[RESEND PATCH 0/4]Add SMP support for MSM8660, MSM8960 and MSM8974

[RESEND PATCH 0/4]Add SMP support for MSM8660, MSM8960 and MSM8974

[Rohit Vaswani]

This series re-organizes the platsmp.c and adds SMP support for
MSM8660, MSM8960 and MSM8974. This is based on 3.11-rc3.
We convert to using the cpus property in device tree and
add a "enable-method" property for arm32.
This helps select the appropriate release sequence for the
secondary cores based on the cpu.

Rohit Vaswani (4):
  ARM: msm: Remove pen_release usage
  ARM: msm: Re-organize platsmp to make it extensible
  ARM: msm: Add SMP support for 8960
  ARM: msm: Add support for 8974 SMP

 Documentation/devicetree/bindings/arm/cpus.txt     |   9 +
 Documentation/devicetree/bindings/arm/msm/kpss.txt |  16 ++
 Documentation/devicetree/bindings/arm/msm/scss.txt |  15 ++
 arch/arm/boot/dts/msm8660-surf.dts                 |  23 +-
 arch/arm/boot/dts/msm8960-cdp.dts                  |  22 ++
 arch/arm/boot/dts/msm8974.dts                      |  23 ++
 arch/arm/mach-msm/Makefile                         |   2 +-
 arch/arm/mach-msm/board-dt-8974.c                  |   3 +
 arch/arm/mach-msm/headsmp.S                        |  39 ---
 arch/arm/mach-msm/hotplug.c                        |  31 +--
 arch/arm/mach-msm/platsmp.c                        | 263 ++++++++++++++++-----
 arch/arm/mach-msm/scm-boot.h                       |   8 +-
 12 files changed, 330 insertions(+), 124 deletions(-)
 create mode 100644 Documentation/devicetree/bindings/arm/msm/kpss.txt
 create mode 100644 Documentation/devicetree/bindings/arm/msm/scss.txt
 delete mode 100644 arch/arm/mach-msm/headsmp.S

-- 
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
hosted by The Linux Foundation

[RESEND PATCH 0/4]Add SMP support for MSM8660, MSM8960 and MSM8974

[Rohit Vaswani]

This series re-organizes the platsmp.c and adds SMP support for
MSM8660, MSM8960 and MSM8974. This is based on 3.11-rc3.
We convert to using the cpus property in device tree and
add a "enable-method" property for arm32.
This helps select the appropriate release sequence for the
secondary cores based on the cpu.

Rohit Vaswani (4):
  ARM: msm: Remove pen_release usage
  ARM: msm: Re-organize platsmp to make it extensible
  ARM: msm: Add SMP support for 8960
  ARM: msm: Add support for 8974 SMP

 Documentation/devicetree/bindings/arm/cpus.txt     |   9 +
 Documentation/devicetree/bindings/arm/msm/kpss.txt |  16 ++
 Documentation/devicetree/bindings/arm/msm/scss.txt |  15 ++
 arch/arm/boot/dts/msm8660-surf.dts                 |  23 +-
 arch/arm/boot/dts/msm8960-cdp.dts                  |  22 ++
 arch/arm/boot/dts/msm8974.dts                      |  23 ++
 arch/arm/mach-msm/Makefile                         |   2 +-
 arch/arm/mach-msm/board-dt-8974.c                  |   3 +
 arch/arm/mach-msm/headsmp.S                        |  39 ---
 arch/arm/mach-msm/hotplug.c                        |  31 +--
 arch/arm/mach-msm/platsmp.c                        | 263 ++++++++++++++++-----
 arch/arm/mach-msm/scm-boot.h                       |   8 +-
 12 files changed, 330 insertions(+), 124 deletions(-)
 create mode 100644 Documentation/devicetree/bindings/arm/msm/kpss.txt
 create mode 100644 Documentation/devicetree/bindings/arm/msm/scss.txt
 delete mode 100644 arch/arm/mach-msm/headsmp.S

-- 
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
hosted by The Linux Foundation

[RESEND PATCH 1/4] ARM: msm: Remove pen_release usage

[Rohit Vaswani]

pen_release is no longer required as the synchronization
is now managed by generic arm code.
This is done as suggested in https://lkml.org/lkml/2013/6/4/184

Signed-off-by: Rohit Vaswani <rvaswani@codeaurora.org>
---
 arch/arm/mach-msm/Makefile  |  2 +-
 arch/arm/mach-msm/headsmp.S | 39 ---------------------------------------
 arch/arm/mach-msm/hotplug.c | 31 ++++---------------------------
 arch/arm/mach-msm/platsmp.c | 35 +++--------------------------------
 4 files changed, 8 insertions(+), 99 deletions(-)
 delete mode 100644 arch/arm/mach-msm/headsmp.S

diff --git a/arch/arm/mach-msm/Makefile b/arch/arm/mach-msm/Makefile
index 80e3b15..14e22f9 100644
--- a/arch/arm/mach-msm/Makefile
+++ b/arch/arm/mach-msm/Makefile
@@ -20,7 +20,7 @@ obj-$(CONFIG_MSM_SCM) += scm.o scm-boot.o
 CFLAGS_scm.o :=$(call as-instr,.arch_extension sec,-DREQUIRES_SEC=1)
 
 obj-$(CONFIG_HOTPLUG_CPU) += hotplug.o
-obj-$(CONFIG_SMP) += headsmp.o platsmp.o
+obj-$(CONFIG_SMP) += platsmp.o
 
 obj-$(CONFIG_MACH_TROUT) += board-trout.o board-trout-gpio.o board-trout-mmc.o devices-msm7x00.o
 obj-$(CONFIG_MACH_TROUT) += board-trout.o board-trout-gpio.o board-trout-mmc.o board-trout-panel.o devices-msm7x00.o
diff --git a/arch/arm/mach-msm/headsmp.S b/arch/arm/mach-msm/headsmp.S
deleted file mode 100644
index 6c62c3f..0000000
--- a/arch/arm/mach-msm/headsmp.S
+++ /dev/null
@@ -1,39 +0,0 @@
-/*
- *  linux/arch/arm/mach-realview/headsmp.S
- *
- *  Copyright (c) 2003 ARM Limited
- *  All Rights Reserved
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- */
-#include <linux/linkage.h>
-#include <linux/init.h>
-
-/*
- * MSM specific entry point for secondary CPUs.  This provides
- * a "holding pen" into which all secondary cores are held until we're
- * ready for them to initialise.
- */
-ENTRY(msm_secondary_startup)
-	mrc	p15, 0, r0, c0, c0, 5
-	and	r0, r0, #15
-	adr	r4, 1f
-	ldmia	r4, {r5, r6}
-	sub	r4, r4, r5
-	add	r6, r6, r4
-pen:	ldr	r7, [r6]
-	cmp	r7, r0
-	bne	pen
-
-	/*
-	 * we've been released from the holding pen: secondary_stack
-	 * should now contain the SVC stack for this core
-	 */
-	b	secondary_startup
-ENDPROC(msm_secondary_startup)
-
-	.align
-1:	.long	.
-	.long	pen_release
diff --git a/arch/arm/mach-msm/hotplug.c b/arch/arm/mach-msm/hotplug.c
index 326a872..cea80fc 100644
--- a/arch/arm/mach-msm/hotplug.c
+++ b/arch/arm/mach-msm/hotplug.c
@@ -24,33 +24,10 @@ static inline void cpu_leave_lowpower(void)
 
 static inline void platform_do_lowpower(unsigned int cpu)
 {
-	/* Just enter wfi for now. TODO: Properly shut off the cpu. */
-	for (;;) {
-		/*
-		 * here's the WFI
-		 */
-		asm("wfi"
-		    :
-		    :
-		    : "memory", "cc");
-
-		if (pen_release == cpu_logical_map(cpu)) {
-			/*
-			 * OK, proper wakeup, we're done
-			 */
-			break;
-		}
-
-		/*
-		 * getting here, means that we have come out of WFI without
-		 * having been woken up - this shouldn't happen
-		 *
-		 * The trouble is, letting people know about this is not really
-		 * possible, since we are currently running incoherently, and
-		 * therefore cannot safely call printk() or anything else
-		 */
-		pr_debug("CPU%u: spurious wakeup call\n", cpu);
-	}
+	asm("wfi"
+	    :
+	    :
+	    : "memory", "cc");
 }
 
 /*
diff --git a/arch/arm/mach-msm/platsmp.c b/arch/arm/mach-msm/platsmp.c
index 3f06edc..5b481db 100644
--- a/arch/arm/mach-msm/platsmp.c
+++ b/arch/arm/mach-msm/platsmp.c
@@ -28,7 +28,7 @@
 #define SCSS_CPU1CORE_RESET 0xD80
 #define SCSS_DBG_STATUS_CORE_PWRDUP 0xE64
 
-extern void msm_secondary_startup(void);
+extern void secondary_startup(void);
 
 static DEFINE_SPINLOCK(boot_lock);
 
@@ -41,13 +41,6 @@ static inline int get_core_count(void)
 static void msm_secondary_init(unsigned int cpu)
 {
 	/*
-	 * let the primary processor know we're out of the
-	 * pen, then head off into the C entry point
-	 */
-	pen_release = -1;
-	smp_wmb();
-
-	/*
 	 * Synchronise with the boot thread.
 	 */
 	spin_lock(&boot_lock);
@@ -57,7 +50,7 @@ static void msm_secondary_init(unsigned int cpu)
 static void prepare_cold_cpu(unsigned int cpu)
 {
 	int ret;
-	ret = scm_set_boot_addr(virt_to_phys(msm_secondary_startup),
+	ret = scm_set_boot_addr(virt_to_phys(secondary_startup),
 				SCM_FLAG_COLDBOOT_CPU1);
 	if (ret == 0) {
 		void __iomem *sc1_base_ptr;
@@ -75,7 +68,6 @@ static void prepare_cold_cpu(unsigned int cpu)
 
 static int msm_boot_secondary(unsigned int cpu, struct task_struct *idle)
 {
-	unsigned long timeout;
 	static int cold_boot_done;
 
 	/* Only need to bring cpu out of reset this way once */
@@ -91,40 +83,19 @@ static int msm_boot_secondary(unsigned int cpu, struct task_struct *idle)
 	spin_lock(&boot_lock);
 
 	/*
-	 * The secondary processor is waiting to be released from
-	 * the holding pen - release it, then wait for it to flag
-	 * that it has been released by resetting pen_release.
-	 *
-	 * Note that "pen_release" is the hardware CPU ID, whereas
-	 * "cpu" is Linux's internal ID.
-	 */
-	pen_release = cpu_logical_map(cpu);
-	__cpuc_flush_dcache_area((void *)&pen_release, sizeof(pen_release));
-	outer_clean_range(__pa(&pen_release), __pa(&pen_release + 1));
-
-	/*
 	 * Send the secondary CPU a soft interrupt, thereby causing
 	 * the boot monitor to read the system wide flags register,
 	 * and branch to the address found there.
 	 */
 	arch_send_wakeup_ipi_mask(cpumask_of(cpu));
 
-	timeout = jiffies + (1 * HZ);
-	while (time_before(jiffies, timeout)) {
-		smp_rmb();
-		if (pen_release == -1)
-			break;
-
-		udelay(10);
-	}
-
 	/*
 	 * now the secondary core is starting up let it run its
 	 * calibrations, then wait for it to finish
 	 */
 	spin_unlock(&boot_lock);
 
-	return pen_release != -1 ? -ENOSYS : 0;
+	return 0;
 }
 
 /*
-- 
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
hosted by The Linux Foundation

[RESEND PATCH 1/4] ARM: msm: Remove pen_release usage

[Rohit Vaswani]

pen_release is no longer required as the synchronization
is now managed by generic arm code.
This is done as suggested in https://lkml.org/lkml/2013/6/4/184

Signed-off-by: Rohit Vaswani <rvaswani@codeaurora.org>
---
 arch/arm/mach-msm/Makefile  |  2 +-
 arch/arm/mach-msm/headsmp.S | 39 ---------------------------------------
 arch/arm/mach-msm/hotplug.c | 31 ++++---------------------------
 arch/arm/mach-msm/platsmp.c | 35 +++--------------------------------
 4 files changed, 8 insertions(+), 99 deletions(-)
 delete mode 100644 arch/arm/mach-msm/headsmp.S

diff --git a/arch/arm/mach-msm/Makefile b/arch/arm/mach-msm/Makefile
index 80e3b15..14e22f9 100644
--- a/arch/arm/mach-msm/Makefile
+++ b/arch/arm/mach-msm/Makefile
@@ -20,7 +20,7 @@ obj-$(CONFIG_MSM_SCM) += scm.o scm-boot.o
 CFLAGS_scm.o :=$(call as-instr,.arch_extension sec,-DREQUIRES_SEC=1)
 
 obj-$(CONFIG_HOTPLUG_CPU) += hotplug.o
-obj-$(CONFIG_SMP) += headsmp.o platsmp.o
+obj-$(CONFIG_SMP) += platsmp.o
 
 obj-$(CONFIG_MACH_TROUT) += board-trout.o board-trout-gpio.o board-trout-mmc.o devices-msm7x00.o
 obj-$(CONFIG_MACH_TROUT) += board-trout.o board-trout-gpio.o board-trout-mmc.o board-trout-panel.o devices-msm7x00.o
diff --git a/arch/arm/mach-msm/headsmp.S b/arch/arm/mach-msm/headsmp.S
deleted file mode 100644
index 6c62c3f..0000000
--- a/arch/arm/mach-msm/headsmp.S
+++ /dev/null
@@ -1,39 +0,0 @@
-/*
- *  linux/arch/arm/mach-realview/headsmp.S
- *
- *  Copyright (c) 2003 ARM Limited
- *  All Rights Reserved
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- */
-#include <linux/linkage.h>
-#include <linux/init.h>
-
-/*
- * MSM specific entry point for secondary CPUs.  This provides
- * a "holding pen" into which all secondary cores are held until we're
- * ready for them to initialise.
- */
-ENTRY(msm_secondary_startup)
-	mrc	p15, 0, r0, c0, c0, 5
-	and	r0, r0, #15
-	adr	r4, 1f
-	ldmia	r4, {r5, r6}
-	sub	r4, r4, r5
-	add	r6, r6, r4
-pen:	ldr	r7, [r6]
-	cmp	r7, r0
-	bne	pen
-
-	/*
-	 * we've been released from the holding pen: secondary_stack
-	 * should now contain the SVC stack for this core
-	 */
-	b	secondary_startup
-ENDPROC(msm_secondary_startup)
-
-	.align
-1:	.long	.
-	.long	pen_release
diff --git a/arch/arm/mach-msm/hotplug.c b/arch/arm/mach-msm/hotplug.c
index 326a872..cea80fc 100644
--- a/arch/arm/mach-msm/hotplug.c
+++ b/arch/arm/mach-msm/hotplug.c
@@ -24,33 +24,10 @@ static inline void cpu_leave_lowpower(void)
 
 static inline void platform_do_lowpower(unsigned int cpu)
 {
-	/* Just enter wfi for now. TODO: Properly shut off the cpu. */
-	for (;;) {
-		/*
-		 * here's the WFI
-		 */
-		asm("wfi"
-		    :
-		    :
-		    : "memory", "cc");
-
-		if (pen_release == cpu_logical_map(cpu)) {
-			/*
-			 * OK, proper wakeup, we're done
-			 */
-			break;
-		}
-
-		/*
-		 * getting here, means that we have come out of WFI without
-		 * having been woken up - this shouldn't happen
-		 *
-		 * The trouble is, letting people know about this is not really
-		 * possible, since we are currently running incoherently, and
-		 * therefore cannot safely call printk() or anything else
-		 */
-		pr_debug("CPU%u: spurious wakeup call\n", cpu);
-	}
+	asm("wfi"
+	    :
+	    :
+	    : "memory", "cc");
 }
 
 /*
diff --git a/arch/arm/mach-msm/platsmp.c b/arch/arm/mach-msm/platsmp.c
index 3f06edc..5b481db 100644
--- a/arch/arm/mach-msm/platsmp.c
+++ b/arch/arm/mach-msm/platsmp.c
@@ -28,7 +28,7 @@
 #define SCSS_CPU1CORE_RESET 0xD80
 #define SCSS_DBG_STATUS_CORE_PWRDUP 0xE64
 
-extern void msm_secondary_startup(void);
+extern void secondary_startup(void);
 
 static DEFINE_SPINLOCK(boot_lock);
 
@@ -41,13 +41,6 @@ static inline int get_core_count(void)
 static void msm_secondary_init(unsigned int cpu)
 {
 	/*
-	 * let the primary processor know we're out of the
-	 * pen, then head off into the C entry point
-	 */
-	pen_release = -1;
-	smp_wmb();
-
-	/*
 	 * Synchronise with the boot thread.
 	 */
 	spin_lock(&boot_lock);
@@ -57,7 +50,7 @@ static void msm_secondary_init(unsigned int cpu)
 static void prepare_cold_cpu(unsigned int cpu)
 {
 	int ret;
-	ret = scm_set_boot_addr(virt_to_phys(msm_secondary_startup),
+	ret = scm_set_boot_addr(virt_to_phys(secondary_startup),
 				SCM_FLAG_COLDBOOT_CPU1);
 	if (ret == 0) {
 		void __iomem *sc1_base_ptr;
@@ -75,7 +68,6 @@ static void prepare_cold_cpu(unsigned int cpu)
 
 static int msm_boot_secondary(unsigned int cpu, struct task_struct *idle)
 {
-	unsigned long timeout;
 	static int cold_boot_done;
 
 	/* Only need to bring cpu out of reset this way once */
@@ -91,40 +83,19 @@ static int msm_boot_secondary(unsigned int cpu, struct task_struct *idle)
 	spin_lock(&boot_lock);
 
 	/*
-	 * The secondary processor is waiting to be released from
-	 * the holding pen - release it, then wait for it to flag
-	 * that it has been released by resetting pen_release.
-	 *
-	 * Note that "pen_release" is the hardware CPU ID, whereas
-	 * "cpu" is Linux's internal ID.
-	 */
-	pen_release = cpu_logical_map(cpu);
-	__cpuc_flush_dcache_area((void *)&pen_release, sizeof(pen_release));
-	outer_clean_range(__pa(&pen_release), __pa(&pen_release + 1));
-
-	/*
 	 * Send the secondary CPU a soft interrupt, thereby causing
 	 * the boot monitor to read the system wide flags register,
 	 * and branch to the address found there.
 	 */
 	arch_send_wakeup_ipi_mask(cpumask_of(cpu));
 
-	timeout = jiffies + (1 * HZ);
-	while (time_before(jiffies, timeout)) {
-		smp_rmb();
-		if (pen_release == -1)
-			break;
-
-		udelay(10);
-	}
-
 	/*
 	 * now the secondary core is starting up let it run its
 	 * calibrations, then wait for it to finish
 	 */
 	spin_unlock(&boot_lock);
 
-	return pen_release != -1 ? -ENOSYS : 0;
+	return 0;
 }
 
 /*
-- 
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
hosted by The Linux Foundation

[RESEND PATCH 2/4] ARM: msm: Re-organize platsmp to make it extensible

[Rohit Vaswani]

This makes it easy to add SMP support for new targets
by adding cpus property and the release sequence.
We add the enable-method property for the cpus property to
specify which release sequence to use.
While at it, add the 8660 cpus bindings to make SMP work.

Signed-off-by: Rohit Vaswani <rvaswani@codeaurora.org>
---
 Documentation/devicetree/bindings/arm/cpus.txt     |  6 ++
 Documentation/devicetree/bindings/arm/msm/scss.txt | 15 ++++
 arch/arm/boot/dts/msm8660-surf.dts                 | 23 +++++-
 arch/arm/mach-msm/platsmp.c                        | 94 +++++++++++++++++-----
 4 files changed, 115 insertions(+), 23 deletions(-)
 create mode 100644 Documentation/devicetree/bindings/arm/msm/scss.txt

diff --git a/Documentation/devicetree/bindings/arm/cpus.txt b/Documentation/devicetree/bindings/arm/cpus.txt
index f32494d..327aad2 100644
--- a/Documentation/devicetree/bindings/arm/cpus.txt
+++ b/Documentation/devicetree/bindings/arm/cpus.txt
@@ -44,6 +44,12 @@ For the ARM architecture every CPU node must contain the following properties:
 		"marvell,mohawk"
 		"marvell,xsc3"
 		"marvell,xscale"
+		"qcom,scorpion"
+- enable-method: Specifies the method used to enable or take the secondary cores
+		 out of reset. This allows different reset sequence for
+		 different types of cpus.
+		 This should be one of:
+		 "qcom,scss"
 
 Example:
 
diff --git a/Documentation/devicetree/bindings/arm/msm/scss.txt b/Documentation/devicetree/bindings/arm/msm/scss.txt
new file mode 100644
index 0000000..21c3e26
--- /dev/null
+++ b/Documentation/devicetree/bindings/arm/msm/scss.txt
@@ -0,0 +1,15 @@
+* SCSS - Scorpion Sub-system
+
+Properties
+
+- compatible : Should contain "qcom,scss".
+
+- reg: Specifies the base address for the SCSS registers used for
+       booting up secondary cores.
+
+Example:
+
+	scss@902000 {
+		compatible = "qcom,scss";
+		reg = <0x00902000 0x2000>;
+	};
diff --git a/arch/arm/boot/dts/msm8660-surf.dts b/arch/arm/boot/dts/msm8660-surf.dts
index cdc010e..203e51a 100644
--- a/arch/arm/boot/dts/msm8660-surf.dts
+++ b/arch/arm/boot/dts/msm8660-surf.dts
@@ -7,6 +7,22 @@
 	compatible = "qcom,msm8660-surf", "qcom,msm8660";
 	interrupt-parent = <&intc>;
 
+	cpus {
+		#address-cells = <1>;
+		#size-cells = <0>;
+		compatible = "qcom,scorpion";
+		device_type = "cpu";
+		enable-method = "qcom,scss";
+
+		cpu@0 {
+			reg = <0>;
+		};
+
+		cpu@1 {
+			reg = <1>;
+		};
+	};
+
 	intc: interrupt-controller@2080000 {
 		compatible = "qcom,msm-8660-qgic";
 		interrupt-controller;
@@ -37,7 +53,12 @@
 		#interrupt-cells = <2>;
 	};
 
-	serial@19c40000 {
+	scss@902000 {
+		compatible = "qcom,scss";
+		reg = <0x00902000 0x2000>;
+	};
+
+	serial@19c400000 {
 		compatible = "qcom,msm-hsuart", "qcom,msm-uart";
 		reg = <0x19c40000 0x1000>,
 		      <0x19c00000 0x1000>;
diff --git a/arch/arm/mach-msm/platsmp.c b/arch/arm/mach-msm/platsmp.c
index 5b481db..17022e0 100644
--- a/arch/arm/mach-msm/platsmp.c
+++ b/arch/arm/mach-msm/platsmp.c
@@ -13,6 +13,8 @@
 #include <linux/delay.h>
 #include <linux/device.h>
 #include <linux/jiffies.h>
+#include <linux/of.h>
+#include <linux/of_address.h>
 #include <linux/smp.h>
 #include <linux/io.h>
 
@@ -47,35 +49,63 @@ static void msm_secondary_init(unsigned int cpu)
 	spin_unlock(&boot_lock);
 }
 
-static void prepare_cold_cpu(unsigned int cpu)
+static int scorpion_release_secondary(void)
 {
-	int ret;
-	ret = scm_set_boot_addr(virt_to_phys(secondary_startup),
-				SCM_FLAG_COLDBOOT_CPU1);
-	if (ret == 0) {
-		void __iomem *sc1_base_ptr;
-		sc1_base_ptr = ioremap_nocache(0x00902000, SZ_4K*2);
-		if (sc1_base_ptr) {
-			writel(0, sc1_base_ptr + VDD_SC1_ARRAY_CLAMP_GFS_CTL);
-			writel(0, sc1_base_ptr + SCSS_CPU1CORE_RESET);
-			writel(3, sc1_base_ptr + SCSS_DBG_STATUS_CORE_PWRDUP);
-			iounmap(sc1_base_ptr);
-		}
-	} else
-		printk(KERN_DEBUG "Failed to set secondary core boot "
-				  "address\n");
+	void __iomem *sc1_base_ptr;
+	struct device_node *dn = NULL;
+
+	dn = of_find_compatible_node(dn, NULL, "qcom,scss");
+	if (!dn) {
+		pr_err("%s: Missing scss node in device tree\n", __func__);
+		return -ENXIO;
+	}
+
+	sc1_base_ptr = of_iomap(dn, 0);
+	if (sc1_base_ptr) {
+		writel_relaxed(0, sc1_base_ptr + VDD_SC1_ARRAY_CLAMP_GFS_CTL);
+		writel_relaxed(0, sc1_base_ptr + SCSS_CPU1CORE_RESET);
+		writel_relaxed(3, sc1_base_ptr + SCSS_DBG_STATUS_CORE_PWRDUP);
+		mb();
+		iounmap(sc1_base_ptr);
+	} else {
+		return -ENOMEM;
+	}
+
+	return 0;
 }
 
-static int msm_boot_secondary(unsigned int cpu, struct task_struct *idle)
+static DEFINE_PER_CPU(int, cold_boot_done);
+
+static void boot_cold_cpu(unsigned int cpu)
 {
-	static int cold_boot_done;
+	const char *enable_method;
+	struct device_node *dn = NULL;
 
-	/* Only need to bring cpu out of reset this way once */
-	if (cold_boot_done == false) {
-		prepare_cold_cpu(cpu);
-		cold_boot_done = true;
+	dn = of_find_node_by_name(dn, "cpus");
+	if (!dn) {
+		pr_err("%s: Missing node cpus in device tree\n", __func__);
+		return;
 	}
 
+	enable_method = of_get_property(dn, "enable-method", NULL);
+	if (!enable_method) {
+			pr_err("%s: cpus node is missing enable-method property\n",
+					__func__);
+	} else if (!strcmp(enable_method, "qcom,scss")) {
+		if (per_cpu(cold_boot_done, cpu) == false) {
+			scorpion_release_secondary();
+			per_cpu(cold_boot_done, cpu) = true;
+		}
+	} else {
+		pr_err("%s: Invalid enable-method property: %s\n",
+				__func__, enable_method);
+	}
+}
+
+static int msm_boot_secondary(unsigned int cpu, struct task_struct *idle)
+{
+	boot_cold_cpu(cpu);
+
 	/*
 	 * set synchronisation state between this boot processor
 	 * and the secondary one
@@ -118,8 +148,28 @@ static void __init msm_smp_init_cpus(void)
 		set_cpu_possible(i, true);
 }
 
+static const int cold_boot_flags[] __initconst = {
+	0,
+	SCM_FLAG_COLDBOOT_CPU1,
+};
+
 static void __init msm_smp_prepare_cpus(unsigned int max_cpus)
 {
+	int cpu, map;
+	unsigned int flags = 0;
+
+	for_each_present_cpu(cpu) {
+		map = cpu_logical_map(cpu);
+		if (map > ARRAY_SIZE(cold_boot_flags)) {
+			set_cpu_present(cpu, false);
+			__WARN();
+			continue;
+		}
+		flags |= cold_boot_flags[map];
+	}
+
+	if (scm_set_boot_addr(virt_to_phys(secondary_startup), flags))
+		pr_warn("Failed to set CPU boot address\n");
 }
 
 struct smp_operations msm_smp_ops __initdata = {
-- 
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
hosted by The Linux Foundation

[RESEND PATCH 2/4] ARM: msm: Re-organize platsmp to make it extensible

[Rohit Vaswani]

This makes it easy to add SMP support for new targets
by adding cpus property and the release sequence.
We add the enable-method property for the cpus property to
specify which release sequence to use.
While at it, add the 8660 cpus bindings to make SMP work.

Signed-off-by: Rohit Vaswani <rvaswani@codeaurora.org>
---
 Documentation/devicetree/bindings/arm/cpus.txt     |  6 ++
 Documentation/devicetree/bindings/arm/msm/scss.txt | 15 ++++
 arch/arm/boot/dts/msm8660-surf.dts                 | 23 +++++-
 arch/arm/mach-msm/platsmp.c                        | 94 +++++++++++++++++-----
 4 files changed, 115 insertions(+), 23 deletions(-)
 create mode 100644 Documentation/devicetree/bindings/arm/msm/scss.txt

diff --git a/Documentation/devicetree/bindings/arm/cpus.txt b/Documentation/devicetree/bindings/arm/cpus.txt
index f32494d..327aad2 100644
--- a/Documentation/devicetree/bindings/arm/cpus.txt
+++ b/Documentation/devicetree/bindings/arm/cpus.txt
@@ -44,6 +44,12 @@ For the ARM architecture every CPU node must contain the following properties:
 		"marvell,mohawk"
 		"marvell,xsc3"
 		"marvell,xscale"
+		"qcom,scorpion"
+- enable-method: Specifies the method used to enable or take the secondary cores
+		 out of reset. This allows different reset sequence for
+		 different types of cpus.
+		 This should be one of:
+		 "qcom,scss"
 
 Example:
 
diff --git a/Documentation/devicetree/bindings/arm/msm/scss.txt b/Documentation/devicetree/bindings/arm/msm/scss.txt
new file mode 100644
index 0000000..21c3e26
--- /dev/null
+++ b/Documentation/devicetree/bindings/arm/msm/scss.txt
@@ -0,0 +1,15 @@
+* SCSS - Scorpion Sub-system
+
+Properties
+
+- compatible : Should contain "qcom,scss".
+
+- reg: Specifies the base address for the SCSS registers used for
+       booting up secondary cores.
+
+Example:
+
+	scss at 902000 {
+		compatible = "qcom,scss";
+		reg = <0x00902000 0x2000>;
+	};
diff --git a/arch/arm/boot/dts/msm8660-surf.dts b/arch/arm/boot/dts/msm8660-surf.dts
index cdc010e..203e51a 100644
--- a/arch/arm/boot/dts/msm8660-surf.dts
+++ b/arch/arm/boot/dts/msm8660-surf.dts
@@ -7,6 +7,22 @@
 	compatible = "qcom,msm8660-surf", "qcom,msm8660";
 	interrupt-parent = <&intc>;
 
+	cpus {
+		#address-cells = <1>;
+		#size-cells = <0>;
+		compatible = "qcom,scorpion";
+		device_type = "cpu";
+		enable-method = "qcom,scss";
+
+		cpu at 0 {
+			reg = <0>;
+		};
+
+		cpu at 1 {
+			reg = <1>;
+		};
+	};
+
 	intc: interrupt-controller at 2080000 {
 		compatible = "qcom,msm-8660-qgic";
 		interrupt-controller;
@@ -37,7 +53,12 @@
 		#interrupt-cells = <2>;
 	};
 
-	serial at 19c40000 {
+	scss at 902000 {
+		compatible = "qcom,scss";
+		reg = <0x00902000 0x2000>;
+	};
+
+	serial at 19c400000 {
 		compatible = "qcom,msm-hsuart", "qcom,msm-uart";
 		reg = <0x19c40000 0x1000>,
 		      <0x19c00000 0x1000>;
diff --git a/arch/arm/mach-msm/platsmp.c b/arch/arm/mach-msm/platsmp.c
index 5b481db..17022e0 100644
--- a/arch/arm/mach-msm/platsmp.c
+++ b/arch/arm/mach-msm/platsmp.c
@@ -13,6 +13,8 @@
 #include <linux/delay.h>
 #include <linux/device.h>
 #include <linux/jiffies.h>
+#include <linux/of.h>
+#include <linux/of_address.h>
 #include <linux/smp.h>
 #include <linux/io.h>
 
@@ -47,35 +49,63 @@ static void msm_secondary_init(unsigned int cpu)
 	spin_unlock(&boot_lock);
 }
 
-static void prepare_cold_cpu(unsigned int cpu)
+static int scorpion_release_secondary(void)
 {
-	int ret;
-	ret = scm_set_boot_addr(virt_to_phys(secondary_startup),
-				SCM_FLAG_COLDBOOT_CPU1);
-	if (ret == 0) {
-		void __iomem *sc1_base_ptr;
-		sc1_base_ptr = ioremap_nocache(0x00902000, SZ_4K*2);
-		if (sc1_base_ptr) {
-			writel(0, sc1_base_ptr + VDD_SC1_ARRAY_CLAMP_GFS_CTL);
-			writel(0, sc1_base_ptr + SCSS_CPU1CORE_RESET);
-			writel(3, sc1_base_ptr + SCSS_DBG_STATUS_CORE_PWRDUP);
-			iounmap(sc1_base_ptr);
-		}
-	} else
-		printk(KERN_DEBUG "Failed to set secondary core boot "
-				  "address\n");
+	void __iomem *sc1_base_ptr;
+	struct device_node *dn = NULL;
+
+	dn = of_find_compatible_node(dn, NULL, "qcom,scss");
+	if (!dn) {
+		pr_err("%s: Missing scss node in device tree\n", __func__);
+		return -ENXIO;
+	}
+
+	sc1_base_ptr = of_iomap(dn, 0);
+	if (sc1_base_ptr) {
+		writel_relaxed(0, sc1_base_ptr + VDD_SC1_ARRAY_CLAMP_GFS_CTL);
+		writel_relaxed(0, sc1_base_ptr + SCSS_CPU1CORE_RESET);
+		writel_relaxed(3, sc1_base_ptr + SCSS_DBG_STATUS_CORE_PWRDUP);
+		mb();
+		iounmap(sc1_base_ptr);
+	} else {
+		return -ENOMEM;
+	}
+
+	return 0;
 }
 
-static int msm_boot_secondary(unsigned int cpu, struct task_struct *idle)
+static DEFINE_PER_CPU(int, cold_boot_done);
+
+static void boot_cold_cpu(unsigned int cpu)
 {
-	static int cold_boot_done;
+	const char *enable_method;
+	struct device_node *dn = NULL;
 
-	/* Only need to bring cpu out of reset this way once */
-	if (cold_boot_done == false) {
-		prepare_cold_cpu(cpu);
-		cold_boot_done = true;
+	dn = of_find_node_by_name(dn, "cpus");
+	if (!dn) {
+		pr_err("%s: Missing node cpus in device tree\n", __func__);
+		return;
 	}
 
+	enable_method = of_get_property(dn, "enable-method", NULL);
+	if (!enable_method) {
+			pr_err("%s: cpus node is missing enable-method property\n",
+					__func__);
+	} else if (!strcmp(enable_method, "qcom,scss")) {
+		if (per_cpu(cold_boot_done, cpu) == false) {
+			scorpion_release_secondary();
+			per_cpu(cold_boot_done, cpu) = true;
+		}
+	} else {
+		pr_err("%s: Invalid enable-method property: %s\n",
+				__func__, enable_method);
+	}
+}
+
+static int msm_boot_secondary(unsigned int cpu, struct task_struct *idle)
+{
+	boot_cold_cpu(cpu);
+
 	/*
 	 * set synchronisation state between this boot processor
 	 * and the secondary one
@@ -118,8 +148,28 @@ static void __init msm_smp_init_cpus(void)
 		set_cpu_possible(i, true);
 }
 
+static const int cold_boot_flags[] __initconst = {
+	0,
+	SCM_FLAG_COLDBOOT_CPU1,
+};
+
 static void __init msm_smp_prepare_cpus(unsigned int max_cpus)
 {
+	int cpu, map;
+	unsigned int flags = 0;
+
+	for_each_present_cpu(cpu) {
+		map = cpu_logical_map(cpu);
+		if (map > ARRAY_SIZE(cold_boot_flags)) {
+			set_cpu_present(cpu, false);
+			__WARN();
+			continue;
+		}
+		flags |= cold_boot_flags[map];
+	}
+
+	if (scm_set_boot_addr(virt_to_phys(secondary_startup), flags))
+		pr_warn("Failed to set CPU boot address\n");
 }
 
 struct smp_operations msm_smp_ops __initdata = {
-- 
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
hosted by The Linux Foundation

[PATCH 3/4] ARM: msm: Add SMP support for 8960

[Rohit Vaswani]

Add the cpus bindings and the Krait release sequence
to make SMP work for MSM8960

Signed-off-by: Rohit Vaswani <rvaswani@codeaurora.org>
---
 Documentation/devicetree/bindings/arm/cpus.txt     |  2 +
 Documentation/devicetree/bindings/arm/msm/kpss.txt | 16 ++++++
 arch/arm/boot/dts/msm8960-cdp.dts                  | 22 +++++++++
 arch/arm/mach-msm/platsmp.c                        | 57 ++++++++++++++++++++++
 arch/arm/mach-msm/scm-boot.h                       |  8 +--
 5 files changed, 102 insertions(+), 3 deletions(-)
 create mode 100644 Documentation/devicetree/bindings/arm/msm/kpss.txt

diff --git a/Documentation/devicetree/bindings/arm/cpus.txt b/Documentation/devicetree/bindings/arm/cpus.txt
index 327aad2..1132eac 100644
--- a/Documentation/devicetree/bindings/arm/cpus.txt
+++ b/Documentation/devicetree/bindings/arm/cpus.txt
@@ -45,11 +45,13 @@ For the ARM architecture every CPU node must contain the following properties:
 		"marvell,xsc3"
 		"marvell,xscale"
 		"qcom,scorpion"
+		"qcom,krait"
 - enable-method: Specifies the method used to enable or take the secondary cores
 		 out of reset. This allows different reset sequence for
 		 different types of cpus.
 		 This should be one of:
 		 "qcom,scss"
+		 "qcom,kpssv1"
 
 Example:
 
diff --git a/Documentation/devicetree/bindings/arm/msm/kpss.txt b/Documentation/devicetree/bindings/arm/msm/kpss.txt
new file mode 100644
index 0000000..7272340
--- /dev/null
+++ b/Documentation/devicetree/bindings/arm/msm/kpss.txt
@@ -0,0 +1,16 @@
+* KPSS - Krait Processor Sub-system
+
+Properties
+
+- compatible : Should contain "qcom,kpss".
+
+- reg: Specifies the base address for the KPSS registers used for
+       booting up secondary cores.
+
+Example:
+
+	kpss@2088000 {
+		compatible = "qcom,kpss";
+		reg = <0x02088000 0x1000
+			0x02098000 0x2000>;
+	};
diff --git a/arch/arm/boot/dts/msm8960-cdp.dts b/arch/arm/boot/dts/msm8960-cdp.dts
index db2060c..8c82d5e 100644
--- a/arch/arm/boot/dts/msm8960-cdp.dts
+++ b/arch/arm/boot/dts/msm8960-cdp.dts
@@ -7,6 +7,22 @@
 	compatible = "qcom,msm8960-cdp", "qcom,msm8960";
 	interrupt-parent = <&intc>;
 
+	cpus {
+		#address-cells = <1>;
+		#size-cells = <0>;
+		compatible = "qcom,krait";
+		device_type = "cpu";
+		enable-method = "qcom,kpssv1";
+
+		cpu@0 {
+			reg = <0>;
+		};
+
+		cpu@1 {
+			reg = <1>;
+		};
+	};
+
 	intc: interrupt-controller@2000000 {
 		compatible = "qcom,msm-qgic2";
 		interrupt-controller;
@@ -37,6 +53,12 @@
 		reg = <0xfd510000 0x4000>;
 	};
 
+	kpss@2088000 {
+		compatible = "qcom,kpss";
+		reg = <0x02088000 0x1000
+			0x02098000 0x2000>;
+	};
+
 	serial@16440000 {
 		compatible = "qcom,msm-hsuart", "qcom,msm-uart";
 		reg = <0x16440000 0x1000>,
diff --git a/arch/arm/mach-msm/platsmp.c b/arch/arm/mach-msm/platsmp.c
index 17022e0..82eb079 100644
--- a/arch/arm/mach-msm/platsmp.c
+++ b/arch/arm/mach-msm/platsmp.c
@@ -74,6 +74,56 @@ static int scorpion_release_secondary(void)
 	return 0;
 }
 
+static int msm8960_release_secondary(unsigned int cpu)
+{
+	void __iomem *reg;
+	struct device_node *dn = NULL;
+
+	if (cpu == 0 || cpu >= num_possible_cpus())
+		return -EINVAL;
+
+	dn = of_find_compatible_node(dn, NULL, "qcom,kpss");
+	if (!dn) {
+		pr_err("%s : Missing kpss node from device tree\n", __func__);
+		return -ENXIO;
+	}
+
+	reg = of_iomap(dn, cpu);
+	if (!reg)
+		return -ENOMEM;
+
+	pr_debug("Starting secondary CPU %d\n", cpu);
+
+	/* Turn on CPU Rail */
+	writel_relaxed(0xA4, reg+0x1014);
+	mb();
+	udelay(512);
+
+	/* Krait bring-up sequence */
+	writel_relaxed(0x109, reg+0x04);
+	writel_relaxed(0x101, reg+0x04);
+	mb();
+	ndelay(300);
+
+	writel_relaxed(0x121, reg+0x04);
+	mb();
+	udelay(2);
+
+	writel_relaxed(0x120, reg+0x04);
+	mb();
+	udelay(2);
+
+	writel_relaxed(0x100, reg+0x04);
+	mb();
+	udelay(100);
+
+	writel_relaxed(0x180, reg+0x04);
+	mb();
+
+	iounmap(reg);
+	return 0;
+}
+
 static DEFINE_PER_CPU(int, cold_boot_done);
 
 static void boot_cold_cpu(unsigned int cpu)
@@ -96,6 +146,11 @@ static void boot_cold_cpu(unsigned int cpu)
 			scorpion_release_secondary();
 			per_cpu(cold_boot_done, cpu) = true;
 		}
+	} else if (!strcmp(enable_method, "qcom,kpssv1")) {
+		if (per_cpu(cold_boot_done, cpu) == false) {
+			msm8960_release_secondary(cpu);
+			per_cpu(cold_boot_done, cpu) = true;
+		}
 	} else {
 		pr_err("%s: Invalid enable-method property: %s\n",
 				__func__, enable_method);
@@ -151,6 +206,8 @@ static void __init msm_smp_init_cpus(void)
 static const int cold_boot_flags[] __initconst = {
 	0,
 	SCM_FLAG_COLDBOOT_CPU1,
+	SCM_FLAG_COLDBOOT_CPU2,
+	SCM_FLAG_COLDBOOT_CPU3,
 };
 
 static void __init msm_smp_prepare_cpus(unsigned int max_cpus)
diff --git a/arch/arm/mach-msm/scm-boot.h b/arch/arm/mach-msm/scm-boot.h
index 7be32ff..6aabb24 100644
--- a/arch/arm/mach-msm/scm-boot.h
+++ b/arch/arm/mach-msm/scm-boot.h
@@ -13,9 +13,11 @@
 #define __MACH_SCM_BOOT_H
 
 #define SCM_BOOT_ADDR			0x1
-#define SCM_FLAG_COLDBOOT_CPU1		0x1
-#define SCM_FLAG_WARMBOOT_CPU1		0x2
-#define SCM_FLAG_WARMBOOT_CPU0		0x4
+#define SCM_FLAG_COLDBOOT_CPU1		0x01
+#define SCM_FLAG_COLDBOOT_CPU2		0x08
+#define SCM_FLAG_COLDBOOT_CPU3		0x20
+#define SCM_FLAG_WARMBOOT_CPU0		0x04
+#define SCM_FLAG_WARMBOOT_CPU1		0x02
 
 int scm_set_boot_addr(phys_addr_t addr, int flags);
 
-- 
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
hosted by The Linux Foundation

[PATCH 3/4] ARM: msm: Add SMP support for 8960

[Rohit Vaswani]

Add the cpus bindings and the Krait release sequence
to make SMP work for MSM8960

Signed-off-by: Rohit Vaswani <rvaswani@codeaurora.org>
---
 Documentation/devicetree/bindings/arm/cpus.txt     |  2 +
 Documentation/devicetree/bindings/arm/msm/kpss.txt | 16 ++++++
 arch/arm/boot/dts/msm8960-cdp.dts                  | 22 +++++++++
 arch/arm/mach-msm/platsmp.c                        | 57 ++++++++++++++++++++++
 arch/arm/mach-msm/scm-boot.h                       |  8 +--
 5 files changed, 102 insertions(+), 3 deletions(-)
 create mode 100644 Documentation/devicetree/bindings/arm/msm/kpss.txt

diff --git a/Documentation/devicetree/bindings/arm/cpus.txt b/Documentation/devicetree/bindings/arm/cpus.txt
index 327aad2..1132eac 100644
--- a/Documentation/devicetree/bindings/arm/cpus.txt
+++ b/Documentation/devicetree/bindings/arm/cpus.txt
@@ -45,11 +45,13 @@ For the ARM architecture every CPU node must contain the following properties:
 		"marvell,xsc3"
 		"marvell,xscale"
 		"qcom,scorpion"
+		"qcom,krait"
 - enable-method: Specifies the method used to enable or take the secondary cores
 		 out of reset. This allows different reset sequence for
 		 different types of cpus.
 		 This should be one of:
 		 "qcom,scss"
+		 "qcom,kpssv1"
 
 Example:
 
diff --git a/Documentation/devicetree/bindings/arm/msm/kpss.txt b/Documentation/devicetree/bindings/arm/msm/kpss.txt
new file mode 100644
index 0000000..7272340
--- /dev/null
+++ b/Documentation/devicetree/bindings/arm/msm/kpss.txt
@@ -0,0 +1,16 @@
+* KPSS - Krait Processor Sub-system
+
+Properties
+
+- compatible : Should contain "qcom,kpss".
+
+- reg: Specifies the base address for the KPSS registers used for
+       booting up secondary cores.
+
+Example:
+
+	kpss at 2088000 {
+		compatible = "qcom,kpss";
+		reg = <0x02088000 0x1000
+			0x02098000 0x2000>;
+	};
diff --git a/arch/arm/boot/dts/msm8960-cdp.dts b/arch/arm/boot/dts/msm8960-cdp.dts
index db2060c..8c82d5e 100644
--- a/arch/arm/boot/dts/msm8960-cdp.dts
+++ b/arch/arm/boot/dts/msm8960-cdp.dts
@@ -7,6 +7,22 @@
 	compatible = "qcom,msm8960-cdp", "qcom,msm8960";
 	interrupt-parent = <&intc>;
 
+	cpus {
+		#address-cells = <1>;
+		#size-cells = <0>;
+		compatible = "qcom,krait";
+		device_type = "cpu";
+		enable-method = "qcom,kpssv1";
+
+		cpu at 0 {
+			reg = <0>;
+		};
+
+		cpu at 1 {
+			reg = <1>;
+		};
+	};
+
 	intc: interrupt-controller at 2000000 {
 		compatible = "qcom,msm-qgic2";
 		interrupt-controller;
@@ -37,6 +53,12 @@
 		reg = <0xfd510000 0x4000>;
 	};
 
+	kpss at 2088000 {
+		compatible = "qcom,kpss";
+		reg = <0x02088000 0x1000
+			0x02098000 0x2000>;
+	};
+
 	serial at 16440000 {
 		compatible = "qcom,msm-hsuart", "qcom,msm-uart";
 		reg = <0x16440000 0x1000>,
diff --git a/arch/arm/mach-msm/platsmp.c b/arch/arm/mach-msm/platsmp.c
index 17022e0..82eb079 100644
--- a/arch/arm/mach-msm/platsmp.c
+++ b/arch/arm/mach-msm/platsmp.c
@@ -74,6 +74,56 @@ static int scorpion_release_secondary(void)
 	return 0;
 }
 
+static int msm8960_release_secondary(unsigned int cpu)
+{
+	void __iomem *reg;
+	struct device_node *dn = NULL;
+
+	if (cpu == 0 || cpu >= num_possible_cpus())
+		return -EINVAL;
+
+	dn = of_find_compatible_node(dn, NULL, "qcom,kpss");
+	if (!dn) {
+		pr_err("%s : Missing kpss node from device tree\n", __func__);
+		return -ENXIO;
+	}
+
+	reg = of_iomap(dn, cpu);
+	if (!reg)
+		return -ENOMEM;
+
+	pr_debug("Starting secondary CPU %d\n", cpu);
+
+	/* Turn on CPU Rail */
+	writel_relaxed(0xA4, reg+0x1014);
+	mb();
+	udelay(512);
+
+	/* Krait bring-up sequence */
+	writel_relaxed(0x109, reg+0x04);
+	writel_relaxed(0x101, reg+0x04);
+	mb();
+	ndelay(300);
+
+	writel_relaxed(0x121, reg+0x04);
+	mb();
+	udelay(2);
+
+	writel_relaxed(0x120, reg+0x04);
+	mb();
+	udelay(2);
+
+	writel_relaxed(0x100, reg+0x04);
+	mb();
+	udelay(100);
+
+	writel_relaxed(0x180, reg+0x04);
+	mb();
+
+	iounmap(reg);
+	return 0;
+}
+
 static DEFINE_PER_CPU(int, cold_boot_done);
 
 static void boot_cold_cpu(unsigned int cpu)
@@ -96,6 +146,11 @@ static void boot_cold_cpu(unsigned int cpu)
 			scorpion_release_secondary();
 			per_cpu(cold_boot_done, cpu) = true;
 		}
+	} else if (!strcmp(enable_method, "qcom,kpssv1")) {
+		if (per_cpu(cold_boot_done, cpu) == false) {
+			msm8960_release_secondary(cpu);
+			per_cpu(cold_boot_done, cpu) = true;
+		}
 	} else {
 		pr_err("%s: Invalid enable-method property: %s\n",
 				__func__, enable_method);
@@ -151,6 +206,8 @@ static void __init msm_smp_init_cpus(void)
 static const int cold_boot_flags[] __initconst = {
 	0,
 	SCM_FLAG_COLDBOOT_CPU1,
+	SCM_FLAG_COLDBOOT_CPU2,
+	SCM_FLAG_COLDBOOT_CPU3,
 };
 
 static void __init msm_smp_prepare_cpus(unsigned int max_cpus)
diff --git a/arch/arm/mach-msm/scm-boot.h b/arch/arm/mach-msm/scm-boot.h
index 7be32ff..6aabb24 100644
--- a/arch/arm/mach-msm/scm-boot.h
+++ b/arch/arm/mach-msm/scm-boot.h
@@ -13,9 +13,11 @@
 #define __MACH_SCM_BOOT_H
 
 #define SCM_BOOT_ADDR			0x1
-#define SCM_FLAG_COLDBOOT_CPU1		0x1
-#define SCM_FLAG_WARMBOOT_CPU1		0x2
-#define SCM_FLAG_WARMBOOT_CPU0		0x4
+#define SCM_FLAG_COLDBOOT_CPU1		0x01
+#define SCM_FLAG_COLDBOOT_CPU2		0x08
+#define SCM_FLAG_COLDBOOT_CPU3		0x20
+#define SCM_FLAG_WARMBOOT_CPU0		0x04
+#define SCM_FLAG_WARMBOOT_CPU1		0x02
 
 int scm_set_boot_addr(phys_addr_t addr, int flags);
 
-- 
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
hosted by The Linux Foundation

[RESEND PATCH 4/4] ARM: msm: Add support for 8974 SMP

[Rohit Vaswani]

Add the cpus bindings and the Kraitv2 release sequence
to make SMP work for 2 cores on MSM8974.

Signed-off-by: Rohit Vaswani <rvaswani@codeaurora.org>
---
 Documentation/devicetree/bindings/arm/cpus.txt |  1 +
 arch/arm/boot/dts/msm8974.dts                  | 23 ++++++++
 arch/arm/mach-msm/board-dt-8974.c              |  3 +
 arch/arm/mach-msm/platsmp.c                    | 79 ++++++++++++++++++++++++++
 4 files changed, 106 insertions(+)

diff --git a/Documentation/devicetree/bindings/arm/cpus.txt b/Documentation/devicetree/bindings/arm/cpus.txt
index 1132eac..7c3c677 100644
--- a/Documentation/devicetree/bindings/arm/cpus.txt
+++ b/Documentation/devicetree/bindings/arm/cpus.txt
@@ -52,6 +52,7 @@ For the ARM architecture every CPU node must contain the following properties:
 		 This should be one of:
 		 "qcom,scss"
 		 "qcom,kpssv1"
+		 "qcom,kpssv2"
 
 Example:
 
diff --git a/arch/arm/boot/dts/msm8974.dts b/arch/arm/boot/dts/msm8974.dts
index c31c097..ef35a9b 100644
--- a/arch/arm/boot/dts/msm8974.dts
+++ b/arch/arm/boot/dts/msm8974.dts
@@ -7,6 +7,22 @@
 	compatible = "qcom,msm8974";
 	interrupt-parent = <&intc>;
 
+	cpus {
+		#address-cells = <1>;
+		#size-cells = <0>;
+		compatible = "qcom,krait";
+		device_type = "cpu";
+		enable-method = "qcom,kpssv2";
+
+		cpu@0 {
+			reg = <0>;
+		};
+
+		cpu@1 {
+			reg = <1>;
+		};
+	};
+
 	intc: interrupt-controller@f9000000 {
 		compatible = "qcom,msm-qgic2";
 		interrupt-controller;
@@ -23,4 +39,11 @@
 			     <1 1 0xf08>;
 		clock-frequency = <19200000>;
 	};
+
+	kpss@f9012000 {
+		compatible = "qcom,kpss";
+		reg = <0xf9012000 0x1000>,
+		      <0xf9088000 0x1000>,
+		      <0xf9098000 0x1000>;
+	};
 };
diff --git a/arch/arm/mach-msm/board-dt-8974.c b/arch/arm/mach-msm/board-dt-8974.c
index d7f84f2..06119f9 100644
--- a/arch/arm/mach-msm/board-dt-8974.c
+++ b/arch/arm/mach-msm/board-dt-8974.c
@@ -13,11 +13,14 @@
 #include <linux/of_platform.h>
 #include <asm/mach/arch.h>
 
+#include "common.h"
+
 static const char * const msm8974_dt_match[] __initconst = {
 	"qcom,msm8974",
 	NULL
 };
 
 DT_MACHINE_START(MSM8974_DT, "Qualcomm MSM (Flattened Device Tree)")
+	.smp = smp_ops(msm_smp_ops),
 	.dt_compat = msm8974_dt_match,
 MACHINE_END
diff --git a/arch/arm/mach-msm/platsmp.c b/arch/arm/mach-msm/platsmp.c
index 82eb079..0fdae69 100644
--- a/arch/arm/mach-msm/platsmp.c
+++ b/arch/arm/mach-msm/platsmp.c
@@ -124,6 +124,80 @@ static int msm8960_release_secondary(unsigned int cpu)
 	return 0;
 }
 
+static int msm8974_release_secondary(unsigned int cpu)
+{
+	void __iomem *reg;
+	void __iomem *l2_saw_base;
+	struct device_node *dn = NULL;
+	unsigned apc_pwr_gate_ctl = 0x14;
+	unsigned reg_val;
+
+	if (cpu == 0 || cpu >= num_possible_cpus())
+		return -EINVAL;
+
+	dn = of_find_compatible_node(dn, NULL, "qcom,kpss");
+	if (!dn) {
+		pr_err("%s : Missing kpss node from device tree\n", __func__);
+		return -ENXIO;
+	}
+
+	reg = of_iomap(dn, cpu+1);
+	if (!reg)
+		return -ENOMEM;
+
+	pr_debug("Starting secondary CPU %d\n", cpu);
+
+	/* Turn on the BHS, turn off LDO Bypass and power down LDO */
+	reg_val =  0x403f0001;
+	writel_relaxed(reg_val, reg + apc_pwr_gate_ctl);
+
+	/* complete the above write before the delay */
+	mb();
+	/* wait for the bhs to settle */
+	udelay(1);
+
+	/* Turn on BHS segments */
+	reg_val |= 0x3f << 1;
+	writel_relaxed(reg_val, reg + apc_pwr_gate_ctl);
+
+	/* complete the above write before the delay */
+	mb();
+	 /* wait for the bhs to settle */
+	udelay(1);
+
+	/* Finally turn on the bypass so that BHS supplies power */
+	reg_val |= 0x3f << 8;
+	writel_relaxed(reg_val, reg + apc_pwr_gate_ctl);
+
+	/* enable max phases */
+	l2_saw_base = of_iomap(dn, 0);
+	if (!l2_saw_base) {
+		return -ENOMEM;
+	}
+	writel_relaxed(0x10003, l2_saw_base + 0x1c);
+	mb();
+	udelay(50);
+
+	iounmap(l2_saw_base);
+
+	writel_relaxed(0x021, reg+0x04);
+	mb();
+	udelay(2);
+
+	writel_relaxed(0x020, reg+0x04);
+	mb();
+	udelay(2);
+
+	writel_relaxed(0x000, reg+0x04);
+	mb();
+
+	writel_relaxed(0x080, reg+0x04);
+	mb();
+
+	iounmap(reg);
+	return 0;
+}
+
 static DEFINE_PER_CPU(int, cold_boot_done);
 
 static void boot_cold_cpu(unsigned int cpu)
@@ -151,6 +225,11 @@ static void boot_cold_cpu(unsigned int cpu)
 			msm8960_release_secondary(cpu);
 			per_cpu(cold_boot_done, cpu) = true;
 		}
+	} else if (!strcmp(enable_method, "qcom,kpssv2")) {
+		if (per_cpu(cold_boot_done, cpu) == false) {
+			msm8974_release_secondary(cpu);
+			per_cpu(cold_boot_done, cpu) = true;
+		}
 	} else {
 		pr_err("%s: Invalid enable-method property: %s\n",
 				__func__, enable_method);
-- 
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
hosted by The Linux Foundation

[RESEND PATCH 4/4] ARM: msm: Add support for 8974 SMP

[Rohit Vaswani]

Add the cpus bindings and the Kraitv2 release sequence
to make SMP work for 2 cores on MSM8974.

Signed-off-by: Rohit Vaswani <rvaswani@codeaurora.org>
---
 Documentation/devicetree/bindings/arm/cpus.txt |  1 +
 arch/arm/boot/dts/msm8974.dts                  | 23 ++++++++
 arch/arm/mach-msm/board-dt-8974.c              |  3 +
 arch/arm/mach-msm/platsmp.c                    | 79 ++++++++++++++++++++++++++
 4 files changed, 106 insertions(+)

diff --git a/Documentation/devicetree/bindings/arm/cpus.txt b/Documentation/devicetree/bindings/arm/cpus.txt
index 1132eac..7c3c677 100644
--- a/Documentation/devicetree/bindings/arm/cpus.txt
+++ b/Documentation/devicetree/bindings/arm/cpus.txt
@@ -52,6 +52,7 @@ For the ARM architecture every CPU node must contain the following properties:
 		 This should be one of:
 		 "qcom,scss"
 		 "qcom,kpssv1"
+		 "qcom,kpssv2"
 
 Example:
 
diff --git a/arch/arm/boot/dts/msm8974.dts b/arch/arm/boot/dts/msm8974.dts
index c31c097..ef35a9b 100644
--- a/arch/arm/boot/dts/msm8974.dts
+++ b/arch/arm/boot/dts/msm8974.dts
@@ -7,6 +7,22 @@
 	compatible = "qcom,msm8974";
 	interrupt-parent = <&intc>;
 
+	cpus {
+		#address-cells = <1>;
+		#size-cells = <0>;
+		compatible = "qcom,krait";
+		device_type = "cpu";
+		enable-method = "qcom,kpssv2";
+
+		cpu at 0 {
+			reg = <0>;
+		};
+
+		cpu at 1 {
+			reg = <1>;
+		};
+	};
+
 	intc: interrupt-controller at f9000000 {
 		compatible = "qcom,msm-qgic2";
 		interrupt-controller;
@@ -23,4 +39,11 @@
 			     <1 1 0xf08>;
 		clock-frequency = <19200000>;
 	};
+
+	kpss at f9012000 {
+		compatible = "qcom,kpss";
+		reg = <0xf9012000 0x1000>,
+		      <0xf9088000 0x1000>,
+		      <0xf9098000 0x1000>;
+	};
 };
diff --git a/arch/arm/mach-msm/board-dt-8974.c b/arch/arm/mach-msm/board-dt-8974.c
index d7f84f2..06119f9 100644
--- a/arch/arm/mach-msm/board-dt-8974.c
+++ b/arch/arm/mach-msm/board-dt-8974.c
@@ -13,11 +13,14 @@
 #include <linux/of_platform.h>
 #include <asm/mach/arch.h>
 
+#include "common.h"
+
 static const char * const msm8974_dt_match[] __initconst = {
 	"qcom,msm8974",
 	NULL
 };
 
 DT_MACHINE_START(MSM8974_DT, "Qualcomm MSM (Flattened Device Tree)")
+	.smp = smp_ops(msm_smp_ops),
 	.dt_compat = msm8974_dt_match,
 MACHINE_END
diff --git a/arch/arm/mach-msm/platsmp.c b/arch/arm/mach-msm/platsmp.c
index 82eb079..0fdae69 100644
--- a/arch/arm/mach-msm/platsmp.c
+++ b/arch/arm/mach-msm/platsmp.c
@@ -124,6 +124,80 @@ static int msm8960_release_secondary(unsigned int cpu)
 	return 0;
 }
 
+static int msm8974_release_secondary(unsigned int cpu)
+{
+	void __iomem *reg;
+	void __iomem *l2_saw_base;
+	struct device_node *dn = NULL;
+	unsigned apc_pwr_gate_ctl = 0x14;
+	unsigned reg_val;
+
+	if (cpu == 0 || cpu >= num_possible_cpus())
+		return -EINVAL;
+
+	dn = of_find_compatible_node(dn, NULL, "qcom,kpss");
+	if (!dn) {
+		pr_err("%s : Missing kpss node from device tree\n", __func__);
+		return -ENXIO;
+	}
+
+	reg = of_iomap(dn, cpu+1);
+	if (!reg)
+		return -ENOMEM;
+
+	pr_debug("Starting secondary CPU %d\n", cpu);
+
+	/* Turn on the BHS, turn off LDO Bypass and power down LDO */
+	reg_val =  0x403f0001;
+	writel_relaxed(reg_val, reg + apc_pwr_gate_ctl);
+
+	/* complete the above write before the delay */
+	mb();
+	/* wait for the bhs to settle */
+	udelay(1);
+
+	/* Turn on BHS segments */
+	reg_val |= 0x3f << 1;
+	writel_relaxed(reg_val, reg + apc_pwr_gate_ctl);
+
+	/* complete the above write before the delay */
+	mb();
+	 /* wait for the bhs to settle */
+	udelay(1);
+
+	/* Finally turn on the bypass so that BHS supplies power */
+	reg_val |= 0x3f << 8;
+	writel_relaxed(reg_val, reg + apc_pwr_gate_ctl);
+
+	/* enable max phases */
+	l2_saw_base = of_iomap(dn, 0);
+	if (!l2_saw_base) {
+		return -ENOMEM;
+	}
+	writel_relaxed(0x10003, l2_saw_base + 0x1c);
+	mb();
+	udelay(50);
+
+	iounmap(l2_saw_base);
+
+	writel_relaxed(0x021, reg+0x04);
+	mb();
+	udelay(2);
+
+	writel_relaxed(0x020, reg+0x04);
+	mb();
+	udelay(2);
+
+	writel_relaxed(0x000, reg+0x04);
+	mb();
+
+	writel_relaxed(0x080, reg+0x04);
+	mb();
+
+	iounmap(reg);
+	return 0;
+}
+
 static DEFINE_PER_CPU(int, cold_boot_done);
 
 static void boot_cold_cpu(unsigned int cpu)
@@ -151,6 +225,11 @@ static void boot_cold_cpu(unsigned int cpu)
 			msm8960_release_secondary(cpu);
 			per_cpu(cold_boot_done, cpu) = true;
 		}
+	} else if (!strcmp(enable_method, "qcom,kpssv2")) {
+		if (per_cpu(cold_boot_done, cpu) == false) {
+			msm8974_release_secondary(cpu);
+			per_cpu(cold_boot_done, cpu) = true;
+		}
 	} else {
 		pr_err("%s: Invalid enable-method property: %s\n",
 				__func__, enable_method);
-- 
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
hosted by The Linux Foundation

Re: [PATCH 3/4] ARM: msm: Add SMP support for 8960

[Kumar Gala]

On Aug 1, 2013, at 9:15 PM, Rohit Vaswani wrote:

> Add the cpus bindings and the Krait release sequence
> to make SMP work for MSM8960
> 
> Signed-off-by: Rohit Vaswani <rvaswani@codeaurora.org>
> ---
> Documentation/devicetree/bindings/arm/cpus.txt     |  2 +
> Documentation/devicetree/bindings/arm/msm/kpss.txt | 16 ++++++
> arch/arm/boot/dts/msm8960-cdp.dts                  | 22 +++++++++
> arch/arm/mach-msm/platsmp.c                        | 57 ++++++++++++++++++++++
> arch/arm/mach-msm/scm-boot.h                       |  8 +--
> 5 files changed, 102 insertions(+), 3 deletions(-)
> create mode 100644 Documentation/devicetree/bindings/arm/msm/kpss.txt
> 
> diff --git a/Documentation/devicetree/bindings/arm/cpus.txt b/Documentation/devicetree/bindings/arm/cpus.txt
> index 327aad2..1132eac 100644
> --- a/Documentation/devicetree/bindings/arm/cpus.txt
> +++ b/Documentation/devicetree/bindings/arm/cpus.txt
> @@ -45,11 +45,13 @@ For the ARM architecture every CPU node must contain the following properties:
> 		"marvell,xsc3"
> 		"marvell,xscale"
> 		"qcom,scorpion"
> +		"qcom,krait"
> - enable-method: Specifies the method used to enable or take the secondary cores
> 		 out of reset. This allows different reset sequence for
> 		 different types of cpus.
> 		 This should be one of:
> 		 "qcom,scss"
> +		 "qcom,kpssv1"
> 
> Example:
> 
> diff --git a/Documentation/devicetree/bindings/arm/msm/kpss.txt b/Documentation/devicetree/bindings/arm/msm/kpss.txt
> new file mode 100644
> index 0000000..7272340
> --- /dev/null
> +++ b/Documentation/devicetree/bindings/arm/msm/kpss.txt
> @@ -0,0 +1,16 @@
> +* KPSS - Krait Processor Sub-system
> +
> +Properties
> +
> +- compatible : Should contain "qcom,kpss".
> +
> +- reg: Specifies the base address for the KPSS registers used for
> +       booting up secondary cores.
> +
> +Example:
> +
> +	kpss@2088000 {
> +		compatible = "qcom,kpss";
> +		reg = <0x02088000 0x1000
> +			0x02098000 0x2000>;
> +	};
> diff --git a/arch/arm/boot/dts/msm8960-cdp.dts b/arch/arm/boot/dts/msm8960-cdp.dts
> index db2060c..8c82d5e 100644
> --- a/arch/arm/boot/dts/msm8960-cdp.dts
> +++ b/arch/arm/boot/dts/msm8960-cdp.dts
> @@ -7,6 +7,22 @@
> 	compatible = "qcom,msm8960-cdp", "qcom,msm8960";
> 	interrupt-parent = <&intc>;
> 
> +	cpus {
> +		#address-cells = <1>;
> +		#size-cells = <0>;
> +		compatible = "qcom,krait";
> +		device_type = "cpu";
> +		enable-method = "qcom,kpssv1";
> +
> +		cpu@0 {
> +			reg = <0>;
> +		};
> +
> +		cpu@1 {
> +			reg = <1>;
> +		};
> +	};
> +
> 	intc: interrupt-controller@2000000 {
> 		compatible = "qcom,msm-qgic2";
> 		interrupt-controller;
> @@ -37,6 +53,12 @@
> 		reg = <0xfd510000 0x4000>;
> 	};
> 
> +	kpss@2088000 {
> +		compatible = "qcom,kpss";
> +		reg = <0x02088000 0x1000
> +			0x02098000 0x2000>;
> +	};
> +
> 	serial@16440000 {
> 		compatible = "qcom,msm-hsuart", "qcom,msm-uart";
> 		reg = <0x16440000 0x1000>,
> diff --git a/arch/arm/mach-msm/platsmp.c b/arch/arm/mach-msm/platsmp.c
> index 17022e0..82eb079 100644
> --- a/arch/arm/mach-msm/platsmp.c
> +++ b/arch/arm/mach-msm/platsmp.c
> @@ -74,6 +74,56 @@ static int scorpion_release_secondary(void)
> 	return 0;
> }
> 
> +static int msm8960_release_secondary(unsigned int cpu)
> +{
> +	void __iomem *reg;
> +	struct device_node *dn = NULL;
> +
> +	if (cpu == 0 || cpu >= num_possible_cpus())
> +		return -EINVAL;
> +
> +	dn = of_find_compatible_node(dn, NULL, "qcom,kpss");
> +	if (!dn) {
> +		pr_err("%s : Missing kpss node from device tree\n", __func__);
> +		return -ENXIO;
> +	}
> +
> +	reg = of_iomap(dn, cpu);
> +	if (!reg)
> +		return -ENOMEM;
> +
> +	pr_debug("Starting secondary CPU %d\n", cpu);
> +
> +	/* Turn on CPU Rail */
> +	writel_relaxed(0xA4, reg+0x1014);

Is there some reason we are using magic numbers for both values and offsets?

> +	mb();
> +	udelay(512);
> +
> +	/* Krait bring-up sequence */
> +	writel_relaxed(0x109, reg+0x04);
> +	writel_relaxed(0x101, reg+0x04);
> +	mb();
> +	ndelay(300);
> +
> +	writel_relaxed(0x121, reg+0x04);
> +	mb();
> +	udelay(2);
> +
> +	writel_relaxed(0x120, reg+0x04);
> +	mb();
> +	udelay(2);
> +
> +	writel_relaxed(0x100, reg+0x04);
> +	mb();
> +	udelay(100);
> +
> +	writel_relaxed(0x180, reg+0x04);
> +	mb();
> +
> +	iounmap(reg);
> +	return 0;
> +}
> +
> static DEFINE_PER_CPU(int, cold_boot_done);
> 
> static void boot_cold_cpu(unsigned int cpu)
> @@ -96,6 +146,11 @@ static void boot_cold_cpu(unsigned int cpu)
> 			scorpion_release_secondary();
> 			per_cpu(cold_boot_done, cpu) = true;
> 		}
> +	} else if (!strcmp(enable_method, "qcom,kpssv1")) {
> +		if (per_cpu(cold_boot_done, cpu) == false) {
> +			msm8960_release_secondary(cpu);

Is this really msm8960 specific, if so than we should be doing something other than comparing against "qcom,kpssv1" or we should change the function to kpssv1_release_secondary()

> +			per_cpu(cold_boot_done, cpu) = true;
> +		}
> 	} else {
> 		pr_err("%s: Invalid enable-method property: %s\n",
> 				__func__, enable_method);
> @@ -151,6 +206,8 @@ static void __init msm_smp_init_cpus(void)
> static const int cold_boot_flags[] __initconst = {
> 	0,
> 	SCM_FLAG_COLDBOOT_CPU1,
> +	SCM_FLAG_COLDBOOT_CPU2,
> +	SCM_FLAG_COLDBOOT_CPU3,
> };
> 
> static void __init msm_smp_prepare_cpus(unsigned int max_cpus)
> diff --git a/arch/arm/mach-msm/scm-boot.h b/arch/arm/mach-msm/scm-boot.h
> index 7be32ff..6aabb24 100644
> --- a/arch/arm/mach-msm/scm-boot.h
> +++ b/arch/arm/mach-msm/scm-boot.h
> @@ -13,9 +13,11 @@
> #define __MACH_SCM_BOOT_H
> 
> #define SCM_BOOT_ADDR			0x1
> -#define SCM_FLAG_COLDBOOT_CPU1		0x1
> -#define SCM_FLAG_WARMBOOT_CPU1		0x2
> -#define SCM_FLAG_WARMBOOT_CPU0		0x4
> +#define SCM_FLAG_COLDBOOT_CPU1		0x01
> +#define SCM_FLAG_COLDBOOT_CPU2		0x08
> +#define SCM_FLAG_COLDBOOT_CPU3		0x20
> +#define SCM_FLAG_WARMBOOT_CPU0		0x04
> +#define SCM_FLAG_WARMBOOT_CPU1		0x02
> 
> int scm_set_boot_addr(phys_addr_t addr, int flags);
> 
> -- 
> The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
> hosted by The Linux Foundation
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-arm-msm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

--
Employee of Qualcomm Innovation Center, Inc.
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation

[PATCH 3/4] ARM: msm: Add SMP support for 8960

[Kumar Gala]

On Aug 1, 2013, at 9:15 PM, Rohit Vaswani wrote:

> Add the cpus bindings and the Krait release sequence
> to make SMP work for MSM8960
> 
> Signed-off-by: Rohit Vaswani <rvaswani@codeaurora.org>
> ---
> Documentation/devicetree/bindings/arm/cpus.txt     |  2 +
> Documentation/devicetree/bindings/arm/msm/kpss.txt | 16 ++++++
> arch/arm/boot/dts/msm8960-cdp.dts                  | 22 +++++++++
> arch/arm/mach-msm/platsmp.c                        | 57 ++++++++++++++++++++++
> arch/arm/mach-msm/scm-boot.h                       |  8 +--
> 5 files changed, 102 insertions(+), 3 deletions(-)
> create mode 100644 Documentation/devicetree/bindings/arm/msm/kpss.txt
> 
> diff --git a/Documentation/devicetree/bindings/arm/cpus.txt b/Documentation/devicetree/bindings/arm/cpus.txt
> index 327aad2..1132eac 100644
> --- a/Documentation/devicetree/bindings/arm/cpus.txt
> +++ b/Documentation/devicetree/bindings/arm/cpus.txt
> @@ -45,11 +45,13 @@ For the ARM architecture every CPU node must contain the following properties:
> 		"marvell,xsc3"
> 		"marvell,xscale"
> 		"qcom,scorpion"
> +		"qcom,krait"
> - enable-method: Specifies the method used to enable or take the secondary cores
> 		 out of reset. This allows different reset sequence for
> 		 different types of cpus.
> 		 This should be one of:
> 		 "qcom,scss"
> +		 "qcom,kpssv1"
> 
> Example:
> 
> diff --git a/Documentation/devicetree/bindings/arm/msm/kpss.txt b/Documentation/devicetree/bindings/arm/msm/kpss.txt
> new file mode 100644
> index 0000000..7272340
> --- /dev/null
> +++ b/Documentation/devicetree/bindings/arm/msm/kpss.txt
> @@ -0,0 +1,16 @@
> +* KPSS - Krait Processor Sub-system
> +
> +Properties
> +
> +- compatible : Should contain "qcom,kpss".
> +
> +- reg: Specifies the base address for the KPSS registers used for
> +       booting up secondary cores.
> +
> +Example:
> +
> +	kpss at 2088000 {
> +		compatible = "qcom,kpss";
> +		reg = <0x02088000 0x1000
> +			0x02098000 0x2000>;
> +	};
> diff --git a/arch/arm/boot/dts/msm8960-cdp.dts b/arch/arm/boot/dts/msm8960-cdp.dts
> index db2060c..8c82d5e 100644
> --- a/arch/arm/boot/dts/msm8960-cdp.dts
> +++ b/arch/arm/boot/dts/msm8960-cdp.dts
> @@ -7,6 +7,22 @@
> 	compatible = "qcom,msm8960-cdp", "qcom,msm8960";
> 	interrupt-parent = <&intc>;
> 
> +	cpus {
> +		#address-cells = <1>;
> +		#size-cells = <0>;
> +		compatible = "qcom,krait";
> +		device_type = "cpu";
> +		enable-method = "qcom,kpssv1";
> +
> +		cpu at 0 {
> +			reg = <0>;
> +		};
> +
> +		cpu at 1 {
> +			reg = <1>;
> +		};
> +	};
> +
> 	intc: interrupt-controller at 2000000 {
> 		compatible = "qcom,msm-qgic2";
> 		interrupt-controller;
> @@ -37,6 +53,12 @@
> 		reg = <0xfd510000 0x4000>;
> 	};
> 
> +	kpss at 2088000 {
> +		compatible = "qcom,kpss";
> +		reg = <0x02088000 0x1000
> +			0x02098000 0x2000>;
> +	};
> +
> 	serial at 16440000 {
> 		compatible = "qcom,msm-hsuart", "qcom,msm-uart";
> 		reg = <0x16440000 0x1000>,
> diff --git a/arch/arm/mach-msm/platsmp.c b/arch/arm/mach-msm/platsmp.c
> index 17022e0..82eb079 100644
> --- a/arch/arm/mach-msm/platsmp.c
> +++ b/arch/arm/mach-msm/platsmp.c
> @@ -74,6 +74,56 @@ static int scorpion_release_secondary(void)
> 	return 0;
> }
> 
> +static int msm8960_release_secondary(unsigned int cpu)
> +{
> +	void __iomem *reg;
> +	struct device_node *dn = NULL;
> +
> +	if (cpu == 0 || cpu >= num_possible_cpus())
> +		return -EINVAL;
> +
> +	dn = of_find_compatible_node(dn, NULL, "qcom,kpss");
> +	if (!dn) {
> +		pr_err("%s : Missing kpss node from device tree\n", __func__);
> +		return -ENXIO;
> +	}
> +
> +	reg = of_iomap(dn, cpu);
> +	if (!reg)
> +		return -ENOMEM;
> +
> +	pr_debug("Starting secondary CPU %d\n", cpu);
> +
> +	/* Turn on CPU Rail */
> +	writel_relaxed(0xA4, reg+0x1014);

Is there some reason we are using magic numbers for both values and offsets?

> +	mb();
> +	udelay(512);
> +
> +	/* Krait bring-up sequence */
> +	writel_relaxed(0x109, reg+0x04);
> +	writel_relaxed(0x101, reg+0x04);
> +	mb();
> +	ndelay(300);
> +
> +	writel_relaxed(0x121, reg+0x04);
> +	mb();
> +	udelay(2);
> +
> +	writel_relaxed(0x120, reg+0x04);
> +	mb();
> +	udelay(2);
> +
> +	writel_relaxed(0x100, reg+0x04);
> +	mb();
> +	udelay(100);
> +
> +	writel_relaxed(0x180, reg+0x04);
> +	mb();
> +
> +	iounmap(reg);
> +	return 0;
> +}
> +
> static DEFINE_PER_CPU(int, cold_boot_done);
> 
> static void boot_cold_cpu(unsigned int cpu)
> @@ -96,6 +146,11 @@ static void boot_cold_cpu(unsigned int cpu)
> 			scorpion_release_secondary();
> 			per_cpu(cold_boot_done, cpu) = true;
> 		}
> +	} else if (!strcmp(enable_method, "qcom,kpssv1")) {
> +		if (per_cpu(cold_boot_done, cpu) == false) {
> +			msm8960_release_secondary(cpu);

Is this really msm8960 specific, if so than we should be doing something other than comparing against "qcom,kpssv1" or we should change the function to kpssv1_release_secondary()

> +			per_cpu(cold_boot_done, cpu) = true;
> +		}
> 	} else {
> 		pr_err("%s: Invalid enable-method property: %s\n",
> 				__func__, enable_method);
> @@ -151,6 +206,8 @@ static void __init msm_smp_init_cpus(void)
> static const int cold_boot_flags[] __initconst = {
> 	0,
> 	SCM_FLAG_COLDBOOT_CPU1,
> +	SCM_FLAG_COLDBOOT_CPU2,
> +	SCM_FLAG_COLDBOOT_CPU3,
> };
> 
> static void __init msm_smp_prepare_cpus(unsigned int max_cpus)
> diff --git a/arch/arm/mach-msm/scm-boot.h b/arch/arm/mach-msm/scm-boot.h
> index 7be32ff..6aabb24 100644
> --- a/arch/arm/mach-msm/scm-boot.h
> +++ b/arch/arm/mach-msm/scm-boot.h
> @@ -13,9 +13,11 @@
> #define __MACH_SCM_BOOT_H
> 
> #define SCM_BOOT_ADDR			0x1
> -#define SCM_FLAG_COLDBOOT_CPU1		0x1
> -#define SCM_FLAG_WARMBOOT_CPU1		0x2
> -#define SCM_FLAG_WARMBOOT_CPU0		0x4
> +#define SCM_FLAG_COLDBOOT_CPU1		0x01
> +#define SCM_FLAG_COLDBOOT_CPU2		0x08
> +#define SCM_FLAG_COLDBOOT_CPU3		0x20
> +#define SCM_FLAG_WARMBOOT_CPU0		0x04
> +#define SCM_FLAG_WARMBOOT_CPU1		0x02
> 
> int scm_set_boot_addr(phys_addr_t addr, int flags);
> 
> -- 
> The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
> hosted by The Linux Foundation
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-arm-msm" in
> the body of a message to majordomo at vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

--
Employee of Qualcomm Innovation Center, Inc.
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation

Re: [RESEND PATCH 4/4] ARM: msm: Add support for 8974 SMP

[Kumar Gala]

On Aug 1, 2013, at 9:15 PM, Rohit Vaswani wrote:

> Add the cpus bindings and the Kraitv2 release sequence
> to make SMP work for 2 cores on MSM8974.
> 
> Signed-off-by: Rohit Vaswani <rvaswani@codeaurora.org>
> ---
> Documentation/devicetree/bindings/arm/cpus.txt |  1 +
> arch/arm/boot/dts/msm8974.dts                  | 23 ++++++++
> arch/arm/mach-msm/board-dt-8974.c              |  3 +
> arch/arm/mach-msm/platsmp.c                    | 79 ++++++++++++++++++++++++++
> 4 files changed, 106 insertions(+)
> 
> diff --git a/Documentation/devicetree/bindings/arm/cpus.txt b/Documentation/devicetree/bindings/arm/cpus.txt
> index 1132eac..7c3c677 100644
> --- a/Documentation/devicetree/bindings/arm/cpus.txt
> +++ b/Documentation/devicetree/bindings/arm/cpus.txt
> @@ -52,6 +52,7 @@ For the ARM architecture every CPU node must contain the following properties:
> 		 This should be one of:
> 		 "qcom,scss"
> 		 "qcom,kpssv1"
> +		 "qcom,kpssv2"
> 
> Example:
> 
> diff --git a/arch/arm/boot/dts/msm8974.dts b/arch/arm/boot/dts/msm8974.dts
> index c31c097..ef35a9b 100644
> --- a/arch/arm/boot/dts/msm8974.dts
> +++ b/arch/arm/boot/dts/msm8974.dts
> @@ -7,6 +7,22 @@
> 	compatible = "qcom,msm8974";
> 	interrupt-parent = <&intc>;
> 
> +	cpus {
> +		#address-cells = <1>;
> +		#size-cells = <0>;
> +		compatible = "qcom,krait";
> +		device_type = "cpu";
> +		enable-method = "qcom,kpssv2";
> +
> +		cpu@0 {
> +			reg = <0>;
> +		};
> +
> +		cpu@1 {
> +			reg = <1>;
> +		};

Any reason not to have all 4 cores?

> +	};
> +
> 	intc: interrupt-controller@f9000000 {
> 		compatible = "qcom,msm-qgic2";
> 		interrupt-controller;
> @@ -23,4 +39,11 @@
> 			     <1 1 0xf08>;
> 		clock-frequency = <19200000>;
> 	};
> +
> +	kpss@f9012000 {
> +		compatible = "qcom,kpss";
> +		reg = <0xf9012000 0x1000>,
> +		      <0xf9088000 0x1000>,
> +		      <0xf9098000 0x1000>;

we should probably have regnmaes to go along with this.

Also this doesn't really match the binding spec, as you've included the L2 register, we should cleanup the binding spec to be more precise.

> +	};
> };
> diff --git a/arch/arm/mach-msm/board-dt-8974.c b/arch/arm/mach-msm/board-dt-8974.c
> index d7f84f2..06119f9 100644
> --- a/arch/arm/mach-msm/board-dt-8974.c
> +++ b/arch/arm/mach-msm/board-dt-8974.c
> @@ -13,11 +13,14 @@
> #include <linux/of_platform.h>
> #include <asm/mach/arch.h>
> 
> +#include "common.h"
> +
> static const char * const msm8974_dt_match[] __initconst = {
> 	"qcom,msm8974",
> 	NULL
> };
> 
> DT_MACHINE_START(MSM8974_DT, "Qualcomm MSM (Flattened Device Tree)")
> +	.smp = smp_ops(msm_smp_ops),
> 	.dt_compat = msm8974_dt_match,
> MACHINE_END
> diff --git a/arch/arm/mach-msm/platsmp.c b/arch/arm/mach-msm/platsmp.c
> index 82eb079..0fdae69 100644
> --- a/arch/arm/mach-msm/platsmp.c
> +++ b/arch/arm/mach-msm/platsmp.c
> @@ -124,6 +124,80 @@ static int msm8960_release_secondary(unsigned int cpu)
> 	return 0;
> }
> 
> +static int msm8974_release_secondary(unsigned int cpu)
> +{
> +	void __iomem *reg;
> +	void __iomem *l2_saw_base;
> +	struct device_node *dn = NULL;
> +	unsigned apc_pwr_gate_ctl = 0x14;
> +	unsigned reg_val;
> +
> +	if (cpu == 0 || cpu >= num_possible_cpus())
> +		return -EINVAL;
> +
> +	dn = of_find_compatible_node(dn, NULL, "qcom,kpss");
> +	if (!dn) {
> +		pr_err("%s : Missing kpss node from device tree\n", __func__);
> +		return -ENXIO;
> +	}
> +
> +	reg = of_iomap(dn, cpu+1);
> +	if (!reg)
> +		return -ENOMEM;
> +
> +	pr_debug("Starting secondary CPU %d\n", cpu);
> +
> +	/* Turn on the BHS, turn off LDO Bypass and power down LDO */
> +	reg_val =  0x403f0001;
> +	writel_relaxed(reg_val, reg + apc_pwr_gate_ctl);
> +
> +	/* complete the above write before the delay */
> +	mb();
> +	/* wait for the bhs to settle */
> +	udelay(1);
> +
> +	/* Turn on BHS segments */
> +	reg_val |= 0x3f << 1;
> +	writel_relaxed(reg_val, reg + apc_pwr_gate_ctl);
> +
> +	/* complete the above write before the delay */
> +	mb();
> +	 /* wait for the bhs to settle */
> +	udelay(1);
> +
> +	/* Finally turn on the bypass so that BHS supplies power */
> +	reg_val |= 0x3f << 8;
> +	writel_relaxed(reg_val, reg + apc_pwr_gate_ctl);
> +
> +	/* enable max phases */
> +	l2_saw_base = of_iomap(dn, 0);
> +	if (!l2_saw_base) {
> +		return -ENOMEM;
> +	}
> +	writel_relaxed(0x10003, l2_saw_base + 0x1c);
> +	mb();
> +	udelay(50);
> +
> +	iounmap(l2_saw_base);
> +
> +	writel_relaxed(0x021, reg+0x04);
> +	mb();
> +	udelay(2);
> +
> +	writel_relaxed(0x020, reg+0x04);
> +	mb();
> +	udelay(2);
> +
> +	writel_relaxed(0x000, reg+0x04);
> +	mb();
> +
> +	writel_relaxed(0x080, reg+0x04);
> +	mb();
> +
> +	iounmap(reg);
> +	return 0;
> +}
> +
> static DEFINE_PER_CPU(int, cold_boot_done);
> 
> static void boot_cold_cpu(unsigned int cpu)
> @@ -151,6 +225,11 @@ static void boot_cold_cpu(unsigned int cpu)
> 			msm8960_release_secondary(cpu);
> 			per_cpu(cold_boot_done, cpu) = true;
> 		}
> +	} else if (!strcmp(enable_method, "qcom,kpssv2")) {
> +		if (per_cpu(cold_boot_done, cpu) == false) {
> +			msm8974_release_secondary(cpu);

same comment as on the 8960 patch.

> +			per_cpu(cold_boot_done, cpu) = true;
> +		}
> 	} else {
> 		pr_err("%s: Invalid enable-method property: %s\n",
> 				__func__, enable_method);
> -- 
> The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
> hosted by The Linux Foundation
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-arm-msm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

--
Employee of Qualcomm Innovation Center, Inc.
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation

[RESEND PATCH 4/4] ARM: msm: Add support for 8974 SMP

[Kumar Gala]

On Aug 1, 2013, at 9:15 PM, Rohit Vaswani wrote:

> Add the cpus bindings and the Kraitv2 release sequence
> to make SMP work for 2 cores on MSM8974.
> 
> Signed-off-by: Rohit Vaswani <rvaswani@codeaurora.org>
> ---
> Documentation/devicetree/bindings/arm/cpus.txt |  1 +
> arch/arm/boot/dts/msm8974.dts                  | 23 ++++++++
> arch/arm/mach-msm/board-dt-8974.c              |  3 +
> arch/arm/mach-msm/platsmp.c                    | 79 ++++++++++++++++++++++++++
> 4 files changed, 106 insertions(+)
> 
> diff --git a/Documentation/devicetree/bindings/arm/cpus.txt b/Documentation/devicetree/bindings/arm/cpus.txt
> index 1132eac..7c3c677 100644
> --- a/Documentation/devicetree/bindings/arm/cpus.txt
> +++ b/Documentation/devicetree/bindings/arm/cpus.txt
> @@ -52,6 +52,7 @@ For the ARM architecture every CPU node must contain the following properties:
> 		 This should be one of:
> 		 "qcom,scss"
> 		 "qcom,kpssv1"
> +		 "qcom,kpssv2"
> 
> Example:
> 
> diff --git a/arch/arm/boot/dts/msm8974.dts b/arch/arm/boot/dts/msm8974.dts
> index c31c097..ef35a9b 100644
> --- a/arch/arm/boot/dts/msm8974.dts
> +++ b/arch/arm/boot/dts/msm8974.dts
> @@ -7,6 +7,22 @@
> 	compatible = "qcom,msm8974";
> 	interrupt-parent = <&intc>;
> 
> +	cpus {
> +		#address-cells = <1>;
> +		#size-cells = <0>;
> +		compatible = "qcom,krait";
> +		device_type = "cpu";
> +		enable-method = "qcom,kpssv2";
> +
> +		cpu at 0 {
> +			reg = <0>;
> +		};
> +
> +		cpu at 1 {
> +			reg = <1>;
> +		};

Any reason not to have all 4 cores?

> +	};
> +
> 	intc: interrupt-controller at f9000000 {
> 		compatible = "qcom,msm-qgic2";
> 		interrupt-controller;
> @@ -23,4 +39,11 @@
> 			     <1 1 0xf08>;
> 		clock-frequency = <19200000>;
> 	};
> +
> +	kpss at f9012000 {
> +		compatible = "qcom,kpss";
> +		reg = <0xf9012000 0x1000>,
> +		      <0xf9088000 0x1000>,
> +		      <0xf9098000 0x1000>;

we should probably have regnmaes to go along with this.

Also this doesn't really match the binding spec, as you've included the L2 register, we should cleanup the binding spec to be more precise.

> +	};
> };
> diff --git a/arch/arm/mach-msm/board-dt-8974.c b/arch/arm/mach-msm/board-dt-8974.c
> index d7f84f2..06119f9 100644
> --- a/arch/arm/mach-msm/board-dt-8974.c
> +++ b/arch/arm/mach-msm/board-dt-8974.c
> @@ -13,11 +13,14 @@
> #include <linux/of_platform.h>
> #include <asm/mach/arch.h>
> 
> +#include "common.h"
> +
> static const char * const msm8974_dt_match[] __initconst = {
> 	"qcom,msm8974",
> 	NULL
> };
> 
> DT_MACHINE_START(MSM8974_DT, "Qualcomm MSM (Flattened Device Tree)")
> +	.smp = smp_ops(msm_smp_ops),
> 	.dt_compat = msm8974_dt_match,
> MACHINE_END
> diff --git a/arch/arm/mach-msm/platsmp.c b/arch/arm/mach-msm/platsmp.c
> index 82eb079..0fdae69 100644
> --- a/arch/arm/mach-msm/platsmp.c
> +++ b/arch/arm/mach-msm/platsmp.c
> @@ -124,6 +124,80 @@ static int msm8960_release_secondary(unsigned int cpu)
> 	return 0;
> }
> 
> +static int msm8974_release_secondary(unsigned int cpu)
> +{
> +	void __iomem *reg;
> +	void __iomem *l2_saw_base;
> +	struct device_node *dn = NULL;
> +	unsigned apc_pwr_gate_ctl = 0x14;
> +	unsigned reg_val;
> +
> +	if (cpu == 0 || cpu >= num_possible_cpus())
> +		return -EINVAL;
> +
> +	dn = of_find_compatible_node(dn, NULL, "qcom,kpss");
> +	if (!dn) {
> +		pr_err("%s : Missing kpss node from device tree\n", __func__);
> +		return -ENXIO;
> +	}
> +
> +	reg = of_iomap(dn, cpu+1);
> +	if (!reg)
> +		return -ENOMEM;
> +
> +	pr_debug("Starting secondary CPU %d\n", cpu);
> +
> +	/* Turn on the BHS, turn off LDO Bypass and power down LDO */
> +	reg_val =  0x403f0001;
> +	writel_relaxed(reg_val, reg + apc_pwr_gate_ctl);
> +
> +	/* complete the above write before the delay */
> +	mb();
> +	/* wait for the bhs to settle */
> +	udelay(1);
> +
> +	/* Turn on BHS segments */
> +	reg_val |= 0x3f << 1;
> +	writel_relaxed(reg_val, reg + apc_pwr_gate_ctl);
> +
> +	/* complete the above write before the delay */
> +	mb();
> +	 /* wait for the bhs to settle */
> +	udelay(1);
> +
> +	/* Finally turn on the bypass so that BHS supplies power */
> +	reg_val |= 0x3f << 8;
> +	writel_relaxed(reg_val, reg + apc_pwr_gate_ctl);
> +
> +	/* enable max phases */
> +	l2_saw_base = of_iomap(dn, 0);
> +	if (!l2_saw_base) {
> +		return -ENOMEM;
> +	}
> +	writel_relaxed(0x10003, l2_saw_base + 0x1c);
> +	mb();
> +	udelay(50);
> +
> +	iounmap(l2_saw_base);
> +
> +	writel_relaxed(0x021, reg+0x04);
> +	mb();
> +	udelay(2);
> +
> +	writel_relaxed(0x020, reg+0x04);
> +	mb();
> +	udelay(2);
> +
> +	writel_relaxed(0x000, reg+0x04);
> +	mb();
> +
> +	writel_relaxed(0x080, reg+0x04);
> +	mb();
> +
> +	iounmap(reg);
> +	return 0;
> +}
> +
> static DEFINE_PER_CPU(int, cold_boot_done);
> 
> static void boot_cold_cpu(unsigned int cpu)
> @@ -151,6 +225,11 @@ static void boot_cold_cpu(unsigned int cpu)
> 			msm8960_release_secondary(cpu);
> 			per_cpu(cold_boot_done, cpu) = true;
> 		}
> +	} else if (!strcmp(enable_method, "qcom,kpssv2")) {
> +		if (per_cpu(cold_boot_done, cpu) == false) {
> +			msm8974_release_secondary(cpu);

same comment as on the 8960 patch.

> +			per_cpu(cold_boot_done, cpu) = true;
> +		}
> 	} else {
> 		pr_err("%s: Invalid enable-method property: %s\n",
> 				__func__, enable_method);
> -- 
> The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
> hosted by The Linux Foundation
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-arm-msm" in
> the body of a message to majordomo at vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

--
Employee of Qualcomm Innovation Center, Inc.
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation

Re: [RESEND PATCH 2/4] ARM: msm: Re-organize platsmp to make it extensible

[Mark Rutland]

Hi,

Apologies for the long delay for review on this.

I really like the direction this is going, but I have some qualms with
the implementation.

On Fri, Aug 02, 2013 at 03:15:23AM +0100, Rohit Vaswani wrote:
> This makes it easy to add SMP support for new targets
> by adding cpus property and the release sequence.
> We add the enable-method property for the cpus property to
> specify which release sequence to use.
> While at it, add the 8660 cpus bindings to make SMP work.
> 
> Signed-off-by: Rohit Vaswani <rvaswani@codeaurora.org>
> ---
>  Documentation/devicetree/bindings/arm/cpus.txt     |  6 ++
>  Documentation/devicetree/bindings/arm/msm/scss.txt | 15 ++++
>  arch/arm/boot/dts/msm8660-surf.dts                 | 23 +++++-
>  arch/arm/mach-msm/platsmp.c                        | 94 +++++++++++++++++-----
>  4 files changed, 115 insertions(+), 23 deletions(-)
>  create mode 100644 Documentation/devicetree/bindings/arm/msm/scss.txt
> 
> diff --git a/Documentation/devicetree/bindings/arm/cpus.txt b/Documentation/devicetree/bindings/arm/cpus.txt
> index f32494d..327aad2 100644
> --- a/Documentation/devicetree/bindings/arm/cpus.txt
> +++ b/Documentation/devicetree/bindings/arm/cpus.txt
> @@ -44,6 +44,12 @@ For the ARM architecture every CPU node must contain the following properties:
>  		"marvell,mohawk"
>  		"marvell,xsc3"
>  		"marvell,xscale"
> +		"qcom,scorpion"
> +- enable-method: Specifies the method used to enable or take the secondary cores
> +		 out of reset. This allows different reset sequence for
> +		 different types of cpus.
> +		 This should be one of:
> +		 "qcom,scss"

While I'd certainly like to see a move to using enable-method for
32-bit, I think this needs a bit more thought:

What does "qcom,scss" mean, precisely? It seems to be that we poke some
registers in a "qcom,scss" device. I think we need to document
enable-methods *very* carefully (and we need to do that for 64-bit as
well with psci), as it's very likely there'll be subtle
incompatibilities between platforms, especially if firmware is involved.
We should try to leaves as little room for error as possible.

I don't want to see this devolve into meaning "whatever the Linux driver
for this happens to do at the current point in time", as that just leads
to breakage as we have no clear distinction between actual requirements
and implementation details.

Given we already have platforms without an enable-method, we can't make
it a required property either -- those platforms are already booting by
figuring out an enable method from the platform's compatible string.

With PSCI, enable-method also implies a method for disabling a
particular CPU, so it would be nice for the description to cover this.

>  
>  Example:
>  
> diff --git a/Documentation/devicetree/bindings/arm/msm/scss.txt b/Documentation/devicetree/bindings/arm/msm/scss.txt
> new file mode 100644
> index 0000000..21c3e26
> --- /dev/null
> +++ b/Documentation/devicetree/bindings/arm/msm/scss.txt
> @@ -0,0 +1,15 @@
> +* SCSS - Scorpion Sub-system
> +
> +Properties
> +
> +- compatible : Should contain "qcom,scss".
> +
> +- reg: Specifies the base address for the SCSS registers used for
> +       booting up secondary cores.
> +
> +Example:
> +
> +	scss@902000 {
> +		compatible = "qcom,scss";
> +		reg = <0x00902000 0x2000>;
> +	};
> diff --git a/arch/arm/boot/dts/msm8660-surf.dts b/arch/arm/boot/dts/msm8660-surf.dts
> index cdc010e..203e51a 100644
> --- a/arch/arm/boot/dts/msm8660-surf.dts
> +++ b/arch/arm/boot/dts/msm8660-surf.dts
> @@ -7,6 +7,22 @@
>  	compatible = "qcom,msm8660-surf", "qcom,msm8660";
>  	interrupt-parent = <&intc>;
>  
> +	cpus {
> +		#address-cells = <1>;
> +		#size-cells = <0>;
> +		compatible = "qcom,scorpion";
> +		device_type = "cpu";
> +		enable-method = "qcom,scss";
> +
> +		cpu@0 {
> +			reg = <0>;
> +		};
> +
> +		cpu@1 {
> +			reg = <1>;
> +		};
> +	};
> +

I *really* like moving the common properties out into the /cpus node --
ePAPR suggests it, it's less error prone, and it takes up less space.
However, I'm not sure our generic/arch code handles it properly, and I
think we need to audit that before we can start writing dts that depend
on it. I'd be happy to be wrong here if anyone can correct me. :)

We probably need to factor out the way we read properties for cpu nodes,
falling back to ones present in /cpus if not present. There's already a
lot of pain getting the node for a logical (rather than physical) cpu
id.

Sudeep recently factored out finding the cpu node for a logical cpu id
[1,2] in generic code with a per-arch callback, it shouldn't be too hard
to have shims around that to read (optionally) common properties.

[...]

> -static void prepare_cold_cpu(unsigned int cpu)
> +static int scorpion_release_secondary(void)
>  {
> -	int ret;
> -	ret = scm_set_boot_addr(virt_to_phys(secondary_startup),
> -				SCM_FLAG_COLDBOOT_CPU1);
> -	if (ret == 0) {
> -		void __iomem *sc1_base_ptr;
> -		sc1_base_ptr = ioremap_nocache(0x00902000, SZ_4K*2);
> -		if (sc1_base_ptr) {
> -			writel(0, sc1_base_ptr + VDD_SC1_ARRAY_CLAMP_GFS_CTL);
> -			writel(0, sc1_base_ptr + SCSS_CPU1CORE_RESET);
> -			writel(3, sc1_base_ptr + SCSS_DBG_STATUS_CORE_PWRDUP);
> -			iounmap(sc1_base_ptr);
> -		}
> -	} else
> -		printk(KERN_DEBUG "Failed to set secondary core boot "
> -				  "address\n");
> +	void __iomem *sc1_base_ptr;
> +	struct device_node *dn = NULL;
> +
> +	dn = of_find_compatible_node(dn, NULL, "qcom,scss");
> +	if (!dn) {
> +		pr_err("%s: Missing scss node in device tree\n", __func__);
> +		return -ENXIO;
> +	}
> +
> +	sc1_base_ptr = of_iomap(dn, 0);
> +	if (sc1_base_ptr) {
> +		writel_relaxed(0, sc1_base_ptr + VDD_SC1_ARRAY_CLAMP_GFS_CTL);
> +		writel_relaxed(0, sc1_base_ptr + SCSS_CPU1CORE_RESET);
> +		writel_relaxed(3, sc1_base_ptr + SCSS_DBG_STATUS_CORE_PWRDUP);
> +		mb();
> +		iounmap(sc1_base_ptr);

Does this boot *all* secondary CPUs directly into the kernel? Is that
safe (e.g. if the kernel supports fewer CPUs than are physically
present)?

Is this a one-time thing, or are we able to dynamically hotplug CPUs? If
the latter, the map/unmap looks odd to me.

> +	} else {
> +		return -ENOMEM;
> +	}
> +
> +	return 0;
>  }
>  
> -static int msm_boot_secondary(unsigned int cpu, struct task_struct *idle)
> +static DEFINE_PER_CPU(int, cold_boot_done);
> +
> +static void boot_cold_cpu(unsigned int cpu)
>  {
> -	static int cold_boot_done;
> +	const char *enable_method;
> +	struct device_node *dn = NULL;
>  
> -	/* Only need to bring cpu out of reset this way once */
> -	if (cold_boot_done == false) {
> -		prepare_cold_cpu(cpu);
> -		cold_boot_done = true;
> +	dn = of_find_node_by_name(dn, "cpus");
> +	if (!dn) {
> +		pr_err("%s: Missing node cpus in device tree\n", __func__);
> +		return;
>  	}
>  
> +	enable_method = of_get_property(dn, "enable-method", NULL);

Please factor this out from the platform code.

If we're going to use enable-method, it should be decoupled from
platform code -- common code should parse it to find the appropriate
handler. Also, we *must* support having an enable-method per-cpu as KVM
does for PSCI (though I definitely would like support for shared
properties as mentioned above).

> +	if (!enable_method) {
> +			pr_err("%s: cpus node is missing enable-method property\n",
> +					__func__);
> +	} else if (!strcmp(enable_method, "qcom,scss")) {
> +		if (per_cpu(cold_boot_done, cpu) == false) {
> +			scorpion_release_secondary();
> +			per_cpu(cold_boot_done, cpu) = true;
> +		}
> +	} else {
> +		pr_err("%s: Invalid enable-method property: %s\n",
> +				__func__, enable_method);
> +	}
> +}
> +
> +static int msm_boot_secondary(unsigned int cpu, struct task_struct *idle)
> +{
> +	boot_cold_cpu(cpu);
> +
>  	/*
>  	 * set synchronisation state between this boot processor
>  	 * and the secondary one
> @@ -118,8 +148,28 @@ static void __init msm_smp_init_cpus(void)
>  		set_cpu_possible(i, true);
>  }
>  
> +static const int cold_boot_flags[] __initconst = {
> +	0,
> +	SCM_FLAG_COLDBOOT_CPU1,
> +};

So we only ever support booting two CPUs?

Is the mapping of MPIDR to register bits arbitrary? Or do we know what
they would be for four CPUs, four clusters, and beyond?

> +
>  static void __init msm_smp_prepare_cpus(unsigned int max_cpus)
>  {
> +	int cpu, map;
> +	unsigned int flags = 0;
> +
> +	for_each_present_cpu(cpu) {
> +		map = cpu_logical_map(cpu);
> +		if (map > ARRAY_SIZE(cold_boot_flags)) {
> +			set_cpu_present(cpu, false);
> +			__WARN();
> +			continue;
> +		}
> +		flags |= cold_boot_flags[map];

It's a shame that this leaves a window where some CPUs seem bootable,
but aren't (though I can't offer a better way of handling this given we
have dts without enable-method properties).

> +	}
> +
> +	if (scm_set_boot_addr(virt_to_phys(secondary_startup), flags))
> +		pr_warn("Failed to set CPU boot address\n");
>  }
>  
>  struct smp_operations msm_smp_ops __initdata = {
> -- 
> The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
> hosted by The Linux Foundation
> 

Thanks,
Mark

[1] http://lists.infradead.org/pipermail/linux-arm-kernel/2013-July/184150.html
[2] http://lists.infradead.org/pipermail/linux-arm-kernel/2013-August/189619.html

Re: [RESEND PATCH 2/4] ARM: msm: Re-organize platsmp to make it extensible

[Mark Rutland]

Hi,

Apologies for the long delay for review on this.

I really like the direction this is going, but I have some qualms with
the implementation.

On Fri, Aug 02, 2013 at 03:15:23AM +0100, Rohit Vaswani wrote:
> This makes it easy to add SMP support for new targets
> by adding cpus property and the release sequence.
> We add the enable-method property for the cpus property to
> specify which release sequence to use.
> While at it, add the 8660 cpus bindings to make SMP work.
> 
> Signed-off-by: Rohit Vaswani <rvaswani@codeaurora.org>
> ---
>  Documentation/devicetree/bindings/arm/cpus.txt     |  6 ++
>  Documentation/devicetree/bindings/arm/msm/scss.txt | 15 ++++
>  arch/arm/boot/dts/msm8660-surf.dts                 | 23 +++++-
>  arch/arm/mach-msm/platsmp.c                        | 94 +++++++++++++++++-----
>  4 files changed, 115 insertions(+), 23 deletions(-)
>  create mode 100644 Documentation/devicetree/bindings/arm/msm/scss.txt
> 
> diff --git a/Documentation/devicetree/bindings/arm/cpus.txt b/Documentation/devicetree/bindings/arm/cpus.txt
> index f32494d..327aad2 100644
> --- a/Documentation/devicetree/bindings/arm/cpus.txt
> +++ b/Documentation/devicetree/bindings/arm/cpus.txt
> @@ -44,6 +44,12 @@ For the ARM architecture every CPU node must contain the following properties:
>  		"marvell,mohawk"
>  		"marvell,xsc3"
>  		"marvell,xscale"
> +		"qcom,scorpion"
> +- enable-method: Specifies the method used to enable or take the secondary cores
> +		 out of reset. This allows different reset sequence for
> +		 different types of cpus.
> +		 This should be one of:
> +		 "qcom,scss"

While I'd certainly like to see a move to using enable-method for
32-bit, I think this needs a bit more thought:

What does "qcom,scss" mean, precisely? It seems to be that we poke some
registers in a "qcom,scss" device. I think we need to document
enable-methods *very* carefully (and we need to do that for 64-bit as
well with psci), as it's very likely there'll be subtle
incompatibilities between platforms, especially if firmware is involved.
We should try to leaves as little room for error as possible.

I don't want to see this devolve into meaning "whatever the Linux driver
for this happens to do at the current point in time", as that just leads
to breakage as we have no clear distinction between actual requirements
and implementation details.

Given we already have platforms without an enable-method, we can't make
it a required property either -- those platforms are already booting by
figuring out an enable method from the platform's compatible string.

With PSCI, enable-method also implies a method for disabling a
particular CPU, so it would be nice for the description to cover this.

>  
>  Example:
>  
> diff --git a/Documentation/devicetree/bindings/arm/msm/scss.txt b/Documentation/devicetree/bindings/arm/msm/scss.txt
> new file mode 100644
> index 0000000..21c3e26
> --- /dev/null
> +++ b/Documentation/devicetree/bindings/arm/msm/scss.txt
> @@ -0,0 +1,15 @@
> +* SCSS - Scorpion Sub-system
> +
> +Properties
> +
> +- compatible : Should contain "qcom,scss".
> +
> +- reg: Specifies the base address for the SCSS registers used for
> +       booting up secondary cores.
> +
> +Example:
> +
> +	scss@902000 {
> +		compatible = "qcom,scss";
> +		reg = <0x00902000 0x2000>;
> +	};
> diff --git a/arch/arm/boot/dts/msm8660-surf.dts b/arch/arm/boot/dts/msm8660-surf.dts
> index cdc010e..203e51a 100644
> --- a/arch/arm/boot/dts/msm8660-surf.dts
> +++ b/arch/arm/boot/dts/msm8660-surf.dts
> @@ -7,6 +7,22 @@
>  	compatible = "qcom,msm8660-surf", "qcom,msm8660";
>  	interrupt-parent = <&intc>;
>  
> +	cpus {
> +		#address-cells = <1>;
> +		#size-cells = <0>;
> +		compatible = "qcom,scorpion";
> +		device_type = "cpu";
> +		enable-method = "qcom,scss";
> +
> +		cpu@0 {
> +			reg = <0>;
> +		};
> +
> +		cpu@1 {
> +			reg = <1>;
> +		};
> +	};
> +

I *really* like moving the common properties out into the /cpus node --
ePAPR suggests it, it's less error prone, and it takes up less space.
However, I'm not sure our generic/arch code handles it properly, and I
think we need to audit that before we can start writing dts that depend
on it. I'd be happy to be wrong here if anyone can correct me. :)

We probably need to factor out the way we read properties for cpu nodes,
falling back to ones present in /cpus if not present. There's already a
lot of pain getting the node for a logical (rather than physical) cpu
id.

Sudeep recently factored out finding the cpu node for a logical cpu id
[1,2] in generic code with a per-arch callback, it shouldn't be too hard
to have shims around that to read (optionally) common properties.

[...]

> -static void prepare_cold_cpu(unsigned int cpu)
> +static int scorpion_release_secondary(void)
>  {
> -	int ret;
> -	ret = scm_set_boot_addr(virt_to_phys(secondary_startup),
> -				SCM_FLAG_COLDBOOT_CPU1);
> -	if (ret == 0) {
> -		void __iomem *sc1_base_ptr;
> -		sc1_base_ptr = ioremap_nocache(0x00902000, SZ_4K*2);
> -		if (sc1_base_ptr) {
> -			writel(0, sc1_base_ptr + VDD_SC1_ARRAY_CLAMP_GFS_CTL);
> -			writel(0, sc1_base_ptr + SCSS_CPU1CORE_RESET);
> -			writel(3, sc1_base_ptr + SCSS_DBG_STATUS_CORE_PWRDUP);
> -			iounmap(sc1_base_ptr);
> -		}
> -	} else
> -		printk(KERN_DEBUG "Failed to set secondary core boot "
> -				  "address\n");
> +	void __iomem *sc1_base_ptr;
> +	struct device_node *dn = NULL;
> +
> +	dn = of_find_compatible_node(dn, NULL, "qcom,scss");
> +	if (!dn) {
> +		pr_err("%s: Missing scss node in device tree\n", __func__);
> +		return -ENXIO;
> +	}
> +
> +	sc1_base_ptr = of_iomap(dn, 0);
> +	if (sc1_base_ptr) {
> +		writel_relaxed(0, sc1_base_ptr + VDD_SC1_ARRAY_CLAMP_GFS_CTL);
> +		writel_relaxed(0, sc1_base_ptr + SCSS_CPU1CORE_RESET);
> +		writel_relaxed(3, sc1_base_ptr + SCSS_DBG_STATUS_CORE_PWRDUP);
> +		mb();
> +		iounmap(sc1_base_ptr);

Does this boot *all* secondary CPUs directly into the kernel? Is that
safe (e.g. if the kernel supports fewer CPUs than are physically
present)?

Is this a one-time thing, or are we able to dynamically hotplug CPUs? If
the latter, the map/unmap looks odd to me.

> +	} else {
> +		return -ENOMEM;
> +	}
> +
> +	return 0;
>  }
>  
> -static int msm_boot_secondary(unsigned int cpu, struct task_struct *idle)
> +static DEFINE_PER_CPU(int, cold_boot_done);
> +
> +static void boot_cold_cpu(unsigned int cpu)
>  {
> -	static int cold_boot_done;
> +	const char *enable_method;
> +	struct device_node *dn = NULL;
>  
> -	/* Only need to bring cpu out of reset this way once */
> -	if (cold_boot_done == false) {
> -		prepare_cold_cpu(cpu);
> -		cold_boot_done = true;
> +	dn = of_find_node_by_name(dn, "cpus");
> +	if (!dn) {
> +		pr_err("%s: Missing node cpus in device tree\n", __func__);
> +		return;
>  	}
>  
> +	enable_method = of_get_property(dn, "enable-method", NULL);

Please factor this out from the platform code.

If we're going to use enable-method, it should be decoupled from
platform code -- common code should parse it to find the appropriate
handler. Also, we *must* support having an enable-method per-cpu as KVM
does for PSCI (though I definitely would like support for shared
properties as mentioned above).

> +	if (!enable_method) {
> +			pr_err("%s: cpus node is missing enable-method property\n",
> +					__func__);
> +	} else if (!strcmp(enable_method, "qcom,scss")) {
> +		if (per_cpu(cold_boot_done, cpu) == false) {
> +			scorpion_release_secondary();
> +			per_cpu(cold_boot_done, cpu) = true;
> +		}
> +	} else {
> +		pr_err("%s: Invalid enable-method property: %s\n",
> +				__func__, enable_method);
> +	}
> +}
> +
> +static int msm_boot_secondary(unsigned int cpu, struct task_struct *idle)
> +{
> +	boot_cold_cpu(cpu);
> +
>  	/*
>  	 * set synchronisation state between this boot processor
>  	 * and the secondary one
> @@ -118,8 +148,28 @@ static void __init msm_smp_init_cpus(void)
>  		set_cpu_possible(i, true);
>  }
>  
> +static const int cold_boot_flags[] __initconst = {
> +	0,
> +	SCM_FLAG_COLDBOOT_CPU1,
> +};

So we only ever support booting two CPUs?

Is the mapping of MPIDR to register bits arbitrary? Or do we know what
they would be for four CPUs, four clusters, and beyond?

> +
>  static void __init msm_smp_prepare_cpus(unsigned int max_cpus)
>  {
> +	int cpu, map;
> +	unsigned int flags = 0;
> +
> +	for_each_present_cpu(cpu) {
> +		map = cpu_logical_map(cpu);
> +		if (map > ARRAY_SIZE(cold_boot_flags)) {
> +			set_cpu_present(cpu, false);
> +			__WARN();
> +			continue;
> +		}
> +		flags |= cold_boot_flags[map];

It's a shame that this leaves a window where some CPUs seem bootable,
but aren't (though I can't offer a better way of handling this given we
have dts without enable-method properties).

> +	}
> +
> +	if (scm_set_boot_addr(virt_to_phys(secondary_startup), flags))
> +		pr_warn("Failed to set CPU boot address\n");
>  }
>  
>  struct smp_operations msm_smp_ops __initdata = {
> -- 
> The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
> hosted by The Linux Foundation
> 

Thanks,
Mark

[1] http://lists.infradead.org/pipermail/linux-arm-kernel/2013-July/184150.html
[2] http://lists.infradead.org/pipermail/linux-arm-kernel/2013-August/189619.html

[RESEND PATCH 2/4] ARM: msm: Re-organize platsmp to make it extensible

[Mark Rutland]

Hi,

Apologies for the long delay for review on this.

I really like the direction this is going, but I have some qualms with
the implementation.

On Fri, Aug 02, 2013 at 03:15:23AM +0100, Rohit Vaswani wrote:
> This makes it easy to add SMP support for new targets
> by adding cpus property and the release sequence.
> We add the enable-method property for the cpus property to
> specify which release sequence to use.
> While at it, add the 8660 cpus bindings to make SMP work.
> 
> Signed-off-by: Rohit Vaswani <rvaswani@codeaurora.org>
> ---
>  Documentation/devicetree/bindings/arm/cpus.txt     |  6 ++
>  Documentation/devicetree/bindings/arm/msm/scss.txt | 15 ++++
>  arch/arm/boot/dts/msm8660-surf.dts                 | 23 +++++-
>  arch/arm/mach-msm/platsmp.c                        | 94 +++++++++++++++++-----
>  4 files changed, 115 insertions(+), 23 deletions(-)
>  create mode 100644 Documentation/devicetree/bindings/arm/msm/scss.txt
> 
> diff --git a/Documentation/devicetree/bindings/arm/cpus.txt b/Documentation/devicetree/bindings/arm/cpus.txt
> index f32494d..327aad2 100644
> --- a/Documentation/devicetree/bindings/arm/cpus.txt
> +++ b/Documentation/devicetree/bindings/arm/cpus.txt
> @@ -44,6 +44,12 @@ For the ARM architecture every CPU node must contain the following properties:
>  		"marvell,mohawk"
>  		"marvell,xsc3"
>  		"marvell,xscale"
> +		"qcom,scorpion"
> +- enable-method: Specifies the method used to enable or take the secondary cores
> +		 out of reset. This allows different reset sequence for
> +		 different types of cpus.
> +		 This should be one of:
> +		 "qcom,scss"

While I'd certainly like to see a move to using enable-method for
32-bit, I think this needs a bit more thought:

What does "qcom,scss" mean, precisely? It seems to be that we poke some
registers in a "qcom,scss" device. I think we need to document
enable-methods *very* carefully (and we need to do that for 64-bit as
well with psci), as it's very likely there'll be subtle
incompatibilities between platforms, especially if firmware is involved.
We should try to leaves as little room for error as possible.

I don't want to see this devolve into meaning "whatever the Linux driver
for this happens to do at the current point in time", as that just leads
to breakage as we have no clear distinction between actual requirements
and implementation details.

Given we already have platforms without an enable-method, we can't make
it a required property either -- those platforms are already booting by
figuring out an enable method from the platform's compatible string.

With PSCI, enable-method also implies a method for disabling a
particular CPU, so it would be nice for the description to cover this.

>  
>  Example:
>  
> diff --git a/Documentation/devicetree/bindings/arm/msm/scss.txt b/Documentation/devicetree/bindings/arm/msm/scss.txt
> new file mode 100644
> index 0000000..21c3e26
> --- /dev/null
> +++ b/Documentation/devicetree/bindings/arm/msm/scss.txt
> @@ -0,0 +1,15 @@
> +* SCSS - Scorpion Sub-system
> +
> +Properties
> +
> +- compatible : Should contain "qcom,scss".
> +
> +- reg: Specifies the base address for the SCSS registers used for
> +       booting up secondary cores.
> +
> +Example:
> +
> +	scss at 902000 {
> +		compatible = "qcom,scss";
> +		reg = <0x00902000 0x2000>;
> +	};
> diff --git a/arch/arm/boot/dts/msm8660-surf.dts b/arch/arm/boot/dts/msm8660-surf.dts
> index cdc010e..203e51a 100644
> --- a/arch/arm/boot/dts/msm8660-surf.dts
> +++ b/arch/arm/boot/dts/msm8660-surf.dts
> @@ -7,6 +7,22 @@
>  	compatible = "qcom,msm8660-surf", "qcom,msm8660";
>  	interrupt-parent = <&intc>;
>  
> +	cpus {
> +		#address-cells = <1>;
> +		#size-cells = <0>;
> +		compatible = "qcom,scorpion";
> +		device_type = "cpu";
> +		enable-method = "qcom,scss";
> +
> +		cpu at 0 {
> +			reg = <0>;
> +		};
> +
> +		cpu at 1 {
> +			reg = <1>;
> +		};
> +	};
> +

I *really* like moving the common properties out into the /cpus node --
ePAPR suggests it, it's less error prone, and it takes up less space.
However, I'm not sure our generic/arch code handles it properly, and I
think we need to audit that before we can start writing dts that depend
on it. I'd be happy to be wrong here if anyone can correct me. :)

We probably need to factor out the way we read properties for cpu nodes,
falling back to ones present in /cpus if not present. There's already a
lot of pain getting the node for a logical (rather than physical) cpu
id.

Sudeep recently factored out finding the cpu node for a logical cpu id
[1,2] in generic code with a per-arch callback, it shouldn't be too hard
to have shims around that to read (optionally) common properties.

[...]

> -static void prepare_cold_cpu(unsigned int cpu)
> +static int scorpion_release_secondary(void)
>  {
> -	int ret;
> -	ret = scm_set_boot_addr(virt_to_phys(secondary_startup),
> -				SCM_FLAG_COLDBOOT_CPU1);
> -	if (ret == 0) {
> -		void __iomem *sc1_base_ptr;
> -		sc1_base_ptr = ioremap_nocache(0x00902000, SZ_4K*2);
> -		if (sc1_base_ptr) {
> -			writel(0, sc1_base_ptr + VDD_SC1_ARRAY_CLAMP_GFS_CTL);
> -			writel(0, sc1_base_ptr + SCSS_CPU1CORE_RESET);
> -			writel(3, sc1_base_ptr + SCSS_DBG_STATUS_CORE_PWRDUP);
> -			iounmap(sc1_base_ptr);
> -		}
> -	} else
> -		printk(KERN_DEBUG "Failed to set secondary core boot "
> -				  "address\n");
> +	void __iomem *sc1_base_ptr;
> +	struct device_node *dn = NULL;
> +
> +	dn = of_find_compatible_node(dn, NULL, "qcom,scss");
> +	if (!dn) {
> +		pr_err("%s: Missing scss node in device tree\n", __func__);
> +		return -ENXIO;
> +	}
> +
> +	sc1_base_ptr = of_iomap(dn, 0);
> +	if (sc1_base_ptr) {
> +		writel_relaxed(0, sc1_base_ptr + VDD_SC1_ARRAY_CLAMP_GFS_CTL);
> +		writel_relaxed(0, sc1_base_ptr + SCSS_CPU1CORE_RESET);
> +		writel_relaxed(3, sc1_base_ptr + SCSS_DBG_STATUS_CORE_PWRDUP);
> +		mb();
> +		iounmap(sc1_base_ptr);

Does this boot *all* secondary CPUs directly into the kernel? Is that
safe (e.g. if the kernel supports fewer CPUs than are physically
present)?

Is this a one-time thing, or are we able to dynamically hotplug CPUs? If
the latter, the map/unmap looks odd to me.

> +	} else {
> +		return -ENOMEM;
> +	}
> +
> +	return 0;
>  }
>  
> -static int msm_boot_secondary(unsigned int cpu, struct task_struct *idle)
> +static DEFINE_PER_CPU(int, cold_boot_done);
> +
> +static void boot_cold_cpu(unsigned int cpu)
>  {
> -	static int cold_boot_done;
> +	const char *enable_method;
> +	struct device_node *dn = NULL;
>  
> -	/* Only need to bring cpu out of reset this way once */
> -	if (cold_boot_done == false) {
> -		prepare_cold_cpu(cpu);
> -		cold_boot_done = true;
> +	dn = of_find_node_by_name(dn, "cpus");
> +	if (!dn) {
> +		pr_err("%s: Missing node cpus in device tree\n", __func__);
> +		return;
>  	}
>  
> +	enable_method = of_get_property(dn, "enable-method", NULL);

Please factor this out from the platform code.

If we're going to use enable-method, it should be decoupled from
platform code -- common code should parse it to find the appropriate
handler. Also, we *must* support having an enable-method per-cpu as KVM
does for PSCI (though I definitely would like support for shared
properties as mentioned above).

> +	if (!enable_method) {
> +			pr_err("%s: cpus node is missing enable-method property\n",
> +					__func__);
> +	} else if (!strcmp(enable_method, "qcom,scss")) {
> +		if (per_cpu(cold_boot_done, cpu) == false) {
> +			scorpion_release_secondary();
> +			per_cpu(cold_boot_done, cpu) = true;
> +		}
> +	} else {
> +		pr_err("%s: Invalid enable-method property: %s\n",
> +				__func__, enable_method);
> +	}
> +}
> +
> +static int msm_boot_secondary(unsigned int cpu, struct task_struct *idle)
> +{
> +	boot_cold_cpu(cpu);
> +
>  	/*
>  	 * set synchronisation state between this boot processor
>  	 * and the secondary one
> @@ -118,8 +148,28 @@ static void __init msm_smp_init_cpus(void)
>  		set_cpu_possible(i, true);
>  }
>  
> +static const int cold_boot_flags[] __initconst = {
> +	0,
> +	SCM_FLAG_COLDBOOT_CPU1,
> +};

So we only ever support booting two CPUs?

Is the mapping of MPIDR to register bits arbitrary? Or do we know what
they would be for four CPUs, four clusters, and beyond?

> +
>  static void __init msm_smp_prepare_cpus(unsigned int max_cpus)
>  {
> +	int cpu, map;
> +	unsigned int flags = 0;
> +
> +	for_each_present_cpu(cpu) {
> +		map = cpu_logical_map(cpu);
> +		if (map > ARRAY_SIZE(cold_boot_flags)) {
> +			set_cpu_present(cpu, false);
> +			__WARN();
> +			continue;
> +		}
> +		flags |= cold_boot_flags[map];

It's a shame that this leaves a window where some CPUs seem bootable,
but aren't (though I can't offer a better way of handling this given we
have dts without enable-method properties).

> +	}
> +
> +	if (scm_set_boot_addr(virt_to_phys(secondary_startup), flags))
> +		pr_warn("Failed to set CPU boot address\n");
>  }
>  
>  struct smp_operations msm_smp_ops __initdata = {
> -- 
> The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
> hosted by The Linux Foundation
> 

Thanks,
Mark

[1] http://lists.infradead.org/pipermail/linux-arm-kernel/2013-July/184150.html
[2] http://lists.infradead.org/pipermail/linux-arm-kernel/2013-August/189619.html

Re: [PATCH 3/4] ARM: msm: Add SMP support for 8960

[Mark Rutland]

On Fri, Aug 02, 2013 at 03:15:24AM +0100, Rohit Vaswani wrote:
> Add the cpus bindings and the Krait release sequence
> to make SMP work for MSM8960
> 
> Signed-off-by: Rohit Vaswani <rvaswani@codeaurora.org>
> ---
>  Documentation/devicetree/bindings/arm/cpus.txt     |  2 +
>  Documentation/devicetree/bindings/arm/msm/kpss.txt | 16 ++++++
>  arch/arm/boot/dts/msm8960-cdp.dts                  | 22 +++++++++
>  arch/arm/mach-msm/platsmp.c                        | 57 ++++++++++++++++++++++
>  arch/arm/mach-msm/scm-boot.h                       |  8 +--
>  5 files changed, 102 insertions(+), 3 deletions(-)
>  create mode 100644 Documentation/devicetree/bindings/arm/msm/kpss.txt
> 
> diff --git a/Documentation/devicetree/bindings/arm/cpus.txt b/Documentation/devicetree/bindings/arm/cpus.txt
> index 327aad2..1132eac 100644
> --- a/Documentation/devicetree/bindings/arm/cpus.txt
> +++ b/Documentation/devicetree/bindings/arm/cpus.txt
> @@ -45,11 +45,13 @@ For the ARM architecture every CPU node must contain the following properties:
>  		"marvell,xsc3"
>  		"marvell,xscale"
>  		"qcom,scorpion"
> +		"qcom,krait"
>  - enable-method: Specifies the method used to enable or take the secondary cores
>  		 out of reset. This allows different reset sequence for
>  		 different types of cpus.
>  		 This should be one of:
>  		 "qcom,scss"
> +		 "qcom,kpssv1"

Hopefully (though this series implies otherwise) we won't have an
explosion of enable-methods. We haven't listed any common ones yet (e.g.
PSCI), and both this and qcom,scss are "poke some cpu-specific
registers".

I take it by the "v1" suffix you're expecting more variation here?

>  
>  Example:
>  
> diff --git a/Documentation/devicetree/bindings/arm/msm/kpss.txt b/Documentation/devicetree/bindings/arm/msm/kpss.txt
> new file mode 100644
> index 0000000..7272340
> --- /dev/null
> +++ b/Documentation/devicetree/bindings/arm/msm/kpss.txt
> @@ -0,0 +1,16 @@
> +* KPSS - Krait Processor Sub-system
> +
> +Properties
> +
> +- compatible : Should contain "qcom,kpss".
> +
> +- reg: Specifies the base address for the KPSS registers used for
> +       booting up secondary cores.
> +
> +Example:
> +
> +	kpss@2088000 {
> +		compatible = "qcom,kpss";
> +		reg = <0x02088000 0x1000
> +			0x02098000 0x2000>;
> +	};

What's the secondary bank of registers? The binding only mentions one...

Is this a register bank per-cpu? There's no linkage to CPU ID, which
means that handling logical mapping is going to get quite painful.

For the vaguely standard "spin-table" enable-method, the address to poke
(cpu-release-addr) may be stored inside a specific cpu node. Following
that style may make more sense here, unless the kpss hardware is used
for anything more than processor hotplug.

We could have the cpu node refer to the specific kpss/register combo,
which would also allow for future expansion if the kpss unit is
per-cluster:

/ {
	cpus {
		device_type = "cpu";
		compatible = "qcom,krait";
		enable-method = "qcom,kpssv1";

		cpu@0 {
			reg = <0>;
			qcom,kpss-reg = <&kpss 1>; /* reg[1] in kpss */
		};

		cpu@1 {
			reg = <1>;
			qcom,kpss-reg = <&kpss 0>; /* reg[0] in kpss */
		};
	}

	kpss: kpss@2088000 {
		compatible = "qcom,kpss";
		reg = <0x02088000 0x1000>,
		      <0x02098000 0x2000>;
	};
}

> diff --git a/arch/arm/boot/dts/msm8960-cdp.dts b/arch/arm/boot/dts/msm8960-cdp.dts
> index db2060c..8c82d5e 100644
> --- a/arch/arm/boot/dts/msm8960-cdp.dts
> +++ b/arch/arm/boot/dts/msm8960-cdp.dts
> @@ -7,6 +7,22 @@
>  	compatible = "qcom,msm8960-cdp", "qcom,msm8960";
>  	interrupt-parent = <&intc>;
>  
> +	cpus {
> +		#address-cells = <1>;
> +		#size-cells = <0>;
> +		compatible = "qcom,krait";
> +		device_type = "cpu";
> +		enable-method = "qcom,kpssv1";
> +
> +		cpu@0 {
> +			reg = <0>;
> +		};
> +
> +		cpu@1 {
> +			reg = <1>;
> +		};
> +	};

Similarly to my comments on the first patch, I like making properties
shared, but we *need* to have common infrastructure before we can do
things this way.

> +
>  	intc: interrupt-controller@2000000 {
>  		compatible = "qcom,msm-qgic2";
>  		interrupt-controller;
> @@ -37,6 +53,12 @@
>  		reg = <0xfd510000 0x4000>;
>  	};
>  
> +	kpss@2088000 {
> +		compatible = "qcom,kpss";
> +		reg = <0x02088000 0x1000
> +			0x02098000 0x2000>;
> +	};
> +
>  	serial@16440000 {
>  		compatible = "qcom,msm-hsuart", "qcom,msm-uart";
>  		reg = <0x16440000 0x1000>,
> diff --git a/arch/arm/mach-msm/platsmp.c b/arch/arm/mach-msm/platsmp.c
> index 17022e0..82eb079 100644
> --- a/arch/arm/mach-msm/platsmp.c
> +++ b/arch/arm/mach-msm/platsmp.c
> @@ -74,6 +74,56 @@ static int scorpion_release_secondary(void)
>  	return 0;
>  }
>  
> +static int msm8960_release_secondary(unsigned int cpu)
> +{
> +	void __iomem *reg;
> +	struct device_node *dn = NULL;
> +
> +	if (cpu == 0 || cpu >= num_possible_cpus())
> +		return -EINVAL;

We seem to describe a reg bank for CPU0. Is this check because we don't
(yet) have a way of hotplugging CPU0 off? Is CPU0 special in that
regard?

> +
> +	dn = of_find_compatible_node(dn, NULL, "qcom,kpss");
> +	if (!dn) {
> +		pr_err("%s : Missing kpss node from device tree\n", __func__);
> +		return -ENXIO;
> +	}
> +
> +	reg = of_iomap(dn, cpu);

That doesn't handle logical id mapping.

> +	if (!reg)
> +		return -ENOMEM;
> +
> +	pr_debug("Starting secondary CPU %d\n", cpu);
> +
> +	/* Turn on CPU Rail */
> +	writel_relaxed(0xA4, reg+0x1014);

Symbolic names for registers please.

[...]

> @@ -151,6 +206,8 @@ static void __init msm_smp_init_cpus(void)
>  static const int cold_boot_flags[] __initconst = {
>  	0,
>  	SCM_FLAG_COLDBOOT_CPU1,
> +	SCM_FLAG_COLDBOOT_CPU2,
> +	SCM_FLAG_COLDBOOT_CPU3,
>  };
>  
>  static void __init msm_smp_prepare_cpus(unsigned int max_cpus)
> diff --git a/arch/arm/mach-msm/scm-boot.h b/arch/arm/mach-msm/scm-boot.h
> index 7be32ff..6aabb24 100644
> --- a/arch/arm/mach-msm/scm-boot.h
> +++ b/arch/arm/mach-msm/scm-boot.h
> @@ -13,9 +13,11 @@
>  #define __MACH_SCM_BOOT_H
>  
>  #define SCM_BOOT_ADDR			0x1
> -#define SCM_FLAG_COLDBOOT_CPU1		0x1
> -#define SCM_FLAG_WARMBOOT_CPU1		0x2
> -#define SCM_FLAG_WARMBOOT_CPU0		0x4
> +#define SCM_FLAG_COLDBOOT_CPU1		0x01
> +#define SCM_FLAG_COLDBOOT_CPU2		0x08
> +#define SCM_FLAG_COLDBOOT_CPU3		0x20
> +#define SCM_FLAG_WARMBOOT_CPU0		0x04
> +#define SCM_FLAG_WARMBOOT_CPU1		0x02

Is there any obvious sequencing for these values?

How will they be extended in future for more CPUs/clusters?

Do we possibly need this information in DT?

Thanks,
Mark.

Re: [PATCH 3/4] ARM: msm: Add SMP support for 8960

[Mark Rutland]

On Fri, Aug 02, 2013 at 03:15:24AM +0100, Rohit Vaswani wrote:
> Add the cpus bindings and the Krait release sequence
> to make SMP work for MSM8960
> 
> Signed-off-by: Rohit Vaswani <rvaswani@codeaurora.org>
> ---
>  Documentation/devicetree/bindings/arm/cpus.txt     |  2 +
>  Documentation/devicetree/bindings/arm/msm/kpss.txt | 16 ++++++
>  arch/arm/boot/dts/msm8960-cdp.dts                  | 22 +++++++++
>  arch/arm/mach-msm/platsmp.c                        | 57 ++++++++++++++++++++++
>  arch/arm/mach-msm/scm-boot.h                       |  8 +--
>  5 files changed, 102 insertions(+), 3 deletions(-)
>  create mode 100644 Documentation/devicetree/bindings/arm/msm/kpss.txt
> 
> diff --git a/Documentation/devicetree/bindings/arm/cpus.txt b/Documentation/devicetree/bindings/arm/cpus.txt
> index 327aad2..1132eac 100644
> --- a/Documentation/devicetree/bindings/arm/cpus.txt
> +++ b/Documentation/devicetree/bindings/arm/cpus.txt
> @@ -45,11 +45,13 @@ For the ARM architecture every CPU node must contain the following properties:
>  		"marvell,xsc3"
>  		"marvell,xscale"
>  		"qcom,scorpion"
> +		"qcom,krait"
>  - enable-method: Specifies the method used to enable or take the secondary cores
>  		 out of reset. This allows different reset sequence for
>  		 different types of cpus.
>  		 This should be one of:
>  		 "qcom,scss"
> +		 "qcom,kpssv1"

Hopefully (though this series implies otherwise) we won't have an
explosion of enable-methods. We haven't listed any common ones yet (e.g.
PSCI), and both this and qcom,scss are "poke some cpu-specific
registers".

I take it by the "v1" suffix you're expecting more variation here?

>  
>  Example:
>  
> diff --git a/Documentation/devicetree/bindings/arm/msm/kpss.txt b/Documentation/devicetree/bindings/arm/msm/kpss.txt
> new file mode 100644
> index 0000000..7272340
> --- /dev/null
> +++ b/Documentation/devicetree/bindings/arm/msm/kpss.txt
> @@ -0,0 +1,16 @@
> +* KPSS - Krait Processor Sub-system
> +
> +Properties
> +
> +- compatible : Should contain "qcom,kpss".
> +
> +- reg: Specifies the base address for the KPSS registers used for
> +       booting up secondary cores.
> +
> +Example:
> +
> +	kpss@2088000 {
> +		compatible = "qcom,kpss";
> +		reg = <0x02088000 0x1000
> +			0x02098000 0x2000>;
> +	};

What's the secondary bank of registers? The binding only mentions one...

Is this a register bank per-cpu? There's no linkage to CPU ID, which
means that handling logical mapping is going to get quite painful.

For the vaguely standard "spin-table" enable-method, the address to poke
(cpu-release-addr) may be stored inside a specific cpu node. Following
that style may make more sense here, unless the kpss hardware is used
for anything more than processor hotplug.

We could have the cpu node refer to the specific kpss/register combo,
which would also allow for future expansion if the kpss unit is
per-cluster:

/ {
	cpus {
		device_type = "cpu";
		compatible = "qcom,krait";
		enable-method = "qcom,kpssv1";

		cpu@0 {
			reg = <0>;
			qcom,kpss-reg = <&kpss 1>; /* reg[1] in kpss */
		};

		cpu@1 {
			reg = <1>;
			qcom,kpss-reg = <&kpss 0>; /* reg[0] in kpss */
		};
	}

	kpss: kpss@2088000 {
		compatible = "qcom,kpss";
		reg = <0x02088000 0x1000>,
		      <0x02098000 0x2000>;
	};
}

> diff --git a/arch/arm/boot/dts/msm8960-cdp.dts b/arch/arm/boot/dts/msm8960-cdp.dts
> index db2060c..8c82d5e 100644
> --- a/arch/arm/boot/dts/msm8960-cdp.dts
> +++ b/arch/arm/boot/dts/msm8960-cdp.dts
> @@ -7,6 +7,22 @@
>  	compatible = "qcom,msm8960-cdp", "qcom,msm8960";
>  	interrupt-parent = <&intc>;
>  
> +	cpus {
> +		#address-cells = <1>;
> +		#size-cells = <0>;
> +		compatible = "qcom,krait";
> +		device_type = "cpu";
> +		enable-method = "qcom,kpssv1";
> +
> +		cpu@0 {
> +			reg = <0>;
> +		};
> +
> +		cpu@1 {
> +			reg = <1>;
> +		};
> +	};

Similarly to my comments on the first patch, I like making properties
shared, but we *need* to have common infrastructure before we can do
things this way.

> +
>  	intc: interrupt-controller@2000000 {
>  		compatible = "qcom,msm-qgic2";
>  		interrupt-controller;
> @@ -37,6 +53,12 @@
>  		reg = <0xfd510000 0x4000>;
>  	};
>  
> +	kpss@2088000 {
> +		compatible = "qcom,kpss";
> +		reg = <0x02088000 0x1000
> +			0x02098000 0x2000>;
> +	};
> +
>  	serial@16440000 {
>  		compatible = "qcom,msm-hsuart", "qcom,msm-uart";
>  		reg = <0x16440000 0x1000>,
> diff --git a/arch/arm/mach-msm/platsmp.c b/arch/arm/mach-msm/platsmp.c
> index 17022e0..82eb079 100644
> --- a/arch/arm/mach-msm/platsmp.c
> +++ b/arch/arm/mach-msm/platsmp.c
> @@ -74,6 +74,56 @@ static int scorpion_release_secondary(void)
>  	return 0;
>  }
>  
> +static int msm8960_release_secondary(unsigned int cpu)
> +{
> +	void __iomem *reg;
> +	struct device_node *dn = NULL;
> +
> +	if (cpu == 0 || cpu >= num_possible_cpus())
> +		return -EINVAL;

We seem to describe a reg bank for CPU0. Is this check because we don't
(yet) have a way of hotplugging CPU0 off? Is CPU0 special in that
regard?

> +
> +	dn = of_find_compatible_node(dn, NULL, "qcom,kpss");
> +	if (!dn) {
> +		pr_err("%s : Missing kpss node from device tree\n", __func__);
> +		return -ENXIO;
> +	}
> +
> +	reg = of_iomap(dn, cpu);

That doesn't handle logical id mapping.

> +	if (!reg)
> +		return -ENOMEM;
> +
> +	pr_debug("Starting secondary CPU %d\n", cpu);
> +
> +	/* Turn on CPU Rail */
> +	writel_relaxed(0xA4, reg+0x1014);

Symbolic names for registers please.

[...]

> @@ -151,6 +206,8 @@ static void __init msm_smp_init_cpus(void)
>  static const int cold_boot_flags[] __initconst = {
>  	0,
>  	SCM_FLAG_COLDBOOT_CPU1,
> +	SCM_FLAG_COLDBOOT_CPU2,
> +	SCM_FLAG_COLDBOOT_CPU3,
>  };
>  
>  static void __init msm_smp_prepare_cpus(unsigned int max_cpus)
> diff --git a/arch/arm/mach-msm/scm-boot.h b/arch/arm/mach-msm/scm-boot.h
> index 7be32ff..6aabb24 100644
> --- a/arch/arm/mach-msm/scm-boot.h
> +++ b/arch/arm/mach-msm/scm-boot.h
> @@ -13,9 +13,11 @@
>  #define __MACH_SCM_BOOT_H
>  
>  #define SCM_BOOT_ADDR			0x1
> -#define SCM_FLAG_COLDBOOT_CPU1		0x1
> -#define SCM_FLAG_WARMBOOT_CPU1		0x2
> -#define SCM_FLAG_WARMBOOT_CPU0		0x4
> +#define SCM_FLAG_COLDBOOT_CPU1		0x01
> +#define SCM_FLAG_COLDBOOT_CPU2		0x08
> +#define SCM_FLAG_COLDBOOT_CPU3		0x20
> +#define SCM_FLAG_WARMBOOT_CPU0		0x04
> +#define SCM_FLAG_WARMBOOT_CPU1		0x02

Is there any obvious sequencing for these values?

How will they be extended in future for more CPUs/clusters?

Do we possibly need this information in DT?

Thanks,
Mark.

[PATCH 3/4] ARM: msm: Add SMP support for 8960

[Mark Rutland]

On Fri, Aug 02, 2013 at 03:15:24AM +0100, Rohit Vaswani wrote:
> Add the cpus bindings and the Krait release sequence
> to make SMP work for MSM8960
> 
> Signed-off-by: Rohit Vaswani <rvaswani@codeaurora.org>
> ---
>  Documentation/devicetree/bindings/arm/cpus.txt     |  2 +
>  Documentation/devicetree/bindings/arm/msm/kpss.txt | 16 ++++++
>  arch/arm/boot/dts/msm8960-cdp.dts                  | 22 +++++++++
>  arch/arm/mach-msm/platsmp.c                        | 57 ++++++++++++++++++++++
>  arch/arm/mach-msm/scm-boot.h                       |  8 +--
>  5 files changed, 102 insertions(+), 3 deletions(-)
>  create mode 100644 Documentation/devicetree/bindings/arm/msm/kpss.txt
> 
> diff --git a/Documentation/devicetree/bindings/arm/cpus.txt b/Documentation/devicetree/bindings/arm/cpus.txt
> index 327aad2..1132eac 100644
> --- a/Documentation/devicetree/bindings/arm/cpus.txt
> +++ b/Documentation/devicetree/bindings/arm/cpus.txt
> @@ -45,11 +45,13 @@ For the ARM architecture every CPU node must contain the following properties:
>  		"marvell,xsc3"
>  		"marvell,xscale"
>  		"qcom,scorpion"
> +		"qcom,krait"
>  - enable-method: Specifies the method used to enable or take the secondary cores
>  		 out of reset. This allows different reset sequence for
>  		 different types of cpus.
>  		 This should be one of:
>  		 "qcom,scss"
> +		 "qcom,kpssv1"

Hopefully (though this series implies otherwise) we won't have an
explosion of enable-methods. We haven't listed any common ones yet (e.g.
PSCI), and both this and qcom,scss are "poke some cpu-specific
registers".

I take it by the "v1" suffix you're expecting more variation here?

>  
>  Example:
>  
> diff --git a/Documentation/devicetree/bindings/arm/msm/kpss.txt b/Documentation/devicetree/bindings/arm/msm/kpss.txt
> new file mode 100644
> index 0000000..7272340
> --- /dev/null
> +++ b/Documentation/devicetree/bindings/arm/msm/kpss.txt
> @@ -0,0 +1,16 @@
> +* KPSS - Krait Processor Sub-system
> +
> +Properties
> +
> +- compatible : Should contain "qcom,kpss".
> +
> +- reg: Specifies the base address for the KPSS registers used for
> +       booting up secondary cores.
> +
> +Example:
> +
> +	kpss at 2088000 {
> +		compatible = "qcom,kpss";
> +		reg = <0x02088000 0x1000
> +			0x02098000 0x2000>;
> +	};

What's the secondary bank of registers? The binding only mentions one...

Is this a register bank per-cpu? There's no linkage to CPU ID, which
means that handling logical mapping is going to get quite painful.

For the vaguely standard "spin-table" enable-method, the address to poke
(cpu-release-addr) may be stored inside a specific cpu node. Following
that style may make more sense here, unless the kpss hardware is used
for anything more than processor hotplug.

We could have the cpu node refer to the specific kpss/register combo,
which would also allow for future expansion if the kpss unit is
per-cluster:

/ {
	cpus {
		device_type = "cpu";
		compatible = "qcom,krait";
		enable-method = "qcom,kpssv1";

		cpu at 0 {
			reg = <0>;
			qcom,kpss-reg = <&kpss 1>; /* reg[1] in kpss */
		};

		cpu at 1 {
			reg = <1>;
			qcom,kpss-reg = <&kpss 0>; /* reg[0] in kpss */
		};
	}

	kpss: kpss at 2088000 {
		compatible = "qcom,kpss";
		reg = <0x02088000 0x1000>,
		      <0x02098000 0x2000>;
	};
}

> diff --git a/arch/arm/boot/dts/msm8960-cdp.dts b/arch/arm/boot/dts/msm8960-cdp.dts
> index db2060c..8c82d5e 100644
> --- a/arch/arm/boot/dts/msm8960-cdp.dts
> +++ b/arch/arm/boot/dts/msm8960-cdp.dts
> @@ -7,6 +7,22 @@
>  	compatible = "qcom,msm8960-cdp", "qcom,msm8960";
>  	interrupt-parent = <&intc>;
>  
> +	cpus {
> +		#address-cells = <1>;
> +		#size-cells = <0>;
> +		compatible = "qcom,krait";
> +		device_type = "cpu";
> +		enable-method = "qcom,kpssv1";
> +
> +		cpu at 0 {
> +			reg = <0>;
> +		};
> +
> +		cpu at 1 {
> +			reg = <1>;
> +		};
> +	};

Similarly to my comments on the first patch, I like making properties
shared, but we *need* to have common infrastructure before we can do
things this way.

> +
>  	intc: interrupt-controller at 2000000 {
>  		compatible = "qcom,msm-qgic2";
>  		interrupt-controller;
> @@ -37,6 +53,12 @@
>  		reg = <0xfd510000 0x4000>;
>  	};
>  
> +	kpss at 2088000 {
> +		compatible = "qcom,kpss";
> +		reg = <0x02088000 0x1000
> +			0x02098000 0x2000>;
> +	};
> +
>  	serial at 16440000 {
>  		compatible = "qcom,msm-hsuart", "qcom,msm-uart";
>  		reg = <0x16440000 0x1000>,
> diff --git a/arch/arm/mach-msm/platsmp.c b/arch/arm/mach-msm/platsmp.c
> index 17022e0..82eb079 100644
> --- a/arch/arm/mach-msm/platsmp.c
> +++ b/arch/arm/mach-msm/platsmp.c
> @@ -74,6 +74,56 @@ static int scorpion_release_secondary(void)
>  	return 0;
>  }
>  
> +static int msm8960_release_secondary(unsigned int cpu)
> +{
> +	void __iomem *reg;
> +	struct device_node *dn = NULL;
> +
> +	if (cpu == 0 || cpu >= num_possible_cpus())
> +		return -EINVAL;

We seem to describe a reg bank for CPU0. Is this check because we don't
(yet) have a way of hotplugging CPU0 off? Is CPU0 special in that
regard?

> +
> +	dn = of_find_compatible_node(dn, NULL, "qcom,kpss");
> +	if (!dn) {
> +		pr_err("%s : Missing kpss node from device tree\n", __func__);
> +		return -ENXIO;
> +	}
> +
> +	reg = of_iomap(dn, cpu);

That doesn't handle logical id mapping.

> +	if (!reg)
> +		return -ENOMEM;
> +
> +	pr_debug("Starting secondary CPU %d\n", cpu);
> +
> +	/* Turn on CPU Rail */
> +	writel_relaxed(0xA4, reg+0x1014);

Symbolic names for registers please.

[...]

> @@ -151,6 +206,8 @@ static void __init msm_smp_init_cpus(void)
>  static const int cold_boot_flags[] __initconst = {
>  	0,
>  	SCM_FLAG_COLDBOOT_CPU1,
> +	SCM_FLAG_COLDBOOT_CPU2,
> +	SCM_FLAG_COLDBOOT_CPU3,
>  };
>  
>  static void __init msm_smp_prepare_cpus(unsigned int max_cpus)
> diff --git a/arch/arm/mach-msm/scm-boot.h b/arch/arm/mach-msm/scm-boot.h
> index 7be32ff..6aabb24 100644
> --- a/arch/arm/mach-msm/scm-boot.h
> +++ b/arch/arm/mach-msm/scm-boot.h
> @@ -13,9 +13,11 @@
>  #define __MACH_SCM_BOOT_H
>  
>  #define SCM_BOOT_ADDR			0x1
> -#define SCM_FLAG_COLDBOOT_CPU1		0x1
> -#define SCM_FLAG_WARMBOOT_CPU1		0x2
> -#define SCM_FLAG_WARMBOOT_CPU0		0x4
> +#define SCM_FLAG_COLDBOOT_CPU1		0x01
> +#define SCM_FLAG_COLDBOOT_CPU2		0x08
> +#define SCM_FLAG_COLDBOOT_CPU3		0x20
> +#define SCM_FLAG_WARMBOOT_CPU0		0x04
> +#define SCM_FLAG_WARMBOOT_CPU1		0x02

Is there any obvious sequencing for these values?

How will they be extended in future for more CPUs/clusters?

Do we possibly need this information in DT?

Thanks,
Mark.

Re: [RESEND PATCH 4/4] ARM: msm: Add support for 8974 SMP

[Mark Rutland]

On Fri, Aug 02, 2013 at 03:15:25AM +0100, Rohit Vaswani wrote:
> Add the cpus bindings and the Kraitv2 release sequence
> to make SMP work for 2 cores on MSM8974.
> 
> Signed-off-by: Rohit Vaswani <rvaswani@codeaurora.org>
> ---
>  Documentation/devicetree/bindings/arm/cpus.txt |  1 +
>  arch/arm/boot/dts/msm8974.dts                  | 23 ++++++++
>  arch/arm/mach-msm/board-dt-8974.c              |  3 +
>  arch/arm/mach-msm/platsmp.c                    | 79 ++++++++++++++++++++++++++
>  4 files changed, 106 insertions(+)
> 
> diff --git a/Documentation/devicetree/bindings/arm/cpus.txt b/Documentation/devicetree/bindings/arm/cpus.txt
> index 1132eac..7c3c677 100644
> --- a/Documentation/devicetree/bindings/arm/cpus.txt
> +++ b/Documentation/devicetree/bindings/arm/cpus.txt
> @@ -52,6 +52,7 @@ For the ARM architecture every CPU node must contain the following properties:
>  		 This should be one of:
>  		 "qcom,scss"
>  		 "qcom,kpssv1"
> +		 "qcom,kpssv2"

I guess I should've looked at the whole series before responding, that
answers my prior question about what variation we expect.

>  
>  Example:
>  
> diff --git a/arch/arm/boot/dts/msm8974.dts b/arch/arm/boot/dts/msm8974.dts
> index c31c097..ef35a9b 100644
> --- a/arch/arm/boot/dts/msm8974.dts
> +++ b/arch/arm/boot/dts/msm8974.dts
> @@ -7,6 +7,22 @@
>  	compatible = "qcom,msm8974";
>  	interrupt-parent = <&intc>;
>  
> +	cpus {
> +		#address-cells = <1>;
> +		#size-cells = <0>;
> +		compatible = "qcom,krait";
> +		device_type = "cpu";
> +		enable-method = "qcom,kpssv2";
> +
> +		cpu@0 {
> +			reg = <0>;
> +		};
> +
> +		cpu@1 {
> +			reg = <1>;
> +		};
> +	};
> +
>  	intc: interrupt-controller@f9000000 {
>  		compatible = "qcom,msm-qgic2";
>  		interrupt-controller;
> @@ -23,4 +39,11 @@
>  			     <1 1 0xf08>;
>  		clock-frequency = <19200000>;
>  	};
> +
> +	kpss@f9012000 {
> +		compatible = "qcom,kpss";
> +		reg = <0xf9012000 0x1000>,
> +		      <0xf9088000 0x1000>,
> +		      <0xf9098000 0x1000>;
> +	};

In the previous examples, the number of CPUs was equal to the number of
kpss reg values. Why does it differ here. Either:

* We always have the extra regsiter here, and it should be described
  even if we don't use it.

* It's a different hardware block, and needs a more specific
  compatible string.

Eitehr way this strengthens my feeling that we need to define the linkage
from a CPU to the portion of the kpss which affects it.

>  };
> diff --git a/arch/arm/mach-msm/board-dt-8974.c b/arch/arm/mach-msm/board-dt-8974.c
> index d7f84f2..06119f9 100644
> --- a/arch/arm/mach-msm/board-dt-8974.c
> +++ b/arch/arm/mach-msm/board-dt-8974.c
> @@ -13,11 +13,14 @@
>  #include <linux/of_platform.h>
>  #include <asm/mach/arch.h>
>  
> +#include "common.h"
> +
>  static const char * const msm8974_dt_match[] __initconst = {
>  	"qcom,msm8974",
>  	NULL
>  };
>  
>  DT_MACHINE_START(MSM8974_DT, "Qualcomm MSM (Flattened Device Tree)")
> +	.smp = smp_ops(msm_smp_ops),
>  	.dt_compat = msm8974_dt_match,
>  MACHINE_END
> diff --git a/arch/arm/mach-msm/platsmp.c b/arch/arm/mach-msm/platsmp.c
> index 82eb079..0fdae69 100644
> --- a/arch/arm/mach-msm/platsmp.c
> +++ b/arch/arm/mach-msm/platsmp.c
> @@ -124,6 +124,80 @@ static int msm8960_release_secondary(unsigned int cpu)
>  	return 0;
>  }
>  
> +static int msm8974_release_secondary(unsigned int cpu)
> +{
> +	void __iomem *reg;
> +	void __iomem *l2_saw_base;
> +	struct device_node *dn = NULL;
> +	unsigned apc_pwr_gate_ctl = 0x14;
> +	unsigned reg_val;
> +
> +	if (cpu == 0 || cpu >= num_possible_cpus())
> +		return -EINVAL;
> +
> +	dn = of_find_compatible_node(dn, NULL, "qcom,kpss");
> +	if (!dn) {
> +		pr_err("%s : Missing kpss node from device tree\n", __func__);
> +		return -ENXIO;
> +	}
> +
> +	reg = of_iomap(dn, cpu+1);

This looks very fishy given the prior patch being one off from this.
why is reg[0] now different?

> +	if (!reg)
> +		return -ENOMEM;
> +
> +	pr_debug("Starting secondary CPU %d\n", cpu);
> +
> +	/* Turn on the BHS, turn off LDO Bypass and power down LDO */
> +	reg_val =  0x403f0001;

Magic number?

> +	writel_relaxed(reg_val, reg + apc_pwr_gate_ctl);
> +
> +	/* complete the above write before the delay */
> +	mb();

Use writel?

> +	/* wait for the bhs to settle */
> +	udelay(1);
> +
> +	/* Turn on BHS segments */
> +	reg_val |= 0x3f << 1;
> +	writel_relaxed(reg_val, reg + apc_pwr_gate_ctl);
> +
> +	/* complete the above write before the delay */
> +	mb();

Use writel again?

> +	 /* wait for the bhs to settle */
> +	udelay(1);
> +
> +	/* Finally turn on the bypass so that BHS supplies power */
> +	reg_val |= 0x3f << 8;
> +	writel_relaxed(reg_val, reg + apc_pwr_gate_ctl);
> +
> +	/* enable max phases */
> +	l2_saw_base = of_iomap(dn, 0);
> +	if (!l2_saw_base) {
> +		return -ENOMEM;
> +	}

What? 

You've just lost your only reference to the mapping in reg.

Why do you not do this at the start, before poking everything else? Even
better, do it at probe time and fail once rather than for each CPU you
have no chance of bringing up.

[...]
>  static void boot_cold_cpu(unsigned int cpu)
> @@ -151,6 +225,11 @@ static void boot_cold_cpu(unsigned int cpu)
>  			msm8960_release_secondary(cpu);
>  			per_cpu(cold_boot_done, cpu) = true;
>  		}
> +	} else if (!strcmp(enable_method, "qcom,kpssv2")) {
> +		if (per_cpu(cold_boot_done, cpu) == false) {
> +			msm8974_release_secondary(cpu);
> +			per_cpu(cold_boot_done, cpu) = true;
> +		}
>  	} else {
>  		pr_err("%s: Invalid enable-method property: %s\n",
>  				__func__, enable_method);

The enable-method parsing really should be moved out to common code. We
could make it scan the enable-method if a machine has no smp ops (which
is more general than the PSCI fallback that's been suggested before).

Thanks,
Mark.

Re: [RESEND PATCH 4/4] ARM: msm: Add support for 8974 SMP

[Mark Rutland]

On Fri, Aug 02, 2013 at 03:15:25AM +0100, Rohit Vaswani wrote:
> Add the cpus bindings and the Kraitv2 release sequence
> to make SMP work for 2 cores on MSM8974.
> 
> Signed-off-by: Rohit Vaswani <rvaswani@codeaurora.org>
> ---
>  Documentation/devicetree/bindings/arm/cpus.txt |  1 +
>  arch/arm/boot/dts/msm8974.dts                  | 23 ++++++++
>  arch/arm/mach-msm/board-dt-8974.c              |  3 +
>  arch/arm/mach-msm/platsmp.c                    | 79 ++++++++++++++++++++++++++
>  4 files changed, 106 insertions(+)
> 
> diff --git a/Documentation/devicetree/bindings/arm/cpus.txt b/Documentation/devicetree/bindings/arm/cpus.txt
> index 1132eac..7c3c677 100644
> --- a/Documentation/devicetree/bindings/arm/cpus.txt
> +++ b/Documentation/devicetree/bindings/arm/cpus.txt
> @@ -52,6 +52,7 @@ For the ARM architecture every CPU node must contain the following properties:
>  		 This should be one of:
>  		 "qcom,scss"
>  		 "qcom,kpssv1"
> +		 "qcom,kpssv2"

I guess I should've looked at the whole series before responding, that
answers my prior question about what variation we expect.

>  
>  Example:
>  
> diff --git a/arch/arm/boot/dts/msm8974.dts b/arch/arm/boot/dts/msm8974.dts
> index c31c097..ef35a9b 100644
> --- a/arch/arm/boot/dts/msm8974.dts
> +++ b/arch/arm/boot/dts/msm8974.dts
> @@ -7,6 +7,22 @@
>  	compatible = "qcom,msm8974";
>  	interrupt-parent = <&intc>;
>  
> +	cpus {
> +		#address-cells = <1>;
> +		#size-cells = <0>;
> +		compatible = "qcom,krait";
> +		device_type = "cpu";
> +		enable-method = "qcom,kpssv2";
> +
> +		cpu@0 {
> +			reg = <0>;
> +		};
> +
> +		cpu@1 {
> +			reg = <1>;
> +		};
> +	};
> +
>  	intc: interrupt-controller@f9000000 {
>  		compatible = "qcom,msm-qgic2";
>  		interrupt-controller;
> @@ -23,4 +39,11 @@
>  			     <1 1 0xf08>;
>  		clock-frequency = <19200000>;
>  	};
> +
> +	kpss@f9012000 {
> +		compatible = "qcom,kpss";
> +		reg = <0xf9012000 0x1000>,
> +		      <0xf9088000 0x1000>,
> +		      <0xf9098000 0x1000>;
> +	};

In the previous examples, the number of CPUs was equal to the number of
kpss reg values. Why does it differ here. Either:

* We always have the extra regsiter here, and it should be described
  even if we don't use it.

* It's a different hardware block, and needs a more specific
  compatible string.

Eitehr way this strengthens my feeling that we need to define the linkage
from a CPU to the portion of the kpss which affects it.

>  };
> diff --git a/arch/arm/mach-msm/board-dt-8974.c b/arch/arm/mach-msm/board-dt-8974.c
> index d7f84f2..06119f9 100644
> --- a/arch/arm/mach-msm/board-dt-8974.c
> +++ b/arch/arm/mach-msm/board-dt-8974.c
> @@ -13,11 +13,14 @@
>  #include <linux/of_platform.h>
>  #include <asm/mach/arch.h>
>  
> +#include "common.h"
> +
>  static const char * const msm8974_dt_match[] __initconst = {
>  	"qcom,msm8974",
>  	NULL
>  };
>  
>  DT_MACHINE_START(MSM8974_DT, "Qualcomm MSM (Flattened Device Tree)")
> +	.smp = smp_ops(msm_smp_ops),
>  	.dt_compat = msm8974_dt_match,
>  MACHINE_END
> diff --git a/arch/arm/mach-msm/platsmp.c b/arch/arm/mach-msm/platsmp.c
> index 82eb079..0fdae69 100644
> --- a/arch/arm/mach-msm/platsmp.c
> +++ b/arch/arm/mach-msm/platsmp.c
> @@ -124,6 +124,80 @@ static int msm8960_release_secondary(unsigned int cpu)
>  	return 0;
>  }
>  
> +static int msm8974_release_secondary(unsigned int cpu)
> +{
> +	void __iomem *reg;
> +	void __iomem *l2_saw_base;
> +	struct device_node *dn = NULL;
> +	unsigned apc_pwr_gate_ctl = 0x14;
> +	unsigned reg_val;
> +
> +	if (cpu == 0 || cpu >= num_possible_cpus())
> +		return -EINVAL;
> +
> +	dn = of_find_compatible_node(dn, NULL, "qcom,kpss");
> +	if (!dn) {
> +		pr_err("%s : Missing kpss node from device tree\n", __func__);
> +		return -ENXIO;
> +	}
> +
> +	reg = of_iomap(dn, cpu+1);

This looks very fishy given the prior patch being one off from this.
why is reg[0] now different?

> +	if (!reg)
> +		return -ENOMEM;
> +
> +	pr_debug("Starting secondary CPU %d\n", cpu);
> +
> +	/* Turn on the BHS, turn off LDO Bypass and power down LDO */
> +	reg_val =  0x403f0001;

Magic number?

> +	writel_relaxed(reg_val, reg + apc_pwr_gate_ctl);
> +
> +	/* complete the above write before the delay */
> +	mb();

Use writel?

> +	/* wait for the bhs to settle */
> +	udelay(1);
> +
> +	/* Turn on BHS segments */
> +	reg_val |= 0x3f << 1;
> +	writel_relaxed(reg_val, reg + apc_pwr_gate_ctl);
> +
> +	/* complete the above write before the delay */
> +	mb();

Use writel again?

> +	 /* wait for the bhs to settle */
> +	udelay(1);
> +
> +	/* Finally turn on the bypass so that BHS supplies power */
> +	reg_val |= 0x3f << 8;
> +	writel_relaxed(reg_val, reg + apc_pwr_gate_ctl);
> +
> +	/* enable max phases */
> +	l2_saw_base = of_iomap(dn, 0);
> +	if (!l2_saw_base) {
> +		return -ENOMEM;
> +	}

What? 

You've just lost your only reference to the mapping in reg.

Why do you not do this at the start, before poking everything else? Even
better, do it at probe time and fail once rather than for each CPU you
have no chance of bringing up.

[...]
>  static void boot_cold_cpu(unsigned int cpu)
> @@ -151,6 +225,11 @@ static void boot_cold_cpu(unsigned int cpu)
>  			msm8960_release_secondary(cpu);
>  			per_cpu(cold_boot_done, cpu) = true;
>  		}
> +	} else if (!strcmp(enable_method, "qcom,kpssv2")) {
> +		if (per_cpu(cold_boot_done, cpu) == false) {
> +			msm8974_release_secondary(cpu);
> +			per_cpu(cold_boot_done, cpu) = true;
> +		}
>  	} else {
>  		pr_err("%s: Invalid enable-method property: %s\n",
>  				__func__, enable_method);

The enable-method parsing really should be moved out to common code. We
could make it scan the enable-method if a machine has no smp ops (which
is more general than the PSCI fallback that's been suggested before).

Thanks,
Mark.

[RESEND PATCH 4/4] ARM: msm: Add support for 8974 SMP

[Mark Rutland]

On Fri, Aug 02, 2013 at 03:15:25AM +0100, Rohit Vaswani wrote:
> Add the cpus bindings and the Kraitv2 release sequence
> to make SMP work for 2 cores on MSM8974.
> 
> Signed-off-by: Rohit Vaswani <rvaswani@codeaurora.org>
> ---
>  Documentation/devicetree/bindings/arm/cpus.txt |  1 +
>  arch/arm/boot/dts/msm8974.dts                  | 23 ++++++++
>  arch/arm/mach-msm/board-dt-8974.c              |  3 +
>  arch/arm/mach-msm/platsmp.c                    | 79 ++++++++++++++++++++++++++
>  4 files changed, 106 insertions(+)
> 
> diff --git a/Documentation/devicetree/bindings/arm/cpus.txt b/Documentation/devicetree/bindings/arm/cpus.txt
> index 1132eac..7c3c677 100644
> --- a/Documentation/devicetree/bindings/arm/cpus.txt
> +++ b/Documentation/devicetree/bindings/arm/cpus.txt
> @@ -52,6 +52,7 @@ For the ARM architecture every CPU node must contain the following properties:
>  		 This should be one of:
>  		 "qcom,scss"
>  		 "qcom,kpssv1"
> +		 "qcom,kpssv2"

I guess I should've looked at the whole series before responding, that
answers my prior question about what variation we expect.

>  
>  Example:
>  
> diff --git a/arch/arm/boot/dts/msm8974.dts b/arch/arm/boot/dts/msm8974.dts
> index c31c097..ef35a9b 100644
> --- a/arch/arm/boot/dts/msm8974.dts
> +++ b/arch/arm/boot/dts/msm8974.dts
> @@ -7,6 +7,22 @@
>  	compatible = "qcom,msm8974";
>  	interrupt-parent = <&intc>;
>  
> +	cpus {
> +		#address-cells = <1>;
> +		#size-cells = <0>;
> +		compatible = "qcom,krait";
> +		device_type = "cpu";
> +		enable-method = "qcom,kpssv2";
> +
> +		cpu at 0 {
> +			reg = <0>;
> +		};
> +
> +		cpu at 1 {
> +			reg = <1>;
> +		};
> +	};
> +
>  	intc: interrupt-controller at f9000000 {
>  		compatible = "qcom,msm-qgic2";
>  		interrupt-controller;
> @@ -23,4 +39,11 @@
>  			     <1 1 0xf08>;
>  		clock-frequency = <19200000>;
>  	};
> +
> +	kpss at f9012000 {
> +		compatible = "qcom,kpss";
> +		reg = <0xf9012000 0x1000>,
> +		      <0xf9088000 0x1000>,
> +		      <0xf9098000 0x1000>;
> +	};

In the previous examples, the number of CPUs was equal to the number of
kpss reg values. Why does it differ here. Either:

* We always have the extra regsiter here, and it should be described
  even if we don't use it.

* It's a different hardware block, and needs a more specific
  compatible string.

Eitehr way this strengthens my feeling that we need to define the linkage
from a CPU to the portion of the kpss which affects it.

>  };
> diff --git a/arch/arm/mach-msm/board-dt-8974.c b/arch/arm/mach-msm/board-dt-8974.c
> index d7f84f2..06119f9 100644
> --- a/arch/arm/mach-msm/board-dt-8974.c
> +++ b/arch/arm/mach-msm/board-dt-8974.c
> @@ -13,11 +13,14 @@
>  #include <linux/of_platform.h>
>  #include <asm/mach/arch.h>
>  
> +#include "common.h"
> +
>  static const char * const msm8974_dt_match[] __initconst = {
>  	"qcom,msm8974",
>  	NULL
>  };
>  
>  DT_MACHINE_START(MSM8974_DT, "Qualcomm MSM (Flattened Device Tree)")
> +	.smp = smp_ops(msm_smp_ops),
>  	.dt_compat = msm8974_dt_match,
>  MACHINE_END
> diff --git a/arch/arm/mach-msm/platsmp.c b/arch/arm/mach-msm/platsmp.c
> index 82eb079..0fdae69 100644
> --- a/arch/arm/mach-msm/platsmp.c
> +++ b/arch/arm/mach-msm/platsmp.c
> @@ -124,6 +124,80 @@ static int msm8960_release_secondary(unsigned int cpu)
>  	return 0;
>  }
>  
> +static int msm8974_release_secondary(unsigned int cpu)
> +{
> +	void __iomem *reg;
> +	void __iomem *l2_saw_base;
> +	struct device_node *dn = NULL;
> +	unsigned apc_pwr_gate_ctl = 0x14;
> +	unsigned reg_val;
> +
> +	if (cpu == 0 || cpu >= num_possible_cpus())
> +		return -EINVAL;
> +
> +	dn = of_find_compatible_node(dn, NULL, "qcom,kpss");
> +	if (!dn) {
> +		pr_err("%s : Missing kpss node from device tree\n", __func__);
> +		return -ENXIO;
> +	}
> +
> +	reg = of_iomap(dn, cpu+1);

This looks very fishy given the prior patch being one off from this.
why is reg[0] now different?

> +	if (!reg)
> +		return -ENOMEM;
> +
> +	pr_debug("Starting secondary CPU %d\n", cpu);
> +
> +	/* Turn on the BHS, turn off LDO Bypass and power down LDO */
> +	reg_val =  0x403f0001;

Magic number?

> +	writel_relaxed(reg_val, reg + apc_pwr_gate_ctl);
> +
> +	/* complete the above write before the delay */
> +	mb();

Use writel?

> +	/* wait for the bhs to settle */
> +	udelay(1);
> +
> +	/* Turn on BHS segments */
> +	reg_val |= 0x3f << 1;
> +	writel_relaxed(reg_val, reg + apc_pwr_gate_ctl);
> +
> +	/* complete the above write before the delay */
> +	mb();

Use writel again?

> +	 /* wait for the bhs to settle */
> +	udelay(1);
> +
> +	/* Finally turn on the bypass so that BHS supplies power */
> +	reg_val |= 0x3f << 8;
> +	writel_relaxed(reg_val, reg + apc_pwr_gate_ctl);
> +
> +	/* enable max phases */
> +	l2_saw_base = of_iomap(dn, 0);
> +	if (!l2_saw_base) {
> +		return -ENOMEM;
> +	}

What? 

You've just lost your only reference to the mapping in reg.

Why do you not do this at the start, before poking everything else? Even
better, do it at probe time and fail once rather than for each CPU you
have no chance of bringing up.

[...]
>  static void boot_cold_cpu(unsigned int cpu)
> @@ -151,6 +225,11 @@ static void boot_cold_cpu(unsigned int cpu)
>  			msm8960_release_secondary(cpu);
>  			per_cpu(cold_boot_done, cpu) = true;
>  		}
> +	} else if (!strcmp(enable_method, "qcom,kpssv2")) {
> +		if (per_cpu(cold_boot_done, cpu) == false) {
> +			msm8974_release_secondary(cpu);
> +			per_cpu(cold_boot_done, cpu) = true;
> +		}
>  	} else {
>  		pr_err("%s: Invalid enable-method property: %s\n",
>  				__func__, enable_method);

The enable-method parsing really should be moved out to common code. We
could make it scan the enable-method if a machine has no smp ops (which
is more general than the PSCI fallback that's been suggested before).

Thanks,
Mark.

Re: [RESEND PATCH 2/4] ARM: msm: Re-organize platsmp to make it extensible

[Rohit Vaswani]

On 8/12/2013 8:50 AM, Mark Rutland wrote:
> Hi,
>
> Apologies for the long delay for review on this.
>
> I really like the direction this is going, but I have some qualms with
> the implementation.

Thanks for your review, but a few direct recommendations would help in 
making the implementation right.
>
> On Fri, Aug 02, 2013 at 03:15:23AM +0100, Rohit Vaswani wrote:
>> This makes it easy to add SMP support for new targets
>> by adding cpus property and the release sequence.
>> We add the enable-method property for the cpus property to
>> specify which release sequence to use.
>> While at it, add the 8660 cpus bindings to make SMP work.
>>
>> Signed-off-by: Rohit Vaswani <rvaswani@codeaurora.org>
>> ---
>>   Documentation/devicetree/bindings/arm/cpus.txt     |  6 ++
>>   Documentation/devicetree/bindings/arm/msm/scss.txt | 15 ++++
>>   arch/arm/boot/dts/msm8660-surf.dts                 | 23 +++++-
>>   arch/arm/mach-msm/platsmp.c                        | 94 +++++++++++++++++-----
>>   4 files changed, 115 insertions(+), 23 deletions(-)
>>   create mode 100644 Documentation/devicetree/bindings/arm/msm/scss.txt
>>
>> diff --git a/Documentation/devicetree/bindings/arm/cpus.txt b/Documentation/devicetree/bindings/arm/cpus.txt
>> index f32494d..327aad2 100644
>> --- a/Documentation/devicetree/bindings/arm/cpus.txt
>> +++ b/Documentation/devicetree/bindings/arm/cpus.txt
>> @@ -44,6 +44,12 @@ For the ARM architecture every CPU node must contain the following properties:
>>   		"marvell,mohawk"
>>   		"marvell,xsc3"
>>   		"marvell,xscale"
>> +		"qcom,scorpion"
>> +- enable-method: Specifies the method used to enable or take the secondary cores
>> +		 out of reset. This allows different reset sequence for
>> +		 different types of cpus.
>> +		 This should be one of:
>> +		 "qcom,scss"
> While I'd certainly like to see a move to using enable-method for
> 32-bit, I think this needs a bit more thought:
>
> What does "qcom,scss" mean, precisely? It seems to be that we poke some
> registers in a "qcom,scss" device. I think we need to document
> enable-methods *very* carefully (and we need to do that for 64-bit as
> well with psci), as it's very likely there'll be subtle
> incompatibilities between platforms, especially if firmware is involved.
> We should try to leaves as little room for error as possible.
Yes qcom,scss implies poking registers in the qcom,scss device. How 
could I make that more clear in the documentation ?

>
> I don't want to see this devolve into meaning "whatever the Linux driver
> for this happens to do at the current point in time", as that just leads
> to breakage as we have no clear distinction between actual requirements
> and implementation details.
>
> Given we already have platforms without an enable-method, we can't make
> it a required property either -- those platforms are already booting by
> figuring out an enable method from the platform's compatible string.
So, you recommend we continue to using the platform compatible string as 
well ?
We currently don't have a perfect method to use enable-method in generic 
code. More on this below...
>
> With PSCI, enable-method also implies a method for disabling a
> particular CPU, so it would be nice for the description to cover this.
>
>>   
>>   Example:
>>   
>> diff --git a/Documentation/devicetree/bindings/arm/msm/scss.txt b/Documentation/devicetree/bindings/arm/msm/scss.txt
>> new file mode 100644
>> index 0000000..21c3e26
>> --- /dev/null
>> +++ b/Documentation/devicetree/bindings/arm/msm/scss.txt
>> @@ -0,0 +1,15 @@
>> +* SCSS - Scorpion Sub-system
>> +
>> +Properties
>> +
>> +- compatible : Should contain "qcom,scss".
>> +
>> +- reg: Specifies the base address for the SCSS registers used for
>> +       booting up secondary cores.
>> +
>> +Example:
>> +
>> +	scss@902000 {
>> +		compatible = "qcom,scss";
>> +		reg = <0x00902000 0x2000>;
>> +	};
>> diff --git a/arch/arm/boot/dts/msm8660-surf.dts b/arch/arm/boot/dts/msm8660-surf.dts
>> index cdc010e..203e51a 100644
>> --- a/arch/arm/boot/dts/msm8660-surf.dts
>> +++ b/arch/arm/boot/dts/msm8660-surf.dts
>> @@ -7,6 +7,22 @@
>>   	compatible = "qcom,msm8660-surf", "qcom,msm8660";
>>   	interrupt-parent = <&intc>;
>>   
>> +	cpus {
>> +		#address-cells = <1>;
>> +		#size-cells = <0>;
>> +		compatible = "qcom,scorpion";
>> +		device_type = "cpu";
>> +		enable-method = "qcom,scss";
>> +
>> +		cpu@0 {
>> +			reg = <0>;
>> +		};
>> +
>> +		cpu@1 {
>> +			reg = <1>;
>> +		};
>> +	};
>> +
> I *really* like moving the common properties out into the /cpus node --
> ePAPR suggests it, it's less error prone, and it takes up less space.
> However, I'm not sure our generic/arch code handles it properly, and I
> think we need to audit that before we can start writing dts that depend
> on it. I'd be happy to be wrong here if anyone can correct me. :)
>
> We probably need to factor out the way we read properties for cpu nodes,
> falling back to ones present in /cpus if not present. There's already a
> lot of pain getting the node for a logical (rather than physical) cpu
> id.
>
> Sudeep recently factored out finding the cpu node for a logical cpu id
> [1,2] in generic code with a per-arch callback, it shouldn't be too hard
> to have shims around that to read (optionally) common properties.
>
> [...]
>
>> -static void prepare_cold_cpu(unsigned int cpu)
>> +static int scorpion_release_secondary(void)
>>   {
>> -	int ret;
>> -	ret = scm_set_boot_addr(virt_to_phys(secondary_startup),
>> -				SCM_FLAG_COLDBOOT_CPU1);
>> -	if (ret == 0) {
>> -		void __iomem *sc1_base_ptr;
>> -		sc1_base_ptr = ioremap_nocache(0x00902000, SZ_4K*2);
>> -		if (sc1_base_ptr) {
>> -			writel(0, sc1_base_ptr + VDD_SC1_ARRAY_CLAMP_GFS_CTL);
>> -			writel(0, sc1_base_ptr + SCSS_CPU1CORE_RESET);
>> -			writel(3, sc1_base_ptr + SCSS_DBG_STATUS_CORE_PWRDUP);
>> -			iounmap(sc1_base_ptr);
>> -		}
>> -	} else
>> -		printk(KERN_DEBUG "Failed to set secondary core boot "
>> -				  "address\n");
>> +	void __iomem *sc1_base_ptr;
>> +	struct device_node *dn = NULL;
>> +
>> +	dn = of_find_compatible_node(dn, NULL, "qcom,scss");
>> +	if (!dn) {
>> +		pr_err("%s: Missing scss node in device tree\n", __func__);
>> +		return -ENXIO;
>> +	}
>> +
>> +	sc1_base_ptr = of_iomap(dn, 0);
>> +	if (sc1_base_ptr) {
>> +		writel_relaxed(0, sc1_base_ptr + VDD_SC1_ARRAY_CLAMP_GFS_CTL);
>> +		writel_relaxed(0, sc1_base_ptr + SCSS_CPU1CORE_RESET);
>> +		writel_relaxed(3, sc1_base_ptr + SCSS_DBG_STATUS_CORE_PWRDUP);
>> +		mb();
>> +		iounmap(sc1_base_ptr);
> Does this boot *all* secondary CPUs directly into the kernel? Is that
> safe (e.g. if the kernel supports fewer CPUs than are physically
> present)?
Im not sure I understand the concern with safety here. The CPU for which 
the release_secondary will be called will be taken out of reset with 
this sequence.

>
> Is this a one-time thing, or are we able to dynamically hotplug CPUs? If
> the latter, the map/unmap looks odd to me.

This is a one-time thing done for each CPU that's specified in the 
device tree (or based on the command line over-ride).
>> +	} else {
>> +		return -ENOMEM;
>> +	}
>> +
>> +	return 0;
>>   }
>>   
>> -static int msm_boot_secondary(unsigned int cpu, struct task_struct *idle)
>> +static DEFINE_PER_CPU(int, cold_boot_done);
>> +
>> +static void boot_cold_cpu(unsigned int cpu)
>>   {
>> -	static int cold_boot_done;
>> +	const char *enable_method;
>> +	struct device_node *dn = NULL;
>>   
>> -	/* Only need to bring cpu out of reset this way once */
>> -	if (cold_boot_done == false) {
>> -		prepare_cold_cpu(cpu);
>> -		cold_boot_done = true;
>> +	dn = of_find_node_by_name(dn, "cpus");
>> +	if (!dn) {
>> +		pr_err("%s: Missing node cpus in device tree\n", __func__);
>> +		return;
>>   	}
>>   
>> +	enable_method = of_get_property(dn, "enable-method", NULL);
> Please factor this out from the platform code.
>
> If we're going to use enable-method, it should be decoupled from
> platform code -- common code should parse it to find the appropriate
> handler. Also, we *must* support having an enable-method per-cpu as KVM
> does for PSCI (though I definitely would like support for shared
> properties as mentioned above).
Currently with most platforms, smp.c calls into the boot_secondary 
function which decides which cpu it is
and then calls the appropriate function. This is because smp ops allow 
only 1 callback function to be registered...
Hence, using the enable-method in platform specific code works.

If we create a generic property should it mandate having generic code 
handle that ?
I currently don't have a good way of incorporating enable-method in 
generic code as it would mean to establish a mechanism to
associate the enable-method string with cpu specific boot_secondary 
functions.
You have an approach in mind that I can try ?

>> +	if (!enable_method) {
>> +			pr_err("%s: cpus node is missing enable-method property\n",
>> +					__func__);
>> +	} else if (!strcmp(enable_method, "qcom,scss")) {
>> +		if (per_cpu(cold_boot_done, cpu) == false) {
>> +			scorpion_release_secondary();
>> +			per_cpu(cold_boot_done, cpu) = true;
>> +		}
>> +	} else {
>> +		pr_err("%s: Invalid enable-method property: %s\n",
>> +				__func__, enable_method);
>> +	}
>> +}
>> +
>> +static int msm_boot_secondary(unsigned int cpu, struct task_struct *idle)
>> +{
>> +	boot_cold_cpu(cpu);
>> +
>>   	/*
>>   	 * set synchronisation state between this boot processor
>>   	 * and the secondary one
>> @@ -118,8 +148,28 @@ static void __init msm_smp_init_cpus(void)
>>   		set_cpu_possible(i, true);
>>   }
>>   
>> +static const int cold_boot_flags[] __initconst = {
>> +	0,
>> +	SCM_FLAG_COLDBOOT_CPU1,
>> +};
> So we only ever support booting two CPUs?
The next patch which adds support for a quadcore chip, adds more flags.
>
> Is the mapping of MPIDR to register bits arbitrary? Or do we know what
> they would be for four CPUs, four clusters, and beyond?
Four cpus.

>
>> +
>>   static void __init msm_smp_prepare_cpus(unsigned int max_cpus)
>>   {
>> +	int cpu, map;
>> +	unsigned int flags = 0;
>> +
>> +	for_each_present_cpu(cpu) {
>> +		map = cpu_logical_map(cpu);
>> +		if (map > ARRAY_SIZE(cold_boot_flags)) {
>> +			set_cpu_present(cpu, false);
>> +			__WARN();
>> +			continue;
>> +		}
>> +		flags |= cold_boot_flags[map];
> It's a shame that this leaves a window where some CPUs seem bootable,
> but aren't (though I can't offer a better way of handling this given we
> have dts without enable-method properties).
Any CPU that seems bootable and is defined in DT, should be bootable.
The cold_boot_flags are just a mechanism to set appropriate values for 
the scm call and
they aren't used as a method to disallow CPUs from being bootable and 
hence the __WARN() if this is done incorrectly.

>
>> +	}
>> +
>> +	if (scm_set_boot_addr(virt_to_phys(secondary_startup), flags))
>> +		pr_warn("Failed to set CPU boot address\n");
>>   }
>>   
>>   struct smp_operations msm_smp_ops __initdata = {
>> -- 
>> The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
>> hosted by The Linux Foundation
>>
> Thanks,
> Mark
>
> [1] http://lists.infradead.org/pipermail/linux-arm-kernel/2013-July/184150.html
> [2] http://lists.infradead.org/pipermail/linux-arm-kernel/2013-August/189619.html
> --
> To unsubscribe from this list: send the line "unsubscribe linux-arm-msm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html


Thanks,
Rohit Vaswani

-- 
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum, hosted by The Linux Foundation

Re: [RESEND PATCH 2/4] ARM: msm: Re-organize platsmp to make it extensible

[Rohit Vaswani]

On 8/12/2013 8:50 AM, Mark Rutland wrote:
> Hi,
>
> Apologies for the long delay for review on this.
>
> I really like the direction this is going, but I have some qualms with
> the implementation.

Thanks for your review, but a few direct recommendations would help in 
making the implementation right.
>
> On Fri, Aug 02, 2013 at 03:15:23AM +0100, Rohit Vaswani wrote:
>> This makes it easy to add SMP support for new targets
>> by adding cpus property and the release sequence.
>> We add the enable-method property for the cpus property to
>> specify which release sequence to use.
>> While at it, add the 8660 cpus bindings to make SMP work.
>>
>> Signed-off-by: Rohit Vaswani <rvaswani@codeaurora.org>
>> ---
>>   Documentation/devicetree/bindings/arm/cpus.txt     |  6 ++
>>   Documentation/devicetree/bindings/arm/msm/scss.txt | 15 ++++
>>   arch/arm/boot/dts/msm8660-surf.dts                 | 23 +++++-
>>   arch/arm/mach-msm/platsmp.c                        | 94 +++++++++++++++++-----
>>   4 files changed, 115 insertions(+), 23 deletions(-)
>>   create mode 100644 Documentation/devicetree/bindings/arm/msm/scss.txt
>>
>> diff --git a/Documentation/devicetree/bindings/arm/cpus.txt b/Documentation/devicetree/bindings/arm/cpus.txt
>> index f32494d..327aad2 100644
>> --- a/Documentation/devicetree/bindings/arm/cpus.txt
>> +++ b/Documentation/devicetree/bindings/arm/cpus.txt
>> @@ -44,6 +44,12 @@ For the ARM architecture every CPU node must contain the following properties:
>>   		"marvell,mohawk"
>>   		"marvell,xsc3"
>>   		"marvell,xscale"
>> +		"qcom,scorpion"
>> +- enable-method: Specifies the method used to enable or take the secondary cores
>> +		 out of reset. This allows different reset sequence for
>> +		 different types of cpus.
>> +		 This should be one of:
>> +		 "qcom,scss"
> While I'd certainly like to see a move to using enable-method for
> 32-bit, I think this needs a bit more thought:
>
> What does "qcom,scss" mean, precisely? It seems to be that we poke some
> registers in a "qcom,scss" device. I think we need to document
> enable-methods *very* carefully (and we need to do that for 64-bit as
> well with psci), as it's very likely there'll be subtle
> incompatibilities between platforms, especially if firmware is involved.
> We should try to leaves as little room for error as possible.
Yes qcom,scss implies poking registers in the qcom,scss device. How 
could I make that more clear in the documentation ?

>
> I don't want to see this devolve into meaning "whatever the Linux driver
> for this happens to do at the current point in time", as that just leads
> to breakage as we have no clear distinction between actual requirements
> and implementation details.
>
> Given we already have platforms without an enable-method, we can't make
> it a required property either -- those platforms are already booting by
> figuring out an enable method from the platform's compatible string.
So, you recommend we continue to using the platform compatible string as 
well ?
We currently don't have a perfect method to use enable-method in generic 
code. More on this below...
>
> With PSCI, enable-method also implies a method for disabling a
> particular CPU, so it would be nice for the description to cover this.
>
>>   
>>   Example:
>>   
>> diff --git a/Documentation/devicetree/bindings/arm/msm/scss.txt b/Documentation/devicetree/bindings/arm/msm/scss.txt
>> new file mode 100644
>> index 0000000..21c3e26
>> --- /dev/null
>> +++ b/Documentation/devicetree/bindings/arm/msm/scss.txt
>> @@ -0,0 +1,15 @@
>> +* SCSS - Scorpion Sub-system
>> +
>> +Properties
>> +
>> +- compatible : Should contain "qcom,scss".
>> +
>> +- reg: Specifies the base address for the SCSS registers used for
>> +       booting up secondary cores.
>> +
>> +Example:
>> +
>> +	scss@902000 {
>> +		compatible = "qcom,scss";
>> +		reg = <0x00902000 0x2000>;
>> +	};
>> diff --git a/arch/arm/boot/dts/msm8660-surf.dts b/arch/arm/boot/dts/msm8660-surf.dts
>> index cdc010e..203e51a 100644
>> --- a/arch/arm/boot/dts/msm8660-surf.dts
>> +++ b/arch/arm/boot/dts/msm8660-surf.dts
>> @@ -7,6 +7,22 @@
>>   	compatible = "qcom,msm8660-surf", "qcom,msm8660";
>>   	interrupt-parent = <&intc>;
>>   
>> +	cpus {
>> +		#address-cells = <1>;
>> +		#size-cells = <0>;
>> +		compatible = "qcom,scorpion";
>> +		device_type = "cpu";
>> +		enable-method = "qcom,scss";
>> +
>> +		cpu@0 {
>> +			reg = <0>;
>> +		};
>> +
>> +		cpu@1 {
>> +			reg = <1>;
>> +		};
>> +	};
>> +
> I *really* like moving the common properties out into the /cpus node --
> ePAPR suggests it, it's less error prone, and it takes up less space.
> However, I'm not sure our generic/arch code handles it properly, and I
> think we need to audit that before we can start writing dts that depend
> on it. I'd be happy to be wrong here if anyone can correct me. :)
>
> We probably need to factor out the way we read properties for cpu nodes,
> falling back to ones present in /cpus if not present. There's already a
> lot of pain getting the node for a logical (rather than physical) cpu
> id.
>
> Sudeep recently factored out finding the cpu node for a logical cpu id
> [1,2] in generic code with a per-arch callback, it shouldn't be too hard
> to have shims around that to read (optionally) common properties.
>
> [...]
>
>> -static void prepare_cold_cpu(unsigned int cpu)
>> +static int scorpion_release_secondary(void)
>>   {
>> -	int ret;
>> -	ret = scm_set_boot_addr(virt_to_phys(secondary_startup),
>> -				SCM_FLAG_COLDBOOT_CPU1);
>> -	if (ret == 0) {
>> -		void __iomem *sc1_base_ptr;
>> -		sc1_base_ptr = ioremap_nocache(0x00902000, SZ_4K*2);
>> -		if (sc1_base_ptr) {
>> -			writel(0, sc1_base_ptr + VDD_SC1_ARRAY_CLAMP_GFS_CTL);
>> -			writel(0, sc1_base_ptr + SCSS_CPU1CORE_RESET);
>> -			writel(3, sc1_base_ptr + SCSS_DBG_STATUS_CORE_PWRDUP);
>> -			iounmap(sc1_base_ptr);
>> -		}
>> -	} else
>> -		printk(KERN_DEBUG "Failed to set secondary core boot "
>> -				  "address\n");
>> +	void __iomem *sc1_base_ptr;
>> +	struct device_node *dn = NULL;
>> +
>> +	dn = of_find_compatible_node(dn, NULL, "qcom,scss");
>> +	if (!dn) {
>> +		pr_err("%s: Missing scss node in device tree\n", __func__);
>> +		return -ENXIO;
>> +	}
>> +
>> +	sc1_base_ptr = of_iomap(dn, 0);
>> +	if (sc1_base_ptr) {
>> +		writel_relaxed(0, sc1_base_ptr + VDD_SC1_ARRAY_CLAMP_GFS_CTL);
>> +		writel_relaxed(0, sc1_base_ptr + SCSS_CPU1CORE_RESET);
>> +		writel_relaxed(3, sc1_base_ptr + SCSS_DBG_STATUS_CORE_PWRDUP);
>> +		mb();
>> +		iounmap(sc1_base_ptr);
> Does this boot *all* secondary CPUs directly into the kernel? Is that
> safe (e.g. if the kernel supports fewer CPUs than are physically
> present)?
Im not sure I understand the concern with safety here. The CPU for which 
the release_secondary will be called will be taken out of reset with 
this sequence.

>
> Is this a one-time thing, or are we able to dynamically hotplug CPUs? If
> the latter, the map/unmap looks odd to me.

This is a one-time thing done for each CPU that's specified in the 
device tree (or based on the command line over-ride).
>> +	} else {
>> +		return -ENOMEM;
>> +	}
>> +
>> +	return 0;
>>   }
>>   
>> -static int msm_boot_secondary(unsigned int cpu, struct task_struct *idle)
>> +static DEFINE_PER_CPU(int, cold_boot_done);
>> +
>> +static void boot_cold_cpu(unsigned int cpu)
>>   {
>> -	static int cold_boot_done;
>> +	const char *enable_method;
>> +	struct device_node *dn = NULL;
>>   
>> -	/* Only need to bring cpu out of reset this way once */
>> -	if (cold_boot_done == false) {
>> -		prepare_cold_cpu(cpu);
>> -		cold_boot_done = true;
>> +	dn = of_find_node_by_name(dn, "cpus");
>> +	if (!dn) {
>> +		pr_err("%s: Missing node cpus in device tree\n", __func__);
>> +		return;
>>   	}
>>   
>> +	enable_method = of_get_property(dn, "enable-method", NULL);
> Please factor this out from the platform code.
>
> If we're going to use enable-method, it should be decoupled from
> platform code -- common code should parse it to find the appropriate
> handler. Also, we *must* support having an enable-method per-cpu as KVM
> does for PSCI (though I definitely would like support for shared
> properties as mentioned above).
Currently with most platforms, smp.c calls into the boot_secondary 
function which decides which cpu it is
and then calls the appropriate function. This is because smp ops allow 
only 1 callback function to be registered...
Hence, using the enable-method in platform specific code works.

If we create a generic property should it mandate having generic code 
handle that ?
I currently don't have a good way of incorporating enable-method in 
generic code as it would mean to establish a mechanism to
associate the enable-method string with cpu specific boot_secondary 
functions.
You have an approach in mind that I can try ?

>> +	if (!enable_method) {
>> +			pr_err("%s: cpus node is missing enable-method property\n",
>> +					__func__);
>> +	} else if (!strcmp(enable_method, "qcom,scss")) {
>> +		if (per_cpu(cold_boot_done, cpu) == false) {
>> +			scorpion_release_secondary();
>> +			per_cpu(cold_boot_done, cpu) = true;
>> +		}
>> +	} else {
>> +		pr_err("%s: Invalid enable-method property: %s\n",
>> +				__func__, enable_method);
>> +	}
>> +}
>> +
>> +static int msm_boot_secondary(unsigned int cpu, struct task_struct *idle)
>> +{
>> +	boot_cold_cpu(cpu);
>> +
>>   	/*
>>   	 * set synchronisation state between this boot processor
>>   	 * and the secondary one
>> @@ -118,8 +148,28 @@ static void __init msm_smp_init_cpus(void)
>>   		set_cpu_possible(i, true);
>>   }
>>   
>> +static const int cold_boot_flags[] __initconst = {
>> +	0,
>> +	SCM_FLAG_COLDBOOT_CPU1,
>> +};
> So we only ever support booting two CPUs?
The next patch which adds support for a quadcore chip, adds more flags.
>
> Is the mapping of MPIDR to register bits arbitrary? Or do we know what
> they would be for four CPUs, four clusters, and beyond?
Four cpus.

>
>> +
>>   static void __init msm_smp_prepare_cpus(unsigned int max_cpus)
>>   {
>> +	int cpu, map;
>> +	unsigned int flags = 0;
>> +
>> +	for_each_present_cpu(cpu) {
>> +		map = cpu_logical_map(cpu);
>> +		if (map > ARRAY_SIZE(cold_boot_flags)) {
>> +			set_cpu_present(cpu, false);
>> +			__WARN();
>> +			continue;
>> +		}
>> +		flags |= cold_boot_flags[map];
> It's a shame that this leaves a window where some CPUs seem bootable,
> but aren't (though I can't offer a better way of handling this given we
> have dts without enable-method properties).
Any CPU that seems bootable and is defined in DT, should be bootable.
The cold_boot_flags are just a mechanism to set appropriate values for 
the scm call and
they aren't used as a method to disallow CPUs from being bootable and 
hence the __WARN() if this is done incorrectly.

>
>> +	}
>> +
>> +	if (scm_set_boot_addr(virt_to_phys(secondary_startup), flags))
>> +		pr_warn("Failed to set CPU boot address\n");
>>   }
>>   
>>   struct smp_operations msm_smp_ops __initdata = {
>> -- 
>> The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
>> hosted by The Linux Foundation
>>
> Thanks,
> Mark
>
> [1] http://lists.infradead.org/pipermail/linux-arm-kernel/2013-July/184150.html
> [2] http://lists.infradead.org/pipermail/linux-arm-kernel/2013-August/189619.html
> --
> To unsubscribe from this list: send the line "unsubscribe linux-arm-msm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html


Thanks,
Rohit Vaswani

-- 
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum, hosted by The Linux Foundation

[RESEND PATCH 2/4] ARM: msm: Re-organize platsmp to make it extensible

[Rohit Vaswani]

On 8/12/2013 8:50 AM, Mark Rutland wrote:
> Hi,
>
> Apologies for the long delay for review on this.
>
> I really like the direction this is going, but I have some qualms with
> the implementation.

Thanks for your review, but a few direct recommendations would help in 
making the implementation right.
>
> On Fri, Aug 02, 2013 at 03:15:23AM +0100, Rohit Vaswani wrote:
>> This makes it easy to add SMP support for new targets
>> by adding cpus property and the release sequence.
>> We add the enable-method property for the cpus property to
>> specify which release sequence to use.
>> While at it, add the 8660 cpus bindings to make SMP work.
>>
>> Signed-off-by: Rohit Vaswani <rvaswani@codeaurora.org>
>> ---
>>   Documentation/devicetree/bindings/arm/cpus.txt     |  6 ++
>>   Documentation/devicetree/bindings/arm/msm/scss.txt | 15 ++++
>>   arch/arm/boot/dts/msm8660-surf.dts                 | 23 +++++-
>>   arch/arm/mach-msm/platsmp.c                        | 94 +++++++++++++++++-----
>>   4 files changed, 115 insertions(+), 23 deletions(-)
>>   create mode 100644 Documentation/devicetree/bindings/arm/msm/scss.txt
>>
>> diff --git a/Documentation/devicetree/bindings/arm/cpus.txt b/Documentation/devicetree/bindings/arm/cpus.txt
>> index f32494d..327aad2 100644
>> --- a/Documentation/devicetree/bindings/arm/cpus.txt
>> +++ b/Documentation/devicetree/bindings/arm/cpus.txt
>> @@ -44,6 +44,12 @@ For the ARM architecture every CPU node must contain the following properties:
>>   		"marvell,mohawk"
>>   		"marvell,xsc3"
>>   		"marvell,xscale"
>> +		"qcom,scorpion"
>> +- enable-method: Specifies the method used to enable or take the secondary cores
>> +		 out of reset. This allows different reset sequence for
>> +		 different types of cpus.
>> +		 This should be one of:
>> +		 "qcom,scss"
> While I'd certainly like to see a move to using enable-method for
> 32-bit, I think this needs a bit more thought:
>
> What does "qcom,scss" mean, precisely? It seems to be that we poke some
> registers in a "qcom,scss" device. I think we need to document
> enable-methods *very* carefully (and we need to do that for 64-bit as
> well with psci), as it's very likely there'll be subtle
> incompatibilities between platforms, especially if firmware is involved.
> We should try to leaves as little room for error as possible.
Yes qcom,scss implies poking registers in the qcom,scss device. How 
could I make that more clear in the documentation ?

>
> I don't want to see this devolve into meaning "whatever the Linux driver
> for this happens to do at the current point in time", as that just leads
> to breakage as we have no clear distinction between actual requirements
> and implementation details.
>
> Given we already have platforms without an enable-method, we can't make
> it a required property either -- those platforms are already booting by
> figuring out an enable method from the platform's compatible string.
So, you recommend we continue to using the platform compatible string as 
well ?
We currently don't have a perfect method to use enable-method in generic 
code. More on this below...
>
> With PSCI, enable-method also implies a method for disabling a
> particular CPU, so it would be nice for the description to cover this.
>
>>   
>>   Example:
>>   
>> diff --git a/Documentation/devicetree/bindings/arm/msm/scss.txt b/Documentation/devicetree/bindings/arm/msm/scss.txt
>> new file mode 100644
>> index 0000000..21c3e26
>> --- /dev/null
>> +++ b/Documentation/devicetree/bindings/arm/msm/scss.txt
>> @@ -0,0 +1,15 @@
>> +* SCSS - Scorpion Sub-system
>> +
>> +Properties
>> +
>> +- compatible : Should contain "qcom,scss".
>> +
>> +- reg: Specifies the base address for the SCSS registers used for
>> +       booting up secondary cores.
>> +
>> +Example:
>> +
>> +	scss at 902000 {
>> +		compatible = "qcom,scss";
>> +		reg = <0x00902000 0x2000>;
>> +	};
>> diff --git a/arch/arm/boot/dts/msm8660-surf.dts b/arch/arm/boot/dts/msm8660-surf.dts
>> index cdc010e..203e51a 100644
>> --- a/arch/arm/boot/dts/msm8660-surf.dts
>> +++ b/arch/arm/boot/dts/msm8660-surf.dts
>> @@ -7,6 +7,22 @@
>>   	compatible = "qcom,msm8660-surf", "qcom,msm8660";
>>   	interrupt-parent = <&intc>;
>>   
>> +	cpus {
>> +		#address-cells = <1>;
>> +		#size-cells = <0>;
>> +		compatible = "qcom,scorpion";
>> +		device_type = "cpu";
>> +		enable-method = "qcom,scss";
>> +
>> +		cpu at 0 {
>> +			reg = <0>;
>> +		};
>> +
>> +		cpu at 1 {
>> +			reg = <1>;
>> +		};
>> +	};
>> +
> I *really* like moving the common properties out into the /cpus node --
> ePAPR suggests it, it's less error prone, and it takes up less space.
> However, I'm not sure our generic/arch code handles it properly, and I
> think we need to audit that before we can start writing dts that depend
> on it. I'd be happy to be wrong here if anyone can correct me. :)
>
> We probably need to factor out the way we read properties for cpu nodes,
> falling back to ones present in /cpus if not present. There's already a
> lot of pain getting the node for a logical (rather than physical) cpu
> id.
>
> Sudeep recently factored out finding the cpu node for a logical cpu id
> [1,2] in generic code with a per-arch callback, it shouldn't be too hard
> to have shims around that to read (optionally) common properties.
>
> [...]
>
>> -static void prepare_cold_cpu(unsigned int cpu)
>> +static int scorpion_release_secondary(void)
>>   {
>> -	int ret;
>> -	ret = scm_set_boot_addr(virt_to_phys(secondary_startup),
>> -				SCM_FLAG_COLDBOOT_CPU1);
>> -	if (ret == 0) {
>> -		void __iomem *sc1_base_ptr;
>> -		sc1_base_ptr = ioremap_nocache(0x00902000, SZ_4K*2);
>> -		if (sc1_base_ptr) {
>> -			writel(0, sc1_base_ptr + VDD_SC1_ARRAY_CLAMP_GFS_CTL);
>> -			writel(0, sc1_base_ptr + SCSS_CPU1CORE_RESET);
>> -			writel(3, sc1_base_ptr + SCSS_DBG_STATUS_CORE_PWRDUP);
>> -			iounmap(sc1_base_ptr);
>> -		}
>> -	} else
>> -		printk(KERN_DEBUG "Failed to set secondary core boot "
>> -				  "address\n");
>> +	void __iomem *sc1_base_ptr;
>> +	struct device_node *dn = NULL;
>> +
>> +	dn = of_find_compatible_node(dn, NULL, "qcom,scss");
>> +	if (!dn) {
>> +		pr_err("%s: Missing scss node in device tree\n", __func__);
>> +		return -ENXIO;
>> +	}
>> +
>> +	sc1_base_ptr = of_iomap(dn, 0);
>> +	if (sc1_base_ptr) {
>> +		writel_relaxed(0, sc1_base_ptr + VDD_SC1_ARRAY_CLAMP_GFS_CTL);
>> +		writel_relaxed(0, sc1_base_ptr + SCSS_CPU1CORE_RESET);
>> +		writel_relaxed(3, sc1_base_ptr + SCSS_DBG_STATUS_CORE_PWRDUP);
>> +		mb();
>> +		iounmap(sc1_base_ptr);
> Does this boot *all* secondary CPUs directly into the kernel? Is that
> safe (e.g. if the kernel supports fewer CPUs than are physically
> present)?
Im not sure I understand the concern with safety here. The CPU for which 
the release_secondary will be called will be taken out of reset with 
this sequence.

>
> Is this a one-time thing, or are we able to dynamically hotplug CPUs? If
> the latter, the map/unmap looks odd to me.

This is a one-time thing done for each CPU that's specified in the 
device tree (or based on the command line over-ride).
>> +	} else {
>> +		return -ENOMEM;
>> +	}
>> +
>> +	return 0;
>>   }
>>   
>> -static int msm_boot_secondary(unsigned int cpu, struct task_struct *idle)
>> +static DEFINE_PER_CPU(int, cold_boot_done);
>> +
>> +static void boot_cold_cpu(unsigned int cpu)
>>   {
>> -	static int cold_boot_done;
>> +	const char *enable_method;
>> +	struct device_node *dn = NULL;
>>   
>> -	/* Only need to bring cpu out of reset this way once */
>> -	if (cold_boot_done == false) {
>> -		prepare_cold_cpu(cpu);
>> -		cold_boot_done = true;
>> +	dn = of_find_node_by_name(dn, "cpus");
>> +	if (!dn) {
>> +		pr_err("%s: Missing node cpus in device tree\n", __func__);
>> +		return;
>>   	}
>>   
>> +	enable_method = of_get_property(dn, "enable-method", NULL);
> Please factor this out from the platform code.
>
> If we're going to use enable-method, it should be decoupled from
> platform code -- common code should parse it to find the appropriate
> handler. Also, we *must* support having an enable-method per-cpu as KVM
> does for PSCI (though I definitely would like support for shared
> properties as mentioned above).
Currently with most platforms, smp.c calls into the boot_secondary 
function which decides which cpu it is
and then calls the appropriate function. This is because smp ops allow 
only 1 callback function to be registered...
Hence, using the enable-method in platform specific code works.

If we create a generic property should it mandate having generic code 
handle that ?
I currently don't have a good way of incorporating enable-method in 
generic code as it would mean to establish a mechanism to
associate the enable-method string with cpu specific boot_secondary 
functions.
You have an approach in mind that I can try ?

>> +	if (!enable_method) {
>> +			pr_err("%s: cpus node is missing enable-method property\n",
>> +					__func__);
>> +	} else if (!strcmp(enable_method, "qcom,scss")) {
>> +		if (per_cpu(cold_boot_done, cpu) == false) {
>> +			scorpion_release_secondary();
>> +			per_cpu(cold_boot_done, cpu) = true;
>> +		}
>> +	} else {
>> +		pr_err("%s: Invalid enable-method property: %s\n",
>> +				__func__, enable_method);
>> +	}
>> +}
>> +
>> +static int msm_boot_secondary(unsigned int cpu, struct task_struct *idle)
>> +{
>> +	boot_cold_cpu(cpu);
>> +
>>   	/*
>>   	 * set synchronisation state between this boot processor
>>   	 * and the secondary one
>> @@ -118,8 +148,28 @@ static void __init msm_smp_init_cpus(void)
>>   		set_cpu_possible(i, true);
>>   }
>>   
>> +static const int cold_boot_flags[] __initconst = {
>> +	0,
>> +	SCM_FLAG_COLDBOOT_CPU1,
>> +};
> So we only ever support booting two CPUs?
The next patch which adds support for a quadcore chip, adds more flags.
>
> Is the mapping of MPIDR to register bits arbitrary? Or do we know what
> they would be for four CPUs, four clusters, and beyond?
Four cpus.

>
>> +
>>   static void __init msm_smp_prepare_cpus(unsigned int max_cpus)
>>   {
>> +	int cpu, map;
>> +	unsigned int flags = 0;
>> +
>> +	for_each_present_cpu(cpu) {
>> +		map = cpu_logical_map(cpu);
>> +		if (map > ARRAY_SIZE(cold_boot_flags)) {
>> +			set_cpu_present(cpu, false);
>> +			__WARN();
>> +			continue;
>> +		}
>> +		flags |= cold_boot_flags[map];
> It's a shame that this leaves a window where some CPUs seem bootable,
> but aren't (though I can't offer a better way of handling this given we
> have dts without enable-method properties).
Any CPU that seems bootable and is defined in DT, should be bootable.
The cold_boot_flags are just a mechanism to set appropriate values for 
the scm call and
they aren't used as a method to disallow CPUs from being bootable and 
hence the __WARN() if this is done incorrectly.

>
>> +	}
>> +
>> +	if (scm_set_boot_addr(virt_to_phys(secondary_startup), flags))
>> +		pr_warn("Failed to set CPU boot address\n");
>>   }
>>   
>>   struct smp_operations msm_smp_ops __initdata = {
>> -- 
>> The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
>> hosted by The Linux Foundation
>>
> Thanks,
> Mark
>
> [1] http://lists.infradead.org/pipermail/linux-arm-kernel/2013-July/184150.html
> [2] http://lists.infradead.org/pipermail/linux-arm-kernel/2013-August/189619.html
> --
> To unsubscribe from this list: send the line "unsubscribe linux-arm-msm" in
> the body of a message to majordomo at vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html


Thanks,
Rohit Vaswani

-- 
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum, hosted by The Linux Foundation

Re: [RESEND PATCH 4/4] ARM: msm: Add support for 8974 SMP

[Rohit Vaswani]

On 8/12/2013 9:39 AM, Mark Rutland wrote:
> On Fri, Aug 02, 2013 at 03:15:25AM +0100, Rohit Vaswani wrote:
>> Add the cpus bindings and the Kraitv2 release sequence
>> to make SMP work for 2 cores on MSM8974.
>>
>> Signed-off-by: Rohit Vaswani <rvaswani@codeaurora.org>
>> ---
>>   Documentation/devicetree/bindings/arm/cpus.txt |  1 +
>>   arch/arm/boot/dts/msm8974.dts                  | 23 ++++++++
>>   arch/arm/mach-msm/board-dt-8974.c              |  3 +
>>   arch/arm/mach-msm/platsmp.c                    | 79 ++++++++++++++++++++++++++
>>   4 files changed, 106 insertions(+)
>>
>> diff --git a/Documentation/devicetree/bindings/arm/cpus.txt b/Documentation/devicetree/bindings/arm/cpus.txt
>> index 1132eac..7c3c677 100644
>> --- a/Documentation/devicetree/bindings/arm/cpus.txt
>> +++ b/Documentation/devicetree/bindings/arm/cpus.txt
>> @@ -52,6 +52,7 @@ For the ARM architecture every CPU node must contain the following properties:
>>   		 This should be one of:
>>   		 "qcom,scss"
>>   		 "qcom,kpssv1"
>> +		 "qcom,kpssv2"
> I guess I should've looked at the whole series before responding, that
> answers my prior question about what variation we expect.
>
>>   
>>   Example:
>>   
>> diff --git a/arch/arm/boot/dts/msm8974.dts b/arch/arm/boot/dts/msm8974.dts
>> index c31c097..ef35a9b 100644
>> --- a/arch/arm/boot/dts/msm8974.dts
>> +++ b/arch/arm/boot/dts/msm8974.dts
>> @@ -7,6 +7,22 @@
>>   	compatible = "qcom,msm8974";
>>   	interrupt-parent = <&intc>;
>>   
>> +	cpus {
>> +		#address-cells = <1>;
>> +		#size-cells = <0>;
>> +		compatible = "qcom,krait";
>> +		device_type = "cpu";
>> +		enable-method = "qcom,kpssv2";
>> +
>> +		cpu@0 {
>> +			reg = <0>;
>> +		};
>> +
>> +		cpu@1 {
>> +			reg = <1>;
>> +		};
>> +	};
>> +
>>   	intc: interrupt-controller@f9000000 {
>>   		compatible = "qcom,msm-qgic2";
>>   		interrupt-controller;
>> @@ -23,4 +39,11 @@
>>   			     <1 1 0xf08>;
>>   		clock-frequency = <19200000>;
>>   	};
>> +
>> +	kpss@f9012000 {
>> +		compatible = "qcom,kpss";
>> +		reg = <0xf9012000 0x1000>,
>> +		      <0xf9088000 0x1000>,
>> +		      <0xf9098000 0x1000>;
>> +	};
> In the previous examples, the number of CPUs was equal to the number of
> kpss reg values. Why does it differ here. Either:
>
> * We always have the extra regsiter here, and it should be described
>    even if we don't use it.
>
> * It's a different hardware block, and needs a more specific
>    compatible string.
>
> Eitehr way this strengthens my feeling that we need to define the linkage
> from a CPU to the portion of the kpss which affects it.
Will add documentation for each of the registers. We have one for each 
CPU and one within the KPSS (Krait Processor Sub-System) e.g the L2 saw 
base in this case.
>
>>   };
>> diff --git a/arch/arm/mach-msm/board-dt-8974.c b/arch/arm/mach-msm/board-dt-8974.c
>> index d7f84f2..06119f9 100644
>> --- a/arch/arm/mach-msm/board-dt-8974.c
>> +++ b/arch/arm/mach-msm/board-dt-8974.c
>> @@ -13,11 +13,14 @@
>>   #include <linux/of_platform.h>
>>   #include <asm/mach/arch.h>
>>   
>> +#include "common.h"
>> +
>>   static const char * const msm8974_dt_match[] __initconst = {
>>   	"qcom,msm8974",
>>   	NULL
>>   };
>>   
>>   DT_MACHINE_START(MSM8974_DT, "Qualcomm MSM (Flattened Device Tree)")
>> +	.smp = smp_ops(msm_smp_ops),
>>   	.dt_compat = msm8974_dt_match,
>>   MACHINE_END
>> diff --git a/arch/arm/mach-msm/platsmp.c b/arch/arm/mach-msm/platsmp.c
>> index 82eb079..0fdae69 100644
>> --- a/arch/arm/mach-msm/platsmp.c
>> +++ b/arch/arm/mach-msm/platsmp.c
>> @@ -124,6 +124,80 @@ static int msm8960_release_secondary(unsigned int cpu)
>>   	return 0;
>>   }
>>   
>> +static int msm8974_release_secondary(unsigned int cpu)
>> +{
>> +	void __iomem *reg;
>> +	void __iomem *l2_saw_base;
>> +	struct device_node *dn = NULL;
>> +	unsigned apc_pwr_gate_ctl = 0x14;
>> +	unsigned reg_val;
>> +
>> +	if (cpu == 0 || cpu >= num_possible_cpus())
>> +		return -EINVAL;
>> +
>> +	dn = of_find_compatible_node(dn, NULL, "qcom,kpss");
>> +	if (!dn) {
>> +		pr_err("%s : Missing kpss node from device tree\n", __func__);
>> +		return -ENXIO;
>> +	}
>> +
>> +	reg = of_iomap(dn, cpu+1);
> This looks very fishy given the prior patch being one off from this.
> why is reg[0] now different?
>
>> +	if (!reg)
>> +		return -ENOMEM;
>> +
>> +	pr_debug("Starting secondary CPU %d\n", cpu);
>> +
>> +	/* Turn on the BHS, turn off LDO Bypass and power down LDO */
>> +	reg_val =  0x403f0001;
> Magic number?
It represents the comment above it. It didnt seem clean to define 4 
different offsets with #defines within a single register for the purpose 
of 1 write.
>
>> +	writel_relaxed(reg_val, reg + apc_pwr_gate_ctl);
>> +
>> +	/* complete the above write before the delay */
>> +	mb();
> Use writel?
>
>> +	/* wait for the bhs to settle */
>> +	udelay(1);
>> +
>> +	/* Turn on BHS segments */
>> +	reg_val |= 0x3f << 1;
>> +	writel_relaxed(reg_val, reg + apc_pwr_gate_ctl);
>> +
>> +	/* complete the above write before the delay */
>> +	mb();
> Use writel again?
>
>> +	 /* wait for the bhs to settle */
>> +	udelay(1);
>> +
>> +	/* Finally turn on the bypass so that BHS supplies power */
>> +	reg_val |= 0x3f << 8;
>> +	writel_relaxed(reg_val, reg + apc_pwr_gate_ctl);
>> +
>> +	/* enable max phases */
>> +	l2_saw_base = of_iomap(dn, 0);
>> +	if (!l2_saw_base) {
>> +		return -ENOMEM;
>> +	}
> What?
>
> You've just lost your only reference to the mapping in reg.
>
> Why do you not do this at the start, before poking everything else? Even
> better, do it at probe time and fail once rather than for each CPU you
> have no chance of bringing up.
Will do.

>
> [...]
>>   static void boot_cold_cpu(unsigned int cpu)
>> @@ -151,6 +225,11 @@ static void boot_cold_cpu(unsigned int cpu)
>>   			msm8960_release_secondary(cpu);
>>   			per_cpu(cold_boot_done, cpu) = true;
>>   		}
>> +	} else if (!strcmp(enable_method, "qcom,kpssv2")) {
>> +		if (per_cpu(cold_boot_done, cpu) == false) {
>> +			msm8974_release_secondary(cpu);
>> +			per_cpu(cold_boot_done, cpu) = true;
>> +		}
>>   	} else {
>>   		pr_err("%s: Invalid enable-method property: %s\n",
>>   				__func__, enable_method);
> The enable-method parsing really should be moved out to common code. We
> could make it scan the enable-method if a machine has no smp ops (which
> is more general than the PSCI fallback that's been suggested before).
But we have smp ops like every other SoC. I might need to go back to the 
drawing board for incorporating enable-method into generic code.
Any suggestions are appreciated.
If enable-method seems cumbersome to be enforced for every SoC, I would 
be comfortable using the cpu compatible string as you suggested in the 
previous patch.
>
> Thanks,
> Mark.
>
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel


Thanks,
Rohit Vaswani

-- 
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum, hosted by The Linux Foundation

Re: [RESEND PATCH 4/4] ARM: msm: Add support for 8974 SMP

[Rohit Vaswani]

On 8/12/2013 9:39 AM, Mark Rutland wrote:
> On Fri, Aug 02, 2013 at 03:15:25AM +0100, Rohit Vaswani wrote:
>> Add the cpus bindings and the Kraitv2 release sequence
>> to make SMP work for 2 cores on MSM8974.
>>
>> Signed-off-by: Rohit Vaswani <rvaswani@codeaurora.org>
>> ---
>>   Documentation/devicetree/bindings/arm/cpus.txt |  1 +
>>   arch/arm/boot/dts/msm8974.dts                  | 23 ++++++++
>>   arch/arm/mach-msm/board-dt-8974.c              |  3 +
>>   arch/arm/mach-msm/platsmp.c                    | 79 ++++++++++++++++++++++++++
>>   4 files changed, 106 insertions(+)
>>
>> diff --git a/Documentation/devicetree/bindings/arm/cpus.txt b/Documentation/devicetree/bindings/arm/cpus.txt
>> index 1132eac..7c3c677 100644
>> --- a/Documentation/devicetree/bindings/arm/cpus.txt
>> +++ b/Documentation/devicetree/bindings/arm/cpus.txt
>> @@ -52,6 +52,7 @@ For the ARM architecture every CPU node must contain the following properties:
>>   		 This should be one of:
>>   		 "qcom,scss"
>>   		 "qcom,kpssv1"
>> +		 "qcom,kpssv2"
> I guess I should've looked at the whole series before responding, that
> answers my prior question about what variation we expect.
>
>>   
>>   Example:
>>   
>> diff --git a/arch/arm/boot/dts/msm8974.dts b/arch/arm/boot/dts/msm8974.dts
>> index c31c097..ef35a9b 100644
>> --- a/arch/arm/boot/dts/msm8974.dts
>> +++ b/arch/arm/boot/dts/msm8974.dts
>> @@ -7,6 +7,22 @@
>>   	compatible = "qcom,msm8974";
>>   	interrupt-parent = <&intc>;
>>   
>> +	cpus {
>> +		#address-cells = <1>;
>> +		#size-cells = <0>;
>> +		compatible = "qcom,krait";
>> +		device_type = "cpu";
>> +		enable-method = "qcom,kpssv2";
>> +
>> +		cpu@0 {
>> +			reg = <0>;
>> +		};
>> +
>> +		cpu@1 {
>> +			reg = <1>;
>> +		};
>> +	};
>> +
>>   	intc: interrupt-controller@f9000000 {
>>   		compatible = "qcom,msm-qgic2";
>>   		interrupt-controller;
>> @@ -23,4 +39,11 @@
>>   			     <1 1 0xf08>;
>>   		clock-frequency = <19200000>;
>>   	};
>> +
>> +	kpss@f9012000 {
>> +		compatible = "qcom,kpss";
>> +		reg = <0xf9012000 0x1000>,
>> +		      <0xf9088000 0x1000>,
>> +		      <0xf9098000 0x1000>;
>> +	};
> In the previous examples, the number of CPUs was equal to the number of
> kpss reg values. Why does it differ here. Either:
>
> * We always have the extra regsiter here, and it should be described
>    even if we don't use it.
>
> * It's a different hardware block, and needs a more specific
>    compatible string.
>
> Eitehr way this strengthens my feeling that we need to define the linkage
> from a CPU to the portion of the kpss which affects it.
Will add documentation for each of the registers. We have one for each 
CPU and one within the KPSS (Krait Processor Sub-System) e.g the L2 saw 
base in this case.
>
>>   };
>> diff --git a/arch/arm/mach-msm/board-dt-8974.c b/arch/arm/mach-msm/board-dt-8974.c
>> index d7f84f2..06119f9 100644
>> --- a/arch/arm/mach-msm/board-dt-8974.c
>> +++ b/arch/arm/mach-msm/board-dt-8974.c
>> @@ -13,11 +13,14 @@
>>   #include <linux/of_platform.h>
>>   #include <asm/mach/arch.h>
>>   
>> +#include "common.h"
>> +
>>   static const char * const msm8974_dt_match[] __initconst = {
>>   	"qcom,msm8974",
>>   	NULL
>>   };
>>   
>>   DT_MACHINE_START(MSM8974_DT, "Qualcomm MSM (Flattened Device Tree)")
>> +	.smp = smp_ops(msm_smp_ops),
>>   	.dt_compat = msm8974_dt_match,
>>   MACHINE_END
>> diff --git a/arch/arm/mach-msm/platsmp.c b/arch/arm/mach-msm/platsmp.c
>> index 82eb079..0fdae69 100644
>> --- a/arch/arm/mach-msm/platsmp.c
>> +++ b/arch/arm/mach-msm/platsmp.c
>> @@ -124,6 +124,80 @@ static int msm8960_release_secondary(unsigned int cpu)
>>   	return 0;
>>   }
>>   
>> +static int msm8974_release_secondary(unsigned int cpu)
>> +{
>> +	void __iomem *reg;
>> +	void __iomem *l2_saw_base;
>> +	struct device_node *dn = NULL;
>> +	unsigned apc_pwr_gate_ctl = 0x14;
>> +	unsigned reg_val;
>> +
>> +	if (cpu == 0 || cpu >= num_possible_cpus())
>> +		return -EINVAL;
>> +
>> +	dn = of_find_compatible_node(dn, NULL, "qcom,kpss");
>> +	if (!dn) {
>> +		pr_err("%s : Missing kpss node from device tree\n", __func__);
>> +		return -ENXIO;
>> +	}
>> +
>> +	reg = of_iomap(dn, cpu+1);
> This looks very fishy given the prior patch being one off from this.
> why is reg[0] now different?
>
>> +	if (!reg)
>> +		return -ENOMEM;
>> +
>> +	pr_debug("Starting secondary CPU %d\n", cpu);
>> +
>> +	/* Turn on the BHS, turn off LDO Bypass and power down LDO */
>> +	reg_val =  0x403f0001;
> Magic number?
It represents the comment above it. It didnt seem clean to define 4 
different offsets with #defines within a single register for the purpose 
of 1 write.
>
>> +	writel_relaxed(reg_val, reg + apc_pwr_gate_ctl);
>> +
>> +	/* complete the above write before the delay */
>> +	mb();
> Use writel?
>
>> +	/* wait for the bhs to settle */
>> +	udelay(1);
>> +
>> +	/* Turn on BHS segments */
>> +	reg_val |= 0x3f << 1;
>> +	writel_relaxed(reg_val, reg + apc_pwr_gate_ctl);
>> +
>> +	/* complete the above write before the delay */
>> +	mb();
> Use writel again?
>
>> +	 /* wait for the bhs to settle */
>> +	udelay(1);
>> +
>> +	/* Finally turn on the bypass so that BHS supplies power */
>> +	reg_val |= 0x3f << 8;
>> +	writel_relaxed(reg_val, reg + apc_pwr_gate_ctl);
>> +
>> +	/* enable max phases */
>> +	l2_saw_base = of_iomap(dn, 0);
>> +	if (!l2_saw_base) {
>> +		return -ENOMEM;
>> +	}
> What?
>
> You've just lost your only reference to the mapping in reg.
>
> Why do you not do this at the start, before poking everything else? Even
> better, do it at probe time and fail once rather than for each CPU you
> have no chance of bringing up.
Will do.

>
> [...]
>>   static void boot_cold_cpu(unsigned int cpu)
>> @@ -151,6 +225,11 @@ static void boot_cold_cpu(unsigned int cpu)
>>   			msm8960_release_secondary(cpu);
>>   			per_cpu(cold_boot_done, cpu) = true;
>>   		}
>> +	} else if (!strcmp(enable_method, "qcom,kpssv2")) {
>> +		if (per_cpu(cold_boot_done, cpu) == false) {
>> +			msm8974_release_secondary(cpu);
>> +			per_cpu(cold_boot_done, cpu) = true;
>> +		}
>>   	} else {
>>   		pr_err("%s: Invalid enable-method property: %s\n",
>>   				__func__, enable_method);
> The enable-method parsing really should be moved out to common code. We
> could make it scan the enable-method if a machine has no smp ops (which
> is more general than the PSCI fallback that's been suggested before).
But we have smp ops like every other SoC. I might need to go back to the 
drawing board for incorporating enable-method into generic code.
Any suggestions are appreciated.
If enable-method seems cumbersome to be enforced for every SoC, I would 
be comfortable using the cpu compatible string as you suggested in the 
previous patch.
>
> Thanks,
> Mark.
>
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel


Thanks,
Rohit Vaswani

-- 
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum, hosted by The Linux Foundation

[RESEND PATCH 4/4] ARM: msm: Add support for 8974 SMP

[Rohit Vaswani]

On 8/12/2013 9:39 AM, Mark Rutland wrote:
> On Fri, Aug 02, 2013 at 03:15:25AM +0100, Rohit Vaswani wrote:
>> Add the cpus bindings and the Kraitv2 release sequence
>> to make SMP work for 2 cores on MSM8974.
>>
>> Signed-off-by: Rohit Vaswani <rvaswani@codeaurora.org>
>> ---
>>   Documentation/devicetree/bindings/arm/cpus.txt |  1 +
>>   arch/arm/boot/dts/msm8974.dts                  | 23 ++++++++
>>   arch/arm/mach-msm/board-dt-8974.c              |  3 +
>>   arch/arm/mach-msm/platsmp.c                    | 79 ++++++++++++++++++++++++++
>>   4 files changed, 106 insertions(+)
>>
>> diff --git a/Documentation/devicetree/bindings/arm/cpus.txt b/Documentation/devicetree/bindings/arm/cpus.txt
>> index 1132eac..7c3c677 100644
>> --- a/Documentation/devicetree/bindings/arm/cpus.txt
>> +++ b/Documentation/devicetree/bindings/arm/cpus.txt
>> @@ -52,6 +52,7 @@ For the ARM architecture every CPU node must contain the following properties:
>>   		 This should be one of:
>>   		 "qcom,scss"
>>   		 "qcom,kpssv1"
>> +		 "qcom,kpssv2"
> I guess I should've looked at the whole series before responding, that
> answers my prior question about what variation we expect.
>
>>   
>>   Example:
>>   
>> diff --git a/arch/arm/boot/dts/msm8974.dts b/arch/arm/boot/dts/msm8974.dts
>> index c31c097..ef35a9b 100644
>> --- a/arch/arm/boot/dts/msm8974.dts
>> +++ b/arch/arm/boot/dts/msm8974.dts
>> @@ -7,6 +7,22 @@
>>   	compatible = "qcom,msm8974";
>>   	interrupt-parent = <&intc>;
>>   
>> +	cpus {
>> +		#address-cells = <1>;
>> +		#size-cells = <0>;
>> +		compatible = "qcom,krait";
>> +		device_type = "cpu";
>> +		enable-method = "qcom,kpssv2";
>> +
>> +		cpu at 0 {
>> +			reg = <0>;
>> +		};
>> +
>> +		cpu at 1 {
>> +			reg = <1>;
>> +		};
>> +	};
>> +
>>   	intc: interrupt-controller at f9000000 {
>>   		compatible = "qcom,msm-qgic2";
>>   		interrupt-controller;
>> @@ -23,4 +39,11 @@
>>   			     <1 1 0xf08>;
>>   		clock-frequency = <19200000>;
>>   	};
>> +
>> +	kpss at f9012000 {
>> +		compatible = "qcom,kpss";
>> +		reg = <0xf9012000 0x1000>,
>> +		      <0xf9088000 0x1000>,
>> +		      <0xf9098000 0x1000>;
>> +	};
> In the previous examples, the number of CPUs was equal to the number of
> kpss reg values. Why does it differ here. Either:
>
> * We always have the extra regsiter here, and it should be described
>    even if we don't use it.
>
> * It's a different hardware block, and needs a more specific
>    compatible string.
>
> Eitehr way this strengthens my feeling that we need to define the linkage
> from a CPU to the portion of the kpss which affects it.
Will add documentation for each of the registers. We have one for each 
CPU and one within the KPSS (Krait Processor Sub-System) e.g the L2 saw 
base in this case.
>
>>   };
>> diff --git a/arch/arm/mach-msm/board-dt-8974.c b/arch/arm/mach-msm/board-dt-8974.c
>> index d7f84f2..06119f9 100644
>> --- a/arch/arm/mach-msm/board-dt-8974.c
>> +++ b/arch/arm/mach-msm/board-dt-8974.c
>> @@ -13,11 +13,14 @@
>>   #include <linux/of_platform.h>
>>   #include <asm/mach/arch.h>
>>   
>> +#include "common.h"
>> +
>>   static const char * const msm8974_dt_match[] __initconst = {
>>   	"qcom,msm8974",
>>   	NULL
>>   };
>>   
>>   DT_MACHINE_START(MSM8974_DT, "Qualcomm MSM (Flattened Device Tree)")
>> +	.smp = smp_ops(msm_smp_ops),
>>   	.dt_compat = msm8974_dt_match,
>>   MACHINE_END
>> diff --git a/arch/arm/mach-msm/platsmp.c b/arch/arm/mach-msm/platsmp.c
>> index 82eb079..0fdae69 100644
>> --- a/arch/arm/mach-msm/platsmp.c
>> +++ b/arch/arm/mach-msm/platsmp.c
>> @@ -124,6 +124,80 @@ static int msm8960_release_secondary(unsigned int cpu)
>>   	return 0;
>>   }
>>   
>> +static int msm8974_release_secondary(unsigned int cpu)
>> +{
>> +	void __iomem *reg;
>> +	void __iomem *l2_saw_base;
>> +	struct device_node *dn = NULL;
>> +	unsigned apc_pwr_gate_ctl = 0x14;
>> +	unsigned reg_val;
>> +
>> +	if (cpu == 0 || cpu >= num_possible_cpus())
>> +		return -EINVAL;
>> +
>> +	dn = of_find_compatible_node(dn, NULL, "qcom,kpss");
>> +	if (!dn) {
>> +		pr_err("%s : Missing kpss node from device tree\n", __func__);
>> +		return -ENXIO;
>> +	}
>> +
>> +	reg = of_iomap(dn, cpu+1);
> This looks very fishy given the prior patch being one off from this.
> why is reg[0] now different?
>
>> +	if (!reg)
>> +		return -ENOMEM;
>> +
>> +	pr_debug("Starting secondary CPU %d\n", cpu);
>> +
>> +	/* Turn on the BHS, turn off LDO Bypass and power down LDO */
>> +	reg_val =  0x403f0001;
> Magic number?
It represents the comment above it. It didnt seem clean to define 4 
different offsets with #defines within a single register for the purpose 
of 1 write.
>
>> +	writel_relaxed(reg_val, reg + apc_pwr_gate_ctl);
>> +
>> +	/* complete the above write before the delay */
>> +	mb();
> Use writel?
>
>> +	/* wait for the bhs to settle */
>> +	udelay(1);
>> +
>> +	/* Turn on BHS segments */
>> +	reg_val |= 0x3f << 1;
>> +	writel_relaxed(reg_val, reg + apc_pwr_gate_ctl);
>> +
>> +	/* complete the above write before the delay */
>> +	mb();
> Use writel again?
>
>> +	 /* wait for the bhs to settle */
>> +	udelay(1);
>> +
>> +	/* Finally turn on the bypass so that BHS supplies power */
>> +	reg_val |= 0x3f << 8;
>> +	writel_relaxed(reg_val, reg + apc_pwr_gate_ctl);
>> +
>> +	/* enable max phases */
>> +	l2_saw_base = of_iomap(dn, 0);
>> +	if (!l2_saw_base) {
>> +		return -ENOMEM;
>> +	}
> What?
>
> You've just lost your only reference to the mapping in reg.
>
> Why do you not do this at the start, before poking everything else? Even
> better, do it at probe time and fail once rather than for each CPU you
> have no chance of bringing up.
Will do.

>
> [...]
>>   static void boot_cold_cpu(unsigned int cpu)
>> @@ -151,6 +225,11 @@ static void boot_cold_cpu(unsigned int cpu)
>>   			msm8960_release_secondary(cpu);
>>   			per_cpu(cold_boot_done, cpu) = true;
>>   		}
>> +	} else if (!strcmp(enable_method, "qcom,kpssv2")) {
>> +		if (per_cpu(cold_boot_done, cpu) == false) {
>> +			msm8974_release_secondary(cpu);
>> +			per_cpu(cold_boot_done, cpu) = true;
>> +		}
>>   	} else {
>>   		pr_err("%s: Invalid enable-method property: %s\n",
>>   				__func__, enable_method);
> The enable-method parsing really should be moved out to common code. We
> could make it scan the enable-method if a machine has no smp ops (which
> is more general than the PSCI fallback that's been suggested before).
But we have smp ops like every other SoC. I might need to go back to the 
drawing board for incorporating enable-method into generic code.
Any suggestions are appreciated.
If enable-method seems cumbersome to be enforced for every SoC, I would 
be comfortable using the cpu compatible string as you suggested in the 
previous patch.
>
> Thanks,
> Mark.
>
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel


Thanks,
Rohit Vaswani

-- 
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum, hosted by The Linux Foundation

Re: [PATCH 3/4] ARM: msm: Add SMP support for 8960

[Rohit Vaswani]

On 8/2/2013 8:43 AM, Kumar Gala wrote:
> On Aug 1, 2013, at 9:15 PM, Rohit Vaswani wrote:
>
>> Add the cpus bindings and the Krait release sequence
>> to make SMP work for MSM8960
>>
>> Signed-off-by: Rohit Vaswani <rvaswani@codeaurora.org>
>> ---
>> Documentation/devicetree/bindings/arm/cpus.txt     |  2 +
>> Documentation/devicetree/bindings/arm/msm/kpss.txt | 16 ++++++
>> arch/arm/boot/dts/msm8960-cdp.dts                  | 22 +++++++++
>> arch/arm/mach-msm/platsmp.c                        | 57 ++++++++++++++++++++++
>> arch/arm/mach-msm/scm-boot.h                       |  8 +--
>> 5 files changed, 102 insertions(+), 3 deletions(-)
>> create mode 100644 Documentation/devicetree/bindings/arm/msm/kpss.txt
>>
>> diff --git a/Documentation/devicetree/bindings/arm/cpus.txt b/Documentation/devicetree/bindings/arm/cpus.txt
>> index 327aad2..1132eac 100644
>> --- a/Documentation/devicetree/bindings/arm/cpus.txt
>> +++ b/Documentation/devicetree/bindings/arm/cpus.txt
>> @@ -45,11 +45,13 @@ For the ARM architecture every CPU node must contain the following properties:
>> 		"marvell,xsc3"
>> 		"marvell,xscale"
>> 		"qcom,scorpion"
>> +		"qcom,krait"
>> - enable-method: Specifies the method used to enable or take the secondary cores
>> 		 out of reset. This allows different reset sequence for
>> 		 different types of cpus.
>> 		 This should be one of:
>> 		 "qcom,scss"
>> +		 "qcom,kpssv1"
>>
>> Example:
>>
>> diff --git a/Documentation/devicetree/bindings/arm/msm/kpss.txt b/Documentation/devicetree/bindings/arm/msm/kpss.txt
>> new file mode 100644
>> index 0000000..7272340
>> --- /dev/null
>> +++ b/Documentation/devicetree/bindings/arm/msm/kpss.txt
>> @@ -0,0 +1,16 @@
>> +* KPSS - Krait Processor Sub-system
>> +
>> +Properties
>> +
>> +- compatible : Should contain "qcom,kpss".
>> +
>> +- reg: Specifies the base address for the KPSS registers used for
>> +       booting up secondary cores.
>> +
>> +Example:
>> +
>> +	kpss@2088000 {
>> +		compatible = "qcom,kpss";
>> +		reg = <0x02088000 0x1000
>> +			0x02098000 0x2000>;
>> +	};
>> diff --git a/arch/arm/boot/dts/msm8960-cdp.dts b/arch/arm/boot/dts/msm8960-cdp.dts
>> index db2060c..8c82d5e 100644
>> --- a/arch/arm/boot/dts/msm8960-cdp.dts
>> +++ b/arch/arm/boot/dts/msm8960-cdp.dts
>> @@ -7,6 +7,22 @@
>> 	compatible = "qcom,msm8960-cdp", "qcom,msm8960";
>> 	interrupt-parent = <&intc>;
>>
>> +	cpus {
>> +		#address-cells = <1>;
>> +		#size-cells = <0>;
>> +		compatible = "qcom,krait";
>> +		device_type = "cpu";
>> +		enable-method = "qcom,kpssv1";
>> +
>> +		cpu@0 {
>> +			reg = <0>;
>> +		};
>> +
>> +		cpu@1 {
>> +			reg = <1>;
>> +		};
>> +	};
>> +
>> 	intc: interrupt-controller@2000000 {
>> 		compatible = "qcom,msm-qgic2";
>> 		interrupt-controller;
>> @@ -37,6 +53,12 @@
>> 		reg = <0xfd510000 0x4000>;
>> 	};
>>
>> +	kpss@2088000 {
>> +		compatible = "qcom,kpss";
>> +		reg = <0x02088000 0x1000
>> +			0x02098000 0x2000>;
>> +	};
>> +
>> 	serial@16440000 {
>> 		compatible = "qcom,msm-hsuart", "qcom,msm-uart";
>> 		reg = <0x16440000 0x1000>,
>> diff --git a/arch/arm/mach-msm/platsmp.c b/arch/arm/mach-msm/platsmp.c
>> index 17022e0..82eb079 100644
>> --- a/arch/arm/mach-msm/platsmp.c
>> +++ b/arch/arm/mach-msm/platsmp.c
>> @@ -74,6 +74,56 @@ static int scorpion_release_secondary(void)
>> 	return 0;
>> }
>>
>> +static int msm8960_release_secondary(unsigned int cpu)
>> +{
>> +	void __iomem *reg;
>> +	struct device_node *dn = NULL;
>> +
>> +	if (cpu == 0 || cpu >= num_possible_cpus())
>> +		return -EINVAL;
>> +
>> +	dn = of_find_compatible_node(dn, NULL, "qcom,kpss");
>> +	if (!dn) {
>> +		pr_err("%s : Missing kpss node from device tree\n", __func__);
>> +		return -ENXIO;
>> +	}
>> +
>> +	reg = of_iomap(dn, cpu);
>> +	if (!reg)
>> +		return -ENOMEM;
>> +
>> +	pr_debug("Starting secondary CPU %d\n", cpu);
>> +
>> +	/* Turn on CPU Rail */
>> +	writel_relaxed(0xA4, reg+0x1014);
> Is there some reason we are using magic numbers for both values and offsets?
Yes. The names are not defined by hardware spec. Would you prefer having 
multiple #defines for each value, register ?
>
>> +	mb();
>> +	udelay(512);
>> +
>> +	/* Krait bring-up sequence */
>> +	writel_relaxed(0x109, reg+0x04);
>> +	writel_relaxed(0x101, reg+0x04);
>> +	mb();
>> +	ndelay(300);
>> +
>> +	writel_relaxed(0x121, reg+0x04);
>> +	mb();
>> +	udelay(2);
>> +
>> +	writel_relaxed(0x120, reg+0x04);
>> +	mb();
>> +	udelay(2);
>> +
>> +	writel_relaxed(0x100, reg+0x04);
>> +	mb();
>> +	udelay(100);
>> +
>> +	writel_relaxed(0x180, reg+0x04);
>> +	mb();
>> +
>> +	iounmap(reg);
>> +	return 0;
>> +}
>> +
>> static DEFINE_PER_CPU(int, cold_boot_done);
>>
>> static void boot_cold_cpu(unsigned int cpu)
>> @@ -96,6 +146,11 @@ static void boot_cold_cpu(unsigned int cpu)
>> 			scorpion_release_secondary();
>> 			per_cpu(cold_boot_done, cpu) = true;
>> 		}
>> +	} else if (!strcmp(enable_method, "qcom,kpssv1")) {
>> +		if (per_cpu(cold_boot_done, cpu) == false) {
>> +			msm8960_release_secondary(cpu);
> Is this really msm8960 specific, if so than we should be doing something other than comparing against "qcom,kpssv1" or we should change the function to kpssv1_release_secondary()
Will do.
>
>> +			per_cpu(cold_boot_done, cpu) = true;
>> +		}
>> 	} else {
>> 		pr_err("%s: Invalid enable-method property: %s\n",
>> 				__func__, enable_method);
>> @@ -151,6 +206,8 @@ static void __init msm_smp_init_cpus(void)
>> static const int cold_boot_flags[] __initconst = {
>> 	0,
>> 	SCM_FLAG_COLDBOOT_CPU1,
>> +	SCM_FLAG_COLDBOOT_CPU2,
>> +	SCM_FLAG_COLDBOOT_CPU3,
>> };
>>
>> static void __init msm_smp_prepare_cpus(unsigned int max_cpus)
>> diff --git a/arch/arm/mach-msm/scm-boot.h b/arch/arm/mach-msm/scm-boot.h
>> index 7be32ff..6aabb24 100644
>> --- a/arch/arm/mach-msm/scm-boot.h
>> +++ b/arch/arm/mach-msm/scm-boot.h
>> @@ -13,9 +13,11 @@
>> #define __MACH_SCM_BOOT_H
>>
>> #define SCM_BOOT_ADDR			0x1
>> -#define SCM_FLAG_COLDBOOT_CPU1		0x1
>> -#define SCM_FLAG_WARMBOOT_CPU1		0x2
>> -#define SCM_FLAG_WARMBOOT_CPU0		0x4
>> +#define SCM_FLAG_COLDBOOT_CPU1		0x01
>> +#define SCM_FLAG_COLDBOOT_CPU2		0x08
>> +#define SCM_FLAG_COLDBOOT_CPU3		0x20
>> +#define SCM_FLAG_WARMBOOT_CPU0		0x04
>> +#define SCM_FLAG_WARMBOOT_CPU1		0x02
>>
>> int scm_set_boot_addr(phys_addr_t addr, int flags);
>>
>> -- 
>> The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
>> hosted by The Linux Foundation
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-arm-msm" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> --
> Employee of Qualcomm Innovation Center, Inc.
> Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-arm-msm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html


Thanks,
Rohit Vaswani

-- 
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum, hosted by The Linux Foundation

[PATCH 3/4] ARM: msm: Add SMP support for 8960

[Rohit Vaswani]

On 8/2/2013 8:43 AM, Kumar Gala wrote:
> On Aug 1, 2013, at 9:15 PM, Rohit Vaswani wrote:
>
>> Add the cpus bindings and the Krait release sequence
>> to make SMP work for MSM8960
>>
>> Signed-off-by: Rohit Vaswani <rvaswani@codeaurora.org>
>> ---
>> Documentation/devicetree/bindings/arm/cpus.txt     |  2 +
>> Documentation/devicetree/bindings/arm/msm/kpss.txt | 16 ++++++
>> arch/arm/boot/dts/msm8960-cdp.dts                  | 22 +++++++++
>> arch/arm/mach-msm/platsmp.c                        | 57 ++++++++++++++++++++++
>> arch/arm/mach-msm/scm-boot.h                       |  8 +--
>> 5 files changed, 102 insertions(+), 3 deletions(-)
>> create mode 100644 Documentation/devicetree/bindings/arm/msm/kpss.txt
>>
>> diff --git a/Documentation/devicetree/bindings/arm/cpus.txt b/Documentation/devicetree/bindings/arm/cpus.txt
>> index 327aad2..1132eac 100644
>> --- a/Documentation/devicetree/bindings/arm/cpus.txt
>> +++ b/Documentation/devicetree/bindings/arm/cpus.txt
>> @@ -45,11 +45,13 @@ For the ARM architecture every CPU node must contain the following properties:
>> 		"marvell,xsc3"
>> 		"marvell,xscale"
>> 		"qcom,scorpion"
>> +		"qcom,krait"
>> - enable-method: Specifies the method used to enable or take the secondary cores
>> 		 out of reset. This allows different reset sequence for
>> 		 different types of cpus.
>> 		 This should be one of:
>> 		 "qcom,scss"
>> +		 "qcom,kpssv1"
>>
>> Example:
>>
>> diff --git a/Documentation/devicetree/bindings/arm/msm/kpss.txt b/Documentation/devicetree/bindings/arm/msm/kpss.txt
>> new file mode 100644
>> index 0000000..7272340
>> --- /dev/null
>> +++ b/Documentation/devicetree/bindings/arm/msm/kpss.txt
>> @@ -0,0 +1,16 @@
>> +* KPSS - Krait Processor Sub-system
>> +
>> +Properties
>> +
>> +- compatible : Should contain "qcom,kpss".
>> +
>> +- reg: Specifies the base address for the KPSS registers used for
>> +       booting up secondary cores.
>> +
>> +Example:
>> +
>> +	kpss at 2088000 {
>> +		compatible = "qcom,kpss";
>> +		reg = <0x02088000 0x1000
>> +			0x02098000 0x2000>;
>> +	};
>> diff --git a/arch/arm/boot/dts/msm8960-cdp.dts b/arch/arm/boot/dts/msm8960-cdp.dts
>> index db2060c..8c82d5e 100644
>> --- a/arch/arm/boot/dts/msm8960-cdp.dts
>> +++ b/arch/arm/boot/dts/msm8960-cdp.dts
>> @@ -7,6 +7,22 @@
>> 	compatible = "qcom,msm8960-cdp", "qcom,msm8960";
>> 	interrupt-parent = <&intc>;
>>
>> +	cpus {
>> +		#address-cells = <1>;
>> +		#size-cells = <0>;
>> +		compatible = "qcom,krait";
>> +		device_type = "cpu";
>> +		enable-method = "qcom,kpssv1";
>> +
>> +		cpu at 0 {
>> +			reg = <0>;
>> +		};
>> +
>> +		cpu at 1 {
>> +			reg = <1>;
>> +		};
>> +	};
>> +
>> 	intc: interrupt-controller at 2000000 {
>> 		compatible = "qcom,msm-qgic2";
>> 		interrupt-controller;
>> @@ -37,6 +53,12 @@
>> 		reg = <0xfd510000 0x4000>;
>> 	};
>>
>> +	kpss at 2088000 {
>> +		compatible = "qcom,kpss";
>> +		reg = <0x02088000 0x1000
>> +			0x02098000 0x2000>;
>> +	};
>> +
>> 	serial at 16440000 {
>> 		compatible = "qcom,msm-hsuart", "qcom,msm-uart";
>> 		reg = <0x16440000 0x1000>,
>> diff --git a/arch/arm/mach-msm/platsmp.c b/arch/arm/mach-msm/platsmp.c
>> index 17022e0..82eb079 100644
>> --- a/arch/arm/mach-msm/platsmp.c
>> +++ b/arch/arm/mach-msm/platsmp.c
>> @@ -74,6 +74,56 @@ static int scorpion_release_secondary(void)
>> 	return 0;
>> }
>>
>> +static int msm8960_release_secondary(unsigned int cpu)
>> +{
>> +	void __iomem *reg;
>> +	struct device_node *dn = NULL;
>> +
>> +	if (cpu == 0 || cpu >= num_possible_cpus())
>> +		return -EINVAL;
>> +
>> +	dn = of_find_compatible_node(dn, NULL, "qcom,kpss");
>> +	if (!dn) {
>> +		pr_err("%s : Missing kpss node from device tree\n", __func__);
>> +		return -ENXIO;
>> +	}
>> +
>> +	reg = of_iomap(dn, cpu);
>> +	if (!reg)
>> +		return -ENOMEM;
>> +
>> +	pr_debug("Starting secondary CPU %d\n", cpu);
>> +
>> +	/* Turn on CPU Rail */
>> +	writel_relaxed(0xA4, reg+0x1014);
> Is there some reason we are using magic numbers for both values and offsets?
Yes. The names are not defined by hardware spec. Would you prefer having 
multiple #defines for each value, register ?
>
>> +	mb();
>> +	udelay(512);
>> +
>> +	/* Krait bring-up sequence */
>> +	writel_relaxed(0x109, reg+0x04);
>> +	writel_relaxed(0x101, reg+0x04);
>> +	mb();
>> +	ndelay(300);
>> +
>> +	writel_relaxed(0x121, reg+0x04);
>> +	mb();
>> +	udelay(2);
>> +
>> +	writel_relaxed(0x120, reg+0x04);
>> +	mb();
>> +	udelay(2);
>> +
>> +	writel_relaxed(0x100, reg+0x04);
>> +	mb();
>> +	udelay(100);
>> +
>> +	writel_relaxed(0x180, reg+0x04);
>> +	mb();
>> +
>> +	iounmap(reg);
>> +	return 0;
>> +}
>> +
>> static DEFINE_PER_CPU(int, cold_boot_done);
>>
>> static void boot_cold_cpu(unsigned int cpu)
>> @@ -96,6 +146,11 @@ static void boot_cold_cpu(unsigned int cpu)
>> 			scorpion_release_secondary();
>> 			per_cpu(cold_boot_done, cpu) = true;
>> 		}
>> +	} else if (!strcmp(enable_method, "qcom,kpssv1")) {
>> +		if (per_cpu(cold_boot_done, cpu) == false) {
>> +			msm8960_release_secondary(cpu);
> Is this really msm8960 specific, if so than we should be doing something other than comparing against "qcom,kpssv1" or we should change the function to kpssv1_release_secondary()
Will do.
>
>> +			per_cpu(cold_boot_done, cpu) = true;
>> +		}
>> 	} else {
>> 		pr_err("%s: Invalid enable-method property: %s\n",
>> 				__func__, enable_method);
>> @@ -151,6 +206,8 @@ static void __init msm_smp_init_cpus(void)
>> static const int cold_boot_flags[] __initconst = {
>> 	0,
>> 	SCM_FLAG_COLDBOOT_CPU1,
>> +	SCM_FLAG_COLDBOOT_CPU2,
>> +	SCM_FLAG_COLDBOOT_CPU3,
>> };
>>
>> static void __init msm_smp_prepare_cpus(unsigned int max_cpus)
>> diff --git a/arch/arm/mach-msm/scm-boot.h b/arch/arm/mach-msm/scm-boot.h
>> index 7be32ff..6aabb24 100644
>> --- a/arch/arm/mach-msm/scm-boot.h
>> +++ b/arch/arm/mach-msm/scm-boot.h
>> @@ -13,9 +13,11 @@
>> #define __MACH_SCM_BOOT_H
>>
>> #define SCM_BOOT_ADDR			0x1
>> -#define SCM_FLAG_COLDBOOT_CPU1		0x1
>> -#define SCM_FLAG_WARMBOOT_CPU1		0x2
>> -#define SCM_FLAG_WARMBOOT_CPU0		0x4
>> +#define SCM_FLAG_COLDBOOT_CPU1		0x01
>> +#define SCM_FLAG_COLDBOOT_CPU2		0x08
>> +#define SCM_FLAG_COLDBOOT_CPU3		0x20
>> +#define SCM_FLAG_WARMBOOT_CPU0		0x04
>> +#define SCM_FLAG_WARMBOOT_CPU1		0x02
>>
>> int scm_set_boot_addr(phys_addr_t addr, int flags);
>>
>> -- 
>> The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
>> hosted by The Linux Foundation
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-arm-msm" in
>> the body of a message to majordomo at vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> --
> Employee of Qualcomm Innovation Center, Inc.
> Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-arm-msm" in
> the body of a message to majordomo at vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html


Thanks,
Rohit Vaswani

-- 
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum, hosted by The Linux Foundation

Re: [RESEND PATCH 4/4] ARM: msm: Add support for 8974 SMP

[Rohit Vaswani]

On 8/2/2013 8:46 AM, Kumar Gala wrote:
> On Aug 1, 2013, at 9:15 PM, Rohit Vaswani wrote:
>
>> Add the cpus bindings and the Kraitv2 release sequence
>> to make SMP work for 2 cores on MSM8974.
>>
>> Signed-off-by: Rohit Vaswani <rvaswani@codeaurora.org>
>> ---
>> Documentation/devicetree/bindings/arm/cpus.txt |  1 +
>> arch/arm/boot/dts/msm8974.dts                  | 23 ++++++++
>> arch/arm/mach-msm/board-dt-8974.c              |  3 +
>> arch/arm/mach-msm/platsmp.c                    | 79 ++++++++++++++++++++++++++
>> 4 files changed, 106 insertions(+)
>>
>> diff --git a/Documentation/devicetree/bindings/arm/cpus.txt b/Documentation/devicetree/bindings/arm/cpus.txt
>> index 1132eac..7c3c677 100644
>> --- a/Documentation/devicetree/bindings/arm/cpus.txt
>> +++ b/Documentation/devicetree/bindings/arm/cpus.txt
>> @@ -52,6 +52,7 @@ For the ARM architecture every CPU node must contain the following properties:
>> 		 This should be one of:
>> 		 "qcom,scss"
>> 		 "qcom,kpssv1"
>> +		 "qcom,kpssv2"
>>
>> Example:
>>
>> diff --git a/arch/arm/boot/dts/msm8974.dts b/arch/arm/boot/dts/msm8974.dts
>> index c31c097..ef35a9b 100644
>> --- a/arch/arm/boot/dts/msm8974.dts
>> +++ b/arch/arm/boot/dts/msm8974.dts
>> @@ -7,6 +7,22 @@
>> 	compatible = "qcom,msm8974";
>> 	interrupt-parent = <&intc>;
>>
>> +	cpus {
>> +		#address-cells = <1>;
>> +		#size-cells = <0>;
>> +		compatible = "qcom,krait";
>> +		device_type = "cpu";
>> +		enable-method = "qcom,kpssv2";
>> +
>> +		cpu@0 {
>> +			reg = <0>;
>> +		};
>> +
>> +		cpu@1 {
>> +			reg = <1>;
>> +		};
> Any reason not to have all 4 cores?
>
>> +	};
>> +
>> 	intc: interrupt-controller@f9000000 {
>> 		compatible = "qcom,msm-qgic2";
>> 		interrupt-controller;
>> @@ -23,4 +39,11 @@
>> 			     <1 1 0xf08>;
>> 		clock-frequency = <19200000>;
>> 	};
>> +
>> +	kpss@f9012000 {
>> +		compatible = "qcom,kpss";
>> +		reg = <0xf9012000 0x1000>,
>> +		      <0xf9088000 0x1000>,
>> +		      <0xf9098000 0x1000>;
> we should probably have regnmaes to go along with this.
>
> Also this doesn't really match the binding spec, as you've included the L2 register, we should cleanup the binding spec to be more precise.
Its part of the Krait Processor Sub-System. I will add the reg-names to 
clear this up.

>
>> +	};
>> };
>> diff --git a/arch/arm/mach-msm/board-dt-8974.c b/arch/arm/mach-msm/board-dt-8974.c
>> index d7f84f2..06119f9 100644
>> --- a/arch/arm/mach-msm/board-dt-8974.c
>> +++ b/arch/arm/mach-msm/board-dt-8974.c
>> @@ -13,11 +13,14 @@
>> #include <linux/of_platform.h>
>> #include <asm/mach/arch.h>
>>
>> +#include "common.h"
>> +
>> static const char * const msm8974_dt_match[] __initconst = {
>> 	"qcom,msm8974",
>> 	NULL
>> };
>>
>> DT_MACHINE_START(MSM8974_DT, "Qualcomm MSM (Flattened Device Tree)")
>> +	.smp = smp_ops(msm_smp_ops),
>> 	.dt_compat = msm8974_dt_match,
>> MACHINE_END
>> diff --git a/arch/arm/mach-msm/platsmp.c b/arch/arm/mach-msm/platsmp.c
>> index 82eb079..0fdae69 100644
>> --- a/arch/arm/mach-msm/platsmp.c
>> +++ b/arch/arm/mach-msm/platsmp.c
>> @@ -124,6 +124,80 @@ static int msm8960_release_secondary(unsigned int cpu)
>> 	return 0;
>> }
>>
>> +static int msm8974_release_secondary(unsigned int cpu)
>> +{
>> +	void __iomem *reg;
>> +	void __iomem *l2_saw_base;
>> +	struct device_node *dn = NULL;
>> +	unsigned apc_pwr_gate_ctl = 0x14;
>> +	unsigned reg_val;
>> +
>> +	if (cpu == 0 || cpu >= num_possible_cpus())
>> +		return -EINVAL;
>> +
>> +	dn = of_find_compatible_node(dn, NULL, "qcom,kpss");
>> +	if (!dn) {
>> +		pr_err("%s : Missing kpss node from device tree\n", __func__);
>> +		return -ENXIO;
>> +	}
>> +
>> +	reg = of_iomap(dn, cpu+1);
>> +	if (!reg)
>> +		return -ENOMEM;
>> +
>> +	pr_debug("Starting secondary CPU %d\n", cpu);
>> +
>> +	/* Turn on the BHS, turn off LDO Bypass and power down LDO */
>> +	reg_val =  0x403f0001;
>> +	writel_relaxed(reg_val, reg + apc_pwr_gate_ctl);
>> +
>> +	/* complete the above write before the delay */
>> +	mb();
>> +	/* wait for the bhs to settle */
>> +	udelay(1);
>> +
>> +	/* Turn on BHS segments */
>> +	reg_val |= 0x3f << 1;
>> +	writel_relaxed(reg_val, reg + apc_pwr_gate_ctl);
>> +
>> +	/* complete the above write before the delay */
>> +	mb();
>> +	 /* wait for the bhs to settle */
>> +	udelay(1);
>> +
>> +	/* Finally turn on the bypass so that BHS supplies power */
>> +	reg_val |= 0x3f << 8;
>> +	writel_relaxed(reg_val, reg + apc_pwr_gate_ctl);
>> +
>> +	/* enable max phases */
>> +	l2_saw_base = of_iomap(dn, 0);
>> +	if (!l2_saw_base) {
>> +		return -ENOMEM;
>> +	}
>> +	writel_relaxed(0x10003, l2_saw_base + 0x1c);
>> +	mb();
>> +	udelay(50);
>> +
>> +	iounmap(l2_saw_base);
>> +
>> +	writel_relaxed(0x021, reg+0x04);
>> +	mb();
>> +	udelay(2);
>> +
>> +	writel_relaxed(0x020, reg+0x04);
>> +	mb();
>> +	udelay(2);
>> +
>> +	writel_relaxed(0x000, reg+0x04);
>> +	mb();
>> +
>> +	writel_relaxed(0x080, reg+0x04);
>> +	mb();
>> +
>> +	iounmap(reg);
>> +	return 0;
>> +}
>> +
>> static DEFINE_PER_CPU(int, cold_boot_done);
>>
>> static void boot_cold_cpu(unsigned int cpu)
>> @@ -151,6 +225,11 @@ static void boot_cold_cpu(unsigned int cpu)
>> 			msm8960_release_secondary(cpu);
>> 			per_cpu(cold_boot_done, cpu) = true;
>> 		}
>> +	} else if (!strcmp(enable_method, "qcom,kpssv2")) {
>> +		if (per_cpu(cold_boot_done, cpu) == false) {
>> +			msm8974_release_secondary(cpu);
> same comment as on the 8960 patch.
>
>> +			per_cpu(cold_boot_done, cpu) = true;
>> +		}
>> 	} else {
>> 		pr_err("%s: Invalid enable-method property: %s\n",
>> 				__func__, enable_method);
>> -- 
>> The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
>> hosted by The Linux Foundation
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-arm-msm" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> --
> Employee of Qualcomm Innovation Center, Inc.
> Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation
>
>
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel


Thanks,
Rohit Vaswani

-- 
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum, hosted by The Linux Foundation

[RESEND PATCH 4/4] ARM: msm: Add support for 8974 SMP

[Rohit Vaswani]

On 8/2/2013 8:46 AM, Kumar Gala wrote:
> On Aug 1, 2013, at 9:15 PM, Rohit Vaswani wrote:
>
>> Add the cpus bindings and the Kraitv2 release sequence
>> to make SMP work for 2 cores on MSM8974.
>>
>> Signed-off-by: Rohit Vaswani <rvaswani@codeaurora.org>
>> ---
>> Documentation/devicetree/bindings/arm/cpus.txt |  1 +
>> arch/arm/boot/dts/msm8974.dts                  | 23 ++++++++
>> arch/arm/mach-msm/board-dt-8974.c              |  3 +
>> arch/arm/mach-msm/platsmp.c                    | 79 ++++++++++++++++++++++++++
>> 4 files changed, 106 insertions(+)
>>
>> diff --git a/Documentation/devicetree/bindings/arm/cpus.txt b/Documentation/devicetree/bindings/arm/cpus.txt
>> index 1132eac..7c3c677 100644
>> --- a/Documentation/devicetree/bindings/arm/cpus.txt
>> +++ b/Documentation/devicetree/bindings/arm/cpus.txt
>> @@ -52,6 +52,7 @@ For the ARM architecture every CPU node must contain the following properties:
>> 		 This should be one of:
>> 		 "qcom,scss"
>> 		 "qcom,kpssv1"
>> +		 "qcom,kpssv2"
>>
>> Example:
>>
>> diff --git a/arch/arm/boot/dts/msm8974.dts b/arch/arm/boot/dts/msm8974.dts
>> index c31c097..ef35a9b 100644
>> --- a/arch/arm/boot/dts/msm8974.dts
>> +++ b/arch/arm/boot/dts/msm8974.dts
>> @@ -7,6 +7,22 @@
>> 	compatible = "qcom,msm8974";
>> 	interrupt-parent = <&intc>;
>>
>> +	cpus {
>> +		#address-cells = <1>;
>> +		#size-cells = <0>;
>> +		compatible = "qcom,krait";
>> +		device_type = "cpu";
>> +		enable-method = "qcom,kpssv2";
>> +
>> +		cpu at 0 {
>> +			reg = <0>;
>> +		};
>> +
>> +		cpu at 1 {
>> +			reg = <1>;
>> +		};
> Any reason not to have all 4 cores?
>
>> +	};
>> +
>> 	intc: interrupt-controller at f9000000 {
>> 		compatible = "qcom,msm-qgic2";
>> 		interrupt-controller;
>> @@ -23,4 +39,11 @@
>> 			     <1 1 0xf08>;
>> 		clock-frequency = <19200000>;
>> 	};
>> +
>> +	kpss at f9012000 {
>> +		compatible = "qcom,kpss";
>> +		reg = <0xf9012000 0x1000>,
>> +		      <0xf9088000 0x1000>,
>> +		      <0xf9098000 0x1000>;
> we should probably have regnmaes to go along with this.
>
> Also this doesn't really match the binding spec, as you've included the L2 register, we should cleanup the binding spec to be more precise.
Its part of the Krait Processor Sub-System. I will add the reg-names to 
clear this up.

>
>> +	};
>> };
>> diff --git a/arch/arm/mach-msm/board-dt-8974.c b/arch/arm/mach-msm/board-dt-8974.c
>> index d7f84f2..06119f9 100644
>> --- a/arch/arm/mach-msm/board-dt-8974.c
>> +++ b/arch/arm/mach-msm/board-dt-8974.c
>> @@ -13,11 +13,14 @@
>> #include <linux/of_platform.h>
>> #include <asm/mach/arch.h>
>>
>> +#include "common.h"
>> +
>> static const char * const msm8974_dt_match[] __initconst = {
>> 	"qcom,msm8974",
>> 	NULL
>> };
>>
>> DT_MACHINE_START(MSM8974_DT, "Qualcomm MSM (Flattened Device Tree)")
>> +	.smp = smp_ops(msm_smp_ops),
>> 	.dt_compat = msm8974_dt_match,
>> MACHINE_END
>> diff --git a/arch/arm/mach-msm/platsmp.c b/arch/arm/mach-msm/platsmp.c
>> index 82eb079..0fdae69 100644
>> --- a/arch/arm/mach-msm/platsmp.c
>> +++ b/arch/arm/mach-msm/platsmp.c
>> @@ -124,6 +124,80 @@ static int msm8960_release_secondary(unsigned int cpu)
>> 	return 0;
>> }
>>
>> +static int msm8974_release_secondary(unsigned int cpu)
>> +{
>> +	void __iomem *reg;
>> +	void __iomem *l2_saw_base;
>> +	struct device_node *dn = NULL;
>> +	unsigned apc_pwr_gate_ctl = 0x14;
>> +	unsigned reg_val;
>> +
>> +	if (cpu == 0 || cpu >= num_possible_cpus())
>> +		return -EINVAL;
>> +
>> +	dn = of_find_compatible_node(dn, NULL, "qcom,kpss");
>> +	if (!dn) {
>> +		pr_err("%s : Missing kpss node from device tree\n", __func__);
>> +		return -ENXIO;
>> +	}
>> +
>> +	reg = of_iomap(dn, cpu+1);
>> +	if (!reg)
>> +		return -ENOMEM;
>> +
>> +	pr_debug("Starting secondary CPU %d\n", cpu);
>> +
>> +	/* Turn on the BHS, turn off LDO Bypass and power down LDO */
>> +	reg_val =  0x403f0001;
>> +	writel_relaxed(reg_val, reg + apc_pwr_gate_ctl);
>> +
>> +	/* complete the above write before the delay */
>> +	mb();
>> +	/* wait for the bhs to settle */
>> +	udelay(1);
>> +
>> +	/* Turn on BHS segments */
>> +	reg_val |= 0x3f << 1;
>> +	writel_relaxed(reg_val, reg + apc_pwr_gate_ctl);
>> +
>> +	/* complete the above write before the delay */
>> +	mb();
>> +	 /* wait for the bhs to settle */
>> +	udelay(1);
>> +
>> +	/* Finally turn on the bypass so that BHS supplies power */
>> +	reg_val |= 0x3f << 8;
>> +	writel_relaxed(reg_val, reg + apc_pwr_gate_ctl);
>> +
>> +	/* enable max phases */
>> +	l2_saw_base = of_iomap(dn, 0);
>> +	if (!l2_saw_base) {
>> +		return -ENOMEM;
>> +	}
>> +	writel_relaxed(0x10003, l2_saw_base + 0x1c);
>> +	mb();
>> +	udelay(50);
>> +
>> +	iounmap(l2_saw_base);
>> +
>> +	writel_relaxed(0x021, reg+0x04);
>> +	mb();
>> +	udelay(2);
>> +
>> +	writel_relaxed(0x020, reg+0x04);
>> +	mb();
>> +	udelay(2);
>> +
>> +	writel_relaxed(0x000, reg+0x04);
>> +	mb();
>> +
>> +	writel_relaxed(0x080, reg+0x04);
>> +	mb();
>> +
>> +	iounmap(reg);
>> +	return 0;
>> +}
>> +
>> static DEFINE_PER_CPU(int, cold_boot_done);
>>
>> static void boot_cold_cpu(unsigned int cpu)
>> @@ -151,6 +225,11 @@ static void boot_cold_cpu(unsigned int cpu)
>> 			msm8960_release_secondary(cpu);
>> 			per_cpu(cold_boot_done, cpu) = true;
>> 		}
>> +	} else if (!strcmp(enable_method, "qcom,kpssv2")) {
>> +		if (per_cpu(cold_boot_done, cpu) == false) {
>> +			msm8974_release_secondary(cpu);
> same comment as on the 8960 patch.
>
>> +			per_cpu(cold_boot_done, cpu) = true;
>> +		}
>> 	} else {
>> 		pr_err("%s: Invalid enable-method property: %s\n",
>> 				__func__, enable_method);
>> -- 
>> The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
>> hosted by The Linux Foundation
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-arm-msm" in
>> the body of a message to majordomo at vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> --
> Employee of Qualcomm Innovation Center, Inc.
> Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation
>
>
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel


Thanks,
Rohit Vaswani

-- 
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum, hosted by The Linux Foundation

Re: [RESEND PATCH 2/4] ARM: msm: Re-organize platsmp to make it extensible

[Mark Rutland]

On Wed, Aug 14, 2013 at 09:55:31PM +0100, Rohit Vaswani wrote:
> On 8/12/2013 8:50 AM, Mark Rutland wrote:
> > Hi,
> >
> > Apologies for the long delay for review on this.
> >
> > I really like the direction this is going, but I have some qualms with
> > the implementation.
> 
> Thanks for your review, but a few direct recommendations would help in
> making the implementation right.

Sorry, I'll try to be clearer below :)

> >
> > On Fri, Aug 02, 2013 at 03:15:23AM +0100, Rohit Vaswani wrote:
> >> This makes it easy to add SMP support for new targets
> >> by adding cpus property and the release sequence.
> >> We add the enable-method property for the cpus property to
> >> specify which release sequence to use.
> >> While at it, add the 8660 cpus bindings to make SMP work.
> >>
> >> Signed-off-by: Rohit Vaswani <rvaswani@codeaurora.org>
> >> ---
> >>   Documentation/devicetree/bindings/arm/cpus.txt     |  6 ++
> >>   Documentation/devicetree/bindings/arm/msm/scss.txt | 15 ++++
> >>   arch/arm/boot/dts/msm8660-surf.dts                 | 23 +++++-
> >>   arch/arm/mach-msm/platsmp.c                        | 94 +++++++++++++++++-----
> >>   4 files changed, 115 insertions(+), 23 deletions(-)
> >>   create mode 100644 Documentation/devicetree/bindings/arm/msm/scss.txt
> >>
> >> diff --git a/Documentation/devicetree/bindings/arm/cpus.txt b/Documentation/devicetree/bindings/arm/cpus.txt
> >> index f32494d..327aad2 100644
> >> --- a/Documentation/devicetree/bindings/arm/cpus.txt
> >> +++ b/Documentation/devicetree/bindings/arm/cpus.txt
> >> @@ -44,6 +44,12 @@ For the ARM architecture every CPU node must contain the following properties:
> >>              "marvell,mohawk"
> >>              "marvell,xsc3"
> >>              "marvell,xscale"
> >> +            "qcom,scorpion"
> >> +- enable-method: Specifies the method used to enable or take the secondary cores
> >> +             out of reset. This allows different reset sequence for
> >> +             different types of cpus.
> >> +             This should be one of:
> >> +             "qcom,scss"
> > While I'd certainly like to see a move to using enable-method for
> > 32-bit, I think this needs a bit more thought:
> >
> > What does "qcom,scss" mean, precisely? It seems to be that we poke some
> > registers in a "qcom,scss" device. I think we need to document
> > enable-methods *very* carefully (and we need to do that for 64-bit as
> > well with psci), as it's very likely there'll be subtle
> > incompatibilities between platforms, especially if firmware is involved.
> > We should try to leaves as little room for error as possible.
> Yes qcom,scss implies poking registers in the qcom,scss device. How
> could I make that more clear in the documentation ?

A description of that should be in the binding document to that effect.

Just having "qcom,scss" may not be enough, as we don't necessarily know
which bit in a register needs to be poked for a specific CPU. While
today you may have a single cluster and it's the bit at offset
MPIDR.Aff0 in the register, that might not be true in future (either
because you have multiple clusters or some arbitrary mappnig from CPU to
bit), and we may need to describe that.

Presumably we need to write the start address for the secondary CPU
somewhere -- is that a register in the SCSS, does the CPU start at some
fixed address that we need to install a trampoline at, or is there some
other mechanism for controlling this?

We need to try and make as few assumptions as possible or we can't have
a stable binding here. We already have variations on "spin-table", for
example on arm64 where firmware may have a wfe, so the OS *must* issue a
sev to function with all implementations. At the opposite end, PSCI is
pretty much identical across arm and arm64 (at present).

We'll probably want to factor the enable methods out into a separate
file or directory, as I can see this getting quite large.

> 
> >
> > I don't want to see this devolve into meaning "whatever the Linux driver
> > for this happens to do at the current point in time", as that just leads
> > to breakage as we have no clear distinction between actual requirements
> > and implementation details.
> >
> > Given we already have platforms without an enable-method, we can't make
> > it a required property either -- those platforms are already booting by
> > figuring out an enable method from the platform's compatible string.
> So, you recommend we continue to using the platform compatible string as
> well ?

I'm not making a recommendation to do that, but we need to consider the
trade-off. Adding a new enable-method for each platform and then finding
it can't be shared because we didn't document the general case is
something I'd like to avoid.

> We currently don't have a perfect method to use enable-method in generic
> code. More on this below...

We certainly need to factor out enable-method detection into generic
code. Ideally, if we don't have an smp_ops we'd fall back to
enable-method detection.

> >
> > With PSCI, enable-method also implies a method for disabling a
> > particular CPU, so it would be nice for the description to cover this.
> >
> >>
> >>   Example:
> >>
> >> diff --git a/Documentation/devicetree/bindings/arm/msm/scss.txt b/Documentation/devicetree/bindings/arm/msm/scss.txt
> >> new file mode 100644
> >> index 0000000..21c3e26
> >> --- /dev/null
> >> +++ b/Documentation/devicetree/bindings/arm/msm/scss.txt
> >> @@ -0,0 +1,15 @@
> >> +* SCSS - Scorpion Sub-system
> >> +
> >> +Properties
> >> +
> >> +- compatible : Should contain "qcom,scss".
> >> +
> >> +- reg: Specifies the base address for the SCSS registers used for
> >> +       booting up secondary cores.
> >> +
> >> +Example:
> >> +
> >> +    scss@902000 {
> >> +            compatible = "qcom,scss";
> >> +            reg = <0x00902000 0x2000>;
> >> +    };
> >> diff --git a/arch/arm/boot/dts/msm8660-surf.dts b/arch/arm/boot/dts/msm8660-surf.dts
> >> index cdc010e..203e51a 100644
> >> --- a/arch/arm/boot/dts/msm8660-surf.dts
> >> +++ b/arch/arm/boot/dts/msm8660-surf.dts
> >> @@ -7,6 +7,22 @@
> >>      compatible = "qcom,msm8660-surf", "qcom,msm8660";
> >>      interrupt-parent = <&intc>;
> >>
> >> +    cpus {
> >> +            #address-cells = <1>;
> >> +            #size-cells = <0>;
> >> +            compatible = "qcom,scorpion";
> >> +            device_type = "cpu";
> >> +            enable-method = "qcom,scss";
> >> +
> >> +            cpu@0 {
> >> +                    reg = <0>;
> >> +            };
> >> +
> >> +            cpu@1 {
> >> +                    reg = <1>;
> >> +            };
> >> +    };
> >> +
> > I *really* like moving the common properties out into the /cpus node --
> > ePAPR suggests it, it's less error prone, and it takes up less space.
> > However, I'm not sure our generic/arch code handles it properly, and I
> > think we need to audit that before we can start writing dts that depend
> > on it. I'd be happy to be wrong here if anyone can correct me. :)

I was trying to point out that here we have the properties under /cpus,
but the PSCI code in kvmtool currently drops these per-cpu. Either is
technically valid, and the recommendation of ePAPR is to look in a given
/cpus/cpu@N node first, then fall back to the /cpus node.

> >
> > We probably need to factor out the way we read properties for cpu nodes,
> > falling back to ones present in /cpus if not present. There's already a
> > lot of pain getting the node for a logical (rather than physical) cpu
> > id.
> >
> > Sudeep recently factored out finding the cpu node for a logical cpu id
> > [1,2] in generic code with a per-arch callback, it shouldn't be too hard
> > to have shims around that to read (optionally) common properties.
> >

This was my attempt at a suggestion. We want some generic code for
grabbing cpu properties that falls back to the /cpus node. Something
like a family of of_get_cpu_{property,u32,string} functions (we likely
don't need much more than string at the moment).

Sudeep's patches for getting the cpu node for a logical cpu id could be
the basis for this.

> > [...]
> >
> >> -static void prepare_cold_cpu(unsigned int cpu)
> >> +static int scorpion_release_secondary(void)
> >>   {
> >> -    int ret;
> >> -    ret = scm_set_boot_addr(virt_to_phys(secondary_startup),
> >> -                            SCM_FLAG_COLDBOOT_CPU1);
> >> -    if (ret == 0) {
> >> -            void __iomem *sc1_base_ptr;
> >> -            sc1_base_ptr = ioremap_nocache(0x00902000, SZ_4K*2);
> >> -            if (sc1_base_ptr) {
> >> -                    writel(0, sc1_base_ptr + VDD_SC1_ARRAY_CLAMP_GFS_CTL);
> >> -                    writel(0, sc1_base_ptr + SCSS_CPU1CORE_RESET);
> >> -                    writel(3, sc1_base_ptr + SCSS_DBG_STATUS_CORE_PWRDUP);
> >> -                    iounmap(sc1_base_ptr);
> >> -            }
> >> -    } else
> >> -            printk(KERN_DEBUG "Failed to set secondary core boot "
> >> -                              "address\n");
> >> +    void __iomem *sc1_base_ptr;
> >> +    struct device_node *dn = NULL;
> >> +
> >> +    dn = of_find_compatible_node(dn, NULL, "qcom,scss");
> >> +    if (!dn) {
> >> +            pr_err("%s: Missing scss node in device tree\n", __func__);
> >> +            return -ENXIO;
> >> +    }
> >> +
> >> +    sc1_base_ptr = of_iomap(dn, 0);
> >> +    if (sc1_base_ptr) {
> >> +            writel_relaxed(0, sc1_base_ptr + VDD_SC1_ARRAY_CLAMP_GFS_CTL);
> >> +            writel_relaxed(0, sc1_base_ptr + SCSS_CPU1CORE_RESET);
> >> +            writel_relaxed(3, sc1_base_ptr + SCSS_DBG_STATUS_CORE_PWRDUP);
> >> +            mb();
> >> +            iounmap(sc1_base_ptr);
> > Does this boot *all* secondary CPUs directly into the kernel? Is that
> > safe (e.g. if the kernel supports fewer CPUs than are physically
> > present)?
> Im not sure I understand the concern with safety here. The CPU for which
> the release_secondary will be called will be taken out of reset with
> this sequence.

I was confused that this function was no longer taking a cpu property.
To me that seems to imply that what this was doing was not specific to a
particular CPU,

I must have misunderstood something here -- how does this take a
particular CPU out of reset? It looks like it's poking some shared
registers in the "qcom,scss" device that aren't specific to a single
CPU.

Is there only one secondary? How does this generalise to more?

> 
> >
> > Is this a one-time thing, or are we able to dynamically hotplug CPUs? If
> > the latter, the map/unmap looks odd to me.
> 
> This is a one-time thing done for each CPU that's specified in the
> device tree (or based on the command line over-ride).

Ok. Is it possible for hot unplug to be implemented based on this later?

> >> +    } else {
> >> +            return -ENOMEM;
> >> +    }
> >> +
> >> +    return 0;
> >>   }
> >>
> >> -static int msm_boot_secondary(unsigned int cpu, struct task_struct *idle)
> >> +static DEFINE_PER_CPU(int, cold_boot_done);
> >> +
> >> +static void boot_cold_cpu(unsigned int cpu)
> >>   {
> >> -    static int cold_boot_done;
> >> +    const char *enable_method;
> >> +    struct device_node *dn = NULL;
> >>
> >> -    /* Only need to bring cpu out of reset this way once */
> >> -    if (cold_boot_done == false) {
> >> -            prepare_cold_cpu(cpu);
> >> -            cold_boot_done = true;
> >> +    dn = of_find_node_by_name(dn, "cpus");
> >> +    if (!dn) {
> >> +            pr_err("%s: Missing node cpus in device tree\n", __func__);
> >> +            return;
> >>      }
> >>
> >> +    enable_method = of_get_property(dn, "enable-method", NULL);
> > Please factor this out from the platform code.
> >
> > If we're going to use enable-method, it should be decoupled from
> > platform code -- common code should parse it to find the appropriate
> > handler. Also, we *must* support having an enable-method per-cpu as KVM
> > does for PSCI (though I definitely would like support for shared
> > properties as mentioned above).
> Currently with most platforms, smp.c calls into the boot_secondary
> function which decides which cpu it is
> and then calls the appropriate function. This is because smp ops allow
> only 1 callback function to be registered...
> Hence, using the enable-method in platform specific code works.
> 
> If we create a generic property should it mandate having generic code
> handle that ?

I'd certainly prefer to move enable-method handling into generic code.
Is there no way we could do this if we had a NULL smp_ops entry?

> I currently don't have a good way of incorporating enable-method in
> generic code as it would mean to establish a mechanism to
> associate the enable-method string with cpu specific boot_secondary
> functions.

If we have cpu-specific boot_secondary functions, then surely the
enable-method isn't enough to tell us how to boot the secondary?

Does the cpu's compatible string not give us enough, or is this still
platform-specific? If it's the latter, I'm not sure having the
enable-method buys us anything, as we still need additional platform
information to be able to use it.

> You have an approach in mind that I can try ?

Not at present, but I'm sure we can think of something.

> 
> >> +    if (!enable_method) {
> >> +                    pr_err("%s: cpus node is missing enable-method property\n",
> >> +                                    __func__);
> >> +    } else if (!strcmp(enable_method, "qcom,scss")) {
> >> +            if (per_cpu(cold_boot_done, cpu) == false) {
> >> +                    scorpion_release_secondary();
> >> +                    per_cpu(cold_boot_done, cpu) = true;
> >> +            }
> >> +    } else {
> >> +            pr_err("%s: Invalid enable-method property: %s\n",
> >> +                            __func__, enable_method);
> >> +    }
> >> +}
> >> +
> >> +static int msm_boot_secondary(unsigned int cpu, struct task_struct *idle)
> >> +{
> >> +    boot_cold_cpu(cpu);
> >> +
> >>      /*
> >>       * set synchronisation state between this boot processor
> >>       * and the secondary one
> >> @@ -118,8 +148,28 @@ static void __init msm_smp_init_cpus(void)
> >>              set_cpu_possible(i, true);
> >>   }
> >>
> >> +static const int cold_boot_flags[] __initconst = {
> >> +    0,
> >> +    SCM_FLAG_COLDBOOT_CPU1,
> >> +};
> > So we only ever support booting two CPUs?
> The next patch which adds support for a quadcore chip, adds more flags.
> >
> > Is the mapping of MPIDR to register bits arbitrary? Or do we know what
> > they would be for four CPUs, four clusters, and beyond?
> Four cpus.

Ok. is SCSS based booting always going to be hard-limited to four CPUs,
or do we possibly need to encode some (optional) information in each CPU
node in the dt (i.e. the bits of the register that needs to be poked).

> 
> >
> >> +
> >>   static void __init msm_smp_prepare_cpus(unsigned int max_cpus)
> >>   {
> >> +    int cpu, map;
> >> +    unsigned int flags = 0;
> >> +
> >> +    for_each_present_cpu(cpu) {
> >> +            map = cpu_logical_map(cpu);
> >> +            if (map > ARRAY_SIZE(cold_boot_flags)) {
> >> +                    set_cpu_present(cpu, false);
> >> +                    __WARN();
> >> +                    continue;
> >> +            }
> >> +            flags |= cold_boot_flags[map];
> > It's a shame that this leaves a window where some CPUs seem bootable,
> > but aren't (though I can't offer a better way of handling this given we
> > have dts without enable-method properties).
> Any CPU that seems bootable and is defined in DT, should be bootable.
> The cold_boot_flags are just a mechanism to set appropriate values for
> the scm call and
> they aren't used as a method to disallow CPUs from being bootable and
> hence the __WARN() if this is done incorrectly.

Ok. That sounds reasonable.

Thanks,
Mark.

Re: [RESEND PATCH 2/4] ARM: msm: Re-organize platsmp to make it extensible

[Mark Rutland]

On Wed, Aug 14, 2013 at 09:55:31PM +0100, Rohit Vaswani wrote:
> On 8/12/2013 8:50 AM, Mark Rutland wrote:
> > Hi,
> >
> > Apologies for the long delay for review on this.
> >
> > I really like the direction this is going, but I have some qualms with
> > the implementation.
> 
> Thanks for your review, but a few direct recommendations would help in
> making the implementation right.

Sorry, I'll try to be clearer below :)

> >
> > On Fri, Aug 02, 2013 at 03:15:23AM +0100, Rohit Vaswani wrote:
> >> This makes it easy to add SMP support for new targets
> >> by adding cpus property and the release sequence.
> >> We add the enable-method property for the cpus property to
> >> specify which release sequence to use.
> >> While at it, add the 8660 cpus bindings to make SMP work.
> >>
> >> Signed-off-by: Rohit Vaswani <rvaswani@codeaurora.org>
> >> ---
> >>   Documentation/devicetree/bindings/arm/cpus.txt     |  6 ++
> >>   Documentation/devicetree/bindings/arm/msm/scss.txt | 15 ++++
> >>   arch/arm/boot/dts/msm8660-surf.dts                 | 23 +++++-
> >>   arch/arm/mach-msm/platsmp.c                        | 94 +++++++++++++++++-----
> >>   4 files changed, 115 insertions(+), 23 deletions(-)
> >>   create mode 100644 Documentation/devicetree/bindings/arm/msm/scss.txt
> >>
> >> diff --git a/Documentation/devicetree/bindings/arm/cpus.txt b/Documentation/devicetree/bindings/arm/cpus.txt
> >> index f32494d..327aad2 100644
> >> --- a/Documentation/devicetree/bindings/arm/cpus.txt
> >> +++ b/Documentation/devicetree/bindings/arm/cpus.txt
> >> @@ -44,6 +44,12 @@ For the ARM architecture every CPU node must contain the following properties:
> >>              "marvell,mohawk"
> >>              "marvell,xsc3"
> >>              "marvell,xscale"
> >> +            "qcom,scorpion"
> >> +- enable-method: Specifies the method used to enable or take the secondary cores
> >> +             out of reset. This allows different reset sequence for
> >> +             different types of cpus.
> >> +             This should be one of:
> >> +             "qcom,scss"
> > While I'd certainly like to see a move to using enable-method for
> > 32-bit, I think this needs a bit more thought:
> >
> > What does "qcom,scss" mean, precisely? It seems to be that we poke some
> > registers in a "qcom,scss" device. I think we need to document
> > enable-methods *very* carefully (and we need to do that for 64-bit as
> > well with psci), as it's very likely there'll be subtle
> > incompatibilities between platforms, especially if firmware is involved.
> > We should try to leaves as little room for error as possible.
> Yes qcom,scss implies poking registers in the qcom,scss device. How
> could I make that more clear in the documentation ?

A description of that should be in the binding document to that effect.

Just having "qcom,scss" may not be enough, as we don't necessarily know
which bit in a register needs to be poked for a specific CPU. While
today you may have a single cluster and it's the bit at offset
MPIDR.Aff0 in the register, that might not be true in future (either
because you have multiple clusters or some arbitrary mappnig from CPU to
bit), and we may need to describe that.

Presumably we need to write the start address for the secondary CPU
somewhere -- is that a register in the SCSS, does the CPU start at some
fixed address that we need to install a trampoline at, or is there some
other mechanism for controlling this?

We need to try and make as few assumptions as possible or we can't have
a stable binding here. We already have variations on "spin-table", for
example on arm64 where firmware may have a wfe, so the OS *must* issue a
sev to function with all implementations. At the opposite end, PSCI is
pretty much identical across arm and arm64 (at present).

We'll probably want to factor the enable methods out into a separate
file or directory, as I can see this getting quite large.

> 
> >
> > I don't want to see this devolve into meaning "whatever the Linux driver
> > for this happens to do at the current point in time", as that just leads
> > to breakage as we have no clear distinction between actual requirements
> > and implementation details.
> >
> > Given we already have platforms without an enable-method, we can't make
> > it a required property either -- those platforms are already booting by
> > figuring out an enable method from the platform's compatible string.
> So, you recommend we continue to using the platform compatible string as
> well ?

I'm not making a recommendation to do that, but we need to consider the
trade-off. Adding a new enable-method for each platform and then finding
it can't be shared because we didn't document the general case is
something I'd like to avoid.

> We currently don't have a perfect method to use enable-method in generic
> code. More on this below...

We certainly need to factor out enable-method detection into generic
code. Ideally, if we don't have an smp_ops we'd fall back to
enable-method detection.

> >
> > With PSCI, enable-method also implies a method for disabling a
> > particular CPU, so it would be nice for the description to cover this.
> >
> >>
> >>   Example:
> >>
> >> diff --git a/Documentation/devicetree/bindings/arm/msm/scss.txt b/Documentation/devicetree/bindings/arm/msm/scss.txt
> >> new file mode 100644
> >> index 0000000..21c3e26
> >> --- /dev/null
> >> +++ b/Documentation/devicetree/bindings/arm/msm/scss.txt
> >> @@ -0,0 +1,15 @@
> >> +* SCSS - Scorpion Sub-system
> >> +
> >> +Properties
> >> +
> >> +- compatible : Should contain "qcom,scss".
> >> +
> >> +- reg: Specifies the base address for the SCSS registers used for
> >> +       booting up secondary cores.
> >> +
> >> +Example:
> >> +
> >> +    scss@902000 {
> >> +            compatible = "qcom,scss";
> >> +            reg = <0x00902000 0x2000>;
> >> +    };
> >> diff --git a/arch/arm/boot/dts/msm8660-surf.dts b/arch/arm/boot/dts/msm8660-surf.dts
> >> index cdc010e..203e51a 100644
> >> --- a/arch/arm/boot/dts/msm8660-surf.dts
> >> +++ b/arch/arm/boot/dts/msm8660-surf.dts
> >> @@ -7,6 +7,22 @@
> >>      compatible = "qcom,msm8660-surf", "qcom,msm8660";
> >>      interrupt-parent = <&intc>;
> >>
> >> +    cpus {
> >> +            #address-cells = <1>;
> >> +            #size-cells = <0>;
> >> +            compatible = "qcom,scorpion";
> >> +            device_type = "cpu";
> >> +            enable-method = "qcom,scss";
> >> +
> >> +            cpu@0 {
> >> +                    reg = <0>;
> >> +            };
> >> +
> >> +            cpu@1 {
> >> +                    reg = <1>;
> >> +            };
> >> +    };
> >> +
> > I *really* like moving the common properties out into the /cpus node --
> > ePAPR suggests it, it's less error prone, and it takes up less space.
> > However, I'm not sure our generic/arch code handles it properly, and I
> > think we need to audit that before we can start writing dts that depend
> > on it. I'd be happy to be wrong here if anyone can correct me. :)

I was trying to point out that here we have the properties under /cpus,
but the PSCI code in kvmtool currently drops these per-cpu. Either is
technically valid, and the recommendation of ePAPR is to look in a given
/cpus/cpu@N node first, then fall back to the /cpus node.

> >
> > We probably need to factor out the way we read properties for cpu nodes,
> > falling back to ones present in /cpus if not present. There's already a
> > lot of pain getting the node for a logical (rather than physical) cpu
> > id.
> >
> > Sudeep recently factored out finding the cpu node for a logical cpu id
> > [1,2] in generic code with a per-arch callback, it shouldn't be too hard
> > to have shims around that to read (optionally) common properties.
> >

This was my attempt at a suggestion. We want some generic code for
grabbing cpu properties that falls back to the /cpus node. Something
like a family of of_get_cpu_{property,u32,string} functions (we likely
don't need much more than string at the moment).

Sudeep's patches for getting the cpu node for a logical cpu id could be
the basis for this.

> > [...]
> >
> >> -static void prepare_cold_cpu(unsigned int cpu)
> >> +static int scorpion_release_secondary(void)
> >>   {
> >> -    int ret;
> >> -    ret = scm_set_boot_addr(virt_to_phys(secondary_startup),
> >> -                            SCM_FLAG_COLDBOOT_CPU1);
> >> -    if (ret == 0) {
> >> -            void __iomem *sc1_base_ptr;
> >> -            sc1_base_ptr = ioremap_nocache(0x00902000, SZ_4K*2);
> >> -            if (sc1_base_ptr) {
> >> -                    writel(0, sc1_base_ptr + VDD_SC1_ARRAY_CLAMP_GFS_CTL);
> >> -                    writel(0, sc1_base_ptr + SCSS_CPU1CORE_RESET);
> >> -                    writel(3, sc1_base_ptr + SCSS_DBG_STATUS_CORE_PWRDUP);
> >> -                    iounmap(sc1_base_ptr);
> >> -            }
> >> -    } else
> >> -            printk(KERN_DEBUG "Failed to set secondary core boot "
> >> -                              "address\n");
> >> +    void __iomem *sc1_base_ptr;
> >> +    struct device_node *dn = NULL;
> >> +
> >> +    dn = of_find_compatible_node(dn, NULL, "qcom,scss");
> >> +    if (!dn) {
> >> +            pr_err("%s: Missing scss node in device tree\n", __func__);
> >> +            return -ENXIO;
> >> +    }
> >> +
> >> +    sc1_base_ptr = of_iomap(dn, 0);
> >> +    if (sc1_base_ptr) {
> >> +            writel_relaxed(0, sc1_base_ptr + VDD_SC1_ARRAY_CLAMP_GFS_CTL);
> >> +            writel_relaxed(0, sc1_base_ptr + SCSS_CPU1CORE_RESET);
> >> +            writel_relaxed(3, sc1_base_ptr + SCSS_DBG_STATUS_CORE_PWRDUP);
> >> +            mb();
> >> +            iounmap(sc1_base_ptr);
> > Does this boot *all* secondary CPUs directly into the kernel? Is that
> > safe (e.g. if the kernel supports fewer CPUs than are physically
> > present)?
> Im not sure I understand the concern with safety here. The CPU for which
> the release_secondary will be called will be taken out of reset with
> this sequence.

I was confused that this function was no longer taking a cpu property.
To me that seems to imply that what this was doing was not specific to a
particular CPU,

I must have misunderstood something here -- how does this take a
particular CPU out of reset? It looks like it's poking some shared
registers in the "qcom,scss" device that aren't specific to a single
CPU.

Is there only one secondary? How does this generalise to more?

> 
> >
> > Is this a one-time thing, or are we able to dynamically hotplug CPUs? If
> > the latter, the map/unmap looks odd to me.
> 
> This is a one-time thing done for each CPU that's specified in the
> device tree (or based on the command line over-ride).

Ok. Is it possible for hot unplug to be implemented based on this later?

> >> +    } else {
> >> +            return -ENOMEM;
> >> +    }
> >> +
> >> +    return 0;
> >>   }
> >>
> >> -static int msm_boot_secondary(unsigned int cpu, struct task_struct *idle)
> >> +static DEFINE_PER_CPU(int, cold_boot_done);
> >> +
> >> +static void boot_cold_cpu(unsigned int cpu)
> >>   {
> >> -    static int cold_boot_done;
> >> +    const char *enable_method;
> >> +    struct device_node *dn = NULL;
> >>
> >> -    /* Only need to bring cpu out of reset this way once */
> >> -    if (cold_boot_done == false) {
> >> -            prepare_cold_cpu(cpu);
> >> -            cold_boot_done = true;
> >> +    dn = of_find_node_by_name(dn, "cpus");
> >> +    if (!dn) {
> >> +            pr_err("%s: Missing node cpus in device tree\n", __func__);
> >> +            return;
> >>      }
> >>
> >> +    enable_method = of_get_property(dn, "enable-method", NULL);
> > Please factor this out from the platform code.
> >
> > If we're going to use enable-method, it should be decoupled from
> > platform code -- common code should parse it to find the appropriate
> > handler. Also, we *must* support having an enable-method per-cpu as KVM
> > does for PSCI (though I definitely would like support for shared
> > properties as mentioned above).
> Currently with most platforms, smp.c calls into the boot_secondary
> function which decides which cpu it is
> and then calls the appropriate function. This is because smp ops allow
> only 1 callback function to be registered...
> Hence, using the enable-method in platform specific code works.
> 
> If we create a generic property should it mandate having generic code
> handle that ?

I'd certainly prefer to move enable-method handling into generic code.
Is there no way we could do this if we had a NULL smp_ops entry?

> I currently don't have a good way of incorporating enable-method in
> generic code as it would mean to establish a mechanism to
> associate the enable-method string with cpu specific boot_secondary
> functions.

If we have cpu-specific boot_secondary functions, then surely the
enable-method isn't enough to tell us how to boot the secondary?

Does the cpu's compatible string not give us enough, or is this still
platform-specific? If it's the latter, I'm not sure having the
enable-method buys us anything, as we still need additional platform
information to be able to use it.

> You have an approach in mind that I can try ?

Not at present, but I'm sure we can think of something.

> 
> >> +    if (!enable_method) {
> >> +                    pr_err("%s: cpus node is missing enable-method property\n",
> >> +                                    __func__);
> >> +    } else if (!strcmp(enable_method, "qcom,scss")) {
> >> +            if (per_cpu(cold_boot_done, cpu) == false) {
> >> +                    scorpion_release_secondary();
> >> +                    per_cpu(cold_boot_done, cpu) = true;
> >> +            }
> >> +    } else {
> >> +            pr_err("%s: Invalid enable-method property: %s\n",
> >> +                            __func__, enable_method);
> >> +    }
> >> +}
> >> +
> >> +static int msm_boot_secondary(unsigned int cpu, struct task_struct *idle)
> >> +{
> >> +    boot_cold_cpu(cpu);
> >> +
> >>      /*
> >>       * set synchronisation state between this boot processor
> >>       * and the secondary one
> >> @@ -118,8 +148,28 @@ static void __init msm_smp_init_cpus(void)
> >>              set_cpu_possible(i, true);
> >>   }
> >>
> >> +static const int cold_boot_flags[] __initconst = {
> >> +    0,
> >> +    SCM_FLAG_COLDBOOT_CPU1,
> >> +};
> > So we only ever support booting two CPUs?
> The next patch which adds support for a quadcore chip, adds more flags.
> >
> > Is the mapping of MPIDR to register bits arbitrary? Or do we know what
> > they would be for four CPUs, four clusters, and beyond?
> Four cpus.

Ok. is SCSS based booting always going to be hard-limited to four CPUs,
or do we possibly need to encode some (optional) information in each CPU
node in the dt (i.e. the bits of the register that needs to be poked).

> 
> >
> >> +
> >>   static void __init msm_smp_prepare_cpus(unsigned int max_cpus)
> >>   {
> >> +    int cpu, map;
> >> +    unsigned int flags = 0;
> >> +
> >> +    for_each_present_cpu(cpu) {
> >> +            map = cpu_logical_map(cpu);
> >> +            if (map > ARRAY_SIZE(cold_boot_flags)) {
> >> +                    set_cpu_present(cpu, false);
> >> +                    __WARN();
> >> +                    continue;
> >> +            }
> >> +            flags |= cold_boot_flags[map];
> > It's a shame that this leaves a window where some CPUs seem bootable,
> > but aren't (though I can't offer a better way of handling this given we
> > have dts without enable-method properties).
> Any CPU that seems bootable and is defined in DT, should be bootable.
> The cold_boot_flags are just a mechanism to set appropriate values for
> the scm call and
> they aren't used as a method to disallow CPUs from being bootable and
> hence the __WARN() if this is done incorrectly.

Ok. That sounds reasonable.

Thanks,
Mark.

[RESEND PATCH 2/4] ARM: msm: Re-organize platsmp to make it extensible

[Mark Rutland]

On Wed, Aug 14, 2013 at 09:55:31PM +0100, Rohit Vaswani wrote:
> On 8/12/2013 8:50 AM, Mark Rutland wrote:
> > Hi,
> >
> > Apologies for the long delay for review on this.
> >
> > I really like the direction this is going, but I have some qualms with
> > the implementation.
> 
> Thanks for your review, but a few direct recommendations would help in
> making the implementation right.

Sorry, I'll try to be clearer below :)

> >
> > On Fri, Aug 02, 2013 at 03:15:23AM +0100, Rohit Vaswani wrote:
> >> This makes it easy to add SMP support for new targets
> >> by adding cpus property and the release sequence.
> >> We add the enable-method property for the cpus property to
> >> specify which release sequence to use.
> >> While at it, add the 8660 cpus bindings to make SMP work.
> >>
> >> Signed-off-by: Rohit Vaswani <rvaswani@codeaurora.org>
> >> ---
> >>   Documentation/devicetree/bindings/arm/cpus.txt     |  6 ++
> >>   Documentation/devicetree/bindings/arm/msm/scss.txt | 15 ++++
> >>   arch/arm/boot/dts/msm8660-surf.dts                 | 23 +++++-
> >>   arch/arm/mach-msm/platsmp.c                        | 94 +++++++++++++++++-----
> >>   4 files changed, 115 insertions(+), 23 deletions(-)
> >>   create mode 100644 Documentation/devicetree/bindings/arm/msm/scss.txt
> >>
> >> diff --git a/Documentation/devicetree/bindings/arm/cpus.txt b/Documentation/devicetree/bindings/arm/cpus.txt
> >> index f32494d..327aad2 100644
> >> --- a/Documentation/devicetree/bindings/arm/cpus.txt
> >> +++ b/Documentation/devicetree/bindings/arm/cpus.txt
> >> @@ -44,6 +44,12 @@ For the ARM architecture every CPU node must contain the following properties:
> >>              "marvell,mohawk"
> >>              "marvell,xsc3"
> >>              "marvell,xscale"
> >> +            "qcom,scorpion"
> >> +- enable-method: Specifies the method used to enable or take the secondary cores
> >> +             out of reset. This allows different reset sequence for
> >> +             different types of cpus.
> >> +             This should be one of:
> >> +             "qcom,scss"
> > While I'd certainly like to see a move to using enable-method for
> > 32-bit, I think this needs a bit more thought:
> >
> > What does "qcom,scss" mean, precisely? It seems to be that we poke some
> > registers in a "qcom,scss" device. I think we need to document
> > enable-methods *very* carefully (and we need to do that for 64-bit as
> > well with psci), as it's very likely there'll be subtle
> > incompatibilities between platforms, especially if firmware is involved.
> > We should try to leaves as little room for error as possible.
> Yes qcom,scss implies poking registers in the qcom,scss device. How
> could I make that more clear in the documentation ?

A description of that should be in the binding document to that effect.

Just having "qcom,scss" may not be enough, as we don't necessarily know
which bit in a register needs to be poked for a specific CPU. While
today you may have a single cluster and it's the bit at offset
MPIDR.Aff0 in the register, that might not be true in future (either
because you have multiple clusters or some arbitrary mappnig from CPU to
bit), and we may need to describe that.

Presumably we need to write the start address for the secondary CPU
somewhere -- is that a register in the SCSS, does the CPU start at some
fixed address that we need to install a trampoline at, or is there some
other mechanism for controlling this?

We need to try and make as few assumptions as possible or we can't have
a stable binding here. We already have variations on "spin-table", for
example on arm64 where firmware may have a wfe, so the OS *must* issue a
sev to function with all implementations. At the opposite end, PSCI is
pretty much identical across arm and arm64 (at present).

We'll probably want to factor the enable methods out into a separate
file or directory, as I can see this getting quite large.

> 
> >
> > I don't want to see this devolve into meaning "whatever the Linux driver
> > for this happens to do at the current point in time", as that just leads
> > to breakage as we have no clear distinction between actual requirements
> > and implementation details.
> >
> > Given we already have platforms without an enable-method, we can't make
> > it a required property either -- those platforms are already booting by
> > figuring out an enable method from the platform's compatible string.
> So, you recommend we continue to using the platform compatible string as
> well ?

I'm not making a recommendation to do that, but we need to consider the
trade-off. Adding a new enable-method for each platform and then finding
it can't be shared because we didn't document the general case is
something I'd like to avoid.

> We currently don't have a perfect method to use enable-method in generic
> code. More on this below...

We certainly need to factor out enable-method detection into generic
code. Ideally, if we don't have an smp_ops we'd fall back to
enable-method detection.

> >
> > With PSCI, enable-method also implies a method for disabling a
> > particular CPU, so it would be nice for the description to cover this.
> >
> >>
> >>   Example:
> >>
> >> diff --git a/Documentation/devicetree/bindings/arm/msm/scss.txt b/Documentation/devicetree/bindings/arm/msm/scss.txt
> >> new file mode 100644
> >> index 0000000..21c3e26
> >> --- /dev/null
> >> +++ b/Documentation/devicetree/bindings/arm/msm/scss.txt
> >> @@ -0,0 +1,15 @@
> >> +* SCSS - Scorpion Sub-system
> >> +
> >> +Properties
> >> +
> >> +- compatible : Should contain "qcom,scss".
> >> +
> >> +- reg: Specifies the base address for the SCSS registers used for
> >> +       booting up secondary cores.
> >> +
> >> +Example:
> >> +
> >> +    scss at 902000 {
> >> +            compatible = "qcom,scss";
> >> +            reg = <0x00902000 0x2000>;
> >> +    };
> >> diff --git a/arch/arm/boot/dts/msm8660-surf.dts b/arch/arm/boot/dts/msm8660-surf.dts
> >> index cdc010e..203e51a 100644
> >> --- a/arch/arm/boot/dts/msm8660-surf.dts
> >> +++ b/arch/arm/boot/dts/msm8660-surf.dts
> >> @@ -7,6 +7,22 @@
> >>      compatible = "qcom,msm8660-surf", "qcom,msm8660";
> >>      interrupt-parent = <&intc>;
> >>
> >> +    cpus {
> >> +            #address-cells = <1>;
> >> +            #size-cells = <0>;
> >> +            compatible = "qcom,scorpion";
> >> +            device_type = "cpu";
> >> +            enable-method = "qcom,scss";
> >> +
> >> +            cpu at 0 {
> >> +                    reg = <0>;
> >> +            };
> >> +
> >> +            cpu at 1 {
> >> +                    reg = <1>;
> >> +            };
> >> +    };
> >> +
> > I *really* like moving the common properties out into the /cpus node --
> > ePAPR suggests it, it's less error prone, and it takes up less space.
> > However, I'm not sure our generic/arch code handles it properly, and I
> > think we need to audit that before we can start writing dts that depend
> > on it. I'd be happy to be wrong here if anyone can correct me. :)

I was trying to point out that here we have the properties under /cpus,
but the PSCI code in kvmtool currently drops these per-cpu. Either is
technically valid, and the recommendation of ePAPR is to look in a given
/cpus/cpu at N node first, then fall back to the /cpus node.

> >
> > We probably need to factor out the way we read properties for cpu nodes,
> > falling back to ones present in /cpus if not present. There's already a
> > lot of pain getting the node for a logical (rather than physical) cpu
> > id.
> >
> > Sudeep recently factored out finding the cpu node for a logical cpu id
> > [1,2] in generic code with a per-arch callback, it shouldn't be too hard
> > to have shims around that to read (optionally) common properties.
> >

This was my attempt at a suggestion. We want some generic code for
grabbing cpu properties that falls back to the /cpus node. Something
like a family of of_get_cpu_{property,u32,string} functions (we likely
don't need much more than string at the moment).

Sudeep's patches for getting the cpu node for a logical cpu id could be
the basis for this.

> > [...]
> >
> >> -static void prepare_cold_cpu(unsigned int cpu)
> >> +static int scorpion_release_secondary(void)
> >>   {
> >> -    int ret;
> >> -    ret = scm_set_boot_addr(virt_to_phys(secondary_startup),
> >> -                            SCM_FLAG_COLDBOOT_CPU1);
> >> -    if (ret == 0) {
> >> -            void __iomem *sc1_base_ptr;
> >> -            sc1_base_ptr = ioremap_nocache(0x00902000, SZ_4K*2);
> >> -            if (sc1_base_ptr) {
> >> -                    writel(0, sc1_base_ptr + VDD_SC1_ARRAY_CLAMP_GFS_CTL);
> >> -                    writel(0, sc1_base_ptr + SCSS_CPU1CORE_RESET);
> >> -                    writel(3, sc1_base_ptr + SCSS_DBG_STATUS_CORE_PWRDUP);
> >> -                    iounmap(sc1_base_ptr);
> >> -            }
> >> -    } else
> >> -            printk(KERN_DEBUG "Failed to set secondary core boot "
> >> -                              "address\n");
> >> +    void __iomem *sc1_base_ptr;
> >> +    struct device_node *dn = NULL;
> >> +
> >> +    dn = of_find_compatible_node(dn, NULL, "qcom,scss");
> >> +    if (!dn) {
> >> +            pr_err("%s: Missing scss node in device tree\n", __func__);
> >> +            return -ENXIO;
> >> +    }
> >> +
> >> +    sc1_base_ptr = of_iomap(dn, 0);
> >> +    if (sc1_base_ptr) {
> >> +            writel_relaxed(0, sc1_base_ptr + VDD_SC1_ARRAY_CLAMP_GFS_CTL);
> >> +            writel_relaxed(0, sc1_base_ptr + SCSS_CPU1CORE_RESET);
> >> +            writel_relaxed(3, sc1_base_ptr + SCSS_DBG_STATUS_CORE_PWRDUP);
> >> +            mb();
> >> +            iounmap(sc1_base_ptr);
> > Does this boot *all* secondary CPUs directly into the kernel? Is that
> > safe (e.g. if the kernel supports fewer CPUs than are physically
> > present)?
> Im not sure I understand the concern with safety here. The CPU for which
> the release_secondary will be called will be taken out of reset with
> this sequence.

I was confused that this function was no longer taking a cpu property.
To me that seems to imply that what this was doing was not specific to a
particular CPU,

I must have misunderstood something here -- how does this take a
particular CPU out of reset? It looks like it's poking some shared
registers in the "qcom,scss" device that aren't specific to a single
CPU.

Is there only one secondary? How does this generalise to more?

> 
> >
> > Is this a one-time thing, or are we able to dynamically hotplug CPUs? If
> > the latter, the map/unmap looks odd to me.
> 
> This is a one-time thing done for each CPU that's specified in the
> device tree (or based on the command line over-ride).

Ok. Is it possible for hot unplug to be implemented based on this later?

> >> +    } else {
> >> +            return -ENOMEM;
> >> +    }
> >> +
> >> +    return 0;
> >>   }
> >>
> >> -static int msm_boot_secondary(unsigned int cpu, struct task_struct *idle)
> >> +static DEFINE_PER_CPU(int, cold_boot_done);
> >> +
> >> +static void boot_cold_cpu(unsigned int cpu)
> >>   {
> >> -    static int cold_boot_done;
> >> +    const char *enable_method;
> >> +    struct device_node *dn = NULL;
> >>
> >> -    /* Only need to bring cpu out of reset this way once */
> >> -    if (cold_boot_done == false) {
> >> -            prepare_cold_cpu(cpu);
> >> -            cold_boot_done = true;
> >> +    dn = of_find_node_by_name(dn, "cpus");
> >> +    if (!dn) {
> >> +            pr_err("%s: Missing node cpus in device tree\n", __func__);
> >> +            return;
> >>      }
> >>
> >> +    enable_method = of_get_property(dn, "enable-method", NULL);
> > Please factor this out from the platform code.
> >
> > If we're going to use enable-method, it should be decoupled from
> > platform code -- common code should parse it to find the appropriate
> > handler. Also, we *must* support having an enable-method per-cpu as KVM
> > does for PSCI (though I definitely would like support for shared
> > properties as mentioned above).
> Currently with most platforms, smp.c calls into the boot_secondary
> function which decides which cpu it is
> and then calls the appropriate function. This is because smp ops allow
> only 1 callback function to be registered...
> Hence, using the enable-method in platform specific code works.
> 
> If we create a generic property should it mandate having generic code
> handle that ?

I'd certainly prefer to move enable-method handling into generic code.
Is there no way we could do this if we had a NULL smp_ops entry?

> I currently don't have a good way of incorporating enable-method in
> generic code as it would mean to establish a mechanism to
> associate the enable-method string with cpu specific boot_secondary
> functions.

If we have cpu-specific boot_secondary functions, then surely the
enable-method isn't enough to tell us how to boot the secondary?

Does the cpu's compatible string not give us enough, or is this still
platform-specific? If it's the latter, I'm not sure having the
enable-method buys us anything, as we still need additional platform
information to be able to use it.

> You have an approach in mind that I can try ?

Not at present, but I'm sure we can think of something.

> 
> >> +    if (!enable_method) {
> >> +                    pr_err("%s: cpus node is missing enable-method property\n",
> >> +                                    __func__);
> >> +    } else if (!strcmp(enable_method, "qcom,scss")) {
> >> +            if (per_cpu(cold_boot_done, cpu) == false) {
> >> +                    scorpion_release_secondary();
> >> +                    per_cpu(cold_boot_done, cpu) = true;
> >> +            }
> >> +    } else {
> >> +            pr_err("%s: Invalid enable-method property: %s\n",
> >> +                            __func__, enable_method);
> >> +    }
> >> +}
> >> +
> >> +static int msm_boot_secondary(unsigned int cpu, struct task_struct *idle)
> >> +{
> >> +    boot_cold_cpu(cpu);
> >> +
> >>      /*
> >>       * set synchronisation state between this boot processor
> >>       * and the secondary one
> >> @@ -118,8 +148,28 @@ static void __init msm_smp_init_cpus(void)
> >>              set_cpu_possible(i, true);
> >>   }
> >>
> >> +static const int cold_boot_flags[] __initconst = {
> >> +    0,
> >> +    SCM_FLAG_COLDBOOT_CPU1,
> >> +};
> > So we only ever support booting two CPUs?
> The next patch which adds support for a quadcore chip, adds more flags.
> >
> > Is the mapping of MPIDR to register bits arbitrary? Or do we know what
> > they would be for four CPUs, four clusters, and beyond?
> Four cpus.

Ok. is SCSS based booting always going to be hard-limited to four CPUs,
or do we possibly need to encode some (optional) information in each CPU
node in the dt (i.e. the bits of the register that needs to be poked).

> 
> >
> >> +
> >>   static void __init msm_smp_prepare_cpus(unsigned int max_cpus)
> >>   {
> >> +    int cpu, map;
> >> +    unsigned int flags = 0;
> >> +
> >> +    for_each_present_cpu(cpu) {
> >> +            map = cpu_logical_map(cpu);
> >> +            if (map > ARRAY_SIZE(cold_boot_flags)) {
> >> +                    set_cpu_present(cpu, false);
> >> +                    __WARN();
> >> +                    continue;
> >> +            }
> >> +            flags |= cold_boot_flags[map];
> > It's a shame that this leaves a window where some CPUs seem bootable,
> > but aren't (though I can't offer a better way of handling this given we
> > have dts without enable-method properties).
> Any CPU that seems bootable and is defined in DT, should be bootable.
> The cold_boot_flags are just a mechanism to set appropriate values for
> the scm call and
> they aren't used as a method to disallow CPUs from being bootable and
> hence the __WARN() if this is done incorrectly.

Ok. That sounds reasonable.

Thanks,
Mark.

Re: [RESEND PATCH 4/4] ARM: msm: Add support for 8974 SMP

[Mark Rutland]

On Wed, Aug 14, 2013 at 11:38:44PM +0100, Rohit Vaswani wrote:
> On 8/12/2013 9:39 AM, Mark Rutland wrote:
> > On Fri, Aug 02, 2013 at 03:15:25AM +0100, Rohit Vaswani wrote:
> >> Add the cpus bindings and the Kraitv2 release sequence
> >> to make SMP work for 2 cores on MSM8974.
> >>
> >> Signed-off-by: Rohit Vaswani <rvaswani@codeaurora.org>
> >> ---
> >>   Documentation/devicetree/bindings/arm/cpus.txt |  1 +
> >>   arch/arm/boot/dts/msm8974.dts                  | 23 ++++++++
> >>   arch/arm/mach-msm/board-dt-8974.c              |  3 +
> >>   arch/arm/mach-msm/platsmp.c                    | 79 ++++++++++++++++++++++++++
> >>   4 files changed, 106 insertions(+)
> >>
> >> diff --git a/Documentation/devicetree/bindings/arm/cpus.txt b/Documentation/devicetree/bindings/arm/cpus.txt
> >> index 1132eac..7c3c677 100644
> >> --- a/Documentation/devicetree/bindings/arm/cpus.txt
> >> +++ b/Documentation/devicetree/bindings/arm/cpus.txt
> >> @@ -52,6 +52,7 @@ For the ARM architecture every CPU node must contain the following properties:
> >>   		 This should be one of:
> >>   		 "qcom,scss"
> >>   		 "qcom,kpssv1"
> >> +		 "qcom,kpssv2"
> > I guess I should've looked at the whole series before responding, that
> > answers my prior question about what variation we expect.
> >
> >>   
> >>   Example:
> >>   
> >> diff --git a/arch/arm/boot/dts/msm8974.dts b/arch/arm/boot/dts/msm8974.dts
> >> index c31c097..ef35a9b 100644
> >> --- a/arch/arm/boot/dts/msm8974.dts
> >> +++ b/arch/arm/boot/dts/msm8974.dts
> >> @@ -7,6 +7,22 @@
> >>   	compatible = "qcom,msm8974";
> >>   	interrupt-parent = <&intc>;
> >>   
> >> +	cpus {
> >> +		#address-cells = <1>;
> >> +		#size-cells = <0>;
> >> +		compatible = "qcom,krait";
> >> +		device_type = "cpu";
> >> +		enable-method = "qcom,kpssv2";
> >> +
> >> +		cpu@0 {
> >> +			reg = <0>;
> >> +		};
> >> +
> >> +		cpu@1 {
> >> +			reg = <1>;
> >> +		};
> >> +	};
> >> +
> >>   	intc: interrupt-controller@f9000000 {
> >>   		compatible = "qcom,msm-qgic2";
> >>   		interrupt-controller;
> >> @@ -23,4 +39,11 @@
> >>   			     <1 1 0xf08>;
> >>   		clock-frequency = <19200000>;
> >>   	};
> >> +
> >> +	kpss@f9012000 {
> >> +		compatible = "qcom,kpss";
> >> +		reg = <0xf9012000 0x1000>,
> >> +		      <0xf9088000 0x1000>,
> >> +		      <0xf9098000 0x1000>;
> >> +	};
> > In the previous examples, the number of CPUs was equal to the number of
> > kpss reg values. Why does it differ here. Either:
> >
> > * We always have the extra regsiter here, and it should be described
> >    even if we don't use it.
> >
> > * It's a different hardware block, and needs a more specific
> >    compatible string.
> >
> > Eitehr way this strengthens my feeling that we need to define the linkage
> > from a CPU to the portion of the kpss which affects it.
> Will add documentation for each of the registers. We have one for each 
> CPU and one within the KPSS (Krait Processor Sub-System) e.g the L2 saw 
> base in this case.

Ok.

So the previous example had no L2 saw base?

> >
> >>   };
> >> diff --git a/arch/arm/mach-msm/board-dt-8974.c b/arch/arm/mach-msm/board-dt-8974.c
> >> index d7f84f2..06119f9 100644
> >> --- a/arch/arm/mach-msm/board-dt-8974.c
> >> +++ b/arch/arm/mach-msm/board-dt-8974.c
> >> @@ -13,11 +13,14 @@
> >>   #include <linux/of_platform.h>
> >>   #include <asm/mach/arch.h>
> >>   
> >> +#include "common.h"
> >> +
> >>   static const char * const msm8974_dt_match[] __initconst = {
> >>   	"qcom,msm8974",
> >>   	NULL
> >>   };
> >>   
> >>   DT_MACHINE_START(MSM8974_DT, "Qualcomm MSM (Flattened Device Tree)")
> >> +	.smp = smp_ops(msm_smp_ops),
> >>   	.dt_compat = msm8974_dt_match,
> >>   MACHINE_END
> >> diff --git a/arch/arm/mach-msm/platsmp.c b/arch/arm/mach-msm/platsmp.c
> >> index 82eb079..0fdae69 100644
> >> --- a/arch/arm/mach-msm/platsmp.c
> >> +++ b/arch/arm/mach-msm/platsmp.c
> >> @@ -124,6 +124,80 @@ static int msm8960_release_secondary(unsigned int cpu)
> >>   	return 0;
> >>   }
> >>   
> >> +static int msm8974_release_secondary(unsigned int cpu)
> >> +{
> >> +	void __iomem *reg;
> >> +	void __iomem *l2_saw_base;
> >> +	struct device_node *dn = NULL;
> >> +	unsigned apc_pwr_gate_ctl = 0x14;
> >> +	unsigned reg_val;
> >> +
> >> +	if (cpu == 0 || cpu >= num_possible_cpus())
> >> +		return -EINVAL;
> >> +
> >> +	dn = of_find_compatible_node(dn, NULL, "qcom,kpss");
> >> +	if (!dn) {
> >> +		pr_err("%s : Missing kpss node from device tree\n", __func__);
> >> +		return -ENXIO;
> >> +	}
> >> +
> >> +	reg = of_iomap(dn, cpu+1);
> > This looks very fishy given the prior patch being one off from this.
> > why is reg[0] now different?
> >
> >> +	if (!reg)
> >> +		return -ENOMEM;
> >> +
> >> +	pr_debug("Starting secondary CPU %d\n", cpu);
> >> +
> >> +	/* Turn on the BHS, turn off LDO Bypass and power down LDO */
> >> +	reg_val =  0x403f0001;
> > Magic number?
> It represents the comment above it. It didnt seem clean to define 4 
> different offsets with #defines within a single register for the purpose 
> of 1 write.

In general I'd prefer having symbolic names, but I'm not going to argue
here.

> >
> >> +	writel_relaxed(reg_val, reg + apc_pwr_gate_ctl);
> >> +
> >> +	/* complete the above write before the delay */
> >> +	mb();
> > Use writel?
> >
> >> +	/* wait for the bhs to settle */
> >> +	udelay(1);
> >> +
> >> +	/* Turn on BHS segments */
> >> +	reg_val |= 0x3f << 1;
> >> +	writel_relaxed(reg_val, reg + apc_pwr_gate_ctl);
> >> +
> >> +	/* complete the above write before the delay */
> >> +	mb();
> > Use writel again?
> >
> >> +	 /* wait for the bhs to settle */
> >> +	udelay(1);
> >> +
> >> +	/* Finally turn on the bypass so that BHS supplies power */
> >> +	reg_val |= 0x3f << 8;
> >> +	writel_relaxed(reg_val, reg + apc_pwr_gate_ctl);
> >> +
> >> +	/* enable max phases */
> >> +	l2_saw_base = of_iomap(dn, 0);
> >> +	if (!l2_saw_base) {
> >> +		return -ENOMEM;
> >> +	}
> > What?
> >
> > You've just lost your only reference to the mapping in reg.
> >
> > Why do you not do this at the start, before poking everything else? Even
> > better, do it at probe time and fail once rather than for each CPU you
> > have no chance of bringing up.
> Will do.
> 
> >
> > [...]
> >>   static void boot_cold_cpu(unsigned int cpu)
> >> @@ -151,6 +225,11 @@ static void boot_cold_cpu(unsigned int cpu)
> >>   			msm8960_release_secondary(cpu);
> >>   			per_cpu(cold_boot_done, cpu) = true;
> >>   		}
> >> +	} else if (!strcmp(enable_method, "qcom,kpssv2")) {
> >> +		if (per_cpu(cold_boot_done, cpu) == false) {
> >> +			msm8974_release_secondary(cpu);
> >> +			per_cpu(cold_boot_done, cpu) = true;
> >> +		}
> >>   	} else {
> >>   		pr_err("%s: Invalid enable-method property: %s\n",
> >>   				__func__, enable_method);
> > The enable-method parsing really should be moved out to common code. We
> > could make it scan the enable-method if a machine has no smp ops (which
> > is more general than the PSCI fallback that's been suggested before).
> But we have smp ops like every other SoC. I might need to go back to the 
> drawing board for incorporating enable-method into generic code.
> Any suggestions are appreciated.

I think we need generic infrastructure that checks if we have a NULL
smp_ops and tries to find one based on any enable-method properties in
the dt.

> If enable-method seems cumbersome to be enforced for every SoC, I would 
> be comfortable using the cpu compatible string as you suggested in the 
> previous patch.

I'm not sure cpu compatible string alone gives us enough, the
integration of the SoC and particular board will affect how we bring up
secondaries.

Thanks,
Mark.

Re: [RESEND PATCH 4/4] ARM: msm: Add support for 8974 SMP

[Mark Rutland]

On Wed, Aug 14, 2013 at 11:38:44PM +0100, Rohit Vaswani wrote:
> On 8/12/2013 9:39 AM, Mark Rutland wrote:
> > On Fri, Aug 02, 2013 at 03:15:25AM +0100, Rohit Vaswani wrote:
> >> Add the cpus bindings and the Kraitv2 release sequence
> >> to make SMP work for 2 cores on MSM8974.
> >>
> >> Signed-off-by: Rohit Vaswani <rvaswani@codeaurora.org>
> >> ---
> >>   Documentation/devicetree/bindings/arm/cpus.txt |  1 +
> >>   arch/arm/boot/dts/msm8974.dts                  | 23 ++++++++
> >>   arch/arm/mach-msm/board-dt-8974.c              |  3 +
> >>   arch/arm/mach-msm/platsmp.c                    | 79 ++++++++++++++++++++++++++
> >>   4 files changed, 106 insertions(+)
> >>
> >> diff --git a/Documentation/devicetree/bindings/arm/cpus.txt b/Documentation/devicetree/bindings/arm/cpus.txt
> >> index 1132eac..7c3c677 100644
> >> --- a/Documentation/devicetree/bindings/arm/cpus.txt
> >> +++ b/Documentation/devicetree/bindings/arm/cpus.txt
> >> @@ -52,6 +52,7 @@ For the ARM architecture every CPU node must contain the following properties:
> >>   		 This should be one of:
> >>   		 "qcom,scss"
> >>   		 "qcom,kpssv1"
> >> +		 "qcom,kpssv2"
> > I guess I should've looked at the whole series before responding, that
> > answers my prior question about what variation we expect.
> >
> >>   
> >>   Example:
> >>   
> >> diff --git a/arch/arm/boot/dts/msm8974.dts b/arch/arm/boot/dts/msm8974.dts
> >> index c31c097..ef35a9b 100644
> >> --- a/arch/arm/boot/dts/msm8974.dts
> >> +++ b/arch/arm/boot/dts/msm8974.dts
> >> @@ -7,6 +7,22 @@
> >>   	compatible = "qcom,msm8974";
> >>   	interrupt-parent = <&intc>;
> >>   
> >> +	cpus {
> >> +		#address-cells = <1>;
> >> +		#size-cells = <0>;
> >> +		compatible = "qcom,krait";
> >> +		device_type = "cpu";
> >> +		enable-method = "qcom,kpssv2";
> >> +
> >> +		cpu@0 {
> >> +			reg = <0>;
> >> +		};
> >> +
> >> +		cpu@1 {
> >> +			reg = <1>;
> >> +		};
> >> +	};
> >> +
> >>   	intc: interrupt-controller@f9000000 {
> >>   		compatible = "qcom,msm-qgic2";
> >>   		interrupt-controller;
> >> @@ -23,4 +39,11 @@
> >>   			     <1 1 0xf08>;
> >>   		clock-frequency = <19200000>;
> >>   	};
> >> +
> >> +	kpss@f9012000 {
> >> +		compatible = "qcom,kpss";
> >> +		reg = <0xf9012000 0x1000>,
> >> +		      <0xf9088000 0x1000>,
> >> +		      <0xf9098000 0x1000>;
> >> +	};
> > In the previous examples, the number of CPUs was equal to the number of
> > kpss reg values. Why does it differ here. Either:
> >
> > * We always have the extra regsiter here, and it should be described
> >    even if we don't use it.
> >
> > * It's a different hardware block, and needs a more specific
> >    compatible string.
> >
> > Eitehr way this strengthens my feeling that we need to define the linkage
> > from a CPU to the portion of the kpss which affects it.
> Will add documentation for each of the registers. We have one for each 
> CPU and one within the KPSS (Krait Processor Sub-System) e.g the L2 saw 
> base in this case.

Ok.

So the previous example had no L2 saw base?

> >
> >>   };
> >> diff --git a/arch/arm/mach-msm/board-dt-8974.c b/arch/arm/mach-msm/board-dt-8974.c
> >> index d7f84f2..06119f9 100644
> >> --- a/arch/arm/mach-msm/board-dt-8974.c
> >> +++ b/arch/arm/mach-msm/board-dt-8974.c
> >> @@ -13,11 +13,14 @@
> >>   #include <linux/of_platform.h>
> >>   #include <asm/mach/arch.h>
> >>   
> >> +#include "common.h"
> >> +
> >>   static const char * const msm8974_dt_match[] __initconst = {
> >>   	"qcom,msm8974",
> >>   	NULL
> >>   };
> >>   
> >>   DT_MACHINE_START(MSM8974_DT, "Qualcomm MSM (Flattened Device Tree)")
> >> +	.smp = smp_ops(msm_smp_ops),
> >>   	.dt_compat = msm8974_dt_match,
> >>   MACHINE_END
> >> diff --git a/arch/arm/mach-msm/platsmp.c b/arch/arm/mach-msm/platsmp.c
> >> index 82eb079..0fdae69 100644
> >> --- a/arch/arm/mach-msm/platsmp.c
> >> +++ b/arch/arm/mach-msm/platsmp.c
> >> @@ -124,6 +124,80 @@ static int msm8960_release_secondary(unsigned int cpu)
> >>   	return 0;
> >>   }
> >>   
> >> +static int msm8974_release_secondary(unsigned int cpu)
> >> +{
> >> +	void __iomem *reg;
> >> +	void __iomem *l2_saw_base;
> >> +	struct device_node *dn = NULL;
> >> +	unsigned apc_pwr_gate_ctl = 0x14;
> >> +	unsigned reg_val;
> >> +
> >> +	if (cpu == 0 || cpu >= num_possible_cpus())
> >> +		return -EINVAL;
> >> +
> >> +	dn = of_find_compatible_node(dn, NULL, "qcom,kpss");
> >> +	if (!dn) {
> >> +		pr_err("%s : Missing kpss node from device tree\n", __func__);
> >> +		return -ENXIO;
> >> +	}
> >> +
> >> +	reg = of_iomap(dn, cpu+1);
> > This looks very fishy given the prior patch being one off from this.
> > why is reg[0] now different?
> >
> >> +	if (!reg)
> >> +		return -ENOMEM;
> >> +
> >> +	pr_debug("Starting secondary CPU %d\n", cpu);
> >> +
> >> +	/* Turn on the BHS, turn off LDO Bypass and power down LDO */
> >> +	reg_val =  0x403f0001;
> > Magic number?
> It represents the comment above it. It didnt seem clean to define 4 
> different offsets with #defines within a single register for the purpose 
> of 1 write.

In general I'd prefer having symbolic names, but I'm not going to argue
here.

> >
> >> +	writel_relaxed(reg_val, reg + apc_pwr_gate_ctl);
> >> +
> >> +	/* complete the above write before the delay */
> >> +	mb();
> > Use writel?
> >
> >> +	/* wait for the bhs to settle */
> >> +	udelay(1);
> >> +
> >> +	/* Turn on BHS segments */
> >> +	reg_val |= 0x3f << 1;
> >> +	writel_relaxed(reg_val, reg + apc_pwr_gate_ctl);
> >> +
> >> +	/* complete the above write before the delay */
> >> +	mb();
> > Use writel again?
> >
> >> +	 /* wait for the bhs to settle */
> >> +	udelay(1);
> >> +
> >> +	/* Finally turn on the bypass so that BHS supplies power */
> >> +	reg_val |= 0x3f << 8;
> >> +	writel_relaxed(reg_val, reg + apc_pwr_gate_ctl);
> >> +
> >> +	/* enable max phases */
> >> +	l2_saw_base = of_iomap(dn, 0);
> >> +	if (!l2_saw_base) {
> >> +		return -ENOMEM;
> >> +	}
> > What?
> >
> > You've just lost your only reference to the mapping in reg.
> >
> > Why do you not do this at the start, before poking everything else? Even
> > better, do it at probe time and fail once rather than for each CPU you
> > have no chance of bringing up.
> Will do.
> 
> >
> > [...]
> >>   static void boot_cold_cpu(unsigned int cpu)
> >> @@ -151,6 +225,11 @@ static void boot_cold_cpu(unsigned int cpu)
> >>   			msm8960_release_secondary(cpu);
> >>   			per_cpu(cold_boot_done, cpu) = true;
> >>   		}
> >> +	} else if (!strcmp(enable_method, "qcom,kpssv2")) {
> >> +		if (per_cpu(cold_boot_done, cpu) == false) {
> >> +			msm8974_release_secondary(cpu);
> >> +			per_cpu(cold_boot_done, cpu) = true;
> >> +		}
> >>   	} else {
> >>   		pr_err("%s: Invalid enable-method property: %s\n",
> >>   				__func__, enable_method);
> > The enable-method parsing really should be moved out to common code. We
> > could make it scan the enable-method if a machine has no smp ops (which
> > is more general than the PSCI fallback that's been suggested before).
> But we have smp ops like every other SoC. I might need to go back to the 
> drawing board for incorporating enable-method into generic code.
> Any suggestions are appreciated.

I think we need generic infrastructure that checks if we have a NULL
smp_ops and tries to find one based on any enable-method properties in
the dt.

> If enable-method seems cumbersome to be enforced for every SoC, I would 
> be comfortable using the cpu compatible string as you suggested in the 
> previous patch.

I'm not sure cpu compatible string alone gives us enough, the
integration of the SoC and particular board will affect how we bring up
secondaries.

Thanks,
Mark.

[RESEND PATCH 4/4] ARM: msm: Add support for 8974 SMP

[Mark Rutland]

On Wed, Aug 14, 2013 at 11:38:44PM +0100, Rohit Vaswani wrote:
> On 8/12/2013 9:39 AM, Mark Rutland wrote:
> > On Fri, Aug 02, 2013 at 03:15:25AM +0100, Rohit Vaswani wrote:
> >> Add the cpus bindings and the Kraitv2 release sequence
> >> to make SMP work for 2 cores on MSM8974.
> >>
> >> Signed-off-by: Rohit Vaswani <rvaswani@codeaurora.org>
> >> ---
> >>   Documentation/devicetree/bindings/arm/cpus.txt |  1 +
> >>   arch/arm/boot/dts/msm8974.dts                  | 23 ++++++++
> >>   arch/arm/mach-msm/board-dt-8974.c              |  3 +
> >>   arch/arm/mach-msm/platsmp.c                    | 79 ++++++++++++++++++++++++++
> >>   4 files changed, 106 insertions(+)
> >>
> >> diff --git a/Documentation/devicetree/bindings/arm/cpus.txt b/Documentation/devicetree/bindings/arm/cpus.txt
> >> index 1132eac..7c3c677 100644
> >> --- a/Documentation/devicetree/bindings/arm/cpus.txt
> >> +++ b/Documentation/devicetree/bindings/arm/cpus.txt
> >> @@ -52,6 +52,7 @@ For the ARM architecture every CPU node must contain the following properties:
> >>   		 This should be one of:
> >>   		 "qcom,scss"
> >>   		 "qcom,kpssv1"
> >> +		 "qcom,kpssv2"
> > I guess I should've looked at the whole series before responding, that
> > answers my prior question about what variation we expect.
> >
> >>   
> >>   Example:
> >>   
> >> diff --git a/arch/arm/boot/dts/msm8974.dts b/arch/arm/boot/dts/msm8974.dts
> >> index c31c097..ef35a9b 100644
> >> --- a/arch/arm/boot/dts/msm8974.dts
> >> +++ b/arch/arm/boot/dts/msm8974.dts
> >> @@ -7,6 +7,22 @@
> >>   	compatible = "qcom,msm8974";
> >>   	interrupt-parent = <&intc>;
> >>   
> >> +	cpus {
> >> +		#address-cells = <1>;
> >> +		#size-cells = <0>;
> >> +		compatible = "qcom,krait";
> >> +		device_type = "cpu";
> >> +		enable-method = "qcom,kpssv2";
> >> +
> >> +		cpu at 0 {
> >> +			reg = <0>;
> >> +		};
> >> +
> >> +		cpu at 1 {
> >> +			reg = <1>;
> >> +		};
> >> +	};
> >> +
> >>   	intc: interrupt-controller at f9000000 {
> >>   		compatible = "qcom,msm-qgic2";
> >>   		interrupt-controller;
> >> @@ -23,4 +39,11 @@
> >>   			     <1 1 0xf08>;
> >>   		clock-frequency = <19200000>;
> >>   	};
> >> +
> >> +	kpss at f9012000 {
> >> +		compatible = "qcom,kpss";
> >> +		reg = <0xf9012000 0x1000>,
> >> +		      <0xf9088000 0x1000>,
> >> +		      <0xf9098000 0x1000>;
> >> +	};
> > In the previous examples, the number of CPUs was equal to the number of
> > kpss reg values. Why does it differ here. Either:
> >
> > * We always have the extra regsiter here, and it should be described
> >    even if we don't use it.
> >
> > * It's a different hardware block, and needs a more specific
> >    compatible string.
> >
> > Eitehr way this strengthens my feeling that we need to define the linkage
> > from a CPU to the portion of the kpss which affects it.
> Will add documentation for each of the registers. We have one for each 
> CPU and one within the KPSS (Krait Processor Sub-System) e.g the L2 saw 
> base in this case.

Ok.

So the previous example had no L2 saw base?

> >
> >>   };
> >> diff --git a/arch/arm/mach-msm/board-dt-8974.c b/arch/arm/mach-msm/board-dt-8974.c
> >> index d7f84f2..06119f9 100644
> >> --- a/arch/arm/mach-msm/board-dt-8974.c
> >> +++ b/arch/arm/mach-msm/board-dt-8974.c
> >> @@ -13,11 +13,14 @@
> >>   #include <linux/of_platform.h>
> >>   #include <asm/mach/arch.h>
> >>   
> >> +#include "common.h"
> >> +
> >>   static const char * const msm8974_dt_match[] __initconst = {
> >>   	"qcom,msm8974",
> >>   	NULL
> >>   };
> >>   
> >>   DT_MACHINE_START(MSM8974_DT, "Qualcomm MSM (Flattened Device Tree)")
> >> +	.smp = smp_ops(msm_smp_ops),
> >>   	.dt_compat = msm8974_dt_match,
> >>   MACHINE_END
> >> diff --git a/arch/arm/mach-msm/platsmp.c b/arch/arm/mach-msm/platsmp.c
> >> index 82eb079..0fdae69 100644
> >> --- a/arch/arm/mach-msm/platsmp.c
> >> +++ b/arch/arm/mach-msm/platsmp.c
> >> @@ -124,6 +124,80 @@ static int msm8960_release_secondary(unsigned int cpu)
> >>   	return 0;
> >>   }
> >>   
> >> +static int msm8974_release_secondary(unsigned int cpu)
> >> +{
> >> +	void __iomem *reg;
> >> +	void __iomem *l2_saw_base;
> >> +	struct device_node *dn = NULL;
> >> +	unsigned apc_pwr_gate_ctl = 0x14;
> >> +	unsigned reg_val;
> >> +
> >> +	if (cpu == 0 || cpu >= num_possible_cpus())
> >> +		return -EINVAL;
> >> +
> >> +	dn = of_find_compatible_node(dn, NULL, "qcom,kpss");
> >> +	if (!dn) {
> >> +		pr_err("%s : Missing kpss node from device tree\n", __func__);
> >> +		return -ENXIO;
> >> +	}
> >> +
> >> +	reg = of_iomap(dn, cpu+1);
> > This looks very fishy given the prior patch being one off from this.
> > why is reg[0] now different?
> >
> >> +	if (!reg)
> >> +		return -ENOMEM;
> >> +
> >> +	pr_debug("Starting secondary CPU %d\n", cpu);
> >> +
> >> +	/* Turn on the BHS, turn off LDO Bypass and power down LDO */
> >> +	reg_val =  0x403f0001;
> > Magic number?
> It represents the comment above it. It didnt seem clean to define 4 
> different offsets with #defines within a single register for the purpose 
> of 1 write.

In general I'd prefer having symbolic names, but I'm not going to argue
here.

> >
> >> +	writel_relaxed(reg_val, reg + apc_pwr_gate_ctl);
> >> +
> >> +	/* complete the above write before the delay */
> >> +	mb();
> > Use writel?
> >
> >> +	/* wait for the bhs to settle */
> >> +	udelay(1);
> >> +
> >> +	/* Turn on BHS segments */
> >> +	reg_val |= 0x3f << 1;
> >> +	writel_relaxed(reg_val, reg + apc_pwr_gate_ctl);
> >> +
> >> +	/* complete the above write before the delay */
> >> +	mb();
> > Use writel again?
> >
> >> +	 /* wait for the bhs to settle */
> >> +	udelay(1);
> >> +
> >> +	/* Finally turn on the bypass so that BHS supplies power */
> >> +	reg_val |= 0x3f << 8;
> >> +	writel_relaxed(reg_val, reg + apc_pwr_gate_ctl);
> >> +
> >> +	/* enable max phases */
> >> +	l2_saw_base = of_iomap(dn, 0);
> >> +	if (!l2_saw_base) {
> >> +		return -ENOMEM;
> >> +	}
> > What?
> >
> > You've just lost your only reference to the mapping in reg.
> >
> > Why do you not do this at the start, before poking everything else? Even
> > better, do it at probe time and fail once rather than for each CPU you
> > have no chance of bringing up.
> Will do.
> 
> >
> > [...]
> >>   static void boot_cold_cpu(unsigned int cpu)
> >> @@ -151,6 +225,11 @@ static void boot_cold_cpu(unsigned int cpu)
> >>   			msm8960_release_secondary(cpu);
> >>   			per_cpu(cold_boot_done, cpu) = true;
> >>   		}
> >> +	} else if (!strcmp(enable_method, "qcom,kpssv2")) {
> >> +		if (per_cpu(cold_boot_done, cpu) == false) {
> >> +			msm8974_release_secondary(cpu);
> >> +			per_cpu(cold_boot_done, cpu) = true;
> >> +		}
> >>   	} else {
> >>   		pr_err("%s: Invalid enable-method property: %s\n",
> >>   				__func__, enable_method);
> > The enable-method parsing really should be moved out to common code. We
> > could make it scan the enable-method if a machine has no smp ops (which
> > is more general than the PSCI fallback that's been suggested before).
> But we have smp ops like every other SoC. I might need to go back to the 
> drawing board for incorporating enable-method into generic code.
> Any suggestions are appreciated.

I think we need generic infrastructure that checks if we have a NULL
smp_ops and tries to find one based on any enable-method properties in
the dt.

> If enable-method seems cumbersome to be enforced for every SoC, I would 
> be comfortable using the cpu compatible string as you suggested in the 
> previous patch.

I'm not sure cpu compatible string alone gives us enough, the
integration of the SoC and particular board will affect how we bring up
secondaries.

Thanks,
Mark.

Re: [RESEND PATCH 2/4] ARM: msm: Re-organize platsmp to make it extensible

[David Rientjes]

On Mon, 12 Aug 2013, Mark Rutland wrote:

> >  static void __init msm_smp_prepare_cpus(unsigned int max_cpus)
> >  {
> > +	int cpu, map;
> > +	unsigned int flags = 0;
> > +
> > +	for_each_present_cpu(cpu) {
> > +		map = cpu_logical_map(cpu);
> > +		if (map > ARRAY_SIZE(cold_boot_flags)) {
> > +			set_cpu_present(cpu, false);
> > +			__WARN();
> > +			continue;
> > +		}
> > +		flags |= cold_boot_flags[map];

__WARN() can't be used in generic code because it's possible to have 
CONFIG_BUG=n, you probably want something like WARN_ON(1) instead.

Re: [RESEND PATCH 2/4] ARM: msm: Re-organize platsmp to make it extensible

[David Rientjes]

On Mon, 12 Aug 2013, Mark Rutland wrote:

> >  static void __init msm_smp_prepare_cpus(unsigned int max_cpus)
> >  {
> > +	int cpu, map;
> > +	unsigned int flags = 0;
> > +
> > +	for_each_present_cpu(cpu) {
> > +		map = cpu_logical_map(cpu);
> > +		if (map > ARRAY_SIZE(cold_boot_flags)) {
> > +			set_cpu_present(cpu, false);
> > +			__WARN();
> > +			continue;
> > +		}
> > +		flags |= cold_boot_flags[map];

__WARN() can't be used in generic code because it's possible to have 
CONFIG_BUG=n, you probably want something like WARN_ON(1) instead.

[RESEND PATCH 2/4] ARM: msm: Re-organize platsmp to make it extensible

[David Rientjes]

On Mon, 12 Aug 2013, Mark Rutland wrote:

> >  static void __init msm_smp_prepare_cpus(unsigned int max_cpus)
> >  {
> > +	int cpu, map;
> > +	unsigned int flags = 0;
> > +
> > +	for_each_present_cpu(cpu) {
> > +		map = cpu_logical_map(cpu);
> > +		if (map > ARRAY_SIZE(cold_boot_flags)) {
> > +			set_cpu_present(cpu, false);
> > +			__WARN();
> > +			continue;
> > +		}
> > +		flags |= cold_boot_flags[map];

__WARN() can't be used in generic code because it's possible to have 
CONFIG_BUG=n, you probably want something like WARN_ON(1) instead.