[U-Boot] [PATCH 1/2] ARM: move #ifdef to match the error handling code

[U-Boot] [PATCH 1/2] ARM: move #ifdef to match the error handling code

[Masahiro Yamada]

Match the #ifdef ... #endif and the code,

   ret = do_something();
   if (ret)
           return ret;

This will make it easier to add more #ifdef'ed code.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
---

 arch/arm/lib/bootm-fdt.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/arch/arm/lib/bootm-fdt.c b/arch/arm/lib/bootm-fdt.c
index 7677358..76b75d8 100644
--- a/arch/arm/lib/bootm-fdt.c
+++ b/arch/arm/lib/bootm-fdt.c
@@ -42,11 +42,14 @@ int arch_fixup_fdt(void *blob)
 	}
 
 	ret = fdt_fixup_memory_banks(blob, start, size, CONFIG_NR_DRAM_BANKS);
-#ifdef CONFIG_ARMV7_NONSEC
 	if (ret)
 		return ret;
 
+#ifdef CONFIG_ARMV7_NONSEC
 	ret = psci_update_dt(blob);
+	if (ret)
+		return ret;
 #endif
-	return ret;
+
+	return 0;
 }
-- 
1.9.1

[U-Boot] [PATCH 2/2] arm64: add better spin-table support

[Masahiro Yamada]

There are two enable methods supported by ARM64 Linux; psci and
spin-table.  The latter is simpler and easier to use for quick SoC
bring-up.

So, I used the spin-table for my first ARMv8 SoC porting, but I
found its support in U-Boot was poor.  It is true there exists a
code fragment for the spin code in arch/arm/cpu/armv8/start.S,
but I see some problems:

  - We must hard-code CPU_RELEASE_ADDR so that it matches the
    "cpu-release-addr" property in the DT that comes from the
    kernel tree.

  - The Documentation/arm64/booting.txt in Linux requires that
    the release address must be zero-initialized, but it is not
    cared by the common code in U-Boot.  So, we must do it in a
    board specific manner.

  - There is no systematic way to protect the spin code from the
    kernel.  U-Boot relocates itself during the boot, so it is
    difficult to predict where the spin code will be located
    after the relocation, which makes it even more difficult to
    hard-code /memreserve/ in the DT of the kernel.  One possible
    work-around would be to pre-fetch the spin-code into the
    I-cache of secondary CPUs, but this is an unsafe solution.

So, here is a patch to solve the problems.  In this approach, the DT
is run-time modified to reserve the spin code (+ cpu-release-addr).
Also, the "cpu-release-addr" property is set to an appropriate
address after the relocation, which means we no longer need the
hard-coded CPU_RELEASE_ADDR.

Currently this patch only supports ARMv8, but theoretically nothing
about the spin-table is arch-specific.  Perhaps, we might want to
support it on PowerPC in the future.  So, I put the DT fixup code
into the common/ directory.  Very little code must be written in
assembler, which went to the arch/arm/cpu/armv8/ directory.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
---

 arch/arm/cpu/armv8/Kconfig         | 18 +++++++++++
 arch/arm/cpu/armv8/Makefile        |  1 +
 arch/arm/cpu/armv8/spin_table_v8.S | 22 ++++++++++++++
 arch/arm/cpu/armv8/start.S         | 10 +++---
 arch/arm/lib/bootm-fdt.c           |  7 +++++
 common/Makefile                    |  1 +
 common/spin_table.c                | 62 ++++++++++++++++++++++++++++++++++++++
 include/spin_table.h               | 14 +++++++++
 8 files changed, 131 insertions(+), 4 deletions(-)
 create mode 100644 arch/arm/cpu/armv8/spin_table_v8.S
 create mode 100644 common/spin_table.c
 create mode 100644 include/spin_table.h

diff --git a/arch/arm/cpu/armv8/Kconfig b/arch/arm/cpu/armv8/Kconfig
index 3d19bbf..019b625 100644
--- a/arch/arm/cpu/armv8/Kconfig
+++ b/arch/arm/cpu/armv8/Kconfig
@@ -3,4 +3,22 @@ if ARM64
 config ARMV8_MULTIENTRY
         boolean "Enable multiple CPUs to enter into U-Boot"
 
+config SPIN_TABLE
+	bool "Support spin-table enable method"
+	depends on ARMV8_MULTIENTRY && OF_LIBFDT
+	help
+	  Say Y here to support "spin-table" enable method for booting Linux.
+
+	  To use this feature, you must do:
+	    - Specify enable-method = "spin-table" in each CPU node in the
+	      Device Tree you are using to boot the kernel
+	    - Let secondary CPUs in U-Boot (in a board specific manner)
+	      before the master CPU jumps to the kernel
+
+	  U-Boot automatically does:
+	    - Set "cpu-release-addr" property of each CPU node
+	      (overwrites it if already exists).
+	    - Reserve the code for the spin-table and the release address
+	      via a /memreserve/ region in the Device Tree.
+
 endif
diff --git a/arch/arm/cpu/armv8/Makefile b/arch/arm/cpu/armv8/Makefile
index 1c85aa9..2e3f421 100644
--- a/arch/arm/cpu/armv8/Makefile
+++ b/arch/arm/cpu/armv8/Makefile
@@ -15,6 +15,7 @@ obj-y	+= cache.o
 obj-y	+= tlb.o
 obj-y	+= transition.o
 obj-y	+= fwcall.o
+obj-$(CONFIG_SPIN_TABLE) += spin_table_v8.o
 
 obj-$(CONFIG_FSL_LAYERSCAPE) += fsl-layerscape/
 obj-$(CONFIG_ARCH_ZYNQMP) += zynqmp/
diff --git a/arch/arm/cpu/armv8/spin_table_v8.S b/arch/arm/cpu/armv8/spin_table_v8.S
new file mode 100644
index 0000000..2f1bd61
--- /dev/null
+++ b/arch/arm/cpu/armv8/spin_table_v8.S
@@ -0,0 +1,22 @@
+/*
+ * Copyright (C) 2016 Masahiro Yamada <yamada.masahiro@socionext.com>
+ *
+ * SPDX-License-Identifier:	GPL-2.0+
+ */
+
+#include <linux/linkage.h>
+
+ENTRY(spin_table_secondary_jump)
+.globl spin_table_reserve_begin
+spin_table_reserve_begin:
+0:	wfe
+	ldr	x0, spin_table_cpu_release_addr
+	cbz	x0, 0b
+	br	x0
+.globl spin_table_cpu_release_addr
+	.align	3
+spin_table_cpu_release_addr:
+	.quad	0
+.globl spin_table_reserve_end
+spin_table_reserve_end:
+ENDPROC(spin_table_secondary_jump)
diff --git a/arch/arm/cpu/armv8/start.S b/arch/arm/cpu/armv8/start.S
index 670e323..15ca864 100644
--- a/arch/arm/cpu/armv8/start.S
+++ b/arch/arm/cpu/armv8/start.S
@@ -94,7 +94,11 @@ reset:
 	/* Processor specific initialization */
 	bl	lowlevel_init
 
-#ifdef CONFIG_ARMV8_MULTIENTRY
+#if defined(CONFIG_SPIN_TABLE)
+	branch_if_master x0, x1, master_cpu
+	b	spin_table_secondary_jump
+	/* never return */
+#elif defined(CONFIG_ARMV8_MULTIENTRY)
 	branch_if_master x0, x1, master_cpu
 
 	/*
@@ -106,10 +110,8 @@ slave_cpu:
 	ldr	x0, [x1]
 	cbz	x0, slave_cpu
 	br	x0			/* branch to the given address */
-master_cpu:
-	/* On the master CPU */
 #endif /* CONFIG_ARMV8_MULTIENTRY */
-
+master_cpu:
 	bl	_main
 
 #ifdef CONFIG_SYS_RESET_SCTRL
diff --git a/arch/arm/lib/bootm-fdt.c b/arch/arm/lib/bootm-fdt.c
index 76b75d8..ec581ac 100644
--- a/arch/arm/lib/bootm-fdt.c
+++ b/arch/arm/lib/bootm-fdt.c
@@ -21,6 +21,7 @@
 #include <asm/armv7.h>
 #endif
 #include <asm/psci.h>
+#include <spin_table.h>
 
 DECLARE_GLOBAL_DATA_PTR;
 
@@ -51,5 +52,11 @@ int arch_fixup_fdt(void *blob)
 		return ret;
 #endif
 
+#ifdef CONFIG_SPIN_TABLE
+	ret = spin_table_update_dt(blob);
+	if (ret)
+		return ret;
+#endif
+
 	return 0;
 }
diff --git a/common/Makefile b/common/Makefile
index 1557a04..f4eb381 100644
--- a/common/Makefile
+++ b/common/Makefile
@@ -141,6 +141,7 @@ obj-$(CONFIG_ANDROID_BOOT_IMAGE) += image-android.o
 obj-$(CONFIG_$(SPL_)OF_LIBFDT) += image-fdt.o
 obj-$(CONFIG_$(SPL_)FIT) += image-fit.o
 obj-$(CONFIG_$(SPL_)FIT_SIGNATURE) += image-sig.o
+obj-$(CONFIG_$(SPL_)SPIN_TABLE) += spin_table.o
 obj-$(CONFIG_IO_TRACE) += iotrace.o
 obj-y += memsize.o
 obj-y += stdio.o
diff --git a/common/spin_table.c b/common/spin_table.c
new file mode 100644
index 0000000..046af8f
--- /dev/null
+++ b/common/spin_table.c
@@ -0,0 +1,62 @@
+/*
+ * Copyright (C) 2016 Masahiro Yamada <yamada.masahiro@socionext.com>
+ *
+ * SPDX-License-Identifier:	GPL-2.0+
+ */
+
+#include <common.h>
+#include <libfdt.h>
+#include <spin_table.h>
+
+int spin_table_update_dt(void *fdt)
+{
+	int cpus_offset, offset;
+	const char *prop;
+	int ret;
+	unsigned long rsv_addr = (unsigned long)&spin_table_reserve_begin;
+	unsigned long rsv_size = &spin_table_reserve_end -
+						&spin_table_reserve_begin;
+
+	cpus_offset = fdt_path_offset(fdt, "/cpus");
+	if (cpus_offset < 0)
+		return -ENODEV;
+
+	for (offset = fdt_first_subnode(fdt, cpus_offset);
+	     offset >= 0;
+	     offset = fdt_next_subnode(fdt, offset)) {
+		prop = fdt_getprop(fdt, offset, "device_type", NULL);
+		if (!prop || strcmp(prop, "cpu"))
+			continue;
+
+		/*
+		 * In the first loop, we check if every CPU node specifies
+		 * spin-table.  Otherwise, just return successfully to not
+		 * disturb other methods, like psci.
+		 */
+		prop = fdt_getprop(fdt, offset, "enable-method", NULL);
+		if (!prop || strcmp(prop, "spin-table"))
+			return 0;
+	}
+
+	for (offset = fdt_first_subnode(fdt, cpus_offset);
+	     offset >= 0;
+	     offset = fdt_next_subnode(fdt, offset)) {
+		prop = fdt_getprop(fdt, offset, "device_type", NULL);
+		if (!prop || strcmp(prop, "cpu"))
+			continue;
+
+		ret = fdt_setprop_u64(fdt, offset, "cpu-release-addr",
+				(unsigned long)&spin_table_cpu_release_addr);
+		if (ret)
+			return -ENOSPC;
+	}
+
+	ret = fdt_add_mem_rsv(fdt, rsv_addr, rsv_size);
+	if (ret)
+		return -ENOSPC;
+
+	printf("   Reserved memory region for spin-table: addr=%lx size=%lx\n",
+	       rsv_addr, rsv_size);
+
+	return 0;
+}
diff --git a/include/spin_table.h b/include/spin_table.h
new file mode 100644
index 0000000..cf455a1
--- /dev/null
+++ b/include/spin_table.h
@@ -0,0 +1,14 @@
+/*
+ * SPDX-License-Identifier:	GPL-2.0+
+ */
+
+#ifndef __SPIN_TABLE_H__
+#define __SPIN_TABLE_H__
+
+extern u64 spin_table_cpu_release_addr;
+extern char spin_table_reserve_begin;
+extern char spin_table_reserve_end;
+
+int spin_table_update_dt(void *fdt);
+
+#endif /* __SPIN_TABLE_H__ */
-- 
1.9.1

[U-Boot] [PATCH 2/2] arm64: add better spin-table support

[Mark Rutland]

On Fri, Jun 17, 2016 at 09:51:49PM +0900, Masahiro Yamada wrote:
> There are two enable methods supported by ARM64 Linux; psci and
> spin-table.  The latter is simpler and easier to use for quick SoC
> bring-up.
> 
> So, I used the spin-table for my first ARMv8 SoC porting, but I
> found its support in U-Boot was poor.  It is true there exists a
> code fragment for the spin code in arch/arm/cpu/armv8/start.S,
> but I see some problems:
> 
>   - We must hard-code CPU_RELEASE_ADDR so that it matches the
>     "cpu-release-addr" property in the DT that comes from the
>     kernel tree.
> 
>   - The Documentation/arm64/booting.txt in Linux requires that
>     the release address must be zero-initialized, but it is not
>     cared by the common code in U-Boot.  So, we must do it in a
>     board specific manner.
> 
>   - There is no systematic way to protect the spin code from the
>     kernel.  U-Boot relocates itself during the boot, so it is
>     difficult to predict where the spin code will be located
>     after the relocation, which makes it even more difficult to
>     hard-code /memreserve/ in the DT of the kernel.  One possible
>     work-around would be to pre-fetch the spin-code into the
>     I-cache of secondary CPUs, but this is an unsafe solution.
> 
> So, here is a patch to solve the problems.  In this approach, the DT
> is run-time modified to reserve the spin code (+ cpu-release-addr).
> Also, the "cpu-release-addr" property is set to an appropriate
> address after the relocation, which means we no longer need the
> hard-coded CPU_RELEASE_ADDR.
> 
> Currently this patch only supports ARMv8, but theoretically nothing
> about the spin-table is arch-specific.  Perhaps, we might want to
> support it on PowerPC in the future.  So, I put the DT fixup code
> into the common/ directory.  Very little code must be written in
> assembler, which went to the arch/arm/cpu/armv8/ directory.

It's worth noting that while both arm64 and PPC have something called
"spin-table", they're not quite the same. The arm64 version just reused
the enable-method and property names for something different (and
arch-specific).

I have no strong feelings about where the code lives, however.

Otherwise, these sound like useful improvements.

Thanks,
Mark.

> 
> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
> ---
> 
>  arch/arm/cpu/armv8/Kconfig         | 18 +++++++++++
>  arch/arm/cpu/armv8/Makefile        |  1 +
>  arch/arm/cpu/armv8/spin_table_v8.S | 22 ++++++++++++++
>  arch/arm/cpu/armv8/start.S         | 10 +++---
>  arch/arm/lib/bootm-fdt.c           |  7 +++++
>  common/Makefile                    |  1 +
>  common/spin_table.c                | 62 ++++++++++++++++++++++++++++++++++++++
>  include/spin_table.h               | 14 +++++++++
>  8 files changed, 131 insertions(+), 4 deletions(-)
>  create mode 100644 arch/arm/cpu/armv8/spin_table_v8.S
>  create mode 100644 common/spin_table.c
>  create mode 100644 include/spin_table.h
> 
> diff --git a/arch/arm/cpu/armv8/Kconfig b/arch/arm/cpu/armv8/Kconfig
> index 3d19bbf..019b625 100644
> --- a/arch/arm/cpu/armv8/Kconfig
> +++ b/arch/arm/cpu/armv8/Kconfig
> @@ -3,4 +3,22 @@ if ARM64
>  config ARMV8_MULTIENTRY
>          boolean "Enable multiple CPUs to enter into U-Boot"
>  
> +config SPIN_TABLE
> +	bool "Support spin-table enable method"
> +	depends on ARMV8_MULTIENTRY && OF_LIBFDT
> +	help
> +	  Say Y here to support "spin-table" enable method for booting Linux.
> +
> +	  To use this feature, you must do:
> +	    - Specify enable-method = "spin-table" in each CPU node in the
> +	      Device Tree you are using to boot the kernel
> +	    - Let secondary CPUs in U-Boot (in a board specific manner)
> +	      before the master CPU jumps to the kernel
> +
> +	  U-Boot automatically does:
> +	    - Set "cpu-release-addr" property of each CPU node
> +	      (overwrites it if already exists).
> +	    - Reserve the code for the spin-table and the release address
> +	      via a /memreserve/ region in the Device Tree.
> +
>  endif
> diff --git a/arch/arm/cpu/armv8/Makefile b/arch/arm/cpu/armv8/Makefile
> index 1c85aa9..2e3f421 100644
> --- a/arch/arm/cpu/armv8/Makefile
> +++ b/arch/arm/cpu/armv8/Makefile
> @@ -15,6 +15,7 @@ obj-y	+= cache.o
>  obj-y	+= tlb.o
>  obj-y	+= transition.o
>  obj-y	+= fwcall.o
> +obj-$(CONFIG_SPIN_TABLE) += spin_table_v8.o
>  
>  obj-$(CONFIG_FSL_LAYERSCAPE) += fsl-layerscape/
>  obj-$(CONFIG_ARCH_ZYNQMP) += zynqmp/
> diff --git a/arch/arm/cpu/armv8/spin_table_v8.S b/arch/arm/cpu/armv8/spin_table_v8.S
> new file mode 100644
> index 0000000..2f1bd61
> --- /dev/null
> +++ b/arch/arm/cpu/armv8/spin_table_v8.S
> @@ -0,0 +1,22 @@
> +/*
> + * Copyright (C) 2016 Masahiro Yamada <yamada.masahiro@socionext.com>
> + *
> + * SPDX-License-Identifier:	GPL-2.0+
> + */
> +
> +#include <linux/linkage.h>
> +
> +ENTRY(spin_table_secondary_jump)
> +.globl spin_table_reserve_begin
> +spin_table_reserve_begin:
> +0:	wfe
> +	ldr	x0, spin_table_cpu_release_addr
> +	cbz	x0, 0b
> +	br	x0
> +.globl spin_table_cpu_release_addr
> +	.align	3
> +spin_table_cpu_release_addr:
> +	.quad	0
> +.globl spin_table_reserve_end
> +spin_table_reserve_end:
> +ENDPROC(spin_table_secondary_jump)
> diff --git a/arch/arm/cpu/armv8/start.S b/arch/arm/cpu/armv8/start.S
> index 670e323..15ca864 100644
> --- a/arch/arm/cpu/armv8/start.S
> +++ b/arch/arm/cpu/armv8/start.S
> @@ -94,7 +94,11 @@ reset:
>  	/* Processor specific initialization */
>  	bl	lowlevel_init
>  
> -#ifdef CONFIG_ARMV8_MULTIENTRY
> +#if defined(CONFIG_SPIN_TABLE)
> +	branch_if_master x0, x1, master_cpu
> +	b	spin_table_secondary_jump
> +	/* never return */
> +#elif defined(CONFIG_ARMV8_MULTIENTRY)
>  	branch_if_master x0, x1, master_cpu
>  
>  	/*
> @@ -106,10 +110,8 @@ slave_cpu:
>  	ldr	x0, [x1]
>  	cbz	x0, slave_cpu
>  	br	x0			/* branch to the given address */
> -master_cpu:
> -	/* On the master CPU */
>  #endif /* CONFIG_ARMV8_MULTIENTRY */
> -
> +master_cpu:
>  	bl	_main
>  
>  #ifdef CONFIG_SYS_RESET_SCTRL
> diff --git a/arch/arm/lib/bootm-fdt.c b/arch/arm/lib/bootm-fdt.c
> index 76b75d8..ec581ac 100644
> --- a/arch/arm/lib/bootm-fdt.c
> +++ b/arch/arm/lib/bootm-fdt.c
> @@ -21,6 +21,7 @@
>  #include <asm/armv7.h>
>  #endif
>  #include <asm/psci.h>
> +#include <spin_table.h>
>  
>  DECLARE_GLOBAL_DATA_PTR;
>  
> @@ -51,5 +52,11 @@ int arch_fixup_fdt(void *blob)
>  		return ret;
>  #endif
>  
> +#ifdef CONFIG_SPIN_TABLE
> +	ret = spin_table_update_dt(blob);
> +	if (ret)
> +		return ret;
> +#endif
> +
>  	return 0;
>  }
> diff --git a/common/Makefile b/common/Makefile
> index 1557a04..f4eb381 100644
> --- a/common/Makefile
> +++ b/common/Makefile
> @@ -141,6 +141,7 @@ obj-$(CONFIG_ANDROID_BOOT_IMAGE) += image-android.o
>  obj-$(CONFIG_$(SPL_)OF_LIBFDT) += image-fdt.o
>  obj-$(CONFIG_$(SPL_)FIT) += image-fit.o
>  obj-$(CONFIG_$(SPL_)FIT_SIGNATURE) += image-sig.o
> +obj-$(CONFIG_$(SPL_)SPIN_TABLE) += spin_table.o
>  obj-$(CONFIG_IO_TRACE) += iotrace.o
>  obj-y += memsize.o
>  obj-y += stdio.o
> diff --git a/common/spin_table.c b/common/spin_table.c
> new file mode 100644
> index 0000000..046af8f
> --- /dev/null
> +++ b/common/spin_table.c
> @@ -0,0 +1,62 @@
> +/*
> + * Copyright (C) 2016 Masahiro Yamada <yamada.masahiro@socionext.com>
> + *
> + * SPDX-License-Identifier:	GPL-2.0+
> + */
> +
> +#include <common.h>
> +#include <libfdt.h>
> +#include <spin_table.h>
> +
> +int spin_table_update_dt(void *fdt)
> +{
> +	int cpus_offset, offset;
> +	const char *prop;
> +	int ret;
> +	unsigned long rsv_addr = (unsigned long)&spin_table_reserve_begin;
> +	unsigned long rsv_size = &spin_table_reserve_end -
> +						&spin_table_reserve_begin;
> +
> +	cpus_offset = fdt_path_offset(fdt, "/cpus");
> +	if (cpus_offset < 0)
> +		return -ENODEV;
> +
> +	for (offset = fdt_first_subnode(fdt, cpus_offset);
> +	     offset >= 0;
> +	     offset = fdt_next_subnode(fdt, offset)) {
> +		prop = fdt_getprop(fdt, offset, "device_type", NULL);
> +		if (!prop || strcmp(prop, "cpu"))
> +			continue;
> +
> +		/*
> +		 * In the first loop, we check if every CPU node specifies
> +		 * spin-table.  Otherwise, just return successfully to not
> +		 * disturb other methods, like psci.
> +		 */
> +		prop = fdt_getprop(fdt, offset, "enable-method", NULL);
> +		if (!prop || strcmp(prop, "spin-table"))
> +			return 0;
> +	}
> +
> +	for (offset = fdt_first_subnode(fdt, cpus_offset);
> +	     offset >= 0;
> +	     offset = fdt_next_subnode(fdt, offset)) {
> +		prop = fdt_getprop(fdt, offset, "device_type", NULL);
> +		if (!prop || strcmp(prop, "cpu"))
> +			continue;
> +
> +		ret = fdt_setprop_u64(fdt, offset, "cpu-release-addr",
> +				(unsigned long)&spin_table_cpu_release_addr);
> +		if (ret)
> +			return -ENOSPC;
> +	}
> +
> +	ret = fdt_add_mem_rsv(fdt, rsv_addr, rsv_size);
> +	if (ret)
> +		return -ENOSPC;
> +
> +	printf("   Reserved memory region for spin-table: addr=%lx size=%lx\n",
> +	       rsv_addr, rsv_size);
> +
> +	return 0;
> +}
> diff --git a/include/spin_table.h b/include/spin_table.h
> new file mode 100644
> index 0000000..cf455a1
> --- /dev/null
> +++ b/include/spin_table.h
> @@ -0,0 +1,14 @@
> +/*
> + * SPDX-License-Identifier:	GPL-2.0+
> + */
> +
> +#ifndef __SPIN_TABLE_H__
> +#define __SPIN_TABLE_H__
> +
> +extern u64 spin_table_cpu_release_addr;
> +extern char spin_table_reserve_begin;
> +extern char spin_table_reserve_end;
> +
> +int spin_table_update_dt(void *fdt);
> +
> +#endif /* __SPIN_TABLE_H__ */
> -- 
> 1.9.1
> 

[U-Boot] [PATCH 2/2] arm64: add better spin-table support

[york sun]

On 06/17/2016 05:51 AM, Masahiro Yamada wrote:
> There are two enable methods supported by ARM64 Linux; psci and
> spin-table.  The latter is simpler and easier to use for quick SoC
> bring-up.
>
> So, I used the spin-table for my first ARMv8 SoC porting, but I
> found its support in U-Boot was poor.  It is true there exists a
> code fragment for the spin code in arch/arm/cpu/armv8/start.S,
> but I see some problems:
>
>    - We must hard-code CPU_RELEASE_ADDR so that it matches the
>      "cpu-release-addr" property in the DT that comes from the
>      kernel tree.
>
>    - The Documentation/arm64/booting.txt in Linux requires that
>      the release address must be zero-initialized, but it is not
>      cared by the common code in U-Boot.  So, we must do it in a
>      board specific manner.
>
>    - There is no systematic way to protect the spin code from the
>      kernel.  U-Boot relocates itself during the boot, so it is
>      difficult to predict where the spin code will be located
>      after the relocation, which makes it even more difficult to
>      hard-code /memreserve/ in the DT of the kernel.  One possible
>      work-around would be to pre-fetch the spin-code into the
>      I-cache of secondary CPUs, but this is an unsafe solution.
>
> So, here is a patch to solve the problems.  In this approach, the DT
> is run-time modified to reserve the spin code (+ cpu-release-addr).
> Also, the "cpu-release-addr" property is set to an appropriate
> address after the relocation, which means we no longer need the
> hard-coded CPU_RELEASE_ADDR.
>
> Currently this patch only supports ARMv8, but theoretically nothing
> about the spin-table is arch-specific.  Perhaps, we might want to
> support it on PowerPC in the future.  So, I put the DT fixup code
> into the common/ directory.  Very little code must be written in
> assembler, which went to the arch/arm/cpu/armv8/ directory.
>
> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
> ---
>
>   arch/arm/cpu/armv8/Kconfig         | 18 +++++++++++
>   arch/arm/cpu/armv8/Makefile        |  1 +
>   arch/arm/cpu/armv8/spin_table_v8.S | 22 ++++++++++++++
>   arch/arm/cpu/armv8/start.S         | 10 +++---
>   arch/arm/lib/bootm-fdt.c           |  7 +++++
>   common/Makefile                    |  1 +
>   common/spin_table.c                | 62 ++++++++++++++++++++++++++++++++++++++
>   include/spin_table.h               | 14 +++++++++
>   8 files changed, 131 insertions(+), 4 deletions(-)
>   create mode 100644 arch/arm/cpu/armv8/spin_table_v8.S
>   create mode 100644 common/spin_table.c
>   create mode 100644 include/spin_table.h
>
> diff --git a/arch/arm/cpu/armv8/Kconfig b/arch/arm/cpu/armv8/Kconfig
> index 3d19bbf..019b625 100644
> --- a/arch/arm/cpu/armv8/Kconfig
> +++ b/arch/arm/cpu/armv8/Kconfig
> @@ -3,4 +3,22 @@ if ARM64
>   config ARMV8_MULTIENTRY
>           boolean "Enable multiple CPUs to enter into U-Boot"
>
> +config SPIN_TABLE
> +	bool "Support spin-table enable method"
> +	depends on ARMV8_MULTIENTRY && OF_LIBFDT
> +	help
> +	  Say Y here to support "spin-table" enable method for booting Linux.
> +
> +	  To use this feature, you must do:
> +	    - Specify enable-method = "spin-table" in each CPU node in the
> +	      Device Tree you are using to boot the kernel
> +	    - Let secondary CPUs in U-Boot (in a board specific manner)
> +	      before the master CPU jumps to the kernel
> +
> +	  U-Boot automatically does:
> +	    - Set "cpu-release-addr" property of each CPU node
> +	      (overwrites it if already exists).
> +	    - Reserve the code for the spin-table and the release address
> +	      via a /memreserve/ region in the Device Tree.
> +
>   endif
> diff --git a/arch/arm/cpu/armv8/Makefile b/arch/arm/cpu/armv8/Makefile
> index 1c85aa9..2e3f421 100644
> --- a/arch/arm/cpu/armv8/Makefile
> +++ b/arch/arm/cpu/armv8/Makefile
> @@ -15,6 +15,7 @@ obj-y	+= cache.o
>   obj-y	+= tlb.o
>   obj-y	+= transition.o
>   obj-y	+= fwcall.o
> +obj-$(CONFIG_SPIN_TABLE) += spin_table_v8.o
>
>   obj-$(CONFIG_FSL_LAYERSCAPE) += fsl-layerscape/
>   obj-$(CONFIG_ARCH_ZYNQMP) += zynqmp/
> diff --git a/arch/arm/cpu/armv8/spin_table_v8.S b/arch/arm/cpu/armv8/spin_table_v8.S
> new file mode 100644
> index 0000000..2f1bd61
> --- /dev/null
> +++ b/arch/arm/cpu/armv8/spin_table_v8.S
> @@ -0,0 +1,22 @@
> +/*
> + * Copyright (C) 2016 Masahiro Yamada <yamada.masahiro@socionext.com>
> + *
> + * SPDX-License-Identifier:	GPL-2.0+
> + */
> +
> +#include <linux/linkage.h>
> +
> +ENTRY(spin_table_secondary_jump)
> +.globl spin_table_reserve_begin
> +spin_table_reserve_begin:
> +0:	wfe
> +	ldr	x0, spin_table_cpu_release_addr
> +	cbz	x0, 0b
> +	br	x0
> +.globl spin_table_cpu_release_addr
> +	.align	3
> +spin_table_cpu_release_addr:
> +	.quad	0
> +.globl spin_table_reserve_end
> +spin_table_reserve_end:
> +ENDPROC(spin_table_secondary_jump)
> diff --git a/arch/arm/cpu/armv8/start.S b/arch/arm/cpu/armv8/start.S
> index 670e323..15ca864 100644
> --- a/arch/arm/cpu/armv8/start.S
> +++ b/arch/arm/cpu/armv8/start.S
> @@ -94,7 +94,11 @@ reset:
>   	/* Processor specific initialization */
>   	bl	lowlevel_init
>
> -#ifdef CONFIG_ARMV8_MULTIENTRY
> +#if defined(CONFIG_SPIN_TABLE)
> +	branch_if_master x0, x1, master_cpu
> +	b	spin_table_secondary_jump
> +	/* never return */
> +#elif defined(CONFIG_ARMV8_MULTIENTRY)
>   	branch_if_master x0, x1, master_cpu
>
>   	/*
> @@ -106,10 +110,8 @@ slave_cpu:
>   	ldr	x0, [x1]
>   	cbz	x0, slave_cpu
>   	br	x0			/* branch to the given address */
> -master_cpu:
> -	/* On the master CPU */
>   #endif /* CONFIG_ARMV8_MULTIENTRY */
> -
> +master_cpu:
>   	bl	_main
>
>   #ifdef CONFIG_SYS_RESET_SCTRL
> diff --git a/arch/arm/lib/bootm-fdt.c b/arch/arm/lib/bootm-fdt.c
> index 76b75d8..ec581ac 100644
> --- a/arch/arm/lib/bootm-fdt.c
> +++ b/arch/arm/lib/bootm-fdt.c
> @@ -21,6 +21,7 @@
>   #include <asm/armv7.h>
>   #endif
>   #include <asm/psci.h>
> +#include <spin_table.h>
>
>   DECLARE_GLOBAL_DATA_PTR;
>
> @@ -51,5 +52,11 @@ int arch_fixup_fdt(void *blob)
>   		return ret;
>   #endif
>
> +#ifdef CONFIG_SPIN_TABLE
> +	ret = spin_table_update_dt(blob);
> +	if (ret)
> +		return ret;
> +#endif
> +
>   	return 0;
>   }
> diff --git a/common/Makefile b/common/Makefile
> index 1557a04..f4eb381 100644
> --- a/common/Makefile
> +++ b/common/Makefile
> @@ -141,6 +141,7 @@ obj-$(CONFIG_ANDROID_BOOT_IMAGE) += image-android.o
>   obj-$(CONFIG_$(SPL_)OF_LIBFDT) += image-fdt.o
>   obj-$(CONFIG_$(SPL_)FIT) += image-fit.o
>   obj-$(CONFIG_$(SPL_)FIT_SIGNATURE) += image-sig.o
> +obj-$(CONFIG_$(SPL_)SPIN_TABLE) += spin_table.o
>   obj-$(CONFIG_IO_TRACE) += iotrace.o
>   obj-y += memsize.o
>   obj-y += stdio.o
> diff --git a/common/spin_table.c b/common/spin_table.c
> new file mode 100644
> index 0000000..046af8f
> --- /dev/null
> +++ b/common/spin_table.c
> @@ -0,0 +1,62 @@
> +/*
> + * Copyright (C) 2016 Masahiro Yamada <yamada.masahiro@socionext.com>
> + *
> + * SPDX-License-Identifier:	GPL-2.0+
> + */
> +
> +#include <common.h>
> +#include <libfdt.h>
> +#include <spin_table.h>
> +
> +int spin_table_update_dt(void *fdt)
> +{
> +	int cpus_offset, offset;
> +	const char *prop;
> +	int ret;
> +	unsigned long rsv_addr = (unsigned long)&spin_table_reserve_begin;
> +	unsigned long rsv_size = &spin_table_reserve_end -
> +						&spin_table_reserve_begin;
> +
> +	cpus_offset = fdt_path_offset(fdt, "/cpus");
> +	if (cpus_offset < 0)
> +		return -ENODEV;
> +
> +	for (offset = fdt_first_subnode(fdt, cpus_offset);
> +	     offset >= 0;
> +	     offset = fdt_next_subnode(fdt, offset)) {
> +		prop = fdt_getprop(fdt, offset, "device_type", NULL);
> +		if (!prop || strcmp(prop, "cpu"))
> +			continue;
> +
> +		/*
> +		 * In the first loop, we check if every CPU node specifies
> +		 * spin-table.  Otherwise, just return successfully to not
> +		 * disturb other methods, like psci.
> +		 */
> +		prop = fdt_getprop(fdt, offset, "enable-method", NULL);
> +		if (!prop || strcmp(prop, "spin-table"))
> +			return 0;
> +	}
> +
> +	for (offset = fdt_first_subnode(fdt, cpus_offset);
> +	     offset >= 0;
> +	     offset = fdt_next_subnode(fdt, offset)) {
> +		prop = fdt_getprop(fdt, offset, "device_type", NULL);
> +		if (!prop || strcmp(prop, "cpu"))
> +			continue;
> +
> +		ret = fdt_setprop_u64(fdt, offset, "cpu-release-addr",
> +				(unsigned long)&spin_table_cpu_release_addr);
> +		if (ret)
> +			return -ENOSPC;
> +	}
> +
> +	ret = fdt_add_mem_rsv(fdt, rsv_addr, rsv_size);
> +	if (ret)
> +		return -ENOSPC;
> +
> +	printf("   Reserved memory region for spin-table: addr=%lx size=%lx\n",
> +	       rsv_addr, rsv_size);
> +
> +	return 0;
> +}
> diff --git a/include/spin_table.h b/include/spin_table.h
> new file mode 100644
> index 0000000..cf455a1
> --- /dev/null
> +++ b/include/spin_table.h
> @@ -0,0 +1,14 @@
> +/*
> + * SPDX-License-Identifier:	GPL-2.0+
> + */
> +
> +#ifndef __SPIN_TABLE_H__
> +#define __SPIN_TABLE_H__
> +
> +extern u64 spin_table_cpu_release_addr;
> +extern char spin_table_reserve_begin;
> +extern char spin_table_reserve_end;
> +
> +int spin_table_update_dt(void *fdt);
> +
> +#endif /* __SPIN_TABLE_H__ */
>

Masahiro,

Because ARMv8 had no spin table support at time of our bring-up, and the 
core id is SoC specific, we put the code under SoC folder, now 
arch/arm/cpu/armv8/fsl-layerscape/fdt.c and 
arch/arm/cpu/armv8/fsl-layerscape/lowlevel.S. We actually use multiple 
tables. More specifically one table per core. The spin table is 
protected from kernel by fdt_add_mem_rsv() function.

It will be great if we can consolidate the code and reuse what we 
already have.

York

[U-Boot] [PATCH 2/2] arm64: add better spin-table support

[Linus Walleij]

On Fri, Jun 17, 2016 at 2:51 PM, Masahiro Yamada
<yamada.masahiro@socionext.com> wrote:

> There are two enable methods supported by ARM64 Linux; psci and
> spin-table.  The latter is simpler and easier to use for quick SoC
> bring-up.
>
> So, I used the spin-table for my first ARMv8 SoC porting, but I
> found its support in U-Boot was poor.  It is true there exists a
> code fragment for the spin code in arch/arm/cpu/armv8/start.S,
> but I see some problems:

Is part of the motivation for this approach to boot an ARMv8 system
without using the ARM Trusted Firmware?

Yours,
Linus Walleij

[U-Boot] [PATCH 2/2] arm64: add better spin-table support

[Masahiro Yamada]

2016-06-18 18:40 GMT+09:00 Linus Walleij <linus.walleij@linaro.org>:
> On Fri, Jun 17, 2016 at 2:51 PM, Masahiro Yamada
> <yamada.masahiro@socionext.com> wrote:
>
>> There are two enable methods supported by ARM64 Linux; psci and
>> spin-table.  The latter is simpler and easier to use for quick SoC
>> bring-up.
>>
>> So, I used the spin-table for my first ARMv8 SoC porting, but I
>> found its support in U-Boot was poor.  It is true there exists a
>> code fragment for the spin code in arch/arm/cpu/armv8/start.S,
>> but I see some problems:
>
> Is part of the motivation for this approach to boot an ARMv8 system
> without using the ARM Trusted Firmware?
>
> Yours,
> Linus Walleij

Yes, exactly.

It would be the best choice
to switch over to PSCI with ATF in a long run,
but, I decided to use spin-table for the initial SoC bring-up
because of tight schedule.

If I use spin-table, I can keep the boot sequence very simple.


-- 
Best Regards
Masahiro Yamada

[U-Boot] [PATCH 2/2] arm64: add better spin-table support

[Masahiro Yamada]

2016-06-18 0:05 GMT+09:00 Mark Rutland <mark.rutland@arm.com>:
> On Fri, Jun 17, 2016 at 09:51:49PM +0900, Masahiro Yamada wrote:
>> There are two enable methods supported by ARM64 Linux; psci and
>> spin-table.  The latter is simpler and easier to use for quick SoC
>> bring-up.
>>
>> So, I used the spin-table for my first ARMv8 SoC porting, but I
>> found its support in U-Boot was poor.  It is true there exists a
>> code fragment for the spin code in arch/arm/cpu/armv8/start.S,
>> but I see some problems:
>>
>>   - We must hard-code CPU_RELEASE_ADDR so that it matches the
>>     "cpu-release-addr" property in the DT that comes from the
>>     kernel tree.
>>
>>   - The Documentation/arm64/booting.txt in Linux requires that
>>     the release address must be zero-initialized, but it is not
>>     cared by the common code in U-Boot.  So, we must do it in a
>>     board specific manner.
>>
>>   - There is no systematic way to protect the spin code from the
>>     kernel.  U-Boot relocates itself during the boot, so it is
>>     difficult to predict where the spin code will be located
>>     after the relocation, which makes it even more difficult to
>>     hard-code /memreserve/ in the DT of the kernel.  One possible
>>     work-around would be to pre-fetch the spin-code into the
>>     I-cache of secondary CPUs, but this is an unsafe solution.
>>
>> So, here is a patch to solve the problems.  In this approach, the DT
>> is run-time modified to reserve the spin code (+ cpu-release-addr).
>> Also, the "cpu-release-addr" property is set to an appropriate
>> address after the relocation, which means we no longer need the
>> hard-coded CPU_RELEASE_ADDR.
>>
>> Currently this patch only supports ARMv8, but theoretically nothing
>> about the spin-table is arch-specific.  Perhaps, we might want to
>> support it on PowerPC in the future.  So, I put the DT fixup code
>> into the common/ directory.  Very little code must be written in
>> assembler, which went to the arch/arm/cpu/armv8/ directory.
>
> It's worth noting that while both arm64 and PPC have something called
> "spin-table", they're not quite the same. The arm64 version just reused
> the enable-method and property names for something different (and
> arch-specific).
>
> I have no strong feelings about where the code lives, however.
>


I did not know this.  Thanks.

Maybe I should consider moving all the code to arch/arm/cpu/armv8/
and renaming CONFIG_SPIN_TABLE to CONFIG_ARMV8_SPIN_TABLE.


-- 
Best Regards
Masahiro Yamada

[U-Boot] [PATCH 2/2] arm64: add better spin-table support

[André Przywara]

Hi,

On 19/06/16 09:57, Masahiro Yamada wrote:
> 2016-06-18 18:40 GMT+09:00 Linus Walleij <linus.walleij@linaro.org>:
>> On Fri, Jun 17, 2016 at 2:51 PM, Masahiro Yamada
>> <yamada.masahiro@socionext.com> wrote:
>>
>>> There are two enable methods supported by ARM64 Linux; psci and
>>> spin-table.  The latter is simpler and easier to use for quick SoC
>>> bring-up.
>>>
>>> So, I used the spin-table for my first ARMv8 SoC porting, but I
>>> found its support in U-Boot was poor.  It is true there exists a
>>> code fragment for the spin code in arch/arm/cpu/armv8/start.S,
>>> but I see some problems:
>>
>> Is part of the motivation for this approach to boot an ARMv8 system
>> without using the ARM Trusted Firmware?
>>
>> Yours,
>> Linus Walleij
> 
> Yes, exactly.
> 
> It would be the best choice
> to switch over to PSCI with ATF in a long run,
> but, I decided to use spin-table for the initial SoC bring-up
> because of tight schedule.

So if you don't have an ATF port ready, why not use U-Boot's PSCI
implementation meanwhile? I think there are efforts underway to make
PSCI enablement for random new boards a walk in the park (by making the
PSCI support as generic as possible, CCing Chen-Yu for this).

IIRC the spin-table boot method was just introduced to cope with cores
that don't have EL3 and thus cannot provide PSCI services the normal way
(and that don't want to or cannot sacrifice EL2 for that).
So I am a bit wary of proliferating this SMP method.

Wouldn't it be better to help making U-Boot's PSCI stack as easy to use
as possible? I don't see technical reasons that adding PSCI support for
a board should be harder or more involved than adding spin-table support
- in the end you need to tell it about the SMP pen, maybe providing (or
faking?) reset and shutdown for 0.2 compliance.

Cheers,
Andre.

> If I use spin-table, I can keep the boot sequence very simple.

[U-Boot] [PATCH 2/2] arm64: add better spin-table support

[york sun]

On 06/19/2016 03:34 AM, Andr? Przywara wrote:
> Hi,
>
> On 19/06/16 09:57, Masahiro Yamada wrote:
>> 2016-06-18 18:40 GMT+09:00 Linus Walleij <linus.walleij@linaro.org>:
>>> On Fri, Jun 17, 2016 at 2:51 PM, Masahiro Yamada
>>> <yamada.masahiro@socionext.com> wrote:
>>>
>>>> There are two enable methods supported by ARM64 Linux; psci and
>>>> spin-table.  The latter is simpler and easier to use for quick SoC
>>>> bring-up.
>>>>
>>>> So, I used the spin-table for my first ARMv8 SoC porting, but I
>>>> found its support in U-Boot was poor.  It is true there exists a
>>>> code fragment for the spin code in arch/arm/cpu/armv8/start.S,
>>>> but I see some problems:
>>>
>>> Is part of the motivation for this approach to boot an ARMv8 system
>>> without using the ARM Trusted Firmware?
>>>
>>> Yours,
>>> Linus Walleij
>>
>> Yes, exactly.
>>
>> It would be the best choice
>> to switch over to PSCI with ATF in a long run,
>> but, I decided to use spin-table for the initial SoC bring-up
>> because of tight schedule.
>
> So if you don't have an ATF port ready, why not use U-Boot's PSCI
> implementation meanwhile? I think there are efforts underway to make
> PSCI enablement for random new boards a walk in the park (by making the
> PSCI support as generic as possible, CCing Chen-Yu for this).
>
> IIRC the spin-table boot method was just introduced to cope with cores
> that don't have EL3 and thus cannot provide PSCI services the normal way
> (and that don't want to or cannot sacrifice EL2 for that).
> So I am a bit wary of proliferating this SMP method.
>
> Wouldn't it be better to help making U-Boot's PSCI stack as easy to use
> as possible? I don't see technical reasons that adding PSCI support for
> a board should be harder or more involved than adding spin-table support
> - in the end you need to tell it about the SMP pen, maybe providing (or
> faking?) reset and shutdown for 0.2 compliance.
>

We have a team working on PSCI for ARMv8. The patches are floating and 
need some minor fix. With these patches, PSCI will be used instead of 
spin-table. However, a trusted firmware (or other kind) is required.

York

[U-Boot] [U-Boot, 1/2] ARM: move #ifdef to match the error handling code

[Tom Rini]

On Fri, Jun 17, 2016 at 09:51:48PM +0900, Masahiro Yamada wrote:

> Match the #ifdef ... #endif and the code,
> 
>    ret = do_something();
>    if (ret)
>            return ret;
> 
> This will make it easier to add more #ifdef'ed code.
> 
> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

Applied to u-boot/master, thanks!

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20160624/cadd5d25/attachment.sig>

[U-Boot] [PATCH 2/2] arm64: add better spin-table support

[Masahiro Yamada]

Hi Andre,


2016-06-19 19:33 GMT+09:00 Andr? Przywara <andre.przywara@arm.com>:
> Hi,
>
> On 19/06/16 09:57, Masahiro Yamada wrote:
>> 2016-06-18 18:40 GMT+09:00 Linus Walleij <linus.walleij@linaro.org>:
>>> On Fri, Jun 17, 2016 at 2:51 PM, Masahiro Yamada
>>> <yamada.masahiro@socionext.com> wrote:
>>>
>>>> There are two enable methods supported by ARM64 Linux; psci and
>>>> spin-table.  The latter is simpler and easier to use for quick SoC
>>>> bring-up.
>>>>
>>>> So, I used the spin-table for my first ARMv8 SoC porting, but I
>>>> found its support in U-Boot was poor.  It is true there exists a
>>>> code fragment for the spin code in arch/arm/cpu/armv8/start.S,
>>>> but I see some problems:
>>>
>>> Is part of the motivation for this approach to boot an ARMv8 system
>>> without using the ARM Trusted Firmware?
>>>
>>> Yours,
>>> Linus Walleij
>>
>> Yes, exactly.
>>
>> It would be the best choice
>> to switch over to PSCI with ATF in a long run,
>> but, I decided to use spin-table for the initial SoC bring-up
>> because of tight schedule.
>
> So if you don't have an ATF port ready, why not use U-Boot's PSCI
> implementation meanwhile? I think there are efforts underway to make
> PSCI enablement for random new boards a walk in the park (by making the
> PSCI support as generic as possible, CCing Chen-Yu for this).


So, you mean U-Boot can serve PSCI for ARMv8 SoCs without ATF, right?
(and efforts underway for further improvement)

I know PSCI support is available for ARMv7
(arch/arm/cpu/armv7/psci.S),
but I could not find PSCI implementation
in arch/arm/cpu/armv8/ directory in the mainline U-Boot.

I checked Chen-Yu's patches on Patchwork,
but I think they are mostly for improvement of ARMv7 PSCI support.

If I am missing something,
could you point me to the reference, please?



> IIRC the spin-table boot method was just introduced to cope with cores
> that don't have EL3 and thus cannot provide PSCI services the normal way
> (and that don't want to or cannot sacrifice EL2 for that).
> So I am a bit wary of proliferating this SMP method.

Proliferating?

Many of ARM 32bit SoCs have vendor-specific SMP methods.
On the other hand, ARM64 only has two methods; psci and spin-table.

Is this a problem?



> Wouldn't it be better to help making U-Boot's PSCI stack as easy to use
> as possible? I don't see technical reasons that adding PSCI support for
> a board should be harder or more involved than adding spin-table support
> - in the end you need to tell it about the SMP pen, maybe providing (or
> faking?) reset and shutdown for 0.2 compliance.


Right.

My motivation is to bring up Linux quickly
before ATF becomes ready.

From your statement, I felt
efforts for the ARMv8 PSCI implementation in U-Boot are underway.
If so, I am definitely interested in it.


-- 
Best Regards
Masahiro Yamada

[U-Boot] [PATCH 2/2] arm64: add better spin-table support

[Masahiro Yamada]

Hi York,

2016-06-21 1:30 GMT+09:00 york sun <york.sun@nxp.com>:
> On 06/19/2016 03:34 AM, Andr? Przywara wrote:
>> Hi,
>>
>> On 19/06/16 09:57, Masahiro Yamada wrote:
>>> 2016-06-18 18:40 GMT+09:00 Linus Walleij <linus.walleij@linaro.org>:
>>>> On Fri, Jun 17, 2016 at 2:51 PM, Masahiro Yamada
>>>> <yamada.masahiro@socionext.com> wrote:
>>>>
>>>>> There are two enable methods supported by ARM64 Linux; psci and
>>>>> spin-table.  The latter is simpler and easier to use for quick SoC
>>>>> bring-up.
>>>>>
>>>>> So, I used the spin-table for my first ARMv8 SoC porting, but I
>>>>> found its support in U-Boot was poor.  It is true there exists a
>>>>> code fragment for the spin code in arch/arm/cpu/armv8/start.S,
>>>>> but I see some problems:
>>>>
>>>> Is part of the motivation for this approach to boot an ARMv8 system
>>>> without using the ARM Trusted Firmware?
>>>>
>>>> Yours,
>>>> Linus Walleij
>>>
>>> Yes, exactly.
>>>
>>> It would be the best choice
>>> to switch over to PSCI with ATF in a long run,
>>> but, I decided to use spin-table for the initial SoC bring-up
>>> because of tight schedule.
>>
>> So if you don't have an ATF port ready, why not use U-Boot's PSCI
>> implementation meanwhile? I think there are efforts underway to make
>> PSCI enablement for random new boards a walk in the park (by making the
>> PSCI support as generic as possible, CCing Chen-Yu for this).
>>
>> IIRC the spin-table boot method was just introduced to cope with cores
>> that don't have EL3 and thus cannot provide PSCI services the normal way
>> (and that don't want to or cannot sacrifice EL2 for that).
>> So I am a bit wary of proliferating this SMP method.
>>
>> Wouldn't it be better to help making U-Boot's PSCI stack as easy to use
>> as possible? I don't see technical reasons that adding PSCI support for
>> a board should be harder or more involved than adding spin-table support
>> - in the end you need to tell it about the SMP pen, maybe providing (or
>> faking?) reset and shutdown for 0.2 compliance.
>>
>
> We have a team working on PSCI for ARMv8. The patches are floating and
> need some minor fix. With these patches, PSCI will be used instead of
> spin-table. However, a trusted firmware (or other kind) is required.
>


That's great.


So, what should we do about this patch?


I admit PSCI with ATF is the best way in the end,
but I believe having a simpler alternative should not hurt.




-- 
Best Regards
Masahiro Yamada

[U-Boot] [PATCH 2/2] arm64: add better spin-table support

[Andre Przywara]

Hi Masahiro,

On 25/06/16 05:08, Masahiro Yamada wrote:
> Hi Andre,
> 
> 
> 2016-06-19 19:33 GMT+09:00 Andr? Przywara <andre.przywara@arm.com>:
>> Hi,
>>
>> On 19/06/16 09:57, Masahiro Yamada wrote:
>>> 2016-06-18 18:40 GMT+09:00 Linus Walleij <linus.walleij@linaro.org>:
>>>> On Fri, Jun 17, 2016 at 2:51 PM, Masahiro Yamada
>>>> <yamada.masahiro@socionext.com> wrote:
>>>>
>>>>> There are two enable methods supported by ARM64 Linux; psci and
>>>>> spin-table.  The latter is simpler and easier to use for quick SoC
>>>>> bring-up.
>>>>>
>>>>> So, I used the spin-table for my first ARMv8 SoC porting, but I
>>>>> found its support in U-Boot was poor.  It is true there exists a
>>>>> code fragment for the spin code in arch/arm/cpu/armv8/start.S,
>>>>> but I see some problems:
>>>>
>>>> Is part of the motivation for this approach to boot an ARMv8 system
>>>> without using the ARM Trusted Firmware?
>>>>
>>>> Yours,
>>>> Linus Walleij
>>>
>>> Yes, exactly.
>>>
>>> It would be the best choice
>>> to switch over to PSCI with ATF in a long run,
>>> but, I decided to use spin-table for the initial SoC bring-up
>>> because of tight schedule.
>>
>> So if you don't have an ATF port ready, why not use U-Boot's PSCI
>> implementation meanwhile? I think there are efforts underway to make
>> PSCI enablement for random new boards a walk in the park (by making the
>> PSCI support as generic as possible, CCing Chen-Yu for this).
> 
> 
> So, you mean U-Boot can serve PSCI for ARMv8 SoCs without ATF, right?
> (and efforts underway for further improvement)

Yes, if I am not mistaken then there are patches floating around to
achieve this.

> I know PSCI support is available for ARMv7
> (arch/arm/cpu/armv7/psci.S),
> but I could not find PSCI implementation
> in arch/arm/cpu/armv8/ directory in the mainline U-Boot.

I think that's work in progress (as in "on the ML").

> I checked Chen-Yu's patches on Patchwork,
> but I think they are mostly for improvement of ARMv7 PSCI support.
> 
> If I am missing something,
> could you point me to the reference, please?

I think you found this yourself already in that other mail of yours?

>> IIRC the spin-table boot method was just introduced to cope with cores
>> that don't have EL3 and thus cannot provide PSCI services the normal way
>> (and that don't want to or cannot sacrifice EL2 for that).
>> So I am a bit wary of proliferating this SMP method.
> 
> Proliferating?
> 
> Many of ARM 32bit SoCs have vendor-specific SMP methods.

Yes, and this is a massive PITA, but hard to change now.

> On the other hand, ARM64 only has two methods; psci and spin-table.

For a reason: Actually we just wanted to have PSCI only, but it turns
out that it's not easily possible on cores that don't implement EL3
(which is architecturally allowed and also implemented).
So spin-table support was added to provide a fall-back for those cases.
But if you have the choice, then PSCI is your friend.
Please note that any support for another SMP method is very unlikely to
ever get merged into upstream Linux.

> Is this a problem?

As mentioned, if you have a proper PSCI framework (as we are about to
get for U-Boot and which ATF already provides), adding SMP support is
not more involved than implementing spin tables. After all it's about
specifying the pen address and possibly providing per-core power hooks.

Cheers,
Andre.

>> Wouldn't it be better to help making U-Boot's PSCI stack as easy to use
>> as possible? I don't see technical reasons that adding PSCI support for
>> a board should be harder or more involved than adding spin-table support
>> - in the end you need to tell it about the SMP pen, maybe providing (or
>> faking?) reset and shutdown for 0.2 compliance.
> 
> 
> Right.
> 
> My motivation is to bring up Linux quickly
> before ATF becomes ready.
> 
> From your statement, I felt
> efforts for the ARMv8 PSCI implementation in U-Boot are underway.
> If so, I am definitely interested in it.
> 
> 

[U-Boot] [PATCH 2/2] arm64: add better spin-table support

[york sun]

On 06/24/2016 09:15 PM, Masahiro Yamada wrote:
> Hi York,
>
> 2016-06-21 1:30 GMT+09:00 york sun <york.sun@nxp.com>:
>> On 06/19/2016 03:34 AM, Andr? Przywara wrote:
>>> Hi,
>>>
>>> On 19/06/16 09:57, Masahiro Yamada wrote:
>>>> 2016-06-18 18:40 GMT+09:00 Linus Walleij <linus.walleij@linaro.org>:
>>>>> On Fri, Jun 17, 2016 at 2:51 PM, Masahiro Yamada
>>>>> <yamada.masahiro@socionext.com> wrote:
>>>>>
>>>>>> There are two enable methods supported by ARM64 Linux; psci and
>>>>>> spin-table.  The latter is simpler and easier to use for quick SoC
>>>>>> bring-up.
>>>>>>
>>>>>> So, I used the spin-table for my first ARMv8 SoC porting, but I
>>>>>> found its support in U-Boot was poor.  It is true there exists a
>>>>>> code fragment for the spin code in arch/arm/cpu/armv8/start.S,
>>>>>> but I see some problems:
>>>>>
>>>>> Is part of the motivation for this approach to boot an ARMv8 system
>>>>> without using the ARM Trusted Firmware?
>>>>>
>>>>> Yours,
>>>>> Linus Walleij
>>>>
>>>> Yes, exactly.
>>>>
>>>> It would be the best choice
>>>> to switch over to PSCI with ATF in a long run,
>>>> but, I decided to use spin-table for the initial SoC bring-up
>>>> because of tight schedule.
>>>
>>> So if you don't have an ATF port ready, why not use U-Boot's PSCI
>>> implementation meanwhile? I think there are efforts underway to make
>>> PSCI enablement for random new boards a walk in the park (by making the
>>> PSCI support as generic as possible, CCing Chen-Yu for this).
>>>
>>> IIRC the spin-table boot method was just introduced to cope with cores
>>> that don't have EL3 and thus cannot provide PSCI services the normal way
>>> (and that don't want to or cannot sacrifice EL2 for that).
>>> So I am a bit wary of proliferating this SMP method.
>>>
>>> Wouldn't it be better to help making U-Boot's PSCI stack as easy to use
>>> as possible? I don't see technical reasons that adding PSCI support for
>>> a board should be harder or more involved than adding spin-table support
>>> - in the end you need to tell it about the SMP pen, maybe providing (or
>>> faking?) reset and shutdown for 0.2 compliance.
>>>
>>
>> We have a team working on PSCI for ARMv8. The patches are floating and
>> need some minor fix. With these patches, PSCI will be used instead of
>> spin-table. However, a trusted firmware (or other kind) is required.
>>
>
>
> That's great.
>
>
> So, what should we do about this patch?
>
>
> I admit PSCI with ATF is the best way in the end,
> but I believe having a simpler alternative should not hurt.
>

I support using spin table as alternative. I see you already sent out v2 
patch. It would be easier to maintain if we consolidate existing code 
with generic code.

York

[U-Boot] [PATCH 2/2] arm64: add better spin-table support

[Chen-Yu Tsai]

Hi,

On Mon, Jun 27, 2016 at 6:47 PM, Andre Przywara <andre.przywara@arm.com> wrote:
> Hi Masahiro,
>
> On 25/06/16 05:08, Masahiro Yamada wrote:
>> Hi Andre,
>>
>>
>> 2016-06-19 19:33 GMT+09:00 Andr? Przywara <andre.przywara@arm.com>:
>>> Hi,
>>>
>>> On 19/06/16 09:57, Masahiro Yamada wrote:
>>>> 2016-06-18 18:40 GMT+09:00 Linus Walleij <linus.walleij@linaro.org>:
>>>>> On Fri, Jun 17, 2016 at 2:51 PM, Masahiro Yamada
>>>>> <yamada.masahiro@socionext.com> wrote:
>>>>>
>>>>>> There are two enable methods supported by ARM64 Linux; psci and
>>>>>> spin-table.  The latter is simpler and easier to use for quick SoC
>>>>>> bring-up.
>>>>>>
>>>>>> So, I used the spin-table for my first ARMv8 SoC porting, but I
>>>>>> found its support in U-Boot was poor.  It is true there exists a
>>>>>> code fragment for the spin code in arch/arm/cpu/armv8/start.S,
>>>>>> but I see some problems:
>>>>>
>>>>> Is part of the motivation for this approach to boot an ARMv8 system
>>>>> without using the ARM Trusted Firmware?
>>>>>
>>>>> Yours,
>>>>> Linus Walleij
>>>>
>>>> Yes, exactly.
>>>>
>>>> It would be the best choice
>>>> to switch over to PSCI with ATF in a long run,
>>>> but, I decided to use spin-table for the initial SoC bring-up
>>>> because of tight schedule.
>>>
>>> So if you don't have an ATF port ready, why not use U-Boot's PSCI
>>> implementation meanwhile? I think there are efforts underway to make
>>> PSCI enablement for random new boards a walk in the park (by making the
>>> PSCI support as generic as possible, CCing Chen-Yu for this).
>>
>>
>> So, you mean U-Boot can serve PSCI for ARMv8 SoCs without ATF, right?
>> (and efforts underway for further improvement)
>
> Yes, if I am not mistaken then there are patches floating around to
> achieve this.

The first part of the PSCI rework is already in master.
The second part is here:

    http://lists.denx.de/pipermail/u-boot/2016-June/258484.html

Or a rebased version:

    https://github.com/wens/u-boot-sunxi/tree/psci-improve-part2

>> I know PSCI support is available for ARMv7
>> (arch/arm/cpu/armv7/psci.S),
>> but I could not find PSCI implementation
>> in arch/arm/cpu/armv8/ directory in the mainline U-Boot.
>
> I think that's work in progress (as in "on the ML").

My work is still focused on ARMv7. However, with a lot of the code
rewritten in C and modularized, I see no reason it could be moved
up to arch/arm and reused for ARMv8. Same probably goes for the
non-secure related stuff, but I haven't looked into those yet.

The remaining bits of assembly for PSCI are:

  - exception vector table
  - PSCI secure monitor entry
  - stack setup
  - CPU entry function
  - ARMv7(?) SMP toggling (can be done in C with inline assembly)
  - ARMv7 specific cache flush

My only ARMv8 board (Pine64) is still gathering dust, and Andre
is already working on an ATF port, so it's unlikely I will spend
time there. Also I'm very unfamiliar with ARMv8.

>> I checked Chen-Yu's patches on Patchwork,
>> but I think they are mostly for improvement of ARMv7 PSCI support.
>>
>> If I am missing something,
>> could you point me to the reference, please?
>
> I think you found this yourself already in that other mail of yours?
>
>>> IIRC the spin-table boot method was just introduced to cope with cores
>>> that don't have EL3 and thus cannot provide PSCI services the normal way
>>> (and that don't want to or cannot sacrifice EL2 for that).
>>> So I am a bit wary of proliferating this SMP method.
>>
>> Proliferating?
>>
>> Many of ARM 32bit SoCs have vendor-specific SMP methods.
>
> Yes, and this is a massive PITA, but hard to change now.

Get everyone on board with PSCI? :)

Regards
ChenYu

>> On the other hand, ARM64 only has two methods; psci and spin-table.
>
> For a reason: Actually we just wanted to have PSCI only, but it turns
> out that it's not easily possible on cores that don't implement EL3
> (which is architecturally allowed and also implemented).
> So spin-table support was added to provide a fall-back for those cases.
> But if you have the choice, then PSCI is your friend.
> Please note that any support for another SMP method is very unlikely to
> ever get merged into upstream Linux.
>
>> Is this a problem?
>
> As mentioned, if you have a proper PSCI framework (as we are about to
> get for U-Boot and which ATF already provides), adding SMP support is
> not more involved than implementing spin tables. After all it's about
> specifying the pen address and possibly providing per-core power hooks.
>
> Cheers,
> Andre.
>
>>> Wouldn't it be better to help making U-Boot's PSCI stack as easy to use
>>> as possible? I don't see technical reasons that adding PSCI support for
>>> a board should be harder or more involved than adding spin-table support
>>> - in the end you need to tell it about the SMP pen, maybe providing (or
>>> faking?) reset and shutdown for 0.2 compliance.
>>
>>
>> Right.
>>
>> My motivation is to bring up Linux quickly
>> before ATF becomes ready.
>>
>> From your statement, I felt
>> efforts for the ARMv8 PSCI implementation in U-Boot are underway.
>> If so, I am definitely interested in it.
>>
>>
> _______________________________________________
> U-Boot mailing list
> U-Boot at lists.denx.de
> http://lists.denx.de/mailman/listinfo/u-boot