[U-Boot] [PATCH v3 00/13] ARMv7: add PSCI support to u-boot

[U-Boot] [PATCH v3 00/13] ARMv7: add PSCI support to u-boot

[Marc Zyngier]

PSCI is an ARM standard that provides a generic interface that
supervisory software can use to manage power in the following
situations:
- Core idle management
- CPU hotplug
- big.LITTLE migration models
- System shutdown and reset

It basically allows the kernel to offload these tasks to the firmware,
and rely on common kernel side code.

More importantly, it gives a way to ensure that CPUs enter the kernel
at the appropriate exception level (ie HYP mode, to allow the use of
the virtualization extensions), even across events like CPUs being
powered off/on or suspended.

The main idea here is to turn some of the existing u-boot code into a
separate section that can live in secure RAM (or a reserved page of
memory), containing a secure monitor that will implement the PSCI
operations. This code will still be alive when u-boot is long gone,
hence the need for a piece of memory that will not be touched by the
OS.

This patch series contains 4 parts:
- the first four patches are just bug fixes
- the next two refactor the HYP/non-secure code to allow relocation
  in secure memory
- the next four contain the generic PSCI code and DT infrastructure
- the last three implement the CPU_ON method of the Allwinner A20 (aka sun7i).

I realize the A20 u-boot code is not upstream yet (BTW is anyone
actively working on that?), but hopefully that should give a good idea
of how things are structured so far. The patches are against the
mainline u-boot tree as of today, merged with the sunxi u-boot tree
of the day and the first 10 patches will directly apply to mainline
u-boot.

As for using this code, it goes like this:
sun7i# ext2load mmc 0:1 0x40008000 zImage ; ext2load mmc 0:1 0x60000000 sun7i-a20-cubietruck.dtb
2270120 bytes read in 117 ms (18.5 MiB/s)
9138 bytes read in 3 ms (2.9 MiB/s)
sun7i# fdt addr 0x60000000 ; fdt resize ; fdt set ethernet0 mac-address "[5a fe b0 07 b0 07]"
sun7i# setenv bootargs console=ttyS0,115200 earlyprintk ip=dhcp root=/dev/nfs nfsroot=/backup/a20_root,tcp
sun7i# bootz 0x40008000 - 0x60000000

The kernel now boots in HYP mode, finds its secondary CPU without any
SMP code present in the kernel, and runs KVM out of the box.
I've been told the Xen/ARM guys managed to do the same fairly easily.

This code has also been tested on a VExpress TC2, running KVM with all
5 CPUs, in order to make sure there was no obvious regression.

I'm wildly cross-posting this patch series, including to lists I'm not
subscribed to. Please keep me on Cc for any comment you may have.

The code is also available at:
git://git.kernel.org/pub/scm/linux/kernel/git/maz/u-boot.git wip/psci

Cheers,

        M.

Ma Haijun (1):
  ARM: convert arch_fixup_memory_node to a generic FDT fixup function

Marc Zyngier (12):
  ARM: HYP/non-sec: move switch to non-sec to the last boot phase
  ARM: HYP/non-sec: add a barrier after setting SCR.NS==1
  ARM: non-sec: reset CNTVOFF to zero
  ARM: add missing HYP mode constant
  ARM: HYP/non-sec: add separate section for secure code
  ARM: HYP/non-sec: allow relocation to secure RAM
  ARM: HYP/non-sec: add generic ARMv7 PSCI code
  ARM: HYP/non-sec: add the option for a second-stage monitor
  ARM: HYP/non-sec/PSCI: emit DT nodes
  sunxi: fix SRAM_B/SRAM_D memory map
  sunxi: HYP/non-sec: add sun7i PSCI backend
  sunxi: HYP/non-sec: configure CNTFRQ on all CPUs

 arch/arm/config.mk                      |   2 +-
 arch/arm/cpu/armv7/Makefile             |   5 +
 arch/arm/cpu/armv7/nonsec_virt.S        | 168 +++++++++++++++++---------------
 arch/arm/cpu/armv7/psci.S               | 105 ++++++++++++++++++++
 arch/arm/cpu/armv7/sunxi/Makefile       |   3 +
 arch/arm/cpu/armv7/sunxi/psci.S         | 162 ++++++++++++++++++++++++++++++
 arch/arm/cpu/armv7/virt-dt.c            | 100 +++++++++++++++++++
 arch/arm/cpu/armv7/virt-v7.c            |  59 ++++-------
 arch/arm/cpu/u-boot.lds                 |  30 ++++++
 arch/arm/include/asm/arch-sunxi/cpu.h   |   4 +-
 arch/arm/include/asm/armv7.h            |  11 ++-
 arch/arm/include/asm/proc-armv/ptrace.h |   2 +
 arch/arm/include/asm/psci.h             |  35 +++++++
 arch/arm/include/asm/secure.h           |  26 +++++
 arch/arm/lib/bootm-fdt.c                |  13 ++-
 arch/arm/lib/bootm.c                    |  27 +++--
 arch/arm/lib/interrupts.c               |   2 +-
 arch/arm/lib/sections.c                 |   2 +
 common/image-fdt.c                      |   7 +-
 include/common.h                        |   6 +-
 include/configs/sun7i.h                 |   7 ++
 21 files changed, 627 insertions(+), 149 deletions(-)
 create mode 100644 arch/arm/cpu/armv7/psci.S
 create mode 100644 arch/arm/cpu/armv7/sunxi/psci.S
 create mode 100644 arch/arm/cpu/armv7/virt-dt.c
 create mode 100644 arch/arm/include/asm/psci.h
 create mode 100644 arch/arm/include/asm/secure.h

-- 
1.8.5.1

[U-Boot] [PATCH v3 01/13] ARM: HYP/non-sec: move switch to non-sec to the last boot phase

[Marc Zyngier]

Having the switch to non-secure in the "prep" phase is causing
all kind of troubles, as that stage can be called multiple times.

Instead, move the switch to non-secure to the last possible phase,
when there is no turning back anymore.

Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 arch/arm/lib/bootm.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/arch/arm/lib/bootm.c b/arch/arm/lib/bootm.c
index a8295bf..68554c8 100644
--- a/arch/arm/lib/bootm.c
+++ b/arch/arm/lib/bootm.c
@@ -242,7 +242,6 @@ static void boot_prep_linux(bootm_headers_t *images)
 		printf("FDT and ATAGS support not compiled in - hanging\n");
 		hang();
 	}
-	do_nonsec_virt_switch();
 }
 
 /* Subcommand: GO */
@@ -287,8 +286,10 @@ static void boot_jump_linux(bootm_headers_t *images, int flag)
 	else
 		r2 = gd->bd->bi_boot_params;
 
-	if (!fake)
+	if (!fake) {
+		do_nonsec_virt_switch();
 		kernel_entry(0, machid, r2);
+	}
 #endif
 }
 
-- 
1.8.5.1

[U-Boot] [PATCH v3 02/13] ARM: HYP/non-sec: add a barrier after setting SCR.NS==1

[Marc Zyngier]

A CP15 instruction execution can be reordered, requiring an
isb to be sure it is executed in program order.

Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 arch/arm/cpu/armv7/nonsec_virt.S | 1 +
 1 file changed, 1 insertion(+)

diff --git a/arch/arm/cpu/armv7/nonsec_virt.S b/arch/arm/cpu/armv7/nonsec_virt.S
index 6367e09..12de5c2 100644
--- a/arch/arm/cpu/armv7/nonsec_virt.S
+++ b/arch/arm/cpu/armv7/nonsec_virt.S
@@ -46,6 +46,7 @@ _secure_monitor:
 #endif
 
 	mcr	p15, 0, r1, c1, c1, 0		@ write SCR (with NS bit set)
+	isb
 
 #ifdef CONFIG_ARMV7_VIRT
 	mrceq	p15, 0, r0, c12, c0, 1		@ get MVBAR value
-- 
1.8.5.1

[U-Boot] [PATCH v3 03/13] ARM: non-sec: reset CNTVOFF to zero

[Marc Zyngier]

Before switching to non-secure, make sure that CNTVOFF is set
to zero on all CPUs. Otherwise, kernel running in non-secure
without HYP enabled (hence using virtual timers) may observe
timers that are not synchronized, effectively seeing time
going backward...

Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 arch/arm/cpu/armv7/nonsec_virt.S | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/arch/arm/cpu/armv7/nonsec_virt.S b/arch/arm/cpu/armv7/nonsec_virt.S
index 12de5c2..b5c946f 100644
--- a/arch/arm/cpu/armv7/nonsec_virt.S
+++ b/arch/arm/cpu/armv7/nonsec_virt.S
@@ -38,10 +38,10 @@ _secure_monitor:
 	bic	r1, r1, #0x4e			@ clear IRQ, FIQ, EA, nET bits
 	orr	r1, r1, #0x31			@ enable NS, AW, FW bits
 
-#ifdef CONFIG_ARMV7_VIRT
 	mrc	p15, 0, r0, c0, c1, 1		@ read ID_PFR1
 	and	r0, r0, #CPUID_ARM_VIRT_MASK	@ mask virtualization bits
 	cmp	r0, #(1 << CPUID_ARM_VIRT_SHIFT)
+#ifdef CONFIG_ARMV7_VIRT
 	orreq	r1, r1, #0x100			@ allow HVC instruction
 #endif
 
@@ -52,7 +52,14 @@ _secure_monitor:
 	mrceq	p15, 0, r0, c12, c0, 1		@ get MVBAR value
 	mcreq	p15, 4, r0, c12, c0, 0		@ write HVBAR
 #endif
+	bne	1f
 
+	@ Reset CNTVOFF to 0 before leaving monitor mode
+	mrc	p15, 0, r0, c0, c1, 1		@ read ID_PFR1
+	ands	r0, r0, #CPUID_ARM_GENTIMER_MASK	@ test arch timer bits
+	movne	r0, #0
+	mcrrne	p15, 4, r0, r0, c14		@ Reset CNTVOFF to zero
+1:
 	movs	pc, lr				@ return to non-secure SVC
 
 _hyp_trap:
-- 
1.8.5.1

[U-Boot] [PATCH v3 04/13] ARM: add missing HYP mode constant

[Marc Zyngier]

In order to be able to use the various mode constants (far more
readable than random hex values), add the missing HYP and A
values.

Also update arm/lib/interrupts.c to display HYP instead of an
unknown value.

Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 arch/arm/include/asm/proc-armv/ptrace.h | 2 ++
 arch/arm/lib/interrupts.c               | 2 +-
 2 files changed, 3 insertions(+), 1 deletion(-)

diff --git a/arch/arm/include/asm/proc-armv/ptrace.h b/arch/arm/include/asm/proc-armv/ptrace.h
index 21aef58..71df5a9 100644
--- a/arch/arm/include/asm/proc-armv/ptrace.h
+++ b/arch/arm/include/asm/proc-armv/ptrace.h
@@ -38,12 +38,14 @@ struct pt_regs {
 #define IRQ_MODE	0x12
 #define SVC_MODE	0x13
 #define ABT_MODE	0x17
+#define HYP_MODE	0x1a
 #define UND_MODE	0x1b
 #define SYSTEM_MODE	0x1f
 #define MODE_MASK	0x1f
 #define T_BIT		0x20
 #define F_BIT		0x40
 #define I_BIT		0x80
+#define A_BIT		0x100
 #define CC_V_BIT	(1 << 28)
 #define CC_C_BIT	(1 << 29)
 #define CC_Z_BIT	(1 << 30)
diff --git a/arch/arm/lib/interrupts.c b/arch/arm/lib/interrupts.c
index 603bf14..fa553c4 100644
--- a/arch/arm/lib/interrupts.c
+++ b/arch/arm/lib/interrupts.c
@@ -103,7 +103,7 @@ void show_regs (struct pt_regs *regs)
 	"UK12_26",	"UK13_26",	"UK14_26",	"UK15_26",
 	"USER_32",	"FIQ_32",	"IRQ_32",	"SVC_32",
 	"UK4_32",	"UK5_32",	"UK6_32",	"ABT_32",
-	"UK8_32",	"UK9_32",	"UK10_32",	"UND_32",
+	"UK8_32",	"UK9_32",	"HYP_32",	"UND_32",
 	"UK12_32",	"UK13_32",	"UK14_32",	"SYS_32",
 	};
 
-- 
1.8.5.1

[U-Boot] [PATCH v3 05/13] ARM: HYP/non-sec: add separate section for secure code

[Marc Zyngier]

In anticipation of refactoring the HYP/non-secure code to run
from secure RAM, add a new linker section that will contain that
code.

Nothing is using it just yet.

Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 arch/arm/config.mk      |  2 +-
 arch/arm/cpu/u-boot.lds | 30 ++++++++++++++++++++++++++++++
 arch/arm/lib/sections.c |  2 ++
 3 files changed, 33 insertions(+), 1 deletion(-)

diff --git a/arch/arm/config.mk b/arch/arm/config.mk
index 98c1253..5db7e73 100644
--- a/arch/arm/config.mk
+++ b/arch/arm/config.mk
@@ -110,5 +110,5 @@ endif
 ifdef CONFIG_ARM64
 OBJCFLAGS += -j .text -j .rodata -j .data -j .u_boot_list -j .rela.dyn
 else
-OBJCFLAGS += -j .text -j .rodata -j .hash -j .data -j .got.plt -j .u_boot_list -j .rel.dyn
+OBJCFLAGS += -j .text -j .secure_text -j .rodata -j .hash -j .data -j .got.plt -j .u_boot_list -j .rel.dyn
 endif
diff --git a/arch/arm/cpu/u-boot.lds b/arch/arm/cpu/u-boot.lds
index 4da5d24..4f04edc 100644
--- a/arch/arm/cpu/u-boot.lds
+++ b/arch/arm/cpu/u-boot.lds
@@ -7,6 +7,8 @@
  * SPDX-License-Identifier:	GPL-2.0+
  */
 
+#include <config.h>
+
 OUTPUT_FORMAT("elf32-littlearm", "elf32-littlearm", "elf32-littlearm")
 OUTPUT_ARCH(arm)
 ENTRY(_start)
@@ -22,6 +24,34 @@ SECTIONS
 		*(.text*)
 	}
 
+#if defined(CONFIG_ARMV7_NONSEC) || defined(CONFIG_ARMV7_VIRT) || defined(CONFIG_ARMV7_PSCI)
+
+#ifndef CONFIG_ARMV7_SECURE_BASE
+#define CONFIG_ARMV7_SECURE_BASE
+#endif
+
+	.__secure_start : {
+		. = ALIGN(0x1000);
+		*(.__secure_start)
+	}
+
+	.secure_text CONFIG_ARMV7_SECURE_BASE :
+		AT(ADDR(.__secure_start) + SIZEOF(.__secure_start))
+	{
+		*(._secure.text)
+	}
+
+	. = LOADADDR(.__secure_start) +
+		SIZEOF(.__secure_start) +
+		SIZEOF(.secure_text);
+
+	__secure_end_lma = .;
+	.__secure_end : AT(__secure_end_lma) {
+		*(.__secure_end)
+		LONG(0x1d1071c);	/* Must output something to reset LMA */
+	}
+#endif
+
 	. = ALIGN(4);
 	.rodata : { *(SORT_BY_ALIGNMENT(SORT_BY_NAME(.rodata*))) }
 
diff --git a/arch/arm/lib/sections.c b/arch/arm/lib/sections.c
index e35687c..c141923 100644
--- a/arch/arm/lib/sections.c
+++ b/arch/arm/lib/sections.c
@@ -25,3 +25,5 @@ char __image_copy_start[0] __attribute__((section(".__image_copy_start")));
 char __image_copy_end[0] __attribute__((section(".__image_copy_end")));
 char __rel_dyn_start[0] __attribute__((section(".__rel_dyn_start")));
 char __rel_dyn_end[0] __attribute__((section(".__rel_dyn_end")));
+char __secure_start[0] __attribute__((section(".__secure_start")));
+char __secure_end[0] __attribute__((section(".__secure_end")));
-- 
1.8.5.1

[U-Boot] [PATCH v3 06/13] ARM: HYP/non-sec: allow relocation to secure RAM

[Marc Zyngier]

The current non-sec switching code suffers from one major issue:
it cannot run in secure RAM, as a large part of u-boot still needs
to be run while we're switched to non-secure.

This patch reworks the whole HYP/non-secure strategy by:
- making sure the secure code is the *last* thing u-boot executes
  before entering the payload
- performing an exception return from secure mode directly into
  the payload
- allowing the code to be dynamically relocated to secure RAM
  before switching to non-secure.

This involves quite a bit of horrible code, specially as u-boot
relocation is quite primitive.

Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 arch/arm/cpu/armv7/nonsec_virt.S | 161 +++++++++++++++++++--------------------
 arch/arm/cpu/armv7/virt-v7.c     |  59 +++++---------
 arch/arm/include/asm/armv7.h     |  10 ++-
 arch/arm/include/asm/secure.h    |  26 +++++++
 arch/arm/lib/bootm.c             |  22 +++---
 5 files changed, 138 insertions(+), 140 deletions(-)
 create mode 100644 arch/arm/include/asm/secure.h

diff --git a/arch/arm/cpu/armv7/nonsec_virt.S b/arch/arm/cpu/armv7/nonsec_virt.S
index b5c946f..2a43e3c 100644
--- a/arch/arm/cpu/armv7/nonsec_virt.S
+++ b/arch/arm/cpu/armv7/nonsec_virt.S
@@ -10,10 +10,13 @@
 #include <linux/linkage.h>
 #include <asm/gic.h>
 #include <asm/armv7.h>
+#include <asm/proc-armv/ptrace.h>
 
 .arch_extension sec
 .arch_extension virt
 
+	.pushsection ._secure.text, "ax"
+
 	.align	5
 /* the vector table for secure state and HYP mode */
 _monitor_vectors:
@@ -22,51 +25,86 @@ _monitor_vectors:
 	adr pc, _secure_monitor
 	.word 0
 	.word 0
-	adr pc, _hyp_trap
+	.word 0
 	.word 0
 	.word 0
 
+.macro is_cpu_virt_capable	tmp
+	mrc	p15, 0, \tmp, c0, c1, 1		@ read ID_PFR1
+	and	\tmp, \tmp, #CPUID_ARM_VIRT_MASK	@ mask virtualization bits
+	cmp	\tmp, #(1 << CPUID_ARM_VIRT_SHIFT)
+.endm
+
 /*
  * secure monitor handler
  * U-boot calls this "software interrupt" in start.S
  * This is executed on a "smc" instruction, we use a "smc #0" to switch
  * to non-secure state.
- * We use only r0 and r1 here, due to constraints in the caller.
+ * r0, r1, r2: passed to the callee
+ * ip: target PC
  */
 _secure_monitor:
-	mrc	p15, 0, r1, c1, c1, 0		@ read SCR
-	bic	r1, r1, #0x4e			@ clear IRQ, FIQ, EA, nET bits
-	orr	r1, r1, #0x31			@ enable NS, AW, FW bits
+	mrc	p15, 0, r5, c1, c1, 0		@ read SCR
+	bic	r5, r5, #0x4e			@ clear IRQ, FIQ, EA, nET bits
+	orr	r5, r5, #0x31			@ enable NS, AW, FW bits
 
-	mrc	p15, 0, r0, c0, c1, 1		@ read ID_PFR1
-	and	r0, r0, #CPUID_ARM_VIRT_MASK	@ mask virtualization bits
-	cmp	r0, #(1 << CPUID_ARM_VIRT_SHIFT)
+	mov	r6, #SVC_MODE			@ default mode is SVC
+	is_cpu_virt_capable r4
 #ifdef CONFIG_ARMV7_VIRT
-	orreq	r1, r1, #0x100			@ allow HVC instruction
+	orreq	r5, r5, #0x100			@ allow HVC instruction
+	moveq	r6, #HYP_MODE			@ Enter the kernel as HYP
 #endif
 
-	mcr	p15, 0, r1, c1, c1, 0		@ write SCR (with NS bit set)
+	mcr	p15, 0, r5, c1, c1, 0		@ write SCR (with NS bit set)
 	isb
 
-#ifdef CONFIG_ARMV7_VIRT
-	mrceq	p15, 0, r0, c12, c0, 1		@ get MVBAR value
-	mcreq	p15, 4, r0, c12, c0, 0		@ write HVBAR
-#endif
 	bne	1f
 
 	@ Reset CNTVOFF to 0 before leaving monitor mode
-	mrc	p15, 0, r0, c0, c1, 1		@ read ID_PFR1
-	ands	r0, r0, #CPUID_ARM_GENTIMER_MASK	@ test arch timer bits
-	movne	r0, #0
-	mcrrne	p15, 4, r0, r0, c14		@ Reset CNTVOFF to zero
+	mrc	p15, 0, r4, c0, c1, 1		@ read ID_PFR1
+	ands	r4, r4, #CPUID_ARM_GENTIMER_MASK	@ test arch timer bits
+	movne	r4, #0
+	mcrrne	p15, 4, r4, r4, c14		@ Reset CNTVOFF to zero
 1:
-	movs	pc, lr				@ return to non-secure SVC
-
-_hyp_trap:
-	mrs	lr, elr_hyp	@ for older asm: .byte 0x00, 0xe3, 0x0e, 0xe1
-	mov pc, lr				@ do no switch modes, but
-						@ return to caller
-
+	mov	lr, ip
+	mov	ip, #(F_BIT | I_BIT | A_BIT)	@ Set A, I and F
+	tst	lr, #1				@ Check for Thumb PC
+	orrne	ip, ip, #T_BIT			@ Set T if Thumb
+	orr	ip, ip, r6			@ Slot target mode in
+	msr	spsr_cxfs, ip			@ Set full SPSR
+	movs	pc, lr				@ ERET to non-secure
+
+ENTRY(_do_nonsec_entry)
+	mov	ip, r0
+	mov	r0, r1
+	mov	r1, r2
+	mov	r2, r3
+	smc	#0
+ENDPROC(_do_nonsec_entry)
+
+.macro get_cbar_addr	addr
+#ifdef CONFIG_ARM_GIC_BASE_ADDRESS
+	ldr	\addr, =CONFIG_ARM_GIC_BASE_ADDRESS
+#else
+	mrc	p15, 4, \addr, c15, c0, 0	@ read CBAR
+	bfc	\addr, #0, #15			@ clear reserved bits
+#endif
+.endm
+
+.macro get_gicd_addr	addr
+	get_cbar_addr	\addr
+	add	\addr, \addr, #GIC_DIST_OFFSET	@ GIC dist i/f offset
+.endm
+
+.macro get_gicc_addr	addr, tmp
+	get_cbar_addr	\addr
+	is_cpu_virt_capable \tmp
+	movne	\tmp, #GIC_CPU_OFFSET_A9	@ GIC CPU offset for A9
+	moveq	\tmp, #GIC_CPU_OFFSET_A15	@ GIC CPU offset for A15/A7
+	add	\addr, \addr, \tmp
+.endm
+
+#ifndef CONFIG_ARMV7_PSCI
 /*
  * Secondary CPUs start here and call the code for the core specific parts
  * of the non-secure and HYP mode transition. The GIC distributor specific
@@ -74,31 +112,21 @@ _hyp_trap:
  * Then they go back to wfi and wait to be woken up by the kernel again.
  */
 ENTRY(_smp_pen)
-	mrs	r0, cpsr
-	orr	r0, r0, #0xc0
-	msr	cpsr, r0			@ disable interrupts
-	ldr	r1, =_start
-	mcr	p15, 0, r1, c12, c0, 0		@ set VBAR
+	cpsid	i
+	cpsid	f
 
 	bl	_nonsec_init
-	mov	r12, r0				@ save GICC address
-#ifdef CONFIG_ARMV7_VIRT
-	bl	_switch_to_hyp
-#endif
-
-	ldr	r1, [r12, #GICC_IAR]		@ acknowledge IPI
-	str	r1, [r12, #GICC_EOIR]		@ signal end of interrupt
 
 	adr	r0, _smp_pen			@ do not use this address again
 	b	smp_waitloop			@ wait for IPIs, board specific
 ENDPROC(_smp_pen)
+#endif
 
 /*
  * Switch a core to non-secure state.
  *
  *  1. initialize the GIC per-core interface
  *  2. allow coprocessor access in non-secure modes
- *  3. switch the cpu mode (by calling "smc #0")
  *
  * Called from smp_pen by secondary cores and directly by the BSP.
  * Do not assume that the stack is available and only use registers
@@ -108,38 +136,23 @@ ENDPROC(_smp_pen)
  * though, but we check this in C before calling this function.
  */
 ENTRY(_nonsec_init)
-#ifdef CONFIG_ARM_GIC_BASE_ADDRESS
-	ldr	r2, =CONFIG_ARM_GIC_BASE_ADDRESS
-#else
-	mrc	p15, 4, r2, c15, c0, 0		@ read CBAR
-	bfc	r2, #0, #15			@ clear reserved bits
-#endif
-	add	r3, r2, #GIC_DIST_OFFSET	@ GIC dist i/f offset
+	get_gicd_addr	r3
+
 	mvn	r1, #0				@ all bits to 1
 	str	r1, [r3, #GICD_IGROUPRn]	@ allow private interrupts
 
-	mrc	p15, 0, r0, c0, c0, 0		@ read MIDR
-	ldr	r1, =MIDR_PRIMARY_PART_MASK
-	and	r0, r0, r1			@ mask out variant and revision
+	get_gicc_addr	r3, r1
 
-	ldr	r1, =MIDR_CORTEX_A7_R0P0 & MIDR_PRIMARY_PART_MASK
-	cmp	r0, r1				@ check for Cortex-A7
-
-	ldr	r1, =MIDR_CORTEX_A15_R0P0 & MIDR_PRIMARY_PART_MASK
-	cmpne	r0, r1				@ check for Cortex-A15
-
-	movne	r1, #GIC_CPU_OFFSET_A9		@ GIC CPU offset for A9
-	moveq	r1, #GIC_CPU_OFFSET_A15		@ GIC CPU offset for A15/A7
-	add	r3, r2, r1			@ r3 = GIC CPU i/f addr
-
-	mov	r1, #1				@ set GICC_CTLR[enable]
+	mov	r1, #3				@ Enable both groups
 	str	r1, [r3, #GICC_CTLR]		@ and clear all other bits
 	mov	r1, #0xff
 	str	r1, [r3, #GICC_PMR]		@ set priority mask register
 
+	mrc	p15, 0, r0, c1, c1, 2
 	movw	r1, #0x3fff
-	movt	r1, #0x0006
-	mcr	p15, 0, r1, c1, c1, 2		@ NSACR = all copros to non-sec
+	movt	r1, #0x0004
+	orr	r0, r0, r1
+	mcr	p15, 0, r0, c1, c1, 2		@ NSACR = all copros to non-sec
 
 /* The CNTFRQ register of the generic timer needs to be
  * programmed in secure state. Some primary bootloaders / firmware
@@ -157,21 +170,9 @@ ENTRY(_nonsec_init)
 
 	adr	r1, _monitor_vectors
 	mcr	p15, 0, r1, c12, c0, 1		@ set MVBAR to secure vectors
-
-	mrc	p15, 0, ip, c12, c0, 0		@ save secure copy of VBAR
-
 	isb
-	smc	#0				@ call into MONITOR mode
-
-	mcr	p15, 0, ip, c12, c0, 0		@ write non-secure copy of VBAR
-
-	mov	r1, #1
-	str	r1, [r3, #GICC_CTLR]		@ enable non-secure CPU i/f
-	add	r2, r2, #GIC_DIST_OFFSET
-	str	r1, [r2, #GICD_CTLR]		@ allow private interrupts
 
 	mov	r0, r3				@ return GICC address
-
 	bx	lr
 ENDPROC(_nonsec_init)
 
@@ -183,18 +184,10 @@ ENTRY(smp_waitloop)
 	ldr	r1, [r1]
 	cmp	r0, r1			@ make sure we dont execute this code
 	beq	smp_waitloop		@ again (due to a spurious wakeup)
-	mov	pc, r1
+	mov	r0, r1
+	b	_do_nonsec_entry
 ENDPROC(smp_waitloop)
 .weak smp_waitloop
 #endif
 
-ENTRY(_switch_to_hyp)
-	mov	r0, lr
-	mov	r1, sp				@ save SVC copy of LR and SP
-	isb
-	hvc #0			 @ for older asm: .byte 0x70, 0x00, 0x40, 0xe1
-	mov	sp, r1
-	mov	lr, r0				@ restore SVC copy of LR and SP
-
-	bx	lr
-ENDPROC(_switch_to_hyp)
+	.popsection
diff --git a/arch/arm/cpu/armv7/virt-v7.c b/arch/arm/cpu/armv7/virt-v7.c
index 2cd604f..6500030 100644
--- a/arch/arm/cpu/armv7/virt-v7.c
+++ b/arch/arm/cpu/armv7/virt-v7.c
@@ -13,17 +13,10 @@
 #include <asm/armv7.h>
 #include <asm/gic.h>
 #include <asm/io.h>
+#include <asm/secure.h>
 
 unsigned long gic_dist_addr;
 
-static unsigned int read_cpsr(void)
-{
-	unsigned int reg;
-
-	asm volatile ("mrs %0, cpsr\n" : "=r" (reg));
-	return reg;
-}
-
 static unsigned int read_id_pfr1(void)
 {
 	unsigned int reg;
@@ -72,6 +65,18 @@ static unsigned long get_gicd_base_address(void)
 #endif
 }
 
+static void relocate_secure_section(void)
+{
+#ifdef CONFIG_ARMV7_SECURE_BASE
+	size_t sz = __secure_end - __secure_start;
+
+	memcpy((void *)CONFIG_ARMV7_SECURE_BASE, __secure_start, sz);
+	flush_dcache_range(CONFIG_ARMV7_SECURE_BASE,
+			   CONFIG_ARMV7_SECURE_BASE + sz + 1);
+	invalidate_icache_all();
+#endif
+}
+
 static void kick_secondary_cpus_gic(unsigned long gicdaddr)
 {
 	/* kick all CPUs (except this one) by writing to GICD_SGIR */
@@ -83,35 +88,7 @@ void __weak smp_kick_all_cpus(void)
 	kick_secondary_cpus_gic(gic_dist_addr);
 }
 
-int armv7_switch_hyp(void)
-{
-	unsigned int reg;
-
-	/* check whether we are in HYP mode already */
-	if ((read_cpsr() & 0x1f) == 0x1a) {
-		debug("CPU already in HYP mode\n");
-		return 0;
-	}
-
-	/* check whether the CPU supports the virtualization extensions */
-	reg = read_id_pfr1();
-	if ((reg & CPUID_ARM_VIRT_MASK) != 1 << CPUID_ARM_VIRT_SHIFT) {
-		printf("HYP mode: Virtualization extensions not implemented.\n");
-		return -1;
-	}
-
-	/* call the HYP switching code on this CPU also */
-	_switch_to_hyp();
-
-	if ((read_cpsr() & 0x1F) != 0x1a) {
-		printf("HYP mode: switch not successful.\n");
-		return -1;
-	}
-
-	return 0;
-}
-
-int armv7_switch_nonsec(void)
+int armv7_init_nonsec(void)
 {
 	unsigned int reg;
 	unsigned itlinesnr, i;
@@ -147,11 +124,13 @@ int armv7_switch_nonsec(void)
 	for (i = 1; i <= itlinesnr; i++)
 		writel((unsigned)-1, gic_dist_addr + GICD_IGROUPRn + 4 * i);
 
-	smp_set_core_boot_addr((unsigned long)_smp_pen, -1);
+#ifndef CONFIG_ARMV7_PSCI
+	smp_set_core_boot_addr((unsigned long)secure_ram_addr(_smp_pen), -1);
 	smp_kick_all_cpus();
+#endif
 
 	/* call the non-sec switching code on this CPU also */
-	_nonsec_init();
-
+	relocate_secure_section();
+	secure_ram_addr(_nonsec_init)();
 	return 0;
 }
diff --git a/arch/arm/include/asm/armv7.h b/arch/arm/include/asm/armv7.h
index 395444e..11476dd 100644
--- a/arch/arm/include/asm/armv7.h
+++ b/arch/arm/include/asm/armv7.h
@@ -78,13 +78,17 @@ void v7_outer_cache_inval_range(u32 start, u32 end);
 
 #if defined(CONFIG_ARMV7_NONSEC) || defined(CONFIG_ARMV7_VIRT)
 
-int armv7_switch_nonsec(void);
-int armv7_switch_hyp(void);
+int armv7_init_nonsec(void);
 
 /* defined in assembly file */
 unsigned int _nonsec_init(void);
+void _do_nonsec_entry(void *target_pc, unsigned long r0,
+		      unsigned long r1, unsigned long r2);
 void _smp_pen(void);
-void _switch_to_hyp(void);
+
+extern char __secure_start[];
+extern char __secure_end[];
+
 #endif /* CONFIG_ARMV7_NONSEC || CONFIG_ARMV7_VIRT */
 
 #endif /* ! __ASSEMBLY__ */
diff --git a/arch/arm/include/asm/secure.h b/arch/arm/include/asm/secure.h
new file mode 100644
index 0000000..effdb18
--- /dev/null
+++ b/arch/arm/include/asm/secure.h
@@ -0,0 +1,26 @@
+#ifndef __ASM_SECURE_H
+#define __ASM_SECURE_H
+
+#include <config.h>
+
+#ifdef CONFIG_ARMV7_SECURE_BASE
+/*
+ * Warning, horror ahead.
+ *
+ * The target code lives in our "secure ram", but u-boot doesn't know
+ * that, and has blindly added reloc_off to every relocation
+ * entry. Gahh. Do the opposite conversion. This hack also prevents
+ * GCC from generating code veeners, which u-boot doesn't relocate at
+ * all...
+ */
+#define secure_ram_addr(_fn) ({						\
+			DECLARE_GLOBAL_DATA_PTR;			\
+			void *__fn = _fn;				\
+			typeof(_fn) *__tmp = (__fn - gd->reloc_off);	\
+			__tmp;						\
+		})
+#else
+#define secure_ram_addr(_fn)	(_fn)
+#endif
+
+#endif
diff --git a/arch/arm/lib/bootm.c b/arch/arm/lib/bootm.c
index 68554c8..ecc25f9 100644
--- a/arch/arm/lib/bootm.c
+++ b/arch/arm/lib/bootm.c
@@ -20,6 +20,7 @@
 #include <libfdt.h>
 #include <fdt_support.h>
 #include <asm/bootm.h>
+#include <asm/secure.h>
 #include <linux/compiler.h>
 
 #if defined(CONFIG_ARMV7_NONSEC) || defined(CONFIG_ARMV7_VIRT)
@@ -185,26 +186,16 @@ static void setup_end_tag(bd_t *bd)
 
 __weak void setup_board_tags(struct tag **in_params) {}
 
+#ifdef CONFIG_ARM64
 static void do_nonsec_virt_switch(void)
 {
-#if defined(CONFIG_ARMV7_NONSEC) || defined(CONFIG_ARMV7_VIRT)
-	if (armv7_switch_nonsec() == 0)
-#ifdef CONFIG_ARMV7_VIRT
-		if (armv7_switch_hyp() == 0)
-			debug("entered HYP mode\n");
-#else
-		debug("entered non-secure state\n");
-#endif
-#endif
-
-#ifdef CONFIG_ARM64
 	smp_kick_all_cpus();
 	armv8_switch_to_el2();
 #ifdef CONFIG_ARMV8_SWITCH_TO_EL1
 	armv8_switch_to_el1();
 #endif
-#endif
 }
+#endif
 
 /* Subcommand: PREP */
 static void boot_prep_linux(bootm_headers_t *images)
@@ -287,8 +278,13 @@ static void boot_jump_linux(bootm_headers_t *images, int flag)
 		r2 = gd->bd->bi_boot_params;
 
 	if (!fake) {
-		do_nonsec_virt_switch();
+#if defined(CONFIG_ARMV7_NONSEC) || defined(CONFIG_ARMV7_VIRT)
+		armv7_init_nonsec();
+		secure_ram_addr(_do_nonsec_entry)(kernel_entry,
+						  0, machid, r2);
+#else
 		kernel_entry(0, machid, r2);
+#endif
 	}
 #endif
 }
-- 
1.8.5.1

[U-Boot] [PATCH v3 07/13] ARM: HYP/non-sec: add generic ARMv7 PSCI code

[Marc Zyngier]

Implement core support for PSCI. As this is generic code, it doesn't
implement anything really useful (all the functions are returning
Not Implemented).

Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 arch/arm/cpu/armv7/Makefile |   4 ++
 arch/arm/cpu/armv7/psci.S   | 105 ++++++++++++++++++++++++++++++++++++++++++++
 arch/arm/include/asm/psci.h |  35 +++++++++++++++
 3 files changed, 144 insertions(+)
 create mode 100644 arch/arm/cpu/armv7/psci.S
 create mode 100644 arch/arm/include/asm/psci.h

diff --git a/arch/arm/cpu/armv7/Makefile b/arch/arm/cpu/armv7/Makefile
index 28aa4e1..c749fb6 100644
--- a/arch/arm/cpu/armv7/Makefile
+++ b/arch/arm/cpu/armv7/Makefile
@@ -24,6 +24,10 @@ obj-y	+= nonsec_virt.o
 obj-y	+= virt-v7.o
 endif
 
+ifneq ($(CONFIG_ARMV7_PSCI),)
+obj-y	+= psci.o
+endif
+
 obj-$(CONFIG_OMAP_COMMON) += omap-common/
 obj-$(CONFIG_TEGRA) += tegra-common/
 
diff --git a/arch/arm/cpu/armv7/psci.S b/arch/arm/cpu/armv7/psci.S
new file mode 100644
index 0000000..a9341e0
--- /dev/null
+++ b/arch/arm/cpu/armv7/psci.S
@@ -0,0 +1,105 @@
+/*
+ * Copyright (C) 2013 - ARM Ltd
+ * Author: Marc Zyngier <marc.zyngier@arm.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <config.h>
+#include <linux/linkage.h>
+#include <asm/psci.h>
+
+	.pushsection ._secure.text, "ax"
+
+	.arch_extension	sec
+
+	.align	5
+	.globl _psci_vectors
+_psci_vectors:
+	b	default_psci_vector	@ reset
+	b	default_psci_vector	@ undef
+	b	_smc_psci		@ smc
+	b	default_psci_vector	@ pabort
+	b	default_psci_vector	@ dabort
+	b	default_psci_vector	@ hyp
+	b	default_psci_vector	@ irq
+	b	psci_fiq_enter		@ fiq
+
+ENTRY(psci_fiq_enter)
+	movs	pc, lr
+ENDPROC(psci_fiq_enter)
+.weak psci_fiq_enter
+
+ENTRY(default_psci_vector)
+	movs	pc, lr
+ENDPROC(default_psci_vector)
+.weak default_psci_vector
+
+ENTRY(psci_cpu_suspend)
+ENTRY(psci_cpu_off)
+ENTRY(psci_cpu_on)
+ENTRY(psci_migrate)
+	mov	r0, #ARM_PSCI_RET_NI	@ Return -1 (Not Implemented)
+	mov	pc, lr
+ENDPROC(psci_migrate)
+ENDPROC(psci_cpu_on)
+ENDPROC(psci_cpu_off)
+ENDPROC(psci_cpu_suspend)
+.weak psci_cpu_suspend
+.weak psci_cpu_off
+.weak psci_cpu_on
+.weak psci_migrate
+
+_psci_table:
+	.word	ARM_PSCI_FN_CPU_SUSPEND
+	.word	psci_cpu_suspend
+	.word	ARM_PSCI_FN_CPU_OFF
+	.word	psci_cpu_off
+	.word	ARM_PSCI_FN_CPU_ON
+	.word	psci_cpu_on
+	.word	ARM_PSCI_FN_MIGRATE
+	.word	psci_migrate
+	.word	0
+	.word	0
+
+_smc_psci:
+	push	{r3-r7,lr}
+
+	@ Switch to secure
+	mrc	p15, 0, r7, c1, c1, 0
+	bic	r4, r7, #1
+	mcr	p15, 0, r4, c1, c1, 0
+	isb
+
+	adr	r4, _psci_table
+1:	ldr	r5, [r4]	@ Load PSCI function ID
+	ldr	r6, [r4, #4]	@ Load target PC
+	cmp	r5, #0		@ If reach the end, bail out
+	mvneq	r0, #0		@ Return -1 (Not Implemented)
+	beq	2f
+	cmp	r0, r5		@ If not matching, try next entry
+	addne	r4, r4, #8
+	bne	1b
+	cmp	r6, #0		@ Not implemented
+	moveq	r0, #ARM_PSCI_RET_NI
+	beq	2f
+
+	blx	r6		@ Execute PSCI function
+
+	@ Switch back to non-secure
+	mcr	p15, 0, r7, c1, c1, 0
+
+2:	pop	{r3-r7, lr}
+	movs	pc, lr		@ Return to the kernel
+
+	.popsection
diff --git a/arch/arm/include/asm/psci.h b/arch/arm/include/asm/psci.h
new file mode 100644
index 0000000..704b4b0
--- /dev/null
+++ b/arch/arm/include/asm/psci.h
@@ -0,0 +1,35 @@
+/*
+ * Copyright (C) 2013 - ARM Ltd
+ * Author: Marc Zyngier <marc.zyngier@arm.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#ifndef __ARM_PSCI_H__
+#define __ARM_PSCI_H__
+
+/* PSCI interface */
+#define ARM_PSCI_FN_BASE		0x95c1ba5e
+#define ARM_PSCI_FN(n)			(ARM_PSCI_FN_BASE + (n))
+
+#define ARM_PSCI_FN_CPU_SUSPEND		ARM_PSCI_FN(0)
+#define ARM_PSCI_FN_CPU_OFF		ARM_PSCI_FN(1)
+#define ARM_PSCI_FN_CPU_ON		ARM_PSCI_FN(2)
+#define ARM_PSCI_FN_MIGRATE		ARM_PSCI_FN(3)
+
+#define ARM_PSCI_RET_SUCCESS		0
+#define ARM_PSCI_RET_NI			(-1)
+#define ARM_PSCI_RET_INVAL		(-2)
+#define ARM_PSCI_RET_DENIED		(-3)
+
+#endif /* __ARM_PSCI_H__ */
-- 
1.8.5.1

[U-Boot] [PATCH v3 08/13] ARM: HYP/non-sec: add the option for a second-stage monitor

[Marc Zyngier]

Allow the switch to a second stage secure monitor just before
switching to non-secure.

This allows a resident piece of firmware to be active once the
kernel has been entered (the u-boot monitor is dead anyway,
its pages being reused).

Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 arch/arm/cpu/armv7/nonsec_virt.S | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)

diff --git a/arch/arm/cpu/armv7/nonsec_virt.S b/arch/arm/cpu/armv7/nonsec_virt.S
index 2a43e3c..745670e 100644
--- a/arch/arm/cpu/armv7/nonsec_virt.S
+++ b/arch/arm/cpu/armv7/nonsec_virt.S
@@ -44,10 +44,19 @@ _monitor_vectors:
  * ip: target PC
  */
 _secure_monitor:
+#ifdef CONFIG_ARMV7_PSCI
+	ldr	r5, =_psci_vectors		@ Switch to the next monitor
+	mcr	p15, 0, r5, c12, c0, 1
+	isb
+
+	@ Obtain a secure stack, and configure the PSCI backend
+	bl	psci_arch_init
+#endif
+
 	mrc	p15, 0, r5, c1, c1, 0		@ read SCR
-	bic	r5, r5, #0x4e			@ clear IRQ, FIQ, EA, nET bits
+	bic	r5, r5, #0x4a			@ clear IRQ, EA, nET bits
 	orr	r5, r5, #0x31			@ enable NS, AW, FW bits
-
+						@ FIQ preserved for secure mode
 	mov	r6, #SVC_MODE			@ default mode is SVC
 	is_cpu_virt_capable r4
 #ifdef CONFIG_ARMV7_VIRT
-- 
1.8.5.1

[U-Boot] [PATCH v3 09/13] ARM: convert arch_fixup_memory_node to a generic FDT fixup function

[Marc Zyngier]

From: Ma Haijun <mahaijuns@gmail.com>

Some architecture needs extra device tree setup. Instead of adding
yet another hook, convert arch_fixup_memory_node to be a generic
FDT fixup function.

[maz: collapsed 3 patches into one, rewrote commit message]

Signed-off-by: Ma Haijun <mahaijuns@gmail.com>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 arch/arm/lib/bootm-fdt.c | 2 +-
 arch/arm/lib/bootm.c     | 2 +-
 common/image-fdt.c       | 7 +++++--
 include/common.h         | 6 +++---
 4 files changed, 10 insertions(+), 7 deletions(-)

diff --git a/arch/arm/lib/bootm-fdt.c b/arch/arm/lib/bootm-fdt.c
index e40691d..8394e15 100644
--- a/arch/arm/lib/bootm-fdt.c
+++ b/arch/arm/lib/bootm-fdt.c
@@ -20,7 +20,7 @@
 
 DECLARE_GLOBAL_DATA_PTR;
 
-int arch_fixup_memory_node(void *blob)
+int arch_fixup_fdt(void *blob)
 {
 	bd_t *bd = gd->bd;
 	int bank;
diff --git a/arch/arm/lib/bootm.c b/arch/arm/lib/bootm.c
index ecc25f9..866280a 100644
--- a/arch/arm/lib/bootm.c
+++ b/arch/arm/lib/bootm.c
@@ -357,7 +357,7 @@ void boot_prep_vxworks(bootm_headers_t *images)
 	if (images->ft_addr) {
 		off = fdt_path_offset(images->ft_addr, "/memory");
 		if (off < 0) {
-			if (arch_fixup_memory_node(images->ft_addr))
+			if (arch_fixup_fdt(images->ft_addr))
 				puts("## WARNING: fixup memory failed!\n");
 		}
 	}
diff --git a/common/image-fdt.c b/common/image-fdt.c
index 6f9ce7d..398e576 100644
--- a/common/image-fdt.c
+++ b/common/image-fdt.c
@@ -445,7 +445,7 @@ __weak int ft_verify_fdt(void *fdt)
 	return 1;
 }
 
-__weak int arch_fixup_memory_node(void *blob)
+__weak int arch_fixup_fdt(void *blob)
 {
 	return 0;
 }
@@ -462,7 +462,10 @@ int image_setup_libfdt(bootm_headers_t *images, void *blob,
 		puts(" - must RESET the board to recover.\n");
 		return -1;
 	}
-	arch_fixup_memory_node(blob);
+	if (arch_fixup_fdt(blob) < 0) {
+		puts("ERROR: arch specific fdt fixup failed");
+		return -1;
+	}
 	if (IMAAGE_OF_BOARD_SETUP)
 		ft_board_setup(blob, gd->bd);
 	fdt_fixup_ethernet(blob);
diff --git a/include/common.h b/include/common.h
index 7b4a6e9..0b5252e 100644
--- a/include/common.h
+++ b/include/common.h
@@ -317,14 +317,14 @@ int update_flash_size(int flash_size);
 void board_show_dram(ulong size);
 
 /**
- * arch_fixup_memory_node() - Write arch-specific memory information to fdt
+ * arch_fixup_fdt() - Write arch-specific information to fdt
  *
- * Defined in arch/$(ARCH)/lib/bootm.c
+ * Defined in arch/$(ARCH)/lib/bootm-fdt.c
  *
  * @blob:	FDT blob to write to
  * @return 0 if ok, or -ve FDT_ERR_... on failure
  */
-int arch_fixup_memory_node(void *blob);
+int arch_fixup_fdt(void *blob);
 
 /* common/flash.c */
 void flash_perror (int);
-- 
1.8.5.1

[U-Boot] [PATCH v3 10/13] ARM: HYP/non-sec/PSCI: emit DT nodes

[Marc Zyngier]

Generate the PSCI node in the device tree.

Also add a reserve section for the "secure" code that lives in
in normal RAM, so that the kernel knows it'd better not trip on
it.

Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 arch/arm/cpu/armv7/Makefile  |   1 +
 arch/arm/cpu/armv7/virt-dt.c | 100 +++++++++++++++++++++++++++++++++++++++++++
 arch/arm/include/asm/armv7.h |   1 +
 arch/arm/lib/bootm-fdt.c     |  11 ++++-
 4 files changed, 111 insertions(+), 2 deletions(-)
 create mode 100644 arch/arm/cpu/armv7/virt-dt.c

diff --git a/arch/arm/cpu/armv7/Makefile b/arch/arm/cpu/armv7/Makefile
index c749fb6..ef74c49 100644
--- a/arch/arm/cpu/armv7/Makefile
+++ b/arch/arm/cpu/armv7/Makefile
@@ -22,6 +22,7 @@ endif
 ifneq ($(CONFIG_ARMV7_NONSEC)$(CONFIG_ARMV7_VIRT),)
 obj-y	+= nonsec_virt.o
 obj-y	+= virt-v7.o
+obj-y	+= virt-dt.o
 endif
 
 ifneq ($(CONFIG_ARMV7_PSCI),)
diff --git a/arch/arm/cpu/armv7/virt-dt.c b/arch/arm/cpu/armv7/virt-dt.c
new file mode 100644
index 0000000..0b0d6a7
--- /dev/null
+++ b/arch/arm/cpu/armv7/virt-dt.c
@@ -0,0 +1,100 @@
+/*
+ * Copyright (C) 2013 - ARM Ltd
+ * Author: Marc Zyngier <marc.zyngier@arm.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <common.h>
+#include <stdio_dev.h>
+#include <linux/ctype.h>
+#include <linux/types.h>
+#include <asm/global_data.h>
+#include <libfdt.h>
+#include <fdt_support.h>
+#include <asm/armv7.h>
+#include <asm/psci.h>
+
+static int fdt_psci(void *fdt)
+{
+#ifdef CONFIG_ARMV7_PSCI
+	int nodeoff;
+	int tmp;
+
+	nodeoff = fdt_path_offset(fdt, "/cpus");
+	if (nodeoff < 0) {
+		printf("couldn't find /cpus\n");
+		return nodeoff;
+	}
+
+	/* add 'enable-method = "psci"' to each cpu node */
+	for (tmp = fdt_first_subnode(fdt, nodeoff);
+	     tmp >= 0;
+	     tmp = fdt_next_subnode(fdt, tmp)) {
+		const struct fdt_property *prop;
+		int len;
+
+		prop = fdt_get_property(fdt, tmp, "device_type", &len);
+		if (!prop)
+			continue;
+		if (len < 4)
+			continue;
+		if (strcmp(prop->data, "cpu"))
+			continue;
+
+		fdt_setprop_string(fdt, tmp, "enable-method", "psci");
+	}
+
+	nodeoff = fdt_path_offset(fdt, "/psci");
+	if (nodeoff < 0) {
+		nodeoff = fdt_path_offset(fdt, "/");
+		if (nodeoff < 0)
+			return nodeoff;
+
+		nodeoff = fdt_add_subnode(fdt, nodeoff, "psci");
+		if (nodeoff < 0)
+			return nodeoff;
+	}
+
+	tmp = fdt_setprop_string(fdt, nodeoff, "compatible", "arm,psci");
+	if (tmp)
+		return tmp;
+	tmp = fdt_setprop_string(fdt, nodeoff, "method", "smc");
+	if (tmp)
+		return tmp;
+	tmp = fdt_setprop_u32(fdt, nodeoff, "cpu_suspend", ARM_PSCI_FN_CPU_SUSPEND);
+	if (tmp)
+		return tmp;
+	tmp = fdt_setprop_u32(fdt, nodeoff, "cpu_off", ARM_PSCI_FN_CPU_OFF);
+	if (tmp)
+		return tmp;
+	tmp = fdt_setprop_u32(fdt, nodeoff, "cpu_on", ARM_PSCI_FN_CPU_ON);
+	if (tmp)
+		return tmp;
+	tmp = fdt_setprop_u32(fdt, nodeoff, "migrate", ARM_PSCI_FN_MIGRATE);
+	if (tmp)
+		return tmp;
+#endif
+	return 0;
+}
+
+int armv7_update_dt(void *fdt)
+{
+#ifndef CONFIG_ARMV7_SECURE_BASE
+	/* secure code lives in RAM, keep it alive */
+	fdt_add_mem_rsv(fdt, (unsigned long)__secure_start,
+			__secure_end - __secure_start);
+#endif
+
+	return fdt_psci(fdt);
+}
diff --git a/arch/arm/include/asm/armv7.h b/arch/arm/include/asm/armv7.h
index 11476dd..323f282 100644
--- a/arch/arm/include/asm/armv7.h
+++ b/arch/arm/include/asm/armv7.h
@@ -79,6 +79,7 @@ void v7_outer_cache_inval_range(u32 start, u32 end);
 #if defined(CONFIG_ARMV7_NONSEC) || defined(CONFIG_ARMV7_VIRT)
 
 int armv7_init_nonsec(void);
+int armv7_update_dt(void *fdt);
 
 /* defined in assembly file */
 unsigned int _nonsec_init(void);
diff --git a/arch/arm/lib/bootm-fdt.c b/arch/arm/lib/bootm-fdt.c
index 8394e15..ccb76c7 100644
--- a/arch/arm/lib/bootm-fdt.c
+++ b/arch/arm/lib/bootm-fdt.c
@@ -17,13 +17,14 @@
 
 #include <common.h>
 #include <fdt_support.h>
+#include <asm/armv7.h>
 
 DECLARE_GLOBAL_DATA_PTR;
 
 int arch_fixup_fdt(void *blob)
 {
 	bd_t *bd = gd->bd;
-	int bank;
+	int bank, ret;
 	u64 start[CONFIG_NR_DRAM_BANKS];
 	u64 size[CONFIG_NR_DRAM_BANKS];
 
@@ -32,5 +33,11 @@ int arch_fixup_fdt(void *blob)
 		size[bank] = bd->bi_dram[bank].size;
 	}
 
-	return fdt_fixup_memory_banks(blob, start, size, CONFIG_NR_DRAM_BANKS);
+	ret = fdt_fixup_memory_banks(blob, start, size, CONFIG_NR_DRAM_BANKS);
+	if (ret)
+		return ret;
+
+#if defined(CONFIG_ARMV7_NONSEC) || defined(CONFIG_ARMV7_VIRT)
+	return armv7_update_dt(blob);
+#endif
 }
-- 
1.8.5.1

[U-Boot] [PATCH v3 11/13] sunxi: fix SRAM_B/SRAM_D memory map

[Marc Zyngier]

Move the B and D SRAM bank to their actual location (or at least
where the documentation pretends they are).

Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 arch/arm/include/asm/arch-sunxi/cpu.h | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/arch/arm/include/asm/arch-sunxi/cpu.h b/arch/arm/include/asm/arch-sunxi/cpu.h
index 17facc3..aea9a95 100644
--- a/arch/arm/include/asm/arch-sunxi/cpu.h
+++ b/arch/arm/include/asm/arch-sunxi/cpu.h
@@ -31,8 +31,8 @@
 #define SUNXI_SRAM_A2_BASE		0x00004000	/* 16 kiB */
 #define SUNXI_SRAM_A3_BASE		0x00008000	/* 13 kiB */
 #define SUNXI_SRAM_A4_BASE		0x0000b400	/* 3 kiB */
-#define SUNXI_SRAM_D_BASE		0x01c00000
-#define SUNXI_SRAM_B_BASE		0x01c00000	/* 64 kiB (secure) */
+#define SUNXI_SRAM_D_BASE		0x00010000	/* 4 kiB */
+#define SUNXI_SRAM_B_BASE		0x00020000	/* 64 kiB (secure) */
 
 #define SUNXI_SRAMC_BASE		0x01c00000
 #define SUNXI_DRAMC_BASE		0x01c01000
-- 
1.8.5.1

[U-Boot] [PATCH v3 12/13] sunxi: HYP/non-sec: add sun7i PSCI backend

[Marc Zyngier]

So far, only supporting the CPU_ON method.
Other functions can be added later.

Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 arch/arm/cpu/armv7/sunxi/Makefile |   3 +
 arch/arm/cpu/armv7/sunxi/psci.S   | 162 ++++++++++++++++++++++++++++++++++++++
 include/configs/sun7i.h           |   6 ++
 3 files changed, 171 insertions(+)
 create mode 100644 arch/arm/cpu/armv7/sunxi/psci.S

diff --git a/arch/arm/cpu/armv7/sunxi/Makefile b/arch/arm/cpu/armv7/sunxi/Makefile
index f71a26d..0369049 100644
--- a/arch/arm/cpu/armv7/sunxi/Makefile
+++ b/arch/arm/cpu/armv7/sunxi/Makefile
@@ -43,6 +43,9 @@ obj-y	+= cpu_info.o
 ifdef CONFIG_CMD_WATCHDOG
 obj-$(CONFIG_CMD_WATCHDOG)	+= cmd_watchdog.o
 endif
+ifdef CONFIG_ARMV7_PSCI
+obj-y	+= psci.o
+endif
 endif
 
 ifdef CONFIG_SPL_BUILD
diff --git a/arch/arm/cpu/armv7/sunxi/psci.S b/arch/arm/cpu/armv7/sunxi/psci.S
new file mode 100644
index 0000000..0084c81
--- /dev/null
+++ b/arch/arm/cpu/armv7/sunxi/psci.S
@@ -0,0 +1,162 @@
+/*
+ * Copyright (C) 2013 - ARM Ltd
+ * Author: Marc Zyngier <marc.zyngier@arm.com>
+ *
+ * Based on code by Carl van Schaik <carl@ok-labs.com>.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <config.h>
+#include <asm/psci.h>
+#include <asm/arch/cpu.h>
+
+/*
+ * Memory layout:
+ *
+ * SECURE_RAM to text_end :
+ *	._secure_text section
+ * text_end to ALIGN_PAGE(text_end):
+ *	nothing
+ * ALIGN_PAGE(text_end) to ALIGN_PAGE(text_end) + 0x1000)
+ *	1kB of stack per CPU (4 CPUs max).
+ */
+
+	.pushsection ._secure.text, "ax"
+
+	.arch_extension sec
+
+#define	ONE_MS			(CONFIG_SYS_CLK_FREQ / 1000)
+#define	TEN_MS			(10 * ONE_MS)
+
+.macro	timer_wait	reg, ticks
+	@ Program CNTP_TVAL
+	movw	\reg, #(\ticks & 0xffff)
+	movt	\reg, #(\ticks >> 16)
+	mcr	p15, 0, \reg, c14, c2, 0
+	isb
+	@ Enable physical timer, mask interrupt
+	mov	\reg, #3
+	mcr	p15, 0, \reg, c14, c2, 1
+	@ Poll physical timer until ISTATUS is on
+1:	isb
+	mrc	p15, 0, \reg, c14, c2, 1
+	ands	\reg, \reg, #4
+	bne	1b
+	@ Disable timer
+	mov	\reg, #0
+	mcr	p15, 0, \reg, c14, c2, 1
+	isb
+.endm
+
+.globl	psci_arch_init
+psci_arch_init:
+	mrc	p15, 0, r5, c1, c1, 0	@ Read SCR
+	bic	r5, r5, #1		@ Secure mode
+	mcr	p15, 0, r5, c1, c1, 0	@ Write SCR
+	isb
+
+	mrc	p15, 0, r4, c0, c0, 5	@ MPIDR
+	and	r4, r4, #3		@ cpu number in cluster
+	mov	r5, #400		@ 1kB of stack per CPU
+	mul	r4, r4, r5
+
+	adr	r5, text_end		@ end of text
+	add	r5, r5, #0x2000		@ Skip two pages
+	lsr	r5, r5, #12		@ Align to start of page
+	lsl	r5, r5, #12
+	sub	sp, r5, r4		@ here's our stack!
+
+	bx	lr
+
+	@ r1 = target CPU
+	@ r2 = target PC
+.globl	psci_cpu_on
+psci_cpu_on:
+	adr	r0, _target_pc
+	str	r2, [r0]
+	dsb
+
+	movw	r0, #(SUNXI_CPUCFG_BASE & 0xffff)
+	movt	r0, #(SUNXI_CPUCFG_BASE >> 16)
+
+	@ CPU mask
+	and	r1, r1, #3	@ only care about first cluster
+	mov	r4, #1
+	lsl	r4, r4, r1
+
+	adr	r6, _sunxi_cpu_entry
+	str	r6, [r0, #0x1a4] @ PRIVATE_REG (boot vector)
+
+	@ Assert reset on target CPU
+	mov	r6, #0
+	lsl	r5, r1, #6	@ 64 bytes per CPU
+	add	r5, r5, #0x40	@ Offset from base
+	add	r5, r5, r0	@ CPU control block
+	str	r6, [r5]	@ Reset CPU
+
+	@ l1 invalidate
+	ldr	r6, [r0, #0x184]
+	bic	r6, r6, r4
+	str	r6, [r0, #0x184]
+
+	@ Lock CPU
+	ldr	r6, [r0, #0x1e4]
+	bic	r6, r6, r4
+	str	r6, [r0, #0x1e4]
+
+	@ Release power clamp
+	movw	r6, #0x1ff
+	movt	r6, #0
+1:	lsrs	r6, r6, #1
+	str	r6, [r0, #0x1b0]
+	bne	1b
+
+	timer_wait r1, TEN_MS
+
+	@ Clear power gating
+	ldr	r6, [r0, #0x1b4]
+	bic	r6, r6, #1
+	str	r6, [r0, #0x1b4]
+
+	@ Deassert reset on target CPU
+	mov	r6, #3
+	str	r6, [r5]
+
+	@ Unlock CPU
+	ldr	r6, [r0, #0x1e4]
+	orr	r6, r6, r4
+	str	r6, [r0, #0x1e4]
+
+	mov	r0, #ARM_PSCI_RET_SUCCESS	@ Return PSCI_RET_SUCCESS
+	mov	pc, lr
+
+_target_pc:
+	.word	0
+
+_sunxi_cpu_entry:
+	@ Set SMP bit
+	mrc	p15, 0, r0, c1, c0, 1
+	orr	r0, r0, #0x40
+	mcr	p15, 0, r0, c1, c0, 1
+	isb
+
+	bl	_nonsec_init
+	bl	psci_arch_init
+
+	adr	r0, _target_pc
+	ldr	r0, [r0]
+	b	_do_nonsec_entry
+
+text_end:
+	.popsection
diff --git a/include/configs/sun7i.h b/include/configs/sun7i.h
index a6ede2a..c3c24fb 100644
--- a/include/configs/sun7i.h
+++ b/include/configs/sun7i.h
@@ -38,6 +38,12 @@
 #define CONFIG_BOARD_POSTCLK_INIT 1
 #endif
 
+#define CONFIG_ARMV7_VIRT		1
+#define CONFIG_ARMV7_NONSEC		1
+#define CONFIG_ARMV7_PSCI		1
+#define CONFIG_ARMV7_PSCI_NR_CPUS	2
+#define CONFIG_ARMV7_SECURE_BASE	SUNXI_SRAM_B_BASE
+
 /*
  * Include common sunxi configuration where most the settings are
  */
-- 
1.8.5.1

[U-Boot] [PATCH v3 13/13] sunxi: HYP/non-sec: configure CNTFRQ on all CPUs

[Marc Zyngier]

CNTFRQ needs to be properly configured on all CPUs. Otherwise,
virtual machines hoping to find valuable information on secondary
CPUs will be disapointed...

Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 include/configs/sun7i.h | 1 +
 1 file changed, 1 insertion(+)

diff --git a/include/configs/sun7i.h b/include/configs/sun7i.h
index c3c24fb..9098541 100644
--- a/include/configs/sun7i.h
+++ b/include/configs/sun7i.h
@@ -43,6 +43,7 @@
 #define CONFIG_ARMV7_PSCI		1
 #define CONFIG_ARMV7_PSCI_NR_CPUS	2
 #define CONFIG_ARMV7_SECURE_BASE	SUNXI_SRAM_B_BASE
+#define CONFIG_SYS_CLK_FREQ		24000000
 
 /*
  * Include common sunxi configuration where most the settings are
-- 
1.8.5.1

[U-Boot] [PATCH v3 00/13] ARMv7: add PSCI support to u-boot

[Albert ARIBAUD]

Hi Marc,

On Sat, 15 Feb 2014 13:36:24 +0000, Marc Zyngier <marc.zyngier@arm.com>
wrote:

> PSCI is an ARM standard that provides a generic interface that
> supervisory software can use to manage power in the following
> situations:

There seems to be no history information either in the cover letter or
in individual patches. This makes it hard(er) for reviewers to
determine if/how their previous comments were handled.

Amicalement,
-- 
Albert.

[U-Boot] [PATCH v3 00/13] ARMv7: add PSCI support to u-boot

[Marc Zyngier]

Hi Albert,

On 2014-02-15 14:45, Albert ARIBAUD wrote:
> On Sat, 15 Feb 2014 13:36:24 +0000, Marc Zyngier 
> <marc.zyngier@arm.com>
> wrote:
>
>> PSCI is an ARM standard that provides a generic interface that
>> supervisory software can use to manage power in the following
>> situations:
>
> There seems to be no history information either in the cover letter 
> or
> in individual patches. This makes it hard(er) for reviewers to
> determine if/how their previous comments were handled.

Doh. I knew I had forgotten something when I rushed to send this series 
out and bugger off to do something else.

Thanks for reminding me! ;-)

So here we go:

 From v2:
- Dropped the secure stack allocation from the generic PSCI code. There 
was too little space there for it to be really useful, and the arch code 
knows a lot better about its requirements anyway. It is now the 
responsibility of the arch code to provide a stack. This allows it to 
get rid of the silly game with the thread registers that was confusing 
everyone...
- Added provision for FIQ handling in secure mode. Allwinner A20 is 
going to require this for CPU_OFF.
- Better integration of the FDT injection code with the rest of the 
code, fixing the truncated FDT issue that people have been reporting 
(courtesy of Ma Haijun).
- Cleanup of the AW-specific code (stack allocation, timer macro).
- Rebased on mainline u-boot (on top of 22a240c32c13).

> Amicalement,

Pas mieux,

         M.
-- 
Fast, cheap, reliable. Pick two.

[U-Boot] [PATCH v3 00/13] ARMv7: add PSCI support to u-boot

[Albert ARIBAUD]

Hi Marc,

On Sat, 15 Feb 2014 13:36:24 +0000, Marc Zyngier
<marc.zyngier-5wv7dgnIgG8@public.gmane.org> wrote:

> PSCI is an ARM standard that provides a generic interface that
> supervisory software can use to manage power in the following
> situations:
> - Core idle management
> - CPU hotplug
> - big.LITTLE migration models
> - System shutdown and reset
> 
> It basically allows the kernel to offload these tasks to the firmware,
> and rely on common kernel side code.
> 
> More importantly, it gives a way to ensure that CPUs enter the kernel
> at the appropriate exception level (ie HYP mode, to allow the use of
> the virtualization extensions), even across events like CPUs being
> powered off/on or suspended.
> 
> The main idea here is to turn some of the existing u-boot code into a
> separate section that can live in secure RAM (or a reserved page of
> memory), containing a secure monitor that will implement the PSCI
> operations. This code will still be alive when u-boot is long gone,
> hence the need for a piece of memory that will not be touched by the
> OS.
> 
> This patch series contains 4 parts:
> - the first four patches are just bug fixes
> - the next two refactor the HYP/non-secure code to allow relocation
>   in secure memory
> - the next four contain the generic PSCI code and DT infrastructure
> - the last three implement the CPU_ON method of the Allwinner A20 (aka sun7i).
> 
> I realize the A20 u-boot code is not upstream yet (BTW is anyone
> actively working on that?), but hopefully that should give a good idea
> of how things are structured so far. The patches are against the
> mainline u-boot tree as of today, merged with the sunxi u-boot tree
> of the day and the first 10 patches will directly apply to mainline
> u-boot.
> 
> As for using this code, it goes like this:
> sun7i# ext2load mmc 0:1 0x40008000 zImage ; ext2load mmc 0:1 0x60000000 sun7i-a20-cubietruck.dtb
> 2270120 bytes read in 117 ms (18.5 MiB/s)
> 9138 bytes read in 3 ms (2.9 MiB/s)
> sun7i# fdt addr 0x60000000 ; fdt resize ; fdt set ethernet0 mac-address "[5a fe b0 07 b0 07]"
> sun7i# setenv bootargs console=ttyS0,115200 earlyprintk ip=dhcp root=/dev/nfs nfsroot=/backup/a20_root,tcp
> sun7i# bootz 0x40008000 - 0x60000000
> 
> The kernel now boots in HYP mode, finds its secondary CPU without any
> SMP code present in the kernel, and runs KVM out of the box.
> I've been told the Xen/ARM guys managed to do the same fairly easily.
> 
> This code has also been tested on a VExpress TC2, running KVM with all
> 5 CPUs, in order to make sure there was no obvious regression.
> 
> I'm wildly cross-posting this patch series, including to lists I'm not
> subscribed to. Please keep me on Cc for any comment you may have.
> 
> The code is also available at:
> git://git.kernel.org/pub/scm/linux/kernel/git/maz/u-boot.git wip/psci
> 
> Cheers,
> 
>         M.

Marc, I'm unclear what you want to do with this series. You mention
that its first 10 patches will apply to U-Boot, but I am not sure
whether you are just indicating that it is possible to apply them or
asking for these 10 patches to go in U-Boot mainline.  Or is it
something else yet?

Amicalement,
-- 
Albert.

[U-Boot] [PATCH v3 00/13] ARMv7: add PSCI support to u-boot

[Marc Zyngier]

On 16/04/14 15:45, Albert ARIBAUD wrote:
> Hi Marc,
> 
> On Sat, 15 Feb 2014 13:36:24 +0000, Marc Zyngier
> <marc.zyngier-5wv7dgnIgG8@public.gmane.org> wrote:
> 
>> PSCI is an ARM standard that provides a generic interface that
>> supervisory software can use to manage power in the following
>> situations:
>> - Core idle management
>> - CPU hotplug
>> - big.LITTLE migration models
>> - System shutdown and reset
>>
>> It basically allows the kernel to offload these tasks to the firmware,
>> and rely on common kernel side code.
>>
>> More importantly, it gives a way to ensure that CPUs enter the kernel
>> at the appropriate exception level (ie HYP mode, to allow the use of
>> the virtualization extensions), even across events like CPUs being
>> powered off/on or suspended.
>>
>> The main idea here is to turn some of the existing u-boot code into a
>> separate section that can live in secure RAM (or a reserved page of
>> memory), containing a secure monitor that will implement the PSCI
>> operations. This code will still be alive when u-boot is long gone,
>> hence the need for a piece of memory that will not be touched by the
>> OS.
>>
>> This patch series contains 4 parts:
>> - the first four patches are just bug fixes
>> - the next two refactor the HYP/non-secure code to allow relocation
>>   in secure memory
>> - the next four contain the generic PSCI code and DT infrastructure
>> - the last three implement the CPU_ON method of the Allwinner A20 (aka sun7i).
>>
>> I realize the A20 u-boot code is not upstream yet (BTW is anyone
>> actively working on that?), but hopefully that should give a good idea
>> of how things are structured so far. The patches are against the
>> mainline u-boot tree as of today, merged with the sunxi u-boot tree
>> of the day and the first 10 patches will directly apply to mainline
>> u-boot.
>>
>> As for using this code, it goes like this:
>> sun7i# ext2load mmc 0:1 0x40008000 zImage ; ext2load mmc 0:1 0x60000000 sun7i-a20-cubietruck.dtb
>> 2270120 bytes read in 117 ms (18.5 MiB/s)
>> 9138 bytes read in 3 ms (2.9 MiB/s)
>> sun7i# fdt addr 0x60000000 ; fdt resize ; fdt set ethernet0 mac-address "[5a fe b0 07 b0 07]"
>> sun7i# setenv bootargs console=ttyS0,115200 earlyprintk ip=dhcp root=/dev/nfs nfsroot=/backup/a20_root,tcp
>> sun7i# bootz 0x40008000 - 0x60000000
>>
>> The kernel now boots in HYP mode, finds its secondary CPU without any
>> SMP code present in the kernel, and runs KVM out of the box.
>> I've been told the Xen/ARM guys managed to do the same fairly easily.
>>
>> This code has also been tested on a VExpress TC2, running KVM with all
>> 5 CPUs, in order to make sure there was no obvious regression.
>>
>> I'm wildly cross-posting this patch series, including to lists I'm not
>> subscribed to. Please keep me on Cc for any comment you may have.
>>
>> The code is also available at:
>> git://git.kernel.org/pub/scm/linux/kernel/git/maz/u-boot.git wip/psci
>>
>> Cheers,
>>
>>         M.
> 
> Marc, I'm unclear what you want to do with this series. You mention
> that its first 10 patches will apply to U-Boot, but I am not sure
> whether you are just indicating that it is possible to apply them or
> asking for these 10 patches to go in U-Boot mainline.  Or is it
> something else yet?

Well, I rarely write code just for the sake of forking a critical
project ;-)

So let's be 100% explicit: Yes, I'm hereby asking for these patches to
be merged. They offer a service that is required by the Linux kernel as
well as Xen. They are in active use on the Allwinner sun7i platform as
well as Versatile Express (though the later doesn't have a PSCI
implementation).

Now, given that two months have gone past without much comment other
than the odd "hey, works great", I don't really know where to take that.

Are you willing to review the patches?

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

[U-Boot] [PATCH v3 00/13] ARMv7: add PSCI support to u-boot

[Jon Loeliger]

On 04/16/2014 11:09 AM, Marc Zyngier wrote:

>> Marc, I'm unclear what you want to do with this series. You mention
>> that its first 10 patches will apply to U-Boot, but I am not sure
>> whether you are just indicating that it is possible to apply them or
>> asking for these 10 patches to go in U-Boot mainline.  Or is it
>> something else yet?
>
> Well, I rarely write code just for the sake of forking a critical
> project ;-)
>
> So let's be 100% explicit: Yes, I'm hereby asking for these patches to
> be merged. They offer a service that is required by the Linux kernel as
> well as Xen. They are in active use on the Allwinner sun7i platform as
> well as Versatile Express (though the later doesn't have a PSCI
> implementation).
>
> Now, given that two months have gone past without much comment other
> than the odd "hey, works great", I don't really know where to take that.
>
> Are you willing to review the patches?
>
> Thanks,
> 	M.

I too need them.  I am actively working on using them and
bringing up another platform.

They will need to be rebased to top of tree, though.

jdl

[U-Boot] [PATCH v3 00/13] ARMv7: add PSCI support to u-boot

[Albert ARIBAUD]

Hi Marc,

On Wed, 16 Apr 2014 17:09:07 +0100, Marc Zyngier <marc.zyngier@arm.com>
wrote:

> On 16/04/14 15:45, Albert ARIBAUD wrote:
> > Hi Marc,
> > 
> > On Sat, 15 Feb 2014 13:36:24 +0000, Marc Zyngier
> > <marc.zyngier-5wv7dgnIgG8@public.gmane.org> wrote:
> > 
> >> PSCI is an ARM standard that provides a generic interface that
> >> supervisory software can use to manage power in the following
> >> situations:
> >> - Core idle management
> >> - CPU hotplug
> >> - big.LITTLE migration models
> >> - System shutdown and reset
> >>
> >> It basically allows the kernel to offload these tasks to the firmware,
> >> and rely on common kernel side code.
> >>
> >> More importantly, it gives a way to ensure that CPUs enter the kernel
> >> at the appropriate exception level (ie HYP mode, to allow the use of
> >> the virtualization extensions), even across events like CPUs being
> >> powered off/on or suspended.
> >>
> >> The main idea here is to turn some of the existing u-boot code into a
> >> separate section that can live in secure RAM (or a reserved page of
> >> memory), containing a secure monitor that will implement the PSCI
> >> operations. This code will still be alive when u-boot is long gone,
> >> hence the need for a piece of memory that will not be touched by the
> >> OS.
> >>
> >> This patch series contains 4 parts:
> >> - the first four patches are just bug fixes
> >> - the next two refactor the HYP/non-secure code to allow relocation
> >>   in secure memory
> >> - the next four contain the generic PSCI code and DT infrastructure
> >> - the last three implement the CPU_ON method of the Allwinner A20 (aka sun7i).
> >>
> >> I realize the A20 u-boot code is not upstream yet (BTW is anyone
> >> actively working on that?), but hopefully that should give a good idea
> >> of how things are structured so far. The patches are against the
> >> mainline u-boot tree as of today, merged with the sunxi u-boot tree
> >> of the day and the first 10 patches will directly apply to mainline
> >> u-boot.
> >>
> >> As for using this code, it goes like this:
> >> sun7i# ext2load mmc 0:1 0x40008000 zImage ; ext2load mmc 0:1 0x60000000 sun7i-a20-cubietruck.dtb
> >> 2270120 bytes read in 117 ms (18.5 MiB/s)
> >> 9138 bytes read in 3 ms (2.9 MiB/s)
> >> sun7i# fdt addr 0x60000000 ; fdt resize ; fdt set ethernet0 mac-address "[5a fe b0 07 b0 07]"
> >> sun7i# setenv bootargs console=ttyS0,115200 earlyprintk ip=dhcp root=/dev/nfs nfsroot=/backup/a20_root,tcp
> >> sun7i# bootz 0x40008000 - 0x60000000
> >>
> >> The kernel now boots in HYP mode, finds its secondary CPU without any
> >> SMP code present in the kernel, and runs KVM out of the box.
> >> I've been told the Xen/ARM guys managed to do the same fairly easily.
> >>
> >> This code has also been tested on a VExpress TC2, running KVM with all
> >> 5 CPUs, in order to make sure there was no obvious regression.
> >>
> >> I'm wildly cross-posting this patch series, including to lists I'm not
> >> subscribed to. Please keep me on Cc for any comment you may have.
> >>
> >> The code is also available at:
> >> git://git.kernel.org/pub/scm/linux/kernel/git/maz/u-boot.git wip/psci
> >>
> >> Cheers,
> >>
> >>         M.
> > 
> > Marc, I'm unclear what you want to do with this series. You mention
> > that its first 10 patches will apply to U-Boot, but I am not sure
> > whether you are just indicating that it is possible to apply them or
> > asking for these 10 patches to go in U-Boot mainline.  Or is it
> > something else yet?
> 
> Well, I rarely write code just for the sake of forking a critical
> project ;-)
> 
> So let's be 100% explicit: Yes, I'm hereby asking for these patches to
> be merged. They offer a service that is required by the Linux kernel as
> well as Xen. They are in active use on the Allwinner sun7i platform as
> well as Versatile Express (though the later doesn't have a PSCI
> implementation).
> 
> Now, given that two months have gone past without much comment other
> than the odd "hey, works great", I don't really know where to take that.
> 
> Are you willing to review the patches?

Well, I rarely ask about patches just for the sake of conversation. O:-)

So yes, I am willing to review them -- and I suspect others are, as
well. Nobody commented the V3 series on the U-Boot list -- save for
Jon's comment about the series needing a rebase -- which could mean no
one here is unhappy with them... or they were discussed and possibly
acted upon on linux-sunxi, where the replies were redirected. I don't
follow linux-sunx closely, so I couldn't tell. :)

Still, I am trying to figure out the whole Allwinner nebula and see how
things are supposed to work out between their various SoCs and make
sure to avoid duplicate/incompatible effort (you're mentioning the A20,
there seems to be A31 work underway too elsewhere). I am starting to
wonder whether an ARM allwinner sub-repo might make sense. Tom,
Wolfgang?

> Thanks,
> 
> 	M.

Amicalement,
-- 
Albert.

[U-Boot] [PATCH v3 00/13] ARMv7: add PSCI support to u-boot

[Marc Zyngier]

On Thu, Apr 17 2014 at  9:34:24 am BST, Albert ARIBAUD <albert.u.boot@aribaud.net> wrote:
> Hi Marc,
>
> On Wed, 16 Apr 2014 17:09:07 +0100, Marc Zyngier <marc.zyngier@arm.com>
> wrote:
>
>> On 16/04/14 15:45, Albert ARIBAUD wrote:
>> > Hi Marc,
>> > 
>> > On Sat, 15 Feb 2014 13:36:24 +0000, Marc Zyngier
>> > <marc.zyngier-5wv7dgnIgG8@public.gmane.org> wrote:
>> > 
>> >> PSCI is an ARM standard that provides a generic interface that
>> >> supervisory software can use to manage power in the following
>> >> situations:
>> >> - Core idle management
>> >> - CPU hotplug
>> >> - big.LITTLE migration models
>> >> - System shutdown and reset
>> >>
>> >> It basically allows the kernel to offload these tasks to the firmware,
>> >> and rely on common kernel side code.
>> >>
>> >> More importantly, it gives a way to ensure that CPUs enter the kernel
>> >> at the appropriate exception level (ie HYP mode, to allow the use of
>> >> the virtualization extensions), even across events like CPUs being
>> >> powered off/on or suspended.
>> >>
>> >> The main idea here is to turn some of the existing u-boot code into a
>> >> separate section that can live in secure RAM (or a reserved page of
>> >> memory), containing a secure monitor that will implement the PSCI
>> >> operations. This code will still be alive when u-boot is long gone,
>> >> hence the need for a piece of memory that will not be touched by the
>> >> OS.
>> >>
>> >> This patch series contains 4 parts:
>> >> - the first four patches are just bug fixes
>> >> - the next two refactor the HYP/non-secure code to allow relocation
>> >>   in secure memory
>> >> - the next four contain the generic PSCI code and DT infrastructure
>> >> - the last three implement the CPU_ON method of the Allwinner A20 (aka sun7i).
>> >>
>> >> I realize the A20 u-boot code is not upstream yet (BTW is anyone
>> >> actively working on that?), but hopefully that should give a good idea
>> >> of how things are structured so far. The patches are against the
>> >> mainline u-boot tree as of today, merged with the sunxi u-boot tree
>> >> of the day and the first 10 patches will directly apply to mainline
>> >> u-boot.
>> >>
>> >> As for using this code, it goes like this:
>> >> sun7i# ext2load mmc 0:1 0x40008000 zImage ; ext2load mmc 0:1 0x60000000 sun7i-a20-cubietruck.dtb
>> >> 2270120 bytes read in 117 ms (18.5 MiB/s)
>> >> 9138 bytes read in 3 ms (2.9 MiB/s)
>> >> sun7i# fdt addr 0x60000000 ; fdt resize ; fdt set ethernet0 mac-address "[5a fe b0 07 b0 07]"
>> >> sun7i# setenv bootargs console=ttyS0,115200 earlyprintk ip=dhcp root=/dev/nfs nfsroot=/backup/a20_root,tcp
>> >> sun7i# bootz 0x40008000 - 0x60000000
>> >>
>> >> The kernel now boots in HYP mode, finds its secondary CPU without any
>> >> SMP code present in the kernel, and runs KVM out of the box.
>> >> I've been told the Xen/ARM guys managed to do the same fairly easily.
>> >>
>> >> This code has also been tested on a VExpress TC2, running KVM with all
>> >> 5 CPUs, in order to make sure there was no obvious regression.
>> >>
>> >> I'm wildly cross-posting this patch series, including to lists I'm not
>> >> subscribed to. Please keep me on Cc for any comment you may have.
>> >>
>> >> The code is also available at:
>> >> git://git.kernel.org/pub/scm/linux/kernel/git/maz/u-boot.git wip/psci
>> >>
>> >> Cheers,
>> >>
>> >>         M.
>> > 
>> > Marc, I'm unclear what you want to do with this series. You mention
>> > that its first 10 patches will apply to U-Boot, but I am not sure
>> > whether you are just indicating that it is possible to apply them or
>> > asking for these 10 patches to go in U-Boot mainline.  Or is it
>> > something else yet?
>> 
>> Well, I rarely write code just for the sake of forking a critical
>> project ;-)
>> 
>> So let's be 100% explicit: Yes, I'm hereby asking for these patches to
>> be merged. They offer a service that is required by the Linux kernel as
>> well as Xen. They are in active use on the Allwinner sun7i platform as
>> well as Versatile Express (though the later doesn't have a PSCI
>> implementation).
>> 
>> Now, given that two months have gone past without much comment other
>> than the odd "hey, works great", I don't really know where to take that.
>> 
>> Are you willing to review the patches?
>
> Well, I rarely ask about patches just for the sake of conversation. O:-)
>
> So yes, I am willing to review them -- and I suspect others are, as
> well. Nobody commented the V3 series on the U-Boot list -- save for
> Jon's comment about the series needing a rebase -- which could mean no
> one here is unhappy with them... or they were discussed and possibly
> acted upon on linux-sunxi, where the replies were redirected. I don't
> follow linux-sunx closely, so I couldn't tell. :)

No, so far there hasn't been much discussion, and people seem happy with
it. I have a couple of fixes lined up, but nothing major.

Also, a number of the patches are actually fixes that should really make
it into the U-Boot tree, no matter if the PSCI code is merged or
not. Some of them make the kernel go completely bonkers, other introduce
the risk of U-Boot falling over in style.

> Still, I am trying to figure out the whole Allwinner nebula and see how
> things are supposed to work out between their various SoCs and make
> sure to avoid duplicate/incompatible effort (you're mentioning the A20,
> there seems to be A31 work underway too elsewhere). I am starting to
> wonder whether an ARM allwinner sub-repo might make sense. Tom,
> Wolfgang?

Ian Campbell (cc-ed) is actively pushing out patches to support the A20
in mainline U-Boot (I believe you've been on the receiving end of
these), and I plan to rebase my series on top of his. Still, the A20
support is only a small part of the code, used as an example of how to
implement PSCI on a rather simple platform. This can easily be split out
and merged via different trees.

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny.

[U-Boot] [PATCH v3 00/13] ARMv7: add PSCI support to u-boot

[Albert ARIBAUD]

Hi Marc,

On Thu, 17 Apr 2014 09:58:19 +0100, Marc Zyngier <marc.zyngier@arm.com>
wrote:

> On Thu, Apr 17 2014 at  9:34:24 am BST, Albert ARIBAUD <albert.u.boot@aribaud.net> wrote:
> > Hi Marc,
> >
> > On Wed, 16 Apr 2014 17:09:07 +0100, Marc Zyngier <marc.zyngier@arm.com>
> > wrote:
> >
> >> On 16/04/14 15:45, Albert ARIBAUD wrote:
> >> > Hi Marc,
> >> > 
> >> > On Sat, 15 Feb 2014 13:36:24 +0000, Marc Zyngier
> >> > <marc.zyngier-5wv7dgnIgG8@public.gmane.org> wrote:
> >> > 
> >> >> PSCI is an ARM standard that provides a generic interface that
> >> >> supervisory software can use to manage power in the following
> >> >> situations:
> >> >> - Core idle management
> >> >> - CPU hotplug
> >> >> - big.LITTLE migration models
> >> >> - System shutdown and reset
> >> >>
> >> >> It basically allows the kernel to offload these tasks to the firmware,
> >> >> and rely on common kernel side code.
> >> >>
> >> >> More importantly, it gives a way to ensure that CPUs enter the kernel
> >> >> at the appropriate exception level (ie HYP mode, to allow the use of
> >> >> the virtualization extensions), even across events like CPUs being
> >> >> powered off/on or suspended.
> >> >>
> >> >> The main idea here is to turn some of the existing u-boot code into a
> >> >> separate section that can live in secure RAM (or a reserved page of
> >> >> memory), containing a secure monitor that will implement the PSCI
> >> >> operations. This code will still be alive when u-boot is long gone,
> >> >> hence the need for a piece of memory that will not be touched by the
> >> >> OS.
> >> >>
> >> >> This patch series contains 4 parts:
> >> >> - the first four patches are just bug fixes
> >> >> - the next two refactor the HYP/non-secure code to allow relocation
> >> >>   in secure memory
> >> >> - the next four contain the generic PSCI code and DT infrastructure
> >> >> - the last three implement the CPU_ON method of the Allwinner A20 (aka sun7i).
> >> >>
> >> >> I realize the A20 u-boot code is not upstream yet (BTW is anyone
> >> >> actively working on that?), but hopefully that should give a good idea
> >> >> of how things are structured so far. The patches are against the
> >> >> mainline u-boot tree as of today, merged with the sunxi u-boot tree
> >> >> of the day and the first 10 patches will directly apply to mainline
> >> >> u-boot.
> >> >>
> >> >> As for using this code, it goes like this:
> >> >> sun7i# ext2load mmc 0:1 0x40008000 zImage ; ext2load mmc 0:1 0x60000000 sun7i-a20-cubietruck.dtb
> >> >> 2270120 bytes read in 117 ms (18.5 MiB/s)
> >> >> 9138 bytes read in 3 ms (2.9 MiB/s)
> >> >> sun7i# fdt addr 0x60000000 ; fdt resize ; fdt set ethernet0 mac-address "[5a fe b0 07 b0 07]"
> >> >> sun7i# setenv bootargs console=ttyS0,115200 earlyprintk ip=dhcp root=/dev/nfs nfsroot=/backup/a20_root,tcp
> >> >> sun7i# bootz 0x40008000 - 0x60000000
> >> >>
> >> >> The kernel now boots in HYP mode, finds its secondary CPU without any
> >> >> SMP code present in the kernel, and runs KVM out of the box.
> >> >> I've been told the Xen/ARM guys managed to do the same fairly easily.
> >> >>
> >> >> This code has also been tested on a VExpress TC2, running KVM with all
> >> >> 5 CPUs, in order to make sure there was no obvious regression.
> >> >>
> >> >> I'm wildly cross-posting this patch series, including to lists I'm not
> >> >> subscribed to. Please keep me on Cc for any comment you may have.
> >> >>
> >> >> The code is also available at:
> >> >> git://git.kernel.org/pub/scm/linux/kernel/git/maz/u-boot.git wip/psci
> >> >>
> >> >> Cheers,
> >> >>
> >> >>         M.
> >> > 
> >> > Marc, I'm unclear what you want to do with this series. You mention
> >> > that its first 10 patches will apply to U-Boot, but I am not sure
> >> > whether you are just indicating that it is possible to apply them or
> >> > asking for these 10 patches to go in U-Boot mainline.  Or is it
> >> > something else yet?
> >> 
> >> Well, I rarely write code just for the sake of forking a critical
> >> project ;-)
> >> 
> >> So let's be 100% explicit: Yes, I'm hereby asking for these patches to
> >> be merged. They offer a service that is required by the Linux kernel as
> >> well as Xen. They are in active use on the Allwinner sun7i platform as
> >> well as Versatile Express (though the later doesn't have a PSCI
> >> implementation).
> >> 
> >> Now, given that two months have gone past without much comment other
> >> than the odd "hey, works great", I don't really know where to take that.
> >> 
> >> Are you willing to review the patches?
> >
> > Well, I rarely ask about patches just for the sake of conversation. O:-)
> >
> > So yes, I am willing to review them -- and I suspect others are, as
> > well. Nobody commented the V3 series on the U-Boot list -- save for
> > Jon's comment about the series needing a rebase -- which could mean no
> > one here is unhappy with them... or they were discussed and possibly
> > acted upon on linux-sunxi, where the replies were redirected. I don't
> > follow linux-sunx closely, so I couldn't tell. :)
> 
> No, so far there hasn't been much discussion, and people seem happy with
> it. I have a couple of fixes lined up, but nothing major.
> 
> Also, a number of the patches are actually fixes that should really make
> it into the U-Boot tree, no matter if the PSCI code is merged or
> not. Some of them make the kernel go completely bonkers, other introduce
> the risk of U-Boot falling over in style.
> 
> > Still, I am trying to figure out the whole Allwinner nebula and see how
> > things are supposed to work out between their various SoCs and make
> > sure to avoid duplicate/incompatible effort (you're mentioning the A20,
> > there seems to be A31 work underway too elsewhere). I am starting to
> > wonder whether an ARM allwinner sub-repo might make sense. Tom,
> > Wolfgang?
> 
> Ian Campbell (cc-ed) is actively pushing out patches to support the A20
> in mainline U-Boot (I believe you've been on the receiving end of
> these), and I plan to rebase my series on top of his. Still, the A20
> support is only a small part of the code, used as an example of how to
> implement PSCI on a rather simple platform. This can easily be split out
> and merged via different trees.

I have seen Ian's patches, but didn't have time until now to look at
them.

OK, so, to me, the best course of actions is to:

1) isolate those patches in your series which are fixes unrelated to
AllWinner SoCs and get them in. I'd prefer a separate series for this.

2) Get Ian's Allwinner-related patches (reviewed and) applied.

3) Get your Allwinner-related patches (reviewed and) applied.

> Thanks,
> 
> 	M.

Amicalement,
-- 
Albert.

[U-Boot] [PATCH v3 00/13] ARMv7: add PSCI support to u-boot

[Ian Campbell]

On Thu, 2014-04-17 at 09:58 +0100, Marc Zyngier wrote:
> Ian Campbell (cc-ed) is actively pushing out patches to support the A20
> in mainline U-Boot (I believe you've been on the receiving end of
> these),

Need to make a few more cleanups based on the review but v$NEXT should
be along soon.

> and I plan to rebase my series on top of his.

In fact I was banking on it and have deliberately left SMP support out
of the upstreaming series.

Ian.

[U-Boot] [PATCH v3 00/13] ARMv7: add PSCI support to u-boot

[Jon Loeliger]

> No, so far there hasn't been much discussion, and people seem happy with
> it. I have a couple of fixes lined up, but nothing major.

So, I think PSCI 0.2 calls for function numbers in the 0x8400xxxx range.
Seems like we'll have to fix this in one of your patches:

    /* PSCI interface */
    #define ARM_PSCI_FN_BASE                0x95c1ba5e

to be:

    #define ARM_PSCI_FN_BASE                0x84000000

Just thought I'd toss that out there, you know, if you were collecting
fixes for a repost of your patches... :-)

jdl


On Thu, Apr 17, 2014 at 3:58 AM, Marc Zyngier <marc.zyngier@arm.com> wrote:
> On Thu, Apr 17 2014 at  9:34:24 am BST, Albert ARIBAUD <albert.u.boot@aribaud.net> wrote:
>> Hi Marc,
>>
>> On Wed, 16 Apr 2014 17:09:07 +0100, Marc Zyngier <marc.zyngier@arm.com>
>> wrote:
>>
>>> On 16/04/14 15:45, Albert ARIBAUD wrote:
>>> > Hi Marc,
>>> >
>>> > On Sat, 15 Feb 2014 13:36:24 +0000, Marc Zyngier
>>> > <marc.zyngier-5wv7dgnIgG8@public.gmane.org> wrote:
>>> >
>>> >> PSCI is an ARM standard that provides a generic interface that
>>> >> supervisory software can use to manage power in the following
>>> >> situations:
>>> >> - Core idle management
>>> >> - CPU hotplug
>>> >> - big.LITTLE migration models
>>> >> - System shutdown and reset
>>> >>
>>> >> It basically allows the kernel to offload these tasks to the firmware,
>>> >> and rely on common kernel side code.
>>> >>
>>> >> More importantly, it gives a way to ensure that CPUs enter the kernel
>>> >> at the appropriate exception level (ie HYP mode, to allow the use of
>>> >> the virtualization extensions), even across events like CPUs being
>>> >> powered off/on or suspended.
>>> >>
>>> >> The main idea here is to turn some of the existing u-boot code into a
>>> >> separate section that can live in secure RAM (or a reserved page of
>>> >> memory), containing a secure monitor that will implement the PSCI
>>> >> operations. This code will still be alive when u-boot is long gone,
>>> >> hence the need for a piece of memory that will not be touched by the
>>> >> OS.
>>> >>
>>> >> This patch series contains 4 parts:
>>> >> - the first four patches are just bug fixes
>>> >> - the next two refactor the HYP/non-secure code to allow relocation
>>> >>   in secure memory
>>> >> - the next four contain the generic PSCI code and DT infrastructure
>>> >> - the last three implement the CPU_ON method of the Allwinner A20 (aka sun7i).
>>> >>
>>> >> I realize the A20 u-boot code is not upstream yet (BTW is anyone
>>> >> actively working on that?), but hopefully that should give a good idea
>>> >> of how things are structured so far. The patches are against the
>>> >> mainline u-boot tree as of today, merged with the sunxi u-boot tree
>>> >> of the day and the first 10 patches will directly apply to mainline
>>> >> u-boot.
>>> >>
>>> >> As for using this code, it goes like this:
>>> >> sun7i# ext2load mmc 0:1 0x40008000 zImage ; ext2load mmc 0:1 0x60000000 sun7i-a20-cubietruck.dtb
>>> >> 2270120 bytes read in 117 ms (18.5 MiB/s)
>>> >> 9138 bytes read in 3 ms (2.9 MiB/s)
>>> >> sun7i# fdt addr 0x60000000 ; fdt resize ; fdt set ethernet0 mac-address "[5a fe b0 07 b0 07]"
>>> >> sun7i# setenv bootargs console=ttyS0,115200 earlyprintk ip=dhcp root=/dev/nfs nfsroot=/backup/a20_root,tcp
>>> >> sun7i# bootz 0x40008000 - 0x60000000
>>> >>
>>> >> The kernel now boots in HYP mode, finds its secondary CPU without any
>>> >> SMP code present in the kernel, and runs KVM out of the box.
>>> >> I've been told the Xen/ARM guys managed to do the same fairly easily.
>>> >>
>>> >> This code has also been tested on a VExpress TC2, running KVM with all
>>> >> 5 CPUs, in order to make sure there was no obvious regression.
>>> >>
>>> >> I'm wildly cross-posting this patch series, including to lists I'm not
>>> >> subscribed to. Please keep me on Cc for any comment you may have.
>>> >>
>>> >> The code is also available at:
>>> >> git://git.kernel.org/pub/scm/linux/kernel/git/maz/u-boot.git wip/psci
>>> >>
>>> >> Cheers,
>>> >>
>>> >>         M.
>>> >
>>> > Marc, I'm unclear what you want to do with this series. You mention
>>> > that its first 10 patches will apply to U-Boot, but I am not sure
>>> > whether you are just indicating that it is possible to apply them or
>>> > asking for these 10 patches to go in U-Boot mainline.  Or is it
>>> > something else yet?
>>>
>>> Well, I rarely write code just for the sake of forking a critical
>>> project ;-)
>>>
>>> So let's be 100% explicit: Yes, I'm hereby asking for these patches to
>>> be merged. They offer a service that is required by the Linux kernel as
>>> well as Xen. They are in active use on the Allwinner sun7i platform as
>>> well as Versatile Express (though the later doesn't have a PSCI
>>> implementation).
>>>
>>> Now, given that two months have gone past without much comment other
>>> than the odd "hey, works great", I don't really know where to take that.
>>>
>>> Are you willing to review the patches?
>>
>> Well, I rarely ask about patches just for the sake of conversation. O:-)
>>
>> So yes, I am willing to review them -- and I suspect others are, as
>> well. Nobody commented the V3 series on the U-Boot list -- save for
>> Jon's comment about the series needing a rebase -- which could mean no
>> one here is unhappy with them... or they were discussed and possibly
>> acted upon on linux-sunxi, where the replies were redirected. I don't
>> follow linux-sunx closely, so I couldn't tell. :)
>
> No, so far there hasn't been much discussion, and people seem happy with
> it. I have a couple of fixes lined up, but nothing major.
>
> Also, a number of the patches are actually fixes that should really make
> it into the U-Boot tree, no matter if the PSCI code is merged or
> not. Some of them make the kernel go completely bonkers, other introduce
> the risk of U-Boot falling over in style.
>
>> Still, I am trying to figure out the whole Allwinner nebula and see how
>> things are supposed to work out between their various SoCs and make
>> sure to avoid duplicate/incompatible effort (you're mentioning the A20,
>> there seems to be A31 work underway too elsewhere). I am starting to
>> wonder whether an ARM allwinner sub-repo might make sense. Tom,
>> Wolfgang?
>
> Ian Campbell (cc-ed) is actively pushing out patches to support the A20
> in mainline U-Boot (I believe you've been on the receiving end of
> these), and I plan to rebase my series on top of his. Still, the A20
> support is only a small part of the code, used as an example of how to
> implement PSCI on a rather simple platform. This can easily be split out
> and merged via different trees.
>
> Thanks,
>
>         M.
> --
> Jazz is not dead. It just smells funny.
> _______________________________________________
> U-Boot mailing list
> U-Boot at lists.denx.de
> http://lists.denx.de/mailman/listinfo/u-boot

[U-Boot] Fwd: [PATCH v3 00/13] ARMv7: add PSCI support to u-boot

[Jon Loeliger]

[ Drat.  I meant to send this to the U-Boot list, not just Albert.  --jdl]

---------- Forwarded message ----------
From: Jon Loeliger <loeliger@gmail.com>
Date: Thu, Apr 17, 2014 at 11:36 AM
Subject: Re: [U-Boot] [PATCH v3 00/13] ARMv7: add PSCI support to u-boot
To: Albert ARIBAUD <albert.u.boot@aribaud.net>


On Thu, Apr 17, 2014 at 3:34 AM, Albert ARIBAUD
<albert.u.boot@aribaud.net> wrote:
> Hi Marc,
>

> So yes, I am willing to review them -- and I suspect others are, as
> well. Nobody commented the V3 series on the U-Boot list -- save for
> Jon's comment about the series needing a rebase -- which could mean no
> one here is unhappy with them...

So, not *unhappy* with them, but definitely some review is needed.
Also, there are aspects of the implementation that will need to be
generalized a bit.  For example, the sunxi code uses a magic register
in its implementation that allows a core to come out of reset at a known
(ie, given) non-0 address.  My A9 core has CPUs coming out of reset
at a fixed address of 0.  That means my "secure text" must be at 0,
and it must have a "secure vector" with a "secure reset" laid down at 0.
Ultimately, this means that a small modification (adding a secure vector
text section) to the LDS file will be needed to ensure that it is placed at 0
within the secure text section itself.

No, I don't have all this working quite yet. :-)

jdl

[U-Boot] [PATCH v3 00/13] ARMv7: add PSCI support to u-boot

[Marc Zyngier]

On Thu, Apr 17 2014 at 08:55:56 PM, Jon Loeliger <loeliger@gmail.com> wrote:
>> No, so far there hasn't been much discussion, and people seem happy with
>> it. I have a couple of fixes lined up, but nothing major.
>
> So, I think PSCI 0.2 calls for function numbers in the 0x8400xxxx range.
> Seems like we'll have to fix this in one of your patches:
>
>     /* PSCI interface */
>     #define ARM_PSCI_FN_BASE                0x95c1ba5e
>
> to be:
>
>     #define ARM_PSCI_FN_BASE                0x84000000

That would be the case if what we have here was a 0.2 implementation. It
is not. This is the old 0.1 implementation, which is supported in Linux,
KVM and Xen. 0.2 isn't supported anywhere so far (I'm close to merge the
KVM patches, but Linux has no binding just yet).

In 0.1, the number space is entierely undefined. For this
implementation, I used the same numbering as the one documented in the
Linux kernel (Documentation/devicetree/bindings/arm/psci.txt) and KVM
for obvious reasons (arch/arm/include/uapi/asm/kvm.h).

Furthermore, the numbering space is not enough to make this a 0.2
implementation, the extent of the functionallity to implement is much
larger.

When the Linux kernel actually supports 0.2, I'll be happy to add 0.2
support *aside* 0.1.

> Just thought I'd toss that out there, you know, if you were collecting
> fixes for a repost of your patches... :-)

Always happy to stash some more on top of what I already have. I already
rebased the whole pile of crap on top of U-Boot as of yesterday
afternoon, together with Ian's mainline sunxi stuff (see the ijc-merge
branch in my korg repo). Still need to apply fixes on top, but it is
looking OK so far.

I'm be AFK until Tuesday (WTF am I doing now???), but hopefully will get
back to it quickly enough. Expect a drop sometime next week.

	M.
-- 
Jazz is not dead. It just smells funny.

[U-Boot] Fwd: [PATCH v3 00/13] ARMv7: add PSCI support to u-boot

[Marc Zyngier]

On Thu, Apr 17 2014 at 09:01:07 PM, Jon Loeliger <loeliger@gmail.com> wrote:
> [ Drat.  I meant to send this to the U-Boot list, not just Albert.  --jdl]
>
> ---------- Forwarded message ----------
> From: Jon Loeliger <loeliger@gmail.com>
> Date: Thu, Apr 17, 2014 at 11:36 AM
> Subject: Re: [U-Boot] [PATCH v3 00/13] ARMv7: add PSCI support to u-boot
> To: Albert ARIBAUD <albert.u.boot@aribaud.net>
>
>
> On Thu, Apr 17, 2014 at 3:34 AM, Albert ARIBAUD
> <albert.u.boot@aribaud.net> wrote:
>> Hi Marc,
>>
>
>> So yes, I am willing to review them -- and I suspect others are, as
>> well. Nobody commented the V3 series on the U-Boot list -- save for
>> Jon's comment about the series needing a rebase -- which could mean no
>> one here is unhappy with them...
>
> So, not *unhappy* with them, but definitely some review is needed.

That's an understatement. I wouldn't expect such infrastructure patches
to go in without a good discussion about the approach I've taken here.

> Also, there are aspects of the implementation that will need to be
> generalized a bit.  For example, the sunxi code uses a magic register
> in its implementation that allows a core to come out of reset at a known
> (ie, given) non-0 address.  My A9 core has CPUs coming out of reset
> at a fixed address of 0.  That means my "secure text" must be at 0,
> and it must have a "secure vector" with a "secure reset" laid down at 0.
> Ultimately, this means that a small modification (adding a secure vector
> text section) to the LDS file will be needed to ensure that it is placed at 0
> within the secure text section itself.

Yup, there is certainly some flexibility to be added to accomodate for
all the various cases, depending on how secondary CPUs are released.

Patches are most welcome! ;-)

> No, I don't have all this working quite yet. :-)

Looking forward to seeing the result!

	M.
-- 
Jazz is not dead. It just smells funny.

[U-Boot] [PATCH v3 10/13] ARM: HYP/non-sec/PSCI: emit DT nodes

[Ian Campbell]

On Sat, 2014-02-15 at 13:36 +0000, Marc Zyngier wrote:
> @@ -32,5 +33,11 @@ int arch_fixup_fdt(void *blob)
>  		size[bank] = bd->bi_dram[bank].size;
>  	}
>  
> -	return fdt_fixup_memory_banks(blob, start, size, CONFIG_NR_DRAM_BANKS);
> +	ret = fdt_fixup_memory_banks(blob, start, size, CONFIG_NR_DRAM_BANKS);
> +	if (ret)
> +		return ret;
> +
> +#if defined(CONFIG_ARMV7_NONSEC) || defined(CONFIG_ARMV7_VIRT)
> +	return armv7_update_dt(blob);
> +#endif

For platforms where neither of these CONFIG_* are set this results in:
arch/arm/lib/bootm-fdt.c: In function ?arch_fixup_fdt?:
arch/arm/lib/bootm-fdt.c:43:1: warning: control reaches end of non-void function [-Wreturn-type]
 }

Ian.

[U-Boot] [PATCH v3 10/13] ARM: HYP/non-sec/PSCI: emit DT nodes

[Marc Zyngier]

Hi Ian,

On 2014-05-07 20:24, Ian Campbell wrote:
> On Sat, 2014-02-15 at 13:36 +0000, Marc Zyngier wrote:
>> @@ -32,5 +33,11 @@ int arch_fixup_fdt(void *blob)
>>  		size[bank] = bd->bi_dram[bank].size;
>>  	}
>>
>> -	return fdt_fixup_memory_banks(blob, start, size, 
>> CONFIG_NR_DRAM_BANKS);
>> +	ret = fdt_fixup_memory_banks(blob, start, size, 
>> CONFIG_NR_DRAM_BANKS);
>> +	if (ret)
>> +		return ret;
>> +
>> +#if defined(CONFIG_ARMV7_NONSEC) || defined(CONFIG_ARMV7_VIRT)
>> +	return armv7_update_dt(blob);
>> +#endif
>
> For platforms where neither of these CONFIG_* are set this results 
> in:
> arch/arm/lib/bootm-fdt.c: In function ?arch_fixup_fdt?:
> arch/arm/lib/bootm-fdt.c:43:1: warning: control reaches end of
> non-void function [-Wreturn-type]
>  }

I'm afraid this is old news. ;-)

This bug has been fixed in v4:
http://lists.denx.de/pipermail/u-boot/2014-April/178280.html

Thanks,

         M.
-- 
Fast, cheap, reliable. Pick two.

[U-Boot] [PATCH v3 10/13] ARM: HYP/non-sec/PSCI: emit DT nodes

[Ian Campbell]

On Thu, 2014-05-08 at 07:22 +0100, Marc Zyngier wrote:
> Hi Ian,
> 
> On 2014-05-07 20:24, Ian Campbell wrote:
> > On Sat, 2014-02-15 at 13:36 +0000, Marc Zyngier wrote:
> >> @@ -32,5 +33,11 @@ int arch_fixup_fdt(void *blob)
> >>  		size[bank] = bd->bi_dram[bank].size;
> >>  	}
> >>
> >> -	return fdt_fixup_memory_banks(blob, start, size, 
> >> CONFIG_NR_DRAM_BANKS);
> >> +	ret = fdt_fixup_memory_banks(blob, start, size, 
> >> CONFIG_NR_DRAM_BANKS);
> >> +	if (ret)
> >> +		return ret;
> >> +
> >> +#if defined(CONFIG_ARMV7_NONSEC) || defined(CONFIG_ARMV7_VIRT)
> >> +	return armv7_update_dt(blob);
> >> +#endif
> >
> > For platforms where neither of these CONFIG_* are set this results 
> > in:
> > arch/arm/lib/bootm-fdt.c: In function ?arch_fixup_fdt?:
> > arch/arm/lib/bootm-fdt.c:43:1: warning: control reaches end of
> > non-void function [-Wreturn-type]
> >  }
> 
> I'm afraid this is old news. ;-)
> 
> This bug has been fixed in v4:
> http://lists.denx.de/pipermail/u-boot/2014-April/178280.html

AH, I thought I had the latest from your tree and I was only ccd on v3
so I missed it.

Ian.

[U-Boot] smp_kick_all_cpus() function's role

[TigerLiu at via-alliance.com]

Hi, Marc:
I am studying ARMv8's u-boot code with FVP model.
In do_nonsec_virt_switch() function in bootm.c :
It will call smp_kick_all_cpus() function :
It seems it would set GICD_SGIR[24] = 1, forward the interrupt to all
CPU interfaces except tha tof the processor that requested the
interrupt.

So, who generated the interrupt(which would be forwarded to other
cores)?

Best wishes,

[U-Boot] smp_kick_all_cpus() function's role

[Marc Zyngier]

Hi Liu,

On 30/05/14 03:25, TigerLiu at via-alliance.com wrote:
> Hi, Marc:
> I am studying ARMv8's u-boot code with FVP model.
> In do_nonsec_virt_switch() function in bootm.c :
> It will call smp_kick_all_cpus() function :
> It seems it would set GICD_SGIR[24] = 1, forward the interrupt to all
> CPU interfaces except tha tof the processor that requested the
> interrupt.
> 
> So, who generated the interrupt(which would be forwarded to other
> cores)?

I suggest you have a look at the GICv2 architecture document, section
4.3.15, which describes the GICD_SGIR register. Writing to this register
generates the interrupt (SGI number in GICD_SGIR[3:0}), and
GICD_SGIR[25:24] determines who gets it.

In short, if you're setting GICD_SGIR[24] to 1, you're sending SGI0 to
all CPUs but yourself. This seems to match the name of the function,
doesn't it?

	M.
-- 
Jazz is not dead. It just smells funny...

[U-Boot] smp_kick_all_cpus() function's role

[TigerLiu at via-alliance.com]

Hi? Marc:
>In short, if you're setting GICD_SGIR[24] to 1, you're sending SGI0 to
>all CPUs but yourself. This seems to match the name of the function,
>doesn't it?
I described my understanding based on 2014.07-RC2 u-boot source code:
(For ARMv8 cores)
1. smp_kick_all_cpus() will send SGI0 to all other cores except BSP.
   These non-BSP cores handled this SGI0 in gic_wait_for_interrupt(), and then switched to EL2/EL1 .
   These code is implemented in lowlevel_init in arch/arm/cpu/armv8/start.S.
  Is my understanding right?
2. if runing with ATF(Arm Trusted Firmware) + Uboot.bin
   ATF has put non-BSP cores to WFI state.
   So, before jumping to u-boot's entrypoint, there is only a BSP .
   So, smp_kick_all_cpus() could wake up these non-BSP cores?

Best wishes,

[U-Boot] smp_kick_all_cpus() function's role

[Marc Zyngier]

On Tue, Jun 03 2014 at  3:16:19 am BST, "TigerLiu at via-alliance.com" <TigerLiu@via-alliance.com> wrote:
> Hi? Marc:
>>In short, if you're setting GICD_SGIR[24] to 1, you're sending SGI0 to
>>all CPUs but yourself. This seems to match the name of the function,
>>doesn't it?
> I described my understanding based on 2014.07-RC2 u-boot source code:
> (For ARMv8 cores)
> 1. smp_kick_all_cpus() will send SGI0 to all other cores except BSP.
>    These non-BSP cores handled this SGI0 in gic_wait_for_interrupt(), and then switched to EL2/EL1 .
>    These code is implemented in lowlevel_init in arch/arm/cpu/armv8/start.S.
>   Is my understanding right?

I can't tell, I haven't read that bit of code. But that seems similar to
what ARMv7 used to do.

> 2. if runing with ATF(Arm Trusted Firmware) + Uboot.bin
>    ATF has put non-BSP cores to WFI state.
>    So, before jumping to u-boot's entrypoint, there is only a BSP .
>    So, smp_kick_all_cpus() could wake up these non-BSP cores?

My understanding is that if you're using the Trusted Firmware, then you
have an implementation of PSCI, and that's what you must use to bring
the CPUs into u-boot. U-Boot will be running non-secure anyway, so it
requires the firmware to perform S to NS transition on its behalf.

	M.
-- 
Jazz is not dead. It just smells funny.

[U-Boot] smp_kick_all_cpus() function's role

[TigerLiu at via-alliance.com]

Hi, Marc:
>My understanding is that if you're using the Trusted Firmware, then you
>have an implementation of PSCI, and that's what you must use to bring
>the CPUs into u-boot. U-Boot will be running non-secure anyway, so it
>requires the firmware to perform S to NS transition on its behalf.
Do you mean :
Waking up Non-BSP cores through PSCI interface, and then let them switch
to Non-Secure state through smp_kick_all_cpus()?

And another question:
1. how to determine successfully transitioning to Non-Secure?
   Is there any register to indicate current state is Non-Secure state?
   And after transitioning to non-secure state, I tried to access SCR
register,but it caused system hang.

Best wishes,

[U-Boot] smp_kick_all_cpus() function's role

[Marc Zyngier]

On Tue, Jun 03 2014 at 10:41:51 am BST, "TigerLiu at via-alliance.com" <TigerLiu@via-alliance.com> wrote:
> Hi, Marc:
>>My understanding is that if you're using the Trusted Firmware, then you
>>have an implementation of PSCI, and that's what you must use to bring
>>the CPUs into u-boot. U-Boot will be running non-secure anyway, so it
>>requires the firmware to perform S to NS transition on its behalf.
> Do you mean :
> Waking up Non-BSP cores through PSCI interface, and then let them switch
> to Non-Secure state through smp_kick_all_cpus()?

No. You don't need smp_kick_all_cpus at all. Just call the PSCI firmware
to wake up the secondary CPUs, and they will be directly placed in
non-secure mode.

> And another question:
> 1. how to determine successfully transitioning to Non-Secure?
>    Is there any register to indicate current state is Non-Secure state?
>    And after transitioning to non-secure state, I tried to access SCR
> register,but it caused system hang.

No, there is no architectural way. But if you go from EL3 to EL2,
looking at the mode in PSTATE is pretty easy.

	M.
-- 
Jazz is not dead. It just smells funny.

[U-Boot] [PATCH v3 06/13] ARM: HYP/non-sec: allow relocation to secure RAM

[surya.satyavolu at sirabtech.com]

I am trying to bring up xen suing u-boot that has this patch. Unfortunately as soon as the code tries to call _nonsec_init through secure_ram_addr in arm7_init_nonsec function in virt-v7.c I get an undefined instruction exception. I suspect the CONFIG_ARMV7_SECURE_BASE needs to be defined to a particular value. What should that be defined to for omap5432?
Surya

On Saturday, February 15, 2014 at 5:36:30 AM UTC-8, Marc Zyngier wrote:
> The current non-sec switching code suffers from one major issue:
> it cannot run in secure RAM, as a large part of u-boot still needs
> to be run while we're switched to non-secure.
> 
> This patch reworks the whole HYP/non-secure strategy by:
> - making sure the secure code is the *last* thing u-boot executes
>   before entering the payload
> - performing an exception return from secure mode directly into
>   the payload
> - allowing the code to be dynamically relocated to secure RAM
>   before switching to non-secure.
> 
> This involves quite a bit of horrible code, specially as u-boot
> relocation is quite primitive.
> 
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
>  arch/arm/cpu/armv7/nonsec_virt.S | 161 +++++++++++++++++++--------------------
>  arch/arm/cpu/armv7/virt-v7.c     |  59 +++++---------
>  arch/arm/include/asm/armv7.h     |  10 ++-
>  arch/arm/include/asm/secure.h    |  26 +++++++
>  arch/arm/lib/bootm.c             |  22 +++---
>  5 files changed, 138 insertions(+), 140 deletions(-)
>  create mode 100644 arch/arm/include/asm/secure.h
> 
> diff --git a/arch/arm/cpu/armv7/nonsec_virt.S b/arch/arm/cpu/armv7/nonsec_virt.S
> index b5c946f..2a43e3c 100644
> --- a/arch/arm/cpu/armv7/nonsec_virt.S
> +++ b/arch/arm/cpu/armv7/nonsec_virt.S
> @@ -10,10 +10,13 @@
>  #include <linux/linkage.h>
>  #include <asm/gic.h>
>  #include <asm/armv7.h>
> +#include <asm/proc-armv/ptrace.h>
>  
>  .arch_extension sec
>  .arch_extension virt
>  
> +	.pushsection ._secure.text, "ax"
> +
>  	.align	5
>  /* the vector table for secure state and HYP mode */
>  _monitor_vectors:
> @@ -22,51 +25,86 @@ _monitor_vectors:
>  	adr pc, _secure_monitor
>  	.word 0
>  	.word 0
> -	adr pc, _hyp_trap
> +	.word 0
>  	.word 0
>  	.word 0
>  
> +.macro is_cpu_virt_capable	tmp
> +	mrc	p15, 0, \tmp, c0, c1, 1		@ read ID_PFR1
> +	and	\tmp, \tmp, #CPUID_ARM_VIRT_MASK	@ mask virtualization bits
> +	cmp	\tmp, #(1 << CPUID_ARM_VIRT_SHIFT)
> +.endm
> +
>  /*
>   * secure monitor handler
>   * U-boot calls this "software interrupt" in start.S
>   * This is executed on a "smc" instruction, we use a "smc #0" to switch
>   * to non-secure state.
> - * We use only r0 and r1 here, due to constraints in the caller.
> + * r0, r1, r2: passed to the callee
> + * ip: target PC
>   */
>  _secure_monitor:
> -	mrc	p15, 0, r1, c1, c1, 0		@ read SCR
> -	bic	r1, r1, #0x4e			@ clear IRQ, FIQ, EA, nET bits
> -	orr	r1, r1, #0x31			@ enable NS, AW, FW bits
> +	mrc	p15, 0, r5, c1, c1, 0		@ read SCR
> +	bic	r5, r5, #0x4e			@ clear IRQ, FIQ, EA, nET bits
> +	orr	r5, r5, #0x31			@ enable NS, AW, FW bits
>  
> -	mrc	p15, 0, r0, c0, c1, 1		@ read ID_PFR1
> -	and	r0, r0, #CPUID_ARM_VIRT_MASK	@ mask virtualization bits
> -	cmp	r0, #(1 << CPUID_ARM_VIRT_SHIFT)
> +	mov	r6, #SVC_MODE			@ default mode is SVC
> +	is_cpu_virt_capable r4
>  #ifdef CONFIG_ARMV7_VIRT
> -	orreq	r1, r1, #0x100			@ allow HVC instruction
> +	orreq	r5, r5, #0x100			@ allow HVC instruction
> +	moveq	r6, #HYP_MODE			@ Enter the kernel as HYP
>  #endif
>  
> -	mcr	p15, 0, r1, c1, c1, 0		@ write SCR (with NS bit set)
> +	mcr	p15, 0, r5, c1, c1, 0		@ write SCR (with NS bit set)
>  	isb
>  
> -#ifdef CONFIG_ARMV7_VIRT
> -	mrceq	p15, 0, r0, c12, c0, 1		@ get MVBAR value
> -	mcreq	p15, 4, r0, c12, c0, 0		@ write HVBAR
> -#endif
>  	bne	1f
>  
>  	@ Reset CNTVOFF to 0 before leaving monitor mode
> -	mrc	p15, 0, r0, c0, c1, 1		@ read ID_PFR1
> -	ands	r0, r0, #CPUID_ARM_GENTIMER_MASK	@ test arch timer bits
> -	movne	r0, #0
> -	mcrrne	p15, 4, r0, r0, c14		@ Reset CNTVOFF to zero
> +	mrc	p15, 0, r4, c0, c1, 1		@ read ID_PFR1
> +	ands	r4, r4, #CPUID_ARM_GENTIMER_MASK	@ test arch timer bits
> +	movne	r4, #0
> +	mcrrne	p15, 4, r4, r4, c14		@ Reset CNTVOFF to zero
>  1:
> -	movs	pc, lr				@ return to non-secure SVC
> -
> -_hyp_trap:
> -	mrs	lr, elr_hyp	@ for older asm: .byte 0x00, 0xe3, 0x0e, 0xe1
> -	mov pc, lr				@ do no switch modes, but
> -						@ return to caller
> -
> +	mov	lr, ip
> +	mov	ip, #(F_BIT | I_BIT | A_BIT)	@ Set A, I and F
> +	tst	lr, #1				@ Check for Thumb PC
> +	orrne	ip, ip, #T_BIT			@ Set T if Thumb
> +	orr	ip, ip, r6			@ Slot target mode in
> +	msr	spsr_cxfs, ip			@ Set full SPSR
> +	movs	pc, lr				@ ERET to non-secure
> +
> +ENTRY(_do_nonsec_entry)
> +	mov	ip, r0
> +	mov	r0, r1
> +	mov	r1, r2
> +	mov	r2, r3
> +	smc	#0
> +ENDPROC(_do_nonsec_entry)
> +
> +.macro get_cbar_addr	addr
> +#ifdef CONFIG_ARM_GIC_BASE_ADDRESS
> +	ldr	\addr, =CONFIG_ARM_GIC_BASE_ADDRESS
> +#else
> +	mrc	p15, 4, \addr, c15, c0, 0	@ read CBAR
> +	bfc	\addr, #0, #15			@ clear reserved bits
> +#endif
> +.endm
> +
> +.macro get_gicd_addr	addr
> +	get_cbar_addr	\addr
> +	add	\addr, \addr, #GIC_DIST_OFFSET	@ GIC dist i/f offset
> +.endm
> +
> +.macro get_gicc_addr	addr, tmp
> +	get_cbar_addr	\addr
> +	is_cpu_virt_capable \tmp
> +	movne	\tmp, #GIC_CPU_OFFSET_A9	@ GIC CPU offset for A9
> +	moveq	\tmp, #GIC_CPU_OFFSET_A15	@ GIC CPU offset for A15/A7
> +	add	\addr, \addr, \tmp
> +.endm
> +
> +#ifndef CONFIG_ARMV7_PSCI
>  /*
>   * Secondary CPUs start here and call the code for the core specific parts
>   * of the non-secure and HYP mode transition. The GIC distributor specific
> @@ -74,31 +112,21 @@ _hyp_trap:
>   * Then they go back to wfi and wait to be woken up by the kernel again.
>   */
>  ENTRY(_smp_pen)
> -	mrs	r0, cpsr
> -	orr	r0, r0, #0xc0
> -	msr	cpsr, r0			@ disable interrupts
> -	ldr	r1, =_start
> -	mcr	p15, 0, r1, c12, c0, 0		@ set VBAR
> +	cpsid	i
> +	cpsid	f
>  
>  	bl	_nonsec_init
> -	mov	r12, r0				@ save GICC address
> -#ifdef CONFIG_ARMV7_VIRT
> -	bl	_switch_to_hyp
> -#endif
> -
> -	ldr	r1, [r12, #GICC_IAR]		@ acknowledge IPI
> -	str	r1, [r12, #GICC_EOIR]		@ signal end of interrupt
>  
>  	adr	r0, _smp_pen			@ do not use this address again
>  	b	smp_waitloop			@ wait for IPIs, board specific
>  ENDPROC(_smp_pen)
> +#endif
>  
>  /*
>   * Switch a core to non-secure state.
>   *
>   *  1. initialize the GIC per-core interface
>   *  2. allow coprocessor access in non-secure modes
> - *  3. switch the cpu mode (by calling "smc #0")
>   *
>   * Called from smp_pen by secondary cores and directly by the BSP.
>   * Do not assume that the stack is available and only use registers
> @@ -108,38 +136,23 @@ ENDPROC(_smp_pen)
>   * though, but we check this in C before calling this function.
>   */
>  ENTRY(_nonsec_init)
> -#ifdef CONFIG_ARM_GIC_BASE_ADDRESS
> -	ldr	r2, =CONFIG_ARM_GIC_BASE_ADDRESS
> -#else
> -	mrc	p15, 4, r2, c15, c0, 0		@ read CBAR
> -	bfc	r2, #0, #15			@ clear reserved bits
> -#endif
> -	add	r3, r2, #GIC_DIST_OFFSET	@ GIC dist i/f offset
> +	get_gicd_addr	r3
> +
>  	mvn	r1, #0				@ all bits to 1
>  	str	r1, [r3, #GICD_IGROUPRn]	@ allow private interrupts
>  
> -	mrc	p15, 0, r0, c0, c0, 0		@ read MIDR
> -	ldr	r1, =MIDR_PRIMARY_PART_MASK
> -	and	r0, r0, r1			@ mask out variant and revision
> +	get_gicc_addr	r3, r1
>  
> -	ldr	r1, =MIDR_CORTEX_A7_R0P0 & MIDR_PRIMARY_PART_MASK
> -	cmp	r0, r1				@ check for Cortex-A7
> -
> -	ldr	r1, =MIDR_CORTEX_A15_R0P0 & MIDR_PRIMARY_PART_MASK
> -	cmpne	r0, r1				@ check for Cortex-A15
> -
> -	movne	r1, #GIC_CPU_OFFSET_A9		@ GIC CPU offset for A9
> -	moveq	r1, #GIC_CPU_OFFSET_A15		@ GIC CPU offset for A15/A7
> -	add	r3, r2, r1			@ r3 = GIC CPU i/f addr
> -
> -	mov	r1, #1				@ set GICC_CTLR[enable]
> +	mov	r1, #3				@ Enable both groups
>  	str	r1, [r3, #GICC_CTLR]		@ and clear all other bits
>  	mov	r1, #0xff
>  	str	r1, [r3, #GICC_PMR]		@ set priority mask register
>  
> +	mrc	p15, 0, r0, c1, c1, 2
>  	movw	r1, #0x3fff
> -	movt	r1, #0x0006
> -	mcr	p15, 0, r1, c1, c1, 2		@ NSACR = all copros to non-sec
> +	movt	r1, #0x0004
> +	orr	r0, r0, r1
> +	mcr	p15, 0, r0, c1, c1, 2		@ NSACR = all copros to non-sec
>  
>  /* The CNTFRQ register of the generic timer needs to be
>   * programmed in secure state. Some primary bootloaders / firmware
> @@ -157,21 +170,9 @@ ENTRY(_nonsec_init)
>  
>  	adr	r1, _monitor_vectors
>  	mcr	p15, 0, r1, c12, c0, 1		@ set MVBAR to secure vectors
> -
> -	mrc	p15, 0, ip, c12, c0, 0		@ save secure copy of VBAR
> -
>  	isb
> -	smc	#0				@ call into MONITOR mode
> -
> -	mcr	p15, 0, ip, c12, c0, 0		@ write non-secure copy of VBAR
> -
> -	mov	r1, #1
> -	str	r1, [r3, #GICC_CTLR]		@ enable non-secure CPU i/f
> -	add	r2, r2, #GIC_DIST_OFFSET
> -	str	r1, [r2, #GICD_CTLR]		@ allow private interrupts
>  
>  	mov	r0, r3				@ return GICC address
> -
>  	bx	lr
>  ENDPROC(_nonsec_init)
>  
> @@ -183,18 +184,10 @@ ENTRY(smp_waitloop)
>  	ldr	r1, [r1]
>  	cmp	r0, r1			@ make sure we dont execute this code
>  	beq	smp_waitloop		@ again (due to a spurious wakeup)
> -	mov	pc, r1
> +	mov	r0, r1
> +	b	_do_nonsec_entry
>  ENDPROC(smp_waitloop)
>  .weak smp_waitloop
>  #endif
>  
> -ENTRY(_switch_to_hyp)
> -	mov	r0, lr
> -	mov	r1, sp				@ save SVC copy of LR and SP
> -	isb
> -	hvc #0			 @ for older asm: .byte 0x70, 0x00, 0x40, 0xe1
> -	mov	sp, r1
> -	mov	lr, r0				@ restore SVC copy of LR and SP
> -
> -	bx	lr
> -ENDPROC(_switch_to_hyp)
> +	.popsection
> diff --git a/arch/arm/cpu/armv7/virt-v7.c b/arch/arm/cpu/armv7/virt-v7.c
> index 2cd604f..6500030 100644
> --- a/arch/arm/cpu/armv7/virt-v7.c
> +++ b/arch/arm/cpu/armv7/virt-v7.c
> @@ -13,17 +13,10 @@
>  #include <asm/armv7.h>
>  #include <asm/gic.h>
>  #include <asm/io.h>
> +#include <asm/secure.h>
>  
>  unsigned long gic_dist_addr;
>  
> -static unsigned int read_cpsr(void)
> -{
> -	unsigned int reg;
> -
> -	asm volatile ("mrs %0, cpsr\n" : "=r" (reg));
> -	return reg;
> -}
> -
>  static unsigned int read_id_pfr1(void)
>  {
>  	unsigned int reg;
> @@ -72,6 +65,18 @@ static unsigned long get_gicd_base_address(void)
>  #endif
>  }
>  
> +static void relocate_secure_section(void)
> +{
> +#ifdef CONFIG_ARMV7_SECURE_BASE
> +	size_t sz = __secure_end - __secure_start;
> +
> +	memcpy((void *)CONFIG_ARMV7_SECURE_BASE, __secure_start, sz);
> +	flush_dcache_range(CONFIG_ARMV7_SECURE_BASE,
> +			   CONFIG_ARMV7_SECURE_BASE + sz + 1);
> +	invalidate_icache_all();
> +#endif
> +}
> +
>  static void kick_secondary_cpus_gic(unsigned long gicdaddr)
>  {
>  	/* kick all CPUs (except this one) by writing to GICD_SGIR */
> @@ -83,35 +88,7 @@ void __weak smp_kick_all_cpus(void)
>  	kick_secondary_cpus_gic(gic_dist_addr);
>  }
>  
> -int armv7_switch_hyp(void)
> -{
> -	unsigned int reg;
> -
> -	/* check whether we are in HYP mode already */
> -	if ((read_cpsr() & 0x1f) == 0x1a) {
> -		debug("CPU already in HYP mode\n");
> -		return 0;
> -	}
> -
> -	/* check whether the CPU supports the virtualization extensions */
> -	reg = read_id_pfr1();
> -	if ((reg & CPUID_ARM_VIRT_MASK) != 1 << CPUID_ARM_VIRT_SHIFT) {
> -		printf("HYP mode: Virtualization extensions not implemented.\n");
> -		return -1;
> -	}
> -
> -	/* call the HYP switching code on this CPU also */
> -	_switch_to_hyp();
> -
> -	if ((read_cpsr() & 0x1F) != 0x1a) {
> -		printf("HYP mode: switch not successful.\n");
> -		return -1;
> -	}
> -
> -	return 0;
> -}
> -
> -int armv7_switch_nonsec(void)
> +int armv7_init_nonsec(void)
>  {
>  	unsigned int reg;
>  	unsigned itlinesnr, i;
> @@ -147,11 +124,13 @@ int armv7_switch_nonsec(void)
>  	for (i = 1; i <= itlinesnr; i++)
>  		writel((unsigned)-1, gic_dist_addr + GICD_IGROUPRn + 4 * i);
>  
> -	smp_set_core_boot_addr((unsigned long)_smp_pen, -1);
> +#ifndef CONFIG_ARMV7_PSCI
> +	smp_set_core_boot_addr((unsigned long)secure_ram_addr(_smp_pen), -1);
>  	smp_kick_all_cpus();
> +#endif
>  
>  	/* call the non-sec switching code on this CPU also */
> -	_nonsec_init();
> -
> +	relocate_secure_section();
> +	secure_ram_addr(_nonsec_init)();
>  	return 0;
>  }
> diff --git a/arch/arm/include/asm/armv7.h b/arch/arm/include/asm/armv7.h
> index 395444e..11476dd 100644
> --- a/arch/arm/include/asm/armv7.h
> +++ b/arch/arm/include/asm/armv7.h
> @@ -78,13 +78,17 @@ void v7_outer_cache_inval_range(u32 start, u32 end);
>  
>  #if defined(CONFIG_ARMV7_NONSEC) || defined(CONFIG_ARMV7_VIRT)
>  
> -int armv7_switch_nonsec(void);
> -int armv7_switch_hyp(void);
> +int armv7_init_nonsec(void);
>  
>  /* defined in assembly file */
>  unsigned int _nonsec_init(void);
> +void _do_nonsec_entry(void *target_pc, unsigned long r0,
> +		      unsigned long r1, unsigned long r2);
>  void _smp_pen(void);
> -void _switch_to_hyp(void);
> +
> +extern char __secure_start[];
> +extern char __secure_end[];
> +
>  #endif /* CONFIG_ARMV7_NONSEC || CONFIG_ARMV7_VIRT */
>  
>  #endif /* ! __ASSEMBLY__ */
> diff --git a/arch/arm/include/asm/secure.h b/arch/arm/include/asm/secure.h
> new file mode 100644
> index 0000000..effdb18
> --- /dev/null
> +++ b/arch/arm/include/asm/secure.h
> @@ -0,0 +1,26 @@
> +#ifndef __ASM_SECURE_H
> +#define __ASM_SECURE_H
> +
> +#include <config.h>
> +
> +#ifdef CONFIG_ARMV7_SECURE_BASE
> +/*
> + * Warning, horror ahead.
> + *
> + * The target code lives in our "secure ram", but u-boot doesn't know
> + * that, and has blindly added reloc_off to every relocation
> + * entry. Gahh. Do the opposite conversion. This hack also prevents
> + * GCC from generating code veeners, which u-boot doesn't relocate at
> + * all...
> + */
> +#define secure_ram_addr(_fn) ({						\
> +			DECLARE_GLOBAL_DATA_PTR;			\
> +			void *__fn = _fn;				\
> +			typeof(_fn) *__tmp = (__fn - gd->reloc_off);	\
> +			__tmp;						\
> +		})
> +#else
> +#define secure_ram_addr(_fn)	(_fn)
> +#endif
> +
> +#endif
> diff --git a/arch/arm/lib/bootm.c b/arch/arm/lib/bootm.c
> index 68554c8..ecc25f9 100644
> --- a/arch/arm/lib/bootm.c
> +++ b/arch/arm/lib/bootm.c
> @@ -20,6 +20,7 @@
>  #include <libfdt.h>
>  #include <fdt_support.h>
>  #include <asm/bootm.h>
> +#include <asm/secure.h>
>  #include <linux/compiler.h>
>  
>  #if defined(CONFIG_ARMV7_NONSEC) || defined(CONFIG_ARMV7_VIRT)
> @@ -185,26 +186,16 @@ static void setup_end_tag(bd_t *bd)
>  
>  __weak void setup_board_tags(struct tag **in_params) {}
>  
> +#ifdef CONFIG_ARM64
>  static void do_nonsec_virt_switch(void)
>  {
> -#if defined(CONFIG_ARMV7_NONSEC) || defined(CONFIG_ARMV7_VIRT)
> -	if (armv7_switch_nonsec() == 0)
> -#ifdef CONFIG_ARMV7_VIRT
> -		if (armv7_switch_hyp() == 0)
> -			debug("entered HYP mode\n");
> -#else
> -		debug("entered non-secure state\n");
> -#endif
> -#endif
> -
> -#ifdef CONFIG_ARM64
>  	smp_kick_all_cpus();
>  	armv8_switch_to_el2();
>  #ifdef CONFIG_ARMV8_SWITCH_TO_EL1
>  	armv8_switch_to_el1();
>  #endif
> -#endif
>  }
> +#endif
>  
>  /* Subcommand: PREP */
>  static void boot_prep_linux(bootm_headers_t *images)
> @@ -287,8 +278,13 @@ static void boot_jump_linux(bootm_headers_t *images, int flag)
>  		r2 = gd->bd->bi_boot_params;
>  
>  	if (!fake) {
> -		do_nonsec_virt_switch();
> +#if defined(CONFIG_ARMV7_NONSEC) || defined(CONFIG_ARMV7_VIRT)
> +		armv7_init_nonsec();
> +		secure_ram_addr(_do_nonsec_entry)(kernel_entry,
> +						  0, machid, r2);
> +#else
>  		kernel_entry(0, machid, r2);
> +#endif
>  	}
>  #endif
>  }
> -- 
> 1.8.5.1

[U-Boot] [PATCH v3 06/13] ARM: HYP/non-sec: allow relocation to secure RAM

[Marc Zyngier]

On 18/02/15 17:42, surya.satyavolu at sirabtech.com wrote:
> I am trying to bring up xen suing u-boot that has this patch.
> Unfortunately as soon as the code tries to call _nonsec_init through
> secure_ram_addr in arm7_init_nonsec function in virt-v7.c I get an
> undefined instruction exception. I suspect the
> CONFIG_ARMV7_SECURE_BASE needs to be defined to a particular value.
> What should that be defined to for omap5432?

I'm afraid you're barking up the wrong tree.

TI, in its infinite wisdom, drops directly to *non-secure*. So, there is
nothing you can actually do (maybe there's a way to go back to secure
mode, but that's certainly not documented).

The consequence of the above is that you cannot implement PSCI on
OMAP4/5. You'll have to add you own code to promote your CPUs to HYP
(there is a secure call for this).

Good luck,

	M.
-- 
Jazz is not dead. It just smells funny...