[U-Boot] [PATCH 1/2 v3] 85xx: MP Boot Page Translation update

[U-Boot] [PATCH 1/2 v3] 85xx: MP Boot Page Translation update

[Peter Tyser]

This change has 3 goals:
- Have secondary cores be released into spin loops at their 'true'
  address in SDRAM.  Previously, secondary cores were put into spin
  loops in the 0xfffffxxx address range which required that boot page
  translation was always enabled while cores were in their spin loops.

- Allow the TLB window that the primary core uses to access the
  secondary cores boot page to be placed at any address.  Previously, a
  TLB window at 0xfffff000 was always used to access the seconary cores'
  boot page.  This TLB address requirement overlapped with other
  peripherals on some boards (eg XPedite5370).  By default, the boot
  page TLB will still use the 0xfffffxxx address range, but this can be
  overridden on a board-by-board basis by defining a custom
  CONFIG_BPTR_VIRT_ADDR.  Note that the TLB used to map the boot page
  remains in use while U-Boot executes.  Previously it was only
  temporarily used, then restored to its initial value.

- Allow Boot Page Translation to be disabled on bootup.  Previously,
  Boot Page Translation was always left enabled after secondary cores
  were brought out of reset.  This caused the 0xfffffxxx address range
  to somewhat "magically" be translated to an address in SDRAM.  Some
  boards may not want this oddity in their memory map, so defining
  CONFIG_MPC8xxx_DISABLE_BPTR will turn off Boot Page Translation after
  the secondary cores are initialized.

These changes are only applicable to 85xx boards with CONFIG_MP defined.

Signed-off-by: Peter Tyser <ptyser@xes-inc.com>
---
Hi Kumar,
Hopefully this isn't too invasive...  I tried to address your previous
v2 comments of:
- Its possible that the location in memory is not mapped in the TLBs.
  We typically max out @ 2G of DDR mapped (via the TLBs).  So what
  happens if we have 3G of DDR?
- We expect the the address that the spin table is at to be marked
  cache-inhibited.

Changes since v1:
- Use clrbits_be32() instead of in/out_be32()

- Boot page translation is only disabled when CONFIG_MPC8xxx_DISABLT_BPTR
  is defined.

- Instead of zeroing out BPTR when disabling translation, this
  patch only disables translation, but maintins the translation
  address.  This should make it easier to properly re-enable
  translation if need be.

Changes since v2:
- Use CONFIG_BPTR_VIRT_ADDR to determine virtual address of secondary
  cores' boot page
- Split get_spin_addr() into get_[phys|virt]_addr()

 cpu/mpc85xx/fdt.c        |    2 +-
 cpu/mpc85xx/mp.c         |   70 ++++++++++++++++++++++++++++++----------------
 cpu/mpc85xx/mp.h         |    3 +-
 cpu/mpc85xx/release.S    |   34 +++++++++++++++++++---
 include/asm-ppc/config.h |   10 ++++++
 5 files changed, 88 insertions(+), 31 deletions(-)

diff --git a/cpu/mpc85xx/fdt.c b/cpu/mpc85xx/fdt.c
index efb6518..de2dcac 100644
--- a/cpu/mpc85xx/fdt.c
+++ b/cpu/mpc85xx/fdt.c
@@ -43,7 +43,7 @@ extern void ft_fixup_num_cores(void *blob);
 void ft_fixup_cpu(void *blob, u64 memory_limit)
 {
 	int off;
-	ulong spin_tbl_addr = get_spin_addr();
+	ulong spin_tbl_addr = get_spin_phys_addr();
 	u32 bootpg = determine_mp_bootpg();
 	u32 id = get_my_id();
 
diff --git a/cpu/mpc85xx/mp.c b/cpu/mpc85xx/mp.c
index b5c6020..00b6450 100644
--- a/cpu/mpc85xx/mp.c
+++ b/cpu/mpc85xx/mp.c
@@ -52,10 +52,10 @@ int cpu_status(int nr)
 	u32 *table, id = get_my_id();
 
 	if (nr == id) {
-		table = (u32 *)get_spin_addr();
+		table = (u32 *)get_spin_virt_addr();
 		printf("table base @ 0x%p\n", table);
 	} else {
-		table = (u32 *)get_spin_addr() + nr * NUM_BOOT_ENTRY;
+		table = (u32 *)get_spin_virt_addr() + nr * NUM_BOOT_ENTRY;
 		printf("Running on cpu %d\n", id);
 		printf("\n");
 		printf("table @ 0x%p\n", table);
@@ -77,7 +77,7 @@ static u8 boot_entry_map[4] = {
 
 int cpu_release(int nr, int argc, char *argv[])
 {
-	u32 i, val, *table = (u32 *)get_spin_addr() + nr * NUM_BOOT_ENTRY;
+	u32 i, val, *table = (u32 *)get_spin_virt_addr() + nr * NUM_BOOT_ENTRY;
 	u64 boot_addr;
 
 	if (nr == get_my_id()) {
@@ -124,23 +124,29 @@ u32 determine_mp_bootpg(void)
 	return (gd->ram_size - 4096);
 }
 
-ulong get_spin_addr(void)
+ulong get_spin_phys_addr(void)
 {
 	extern ulong __secondary_start_page;
 	extern ulong __spin_table;
 
-	ulong addr =
-		(ulong)&__spin_table - (ulong)&__secondary_start_page;
-	addr += 0xfffff000;
+	return (determine_mp_bootpg() +
+		(ulong)&__spin_table - (ulong)&__secondary_start_page);
+}
+
+ulong get_spin_virt_addr(void)
+{
+	extern ulong __secondary_start_page;
+	extern ulong __spin_table;
 
-	return addr;
+	return (CONFIG_BPTR_VIRT_ADDR +
+		(ulong)&__spin_table - (ulong)&__secondary_start_page);
 }
 
 #ifdef CONFIG_FSL_CORENET
 static void plat_mp_up(unsigned long bootpg)
 {
 	u32 up, cpu_up_mask, whoami;
-	u32 *table = (u32 *)get_spin_addr();
+	u32 *table = (u32 *)get_spin_virt_addr();
 	volatile ccsr_gur_t *gur;
 	volatile ccsr_local_t *ccm;
 	volatile ccsr_rcpm_t *rcpm;
@@ -194,12 +200,23 @@ static void plat_mp_up(unsigned long bootpg)
 	mtspr(SPRN_TBWU, 0);
 	mtspr(SPRN_TBWL, 0);
 	out_be32(&rcpm->ctbenrl, (1 << nr_cpus) - 1);
+
+#ifdef CONFIG_MPC8xxx_DISABLE_BPTR
+	/*
+	 * Disabling Boot Page Translation allows the memory region 0xfffff000
+	 * to 0xffffffff to be used normally.  Leaving Boot Page Translation
+	 * enabled remaps 0xfffff000 to SDRAM which makes that memory region
+	 * unusable for normal operation but it does allow OSes to easily
+	 * reset a processor core to put it back into U-Boot's spinloop.
+	 */
+	clrbits_be32(&ecm->bptr, 0x80000000);
+#endif
 }
 #else
 static void plat_mp_up(unsigned long bootpg)
 {
 	u32 up, cpu_up_mask, whoami;
-	u32 *table = (u32 *)get_spin_addr();
+	u32 *table = (u32 *)get_spin_virt_addr();
 	volatile u32 bpcr;
 	volatile ccsr_local_ecm_t *ecm = (void *)(CONFIG_SYS_MPC85xx_ECM_ADDR);
 	volatile ccsr_gur_t *gur = (void *)(CONFIG_SYS_MPC85xx_GUTS_ADDR);
@@ -256,6 +273,17 @@ static void plat_mp_up(unsigned long bootpg)
 
 	devdisr &= ~(MPC85xx_DEVDISR_TB0 | MPC85xx_DEVDISR_TB1);
 	out_be32(&gur->devdisr, devdisr);
+
+#ifdef CONFIG_MPC8xxx_DISABLE_BPTR
+	/*
+	 * Disabling Boot Page Translation allows the memory region 0xfffff000
+	 * to 0xffffffff to be used normally.  Leaving Boot Page Translation
+	 * enabled remaps 0xfffff000 to SDRAM which makes that memory region
+	 * unusable for normal operation but it does allow OSes to easily
+	 * reset a processor core to put it back into U-Boot's spinloop.
+	 */
+	clrbits_be32(&ecm->bptr, 0x80000000);
+#endif
 }
 #endif
 
@@ -269,33 +297,27 @@ void cpu_mp_lmb_reserve(struct lmb *lmb)
 void setup_mp(void)
 {
 	extern ulong __secondary_start_page;
+	extern ulong __bootpg_addr;
 	ulong fixup = (ulong)&__secondary_start_page;
 	u32 bootpg = determine_mp_bootpg();
 
+	/* Store the bootpg's SDRAM address for use by secondary CPU cores */
+	__bootpg_addr = bootpg;
+
 	/* look for the tlb covering the reset page, there better be one */
-	int i = find_tlb_idx((void *)0xfffff000, 1);
+	int i = find_tlb_idx((void *)CONFIG_BPTR_VIRT_ADDR, 1);
 
 	/* we found a match */
 	if (i != -1) {
 		/* map reset page to bootpg so we can copy code there */
 		disable_tlb(i);
 
-		set_tlb(1, 0xfffff000, bootpg, /* tlb, epn, rpn */
-			MAS3_SX|MAS3_SW|MAS3_SR, MAS2_M, /* perms, wimge */
-			0, i, BOOKE_PAGESZ_4K, 1); /* ts, esel, tsize, iprot */
-
-		memcpy((void *)0xfffff000, (void *)fixup, 4096);
-		flush_cache(0xfffff000, 4096);
-
-		disable_tlb(i);
-
-		/* setup reset page back to 1:1, we'll use HW boot translation
-		 * to map this where we want
-		 */
-		set_tlb(1, 0xfffff000, 0xfffff000, /* tlb, epn, rpn */
+		set_tlb(1, CONFIG_BPTR_VIRT_ADDR, bootpg, /* tlb, epn, rpn */
 			MAS3_SX|MAS3_SW|MAS3_SR, MAS2_I, /* perms, wimge */
 			0, i, BOOKE_PAGESZ_4K, 1); /* ts, esel, tsize, iprot */
 
+		memcpy((void *)CONFIG_BPTR_VIRT_ADDR, (void *)fixup, 4096);
+
 		plat_mp_up(bootpg);
 	} else {
 		puts("WARNING: No reset page TLB. "
diff --git a/cpu/mpc85xx/mp.h b/cpu/mpc85xx/mp.h
index 2c2929e..3422cc1 100644
--- a/cpu/mpc85xx/mp.h
+++ b/cpu/mpc85xx/mp.h
@@ -3,7 +3,8 @@
 
 #include <asm/mp.h>
 
-ulong get_spin_addr(void);
+ulong get_spin_phys_addr(void);
+ulong get_spin_virt_addr(void);
 u32 get_my_id(void);
 
 #define BOOT_ENTRY_ADDR_UPPER	0
diff --git a/cpu/mpc85xx/release.S b/cpu/mpc85xx/release.S
index ecbd0d5..a1ae78a 100644
--- a/cpu/mpc85xx/release.S
+++ b/cpu/mpc85xx/release.S
@@ -138,23 +138,38 @@ __secondary_start_page:
 	stw	r3,ENTRY_R6_UPPER(r10)
 	stw	r3,ENTRY_R6_LOWER(r10)
 
+	/* load r13 with the address of the 'bootpg' in SDRAM */
+	lis	r13,toreset(__bootpg_addr)@h
+	ori	r13,r13,toreset(__bootpg_addr)@l
+	lwz	r13,0(r13)
+
 	/* setup mapping for AS = 1, and jump there */
 	lis	r11,(MAS0_TLBSEL(1)|MAS0_ESEL(1))@h
 	mtspr	SPRN_MAS0,r11
 	lis	r11,(MAS1_VALID|MAS1_IPROT)@h
 	ori	r11,r11,(MAS1_TS|MAS1_TSIZE(BOOKE_PAGESZ_4K))@l
 	mtspr	SPRN_MAS1,r11
-	lis	r11,(0xfffff000|MAS2_I)@h
-	ori	r11,r11,(0xfffff000|MAS2_I)@l
+	oris	r11,r13,(MAS2_I)@h
+	ori	r11,r13,(MAS2_I)@l
 	mtspr	SPRN_MAS2,r11
-	lis	r11,(0xfffff000|MAS3_SX|MAS3_SW|MAS3_SR)@h
-	ori	r11,r11,(0xfffff000|MAS3_SX|MAS3_SW|MAS3_SR)@l
+	oris	r11,r13,(MAS3_SX|MAS3_SW|MAS3_SR)@h
+	ori	r11,r13,(MAS3_SX|MAS3_SW|MAS3_SR)@l
 	mtspr	SPRN_MAS3,r11
 	tlbwe
 
 	bl	1f
 1:	mflr	r11
-	addi	r11,r11,28
+	/*
+	 * OR in 0xfff to create a mask of the bootpg SDRAM address.  We use
+	 * this mask to fixup the cpu spin table and the address that we want
+	 * to jump to, eg change them from 0xfffffxxx to 0x7ffffxxx if the
+	 * bootpg is at 0x7ffff000 in SDRAM.
+	 */
+	ori	r13,r13,0xfff
+	and	r11, r11, r13
+	and	r10, r10, r13
+
+	addi	r11,r11,(2f-1b)
 	mfmsr	r13
 	ori	r12,r13,MSR_IS|MSR_DS at l
 
@@ -227,6 +242,15 @@ __secondary_start_page:
 	mtspr	SPRN_SRR1,r13
 	rfi
 
+	/*
+	 * Allocate some space for the SDRAM address of the bootpg.
+	 * This variable has to be in the boot page so that it can
+	 * be accessed by secondary cores when they come out of reset.
+	 */
+	.globl __bootpg_addr
+__bootpg_addr:
+	.long	0
+
 	.align L1_CACHE_SHIFT
 	.globl __spin_table
 __spin_table:
diff --git a/include/asm-ppc/config.h b/include/asm-ppc/config.h
index eba7901..af0853b 100644
--- a/include/asm-ppc/config.h
+++ b/include/asm-ppc/config.h
@@ -47,6 +47,16 @@
 #define CONFIG_MAX_CPUS		1
 #endif
 
+/*
+ * Provide a default boot page translation virtual address that lines up with
+ * Freescale's default e500 reset page.
+ */
+#if (defined(CONFIG_E500) && defined(CONFIG_MP))
+#ifndef CONFIG_BPTR_VIRT_ADDR
+#define CONFIG_BPTR_VIRT_ADDR	0xfffff000
+#endif
+#endif
+
 /* Relocation to SDRAM works on all PPC boards */
 #define CONFIG_RELOC_FIXUP_WORKS
 
-- 
1.6.2.1

[U-Boot] [PATCH 2/2 v3] xpedite5370: Enable multi-core support

[Peter Tyser]

Signed-off-by: Peter Tyser <ptyser@xes-inc.com>
---
 board/xes/xpedite5370/tlb.c   |   13 +++++++++----
 include/configs/XPEDITE5370.h |    8 ++++++++
 2 files changed, 17 insertions(+), 4 deletions(-)

diff --git a/board/xes/xpedite5370/tlb.c b/board/xes/xpedite5370/tlb.c
index caafa30..a465ce3 100644
--- a/board/xes/xpedite5370/tlb.c
+++ b/board/xes/xpedite5370/tlb.c
@@ -61,32 +61,37 @@ struct fsl_e_tlb_entry tlb_table[] = {
 		MAS3_SX|MAS3_SW|MAS3_SR, MAS2_I|MAS2_G,
 		0, 2, BOOKE_PAGESZ_1M, 1),
 
+	/* **M** - Boot page for secondary processors */
+	SET_TLB_ENTRY(1, CONFIG_BPTR_VIRT_ADDR, CONFIG_BPTR_VIRT_ADDR,
+		MAS3_SX|MAS3_SW|MAS3_SR, MAS2_M,
+		0, 3, BOOKE_PAGESZ_4K, 1),
+
 #ifdef CONFIG_PCIE1
 	/* *I*G* - PCIe */
 	SET_TLB_ENTRY(1, CONFIG_SYS_PCIE1_MEM_PHYS, CONFIG_SYS_PCIE1_MEM_PHYS,
 		MAS3_SX|MAS3_SW|MAS3_SR, MAS2_I|MAS2_G,
-		0, 3, BOOKE_PAGESZ_1G, 1),
+		0, 4, BOOKE_PAGESZ_1G, 1),
 #endif
 
 #ifdef CONFIG_PCIE2
 	/* *I*G* - PCIe */
 	SET_TLB_ENTRY(1, CONFIG_SYS_PCIE2_MEM_PHYS, CONFIG_SYS_PCIE2_MEM_PHYS,
 		MAS3_SX|MAS3_SW|MAS3_SR, MAS2_I|MAS2_G,
-		0, 4, BOOKE_PAGESZ_256M, 1),
+		0, 5, BOOKE_PAGESZ_256M, 1),
 #endif
 
 #ifdef CONFIG_PCIE3
 	/* *I*G* - PCIe */
 	SET_TLB_ENTRY(1, CONFIG_SYS_PCIE3_MEM_PHYS, CONFIG_SYS_PCIE3_MEM_PHYS,
 		MAS3_SX|MAS3_SW|MAS3_SR, MAS2_I|MAS2_G,
-		0, 5, BOOKE_PAGESZ_256M, 1),
+		0, 6, BOOKE_PAGESZ_256M, 1),
 #endif
 
 #if defined(CONFIG_PCIE1) || defined(CONFIG_PCIE2) || defined(CONFIG_PCIE3)
 	/* *I*G* - PCIe */
 	SET_TLB_ENTRY(1, CONFIG_SYS_PCIE1_IO_PHYS, CONFIG_SYS_PCIE1_IO_PHYS,
 		MAS3_SX|MAS3_SW|MAS3_SR, MAS2_I|MAS2_G,
-		0, 6, BOOKE_PAGESZ_64M, 1),
+		0, 7, BOOKE_PAGESZ_64M, 1),
 #endif
 
 };
diff --git a/include/configs/XPEDITE5370.h b/include/configs/XPEDITE5370.h
index 26b798b..7782df3 100644
--- a/include/configs/XPEDITE5370.h
+++ b/include/configs/XPEDITE5370.h
@@ -49,6 +49,13 @@
 #define CONFIG_FSL_LAW		1	/* Use common FSL init code */
 
 /*
+ * Multicore config
+ */
+#define CONFIG_MP
+#define CONFIG_BPTR_VIRT_ADDR	0xee000000	/* virt boot page address */
+#define CONFIG_MPC8xxx_DISABLE_BPTR		/* Don't leave BPTR enabled */
+
+/*
  * DDR config
  */
 #define CONFIG_FSL_DDR2
@@ -109,6 +116,7 @@ extern unsigned long get_board_ddr_clk(unsigned long dummy);
  * 0xe000_0000	0xe7ff_ffff	SRAM/SSRAM/L1 Cache	128M non-cacheable
  * 0xe800_0000	0xe87f_ffff	PCIe1 IO		8M non-cacheable
  * 0xe880_0000	0xe8ff_ffff	PCIe2 IO		8M non-cacheable
+ * 0xee00_0000	0xee00_ffff	Boot page translation	4K non-cacheable
  * 0xef00_0000	0xef0f_ffff	CCSR/IMMR		1M non-cacheable
  * 0xef80_0000	0xef8f_ffff	NAND Flash		1M non-cacheable
  * 0xf000_0000	0xf7ff_ffff	NOR Flash 2		128M non-cacheable
-- 
1.6.2.1

[U-Boot] [PATCH 1/2 v3] 85xx: MP Boot Page Translation update

[Kumar Gala]

On Oct 23, 2009, at 3:55 PM, Peter Tyser wrote:

> This change has 3 goals:
> - Have secondary cores be released into spin loops at their 'true'
>  address in SDRAM.  Previously, secondary cores were put into spin
>  loops in the 0xfffffxxx address range which required that boot page
>  translation was always enabled while cores were in their spin loops.
>
> - Allow the TLB window that the primary core uses to access the
>  secondary cores boot page to be placed at any address.  Previously, a
>  TLB window at 0xfffff000 was always used to access the seconary  
> cores'
>  boot page.  This TLB address requirement overlapped with other
>  peripherals on some boards (eg XPedite5370).  By default, the boot
>  page TLB will still use the 0xfffffxxx address range, but this can be
>  overridden on a board-by-board basis by defining a custom
>  CONFIG_BPTR_VIRT_ADDR.  Note that the TLB used to map the boot page
>  remains in use while U-Boot executes.  Previously it was only
>  temporarily used, then restored to its initial value.
>
> - Allow Boot Page Translation to be disabled on bootup.  Previously,
>  Boot Page Translation was always left enabled after secondary cores
>  were brought out of reset.  This caused the 0xfffffxxx address range
>  to somewhat "magically" be translated to an address in SDRAM.  Some
>  boards may not want this oddity in their memory map, so defining
>  CONFIG_MPC8xxx_DISABLE_BPTR will turn off Boot Page Translation after
>  the secondary cores are initialized.
>
> These changes are only applicable to 85xx boards with CONFIG_MP  
> defined.
>
> Signed-off-by: Peter Tyser <ptyser@xes-inc.com>
> ---
> Hi Kumar,
> Hopefully this isn't too invasive...  I tried to address your previous
> v2 comments of:
> - Its possible that the location in memory is not mapped in the TLBs.
>  We typically max out @ 2G of DDR mapped (via the TLBs).  So what
>  happens if we have 3G of DDR?
> - We expect the the address that the spin table is at to be marked
>  cache-inhibited.
>
> Changes since v1:
> - Use clrbits_be32() instead of in/out_be32()
>
> - Boot page translation is only disabled when  
> CONFIG_MPC8xxx_DISABLT_BPTR
>  is defined.
>
> - Instead of zeroing out BPTR when disabling translation, this
>  patch only disables translation, but maintins the translation
>  address.  This should make it easier to properly re-enable
>  translation if need be.
>
> Changes since v2:
> - Use CONFIG_BPTR_VIRT_ADDR to determine virtual address of secondary
>  cores' boot page
> - Split get_spin_addr() into get_[phys|virt]_addr()
>
> cpu/mpc85xx/fdt.c        |    2 +-
> cpu/mpc85xx/mp.c         |   70 +++++++++++++++++++++++++++++ 
> +----------------
> cpu/mpc85xx/mp.h         |    3 +-
> cpu/mpc85xx/release.S    |   34 +++++++++++++++++++---
> include/asm-ppc/config.h |   10 ++++++
> 5 files changed, 88 insertions(+), 31 deletions(-)

applied to 85xx

- k

[U-Boot] [PATCH 2/2 v3] xpedite5370: Enable multi-core support

[Kumar Gala]

On Oct 23, 2009, at 3:55 PM, Peter Tyser wrote:

> Signed-off-by: Peter Tyser <ptyser@xes-inc.com>
> ---
> board/xes/xpedite5370/tlb.c   |   13 +++++++++----
> include/configs/XPEDITE5370.h |    8 ++++++++
> 2 files changed, 17 insertions(+), 4 deletions(-)

applied to 85xx

- k