[U-Boot] [PATCH] ppc/85xx: Fix bug in setup_mp code

[U-Boot] [PATCH] ppc/85xx: Fix bug in setup_mp code

[Kumar Gala]

Its possible that we try and copy the boot page code out of flash into a
DDR location that doesn't have a TLB cover it.  For example, if we have
3G of DDR we typically only map the first 2G.  In the cases of 4G+ this
wasn't an issue since the reset page TLB mapping covered the last page
of memory which we wanted to copy to.

We now change the physical address of the reset page TLB to map to the
true physical location of the boot page code, copy and than set the
TLB back to its 1:1 mapping of the reset page.

Signed-off-by: Kumar Gala <galak@kernel.crashing.org>
---
 cpu/mpc85xx/mp.c |   32 +++++++++++++++++++++++++++++---
 1 files changed, 29 insertions(+), 3 deletions(-)

diff --git a/cpu/mpc85xx/mp.c b/cpu/mpc85xx/mp.c
index 2df55c7..fa65bed 100644
--- a/cpu/mpc85xx/mp.c
+++ b/cpu/mpc85xx/mp.c
@@ -25,6 +25,7 @@
 #include <ioports.h>
 #include <lmb.h>
 #include <asm/io.h>
+#include <asm/mmu.h>
 #include "mp.h"
 
 DECLARE_GLOBAL_DATA_PTR;
@@ -209,8 +210,33 @@ void setup_mp(void)
 	ulong fixup = (ulong)&__secondary_start_page;
 	u32 bootpg = determine_mp_bootpg();
 
-	memcpy((void *)bootpg, (void *)fixup, 4096);
-	flush_cache(bootpg, 4096);
+	/* look for the tlb covering the reset page, there better be one */
+	int i = find_tlb_idx((void *)0xfffff000, 1);
 
-	pq3_mp_up(bootpg);
+	/* we found a match */
+	if (i != -1) {
+		/* map reset page to bootpg so we can copy code there */
+		disable_tlb(i);
+	
+		set_tlb(1, 0xfffff000, bootpg, /* tlb, epn, rpn */
+			MAS3_SX|MAS3_SW|MAS3_SR, MAS2_M, /* perms, wimge */
+			0, i, BOOKE_PAGESZ_4K, 1); /* ts, esel, tsize, iprot */
+
+		memcpy((void *)0xfffff000, (void *)fixup, 4096);
+		flush_cache(0xfffff000, 4096);
+
+		disable_tlb(i);
+
+		/* setup reset page back to 1:1, we'll use HW boot translation
+		 * to map this where we want
+		 */
+		set_tlb(1, 0xfffff000, 0xfffff000, /* tlb, epn, rpn */
+			MAS3_SX|MAS3_SW|MAS3_SR, MAS2_I, /* perms, wimge */
+			0, i, BOOKE_PAGESZ_4K, 1); /* ts, esel, tsize, iprot */
+
+		pq3_mp_up(bootpg);
+	} else {
+		puts("WARNING: No reset page TLB. "
+			"Skipping secondary core setup\n");
+	}
 }
-- 
1.6.0.6

[U-Boot] [PATCH] ppc/85xx: Fix bug in setup_mp code

[Peter Tyser]

On Thu, 2009-09-03 at 08:58 -0500, Kumar Gala wrote:
> Its possible that we try and copy the boot page code out of flash into a
> DDR location that doesn't have a TLB cover it.  For example, if we have
> 3G of DDR we typically only map the first 2G.  In the cases of 4G+ this
> wasn't an issue since the reset page TLB mapping covered the last page
> of memory which we wanted to copy to.
> 
> We now change the physical address of the reset page TLB to map to the
> true physical location of the boot page code, copy and than set the
> TLB back to its 1:1 mapping of the reset page.
> 
> Signed-off-by: Kumar Gala <galak@kernel.crashing.org>
> ---
>  cpu/mpc85xx/mp.c |   32 +++++++++++++++++++++++++++++---
>  1 files changed, 29 insertions(+), 3 deletions(-)
> 
> diff --git a/cpu/mpc85xx/mp.c b/cpu/mpc85xx/mp.c
> index 2df55c7..fa65bed 100644
> --- a/cpu/mpc85xx/mp.c
> +++ b/cpu/mpc85xx/mp.c
> @@ -25,6 +25,7 @@
>  #include <ioports.h>
>  #include <lmb.h>
>  #include <asm/io.h>
> +#include <asm/mmu.h>
>  #include "mp.h"
>  
>  DECLARE_GLOBAL_DATA_PTR;
> @@ -209,8 +210,33 @@ void setup_mp(void)
>  	ulong fixup = (ulong)&__secondary_start_page;
>  	u32 bootpg = determine_mp_bootpg();
>  
> -	memcpy((void *)bootpg, (void *)fixup, 4096);
> -	flush_cache(bootpg, 4096);
> +	/* look for the tlb covering the reset page, there better be one */
> +	int i = find_tlb_idx((void *)0xfffff000, 1);
>  
> -	pq3_mp_up(bootpg);
> +	/* we found a match */
> +	if (i != -1) {
> +		/* map reset page to bootpg so we can copy code there */
> +		disable_tlb(i);
> +	
> +		set_tlb(1, 0xfffff000, bootpg, /* tlb, epn, rpn */
> +			MAS3_SX|MAS3_SW|MAS3_SR, MAS2_M, /* perms, wimge */
> +			0, i, BOOKE_PAGESZ_4K, 1); /* ts, esel, tsize, iprot */
> +
> +		memcpy((void *)0xfffff000, (void *)fixup, 4096);
> +		flush_cache(0xfffff000, 4096);
> +
> +		disable_tlb(i);
> +
> +		/* setup reset page back to 1:1, we'll use HW boot translation
> +		 * to map this where we want
> +		 */
> +		set_tlb(1, 0xfffff000, 0xfffff000, /* tlb, epn, rpn */
> +			MAS3_SX|MAS3_SW|MAS3_SR, MAS2_I, /* perms, wimge */
> +			0, i, BOOKE_PAGESZ_4K, 1); /* ts, esel, tsize, iprot */

X-ES's mpc8572/8561 boards don't currently use 1 TLB to just map the
0xfffff000 region, we use same TLB that is used for flash (covering
0xf0000000-0xffffffff).

It looks like a fair number of other mpc85xx non-MP boards currently use
the 0xfffff000 region for flash too.  This patch wouldn't affect them,
but I assume when those vendors produce a board with MP support, they'd
like to keep the same memory map, in which case they'd run into the same
issue as I'm having with not reserving the 0xfffffxxx region for BPTR
use.

In any case, I was wondering if you'd be up for changing this patch to
save/restore the TLB you're disabling instead of hardcoding the
"restore".  I could send a follow-up patch if you're OK with the concept
too.

I feel like our use of the 0xfffff000 memory region is starting to make
us the red headed stepchild of the mpc85xx MP family:)

Thanks,
Peter

[U-Boot] [PATCH] ppc/85xx: Fix bug in setup_mp code

[Kumar Gala]

On Sep 3, 2009, at 9:41 AM, Peter Tyser wrote:

> On Thu, 2009-09-03 at 08:58 -0500, Kumar Gala wrote:
>> Its possible that we try and copy the boot page code out of flash  
>> into a
>> DDR location that doesn't have a TLB cover it.  For example, if we  
>> have
>> 3G of DDR we typically only map the first 2G.  In the cases of 4G+  
>> this
>> wasn't an issue since the reset page TLB mapping covered the last  
>> page
>> of memory which we wanted to copy to.
>>
>> We now change the physical address of the reset page TLB to map to  
>> the
>> true physical location of the boot page code, copy and than set the
>> TLB back to its 1:1 mapping of the reset page.
>>
>> Signed-off-by: Kumar Gala <galak@kernel.crashing.org>
>> ---
>> cpu/mpc85xx/mp.c |   32 +++++++++++++++++++++++++++++---
>> 1 files changed, 29 insertions(+), 3 deletions(-)
>>
>> diff --git a/cpu/mpc85xx/mp.c b/cpu/mpc85xx/mp.c
>> index 2df55c7..fa65bed 100644
>> --- a/cpu/mpc85xx/mp.c
>> +++ b/cpu/mpc85xx/mp.c
>> @@ -25,6 +25,7 @@
>> #include <ioports.h>
>> #include <lmb.h>
>> #include <asm/io.h>
>> +#include <asm/mmu.h>
>> #include "mp.h"
>>
>> DECLARE_GLOBAL_DATA_PTR;
>> @@ -209,8 +210,33 @@ void setup_mp(void)
>> 	ulong fixup = (ulong)&__secondary_start_page;
>> 	u32 bootpg = determine_mp_bootpg();
>>
>> -	memcpy((void *)bootpg, (void *)fixup, 4096);
>> -	flush_cache(bootpg, 4096);
>> +	/* look for the tlb covering the reset page, there better be one */
>> +	int i = find_tlb_idx((void *)0xfffff000, 1);
>>
>> -	pq3_mp_up(bootpg);
>> +	/* we found a match */
>> +	if (i != -1) {
>> +		/* map reset page to bootpg so we can copy code there */
>> +		disable_tlb(i);
>> +	
>> +		set_tlb(1, 0xfffff000, bootpg, /* tlb, epn, rpn */
>> +			MAS3_SX|MAS3_SW|MAS3_SR, MAS2_M, /* perms, wimge */
>> +			0, i, BOOKE_PAGESZ_4K, 1); /* ts, esel, tsize, iprot */
>> +
>> +		memcpy((void *)0xfffff000, (void *)fixup, 4096);
>> +		flush_cache(0xfffff000, 4096);
>> +
>> +		disable_tlb(i);
>> +
>> +		/* setup reset page back to 1:1, we'll use HW boot translation
>> +		 * to map this where we want
>> +		 */
>> +		set_tlb(1, 0xfffff000, 0xfffff000, /* tlb, epn, rpn */
>> +			MAS3_SX|MAS3_SW|MAS3_SR, MAS2_I, /* perms, wimge */
>> +			0, i, BOOKE_PAGESZ_4K, 1); /* ts, esel, tsize, iprot */
>
> X-ES's mpc8572/8561 boards don't currently use 1 TLB to just map the
> 0xfffff000 region, we use same TLB that is used for flash (covering
> 0xf0000000-0xffffffff).
>
> It looks like a fair number of other mpc85xx non-MP boards currently  
> use
> the 0xfffff000 region for flash too.  This patch wouldn't affect them,
> but I assume when those vendors produce a board with MP support,  
> they'd
> like to keep the same memory map, in which case they'd run into the  
> same
> issue as I'm having with not reserving the 0xfffffxxx region for BPTR
> use.
>
> In any case, I was wondering if you'd be up for changing this patch to
> save/restore the TLB you're disabling instead of hardcoding the
> "restore".  I could send a follow-up patch if you're OK with the  
> concept
> too.

I'm fine w/the concept.  I'm cleaning up some PCI code at this point  
so feel free to send me a patch to do this.

> I feel like our use of the 0xfffff000 memory region is starting to  
> make
> us the red headed stepchild of the mpc85xx MP family:)

:)

- k

[U-Boot] [PATCH] ppc/85xx: Fix bug in setup_mp code

[Kumar Gala]

On Sep 3, 2009, at 8:58 AM, Kumar Gala wrote:

> Its possible that we try and copy the boot page code out of flash  
> into a
> DDR location that doesn't have a TLB cover it.  For example, if we  
> have
> 3G of DDR we typically only map the first 2G.  In the cases of 4G+  
> this
> wasn't an issue since the reset page TLB mapping covered the last page
> of memory which we wanted to copy to.
>
> We now change the physical address of the reset page TLB to map to the
> true physical location of the boot page code, copy and than set the
> TLB back to its 1:1 mapping of the reset page.
>
> Signed-off-by: Kumar Gala <galak@kernel.crashing.org>
> ---
> cpu/mpc85xx/mp.c |   32 +++++++++++++++++++++++++++++---
> 1 files changed, 29 insertions(+), 3 deletions(-)

applied to 85xx

- k

[U-Boot] [PATCH] ppc/85xx: Fix bug in setup_mp code

[Swarthout Edward L-SWARTHOU]

From: Kumar Gala

> Its possible that we try and copy the boot page code out of flash into

> a DDR location that doesn't have a TLB cover it.  For example, if we 
> have 3G of DDR we typically only map the first 2G.  In the cases of 
> 4G+ this wasn't an issue since the reset page TLB mapping covered the 
> last page of memory which we wanted to copy to.
> 
> We now change the physical address of the reset page TLB to map to the

> true physical location of the boot page code, copy and than set the 
> TLB back to its 1:1 mapping of the reset page.

Why should a boot page TLB still exist?  And if it does exist,
it may be serving a good purpose and shouldn't be touched.

Why not just pick an unused virtual address and map it to bootpg?

This TLB grabbing is giving me problems on my board where I have the
original 1M bootpage mapped to CPC sram and I want to keep active.

-Ed

[U-Boot] [PATCH] ppc/85xx: Fix bug in setup_mp code

[Peter Tyser]

Hi Ed,

On Thu, 2010-01-21 at 15:06 -0700, Swarthout Edward L-SWARTHOU wrote:
> From: Kumar Gala
> 
> > Its possible that we try and copy the boot page code out of flash into
> 
> > a DDR location that doesn't have a TLB cover it.  For example, if we 
> > have 3G of DDR we typically only map the first 2G.  In the cases of 
> > 4G+ this wasn't an issue since the reset page TLB mapping covered the 
> > last page of memory which we wanted to copy to.
> > 
> > We now change the physical address of the reset page TLB to map to the
> 
> > true physical location of the boot page code, copy and than set the 
> > TLB back to its 1:1 mapping of the reset page.
> 
> Why should a boot page TLB still exist?  And if it does exist,
> it may be serving a good purpose and shouldn't be touched.

I imagine Kumar assumed the bootpage was only used for its "intended"
purpose?  Our boards have a large flash mapped on top of the original
bootpage, which caused problems for me too.

> Why not just pick an unused virtual address and map it to bootpg?

You can actually do what you describe since commit
5ccd29c3679b3669b0bde5c501c1aa0f325a7acb.  The CONFIG_BPTR_VIRT_ADDR
feature is enabled for the XPedite5370 in commit
48618126f78f05042dae428811809b594f747eb9.

> This TLB grabbing is giving me problems on my board where I have the
> original 1M bootpage mapped to CPC sram and I want to keep active.

I believe reproducing 48618126f78f05042dae428811809b594f747eb9 for your
board should be a workaround.

Best,
Peter