[PATCH 0/2] arm: fix kexec for ARMv7

[PATCH 0/2] arm: fix kexec for ARMv7

[Saeed Bishara]

Those two patches needed to make kexec work for ARMv7 based cpu's. The second one needed when L2 is used. 
-- 
1.6.0.4

[PATCH 1/2] arm: disable L2 cache in the v7 finish function

[Saeed Bishara]

Signed-off-by: Saeed Bishara <saeed@marvell.com>
---
 arch/arm/mm/proc-v7.S |    5 +++++
 1 files changed, 5 insertions(+), 0 deletions(-)

diff --git a/arch/arm/mm/proc-v7.S b/arch/arm/mm/proc-v7.S
index 7aaf88a..06cc36c 100644
--- a/arch/arm/mm/proc-v7.S
+++ b/arch/arm/mm/proc-v7.S
@@ -52,6 +52,11 @@ ENTRY(cpu_v7_proc_fin)
 	bic	r0, r0, #0x1000			@ ...i............
 	bic	r0, r0, #0x0006			@ .............ca.
 	mcr	p15, 0, r0, c1, c0, 0		@ disable caches
+#ifdef CONFIG_OUTER_CACHE
+	mrc	p15, 0, r0, c1, c0, 1
+	bic	r0, r0, #0x2
+	mcr	p15, 0, r0, c1, c0, 1		@ disable L2 cache
+#endif
 	ldmfd	sp!, {pc}
 ENDPROC(cpu_v7_proc_fin)
 
-- 
1.6.0.4

[PATCH 2/2] arm: invalidate TLBs when enabling mmu

[Saeed Bishara]

Signed-off-by: Saeed Bishara <saeed@marvell.com>
---
 arch/arm/boot/compressed/head.S |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

diff --git a/arch/arm/boot/compressed/head.S b/arch/arm/boot/compressed/head.S
index 4fddc50..a1ab79f 100644
--- a/arch/arm/boot/compressed/head.S
+++ b/arch/arm/boot/compressed/head.S
@@ -489,6 +489,7 @@ __armv7_mmu_cache_on:
 		mcr	p15, 0, r0, c1, c0, 0	@ load control register
 		mrc	p15, 0, r0, c1, c0, 0	@ and read it back
 		mov	r0, #0
+		mcr	p15, 0, r0, c8, c7, 0	@ invalidate I,D TLBs
 		mcr	p15, 0, r0, c7, c5, 4	@ ISB
 		mov	pc, r12
 
-- 
1.6.0.4

[PATCH 1/2] arm: disable L2 cache in the v7 finish function

[Catalin Marinas]

On Tue, 2010-03-09 at 14:07 +0000, Saeed Bishara wrote:
> Signed-off-by: Saeed Bishara <saeed@marvell.com>
> ---
>  arch/arm/mm/proc-v7.S |    5 +++++
>  1 files changed, 5 insertions(+), 0 deletions(-)
> 
> diff --git a/arch/arm/mm/proc-v7.S b/arch/arm/mm/proc-v7.S
> index 7aaf88a..06cc36c 100644
> --- a/arch/arm/mm/proc-v7.S
> +++ b/arch/arm/mm/proc-v7.S
> @@ -52,6 +52,11 @@ ENTRY(cpu_v7_proc_fin)
>         bic     r0, r0, #0x1000                 @ ...i............
>         bic     r0, r0, #0x0006                 @ .............ca.
>         mcr     p15, 0, r0, c1, c0, 0           @ disable caches
> +#ifdef CONFIG_OUTER_CACHE
> +       mrc     p15, 0, r0, c1, c0, 1
> +       bic     r0, r0, #0x2
> +       mcr     p15, 0, r0, c1, c0, 1           @ disable L2 cache
> +#endif
>         ldmfd   sp!, {pc}
>  ENDPROC(cpu_v7_proc_fin)

NACK.

I'm not sure why kexec doesn't work but bit 1 in this register has
different meanings on Cortex-A8 and A9.

Also, on Cortex-A8, it means L2EN but this refers to the inner L2 rather
than the outer cache (that's configurable via the L2 Auxiliary Cache
Control Register but the Linux meaning of outer cache is a separate
device outside the CPU acting as a cache controller).

-- 
Catalin

[PATCH 2/2] arm: invalidate TLBs when enabling mmu

[Catalin Marinas]

On Tue, 2010-03-09 at 14:07 +0000, Saeed Bishara wrote:
> Signed-off-by: Saeed Bishara <saeed@marvell.com>
> ---
>  arch/arm/boot/compressed/head.S |    1 +
>  1 files changed, 1 insertions(+), 0 deletions(-)
> 
> diff --git a/arch/arm/boot/compressed/head.S
> b/arch/arm/boot/compressed/head.S
> index 4fddc50..a1ab79f 100644
> --- a/arch/arm/boot/compressed/head.S
> +++ b/arch/arm/boot/compressed/head.S
> @@ -489,6 +489,7 @@ __armv7_mmu_cache_on:
>                 mcr     p15, 0, r0, c1, c0, 0   @ load control
> register
>                 mrc     p15, 0, r0, c1, c0, 0   @ and read it back
>                 mov     r0, #0
> +               mcr     p15, 0, r0, c8, c7, 0   @ invalidate I,D TLBs
>                 mcr     p15, 0, r0, c7, c5, 4   @ ISB
>                 mov     pc, r12

The TLB invalidating is done earlier in the __armv7_mmu_cache_on
function, why do you need to do it again?

-- 
Catalin

[PATCH 1/2] arm: disable L2 cache in the v7 finish function

[Tony Lindgren]

* Saeed Bishara <saeed@marvell.com> [100309 06:13]:
> Signed-off-by: Saeed Bishara <saeed@marvell.com>
> ---
>  arch/arm/mm/proc-v7.S |    5 +++++
>  1 files changed, 5 insertions(+), 0 deletions(-)
> 
> diff --git a/arch/arm/mm/proc-v7.S b/arch/arm/mm/proc-v7.S
> index 7aaf88a..06cc36c 100644
> --- a/arch/arm/mm/proc-v7.S
> +++ b/arch/arm/mm/proc-v7.S
> @@ -52,6 +52,11 @@ ENTRY(cpu_v7_proc_fin)
>  	bic	r0, r0, #0x1000			@ ...i............
>  	bic	r0, r0, #0x0006			@ .............ca.
>  	mcr	p15, 0, r0, c1, c0, 0		@ disable caches
> +#ifdef CONFIG_OUTER_CACHE
> +	mrc	p15, 0, r0, c1, c0, 1
> +	bic	r0, r0, #0x2
> +	mcr	p15, 0, r0, c1, c0, 1		@ disable L2 cache
> +#endif
>  	ldmfd	sp!, {pc}
>  ENDPROC(cpu_v7_proc_fin)

This one is a bit tricky. I know of three different ways
to disable the L2 cache depending on the hardware. One of
them is what you have above. The second is via SMI, and
then the first one won't work. The third is via external
secure monitor calls used on some omaps at least. And in
the third case the first two methods won't work.

AFAIK disabling the c bit should also disable the outer
cache. Anybody know if there's something else to it?

I'm currently thinking we should call outer_clean_range
before hitting cpu_v6_proc_fin, then disable cache.

Regards,

Tony

[PATCH 1/2] arm: disable L2 cache in the v7 finish function

[Tony Lindgren]

* Catalin Marinas <catalin.marinas@arm.com> [100309 09:05]:
> On Tue, 2010-03-09 at 14:07 +0000, Saeed Bishara wrote:
> > Signed-off-by: Saeed Bishara <saeed@marvell.com>
> > ---
> >  arch/arm/mm/proc-v7.S |    5 +++++
> >  1 files changed, 5 insertions(+), 0 deletions(-)
> > 
> > diff --git a/arch/arm/mm/proc-v7.S b/arch/arm/mm/proc-v7.S
> > index 7aaf88a..06cc36c 100644
> > --- a/arch/arm/mm/proc-v7.S
> > +++ b/arch/arm/mm/proc-v7.S
> > @@ -52,6 +52,11 @@ ENTRY(cpu_v7_proc_fin)
> >         bic     r0, r0, #0x1000                 @ ...i............
> >         bic     r0, r0, #0x0006                 @ .............ca.
> >         mcr     p15, 0, r0, c1, c0, 0           @ disable caches
> > +#ifdef CONFIG_OUTER_CACHE
> > +       mrc     p15, 0, r0, c1, c0, 1
> > +       bic     r0, r0, #0x2
> > +       mcr     p15, 0, r0, c1, c0, 1           @ disable L2 cache
> > +#endif
> >         ldmfd   sp!, {pc}
> >  ENDPROC(cpu_v7_proc_fin)
> 
> NACK.
> 
> I'm not sure why kexec doesn't work but bit 1 in this register has
> different meanings on Cortex-A8 and A9.
> 
> Also, on Cortex-A8, it means L2EN but this refers to the inner L2 rather
> than the outer cache (that's configurable via the L2 Auxiliary Cache
> Control Register but the Linux meaning of outer cache is a separate
> device outside the CPU acting as a cache controller).

Plus there are two other ways to disable the inner L2 and then
L2EN won't work like I replied earlier.

Regards,

Tony

[PATCH 1/2] arm: disable L2 cache in the v7 finish function

[Woodruff, Richard]

> From: linux-arm-kernel-bounces at lists.infradead.org [mailto:linux-arm-kernel-
> bounces at lists.infradead.org] On Behalf Of Tony Lindgren
> Sent: Wednesday, March 10, 2010 3:53 PM

> > +#ifdef CONFIG_OUTER_CACHE
> > +   mrc     p15, 0, r0, c1, c0, 1
> > +   bic     r0, r0, #0x2
> > +   mcr     p15, 0, r0, c1, c0, 1           @ disable L2 cache
> > +#endif
> >     ldmfd   sp!, {pc}
> >  ENDPROC(cpu_v7_proc_fin)
>
> This one is a bit tricky. I know of three different ways
> to disable the L2 cache depending on the hardware. One of
> them is what you have above. The second is via SMI, and
> then the first one won't work. The third is via external
> secure monitor calls used on some omaps at least. And in
> the third case the first two methods won't work.

On A8 cores with integrated L2 cache the L2EN bit is banked after r0px series (you won't find r0 systems anywhere). As such the above code is good on OMAP3 and many other A8's.  Other bits in this same register have issues which you warn of.  But not the L2EN.

On A9 which uses outer cache the above won't work.  The external PL310 (or variant) is memory mapped.  To handle a different sequence is needed all together.

Regards,
Richard W.

[PATCH 1/2] arm: disable L2 cache in the v7 finish function

[Tony Lindgren]

* Woodruff, Richard <r-woodruff2@ti.com> [100319 12:51]:
> 
> > From: linux-arm-kernel-bounces at lists.infradead.org [mailto:linux-arm-kernel-
> > bounces at lists.infradead.org] On Behalf Of Tony Lindgren
> > Sent: Wednesday, March 10, 2010 3:53 PM
> 
> > > +#ifdef CONFIG_OUTER_CACHE
> > > +   mrc     p15, 0, r0, c1, c0, 1
> > > +   bic     r0, r0, #0x2
> > > +   mcr     p15, 0, r0, c1, c0, 1           @ disable L2 cache
> > > +#endif
> > >     ldmfd   sp!, {pc}
> > >  ENDPROC(cpu_v7_proc_fin)
> >
> > This one is a bit tricky. I know of three different ways
> > to disable the L2 cache depending on the hardware. One of
> > them is what you have above. The second is via SMI, and
> > then the first one won't work. The third is via external
> > secure monitor calls used on some omaps at least. And in
> > the third case the first two methods won't work.
> 
> On A8 cores with integrated L2 cache the L2EN bit is banked after r0px series (you won't find r0 systems anywhere). As such the above code is good on OMAP3 and many other A8's.  Other bits in this same register have issues which you warn of.  But not the L2EN.
> 
> On A9 which uses outer cache the above won't work.  The external PL310 (or variant) is memory mapped.  To handle a different sequence is needed all together.

Sounds like we need something like:

ENTRY(cpu_v7_proc_fin)
...
	mcr	p15, 0, r0, c1, c0, 0	@ disable caches
	b	cpu_v7_arch_proc_fin	@ disable L2 cache
	ldmfd	sp!, {pc}
ENDPROC(cpu_v7_proc_fin)

And then cpu_v7_arch_proc_fin can be implemented for the platforms
that need special handling.

Regards,

Tony

[PATCH 1/2] arm: disable L2 cache in the v7 finish function

[Eric Miao]

On Tue, Mar 23, 2010 at 5:00 AM, Tony Lindgren <tony@atomide.com> wrote:
> * Woodruff, Richard <r-woodruff2@ti.com> [100319 12:51]:
>>
>> > From: linux-arm-kernel-bounces at lists.infradead.org [mailto:linux-arm-kernel-
>> > bounces at lists.infradead.org] On Behalf Of Tony Lindgren
>> > Sent: Wednesday, March 10, 2010 3:53 PM
>>
>> > > +#ifdef CONFIG_OUTER_CACHE
>> > > + ? mrc ? ? p15, 0, r0, c1, c0, 1
>> > > + ? bic ? ? r0, r0, #0x2
>> > > + ? mcr ? ? p15, 0, r0, c1, c0, 1 ? ? ? ? ? @ disable L2 cache
>> > > +#endif
>> > > ? ? ldmfd ? sp!, {pc}
>> > > ?ENDPROC(cpu_v7_proc_fin)
>> >
>> > This one is a bit tricky. I know of three different ways
>> > to disable the L2 cache depending on the hardware. One of
>> > them is what you have above. The second is via SMI, and
>> > then the first one won't work. The third is via external
>> > secure monitor calls used on some omaps at least. And in
>> > the third case the first two methods won't work.
>>
>> On A8 cores with integrated L2 cache the L2EN bit is banked after r0px series (you won't find r0 systems anywhere). As such the above code is good on OMAP3 and many other A8's. ?Other bits in this same register have issues which you warn of. ?But not the L2EN.
>>
>> On A9 which uses outer cache the above won't work. ?The external PL310 (or variant) is memory mapped. ?To handle a different sequence is needed all together.
>
> Sounds like we need something like:
>
> ENTRY(cpu_v7_proc_fin)
> ...
> ? ? ? ?mcr ? ? p15, 0, r0, c1, c0, 0 ? @ disable caches
> ? ? ? ?b ? ? ? cpu_v7_arch_proc_fin ? ?@ disable L2 cache
> ? ? ? ?ldmfd ? sp!, {pc}
> ENDPROC(cpu_v7_proc_fin)
>
> And then cpu_v7_arch_proc_fin can be implemented for the platforms
> that need special handling.
>

Yeah, sort of I agree.

For outer cache, I also doubt that we might need to flush the L2 before
disabling, though this may depend on the exact behavior of the specific
cache controller. Introduce something like outer_cache_fns.finish() may
be something feasible.


> Regards,
>
> Tony
>
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
>

[PATCH 2/2] arm: invalidate TLBs when enabling mmu

[Eric Miao]

On Wed, Mar 10, 2010 at 12:45 AM, Catalin Marinas
<catalin.marinas@arm.com> wrote:
> On Tue, 2010-03-09 at 14:07 +0000, Saeed Bishara wrote:
>> Signed-off-by: Saeed Bishara <saeed@marvell.com>
>> ---
>> ?arch/arm/boot/compressed/head.S | ? ?1 +
>> ?1 files changed, 1 insertions(+), 0 deletions(-)
>>
>> diff --git a/arch/arm/boot/compressed/head.S
>> b/arch/arm/boot/compressed/head.S
>> index 4fddc50..a1ab79f 100644
>> --- a/arch/arm/boot/compressed/head.S
>> +++ b/arch/arm/boot/compressed/head.S
>> @@ -489,6 +489,7 @@ __armv7_mmu_cache_on:
>> ? ? ? ? ? ? ? ? mcr ? ? p15, 0, r0, c1, c0, 0 ? @ load control
>> register
>> ? ? ? ? ? ? ? ? mrc ? ? p15, 0, r0, c1, c0, 0 ? @ and read it back
>> ? ? ? ? ? ? ? ? mov ? ? r0, #0
>> + ? ? ? ? ? ? ? mcr ? ? p15, 0, r0, c8, c7, 0 ? @ invalidate I,D TLBs
>> ? ? ? ? ? ? ? ? mcr ? ? p15, 0, r0, c7, c5, 4 ? @ ISB
>> ? ? ? ? ? ? ? ? mov ? ? pc, r12
>
> The TLB invalidating is done earlier in the __armv7_mmu_cache_on
> function, why do you need to do it again?
>

Well, the only difference between these two "invalidate"s looks like one is
before control register load and one after.

We do have the problem of slow decompressing when this invalidate
change is not there. I do suspect this might be Dove specific. But yet,
Catalin, could you confirm the behavior difference of "invalidate" before
and after MMU is actaully turned on?

[PATCH 2/2] arm: invalidate TLBs when enabling mmu

[Bryan Wu]

On 04/14/2010 10:49 AM, Eric Miao wrote:
> On Wed, Mar 10, 2010 at 12:45 AM, Catalin Marinas
> <catalin.marinas@arm.com>  wrote:
>> On Tue, 2010-03-09 at 14:07 +0000, Saeed Bishara wrote:
>>> Signed-off-by: Saeed Bishara<saeed@marvell.com>
>>> ---
>>>   arch/arm/boot/compressed/head.S |    1 +
>>>   1 files changed, 1 insertions(+), 0 deletions(-)
>>>
>>> diff --git a/arch/arm/boot/compressed/head.S
>>> b/arch/arm/boot/compressed/head.S
>>> index 4fddc50..a1ab79f 100644
>>> --- a/arch/arm/boot/compressed/head.S
>>> +++ b/arch/arm/boot/compressed/head.S
>>> @@ -489,6 +489,7 @@ __armv7_mmu_cache_on:
>>>                  mcr     p15, 0, r0, c1, c0, 0   @ load control
>>> register
>>>                  mrc     p15, 0, r0, c1, c0, 0   @ and read it back
>>>                  mov     r0, #0
>>> +               mcr     p15, 0, r0, c8, c7, 0   @ invalidate I,D TLBs
>>>                  mcr     p15, 0, r0, c7, c5, 4   @ ISB
>>>                  mov     pc, r12
>>
>> The TLB invalidating is done earlier in the __armv7_mmu_cache_on
>> function, why do you need to do it again?
>>
>
> Well, the only difference between these two "invalidate"s looks like one is
> before control register load and one after.
>
> We do have the problem of slow decompressing when this invalidate
> change is not there. I do suspect this might be Dove specific. But yet,
> Catalin, could you confirm the behavior difference of "invalidate" before
> and after MMU is actaully turned on?
>

I also met this slow decompressing issue without this patch when I was trying 
kexec on Freescale imx51 babbage board.

Sascha, do you have any comments on this?

Thanks,
-- 
Bryan Wu <bryan.wu@canonical.com>
Kernel Developer    +86.138-1617-6545 Mobile
Ubuntu Kernel Team | Hardware Enablement Team
Canonical Ltd.      www.canonical.com
Ubuntu - Linux for human beings | www.ubuntu.com

[PATCH 2/2] arm: invalidate TLBs when enabling mmu

[Russell King - ARM Linux]

On Tue, Mar 09, 2010 at 04:07:03PM +0200, Saeed Bishara wrote:
> Signed-off-by: Saeed Bishara <saeed@marvell.com>
> ---
>  arch/arm/boot/compressed/head.S |    1 +
>  1 files changed, 1 insertions(+), 0 deletions(-)
> 
> diff --git a/arch/arm/boot/compressed/head.S b/arch/arm/boot/compressed/head.S
> index 4fddc50..a1ab79f 100644
> --- a/arch/arm/boot/compressed/head.S
> +++ b/arch/arm/boot/compressed/head.S
> @@ -489,6 +489,7 @@ __armv7_mmu_cache_on:
>  		mcr	p15, 0, r0, c1, c0, 0	@ load control register
>  		mrc	p15, 0, r0, c1, c0, 0	@ and read it back
>  		mov	r0, #0
> +		mcr	p15, 0, r0, c8, c7, 0	@ invalidate I,D TLBs
>  		mcr	p15, 0, r0, c7, c5, 4	@ ISB
>  		mov	pc, r12
>  

This can't be unconditional - if we're running on PMSA (iow, uclinux)
we should not execute this instruction.  Notice that the previous one
is conditional.

The other question is whether this should be done before or after the
ISB - if it's done before, my understanding is that it could occur
unordered with respect to the MMU being enabled - if that's indeed
the problem.

[PATCH 2/2] arm: invalidate TLBs when enabling mmu

[Eric Miao]

On Thu, Apr 15, 2010 at 2:27 AM, Russell King - ARM Linux
<linux@arm.linux.org.uk> wrote:
> On Tue, Mar 09, 2010 at 04:07:03PM +0200, Saeed Bishara wrote:
>> Signed-off-by: Saeed Bishara <saeed@marvell.com>
>> ---
>>  arch/arm/boot/compressed/head.S |    1 +
>>  1 files changed, 1 insertions(+), 0 deletions(-)
>>
>> diff --git a/arch/arm/boot/compressed/head.S b/arch/arm/boot/compressed/head.S
>> index 4fddc50..a1ab79f 100644
>> --- a/arch/arm/boot/compressed/head.S
>> +++ b/arch/arm/boot/compressed/head.S
>> @@ -489,6 +489,7 @@ __armv7_mmu_cache_on:
>>               mcr     p15, 0, r0, c1, c0, 0   @ load control register
>>               mrc     p15, 0, r0, c1, c0, 0   @ and read it back
>>               mov     r0, #0
>> +             mcr     p15, 0, r0, c8, c7, 0   @ invalidate I,D TLBs
>>               mcr     p15, 0, r0, c7, c5, 4   @ ISB
>>               mov     pc, r12
>>
>
> This can't be unconditional - if we're running on PMSA (iow, uclinux)
> we should not execute this instruction.  Notice that the previous one
> is conditional.
>

This is true and needs to be fixed.

> The other question is whether this should be done before or after the
> ISB - if it's done before, my understanding is that it could occur
> unordered with respect to the MMU being enabled - if that's indeed
> the problem.
>

Another noticeable difference is the slow decompressing happens so far on
kexec only, a normal boot without this additional "invalidate" does not
exhibit such slowness. This leads to me to suspect that the MMU state might
not be same between a normal bootup and a kexec soft reboot.

I may overlooked, but it looks to me that after a 1:1 mapping of ther user
space area (from 0x0000_0000 to TASK_SIZE, which I guess will cause some
other issue if the kexec kernel start entry starts beyond TASK_SIZE?), the
MMU is NOT actually turned off through the reset. This might cause weird
behavior with the original sequence of __armv7_mmu_cache_on ??

[PATCH 2/2] arm: invalidate TLBs when enabling mmu

[Eric Miao]

On Thu, Apr 15, 2010 at 8:24 PM, Eric Miao <eric.y.miao@gmail.com> wrote:
> On Thu, Apr 15, 2010 at 2:27 AM, Russell King - ARM Linux
> <linux@arm.linux.org.uk> wrote:
>> On Tue, Mar 09, 2010 at 04:07:03PM +0200, Saeed Bishara wrote:
>>> Signed-off-by: Saeed Bishara <saeed@marvell.com>
>>> ---
>>> ?arch/arm/boot/compressed/head.S | ? ?1 +
>>> ?1 files changed, 1 insertions(+), 0 deletions(-)
>>>
>>> diff --git a/arch/arm/boot/compressed/head.S b/arch/arm/boot/compressed/head.S
>>> index 4fddc50..a1ab79f 100644
>>> --- a/arch/arm/boot/compressed/head.S
>>> +++ b/arch/arm/boot/compressed/head.S
>>> @@ -489,6 +489,7 @@ __armv7_mmu_cache_on:
>>> ? ? ? ? ? ? ? mcr ? ? p15, 0, r0, c1, c0, 0 ? @ load control register
>>> ? ? ? ? ? ? ? mrc ? ? p15, 0, r0, c1, c0, 0 ? @ and read it back
>>> ? ? ? ? ? ? ? mov ? ? r0, #0
>>> + ? ? ? ? ? ? mcr ? ? p15, 0, r0, c8, c7, 0 ? @ invalidate I,D TLBs
>>> ? ? ? ? ? ? ? mcr ? ? p15, 0, r0, c7, c5, 4 ? @ ISB
>>> ? ? ? ? ? ? ? mov ? ? pc, r12
>>>
>>
>> This can't be unconditional - if we're running on PMSA (iow, uclinux)
>> we should not execute this instruction. ?Notice that the previous one
>> is conditional.
>>
>
> This is true and needs to be fixed.
>
>> The other question is whether this should be done before or after the
>> ISB - if it's done before, my understanding is that it could occur
>> unordered with respect to the MMU being enabled - if that's indeed
>> the problem.
>>
>
> Another noticeable difference is the slow decompressing happens so far on
> kexec only, a normal boot without this additional "invalidate" does not
> exhibit such slowness. This leads to me to suspect that the MMU state might
> not be same between a normal bootup and a kexec soft reboot.
>

BTW: this happens on imx51 as well, so I suspect it's not Marvell dove
specific.

> I may overlooked, but it looks to me that after a 1:1 mapping of ther user
> space area (from 0x0000_0000 to TASK_SIZE, which I guess will cause some
> other issue if the kexec kernel start entry starts beyond TASK_SIZE?), the
> MMU is NOT actually turned off through the reset. This might cause weird
> behavior with the original sequence of __armv7_mmu_cache_on ??
>

[PATCH 2/2] arm: invalidate TLBs when enabling mmu

[Russell King - ARM Linux]

On Thu, Apr 15, 2010 at 08:24:02PM +0800, Eric Miao wrote:
> Another noticeable difference is the slow decompressing happens so far on
> kexec only, a normal boot without this additional "invalidate" does not
> exhibit such slowness. This leads to me to suspect that the MMU state might
> not be same between a normal bootup and a kexec soft reboot.
> 
> I may overlooked, but it looks to me that after a 1:1 mapping of ther user
> space area (from 0x0000_0000 to TASK_SIZE, which I guess will cause some
> other issue if the kexec kernel start entry starts beyond TASK_SIZE?), the
> MMU is NOT actually turned off through the reset. This might cause weird
> behavior with the original sequence of __armv7_mmu_cache_on ??

For kexec:

        cpu_proc_fin();
        setup_mm_for_reboot(0); /* mode is not used, so just pass 0*/
        cpu_reset(reboot_code_buffer_phys);

cpu_proc_fin() disables interrupts, flushes the cache and then disables
caches.

setup_mm_for_reboot() creates 1:1 mappings for the entire user address
space, which ends up being strongly ordered.

cpu_reset() is supposed to invalidate I/D caches, invalidate TLBs, and
disable the MMU prior to calling the passed address.  However, neither
ARMv6 nor ARMv7 does this - and I'm confused as to why not.