Kexec on arm64

[mark.rutland@arm.com (Mark Rutland)]

On Fri, Jul 25, 2014 at 12:48:04PM +0100, Arun Chandran wrote:
> Hi Geoff,
> 
> On Fri, Jul 25, 2014 at 5:47 AM, Geoff Levand <geoff@infradead.org> wrote:
> > Hi,
> >
> > On Thu, 2014-07-24 at 10:36 +0100, Mark Rutland wrote:
> >> On Thu, Jul 24, 2014 at 01:38:07AM +0100, Geoff Levand wrote:
> >
> >> > All memory management for the main cpu is done by the arch code.  Kexec
> >> > and cpu hot plug only work with the secondary cpus, so the problem would
> >> > be in the arch memory code, either in setup_restart() for shutdown, or
> >> > in the startup code.
> >>
> >> It's possible that soft_restart and setup_restart are a little dodgy, as
> >> they also rely on the compiler being smart and not touching the stack
> >> after setup_restart().
> >>
> >> However, they provide absolutely no guarantee that any data has been
> >> flushed out to the PoC [1]. If you require any data to be flushed out to the
> >> PoC so as to be visible to noncacheable accesses, you will need to
> >> ensure that this is flushed by VA before soft_restart is called. Data
> >> may have migrated to another cache (e.g. another CPU, or the L3) where
> >> it is not visible.
> >
> > OK, kexec's reset routine relocate_new_kernel does use a few global
> > variables that are set by the main kexec routines.  I added a call to
> > __flush_dcache_area(), which uses 'dc civac' for those.
> >
> > I had thought the call to __flush_dcache_all, which uses 'dc cisw', in
> > flush_cache_all() would be enough.
> >
> > Arun, I also fixed UP builds.  Could you pull my latest and try with L3
> > enabled?
> >
> I got this working. As 'Mark Rutland' pointed in another mail that
> it could be problem with flushing the cache; I did a read of
> 1GB data from start of RAM to a volatile var. I assume that
> this will clear and invalidate all that in cache (L1=32K, L2=256 K, L3=8M)

You've managed to get the cache to evict some lines, which proves my
theory, but this is absolute nonsense and guarantees nothing.

So NAK to this.

If you need to perform cache maintenance to guarantee data is visible to
non-cacheable accesses  you _must_ use the architected mechanism for
cleaning data to the PoC: DC CVAC. We have wrappers for flushing ranges.

Anything else is nonsense and does not provide the guarantee you need.

That said, I still am not sure what guarantee you are attempting to get.
Which data do you need out at the PoC?

Thanks,
Mark.

> 
> ###################
> diff --git a/arch/arm64/kernel/process.c b/arch/arm64/kernel/process.c
> index 786daa6..90418f3 100644
> --- a/arch/arm64/kernel/process.c
> +++ b/arch/arm64/kernel/process.c
> @@ -63,6 +63,10 @@ static inline void smp_secondary_shutdown(void) {}
> 
>  static void setup_restart(void)
>  {
> +       volatile u64 tmp;
> +       volatile u64 *addr;
> +
> +       addr = (u64 *)(0xffffffc000000000);
>         /*
>          * Tell the mm system that we are going to reboot -
>          * we may need it to insert some 1:1 mappings so that
> @@ -75,6 +79,11 @@ static void setup_restart(void)
>         /* Clean and invalidate caches */
>         flush_cache_all();
> 
> +       for ( ;addr < (u64 *)0xffffffc040000000; addr++)
> +       {
> +               tmp = *addr;
> +       }
> +
>         /* Turn D-cache off */
>         cpu_cache_off();
> 
> ###################
> 
> With the above change latest kernel @
> https://git.linaro.org/people/geoff.levand/linux-kexec.git
> is able to do kexec with L3 enabled in UP scenario.
> 
> --Arun
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> > -Geoff
> >
>