[U-Boot] ARM64: How to protect spin-table code from U-Boot?

[U-Boot] ARM64: How to protect spin-table code from U-Boot?

[Masahiro Yamada]

Hi.

I assume the following code in
arch/arm/cpu/armv8/start.S is for spin-table.



#ifdef CONFIG_ARMV8_MULTIENTRY
       branch_if_master x0, x1, master_cpu

       /*
        * Slave CPUs
        */
slave_cpu:
       wfe
       ldr x1, =CPU_RELEASE_ADDR
       ldr x0, [x1]
       cbz x0, slave_cpu
       br x0 /* branch to the given address */
master_cpu:
       /* On the master CPU */
#endif /* CONFIG_ARMV8_MULTIENTRY */



As Documentation/arm64/booting.txt of Linux says,
slave CPUs should spin outside of the kernel in a
reserved area of memory.

U-Boot generally works on DRAM, so the code for spin-table
should be reserved in Device Tree.

Otherwise, the code above and the variable "CPU_RELEASE_ADDR"
has been destroyed by the kernel by the time slave CPUs are kicked.

Now, I locally work-around this problem by pre-fetching necessary code
to the I-cache, but this solution is unstable.


My question is, is there a solution to protect spin-table code already?
(or on-going work to solve the problem?)

One problem specific for U-Boot is that,
U-Boot relocates itself to the tail of DRAM.
So, it is difficult to reserve the code statically at the compile time of DT.


Thought?


-- 
Best Regards
Masahiro Yamada

[U-Boot] ARM64: How to protect spin-table code from U-Boot?

[Alexander Graf]

On 07.05.16 09:12, Masahiro Yamada wrote:
> Hi.
> 
> I assume the following code in
> arch/arm/cpu/armv8/start.S is for spin-table.
> 
> 
> 
> #ifdef CONFIG_ARMV8_MULTIENTRY
>        branch_if_master x0, x1, master_cpu
> 
>        /*
>         * Slave CPUs
>         */
> slave_cpu:
>        wfe
>        ldr x1, =CPU_RELEASE_ADDR
>        ldr x0, [x1]
>        cbz x0, slave_cpu
>        br x0 /* branch to the given address */
> master_cpu:
>        /* On the master CPU */
> #endif /* CONFIG_ARMV8_MULTIENTRY */
> 
> 
> 
> As Documentation/arm64/booting.txt of Linux says,
> slave CPUs should spin outside of the kernel in a
> reserved area of memory.
> 
> U-Boot generally works on DRAM, so the code for spin-table
> should be reserved in Device Tree.
> 
> Otherwise, the code above and the variable "CPU_RELEASE_ADDR"
> has been destroyed by the kernel by the time slave CPUs are kicked.
> 
> Now, I locally work-around this problem by pre-fetching necessary code
> to the I-cache, but this solution is unstable.
> 
> 
> My question is, is there a solution to protect spin-table code already?
> (or on-going work to solve the problem?)
> 
> One problem specific for U-Boot is that,
> U-Boot relocates itself to the tail of DRAM.
> So, it is difficult to reserve the code statically at the compile time of DT.

I assume your SoC has working EL3? If so, why don't you just provide the
respective PSCI cpu wakeup calls via ATF instead of using spin tables?


Alex

[U-Boot] ARM64: How to protect spin-table code from U-Boot?

[Masahiro Yamada]

Hi Alex,


2016-05-07 20:30 GMT+09:00 Alexander Graf <agraf@suse.de>:
>
>
> On 07.05.16 09:12, Masahiro Yamada wrote:
>> Hi.
>>
>> I assume the following code in
>> arch/arm/cpu/armv8/start.S is for spin-table.
>>
>>
>>
>> #ifdef CONFIG_ARMV8_MULTIENTRY
>>        branch_if_master x0, x1, master_cpu
>>
>>        /*
>>         * Slave CPUs
>>         */
>> slave_cpu:
>>        wfe
>>        ldr x1, =CPU_RELEASE_ADDR
>>        ldr x0, [x1]
>>        cbz x0, slave_cpu
>>        br x0 /* branch to the given address */
>> master_cpu:
>>        /* On the master CPU */
>> #endif /* CONFIG_ARMV8_MULTIENTRY */
>>
>>
>>
>> As Documentation/arm64/booting.txt of Linux says,
>> slave CPUs should spin outside of the kernel in a
>> reserved area of memory.
>>
>> U-Boot generally works on DRAM, so the code for spin-table
>> should be reserved in Device Tree.
>>
>> Otherwise, the code above and the variable "CPU_RELEASE_ADDR"
>> has been destroyed by the kernel by the time slave CPUs are kicked.
>>
>> Now, I locally work-around this problem by pre-fetching necessary code
>> to the I-cache, but this solution is unstable.
>>
>>
>> My question is, is there a solution to protect spin-table code already?
>> (or on-going work to solve the problem?)
>>
>> One problem specific for U-Boot is that,
>> U-Boot relocates itself to the tail of DRAM.
>> So, it is difficult to reserve the code statically at the compile time of DT.
>
> I assume your SoC has working EL3? If so, why don't you just provide the
> respective PSCI cpu wakeup calls via ATF instead of using spin tables?
>


I am planning to switch to ARM Trusted Firmware in the future,
but there are several things to study before staring to use it.
(and I guess there are SoC-specific parts that should be implemented in ATF)

I needed to bring-up my first ARMv8 SoC quickly.
I am familiar with U-Boot already, so I chose to use U-Boot alone
in my early development phase.

A good thing about spin-table is that it is really simple.

Moreover, if we have something, it should be correct.
(or should be deleted if it is not working.)
I do not like the half-way house like "we implemented it, but not working".

If nobody has taken care about it yet, I am happy to work on it.
Any comment is very appreciated, of course.


-- 
Best Regards
Masahiro Yamada

[U-Boot] ARM64: How to protect spin-table code from U-Boot?

[Alexander Graf]

> Am 09.05.2016 um 00:57 schrieb Masahiro Yamada <yamada.masahiro@socionext.com>:
> 
> Hi Alex,
> 
> 
> 2016-05-07 20:30 GMT+09:00 Alexander Graf <agraf@suse.de>:
>> 
>> 
>>> On 07.05.16 09:12, Masahiro Yamada wrote:
>>> Hi.
>>> 
>>> I assume the following code in
>>> arch/arm/cpu/armv8/start.S is for spin-table.
>>> 
>>> 
>>> 
>>> #ifdef CONFIG_ARMV8_MULTIENTRY
>>>       branch_if_master x0, x1, master_cpu
>>> 
>>>       /*
>>>        * Slave CPUs
>>>        */
>>> slave_cpu:
>>>       wfe
>>>       ldr x1, =CPU_RELEASE_ADDR
>>>       ldr x0, [x1]
>>>       cbz x0, slave_cpu
>>>       br x0 /* branch to the given address */
>>> master_cpu:
>>>       /* On the master CPU */
>>> #endif /* CONFIG_ARMV8_MULTIENTRY */
>>> 
>>> 
>>> 
>>> As Documentation/arm64/booting.txt of Linux says,
>>> slave CPUs should spin outside of the kernel in a
>>> reserved area of memory.
>>> 
>>> U-Boot generally works on DRAM, so the code for spin-table
>>> should be reserved in Device Tree.
>>> 
>>> Otherwise, the code above and the variable "CPU_RELEASE_ADDR"
>>> has been destroyed by the kernel by the time slave CPUs are kicked.
>>> 
>>> Now, I locally work-around this problem by pre-fetching necessary code
>>> to the I-cache, but this solution is unstable.
>>> 
>>> 
>>> My question is, is there a solution to protect spin-table code already?
>>> (or on-going work to solve the problem?)
>>> 
>>> One problem specific for U-Boot is that,
>>> U-Boot relocates itself to the tail of DRAM.
>>> So, it is difficult to reserve the code statically at the compile time of DT.
>> 
>> I assume your SoC has working EL3? If so, why don't you just provide the
>> respective PSCI cpu wakeup calls via ATF instead of using spin tables?
> 
> 
> I am planning to switch to ARM Trusted Firmware in the future,
> but there are several things to study before staring to use it.
> (and I guess there are SoC-specific parts that should be implemented in ATF)
> 
> I needed to bring-up my first ARMv8 SoC quickly.
> I am familiar with U-Boot already, so I chose to use U-Boot alone
> in my early development phase.
> 
> A good thing about spin-table is that it is really simple.
> 
> Moreover, if we have something, it should be correct.
> (or should be deleted if it is not working.)
> I do not like the half-way house like "we implemented it, but not working".
> 
> If nobody has taken care about it yet, I am happy to work on it.
> Any comment is very appreciated, of course.

Of course, I was really just asking :).

There are memory reservation functions in U-Boot that get translated to reserve fdt entries on boot. You can probably just call those from your board file.

Alex

> 
> 
> -- 
> Best Regards
> Masahiro Yamada