Odroid XU3 support

Odroid XU3 support

[Ivan Pavić2]

Hello,

i'm trying to boot dom0 linux on exynos5422 platform on A15 (big cpu cluster). 
From the past mailing lists on this link: 
http://lists.xen.org/archives/html/xen-devel/2016-02/msg02275.html) 
I'm not sure that is well defined how to change A7's for A15's. 
Should change be in DTS or should one change cpu power up routines in linux? 
So far every time it boots from A7's. 
Additionally, I found this in xen boot.ini file:

# use the A7s for dom0 and pin them to it
setenv xen_bootargs sync_console console=dtuart dtuart=/serial@12C20000 \
dom0_mem=800M dom0_max_vcpus=4 dom0_vcpus_pin

How this defines A7s for dom0?

How can I check which core is currently active? 
Judging by this link on big.LITTLE architecture:
http://forum.odroid.com/viewtopic.php?f=65&t=2580

result of: cat /proc/cpuinfo | grep "CPU part" is
CPU part    : 0xc07

which stands for A7.

Regards,

Ivan Pavic.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Odroid XU3 support

[Julien Grall]

(CC the author of the Odroid support in Xen)

On 25/04/2016 11:01, Ivan Pavić2 wrote:
> Hello,

Hello,

I have CCed Suriyan who added the support of Odroid in Xen. He might be 
able to help you here.

>
> i'm trying to boot dom0 linux on exynos5422 platform on A15 (big cpu cluster).
>  From the past mailing lists on this link:
> http://lists.xen.org/archives/html/xen-devel/2016-02/msg02275.html)
> I'm not sure that is well defined how to change A7's for A15's.
> Should change be in DTS or should one change cpu power up routines in linux?
> So far every time it boots from A7's.
> Additionally, I found this in xen boot.ini file:
>
> # use the A7s for dom0 and pin them to it
> setenv xen_bootargs sync_console console=dtuart dtuart=/serial@12C20000 \
> dom0_mem=800M dom0_max_vcpus=4 dom0_vcpus_pin
>
> How this defines A7s for dom0?
>
> How can I check which core is currently active?
> Judging by this link on big.LITTLE architecture:
> http://forum.odroid.com/viewtopic.php?f=65&t=2580
>
> result of: cat /proc/cpuinfo | grep "CPU part" is
> CPU part    : 0xc07
>
> which stands for A7.
>
> Regards,

Regards,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Odroid XU3 support

[Suriyan Ramasami]

[-- Attachment #1.1: Type: text/plain, Size: 1815 bytes --]

On Tue, Apr 26, 2016 at 3:31 AM, Julien Grall <julien.grall@arm.com> wrote:

> (CC the author of the Odroid support in Xen)
>
> On 25/04/2016 11:01, Ivan Pavić2 wrote:
>
>> Hello,
>>
>
> Hello,
>
> I have CCed Suriyan who added the support of Odroid in Xen. He might be
> able to help you here.
>

>
>> i'm trying to boot dom0 linux on exynos5422 platform on A15 (big cpu
>> cluster).
>>  From the past mailing lists on this link:
>> http://lists.xen.org/archives/html/xen-devel/2016-02/msg02275.html)
>> I'm not sure that is well defined how to change A7's for A15's.
>> Should change be in DTS or should one change cpu power up routines in
>> linux?
>> So far every time it boots from A7's.
>>
>
The Odroid XU3/XU4 afaict has a hardware pin which dictates if a A7 or an
A15 core should boot up. It is currently set to boot up from the A7s.


> Additionally, I found this in xen boot.ini file:
>>
>> # use the A7s for dom0 and pin them to it
>> setenv xen_bootargs sync_console console=dtuart dtuart=/serial@12C20000 \
>> dom0_mem=800M dom0_max_vcpus=4 dom0_vcpus_pin
>>
>> How this defines A7s for dom0?
>>
>> As mentioned above, the hardware by design boots up from the A7 cluster.
Hence, the first 4 cpus are A7s and the next 4 are the A15s.


> How can I check which core is currently active?
>> Judging by this link on big.LITTLE architecture:
>> http://forum.odroid.com/viewtopic.php?f=65&t=2580
>>
>> result of: cat /proc/cpuinfo | grep "CPU part" is
>> CPU part    : 0xc07
>>
>> which stands for A7.
>>
>> If you do this in dom0, it will show all of them to be 0xc07. They are
vCPUs after all.
I am not sure if I answered your question. Please feel free to probe more.
Thanks!
-Suriyan


> Regards,
>>
>
> Regards,
>
> --
> Julien Grall
>

[-- Attachment #1.2: Type: text/html, Size: 3594 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Odroid XU3 support

[Julien Grall]

Hello,

On 27/04/16 23:53, Suriyan Ramasami wrote:

>         How can I check which core is currently active?
>         Judging by this link on big.LITTLE architecture:
>         http://forum.odroid.com/viewtopic.php?f=65&t=2580
>
>         result of: cat /proc/cpuinfo | grep "CPU part" is
>         CPU part    : 0xc07
>
>         which stands for A7.
>
> If you do this in dom0, it will show all of them to be 0xc07. They are
> vCPUs after all.

Which is not a good idea. This means that Linux is not able to detect 
potential errata for the underlying cores (in this case the cortex-A15). 
Also some userspace may do some runtime optimization based on the kind 
of CPUs available in the guest.

Xen is not ready for big.LITTLE, so I would highly recommend you to 
disable either all the Cortex-A15 or Cortex-A7.

For your information, I am planning to send a patch to park any CPUs 
whose MIDR is not matching the boot CPU one.

Regards,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Odroid XU3 support

[Ivan Pavić2]

> Xen is not ready for big.LITTLE, so I would highly recommend you to
> disable either all the Cortex-A15 or Cortex-A7.

> For your information, I am planning to send a patch to park any CPUs
> whose MIDR is not matching the boot CPU one.

> Julien Grall

Ok, 

I decided to use A15s... How can I  disable A7s in software and use
only A15s... Additionally how can I be sure that I'm using A15 cluster?

This maybe more ODROID related:

Suriyan Ramasami wrote:

> The Odroid XU3/XU4 afaict has a hardware pin which dictates if a A7
> or an A15 core should boot up. It is currently set to boot up from the A7s.

This may sound silly but where is that PIN on XU3 board.? Can 
this be modfied(is it accessible)? If not, how can I do it in software... 
Where to change order? Where to disable A7s? Probably in uBoot? 

Regards, Ivan Pavic.
  
 

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Odroid XU3 support

[Suriyan Ramasami]

[-- Attachment #1.1: Type: text/plain, Size: 1546 bytes --]

Hello Julien,
    Thank you for the advice. I do have a follow up question.

On Thu, Apr 28, 2016 at 2:50 AM, Julien Grall <julien.grall@arm.com> wrote:

> Hello,
>
> On 27/04/16 23:53, Suriyan Ramasami wrote:
>
>         How can I check which core is currently active?
>>         Judging by this link on big.LITTLE architecture:
>>         http://forum.odroid.com/viewtopic.php?f=65&t=2580
>>
>>         result of: cat /proc/cpuinfo | grep "CPU part" is
>>         CPU part    : 0xc07
>>
>>         which stands for A7.
>>
>> If you do this in dom0, it will show all of them to be 0xc07. They are
>> vCPUs after all.
>>
>
> Which is not a good idea. This means that Linux is not able to detect
> potential errata for the underlying cores (in this case the cortex-A15).
> Also some userspace may do some runtime optimization based on the kind of
> CPUs available in the guest.
>
> Xen is not ready for big.LITTLE, so I would highly recommend you to
> disable either all the Cortex-A15 or Cortex-A7.
>
>
Ian did recommend that if they were in their own cpu pools it would avoid
mixing them in a guest. I was researching that angle. What is your take on
that?

If Linux is not recognizing it, that is a dom0/domU issue, is it not?

Nonetheless, to start with, to add support, I think there would be less
resistance if the boot cluster (a7) cpus are enabled and the other cluster
disabled (a15)

For your information, I am planning to send a patch to park any CPUs whose
> MIDR is not matching the boot CPU one.
>
> Regards,
>
> --
> Julien Grall
>

[-- Attachment #1.2: Type: text/html, Size: 2458 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Odroid XU3 support

[Julien Grall]

On 28/04/16 14:26, Suriyan Ramasami wrote:
> Hello Julien,

Hello Suriyan,

>      Thank you for the advice. I do have a follow up question.
>
> On Thu, Apr 28, 2016 at 2:50 AM, Julien Grall <julien.grall@arm.com
> <mailto:julien.grall@arm.com>> wrote:
>
>     Hello,
>
>     On 27/04/16 23:53, Suriyan Ramasami wrote:
>
>                  How can I check which core is currently active?
>                  Judging by this link on big.LITTLE architecture:
>         http://forum.odroid.com/viewtopic.php?f=65&t=2580
>
>                  result of: cat /proc/cpuinfo | grep "CPU part" is
>                  CPU part    : 0xc07
>
>                  which stands for A7.
>
>         If you do this in dom0, it will show all of them to be 0xc07.
>         They are
>         vCPUs after all.
>
>
>     Which is not a good idea. This means that Linux is not able to
>     detect potential errata for the underlying cores (in this case the
>     cortex-A15). Also some userspace may do some runtime optimization
>     based on the kind of CPUs available in the guest.
>
>     Xen is not ready for big.LITTLE, so I would highly recommend you to
>     disable either all the Cortex-A15 or Cortex-A7.
>
> Ian did recommend that if they were in their own cpu pools it would
> avoid mixing them in a guest. I was researching that angle. What is your
> take on that?

I have the same idea in mind. The pools are created at boot time by Xen, 
physical CPUs will be assigned to a pool depending on the CPU ID.

However this is not enough to get support of big.LITTLE. You will at 
least need to modify the domain creation code to set the vMIDR based on 
the based where the vCPU will run.
Furthermore Xen is expecting all the CPUs to have the same set of 
features, hence the boot CPU data is often used to know what could be 
run. This would need some work to get a system wide safe value (see 
arch/arm64/kernel/cpufeature.c in Linux).

>
> If Linux is not recognizing it, that is a dom0/domU issue, is it not?

This is a Xen issue. Xen is exposing the wrong CPU ID to the domain and 
therefore the kernel may not apply properly errata or apply wrong 
optimization.

Regards,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Odroid XU3 support

[Julien Grall]

(CC Stefano)

I forgot the CC stefano on this mail.

On 05/05/16 12:39, Julien Grall wrote:
>
>
> On 28/04/16 14:26, Suriyan Ramasami wrote:
>> Hello Julien,
>
> Hello Suriyan,
>
>>      Thank you for the advice. I do have a follow up question.
>>
>> On Thu, Apr 28, 2016 at 2:50 AM, Julien Grall <julien.grall@arm.com
>> <mailto:julien.grall@arm.com>> wrote:
>>
>>     Hello,
>>
>>     On 27/04/16 23:53, Suriyan Ramasami wrote:
>>
>>                  How can I check which core is currently active?
>>                  Judging by this link on big.LITTLE architecture:
>>         http://forum.odroid.com/viewtopic.php?f=65&t=2580
>>
>>                  result of: cat /proc/cpuinfo | grep "CPU part" is
>>                  CPU part    : 0xc07
>>
>>                  which stands for A7.
>>
>>         If you do this in dom0, it will show all of them to be 0xc07.
>>         They are
>>         vCPUs after all.
>>
>>
>>     Which is not a good idea. This means that Linux is not able to
>>     detect potential errata for the underlying cores (in this case the
>>     cortex-A15). Also some userspace may do some runtime optimization
>>     based on the kind of CPUs available in the guest.
>>
>>     Xen is not ready for big.LITTLE, so I would highly recommend you to
>>     disable either all the Cortex-A15 or Cortex-A7.
>>
>> Ian did recommend that if they were in their own cpu pools it would
>> avoid mixing them in a guest. I was researching that angle. What is your
>> take on that?
>
> I have the same idea in mind. The pools are created at boot time by Xen,
> physical CPUs will be assigned to a pool depending on the CPU ID.
>
> However this is not enough to get support of big.LITTLE. You will at
> least need to modify the domain creation code to set the vMIDR based on
> the based where the vCPU will run.
> Furthermore Xen is expecting all the CPUs to have the same set of
> features, hence the boot CPU data is often used to know what could be
> run. This would need some work to get a system wide safe value (see
> arch/arm64/kernel/cpufeature.c in Linux).
>
>>
>> If Linux is not recognizing it, that is a dom0/domU issue, is it not?
>
> This is a Xen issue. Xen is exposing the wrong CPU ID to the domain and
> therefore the kernel may not apply properly errata or apply wrong
> optimization.
>
> Regards,
>

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Odroid XU3 support

[Peng Fan]

Hi Julien,

On Thu, Apr 28, 2016 at 10:50:33AM +0100, Julien Grall wrote:
>Hello,
>
>On 27/04/16 23:53, Suriyan Ramasami wrote:
>
>>        How can I check which core is currently active?
>>        Judging by this link on big.LITTLE architecture:
>>        http://forum.odroid.com/viewtopic.php?f=65&t=2580
>>
>>        result of: cat /proc/cpuinfo | grep "CPU part" is
>>        CPU part    : 0xc07
>>
>>        which stands for A7.
>>
>>If you do this in dom0, it will show all of them to be 0xc07. They are
>>vCPUs after all.
>
>Which is not a good idea. This means that Linux is not able to detect
>potential errata for the underlying cores (in this case the cortex-A15). Also
>some userspace may do some runtime optimization based on the kind of CPUs
>available in the guest.
>
>Xen is not ready for big.LITTLE, so I would highly recommend you to disable
>either all the Cortex-A15 or Cortex-A7.

For Cortex A53 + A72 or A53 + A57, xen also not ready?

If need to let xen support A53 + A72 or A53 + A57, what work may need to be done?

Thanks,
Peng
>
>For your information, I am planning to send a patch to park any CPUs whose
>MIDR is not matching the boot CPU one.
>
>Regards,
>
>-- 
>Julien Grall
>
>_______________________________________________
>Xen-devel mailing list
>Xen-devel@lists.xen.org
>http://lists.xen.org/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Odroid XU3 support

[Julien Grall]

On 08/05/2016 12:59, Peng Fan wrote:
> Hi Julien,

Hello Peng,

> On Thu, Apr 28, 2016 at 10:50:33AM +0100, Julien Grall wrote:
>> Hello,
>>
>> On 27/04/16 23:53, Suriyan Ramasami wrote:
>>
>>>         How can I check which core is currently active?
>>>         Judging by this link on big.LITTLE architecture:
>>>         http://forum.odroid.com/viewtopic.php?f=65&t=2580
>>>
>>>         result of: cat /proc/cpuinfo | grep "CPU part" is
>>>         CPU part    : 0xc07
>>>
>>>         which stands for A7.
>>>
>>> If you do this in dom0, it will show all of them to be 0xc07. They are
>>> vCPUs after all.
>>
>> Which is not a good idea. This means that Linux is not able to detect
>> potential errata for the underlying cores (in this case the cortex-A15). Also
>> some userspace may do some runtime optimization based on the kind of CPUs
>> available in the guest.
>>
>> Xen is not ready for big.LITTLE, so I would highly recommend you to disable
>> either all the Cortex-A15 or Cortex-A7.
>
> For Cortex A53 + A72 or A53 + A57, xen also not ready?

Xen is not ready at all for big.LITTLE, no matter the combinations of 
the CPUs.

Some of the cores you mentioned (especially A53 and A57) may contain 
errata that requires workaround in the Linux kernel to avoid potential 
data corruption or deadlock. As the vPIDR will always be equal to the 
boot CPU vMIDR, Linux won't apply the necessary workaround to itself.

So using big and little cores at the same time with Xen is not safe at 
all, even if you create CPU pool with current version of Xen.

>
> If need to let xen support A53 + A72 or A53 + A57, what work may need to be done?

I gave some insights later this thread. See:

http://lists.xen.org/archives/html/xen-devel/2016-05/msg00466.html

Regards,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Odroid XU3 support

[Peng Fan]

Hi Julien,

On Mon, May 09, 2016 at 10:49:58AM +0100, Julien Grall wrote:
>
>
>On 08/05/2016 12:59, Peng Fan wrote:
>>Hi Julien,
>
>Hello Peng,
>
>>On Thu, Apr 28, 2016 at 10:50:33AM +0100, Julien Grall wrote:
>>>Hello,
>>>
>>>On 27/04/16 23:53, Suriyan Ramasami wrote:
>>>
>>>>        How can I check which core is currently active?
>>>>        Judging by this link on big.LITTLE architecture:
>>>>        http://forum.odroid.com/viewtopic.php?f=65&t=2580
>>>>
>>>>        result of: cat /proc/cpuinfo | grep "CPU part" is
>>>>        CPU part    : 0xc07
>>>>
>>>>        which stands for A7.
>>>>
>>>>If you do this in dom0, it will show all of them to be 0xc07. They are
>>>>vCPUs after all.
>>>
>>>Which is not a good idea. This means that Linux is not able to detect
>>>potential errata for the underlying cores (in this case the cortex-A15). Also
>>>some userspace may do some runtime optimization based on the kind of CPUs
>>>available in the guest.
>>>
>>>Xen is not ready for big.LITTLE, so I would highly recommend you to disable
>>>either all the Cortex-A15 or Cortex-A7.
>>
>>For Cortex A53 + A72 or A53 + A57, xen also not ready?
>
>Xen is not ready at all for big.LITTLE, no matter the combinations of the
>CPUs.
>
>Some of the cores you mentioned (especially A53 and A57) may contain errata
>that requires workaround in the Linux kernel to avoid potential data
>corruption or deadlock. As the vPIDR will always be equal to the boot CPU
>vMIDR, Linux won't apply the necessary workaround to itself.
>
>So using big and little cores at the same time with Xen is not safe at all,
>even if you create CPU pool with current version of Xen.
>
>>
>>If need to let xen support A53 + A72 or A53 + A57, what work may need to be done?
>
>I gave some insights later this thread. See:
>
>http://lists.xen.org/archives/html/xen-devel/2016-05/msg00466.html

Thanks for this info. Do you have plan to add bit.LITTLE support for xen?
I would like to use this on AArch64 platform-:)

Thanks,
Peng.

>
>Regards,
>
>-- 
>Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Odroid XU3 support

[Julien Grall]

On 10/05/16 08:53, Peng Fan wrote:
> Hi Julien,

Hello Peng,

>
> Thanks for this info. Do you have plan to add bit.LITTLE support for xen?

It is not on my top priority for Xen. Feel free to send a patch series 
to add support for big.LITTLE.

Regards,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel