Re: [PATCH 4/4] xen/arm: update the docs about heterogeneous computing

[Julien Grall <julien.grall@arm.com>]

Hi,

On 17/02/2018 00:31, Stefano Stabellini wrote:
> On Fri, 16 Feb 2018, Julien Grall wrote:
>> On 16/02/2018 21:15, Stefano Stabellini wrote:
>>> On Fri, 16 Feb 2018, Julien Grall wrote:
>>>> On 16/02/2018 20:50, Stefano Stabellini wrote:
>>>>> On Fri, 16 Feb 2018, Julien Grall wrote:
>>>>>> Hi Stefano,
>>>>>>
>>>>>> On 15/02/18 23:17, Stefano Stabellini wrote:
>>>>>>> Update the documentation of the hmp-unsafe option to explain how to
>>>>>>> use
>>>>>>> it safely, together with the right cpu affinity setting, on
>>>>>>> big.LITTLE
>>>>>>> systems.
>>>>>>>
>>>>>>> Also update the warning message to point users to the docs.
>>>>>>>
>>>>>>> Signed-off-by: Stefano Stabellini <sstabellini@kernel.org>
>>>>>>> CC: jbeulich@suse.com
>>>>>>> CC: konrad.wilk@oracle.com
>>>>>>> CC: tim@xen.org
>>>>>>> CC: wei.liu2@citrix.com
>>>>>>> CC: andrew.cooper3@citrix.com
>>>>>>> CC: George.Dunlap@eu.citrix.com
>>>>>>> CC: ian.jackson@eu.citrix.com
>>>>>>>
>>>>>>> ---
>>>>>>>      docs/misc/xen-command-line.markdown | 10 +++++++++-
>>>>>>>      xen/arch/arm/smpboot.c              |  9 +++++----
>>>>>>>      2 files changed, 14 insertions(+), 5 deletions(-)
>>>>>>>
>>>>>>> diff --git a/docs/misc/xen-command-line.markdown
>>>>>>> b/docs/misc/xen-command-line.markdown
>>>>>>> index 2184cb9..a1ebeea 100644
>>>>>>> --- a/docs/misc/xen-command-line.markdown
>>>>>>> +++ b/docs/misc/xen-command-line.markdown
>>>>>>> @@ -1007,7 +1007,15 @@ Control Xens use of the APEI Hardware Error
>>>>>>> Source
>>>>>>> Table, should one be found.
>>>>>>>        Say yes at your own risk if you want to enable heterogenous
>>>>>>> computing
>>>>>>>      (such as big.LITTLE). This may result to an unstable and
>>>>>>> insecure
>>>>>>> -platform. When the option is disabled (default), CPUs that are not
>>>>>>> +platform, unless you manually specify the cpu affinity of all
>>>>>>> domains
>>>>>>> so
>>>>>>> +that all vcpus are scheduled on the same class of pcpus (big or
>>>>>>> LITTLE
>>>>>>> +but not both). vcpu migration between big cores and LITTLE cores is
>>>>>>> not
>>>>>>> +supported. Thus, if the first 4 pcpus are big and the last 4 are
>>>>>>> LITTLE,
>>>>>>> +all domains need to have either cpus = "0-3" or cpus = "4-7" in
>>>>>>> their
>>>>>>> VM
>>>>>>> +config. Moreover, dom0_vcpus_pin needs to be passed on the Xen
>>>>>>> command
>>>>>>> +line.
>>>>>>
>>>>>> In your example here you suggest to have all the vCPUs of a guest to
>>>>>> either on
>>>>>> big or LITTLE cores. How about giving an example where the guest can
>>>>>> have
>>>>>> 2
>>>>>> LITTLE vCPUs and one big vCPU?
>>>>>
>>>>> I would rather discourage it at the moment, given that it requires more
>>>>> complex cpu affinity settings, or vcpu pinning. Also, I am afraid that
>>>>> without matching corresponding topology information on the guest device
>>>>> tree, guests might not work as expected in such a scenario.
>>>>>
>>>>> What do you think?
>>>>
>>>> You already know my view on this. I would rather strongly discourage
>>>> anyone
>>>> pinning all vCPUs of a domain to big cores. We should avoid to provide
>>>> shortcuts to use that could have potentially damageable impact on their
>>>> platform without telling them.
>>>
>>> Do you have a link to a doc somewhere that provides more details about
>>> this? We could add a link to it here to inform users. It would be
>>> useful.
>>
>> This is quite well described in
>> https://xenbits.xen.org/docs/unstable/man/xl.cfg.5.html#CPU-Allocation see
>> "cpus".
> 
> OK, I'll add the link in a new big.LITTLE doc. Also, do you have any
> documentation or link about big core being potentially damaging? It
> would be good to provide information about that too in the big.LITTLE
> doc.

I don't have specific documentation to point on it but I would quite 
interesting to know what is your documentation regarding why always 
running on big is safe. I provided you quite a few insights why this may 
not safe on all platforms and we all remember those phones burning you 
when playing game or watching a video. So I don't feel Xen Project 
should encourage those setups by default.

I would recommend you to read the thread about big.LITTLE in Xen from 
2016: 
https://lists.xenproject.org/archives/html/xen-devel/2016-09/msg01802.html

A few interesting things from that conversation:

"big.LITTLE is a generic term to have 'power hungry and powerful core 
powerful' (big) with slower and battery-saving cores (LITTLE)."

"The use case of big.LITTLE is big cores are used for short period of 
burst and little core are used for the rest (e.g listening audio, 
fetching mail...). If you want to reduce latency when switch between big 
and little CPUs, you may want to put them within the same cluster."

Cheers,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel