sched: Question about big and little cores system with SMP and EAS

sched: Question about big and little cores system with SMP and EAS

[Ley Foon Tan]

Hi all

Would like to ask the experts here regarding the Symmetric
Multi-Processing mode (SMP) with Energy aware scheduler (EAS) support
on the big + little cores system.

Hardware system:
Big and little cores have almost the same ISA, but the big core has
some extension instructions that little core doesn't have.

With this hardware system, is it still possible to use SMP with EAS?
Any suggestions supporting this kind of hardware system in Linux?

Thanks in advance.

Regards
Ley Foon

Re: sched: Question about big and little cores system with SMP and EAS

[Peter Zijlstra]

On Wed, Jun 16, 2021 at 07:29:26PM +0800, Ley Foon Tan wrote:
> Hi all
> 
> Would like to ask the experts here regarding the Symmetric
> Multi-Processing mode (SMP) with Energy aware scheduler (EAS) support
> on the big + little cores system.

And the you ask a question unrelated to either Symmetric MP or EAS :-)

> Hardware system:
> Big and little cores have almost the same ISA, but the big core has
> some extension instructions that little core doesn't have.

That problem is unrelated to big.Little / EAS, also by definition that
is not SMP seeing how the 'S' is a blatant lie.

The simplest solution is to simply disallow usage of the extended ISA
and force mandate the common subset. The complicated answer is something
along the lines of:

  https://lkml.kernel.org/r/20210608180313.11502-1-will@kernel.org

Re: sched: Question about big and little cores system with SMP and EAS

[Dietmar Eggemann]

- Quentin Perret <quentin.perret@arm.com>
+ Quentin Perret <qperret@google.com>

On 16/06/2021 13:39, Peter Zijlstra wrote:
> On Wed, Jun 16, 2021 at 07:29:26PM +0800, Ley Foon Tan wrote:
>> Hi all
>>
>> Would like to ask the experts here regarding the Symmetric
>> Multi-Processing mode (SMP) with Energy aware scheduler (EAS) support
>> on the big + little cores system.
> 
> And the you ask a question unrelated to either Symmetric MP or EAS :-)
> 
>> Hardware system:
>> Big and little cores have almost the same ISA, but the big core has
>> some extension instructions that little core doesn't have.
> 
> That problem is unrelated to big.Little / EAS, also by definition that
> is not SMP seeing how the 'S' is a blatant lie.
> 
> The simplest solution is to simply disallow usage of the extended ISA
> and force mandate the common subset. The complicated answer is something
> along the lines of:
> 
>   https://lkml.kernel.org/r/20210608180313.11502-1-will@kernel.org

We don't encourage asymmetric ISA extensions for EAS*/CAS** on
big.Little systems.
It would be simply a nightmare to schedule tasks on such systems.

The exception to this is the 'asymmetric 32-bit Soc' to support 32bit
legacy Apps. The nightmare for scheduling is reduced in this case to CPU
affinity, something the task scheduler has to live with already today.
(+ DL admission control for 32bit tasks).

*  Documentation/scheduler/sched-energy.rst
** Documentation/scheduler/sched-capacity.rst

Re: sched: Question about big and little cores system with SMP and EAS

[Ley Foon Tan]

On Wed, Jun 16, 2021 at 9:20 PM Dietmar Eggemann
<dietmar.eggemann@arm.com> wrote:
>
> - Quentin Perret <quentin.perret@arm.com>
> + Quentin Perret <qperret@google.com>
>
> On 16/06/2021 13:39, Peter Zijlstra wrote:
> > On Wed, Jun 16, 2021 at 07:29:26PM +0800, Ley Foon Tan wrote:
> >> Hi all
> >>
> >> Would like to ask the experts here regarding the Symmetric
> >> Multi-Processing mode (SMP) with Energy aware scheduler (EAS) support
> >> on the big + little cores system.
> >
> > And the you ask a question unrelated to either Symmetric MP or EAS :-)
> >
> >> Hardware system:
> >> Big and little cores have almost the same ISA, but the big core has
> >> some extension instructions that little core doesn't have.
> >
> > That problem is unrelated to big.Little / EAS, also by definition that
> > is not SMP seeing how the 'S' is a blatant lie.
> >
> > The simplest solution is to simply disallow usage of the extended ISA
> > and force mandate the common subset. The complicated answer is something
> > along the lines of:
> >
> >   https://lkml.kernel.org/r/20210608180313.11502-1-will@kernel.org
>
> We don't encourage asymmetric ISA extensions for EAS*/CAS** on
> big.Little systems.
> It would be simply a nightmare to schedule tasks on such systems.
>
> The exception to this is the 'asymmetric 32-bit Soc' to support 32bit
> legacy Apps. The nightmare for scheduling is reduced in this case to CPU
> affinity, something the task scheduler has to live with already today.
> (+ DL admission control for 32bit tasks).
>
> *  Documentation/scheduler/sched-energy.rst
> ** Documentation/scheduler/sched-capacity.rst

Thanks for the reply.
Yes, forsee it is very complicated and nightmare for software to
support for SMP mode but HW is not real "symmetric".
That's why post the question here to ask the advice and comment from
experts here. So that can feedback to HW team.
Asymmetric extension instructions issue should more complicated than
asymmetric 32-bit app support, it can happen in all the areas (kernel,
app, library and etc).

Thanks!

Regards
Ley Foon

Re: sched: Question about big and little cores system with SMP and EAS

[Morten Rasmussen]

On Thu, Jun 17, 2021 at 01:00:12PM +0800, Ley Foon Tan wrote:
> On Wed, Jun 16, 2021 at 9:20 PM Dietmar Eggemann
> <dietmar.eggemann@arm.com> wrote:
> >
> > - Quentin Perret <quentin.perret@arm.com>
> > + Quentin Perret <qperret@google.com>
> >
> > On 16/06/2021 13:39, Peter Zijlstra wrote:
> > > On Wed, Jun 16, 2021 at 07:29:26PM +0800, Ley Foon Tan wrote:
> > >> Hi all
> > >>
> > >> Would like to ask the experts here regarding the Symmetric
> > >> Multi-Processing mode (SMP) with Energy aware scheduler (EAS) support
> > >> on the big + little cores system.
> > >
> > > And the you ask a question unrelated to either Symmetric MP or EAS :-)
> > >
> > >> Hardware system:
> > >> Big and little cores have almost the same ISA, but the big core has
> > >> some extension instructions that little core doesn't have.
> > >
> > > That problem is unrelated to big.Little / EAS, also by definition that
> > > is not SMP seeing how the 'S' is a blatant lie.
> > >
> > > The simplest solution is to simply disallow usage of the extended ISA
> > > and force mandate the common subset. The complicated answer is something
> > > along the lines of:
> > >
> > >   https://lkml.kernel.org/r/20210608180313.11502-1-will@kernel.org
> >
> > We don't encourage asymmetric ISA extensions for EAS*/CAS** on
> > big.Little systems.
> > It would be simply a nightmare to schedule tasks on such systems.
> >
> > The exception to this is the 'asymmetric 32-bit Soc' to support 32bit
> > legacy Apps. The nightmare for scheduling is reduced in this case to CPU
> > affinity, something the task scheduler has to live with already today.
> > (+ DL admission control for 32bit tasks).
> >
> > *  Documentation/scheduler/sched-energy.rst
> > ** Documentation/scheduler/sched-capacity.rst
> 
> Thanks for the reply.
> Yes, forsee it is very complicated and nightmare for software to
> support for SMP mode but HW is not real "symmetric".
> That's why post the question here to ask the advice and comment from
> experts here. So that can feedback to HW team.
> Asymmetric extension instructions issue should more complicated than
> asymmetric 32-bit app support, it can happen in all the areas (kernel,
> app, library and etc).

Indeed. Detecting what extensions a task might use difficult, if not
impossible. Also, we certainly don't want to end up in situation where
the CPU subsets supporting two extensions are disjoint and a task
requires both extensions.

Morten