Re: [Qemu-devel] [PATCH RFC V2 1/4] Use Aff1 with mpidr

[Shlomo Pongratz <shlomo.pongratz@huawei.com>]

> -----Original Message-----
> From: Igor Mammedov [mailto:imammedo@redhat.com]
> Sent: Wednesday, 27 May, 2015 7:12 PM
> To: shlomopongratz@gmail.com
> Cc: qemu-devel@nongnu.org; peter.maydell@linaro.org; Shlomo Pongratz
> Subject: Re: [Qemu-devel] [PATCH RFC V2 1/4] Use Aff1 with mpidr
> 
> On Wed,  6 May 2015 17:04:39 +0300
> shlomopongratz@gmail.com wrote:
> 
> > From: Shlomo Pongratz <shlomo.pongratz@huawei.com>
> >
> > In order to support up to 128 cores with GIC-500 (GICv3
> > implementation)
> > affinity1 must be used. GIC-500 support up to 32 clusters with up to
> > 8 cores in a cluster. So for example, if one wishes to have 16 cores,
> > the options are: 2 clusters of 8 cores each, 4 clusters with 4 cores
> > each Currently only the first option is supported.
> > In order to have more flexible scheme the virt machine must pass the
> > desired scheme.
> >
> > Signed-off-by: Shlomo Pongratz <shlomo.pongratz@huawei.com>
> > ---
> >  target-arm/helper.c | 12 ++++++++++--
> >  target-arm/psci.c   | 18 ++++++++++++++++--
> >  2 files changed, 26 insertions(+), 4 deletions(-)
> >
> > diff --git a/target-arm/helper.c b/target-arm/helper.c index
> > f8f8d76..f555c0b 100644
> > --- a/target-arm/helper.c
> > +++ b/target-arm/helper.c
> > @@ -2035,11 +2035,19 @@ static const ARMCPRegInfo
> > strongarm_cp_reginfo[] = {  static uint64_t mpidr_read(CPUARMState
> > *env, const ARMCPRegInfo *ri)  {
> >      CPUState *cs = CPU(arm_env_get_cpu(env));
> > -    uint32_t mpidr = cs->cpu_index;
> > -    /* We don't support setting cluster ID ([8..11]) (known as Aff1
> > +    uint32_t mpidr, aff0, aff1;
> > +    uint32_t cpuid = cs->cpu_index;
> > +    /* We don't support setting cluster ID ([16..23]) (known as Aff2
> >       * in later ARM ARM versions), or any of the higher affinity level fields,
> >       * so these bits always RAZ.
> >       */
> > +    /* Currently GIC-500 code supports 64 cores in 16 clusters with 8 cores
> each
> > +     * Future code will remove this limitation.
> > +     * This code is valid for GIC-400 too.
> > +     */
> > +    aff0 = cpuid % 8;
> Even though GIC-500 spec says that it supports 8 cores I haven't found in it
> how it encodes aff0 but encoding is specified by respective CPU specs. I've
> checked a53, a57, a72 specs and they limit aff0 to max 0x3 value.
> I think encoding should be CPU type specific i.e. not defined by what GIC can
> support and once we add CPU type with 8 cores, it would provide it's own
> version of mpidr_read since it would be defined by spec how to encode aff0.

Assume we let the processor select its mpidr than for cortex-57 it will select 0-3 now how a system with GICv2 with 8 cores will work?
Note that GICv2 that doesn't affinity levels. In the GIC-500 it is written that with GICv2 backwards compatibility " This mode supports up to eight cores. The eight cores supported 
by the GIC-500 are the cores with the lowest affinity numbers". So all 8 cores must have aff0 0 to 7.
It is obvious that with GICv3 if we have 2 cortex-a57 we should have affinities 0.0.0.{0-3} and 0.0.1.{0-3} but with GICv2 we should have 0.0.0.{0-7}.
Now the only component that knows which GIC is used is the virt, it can set a property telling the GIC and the system what is the size of the lowest level affinity.
I think this number is the upper power of two of the SMP number divided by the (GICD_TYPER.CPUNumber + 1) but I'm not sure.

Best regards,

S.P.

> 
> > +    aff1 = cpuid / 8;
> > +    mpidr = (aff1 << 8) | aff0;
> >      if (arm_feature(env, ARM_FEATURE_V7MP)) {
> >          mpidr |= (1U << 31);
> >          /* Cores which are uniprocessor (non-coherent) diff --git
> > a/target-arm/psci.c b/target-arm/psci.c index d8fafab..64fbe61 100644
> > --- a/target-arm/psci.c
> > +++ b/target-arm/psci.c
> > @@ -86,6 +86,7 @@ void arm_handle_psci_call(ARMCPU *cpu)
> >      CPUARMState *env = &cpu->env;
> >      uint64_t param[4];
> >      uint64_t context_id, mpidr;
> > +    uint32_t core, Aff1, Aff0;
> >      target_ulong entry;
> >      int32_t ret = 0;
> >      int i;
> > @@ -121,7 +122,16 @@ void arm_handle_psci_call(ARMCPU *cpu)
> >
> >          switch (param[2]) {
> >          case 0:
> > -            target_cpu_state = qemu_get_cpu(mpidr & 0xff);
> > +            /* MPIDR_EL1 [RES0:affinity-3:RES1:U:RES0:MT:affinity-1:affinity-0]
> > +             * GIC 500 code currently supports 32 clusters with 8 cores each
> > +             * but no more than 128 cores. Future version will have flexible
> > +             * affinity selection
> > +             * GIC 400 supports 8 cores so 0x7 for Aff0 is O.K. too
> > +             */
> > +            Aff1 = (mpidr & 0xff00) >> (8 - 3); /* Shift by 8 multiply by 8 */
> > +            Aff0 = mpidr & 0x7;
> > +            core = Aff1 + Aff0;
> > +            target_cpu_state = qemu_get_cpu(core);
> >              if (!target_cpu_state) {
> >                  ret = QEMU_PSCI_RET_INVALID_PARAMS;
> >                  break;
> > @@ -153,7 +163,11 @@ void arm_handle_psci_call(ARMCPU *cpu)
> >          context_id = param[3];
> >
> >          /* change to the cpu we are powering up */
> > -        target_cpu_state = qemu_get_cpu(mpidr & 0xff);
> > +        /* Currently supports 64 cores in 16 clusters with 8 cores each */
> > +        Aff1 = (mpidr & 0xff00) >> (8 - 3); /* Shift by 8 multiply by 8 */
> > +        Aff0 = mpidr & 0x7;
> > +        core = Aff1 + Aff0;
> > +        target_cpu_state = qemu_get_cpu(core);
> >          if (!target_cpu_state) {
> >              ret = QEMU_PSCI_RET_INVALID_PARAMS;
> >              break;