[PATCH 0/5] gicv3: Use right number of prio bits for the CPU

[PATCH 0/5] gicv3: Use right number of prio bits for the CPU

[Peter Maydell]

This patchset fills in an odd inconsistency in our GICv3 emulation
that I noticed while I was doing the GICv4 work. At the moment we
allow the CPU to specify the number of bits of virtual priority
(via the ARMCPU::gic_vpribits field), but we always use 8 bits of
physical priority, even though to my knowledge no real Arm CPU
hardware has that many.

This series makes the GICv3 emulation use a runtime-configurable
number of physical priority bits, and sets it to match the number
used by the various CPUs we implement (which is 5 for all the
Cortex-Axx CPUs we emulate). Because changing the number of
priority bits is a migration compatibility break, we use a compat
property to keep the number of priority bits at 8 for older
versions of the virt board.

There is one TODO left in this series, which is that I don't know
the right value to use for the A64FX, so I've guessed that it
is 5, like all the Arm implementations.

Patch 1 is an independent bugfix; patch 5 is cleanup.

thanks
-- PMM

Peter Maydell (5):
  hw/intc/arm_gicv3: report correct PRIbits field in ICV_CTLR_EL1
  hw/intc/arm_gicv3_kvm.c: Stop using GIC_MIN_BPR constant
  hw/intc/arm_gicv3: Support configurable number of physical priority
    bits
  hw/intc/arm_gicv3: Use correct number of priority bits for the CPU
  hw/intc/arm_gicv3: Provide ich_num_aprs()

 include/hw/intc/arm_gicv3_common.h |   8 +-
 target/arm/cpu.h                   |   1 +
 hw/core/machine.c                  |   4 +-
 hw/intc/arm_gicv3_common.c         |   5 +
 hw/intc/arm_gicv3_cpuif.c          | 208 ++++++++++++++++++++---------
 hw/intc/arm_gicv3_kvm.c            |  16 ++-
 target/arm/cpu64.c                 |   9 ++
 7 files changed, 179 insertions(+), 72 deletions(-)

-- 
2.25.1

[PATCH 1/5] hw/intc/arm_gicv3: report correct PRIbits field in ICV_CTLR_EL1

[Peter Maydell]

As noted in the comment, the PRIbits field in ICV_CTLR_EL1 is
supposed to match the ICH_VTR_EL2 PRIbits setting; that is, it is the
virtual priority bit setting, not the physical priority bit setting.
(For QEMU currently we always implement 8 bits of physical priority,
so the PRIbits field was previously 7, since it is defined to be
"priority bits - 1".)

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 hw/intc/arm_gicv3_cpuif.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/hw/intc/arm_gicv3_cpuif.c b/hw/intc/arm_gicv3_cpuif.c
index 9efba798f82..d3b92a36636 100644
--- a/hw/intc/arm_gicv3_cpuif.c
+++ b/hw/intc/arm_gicv3_cpuif.c
@@ -657,7 +657,7 @@ static uint64_t icv_ctlr_read(CPUARMState *env, const ARMCPRegInfo *ri)
      * should match the ones reported in ich_vtr_read().
      */
     value = ICC_CTLR_EL1_A3V | (1 << ICC_CTLR_EL1_IDBITS_SHIFT) |
-        (7 << ICC_CTLR_EL1_PRIBITS_SHIFT);
+        ((cs->vpribits - 1) << ICC_CTLR_EL1_PRIBITS_SHIFT);
 
     if (cs->ich_vmcr_el2 & ICH_VMCR_EL2_VEOIM) {
         value |= ICC_CTLR_EL1_EOIMODE;
-- 
2.25.1

[PATCH 2/5] hw/intc/arm_gicv3_kvm.c: Stop using GIC_MIN_BPR constant

[Peter Maydell]

The GIC_MIN_BPR constant defines the minimum BPR value that the TCG
emulated GICv3 supports.  We're currently using this also as the
value we reset the KVM GICv3 ICC_BPR registers to, but this is only
right by accident.

We want to make the emulated GICv3 use a configurable number of
priority bits, which means that GIC_MIN_BPR will no longer be a
constant.  Replace the uses in the KVM reset code with literal 0,
plus a constant explaining why this is reasonable.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 hw/intc/arm_gicv3_kvm.c | 16 +++++++++++++---
 1 file changed, 13 insertions(+), 3 deletions(-)

diff --git a/hw/intc/arm_gicv3_kvm.c b/hw/intc/arm_gicv3_kvm.c
index 2922c516e56..3ca643ecba4 100644
--- a/hw/intc/arm_gicv3_kvm.c
+++ b/hw/intc/arm_gicv3_kvm.c
@@ -673,9 +673,19 @@ static void arm_gicv3_icc_reset(CPUARMState *env, const ARMCPRegInfo *ri)
     s = c->gic;
 
     c->icc_pmr_el1 = 0;
-    c->icc_bpr[GICV3_G0] = GIC_MIN_BPR;
-    c->icc_bpr[GICV3_G1] = GIC_MIN_BPR;
-    c->icc_bpr[GICV3_G1NS] = GIC_MIN_BPR;
+    /*
+     * Architecturally the reset value of the ICC_BPR registers
+     * is UNKNOWN. We set them all to 0 here; when the kernel
+     * uses these values to program the ICH_VMCR_EL2 fields that
+     * determine the guest-visible ICC_BPR register values, the
+     * hardware's "writing a value less than the minimum sets
+     * the field to the minimum value" behaviour will result in
+     * them effectively resetting to the correct minimum value
+     * for the host GIC.
+     */
+    c->icc_bpr[GICV3_G0] = 0;
+    c->icc_bpr[GICV3_G1] = 0;
+    c->icc_bpr[GICV3_G1NS] = 0;
 
     c->icc_sre_el1 = 0x7;
     memset(c->icc_apr, 0, sizeof(c->icc_apr));
-- 
2.25.1

[PATCH 3/5] hw/intc/arm_gicv3: Support configurable number of physical priority bits

[Peter Maydell]

The GICv3 code has always supported a configurable number of virtual
priority and preemption bits, but our implementation currently
hardcodes the number of physical priority bits at 8.  This is not
what most hardware implementations provide; for instance the
Cortex-A53 provides only 5 bits of physical priority.

Make the number of physical priority/preemption bits driven by fields
in the GICv3CPUState, the way that we already do for virtual
priority/preemption bits.  We set cs->pribits to 8, so there is no
behavioural change in this commit.  A following commit will add the
machinery for CPUs to set this to the correct value for their
implementation.

Note that changing the number of priority bits would be a migration
compatibility break, because the semantics of the icc_apr[][] array
changes.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 include/hw/intc/arm_gicv3_common.h |   7 +-
 hw/intc/arm_gicv3_cpuif.c          | 182 ++++++++++++++++++++---------
 2 files changed, 130 insertions(+), 59 deletions(-)

diff --git a/include/hw/intc/arm_gicv3_common.h b/include/hw/intc/arm_gicv3_common.h
index 4e416100559..46677ec345c 100644
--- a/include/hw/intc/arm_gicv3_common.h
+++ b/include/hw/intc/arm_gicv3_common.h
@@ -51,11 +51,6 @@
 /* Maximum number of list registers (architectural limit) */
 #define GICV3_LR_MAX 16
 
-/* Minimum BPR for Secure, or when security not enabled */
-#define GIC_MIN_BPR 0
-/* Minimum BPR for Nonsecure when security is enabled */
-#define GIC_MIN_BPR_NS (GIC_MIN_BPR + 1)
-
 /* For some distributor fields we want to model the array of 32-bit
  * register values which hold various bitmaps corresponding to enabled,
  * pending, etc bits. These macros and functions facilitate that; the
@@ -206,6 +201,8 @@ struct GICv3CPUState {
     int num_list_regs;
     int vpribits; /* number of virtual priority bits */
     int vprebits; /* number of virtual preemption bits */
+    int pribits; /* number of physical priority bits */
+    int prebits; /* number of physical preemption bits */
 
     /* Current highest priority pending interrupt for this CPU.
      * This is cached information that can be recalculated from the
diff --git a/hw/intc/arm_gicv3_cpuif.c b/hw/intc/arm_gicv3_cpuif.c
index d3b92a36636..8499a49be39 100644
--- a/hw/intc/arm_gicv3_cpuif.c
+++ b/hw/intc/arm_gicv3_cpuif.c
@@ -787,6 +787,36 @@ static uint64_t icv_iar_read(CPUARMState *env, const ARMCPRegInfo *ri)
     return intid;
 }
 
+static uint32_t icc_fullprio_mask(GICv3CPUState *cs)
+{
+    /*
+     * Return a mask word which clears the unimplemented priority bits
+     * from a priority value for a physical interrupt. (Not to be confused
+     * with the group priority, whose mask depends on the value of BPR
+     * for the interrupt group.)
+     */
+    return ~0U << (8 - cs->pribits);
+}
+
+static inline int icc_min_bpr(GICv3CPUState *cs)
+{
+    /* The minimum BPR for the physical interface. */
+    return 7 - cs->prebits;
+}
+
+static inline int icc_min_bpr_ns(GICv3CPUState *cs)
+{
+    return icc_min_bpr(cs) + 1;
+}
+
+static inline int icc_num_aprs(GICv3CPUState *cs)
+{
+    /* Return the number of APR registers (1, 2, or 4) */
+    int aprmax = 1 << MAX(cs->prebits - 5, 0);
+    assert(aprmax <= ARRAY_SIZE(cs->icc_apr[0]));
+    return aprmax;
+}
+
 static int icc_highest_active_prio(GICv3CPUState *cs)
 {
     /* Calculate the current running priority based on the set bits
@@ -794,14 +824,14 @@ static int icc_highest_active_prio(GICv3CPUState *cs)
      */
     int i;
 
-    for (i = 0; i < ARRAY_SIZE(cs->icc_apr[0]); i++) {
+    for (i = 0; i < icc_num_aprs(cs); i++) {
         uint32_t apr = cs->icc_apr[GICV3_G0][i] |
             cs->icc_apr[GICV3_G1][i] | cs->icc_apr[GICV3_G1NS][i];
 
         if (!apr) {
             continue;
         }
-        return (i * 32 + ctz32(apr)) << (GIC_MIN_BPR + 1);
+        return (i * 32 + ctz32(apr)) << (icc_min_bpr(cs) + 1);
     }
     /* No current active interrupts: return idle priority */
     return 0xff;
@@ -980,7 +1010,7 @@ static void icc_pmr_write(CPUARMState *env, const ARMCPRegInfo *ri,
 
     trace_gicv3_icc_pmr_write(gicv3_redist_affid(cs), value);
 
-    value &= 0xff;
+    value &= icc_fullprio_mask(cs);
 
     if (arm_feature(env, ARM_FEATURE_EL3) && !arm_is_secure(env) &&
         (env->cp15.scr_el3 & SCR_FIQ)) {
@@ -1004,7 +1034,7 @@ static void icc_activate_irq(GICv3CPUState *cs, int irq)
      */
     uint32_t mask = icc_gprio_mask(cs, cs->hppi.grp);
     int prio = cs->hppi.prio & mask;
-    int aprbit = prio >> 1;
+    int aprbit = prio >> (8 - cs->prebits);
     int regno = aprbit / 32;
     int regbit = aprbit % 32;
 
@@ -1162,7 +1192,7 @@ static void icc_drop_prio(GICv3CPUState *cs, int grp)
      */
     int i;
 
-    for (i = 0; i < ARRAY_SIZE(cs->icc_apr[grp]); i++) {
+    for (i = 0; i < icc_num_aprs(cs); i++) {
         uint64_t *papr = &cs->icc_apr[grp][i];
 
         if (!*papr) {
@@ -1590,7 +1620,7 @@ static void icc_bpr_write(CPUARMState *env, const ARMCPRegInfo *ri,
         return;
     }
 
-    minval = (grp == GICV3_G1NS) ? GIC_MIN_BPR_NS : GIC_MIN_BPR;
+    minval = (grp == GICV3_G1NS) ? icc_min_bpr_ns(cs) : icc_min_bpr(cs);
     if (value < minval) {
         value = minval;
     }
@@ -2171,19 +2201,19 @@ static void icc_reset(CPUARMState *env, const ARMCPRegInfo *ri)
 
     cs->icc_ctlr_el1[GICV3_S] = ICC_CTLR_EL1_A3V |
         (1 << ICC_CTLR_EL1_IDBITS_SHIFT) |
-        (7 << ICC_CTLR_EL1_PRIBITS_SHIFT);
+        ((cs->pribits - 1) << ICC_CTLR_EL1_PRIBITS_SHIFT);
     cs->icc_ctlr_el1[GICV3_NS] = ICC_CTLR_EL1_A3V |
         (1 << ICC_CTLR_EL1_IDBITS_SHIFT) |
-        (7 << ICC_CTLR_EL1_PRIBITS_SHIFT);
+        ((cs->pribits - 1) << ICC_CTLR_EL1_PRIBITS_SHIFT);
     cs->icc_pmr_el1 = 0;
-    cs->icc_bpr[GICV3_G0] = GIC_MIN_BPR;
-    cs->icc_bpr[GICV3_G1] = GIC_MIN_BPR;
-    cs->icc_bpr[GICV3_G1NS] = GIC_MIN_BPR_NS;
+    cs->icc_bpr[GICV3_G0] = icc_min_bpr(cs);
+    cs->icc_bpr[GICV3_G1] = icc_min_bpr(cs);
+    cs->icc_bpr[GICV3_G1NS] = icc_min_bpr_ns(cs);
     memset(cs->icc_apr, 0, sizeof(cs->icc_apr));
     memset(cs->icc_igrpen, 0, sizeof(cs->icc_igrpen));
     cs->icc_ctlr_el3 = ICC_CTLR_EL3_NDS | ICC_CTLR_EL3_A3V |
         (1 << ICC_CTLR_EL3_IDBITS_SHIFT) |
-        (7 << ICC_CTLR_EL3_PRIBITS_SHIFT);
+        ((cs->pribits - 1) << ICC_CTLR_EL3_PRIBITS_SHIFT);
 
     memset(cs->ich_apr, 0, sizeof(cs->ich_apr));
     cs->ich_hcr_el2 = 0;
@@ -2238,27 +2268,6 @@ static const ARMCPRegInfo gicv3_cpuif_reginfo[] = {
       .readfn = icc_ap_read,
       .writefn = icc_ap_write,
     },
-    { .name = "ICC_AP0R1_EL1", .state = ARM_CP_STATE_BOTH,
-      .opc0 = 3, .opc1 = 0, .crn = 12, .crm = 8, .opc2 = 5,
-      .type = ARM_CP_IO | ARM_CP_NO_RAW,
-      .access = PL1_RW, .accessfn = gicv3_fiq_access,
-      .readfn = icc_ap_read,
-      .writefn = icc_ap_write,
-    },
-    { .name = "ICC_AP0R2_EL1", .state = ARM_CP_STATE_BOTH,
-      .opc0 = 3, .opc1 = 0, .crn = 12, .crm = 8, .opc2 = 6,
-      .type = ARM_CP_IO | ARM_CP_NO_RAW,
-      .access = PL1_RW, .accessfn = gicv3_fiq_access,
-      .readfn = icc_ap_read,
-      .writefn = icc_ap_write,
-    },
-    { .name = "ICC_AP0R3_EL1", .state = ARM_CP_STATE_BOTH,
-      .opc0 = 3, .opc1 = 0, .crn = 12, .crm = 8, .opc2 = 7,
-      .type = ARM_CP_IO | ARM_CP_NO_RAW,
-      .access = PL1_RW, .accessfn = gicv3_fiq_access,
-      .readfn = icc_ap_read,
-      .writefn = icc_ap_write,
-    },
     /* All the ICC_AP1R*_EL1 registers are banked */
     { .name = "ICC_AP1R0_EL1", .state = ARM_CP_STATE_BOTH,
       .opc0 = 3, .opc1 = 0, .crn = 12, .crm = 9, .opc2 = 0,
@@ -2267,27 +2276,6 @@ static const ARMCPRegInfo gicv3_cpuif_reginfo[] = {
       .readfn = icc_ap_read,
       .writefn = icc_ap_write,
     },
-    { .name = "ICC_AP1R1_EL1", .state = ARM_CP_STATE_BOTH,
-      .opc0 = 3, .opc1 = 0, .crn = 12, .crm = 9, .opc2 = 1,
-      .type = ARM_CP_IO | ARM_CP_NO_RAW,
-      .access = PL1_RW, .accessfn = gicv3_irq_access,
-      .readfn = icc_ap_read,
-      .writefn = icc_ap_write,
-    },
-    { .name = "ICC_AP1R2_EL1", .state = ARM_CP_STATE_BOTH,
-      .opc0 = 3, .opc1 = 0, .crn = 12, .crm = 9, .opc2 = 2,
-      .type = ARM_CP_IO | ARM_CP_NO_RAW,
-      .access = PL1_RW, .accessfn = gicv3_irq_access,
-      .readfn = icc_ap_read,
-      .writefn = icc_ap_write,
-    },
-    { .name = "ICC_AP1R3_EL1", .state = ARM_CP_STATE_BOTH,
-      .opc0 = 3, .opc1 = 0, .crn = 12, .crm = 9, .opc2 = 3,
-      .type = ARM_CP_IO | ARM_CP_NO_RAW,
-      .access = PL1_RW, .accessfn = gicv3_irq_access,
-      .readfn = icc_ap_read,
-      .writefn = icc_ap_write,
-    },
     { .name = "ICC_DIR_EL1", .state = ARM_CP_STATE_BOTH,
       .opc0 = 3, .opc1 = 0, .crn = 12, .crm = 11, .opc2 = 1,
       .type = ARM_CP_IO | ARM_CP_NO_RAW,
@@ -2430,6 +2418,54 @@ static const ARMCPRegInfo gicv3_cpuif_reginfo[] = {
     },
 };
 
+static const ARMCPRegInfo gicv3_cpuif_icc_apxr1_reginfo[] = {
+    { .name = "ICC_AP0R1_EL1", .state = ARM_CP_STATE_BOTH,
+      .opc0 = 3, .opc1 = 0, .crn = 12, .crm = 8, .opc2 = 5,
+      .type = ARM_CP_IO | ARM_CP_NO_RAW,
+      .access = PL1_RW, .accessfn = gicv3_fiq_access,
+      .readfn = icc_ap_read,
+      .writefn = icc_ap_write,
+    },
+    { .name = "ICC_AP1R1_EL1", .state = ARM_CP_STATE_BOTH,
+      .opc0 = 3, .opc1 = 0, .crn = 12, .crm = 9, .opc2 = 1,
+      .type = ARM_CP_IO | ARM_CP_NO_RAW,
+      .access = PL1_RW, .accessfn = gicv3_irq_access,
+      .readfn = icc_ap_read,
+      .writefn = icc_ap_write,
+    },
+};
+
+static const ARMCPRegInfo gicv3_cpuif_icc_apxr23_reginfo[] = {
+    { .name = "ICC_AP0R2_EL1", .state = ARM_CP_STATE_BOTH,
+      .opc0 = 3, .opc1 = 0, .crn = 12, .crm = 8, .opc2 = 6,
+      .type = ARM_CP_IO | ARM_CP_NO_RAW,
+      .access = PL1_RW, .accessfn = gicv3_fiq_access,
+      .readfn = icc_ap_read,
+      .writefn = icc_ap_write,
+    },
+    { .name = "ICC_AP0R3_EL1", .state = ARM_CP_STATE_BOTH,
+      .opc0 = 3, .opc1 = 0, .crn = 12, .crm = 8, .opc2 = 7,
+      .type = ARM_CP_IO | ARM_CP_NO_RAW,
+      .access = PL1_RW, .accessfn = gicv3_fiq_access,
+      .readfn = icc_ap_read,
+      .writefn = icc_ap_write,
+    },
+    { .name = "ICC_AP1R2_EL1", .state = ARM_CP_STATE_BOTH,
+      .opc0 = 3, .opc1 = 0, .crn = 12, .crm = 9, .opc2 = 2,
+      .type = ARM_CP_IO | ARM_CP_NO_RAW,
+      .access = PL1_RW, .accessfn = gicv3_irq_access,
+      .readfn = icc_ap_read,
+      .writefn = icc_ap_write,
+    },
+    { .name = "ICC_AP1R3_EL1", .state = ARM_CP_STATE_BOTH,
+      .opc0 = 3, .opc1 = 0, .crn = 12, .crm = 9, .opc2 = 3,
+      .type = ARM_CP_IO | ARM_CP_NO_RAW,
+      .access = PL1_RW, .accessfn = gicv3_irq_access,
+      .readfn = icc_ap_read,
+      .writefn = icc_ap_write,
+    },
+};
+
 static uint64_t ich_ap_read(CPUARMState *env, const ARMCPRegInfo *ri)
 {
     GICv3CPUState *cs = icc_cs_from_env(env);
@@ -2763,6 +2799,44 @@ void gicv3_init_cpuif(GICv3State *s)
          * get back to the GICv3CPUState from the CPUARMState.
          */
         define_arm_cp_regs(cpu, gicv3_cpuif_reginfo);
+
+        /*
+         * For the moment, retain the existing behaviour of 8 priority bits;
+         * in a following commit we will take this from the CPU state,
+         * as we do for the virtual priority bits.
+         */
+        cs->pribits = 8;
+        /*
+         * The GICv3 has separate ID register fields for virtual priority
+         * and preemption bit values, but only a single ID register field
+         * for the physical priority bits. The preemption bit count is
+         * always the same as the priority bit count, except that 8 bits
+         * of priority means 7 preemption bits. We precalculate the
+         * preemption bits because it simplifies the code and makes the
+         * parallels between the virtual and physical bits of the GIC
+         * a bit clearer.
+         */
+        cs->prebits = cs->pribits;
+        if (cs->prebits == 8) {
+            cs->prebits--;
+        }
+        /*
+         * Check that CPU code defining pribits didn't violate
+         * architectural constraints our implementation relies on.
+         */
+        g_assert(cs->pribits >= 4 && cs->pribits <= 8);
+
+        /*
+         * gicv3_cpuif_reginfo[] defines ICC_AP*R0_EL1; add definitions
+         * for ICC_AP*R{1,2,3}_EL1 if the prebits value requires them.
+         */
+        if (cs->prebits >= 6) {
+            define_arm_cp_regs(cpu, gicv3_cpuif_icc_apxr1_reginfo);
+        }
+        if (cs->prebits == 7) {
+            define_arm_cp_regs(cpu, gicv3_cpuif_icc_apxr23_reginfo);
+        }
+
         if (arm_feature(&cpu->env, ARM_FEATURE_EL2)
             && cpu->gic_num_lrs) {
             int j;
-- 
2.25.1

[PATCH 4/5] hw/intc/arm_gicv3: Use correct number of priority bits for the CPU

[Peter Maydell]

Make the GICv3 set its number of bits of physical priority from the
implementation-specific value provided in the CPU state struct, in
the same way we already do for virtual priority bits.  Because this
would be a migration compatibility break, we provide a property
force-8-bit-prio which is enabled for 7.0 and earlier versioned board
models to retain the legacy "always use 8 bits" behaviour.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
I have guessed at the right value for the A64FX, but if we can
find the correct ICC_CTLR_EL1 value that would be better.
---
 include/hw/intc/arm_gicv3_common.h |  1 +
 target/arm/cpu.h                   |  1 +
 hw/core/machine.c                  |  4 +++-
 hw/intc/arm_gicv3_common.c         |  5 +++++
 hw/intc/arm_gicv3_cpuif.c          | 14 ++++++++++----
 target/arm/cpu64.c                 |  9 +++++++++
 6 files changed, 29 insertions(+), 5 deletions(-)

diff --git a/include/hw/intc/arm_gicv3_common.h b/include/hw/intc/arm_gicv3_common.h
index 46677ec345c..ab5182a28a2 100644
--- a/include/hw/intc/arm_gicv3_common.h
+++ b/include/hw/intc/arm_gicv3_common.h
@@ -248,6 +248,7 @@ struct GICv3State {
     uint32_t revision;
     bool lpi_enable;
     bool security_extn;
+    bool force_8bit_prio;
     bool irq_reset_nonsecure;
     bool gicd_no_migration_shift_bug;
 
diff --git a/target/arm/cpu.h b/target/arm/cpu.h
index ca01f909a86..f8873bdbb97 100644
--- a/target/arm/cpu.h
+++ b/target/arm/cpu.h
@@ -993,6 +993,7 @@ struct ArchCPU {
     int gic_num_lrs; /* number of list registers */
     int gic_vpribits; /* number of virtual priority bits */
     int gic_vprebits; /* number of virtual preemption bits */
+    int gic_pribits; /* number of physical priority bits */
 
     /* Whether the cfgend input is high (i.e. this CPU should reset into
      * big-endian mode).  This setting isn't used directly: instead it modifies
diff --git a/hw/core/machine.c b/hw/core/machine.c
index cb9bbc844d2..db012376785 100644
--- a/hw/core/machine.c
+++ b/hw/core/machine.c
@@ -37,7 +37,9 @@
 #include "hw/virtio/virtio.h"
 #include "hw/virtio/virtio-pci.h"
 
-GlobalProperty hw_compat_7_0[] = {};
+GlobalProperty hw_compat_7_0[] = {
+    { "arm-gicv3-common", "force-8-bit-prio", "on" },
+};
 const size_t hw_compat_7_0_len = G_N_ELEMENTS(hw_compat_7_0);
 
 GlobalProperty hw_compat_6_2[] = {
diff --git a/hw/intc/arm_gicv3_common.c b/hw/intc/arm_gicv3_common.c
index 5634c6fc788..351843db4aa 100644
--- a/hw/intc/arm_gicv3_common.c
+++ b/hw/intc/arm_gicv3_common.c
@@ -563,6 +563,11 @@ static Property arm_gicv3_common_properties[] = {
     DEFINE_PROP_UINT32("revision", GICv3State, revision, 3),
     DEFINE_PROP_BOOL("has-lpi", GICv3State, lpi_enable, 0),
     DEFINE_PROP_BOOL("has-security-extensions", GICv3State, security_extn, 0),
+    /*
+     * Compatibility property: force 8 bits of physical priority, even
+     * if the CPU being emulated should have fewer.
+     */
+    DEFINE_PROP_BOOL("force-8-bit-prio", GICv3State, force_8bit_prio, 0),
     DEFINE_PROP_ARRAY("redist-region-count", GICv3State, nb_redist_regions,
                       redist_region_count, qdev_prop_uint32, uint32_t),
     DEFINE_PROP_LINK("sysmem", GICv3State, dma, TYPE_MEMORY_REGION,
diff --git a/hw/intc/arm_gicv3_cpuif.c b/hw/intc/arm_gicv3_cpuif.c
index 8499a49be39..e277a807bd5 100644
--- a/hw/intc/arm_gicv3_cpuif.c
+++ b/hw/intc/arm_gicv3_cpuif.c
@@ -2801,11 +2801,17 @@ void gicv3_init_cpuif(GICv3State *s)
         define_arm_cp_regs(cpu, gicv3_cpuif_reginfo);
 
         /*
-         * For the moment, retain the existing behaviour of 8 priority bits;
-         * in a following commit we will take this from the CPU state,
-         * as we do for the virtual priority bits.
+         * The CPU implementation specifies the number of supported
+         * bits of physical priority. For backwards compatibility
+         * of migration, we have a compat property that forces use
+         * of 8 priority bits regardless of what the CPU really has.
          */
-        cs->pribits = 8;
+        if (s->force_8bit_prio) {
+            cs->pribits = 8;
+        } else {
+            cs->pribits = cpu->gic_pribits;
+        }
+
         /*
          * The GICv3 has separate ID register fields for virtual priority
          * and preemption bit values, but only a single ID register field
diff --git a/target/arm/cpu64.c b/target/arm/cpu64.c
index c841d55d0e9..490231b90f3 100644
--- a/target/arm/cpu64.c
+++ b/target/arm/cpu64.c
@@ -143,6 +143,7 @@ static void aarch64_a57_initfn(Object *obj)
     cpu->gic_num_lrs = 4;
     cpu->gic_vpribits = 5;
     cpu->gic_vprebits = 5;
+    cpu->gic_pribits = 5;
     define_arm_cp_regs(cpu, cortex_a72_a57_a53_cp_reginfo);
 }
 
@@ -196,6 +197,7 @@ static void aarch64_a53_initfn(Object *obj)
     cpu->gic_num_lrs = 4;
     cpu->gic_vpribits = 5;
     cpu->gic_vprebits = 5;
+    cpu->gic_pribits = 5;
     define_arm_cp_regs(cpu, cortex_a72_a57_a53_cp_reginfo);
 }
 
@@ -247,6 +249,7 @@ static void aarch64_a72_initfn(Object *obj)
     cpu->gic_num_lrs = 4;
     cpu->gic_vpribits = 5;
     cpu->gic_vprebits = 5;
+    cpu->gic_pribits = 5;
     define_arm_cp_regs(cpu, cortex_a72_a57_a53_cp_reginfo);
 }
 
@@ -961,6 +964,12 @@ static void aarch64_a64fx_initfn(Object *obj)
     cpu->gic_num_lrs = 4;
     cpu->gic_vpribits = 5;
     cpu->gic_vprebits = 5;
+    /*
+     * TODO: What does the real A64FX GICv3 provide ?
+     * This is a guess based on what other Arm CPUs do; to find the correct
+     * answer we need the value of the A64FX's ICC_CTLR_EL1 register.
+     */
+    cpu->gic_pribits = 5;
 
     /* Suppport of A64FX's vector length are 128,256 and 512bit only */
     aarch64_add_sve_properties(obj);
-- 
2.25.1

[PATCH 5/5] hw/intc/arm_gicv3: Provide ich_num_aprs()

[Peter Maydell]

We previously open-coded the expression for the number of virtual APR
registers and the assertion that it was not going to cause us to
overflow the cs->ich_apr[] array.  Factor this out into a new
ich_num_aprs() function, for consistency with the icc_num_aprs()
function we just added for the physical APR handling.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 hw/intc/arm_gicv3_cpuif.c | 18 ++++++++++--------
 1 file changed, 10 insertions(+), 8 deletions(-)

diff --git a/hw/intc/arm_gicv3_cpuif.c b/hw/intc/arm_gicv3_cpuif.c
index e277a807bd5..5418ad9bbc5 100644
--- a/hw/intc/arm_gicv3_cpuif.c
+++ b/hw/intc/arm_gicv3_cpuif.c
@@ -49,6 +49,14 @@ static inline int icv_min_vbpr(GICv3CPUState *cs)
     return 7 - cs->vprebits;
 }
 
+static inline int ich_num_aprs(GICv3CPUState *cs)
+{
+    /* Return the number of virtual APR registers (1, 2, or 4) */
+    int aprmax = 1 << (cs->vprebits - 5);
+    assert(aprmax <= ARRAY_SIZE(cs->ich_apr[0]));
+    return aprmax;
+}
+
 /* Simple accessor functions for LR fields */
 static uint32_t ich_lr_vintid(uint64_t lr)
 {
@@ -145,11 +153,8 @@ static int ich_highest_active_virt_prio(GICv3CPUState *cs)
      * in the ICH Active Priority Registers.
      */
     int i;
-    int aprmax = 1 << (cs->vprebits - 5);
 
-    assert(aprmax <= ARRAY_SIZE(cs->ich_apr[0]));
-
-    for (i = 0; i < aprmax; i++) {
+    for (i = 0; i < ich_num_aprs(cs); i++) {
         uint32_t apr = cs->ich_apr[GICV3_G0][i] |
             cs->ich_apr[GICV3_G1NS][i];
 
@@ -1333,11 +1338,8 @@ static int icv_drop_prio(GICv3CPUState *cs)
      * 32 bits are actually relevant.
      */
     int i;
-    int aprmax = 1 << (cs->vprebits - 5);
 
-    assert(aprmax <= ARRAY_SIZE(cs->ich_apr[0]));
-
-    for (i = 0; i < aprmax; i++) {
+    for (i = 0; i < ich_num_aprs(cs); i++) {
         uint64_t *papr0 = &cs->ich_apr[GICV3_G0][i];
         uint64_t *papr1 = &cs->ich_apr[GICV3_G1NS][i];
         int apr0count, apr1count;
-- 
2.25.1

Re: [PATCH 4/5] hw/intc/arm_gicv3: Use correct number of priority bits for the CPU

[Peter Maydell]

On Fri, 6 May 2022 at 17:21, Peter Maydell <peter.maydell@linaro.org> wrote:
>
> Make the GICv3 set its number of bits of physical priority from the
> implementation-specific value provided in the CPU state struct, in
> the same way we already do for virtual priority bits.  Because this
> would be a migration compatibility break, we provide a property
> force-8-bit-prio which is enabled for 7.0 and earlier versioned board
> models to retain the legacy "always use 8 bits" behaviour.
>
> Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
> ---
> I have guessed at the right value for the A64FX, but if we can
> find the correct ICC_CTLR_EL1 value that would be better.

Shuuichirou, Itaru: do either of you know the right setting for
the A64FX for this? If you can find what the hardware value of
the ICC_CTLR_EL3 or ICC_CTLR_EL1 register is (more specifically,
the PRIBits subfield) that should be enough to tell us.

> @@ -961,6 +964,12 @@ static void aarch64_a64fx_initfn(Object *obj)
>      cpu->gic_num_lrs = 4;
>      cpu->gic_vpribits = 5;
>      cpu->gic_vprebits = 5;
> +    /*
> +     * TODO: What does the real A64FX GICv3 provide ?
> +     * This is a guess based on what other Arm CPUs do; to find the correct
> +     * answer we need the value of the A64FX's ICC_CTLR_EL1 register.
> +     */
> +    cpu->gic_pribits = 5;
>
>      /* Suppport of A64FX's vector length are 128,256 and 512bit only */
>      aarch64_add_sve_properties(obj);
> --
> 2.25.1

thanks
-- PMM

Re: [PATCH 4/5] hw/intc/arm_gicv3: Use correct number of priority bits for the CPU

[Itaru Kitayama]

[-- Attachment #1: Type: text/plain, Size: 1807 bytes --]

Peter,
I’ll talk with Shuichiro this coming Monday (here most of us on vacation),
and get back to you.

Itaru.

On Sat, May 7, 2022 at 1:34 Peter Maydell <peter.maydell@linaro.org> wrote:

> On Fri, 6 May 2022 at 17:21, Peter Maydell <peter.maydell@linaro.org>
> wrote:
> >
> > Make the GICv3 set its number of bits of physical priority from the
> > implementation-specific value provided in the CPU state struct, in
> > the same way we already do for virtual priority bits.  Because this
> > would be a migration compatibility break, we provide a property
> > force-8-bit-prio which is enabled for 7.0 and earlier versioned board
> > models to retain the legacy "always use 8 bits" behaviour.
> >
> > Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
> > ---
> > I have guessed at the right value for the A64FX, but if we can
> > find the correct ICC_CTLR_EL1 value that would be better.
>
> Shuuichirou, Itaru: do either of you know the right setting for
> the A64FX for this? If you can find what the hardware value of
> the ICC_CTLR_EL3 or ICC_CTLR_EL1 register is (more specifically,
> the PRIBits subfield) that should be enough to tell us.
>
> > @@ -961,6 +964,12 @@ static void aarch64_a64fx_initfn(Object *obj)
> >      cpu->gic_num_lrs = 4;
> >      cpu->gic_vpribits = 5;
> >      cpu->gic_vprebits = 5;
> > +    /*
> > +     * TODO: What does the real A64FX GICv3 provide ?
> > +     * This is a guess based on what other Arm CPUs do; to find the
> correct
> > +     * answer we need the value of the A64FX's ICC_CTLR_EL1 register.
> > +     */
> > +    cpu->gic_pribits = 5;
> >
> >      /* Suppport of A64FX's vector length are 128,256 and 512bit only */
> >      aarch64_add_sve_properties(obj);
> > --
> > 2.25.1
>
> thanks
> -- PMM
>

[-- Attachment #2: Type: text/html, Size: 2490 bytes --]

Re: [PATCH 0/5] gicv3: Use right number of prio bits for the CPU

[Richard Henderson]

On 5/6/22 11:21, Peter Maydell wrote:
> This patchset fills in an odd inconsistency in our GICv3 emulation
> that I noticed while I was doing the GICv4 work. At the moment we
> allow the CPU to specify the number of bits of virtual priority
> (via the ARMCPU::gic_vpribits field), but we always use 8 bits of
> physical priority, even though to my knowledge no real Arm CPU
> hardware has that many.
> 
> This series makes the GICv3 emulation use a runtime-configurable
> number of physical priority bits, and sets it to match the number
> used by the various CPUs we implement (which is 5 for all the
> Cortex-Axx CPUs we emulate). Because changing the number of
> priority bits is a migration compatibility break, we use a compat
> property to keep the number of priority bits at 8 for older
> versions of the virt board.
> 
> There is one TODO left in this series, which is that I don't know
> the right value to use for the A64FX, so I've guessed that it
> is 5, like all the Arm implementations.
> 
> Patch 1 is an independent bugfix; patch 5 is cleanup.
> 
> thanks
> -- PMM
> 
> Peter Maydell (5):
>    hw/intc/arm_gicv3: report correct PRIbits field in ICV_CTLR_EL1
>    hw/intc/arm_gicv3_kvm.c: Stop using GIC_MIN_BPR constant
>    hw/intc/arm_gicv3: Support configurable number of physical priority
>      bits
>    hw/intc/arm_gicv3: Use correct number of priority bits for the CPU
>    hw/intc/arm_gicv3: Provide ich_num_aprs()
> 

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>

r~

Re: [PATCH 5/5] hw/intc/arm_gicv3: Provide ich_num_aprs()

[Richard Henderson]

On 5/6/22 11:21, Peter Maydell wrote:
> @@ -145,11 +153,8 @@ static int ich_highest_active_virt_prio(GICv3CPUState *cs)
>        * in the ICH Active Priority Registers.
>        */
>       int i;
> -    int aprmax = 1 << (cs->vprebits - 5);
>   
> -    assert(aprmax <= ARRAY_SIZE(cs->ich_apr[0]));
> -
> -    for (i = 0; i < aprmax; i++) {
> +    for (i = 0; i < ich_num_aprs(cs); i++) {

Better to retain the local variable for the loop bound.


r~

RE: [PATCH 4/5] hw/intc/arm_gicv3: Use correct number of priority bits for the CPU

[ishii.shuuichir]

Hi, Peter.

> Shuuichirou, Itaru: do either of you know the right setting for the A64FX for this? If
> you can find what the hardware value of the ICC_CTLR_EL3 or ICC_CTLR_EL1
> register is (more specifically, the PRIBits subfield) that should be enough to tell
> us.

The value of the PRIbits field in the A64FX is 0x4.
Therefore, the following values is fine.

> > +    cpu->gic_pribits = 5;

Best regards,
Shuuichirou.

P.S.
Itaru, thank you for the follow-up.

> -----Original Message-----
> From: Peter Maydell <peter.maydell@linaro.org>
> Sent: Saturday, May 7, 2022 1:34 AM
> To: qemu-arm@nongnu.org; qemu-devel@nongnu.org
> Cc: Ishii, Shuuichirou/石井 周一郎 <ishii.shuuichir@fujitsu.com>; Itaru Kitayama
> <itaru.kitayama@gmail.com>
> Subject: Re: [PATCH 4/5] hw/intc/arm_gicv3: Use correct number of priority bits
> for the CPU
> 
> On Fri, 6 May 2022 at 17:21, Peter Maydell <peter.maydell@linaro.org> wrote:
> >
> > Make the GICv3 set its number of bits of physical priority from the
> > implementation-specific value provided in the CPU state struct, in the
> > same way we already do for virtual priority bits.  Because this would
> > be a migration compatibility break, we provide a property
> > force-8-bit-prio which is enabled for 7.0 and earlier versioned board
> > models to retain the legacy "always use 8 bits" behaviour.
> >
> > Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
> > ---
> > I have guessed at the right value for the A64FX, but if we can find
> > the correct ICC_CTLR_EL1 value that would be better.
> 
> Shuuichirou, Itaru: do either of you know the right setting for the A64FX for this? If
> you can find what the hardware value of the ICC_CTLR_EL3 or ICC_CTLR_EL1
> register is (more specifically, the PRIBits subfield) that should be enough to tell
> us.
> 
> > @@ -961,6 +964,12 @@ static void aarch64_a64fx_initfn(Object *obj)
> >      cpu->gic_num_lrs = 4;
> >      cpu->gic_vpribits = 5;
> >      cpu->gic_vprebits = 5;
> > +    /*
> > +     * TODO: What does the real A64FX GICv3 provide ?
> > +     * This is a guess based on what other Arm CPUs do; to find the correct
> > +     * answer we need the value of the A64FX's ICC_CTLR_EL1 register.
> > +     */
> > +    cpu->gic_pribits = 5;
> >
> >      /* Suppport of A64FX's vector length are 128,256 and 512bit only */
> >      aarch64_add_sve_properties(obj);
> > --
> > 2.25.1
> 
> thanks
> -- PMM

Re: [PATCH 4/5] hw/intc/arm_gicv3: Use correct number of priority bits for the CPU

[Peter Maydell]

On Mon, 9 May 2022 at 23:55, ishii.shuuichir@fujitsu.com
<ishii.shuuichir@fujitsu.com> wrote:
>
> Hi, Peter.
>
> > Shuuichirou, Itaru: do either of you know the right setting for the A64FX for this? If
> > you can find what the hardware value of the ICC_CTLR_EL3 or ICC_CTLR_EL1
> > register is (more specifically, the PRIBits subfield) that should be enough to tell
> > us.
>
> The value of the PRIbits field in the A64FX is 0x4.
> Therefore, the following values is fine.
>
> > > +    cpu->gic_pribits = 5;

Great, thanks very much for confirming this.

-- PMM

Re: [PATCH 0/5] gicv3: Use right number of prio bits for the CPU

[Peter Maydell]

On Sat, 7 May 2022 at 12:35, Richard Henderson
<richard.henderson@linaro.org> wrote:
>
> On 5/6/22 11:21, Peter Maydell wrote:
> > This patchset fills in an odd inconsistency in our GICv3 emulation
> > that I noticed while I was doing the GICv4 work. At the moment we
> > allow the CPU to specify the number of bits of virtual priority
> > (via the ARMCPU::gic_vpribits field), but we always use 8 bits of
> > physical priority, even though to my knowledge no real Arm CPU
> > hardware has that many.
> >
> > This series makes the GICv3 emulation use a runtime-configurable
> > number of physical priority bits, and sets it to match the number
> > used by the various CPUs we implement (which is 5 for all the
> > Cortex-Axx CPUs we emulate). Because changing the number of
> > priority bits is a migration compatibility break, we use a compat
> > property to keep the number of priority bits at 8 for older
> > versions of the virt board.
> >
> > There is one TODO left in this series, which is that I don't know
> > the right value to use for the A64FX, so I've guessed that it
> > is 5, like all the Arm implementations.
> >
> > Patch 1 is an independent bugfix; patch 5 is cleanup.
> >
> > thanks
> > -- PMM
> >
> > Peter Maydell (5):
> >    hw/intc/arm_gicv3: report correct PRIbits field in ICV_CTLR_EL1
> >    hw/intc/arm_gicv3_kvm.c: Stop using GIC_MIN_BPR constant
> >    hw/intc/arm_gicv3: Support configurable number of physical priority
> >      bits
> >    hw/intc/arm_gicv3: Use correct number of priority bits for the CPU
> >    hw/intc/arm_gicv3: Provide ich_num_aprs()
> >
>
> Reviewed-by: Richard Henderson <richard.henderson@linaro.org>

Thanks; I've applied this to target-arm.next, with the 'TODO' note
for the A64FX removed, with the "retain local variable" tweak in
the last patch made, and with "cpu->gic_pribits = 5" statements added
for the new cortex-a76 and neoverse-n1 CPU types (confirmed correct
via their TRMs).

-- PMM