[PATCH v3 0/7] xen/arm: Emulate ID registers

[PATCH v3 0/7] xen/arm: Emulate ID registers

[Bertrand Marquis]

The goal of this serie is to emulate coprocessor ID registers so that
Xen only publish to guest features that are supported by Xen and can
actually be used by guests.
One practical example where this is required are SVE support which is
forbidden by Xen as it is not supported, but if Linux is compiled with
it, it will crash on boot. An other one is AMU which is also forbidden
by Xen but one Linux compiled with it would crash if the platform
supports it.

To be able to emulate the coprocessor registers defining what features
are supported by the hardware, the TID3 bit of HCR must be disabled and
Xen must emulated the values of those registers when an exception is
catched when a guest is accessing it.

This serie is first creating a guest cpuinfo structure which will
contain the values that we want to publish to the guests and then
provides the proper emulationg for those registers when Xen is getting
an exception due to an access to any of those registers.

This is a first simple implementation to solve the problem and the way
to define the values that we provide to guests and which features are
disabled will be in a future patchset enhance so that we could decide
per guest what can be used or not and depending on this deduce the bits
to activate in HCR and the values that we must publish on ID registers.

---
Changes in V2:
  Fix First patch to properly handle DFR1 register and increase dbg32
  size. Other patches have just been rebased.

Changes in V3:
  Add handling of reserved registers as RAZ
  Minor fixes described in each patch

Bertrand Marquis (7):
  xen/arm: Add ID registers and complete cpuinfo
  xen/arm: Add arm64 ID registers definitions
  xen/arm: create a cpuinfo structure for guest
  xen/arm: Add handler for ID registers on arm64
  xen/arm: Add handler for cp15 ID registers
  xen/arm: Add CP10 exception support to handle MVFR
  xen/arm: Activate TID3 in HCR_EL2

 xen/arch/arm/arm64/vsysreg.c        | 53 ++++++++++++++++++++
 xen/arch/arm/cpufeature.c           | 69 ++++++++++++++++++++++++++
 xen/arch/arm/traps.c                |  7 ++-
 xen/arch/arm/vcpreg.c               | 76 +++++++++++++++++++++++++++++
 xen/include/asm-arm/arm64/hsr.h     | 66 +++++++++++++++++++++++++
 xen/include/asm-arm/arm64/sysregs.h | 28 +++++++++++
 xen/include/asm-arm/cpregs.h        | 37 ++++++++++++++
 xen/include/asm-arm/cpufeature.h    | 58 ++++++++++++++++++----
 xen/include/asm-arm/perfc_defn.h    |  1 +
 xen/include/asm-arm/traps.h         |  1 +
 10 files changed, 386 insertions(+), 10 deletions(-)

-- 
2.17.1

[PATCH v3 1/7] xen/arm: Add ID registers and complete cpuinfo

[Bertrand Marquis]

Add definition and entries in cpuinfo for ID registers introduced in
newer Arm Architecture reference manual:
- ID_PFR2: processor feature register 2
- ID_DFR1: debug feature register 1
- ID_MMFR4 and ID_MMFR5: Memory model feature registers 4 and 5
- ID_ISA6: ISA Feature register 6
Add more bitfield definitions in PFR fields of cpuinfo.
Add MVFR2 register definition for aarch32.
Add mvfr values in cpuinfo.
Add some registers definition for arm64 in sysregs as some are not
always know by compilers.
Initialize the new values added in cpuinfo in identify_cpu during init.

Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>

---
Changes in V2:
  Fix dbg32 table size and add proper initialisation of the second entry
  of the table by reading ID_DFR1 register.
Changes in V3:
  Fix typo in commit title
  Add MVFR2 definition and handling on aarch32 and remove specific case
  for mvfr field in cpuinfo (now the same on arm64 and arm32).
  Add MMFR4 definition if not known by the compiler.

---
 xen/arch/arm/cpufeature.c           | 18 ++++++++++
 xen/include/asm-arm/arm64/sysregs.h | 28 +++++++++++++++
 xen/include/asm-arm/cpregs.h        | 12 +++++++
 xen/include/asm-arm/cpufeature.h    | 56 ++++++++++++++++++++++++-----
 4 files changed, 105 insertions(+), 9 deletions(-)

diff --git a/xen/arch/arm/cpufeature.c b/xen/arch/arm/cpufeature.c
index 44126dbf07..bc7ee5ac95 100644
--- a/xen/arch/arm/cpufeature.c
+++ b/xen/arch/arm/cpufeature.c
@@ -114,15 +114,20 @@ void identify_cpu(struct cpuinfo_arm *c)
 
         c->mm64.bits[0]  = READ_SYSREG64(ID_AA64MMFR0_EL1);
         c->mm64.bits[1]  = READ_SYSREG64(ID_AA64MMFR1_EL1);
+        c->mm64.bits[2]  = READ_SYSREG64(ID_AA64MMFR2_EL1);
 
         c->isa64.bits[0] = READ_SYSREG64(ID_AA64ISAR0_EL1);
         c->isa64.bits[1] = READ_SYSREG64(ID_AA64ISAR1_EL1);
+
+        c->zfr64.bits[0] = READ_SYSREG64(ID_AA64ZFR0_EL1);
 #endif
 
         c->pfr32.bits[0] = READ_SYSREG32(ID_PFR0_EL1);
         c->pfr32.bits[1] = READ_SYSREG32(ID_PFR1_EL1);
+        c->pfr32.bits[2] = READ_SYSREG32(ID_PFR2_EL1);
 
         c->dbg32.bits[0] = READ_SYSREG32(ID_DFR0_EL1);
+        c->dbg32.bits[1] = READ_SYSREG32(ID_DFR1_EL1);
 
         c->aux32.bits[0] = READ_SYSREG32(ID_AFR0_EL1);
 
@@ -130,6 +135,8 @@ void identify_cpu(struct cpuinfo_arm *c)
         c->mm32.bits[1]  = READ_SYSREG32(ID_MMFR1_EL1);
         c->mm32.bits[2]  = READ_SYSREG32(ID_MMFR2_EL1);
         c->mm32.bits[3]  = READ_SYSREG32(ID_MMFR3_EL1);
+        c->mm32.bits[4]  = READ_SYSREG32(ID_MMFR4_EL1);
+        c->mm32.bits[5]  = READ_SYSREG32(ID_MMFR5_EL1);
 
         c->isa32.bits[0] = READ_SYSREG32(ID_ISAR0_EL1);
         c->isa32.bits[1] = READ_SYSREG32(ID_ISAR1_EL1);
@@ -137,6 +144,17 @@ void identify_cpu(struct cpuinfo_arm *c)
         c->isa32.bits[3] = READ_SYSREG32(ID_ISAR3_EL1);
         c->isa32.bits[4] = READ_SYSREG32(ID_ISAR4_EL1);
         c->isa32.bits[5] = READ_SYSREG32(ID_ISAR5_EL1);
+        c->isa32.bits[6] = READ_SYSREG32(ID_ISAR6_EL1);
+
+#ifdef CONFIG_ARM_64
+        c->mvfr.bits[0] = READ_SYSREG64(MVFR0_EL1);
+        c->mvfr.bits[1] = READ_SYSREG64(MVFR1_EL1);
+        c->mvfr.bits[2] = READ_SYSREG64(MVFR2_EL1);
+#else
+        c->mvfr.bits[0] = READ_CP32(MVFR0);
+        c->mvfr.bits[1] = READ_CP32(MVFR1);
+        c->mvfr.bits[2] = READ_CP32(MVFR2);
+#endif
 }
 
 /*
diff --git a/xen/include/asm-arm/arm64/sysregs.h b/xen/include/asm-arm/arm64/sysregs.h
index c60029d38f..077fd95fb7 100644
--- a/xen/include/asm-arm/arm64/sysregs.h
+++ b/xen/include/asm-arm/arm64/sysregs.h
@@ -57,6 +57,34 @@
 #define ICH_AP1R2_EL2             __AP1Rx_EL2(2)
 #define ICH_AP1R3_EL2             __AP1Rx_EL2(3)
 
+/*
+ * Define ID coprocessor registers if they are not
+ * already defined by the compiler.
+ *
+ * Values picked from linux kernel
+ */
+#ifndef ID_AA64MMFR2_EL1
+#define ID_AA64MMFR2_EL1            S3_0_C0_C7_2
+#endif
+#ifndef ID_PFR2_EL1
+#define ID_PFR2_EL1                 S3_0_C0_C3_4
+#endif
+#ifndef ID_MMFR4_EL1
+#define ID_MMFR4_EL1                S3_0_C0_C2_6
+#endif
+#ifndef ID_MMFR5_EL1
+#define ID_MMFR5_EL1                S3_0_C0_C3_6
+#endif
+#ifndef ID_ISAR6_EL1
+#define ID_ISAR6_EL1                S3_0_C0_C2_7
+#endif
+#ifndef ID_AA64ZFR0_EL1
+#define ID_AA64ZFR0_EL1             S3_0_C0_C4_4
+#endif
+#ifndef ID_DFR1_EL1
+#define ID_DFR1_EL1                 S3_0_C0_C3_5
+#endif
+
 /* Access to system registers */
 
 #define READ_SYSREG32(name) ((uint32_t)READ_SYSREG64(name))
diff --git a/xen/include/asm-arm/cpregs.h b/xen/include/asm-arm/cpregs.h
index 8fd344146e..2690ddeb7a 100644
--- a/xen/include/asm-arm/cpregs.h
+++ b/xen/include/asm-arm/cpregs.h
@@ -63,6 +63,8 @@
 #define FPSID           p10,7,c0,c0,0   /* Floating-Point System ID Register */
 #define FPSCR           p10,7,c1,c0,0   /* Floating-Point Status and Control Register */
 #define MVFR0           p10,7,c7,c0,0   /* Media and VFP Feature Register 0 */
+#define MVFR1           p10,7,c6,c0,0   /* Media and VFP Feature Register 1 */
+#define MVFR2           p10,7,c5,c0,0   /* Media and VFP Feature Register 2 */
 #define FPEXC           p10,7,c8,c0,0   /* Floating-Point Exception Control Register */
 #define FPINST          p10,7,c9,c0,0   /* Floating-Point Instruction Register */
 #define FPINST2         p10,7,c10,c0,0  /* Floating-point Instruction Register 2 */
@@ -108,18 +110,23 @@
 #define MPIDR           p15,0,c0,c0,5   /* Multiprocessor Affinity Register */
 #define ID_PFR0         p15,0,c0,c1,0   /* Processor Feature Register 0 */
 #define ID_PFR1         p15,0,c0,c1,1   /* Processor Feature Register 1 */
+#define ID_PFR2         p15,0,c0,c3,4   /* Processor Feature Register 2 */
 #define ID_DFR0         p15,0,c0,c1,2   /* Debug Feature Register 0 */
+#define ID_DFR1         p15,0,c0,c3,5   /* Debug Feature Register 1 */
 #define ID_AFR0         p15,0,c0,c1,3   /* Auxiliary Feature Register 0 */
 #define ID_MMFR0        p15,0,c0,c1,4   /* Memory Model Feature Register 0 */
 #define ID_MMFR1        p15,0,c0,c1,5   /* Memory Model Feature Register 1 */
 #define ID_MMFR2        p15,0,c0,c1,6   /* Memory Model Feature Register 2 */
 #define ID_MMFR3        p15,0,c0,c1,7   /* Memory Model Feature Register 3 */
+#define ID_MMFR4        p15,0,c0,c2,6   /* Memory Model Feature Register 4 */
+#define ID_MMFR5        p15,0,c0,c3,6   /* Memory Model Feature Register 5 */
 #define ID_ISAR0        p15,0,c0,c2,0   /* ISA Feature Register 0 */
 #define ID_ISAR1        p15,0,c0,c2,1   /* ISA Feature Register 1 */
 #define ID_ISAR2        p15,0,c0,c2,2   /* ISA Feature Register 2 */
 #define ID_ISAR3        p15,0,c0,c2,3   /* ISA Feature Register 3 */
 #define ID_ISAR4        p15,0,c0,c2,4   /* ISA Feature Register 4 */
 #define ID_ISAR5        p15,0,c0,c2,5   /* ISA Feature Register 5 */
+#define ID_ISAR6        p15,0,c0,c2,7   /* ISA Feature Register 6 */
 #define CCSIDR          p15,1,c0,c0,0   /* Cache Size ID Registers */
 #define CLIDR           p15,1,c0,c0,1   /* Cache Level ID Register */
 #define CSSELR          p15,2,c0,c0,0   /* Cache Size Selection Register */
@@ -312,18 +319,23 @@
 #define HSTR_EL2                HSTR
 #define ID_AFR0_EL1             ID_AFR0
 #define ID_DFR0_EL1             ID_DFR0
+#define ID_DFR1_EL1             ID_DFR1
 #define ID_ISAR0_EL1            ID_ISAR0
 #define ID_ISAR1_EL1            ID_ISAR1
 #define ID_ISAR2_EL1            ID_ISAR2
 #define ID_ISAR3_EL1            ID_ISAR3
 #define ID_ISAR4_EL1            ID_ISAR4
 #define ID_ISAR5_EL1            ID_ISAR5
+#define ID_ISAR6_EL1            ID_ISAR6
 #define ID_MMFR0_EL1            ID_MMFR0
 #define ID_MMFR1_EL1            ID_MMFR1
 #define ID_MMFR2_EL1            ID_MMFR2
 #define ID_MMFR3_EL1            ID_MMFR3
+#define ID_MMFR4_EL1            ID_MMFR4
+#define ID_MMFR5_EL1            ID_MMFR5
 #define ID_PFR0_EL1             ID_PFR0
 #define ID_PFR1_EL1             ID_PFR1
+#define ID_PFR2_EL1             ID_PFR2
 #define IFSR32_EL2              IFSR
 #define MDCR_EL2                HDCR
 #define MIDR_EL1                MIDR
diff --git a/xen/include/asm-arm/cpufeature.h b/xen/include/asm-arm/cpufeature.h
index c7b5052992..6cf83d775b 100644
--- a/xen/include/asm-arm/cpufeature.h
+++ b/xen/include/asm-arm/cpufeature.h
@@ -148,6 +148,7 @@ struct cpuinfo_arm {
     union {
         uint64_t bits[2];
         struct {
+            /* PFR0 */
             unsigned long el0:4;
             unsigned long el1:4;
             unsigned long el2:4;
@@ -155,9 +156,23 @@ struct cpuinfo_arm {
             unsigned long fp:4;   /* Floating Point */
             unsigned long simd:4; /* Advanced SIMD */
             unsigned long gic:4;  /* GIC support */
-            unsigned long __res0:28;
+            unsigned long ras:4;
+            unsigned long sve:4;
+            unsigned long sel2:4;
+            unsigned long mpam:4;
+            unsigned long amu:4;
+            unsigned long dit:4;
+            unsigned long __res0:4;
             unsigned long csv2:4;
-            unsigned long __res1:4;
+            unsigned long cvs3:4;
+
+            /* PFR1 */
+            unsigned long bt:4;
+            unsigned long ssbs:4;
+            unsigned long mte:4;
+            unsigned long ras_frac:4;
+            unsigned long mpam_frac:4;
+            unsigned long __res1:44;
         };
     } pfr64;
 
@@ -170,7 +185,7 @@ struct cpuinfo_arm {
     } aux64;
 
     union {
-        uint64_t bits[2];
+        uint64_t bits[3];
         struct {
             unsigned long pa_range:4;
             unsigned long asid_bits:4;
@@ -190,6 +205,8 @@ struct cpuinfo_arm {
             unsigned long pan:4;
             unsigned long __res1:8;
             unsigned long __res2:32;
+
+            unsigned long __res3:64;
         };
     } mm64;
 
@@ -197,6 +214,10 @@ struct cpuinfo_arm {
         uint64_t bits[2];
     } isa64;
 
+    struct {
+        uint64_t bits[1];
+    } zfr64;
+
 #endif
 
     /*
@@ -204,25 +225,38 @@ struct cpuinfo_arm {
      * when running in 32-bit mode.
      */
     union {
-        uint32_t bits[2];
+        uint32_t bits[3];
         struct {
+            /* PFR0 */
             unsigned long arm:4;
             unsigned long thumb:4;
             unsigned long jazelle:4;
             unsigned long thumbee:4;
-            unsigned long __res0:16;
+            unsigned long csv2:4;
+            unsigned long amu:4;
+            unsigned long dit:4;
+            unsigned long ras:4;
 
+            /* PFR1 */
             unsigned long progmodel:4;
             unsigned long security:4;
             unsigned long mprofile:4;
             unsigned long virt:4;
             unsigned long gentimer:4;
-            unsigned long __res1:12;
+            unsigned long sec_frac:4;
+            unsigned long virt_frac:4;
+            unsigned long gic:4;
+
+            /* PFR2 */
+            unsigned long csv3:4;
+            unsigned long ssbs:4;
+            unsigned long ras_frac:4;
+            unsigned long __res2:20;
         };
     } pfr32;
 
     struct {
-        uint32_t bits[1];
+        uint32_t bits[2];
     } dbg32;
 
     struct {
@@ -230,12 +264,16 @@ struct cpuinfo_arm {
     } aux32;
 
     struct {
-        uint32_t bits[4];
+        uint32_t bits[6];
     } mm32;
 
     struct {
-        uint32_t bits[6];
+        uint32_t bits[7];
     } isa32;
+
+    struct {
+        uint64_t bits[3];
+    } mvfr;
 };
 
 extern struct cpuinfo_arm boot_cpu_data;
-- 
2.17.1

[PATCH v3 2/7] xen/arm: Add arm64 ID registers definitions

[Bertrand Marquis]

Add coprocessor registers definitions for all ID registers trapped
through the TID3 bit of HSR.
Those are the one that will be emulated in Xen to only publish to guests
the features that are supported by Xen and that are accessible to
guests.
Also define a case to catch all reserved registers that should be
handled as RAZ.

Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
---
Changes in V2: Rebase
Changes in V3:
  Add case definition for reserved registers.

---
 xen/include/asm-arm/arm64/hsr.h | 66 +++++++++++++++++++++++++++++++++
 1 file changed, 66 insertions(+)

diff --git a/xen/include/asm-arm/arm64/hsr.h b/xen/include/asm-arm/arm64/hsr.h
index ca931dd2fe..ffe0f0007e 100644
--- a/xen/include/asm-arm/arm64/hsr.h
+++ b/xen/include/asm-arm/arm64/hsr.h
@@ -110,6 +110,72 @@
 #define HSR_SYSREG_CNTP_CTL_EL0   HSR_SYSREG(3,3,c14,c2,1)
 #define HSR_SYSREG_CNTP_CVAL_EL0  HSR_SYSREG(3,3,c14,c2,2)
 
+/* Those registers are used when HCR_EL2.TID3 is set */
+#define HSR_SYSREG_ID_PFR0_EL1    HSR_SYSREG(3,0,c0,c1,0)
+#define HSR_SYSREG_ID_PFR1_EL1    HSR_SYSREG(3,0,c0,c1,1)
+#define HSR_SYSREG_ID_PFR2_EL1    HSR_SYSREG(3,0,c0,c3,4)
+#define HSR_SYSREG_ID_DFR0_EL1    HSR_SYSREG(3,0,c0,c1,2)
+#define HSR_SYSREG_ID_DFR1_EL1    HSR_SYSREG(3,0,c0,c3,5)
+#define HSR_SYSREG_ID_AFR0_EL1    HSR_SYSREG(3,0,c0,c1,3)
+#define HSR_SYSREG_ID_MMFR0_EL1   HSR_SYSREG(3,0,c0,c1,4)
+#define HSR_SYSREG_ID_MMFR1_EL1   HSR_SYSREG(3,0,c0,c1,5)
+#define HSR_SYSREG_ID_MMFR2_EL1   HSR_SYSREG(3,0,c0,c1,6)
+#define HSR_SYSREG_ID_MMFR3_EL1   HSR_SYSREG(3,0,c0,c1,7)
+#define HSR_SYSREG_ID_MMFR4_EL1   HSR_SYSREG(3,0,c0,c2,6)
+#define HSR_SYSREG_ID_MMFR5_EL1   HSR_SYSREG(3,0,c0,c3,6)
+#define HSR_SYSREG_ID_ISAR0_EL1   HSR_SYSREG(3,0,c0,c2,0)
+#define HSR_SYSREG_ID_ISAR1_EL1   HSR_SYSREG(3,0,c0,c2,1)
+#define HSR_SYSREG_ID_ISAR2_EL1   HSR_SYSREG(3,0,c0,c2,2)
+#define HSR_SYSREG_ID_ISAR3_EL1   HSR_SYSREG(3,0,c0,c2,3)
+#define HSR_SYSREG_ID_ISAR4_EL1   HSR_SYSREG(3,0,c0,c2,4)
+#define HSR_SYSREG_ID_ISAR5_EL1   HSR_SYSREG(3,0,c0,c2,5)
+#define HSR_SYSREG_ID_ISAR6_EL1   HSR_SYSREG(3,0,c0,c2,7)
+#define HSR_SYSREG_MVFR0_EL1      HSR_SYSREG(3,0,c0,c3,0)
+#define HSR_SYSREG_MVFR1_EL1      HSR_SYSREG(3,0,c0,c3,1)
+#define HSR_SYSREG_MVFR2_EL1      HSR_SYSREG(3,0,c0,c3,2)
+
+#define HSR_SYSREG_ID_AA64PFR0_EL1   HSR_SYSREG(3,0,c0,c4,0)
+#define HSR_SYSREG_ID_AA64PFR1_EL1   HSR_SYSREG(3,0,c0,c4,1)
+#define HSR_SYSREG_ID_AA64DFR0_EL1   HSR_SYSREG(3,0,c0,c5,0)
+#define HSR_SYSREG_ID_AA64DFR1_EL1   HSR_SYSREG(3,0,c0,c5,1)
+#define HSR_SYSREG_ID_AA64ISAR0_EL1  HSR_SYSREG(3,0,c0,c6,0)
+#define HSR_SYSREG_ID_AA64ISAR1_EL1  HSR_SYSREG(3,0,c0,c6,1)
+#define HSR_SYSREG_ID_AA64MMFR0_EL1  HSR_SYSREG(3,0,c0,c7,0)
+#define HSR_SYSREG_ID_AA64MMFR1_EL1  HSR_SYSREG(3,0,c0,c7,1)
+#define HSR_SYSREG_ID_AA64MMFR2_EL1  HSR_SYSREG(3,0,c0,c7,2)
+#define HSR_SYSREG_ID_AA64AFR0_EL1   HSR_SYSREG(3,0,c0,c5,4)
+#define HSR_SYSREG_ID_AA64AFR1_EL1   HSR_SYSREG(3,0,c0,c5,5)
+#define HSR_SYSREG_ID_AA64ZFR0_EL1   HSR_SYSREG(3,0,c0,c4,4)
+
+/*
+ * Those cases are catching all Reserved registers trapped by TID3 which
+ * currently have no assignment.
+ * HCR.TID3 is trapping all registers in the group 3:
+ * Op0 == 3, op1 == 0, CRn == c0,CRm == {c1-c7}, op2 == {0-7}.
+ */
+#define HSR_SYSREG_TID3_RESERVED_CASE  case HSR_SYSREG(3,0,c0,c3,3): \
+                                       case HSR_SYSREG(3,0,c0,c3,7): \
+                                       case HSR_SYSREG(3,0,c0,c4,2): \
+                                       case HSR_SYSREG(3,0,c0,c4,3): \
+                                       case HSR_SYSREG(3,0,c0,c4,5): \
+                                       case HSR_SYSREG(3,0,c0,c4,6): \
+                                       case HSR_SYSREG(3,0,c0,c4,7): \
+                                       case HSR_SYSREG(3,0,c0,c5,2): \
+                                       case HSR_SYSREG(3,0,c0,c5,3): \
+                                       case HSR_SYSREG(3,0,c0,c5,6): \
+                                       case HSR_SYSREG(3,0,c0,c5,7): \
+                                       case HSR_SYSREG(3,0,c0,c6,2): \
+                                       case HSR_SYSREG(3,0,c0,c6,3): \
+                                       case HSR_SYSREG(3,0,c0,c6,4): \
+                                       case HSR_SYSREG(3,0,c0,c6,5): \
+                                       case HSR_SYSREG(3,0,c0,c6,6): \
+                                       case HSR_SYSREG(3,0,c0,c6,7): \
+                                       case HSR_SYSREG(3,0,c0,c7,3): \
+                                       case HSR_SYSREG(3,0,c0,c7,4): \
+                                       case HSR_SYSREG(3,0,c0,c7,5): \
+                                       case HSR_SYSREG(3,0,c0,c7,6): \
+                                       case HSR_SYSREG(3,0,c0,c7,7)
+
 #endif /* __ASM_ARM_ARM64_HSR_H */
 
 /*
-- 
2.17.1

[PATCH v3 3/7] xen/arm: create a cpuinfo structure for guest

[Bertrand Marquis]

Create a cpuinfo structure for guest and mask into it the features that
we do not support in Xen or that we do not want to publish to guests.

Modify some values in the cpuinfo structure for guests to mask some
features which we do not want to allow to guests (like AMU) or we do not
support (like SVE).

The code is trying to group together registers modifications for the
same feature to be able in the long term to easily enable/disable a
feature depending on user parameters or add other registers modification
in the same place (like enabling/disabling HCR bits).

Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
---
Changes in V2: Rebase
Changes in V3:
  Use current_cpu_data info instead of recalling identify_cpu

---
 xen/arch/arm/cpufeature.c        | 51 ++++++++++++++++++++++++++++++++
 xen/include/asm-arm/cpufeature.h |  2 ++
 2 files changed, 53 insertions(+)

diff --git a/xen/arch/arm/cpufeature.c b/xen/arch/arm/cpufeature.c
index bc7ee5ac95..7255383504 100644
--- a/xen/arch/arm/cpufeature.c
+++ b/xen/arch/arm/cpufeature.c
@@ -24,6 +24,8 @@
 
 DECLARE_BITMAP(cpu_hwcaps, ARM_NCAPS);
 
+struct cpuinfo_arm __read_mostly guest_cpuinfo;
+
 void update_cpu_capabilities(const struct arm_cpu_capabilities *caps,
                              const char *info)
 {
@@ -157,6 +159,55 @@ void identify_cpu(struct cpuinfo_arm *c)
 #endif
 }
 
+/*
+ * This function is creating a cpuinfo structure with values modified to mask
+ * all cpu features that should not be published to guest.
+ * The created structure is then used to provide ID registers values to guests.
+ */
+static int __init create_guest_cpuinfo(void)
+{
+    /*
+     * TODO: The code is currently using only the features detected on the boot
+     * core. In the long term we should try to compute values containing only
+     * features supported by all cores.
+     */
+    guest_cpuinfo = current_cpu_data;
+
+#ifdef CONFIG_ARM_64
+    /* Disable MPAM as xen does not support it */
+    guest_cpuinfo.pfr64.mpam = 0;
+    guest_cpuinfo.pfr64.mpam_frac = 0;
+
+    /* Disable SVE as Xen does not support it */
+    guest_cpuinfo.pfr64.sve = 0;
+    guest_cpuinfo.zfr64.bits[0] = 0;
+
+    /* Disable MTE as Xen does not support it */
+    guest_cpuinfo.pfr64.mte = 0;
+#endif
+
+    /* Disable AMU */
+#ifdef CONFIG_ARM_64
+    guest_cpuinfo.pfr64.amu = 0;
+#endif
+    guest_cpuinfo.pfr32.amu = 0;
+
+    /* Disable RAS as Xen does not support it */
+#ifdef CONFIG_ARM_64
+    guest_cpuinfo.pfr64.ras = 0;
+    guest_cpuinfo.pfr64.ras_frac = 0;
+#endif
+    guest_cpuinfo.pfr32.ras = 0;
+    guest_cpuinfo.pfr32.ras_frac = 0;
+
+    return 0;
+}
+/*
+ * This function needs to be run after all smp are started to have
+ * cpuinfo structures for all cores.
+ */
+__initcall(create_guest_cpuinfo);
+
 /*
  * Local variables:
  * mode: C
diff --git a/xen/include/asm-arm/cpufeature.h b/xen/include/asm-arm/cpufeature.h
index 6cf83d775b..10b62bd324 100644
--- a/xen/include/asm-arm/cpufeature.h
+++ b/xen/include/asm-arm/cpufeature.h
@@ -283,6 +283,8 @@ extern void identify_cpu(struct cpuinfo_arm *);
 extern struct cpuinfo_arm cpu_data[];
 #define current_cpu_data cpu_data[smp_processor_id()]
 
+extern struct cpuinfo_arm guest_cpuinfo;
+
 #endif /* __ASSEMBLY__ */
 
 #endif
-- 
2.17.1

[PATCH v3 4/7] xen/arm: Add handler for ID registers on arm64

[Bertrand Marquis]

Add vsysreg emulation for registers trapped when TID3 bit is activated
in HSR.
The emulation is returning the value stored in cpuinfo_guest structure
for know registers and is handling reserved registers as RAZ.

Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
---
Changes in V2: Rebase
Changes in V3:
  Fix commit message
  Fix code style for GENERATE_TID3_INFO declaration
  Add handling of reserved registers as RAZ.

---
 xen/arch/arm/arm64/vsysreg.c | 53 ++++++++++++++++++++++++++++++++++++
 1 file changed, 53 insertions(+)

diff --git a/xen/arch/arm/arm64/vsysreg.c b/xen/arch/arm/arm64/vsysreg.c
index 8a85507d9d..ef7a11dbdd 100644
--- a/xen/arch/arm/arm64/vsysreg.c
+++ b/xen/arch/arm/arm64/vsysreg.c
@@ -69,6 +69,14 @@ TVM_REG(CONTEXTIDR_EL1)
         break;                                                          \
     }
 
+/* Macro to generate easily case for ID co-processor emulation */
+#define GENERATE_TID3_INFO(reg, field, offset)                          \
+    case HSR_SYSREG_##reg:                                              \
+    {                                                                   \
+        return handle_ro_read_val(regs, regidx, hsr.sysreg.read, hsr,   \
+                          1, guest_cpuinfo.field.bits[offset]);         \
+    }
+
 void do_sysreg(struct cpu_user_regs *regs,
                const union hsr hsr)
 {
@@ -259,6 +267,51 @@ void do_sysreg(struct cpu_user_regs *regs,
          */
         return handle_raz_wi(regs, regidx, hsr.sysreg.read, hsr, 1);
 
+    /*
+     * HCR_EL2.TID3
+     *
+     * This is trapping most Identification registers used by a guest
+     * to identify the processor features
+     */
+    GENERATE_TID3_INFO(ID_PFR0_EL1, pfr32, 0)
+    GENERATE_TID3_INFO(ID_PFR1_EL1, pfr32, 1)
+    GENERATE_TID3_INFO(ID_PFR2_EL1, pfr32, 2)
+    GENERATE_TID3_INFO(ID_DFR0_EL1, dbg32, 0)
+    GENERATE_TID3_INFO(ID_DFR1_EL1, dbg32, 1)
+    GENERATE_TID3_INFO(ID_AFR0_EL1, aux32, 0)
+    GENERATE_TID3_INFO(ID_MMFR0_EL1, mm32, 0)
+    GENERATE_TID3_INFO(ID_MMFR1_EL1, mm32, 1)
+    GENERATE_TID3_INFO(ID_MMFR2_EL1, mm32, 2)
+    GENERATE_TID3_INFO(ID_MMFR3_EL1, mm32, 3)
+    GENERATE_TID3_INFO(ID_MMFR4_EL1, mm32, 4)
+    GENERATE_TID3_INFO(ID_MMFR5_EL1, mm32, 5)
+    GENERATE_TID3_INFO(ID_ISAR0_EL1, isa32, 0)
+    GENERATE_TID3_INFO(ID_ISAR1_EL1, isa32, 1)
+    GENERATE_TID3_INFO(ID_ISAR2_EL1, isa32, 2)
+    GENERATE_TID3_INFO(ID_ISAR3_EL1, isa32, 3)
+    GENERATE_TID3_INFO(ID_ISAR4_EL1, isa32, 4)
+    GENERATE_TID3_INFO(ID_ISAR5_EL1, isa32, 5)
+    GENERATE_TID3_INFO(ID_ISAR6_EL1, isa32, 6)
+    GENERATE_TID3_INFO(MVFR0_EL1, mvfr, 0)
+    GENERATE_TID3_INFO(MVFR1_EL1, mvfr, 1)
+    GENERATE_TID3_INFO(MVFR2_EL1, mvfr, 2)
+    GENERATE_TID3_INFO(ID_AA64PFR0_EL1, pfr64, 0)
+    GENERATE_TID3_INFO(ID_AA64PFR1_EL1, pfr64, 1)
+    GENERATE_TID3_INFO(ID_AA64DFR0_EL1, dbg64, 0)
+    GENERATE_TID3_INFO(ID_AA64DFR1_EL1, dbg64, 1)
+    GENERATE_TID3_INFO(ID_AA64ISAR0_EL1, isa64, 0)
+    GENERATE_TID3_INFO(ID_AA64ISAR1_EL1, isa64, 1)
+    GENERATE_TID3_INFO(ID_AA64MMFR0_EL1, mm64, 0)
+    GENERATE_TID3_INFO(ID_AA64MMFR1_EL1, mm64, 1)
+    GENERATE_TID3_INFO(ID_AA64MMFR2_EL1, mm64, 2)
+    GENERATE_TID3_INFO(ID_AA64AFR0_EL1, aux64, 0)
+    GENERATE_TID3_INFO(ID_AA64AFR1_EL1, aux64, 1)
+    GENERATE_TID3_INFO(ID_AA64ZFR0_EL1, zfr64, 0)
+
+    HSR_SYSREG_TID3_RESERVED_CASE:
+        /* Handle all reserved registers as RAZ */
+        return handle_ro_raz(regs, regidx, hsr.sysreg.read, hsr, 1);
+
     /*
      * HCR_EL2.TIDCP
      *
-- 
2.17.1

[PATCH v3 5/7] xen/arm: Add handler for cp15 ID registers

[Bertrand Marquis]

Add support for emulation of cp15 based ID registers (on arm32 or when
running a 32bit guest on arm64).
The handlers are returning the values stored in the guest_cpuinfo
structure for known registers and RAZ for all reserved registers.
In the current status the MVFR registers are no supported.

Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
---
Changes in V2: Rebase
Changes in V3:
  Add case definition for reserved registers
  Add handling of reserved registers as RAZ.
  Fix code style in GENERATE_TID3_INFO declaration

---
 xen/arch/arm/vcpreg.c        | 39 ++++++++++++++++++++++++++++++++++++
 xen/include/asm-arm/cpregs.h | 25 +++++++++++++++++++++++
 2 files changed, 64 insertions(+)

diff --git a/xen/arch/arm/vcpreg.c b/xen/arch/arm/vcpreg.c
index cdc91cdf5b..d371a1c38c 100644
--- a/xen/arch/arm/vcpreg.c
+++ b/xen/arch/arm/vcpreg.c
@@ -155,6 +155,14 @@ TVM_REG32(CONTEXTIDR, CONTEXTIDR_EL1)
         break;                                                      \
     }
 
+/* Macro to generate easily case for ID co-processor emulation */
+#define GENERATE_TID3_INFO(reg, field, offset)                      \
+    case HSR_CPREG32(reg):                                          \
+    {                                                               \
+        return handle_ro_read_val(regs, regidx, cp32.read, hsr,     \
+                          1, guest_cpuinfo.field.bits[offset]);     \
+    }
+
 void do_cp15_32(struct cpu_user_regs *regs, const union hsr hsr)
 {
     const struct hsr_cp32 cp32 = hsr.cp32;
@@ -286,6 +294,37 @@ void do_cp15_32(struct cpu_user_regs *regs, const union hsr hsr)
          */
         return handle_raz_wi(regs, regidx, cp32.read, hsr, 1);
 
+    /*
+     * HCR_EL2.TID3
+     *
+     * This is trapping most Identification registers used by a guest
+     * to identify the processor features
+     */
+    GENERATE_TID3_INFO(ID_PFR0, pfr32, 0)
+    GENERATE_TID3_INFO(ID_PFR1, pfr32, 1)
+    GENERATE_TID3_INFO(ID_PFR2, pfr32, 2)
+    GENERATE_TID3_INFO(ID_DFR0, dbg32, 0)
+    GENERATE_TID3_INFO(ID_DFR1, dbg32, 1)
+    GENERATE_TID3_INFO(ID_AFR0, aux32, 0)
+    GENERATE_TID3_INFO(ID_MMFR0, mm32, 0)
+    GENERATE_TID3_INFO(ID_MMFR1, mm32, 1)
+    GENERATE_TID3_INFO(ID_MMFR2, mm32, 2)
+    GENERATE_TID3_INFO(ID_MMFR3, mm32, 3)
+    GENERATE_TID3_INFO(ID_MMFR4, mm32, 4)
+    GENERATE_TID3_INFO(ID_MMFR5, mm32, 5)
+    GENERATE_TID3_INFO(ID_ISAR0, isa32, 0)
+    GENERATE_TID3_INFO(ID_ISAR1, isa32, 1)
+    GENERATE_TID3_INFO(ID_ISAR2, isa32, 2)
+    GENERATE_TID3_INFO(ID_ISAR3, isa32, 3)
+    GENERATE_TID3_INFO(ID_ISAR4, isa32, 4)
+    GENERATE_TID3_INFO(ID_ISAR5, isa32, 5)
+    GENERATE_TID3_INFO(ID_ISAR6, isa32, 6)
+    /* MVFR registers are in cp10 no cp15 */
+
+    HSR_CPREG32_TID3_RESERVED_CASE:
+        /* Handle all reserved registers as RAZ */
+        return handle_ro_raz(regs, regidx, cp32.read, hsr, 1);
+
     /*
      * HCR_EL2.TIDCP
      *
diff --git a/xen/include/asm-arm/cpregs.h b/xen/include/asm-arm/cpregs.h
index 2690ddeb7a..5cb1ad5cbe 100644
--- a/xen/include/asm-arm/cpregs.h
+++ b/xen/include/asm-arm/cpregs.h
@@ -133,6 +133,31 @@
 #define VPIDR           p15,4,c0,c0,0   /* Virtualization Processor ID Register */
 #define VMPIDR          p15,4,c0,c0,5   /* Virtualization Multiprocessor ID Register */
 
+/*
+ * Those cases are catching all Reserved registers trapped by TID3 which
+ * currently have no assignment.
+ * HCR.TID3 is trapping all registers in the group 3:
+ * coproc == p15, opc1 == 0, CRn == c0, CRm == {c2-c7}, opc2 == {0-7}.
+ */
+#define HSR_CPREG32_TID3_CASES(REG)     case HSR_CPREG32(p15,0,c0,REG,0): \
+                                        case HSR_CPREG32(p15,0,c0,REG,1): \
+                                        case HSR_CPREG32(p15,0,c0,REG,2): \
+                                        case HSR_CPREG32(p15,0,c0,REG,3): \
+                                        case HSR_CPREG32(p15,0,c0,REG,4): \
+                                        case HSR_CPREG32(p15,0,c0,REG,5): \
+                                        case HSR_CPREG32(p15,0,c0,REG,6): \
+                                        case HSR_CPREG32(p15,0,c0,REG,7)
+
+#define HSR_CPREG32_TID3_RESERVED_CASE  case HSR_CPREG32(p15,0,c0,c3,0): \
+                                        case HSR_CPREG32(p15,0,c0,c3,1): \
+                                        case HSR_CPREG32(p15,0,c0,c3,2): \
+                                        case HSR_CPREG32(p15,0,c0,c3,3): \
+                                        case HSR_CPREG32(p15,0,c0,c3,7): \
+                                        HSR_CPREG32_TID3_CASES(c4): \
+                                        HSR_CPREG32_TID3_CASES(c5): \
+                                        HSR_CPREG32_TID3_CASES(c6): \
+                                        HSR_CPREG32_TID3_CASES(c7)
+
 /* CP15 CR1: System Control Registers */
 #define SCTLR           p15,0,c1,c0,0   /* System Control Register */
 #define ACTLR           p15,0,c1,c0,1   /* Auxiliary Control Register */
-- 
2.17.1

[PATCH v3 6/7] xen/arm: Add CP10 exception support to handle MVFR

[Bertrand Marquis]

Add support for cp10 exceptions decoding to be able to emulate the
values for MVFR0, MVFR1 and MVFR2 when TID3 bit of HSR is activated.
This is required for aarch32 guests accessing MVFR registers using
vmrs and vmsr instructions.

Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
---
Changes in V2: Rebase
Changes in V3:
  Add case for MVFR2, fix typo VMFR <-> MVFR.

---
 xen/arch/arm/traps.c             |  5 ++++
 xen/arch/arm/vcpreg.c            | 39 +++++++++++++++++++++++++++++++-
 xen/include/asm-arm/perfc_defn.h |  1 +
 xen/include/asm-arm/traps.h      |  1 +
 4 files changed, 45 insertions(+), 1 deletion(-)

diff --git a/xen/arch/arm/traps.c b/xen/arch/arm/traps.c
index 22bd1bd4c6..28d9d64558 100644
--- a/xen/arch/arm/traps.c
+++ b/xen/arch/arm/traps.c
@@ -2097,6 +2097,11 @@ void do_trap_guest_sync(struct cpu_user_regs *regs)
         perfc_incr(trap_cp14_dbg);
         do_cp14_dbg(regs, hsr);
         break;
+    case HSR_EC_CP10:
+        GUEST_BUG_ON(!psr_mode_is_32bit(regs));
+        perfc_incr(trap_cp10);
+        do_cp10(regs, hsr);
+        break;
     case HSR_EC_CP:
         GUEST_BUG_ON(!psr_mode_is_32bit(regs));
         perfc_incr(trap_cp);
diff --git a/xen/arch/arm/vcpreg.c b/xen/arch/arm/vcpreg.c
index d371a1c38c..da4e22a467 100644
--- a/xen/arch/arm/vcpreg.c
+++ b/xen/arch/arm/vcpreg.c
@@ -319,7 +319,7 @@ void do_cp15_32(struct cpu_user_regs *regs, const union hsr hsr)
     GENERATE_TID3_INFO(ID_ISAR4, isa32, 4)
     GENERATE_TID3_INFO(ID_ISAR5, isa32, 5)
     GENERATE_TID3_INFO(ID_ISAR6, isa32, 6)
-    /* MVFR registers are in cp10 no cp15 */
+    /* MVFR registers are in cp10 not cp15 */
 
     HSR_CPREG32_TID3_RESERVED_CASE:
         /* Handle all reserved registers as RAZ */
@@ -638,6 +638,43 @@ void do_cp14_dbg(struct cpu_user_regs *regs, const union hsr hsr)
     inject_undef_exception(regs, hsr);
 }
 
+void do_cp10(struct cpu_user_regs *regs, const union hsr hsr)
+{
+    const struct hsr_cp32 cp32 = hsr.cp32;
+    int regidx = cp32.reg;
+
+    if ( !check_conditional_instr(regs, hsr) )
+    {
+        advance_pc(regs, hsr);
+        return;
+    }
+
+    switch ( hsr.bits & HSR_CP32_REGS_MASK )
+    {
+    /*
+     * HSR.TID3 is trapping access to MVFR register used to identify the
+     * VFP/Simd using VMRS/VMSR instructions.
+     * Exception encoding is using MRC/MCR standard with the reg field in Crn
+     * as are declared MVFR0 and MVFR1 in cpregs.h
+     */
+    GENERATE_TID3_INFO(MVFR0, mvfr, 0)
+    GENERATE_TID3_INFO(MVFR1, mvfr, 1)
+    GENERATE_TID3_INFO(MVFR2, mvfr, 2)
+
+    default:
+        gdprintk(XENLOG_ERR,
+                 "%s p10, %d, r%d, cr%d, cr%d, %d @ 0x%"PRIregister"\n",
+                 cp32.read ? "mrc" : "mcr",
+                 cp32.op1, cp32.reg, cp32.crn, cp32.crm, cp32.op2, regs->pc);
+        gdprintk(XENLOG_ERR, "unhandled 32-bit CP10 access %#x\n",
+                 hsr.bits & HSR_CP32_REGS_MASK);
+        inject_undef_exception(regs, hsr);
+        return;
+    }
+
+    advance_pc(regs, hsr);
+}
+
 void do_cp(struct cpu_user_regs *regs, const union hsr hsr)
 {
     const struct hsr_cp cp = hsr.cp;
diff --git a/xen/include/asm-arm/perfc_defn.h b/xen/include/asm-arm/perfc_defn.h
index 6a83185163..31f071222b 100644
--- a/xen/include/asm-arm/perfc_defn.h
+++ b/xen/include/asm-arm/perfc_defn.h
@@ -11,6 +11,7 @@ PERFCOUNTER(trap_cp15_64,  "trap: cp15 64-bit access")
 PERFCOUNTER(trap_cp14_32,  "trap: cp14 32-bit access")
 PERFCOUNTER(trap_cp14_64,  "trap: cp14 64-bit access")
 PERFCOUNTER(trap_cp14_dbg, "trap: cp14 dbg access")
+PERFCOUNTER(trap_cp10,     "trap: cp10 access")
 PERFCOUNTER(trap_cp,       "trap: cp access")
 PERFCOUNTER(trap_smc32,    "trap: 32-bit smc")
 PERFCOUNTER(trap_hvc32,    "trap: 32-bit hvc")
diff --git a/xen/include/asm-arm/traps.h b/xen/include/asm-arm/traps.h
index 997c37884e..c4a3d0fb1b 100644
--- a/xen/include/asm-arm/traps.h
+++ b/xen/include/asm-arm/traps.h
@@ -62,6 +62,7 @@ void do_cp15_64(struct cpu_user_regs *regs, const union hsr hsr);
 void do_cp14_32(struct cpu_user_regs *regs, const union hsr hsr);
 void do_cp14_64(struct cpu_user_regs *regs, const union hsr hsr);
 void do_cp14_dbg(struct cpu_user_regs *regs, const union hsr hsr);
+void do_cp10(struct cpu_user_regs *regs, const union hsr hsr);
 void do_cp(struct cpu_user_regs *regs, const union hsr hsr);
 
 /* SMCCC handling */
-- 
2.17.1

[PATCH v3 7/7] xen/arm: Activate TID3 in HCR_EL2

[Bertrand Marquis]

Activate TID3 bit in HSR register when starting a guest.
This will trap all coprecessor ID registers so that we can give to guest
values corresponding to what they can actually use and mask some
features to guests even though they would be supported by the underlying
hardware (like SVE or MPAM).

Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
---
Changes in V2: Rebase
Changes in V3: Rebase

---
 xen/arch/arm/traps.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/xen/arch/arm/traps.c b/xen/arch/arm/traps.c
index 28d9d64558..c1a9ad6056 100644
--- a/xen/arch/arm/traps.c
+++ b/xen/arch/arm/traps.c
@@ -98,7 +98,7 @@ register_t get_default_hcr_flags(void)
 {
     return  (HCR_PTW|HCR_BSU_INNER|HCR_AMO|HCR_IMO|HCR_FMO|HCR_VM|
              (vwfi != NATIVE ? (HCR_TWI|HCR_TWE) : 0) |
-             HCR_TSC|HCR_TAC|HCR_SWIO|HCR_TIDCP|HCR_FB|HCR_TSW);
+             HCR_TID3|HCR_TSC|HCR_TAC|HCR_SWIO|HCR_TIDCP|HCR_FB|HCR_TSW);
 }
 
 static enum {
-- 
2.17.1

Re: [PATCH v3 4/7] xen/arm: Add handler for ID registers on arm64

[Stefano Stabellini]

On Wed, 9 Dec 2020, Bertrand Marquis wrote:
> Add vsysreg emulation for registers trapped when TID3 bit is activated
> in HSR.
> The emulation is returning the value stored in cpuinfo_guest structure
> for know registers and is handling reserved registers as RAZ.
> 
> Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
> ---
> Changes in V2: Rebase
> Changes in V3:
>   Fix commit message
>   Fix code style for GENERATE_TID3_INFO declaration
>   Add handling of reserved registers as RAZ.
> 
> ---
>  xen/arch/arm/arm64/vsysreg.c | 53 ++++++++++++++++++++++++++++++++++++
>  1 file changed, 53 insertions(+)
> 
> diff --git a/xen/arch/arm/arm64/vsysreg.c b/xen/arch/arm/arm64/vsysreg.c
> index 8a85507d9d..ef7a11dbdd 100644
> --- a/xen/arch/arm/arm64/vsysreg.c
> +++ b/xen/arch/arm/arm64/vsysreg.c
> @@ -69,6 +69,14 @@ TVM_REG(CONTEXTIDR_EL1)
>          break;                                                          \
>      }
>  
> +/* Macro to generate easily case for ID co-processor emulation */
> +#define GENERATE_TID3_INFO(reg, field, offset)                          \
> +    case HSR_SYSREG_##reg:                                              \
> +    {                                                                   \
> +        return handle_ro_read_val(regs, regidx, hsr.sysreg.read, hsr,   \
> +                          1, guest_cpuinfo.field.bits[offset]);         \

[...]

> +    HSR_SYSREG_TID3_RESERVED_CASE:
> +        /* Handle all reserved registers as RAZ */
> +        return handle_ro_raz(regs, regidx, hsr.sysreg.read, hsr, 1);


We are implementing both the known and the implementation defined
registers as read-as-zero. On write, we inject an exception.

However, reading the manual, it looks like the implementation defined
registers should be read-as-zero/write-ignore, is that right?

I couldn't easily find in the manual if it is OK to inject an exception
on write to a known register.

Re: [PATCH v3 5/7] xen/arm: Add handler for cp15 ID registers

[Stefano Stabellini]

On Wed, 9 Dec 2020, Bertrand Marquis wrote:
> Add support for emulation of cp15 based ID registers (on arm32 or when
> running a 32bit guest on arm64).
> The handlers are returning the values stored in the guest_cpuinfo
> structure for known registers and RAZ for all reserved registers.
> In the current status the MVFR registers are no supported.
> 
> Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
> ---
> Changes in V2: Rebase
> Changes in V3:
>   Add case definition for reserved registers
>   Add handling of reserved registers as RAZ.
>   Fix code style in GENERATE_TID3_INFO declaration
> 
> ---
>  xen/arch/arm/vcpreg.c        | 39 ++++++++++++++++++++++++++++++++++++
>  xen/include/asm-arm/cpregs.h | 25 +++++++++++++++++++++++
>  2 files changed, 64 insertions(+)
> 
> diff --git a/xen/arch/arm/vcpreg.c b/xen/arch/arm/vcpreg.c
> index cdc91cdf5b..d371a1c38c 100644
> --- a/xen/arch/arm/vcpreg.c
> +++ b/xen/arch/arm/vcpreg.c
> @@ -155,6 +155,14 @@ TVM_REG32(CONTEXTIDR, CONTEXTIDR_EL1)
>          break;                                                      \
>      }
>  
> +/* Macro to generate easily case for ID co-processor emulation */
> +#define GENERATE_TID3_INFO(reg, field, offset)                      \
> +    case HSR_CPREG32(reg):                                          \
> +    {                                                               \
> +        return handle_ro_read_val(regs, regidx, cp32.read, hsr,     \
> +                          1, guest_cpuinfo.field.bits[offset]);     \
> +    }
> +
>  void do_cp15_32(struct cpu_user_regs *regs, const union hsr hsr)
>  {
>      const struct hsr_cp32 cp32 = hsr.cp32;
> @@ -286,6 +294,37 @@ void do_cp15_32(struct cpu_user_regs *regs, const union hsr hsr)
>           */
>          return handle_raz_wi(regs, regidx, cp32.read, hsr, 1);
>  
> +    /*
> +     * HCR_EL2.TID3
> +     *
> +     * This is trapping most Identification registers used by a guest
> +     * to identify the processor features
> +     */
> +    GENERATE_TID3_INFO(ID_PFR0, pfr32, 0)
> +    GENERATE_TID3_INFO(ID_PFR1, pfr32, 1)
> +    GENERATE_TID3_INFO(ID_PFR2, pfr32, 2)
> +    GENERATE_TID3_INFO(ID_DFR0, dbg32, 0)
> +    GENERATE_TID3_INFO(ID_DFR1, dbg32, 1)
> +    GENERATE_TID3_INFO(ID_AFR0, aux32, 0)
> +    GENERATE_TID3_INFO(ID_MMFR0, mm32, 0)
> +    GENERATE_TID3_INFO(ID_MMFR1, mm32, 1)
> +    GENERATE_TID3_INFO(ID_MMFR2, mm32, 2)
> +    GENERATE_TID3_INFO(ID_MMFR3, mm32, 3)
> +    GENERATE_TID3_INFO(ID_MMFR4, mm32, 4)
> +    GENERATE_TID3_INFO(ID_MMFR5, mm32, 5)
> +    GENERATE_TID3_INFO(ID_ISAR0, isa32, 0)
> +    GENERATE_TID3_INFO(ID_ISAR1, isa32, 1)
> +    GENERATE_TID3_INFO(ID_ISAR2, isa32, 2)
> +    GENERATE_TID3_INFO(ID_ISAR3, isa32, 3)
> +    GENERATE_TID3_INFO(ID_ISAR4, isa32, 4)
> +    GENERATE_TID3_INFO(ID_ISAR5, isa32, 5)
> +    GENERATE_TID3_INFO(ID_ISAR6, isa32, 6)
> +    /* MVFR registers are in cp10 no cp15 */
> +
> +    HSR_CPREG32_TID3_RESERVED_CASE:
> +        /* Handle all reserved registers as RAZ */
> +        return handle_ro_raz(regs, regidx, cp32.read, hsr, 1);

Same question as for the aarch64 case: do we need to do write-ignore
for the reserved registers?


>      /*
>       * HCR_EL2.TIDCP
>       *
> diff --git a/xen/include/asm-arm/cpregs.h b/xen/include/asm-arm/cpregs.h
> index 2690ddeb7a..5cb1ad5cbe 100644
> --- a/xen/include/asm-arm/cpregs.h
> +++ b/xen/include/asm-arm/cpregs.h
> @@ -133,6 +133,31 @@
>  #define VPIDR           p15,4,c0,c0,0   /* Virtualization Processor ID Register */
>  #define VMPIDR          p15,4,c0,c0,5   /* Virtualization Multiprocessor ID Register */
>  
> +/*
> + * Those cases are catching all Reserved registers trapped by TID3 which
> + * currently have no assignment.
> + * HCR.TID3 is trapping all registers in the group 3:
> + * coproc == p15, opc1 == 0, CRn == c0, CRm == {c2-c7}, opc2 == {0-7}.
> + */
> +#define HSR_CPREG32_TID3_CASES(REG)     case HSR_CPREG32(p15,0,c0,REG,0): \
> +                                        case HSR_CPREG32(p15,0,c0,REG,1): \
> +                                        case HSR_CPREG32(p15,0,c0,REG,2): \
> +                                        case HSR_CPREG32(p15,0,c0,REG,3): \
> +                                        case HSR_CPREG32(p15,0,c0,REG,4): \
> +                                        case HSR_CPREG32(p15,0,c0,REG,5): \
> +                                        case HSR_CPREG32(p15,0,c0,REG,6): \
> +                                        case HSR_CPREG32(p15,0,c0,REG,7)
> +
> +#define HSR_CPREG32_TID3_RESERVED_CASE  case HSR_CPREG32(p15,0,c0,c3,0): \
> +                                        case HSR_CPREG32(p15,0,c0,c3,1): \
> +                                        case HSR_CPREG32(p15,0,c0,c3,2): \
> +                                        case HSR_CPREG32(p15,0,c0,c3,3): \
> +                                        case HSR_CPREG32(p15,0,c0,c3,7): \
> +                                        HSR_CPREG32_TID3_CASES(c4): \
> +                                        HSR_CPREG32_TID3_CASES(c5): \
> +                                        HSR_CPREG32_TID3_CASES(c6): \
> +                                        HSR_CPREG32_TID3_CASES(c7)

The following are missing, is it a problem?

p15,0,c0,c0,2
p15,0,c0,c0,3
p15,0,c0,c0,4
p15,0,c0,c0,6
p15,0,c0,c0,7

Re: [PATCH v3 6/7] xen/arm: Add CP10 exception support to handle MVFR

[Stefano Stabellini]

On Wed, 9 Dec 2020, Bertrand Marquis wrote:
> Add support for cp10 exceptions decoding to be able to emulate the
> values for MVFR0, MVFR1 and MVFR2 when TID3 bit of HSR is activated.
> This is required for aarch32 guests accessing MVFR registers using
> vmrs and vmsr instructions.
> 
> Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
> ---
> Changes in V2: Rebase
> Changes in V3:
>   Add case for MVFR2, fix typo VMFR <-> MVFR.
> 
> ---
>  xen/arch/arm/traps.c             |  5 ++++
>  xen/arch/arm/vcpreg.c            | 39 +++++++++++++++++++++++++++++++-
>  xen/include/asm-arm/perfc_defn.h |  1 +
>  xen/include/asm-arm/traps.h      |  1 +
>  4 files changed, 45 insertions(+), 1 deletion(-)
> 
> diff --git a/xen/arch/arm/traps.c b/xen/arch/arm/traps.c
> index 22bd1bd4c6..28d9d64558 100644
> --- a/xen/arch/arm/traps.c
> +++ b/xen/arch/arm/traps.c
> @@ -2097,6 +2097,11 @@ void do_trap_guest_sync(struct cpu_user_regs *regs)
>          perfc_incr(trap_cp14_dbg);
>          do_cp14_dbg(regs, hsr);
>          break;
> +    case HSR_EC_CP10:
> +        GUEST_BUG_ON(!psr_mode_is_32bit(regs));
> +        perfc_incr(trap_cp10);
> +        do_cp10(regs, hsr);
> +        break;
>      case HSR_EC_CP:
>          GUEST_BUG_ON(!psr_mode_is_32bit(regs));
>          perfc_incr(trap_cp);
> diff --git a/xen/arch/arm/vcpreg.c b/xen/arch/arm/vcpreg.c
> index d371a1c38c..da4e22a467 100644
> --- a/xen/arch/arm/vcpreg.c
> +++ b/xen/arch/arm/vcpreg.c
> @@ -319,7 +319,7 @@ void do_cp15_32(struct cpu_user_regs *regs, const union hsr hsr)
>      GENERATE_TID3_INFO(ID_ISAR4, isa32, 4)
>      GENERATE_TID3_INFO(ID_ISAR5, isa32, 5)
>      GENERATE_TID3_INFO(ID_ISAR6, isa32, 6)
> -    /* MVFR registers are in cp10 no cp15 */
> +    /* MVFR registers are in cp10 not cp15 */
>  
>      HSR_CPREG32_TID3_RESERVED_CASE:
>          /* Handle all reserved registers as RAZ */
> @@ -638,6 +638,43 @@ void do_cp14_dbg(struct cpu_user_regs *regs, const union hsr hsr)
>      inject_undef_exception(regs, hsr);
>  }
>  
> +void do_cp10(struct cpu_user_regs *regs, const union hsr hsr)
> +{
> +    const struct hsr_cp32 cp32 = hsr.cp32;
> +    int regidx = cp32.reg;
> +
> +    if ( !check_conditional_instr(regs, hsr) )
> +    {
> +        advance_pc(regs, hsr);
> +        return;
> +    }
> +
> +    switch ( hsr.bits & HSR_CP32_REGS_MASK )
> +    {
> +    /*
> +     * HSR.TID3 is trapping access to MVFR register used to identify the
          ^ HCR

> +     * VFP/Simd using VMRS/VMSR instructions.
> +     * Exception encoding is using MRC/MCR standard with the reg field in Crn
> +     * as are declared MVFR0 and MVFR1 in cpregs.h
> +     */
> +    GENERATE_TID3_INFO(MVFR0, mvfr, 0)
> +    GENERATE_TID3_INFO(MVFR1, mvfr, 1)
> +    GENERATE_TID3_INFO(MVFR2, mvfr, 2)
> +
> +    default:
> +        gdprintk(XENLOG_ERR,
> +                 "%s p10, %d, r%d, cr%d, cr%d, %d @ 0x%"PRIregister"\n",
> +                 cp32.read ? "mrc" : "mcr",
> +                 cp32.op1, cp32.reg, cp32.crn, cp32.crm, cp32.op2, regs->pc);
> +        gdprintk(XENLOG_ERR, "unhandled 32-bit CP10 access %#x\n",
> +                 hsr.bits & HSR_CP32_REGS_MASK);
> +        inject_undef_exception(regs, hsr);
> +        return;

I take we are sure there are no other cp10 registers of interest?


> +    }
> +
> +    advance_pc(regs, hsr);
> +}
> +
>  void do_cp(struct cpu_user_regs *regs, const union hsr hsr)
>  {
>      const struct hsr_cp cp = hsr.cp;
> diff --git a/xen/include/asm-arm/perfc_defn.h b/xen/include/asm-arm/perfc_defn.h
> index 6a83185163..31f071222b 100644
> --- a/xen/include/asm-arm/perfc_defn.h
> +++ b/xen/include/asm-arm/perfc_defn.h
> @@ -11,6 +11,7 @@ PERFCOUNTER(trap_cp15_64,  "trap: cp15 64-bit access")
>  PERFCOUNTER(trap_cp14_32,  "trap: cp14 32-bit access")
>  PERFCOUNTER(trap_cp14_64,  "trap: cp14 64-bit access")
>  PERFCOUNTER(trap_cp14_dbg, "trap: cp14 dbg access")
> +PERFCOUNTER(trap_cp10,     "trap: cp10 access")
>  PERFCOUNTER(trap_cp,       "trap: cp access")
>  PERFCOUNTER(trap_smc32,    "trap: 32-bit smc")
>  PERFCOUNTER(trap_hvc32,    "trap: 32-bit hvc")
> diff --git a/xen/include/asm-arm/traps.h b/xen/include/asm-arm/traps.h
> index 997c37884e..c4a3d0fb1b 100644
> --- a/xen/include/asm-arm/traps.h
> +++ b/xen/include/asm-arm/traps.h
> @@ -62,6 +62,7 @@ void do_cp15_64(struct cpu_user_regs *regs, const union hsr hsr);
>  void do_cp14_32(struct cpu_user_regs *regs, const union hsr hsr);
>  void do_cp14_64(struct cpu_user_regs *regs, const union hsr hsr);
>  void do_cp14_dbg(struct cpu_user_regs *regs, const union hsr hsr);
> +void do_cp10(struct cpu_user_regs *regs, const union hsr hsr);
>  void do_cp(struct cpu_user_regs *regs, const union hsr hsr);
>  
>  /* SMCCC handling */
> -- 
> 2.17.1
> 

Re: [PATCH v3 1/7] xen/arm: Add ID registers and complete cpuinfo

[Stefano Stabellini]

On Wed, 9 Dec 2020, Bertrand Marquis wrote:
> Add definition and entries in cpuinfo for ID registers introduced in
> newer Arm Architecture reference manual:
> - ID_PFR2: processor feature register 2
> - ID_DFR1: debug feature register 1
> - ID_MMFR4 and ID_MMFR5: Memory model feature registers 4 and 5
> - ID_ISA6: ISA Feature register 6
> Add more bitfield definitions in PFR fields of cpuinfo.
> Add MVFR2 register definition for aarch32.
> Add mvfr values in cpuinfo.
> Add some registers definition for arm64 in sysregs as some are not
> always know by compilers.
> Initialize the new values added in cpuinfo in identify_cpu during init.
> 
> Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
> Changes in V2:
>   Fix dbg32 table size and add proper initialisation of the second entry
>   of the table by reading ID_DFR1 register.
> Changes in V3:
>   Fix typo in commit title
>   Add MVFR2 definition and handling on aarch32 and remove specific case
>   for mvfr field in cpuinfo (now the same on arm64 and arm32).
>   Add MMFR4 definition if not known by the compiler.
> 
> ---
>  xen/arch/arm/cpufeature.c           | 18 ++++++++++
>  xen/include/asm-arm/arm64/sysregs.h | 28 +++++++++++++++
>  xen/include/asm-arm/cpregs.h        | 12 +++++++
>  xen/include/asm-arm/cpufeature.h    | 56 ++++++++++++++++++++++++-----
>  4 files changed, 105 insertions(+), 9 deletions(-)
> 
> diff --git a/xen/arch/arm/cpufeature.c b/xen/arch/arm/cpufeature.c
> index 44126dbf07..bc7ee5ac95 100644
> --- a/xen/arch/arm/cpufeature.c
> +++ b/xen/arch/arm/cpufeature.c
> @@ -114,15 +114,20 @@ void identify_cpu(struct cpuinfo_arm *c)
>  
>          c->mm64.bits[0]  = READ_SYSREG64(ID_AA64MMFR0_EL1);
>          c->mm64.bits[1]  = READ_SYSREG64(ID_AA64MMFR1_EL1);
> +        c->mm64.bits[2]  = READ_SYSREG64(ID_AA64MMFR2_EL1);
>  
>          c->isa64.bits[0] = READ_SYSREG64(ID_AA64ISAR0_EL1);
>          c->isa64.bits[1] = READ_SYSREG64(ID_AA64ISAR1_EL1);
> +
> +        c->zfr64.bits[0] = READ_SYSREG64(ID_AA64ZFR0_EL1);
>  #endif
>  
>          c->pfr32.bits[0] = READ_SYSREG32(ID_PFR0_EL1);
>          c->pfr32.bits[1] = READ_SYSREG32(ID_PFR1_EL1);
> +        c->pfr32.bits[2] = READ_SYSREG32(ID_PFR2_EL1);
>  
>          c->dbg32.bits[0] = READ_SYSREG32(ID_DFR0_EL1);
> +        c->dbg32.bits[1] = READ_SYSREG32(ID_DFR1_EL1);
>  
>          c->aux32.bits[0] = READ_SYSREG32(ID_AFR0_EL1);
>  
> @@ -130,6 +135,8 @@ void identify_cpu(struct cpuinfo_arm *c)
>          c->mm32.bits[1]  = READ_SYSREG32(ID_MMFR1_EL1);
>          c->mm32.bits[2]  = READ_SYSREG32(ID_MMFR2_EL1);
>          c->mm32.bits[3]  = READ_SYSREG32(ID_MMFR3_EL1);
> +        c->mm32.bits[4]  = READ_SYSREG32(ID_MMFR4_EL1);
> +        c->mm32.bits[5]  = READ_SYSREG32(ID_MMFR5_EL1);
>  
>          c->isa32.bits[0] = READ_SYSREG32(ID_ISAR0_EL1);
>          c->isa32.bits[1] = READ_SYSREG32(ID_ISAR1_EL1);
> @@ -137,6 +144,17 @@ void identify_cpu(struct cpuinfo_arm *c)
>          c->isa32.bits[3] = READ_SYSREG32(ID_ISAR3_EL1);
>          c->isa32.bits[4] = READ_SYSREG32(ID_ISAR4_EL1);
>          c->isa32.bits[5] = READ_SYSREG32(ID_ISAR5_EL1);
> +        c->isa32.bits[6] = READ_SYSREG32(ID_ISAR6_EL1);
> +
> +#ifdef CONFIG_ARM_64
> +        c->mvfr.bits[0] = READ_SYSREG64(MVFR0_EL1);
> +        c->mvfr.bits[1] = READ_SYSREG64(MVFR1_EL1);
> +        c->mvfr.bits[2] = READ_SYSREG64(MVFR2_EL1);
> +#else
> +        c->mvfr.bits[0] = READ_CP32(MVFR0);
> +        c->mvfr.bits[1] = READ_CP32(MVFR1);
> +        c->mvfr.bits[2] = READ_CP32(MVFR2);
> +#endif
>  }
>  
>  /*
> diff --git a/xen/include/asm-arm/arm64/sysregs.h b/xen/include/asm-arm/arm64/sysregs.h
> index c60029d38f..077fd95fb7 100644
> --- a/xen/include/asm-arm/arm64/sysregs.h
> +++ b/xen/include/asm-arm/arm64/sysregs.h
> @@ -57,6 +57,34 @@
>  #define ICH_AP1R2_EL2             __AP1Rx_EL2(2)
>  #define ICH_AP1R3_EL2             __AP1Rx_EL2(3)
>  
> +/*
> + * Define ID coprocessor registers if they are not
> + * already defined by the compiler.
> + *
> + * Values picked from linux kernel
> + */
> +#ifndef ID_AA64MMFR2_EL1
> +#define ID_AA64MMFR2_EL1            S3_0_C0_C7_2
> +#endif
> +#ifndef ID_PFR2_EL1
> +#define ID_PFR2_EL1                 S3_0_C0_C3_4
> +#endif
> +#ifndef ID_MMFR4_EL1
> +#define ID_MMFR4_EL1                S3_0_C0_C2_6
> +#endif
> +#ifndef ID_MMFR5_EL1
> +#define ID_MMFR5_EL1                S3_0_C0_C3_6
> +#endif
> +#ifndef ID_ISAR6_EL1
> +#define ID_ISAR6_EL1                S3_0_C0_C2_7
> +#endif
> +#ifndef ID_AA64ZFR0_EL1
> +#define ID_AA64ZFR0_EL1             S3_0_C0_C4_4
> +#endif
> +#ifndef ID_DFR1_EL1
> +#define ID_DFR1_EL1                 S3_0_C0_C3_5
> +#endif
> +
>  /* Access to system registers */
>  
>  #define READ_SYSREG32(name) ((uint32_t)READ_SYSREG64(name))
> diff --git a/xen/include/asm-arm/cpregs.h b/xen/include/asm-arm/cpregs.h
> index 8fd344146e..2690ddeb7a 100644
> --- a/xen/include/asm-arm/cpregs.h
> +++ b/xen/include/asm-arm/cpregs.h
> @@ -63,6 +63,8 @@
>  #define FPSID           p10,7,c0,c0,0   /* Floating-Point System ID Register */
>  #define FPSCR           p10,7,c1,c0,0   /* Floating-Point Status and Control Register */
>  #define MVFR0           p10,7,c7,c0,0   /* Media and VFP Feature Register 0 */
> +#define MVFR1           p10,7,c6,c0,0   /* Media and VFP Feature Register 1 */
> +#define MVFR2           p10,7,c5,c0,0   /* Media and VFP Feature Register 2 */
>  #define FPEXC           p10,7,c8,c0,0   /* Floating-Point Exception Control Register */
>  #define FPINST          p10,7,c9,c0,0   /* Floating-Point Instruction Register */
>  #define FPINST2         p10,7,c10,c0,0  /* Floating-point Instruction Register 2 */
> @@ -108,18 +110,23 @@
>  #define MPIDR           p15,0,c0,c0,5   /* Multiprocessor Affinity Register */
>  #define ID_PFR0         p15,0,c0,c1,0   /* Processor Feature Register 0 */
>  #define ID_PFR1         p15,0,c0,c1,1   /* Processor Feature Register 1 */
> +#define ID_PFR2         p15,0,c0,c3,4   /* Processor Feature Register 2 */
>  #define ID_DFR0         p15,0,c0,c1,2   /* Debug Feature Register 0 */
> +#define ID_DFR1         p15,0,c0,c3,5   /* Debug Feature Register 1 */
>  #define ID_AFR0         p15,0,c0,c1,3   /* Auxiliary Feature Register 0 */
>  #define ID_MMFR0        p15,0,c0,c1,4   /* Memory Model Feature Register 0 */
>  #define ID_MMFR1        p15,0,c0,c1,5   /* Memory Model Feature Register 1 */
>  #define ID_MMFR2        p15,0,c0,c1,6   /* Memory Model Feature Register 2 */
>  #define ID_MMFR3        p15,0,c0,c1,7   /* Memory Model Feature Register 3 */
> +#define ID_MMFR4        p15,0,c0,c2,6   /* Memory Model Feature Register 4 */
> +#define ID_MMFR5        p15,0,c0,c3,6   /* Memory Model Feature Register 5 */
>  #define ID_ISAR0        p15,0,c0,c2,0   /* ISA Feature Register 0 */
>  #define ID_ISAR1        p15,0,c0,c2,1   /* ISA Feature Register 1 */
>  #define ID_ISAR2        p15,0,c0,c2,2   /* ISA Feature Register 2 */
>  #define ID_ISAR3        p15,0,c0,c2,3   /* ISA Feature Register 3 */
>  #define ID_ISAR4        p15,0,c0,c2,4   /* ISA Feature Register 4 */
>  #define ID_ISAR5        p15,0,c0,c2,5   /* ISA Feature Register 5 */
> +#define ID_ISAR6        p15,0,c0,c2,7   /* ISA Feature Register 6 */
>  #define CCSIDR          p15,1,c0,c0,0   /* Cache Size ID Registers */
>  #define CLIDR           p15,1,c0,c0,1   /* Cache Level ID Register */
>  #define CSSELR          p15,2,c0,c0,0   /* Cache Size Selection Register */
> @@ -312,18 +319,23 @@
>  #define HSTR_EL2                HSTR
>  #define ID_AFR0_EL1             ID_AFR0
>  #define ID_DFR0_EL1             ID_DFR0
> +#define ID_DFR1_EL1             ID_DFR1
>  #define ID_ISAR0_EL1            ID_ISAR0
>  #define ID_ISAR1_EL1            ID_ISAR1
>  #define ID_ISAR2_EL1            ID_ISAR2
>  #define ID_ISAR3_EL1            ID_ISAR3
>  #define ID_ISAR4_EL1            ID_ISAR4
>  #define ID_ISAR5_EL1            ID_ISAR5
> +#define ID_ISAR6_EL1            ID_ISAR6
>  #define ID_MMFR0_EL1            ID_MMFR0
>  #define ID_MMFR1_EL1            ID_MMFR1
>  #define ID_MMFR2_EL1            ID_MMFR2
>  #define ID_MMFR3_EL1            ID_MMFR3
> +#define ID_MMFR4_EL1            ID_MMFR4
> +#define ID_MMFR5_EL1            ID_MMFR5
>  #define ID_PFR0_EL1             ID_PFR0
>  #define ID_PFR1_EL1             ID_PFR1
> +#define ID_PFR2_EL1             ID_PFR2
>  #define IFSR32_EL2              IFSR
>  #define MDCR_EL2                HDCR
>  #define MIDR_EL1                MIDR
> diff --git a/xen/include/asm-arm/cpufeature.h b/xen/include/asm-arm/cpufeature.h
> index c7b5052992..6cf83d775b 100644
> --- a/xen/include/asm-arm/cpufeature.h
> +++ b/xen/include/asm-arm/cpufeature.h
> @@ -148,6 +148,7 @@ struct cpuinfo_arm {
>      union {
>          uint64_t bits[2];
>          struct {
> +            /* PFR0 */
>              unsigned long el0:4;
>              unsigned long el1:4;
>              unsigned long el2:4;
> @@ -155,9 +156,23 @@ struct cpuinfo_arm {
>              unsigned long fp:4;   /* Floating Point */
>              unsigned long simd:4; /* Advanced SIMD */
>              unsigned long gic:4;  /* GIC support */
> -            unsigned long __res0:28;
> +            unsigned long ras:4;
> +            unsigned long sve:4;
> +            unsigned long sel2:4;
> +            unsigned long mpam:4;
> +            unsigned long amu:4;
> +            unsigned long dit:4;
> +            unsigned long __res0:4;
>              unsigned long csv2:4;
> -            unsigned long __res1:4;
> +            unsigned long cvs3:4;
> +
> +            /* PFR1 */
> +            unsigned long bt:4;
> +            unsigned long ssbs:4;
> +            unsigned long mte:4;
> +            unsigned long ras_frac:4;
> +            unsigned long mpam_frac:4;
> +            unsigned long __res1:44;
>          };
>      } pfr64;
>  
> @@ -170,7 +185,7 @@ struct cpuinfo_arm {
>      } aux64;
>  
>      union {
> -        uint64_t bits[2];
> +        uint64_t bits[3];
>          struct {
>              unsigned long pa_range:4;
>              unsigned long asid_bits:4;
> @@ -190,6 +205,8 @@ struct cpuinfo_arm {
>              unsigned long pan:4;
>              unsigned long __res1:8;
>              unsigned long __res2:32;
> +
> +            unsigned long __res3:64;
>          };
>      } mm64;
>  
> @@ -197,6 +214,10 @@ struct cpuinfo_arm {
>          uint64_t bits[2];
>      } isa64;
>  
> +    struct {
> +        uint64_t bits[1];
> +    } zfr64;
> +
>  #endif
>  
>      /*
> @@ -204,25 +225,38 @@ struct cpuinfo_arm {
>       * when running in 32-bit mode.
>       */
>      union {
> -        uint32_t bits[2];
> +        uint32_t bits[3];
>          struct {
> +            /* PFR0 */
>              unsigned long arm:4;
>              unsigned long thumb:4;
>              unsigned long jazelle:4;
>              unsigned long thumbee:4;
> -            unsigned long __res0:16;
> +            unsigned long csv2:4;
> +            unsigned long amu:4;
> +            unsigned long dit:4;
> +            unsigned long ras:4;
>  
> +            /* PFR1 */
>              unsigned long progmodel:4;
>              unsigned long security:4;
>              unsigned long mprofile:4;
>              unsigned long virt:4;
>              unsigned long gentimer:4;
> -            unsigned long __res1:12;
> +            unsigned long sec_frac:4;
> +            unsigned long virt_frac:4;
> +            unsigned long gic:4;
> +
> +            /* PFR2 */
> +            unsigned long csv3:4;
> +            unsigned long ssbs:4;
> +            unsigned long ras_frac:4;
> +            unsigned long __res2:20;
>          };
>      } pfr32;
>  
>      struct {
> -        uint32_t bits[1];
> +        uint32_t bits[2];
>      } dbg32;
>  
>      struct {
> @@ -230,12 +264,16 @@ struct cpuinfo_arm {
>      } aux32;
>  
>      struct {
> -        uint32_t bits[4];
> +        uint32_t bits[6];
>      } mm32;
>  
>      struct {
> -        uint32_t bits[6];
> +        uint32_t bits[7];
>      } isa32;
> +
> +    struct {
> +        uint64_t bits[3];
> +    } mvfr;
>  };
>  
>  extern struct cpuinfo_arm boot_cpu_data;
> -- 
> 2.17.1
> 

Re: [PATCH v3 2/7] xen/arm: Add arm64 ID registers definitions

[Stefano Stabellini]

On Wed, 9 Dec 2020, Bertrand Marquis wrote:
> Add coprocessor registers definitions for all ID registers trapped
> through the TID3 bit of HSR.
> Those are the one that will be emulated in Xen to only publish to guests
> the features that are supported by Xen and that are accessible to
> guests.
> Also define a case to catch all reserved registers that should be
> handled as RAZ.
> 
> Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
> Changes in V2: Rebase
> Changes in V3:
>   Add case definition for reserved registers.
> 
> ---
>  xen/include/asm-arm/arm64/hsr.h | 66 +++++++++++++++++++++++++++++++++
>  1 file changed, 66 insertions(+)
> 
> diff --git a/xen/include/asm-arm/arm64/hsr.h b/xen/include/asm-arm/arm64/hsr.h
> index ca931dd2fe..ffe0f0007e 100644
> --- a/xen/include/asm-arm/arm64/hsr.h
> +++ b/xen/include/asm-arm/arm64/hsr.h
> @@ -110,6 +110,72 @@
>  #define HSR_SYSREG_CNTP_CTL_EL0   HSR_SYSREG(3,3,c14,c2,1)
>  #define HSR_SYSREG_CNTP_CVAL_EL0  HSR_SYSREG(3,3,c14,c2,2)
>  
> +/* Those registers are used when HCR_EL2.TID3 is set */
> +#define HSR_SYSREG_ID_PFR0_EL1    HSR_SYSREG(3,0,c0,c1,0)
> +#define HSR_SYSREG_ID_PFR1_EL1    HSR_SYSREG(3,0,c0,c1,1)
> +#define HSR_SYSREG_ID_PFR2_EL1    HSR_SYSREG(3,0,c0,c3,4)
> +#define HSR_SYSREG_ID_DFR0_EL1    HSR_SYSREG(3,0,c0,c1,2)
> +#define HSR_SYSREG_ID_DFR1_EL1    HSR_SYSREG(3,0,c0,c3,5)
> +#define HSR_SYSREG_ID_AFR0_EL1    HSR_SYSREG(3,0,c0,c1,3)
> +#define HSR_SYSREG_ID_MMFR0_EL1   HSR_SYSREG(3,0,c0,c1,4)
> +#define HSR_SYSREG_ID_MMFR1_EL1   HSR_SYSREG(3,0,c0,c1,5)
> +#define HSR_SYSREG_ID_MMFR2_EL1   HSR_SYSREG(3,0,c0,c1,6)
> +#define HSR_SYSREG_ID_MMFR3_EL1   HSR_SYSREG(3,0,c0,c1,7)
> +#define HSR_SYSREG_ID_MMFR4_EL1   HSR_SYSREG(3,0,c0,c2,6)
> +#define HSR_SYSREG_ID_MMFR5_EL1   HSR_SYSREG(3,0,c0,c3,6)
> +#define HSR_SYSREG_ID_ISAR0_EL1   HSR_SYSREG(3,0,c0,c2,0)
> +#define HSR_SYSREG_ID_ISAR1_EL1   HSR_SYSREG(3,0,c0,c2,1)
> +#define HSR_SYSREG_ID_ISAR2_EL1   HSR_SYSREG(3,0,c0,c2,2)
> +#define HSR_SYSREG_ID_ISAR3_EL1   HSR_SYSREG(3,0,c0,c2,3)
> +#define HSR_SYSREG_ID_ISAR4_EL1   HSR_SYSREG(3,0,c0,c2,4)
> +#define HSR_SYSREG_ID_ISAR5_EL1   HSR_SYSREG(3,0,c0,c2,5)
> +#define HSR_SYSREG_ID_ISAR6_EL1   HSR_SYSREG(3,0,c0,c2,7)
> +#define HSR_SYSREG_MVFR0_EL1      HSR_SYSREG(3,0,c0,c3,0)
> +#define HSR_SYSREG_MVFR1_EL1      HSR_SYSREG(3,0,c0,c3,1)
> +#define HSR_SYSREG_MVFR2_EL1      HSR_SYSREG(3,0,c0,c3,2)
> +
> +#define HSR_SYSREG_ID_AA64PFR0_EL1   HSR_SYSREG(3,0,c0,c4,0)
> +#define HSR_SYSREG_ID_AA64PFR1_EL1   HSR_SYSREG(3,0,c0,c4,1)
> +#define HSR_SYSREG_ID_AA64DFR0_EL1   HSR_SYSREG(3,0,c0,c5,0)
> +#define HSR_SYSREG_ID_AA64DFR1_EL1   HSR_SYSREG(3,0,c0,c5,1)
> +#define HSR_SYSREG_ID_AA64ISAR0_EL1  HSR_SYSREG(3,0,c0,c6,0)
> +#define HSR_SYSREG_ID_AA64ISAR1_EL1  HSR_SYSREG(3,0,c0,c6,1)
> +#define HSR_SYSREG_ID_AA64MMFR0_EL1  HSR_SYSREG(3,0,c0,c7,0)
> +#define HSR_SYSREG_ID_AA64MMFR1_EL1  HSR_SYSREG(3,0,c0,c7,1)
> +#define HSR_SYSREG_ID_AA64MMFR2_EL1  HSR_SYSREG(3,0,c0,c7,2)
> +#define HSR_SYSREG_ID_AA64AFR0_EL1   HSR_SYSREG(3,0,c0,c5,4)
> +#define HSR_SYSREG_ID_AA64AFR1_EL1   HSR_SYSREG(3,0,c0,c5,5)
> +#define HSR_SYSREG_ID_AA64ZFR0_EL1   HSR_SYSREG(3,0,c0,c4,4)
> +
> +/*
> + * Those cases are catching all Reserved registers trapped by TID3 which
> + * currently have no assignment.
> + * HCR.TID3 is trapping all registers in the group 3:
> + * Op0 == 3, op1 == 0, CRn == c0,CRm == {c1-c7}, op2 == {0-7}.
> + */
> +#define HSR_SYSREG_TID3_RESERVED_CASE  case HSR_SYSREG(3,0,c0,c3,3): \
> +                                       case HSR_SYSREG(3,0,c0,c3,7): \
> +                                       case HSR_SYSREG(3,0,c0,c4,2): \
> +                                       case HSR_SYSREG(3,0,c0,c4,3): \
> +                                       case HSR_SYSREG(3,0,c0,c4,5): \
> +                                       case HSR_SYSREG(3,0,c0,c4,6): \
> +                                       case HSR_SYSREG(3,0,c0,c4,7): \
> +                                       case HSR_SYSREG(3,0,c0,c5,2): \
> +                                       case HSR_SYSREG(3,0,c0,c5,3): \
> +                                       case HSR_SYSREG(3,0,c0,c5,6): \
> +                                       case HSR_SYSREG(3,0,c0,c5,7): \
> +                                       case HSR_SYSREG(3,0,c0,c6,2): \
> +                                       case HSR_SYSREG(3,0,c0,c6,3): \
> +                                       case HSR_SYSREG(3,0,c0,c6,4): \
> +                                       case HSR_SYSREG(3,0,c0,c6,5): \
> +                                       case HSR_SYSREG(3,0,c0,c6,6): \
> +                                       case HSR_SYSREG(3,0,c0,c6,7): \
> +                                       case HSR_SYSREG(3,0,c0,c7,3): \
> +                                       case HSR_SYSREG(3,0,c0,c7,4): \
> +                                       case HSR_SYSREG(3,0,c0,c7,5): \
> +                                       case HSR_SYSREG(3,0,c0,c7,6): \
> +                                       case HSR_SYSREG(3,0,c0,c7,7)
> +
>  #endif /* __ASM_ARM_ARM64_HSR_H */
>  
>  /*
> -- 
> 2.17.1
> 

Re: [PATCH v3 3/7] xen/arm: create a cpuinfo structure for guest

[Stefano Stabellini]

On Wed, 9 Dec 2020, Bertrand Marquis wrote:
> Create a cpuinfo structure for guest and mask into it the features that
> we do not support in Xen or that we do not want to publish to guests.
> 
> Modify some values in the cpuinfo structure for guests to mask some
> features which we do not want to allow to guests (like AMU) or we do not
> support (like SVE).
> 
> The code is trying to group together registers modifications for the
> same feature to be able in the long term to easily enable/disable a
> feature depending on user parameters or add other registers modification
> in the same place (like enabling/disabling HCR bits).
> 
> Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
> Changes in V2: Rebase
> Changes in V3:
>   Use current_cpu_data info instead of recalling identify_cpu
> 
> ---
>  xen/arch/arm/cpufeature.c        | 51 ++++++++++++++++++++++++++++++++
>  xen/include/asm-arm/cpufeature.h |  2 ++
>  2 files changed, 53 insertions(+)
> 
> diff --git a/xen/arch/arm/cpufeature.c b/xen/arch/arm/cpufeature.c
> index bc7ee5ac95..7255383504 100644
> --- a/xen/arch/arm/cpufeature.c
> +++ b/xen/arch/arm/cpufeature.c
> @@ -24,6 +24,8 @@
>  
>  DECLARE_BITMAP(cpu_hwcaps, ARM_NCAPS);
>  
> +struct cpuinfo_arm __read_mostly guest_cpuinfo;
> +
>  void update_cpu_capabilities(const struct arm_cpu_capabilities *caps,
>                               const char *info)
>  {
> @@ -157,6 +159,55 @@ void identify_cpu(struct cpuinfo_arm *c)
>  #endif
>  }
>  
> +/*
> + * This function is creating a cpuinfo structure with values modified to mask
> + * all cpu features that should not be published to guest.
> + * The created structure is then used to provide ID registers values to guests.
> + */
> +static int __init create_guest_cpuinfo(void)
> +{
> +    /*
> +     * TODO: The code is currently using only the features detected on the boot
> +     * core. In the long term we should try to compute values containing only
> +     * features supported by all cores.
> +     */
> +    guest_cpuinfo = current_cpu_data;
> +
> +#ifdef CONFIG_ARM_64
> +    /* Disable MPAM as xen does not support it */
> +    guest_cpuinfo.pfr64.mpam = 0;
> +    guest_cpuinfo.pfr64.mpam_frac = 0;
> +
> +    /* Disable SVE as Xen does not support it */
> +    guest_cpuinfo.pfr64.sve = 0;
> +    guest_cpuinfo.zfr64.bits[0] = 0;
> +
> +    /* Disable MTE as Xen does not support it */
> +    guest_cpuinfo.pfr64.mte = 0;
> +#endif
> +
> +    /* Disable AMU */
> +#ifdef CONFIG_ARM_64
> +    guest_cpuinfo.pfr64.amu = 0;
> +#endif
> +    guest_cpuinfo.pfr32.amu = 0;
> +
> +    /* Disable RAS as Xen does not support it */
> +#ifdef CONFIG_ARM_64
> +    guest_cpuinfo.pfr64.ras = 0;
> +    guest_cpuinfo.pfr64.ras_frac = 0;
> +#endif
> +    guest_cpuinfo.pfr32.ras = 0;
> +    guest_cpuinfo.pfr32.ras_frac = 0;
> +
> +    return 0;
> +}
> +/*
> + * This function needs to be run after all smp are started to have
> + * cpuinfo structures for all cores.
> + */
> +__initcall(create_guest_cpuinfo);
> +
>  /*
>   * Local variables:
>   * mode: C
> diff --git a/xen/include/asm-arm/cpufeature.h b/xen/include/asm-arm/cpufeature.h
> index 6cf83d775b..10b62bd324 100644
> --- a/xen/include/asm-arm/cpufeature.h
> +++ b/xen/include/asm-arm/cpufeature.h
> @@ -283,6 +283,8 @@ extern void identify_cpu(struct cpuinfo_arm *);
>  extern struct cpuinfo_arm cpu_data[];
>  #define current_cpu_data cpu_data[smp_processor_id()]
>  
> +extern struct cpuinfo_arm guest_cpuinfo;
> +
>  #endif /* __ASSEMBLY__ */
>  
>  #endif
> -- 
> 2.17.1
> 

Re: [PATCH v3 7/7] xen/arm: Activate TID3 in HCR_EL2

[Stefano Stabellini]

On Wed, 9 Dec 2020, Bertrand Marquis wrote:
> Activate TID3 bit in HSR register when starting a guest.
> This will trap all coprecessor ID registers so that we can give to guest
> values corresponding to what they can actually use and mask some
> features to guests even though they would be supported by the underlying
> hardware (like SVE or MPAM).
> 
> Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
> Changes in V2: Rebase
> Changes in V3: Rebase
> 
> ---
>  xen/arch/arm/traps.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/xen/arch/arm/traps.c b/xen/arch/arm/traps.c
> index 28d9d64558..c1a9ad6056 100644
> --- a/xen/arch/arm/traps.c
> +++ b/xen/arch/arm/traps.c
> @@ -98,7 +98,7 @@ register_t get_default_hcr_flags(void)
>  {
>      return  (HCR_PTW|HCR_BSU_INNER|HCR_AMO|HCR_IMO|HCR_FMO|HCR_VM|
>               (vwfi != NATIVE ? (HCR_TWI|HCR_TWE) : 0) |
> -             HCR_TSC|HCR_TAC|HCR_SWIO|HCR_TIDCP|HCR_FB|HCR_TSW);
> +             HCR_TID3|HCR_TSC|HCR_TAC|HCR_SWIO|HCR_TIDCP|HCR_FB|HCR_TSW);
>  }
>  
>  static enum {
> -- 
> 2.17.1
> 

Re: [PATCH v3 1/7] xen/arm: Add ID registers and complete cpuinfo

[Julien Grall]

Hi Bertrand,

On 09/12/2020 16:30, Bertrand Marquis wrote:
> Add definition and entries in cpuinfo for ID registers introduced in
> newer Arm Architecture reference manual:
> - ID_PFR2: processor feature register 2
> - ID_DFR1: debug feature register 1
> - ID_MMFR4 and ID_MMFR5: Memory model feature registers 4 and 5
> - ID_ISA6: ISA Feature register 6
> Add more bitfield definitions in PFR fields of cpuinfo.
> Add MVFR2 register definition for aarch32.
> Add mvfr values in cpuinfo.
> Add some registers definition for arm64 in sysregs as some are not
> always know by compilers.
> Initialize the new values added in cpuinfo in identify_cpu during init.
> 
> Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
> 
> ---
> Changes in V2:
>    Fix dbg32 table size and add proper initialisation of the second entry
>    of the table by reading ID_DFR1 register.
> Changes in V3:
>    Fix typo in commit title
>    Add MVFR2 definition and handling on aarch32 and remove specific case
>    for mvfr field in cpuinfo (now the same on arm64 and arm32).
>    Add MMFR4 definition if not known by the compiler.
> 
> ---
>   xen/arch/arm/cpufeature.c           | 18 ++++++++++
>   xen/include/asm-arm/arm64/sysregs.h | 28 +++++++++++++++
>   xen/include/asm-arm/cpregs.h        | 12 +++++++
>   xen/include/asm-arm/cpufeature.h    | 56 ++++++++++++++++++++++++-----
>   4 files changed, 105 insertions(+), 9 deletions(-)
> 
> diff --git a/xen/arch/arm/cpufeature.c b/xen/arch/arm/cpufeature.c
> index 44126dbf07..bc7ee5ac95 100644
> --- a/xen/arch/arm/cpufeature.c
> +++ b/xen/arch/arm/cpufeature.c
> @@ -114,15 +114,20 @@ void identify_cpu(struct cpuinfo_arm *c)
>   
>           c->mm64.bits[0]  = READ_SYSREG64(ID_AA64MMFR0_EL1);
>           c->mm64.bits[1]  = READ_SYSREG64(ID_AA64MMFR1_EL1);
> +        c->mm64.bits[2]  = READ_SYSREG64(ID_AA64MMFR2_EL1);
>   
>           c->isa64.bits[0] = READ_SYSREG64(ID_AA64ISAR0_EL1);
>           c->isa64.bits[1] = READ_SYSREG64(ID_AA64ISAR1_EL1);
> +
> +        c->zfr64.bits[0] = READ_SYSREG64(ID_AA64ZFR0_EL1);
>   #endif
>   
>           c->pfr32.bits[0] = READ_SYSREG32(ID_PFR0_EL1);
>           c->pfr32.bits[1] = READ_SYSREG32(ID_PFR1_EL1);
> +        c->pfr32.bits[2] = READ_SYSREG32(ID_PFR2_EL1);
>   
>           c->dbg32.bits[0] = READ_SYSREG32(ID_DFR0_EL1);
> +        c->dbg32.bits[1] = READ_SYSREG32(ID_DFR1_EL1);
>   
>           c->aux32.bits[0] = READ_SYSREG32(ID_AFR0_EL1);
>   
> @@ -130,6 +135,8 @@ void identify_cpu(struct cpuinfo_arm *c)
>           c->mm32.bits[1]  = READ_SYSREG32(ID_MMFR1_EL1);
>           c->mm32.bits[2]  = READ_SYSREG32(ID_MMFR2_EL1);
>           c->mm32.bits[3]  = READ_SYSREG32(ID_MMFR3_EL1);
> +        c->mm32.bits[4]  = READ_SYSREG32(ID_MMFR4_EL1);
> +        c->mm32.bits[5]  = READ_SYSREG32(ID_MMFR5_EL1);

Please don't introduce any more use of READ_SYSREG32(), they are wrong 
on Armv8 because system registers are always 64-bit.

>   
>           c->isa32.bits[0] = READ_SYSREG32(ID_ISAR0_EL1);
>           c->isa32.bits[1] = READ_SYSREG32(ID_ISAR1_EL1);
> @@ -137,6 +144,17 @@ void identify_cpu(struct cpuinfo_arm *c)
>           c->isa32.bits[3] = READ_SYSREG32(ID_ISAR3_EL1);
>           c->isa32.bits[4] = READ_SYSREG32(ID_ISAR4_EL1);
>           c->isa32.bits[5] = READ_SYSREG32(ID_ISAR5_EL1);
> +        c->isa32.bits[6] = READ_SYSREG32(ID_ISAR6_EL1);
> +
> +#ifdef CONFIG_ARM_64
> +        c->mvfr.bits[0] = READ_SYSREG64(MVFR0_EL1);
> +        c->mvfr.bits[1] = READ_SYSREG64(MVFR1_EL1);
> +        c->mvfr.bits[2] = READ_SYSREG64(MVFR2_EL1);
> +#else
> +        c->mvfr.bits[0] = READ_CP32(MVFR0);
> +        c->mvfr.bits[1] = READ_CP32(MVFR1);
> +        c->mvfr.bits[2] = READ_CP32(MVFR2);
> +#endif

READ_SYSREG() will do the job to either use READ_SYSREG64() or 
READ_CP32() depending on the arch used.

>   }
>   
>   /*
> diff --git a/xen/include/asm-arm/arm64/sysregs.h b/xen/include/asm-arm/arm64/sysregs.h
> index c60029d38f..077fd95fb7 100644
> --- a/xen/include/asm-arm/arm64/sysregs.h
> +++ b/xen/include/asm-arm/arm64/sysregs.h
> @@ -57,6 +57,34 @@
>   #define ICH_AP1R2_EL2             __AP1Rx_EL2(2)
>   #define ICH_AP1R3_EL2             __AP1Rx_EL2(3)
>   
> +/*
> + * Define ID coprocessor registers if they are not
> + * already defined by the compiler.
> + *
> + * Values picked from linux kernel
> + */
> +#ifndef ID_AA64MMFR2_EL1

I am a bit puzzled how this meant to work. Will the libc/compiler 
headers define ID_AA64MMFR2_EL1?

> +#define ID_AA64MMFR2_EL1            S3_0_C0_C7_2
> +#endif
> +#ifndef ID_PFR2_EL1
> +#define ID_PFR2_EL1                 S3_0_C0_C3_4
> +#endif
> +#ifndef ID_MMFR4_EL1
> +#define ID_MMFR4_EL1                S3_0_C0_C2_6
> +#endif
> +#ifndef ID_MMFR5_EL1
> +#define ID_MMFR5_EL1                S3_0_C0_C3_6
> +#endif
> +#ifndef ID_ISAR6_EL1
> +#define ID_ISAR6_EL1                S3_0_C0_C2_7
> +#endif
> +#ifndef ID_AA64ZFR0_EL1
> +#define ID_AA64ZFR0_EL1             S3_0_C0_C4_4
> +#endif
> +#ifndef ID_DFR1_EL1
> +#define ID_DFR1_EL1                 S3_0_C0_C3_5
> +#endif
> +
>   /* Access to system registers */
>   
>   #define READ_SYSREG32(name) ((uint32_t)READ_SYSREG64(name))
> diff --git a/xen/include/asm-arm/cpregs.h b/xen/include/asm-arm/cpregs.h
> index 8fd344146e..2690ddeb7a 100644
> --- a/xen/include/asm-arm/cpregs.h
> +++ b/xen/include/asm-arm/cpregs.h
> @@ -63,6 +63,8 @@
>   #define FPSID           p10,7,c0,c0,0   /* Floating-Point System ID Register */
>   #define FPSCR           p10,7,c1,c0,0   /* Floating-Point Status and Control Register */
>   #define MVFR0           p10,7,c7,c0,0   /* Media and VFP Feature Register 0 */
> +#define MVFR1           p10,7,c6,c0,0   /* Media and VFP Feature Register 1 */
> +#define MVFR2           p10,7,c5,c0,0   /* Media and VFP Feature Register 2 */
>   #define FPEXC           p10,7,c8,c0,0   /* Floating-Point Exception Control Register */
>   #define FPINST          p10,7,c9,c0,0   /* Floating-Point Instruction Register */
>   #define FPINST2         p10,7,c10,c0,0  /* Floating-point Instruction Register 2 */
> @@ -108,18 +110,23 @@
>   #define MPIDR           p15,0,c0,c0,5   /* Multiprocessor Affinity Register */
>   #define ID_PFR0         p15,0,c0,c1,0   /* Processor Feature Register 0 */
>   #define ID_PFR1         p15,0,c0,c1,1   /* Processor Feature Register 1 */
> +#define ID_PFR2         p15,0,c0,c3,4   /* Processor Feature Register 2 */
>   #define ID_DFR0         p15,0,c0,c1,2   /* Debug Feature Register 0 */
> +#define ID_DFR1         p15,0,c0,c3,5   /* Debug Feature Register 1 */
>   #define ID_AFR0         p15,0,c0,c1,3   /* Auxiliary Feature Register 0 */
>   #define ID_MMFR0        p15,0,c0,c1,4   /* Memory Model Feature Register 0 */
>   #define ID_MMFR1        p15,0,c0,c1,5   /* Memory Model Feature Register 1 */
>   #define ID_MMFR2        p15,0,c0,c1,6   /* Memory Model Feature Register 2 */
>   #define ID_MMFR3        p15,0,c0,c1,7   /* Memory Model Feature Register 3 */
> +#define ID_MMFR4        p15,0,c0,c2,6   /* Memory Model Feature Register 4 */
> +#define ID_MMFR5        p15,0,c0,c3,6   /* Memory Model Feature Register 5 */
>   #define ID_ISAR0        p15,0,c0,c2,0   /* ISA Feature Register 0 */
>   #define ID_ISAR1        p15,0,c0,c2,1   /* ISA Feature Register 1 */
>   #define ID_ISAR2        p15,0,c0,c2,2   /* ISA Feature Register 2 */
>   #define ID_ISAR3        p15,0,c0,c2,3   /* ISA Feature Register 3 */
>   #define ID_ISAR4        p15,0,c0,c2,4   /* ISA Feature Register 4 */
>   #define ID_ISAR5        p15,0,c0,c2,5   /* ISA Feature Register 5 */
> +#define ID_ISAR6        p15,0,c0,c2,7   /* ISA Feature Register 6 */
>   #define CCSIDR          p15,1,c0,c0,0   /* Cache Size ID Registers */
>   #define CLIDR           p15,1,c0,c0,1   /* Cache Level ID Register */
>   #define CSSELR          p15,2,c0,c0,0   /* Cache Size Selection Register */
> @@ -312,18 +319,23 @@
>   #define HSTR_EL2                HSTR
>   #define ID_AFR0_EL1             ID_AFR0
>   #define ID_DFR0_EL1             ID_DFR0
> +#define ID_DFR1_EL1             ID_DFR1
>   #define ID_ISAR0_EL1            ID_ISAR0
>   #define ID_ISAR1_EL1            ID_ISAR1
>   #define ID_ISAR2_EL1            ID_ISAR2
>   #define ID_ISAR3_EL1            ID_ISAR3
>   #define ID_ISAR4_EL1            ID_ISAR4
>   #define ID_ISAR5_EL1            ID_ISAR5
> +#define ID_ISAR6_EL1            ID_ISAR6
>   #define ID_MMFR0_EL1            ID_MMFR0
>   #define ID_MMFR1_EL1            ID_MMFR1
>   #define ID_MMFR2_EL1            ID_MMFR2
>   #define ID_MMFR3_EL1            ID_MMFR3
> +#define ID_MMFR4_EL1            ID_MMFR4
> +#define ID_MMFR5_EL1            ID_MMFR5
>   #define ID_PFR0_EL1             ID_PFR0
>   #define ID_PFR1_EL1             ID_PFR1
> +#define ID_PFR2_EL1             ID_PFR2
>   #define IFSR32_EL2              IFSR
>   #define MDCR_EL2                HDCR
>   #define MIDR_EL1                MIDR
> diff --git a/xen/include/asm-arm/cpufeature.h b/xen/include/asm-arm/cpufeature.h
> index c7b5052992..6cf83d775b 100644
> --- a/xen/include/asm-arm/cpufeature.h
> +++ b/xen/include/asm-arm/cpufeature.h
> @@ -148,6 +148,7 @@ struct cpuinfo_arm {
>       union {
>           uint64_t bits[2];
>           struct {
> +            /* PFR0 */
>               unsigned long el0:4;
>               unsigned long el1:4;
>               unsigned long el2:4;
> @@ -155,9 +156,23 @@ struct cpuinfo_arm {
>               unsigned long fp:4;   /* Floating Point */
>               unsigned long simd:4; /* Advanced SIMD */
>               unsigned long gic:4;  /* GIC support */
> -            unsigned long __res0:28;
> +            unsigned long ras:4;
> +            unsigned long sve:4;
> +            unsigned long sel2:4;
> +            unsigned long mpam:4;
> +            unsigned long amu:4;
> +            unsigned long dit:4;
> +            unsigned long __res0:4;
>               unsigned long csv2:4;
> -            unsigned long __res1:4;
> +            unsigned long cvs3:4;
> +
> +            /* PFR1 */
> +            unsigned long bt:4;
> +            unsigned long ssbs:4;
> +            unsigned long mte:4;
> +            unsigned long ras_frac:4;
> +            unsigned long mpam_frac:4;
> +            unsigned long __res1:44;
>           };
>       } pfr64;
>   
> @@ -170,7 +185,7 @@ struct cpuinfo_arm {
>       } aux64;
>   
>       union {
> -        uint64_t bits[2];
> +        uint64_t bits[3];
>           struct {
>               unsigned long pa_range:4;
>               unsigned long asid_bits:4;
> @@ -190,6 +205,8 @@ struct cpuinfo_arm {
>               unsigned long pan:4;
>               unsigned long __res1:8;
>               unsigned long __res2:32;
> +
> +            unsigned long __res3:64;
>           };
>       } mm64;
>   
> @@ -197,6 +214,10 @@ struct cpuinfo_arm {
>           uint64_t bits[2];
>       } isa64;
>   
> +    struct {
> +        uint64_t bits[1];
> +    } zfr64;
> +
>   #endif
>   
>       /*
> @@ -204,25 +225,38 @@ struct cpuinfo_arm {
>        * when running in 32-bit mode.
>        */
>       union {
> -        uint32_t bits[2];
> +        uint32_t bits[3];
>           struct {
> +            /* PFR0 */
>               unsigned long arm:4;
>               unsigned long thumb:4;
>               unsigned long jazelle:4;
>               unsigned long thumbee:4;
> -            unsigned long __res0:16;
> +            unsigned long csv2:4;
> +            unsigned long amu:4;
> +            unsigned long dit:4;
> +            unsigned long ras:4;
>   
> +            /* PFR1 */
>               unsigned long progmodel:4;
>               unsigned long security:4;
>               unsigned long mprofile:4;
>               unsigned long virt:4;
>               unsigned long gentimer:4;
> -            unsigned long __res1:12;
> +            unsigned long sec_frac:4;
> +            unsigned long virt_frac:4;
> +            unsigned long gic:4;
> +
> +            /* PFR2 */
> +            unsigned long csv3:4;
> +            unsigned long ssbs:4;
> +            unsigned long ras_frac:4;
> +            unsigned long __res2:20;
>           };
>       } pfr32;
>   
>       struct {
> -        uint32_t bits[1];
> +        uint32_t bits[2];
>       } dbg32;
>   
>       struct {
> @@ -230,12 +264,16 @@ struct cpuinfo_arm {
>       } aux32;
>   
>       struct {
> -        uint32_t bits[4];
> +        uint32_t bits[6];
>       } mm32;
>   
>       struct {
> -        uint32_t bits[6];
> +        uint32_t bits[7];
>       } isa32;
> +
> +    struct {
> +        uint64_t bits[3];

Shouldn't this be register_t?

> +    } mvfr;
>   };
>   
>   extern struct cpuinfo_arm boot_cpu_data;
> 

-- 
Julien Grall

Re: [PATCH v3 2/7] xen/arm: Add arm64 ID registers definitions

[Julien Grall]

Hi Bertrand,

On 09/12/2020 16:30, Bertrand Marquis wrote:
> Add coprocessor registers definitions for all ID registers trapped
> through the TID3 bit of HSR.
> Those are the one that will be emulated in Xen to only publish to guests
> the features that are supported by Xen and that are accessible to
> guests.
> Also define a case to catch all reserved registers that should be
> handled as RAZ.
> 
> Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
> ---
> Changes in V2: Rebase
> Changes in V3:
>    Add case definition for reserved registers.
> 
> ---
>   xen/include/asm-arm/arm64/hsr.h | 66 +++++++++++++++++++++++++++++++++
>   1 file changed, 66 insertions(+)
> 
> diff --git a/xen/include/asm-arm/arm64/hsr.h b/xen/include/asm-arm/arm64/hsr.h
> index ca931dd2fe..ffe0f0007e 100644
> --- a/xen/include/asm-arm/arm64/hsr.h
> +++ b/xen/include/asm-arm/arm64/hsr.h
> @@ -110,6 +110,72 @@
>   #define HSR_SYSREG_CNTP_CTL_EL0   HSR_SYSREG(3,3,c14,c2,1)
>   #define HSR_SYSREG_CNTP_CVAL_EL0  HSR_SYSREG(3,3,c14,c2,2)
>   
> +/* Those registers are used when HCR_EL2.TID3 is set */
> +#define HSR_SYSREG_ID_PFR0_EL1    HSR_SYSREG(3,0,c0,c1,0)
> +#define HSR_SYSREG_ID_PFR1_EL1    HSR_SYSREG(3,0,c0,c1,1)
> +#define HSR_SYSREG_ID_PFR2_EL1    HSR_SYSREG(3,0,c0,c3,4)
> +#define HSR_SYSREG_ID_DFR0_EL1    HSR_SYSREG(3,0,c0,c1,2)
> +#define HSR_SYSREG_ID_DFR1_EL1    HSR_SYSREG(3,0,c0,c3,5)
> +#define HSR_SYSREG_ID_AFR0_EL1    HSR_SYSREG(3,0,c0,c1,3)
> +#define HSR_SYSREG_ID_MMFR0_EL1   HSR_SYSREG(3,0,c0,c1,4)
> +#define HSR_SYSREG_ID_MMFR1_EL1   HSR_SYSREG(3,0,c0,c1,5)
> +#define HSR_SYSREG_ID_MMFR2_EL1   HSR_SYSREG(3,0,c0,c1,6)
> +#define HSR_SYSREG_ID_MMFR3_EL1   HSR_SYSREG(3,0,c0,c1,7)
> +#define HSR_SYSREG_ID_MMFR4_EL1   HSR_SYSREG(3,0,c0,c2,6)
> +#define HSR_SYSREG_ID_MMFR5_EL1   HSR_SYSREG(3,0,c0,c3,6)
> +#define HSR_SYSREG_ID_ISAR0_EL1   HSR_SYSREG(3,0,c0,c2,0)
> +#define HSR_SYSREG_ID_ISAR1_EL1   HSR_SYSREG(3,0,c0,c2,1)
> +#define HSR_SYSREG_ID_ISAR2_EL1   HSR_SYSREG(3,0,c0,c2,2)
> +#define HSR_SYSREG_ID_ISAR3_EL1   HSR_SYSREG(3,0,c0,c2,3)
> +#define HSR_SYSREG_ID_ISAR4_EL1   HSR_SYSREG(3,0,c0,c2,4)
> +#define HSR_SYSREG_ID_ISAR5_EL1   HSR_SYSREG(3,0,c0,c2,5)
> +#define HSR_SYSREG_ID_ISAR6_EL1   HSR_SYSREG(3,0,c0,c2,7)
> +#define HSR_SYSREG_MVFR0_EL1      HSR_SYSREG(3,0,c0,c3,0)
> +#define HSR_SYSREG_MVFR1_EL1      HSR_SYSREG(3,0,c0,c3,1)
> +#define HSR_SYSREG_MVFR2_EL1      HSR_SYSREG(3,0,c0,c3,2)
> +
> +#define HSR_SYSREG_ID_AA64PFR0_EL1   HSR_SYSREG(3,0,c0,c4,0)
> +#define HSR_SYSREG_ID_AA64PFR1_EL1   HSR_SYSREG(3,0,c0,c4,1)
> +#define HSR_SYSREG_ID_AA64DFR0_EL1   HSR_SYSREG(3,0,c0,c5,0)
> +#define HSR_SYSREG_ID_AA64DFR1_EL1   HSR_SYSREG(3,0,c0,c5,1)
> +#define HSR_SYSREG_ID_AA64ISAR0_EL1  HSR_SYSREG(3,0,c0,c6,0)
> +#define HSR_SYSREG_ID_AA64ISAR1_EL1  HSR_SYSREG(3,0,c0,c6,1)
> +#define HSR_SYSREG_ID_AA64MMFR0_EL1  HSR_SYSREG(3,0,c0,c7,0)
> +#define HSR_SYSREG_ID_AA64MMFR1_EL1  HSR_SYSREG(3,0,c0,c7,1)
> +#define HSR_SYSREG_ID_AA64MMFR2_EL1  HSR_SYSREG(3,0,c0,c7,2)
> +#define HSR_SYSREG_ID_AA64AFR0_EL1   HSR_SYSREG(3,0,c0,c5,4)
> +#define HSR_SYSREG_ID_AA64AFR1_EL1   HSR_SYSREG(3,0,c0,c5,5)
> +#define HSR_SYSREG_ID_AA64ZFR0_EL1   HSR_SYSREG(3,0,c0,c4,4)
> +
> +/*
> + * Those cases are catching all Reserved registers trapped by TID3 which
> + * currently have no assignment.
> + * HCR.TID3 is trapping all registers in the group 3:
> + * Op0 == 3, op1 == 0, CRn == c0,CRm == {c1-c7}, op2 == {0-7}.
> + */
> +#define HSR_SYSREG_TID3_RESERVED_CASE  case HSR_SYSREG(3,0,c0,c3,3): \
> +                                       case HSR_SYSREG(3,0,c0,c3,7): \
> +                                       case HSR_SYSREG(3,0,c0,c4,2): \
> +                                       case HSR_SYSREG(3,0,c0,c4,3): \
> +                                       case HSR_SYSREG(3,0,c0,c4,5): \
> +                                       case HSR_SYSREG(3,0,c0,c4,6): \
> +                                       case HSR_SYSREG(3,0,c0,c4,7): \
> +                                       case HSR_SYSREG(3,0,c0,c5,2): \
> +                                       case HSR_SYSREG(3,0,c0,c5,3): \
> +                                       case HSR_SYSREG(3,0,c0,c5,6): \
> +                                       case HSR_SYSREG(3,0,c0,c5,7): \
> +                                       case HSR_SYSREG(3,0,c0,c6,2): \
> +                                       case HSR_SYSREG(3,0,c0,c6,3): \
> +                                       case HSR_SYSREG(3,0,c0,c6,4): \
> +                                       case HSR_SYSREG(3,0,c0,c6,5): \
> +                                       case HSR_SYSREG(3,0,c0,c6,6): \
> +                                       case HSR_SYSREG(3,0,c0,c6,7): \
> +                                       case HSR_SYSREG(3,0,c0,c7,3): \
> +                                       case HSR_SYSREG(3,0,c0,c7,4): \
> +                                       case HSR_SYSREG(3,0,c0,c7,5): \
> +                                       case HSR_SYSREG(3,0,c0,c7,6): \
> +                                       case HSR_SYSREG(3,0,c0,c7,7)

I don't like the idea to define the list of case in a header that is 
used by multiple source. Please define it directly in the source file 
that use it.

Cheers,

-- 
Julien Grall

Re: [PATCH v3 3/7] xen/arm: create a cpuinfo structure for guest

[Julien Grall]

Hi Bertand,

On 09/12/2020 16:30, Bertrand Marquis wrote:
> Create a cpuinfo structure for guest and mask into it the features that
> we do not support in Xen or that we do not want to publish to guests.
> 
> Modify some values in the cpuinfo structure for guests to mask some
> features which we do not want to allow to guests (like AMU) or we do not
> support (like SVE).
> 
> The code is trying to group together registers modifications for the
> same feature to be able in the long term to easily enable/disable a
> feature depending on user parameters or add other registers modification
> in the same place (like enabling/disabling HCR bits).
> 
> Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
> ---
> Changes in V2: Rebase
> Changes in V3:
>    Use current_cpu_data info instead of recalling identify_cpu
> 
> ---
>   xen/arch/arm/cpufeature.c        | 51 ++++++++++++++++++++++++++++++++
>   xen/include/asm-arm/cpufeature.h |  2 ++
>   2 files changed, 53 insertions(+)
> 
> diff --git a/xen/arch/arm/cpufeature.c b/xen/arch/arm/cpufeature.c
> index bc7ee5ac95..7255383504 100644
> --- a/xen/arch/arm/cpufeature.c
> +++ b/xen/arch/arm/cpufeature.c
> @@ -24,6 +24,8 @@
>   
>   DECLARE_BITMAP(cpu_hwcaps, ARM_NCAPS);
>   
> +struct cpuinfo_arm __read_mostly guest_cpuinfo;
> +
>   void update_cpu_capabilities(const struct arm_cpu_capabilities *caps,
>                                const char *info)
>   {
> @@ -157,6 +159,55 @@ void identify_cpu(struct cpuinfo_arm *c)
>   #endif
>   }
>   
> +/*
> + * This function is creating a cpuinfo structure with values modified to mask
> + * all cpu features that should not be published to guest.
> + * The created structure is then used to provide ID registers values to guests.
> + */
> +static int __init create_guest_cpuinfo(void)
> +{
> +    /*
> +     * TODO: The code is currently using only the features detected on the boot
> +     * core. In the long term we should try to compute values containing only
> +     * features supported by all cores.
> +     */
> +    guest_cpuinfo = current_cpu_data;

It would be more logical to use boot_cpu_data as this would be easier to 
match with your comment.

> +
> +#ifdef CONFIG_ARM_64
> +    /* Disable MPAM as xen does not support it */
> +    guest_cpuinfo.pfr64.mpam = 0;
> +    guest_cpuinfo.pfr64.mpam_frac = 0;
> +
> +    /* Disable SVE as Xen does not support it */
> +    guest_cpuinfo.pfr64.sve = 0;
> +    guest_cpuinfo.zfr64.bits[0] = 0;
> +
> +    /* Disable MTE as Xen does not support it */
> +    guest_cpuinfo.pfr64.mte = 0;
> +#endif
> +
> +    /* Disable AMU */
> +#ifdef CONFIG_ARM_64
> +    guest_cpuinfo.pfr64.amu = 0;
> +#endif
> +    guest_cpuinfo.pfr32.amu = 0;
> +
> +    /* Disable RAS as Xen does not support it */
> +#ifdef CONFIG_ARM_64
> +    guest_cpuinfo.pfr64.ras = 0;
> +    guest_cpuinfo.pfr64.ras_frac = 0;
> +#endif
> +    guest_cpuinfo.pfr32.ras = 0;
> +    guest_cpuinfo.pfr32.ras_frac = 0;

How about all the fields that are currently marked as RES0/RES1? 
Shouldn't we make sure they will stay like that even if newer 
architecture use them?

> +
> +    return 0;
> +}
> +/*
> + * This function needs to be run after all smp are started to have
> + * cpuinfo structures for all cores.
> + */
> +__initcall(create_guest_cpuinfo);
> +
>   /*
>    * Local variables:
>    * mode: C
> diff --git a/xen/include/asm-arm/cpufeature.h b/xen/include/asm-arm/cpufeature.h
> index 6cf83d775b..10b62bd324 100644
> --- a/xen/include/asm-arm/cpufeature.h
> +++ b/xen/include/asm-arm/cpufeature.h
> @@ -283,6 +283,8 @@ extern void identify_cpu(struct cpuinfo_arm *);
>   extern struct cpuinfo_arm cpu_data[];
>   #define current_cpu_data cpu_data[smp_processor_id()]
>   
> +extern struct cpuinfo_arm guest_cpuinfo;
> +
>   #endif /* __ASSEMBLY__ */
>   
>   #endif
> 

-- 
Julien Grall

Re: [PATCH v3 4/7] xen/arm: Add handler for ID registers on arm64

[Julien Grall]

On 09/12/2020 16:30, Bertrand Marquis wrote:
> Add vsysreg emulation for registers trapped when TID3 bit is activated
> in HSR.
> The emulation is returning the value stored in cpuinfo_guest structure
> for know registers and is handling reserved registers as RAZ.
> 
> Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
> ---
> Changes in V2: Rebase
> Changes in V3:
>    Fix commit message
>    Fix code style for GENERATE_TID3_INFO declaration
>    Add handling of reserved registers as RAZ.
> 
> ---
>   xen/arch/arm/arm64/vsysreg.c | 53 ++++++++++++++++++++++++++++++++++++
>   1 file changed, 53 insertions(+)
> 
> diff --git a/xen/arch/arm/arm64/vsysreg.c b/xen/arch/arm/arm64/vsysreg.c
> index 8a85507d9d..ef7a11dbdd 100644
> --- a/xen/arch/arm/arm64/vsysreg.c
> +++ b/xen/arch/arm/arm64/vsysreg.c
> @@ -69,6 +69,14 @@ TVM_REG(CONTEXTIDR_EL1)
>           break;                                                          \
>       }
>   
> +/* Macro to generate easily case for ID co-processor emulation */
> +#define GENERATE_TID3_INFO(reg, field, offset)                          \
> +    case HSR_SYSREG_##reg:                                              \
> +    {                                                                   \
> +        return handle_ro_read_val(regs, regidx, hsr.sysreg.read, hsr,   \
> +                          1, guest_cpuinfo.field.bits[offset]);         \

The indentation looks wrong here. The "1" should be aligned with "regs".

> +    }
> +
>   void do_sysreg(struct cpu_user_regs *regs,
>                  const union hsr hsr)
>   {
> @@ -259,6 +267,51 @@ void do_sysreg(struct cpu_user_regs *regs,
>            */
>           return handle_raz_wi(regs, regidx, hsr.sysreg.read, hsr, 1);
>   
> +    /*
> +     * HCR_EL2.TID3
> +     *
> +     * This is trapping most Identification registers used by a guest
> +     * to identify the processor features
> +     */
> +    GENERATE_TID3_INFO(ID_PFR0_EL1, pfr32, 0)
> +    GENERATE_TID3_INFO(ID_PFR1_EL1, pfr32, 1)
> +    GENERATE_TID3_INFO(ID_PFR2_EL1, pfr32, 2)
> +    GENERATE_TID3_INFO(ID_DFR0_EL1, dbg32, 0)
> +    GENERATE_TID3_INFO(ID_DFR1_EL1, dbg32, 1)
> +    GENERATE_TID3_INFO(ID_AFR0_EL1, aux32, 0)
> +    GENERATE_TID3_INFO(ID_MMFR0_EL1, mm32, 0)
> +    GENERATE_TID3_INFO(ID_MMFR1_EL1, mm32, 1)
> +    GENERATE_TID3_INFO(ID_MMFR2_EL1, mm32, 2)
> +    GENERATE_TID3_INFO(ID_MMFR3_EL1, mm32, 3)
> +    GENERATE_TID3_INFO(ID_MMFR4_EL1, mm32, 4)
> +    GENERATE_TID3_INFO(ID_MMFR5_EL1, mm32, 5)
> +    GENERATE_TID3_INFO(ID_ISAR0_EL1, isa32, 0)
> +    GENERATE_TID3_INFO(ID_ISAR1_EL1, isa32, 1)
> +    GENERATE_TID3_INFO(ID_ISAR2_EL1, isa32, 2)
> +    GENERATE_TID3_INFO(ID_ISAR3_EL1, isa32, 3)
> +    GENERATE_TID3_INFO(ID_ISAR4_EL1, isa32, 4)
> +    GENERATE_TID3_INFO(ID_ISAR5_EL1, isa32, 5)
> +    GENERATE_TID3_INFO(ID_ISAR6_EL1, isa32, 6)
> +    GENERATE_TID3_INFO(MVFR0_EL1, mvfr, 0)
> +    GENERATE_TID3_INFO(MVFR1_EL1, mvfr, 1)
> +    GENERATE_TID3_INFO(MVFR2_EL1, mvfr, 2)
> +    GENERATE_TID3_INFO(ID_AA64PFR0_EL1, pfr64, 0)
> +    GENERATE_TID3_INFO(ID_AA64PFR1_EL1, pfr64, 1)
> +    GENERATE_TID3_INFO(ID_AA64DFR0_EL1, dbg64, 0)
> +    GENERATE_TID3_INFO(ID_AA64DFR1_EL1, dbg64, 1)
> +    GENERATE_TID3_INFO(ID_AA64ISAR0_EL1, isa64, 0)
> +    GENERATE_TID3_INFO(ID_AA64ISAR1_EL1, isa64, 1)
> +    GENERATE_TID3_INFO(ID_AA64MMFR0_EL1, mm64, 0)
> +    GENERATE_TID3_INFO(ID_AA64MMFR1_EL1, mm64, 1)
> +    GENERATE_TID3_INFO(ID_AA64MMFR2_EL1, mm64, 2)
> +    GENERATE_TID3_INFO(ID_AA64AFR0_EL1, aux64, 0)
> +    GENERATE_TID3_INFO(ID_AA64AFR1_EL1, aux64, 1)
> +    GENERATE_TID3_INFO(ID_AA64ZFR0_EL1, zfr64, 0)
> +
> +    HSR_SYSREG_TID3_RESERVED_CASE:
> +        /* Handle all reserved registers as RAZ */
> +        return handle_ro_raz(regs, regidx, hsr.sysreg.read, hsr, 1);
> +
>       /*
>        * HCR_EL2.TIDCP
>        *
> 

-- 
Julien Grall

Re: [PATCH v3 6/7] xen/arm: Add CP10 exception support to handle MVFR

[Julien Grall]

Hi Bertrand,

On 09/12/2020 16:30, Bertrand Marquis wrote:
> Add support for cp10 exceptions decoding to be able to emulate the
> values for MVFR0, MVFR1 and MVFR2 when TID3 bit of HSR is activated.
> This is required for aarch32 guests accessing MVFR registers using
> vmrs and vmsr instructions.
> 
> Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
> ---
> Changes in V2: Rebase
> Changes in V3:
>    Add case for MVFR2, fix typo VMFR <-> MVFR.
> 
> ---
>   xen/arch/arm/traps.c             |  5 ++++
>   xen/arch/arm/vcpreg.c            | 39 +++++++++++++++++++++++++++++++-
>   xen/include/asm-arm/perfc_defn.h |  1 +
>   xen/include/asm-arm/traps.h      |  1 +
>   4 files changed, 45 insertions(+), 1 deletion(-)
> 
> diff --git a/xen/arch/arm/traps.c b/xen/arch/arm/traps.c
> index 22bd1bd4c6..28d9d64558 100644
> --- a/xen/arch/arm/traps.c
> +++ b/xen/arch/arm/traps.c
> @@ -2097,6 +2097,11 @@ void do_trap_guest_sync(struct cpu_user_regs *regs)
>           perfc_incr(trap_cp14_dbg);
>           do_cp14_dbg(regs, hsr);
>           break;
> +    case HSR_EC_CP10:
> +        GUEST_BUG_ON(!psr_mode_is_32bit(regs));
> +        perfc_incr(trap_cp10);
> +        do_cp10(regs, hsr);
> +        break;
>       case HSR_EC_CP:
>           GUEST_BUG_ON(!psr_mode_is_32bit(regs));
>           perfc_incr(trap_cp);
> diff --git a/xen/arch/arm/vcpreg.c b/xen/arch/arm/vcpreg.c
> index d371a1c38c..da4e22a467 100644
> --- a/xen/arch/arm/vcpreg.c
> +++ b/xen/arch/arm/vcpreg.c
> @@ -319,7 +319,7 @@ void do_cp15_32(struct cpu_user_regs *regs, const union hsr hsr)
>       GENERATE_TID3_INFO(ID_ISAR4, isa32, 4)
>       GENERATE_TID3_INFO(ID_ISAR5, isa32, 5)
>       GENERATE_TID3_INFO(ID_ISAR6, isa32, 6)
> -    /* MVFR registers are in cp10 no cp15 */
> +    /* MVFR registers are in cp10 not cp15 */

The code was originally added in the previous patch. Please either 
introduce the comment here or fold it in the previous patch.

>   
>       HSR_CPREG32_TID3_RESERVED_CASE:
>           /* Handle all reserved registers as RAZ */
> @@ -638,6 +638,43 @@ void do_cp14_dbg(struct cpu_user_regs *regs, const union hsr hsr)
>       inject_undef_exception(regs, hsr);
>   }
>   
> +void do_cp10(struct cpu_user_regs *regs, const union hsr hsr)
> +{
> +    const struct hsr_cp32 cp32 = hsr.cp32;
> +    int regidx = cp32.reg;
> +
> +    if ( !check_conditional_instr(regs, hsr) )
> +    {
> +        advance_pc(regs, hsr);
> +        return;
> +    }
> +
> +    switch ( hsr.bits & HSR_CP32_REGS_MASK )
> +    {
> +    /*
> +     * HSR.TID3 is trapping access to MVFR register used to identify the
> +     * VFP/Simd using VMRS/VMSR instructions.
> +     * Exception encoding is using MRC/MCR standard with the reg field in Crn
> +     * as are declared MVFR0 and MVFR1 in cpregs.h
> +     */
> +    GENERATE_TID3_INFO(MVFR0, mvfr, 0)
> +    GENERATE_TID3_INFO(MVFR1, mvfr, 1)
> +    GENERATE_TID3_INFO(MVFR2, mvfr, 2)
> +
> +    default:
> +        gdprintk(XENLOG_ERR,
> +                 "%s p10, %d, r%d, cr%d, cr%d, %d @ 0x%"PRIregister"\n",
> +                 cp32.read ? "mrc" : "mcr",
> +                 cp32.op1, cp32.reg, cp32.crn, cp32.crm, cp32.op2, regs->pc);
> +        gdprintk(XENLOG_ERR, "unhandled 32-bit CP10 access %#x\n",
> +                 hsr.bits & HSR_CP32_REGS_MASK);
> +        inject_undef_exception(regs, hsr);
> +        return;
> +    }
> +
> +    advance_pc(regs, hsr);
> +}
> +
>   void do_cp(struct cpu_user_regs *regs, const union hsr hsr)
>   {
>       const struct hsr_cp cp = hsr.cp;
> diff --git a/xen/include/asm-arm/perfc_defn.h b/xen/include/asm-arm/perfc_defn.h
> index 6a83185163..31f071222b 100644
> --- a/xen/include/asm-arm/perfc_defn.h
> +++ b/xen/include/asm-arm/perfc_defn.h
> @@ -11,6 +11,7 @@ PERFCOUNTER(trap_cp15_64,  "trap: cp15 64-bit access")
>   PERFCOUNTER(trap_cp14_32,  "trap: cp14 32-bit access")
>   PERFCOUNTER(trap_cp14_64,  "trap: cp14 64-bit access")
>   PERFCOUNTER(trap_cp14_dbg, "trap: cp14 dbg access")
> +PERFCOUNTER(trap_cp10,     "trap: cp10 access")
>   PERFCOUNTER(trap_cp,       "trap: cp access")
>   PERFCOUNTER(trap_smc32,    "trap: 32-bit smc")
>   PERFCOUNTER(trap_hvc32,    "trap: 32-bit hvc")
> diff --git a/xen/include/asm-arm/traps.h b/xen/include/asm-arm/traps.h
> index 997c37884e..c4a3d0fb1b 100644
> --- a/xen/include/asm-arm/traps.h
> +++ b/xen/include/asm-arm/traps.h
> @@ -62,6 +62,7 @@ void do_cp15_64(struct cpu_user_regs *regs, const union hsr hsr);
>   void do_cp14_32(struct cpu_user_regs *regs, const union hsr hsr);
>   void do_cp14_64(struct cpu_user_regs *regs, const union hsr hsr);
>   void do_cp14_dbg(struct cpu_user_regs *regs, const union hsr hsr);
> +void do_cp10(struct cpu_user_regs *regs, const union hsr hsr);
>   void do_cp(struct cpu_user_regs *regs, const union hsr hsr);
>   
>   /* SMCCC handling */
> 

-- 
Julien Grall

Re: [PATCH v3 7/7] xen/arm: Activate TID3 in HCR_EL2

[Julien Grall]

Hi Bertrand,

On 09/12/2020 16:31, Bertrand Marquis wrote:
> Activate TID3 bit in HSR register when starting a guest.

s/HSR/HCR/

> This will trap all coprecessor ID registers so that we can give to guest
> values corresponding to what they can actually use and mask some
> features to guests even though they would be supported by the underlying
> hardware (like SVE or MPAM).

So this will make sure the guest will not be able to identify the 
feature. Did you check that the features are effectively not accessible 
by the guest? IOW it should trap.

Cheers,

-- 
Julien Grall

Re: [PATCH v3 3/7] xen/arm: create a cpuinfo structure for guest

[Julien Grall]

Hi,

On 09/12/2020 16:30, Bertrand Marquis wrote:
> +    /* Disable MPAM as xen does not support it */

I am going to be picky :). I think we want to say "hide" rather than 
"disable" because the latter is done differently via the HCR_EL2.

Cheers,

-- 
Julien Grall

Re: [PATCH v3 2/7] xen/arm: Add arm64 ID registers definitions

[Stefano Stabellini]

On Wed, 9 Dec 2020, Julien Grall wrote:
> Hi Bertrand,
> 
> On 09/12/2020 16:30, Bertrand Marquis wrote:
> > Add coprocessor registers definitions for all ID registers trapped
> > through the TID3 bit of HSR.
> > Those are the one that will be emulated in Xen to only publish to guests
> > the features that are supported by Xen and that are accessible to
> > guests.
> > Also define a case to catch all reserved registers that should be
> > handled as RAZ.
> > 
> > Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
> > ---
> > Changes in V2: Rebase
> > Changes in V3:
> >    Add case definition for reserved registers.
> > 
> > ---
> >   xen/include/asm-arm/arm64/hsr.h | 66 +++++++++++++++++++++++++++++++++
> >   1 file changed, 66 insertions(+)
> > 
> > diff --git a/xen/include/asm-arm/arm64/hsr.h
> > b/xen/include/asm-arm/arm64/hsr.h
> > index ca931dd2fe..ffe0f0007e 100644
> > --- a/xen/include/asm-arm/arm64/hsr.h
> > +++ b/xen/include/asm-arm/arm64/hsr.h
> > @@ -110,6 +110,72 @@
> >   #define HSR_SYSREG_CNTP_CTL_EL0   HSR_SYSREG(3,3,c14,c2,1)
> >   #define HSR_SYSREG_CNTP_CVAL_EL0  HSR_SYSREG(3,3,c14,c2,2)
> >   +/* Those registers are used when HCR_EL2.TID3 is set */
> > +#define HSR_SYSREG_ID_PFR0_EL1    HSR_SYSREG(3,0,c0,c1,0)
> > +#define HSR_SYSREG_ID_PFR1_EL1    HSR_SYSREG(3,0,c0,c1,1)
> > +#define HSR_SYSREG_ID_PFR2_EL1    HSR_SYSREG(3,0,c0,c3,4)
> > +#define HSR_SYSREG_ID_DFR0_EL1    HSR_SYSREG(3,0,c0,c1,2)
> > +#define HSR_SYSREG_ID_DFR1_EL1    HSR_SYSREG(3,0,c0,c3,5)
> > +#define HSR_SYSREG_ID_AFR0_EL1    HSR_SYSREG(3,0,c0,c1,3)
> > +#define HSR_SYSREG_ID_MMFR0_EL1   HSR_SYSREG(3,0,c0,c1,4)
> > +#define HSR_SYSREG_ID_MMFR1_EL1   HSR_SYSREG(3,0,c0,c1,5)
> > +#define HSR_SYSREG_ID_MMFR2_EL1   HSR_SYSREG(3,0,c0,c1,6)
> > +#define HSR_SYSREG_ID_MMFR3_EL1   HSR_SYSREG(3,0,c0,c1,7)
> > +#define HSR_SYSREG_ID_MMFR4_EL1   HSR_SYSREG(3,0,c0,c2,6)
> > +#define HSR_SYSREG_ID_MMFR5_EL1   HSR_SYSREG(3,0,c0,c3,6)
> > +#define HSR_SYSREG_ID_ISAR0_EL1   HSR_SYSREG(3,0,c0,c2,0)
> > +#define HSR_SYSREG_ID_ISAR1_EL1   HSR_SYSREG(3,0,c0,c2,1)
> > +#define HSR_SYSREG_ID_ISAR2_EL1   HSR_SYSREG(3,0,c0,c2,2)
> > +#define HSR_SYSREG_ID_ISAR3_EL1   HSR_SYSREG(3,0,c0,c2,3)
> > +#define HSR_SYSREG_ID_ISAR4_EL1   HSR_SYSREG(3,0,c0,c2,4)
> > +#define HSR_SYSREG_ID_ISAR5_EL1   HSR_SYSREG(3,0,c0,c2,5)
> > +#define HSR_SYSREG_ID_ISAR6_EL1   HSR_SYSREG(3,0,c0,c2,7)
> > +#define HSR_SYSREG_MVFR0_EL1      HSR_SYSREG(3,0,c0,c3,0)
> > +#define HSR_SYSREG_MVFR1_EL1      HSR_SYSREG(3,0,c0,c3,1)
> > +#define HSR_SYSREG_MVFR2_EL1      HSR_SYSREG(3,0,c0,c3,2)
> > +
> > +#define HSR_SYSREG_ID_AA64PFR0_EL1   HSR_SYSREG(3,0,c0,c4,0)
> > +#define HSR_SYSREG_ID_AA64PFR1_EL1   HSR_SYSREG(3,0,c0,c4,1)
> > +#define HSR_SYSREG_ID_AA64DFR0_EL1   HSR_SYSREG(3,0,c0,c5,0)
> > +#define HSR_SYSREG_ID_AA64DFR1_EL1   HSR_SYSREG(3,0,c0,c5,1)
> > +#define HSR_SYSREG_ID_AA64ISAR0_EL1  HSR_SYSREG(3,0,c0,c6,0)
> > +#define HSR_SYSREG_ID_AA64ISAR1_EL1  HSR_SYSREG(3,0,c0,c6,1)
> > +#define HSR_SYSREG_ID_AA64MMFR0_EL1  HSR_SYSREG(3,0,c0,c7,0)
> > +#define HSR_SYSREG_ID_AA64MMFR1_EL1  HSR_SYSREG(3,0,c0,c7,1)
> > +#define HSR_SYSREG_ID_AA64MMFR2_EL1  HSR_SYSREG(3,0,c0,c7,2)
> > +#define HSR_SYSREG_ID_AA64AFR0_EL1   HSR_SYSREG(3,0,c0,c5,4)
> > +#define HSR_SYSREG_ID_AA64AFR1_EL1   HSR_SYSREG(3,0,c0,c5,5)
> > +#define HSR_SYSREG_ID_AA64ZFR0_EL1   HSR_SYSREG(3,0,c0,c4,4)
> > +
> > +/*
> > + * Those cases are catching all Reserved registers trapped by TID3 which
> > + * currently have no assignment.
> > + * HCR.TID3 is trapping all registers in the group 3:
> > + * Op0 == 3, op1 == 0, CRn == c0,CRm == {c1-c7}, op2 == {0-7}.
> > + */
> > +#define HSR_SYSREG_TID3_RESERVED_CASE  case HSR_SYSREG(3,0,c0,c3,3): \
> > +                                       case HSR_SYSREG(3,0,c0,c3,7): \
> > +                                       case HSR_SYSREG(3,0,c0,c4,2): \
> > +                                       case HSR_SYSREG(3,0,c0,c4,3): \
> > +                                       case HSR_SYSREG(3,0,c0,c4,5): \
> > +                                       case HSR_SYSREG(3,0,c0,c4,6): \
> > +                                       case HSR_SYSREG(3,0,c0,c4,7): \
> > +                                       case HSR_SYSREG(3,0,c0,c5,2): \
> > +                                       case HSR_SYSREG(3,0,c0,c5,3): \
> > +                                       case HSR_SYSREG(3,0,c0,c5,6): \
> > +                                       case HSR_SYSREG(3,0,c0,c5,7): \
> > +                                       case HSR_SYSREG(3,0,c0,c6,2): \
> > +                                       case HSR_SYSREG(3,0,c0,c6,3): \
> > +                                       case HSR_SYSREG(3,0,c0,c6,4): \
> > +                                       case HSR_SYSREG(3,0,c0,c6,5): \
> > +                                       case HSR_SYSREG(3,0,c0,c6,6): \
> > +                                       case HSR_SYSREG(3,0,c0,c6,7): \
> > +                                       case HSR_SYSREG(3,0,c0,c7,3): \
> > +                                       case HSR_SYSREG(3,0,c0,c7,4): \
> > +                                       case HSR_SYSREG(3,0,c0,c7,5): \
> > +                                       case HSR_SYSREG(3,0,c0,c7,6): \
> > +                                       case HSR_SYSREG(3,0,c0,c7,7)
> 
> I don't like the idea to define the list of case in a header that is used by
> multiple source. Please define it directly in the source file that use it.

At that point it might be best to open-coding it in do_sysreg? I mean no
#define at all. 

Re: [PATCH v3 5/7] xen/arm: Add handler for cp15 ID registers

[Bertrand Marquis]

Hi Stefano,

> On 9 Dec 2020, at 19:54, Stefano Stabellini <sstabellini@kernel.org> wrote:
> 
> On Wed, 9 Dec 2020, Bertrand Marquis wrote:
>> Add support for emulation of cp15 based ID registers (on arm32 or when
>> running a 32bit guest on arm64).
>> The handlers are returning the values stored in the guest_cpuinfo
>> structure for known registers and RAZ for all reserved registers.
>> In the current status the MVFR registers are no supported.
>> 
>> Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
>> ---
>> Changes in V2: Rebase
>> Changes in V3:
>>  Add case definition for reserved registers
>>  Add handling of reserved registers as RAZ.
>>  Fix code style in GENERATE_TID3_INFO declaration
>> 
>> ---
>> xen/arch/arm/vcpreg.c        | 39 ++++++++++++++++++++++++++++++++++++
>> xen/include/asm-arm/cpregs.h | 25 +++++++++++++++++++++++
>> 2 files changed, 64 insertions(+)
>> 
>> diff --git a/xen/arch/arm/vcpreg.c b/xen/arch/arm/vcpreg.c
>> index cdc91cdf5b..d371a1c38c 100644
>> --- a/xen/arch/arm/vcpreg.c
>> +++ b/xen/arch/arm/vcpreg.c
>> @@ -155,6 +155,14 @@ TVM_REG32(CONTEXTIDR, CONTEXTIDR_EL1)
>>         break;                                                      \
>>     }
>> 
>> +/* Macro to generate easily case for ID co-processor emulation */
>> +#define GENERATE_TID3_INFO(reg, field, offset)                      \
>> +    case HSR_CPREG32(reg):                                          \
>> +    {                                                               \
>> +        return handle_ro_read_val(regs, regidx, cp32.read, hsr,     \
>> +                          1, guest_cpuinfo.field.bits[offset]);     \
>> +    }
>> +
>> void do_cp15_32(struct cpu_user_regs *regs, const union hsr hsr)
>> {
>>     const struct hsr_cp32 cp32 = hsr.cp32;
>> @@ -286,6 +294,37 @@ void do_cp15_32(struct cpu_user_regs *regs, const union hsr hsr)
>>          */
>>         return handle_raz_wi(regs, regidx, cp32.read, hsr, 1);
>> 
>> +    /*
>> +     * HCR_EL2.TID3
>> +     *
>> +     * This is trapping most Identification registers used by a guest
>> +     * to identify the processor features
>> +     */
>> +    GENERATE_TID3_INFO(ID_PFR0, pfr32, 0)
>> +    GENERATE_TID3_INFO(ID_PFR1, pfr32, 1)
>> +    GENERATE_TID3_INFO(ID_PFR2, pfr32, 2)
>> +    GENERATE_TID3_INFO(ID_DFR0, dbg32, 0)
>> +    GENERATE_TID3_INFO(ID_DFR1, dbg32, 1)
>> +    GENERATE_TID3_INFO(ID_AFR0, aux32, 0)
>> +    GENERATE_TID3_INFO(ID_MMFR0, mm32, 0)
>> +    GENERATE_TID3_INFO(ID_MMFR1, mm32, 1)
>> +    GENERATE_TID3_INFO(ID_MMFR2, mm32, 2)
>> +    GENERATE_TID3_INFO(ID_MMFR3, mm32, 3)
>> +    GENERATE_TID3_INFO(ID_MMFR4, mm32, 4)
>> +    GENERATE_TID3_INFO(ID_MMFR5, mm32, 5)
>> +    GENERATE_TID3_INFO(ID_ISAR0, isa32, 0)
>> +    GENERATE_TID3_INFO(ID_ISAR1, isa32, 1)
>> +    GENERATE_TID3_INFO(ID_ISAR2, isa32, 2)
>> +    GENERATE_TID3_INFO(ID_ISAR3, isa32, 3)
>> +    GENERATE_TID3_INFO(ID_ISAR4, isa32, 4)
>> +    GENERATE_TID3_INFO(ID_ISAR5, isa32, 5)
>> +    GENERATE_TID3_INFO(ID_ISAR6, isa32, 6)
>> +    /* MVFR registers are in cp10 no cp15 */
>> +
>> +    HSR_CPREG32_TID3_RESERVED_CASE:
>> +        /* Handle all reserved registers as RAZ */
>> +        return handle_ro_raz(regs, regidx, cp32.read, hsr, 1);
> 
> Same question as for the aarch64 case: do we need to do write-ignore
> for the reserved registers?

Arm architecture reference manual is listing all those registers as RO including
the reserved ones (cf table D12-2). This said I have no objection to make them
write ignore but from my understanding this would not reflect the hardware
behaviour.

> 
> 
>>     /*
>>      * HCR_EL2.TIDCP
>>      *
>> diff --git a/xen/include/asm-arm/cpregs.h b/xen/include/asm-arm/cpregs.h
>> index 2690ddeb7a..5cb1ad5cbe 100644
>> --- a/xen/include/asm-arm/cpregs.h
>> +++ b/xen/include/asm-arm/cpregs.h
>> @@ -133,6 +133,31 @@
>> #define VPIDR           p15,4,c0,c0,0   /* Virtualization Processor ID Register */
>> #define VMPIDR          p15,4,c0,c0,5   /* Virtualization Multiprocessor ID Register */
>> 
>> +/*
>> + * Those cases are catching all Reserved registers trapped by TID3 which
>> + * currently have no assignment.
>> + * HCR.TID3 is trapping all registers in the group 3:
>> + * coproc == p15, opc1 == 0, CRn == c0, CRm == {c2-c7}, opc2 == {0-7}.
>> + */
>> +#define HSR_CPREG32_TID3_CASES(REG)     case HSR_CPREG32(p15,0,c0,REG,0): \
>> +                                        case HSR_CPREG32(p15,0,c0,REG,1): \
>> +                                        case HSR_CPREG32(p15,0,c0,REG,2): \
>> +                                        case HSR_CPREG32(p15,0,c0,REG,3): \
>> +                                        case HSR_CPREG32(p15,0,c0,REG,4): \
>> +                                        case HSR_CPREG32(p15,0,c0,REG,5): \
>> +                                        case HSR_CPREG32(p15,0,c0,REG,6): \
>> +                                        case HSR_CPREG32(p15,0,c0,REG,7)
>> +
>> +#define HSR_CPREG32_TID3_RESERVED_CASE  case HSR_CPREG32(p15,0,c0,c3,0): \
>> +                                        case HSR_CPREG32(p15,0,c0,c3,1): \
>> +                                        case HSR_CPREG32(p15,0,c0,c3,2): \
>> +                                        case HSR_CPREG32(p15,0,c0,c3,3): \
>> +                                        case HSR_CPREG32(p15,0,c0,c3,7): \
>> +                                        HSR_CPREG32_TID3_CASES(c4): \
>> +                                        HSR_CPREG32_TID3_CASES(c5): \
>> +                                        HSR_CPREG32_TID3_CASES(c6): \
>> +                                        HSR_CPREG32_TID3_CASES(c7)
> 
> The following are missing, is it a problem?
> 
> p15,0,c0,c0,2
> p15,0,c0,c0,3
> p15,0,c0,c0,4
> p15,0,c0,c0,6
> p15,0,c0,c0,7

HCR.TID3 documentation is saying that access to "coproc == p15, opc1 == 0, 
CRn == c0, CRm == {c2-c7}, opc2 == {0-7}” are trapped so CRm = c0 is not handled here.

Cheers
Bertradn

Re: [PATCH v3 1/7] xen/arm: Add ID registers and complete cpuinfo

[Bertrand Marquis]

Hi Julien,

> On 9 Dec 2020, at 23:03, Julien Grall <julien@xen.org> wrote:
> 
> Hi Bertrand,
> 
> On 09/12/2020 16:30, Bertrand Marquis wrote:
>> Add definition and entries in cpuinfo for ID registers introduced in
>> newer Arm Architecture reference manual:
>> - ID_PFR2: processor feature register 2
>> - ID_DFR1: debug feature register 1
>> - ID_MMFR4 and ID_MMFR5: Memory model feature registers 4 and 5
>> - ID_ISA6: ISA Feature register 6
>> Add more bitfield definitions in PFR fields of cpuinfo.
>> Add MVFR2 register definition for aarch32.
>> Add mvfr values in cpuinfo.
>> Add some registers definition for arm64 in sysregs as some are not
>> always know by compilers.
>> Initialize the new values added in cpuinfo in identify_cpu during init.
>> Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
>> ---
>> Changes in V2:
>>   Fix dbg32 table size and add proper initialisation of the second entry
>>   of the table by reading ID_DFR1 register.
>> Changes in V3:
>>   Fix typo in commit title
>>   Add MVFR2 definition and handling on aarch32 and remove specific case
>>   for mvfr field in cpuinfo (now the same on arm64 and arm32).
>>   Add MMFR4 definition if not known by the compiler.
>> ---
>>  xen/arch/arm/cpufeature.c           | 18 ++++++++++
>>  xen/include/asm-arm/arm64/sysregs.h | 28 +++++++++++++++
>>  xen/include/asm-arm/cpregs.h        | 12 +++++++
>>  xen/include/asm-arm/cpufeature.h    | 56 ++++++++++++++++++++++++-----
>>  4 files changed, 105 insertions(+), 9 deletions(-)
>> diff --git a/xen/arch/arm/cpufeature.c b/xen/arch/arm/cpufeature.c
>> index 44126dbf07..bc7ee5ac95 100644
>> --- a/xen/arch/arm/cpufeature.c
>> +++ b/xen/arch/arm/cpufeature.c
>> @@ -114,15 +114,20 @@ void identify_cpu(struct cpuinfo_arm *c)
>>            c->mm64.bits[0]  = READ_SYSREG64(ID_AA64MMFR0_EL1);
>>          c->mm64.bits[1]  = READ_SYSREG64(ID_AA64MMFR1_EL1);
>> +        c->mm64.bits[2]  = READ_SYSREG64(ID_AA64MMFR2_EL1);
>>            c->isa64.bits[0] = READ_SYSREG64(ID_AA64ISAR0_EL1);
>>          c->isa64.bits[1] = READ_SYSREG64(ID_AA64ISAR1_EL1);
>> +
>> +        c->zfr64.bits[0] = READ_SYSREG64(ID_AA64ZFR0_EL1);
>>  #endif
>>            c->pfr32.bits[0] = READ_SYSREG32(ID_PFR0_EL1);
>>          c->pfr32.bits[1] = READ_SYSREG32(ID_PFR1_EL1);
>> +        c->pfr32.bits[2] = READ_SYSREG32(ID_PFR2_EL1);
>>            c->dbg32.bits[0] = READ_SYSREG32(ID_DFR0_EL1);
>> +        c->dbg32.bits[1] = READ_SYSREG32(ID_DFR1_EL1);
>>            c->aux32.bits[0] = READ_SYSREG32(ID_AFR0_EL1);
>>  @@ -130,6 +135,8 @@ void identify_cpu(struct cpuinfo_arm *c)
>>          c->mm32.bits[1]  = READ_SYSREG32(ID_MMFR1_EL1);
>>          c->mm32.bits[2]  = READ_SYSREG32(ID_MMFR2_EL1);
>>          c->mm32.bits[3]  = READ_SYSREG32(ID_MMFR3_EL1);
>> +        c->mm32.bits[4]  = READ_SYSREG32(ID_MMFR4_EL1);
>> +        c->mm32.bits[5]  = READ_SYSREG32(ID_MMFR5_EL1);
> 
> Please don't introduce any more use of READ_SYSREG32(), they are wrong on Armv8 because system registers are always 64-bit.

I followed the existing implementation but ...

> 
>>            c->isa32.bits[0] = READ_SYSREG32(ID_ISAR0_EL1);
>>          c->isa32.bits[1] = READ_SYSREG32(ID_ISAR1_EL1);
>> @@ -137,6 +144,17 @@ void identify_cpu(struct cpuinfo_arm *c)
>>          c->isa32.bits[3] = READ_SYSREG32(ID_ISAR3_EL1);
>>          c->isa32.bits[4] = READ_SYSREG32(ID_ISAR4_EL1);
>>          c->isa32.bits[5] = READ_SYSREG32(ID_ISAR5_EL1);
>> +        c->isa32.bits[6] = READ_SYSREG32(ID_ISAR6_EL1);
>> +
>> +#ifdef CONFIG_ARM_64
>> +        c->mvfr.bits[0] = READ_SYSREG64(MVFR0_EL1);
>> +        c->mvfr.bits[1] = READ_SYSREG64(MVFR1_EL1);
>> +        c->mvfr.bits[2] = READ_SYSREG64(MVFR2_EL1);
>> +#else
>> +        c->mvfr.bits[0] = READ_CP32(MVFR0);
>> +        c->mvfr.bits[1] = READ_CP32(MVFR1);
>> +        c->mvfr.bits[2] = READ_CP32(MVFR2);
>> +#endif
> 
> READ_SYSREG() will do the job to either use READ_SYSREG64() or READ_CP32() depending on the arch used.

.. I can modify the ones I added and the existing ones to user READ_SYSREG instead.
Please confirm if you want me to do that.

> 
>>  }
>>    /*
>> diff --git a/xen/include/asm-arm/arm64/sysregs.h b/xen/include/asm-arm/arm64/sysregs.h
>> index c60029d38f..077fd95fb7 100644
>> --- a/xen/include/asm-arm/arm64/sysregs.h
>> +++ b/xen/include/asm-arm/arm64/sysregs.h
>> @@ -57,6 +57,34 @@
>>  #define ICH_AP1R2_EL2             __AP1Rx_EL2(2)
>>  #define ICH_AP1R3_EL2             __AP1Rx_EL2(3)
>>  +/*
>> + * Define ID coprocessor registers if they are not
>> + * already defined by the compiler.
>> + *
>> + * Values picked from linux kernel
>> + */
>> +#ifndef ID_AA64MMFR2_EL1
> 
> I am a bit puzzled how this meant to work. Will the libc/compiler headers define ID_AA64MMFR2_EL1?

I tested this and if the compiler has a definition for the register, I am not entering the ifndef.
So there is no header defining this but if the compiler has the definition for this the ifndef will
not be entered.

> 
>> +#define ID_AA64MMFR2_EL1            S3_0_C0_C7_2
>> +#endif
>> +#ifndef ID_PFR2_EL1
>> +#define ID_PFR2_EL1                 S3_0_C0_C3_4
>> +#endif
>> +#ifndef ID_MMFR4_EL1
>> +#define ID_MMFR4_EL1                S3_0_C0_C2_6
>> +#endif
>> +#ifndef ID_MMFR5_EL1
>> +#define ID_MMFR5_EL1                S3_0_C0_C3_6
>> +#endif
>> +#ifndef ID_ISAR6_EL1
>> +#define ID_ISAR6_EL1                S3_0_C0_C2_7
>> +#endif
>> +#ifndef ID_AA64ZFR0_EL1
>> +#define ID_AA64ZFR0_EL1             S3_0_C0_C4_4
>> +#endif
>> +#ifndef ID_DFR1_EL1
>> +#define ID_DFR1_EL1                 S3_0_C0_C3_5
>> +#endif
>> +
>>  /* Access to system registers */
>>    #define READ_SYSREG32(name) ((uint32_t)READ_SYSREG64(name))
>> diff --git a/xen/include/asm-arm/cpregs.h b/xen/include/asm-arm/cpregs.h
>> index 8fd344146e..2690ddeb7a 100644
>> --- a/xen/include/asm-arm/cpregs.h
>> +++ b/xen/include/asm-arm/cpregs.h
>> @@ -63,6 +63,8 @@
>>  #define FPSID           p10,7,c0,c0,0   /* Floating-Point System ID Register */
>>  #define FPSCR           p10,7,c1,c0,0   /* Floating-Point Status and Control Register */
>>  #define MVFR0           p10,7,c7,c0,0   /* Media and VFP Feature Register 0 */
>> +#define MVFR1           p10,7,c6,c0,0   /* Media and VFP Feature Register 1 */
>> +#define MVFR2           p10,7,c5,c0,0   /* Media and VFP Feature Register 2 */
>>  #define FPEXC           p10,7,c8,c0,0   /* Floating-Point Exception Control Register */
>>  #define FPINST          p10,7,c9,c0,0   /* Floating-Point Instruction Register */
>>  #define FPINST2         p10,7,c10,c0,0  /* Floating-point Instruction Register 2 */
>> @@ -108,18 +110,23 @@
>>  #define MPIDR           p15,0,c0,c0,5   /* Multiprocessor Affinity Register */
>>  #define ID_PFR0         p15,0,c0,c1,0   /* Processor Feature Register 0 */
>>  #define ID_PFR1         p15,0,c0,c1,1   /* Processor Feature Register 1 */
>> +#define ID_PFR2         p15,0,c0,c3,4   /* Processor Feature Register 2 */
>>  #define ID_DFR0         p15,0,c0,c1,2   /* Debug Feature Register 0 */
>> +#define ID_DFR1         p15,0,c0,c3,5   /* Debug Feature Register 1 */
>>  #define ID_AFR0         p15,0,c0,c1,3   /* Auxiliary Feature Register 0 */
>>  #define ID_MMFR0        p15,0,c0,c1,4   /* Memory Model Feature Register 0 */
>>  #define ID_MMFR1        p15,0,c0,c1,5   /* Memory Model Feature Register 1 */
>>  #define ID_MMFR2        p15,0,c0,c1,6   /* Memory Model Feature Register 2 */
>>  #define ID_MMFR3        p15,0,c0,c1,7   /* Memory Model Feature Register 3 */
>> +#define ID_MMFR4        p15,0,c0,c2,6   /* Memory Model Feature Register 4 */
>> +#define ID_MMFR5        p15,0,c0,c3,6   /* Memory Model Feature Register 5 */
>>  #define ID_ISAR0        p15,0,c0,c2,0   /* ISA Feature Register 0 */
>>  #define ID_ISAR1        p15,0,c0,c2,1   /* ISA Feature Register 1 */
>>  #define ID_ISAR2        p15,0,c0,c2,2   /* ISA Feature Register 2 */
>>  #define ID_ISAR3        p15,0,c0,c2,3   /* ISA Feature Register 3 */
>>  #define ID_ISAR4        p15,0,c0,c2,4   /* ISA Feature Register 4 */
>>  #define ID_ISAR5        p15,0,c0,c2,5   /* ISA Feature Register 5 */
>> +#define ID_ISAR6        p15,0,c0,c2,7   /* ISA Feature Register 6 */
>>  #define CCSIDR          p15,1,c0,c0,0   /* Cache Size ID Registers */
>>  #define CLIDR           p15,1,c0,c0,1   /* Cache Level ID Register */
>>  #define CSSELR          p15,2,c0,c0,0   /* Cache Size Selection Register */
>> @@ -312,18 +319,23 @@
>>  #define HSTR_EL2                HSTR
>>  #define ID_AFR0_EL1             ID_AFR0
>>  #define ID_DFR0_EL1             ID_DFR0
>> +#define ID_DFR1_EL1             ID_DFR1
>>  #define ID_ISAR0_EL1            ID_ISAR0
>>  #define ID_ISAR1_EL1            ID_ISAR1
>>  #define ID_ISAR2_EL1            ID_ISAR2
>>  #define ID_ISAR3_EL1            ID_ISAR3
>>  #define ID_ISAR4_EL1            ID_ISAR4
>>  #define ID_ISAR5_EL1            ID_ISAR5
>> +#define ID_ISAR6_EL1            ID_ISAR6
>>  #define ID_MMFR0_EL1            ID_MMFR0
>>  #define ID_MMFR1_EL1            ID_MMFR1
>>  #define ID_MMFR2_EL1            ID_MMFR2
>>  #define ID_MMFR3_EL1            ID_MMFR3
>> +#define ID_MMFR4_EL1            ID_MMFR4
>> +#define ID_MMFR5_EL1            ID_MMFR5
>>  #define ID_PFR0_EL1             ID_PFR0
>>  #define ID_PFR1_EL1             ID_PFR1
>> +#define ID_PFR2_EL1             ID_PFR2
>>  #define IFSR32_EL2              IFSR
>>  #define MDCR_EL2                HDCR
>>  #define MIDR_EL1                MIDR
>> diff --git a/xen/include/asm-arm/cpufeature.h b/xen/include/asm-arm/cpufeature.h
>> index c7b5052992..6cf83d775b 100644
>> --- a/xen/include/asm-arm/cpufeature.h
>> +++ b/xen/include/asm-arm/cpufeature.h
>> @@ -148,6 +148,7 @@ struct cpuinfo_arm {
>>      union {
>>          uint64_t bits[2];
>>          struct {
>> +            /* PFR0 */
>>              unsigned long el0:4;
>>              unsigned long el1:4;
>>              unsigned long el2:4;
>> @@ -155,9 +156,23 @@ struct cpuinfo_arm {
>>              unsigned long fp:4;   /* Floating Point */
>>              unsigned long simd:4; /* Advanced SIMD */
>>              unsigned long gic:4;  /* GIC support */
>> -            unsigned long __res0:28;
>> +            unsigned long ras:4;
>> +            unsigned long sve:4;
>> +            unsigned long sel2:4;
>> +            unsigned long mpam:4;
>> +            unsigned long amu:4;
>> +            unsigned long dit:4;
>> +            unsigned long __res0:4;
>>              unsigned long csv2:4;
>> -            unsigned long __res1:4;
>> +            unsigned long cvs3:4;
>> +
>> +            /* PFR1 */
>> +            unsigned long bt:4;
>> +            unsigned long ssbs:4;
>> +            unsigned long mte:4;
>> +            unsigned long ras_frac:4;
>> +            unsigned long mpam_frac:4;
>> +            unsigned long __res1:44;
>>          };
>>      } pfr64;
>>  @@ -170,7 +185,7 @@ struct cpuinfo_arm {
>>      } aux64;
>>        union {
>> -        uint64_t bits[2];
>> +        uint64_t bits[3];
>>          struct {
>>              unsigned long pa_range:4;
>>              unsigned long asid_bits:4;
>> @@ -190,6 +205,8 @@ struct cpuinfo_arm {
>>              unsigned long pan:4;
>>              unsigned long __res1:8;
>>              unsigned long __res2:32;
>> +
>> +            unsigned long __res3:64;
>>          };
>>      } mm64;
>>  @@ -197,6 +214,10 @@ struct cpuinfo_arm {
>>          uint64_t bits[2];
>>      } isa64;
>>  +    struct {
>> +        uint64_t bits[1];
>> +    } zfr64;
>> +
>>  #endif
>>        /*
>> @@ -204,25 +225,38 @@ struct cpuinfo_arm {
>>       * when running in 32-bit mode.
>>       */
>>      union {
>> -        uint32_t bits[2];
>> +        uint32_t bits[3];
>>          struct {
>> +            /* PFR0 */
>>              unsigned long arm:4;
>>              unsigned long thumb:4;
>>              unsigned long jazelle:4;
>>              unsigned long thumbee:4;
>> -            unsigned long __res0:16;
>> +            unsigned long csv2:4;
>> +            unsigned long amu:4;
>> +            unsigned long dit:4;
>> +            unsigned long ras:4;
>>  +            /* PFR1 */
>>              unsigned long progmodel:4;
>>              unsigned long security:4;
>>              unsigned long mprofile:4;
>>              unsigned long virt:4;
>>              unsigned long gentimer:4;
>> -            unsigned long __res1:12;
>> +            unsigned long sec_frac:4;
>> +            unsigned long virt_frac:4;
>> +            unsigned long gic:4;
>> +
>> +            /* PFR2 */
>> +            unsigned long csv3:4;
>> +            unsigned long ssbs:4;
>> +            unsigned long ras_frac:4;
>> +            unsigned long __res2:20;
>>          };
>>      } pfr32;
>>        struct {
>> -        uint32_t bits[1];
>> +        uint32_t bits[2];
>>      } dbg32;
>>        struct {
>> @@ -230,12 +264,16 @@ struct cpuinfo_arm {
>>      } aux32;
>>        struct {
>> -        uint32_t bits[4];
>> +        uint32_t bits[6];
>>      } mm32;
>>        struct {
>> -        uint32_t bits[6];
>> +        uint32_t bits[7];
>>      } isa32;
>> +
>> +    struct {
>> +        uint64_t bits[3];
> 
> Shouldn't this be register_t?

I followed the scheme of the rest of the structure which
is always using uint64_t or uint32_t for bits definitions.

Why should I use register_t type for this one ?

Cheers
Bertrand

> 
>> +    } mvfr;
>>  };
>>    extern struct cpuinfo_arm boot_cpu_data;
> 
> -- 
> Julien Grall

Re: [PATCH v3 4/7] xen/arm: Add handler for ID registers on arm64

[Bertrand Marquis]

Hi Stefano,

> On 9 Dec 2020, at 19:38, Stefano Stabellini <sstabellini@kernel.org> wrote:
> 
> On Wed, 9 Dec 2020, Bertrand Marquis wrote:
>> Add vsysreg emulation for registers trapped when TID3 bit is activated
>> in HSR.
>> The emulation is returning the value stored in cpuinfo_guest structure
>> for know registers and is handling reserved registers as RAZ.
>> 
>> Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
>> ---
>> Changes in V2: Rebase
>> Changes in V3:
>>  Fix commit message
>>  Fix code style for GENERATE_TID3_INFO declaration
>>  Add handling of reserved registers as RAZ.
>> 
>> ---
>> xen/arch/arm/arm64/vsysreg.c | 53 ++++++++++++++++++++++++++++++++++++
>> 1 file changed, 53 insertions(+)
>> 
>> diff --git a/xen/arch/arm/arm64/vsysreg.c b/xen/arch/arm/arm64/vsysreg.c
>> index 8a85507d9d..ef7a11dbdd 100644
>> --- a/xen/arch/arm/arm64/vsysreg.c
>> +++ b/xen/arch/arm/arm64/vsysreg.c
>> @@ -69,6 +69,14 @@ TVM_REG(CONTEXTIDR_EL1)
>>         break;                                                          \
>>     }
>> 
>> +/* Macro to generate easily case for ID co-processor emulation */
>> +#define GENERATE_TID3_INFO(reg, field, offset)                          \
>> +    case HSR_SYSREG_##reg:                                              \
>> +    {                                                                   \
>> +        return handle_ro_read_val(regs, regidx, hsr.sysreg.read, hsr,   \
>> +                          1, guest_cpuinfo.field.bits[offset]);         \
> 
> [...]
> 
>> +    HSR_SYSREG_TID3_RESERVED_CASE:
>> +        /* Handle all reserved registers as RAZ */
>> +        return handle_ro_raz(regs, regidx, hsr.sysreg.read, hsr, 1);
> 
> 
> We are implementing both the known and the implementation defined
> registers as read-as-zero. On write, we inject an exception.
> 
> However, reading the manual, it looks like the implementation defined
> registers should be read-as-zero/write-ignore, is that right?

In the documentation, I did find all those defined as RO (Arm Architecture
reference manual, chapter D12.3.1). Do you think we should handle Read
only register as write ignore ? now i think of it RO does not explicitely mean
if writes are ignored or should generate an exception.

> 
> I couldn't easily find in the manual if it is OK to inject an exception
> on write to a known register.

I am actually unsure if it should or not.
I will try to run a test to check what is happening if this is done on the
real hardware and come back to you on this one.

Cheers
Bertrand

Re: [PATCH v3 4/7] xen/arm: Add handler for ID registers on arm64

[Bertrand Marquis]

Hi Julien,

> On 9 Dec 2020, at 23:13, Julien Grall <julien@xen.org> wrote:
> 
> 
> 
> On 09/12/2020 16:30, Bertrand Marquis wrote:
>> Add vsysreg emulation for registers trapped when TID3 bit is activated
>> in HSR.
>> The emulation is returning the value stored in cpuinfo_guest structure
>> for know registers and is handling reserved registers as RAZ.
>> Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
>> ---
>> Changes in V2: Rebase
>> Changes in V3:
>>   Fix commit message
>>   Fix code style for GENERATE_TID3_INFO declaration
>>   Add handling of reserved registers as RAZ.
>> ---
>>  xen/arch/arm/arm64/vsysreg.c | 53 ++++++++++++++++++++++++++++++++++++
>>  1 file changed, 53 insertions(+)
>> diff --git a/xen/arch/arm/arm64/vsysreg.c b/xen/arch/arm/arm64/vsysreg.c
>> index 8a85507d9d..ef7a11dbdd 100644
>> --- a/xen/arch/arm/arm64/vsysreg.c
>> +++ b/xen/arch/arm/arm64/vsysreg.c
>> @@ -69,6 +69,14 @@ TVM_REG(CONTEXTIDR_EL1)
>>          break;                                                          \
>>      }
>>  +/* Macro to generate easily case for ID co-processor emulation */
>> +#define GENERATE_TID3_INFO(reg, field, offset)                          \
>> +    case HSR_SYSREG_##reg:                                              \
>> +    {                                                                   \
>> +        return handle_ro_read_val(regs, regidx, hsr.sysreg.read, hsr,   \
>> +                          1, guest_cpuinfo.field.bits[offset]);         \
> 
> The indentation looks wrong here. The "1" should be aligned with "regs".

Right, I will fix that in v4.

Cheers
Bertrand

> 
>> +    }
>> +
>>  void do_sysreg(struct cpu_user_regs *regs,
>>                 const union hsr hsr)
>>  {
>> @@ -259,6 +267,51 @@ void do_sysreg(struct cpu_user_regs *regs,
>>           */
>>          return handle_raz_wi(regs, regidx, hsr.sysreg.read, hsr, 1);
>>  +    /*
>> +     * HCR_EL2.TID3
>> +     *
>> +     * This is trapping most Identification registers used by a guest
>> +     * to identify the processor features
>> +     */
>> +    GENERATE_TID3_INFO(ID_PFR0_EL1, pfr32, 0)
>> +    GENERATE_TID3_INFO(ID_PFR1_EL1, pfr32, 1)
>> +    GENERATE_TID3_INFO(ID_PFR2_EL1, pfr32, 2)
>> +    GENERATE_TID3_INFO(ID_DFR0_EL1, dbg32, 0)
>> +    GENERATE_TID3_INFO(ID_DFR1_EL1, dbg32, 1)
>> +    GENERATE_TID3_INFO(ID_AFR0_EL1, aux32, 0)
>> +    GENERATE_TID3_INFO(ID_MMFR0_EL1, mm32, 0)
>> +    GENERATE_TID3_INFO(ID_MMFR1_EL1, mm32, 1)
>> +    GENERATE_TID3_INFO(ID_MMFR2_EL1, mm32, 2)
>> +    GENERATE_TID3_INFO(ID_MMFR3_EL1, mm32, 3)
>> +    GENERATE_TID3_INFO(ID_MMFR4_EL1, mm32, 4)
>> +    GENERATE_TID3_INFO(ID_MMFR5_EL1, mm32, 5)
>> +    GENERATE_TID3_INFO(ID_ISAR0_EL1, isa32, 0)
>> +    GENERATE_TID3_INFO(ID_ISAR1_EL1, isa32, 1)
>> +    GENERATE_TID3_INFO(ID_ISAR2_EL1, isa32, 2)
>> +    GENERATE_TID3_INFO(ID_ISAR3_EL1, isa32, 3)
>> +    GENERATE_TID3_INFO(ID_ISAR4_EL1, isa32, 4)
>> +    GENERATE_TID3_INFO(ID_ISAR5_EL1, isa32, 5)
>> +    GENERATE_TID3_INFO(ID_ISAR6_EL1, isa32, 6)
>> +    GENERATE_TID3_INFO(MVFR0_EL1, mvfr, 0)
>> +    GENERATE_TID3_INFO(MVFR1_EL1, mvfr, 1)
>> +    GENERATE_TID3_INFO(MVFR2_EL1, mvfr, 2)
>> +    GENERATE_TID3_INFO(ID_AA64PFR0_EL1, pfr64, 0)
>> +    GENERATE_TID3_INFO(ID_AA64PFR1_EL1, pfr64, 1)
>> +    GENERATE_TID3_INFO(ID_AA64DFR0_EL1, dbg64, 0)
>> +    GENERATE_TID3_INFO(ID_AA64DFR1_EL1, dbg64, 1)
>> +    GENERATE_TID3_INFO(ID_AA64ISAR0_EL1, isa64, 0)
>> +    GENERATE_TID3_INFO(ID_AA64ISAR1_EL1, isa64, 1)
>> +    GENERATE_TID3_INFO(ID_AA64MMFR0_EL1, mm64, 0)
>> +    GENERATE_TID3_INFO(ID_AA64MMFR1_EL1, mm64, 1)
>> +    GENERATE_TID3_INFO(ID_AA64MMFR2_EL1, mm64, 2)
>> +    GENERATE_TID3_INFO(ID_AA64AFR0_EL1, aux64, 0)
>> +    GENERATE_TID3_INFO(ID_AA64AFR1_EL1, aux64, 1)
>> +    GENERATE_TID3_INFO(ID_AA64ZFR0_EL1, zfr64, 0)
>> +
>> +    HSR_SYSREG_TID3_RESERVED_CASE:
>> +        /* Handle all reserved registers as RAZ */
>> +        return handle_ro_raz(regs, regidx, hsr.sysreg.read, hsr, 1);
>> +
>>      /*
>>       * HCR_EL2.TIDCP
>>       *
> 
> -- 
> Julien Grall

Re: [PATCH v3 6/7] xen/arm: Add CP10 exception support to handle MVFR

[Bertrand Marquis]

Hi Stefano,

> On 9 Dec 2020, at 21:04, Stefano Stabellini <sstabellini@kernel.org> wrote:
> 
> On Wed, 9 Dec 2020, Bertrand Marquis wrote:
>> Add support for cp10 exceptions decoding to be able to emulate the
>> values for MVFR0, MVFR1 and MVFR2 when TID3 bit of HSR is activated.
>> This is required for aarch32 guests accessing MVFR registers using
>> vmrs and vmsr instructions.
>> 
>> Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
>> ---
>> Changes in V2: Rebase
>> Changes in V3:
>>  Add case for MVFR2, fix typo VMFR <-> MVFR.
>> 
>> ---
>> xen/arch/arm/traps.c             |  5 ++++
>> xen/arch/arm/vcpreg.c            | 39 +++++++++++++++++++++++++++++++-
>> xen/include/asm-arm/perfc_defn.h |  1 +
>> xen/include/asm-arm/traps.h      |  1 +
>> 4 files changed, 45 insertions(+), 1 deletion(-)
>> 
>> diff --git a/xen/arch/arm/traps.c b/xen/arch/arm/traps.c
>> index 22bd1bd4c6..28d9d64558 100644
>> --- a/xen/arch/arm/traps.c
>> +++ b/xen/arch/arm/traps.c
>> @@ -2097,6 +2097,11 @@ void do_trap_guest_sync(struct cpu_user_regs *regs)
>>         perfc_incr(trap_cp14_dbg);
>>         do_cp14_dbg(regs, hsr);
>>         break;
>> +    case HSR_EC_CP10:
>> +        GUEST_BUG_ON(!psr_mode_is_32bit(regs));
>> +        perfc_incr(trap_cp10);
>> +        do_cp10(regs, hsr);
>> +        break;
>>     case HSR_EC_CP:
>>         GUEST_BUG_ON(!psr_mode_is_32bit(regs));
>>         perfc_incr(trap_cp);
>> diff --git a/xen/arch/arm/vcpreg.c b/xen/arch/arm/vcpreg.c
>> index d371a1c38c..da4e22a467 100644
>> --- a/xen/arch/arm/vcpreg.c
>> +++ b/xen/arch/arm/vcpreg.c
>> @@ -319,7 +319,7 @@ void do_cp15_32(struct cpu_user_regs *regs, const union hsr hsr)
>>     GENERATE_TID3_INFO(ID_ISAR4, isa32, 4)
>>     GENERATE_TID3_INFO(ID_ISAR5, isa32, 5)
>>     GENERATE_TID3_INFO(ID_ISAR6, isa32, 6)
>> -    /* MVFR registers are in cp10 no cp15 */
>> +    /* MVFR registers are in cp10 not cp15 */
>> 
>>     HSR_CPREG32_TID3_RESERVED_CASE:
>>         /* Handle all reserved registers as RAZ */
>> @@ -638,6 +638,43 @@ void do_cp14_dbg(struct cpu_user_regs *regs, const union hsr hsr)
>>     inject_undef_exception(regs, hsr);
>> }
>> 
>> +void do_cp10(struct cpu_user_regs *regs, const union hsr hsr)
>> +{
>> +    const struct hsr_cp32 cp32 = hsr.cp32;
>> +    int regidx = cp32.reg;
>> +
>> +    if ( !check_conditional_instr(regs, hsr) )
>> +    {
>> +        advance_pc(regs, hsr);
>> +        return;
>> +    }
>> +
>> +    switch ( hsr.bits & HSR_CP32_REGS_MASK )
>> +    {
>> +    /*
>> +     * HSR.TID3 is trapping access to MVFR register used to identify the
>          ^ HCR

ack, will fix the typo in v4.

> 
>> +     * VFP/Simd using VMRS/VMSR instructions.
>> +     * Exception encoding is using MRC/MCR standard with the reg field in Crn
>> +     * as are declared MVFR0 and MVFR1 in cpregs.h
>> +     */
>> +    GENERATE_TID3_INFO(MVFR0, mvfr, 0)
>> +    GENERATE_TID3_INFO(MVFR1, mvfr, 1)
>> +    GENERATE_TID3_INFO(MVFR2, mvfr, 2)
>> +
>> +    default:
>> +        gdprintk(XENLOG_ERR,
>> +                 "%s p10, %d, r%d, cr%d, cr%d, %d @ 0x%"PRIregister"\n",
>> +                 cp32.read ? "mrc" : "mcr",
>> +                 cp32.op1, cp32.reg, cp32.crn, cp32.crm, cp32.op2, regs->pc);
>> +        gdprintk(XENLOG_ERR, "unhandled 32-bit CP10 access %#x\n",
>> +                 hsr.bits & HSR_CP32_REGS_MASK);
>> +        inject_undef_exception(regs, hsr);
>> +        return;
> 
> I take we are sure there are no other cp10 registers of interest?

Documentation is saying:
"VMRS access to MVFR0, MVFR1, and MVFR2, are trapped to EL2, reported using EC
syndrome value 0x08"

So this is my understanding yes.

Cheers
Bertrand

> 
> 
>> +    }
>> +
>> +    advance_pc(regs, hsr);
>> +}
>> +
>> void do_cp(struct cpu_user_regs *regs, const union hsr hsr)
>> {
>>     const struct hsr_cp cp = hsr.cp;
>> diff --git a/xen/include/asm-arm/perfc_defn.h b/xen/include/asm-arm/perfc_defn.h
>> index 6a83185163..31f071222b 100644
>> --- a/xen/include/asm-arm/perfc_defn.h
>> +++ b/xen/include/asm-arm/perfc_defn.h
>> @@ -11,6 +11,7 @@ PERFCOUNTER(trap_cp15_64,  "trap: cp15 64-bit access")
>> PERFCOUNTER(trap_cp14_32,  "trap: cp14 32-bit access")
>> PERFCOUNTER(trap_cp14_64,  "trap: cp14 64-bit access")
>> PERFCOUNTER(trap_cp14_dbg, "trap: cp14 dbg access")
>> +PERFCOUNTER(trap_cp10,     "trap: cp10 access")
>> PERFCOUNTER(trap_cp,       "trap: cp access")
>> PERFCOUNTER(trap_smc32,    "trap: 32-bit smc")
>> PERFCOUNTER(trap_hvc32,    "trap: 32-bit hvc")
>> diff --git a/xen/include/asm-arm/traps.h b/xen/include/asm-arm/traps.h
>> index 997c37884e..c4a3d0fb1b 100644
>> --- a/xen/include/asm-arm/traps.h
>> +++ b/xen/include/asm-arm/traps.h
>> @@ -62,6 +62,7 @@ void do_cp15_64(struct cpu_user_regs *regs, const union hsr hsr);
>> void do_cp14_32(struct cpu_user_regs *regs, const union hsr hsr);
>> void do_cp14_64(struct cpu_user_regs *regs, const union hsr hsr);
>> void do_cp14_dbg(struct cpu_user_regs *regs, const union hsr hsr);
>> +void do_cp10(struct cpu_user_regs *regs, const union hsr hsr);
>> void do_cp(struct cpu_user_regs *regs, const union hsr hsr);
>> 
>> /* SMCCC handling */
>> -- 
>> 2.17.1

Re: [PATCH v3 6/7] xen/arm: Add CP10 exception support to handle MVFR

[Bertrand Marquis]

Hi Julien

> On 9 Dec 2020, at 23:15, Julien Grall <julien@xen.org> wrote:
> 
> Hi Bertrand,
> 
> On 09/12/2020 16:30, Bertrand Marquis wrote:
>> Add support for cp10 exceptions decoding to be able to emulate the
>> values for MVFR0, MVFR1 and MVFR2 when TID3 bit of HSR is activated.
>> This is required for aarch32 guests accessing MVFR registers using
>> vmrs and vmsr instructions.
>> Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
>> ---
>> Changes in V2: Rebase
>> Changes in V3:
>>   Add case for MVFR2, fix typo VMFR <-> MVFR.
>> ---
>>  xen/arch/arm/traps.c             |  5 ++++
>>  xen/arch/arm/vcpreg.c            | 39 +++++++++++++++++++++++++++++++-
>>  xen/include/asm-arm/perfc_defn.h |  1 +
>>  xen/include/asm-arm/traps.h      |  1 +
>>  4 files changed, 45 insertions(+), 1 deletion(-)
>> diff --git a/xen/arch/arm/traps.c b/xen/arch/arm/traps.c
>> index 22bd1bd4c6..28d9d64558 100644
>> --- a/xen/arch/arm/traps.c
>> +++ b/xen/arch/arm/traps.c
>> @@ -2097,6 +2097,11 @@ void do_trap_guest_sync(struct cpu_user_regs *regs)
>>          perfc_incr(trap_cp14_dbg);
>>          do_cp14_dbg(regs, hsr);
>>          break;
>> +    case HSR_EC_CP10:
>> +        GUEST_BUG_ON(!psr_mode_is_32bit(regs));
>> +        perfc_incr(trap_cp10);
>> +        do_cp10(regs, hsr);
>> +        break;
>>      case HSR_EC_CP:
>>          GUEST_BUG_ON(!psr_mode_is_32bit(regs));
>>          perfc_incr(trap_cp);
>> diff --git a/xen/arch/arm/vcpreg.c b/xen/arch/arm/vcpreg.c
>> index d371a1c38c..da4e22a467 100644
>> --- a/xen/arch/arm/vcpreg.c
>> +++ b/xen/arch/arm/vcpreg.c
>> @@ -319,7 +319,7 @@ void do_cp15_32(struct cpu_user_regs *regs, const union hsr hsr)
>>      GENERATE_TID3_INFO(ID_ISAR4, isa32, 4)
>>      GENERATE_TID3_INFO(ID_ISAR5, isa32, 5)
>>      GENERATE_TID3_INFO(ID_ISAR6, isa32, 6)
>> -    /* MVFR registers are in cp10 no cp15 */
>> +    /* MVFR registers are in cp10 not cp15 */
> 
> The code was originally added in the previous patch. Please either introduce the comment here or fold it in the previous patch.

Right i will fold it back in previous patch.

Regards
Bertrand

> 
>>        HSR_CPREG32_TID3_RESERVED_CASE:
>>          /* Handle all reserved registers as RAZ */
>> @@ -638,6 +638,43 @@ void do_cp14_dbg(struct cpu_user_regs *regs, const union hsr hsr)
>>      inject_undef_exception(regs, hsr);
>>  }
>>  +void do_cp10(struct cpu_user_regs *regs, const union hsr hsr)
>> +{
>> +    const struct hsr_cp32 cp32 = hsr.cp32;
>> +    int regidx = cp32.reg;
>> +
>> +    if ( !check_conditional_instr(regs, hsr) )
>> +    {
>> +        advance_pc(regs, hsr);
>> +        return;
>> +    }
>> +
>> +    switch ( hsr.bits & HSR_CP32_REGS_MASK )
>> +    {
>> +    /*
>> +     * HSR.TID3 is trapping access to MVFR register used to identify the
>> +     * VFP/Simd using VMRS/VMSR instructions.
>> +     * Exception encoding is using MRC/MCR standard with the reg field in Crn
>> +     * as are declared MVFR0 and MVFR1 in cpregs.h
>> +     */
>> +    GENERATE_TID3_INFO(MVFR0, mvfr, 0)
>> +    GENERATE_TID3_INFO(MVFR1, mvfr, 1)
>> +    GENERATE_TID3_INFO(MVFR2, mvfr, 2)
>> +
>> +    default:
>> +        gdprintk(XENLOG_ERR,
>> +                 "%s p10, %d, r%d, cr%d, cr%d, %d @ 0x%"PRIregister"\n",
>> +                 cp32.read ? "mrc" : "mcr",
>> +                 cp32.op1, cp32.reg, cp32.crn, cp32.crm, cp32.op2, regs->pc);
>> +        gdprintk(XENLOG_ERR, "unhandled 32-bit CP10 access %#x\n",
>> +                 hsr.bits & HSR_CP32_REGS_MASK);
>> +        inject_undef_exception(regs, hsr);
>> +        return;
>> +    }
>> +
>> +    advance_pc(regs, hsr);
>> +}
>> +
>>  void do_cp(struct cpu_user_regs *regs, const union hsr hsr)
>>  {
>>      const struct hsr_cp cp = hsr.cp;
>> diff --git a/xen/include/asm-arm/perfc_defn.h b/xen/include/asm-arm/perfc_defn.h
>> index 6a83185163..31f071222b 100644
>> --- a/xen/include/asm-arm/perfc_defn.h
>> +++ b/xen/include/asm-arm/perfc_defn.h
>> @@ -11,6 +11,7 @@ PERFCOUNTER(trap_cp15_64,  "trap: cp15 64-bit access")
>>  PERFCOUNTER(trap_cp14_32,  "trap: cp14 32-bit access")
>>  PERFCOUNTER(trap_cp14_64,  "trap: cp14 64-bit access")
>>  PERFCOUNTER(trap_cp14_dbg, "trap: cp14 dbg access")
>> +PERFCOUNTER(trap_cp10,     "trap: cp10 access")
>>  PERFCOUNTER(trap_cp,       "trap: cp access")
>>  PERFCOUNTER(trap_smc32,    "trap: 32-bit smc")
>>  PERFCOUNTER(trap_hvc32,    "trap: 32-bit hvc")
>> diff --git a/xen/include/asm-arm/traps.h b/xen/include/asm-arm/traps.h
>> index 997c37884e..c4a3d0fb1b 100644
>> --- a/xen/include/asm-arm/traps.h
>> +++ b/xen/include/asm-arm/traps.h
>> @@ -62,6 +62,7 @@ void do_cp15_64(struct cpu_user_regs *regs, const union hsr hsr);
>>  void do_cp14_32(struct cpu_user_regs *regs, const union hsr hsr);
>>  void do_cp14_64(struct cpu_user_regs *regs, const union hsr hsr);
>>  void do_cp14_dbg(struct cpu_user_regs *regs, const union hsr hsr);
>> +void do_cp10(struct cpu_user_regs *regs, const union hsr hsr);
>>  void do_cp(struct cpu_user_regs *regs, const union hsr hsr);
>>    /* SMCCC handling */
> 
> -- 
> Julien Grall

Re: [PATCH v3 7/7] xen/arm: Activate TID3 in HCR_EL2

[Bertrand Marquis]

Hi Julien,

> On 9 Dec 2020, at 23:17, Julien Grall <julien@xen.org> wrote:
> 
> Hi Bertrand,
> 
> On 09/12/2020 16:31, Bertrand Marquis wrote:
>> Activate TID3 bit in HSR register when starting a guest.
> 
> s/HSR/HCR/
> 

Right, I did it a lot thanks for the review.
I will fix that in V4.

>> This will trap all coprecessor ID registers so that we can give to guest
>> values corresponding to what they can actually use and mask some
>> features to guests even though they would be supported by the underlying
>> hardware (like SVE or MPAM).
> 
> So this will make sure the guest will not be able to identify the feature. Did you check that the features are effectively not accessible by the guest? IOW it should trap.

For SVE yes I checked and with the serie a Linux kernel with SVE support activated on a target with SVE is now working (was crashing before).
For MPAM, I have no target available with MPAM support so I could not test that but your recent XSA patch did turn the access to the guest off.

With my SVE test, I could confirm that access are trapped and properly emulated.

Cheers
Bertrand

> 
> Cheers,
> 
> -- 
> Julien Grall
> 

Re: [PATCH v3 1/7] xen/arm: Add ID registers and complete cpuinfo

[Julien Grall]

Hi Bertrand,

On 10/12/2020 15:14, Bertrand Marquis wrote:
> Hi Julien,
> 
>> On 9 Dec 2020, at 23:03, Julien Grall <julien@xen.org> wrote:
>>
>> Hi Bertrand,
>>
>> On 09/12/2020 16:30, Bertrand Marquis wrote:
>>> Add definition and entries in cpuinfo for ID registers introduced in
>>> newer Arm Architecture reference manual:
>>> - ID_PFR2: processor feature register 2
>>> - ID_DFR1: debug feature register 1
>>> - ID_MMFR4 and ID_MMFR5: Memory model feature registers 4 and 5
>>> - ID_ISA6: ISA Feature register 6
>>> Add more bitfield definitions in PFR fields of cpuinfo.
>>> Add MVFR2 register definition for aarch32.
>>> Add mvfr values in cpuinfo.
>>> Add some registers definition for arm64 in sysregs as some are not
>>> always know by compilers.
>>> Initialize the new values added in cpuinfo in identify_cpu during init.
>>> Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
>>> ---
>>> Changes in V2:
>>>    Fix dbg32 table size and add proper initialisation of the second entry
>>>    of the table by reading ID_DFR1 register.
>>> Changes in V3:
>>>    Fix typo in commit title
>>>    Add MVFR2 definition and handling on aarch32 and remove specific case
>>>    for mvfr field in cpuinfo (now the same on arm64 and arm32).
>>>    Add MMFR4 definition if not known by the compiler.
>>> ---
>>>   xen/arch/arm/cpufeature.c           | 18 ++++++++++
>>>   xen/include/asm-arm/arm64/sysregs.h | 28 +++++++++++++++
>>>   xen/include/asm-arm/cpregs.h        | 12 +++++++
>>>   xen/include/asm-arm/cpufeature.h    | 56 ++++++++++++++++++++++++-----
>>>   4 files changed, 105 insertions(+), 9 deletions(-)
>>> diff --git a/xen/arch/arm/cpufeature.c b/xen/arch/arm/cpufeature.c
>>> index 44126dbf07..bc7ee5ac95 100644
>>> --- a/xen/arch/arm/cpufeature.c
>>> +++ b/xen/arch/arm/cpufeature.c
>>> @@ -114,15 +114,20 @@ void identify_cpu(struct cpuinfo_arm *c)
>>>             c->mm64.bits[0]  = READ_SYSREG64(ID_AA64MMFR0_EL1);
>>>           c->mm64.bits[1]  = READ_SYSREG64(ID_AA64MMFR1_EL1);
>>> +        c->mm64.bits[2]  = READ_SYSREG64(ID_AA64MMFR2_EL1);
>>>             c->isa64.bits[0] = READ_SYSREG64(ID_AA64ISAR0_EL1);
>>>           c->isa64.bits[1] = READ_SYSREG64(ID_AA64ISAR1_EL1);
>>> +
>>> +        c->zfr64.bits[0] = READ_SYSREG64(ID_AA64ZFR0_EL1);
>>>   #endif
>>>             c->pfr32.bits[0] = READ_SYSREG32(ID_PFR0_EL1);
>>>           c->pfr32.bits[1] = READ_SYSREG32(ID_PFR1_EL1);
>>> +        c->pfr32.bits[2] = READ_SYSREG32(ID_PFR2_EL1);
>>>             c->dbg32.bits[0] = READ_SYSREG32(ID_DFR0_EL1);
>>> +        c->dbg32.bits[1] = READ_SYSREG32(ID_DFR1_EL1);
>>>             c->aux32.bits[0] = READ_SYSREG32(ID_AFR0_EL1);
>>>   @@ -130,6 +135,8 @@ void identify_cpu(struct cpuinfo_arm *c)
>>>           c->mm32.bits[1]  = READ_SYSREG32(ID_MMFR1_EL1);
>>>           c->mm32.bits[2]  = READ_SYSREG32(ID_MMFR2_EL1);
>>>           c->mm32.bits[3]  = READ_SYSREG32(ID_MMFR3_EL1);
>>> +        c->mm32.bits[4]  = READ_SYSREG32(ID_MMFR4_EL1);
>>> +        c->mm32.bits[5]  = READ_SYSREG32(ID_MMFR5_EL1);
>>
>> Please don't introduce any more use of READ_SYSREG32(), they are wrong on Armv8 because system registers are always 64-bit.
> 
> I followed the existing implementation but ...
> 
>>
>>>             c->isa32.bits[0] = READ_SYSREG32(ID_ISAR0_EL1);
>>>           c->isa32.bits[1] = READ_SYSREG32(ID_ISAR1_EL1);
>>> @@ -137,6 +144,17 @@ void identify_cpu(struct cpuinfo_arm *c)
>>>           c->isa32.bits[3] = READ_SYSREG32(ID_ISAR3_EL1);
>>>           c->isa32.bits[4] = READ_SYSREG32(ID_ISAR4_EL1);
>>>           c->isa32.bits[5] = READ_SYSREG32(ID_ISAR5_EL1);
>>> +        c->isa32.bits[6] = READ_SYSREG32(ID_ISAR6_EL1);
>>> +
>>> +#ifdef CONFIG_ARM_64
>>> +        c->mvfr.bits[0] = READ_SYSREG64(MVFR0_EL1);
>>> +        c->mvfr.bits[1] = READ_SYSREG64(MVFR1_EL1);
>>> +        c->mvfr.bits[2] = READ_SYSREG64(MVFR2_EL1);
>>> +#else
>>> +        c->mvfr.bits[0] = READ_CP32(MVFR0);
>>> +        c->mvfr.bits[1] = READ_CP32(MVFR1);
>>> +        c->mvfr.bits[2] = READ_CP32(MVFR2);
>>> +#endif
>>
>> READ_SYSREG() will do the job to either use READ_SYSREG64() or READ_CP32() depending on the arch used.
> 
> .. I can modify the ones I added and the existing ones to user READ_SYSREG instead.
> Please confirm if you want me to do that.

Yes, please use READ_SYSREG() for the new ones. We can convert the 
others separately.

> 
>>
>>>   }
>>>     /*
>>> diff --git a/xen/include/asm-arm/arm64/sysregs.h b/xen/include/asm-arm/arm64/sysregs.h
>>> index c60029d38f..077fd95fb7 100644
>>> --- a/xen/include/asm-arm/arm64/sysregs.h
>>> +++ b/xen/include/asm-arm/arm64/sysregs.h
>>> @@ -57,6 +57,34 @@
>>>   #define ICH_AP1R2_EL2             __AP1Rx_EL2(2)
>>>   #define ICH_AP1R3_EL2             __AP1Rx_EL2(3)
>>>   +/*
>>> + * Define ID coprocessor registers if they are not
>>> + * already defined by the compiler.
>>> + *
>>> + * Values picked from linux kernel
>>> + */
>>> +#ifndef ID_AA64MMFR2_EL1
>>
>> I am a bit puzzled how this meant to work. Will the libc/compiler headers define ID_AA64MMFR2_EL1?
> 
> I tested this and if the compiler has a definition for the register, I am not entering the ifndef.
> So there is no header defining this but if the compiler has the definition for this the ifndef will
> not be entered.

Good to hear :).

> 
>>
>>> +#define ID_AA64MMFR2_EL1            S3_0_C0_C7_2
>>> +#endif
>>> +#ifndef ID_PFR2_EL1
>>> +#define ID_PFR2_EL1                 S3_0_C0_C3_4
>>> +#endif
>>> +#ifndef ID_MMFR4_EL1
>>> +#define ID_MMFR4_EL1                S3_0_C0_C2_6
>>> +#endif
>>> +#ifndef ID_MMFR5_EL1
>>> +#define ID_MMFR5_EL1                S3_0_C0_C3_6
>>> +#endif
>>> +#ifndef ID_ISAR6_EL1
>>> +#define ID_ISAR6_EL1                S3_0_C0_C2_7
>>> +#endif
>>> +#ifndef ID_AA64ZFR0_EL1
>>> +#define ID_AA64ZFR0_EL1             S3_0_C0_C4_4
>>> +#endif
>>> +#ifndef ID_DFR1_EL1
>>> +#define ID_DFR1_EL1                 S3_0_C0_C3_5
>>> +#endif
>>> +
>>>   /* Access to system registers */
>>>     #define READ_SYSREG32(name) ((uint32_t)READ_SYSREG64(name))
>>> diff --git a/xen/include/asm-arm/cpregs.h b/xen/include/asm-arm/cpregs.h
>>> index 8fd344146e..2690ddeb7a 100644
>>> --- a/xen/include/asm-arm/cpregs.h
>>> +++ b/xen/include/asm-arm/cpregs.h
>>> @@ -63,6 +63,8 @@
>>>   #define FPSID           p10,7,c0,c0,0   /* Floating-Point System ID Register */
>>>   #define FPSCR           p10,7,c1,c0,0   /* Floating-Point Status and Control Register */
>>>   #define MVFR0           p10,7,c7,c0,0   /* Media and VFP Feature Register 0 */
>>> +#define MVFR1           p10,7,c6,c0,0   /* Media and VFP Feature Register 1 */
>>> +#define MVFR2           p10,7,c5,c0,0   /* Media and VFP Feature Register 2 */
>>>   #define FPEXC           p10,7,c8,c0,0   /* Floating-Point Exception Control Register */
>>>   #define FPINST          p10,7,c9,c0,0   /* Floating-Point Instruction Register */
>>>   #define FPINST2         p10,7,c10,c0,0  /* Floating-point Instruction Register 2 */
>>> @@ -108,18 +110,23 @@
>>>   #define MPIDR           p15,0,c0,c0,5   /* Multiprocessor Affinity Register */
>>>   #define ID_PFR0         p15,0,c0,c1,0   /* Processor Feature Register 0 */
>>>   #define ID_PFR1         p15,0,c0,c1,1   /* Processor Feature Register 1 */
>>> +#define ID_PFR2         p15,0,c0,c3,4   /* Processor Feature Register 2 */
>>>   #define ID_DFR0         p15,0,c0,c1,2   /* Debug Feature Register 0 */
>>> +#define ID_DFR1         p15,0,c0,c3,5   /* Debug Feature Register 1 */
>>>   #define ID_AFR0         p15,0,c0,c1,3   /* Auxiliary Feature Register 0 */
>>>   #define ID_MMFR0        p15,0,c0,c1,4   /* Memory Model Feature Register 0 */
>>>   #define ID_MMFR1        p15,0,c0,c1,5   /* Memory Model Feature Register 1 */
>>>   #define ID_MMFR2        p15,0,c0,c1,6   /* Memory Model Feature Register 2 */
>>>   #define ID_MMFR3        p15,0,c0,c1,7   /* Memory Model Feature Register 3 */
>>> +#define ID_MMFR4        p15,0,c0,c2,6   /* Memory Model Feature Register 4 */
>>> +#define ID_MMFR5        p15,0,c0,c3,6   /* Memory Model Feature Register 5 */
>>>   #define ID_ISAR0        p15,0,c0,c2,0   /* ISA Feature Register 0 */
>>>   #define ID_ISAR1        p15,0,c0,c2,1   /* ISA Feature Register 1 */
>>>   #define ID_ISAR2        p15,0,c0,c2,2   /* ISA Feature Register 2 */
>>>   #define ID_ISAR3        p15,0,c0,c2,3   /* ISA Feature Register 3 */
>>>   #define ID_ISAR4        p15,0,c0,c2,4   /* ISA Feature Register 4 */
>>>   #define ID_ISAR5        p15,0,c0,c2,5   /* ISA Feature Register 5 */
>>> +#define ID_ISAR6        p15,0,c0,c2,7   /* ISA Feature Register 6 */
>>>   #define CCSIDR          p15,1,c0,c0,0   /* Cache Size ID Registers */
>>>   #define CLIDR           p15,1,c0,c0,1   /* Cache Level ID Register */
>>>   #define CSSELR          p15,2,c0,c0,0   /* Cache Size Selection Register */
>>> @@ -312,18 +319,23 @@
>>>   #define HSTR_EL2                HSTR
>>>   #define ID_AFR0_EL1             ID_AFR0
>>>   #define ID_DFR0_EL1             ID_DFR0
>>> +#define ID_DFR1_EL1             ID_DFR1
>>>   #define ID_ISAR0_EL1            ID_ISAR0
>>>   #define ID_ISAR1_EL1            ID_ISAR1
>>>   #define ID_ISAR2_EL1            ID_ISAR2
>>>   #define ID_ISAR3_EL1            ID_ISAR3
>>>   #define ID_ISAR4_EL1            ID_ISAR4
>>>   #define ID_ISAR5_EL1            ID_ISAR5
>>> +#define ID_ISAR6_EL1            ID_ISAR6
>>>   #define ID_MMFR0_EL1            ID_MMFR0
>>>   #define ID_MMFR1_EL1            ID_MMFR1
>>>   #define ID_MMFR2_EL1            ID_MMFR2
>>>   #define ID_MMFR3_EL1            ID_MMFR3
>>> +#define ID_MMFR4_EL1            ID_MMFR4
>>> +#define ID_MMFR5_EL1            ID_MMFR5
>>>   #define ID_PFR0_EL1             ID_PFR0
>>>   #define ID_PFR1_EL1             ID_PFR1
>>> +#define ID_PFR2_EL1             ID_PFR2
>>>   #define IFSR32_EL2              IFSR
>>>   #define MDCR_EL2                HDCR
>>>   #define MIDR_EL1                MIDR
>>> diff --git a/xen/include/asm-arm/cpufeature.h b/xen/include/asm-arm/cpufeature.h
>>> index c7b5052992..6cf83d775b 100644
>>> --- a/xen/include/asm-arm/cpufeature.h
>>> +++ b/xen/include/asm-arm/cpufeature.h
>>> @@ -148,6 +148,7 @@ struct cpuinfo_arm {
>>>       union {
>>>           uint64_t bits[2];
>>>           struct {
>>> +            /* PFR0 */
>>>               unsigned long el0:4;
>>>               unsigned long el1:4;
>>>               unsigned long el2:4;
>>> @@ -155,9 +156,23 @@ struct cpuinfo_arm {
>>>               unsigned long fp:4;   /* Floating Point */
>>>               unsigned long simd:4; /* Advanced SIMD */
>>>               unsigned long gic:4;  /* GIC support */
>>> -            unsigned long __res0:28;
>>> +            unsigned long ras:4;
>>> +            unsigned long sve:4;
>>> +            unsigned long sel2:4;
>>> +            unsigned long mpam:4;
>>> +            unsigned long amu:4;
>>> +            unsigned long dit:4;
>>> +            unsigned long __res0:4;
>>>               unsigned long csv2:4;
>>> -            unsigned long __res1:4;
>>> +            unsigned long cvs3:4;
>>> +
>>> +            /* PFR1 */
>>> +            unsigned long bt:4;
>>> +            unsigned long ssbs:4;
>>> +            unsigned long mte:4;
>>> +            unsigned long ras_frac:4;
>>> +            unsigned long mpam_frac:4;
>>> +            unsigned long __res1:44;
>>>           };
>>>       } pfr64;
>>>   @@ -170,7 +185,7 @@ struct cpuinfo_arm {
>>>       } aux64;
>>>         union {
>>> -        uint64_t bits[2];
>>> +        uint64_t bits[3];
>>>           struct {
>>>               unsigned long pa_range:4;
>>>               unsigned long asid_bits:4;
>>> @@ -190,6 +205,8 @@ struct cpuinfo_arm {
>>>               unsigned long pan:4;
>>>               unsigned long __res1:8;
>>>               unsigned long __res2:32;
>>> +
>>> +            unsigned long __res3:64;
>>>           };
>>>       } mm64;
>>>   @@ -197,6 +214,10 @@ struct cpuinfo_arm {
>>>           uint64_t bits[2];
>>>       } isa64;
>>>   +    struct {
>>> +        uint64_t bits[1];
>>> +    } zfr64;
>>> +
>>>   #endif
>>>         /*
>>> @@ -204,25 +225,38 @@ struct cpuinfo_arm {
>>>        * when running in 32-bit mode.
>>>        */
>>>       union {
>>> -        uint32_t bits[2];
>>> +        uint32_t bits[3];
>>>           struct {
>>> +            /* PFR0 */
>>>               unsigned long arm:4;
>>>               unsigned long thumb:4;
>>>               unsigned long jazelle:4;
>>>               unsigned long thumbee:4;
>>> -            unsigned long __res0:16;
>>> +            unsigned long csv2:4;
>>> +            unsigned long amu:4;
>>> +            unsigned long dit:4;
>>> +            unsigned long ras:4;
>>>   +            /* PFR1 */
>>>               unsigned long progmodel:4;
>>>               unsigned long security:4;
>>>               unsigned long mprofile:4;
>>>               unsigned long virt:4;
>>>               unsigned long gentimer:4;
>>> -            unsigned long __res1:12;
>>> +            unsigned long sec_frac:4;
>>> +            unsigned long virt_frac:4;
>>> +            unsigned long gic:4;
>>> +
>>> +            /* PFR2 */
>>> +            unsigned long csv3:4;
>>> +            unsigned long ssbs:4;
>>> +            unsigned long ras_frac:4;
>>> +            unsigned long __res2:20;
>>>           };
>>>       } pfr32;
>>>         struct {
>>> -        uint32_t bits[1];
>>> +        uint32_t bits[2];
>>>       } dbg32;
>>>         struct {
>>> @@ -230,12 +264,16 @@ struct cpuinfo_arm {
>>>       } aux32;
>>>         struct {
>>> -        uint32_t bits[4];
>>> +        uint32_t bits[6];
>>>       } mm32;
>>>         struct {
>>> -        uint32_t bits[6];
>>> +        uint32_t bits[7];
>>>       } isa32;
>>> +
>>> +    struct {
>>> +        uint64_t bits[3];
>>
>> Shouldn't this be register_t?
> 
> I followed the scheme of the rest of the structure which
> is always using uint64_t or uint32_t for bits definitions.

Right, but I am sure you will not be surprised if I tell you this is 
buggy ;). The historical reason is, IIRC, the original spec of Armv8.0 
described them as 32-bit registers.

The spec was updated a while ago to clarify they are 64-bit when running 
in AArch64. But a majority of them still have the top 32-bit RES0 
(thankfully!).

> 
> Why should I use register_t type for this one ?

Because the value is 32-bit on AArch32 and 64-bit for AArch64. I am ok 
to use 64-bit still for AArch32, but it sounds a bit of a waste of memory.

What I care the most here is we use 64-bit for the new registers on 
AArch64. Otherwise, we are going to soon discover that a bit above 32 
was allocated and not detected by Xen. I don't want to be the one doing 
the debugging!

Admittly, this is not a new issue. However, the more offending code we 
had the more it will be difficul to get Xen to be fully compliant with 
the Armv8 spec.

Cheers,

-- 
Julien Grall

Re: [PATCH v3 2/7] xen/arm: Add arm64 ID registers definitions

[Julien Grall]

On 10/12/2020 02:30, Stefano Stabellini wrote:
> On Wed, 9 Dec 2020, Julien Grall wrote:
>> Hi Bertrand,
>>
>> On 09/12/2020 16:30, Bertrand Marquis wrote:
>>> Add coprocessor registers definitions for all ID registers trapped
>>> through the TID3 bit of HSR.
>>> Those are the one that will be emulated in Xen to only publish to guests
>>> the features that are supported by Xen and that are accessible to
>>> guests.
>>> Also define a case to catch all reserved registers that should be
>>> handled as RAZ.
>>>
>>> Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
>>> ---
>>> Changes in V2: Rebase
>>> Changes in V3:
>>>     Add case definition for reserved registers.
>>>
>>> ---
>>>    xen/include/asm-arm/arm64/hsr.h | 66 +++++++++++++++++++++++++++++++++
>>>    1 file changed, 66 insertions(+)
>>>
>>> diff --git a/xen/include/asm-arm/arm64/hsr.h
>>> b/xen/include/asm-arm/arm64/hsr.h
>>> index ca931dd2fe..ffe0f0007e 100644
>>> --- a/xen/include/asm-arm/arm64/hsr.h
>>> +++ b/xen/include/asm-arm/arm64/hsr.h
>>> @@ -110,6 +110,72 @@
>>>    #define HSR_SYSREG_CNTP_CTL_EL0   HSR_SYSREG(3,3,c14,c2,1)
>>>    #define HSR_SYSREG_CNTP_CVAL_EL0  HSR_SYSREG(3,3,c14,c2,2)
>>>    +/* Those registers are used when HCR_EL2.TID3 is set */
>>> +#define HSR_SYSREG_ID_PFR0_EL1    HSR_SYSREG(3,0,c0,c1,0)
>>> +#define HSR_SYSREG_ID_PFR1_EL1    HSR_SYSREG(3,0,c0,c1,1)
>>> +#define HSR_SYSREG_ID_PFR2_EL1    HSR_SYSREG(3,0,c0,c3,4)
>>> +#define HSR_SYSREG_ID_DFR0_EL1    HSR_SYSREG(3,0,c0,c1,2)
>>> +#define HSR_SYSREG_ID_DFR1_EL1    HSR_SYSREG(3,0,c0,c3,5)
>>> +#define HSR_SYSREG_ID_AFR0_EL1    HSR_SYSREG(3,0,c0,c1,3)
>>> +#define HSR_SYSREG_ID_MMFR0_EL1   HSR_SYSREG(3,0,c0,c1,4)
>>> +#define HSR_SYSREG_ID_MMFR1_EL1   HSR_SYSREG(3,0,c0,c1,5)
>>> +#define HSR_SYSREG_ID_MMFR2_EL1   HSR_SYSREG(3,0,c0,c1,6)
>>> +#define HSR_SYSREG_ID_MMFR3_EL1   HSR_SYSREG(3,0,c0,c1,7)
>>> +#define HSR_SYSREG_ID_MMFR4_EL1   HSR_SYSREG(3,0,c0,c2,6)
>>> +#define HSR_SYSREG_ID_MMFR5_EL1   HSR_SYSREG(3,0,c0,c3,6)
>>> +#define HSR_SYSREG_ID_ISAR0_EL1   HSR_SYSREG(3,0,c0,c2,0)
>>> +#define HSR_SYSREG_ID_ISAR1_EL1   HSR_SYSREG(3,0,c0,c2,1)
>>> +#define HSR_SYSREG_ID_ISAR2_EL1   HSR_SYSREG(3,0,c0,c2,2)
>>> +#define HSR_SYSREG_ID_ISAR3_EL1   HSR_SYSREG(3,0,c0,c2,3)
>>> +#define HSR_SYSREG_ID_ISAR4_EL1   HSR_SYSREG(3,0,c0,c2,4)
>>> +#define HSR_SYSREG_ID_ISAR5_EL1   HSR_SYSREG(3,0,c0,c2,5)
>>> +#define HSR_SYSREG_ID_ISAR6_EL1   HSR_SYSREG(3,0,c0,c2,7)
>>> +#define HSR_SYSREG_MVFR0_EL1      HSR_SYSREG(3,0,c0,c3,0)
>>> +#define HSR_SYSREG_MVFR1_EL1      HSR_SYSREG(3,0,c0,c3,1)
>>> +#define HSR_SYSREG_MVFR2_EL1      HSR_SYSREG(3,0,c0,c3,2)
>>> +
>>> +#define HSR_SYSREG_ID_AA64PFR0_EL1   HSR_SYSREG(3,0,c0,c4,0)
>>> +#define HSR_SYSREG_ID_AA64PFR1_EL1   HSR_SYSREG(3,0,c0,c4,1)
>>> +#define HSR_SYSREG_ID_AA64DFR0_EL1   HSR_SYSREG(3,0,c0,c5,0)
>>> +#define HSR_SYSREG_ID_AA64DFR1_EL1   HSR_SYSREG(3,0,c0,c5,1)
>>> +#define HSR_SYSREG_ID_AA64ISAR0_EL1  HSR_SYSREG(3,0,c0,c6,0)
>>> +#define HSR_SYSREG_ID_AA64ISAR1_EL1  HSR_SYSREG(3,0,c0,c6,1)
>>> +#define HSR_SYSREG_ID_AA64MMFR0_EL1  HSR_SYSREG(3,0,c0,c7,0)
>>> +#define HSR_SYSREG_ID_AA64MMFR1_EL1  HSR_SYSREG(3,0,c0,c7,1)
>>> +#define HSR_SYSREG_ID_AA64MMFR2_EL1  HSR_SYSREG(3,0,c0,c7,2)
>>> +#define HSR_SYSREG_ID_AA64AFR0_EL1   HSR_SYSREG(3,0,c0,c5,4)
>>> +#define HSR_SYSREG_ID_AA64AFR1_EL1   HSR_SYSREG(3,0,c0,c5,5)
>>> +#define HSR_SYSREG_ID_AA64ZFR0_EL1   HSR_SYSREG(3,0,c0,c4,4)
>>> +
>>> +/*
>>> + * Those cases are catching all Reserved registers trapped by TID3 which
>>> + * currently have no assignment.
>>> + * HCR.TID3 is trapping all registers in the group 3:
>>> + * Op0 == 3, op1 == 0, CRn == c0,CRm == {c1-c7}, op2 == {0-7}.
>>> + */
>>> +#define HSR_SYSREG_TID3_RESERVED_CASE  case HSR_SYSREG(3,0,c0,c3,3): \
>>> +                                       case HSR_SYSREG(3,0,c0,c3,7): \
>>> +                                       case HSR_SYSREG(3,0,c0,c4,2): \
>>> +                                       case HSR_SYSREG(3,0,c0,c4,3): \
>>> +                                       case HSR_SYSREG(3,0,c0,c4,5): \
>>> +                                       case HSR_SYSREG(3,0,c0,c4,6): \
>>> +                                       case HSR_SYSREG(3,0,c0,c4,7): \
>>> +                                       case HSR_SYSREG(3,0,c0,c5,2): \
>>> +                                       case HSR_SYSREG(3,0,c0,c5,3): \
>>> +                                       case HSR_SYSREG(3,0,c0,c5,6): \
>>> +                                       case HSR_SYSREG(3,0,c0,c5,7): \
>>> +                                       case HSR_SYSREG(3,0,c0,c6,2): \
>>> +                                       case HSR_SYSREG(3,0,c0,c6,3): \
>>> +                                       case HSR_SYSREG(3,0,c0,c6,4): \
>>> +                                       case HSR_SYSREG(3,0,c0,c6,5): \
>>> +                                       case HSR_SYSREG(3,0,c0,c6,6): \
>>> +                                       case HSR_SYSREG(3,0,c0,c6,7): \
>>> +                                       case HSR_SYSREG(3,0,c0,c7,3): \
>>> +                                       case HSR_SYSREG(3,0,c0,c7,4): \
>>> +                                       case HSR_SYSREG(3,0,c0,c7,5): \
>>> +                                       case HSR_SYSREG(3,0,c0,c7,6): \
>>> +                                       case HSR_SYSREG(3,0,c0,c7,7)
>>
>> I don't like the idea to define the list of case in a header that is used by
>> multiple source. Please define it directly in the source file that use it.
> 
> At that point it might be best to open-coding it in do_sysreg? I mean no
> #define at all.

I am happpy with that.

Cheers,

-- 
Julien Grall

Re: [PATCH v3 3/7] xen/arm: create a cpuinfo structure for guest

[Bertrand Marquis]

Hi Julien,

> On 9 Dec 2020, at 23:09, Julien Grall <julien@xen.org> wrote:
> 
> Hi Bertand,
> 
> On 09/12/2020 16:30, Bertrand Marquis wrote:
>> Create a cpuinfo structure for guest and mask into it the features that
>> we do not support in Xen or that we do not want to publish to guests.
>> Modify some values in the cpuinfo structure for guests to mask some
>> features which we do not want to allow to guests (like AMU) or we do not
>> support (like SVE).
>> The code is trying to group together registers modifications for the
>> same feature to be able in the long term to easily enable/disable a
>> feature depending on user parameters or add other registers modification
>> in the same place (like enabling/disabling HCR bits).
>> Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
>> ---
>> Changes in V2: Rebase
>> Changes in V3:
>>   Use current_cpu_data info instead of recalling identify_cpu
>> ---
>>  xen/arch/arm/cpufeature.c        | 51 ++++++++++++++++++++++++++++++++
>>  xen/include/asm-arm/cpufeature.h |  2 ++
>>  2 files changed, 53 insertions(+)
>> diff --git a/xen/arch/arm/cpufeature.c b/xen/arch/arm/cpufeature.c
>> index bc7ee5ac95..7255383504 100644
>> --- a/xen/arch/arm/cpufeature.c
>> +++ b/xen/arch/arm/cpufeature.c
>> @@ -24,6 +24,8 @@
>>    DECLARE_BITMAP(cpu_hwcaps, ARM_NCAPS);
>>  +struct cpuinfo_arm __read_mostly guest_cpuinfo;
>> +
>>  void update_cpu_capabilities(const struct arm_cpu_capabilities *caps,
>>                               const char *info)
>>  {
>> @@ -157,6 +159,55 @@ void identify_cpu(struct cpuinfo_arm *c)
>>  #endif
>>  }
>>  +/*
>> + * This function is creating a cpuinfo structure with values modified to mask
>> + * all cpu features that should not be published to guest.
>> + * The created structure is then used to provide ID registers values to guests.
>> + */
>> +static int __init create_guest_cpuinfo(void)
>> +{
>> +    /*
>> +     * TODO: The code is currently using only the features detected on the boot
>> +     * core. In the long term we should try to compute values containing only
>> +     * features supported by all cores.
>> +     */
>> +    guest_cpuinfo = current_cpu_data;
> 
> It would be more logical to use boot_cpu_data as this would be easier to match with your comment.

Agree, I will fix that in V4.

> 
>> +
>> +#ifdef CONFIG_ARM_64
>> +    /* Disable MPAM as xen does not support it */
>> +    guest_cpuinfo.pfr64.mpam = 0;
>> +    guest_cpuinfo.pfr64.mpam_frac = 0;
>> +
>> +    /* Disable SVE as Xen does not support it */
>> +    guest_cpuinfo.pfr64.sve = 0;
>> +    guest_cpuinfo.zfr64.bits[0] = 0;
>> +
>> +    /* Disable MTE as Xen does not support it */
>> +    guest_cpuinfo.pfr64.mte = 0;
>> +#endif
>> +
>> +    /* Disable AMU */
>> +#ifdef CONFIG_ARM_64
>> +    guest_cpuinfo.pfr64.amu = 0;
>> +#endif
>> +    guest_cpuinfo.pfr32.amu = 0;
>> +
>> +    /* Disable RAS as Xen does not support it */
>> +#ifdef CONFIG_ARM_64
>> +    guest_cpuinfo.pfr64.ras = 0;
>> +    guest_cpuinfo.pfr64.ras_frac = 0;
>> +#endif
>> +    guest_cpuinfo.pfr32.ras = 0;
>> +    guest_cpuinfo.pfr32.ras_frac = 0;
> 
> How about all the fields that are currently marked as RES0/RES1? Shouldn't we make sure they will stay like that even if newer architecture use them?

Definitely we can do more then this here (including allowing to enable some things for dom0 or for test reasons).
This is a first try to solve current issues with MPAM and SVE and I plan to continue to enhance this in the future
to enable more customisation here.
I do think we could do a bit more here to have some features controlled by the user but this will need a bit of
discussion to agree on a strategy.

Could we agree to keep the current scope for this serie (to have this in next release) and work then on future
enhancements like this ?

Cheers
Bertrand

> 
>> +
>> +    return 0;
>> +}
>> +/*
>> + * This function needs to be run after all smp are started to have
>> + * cpuinfo structures for all cores.
>> + */
>> +__initcall(create_guest_cpuinfo);
>> +
>>  /*
>>   * Local variables:
>>   * mode: C
>> diff --git a/xen/include/asm-arm/cpufeature.h b/xen/include/asm-arm/cpufeature.h
>> index 6cf83d775b..10b62bd324 100644
>> --- a/xen/include/asm-arm/cpufeature.h
>> +++ b/xen/include/asm-arm/cpufeature.h
>> @@ -283,6 +283,8 @@ extern void identify_cpu(struct cpuinfo_arm *);
>>  extern struct cpuinfo_arm cpu_data[];
>>  #define current_cpu_data cpu_data[smp_processor_id()]
>>  +extern struct cpuinfo_arm guest_cpuinfo;
>> +
>>  #endif /* __ASSEMBLY__ */
>>    #endif
> 
> -- 
> Julien Grall

Re: [PATCH v3 3/7] xen/arm: create a cpuinfo structure for guest

[Bertrand Marquis]

Hi,

> On 9 Dec 2020, at 23:22, Julien Grall <julien@xen.org> wrote:
> 
> Hi,
> 
> On 09/12/2020 16:30, Bertrand Marquis wrote:
>> +    /* Disable MPAM as xen does not support it */
> 
> I am going to be picky :). I think we want to say "hide" rather than "disable" because the latter is done differently via the HCR_EL2.

That does make sense as we are not really disabling but hiding you are right.
I will fix that in V4.

Cheers
Bertrand

> 
> Cheers,
> 
> -- 
> Julien Grall

Re: [PATCH v3 1/7] xen/arm: Add ID registers and complete cpuinfo

[Bertrand Marquis]

Hi,

> On 10 Dec 2020, at 15:45, Julien Grall <julien@xen.org> wrote:
> 
> Hi Bertrand,
> 
> On 10/12/2020 15:14, Bertrand Marquis wrote:
>> Hi Julien,
>>> On 9 Dec 2020, at 23:03, Julien Grall <julien@xen.org> wrote:
>>> 
>>> Hi Bertrand,
>>> 
>>> On 09/12/2020 16:30, Bertrand Marquis wrote:
>>>> Add definition and entries in cpuinfo for ID registers introduced in
>>>> newer Arm Architecture reference manual:
>>>> - ID_PFR2: processor feature register 2
>>>> - ID_DFR1: debug feature register 1
>>>> - ID_MMFR4 and ID_MMFR5: Memory model feature registers 4 and 5
>>>> - ID_ISA6: ISA Feature register 6
>>>> Add more bitfield definitions in PFR fields of cpuinfo.
>>>> Add MVFR2 register definition for aarch32.
>>>> Add mvfr values in cpuinfo.
>>>> Add some registers definition for arm64 in sysregs as some are not
>>>> always know by compilers.
>>>> Initialize the new values added in cpuinfo in identify_cpu during init.
>>>> Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
>>>> ---
>>>> Changes in V2:
>>>>   Fix dbg32 table size and add proper initialisation of the second entry
>>>>   of the table by reading ID_DFR1 register.
>>>> Changes in V3:
>>>>   Fix typo in commit title
>>>>   Add MVFR2 definition and handling on aarch32 and remove specific case
>>>>   for mvfr field in cpuinfo (now the same on arm64 and arm32).
>>>>   Add MMFR4 definition if not known by the compiler.
>>>> ---
>>>>  xen/arch/arm/cpufeature.c           | 18 ++++++++++
>>>>  xen/include/asm-arm/arm64/sysregs.h | 28 +++++++++++++++
>>>>  xen/include/asm-arm/cpregs.h        | 12 +++++++
>>>>  xen/include/asm-arm/cpufeature.h    | 56 ++++++++++++++++++++++++-----
>>>>  4 files changed, 105 insertions(+), 9 deletions(-)
>>>> diff --git a/xen/arch/arm/cpufeature.c b/xen/arch/arm/cpufeature.c
>>>> index 44126dbf07..bc7ee5ac95 100644
>>>> --- a/xen/arch/arm/cpufeature.c
>>>> +++ b/xen/arch/arm/cpufeature.c
>>>> @@ -114,15 +114,20 @@ void identify_cpu(struct cpuinfo_arm *c)
>>>>            c->mm64.bits[0]  = READ_SYSREG64(ID_AA64MMFR0_EL1);
>>>>          c->mm64.bits[1]  = READ_SYSREG64(ID_AA64MMFR1_EL1);
>>>> +        c->mm64.bits[2]  = READ_SYSREG64(ID_AA64MMFR2_EL1);
>>>>            c->isa64.bits[0] = READ_SYSREG64(ID_AA64ISAR0_EL1);
>>>>          c->isa64.bits[1] = READ_SYSREG64(ID_AA64ISAR1_EL1);
>>>> +
>>>> +        c->zfr64.bits[0] = READ_SYSREG64(ID_AA64ZFR0_EL1);
>>>>  #endif
>>>>            c->pfr32.bits[0] = READ_SYSREG32(ID_PFR0_EL1);
>>>>          c->pfr32.bits[1] = READ_SYSREG32(ID_PFR1_EL1);
>>>> +        c->pfr32.bits[2] = READ_SYSREG32(ID_PFR2_EL1);
>>>>            c->dbg32.bits[0] = READ_SYSREG32(ID_DFR0_EL1);
>>>> +        c->dbg32.bits[1] = READ_SYSREG32(ID_DFR1_EL1);
>>>>            c->aux32.bits[0] = READ_SYSREG32(ID_AFR0_EL1);
>>>>  @@ -130,6 +135,8 @@ void identify_cpu(struct cpuinfo_arm *c)
>>>>          c->mm32.bits[1]  = READ_SYSREG32(ID_MMFR1_EL1);
>>>>          c->mm32.bits[2]  = READ_SYSREG32(ID_MMFR2_EL1);
>>>>          c->mm32.bits[3]  = READ_SYSREG32(ID_MMFR3_EL1);
>>>> +        c->mm32.bits[4]  = READ_SYSREG32(ID_MMFR4_EL1);
>>>> +        c->mm32.bits[5]  = READ_SYSREG32(ID_MMFR5_EL1);
>>> 
>>> Please don't introduce any more use of READ_SYSREG32(), they are wrong on Armv8 because system registers are always 64-bit.
>> I followed the existing implementation but ...
>>> 
>>>>            c->isa32.bits[0] = READ_SYSREG32(ID_ISAR0_EL1);
>>>>          c->isa32.bits[1] = READ_SYSREG32(ID_ISAR1_EL1);
>>>> @@ -137,6 +144,17 @@ void identify_cpu(struct cpuinfo_arm *c)
>>>>          c->isa32.bits[3] = READ_SYSREG32(ID_ISAR3_EL1);
>>>>          c->isa32.bits[4] = READ_SYSREG32(ID_ISAR4_EL1);
>>>>          c->isa32.bits[5] = READ_SYSREG32(ID_ISAR5_EL1);
>>>> +        c->isa32.bits[6] = READ_SYSREG32(ID_ISAR6_EL1);
>>>> +
>>>> +#ifdef CONFIG_ARM_64
>>>> +        c->mvfr.bits[0] = READ_SYSREG64(MVFR0_EL1);
>>>> +        c->mvfr.bits[1] = READ_SYSREG64(MVFR1_EL1);
>>>> +        c->mvfr.bits[2] = READ_SYSREG64(MVFR2_EL1);
>>>> +#else
>>>> +        c->mvfr.bits[0] = READ_CP32(MVFR0);
>>>> +        c->mvfr.bits[1] = READ_CP32(MVFR1);
>>>> +        c->mvfr.bits[2] = READ_CP32(MVFR2);
>>>> +#endif
>>> 
>>> READ_SYSREG() will do the job to either use READ_SYSREG64() or READ_CP32() depending on the arch used.
>> .. I can modify the ones I added and the existing ones to user READ_SYSREG instead.
>> Please confirm if you want me to do that.
> 
> Yes, please use READ_SYSREG() for the new ones. We can convert the others separately.
> 
>>> 
>>>>  }
>>>>    /*
>>>> diff --git a/xen/include/asm-arm/arm64/sysregs.h b/xen/include/asm-arm/arm64/sysregs.h
>>>> index c60029d38f..077fd95fb7 100644
>>>> --- a/xen/include/asm-arm/arm64/sysregs.h
>>>> +++ b/xen/include/asm-arm/arm64/sysregs.h
>>>> @@ -57,6 +57,34 @@
>>>>  #define ICH_AP1R2_EL2             __AP1Rx_EL2(2)
>>>>  #define ICH_AP1R3_EL2             __AP1Rx_EL2(3)
>>>>  +/*
>>>> + * Define ID coprocessor registers if they are not
>>>> + * already defined by the compiler.
>>>> + *
>>>> + * Values picked from linux kernel
>>>> + */
>>>> +#ifndef ID_AA64MMFR2_EL1
>>> 
>>> I am a bit puzzled how this meant to work. Will the libc/compiler headers define ID_AA64MMFR2_EL1?
>> I tested this and if the compiler has a definition for the register, I am not entering the ifndef.
>> So there is no header defining this but if the compiler has the definition for this the ifndef will
>> not be entered.
> 
> Good to hear :).
> 
>>> 
>>>> +#define ID_AA64MMFR2_EL1            S3_0_C0_C7_2
>>>> +#endif
>>>> +#ifndef ID_PFR2_EL1
>>>> +#define ID_PFR2_EL1                 S3_0_C0_C3_4
>>>> +#endif
>>>> +#ifndef ID_MMFR4_EL1
>>>> +#define ID_MMFR4_EL1                S3_0_C0_C2_6
>>>> +#endif
>>>> +#ifndef ID_MMFR5_EL1
>>>> +#define ID_MMFR5_EL1                S3_0_C0_C3_6
>>>> +#endif
>>>> +#ifndef ID_ISAR6_EL1
>>>> +#define ID_ISAR6_EL1                S3_0_C0_C2_7
>>>> +#endif
>>>> +#ifndef ID_AA64ZFR0_EL1
>>>> +#define ID_AA64ZFR0_EL1             S3_0_C0_C4_4
>>>> +#endif
>>>> +#ifndef ID_DFR1_EL1
>>>> +#define ID_DFR1_EL1                 S3_0_C0_C3_5
>>>> +#endif
>>>> +
>>>>  /* Access to system registers */
>>>>    #define READ_SYSREG32(name) ((uint32_t)READ_SYSREG64(name))
>>>> diff --git a/xen/include/asm-arm/cpregs.h b/xen/include/asm-arm/cpregs.h
>>>> index 8fd344146e..2690ddeb7a 100644
>>>> --- a/xen/include/asm-arm/cpregs.h
>>>> +++ b/xen/include/asm-arm/cpregs.h
>>>> @@ -63,6 +63,8 @@
>>>>  #define FPSID           p10,7,c0,c0,0   /* Floating-Point System ID Register */
>>>>  #define FPSCR           p10,7,c1,c0,0   /* Floating-Point Status and Control Register */
>>>>  #define MVFR0           p10,7,c7,c0,0   /* Media and VFP Feature Register 0 */
>>>> +#define MVFR1           p10,7,c6,c0,0   /* Media and VFP Feature Register 1 */
>>>> +#define MVFR2           p10,7,c5,c0,0   /* Media and VFP Feature Register 2 */
>>>>  #define FPEXC           p10,7,c8,c0,0   /* Floating-Point Exception Control Register */
>>>>  #define FPINST          p10,7,c9,c0,0   /* Floating-Point Instruction Register */
>>>>  #define FPINST2         p10,7,c10,c0,0  /* Floating-point Instruction Register 2 */
>>>> @@ -108,18 +110,23 @@
>>>>  #define MPIDR           p15,0,c0,c0,5   /* Multiprocessor Affinity Register */
>>>>  #define ID_PFR0         p15,0,c0,c1,0   /* Processor Feature Register 0 */
>>>>  #define ID_PFR1         p15,0,c0,c1,1   /* Processor Feature Register 1 */
>>>> +#define ID_PFR2         p15,0,c0,c3,4   /* Processor Feature Register 2 */
>>>>  #define ID_DFR0         p15,0,c0,c1,2   /* Debug Feature Register 0 */
>>>> +#define ID_DFR1         p15,0,c0,c3,5   /* Debug Feature Register 1 */
>>>>  #define ID_AFR0         p15,0,c0,c1,3   /* Auxiliary Feature Register 0 */
>>>>  #define ID_MMFR0        p15,0,c0,c1,4   /* Memory Model Feature Register 0 */
>>>>  #define ID_MMFR1        p15,0,c0,c1,5   /* Memory Model Feature Register 1 */
>>>>  #define ID_MMFR2        p15,0,c0,c1,6   /* Memory Model Feature Register 2 */
>>>>  #define ID_MMFR3        p15,0,c0,c1,7   /* Memory Model Feature Register 3 */
>>>> +#define ID_MMFR4        p15,0,c0,c2,6   /* Memory Model Feature Register 4 */
>>>> +#define ID_MMFR5        p15,0,c0,c3,6   /* Memory Model Feature Register 5 */
>>>>  #define ID_ISAR0        p15,0,c0,c2,0   /* ISA Feature Register 0 */
>>>>  #define ID_ISAR1        p15,0,c0,c2,1   /* ISA Feature Register 1 */
>>>>  #define ID_ISAR2        p15,0,c0,c2,2   /* ISA Feature Register 2 */
>>>>  #define ID_ISAR3        p15,0,c0,c2,3   /* ISA Feature Register 3 */
>>>>  #define ID_ISAR4        p15,0,c0,c2,4   /* ISA Feature Register 4 */
>>>>  #define ID_ISAR5        p15,0,c0,c2,5   /* ISA Feature Register 5 */
>>>> +#define ID_ISAR6        p15,0,c0,c2,7   /* ISA Feature Register 6 */
>>>>  #define CCSIDR          p15,1,c0,c0,0   /* Cache Size ID Registers */
>>>>  #define CLIDR           p15,1,c0,c0,1   /* Cache Level ID Register */
>>>>  #define CSSELR          p15,2,c0,c0,0   /* Cache Size Selection Register */
>>>> @@ -312,18 +319,23 @@
>>>>  #define HSTR_EL2                HSTR
>>>>  #define ID_AFR0_EL1             ID_AFR0
>>>>  #define ID_DFR0_EL1             ID_DFR0
>>>> +#define ID_DFR1_EL1             ID_DFR1
>>>>  #define ID_ISAR0_EL1            ID_ISAR0
>>>>  #define ID_ISAR1_EL1            ID_ISAR1
>>>>  #define ID_ISAR2_EL1            ID_ISAR2
>>>>  #define ID_ISAR3_EL1            ID_ISAR3
>>>>  #define ID_ISAR4_EL1            ID_ISAR4
>>>>  #define ID_ISAR5_EL1            ID_ISAR5
>>>> +#define ID_ISAR6_EL1            ID_ISAR6
>>>>  #define ID_MMFR0_EL1            ID_MMFR0
>>>>  #define ID_MMFR1_EL1            ID_MMFR1
>>>>  #define ID_MMFR2_EL1            ID_MMFR2
>>>>  #define ID_MMFR3_EL1            ID_MMFR3
>>>> +#define ID_MMFR4_EL1            ID_MMFR4
>>>> +#define ID_MMFR5_EL1            ID_MMFR5
>>>>  #define ID_PFR0_EL1             ID_PFR0
>>>>  #define ID_PFR1_EL1             ID_PFR1
>>>> +#define ID_PFR2_EL1             ID_PFR2
>>>>  #define IFSR32_EL2              IFSR
>>>>  #define MDCR_EL2                HDCR
>>>>  #define MIDR_EL1                MIDR
>>>> diff --git a/xen/include/asm-arm/cpufeature.h b/xen/include/asm-arm/cpufeature.h
>>>> index c7b5052992..6cf83d775b 100644
>>>> --- a/xen/include/asm-arm/cpufeature.h
>>>> +++ b/xen/include/asm-arm/cpufeature.h
>>>> @@ -148,6 +148,7 @@ struct cpuinfo_arm {
>>>>      union {
>>>>          uint64_t bits[2];
>>>>          struct {
>>>> +            /* PFR0 */
>>>>              unsigned long el0:4;
>>>>              unsigned long el1:4;
>>>>              unsigned long el2:4;
>>>> @@ -155,9 +156,23 @@ struct cpuinfo_arm {
>>>>              unsigned long fp:4;   /* Floating Point */
>>>>              unsigned long simd:4; /* Advanced SIMD */
>>>>              unsigned long gic:4;  /* GIC support */
>>>> -            unsigned long __res0:28;
>>>> +            unsigned long ras:4;
>>>> +            unsigned long sve:4;
>>>> +            unsigned long sel2:4;
>>>> +            unsigned long mpam:4;
>>>> +            unsigned long amu:4;
>>>> +            unsigned long dit:4;
>>>> +            unsigned long __res0:4;
>>>>              unsigned long csv2:4;
>>>> -            unsigned long __res1:4;
>>>> +            unsigned long cvs3:4;
>>>> +
>>>> +            /* PFR1 */
>>>> +            unsigned long bt:4;
>>>> +            unsigned long ssbs:4;
>>>> +            unsigned long mte:4;
>>>> +            unsigned long ras_frac:4;
>>>> +            unsigned long mpam_frac:4;
>>>> +            unsigned long __res1:44;
>>>>          };
>>>>      } pfr64;
>>>>  @@ -170,7 +185,7 @@ struct cpuinfo_arm {
>>>>      } aux64;
>>>>        union {
>>>> -        uint64_t bits[2];
>>>> +        uint64_t bits[3];
>>>>          struct {
>>>>              unsigned long pa_range:4;
>>>>              unsigned long asid_bits:4;
>>>> @@ -190,6 +205,8 @@ struct cpuinfo_arm {
>>>>              unsigned long pan:4;
>>>>              unsigned long __res1:8;
>>>>              unsigned long __res2:32;
>>>> +
>>>> +            unsigned long __res3:64;
>>>>          };
>>>>      } mm64;
>>>>  @@ -197,6 +214,10 @@ struct cpuinfo_arm {
>>>>          uint64_t bits[2];
>>>>      } isa64;
>>>>  +    struct {
>>>> +        uint64_t bits[1];
>>>> +    } zfr64;
>>>> +
>>>>  #endif
>>>>        /*
>>>> @@ -204,25 +225,38 @@ struct cpuinfo_arm {
>>>>       * when running in 32-bit mode.
>>>>       */
>>>>      union {
>>>> -        uint32_t bits[2];
>>>> +        uint32_t bits[3];
>>>>          struct {
>>>> +            /* PFR0 */
>>>>              unsigned long arm:4;
>>>>              unsigned long thumb:4;
>>>>              unsigned long jazelle:4;
>>>>              unsigned long thumbee:4;
>>>> -            unsigned long __res0:16;
>>>> +            unsigned long csv2:4;
>>>> +            unsigned long amu:4;
>>>> +            unsigned long dit:4;
>>>> +            unsigned long ras:4;
>>>>  +            /* PFR1 */
>>>>              unsigned long progmodel:4;
>>>>              unsigned long security:4;
>>>>              unsigned long mprofile:4;
>>>>              unsigned long virt:4;
>>>>              unsigned long gentimer:4;
>>>> -            unsigned long __res1:12;
>>>> +            unsigned long sec_frac:4;
>>>> +            unsigned long virt_frac:4;
>>>> +            unsigned long gic:4;
>>>> +
>>>> +            /* PFR2 */
>>>> +            unsigned long csv3:4;
>>>> +            unsigned long ssbs:4;
>>>> +            unsigned long ras_frac:4;
>>>> +            unsigned long __res2:20;
>>>>          };
>>>>      } pfr32;
>>>>        struct {
>>>> -        uint32_t bits[1];
>>>> +        uint32_t bits[2];
>>>>      } dbg32;
>>>>        struct {
>>>> @@ -230,12 +264,16 @@ struct cpuinfo_arm {
>>>>      } aux32;
>>>>        struct {
>>>> -        uint32_t bits[4];
>>>> +        uint32_t bits[6];
>>>>      } mm32;
>>>>        struct {
>>>> -        uint32_t bits[6];
>>>> +        uint32_t bits[7];
>>>>      } isa32;
>>>> +
>>>> +    struct {
>>>> +        uint64_t bits[3];
>>> 
>>> Shouldn't this be register_t?
>> I followed the scheme of the rest of the structure which
>> is always using uint64_t or uint32_t for bits definitions.
> 
> Right, but I am sure you will not be surprised if I tell you this is buggy ;). The historical reason is, IIRC, the original spec of Armv8.0 described them as 32-bit registers.
> 
> The spec was updated a while ago to clarify they are 64-bit when running in AArch64. But a majority of them still have the top 32-bit RES0 (thankfully!).
> 
>> Why should I use register_t type for this one ?
> 
> Because the value is 32-bit on AArch32 and 64-bit for AArch64. I am ok to use 64-bit still for AArch32, but it sounds a bit of a waste of memory.
> 
> What I care the most here is we use 64-bit for the new registers on AArch64. Otherwise, we are going to soon discover that a bit above 32 was allocated and not detected by Xen. I don't want to be the one doing the debugging!
> 
> Admittly, this is not a new issue. However, the more offending code we had the more it will be difficul to get Xen to be fully compliant with the Armv8 spec.

I agree with that so I propose to do the following:
- create a new serie to fix those: 
	- use READ_SYSREG in cpufeature.c
	- use register_t in cpuinfo structure
- rebase my serie on top of this one and use the proper types on it

Might require more work but seem to be the right approach.

Cheers
Bertrand

> 
> Cheers,
> 
> -- 
> Julien Grall

Re: [PATCH v3 2/7] xen/arm: Add arm64 ID registers definitions

[Bertrand Marquis]

Hi,

> On 10 Dec 2020, at 15:46, Julien Grall <julien@xen.org> wrote:
> 
> 
> 
> On 10/12/2020 02:30, Stefano Stabellini wrote:
>> On Wed, 9 Dec 2020, Julien Grall wrote:
>>> Hi Bertrand,
>>> 
>>> On 09/12/2020 16:30, Bertrand Marquis wrote:
>>>> Add coprocessor registers definitions for all ID registers trapped
>>>> through the TID3 bit of HSR.
>>>> Those are the one that will be emulated in Xen to only publish to guests
>>>> the features that are supported by Xen and that are accessible to
>>>> guests.
>>>> Also define a case to catch all reserved registers that should be
>>>> handled as RAZ.
>>>> 
>>>> Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
>>>> ---
>>>> Changes in V2: Rebase
>>>> Changes in V3:
>>>>    Add case definition for reserved registers.
>>>> 
>>>> ---
>>>>   xen/include/asm-arm/arm64/hsr.h | 66 +++++++++++++++++++++++++++++++++
>>>>   1 file changed, 66 insertions(+)
>>>> 
>>>> diff --git a/xen/include/asm-arm/arm64/hsr.h
>>>> b/xen/include/asm-arm/arm64/hsr.h
>>>> index ca931dd2fe..ffe0f0007e 100644
>>>> --- a/xen/include/asm-arm/arm64/hsr.h
>>>> +++ b/xen/include/asm-arm/arm64/hsr.h
>>>> @@ -110,6 +110,72 @@
>>>>   #define HSR_SYSREG_CNTP_CTL_EL0   HSR_SYSREG(3,3,c14,c2,1)
>>>>   #define HSR_SYSREG_CNTP_CVAL_EL0  HSR_SYSREG(3,3,c14,c2,2)
>>>>   +/* Those registers are used when HCR_EL2.TID3 is set */
>>>> +#define HSR_SYSREG_ID_PFR0_EL1    HSR_SYSREG(3,0,c0,c1,0)
>>>> +#define HSR_SYSREG_ID_PFR1_EL1    HSR_SYSREG(3,0,c0,c1,1)
>>>> +#define HSR_SYSREG_ID_PFR2_EL1    HSR_SYSREG(3,0,c0,c3,4)
>>>> +#define HSR_SYSREG_ID_DFR0_EL1    HSR_SYSREG(3,0,c0,c1,2)
>>>> +#define HSR_SYSREG_ID_DFR1_EL1    HSR_SYSREG(3,0,c0,c3,5)
>>>> +#define HSR_SYSREG_ID_AFR0_EL1    HSR_SYSREG(3,0,c0,c1,3)
>>>> +#define HSR_SYSREG_ID_MMFR0_EL1   HSR_SYSREG(3,0,c0,c1,4)
>>>> +#define HSR_SYSREG_ID_MMFR1_EL1   HSR_SYSREG(3,0,c0,c1,5)
>>>> +#define HSR_SYSREG_ID_MMFR2_EL1   HSR_SYSREG(3,0,c0,c1,6)
>>>> +#define HSR_SYSREG_ID_MMFR3_EL1   HSR_SYSREG(3,0,c0,c1,7)
>>>> +#define HSR_SYSREG_ID_MMFR4_EL1   HSR_SYSREG(3,0,c0,c2,6)
>>>> +#define HSR_SYSREG_ID_MMFR5_EL1   HSR_SYSREG(3,0,c0,c3,6)
>>>> +#define HSR_SYSREG_ID_ISAR0_EL1   HSR_SYSREG(3,0,c0,c2,0)
>>>> +#define HSR_SYSREG_ID_ISAR1_EL1   HSR_SYSREG(3,0,c0,c2,1)
>>>> +#define HSR_SYSREG_ID_ISAR2_EL1   HSR_SYSREG(3,0,c0,c2,2)
>>>> +#define HSR_SYSREG_ID_ISAR3_EL1   HSR_SYSREG(3,0,c0,c2,3)
>>>> +#define HSR_SYSREG_ID_ISAR4_EL1   HSR_SYSREG(3,0,c0,c2,4)
>>>> +#define HSR_SYSREG_ID_ISAR5_EL1   HSR_SYSREG(3,0,c0,c2,5)
>>>> +#define HSR_SYSREG_ID_ISAR6_EL1   HSR_SYSREG(3,0,c0,c2,7)
>>>> +#define HSR_SYSREG_MVFR0_EL1      HSR_SYSREG(3,0,c0,c3,0)
>>>> +#define HSR_SYSREG_MVFR1_EL1      HSR_SYSREG(3,0,c0,c3,1)
>>>> +#define HSR_SYSREG_MVFR2_EL1      HSR_SYSREG(3,0,c0,c3,2)
>>>> +
>>>> +#define HSR_SYSREG_ID_AA64PFR0_EL1   HSR_SYSREG(3,0,c0,c4,0)
>>>> +#define HSR_SYSREG_ID_AA64PFR1_EL1   HSR_SYSREG(3,0,c0,c4,1)
>>>> +#define HSR_SYSREG_ID_AA64DFR0_EL1   HSR_SYSREG(3,0,c0,c5,0)
>>>> +#define HSR_SYSREG_ID_AA64DFR1_EL1   HSR_SYSREG(3,0,c0,c5,1)
>>>> +#define HSR_SYSREG_ID_AA64ISAR0_EL1  HSR_SYSREG(3,0,c0,c6,0)
>>>> +#define HSR_SYSREG_ID_AA64ISAR1_EL1  HSR_SYSREG(3,0,c0,c6,1)
>>>> +#define HSR_SYSREG_ID_AA64MMFR0_EL1  HSR_SYSREG(3,0,c0,c7,0)
>>>> +#define HSR_SYSREG_ID_AA64MMFR1_EL1  HSR_SYSREG(3,0,c0,c7,1)
>>>> +#define HSR_SYSREG_ID_AA64MMFR2_EL1  HSR_SYSREG(3,0,c0,c7,2)
>>>> +#define HSR_SYSREG_ID_AA64AFR0_EL1   HSR_SYSREG(3,0,c0,c5,4)
>>>> +#define HSR_SYSREG_ID_AA64AFR1_EL1   HSR_SYSREG(3,0,c0,c5,5)
>>>> +#define HSR_SYSREG_ID_AA64ZFR0_EL1   HSR_SYSREG(3,0,c0,c4,4)
>>>> +
>>>> +/*
>>>> + * Those cases are catching all Reserved registers trapped by TID3 which
>>>> + * currently have no assignment.
>>>> + * HCR.TID3 is trapping all registers in the group 3:
>>>> + * Op0 == 3, op1 == 0, CRn == c0,CRm == {c1-c7}, op2 == {0-7}.
>>>> + */
>>>> +#define HSR_SYSREG_TID3_RESERVED_CASE  case HSR_SYSREG(3,0,c0,c3,3): \
>>>> +                                       case HSR_SYSREG(3,0,c0,c3,7): \
>>>> +                                       case HSR_SYSREG(3,0,c0,c4,2): \
>>>> +                                       case HSR_SYSREG(3,0,c0,c4,3): \
>>>> +                                       case HSR_SYSREG(3,0,c0,c4,5): \
>>>> +                                       case HSR_SYSREG(3,0,c0,c4,6): \
>>>> +                                       case HSR_SYSREG(3,0,c0,c4,7): \
>>>> +                                       case HSR_SYSREG(3,0,c0,c5,2): \
>>>> +                                       case HSR_SYSREG(3,0,c0,c5,3): \
>>>> +                                       case HSR_SYSREG(3,0,c0,c5,6): \
>>>> +                                       case HSR_SYSREG(3,0,c0,c5,7): \
>>>> +                                       case HSR_SYSREG(3,0,c0,c6,2): \
>>>> +                                       case HSR_SYSREG(3,0,c0,c6,3): \
>>>> +                                       case HSR_SYSREG(3,0,c0,c6,4): \
>>>> +                                       case HSR_SYSREG(3,0,c0,c6,5): \
>>>> +                                       case HSR_SYSREG(3,0,c0,c6,6): \
>>>> +                                       case HSR_SYSREG(3,0,c0,c6,7): \
>>>> +                                       case HSR_SYSREG(3,0,c0,c7,3): \
>>>> +                                       case HSR_SYSREG(3,0,c0,c7,4): \
>>>> +                                       case HSR_SYSREG(3,0,c0,c7,5): \
>>>> +                                       case HSR_SYSREG(3,0,c0,c7,6): \
>>>> +                                       case HSR_SYSREG(3,0,c0,c7,7)
>>> 
>>> I don't like the idea to define the list of case in a header that is used by
>>> multiple source. Please define it directly in the source file that use it.
>> At that point it might be best to open-coding it in do_sysreg? I mean no
>> #define at all.
> 
> I am happpy with that.

Then i will fix both cp15 and arm64 code to do it this way.

Cheers
Bertrand

> 
> Cheers,
> 
> -- 
> Julien Grall
> 

Re: [PATCH v3 3/7] xen/arm: create a cpuinfo structure for guest

[Julien Grall]

On 10/12/2020 15:48, Bertrand Marquis wrote:
> Hi Julien,
> 
>> On 9 Dec 2020, at 23:09, Julien Grall <julien@xen.org> wrote:
>>
>> Hi Bertand,
>>
>> On 09/12/2020 16:30, Bertrand Marquis wrote:
>>> Create a cpuinfo structure for guest and mask into it the features that
>>> we do not support in Xen or that we do not want to publish to guests.
>>> Modify some values in the cpuinfo structure for guests to mask some
>>> features which we do not want to allow to guests (like AMU) or we do not
>>> support (like SVE).
>>> The code is trying to group together registers modifications for the
>>> same feature to be able in the long term to easily enable/disable a
>>> feature depending on user parameters or add other registers modification
>>> in the same place (like enabling/disabling HCR bits).
>>> Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
>>> ---
>>> Changes in V2: Rebase
>>> Changes in V3:
>>>    Use current_cpu_data info instead of recalling identify_cpu
>>> ---
>>>   xen/arch/arm/cpufeature.c        | 51 ++++++++++++++++++++++++++++++++
>>>   xen/include/asm-arm/cpufeature.h |  2 ++
>>>   2 files changed, 53 insertions(+)
>>> diff --git a/xen/arch/arm/cpufeature.c b/xen/arch/arm/cpufeature.c
>>> index bc7ee5ac95..7255383504 100644
>>> --- a/xen/arch/arm/cpufeature.c
>>> +++ b/xen/arch/arm/cpufeature.c
>>> @@ -24,6 +24,8 @@
>>>     DECLARE_BITMAP(cpu_hwcaps, ARM_NCAPS);
>>>   +struct cpuinfo_arm __read_mostly guest_cpuinfo;
>>> +
>>>   void update_cpu_capabilities(const struct arm_cpu_capabilities *caps,
>>>                                const char *info)
>>>   {
>>> @@ -157,6 +159,55 @@ void identify_cpu(struct cpuinfo_arm *c)
>>>   #endif
>>>   }
>>>   +/*
>>> + * This function is creating a cpuinfo structure with values modified to mask
>>> + * all cpu features that should not be published to guest.
>>> + * The created structure is then used to provide ID registers values to guests.
>>> + */
>>> +static int __init create_guest_cpuinfo(void)
>>> +{
>>> +    /*
>>> +     * TODO: The code is currently using only the features detected on the boot
>>> +     * core. In the long term we should try to compute values containing only
>>> +     * features supported by all cores.
>>> +     */
>>> +    guest_cpuinfo = current_cpu_data;
>>
>> It would be more logical to use boot_cpu_data as this would be easier to match with your comment.
> 
> Agree, I will fix that in V4.
> 
>>
>>> +
>>> +#ifdef CONFIG_ARM_64
>>> +    /* Disable MPAM as xen does not support it */
>>> +    guest_cpuinfo.pfr64.mpam = 0;
>>> +    guest_cpuinfo.pfr64.mpam_frac = 0;
>>> +
>>> +    /* Disable SVE as Xen does not support it */
>>> +    guest_cpuinfo.pfr64.sve = 0;
>>> +    guest_cpuinfo.zfr64.bits[0] = 0;
>>> +
>>> +    /* Disable MTE as Xen does not support it */
>>> +    guest_cpuinfo.pfr64.mte = 0;
>>> +#endif
>>> +
>>> +    /* Disable AMU */
>>> +#ifdef CONFIG_ARM_64
>>> +    guest_cpuinfo.pfr64.amu = 0;
>>> +#endif
>>> +    guest_cpuinfo.pfr32.amu = 0;
>>> +
>>> +    /* Disable RAS as Xen does not support it */
>>> +#ifdef CONFIG_ARM_64
>>> +    guest_cpuinfo.pfr64.ras = 0;
>>> +    guest_cpuinfo.pfr64.ras_frac = 0;
>>> +#endif
>>> +    guest_cpuinfo.pfr32.ras = 0;
>>> +    guest_cpuinfo.pfr32.ras_frac = 0;
>>
>> How about all the fields that are currently marked as RES0/RES1? Shouldn't we make sure they will stay like that even if newer architecture use them?
> 
> Definitely we can do more then this here (including allowing to enable some things for dom0 or for test reasons).
> This is a first try to solve current issues with MPAM and SVE and I plan to continue to enhance this in the future
> to enable more customisation here.
> I do think we could do a bit more here to have some features controlled by the user but this will need a bit of
> discussion to agree on a strategy.

I think you misunderstood my comment. I am not asking whether we want to 
customize the value per domain. Instead, I am raising questions for the 
strategy taken in this patch.

I am going to leave the safety aside, because I think this is orthogonal 
to this patch.

This patch is introducing a deny list. This means that all the features 
will be exposed to a domain unless someone determine that this is not
supported by Xen.

This means we will always try to catch up with what Arm decided to 
invent and attempt to fix it before the silicon is out.

Instead, I think it would be better to use an allow list. We should only 
expose features to the guest we know works (this could possibly be just 
the Armv8.0 one).

Cheers,

-- 
Julien Grall

Re: [PATCH v3 3/7] xen/arm: create a cpuinfo structure for guest

[Bertrand Marquis]

Hi Julien,

> On 10 Dec 2020, at 16:05, Julien Grall <julien@xen.org> wrote:
> 
> 
> 
> On 10/12/2020 15:48, Bertrand Marquis wrote:
>> Hi Julien,
>>> On 9 Dec 2020, at 23:09, Julien Grall <julien@xen.org> wrote:
>>> 
>>> Hi Bertand,
>>> 
>>> On 09/12/2020 16:30, Bertrand Marquis wrote:
>>>> Create a cpuinfo structure for guest and mask into it the features that
>>>> we do not support in Xen or that we do not want to publish to guests.
>>>> Modify some values in the cpuinfo structure for guests to mask some
>>>> features which we do not want to allow to guests (like AMU) or we do not
>>>> support (like SVE).
>>>> The code is trying to group together registers modifications for the
>>>> same feature to be able in the long term to easily enable/disable a
>>>> feature depending on user parameters or add other registers modification
>>>> in the same place (like enabling/disabling HCR bits).
>>>> Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
>>>> ---
>>>> Changes in V2: Rebase
>>>> Changes in V3:
>>>>   Use current_cpu_data info instead of recalling identify_cpu
>>>> ---
>>>>  xen/arch/arm/cpufeature.c        | 51 ++++++++++++++++++++++++++++++++
>>>>  xen/include/asm-arm/cpufeature.h |  2 ++
>>>>  2 files changed, 53 insertions(+)
>>>> diff --git a/xen/arch/arm/cpufeature.c b/xen/arch/arm/cpufeature.c
>>>> index bc7ee5ac95..7255383504 100644
>>>> --- a/xen/arch/arm/cpufeature.c
>>>> +++ b/xen/arch/arm/cpufeature.c
>>>> @@ -24,6 +24,8 @@
>>>>    DECLARE_BITMAP(cpu_hwcaps, ARM_NCAPS);
>>>>  +struct cpuinfo_arm __read_mostly guest_cpuinfo;
>>>> +
>>>>  void update_cpu_capabilities(const struct arm_cpu_capabilities *caps,
>>>>                               const char *info)
>>>>  {
>>>> @@ -157,6 +159,55 @@ void identify_cpu(struct cpuinfo_arm *c)
>>>>  #endif
>>>>  }
>>>>  +/*
>>>> + * This function is creating a cpuinfo structure with values modified to mask
>>>> + * all cpu features that should not be published to guest.
>>>> + * The created structure is then used to provide ID registers values to guests.
>>>> + */
>>>> +static int __init create_guest_cpuinfo(void)
>>>> +{
>>>> +    /*
>>>> +     * TODO: The code is currently using only the features detected on the boot
>>>> +     * core. In the long term we should try to compute values containing only
>>>> +     * features supported by all cores.
>>>> +     */
>>>> +    guest_cpuinfo = current_cpu_data;
>>> 
>>> It would be more logical to use boot_cpu_data as this would be easier to match with your comment.
>> Agree, I will fix that in V4.
>>> 
>>>> +
>>>> +#ifdef CONFIG_ARM_64
>>>> +    /* Disable MPAM as xen does not support it */
>>>> +    guest_cpuinfo.pfr64.mpam = 0;
>>>> +    guest_cpuinfo.pfr64.mpam_frac = 0;
>>>> +
>>>> +    /* Disable SVE as Xen does not support it */
>>>> +    guest_cpuinfo.pfr64.sve = 0;
>>>> +    guest_cpuinfo.zfr64.bits[0] = 0;
>>>> +
>>>> +    /* Disable MTE as Xen does not support it */
>>>> +    guest_cpuinfo.pfr64.mte = 0;
>>>> +#endif
>>>> +
>>>> +    /* Disable AMU */
>>>> +#ifdef CONFIG_ARM_64
>>>> +    guest_cpuinfo.pfr64.amu = 0;
>>>> +#endif
>>>> +    guest_cpuinfo.pfr32.amu = 0;
>>>> +
>>>> +    /* Disable RAS as Xen does not support it */
>>>> +#ifdef CONFIG_ARM_64
>>>> +    guest_cpuinfo.pfr64.ras = 0;
>>>> +    guest_cpuinfo.pfr64.ras_frac = 0;
>>>> +#endif
>>>> +    guest_cpuinfo.pfr32.ras = 0;
>>>> +    guest_cpuinfo.pfr32.ras_frac = 0;
>>> 
>>> How about all the fields that are currently marked as RES0/RES1? Shouldn't we make sure they will stay like that even if newer architecture use them?
>> Definitely we can do more then this here (including allowing to enable some things for dom0 or for test reasons).
>> This is a first try to solve current issues with MPAM and SVE and I plan to continue to enhance this in the future
>> to enable more customisation here.
>> I do think we could do a bit more here to have some features controlled by the user but this will need a bit of
>> discussion to agree on a strategy.
> 
> I think you misunderstood my comment. I am not asking whether we want to customize the value per domain. Instead, I am raising questions for the strategy taken in this patch.
> 
> I am going to leave the safety aside, because I think this is orthogonal to this patch.
> 
> This patch is introducing a deny list. This means that all the features will be exposed to a domain unless someone determine that this is not
> supported by Xen.
> 
> This means we will always try to catch up with what Arm decided to invent and attempt to fix it before the silicon is out.
> 
> Instead, I think it would be better to use an allow list. We should only expose features to the guest we know works (this could possibly be just the Armv8.0 one).

I understood that and I fully agree that this is what we should do: only expose what we support and know and let everything else as “disabled”.
And I definitely want to do that and I think with this serie we have the required support to do that, we will need to rework how we initialise the
guest_cpuinfo structure.

I just want to leave this discussion for after so that we can at least right now have a current linux booting without the need to modify the linux
kernel binary to remove things like SVE.

Regards
Bertrand

> 
> Cheers,
> 
> -- 
> Julien Grall

Re: [PATCH v3 3/7] xen/arm: create a cpuinfo structure for guest

[Julien Grall]

On 10/12/2020 16:17, Bertrand Marquis wrote:
> Hi Julien,

Hi Bertrand,

>> On 10 Dec 2020, at 16:05, Julien Grall <julien@xen.org> wrote:
>>
>>
>>
>> On 10/12/2020 15:48, Bertrand Marquis wrote:
>>> Hi Julien,
>>>> On 9 Dec 2020, at 23:09, Julien Grall <julien@xen.org> wrote:
>>>>
>>>> Hi Bertand,
>>>>
>>>> On 09/12/2020 16:30, Bertrand Marquis wrote:
>>>>> Create a cpuinfo structure for guest and mask into it the features that
>>>>> we do not support in Xen or that we do not want to publish to guests.
>>>>> Modify some values in the cpuinfo structure for guests to mask some
>>>>> features which we do not want to allow to guests (like AMU) or we do not
>>>>> support (like SVE).
>>>>> The code is trying to group together registers modifications for the
>>>>> same feature to be able in the long term to easily enable/disable a
>>>>> feature depending on user parameters or add other registers modification
>>>>> in the same place (like enabling/disabling HCR bits).
>>>>> Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
>>>>> ---
>>>>> Changes in V2: Rebase
>>>>> Changes in V3:
>>>>>    Use current_cpu_data info instead of recalling identify_cpu
>>>>> ---
>>>>>   xen/arch/arm/cpufeature.c        | 51 ++++++++++++++++++++++++++++++++
>>>>>   xen/include/asm-arm/cpufeature.h |  2 ++
>>>>>   2 files changed, 53 insertions(+)
>>>>> diff --git a/xen/arch/arm/cpufeature.c b/xen/arch/arm/cpufeature.c
>>>>> index bc7ee5ac95..7255383504 100644
>>>>> --- a/xen/arch/arm/cpufeature.c
>>>>> +++ b/xen/arch/arm/cpufeature.c
>>>>> @@ -24,6 +24,8 @@
>>>>>     DECLARE_BITMAP(cpu_hwcaps, ARM_NCAPS);
>>>>>   +struct cpuinfo_arm __read_mostly guest_cpuinfo;
>>>>> +
>>>>>   void update_cpu_capabilities(const struct arm_cpu_capabilities *caps,
>>>>>                                const char *info)
>>>>>   {
>>>>> @@ -157,6 +159,55 @@ void identify_cpu(struct cpuinfo_arm *c)
>>>>>   #endif
>>>>>   }
>>>>>   +/*
>>>>> + * This function is creating a cpuinfo structure with values modified to mask
>>>>> + * all cpu features that should not be published to guest.
>>>>> + * The created structure is then used to provide ID registers values to guests.
>>>>> + */
>>>>> +static int __init create_guest_cpuinfo(void)
>>>>> +{
>>>>> +    /*
>>>>> +     * TODO: The code is currently using only the features detected on the boot
>>>>> +     * core. In the long term we should try to compute values containing only
>>>>> +     * features supported by all cores.
>>>>> +     */
>>>>> +    guest_cpuinfo = current_cpu_data;
>>>>
>>>> It would be more logical to use boot_cpu_data as this would be easier to match with your comment.
>>> Agree, I will fix that in V4.
>>>>
>>>>> +
>>>>> +#ifdef CONFIG_ARM_64
>>>>> +    /* Disable MPAM as xen does not support it */
>>>>> +    guest_cpuinfo.pfr64.mpam = 0;
>>>>> +    guest_cpuinfo.pfr64.mpam_frac = 0;
>>>>> +
>>>>> +    /* Disable SVE as Xen does not support it */
>>>>> +    guest_cpuinfo.pfr64.sve = 0;
>>>>> +    guest_cpuinfo.zfr64.bits[0] = 0;
>>>>> +
>>>>> +    /* Disable MTE as Xen does not support it */
>>>>> +    guest_cpuinfo.pfr64.mte = 0;
>>>>> +#endif
>>>>> +
>>>>> +    /* Disable AMU */
>>>>> +#ifdef CONFIG_ARM_64
>>>>> +    guest_cpuinfo.pfr64.amu = 0;
>>>>> +#endif
>>>>> +    guest_cpuinfo.pfr32.amu = 0;
>>>>> +
>>>>> +    /* Disable RAS as Xen does not support it */
>>>>> +#ifdef CONFIG_ARM_64
>>>>> +    guest_cpuinfo.pfr64.ras = 0;
>>>>> +    guest_cpuinfo.pfr64.ras_frac = 0;
>>>>> +#endif
>>>>> +    guest_cpuinfo.pfr32.ras = 0;
>>>>> +    guest_cpuinfo.pfr32.ras_frac = 0;
>>>>
>>>> How about all the fields that are currently marked as RES0/RES1? Shouldn't we make sure they will stay like that even if newer architecture use them?
>>> Definitely we can do more then this here (including allowing to enable some things for dom0 or for test reasons).
>>> This is a first try to solve current issues with MPAM and SVE and I plan to continue to enhance this in the future
>>> to enable more customisation here.
>>> I do think we could do a bit more here to have some features controlled by the user but this will need a bit of
>>> discussion to agree on a strategy.
>>
>> I think you misunderstood my comment. I am not asking whether we want to customize the value per domain. Instead, I am raising questions for the strategy taken in this patch.
>>
>> I am going to leave the safety aside, because I think this is orthogonal to this patch.
>>
>> This patch is introducing a deny list. This means that all the features will be exposed to a domain unless someone determine that this is not
>> supported by Xen.
>>
>> This means we will always try to catch up with what Arm decided to invent and attempt to fix it before the silicon is out.
>>
>> Instead, I think it would be better to use an allow list. We should only expose features to the guest we know works (this could possibly be just the Armv8.0 one).
> 
> I understood that and I fully agree that this is what we should do: only expose what we support and know and let everything else as “disabled”.
> And I definitely want to do that and I think with this serie we have the required support to do that, we will need to rework how we initialise the
> guest_cpuinfo structure.
> 
> I just want to leave this discussion for after so that we can at least right now have a current linux booting without the need to modify the linux
> kernel binary to remove things like SVE.

Ok. So this patch is more to fill the gap rather than the final 
solution. This should be clarified in the commit message.

Although, it is still unclear to me why this can't be an allowlist from 
the start. As you said, the infrastructure is already there. So it would 
be a matter of copying bits we know work with Xen (rather than cloberring).

Cheers,

-- 
Julien Grall

Re: [PATCH v3 3/7] xen/arm: create a cpuinfo structure for guest

[Bertrand Marquis]

Hi Julien,

> On 10 Dec 2020, at 16:30, Julien Grall <julien@xen.org> wrote:
> 
> 
> 
> On 10/12/2020 16:17, Bertrand Marquis wrote:
>> Hi Julien,
> 
> Hi Bertrand,
> 
>>> On 10 Dec 2020, at 16:05, Julien Grall <julien@xen.org> wrote:
>>> 
>>> 
>>> 
>>> On 10/12/2020 15:48, Bertrand Marquis wrote:
>>>> Hi Julien,
>>>>> On 9 Dec 2020, at 23:09, Julien Grall <julien@xen.org> wrote:
>>>>> 
>>>>> Hi Bertand,
>>>>> 
>>>>> On 09/12/2020 16:30, Bertrand Marquis wrote:
>>>>>> Create a cpuinfo structure for guest and mask into it the features that
>>>>>> we do not support in Xen or that we do not want to publish to guests.
>>>>>> Modify some values in the cpuinfo structure for guests to mask some
>>>>>> features which we do not want to allow to guests (like AMU) or we do not
>>>>>> support (like SVE).
>>>>>> The code is trying to group together registers modifications for the
>>>>>> same feature to be able in the long term to easily enable/disable a
>>>>>> feature depending on user parameters or add other registers modification
>>>>>> in the same place (like enabling/disabling HCR bits).
>>>>>> Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
>>>>>> ---
>>>>>> Changes in V2: Rebase
>>>>>> Changes in V3:
>>>>>>   Use current_cpu_data info instead of recalling identify_cpu
>>>>>> ---
>>>>>>  xen/arch/arm/cpufeature.c        | 51 ++++++++++++++++++++++++++++++++
>>>>>>  xen/include/asm-arm/cpufeature.h |  2 ++
>>>>>>  2 files changed, 53 insertions(+)
>>>>>> diff --git a/xen/arch/arm/cpufeature.c b/xen/arch/arm/cpufeature.c
>>>>>> index bc7ee5ac95..7255383504 100644
>>>>>> --- a/xen/arch/arm/cpufeature.c
>>>>>> +++ b/xen/arch/arm/cpufeature.c
>>>>>> @@ -24,6 +24,8 @@
>>>>>>    DECLARE_BITMAP(cpu_hwcaps, ARM_NCAPS);
>>>>>>  +struct cpuinfo_arm __read_mostly guest_cpuinfo;
>>>>>> +
>>>>>>  void update_cpu_capabilities(const struct arm_cpu_capabilities *caps,
>>>>>>                               const char *info)
>>>>>>  {
>>>>>> @@ -157,6 +159,55 @@ void identify_cpu(struct cpuinfo_arm *c)
>>>>>>  #endif
>>>>>>  }
>>>>>>  +/*
>>>>>> + * This function is creating a cpuinfo structure with values modified to mask
>>>>>> + * all cpu features that should not be published to guest.
>>>>>> + * The created structure is then used to provide ID registers values to guests.
>>>>>> + */
>>>>>> +static int __init create_guest_cpuinfo(void)
>>>>>> +{
>>>>>> +    /*
>>>>>> +     * TODO: The code is currently using only the features detected on the boot
>>>>>> +     * core. In the long term we should try to compute values containing only
>>>>>> +     * features supported by all cores.
>>>>>> +     */
>>>>>> +    guest_cpuinfo = current_cpu_data;
>>>>> 
>>>>> It would be more logical to use boot_cpu_data as this would be easier to match with your comment.
>>>> Agree, I will fix that in V4.
>>>>> 
>>>>>> +
>>>>>> +#ifdef CONFIG_ARM_64
>>>>>> +    /* Disable MPAM as xen does not support it */
>>>>>> +    guest_cpuinfo.pfr64.mpam = 0;
>>>>>> +    guest_cpuinfo.pfr64.mpam_frac = 0;
>>>>>> +
>>>>>> +    /* Disable SVE as Xen does not support it */
>>>>>> +    guest_cpuinfo.pfr64.sve = 0;
>>>>>> +    guest_cpuinfo.zfr64.bits[0] = 0;
>>>>>> +
>>>>>> +    /* Disable MTE as Xen does not support it */
>>>>>> +    guest_cpuinfo.pfr64.mte = 0;
>>>>>> +#endif
>>>>>> +
>>>>>> +    /* Disable AMU */
>>>>>> +#ifdef CONFIG_ARM_64
>>>>>> +    guest_cpuinfo.pfr64.amu = 0;
>>>>>> +#endif
>>>>>> +    guest_cpuinfo.pfr32.amu = 0;
>>>>>> +
>>>>>> +    /* Disable RAS as Xen does not support it */
>>>>>> +#ifdef CONFIG_ARM_64
>>>>>> +    guest_cpuinfo.pfr64.ras = 0;
>>>>>> +    guest_cpuinfo.pfr64.ras_frac = 0;
>>>>>> +#endif
>>>>>> +    guest_cpuinfo.pfr32.ras = 0;
>>>>>> +    guest_cpuinfo.pfr32.ras_frac = 0;
>>>>> 
>>>>> How about all the fields that are currently marked as RES0/RES1? Shouldn't we make sure they will stay like that even if newer architecture use them?
>>>> Definitely we can do more then this here (including allowing to enable some things for dom0 or for test reasons).
>>>> This is a first try to solve current issues with MPAM and SVE and I plan to continue to enhance this in the future
>>>> to enable more customisation here.
>>>> I do think we could do a bit more here to have some features controlled by the user but this will need a bit of
>>>> discussion to agree on a strategy.
>>> 
>>> I think you misunderstood my comment. I am not asking whether we want to customize the value per domain. Instead, I am raising questions for the strategy taken in this patch.
>>> 
>>> I am going to leave the safety aside, because I think this is orthogonal to this patch.
>>> 
>>> This patch is introducing a deny list. This means that all the features will be exposed to a domain unless someone determine that this is not
>>> supported by Xen.
>>> 
>>> This means we will always try to catch up with what Arm decided to invent and attempt to fix it before the silicon is out.
>>> 
>>> Instead, I think it would be better to use an allow list. We should only expose features to the guest we know works (this could possibly be just the Armv8.0 one).
>> I understood that and I fully agree that this is what we should do: only expose what we support and know and let everything else as “disabled”.
>> And I definitely want to do that and I think with this serie we have the required support to do that, we will need to rework how we initialise the
>> guest_cpuinfo structure.
>> I just want to leave this discussion for after so that we can at least right now have a current linux booting without the need to modify the linux
>> kernel binary to remove things like SVE.
> 
> Ok. So this patch is more to fill the gap rather than the final solution. This should be clarified in the commit message.

I can add that in the commit message.

> 
> Although, it is still unclear to me why this can't be an allowlist from the start. As you said, the infrastructure is already there. So it would be a matter of copying bits we know work with Xen (rather than cloberring).

The analysis to do that properly might require more work as we should start from everything off and then going one by one and making sure we not only do that here but also in HCR register bits.

Here we are just explicitely “hiding” features which is somehow easy to review and check.

I will not have time to investigate deeper then what I did already before the next xen release.
So best i can do is change this patch to not modify anything in the guest_cpuinfo so that someone else can do that work on top of this serie.
I have planned some time to work on continuing this work, but not before march.

Regards
Bertrand

> 
> Cheers,
> 
> -- 
> Julien Grall

Re: [PATCH v3 4/7] xen/arm: Add handler for ID registers on arm64

[Stefano Stabellini]

On Thu, 10 Dec 2020, Bertrand Marquis wrote:
> Hi Stefano,
> 
> > On 9 Dec 2020, at 19:38, Stefano Stabellini <sstabellini@kernel.org> wrote:
> > 
> > On Wed, 9 Dec 2020, Bertrand Marquis wrote:
> >> Add vsysreg emulation for registers trapped when TID3 bit is activated
> >> in HSR.
> >> The emulation is returning the value stored in cpuinfo_guest structure
> >> for know registers and is handling reserved registers as RAZ.
> >> 
> >> Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
> >> ---
> >> Changes in V2: Rebase
> >> Changes in V3:
> >>  Fix commit message
> >>  Fix code style for GENERATE_TID3_INFO declaration
> >>  Add handling of reserved registers as RAZ.
> >> 
> >> ---
> >> xen/arch/arm/arm64/vsysreg.c | 53 ++++++++++++++++++++++++++++++++++++
> >> 1 file changed, 53 insertions(+)
> >> 
> >> diff --git a/xen/arch/arm/arm64/vsysreg.c b/xen/arch/arm/arm64/vsysreg.c
> >> index 8a85507d9d..ef7a11dbdd 100644
> >> --- a/xen/arch/arm/arm64/vsysreg.c
> >> +++ b/xen/arch/arm/arm64/vsysreg.c
> >> @@ -69,6 +69,14 @@ TVM_REG(CONTEXTIDR_EL1)
> >>         break;                                                          \
> >>     }
> >> 
> >> +/* Macro to generate easily case for ID co-processor emulation */
> >> +#define GENERATE_TID3_INFO(reg, field, offset)                          \
> >> +    case HSR_SYSREG_##reg:                                              \
> >> +    {                                                                   \
> >> +        return handle_ro_read_val(regs, regidx, hsr.sysreg.read, hsr,   \
> >> +                          1, guest_cpuinfo.field.bits[offset]);         \
> > 
> > [...]
> > 
> >> +    HSR_SYSREG_TID3_RESERVED_CASE:
> >> +        /* Handle all reserved registers as RAZ */
> >> +        return handle_ro_raz(regs, regidx, hsr.sysreg.read, hsr, 1);
> > 
> > 
> > We are implementing both the known and the implementation defined
> > registers as read-as-zero. On write, we inject an exception.
> > 
> > However, reading the manual, it looks like the implementation defined
> > registers should be read-as-zero/write-ignore, is that right?
> 
> In the documentation, I did find all those defined as RO (Arm Architecture
> reference manual, chapter D12.3.1). Do you think we should handle Read
> only register as write ignore ? now i think of it RO does not explicitely mean
> if writes are ignored or should generate an exception.
> 
> > 
> > I couldn't easily find in the manual if it is OK to inject an exception
> > on write to a known register.
> 
> I am actually unsure if it should or not.
> I will try to run a test to check what is happening if this is done on the
> real hardware and come back to you on this one.

Yeah, that's the best way to do it: if writes are ignored on real
hardware, let's turn this into read-only/write-ignore, otherwise if they
generate an exception then let's keep the code as is.

Also you might want to do that both for a known register and also for an
unknown register to see if it makes a difference.

Thank you!

Re: [PATCH v3 5/7] xen/arm: Add handler for cp15 ID registers

[Stefano Stabellini]

[-- Attachment #1: Type: text/plain, Size: 6382 bytes --]

On Thu, 10 Dec 2020, Bertrand Marquis wrote:
> > On 9 Dec 2020, at 19:54, Stefano Stabellini <sstabellini@kernel.org> wrote:
> > 
> > On Wed, 9 Dec 2020, Bertrand Marquis wrote:
> >> Add support for emulation of cp15 based ID registers (on arm32 or when
> >> running a 32bit guest on arm64).
> >> The handlers are returning the values stored in the guest_cpuinfo
> >> structure for known registers and RAZ for all reserved registers.
> >> In the current status the MVFR registers are no supported.
> >> 
> >> Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
> >> ---
> >> Changes in V2: Rebase
> >> Changes in V3:
> >>  Add case definition for reserved registers
> >>  Add handling of reserved registers as RAZ.
> >>  Fix code style in GENERATE_TID3_INFO declaration
> >> 
> >> ---
> >> xen/arch/arm/vcpreg.c        | 39 ++++++++++++++++++++++++++++++++++++
> >> xen/include/asm-arm/cpregs.h | 25 +++++++++++++++++++++++
> >> 2 files changed, 64 insertions(+)
> >> 
> >> diff --git a/xen/arch/arm/vcpreg.c b/xen/arch/arm/vcpreg.c
> >> index cdc91cdf5b..d371a1c38c 100644
> >> --- a/xen/arch/arm/vcpreg.c
> >> +++ b/xen/arch/arm/vcpreg.c
> >> @@ -155,6 +155,14 @@ TVM_REG32(CONTEXTIDR, CONTEXTIDR_EL1)
> >>         break;                                                      \
> >>     }
> >> 
> >> +/* Macro to generate easily case for ID co-processor emulation */
> >> +#define GENERATE_TID3_INFO(reg, field, offset)                      \
> >> +    case HSR_CPREG32(reg):                                          \
> >> +    {                                                               \
> >> +        return handle_ro_read_val(regs, regidx, cp32.read, hsr,     \
> >> +                          1, guest_cpuinfo.field.bits[offset]);     \
> >> +    }
> >> +
> >> void do_cp15_32(struct cpu_user_regs *regs, const union hsr hsr)
> >> {
> >>     const struct hsr_cp32 cp32 = hsr.cp32;
> >> @@ -286,6 +294,37 @@ void do_cp15_32(struct cpu_user_regs *regs, const union hsr hsr)
> >>          */
> >>         return handle_raz_wi(regs, regidx, cp32.read, hsr, 1);
> >> 
> >> +    /*
> >> +     * HCR_EL2.TID3
> >> +     *
> >> +     * This is trapping most Identification registers used by a guest
> >> +     * to identify the processor features
> >> +     */
> >> +    GENERATE_TID3_INFO(ID_PFR0, pfr32, 0)
> >> +    GENERATE_TID3_INFO(ID_PFR1, pfr32, 1)
> >> +    GENERATE_TID3_INFO(ID_PFR2, pfr32, 2)
> >> +    GENERATE_TID3_INFO(ID_DFR0, dbg32, 0)
> >> +    GENERATE_TID3_INFO(ID_DFR1, dbg32, 1)
> >> +    GENERATE_TID3_INFO(ID_AFR0, aux32, 0)
> >> +    GENERATE_TID3_INFO(ID_MMFR0, mm32, 0)
> >> +    GENERATE_TID3_INFO(ID_MMFR1, mm32, 1)
> >> +    GENERATE_TID3_INFO(ID_MMFR2, mm32, 2)
> >> +    GENERATE_TID3_INFO(ID_MMFR3, mm32, 3)
> >> +    GENERATE_TID3_INFO(ID_MMFR4, mm32, 4)
> >> +    GENERATE_TID3_INFO(ID_MMFR5, mm32, 5)
> >> +    GENERATE_TID3_INFO(ID_ISAR0, isa32, 0)
> >> +    GENERATE_TID3_INFO(ID_ISAR1, isa32, 1)
> >> +    GENERATE_TID3_INFO(ID_ISAR2, isa32, 2)
> >> +    GENERATE_TID3_INFO(ID_ISAR3, isa32, 3)
> >> +    GENERATE_TID3_INFO(ID_ISAR4, isa32, 4)
> >> +    GENERATE_TID3_INFO(ID_ISAR5, isa32, 5)
> >> +    GENERATE_TID3_INFO(ID_ISAR6, isa32, 6)
> >> +    /* MVFR registers are in cp10 no cp15 */
> >> +
> >> +    HSR_CPREG32_TID3_RESERVED_CASE:
> >> +        /* Handle all reserved registers as RAZ */
> >> +        return handle_ro_raz(regs, regidx, cp32.read, hsr, 1);
> > 
> > Same question as for the aarch64 case: do we need to do write-ignore
> > for the reserved registers?
> 
> Arm architecture reference manual is listing all those registers as RO including
> the reserved ones (cf table D12-2). This said I have no objection to make them
> write ignore but from my understanding this would not reflect the hardware
> behaviour.

I think so too, but if you are going to run a test on ARMv8 that's even
better. Then we can apply the same policy (ignore or exception) here
too.

 
> >>     /*
> >>      * HCR_EL2.TIDCP
> >>      *
> >> diff --git a/xen/include/asm-arm/cpregs.h b/xen/include/asm-arm/cpregs.h
> >> index 2690ddeb7a..5cb1ad5cbe 100644
> >> --- a/xen/include/asm-arm/cpregs.h
> >> +++ b/xen/include/asm-arm/cpregs.h
> >> @@ -133,6 +133,31 @@
> >> #define VPIDR           p15,4,c0,c0,0   /* Virtualization Processor ID Register */
> >> #define VMPIDR          p15,4,c0,c0,5   /* Virtualization Multiprocessor ID Register */
> >> 
> >> +/*
> >> + * Those cases are catching all Reserved registers trapped by TID3 which
> >> + * currently have no assignment.
> >> + * HCR.TID3 is trapping all registers in the group 3:
> >> + * coproc == p15, opc1 == 0, CRn == c0, CRm == {c2-c7}, opc2 == {0-7}.
> >> + */
> >> +#define HSR_CPREG32_TID3_CASES(REG)     case HSR_CPREG32(p15,0,c0,REG,0): \
> >> +                                        case HSR_CPREG32(p15,0,c0,REG,1): \
> >> +                                        case HSR_CPREG32(p15,0,c0,REG,2): \
> >> +                                        case HSR_CPREG32(p15,0,c0,REG,3): \
> >> +                                        case HSR_CPREG32(p15,0,c0,REG,4): \
> >> +                                        case HSR_CPREG32(p15,0,c0,REG,5): \
> >> +                                        case HSR_CPREG32(p15,0,c0,REG,6): \
> >> +                                        case HSR_CPREG32(p15,0,c0,REG,7)
> >> +
> >> +#define HSR_CPREG32_TID3_RESERVED_CASE  case HSR_CPREG32(p15,0,c0,c3,0): \
> >> +                                        case HSR_CPREG32(p15,0,c0,c3,1): \
> >> +                                        case HSR_CPREG32(p15,0,c0,c3,2): \
> >> +                                        case HSR_CPREG32(p15,0,c0,c3,3): \
> >> +                                        case HSR_CPREG32(p15,0,c0,c3,7): \
> >> +                                        HSR_CPREG32_TID3_CASES(c4): \
> >> +                                        HSR_CPREG32_TID3_CASES(c5): \
> >> +                                        HSR_CPREG32_TID3_CASES(c6): \
> >> +                                        HSR_CPREG32_TID3_CASES(c7)
> > 
> > The following are missing, is it a problem?
> > 
> > p15,0,c0,c0,2
> > p15,0,c0,c0,3
> > p15,0,c0,c0,4
> > p15,0,c0,c0,6
> > p15,0,c0,c0,7
> 
> HCR.TID3 documentation is saying that access to "coproc == p15, opc1 == 0, 
> CRn == c0, CRm == {c2-c7}, opc2 == {0-7}” are trapped so CRm = c0 is not handled here.

OK, thank you for checking

Re: [PATCH v3 4/7] xen/arm: Add handler for ID registers on arm64

[Bertrand Marquis]

Hi Stefano,

> On 10 Dec 2020, at 22:29, Stefano Stabellini <sstabellini@kernel.org> wrote:
> 
> On Thu, 10 Dec 2020, Bertrand Marquis wrote:
>> Hi Stefano,
>> 
>>> On 9 Dec 2020, at 19:38, Stefano Stabellini <sstabellini@kernel.org> wrote:
>>> 
>>> On Wed, 9 Dec 2020, Bertrand Marquis wrote:
>>>> Add vsysreg emulation for registers trapped when TID3 bit is activated
>>>> in HSR.
>>>> The emulation is returning the value stored in cpuinfo_guest structure
>>>> for know registers and is handling reserved registers as RAZ.
>>>> 
>>>> Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
>>>> ---
>>>> Changes in V2: Rebase
>>>> Changes in V3:
>>>> Fix commit message
>>>> Fix code style for GENERATE_TID3_INFO declaration
>>>> Add handling of reserved registers as RAZ.
>>>> 
>>>> ---
>>>> xen/arch/arm/arm64/vsysreg.c | 53 ++++++++++++++++++++++++++++++++++++
>>>> 1 file changed, 53 insertions(+)
>>>> 
>>>> diff --git a/xen/arch/arm/arm64/vsysreg.c b/xen/arch/arm/arm64/vsysreg.c
>>>> index 8a85507d9d..ef7a11dbdd 100644
>>>> --- a/xen/arch/arm/arm64/vsysreg.c
>>>> +++ b/xen/arch/arm/arm64/vsysreg.c
>>>> @@ -69,6 +69,14 @@ TVM_REG(CONTEXTIDR_EL1)
>>>>        break;                                                          \
>>>>    }
>>>> 
>>>> +/* Macro to generate easily case for ID co-processor emulation */
>>>> +#define GENERATE_TID3_INFO(reg, field, offset)                          \
>>>> +    case HSR_SYSREG_##reg:                                              \
>>>> +    {                                                                   \
>>>> +        return handle_ro_read_val(regs, regidx, hsr.sysreg.read, hsr,   \
>>>> +                          1, guest_cpuinfo.field.bits[offset]);         \
>>> 
>>> [...]
>>> 
>>>> +    HSR_SYSREG_TID3_RESERVED_CASE:
>>>> +        /* Handle all reserved registers as RAZ */
>>>> +        return handle_ro_raz(regs, regidx, hsr.sysreg.read, hsr, 1);
>>> 
>>> 
>>> We are implementing both the known and the implementation defined
>>> registers as read-as-zero. On write, we inject an exception.
>>> 
>>> However, reading the manual, it looks like the implementation defined
>>> registers should be read-as-zero/write-ignore, is that right?
>> 
>> In the documentation, I did find all those defined as RO (Arm Architecture
>> reference manual, chapter D12.3.1). Do you think we should handle Read
>> only register as write ignore ? now i think of it RO does not explicitely mean
>> if writes are ignored or should generate an exception.
>> 
>>> 
>>> I couldn't easily find in the manual if it is OK to inject an exception
>>> on write to a known register.
>> 
>> I am actually unsure if it should or not.
>> I will try to run a test to check what is happening if this is done on the
>> real hardware and come back to you on this one.
> 
> Yeah, that's the best way to do it: if writes are ignored on real
> hardware, let's turn this into read-only/write-ignore, otherwise if they
> generate an exception then let's keep the code as is.
> 
> Also you might want to do that both for a known register and also for an
> unknown register to see if it makes a difference.

I did a test with the following:
- WRITE_SYSREG64(0xf, S3_0_C0_C3_3)
- WRITE_SYSREG64(0xf, ID_MMFR0_EL1)
- WRITE_SYSREG64(0xf, ID_AA64MMFR0_EL1)

All generate exceptions like:
Hypervisor Trap. HSR=0x2000000 EC=0x0 IL=1 Syndrome=0x0

So I think it is right to generate an exception if one of them is accessed.

Regards
Bertrand

> 
> Thank you!

Re: [PATCH v3 4/7] xen/arm: Add handler for ID registers on arm64

[Stefano Stabellini]

On Fri, 11 Dec 2020, Bertrand Marquis wrote:
> Hi Stefano,
> 
> > On 10 Dec 2020, at 22:29, Stefano Stabellini <sstabellini@kernel.org> wrote:
> > 
> > On Thu, 10 Dec 2020, Bertrand Marquis wrote:
> >> Hi Stefano,
> >> 
> >>> On 9 Dec 2020, at 19:38, Stefano Stabellini <sstabellini@kernel.org> wrote:
> >>> 
> >>> On Wed, 9 Dec 2020, Bertrand Marquis wrote:
> >>>> Add vsysreg emulation for registers trapped when TID3 bit is activated
> >>>> in HSR.
> >>>> The emulation is returning the value stored in cpuinfo_guest structure
> >>>> for know registers and is handling reserved registers as RAZ.
> >>>> 
> >>>> Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
> >>>> ---
> >>>> Changes in V2: Rebase
> >>>> Changes in V3:
> >>>> Fix commit message
> >>>> Fix code style for GENERATE_TID3_INFO declaration
> >>>> Add handling of reserved registers as RAZ.
> >>>> 
> >>>> ---
> >>>> xen/arch/arm/arm64/vsysreg.c | 53 ++++++++++++++++++++++++++++++++++++
> >>>> 1 file changed, 53 insertions(+)
> >>>> 
> >>>> diff --git a/xen/arch/arm/arm64/vsysreg.c b/xen/arch/arm/arm64/vsysreg.c
> >>>> index 8a85507d9d..ef7a11dbdd 100644
> >>>> --- a/xen/arch/arm/arm64/vsysreg.c
> >>>> +++ b/xen/arch/arm/arm64/vsysreg.c
> >>>> @@ -69,6 +69,14 @@ TVM_REG(CONTEXTIDR_EL1)
> >>>>        break;                                                          \
> >>>>    }
> >>>> 
> >>>> +/* Macro to generate easily case for ID co-processor emulation */
> >>>> +#define GENERATE_TID3_INFO(reg, field, offset)                          \
> >>>> +    case HSR_SYSREG_##reg:                                              \
> >>>> +    {                                                                   \
> >>>> +        return handle_ro_read_val(regs, regidx, hsr.sysreg.read, hsr,   \
> >>>> +                          1, guest_cpuinfo.field.bits[offset]);         \
> >>> 
> >>> [...]
> >>> 
> >>>> +    HSR_SYSREG_TID3_RESERVED_CASE:
> >>>> +        /* Handle all reserved registers as RAZ */
> >>>> +        return handle_ro_raz(regs, regidx, hsr.sysreg.read, hsr, 1);
> >>> 
> >>> 
> >>> We are implementing both the known and the implementation defined
> >>> registers as read-as-zero. On write, we inject an exception.
> >>> 
> >>> However, reading the manual, it looks like the implementation defined
> >>> registers should be read-as-zero/write-ignore, is that right?
> >> 
> >> In the documentation, I did find all those defined as RO (Arm Architecture
> >> reference manual, chapter D12.3.1). Do you think we should handle Read
> >> only register as write ignore ? now i think of it RO does not explicitely mean
> >> if writes are ignored or should generate an exception.
> >> 
> >>> 
> >>> I couldn't easily find in the manual if it is OK to inject an exception
> >>> on write to a known register.
> >> 
> >> I am actually unsure if it should or not.
> >> I will try to run a test to check what is happening if this is done on the
> >> real hardware and come back to you on this one.
> > 
> > Yeah, that's the best way to do it: if writes are ignored on real
> > hardware, let's turn this into read-only/write-ignore, otherwise if they
> > generate an exception then let's keep the code as is.
> > 
> > Also you might want to do that both for a known register and also for an
> > unknown register to see if it makes a difference.
> 
> I did a test with the following:
> - WRITE_SYSREG64(0xf, S3_0_C0_C3_3)
> - WRITE_SYSREG64(0xf, ID_MMFR0_EL1)
> - WRITE_SYSREG64(0xf, ID_AA64MMFR0_EL1)
> 
> All generate exceptions like:
> Hypervisor Trap. HSR=0x2000000 EC=0x0 IL=1 Syndrome=0x0
> 
> So I think it is right to generate an exception if one of them is accessed.

Great, thanks for checking. In that case the patch is fine as is.