* [PATCH v3 0/7] xen/arm: Sanitize cpuinfo
@ 2021-08-25 13:18 Bertrand Marquis
2021-08-25 13:18 ` [PATCH v3 1/7] xen/arm: Import ID registers definitions from Linux Bertrand Marquis
` (6 more replies)
0 siblings, 7 replies; 26+ messages in thread
From: Bertrand Marquis @ 2021-08-25 13:18 UTC (permalink / raw)
To: xen-devel
Cc: Stefano Stabellini, Julien Grall, Volodymyr Babchuk,
Andrew Cooper, George Dunlap, Ian Jackson, Jan Beulich, Wei Liu
On arm architecture we might have heterogeneous platforms with different
types of cores. As a guest can potentialy run on any of those cores we
have to present them cpu features which are compatible with all cores
and discard the features which are only available on some cores.
As the features can be fairly complex, the way to deduce from 2
different features what should be the acceptable minimal feature can be
complex (and sometime impossible).
To reduce the implementation effort in Xen, this serie is importing the
structures and filtering system used by Linux in order to build a
cpuinfo containing the best values compatible with all cores on the
platform.
The serie start by importing the necessary code and structure from Linux
and then use it to sanitize the boot cpuinfo.
It is simplifying p2m code which was searching for the best support
value for PAR and VMID size.
On big.LITTLE platforms the current behaviour is kept and if hmp-unsafe
is provided in Xen command line then Xen is using all cores. This serie
is tainting Xen with CPU_OUT_OF_SPEC when different MIDR are found.
It is also adding DCZID and CTR registers in cpuinfo in order to check
for incoherent values between cores for those 2 registers. Xen is
tainted if different DCZID registers are found and CTR register is
sanitized when possible and in this case CTR register (and other
registers catched by HCR.TID2) is emulated.
Changes in v3:
- restore hmp-unsafe need to use all core when different midr are found
- taint Xen if different MIDR are found
- sanitize system_cpuinfo only if core is not stopped by Xen on boot
Changes in v2:
- Sanitize DCZID register
- Sanitize CTR_EL0 and add emulation of registers catched by TID2
- rename cpu_boot_data to system_cpuinfo
Bertrand Marquis (7):
xen/arm: Import ID registers definitions from Linux
xen/arm: Import ID features sanitize from linux
xen/arm: Rename cpu_boot_data to system_cpuinfo
xen/arm: Sanitize cpuinfo ID registers fields
xen/arm: Use sanitize values for p2m
xen/arm: Taint Xen on incompatible DCZID values
xen/arm: Sanitize CTR_EL0 and emulate it if needed
xen/arch/arm/arm64/Makefile | 1 +
xen/arch/arm/arm64/cpufeature.c | 644 +++++++++++++++++++++++++
xen/arch/arm/arm64/vsysreg.c | 40 ++
xen/arch/arm/cpufeature.c | 12 +-
xen/arch/arm/domain.c | 8 +
xen/arch/arm/p2m.c | 30 +-
xen/arch/arm/setup.c | 36 +-
xen/arch/arm/smpboot.c | 36 +-
xen/arch/arm/vcpreg.c | 45 ++
xen/common/kernel.c | 6 +-
xen/include/asm-arm/arm64/cpufeature.h | 104 ++++
xen/include/asm-arm/arm64/hsr.h | 6 +
xen/include/asm-arm/arm64/sysregs.h | 312 ++++++++++++
xen/include/asm-arm/cpufeature.h | 34 +-
xen/include/asm-arm/processor.h | 18 +-
xen/include/xen/lib.h | 1 +
16 files changed, 1274 insertions(+), 59 deletions(-)
create mode 100644 xen/arch/arm/arm64/cpufeature.c
create mode 100644 xen/include/asm-arm/arm64/cpufeature.h
--
2.17.1
^ permalink raw reply [flat|nested] 26+ messages in thread
* [PATCH v3 1/7] xen/arm: Import ID registers definitions from Linux
2021-08-25 13:18 [PATCH v3 0/7] xen/arm: Sanitize cpuinfo Bertrand Marquis
@ 2021-08-25 13:18 ` Bertrand Marquis
2021-08-27 14:40 ` Julien Grall
2021-08-25 13:18 ` [PATCH v3 2/7] xen/arm: Import ID features sanitize from linux Bertrand Marquis
` (5 subsequent siblings)
6 siblings, 1 reply; 26+ messages in thread
From: Bertrand Marquis @ 2021-08-25 13:18 UTC (permalink / raw)
To: xen-devel; +Cc: Stefano Stabellini, Julien Grall, Volodymyr Babchuk
Import some ID registers definitions from Linux sysreg header to have
required shift definitions for all ID registers fields.
Those are required to reuse the cpufeature sanitization system from
Linux kernel.
Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
---
Changes in v3: none
Changes in v2: Rebase
---
xen/include/asm-arm/arm64/sysregs.h | 312 ++++++++++++++++++++++++++++
1 file changed, 312 insertions(+)
diff --git a/xen/include/asm-arm/arm64/sysregs.h b/xen/include/asm-arm/arm64/sysregs.h
index 795901e1ba..d7e4772f21 100644
--- a/xen/include/asm-arm/arm64/sysregs.h
+++ b/xen/include/asm-arm/arm64/sysregs.h
@@ -85,6 +85,318 @@
#define ID_DFR1_EL1 S3_0_C0_C3_5
#endif
+/* ID registers (imported from arm64/include/asm/sysreg.h in Linux) */
+
+/* id_aa64isar0 */
+#define ID_AA64ISAR0_RNDR_SHIFT 60
+#define ID_AA64ISAR0_TLB_SHIFT 56
+#define ID_AA64ISAR0_TS_SHIFT 52
+#define ID_AA64ISAR0_FHM_SHIFT 48
+#define ID_AA64ISAR0_DP_SHIFT 44
+#define ID_AA64ISAR0_SM4_SHIFT 40
+#define ID_AA64ISAR0_SM3_SHIFT 36
+#define ID_AA64ISAR0_SHA3_SHIFT 32
+#define ID_AA64ISAR0_RDM_SHIFT 28
+#define ID_AA64ISAR0_ATOMICS_SHIFT 20
+#define ID_AA64ISAR0_CRC32_SHIFT 16
+#define ID_AA64ISAR0_SHA2_SHIFT 12
+#define ID_AA64ISAR0_SHA1_SHIFT 8
+#define ID_AA64ISAR0_AES_SHIFT 4
+
+#define ID_AA64ISAR0_TLB_RANGE_NI 0x0
+#define ID_AA64ISAR0_TLB_RANGE 0x2
+
+/* id_aa64isar1 */
+#define ID_AA64ISAR1_I8MM_SHIFT 52
+#define ID_AA64ISAR1_DGH_SHIFT 48
+#define ID_AA64ISAR1_BF16_SHIFT 44
+#define ID_AA64ISAR1_SPECRES_SHIFT 40
+#define ID_AA64ISAR1_SB_SHIFT 36
+#define ID_AA64ISAR1_FRINTTS_SHIFT 32
+#define ID_AA64ISAR1_GPI_SHIFT 28
+#define ID_AA64ISAR1_GPA_SHIFT 24
+#define ID_AA64ISAR1_LRCPC_SHIFT 20
+#define ID_AA64ISAR1_FCMA_SHIFT 16
+#define ID_AA64ISAR1_JSCVT_SHIFT 12
+#define ID_AA64ISAR1_API_SHIFT 8
+#define ID_AA64ISAR1_APA_SHIFT 4
+#define ID_AA64ISAR1_DPB_SHIFT 0
+
+#define ID_AA64ISAR1_APA_NI 0x0
+#define ID_AA64ISAR1_APA_ARCHITECTED 0x1
+#define ID_AA64ISAR1_APA_ARCH_EPAC 0x2
+#define ID_AA64ISAR1_APA_ARCH_EPAC2 0x3
+#define ID_AA64ISAR1_APA_ARCH_EPAC2_FPAC 0x4
+#define ID_AA64ISAR1_APA_ARCH_EPAC2_FPAC_CMB 0x5
+#define ID_AA64ISAR1_API_NI 0x0
+#define ID_AA64ISAR1_API_IMP_DEF 0x1
+#define ID_AA64ISAR1_API_IMP_DEF_EPAC 0x2
+#define ID_AA64ISAR1_API_IMP_DEF_EPAC2 0x3
+#define ID_AA64ISAR1_API_IMP_DEF_EPAC2_FPAC 0x4
+#define ID_AA64ISAR1_API_IMP_DEF_EPAC2_FPAC_CMB 0x5
+#define ID_AA64ISAR1_GPA_NI 0x0
+#define ID_AA64ISAR1_GPA_ARCHITECTED 0x1
+#define ID_AA64ISAR1_GPI_NI 0x0
+#define ID_AA64ISAR1_GPI_IMP_DEF 0x1
+
+/* id_aa64pfr0 */
+#define ID_AA64PFR0_CSV3_SHIFT 60
+#define ID_AA64PFR0_CSV2_SHIFT 56
+#define ID_AA64PFR0_DIT_SHIFT 48
+#define ID_AA64PFR0_AMU_SHIFT 44
+#define ID_AA64PFR0_MPAM_SHIFT 40
+#define ID_AA64PFR0_SEL2_SHIFT 36
+#define ID_AA64PFR0_SVE_SHIFT 32
+#define ID_AA64PFR0_RAS_SHIFT 28
+#define ID_AA64PFR0_GIC_SHIFT 24
+#define ID_AA64PFR0_ASIMD_SHIFT 20
+#define ID_AA64PFR0_FP_SHIFT 16
+#define ID_AA64PFR0_EL3_SHIFT 12
+#define ID_AA64PFR0_EL2_SHIFT 8
+#define ID_AA64PFR0_EL1_SHIFT 4
+#define ID_AA64PFR0_EL0_SHIFT 0
+
+#define ID_AA64PFR0_AMU 0x1
+#define ID_AA64PFR0_SVE 0x1
+#define ID_AA64PFR0_RAS_V1 0x1
+#define ID_AA64PFR0_FP_NI 0xf
+#define ID_AA64PFR0_FP_SUPPORTED 0x0
+#define ID_AA64PFR0_ASIMD_NI 0xf
+#define ID_AA64PFR0_ASIMD_SUPPORTED 0x0
+#define ID_AA64PFR0_EL1_64BIT_ONLY 0x1
+#define ID_AA64PFR0_EL1_32BIT_64BIT 0x2
+#define ID_AA64PFR0_EL0_64BIT_ONLY 0x1
+#define ID_AA64PFR0_EL0_32BIT_64BIT 0x2
+
+/* id_aa64pfr1 */
+#define ID_AA64PFR1_MPAMFRAC_SHIFT 16
+#define ID_AA64PFR1_RASFRAC_SHIFT 12
+#define ID_AA64PFR1_MTE_SHIFT 8
+#define ID_AA64PFR1_SSBS_SHIFT 4
+#define ID_AA64PFR1_BT_SHIFT 0
+
+#define ID_AA64PFR1_SSBS_PSTATE_NI 0
+#define ID_AA64PFR1_SSBS_PSTATE_ONLY 1
+#define ID_AA64PFR1_SSBS_PSTATE_INSNS 2
+#define ID_AA64PFR1_BT_BTI 0x1
+
+#define ID_AA64PFR1_MTE_NI 0x0
+#define ID_AA64PFR1_MTE_EL0 0x1
+#define ID_AA64PFR1_MTE 0x2
+
+/* id_aa64zfr0 */
+#define ID_AA64ZFR0_F64MM_SHIFT 56
+#define ID_AA64ZFR0_F32MM_SHIFT 52
+#define ID_AA64ZFR0_I8MM_SHIFT 44
+#define ID_AA64ZFR0_SM4_SHIFT 40
+#define ID_AA64ZFR0_SHA3_SHIFT 32
+#define ID_AA64ZFR0_BF16_SHIFT 20
+#define ID_AA64ZFR0_BITPERM_SHIFT 16
+#define ID_AA64ZFR0_AES_SHIFT 4
+#define ID_AA64ZFR0_SVEVER_SHIFT 0
+
+#define ID_AA64ZFR0_F64MM 0x1
+#define ID_AA64ZFR0_F32MM 0x1
+#define ID_AA64ZFR0_I8MM 0x1
+#define ID_AA64ZFR0_BF16 0x1
+#define ID_AA64ZFR0_SM4 0x1
+#define ID_AA64ZFR0_SHA3 0x1
+#define ID_AA64ZFR0_BITPERM 0x1
+#define ID_AA64ZFR0_AES 0x1
+#define ID_AA64ZFR0_AES_PMULL 0x2
+#define ID_AA64ZFR0_SVEVER_SVE2 0x1
+
+/* id_aa64mmfr0 */
+#define ID_AA64MMFR0_ECV_SHIFT 60
+#define ID_AA64MMFR0_FGT_SHIFT 56
+#define ID_AA64MMFR0_EXS_SHIFT 44
+#define ID_AA64MMFR0_TGRAN4_2_SHIFT 40
+#define ID_AA64MMFR0_TGRAN64_2_SHIFT 36
+#define ID_AA64MMFR0_TGRAN16_2_SHIFT 32
+#define ID_AA64MMFR0_TGRAN4_SHIFT 28
+#define ID_AA64MMFR0_TGRAN64_SHIFT 24
+#define ID_AA64MMFR0_TGRAN16_SHIFT 20
+#define ID_AA64MMFR0_BIGENDEL0_SHIFT 16
+#define ID_AA64MMFR0_SNSMEM_SHIFT 12
+#define ID_AA64MMFR0_BIGENDEL_SHIFT 8
+#define ID_AA64MMFR0_ASID_SHIFT 4
+#define ID_AA64MMFR0_PARANGE_SHIFT 0
+
+#define ID_AA64MMFR0_TGRAN4_NI 0xf
+#define ID_AA64MMFR0_TGRAN4_SUPPORTED 0x0
+#define ID_AA64MMFR0_TGRAN64_NI 0xf
+#define ID_AA64MMFR0_TGRAN64_SUPPORTED 0x0
+#define ID_AA64MMFR0_TGRAN16_NI 0x0
+#define ID_AA64MMFR0_TGRAN16_SUPPORTED 0x1
+#define ID_AA64MMFR0_PARANGE_48 0x5
+#define ID_AA64MMFR0_PARANGE_52 0x6
+
+/* id_aa64mmfr1 */
+#define ID_AA64MMFR1_ETS_SHIFT 36
+#define ID_AA64MMFR1_TWED_SHIFT 32
+#define ID_AA64MMFR1_XNX_SHIFT 28
+#define ID_AA64MMFR1_SPECSEI_SHIFT 24
+#define ID_AA64MMFR1_PAN_SHIFT 20
+#define ID_AA64MMFR1_LOR_SHIFT 16
+#define ID_AA64MMFR1_HPD_SHIFT 12
+#define ID_AA64MMFR1_VHE_SHIFT 8
+#define ID_AA64MMFR1_VMIDBITS_SHIFT 4
+#define ID_AA64MMFR1_HADBS_SHIFT 0
+
+#define ID_AA64MMFR1_VMIDBITS_8 0
+#define ID_AA64MMFR1_VMIDBITS_16 2
+
+/* id_aa64mmfr2 */
+#define ID_AA64MMFR2_E0PD_SHIFT 60
+#define ID_AA64MMFR2_EVT_SHIFT 56
+#define ID_AA64MMFR2_BBM_SHIFT 52
+#define ID_AA64MMFR2_TTL_SHIFT 48
+#define ID_AA64MMFR2_FWB_SHIFT 40
+#define ID_AA64MMFR2_IDS_SHIFT 36
+#define ID_AA64MMFR2_AT_SHIFT 32
+#define ID_AA64MMFR2_ST_SHIFT 28
+#define ID_AA64MMFR2_NV_SHIFT 24
+#define ID_AA64MMFR2_CCIDX_SHIFT 20
+#define ID_AA64MMFR2_LVA_SHIFT 16
+#define ID_AA64MMFR2_IESB_SHIFT 12
+#define ID_AA64MMFR2_LSM_SHIFT 8
+#define ID_AA64MMFR2_UAO_SHIFT 4
+#define ID_AA64MMFR2_CNP_SHIFT 0
+
+/* id_aa64dfr0 */
+#define ID_AA64DFR0_DOUBLELOCK_SHIFT 36
+#define ID_AA64DFR0_PMSVER_SHIFT 32
+#define ID_AA64DFR0_CTX_CMPS_SHIFT 28
+#define ID_AA64DFR0_WRPS_SHIFT 20
+#define ID_AA64DFR0_BRPS_SHIFT 12
+#define ID_AA64DFR0_PMUVER_SHIFT 8
+#define ID_AA64DFR0_TRACEVER_SHIFT 4
+#define ID_AA64DFR0_DEBUGVER_SHIFT 0
+
+#define ID_AA64DFR0_PMUVER_8_0 0x1
+#define ID_AA64DFR0_PMUVER_8_1 0x4
+#define ID_AA64DFR0_PMUVER_8_4 0x5
+#define ID_AA64DFR0_PMUVER_8_5 0x6
+#define ID_AA64DFR0_PMUVER_IMP_DEF 0xf
+
+#define ID_DFR0_PERFMON_SHIFT 24
+
+#define ID_DFR0_PERFMON_8_1 0x4
+
+#define ID_ISAR4_SWP_FRAC_SHIFT 28
+#define ID_ISAR4_PSR_M_SHIFT 24
+#define ID_ISAR4_SYNCH_PRIM_FRAC_SHIFT 20
+#define ID_ISAR4_BARRIER_SHIFT 16
+#define ID_ISAR4_SMC_SHIFT 12
+#define ID_ISAR4_WRITEBACK_SHIFT 8
+#define ID_ISAR4_WITHSHIFTS_SHIFT 4
+#define ID_ISAR4_UNPRIV_SHIFT 0
+
+#define ID_DFR1_MTPMU_SHIFT 0
+
+#define ID_ISAR0_DIVIDE_SHIFT 24
+#define ID_ISAR0_DEBUG_SHIFT 20
+#define ID_ISAR0_COPROC_SHIFT 16
+#define ID_ISAR0_CMPBRANCH_SHIFT 12
+#define ID_ISAR0_BITFIELD_SHIFT 8
+#define ID_ISAR0_BITCOUNT_SHIFT 4
+#define ID_ISAR0_SWAP_SHIFT 0
+
+#define ID_ISAR5_RDM_SHIFT 24
+#define ID_ISAR5_CRC32_SHIFT 16
+#define ID_ISAR5_SHA2_SHIFT 12
+#define ID_ISAR5_SHA1_SHIFT 8
+#define ID_ISAR5_AES_SHIFT 4
+#define ID_ISAR5_SEVL_SHIFT 0
+
+#define ID_ISAR6_I8MM_SHIFT 24
+#define ID_ISAR6_BF16_SHIFT 20
+#define ID_ISAR6_SPECRES_SHIFT 16
+#define ID_ISAR6_SB_SHIFT 12
+#define ID_ISAR6_FHM_SHIFT 8
+#define ID_ISAR6_DP_SHIFT 4
+#define ID_ISAR6_JSCVT_SHIFT 0
+
+#define ID_MMFR0_INNERSHR_SHIFT 28
+#define ID_MMFR0_FCSE_SHIFT 24
+#define ID_MMFR0_AUXREG_SHIFT 20
+#define ID_MMFR0_TCM_SHIFT 16
+#define ID_MMFR0_SHARELVL_SHIFT 12
+#define ID_MMFR0_OUTERSHR_SHIFT 8
+#define ID_MMFR0_PMSA_SHIFT 4
+#define ID_MMFR0_VMSA_SHIFT 0
+
+#define ID_MMFR4_EVT_SHIFT 28
+#define ID_MMFR4_CCIDX_SHIFT 24
+#define ID_MMFR4_LSM_SHIFT 20
+#define ID_MMFR4_HPDS_SHIFT 16
+#define ID_MMFR4_CNP_SHIFT 12
+#define ID_MMFR4_XNX_SHIFT 8
+#define ID_MMFR4_AC2_SHIFT 4
+#define ID_MMFR4_SPECSEI_SHIFT 0
+
+#define ID_MMFR5_ETS_SHIFT 0
+
+#define ID_PFR0_DIT_SHIFT 24
+#define ID_PFR0_CSV2_SHIFT 16
+#define ID_PFR0_STATE3_SHIFT 12
+#define ID_PFR0_STATE2_SHIFT 8
+#define ID_PFR0_STATE1_SHIFT 4
+#define ID_PFR0_STATE0_SHIFT 0
+
+#define ID_DFR0_PERFMON_SHIFT 24
+#define ID_DFR0_MPROFDBG_SHIFT 20
+#define ID_DFR0_MMAPTRC_SHIFT 16
+#define ID_DFR0_COPTRC_SHIFT 12
+#define ID_DFR0_MMAPDBG_SHIFT 8
+#define ID_DFR0_COPSDBG_SHIFT 4
+#define ID_DFR0_COPDBG_SHIFT 0
+
+#define ID_PFR2_SSBS_SHIFT 4
+#define ID_PFR2_CSV3_SHIFT 0
+
+#define MVFR0_FPROUND_SHIFT 28
+#define MVFR0_FPSHVEC_SHIFT 24
+#define MVFR0_FPSQRT_SHIFT 20
+#define MVFR0_FPDIVIDE_SHIFT 16
+#define MVFR0_FPTRAP_SHIFT 12
+#define MVFR0_FPDP_SHIFT 8
+#define MVFR0_FPSP_SHIFT 4
+#define MVFR0_SIMD_SHIFT 0
+
+#define MVFR1_SIMDFMAC_SHIFT 28
+#define MVFR1_FPHP_SHIFT 24
+#define MVFR1_SIMDHP_SHIFT 20
+#define MVFR1_SIMDSP_SHIFT 16
+#define MVFR1_SIMDINT_SHIFT 12
+#define MVFR1_SIMDLS_SHIFT 8
+#define MVFR1_FPDNAN_SHIFT 4
+#define MVFR1_FPFTZ_SHIFT 0
+
+#define ID_PFR1_GIC_SHIFT 28
+#define ID_PFR1_VIRT_FRAC_SHIFT 24
+#define ID_PFR1_SEC_FRAC_SHIFT 20
+#define ID_PFR1_GENTIMER_SHIFT 16
+#define ID_PFR1_VIRTUALIZATION_SHIFT 12
+#define ID_PFR1_MPROGMOD_SHIFT 8
+#define ID_PFR1_SECURITY_SHIFT 4
+#define ID_PFR1_PROGMOD_SHIFT 0
+
+#define MVFR2_FPMISC_SHIFT 4
+#define MVFR2_SIMDMISC_SHIFT 0
+
+#define DCZID_DZP_SHIFT 4
+#define DCZID_BS_SHIFT 0
+
+/*
+ * The ZCR_ELx_LEN_* definitions intentionally include bits [8:4] which
+ * are reserved by the SVE architecture for future expansion of the LEN
+ * field, with compatible semantics.
+ */
+#define ZCR_ELx_LEN_SHIFT 0
+#define ZCR_ELx_LEN_SIZE 9
+#define ZCR_ELx_LEN_MASK 0x1ff
+
/* Access to system registers */
#define WRITE_SYSREG64(v, name) do { \
--
2.17.1
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [PATCH v3 2/7] xen/arm: Import ID features sanitize from linux
2021-08-25 13:18 [PATCH v3 0/7] xen/arm: Sanitize cpuinfo Bertrand Marquis
2021-08-25 13:18 ` [PATCH v3 1/7] xen/arm: Import ID registers definitions from Linux Bertrand Marquis
@ 2021-08-25 13:18 ` Bertrand Marquis
2021-09-03 22:48 ` Stefano Stabellini
2021-08-25 13:18 ` [PATCH v3 3/7] xen/arm: Rename cpu_boot_data to system_cpuinfo Bertrand Marquis
` (4 subsequent siblings)
6 siblings, 1 reply; 26+ messages in thread
From: Bertrand Marquis @ 2021-08-25 13:18 UTC (permalink / raw)
To: xen-devel; +Cc: Stefano Stabellini, Julien Grall, Volodymyr Babchuk
Import structures declared in Linux file arch/arm64/kernel/cpufeature.c
and the required types from arch/arm64/include/asm/cpufeature.h.
Current code has been imported from Linux 5.13-rc5 (Commit ID
cd1245d75ce93b8fd206f4b34eb58bcfe156d5e9) and copied into cpufeature.c
in arm64 code and cpufeature.h in arm64 specific headers.
Those structure will be used to sanitize the cpu features available to
the ones availble on all cores of a system even if we are on an
heterogeneous platform (from example a big/LITTLE).
For each feature field of all ID registers, those structures define what
is the safest value and if we can allow to have different values in
different cores.
This patch is introducing Linux code without any changes to it.
Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
---
Changes in v3: none
Changes in v2:
- Move add to Makefile to following patch to allow bisection
- Remove GPL text as SPDL is there
- Re-add introduction comment from Linux Kernel file
- Rename cpusanitize.c to cpufeature.c to keep Linux file name
- Move structures imported from linux headers into a new cpufeature.h
header in asm-arm/arm64.
- Move comment about imported code origin to the file header
- Remove not needed linux function declarations instead of removing them
in the following patch
- Add original arm64_ftr_safe_value from Linux
- include kernel.h to use max()
- remove unused ftr_single32 as we will not use it
- remove ctr associated structures that we cannot use (keep the one
defining sanitization bits)
---
xen/arch/arm/arm64/cpufeature.c | 504 +++++++++++++++++++++++++
xen/include/asm-arm/arm64/cpufeature.h | 104 +++++
2 files changed, 608 insertions(+)
create mode 100644 xen/arch/arm/arm64/cpufeature.c
create mode 100644 xen/include/asm-arm/arm64/cpufeature.h
diff --git a/xen/arch/arm/arm64/cpufeature.c b/xen/arch/arm/arm64/cpufeature.c
new file mode 100644
index 0000000000..5777e33e5c
--- /dev/null
+++ b/xen/arch/arm/arm64/cpufeature.c
@@ -0,0 +1,504 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Contains CPU feature definitions
+ *
+ * The following structures have been imported directly from Linux kernel and
+ * should be kept in sync.
+ * The current version has been imported from arch/arm64/kernel/cpufeature.c
+ * from kernel version 5.13-rc5 together with the required structures and
+ * macros from arch/arm64/include/asm/cpufeature.h which are stored in
+ * include/asm-arm/arm64/cpufeature.h
+ *
+ * Copyright (C) 2021 Arm Ltd.
+ * based on code from the Linux kernel, which is:
+ * Copyright (C) 2015 ARM Ltd.
+ *
+ * A note for the weary kernel hacker: the code here is confusing and hard to
+ * follow! That's partly because it's solving a nasty problem, but also because
+ * there's a little bit of over-abstraction that tends to obscure what's going
+ * on behind a maze of helper functions and macros.
+ *
+ * The basic problem is that hardware folks have started gluing together CPUs
+ * with distinct architectural features; in some cases even creating SoCs where
+ * user-visible instructions are available only on a subset of the available
+ * cores. We try to address this by snapshotting the feature registers of the
+ * boot CPU and comparing these with the feature registers of each secondary
+ * CPU when bringing them up. If there is a mismatch, then we update the
+ * snapshot state to indicate the lowest-common denominator of the feature,
+ * known as the "safe" value. This snapshot state can be queried to view the
+ * "sanitised" value of a feature register.
+ *
+ * The sanitised register values are used to decide which capabilities we
+ * have in the system. These may be in the form of traditional "hwcaps"
+ * advertised to userspace or internal "cpucaps" which are used to configure
+ * things like alternative patching and static keys. While a feature mismatch
+ * may result in a TAINT_CPU_OUT_OF_SPEC kernel taint, a capability mismatch
+ * may prevent a CPU from being onlined at all.
+ *
+ * Some implementation details worth remembering:
+ *
+ * - Mismatched features are *always* sanitised to a "safe" value, which
+ * usually indicates that the feature is not supported.
+ *
+ * - A mismatched feature marked with FTR_STRICT will cause a "SANITY CHECK"
+ * warning when onlining an offending CPU and the kernel will be tainted
+ * with TAINT_CPU_OUT_OF_SPEC.
+ *
+ * - Features marked as FTR_VISIBLE have their sanitised value visible to
+ * userspace. FTR_VISIBLE features in registers that are only visible
+ * to EL0 by trapping *must* have a corresponding HWCAP so that late
+ * onlining of CPUs cannot lead to features disappearing at runtime.
+ *
+ * - A "feature" is typically a 4-bit register field. A "capability" is the
+ * high-level description derived from the sanitised field value.
+ *
+ * - Read the Arm ARM (DDI 0487F.a) section D13.1.3 ("Principles of the ID
+ * scheme for fields in ID registers") to understand when feature fields
+ * may be signed or unsigned (FTR_SIGNED and FTR_UNSIGNED accordingly).
+ *
+ * - KVM exposes its own view of the feature registers to guest operating
+ * systems regardless of FTR_VISIBLE. This is typically driven from the
+ * sanitised register values to allow virtual CPUs to be migrated between
+ * arbitrary physical CPUs, but some features not present on the host are
+ * also advertised and emulated. Look at sys_reg_descs[] for the gory
+ * details.
+ *
+ * - If the arm64_ftr_bits[] for a register has a missing field, then this
+ * field is treated as STRICT RES0, including for read_sanitised_ftr_reg().
+ * This is stronger than FTR_HIDDEN and can be used to hide features from
+ * KVM guests.
+ */
+
+#include <xen/types.h>
+#include <xen/kernel.h>
+#include <asm/sysregs.h>
+#include <asm/cpufeature.h>
+#include <asm/arm64/cpufeature.h>
+
+#define __ARM64_FTR_BITS(SIGNED, VISIBLE, STRICT, TYPE, SHIFT, WIDTH, SAFE_VAL) \
+ { \
+ .sign = SIGNED, \
+ .visible = VISIBLE, \
+ .strict = STRICT, \
+ .type = TYPE, \
+ .shift = SHIFT, \
+ .width = WIDTH, \
+ .safe_val = SAFE_VAL, \
+ }
+
+/* Define a feature with unsigned values */
+#define ARM64_FTR_BITS(VISIBLE, STRICT, TYPE, SHIFT, WIDTH, SAFE_VAL) \
+ __ARM64_FTR_BITS(FTR_UNSIGNED, VISIBLE, STRICT, TYPE, SHIFT, WIDTH, SAFE_VAL)
+
+/* Define a feature with a signed value */
+#define S_ARM64_FTR_BITS(VISIBLE, STRICT, TYPE, SHIFT, WIDTH, SAFE_VAL) \
+ __ARM64_FTR_BITS(FTR_SIGNED, VISIBLE, STRICT, TYPE, SHIFT, WIDTH, SAFE_VAL)
+
+#define ARM64_FTR_END \
+ { \
+ .width = 0, \
+ }
+
+/*
+ * NOTE: Any changes to the visibility of features should be kept in
+ * sync with the documentation of the CPU feature register ABI.
+ */
+static const struct arm64_ftr_bits ftr_id_aa64isar0[] = {
+ ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR0_RNDR_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR0_TLB_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR0_TS_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR0_FHM_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR0_DP_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR0_SM4_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR0_SM3_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR0_SHA3_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR0_RDM_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR0_ATOMICS_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR0_CRC32_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR0_SHA2_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR0_SHA1_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR0_AES_SHIFT, 4, 0),
+ ARM64_FTR_END,
+};
+
+static const struct arm64_ftr_bits ftr_id_aa64isar1[] = {
+ ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR1_I8MM_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR1_DGH_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR1_BF16_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR1_SPECRES_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR1_SB_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR1_FRINTTS_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_PTR_AUTH),
+ FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR1_GPI_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_PTR_AUTH),
+ FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR1_GPA_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR1_LRCPC_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR1_FCMA_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR1_JSCVT_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_PTR_AUTH),
+ FTR_STRICT, FTR_EXACT, ID_AA64ISAR1_API_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_PTR_AUTH),
+ FTR_STRICT, FTR_EXACT, ID_AA64ISAR1_APA_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR1_DPB_SHIFT, 4, 0),
+ ARM64_FTR_END,
+};
+
+static const struct arm64_ftr_bits ftr_id_aa64pfr0[] = {
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64PFR0_CSV3_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64PFR0_CSV2_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR0_DIT_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64PFR0_AMU_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR0_MPAM_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64PFR0_SEL2_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_SVE),
+ FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR0_SVE_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR0_RAS_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR0_GIC_SHIFT, 4, 0),
+ S_ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR0_ASIMD_SHIFT, 4, ID_AA64PFR0_ASIMD_NI),
+ S_ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR0_FP_SHIFT, 4, ID_AA64PFR0_FP_NI),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64PFR0_EL3_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64PFR0_EL2_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64PFR0_EL1_SHIFT, 4, ID_AA64PFR0_EL1_64BIT_ONLY),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64PFR0_EL0_SHIFT, 4, ID_AA64PFR0_EL0_64BIT_ONLY),
+ ARM64_FTR_END,
+};
+
+static const struct arm64_ftr_bits ftr_id_aa64pfr1[] = {
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR1_MPAMFRAC_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR1_RASFRAC_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_MTE),
+ FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR1_MTE_SHIFT, 4, ID_AA64PFR1_MTE_NI),
+ ARM64_FTR_BITS(FTR_VISIBLE, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64PFR1_SSBS_SHIFT, 4, ID_AA64PFR1_SSBS_PSTATE_NI),
+ ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_BTI),
+ FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR1_BT_SHIFT, 4, 0),
+ ARM64_FTR_END,
+};
+
+static const struct arm64_ftr_bits ftr_id_aa64zfr0[] = {
+ ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_SVE),
+ FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ZFR0_F64MM_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_SVE),
+ FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ZFR0_F32MM_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_SVE),
+ FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ZFR0_I8MM_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_SVE),
+ FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ZFR0_SM4_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_SVE),
+ FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ZFR0_SHA3_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_SVE),
+ FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ZFR0_BF16_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_SVE),
+ FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ZFR0_BITPERM_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_SVE),
+ FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ZFR0_AES_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_SVE),
+ FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ZFR0_SVEVER_SHIFT, 4, 0),
+ ARM64_FTR_END,
+};
+
+static const struct arm64_ftr_bits ftr_id_aa64mmfr0[] = {
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR0_ECV_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR0_FGT_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR0_EXS_SHIFT, 4, 0),
+ /*
+ * Page size not being supported at Stage-2 is not fatal. You
+ * just give up KVM if PAGE_SIZE isn't supported there. Go fix
+ * your favourite nesting hypervisor.
+ *
+ * There is a small corner case where the hypervisor explicitly
+ * advertises a given granule size at Stage-2 (value 2) on some
+ * vCPUs, and uses the fallback to Stage-1 (value 0) for other
+ * vCPUs. Although this is not forbidden by the architecture, it
+ * indicates that the hypervisor is being silly (or buggy).
+ *
+ * We make no effort to cope with this and pretend that if these
+ * fields are inconsistent across vCPUs, then it isn't worth
+ * trying to bring KVM up.
+ */
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_EXACT, ID_AA64MMFR0_TGRAN4_2_SHIFT, 4, 1),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_EXACT, ID_AA64MMFR0_TGRAN64_2_SHIFT, 4, 1),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_EXACT, ID_AA64MMFR0_TGRAN16_2_SHIFT, 4, 1),
+ /*
+ * We already refuse to boot CPUs that don't support our configured
+ * page size, so we can only detect mismatches for a page size other
+ * than the one we're currently using. Unfortunately, SoCs like this
+ * exist in the wild so, even though we don't like it, we'll have to go
+ * along with it and treat them as non-strict.
+ */
+ S_ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64MMFR0_TGRAN4_SHIFT, 4, ID_AA64MMFR0_TGRAN4_NI),
+ S_ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64MMFR0_TGRAN64_SHIFT, 4, ID_AA64MMFR0_TGRAN64_NI),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64MMFR0_TGRAN16_SHIFT, 4, ID_AA64MMFR0_TGRAN16_NI),
+
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR0_BIGENDEL0_SHIFT, 4, 0),
+ /* Linux shouldn't care about secure memory */
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64MMFR0_SNSMEM_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR0_BIGENDEL_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR0_ASID_SHIFT, 4, 0),
+ /*
+ * Differing PARange is fine as long as all peripherals and memory are mapped
+ * within the minimum PARange of all CPUs
+ */
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64MMFR0_PARANGE_SHIFT, 4, 0),
+ ARM64_FTR_END,
+};
+
+static const struct arm64_ftr_bits ftr_id_aa64mmfr1[] = {
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR1_ETS_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR1_TWED_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR1_XNX_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_HIGHER_SAFE, ID_AA64MMFR1_SPECSEI_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR1_PAN_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR1_LOR_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR1_HPD_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR1_VHE_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR1_VMIDBITS_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR1_HADBS_SHIFT, 4, 0),
+ ARM64_FTR_END,
+};
+
+static const struct arm64_ftr_bits ftr_id_aa64mmfr2[] = {
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_E0PD_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_EVT_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_BBM_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_TTL_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_FWB_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_IDS_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_AT_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_ST_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_NV_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_CCIDX_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_LVA_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_IESB_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_LSM_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_UAO_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_CNP_SHIFT, 4, 0),
+ ARM64_FTR_END,
+};
+
+static const struct arm64_ftr_bits ftr_ctr[] = {
+ ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_EXACT, 31, 1, 1), /* RES1 */
+ ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, CTR_DIC_SHIFT, 1, 1),
+ ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, CTR_IDC_SHIFT, 1, 1),
+ ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_HIGHER_OR_ZERO_SAFE, CTR_CWG_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_HIGHER_OR_ZERO_SAFE, CTR_ERG_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, CTR_DMINLINE_SHIFT, 4, 1),
+ /*
+ * Linux can handle differing I-cache policies. Userspace JITs will
+ * make use of *minLine.
+ * If we have differing I-cache policies, report it as the weakest - VIPT.
+ */
+ ARM64_FTR_BITS(FTR_VISIBLE, FTR_NONSTRICT, FTR_EXACT, CTR_L1IP_SHIFT, 2, ICACHE_POLICY_VIPT), /* L1Ip */
+ ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, CTR_IMINLINE_SHIFT, 4, 0),
+ ARM64_FTR_END,
+};
+
+static const struct arm64_ftr_bits ftr_id_mmfr0[] = {
+ S_ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR0_INNERSHR_SHIFT, 4, 0xf),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR0_FCSE_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_MMFR0_AUXREG_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR0_TCM_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR0_SHARELVL_SHIFT, 4, 0),
+ S_ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR0_OUTERSHR_SHIFT, 4, 0xf),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR0_PMSA_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR0_VMSA_SHIFT, 4, 0),
+ ARM64_FTR_END,
+};
+
+static const struct arm64_ftr_bits ftr_id_aa64dfr0[] = {
+ S_ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64DFR0_DOUBLELOCK_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64DFR0_PMSVER_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64DFR0_CTX_CMPS_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64DFR0_WRPS_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64DFR0_BRPS_SHIFT, 4, 0),
+ /*
+ * We can instantiate multiple PMU instances with different levels
+ * of support.
+ */
+ S_ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_EXACT, ID_AA64DFR0_PMUVER_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_EXACT, ID_AA64DFR0_DEBUGVER_SHIFT, 4, 0x6),
+ ARM64_FTR_END,
+};
+
+static const struct arm64_ftr_bits ftr_mvfr2[] = {
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, MVFR2_FPMISC_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, MVFR2_SIMDMISC_SHIFT, 4, 0),
+ ARM64_FTR_END,
+};
+
+static const struct arm64_ftr_bits ftr_dczid[] = {
+ ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_EXACT, DCZID_DZP_SHIFT, 1, 1),
+ ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, DCZID_BS_SHIFT, 4, 0),
+ ARM64_FTR_END,
+};
+
+static const struct arm64_ftr_bits ftr_id_isar0[] = {
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR0_DIVIDE_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR0_DEBUG_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR0_COPROC_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR0_CMPBRANCH_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR0_BITFIELD_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR0_BITCOUNT_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR0_SWAP_SHIFT, 4, 0),
+ ARM64_FTR_END,
+};
+
+static const struct arm64_ftr_bits ftr_id_isar5[] = {
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR5_RDM_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR5_CRC32_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR5_SHA2_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR5_SHA1_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR5_AES_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR5_SEVL_SHIFT, 4, 0),
+ ARM64_FTR_END,
+};
+
+static const struct arm64_ftr_bits ftr_id_mmfr4[] = {
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR4_EVT_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR4_CCIDX_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR4_LSM_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR4_HPDS_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR4_CNP_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR4_XNX_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR4_AC2_SHIFT, 4, 0),
+
+ /*
+ * SpecSEI = 1 indicates that the PE might generate an SError on an
+ * external abort on speculative read. It is safe to assume that an
+ * SError might be generated than it will not be. Hence it has been
+ * classified as FTR_HIGHER_SAFE.
+ */
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_HIGHER_SAFE, ID_MMFR4_SPECSEI_SHIFT, 4, 0),
+ ARM64_FTR_END,
+};
+
+static const struct arm64_ftr_bits ftr_id_isar4[] = {
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR4_SWP_FRAC_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR4_PSR_M_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR4_SYNCH_PRIM_FRAC_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR4_BARRIER_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR4_SMC_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR4_WRITEBACK_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR4_WITHSHIFTS_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR4_UNPRIV_SHIFT, 4, 0),
+ ARM64_FTR_END,
+};
+
+static const struct arm64_ftr_bits ftr_id_mmfr5[] = {
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR5_ETS_SHIFT, 4, 0),
+ ARM64_FTR_END,
+};
+
+static const struct arm64_ftr_bits ftr_id_isar6[] = {
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR6_I8MM_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR6_BF16_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR6_SPECRES_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR6_SB_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR6_FHM_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR6_DP_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR6_JSCVT_SHIFT, 4, 0),
+ ARM64_FTR_END,
+};
+
+static const struct arm64_ftr_bits ftr_id_pfr0[] = {
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_PFR0_DIT_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_PFR0_CSV2_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_PFR0_STATE3_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_PFR0_STATE2_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_PFR0_STATE1_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_PFR0_STATE0_SHIFT, 4, 0),
+ ARM64_FTR_END,
+};
+
+static const struct arm64_ftr_bits ftr_id_pfr1[] = {
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_PFR1_GIC_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_PFR1_VIRT_FRAC_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_PFR1_SEC_FRAC_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_PFR1_GENTIMER_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_PFR1_VIRTUALIZATION_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_PFR1_MPROGMOD_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_PFR1_SECURITY_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_PFR1_PROGMOD_SHIFT, 4, 0),
+ ARM64_FTR_END,
+};
+
+static const struct arm64_ftr_bits ftr_id_pfr2[] = {
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_PFR2_SSBS_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_PFR2_CSV3_SHIFT, 4, 0),
+ ARM64_FTR_END,
+};
+
+static const struct arm64_ftr_bits ftr_id_dfr0[] = {
+ /* [31:28] TraceFilt */
+ S_ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_DFR0_PERFMON_SHIFT, 4, 0xf),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_DFR0_MPROFDBG_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_DFR0_MMAPTRC_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_DFR0_COPTRC_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_DFR0_MMAPDBG_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_DFR0_COPSDBG_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_DFR0_COPDBG_SHIFT, 4, 0),
+ ARM64_FTR_END,
+};
+
+static const struct arm64_ftr_bits ftr_id_dfr1[] = {
+ S_ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_DFR1_MTPMU_SHIFT, 4, 0),
+ ARM64_FTR_END,
+};
+
+static const struct arm64_ftr_bits ftr_zcr[] = {
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE,
+ ZCR_ELx_LEN_SHIFT, ZCR_ELx_LEN_SIZE, 0), /* LEN */
+ ARM64_FTR_END,
+};
+
+/*
+ * Common ftr bits for a 32bit register with all hidden, strict
+ * attributes, with 4bit feature fields and a default safe value of
+ * 0. Covers the following 32bit registers:
+ * id_isar[1-4], id_mmfr[1-3], id_pfr1, mvfr[0-1]
+ */
+static const struct arm64_ftr_bits ftr_generic_32bits[] = {
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, 28, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, 24, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, 20, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, 16, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, 12, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, 8, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, 4, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, 0, 4, 0),
+ ARM64_FTR_END,
+};
+
+static const struct arm64_ftr_bits ftr_raz[] = {
+ ARM64_FTR_END,
+};
+
+static s64 arm64_ftr_safe_value(const struct arm64_ftr_bits *ftrp, s64 new,
+ s64 cur)
+{
+ s64 ret = 0;
+
+ switch (ftrp->type) {
+ case FTR_EXACT:
+ ret = ftrp->safe_val;
+ break;
+ case FTR_LOWER_SAFE:
+ ret = min(new, cur);
+ break;
+ case FTR_HIGHER_OR_ZERO_SAFE:
+ if (!cur || !new)
+ break;
+ fallthrough;
+ case FTR_HIGHER_SAFE:
+ ret = max(new, cur);
+ break;
+ default:
+ BUG();
+ }
+
+ return ret;
+}
+
+/*
+ * End of imported linux structures and code
+ */
+
diff --git a/xen/include/asm-arm/arm64/cpufeature.h b/xen/include/asm-arm/arm64/cpufeature.h
new file mode 100644
index 0000000000..d9b9fa77cb
--- /dev/null
+++ b/xen/include/asm-arm/arm64/cpufeature.h
@@ -0,0 +1,104 @@
+#ifndef __ASM_ARM_ARM64_CPUFEATURES_H
+#define __ASM_ARM_ARM64_CPUFEATURES_H
+
+/*
+ * CPU feature register tracking
+ *
+ * The safe value of a CPUID feature field is dependent on the implications
+ * of the values assigned to it by the architecture. Based on the relationship
+ * between the values, the features are classified into 3 types - LOWER_SAFE,
+ * HIGHER_SAFE and EXACT.
+ *
+ * The lowest value of all the CPUs is chosen for LOWER_SAFE and highest
+ * for HIGHER_SAFE. It is expected that all CPUs have the same value for
+ * a field when EXACT is specified, failing which, the safe value specified
+ * in the table is chosen.
+ */
+
+enum ftr_type {
+ FTR_EXACT, /* Use a predefined safe value */
+ FTR_LOWER_SAFE, /* Smaller value is safe */
+ FTR_HIGHER_SAFE, /* Bigger value is safe */
+ FTR_HIGHER_OR_ZERO_SAFE, /* Bigger value is safe, but 0 is biggest */
+};
+
+#define FTR_STRICT true /* SANITY check strict matching required */
+#define FTR_NONSTRICT false /* SANITY check ignored */
+
+#define FTR_SIGNED true /* Value should be treated as signed */
+#define FTR_UNSIGNED false /* Value should be treated as unsigned */
+
+#define FTR_VISIBLE true /* Feature visible to the user space */
+#define FTR_HIDDEN false /* Feature is hidden from the user */
+
+#define FTR_VISIBLE_IF_IS_ENABLED(config) \
+ (IS_ENABLED(config) ? FTR_VISIBLE : FTR_HIDDEN)
+
+struct arm64_ftr_bits {
+ bool sign; /* Value is signed ? */
+ bool visible;
+ bool strict; /* CPU Sanity check: strict matching required ? */
+ enum ftr_type type;
+ u8 shift;
+ u8 width;
+ s64 safe_val; /* safe value for FTR_EXACT features */
+};
+
+static inline int __attribute_const__
+cpuid_feature_extract_signed_field_width(u64 features, int field, int width)
+{
+ return (s64)(features << (64 - width - field)) >> (64 - width);
+}
+
+static inline int __attribute_const__
+cpuid_feature_extract_signed_field(u64 features, int field)
+{
+ return cpuid_feature_extract_signed_field_width(features, field, 4);
+}
+
+static inline unsigned int __attribute_const__
+cpuid_feature_extract_unsigned_field_width(u64 features, int field, int width)
+{
+ return (u64)(features << (64 - width - field)) >> (64 - width);
+}
+
+static inline unsigned int __attribute_const__
+cpuid_feature_extract_unsigned_field(u64 features, int field)
+{
+ return cpuid_feature_extract_unsigned_field_width(features, field, 4);
+}
+
+static inline u64 arm64_ftr_mask(const struct arm64_ftr_bits *ftrp)
+{
+ return (u64)GENMASK(ftrp->shift + ftrp->width - 1, ftrp->shift);
+}
+
+static inline int __attribute_const__
+cpuid_feature_extract_field_width(u64 features, int field, int width, bool sign)
+{
+ return (sign) ?
+ cpuid_feature_extract_signed_field_width(features, field, width) :
+ cpuid_feature_extract_unsigned_field_width(features, field, width);
+}
+
+static inline int __attribute_const__
+cpuid_feature_extract_field(u64 features, int field, bool sign)
+{
+ return cpuid_feature_extract_field_width(features, field, 4, sign);
+}
+
+static inline s64 arm64_ftr_value(const struct arm64_ftr_bits *ftrp, u64 val)
+{
+ return (s64)cpuid_feature_extract_field_width(val, ftrp->shift, ftrp->width, ftrp->sign);
+}
+
+#endif /* _ASM_ARM_ARM64_CPUFEATURES_H */
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
--
2.17.1
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [PATCH v3 3/7] xen/arm: Rename cpu_boot_data to system_cpuinfo
2021-08-25 13:18 [PATCH v3 0/7] xen/arm: Sanitize cpuinfo Bertrand Marquis
2021-08-25 13:18 ` [PATCH v3 1/7] xen/arm: Import ID registers definitions from Linux Bertrand Marquis
2021-08-25 13:18 ` [PATCH v3 2/7] xen/arm: Import ID features sanitize from linux Bertrand Marquis
@ 2021-08-25 13:18 ` Bertrand Marquis
2021-09-03 22:48 ` Stefano Stabellini
2021-08-25 13:18 ` [PATCH v3 4/7] xen/arm: Sanitize cpuinfo ID registers fields Bertrand Marquis
` (3 subsequent siblings)
6 siblings, 1 reply; 26+ messages in thread
From: Bertrand Marquis @ 2021-08-25 13:18 UTC (permalink / raw)
To: xen-devel; +Cc: Stefano Stabellini, Julien Grall, Volodymyr Babchuk
As we will sanitize the content of boot_cpu_data it will not really
contain the boot cpu information but the system sanitize information.
Rename the structure to system_cpuinfo so the user is informed that this
is the system wide available feature and not anymore the features of the
boot cpu.
The original boot cpu data is still available in cpu_data.
Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
---
Changes in v3: none
Changes in v2:
- patch introduced in v2
---
xen/arch/arm/cpufeature.c | 8 ++------
xen/arch/arm/setup.c | 34 ++++++++++++++++++--------------
xen/arch/arm/smpboot.c | 6 +++---
xen/include/asm-arm/cpufeature.h | 6 +++---
4 files changed, 27 insertions(+), 27 deletions(-)
diff --git a/xen/arch/arm/cpufeature.c b/xen/arch/arm/cpufeature.c
index 1d88783809..f600a611bd 100644
--- a/xen/arch/arm/cpufeature.c
+++ b/xen/arch/arm/cpufeature.c
@@ -169,12 +169,8 @@ void identify_cpu(struct cpuinfo_arm *c)
*/
static int __init create_guest_cpuinfo(void)
{
- /*
- * TODO: The code is currently using only the features detected on the boot
- * core. In the long term we should try to compute values containing only
- * features supported by all cores.
- */
- guest_cpuinfo = boot_cpu_data;
+ /* Use the sanitized cpuinfo as initial guest cpuinfo */
+ guest_cpuinfo = system_cpuinfo;
#ifdef CONFIG_ARM_64
/* Hide MPAM support as xen does not support it */
diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
index 63a908e325..3798c5ade0 100644
--- a/xen/arch/arm/setup.c
+++ b/xen/arch/arm/setup.c
@@ -56,7 +56,11 @@
struct bootinfo __initdata bootinfo;
-struct cpuinfo_arm __read_mostly boot_cpu_data;
+/*
+ * Sanitized version of cpuinfo containing only features available on all
+ * cores (only on arm64 as there is no sanitization support on arm32).
+ */
+struct cpuinfo_arm __read_mostly system_cpuinfo;
#ifdef CONFIG_ACPI
bool __read_mostly acpi_disabled;
@@ -100,7 +104,7 @@ static const char * __initdata processor_implementers[] = {
static void __init processor_id(void)
{
const char *implementer = "Unknown";
- struct cpuinfo_arm *c = &boot_cpu_data;
+ struct cpuinfo_arm *c = &system_cpuinfo;
identify_cpu(c);
current_cpu_data = *c;
@@ -120,7 +124,7 @@ static void __init processor_id(void)
#if defined(CONFIG_ARM_64)
printk("64-bit Execution:\n");
printk(" Processor Features: %016"PRIx64" %016"PRIx64"\n",
- boot_cpu_data.pfr64.bits[0], boot_cpu_data.pfr64.bits[1]);
+ system_cpuinfo.pfr64.bits[0], system_cpuinfo.pfr64.bits[1]);
printk(" Exception Levels: EL3:%s EL2:%s EL1:%s EL0:%s\n",
cpu_has_el3_32 ? "64+32" : cpu_has_el3_64 ? "64" : "No",
cpu_has_el2_32 ? "64+32" : cpu_has_el2_64 ? "64" : "No",
@@ -144,13 +148,13 @@ static void __init processor_id(void)
boot_cpu_feature64(simd));
printk(" Debug Features: %016"PRIx64" %016"PRIx64"\n",
- boot_cpu_data.dbg64.bits[0], boot_cpu_data.dbg64.bits[1]);
+ system_cpuinfo.dbg64.bits[0], system_cpuinfo.dbg64.bits[1]);
printk(" Auxiliary Features: %016"PRIx64" %016"PRIx64"\n",
- boot_cpu_data.aux64.bits[0], boot_cpu_data.aux64.bits[1]);
+ system_cpuinfo.aux64.bits[0], system_cpuinfo.aux64.bits[1]);
printk(" Memory Model Features: %016"PRIx64" %016"PRIx64"\n",
- boot_cpu_data.mm64.bits[0], boot_cpu_data.mm64.bits[1]);
+ system_cpuinfo.mm64.bits[0], system_cpuinfo.mm64.bits[1]);
printk(" ISA Features: %016"PRIx64" %016"PRIx64"\n",
- boot_cpu_data.isa64.bits[0], boot_cpu_data.isa64.bits[1]);
+ system_cpuinfo.isa64.bits[0], system_cpuinfo.isa64.bits[1]);
#endif
/*
@@ -161,7 +165,7 @@ static void __init processor_id(void)
{
printk("32-bit Execution:\n");
printk(" Processor Features: %"PRIregister":%"PRIregister"\n",
- boot_cpu_data.pfr32.bits[0], boot_cpu_data.pfr32.bits[1]);
+ system_cpuinfo.pfr32.bits[0], system_cpuinfo.pfr32.bits[1]);
printk(" Instruction Sets:%s%s%s%s%s%s\n",
cpu_has_aarch32 ? " AArch32" : "",
cpu_has_arm ? " A32" : "",
@@ -174,18 +178,18 @@ static void __init processor_id(void)
cpu_has_security ? " Security" : "");
printk(" Debug Features: %"PRIregister"\n",
- boot_cpu_data.dbg32.bits[0]);
+ system_cpuinfo.dbg32.bits[0]);
printk(" Auxiliary Features: %"PRIregister"\n",
- boot_cpu_data.aux32.bits[0]);
+ system_cpuinfo.aux32.bits[0]);
printk(" Memory Model Features: %"PRIregister" %"PRIregister"\n"
" %"PRIregister" %"PRIregister"\n",
- boot_cpu_data.mm32.bits[0], boot_cpu_data.mm32.bits[1],
- boot_cpu_data.mm32.bits[2], boot_cpu_data.mm32.bits[3]);
+ system_cpuinfo.mm32.bits[0], system_cpuinfo.mm32.bits[1],
+ system_cpuinfo.mm32.bits[2], system_cpuinfo.mm32.bits[3]);
printk(" ISA Features: %"PRIregister" %"PRIregister" %"PRIregister"\n"
" %"PRIregister" %"PRIregister" %"PRIregister"\n",
- boot_cpu_data.isa32.bits[0], boot_cpu_data.isa32.bits[1],
- boot_cpu_data.isa32.bits[2], boot_cpu_data.isa32.bits[3],
- boot_cpu_data.isa32.bits[4], boot_cpu_data.isa32.bits[5]);
+ system_cpuinfo.isa32.bits[0], system_cpuinfo.isa32.bits[1],
+ system_cpuinfo.isa32.bits[2], system_cpuinfo.isa32.bits[3],
+ system_cpuinfo.isa32.bits[4], system_cpuinfo.isa32.bits[5]);
}
else
{
diff --git a/xen/arch/arm/smpboot.c b/xen/arch/arm/smpboot.c
index a1ee3146ef..c9f2827d56 100644
--- a/xen/arch/arm/smpboot.c
+++ b/xen/arch/arm/smpboot.c
@@ -124,7 +124,7 @@ static void __init dt_smp_init_cpus(void)
bool bootcpu_valid = false;
int rc;
- mpidr = boot_cpu_data.mpidr.bits & MPIDR_HWID_MASK;
+ mpidr = system_cpuinfo.mpidr.bits & MPIDR_HWID_MASK;
if ( !cpus )
{
@@ -319,13 +319,13 @@ void start_secondary(void)
* now.
*/
if ( !opt_hmp_unsafe &&
- current_cpu_data.midr.bits != boot_cpu_data.midr.bits )
+ current_cpu_data.midr.bits != system_cpuinfo.midr.bits )
{
printk(XENLOG_ERR
"CPU%u MIDR (0x%"PRIregister") does not match boot CPU MIDR (0x%"PRIregister"),\n"
XENLOG_ERR "disable cpu (see big.LITTLE.txt under docs/).\n",
smp_processor_id(), current_cpu_data.midr.bits,
- boot_cpu_data.midr.bits);
+ system_cpuinfo.midr.bits);
stop_cpu();
}
diff --git a/xen/include/asm-arm/cpufeature.h b/xen/include/asm-arm/cpufeature.h
index ba48db3eac..8f2b8e7830 100644
--- a/xen/include/asm-arm/cpufeature.h
+++ b/xen/include/asm-arm/cpufeature.h
@@ -3,7 +3,7 @@
#ifdef CONFIG_ARM_64
#define cpu_feature64(c, feat) ((c)->pfr64.feat)
-#define boot_cpu_feature64(feat) (boot_cpu_data.pfr64.feat)
+#define boot_cpu_feature64(feat) (system_cpuinfo.pfr64.feat)
#define cpu_feature64_has_el0_32(c) (cpu_feature64(c, el0) == 2)
@@ -21,7 +21,7 @@
#endif
#define cpu_feature32(c, feat) ((c)->pfr32.feat)
-#define boot_cpu_feature32(feat) (boot_cpu_data.pfr32.feat)
+#define boot_cpu_feature32(feat) (system_cpuinfo.pfr32.feat)
#define cpu_has_arm (boot_cpu_feature32(arm) == 1)
#define cpu_has_thumb (boot_cpu_feature32(thumb) >= 1)
@@ -326,7 +326,7 @@ struct cpuinfo_arm {
} mvfr;
};
-extern struct cpuinfo_arm boot_cpu_data;
+extern struct cpuinfo_arm system_cpuinfo;
extern void identify_cpu(struct cpuinfo_arm *);
--
2.17.1
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [PATCH v3 4/7] xen/arm: Sanitize cpuinfo ID registers fields
2021-08-25 13:18 [PATCH v3 0/7] xen/arm: Sanitize cpuinfo Bertrand Marquis
` (2 preceding siblings ...)
2021-08-25 13:18 ` [PATCH v3 3/7] xen/arm: Rename cpu_boot_data to system_cpuinfo Bertrand Marquis
@ 2021-08-25 13:18 ` Bertrand Marquis
2021-09-03 22:48 ` Stefano Stabellini
2021-08-25 13:18 ` [PATCH v3 5/7] xen/arm: Use sanitize values for p2m Bertrand Marquis
` (2 subsequent siblings)
6 siblings, 1 reply; 26+ messages in thread
From: Bertrand Marquis @ 2021-08-25 13:18 UTC (permalink / raw)
To: xen-devel
Cc: Stefano Stabellini, Julien Grall, Volodymyr Babchuk,
Andrew Cooper, George Dunlap, Ian Jackson, Jan Beulich, Wei Liu
Define a sanitize_cpu function to be called on secondary cores to
sanitize the system cpuinfo structure.
The safest value is taken when possible and the system is marked tainted
if we encounter values which are incompatible with each other.
Call the update_system_features function on all secondary cores that are
kept running and taint the system if different midr are found between
cores but hmp-unsafe=true was passed on Xen command line.
This is only supported on arm64 so update_system_features is an empty
static inline on arm32.
The patch is adding a new TAINT_CPU_OUT_OF_SPEC to warn the user if
Xen is running on a system with features differences between cores which
are not supported.
The patch is disabling CTR_EL0, DCZID_EL0 and ZCRusing #if 0 with a TODO
as this patch is not handling sanitization of those registers.
CTR_EL0/DCZID will be handled in a future patch to properly handle
different cache attributes when possible.
ZCR should be sanitize once we add support for SVE in Xen.
Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
---
Changes in v3:
- in case of different midr but hmp-unsafe passed on the command line,
enable all cores anyway but taint Xen with CPU_OUT_OF_SPEC.
- use current core info to sanitize cpu only if we keep it on
Changes in v2:
- add compilation of cpufeature.c in this patch instead of previous one
- remove functions reused from linux code and moved to header
- rename sanitize_cpu to update_system_features
- change to Linux coding style
- remove dev comments
- surround currently not used Linux structures with #if 0 and adapt the
commit message
- add missing aa64dfr1 register
- add TODO for CTR, DCZID and ZCR
- add CPU_OUT_OF_SPEC support to print_taint
- use system_cpuinfo instead of boot_cpu_data
---
xen/arch/arm/arm64/Makefile | 1 +
xen/arch/arm/arm64/cpufeature.c | 121 +++++++++++++++++++++++++++++++
xen/arch/arm/smpboot.c | 34 +++++++--
xen/common/kernel.c | 6 +-
xen/include/asm-arm/cpufeature.h | 9 +++
xen/include/xen/lib.h | 1 +
6 files changed, 162 insertions(+), 10 deletions(-)
diff --git a/xen/arch/arm/arm64/Makefile b/xen/arch/arm/arm64/Makefile
index 40642ff574..701d66883d 100644
--- a/xen/arch/arm/arm64/Makefile
+++ b/xen/arch/arm/arm64/Makefile
@@ -1,6 +1,7 @@
obj-y += lib/
obj-y += cache.o
+obj-y += cpufeature.o
obj-$(CONFIG_HARDEN_BRANCH_PREDICTOR) += bpi.o
obj-$(CONFIG_EARLY_PRINTK) += debug.o
obj-y += domctl.o
diff --git a/xen/arch/arm/arm64/cpufeature.c b/xen/arch/arm/arm64/cpufeature.c
index 5777e33e5c..61f629ebaa 100644
--- a/xen/arch/arm/arm64/cpufeature.c
+++ b/xen/arch/arm/arm64/cpufeature.c
@@ -275,6 +275,9 @@ static const struct arm64_ftr_bits ftr_id_aa64mmfr2[] = {
ARM64_FTR_END,
};
+#if 0
+/* TODO: use this to sanitize the cache line size among cores */
+
static const struct arm64_ftr_bits ftr_ctr[] = {
ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_EXACT, 31, 1, 1), /* RES1 */
ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, CTR_DIC_SHIFT, 1, 1),
@@ -291,6 +294,7 @@ static const struct arm64_ftr_bits ftr_ctr[] = {
ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, CTR_IMINLINE_SHIFT, 4, 0),
ARM64_FTR_END,
};
+#endif
static const struct arm64_ftr_bits ftr_id_mmfr0[] = {
S_ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR0_INNERSHR_SHIFT, 4, 0xf),
@@ -325,11 +329,14 @@ static const struct arm64_ftr_bits ftr_mvfr2[] = {
ARM64_FTR_END,
};
+#if 0
+/* TODO: handle this when sanitizing cache related registers */
static const struct arm64_ftr_bits ftr_dczid[] = {
ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_EXACT, DCZID_DZP_SHIFT, 1, 1),
ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, DCZID_BS_SHIFT, 4, 0),
ARM64_FTR_END,
};
+#endif
static const struct arm64_ftr_bits ftr_id_isar0[] = {
ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR0_DIVIDE_SHIFT, 4, 0),
@@ -444,11 +451,15 @@ static const struct arm64_ftr_bits ftr_id_dfr1[] = {
ARM64_FTR_END,
};
+#if 0
+/* TODO: use this to sanitize SVE once we support it */
+
static const struct arm64_ftr_bits ftr_zcr[] = {
ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE,
ZCR_ELx_LEN_SHIFT, ZCR_ELx_LEN_SIZE, 0), /* LEN */
ARM64_FTR_END,
};
+#endif
/*
* Common ftr bits for a 32bit register with all hidden, strict
@@ -502,3 +513,113 @@ static s64 arm64_ftr_safe_value(const struct arm64_ftr_bits *ftrp, s64 new,
* End of imported linux structures and code
*/
+static void sanitize_reg(u64 *cur_reg, u64 new_reg, const char *reg_name,
+ const struct arm64_ftr_bits *ftrp)
+{
+ int taint = 0;
+ u64 old_reg = *cur_reg;
+
+ for (;ftrp->width != 0;ftrp++)
+ {
+ u64 mask;
+ s64 cur_field = arm64_ftr_value(ftrp, *cur_reg);
+ s64 new_field = arm64_ftr_value(ftrp, new_reg);
+
+ if (cur_field == new_field)
+ continue;
+
+ if (ftrp->strict)
+ taint = 1;
+
+ mask = arm64_ftr_mask(ftrp);
+
+ *cur_reg &= ~mask;
+ *cur_reg |= (arm64_ftr_safe_value(ftrp, new_field, cur_field)
+ << ftrp->shift) & mask;
+ }
+
+ if (old_reg != new_reg)
+ printk(XENLOG_DEBUG "SANITY DIF: %s 0x%"PRIx64" -> 0x%"PRIx64"\n",
+ reg_name, old_reg, new_reg);
+ if (old_reg != *cur_reg)
+ printk(XENLOG_DEBUG "SANITY FIX: %s 0x%"PRIx64" -> 0x%"PRIx64"\n",
+ reg_name, old_reg, *cur_reg);
+
+ if (taint)
+ {
+ printk(XENLOG_WARNING "SANITY CHECK: Unexpected variation in %s.\n",
+ reg_name);
+ add_taint(TAINT_CPU_OUT_OF_SPEC);
+ }
+}
+
+
+/*
+ * This function should be called on secondary cores to sanitize the boot cpu
+ * cpuinfo.
+ */
+void update_system_features(const struct cpuinfo_arm *new)
+{
+
+#define SANITIZE_REG(field, num, reg) \
+ sanitize_reg(&system_cpuinfo.field.bits[num], new->field.bits[num], \
+ #reg, ftr_##reg)
+
+#define SANITIZE_ID_REG(field, num, reg) \
+ sanitize_reg(&system_cpuinfo.field.bits[num], new->field.bits[num], \
+ #reg, ftr_id_##reg)
+
+#define SANITIZE_RAZ_REG(field, num, reg) \
+ sanitize_reg(&system_cpuinfo.field.bits[num], new->field.bits[num], \
+ #reg, ftr_raz)
+
+#define SANITIZE_GENERIC_REG(field, num, reg) \
+ sanitize_reg(&system_cpuinfo.field.bits[num], new->field.bits[num], \
+ #reg, ftr_generic_32bits)
+
+ SANITIZE_ID_REG(pfr64, 0, aa64pfr0);
+ SANITIZE_ID_REG(pfr64, 1, aa64pfr1);
+
+ SANITIZE_ID_REG(dbg64, 0, aa64dfr0);
+ SANITIZE_RAZ_REG(dbg64, 1, aa64dfr1);
+
+ SANITIZE_ID_REG(mm64, 0, aa64mmfr0);
+ SANITIZE_ID_REG(mm64, 1, aa64mmfr1);
+ SANITIZE_ID_REG(mm64, 2, aa64mmfr2);
+
+ SANITIZE_ID_REG(isa64, 0, aa64isar0);
+ SANITIZE_ID_REG(isa64, 1, aa64isar1);
+
+ SANITIZE_ID_REG(zfr64, 0, aa64zfr0);
+
+ if ( cpu_feature64_has_el0_32(&system_cpuinfo) )
+ {
+ SANITIZE_ID_REG(pfr32, 0, pfr0);
+ SANITIZE_ID_REG(pfr32, 1, pfr1);
+ SANITIZE_ID_REG(pfr32, 2, pfr2);
+
+ SANITIZE_ID_REG(dbg32, 0, dfr0);
+ SANITIZE_ID_REG(dbg32, 1, dfr1);
+
+ SANITIZE_ID_REG(mm32, 0, mmfr0);
+ SANITIZE_GENERIC_REG(mm32, 1, mmfr1);
+ SANITIZE_GENERIC_REG(mm32, 2, mmfr2);
+ SANITIZE_GENERIC_REG(mm32, 3, mmfr3);
+ SANITIZE_ID_REG(mm32, 4, mmfr4);
+ SANITIZE_ID_REG(mm32, 5, mmfr5);
+
+ SANITIZE_ID_REG(isa32, 0, isar0);
+ SANITIZE_GENERIC_REG(isa32, 1, isar1);
+ SANITIZE_GENERIC_REG(isa32, 2, isar2);
+ SANITIZE_GENERIC_REG(isa32, 3, isar3);
+ SANITIZE_ID_REG(isa32, 4, isar4);
+ SANITIZE_ID_REG(isa32, 5, isar5);
+ SANITIZE_ID_REG(isa32, 6, isar6);
+
+ SANITIZE_GENERIC_REG(mvfr, 0, mvfr0);
+ SANITIZE_GENERIC_REG(mvfr, 1, mvfr1);
+#ifndef MVFR2_MAYBE_UNDEFINED
+ SANITIZE_REG(mvfr, 2, mvfr2);
+#endif
+ }
+}
diff --git a/xen/arch/arm/smpboot.c b/xen/arch/arm/smpboot.c
index c9f2827d56..60c0e82fc5 100644
--- a/xen/arch/arm/smpboot.c
+++ b/xen/arch/arm/smpboot.c
@@ -318,15 +318,26 @@ void start_secondary(void)
* is manually specified for all domains). Better to park them for
* now.
*/
- if ( !opt_hmp_unsafe &&
- current_cpu_data.midr.bits != system_cpuinfo.midr.bits )
+ if ( current_cpu_data.midr.bits != system_cpuinfo.midr.bits )
{
- printk(XENLOG_ERR
- "CPU%u MIDR (0x%"PRIregister") does not match boot CPU MIDR (0x%"PRIregister"),\n"
- XENLOG_ERR "disable cpu (see big.LITTLE.txt under docs/).\n",
- smp_processor_id(), current_cpu_data.midr.bits,
- system_cpuinfo.midr.bits);
- stop_cpu();
+ if ( !opt_hmp_unsafe )
+ {
+ printk(XENLOG_ERR
+ "CPU%u MIDR (0x%"PRIregister") does not match boot CPU MIDR (0x%"PRIregister"),\n"
+ XENLOG_ERR "disable cpu (see big.LITTLE.txt under docs/).\n",
+ smp_processor_id(), current_cpu_data.midr.bits,
+ system_cpuinfo.midr.bits);
+ stop_cpu();
+ }
+ else
+ {
+ printk(XENLOG_ERR
+ "CPU%u MIDR (0x%"PRIregister") does not match boot CPU MIDR (0x%"PRIregister"),\n"
+ XENLOG_ERR "hmp-unsafe turned on so tainting Xen and keep core on!!\n",
+ smp_processor_id(), current_cpu_data.midr.bits,
+ system_cpuinfo.midr.bits);
+ add_taint(TAINT_CPU_OUT_OF_SPEC);
+ }
}
if ( dcache_line_bytes != read_dcache_line_bytes() )
@@ -337,6 +348,13 @@ void start_secondary(void)
stop_cpu();
}
+ /*
+ * system features must be updated only if we do not stop the core or
+ * we might disable features due to a non used core (for example when
+ * booting on big cores on a big.LITTLE system with hmp_unsafe)
+ */
+ update_system_features(¤t_cpu_data);
+
mmu_init_secondary_cpu();
gic_init_secondary_cpu();
diff --git a/xen/common/kernel.c b/xen/common/kernel.c
index d77756a81e..e119e5401f 100644
--- a/xen/common/kernel.c
+++ b/xen/common/kernel.c
@@ -327,6 +327,7 @@ unsigned int tainted;
* 'H' - HVM forced emulation prefix is permitted.
* 'M' - Machine had a machine check experience.
* 'U' - Platform is unsecure (usually due to an errata on the platform).
+ * 'S' - Out of spec CPU (One core has a feature incompatible with others).
*
* The string is overwritten by the next call to print_taint().
*/
@@ -334,12 +335,13 @@ char *print_tainted(char *str)
{
if ( tainted )
{
- snprintf(str, TAINT_STRING_MAX_LEN, "Tainted: %c%c%c%c%c",
+ snprintf(str, TAINT_STRING_MAX_LEN, "Tainted: %c%c%c%c%c%c",
tainted & TAINT_MACHINE_UNSECURE ? 'U' : ' ',
tainted & TAINT_MACHINE_CHECK ? 'M' : ' ',
tainted & TAINT_SYNC_CONSOLE ? 'C' : ' ',
tainted & TAINT_ERROR_INJECT ? 'E' : ' ',
- tainted & TAINT_HVM_FEP ? 'H' : ' ');
+ tainted & TAINT_HVM_FEP ? 'H' : ' ',
+ tainted & TAINT_CPU_OUT_OF_SPEC ? 'S' : ' ');
}
else
{
diff --git a/xen/include/asm-arm/cpufeature.h b/xen/include/asm-arm/cpufeature.h
index 8f2b8e7830..52cb3133e0 100644
--- a/xen/include/asm-arm/cpufeature.h
+++ b/xen/include/asm-arm/cpufeature.h
@@ -330,6 +330,15 @@ extern struct cpuinfo_arm system_cpuinfo;
extern void identify_cpu(struct cpuinfo_arm *);
+#ifdef CONFIG_ARM_64
+extern void update_system_features(const struct cpuinfo_arm *);
+#else
+static inline void update_system_features(const struct cpuinfo_arm *cpuinfo)
+{
+ /* Not supported on arm32 */
+}
+#endif
+
extern struct cpuinfo_arm cpu_data[];
#define current_cpu_data cpu_data[smp_processor_id()]
diff --git a/xen/include/xen/lib.h b/xen/include/xen/lib.h
index 1198c7c0b2..c6987973bf 100644
--- a/xen/include/xen/lib.h
+++ b/xen/include/xen/lib.h
@@ -192,6 +192,7 @@ uint64_t muldiv64(uint64_t a, uint32_t b, uint32_t c);
#define TAINT_ERROR_INJECT (1u << 2)
#define TAINT_HVM_FEP (1u << 3)
#define TAINT_MACHINE_UNSECURE (1u << 4)
+#define TAINT_CPU_OUT_OF_SPEC (1u << 5)
extern unsigned int tainted;
#define TAINT_STRING_MAX_LEN 20
extern char *print_tainted(char *str);
--
2.17.1
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [PATCH v3 5/7] xen/arm: Use sanitize values for p2m
2021-08-25 13:18 [PATCH v3 0/7] xen/arm: Sanitize cpuinfo Bertrand Marquis
` (3 preceding siblings ...)
2021-08-25 13:18 ` [PATCH v3 4/7] xen/arm: Sanitize cpuinfo ID registers fields Bertrand Marquis
@ 2021-08-25 13:18 ` Bertrand Marquis
2021-09-03 22:48 ` Stefano Stabellini
2021-08-25 13:18 ` [PATCH v3 6/7] xen/arm: Taint Xen on incompatible DCZID values Bertrand Marquis
2021-08-25 13:18 ` [PATCH v3 7/7] xen/arm: Sanitize CTR_EL0 and emulate it if needed Bertrand Marquis
6 siblings, 1 reply; 26+ messages in thread
From: Bertrand Marquis @ 2021-08-25 13:18 UTC (permalink / raw)
To: xen-devel; +Cc: Stefano Stabellini, Julien Grall, Volodymyr Babchuk
Replace the code in p2m trying to find a sane value for the VMID size
supported and the PAR to use. We are now using the boot cpuinfo as the
values there are sanitized during boot and the value for those
parameters is now the safest possible value on the system.
Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
---
Changes in v3: drop arm32 mention in commmit message
Changes in v2:
- use system_cpuinfo
---
xen/arch/arm/p2m.c | 30 ++++++++++--------------------
1 file changed, 10 insertions(+), 20 deletions(-)
diff --git a/xen/arch/arm/p2m.c b/xen/arch/arm/p2m.c
index eff9a105e7..41b6430c30 100644
--- a/xen/arch/arm/p2m.c
+++ b/xen/arch/arm/p2m.c
@@ -2045,31 +2045,21 @@ void __init setup_virt_paging(void)
[7] = { 0 } /* Invalid */
};
- unsigned int i, cpu;
+ unsigned int i;
unsigned int pa_range = 0x10; /* Larger than any possible value */
- bool vmid_8_bit = false;
-
- for_each_online_cpu ( cpu )
- {
- const struct cpuinfo_arm *info = &cpu_data[cpu];
- /*
- * Restrict "p2m_ipa_bits" if needed. As P2M table is always configured
- * with IPA bits == PA bits, compare against "pabits".
- */
- if ( pa_range_info[info->mm64.pa_range].pabits < p2m_ipa_bits )
- p2m_ipa_bits = pa_range_info[info->mm64.pa_range].pabits;
-
- /* Set a flag if the current cpu does not support 16 bit VMIDs. */
- if ( info->mm64.vmid_bits != MM64_VMID_16_BITS_SUPPORT )
- vmid_8_bit = true;
- }
+ /*
+ * Restrict "p2m_ipa_bits" if needed. As P2M table is always configured
+ * with IPA bits == PA bits, compare against "pabits".
+ */
+ if ( pa_range_info[system_cpuinfo.mm64.pa_range].pabits < p2m_ipa_bits )
+ p2m_ipa_bits = pa_range_info[system_cpuinfo.mm64.pa_range].pabits;
/*
- * If the flag is not set then it means all CPUs support 16-bit
- * VMIDs.
+ * cpu info sanitization made sure we support 16bits VMID only if all
+ * cores are supporting it.
*/
- if ( !vmid_8_bit )
+ if ( system_cpuinfo.mm64.vmid_bits == MM64_VMID_16_BITS_SUPPORT )
max_vmid = MAX_VMID_16_BIT;
/* Choose suitable "pa_range" according to the resulted "p2m_ipa_bits". */
--
2.17.1
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [PATCH v3 6/7] xen/arm: Taint Xen on incompatible DCZID values
2021-08-25 13:18 [PATCH v3 0/7] xen/arm: Sanitize cpuinfo Bertrand Marquis
` (4 preceding siblings ...)
2021-08-25 13:18 ` [PATCH v3 5/7] xen/arm: Use sanitize values for p2m Bertrand Marquis
@ 2021-08-25 13:18 ` Bertrand Marquis
2021-09-03 22:49 ` Stefano Stabellini
2021-08-25 13:18 ` [PATCH v3 7/7] xen/arm: Sanitize CTR_EL0 and emulate it if needed Bertrand Marquis
6 siblings, 1 reply; 26+ messages in thread
From: Bertrand Marquis @ 2021-08-25 13:18 UTC (permalink / raw)
To: xen-devel; +Cc: Stefano Stabellini, Julien Grall, Volodymyr Babchuk
Use arm64 cpu feature sanitization to TAIN Xen if different DCZID values
are found (ftr_dczid is using only STRICT method).
In this case actual memory being cleaned by DC ZVA operations would be
different depending on the cores which could make a guest zeroing too
much or too little memory if it is merged between CPUs.
We could, on processor supporting it, trap access to DCZID_EL0 register
using HFGRTR_EL2 register but this would not solve the case where a
process is being migrated during a copy or if it cached the value of the
register.
Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
---
Change in v3: none
Change in v2: Patch introduced in v2
---
xen/arch/arm/arm64/cpufeature.c | 14 +++++++++++---
xen/arch/arm/cpufeature.c | 2 ++
xen/include/asm-arm/cpufeature.h | 8 ++++++++
3 files changed, 21 insertions(+), 3 deletions(-)
diff --git a/xen/arch/arm/arm64/cpufeature.c b/xen/arch/arm/arm64/cpufeature.c
index 61f629ebaa..b1936ef1d6 100644
--- a/xen/arch/arm/arm64/cpufeature.c
+++ b/xen/arch/arm/arm64/cpufeature.c
@@ -329,14 +329,11 @@ static const struct arm64_ftr_bits ftr_mvfr2[] = {
ARM64_FTR_END,
};
-#if 0
-/* TODO: handle this when sanitizing cache related registers */
static const struct arm64_ftr_bits ftr_dczid[] = {
ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_EXACT, DCZID_DZP_SHIFT, 1, 1),
ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, DCZID_BS_SHIFT, 4, 0),
ARM64_FTR_END,
};
-#endif
static const struct arm64_ftr_bits ftr_id_isar0[] = {
ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR0_DIVIDE_SHIFT, 4, 0),
@@ -592,6 +589,17 @@ void update_system_features(const struct cpuinfo_arm *new)
SANITIZE_ID_REG(zfr64, 0, aa64zfr0);
+ /*
+ * Comment from Linux:
+ * Userspace may perform DC ZVA instructions. Mismatched block sizes
+ * could result in too much or too little memory being zeroed if a
+ * process is preempted and migrated between CPUs.
+ *
+ * ftr_dczid is using STRICT comparison so we will taint Xen if different
+ * values are found.
+ */
+ SANITIZE_REG(dczid, 0, dczid);
+
if ( cpu_feature64_has_el0_32(&system_cpuinfo) )
{
SANITIZE_ID_REG(pfr32, 0, pfr0);
diff --git a/xen/arch/arm/cpufeature.c b/xen/arch/arm/cpufeature.c
index f600a611bd..113f20f601 100644
--- a/xen/arch/arm/cpufeature.c
+++ b/xen/arch/arm/cpufeature.c
@@ -125,6 +125,8 @@ void identify_cpu(struct cpuinfo_arm *c)
c->zfr64.bits[0] = READ_SYSREG(ID_AA64ZFR0_EL1);
+ c->dczid.bits[0] = READ_SYSREG(DCZID_EL0);
+
aarch32_el0 = cpu_feature64_has_el0_32(c);
#endif
diff --git a/xen/include/asm-arm/cpufeature.h b/xen/include/asm-arm/cpufeature.h
index 52cb3133e0..5219fd3bab 100644
--- a/xen/include/asm-arm/cpufeature.h
+++ b/xen/include/asm-arm/cpufeature.h
@@ -259,6 +259,14 @@ struct cpuinfo_arm {
register_t bits[1];
} zfr64;
+ /*
+ * DCZID is only used to check for incoherent values between cores
+ * and taint Xen in this case
+ */
+ struct {
+ register_t bits[1];
+ } dczid;
+
#endif
/*
--
2.17.1
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [PATCH v3 7/7] xen/arm: Sanitize CTR_EL0 and emulate it if needed
2021-08-25 13:18 [PATCH v3 0/7] xen/arm: Sanitize cpuinfo Bertrand Marquis
` (5 preceding siblings ...)
2021-08-25 13:18 ` [PATCH v3 6/7] xen/arm: Taint Xen on incompatible DCZID values Bertrand Marquis
@ 2021-08-25 13:18 ` Bertrand Marquis
2021-08-27 15:05 ` Julien Grall
6 siblings, 1 reply; 26+ messages in thread
From: Bertrand Marquis @ 2021-08-25 13:18 UTC (permalink / raw)
To: xen-devel; +Cc: Stefano Stabellini, Julien Grall, Volodymyr Babchuk
Sanitize CTR_EL0 value between cores.
In most cases different values will taint Xen but if different
i-cache policies are found, we choose the one which will be compatible
between all cores in terms of invalidation/data cache flushing strategy.
In this case we need to activate the TID2 bit in HCR to emulate the
TCR_EL0 register for guests. This patch is not activating TID2 bit all
the time to limit the overhead when possible.
When TID2 is activate we also need to emulate the CCSIDR, CSSELR and
CLIDR registers which is done here for both 32 and 64bit versions of the
registers.
Add CTR register field definitions using Linux value and define names
and use the opportunity to rename CTR_L1Ip to use an upper case name
instead. The patch is also defining ICACHE_POLICY_xxx instead of only
having CTR_L1IP_xxx. Code using those defines is also updated by this
patch (arm32 setup).
On most big/LITTLE platforms this patch will activate TID2 and emulate
VIPT type of i-cache for all cores (as most LITTLE cores are VIPT where
big ones are PIPT). This is the case for example on Juno boards.
On platforms with only the same type of cores, this patch should not
modify the behaviour.
Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
---
Changes in v3: none
Change in v2: Patch introduced in v2
---
xen/arch/arm/arm64/cpufeature.c | 19 +++++++++++---
xen/arch/arm/arm64/vsysreg.c | 40 ++++++++++++++++++++++++++++
xen/arch/arm/cpufeature.c | 2 ++
xen/arch/arm/domain.c | 8 ++++++
xen/arch/arm/setup.c | 2 +-
xen/arch/arm/vcpreg.c | 45 ++++++++++++++++++++++++++++++++
xen/include/asm-arm/arm64/hsr.h | 6 +++++
xen/include/asm-arm/cpufeature.h | 11 ++++++++
xen/include/asm-arm/processor.h | 18 ++++++++++---
9 files changed, 143 insertions(+), 8 deletions(-)
diff --git a/xen/arch/arm/arm64/cpufeature.c b/xen/arch/arm/arm64/cpufeature.c
index b1936ef1d6..334d590ba0 100644
--- a/xen/arch/arm/arm64/cpufeature.c
+++ b/xen/arch/arm/arm64/cpufeature.c
@@ -275,9 +275,6 @@ static const struct arm64_ftr_bits ftr_id_aa64mmfr2[] = {
ARM64_FTR_END,
};
-#if 0
-/* TODO: use this to sanitize the cache line size among cores */
-
static const struct arm64_ftr_bits ftr_ctr[] = {
ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_EXACT, 31, 1, 1), /* RES1 */
ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, CTR_DIC_SHIFT, 1, 1),
@@ -294,7 +291,6 @@ static const struct arm64_ftr_bits ftr_ctr[] = {
ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, CTR_IMINLINE_SHIFT, 4, 0),
ARM64_FTR_END,
};
-#endif
static const struct arm64_ftr_bits ftr_id_mmfr0[] = {
S_ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR0_INNERSHR_SHIFT, 4, 0xf),
@@ -510,6 +506,12 @@ static s64 arm64_ftr_safe_value(const struct arm64_ftr_bits *ftrp, s64 new,
* End of imported linux structures and code
*/
+/*
+ * This is set to true if we have different type of i-caches on cores
+ * and used to activate TID2 bit to emulate CTR_EL0 register for guests
+ */
+bool mismatch_cache_type = false;
+
static void sanitize_reg(u64 *cur_reg, u64 new_reg, const char *reg_name,
const struct arm64_ftr_bits *ftrp)
{
@@ -600,6 +602,15 @@ void update_system_features(const struct cpuinfo_arm *new)
*/
SANITIZE_REG(dczid, 0, dczid);
+ SANITIZE_REG(ctr, 0, ctr);
+
+ /*
+ * If CTR is different among cores, set mismatch_cache_type to activate
+ * TID2 bit in HCR and emulate CTR register access for guests.
+ */
+ if ( system_cpuinfo.ctr.bits[0] != new->ctr.bits[0] )
+ mismatch_cache_type = true;
+
if ( cpu_feature64_has_el0_32(&system_cpuinfo) )
{
SANITIZE_ID_REG(pfr32, 0, pfr0);
diff --git a/xen/arch/arm/arm64/vsysreg.c b/xen/arch/arm/arm64/vsysreg.c
index 887266dd46..17212bd7ae 100644
--- a/xen/arch/arm/arm64/vsysreg.c
+++ b/xen/arch/arm/arm64/vsysreg.c
@@ -341,6 +341,46 @@ void do_sysreg(struct cpu_user_regs *regs,
case HSR_SYSREG(3,0,c0,c7,7):
return handle_ro_raz(regs, regidx, hsr.sysreg.read, hsr, 1);
+ /*
+ * HCR_EL2.TID2
+ *
+ * registers related to cache detection
+ */
+ case HSR_SYSREG_CTR_EL0:
+ return handle_ro_read_val(regs, regidx, hsr.sysreg.read, hsr, 1,
+ system_cpuinfo.ctr.bits[0]);
+
+ case HSR_SYSREG_CLIDR_EL1:
+ return handle_ro_read_val(regs, regidx, hsr.sysreg.read, hsr, 1,
+ READ_SYSREG(CLIDR_EL1));
+
+ case HSR_SYSREG_CSSELR_EL1:
+ if ( psr_mode_is_user(regs) )
+ return inject_undef_exception(regs, hsr);
+ if ( hsr.sysreg.read )
+ set_user_reg(regs, regidx, v->arch.csselr);
+ else
+ v->arch.csselr = get_user_reg(regs, regidx);
+ break;
+
+ case HSR_SYSREG_CCSIDR_EL1:
+ if ( psr_mode_is_user(regs) )
+ return inject_undef_exception(regs, hsr);
+ if ( hsr.sysreg.read )
+ {
+ /* we need to set CSSELR and do the read of CCSIDR atomically */
+ WRITE_SYSREG(v->arch.csselr, CSSELR_EL1);
+ set_user_reg(regs, regidx, READ_SYSREG(CCSIDR_EL1));
+ }
+ break;
+
+ case HSR_SYSREG_CCSIDR2_EL1:
+ /*
+ * This would need to return a properly defined value if CCIDX is
+ * implemented in the processor
+ */
+ return inject_undef_exception(regs, hsr);
+
/*
* HCR_EL2.TIDCP
*
diff --git a/xen/arch/arm/cpufeature.c b/xen/arch/arm/cpufeature.c
index 113f20f601..6e51f530a8 100644
--- a/xen/arch/arm/cpufeature.c
+++ b/xen/arch/arm/cpufeature.c
@@ -127,6 +127,8 @@ void identify_cpu(struct cpuinfo_arm *c)
c->dczid.bits[0] = READ_SYSREG(DCZID_EL0);
+ c->ctr.bits[0] = READ_SYSREG(CTR_EL0);
+
aarch32_el0 = cpu_feature64_has_el0_32(c);
#endif
diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
index 19c756ac3d..7a97fde3e7 100644
--- a/xen/arch/arm/domain.c
+++ b/xen/arch/arm/domain.c
@@ -585,6 +585,14 @@ int arch_vcpu_create(struct vcpu *v)
v->arch.vmpidr = MPIDR_SMP | vcpuid_to_vaffinity(v->vcpu_id);
v->arch.hcr_el2 = get_default_hcr_flags();
+#ifdef CONFIG_ARM64
+ /*
+ * Only activated TID2 to catch access to CTR_EL0 if the platform has some
+ * mismatching i-cache types among cores
+ */
+ if ( mismatch_cache_type )
+ v->arch.hcr_el2 |= HCR_TID2;
+#endif
if ( (rc = vcpu_vgic_init(v)) != 0 )
goto fail;
diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
index 3798c5ade0..33b7bfb59c 100644
--- a/xen/arch/arm/setup.c
+++ b/xen/arch/arm/setup.c
@@ -627,7 +627,7 @@ static void __init setup_mm(void)
panic("No memory bank\n");
/* We only supports instruction caches implementing the IVIPT extension. */
- if ( ((ctr >> CTR_L1Ip_SHIFT) & CTR_L1Ip_MASK) == CTR_L1Ip_AIVIVT )
+ if ( ((ctr >> CTR_L1IP_SHIFT) & CTR_L1IP_MASK) == ICACHE_POLICY_AIVIVT )
panic("AIVIVT instruction cache not supported\n");
init_pdx();
diff --git a/xen/arch/arm/vcpreg.c b/xen/arch/arm/vcpreg.c
index 33259c4194..5ffed96ded 100644
--- a/xen/arch/arm/vcpreg.c
+++ b/xen/arch/arm/vcpreg.c
@@ -361,6 +361,51 @@ void do_cp15_32(struct cpu_user_regs *regs, const union hsr hsr)
HSR_CPREG32_TID3_CASES(c7):
return handle_ro_raz(regs, regidx, cp32.read, hsr, 1);
+#ifdef CONFIG_ARM64
+ /*
+ * HCR_EL2.TID2
+ *
+ * registers related to cache detection
+ * Only supported on arm64 as we do not sanitize cpuinfo on arm32 so we
+ * do not need to emulate those.
+ */
+ case HSR_CPREG32(CTR):
+ return handle_ro_read_val(regs, regidx, cp32.read, hsr, 1,
+ system_cpuinfo.ctr.bits[0]);
+
+ case HSR_CPREG32(CLIDR):
+ return handle_ro_read_val(regs, regidx, cp32.read, hsr, 1,
+ READ_SYSREG(CLIDR_EL1));
+
+ case HSR_CPREG32(CSSELR):
+ if ( psr_mode_is_user(regs) )
+ return inject_undef_exception(regs, hsr);
+ if ( cp32.read )
+ set_user_reg(regs, regidx, v->arch.csselr);
+ else
+ v->arch.csselr = get_user_reg(regs, regidx);
+ break;
+
+ case HSR_CPREG32(CCSIDR):
+ if ( psr_mode_is_user(regs) )
+ return inject_undef_exception(regs, hsr);
+ if ( cp32.read )
+ {
+ /* we need to set CSSELR and do the read of CCSIDR atomically */
+ WRITE_SYSREG(v->arch.csselr, CSSELR_EL1);
+ set_user_reg(regs, regidx, READ_SYSREG(CCSIDR_EL1));
+ }
+ break;
+
+ case HSR_CPREG32(CCSIDR2):
+ /*
+ * This would need to return a properly defined value if CCIDX is
+ * implemented in the processor
+ */
+ return inject_undef_exception(regs, hsr);
+
+#endif
+
/*
* HCR_EL2.TIDCP
*
diff --git a/xen/include/asm-arm/arm64/hsr.h b/xen/include/asm-arm/arm64/hsr.h
index e691d41c17..c33980e4e5 100644
--- a/xen/include/asm-arm/arm64/hsr.h
+++ b/xen/include/asm-arm/arm64/hsr.h
@@ -147,6 +147,12 @@
#define HSR_SYSREG_ID_AA64AFR1_EL1 HSR_SYSREG(3,0,c0,c5,5)
#define HSR_SYSREG_ID_AA64ZFR0_EL1 HSR_SYSREG(3,0,c0,c4,4)
+#define HSR_SYSREG_CTR_EL0 HSR_SYSREG(3,3,c0,c0,1)
+#define HSR_SYSREG_CLIDR_EL1 HSR_SYSREG(3,1,c0,c0,1)
+#define HSR_SYSREG_CSSELR_EL1 HSR_SYSREG(3,2,c0,c0,0)
+#define HSR_SYSREG_CCSIDR_EL1 HSR_SYSREG(3,1,c0,c0,0)
+#define HSR_SYSREG_CCSIDR2_EL1 HSR_SYSREG(3,1,c0,c0,2)
+
#endif /* __ASM_ARM_ARM64_HSR_H */
/*
diff --git a/xen/include/asm-arm/cpufeature.h b/xen/include/asm-arm/cpufeature.h
index 5219fd3bab..ca6e827fcb 100644
--- a/xen/include/asm-arm/cpufeature.h
+++ b/xen/include/asm-arm/cpufeature.h
@@ -267,6 +267,14 @@ struct cpuinfo_arm {
register_t bits[1];
} dczid;
+ /*
+ * CTR is only used to check for different cache types or policies and
+ * taint Xen in this case
+ */
+ struct {
+ register_t bits[1];
+ } ctr;
+
#endif
/*
@@ -339,6 +347,9 @@ extern struct cpuinfo_arm system_cpuinfo;
extern void identify_cpu(struct cpuinfo_arm *);
#ifdef CONFIG_ARM_64
+
+extern bool mismatched_cache_type;
+
extern void update_system_features(const struct cpuinfo_arm *);
#else
static inline void update_system_features(const struct cpuinfo_arm *cpuinfo)
diff --git a/xen/include/asm-arm/processor.h b/xen/include/asm-arm/processor.h
index 2577e9a244..8c9843e12b 100644
--- a/xen/include/asm-arm/processor.h
+++ b/xen/include/asm-arm/processor.h
@@ -7,9 +7,21 @@
#include <public/arch-arm.h>
/* CTR Cache Type Register */
-#define CTR_L1Ip_MASK 0x3
-#define CTR_L1Ip_SHIFT 14
-#define CTR_L1Ip_AIVIVT 0x1
+#define CTR_L1IP_MASK 0x3
+#define CTR_L1IP_SHIFT 14
+#define CTR_DMINLINE_SHIFT 16
+#define CTR_IMINLINE_SHIFT 0
+#define CTR_IMINLINE_MASK 0xf
+#define CTR_ERG_SHIFT 20
+#define CTR_CWG_SHIFT 24
+#define CTR_CWG_MASK 15
+#define CTR_IDC_SHIFT 28
+#define CTR_DIC_SHIFT 29
+
+#define ICACHE_POLICY_VPIPT 0
+#define ICACHE_POLICY_AIVIVT 1
+#define ICACHE_POLICY_VIPT 2
+#define ICACHE_POLICY_PIPT 3
/* MIDR Main ID Register */
#define MIDR_REVISION_MASK 0xf
--
2.17.1
^ permalink raw reply related [flat|nested] 26+ messages in thread
* Re: [PATCH v3 1/7] xen/arm: Import ID registers definitions from Linux
2021-08-25 13:18 ` [PATCH v3 1/7] xen/arm: Import ID registers definitions from Linux Bertrand Marquis
@ 2021-08-27 14:40 ` Julien Grall
0 siblings, 0 replies; 26+ messages in thread
From: Julien Grall @ 2021-08-27 14:40 UTC (permalink / raw)
To: Bertrand Marquis, xen-devel; +Cc: Stefano Stabellini, Volodymyr Babchuk
Hi Bertrand,
On 25/08/2021 14:18, Bertrand Marquis wrote:
> Import some ID registers definitions from Linux sysreg header to have
> required shift definitions for all ID registers fields.
>
> Those are required to reuse the cpufeature sanitization system from
> Linux kernel.
>
> Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
Acked-by: Julien Grall <jgrall@amazon.com>
Cheers,
--
Julien Grall
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [PATCH v3 7/7] xen/arm: Sanitize CTR_EL0 and emulate it if needed
2021-08-25 13:18 ` [PATCH v3 7/7] xen/arm: Sanitize CTR_EL0 and emulate it if needed Bertrand Marquis
@ 2021-08-27 15:05 ` Julien Grall
2021-08-31 13:17 ` Bertrand Marquis
0 siblings, 1 reply; 26+ messages in thread
From: Julien Grall @ 2021-08-27 15:05 UTC (permalink / raw)
To: Bertrand Marquis, xen-devel; +Cc: Stefano Stabellini, Volodymyr Babchuk
Hi Bertrand,
On 25/08/2021 14:18, Bertrand Marquis wrote:
> Sanitize CTR_EL0 value between cores.
>
> In most cases different values will taint Xen but if different
> i-cache policies are found, we choose the one which will be compatible
> between all cores in terms of invalidation/data cache flushing strategy.
I understand that all the CPUs in Xen needs to agree on the cache flush
strategy. However...
>
> In this case we need to activate the TID2 bit in HCR to emulate the
> TCR_EL0 register for guests. This patch is not activating TID2 bit all
> the time to limit the overhead when possible.
as we discussed in an earlier version, a vCPU is unlikely (at least in
short/medium) to be able move across pCPU of different type. So the vCPU
would be pinned to a set of pCPUs. IOW, the guest would have to be
big.LITTLE aware and therefore would be able to do its own strategy
decision.
So I think we should be able to get away from trappings the registers.
> When TID2 is activate we also need to emulate the CCSIDR, CSSELR and
> CLIDR registers which is done here for both 32 and 64bit versions of the
> registers.
>
> Add CTR register field definitions using Linux value and define names
> and use the opportunity to rename CTR_L1Ip to use an upper case name
> instead. The patch is also defining ICACHE_POLICY_xxx instead of only
> having CTR_L1IP_xxx. Code using those defines is also updated by this
> patch (arm32 setup).
>
> On most big/LITTLE platforms this patch will activate TID2 and emulate
> VIPT type of i-cache for all cores (as most LITTLE cores are VIPT where
> big ones are PIPT). This is the case for example on Juno boards.
>
> On platforms with only the same type of cores, this patch should not
> modify the behaviour.
>
> Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
> ---
> Changes in v3: none
> Change in v2: Patch introduced in v2
> ---
> xen/arch/arm/arm64/cpufeature.c | 19 +++++++++++---
> xen/arch/arm/arm64/vsysreg.c | 40 ++++++++++++++++++++++++++++
> xen/arch/arm/cpufeature.c | 2 ++
> xen/arch/arm/domain.c | 8 ++++++
> xen/arch/arm/setup.c | 2 +-
> xen/arch/arm/vcpreg.c | 45 ++++++++++++++++++++++++++++++++
> xen/include/asm-arm/arm64/hsr.h | 6 +++++
> xen/include/asm-arm/cpufeature.h | 11 ++++++++
> xen/include/asm-arm/processor.h | 18 ++++++++++---
> 9 files changed, 143 insertions(+), 8 deletions(-)
>
> diff --git a/xen/arch/arm/arm64/cpufeature.c b/xen/arch/arm/arm64/cpufeature.c
> index b1936ef1d6..334d590ba0 100644
> --- a/xen/arch/arm/arm64/cpufeature.c
> +++ b/xen/arch/arm/arm64/cpufeature.c
> @@ -275,9 +275,6 @@ static const struct arm64_ftr_bits ftr_id_aa64mmfr2[] = {
> ARM64_FTR_END,
> };
>
> -#if 0
> -/* TODO: use this to sanitize the cache line size among cores */
> -
> static const struct arm64_ftr_bits ftr_ctr[] = {
> ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_EXACT, 31, 1, 1), /* RES1 */
> ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, CTR_DIC_SHIFT, 1, 1),
> @@ -294,7 +291,6 @@ static const struct arm64_ftr_bits ftr_ctr[] = {
> ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, CTR_IMINLINE_SHIFT, 4, 0),
> ARM64_FTR_END,
> };
> -#endif
>
> static const struct arm64_ftr_bits ftr_id_mmfr0[] = {
> S_ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR0_INNERSHR_SHIFT, 4, 0xf),
> @@ -510,6 +506,12 @@ static s64 arm64_ftr_safe_value(const struct arm64_ftr_bits *ftrp, s64 new,
> * End of imported linux structures and code
> */
>
> +/*
> + * This is set to true if we have different type of i-caches on cores
> + * and used to activate TID2 bit to emulate CTR_EL0 register for guests
> + */
> +bool mismatch_cache_type = false;
If we are still planning to trap and emulate the registers, then this
needs to be an HW capability (see cpus_set_cap()).
> +
> static void sanitize_reg(u64 *cur_reg, u64 new_reg, const char *reg_name,
> const struct arm64_ftr_bits *ftrp)
> {
> @@ -600,6 +602,15 @@ void update_system_features(const struct cpuinfo_arm *new)
> */
> SANITIZE_REG(dczid, 0, dczid);
>
> + SANITIZE_REG(ctr, 0, ctr);
> +
> + /*
> + * If CTR is different among cores, set mismatch_cache_type to activate
> + * TID2 bit in HCR and emulate CTR register access for guests.
> + */
> + if ( system_cpuinfo.ctr.bits[0] != new->ctr.bits[0] )
> + mismatch_cache_type = true;
> +
> if ( cpu_feature64_has_el0_32(&system_cpuinfo) )
> {
> SANITIZE_ID_REG(pfr32, 0, pfr0);
> diff --git a/xen/arch/arm/arm64/vsysreg.c b/xen/arch/arm/arm64/vsysreg.c
> index 887266dd46..17212bd7ae 100644
> --- a/xen/arch/arm/arm64/vsysreg.c
> +++ b/xen/arch/arm/arm64/vsysreg.c
> @@ -341,6 +341,46 @@ void do_sysreg(struct cpu_user_regs *regs,
> case HSR_SYSREG(3,0,c0,c7,7):
> return handle_ro_raz(regs, regidx, hsr.sysreg.read, hsr, 1);
>
> + /*
> + * HCR_EL2.TID2
> + *
> + * registers related to cache detection
> + */
> + case HSR_SYSREG_CTR_EL0:
> + return handle_ro_read_val(regs, regidx, hsr.sysreg.read, hsr, 1,
> + system_cpuinfo.ctr.bits[0]);
Coding style: This needs to be aligned with the first argument.
> +
> + case HSR_SYSREG_CLIDR_EL1:
> + return handle_ro_read_val(regs, regidx, hsr.sysreg.read, hsr, 1,
> + READ_SYSREG(CLIDR_EL1));
Same.
> +
> + case HSR_SYSREG_CSSELR_EL1:
> + if ( psr_mode_is_user(regs) )
> + return inject_undef_exception(regs, hsr);
> + if ( hsr.sysreg.read )
> + set_user_reg(regs, regidx, v->arch.csselr);
> + else
> + v->arch.csselr = get_user_reg(regs, regidx);
> + break;
> +
> + case HSR_SYSREG_CCSIDR_EL1:
> + if ( psr_mode_is_user(regs) )
> + return inject_undef_exception(regs, hsr);
> + if ( hsr.sysreg.read )
> + {
> + /* we need to set CSSELR and do the read of CCSIDR atomically */
I couldn't find this requirement in the Arm Arm. Do you have the section
at hand?
> + WRITE_SYSREG(v->arch.csselr, CSSELR_EL1);
> + set_user_reg(regs, regidx, READ_SYSREG(CCSIDR_EL1));
> + }
From the Arm Arm (D13.2.25 in ARM DDI 0487F.c):
"
If CSSELR_EL1.Level is programmed to a cache level that is not
implemented, then on a read of the CCSIDR_EL1
the behavior is CONSTRAINED UNPREDICTABLE, and can be one of the following:
• The CCSIDR_EL1 read is treated as NOP.
• The CCSIDR_EL1 read is UNDEFINED.
• The CCSIDR_EL1 read returns an UNKNOWN value.
"
We can't trust the guest here, so we need to prevent any of this
behavior (in particular 1 and 2) to happen. The options are:
1) Sanitize the values from the guest
2) Make sure the register is 0 before reading (for the NOP case to
avoid leaking a value from Xen) and catch the undefined (some similar to
the extable on x86).
> + break;
> +
> + case HSR_SYSREG_CCSIDR2_EL1:
> + /*
> + * This would need to return a properly defined value if CCIDX is
> + * implemented in the processor
> + */
> + return inject_undef_exception(regs, hsr);
> +
> /*
> * HCR_EL2.TIDCP
> *
> diff --git a/xen/arch/arm/cpufeature.c b/xen/arch/arm/cpufeature.c
> index 113f20f601..6e51f530a8 100644
> --- a/xen/arch/arm/cpufeature.c
> +++ b/xen/arch/arm/cpufeature.c
> @@ -127,6 +127,8 @@ void identify_cpu(struct cpuinfo_arm *c)
>
> c->dczid.bits[0] = READ_SYSREG(DCZID_EL0);
>
> + c->ctr.bits[0] = READ_SYSREG(CTR_EL0);
> +
> aarch32_el0 = cpu_feature64_has_el0_32(c);
> #endif
>
> diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
> index 19c756ac3d..7a97fde3e7 100644
> --- a/xen/arch/arm/domain.c
> +++ b/xen/arch/arm/domain.c
> @@ -585,6 +585,14 @@ int arch_vcpu_create(struct vcpu *v)
> v->arch.vmpidr = MPIDR_SMP | vcpuid_to_vaffinity(v->vcpu_id);
>
> v->arch.hcr_el2 = get_default_hcr_flags();
> +#ifdef CONFIG_ARM64
This #ifdef could be droppped if we use an HW caps.
> + /*
> + * Only activated TID2 to catch access to CTR_EL0 if the platform has some
> + * mismatching i-cache types among cores
> + */
> + if ( mismatch_cache_type )
> + v->arch.hcr_el2 |= HCR_TID2;
> +#endif
>
> if ( (rc = vcpu_vgic_init(v)) != 0 )
> goto fail;
> diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
> index 3798c5ade0..33b7bfb59c 100644
> --- a/xen/arch/arm/setup.c
> +++ b/xen/arch/arm/setup.c
> @@ -627,7 +627,7 @@ static void __init setup_mm(void)
> panic("No memory bank\n");
>
> /* We only supports instruction caches implementing the IVIPT extension. */
> - if ( ((ctr >> CTR_L1Ip_SHIFT) & CTR_L1Ip_MASK) == CTR_L1Ip_AIVIVT )
> + if ( ((ctr >> CTR_L1IP_SHIFT) & CTR_L1IP_MASK) == ICACHE_POLICY_AIVIVT )
> panic("AIVIVT instruction cache not supported\n");
>
> init_pdx();
> diff --git a/xen/arch/arm/vcpreg.c b/xen/arch/arm/vcpreg.c
> index 33259c4194..5ffed96ded 100644
> --- a/xen/arch/arm/vcpreg.c
> +++ b/xen/arch/arm/vcpreg.c
> @@ -361,6 +361,51 @@ void do_cp15_32(struct cpu_user_regs *regs, const union hsr hsr)
> HSR_CPREG32_TID3_CASES(c7):
> return handle_ro_raz(regs, regidx, cp32.read, hsr, 1);
>
> +#ifdef CONFIG_ARM64
> + /*
> + * HCR_EL2.TID2
> + *
> + * registers related to cache detection
> + * Only supported on arm64 as we do not sanitize cpuinfo on arm32 so we
> + * do not need to emulate those.
> + */
> + case HSR_CPREG32(CTR):
> + return handle_ro_read_val(regs, regidx, cp32.read, hsr, 1,
> + system_cpuinfo.ctr.bits[0]);
> +
> + case HSR_CPREG32(CLIDR):
> + return handle_ro_read_val(regs, regidx, cp32.read, hsr, 1,
> + READ_SYSREG(CLIDR_EL1));
> +
> + case HSR_CPREG32(CSSELR):
> + if ( psr_mode_is_user(regs) )
> + return inject_undef_exception(regs, hsr);
> + if ( cp32.read )
> + set_user_reg(regs, regidx, v->arch.csselr);
> + else
> + v->arch.csselr = get_user_reg(regs, regidx);
> + break;
> +
> + case HSR_CPREG32(CCSIDR):
> + if ( psr_mode_is_user(regs) )
> + return inject_undef_exception(regs, hsr);
> + if ( cp32.read )
> + {
> + /* we need to set CSSELR and do the read of CCSIDR atomically */
> + WRITE_SYSREG(v->arch.csselr, CSSELR_EL1);
> + set_user_reg(regs, regidx, READ_SYSREG(CCSIDR_EL1));
> + }
> + break;
> +
> + case HSR_CPREG32(CCSIDR2):
> + /*
> + * This would need to return a properly defined value if CCIDX is
> + * implemented in the processor
> + */
> + return inject_undef_exception(regs, hsr);
> +
> +#endif
> +
> /*
> * HCR_EL2.TIDCP
> *
> diff --git a/xen/include/asm-arm/arm64/hsr.h b/xen/include/asm-arm/arm64/hsr.h
> index e691d41c17..c33980e4e5 100644
> --- a/xen/include/asm-arm/arm64/hsr.h
> +++ b/xen/include/asm-arm/arm64/hsr.h
> @@ -147,6 +147,12 @@
> #define HSR_SYSREG_ID_AA64AFR1_EL1 HSR_SYSREG(3,0,c0,c5,5)
> #define HSR_SYSREG_ID_AA64ZFR0_EL1 HSR_SYSREG(3,0,c0,c4,4)
>
> +#define HSR_SYSREG_CTR_EL0 HSR_SYSREG(3,3,c0,c0,1)
> +#define HSR_SYSREG_CLIDR_EL1 HSR_SYSREG(3,1,c0,c0,1)
> +#define HSR_SYSREG_CSSELR_EL1 HSR_SYSREG(3,2,c0,c0,0)
> +#define HSR_SYSREG_CCSIDR_EL1 HSR_SYSREG(3,1,c0,c0,0)
> +#define HSR_SYSREG_CCSIDR2_EL1 HSR_SYSREG(3,1,c0,c0,2)
> +
> #endif /* __ASM_ARM_ARM64_HSR_H */
>
> /*
> diff --git a/xen/include/asm-arm/cpufeature.h b/xen/include/asm-arm/cpufeature.h
> index 5219fd3bab..ca6e827fcb 100644
> --- a/xen/include/asm-arm/cpufeature.h
> +++ b/xen/include/asm-arm/cpufeature.h
> @@ -267,6 +267,14 @@ struct cpuinfo_arm {
> register_t bits[1];
> } dczid;
>
> + /*
> + * CTR is only used to check for different cache types or policies and
> + * taint Xen in this case
> + */
> + struct {
> + register_t bits[1];
> + } ctr;
> +
> #endif
>
> /*
> @@ -339,6 +347,9 @@ extern struct cpuinfo_arm system_cpuinfo;
> extern void identify_cpu(struct cpuinfo_arm *);
>
> #ifdef CONFIG_ARM_64
> +
> +extern bool mismatched_cache_type;
> +
> extern void update_system_features(const struct cpuinfo_arm *);
> #else
> static inline void update_system_features(const struct cpuinfo_arm *cpuinfo)
> diff --git a/xen/include/asm-arm/processor.h b/xen/include/asm-arm/processor.h
> index 2577e9a244..8c9843e12b 100644
> --- a/xen/include/asm-arm/processor.h
> +++ b/xen/include/asm-arm/processor.h
> @@ -7,9 +7,21 @@
> #include <public/arch-arm.h>
>
> /* CTR Cache Type Register */
> -#define CTR_L1Ip_MASK 0x3
> -#define CTR_L1Ip_SHIFT 14
> -#define CTR_L1Ip_AIVIVT 0x1
> +#define CTR_L1IP_MASK 0x3
> +#define CTR_L1IP_SHIFT 14
> +#define CTR_DMINLINE_SHIFT 16
> +#define CTR_IMINLINE_SHIFT 0
> +#define CTR_IMINLINE_MASK 0xf
> +#define CTR_ERG_SHIFT 20
> +#define CTR_CWG_SHIFT 24
> +#define CTR_CWG_MASK 15
> +#define CTR_IDC_SHIFT 28
> +#define CTR_DIC_SHIFT 29
> +
> +#define ICACHE_POLICY_VPIPT 0
> +#define ICACHE_POLICY_AIVIVT 1
> +#define ICACHE_POLICY_VIPT 2
> +#define ICACHE_POLICY_PIPT 3
>
> /* MIDR Main ID Register */
> #define MIDR_REVISION_MASK 0xf
>
Cheers,
--
Julien Grall
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [PATCH v3 7/7] xen/arm: Sanitize CTR_EL0 and emulate it if needed
2021-08-27 15:05 ` Julien Grall
@ 2021-08-31 13:17 ` Bertrand Marquis
2021-08-31 14:47 ` Julien Grall
0 siblings, 1 reply; 26+ messages in thread
From: Bertrand Marquis @ 2021-08-31 13:17 UTC (permalink / raw)
To: Julien Grall; +Cc: xen-devel, Stefano Stabellini, Volodymyr Babchuk
Hi Julien,
> On 27 Aug 2021, at 16:05, Julien Grall <julien@xen.org> wrote:
>
> Hi Bertrand,
>
> On 25/08/2021 14:18, Bertrand Marquis wrote:
>> Sanitize CTR_EL0 value between cores.
>> In most cases different values will taint Xen but if different
>> i-cache policies are found, we choose the one which will be compatible
>> between all cores in terms of invalidation/data cache flushing strategy.
>
> I understand that all the CPUs in Xen needs to agree on the cache flush strategy. However...
>
>> In this case we need to activate the TID2 bit in HCR to emulate the
>> TCR_EL0 register for guests. This patch is not activating TID2 bit all
>> the time to limit the overhead when possible.
>
> as we discussed in an earlier version, a vCPU is unlikely (at least in short/medium) to be able move across pCPU of different type. So the vCPU would be pinned to a set of pCPUs. IOW, the guest would have to be big.LITTLE aware and therefore would be able to do its own strategy decision.
>
> So I think we should be able to get away from trappings the registers.
I do agree that we should be able to get away from that in the long term once
we have cpupools properly set but right now this is the only way to have
something useable (I will not say right).
I will work on finding a way to setup properly cpupools (or something else as
we discussed earlier) but in the short term I think this is the best we can do.
An other solution would be to discard this patch from the serie for now until
I have worked a proper solution for this case.
Should we discard or merge or do you have an other idea ?
>
>> When TID2 is activate we also need to emulate the CCSIDR, CSSELR and
>> CLIDR registers which is done here for both 32 and 64bit versions of the
>> registers.
>> Add CTR register field definitions using Linux value and define names
>> and use the opportunity to rename CTR_L1Ip to use an upper case name
>> instead. The patch is also defining ICACHE_POLICY_xxx instead of only
>> having CTR_L1IP_xxx. Code using those defines is also updated by this
>> patch (arm32 setup).
>> On most big/LITTLE platforms this patch will activate TID2 and emulate
>> VIPT type of i-cache for all cores (as most LITTLE cores are VIPT where
>> big ones are PIPT). This is the case for example on Juno boards.
>> On platforms with only the same type of cores, this patch should not
>> modify the behaviour.
>> Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
>> ---
>> Changes in v3: none
>> Change in v2: Patch introduced in v2
>> ---
>> xen/arch/arm/arm64/cpufeature.c | 19 +++++++++++---
>> xen/arch/arm/arm64/vsysreg.c | 40 ++++++++++++++++++++++++++++
>> xen/arch/arm/cpufeature.c | 2 ++
>> xen/arch/arm/domain.c | 8 ++++++
>> xen/arch/arm/setup.c | 2 +-
>> xen/arch/arm/vcpreg.c | 45 ++++++++++++++++++++++++++++++++
>> xen/include/asm-arm/arm64/hsr.h | 6 +++++
>> xen/include/asm-arm/cpufeature.h | 11 ++++++++
>> xen/include/asm-arm/processor.h | 18 ++++++++++---
>> 9 files changed, 143 insertions(+), 8 deletions(-)
>> diff --git a/xen/arch/arm/arm64/cpufeature.c b/xen/arch/arm/arm64/cpufeature.c
>> index b1936ef1d6..334d590ba0 100644
>> --- a/xen/arch/arm/arm64/cpufeature.c
>> +++ b/xen/arch/arm/arm64/cpufeature.c
>> @@ -275,9 +275,6 @@ static const struct arm64_ftr_bits ftr_id_aa64mmfr2[] = {
>> ARM64_FTR_END,
>> };
>> -#if 0
>> -/* TODO: use this to sanitize the cache line size among cores */
>> -
>> static const struct arm64_ftr_bits ftr_ctr[] = {
>> ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_EXACT, 31, 1, 1), /* RES1 */
>> ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, CTR_DIC_SHIFT, 1, 1),
>> @@ -294,7 +291,6 @@ static const struct arm64_ftr_bits ftr_ctr[] = {
>> ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, CTR_IMINLINE_SHIFT, 4, 0),
>> ARM64_FTR_END,
>> };
>> -#endif
>> static const struct arm64_ftr_bits ftr_id_mmfr0[] = {
>> S_ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR0_INNERSHR_SHIFT, 4, 0xf),
>> @@ -510,6 +506,12 @@ static s64 arm64_ftr_safe_value(const struct arm64_ftr_bits *ftrp, s64 new,
>> * End of imported linux structures and code
>> */
>> +/*
>> + * This is set to true if we have different type of i-caches on cores
>> + * and used to activate TID2 bit to emulate CTR_EL0 register for guests
>> + */
>> +bool mismatch_cache_type = false;
>
> If we are still planning to trap and emulate the registers, then this needs to be an HW capability (see cpus_set_cap()).
Ok
>
>> +
>> static void sanitize_reg(u64 *cur_reg, u64 new_reg, const char *reg_name,
>> const struct arm64_ftr_bits *ftrp)
>> {
>> @@ -600,6 +602,15 @@ void update_system_features(const struct cpuinfo_arm *new)
>> */
>> SANITIZE_REG(dczid, 0, dczid);
>> + SANITIZE_REG(ctr, 0, ctr);
>> +
>> + /*
>> + * If CTR is different among cores, set mismatch_cache_type to activate
>> + * TID2 bit in HCR and emulate CTR register access for guests.
>> + */
>> + if ( system_cpuinfo.ctr.bits[0] != new->ctr.bits[0] )
>> + mismatch_cache_type = true;
>> +
>> if ( cpu_feature64_has_el0_32(&system_cpuinfo) )
>> {
>> SANITIZE_ID_REG(pfr32, 0, pfr0);
>> diff --git a/xen/arch/arm/arm64/vsysreg.c b/xen/arch/arm/arm64/vsysreg.c
>> index 887266dd46..17212bd7ae 100644
>> --- a/xen/arch/arm/arm64/vsysreg.c
>> +++ b/xen/arch/arm/arm64/vsysreg.c
>> @@ -341,6 +341,46 @@ void do_sysreg(struct cpu_user_regs *regs,
>> case HSR_SYSREG(3,0,c0,c7,7):
>> return handle_ro_raz(regs, regidx, hsr.sysreg.read, hsr, 1);
>> + /*
>> + * HCR_EL2.TID2
>> + *
>> + * registers related to cache detection
>> + */
>> + case HSR_SYSREG_CTR_EL0:
>> + return handle_ro_read_val(regs, regidx, hsr.sysreg.read, hsr, 1,
>> + system_cpuinfo.ctr.bits[0]);
>
> Coding style: This needs to be aligned with the first argument.
Sure I will fix that
>
>> +
>> + case HSR_SYSREG_CLIDR_EL1:
>> + return handle_ro_read_val(regs, regidx, hsr.sysreg.read, hsr, 1,
>> + READ_SYSREG(CLIDR_EL1));
>
> Same.
Ok
>
>> +
>> + case HSR_SYSREG_CSSELR_EL1:
>> + if ( psr_mode_is_user(regs) )
>> + return inject_undef_exception(regs, hsr);
>> + if ( hsr.sysreg.read )
>> + set_user_reg(regs, regidx, v->arch.csselr);
>> + else
>> + v->arch.csselr = get_user_reg(regs, regidx);
>> + break;
>> +
>> + case HSR_SYSREG_CCSIDR_EL1:
>> + if ( psr_mode_is_user(regs) )
>> + return inject_undef_exception(regs, hsr);
>> + if ( hsr.sysreg.read )
>> + {
>> + /* we need to set CSSELR and do the read of CCSIDR atomically */
>
> I couldn't find this requirement in the Arm Arm. Do you have the section at hand?
If we get interrupted, someone could program CSSELR differently and the next read
will not be reflecting what the guest actually wants to do.
The code is not preemptible right now so this cannot be an issue but I added the
comment more as a warning.
This is not something from the documentation, this is because value written
in CSSELR is defining what is read from CCSIDR
>
>> + WRITE_SYSREG(v->arch.csselr, CSSELR_EL1);
>> + set_user_reg(regs, regidx, READ_SYSREG(CCSIDR_EL1));
>> + }
>
> From the Arm Arm (D13.2.25 in ARM DDI 0487F.c):
>
> "
> If CSSELR_EL1.Level is programmed to a cache level that is not implemented, then on a read of the CCSIDR_EL1
> the behavior is CONSTRAINED UNPREDICTABLE, and can be one of the following:
> • The CCSIDR_EL1 read is treated as NOP.
> • The CCSIDR_EL1 read is UNDEFINED.
> • The CCSIDR_EL1 read returns an UNKNOWN value.
> "
>
> We can't trust the guest here, so we need to prevent any of this behavior (in particular 1 and 2) to happen. The options are:
> 1) Sanitize the values from the guest
> 2) Make sure the register is 0 before reading (for the NOP case to avoid leaking a value from Xen) and catch the undefined (some similar to the extable on x86).
Good catch I should have thought of that.
I will check this but my preference would go to option 1 if feasible.
>
>> + break;
>> +
>> + case HSR_SYSREG_CCSIDR2_EL1:
>> + /*
>> + * This would need to return a properly defined value if CCIDX is
>> + * implemented in the processor
>> + */
>> + return inject_undef_exception(regs, hsr);
>> +
>> /*
>> * HCR_EL2.TIDCP
>> *
>> diff --git a/xen/arch/arm/cpufeature.c b/xen/arch/arm/cpufeature.c
>> index 113f20f601..6e51f530a8 100644
>> --- a/xen/arch/arm/cpufeature.c
>> +++ b/xen/arch/arm/cpufeature.c
>> @@ -127,6 +127,8 @@ void identify_cpu(struct cpuinfo_arm *c)
>> c->dczid.bits[0] = READ_SYSREG(DCZID_EL0);
>> + c->ctr.bits[0] = READ_SYSREG(CTR_EL0);
>> +
>> aarch32_el0 = cpu_feature64_has_el0_32(c);
>> #endif
>> diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
>> index 19c756ac3d..7a97fde3e7 100644
>> --- a/xen/arch/arm/domain.c
>> +++ b/xen/arch/arm/domain.c
>> @@ -585,6 +585,14 @@ int arch_vcpu_create(struct vcpu *v)
>> v->arch.vmpidr = MPIDR_SMP | vcpuid_to_vaffinity(v->vcpu_id);
>> v->arch.hcr_el2 = get_default_hcr_flags();
>> +#ifdef CONFIG_ARM64
>
> This #ifdef could be droppped if we use an HW caps.
Ack.
>
>> + /*
>> + * Only activated TID2 to catch access to CTR_EL0 if the platform has some
>> + * mismatching i-cache types among cores
>> + */
>> + if ( mismatch_cache_type )
>> + v->arch.hcr_el2 |= HCR_TID2;
>> +#endif
>> if ( (rc = vcpu_vgic_init(v)) != 0 )
>> goto fail;
>> diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
>> index 3798c5ade0..33b7bfb59c 100644
>> --- a/xen/arch/arm/setup.c
>> +++ b/xen/arch/arm/setup.c
>> @@ -627,7 +627,7 @@ static void __init setup_mm(void)
>> panic("No memory bank\n");
>> /* We only supports instruction caches implementing the IVIPT extension. */
>> - if ( ((ctr >> CTR_L1Ip_SHIFT) & CTR_L1Ip_MASK) == CTR_L1Ip_AIVIVT )
>> + if ( ((ctr >> CTR_L1IP_SHIFT) & CTR_L1IP_MASK) == ICACHE_POLICY_AIVIVT )
>> panic("AIVIVT instruction cache not supported\n");
>> init_pdx();
>> diff --git a/xen/arch/arm/vcpreg.c b/xen/arch/arm/vcpreg.c
>> index 33259c4194..5ffed96ded 100644
>> --- a/xen/arch/arm/vcpreg.c
>> +++ b/xen/arch/arm/vcpreg.c
>> @@ -361,6 +361,51 @@ void do_cp15_32(struct cpu_user_regs *regs, const union hsr hsr)
>> HSR_CPREG32_TID3_CASES(c7):
>> return handle_ro_raz(regs, regidx, cp32.read, hsr, 1);
>> +#ifdef CONFIG_ARM64
>> + /*
>> + * HCR_EL2.TID2
>> + *
>> + * registers related to cache detection
>> + * Only supported on arm64 as we do not sanitize cpuinfo on arm32 so we
>> + * do not need to emulate those.
>> + */
>> + case HSR_CPREG32(CTR):
>> + return handle_ro_read_val(regs, regidx, cp32.read, hsr, 1,
>> + system_cpuinfo.ctr.bits[0]);
>> +
>> + case HSR_CPREG32(CLIDR):
>> + return handle_ro_read_val(regs, regidx, cp32.read, hsr, 1,
>> + READ_SYSREG(CLIDR_EL1));
>> +
>> + case HSR_CPREG32(CSSELR):
>> + if ( psr_mode_is_user(regs) )
>> + return inject_undef_exception(regs, hsr);
>> + if ( cp32.read )
>> + set_user_reg(regs, regidx, v->arch.csselr);
>> + else
>> + v->arch.csselr = get_user_reg(regs, regidx);
>> + break;
>> +
>> + case HSR_CPREG32(CCSIDR):
>> + if ( psr_mode_is_user(regs) )
>> + return inject_undef_exception(regs, hsr);
>> + if ( cp32.read )
>> + {
>> + /* we need to set CSSELR and do the read of CCSIDR atomically */
>> + WRITE_SYSREG(v->arch.csselr, CSSELR_EL1);
>> + set_user_reg(regs, regidx, READ_SYSREG(CCSIDR_EL1));
>> + }
>> + break;
>> +
>> + case HSR_CPREG32(CCSIDR2):
>> + /*
>> + * This would need to return a properly defined value if CCIDX is
>> + * implemented in the processor
>> + */
>> + return inject_undef_exception(regs, hsr);
>> +
>> +#endif
>> +
>> /*
>> * HCR_EL2.TIDCP
>> *
>> diff --git a/xen/include/asm-arm/arm64/hsr.h b/xen/include/asm-arm/arm64/hsr.h
>> index e691d41c17..c33980e4e5 100644
>> --- a/xen/include/asm-arm/arm64/hsr.h
>> +++ b/xen/include/asm-arm/arm64/hsr.h
>> @@ -147,6 +147,12 @@
>> #define HSR_SYSREG_ID_AA64AFR1_EL1 HSR_SYSREG(3,0,c0,c5,5)
>> #define HSR_SYSREG_ID_AA64ZFR0_EL1 HSR_SYSREG(3,0,c0,c4,4)
>> +#define HSR_SYSREG_CTR_EL0 HSR_SYSREG(3,3,c0,c0,1)
>> +#define HSR_SYSREG_CLIDR_EL1 HSR_SYSREG(3,1,c0,c0,1)
>> +#define HSR_SYSREG_CSSELR_EL1 HSR_SYSREG(3,2,c0,c0,0)
>> +#define HSR_SYSREG_CCSIDR_EL1 HSR_SYSREG(3,1,c0,c0,0)
>> +#define HSR_SYSREG_CCSIDR2_EL1 HSR_SYSREG(3,1,c0,c0,2)
>> +
>> #endif /* __ASM_ARM_ARM64_HSR_H */
>> /*
>> diff --git a/xen/include/asm-arm/cpufeature.h b/xen/include/asm-arm/cpufeature.h
>> index 5219fd3bab..ca6e827fcb 100644
>> --- a/xen/include/asm-arm/cpufeature.h
>> +++ b/xen/include/asm-arm/cpufeature.h
>> @@ -267,6 +267,14 @@ struct cpuinfo_arm {
>> register_t bits[1];
>> } dczid;
>> + /*
>> + * CTR is only used to check for different cache types or policies and
>> + * taint Xen in this case
>> + */
>> + struct {
>> + register_t bits[1];
>> + } ctr;
>> +
>> #endif
>> /*
>> @@ -339,6 +347,9 @@ extern struct cpuinfo_arm system_cpuinfo;
>> extern void identify_cpu(struct cpuinfo_arm *);
>> #ifdef CONFIG_ARM_64
>> +
>> +extern bool mismatched_cache_type;
>> +
>> extern void update_system_features(const struct cpuinfo_arm *);
>> #else
>> static inline void update_system_features(const struct cpuinfo_arm *cpuinfo)
>> diff --git a/xen/include/asm-arm/processor.h b/xen/include/asm-arm/processor.h
>> index 2577e9a244..8c9843e12b 100644
>> --- a/xen/include/asm-arm/processor.h
>> +++ b/xen/include/asm-arm/processor.h
>> @@ -7,9 +7,21 @@
>> #include <public/arch-arm.h>
>> /* CTR Cache Type Register */
>> -#define CTR_L1Ip_MASK 0x3
>> -#define CTR_L1Ip_SHIFT 14
>> -#define CTR_L1Ip_AIVIVT 0x1
>> +#define CTR_L1IP_MASK 0x3
>> +#define CTR_L1IP_SHIFT 14
>> +#define CTR_DMINLINE_SHIFT 16
>> +#define CTR_IMINLINE_SHIFT 0
>> +#define CTR_IMINLINE_MASK 0xf
>> +#define CTR_ERG_SHIFT 20
>> +#define CTR_CWG_SHIFT 24
>> +#define CTR_CWG_MASK 15
>> +#define CTR_IDC_SHIFT 28
>> +#define CTR_DIC_SHIFT 29
>> +
>> +#define ICACHE_POLICY_VPIPT 0
>> +#define ICACHE_POLICY_AIVIVT 1
>> +#define ICACHE_POLICY_VIPT 2
>> +#define ICACHE_POLICY_PIPT 3
>> /* MIDR Main ID Register */
>> #define MIDR_REVISION_MASK 0xf
>
> Cheers,
Thanks for the review
Cheers
Bertrand
>
> --
> Julien Grall
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [PATCH v3 7/7] xen/arm: Sanitize CTR_EL0 and emulate it if needed
2021-08-31 13:17 ` Bertrand Marquis
@ 2021-08-31 14:47 ` Julien Grall
2021-08-31 16:15 ` Bertrand Marquis
0 siblings, 1 reply; 26+ messages in thread
From: Julien Grall @ 2021-08-31 14:47 UTC (permalink / raw)
To: Bertrand Marquis; +Cc: xen-devel, Stefano Stabellini, Volodymyr Babchuk
On 31/08/2021 14:17, Bertrand Marquis wrote:
> Hi Julien,
Hi Bertrand,
>
>> On 27 Aug 2021, at 16:05, Julien Grall <julien@xen.org> wrote:
>>
>> Hi Bertrand,
>>
>> On 25/08/2021 14:18, Bertrand Marquis wrote:
>>> Sanitize CTR_EL0 value between cores.
>>> In most cases different values will taint Xen but if different
>>> i-cache policies are found, we choose the one which will be compatible
>>> between all cores in terms of invalidation/data cache flushing strategy.
>>
>> I understand that all the CPUs in Xen needs to agree on the cache flush strategy. However...
>>
>>> In this case we need to activate the TID2 bit in HCR to emulate the
>>> TCR_EL0 register for guests. This patch is not activating TID2 bit all
>>> the time to limit the overhead when possible.
>>
>> as we discussed in an earlier version, a vCPU is unlikely (at least in short/medium) to be able move across pCPU of different type. So the vCPU would be pinned to a set of pCPUs. IOW, the guest would have to be big.LITTLE aware and therefore would be able to do its own strategy decision.
>>
>> So I think we should be able to get away from trappings the registers.
>
> I do agree that we should be able to get away from that in the long term once
> we have cpupools properly set but right now this is the only way to have
> something useable (I will not say right).
> I will work on finding a way to setup properly cpupools (or something else as
> we discussed earlier) but in the short term I think this is the best we can do.
My concern is you are making look like Xen will be able to deal nicely
with big.LITTLE when in fact there are a lot more potential issue by
allow a vCPU moving accross pCPU of different type (the errata is one
example).
>
> An other solution would be to discard this patch from the serie for now until
> I have worked a proper solution for this case.
>
> Should we discard or merge or do you have an other idea ?
Please correct me if I am wrong, at the moment, it doesn't look like
this patch will be part of the longer plan. If so, then I think it
should be parked for now.
This would also have the advantage to avoid spending too much time on
resolving the emulation issue I mentioned in my previous answer.
No need to resend a new version of this series yet. You can wait until
the rest of the series get more feedback.
[...]
> If we get interrupted, someone could program CSSELR differently and the next read
> will not be reflecting what the guest actually wants to do
AFAICT, CSSELR is preserved during the context switch of vCPU. So that
someone would have to be Xen, right?
If so, what you describe would also be an issue even if we didn't trap
the register. Therefore, if Xen would ever use CSSELR, then that code
would need to save the value, use the register and then restore the
value with preemption disabled.
>
> The code is not preemptible right now so this cannot be an issue but I added the
> comment more as a warning.
>
> This is not something from the documentation, this is because value written
> in CSSELR is defining what is read from CCSIDR
>
>>
>>> + WRITE_SYSREG(v->arch.csselr, CSSELR_EL1);
>>> + set_user_reg(regs, regidx, READ_SYSREG(CCSIDR_EL1));
>>> + }
Cheers,
--
Julien Grall
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [PATCH v3 7/7] xen/arm: Sanitize CTR_EL0 and emulate it if needed
2021-08-31 14:47 ` Julien Grall
@ 2021-08-31 16:15 ` Bertrand Marquis
2021-09-03 22:49 ` Stefano Stabellini
0 siblings, 1 reply; 26+ messages in thread
From: Bertrand Marquis @ 2021-08-31 16:15 UTC (permalink / raw)
To: Julien Grall; +Cc: xen-devel, Stefano Stabellini, Volodymyr Babchuk
Hi Julien,
> On 31 Aug 2021, at 15:47, Julien Grall <julien@xen.org> wrote:
>
>
>
> On 31/08/2021 14:17, Bertrand Marquis wrote:
>> Hi Julien,
>
> Hi Bertrand,
>
>>> On 27 Aug 2021, at 16:05, Julien Grall <julien@xen.org> wrote:
>>>
>>> Hi Bertrand,
>>>
>>> On 25/08/2021 14:18, Bertrand Marquis wrote:
>>>> Sanitize CTR_EL0 value between cores.
>>>> In most cases different values will taint Xen but if different
>>>> i-cache policies are found, we choose the one which will be compatible
>>>> between all cores in terms of invalidation/data cache flushing strategy.
>>>
>>> I understand that all the CPUs in Xen needs to agree on the cache flush strategy. However...
>>>
>>>> In this case we need to activate the TID2 bit in HCR to emulate the
>>>> TCR_EL0 register for guests. This patch is not activating TID2 bit all
>>>> the time to limit the overhead when possible.
>>>
>>> as we discussed in an earlier version, a vCPU is unlikely (at least in short/medium) to be able move across pCPU of different type. So the vCPU would be pinned to a set of pCPUs. IOW, the guest would have to be big.LITTLE aware and therefore would be able to do its own strategy decision.
>>>
>>> So I think we should be able to get away from trappings the registers.
>> I do agree that we should be able to get away from that in the long term once
>> we have cpupools properly set but right now this is the only way to have
>> something useable (I will not say right).
>> I will work on finding a way to setup properly cpupools (or something else as
>> we discussed earlier) but in the short term I think this is the best we can do.
>
> My concern is you are making look like Xen will be able to deal nicely with big.LITTLE when in fact there are a lot more potential issue by allow a vCPU moving accross pCPU of different type (the errata is one example).
I agree and this is why Xen is tainted.
>
>> An other solution would be to discard this patch from the serie for now until
>> I have worked a proper solution for this case.
>> Should we discard or merge or do you have an other idea ?
> Please correct me if I am wrong, at the moment, it doesn't look like this patch will be part of the longer plan. If so, then I think it should be parked for now.
Not sure it depends on what the final solution would be but this is highly possible I agree.
>
> This would also have the advantage to avoid spending too much time on resolving the emulation issue I mentioned in my previous answer.
>
> No need to resend a new version of this series yet. You can wait until the rest of the series get more feedback.
Ok, I will wait for feedback and next serie will not include this patch anymore.
>
> [...]
>
>> If we get interrupted, someone could program CSSELR differently and the next read
>> will not be reflecting what the guest actually wants to do
>
> AFAICT, CSSELR is preserved during the context switch of vCPU. So that someone would have to be Xen, right?
>
> If so, what you describe would also be an issue even if we didn't trap the register. Therefore, if Xen would ever use CSSELR, then that code would need to save the value, use the register and then restore the value with preemption disabled.
I could just remove the comment, I added it as information, but if you think it is misleading no problem.
Anyway as we will park this for now no need to discuss that further.
Cheers
Bertrand
>
>> The code is not preemptible right now so this cannot be an issue but I added the
>> comment more as a warning.
>> This is not something from the documentation, this is because value written
>> in CSSELR is defining what is read from CCSIDR
>>>
>>>> + WRITE_SYSREG(v->arch.csselr, CSSELR_EL1);
>>>> + set_user_reg(regs, regidx, READ_SYSREG(CCSIDR_EL1));
>>>> + }
>
> Cheers,
>
> --
> Julien Grall
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [PATCH v3 2/7] xen/arm: Import ID features sanitize from linux
2021-08-25 13:18 ` [PATCH v3 2/7] xen/arm: Import ID features sanitize from linux Bertrand Marquis
@ 2021-09-03 22:48 ` Stefano Stabellini
0 siblings, 0 replies; 26+ messages in thread
From: Stefano Stabellini @ 2021-09-03 22:48 UTC (permalink / raw)
To: Bertrand Marquis
Cc: xen-devel, Stefano Stabellini, Julien Grall, Volodymyr Babchuk
On Wed, 25 Aug 2021, Bertrand Marquis wrote:
> Import structures declared in Linux file arch/arm64/kernel/cpufeature.c
> and the required types from arch/arm64/include/asm/cpufeature.h.
>
> Current code has been imported from Linux 5.13-rc5 (Commit ID
> cd1245d75ce93b8fd206f4b34eb58bcfe156d5e9) and copied into cpufeature.c
> in arm64 code and cpufeature.h in arm64 specific headers.
>
> Those structure will be used to sanitize the cpu features available to
> the ones availble on all cores of a system even if we are on an
> heterogeneous platform (from example a big/LITTLE).
>
> For each feature field of all ID registers, those structures define what
> is the safest value and if we can allow to have different values in
> different cores.
>
> This patch is introducing Linux code without any changes to it.
>
> Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
> ---
> Changes in v3: none
> Changes in v2:
> - Move add to Makefile to following patch to allow bisection
> - Remove GPL text as SPDL is there
> - Re-add introduction comment from Linux Kernel file
> - Rename cpusanitize.c to cpufeature.c to keep Linux file name
> - Move structures imported from linux headers into a new cpufeature.h
> header in asm-arm/arm64.
> - Move comment about imported code origin to the file header
> - Remove not needed linux function declarations instead of removing them
> in the following patch
> - Add original arm64_ftr_safe_value from Linux
> - include kernel.h to use max()
> - remove unused ftr_single32 as we will not use it
> - remove ctr associated structures that we cannot use (keep the one
> defining sanitization bits)
> ---
> xen/arch/arm/arm64/cpufeature.c | 504 +++++++++++++++++++++++++
> xen/include/asm-arm/arm64/cpufeature.h | 104 +++++
> 2 files changed, 608 insertions(+)
> create mode 100644 xen/arch/arm/arm64/cpufeature.c
> create mode 100644 xen/include/asm-arm/arm64/cpufeature.h
>
> diff --git a/xen/arch/arm/arm64/cpufeature.c b/xen/arch/arm/arm64/cpufeature.c
> new file mode 100644
> index 0000000000..5777e33e5c
> --- /dev/null
> +++ b/xen/arch/arm/arm64/cpufeature.c
> @@ -0,0 +1,504 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/*
> + * Contains CPU feature definitions
> + *
> + * The following structures have been imported directly from Linux kernel and
> + * should be kept in sync.
> + * The current version has been imported from arch/arm64/kernel/cpufeature.c
> + * from kernel version 5.13-rc5 together with the required structures and
> + * macros from arch/arm64/include/asm/cpufeature.h which are stored in
> + * include/asm-arm/arm64/cpufeature.h
> + *
> + * Copyright (C) 2021 Arm Ltd.
> + * based on code from the Linux kernel, which is:
> + * Copyright (C) 2015 ARM Ltd.
> + *
> + * A note for the weary kernel hacker: the code here is confusing and hard to
> + * follow! That's partly because it's solving a nasty problem, but also because
> + * there's a little bit of over-abstraction that tends to obscure what's going
> + * on behind a maze of helper functions and macros.
> + *
> + * The basic problem is that hardware folks have started gluing together CPUs
> + * with distinct architectural features; in some cases even creating SoCs where
> + * user-visible instructions are available only on a subset of the available
> + * cores. We try to address this by snapshotting the feature registers of the
> + * boot CPU and comparing these with the feature registers of each secondary
> + * CPU when bringing them up. If there is a mismatch, then we update the
> + * snapshot state to indicate the lowest-common denominator of the feature,
> + * known as the "safe" value. This snapshot state can be queried to view the
> + * "sanitised" value of a feature register.
> + *
> + * The sanitised register values are used to decide which capabilities we
> + * have in the system. These may be in the form of traditional "hwcaps"
> + * advertised to userspace or internal "cpucaps" which are used to configure
> + * things like alternative patching and static keys. While a feature mismatch
> + * may result in a TAINT_CPU_OUT_OF_SPEC kernel taint, a capability mismatch
> + * may prevent a CPU from being onlined at all.
> + *
> + * Some implementation details worth remembering:
> + *
> + * - Mismatched features are *always* sanitised to a "safe" value, which
> + * usually indicates that the feature is not supported.
> + *
> + * - A mismatched feature marked with FTR_STRICT will cause a "SANITY CHECK"
> + * warning when onlining an offending CPU and the kernel will be tainted
> + * with TAINT_CPU_OUT_OF_SPEC.
> + *
> + * - Features marked as FTR_VISIBLE have their sanitised value visible to
> + * userspace. FTR_VISIBLE features in registers that are only visible
> + * to EL0 by trapping *must* have a corresponding HWCAP so that late
> + * onlining of CPUs cannot lead to features disappearing at runtime.
> + *
> + * - A "feature" is typically a 4-bit register field. A "capability" is the
> + * high-level description derived from the sanitised field value.
> + *
> + * - Read the Arm ARM (DDI 0487F.a) section D13.1.3 ("Principles of the ID
> + * scheme for fields in ID registers") to understand when feature fields
> + * may be signed or unsigned (FTR_SIGNED and FTR_UNSIGNED accordingly).
> + *
> + * - KVM exposes its own view of the feature registers to guest operating
> + * systems regardless of FTR_VISIBLE. This is typically driven from the
> + * sanitised register values to allow virtual CPUs to be migrated between
> + * arbitrary physical CPUs, but some features not present on the host are
> + * also advertised and emulated. Look at sys_reg_descs[] for the gory
> + * details.
> + *
> + * - If the arm64_ftr_bits[] for a register has a missing field, then this
> + * field is treated as STRICT RES0, including for read_sanitised_ftr_reg().
> + * This is stronger than FTR_HIDDEN and can be used to hide features from
> + * KVM guests.
> + */
> +
> +#include <xen/types.h>
> +#include <xen/kernel.h>
> +#include <asm/sysregs.h>
> +#include <asm/cpufeature.h>
> +#include <asm/arm64/cpufeature.h>
> +
> +#define __ARM64_FTR_BITS(SIGNED, VISIBLE, STRICT, TYPE, SHIFT, WIDTH, SAFE_VAL) \
> + { \
> + .sign = SIGNED, \
> + .visible = VISIBLE, \
> + .strict = STRICT, \
> + .type = TYPE, \
> + .shift = SHIFT, \
> + .width = WIDTH, \
> + .safe_val = SAFE_VAL, \
> + }
> +
> +/* Define a feature with unsigned values */
> +#define ARM64_FTR_BITS(VISIBLE, STRICT, TYPE, SHIFT, WIDTH, SAFE_VAL) \
> + __ARM64_FTR_BITS(FTR_UNSIGNED, VISIBLE, STRICT, TYPE, SHIFT, WIDTH, SAFE_VAL)
> +
> +/* Define a feature with a signed value */
> +#define S_ARM64_FTR_BITS(VISIBLE, STRICT, TYPE, SHIFT, WIDTH, SAFE_VAL) \
> + __ARM64_FTR_BITS(FTR_SIGNED, VISIBLE, STRICT, TYPE, SHIFT, WIDTH, SAFE_VAL)
> +
> +#define ARM64_FTR_END \
> + { \
> + .width = 0, \
> + }
> +
> +/*
> + * NOTE: Any changes to the visibility of features should be kept in
> + * sync with the documentation of the CPU feature register ABI.
> + */
> +static const struct arm64_ftr_bits ftr_id_aa64isar0[] = {
> + ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR0_RNDR_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR0_TLB_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR0_TS_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR0_FHM_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR0_DP_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR0_SM4_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR0_SM3_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR0_SHA3_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR0_RDM_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR0_ATOMICS_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR0_CRC32_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR0_SHA2_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR0_SHA1_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR0_AES_SHIFT, 4, 0),
> + ARM64_FTR_END,
> +};
> +
> +static const struct arm64_ftr_bits ftr_id_aa64isar1[] = {
> + ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR1_I8MM_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR1_DGH_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR1_BF16_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR1_SPECRES_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR1_SB_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR1_FRINTTS_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_PTR_AUTH),
> + FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR1_GPI_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_PTR_AUTH),
> + FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR1_GPA_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR1_LRCPC_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR1_FCMA_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR1_JSCVT_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_PTR_AUTH),
> + FTR_STRICT, FTR_EXACT, ID_AA64ISAR1_API_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_PTR_AUTH),
> + FTR_STRICT, FTR_EXACT, ID_AA64ISAR1_APA_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR1_DPB_SHIFT, 4, 0),
> + ARM64_FTR_END,
> +};
> +
> +static const struct arm64_ftr_bits ftr_id_aa64pfr0[] = {
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64PFR0_CSV3_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64PFR0_CSV2_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR0_DIT_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64PFR0_AMU_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR0_MPAM_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64PFR0_SEL2_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_SVE),
> + FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR0_SVE_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR0_RAS_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR0_GIC_SHIFT, 4, 0),
> + S_ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR0_ASIMD_SHIFT, 4, ID_AA64PFR0_ASIMD_NI),
> + S_ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR0_FP_SHIFT, 4, ID_AA64PFR0_FP_NI),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64PFR0_EL3_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64PFR0_EL2_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64PFR0_EL1_SHIFT, 4, ID_AA64PFR0_EL1_64BIT_ONLY),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64PFR0_EL0_SHIFT, 4, ID_AA64PFR0_EL0_64BIT_ONLY),
> + ARM64_FTR_END,
> +};
> +
> +static const struct arm64_ftr_bits ftr_id_aa64pfr1[] = {
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR1_MPAMFRAC_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR1_RASFRAC_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_MTE),
> + FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR1_MTE_SHIFT, 4, ID_AA64PFR1_MTE_NI),
> + ARM64_FTR_BITS(FTR_VISIBLE, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64PFR1_SSBS_SHIFT, 4, ID_AA64PFR1_SSBS_PSTATE_NI),
> + ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_BTI),
> + FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR1_BT_SHIFT, 4, 0),
> + ARM64_FTR_END,
> +};
> +
> +static const struct arm64_ftr_bits ftr_id_aa64zfr0[] = {
> + ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_SVE),
> + FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ZFR0_F64MM_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_SVE),
> + FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ZFR0_F32MM_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_SVE),
> + FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ZFR0_I8MM_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_SVE),
> + FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ZFR0_SM4_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_SVE),
> + FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ZFR0_SHA3_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_SVE),
> + FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ZFR0_BF16_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_SVE),
> + FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ZFR0_BITPERM_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_SVE),
> + FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ZFR0_AES_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_SVE),
> + FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ZFR0_SVEVER_SHIFT, 4, 0),
> + ARM64_FTR_END,
> +};
> +
> +static const struct arm64_ftr_bits ftr_id_aa64mmfr0[] = {
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR0_ECV_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR0_FGT_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR0_EXS_SHIFT, 4, 0),
> + /*
> + * Page size not being supported at Stage-2 is not fatal. You
> + * just give up KVM if PAGE_SIZE isn't supported there. Go fix
> + * your favourite nesting hypervisor.
> + *
> + * There is a small corner case where the hypervisor explicitly
> + * advertises a given granule size at Stage-2 (value 2) on some
> + * vCPUs, and uses the fallback to Stage-1 (value 0) for other
> + * vCPUs. Although this is not forbidden by the architecture, it
> + * indicates that the hypervisor is being silly (or buggy).
> + *
> + * We make no effort to cope with this and pretend that if these
> + * fields are inconsistent across vCPUs, then it isn't worth
> + * trying to bring KVM up.
> + */
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_EXACT, ID_AA64MMFR0_TGRAN4_2_SHIFT, 4, 1),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_EXACT, ID_AA64MMFR0_TGRAN64_2_SHIFT, 4, 1),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_EXACT, ID_AA64MMFR0_TGRAN16_2_SHIFT, 4, 1),
> + /*
> + * We already refuse to boot CPUs that don't support our configured
> + * page size, so we can only detect mismatches for a page size other
> + * than the one we're currently using. Unfortunately, SoCs like this
> + * exist in the wild so, even though we don't like it, we'll have to go
> + * along with it and treat them as non-strict.
> + */
> + S_ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64MMFR0_TGRAN4_SHIFT, 4, ID_AA64MMFR0_TGRAN4_NI),
> + S_ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64MMFR0_TGRAN64_SHIFT, 4, ID_AA64MMFR0_TGRAN64_NI),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64MMFR0_TGRAN16_SHIFT, 4, ID_AA64MMFR0_TGRAN16_NI),
> +
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR0_BIGENDEL0_SHIFT, 4, 0),
> + /* Linux shouldn't care about secure memory */
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64MMFR0_SNSMEM_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR0_BIGENDEL_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR0_ASID_SHIFT, 4, 0),
> + /*
> + * Differing PARange is fine as long as all peripherals and memory are mapped
> + * within the minimum PARange of all CPUs
> + */
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64MMFR0_PARANGE_SHIFT, 4, 0),
> + ARM64_FTR_END,
> +};
> +
> +static const struct arm64_ftr_bits ftr_id_aa64mmfr1[] = {
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR1_ETS_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR1_TWED_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR1_XNX_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_HIGHER_SAFE, ID_AA64MMFR1_SPECSEI_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR1_PAN_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR1_LOR_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR1_HPD_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR1_VHE_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR1_VMIDBITS_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR1_HADBS_SHIFT, 4, 0),
> + ARM64_FTR_END,
> +};
> +
> +static const struct arm64_ftr_bits ftr_id_aa64mmfr2[] = {
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_E0PD_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_EVT_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_BBM_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_TTL_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_FWB_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_IDS_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_AT_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_ST_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_NV_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_CCIDX_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_LVA_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_IESB_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_LSM_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_UAO_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_CNP_SHIFT, 4, 0),
> + ARM64_FTR_END,
> +};
> +
> +static const struct arm64_ftr_bits ftr_ctr[] = {
> + ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_EXACT, 31, 1, 1), /* RES1 */
> + ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, CTR_DIC_SHIFT, 1, 1),
> + ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, CTR_IDC_SHIFT, 1, 1),
> + ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_HIGHER_OR_ZERO_SAFE, CTR_CWG_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_HIGHER_OR_ZERO_SAFE, CTR_ERG_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, CTR_DMINLINE_SHIFT, 4, 1),
> + /*
> + * Linux can handle differing I-cache policies. Userspace JITs will
> + * make use of *minLine.
> + * If we have differing I-cache policies, report it as the weakest - VIPT.
> + */
> + ARM64_FTR_BITS(FTR_VISIBLE, FTR_NONSTRICT, FTR_EXACT, CTR_L1IP_SHIFT, 2, ICACHE_POLICY_VIPT), /* L1Ip */
> + ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, CTR_IMINLINE_SHIFT, 4, 0),
> + ARM64_FTR_END,
> +};
> +
> +static const struct arm64_ftr_bits ftr_id_mmfr0[] = {
> + S_ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR0_INNERSHR_SHIFT, 4, 0xf),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR0_FCSE_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_MMFR0_AUXREG_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR0_TCM_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR0_SHARELVL_SHIFT, 4, 0),
> + S_ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR0_OUTERSHR_SHIFT, 4, 0xf),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR0_PMSA_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR0_VMSA_SHIFT, 4, 0),
> + ARM64_FTR_END,
> +};
> +
> +static const struct arm64_ftr_bits ftr_id_aa64dfr0[] = {
> + S_ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64DFR0_DOUBLELOCK_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64DFR0_PMSVER_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64DFR0_CTX_CMPS_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64DFR0_WRPS_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64DFR0_BRPS_SHIFT, 4, 0),
> + /*
> + * We can instantiate multiple PMU instances with different levels
> + * of support.
> + */
> + S_ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_EXACT, ID_AA64DFR0_PMUVER_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_EXACT, ID_AA64DFR0_DEBUGVER_SHIFT, 4, 0x6),
> + ARM64_FTR_END,
> +};
> +
> +static const struct arm64_ftr_bits ftr_mvfr2[] = {
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, MVFR2_FPMISC_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, MVFR2_SIMDMISC_SHIFT, 4, 0),
> + ARM64_FTR_END,
> +};
> +
> +static const struct arm64_ftr_bits ftr_dczid[] = {
> + ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_EXACT, DCZID_DZP_SHIFT, 1, 1),
> + ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, DCZID_BS_SHIFT, 4, 0),
> + ARM64_FTR_END,
> +};
> +
> +static const struct arm64_ftr_bits ftr_id_isar0[] = {
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR0_DIVIDE_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR0_DEBUG_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR0_COPROC_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR0_CMPBRANCH_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR0_BITFIELD_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR0_BITCOUNT_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR0_SWAP_SHIFT, 4, 0),
> + ARM64_FTR_END,
> +};
> +
> +static const struct arm64_ftr_bits ftr_id_isar5[] = {
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR5_RDM_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR5_CRC32_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR5_SHA2_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR5_SHA1_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR5_AES_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR5_SEVL_SHIFT, 4, 0),
> + ARM64_FTR_END,
> +};
> +
> +static const struct arm64_ftr_bits ftr_id_mmfr4[] = {
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR4_EVT_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR4_CCIDX_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR4_LSM_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR4_HPDS_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR4_CNP_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR4_XNX_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR4_AC2_SHIFT, 4, 0),
> +
> + /*
> + * SpecSEI = 1 indicates that the PE might generate an SError on an
> + * external abort on speculative read. It is safe to assume that an
> + * SError might be generated than it will not be. Hence it has been
> + * classified as FTR_HIGHER_SAFE.
> + */
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_HIGHER_SAFE, ID_MMFR4_SPECSEI_SHIFT, 4, 0),
> + ARM64_FTR_END,
> +};
> +
> +static const struct arm64_ftr_bits ftr_id_isar4[] = {
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR4_SWP_FRAC_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR4_PSR_M_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR4_SYNCH_PRIM_FRAC_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR4_BARRIER_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR4_SMC_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR4_WRITEBACK_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR4_WITHSHIFTS_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR4_UNPRIV_SHIFT, 4, 0),
> + ARM64_FTR_END,
> +};
> +
> +static const struct arm64_ftr_bits ftr_id_mmfr5[] = {
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR5_ETS_SHIFT, 4, 0),
> + ARM64_FTR_END,
> +};
> +
> +static const struct arm64_ftr_bits ftr_id_isar6[] = {
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR6_I8MM_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR6_BF16_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR6_SPECRES_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR6_SB_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR6_FHM_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR6_DP_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR6_JSCVT_SHIFT, 4, 0),
> + ARM64_FTR_END,
> +};
> +
> +static const struct arm64_ftr_bits ftr_id_pfr0[] = {
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_PFR0_DIT_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_PFR0_CSV2_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_PFR0_STATE3_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_PFR0_STATE2_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_PFR0_STATE1_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_PFR0_STATE0_SHIFT, 4, 0),
> + ARM64_FTR_END,
> +};
> +
> +static const struct arm64_ftr_bits ftr_id_pfr1[] = {
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_PFR1_GIC_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_PFR1_VIRT_FRAC_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_PFR1_SEC_FRAC_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_PFR1_GENTIMER_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_PFR1_VIRTUALIZATION_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_PFR1_MPROGMOD_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_PFR1_SECURITY_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_PFR1_PROGMOD_SHIFT, 4, 0),
> + ARM64_FTR_END,
> +};
> +
> +static const struct arm64_ftr_bits ftr_id_pfr2[] = {
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_PFR2_SSBS_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_PFR2_CSV3_SHIFT, 4, 0),
> + ARM64_FTR_END,
> +};
> +
> +static const struct arm64_ftr_bits ftr_id_dfr0[] = {
> + /* [31:28] TraceFilt */
> + S_ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_DFR0_PERFMON_SHIFT, 4, 0xf),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_DFR0_MPROFDBG_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_DFR0_MMAPTRC_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_DFR0_COPTRC_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_DFR0_MMAPDBG_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_DFR0_COPSDBG_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_DFR0_COPDBG_SHIFT, 4, 0),
> + ARM64_FTR_END,
> +};
> +
> +static const struct arm64_ftr_bits ftr_id_dfr1[] = {
> + S_ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_DFR1_MTPMU_SHIFT, 4, 0),
> + ARM64_FTR_END,
> +};
> +
> +static const struct arm64_ftr_bits ftr_zcr[] = {
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE,
> + ZCR_ELx_LEN_SHIFT, ZCR_ELx_LEN_SIZE, 0), /* LEN */
> + ARM64_FTR_END,
> +};
> +
> +/*
> + * Common ftr bits for a 32bit register with all hidden, strict
> + * attributes, with 4bit feature fields and a default safe value of
> + * 0. Covers the following 32bit registers:
> + * id_isar[1-4], id_mmfr[1-3], id_pfr1, mvfr[0-1]
> + */
> +static const struct arm64_ftr_bits ftr_generic_32bits[] = {
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, 28, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, 24, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, 20, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, 16, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, 12, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, 8, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, 4, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, 0, 4, 0),
> + ARM64_FTR_END,
> +};
> +
> +static const struct arm64_ftr_bits ftr_raz[] = {
> + ARM64_FTR_END,
> +};
> +
> +static s64 arm64_ftr_safe_value(const struct arm64_ftr_bits *ftrp, s64 new,
> + s64 cur)
> +{
> + s64 ret = 0;
> +
> + switch (ftrp->type) {
> + case FTR_EXACT:
> + ret = ftrp->safe_val;
> + break;
> + case FTR_LOWER_SAFE:
> + ret = min(new, cur);
> + break;
> + case FTR_HIGHER_OR_ZERO_SAFE:
> + if (!cur || !new)
> + break;
> + fallthrough;
> + case FTR_HIGHER_SAFE:
> + ret = max(new, cur);
> + break;
> + default:
> + BUG();
> + }
> +
> + return ret;
> +}
> +
> +/*
> + * End of imported linux structures and code
> + */
> +
> diff --git a/xen/include/asm-arm/arm64/cpufeature.h b/xen/include/asm-arm/arm64/cpufeature.h
> new file mode 100644
> index 0000000000..d9b9fa77cb
> --- /dev/null
> +++ b/xen/include/asm-arm/arm64/cpufeature.h
> @@ -0,0 +1,104 @@
> +#ifndef __ASM_ARM_ARM64_CPUFEATURES_H
> +#define __ASM_ARM_ARM64_CPUFEATURES_H
> +
> +/*
> + * CPU feature register tracking
> + *
> + * The safe value of a CPUID feature field is dependent on the implications
> + * of the values assigned to it by the architecture. Based on the relationship
> + * between the values, the features are classified into 3 types - LOWER_SAFE,
> + * HIGHER_SAFE and EXACT.
> + *
> + * The lowest value of all the CPUs is chosen for LOWER_SAFE and highest
> + * for HIGHER_SAFE. It is expected that all CPUs have the same value for
> + * a field when EXACT is specified, failing which, the safe value specified
> + * in the table is chosen.
> + */
> +
> +enum ftr_type {
> + FTR_EXACT, /* Use a predefined safe value */
> + FTR_LOWER_SAFE, /* Smaller value is safe */
> + FTR_HIGHER_SAFE, /* Bigger value is safe */
> + FTR_HIGHER_OR_ZERO_SAFE, /* Bigger value is safe, but 0 is biggest */
> +};
> +
> +#define FTR_STRICT true /* SANITY check strict matching required */
> +#define FTR_NONSTRICT false /* SANITY check ignored */
> +
> +#define FTR_SIGNED true /* Value should be treated as signed */
> +#define FTR_UNSIGNED false /* Value should be treated as unsigned */
> +
> +#define FTR_VISIBLE true /* Feature visible to the user space */
> +#define FTR_HIDDEN false /* Feature is hidden from the user */
> +
> +#define FTR_VISIBLE_IF_IS_ENABLED(config) \
> + (IS_ENABLED(config) ? FTR_VISIBLE : FTR_HIDDEN)
> +
> +struct arm64_ftr_bits {
> + bool sign; /* Value is signed ? */
> + bool visible;
> + bool strict; /* CPU Sanity check: strict matching required ? */
> + enum ftr_type type;
> + u8 shift;
> + u8 width;
> + s64 safe_val; /* safe value for FTR_EXACT features */
> +};
> +
> +static inline int __attribute_const__
> +cpuid_feature_extract_signed_field_width(u64 features, int field, int width)
> +{
> + return (s64)(features << (64 - width - field)) >> (64 - width);
> +}
> +
> +static inline int __attribute_const__
> +cpuid_feature_extract_signed_field(u64 features, int field)
> +{
> + return cpuid_feature_extract_signed_field_width(features, field, 4);
> +}
> +
> +static inline unsigned int __attribute_const__
> +cpuid_feature_extract_unsigned_field_width(u64 features, int field, int width)
> +{
> + return (u64)(features << (64 - width - field)) >> (64 - width);
> +}
> +
> +static inline unsigned int __attribute_const__
> +cpuid_feature_extract_unsigned_field(u64 features, int field)
> +{
> + return cpuid_feature_extract_unsigned_field_width(features, field, 4);
> +}
> +
> +static inline u64 arm64_ftr_mask(const struct arm64_ftr_bits *ftrp)
> +{
> + return (u64)GENMASK(ftrp->shift + ftrp->width - 1, ftrp->shift);
> +}
> +
> +static inline int __attribute_const__
> +cpuid_feature_extract_field_width(u64 features, int field, int width, bool sign)
> +{
> + return (sign) ?
> + cpuid_feature_extract_signed_field_width(features, field, width) :
> + cpuid_feature_extract_unsigned_field_width(features, field, width);
> +}
> +
> +static inline int __attribute_const__
> +cpuid_feature_extract_field(u64 features, int field, bool sign)
> +{
> + return cpuid_feature_extract_field_width(features, field, 4, sign);
> +}
> +
> +static inline s64 arm64_ftr_value(const struct arm64_ftr_bits *ftrp, u64 val)
> +{
> + return (s64)cpuid_feature_extract_field_width(val, ftrp->shift, ftrp->width, ftrp->sign);
> +}
> +
> +#endif /* _ASM_ARM_ARM64_CPUFEATURES_H */
> +
> +/*
> + * Local variables:
> + * mode: C
> + * c-file-style: "BSD"
> + * c-basic-offset: 4
> + * indent-tabs-mode: nil
> + * End:
> + */
> --
> 2.17.1
>
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [PATCH v3 3/7] xen/arm: Rename cpu_boot_data to system_cpuinfo
2021-08-25 13:18 ` [PATCH v3 3/7] xen/arm: Rename cpu_boot_data to system_cpuinfo Bertrand Marquis
@ 2021-09-03 22:48 ` Stefano Stabellini
0 siblings, 0 replies; 26+ messages in thread
From: Stefano Stabellini @ 2021-09-03 22:48 UTC (permalink / raw)
To: Bertrand Marquis
Cc: xen-devel, Stefano Stabellini, Julien Grall, Volodymyr Babchuk
On Wed, 25 Aug 2021, Bertrand Marquis wrote:
> As we will sanitize the content of boot_cpu_data it will not really
> contain the boot cpu information but the system sanitize information.
> Rename the structure to system_cpuinfo so the user is informed that this
> is the system wide available feature and not anymore the features of the
> boot cpu.
> The original boot cpu data is still available in cpu_data.
>
> Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
> ---
> Changes in v3: none
> Changes in v2:
> - patch introduced in v2
> ---
> xen/arch/arm/cpufeature.c | 8 ++------
> xen/arch/arm/setup.c | 34 ++++++++++++++++++--------------
> xen/arch/arm/smpboot.c | 6 +++---
> xen/include/asm-arm/cpufeature.h | 6 +++---
> 4 files changed, 27 insertions(+), 27 deletions(-)
>
> diff --git a/xen/arch/arm/cpufeature.c b/xen/arch/arm/cpufeature.c
> index 1d88783809..f600a611bd 100644
> --- a/xen/arch/arm/cpufeature.c
> +++ b/xen/arch/arm/cpufeature.c
> @@ -169,12 +169,8 @@ void identify_cpu(struct cpuinfo_arm *c)
> */
> static int __init create_guest_cpuinfo(void)
> {
> - /*
> - * TODO: The code is currently using only the features detected on the boot
> - * core. In the long term we should try to compute values containing only
> - * features supported by all cores.
> - */
> - guest_cpuinfo = boot_cpu_data;
> + /* Use the sanitized cpuinfo as initial guest cpuinfo */
> + guest_cpuinfo = system_cpuinfo;
>
> #ifdef CONFIG_ARM_64
> /* Hide MPAM support as xen does not support it */
> diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
> index 63a908e325..3798c5ade0 100644
> --- a/xen/arch/arm/setup.c
> +++ b/xen/arch/arm/setup.c
> @@ -56,7 +56,11 @@
>
> struct bootinfo __initdata bootinfo;
>
> -struct cpuinfo_arm __read_mostly boot_cpu_data;
> +/*
> + * Sanitized version of cpuinfo containing only features available on all
> + * cores (only on arm64 as there is no sanitization support on arm32).
> + */
> +struct cpuinfo_arm __read_mostly system_cpuinfo;
>
> #ifdef CONFIG_ACPI
> bool __read_mostly acpi_disabled;
> @@ -100,7 +104,7 @@ static const char * __initdata processor_implementers[] = {
> static void __init processor_id(void)
> {
> const char *implementer = "Unknown";
> - struct cpuinfo_arm *c = &boot_cpu_data;
> + struct cpuinfo_arm *c = &system_cpuinfo;
>
> identify_cpu(c);
> current_cpu_data = *c;
> @@ -120,7 +124,7 @@ static void __init processor_id(void)
> #if defined(CONFIG_ARM_64)
> printk("64-bit Execution:\n");
> printk(" Processor Features: %016"PRIx64" %016"PRIx64"\n",
> - boot_cpu_data.pfr64.bits[0], boot_cpu_data.pfr64.bits[1]);
> + system_cpuinfo.pfr64.bits[0], system_cpuinfo.pfr64.bits[1]);
> printk(" Exception Levels: EL3:%s EL2:%s EL1:%s EL0:%s\n",
> cpu_has_el3_32 ? "64+32" : cpu_has_el3_64 ? "64" : "No",
> cpu_has_el2_32 ? "64+32" : cpu_has_el2_64 ? "64" : "No",
> @@ -144,13 +148,13 @@ static void __init processor_id(void)
> boot_cpu_feature64(simd));
>
> printk(" Debug Features: %016"PRIx64" %016"PRIx64"\n",
> - boot_cpu_data.dbg64.bits[0], boot_cpu_data.dbg64.bits[1]);
> + system_cpuinfo.dbg64.bits[0], system_cpuinfo.dbg64.bits[1]);
> printk(" Auxiliary Features: %016"PRIx64" %016"PRIx64"\n",
> - boot_cpu_data.aux64.bits[0], boot_cpu_data.aux64.bits[1]);
> + system_cpuinfo.aux64.bits[0], system_cpuinfo.aux64.bits[1]);
> printk(" Memory Model Features: %016"PRIx64" %016"PRIx64"\n",
> - boot_cpu_data.mm64.bits[0], boot_cpu_data.mm64.bits[1]);
> + system_cpuinfo.mm64.bits[0], system_cpuinfo.mm64.bits[1]);
> printk(" ISA Features: %016"PRIx64" %016"PRIx64"\n",
> - boot_cpu_data.isa64.bits[0], boot_cpu_data.isa64.bits[1]);
> + system_cpuinfo.isa64.bits[0], system_cpuinfo.isa64.bits[1]);
> #endif
>
> /*
> @@ -161,7 +165,7 @@ static void __init processor_id(void)
> {
> printk("32-bit Execution:\n");
> printk(" Processor Features: %"PRIregister":%"PRIregister"\n",
> - boot_cpu_data.pfr32.bits[0], boot_cpu_data.pfr32.bits[1]);
> + system_cpuinfo.pfr32.bits[0], system_cpuinfo.pfr32.bits[1]);
> printk(" Instruction Sets:%s%s%s%s%s%s\n",
> cpu_has_aarch32 ? " AArch32" : "",
> cpu_has_arm ? " A32" : "",
> @@ -174,18 +178,18 @@ static void __init processor_id(void)
> cpu_has_security ? " Security" : "");
>
> printk(" Debug Features: %"PRIregister"\n",
> - boot_cpu_data.dbg32.bits[0]);
> + system_cpuinfo.dbg32.bits[0]);
> printk(" Auxiliary Features: %"PRIregister"\n",
> - boot_cpu_data.aux32.bits[0]);
> + system_cpuinfo.aux32.bits[0]);
> printk(" Memory Model Features: %"PRIregister" %"PRIregister"\n"
> " %"PRIregister" %"PRIregister"\n",
> - boot_cpu_data.mm32.bits[0], boot_cpu_data.mm32.bits[1],
> - boot_cpu_data.mm32.bits[2], boot_cpu_data.mm32.bits[3]);
> + system_cpuinfo.mm32.bits[0], system_cpuinfo.mm32.bits[1],
> + system_cpuinfo.mm32.bits[2], system_cpuinfo.mm32.bits[3]);
> printk(" ISA Features: %"PRIregister" %"PRIregister" %"PRIregister"\n"
> " %"PRIregister" %"PRIregister" %"PRIregister"\n",
> - boot_cpu_data.isa32.bits[0], boot_cpu_data.isa32.bits[1],
> - boot_cpu_data.isa32.bits[2], boot_cpu_data.isa32.bits[3],
> - boot_cpu_data.isa32.bits[4], boot_cpu_data.isa32.bits[5]);
> + system_cpuinfo.isa32.bits[0], system_cpuinfo.isa32.bits[1],
> + system_cpuinfo.isa32.bits[2], system_cpuinfo.isa32.bits[3],
> + system_cpuinfo.isa32.bits[4], system_cpuinfo.isa32.bits[5]);
> }
> else
> {
> diff --git a/xen/arch/arm/smpboot.c b/xen/arch/arm/smpboot.c
> index a1ee3146ef..c9f2827d56 100644
> --- a/xen/arch/arm/smpboot.c
> +++ b/xen/arch/arm/smpboot.c
> @@ -124,7 +124,7 @@ static void __init dt_smp_init_cpus(void)
> bool bootcpu_valid = false;
> int rc;
>
> - mpidr = boot_cpu_data.mpidr.bits & MPIDR_HWID_MASK;
> + mpidr = system_cpuinfo.mpidr.bits & MPIDR_HWID_MASK;
>
> if ( !cpus )
> {
> @@ -319,13 +319,13 @@ void start_secondary(void)
> * now.
> */
> if ( !opt_hmp_unsafe &&
> - current_cpu_data.midr.bits != boot_cpu_data.midr.bits )
> + current_cpu_data.midr.bits != system_cpuinfo.midr.bits )
> {
> printk(XENLOG_ERR
> "CPU%u MIDR (0x%"PRIregister") does not match boot CPU MIDR (0x%"PRIregister"),\n"
> XENLOG_ERR "disable cpu (see big.LITTLE.txt under docs/).\n",
> smp_processor_id(), current_cpu_data.midr.bits,
> - boot_cpu_data.midr.bits);
> + system_cpuinfo.midr.bits);
> stop_cpu();
> }
>
> diff --git a/xen/include/asm-arm/cpufeature.h b/xen/include/asm-arm/cpufeature.h
> index ba48db3eac..8f2b8e7830 100644
> --- a/xen/include/asm-arm/cpufeature.h
> +++ b/xen/include/asm-arm/cpufeature.h
> @@ -3,7 +3,7 @@
>
> #ifdef CONFIG_ARM_64
> #define cpu_feature64(c, feat) ((c)->pfr64.feat)
> -#define boot_cpu_feature64(feat) (boot_cpu_data.pfr64.feat)
> +#define boot_cpu_feature64(feat) (system_cpuinfo.pfr64.feat)
>
> #define cpu_feature64_has_el0_32(c) (cpu_feature64(c, el0) == 2)
>
> @@ -21,7 +21,7 @@
> #endif
>
> #define cpu_feature32(c, feat) ((c)->pfr32.feat)
> -#define boot_cpu_feature32(feat) (boot_cpu_data.pfr32.feat)
> +#define boot_cpu_feature32(feat) (system_cpuinfo.pfr32.feat)
>
> #define cpu_has_arm (boot_cpu_feature32(arm) == 1)
> #define cpu_has_thumb (boot_cpu_feature32(thumb) >= 1)
> @@ -326,7 +326,7 @@ struct cpuinfo_arm {
> } mvfr;
> };
>
> -extern struct cpuinfo_arm boot_cpu_data;
> +extern struct cpuinfo_arm system_cpuinfo;
>
> extern void identify_cpu(struct cpuinfo_arm *);
>
> --
> 2.17.1
>
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [PATCH v3 4/7] xen/arm: Sanitize cpuinfo ID registers fields
2021-08-25 13:18 ` [PATCH v3 4/7] xen/arm: Sanitize cpuinfo ID registers fields Bertrand Marquis
@ 2021-09-03 22:48 ` Stefano Stabellini
2021-09-06 8:24 ` Bertrand Marquis
0 siblings, 1 reply; 26+ messages in thread
From: Stefano Stabellini @ 2021-09-03 22:48 UTC (permalink / raw)
To: Bertrand Marquis
Cc: xen-devel, Stefano Stabellini, Julien Grall, Volodymyr Babchuk,
Andrew Cooper, George Dunlap, Ian Jackson, Jan Beulich, Wei Liu
On Wed, 25 Aug 2021, Bertrand Marquis wrote:
> Define a sanitize_cpu function to be called on secondary cores to
> sanitize the system cpuinfo structure.
>
> The safest value is taken when possible and the system is marked tainted
> if we encounter values which are incompatible with each other.
>
> Call the update_system_features function on all secondary cores that are
> kept running and taint the system if different midr are found between
> cores but hmp-unsafe=true was passed on Xen command line.
>
> This is only supported on arm64 so update_system_features is an empty
> static inline on arm32.
>
> The patch is adding a new TAINT_CPU_OUT_OF_SPEC to warn the user if
> Xen is running on a system with features differences between cores which
> are not supported.
>
> The patch is disabling CTR_EL0, DCZID_EL0 and ZCRusing #if 0 with a TODO
> as this patch is not handling sanitization of those registers.
> CTR_EL0/DCZID will be handled in a future patch to properly handle
> different cache attributes when possible.
> ZCR should be sanitize once we add support for SVE in Xen.
>
> Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
Great patch! It looks good I only have a couple of minor questions
below.
> ---
> Changes in v3:
> - in case of different midr but hmp-unsafe passed on the command line,
> enable all cores anyway but taint Xen with CPU_OUT_OF_SPEC.
> - use current core info to sanitize cpu only if we keep it on
> Changes in v2:
> - add compilation of cpufeature.c in this patch instead of previous one
> - remove functions reused from linux code and moved to header
> - rename sanitize_cpu to update_system_features
> - change to Linux coding style
> - remove dev comments
> - surround currently not used Linux structures with #if 0 and adapt the
> commit message
> - add missing aa64dfr1 register
> - add TODO for CTR, DCZID and ZCR
> - add CPU_OUT_OF_SPEC support to print_taint
> - use system_cpuinfo instead of boot_cpu_data
> ---
> xen/arch/arm/arm64/Makefile | 1 +
> xen/arch/arm/arm64/cpufeature.c | 121 +++++++++++++++++++++++++++++++
> xen/arch/arm/smpboot.c | 34 +++++++--
> xen/common/kernel.c | 6 +-
> xen/include/asm-arm/cpufeature.h | 9 +++
> xen/include/xen/lib.h | 1 +
> 6 files changed, 162 insertions(+), 10 deletions(-)
>
> diff --git a/xen/arch/arm/arm64/Makefile b/xen/arch/arm/arm64/Makefile
> index 40642ff574..701d66883d 100644
> --- a/xen/arch/arm/arm64/Makefile
> +++ b/xen/arch/arm/arm64/Makefile
> @@ -1,6 +1,7 @@
> obj-y += lib/
>
> obj-y += cache.o
> +obj-y += cpufeature.o
> obj-$(CONFIG_HARDEN_BRANCH_PREDICTOR) += bpi.o
> obj-$(CONFIG_EARLY_PRINTK) += debug.o
> obj-y += domctl.o
> diff --git a/xen/arch/arm/arm64/cpufeature.c b/xen/arch/arm/arm64/cpufeature.c
> index 5777e33e5c..61f629ebaa 100644
> --- a/xen/arch/arm/arm64/cpufeature.c
> +++ b/xen/arch/arm/arm64/cpufeature.c
> @@ -275,6 +275,9 @@ static const struct arm64_ftr_bits ftr_id_aa64mmfr2[] = {
> ARM64_FTR_END,
> };
>
> +#if 0
> +/* TODO: use this to sanitize the cache line size among cores */
> +
> static const struct arm64_ftr_bits ftr_ctr[] = {
> ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_EXACT, 31, 1, 1), /* RES1 */
> ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, CTR_DIC_SHIFT, 1, 1),
> @@ -291,6 +294,7 @@ static const struct arm64_ftr_bits ftr_ctr[] = {
> ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, CTR_IMINLINE_SHIFT, 4, 0),
> ARM64_FTR_END,
> };
> +#endif
>
> static const struct arm64_ftr_bits ftr_id_mmfr0[] = {
> S_ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR0_INNERSHR_SHIFT, 4, 0xf),
> @@ -325,11 +329,14 @@ static const struct arm64_ftr_bits ftr_mvfr2[] = {
> ARM64_FTR_END,
> };
>
> +#if 0
> +/* TODO: handle this when sanitizing cache related registers */
> static const struct arm64_ftr_bits ftr_dczid[] = {
> ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_EXACT, DCZID_DZP_SHIFT, 1, 1),
> ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, DCZID_BS_SHIFT, 4, 0),
> ARM64_FTR_END,
> };
> +#endif
>
> static const struct arm64_ftr_bits ftr_id_isar0[] = {
> ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR0_DIVIDE_SHIFT, 4, 0),
> @@ -444,11 +451,15 @@ static const struct arm64_ftr_bits ftr_id_dfr1[] = {
> ARM64_FTR_END,
> };
>
> +#if 0
> +/* TODO: use this to sanitize SVE once we support it */
> +
> static const struct arm64_ftr_bits ftr_zcr[] = {
> ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE,
> ZCR_ELx_LEN_SHIFT, ZCR_ELx_LEN_SIZE, 0), /* LEN */
> ARM64_FTR_END,
> };
> +#endif
>
> /*
> * Common ftr bits for a 32bit register with all hidden, strict
> @@ -502,3 +513,113 @@ static s64 arm64_ftr_safe_value(const struct arm64_ftr_bits *ftrp, s64 new,
> * End of imported linux structures and code
> */
>
> +static void sanitize_reg(u64 *cur_reg, u64 new_reg, const char *reg_name,
> + const struct arm64_ftr_bits *ftrp)
> +{
> + int taint = 0;
> + u64 old_reg = *cur_reg;
> +
> + for (;ftrp->width != 0;ftrp++)
> + {
> + u64 mask;
> + s64 cur_field = arm64_ftr_value(ftrp, *cur_reg);
> + s64 new_field = arm64_ftr_value(ftrp, new_reg);
> +
> + if (cur_field == new_field)
> + continue;
> +
> + if (ftrp->strict)
> + taint = 1;
> +
> + mask = arm64_ftr_mask(ftrp);
> +
> + *cur_reg &= ~mask;
> + *cur_reg |= (arm64_ftr_safe_value(ftrp, new_field, cur_field)
> + << ftrp->shift) & mask;
I wonder why you haven't also imported arm64_ftr_set_value? This seems
to be the open-coded version of it.
> + }
> +
> + if (old_reg != new_reg)
> + printk(XENLOG_DEBUG "SANITY DIF: %s 0x%"PRIx64" -> 0x%"PRIx64"\n",
> + reg_name, old_reg, new_reg);
> + if (old_reg != *cur_reg)
> + printk(XENLOG_DEBUG "SANITY FIX: %s 0x%"PRIx64" -> 0x%"PRIx64"\n",
> + reg_name, old_reg, *cur_reg);
> +
> + if (taint)
> + {
> + printk(XENLOG_WARNING "SANITY CHECK: Unexpected variation in %s.\n",
> + reg_name);
> + add_taint(TAINT_CPU_OUT_OF_SPEC);
> + }
> +}
> +
> +
> +/*
> + * This function should be called on secondary cores to sanitize the boot cpu
> + * cpuinfo.
> + */
> +void update_system_features(const struct cpuinfo_arm *new)
> +{
> +
> +#define SANITIZE_REG(field, num, reg) \
> + sanitize_reg(&system_cpuinfo.field.bits[num], new->field.bits[num], \
> + #reg, ftr_##reg)
> +
> +#define SANITIZE_ID_REG(field, num, reg) \
> + sanitize_reg(&system_cpuinfo.field.bits[num], new->field.bits[num], \
> + #reg, ftr_id_##reg)
> +
> +#define SANITIZE_RAZ_REG(field, num, reg) \
> + sanitize_reg(&system_cpuinfo.field.bits[num], new->field.bits[num], \
> + #reg, ftr_raz)
> +
> +#define SANITIZE_GENERIC_REG(field, num, reg) \
> + sanitize_reg(&system_cpuinfo.field.bits[num], new->field.bits[num], \
> + #reg, ftr_generic_32bits)
> +
> + SANITIZE_ID_REG(pfr64, 0, aa64pfr0);
> + SANITIZE_ID_REG(pfr64, 1, aa64pfr1);
> +
> + SANITIZE_ID_REG(dbg64, 0, aa64dfr0);
> + SANITIZE_RAZ_REG(dbg64, 1, aa64dfr1);
> +
> + SANITIZE_ID_REG(mm64, 0, aa64mmfr0);
> + SANITIZE_ID_REG(mm64, 1, aa64mmfr1);
> + SANITIZE_ID_REG(mm64, 2, aa64mmfr2);
> +
> + SANITIZE_ID_REG(isa64, 0, aa64isar0);
> + SANITIZE_ID_REG(isa64, 1, aa64isar1);
> +
> + SANITIZE_ID_REG(zfr64, 0, aa64zfr0);
> +
> + if ( cpu_feature64_has_el0_32(&system_cpuinfo) )
> + {
> + SANITIZE_ID_REG(pfr32, 0, pfr0);
> + SANITIZE_ID_REG(pfr32, 1, pfr1);
> + SANITIZE_ID_REG(pfr32, 2, pfr2);
> +
> + SANITIZE_ID_REG(dbg32, 0, dfr0);
> + SANITIZE_ID_REG(dbg32, 1, dfr1);
> +
> + SANITIZE_ID_REG(mm32, 0, mmfr0);
> + SANITIZE_GENERIC_REG(mm32, 1, mmfr1);
> + SANITIZE_GENERIC_REG(mm32, 2, mmfr2);
> + SANITIZE_GENERIC_REG(mm32, 3, mmfr3);
> + SANITIZE_ID_REG(mm32, 4, mmfr4);
> + SANITIZE_ID_REG(mm32, 5, mmfr5);
> +
> + SANITIZE_ID_REG(isa32, 0, isar0);
> + SANITIZE_GENERIC_REG(isa32, 1, isar1);
> + SANITIZE_GENERIC_REG(isa32, 2, isar2);
> + SANITIZE_GENERIC_REG(isa32, 3, isar3);
> + SANITIZE_ID_REG(isa32, 4, isar4);
> + SANITIZE_ID_REG(isa32, 5, isar5);
> + SANITIZE_ID_REG(isa32, 6, isar6);
> +
> + SANITIZE_GENERIC_REG(mvfr, 0, mvfr0);
> + SANITIZE_GENERIC_REG(mvfr, 1, mvfr1);
> +#ifndef MVFR2_MAYBE_UNDEFINED
> + SANITIZE_REG(mvfr, 2, mvfr2);
> +#endif
> + }
> +}
Looking at the list of registers we are sanitizing here we are only
missing aux32 and aux64 compared to struct cpuinfo_arm. Is that because
there is nothing to sanitize there?
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [PATCH v3 5/7] xen/arm: Use sanitize values for p2m
2021-08-25 13:18 ` [PATCH v3 5/7] xen/arm: Use sanitize values for p2m Bertrand Marquis
@ 2021-09-03 22:48 ` Stefano Stabellini
0 siblings, 0 replies; 26+ messages in thread
From: Stefano Stabellini @ 2021-09-03 22:48 UTC (permalink / raw)
To: Bertrand Marquis
Cc: xen-devel, Stefano Stabellini, Julien Grall, Volodymyr Babchuk
On Wed, 25 Aug 2021, Bertrand Marquis wrote:
> Replace the code in p2m trying to find a sane value for the VMID size
> supported and the PAR to use. We are now using the boot cpuinfo as the
> values there are sanitized during boot and the value for those
> parameters is now the safest possible value on the system.
>
> Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
> ---
> Changes in v3: drop arm32 mention in commmit message
> Changes in v2:
> - use system_cpuinfo
> ---
> xen/arch/arm/p2m.c | 30 ++++++++++--------------------
> 1 file changed, 10 insertions(+), 20 deletions(-)
>
> diff --git a/xen/arch/arm/p2m.c b/xen/arch/arm/p2m.c
> index eff9a105e7..41b6430c30 100644
> --- a/xen/arch/arm/p2m.c
> +++ b/xen/arch/arm/p2m.c
> @@ -2045,31 +2045,21 @@ void __init setup_virt_paging(void)
> [7] = { 0 } /* Invalid */
> };
>
> - unsigned int i, cpu;
> + unsigned int i;
> unsigned int pa_range = 0x10; /* Larger than any possible value */
> - bool vmid_8_bit = false;
> -
> - for_each_online_cpu ( cpu )
> - {
> - const struct cpuinfo_arm *info = &cpu_data[cpu];
>
> - /*
> - * Restrict "p2m_ipa_bits" if needed. As P2M table is always configured
> - * with IPA bits == PA bits, compare against "pabits".
> - */
> - if ( pa_range_info[info->mm64.pa_range].pabits < p2m_ipa_bits )
> - p2m_ipa_bits = pa_range_info[info->mm64.pa_range].pabits;
> -
> - /* Set a flag if the current cpu does not support 16 bit VMIDs. */
> - if ( info->mm64.vmid_bits != MM64_VMID_16_BITS_SUPPORT )
> - vmid_8_bit = true;
> - }
> + /*
> + * Restrict "p2m_ipa_bits" if needed. As P2M table is always configured
> + * with IPA bits == PA bits, compare against "pabits".
> + */
> + if ( pa_range_info[system_cpuinfo.mm64.pa_range].pabits < p2m_ipa_bits )
> + p2m_ipa_bits = pa_range_info[system_cpuinfo.mm64.pa_range].pabits;
>
> /*
> - * If the flag is not set then it means all CPUs support 16-bit
> - * VMIDs.
> + * cpu info sanitization made sure we support 16bits VMID only if all
> + * cores are supporting it.
> */
> - if ( !vmid_8_bit )
> + if ( system_cpuinfo.mm64.vmid_bits == MM64_VMID_16_BITS_SUPPORT )
> max_vmid = MAX_VMID_16_BIT;
>
> /* Choose suitable "pa_range" according to the resulted "p2m_ipa_bits". */
> --
> 2.17.1
>
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [PATCH v3 6/7] xen/arm: Taint Xen on incompatible DCZID values
2021-08-25 13:18 ` [PATCH v3 6/7] xen/arm: Taint Xen on incompatible DCZID values Bertrand Marquis
@ 2021-09-03 22:49 ` Stefano Stabellini
2021-09-06 7:59 ` Bertrand Marquis
0 siblings, 1 reply; 26+ messages in thread
From: Stefano Stabellini @ 2021-09-03 22:49 UTC (permalink / raw)
To: Bertrand Marquis
Cc: xen-devel, Stefano Stabellini, Julien Grall, Volodymyr Babchuk
On Wed, 25 Aug 2021, Bertrand Marquis wrote:
> Use arm64 cpu feature sanitization to TAIN Xen if different DCZID values
^ TAINT
> are found (ftr_dczid is using only STRICT method).
> In this case actual memory being cleaned by DC ZVA operations would be
> different depending on the cores which could make a guest zeroing too
> much or too little memory if it is merged between CPUs.
>
> We could, on processor supporting it, trap access to DCZID_EL0 register
^ processors
> using HFGRTR_EL2 register but this would not solve the case where a
> process is being migrated during a copy or if it cached the value of the
> register.
>
> Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
> ---
> Change in v3: none
> Change in v2: Patch introduced in v2
> ---
> xen/arch/arm/arm64/cpufeature.c | 14 +++++++++++---
> xen/arch/arm/cpufeature.c | 2 ++
> xen/include/asm-arm/cpufeature.h | 8 ++++++++
> 3 files changed, 21 insertions(+), 3 deletions(-)
>
> diff --git a/xen/arch/arm/arm64/cpufeature.c b/xen/arch/arm/arm64/cpufeature.c
> index 61f629ebaa..b1936ef1d6 100644
> --- a/xen/arch/arm/arm64/cpufeature.c
> +++ b/xen/arch/arm/arm64/cpufeature.c
> @@ -329,14 +329,11 @@ static const struct arm64_ftr_bits ftr_mvfr2[] = {
> ARM64_FTR_END,
> };
>
> -#if 0
> -/* TODO: handle this when sanitizing cache related registers */
> static const struct arm64_ftr_bits ftr_dczid[] = {
> ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_EXACT, DCZID_DZP_SHIFT, 1, 1),
> ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, DCZID_BS_SHIFT, 4, 0),
> ARM64_FTR_END,
> };
> -#endif
>
> static const struct arm64_ftr_bits ftr_id_isar0[] = {
> ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR0_DIVIDE_SHIFT, 4, 0),
> @@ -592,6 +589,17 @@ void update_system_features(const struct cpuinfo_arm *new)
>
> SANITIZE_ID_REG(zfr64, 0, aa64zfr0);
>
> + /*
> + * Comment from Linux:
I don't know if I would keep or remove "Comment from Linux"
In any case:
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
Also I gave a quick test of the series on a ZCU102 and it worked fine.
> + * Userspace may perform DC ZVA instructions. Mismatched block sizes
> + * could result in too much or too little memory being zeroed if a
> + * process is preempted and migrated between CPUs.
> + *
> + * ftr_dczid is using STRICT comparison so we will taint Xen if different
> + * values are found.
> + */
> + SANITIZE_REG(dczid, 0, dczid);
> +
> if ( cpu_feature64_has_el0_32(&system_cpuinfo) )
> {
> SANITIZE_ID_REG(pfr32, 0, pfr0);
> diff --git a/xen/arch/arm/cpufeature.c b/xen/arch/arm/cpufeature.c
> index f600a611bd..113f20f601 100644
> --- a/xen/arch/arm/cpufeature.c
> +++ b/xen/arch/arm/cpufeature.c
> @@ -125,6 +125,8 @@ void identify_cpu(struct cpuinfo_arm *c)
>
> c->zfr64.bits[0] = READ_SYSREG(ID_AA64ZFR0_EL1);
>
> + c->dczid.bits[0] = READ_SYSREG(DCZID_EL0);
> +
> aarch32_el0 = cpu_feature64_has_el0_32(c);
> #endif
>
> diff --git a/xen/include/asm-arm/cpufeature.h b/xen/include/asm-arm/cpufeature.h
> index 52cb3133e0..5219fd3bab 100644
> --- a/xen/include/asm-arm/cpufeature.h
> +++ b/xen/include/asm-arm/cpufeature.h
> @@ -259,6 +259,14 @@ struct cpuinfo_arm {
> register_t bits[1];
> } zfr64;
>
> + /*
> + * DCZID is only used to check for incoherent values between cores
> + * and taint Xen in this case
> + */
> + struct {
> + register_t bits[1];
> + } dczid;
> +
> #endif
>
> /*
> --
> 2.17.1
>
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [PATCH v3 7/7] xen/arm: Sanitize CTR_EL0 and emulate it if needed
2021-08-31 16:15 ` Bertrand Marquis
@ 2021-09-03 22:49 ` Stefano Stabellini
2021-09-06 8:29 ` Bertrand Marquis
0 siblings, 1 reply; 26+ messages in thread
From: Stefano Stabellini @ 2021-09-03 22:49 UTC (permalink / raw)
To: Bertrand Marquis
Cc: Julien Grall, xen-devel, Stefano Stabellini, Volodymyr Babchuk
On Tue, 31 Aug 2021, Bertrand Marquis wrote:
> Hi Julien,
>
> > On 31 Aug 2021, at 15:47, Julien Grall <julien@xen.org> wrote:
> >
> >
> >
> > On 31/08/2021 14:17, Bertrand Marquis wrote:
> >> Hi Julien,
> >
> > Hi Bertrand,
> >
> >>> On 27 Aug 2021, at 16:05, Julien Grall <julien@xen.org> wrote:
> >>>
> >>> Hi Bertrand,
> >>>
> >>> On 25/08/2021 14:18, Bertrand Marquis wrote:
> >>>> Sanitize CTR_EL0 value between cores.
> >>>> In most cases different values will taint Xen but if different
> >>>> i-cache policies are found, we choose the one which will be compatible
> >>>> between all cores in terms of invalidation/data cache flushing strategy.
> >>>
> >>> I understand that all the CPUs in Xen needs to agree on the cache flush strategy. However...
> >>>
> >>>> In this case we need to activate the TID2 bit in HCR to emulate the
> >>>> TCR_EL0 register for guests. This patch is not activating TID2 bit all
> >>>> the time to limit the overhead when possible.
> >>>
> >>> as we discussed in an earlier version, a vCPU is unlikely (at least in short/medium) to be able move across pCPU of different type. So the vCPU would be pinned to a set of pCPUs. IOW, the guest would have to be big.LITTLE aware and therefore would be able to do its own strategy decision.
> >>>
> >>> So I think we should be able to get away from trappings the registers.
> >> I do agree that we should be able to get away from that in the long term once
> >> we have cpupools properly set but right now this is the only way to have
> >> something useable (I will not say right).
> >> I will work on finding a way to setup properly cpupools (or something else as
> >> we discussed earlier) but in the short term I think this is the best we can do.
> >
> > My concern is you are making look like Xen will be able to deal nicely with big.LITTLE when in fact there are a lot more potential issue by allow a vCPU moving accross pCPU of different type (the errata is one example).
>
> I agree and this is why Xen is tainted.
>
> >
> >> An other solution would be to discard this patch from the serie for now until
> >> I have worked a proper solution for this case.
> >> Should we discard or merge or do you have an other idea ?
> > Please correct me if I am wrong, at the moment, it doesn't look like this patch will be part of the longer plan. If so, then I think it should be parked for now.
>
> Not sure it depends on what the final solution would be but this is highly possible I agree.
>
> >
> > This would also have the advantage to avoid spending too much time on resolving the emulation issue I mentioned in my previous answer.
> >
> > No need to resend a new version of this series yet. You can wait until the rest of the series get more feedback.
>
> Ok, I will wait for feedback and next serie will not include this patch anymore.
Would it be worth keeping just the part that sanitizes ctr, without any
of the emulation stuff? That way we could still detect cases where there
is a mismatch between CPUs, print a useful message and taint Xen.
For clarity something like the appended.
diff --git a/xen/arch/arm/arm64/cpufeature.c b/xen/arch/arm/arm64/cpufeature.c
index b1936ef1d6..d2456af2bf 100644
--- a/xen/arch/arm/arm64/cpufeature.c
+++ b/xen/arch/arm/arm64/cpufeature.c
@@ -275,9 +275,6 @@ static const struct arm64_ftr_bits ftr_id_aa64mmfr2[] = {
ARM64_FTR_END,
};
-#if 0
-/* TODO: use this to sanitize the cache line size among cores */
-
static const struct arm64_ftr_bits ftr_ctr[] = {
ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_EXACT, 31, 1, 1), /* RES1 */
ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, CTR_DIC_SHIFT, 1, 1),
@@ -294,7 +291,6 @@ static const struct arm64_ftr_bits ftr_ctr[] = {
ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, CTR_IMINLINE_SHIFT, 4, 0),
ARM64_FTR_END,
};
-#endif
static const struct arm64_ftr_bits ftr_id_mmfr0[] = {
S_ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR0_INNERSHR_SHIFT, 4, 0xf),
@@ -600,6 +596,8 @@ void update_system_features(const struct cpuinfo_arm *new)
*/
SANITIZE_REG(dczid, 0, dczid);
+ SANITIZE_REG(ctr, 0, ctr);
+
if ( cpu_feature64_has_el0_32(&system_cpuinfo) )
{
SANITIZE_ID_REG(pfr32, 0, pfr0);
diff --git a/xen/arch/arm/cpufeature.c b/xen/arch/arm/cpufeature.c
index 113f20f601..6e51f530a8 100644
--- a/xen/arch/arm/cpufeature.c
+++ b/xen/arch/arm/cpufeature.c
@@ -127,6 +127,8 @@ void identify_cpu(struct cpuinfo_arm *c)
c->dczid.bits[0] = READ_SYSREG(DCZID_EL0);
+ c->ctr.bits[0] = READ_SYSREG(CTR_EL0);
+
aarch32_el0 = cpu_feature64_has_el0_32(c);
#endif
diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
index 3798c5ade0..33b7bfb59c 100644
--- a/xen/arch/arm/setup.c
+++ b/xen/arch/arm/setup.c
@@ -627,7 +627,7 @@ static void __init setup_mm(void)
panic("No memory bank\n");
/* We only supports instruction caches implementing the IVIPT extension. */
- if ( ((ctr >> CTR_L1Ip_SHIFT) & CTR_L1Ip_MASK) == CTR_L1Ip_AIVIVT )
+ if ( ((ctr >> CTR_L1IP_SHIFT) & CTR_L1IP_MASK) == ICACHE_POLICY_AIVIVT )
panic("AIVIVT instruction cache not supported\n");
init_pdx();
diff --git a/xen/include/asm-arm/cpufeature.h b/xen/include/asm-arm/cpufeature.h
index 5219fd3bab..ca6e827fcb 100644
--- a/xen/include/asm-arm/cpufeature.h
+++ b/xen/include/asm-arm/cpufeature.h
@@ -267,6 +267,14 @@ struct cpuinfo_arm {
register_t bits[1];
} dczid;
+ /*
+ * CTR is only used to check for different cache types or policies and
+ * taint Xen in this case
+ */
+ struct {
+ register_t bits[1];
+ } ctr;
+
#endif
/*
@@ -339,6 +347,9 @@ extern struct cpuinfo_arm system_cpuinfo;
extern void identify_cpu(struct cpuinfo_arm *);
#ifdef CONFIG_ARM_64
+
+extern bool mismatched_cache_type;
+
extern void update_system_features(const struct cpuinfo_arm *);
#else
static inline void update_system_features(const struct cpuinfo_arm *cpuinfo)
diff --git a/xen/include/asm-arm/processor.h b/xen/include/asm-arm/processor.h
index 2577e9a244..8c9843e12b 100644
--- a/xen/include/asm-arm/processor.h
+++ b/xen/include/asm-arm/processor.h
@@ -7,9 +7,21 @@
#include <public/arch-arm.h>
/* CTR Cache Type Register */
-#define CTR_L1Ip_MASK 0x3
-#define CTR_L1Ip_SHIFT 14
-#define CTR_L1Ip_AIVIVT 0x1
+#define CTR_L1IP_MASK 0x3
+#define CTR_L1IP_SHIFT 14
+#define CTR_DMINLINE_SHIFT 16
+#define CTR_IMINLINE_SHIFT 0
+#define CTR_IMINLINE_MASK 0xf
+#define CTR_ERG_SHIFT 20
+#define CTR_CWG_SHIFT 24
+#define CTR_CWG_MASK 15
+#define CTR_IDC_SHIFT 28
+#define CTR_DIC_SHIFT 29
+
+#define ICACHE_POLICY_VPIPT 0
+#define ICACHE_POLICY_AIVIVT 1
+#define ICACHE_POLICY_VIPT 2
+#define ICACHE_POLICY_PIPT 3
/* MIDR Main ID Register */
#define MIDR_REVISION_MASK 0xf
^ permalink raw reply related [flat|nested] 26+ messages in thread
* Re: [PATCH v3 6/7] xen/arm: Taint Xen on incompatible DCZID values
2021-09-03 22:49 ` Stefano Stabellini
@ 2021-09-06 7:59 ` Bertrand Marquis
2021-09-07 23:17 ` Stefano Stabellini
0 siblings, 1 reply; 26+ messages in thread
From: Bertrand Marquis @ 2021-09-06 7:59 UTC (permalink / raw)
To: Stefano Stabellini; +Cc: xen-devel, Julien Grall, Volodymyr Babchuk
> On 3 Sep 2021, at 23:49, Stefano Stabellini <sstabellini@kernel.org> wrote:
>
> On Wed, 25 Aug 2021, Bertrand Marquis wrote:
>> Use arm64 cpu feature sanitization to TAIN Xen if different DCZID values
> ^ TAINT
>
>
>> are found (ftr_dczid is using only STRICT method).
>> In this case actual memory being cleaned by DC ZVA operations would be
>> different depending on the cores which could make a guest zeroing too
>> much or too little memory if it is merged between CPUs.
>>
>> We could, on processor supporting it, trap access to DCZID_EL0 register
> ^ processors
Could those typos be fixed during commit ?
>
>> using HFGRTR_EL2 register but this would not solve the case where a
>> process is being migrated during a copy or if it cached the value of the
>> register.
>>
>> Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
>> ---
>> Change in v3: none
>> Change in v2: Patch introduced in v2
>> ---
>> xen/arch/arm/arm64/cpufeature.c | 14 +++++++++++---
>> xen/arch/arm/cpufeature.c | 2 ++
>> xen/include/asm-arm/cpufeature.h | 8 ++++++++
>> 3 files changed, 21 insertions(+), 3 deletions(-)
>>
>> diff --git a/xen/arch/arm/arm64/cpufeature.c b/xen/arch/arm/arm64/cpufeature.c
>> index 61f629ebaa..b1936ef1d6 100644
>> --- a/xen/arch/arm/arm64/cpufeature.c
>> +++ b/xen/arch/arm/arm64/cpufeature.c
>> @@ -329,14 +329,11 @@ static const struct arm64_ftr_bits ftr_mvfr2[] = {
>> ARM64_FTR_END,
>> };
>>
>> -#if 0
>> -/* TODO: handle this when sanitizing cache related registers */
>> static const struct arm64_ftr_bits ftr_dczid[] = {
>> ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_EXACT, DCZID_DZP_SHIFT, 1, 1),
>> ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, DCZID_BS_SHIFT, 4, 0),
>> ARM64_FTR_END,
>> };
>> -#endif
>>
>> static const struct arm64_ftr_bits ftr_id_isar0[] = {
>> ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR0_DIVIDE_SHIFT, 4, 0),
>> @@ -592,6 +589,17 @@ void update_system_features(const struct cpuinfo_arm *new)
>>
>> SANITIZE_ID_REG(zfr64, 0, aa64zfr0);
>>
>> + /*
>> + * Comment from Linux:
>
> I don't know if I would keep or remove "Comment from Linux"
I added that because the comment itself does not really apply to Xen.
I could have rephrased the comment/
Anyway I have no objection to remove that statement.
Do I need to send a v2 for that ?
Cheers
Bertrand
>
> In any case:
>
> Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
>
>
> Also I gave a quick test of the series on a ZCU102 and it worked fine.
>
>
>> + * Userspace may perform DC ZVA instructions. Mismatched block sizes
>> + * could result in too much or too little memory being zeroed if a
>> + * process is preempted and migrated between CPUs.
>> + *
>> + * ftr_dczid is using STRICT comparison so we will taint Xen if different
>> + * values are found.
>> + */
>> + SANITIZE_REG(dczid, 0, dczid);
>> +
>> if ( cpu_feature64_has_el0_32(&system_cpuinfo) )
>> {
>> SANITIZE_ID_REG(pfr32, 0, pfr0);
>> diff --git a/xen/arch/arm/cpufeature.c b/xen/arch/arm/cpufeature.c
>> index f600a611bd..113f20f601 100644
>> --- a/xen/arch/arm/cpufeature.c
>> +++ b/xen/arch/arm/cpufeature.c
>> @@ -125,6 +125,8 @@ void identify_cpu(struct cpuinfo_arm *c)
>>
>> c->zfr64.bits[0] = READ_SYSREG(ID_AA64ZFR0_EL1);
>>
>> + c->dczid.bits[0] = READ_SYSREG(DCZID_EL0);
>> +
>> aarch32_el0 = cpu_feature64_has_el0_32(c);
>> #endif
>>
>> diff --git a/xen/include/asm-arm/cpufeature.h b/xen/include/asm-arm/cpufeature.h
>> index 52cb3133e0..5219fd3bab 100644
>> --- a/xen/include/asm-arm/cpufeature.h
>> +++ b/xen/include/asm-arm/cpufeature.h
>> @@ -259,6 +259,14 @@ struct cpuinfo_arm {
>> register_t bits[1];
>> } zfr64;
>>
>> + /*
>> + * DCZID is only used to check for incoherent values between cores
>> + * and taint Xen in this case
>> + */
>> + struct {
>> + register_t bits[1];
>> + } dczid;
>> +
>> #endif
>>
>> /*
>> --
>> 2.17.1
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [PATCH v3 4/7] xen/arm: Sanitize cpuinfo ID registers fields
2021-09-03 22:48 ` Stefano Stabellini
@ 2021-09-06 8:24 ` Bertrand Marquis
0 siblings, 0 replies; 26+ messages in thread
From: Bertrand Marquis @ 2021-09-06 8:24 UTC (permalink / raw)
To: Stefano Stabellini
Cc: Xen-devel, Julien Grall, Volodymyr Babchuk, Andrew Cooper,
George Dunlap, Ian Jackson, Jan Beulich, Wei Liu
Hi Stefano,
> On 3 Sep 2021, at 23:48, Stefano Stabellini <sstabellini@kernel.org> wrote:
>
> On Wed, 25 Aug 2021, Bertrand Marquis wrote:
>> Define a sanitize_cpu function to be called on secondary cores to
>> sanitize the system cpuinfo structure.
>>
>> The safest value is taken when possible and the system is marked tainted
>> if we encounter values which are incompatible with each other.
>>
>> Call the update_system_features function on all secondary cores that are
>> kept running and taint the system if different midr are found between
>> cores but hmp-unsafe=true was passed on Xen command line.
>>
>> This is only supported on arm64 so update_system_features is an empty
>> static inline on arm32.
>>
>> The patch is adding a new TAINT_CPU_OUT_OF_SPEC to warn the user if
>> Xen is running on a system with features differences between cores which
>> are not supported.
>>
>> The patch is disabling CTR_EL0, DCZID_EL0 and ZCRusing #if 0 with a TODO
>> as this patch is not handling sanitization of those registers.
>> CTR_EL0/DCZID will be handled in a future patch to properly handle
>> different cache attributes when possible.
>> ZCR should be sanitize once we add support for SVE in Xen.
>>
>> Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
>
> Great patch! It looks good I only have a couple of minor questions
> below.
Thanks
>
>
>> ---
>> Changes in v3:
>> - in case of different midr but hmp-unsafe passed on the command line,
>> enable all cores anyway but taint Xen with CPU_OUT_OF_SPEC.
>> - use current core info to sanitize cpu only if we keep it on
>> Changes in v2:
>> - add compilation of cpufeature.c in this patch instead of previous one
>> - remove functions reused from linux code and moved to header
>> - rename sanitize_cpu to update_system_features
>> - change to Linux coding style
>> - remove dev comments
>> - surround currently not used Linux structures with #if 0 and adapt the
>> commit message
>> - add missing aa64dfr1 register
>> - add TODO for CTR, DCZID and ZCR
>> - add CPU_OUT_OF_SPEC support to print_taint
>> - use system_cpuinfo instead of boot_cpu_data
>> ---
>> xen/arch/arm/arm64/Makefile | 1 +
>> xen/arch/arm/arm64/cpufeature.c | 121 +++++++++++++++++++++++++++++++
>> xen/arch/arm/smpboot.c | 34 +++++++--
>> xen/common/kernel.c | 6 +-
>> xen/include/asm-arm/cpufeature.h | 9 +++
>> xen/include/xen/lib.h | 1 +
>> 6 files changed, 162 insertions(+), 10 deletions(-)
>>
>> diff --git a/xen/arch/arm/arm64/Makefile b/xen/arch/arm/arm64/Makefile
>> index 40642ff574..701d66883d 100644
>> --- a/xen/arch/arm/arm64/Makefile
>> +++ b/xen/arch/arm/arm64/Makefile
>> @@ -1,6 +1,7 @@
>> obj-y += lib/
>>
>> obj-y += cache.o
>> +obj-y += cpufeature.o
>> obj-$(CONFIG_HARDEN_BRANCH_PREDICTOR) += bpi.o
>> obj-$(CONFIG_EARLY_PRINTK) += debug.o
>> obj-y += domctl.o
>> diff --git a/xen/arch/arm/arm64/cpufeature.c b/xen/arch/arm/arm64/cpufeature.c
>> index 5777e33e5c..61f629ebaa 100644
>> --- a/xen/arch/arm/arm64/cpufeature.c
>> +++ b/xen/arch/arm/arm64/cpufeature.c
>> @@ -275,6 +275,9 @@ static const struct arm64_ftr_bits ftr_id_aa64mmfr2[] = {
>> ARM64_FTR_END,
>> };
>>
>> +#if 0
>> +/* TODO: use this to sanitize the cache line size among cores */
>> +
>> static const struct arm64_ftr_bits ftr_ctr[] = {
>> ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_EXACT, 31, 1, 1), /* RES1 */
>> ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, CTR_DIC_SHIFT, 1, 1),
>> @@ -291,6 +294,7 @@ static const struct arm64_ftr_bits ftr_ctr[] = {
>> ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, CTR_IMINLINE_SHIFT, 4, 0),
>> ARM64_FTR_END,
>> };
>> +#endif
>>
>> static const struct arm64_ftr_bits ftr_id_mmfr0[] = {
>> S_ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR0_INNERSHR_SHIFT, 4, 0xf),
>> @@ -325,11 +329,14 @@ static const struct arm64_ftr_bits ftr_mvfr2[] = {
>> ARM64_FTR_END,
>> };
>>
>> +#if 0
>> +/* TODO: handle this when sanitizing cache related registers */
>> static const struct arm64_ftr_bits ftr_dczid[] = {
>> ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_EXACT, DCZID_DZP_SHIFT, 1, 1),
>> ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, DCZID_BS_SHIFT, 4, 0),
>> ARM64_FTR_END,
>> };
>> +#endif
>>
>> static const struct arm64_ftr_bits ftr_id_isar0[] = {
>> ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR0_DIVIDE_SHIFT, 4, 0),
>> @@ -444,11 +451,15 @@ static const struct arm64_ftr_bits ftr_id_dfr1[] = {
>> ARM64_FTR_END,
>> };
>>
>> +#if 0
>> +/* TODO: use this to sanitize SVE once we support it */
>> +
>> static const struct arm64_ftr_bits ftr_zcr[] = {
>> ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE,
>> ZCR_ELx_LEN_SHIFT, ZCR_ELx_LEN_SIZE, 0), /* LEN */
>> ARM64_FTR_END,
>> };
>> +#endif
>>
>> /*
>> * Common ftr bits for a 32bit register with all hidden, strict
>> @@ -502,3 +513,113 @@ static s64 arm64_ftr_safe_value(const struct arm64_ftr_bits *ftrp, s64 new,
>> * End of imported linux structures and code
>> */
>>
>> +static void sanitize_reg(u64 *cur_reg, u64 new_reg, const char *reg_name,
>> + const struct arm64_ftr_bits *ftrp)
>> +{
>> + int taint = 0;
>> + u64 old_reg = *cur_reg;
>> +
>> + for (;ftrp->width != 0;ftrp++)
>> + {
>> + u64 mask;
>> + s64 cur_field = arm64_ftr_value(ftrp, *cur_reg);
>> + s64 new_field = arm64_ftr_value(ftrp, new_reg);
>> +
>> + if (cur_field == new_field)
>> + continue;
>> +
>> + if (ftrp->strict)
>> + taint = 1;
>> +
>> + mask = arm64_ftr_mask(ftrp);
>> +
>> + *cur_reg &= ~mask;
>> + *cur_reg |= (arm64_ftr_safe_value(ftrp, new_field, cur_field)
>> + << ftrp->shift) & mask;
>
> I wonder why you haven't also imported arm64_ftr_set_value? This seems
> to be the open-coded version of it.
You are right I could have used it instead (no idea why I did not).
I will modify that and send an update.
Good finding :-)
>
>
>> + }
>> +
>> + if (old_reg != new_reg)
>> + printk(XENLOG_DEBUG "SANITY DIF: %s 0x%"PRIx64" -> 0x%"PRIx64"\n",
>> + reg_name, old_reg, new_reg);
>> + if (old_reg != *cur_reg)
>> + printk(XENLOG_DEBUG "SANITY FIX: %s 0x%"PRIx64" -> 0x%"PRIx64"\n",
>> + reg_name, old_reg, *cur_reg);
>> +
>> + if (taint)
>> + {
>> + printk(XENLOG_WARNING "SANITY CHECK: Unexpected variation in %s.\n",
>> + reg_name);
>> + add_taint(TAINT_CPU_OUT_OF_SPEC);
>> + }
>> +}
>> +
>> +
>> +/*
>> + * This function should be called on secondary cores to sanitize the boot cpu
>> + * cpuinfo.
>> + */
>> +void update_system_features(const struct cpuinfo_arm *new)
>> +{
>> +
>> +#define SANITIZE_REG(field, num, reg) \
>> + sanitize_reg(&system_cpuinfo.field.bits[num], new->field.bits[num], \
>> + #reg, ftr_##reg)
>> +
>> +#define SANITIZE_ID_REG(field, num, reg) \
>> + sanitize_reg(&system_cpuinfo.field.bits[num], new->field.bits[num], \
>> + #reg, ftr_id_##reg)
>> +
>> +#define SANITIZE_RAZ_REG(field, num, reg) \
>> + sanitize_reg(&system_cpuinfo.field.bits[num], new->field.bits[num], \
>> + #reg, ftr_raz)
>> +
>> +#define SANITIZE_GENERIC_REG(field, num, reg) \
>> + sanitize_reg(&system_cpuinfo.field.bits[num], new->field.bits[num], \
>> + #reg, ftr_generic_32bits)
>> +
>> + SANITIZE_ID_REG(pfr64, 0, aa64pfr0);
>> + SANITIZE_ID_REG(pfr64, 1, aa64pfr1);
>> +
>> + SANITIZE_ID_REG(dbg64, 0, aa64dfr0);
>> + SANITIZE_RAZ_REG(dbg64, 1, aa64dfr1);
>> +
>> + SANITIZE_ID_REG(mm64, 0, aa64mmfr0);
>> + SANITIZE_ID_REG(mm64, 1, aa64mmfr1);
>> + SANITIZE_ID_REG(mm64, 2, aa64mmfr2);
>> +
>> + SANITIZE_ID_REG(isa64, 0, aa64isar0);
>> + SANITIZE_ID_REG(isa64, 1, aa64isar1);
>> +
>> + SANITIZE_ID_REG(zfr64, 0, aa64zfr0);
>> +
>> + if ( cpu_feature64_has_el0_32(&system_cpuinfo) )
>> + {
>> + SANITIZE_ID_REG(pfr32, 0, pfr0);
>> + SANITIZE_ID_REG(pfr32, 1, pfr1);
>> + SANITIZE_ID_REG(pfr32, 2, pfr2);
>> +
>> + SANITIZE_ID_REG(dbg32, 0, dfr0);
>> + SANITIZE_ID_REG(dbg32, 1, dfr1);
>> +
>> + SANITIZE_ID_REG(mm32, 0, mmfr0);
>> + SANITIZE_GENERIC_REG(mm32, 1, mmfr1);
>> + SANITIZE_GENERIC_REG(mm32, 2, mmfr2);
>> + SANITIZE_GENERIC_REG(mm32, 3, mmfr3);
>> + SANITIZE_ID_REG(mm32, 4, mmfr4);
>> + SANITIZE_ID_REG(mm32, 5, mmfr5);
>> +
>> + SANITIZE_ID_REG(isa32, 0, isar0);
>> + SANITIZE_GENERIC_REG(isa32, 1, isar1);
>> + SANITIZE_GENERIC_REG(isa32, 2, isar2);
>> + SANITIZE_GENERIC_REG(isa32, 3, isar3);
>> + SANITIZE_ID_REG(isa32, 4, isar4);
>> + SANITIZE_ID_REG(isa32, 5, isar5);
>> + SANITIZE_ID_REG(isa32, 6, isar6);
>> +
>> + SANITIZE_GENERIC_REG(mvfr, 0, mvfr0);
>> + SANITIZE_GENERIC_REG(mvfr, 1, mvfr1);
>> +#ifndef MVFR2_MAYBE_UNDEFINED
>> + SANITIZE_REG(mvfr, 2, mvfr2);
>> +#endif
>> + }
>> +}
>
> Looking at the list of registers we are sanitizing here we are only
> missing aux32 and aux64 compared to struct cpuinfo_arm. Is that because
> there is nothing to sanitize there?
Aux registers have no standard bit definition and as such cannot be treated.
Linux is doing the same.
Cheers
Bertrand
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [PATCH v3 7/7] xen/arm: Sanitize CTR_EL0 and emulate it if needed
2021-09-03 22:49 ` Stefano Stabellini
@ 2021-09-06 8:29 ` Bertrand Marquis
2021-09-06 17:36 ` Julien Grall
0 siblings, 1 reply; 26+ messages in thread
From: Bertrand Marquis @ 2021-09-06 8:29 UTC (permalink / raw)
To: Stefano Stabellini; +Cc: Julien Grall, xen-devel, Volodymyr Babchuk
Hi Stefano,
> On 3 Sep 2021, at 23:49, Stefano Stabellini <sstabellini@kernel.org> wrote:
>
> On Tue, 31 Aug 2021, Bertrand Marquis wrote:
>> Hi Julien,
>>
>>> On 31 Aug 2021, at 15:47, Julien Grall <julien@xen.org> wrote:
>>>
>>>
>>>
>>> On 31/08/2021 14:17, Bertrand Marquis wrote:
>>>> Hi Julien,
>>>
>>> Hi Bertrand,
>>>
>>>>> On 27 Aug 2021, at 16:05, Julien Grall <julien@xen.org> wrote:
>>>>>
>>>>> Hi Bertrand,
>>>>>
>>>>> On 25/08/2021 14:18, Bertrand Marquis wrote:
>>>>>> Sanitize CTR_EL0 value between cores.
>>>>>> In most cases different values will taint Xen but if different
>>>>>> i-cache policies are found, we choose the one which will be compatible
>>>>>> between all cores in terms of invalidation/data cache flushing strategy.
>>>>>
>>>>> I understand that all the CPUs in Xen needs to agree on the cache flush strategy. However...
>>>>>
>>>>>> In this case we need to activate the TID2 bit in HCR to emulate the
>>>>>> TCR_EL0 register for guests. This patch is not activating TID2 bit all
>>>>>> the time to limit the overhead when possible.
>>>>>
>>>>> as we discussed in an earlier version, a vCPU is unlikely (at least in short/medium) to be able move across pCPU of different type. So the vCPU would be pinned to a set of pCPUs. IOW, the guest would have to be big.LITTLE aware and therefore would be able to do its own strategy decision.
>>>>>
>>>>> So I think we should be able to get away from trappings the registers.
>>>> I do agree that we should be able to get away from that in the long term once
>>>> we have cpupools properly set but right now this is the only way to have
>>>> something useable (I will not say right).
>>>> I will work on finding a way to setup properly cpupools (or something else as
>>>> we discussed earlier) but in the short term I think this is the best we can do.
>>>
>>> My concern is you are making look like Xen will be able to deal nicely with big.LITTLE when in fact there are a lot more potential issue by allow a vCPU moving accross pCPU of different type (the errata is one example).
>>
>> I agree and this is why Xen is tainted.
>>
>>>
>>>> An other solution would be to discard this patch from the serie for now until
>>>> I have worked a proper solution for this case.
>>>> Should we discard or merge or do you have an other idea ?
>>> Please correct me if I am wrong, at the moment, it doesn't look like this patch will be part of the longer plan. If so, then I think it should be parked for now.
>>
>> Not sure it depends on what the final solution would be but this is highly possible I agree.
>>
>>>
>>> This would also have the advantage to avoid spending too much time on resolving the emulation issue I mentioned in my previous answer.
>>>
>>> No need to resend a new version of this series yet. You can wait until the rest of the series get more feedback.
>>
>> Ok, I will wait for feedback and next serie will not include this patch anymore.
>
> Would it be worth keeping just the part that sanitizes ctr, without any
> of the emulation stuff? That way we could still detect cases where there
> is a mismatch between CPUs, print a useful message and taint Xen.
That’s a good idea, it means removing the emulation part and just keep the sanitization.
@Julien: would you be ok with that ?
Should I send a v4 or should we use Stefano’s patch directly instead ?
Cheers
Bertrand
>
> For clarity something like the appended.
>
>
> diff --git a/xen/arch/arm/arm64/cpufeature.c b/xen/arch/arm/arm64/cpufeature.c
> index b1936ef1d6..d2456af2bf 100644
> --- a/xen/arch/arm/arm64/cpufeature.c
> +++ b/xen/arch/arm/arm64/cpufeature.c
> @@ -275,9 +275,6 @@ static const struct arm64_ftr_bits ftr_id_aa64mmfr2[] = {
> ARM64_FTR_END,
> };
>
> -#if 0
> -/* TODO: use this to sanitize the cache line size among cores */
> -
> static const struct arm64_ftr_bits ftr_ctr[] = {
> ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_EXACT, 31, 1, 1), /* RES1 */
> ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, CTR_DIC_SHIFT, 1, 1),
> @@ -294,7 +291,6 @@ static const struct arm64_ftr_bits ftr_ctr[] = {
> ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, CTR_IMINLINE_SHIFT, 4, 0),
> ARM64_FTR_END,
> };
> -#endif
>
> static const struct arm64_ftr_bits ftr_id_mmfr0[] = {
> S_ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR0_INNERSHR_SHIFT, 4, 0xf),
> @@ -600,6 +596,8 @@ void update_system_features(const struct cpuinfo_arm *new)
> */
> SANITIZE_REG(dczid, 0, dczid);
>
> + SANITIZE_REG(ctr, 0, ctr);
> +
> if ( cpu_feature64_has_el0_32(&system_cpuinfo) )
> {
> SANITIZE_ID_REG(pfr32, 0, pfr0);
> diff --git a/xen/arch/arm/cpufeature.c b/xen/arch/arm/cpufeature.c
> index 113f20f601..6e51f530a8 100644
> --- a/xen/arch/arm/cpufeature.c
> +++ b/xen/arch/arm/cpufeature.c
> @@ -127,6 +127,8 @@ void identify_cpu(struct cpuinfo_arm *c)
>
> c->dczid.bits[0] = READ_SYSREG(DCZID_EL0);
>
> + c->ctr.bits[0] = READ_SYSREG(CTR_EL0);
> +
> aarch32_el0 = cpu_feature64_has_el0_32(c);
> #endif
>
> diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
> index 3798c5ade0..33b7bfb59c 100644
> --- a/xen/arch/arm/setup.c
> +++ b/xen/arch/arm/setup.c
> @@ -627,7 +627,7 @@ static void __init setup_mm(void)
> panic("No memory bank\n");
>
> /* We only supports instruction caches implementing the IVIPT extension. */
> - if ( ((ctr >> CTR_L1Ip_SHIFT) & CTR_L1Ip_MASK) == CTR_L1Ip_AIVIVT )
> + if ( ((ctr >> CTR_L1IP_SHIFT) & CTR_L1IP_MASK) == ICACHE_POLICY_AIVIVT )
> panic("AIVIVT instruction cache not supported\n");
>
> init_pdx();
> diff --git a/xen/include/asm-arm/cpufeature.h b/xen/include/asm-arm/cpufeature.h
> index 5219fd3bab..ca6e827fcb 100644
> --- a/xen/include/asm-arm/cpufeature.h
> +++ b/xen/include/asm-arm/cpufeature.h
> @@ -267,6 +267,14 @@ struct cpuinfo_arm {
> register_t bits[1];
> } dczid;
>
> + /*
> + * CTR is only used to check for different cache types or policies and
> + * taint Xen in this case
> + */
> + struct {
> + register_t bits[1];
> + } ctr;
> +
> #endif
>
> /*
> @@ -339,6 +347,9 @@ extern struct cpuinfo_arm system_cpuinfo;
> extern void identify_cpu(struct cpuinfo_arm *);
>
> #ifdef CONFIG_ARM_64
> +
> +extern bool mismatched_cache_type;
> +
> extern void update_system_features(const struct cpuinfo_arm *);
> #else
> static inline void update_system_features(const struct cpuinfo_arm *cpuinfo)
> diff --git a/xen/include/asm-arm/processor.h b/xen/include/asm-arm/processor.h
> index 2577e9a244..8c9843e12b 100644
> --- a/xen/include/asm-arm/processor.h
> +++ b/xen/include/asm-arm/processor.h
> @@ -7,9 +7,21 @@
> #include <public/arch-arm.h>
>
> /* CTR Cache Type Register */
> -#define CTR_L1Ip_MASK 0x3
> -#define CTR_L1Ip_SHIFT 14
> -#define CTR_L1Ip_AIVIVT 0x1
> +#define CTR_L1IP_MASK 0x3
> +#define CTR_L1IP_SHIFT 14
> +#define CTR_DMINLINE_SHIFT 16
> +#define CTR_IMINLINE_SHIFT 0
> +#define CTR_IMINLINE_MASK 0xf
> +#define CTR_ERG_SHIFT 20
> +#define CTR_CWG_SHIFT 24
> +#define CTR_CWG_MASK 15
> +#define CTR_IDC_SHIFT 28
> +#define CTR_DIC_SHIFT 29
> +
> +#define ICACHE_POLICY_VPIPT 0
> +#define ICACHE_POLICY_AIVIVT 1
> +#define ICACHE_POLICY_VIPT 2
> +#define ICACHE_POLICY_PIPT 3
>
> /* MIDR Main ID Register */
> #define MIDR_REVISION_MASK 0xf
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [PATCH v3 7/7] xen/arm: Sanitize CTR_EL0 and emulate it if needed
2021-09-06 8:29 ` Bertrand Marquis
@ 2021-09-06 17:36 ` Julien Grall
2021-09-07 7:37 ` Bertrand Marquis
0 siblings, 1 reply; 26+ messages in thread
From: Julien Grall @ 2021-09-06 17:36 UTC (permalink / raw)
To: Bertrand Marquis, Stefano Stabellini; +Cc: xen-devel, Volodymyr Babchuk
Hi Bertrand,
On 06/09/2021 09:29, Bertrand Marquis wrote:
>> On 3 Sep 2021, at 23:49, Stefano Stabellini <sstabellini@kernel.org> wrote:
>>
>> On Tue, 31 Aug 2021, Bertrand Marquis wrote:
>>> Hi Julien,
>>>
>>>> On 31 Aug 2021, at 15:47, Julien Grall <julien@xen.org> wrote:
>>>>
>>>>
>>>>
>>>> On 31/08/2021 14:17, Bertrand Marquis wrote:
>>>>> Hi Julien,
>>>>
>>>> Hi Bertrand,
>>>>
>>>>>> On 27 Aug 2021, at 16:05, Julien Grall <julien@xen.org> wrote:
>>>>>>
>>>>>> Hi Bertrand,
>>>>>>
>>>>>> On 25/08/2021 14:18, Bertrand Marquis wrote:
>>>>>>> Sanitize CTR_EL0 value between cores.
>>>>>>> In most cases different values will taint Xen but if different
>>>>>>> i-cache policies are found, we choose the one which will be compatible
>>>>>>> between all cores in terms of invalidation/data cache flushing strategy.
>>>>>>
>>>>>> I understand that all the CPUs in Xen needs to agree on the cache flush strategy. However...
>>>>>>
>>>>>>> In this case we need to activate the TID2 bit in HCR to emulate the
>>>>>>> TCR_EL0 register for guests. This patch is not activating TID2 bit all
>>>>>>> the time to limit the overhead when possible.
>>>>>>
>>>>>> as we discussed in an earlier version, a vCPU is unlikely (at least in short/medium) to be able move across pCPU of different type. So the vCPU would be pinned to a set of pCPUs. IOW, the guest would have to be big.LITTLE aware and therefore would be able to do its own strategy decision.
>>>>>>
>>>>>> So I think we should be able to get away from trappings the registers.
>>>>> I do agree that we should be able to get away from that in the long term once
>>>>> we have cpupools properly set but right now this is the only way to have
>>>>> something useable (I will not say right).
>>>>> I will work on finding a way to setup properly cpupools (or something else as
>>>>> we discussed earlier) but in the short term I think this is the best we can do.
>>>>
>>>> My concern is you are making look like Xen will be able to deal nicely with big.LITTLE when in fact there are a lot more potential issue by allow a vCPU moving accross pCPU of different type (the errata is one example).
>>>
>>> I agree and this is why Xen is tainted.
>>>
>>>>
>>>>> An other solution would be to discard this patch from the serie for now until
>>>>> I have worked a proper solution for this case.
>>>>> Should we discard or merge or do you have an other idea ?
>>>> Please correct me if I am wrong, at the moment, it doesn't look like this patch will be part of the longer plan. If so, then I think it should be parked for now.
>>>
>>> Not sure it depends on what the final solution would be but this is highly possible I agree.
>>>
>>>>
>>>> This would also have the advantage to avoid spending too much time on resolving the emulation issue I mentioned in my previous answer.
>>>>
>>>> No need to resend a new version of this series yet. You can wait until the rest of the series get more feedback.
>>>
>>> Ok, I will wait for feedback and next serie will not include this patch anymore.
>>
>> Would it be worth keeping just the part that sanitizes ctr, without any
>> of the emulation stuff? That way we could still detect cases where there
>> is a mismatch between CPUs, print a useful message and taint Xen.
>
> That’s a good idea, it means removing the emulation part and just keep the sanitization.
>
> @Julien: would you be ok with that ?
I actually thought about suggesting it before Stefano did it. So I am OK
with that.
>
> Should I send a v4 or should we use Stefano’s patch directly instead ?
I would suggest to send a v4. This needs a signed-off-by from Stefano
and a new commit message.
Cheers,
--
Julien Grall
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [PATCH v3 7/7] xen/arm: Sanitize CTR_EL0 and emulate it if needed
2021-09-06 17:36 ` Julien Grall
@ 2021-09-07 7:37 ` Bertrand Marquis
2021-09-07 23:17 ` Stefano Stabellini
0 siblings, 1 reply; 26+ messages in thread
From: Bertrand Marquis @ 2021-09-07 7:37 UTC (permalink / raw)
To: Julien Grall; +Cc: Stefano Stabellini, xen-devel, Volodymyr Babchuk
Hi Julien,
> On 6 Sep 2021, at 18:36, Julien Grall <julien@xen.org> wrote:
>
> Hi Bertrand,
>
> On 06/09/2021 09:29, Bertrand Marquis wrote:
>>> On 3 Sep 2021, at 23:49, Stefano Stabellini <sstabellini@kernel.org> wrote:
>>>
>>> On Tue, 31 Aug 2021, Bertrand Marquis wrote:
>>>> Hi Julien,
>>>>
>>>>> On 31 Aug 2021, at 15:47, Julien Grall <julien@xen.org> wrote:
>>>>>
>>>>>
>>>>>
>>>>> On 31/08/2021 14:17, Bertrand Marquis wrote:
>>>>>> Hi Julien,
>>>>>
>>>>> Hi Bertrand,
>>>>>
>>>>>>> On 27 Aug 2021, at 16:05, Julien Grall <julien@xen.org> wrote:
>>>>>>>
>>>>>>> Hi Bertrand,
>>>>>>>
>>>>>>> On 25/08/2021 14:18, Bertrand Marquis wrote:
>>>>>>>> Sanitize CTR_EL0 value between cores.
>>>>>>>> In most cases different values will taint Xen but if different
>>>>>>>> i-cache policies are found, we choose the one which will be compatible
>>>>>>>> between all cores in terms of invalidation/data cache flushing strategy.
>>>>>>>
>>>>>>> I understand that all the CPUs in Xen needs to agree on the cache flush strategy. However...
>>>>>>>
>>>>>>>> In this case we need to activate the TID2 bit in HCR to emulate the
>>>>>>>> TCR_EL0 register for guests. This patch is not activating TID2 bit all
>>>>>>>> the time to limit the overhead when possible.
>>>>>>>
>>>>>>> as we discussed in an earlier version, a vCPU is unlikely (at least in short/medium) to be able move across pCPU of different type. So the vCPU would be pinned to a set of pCPUs. IOW, the guest would have to be big.LITTLE aware and therefore would be able to do its own strategy decision.
>>>>>>>
>>>>>>> So I think we should be able to get away from trappings the registers.
>>>>>> I do agree that we should be able to get away from that in the long term once
>>>>>> we have cpupools properly set but right now this is the only way to have
>>>>>> something useable (I will not say right).
>>>>>> I will work on finding a way to setup properly cpupools (or something else as
>>>>>> we discussed earlier) but in the short term I think this is the best we can do.
>>>>>
>>>>> My concern is you are making look like Xen will be able to deal nicely with big.LITTLE when in fact there are a lot more potential issue by allow a vCPU moving accross pCPU of different type (the errata is one example).
>>>>
>>>> I agree and this is why Xen is tainted.
>>>>
>>>>>
>>>>>> An other solution would be to discard this patch from the serie for now until
>>>>>> I have worked a proper solution for this case.
>>>>>> Should we discard or merge or do you have an other idea ?
>>>>> Please correct me if I am wrong, at the moment, it doesn't look like this patch will be part of the longer plan. If so, then I think it should be parked for now.
>>>>
>>>> Not sure it depends on what the final solution would be but this is highly possible I agree.
>>>>
>>>>>
>>>>> This would also have the advantage to avoid spending too much time on resolving the emulation issue I mentioned in my previous answer.
>>>>>
>>>>> No need to resend a new version of this series yet. You can wait until the rest of the series get more feedback.
>>>>
>>>> Ok, I will wait for feedback and next serie will not include this patch anymore.
>>>
>>> Would it be worth keeping just the part that sanitizes ctr, without any
>>> of the emulation stuff? That way we could still detect cases where there
>>> is a mismatch between CPUs, print a useful message and taint Xen.
>> That’s a good idea, it means removing the emulation part and just keep the sanitization.
>> @Julien: would you be ok with that ?
>
> I actually thought about suggesting it before Stefano did it. So I am OK with that.
>
>> Should I send a v4 or should we use Stefano’s patch directly instead ?
>
> I would suggest to send a v4. This needs a signed-off-by from Stefano and a new commit message.
Ok I will do that beginning of next week.
Cheers
Bertrand
>
> Cheers,
>
> --
> Julien Grall
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [PATCH v3 6/7] xen/arm: Taint Xen on incompatible DCZID values
2021-09-06 7:59 ` Bertrand Marquis
@ 2021-09-07 23:17 ` Stefano Stabellini
0 siblings, 0 replies; 26+ messages in thread
From: Stefano Stabellini @ 2021-09-07 23:17 UTC (permalink / raw)
To: Bertrand Marquis
Cc: Stefano Stabellini, xen-devel, Julien Grall, Volodymyr Babchuk
On Mon, 6 Sep 2021, Bertrand Marquis wrote:
> > On 3 Sep 2021, at 23:49, Stefano Stabellini <sstabellini@kernel.org> wrote:
> >
> > On Wed, 25 Aug 2021, Bertrand Marquis wrote:
> >> Use arm64 cpu feature sanitization to TAIN Xen if different DCZID values
> > ^ TAINT
> >
> >
> >> are found (ftr_dczid is using only STRICT method).
> >> In this case actual memory being cleaned by DC ZVA operations would be
> >> different depending on the cores which could make a guest zeroing too
> >> much or too little memory if it is merged between CPUs.
> >>
> >> We could, on processor supporting it, trap access to DCZID_EL0 register
> > ^ processors
>
> Could those typos be fixed during commit ?
Yes they can
> >> using HFGRTR_EL2 register but this would not solve the case where a
> >> process is being migrated during a copy or if it cached the value of the
> >> register.
> >>
> >> Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
> >> ---
> >> Change in v3: none
> >> Change in v2: Patch introduced in v2
> >> ---
> >> xen/arch/arm/arm64/cpufeature.c | 14 +++++++++++---
> >> xen/arch/arm/cpufeature.c | 2 ++
> >> xen/include/asm-arm/cpufeature.h | 8 ++++++++
> >> 3 files changed, 21 insertions(+), 3 deletions(-)
> >>
> >> diff --git a/xen/arch/arm/arm64/cpufeature.c b/xen/arch/arm/arm64/cpufeature.c
> >> index 61f629ebaa..b1936ef1d6 100644
> >> --- a/xen/arch/arm/arm64/cpufeature.c
> >> +++ b/xen/arch/arm/arm64/cpufeature.c
> >> @@ -329,14 +329,11 @@ static const struct arm64_ftr_bits ftr_mvfr2[] = {
> >> ARM64_FTR_END,
> >> };
> >>
> >> -#if 0
> >> -/* TODO: handle this when sanitizing cache related registers */
> >> static const struct arm64_ftr_bits ftr_dczid[] = {
> >> ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_EXACT, DCZID_DZP_SHIFT, 1, 1),
> >> ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, DCZID_BS_SHIFT, 4, 0),
> >> ARM64_FTR_END,
> >> };
> >> -#endif
> >>
> >> static const struct arm64_ftr_bits ftr_id_isar0[] = {
> >> ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR0_DIVIDE_SHIFT, 4, 0),
> >> @@ -592,6 +589,17 @@ void update_system_features(const struct cpuinfo_arm *new)
> >>
> >> SANITIZE_ID_REG(zfr64, 0, aa64zfr0);
> >>
> >> + /*
> >> + * Comment from Linux:
> >
> > I don't know if I would keep or remove "Comment from Linux"
>
> I added that because the comment itself does not really apply to Xen.
> I could have rephrased the comment/
> Anyway I have no objection to remove that statement.
>
> Do I need to send a v2 for that ?
No you don't need to resend just for that or the typos. However if you
are going to resend, then an update would be nice :-)
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [PATCH v3 7/7] xen/arm: Sanitize CTR_EL0 and emulate it if needed
2021-09-07 7:37 ` Bertrand Marquis
@ 2021-09-07 23:17 ` Stefano Stabellini
0 siblings, 0 replies; 26+ messages in thread
From: Stefano Stabellini @ 2021-09-07 23:17 UTC (permalink / raw)
To: Bertrand Marquis
Cc: Julien Grall, Stefano Stabellini, xen-devel, Volodymyr Babchuk
[-- Attachment #1: Type: text/plain, Size: 4195 bytes --]
On Tue, 7 Sep 2021, Bertrand Marquis wrote:
> Hi Julien,
>
> > On 6 Sep 2021, at 18:36, Julien Grall <julien@xen.org> wrote:
> >
> > Hi Bertrand,
> >
> > On 06/09/2021 09:29, Bertrand Marquis wrote:
> >>> On 3 Sep 2021, at 23:49, Stefano Stabellini <sstabellini@kernel.org> wrote:
> >>>
> >>> On Tue, 31 Aug 2021, Bertrand Marquis wrote:
> >>>> Hi Julien,
> >>>>
> >>>>> On 31 Aug 2021, at 15:47, Julien Grall <julien@xen.org> wrote:
> >>>>>
> >>>>>
> >>>>>
> >>>>> On 31/08/2021 14:17, Bertrand Marquis wrote:
> >>>>>> Hi Julien,
> >>>>>
> >>>>> Hi Bertrand,
> >>>>>
> >>>>>>> On 27 Aug 2021, at 16:05, Julien Grall <julien@xen.org> wrote:
> >>>>>>>
> >>>>>>> Hi Bertrand,
> >>>>>>>
> >>>>>>> On 25/08/2021 14:18, Bertrand Marquis wrote:
> >>>>>>>> Sanitize CTR_EL0 value between cores.
> >>>>>>>> In most cases different values will taint Xen but if different
> >>>>>>>> i-cache policies are found, we choose the one which will be compatible
> >>>>>>>> between all cores in terms of invalidation/data cache flushing strategy.
> >>>>>>>
> >>>>>>> I understand that all the CPUs in Xen needs to agree on the cache flush strategy. However...
> >>>>>>>
> >>>>>>>> In this case we need to activate the TID2 bit in HCR to emulate the
> >>>>>>>> TCR_EL0 register for guests. This patch is not activating TID2 bit all
> >>>>>>>> the time to limit the overhead when possible.
> >>>>>>>
> >>>>>>> as we discussed in an earlier version, a vCPU is unlikely (at least in short/medium) to be able move across pCPU of different type. So the vCPU would be pinned to a set of pCPUs. IOW, the guest would have to be big.LITTLE aware and therefore would be able to do its own strategy decision.
> >>>>>>>
> >>>>>>> So I think we should be able to get away from trappings the registers.
> >>>>>> I do agree that we should be able to get away from that in the long term once
> >>>>>> we have cpupools properly set but right now this is the only way to have
> >>>>>> something useable (I will not say right).
> >>>>>> I will work on finding a way to setup properly cpupools (or something else as
> >>>>>> we discussed earlier) but in the short term I think this is the best we can do.
> >>>>>
> >>>>> My concern is you are making look like Xen will be able to deal nicely with big.LITTLE when in fact there are a lot more potential issue by allow a vCPU moving accross pCPU of different type (the errata is one example).
> >>>>
> >>>> I agree and this is why Xen is tainted.
> >>>>
> >>>>>
> >>>>>> An other solution would be to discard this patch from the serie for now until
> >>>>>> I have worked a proper solution for this case.
> >>>>>> Should we discard or merge or do you have an other idea ?
> >>>>> Please correct me if I am wrong, at the moment, it doesn't look like this patch will be part of the longer plan. If so, then I think it should be parked for now.
> >>>>
> >>>> Not sure it depends on what the final solution would be but this is highly possible I agree.
> >>>>
> >>>>>
> >>>>> This would also have the advantage to avoid spending too much time on resolving the emulation issue I mentioned in my previous answer.
> >>>>>
> >>>>> No need to resend a new version of this series yet. You can wait until the rest of the series get more feedback.
> >>>>
> >>>> Ok, I will wait for feedback and next serie will not include this patch anymore.
> >>>
> >>> Would it be worth keeping just the part that sanitizes ctr, without any
> >>> of the emulation stuff? That way we could still detect cases where there
> >>> is a mismatch between CPUs, print a useful message and taint Xen.
> >> That’s a good idea, it means removing the emulation part and just keep the sanitization.
> >> @Julien: would you be ok with that ?
> >
> > I actually thought about suggesting it before Stefano did it. So I am OK with that.
> >
> >> Should I send a v4 or should we use Stefano’s patch directly instead ?
> >
> > I would suggest to send a v4. This needs a signed-off-by from Stefano and a new commit message.
>
> Ok I will do that beginning of next week.
Of course you can add my signed-off-by and even my reviewed-by (although
it will look weird as it seems I reviewed my own patch)
^ permalink raw reply [flat|nested] 26+ messages in thread
end of thread, other threads:[~2021-09-07 23:17 UTC | newest]
Thread overview: 26+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-08-25 13:18 [PATCH v3 0/7] xen/arm: Sanitize cpuinfo Bertrand Marquis
2021-08-25 13:18 ` [PATCH v3 1/7] xen/arm: Import ID registers definitions from Linux Bertrand Marquis
2021-08-27 14:40 ` Julien Grall
2021-08-25 13:18 ` [PATCH v3 2/7] xen/arm: Import ID features sanitize from linux Bertrand Marquis
2021-09-03 22:48 ` Stefano Stabellini
2021-08-25 13:18 ` [PATCH v3 3/7] xen/arm: Rename cpu_boot_data to system_cpuinfo Bertrand Marquis
2021-09-03 22:48 ` Stefano Stabellini
2021-08-25 13:18 ` [PATCH v3 4/7] xen/arm: Sanitize cpuinfo ID registers fields Bertrand Marquis
2021-09-03 22:48 ` Stefano Stabellini
2021-09-06 8:24 ` Bertrand Marquis
2021-08-25 13:18 ` [PATCH v3 5/7] xen/arm: Use sanitize values for p2m Bertrand Marquis
2021-09-03 22:48 ` Stefano Stabellini
2021-08-25 13:18 ` [PATCH v3 6/7] xen/arm: Taint Xen on incompatible DCZID values Bertrand Marquis
2021-09-03 22:49 ` Stefano Stabellini
2021-09-06 7:59 ` Bertrand Marquis
2021-09-07 23:17 ` Stefano Stabellini
2021-08-25 13:18 ` [PATCH v3 7/7] xen/arm: Sanitize CTR_EL0 and emulate it if needed Bertrand Marquis
2021-08-27 15:05 ` Julien Grall
2021-08-31 13:17 ` Bertrand Marquis
2021-08-31 14:47 ` Julien Grall
2021-08-31 16:15 ` Bertrand Marquis
2021-09-03 22:49 ` Stefano Stabellini
2021-09-06 8:29 ` Bertrand Marquis
2021-09-06 17:36 ` Julien Grall
2021-09-07 7:37 ` Bertrand Marquis
2021-09-07 23:17 ` Stefano Stabellini
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.