[RFC PATCH v3 00/29] KVM: arm64: Make CPU ID registers writable by userspace

[RFC PATCH v3 00/29] KVM: arm64: Make CPU ID registers writable by userspace

[Reiji Watanabe]

In KVM/arm64, values of ID registers for a guest are mostly same as
its host's values except for bits for feature that KVM doesn't support
and for opt-in features that userspace didn't configure.  Userspace
can use KVM_SET_ONE_REG to a set ID register value, but it fails
if userspace attempts to modify the register value.

This patch series adds support to allow userspace to modify a value of
ID registers (as long as KVM can support features that are indicated
in the registers) so userspace can have more control of configuring
and unconfiguring features for guests.

The patch series is for both VHE and non-VHE, except for protected VMs,
which have a different way of configuring ID registers based on its
different requirements [1].
There was a patch series that tried to achieve the same thing [2].
A few snippets of codes in this series were inspired by or came from [2].

The initial value of ID registers for a vCPU will be the host's value
with bits cleared for unsupported features and for opt-in features that
were not configured. So, the initial value userspace can see (via
KVM_GET_ONE_REG) is the upper limit that can be set for the register.
Any requests to change the value that conflicts with opt-in features'
configuration will fail.

When a guest tries to use a CPU feature that is not exposed to the guest,
trapping it (to emulate a real CPU's behavior) would generally be a
desirable behavior (when it is possible with no or little side effects).
The later patches in the series add codes for this.  Only features that
can be trapped independently will be trapped by this series though.

This series adds kunit tests for new functions in sys_regs.c (except for
trivial ones), and these tests are enabled with a new configuration
option 'CONFIG_KVM_KUNIT_TEST'.

The series is based on v5.16-rc1.

v3:
  - Remove ID register consistency checking across vCPUs [Oliver]
  - Change KVM_CAP_ARM_ID_REG_WRITABLE to
    KVM_CAP_ARM_ID_REG_CONFIGURABLE [Oliver]
  - Add KUnit testing for ID register validation and trap initialization.
  - Change read_id_reg() to take care of ID_AA64PFR0_EL1.GIC
  - Add a helper of read_id_reg() (__read_id_reg()) and use the helper
    instead of directly using __vcpu_sys_reg()
  - Change not to run kvm_id_regs_consistency_check() and
    kvm_vcpu_init_traps() for protected VMs.
  - Update selftest to remove test cases for ID register consistency
    checking across vCPUs and to add test cases for ID_AA64PFR0_EL1.GIC.

v2: https://lore.kernel.org/all/20211103062520.1445832-1-reijiw@google.com/
  - Remove unnecessary line breaks. [Andrew]
  - Use @params for comments. [Andrew]
  - Move arm64_check_features to arch/arm64/kvm/sys_regs.c and
    change that KVM specific feature check function.  [Andrew]
  - Remove unnecessary raz handling from __set_id_reg. [Andrew]
  - Remove sys_val field from the initial id_reg_info and add it
    in the later patch. [Andrew]
  - Call id_reg->init() from id_reg_info_init(). [Andrew]
  - Fix cpuid_feature_cap_perfmon_field() to convert 0xf to 0x0
    (and use it in the following patches).
  - Change kvm_vcpu_first_run_init to set has_run_once to false
    when kvm_id_regs_consistency_check() fails.
  - Add a patch to introduce id_reg_info for ID_AA64MMFR0_EL1,
    which requires special validity checking for TGran*_2 fields.
  - Add patches to introduce id_reg_info for ID_DFR1_EL1 and
    ID_MMFR0_EL1, which are required due to arm64_check_features
    implementation change.
  - Add a new argument, which is a pointer to id_reg_info, for
    id_reg_info's validate()

v1: https://lore.kernel.org/all/20211012043535.500493-1-reijiw@google.com/

[1] https://lore.kernel.org/kvmarm/20211010145636.1950948-1-tabba@google.com/
[2] https://lore.kernel.org/kvm/20201102033422.657391-1-liangpeng10@huawei.com/

Reiji Watanabe (29):
  KVM: arm64: Add has_reset_once flag for vcpu
  KVM: arm64: Save ID registers' sanitized value per vCPU
  KVM: arm64: Introduce struct id_reg_info
  KVM: arm64: Make ID_AA64PFR0_EL1 writable
  KVM: arm64: Make ID_AA64PFR1_EL1 writable
  KVM: arm64: Make ID_AA64ISAR0_EL1 writable
  KVM: arm64: Make ID_AA64ISAR1_EL1 writable
  KVM: arm64: Make ID_AA64MMFR0_EL1 writable
  KVM: arm64: Hide IMPLEMENTATION DEFINED PMU support for the guest
  KVM: arm64: Make ID_AA64DFR0_EL1 writable
  KVM: arm64: Make ID_DFR0_EL1 writable
  KVM: arm64: Make ID_DFR1_EL1 writable
  KVM: arm64: Make ID_MMFR0_EL1 writable
  KVM: arm64: Make MVFR1_EL1 writable
  KVM: arm64: Make ID registers without id_reg_info writable
  KVM: arm64: Add consistency checking for frac fields of ID registers
  KVM: arm64: Introduce KVM_CAP_ARM_ID_REG_CONFIGURABLE capability
  KVM: arm64: Add kunit test for ID register validation
  KVM: arm64: Use vcpu->arch cptr_el2 to track value of cptr_el2 for VHE
  KVM: arm64: Use vcpu->arch.mdcr_el2 to track value of mdcr_el2
  KVM: arm64: Introduce framework to trap disabled features
  KVM: arm64: Trap disabled features of ID_AA64PFR0_EL1
  KVM: arm64: Trap disabled features of ID_AA64PFR1_EL1
  KVM: arm64: Trap disabled features of ID_AA64DFR0_EL1
  KVM: arm64: Trap disabled features of ID_AA64MMFR1_EL1
  KVM: arm64: Trap disabled features of ID_AA64ISAR1_EL1
  KVM: arm64: Initialize trapping of disabled CPU features for the guest
  KVM: arm64: Add kunit test for trap initialization
  KVM: arm64: selftests: Introduce id_reg_test

 Documentation/virt/kvm/api.rst                |    8 +
 arch/arm64/include/asm/cpufeature.h           |    2 +-
 arch/arm64/include/asm/kvm_arm.h              |   32 +
 arch/arm64/include/asm/kvm_host.h             |   15 +
 arch/arm64/include/asm/sysreg.h               |    2 +
 arch/arm64/kvm/Kconfig                        |   11 +
 arch/arm64/kvm/arm.c                          |   12 +-
 arch/arm64/kvm/debug.c                        |   13 +-
 arch/arm64/kvm/hyp/vhe/switch.c               |   14 +-
 arch/arm64/kvm/reset.c                        |    4 +
 arch/arm64/kvm/sys_regs.c                     | 1265 +++++++++++++++--
 arch/arm64/kvm/sys_regs_test.c                | 1109 +++++++++++++++
 include/uapi/linux/kvm.h                      |    1 +
 tools/arch/arm64/include/asm/sysreg.h         |    1 +
 tools/testing/selftests/kvm/.gitignore        |    1 +
 tools/testing/selftests/kvm/Makefile          |    1 +
 .../selftests/kvm/aarch64/id_reg_test.c       | 1128 +++++++++++++++
 17 files changed, 3488 insertions(+), 131 deletions(-)
 create mode 100644 arch/arm64/kvm/sys_regs_test.c
 create mode 100644 tools/testing/selftests/kvm/aarch64/id_reg_test.c

-- 
2.34.0.rc1.387.gb447b232ab-goog

[RFC PATCH v3 01/29] KVM: arm64: Add has_reset_once flag for vcpu

[Reiji Watanabe]

Introduce 'has_reset_once' flag in kvm_vcpu_arch, which indicates
if the vCPU reset has been done once, for later use.

Signed-off-by: Reiji Watanabe <reijiw@google.com>
Reviewed-by: Oliver Upton <oupton@google.com>
---
 arch/arm64/include/asm/kvm_host.h | 2 ++
 arch/arm64/kvm/reset.c            | 4 ++++
 2 files changed, 6 insertions(+)

diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index 2a5f7f38006f..edbe2cb21947 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -385,6 +385,7 @@ struct kvm_vcpu_arch {
 		u64 last_steal;
 		gpa_t base;
 	} steal;
+	bool has_reset_once;
 };
 
 /* Pointer to the vcpu's SVE FFR for sve_{save,load}_state() */
@@ -450,6 +451,7 @@ struct kvm_vcpu_arch {
 
 #define vcpu_has_sve(vcpu) (system_supports_sve() &&			\
 			    ((vcpu)->arch.flags & KVM_ARM64_GUEST_HAS_SVE))
+#define	vcpu_has_reset_once(vcpu) ((vcpu)->arch.has_reset_once)
 
 #ifdef CONFIG_ARM64_PTR_AUTH
 #define vcpu_has_ptrauth(vcpu)						\
diff --git a/arch/arm64/kvm/reset.c b/arch/arm64/kvm/reset.c
index 426bd7fbc3fd..c3a91ab370fa 100644
--- a/arch/arm64/kvm/reset.c
+++ b/arch/arm64/kvm/reset.c
@@ -305,6 +305,10 @@ int kvm_reset_vcpu(struct kvm_vcpu *vcpu)
 	if (loaded)
 		kvm_arch_vcpu_load(vcpu, smp_processor_id());
 	preempt_enable();
+
+	if (!ret && !vcpu->arch.has_reset_once)
+		vcpu->arch.has_reset_once = true;
+
 	return ret;
 }
 
-- 
2.34.0.rc1.387.gb447b232ab-goog

[RFC PATCH v3 02/29] KVM: arm64: Save ID registers' sanitized value per vCPU

[Reiji Watanabe]

Extend sys_regs[] of kvm_cpu_context for ID registers and save ID
registers' sanitized value in the array for the vCPU at the first
vCPU reset. Use the saved ones when ID registers are read by
userspace (via KVM_GET_ONE_REG) or the guest.

Signed-off-by: Reiji Watanabe <reijiw@google.com>
---
 arch/arm64/include/asm/kvm_host.h | 10 +++++++
 arch/arm64/kvm/sys_regs.c         | 43 +++++++++++++++++++------------
 2 files changed, 37 insertions(+), 16 deletions(-)

diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index edbe2cb21947..72db73c79403 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -146,6 +146,14 @@ struct kvm_vcpu_fault_info {
 	u64 disr_el1;		/* Deferred [SError] Status Register */
 };
 
+/*
+ * (Op0, Op1, CRn, CRm, Op2) of ID registers is (3, 0, 0, crm, op2),
+ * where 0<=crm<8, 0<=op2<8.
+ */
+#define KVM_ARM_ID_REG_MAX_NUM 64
+#define IDREG_IDX(id)		((sys_reg_CRm(id) << 3) | sys_reg_Op2(id))
+#define IDREG_SYS_IDX(id)	(ID_REG_BASE + IDREG_IDX(id))
+
 enum vcpu_sysreg {
 	__INVALID_SYSREG__,   /* 0 is reserved as an invalid value */
 	MPIDR_EL1,	/* MultiProcessor Affinity Register */
@@ -210,6 +218,8 @@ enum vcpu_sysreg {
 	CNTP_CVAL_EL0,
 	CNTP_CTL_EL0,
 
+	ID_REG_BASE,
+	ID_REG_END = ID_REG_BASE + KVM_ARM_ID_REG_MAX_NUM - 1,
 	/* Memory Tagging Extension registers */
 	RGSR_EL1,	/* Random Allocation Tag Seed Register */
 	GCR_EL1,	/* Tag Control Register */
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index e3ec1a44f94d..5608d3410660 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -33,6 +33,8 @@
 
 #include "trace.h"
 
+static u64 __read_id_reg(const struct kvm_vcpu *vcpu, u32 id);
+
 /*
  * All of this file is extremely similar to the ARM coproc.c, but the
  * types are different. My gut feeling is that it should be pretty
@@ -273,7 +275,7 @@ static bool trap_loregion(struct kvm_vcpu *vcpu,
 			  struct sys_reg_params *p,
 			  const struct sys_reg_desc *r)
 {
-	u64 val = read_sanitised_ftr_reg(SYS_ID_AA64MMFR1_EL1);
+	u64 val = __read_id_reg(vcpu, SYS_ID_AA64MMFR1_EL1);
 	u32 sr = reg_to_encoding(r);
 
 	if (!(val & (0xfUL << ID_AA64MMFR1_LOR_SHIFT))) {
@@ -1059,17 +1061,9 @@ static bool access_arch_timer(struct kvm_vcpu *vcpu,
 	return true;
 }
 
-/* Read a sanitised cpufeature ID register by sys_reg_desc */
-static u64 read_id_reg(const struct kvm_vcpu *vcpu,
-		struct sys_reg_desc const *r, bool raz)
+static u64 __read_id_reg(const struct kvm_vcpu *vcpu, u32 id)
 {
-	u32 id = reg_to_encoding(r);
-	u64 val;
-
-	if (raz)
-		return 0;
-
-	val = read_sanitised_ftr_reg(id);
+	u64 val = __vcpu_sys_reg(vcpu, IDREG_SYS_IDX(id));
 
 	switch (id) {
 	case SYS_ID_AA64PFR0_EL1:
@@ -1119,6 +1113,14 @@ static u64 read_id_reg(const struct kvm_vcpu *vcpu,
 	return val;
 }
 
+static u64 read_id_reg(const struct kvm_vcpu *vcpu,
+		       struct sys_reg_desc const *r, bool raz)
+{
+	u32 id = reg_to_encoding(r);
+
+	return raz ? 0 : __read_id_reg(vcpu, id);
+}
+
 static unsigned int id_visibility(const struct kvm_vcpu *vcpu,
 				  const struct sys_reg_desc *r)
 {
@@ -1178,6 +1180,16 @@ static unsigned int sve_visibility(const struct kvm_vcpu *vcpu,
 	return REG_HIDDEN;
 }
 
+static void reset_id_reg(struct kvm_vcpu *vcpu, const struct sys_reg_desc *rd)
+{
+	u32 id = reg_to_encoding(rd);
+
+	if (vcpu_has_reset_once(vcpu))
+		return;
+
+	__vcpu_sys_reg(vcpu, IDREG_SYS_IDX(id)) = read_sanitised_ftr_reg(id);
+}
+
 static int set_id_aa64pfr0_el1(struct kvm_vcpu *vcpu,
 			       const struct sys_reg_desc *rd,
 			       const struct kvm_one_reg *reg, void __user *uaddr)
@@ -1223,9 +1235,7 @@ static int set_id_aa64pfr0_el1(struct kvm_vcpu *vcpu,
 /*
  * cpufeature ID register user accessors
  *
- * For now, these registers are immutable for userspace, so no values
- * are stored, and for set_id_reg() we don't allow the effective value
- * to be changed.
+ * We don't allow the effective value to be changed.
  */
 static int __get_id_reg(const struct kvm_vcpu *vcpu,
 			const struct sys_reg_desc *rd, void __user *uaddr,
@@ -1382,6 +1392,7 @@ static unsigned int mte_visibility(const struct kvm_vcpu *vcpu,
 #define ID_SANITISED(name) {			\
 	SYS_DESC(SYS_##name),			\
 	.access	= access_id_reg,		\
+	.reset	= reset_id_reg,			\
 	.get_user = get_id_reg,			\
 	.set_user = set_id_reg,			\
 	.visibility = id_visibility,		\
@@ -1837,8 +1848,8 @@ static bool trap_dbgdidr(struct kvm_vcpu *vcpu,
 	if (p->is_write) {
 		return ignore_write(vcpu, p);
 	} else {
-		u64 dfr = read_sanitised_ftr_reg(SYS_ID_AA64DFR0_EL1);
-		u64 pfr = read_sanitised_ftr_reg(SYS_ID_AA64PFR0_EL1);
+		u64 dfr = __read_id_reg(vcpu, SYS_ID_AA64DFR0_EL1);
+		u64 pfr = __read_id_reg(vcpu, SYS_ID_AA64PFR0_EL1);
 		u32 el3 = !!cpuid_feature_extract_unsigned_field(pfr, ID_AA64PFR0_EL3_SHIFT);
 
 		p->regval = ((((dfr >> ID_AA64DFR0_WRPS_SHIFT) & 0xf) << 28) |
-- 
2.34.0.rc1.387.gb447b232ab-goog

[RFC PATCH v3 03/29] KVM: arm64: Introduce struct id_reg_info

[Reiji Watanabe]

This patch lays the groundwork to make ID registers writable.

Introduce struct id_reg_info for an ID register to manage the
register specific control of its value for the guest, and provide set
of functions commonly used for ID registers to make them writable.

The id_reg_info is used to do register specific initialization,
validation of the ID register and etc.  Not all ID registers must
have the id_reg_info. ID registers that don't have the id_reg_info
are handled in a common way that is applied to all ID registers.

At present, changing an ID register from userspace is allowed only
if the ID register has the id_reg_info, but that will be changed
by the following patches.

No ID register has the structure yet and the following patches
will add the id_reg_info for some ID registers.

Signed-off-by: Reiji Watanabe <reijiw@google.com>
---
 arch/arm64/include/asm/sysreg.h |   1 +
 arch/arm64/kvm/sys_regs.c       | 226 ++++++++++++++++++++++++++++++--
 2 files changed, 218 insertions(+), 9 deletions(-)

diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h
index 16b3f1a1d468..597609f26331 100644
--- a/arch/arm64/include/asm/sysreg.h
+++ b/arch/arm64/include/asm/sysreg.h
@@ -1197,6 +1197,7 @@
 #define ICH_VTR_TDS_MASK	(1 << ICH_VTR_TDS_SHIFT)
 
 #define ARM64_FEATURE_FIELD_BITS	4
+#define ARM64_FEATURE_FIELD_MASK	((1ull << ARM64_FEATURE_FIELD_BITS) - 1)
 
 /* Create a mask for the feature bits of the specified feature. */
 #define ARM64_FEATURE_MASK(x)	(GENMASK_ULL(x##_SHIFT + ARM64_FEATURE_FIELD_BITS - 1, x##_SHIFT))
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index 5608d3410660..1552cd5581b7 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -265,6 +265,181 @@ static bool trap_raz_wi(struct kvm_vcpu *vcpu,
 		return read_zero(vcpu, p);
 }
 
+/*
+ * A value for FCT_LOWER_SAFE must be zero and changing that will affect
+ * ftr_check_types of id_reg_info.
+ */
+enum feature_check_type {
+	FCT_LOWER_SAFE = 0,
+	FCT_HIGHER_SAFE,
+	FCT_HIGHER_OR_ZERO_SAFE,
+	FCT_EXACT,
+	FCT_EXACT_OR_ZERO_SAFE,
+	FCT_IGNORE,	/* Don't check (any value is fine) */
+};
+
+static int arm64_check_feature_one(enum feature_check_type type, int val,
+				   int limit)
+{
+	bool is_safe = false;
+
+	if (val == limit)
+		return 0;
+
+	switch (type) {
+	case FCT_LOWER_SAFE:
+		is_safe = (val <= limit);
+		break;
+	case FCT_HIGHER_OR_ZERO_SAFE:
+		if (val == 0) {
+			is_safe = true;
+			break;
+		}
+		fallthrough;
+	case FCT_HIGHER_SAFE:
+		is_safe = (val >= limit);
+		break;
+	case FCT_EXACT:
+		break;
+	case FCT_EXACT_OR_ZERO_SAFE:
+		is_safe = (val == 0);
+		break;
+	case FCT_IGNORE:
+		is_safe = true;
+		break;
+	default:
+		WARN_ONCE(1, "Unexpected feature_check_type (%d)\n", type);
+		break;
+	}
+
+	return is_safe ? 0 : -1;
+}
+
+#define	FCT_TYPE_MASK		0x7
+#define	FCT_TYPE_SHIFT		1
+#define	FCT_SIGN_MASK		0x1
+#define	FCT_SIGN_SHIFT		0
+#define	FCT_TYPE(val)	((val >> FCT_TYPE_SHIFT) & FCT_TYPE_MASK)
+#define	FCT_SIGN(val)	((val >> FCT_SIGN_SHIFT) & FCT_SIGN_MASK)
+
+#define	MAKE_FCT(shift, type, sign)				\
+	((u64)((((type) & FCT_TYPE_MASK) << FCT_TYPE_SHIFT) |	\
+	       (((sign) & FCT_SIGN_MASK) << FCT_SIGN_SHIFT)) << (shift))
+
+/* For signed field */
+#define	S_FCT(shift, type)	MAKE_FCT(shift, type, 1)
+/* For unigned field */
+#define	U_FCT(shift, type)	MAKE_FCT(shift, type, 0)
+
+/*
+ * @val and @lim are both a value of the ID register. The function checks
+ * if all features indicated in @val can be supported for guests on the host,
+ * which supports features indicated in @lim. @check_types indicates how
+ * features in the ID register needs to be checked.
+ * See comments for id_reg_info's ftr_check_types field for more detail.
+ */
+static int arm64_check_features(u64 check_types, u64 val, u64 lim)
+{
+	int i;
+
+	for (i = 0; i < 64; i += ARM64_FEATURE_FIELD_BITS) {
+		u8 ftr_check = (check_types >> i) & ARM64_FEATURE_FIELD_MASK;
+		bool is_sign = FCT_SIGN(ftr_check);
+		enum feature_check_type fctype = FCT_TYPE(ftr_check);
+		int fval, flim, ret;
+
+		fval = cpuid_feature_extract_field(val, i, is_sign);
+		flim = cpuid_feature_extract_field(lim, i, is_sign);
+
+		ret = arm64_check_feature_one(fctype, fval, flim);
+		if (ret)
+			return -E2BIG;
+	}
+	return 0;
+}
+
+struct id_reg_info {
+	u32	sys_reg;	/* Register ID */
+
+	/*
+	 * Limit value of the register for a vcpu. The value is the sanitized
+	 * system value with bits cleared for unsupported features for the
+	 * guest.
+	 */
+	u64	vcpu_limit_val;
+
+	/*
+	 * The ftr_check_types is comprised of a set of 4 bits fields.
+	 * Each 4 bits field is for a feature indicated by the same bits
+	 * field of the ID register and indicates how the feature support
+	 * for guests needs to be checked.
+	 * The bit 0 indicates that the corresponding ID register field
+	 * is signed(1) or unsigned(0).
+	 * The bits [3:1] hold feature_check_type for the field.
+	 * If all zero, all features in the ID register are treated as unsigned
+	 * fields and checked based on Principles of the ID scheme for fields
+	 * in ID registers (FCT_LOWER_SAFE of feature_check_type).
+	 */
+	u64	ftr_check_types;
+
+	/* Initialization function of the id_reg_info */
+	void (*init)(struct id_reg_info *id_reg);
+
+	/* Register specific validation function */
+	int (*validate)(struct kvm_vcpu *vcpu, const struct id_reg_info *id_reg,
+			u64 val);
+
+	/* Return the reset value of the register for the vCPU */
+	u64 (*get_reset_val)(struct kvm_vcpu *vcpu,
+			     const struct id_reg_info *id_reg);
+};
+
+static void id_reg_info_init(struct id_reg_info *id_reg)
+{
+	id_reg->vcpu_limit_val = read_sanitised_ftr_reg(id_reg->sys_reg);
+	if (id_reg->init)
+		id_reg->init(id_reg);
+}
+
+/*
+ * An ID register that needs special handling to control the value for the
+ * guest must have its own id_reg_info in id_reg_info_table.
+ * (i.e. the reset value is different from the host's sanitized value,
+ * the value is affected by opt-in features, some fields needs specific
+ * validation, etc.)
+ */
+#define	GET_ID_REG_INFO(id)	(id_reg_info_table[IDREG_IDX(id)])
+static struct id_reg_info *id_reg_info_table[KVM_ARM_ID_REG_MAX_NUM] = {};
+
+static int validate_id_reg(struct kvm_vcpu *vcpu,
+			   const struct sys_reg_desc *rd, u64 val)
+{
+	u32 id = reg_to_encoding(rd);
+	const struct id_reg_info *id_reg = GET_ID_REG_INFO(id);
+	u64 limit, check_types;
+	int err;
+
+	if (id_reg) {
+		check_types = id_reg->ftr_check_types;
+		limit = id_reg->vcpu_limit_val;
+	} else {
+		/* All fields are treated as unsigned and FCT_LOWER_SAFE */
+		check_types = 0;
+		limit = read_sanitised_ftr_reg(id);
+	}
+
+	/* Check if the value indicates any feature that is not in the limit. */
+	err = arm64_check_features(check_types, val, limit);
+	if (err)
+		return err;
+
+	if (id_reg && id_reg->validate)
+		/* Run the ID register specific validity check. */
+		err = id_reg->validate(vcpu, id_reg, val);
+
+	return err;
+}
+
 /*
  * ARMv8.1 mandates at least a trivial LORegion implementation, where all the
  * RW registers are RES0 (which we can implement as RAZ/WI). On an ARMv8.0
@@ -1183,11 +1358,19 @@ static unsigned int sve_visibility(const struct kvm_vcpu *vcpu,
 static void reset_id_reg(struct kvm_vcpu *vcpu, const struct sys_reg_desc *rd)
 {
 	u32 id = reg_to_encoding(rd);
+	struct id_reg_info *id_reg;
+	u64 val;
 
 	if (vcpu_has_reset_once(vcpu))
 		return;
 
-	__vcpu_sys_reg(vcpu, IDREG_SYS_IDX(id)) = read_sanitised_ftr_reg(id);
+	id_reg = GET_ID_REG_INFO(id);
+	if (id_reg && id_reg->get_reset_val)
+		val = id_reg->get_reset_val(vcpu, id_reg);
+	else
+		val = read_sanitised_ftr_reg(id);
+
+	__vcpu_sys_reg(vcpu, IDREG_SYS_IDX(id)) = val;
 }
 
 static int set_id_aa64pfr0_el1(struct kvm_vcpu *vcpu,
@@ -1232,11 +1415,7 @@ static int set_id_aa64pfr0_el1(struct kvm_vcpu *vcpu,
 	return 0;
 }
 
-/*
- * cpufeature ID register user accessors
- *
- * We don't allow the effective value to be changed.
- */
+/* cpufeature ID register user accessors */
 static int __get_id_reg(const struct kvm_vcpu *vcpu,
 			const struct sys_reg_desc *rd, void __user *uaddr,
 			bool raz)
@@ -1247,11 +1426,12 @@ static int __get_id_reg(const struct kvm_vcpu *vcpu,
 	return reg_to_user(uaddr, &val, id);
 }
 
-static int __set_id_reg(const struct kvm_vcpu *vcpu,
+static int __set_id_reg(struct kvm_vcpu *vcpu,
 			const struct sys_reg_desc *rd, void __user *uaddr,
 			bool raz)
 {
 	const u64 id = sys_reg_to_index(rd);
+	u32 encoding = reg_to_encoding(rd);
 	int err;
 	u64 val;
 
@@ -1259,10 +1439,22 @@ static int __set_id_reg(const struct kvm_vcpu *vcpu,
 	if (err)
 		return err;
 
-	/* This is what we mean by invariant: you can't change it. */
-	if (val != read_id_reg(vcpu, rd, raz))
+	/* Don't allow to change the reg unless the reg has id_reg_info */
+	if (val != read_id_reg(vcpu, rd, raz) && !GET_ID_REG_INFO(encoding))
 		return -EINVAL;
 
+	if (raz)
+		return 0;
+
+	/* Don't allow to change the reg after the first KVM_RUN. */
+	if (vcpu->arch.has_run_once)
+		return -EINVAL;
+
+	err = validate_id_reg(vcpu, rd, val);
+	if (err)
+		return err;
+
+	__vcpu_sys_reg(vcpu, IDREG_SYS_IDX(encoding)) = val;
 	return 0;
 }
 
@@ -2826,6 +3018,20 @@ int kvm_arm_copy_sys_reg_indices(struct kvm_vcpu *vcpu, u64 __user *uindices)
 	return write_demux_regids(uindices);
 }
 
+static void id_reg_info_init_all(void)
+{
+	int i;
+	struct id_reg_info *id_reg;
+
+	for (i = 0; i < ARRAY_SIZE(id_reg_info_table); i++) {
+		id_reg = (struct id_reg_info *)id_reg_info_table[i];
+		if (!id_reg)
+			continue;
+
+		id_reg_info_init(id_reg);
+	}
+}
+
 void kvm_sys_reg_table_init(void)
 {
 	unsigned int i;
@@ -2860,4 +3066,6 @@ void kvm_sys_reg_table_init(void)
 			break;
 	/* Clear all higher bits. */
 	cache_levels &= (1 << (i*3))-1;
+
+	id_reg_info_init_all();
 }
-- 
2.34.0.rc1.387.gb447b232ab-goog

[RFC PATCH v3 04/29] KVM: arm64: Make ID_AA64PFR0_EL1 writable

[Reiji Watanabe]

This patch adds id_reg_info for ID_AA64PFR0_EL1 to make it writable by
userspace.

The CSV2/CSV3 fields of the register were already writable and values
that were written for them affected all vCPUs before. Now they only
affect the vCPU.
Return an error if userspace tries to set SVE/GIC field of the register
to a value that conflicts with SVE/GIC configuration for the guest.
SIMD/FP/SVE fields of the requested value are validated according to
Arm ARM.

Signed-off-by: Reiji Watanabe <reijiw@google.com>
---
 arch/arm64/kvm/sys_regs.c | 159 ++++++++++++++++++++++++--------------
 1 file changed, 103 insertions(+), 56 deletions(-)

diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index 1552cd5581b7..35400869067a 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -401,6 +401,92 @@ static void id_reg_info_init(struct id_reg_info *id_reg)
 		id_reg->init(id_reg);
 }
 
+#define	kvm_has_gic3(kvm)		\
+	(irqchip_in_kernel(kvm) &&	\
+	 (kvm)->arch.vgic.vgic_model == KVM_DEV_TYPE_ARM_VGIC_V3)
+
+static int validate_id_aa64pfr0_el1(struct kvm_vcpu *vcpu,
+				    const struct id_reg_info *id_reg, u64 val)
+{
+	int fp, simd;
+	bool vcpu_has_sve = vcpu_has_sve(vcpu);
+	bool pfr0_has_sve = id_aa64pfr0_sve(val);
+	int gic;
+
+	simd = cpuid_feature_extract_signed_field(val, ID_AA64PFR0_ASIMD_SHIFT);
+	fp = cpuid_feature_extract_signed_field(val, ID_AA64PFR0_FP_SHIFT);
+	if (simd != fp)
+		return -EINVAL;
+
+	/* fp must be supported when sve is supported */
+	if (pfr0_has_sve && (fp < 0))
+		return -EINVAL;
+
+	/* Check if there is a conflict with a request via KVM_ARM_VCPU_INIT */
+	if (vcpu_has_sve ^ pfr0_has_sve)
+		return -EPERM;
+
+	gic = cpuid_feature_extract_unsigned_field(val, ID_AA64PFR0_GIC_SHIFT);
+	if ((gic > 0) ^ kvm_has_gic3(vcpu->kvm))
+		return -EPERM;
+
+	return 0;
+}
+
+static void init_id_aa64pfr0_el1_info(struct id_reg_info *id_reg)
+{
+	u64 limit = id_reg->vcpu_limit_val;
+	unsigned int gic;
+
+	limit &= ~ARM64_FEATURE_MASK(ID_AA64PFR0_AMU);
+	if (!system_supports_sve())
+		limit &= ~ARM64_FEATURE_MASK(ID_AA64PFR0_SVE);
+
+	/*
+	 * The default is to expose CSV2 == 1 and CSV3 == 1 if the HW
+	 * isn't affected.  Userspace can override this as long as it
+	 * doesn't promise the impossible.
+	 */
+	limit &= ~(ARM64_FEATURE_MASK(ID_AA64PFR0_CSV2) |
+		   ARM64_FEATURE_MASK(ID_AA64PFR0_CSV3));
+
+	if (arm64_get_spectre_v2_state() == SPECTRE_UNAFFECTED)
+		limit |= FIELD_PREP(ARM64_FEATURE_MASK(ID_AA64PFR0_CSV2), 1);
+	if (arm64_get_meltdown_state() == SPECTRE_UNAFFECTED)
+		limit |= FIELD_PREP(ARM64_FEATURE_MASK(ID_AA64PFR0_CSV3), 1);
+
+	gic = cpuid_feature_extract_unsigned_field(limit, ID_AA64PFR0_GIC_SHIFT);
+	if (gic > 1) {
+		/* Limit to GICv3.0/4.0 */
+		limit &= ~ARM64_FEATURE_MASK(ID_AA64PFR0_GIC);
+		limit |= FIELD_PREP(ARM64_FEATURE_MASK(ID_AA64PFR0_GIC), 1);
+	}
+	id_reg->vcpu_limit_val = limit;
+}
+
+static u64 get_reset_id_aa64pfr0_el1(struct kvm_vcpu *vcpu,
+				     const struct id_reg_info *idr)
+{
+	u64 val = idr->vcpu_limit_val;
+
+	if (!vcpu_has_sve(vcpu))
+		val &= ~ARM64_FEATURE_MASK(ID_AA64PFR0_SVE);
+
+	if (!kvm_has_gic3(vcpu->kvm))
+		val &= ~ARM64_FEATURE_MASK(ID_AA64PFR0_GIC);
+
+	return val;
+}
+
+static struct id_reg_info id_aa64pfr0_el1_info = {
+	.sys_reg = SYS_ID_AA64PFR0_EL1,
+	.ftr_check_types = S_FCT(ID_AA64PFR0_ASIMD_SHIFT, FCT_LOWER_SAFE) |
+			   S_FCT(ID_AA64PFR0_FP_SHIFT, FCT_LOWER_SAFE),
+	.init = init_id_aa64pfr0_el1_info,
+	.validate = validate_id_aa64pfr0_el1,
+	.get_reset_val = get_reset_id_aa64pfr0_el1,
+};
+
 /*
  * An ID register that needs special handling to control the value for the
  * guest must have its own id_reg_info in id_reg_info_table.
@@ -409,7 +495,9 @@ static void id_reg_info_init(struct id_reg_info *id_reg)
  * validation, etc.)
  */
 #define	GET_ID_REG_INFO(id)	(id_reg_info_table[IDREG_IDX(id)])
-static struct id_reg_info *id_reg_info_table[KVM_ARM_ID_REG_MAX_NUM] = {};
+static struct id_reg_info *id_reg_info_table[KVM_ARM_ID_REG_MAX_NUM] = {
+	[IDREG_IDX(SYS_ID_AA64PFR0_EL1)] = &id_aa64pfr0_el1_info,
+};
 
 static int validate_id_reg(struct kvm_vcpu *vcpu,
 			   const struct sys_reg_desc *rd, u64 val)
@@ -1239,20 +1327,22 @@ static bool access_arch_timer(struct kvm_vcpu *vcpu,
 static u64 __read_id_reg(const struct kvm_vcpu *vcpu, u32 id)
 {
 	u64 val = __vcpu_sys_reg(vcpu, IDREG_SYS_IDX(id));
+	u64 lim, gic, gic_lim;
+	const struct id_reg_info *id_reg;
 
 	switch (id) {
 	case SYS_ID_AA64PFR0_EL1:
-		if (!vcpu_has_sve(vcpu))
-			val &= ~ARM64_FEATURE_MASK(ID_AA64PFR0_SVE);
-		val &= ~ARM64_FEATURE_MASK(ID_AA64PFR0_AMU);
-		val &= ~ARM64_FEATURE_MASK(ID_AA64PFR0_CSV2);
-		val |= FIELD_PREP(ARM64_FEATURE_MASK(ID_AA64PFR0_CSV2), (u64)vcpu->kvm->arch.pfr0_csv2);
-		val &= ~ARM64_FEATURE_MASK(ID_AA64PFR0_CSV3);
-		val |= FIELD_PREP(ARM64_FEATURE_MASK(ID_AA64PFR0_CSV3), (u64)vcpu->kvm->arch.pfr0_csv3);
-		if (irqchip_in_kernel(vcpu->kvm) &&
-		    vcpu->kvm->arch.vgic.vgic_model == KVM_DEV_TYPE_ARM_VGIC_V3) {
-			val &= ~ARM64_FEATURE_MASK(ID_AA64PFR0_GIC);
-			val |= FIELD_PREP(ARM64_FEATURE_MASK(ID_AA64PFR0_GIC), 1);
+		gic = cpuid_feature_extract_unsigned_field(val, ID_AA64PFR0_GIC_SHIFT);
+		if (kvm_has_gic3(vcpu->kvm) && (gic == 0)) {
+			/*
+			 * This is a case where userspace configured gic3 after
+			 * the vcpu was created, and then it didn't set
+			 * ID_AA64PFR0_EL1.
+			 */
+			id_reg = GET_ID_REG_INFO(id);
+			lim = id_reg->vcpu_limit_val;
+			gic_lim = cpuid_feature_extract_unsigned_field(lim, ID_AA64PFR0_GIC_SHIFT);
+			val |= FIELD_PREP(ARM64_FEATURE_MASK(ID_AA64PFR0_GIC), gic_lim);
 		}
 		break;
 	case SYS_ID_AA64PFR1_EL1:
@@ -1373,48 +1463,6 @@ static void reset_id_reg(struct kvm_vcpu *vcpu, const struct sys_reg_desc *rd)
 	__vcpu_sys_reg(vcpu, IDREG_SYS_IDX(id)) = val;
 }
 
-static int set_id_aa64pfr0_el1(struct kvm_vcpu *vcpu,
-			       const struct sys_reg_desc *rd,
-			       const struct kvm_one_reg *reg, void __user *uaddr)
-{
-	const u64 id = sys_reg_to_index(rd);
-	u8 csv2, csv3;
-	int err;
-	u64 val;
-
-	err = reg_from_user(&val, uaddr, id);
-	if (err)
-		return err;
-
-	/*
-	 * Allow AA64PFR0_EL1.CSV2 to be set from userspace as long as
-	 * it doesn't promise more than what is actually provided (the
-	 * guest could otherwise be covered in ectoplasmic residue).
-	 */
-	csv2 = cpuid_feature_extract_unsigned_field(val, ID_AA64PFR0_CSV2_SHIFT);
-	if (csv2 > 1 ||
-	    (csv2 && arm64_get_spectre_v2_state() != SPECTRE_UNAFFECTED))
-		return -EINVAL;
-
-	/* Same thing for CSV3 */
-	csv3 = cpuid_feature_extract_unsigned_field(val, ID_AA64PFR0_CSV3_SHIFT);
-	if (csv3 > 1 ||
-	    (csv3 && arm64_get_meltdown_state() != SPECTRE_UNAFFECTED))
-		return -EINVAL;
-
-	/* We can only differ with CSV[23], and anything else is an error */
-	val ^= read_id_reg(vcpu, rd, false);
-	val &= ~((0xFUL << ID_AA64PFR0_CSV2_SHIFT) |
-		 (0xFUL << ID_AA64PFR0_CSV3_SHIFT));
-	if (val)
-		return -EINVAL;
-
-	vcpu->kvm->arch.pfr0_csv2 = csv2;
-	vcpu->kvm->arch.pfr0_csv3 = csv3 ;
-
-	return 0;
-}
-
 /* cpufeature ID register user accessors */
 static int __get_id_reg(const struct kvm_vcpu *vcpu,
 			const struct sys_reg_desc *rd, void __user *uaddr,
@@ -1705,8 +1753,7 @@ static const struct sys_reg_desc sys_reg_descs[] = {
 
 	/* AArch64 ID registers */
 	/* CRm=4 */
-	{ SYS_DESC(SYS_ID_AA64PFR0_EL1), .access = access_id_reg,
-	  .get_user = get_id_reg, .set_user = set_id_aa64pfr0_el1, },
+	ID_SANITISED(ID_AA64PFR0_EL1),
 	ID_SANITISED(ID_AA64PFR1_EL1),
 	ID_UNALLOCATED(4,2),
 	ID_UNALLOCATED(4,3),
-- 
2.34.0.rc1.387.gb447b232ab-goog

[RFC PATCH v3 05/29] KVM: arm64: Make ID_AA64PFR1_EL1 writable

[Reiji Watanabe]

This patch adds id_reg_info for ID_AA64PFR1_EL1 to make it writable
by userspace.

Return an error if userspace tries to set MTE field of the register
to a value that conflicts with KVM_CAP_ARM_MTE configuration for
the guest.
Skip fractional feature fields validation at present and they will
be handled by the following patches.

Signed-off-by: Reiji Watanabe <reijiw@google.com>
---
 arch/arm64/include/asm/sysreg.h |  1 +
 arch/arm64/kvm/sys_regs.c       | 44 ++++++++++++++++++++++++++++++---
 2 files changed, 41 insertions(+), 4 deletions(-)

diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h
index 597609f26331..b7ad59fd22e2 100644
--- a/arch/arm64/include/asm/sysreg.h
+++ b/arch/arm64/include/asm/sysreg.h
@@ -800,6 +800,7 @@
 #define ID_AA64PFR0_ELx_32BIT_64BIT	0x2
 
 /* id_aa64pfr1 */
+#define ID_AA64PFR1_CSV2FRAC_SHIFT	32
 #define ID_AA64PFR1_MPAMFRAC_SHIFT	16
 #define ID_AA64PFR1_RASFRAC_SHIFT	12
 #define ID_AA64PFR1_MTE_SHIFT		8
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index 35400869067a..7dc2b0d41b75 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -433,6 +433,21 @@ static int validate_id_aa64pfr0_el1(struct kvm_vcpu *vcpu,
 	return 0;
 }
 
+static int validate_id_aa64pfr1_el1(struct kvm_vcpu *vcpu,
+				    const struct id_reg_info *id_reg, u64 val)
+{
+	bool kvm_mte = kvm_has_mte(vcpu->kvm);
+	unsigned int mte;
+
+	mte = cpuid_feature_extract_unsigned_field(val, ID_AA64PFR1_MTE_SHIFT);
+
+	/* Check if there is a conflict with a request via KVM_ARM_VCPU_INIT. */
+	if (kvm_mte ^ (mte > 0))
+		return -EPERM;
+
+	return 0;
+}
+
 static void init_id_aa64pfr0_el1_info(struct id_reg_info *id_reg)
 {
 	u64 limit = id_reg->vcpu_limit_val;
@@ -464,6 +479,12 @@ static void init_id_aa64pfr0_el1_info(struct id_reg_info *id_reg)
 	id_reg->vcpu_limit_val = limit;
 }
 
+static void init_id_aa64pfr1_el1_info(struct id_reg_info *id_reg)
+{
+	if (!system_supports_mte())
+		id_reg->vcpu_limit_val &= ~ARM64_FEATURE_MASK(ID_AA64PFR1_MTE);
+}
+
 static u64 get_reset_id_aa64pfr0_el1(struct kvm_vcpu *vcpu,
 				     const struct id_reg_info *idr)
 {
@@ -478,6 +499,14 @@ static u64 get_reset_id_aa64pfr0_el1(struct kvm_vcpu *vcpu,
 	return val;
 }
 
+static u64 get_reset_id_aa64pfr1_el1(struct kvm_vcpu *vcpu,
+				     const struct id_reg_info *idr)
+{
+	return kvm_has_mte(vcpu->kvm) ?
+	       idr->vcpu_limit_val :
+	       (idr->vcpu_limit_val & ~(ARM64_FEATURE_MASK(ID_AA64PFR1_MTE)));
+}
+
 static struct id_reg_info id_aa64pfr0_el1_info = {
 	.sys_reg = SYS_ID_AA64PFR0_EL1,
 	.ftr_check_types = S_FCT(ID_AA64PFR0_ASIMD_SHIFT, FCT_LOWER_SAFE) |
@@ -487,6 +516,16 @@ static struct id_reg_info id_aa64pfr0_el1_info = {
 	.get_reset_val = get_reset_id_aa64pfr0_el1,
 };
 
+static struct id_reg_info id_aa64pfr1_el1_info = {
+	.sys_reg = SYS_ID_AA64PFR1_EL1,
+	.ftr_check_types = U_FCT(ID_AA64PFR1_RASFRAC_SHIFT, FCT_IGNORE) |
+			   U_FCT(ID_AA64PFR1_MPAMFRAC_SHIFT, FCT_IGNORE) |
+			   U_FCT(ID_AA64PFR1_CSV2FRAC_SHIFT, FCT_IGNORE),
+	.init = init_id_aa64pfr1_el1_info,
+	.validate = validate_id_aa64pfr1_el1,
+	.get_reset_val = get_reset_id_aa64pfr1_el1,
+};
+
 /*
  * An ID register that needs special handling to control the value for the
  * guest must have its own id_reg_info in id_reg_info_table.
@@ -497,6 +536,7 @@ static struct id_reg_info id_aa64pfr0_el1_info = {
 #define	GET_ID_REG_INFO(id)	(id_reg_info_table[IDREG_IDX(id)])
 static struct id_reg_info *id_reg_info_table[KVM_ARM_ID_REG_MAX_NUM] = {
 	[IDREG_IDX(SYS_ID_AA64PFR0_EL1)] = &id_aa64pfr0_el1_info,
+	[IDREG_IDX(SYS_ID_AA64PFR1_EL1)] = &id_aa64pfr1_el1_info,
 };
 
 static int validate_id_reg(struct kvm_vcpu *vcpu,
@@ -1345,10 +1385,6 @@ static u64 __read_id_reg(const struct kvm_vcpu *vcpu, u32 id)
 			val |= FIELD_PREP(ARM64_FEATURE_MASK(ID_AA64PFR0_GIC), gic_lim);
 		}
 		break;
-	case SYS_ID_AA64PFR1_EL1:
-		if (!kvm_has_mte(vcpu->kvm))
-			val &= ~ARM64_FEATURE_MASK(ID_AA64PFR1_MTE);
-		break;
 	case SYS_ID_AA64ISAR1_EL1:
 		if (!vcpu_has_ptrauth(vcpu))
 			val &= ~(ARM64_FEATURE_MASK(ID_AA64ISAR1_APA) |
-- 
2.34.0.rc1.387.gb447b232ab-goog

[RFC PATCH v3 06/29] KVM: arm64: Make ID_AA64ISAR0_EL1 writable

[Reiji Watanabe]

This patch adds id_reg_info for ID_AA64ISAR0_EL1 to make it writable
by userspace.

Updating sm3, sm4, sha1, sha2 and sha3 fields are allowed only
if values of those fields follow Arm ARM.

Signed-off-by: Reiji Watanabe <reijiw@google.com>
---
 arch/arm64/kvm/sys_regs.c | 29 +++++++++++++++++++++++++++++
 1 file changed, 29 insertions(+)

diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index 7dc2b0d41b75..fdd707462fa8 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -448,6 +448,29 @@ static int validate_id_aa64pfr1_el1(struct kvm_vcpu *vcpu,
 	return 0;
 }
 
+static int validate_id_aa64isar0_el1(struct kvm_vcpu *vcpu,
+				     const struct id_reg_info *id_reg, u64 val)
+{
+	unsigned int sm3, sm4, sha1, sha2, sha3;
+
+	/* Run consistency checkings according to Arm ARM */
+	sm3 = cpuid_feature_extract_unsigned_field(val, ID_AA64ISAR0_SM3_SHIFT);
+	sm4 = cpuid_feature_extract_unsigned_field(val, ID_AA64ISAR0_SM4_SHIFT);
+	if (sm3 != sm4)
+		return -EINVAL;
+
+	sha1 = cpuid_feature_extract_unsigned_field(val, ID_AA64ISAR0_SHA1_SHIFT);
+	sha2 = cpuid_feature_extract_unsigned_field(val, ID_AA64ISAR0_SHA2_SHIFT);
+	if ((sha1 == 0) ^ (sha2 == 0))
+		return -EINVAL;
+
+	sha3 = cpuid_feature_extract_unsigned_field(val, ID_AA64ISAR0_SHA3_SHIFT);
+	if (((sha2 == 2) ^ (sha3 == 1)) || (!sha1 && sha3))
+		return -EINVAL;
+
+	return 0;
+}
+
 static void init_id_aa64pfr0_el1_info(struct id_reg_info *id_reg)
 {
 	u64 limit = id_reg->vcpu_limit_val;
@@ -526,6 +549,11 @@ static struct id_reg_info id_aa64pfr1_el1_info = {
 	.get_reset_val = get_reset_id_aa64pfr1_el1,
 };
 
+static struct id_reg_info id_aa64isar0_el1_info = {
+	.sys_reg = SYS_ID_AA64ISAR0_EL1,
+	.validate = validate_id_aa64isar0_el1,
+};
+
 /*
  * An ID register that needs special handling to control the value for the
  * guest must have its own id_reg_info in id_reg_info_table.
@@ -537,6 +565,7 @@ static struct id_reg_info id_aa64pfr1_el1_info = {
 static struct id_reg_info *id_reg_info_table[KVM_ARM_ID_REG_MAX_NUM] = {
 	[IDREG_IDX(SYS_ID_AA64PFR0_EL1)] = &id_aa64pfr0_el1_info,
 	[IDREG_IDX(SYS_ID_AA64PFR1_EL1)] = &id_aa64pfr1_el1_info,
+	[IDREG_IDX(SYS_ID_AA64ISAR0_EL1)] = &id_aa64isar0_el1_info,
 };
 
 static int validate_id_reg(struct kvm_vcpu *vcpu,
-- 
2.34.0.rc1.387.gb447b232ab-goog

[RFC PATCH v3 07/29] KVM: arm64: Make ID_AA64ISAR1_EL1 writable

[Reiji Watanabe]

This patch adds id_reg_info for ID_AA64ISAR1_EL1 to make it
writable by userspace.

Return an error if userspace tries to set PTRAUTH related fields
of the register to values that conflict with PTRAUTH configuration,
which was configured by KVM_ARM_VCPU_INIT, for the guest.

Signed-off-by: Reiji Watanabe <reijiw@google.com>
---
 arch/arm64/kvm/sys_regs.c | 79 +++++++++++++++++++++++++++++++++++----
 1 file changed, 72 insertions(+), 7 deletions(-)

diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index fdd707462fa8..5812e39602fe 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -358,6 +358,24 @@ static int arm64_check_features(u64 check_types, u64 val, u64 lim)
 	return 0;
 }
 
+#define PTRAUTH_MASK	(ARM64_FEATURE_MASK(ID_AA64ISAR1_APA) |	\
+			 ARM64_FEATURE_MASK(ID_AA64ISAR1_API) | \
+			 ARM64_FEATURE_MASK(ID_AA64ISAR1_GPA) |	\
+			 ARM64_FEATURE_MASK(ID_AA64ISAR1_GPI))
+
+#define aa64isar1_has_apa(val)	\
+	(cpuid_feature_extract_unsigned_field(val, ID_AA64ISAR1_APA_SHIFT) >= \
+	 ID_AA64ISAR1_APA_ARCHITECTED)
+#define aa64isar1_has_api(val)	\
+	(cpuid_feature_extract_unsigned_field(val, ID_AA64ISAR1_API_SHIFT) >= \
+	 ID_AA64ISAR1_API_IMP_DEF)
+#define aa64isar1_has_gpa(val)	\
+	(cpuid_feature_extract_unsigned_field(val, ID_AA64ISAR1_GPA_SHIFT) >= \
+	 ID_AA64ISAR1_GPA_ARCHITECTED)
+#define aa64isar1_has_gpi(val)	\
+	(cpuid_feature_extract_unsigned_field(val, ID_AA64ISAR1_GPI_SHIFT) >= \
+	 ID_AA64ISAR1_GPI_IMP_DEF)
+
 struct id_reg_info {
 	u32	sys_reg;	/* Register ID */
 
@@ -471,6 +489,36 @@ static int validate_id_aa64isar0_el1(struct kvm_vcpu *vcpu,
 	return 0;
 }
 
+static int validate_id_aa64isar1_el1(struct kvm_vcpu *vcpu,
+				     const struct id_reg_info *id_reg, u64 val)
+{
+	bool has_gpi, has_gpa, has_api, has_apa;
+	bool generic, address;
+
+	has_gpi = aa64isar1_has_gpi(val);
+	has_gpa = aa64isar1_has_gpa(val);
+	has_api = aa64isar1_has_api(val);
+	has_apa = aa64isar1_has_apa(val);
+	if ((has_gpi && has_gpa) || (has_api && has_apa))
+		return -EINVAL;
+
+	generic = has_gpi || has_gpa;
+	address = has_api || has_apa;
+	/*
+	 * Since the current KVM guest implementation works by enabling
+	 * both address/generic pointer authentication features,
+	 * return an error if they conflict.
+	 */
+	if (generic ^ address)
+		return -EPERM;
+
+	/* Check if there is a conflict with a request via KVM_ARM_VCPU_INIT */
+	if (vcpu_has_ptrauth(vcpu) ^ (generic && address))
+		return -EPERM;
+
+	return 0;
+}
+
 static void init_id_aa64pfr0_el1_info(struct id_reg_info *id_reg)
 {
 	u64 limit = id_reg->vcpu_limit_val;
@@ -508,6 +556,12 @@ static void init_id_aa64pfr1_el1_info(struct id_reg_info *id_reg)
 		id_reg->vcpu_limit_val &= ~ARM64_FEATURE_MASK(ID_AA64PFR1_MTE);
 }
 
+static void init_id_aa64isar1_el1_info(struct id_reg_info *id_reg)
+{
+	if (!system_has_full_ptr_auth())
+		id_reg->vcpu_limit_val &= ~PTRAUTH_MASK;
+}
+
 static u64 get_reset_id_aa64pfr0_el1(struct kvm_vcpu *vcpu,
 				     const struct id_reg_info *idr)
 {
@@ -530,6 +584,13 @@ static u64 get_reset_id_aa64pfr1_el1(struct kvm_vcpu *vcpu,
 	       (idr->vcpu_limit_val & ~(ARM64_FEATURE_MASK(ID_AA64PFR1_MTE)));
 }
 
+static u64 get_reset_id_aa64isar1_el1(struct kvm_vcpu *vcpu,
+				      const struct id_reg_info *idr)
+{
+	return vcpu_has_ptrauth(vcpu) ?
+	       idr->vcpu_limit_val : (idr->vcpu_limit_val & ~PTRAUTH_MASK);
+}
+
 static struct id_reg_info id_aa64pfr0_el1_info = {
 	.sys_reg = SYS_ID_AA64PFR0_EL1,
 	.ftr_check_types = S_FCT(ID_AA64PFR0_ASIMD_SHIFT, FCT_LOWER_SAFE) |
@@ -554,6 +615,16 @@ static struct id_reg_info id_aa64isar0_el1_info = {
 	.validate = validate_id_aa64isar0_el1,
 };
 
+static struct id_reg_info id_aa64isar1_el1_info = {
+	.sys_reg = SYS_ID_AA64ISAR1_EL1,
+	.ftr_check_types =
+		U_FCT(ID_AA64ISAR1_API_SHIFT, FCT_EXACT_OR_ZERO_SAFE) |
+		U_FCT(ID_AA64ISAR1_APA_SHIFT, FCT_EXACT_OR_ZERO_SAFE),
+	.init = init_id_aa64isar1_el1_info,
+	.validate = validate_id_aa64isar1_el1,
+	.get_reset_val = get_reset_id_aa64isar1_el1,
+};
+
 /*
  * An ID register that needs special handling to control the value for the
  * guest must have its own id_reg_info in id_reg_info_table.
@@ -566,6 +637,7 @@ static struct id_reg_info *id_reg_info_table[KVM_ARM_ID_REG_MAX_NUM] = {
 	[IDREG_IDX(SYS_ID_AA64PFR0_EL1)] = &id_aa64pfr0_el1_info,
 	[IDREG_IDX(SYS_ID_AA64PFR1_EL1)] = &id_aa64pfr1_el1_info,
 	[IDREG_IDX(SYS_ID_AA64ISAR0_EL1)] = &id_aa64isar0_el1_info,
+	[IDREG_IDX(SYS_ID_AA64ISAR1_EL1)] = &id_aa64isar1_el1_info,
 };
 
 static int validate_id_reg(struct kvm_vcpu *vcpu,
@@ -1414,13 +1486,6 @@ static u64 __read_id_reg(const struct kvm_vcpu *vcpu, u32 id)
 			val |= FIELD_PREP(ARM64_FEATURE_MASK(ID_AA64PFR0_GIC), gic_lim);
 		}
 		break;
-	case SYS_ID_AA64ISAR1_EL1:
-		if (!vcpu_has_ptrauth(vcpu))
-			val &= ~(ARM64_FEATURE_MASK(ID_AA64ISAR1_APA) |
-				 ARM64_FEATURE_MASK(ID_AA64ISAR1_API) |
-				 ARM64_FEATURE_MASK(ID_AA64ISAR1_GPA) |
-				 ARM64_FEATURE_MASK(ID_AA64ISAR1_GPI));
-		break;
 	case SYS_ID_AA64DFR0_EL1:
 		/* Limit debug to ARMv8.0 */
 		val &= ~ARM64_FEATURE_MASK(ID_AA64DFR0_DEBUGVER);
-- 
2.34.0.rc1.387.gb447b232ab-goog

[RFC PATCH v3 08/29] KVM: arm64: Make ID_AA64MMFR0_EL1 writable

[Reiji Watanabe]

This patch adds id_reg_info for ID_AA64MMFR0_EL1 to make it
writable by userspace.

Since ID_AA64MMFR0_EL1 stage 2 granule size fields don't follow the
standard ID scheme, we need a special handling to validate those fields.

Signed-off-by: Reiji Watanabe <reijiw@google.com>
---
 arch/arm64/kvm/sys_regs.c | 118 ++++++++++++++++++++++++++++++++++++++
 1 file changed, 118 insertions(+)

diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index 5812e39602fe..772e3d3067b2 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -519,6 +519,113 @@ static int validate_id_aa64isar1_el1(struct kvm_vcpu *vcpu,
 	return 0;
 }
 
+/*
+ * Check if the requested stage2 translation granule size indicated in
+ * @mmfr0 is also indicated in @mmfr0_lim.  This function assumes that
+ * the stage1 granule size indicated in @mmfr0 has been validated already.
+ */
+static int aa64mmfr0_tgran2_check(int field, u64 mmfr0, u64 mmfr0_lim)
+{
+	s64 tgran2, lim_tgran2, rtgran1;
+	int f1;
+	bool is_signed = true;
+
+	tgran2 = cpuid_feature_extract_unsigned_field(mmfr0, field);
+	lim_tgran2 = cpuid_feature_extract_unsigned_field(mmfr0_lim, field);
+	if (tgran2 == lim_tgran2)
+		return 0;
+
+	if (tgran2 && lim_tgran2)
+		return (tgran2 > lim_tgran2) ? -E2BIG : 0;
+
+	/*
+	 * Either tgran2 or lim_tgran2 is zero.
+	 * Need stage1 granule size to validate tgran2.
+	 */
+	switch (field) {
+	case ID_AA64MMFR0_TGRAN4_2_SHIFT:
+		f1 = ID_AA64MMFR0_TGRAN4_SHIFT;
+		break;
+	case ID_AA64MMFR0_TGRAN64_2_SHIFT:
+		f1 = ID_AA64MMFR0_TGRAN64_SHIFT;
+		break;
+	case ID_AA64MMFR0_TGRAN16_2_SHIFT:
+		f1 = ID_AA64MMFR0_TGRAN16_SHIFT;
+		is_signed = false;
+		break;
+	default:
+		/* Should never happen */
+		WARN_ONCE(1, "Unexpected stage2 granule field (%d)\n", field);
+		return 0;
+	}
+
+	/*
+	 * If tgran2 == 0 (&& lim_tgran2 != 0), the requested stage2 granule
+	 * size is indicated in the stage1 granule size field of @mmfr0.
+	 * So, validate the stage1 granule size against the stage2 limit
+	 * granule size.
+	 * If lim_tgran2 == 0 (&& tgran2 != 0), the stage2 limit granule size
+	 * is indicated in the stage1 granule size field of @mmfr0_lim.
+	 * So, validate the requested stage2 granule size against the stage1
+	 * limit granule size.
+	 */
+
+	 /* Get the relevant stage1 granule size to validate tgran2 */
+	if (tgran2 == 0)
+		/* The requested stage1 granule size */
+		rtgran1 = cpuid_feature_extract_field(mmfr0, f1, is_signed);
+	else /* lim_tgran2 == 0 */
+		/* The stage1 limit granule size */
+		rtgran1 = cpuid_feature_extract_field(mmfr0_lim, f1, is_signed);
+
+	/*
+	 * Adjust the value of rtgran1 to compare with stage2 granule size,
+	 * which indicates: 1: Not supported, 2: Supported, etc.
+	 */
+	if (is_signed)
+		/* For signed, -1: Not supported, 0: Supported, etc. */
+		rtgran1 += 0x2;
+	else
+		/* For unsigned, 0: Not supported, 1: Supported, etc. */
+		rtgran1 += 0x1;
+
+	if ((tgran2 == 0) && (rtgran1 > lim_tgran2))
+		/*
+		 * The requested stage1 granule size (== the requested stage2
+		 * granule size) is larger than the stage2 limit granule size.
+		 */
+		return -E2BIG;
+	else if ((lim_tgran2 == 0) && (tgran2 > rtgran1))
+		/*
+		 * The requested stage2 granule size is larger than the stage1
+		 * limit granulze size (== the stage2 limit granule size).
+		 */
+		return -E2BIG;
+
+	return 0;
+}
+
+static int validate_id_aa64mmfr0_el1(struct kvm_vcpu *vcpu,
+				     const struct id_reg_info *id_reg, u64 val)
+{
+	u64 limit = id_reg->vcpu_limit_val;
+	int ret;
+
+	ret = aa64mmfr0_tgran2_check(ID_AA64MMFR0_TGRAN4_2_SHIFT, val, limit);
+	if (ret)
+		return ret;
+
+	ret = aa64mmfr0_tgran2_check(ID_AA64MMFR0_TGRAN64_2_SHIFT, val, limit);
+	if (ret)
+		return ret;
+
+	ret = aa64mmfr0_tgran2_check(ID_AA64MMFR0_TGRAN16_2_SHIFT, val, limit);
+	if (ret)
+		return ret;
+
+	return 0;
+}
+
 static void init_id_aa64pfr0_el1_info(struct id_reg_info *id_reg)
 {
 	u64 limit = id_reg->vcpu_limit_val;
@@ -625,6 +732,16 @@ static struct id_reg_info id_aa64isar1_el1_info = {
 	.get_reset_val = get_reset_id_aa64isar1_el1,
 };
 
+static struct id_reg_info id_aa64mmfr0_el1_info = {
+	.sys_reg = SYS_ID_AA64MMFR0_EL1,
+	.ftr_check_types = S_FCT(ID_AA64MMFR0_TGRAN4_SHIFT, FCT_LOWER_SAFE) |
+			   S_FCT(ID_AA64MMFR0_TGRAN64_SHIFT, FCT_LOWER_SAFE) |
+			   U_FCT(ID_AA64MMFR0_TGRAN4_2_SHIFT, FCT_IGNORE) |
+			   U_FCT(ID_AA64MMFR0_TGRAN64_2_SHIFT, FCT_IGNORE) |
+			   U_FCT(ID_AA64MMFR0_TGRAN16_2_SHIFT, FCT_IGNORE),
+	.validate = validate_id_aa64mmfr0_el1,
+};
+
 /*
  * An ID register that needs special handling to control the value for the
  * guest must have its own id_reg_info in id_reg_info_table.
@@ -638,6 +755,7 @@ static struct id_reg_info *id_reg_info_table[KVM_ARM_ID_REG_MAX_NUM] = {
 	[IDREG_IDX(SYS_ID_AA64PFR1_EL1)] = &id_aa64pfr1_el1_info,
 	[IDREG_IDX(SYS_ID_AA64ISAR0_EL1)] = &id_aa64isar0_el1_info,
 	[IDREG_IDX(SYS_ID_AA64ISAR1_EL1)] = &id_aa64isar1_el1_info,
+	[IDREG_IDX(SYS_ID_AA64MMFR0_EL1)] = &id_aa64mmfr0_el1_info,
 };
 
 static int validate_id_reg(struct kvm_vcpu *vcpu,
-- 
2.34.0.rc1.387.gb447b232ab-goog

[RFC PATCH v3 09/29] KVM: arm64: Hide IMPLEMENTATION DEFINED PMU support for the guest

[Reiji Watanabe]

When ID_AA64DFR0_EL1.PMUVER or ID_DFR0_EL1.PERFMON is 0xf, which
means IMPLEMENTATION DEFINED PMU supported, KVM unconditionally
expose the value for the guest as it is.  Since KVM doesn't support
IMPLEMENTATION DEFINED PMU for the guest, in that case KVM should
exopse 0x0 (PMU is not implemented) instead.

Change cpuid_feature_cap_perfmon_field() to update the field value
to 0x0 when it is 0xf.

Fixes: 8e35aa642ee4 ("arm64: cpufeature: Extract capped perfmon fields")
Signed-off-by: Reiji Watanabe <reijiw@google.com>
---
 arch/arm64/include/asm/cpufeature.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
index ef6be92b1921..fd7ad8193827 100644
--- a/arch/arm64/include/asm/cpufeature.h
+++ b/arch/arm64/include/asm/cpufeature.h
@@ -553,7 +553,7 @@ cpuid_feature_cap_perfmon_field(u64 features, int field, u64 cap)
 
 	/* Treat IMPLEMENTATION DEFINED functionality as unimplemented */
 	if (val == ID_AA64DFR0_PMUVER_IMP_DEF)
-		val = 0;
+		return (features & ~mask);
 
 	if (val > cap) {
 		features &= ~mask;
-- 
2.34.0.rc1.387.gb447b232ab-goog

[RFC PATCH v3 10/29] KVM: arm64: Make ID_AA64DFR0_EL1 writable

[Reiji Watanabe]

This patch adds id_reg_info for ID_AA64DFR0_EL1 to make it writable
by userspace.

Return an error if userspace tries to set PMUVER field of the
register to a value that conflicts with the PMU configuration.

Since number of context-aware breakpoints must be no more than number
of supported breakpoints according to Arm ARM, return an error
if userspace tries to set CTX_CMPS field to such value.

Signed-off-by: Reiji Watanabe <reijiw@google.com>
---
 arch/arm64/kvm/sys_regs.c | 84 ++++++++++++++++++++++++++++++++++-----
 1 file changed, 73 insertions(+), 11 deletions(-)

diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index 772e3d3067b2..0faf458b0efb 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -626,6 +626,45 @@ static int validate_id_aa64mmfr0_el1(struct kvm_vcpu *vcpu,
 	return 0;
 }
 
+static bool id_reg_has_pmu(u64 val, u64 shift, unsigned int min)
+{
+	unsigned int pmu = cpuid_feature_extract_unsigned_field(val, shift);
+
+	/*
+	 * Treat IMPLEMENTATION DEFINED functionality as unimplemented for
+	 * ID_AA64DFR0_EL1.PMUVer/ID_DFR0_EL1.PerfMon.
+	 */
+	if (pmu == 0xf)
+		pmu = 0;
+
+	return (pmu >= min);
+}
+
+static int validate_id_aa64dfr0_el1(struct kvm_vcpu *vcpu,
+				    const struct id_reg_info *id_reg, u64 val)
+{
+	unsigned int brps, ctx_cmps;
+	bool vcpu_pmu, dfr0_pmu;
+
+	brps = cpuid_feature_extract_unsigned_field(val, ID_AA64DFR0_BRPS_SHIFT);
+	ctx_cmps = cpuid_feature_extract_unsigned_field(val, ID_AA64DFR0_CTX_CMPS_SHIFT);
+
+	/*
+	 * Number of context-aware breakpoints can be no more than number of
+	 * supported breakpoints.
+	 */
+	if (ctx_cmps > brps)
+		return -EINVAL;
+
+	vcpu_pmu = kvm_vcpu_has_pmu(vcpu);
+	dfr0_pmu = id_reg_has_pmu(val, ID_AA64DFR0_PMUVER_SHIFT, ID_AA64DFR0_PMUVER_8_0);
+	/* Check if there is a conflict with a request via KVM_ARM_VCPU_INIT */
+	if (vcpu_pmu ^ dfr0_pmu)
+		return -EPERM;
+
+	return 0;
+}
+
 static void init_id_aa64pfr0_el1_info(struct id_reg_info *id_reg)
 {
 	u64 limit = id_reg->vcpu_limit_val;
@@ -669,6 +708,23 @@ static void init_id_aa64isar1_el1_info(struct id_reg_info *id_reg)
 		id_reg->vcpu_limit_val &= ~PTRAUTH_MASK;
 }
 
+static void init_id_aa64dfr0_el1_info(struct id_reg_info *id_reg)
+{
+	u64 limit = id_reg->vcpu_limit_val;
+
+	/* Limit guests to PMUv3 for ARMv8.4 */
+	limit = cpuid_feature_cap_perfmon_field(limit, ID_AA64DFR0_PMUVER_SHIFT,
+						ID_AA64DFR0_PMUVER_8_4);
+	/* Limit debug to ARMv8.0 */
+	limit &= ~ARM64_FEATURE_MASK(ID_AA64DFR0_DEBUGVER);
+	limit |= (FIELD_PREP(ARM64_FEATURE_MASK(ID_AA64DFR0_DEBUGVER), 6));
+
+	/* Hide SPE from guests */
+	limit &= ~ARM64_FEATURE_MASK(ID_AA64DFR0_PMSVER);
+
+	id_reg->vcpu_limit_val = limit;
+}
+
 static u64 get_reset_id_aa64pfr0_el1(struct kvm_vcpu *vcpu,
 				     const struct id_reg_info *idr)
 {
@@ -698,6 +754,14 @@ static u64 get_reset_id_aa64isar1_el1(struct kvm_vcpu *vcpu,
 	       idr->vcpu_limit_val : (idr->vcpu_limit_val & ~PTRAUTH_MASK);
 }
 
+static u64 get_reset_id_aa64dfr0_el1(struct kvm_vcpu *vcpu,
+				     const struct id_reg_info *idr)
+{
+	return kvm_vcpu_has_pmu(vcpu) ?
+	       idr->vcpu_limit_val :
+	       (idr->vcpu_limit_val & ~(ARM64_FEATURE_MASK(ID_AA64DFR0_PMUVER)));
+}
+
 static struct id_reg_info id_aa64pfr0_el1_info = {
 	.sys_reg = SYS_ID_AA64PFR0_EL1,
 	.ftr_check_types = S_FCT(ID_AA64PFR0_ASIMD_SHIFT, FCT_LOWER_SAFE) |
@@ -742,6 +806,14 @@ static struct id_reg_info id_aa64mmfr0_el1_info = {
 	.validate = validate_id_aa64mmfr0_el1,
 };
 
+static struct id_reg_info id_aa64dfr0_el1_info = {
+	.sys_reg = SYS_ID_AA64DFR0_EL1,
+	.ftr_check_types = S_FCT(ID_AA64DFR0_DOUBLELOCK_SHIFT, FCT_LOWER_SAFE),
+	.init = init_id_aa64dfr0_el1_info,
+	.validate = validate_id_aa64dfr0_el1,
+	.get_reset_val = get_reset_id_aa64dfr0_el1,
+};
+
 /*
  * An ID register that needs special handling to control the value for the
  * guest must have its own id_reg_info in id_reg_info_table.
@@ -753,6 +825,7 @@ static struct id_reg_info id_aa64mmfr0_el1_info = {
 static struct id_reg_info *id_reg_info_table[KVM_ARM_ID_REG_MAX_NUM] = {
 	[IDREG_IDX(SYS_ID_AA64PFR0_EL1)] = &id_aa64pfr0_el1_info,
 	[IDREG_IDX(SYS_ID_AA64PFR1_EL1)] = &id_aa64pfr1_el1_info,
+	[IDREG_IDX(SYS_ID_AA64DFR0_EL1)] = &id_aa64dfr0_el1_info,
 	[IDREG_IDX(SYS_ID_AA64ISAR0_EL1)] = &id_aa64isar0_el1_info,
 	[IDREG_IDX(SYS_ID_AA64ISAR1_EL1)] = &id_aa64isar1_el1_info,
 	[IDREG_IDX(SYS_ID_AA64MMFR0_EL1)] = &id_aa64mmfr0_el1_info,
@@ -1604,17 +1677,6 @@ static u64 __read_id_reg(const struct kvm_vcpu *vcpu, u32 id)
 			val |= FIELD_PREP(ARM64_FEATURE_MASK(ID_AA64PFR0_GIC), gic_lim);
 		}
 		break;
-	case SYS_ID_AA64DFR0_EL1:
-		/* Limit debug to ARMv8.0 */
-		val &= ~ARM64_FEATURE_MASK(ID_AA64DFR0_DEBUGVER);
-		val |= FIELD_PREP(ARM64_FEATURE_MASK(ID_AA64DFR0_DEBUGVER), 6);
-		/* Limit guests to PMUv3 for ARMv8.4 */
-		val = cpuid_feature_cap_perfmon_field(val,
-						      ID_AA64DFR0_PMUVER_SHIFT,
-						      kvm_vcpu_has_pmu(vcpu) ? ID_AA64DFR0_PMUVER_8_4 : 0);
-		/* Hide SPE from guests */
-		val &= ~ARM64_FEATURE_MASK(ID_AA64DFR0_PMSVER);
-		break;
 	case SYS_ID_DFR0_EL1:
 		/* Limit guests to PMUv3 for ARMv8.4 */
 		val = cpuid_feature_cap_perfmon_field(val,
-- 
2.34.0.rc1.387.gb447b232ab-goog

[RFC PATCH v3 11/29] KVM: arm64: Make ID_DFR0_EL1 writable

[Reiji Watanabe]

This patch adds id_reg_info for ID_DFR0_EL1 to make it writable
by userspace.

Return an error if userspace tries to set PerfMon field of the
register to a value that conflicts with the PMU configuration.

Signed-off-by: Reiji Watanabe <reijiw@google.com>
---
 arch/arm64/kvm/sys_regs.c | 52 ++++++++++++++++++++++++++++++++++-----
 1 file changed, 46 insertions(+), 6 deletions(-)

diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index 0faf458b0efb..fbd335ac5e6b 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -665,6 +665,27 @@ static int validate_id_aa64dfr0_el1(struct kvm_vcpu *vcpu,
 	return 0;
 }
 
+static int validate_id_dfr0_el1(struct kvm_vcpu *vcpu,
+				const struct id_reg_info *id_reg, u64 val)
+{
+	bool vcpu_pmu, dfr0_pmu;
+	unsigned int perfmon;
+
+	perfmon = cpuid_feature_extract_unsigned_field(val, ID_DFR0_PERFMON_SHIFT);
+	if (perfmon == 1 || perfmon == 2)
+		/* PMUv1 or PMUv2 is not allowed on ARMv8. */
+		return -EINVAL;
+
+	vcpu_pmu = kvm_vcpu_has_pmu(vcpu);
+	dfr0_pmu = id_reg_has_pmu(val, ID_DFR0_PERFMON_SHIFT, ID_DFR0_PERFMON_8_0);
+
+	/* Check if there is a conflict with a request via KVM_ARM_VCPU_INIT */
+	if (vcpu_pmu ^ dfr0_pmu)
+		return -EPERM;
+
+	return 0;
+}
+
 static void init_id_aa64pfr0_el1_info(struct id_reg_info *id_reg)
 {
 	u64 limit = id_reg->vcpu_limit_val;
@@ -725,6 +746,15 @@ static void init_id_aa64dfr0_el1_info(struct id_reg_info *id_reg)
 	id_reg->vcpu_limit_val = limit;
 }
 
+static void init_id_dfr0_el1_info(struct id_reg_info *id_reg)
+{
+	/* Limit guests to PMUv3 for ARMv8.4 */
+	id_reg->vcpu_limit_val =
+		cpuid_feature_cap_perfmon_field(id_reg->vcpu_limit_val,
+						ID_DFR0_PERFMON_SHIFT,
+						ID_DFR0_PERFMON_8_4);
+}
+
 static u64 get_reset_id_aa64pfr0_el1(struct kvm_vcpu *vcpu,
 				     const struct id_reg_info *idr)
 {
@@ -762,6 +792,14 @@ static u64 get_reset_id_aa64dfr0_el1(struct kvm_vcpu *vcpu,
 	       (idr->vcpu_limit_val & ~(ARM64_FEATURE_MASK(ID_AA64DFR0_PMUVER)));
 }
 
+static u64 get_reset_id_dfr0_el1(struct kvm_vcpu *vcpu,
+				 const struct id_reg_info *idr)
+{
+	return kvm_vcpu_has_pmu(vcpu) ?
+	       idr->vcpu_limit_val :
+	       (idr->vcpu_limit_val & ~(ARM64_FEATURE_MASK(ID_DFR0_PERFMON)));
+}
+
 static struct id_reg_info id_aa64pfr0_el1_info = {
 	.sys_reg = SYS_ID_AA64PFR0_EL1,
 	.ftr_check_types = S_FCT(ID_AA64PFR0_ASIMD_SHIFT, FCT_LOWER_SAFE) |
@@ -814,6 +852,13 @@ static struct id_reg_info id_aa64dfr0_el1_info = {
 	.get_reset_val = get_reset_id_aa64dfr0_el1,
 };
 
+static struct id_reg_info id_dfr0_el1_info = {
+	.sys_reg = SYS_ID_DFR0_EL1,
+	.init = init_id_dfr0_el1_info,
+	.validate = validate_id_dfr0_el1,
+	.get_reset_val = get_reset_id_dfr0_el1,
+};
+
 /*
  * An ID register that needs special handling to control the value for the
  * guest must have its own id_reg_info in id_reg_info_table.
@@ -823,6 +868,7 @@ static struct id_reg_info id_aa64dfr0_el1_info = {
  */
 #define	GET_ID_REG_INFO(id)	(id_reg_info_table[IDREG_IDX(id)])
 static struct id_reg_info *id_reg_info_table[KVM_ARM_ID_REG_MAX_NUM] = {
+	[IDREG_IDX(SYS_ID_DFR0_EL1)] = &id_dfr0_el1_info,
 	[IDREG_IDX(SYS_ID_AA64PFR0_EL1)] = &id_aa64pfr0_el1_info,
 	[IDREG_IDX(SYS_ID_AA64PFR1_EL1)] = &id_aa64pfr1_el1_info,
 	[IDREG_IDX(SYS_ID_AA64DFR0_EL1)] = &id_aa64dfr0_el1_info,
@@ -1677,12 +1723,6 @@ static u64 __read_id_reg(const struct kvm_vcpu *vcpu, u32 id)
 			val |= FIELD_PREP(ARM64_FEATURE_MASK(ID_AA64PFR0_GIC), gic_lim);
 		}
 		break;
-	case SYS_ID_DFR0_EL1:
-		/* Limit guests to PMUv3 for ARMv8.4 */
-		val = cpuid_feature_cap_perfmon_field(val,
-						      ID_DFR0_PERFMON_SHIFT,
-						      kvm_vcpu_has_pmu(vcpu) ? ID_DFR0_PERFMON_8_4 : 0);
-		break;
 	}
 
 	return val;
-- 
2.34.0.rc1.387.gb447b232ab-goog

[RFC PATCH v3 12/29] KVM: arm64: Make ID_DFR1_EL1 writable

[Reiji Watanabe]

This patch adds id_reg_info for ID_DFR1_EL1 to make it writable
by userspace.

Signed-off-by: Reiji Watanabe <reijiw@google.com>
---
 arch/arm64/kvm/sys_regs.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index fbd335ac5e6b..dda7001959f6 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -859,6 +859,11 @@ static struct id_reg_info id_dfr0_el1_info = {
 	.get_reset_val = get_reset_id_dfr0_el1,
 };
 
+static struct id_reg_info id_dfr1_el1_info = {
+	.sys_reg = SYS_ID_DFR1_EL1,
+	.ftr_check_types = S_FCT(ID_DFR1_MTPMU_SHIFT, FCT_LOWER_SAFE),
+};
+
 /*
  * An ID register that needs special handling to control the value for the
  * guest must have its own id_reg_info in id_reg_info_table.
@@ -869,6 +874,7 @@ static struct id_reg_info id_dfr0_el1_info = {
 #define	GET_ID_REG_INFO(id)	(id_reg_info_table[IDREG_IDX(id)])
 static struct id_reg_info *id_reg_info_table[KVM_ARM_ID_REG_MAX_NUM] = {
 	[IDREG_IDX(SYS_ID_DFR0_EL1)] = &id_dfr0_el1_info,
+	[IDREG_IDX(SYS_ID_DFR1_EL1)] = &id_dfr1_el1_info,
 	[IDREG_IDX(SYS_ID_AA64PFR0_EL1)] = &id_aa64pfr0_el1_info,
 	[IDREG_IDX(SYS_ID_AA64PFR1_EL1)] = &id_aa64pfr1_el1_info,
 	[IDREG_IDX(SYS_ID_AA64DFR0_EL1)] = &id_aa64dfr0_el1_info,
-- 
2.34.0.rc1.387.gb447b232ab-goog

[RFC PATCH v3 13/29] KVM: arm64: Make ID_MMFR0_EL1 writable

[Reiji Watanabe]

This patch adds id_reg_info for ID_MMFR0_EL1 to make it writable
by userspace.

Signed-off-by: Reiji Watanabe <reijiw@google.com>
---
 arch/arm64/kvm/sys_regs.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index dda7001959f6..5b16d422b37d 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -864,6 +864,12 @@ static struct id_reg_info id_dfr1_el1_info = {
 	.ftr_check_types = S_FCT(ID_DFR1_MTPMU_SHIFT, FCT_LOWER_SAFE),
 };
 
+static struct id_reg_info id_mmfr0_el1_info = {
+	.sys_reg = SYS_ID_MMFR0_EL1,
+	.ftr_check_types = S_FCT(ID_MMFR0_INNERSHR_SHIFT, FCT_LOWER_SAFE) |
+			   S_FCT(ID_MMFR0_OUTERSHR_SHIFT, FCT_LOWER_SAFE),
+};
+
 /*
  * An ID register that needs special handling to control the value for the
  * guest must have its own id_reg_info in id_reg_info_table.
@@ -874,6 +880,7 @@ static struct id_reg_info id_dfr1_el1_info = {
 #define	GET_ID_REG_INFO(id)	(id_reg_info_table[IDREG_IDX(id)])
 static struct id_reg_info *id_reg_info_table[KVM_ARM_ID_REG_MAX_NUM] = {
 	[IDREG_IDX(SYS_ID_DFR0_EL1)] = &id_dfr0_el1_info,
+	[IDREG_IDX(SYS_ID_MMFR0_EL1)] = &id_mmfr0_el1_info,
 	[IDREG_IDX(SYS_ID_DFR1_EL1)] = &id_dfr1_el1_info,
 	[IDREG_IDX(SYS_ID_AA64PFR0_EL1)] = &id_aa64pfr0_el1_info,
 	[IDREG_IDX(SYS_ID_AA64PFR1_EL1)] = &id_aa64pfr1_el1_info,
-- 
2.34.0.rc1.387.gb447b232ab-goog

[RFC PATCH v3 14/29] KVM: arm64: Make MVFR1_EL1 writable

[Reiji Watanabe]

This patch adds id_reg_info for MVFR1_EL1 to make it writable
by userspace.

There are only a few valid combinations of values that can be set
for FPHP and SIMDHP fields according to Arm ARM.  Return an error
when userspace tries to set those fields to values that don't match
any of the valid combinations.

Signed-off-by: Reiji Watanabe <reijiw@google.com>
---
 arch/arm64/kvm/sys_regs.c | 36 ++++++++++++++++++++++++++++++++++++
 1 file changed, 36 insertions(+)

diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index 5b16d422b37d..659ec880d527 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -686,6 +686,36 @@ static int validate_id_dfr0_el1(struct kvm_vcpu *vcpu,
 	return 0;
 }
 
+static int validate_mvfr1_el1(struct kvm_vcpu *vcpu,
+			      const struct id_reg_info *id_reg, u64 val)
+{
+	unsigned int fphp, simdhp;
+	struct fphp_simdhp {
+		unsigned int fphp;
+		unsigned int simdhp;
+	};
+	/* Permitted fphp/simdhp value combinations according to Arm ARM */
+	struct fphp_simdhp valid_fphp_simdhp[3] = {{0, 0}, {2, 1}, {3, 2}};
+	int i;
+	bool is_valid_fphp_simdhp = false;
+
+	fphp = cpuid_feature_extract_unsigned_field(val, MVFR1_FPHP_SHIFT);
+	simdhp = cpuid_feature_extract_unsigned_field(val, MVFR1_SIMDHP_SHIFT);
+
+	for (i = 0; i < ARRAY_SIZE(valid_fphp_simdhp); i++) {
+		if (valid_fphp_simdhp[i].fphp == fphp &&
+		    valid_fphp_simdhp[i].simdhp == simdhp) {
+			is_valid_fphp_simdhp = true;
+			break;
+		}
+	}
+
+	if (!is_valid_fphp_simdhp)
+		return -EINVAL;
+
+	return 0;
+}
+
 static void init_id_aa64pfr0_el1_info(struct id_reg_info *id_reg)
 {
 	u64 limit = id_reg->vcpu_limit_val;
@@ -870,6 +900,11 @@ static struct id_reg_info id_mmfr0_el1_info = {
 			   S_FCT(ID_MMFR0_OUTERSHR_SHIFT, FCT_LOWER_SAFE),
 };
 
+static struct id_reg_info mvfr1_el1_info = {
+	.sys_reg = SYS_MVFR1_EL1,
+	.validate = validate_mvfr1_el1,
+};
+
 /*
  * An ID register that needs special handling to control the value for the
  * guest must have its own id_reg_info in id_reg_info_table.
@@ -881,6 +916,7 @@ static struct id_reg_info id_mmfr0_el1_info = {
 static struct id_reg_info *id_reg_info_table[KVM_ARM_ID_REG_MAX_NUM] = {
 	[IDREG_IDX(SYS_ID_DFR0_EL1)] = &id_dfr0_el1_info,
 	[IDREG_IDX(SYS_ID_MMFR0_EL1)] = &id_mmfr0_el1_info,
+	[IDREG_IDX(SYS_MVFR1_EL1)] = &mvfr1_el1_info,
 	[IDREG_IDX(SYS_ID_DFR1_EL1)] = &id_dfr1_el1_info,
 	[IDREG_IDX(SYS_ID_AA64PFR0_EL1)] = &id_aa64pfr0_el1_info,
 	[IDREG_IDX(SYS_ID_AA64PFR1_EL1)] = &id_aa64pfr1_el1_info,
-- 
2.34.0.rc1.387.gb447b232ab-goog

[RFC PATCH v3 15/29] KVM: arm64: Make ID registers without id_reg_info writable

[Reiji Watanabe]

Make ID registers that don't have id_reg_info writable.

Signed-off-by: Reiji Watanabe <reijiw@google.com>
---
 arch/arm64/kvm/sys_regs.c | 10 +++-------
 1 file changed, 3 insertions(+), 7 deletions(-)

diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index 659ec880d527..35e458cc1e1d 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -1886,16 +1886,12 @@ static int __set_id_reg(struct kvm_vcpu *vcpu,
 	if (err)
 		return err;
 
-	/* Don't allow to change the reg unless the reg has id_reg_info */
-	if (val != read_id_reg(vcpu, rd, raz) && !GET_ID_REG_INFO(encoding))
+	/* Don't allow to change the reg after the first KVM_RUN. */
+	if ((val != read_id_reg(vcpu, rd, raz)) && vcpu->arch.has_run_once)
 		return -EINVAL;
 
 	if (raz)
-		return 0;
-
-	/* Don't allow to change the reg after the first KVM_RUN. */
-	if (vcpu->arch.has_run_once)
-		return -EINVAL;
+		return (val == 0) ? 0 : -EINVAL;
 
 	err = validate_id_reg(vcpu, rd, val);
 	if (err)
-- 
2.34.0.rc1.387.gb447b232ab-goog

[RFC PATCH v3 16/29] KVM: arm64: Add consistency checking for frac fields of ID registers

[Reiji Watanabe]

Feature fractional field of an ID register cannot be simply validated
at KVM_SET_ONE_REG because its validity depends on its (main) feature
field value, which could be in a different ID register (and might be
set later).
Validate fractional fields at the first KVM_RUN instead.

Signed-off-by: Reiji Watanabe <reijiw@google.com>
---
 arch/arm64/include/asm/kvm_host.h |   2 +
 arch/arm64/kvm/arm.c              |   3 +
 arch/arm64/kvm/sys_regs.c         | 125 +++++++++++++++++++++++++++++-
 3 files changed, 127 insertions(+), 3 deletions(-)

diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index 72db73c79403..9dc9970a2d46 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -746,6 +746,8 @@ int kvm_arm_vcpu_arch_has_attr(struct kvm_vcpu *vcpu,
 long kvm_vm_ioctl_mte_copy_tags(struct kvm *kvm,
 				struct kvm_arm_copy_mte_tags *copy_tags);
 
+int kvm_id_regs_consistency_check(const struct kvm_vcpu *vcpu);
+
 /* Guest/host FPSIMD coordination helpers */
 int kvm_arch_vcpu_run_map_fp(struct kvm_vcpu *vcpu);
 void kvm_arch_vcpu_load_fp(struct kvm_vcpu *vcpu);
diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
index 2f03cbfefe67..19c4a78f931d 100644
--- a/arch/arm64/kvm/arm.c
+++ b/arch/arm64/kvm/arm.c
@@ -588,6 +588,9 @@ static int kvm_vcpu_first_run_init(struct kvm_vcpu *vcpu)
 	if (!kvm_arm_vcpu_is_finalized(vcpu))
 		return -EPERM;
 
+	if (!kvm_vm_is_protected(kvm) && kvm_id_regs_consistency_check(vcpu))
+		return -EPERM;
+
 	vcpu->arch.has_run_once = true;
 
 	kvm_arm_vcpu_init_debug(vcpu);
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index 35e458cc1e1d..b848ecea0c59 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -841,9 +841,6 @@ static struct id_reg_info id_aa64pfr0_el1_info = {
 
 static struct id_reg_info id_aa64pfr1_el1_info = {
 	.sys_reg = SYS_ID_AA64PFR1_EL1,
-	.ftr_check_types = U_FCT(ID_AA64PFR1_RASFRAC_SHIFT, FCT_IGNORE) |
-			   U_FCT(ID_AA64PFR1_MPAMFRAC_SHIFT, FCT_IGNORE) |
-			   U_FCT(ID_AA64PFR1_CSV2FRAC_SHIFT, FCT_IGNORE),
 	.init = init_id_aa64pfr1_el1_info,
 	.validate = validate_id_aa64pfr1_el1,
 	.get_reset_val = get_reset_id_aa64pfr1_el1,
@@ -3460,10 +3457,106 @@ int kvm_arm_copy_sys_reg_indices(struct kvm_vcpu *vcpu, u64 __user *uindices)
 	return write_demux_regids(uindices);
 }
 
+/* ID register's fractional field information with its feature field. */
+struct feature_frac {
+	u32	id;
+	u32	shift;
+	u32	frac_id;
+	u32	frac_shift;
+	u8	frac_ftr_check;
+};
+
+static struct feature_frac feature_frac_table[] = {
+	{
+		.frac_id = SYS_ID_AA64PFR1_EL1,
+		.frac_shift = ID_AA64PFR1_RASFRAC_SHIFT,
+		.id = SYS_ID_AA64PFR0_EL1,
+		.shift = ID_AA64PFR0_RAS_SHIFT,
+	},
+	{
+		.frac_id = SYS_ID_AA64PFR1_EL1,
+		.frac_shift = ID_AA64PFR1_MPAMFRAC_SHIFT,
+		.id = SYS_ID_AA64PFR0_EL1,
+		.shift = ID_AA64PFR0_MPAM_SHIFT,
+	},
+	{
+		.frac_id = SYS_ID_AA64PFR1_EL1,
+		.frac_shift = ID_AA64PFR1_CSV2FRAC_SHIFT,
+		.id = SYS_ID_AA64PFR0_EL1,
+		.shift = ID_AA64PFR0_CSV2_SHIFT,
+	},
+};
+
+/*
+ * Return non-zero if the feature/fractional fields pair are not
+ * supported. Return zero otherwise.
+ * This function only checks fractional feature field and assumes
+ * the feature field is valid.
+ */
+static int vcpu_id_reg_feature_frac_check(const struct kvm_vcpu *vcpu,
+					  const struct feature_frac *ftr_frac)
+{
+	u32 id;
+	int fval, flim, ret;
+	u64 val, lim, mask;
+	const struct id_reg_info *id_reg;
+	bool sign = FCT_SIGN(ftr_frac->frac_ftr_check);
+	enum feature_check_type type = FCT_TYPE(ftr_frac->frac_ftr_check);
+
+	/* Check if the feature field value is same as the limit */
+	id = ftr_frac->id;
+	id_reg = GET_ID_REG_INFO(id);
+
+	val = __read_id_reg(vcpu, id);
+	lim = id_reg ? id_reg->vcpu_limit_val : read_sanitised_ftr_reg(id);
+
+	mask = (u64)ARM64_FEATURE_FIELD_MASK << ftr_frac->shift;
+	if ((val & mask) != (lim & mask))
+		/*
+		 * The feature level is smaller than the limit.
+		 * Any fractional version should be fine.
+		 */
+		return 0;
+
+	/* Check the fractional feature field */
+	id = ftr_frac->frac_id;
+	id_reg = GET_ID_REG_INFO(id);
+
+	val = __read_id_reg(vcpu, id);
+	fval = cpuid_feature_extract_field(val, ftr_frac->frac_shift, sign);
+
+	lim = id_reg ? id_reg->vcpu_limit_val : read_sanitised_ftr_reg(id);
+	flim = cpuid_feature_extract_field(lim, ftr_frac->frac_shift, sign);
+
+	ret = arm64_check_feature_one(type, fval, flim);
+	return ret ? -E2BIG : 0;
+}
+
+int kvm_id_regs_consistency_check(const struct kvm_vcpu *vcpu)
+{
+	int i, err;
+	const struct feature_frac *frac;
+
+	/*
+	 * Check ID registers' fractional fields, which aren't checked
+	 * at KVM_SET_ONE_REG.
+	 */
+	for (i = 0; i < ARRAY_SIZE(feature_frac_table); i++) {
+		frac = &feature_frac_table[i];
+		err = vcpu_id_reg_feature_frac_check(vcpu, frac);
+		if (err)
+			return err;
+	}
+	return 0;
+}
+
 static void id_reg_info_init_all(void)
 {
 	int i;
 	struct id_reg_info *id_reg;
+	struct feature_frac *frac;
+	u64 mask = ARM64_FEATURE_FIELD_MASK;
+	u64 org;
 
 	for (i = 0; i < ARRAY_SIZE(id_reg_info_table); i++) {
 		id_reg = (struct id_reg_info *)id_reg_info_table[i];
@@ -3472,6 +3565,32 @@ static void id_reg_info_init_all(void)
 
 		id_reg_info_init(id_reg);
 	}
+
+	for (i = 0; i < ARRAY_SIZE(feature_frac_table); i++) {
+		frac = &feature_frac_table[i];
+		id_reg = GET_ID_REG_INFO(frac->frac_id);
+
+		/*
+		 * An ID register that has fractional fields is expected
+		 * to have its own id_reg_info.
+		 */
+		if (WARN_ON_ONCE(!id_reg))
+			continue;
+
+		/*
+		 * Update the id_reg's ftr_check_types for the fractional
+		 * field with FCT_IGNORE so that the field won't be validated
+		 * when the ID register is set by userspace, which could
+		 * temporarily cause an inconsistency if its (main) feature
+		 * field is not set yet.  Save the original ftr_check_types
+		 * for the fractional field to validate the field later.
+		 */
+		org = (id_reg->ftr_check_types >> frac->frac_shift) & mask;
+		id_reg->ftr_check_types &= ~(mask << frac->frac_shift);
+		id_reg->ftr_check_types |=
+			MAKE_FCT(frac->frac_shift, FCT_IGNORE, FCT_SIGN(org));
+		frac->frac_ftr_check = org;
+	}
 }
 
 void kvm_sys_reg_table_init(void)
-- 
2.34.0.rc1.387.gb447b232ab-goog

[RFC PATCH v3 17/29] KVM: arm64: Introduce KVM_CAP_ARM_ID_REG_CONFIGURABLE capability

[Reiji Watanabe]

Introduce a new capability KVM_CAP_ARM_ID_REG_CONFIGURABLE to indicate
that ID registers are writable by userspace.

Signed-off-by: Reiji Watanabe <reijiw@google.com>
---
 Documentation/virt/kvm/api.rst | 8 ++++++++
 arch/arm64/kvm/arm.c           | 1 +
 include/uapi/linux/kvm.h       | 1 +
 3 files changed, 10 insertions(+)

diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst
index aeeb071c7688..022effa4454d 100644
--- a/Documentation/virt/kvm/api.rst
+++ b/Documentation/virt/kvm/api.rst
@@ -7484,3 +7484,11 @@ The argument to KVM_ENABLE_CAP is also a bitmask, and must be a subset
 of the result of KVM_CHECK_EXTENSION.  KVM will forward to userspace
 the hypercalls whose corresponding bit is in the argument, and return
 ENOSYS for the others.
+
+8.35 KVM_CAP_ARM_ID_REG_CONFIGURABLE
+------------------------------------
+
+:Architectures: arm64
+
+This capability indicates that userspace can modify the ID registers
+via KVM_SET_ONE_REG ioctl.
diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
index 19c4a78f931d..a54579e7ac91 100644
--- a/arch/arm64/kvm/arm.c
+++ b/arch/arm64/kvm/arm.c
@@ -215,6 +215,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
 	case KVM_CAP_SET_GUEST_DEBUG:
 	case KVM_CAP_VCPU_ATTRIBUTES:
 	case KVM_CAP_PTP_KVM:
+	case KVM_CAP_ARM_ID_REG_CONFIGURABLE:
 		r = 1;
 		break;
 	case KVM_CAP_SET_GUEST_DEBUG2:
diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
index 1daa45268de2..9697c06a7f5b 100644
--- a/include/uapi/linux/kvm.h
+++ b/include/uapi/linux/kvm.h
@@ -1131,6 +1131,7 @@ struct kvm_ppc_resize_hpt {
 #define KVM_CAP_EXIT_ON_EMULATION_FAILURE 204
 #define KVM_CAP_ARM_MTE 205
 #define KVM_CAP_VM_MOVE_ENC_CONTEXT_FROM 206
+#define KVM_CAP_ARM_ID_REG_CONFIGURABLE 207
 
 #ifdef KVM_CAP_IRQ_ROUTING
 
-- 
2.34.0.rc1.387.gb447b232ab-goog

[RFC PATCH v3 18/29] KVM: arm64: Add kunit test for ID register validation

[Reiji Watanabe]

Add kunit tests for functions that are used for validation of ID
registers and CONFIG_KVM_KUNIT_TEST option to enable the tests.

Since those tests only cover ID register validation so far, only
a few lines of change are needed in the default arm64.py to run
all of those tests as follows.
-----------------------------------------------------------------------
$ diff tools/testing/kunit/qemu_configs/arm64.py arm64_kvm_min.py
4a5,7
> CONFIG_VIRTUALIZATION=y
> CONFIG_KVM=y
> CONFIG_KVM_KUNIT_TEST=y
12c15
< 			   extra_qemu_params=['-machine virt', '-cpu cortex-a57'])
---
> 			   extra_qemu_params=['-M virt,virtualization=on,mte=on', '-cpu max,sve=on'])
$ tools/testing/kunit/kunit.py run --timeout=60 --jobs=16 --arch=arm64 --cross_compile=aarch64-linux-gnu- --qemu_config arm64_kvm_min.py
[19:58:09] Configuring KUnit Kernel ...
[19:58:09] Building KUnit Kernel ...
Populating config with:
$ make ARCH=arm64 olddefconfig CROSS_COMPILE=aarch64-linux-gnu- O=.kunit
Building with:
$ make ARCH=arm64 --jobs=16 CROSS_COMPILE=aarch64-linux-gnu- O=.kunit
[19:58:12] Starting KUnit Kernel (1/1)...
[19:58:12] ============================================================
Running tests with:
$ qemu-system-aarch64 -nodefaults -m 1024 -kernel .kunit/arch/arm64/boot/Image.gz -append 'mem=1G console=tty kunit_shutdown=halt console=ttyAMA0 kunit_shutdown=reboot' -no-reboot -nographic -serial stdio -M virt,virtualization=on,mte=on -cpu max,sve=on
[19:58:14] ========== kvm-sys-regs-test-suite (14 subtests) ===========
[19:58:14] [PASSED] arm64_check_feature_one_test
[19:58:14] [PASSED] arm64_check_features_test
[19:58:14] [PASSED] vcpu_id_reg_feature_frac_check_test
[19:58:14] [PASSED] validate_id_aa64mmfr0_tgran2_test
[19:58:14] [PASSED] validate_id_aa64mmfr0_tgran2_test
[19:58:14] [PASSED] validate_id_aa64mmfr0_tgran2_test
[19:58:14] [PASSED] validate_id_aa64pfr0_el1_test
[19:58:14] [PASSED] validate_id_aa64pfr1_el1_test
[19:58:14] [PASSED] validate_id_aa64isar0_el1_test
[19:58:14] [PASSED] validate_id_aa64isar1_el1_test
[19:58:14] [PASSED] validate_id_aa64mmfr0_el1_test
[19:58:14] [PASSED] validate_id_aa64dfr0_el1_test
[19:58:14] [PASSED] validate_id_dfr0_el1_test
[19:58:14] [PASSED] validate_mvfr1_el1_test
[19:58:14] ============= [PASSED] kvm-sys-regs-test-suite =============
[19:58:14] ============================================================
[19:58:14] Testing complete. Passed: 14, Failed: 0, Crashed: 0, Skipped: 0, Errors: 0
[19:58:14] Elapsed time: 4.644s total, 0.002s configuring, 2.959s building, 1.682s running
-----------------------------------------------------------------------

Signed-off-by: Reiji Watanabe <reijiw@google.com>
---
 arch/arm64/kvm/Kconfig         |  11 +
 arch/arm64/kvm/sys_regs.c      |   4 +
 arch/arm64/kvm/sys_regs_test.c | 871 +++++++++++++++++++++++++++++++++
 3 files changed, 886 insertions(+)
 create mode 100644 arch/arm64/kvm/sys_regs_test.c

diff --git a/arch/arm64/kvm/Kconfig b/arch/arm64/kvm/Kconfig
index 8ffcbe29395e..48fbdd17b2fd 100644
--- a/arch/arm64/kvm/Kconfig
+++ b/arch/arm64/kvm/Kconfig
@@ -54,4 +54,15 @@ config NVHE_EL2_DEBUG
 
 	  If unsure, say N.
 
+config KVM_KUNIT_TEST
+	bool "KUnit tests for KVM on ARM64 processors" if !KUNIT_ALL_TESTS
+	depends on KVM && KUNIT
+	default KUNIT_ALL_TESTS
+	help
+	  Say Y here to enable KUnit tests for the KVM on ARM64.
+	  Only useful for KVM/ARM development and are not for inclusion into
+	  a production build.
+
+	  If unsure, say N.
+
 endif # VIRTUALIZATION
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index b848ecea0c59..2f96103fc0d2 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -3630,3 +3630,7 @@ void kvm_sys_reg_table_init(void)
 
 	id_reg_info_init_all();
 }
+
+#if IS_ENABLED(CONFIG_KVM_KUNIT_TEST)
+#include "sys_regs_test.c"
+#endif
diff --git a/arch/arm64/kvm/sys_regs_test.c b/arch/arm64/kvm/sys_regs_test.c
new file mode 100644
index 000000000000..8d27c7c361fb
--- /dev/null
+++ b/arch/arm64/kvm/sys_regs_test.c
@@ -0,0 +1,871 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * KUnit tests for arch/arm64/kvm/sys_regs.c.
+ */
+
+#include <linux/module.h>
+#include <kunit/test.h>
+#include <kunit/test.h>
+#include <linux/kvm_host.h>
+#include <asm/cpufeature.h>
+#include "asm/sysreg.h"
+
+/* Some utilities for minimal vcpu/kvm setup for existing testings. */
+static struct kvm_vcpu *test_vcpu_init(struct kunit *test, u32 id,
+				       struct kvm *kvm)
+{
+	struct kvm_vcpu *vcpu;
+
+	vcpu = kunit_kzalloc(test, sizeof(*vcpu), GFP_KERNEL);
+	if (!vcpu)
+		return NULL;
+
+	vcpu->cpu = -1;
+	vcpu->kvm = kvm;
+	vcpu->vcpu_id = id;
+
+	return vcpu;
+}
+
+static void test_vcpu_fini(struct kunit *test, struct kvm_vcpu *vcpu)
+{
+	kunit_kfree(test, vcpu);
+}
+
+static struct kvm *test_kvm_init(struct kunit *test)
+{
+	struct kvm *kvm;
+
+	kvm = kunit_kzalloc(test, sizeof(struct kvm), GFP_KERNEL);
+	if (!kvm)
+		return NULL;
+
+	return kvm;
+}
+
+static void test_kvm_fini(struct kunit *test, struct kvm *kvm)
+{
+	kunit_kfree(test, kvm);
+}
+
+static struct kvm_vcpu *test_kvm_vcpu_init(struct kunit *test)
+{
+	struct kvm_vcpu *vcpu;
+	struct kvm *kvm;
+
+	kvm = test_kvm_init(test);
+	if (!kvm)
+		return NULL;
+
+	vcpu = test_vcpu_init(test, 0, kvm);
+	if (!vcpu) {
+		test_kvm_fini(test, kvm);
+		return NULL;
+	}
+	return vcpu;
+}
+
+static void test_kvm_vcpu_fini(struct kunit *test, struct kvm_vcpu *vcpu)
+{
+	struct kvm *kvm = vcpu->kvm;
+
+	test_vcpu_fini(test, vcpu);
+	if (kvm)
+		test_kvm_fini(test, kvm);
+}
+
+/* Test parameter information to test arm64_check_feature_one() */
+struct check_feature_one_test {
+	enum feature_check_type type;
+	int	value;
+	int	limit;
+	int	expected;
+};
+
+struct check_feature_one_test feature_one_params[] = {
+	{FCT_LOWER_SAFE,  0,  0,  0},
+	{FCT_LOWER_SAFE, -1, -1,  0},
+	{FCT_LOWER_SAFE,  1,  1,  0},
+	{FCT_LOWER_SAFE,  1,  2,  0},
+	{FCT_LOWER_SAFE, -1,  0,  0},
+	{FCT_LOWER_SAFE,  2,  1, -1},
+	{FCT_LOWER_SAFE, -1, -2, -1},
+
+	{FCT_HIGHER_SAFE,  0,  0,  0},
+	{FCT_HIGHER_SAFE, -1, -1,  0},
+	{FCT_HIGHER_SAFE,  1,  1,  0},
+	{FCT_HIGHER_SAFE,  1,  2, -1},
+	{FCT_HIGHER_SAFE, -1,  0, -1},
+	{FCT_HIGHER_SAFE,  2,  1,  0},
+	{FCT_HIGHER_SAFE, -1, -2,  0},
+
+	{FCT_HIGHER_OR_ZERO_SAFE,  0,  0,  0},
+	{FCT_HIGHER_OR_ZERO_SAFE, -1, -1,  0},
+	{FCT_HIGHER_OR_ZERO_SAFE,  1,  1,  0},
+	{FCT_HIGHER_OR_ZERO_SAFE,  1,  2, -1},
+	{FCT_HIGHER_OR_ZERO_SAFE, -1,  0, -1},
+	{FCT_HIGHER_OR_ZERO_SAFE,  2,  1,  0},
+	{FCT_HIGHER_OR_ZERO_SAFE, -1, -2,  0},
+	{FCT_HIGHER_OR_ZERO_SAFE,  0,  2,  0},
+
+	{FCT_EXACT,  0,  0,  0},
+	{FCT_EXACT, -1, -1,  0},
+	{FCT_EXACT,  1,  1,  0},
+	{FCT_EXACT,  1,  2, -1},
+	{FCT_EXACT, -1,  0, -1},
+	{FCT_EXACT,  2,  1, -1},
+	{FCT_EXACT, -1, -2, -1},
+
+	{FCT_IGNORE,  0,  0,  0},
+	{FCT_IGNORE, -1, -1,  0},
+	{FCT_IGNORE,  1,  1,  0},
+	{FCT_IGNORE,  1,  2,  0},
+	{FCT_IGNORE, -1,  0,  0},
+	{FCT_IGNORE,  2,  1,  0},
+	{FCT_IGNORE, -1, -2,  0},
+};
+
+static void feature_one_case_to_desc(struct check_feature_one_test *t,
+				     char *desc)
+{
+	snprintf(desc, KUNIT_PARAM_DESC_SIZE,
+		 "type:%x, value:%d, limit:%d\n", t->type, t->value, t->limit);
+}
+
+void arm64_check_feature_one_test(struct kunit *test)
+{
+	const struct check_feature_one_test *ft = test->param_value;
+
+	KUNIT_EXPECT_EQ(test,
+			arm64_check_feature_one(ft->type, ft->value, ft->limit),
+			ft->expected);
+}
+
+KUNIT_ARRAY_PARAM(feature_one, feature_one_params, feature_one_case_to_desc);
+
+
+/* Test parameter information to test arm64_check_features */
+struct check_features_test {
+	u64	check_types;
+	u64	value;
+	u64	limit;
+	int	expected;
+};
+
+#define	U_FEAT_TEST(shift, type, value, limit, exp)	\
+	{U_FCT(shift, type), (u64)value << shift, (u64)limit << shift, exp}
+
+#define	S_FEAT_TEST(shift, type, value, limit, exp)	\
+	{S_FCT(shift, type), (u64)value << shift, (u64)limit << shift, exp}
+
+struct check_features_test features_params[] = {
+	/* Unsigned */
+	U_FEAT_TEST(0, FCT_LOWER_SAFE, 1, 2, 0),
+	U_FEAT_TEST(0, FCT_HIGHER_SAFE, 1, 2, -E2BIG),
+	U_FEAT_TEST(0, FCT_HIGHER_OR_ZERO_SAFE, 1, 2, -E2BIG),
+	U_FEAT_TEST(0, FCT_EXACT, 1, 2, -E2BIG),
+	U_FEAT_TEST(0, FCT_IGNORE, 1, 2, 0),
+	U_FEAT_TEST(0, FCT_LOWER_SAFE, 1, 0xf, 0),
+	U_FEAT_TEST(0, FCT_HIGHER_SAFE, 1, 0xf, -E2BIG),
+	U_FEAT_TEST(0, FCT_HIGHER_OR_ZERO_SAFE, 1, 0xf, -E2BIG),
+	U_FEAT_TEST(0, FCT_EXACT, 1, 0xf, -E2BIG),
+	U_FEAT_TEST(0, FCT_IGNORE, 1, 0xf, 0),
+	U_FEAT_TEST(60, FCT_LOWER_SAFE, 1, 2, 0),
+	U_FEAT_TEST(60, FCT_HIGHER_SAFE, 1, 2, -E2BIG),
+	U_FEAT_TEST(60, FCT_HIGHER_OR_ZERO_SAFE, 1, 2, -E2BIG),
+	U_FEAT_TEST(60, FCT_EXACT, 1, 2, -E2BIG),
+	U_FEAT_TEST(60, FCT_IGNORE, 1, 2, 0),
+	U_FEAT_TEST(60, FCT_LOWER_SAFE, 1, 0xf, 0),
+	U_FEAT_TEST(60, FCT_HIGHER_SAFE, 1, 0xf, -E2BIG),
+	U_FEAT_TEST(60, FCT_HIGHER_OR_ZERO_SAFE, 1, 0xf, -E2BIG),
+	U_FEAT_TEST(60, FCT_EXACT, 1, 0xf, -E2BIG),
+	U_FEAT_TEST(60, FCT_IGNORE, 1, 0xf, 0),
+
+	/* Signed */
+	S_FEAT_TEST(0, FCT_LOWER_SAFE, 1, 2, 0),
+	S_FEAT_TEST(0, FCT_HIGHER_SAFE, 1, 2, -E2BIG),
+	S_FEAT_TEST(0, FCT_HIGHER_OR_ZERO_SAFE, 1, 2, -E2BIG),
+	S_FEAT_TEST(0, FCT_EXACT, 1, 2, -E2BIG),
+	S_FEAT_TEST(0, FCT_IGNORE, 1, 2, 0),
+	S_FEAT_TEST(0, FCT_LOWER_SAFE, 1, 0xf, -E2BIG),
+	S_FEAT_TEST(0, FCT_HIGHER_SAFE, 1, 0xf, 0),
+	S_FEAT_TEST(0, FCT_HIGHER_OR_ZERO_SAFE, 1, 0xf, 0),
+	S_FEAT_TEST(0, FCT_EXACT, 1, 0xf, -E2BIG),
+	S_FEAT_TEST(0, FCT_IGNORE, 1, 0xf, 0),
+	S_FEAT_TEST(60, FCT_LOWER_SAFE, 1, 2, 0),
+	S_FEAT_TEST(60, FCT_HIGHER_SAFE, 1, 2, -E2BIG),
+	S_FEAT_TEST(60, FCT_HIGHER_OR_ZERO_SAFE, 1, 2, -E2BIG),
+	S_FEAT_TEST(60, FCT_EXACT, 1, 2, -E2BIG),
+	S_FEAT_TEST(60, FCT_IGNORE, 1, 2, 0),
+	S_FEAT_TEST(60, FCT_LOWER_SAFE, 1, 0xf, -E2BIG),
+	S_FEAT_TEST(60, FCT_HIGHER_SAFE, 1, 0xf, 0),
+	S_FEAT_TEST(60, FCT_HIGHER_OR_ZERO_SAFE, 1, 0xf, 0),
+	S_FEAT_TEST(60, FCT_EXACT, 1, 0xf, -E2BIG),
+	S_FEAT_TEST(60, FCT_IGNORE, 1, 0xf, 0),
+};
+
+static void features_case_to_desc(struct check_features_test *t, char *desc)
+{
+	snprintf(desc, KUNIT_PARAM_DESC_SIZE,
+		 "check_types:0x%llx, value:0x%llx, limit:0x%llx\n",
+		 t->check_types, t->value, t->limit);
+}
+
+KUNIT_ARRAY_PARAM(features, features_params, features_case_to_desc);
+
+
+void arm64_check_features_test(struct kunit *test)
+{
+	const struct check_features_test *ft = test->param_value;
+
+	KUNIT_EXPECT_EQ(test,
+		arm64_check_features(ft->check_types, ft->value, ft->limit),
+		ft->expected);
+}
+
+
+/* Test parameter information to test vcpu_id_reg_feature_frac_check */
+struct feat_info {
+	u32	id;
+	u32	shift;
+	u32	value;
+	u32	limit;
+	u8	check_type;
+};
+
+struct frac_check_test {
+	struct feat_info feat;
+	struct feat_info frac_feat;
+	int ret;
+};
+
+#define	FEAT(id, shift, value, limit, type)	{id, shift, value, limit, type}
+
+struct frac_check_test frac_params[] = {
+	{
+		FEAT(SYS_ID_AA64PFR1_EL1, 12, 1, 2, U_FCT(0, FCT_LOWER_SAFE)),
+		FEAT(SYS_ID_AA64PFR1_EL1, 32, 1, 1, U_FCT(0, FCT_LOWER_SAFE)),
+		0,
+	},
+	{
+		FEAT(SYS_ID_AA64PFR1_EL1, 12, 1, 2, U_FCT(0, FCT_LOWER_SAFE)),
+		FEAT(SYS_ID_AA64PFR1_EL1, 32, 1, 2, U_FCT(0, FCT_LOWER_SAFE)),
+		0,
+	},
+	{
+		FEAT(SYS_ID_AA64PFR1_EL1, 12, 1, 2, U_FCT(0, FCT_LOWER_SAFE)),
+		FEAT(SYS_ID_AA64PFR1_EL1, 32, 2, 1, U_FCT(0, FCT_LOWER_SAFE)),
+		0,
+	},
+	{
+		FEAT(SYS_ID_AA64PFR1_EL1, 12, 1, 1, U_FCT(0, FCT_LOWER_SAFE)),
+		FEAT(SYS_ID_AA64PFR1_EL1, 32, 1, 1, U_FCT(0, FCT_LOWER_SAFE)),
+		0,
+	},
+	{
+		FEAT(SYS_ID_AA64PFR1_EL1, 12, 1, 1, U_FCT(0, FCT_LOWER_SAFE)),
+		FEAT(SYS_ID_AA64PFR1_EL1, 32, 1, 2, U_FCT(0, FCT_LOWER_SAFE)),
+		0,
+	},
+	{
+		FEAT(SYS_ID_AA64PFR1_EL1, 12, 1, 1, U_FCT(0, FCT_LOWER_SAFE)),
+		FEAT(SYS_ID_AA64PFR1_EL1, 32, 2, 1, U_FCT(0, FCT_LOWER_SAFE)),
+		-E2BIG,
+	},
+
+};
+
+static void frac_case_to_desc(struct frac_check_test *t, char *desc)
+{
+	struct feat_info *feat = &t->feat;
+	struct feat_info *frac = &t->frac_feat;
+
+	snprintf(desc, KUNIT_PARAM_DESC_SIZE,
+		 "feat - shift:%d, val:%d, lim:%d, frac - shift:%d, val:%d, lim:%d, type:%x\n",
+		 feat->shift, feat->value, feat->limit,
+		 frac->shift, frac->value, frac->limit, frac->check_type);
+}
+
+KUNIT_ARRAY_PARAM(frac, frac_params, frac_case_to_desc);
+
+static void vcpu_id_reg_feature_frac_check_test(struct kunit *test)
+{
+	struct kvm_vcpu *vcpu;
+	u32 id, frac_id;
+	struct id_reg_info id_data, frac_id_data;
+	struct id_reg_info *idr, *frac_idr;
+	struct feature_frac frac_data, *frac = &frac_data;
+	const struct frac_check_test *frct = test->param_value;
+
+	vcpu = test_kvm_vcpu_init(test);
+	KUNIT_EXPECT_TRUE(test, vcpu);
+	if (!vcpu)
+		return;
+
+	id = frct->feat.id;
+	frac_id = frct->frac_feat.id;
+
+	frac->id = id;
+	frac->shift = frct->feat.shift;
+	frac->frac_id = frac_id;
+	frac->frac_shift = frct->frac_feat.shift;
+	frac->frac_ftr_check = frct->frac_feat.check_type;
+
+	idr = GET_ID_REG_INFO(id);
+	frac_idr = GET_ID_REG_INFO(frac_id);
+
+	/* Save the original id_reg_info (and restore later) */
+	memcpy(&id_data, idr, sizeof(id_data));
+	memcpy(&frac_id_data, frac_idr, sizeof(frac_id_data));
+
+	/* The id could be same as the frac_id */
+	idr->vcpu_limit_val = (u64)frct->feat.limit << frac->shift;
+	frac_idr->vcpu_limit_val |=
+			(u64)frct->frac_feat.limit << frac->frac_shift;
+
+	__vcpu_sys_reg(vcpu, IDREG_SYS_IDX(id)) =
+			(u64)frct->feat.value << frac->shift;
+	__vcpu_sys_reg(vcpu, IDREG_SYS_IDX(frac_id)) |=
+			(u64)frct->frac_feat.value << frac->frac_shift;
+
+	KUNIT_EXPECT_EQ(test,
+			vcpu_id_reg_feature_frac_check(vcpu, frac),
+			frct->ret);
+
+	test_kvm_vcpu_fini(test, vcpu);
+
+	/* Restore id_reg_info */
+	memcpy(idr, &id_data, sizeof(id_data));
+	memcpy(frac_idr, &frac_id_data, sizeof(frac_id_data));
+}
+
+/*
+ * Test parameter information to test validate_id_aa64mmfr0_tgran2
+ * and validate_id_aa64mmfr0_el1_test.
+ */
+struct tgran_test {
+	int gran2_field;
+	int gran2;
+	int gran2_lim;
+	int gran1;
+	int gran1_lim;
+	int ret;
+};
+
+struct tgran_test tgran4_2_test_params[] = {
+	{ID_AA64MMFR0_TGRAN4_2_SHIFT, 2, 2,  0,  0, 0},
+	{ID_AA64MMFR0_TGRAN4_2_SHIFT, 2, 1,  0,  0, -E2BIG},
+	{ID_AA64MMFR0_TGRAN4_2_SHIFT, 1, 2,  0,  0, 0},
+	{ID_AA64MMFR0_TGRAN4_2_SHIFT, 0, 0,  0,  0, 0},
+	{ID_AA64MMFR0_TGRAN4_2_SHIFT, 0, 1, -1,  0, 0},
+	{ID_AA64MMFR0_TGRAN4_2_SHIFT, 0, 1,  0,  0, -E2BIG},
+	{ID_AA64MMFR0_TGRAN4_2_SHIFT, 0, 2, -1,  0, 0},
+	{ID_AA64MMFR0_TGRAN4_2_SHIFT, 0, 2,  1,  0, -E2BIG},
+	{ID_AA64MMFR0_TGRAN4_2_SHIFT, 1, 0,  0, -1,  0},
+	{ID_AA64MMFR0_TGRAN4_2_SHIFT, 1, 0,  0,  0,  0},
+	{ID_AA64MMFR0_TGRAN4_2_SHIFT, 2, 0,  0, -1,  -E2BIG},
+	{ID_AA64MMFR0_TGRAN4_2_SHIFT, 2, 0,  0,  0,  0},
+	{ID_AA64MMFR0_TGRAN4_2_SHIFT, 2, 0,  0,  2,  0},
+};
+
+struct tgran_test tgran64_2_test_params[] = {
+	{ID_AA64MMFR0_TGRAN64_2_SHIFT, 2, 2,  0,  0, 0},
+	{ID_AA64MMFR0_TGRAN64_2_SHIFT, 2, 1,  0,  0, -E2BIG},
+	{ID_AA64MMFR0_TGRAN64_2_SHIFT, 1, 2,  0,  0, 0},
+	{ID_AA64MMFR0_TGRAN64_2_SHIFT, 0, 0,  0,  0, 0},
+	{ID_AA64MMFR0_TGRAN64_2_SHIFT, 0, 1, -1,  0, 0},
+	{ID_AA64MMFR0_TGRAN64_2_SHIFT, 0, 1,  0,  0, -E2BIG},
+	{ID_AA64MMFR0_TGRAN64_2_SHIFT, 0, 2, -1,  0, 0},
+	{ID_AA64MMFR0_TGRAN64_2_SHIFT, 0, 2,  1,  0, -E2BIG},
+	{ID_AA64MMFR0_TGRAN64_2_SHIFT, 1, 0,  0, -1, 0},
+	{ID_AA64MMFR0_TGRAN64_2_SHIFT, 1, 0,  0,  0, 0},
+	{ID_AA64MMFR0_TGRAN64_2_SHIFT, 2, 0,  0, -1, -E2BIG},
+	{ID_AA64MMFR0_TGRAN64_2_SHIFT, 2, 0,  0,  0, 0},
+	{ID_AA64MMFR0_TGRAN64_2_SHIFT, 2, 0,  0,  2, 0},
+};
+
+struct tgran_test tgran16_2_test_params[] = {
+	{ID_AA64MMFR0_TGRAN16_2_SHIFT, 2, 2,  0,  0, 0},
+	{ID_AA64MMFR0_TGRAN16_2_SHIFT, 2, 1,  0,  0, -E2BIG},
+	{ID_AA64MMFR0_TGRAN16_2_SHIFT, 1, 2,  0,  0, 0},
+	{ID_AA64MMFR0_TGRAN16_2_SHIFT, 0, 0,  0,  0, 0},
+	{ID_AA64MMFR0_TGRAN16_2_SHIFT, 0, 1,  0,  0, 0},
+	{ID_AA64MMFR0_TGRAN16_2_SHIFT, 0, 1,  1,  0, -E2BIG},
+	{ID_AA64MMFR0_TGRAN16_2_SHIFT, 0, 2,  0,  0, 0},
+	{ID_AA64MMFR0_TGRAN16_2_SHIFT, 0, 2,  2,  0, -E2BIG},
+	{ID_AA64MMFR0_TGRAN16_2_SHIFT, 1, 0,  0,  0, 0},
+	{ID_AA64MMFR0_TGRAN16_2_SHIFT, 1, 0,  0,  1, 0},
+	{ID_AA64MMFR0_TGRAN16_2_SHIFT, 2, 0,  0,  0, -E2BIG},
+	{ID_AA64MMFR0_TGRAN16_2_SHIFT, 2, 0,  0,  1, 0},
+	{ID_AA64MMFR0_TGRAN16_2_SHIFT, 2, 0,  0,  2, 0},
+};
+
+static void tgran2_case_to_desc(struct tgran_test *t, char *desc)
+{
+	snprintf(desc, KUNIT_PARAM_DESC_SIZE,
+		 "gran2(field=%d): val=%d, lim=%d gran1: val=%d limit=%d\n",
+		 t->gran2_field, t->gran2, t->gran2_lim,
+		 t->gran1, t->gran1_lim);
+}
+
+KUNIT_ARRAY_PARAM(tgran4_2, tgran4_2_test_params, tgran2_case_to_desc);
+KUNIT_ARRAY_PARAM(tgran64_2, tgran64_2_test_params, tgran2_case_to_desc);
+KUNIT_ARRAY_PARAM(tgran16_2, tgran16_2_test_params, tgran2_case_to_desc);
+
+#define	MAKE_MMFR0_TGRAN(shift1, gran1, shift2, gran2)		\
+	(((u64)((gran1) & 0xf) << (shift1)) |			\
+	 ((u64)((gran2) & 0xf) << (shift2)))
+
+static int tgran2_to_tgran1_shift(int tgran2_shift)
+{
+	int tgran1_shift = -1;
+
+	switch (tgran2_shift) {
+	case ID_AA64MMFR0_TGRAN4_2_SHIFT:
+		tgran1_shift = ID_AA64MMFR0_TGRAN4_SHIFT;
+		break;
+	case ID_AA64MMFR0_TGRAN64_2_SHIFT:
+		tgran1_shift = ID_AA64MMFR0_TGRAN64_SHIFT;
+		break;
+	case ID_AA64MMFR0_TGRAN16_2_SHIFT:
+		tgran1_shift = ID_AA64MMFR0_TGRAN16_SHIFT;
+		break;
+	default:
+		break;
+	}
+
+	return tgran1_shift;
+}
+
+static void validate_id_aa64mmfr0_tgran2_test(struct kunit *test)
+{
+	const struct tgran_test *t = test->param_value;
+	int shift1, shift2;
+	u64 v, lim;
+
+	shift2 = t->gran2_field;
+	shift1 = tgran2_to_tgran1_shift(shift2);
+	v = MAKE_MMFR0_TGRAN(shift1, t->gran1, shift2, t->gran2);
+	lim = MAKE_MMFR0_TGRAN(shift1, t->gran1_lim, shift2, t->gran2_lim);
+
+	KUNIT_EXPECT_EQ(test, aa64mmfr0_tgran2_check(shift2, v, lim), t->ret);
+}
+
+static void validate_id_aa64pfr0_el1_test(struct kunit *test)
+{
+	struct id_reg_info *id_reg;
+	struct kvm_vcpu *vcpu;
+	u64 v;
+
+	vcpu = test_kvm_vcpu_init(test);
+	KUNIT_EXPECT_TRUE(test, vcpu);
+	if (!vcpu)
+		return;
+
+	id_reg = GET_ID_REG_INFO(SYS_ID_AA64PFR0_EL1);
+
+	v = 0;
+	KUNIT_EXPECT_EQ(test, validate_id_aa64pfr0_el1(vcpu, id_reg, v), 0);
+
+	v = 0x000010000;	/* ASIMD = 0, FP = 1 */
+	KUNIT_EXPECT_NE(test, validate_id_aa64pfr0_el1(vcpu, id_reg, v), 0);
+
+	v = 0x000100000;	/* ASIMD = 1, FP = 0 */
+	KUNIT_EXPECT_NE(test, validate_id_aa64pfr0_el1(vcpu, id_reg, v), 0);
+
+	v = 0x000ff0000;	/* ASIMD = 0xf, FP = 0xf */
+	KUNIT_EXPECT_EQ(test, validate_id_aa64pfr0_el1(vcpu, id_reg, v), 0);
+
+	v = 0x100000000;	/* SVE =1, ASIMD = 0, FP = 0 */
+	KUNIT_EXPECT_NE(test, validate_id_aa64pfr0_el1(vcpu, id_reg, v), 0);
+	if (!system_supports_sve()) {
+		test_kvm_vcpu_fini(test, vcpu);
+		kunit_skip(test, "No SVE support. Partial skip)");
+		/* Not reached */
+	}
+
+	vcpu->arch.flags |= KVM_ARM64_GUEST_HAS_SVE;
+
+	v = 0x100000000;	/* SVE =1, ASIMD = 0, FP = 0 */
+	KUNIT_EXPECT_EQ(test, validate_id_aa64pfr0_el1(vcpu, id_reg, v), 0);
+
+	v = 0x100ff0000;	/* SVE =1, ASIMD = 0, FP = 0 */
+	KUNIT_EXPECT_NE(test, validate_id_aa64pfr0_el1(vcpu, id_reg, v), 0);
+
+	vcpu->arch.flags &= ~KVM_ARM64_GUEST_HAS_SVE;
+
+	v = 0x1000000;		/* GIC = 1 */
+	KUNIT_EXPECT_NE(test, validate_id_aa64pfr0_el1(vcpu, id_reg, v), 0);
+
+	vcpu->kvm->arch.vgic.in_kernel = true;
+	v = 0x1000000;		/* GIC = 1 */
+	KUNIT_EXPECT_NE(test, validate_id_aa64pfr0_el1(vcpu, id_reg, v), 0);
+
+	vcpu->kvm->arch.vgic.vgic_model = KVM_DEV_TYPE_ARM_VGIC_V2;
+	v = 0x1000000;		/* GIC = 1 */
+	KUNIT_EXPECT_NE(test, validate_id_aa64pfr0_el1(vcpu, id_reg, v), 0);
+
+	v = 0;		/* GIC = 0 */
+	KUNIT_EXPECT_EQ(test, validate_id_aa64pfr0_el1(vcpu, id_reg, v), 0);
+
+	v = 0x1000000;		/* GIC = 1 */
+	vcpu->kvm->arch.vgic.vgic_model = KVM_DEV_TYPE_ARM_VGIC_V3;
+	KUNIT_EXPECT_EQ(test, validate_id_aa64pfr0_el1(vcpu, id_reg, v), 0);
+
+	test_kvm_vcpu_fini(test, vcpu);
+}
+
+static void validate_id_aa64pfr1_el1_test(struct kunit *test)
+{
+	struct id_reg_info *id_reg;
+	struct kvm_vcpu *vcpu;
+	u64 v;
+
+	vcpu = test_kvm_vcpu_init(test);
+	KUNIT_EXPECT_TRUE(test, vcpu);
+	if (!vcpu)
+		return;
+
+	id_reg = GET_ID_REG_INFO(SYS_ID_AA64PFR1_EL1);
+
+	v = 0;
+	KUNIT_EXPECT_EQ(test, validate_id_aa64pfr1_el1(vcpu, id_reg, v), 0);
+
+	v = 0x100;	/* MTE = 1*/
+	KUNIT_EXPECT_NE(test, validate_id_aa64pfr1_el1(vcpu, id_reg, v), 0);
+
+	if (!system_supports_mte()) {
+		test_kvm_vcpu_fini(test, vcpu);
+		kunit_skip(test, "(No MTE support. Partial skip)");
+		/* Not reached */
+	}
+
+	vcpu->kvm->arch.mte_enabled = true;
+
+	v = 0x100;	/* MTE = 1*/
+	KUNIT_EXPECT_EQ(test, validate_id_aa64pfr1_el1(vcpu, id_reg, v), 0);
+
+	v = 0x0;
+	vcpu->kvm->arch.mte_enabled = true;
+	KUNIT_EXPECT_NE(test, validate_id_aa64pfr1_el1(vcpu, id_reg, v), 0);
+
+	test_kvm_vcpu_fini(test, vcpu);
+}
+
+static void validate_id_aa64isar0_el1_test(struct kunit *test)
+{
+	struct id_reg_info *id_reg;
+	struct kvm_vcpu *vcpu;
+	u64 v;
+
+	vcpu = test_kvm_vcpu_init(test);
+	KUNIT_EXPECT_TRUE(test, vcpu);
+	if (!vcpu)
+		return;
+
+	id_reg = GET_ID_REG_INFO(SYS_ID_AA64ISAR0_EL1);
+
+	v = 0;
+	KUNIT_EXPECT_EQ(test, validate_id_aa64isar0_el1(vcpu, id_reg, v), 0);
+
+	v = 0x01000000000;	/* SM4 = 0, SM3 = 1 */
+	KUNIT_EXPECT_NE(test, validate_id_aa64isar0_el1(vcpu, id_reg, v), 0);
+
+	v = 0x10000000000;	/* SM4 = 1, SM3 = 0 */
+	KUNIT_EXPECT_NE(test, validate_id_aa64isar0_el1(vcpu, id_reg, v), 0);
+
+	v = 0x11000000000;	/* SM3 = SM4 = 1 */
+	KUNIT_EXPECT_EQ(test, validate_id_aa64isar0_el1(vcpu, id_reg, v), 0);
+
+	v = 0x000000100;	/* SHA2 = 0, SHA1 = 1 */
+	KUNIT_EXPECT_NE(test, validate_id_aa64isar0_el1(vcpu, id_reg, v), 0);
+
+	v = 0x000001000;	/* SHA2 = 1, SHA1 = 0 */
+	KUNIT_EXPECT_NE(test, validate_id_aa64isar0_el1(vcpu, id_reg, v), 0);
+
+	v = 0x000001100;	/* SHA2 = 1, SHA1 = 1 */
+	KUNIT_EXPECT_EQ(test, validate_id_aa64isar0_el1(vcpu, id_reg, v), 0);
+
+	v = 0x100002000;	/* SHA3 = 1, SHA2 = 2 */
+	KUNIT_EXPECT_NE(test, validate_id_aa64isar0_el1(vcpu, id_reg, v), 0);
+
+	v = 0x000002000;	/* SHA3 = 0, SHA2 = 2 */
+	KUNIT_EXPECT_NE(test, validate_id_aa64isar0_el1(vcpu, id_reg, v), 0);
+
+	v = 0x100001000;	/* SHA3 = 1, SHA2 = 1 */
+	KUNIT_EXPECT_NE(test, validate_id_aa64isar0_el1(vcpu, id_reg, v), 0);
+
+	v = 0x200000000;	/* SHA3 = 2, SHA1 = 0 */
+	KUNIT_EXPECT_NE(test, validate_id_aa64isar0_el1(vcpu, id_reg, v), 0);
+
+	v = 0x200001100;	/* SHA3 = 2, SHA2= 1, SHA1 = 1 */
+	KUNIT_EXPECT_EQ(test, validate_id_aa64isar0_el1(vcpu, id_reg, v), 0);
+
+	v = 0x300003300;	/* SHA3 = 3, SHA2 = 3, SHA1 = 3 */
+	KUNIT_EXPECT_EQ(test, validate_id_aa64isar0_el1(vcpu, id_reg, v), 0);
+
+	test_kvm_vcpu_fini(test, vcpu);
+}
+
+static void validate_id_aa64isar1_el1_test(struct kunit *test)
+{
+	struct id_reg_info *id_reg;
+	struct kvm_vcpu *vcpu;
+	u64 v;
+
+	vcpu = test_kvm_vcpu_init(test);
+	KUNIT_EXPECT_TRUE(test, vcpu);
+	if (!vcpu)
+		return;
+
+	id_reg = GET_ID_REG_INFO(SYS_ID_AA64ISAR1_EL1);
+
+	v = 0;
+	KUNIT_EXPECT_EQ(test, validate_id_aa64isar1_el1(vcpu, id_reg, v), 0);
+
+	v = 0x11000110;	/* GPI = 1, GPA = 1, API = 1, APA = 1 */
+	KUNIT_EXPECT_NE(test, validate_id_aa64isar1_el1(vcpu, id_reg, v), 0);
+
+	v = 0x11000100;	/* GPI = 1, GPA = 1, API = 1 */
+	KUNIT_EXPECT_NE(test, validate_id_aa64isar1_el1(vcpu, id_reg, v), 0);
+
+	v = 0x11000010;	/* GPI = 1, GPA = 1, APA = 1 */
+	KUNIT_EXPECT_NE(test, validate_id_aa64isar1_el1(vcpu, id_reg, v), 0);
+
+	v = 0x10000110;	/* GPI = 1, API = 1, APA = 1 */
+	KUNIT_EXPECT_NE(test, validate_id_aa64isar1_el1(vcpu, id_reg, v), 0);
+
+	v = 0x01000110;	/* GPA = 1, API = 1, APA = 1 */
+	KUNIT_EXPECT_NE(test, validate_id_aa64isar1_el1(vcpu, id_reg, v), 0);
+
+	if (!system_has_full_ptr_auth()) {
+		test_kvm_vcpu_fini(test, vcpu);
+		kunit_skip(test, "(No PTRAUTH support. Partial skip)");
+		/* Not reached */
+	}
+
+	vcpu->arch.flags |= KVM_ARM64_GUEST_HAS_PTRAUTH;
+
+	v = 0x10000100;	/* GPI = 1, API = 1 */
+	KUNIT_EXPECT_EQ(test, validate_id_aa64isar1_el1(vcpu, id_reg, v), 0);
+
+	v = 0x10000010;	/* GPI = 1, APA = 1 */
+	KUNIT_EXPECT_EQ(test, validate_id_aa64isar1_el1(vcpu, id_reg, v), 0);
+
+	v = 0x01000100;	/* GPA = 1, API = 1 */
+	KUNIT_EXPECT_EQ(test, validate_id_aa64isar1_el1(vcpu, id_reg, v), 0);
+
+	v = 0x01000010;	/* GPA = 1, APA = 1 */
+	KUNIT_EXPECT_EQ(test, validate_id_aa64isar1_el1(vcpu, id_reg, v), 0);
+
+	v = 0;
+	KUNIT_EXPECT_NE(test, validate_id_aa64isar1_el1(vcpu, id_reg, v), 0);
+
+	test_kvm_vcpu_fini(test, vcpu);
+}
+
+static void validate_id_aa64mmfr0_el1_test(struct kunit *test)
+{
+	struct id_reg_info id_data, *id_reg;
+	const struct tgran_test *t4, *t64, *t16;
+	struct kvm_vcpu *vcpu;
+	int field4, field4_2, field64, field64_2, field16, field16_2;
+	u64 v, v4, lim4, v64, lim64, v16, lim16;
+	int i, j, ret;
+
+	id_reg = GET_ID_REG_INFO(SYS_ID_AA64MMFR0_EL1);
+
+	/* Save the original id_reg_info (and restore later) */
+	memcpy(&id_data, id_reg, sizeof(id_data));
+
+	vcpu = test_kvm_vcpu_init(test);
+
+	t4 = test->param_value;
+	field4_2 = t4->gran2_field;
+	field4 = tgran2_to_tgran1_shift(field4_2);
+	v4 = MAKE_MMFR0_TGRAN(field4, t4->gran1, field4_2, t4->gran2);
+	lim4 = MAKE_MMFR0_TGRAN(field4, t4->gran1_lim, field4_2, t4->gran2_lim);
+
+	/*
+	 * For each given gran4_2 params, test validate_id_aa64mmfr0_el1
+	 * with each of tgran64_2 and tgran16_2 params.
+	 */
+	for (i = 0; i < ARRAY_SIZE(tgran64_2_test_params); i++) {
+		t64 = &tgran64_2_test_params[i];
+		field64_2 = t64->gran2_field;
+		field64 = tgran2_to_tgran1_shift(field64_2);
+		v64 = MAKE_MMFR0_TGRAN(field64, t64->gran1,
+				       field64_2, t64->gran2);
+		lim64 = MAKE_MMFR0_TGRAN(field64, t64->gran1_lim,
+					 field64_2, t64->gran2_lim);
+
+		for (j = 0; j < ARRAY_SIZE(tgran16_2_test_params); j++) {
+			t16 = &tgran16_2_test_params[j];
+
+			field16_2 = t16->gran2_field;
+			field16 = tgran2_to_tgran1_shift(field16_2);
+			v16 = MAKE_MMFR0_TGRAN(field16, t16->gran1,
+					       field16_2, t16->gran2);
+			lim16 = MAKE_MMFR0_TGRAN(field16, t16->gran1_lim,
+						 field16_2, t16->gran2_lim);
+
+			/* Build id_aa64mmfr0_el1 from tgran16/64/4 values */
+			v = v16 | v64 | v4;
+			id_reg->vcpu_limit_val = lim16 | lim64 | lim4;
+
+			ret = t4->ret ? t4->ret : t64->ret;
+			ret = ret ? ret : t16->ret;
+			KUNIT_EXPECT_EQ(test,
+				validate_id_aa64mmfr0_el1(vcpu, id_reg, v),
+				ret);
+		}
+	}
+
+	/* Restore id_reg_info */
+	memcpy(id_reg, &id_data, sizeof(id_data));
+	test_kvm_vcpu_fini(test, vcpu);
+}
+
+static void validate_id_aa64dfr0_el1_test(struct kunit *test)
+{
+	struct id_reg_info *id_reg;
+	struct kvm_vcpu *vcpu;
+	u64 v;
+
+	id_reg = GET_ID_REG_INFO(SYS_ID_AA64DFR0_EL1);
+	vcpu = test_kvm_vcpu_init(test);
+	KUNIT_EXPECT_TRUE(test, vcpu);
+	if (!vcpu)
+		return;
+
+	v = 0;
+	KUNIT_EXPECT_EQ(test, validate_id_aa64dfr0_el1(vcpu, id_reg, v), 0);
+
+	v = 0x10001000;	/* CTX_CMPS = 2, BRPS = 1 */
+	KUNIT_EXPECT_EQ(test, validate_id_aa64dfr0_el1(vcpu, id_reg, v), 0);
+
+	v = 0x20001000;	/* CTX_CMPS = 2, BRPS = 1 */
+	KUNIT_EXPECT_NE(test, validate_id_aa64dfr0_el1(vcpu, id_reg, v), 0);
+
+	v = 0xf00;	/* PMUVER = 0xf */
+	KUNIT_EXPECT_EQ(test, validate_id_aa64dfr0_el1(vcpu, id_reg, v), 0);
+
+	v = 0x100;	/* PMUVER = 1 */
+	KUNIT_EXPECT_NE(test, validate_id_aa64dfr0_el1(vcpu, id_reg, v), 0);
+
+	set_bit(KVM_ARM_VCPU_PMU_V3, vcpu->arch.features);
+
+	v = 0x100;	/* PMUVER = 1 */
+	KUNIT_EXPECT_EQ(test, validate_id_aa64dfr0_el1(vcpu, id_reg, v), 0);
+
+	v = 0x0;	/* PMUVER = 0 */
+	KUNIT_EXPECT_NE(test, validate_id_aa64dfr0_el1(vcpu, id_reg, v), 0);
+
+	test_kvm_vcpu_fini(test, vcpu);
+}
+
+static void validate_id_dfr0_el1_test(struct kunit *test)
+{
+	struct id_reg_info *id_reg;
+	struct kvm_vcpu *vcpu;
+	u64 v;
+
+	id_reg = GET_ID_REG_INFO(SYS_ID_DFR0_EL1);
+	vcpu = test_kvm_vcpu_init(test);
+	KUNIT_EXPECT_TRUE(test, vcpu);
+	if (!vcpu)
+		return;
+
+	v = 0;
+	KUNIT_EXPECT_EQ(test, validate_id_dfr0_el1(vcpu, id_reg, v), 0);
+
+	v = 0xf000000;	/* PERFMON = 0xf */
+	KUNIT_EXPECT_EQ(test, validate_id_dfr0_el1(vcpu, id_reg, v), 0);
+
+	v = 0x1000000;	/* PERFMON = 1 */
+	KUNIT_EXPECT_NE(test, validate_id_dfr0_el1(vcpu, id_reg, v), 0);
+
+	v = 0x2000000;	/* PERFMON = 2 */
+	KUNIT_EXPECT_NE(test, validate_id_dfr0_el1(vcpu, id_reg, v), 0);
+
+	v = 0x3000000;	/* PERFMON = 3 */
+	KUNIT_EXPECT_NE(test, validate_id_dfr0_el1(vcpu, id_reg, v), 0);
+
+	set_bit(KVM_ARM_VCPU_PMU_V3, vcpu->arch.features);
+
+	v = 0x1000000;	/* PERFMON = 1 */
+	KUNIT_EXPECT_NE(test, validate_id_dfr0_el1(vcpu, id_reg, v), 0);
+
+	v = 0x2000000;	/* PERFMON = 2 */
+	KUNIT_EXPECT_NE(test, validate_id_dfr0_el1(vcpu, id_reg, v), 0);
+
+	v = 0x3000000;	/* PERFMON = 3 */
+	KUNIT_EXPECT_EQ(test, validate_id_dfr0_el1(vcpu, id_reg, v), 0);
+
+	v = 0xf000000;	/* PERFMON = 0xf */
+	KUNIT_EXPECT_NE(test, validate_id_dfr0_el1(vcpu, id_reg, v), 0);
+
+	test_kvm_vcpu_fini(test, vcpu);
+}
+
+static void validate_mvfr1_el1_test(struct kunit *test)
+{
+	struct id_reg_info *id_reg;
+	struct kvm_vcpu *vcpu;
+	u64 v;
+
+	id_reg = GET_ID_REG_INFO(SYS_MVFR1_EL1);
+	vcpu = test_kvm_vcpu_init(test);
+	KUNIT_EXPECT_TRUE(test, vcpu);
+	if (!vcpu)
+		return;
+
+	v = 0;
+	KUNIT_EXPECT_EQ(test, validate_mvfr1_el1(vcpu, id_reg, v), 0);
+
+	v = 0x2100000;	/* FPHP = 2, SIMDHP = 1 */
+	KUNIT_EXPECT_EQ(test, validate_mvfr1_el1(vcpu, id_reg, v), 0);
+
+	v = 0x3200000;	/* FPHP = 3, SIMDHP = 2 */
+	KUNIT_EXPECT_EQ(test, validate_mvfr1_el1(vcpu, id_reg, v), 0);
+
+	v = 0x1100000;	/* FPHP = 1, SIMDHP = 1 */
+	KUNIT_EXPECT_NE(test, validate_mvfr1_el1(vcpu, id_reg, v), 0);
+
+	v = 0x2200000;	/* FPHP = 2, SIMDHP = 2 */
+	KUNIT_EXPECT_NE(test, validate_mvfr1_el1(vcpu, id_reg, v), 0);
+
+	v = 0x3300000;	/* FPHP = 3, SIMDHP = 3 */
+	KUNIT_EXPECT_NE(test, validate_mvfr1_el1(vcpu, id_reg, v), 0);
+
+	v = (u64)-1;
+	KUNIT_EXPECT_NE(test, validate_mvfr1_el1(vcpu, id_reg, v), 0);
+
+	test_kvm_vcpu_fini(test, vcpu);
+}
+
+static struct kunit_case kvm_sys_regs_test_cases[] = {
+	KUNIT_CASE_PARAM(arm64_check_feature_one_test, feature_one_gen_params),
+	KUNIT_CASE_PARAM(arm64_check_features_test, features_gen_params),
+	KUNIT_CASE_PARAM(vcpu_id_reg_feature_frac_check_test, frac_gen_params),
+	KUNIT_CASE_PARAM(validate_id_aa64mmfr0_tgran2_test, tgran4_2_gen_params),
+	KUNIT_CASE_PARAM(validate_id_aa64mmfr0_tgran2_test, tgran64_2_gen_params),
+	KUNIT_CASE_PARAM(validate_id_aa64mmfr0_tgran2_test, tgran16_2_gen_params),
+	KUNIT_CASE(validate_id_aa64pfr0_el1_test),
+	KUNIT_CASE(validate_id_aa64pfr1_el1_test),
+	KUNIT_CASE(validate_id_aa64isar0_el1_test),
+	KUNIT_CASE(validate_id_aa64isar1_el1_test),
+	KUNIT_CASE_PARAM(validate_id_aa64mmfr0_el1_test, tgran4_2_gen_params),
+	KUNIT_CASE(validate_id_aa64dfr0_el1_test),
+	KUNIT_CASE(validate_id_dfr0_el1_test),
+	KUNIT_CASE(validate_mvfr1_el1_test),
+	{}
+};
+
+static struct kunit_suite kvm_sys_regs_test_suite = {
+	.name = "kvm-sys-regs-test-suite",
+	.test_cases = kvm_sys_regs_test_cases,
+};
+
+kunit_test_suites(&kvm_sys_regs_test_suite);
+MODULE_LICENSE("GPL");
-- 
2.34.0.rc1.387.gb447b232ab-goog

[RFC PATCH v3 19/29] KVM: arm64: Use vcpu->arch cptr_el2 to track value of cptr_el2 for VHE

[Reiji Watanabe]

Track the baseline guest value for cptr_el2 in struct kvm_vcpu_arch
for VHE.  Use this value when setting cptr_el2 for the guest.

Currently this value is unchanged, but the following patches will set
trapping bits based on features supported for the guest.

No functional change intended.

Signed-off-by: Reiji Watanabe <reijiw@google.com>
---
 arch/arm64/include/asm/kvm_arm.h | 16 ++++++++++++++++
 arch/arm64/kvm/arm.c             |  5 ++++-
 arch/arm64/kvm/hyp/vhe/switch.c  | 14 ++------------
 3 files changed, 22 insertions(+), 13 deletions(-)

diff --git a/arch/arm64/include/asm/kvm_arm.h b/arch/arm64/include/asm/kvm_arm.h
index a39fcf318c77..c1e0e1202f30 100644
--- a/arch/arm64/include/asm/kvm_arm.h
+++ b/arch/arm64/include/asm/kvm_arm.h
@@ -288,6 +288,22 @@
 				 GENMASK(19, 14) |	\
 				 BIT(11))
 
+/*
+ * With VHE (HCR.E2H == 1), accesses to CPACR_EL1 are routed to
+ * CPTR_EL2. In general, CPACR_EL1 has the same layout as CPTR_EL2,
+ * except for some missing controls, such as TAM.
+ * In this case, CPTR_EL2.TAM has the same position with or without
+ * VHE (HCR.E2H == 1) which allows us to use here the CPTR_EL2.TAM
+ * shift value for trapping the AMU accesses.
+ */
+#define CPTR_EL2_VHE_GUEST_DEFAULT	(CPACR_EL1_TTA | CPTR_EL2_TAM)
+
+/*
+ * Bits that are copied from vcpu->arch.cptr_el2 to set cptr_el2 for
+ * guest with VHE.
+ */
+#define CPTR_EL2_VHE_GUEST_TRACKED_MASK	(CPACR_EL1_TTA | CPTR_EL2_TAM)
+
 /* Hyp Debug Configuration Register bits */
 #define MDCR_EL2_E2TB_MASK	(UL(0x3))
 #define MDCR_EL2_E2TB_SHIFT	(UL(24))
diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
index a54579e7ac91..aa4aff2588b8 100644
--- a/arch/arm64/kvm/arm.c
+++ b/arch/arm64/kvm/arm.c
@@ -1128,7 +1128,10 @@ static int kvm_arch_vcpu_ioctl_vcpu_init(struct kvm_vcpu *vcpu,
 	}
 
 	vcpu_reset_hcr(vcpu);
-	vcpu->arch.cptr_el2 = CPTR_EL2_DEFAULT;
+	if (has_vhe())
+		vcpu->arch.cptr_el2 = CPTR_EL2_VHE_GUEST_DEFAULT;
+	else
+		vcpu->arch.cptr_el2 = CPTR_EL2_DEFAULT;
 
 	/*
 	 * Handle the "start in power-off" case.
diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c
index 5a2cb5d9bc4b..0c3f0bf3fbf2 100644
--- a/arch/arm64/kvm/hyp/vhe/switch.c
+++ b/arch/arm64/kvm/hyp/vhe/switch.c
@@ -38,20 +38,10 @@ static void __activate_traps(struct kvm_vcpu *vcpu)
 	___activate_traps(vcpu);
 
 	val = read_sysreg(cpacr_el1);
-	val |= CPACR_EL1_TTA;
+	val &= ~CPTR_EL2_VHE_GUEST_TRACKED_MASK;
+	val |= (vcpu->arch.cptr_el2 & CPTR_EL2_VHE_GUEST_TRACKED_MASK);
 	val &= ~CPACR_EL1_ZEN;
 
-	/*
-	 * With VHE (HCR.E2H == 1), accesses to CPACR_EL1 are routed to
-	 * CPTR_EL2. In general, CPACR_EL1 has the same layout as CPTR_EL2,
-	 * except for some missing controls, such as TAM.
-	 * In this case, CPTR_EL2.TAM has the same position with or without
-	 * VHE (HCR.E2H == 1) which allows us to use here the CPTR_EL2.TAM
-	 * shift value for trapping the AMU accesses.
-	 */
-
-	val |= CPTR_EL2_TAM;
-
 	if (update_fp_enabled(vcpu)) {
 		if (vcpu_has_sve(vcpu))
 			val |= CPACR_EL1_ZEN;
-- 
2.34.0.rc1.387.gb447b232ab-goog

[RFC PATCH v3 20/29] KVM: arm64: Use vcpu->arch.mdcr_el2 to track value of mdcr_el2

[Reiji Watanabe]

Track the baseline guest value for mdcr_el2 in struct kvm_vcpu_arch.
Use this value when setting mdcr_el2 for the guest.

Currently this value is unchanged, but the following patches will set
trapping bits based on features supported for the guest.

No functional change intended.

Signed-off-by: Reiji Watanabe <reijiw@google.com>
---
 arch/arm64/include/asm/kvm_arm.h | 16 ++++++++++++++++
 arch/arm64/kvm/arm.c             |  1 +
 arch/arm64/kvm/debug.c           | 13 ++++---------
 3 files changed, 21 insertions(+), 9 deletions(-)

diff --git a/arch/arm64/include/asm/kvm_arm.h b/arch/arm64/include/asm/kvm_arm.h
index c1e0e1202f30..e81f21995314 100644
--- a/arch/arm64/include/asm/kvm_arm.h
+++ b/arch/arm64/include/asm/kvm_arm.h
@@ -333,6 +333,22 @@
 				 BIT(18) |		\
 				 GENMASK(16, 15))
 
+/*
+ * The default value for the guest below also clears MDCR_EL2_E2PB_MASK
+ * and MDCR_EL2_E2TB_MASK to disable guest access to the profiling and
+ * trace buffers.
+ */
+#define MDCR_GUEST_FLAGS_DEFAULT				\
+	(MDCR_EL2_TPM  | MDCR_EL2_TPMS | MDCR_EL2_TTRF |	\
+	 MDCR_EL2_TPMCR | MDCR_EL2_TDRA | MDCR_EL2_TDOSA)
+
+/* Bits that are copied from vcpu->arch.mdcr_el2 to set mdcr_el2 for guest. */
+#define MDCR_GUEST_FLAGS_TRACKED_MASK				\
+	(MDCR_EL2_TPM  | MDCR_EL2_TPMS | MDCR_EL2_TTRF |	\
+	 MDCR_EL2_TPMCR | MDCR_EL2_TDRA | MDCR_EL2_TDOSA |	\
+	 (MDCR_EL2_E2PB_MASK << MDCR_EL2_E2PB_SHIFT))
+
+
 /* For compatibility with fault code shared with 32-bit */
 #define FSC_FAULT	ESR_ELx_FSC_FAULT
 #define FSC_ACCESS	ESR_ELx_FSC_ACCESS
diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
index aa4aff2588b8..64b104ebee73 100644
--- a/arch/arm64/kvm/arm.c
+++ b/arch/arm64/kvm/arm.c
@@ -1128,6 +1128,7 @@ static int kvm_arch_vcpu_ioctl_vcpu_init(struct kvm_vcpu *vcpu,
 	}
 
 	vcpu_reset_hcr(vcpu);
+	vcpu->arch.mdcr_el2 = MDCR_GUEST_FLAGS_DEFAULT;
 	if (has_vhe())
 		vcpu->arch.cptr_el2 = CPTR_EL2_VHE_GUEST_DEFAULT;
 	else
diff --git a/arch/arm64/kvm/debug.c b/arch/arm64/kvm/debug.c
index db9361338b2a..83330968a411 100644
--- a/arch/arm64/kvm/debug.c
+++ b/arch/arm64/kvm/debug.c
@@ -84,16 +84,11 @@ void kvm_arm_init_debug(void)
 static void kvm_arm_setup_mdcr_el2(struct kvm_vcpu *vcpu)
 {
 	/*
-	 * This also clears MDCR_EL2_E2PB_MASK and MDCR_EL2_E2TB_MASK
-	 * to disable guest access to the profiling and trace buffers
+	 * Keep the vcpu->arch.mdcr_el2 bits that are specified by
+	 * MDCR_GUEST_FLAGS_TRACKED_MASK.
 	 */
-	vcpu->arch.mdcr_el2 = __this_cpu_read(mdcr_el2) & MDCR_EL2_HPMN_MASK;
-	vcpu->arch.mdcr_el2 |= (MDCR_EL2_TPM |
-				MDCR_EL2_TPMS |
-				MDCR_EL2_TTRF |
-				MDCR_EL2_TPMCR |
-				MDCR_EL2_TDRA |
-				MDCR_EL2_TDOSA);
+	vcpu->arch.mdcr_el2 &= MDCR_GUEST_FLAGS_TRACKED_MASK;
+	vcpu->arch.mdcr_el2 |= __this_cpu_read(mdcr_el2) & MDCR_EL2_HPMN_MASK;
 
 	/* Is the VM being debugged by userspace? */
 	if (vcpu->guest_debug)
-- 
2.34.0.rc1.387.gb447b232ab-goog

[RFC PATCH v3 21/29] KVM: arm64: Introduce framework to trap disabled features

[Reiji Watanabe]

When a CPU feature that is supported on the host is not exposed to
its guest, emulating a real CPU's behavior (by trapping or disabling
guest's using the feature) is generally a desirable behavior (when
it's possible without any or little side effect).

Introduce feature_config_ctrl structure, which manages feature
information to program configuration register to trap or disable
the feature when the feature is not exposed to the guest, and
functions that uses the structure to activate trapping the feature.

At present, no feature has feature_config_ctrl yet and the following
patches will add the feature_config_ctrl for several features.

Signed-off-by: Reiji Watanabe <reijiw@google.com>
---
 arch/arm64/kvm/sys_regs.c | 121 +++++++++++++++++++++++++++++++++++++-
 1 file changed, 120 insertions(+), 1 deletion(-)

diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index 2f96103fc0d2..501de08dacb7 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -376,8 +376,38 @@ static int arm64_check_features(u64 check_types, u64 val, u64 lim)
 	(cpuid_feature_extract_unsigned_field(val, ID_AA64ISAR1_GPI_SHIFT) >= \
 	 ID_AA64ISAR1_GPI_IMP_DEF)
 
+enum vcpu_config_reg {
+	VCPU_HCR_EL2 = 1,
+	VCPU_MDCR_EL2,
+	VCPU_CPTR_EL2,
+};
+
+/*
+ * Feature information to program configuration register to trap or disable
+ * guest's using a feature when the feature is not exposed to the guest.
+ */
+struct feature_config_ctrl {
+	/* ID register/field for the feature */
+	u32	ftr_reg;	/* ID register */
+	bool	ftr_signed;	/* Is the feature field signed ? */
+	u8	ftr_shift;	/* Field of ID register for the feature */
+	s8	ftr_min;	/* Min value that indicate the feature */
+
+	/*
+	 * Function to check trapping is needed. This is used when the above
+	 * fields are not enough to determine if trapping is needed.
+	 */
+	bool	(*ftr_need_trap)(struct kvm_vcpu *vcpu);
+
+	/* Configuration register information to trap the feature. */
+	enum vcpu_config_reg cfg_reg;	/* Configuration register */
+	u64	cfg_mask;	/* Field of the configuration register */
+	u64	cfg_val;	/* Value that are set for the field */
+};
+
 struct id_reg_info {
 	u32	sys_reg;	/* Register ID */
+	u64	sys_val;	/* Sanitized system value */
 
 	/*
 	 * Limit value of the register for a vcpu. The value is the sanitized
@@ -410,11 +440,15 @@ struct id_reg_info {
 	/* Return the reset value of the register for the vCPU */
 	u64 (*get_reset_val)(struct kvm_vcpu *vcpu,
 			     const struct id_reg_info *id_reg);
+
+	/* Information to trap features that are disabled for the guest */
+	const struct feature_config_ctrl *(*trap_features)[];
 };
 
 static void id_reg_info_init(struct id_reg_info *id_reg)
 {
-	id_reg->vcpu_limit_val = read_sanitised_ftr_reg(id_reg->sys_reg);
+	id_reg->sys_val = read_sanitised_ftr_reg(id_reg->sys_reg);
+	id_reg->vcpu_limit_val = id_reg->sys_val;
 	if (id_reg->init)
 		id_reg->init(id_reg);
 }
@@ -952,6 +986,47 @@ static int validate_id_reg(struct kvm_vcpu *vcpu,
 	return err;
 }
 
+static void feature_trap_activate(struct kvm_vcpu *vcpu,
+				  const struct feature_config_ctrl *config)
+{
+	u64 *reg_ptr, reg_val;
+
+	switch (config->cfg_reg) {
+	case VCPU_HCR_EL2:
+		reg_ptr = &vcpu->arch.hcr_el2;
+		break;
+	case VCPU_MDCR_EL2:
+		reg_ptr = &vcpu->arch.mdcr_el2;
+		break;
+	case VCPU_CPTR_EL2:
+		reg_ptr = &vcpu->arch.cptr_el2;
+		break;
+	}
+
+	/* Update cfg_mask fields with cfg_val */
+	reg_val = (*reg_ptr & ~config->cfg_mask);
+	reg_val |= config->cfg_val;
+	*reg_ptr = reg_val;
+}
+
+static inline bool feature_avail(const struct feature_config_ctrl *ctrl,
+				 u64 id_val)
+{
+	int field_val = cpuid_feature_extract_field(id_val,
+				ctrl->ftr_shift, ctrl->ftr_signed);
+
+	return (field_val >= ctrl->ftr_min);
+}
+
+static inline bool vcpu_feature_is_available(struct kvm_vcpu *vcpu,
+					const struct feature_config_ctrl *ctrl)
+{
+	u64 val;
+
+	val = __read_id_reg(vcpu, ctrl->ftr_reg);
+	return feature_avail(ctrl, val);
+}
+
 /*
  * ARMv8.1 mandates at least a trivial LORegion implementation, where all the
  * RW registers are RES0 (which we can implement as RAZ/WI). On an ARMv8.0
@@ -1831,6 +1906,42 @@ static int reg_from_user(u64 *val, const void __user *uaddr, u64 id);
 static int reg_to_user(void __user *uaddr, const u64 *val, u64 id);
 static u64 sys_reg_to_index(const struct sys_reg_desc *reg);
 
+static void id_reg_features_trap_activate(struct kvm_vcpu *vcpu,
+					  const struct id_reg_info *id_reg)
+{
+	u64 val;
+	int i = 0;
+	const struct feature_config_ctrl **ctrlp_array, *ctrl;
+
+	if (!id_reg || !id_reg->trap_features)
+		/* No information to trap a feature */
+		return;
+
+	val = __read_id_reg(vcpu, id_reg->sys_reg);
+	if (val == id_reg->sys_val)
+		/* No feature needs to be trapped (no feature is disabled). */
+		return;
+
+	ctrlp_array = *id_reg->trap_features;
+	while ((ctrl = ctrlp_array[i++]) != NULL) {
+		if (ctrl->ftr_need_trap && ctrl->ftr_need_trap(vcpu)) {
+			feature_trap_activate(vcpu, ctrl);
+			continue;
+		}
+
+		if (!feature_avail(ctrl, id_reg->sys_val))
+			/* The feature is not supported on the host. */
+			continue;
+
+		if (feature_avail(ctrl, val))
+			/* The feature is enabled for the guest. */
+			continue;
+
+		/* The feature is supported but disabled. */
+		feature_trap_activate(vcpu, ctrl);
+	}
+}
+
 /* Visibility overrides for SVE-specific control registers */
 static unsigned int sve_visibility(const struct kvm_vcpu *vcpu,
 				   const struct sys_reg_desc *rd)
@@ -3457,6 +3568,14 @@ int kvm_arm_copy_sys_reg_indices(struct kvm_vcpu *vcpu, u64 __user *uindices)
 	return write_demux_regids(uindices);
 }
 
+void kvm_vcpu_init_traps(struct kvm_vcpu *vcpu)
+{
+	int i;
+
+	for (i = 0; i < ARRAY_SIZE(id_reg_info_table); i++)
+		id_reg_features_trap_activate(vcpu, id_reg_info_table[i]);
+}
+
 /* ID register's fractional field information with its feature field. */
 struct feature_frac {
 	u32	id;
-- 
2.34.0.rc1.387.gb447b232ab-goog

[RFC PATCH v3 22/29] KVM: arm64: Trap disabled features of ID_AA64PFR0_EL1

[Reiji Watanabe]

Add feature_config_ctrl for RAS and AMU, which are indicated in
ID_AA64PFR0_EL1, to program configuration registers to trap
guest's using those features when they are not exposed to the guest.

Introduce trap_ras_regs() to change a behavior of guest's access to
the registers, which is currently raz/wi, depending on the feature's
availability for the guest (and inject undefined instruction
exception when guest's RAS register access are trapped and RAS is
not exposed to the guest).  In order to keep the current visibility
of the RAS registers from userspace (always visible), a visibility
function for RAS registers is not added.

No code is added for AMU's access/visibility handler because the
current code already injects the exception for Guest's AMU register
access unconditionally because AMU is never exposed to the guest.

Signed-off-by: Reiji Watanabe <reijiw@google.com>
---
 arch/arm64/kvm/sys_regs.c | 54 +++++++++++++++++++++++++++++++++------
 1 file changed, 46 insertions(+), 8 deletions(-)

diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index 501de08dacb7..42db8cf18fbb 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -405,6 +405,27 @@ struct feature_config_ctrl {
 	u64	cfg_val;	/* Value that are set for the field */
 };
 
+/* For ID_AA64PFR0_EL1 */
+static struct feature_config_ctrl ftr_ctrl_ras = {
+	.ftr_reg = SYS_ID_AA64PFR0_EL1,
+	.ftr_shift = ID_AA64PFR0_RAS_SHIFT,
+	.ftr_min = ID_AA64PFR0_RAS_V1,
+	.ftr_signed = FTR_UNSIGNED,
+	.cfg_reg = VCPU_HCR_EL2,
+	.cfg_mask = (HCR_TERR | HCR_TEA | HCR_FIEN),
+	.cfg_val = (HCR_TERR | HCR_TEA),
+};
+
+static struct feature_config_ctrl ftr_ctrl_amu = {
+	.ftr_reg = SYS_ID_AA64PFR0_EL1,
+	.ftr_shift = ID_AA64PFR0_AMU_SHIFT,
+	.ftr_min = ID_AA64PFR0_AMU,
+	.ftr_signed = FTR_UNSIGNED,
+	.cfg_reg = VCPU_CPTR_EL2,
+	.cfg_mask = CPTR_EL2_TAM,
+	.cfg_val = CPTR_EL2_TAM,
+};
+
 struct id_reg_info {
 	u32	sys_reg;	/* Register ID */
 	u64	sys_val;	/* Sanitized system value */
@@ -871,6 +892,11 @@ static struct id_reg_info id_aa64pfr0_el1_info = {
 	.init = init_id_aa64pfr0_el1_info,
 	.validate = validate_id_aa64pfr0_el1,
 	.get_reset_val = get_reset_id_aa64pfr0_el1,
+	.trap_features = &(const struct feature_config_ctrl *[]) {
+		&ftr_ctrl_ras,
+		&ftr_ctrl_amu,
+		NULL,
+	},
 };
 
 static struct id_reg_info id_aa64pfr1_el1_info = {
@@ -1027,6 +1053,18 @@ static inline bool vcpu_feature_is_available(struct kvm_vcpu *vcpu,
 	return feature_avail(ctrl, val);
 }
 
+static bool trap_ras_regs(struct kvm_vcpu *vcpu,
+			  struct sys_reg_params *p,
+			  const struct sys_reg_desc *r)
+{
+	if (!vcpu_feature_is_available(vcpu, &ftr_ctrl_ras)) {
+		kvm_inject_undefined(vcpu);
+		return false;
+	}
+
+	return trap_raz_wi(vcpu, p, r);
+}
+
 /*
  * ARMv8.1 mandates at least a trivial LORegion implementation, where all the
  * RW registers are RES0 (which we can implement as RAZ/WI). On an ARMv8.0
@@ -2318,14 +2356,14 @@ static const struct sys_reg_desc sys_reg_descs[] = {
 	{ SYS_DESC(SYS_AFSR1_EL1), access_vm_reg, reset_unknown, AFSR1_EL1 },
 	{ SYS_DESC(SYS_ESR_EL1), access_vm_reg, reset_unknown, ESR_EL1 },
 
-	{ SYS_DESC(SYS_ERRIDR_EL1), trap_raz_wi },
-	{ SYS_DESC(SYS_ERRSELR_EL1), trap_raz_wi },
-	{ SYS_DESC(SYS_ERXFR_EL1), trap_raz_wi },
-	{ SYS_DESC(SYS_ERXCTLR_EL1), trap_raz_wi },
-	{ SYS_DESC(SYS_ERXSTATUS_EL1), trap_raz_wi },
-	{ SYS_DESC(SYS_ERXADDR_EL1), trap_raz_wi },
-	{ SYS_DESC(SYS_ERXMISC0_EL1), trap_raz_wi },
-	{ SYS_DESC(SYS_ERXMISC1_EL1), trap_raz_wi },
+	{ SYS_DESC(SYS_ERRIDR_EL1), trap_ras_regs },
+	{ SYS_DESC(SYS_ERRSELR_EL1), trap_ras_regs },
+	{ SYS_DESC(SYS_ERXFR_EL1), trap_ras_regs },
+	{ SYS_DESC(SYS_ERXCTLR_EL1), trap_ras_regs },
+	{ SYS_DESC(SYS_ERXSTATUS_EL1), trap_ras_regs },
+	{ SYS_DESC(SYS_ERXADDR_EL1), trap_ras_regs },
+	{ SYS_DESC(SYS_ERXMISC0_EL1), trap_ras_regs },
+	{ SYS_DESC(SYS_ERXMISC1_EL1), trap_ras_regs },
 
 	MTE_REG(TFSR_EL1),
 	MTE_REG(TFSRE0_EL1),
-- 
2.34.0.rc1.387.gb447b232ab-goog

[RFC PATCH v3 23/29] KVM: arm64: Trap disabled features of ID_AA64PFR1_EL1

[Reiji Watanabe]

Add feature_config_ctrl for MTE, which is indicated in
ID_AA64PFR1_EL1, to program configuration register to trap the
guest's using the feature when it is not exposed to the guest.

Signed-off-by: Reiji Watanabe <reijiw@google.com>
---
 arch/arm64/kvm/sys_regs.c | 15 +++++++++++++++
 1 file changed, 15 insertions(+)

diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index 42db8cf18fbb..73d992ba6e90 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -426,6 +426,17 @@ static struct feature_config_ctrl ftr_ctrl_amu = {
 	.cfg_val = CPTR_EL2_TAM,
 };
 
+/* For ID_AA64PFR1_EL1 */
+static struct feature_config_ctrl ftr_ctrl_mte = {
+	.ftr_reg = SYS_ID_AA64PFR1_EL1,
+	.ftr_shift = ID_AA64PFR1_MTE_SHIFT,
+	.ftr_min = ID_AA64PFR1_MTE_EL0,
+	.ftr_signed = FTR_UNSIGNED,
+	.cfg_reg = VCPU_HCR_EL2,
+	.cfg_mask = (HCR_TID5 | HCR_DCT | HCR_ATA),
+	.cfg_val = HCR_TID5,
+};
+
 struct id_reg_info {
 	u32	sys_reg;	/* Register ID */
 	u64	sys_val;	/* Sanitized system value */
@@ -904,6 +915,10 @@ static struct id_reg_info id_aa64pfr1_el1_info = {
 	.init = init_id_aa64pfr1_el1_info,
 	.validate = validate_id_aa64pfr1_el1,
 	.get_reset_val = get_reset_id_aa64pfr1_el1,
+	.trap_features = &(const struct feature_config_ctrl *[]) {
+		&ftr_ctrl_mte,
+		NULL,
+	},
 };
 
 static struct id_reg_info id_aa64isar0_el1_info = {
-- 
2.34.0.rc1.387.gb447b232ab-goog

[RFC PATCH v3 24/29] KVM: arm64: Trap disabled features of ID_AA64DFR0_EL1

[Reiji Watanabe]

Add feature_config_ctrl for PMUv3, PMS and TraceFilt, which are
indicated in ID_AA64DFR0_EL1, to program configuration registers
to trap guest's using those features when they are not exposed to
the guest.

Signed-off-by: Reiji Watanabe <reijiw@google.com>
---
 arch/arm64/kvm/sys_regs.c | 38 ++++++++++++++++++++++++++++++++++++++
 1 file changed, 38 insertions(+)

diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index 73d992ba6e90..cb18d1fe0658 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -437,6 +437,38 @@ static struct feature_config_ctrl ftr_ctrl_mte = {
 	.cfg_val = HCR_TID5,
 };
 
+/* For ID_AA64DFR0_EL1 */
+static struct feature_config_ctrl ftr_ctrl_pmuv3 = {
+	.ftr_reg = SYS_ID_AA64DFR0_EL1,
+	.ftr_shift = ID_AA64DFR0_PMUVER_SHIFT,
+	.ftr_min = ID_AA64DFR0_PMUVER_8_0,
+	.ftr_signed = FTR_UNSIGNED,
+	.cfg_reg = VCPU_MDCR_EL2,
+	.cfg_mask = MDCR_EL2_TPM,
+	.cfg_val = MDCR_EL2_TPM,
+};
+
+static struct feature_config_ctrl ftr_ctrl_pms = {
+	.ftr_reg = SYS_ID_AA64DFR0_EL1,
+	.ftr_shift = ID_AA64DFR0_PMSVER_SHIFT,
+	.ftr_min = ID_AA64DFR0_PMSVER_8_2,
+	.ftr_signed = FTR_UNSIGNED,
+	.cfg_reg = VCPU_MDCR_EL2,
+	.cfg_mask = (MDCR_EL2_TPMS |
+			(MDCR_EL2_E2PB_MASK << MDCR_EL2_E2PB_SHIFT)),
+	.cfg_val = MDCR_EL2_TPMS,
+};
+
+static struct feature_config_ctrl ftr_ctrl_tracefilt = {
+	.ftr_reg = SYS_ID_AA64DFR0_EL1,
+	.ftr_shift = ID_AA64DFR0_TRACE_FILT_SHIFT,
+	.ftr_min = 1,
+	.ftr_signed = FTR_UNSIGNED,
+	.cfg_reg = VCPU_MDCR_EL2,
+	.cfg_mask = MDCR_EL2_TTRF,
+	.cfg_val = MDCR_EL2_TTRF,
+};
+
 struct id_reg_info {
 	u32	sys_reg;	/* Register ID */
 	u64	sys_val;	/* Sanitized system value */
@@ -952,6 +984,12 @@ static struct id_reg_info id_aa64dfr0_el1_info = {
 	.init = init_id_aa64dfr0_el1_info,
 	.validate = validate_id_aa64dfr0_el1,
 	.get_reset_val = get_reset_id_aa64dfr0_el1,
+	.trap_features = &(const struct feature_config_ctrl *[]) {
+		&ftr_ctrl_pmuv3,
+		&ftr_ctrl_pms,
+		&ftr_ctrl_tracefilt,
+		NULL,
+	},
 };
 
 static struct id_reg_info id_dfr0_el1_info = {
-- 
2.34.0.rc1.387.gb447b232ab-goog

[RFC PATCH v3 25/29] KVM: arm64: Trap disabled features of ID_AA64MMFR1_EL1

[Reiji Watanabe]

Add feature_config_ctrl for LORegions, which is indicated in
ID_AA64MMFR1_EL1, to program configuration register to trap
guest's using the feature when it is not exposed to the guest.

Change trap_loregion() to use vcpu_feature_is_available()
to simplify checking of the feature's availability.

Signed-off-by: Reiji Watanabe <reijiw@google.com>
---
 arch/arm64/kvm/sys_regs.c | 23 +++++++++++++++++++++--
 1 file changed, 21 insertions(+), 2 deletions(-)

diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index cb18d1fe0658..3d3b29515b8b 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -469,6 +469,17 @@ static struct feature_config_ctrl ftr_ctrl_tracefilt = {
 	.cfg_val = MDCR_EL2_TTRF,
 };
 
+/* For ID_AA64MMFR1_EL1 */
+static struct feature_config_ctrl ftr_ctrl_lor = {
+	.ftr_reg = SYS_ID_AA64MMFR1_EL1,
+	.ftr_shift = ID_AA64MMFR1_LOR_SHIFT,
+	.ftr_min = 1,
+	.ftr_signed = FTR_UNSIGNED,
+	.cfg_reg = VCPU_HCR_EL2,
+	.cfg_mask = HCR_TLOR,
+	.cfg_val = HCR_TLOR,
+};
+
 struct id_reg_info {
 	u32	sys_reg;	/* Register ID */
 	u64	sys_val;	/* Sanitized system value */
@@ -992,6 +1003,14 @@ static struct id_reg_info id_aa64dfr0_el1_info = {
 	},
 };
 
+static struct id_reg_info id_aa64mmfr1_el1_info = {
+	.sys_reg = SYS_ID_AA64MMFR1_EL1,
+	.trap_features = &(const struct feature_config_ctrl *[]) {
+		&ftr_ctrl_lor,
+		NULL,
+	},
+};
+
 static struct id_reg_info id_dfr0_el1_info = {
 	.sys_reg = SYS_ID_DFR0_EL1,
 	.init = init_id_dfr0_el1_info,
@@ -1034,6 +1053,7 @@ static struct id_reg_info *id_reg_info_table[KVM_ARM_ID_REG_MAX_NUM] = {
 	[IDREG_IDX(SYS_ID_AA64ISAR0_EL1)] = &id_aa64isar0_el1_info,
 	[IDREG_IDX(SYS_ID_AA64ISAR1_EL1)] = &id_aa64isar1_el1_info,
 	[IDREG_IDX(SYS_ID_AA64MMFR0_EL1)] = &id_aa64mmfr0_el1_info,
+	[IDREG_IDX(SYS_ID_AA64MMFR1_EL1)] = &id_aa64mmfr1_el1_info,
 };
 
 static int validate_id_reg(struct kvm_vcpu *vcpu,
@@ -1128,10 +1148,9 @@ static bool trap_loregion(struct kvm_vcpu *vcpu,
 			  struct sys_reg_params *p,
 			  const struct sys_reg_desc *r)
 {
-	u64 val = __read_id_reg(vcpu, SYS_ID_AA64MMFR1_EL1);
 	u32 sr = reg_to_encoding(r);
 
-	if (!(val & (0xfUL << ID_AA64MMFR1_LOR_SHIFT))) {
+	if (!vcpu_feature_is_available(vcpu, &ftr_ctrl_lor)) {
 		kvm_inject_undefined(vcpu);
 		return false;
 	}
-- 
2.34.0.rc1.387.gb447b232ab-goog

[RFC PATCH v3 26/29] KVM: arm64: Trap disabled features of ID_AA64ISAR1_EL1

[Reiji Watanabe]

Add feature_config_ctrl for PTRAUTH, which is indicated in
ID_AA64ISAR1_EL1, to program configuration register to trap
guest's using the feature when it is not exposed to the guest.

Signed-off-by: Reiji Watanabe <reijiw@google.com>
---
 arch/arm64/kvm/sys_regs.c | 36 ++++++++++++++++++++++++++++++++++++
 1 file changed, 36 insertions(+)

diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index 3d3b29515b8b..f1f975ce7b07 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -376,6 +376,30 @@ static int arm64_check_features(u64 check_types, u64 val, u64 lim)
 	(cpuid_feature_extract_unsigned_field(val, ID_AA64ISAR1_GPI_SHIFT) >= \
 	 ID_AA64ISAR1_GPI_IMP_DEF)
 
+/*
+ * Return true if ptrauth needs to be trapped.
+ * (i.e. if ptrauth is supported on the host but not exposed to the guest)
+ */
+static bool vcpu_need_trap_ptrauth(struct kvm_vcpu *vcpu)
+{
+	u64 val;
+	bool generic, address;
+
+	if (!system_has_full_ptr_auth())
+		/* The feature is not supported. */
+		return false;
+
+	val = __read_id_reg(vcpu, SYS_ID_AA64ISAR1_EL1);
+	generic = aa64isar1_has_gpi(val) || aa64isar1_has_gpa(val);
+	address = aa64isar1_has_api(val) || aa64isar1_has_apa(val);
+	if (generic && address)
+		/* The feature is available. */
+		return false;
+
+	/* The feature is supported but hidden. */
+	return true;
+}
+
 enum vcpu_config_reg {
 	VCPU_HCR_EL2 = 1,
 	VCPU_MDCR_EL2,
@@ -480,6 +504,14 @@ static struct feature_config_ctrl ftr_ctrl_lor = {
 	.cfg_val = HCR_TLOR,
 };
 
+/* For SYS_ID_AA64ISAR1_EL1 */
+static struct feature_config_ctrl ftr_ctrl_ptrauth = {
+	.ftr_need_trap = vcpu_need_trap_ptrauth,
+	.cfg_reg = VCPU_HCR_EL2,
+	.cfg_mask = (HCR_API | HCR_APK),
+	.cfg_val = 0,
+};
+
 struct id_reg_info {
 	u32	sys_reg;	/* Register ID */
 	u64	sys_val;	/* Sanitized system value */
@@ -977,6 +1009,10 @@ static struct id_reg_info id_aa64isar1_el1_info = {
 	.init = init_id_aa64isar1_el1_info,
 	.validate = validate_id_aa64isar1_el1,
 	.get_reset_val = get_reset_id_aa64isar1_el1,
+	.trap_features = &(const struct feature_config_ctrl *[]) {
+		&ftr_ctrl_ptrauth,
+		NULL,
+	},
 };
 
 static struct id_reg_info id_aa64mmfr0_el1_info = {
-- 
2.34.0.rc1.387.gb447b232ab-goog

[RFC PATCH v3 27/29] KVM: arm64: Initialize trapping of disabled CPU features for the guest

[Reiji Watanabe]

Call kvm_vcpu_init_traps() at the first KVM_RUN to initialize trapping
of disabled CPU features for the guest.

Signed-off-by: Reiji Watanabe <reijiw@google.com>
---
 arch/arm64/include/asm/kvm_host.h | 1 +
 arch/arm64/kvm/arm.c              | 2 ++
 2 files changed, 3 insertions(+)

diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index 9dc9970a2d46..a53949cd53c6 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -747,6 +747,7 @@ long kvm_vm_ioctl_mte_copy_tags(struct kvm *kvm,
 				struct kvm_arm_copy_mte_tags *copy_tags);
 
 int kvm_id_regs_consistency_check(const struct kvm_vcpu *vcpu);
+void kvm_vcpu_init_traps(struct kvm_vcpu *vcpu);
 
 /* Guest/host FPSIMD coordination helpers */
 int kvm_arch_vcpu_run_map_fp(struct kvm_vcpu *vcpu);
diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
index 64b104ebee73..25a41ff92aa5 100644
--- a/arch/arm64/kvm/arm.c
+++ b/arch/arm64/kvm/arm.c
@@ -625,6 +625,8 @@ static int kvm_vcpu_first_run_init(struct kvm_vcpu *vcpu)
 	 */
 	if (kvm_vm_is_protected(kvm))
 		kvm_call_hyp_nvhe(__pkvm_vcpu_init_traps, vcpu);
+	else
+		kvm_vcpu_init_traps(vcpu);
 
 	return ret;
 }
-- 
2.34.0.rc1.387.gb447b232ab-goog

[RFC PATCH v3 28/29] KVM: arm64: Add kunit test for trap initialization

[Reiji Watanabe]

Add KUnit tests for functions that initialize traps.

Signed-off-by: Reiji Watanabe <reijiw@google.com>
---
 arch/arm64/kvm/sys_regs_test.c | 238 +++++++++++++++++++++++++++++++++
 1 file changed, 238 insertions(+)

diff --git a/arch/arm64/kvm/sys_regs_test.c b/arch/arm64/kvm/sys_regs_test.c
index 8d27c7c361fb..f73b207be4ee 100644
--- a/arch/arm64/kvm/sys_regs_test.c
+++ b/arch/arm64/kvm/sys_regs_test.c
@@ -844,6 +844,241 @@ static void validate_mvfr1_el1_test(struct kunit *test)
 	test_kvm_vcpu_fini(test, vcpu);
 }
 
+static void feature_trap_activate_test(struct kunit *test)
+{
+	struct kvm_vcpu *vcpu;
+	struct feature_config_ctrl config_data, *config = &config_data;
+	u64 cfg_mask, cfg_val;
+
+	vcpu = test_kvm_vcpu_init(test);
+	KUNIT_EXPECT_TRUE(test, vcpu);
+	if (!vcpu)
+		return;
+
+	vcpu->arch.hcr_el2 = 0;
+	config->ftr_reg = SYS_ID_AA64MMFR1_EL1;
+	config->ftr_shift = 4;
+	config->ftr_min = 2;
+	config->ftr_signed = FTR_UNSIGNED;
+
+	/* Test for hcr_el2 */
+	config->cfg_reg = VCPU_HCR_EL2;
+	cfg_mask = 0x30000800000;
+	cfg_val = 0x30000800000;
+	config->cfg_mask = cfg_mask;
+	config->cfg_val = cfg_val;
+
+	vcpu->arch.hcr_el2 = 0;
+	feature_trap_activate(vcpu, config);
+	KUNIT_EXPECT_EQ(test, vcpu->arch.hcr_el2 & cfg_mask, cfg_val);
+
+	cfg_mask = 0x30000800000;
+	cfg_val = 0;
+	config->cfg_mask = cfg_mask;
+	config->cfg_val = cfg_val;
+
+	vcpu->arch.hcr_el2 = 0;
+	feature_trap_activate(vcpu, config);
+	KUNIT_EXPECT_EQ(test, vcpu->arch.hcr_el2 & cfg_mask, cfg_val);
+
+	/* Test for mdcr_el2 */
+	config->cfg_reg = VCPU_MDCR_EL2;
+	cfg_mask = 0x30000800000;
+	cfg_val = 0x30000800000;
+	config->cfg_mask = cfg_mask;
+	config->cfg_val = cfg_val;
+
+	vcpu->arch.mdcr_el2 = 0;
+	feature_trap_activate(vcpu, config);
+	KUNIT_EXPECT_EQ(test, vcpu->arch.mdcr_el2 & cfg_mask, cfg_val);
+
+	cfg_mask = 0x30000800000;
+	cfg_val = 0x0;
+	config->cfg_mask = cfg_mask;
+	config->cfg_val = cfg_val;
+
+	vcpu->arch.mdcr_el2 = 0;
+	feature_trap_activate(vcpu, config);
+	KUNIT_EXPECT_EQ(test, vcpu->arch.mdcr_el2 & cfg_mask, cfg_val);
+
+	/* Test for cptr_el2 */
+	config->cfg_reg = VCPU_CPTR_EL2;
+	cfg_mask = 0x30000800000;
+	cfg_val = 0x30000800000;
+	config->cfg_mask = cfg_mask;
+	config->cfg_val = cfg_val;
+
+	vcpu->arch.cptr_el2 = 0;
+	feature_trap_activate(vcpu, config);
+	KUNIT_EXPECT_EQ(test, vcpu->arch.cptr_el2 & cfg_mask, cfg_val);
+
+	cfg_mask = 0x30000800000;
+	cfg_val = 0x0;
+	config->cfg_mask = cfg_mask;
+	config->cfg_val = cfg_val;
+
+	vcpu->arch.cptr_el2 = 0;
+	feature_trap_activate(vcpu, config);
+	KUNIT_EXPECT_EQ(test, vcpu->arch.cptr_el2 & cfg_mask, cfg_val);
+
+	test_kvm_vcpu_fini(test, vcpu);
+}
+
+static bool test_need_trap_aa64dfr0(struct kvm_vcpu *vcpu)
+{
+	u64 val;
+
+	val = __vcpu_sys_reg(vcpu, IDREG_SYS_IDX(SYS_ID_AA64DFR0_EL1));
+	return ((val & 0xf) == 0);
+}
+
+static void id_reg_features_trap_activate_test(struct kunit *test)
+{
+	struct kvm_vcpu *vcpu;
+	u32 id;
+	u64 cfg_mask0, cfg_val0, cfg_mask1, cfg_val1, cfg_mask2, cfg_val2;
+	u64 cfg_mask, cfg_val, id_reg_sys_val;
+	struct id_reg_info id_reg_data;
+	struct feature_config_ctrl *config, config0, config1, config2;
+	struct feature_config_ctrl *trap_features[] = {
+		&config0, &config1, &config2, NULL,
+	};
+
+	vcpu = test_kvm_vcpu_init(test);
+	KUNIT_EXPECT_TRUE(test, vcpu);
+	if (!vcpu)
+		return;
+
+	id_reg_sys_val = 0x7777777777777777;
+	id = SYS_ID_AA64DFR0_EL1;
+	id_reg_data.sys_reg = id;
+	id_reg_data.sys_val = id_reg_sys_val;
+	id_reg_data.vcpu_limit_val  = (u64)-1;
+	id_reg_data.trap_features =
+			(const struct feature_config_ctrl *(*)[])trap_features;
+
+	cfg_mask0 = 0x3;
+	cfg_val0 = 0x3;
+	config = &config0;
+	memset(config, 0, sizeof(*config));
+	config->ftr_reg = id;
+	config->ftr_shift = 60;
+	config->ftr_min = 2;
+	config->ftr_signed = FTR_UNSIGNED;
+	config->cfg_reg = VCPU_HCR_EL2;
+	config->cfg_mask = cfg_mask0;
+	config->cfg_val = cfg_val0;
+
+	cfg_mask1 = 0x70000040;
+	cfg_val1 = 0x30000040;
+	config = &config1;
+	memset(config, 0, sizeof(*config));
+	config->ftr_reg = id;
+	config->ftr_need_trap = test_need_trap_aa64dfr0;
+	config->ftr_signed = FTR_UNSIGNED;
+	config->cfg_reg = VCPU_HCR_EL2;
+	config->cfg_mask = cfg_mask1;
+	config->cfg_val = cfg_val1;
+
+	/* Feature with signed ID register field */
+	cfg_mask2 = 0x70000000800;
+	cfg_val2 = 0x30000000800;
+	config = &config2;
+	memset(config, 0, sizeof(*config));
+	config->ftr_reg = id;
+	config->ftr_shift = 4;
+	config->ftr_min = 0;
+	config->ftr_signed = FTR_SIGNED;
+	config->cfg_reg = VCPU_HCR_EL2;
+	config->cfg_mask = cfg_mask2;
+	config->cfg_val = cfg_val2;
+
+	/* Enable features for config0, 1 and 2 */
+	__vcpu_sys_reg(vcpu, IDREG_SYS_IDX(id)) = id_reg_sys_val;
+
+	vcpu->arch.hcr_el2 = 0;
+	id_reg_features_trap_activate(vcpu, &id_reg_data);
+	KUNIT_EXPECT_EQ(test, vcpu->arch.hcr_el2, 0);
+
+	/* Disable features for config0 only */
+	__vcpu_sys_reg(vcpu, IDREG_SYS_IDX(id)) = 0x1;
+	cfg_mask = cfg_mask0;
+	cfg_val = cfg_val0;
+
+	vcpu->arch.hcr_el2 = 0;
+	id_reg_features_trap_activate(vcpu, &id_reg_data);
+	KUNIT_EXPECT_EQ(test, vcpu->arch.hcr_el2 & cfg_mask, cfg_val);
+
+	/* Disable features for config0 and config1 */
+	__vcpu_sys_reg(vcpu, IDREG_SYS_IDX(id)) = 0x0;
+	cfg_mask = (cfg_mask0 | cfg_mask1);
+	cfg_val = (cfg_val0 | cfg_val1);
+
+	vcpu->arch.hcr_el2 = 0;
+	id_reg_features_trap_activate(vcpu, &id_reg_data);
+	KUNIT_EXPECT_EQ(test, vcpu->arch.hcr_el2 & cfg_mask, cfg_val);
+
+	/* Disable features for config0, 1, and 2 */
+	__vcpu_sys_reg(vcpu, IDREG_SYS_IDX(id)) = 0xf0;
+	cfg_mask = (cfg_mask0 | cfg_mask1 | cfg_mask2);
+	cfg_val = (cfg_val0 | cfg_val1 | cfg_val2);
+
+	vcpu->arch.hcr_el2 = 0;
+	id_reg_features_trap_activate(vcpu, &id_reg_data);
+	KUNIT_EXPECT_EQ(test, vcpu->arch.hcr_el2 & cfg_mask, cfg_val);
+
+	/* Test with id_reg_info == NULL */
+	vcpu->arch.hcr_el2 = 0;
+	id_reg_features_trap_activate(vcpu, NULL);
+	KUNIT_EXPECT_EQ(test, vcpu->arch.hcr_el2, 0);
+
+	/* Test with id_reg_data.trap_features = NULL */
+	id_reg_data.trap_features = NULL;
+	__vcpu_sys_reg(vcpu, IDREG_SYS_IDX(id)) = 0xf0;
+
+	vcpu->arch.hcr_el2 = 0;
+	id_reg_features_trap_activate(vcpu, &id_reg_data);
+	KUNIT_EXPECT_EQ(test, vcpu->arch.hcr_el2, 0);
+
+	test_kvm_vcpu_fini(test, vcpu);
+}
+
+static void vcpu_need_trap_ptrauth_test(struct kunit *test)
+{
+	struct kvm_vcpu *vcpu;
+	u32 id = SYS_ID_AA64ISAR1_EL1;
+
+	vcpu = test_kvm_vcpu_init(test);
+	KUNIT_EXPECT_TRUE(test, vcpu);
+	if (!vcpu)
+		return;
+
+	if (system_has_full_ptr_auth()) {
+		__vcpu_sys_reg(vcpu, IDREG_SYS_IDX(id)) = 0x0;
+		KUNIT_EXPECT_TRUE(test, vcpu_need_trap_ptrauth(vcpu));
+
+		/* GPI = 1, API = 1 */
+		__vcpu_sys_reg(vcpu, IDREG_SYS_IDX(id)) = 0x10000100;
+		KUNIT_EXPECT_FALSE(test, vcpu_need_trap_ptrauth(vcpu));
+
+		/* GPI = 1, APA = 1 */
+		__vcpu_sys_reg(vcpu, IDREG_SYS_IDX(id)) = 0x10000010;
+		KUNIT_EXPECT_FALSE(test, vcpu_need_trap_ptrauth(vcpu));
+
+		/* GPA = 1, API = 1 */
+		__vcpu_sys_reg(vcpu, IDREG_SYS_IDX(id)) = 0x01000100;
+		KUNIT_EXPECT_FALSE(test, vcpu_need_trap_ptrauth(vcpu));
+
+		/* GPA = 1, APA = 1 */
+		__vcpu_sys_reg(vcpu, IDREG_SYS_IDX(id)) = 0x01000010;
+		KUNIT_EXPECT_FALSE(test, vcpu_need_trap_ptrauth(vcpu));
+	} else {
+		KUNIT_EXPECT_FALSE(test, vcpu_need_trap_ptrauth(vcpu));
+	}
+
+	test_kvm_vcpu_fini(test, vcpu);
+}
+
 static struct kunit_case kvm_sys_regs_test_cases[] = {
 	KUNIT_CASE_PARAM(arm64_check_feature_one_test, feature_one_gen_params),
 	KUNIT_CASE_PARAM(arm64_check_features_test, features_gen_params),
@@ -859,6 +1094,9 @@ static struct kunit_case kvm_sys_regs_test_cases[] = {
 	KUNIT_CASE(validate_id_aa64dfr0_el1_test),
 	KUNIT_CASE(validate_id_dfr0_el1_test),
 	KUNIT_CASE(validate_mvfr1_el1_test),
+	KUNIT_CASE(vcpu_need_trap_ptrauth_test),
+	KUNIT_CASE(feature_trap_activate_test),
+	KUNIT_CASE(id_reg_features_trap_activate_test),
 	{}
 };
 
-- 
2.34.0.rc1.387.gb447b232ab-goog

[RFC PATCH v3 29/29] KVM: arm64: selftests: Introduce id_reg_test

[Reiji Watanabe]

Introduce a test for aarch64 to validate basic behavior of
KVM_GET_ONE_REG and KVM_SET_ONE_REG for ID registers.

This test runs only when KVM_CAP_ARM_ID_REG_CONFIGURABLE is supported.

Signed-off-by: Reiji Watanabe <reijiw@google.com>
---
 tools/arch/arm64/include/asm/sysreg.h         |    1 +
 tools/testing/selftests/kvm/.gitignore        |    1 +
 tools/testing/selftests/kvm/Makefile          |    1 +
 .../selftests/kvm/aarch64/id_reg_test.c       | 1128 +++++++++++++++++
 4 files changed, 1131 insertions(+)
 create mode 100644 tools/testing/selftests/kvm/aarch64/id_reg_test.c

diff --git a/tools/arch/arm64/include/asm/sysreg.h b/tools/arch/arm64/include/asm/sysreg.h
index 7640fa27be94..be3947c125f1 100644
--- a/tools/arch/arm64/include/asm/sysreg.h
+++ b/tools/arch/arm64/include/asm/sysreg.h
@@ -793,6 +793,7 @@
 #define ID_AA64PFR0_ELx_32BIT_64BIT	0x2
 
 /* id_aa64pfr1 */
+#define ID_AA64PFR1_CSV2FRAC_SHIFT	32
 #define ID_AA64PFR1_MPAMFRAC_SHIFT	16
 #define ID_AA64PFR1_RASFRAC_SHIFT	12
 #define ID_AA64PFR1_MTE_SHIFT		8
diff --git a/tools/testing/selftests/kvm/.gitignore b/tools/testing/selftests/kvm/.gitignore
index d4a830139683..5daf1400f0cf 100644
--- a/tools/testing/selftests/kvm/.gitignore
+++ b/tools/testing/selftests/kvm/.gitignore
@@ -2,6 +2,7 @@
 /aarch64/arch_timer
 /aarch64/debug-exceptions
 /aarch64/get-reg-list
+/aarch64/id_reg_test
 /aarch64/psci_cpu_on_test
 /aarch64/vgic_init
 /s390x/memop
diff --git a/tools/testing/selftests/kvm/Makefile b/tools/testing/selftests/kvm/Makefile
index c4e34717826a..fee6ba13019c 100644
--- a/tools/testing/selftests/kvm/Makefile
+++ b/tools/testing/selftests/kvm/Makefile
@@ -92,6 +92,7 @@ TEST_GEN_PROGS_x86_64 += system_counter_offset_test
 TEST_GEN_PROGS_aarch64 += aarch64/arch_timer
 TEST_GEN_PROGS_aarch64 += aarch64/debug-exceptions
 TEST_GEN_PROGS_aarch64 += aarch64/get-reg-list
+TEST_GEN_PROGS_aarch64 += aarch64/id_reg_test
 TEST_GEN_PROGS_aarch64 += aarch64/psci_cpu_on_test
 TEST_GEN_PROGS_aarch64 += aarch64/vgic_init
 TEST_GEN_PROGS_aarch64 += demand_paging_test
diff --git a/tools/testing/selftests/kvm/aarch64/id_reg_test.c b/tools/testing/selftests/kvm/aarch64/id_reg_test.c
new file mode 100644
index 000000000000..50d60d120e2e
--- /dev/null
+++ b/tools/testing/selftests/kvm/aarch64/id_reg_test.c
@@ -0,0 +1,1128 @@
+// SPDX-License-Identifier: GPL-2.0-only
+
+#define _GNU_SOURCE
+
+#include <stdlib.h>
+#include <time.h>
+#include <pthread.h>
+#include <linux/kvm.h>
+#include <linux/sizes.h>
+
+#include "kvm_util.h"
+#include "processor.h"
+#include "vgic.h"
+
+/*
+ * id_reg_test.c - Tests reading/writing the aarch64's ID registers
+ *
+ * The test validates KVM_SET_ONE_REG/KVM_GET_ONE_REG ioctl for ID
+ * registers as well as reading ID register from the guest works fine.
+ */
+
+/* Reserved ID registers */
+#define	SYS_ID_REG_3_3_EL1		sys_reg(3, 0, 0, 3, 3)
+#define	SYS_ID_REG_3_7_EL1		sys_reg(3, 0, 0, 3, 7)
+
+#define	SYS_ID_REG_4_2_EL1		sys_reg(3, 0, 0, 4, 2)
+#define	SYS_ID_REG_4_3_EL1		sys_reg(3, 0, 0, 4, 3)
+#define	SYS_ID_REG_4_5_EL1		sys_reg(3, 0, 0, 4, 5)
+#define	SYS_ID_REG_4_6_EL1		sys_reg(3, 0, 0, 4, 6)
+#define	SYS_ID_REG_4_7_EL1		sys_reg(3, 0, 0, 4, 7)
+
+#define	SYS_ID_REG_5_2_EL1		sys_reg(3, 0, 0, 5, 2)
+#define	SYS_ID_REG_5_3_EL1		sys_reg(3, 0, 0, 5, 3)
+#define	SYS_ID_REG_5_6_EL1		sys_reg(3, 0, 0, 5, 6)
+#define	SYS_ID_REG_5_7_EL1		sys_reg(3, 0, 0, 5, 7)
+
+#define	SYS_ID_REG_6_2_EL1		sys_reg(3, 0, 0, 6, 2)
+#define	SYS_ID_REG_6_3_EL1		sys_reg(3, 0, 0, 6, 3)
+#define	SYS_ID_REG_6_4_EL1		sys_reg(3, 0, 0, 6, 4)
+#define	SYS_ID_REG_6_5_EL1		sys_reg(3, 0, 0, 6, 5)
+#define	SYS_ID_REG_6_6_EL1		sys_reg(3, 0, 0, 6, 6)
+#define	SYS_ID_REG_6_7_EL1		sys_reg(3, 0, 0, 6, 7)
+
+#define	SYS_ID_REG_7_3_EL1		sys_reg(3, 0, 0, 7, 3)
+#define	SYS_ID_REG_7_4_EL1		sys_reg(3, 0, 0, 7, 4)
+#define	SYS_ID_REG_7_5_EL1		sys_reg(3, 0, 0, 7, 5)
+#define	SYS_ID_REG_7_6_EL1		sys_reg(3, 0, 0, 7, 6)
+#define	SYS_ID_REG_7_7_EL1		sys_reg(3, 0, 0, 7, 7)
+
+#define	READ_ID_REG_FN(name)	read_## name ## _EL1
+
+#define	DEFINE_READ_SYS_REG(reg_name)			\
+uint64_t read_##reg_name(void)				\
+{							\
+	return read_sysreg_s(SYS_##reg_name);		\
+}
+
+#define DEFINE_READ_ID_REG(name)	\
+	DEFINE_READ_SYS_REG(name ## _EL1)
+
+#define	__ID_REG(reg_name)		\
+	.name = #reg_name,		\
+	.id = SYS_## reg_name ##_EL1,	\
+	.read_reg = READ_ID_REG_FN(reg_name),
+
+#define	ID_REG_ENT(reg_name)	\
+	[ID_IDX(reg_name)] = { __ID_REG(reg_name) }
+
+/* Functions to read each ID register */
+/* CRm=1 */
+DEFINE_READ_ID_REG(ID_PFR0)
+DEFINE_READ_ID_REG(ID_PFR1)
+DEFINE_READ_ID_REG(ID_DFR0)
+DEFINE_READ_ID_REG(ID_AFR0)
+DEFINE_READ_ID_REG(ID_MMFR0)
+DEFINE_READ_ID_REG(ID_MMFR1)
+DEFINE_READ_ID_REG(ID_MMFR2)
+DEFINE_READ_ID_REG(ID_MMFR3)
+
+/* CRm=2 */
+DEFINE_READ_ID_REG(ID_ISAR0)
+DEFINE_READ_ID_REG(ID_ISAR1)
+DEFINE_READ_ID_REG(ID_ISAR2)
+DEFINE_READ_ID_REG(ID_ISAR3)
+DEFINE_READ_ID_REG(ID_ISAR4)
+DEFINE_READ_ID_REG(ID_ISAR5)
+DEFINE_READ_ID_REG(ID_MMFR4)
+DEFINE_READ_ID_REG(ID_ISAR6)
+
+/* CRm=3 */
+DEFINE_READ_ID_REG(MVFR0)
+DEFINE_READ_ID_REG(MVFR1)
+DEFINE_READ_ID_REG(MVFR2)
+DEFINE_READ_ID_REG(ID_REG_3_3)
+DEFINE_READ_ID_REG(ID_PFR2)
+DEFINE_READ_ID_REG(ID_DFR1)
+DEFINE_READ_ID_REG(ID_MMFR5)
+DEFINE_READ_ID_REG(ID_REG_3_7)
+
+/* CRm=4 */
+DEFINE_READ_ID_REG(ID_AA64PFR0)
+DEFINE_READ_ID_REG(ID_AA64PFR1)
+DEFINE_READ_ID_REG(ID_REG_4_2)
+DEFINE_READ_ID_REG(ID_REG_4_3)
+DEFINE_READ_ID_REG(ID_AA64ZFR0)
+DEFINE_READ_ID_REG(ID_REG_4_5)
+DEFINE_READ_ID_REG(ID_REG_4_6)
+DEFINE_READ_ID_REG(ID_REG_4_7)
+
+/* CRm=5 */
+DEFINE_READ_ID_REG(ID_AA64DFR0)
+DEFINE_READ_ID_REG(ID_AA64DFR1)
+DEFINE_READ_ID_REG(ID_REG_5_2)
+DEFINE_READ_ID_REG(ID_REG_5_3)
+DEFINE_READ_ID_REG(ID_AA64AFR0)
+DEFINE_READ_ID_REG(ID_AA64AFR1)
+DEFINE_READ_ID_REG(ID_REG_5_6)
+DEFINE_READ_ID_REG(ID_REG_5_7)
+
+/* CRm=6 */
+DEFINE_READ_ID_REG(ID_AA64ISAR0)
+DEFINE_READ_ID_REG(ID_AA64ISAR1)
+DEFINE_READ_ID_REG(ID_REG_6_2)
+DEFINE_READ_ID_REG(ID_REG_6_3)
+DEFINE_READ_ID_REG(ID_REG_6_4)
+DEFINE_READ_ID_REG(ID_REG_6_5)
+DEFINE_READ_ID_REG(ID_REG_6_6)
+DEFINE_READ_ID_REG(ID_REG_6_7)
+
+/* CRm=7 */
+DEFINE_READ_ID_REG(ID_AA64MMFR0)
+DEFINE_READ_ID_REG(ID_AA64MMFR1)
+DEFINE_READ_ID_REG(ID_AA64MMFR2)
+DEFINE_READ_ID_REG(ID_REG_7_3)
+DEFINE_READ_ID_REG(ID_REG_7_4)
+DEFINE_READ_ID_REG(ID_REG_7_5)
+DEFINE_READ_ID_REG(ID_REG_7_6)
+DEFINE_READ_ID_REG(ID_REG_7_7)
+
+#define	ID_IDX(name)	REG_IDX_## name
+
+enum id_reg_idx {
+	/* CRm=1 */
+	ID_IDX(ID_PFR0) = 0,
+	ID_IDX(ID_PFR1),
+	ID_IDX(ID_DFR0),
+	ID_IDX(ID_AFR0),
+	ID_IDX(ID_MMFR0),
+	ID_IDX(ID_MMFR1),
+	ID_IDX(ID_MMFR2),
+	ID_IDX(ID_MMFR3),
+
+	/* CRm=2 */
+	ID_IDX(ID_ISAR0),
+	ID_IDX(ID_ISAR1),
+	ID_IDX(ID_ISAR2),
+	ID_IDX(ID_ISAR3),
+	ID_IDX(ID_ISAR4),
+	ID_IDX(ID_ISAR5),
+	ID_IDX(ID_MMFR4),
+	ID_IDX(ID_ISAR6),
+
+	/* CRm=3 */
+	ID_IDX(MVFR0),
+	ID_IDX(MVFR1),
+	ID_IDX(MVFR2),
+	ID_IDX(ID_REG_3_3),
+	ID_IDX(ID_PFR2),
+	ID_IDX(ID_DFR1),
+	ID_IDX(ID_MMFR5),
+	ID_IDX(ID_REG_3_7),
+
+	/* CRm=4 */
+	ID_IDX(ID_AA64PFR0),
+	ID_IDX(ID_AA64PFR1),
+	ID_IDX(ID_REG_4_2),
+	ID_IDX(ID_REG_4_3),
+	ID_IDX(ID_AA64ZFR0),
+	ID_IDX(ID_REG_4_5),
+	ID_IDX(ID_REG_4_6),
+	ID_IDX(ID_REG_4_7),
+
+	/* CRm=5 */
+	ID_IDX(ID_AA64DFR0),
+	ID_IDX(ID_AA64DFR1),
+	ID_IDX(ID_REG_5_2),
+	ID_IDX(ID_REG_5_3),
+	ID_IDX(ID_AA64AFR0),
+	ID_IDX(ID_AA64AFR1),
+	ID_IDX(ID_REG_5_6),
+	ID_IDX(ID_REG_5_7),
+
+	/* CRm=6 */
+	ID_IDX(ID_AA64ISAR0),
+	ID_IDX(ID_AA64ISAR1),
+	ID_IDX(ID_REG_6_2),
+	ID_IDX(ID_REG_6_3),
+	ID_IDX(ID_REG_6_4),
+	ID_IDX(ID_REG_6_5),
+	ID_IDX(ID_REG_6_6),
+	ID_IDX(ID_REG_6_7),
+
+	/* CRm=7 */
+	ID_IDX(ID_AA64MMFR0),
+	ID_IDX(ID_AA64MMFR1),
+	ID_IDX(ID_AA64MMFR2),
+	ID_IDX(ID_REG_7_3),
+	ID_IDX(ID_REG_7_4),
+	ID_IDX(ID_REG_7_5),
+	ID_IDX(ID_REG_7_6),
+	ID_IDX(ID_REG_7_7),
+};
+
+struct id_reg_test_info {
+	char		*name;
+	uint32_t	id;
+	bool		can_clear;
+	uint64_t	fixed_mask;
+	uint64_t	org_val;
+	uint64_t	user_val;
+	uint64_t	(*read_reg)(void);
+};
+
+#define	ID_REG_INFO(name)	(&id_reg_list[ID_IDX(name)])
+static struct id_reg_test_info id_reg_list[] = {
+	/* CRm=1 */
+	ID_REG_ENT(ID_PFR0),
+	ID_REG_ENT(ID_PFR1),
+	ID_REG_ENT(ID_DFR0),
+	ID_REG_ENT(ID_AFR0),
+	ID_REG_ENT(ID_MMFR0),
+	ID_REG_ENT(ID_MMFR1),
+	ID_REG_ENT(ID_MMFR2),
+	ID_REG_ENT(ID_MMFR3),
+
+	/* CRm=2 */
+	ID_REG_ENT(ID_ISAR0),
+	ID_REG_ENT(ID_ISAR1),
+	ID_REG_ENT(ID_ISAR2),
+	ID_REG_ENT(ID_ISAR3),
+	ID_REG_ENT(ID_ISAR4),
+	ID_REG_ENT(ID_ISAR5),
+	ID_REG_ENT(ID_MMFR4),
+	ID_REG_ENT(ID_ISAR6),
+
+	/* CRm=3 */
+	ID_REG_ENT(MVFR0),
+	ID_REG_ENT(MVFR1),
+	ID_REG_ENT(MVFR2),
+	ID_REG_ENT(ID_REG_3_3),
+	ID_REG_ENT(ID_PFR2),
+	ID_REG_ENT(ID_DFR1),
+	ID_REG_ENT(ID_MMFR5),
+	ID_REG_ENT(ID_REG_3_7),
+
+	/* CRm=4 */
+	ID_REG_ENT(ID_AA64PFR0),
+	ID_REG_ENT(ID_AA64PFR1),
+	ID_REG_ENT(ID_REG_4_2),
+	ID_REG_ENT(ID_REG_4_3),
+	ID_REG_ENT(ID_AA64ZFR0),
+	ID_REG_ENT(ID_REG_4_5),
+	ID_REG_ENT(ID_REG_4_6),
+	ID_REG_ENT(ID_REG_4_7),
+
+	/* CRm=5 */
+	ID_REG_ENT(ID_AA64DFR0),
+	ID_REG_ENT(ID_AA64DFR1),
+	ID_REG_ENT(ID_REG_5_2),
+	ID_REG_ENT(ID_REG_5_3),
+	ID_REG_ENT(ID_AA64AFR0),
+	ID_REG_ENT(ID_AA64AFR1),
+	ID_REG_ENT(ID_REG_5_6),
+	ID_REG_ENT(ID_REG_5_7),
+
+	/* CRm=6 */
+	ID_REG_ENT(ID_AA64ISAR0),
+	ID_REG_ENT(ID_AA64ISAR1),
+	ID_REG_ENT(ID_REG_6_2),
+	ID_REG_ENT(ID_REG_6_3),
+	ID_REG_ENT(ID_REG_6_4),
+	ID_REG_ENT(ID_REG_6_5),
+	ID_REG_ENT(ID_REG_6_6),
+	ID_REG_ENT(ID_REG_6_7),
+
+	/* CRm=7 */
+	ID_REG_ENT(ID_AA64MMFR0),
+	ID_REG_ENT(ID_AA64MMFR1),
+	ID_REG_ENT(ID_AA64MMFR2),
+	ID_REG_ENT(ID_REG_7_3),
+	ID_REG_ENT(ID_REG_7_4),
+	ID_REG_ENT(ID_REG_7_5),
+	ID_REG_ENT(ID_REG_7_6),
+	ID_REG_ENT(ID_REG_7_7),
+};
+
+/* Utilities to get a feature field from ID register value */
+static inline int
+cpuid_signed_field_width(uint64_t id_val, int field, int width)
+{
+	return (s64)(id_val << (64 - width - field)) >> (64 - width);
+}
+
+static unsigned int
+cpuid_unsigned_field_width(uint64_t id_val, int field, int width)
+{
+	return (uint64_t)(id_val << (64 - width - field)) >> (64 - width);
+}
+
+static inline int __attribute_const__
+cpuid_extract_field_width(uint64_t id_val, int field, int width, bool sign)
+{
+	return (sign) ? cpuid_signed_field_width(id_val, field, width) :
+			cpuid_unsigned_field_width(id_val, field, width);
+}
+
+#define	GET_ID_FIELD(regval, shift, is_signed)	\
+	cpuid_extract_field_width(regval, shift, 4, is_signed)
+
+#define	GET_ID_UFIELD(regval, shift)	\
+	cpuid_unsigned_field_width(regval, shift, 4)
+
+#define	UPDATE_ID_UFIELD(regval, shift, fval)	\
+	(((regval) & ~(0xfULL << (shift))) |	\
+	 (((uint64_t)((fval) & 0xf)) << (shift)))
+
+void test_pmu_init(struct kvm_vm *vm, uint32_t vcpu)
+{
+	struct kvm_device_attr attr = {
+		.group = KVM_ARM_VCPU_PMU_V3_CTRL,
+		.attr = KVM_ARM_VCPU_PMU_V3_INIT,
+	};
+	vcpu_ioctl(vm, vcpu, KVM_SET_DEVICE_ATTR, &attr);
+}
+
+void test_sve_init(struct kvm_vm *vm, uint32_t vcpu)
+{
+	int feature = KVM_ARM_VCPU_SVE;
+
+	vcpu_ioctl(vm, vcpu, KVM_ARM_VCPU_FINALIZE, &feature);
+}
+
+#define GICD_BASE_GPA			0x8000000ULL
+#define GICR_BASE_GPA			0x80A0000ULL
+
+void test_vgic_init(struct kvm_vm *vm, uint32_t vcpu)
+{
+	/* We jsut need to configure gic v3 (we don't use it though) */
+	vgic_v3_setup(vm, 1, GICD_BASE_GPA, GICR_BASE_GPA);
+}
+
+#define	MAX_CAPS	2
+struct feature_test_info {
+	char	*name;	/* Feature Name (Debug information) */
+	struct id_reg_test_info	*sreg;	/* ID register for the feature */
+	int	shift;	/* Field of the ID register for the feature */
+	int	min;	/* Min value to indicate the feature */
+	bool	is_sign;	/* Is the field signed or unsigned ? */
+	int	ncaps;		/* Number of valid Capabilities in caps[] */
+	long	caps[MAX_CAPS];	/* Capabilities to indicate this feature */
+	/* struct kvm_enable_cap to use the capability if needed */
+	struct kvm_enable_cap	*opt_in_cap;
+	bool	run_test;	/* Does guest run test for this feature ? */
+	/* Initialization function for the feature as needed */
+	void	(*init_feature)(struct kvm_vm *vm, uint32_t vcpuid);
+	/* struct kvm_vcpu_init to opt-in the feature if needed */
+	struct kvm_vcpu_init	*vcpu_init;
+};
+
+/* Test for optin CPU features */
+static struct feature_test_info feature_test_info_table[] = {
+	{
+		.name = "SVE",
+		.sreg = ID_REG_INFO(ID_AA64PFR0),
+		.shift = ID_AA64PFR0_SVE_SHIFT,
+		.min = 1,
+		.caps = {KVM_CAP_ARM_SVE},
+		.ncaps = 1,
+		.init_feature = test_sve_init,
+		.vcpu_init = &(struct kvm_vcpu_init) {
+			.features = {1ULL << KVM_ARM_VCPU_SVE},
+		},
+	},
+	{
+		.name = "GIC",
+		.sreg = ID_REG_INFO(ID_AA64PFR0),
+		.shift = ID_AA64PFR0_GIC_SHIFT,
+		.min = 1,
+		.caps = {KVM_CAP_IRQCHIP},
+		.ncaps = 1,
+		.init_feature = test_vgic_init,
+	},
+	{
+		.name = "MTE",
+		.sreg = ID_REG_INFO(ID_AA64PFR1),
+		.shift = ID_AA64PFR1_MTE_SHIFT,
+		.min = 2,
+		.caps = {KVM_CAP_ARM_MTE},
+		.ncaps = 1,
+		.opt_in_cap = &(struct kvm_enable_cap) {
+				.cap = KVM_CAP_ARM_MTE,
+		},
+	},
+	{
+		.name = "PMUV3",
+		.sreg = ID_REG_INFO(ID_AA64DFR0),
+		.shift = ID_AA64DFR0_PMUVER_SHIFT,
+		.min = 1,
+		.init_feature = test_pmu_init,
+		.caps = {KVM_CAP_ARM_PMU_V3},
+		.ncaps = 1,
+		.vcpu_init = &(struct kvm_vcpu_init) {
+			.features = {1ULL << KVM_ARM_VCPU_PMU_V3},
+		},
+	},
+	{
+		.name = "PERFMON",
+		.sreg = ID_REG_INFO(ID_DFR0),
+		.shift = ID_DFR0_PERFMON_SHIFT,
+		.min = 3,
+		.init_feature = test_pmu_init,
+		.caps = {KVM_CAP_ARM_PMU_V3},
+		.ncaps = 1,
+		.vcpu_init = &(struct kvm_vcpu_init) {
+			.features = {1ULL << KVM_ARM_VCPU_PMU_V3},
+		},
+	},
+};
+
+static int walk_id_reg_list(int (*fn)(struct id_reg_test_info *sreg, void *arg),
+			    void *arg)
+{
+	int ret = 0, i;
+
+	for (i = 0; i < ARRAY_SIZE(id_reg_list); i++) {
+		ret = fn(&id_reg_list[i], arg);
+		if (ret)
+			break;
+	}
+	return ret;
+}
+
+static int guest_code_id_reg_check_one(struct id_reg_test_info *sreg, void *arg)
+{
+	uint64_t val = sreg->read_reg();
+
+	GUEST_ASSERT_2(val == sreg->user_val, sreg->name, sreg->user_val);
+	return 0;
+}
+
+static void guest_code_id_reg_check_all(uint32_t cpu)
+{
+	walk_id_reg_list(guest_code_id_reg_check_one, NULL);
+	GUEST_DONE();
+}
+
+static void guest_code_do_nothing(uint32_t cpu)
+{
+	GUEST_DONE();
+}
+
+static void guest_code_feature_check(uint32_t cpu)
+{
+	int i;
+	struct feature_test_info *finfo;
+
+	for (i = 0; i < ARRAY_SIZE(feature_test_info_table); i++) {
+		finfo = &feature_test_info_table[i];
+		if (finfo->run_test)
+			guest_code_id_reg_check_one(finfo->sreg, NULL);
+	}
+
+	GUEST_DONE();
+}
+
+static void guest_code_ptrauth_check(uint32_t cpuid)
+{
+	struct id_reg_test_info *sreg = ID_REG_INFO(ID_AA64ISAR1);
+	uint64_t val = sreg->read_reg();
+
+	GUEST_ASSERT_2(val == sreg->user_val, "PTRAUTH", val);
+	GUEST_DONE();
+}
+
+static int reset_id_reg_info_one(struct id_reg_test_info *sreg, void *arg)
+{
+	sreg->user_val = sreg->org_val;
+	return 0;
+}
+
+static void reset_id_reg_info(void)
+{
+	walk_id_reg_list(reset_id_reg_info_one, NULL);
+}
+
+static struct kvm_vm *test_vm_create_cap(uint32_t nvcpus,
+		void (*guest_code)(uint32_t), struct kvm_vcpu_init *init,
+		struct kvm_enable_cap *cap)
+{
+	struct kvm_vm *vm;
+	uint32_t cpuid;
+	uint64_t mem_pages;
+
+	mem_pages = DEFAULT_GUEST_PHY_PAGES + DEFAULT_STACK_PGS * nvcpus;
+	mem_pages += mem_pages / (PTES_PER_MIN_PAGE * 2);
+	mem_pages = vm_adjust_num_guest_pages(VM_MODE_DEFAULT, mem_pages);
+
+	vm = vm_create(VM_MODE_DEFAULT,
+		DEFAULT_GUEST_PHY_PAGES + (DEFAULT_STACK_PGS * nvcpus),
+		O_RDWR);
+	if (cap)
+		vm_enable_cap(vm, cap);
+
+	kvm_vm_elf_load(vm, program_invocation_name);
+
+	if (init && init->target == -1) {
+		struct kvm_vcpu_init preferred;
+
+		vm_ioctl(vm, KVM_ARM_PREFERRED_TARGET, &preferred);
+		init->target = preferred.target;
+	}
+
+	vm_init_descriptor_tables(vm);
+	for (cpuid = 0; cpuid < nvcpus; cpuid++) {
+		if (init)
+			aarch64_vcpu_add_default(vm, cpuid, init, guest_code);
+		else
+			vm_vcpu_add_default(vm, cpuid, guest_code);
+
+		vcpu_init_descriptor_tables(vm, cpuid);
+	}
+
+	ucall_init(vm, NULL);
+	return vm;
+}
+
+static struct kvm_vm *test_vm_create(uint32_t nvcpus,
+				     void (*guest_code)(uint32_t),
+				     struct kvm_vcpu_init *init)
+{
+	return test_vm_create_cap(nvcpus, guest_code, init, NULL);
+}
+
+static void test_vm_free(struct kvm_vm *vm)
+{
+	ucall_uninit(vm);
+	kvm_vm_free(vm);
+}
+
+#define	TEST_RUN(vm, cpu)	\
+	(test_vcpu_run(__func__, __LINE__, vm, cpu, true))
+
+#define	TEST_RUN_NO_SYNC_DATA(vm, cpu)	\
+	(test_vcpu_run(__func__, __LINE__, vm, cpu, false))
+
+static int test_vcpu_run(const char *test_name, int line,
+			 struct kvm_vm *vm, uint32_t vcpuid, bool sync_data)
+{
+	struct ucall uc;
+	int ret;
+
+	if (sync_data) {
+		sync_global_to_guest(vm, id_reg_list);
+		sync_global_to_guest(vm, feature_test_info_table);
+	}
+
+	vcpu_args_set(vm, vcpuid, 1, vcpuid);
+
+	ret = _vcpu_run(vm, vcpuid);
+	if (ret) {
+		ret = errno;
+		goto sync_exit;
+	}
+
+	switch (get_ucall(vm, vcpuid, &uc)) {
+	case UCALL_SYNC:
+	case UCALL_DONE:
+		ret = 0;
+		break;
+	case UCALL_ABORT:
+		TEST_FAIL(
+		    "%s (%s) at line %d (user %s at line %d), args[3]=0x%lx",
+		    (char *)uc.args[0], (char *)uc.args[2], (int)uc.args[1],
+		    test_name, line, uc.args[3]);
+		break;
+	default:
+		TEST_FAIL("Unexpected guest exit\n");
+	}
+
+sync_exit:
+	if (sync_data) {
+		sync_global_from_guest(vm, id_reg_list);
+		sync_global_from_guest(vm, feature_test_info_table);
+	}
+	return ret;
+}
+
+static int set_id_regs_after_run_test_one(struct id_reg_test_info *sreg,
+					  void *arg)
+{
+	struct kvm_vm *vm = arg;
+	struct kvm_one_reg one_reg;
+	uint32_t vcpuid = 0;
+	uint64_t reg_val;
+	int ret;
+
+	one_reg.addr = (uint64_t)&reg_val;
+	one_reg.id = KVM_ARM64_SYS_REG(sreg->id);
+
+	vcpu_ioctl(vm, vcpuid, KVM_GET_ONE_REG, &one_reg);
+	if ((reg_val != 0) && (sreg->can_clear)) {
+		reg_val = 0;
+		ret = _vcpu_ioctl(vm, vcpuid, KVM_SET_ONE_REG, &one_reg);
+		TEST_ASSERT(ret && errno == EINVAL,
+			    "Changing ID reg value should fail\n");
+	}
+
+	vcpu_ioctl(vm, vcpuid, KVM_GET_ONE_REG, &one_reg);
+	ret = _vcpu_ioctl(vm, vcpuid, KVM_SET_ONE_REG, &one_reg);
+	TEST_ASSERT(ret == 0, "Setting the same ID reg value should work\n");
+
+	return 0;
+}
+
+static int set_id_regs_test_one(struct id_reg_test_info *sreg, void *arg)
+{
+	struct kvm_vm *vm = arg;
+	struct kvm_one_reg one_reg;
+	uint32_t vcpuid = 0;
+	uint64_t reg_val;
+
+	one_reg.addr = (uint64_t)&reg_val;
+	reset_id_reg_info();
+
+	one_reg.id = KVM_ARM64_SYS_REG(sreg->id);
+	if (sreg->can_clear) {
+		/* Change the register to 0 when possible */
+		reg_val = 0;
+		vcpu_ioctl(vm, vcpuid, KVM_SET_ONE_REG, &one_reg);
+		vcpu_ioctl(vm, vcpuid, KVM_GET_ONE_REG, &one_reg);
+		TEST_ASSERT(reg_val == 0,
+		    "GET(%s) didn't return 0 but 0x%lx", sreg->name, reg_val);
+	}
+
+	/* Check if we can restore the initial value */
+	reg_val = sreg->org_val;
+	vcpu_ioctl(vm, vcpuid, KVM_SET_ONE_REG, &one_reg);
+	vcpu_ioctl(vm, vcpuid, KVM_GET_ONE_REG, &one_reg);
+	TEST_ASSERT(reg_val == sreg->org_val,
+		    "GET(%s) didn't return 0x%lx but 0x%lx",
+		    sreg->name, sreg->org_val, reg_val);
+	sreg->user_val = sreg->org_val;
+	return 0;
+}
+
+static void set_id_regs_test(void)
+{
+	struct kvm_vm *vm;
+	int ret;
+
+	reset_id_reg_info();
+	vm = test_vm_create(1, guest_code_id_reg_check_all, NULL);
+
+	ret = walk_id_reg_list(set_id_regs_test_one, vm);
+	assert(!ret);
+
+	ret = TEST_RUN(vm, 0);
+	TEST_ASSERT(!ret, "%s TEST_RUN failed, ret=0x%x", __func__, ret);
+
+	ret = walk_id_reg_list(set_id_regs_after_run_test_one, vm);
+	assert(!ret);
+}
+
+static bool caps_are_supported(long *caps, int ncaps)
+{
+	int i;
+
+	for (i = 0; i < ncaps; i++) {
+		if (kvm_check_cap(caps[i]) <= 0)
+			return false;
+	}
+	return true;
+}
+
+static void test_feature_ptrauth(void)
+{
+	struct kvm_one_reg one_reg;
+	struct kvm_vcpu_init init;
+	struct kvm_vm *vm = NULL;
+	struct id_reg_test_info *sreg = ID_REG_INFO(ID_AA64ISAR1);
+	uint32_t vcpu = 0;
+	int64_t rval;
+	int ret;
+	int apa, api, gpa, gpi;
+	char *name = "PTRAUTH";
+	long caps[2] = {KVM_CAP_ARM_PTRAUTH_ADDRESS,
+			KVM_CAP_ARM_PTRAUTH_GENERIC};
+
+	reset_id_reg_info();
+	one_reg.addr = (uint64_t)&rval;
+	one_reg.id = KVM_ARM64_SYS_REG(sreg->id);
+
+	if (caps_are_supported(caps, 2)) {
+
+		/* Test with feature enabled */
+		memset(&init, 0, sizeof(init));
+		init.target = -1;
+		init.features[0] = (1ULL << KVM_ARM_VCPU_PTRAUTH_ADDRESS |
+				    1ULL << KVM_ARM_VCPU_PTRAUTH_GENERIC);
+		vm = test_vm_create_cap(1, guest_code_ptrauth_check, &init,
+					NULL);
+		vcpu_ioctl(vm, vcpu, KVM_GET_ONE_REG, &one_reg);
+		apa = GET_ID_UFIELD(rval, ID_AA64ISAR1_APA_SHIFT);
+		api = GET_ID_UFIELD(rval, ID_AA64ISAR1_API_SHIFT);
+		gpa = GET_ID_UFIELD(rval, ID_AA64ISAR1_GPA_SHIFT);
+		gpi = GET_ID_UFIELD(rval, ID_AA64ISAR1_GPI_SHIFT);
+
+		TEST_ASSERT((apa > 0) || (api > 0),
+			    "Either apa(0x%x) or api(0x%x) must be available",
+			    apa, gpa);
+		TEST_ASSERT((gpa > 0) || (gpi > 0),
+			    "Either gpa(0x%x) or gpi(0x%x) must be available",
+			    gpa, gpi);
+
+		TEST_ASSERT((apa > 0) ^ (api > 0),
+			    "Both apa(0x%x) and api(0x%x) must not be available",
+			    apa, api);
+		TEST_ASSERT((gpa > 0) ^ (gpi > 0),
+			    "Both gpa(0x%x) and gpi(0x%x) must not be available",
+			    gpa, gpi);
+
+		sreg->user_val = rval;
+
+		pr_debug("%s: Test with %s enabled (%s: 0x%lx)\n",
+			 __func__, name, sreg->name, sreg->user_val);
+		ret = TEST_RUN(vm, vcpu);
+		TEST_ASSERT(!ret, "%s:KVM_RUN failed with %s enabled",
+			    __func__, name);
+		test_vm_free(vm);
+	}
+
+	/* Test with feature disabled */
+	reset_id_reg_info();
+
+	vm = test_vm_create(1, guest_code_feature_check, NULL);
+	vcpu_ioctl(vm, vcpu, KVM_GET_ONE_REG, &one_reg);
+
+	apa = GET_ID_UFIELD(rval, ID_AA64ISAR1_APA_SHIFT);
+	api = GET_ID_UFIELD(rval, ID_AA64ISAR1_API_SHIFT);
+	gpa = GET_ID_UFIELD(rval, ID_AA64ISAR1_GPA_SHIFT);
+	gpi = GET_ID_UFIELD(rval, ID_AA64ISAR1_GPI_SHIFT);
+	TEST_ASSERT(!apa && !api && !gpa && !gpi,
+	    "apa(0x%x), api(0x%x), gpa(0x%x), gpi(0x%x) must be zero",
+	    apa, api, gpa, gpi);
+
+	pr_debug("%s: Test with %s disabled (%s: 0x%lx)\n",
+		 __func__, name, sreg->name, sreg->user_val);
+
+	ret = TEST_RUN(vm, vcpu);
+	TEST_ASSERT(!ret, "%s TEST_RUN failed with %s enabled, ret=0x%x",
+		    __func__, name, ret);
+
+	test_vm_free(vm);
+}
+
+static bool feature_caps_are_available(struct feature_test_info *finfo)
+{
+	return ((finfo->ncaps > 0) &&
+		caps_are_supported(finfo->caps, finfo->ncaps));
+}
+
+static void test_feature(struct feature_test_info *finfo)
+{
+	struct id_reg_test_info *sreg = finfo->sreg;
+	struct kvm_one_reg one_reg;
+	struct kvm_vcpu_init init, *initp = NULL;
+	struct kvm_vm *vm = NULL;
+	int64_t fval, reg_val;
+	uint32_t vcpu = 0;
+	bool is_sign = finfo->is_sign;
+	int min = finfo->min;
+	int shift = finfo->shift;
+	int ret;
+
+	pr_debug("%s: %s (reg %s)\n", __func__, finfo->name, sreg->name);
+
+	reset_id_reg_info();
+	finfo->run_test = 1;	/* Indicate that guest runs the test on it */
+	one_reg.addr = (uint64_t)&reg_val;
+	one_reg.id = KVM_ARM64_SYS_REG(sreg->id);
+
+	/* Test with feature enabled if the feature is available */
+	if ((GET_ID_FIELD(sreg->org_val, shift, is_sign) >= min) ||
+	    feature_caps_are_available(finfo)) {
+		if (finfo->vcpu_init) {
+			/*
+			 * Need to enable the feature via
+			 * KVM_ARM_VCPU_INIT.
+			 */
+			memset(&init, 0, sizeof(init));
+			init = *finfo->vcpu_init;
+			init.target = -1;
+			initp = &init;
+		}
+
+		vm = test_vm_create_cap(1, guest_code_feature_check, initp,
+					finfo->opt_in_cap);
+		if (finfo->init_feature)
+			/* Run any required extra process to use the feature */
+			finfo->init_feature(vm, vcpu);
+
+		/* Check if the ID register value indicates the feature */
+		vcpu_ioctl(vm, vcpu, KVM_GET_ONE_REG, &one_reg);
+		fval = GET_ID_FIELD(reg_val, shift, is_sign);
+		TEST_ASSERT(fval >= min, "%s field of %s is too small (%ld)",
+			    finfo->name, sreg->name, fval);
+		sreg->user_val = reg_val;
+
+		pr_debug("%s: Test with %s enabled (%s: 0x%lx)\n",
+			 __func__, finfo->name, sreg->name, sreg->user_val);
+
+		ret = TEST_RUN(vm, vcpu);
+		TEST_ASSERT(!ret, "%s:TEST_RUN failed with %s enabled",
+			    __func__, finfo->name);
+		test_vm_free(vm);
+	}
+
+	/* Test with feature disabled */
+	reset_id_reg_info();
+
+	vm = test_vm_create(1, guest_code_feature_check, NULL);
+	vcpu_ioctl(vm, vcpu, KVM_GET_ONE_REG, &one_reg);
+	fval = GET_ID_FIELD(reg_val, shift, is_sign);
+	if (finfo->vcpu_init || finfo->opt_in_cap) {
+		/*
+		 * If the feature needs to be enabled with KVM_ARM_VCPU_INIT
+		 * or opt-in capabilities, the default value of the ID register
+		 * shouldn't indicate the feature.
+		 */
+		TEST_ASSERT(fval < min, "%s field of %s is too big (%ld)",
+		    finfo->name, sreg->name, fval);
+	} else {
+		/* Update the relevant field to hide the feature. */
+		fval = is_sign ? 0xf : 0x0;
+		reg_val = UPDATE_ID_UFIELD(reg_val, shift, fval);
+		ret = _vcpu_ioctl(vm, vcpu, KVM_SET_ONE_REG, &one_reg);
+		TEST_ASSERT(ret == 0, "Disabling %s failed %d\n",
+			    finfo->name, ret);
+		sreg->user_val = reg_val;
+	}
+
+	pr_debug("%s: Test with %s disabled (%s: 0x%lx)\n",
+		 __func__, finfo->name, sreg->name, sreg->user_val);
+
+	ret = TEST_RUN(vm, vcpu);
+	finfo->run_test = 0;
+	test_vm_free(vm);
+}
+
+static void test_feature_all(void)
+{
+	int i;
+
+	for (i = 0; i < ARRAY_SIZE(feature_test_info_table); i++)
+		test_feature(&feature_test_info_table[i]);
+}
+
+int test_set_reg(struct id_reg_test_info *sreg, uint64_t new_val,
+		 bool guest_run)
+{
+	struct kvm_vm *vm;
+	int ret;
+	uint32_t vcpu = 0;
+	uint64_t reg_val;
+	struct kvm_one_reg one_reg;
+
+	reset_id_reg_info();
+
+	vm = test_vm_create(1, guest_code_id_reg_check_all, NULL);
+	one_reg.id = KVM_ARM64_SYS_REG(sreg->id);
+	one_reg.addr = (uint64_t)&reg_val;
+
+	reg_val = new_val;
+	ret = _vcpu_ioctl(vm, vcpu, KVM_SET_ONE_REG, &one_reg);
+	if (!guest_run)
+		return ret;
+
+	TEST_ASSERT(!ret, "SET_REG(%s=0x%lx) failed, ret=0x%x",
+		    sreg->name, new_val, ret);
+	sreg->user_val = new_val;
+	ret = TEST_RUN(vm, vcpu);
+	test_vm_free(vm);
+	return ret;
+}
+
+int test_feature_frac_vm(struct id_reg_test_info *sreg, uint64_t new_val,
+		      struct id_reg_test_info *frac_sreg, uint64_t frac_new_val)
+{
+	struct kvm_vm *vm;
+	int ret;
+	uint32_t vcpu = 0;
+	uint64_t reg_val;
+	struct kvm_one_reg one_reg;
+
+	reset_id_reg_info();
+
+	vm = test_vm_create(1, guest_code_id_reg_check_all, NULL);
+
+	/* Set feature reg field */
+	one_reg.id = KVM_ARM64_SYS_REG(sreg->id);
+	one_reg.addr = (uint64_t)&reg_val;
+	reg_val = new_val;
+	ret = _vcpu_ioctl(vm, vcpu, KVM_SET_ONE_REG, &one_reg);
+	TEST_ASSERT(!ret, "SET_REG(%s=0x%lx) failed, ret=0x%x",
+		    sreg->name, new_val, ret);
+	sreg->user_val = new_val;
+
+	/* Set fractional reg field */
+	one_reg.id = KVM_ARM64_SYS_REG(frac_sreg->id);
+	one_reg.addr = (uint64_t)&reg_val;
+	reg_val = frac_new_val;
+	vcpu_ioctl(vm, vcpu, KVM_SET_ONE_REG, &one_reg);
+	TEST_ASSERT(!ret, "SET_REG(%s=0x%lx) failed, ret=0x%x",
+		    frac_sreg->name, frac_new_val, ret);
+
+	frac_sreg->user_val = frac_new_val;
+	ret = TEST_RUN(vm, vcpu);
+	test_vm_free(vm);
+	return ret;
+}
+
+struct frac_info {
+	char	*name;
+	struct id_reg_test_info *sreg;
+	struct id_reg_test_info *frac_sreg;
+	int	shift;
+	int	frac_shift;
+};
+
+struct frac_info frac_info_table[] = {
+	{
+		.name = "RAS",
+		.sreg = ID_REG_INFO(ID_AA64PFR0),
+		.shift = ID_AA64PFR0_RAS_SHIFT,
+		.frac_sreg = ID_REG_INFO(ID_AA64PFR1),
+		.frac_shift = ID_AA64PFR1_RASFRAC_SHIFT,
+	},
+	{
+		.name = "MPAM",
+		.sreg = ID_REG_INFO(ID_AA64PFR0),
+		.shift = ID_AA64PFR0_MPAM_SHIFT,
+		.frac_sreg = ID_REG_INFO(ID_AA64PFR1),
+		.frac_shift = ID_AA64PFR1_MPAMFRAC_SHIFT,
+	},
+	{
+		.name = "CSV2",
+		.sreg = ID_REG_INFO(ID_AA64PFR0),
+		.shift = ID_AA64PFR0_CSV2_SHIFT,
+		.frac_sreg = ID_REG_INFO(ID_AA64PFR1),
+		.frac_shift = ID_AA64PFR1_CSV2FRAC_SHIFT,
+	},
+};
+
+void test_feature_frac_one(struct frac_info *frac)
+{
+	uint64_t reg_val, org_fval, frac_reg_val, frac_org_fval;
+	int ret, shift, frac_shift;
+	struct id_reg_test_info *sreg, *frac_sreg;
+
+	reset_id_reg_info();
+
+	sreg = frac->sreg;
+	shift = frac->shift;
+	frac_sreg = frac->frac_sreg;
+	frac_shift = frac->frac_shift;
+
+	pr_debug("%s(%s Frac) reg:%s(shift:%d) frac reg:%s(shift:%d)\n",
+		__func__, frac->name, sreg->name, shift,
+		frac_sreg->name, frac_shift);
+
+	frac_org_fval = GET_ID_UFIELD(frac_sreg->org_val, frac_shift);
+	if (frac_org_fval > 0) {
+		/* Test with smaller frac value */
+		frac_reg_val = UPDATE_ID_UFIELD(frac_sreg->org_val,
+						frac_shift, frac_org_fval - 1);
+		ret = test_set_reg(frac_sreg, frac_reg_val, false);
+		TEST_ASSERT(!ret, "SET smaller %s frac (val:%lx) failed(%d)",
+			    frac->name, frac_reg_val, ret);
+
+		ret = test_feature_frac_vm(sreg, sreg->org_val,
+					   frac_sreg, frac_reg_val);
+		TEST_ASSERT(!ret, "Test smaller %s frac (val:%lx) failed(%d)",
+			    frac->name, frac_reg_val, ret);
+	}
+
+	reset_id_reg_info();
+
+	if (frac_org_fval != 0xf) {
+		/* Test with larger frac value */
+		frac_reg_val = UPDATE_ID_UFIELD(frac_sreg->org_val, frac_shift,
+						frac_org_fval + 1);
+
+		/* Setting larger frac shouldn't fail (at ioctl) */
+		ret = test_set_reg(frac_sreg, frac_reg_val, false);
+		TEST_ASSERT(!ret,
+			"SET larger %s frac (%s org:%lx, val:%lx) failed(%d)",
+			frac->name, frac_sreg->name, frac_sreg->org_val,
+			frac_reg_val, ret);
+
+		/* KVM_RUN with larger frac should fail */
+		ret = test_feature_frac_vm(sreg, sreg->org_val,
+					   frac_sreg, frac_reg_val);
+		TEST_ASSERT(ret,
+			"Test with larger %s frac (%s org:%lx, val:%lx) worked",
+			frac->name, frac_sreg->name, frac_sreg->org_val,
+			frac_reg_val);
+	}
+
+	reset_id_reg_info();
+
+	org_fval = GET_ID_UFIELD(sreg->org_val, shift);
+	if (org_fval == 0) {
+		/* Setting larger val for the feature should fail */
+		reg_val = UPDATE_ID_UFIELD(sreg->org_val, shift, org_fval + 1);
+		ret = test_set_reg(sreg, reg_val, false);
+		TEST_ASSERT(ret, "SET larger %s (val:%lx) worked",
+			    frac->name, reg_val);
+		return;
+	}
+
+	/* Test with smaller feature value */
+	reg_val = UPDATE_ID_UFIELD(sreg->org_val, shift, org_fval - 1);
+	ret = test_set_reg(sreg, reg_val, false);
+	TEST_ASSERT(!ret, "SET smaller %s (val:%lx) failed(%d)",
+		    frac->name, reg_val, ret);
+
+	ret = test_feature_frac_vm(sreg, reg_val, frac_sreg, frac_sreg->org_val);
+	TEST_ASSERT(!ret, "Test with smaller %s (val:%lx) failed(%d)",
+		    frac->name, reg_val, ret);
+
+	if (frac_org_fval > 0) {
+		/* Test with smaller feature and frac value */
+		frac_reg_val = UPDATE_ID_UFIELD(frac_sreg->org_val,
+						frac_shift, frac_org_fval - 1);
+		ret = test_feature_frac_vm(sreg, reg_val, frac_sreg,
+					   frac_reg_val);
+		TEST_ASSERT(!ret,
+		    "Test with smaller %s and frac (val:%lx) failed(%d)",
+		    frac->name, reg_val, ret);
+	}
+
+	if (frac_org_fval != 0xf) {
+		/* Test with smaller feature and larger frac value */
+		frac_reg_val = UPDATE_ID_UFIELD(frac_sreg->org_val,
+						frac_shift, frac_org_fval + 1);
+		ret = test_feature_frac_vm(sreg, reg_val, frac_sreg,
+					   frac_reg_val);
+		TEST_ASSERT(!ret,
+		    "Test with smaller %s and larger frac (val:%lx) failed(%d)",
+		    frac->name, reg_val, ret);
+	}
+}
+
+void test_feature_frac_all(void)
+{
+	int i;
+
+	for (i = 0; i < ARRAY_SIZE(frac_info_table); i++)
+		test_feature_frac_one(&frac_info_table[i]);
+}
+
+void run_test(void)
+{
+	set_id_regs_test();
+	test_feature_all();
+	test_feature_ptrauth();
+	test_feature_frac_all();
+}
+
+static int init_id_reg_info_one(struct id_reg_test_info *sreg, void *arg)
+{
+	uint64_t reg_val;
+	uint32_t vcpuid = 0;
+	int ret;
+	struct kvm_one_reg one_reg;
+	struct kvm_vm *vm = arg;
+
+	one_reg.addr = (uint64_t)&reg_val;
+	one_reg.id = KVM_ARM64_SYS_REG(sreg->id);
+	vcpu_ioctl(vm, vcpuid, KVM_GET_ONE_REG, &one_reg);
+	sreg->org_val = reg_val;
+	sreg->user_val = reg_val;
+	if (sreg->org_val) {
+		reg_val = 0;
+		ret = _vcpu_ioctl(vm, vcpuid, KVM_SET_ONE_REG, &one_reg);
+		if (!ret)
+			sreg->can_clear = true;
+	}
+
+	pr_debug("%s (0x%x): 0x%lx%s\n", sreg->name, sreg->id,
+		 sreg->org_val, sreg->can_clear ? ", can clear" : "");
+
+	return 0;
+}
+
+static void init_id_reg_info(void)
+{
+	struct kvm_vm *vm;
+
+	vm = test_vm_create(1, guest_code_do_nothing, NULL);
+	walk_id_reg_list(init_id_reg_info_one, vm);
+	test_vm_free(vm);
+}
+
+int main(void)
+{
+
+	setbuf(stdout, NULL);
+
+	if (kvm_check_cap(KVM_CAP_ARM_ID_REG_CONFIGURABLE) <= 0) {
+		print_skip("KVM_CAP_ARM_ID_REG_CONFIGURABLE is not supported\n");
+		exit(KSFT_SKIP);
+	}
+
+	init_id_reg_info();
+	run_test();
+	return 0;
+}
-- 
2.34.0.rc1.387.gb447b232ab-goog

Re: [RFC PATCH v3 29/29] KVM: arm64: selftests: Introduce id_reg_test

[Eric Auger]

Hi Reiji,

On 11/17/21 7:43 AM, Reiji Watanabe wrote:
> Introduce a test for aarch64 to validate basic behavior of
> KVM_GET_ONE_REG and KVM_SET_ONE_REG for ID registers.
> 
> This test runs only when KVM_CAP_ARM_ID_REG_CONFIGURABLE is supported.

That's great to get those tests along with the series.

There are several tests actually. I would encourage you to drop a short
comment along with the each main test to summarize what it does.
> 
> Signed-off-by: Reiji Watanabe <reijiw@google.com>
> ---
>  tools/arch/arm64/include/asm/sysreg.h         |    1 +
>  tools/testing/selftests/kvm/.gitignore        |    1 +
>  tools/testing/selftests/kvm/Makefile          |    1 +
>  .../selftests/kvm/aarch64/id_reg_test.c       | 1128 +++++++++++++++++
>  4 files changed, 1131 insertions(+)
>  create mode 100644 tools/testing/selftests/kvm/aarch64/id_reg_test.c
> 
> diff --git a/tools/arch/arm64/include/asm/sysreg.h b/tools/arch/arm64/include/asm/sysreg.h
> index 7640fa27be94..be3947c125f1 100644
> --- a/tools/arch/arm64/include/asm/sysreg.h
> +++ b/tools/arch/arm64/include/asm/sysreg.h
> @@ -793,6 +793,7 @@
>  #define ID_AA64PFR0_ELx_32BIT_64BIT	0x2
>  
>  /* id_aa64pfr1 */
> +#define ID_AA64PFR1_CSV2FRAC_SHIFT	32
>  #define ID_AA64PFR1_MPAMFRAC_SHIFT	16
>  #define ID_AA64PFR1_RASFRAC_SHIFT	12
>  #define ID_AA64PFR1_MTE_SHIFT		8
> diff --git a/tools/testing/selftests/kvm/.gitignore b/tools/testing/selftests/kvm/.gitignore
> index d4a830139683..5daf1400f0cf 100644
> --- a/tools/testing/selftests/kvm/.gitignore
> +++ b/tools/testing/selftests/kvm/.gitignore
> @@ -2,6 +2,7 @@
>  /aarch64/arch_timer
>  /aarch64/debug-exceptions
>  /aarch64/get-reg-list
> +/aarch64/id_reg_test
>  /aarch64/psci_cpu_on_test
>  /aarch64/vgic_init
>  /s390x/memop
> diff --git a/tools/testing/selftests/kvm/Makefile b/tools/testing/selftests/kvm/Makefile
> index c4e34717826a..fee6ba13019c 100644
> --- a/tools/testing/selftests/kvm/Makefile
> +++ b/tools/testing/selftests/kvm/Makefile
> @@ -92,6 +92,7 @@ TEST_GEN_PROGS_x86_64 += system_counter_offset_test
>  TEST_GEN_PROGS_aarch64 += aarch64/arch_timer
>  TEST_GEN_PROGS_aarch64 += aarch64/debug-exceptions
>  TEST_GEN_PROGS_aarch64 += aarch64/get-reg-list
> +TEST_GEN_PROGS_aarch64 += aarch64/id_reg_test
>  TEST_GEN_PROGS_aarch64 += aarch64/psci_cpu_on_test
>  TEST_GEN_PROGS_aarch64 += aarch64/vgic_init
>  TEST_GEN_PROGS_aarch64 += demand_paging_test
> diff --git a/tools/testing/selftests/kvm/aarch64/id_reg_test.c b/tools/testing/selftests/kvm/aarch64/id_reg_test.c
> new file mode 100644
> index 000000000000..50d60d120e2e
> --- /dev/null
> +++ b/tools/testing/selftests/kvm/aarch64/id_reg_test.c
> @@ -0,0 +1,1128 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +
> +#define _GNU_SOURCE
> +
> +#include <stdlib.h>
> +#include <time.h>
> +#include <pthread.h>
> +#include <linux/kvm.h>
> +#include <linux/sizes.h>
> +
> +#include "kvm_util.h"
> +#include "processor.h"
> +#include "vgic.h"
> +
> +/*
> + * id_reg_test.c - Tests reading/writing the aarch64's ID registers
> + *
> + * The test validates KVM_SET_ONE_REG/KVM_GET_ONE_REG ioctl for ID
> + * registers as well as reading ID register from the guest works fine.
> + */
> +
> +/* Reserved ID registers */
> +#define	SYS_ID_REG_3_3_EL1		sys_reg(3, 0, 0, 3, 3)
> +#define	SYS_ID_REG_3_7_EL1		sys_reg(3, 0, 0, 3, 7)
> +
> +#define	SYS_ID_REG_4_2_EL1		sys_reg(3, 0, 0, 4, 2)
> +#define	SYS_ID_REG_4_3_EL1		sys_reg(3, 0, 0, 4, 3)
> +#define	SYS_ID_REG_4_5_EL1		sys_reg(3, 0, 0, 4, 5)
> +#define	SYS_ID_REG_4_6_EL1		sys_reg(3, 0, 0, 4, 6)
> +#define	SYS_ID_REG_4_7_EL1		sys_reg(3, 0, 0, 4, 7)
> +
> +#define	SYS_ID_REG_5_2_EL1		sys_reg(3, 0, 0, 5, 2)
> +#define	SYS_ID_REG_5_3_EL1		sys_reg(3, 0, 0, 5, 3)
> +#define	SYS_ID_REG_5_6_EL1		sys_reg(3, 0, 0, 5, 6)
> +#define	SYS_ID_REG_5_7_EL1		sys_reg(3, 0, 0, 5, 7)
> +
> +#define	SYS_ID_REG_6_2_EL1		sys_reg(3, 0, 0, 6, 2)
> +#define	SYS_ID_REG_6_3_EL1		sys_reg(3, 0, 0, 6, 3)
> +#define	SYS_ID_REG_6_4_EL1		sys_reg(3, 0, 0, 6, 4)
> +#define	SYS_ID_REG_6_5_EL1		sys_reg(3, 0, 0, 6, 5)
> +#define	SYS_ID_REG_6_6_EL1		sys_reg(3, 0, 0, 6, 6)
> +#define	SYS_ID_REG_6_7_EL1		sys_reg(3, 0, 0, 6, 7)
> +
> +#define	SYS_ID_REG_7_3_EL1		sys_reg(3, 0, 0, 7, 3)
> +#define	SYS_ID_REG_7_4_EL1		sys_reg(3, 0, 0, 7, 4)
> +#define	SYS_ID_REG_7_5_EL1		sys_reg(3, 0, 0, 7, 5)
> +#define	SYS_ID_REG_7_6_EL1		sys_reg(3, 0, 0, 7, 6)
> +#define	SYS_ID_REG_7_7_EL1		sys_reg(3, 0, 0, 7, 7)
> +
> +#define	READ_ID_REG_FN(name)	read_## name ## _EL1
> +
> +#define	DEFINE_READ_SYS_REG(reg_name)			\
> +uint64_t read_##reg_name(void)				\
> +{							\
> +	return read_sysreg_s(SYS_##reg_name);		\
> +}
> +
> +#define DEFINE_READ_ID_REG(name)	\
> +	DEFINE_READ_SYS_REG(name ## _EL1)
> +
> +#define	__ID_REG(reg_name)		\
> +	.name = #reg_name,		\
> +	.id = SYS_## reg_name ##_EL1,	\
> +	.read_reg = READ_ID_REG_FN(reg_name),
> +
> +#define	ID_REG_ENT(reg_name)	\
> +	[ID_IDX(reg_name)] = { __ID_REG(reg_name) }
> +
> +/* Functions to read each ID register */
> +/* CRm=1 */
> +DEFINE_READ_ID_REG(ID_PFR0)
> +DEFINE_READ_ID_REG(ID_PFR1)
> +DEFINE_READ_ID_REG(ID_DFR0)
> +DEFINE_READ_ID_REG(ID_AFR0)
> +DEFINE_READ_ID_REG(ID_MMFR0)
> +DEFINE_READ_ID_REG(ID_MMFR1)
> +DEFINE_READ_ID_REG(ID_MMFR2)
> +DEFINE_READ_ID_REG(ID_MMFR3)
> +
> +/* CRm=2 */
> +DEFINE_READ_ID_REG(ID_ISAR0)
> +DEFINE_READ_ID_REG(ID_ISAR1)
> +DEFINE_READ_ID_REG(ID_ISAR2)
> +DEFINE_READ_ID_REG(ID_ISAR3)
> +DEFINE_READ_ID_REG(ID_ISAR4)
> +DEFINE_READ_ID_REG(ID_ISAR5)
> +DEFINE_READ_ID_REG(ID_MMFR4)
> +DEFINE_READ_ID_REG(ID_ISAR6)
> +
> +/* CRm=3 */
> +DEFINE_READ_ID_REG(MVFR0)
> +DEFINE_READ_ID_REG(MVFR1)
> +DEFINE_READ_ID_REG(MVFR2)
> +DEFINE_READ_ID_REG(ID_REG_3_3)
> +DEFINE_READ_ID_REG(ID_PFR2)
> +DEFINE_READ_ID_REG(ID_DFR1)
> +DEFINE_READ_ID_REG(ID_MMFR5)
> +DEFINE_READ_ID_REG(ID_REG_3_7)
> +
> +/* CRm=4 */
> +DEFINE_READ_ID_REG(ID_AA64PFR0)
> +DEFINE_READ_ID_REG(ID_AA64PFR1)
> +DEFINE_READ_ID_REG(ID_REG_4_2)
> +DEFINE_READ_ID_REG(ID_REG_4_3)
> +DEFINE_READ_ID_REG(ID_AA64ZFR0)
> +DEFINE_READ_ID_REG(ID_REG_4_5)
> +DEFINE_READ_ID_REG(ID_REG_4_6)
> +DEFINE_READ_ID_REG(ID_REG_4_7)
> +
> +/* CRm=5 */
> +DEFINE_READ_ID_REG(ID_AA64DFR0)
> +DEFINE_READ_ID_REG(ID_AA64DFR1)
> +DEFINE_READ_ID_REG(ID_REG_5_2)
> +DEFINE_READ_ID_REG(ID_REG_5_3)
> +DEFINE_READ_ID_REG(ID_AA64AFR0)
> +DEFINE_READ_ID_REG(ID_AA64AFR1)
> +DEFINE_READ_ID_REG(ID_REG_5_6)
> +DEFINE_READ_ID_REG(ID_REG_5_7)
> +
> +/* CRm=6 */
> +DEFINE_READ_ID_REG(ID_AA64ISAR0)
> +DEFINE_READ_ID_REG(ID_AA64ISAR1)
> +DEFINE_READ_ID_REG(ID_REG_6_2)
> +DEFINE_READ_ID_REG(ID_REG_6_3)
> +DEFINE_READ_ID_REG(ID_REG_6_4)
> +DEFINE_READ_ID_REG(ID_REG_6_5)
> +DEFINE_READ_ID_REG(ID_REG_6_6)
> +DEFINE_READ_ID_REG(ID_REG_6_7)
> +
> +/* CRm=7 */
> +DEFINE_READ_ID_REG(ID_AA64MMFR0)
> +DEFINE_READ_ID_REG(ID_AA64MMFR1)
> +DEFINE_READ_ID_REG(ID_AA64MMFR2)
> +DEFINE_READ_ID_REG(ID_REG_7_3)
> +DEFINE_READ_ID_REG(ID_REG_7_4)
> +DEFINE_READ_ID_REG(ID_REG_7_5)
> +DEFINE_READ_ID_REG(ID_REG_7_6)
> +DEFINE_READ_ID_REG(ID_REG_7_7)
> +
> +#define	ID_IDX(name)	REG_IDX_## name
> +
> +enum id_reg_idx {
> +	/* CRm=1 */
> +	ID_IDX(ID_PFR0) = 0,
> +	ID_IDX(ID_PFR1),
> +	ID_IDX(ID_DFR0),
> +	ID_IDX(ID_AFR0),
> +	ID_IDX(ID_MMFR0),
> +	ID_IDX(ID_MMFR1),
> +	ID_IDX(ID_MMFR2),
> +	ID_IDX(ID_MMFR3),
> +
> +	/* CRm=2 */
> +	ID_IDX(ID_ISAR0),
> +	ID_IDX(ID_ISAR1),
> +	ID_IDX(ID_ISAR2),
> +	ID_IDX(ID_ISAR3),
> +	ID_IDX(ID_ISAR4),
> +	ID_IDX(ID_ISAR5),
> +	ID_IDX(ID_MMFR4),
> +	ID_IDX(ID_ISAR6),
> +
> +	/* CRm=3 */
> +	ID_IDX(MVFR0),
> +	ID_IDX(MVFR1),
> +	ID_IDX(MVFR2),
> +	ID_IDX(ID_REG_3_3),
> +	ID_IDX(ID_PFR2),
> +	ID_IDX(ID_DFR1),
> +	ID_IDX(ID_MMFR5),
> +	ID_IDX(ID_REG_3_7),
> +
> +	/* CRm=4 */
> +	ID_IDX(ID_AA64PFR0),
> +	ID_IDX(ID_AA64PFR1),
> +	ID_IDX(ID_REG_4_2),
> +	ID_IDX(ID_REG_4_3),
> +	ID_IDX(ID_AA64ZFR0),
> +	ID_IDX(ID_REG_4_5),
> +	ID_IDX(ID_REG_4_6),
> +	ID_IDX(ID_REG_4_7),
> +
> +	/* CRm=5 */
> +	ID_IDX(ID_AA64DFR0),
> +	ID_IDX(ID_AA64DFR1),
> +	ID_IDX(ID_REG_5_2),
> +	ID_IDX(ID_REG_5_3),
> +	ID_IDX(ID_AA64AFR0),
> +	ID_IDX(ID_AA64AFR1),
> +	ID_IDX(ID_REG_5_6),
> +	ID_IDX(ID_REG_5_7),
> +
> +	/* CRm=6 */
> +	ID_IDX(ID_AA64ISAR0),
> +	ID_IDX(ID_AA64ISAR1),
> +	ID_IDX(ID_REG_6_2),
> +	ID_IDX(ID_REG_6_3),
> +	ID_IDX(ID_REG_6_4),
> +	ID_IDX(ID_REG_6_5),
> +	ID_IDX(ID_REG_6_6),
> +	ID_IDX(ID_REG_6_7),
> +
> +	/* CRm=7 */
> +	ID_IDX(ID_AA64MMFR0),
> +	ID_IDX(ID_AA64MMFR1),
> +	ID_IDX(ID_AA64MMFR2),
> +	ID_IDX(ID_REG_7_3),
> +	ID_IDX(ID_REG_7_4),
> +	ID_IDX(ID_REG_7_5),
> +	ID_IDX(ID_REG_7_6),
> +	ID_IDX(ID_REG_7_7),
> +};
> +
> +struct id_reg_test_info {
> +	char		*name;
> +	uint32_t	id;
> +	bool		can_clear;
> +	uint64_t	fixed_mask;
> +	uint64_t	org_val;
nit: original_val? or default_val?
> +	uint64_t	user_val;
> +	uint64_t	(*read_reg)(void);
> +};
> +
> +#define	ID_REG_INFO(name)	(&id_reg_list[ID_IDX(name)])
> +static struct id_reg_test_info id_reg_list[] = {
> +	/* CRm=1 */
> +	ID_REG_ENT(ID_PFR0),
> +	ID_REG_ENT(ID_PFR1),
> +	ID_REG_ENT(ID_DFR0),
> +	ID_REG_ENT(ID_AFR0),
> +	ID_REG_ENT(ID_MMFR0),
> +	ID_REG_ENT(ID_MMFR1),
> +	ID_REG_ENT(ID_MMFR2),
> +	ID_REG_ENT(ID_MMFR3),
> +
> +	/* CRm=2 */
> +	ID_REG_ENT(ID_ISAR0),
> +	ID_REG_ENT(ID_ISAR1),
> +	ID_REG_ENT(ID_ISAR2),
> +	ID_REG_ENT(ID_ISAR3),
> +	ID_REG_ENT(ID_ISAR4),
> +	ID_REG_ENT(ID_ISAR5),
> +	ID_REG_ENT(ID_MMFR4),
> +	ID_REG_ENT(ID_ISAR6),
> +
> +	/* CRm=3 */
> +	ID_REG_ENT(MVFR0),
> +	ID_REG_ENT(MVFR1),
> +	ID_REG_ENT(MVFR2),
> +	ID_REG_ENT(ID_REG_3_3),
> +	ID_REG_ENT(ID_PFR2),
> +	ID_REG_ENT(ID_DFR1),
> +	ID_REG_ENT(ID_MMFR5),
> +	ID_REG_ENT(ID_REG_3_7),
> +
> +	/* CRm=4 */
> +	ID_REG_ENT(ID_AA64PFR0),
> +	ID_REG_ENT(ID_AA64PFR1),
> +	ID_REG_ENT(ID_REG_4_2),
> +	ID_REG_ENT(ID_REG_4_3),
> +	ID_REG_ENT(ID_AA64ZFR0),
> +	ID_REG_ENT(ID_REG_4_5),
> +	ID_REG_ENT(ID_REG_4_6),
> +	ID_REG_ENT(ID_REG_4_7),
> +
> +	/* CRm=5 */
> +	ID_REG_ENT(ID_AA64DFR0),
> +	ID_REG_ENT(ID_AA64DFR1),
> +	ID_REG_ENT(ID_REG_5_2),
> +	ID_REG_ENT(ID_REG_5_3),
> +	ID_REG_ENT(ID_AA64AFR0),
> +	ID_REG_ENT(ID_AA64AFR1),
> +	ID_REG_ENT(ID_REG_5_6),
> +	ID_REG_ENT(ID_REG_5_7),
> +
> +	/* CRm=6 */
> +	ID_REG_ENT(ID_AA64ISAR0),
> +	ID_REG_ENT(ID_AA64ISAR1),
> +	ID_REG_ENT(ID_REG_6_2),
> +	ID_REG_ENT(ID_REG_6_3),
> +	ID_REG_ENT(ID_REG_6_4),
> +	ID_REG_ENT(ID_REG_6_5),
> +	ID_REG_ENT(ID_REG_6_6),
> +	ID_REG_ENT(ID_REG_6_7),
> +
> +	/* CRm=7 */
> +	ID_REG_ENT(ID_AA64MMFR0),
> +	ID_REG_ENT(ID_AA64MMFR1),
> +	ID_REG_ENT(ID_AA64MMFR2),
> +	ID_REG_ENT(ID_REG_7_3),
> +	ID_REG_ENT(ID_REG_7_4),
> +	ID_REG_ENT(ID_REG_7_5),
> +	ID_REG_ENT(ID_REG_7_6),
> +	ID_REG_ENT(ID_REG_7_7),
> +};
> +
> +/* Utilities to get a feature field from ID register value */
> +static inline int
> +cpuid_signed_field_width(uint64_t id_val, int field, int width)
> +{
> +	return (s64)(id_val << (64 - width - field)) >> (64 - width);
> +}
> +
> +static unsigned int
> +cpuid_unsigned_field_width(uint64_t id_val, int field, int width)
> +{
> +	return (uint64_t)(id_val << (64 - width - field)) >> (64 - width);
> +}
> +
> +static inline int __attribute_const__
> +cpuid_extract_field_width(uint64_t id_val, int field, int width, bool sign)
> +{
> +	return (sign) ? cpuid_signed_field_width(id_val, field, width) :
> +			cpuid_unsigned_field_width(id_val, field, width);
> +}
> +
> +#define	GET_ID_FIELD(regval, shift, is_signed)	\
> +	cpuid_extract_field_width(regval, shift, 4, is_signed)
> +
> +#define	GET_ID_UFIELD(regval, shift)	\
> +	cpuid_unsigned_field_width(regval, shift, 4)
> +
> +#define	UPDATE_ID_UFIELD(regval, shift, fval)	\
> +	(((regval) & ~(0xfULL << (shift))) |	\
> +	 (((uint64_t)((fval) & 0xf)) << (shift)))
> +
> +void test_pmu_init(struct kvm_vm *vm, uint32_t vcpu)
I would remove the test_ prefix as it does not test anything but
enhances the initialization instead
> +{
> +	struct kvm_device_attr attr = {
> +		.group = KVM_ARM_VCPU_PMU_V3_CTRL,
> +		.attr = KVM_ARM_VCPU_PMU_V3_INIT,
> +	};
> +	vcpu_ioctl(vm, vcpu, KVM_SET_DEVICE_ATTR, &attr);
> +}
> +
> +void test_sve_init(struct kvm_vm *vm, uint32_t vcpu)
> +{
> +	int feature = KVM_ARM_VCPU_SVE;
> +
> +	vcpu_ioctl(vm, vcpu, KVM_ARM_VCPU_FINALIZE, &feature);
> +}
> +
> +#define GICD_BASE_GPA			0x8000000ULL
> +#define GICR_BASE_GPA			0x80A0000ULL
> +
> +void test_vgic_init(struct kvm_vm *vm, uint32_t vcpu)
> +{
> +	/* We jsut need to configure gic v3 (we don't use it though) */
> +	vgic_v3_setup(vm, 1, GICD_BASE_GPA, GICR_BASE_GPA);
> +}
> +
> +#define	MAX_CAPS	2
> +struct feature_test_info {
> +	char	*name;	/* Feature Name (Debug information) */
> +	struct id_reg_test_info	*sreg;	/* ID register for the feature */
ID register comprising the feature?
> +	int	shift;	/* Field of the ID register for the feature */
I guess you mean feature field bit shift
> +	int	min;	/* Min value to indicate the feature */
Min value that can be assigned to the feature field?
> +	bool	is_sign;	/* Is the field signed or unsigned ? */
> +	int	ncaps;		/* Number of valid Capabilities in caps[] */
> +	long	caps[MAX_CAPS];	/* Capabilities to indicate this feature */
I suggest: KVM_CAP_* capabilities requested to test this feature
> +	/* struct kvm_enable_cap to use the capability if needed */
> +	struct kvm_enable_cap	*opt_in_cap;
> +	bool	run_test;	/* Does guest run test for this feature ? */
s/run_test/guest_run?
> +	/* Initialization function for the feature as needed */
extra init sequence needed besides KVM CAP setting?
> +	void	(*init_feature)(struct kvm_vm *vm, uint32_t vcpuid);
> +	/* struct kvm_vcpu_init to opt-in the feature if needed */
> +	struct kvm_vcpu_init	*vcpu_init;
> +};
> +
> +/* Test for optin CPU features */
opt-in?
> +static struct feature_test_info feature_test_info_table[] = {
> +	{
> +		.name = "SVE",
> +		.sreg = ID_REG_INFO(ID_AA64PFR0),
> +		.shift = ID_AA64PFR0_SVE_SHIFT,
> +		.min = 1,
> +		.caps = {KVM_CAP_ARM_SVE},
> +		.ncaps = 1,
> +		.init_feature = test_sve_init,
> +		.vcpu_init = &(struct kvm_vcpu_init) {
> +			.features = {1ULL << KVM_ARM_VCPU_SVE},
> +		},
> +	},
> +	{
> +		.name = "GIC",
> +		.sreg = ID_REG_INFO(ID_AA64PFR0),
> +		.shift = ID_AA64PFR0_GIC_SHIFT,
> +		.min = 1,
> +		.caps = {KVM_CAP_IRQCHIP},
> +		.ncaps = 1,
> +		.init_feature = test_vgic_init,
> +	},
> +	{
> +		.name = "MTE",
> +		.sreg = ID_REG_INFO(ID_AA64PFR1),
> +		.shift = ID_AA64PFR1_MTE_SHIFT,
> +		.min = 2,
> +		.caps = {KVM_CAP_ARM_MTE},
> +		.ncaps = 1,
> +		.opt_in_cap = &(struct kvm_enable_cap) {
> +				.cap = KVM_CAP_ARM_MTE,
> +		},
> +	},
> +	{
> +		.name = "PMUV3",
> +		.sreg = ID_REG_INFO(ID_AA64DFR0),
> +		.shift = ID_AA64DFR0_PMUVER_SHIFT,
> +		.min = 1,
> +		.init_feature = test_pmu_init,
> +		.caps = {KVM_CAP_ARM_PMU_V3},
> +		.ncaps = 1,
> +		.vcpu_init = &(struct kvm_vcpu_init) {
> +			.features = {1ULL << KVM_ARM_VCPU_PMU_V3},
> +		},
> +	},
> +	{
> +		.name = "PERFMON",
> +		.sreg = ID_REG_INFO(ID_DFR0),
> +		.shift = ID_DFR0_PERFMON_SHIFT,
> +		.min = 3,
> +		.init_feature = test_pmu_init,
> +		.caps = {KVM_CAP_ARM_PMU_V3},
> +		.ncaps = 1,
> +		.vcpu_init = &(struct kvm_vcpu_init) {
> +			.features = {1ULL << KVM_ARM_VCPU_PMU_V3},
> +		},
> +	},
> +};
> +
> +static int walk_id_reg_list(int (*fn)(struct id_reg_test_info *sreg, void *arg),
> +			    void *arg)
> +{
> +	int ret = 0, i;
> +
> +	for (i = 0; i < ARRAY_SIZE(id_reg_list); i++) {
> +		ret = fn(&id_reg_list[i], arg);
> +		if (ret)
> +			break;
none of your fn() function does return something != 0
> +	}
> +	return ret;
> +}
> +
> +static int guest_code_id_reg_check_one(struct id_reg_test_info *sreg, void *arg)
> +{
> +	uint64_t val = sreg->read_reg();
> +
> +	GUEST_ASSERT_2(val == sreg->user_val, sreg->name, sreg->user_val);
> +	return 0;
> +}
> +
> +static void guest_code_id_reg_check_all(uint32_t cpu)
> +{
> +	walk_id_reg_list(guest_code_id_reg_check_one, NULL);
> +	GUEST_DONE();
> +}
> +
> +static void guest_code_do_nothing(uint32_t cpu)
> +{
> +	GUEST_DONE();
> +}
> +
> +static void guest_code_feature_check(uint32_t cpu)
> +{
> +	int i;
> +	struct feature_test_info *finfo;
> +
> +	for (i = 0; i < ARRAY_SIZE(feature_test_info_table); i++) {
> +		finfo = &feature_test_info_table[i];
> +		if (finfo->run_test)
> +			guest_code_id_reg_check_one(finfo->sreg, NULL);
> +	}
> +
> +	GUEST_DONE();
> +}
> +
> +static void guest_code_ptrauth_check(uint32_t cpuid)
> +{
> +	struct id_reg_test_info *sreg = ID_REG_INFO(ID_AA64ISAR1);
> +	uint64_t val = sreg->read_reg();
> +
> +	GUEST_ASSERT_2(val == sreg->user_val, "PTRAUTH", val);
> +	GUEST_DONE();
> +}
> +
> +static int reset_id_reg_info_one(struct id_reg_test_info *sreg, void *arg)
reset_id_reg_user_val_one()?
> +{
> +	sreg->user_val = sreg->org_val;
> +	return 0;
> +}
> +
> +static void reset_id_reg_info(void)
reset_id_reg_user_val()?
> +{
> +	walk_id_reg_list(reset_id_reg_info_one, NULL);
> +}
> +
> +static struct kvm_vm *test_vm_create_cap(uint32_t nvcpus,
> +		void (*guest_code)(uint32_t), struct kvm_vcpu_init *init,
> +		struct kvm_enable_cap *cap)
> +{
> +	struct kvm_vm *vm;
> +	uint32_t cpuid;
> +	uint64_t mem_pages;
> +
> +	mem_pages = DEFAULT_GUEST_PHY_PAGES + DEFAULT_STACK_PGS * nvcpus;
> +	mem_pages += mem_pages / (PTES_PER_MIN_PAGE * 2);
> +	mem_pages = vm_adjust_num_guest_pages(VM_MODE_DEFAULT, mem_pages);


> +
> +	vm = vm_create(VM_MODE_DEFAULT,
> +		DEFAULT_GUEST_PHY_PAGES + (DEFAULT_STACK_PGS * nvcpus),
> +		O_RDWR);
mem_pages must be used instead

augere@virtlab-arm04:~/UPSTREAM/ML/tools/testing/selftests/kvm#
./aarch64/id_reg_test
==== Test Assertion Failure ====
  lib/kvm_util.c:825: vm_adjust_num_guest_pages(vm->mode, npages) == npages
  pid=11439 tid=11439 errno=0 - Success
     1	0x00000000004068cb: vm_userspace_mem_region_add at kvm_util.c:823
     2	0x00000000004071af: vm_create at kvm_util.c:319
     3	0x0000000000401afb: test_vm_create_cap at id_reg_test.c:508
     4	0x00000000004014a3: test_vm_create at id_reg_test.c:541
     5	 (inlined by) init_id_reg_info at id_reg_test.c:1110
     6	 (inlined by) main at id_reg_test.c:1125
     7	0x000003ffa7220de3: ?? ??:0
     8	0x00000000004015eb: _start at :?
  Number of guest pages is not compatible with the host. Try npages=528


Don't you want to check the cap in a first place using kvm_check_cap and
cap->cap
> +	if (cap)
> +		vm_enable_cap(vm, cap);
> +
> +	kvm_vm_elf_load(vm, program_invocation_name);
> +
> +	if (init && init->target == -1) {
> +		struct kvm_vcpu_init preferred;
> +
> +		vm_ioctl(vm, KVM_ARM_PREFERRED_TARGET, &preferred);
> +		init->target = preferred.target;
> +	}
> +
> +	vm_init_descriptor_tables(vm);
> +	for (cpuid = 0; cpuid < nvcpus; cpuid++) {
> +		if (init)
> +			aarch64_vcpu_add_default(vm, cpuid, init, guest_code);
> +		else
> +			vm_vcpu_add_default(vm, cpuid, guest_code);
nit: vm_vcpu_add_default calls aarch64_vcpu_add_default(vm, vcpuid,
NULL, guest_code) so you can unconditionnaly call
aarch64_vcpu_add_default(vm, cpuid, init, guest_code)
> +
> +		vcpu_init_descriptor_tables(vm, cpuid);
> +	}
> +
> +	ucall_init(vm, NULL);
> +	return vm;
> +}
> +
> +static struct kvm_vm *test_vm_create(uint32_t nvcpus,
> +				     void (*guest_code)(uint32_t),
> +				     struct kvm_vcpu_init *init)
> +{
> +	return test_vm_create_cap(nvcpus, guest_code, init, NULL);
> +}
nit: not sure test_vm_create is needed. By the way it is already called
with init = NULL so we can call test_vm_create_cap with 2 NULL args
> +
> +static void test_vm_free(struct kvm_vm *vm)
> +{
> +	ucall_uninit(vm);
> +	kvm_vm_free(vm);
> +}
> +
> +#define	TEST_RUN(vm, cpu)	\
> +	(test_vcpu_run(__func__, __LINE__, vm, cpu, true))
> +
> +#define	TEST_RUN_NO_SYNC_DATA(vm, cpu)	\
> +	(test_vcpu_run(__func__, __LINE__, vm, cpu, false))
> +
> +static int test_vcpu_run(const char *test_name, int line,
> +			 struct kvm_vm *vm, uint32_t vcpuid, bool sync_data)
> +{
> +	struct ucall uc;
> +	int ret;
> +
> +	if (sync_data) {
> +		sync_global_to_guest(vm, id_reg_list);
> +		sync_global_to_guest(vm, feature_test_info_table);
> +	}
> +
> +	vcpu_args_set(vm, vcpuid, 1, vcpuid);
> +
> +	ret = _vcpu_run(vm, vcpuid);
> +	if (ret) {
> +		ret = errno;
> +		goto sync_exit;
> +	}
> +
> +	switch (get_ucall(vm, vcpuid, &uc)) {
> +	case UCALL_SYNC:
> +	case UCALL_DONE:
> +		ret = 0;
> +		break;
> +	case UCALL_ABORT:
> +		TEST_FAIL(
> +		    "%s (%s) at line %d (user %s at line %d), args[3]=0x%lx",
> +		    (char *)uc.args[0], (char *)uc.args[2], (int)uc.args[1],
> +		    test_name, line, uc.args[3]);
> +		break;
> +	default:
> +		TEST_FAIL("Unexpected guest exit\n");
> +	}
> +
> +sync_exit:
> +	if (sync_data) {
> +		sync_global_from_guest(vm, id_reg_list);
> +		sync_global_from_guest(vm, feature_test_info_table);
> +	}
> +	return ret;
> +}
> +
> +static int set_id_regs_after_run_test_one(struct id_reg_test_info *sreg,
> +					  void *arg)
> +{
> +	struct kvm_vm *vm = arg;
> +	struct kvm_one_reg one_reg;
> +	uint32_t vcpuid = 0;
> +	uint64_t reg_val;
> +	int ret;
> +
> +	one_reg.addr = (uint64_t)&reg_val;
> +	one_reg.id = KVM_ARM64_SYS_REG(sreg->id);
> +
> +	vcpu_ioctl(vm, vcpuid, KVM_GET_ONE_REG, &one_reg);
> +	if ((reg_val != 0) && (sreg->can_clear)) {
> +		reg_val = 0;
> +		ret = _vcpu_ioctl(vm, vcpuid, KVM_SET_ONE_REG, &one_reg);
> +		TEST_ASSERT(ret && errno == EINVAL,
> +			    "Changing ID reg value should fail\n");
> +	}
> +
> +	vcpu_ioctl(vm, vcpuid, KVM_GET_ONE_REG, &one_reg);> +	ret = _vcpu_ioctl(vm, vcpuid, KVM_SET_ONE_REG, &one_reg);
> +	TEST_ASSERT(ret == 0, "Setting the same ID reg value should work\n");
> +
> +	return 0;
> +}
> +
> +static int set_id_regs_test_one(struct id_reg_test_info *sreg, void *arg)
if it is a test use test_ prefix
> +{
> +	struct kvm_vm *vm = arg;
> +	struct kvm_one_reg one_reg;
> +	uint32_t vcpuid = 0;
> +	uint64_t reg_val;
> +
> +	one_reg.addr = (uint64_t)&reg_val;
> +	reset_id_reg_info();
> +
> +	one_reg.id = KVM_ARM64_SYS_REG(sreg->id);
> +	if (sreg->can_clear) {
> +		/* Change the register to 0 when possible */
> +		reg_val = 0;
> +		vcpu_ioctl(vm, vcpuid, KVM_SET_ONE_REG, &one_reg);
> +		vcpu_ioctl(vm, vcpuid, KVM_GET_ONE_REG, &one_reg);
> +		TEST_ASSERT(reg_val == 0,
> +		    "GET(%s) didn't return 0 but 0x%lx", sreg->name, reg_val);
> +	}
> +
> +	/* Check if we can restore the initial value */
> +	reg_val = sreg->org_val;
> +	vcpu_ioctl(vm, vcpuid, KVM_SET_ONE_REG, &one_reg);
> +	vcpu_ioctl(vm, vcpuid, KVM_GET_ONE_REG, &one_reg);
> +	TEST_ASSERT(reg_val == sreg->org_val,
> +		    "GET(%s) didn't return 0x%lx but 0x%lx",
> +		    sreg->name, sreg->org_val, reg_val);
> +	sreg->user_val = sreg->org_val;
> +	return 0;
> +}
> +
> +static void set_id_regs_test(void)
if it is a test use test_ prefix
> +{
> +	struct kvm_vm *vm;
> +	int ret;
> +
> +	reset_id_reg_info();
> +	vm = test_vm_create(1, guest_code_id_reg_check_all, NULL);
add test_vm_free()
> +
> +	ret = walk_id_reg_list(set_id_regs_test_one, vm);
> +	assert(!ret);
> +
> +	ret = TEST_RUN(vm, 0);
> +	TEST_ASSERT(!ret, "%s TEST_RUN failed, ret=0x%x", __func__, ret);
> +
> +	ret = walk_id_reg_list(set_id_regs_after_run_test_one, vm);
> +	assert(!ret);
> +}
> +
> +static bool caps_are_supported(long *caps, int ncaps)
> +{
> +	int i;
> +
> +	for (i = 0; i < ncaps; i++) {
> +		if (kvm_check_cap(caps[i]) <= 0)
> +			return false;
> +	}
> +	return true;
> +}
> +
> +static void test_feature_ptrauth(void)
> +{
> +	struct kvm_one_reg one_reg;
> +	struct kvm_vcpu_init init;
> +	struct kvm_vm *vm = NULL;
> +	struct id_reg_test_info *sreg = ID_REG_INFO(ID_AA64ISAR1);
> +	uint32_t vcpu = 0;
> +	int64_t rval;
> +	int ret;
> +	int apa, api, gpa, gpi;
> +	char *name = "PTRAUTH";
> +	long caps[2] = {KVM_CAP_ARM_PTRAUTH_ADDRESS,
> +			KVM_CAP_ARM_PTRAUTH_GENERIC};
> +
> +	reset_id_reg_info();
> +	one_reg.addr = (uint64_t)&rval;
> +	one_reg.id = KVM_ARM64_SYS_REG(sreg->id);
> +
> +	if (caps_are_supported(caps, 2)) {
> +
> +		/* Test with feature enabled */
> +		memset(&init, 0, sizeof(init));
> +		init.target = -1;
> +		init.features[0] = (1ULL << KVM_ARM_VCPU_PTRAUTH_ADDRESS |
> +				    1ULL << KVM_ARM_VCPU_PTRAUTH_GENERIC);
> +		vm = test_vm_create_cap(1, guest_code_ptrauth_check, &init,
> +					NULL);
> +		vcpu_ioctl(vm, vcpu, KVM_GET_ONE_REG, &one_reg);
> +		apa = GET_ID_UFIELD(rval, ID_AA64ISAR1_APA_SHIFT);
> +		api = GET_ID_UFIELD(rval, ID_AA64ISAR1_API_SHIFT);
> +		gpa = GET_ID_UFIELD(rval, ID_AA64ISAR1_GPA_SHIFT);
> +		gpi = GET_ID_UFIELD(rval, ID_AA64ISAR1_GPI_SHIFT);
> +
> +		TEST_ASSERT((apa > 0) || (api > 0),
> +			    "Either apa(0x%x) or api(0x%x) must be available",
> +			    apa, gpa);
> +		TEST_ASSERT((gpa > 0) || (gpi > 0),
> +			    "Either gpa(0x%x) or gpi(0x%x) must be available",
> +			    gpa, gpi);
> +
> +		TEST_ASSERT((apa > 0) ^ (api > 0),
> +			    "Both apa(0x%x) and api(0x%x) must not be available",
> +			    apa, api);
> +		TEST_ASSERT((gpa > 0) ^ (gpi > 0),
> +			    "Both gpa(0x%x) and gpi(0x%x) must not be available",
> +			    gpa, gpi);
> +
> +		sreg->user_val = rval;
> +
> +		pr_debug("%s: Test with %s enabled (%s: 0x%lx)\n",
> +			 __func__, name, sreg->name, sreg->user_val);
> +		ret = TEST_RUN(vm, vcpu);
> +		TEST_ASSERT(!ret, "%s:KVM_RUN failed with %s enabled",
> +			    __func__, name);
> +		test_vm_free(vm);
> +	}
> +
> +	/* Test with feature disabled */
> +	reset_id_reg_info();
> +
> +	vm = test_vm_create(1, guest_code_feature_check, NULL);
> +	vcpu_ioctl(vm, vcpu, KVM_GET_ONE_REG, &one_reg);
> +
> +	apa = GET_ID_UFIELD(rval, ID_AA64ISAR1_APA_SHIFT);
> +	api = GET_ID_UFIELD(rval, ID_AA64ISAR1_API_SHIFT);
> +	gpa = GET_ID_UFIELD(rval, ID_AA64ISAR1_GPA_SHIFT);
> +	gpi = GET_ID_UFIELD(rval, ID_AA64ISAR1_GPI_SHIFT);
> +	TEST_ASSERT(!apa && !api && !gpa && !gpi,
> +	    "apa(0x%x), api(0x%x), gpa(0x%x), gpi(0x%x) must be zero",
> +	    apa, api, gpa, gpi);
> +
> +	pr_debug("%s: Test with %s disabled (%s: 0x%lx)\n",
> +		 __func__, name, sreg->name, sreg->user_val);
> +
> +	ret = TEST_RUN(vm, vcpu);
> +	TEST_ASSERT(!ret, "%s TEST_RUN failed with %s enabled, ret=0x%x",
> +		    __func__, name, ret);
> +
> +	test_vm_free(vm);
> +}
> +
> +static bool feature_caps_are_available(struct feature_test_info *finfo)
> +{
> +	return ((finfo->ncaps > 0) &&
> +		caps_are_supported(finfo->caps, finfo->ncaps));
> +}
> +
comment with short explanation of what the test does
> +static void test_feature(struct feature_test_info *finfo)
> +{
> +	struct id_reg_test_info *sreg = finfo->sreg;
> +	struct kvm_one_reg one_reg;
> +	struct kvm_vcpu_init init, *initp = NULL;
> +	struct kvm_vm *vm = NULL;
> +	int64_t fval, reg_val;
> +	uint32_t vcpu = 0;
> +	bool is_sign = finfo->is_sign;
> +	int min = finfo->min;
> +	int shift = finfo->shift;
> +	int ret;
> +
> +	pr_debug("%s: %s (reg %s)\n", __func__, finfo->name, sreg->name);
> +
> +	reset_id_reg_info();
> +	finfo->run_test = 1;	/* Indicate that guest runs the test on it */
> +	one_reg.addr = (uint64_t)&reg_val;
> +	one_reg.id = KVM_ARM64_SYS_REG(sreg->id);
> +
> +	/* Test with feature enabled if the feature is available */
s/if the feature is available/if the feature is exposed in the default
ID_REG value and if the capabilities are supported at KVM level
> +	if ((GET_ID_FIELD(sreg->org_val, shift, is_sign) >= min) ||
> +	    feature_caps_are_available(finfo)) {
> +		if (finfo->vcpu_init) {
> +			/*
> +			 * Need to enable the feature via
> +			 * KVM_ARM_VCPU_INIT.
> +			 */
> +			memset(&init, 0, sizeof(init));
> +			init = *finfo->vcpu_init;
> +			init.target = -1;
> +			initp = &init;
> +		}
> +
> +		vm = test_vm_create_cap(1, guest_code_feature_check, initp,
> +					finfo->opt_in_cap);
> +		if (finfo->init_feature)
> +			/* Run any required extra process to use the feature */
> +			finfo->init_feature(vm, vcpu);
> +
> +		/* Check if the ID register value indicates the feature */
> +		vcpu_ioctl(vm, vcpu, KVM_GET_ONE_REG, &one_reg);
> +		fval = GET_ID_FIELD(reg_val, shift, is_sign);
> +		TEST_ASSERT(fval >= min, "%s field of %s is too small (%ld)",
> +			    finfo->name, sreg->name, fval);
> +		sreg->user_val = reg_val;
> +
> +		pr_debug("%s: Test with %s enabled (%s: 0x%lx)\n",
> +			 __func__, finfo->name, sreg->name, sreg->user_val);
> +
> +		ret = TEST_RUN(vm, vcpu);
> +		TEST_ASSERT(!ret, "%s:TEST_RUN failed with %s enabled",
> +			    __func__, finfo->name);
> +		test_vm_free(vm);
> +	}
> +
> +	/* Test with feature disabled */
> +	reset_id_reg_info();
> +
> +	vm = test_vm_create(1, guest_code_feature_check, NULL);
> +	vcpu_ioctl(vm, vcpu, KVM_GET_ONE_REG, &one_reg);
> +	fval = GET_ID_FIELD(reg_val, shift, is_sign);
> +	if (finfo->vcpu_init || finfo->opt_in_cap) {
> +		/*
> +		 * If the feature needs to be enabled with KVM_ARM_VCPU_INIT
> +		 * or opt-in capabilities, the default value of the ID register
> +		 * shouldn't indicate the feature.
> +		 */
> +		TEST_ASSERT(fval < min, "%s field of %s is too big (%ld)",
> +		    finfo->name, sreg->name, fval);
> +	} else {
> +		/* Update the relevant field to hide the feature. */
> +		fval = is_sign ? 0xf : 0x0;
> +		reg_val = UPDATE_ID_UFIELD(reg_val, shift, fval);
> +		ret = _vcpu_ioctl(vm, vcpu, KVM_SET_ONE_REG, &one_reg);
> +		TEST_ASSERT(ret == 0, "Disabling %s failed %d\n",
> +			    finfo->name, ret);
> +		sreg->user_val = reg_val;
> +	}
> +
> +	pr_debug("%s: Test with %s disabled (%s: 0x%lx)\n",
> +		 __func__, finfo->name, sreg->name, sreg->user_val);
> +
> +	ret = TEST_RUN(vm, vcpu);
> +	finfo->run_test = 0;
> +	test_vm_free(vm);
> +}
> +
> +static void test_feature_all(void)
> +{
> +	int i;
> +
> +	for (i = 0; i < ARRAY_SIZE(feature_test_info_table); i++)
> +		test_feature(&feature_test_info_table[i]);
> +}
> +
> +int test_set_reg(struct id_reg_test_info *sreg, uint64_t new_val,
> +		 bool guest_run)
> +{
> +	struct kvm_vm *vm;
> +	int ret;
> +	uint32_t vcpu = 0;
> +	uint64_t reg_val;
> +	struct kvm_one_reg one_reg;
> +
> +	reset_id_reg_info();
> +
> +	vm = test_vm_create(1, guest_code_id_reg_check_all, NULL);
> +	one_reg.id = KVM_ARM64_SYS_REG(sreg->id);
> +	one_reg.addr = (uint64_t)&reg_val;
> +
> +	reg_val = new_val;
> +	ret = _vcpu_ioctl(vm, vcpu, KVM_SET_ONE_REG, &one_reg);
> +	if (!guest_run)
> +		return ret;
> +
> +	TEST_ASSERT(!ret, "SET_REG(%s=0x%lx) failed, ret=0x%x",
> +		    sreg->name, new_val, ret);
> +	sreg->user_val = new_val;
> +	ret = TEST_RUN(vm, vcpu);
> +	test_vm_free(vm);
> +	return ret;
> +}
> +
> +int test_feature_frac_vm(struct id_reg_test_info *sreg, uint64_t new_val,
> +		      struct id_reg_test_info *frac_sreg, uint64_t frac_new_val)
> +{
> +	struct kvm_vm *vm;
> +	int ret;
> +	uint32_t vcpu = 0;
> +	uint64_t reg_val;
> +	struct kvm_one_reg one_reg;
> +
> +	reset_id_reg_info();
> +
> +	vm = test_vm_create(1, guest_code_id_reg_check_all, NULL);
> +
> +	/* Set feature reg field */
> +	one_reg.id = KVM_ARM64_SYS_REG(sreg->id);
> +	one_reg.addr = (uint64_t)&reg_val;
> +	reg_val = new_val;
> +	ret = _vcpu_ioctl(vm, vcpu, KVM_SET_ONE_REG, &one_reg);
> +	TEST_ASSERT(!ret, "SET_REG(%s=0x%lx) failed, ret=0x%x",
> +		    sreg->name, new_val, ret);
> +	sreg->user_val = new_val;
> +
> +	/* Set fractional reg field */
> +	one_reg.id = KVM_ARM64_SYS_REG(frac_sreg->id);
> +	one_reg.addr = (uint64_t)&reg_val;
> +	reg_val = frac_new_val;
> +	vcpu_ioctl(vm, vcpu, KVM_SET_ONE_REG, &one_reg);
> +	TEST_ASSERT(!ret, "SET_REG(%s=0x%lx) failed, ret=0x%x",
> +		    frac_sreg->name, frac_new_val, ret);
> +
> +	frac_sreg->user_val = frac_new_val;
> +	ret = TEST_RUN(vm, vcpu);
> +	test_vm_free(vm);
> +	return ret;
> +}
> +
> +struct frac_info {
> +	char	*name;
> +	struct id_reg_test_info *sreg;
> +	struct id_reg_test_info *frac_sreg;
> +	int	shift;
> +	int	frac_shift;
> +};
> +
> +struct frac_info frac_info_table[] = {
> +	{
> +		.name = "RAS",
> +		.sreg = ID_REG_INFO(ID_AA64PFR0),
> +		.shift = ID_AA64PFR0_RAS_SHIFT,
> +		.frac_sreg = ID_REG_INFO(ID_AA64PFR1),
> +		.frac_shift = ID_AA64PFR1_RASFRAC_SHIFT,
> +	},
> +	{
> +		.name = "MPAM",
> +		.sreg = ID_REG_INFO(ID_AA64PFR0),
> +		.shift = ID_AA64PFR0_MPAM_SHIFT,
> +		.frac_sreg = ID_REG_INFO(ID_AA64PFR1),
> +		.frac_shift = ID_AA64PFR1_MPAMFRAC_SHIFT,
> +	},
> +	{
> +		.name = "CSV2",
> +		.sreg = ID_REG_INFO(ID_AA64PFR0),
> +		.shift = ID_AA64PFR0_CSV2_SHIFT,
> +		.frac_sreg = ID_REG_INFO(ID_AA64PFR1),
> +		.frac_shift = ID_AA64PFR1_CSV2FRAC_SHIFT,
> +	},
> +};
> +
> +void test_feature_frac_one(struct frac_info *frac)
> +{
> +	uint64_t reg_val, org_fval, frac_reg_val, frac_org_fval;
> +	int ret, shift, frac_shift;
> +	struct id_reg_test_info *sreg, *frac_sreg;
> +
> +	reset_id_reg_info();
> +
> +	sreg = frac->sreg;
> +	shift = frac->shift;
> +	frac_sreg = frac->frac_sreg;
> +	frac_shift = frac->frac_shift;
> +
> +	pr_debug("%s(%s Frac) reg:%s(shift:%d) frac reg:%s(shift:%d)\n",
> +		__func__, frac->name, sreg->name, shift,
> +		frac_sreg->name, frac_shift);
> +
> +	frac_org_fval = GET_ID_UFIELD(frac_sreg->org_val, frac_shift);
> +	if (frac_org_fval > 0) {
> +		/* Test with smaller frac value */
> +		frac_reg_val = UPDATE_ID_UFIELD(frac_sreg->org_val,
> +						frac_shift, frac_org_fval - 1);
> +		ret = test_set_reg(frac_sreg, frac_reg_val, false);
> +		TEST_ASSERT(!ret, "SET smaller %s frac (val:%lx) failed(%d)",
> +			    frac->name, frac_reg_val, ret);
> +
> +		ret = test_feature_frac_vm(sreg, sreg->org_val,
> +					   frac_sreg, frac_reg_val);
> +		TEST_ASSERT(!ret, "Test smaller %s frac (val:%lx) failed(%d)",
> +			    frac->name, frac_reg_val, ret);
> +	}
> +
> +	reset_id_reg_info();
> +
> +	if (frac_org_fval != 0xf) {
> +		/* Test with larger frac value */
> +		frac_reg_val = UPDATE_ID_UFIELD(frac_sreg->org_val, frac_shift,
> +						frac_org_fval + 1);
> +
> +		/* Setting larger frac shouldn't fail (at ioctl) */
> +		ret = test_set_reg(frac_sreg, frac_reg_val, false);
> +		TEST_ASSERT(!ret,
> +			"SET larger %s frac (%s org:%lx, val:%lx) failed(%d)",
> +			frac->name, frac_sreg->name, frac_sreg->org_val,
> +			frac_reg_val, ret);
> +
> +		/* KVM_RUN with larger frac should fail */
> +		ret = test_feature_frac_vm(sreg, sreg->org_val,
> +					   frac_sreg, frac_reg_val);
> +		TEST_ASSERT(ret,
> +			"Test with larger %s frac (%s org:%lx, val:%lx) worked",
> +			frac->name, frac_sreg->name, frac_sreg->org_val,
> +			frac_reg_val);
> +	}
> +
> +	reset_id_reg_info();
> +
> +	org_fval = GET_ID_UFIELD(sreg->org_val, shift);
> +	if (org_fval == 0) {
> +		/* Setting larger val for the feature should fail */
> +		reg_val = UPDATE_ID_UFIELD(sreg->org_val, shift, org_fval + 1);
> +		ret = test_set_reg(sreg, reg_val, false);
> +		TEST_ASSERT(ret, "SET larger %s (val:%lx) worked",
> +			    frac->name, reg_val);
> +		return;
> +	}
> +
> +	/* Test with smaller feature value */
> +	reg_val = UPDATE_ID_UFIELD(sreg->org_val, shift, org_fval - 1);
> +	ret = test_set_reg(sreg, reg_val, false);
> +	TEST_ASSERT(!ret, "SET smaller %s (val:%lx) failed(%d)",
> +		    frac->name, reg_val, ret);
> +
> +	ret = test_feature_frac_vm(sreg, reg_val, frac_sreg, frac_sreg->org_val);
> +	TEST_ASSERT(!ret, "Test with smaller %s (val:%lx) failed(%d)",
> +		    frac->name, reg_val, ret);
> +
> +	if (frac_org_fval > 0) {
> +		/* Test with smaller feature and frac value */
> +		frac_reg_val = UPDATE_ID_UFIELD(frac_sreg->org_val,
> +						frac_shift, frac_org_fval - 1);
> +		ret = test_feature_frac_vm(sreg, reg_val, frac_sreg,
> +					   frac_reg_val);
> +		TEST_ASSERT(!ret,
> +		    "Test with smaller %s and frac (val:%lx) failed(%d)",
> +		    frac->name, reg_val, ret);
> +	}
> +
> +	if (frac_org_fval != 0xf) {
> +		/* Test with smaller feature and larger frac value */
> +		frac_reg_val = UPDATE_ID_UFIELD(frac_sreg->org_val,
> +						frac_shift, frac_org_fval + 1);
> +		ret = test_feature_frac_vm(sreg, reg_val, frac_sreg,
> +					   frac_reg_val);
> +		TEST_ASSERT(!ret,
> +		    "Test with smaller %s and larger frac (val:%lx) failed(%d)",
> +		    frac->name, reg_val, ret);
> +	}
> +}
> +
> +void test_feature_frac_all(void)
> +{
> +	int i;
> +
> +	for (i = 0; i < ARRAY_SIZE(frac_info_table); i++)
> +		test_feature_frac_one(&frac_info_table[i]);
> +}
> +
> +void run_test(void)
> +{
> +	set_id_regs_test();
> +	test_feature_all();
> +	test_feature_ptrauth();
> +	test_feature_frac_all();
> +}
> +
basic comment would be helpful: attempts to clear a given id_reg and
populate the id_reg with the original value, and can_clear flag?
> +static int init_id_reg_info_one(struct id_reg_test_info *sreg, void *arg)
> +{
> +	uint64_t reg_val;
> +	uint32_t vcpuid = 0;
> +	int ret;
> +	struct kvm_one_reg one_reg;
> +	struct kvm_vm *vm = arg;
> +
> +	one_reg.addr = (uint64_t)&reg_val;
> +	one_reg.id = KVM_ARM64_SYS_REG(sreg->id);
> +	vcpu_ioctl(vm, vcpuid, KVM_GET_ONE_REG, &one_reg);
> +	sreg->org_val = reg_val;
> +	sreg->user_val = reg_val;
nit: add a comment for user_val because it is not obvious why you set it
to reg_val.
> +	if (sreg->org_val) {
> +		reg_val = 0;
> +		ret = _vcpu_ioctl(vm, vcpuid, KVM_SET_ONE_REG, &one_reg);
> +		if (!ret)
> +			sreg->can_clear = true;
> +	}
> +
> +	pr_debug("%s (0x%x): 0x%lx%s\n", sreg->name, sreg->id,
> +		 sreg->org_val, sreg->can_clear ? ", can clear" : "");
> +
> +	return 0;
> +}
> +
add a comment? loop over the idreg list and populates each regid info
with the default, user and can_clear value
> +static void init_id_reg_info(void)
> +{
> +	struct kvm_vm *vm;
> +
> +	vm = test_vm_create(1, guest_code_do_nothing, NULL);
> +	walk_id_reg_list(init_id_reg_info_one, vm);
> +	test_vm_free(vm);
> +}
> +
> +int main(void)
> +{
> +
> +	setbuf(stdout, NULL);
> +
> +	if (kvm_check_cap(KVM_CAP_ARM_ID_REG_CONFIGURABLE) <= 0) {
> +		print_skip("KVM_CAP_ARM_ID_REG_CONFIGURABLE is not supported\n");
> +		exit(KSFT_SKIP);
> +	}
> +
> +	init_id_reg_info();
> +	run_test();
> +	return 0;
> +}
> 

After fixing the mem_pages stuff I get the following error on a cavium
machine.

augere@virtlab-arm04:~/UPSTREAM/ML/tools/testing/selftests/kvm#
./aarch64/id_reg_test
==== Test Assertion Failure ====
  aarch64/id_reg_test.c:814: fval >= min
  pid=11692 tid=11692 errno=4 - Interrupted system call
     1	0x00000000004028d3: test_feature at id_reg_test.c:813
     2	 (inlined by) test_feature_all at id_reg_test.c:863
     3	 (inlined by) run_test at id_reg_test.c:1073
     4	0x000000000040156f: main at id_reg_test.c:1124
     5	0x000003ffa9420de3: ?? ??:0
     6	0x00000000004015eb: _start at :?
  PERFMON field of ID_DFR0 is too small (0)

Fails on
test_feature: PERFMON (reg ID_DFR0)

I will do my utmost to further debug

Eric

Re: [RFC PATCH v3 03/29] KVM: arm64: Introduce struct id_reg_info

[Eric Auger]

Hi Reiji,

On 11/17/21 7:43 AM, Reiji Watanabe wrote:
> This patch lays the groundwork to make ID registers writable.
> 
> Introduce struct id_reg_info for an ID register to manage the
> register specific control of its value for the guest, and provide set
> of functions commonly used for ID registers to make them writable.
> 
> The id_reg_info is used to do register specific initialization,
> validation of the ID register and etc.  Not all ID registers must
> have the id_reg_info. ID registers that don't have the id_reg_info
> are handled in a common way that is applied to all ID registers.
> 
> At present, changing an ID register from userspace is allowed only
> if the ID register has the id_reg_info, but that will be changed
> by the following patches.
> 
> No ID register has the structure yet and the following patches
> will add the id_reg_info for some ID registers.
> 
> Signed-off-by: Reiji Watanabe <reijiw@google.com>
> ---
>  arch/arm64/include/asm/sysreg.h |   1 +
>  arch/arm64/kvm/sys_regs.c       | 226 ++++++++++++++++++++++++++++++--
>  2 files changed, 218 insertions(+), 9 deletions(-)
> 
> diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h
> index 16b3f1a1d468..597609f26331 100644
> --- a/arch/arm64/include/asm/sysreg.h
> +++ b/arch/arm64/include/asm/sysreg.h
> @@ -1197,6 +1197,7 @@
>  #define ICH_VTR_TDS_MASK	(1 << ICH_VTR_TDS_SHIFT)
>  
>  #define ARM64_FEATURE_FIELD_BITS	4
> +#define ARM64_FEATURE_FIELD_MASK	((1ull << ARM64_FEATURE_FIELD_BITS) - 1)
>  
>  /* Create a mask for the feature bits of the specified feature. */
>  #define ARM64_FEATURE_MASK(x)	(GENMASK_ULL(x##_SHIFT + ARM64_FEATURE_FIELD_BITS - 1, x##_SHIFT))
> diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
> index 5608d3410660..1552cd5581b7 100644
> --- a/arch/arm64/kvm/sys_regs.c
> +++ b/arch/arm64/kvm/sys_regs.c
> @@ -265,6 +265,181 @@ static bool trap_raz_wi(struct kvm_vcpu *vcpu,
>  		return read_zero(vcpu, p);
>  }
>  
> +/*
> + * A value for FCT_LOWER_SAFE must be zero and changing that will affect
> + * ftr_check_types of id_reg_info.
> + */
> +enum feature_check_type {
> +	FCT_LOWER_SAFE = 0,
> +	FCT_HIGHER_SAFE,
> +	FCT_HIGHER_OR_ZERO_SAFE,
> +	FCT_EXACT,
> +	FCT_EXACT_OR_ZERO_SAFE,
> +	FCT_IGNORE,	/* Don't check (any value is fine) */
Maybe you can remove the _SAFE suffix (EXACT does not have it).
s/EXACT/EQUAL ?
> +};
> +
> +static int arm64_check_feature_one(enum feature_check_type type, int val,
> +				   int limit)
> +{
> +	bool is_safe = false;
> +
> +	if (val == limit)
> +		return 0;
even if the type is unexpected?
> +
> +	switch (type) {
> +	case FCT_LOWER_SAFE:
> +		is_safe = (val <= limit);
> +		break;
> +	case FCT_HIGHER_OR_ZERO_SAFE:
> +		if (val == 0) {
> +			is_safe = true;
> +			break;
> +		}
> +		fallthrough;
> +	case FCT_HIGHER_SAFE:
> +		is_safe = (val >= limit);
> +		break;
> +	case FCT_EXACT:
> +		break;
> +	case FCT_EXACT_OR_ZERO_SAFE:
> +		is_safe = (val == 0);
> +		break;
> +	case FCT_IGNORE:
> +		is_safe = true;
> +		break;
> +	default:
> +		WARN_ONCE(1, "Unexpected feature_check_type (%d)\n", type);
> +		break;
> +	}
> +
> +	return is_safe ? 0 : -1;
> +}
> +
> +#define	FCT_TYPE_MASK		0x7
> +#define	FCT_TYPE_SHIFT		1
> +#define	FCT_SIGN_MASK		0x1
> +#define	FCT_SIGN_SHIFT		0
> +#define	FCT_TYPE(val)	((val >> FCT_TYPE_SHIFT) & FCT_TYPE_MASK)
> +#define	FCT_SIGN(val)	((val >> FCT_SIGN_SHIFT) & FCT_SIGN_MASK)
> +
> +#define	MAKE_FCT(shift, type, sign)				\
> +	((u64)((((type) & FCT_TYPE_MASK) << FCT_TYPE_SHIFT) |	\
> +	       (((sign) & FCT_SIGN_MASK) << FCT_SIGN_SHIFT)) << (shift))
> +
> +/* For signed field */
> +#define	S_FCT(shift, type)	MAKE_FCT(shift, type, 1)
> +/* For unigned field */
> +#define	U_FCT(shift, type)	MAKE_FCT(shift, type, 0)
> +
> +/*
> + * @val and @lim are both a value of the ID register. The function checks
> + * if all features indicated in @val can be supported for guests on the host,
> + * which supports features indicated in @lim. @check_types indicates how> + * features in the ID register needs to be checked.
> + * See comments for id_reg_info's ftr_check_types field for more detail.
What about RES0 fields which may exist? add a comment to reassure about
the fact they are properly handled if there are?
> + */
> +static int arm64_check_features(u64 check_types, u64 val, u64 lim)
> +{
> +	int i;
> +
> +	for (i = 0; i < 64; i += ARM64_FEATURE_FIELD_BITS) {
> +		u8 ftr_check = (check_types >> i) & ARM64_FEATURE_FIELD_MASK;
> +		bool is_sign = FCT_SIGN(ftr_check);
> +		enum feature_check_type fctype = FCT_TYPE(ftr_check);
> +		int fval, flim, ret;
> +
> +		fval = cpuid_feature_extract_field(val, i, is_sign);
> +		flim = cpuid_feature_extract_field(lim, i, is_sign);
> +
> +		ret = arm64_check_feature_one(fctype, fval, flim);
> +		if (ret)
> +			return -E2BIG;
> +	}
> +	return 0;
> +}
> +
> +struct id_reg_info {
> +	u32	sys_reg;	/* Register ID */
use struct kernel-doc comments instead?
> +
> +	/*
> +	 * Limit value of the register for a vcpu. The value is the sanitized
> +	 * system value with bits cleared for unsupported features for the
> +	 * guest.
> +	 */
> +	u64	vcpu_limit_val;
> +
> +	/*
> +	 * The ftr_check_types is comprised of a set of 4 bits fields.
nit: s/bits field/bit field here and below
> +	 * Each 4 bits field is for a feature indicated by the same bits
> +	 * field of the ID register and indicates how the feature support
> +	 * for guests needs to be checked.
> +	 * The bit 0 indicates that the corresponding ID register field
> +	 * is signed(1) or unsigned(0).
> +	 * The bits [3:1] hold feature_check_type for the field.
> +	 * If all zero, all features in the ID register are treated as unsigned
> +	 * fields and checked based on Principles of the ID scheme for fields
> +	 * in ID registers (FCT_LOWER_SAFE of feature_check_type).
values set by the guest are checked against host ID field values
according to FCT_LOWER_SAFE test? You do not actually explicitly explain
what the check is about although this may be obvious for you?
> +	 */
> +	u64	ftr_check_types;
> +
> +	/* Initialization function of the id_reg_info */
> +	void (*init)(struct id_reg_info *id_reg);
> +
> +	/* Register specific validation function */
validation callback? it does not register anything. We have check
customization means already in ftr_check_types so it is difficult to
guess at that point why this cb is needed, all the more so it applies
after the ftr_checks.
> +	int (*validate)(struct kvm_vcpu *vcpu, const struct id_reg_info *id_reg,
> +			u64 val);
> +
> +	/* Return the reset value of the register for the vCPU */
> +	u64 (*get_reset_val)(struct kvm_vcpu *vcpu,
> +			     const struct id_reg_info *id_reg);
> +};
> +
> +static void id_reg_info_init(struct id_reg_info *id_reg)
> +{
> +	id_reg->vcpu_limit_val = read_sanitised_ftr_reg(id_reg->sys_reg);
> +	if (id_reg->init)
> +		id_reg->init(id_reg);
> +}
> +
> +/*
> + * An ID register that needs special handling to control the value for the
> + * guest must have its own id_reg_info in id_reg_info_table.
> + * (i.e. the reset value is different from the host's sanitized value,
> + * the value is affected by opt-in features, some fields needs specific
s/needs/need
> + * validation, etc.)
> + */
> +#define	GET_ID_REG_INFO(id)	(id_reg_info_table[IDREG_IDX(id)])
> +static struct id_reg_info *id_reg_info_table[KVM_ARM_ID_REG_MAX_NUM] = {};
> +
> +static int validate_id_reg(struct kvm_vcpu *vcpu,
> +			   const struct sys_reg_desc *rd, u64 val)
> +{
> +	u32 id = reg_to_encoding(rd);
> +	const struct id_reg_info *id_reg = GET_ID_REG_INFO(id);
> +	u64 limit, check_types;
> +	int err;
> +
> +	if (id_reg) {
> +		check_types = id_reg->ftr_check_types;
> +		limit = id_reg->vcpu_limit_val;
> +	} else {
> +		/* All fields are treated as unsigned and FCT_LOWER_SAFE */
> +		check_types = 0;
> +		limit = read_sanitised_ftr_reg(id);
> +	}
> +
> +	/* Check if the value indicates any feature that is not in the limit. */
> +	err = arm64_check_features(check_types, val, limit);
> +	if (err)
> +		return err;
> +
> +	if (id_reg && id_reg->validate)
> +		/* Run the ID register specific validity check. */
> +		err = id_reg->validate(vcpu, id_reg, val);
> +
> +	return err;
> +}
> +
>  /*
>   * ARMv8.1 mandates at least a trivial LORegion implementation, where all the
>   * RW registers are RES0 (which we can implement as RAZ/WI). On an ARMv8.0
> @@ -1183,11 +1358,19 @@ static unsigned int sve_visibility(const struct kvm_vcpu *vcpu,
>  static void reset_id_reg(struct kvm_vcpu *vcpu, const struct sys_reg_desc *rd)
>  {
>  	u32 id = reg_to_encoding(rd);
> +	struct id_reg_info *id_reg;
> +	u64 val;
>  
>  	if (vcpu_has_reset_once(vcpu))
>  		return;
>  
> -	__vcpu_sys_reg(vcpu, IDREG_SYS_IDX(id)) = read_sanitised_ftr_reg(id);
> +	id_reg = GET_ID_REG_INFO(id);
> +	if (id_reg && id_reg->get_reset_val)
> +		val = id_reg->get_reset_val(vcpu, id_reg);
> +	else
> +		val = read_sanitised_ftr_reg(id);
> +
> +	__vcpu_sys_reg(vcpu, IDREG_SYS_IDX(id)) = val;
>  }
>  
>  static int set_id_aa64pfr0_el1(struct kvm_vcpu *vcpu,
> @@ -1232,11 +1415,7 @@ static int set_id_aa64pfr0_el1(struct kvm_vcpu *vcpu,
>  	return 0;
>  }
>  
> -/*
> - * cpufeature ID register user accessors
> - *
> - * We don't allow the effective value to be changed.
> - */
> +/* cpufeature ID register user accessors */
>  static int __get_id_reg(const struct kvm_vcpu *vcpu,
>  			const struct sys_reg_desc *rd, void __user *uaddr,
>  			bool raz)
> @@ -1247,11 +1426,12 @@ static int __get_id_reg(const struct kvm_vcpu *vcpu,
>  	return reg_to_user(uaddr, &val, id);
>  }
>  
> -static int __set_id_reg(const struct kvm_vcpu *vcpu,
> +static int __set_id_reg(struct kvm_vcpu *vcpu,
>  			const struct sys_reg_desc *rd, void __user *uaddr,
>  			bool raz)
>  {
>  	const u64 id = sys_reg_to_index(rd);
> +	u32 encoding = reg_to_encoding(rd);
>  	int err;
>  	u64 val;
>  
> @@ -1259,10 +1439,22 @@ static int __set_id_reg(const struct kvm_vcpu *vcpu,
>  	if (err)
>  		return err;
>  
> -	/* This is what we mean by invariant: you can't change it. */
> -	if (val != read_id_reg(vcpu, rd, raz))
> +	/* Don't allow to change the reg unless the reg has id_reg_info */
> +	if (val != read_id_reg(vcpu, rd, raz) && !GET_ID_REG_INFO(encoding))
>  		return -EINVAL;
>  
> +	if (raz)
> +		return 0;
> +
> +	/* Don't allow to change the reg after the first KVM_RUN. */
> +	if (vcpu->arch.has_run_once)
> +		return -EINVAL;
> +
> +	err = validate_id_reg(vcpu, rd, val);
> +	if (err)
> +		return err;
> +
> +	__vcpu_sys_reg(vcpu, IDREG_SYS_IDX(encoding)) = val;
>  	return 0;
>  }
>  
> @@ -2826,6 +3018,20 @@ int kvm_arm_copy_sys_reg_indices(struct kvm_vcpu *vcpu, u64 __user *uindices)
>  	return write_demux_regids(uindices);
>  }
>  
> +static void id_reg_info_init_all(void)
> +{
> +	int i;
> +	struct id_reg_info *id_reg;
> +
> +	for (i = 0; i < ARRAY_SIZE(id_reg_info_table); i++) {
> +		id_reg = (struct id_reg_info *)id_reg_info_table[i];
> +		if (!id_reg)
> +			continue;
> +
> +		id_reg_info_init(id_reg);
> +	}
> +}
> +
>  void kvm_sys_reg_table_init(void)
>  {
>  	unsigned int i;
> @@ -2860,4 +3066,6 @@ void kvm_sys_reg_table_init(void)
>  			break;
>  	/* Clear all higher bits. */
>  	cache_levels &= (1 << (i*3))-1;
> +
> +	id_reg_info_init_all();
>  }
> 
Thanks

Eric

Re: [RFC PATCH v3 02/29] KVM: arm64: Save ID registers' sanitized value per vCPU

[Eric Auger]

Hi Reiji,

On 11/17/21 7:43 AM, Reiji Watanabe wrote:
> Extend sys_regs[] of kvm_cpu_context for ID registers and save ID
> registers' sanitized value in the array for the vCPU at the first
> vCPU reset. Use the saved ones when ID registers are read by
> userspace (via KVM_GET_ONE_REG) or the guest.
> 
> Signed-off-by: Reiji Watanabe <reijiw@google.com>
> ---
>  arch/arm64/include/asm/kvm_host.h | 10 +++++++
>  arch/arm64/kvm/sys_regs.c         | 43 +++++++++++++++++++------------
>  2 files changed, 37 insertions(+), 16 deletions(-)
> 
> diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
> index edbe2cb21947..72db73c79403 100644
> --- a/arch/arm64/include/asm/kvm_host.h
> +++ b/arch/arm64/include/asm/kvm_host.h
> @@ -146,6 +146,14 @@ struct kvm_vcpu_fault_info {
>  	u64 disr_el1;		/* Deferred [SError] Status Register */
>  };
>  
> +/*
> + * (Op0, Op1, CRn, CRm, Op2) of ID registers is (3, 0, 0, crm, op2),
> + * where 0<=crm<8, 0<=op2<8.
> + */
> +#define KVM_ARM_ID_REG_MAX_NUM 64
> +#define IDREG_IDX(id)		((sys_reg_CRm(id) << 3) | sys_reg_Op2(id))
> +#define IDREG_SYS_IDX(id)	(ID_REG_BASE + IDREG_IDX(id))
> +
>  enum vcpu_sysreg {
>  	__INVALID_SYSREG__,   /* 0 is reserved as an invalid value */
>  	MPIDR_EL1,	/* MultiProcessor Affinity Register */
> @@ -210,6 +218,8 @@ enum vcpu_sysreg {
>  	CNTP_CVAL_EL0,
>  	CNTP_CTL_EL0,
>  
> +	ID_REG_BASE,
> +	ID_REG_END = ID_REG_BASE + KVM_ARM_ID_REG_MAX_NUM - 1,
>  	/* Memory Tagging Extension registers */
>  	RGSR_EL1,	/* Random Allocation Tag Seed Register */
>  	GCR_EL1,	/* Tag Control Register */
> diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
> index e3ec1a44f94d..5608d3410660 100644
> --- a/arch/arm64/kvm/sys_regs.c
> +++ b/arch/arm64/kvm/sys_regs.c
> @@ -33,6 +33,8 @@
>  
>  #include "trace.h"
>  
> +static u64 __read_id_reg(const struct kvm_vcpu *vcpu, u32 id);
> +
>  /*
>   * All of this file is extremely similar to the ARM coproc.c, but the
>   * types are different. My gut feeling is that it should be pretty
> @@ -273,7 +275,7 @@ static bool trap_loregion(struct kvm_vcpu *vcpu,
>  			  struct sys_reg_params *p,
>  			  const struct sys_reg_desc *r)
>  {
> -	u64 val = read_sanitised_ftr_reg(SYS_ID_AA64MMFR1_EL1);
> +	u64 val = __read_id_reg(vcpu, SYS_ID_AA64MMFR1_EL1);
>  	u32 sr = reg_to_encoding(r);
>  
>  	if (!(val & (0xfUL << ID_AA64MMFR1_LOR_SHIFT))) {
> @@ -1059,17 +1061,9 @@ static bool access_arch_timer(struct kvm_vcpu *vcpu,
>  	return true;
>  }
>  
> -/* Read a sanitised cpufeature ID register by sys_reg_desc */
> -static u64 read_id_reg(const struct kvm_vcpu *vcpu,
> -		struct sys_reg_desc const *r, bool raz)
> +static u64 __read_id_reg(const struct kvm_vcpu *vcpu, u32 id)
>  {
> -	u32 id = reg_to_encoding(r);
> -	u64 val;
> -
> -	if (raz)
> -		return 0;
> -
> -	val = read_sanitised_ftr_reg(id);
> +	u64 val = __vcpu_sys_reg(vcpu, IDREG_SYS_IDX(id));
>  
>  	switch (id) {
>  	case SYS_ID_AA64PFR0_EL1:
> @@ -1119,6 +1113,14 @@ static u64 read_id_reg(const struct kvm_vcpu *vcpu,
>  	return val;
>  }
>  
> +static u64 read_id_reg(const struct kvm_vcpu *vcpu,
> +		       struct sys_reg_desc const *r, bool raz)
> +{
> +	u32 id = reg_to_encoding(r);
> +
> +	return raz ? 0 : __read_id_reg(vcpu, id);
> +}
> +
>  static unsigned int id_visibility(const struct kvm_vcpu *vcpu,
>  				  const struct sys_reg_desc *r)
>  {
> @@ -1178,6 +1180,16 @@ static unsigned int sve_visibility(const struct kvm_vcpu *vcpu,
>  	return REG_HIDDEN;
>  }
>  
> +static void reset_id_reg(struct kvm_vcpu *vcpu, const struct sys_reg_desc *rd)
> +{
> +	u32 id = reg_to_encoding(rd);
> +
> +	if (vcpu_has_reset_once(vcpu))
> +		return;
> +
> +	__vcpu_sys_reg(vcpu, IDREG_SYS_IDX(id)) = read_sanitised_ftr_reg(id);
> +}
> +
>  static int set_id_aa64pfr0_el1(struct kvm_vcpu *vcpu,
>  			       const struct sys_reg_desc *rd,
>  			       const struct kvm_one_reg *reg, void __user *uaddr)
> @@ -1223,9 +1235,7 @@ static int set_id_aa64pfr0_el1(struct kvm_vcpu *vcpu,
>  /*
>   * cpufeature ID register user accessors
>   *
> - * For now, these registers are immutable for userspace, so no values
> - * are stored, and for set_id_reg() we don't allow the effective value
> - * to be changed.
> + * We don't allow the effective value to be changed.
This change may be moved to a subsequent patch as this patch does not
change the behavior yet.
>   */
>  static int __get_id_reg(const struct kvm_vcpu *vcpu,
>  			const struct sys_reg_desc *rd, void __user *uaddr,
> @@ -1382,6 +1392,7 @@ static unsigned int mte_visibility(const struct kvm_vcpu *vcpu,
>  #define ID_SANITISED(name) {			\
>  	SYS_DESC(SYS_##name),			\
>  	.access	= access_id_reg,		\
> +	.reset	= reset_id_reg,			\
>  	.get_user = get_id_reg,			\
>  	.set_user = set_id_reg,			\
>  	.visibility = id_visibility,		\
> @@ -1837,8 +1848,8 @@ static bool trap_dbgdidr(struct kvm_vcpu *vcpu,
>  	if (p->is_write) {
>  		return ignore_write(vcpu, p);
>  	} else {
> -		u64 dfr = read_sanitised_ftr_reg(SYS_ID_AA64DFR0_EL1);
> -		u64 pfr = read_sanitised_ftr_reg(SYS_ID_AA64PFR0_EL1);
> +		u64 dfr = __read_id_reg(vcpu, SYS_ID_AA64DFR0_EL1);
> +		u64 pfr = __read_id_reg(vcpu, SYS_ID_AA64PFR0_EL1);
>  		u32 el3 = !!cpuid_feature_extract_unsigned_field(pfr, ID_AA64PFR0_EL3_SHIFT);
>  
>  		p->regval = ((((dfr >> ID_AA64DFR0_WRPS_SHIFT) & 0xf) << 28) |
> 
Thanks

Eric

Re: [RFC PATCH v3 02/29] KVM: arm64: Save ID registers' sanitized value per vCPU

[Reiji Watanabe]

Hi Eric,

On Thu, Nov 18, 2021 at 12:37 PM Eric Auger <eauger@redhat.com> wrote:
>
> Hi Reiji,
>
> On 11/17/21 7:43 AM, Reiji Watanabe wrote:
> > Extend sys_regs[] of kvm_cpu_context for ID registers and save ID
> > registers' sanitized value in the array for the vCPU at the first
> > vCPU reset. Use the saved ones when ID registers are read by
> > userspace (via KVM_GET_ONE_REG) or the guest.
> >
> > Signed-off-by: Reiji Watanabe <reijiw@google.com>
> > ---
> >  arch/arm64/include/asm/kvm_host.h | 10 +++++++
> >  arch/arm64/kvm/sys_regs.c         | 43 +++++++++++++++++++------------
> >  2 files changed, 37 insertions(+), 16 deletions(-)
> >
> > diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
> > index edbe2cb21947..72db73c79403 100644
> > --- a/arch/arm64/include/asm/kvm_host.h
> > +++ b/arch/arm64/include/asm/kvm_host.h
> > @@ -146,6 +146,14 @@ struct kvm_vcpu_fault_info {
> >       u64 disr_el1;           /* Deferred [SError] Status Register */
> >  };
> >
> > +/*
> > + * (Op0, Op1, CRn, CRm, Op2) of ID registers is (3, 0, 0, crm, op2),
> > + * where 0<=crm<8, 0<=op2<8.
> > + */
> > +#define KVM_ARM_ID_REG_MAX_NUM 64
> > +#define IDREG_IDX(id)                ((sys_reg_CRm(id) << 3) | sys_reg_Op2(id))
> > +#define IDREG_SYS_IDX(id)    (ID_REG_BASE + IDREG_IDX(id))
> > +
> >  enum vcpu_sysreg {
> >       __INVALID_SYSREG__,   /* 0 is reserved as an invalid value */
> >       MPIDR_EL1,      /* MultiProcessor Affinity Register */
> > @@ -210,6 +218,8 @@ enum vcpu_sysreg {
> >       CNTP_CVAL_EL0,
> >       CNTP_CTL_EL0,
> >
> > +     ID_REG_BASE,
> > +     ID_REG_END = ID_REG_BASE + KVM_ARM_ID_REG_MAX_NUM - 1,
> >       /* Memory Tagging Extension registers */
> >       RGSR_EL1,       /* Random Allocation Tag Seed Register */
> >       GCR_EL1,        /* Tag Control Register */
> > diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
> > index e3ec1a44f94d..5608d3410660 100644
> > --- a/arch/arm64/kvm/sys_regs.c
> > +++ b/arch/arm64/kvm/sys_regs.c
> > @@ -33,6 +33,8 @@
> >
> >  #include "trace.h"
> >
> > +static u64 __read_id_reg(const struct kvm_vcpu *vcpu, u32 id);
> > +
> >  /*
> >   * All of this file is extremely similar to the ARM coproc.c, but the
> >   * types are different. My gut feeling is that it should be pretty
> > @@ -273,7 +275,7 @@ static bool trap_loregion(struct kvm_vcpu *vcpu,
> >                         struct sys_reg_params *p,
> >                         const struct sys_reg_desc *r)
> >  {
> > -     u64 val = read_sanitised_ftr_reg(SYS_ID_AA64MMFR1_EL1);
> > +     u64 val = __read_id_reg(vcpu, SYS_ID_AA64MMFR1_EL1);
> >       u32 sr = reg_to_encoding(r);
> >
> >       if (!(val & (0xfUL << ID_AA64MMFR1_LOR_SHIFT))) {
> > @@ -1059,17 +1061,9 @@ static bool access_arch_timer(struct kvm_vcpu *vcpu,
> >       return true;
> >  }
> >
> > -/* Read a sanitised cpufeature ID register by sys_reg_desc */
> > -static u64 read_id_reg(const struct kvm_vcpu *vcpu,
> > -             struct sys_reg_desc const *r, bool raz)
> > +static u64 __read_id_reg(const struct kvm_vcpu *vcpu, u32 id)
> >  {
> > -     u32 id = reg_to_encoding(r);
> > -     u64 val;
> > -
> > -     if (raz)
> > -             return 0;
> > -
> > -     val = read_sanitised_ftr_reg(id);
> > +     u64 val = __vcpu_sys_reg(vcpu, IDREG_SYS_IDX(id));
> >
> >       switch (id) {
> >       case SYS_ID_AA64PFR0_EL1:
> > @@ -1119,6 +1113,14 @@ static u64 read_id_reg(const struct kvm_vcpu *vcpu,
> >       return val;
> >  }
> >
> > +static u64 read_id_reg(const struct kvm_vcpu *vcpu,
> > +                    struct sys_reg_desc const *r, bool raz)
> > +{
> > +     u32 id = reg_to_encoding(r);
> > +
> > +     return raz ? 0 : __read_id_reg(vcpu, id);
> > +}
> > +
> >  static unsigned int id_visibility(const struct kvm_vcpu *vcpu,
> >                                 const struct sys_reg_desc *r)
> >  {
> > @@ -1178,6 +1180,16 @@ static unsigned int sve_visibility(const struct kvm_vcpu *vcpu,
> >       return REG_HIDDEN;
> >  }
> >
> > +static void reset_id_reg(struct kvm_vcpu *vcpu, const struct sys_reg_desc *rd)
> > +{
> > +     u32 id = reg_to_encoding(rd);
> > +
> > +     if (vcpu_has_reset_once(vcpu))
> > +             return;
> > +
> > +     __vcpu_sys_reg(vcpu, IDREG_SYS_IDX(id)) = read_sanitised_ftr_reg(id);
> > +}
> > +
> >  static int set_id_aa64pfr0_el1(struct kvm_vcpu *vcpu,
> >                              const struct sys_reg_desc *rd,
> >                              const struct kvm_one_reg *reg, void __user *uaddr)
> > @@ -1223,9 +1235,7 @@ static int set_id_aa64pfr0_el1(struct kvm_vcpu *vcpu,
> >  /*
> >   * cpufeature ID register user accessors
> >   *
> > - * For now, these registers are immutable for userspace, so no values
> > - * are stored, and for set_id_reg() we don't allow the effective value
> > - * to be changed.
> > + * We don't allow the effective value to be changed.
> This change may be moved to a subsequent patch as this patch does not
> change the behavior yet.

Thank you for the review.

There are three main parts in the original comments.

 (A) these registers are immutable for userspace
 (B) no values are stored
 (C) we don't allow the effective value to be changed

This patch stores ID register values in sys_regs[].
So, I don't think (B) should be there, and I removed (B).
Since (A) is essentially the same as (C), I removed (A)
(and left (C)).

Do you think it is better to leave (A) in this patch, too ?

Thanks,
Reiji


> >   */
> >  static int __get_id_reg(const struct kvm_vcpu *vcpu,
> >                       const struct sys_reg_desc *rd, void __user *uaddr,
> > @@ -1382,6 +1392,7 @@ static unsigned int mte_visibility(const struct kvm_vcpu *vcpu,
> >  #define ID_SANITISED(name) {                 \
> >       SYS_DESC(SYS_##name),                   \
> >       .access = access_id_reg,                \
> > +     .reset  = reset_id_reg,                 \
> >       .get_user = get_id_reg,                 \
> >       .set_user = set_id_reg,                 \
> >       .visibility = id_visibility,            \
> > @@ -1837,8 +1848,8 @@ static bool trap_dbgdidr(struct kvm_vcpu *vcpu,
> >       if (p->is_write) {
> >               return ignore_write(vcpu, p);
> >       } else {
> > -             u64 dfr = read_sanitised_ftr_reg(SYS_ID_AA64DFR0_EL1);
> > -             u64 pfr = read_sanitised_ftr_reg(SYS_ID_AA64PFR0_EL1);
> > +             u64 dfr = __read_id_reg(vcpu, SYS_ID_AA64DFR0_EL1);
> > +             u64 pfr = __read_id_reg(vcpu, SYS_ID_AA64PFR0_EL1);
> >               u32 el3 = !!cpuid_feature_extract_unsigned_field(pfr, ID_AA64PFR0_EL3_SHIFT);
> >
> >               p->regval = ((((dfr >> ID_AA64DFR0_WRPS_SHIFT) & 0xf) << 28) |
> >
> Thanks
>
> Eric
>

Re: [RFC PATCH v3 03/29] KVM: arm64: Introduce struct id_reg_info

[Reiji Watanabe]

Hi Eric,

On Thu, Nov 18, 2021 at 12:36 PM Eric Auger <eauger@redhat.com> wrote:
>
> Hi Reiji,
>
> On 11/17/21 7:43 AM, Reiji Watanabe wrote:
> > This patch lays the groundwork to make ID registers writable.
> >
> > Introduce struct id_reg_info for an ID register to manage the
> > register specific control of its value for the guest, and provide set
> > of functions commonly used for ID registers to make them writable.
> >
> > The id_reg_info is used to do register specific initialization,
> > validation of the ID register and etc.  Not all ID registers must
> > have the id_reg_info. ID registers that don't have the id_reg_info
> > are handled in a common way that is applied to all ID registers.
> >
> > At present, changing an ID register from userspace is allowed only
> > if the ID register has the id_reg_info, but that will be changed
> > by the following patches.
> >
> > No ID register has the structure yet and the following patches
> > will add the id_reg_info for some ID registers.
> >
> > Signed-off-by: Reiji Watanabe <reijiw@google.com>
> > ---
> >  arch/arm64/include/asm/sysreg.h |   1 +
> >  arch/arm64/kvm/sys_regs.c       | 226 ++++++++++++++++++++++++++++++--
> >  2 files changed, 218 insertions(+), 9 deletions(-)
> >
> > diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h
> > index 16b3f1a1d468..597609f26331 100644
> > --- a/arch/arm64/include/asm/sysreg.h
> > +++ b/arch/arm64/include/asm/sysreg.h
> > @@ -1197,6 +1197,7 @@
> >  #define ICH_VTR_TDS_MASK     (1 << ICH_VTR_TDS_SHIFT)
> >
> >  #define ARM64_FEATURE_FIELD_BITS     4
> > +#define ARM64_FEATURE_FIELD_MASK     ((1ull << ARM64_FEATURE_FIELD_BITS) - 1)
> >
> >  /* Create a mask for the feature bits of the specified feature. */
> >  #define ARM64_FEATURE_MASK(x)        (GENMASK_ULL(x##_SHIFT + ARM64_FEATURE_FIELD_BITS - 1, x##_SHIFT))
> > diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
> > index 5608d3410660..1552cd5581b7 100644
> > --- a/arch/arm64/kvm/sys_regs.c
> > +++ b/arch/arm64/kvm/sys_regs.c
> > @@ -265,6 +265,181 @@ static bool trap_raz_wi(struct kvm_vcpu *vcpu,
> >               return read_zero(vcpu, p);
> >  }
> >
> > +/*
> > + * A value for FCT_LOWER_SAFE must be zero and changing that will affect
> > + * ftr_check_types of id_reg_info.
> > + */
> > +enum feature_check_type {
> > +     FCT_LOWER_SAFE = 0,
> > +     FCT_HIGHER_SAFE,
> > +     FCT_HIGHER_OR_ZERO_SAFE,
> > +     FCT_EXACT,
> > +     FCT_EXACT_OR_ZERO_SAFE,
> > +     FCT_IGNORE,     /* Don't check (any value is fine) */
> Maybe you can remove the _SAFE suffix (EXACT does not have it).

I am inclined to keep 'SAFE' (otherwise, I am likely to forget
if lower is safe or not).

> s/EXACT/EQUAL ?

I will fix that FCT_EXACT to FCT_EQUAL_SAFE.

> > +};
> > +
> > +static int arm64_check_feature_one(enum feature_check_type type, int val,
> > +                                int limit)
> > +{
> > +     bool is_safe = false;
> > +
> > +     if (val == limit)
> > +             return 0;
> even if the type is unexpected?

I will remove it.

> > +
> > +     switch (type) {
> > +     case FCT_LOWER_SAFE:
> > +             is_safe = (val <= limit);
> > +             break;
> > +     case FCT_HIGHER_OR_ZERO_SAFE:
> > +             if (val == 0) {
> > +                     is_safe = true;
> > +                     break;
> > +             }
> > +             fallthrough;
> > +     case FCT_HIGHER_SAFE:
> > +             is_safe = (val >= limit);
> > +             break;
> > +     case FCT_EXACT:
> > +             break;
> > +     case FCT_EXACT_OR_ZERO_SAFE:
> > +             is_safe = (val == 0);
> > +             break;
> > +     case FCT_IGNORE:
> > +             is_safe = true;
> > +             break;
> > +     default:
> > +             WARN_ONCE(1, "Unexpected feature_check_type (%d)\n", type);
> > +             break;
> > +     }
> > +
> > +     return is_safe ? 0 : -1;
> > +}
> > +
> > +#define      FCT_TYPE_MASK           0x7
> > +#define      FCT_TYPE_SHIFT          1
> > +#define      FCT_SIGN_MASK           0x1
> > +#define      FCT_SIGN_SHIFT          0
> > +#define      FCT_TYPE(val)   ((val >> FCT_TYPE_SHIFT) & FCT_TYPE_MASK)
> > +#define      FCT_SIGN(val)   ((val >> FCT_SIGN_SHIFT) & FCT_SIGN_MASK)
> > +
> > +#define      MAKE_FCT(shift, type, sign)                             \
> > +     ((u64)((((type) & FCT_TYPE_MASK) << FCT_TYPE_SHIFT) |   \
> > +            (((sign) & FCT_SIGN_MASK) << FCT_SIGN_SHIFT)) << (shift))
> > +
> > +/* For signed field */
> > +#define      S_FCT(shift, type)      MAKE_FCT(shift, type, 1)
> > +/* For unigned field */
> > +#define      U_FCT(shift, type)      MAKE_FCT(shift, type, 0)
> > +
> > +/*
> > + * @val and @lim are both a value of the ID register. The function checks
> > + * if all features indicated in @val can be supported for guests on the host,
> > + * which supports features indicated in @lim. @check_types indicates how> + * features in the ID register needs to be checked.
> > + * See comments for id_reg_info's ftr_check_types field for more detail.
> What about RES0 fields which may exist? add a comment to reassure about
> the fact they are properly handled if there are?

Any fields including RES0 should be checked based on check_types.
I will explicitly state that in the comment.

> > + */
> > +static int arm64_check_features(u64 check_types, u64 val, u64 lim)
> > +{
> > +     int i;
> > +
> > +     for (i = 0; i < 64; i += ARM64_FEATURE_FIELD_BITS) {
> > +             u8 ftr_check = (check_types >> i) & ARM64_FEATURE_FIELD_MASK;
> > +             bool is_sign = FCT_SIGN(ftr_check);
> > +             enum feature_check_type fctype = FCT_TYPE(ftr_check);
> > +             int fval, flim, ret;
> > +
> > +             fval = cpuid_feature_extract_field(val, i, is_sign);
> > +             flim = cpuid_feature_extract_field(lim, i, is_sign);
> > +
> > +             ret = arm64_check_feature_one(fctype, fval, flim);
> > +             if (ret)
> > +                     return -E2BIG;
> > +     }
> > +     return 0;
> > +}
> > +
> > +struct id_reg_info {
> > +     u32     sys_reg;        /* Register ID */
> use struct kernel-doc comments instead?
> > +
> > +     /*
> > +      * Limit value of the register for a vcpu. The value is the sanitized
> > +      * system value with bits cleared for unsupported features for the
> > +      * guest.
> > +      */
> > +     u64     vcpu_limit_val;
> > +
> > +     /*
> > +      * The ftr_check_types is comprised of a set of 4 bits fields.
> nit: s/bits field/bit field here and below

I will fix them.

> > +      * Each 4 bits field is for a feature indicated by the same bits
> > +      * field of the ID register and indicates how the feature support
> > +      * for guests needs to be checked.
> > +      * The bit 0 indicates that the corresponding ID register field
> > +      * is signed(1) or unsigned(0).
> > +      * The bits [3:1] hold feature_check_type for the field.
> > +      * If all zero, all features in the ID register are treated as unsigned
> > +      * fields and checked based on Principles of the ID scheme for fields
> > +      * in ID registers (FCT_LOWER_SAFE of feature_check_type).
> values set by the guest are checked against host ID field values
> according to FCT_LOWER_SAFE test? You do not actually explicitly explain
> what the check is about although this may be obvious for you?

How about this ?

        /*
         * The ftr_check_types is comprised of a set of 4 bit fields.
         * Each 4 bit field is for a feature indicated by the same bit field
         * of the ID register and indicates how the field needs to be checked
         * (by arm64_check_feature_one) against the host's ID field when
         * userspace tries to set the register.
         * The bit 0 indicates that the corresponding ID register field is
         * signed(1) or unsigned(0). The bits [3:1] hold feature_check_type
         * for the field (FCT_LOWER_SAFE == 0, etc).
         * e.g. for ID_AA64PFR0_EL1.SVE(bits [35:32]), bits[35:32] of
         * ftr_check_types for the register should be 0. It means the SVE
         * field is treated as an unsigned field, and userspace can set the
         * field to a equal or lower value than the host's ID field value.
         */

> > +      */
> > +     u64     ftr_check_types;
> > +
> > +     /* Initialization function of the id_reg_info */
> > +     void (*init)(struct id_reg_info *id_reg);
> > +
> > +     /* Register specific validation function */
> validation callback? it does not register anything. We have check
> customization means already in ftr_check_types so it is difficult to
> guess at that point why this cb is needed, all the more so it applies
> after the ftr_checks.

I am going to add the following comment. Does it look clear enough for you ?

        /*
         * This is an optional ID register specific validation function.
         * When userspace tries to set the ID register, arm64_check_features()
         * will check if the requested value indicates any features that cannot
         * be supported by KVM on the host.  But, some ID register fields need
         * a special checking and this function can be used for such fields.
         * e.g. KVM_CREATE_DEVICE must be used to configure GICv3 for a guest.
         * ID_AA64PFR0_EL1.GIC shouldn't be set to 1 unless GICv3 is configured.
         * The validation function for ID_AA64PFR0_EL1 could be used to check
         * the field is consistent with GICv3 configuration.
         */

> > +     int (*validate)(struct kvm_vcpu *vcpu, const struct id_reg_info *id_reg,
> > +                     u64 val);
> > +
> > +     /* Return the reset value of the register for the vCPU */
> > +     u64 (*get_reset_val)(struct kvm_vcpu *vcpu,
> > +                          const struct id_reg_info *id_reg);
> > +};
> > +
> > +static void id_reg_info_init(struct id_reg_info *id_reg)
> > +{
> > +     id_reg->vcpu_limit_val = read_sanitised_ftr_reg(id_reg->sys_reg);
> > +     if (id_reg->init)
> > +             id_reg->init(id_reg);
> > +}
> > +
> > +/*
> > + * An ID register that needs special handling to control the value for the
> > + * guest must have its own id_reg_info in id_reg_info_table.
> > + * (i.e. the reset value is different from the host's sanitized value,
> > + * the value is affected by opt-in features, some fields needs specific
> s/needs/need

I will fix it.

Thank you for your review !

Regards
Reiji

Re: [RFC PATCH v3 29/29] KVM: arm64: selftests: Introduce id_reg_test

[Reiji Watanabe]

 Hi Eric,

On Thu, Nov 18, 2021 at 12:34 PM Eric Auger <eauger@redhat.com> wrote:
>
> Hi Reiji,
>
> On 11/17/21 7:43 AM, Reiji Watanabe wrote:
> > Introduce a test for aarch64 to validate basic behavior of
> > KVM_GET_ONE_REG and KVM_SET_ONE_REG for ID registers.
> >
> > This test runs only when KVM_CAP_ARM_ID_REG_CONFIGURABLE is supported.
>
> That's great to get those tests along with the series.
>
> There are several tests actually. I would encourage you to drop a short
> comment along with the each main test to summarize what it does.

Thank you for the review !
Yes, I will add short comments for the main test to summarize what it does.


> > +struct id_reg_test_info {
> > +     char            *name;
> > +     uint32_t        id;
> > +     bool            can_clear;
> > +     uint64_t        fixed_mask;
> > +     uint64_t        org_val;
> nit: original_val? or default_val?

That is an original (or initial) value that is set by KVM.
I will change it to original_val.


> > +void test_pmu_init(struct kvm_vm *vm, uint32_t vcpu)
> I would remove the test_ prefix as it does not test anything but
> enhances the initialization instead

Yes, I agree.
I will remove test_ prefix from those names.

> > +{
> > +     struct kvm_device_attr attr = {
> > +             .group = KVM_ARM_VCPU_PMU_V3_CTRL,
> > +             .attr = KVM_ARM_VCPU_PMU_V3_INIT,
> > +     };
> > +     vcpu_ioctl(vm, vcpu, KVM_SET_DEVICE_ATTR, &attr);
> > +}
> > +
> > +void test_sve_init(struct kvm_vm *vm, uint32_t vcpu)
> > +{
> > +     int feature = KVM_ARM_VCPU_SVE;
> > +
> > +     vcpu_ioctl(vm, vcpu, KVM_ARM_VCPU_FINALIZE, &feature);
> > +}
> > +
> > +#define GICD_BASE_GPA                        0x8000000ULL
> > +#define GICR_BASE_GPA                        0x80A0000ULL
> > +
> > +void test_vgic_init(struct kvm_vm *vm, uint32_t vcpu)
> > +{
> > +     /* We jsut need to configure gic v3 (we don't use it though) */
> > +     vgic_v3_setup(vm, 1, GICD_BASE_GPA, GICR_BASE_GPA);
> > +}
> > +
> > +#define      MAX_CAPS        2
> > +struct feature_test_info {
> > +     char    *name;  /* Feature Name (Debug information) */
> > +     struct id_reg_test_info *sreg;  /* ID register for the feature */
> ID register comprising the feature?
> > +     int     shift;  /* Field of the ID register for the feature */
> I guess you mean feature field bit shift
> > +     int     min;    /* Min value to indicate the feature */
> Min value that can be assigned to the feature field?
> > +     bool    is_sign;        /* Is the field signed or unsigned ? */
> > +     int     ncaps;          /* Number of valid Capabilities in caps[] */
> > +     long    caps[MAX_CAPS]; /* Capabilities to indicate this feature */
> I suggest: KVM_CAP_* capabilities requested to test this feature
> > +     /* struct kvm_enable_cap to use the capability if needed */
> > +     struct kvm_enable_cap   *opt_in_cap;
> > +     bool    run_test;       /* Does guest run test for this feature ? */
> s/run_test/guest_run?
> > +     /* Initialization function for the feature as needed */
> extra init sequence needed besides KVM CAP setting?
> > +     void    (*init_feature)(struct kvm_vm *vm, uint32_t vcpuid);
> > +     /* struct kvm_vcpu_init to opt-in the feature if needed */
> > +     struct kvm_vcpu_init    *vcpu_init;
> > +};

I am going to fix the comments as follows.
(Or are any of them still unclear ?)

    /* ID register that identifies the presence of the feature */
    struct id_reg_test_info *sreg;

    /* Bit shift for the field that identifies the presence of the feature */
    int     shift;

    /* Min value of the field that indicates the presence of the feature */
    int     min;    /* Min value to indicate the feature */

    /* KVM_CAP_* Capabilities to indicates that KVM supports this feature */
    long    caps[MAX_CAPS]; /* Capabilities to indicate this feature */

    /* Should the guest check the ID register for this feature ? */
    bool    run_test;

    /*
     * Any extra function to configure the feature if needed.
     * (e.g. KVM_ARM_VCPU_FINALIZE for SVE)
     */
    void    (*init_feature)(struct kvm_vm *vm, uint32_t vcpuid);

> > +
> > +/* Test for optin CPU features */
> opt-in?

I will fix it.

> > +static struct feature_test_info feature_test_info_table[] = {
> > +     {
> > +             .name = "SVE",
> > +             .sreg = ID_REG_INFO(ID_AA64PFR0),
> > +             .shift = ID_AA64PFR0_SVE_SHIFT,
> > +             .min = 1,
> > +             .caps = {KVM_CAP_ARM_SVE},
> > +             .ncaps = 1,
> > +             .init_feature = test_sve_init,
> > +             .vcpu_init = &(struct kvm_vcpu_init) {
> > +                     .features = {1ULL << KVM_ARM_VCPU_SVE},
> > +             },
> > +     },
> > +     {
> > +             .name = "GIC",
> > +             .sreg = ID_REG_INFO(ID_AA64PFR0),
> > +             .shift = ID_AA64PFR0_GIC_SHIFT,
> > +             .min = 1,
> > +             .caps = {KVM_CAP_IRQCHIP},
> > +             .ncaps = 1,
> > +             .init_feature = test_vgic_init,
> > +     },
> > +     {
> > +             .name = "MTE",
> > +             .sreg = ID_REG_INFO(ID_AA64PFR1),
> > +             .shift = ID_AA64PFR1_MTE_SHIFT,
> > +             .min = 2,
> > +             .caps = {KVM_CAP_ARM_MTE},
> > +             .ncaps = 1,
> > +             .opt_in_cap = &(struct kvm_enable_cap) {
> > +                             .cap = KVM_CAP_ARM_MTE,
> > +             },
> > +     },
> > +     {
> > +             .name = "PMUV3",
> > +             .sreg = ID_REG_INFO(ID_AA64DFR0),
> > +             .shift = ID_AA64DFR0_PMUVER_SHIFT,
> > +             .min = 1,
> > +             .init_feature = test_pmu_init,
> > +             .caps = {KVM_CAP_ARM_PMU_V3},
> > +             .ncaps = 1,
> > +             .vcpu_init = &(struct kvm_vcpu_init) {
> > +                     .features = {1ULL << KVM_ARM_VCPU_PMU_V3},
> > +             },
> > +     },
> > +     {
> > +             .name = "PERFMON",
> > +             .sreg = ID_REG_INFO(ID_DFR0),
> > +             .shift = ID_DFR0_PERFMON_SHIFT,
> > +             .min = 3,
> > +             .init_feature = test_pmu_init,
> > +             .caps = {KVM_CAP_ARM_PMU_V3},
> > +             .ncaps = 1,
> > +             .vcpu_init = &(struct kvm_vcpu_init) {
> > +                     .features = {1ULL << KVM_ARM_VCPU_PMU_V3},
> > +             },
> > +     },
> > +};
> > +
> > +static int walk_id_reg_list(int (*fn)(struct id_reg_test_info *sreg, void *arg),
> > +                         void *arg)
> > +{
> > +     int ret = 0, i;
> > +
> > +     for (i = 0; i < ARRAY_SIZE(id_reg_list); i++) {
> > +             ret = fn(&id_reg_list[i], arg);
> > +             if (ret)
> > +                     break;
> none of your fn() function does return something != 0

I will change the callback function to return void instead of int.

> > +     }
> > +     return ret;
> > +}
> > +
> > +static int guest_code_id_reg_check_one(struct id_reg_test_info *sreg, void *arg)
> > +{
> > +     uint64_t val = sreg->read_reg();
> > +
> > +     GUEST_ASSERT_2(val == sreg->user_val, sreg->name, sreg->user_val);
> > +     return 0;
> > +}
> > +
> > +static void guest_code_id_reg_check_all(uint32_t cpu)
> > +{
> > +     walk_id_reg_list(guest_code_id_reg_check_one, NULL);
> > +     GUEST_DONE();
> > +}
> > +
> > +static void guest_code_do_nothing(uint32_t cpu)
> > +{
> > +     GUEST_DONE();
> > +}
> > +
> > +static void guest_code_feature_check(uint32_t cpu)
> > +{
> > +     int i;
> > +     struct feature_test_info *finfo;
> > +
> > +     for (i = 0; i < ARRAY_SIZE(feature_test_info_table); i++) {
> > +             finfo = &feature_test_info_table[i];
> > +             if (finfo->run_test)
> > +                     guest_code_id_reg_check_one(finfo->sreg, NULL);
> > +     }
> > +
> > +     GUEST_DONE();
> > +}
> > +
> > +static void guest_code_ptrauth_check(uint32_t cpuid)
> > +{
> > +     struct id_reg_test_info *sreg = ID_REG_INFO(ID_AA64ISAR1);
> > +     uint64_t val = sreg->read_reg();
> > +
> > +     GUEST_ASSERT_2(val == sreg->user_val, "PTRAUTH", val);
> > +     GUEST_DONE();
> > +}
> > +
> > +static int reset_id_reg_info_one(struct id_reg_test_info *sreg, void *arg)
> reset_id_reg_user_val_one()?

Thanks for the suggestion. I will fix that.

> > +{
> > +     sreg->user_val = sreg->org_val;
> > +     return 0;
> > +}
> > +
> > +static void reset_id_reg_info(void)
> reset_id_reg_user_val()?

 I will fix that.

> > +{
> > +     walk_id_reg_list(reset_id_reg_info_one, NULL);
> > +}
> > +
> > +static struct kvm_vm *test_vm_create_cap(uint32_t nvcpus,
> > +             void (*guest_code)(uint32_t), struct kvm_vcpu_init *init,
> > +             struct kvm_enable_cap *cap)
> > +{
> > +     struct kvm_vm *vm;
> > +     uint32_t cpuid;
> > +     uint64_t mem_pages;
> > +
> > +     mem_pages = DEFAULT_GUEST_PHY_PAGES + DEFAULT_STACK_PGS * nvcpus;
> > +     mem_pages += mem_pages / (PTES_PER_MIN_PAGE * 2);
> > +     mem_pages = vm_adjust_num_guest_pages(VM_MODE_DEFAULT, mem_pages);
>
>
> > +
> > +     vm = vm_create(VM_MODE_DEFAULT,
> > +             DEFAULT_GUEST_PHY_PAGES + (DEFAULT_STACK_PGS * nvcpus),
> > +             O_RDWR);
> mem_pages must be used instead
>
> augere@virtlab-arm04:~/UPSTREAM/ML/tools/testing/selftests/kvm#
> ./aarch64/id_reg_test
> ==== Test Assertion Failure ====
>   lib/kvm_util.c:825: vm_adjust_num_guest_pages(vm->mode, npages) == npages
>   pid=11439 tid=11439 errno=0 - Success
>      1  0x00000000004068cb: vm_userspace_mem_region_add at kvm_util.c:823
>      2  0x00000000004071af: vm_create at kvm_util.c:319
>      3  0x0000000000401afb: test_vm_create_cap at id_reg_test.c:508
>      4  0x00000000004014a3: test_vm_create at id_reg_test.c:541
>      5   (inlined by) init_id_reg_info at id_reg_test.c:1110
>      6   (inlined by) main at id_reg_test.c:1125
>      7  0x000003ffa7220de3: ?? ??:0
>      8  0x00000000004015eb: _start at :?
>   Number of guest pages is not compatible with the host. Try npages=528

Thank you for catching this (It didn't happen in my usual test environment).
I will fix this.

>
>
> Don't you want to check the cap in a first place using kvm_check_cap and
> cap->cap

It is done by the caller before trying to create the vm.


> > +     if (cap)
> > +             vm_enable_cap(vm, cap);
> > +
> > +     kvm_vm_elf_load(vm, program_invocation_name);
> > +
> > +     if (init && init->target == -1) {
> > +             struct kvm_vcpu_init preferred;
> > +
> > +             vm_ioctl(vm, KVM_ARM_PREFERRED_TARGET, &preferred);
> > +             init->target = preferred.target;
> > +     }
> > +
> > +     vm_init_descriptor_tables(vm);
> > +     for (cpuid = 0; cpuid < nvcpus; cpuid++) {
> > +             if (init)
> > +                     aarch64_vcpu_add_default(vm, cpuid, init, guest_code);
> > +             else
> > +                     vm_vcpu_add_default(vm, cpuid, guest_code);
> nit: vm_vcpu_add_default calls aarch64_vcpu_add_default(vm, vcpuid,
> NULL, guest_code) so you can unconditionnaly call
> aarch64_vcpu_add_default(vm, cpuid, init, guest_code)

Oh, thank you ! I will fix that (somehow I overlooked that...).

> > +
> > +             vcpu_init_descriptor_tables(vm, cpuid);
> > +     }
> > +
> > +     ucall_init(vm, NULL);
> > +     return vm;
> > +}
> > +
> > +static struct kvm_vm *test_vm_create(uint32_t nvcpus,
> > +                                  void (*guest_code)(uint32_t),
> > +                                  struct kvm_vcpu_init *init)
> > +{
> > +     return test_vm_create_cap(nvcpus, guest_code, init, NULL);
> > +}
> nit: not sure test_vm_create is needed. By the way it is already called
> with init = NULL so we can call test_vm_create_cap with 2 NULL args

I will remove test_vm_create.


> > +
> > +static void test_vm_free(struct kvm_vm *vm)
> > +{
> > +     ucall_uninit(vm);
> > +     kvm_vm_free(vm);
> > +}
> > +
> > +#define      TEST_RUN(vm, cpu)       \
> > +     (test_vcpu_run(__func__, __LINE__, vm, cpu, true))
> > +
> > +#define      TEST_RUN_NO_SYNC_DATA(vm, cpu)  \
> > +     (test_vcpu_run(__func__, __LINE__, vm, cpu, false))
> > +
> > +static int test_vcpu_run(const char *test_name, int line,
> > +                      struct kvm_vm *vm, uint32_t vcpuid, bool sync_data)
> > +{
> > +     struct ucall uc;
> > +     int ret;
> > +
> > +     if (sync_data) {
> > +             sync_global_to_guest(vm, id_reg_list);
> > +             sync_global_to_guest(vm, feature_test_info_table);
> > +     }
> > +
> > +     vcpu_args_set(vm, vcpuid, 1, vcpuid);
> > +
> > +     ret = _vcpu_run(vm, vcpuid);
> > +     if (ret) {
> > +             ret = errno;
> > +             goto sync_exit;
> > +     }
> > +
> > +     switch (get_ucall(vm, vcpuid, &uc)) {
> > +     case UCALL_SYNC:
> > +     case UCALL_DONE:
> > +             ret = 0;
> > +             break;
> > +     case UCALL_ABORT:
> > +             TEST_FAIL(
> > +                 "%s (%s) at line %d (user %s at line %d), args[3]=0x%lx",
> > +                 (char *)uc.args[0], (char *)uc.args[2], (int)uc.args[1],
> > +                 test_name, line, uc.args[3]);
> > +             break;
> > +     default:
> > +             TEST_FAIL("Unexpected guest exit\n");
> > +     }
> > +
> > +sync_exit:
> > +     if (sync_data) {
> > +             sync_global_from_guest(vm, id_reg_list);
> > +             sync_global_from_guest(vm, feature_test_info_table);
> > +     }
> > +     return ret;
> > +}
> > +
> > +static int set_id_regs_after_run_test_one(struct id_reg_test_info *sreg,
> > +                                       void *arg)
> > +{
> > +     struct kvm_vm *vm = arg;
> > +     struct kvm_one_reg one_reg;
> > +     uint32_t vcpuid = 0;
> > +     uint64_t reg_val;
> > +     int ret;
> > +
> > +     one_reg.addr = (uint64_t)&reg_val;
> > +     one_reg.id = KVM_ARM64_SYS_REG(sreg->id);
> > +
> > +     vcpu_ioctl(vm, vcpuid, KVM_GET_ONE_REG, &one_reg);
> > +     if ((reg_val != 0) && (sreg->can_clear)) {
> > +             reg_val = 0;
> > +             ret = _vcpu_ioctl(vm, vcpuid, KVM_SET_ONE_REG, &one_reg);
> > +             TEST_ASSERT(ret && errno == EINVAL,
> > +                         "Changing ID reg value should fail\n");
> > +     }
> > +
> > +     vcpu_ioctl(vm, vcpuid, KVM_GET_ONE_REG, &one_reg);> +   ret = _vcpu_ioctl(vm, vcpuid, KVM_SET_ONE_REG, &one_reg);
> > +     TEST_ASSERT(ret == 0, "Setting the same ID reg value should work\n");
> > +
> > +     return 0;
> > +}
> > +
> > +static int set_id_regs_test_one(struct id_reg_test_info *sreg, void *arg)
> if it is a test use test_ prefix

I will fix this.


> > +{
> > +     struct kvm_vm *vm = arg;
> > +     struct kvm_one_reg one_reg;
> > +     uint32_t vcpuid = 0;
> > +     uint64_t reg_val;
> > +
> > +     one_reg.addr = (uint64_t)&reg_val;
> > +     reset_id_reg_info();
> > +
> > +     one_reg.id = KVM_ARM64_SYS_REG(sreg->id);
> > +     if (sreg->can_clear) {
> > +             /* Change the register to 0 when possible */
> > +             reg_val = 0;
> > +             vcpu_ioctl(vm, vcpuid, KVM_SET_ONE_REG, &one_reg);
> > +             vcpu_ioctl(vm, vcpuid, KVM_GET_ONE_REG, &one_reg);
> > +             TEST_ASSERT(reg_val == 0,
> > +                 "GET(%s) didn't return 0 but 0x%lx", sreg->name, reg_val);
> > +     }
> > +
> > +     /* Check if we can restore the initial value */
> > +     reg_val = sreg->org_val;
> > +     vcpu_ioctl(vm, vcpuid, KVM_SET_ONE_REG, &one_reg);
> > +     vcpu_ioctl(vm, vcpuid, KVM_GET_ONE_REG, &one_reg);
> > +     TEST_ASSERT(reg_val == sreg->org_val,
> > +                 "GET(%s) didn't return 0x%lx but 0x%lx",
> > +                 sreg->name, sreg->org_val, reg_val);
> > +     sreg->user_val = sreg->org_val;
> > +     return 0;
> > +}
> > +
> > +static void set_id_regs_test(void)
> if it is a test use test_ prefix

I will fix this.

> > +{
> > +     struct kvm_vm *vm;
> > +     int ret;
> > +
> > +     reset_id_reg_info();
> > +     vm = test_vm_create(1, guest_code_id_reg_check_all, NULL);
> add test_vm_free()

I will fix this.

> > +
> > +     ret = walk_id_reg_list(set_id_regs_test_one, vm);
> > +     assert(!ret);
> > +
> > +     ret = TEST_RUN(vm, 0);
> > +     TEST_ASSERT(!ret, "%s TEST_RUN failed, ret=0x%x", __func__, ret);
> > +
> > +     ret = walk_id_reg_list(set_id_regs_after_run_test_one, vm);
> > +     assert(!ret);
> > +}
> > +
> > +static bool caps_are_supported(long *caps, int ncaps)
> > +{
> > +     int i;
> > +
> > +     for (i = 0; i < ncaps; i++) {
> > +             if (kvm_check_cap(caps[i]) <= 0)
> > +                     return false;
> > +     }
> > +     return true;
> > +}
> > +
> > +static void test_feature_ptrauth(void)
> > +{
> > +     struct kvm_one_reg one_reg;
> > +     struct kvm_vcpu_init init;
> > +     struct kvm_vm *vm = NULL;
> > +     struct id_reg_test_info *sreg = ID_REG_INFO(ID_AA64ISAR1);
> > +     uint32_t vcpu = 0;
> > +     int64_t rval;
> > +     int ret;
> > +     int apa, api, gpa, gpi;
> > +     char *name = "PTRAUTH";
> > +     long caps[2] = {KVM_CAP_ARM_PTRAUTH_ADDRESS,
> > +                     KVM_CAP_ARM_PTRAUTH_GENERIC};
> > +
> > +     reset_id_reg_info();
> > +     one_reg.addr = (uint64_t)&rval;
> > +     one_reg.id = KVM_ARM64_SYS_REG(sreg->id);
> > +
> > +     if (caps_are_supported(caps, 2)) {
> > +
> > +             /* Test with feature enabled */
> > +             memset(&init, 0, sizeof(init));
> > +             init.target = -1;
> > +             init.features[0] = (1ULL << KVM_ARM_VCPU_PTRAUTH_ADDRESS |
> > +                                 1ULL << KVM_ARM_VCPU_PTRAUTH_GENERIC);
> > +             vm = test_vm_create_cap(1, guest_code_ptrauth_check, &init,
> > +                                     NULL);
> > +             vcpu_ioctl(vm, vcpu, KVM_GET_ONE_REG, &one_reg);
> > +             apa = GET_ID_UFIELD(rval, ID_AA64ISAR1_APA_SHIFT);
> > +             api = GET_ID_UFIELD(rval, ID_AA64ISAR1_API_SHIFT);
> > +             gpa = GET_ID_UFIELD(rval, ID_AA64ISAR1_GPA_SHIFT);
> > +             gpi = GET_ID_UFIELD(rval, ID_AA64ISAR1_GPI_SHIFT);
> > +
> > +             TEST_ASSERT((apa > 0) || (api > 0),
> > +                         "Either apa(0x%x) or api(0x%x) must be available",
> > +                         apa, gpa);
> > +             TEST_ASSERT((gpa > 0) || (gpi > 0),
> > +                         "Either gpa(0x%x) or gpi(0x%x) must be available",
> > +                         gpa, gpi);
> > +
> > +             TEST_ASSERT((apa > 0) ^ (api > 0),
> > +                         "Both apa(0x%x) and api(0x%x) must not be available",
> > +                         apa, api);
> > +             TEST_ASSERT((gpa > 0) ^ (gpi > 0),
> > +                         "Both gpa(0x%x) and gpi(0x%x) must not be available",
> > +                         gpa, gpi);
> > +
> > +             sreg->user_val = rval;
> > +
> > +             pr_debug("%s: Test with %s enabled (%s: 0x%lx)\n",
> > +                      __func__, name, sreg->name, sreg->user_val);
> > +             ret = TEST_RUN(vm, vcpu);
> > +             TEST_ASSERT(!ret, "%s:KVM_RUN failed with %s enabled",
> > +                         __func__, name);
> > +             test_vm_free(vm);
> > +     }
> > +
> > +     /* Test with feature disabled */
> > +     reset_id_reg_info();
> > +
> > +     vm = test_vm_create(1, guest_code_feature_check, NULL);
> > +     vcpu_ioctl(vm, vcpu, KVM_GET_ONE_REG, &one_reg);
> > +
> > +     apa = GET_ID_UFIELD(rval, ID_AA64ISAR1_APA_SHIFT);
> > +     api = GET_ID_UFIELD(rval, ID_AA64ISAR1_API_SHIFT);
> > +     gpa = GET_ID_UFIELD(rval, ID_AA64ISAR1_GPA_SHIFT);
> > +     gpi = GET_ID_UFIELD(rval, ID_AA64ISAR1_GPI_SHIFT);
> > +     TEST_ASSERT(!apa && !api && !gpa && !gpi,
> > +         "apa(0x%x), api(0x%x), gpa(0x%x), gpi(0x%x) must be zero",
> > +         apa, api, gpa, gpi);
> > +
> > +     pr_debug("%s: Test with %s disabled (%s: 0x%lx)\n",
> > +              __func__, name, sreg->name, sreg->user_val);
> > +
> > +     ret = TEST_RUN(vm, vcpu);
> > +     TEST_ASSERT(!ret, "%s TEST_RUN failed with %s enabled, ret=0x%x",
> > +                 __func__, name, ret);
> > +
> > +     test_vm_free(vm);
> > +}
> > +
> > +static bool feature_caps_are_available(struct feature_test_info *finfo)
> > +{
> > +     return ((finfo->ncaps > 0) &&
> > +             caps_are_supported(finfo->caps, finfo->ncaps));
> > +}
> > +
> comment with short explanation of what the test does

Yes, I will add a short explanation for each test.

> > +static void test_feature(struct feature_test_info *finfo)
> > +{
> > +     struct id_reg_test_info *sreg = finfo->sreg;
> > +     struct kvm_one_reg one_reg;
> > +     struct kvm_vcpu_init init, *initp = NULL;
> > +     struct kvm_vm *vm = NULL;
> > +     int64_t fval, reg_val;
> > +     uint32_t vcpu = 0;
> > +     bool is_sign = finfo->is_sign;
> > +     int min = finfo->min;
> > +     int shift = finfo->shift;
> > +     int ret;
> > +
> > +     pr_debug("%s: %s (reg %s)\n", __func__, finfo->name, sreg->name);
> > +
> > +     reset_id_reg_info();
> > +     finfo->run_test = 1;    /* Indicate that guest runs the test on it */
> > +     one_reg.addr = (uint64_t)&reg_val;
> > +     one_reg.id = KVM_ARM64_SYS_REG(sreg->id);
> > +
> > +     /* Test with feature enabled if the feature is available */
> s/if the feature is available/if the feature is exposed in the default
> ID_REG value and if the capabilities are supported at KVM level

Thank you for the suggestion. I will fix that.


> > +void run_test(void)
> > +{
> > +     set_id_regs_test();
> > +     test_feature_all();
> > +     test_feature_ptrauth();
> > +     test_feature_frac_all();
> > +}
> > +
> basic comment would be helpful: attempts to clear a given id_reg and
> populate the id_reg with the original value, and can_clear flag?

I will add some comments.

> > +static int init_id_reg_info_one(struct id_reg_test_info *sreg, void *arg)
> > +{
> > +     uint64_t reg_val;
> > +     uint32_t vcpuid = 0;
> > +     int ret;
> > +     struct kvm_one_reg one_reg;
> > +     struct kvm_vm *vm = arg;
> > +
> > +     one_reg.addr = (uint64_t)&reg_val;
> > +     one_reg.id = KVM_ARM64_SYS_REG(sreg->id);
> > +     vcpu_ioctl(vm, vcpuid, KVM_GET_ONE_REG, &one_reg);
> > +     sreg->org_val = reg_val;
> > +     sreg->user_val = reg_val;
> nit: add a comment for user_val because it is not obvious why you set it
> to reg_val.

I will add a comment for it.

> > +     if (sreg->org_val) {
> > +             reg_val = 0;
> > +             ret = _vcpu_ioctl(vm, vcpuid, KVM_SET_ONE_REG, &one_reg);
> > +             if (!ret)
> > +                     sreg->can_clear = true;
> > +     }
> > +
> > +     pr_debug("%s (0x%x): 0x%lx%s\n", sreg->name, sreg->id,
> > +              sreg->org_val, sreg->can_clear ? ", can clear" : "");
> > +
> > +     return 0;
> > +}
> > +
> add a comment? loop over the idreg list and populates each regid info
> with the default, user and can_clear value

I will add a comment for the function.

> > +static void init_id_reg_info(void)
> > +{
> > +     struct kvm_vm *vm;
> > +
> > +     vm = test_vm_create(1, guest_code_do_nothing, NULL);
> > +     walk_id_reg_list(init_id_reg_info_one, vm);
> > +     test_vm_free(vm);
> > +}
> > +
> > +int main(void)
> > +{
> > +
> > +     setbuf(stdout, NULL);
> > +
> > +     if (kvm_check_cap(KVM_CAP_ARM_ID_REG_CONFIGURABLE) <= 0) {
> > +             print_skip("KVM_CAP_ARM_ID_REG_CONFIGURABLE is not supported\n");
> > +             exit(KSFT_SKIP);
> > +     }
> > +
> > +     init_id_reg_info();
> > +     run_test();
> > +     return 0;
> > +}
> >
>
> After fixing the mem_pages stuff I get the following error on a cavium
> machine.
>
> augere@virtlab-arm04:~/UPSTREAM/ML/tools/testing/selftests/kvm#
> ./aarch64/id_reg_test
> ==== Test Assertion Failure ====
>   aarch64/id_reg_test.c:814: fval >= min
>   pid=11692 tid=11692 errno=4 - Interrupted system call
>      1  0x00000000004028d3: test_feature at id_reg_test.c:813
>      2   (inlined by) test_feature_all at id_reg_test.c:863
>      3   (inlined by) run_test at id_reg_test.c:1073
>      4  0x000000000040156f: main at id_reg_test.c:1124
>      5  0x000003ffa9420de3: ?? ??:0
>      6  0x00000000004015eb: _start at :?
>   PERFMON field of ID_DFR0 is too small (0)
>
> Fails on
> test_feature: PERFMON (reg ID_DFR0)
>
> I will do my utmost to further debug

Thank you for running it in your environment and reporting this !
This is very interesting...

It implies that the host's ID_DFR0_EL1.PerfMon is zero or 0xf
(meaning FEAT_PMUv3 is not implemented) even though the host's
ID_AA64DFR0_EL1.PMUVer indicates that FEAT_PMUv3 is implemented.

Would it be possible for you to check values of those two
registers on the host (not on the guest) to see if both of them
indicate the presence of FEAT_PMUv3 consistently ?

Thanks,
Reiji

Re: [RFC PATCH v3 01/29] KVM: arm64: Add has_reset_once flag for vcpu

[Marc Zyngier]

On Wed, 17 Nov 2021 06:43:31 +0000,
Reiji Watanabe <reijiw@google.com> wrote:
> 
> Introduce 'has_reset_once' flag in kvm_vcpu_arch, which indicates
> if the vCPU reset has been done once, for later use.

It would be nice if you could at least hint at what this flag is going
to be used for, as being able to reset a vcpu as often as userspace
wants it is part of the KVM ABI.

> 
> Signed-off-by: Reiji Watanabe <reijiw@google.com>
> Reviewed-by: Oliver Upton <oupton@google.com>
> ---
>  arch/arm64/include/asm/kvm_host.h | 2 ++
>  arch/arm64/kvm/reset.c            | 4 ++++
>  2 files changed, 6 insertions(+)
> 
> diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
> index 2a5f7f38006f..edbe2cb21947 100644
> --- a/arch/arm64/include/asm/kvm_host.h
> +++ b/arch/arm64/include/asm/kvm_host.h
> @@ -385,6 +385,7 @@ struct kvm_vcpu_arch {
>  		u64 last_steal;
>  		gpa_t base;
>  	} steal;
> +	bool has_reset_once;

Why can't this be a new flag (part of the vcpu->arch.flags set) rather
than a discrete bool?

Thanks,

	M.

-- 
Without deviation from the norm, progress is not possible.

Re: [RFC PATCH v3 02/29] KVM: arm64: Save ID registers' sanitized value per vCPU

[Marc Zyngier]

On Wed, 17 Nov 2021 06:43:32 +0000,
Reiji Watanabe <reijiw@google.com> wrote:
> 
> Extend sys_regs[] of kvm_cpu_context for ID registers and save ID
> registers' sanitized value in the array for the vCPU at the first
> vCPU reset. Use the saved ones when ID registers are read by
> userspace (via KVM_GET_ONE_REG) or the guest.
> 
> Signed-off-by: Reiji Watanabe <reijiw@google.com>
> ---
>  arch/arm64/include/asm/kvm_host.h | 10 +++++++
>  arch/arm64/kvm/sys_regs.c         | 43 +++++++++++++++++++------------
>  2 files changed, 37 insertions(+), 16 deletions(-)
> 
> diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
> index edbe2cb21947..72db73c79403 100644
> --- a/arch/arm64/include/asm/kvm_host.h
> +++ b/arch/arm64/include/asm/kvm_host.h
> @@ -146,6 +146,14 @@ struct kvm_vcpu_fault_info {
>  	u64 disr_el1;		/* Deferred [SError] Status Register */
>  };
>  
> +/*
> + * (Op0, Op1, CRn, CRm, Op2) of ID registers is (3, 0, 0, crm, op2),
> + * where 0<=crm<8, 0<=op2<8.
> + */
> +#define KVM_ARM_ID_REG_MAX_NUM 64
> +#define IDREG_IDX(id)		((sys_reg_CRm(id) << 3) | sys_reg_Op2(id))
> +#define IDREG_SYS_IDX(id)	(ID_REG_BASE + IDREG_IDX(id))
> +
>  enum vcpu_sysreg {
>  	__INVALID_SYSREG__,   /* 0 is reserved as an invalid value */
>  	MPIDR_EL1,	/* MultiProcessor Affinity Register */
> @@ -210,6 +218,8 @@ enum vcpu_sysreg {
>  	CNTP_CVAL_EL0,
>  	CNTP_CTL_EL0,
>  
> +	ID_REG_BASE,
> +	ID_REG_END = ID_REG_BASE + KVM_ARM_ID_REG_MAX_NUM - 1,

It is rather unclear to me why we want these registers to be
replicated on a per-CPU basis. Yes, this fits the architecture, but
that's also a total waste of memory if you have more than a single
CPU, because we make a point in only exposing homogeneous properties
to the VM (I don't think anyone intends to support vcpu asymmetry in a
VM, and 64 registers per vcpu is not an insignificant memory usage).

If there are no reasons for this to be per-CPU, please move it to be
global to the VM. This also mean that once a vcpu has reset, it
shouldn't be possible to affect the registers. This shouldn't affect
the userspace API though.

Thanks,

	M.

-- 
Without deviation from the norm, progress is not possible.

Re: [RFC PATCH v3 03/29] KVM: arm64: Introduce struct id_reg_info

[Marc Zyngier]

On Wed, 17 Nov 2021 06:43:33 +0000,
Reiji Watanabe <reijiw@google.com> wrote:
> 
> This patch lays the groundwork to make ID registers writable.
> 
> Introduce struct id_reg_info for an ID register to manage the
> register specific control of its value for the guest, and provide set
> of functions commonly used for ID registers to make them writable.
> 
> The id_reg_info is used to do register specific initialization,
> validation of the ID register and etc.  Not all ID registers must
> have the id_reg_info. ID registers that don't have the id_reg_info
> are handled in a common way that is applied to all ID registers.
> 
> At present, changing an ID register from userspace is allowed only
> if the ID register has the id_reg_info, but that will be changed
> by the following patches.
> 
> No ID register has the structure yet and the following patches
> will add the id_reg_info for some ID registers.
> 
> Signed-off-by: Reiji Watanabe <reijiw@google.com>
> ---
>  arch/arm64/include/asm/sysreg.h |   1 +
>  arch/arm64/kvm/sys_regs.c       | 226 ++++++++++++++++++++++++++++++--
>  2 files changed, 218 insertions(+), 9 deletions(-)
> 
> diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h
> index 16b3f1a1d468..597609f26331 100644
> --- a/arch/arm64/include/asm/sysreg.h
> +++ b/arch/arm64/include/asm/sysreg.h
> @@ -1197,6 +1197,7 @@
>  #define ICH_VTR_TDS_MASK	(1 << ICH_VTR_TDS_SHIFT)
>  
>  #define ARM64_FEATURE_FIELD_BITS	4
> +#define ARM64_FEATURE_FIELD_MASK	((1ull << ARM64_FEATURE_FIELD_BITS) - 1)
>  
>  /* Create a mask for the feature bits of the specified feature. */
>  #define ARM64_FEATURE_MASK(x)	(GENMASK_ULL(x##_SHIFT + ARM64_FEATURE_FIELD_BITS - 1, x##_SHIFT))
> diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
> index 5608d3410660..1552cd5581b7 100644
> --- a/arch/arm64/kvm/sys_regs.c
> +++ b/arch/arm64/kvm/sys_regs.c
> @@ -265,6 +265,181 @@ static bool trap_raz_wi(struct kvm_vcpu *vcpu,
>  		return read_zero(vcpu, p);
>  }
>  
> +/*
> + * A value for FCT_LOWER_SAFE must be zero and changing that will affect
> + * ftr_check_types of id_reg_info.
> + */
> +enum feature_check_type {
> +	FCT_LOWER_SAFE = 0,
> +	FCT_HIGHER_SAFE,
> +	FCT_HIGHER_OR_ZERO_SAFE,
> +	FCT_EXACT,
> +	FCT_EXACT_OR_ZERO_SAFE,
> +	FCT_IGNORE,	/* Don't check (any value is fine) */
> +};
> +
> +static int arm64_check_feature_one(enum feature_check_type type, int val,
> +				   int limit)
> +{
> +	bool is_safe = false;
> +
> +	if (val == limit)
> +		return 0;
> +
> +	switch (type) {
> +	case FCT_LOWER_SAFE:
> +		is_safe = (val <= limit);
> +		break;
> +	case FCT_HIGHER_OR_ZERO_SAFE:
> +		if (val == 0) {
> +			is_safe = true;
> +			break;
> +		}
> +		fallthrough;
> +	case FCT_HIGHER_SAFE:
> +		is_safe = (val >= limit);
> +		break;
> +	case FCT_EXACT:
> +		break;
> +	case FCT_EXACT_OR_ZERO_SAFE:
> +		is_safe = (val == 0);
> +		break;
> +	case FCT_IGNORE:
> +		is_safe = true;
> +		break;
> +	default:
> +		WARN_ONCE(1, "Unexpected feature_check_type (%d)\n", type);
> +		break;
> +	}
> +
> +	return is_safe ? 0 : -1;
> +}
> +
> +#define	FCT_TYPE_MASK		0x7
> +#define	FCT_TYPE_SHIFT		1
> +#define	FCT_SIGN_MASK		0x1
> +#define	FCT_SIGN_SHIFT		0
> +#define	FCT_TYPE(val)	((val >> FCT_TYPE_SHIFT) & FCT_TYPE_MASK)
> +#define	FCT_SIGN(val)	((val >> FCT_SIGN_SHIFT) & FCT_SIGN_MASK)
> +
> +#define	MAKE_FCT(shift, type, sign)				\
> +	((u64)((((type) & FCT_TYPE_MASK) << FCT_TYPE_SHIFT) |	\
> +	       (((sign) & FCT_SIGN_MASK) << FCT_SIGN_SHIFT)) << (shift))
> +
> +/* For signed field */
> +#define	S_FCT(shift, type)	MAKE_FCT(shift, type, 1)
> +/* For unigned field */
> +#define	U_FCT(shift, type)	MAKE_FCT(shift, type, 0)
> +
> +/*
> + * @val and @lim are both a value of the ID register. The function checks
> + * if all features indicated in @val can be supported for guests on the host,
> + * which supports features indicated in @lim. @check_types indicates how
> + * features in the ID register needs to be checked.
> + * See comments for id_reg_info's ftr_check_types field for more detail.
> + */
> +static int arm64_check_features(u64 check_types, u64 val, u64 lim)
> +{
> +	int i;
> +
> +	for (i = 0; i < 64; i += ARM64_FEATURE_FIELD_BITS) {
> +		u8 ftr_check = (check_types >> i) & ARM64_FEATURE_FIELD_MASK;
> +		bool is_sign = FCT_SIGN(ftr_check);
> +		enum feature_check_type fctype = FCT_TYPE(ftr_check);
> +		int fval, flim, ret;
> +
> +		fval = cpuid_feature_extract_field(val, i, is_sign);
> +		flim = cpuid_feature_extract_field(lim, i, is_sign);
> +
> +		ret = arm64_check_feature_one(fctype, fval, flim);
> +		if (ret)
> +			return -E2BIG;
> +	}
> +	return 0;
> +}

All this logic seems to reinvent what we already have in
arch/arm64/kernel/cpufeature.c. I'd rather we rely on it and maintain
a single idreg handling library.

Could you outline what is missing in the cpufeature code that requires
you to invent your own? I'm sure Suzuki could help here to make it
directly usable.

Thanks,

	M.

-- 
Without deviation from the norm, progress is not possible.

Re: [RFC PATCH v3 03/29] KVM: arm64: Introduce struct id_reg_info

[Marc Zyngier]

On Fri, 19 Nov 2021 04:47:53 +0000,
Reiji Watanabe <reijiw@google.com> wrote:
> 
> I am going to add the following comment. Does it look clear enough for you ?
> 
>         /*
>          * This is an optional ID register specific validation function.
>          * When userspace tries to set the ID register, arm64_check_features()
>          * will check if the requested value indicates any features that cannot
>          * be supported by KVM on the host.  But, some ID register fields need
>          * a special checking and this function can be used for such fields.
>          * e.g. KVM_CREATE_DEVICE must be used to configure GICv3 for a guest.
>          * ID_AA64PFR0_EL1.GIC shouldn't be set to 1 unless GICv3 is configured.

There is no such requirement. GICv3 has a compatibility interface, and
although KVM doesn't make use of it, there is no reason not to expose
that GICv3 exists on the CPU even if not using it. Even more, this has
been the case forever, and making this change now would probably break
migration.

	M.

-- 
Without deviation from the norm, progress is not possible.

Re: [RFC PATCH v3 04/29] KVM: arm64: Make ID_AA64PFR0_EL1 writable

[Marc Zyngier]

On Wed, 17 Nov 2021 06:43:34 +0000,
Reiji Watanabe <reijiw@google.com> wrote:
> 
> This patch adds id_reg_info for ID_AA64PFR0_EL1 to make it writable by
> userspace.
> 
> The CSV2/CSV3 fields of the register were already writable and values
> that were written for them affected all vCPUs before. Now they only
> affect the vCPU.
> Return an error if userspace tries to set SVE/GIC field of the register
> to a value that conflicts with SVE/GIC configuration for the guest.
> SIMD/FP/SVE fields of the requested value are validated according to
> Arm ARM.
> 
> Signed-off-by: Reiji Watanabe <reijiw@google.com>
> ---
>  arch/arm64/kvm/sys_regs.c | 159 ++++++++++++++++++++++++--------------
>  1 file changed, 103 insertions(+), 56 deletions(-)
> 
> diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
> index 1552cd5581b7..35400869067a 100644
> --- a/arch/arm64/kvm/sys_regs.c
> +++ b/arch/arm64/kvm/sys_regs.c
> @@ -401,6 +401,92 @@ static void id_reg_info_init(struct id_reg_info *id_reg)
>  		id_reg->init(id_reg);
>  }
>  
> +#define	kvm_has_gic3(kvm)		\
> +	(irqchip_in_kernel(kvm) &&	\
> +	 (kvm)->arch.vgic.vgic_model == KVM_DEV_TYPE_ARM_VGIC_V3)
> +
> +static int validate_id_aa64pfr0_el1(struct kvm_vcpu *vcpu,
> +				    const struct id_reg_info *id_reg, u64 val)
> +{
> +	int fp, simd;
> +	bool vcpu_has_sve = vcpu_has_sve(vcpu);
> +	bool pfr0_has_sve = id_aa64pfr0_sve(val);
> +	int gic;
> +
> +	simd = cpuid_feature_extract_signed_field(val, ID_AA64PFR0_ASIMD_SHIFT);
> +	fp = cpuid_feature_extract_signed_field(val, ID_AA64PFR0_FP_SHIFT);
> +	if (simd != fp)
> +		return -EINVAL;
> +
> +	/* fp must be supported when sve is supported */
> +	if (pfr0_has_sve && (fp < 0))
> +		return -EINVAL;
> +
> +	/* Check if there is a conflict with a request via KVM_ARM_VCPU_INIT */
> +	if (vcpu_has_sve ^ pfr0_has_sve)
> +		return -EPERM;
> +
> +	gic = cpuid_feature_extract_unsigned_field(val, ID_AA64PFR0_GIC_SHIFT);
> +	if ((gic > 0) ^ kvm_has_gic3(vcpu->kvm))
> +		return -EPERM;
> +
> +	return 0;
> +}
> +
> +static void init_id_aa64pfr0_el1_info(struct id_reg_info *id_reg)
> +{
> +	u64 limit = id_reg->vcpu_limit_val;
> +	unsigned int gic;
> +
> +	limit &= ~ARM64_FEATURE_MASK(ID_AA64PFR0_AMU);
> +	if (!system_supports_sve())
> +		limit &= ~ARM64_FEATURE_MASK(ID_AA64PFR0_SVE);
> +
> +	/*
> +	 * The default is to expose CSV2 == 1 and CSV3 == 1 if the HW
> +	 * isn't affected.  Userspace can override this as long as it
> +	 * doesn't promise the impossible.
> +	 */
> +	limit &= ~(ARM64_FEATURE_MASK(ID_AA64PFR0_CSV2) |
> +		   ARM64_FEATURE_MASK(ID_AA64PFR0_CSV3));
> +
> +	if (arm64_get_spectre_v2_state() == SPECTRE_UNAFFECTED)
> +		limit |= FIELD_PREP(ARM64_FEATURE_MASK(ID_AA64PFR0_CSV2), 1);
> +	if (arm64_get_meltdown_state() == SPECTRE_UNAFFECTED)
> +		limit |= FIELD_PREP(ARM64_FEATURE_MASK(ID_AA64PFR0_CSV3), 1);
> +
> +	gic = cpuid_feature_extract_unsigned_field(limit, ID_AA64PFR0_GIC_SHIFT);
> +	if (gic > 1) {
> +		/* Limit to GICv3.0/4.0 */
> +		limit &= ~ARM64_FEATURE_MASK(ID_AA64PFR0_GIC);
> +		limit |= FIELD_PREP(ARM64_FEATURE_MASK(ID_AA64PFR0_GIC), 1);
> +	}
> +	id_reg->vcpu_limit_val = limit;
> +}
> +
> +static u64 get_reset_id_aa64pfr0_el1(struct kvm_vcpu *vcpu,
> +				     const struct id_reg_info *idr)
> +{
> +	u64 val = idr->vcpu_limit_val;
> +
> +	if (!vcpu_has_sve(vcpu))
> +		val &= ~ARM64_FEATURE_MASK(ID_AA64PFR0_SVE);
> +
> +	if (!kvm_has_gic3(vcpu->kvm))
> +		val &= ~ARM64_FEATURE_MASK(ID_AA64PFR0_GIC);

No. As I said in a previous email, this breaks migration, and
advertising a GICv3 CPU interface doesn't mean it is usable (the guest
OS must check that it can actually enable ICC_SRE_EL1.SRE -- see what
the Linux GICv3 driver does for an example).

> +
> +	return val;
> +}
> +
> +static struct id_reg_info id_aa64pfr0_el1_info = {
> +	.sys_reg = SYS_ID_AA64PFR0_EL1,
> +	.ftr_check_types = S_FCT(ID_AA64PFR0_ASIMD_SHIFT, FCT_LOWER_SAFE) |
> +			   S_FCT(ID_AA64PFR0_FP_SHIFT, FCT_LOWER_SAFE),
> +	.init = init_id_aa64pfr0_el1_info,
> +	.validate = validate_id_aa64pfr0_el1,
> +	.get_reset_val = get_reset_id_aa64pfr0_el1,
> +};
> +
>  /*
>   * An ID register that needs special handling to control the value for the
>   * guest must have its own id_reg_info in id_reg_info_table.
> @@ -409,7 +495,9 @@ static void id_reg_info_init(struct id_reg_info *id_reg)
>   * validation, etc.)
>   */
>  #define	GET_ID_REG_INFO(id)	(id_reg_info_table[IDREG_IDX(id)])
> -static struct id_reg_info *id_reg_info_table[KVM_ARM_ID_REG_MAX_NUM] = {};
> +static struct id_reg_info *id_reg_info_table[KVM_ARM_ID_REG_MAX_NUM] = {
> +	[IDREG_IDX(SYS_ID_AA64PFR0_EL1)] = &id_aa64pfr0_el1_info,
> +};
>  
>  static int validate_id_reg(struct kvm_vcpu *vcpu,
>  			   const struct sys_reg_desc *rd, u64 val)
> @@ -1239,20 +1327,22 @@ static bool access_arch_timer(struct kvm_vcpu *vcpu,
>  static u64 __read_id_reg(const struct kvm_vcpu *vcpu, u32 id)
>  {
>  	u64 val = __vcpu_sys_reg(vcpu, IDREG_SYS_IDX(id));
> +	u64 lim, gic, gic_lim;
> +	const struct id_reg_info *id_reg;
>  
>  	switch (id) {
>  	case SYS_ID_AA64PFR0_EL1:
> -		if (!vcpu_has_sve(vcpu))
> -			val &= ~ARM64_FEATURE_MASK(ID_AA64PFR0_SVE);
> -		val &= ~ARM64_FEATURE_MASK(ID_AA64PFR0_AMU);
> -		val &= ~ARM64_FEATURE_MASK(ID_AA64PFR0_CSV2);
> -		val |= FIELD_PREP(ARM64_FEATURE_MASK(ID_AA64PFR0_CSV2), (u64)vcpu->kvm->arch.pfr0_csv2);
> -		val &= ~ARM64_FEATURE_MASK(ID_AA64PFR0_CSV3);
> -		val |= FIELD_PREP(ARM64_FEATURE_MASK(ID_AA64PFR0_CSV3), (u64)vcpu->kvm->arch.pfr0_csv3);
> -		if (irqchip_in_kernel(vcpu->kvm) &&
> -		    vcpu->kvm->arch.vgic.vgic_model == KVM_DEV_TYPE_ARM_VGIC_V3) {
> -			val &= ~ARM64_FEATURE_MASK(ID_AA64PFR0_GIC);
> -			val |= FIELD_PREP(ARM64_FEATURE_MASK(ID_AA64PFR0_GIC), 1);
> +		gic = cpuid_feature_extract_unsigned_field(val, ID_AA64PFR0_GIC_SHIFT);
> +		if (kvm_has_gic3(vcpu->kvm) && (gic == 0)) {
> +			/*
> +			 * This is a case where userspace configured gic3 after
> +			 * the vcpu was created, and then it didn't set
> +			 * ID_AA64PFR0_EL1.
> +			 */

Shouldn't that be done at the point where a GICv3 is created, rather
than after the fact?

Thanks,

	M.

-- 
Without deviation from the norm, progress is not possible.

Re: [RFC PATCH v3 21/29] KVM: arm64: Introduce framework to trap disabled features

[Marc Zyngier]

On Wed, 17 Nov 2021 06:43:51 +0000,
Reiji Watanabe <reijiw@google.com> wrote:
> 
> When a CPU feature that is supported on the host is not exposed to
> its guest, emulating a real CPU's behavior (by trapping or disabling
> guest's using the feature) is generally a desirable behavior (when
> it's possible without any or little side effect).
> 
> Introduce feature_config_ctrl structure, which manages feature
> information to program configuration register to trap or disable
> the feature when the feature is not exposed to the guest, and
> functions that uses the structure to activate trapping the feature.
> 
> At present, no feature has feature_config_ctrl yet and the following
> patches will add the feature_config_ctrl for several features.
> 
> Signed-off-by: Reiji Watanabe <reijiw@google.com>
> ---
>  arch/arm64/kvm/sys_regs.c | 121 +++++++++++++++++++++++++++++++++++++-
>  1 file changed, 120 insertions(+), 1 deletion(-)
> 
> diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
> index 2f96103fc0d2..501de08dacb7 100644
> --- a/arch/arm64/kvm/sys_regs.c
> +++ b/arch/arm64/kvm/sys_regs.c
> @@ -376,8 +376,38 @@ static int arm64_check_features(u64 check_types, u64 val, u64 lim)
>  	(cpuid_feature_extract_unsigned_field(val, ID_AA64ISAR1_GPI_SHIFT) >= \
>  	 ID_AA64ISAR1_GPI_IMP_DEF)
>  
> +enum vcpu_config_reg {
> +	VCPU_HCR_EL2 = 1,
> +	VCPU_MDCR_EL2,
> +	VCPU_CPTR_EL2,
> +};
> +
> +/*
> + * Feature information to program configuration register to trap or disable
> + * guest's using a feature when the feature is not exposed to the guest.
> + */
> +struct feature_config_ctrl {
> +	/* ID register/field for the feature */
> +	u32	ftr_reg;	/* ID register */
> +	bool	ftr_signed;	/* Is the feature field signed ? */
> +	u8	ftr_shift;	/* Field of ID register for the feature */
> +	s8	ftr_min;	/* Min value that indicate the feature */
> +
> +	/*
> +	 * Function to check trapping is needed. This is used when the above
> +	 * fields are not enough to determine if trapping is needed.
> +	 */
> +	bool	(*ftr_need_trap)(struct kvm_vcpu *vcpu);
> +
> +	/* Configuration register information to trap the feature. */
> +	enum vcpu_config_reg cfg_reg;	/* Configuration register */
> +	u64	cfg_mask;	/* Field of the configuration register */
> +	u64	cfg_val;	/* Value that are set for the field */

Although this probably works for the use cases you have in mind, some
trap bits are actually working the other way around (clear to trap).
So you probably want to turn this into cfg_set and add a cfg_clear for
a good measure, dropping cfg_mask in the process.

That being said, the current trend is to move to FGT, meaning that a
single register is unlikely to cut it in the long run. I'd rather you
simply have a configuration function here (and the helper you already
have is probably enough).

> +};
> +
>  struct id_reg_info {
>  	u32	sys_reg;	/* Register ID */
> +	u64	sys_val;	/* Sanitized system value */
>  
>  	/*
>  	 * Limit value of the register for a vcpu. The value is the sanitized
> @@ -410,11 +440,15 @@ struct id_reg_info {
>  	/* Return the reset value of the register for the vCPU */
>  	u64 (*get_reset_val)(struct kvm_vcpu *vcpu,
>  			     const struct id_reg_info *id_reg);
> +
> +	/* Information to trap features that are disabled for the guest */
> +	const struct feature_config_ctrl *(*trap_features)[];
>  };
>  
>  static void id_reg_info_init(struct id_reg_info *id_reg)
>  {
> -	id_reg->vcpu_limit_val = read_sanitised_ftr_reg(id_reg->sys_reg);
> +	id_reg->sys_val = read_sanitised_ftr_reg(id_reg->sys_reg);
> +	id_reg->vcpu_limit_val = id_reg->sys_val;
>  	if (id_reg->init)
>  		id_reg->init(id_reg);
>  }
> @@ -952,6 +986,47 @@ static int validate_id_reg(struct kvm_vcpu *vcpu,
>  	return err;
>  }
>  
> +static void feature_trap_activate(struct kvm_vcpu *vcpu,
> +				  const struct feature_config_ctrl *config)
> +{
> +	u64 *reg_ptr, reg_val;
> +
> +	switch (config->cfg_reg) {
> +	case VCPU_HCR_EL2:
> +		reg_ptr = &vcpu->arch.hcr_el2;
> +		break;
> +	case VCPU_MDCR_EL2:
> +		reg_ptr = &vcpu->arch.mdcr_el2;
> +		break;
> +	case VCPU_CPTR_EL2:
> +		reg_ptr = &vcpu->arch.cptr_el2;
> +		break;
> +	}
> +
> +	/* Update cfg_mask fields with cfg_val */
> +	reg_val = (*reg_ptr & ~config->cfg_mask);
> +	reg_val |= config->cfg_val;
> +	*reg_ptr = reg_val;
> +}
> +
> +static inline bool feature_avail(const struct feature_config_ctrl *ctrl,
> +				 u64 id_val)
> +{
> +	int field_val = cpuid_feature_extract_field(id_val,
> +				ctrl->ftr_shift, ctrl->ftr_signed);
> +
> +	return (field_val >= ctrl->ftr_min);
> +}
> +
> +static inline bool vcpu_feature_is_available(struct kvm_vcpu *vcpu,
> +					const struct feature_config_ctrl *ctrl)
> +{
> +	u64 val;
> +
> +	val = __read_id_reg(vcpu, ctrl->ftr_reg);
> +	return feature_avail(ctrl, val);
> +}
> +
>  /*
>   * ARMv8.1 mandates at least a trivial LORegion implementation, where all the
>   * RW registers are RES0 (which we can implement as RAZ/WI). On an ARMv8.0
> @@ -1831,6 +1906,42 @@ static int reg_from_user(u64 *val, const void __user *uaddr, u64 id);
>  static int reg_to_user(void __user *uaddr, const u64 *val, u64 id);
>  static u64 sys_reg_to_index(const struct sys_reg_desc *reg);
>  
> +static void id_reg_features_trap_activate(struct kvm_vcpu *vcpu,
> +					  const struct id_reg_info *id_reg)
> +{
> +	u64 val;
> +	int i = 0;
> +	const struct feature_config_ctrl **ctrlp_array, *ctrl;
> +
> +	if (!id_reg || !id_reg->trap_features)
> +		/* No information to trap a feature */
> +		return;
> +
> +	val = __read_id_reg(vcpu, id_reg->sys_reg);
> +	if (val == id_reg->sys_val)
> +		/* No feature needs to be trapped (no feature is disabled). */
> +		return;
> +
> +	ctrlp_array = *id_reg->trap_features;
> +	while ((ctrl = ctrlp_array[i++]) != NULL) {
> +		if (ctrl->ftr_need_trap && ctrl->ftr_need_trap(vcpu)) {
> +			feature_trap_activate(vcpu, ctrl);
> +			continue;
> +		}
> +
> +		if (!feature_avail(ctrl, id_reg->sys_val))
> +			/* The feature is not supported on the host. */
> +			continue;
> +
> +		if (feature_avail(ctrl, val))
> +			/* The feature is enabled for the guest. */
> +			continue;
> +
> +		/* The feature is supported but disabled. */
> +		feature_trap_activate(vcpu, ctrl);
> +	}
> +}
> +
>  /* Visibility overrides for SVE-specific control registers */
>  static unsigned int sve_visibility(const struct kvm_vcpu *vcpu,
>  				   const struct sys_reg_desc *rd)
> @@ -3457,6 +3568,14 @@ int kvm_arm_copy_sys_reg_indices(struct kvm_vcpu *vcpu, u64 __user *uindices)
>  	return write_demux_regids(uindices);
>  }
>  
> +void kvm_vcpu_init_traps(struct kvm_vcpu *vcpu)

Who is going to call this? At which point? Please document the use
constraints on this.

> +{
> +	int i;
> +
> +	for (i = 0; i < ARRAY_SIZE(id_reg_info_table); i++)
> +		id_reg_features_trap_activate(vcpu, id_reg_info_table[i]);
> +}
> +
>  /* ID register's fractional field information with its feature field. */
>  struct feature_frac {
>  	u32	id;
> -- 
> 2.34.0.rc1.387.gb447b232ab-goog
> 
> 

Thanks,

	M.

-- 
Without deviation from the norm, progress is not possible.

Re: [RFC PATCH v3 29/29] KVM: arm64: selftests: Introduce id_reg_test

[Eric Auger]

Hi Reiji,
On 11/20/21 7:39 AM, Reiji Watanabe wrote:
>  Hi Eric,
> 
> On Thu, Nov 18, 2021 at 12:34 PM Eric Auger <eauger@redhat.com> wrote:
>>
>> Hi Reiji,
>>
>> On 11/17/21 7:43 AM, Reiji Watanabe wrote:
>>> Introduce a test for aarch64 to validate basic behavior of
>>> KVM_GET_ONE_REG and KVM_SET_ONE_REG for ID registers.
>>>
>>> This test runs only when KVM_CAP_ARM_ID_REG_CONFIGURABLE is supported.
>>
>> That's great to get those tests along with the series.
>>
>> There are several tests actually. I would encourage you to drop a short
>> comment along with the each main test to summarize what it does.
> 
> Thank you for the review !
> Yes, I will add short comments for the main test to summarize what it does.
> 
> 
>>> +struct id_reg_test_info {
>>> +     char            *name;
>>> +     uint32_t        id;
>>> +     bool            can_clear;
>>> +     uint64_t        fixed_mask;
>>> +     uint64_t        org_val;
>> nit: original_val? or default_val?
> 
> That is an original (or initial) value that is set by KVM.
> I will change it to original_val.
> 
> 
>>> +void test_pmu_init(struct kvm_vm *vm, uint32_t vcpu)
>> I would remove the test_ prefix as it does not test anything but
>> enhances the initialization instead
> 
> Yes, I agree.
> I will remove test_ prefix from those names.
> 
>>> +{
>>> +     struct kvm_device_attr attr = {
>>> +             .group = KVM_ARM_VCPU_PMU_V3_CTRL,
>>> +             .attr = KVM_ARM_VCPU_PMU_V3_INIT,
>>> +     };
>>> +     vcpu_ioctl(vm, vcpu, KVM_SET_DEVICE_ATTR, &attr);
>>> +}
>>> +
>>> +void test_sve_init(struct kvm_vm *vm, uint32_t vcpu)
>>> +{
>>> +     int feature = KVM_ARM_VCPU_SVE;
>>> +
>>> +     vcpu_ioctl(vm, vcpu, KVM_ARM_VCPU_FINALIZE, &feature);
>>> +}
>>> +
>>> +#define GICD_BASE_GPA                        0x8000000ULL
>>> +#define GICR_BASE_GPA                        0x80A0000ULL
>>> +
>>> +void test_vgic_init(struct kvm_vm *vm, uint32_t vcpu)
>>> +{
>>> +     /* We jsut need to configure gic v3 (we don't use it though) */
>>> +     vgic_v3_setup(vm, 1, GICD_BASE_GPA, GICR_BASE_GPA);
>>> +}
>>> +
>>> +#define      MAX_CAPS        2
>>> +struct feature_test_info {
>>> +     char    *name;  /* Feature Name (Debug information) */
>>> +     struct id_reg_test_info *sreg;  /* ID register for the feature */
>> ID register comprising the feature?
>>> +     int     shift;  /* Field of the ID register for the feature */
>> I guess you mean feature field bit shift
>>> +     int     min;    /* Min value to indicate the feature */
>> Min value that can be assigned to the feature field?
>>> +     bool    is_sign;        /* Is the field signed or unsigned ? */
>>> +     int     ncaps;          /* Number of valid Capabilities in caps[] */
>>> +     long    caps[MAX_CAPS]; /* Capabilities to indicate this feature */
>> I suggest: KVM_CAP_* capabilities requested to test this feature
>>> +     /* struct kvm_enable_cap to use the capability if needed */
>>> +     struct kvm_enable_cap   *opt_in_cap;
>>> +     bool    run_test;       /* Does guest run test for this feature ? */
>> s/run_test/guest_run?
>>> +     /* Initialization function for the feature as needed */
>> extra init sequence needed besides KVM CAP setting?
>>> +     void    (*init_feature)(struct kvm_vm *vm, uint32_t vcpuid);
>>> +     /* struct kvm_vcpu_init to opt-in the feature if needed */
>>> +     struct kvm_vcpu_init    *vcpu_init;
>>> +};
> 
> I am going to fix the comments as follows.
> (Or are any of them still unclear ?)
> 
>     /* ID register that identifies the presence of the feature */
>     struct id_reg_test_info *sreg;
> 
>     /* Bit shift for the field that identifies the presence of the feature */
>     int     shift;
> 
>     /* Min value of the field that indicates the presence of the feature */
>     int     min;    /* Min value to indicate the feature */
> 
>     /* KVM_CAP_* Capabilities to indicates that KVM supports this feature */
>     long    caps[MAX_CAPS]; /* Capabilities to indicate this feature */
> 
>     /* Should the guest check the ID register for this feature ? */
>     bool    run_test;
> 
>     /*
>      * Any extra function to configure the feature if needed.
>      * (e.g. KVM_ARM_VCPU_FINALIZE for SVE)
>      */
>     void    (*init_feature)(struct kvm_vm *vm, uint32_t vcpuid);
> 
>>> +
>>> +/* Test for optin CPU features */
>> opt-in?
> 
> I will fix it.
> 
>>> +static struct feature_test_info feature_test_info_table[] = {
>>> +     {
>>> +             .name = "SVE",
>>> +             .sreg = ID_REG_INFO(ID_AA64PFR0),
>>> +             .shift = ID_AA64PFR0_SVE_SHIFT,
>>> +             .min = 1,
>>> +             .caps = {KVM_CAP_ARM_SVE},
>>> +             .ncaps = 1,
>>> +             .init_feature = test_sve_init,
>>> +             .vcpu_init = &(struct kvm_vcpu_init) {
>>> +                     .features = {1ULL << KVM_ARM_VCPU_SVE},
>>> +             },
>>> +     },
>>> +     {
>>> +             .name = "GIC",
>>> +             .sreg = ID_REG_INFO(ID_AA64PFR0),
>>> +             .shift = ID_AA64PFR0_GIC_SHIFT,
>>> +             .min = 1,
>>> +             .caps = {KVM_CAP_IRQCHIP},
>>> +             .ncaps = 1,
>>> +             .init_feature = test_vgic_init,
>>> +     },
>>> +     {
>>> +             .name = "MTE",
>>> +             .sreg = ID_REG_INFO(ID_AA64PFR1),
>>> +             .shift = ID_AA64PFR1_MTE_SHIFT,
>>> +             .min = 2,
>>> +             .caps = {KVM_CAP_ARM_MTE},
>>> +             .ncaps = 1,
>>> +             .opt_in_cap = &(struct kvm_enable_cap) {
>>> +                             .cap = KVM_CAP_ARM_MTE,
>>> +             },
>>> +     },
>>> +     {
>>> +             .name = "PMUV3",
>>> +             .sreg = ID_REG_INFO(ID_AA64DFR0),
>>> +             .shift = ID_AA64DFR0_PMUVER_SHIFT,
>>> +             .min = 1,
>>> +             .init_feature = test_pmu_init,
>>> +             .caps = {KVM_CAP_ARM_PMU_V3},
>>> +             .ncaps = 1,
>>> +             .vcpu_init = &(struct kvm_vcpu_init) {
>>> +                     .features = {1ULL << KVM_ARM_VCPU_PMU_V3},
>>> +             },
>>> +     },
>>> +     {
>>> +             .name = "PERFMON",
>>> +             .sreg = ID_REG_INFO(ID_DFR0),
>>> +             .shift = ID_DFR0_PERFMON_SHIFT,
>>> +             .min = 3,
>>> +             .init_feature = test_pmu_init,
>>> +             .caps = {KVM_CAP_ARM_PMU_V3},
>>> +             .ncaps = 1,
>>> +             .vcpu_init = &(struct kvm_vcpu_init) {
>>> +                     .features = {1ULL << KVM_ARM_VCPU_PMU_V3},
>>> +             },
>>> +     },
>>> +};
>>> +
>>> +static int walk_id_reg_list(int (*fn)(struct id_reg_test_info *sreg, void *arg),
>>> +                         void *arg)
>>> +{
>>> +     int ret = 0, i;
>>> +
>>> +     for (i = 0; i < ARRAY_SIZE(id_reg_list); i++) {
>>> +             ret = fn(&id_reg_list[i], arg);
>>> +             if (ret)
>>> +                     break;
>> none of your fn() function does return something != 0
> 
> I will change the callback function to return void instead of int.
> 
>>> +     }
>>> +     return ret;
>>> +}
>>> +
>>> +static int guest_code_id_reg_check_one(struct id_reg_test_info *sreg, void *arg)
>>> +{
>>> +     uint64_t val = sreg->read_reg();
>>> +
>>> +     GUEST_ASSERT_2(val == sreg->user_val, sreg->name, sreg->user_val);
>>> +     return 0;
>>> +}
>>> +
>>> +static void guest_code_id_reg_check_all(uint32_t cpu)
>>> +{
>>> +     walk_id_reg_list(guest_code_id_reg_check_one, NULL);
>>> +     GUEST_DONE();
>>> +}
>>> +
>>> +static void guest_code_do_nothing(uint32_t cpu)
>>> +{
>>> +     GUEST_DONE();
>>> +}
>>> +
>>> +static void guest_code_feature_check(uint32_t cpu)
>>> +{
>>> +     int i;
>>> +     struct feature_test_info *finfo;
>>> +
>>> +     for (i = 0; i < ARRAY_SIZE(feature_test_info_table); i++) {
>>> +             finfo = &feature_test_info_table[i];
>>> +             if (finfo->run_test)
>>> +                     guest_code_id_reg_check_one(finfo->sreg, NULL);
>>> +     }
>>> +
>>> +     GUEST_DONE();
>>> +}
>>> +
>>> +static void guest_code_ptrauth_check(uint32_t cpuid)
>>> +{
>>> +     struct id_reg_test_info *sreg = ID_REG_INFO(ID_AA64ISAR1);
>>> +     uint64_t val = sreg->read_reg();
>>> +
>>> +     GUEST_ASSERT_2(val == sreg->user_val, "PTRAUTH", val);
>>> +     GUEST_DONE();
>>> +}
>>> +
>>> +static int reset_id_reg_info_one(struct id_reg_test_info *sreg, void *arg)
>> reset_id_reg_user_val_one()?
> 
> Thanks for the suggestion. I will fix that.
> 
>>> +{
>>> +     sreg->user_val = sreg->org_val;
>>> +     return 0;
>>> +}
>>> +
>>> +static void reset_id_reg_info(void)
>> reset_id_reg_user_val()?
> 
>  I will fix that.
> 
>>> +{
>>> +     walk_id_reg_list(reset_id_reg_info_one, NULL);
>>> +}
>>> +
>>> +static struct kvm_vm *test_vm_create_cap(uint32_t nvcpus,
>>> +             void (*guest_code)(uint32_t), struct kvm_vcpu_init *init,
>>> +             struct kvm_enable_cap *cap)
>>> +{
>>> +     struct kvm_vm *vm;
>>> +     uint32_t cpuid;
>>> +     uint64_t mem_pages;
>>> +
>>> +     mem_pages = DEFAULT_GUEST_PHY_PAGES + DEFAULT_STACK_PGS * nvcpus;
>>> +     mem_pages += mem_pages / (PTES_PER_MIN_PAGE * 2);
>>> +     mem_pages = vm_adjust_num_guest_pages(VM_MODE_DEFAULT, mem_pages);
>>
>>
>>> +
>>> +     vm = vm_create(VM_MODE_DEFAULT,
>>> +             DEFAULT_GUEST_PHY_PAGES + (DEFAULT_STACK_PGS * nvcpus),
>>> +             O_RDWR);
>> mem_pages must be used instead
>>
>> augere@virtlab-arm04:~/UPSTREAM/ML/tools/testing/selftests/kvm#
>> ./aarch64/id_reg_test
>> ==== Test Assertion Failure ====
>>   lib/kvm_util.c:825: vm_adjust_num_guest_pages(vm->mode, npages) == npages
>>   pid=11439 tid=11439 errno=0 - Success
>>      1  0x00000000004068cb: vm_userspace_mem_region_add at kvm_util.c:823
>>      2  0x00000000004071af: vm_create at kvm_util.c:319
>>      3  0x0000000000401afb: test_vm_create_cap at id_reg_test.c:508
>>      4  0x00000000004014a3: test_vm_create at id_reg_test.c:541
>>      5   (inlined by) init_id_reg_info at id_reg_test.c:1110
>>      6   (inlined by) main at id_reg_test.c:1125
>>      7  0x000003ffa7220de3: ?? ??:0
>>      8  0x00000000004015eb: _start at :?
>>   Number of guest pages is not compatible with the host. Try npages=528
> 
> Thank you for catching this (It didn't happen in my usual test environment).
> I will fix this.
> 
>>
>>
>> Don't you want to check the cap in a first place using kvm_check_cap and
>> cap->cap
> 
> It is done by the caller before trying to create the vm.
> 
> 
>>> +     if (cap)
>>> +             vm_enable_cap(vm, cap);
>>> +
>>> +     kvm_vm_elf_load(vm, program_invocation_name);
>>> +
>>> +     if (init && init->target == -1) {
>>> +             struct kvm_vcpu_init preferred;
>>> +
>>> +             vm_ioctl(vm, KVM_ARM_PREFERRED_TARGET, &preferred);
>>> +             init->target = preferred.target;
>>> +     }
>>> +
>>> +     vm_init_descriptor_tables(vm);
>>> +     for (cpuid = 0; cpuid < nvcpus; cpuid++) {
>>> +             if (init)
>>> +                     aarch64_vcpu_add_default(vm, cpuid, init, guest_code);
>>> +             else
>>> +                     vm_vcpu_add_default(vm, cpuid, guest_code);
>> nit: vm_vcpu_add_default calls aarch64_vcpu_add_default(vm, vcpuid,
>> NULL, guest_code) so you can unconditionnaly call
>> aarch64_vcpu_add_default(vm, cpuid, init, guest_code)
> 
> Oh, thank you ! I will fix that (somehow I overlooked that...).
> 
>>> +
>>> +             vcpu_init_descriptor_tables(vm, cpuid);
>>> +     }
>>> +
>>> +     ucall_init(vm, NULL);
>>> +     return vm;
>>> +}
>>> +
>>> +static struct kvm_vm *test_vm_create(uint32_t nvcpus,
>>> +                                  void (*guest_code)(uint32_t),
>>> +                                  struct kvm_vcpu_init *init)
>>> +{
>>> +     return test_vm_create_cap(nvcpus, guest_code, init, NULL);
>>> +}
>> nit: not sure test_vm_create is needed. By the way it is already called
>> with init = NULL so we can call test_vm_create_cap with 2 NULL args
> 
> I will remove test_vm_create.
> 
> 
>>> +
>>> +static void test_vm_free(struct kvm_vm *vm)
>>> +{
>>> +     ucall_uninit(vm);
>>> +     kvm_vm_free(vm);
>>> +}
>>> +
>>> +#define      TEST_RUN(vm, cpu)       \
>>> +     (test_vcpu_run(__func__, __LINE__, vm, cpu, true))
>>> +
>>> +#define      TEST_RUN_NO_SYNC_DATA(vm, cpu)  \
>>> +     (test_vcpu_run(__func__, __LINE__, vm, cpu, false))
>>> +
>>> +static int test_vcpu_run(const char *test_name, int line,
>>> +                      struct kvm_vm *vm, uint32_t vcpuid, bool sync_data)
>>> +{
>>> +     struct ucall uc;
>>> +     int ret;
>>> +
>>> +     if (sync_data) {
>>> +             sync_global_to_guest(vm, id_reg_list);
>>> +             sync_global_to_guest(vm, feature_test_info_table);
>>> +     }
>>> +
>>> +     vcpu_args_set(vm, vcpuid, 1, vcpuid);
>>> +
>>> +     ret = _vcpu_run(vm, vcpuid);
>>> +     if (ret) {
>>> +             ret = errno;
>>> +             goto sync_exit;
>>> +     }
>>> +
>>> +     switch (get_ucall(vm, vcpuid, &uc)) {
>>> +     case UCALL_SYNC:
>>> +     case UCALL_DONE:
>>> +             ret = 0;
>>> +             break;
>>> +     case UCALL_ABORT:
>>> +             TEST_FAIL(
>>> +                 "%s (%s) at line %d (user %s at line %d), args[3]=0x%lx",
>>> +                 (char *)uc.args[0], (char *)uc.args[2], (int)uc.args[1],
>>> +                 test_name, line, uc.args[3]);
>>> +             break;
>>> +     default:
>>> +             TEST_FAIL("Unexpected guest exit\n");
>>> +     }
>>> +
>>> +sync_exit:
>>> +     if (sync_data) {
>>> +             sync_global_from_guest(vm, id_reg_list);
>>> +             sync_global_from_guest(vm, feature_test_info_table);
>>> +     }
>>> +     return ret;
>>> +}
>>> +
>>> +static int set_id_regs_after_run_test_one(struct id_reg_test_info *sreg,
>>> +                                       void *arg)
>>> +{
>>> +     struct kvm_vm *vm = arg;
>>> +     struct kvm_one_reg one_reg;
>>> +     uint32_t vcpuid = 0;
>>> +     uint64_t reg_val;
>>> +     int ret;
>>> +
>>> +     one_reg.addr = (uint64_t)&reg_val;
>>> +     one_reg.id = KVM_ARM64_SYS_REG(sreg->id);
>>> +
>>> +     vcpu_ioctl(vm, vcpuid, KVM_GET_ONE_REG, &one_reg);
>>> +     if ((reg_val != 0) && (sreg->can_clear)) {
>>> +             reg_val = 0;
>>> +             ret = _vcpu_ioctl(vm, vcpuid, KVM_SET_ONE_REG, &one_reg);
>>> +             TEST_ASSERT(ret && errno == EINVAL,
>>> +                         "Changing ID reg value should fail\n");
>>> +     }
>>> +
>>> +     vcpu_ioctl(vm, vcpuid, KVM_GET_ONE_REG, &one_reg);> +   ret = _vcpu_ioctl(vm, vcpuid, KVM_SET_ONE_REG, &one_reg);
>>> +     TEST_ASSERT(ret == 0, "Setting the same ID reg value should work\n");
>>> +
>>> +     return 0;
>>> +}
>>> +
>>> +static int set_id_regs_test_one(struct id_reg_test_info *sreg, void *arg)
>> if it is a test use test_ prefix
> 
> I will fix this.
> 
> 
>>> +{
>>> +     struct kvm_vm *vm = arg;
>>> +     struct kvm_one_reg one_reg;
>>> +     uint32_t vcpuid = 0;
>>> +     uint64_t reg_val;
>>> +
>>> +     one_reg.addr = (uint64_t)&reg_val;
>>> +     reset_id_reg_info();
>>> +
>>> +     one_reg.id = KVM_ARM64_SYS_REG(sreg->id);
>>> +     if (sreg->can_clear) {
>>> +             /* Change the register to 0 when possible */
>>> +             reg_val = 0;
>>> +             vcpu_ioctl(vm, vcpuid, KVM_SET_ONE_REG, &one_reg);
>>> +             vcpu_ioctl(vm, vcpuid, KVM_GET_ONE_REG, &one_reg);
>>> +             TEST_ASSERT(reg_val == 0,
>>> +                 "GET(%s) didn't return 0 but 0x%lx", sreg->name, reg_val);
>>> +     }
>>> +
>>> +     /* Check if we can restore the initial value */
>>> +     reg_val = sreg->org_val;
>>> +     vcpu_ioctl(vm, vcpuid, KVM_SET_ONE_REG, &one_reg);
>>> +     vcpu_ioctl(vm, vcpuid, KVM_GET_ONE_REG, &one_reg);
>>> +     TEST_ASSERT(reg_val == sreg->org_val,
>>> +                 "GET(%s) didn't return 0x%lx but 0x%lx",
>>> +                 sreg->name, sreg->org_val, reg_val);
>>> +     sreg->user_val = sreg->org_val;
>>> +     return 0;
>>> +}
>>> +
>>> +static void set_id_regs_test(void)
>> if it is a test use test_ prefix
> 
> I will fix this.
> 
>>> +{
>>> +     struct kvm_vm *vm;
>>> +     int ret;
>>> +
>>> +     reset_id_reg_info();
>>> +     vm = test_vm_create(1, guest_code_id_reg_check_all, NULL);
>> add test_vm_free()
> 
> I will fix this.
> 
>>> +
>>> +     ret = walk_id_reg_list(set_id_regs_test_one, vm);
>>> +     assert(!ret);
>>> +
>>> +     ret = TEST_RUN(vm, 0);
>>> +     TEST_ASSERT(!ret, "%s TEST_RUN failed, ret=0x%x", __func__, ret);
>>> +
>>> +     ret = walk_id_reg_list(set_id_regs_after_run_test_one, vm);
>>> +     assert(!ret);
>>> +}
>>> +
>>> +static bool caps_are_supported(long *caps, int ncaps)
>>> +{
>>> +     int i;
>>> +
>>> +     for (i = 0; i < ncaps; i++) {
>>> +             if (kvm_check_cap(caps[i]) <= 0)
>>> +                     return false;
>>> +     }
>>> +     return true;
>>> +}
>>> +
>>> +static void test_feature_ptrauth(void)
>>> +{
>>> +     struct kvm_one_reg one_reg;
>>> +     struct kvm_vcpu_init init;
>>> +     struct kvm_vm *vm = NULL;
>>> +     struct id_reg_test_info *sreg = ID_REG_INFO(ID_AA64ISAR1);
>>> +     uint32_t vcpu = 0;
>>> +     int64_t rval;
>>> +     int ret;
>>> +     int apa, api, gpa, gpi;
>>> +     char *name = "PTRAUTH";
>>> +     long caps[2] = {KVM_CAP_ARM_PTRAUTH_ADDRESS,
>>> +                     KVM_CAP_ARM_PTRAUTH_GENERIC};
>>> +
>>> +     reset_id_reg_info();
>>> +     one_reg.addr = (uint64_t)&rval;
>>> +     one_reg.id = KVM_ARM64_SYS_REG(sreg->id);
>>> +
>>> +     if (caps_are_supported(caps, 2)) {
>>> +
>>> +             /* Test with feature enabled */
>>> +             memset(&init, 0, sizeof(init));
>>> +             init.target = -1;
>>> +             init.features[0] = (1ULL << KVM_ARM_VCPU_PTRAUTH_ADDRESS |
>>> +                                 1ULL << KVM_ARM_VCPU_PTRAUTH_GENERIC);
>>> +             vm = test_vm_create_cap(1, guest_code_ptrauth_check, &init,
>>> +                                     NULL);
>>> +             vcpu_ioctl(vm, vcpu, KVM_GET_ONE_REG, &one_reg);
>>> +             apa = GET_ID_UFIELD(rval, ID_AA64ISAR1_APA_SHIFT);
>>> +             api = GET_ID_UFIELD(rval, ID_AA64ISAR1_API_SHIFT);
>>> +             gpa = GET_ID_UFIELD(rval, ID_AA64ISAR1_GPA_SHIFT);
>>> +             gpi = GET_ID_UFIELD(rval, ID_AA64ISAR1_GPI_SHIFT);
>>> +
>>> +             TEST_ASSERT((apa > 0) || (api > 0),
>>> +                         "Either apa(0x%x) or api(0x%x) must be available",
>>> +                         apa, gpa);
>>> +             TEST_ASSERT((gpa > 0) || (gpi > 0),
>>> +                         "Either gpa(0x%x) or gpi(0x%x) must be available",
>>> +                         gpa, gpi);
>>> +
>>> +             TEST_ASSERT((apa > 0) ^ (api > 0),
>>> +                         "Both apa(0x%x) and api(0x%x) must not be available",
>>> +                         apa, api);
>>> +             TEST_ASSERT((gpa > 0) ^ (gpi > 0),
>>> +                         "Both gpa(0x%x) and gpi(0x%x) must not be available",
>>> +                         gpa, gpi);
>>> +
>>> +             sreg->user_val = rval;
>>> +
>>> +             pr_debug("%s: Test with %s enabled (%s: 0x%lx)\n",
>>> +                      __func__, name, sreg->name, sreg->user_val);
>>> +             ret = TEST_RUN(vm, vcpu);
>>> +             TEST_ASSERT(!ret, "%s:KVM_RUN failed with %s enabled",
>>> +                         __func__, name);
>>> +             test_vm_free(vm);
>>> +     }
>>> +
>>> +     /* Test with feature disabled */
>>> +     reset_id_reg_info();
>>> +
>>> +     vm = test_vm_create(1, guest_code_feature_check, NULL);
>>> +     vcpu_ioctl(vm, vcpu, KVM_GET_ONE_REG, &one_reg);
>>> +
>>> +     apa = GET_ID_UFIELD(rval, ID_AA64ISAR1_APA_SHIFT);
>>> +     api = GET_ID_UFIELD(rval, ID_AA64ISAR1_API_SHIFT);
>>> +     gpa = GET_ID_UFIELD(rval, ID_AA64ISAR1_GPA_SHIFT);
>>> +     gpi = GET_ID_UFIELD(rval, ID_AA64ISAR1_GPI_SHIFT);
>>> +     TEST_ASSERT(!apa && !api && !gpa && !gpi,
>>> +         "apa(0x%x), api(0x%x), gpa(0x%x), gpi(0x%x) must be zero",
>>> +         apa, api, gpa, gpi);
>>> +
>>> +     pr_debug("%s: Test with %s disabled (%s: 0x%lx)\n",
>>> +              __func__, name, sreg->name, sreg->user_val);
>>> +
>>> +     ret = TEST_RUN(vm, vcpu);
>>> +     TEST_ASSERT(!ret, "%s TEST_RUN failed with %s enabled, ret=0x%x",
>>> +                 __func__, name, ret);
>>> +
>>> +     test_vm_free(vm);
>>> +}
>>> +
>>> +static bool feature_caps_are_available(struct feature_test_info *finfo)
>>> +{
>>> +     return ((finfo->ncaps > 0) &&
>>> +             caps_are_supported(finfo->caps, finfo->ncaps));
>>> +}
>>> +
>> comment with short explanation of what the test does
> 
> Yes, I will add a short explanation for each test.
> 
>>> +static void test_feature(struct feature_test_info *finfo)
>>> +{
>>> +     struct id_reg_test_info *sreg = finfo->sreg;
>>> +     struct kvm_one_reg one_reg;
>>> +     struct kvm_vcpu_init init, *initp = NULL;
>>> +     struct kvm_vm *vm = NULL;
>>> +     int64_t fval, reg_val;
>>> +     uint32_t vcpu = 0;
>>> +     bool is_sign = finfo->is_sign;
>>> +     int min = finfo->min;
>>> +     int shift = finfo->shift;
>>> +     int ret;
>>> +
>>> +     pr_debug("%s: %s (reg %s)\n", __func__, finfo->name, sreg->name);
>>> +
>>> +     reset_id_reg_info();
>>> +     finfo->run_test = 1;    /* Indicate that guest runs the test on it */
>>> +     one_reg.addr = (uint64_t)&reg_val;
>>> +     one_reg.id = KVM_ARM64_SYS_REG(sreg->id);
>>> +
>>> +     /* Test with feature enabled if the feature is available */
>> s/if the feature is available/if the feature is exposed in the default
>> ID_REG value and if the capabilities are supported at KVM level
> 
> Thank you for the suggestion. I will fix that.
> 
> 
>>> +void run_test(void)
>>> +{
>>> +     set_id_regs_test();
>>> +     test_feature_all();
>>> +     test_feature_ptrauth();
>>> +     test_feature_frac_all();
>>> +}
>>> +
>> basic comment would be helpful: attempts to clear a given id_reg and
>> populate the id_reg with the original value, and can_clear flag?
> 
> I will add some comments.
> 
>>> +static int init_id_reg_info_one(struct id_reg_test_info *sreg, void *arg)
>>> +{
>>> +     uint64_t reg_val;
>>> +     uint32_t vcpuid = 0;
>>> +     int ret;
>>> +     struct kvm_one_reg one_reg;
>>> +     struct kvm_vm *vm = arg;
>>> +
>>> +     one_reg.addr = (uint64_t)&reg_val;
>>> +     one_reg.id = KVM_ARM64_SYS_REG(sreg->id);
>>> +     vcpu_ioctl(vm, vcpuid, KVM_GET_ONE_REG, &one_reg);
>>> +     sreg->org_val = reg_val;
>>> +     sreg->user_val = reg_val;
>> nit: add a comment for user_val because it is not obvious why you set it
>> to reg_val.
> 
> I will add a comment for it.
> 
>>> +     if (sreg->org_val) {
>>> +             reg_val = 0;
>>> +             ret = _vcpu_ioctl(vm, vcpuid, KVM_SET_ONE_REG, &one_reg);
>>> +             if (!ret)
>>> +                     sreg->can_clear = true;
>>> +     }
>>> +
>>> +     pr_debug("%s (0x%x): 0x%lx%s\n", sreg->name, sreg->id,
>>> +              sreg->org_val, sreg->can_clear ? ", can clear" : "");
>>> +
>>> +     return 0;
>>> +}
>>> +
>> add a comment? loop over the idreg list and populates each regid info
>> with the default, user and can_clear value
> 
> I will add a comment for the function.
> 
>>> +static void init_id_reg_info(void)
>>> +{
>>> +     struct kvm_vm *vm;
>>> +
>>> +     vm = test_vm_create(1, guest_code_do_nothing, NULL);
>>> +     walk_id_reg_list(init_id_reg_info_one, vm);
>>> +     test_vm_free(vm);
>>> +}
>>> +
>>> +int main(void)
>>> +{
>>> +
>>> +     setbuf(stdout, NULL);
>>> +
>>> +     if (kvm_check_cap(KVM_CAP_ARM_ID_REG_CONFIGURABLE) <= 0) {
>>> +             print_skip("KVM_CAP_ARM_ID_REG_CONFIGURABLE is not supported\n");
>>> +             exit(KSFT_SKIP);
>>> +     }
>>> +
>>> +     init_id_reg_info();
>>> +     run_test();
>>> +     return 0;
>>> +}
>>>
>>
>> After fixing the mem_pages stuff I get the following error on a cavium
>> machine.
>>
>> augere@virtlab-arm04:~/UPSTREAM/ML/tools/testing/selftests/kvm#
>> ./aarch64/id_reg_test
>> ==== Test Assertion Failure ====
>>   aarch64/id_reg_test.c:814: fval >= min
>>   pid=11692 tid=11692 errno=4 - Interrupted system call
>>      1  0x00000000004028d3: test_feature at id_reg_test.c:813
>>      2   (inlined by) test_feature_all at id_reg_test.c:863
>>      3   (inlined by) run_test at id_reg_test.c:1073
>>      4  0x000000000040156f: main at id_reg_test.c:1124
>>      5  0x000003ffa9420de3: ?? ??:0
>>      6  0x00000000004015eb: _start at :?
>>   PERFMON field of ID_DFR0 is too small (0)
>>
>> Fails on
>> test_feature: PERFMON (reg ID_DFR0)
>>
>> I will do my utmost to further debug
> 
> Thank you for running it in your environment and reporting this !
> This is very interesting...
> 
> It implies that the host's ID_DFR0_EL1.PerfMon is zero or 0xf
> (meaning FEAT_PMUv3 is not implemented) even though the host's
> ID_AA64DFR0_EL1.PMUVer indicates that FEAT_PMUv3 is implemented.
> 
> Would it be possible for you to check values of those two
> registers on the host (not on the guest) to see if both of them
> indicate the presence of FEAT_PMUv3 consistently ?

Here are both values printed in armpmu_register()
[   33.320901] armpmu_register perfmon=0x0 pmuver=0x4

        perfmon =
cpuid_feature_extract_unsigned_field(read_cpuid(ID_DFR0_EL1),
                        ID_DFR0_PERFMON_SHIFT);
        pmuver =
cpuid_feature_extract_unsigned_field(read_cpuid(ID_AA64DFR0_EL1),
                        ID_AA64DFR0_PMUVER_SHIFT);
        printk("%s perfmon=0x%x pmuver=0x%x\n", __func__, perfmon, pmuver);

My machine is a Gigabyte R181-T90 (ThunderX2)

Eric


Eric

> 
> Thanks,
> Reiji
> 

Re: [RFC PATCH v3 01/29] KVM: arm64: Add has_reset_once flag for vcpu

[Reiji Watanabe]

On Sun, Nov 21, 2021 at 4:36 AM Marc Zyngier <maz@kernel.org> wrote:
>
> On Wed, 17 Nov 2021 06:43:31 +0000,
> Reiji Watanabe <reijiw@google.com> wrote:
> >
> > Introduce 'has_reset_once' flag in kvm_vcpu_arch, which indicates
> > if the vCPU reset has been done once, for later use.
>
> It would be nice if you could at least hint at what this flag is going
> to be used for, as being able to reset a vcpu as often as userspace
> wants it is part of the KVM ABI.

I will update the description.


> > Signed-off-by: Reiji Watanabe <reijiw@google.com>
> > Reviewed-by: Oliver Upton <oupton@google.com>
> > ---
> >  arch/arm64/include/asm/kvm_host.h | 2 ++
> >  arch/arm64/kvm/reset.c            | 4 ++++
> >  2 files changed, 6 insertions(+)
> >
> > diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
> > index 2a5f7f38006f..edbe2cb21947 100644
> > --- a/arch/arm64/include/asm/kvm_host.h
> > +++ b/arch/arm64/include/asm/kvm_host.h
> > @@ -385,6 +385,7 @@ struct kvm_vcpu_arch {
> >               u64 last_steal;
> >               gpa_t base;
> >       } steal;
> > +     bool has_reset_once;
>
> Why can't this be a new flag (part of the vcpu->arch.flags set) rather
> than a discrete bool?

Thank you for the suggestion ! I will fix it to use vcpu->arch.flags.

Regards,
Reiji

Re: [RFC PATCH v3 03/29] KVM: arm64: Introduce struct id_reg_info

[Reiji Watanabe]

On Sun, Nov 21, 2021 at 4:37 AM Marc Zyngier <maz@kernel.org> wrote:
>
> On Fri, 19 Nov 2021 04:47:53 +0000,
> Reiji Watanabe <reijiw@google.com> wrote:
> >
> > I am going to add the following comment. Does it look clear enough for you ?
> >
> >         /*
> >          * This is an optional ID register specific validation function.
> >          * When userspace tries to set the ID register, arm64_check_features()
> >          * will check if the requested value indicates any features that cannot
> >          * be supported by KVM on the host.  But, some ID register fields need
> >          * a special checking and this function can be used for such fields.
> >          * e.g. KVM_CREATE_DEVICE must be used to configure GICv3 for a guest.
> >          * ID_AA64PFR0_EL1.GIC shouldn't be set to 1 unless GICv3 is configured.
>
> There is no such requirement. GICv3 has a compatibility interface, and
> although KVM doesn't make use of it, there is no reason not to expose
> that GICv3 exists on the CPU even if not using it. Even more, this has
> been the case forever, and making this change now would probably break
> migration.

Shockingly, I somehow misunderstood what read_id_reg() did for GICv3...
I will use a different example for it.

Thanks,
Reiji

Re: [RFC PATCH v3 02/29] KVM: arm64: Save ID registers' sanitized value per vCPU

[Reiji Watanabe]

On Sun, Nov 21, 2021 at 4:37 AM Marc Zyngier <maz@kernel.org> wrote:
>
> On Wed, 17 Nov 2021 06:43:32 +0000,
> Reiji Watanabe <reijiw@google.com> wrote:
> >
> > Extend sys_regs[] of kvm_cpu_context for ID registers and save ID
> > registers' sanitized value in the array for the vCPU at the first
> > vCPU reset. Use the saved ones when ID registers are read by
> > userspace (via KVM_GET_ONE_REG) or the guest.
> >
> > Signed-off-by: Reiji Watanabe <reijiw@google.com>
> > ---
> >  arch/arm64/include/asm/kvm_host.h | 10 +++++++
> >  arch/arm64/kvm/sys_regs.c         | 43 +++++++++++++++++++------------
> >  2 files changed, 37 insertions(+), 16 deletions(-)
> >
> > diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
> > index edbe2cb21947..72db73c79403 100644
> > --- a/arch/arm64/include/asm/kvm_host.h
> > +++ b/arch/arm64/include/asm/kvm_host.h
> > @@ -146,6 +146,14 @@ struct kvm_vcpu_fault_info {
> >       u64 disr_el1;           /* Deferred [SError] Status Register */
> >  };
> >
> > +/*
> > + * (Op0, Op1, CRn, CRm, Op2) of ID registers is (3, 0, 0, crm, op2),
> > + * where 0<=crm<8, 0<=op2<8.
> > + */
> > +#define KVM_ARM_ID_REG_MAX_NUM 64
> > +#define IDREG_IDX(id)                ((sys_reg_CRm(id) << 3) | sys_reg_Op2(id))
> > +#define IDREG_SYS_IDX(id)    (ID_REG_BASE + IDREG_IDX(id))
> > +
> >  enum vcpu_sysreg {
> >       __INVALID_SYSREG__,   /* 0 is reserved as an invalid value */
> >       MPIDR_EL1,      /* MultiProcessor Affinity Register */
> > @@ -210,6 +218,8 @@ enum vcpu_sysreg {
> >       CNTP_CVAL_EL0,
> >       CNTP_CTL_EL0,
> >
> > +     ID_REG_BASE,
> > +     ID_REG_END = ID_REG_BASE + KVM_ARM_ID_REG_MAX_NUM - 1,
>
> It is rather unclear to me why we want these registers to be
> replicated on a per-CPU basis. Yes, this fits the architecture, but
> that's also a total waste of memory if you have more than a single
> CPU, because we make a point in only exposing homogeneous properties
> to the VM (I don't think anyone intends to support vcpu asymmetry in a
> VM, and 64 registers per vcpu is not an insignificant memory usage).
>
> If there are no reasons for this to be per-CPU, please move it to be
> global to the VM. This also mean that once a vcpu has reset, it
> shouldn't be possible to affect the registers. This shouldn't affect
> the userspace API though.


Currently, userspace can configure different CPU features for each vCPU
with KVM_ARM_VCPU_INIT, which indirectly affect ID registers.
I'm not sure if anyone actually does that though.

Since I personally thought having ID registers per vCPU more naturally
fits the behavior of KVM_ARM_VCPU_INIT and makes more straightforward
behavior of KVM_SET_ONE_REG, I chose that.
(That would be also better in terms of vCPUs scalability for live migration
 considering a case where userspace tries to restore ID registers for
 many vCPUs in parallel during live migration.  Userspace could avoid that,
 and there are ways for KVM to mitigate that though.)

Having ID registers per-VM is of course feasible even while maintaining
the current behavior of KVM_ARM_VCPU_INIT though.

Thanks,
Reiji

Re: [RFC PATCH v3 29/29] KVM: arm64: selftests: Introduce id_reg_test

[Reiji Watanabe]

Hi Eric,

> >> After fixing the mem_pages stuff I get the following error on a cavium
> >> machine.
> >>
> >> augere@virtlab-arm04:~/UPSTREAM/ML/tools/testing/selftests/kvm#
> >> ./aarch64/id_reg_test
> >> ==== Test Assertion Failure ====
> >>   aarch64/id_reg_test.c:814: fval >= min
> >>   pid=11692 tid=11692 errno=4 - Interrupted system call
> >>      1  0x00000000004028d3: test_feature at id_reg_test.c:813
> >>      2   (inlined by) test_feature_all at id_reg_test.c:863
> >>      3   (inlined by) run_test at id_reg_test.c:1073
> >>      4  0x000000000040156f: main at id_reg_test.c:1124
> >>      5  0x000003ffa9420de3: ?? ??:0
> >>      6  0x00000000004015eb: _start at :?
> >>   PERFMON field of ID_DFR0 is too small (0)
> >>
> >> Fails on
> >> test_feature: PERFMON (reg ID_DFR0)
> >>
> >> I will do my utmost to further debug
> >
> > Thank you for running it in your environment and reporting this !
> > This is very interesting...
> >
> > It implies that the host's ID_DFR0_EL1.PerfMon is zero or 0xf
> > (meaning FEAT_PMUv3 is not implemented) even though the host's
> > ID_AA64DFR0_EL1.PMUVer indicates that FEAT_PMUv3 is implemented.
> >
> > Would it be possible for you to check values of those two
> > registers on the host (not on the guest) to see if both of them
> > indicate the presence of FEAT_PMUv3 consistently ?
>
> Here are both values printed in armpmu_register()
> [   33.320901] armpmu_register perfmon=0x0 pmuver=0x4
>
>         perfmon =
> cpuid_feature_extract_unsigned_field(read_cpuid(ID_DFR0_EL1),
>                         ID_DFR0_PERFMON_SHIFT);
>         pmuver =
> cpuid_feature_extract_unsigned_field(read_cpuid(ID_AA64DFR0_EL1),
>                         ID_AA64DFR0_PMUVER_SHIFT);
>         printk("%s perfmon=0x%x pmuver=0x%x\n", __func__, perfmon, pmuver);
>
> My machine is a Gigabyte R181-T90 (ThunderX2)

Thank you for your providing the information !!

Since the test incorrectly expects that ID_DFR0_EL1.PerfMon indicates
PMUv3 on any CPUs that support PMUv3 even when they don't support
32bit EL0, I will fix the test.
(ThunderX2 doesn't seem to support 32bit EL0)

Thanks,
Reiji

Re: [RFC PATCH v3 21/29] KVM: arm64: Introduce framework to trap disabled features

[Reiji Watanabe]

On Sun, Nov 21, 2021 at 10:46 AM Marc Zyngier <maz@kernel.org> wrote:
>
> On Wed, 17 Nov 2021 06:43:51 +0000,
> Reiji Watanabe <reijiw@google.com> wrote:
> >
> > When a CPU feature that is supported on the host is not exposed to
> > its guest, emulating a real CPU's behavior (by trapping or disabling
> > guest's using the feature) is generally a desirable behavior (when
> > it's possible without any or little side effect).
> >
> > Introduce feature_config_ctrl structure, which manages feature
> > information to program configuration register to trap or disable
> > the feature when the feature is not exposed to the guest, and
> > functions that uses the structure to activate trapping the feature.
> >
> > At present, no feature has feature_config_ctrl yet and the following
> > patches will add the feature_config_ctrl for several features.
> >
> > Signed-off-by: Reiji Watanabe <reijiw@google.com>
> > ---
> >  arch/arm64/kvm/sys_regs.c | 121 +++++++++++++++++++++++++++++++++++++-
> >  1 file changed, 120 insertions(+), 1 deletion(-)
> >
> > diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
> > index 2f96103fc0d2..501de08dacb7 100644
> > --- a/arch/arm64/kvm/sys_regs.c
> > +++ b/arch/arm64/kvm/sys_regs.c
> > @@ -376,8 +376,38 @@ static int arm64_check_features(u64 check_types, u64 val, u64 lim)
> >       (cpuid_feature_extract_unsigned_field(val, ID_AA64ISAR1_GPI_SHIFT) >= \
> >        ID_AA64ISAR1_GPI_IMP_DEF)
> >
> > +enum vcpu_config_reg {
> > +     VCPU_HCR_EL2 = 1,
> > +     VCPU_MDCR_EL2,
> > +     VCPU_CPTR_EL2,
> > +};
> > +
> > +/*
> > + * Feature information to program configuration register to trap or disable
> > + * guest's using a feature when the feature is not exposed to the guest.
> > + */
> > +struct feature_config_ctrl {
> > +     /* ID register/field for the feature */
> > +     u32     ftr_reg;        /* ID register */
> > +     bool    ftr_signed;     /* Is the feature field signed ? */
> > +     u8      ftr_shift;      /* Field of ID register for the feature */
> > +     s8      ftr_min;        /* Min value that indicate the feature */
> > +
> > +     /*
> > +      * Function to check trapping is needed. This is used when the above
> > +      * fields are not enough to determine if trapping is needed.
> > +      */
> > +     bool    (*ftr_need_trap)(struct kvm_vcpu *vcpu);
> > +
> > +     /* Configuration register information to trap the feature. */
> > +     enum vcpu_config_reg cfg_reg;   /* Configuration register */
> > +     u64     cfg_mask;       /* Field of the configuration register */
> > +     u64     cfg_val;        /* Value that are set for the field */
>
> Although this probably works for the use cases you have in mind, some
> trap bits are actually working the other way around (clear to trap).
> So you probably want to turn this into cfg_set and add a cfg_clear for
> a good measure, dropping cfg_mask in the process.

Although I was aware of both of the cases (cfg_clear is implicitly
derived from cfg_mask ~ cfg_set), I agree that dropping cfg_mask and
adding cfg_clear would be more explicit and nicer.

> That being said, the current trend is to move to FGT, meaning that a
> single register is unlikely to cut it in the long run. I'd rather you
> simply have a configuration function here (and the helper you already
> have is probably enough).

Thank you for the nice suggestion ! That's a good point (I didn't pay
attention to FGT yet).  I will look into having a configuration function
here.

> > +};
> > +
> >  struct id_reg_info {
> >       u32     sys_reg;        /* Register ID */
> > +     u64     sys_val;        /* Sanitized system value */
> >
> >       /*
> >        * Limit value of the register for a vcpu. The value is the sanitized
> > @@ -410,11 +440,15 @@ struct id_reg_info {
> >       /* Return the reset value of the register for the vCPU */
> >       u64 (*get_reset_val)(struct kvm_vcpu *vcpu,
> >                            const struct id_reg_info *id_reg);
> > +
> > +     /* Information to trap features that are disabled for the guest */
> > +     const struct feature_config_ctrl *(*trap_features)[];
> >  };
> >
> >  static void id_reg_info_init(struct id_reg_info *id_reg)
> >  {
> > -     id_reg->vcpu_limit_val = read_sanitised_ftr_reg(id_reg->sys_reg);
> > +     id_reg->sys_val = read_sanitised_ftr_reg(id_reg->sys_reg);
> > +     id_reg->vcpu_limit_val = id_reg->sys_val;
> >       if (id_reg->init)
> >               id_reg->init(id_reg);
> >  }
> > @@ -952,6 +986,47 @@ static int validate_id_reg(struct kvm_vcpu *vcpu,
> >       return err;
> >  }
> >
> > +static void feature_trap_activate(struct kvm_vcpu *vcpu,
> > +                               const struct feature_config_ctrl *config)
> > +{
> > +     u64 *reg_ptr, reg_val;
> > +
> > +     switch (config->cfg_reg) {
> > +     case VCPU_HCR_EL2:
> > +             reg_ptr = &vcpu->arch.hcr_el2;
> > +             break;
> > +     case VCPU_MDCR_EL2:
> > +             reg_ptr = &vcpu->arch.mdcr_el2;
> > +             break;
> > +     case VCPU_CPTR_EL2:
> > +             reg_ptr = &vcpu->arch.cptr_el2;
> > +             break;
> > +     }
> > +
> > +     /* Update cfg_mask fields with cfg_val */
> > +     reg_val = (*reg_ptr & ~config->cfg_mask);
> > +     reg_val |= config->cfg_val;
> > +     *reg_ptr = reg_val;
> > +}
> > +
> > +static inline bool feature_avail(const struct feature_config_ctrl *ctrl,
> > +                              u64 id_val)
> > +{
> > +     int field_val = cpuid_feature_extract_field(id_val,
> > +                             ctrl->ftr_shift, ctrl->ftr_signed);
> > +
> > +     return (field_val >= ctrl->ftr_min);
> > +}
> > +
> > +static inline bool vcpu_feature_is_available(struct kvm_vcpu *vcpu,
> > +                                     const struct feature_config_ctrl *ctrl)
> > +{
> > +     u64 val;
> > +
> > +     val = __read_id_reg(vcpu, ctrl->ftr_reg);
> > +     return feature_avail(ctrl, val);
> > +}
> > +
> >  /*
> >   * ARMv8.1 mandates at least a trivial LORegion implementation, where all the
> >   * RW registers are RES0 (which we can implement as RAZ/WI). On an ARMv8.0
> > @@ -1831,6 +1906,42 @@ static int reg_from_user(u64 *val, const void __user *uaddr, u64 id);
> >  static int reg_to_user(void __user *uaddr, const u64 *val, u64 id);
> >  static u64 sys_reg_to_index(const struct sys_reg_desc *reg);
> >
> > +static void id_reg_features_trap_activate(struct kvm_vcpu *vcpu,
> > +                                       const struct id_reg_info *id_reg)
> > +{
> > +     u64 val;
> > +     int i = 0;
> > +     const struct feature_config_ctrl **ctrlp_array, *ctrl;
> > +
> > +     if (!id_reg || !id_reg->trap_features)
> > +             /* No information to trap a feature */
> > +             return;
> > +
> > +     val = __read_id_reg(vcpu, id_reg->sys_reg);
> > +     if (val == id_reg->sys_val)
> > +             /* No feature needs to be trapped (no feature is disabled). */
> > +             return;
> > +
> > +     ctrlp_array = *id_reg->trap_features;
> > +     while ((ctrl = ctrlp_array[i++]) != NULL) {
> > +             if (ctrl->ftr_need_trap && ctrl->ftr_need_trap(vcpu)) {
> > +                     feature_trap_activate(vcpu, ctrl);
> > +                     continue;
> > +             }
> > +
> > +             if (!feature_avail(ctrl, id_reg->sys_val))
> > +                     /* The feature is not supported on the host. */
> > +                     continue;
> > +
> > +             if (feature_avail(ctrl, val))
> > +                     /* The feature is enabled for the guest. */
> > +                     continue;
> > +
> > +             /* The feature is supported but disabled. */
> > +             feature_trap_activate(vcpu, ctrl);
> > +     }
> > +}
> > +
> >  /* Visibility overrides for SVE-specific control registers */
> >  static unsigned int sve_visibility(const struct kvm_vcpu *vcpu,
> >                                  const struct sys_reg_desc *rd)
> > @@ -3457,6 +3568,14 @@ int kvm_arm_copy_sys_reg_indices(struct kvm_vcpu *vcpu, u64 __user *uindices)
> >       return write_demux_regids(uindices);
> >  }
> >
> > +void kvm_vcpu_init_traps(struct kvm_vcpu *vcpu)
>
> Who is going to call this? At which point? Please document the use
> constraints on this.

kvm_vcpu_first_run_init() is going to call it (for non-pKVM).
I will document the use constraints on that.

Thanks,
Reiji

Re: [RFC PATCH v3 02/29] KVM: arm64: Save ID registers' sanitized value per vCPU

[Marc Zyngier]

On Tue, 23 Nov 2021 04:39:27 +0000,
Reiji Watanabe <reijiw@google.com> wrote:
> 
> On Sun, Nov 21, 2021 at 4:37 AM Marc Zyngier <maz@kernel.org> wrote:
> >
> > On Wed, 17 Nov 2021 06:43:32 +0000,
> > Reiji Watanabe <reijiw@google.com> wrote:
> > >
> > > Extend sys_regs[] of kvm_cpu_context for ID registers and save ID
> > > registers' sanitized value in the array for the vCPU at the first
> > > vCPU reset. Use the saved ones when ID registers are read by
> > > userspace (via KVM_GET_ONE_REG) or the guest.
> > >
> > > Signed-off-by: Reiji Watanabe <reijiw@google.com>
> > > ---
> > >  arch/arm64/include/asm/kvm_host.h | 10 +++++++
> > >  arch/arm64/kvm/sys_regs.c         | 43 +++++++++++++++++++------------
> > >  2 files changed, 37 insertions(+), 16 deletions(-)
> > >
> > > diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
> > > index edbe2cb21947..72db73c79403 100644
> > > --- a/arch/arm64/include/asm/kvm_host.h
> > > +++ b/arch/arm64/include/asm/kvm_host.h
> > > @@ -146,6 +146,14 @@ struct kvm_vcpu_fault_info {
> > >       u64 disr_el1;           /* Deferred [SError] Status Register */
> > >  };
> > >
> > > +/*
> > > + * (Op0, Op1, CRn, CRm, Op2) of ID registers is (3, 0, 0, crm, op2),
> > > + * where 0<=crm<8, 0<=op2<8.
> > > + */
> > > +#define KVM_ARM_ID_REG_MAX_NUM 64
> > > +#define IDREG_IDX(id)                ((sys_reg_CRm(id) << 3) | sys_reg_Op2(id))
> > > +#define IDREG_SYS_IDX(id)    (ID_REG_BASE + IDREG_IDX(id))
> > > +
> > >  enum vcpu_sysreg {
> > >       __INVALID_SYSREG__,   /* 0 is reserved as an invalid value */
> > >       MPIDR_EL1,      /* MultiProcessor Affinity Register */
> > > @@ -210,6 +218,8 @@ enum vcpu_sysreg {
> > >       CNTP_CVAL_EL0,
> > >       CNTP_CTL_EL0,
> > >
> > > +     ID_REG_BASE,
> > > +     ID_REG_END = ID_REG_BASE + KVM_ARM_ID_REG_MAX_NUM - 1,
> >
> > It is rather unclear to me why we want these registers to be
> > replicated on a per-CPU basis. Yes, this fits the architecture, but
> > that's also a total waste of memory if you have more than a single
> > CPU, because we make a point in only exposing homogeneous properties
> > to the VM (I don't think anyone intends to support vcpu asymmetry in a
> > VM, and 64 registers per vcpu is not an insignificant memory usage).
> >
> > If there are no reasons for this to be per-CPU, please move it to be
> > global to the VM. This also mean that once a vcpu has reset, it
> > shouldn't be possible to affect the registers. This shouldn't affect
> > the userspace API though.
> 
> 
> Currently, userspace can configure different CPU features for each vCPU
> with KVM_ARM_VCPU_INIT, which indirectly affect ID registers.
> I'm not sure if anyone actually does that though.

But the way the ID regs are affected is always global AFAICT. For
example, if you instantiate a PMU, you do so on all vcpus, even of the
architecture allows you to build something completely asymmetric.

> Since I personally thought having ID registers per vCPU more naturally
> fits the behavior of KVM_ARM_VCPU_INIT and makes more straightforward
> behavior of KVM_SET_ONE_REG, I chose that.

I agree that this is the logical approach from an architectural PoV.

> (That would be also better in terms of vCPUs scalability for live migration
>  considering a case where userspace tries to restore ID registers for
>  many vCPUs in parallel during live migration.  Userspace could avoid that,
>  and there are ways for KVM to mitigate that though.)

I think these are orthogonal things. We can expose a per-vcpu view,
but there is no need to have per-vcpu storage and to allow asymmetric
VMs. If I have anything to say about the future of KVM/arm64, it will
be that I don't want to support this at all.

> Having ID registers per-VM is of course feasible even while maintaining
> the current behavior of KVM_ARM_VCPU_INIT though.

Exactly. per-VM storage, and per-vcpu visibility. It will prevent all
sort of odd behaviours.

	M.

-- 
Without deviation from the norm, progress is not possible.

Re: [RFC PATCH v3 00/29] KVM: arm64: Make CPU ID registers writable by userspace

[Alexandru Elisei]

Hi Reiji,

I started reviewing the series, but I ended up being very confused, see
below.

On Tue, Nov 16, 2021 at 10:43:30PM -0800, Reiji Watanabe wrote:
> In KVM/arm64, values of ID registers for a guest are mostly same as
> its host's values except for bits for feature that KVM doesn't support
> and for opt-in features that userspace didn't configure.  Userspace
> can use KVM_SET_ONE_REG to a set ID register value, but it fails
> if userspace attempts to modify the register value.
> 
> This patch series adds support to allow userspace to modify a value of
> ID registers (as long as KVM can support features that are indicated
> in the registers) so userspace can have more control of configuring
> and unconfiguring features for guests.

What not use VCPU features? Isn't that why the field
kvm_vcpu_init->features exists in the first place? This cover letter does
nothing to explaing why any changes are needed.

Do you require finer grained control over certain feature that you cannot
get with the 32 * 7 = 224 feature flag bits from kvm_vcpu_init? Does using
the ID registers simplify certain aspects of the implementation?

Thanks,
Alex

> 
> The patch series is for both VHE and non-VHE, except for protected VMs,
> which have a different way of configuring ID registers based on its
> different requirements [1].
> There was a patch series that tried to achieve the same thing [2].
> A few snippets of codes in this series were inspired by or came from [2].
> 
> The initial value of ID registers for a vCPU will be the host's value
> with bits cleared for unsupported features and for opt-in features that
> were not configured. So, the initial value userspace can see (via
> KVM_GET_ONE_REG) is the upper limit that can be set for the register.
> Any requests to change the value that conflicts with opt-in features'
> configuration will fail.
> 
> When a guest tries to use a CPU feature that is not exposed to the guest,
> trapping it (to emulate a real CPU's behavior) would generally be a
> desirable behavior (when it is possible with no or little side effects).
> The later patches in the series add codes for this.  Only features that
> can be trapped independently will be trapped by this series though.
> 
> This series adds kunit tests for new functions in sys_regs.c (except for
> trivial ones), and these tests are enabled with a new configuration
> option 'CONFIG_KVM_KUNIT_TEST'.
> 
> The series is based on v5.16-rc1.
> 
> v3:
>   - Remove ID register consistency checking across vCPUs [Oliver]
>   - Change KVM_CAP_ARM_ID_REG_WRITABLE to
>     KVM_CAP_ARM_ID_REG_CONFIGURABLE [Oliver]
>   - Add KUnit testing for ID register validation and trap initialization.
>   - Change read_id_reg() to take care of ID_AA64PFR0_EL1.GIC
>   - Add a helper of read_id_reg() (__read_id_reg()) and use the helper
>     instead of directly using __vcpu_sys_reg()
>   - Change not to run kvm_id_regs_consistency_check() and
>     kvm_vcpu_init_traps() for protected VMs.
>   - Update selftest to remove test cases for ID register consistency
>     checking across vCPUs and to add test cases for ID_AA64PFR0_EL1.GIC.
> 
> v2: https://lore.kernel.org/all/20211103062520.1445832-1-reijiw@google.com/
>   - Remove unnecessary line breaks. [Andrew]
>   - Use @params for comments. [Andrew]
>   - Move arm64_check_features to arch/arm64/kvm/sys_regs.c and
>     change that KVM specific feature check function.  [Andrew]
>   - Remove unnecessary raz handling from __set_id_reg. [Andrew]
>   - Remove sys_val field from the initial id_reg_info and add it
>     in the later patch. [Andrew]
>   - Call id_reg->init() from id_reg_info_init(). [Andrew]
>   - Fix cpuid_feature_cap_perfmon_field() to convert 0xf to 0x0
>     (and use it in the following patches).
>   - Change kvm_vcpu_first_run_init to set has_run_once to false
>     when kvm_id_regs_consistency_check() fails.
>   - Add a patch to introduce id_reg_info for ID_AA64MMFR0_EL1,
>     which requires special validity checking for TGran*_2 fields.
>   - Add patches to introduce id_reg_info for ID_DFR1_EL1 and
>     ID_MMFR0_EL1, which are required due to arm64_check_features
>     implementation change.
>   - Add a new argument, which is a pointer to id_reg_info, for
>     id_reg_info's validate()
> 
> v1: https://lore.kernel.org/all/20211012043535.500493-1-reijiw@google.com/
> 
> [1] https://lore.kernel.org/kvmarm/20211010145636.1950948-1-tabba@google.com/
> [2] https://lore.kernel.org/kvm/20201102033422.657391-1-liangpeng10@huawei.com/
> 
> Reiji Watanabe (29):
>   KVM: arm64: Add has_reset_once flag for vcpu
>   KVM: arm64: Save ID registers' sanitized value per vCPU
>   KVM: arm64: Introduce struct id_reg_info
>   KVM: arm64: Make ID_AA64PFR0_EL1 writable
>   KVM: arm64: Make ID_AA64PFR1_EL1 writable
>   KVM: arm64: Make ID_AA64ISAR0_EL1 writable
>   KVM: arm64: Make ID_AA64ISAR1_EL1 writable
>   KVM: arm64: Make ID_AA64MMFR0_EL1 writable
>   KVM: arm64: Hide IMPLEMENTATION DEFINED PMU support for the guest
>   KVM: arm64: Make ID_AA64DFR0_EL1 writable
>   KVM: arm64: Make ID_DFR0_EL1 writable
>   KVM: arm64: Make ID_DFR1_EL1 writable
>   KVM: arm64: Make ID_MMFR0_EL1 writable
>   KVM: arm64: Make MVFR1_EL1 writable
>   KVM: arm64: Make ID registers without id_reg_info writable
>   KVM: arm64: Add consistency checking for frac fields of ID registers
>   KVM: arm64: Introduce KVM_CAP_ARM_ID_REG_CONFIGURABLE capability
>   KVM: arm64: Add kunit test for ID register validation
>   KVM: arm64: Use vcpu->arch cptr_el2 to track value of cptr_el2 for VHE
>   KVM: arm64: Use vcpu->arch.mdcr_el2 to track value of mdcr_el2
>   KVM: arm64: Introduce framework to trap disabled features
>   KVM: arm64: Trap disabled features of ID_AA64PFR0_EL1
>   KVM: arm64: Trap disabled features of ID_AA64PFR1_EL1
>   KVM: arm64: Trap disabled features of ID_AA64DFR0_EL1
>   KVM: arm64: Trap disabled features of ID_AA64MMFR1_EL1
>   KVM: arm64: Trap disabled features of ID_AA64ISAR1_EL1
>   KVM: arm64: Initialize trapping of disabled CPU features for the guest
>   KVM: arm64: Add kunit test for trap initialization
>   KVM: arm64: selftests: Introduce id_reg_test
> 
>  Documentation/virt/kvm/api.rst                |    8 +
>  arch/arm64/include/asm/cpufeature.h           |    2 +-
>  arch/arm64/include/asm/kvm_arm.h              |   32 +
>  arch/arm64/include/asm/kvm_host.h             |   15 +
>  arch/arm64/include/asm/sysreg.h               |    2 +
>  arch/arm64/kvm/Kconfig                        |   11 +
>  arch/arm64/kvm/arm.c                          |   12 +-
>  arch/arm64/kvm/debug.c                        |   13 +-
>  arch/arm64/kvm/hyp/vhe/switch.c               |   14 +-
>  arch/arm64/kvm/reset.c                        |    4 +
>  arch/arm64/kvm/sys_regs.c                     | 1265 +++++++++++++++--
>  arch/arm64/kvm/sys_regs_test.c                | 1109 +++++++++++++++
>  include/uapi/linux/kvm.h                      |    1 +
>  tools/arch/arm64/include/asm/sysreg.h         |    1 +
>  tools/testing/selftests/kvm/.gitignore        |    1 +
>  tools/testing/selftests/kvm/Makefile          |    1 +
>  .../selftests/kvm/aarch64/id_reg_test.c       | 1128 +++++++++++++++
>  17 files changed, 3488 insertions(+), 131 deletions(-)
>  create mode 100644 arch/arm64/kvm/sys_regs_test.c
>  create mode 100644 tools/testing/selftests/kvm/aarch64/id_reg_test.c
> 
> -- 
> 2.34.0.rc1.387.gb447b232ab-goog
> 

Re: [RFC PATCH v3 00/29] KVM: arm64: Make CPU ID registers writable by userspace

[Alexandru Elisei]

Hi Reiji,

The API documentation for KVM_ARM_VCPU_INIT states:

"Userspace can call this function multiple times for a given vcpu,
including after the vcpu has been run. This will reset the vcpu to its
initial state. All calls to this function after the initial call must use
the same target and same set of feature flags, otherwise EINVAL will be
returned."

The consequences of that, according to my understanding:

1. Any changes to the VCPU features made by KVM are observable by
userspace.

2. The features in KVM weren't designed and implemented to be disabled
after being enabled.

With that in mind, I have two questions:

1. What happens when userspace disables a feature via the ID registers
which is set in vcpu->arch.features? Does the feature bit get cleared from
vcpu->arch.features? Does it stay set? If it gets cleared, is it now
possible for userspace to call KVM_ARM_VCPU_INIT again with a different set
of VCPU features (it doesn't look possible to me after looking at the
code). If it stays set, what does it mean when userspace calls
KVM_ARM_VCPU_INIT with a different set of features enabled than what is
present in the ID registers? Should the ID registers be changed to match
the features that userspace set in the last KVM_ARM_VCPU_INIT call (it
looks to me that the ID registers are not changed)?

2. What happens to vcpu->arch.features when userspace enables a feature via
the ID registers which is not present in the bitmap?

Thanks,
Alex

On Tue, Nov 16, 2021 at 10:43:30PM -0800, Reiji Watanabe wrote:
> In KVM/arm64, values of ID registers for a guest are mostly same as
> its host's values except for bits for feature that KVM doesn't support
> and for opt-in features that userspace didn't configure.  Userspace
> can use KVM_SET_ONE_REG to a set ID register value, but it fails
> if userspace attempts to modify the register value.
> 
> This patch series adds support to allow userspace to modify a value of
> ID registers (as long as KVM can support features that are indicated
> in the registers) so userspace can have more control of configuring
> and unconfiguring features for guests.
> 
> The patch series is for both VHE and non-VHE, except for protected VMs,
> which have a different way of configuring ID registers based on its
> different requirements [1].
> There was a patch series that tried to achieve the same thing [2].
> A few snippets of codes in this series were inspired by or came from [2].
> 
> The initial value of ID registers for a vCPU will be the host's value
> with bits cleared for unsupported features and for opt-in features that
> were not configured. So, the initial value userspace can see (via
> KVM_GET_ONE_REG) is the upper limit that can be set for the register.
> Any requests to change the value that conflicts with opt-in features'
> configuration will fail.
> 
> When a guest tries to use a CPU feature that is not exposed to the guest,
> trapping it (to emulate a real CPU's behavior) would generally be a
> desirable behavior (when it is possible with no or little side effects).
> The later patches in the series add codes for this.  Only features that
> can be trapped independently will be trapped by this series though.
> 
> This series adds kunit tests for new functions in sys_regs.c (except for
> trivial ones), and these tests are enabled with a new configuration
> option 'CONFIG_KVM_KUNIT_TEST'.
> 
> The series is based on v5.16-rc1.
> 
> v3:
>   - Remove ID register consistency checking across vCPUs [Oliver]
>   - Change KVM_CAP_ARM_ID_REG_WRITABLE to
>     KVM_CAP_ARM_ID_REG_CONFIGURABLE [Oliver]
>   - Add KUnit testing for ID register validation and trap initialization.
>   - Change read_id_reg() to take care of ID_AA64PFR0_EL1.GIC
>   - Add a helper of read_id_reg() (__read_id_reg()) and use the helper
>     instead of directly using __vcpu_sys_reg()
>   - Change not to run kvm_id_regs_consistency_check() and
>     kvm_vcpu_init_traps() for protected VMs.
>   - Update selftest to remove test cases for ID register consistency
>     checking across vCPUs and to add test cases for ID_AA64PFR0_EL1.GIC.
> 
> v2: https://lore.kernel.org/all/20211103062520.1445832-1-reijiw@google.com/
>   - Remove unnecessary line breaks. [Andrew]
>   - Use @params for comments. [Andrew]
>   - Move arm64_check_features to arch/arm64/kvm/sys_regs.c and
>     change that KVM specific feature check function.  [Andrew]
>   - Remove unnecessary raz handling from __set_id_reg. [Andrew]
>   - Remove sys_val field from the initial id_reg_info and add it
>     in the later patch. [Andrew]
>   - Call id_reg->init() from id_reg_info_init(). [Andrew]
>   - Fix cpuid_feature_cap_perfmon_field() to convert 0xf to 0x0
>     (and use it in the following patches).
>   - Change kvm_vcpu_first_run_init to set has_run_once to false
>     when kvm_id_regs_consistency_check() fails.
>   - Add a patch to introduce id_reg_info for ID_AA64MMFR0_EL1,
>     which requires special validity checking for TGran*_2 fields.
>   - Add patches to introduce id_reg_info for ID_DFR1_EL1 and
>     ID_MMFR0_EL1, which are required due to arm64_check_features
>     implementation change.
>   - Add a new argument, which is a pointer to id_reg_info, for
>     id_reg_info's validate()
> 
> v1: https://lore.kernel.org/all/20211012043535.500493-1-reijiw@google.com/
> 
> [1] https://lore.kernel.org/kvmarm/20211010145636.1950948-1-tabba@google.com/
> [2] https://lore.kernel.org/kvm/20201102033422.657391-1-liangpeng10@huawei.com/
> 
> Reiji Watanabe (29):
>   KVM: arm64: Add has_reset_once flag for vcpu
>   KVM: arm64: Save ID registers' sanitized value per vCPU
>   KVM: arm64: Introduce struct id_reg_info
>   KVM: arm64: Make ID_AA64PFR0_EL1 writable
>   KVM: arm64: Make ID_AA64PFR1_EL1 writable
>   KVM: arm64: Make ID_AA64ISAR0_EL1 writable
>   KVM: arm64: Make ID_AA64ISAR1_EL1 writable
>   KVM: arm64: Make ID_AA64MMFR0_EL1 writable
>   KVM: arm64: Hide IMPLEMENTATION DEFINED PMU support for the guest
>   KVM: arm64: Make ID_AA64DFR0_EL1 writable
>   KVM: arm64: Make ID_DFR0_EL1 writable
>   KVM: arm64: Make ID_DFR1_EL1 writable
>   KVM: arm64: Make ID_MMFR0_EL1 writable
>   KVM: arm64: Make MVFR1_EL1 writable
>   KVM: arm64: Make ID registers without id_reg_info writable
>   KVM: arm64: Add consistency checking for frac fields of ID registers
>   KVM: arm64: Introduce KVM_CAP_ARM_ID_REG_CONFIGURABLE capability
>   KVM: arm64: Add kunit test for ID register validation
>   KVM: arm64: Use vcpu->arch cptr_el2 to track value of cptr_el2 for VHE
>   KVM: arm64: Use vcpu->arch.mdcr_el2 to track value of mdcr_el2
>   KVM: arm64: Introduce framework to trap disabled features
>   KVM: arm64: Trap disabled features of ID_AA64PFR0_EL1
>   KVM: arm64: Trap disabled features of ID_AA64PFR1_EL1
>   KVM: arm64: Trap disabled features of ID_AA64DFR0_EL1
>   KVM: arm64: Trap disabled features of ID_AA64MMFR1_EL1
>   KVM: arm64: Trap disabled features of ID_AA64ISAR1_EL1
>   KVM: arm64: Initialize trapping of disabled CPU features for the guest
>   KVM: arm64: Add kunit test for trap initialization
>   KVM: arm64: selftests: Introduce id_reg_test
> 
>  Documentation/virt/kvm/api.rst                |    8 +
>  arch/arm64/include/asm/cpufeature.h           |    2 +-
>  arch/arm64/include/asm/kvm_arm.h              |   32 +
>  arch/arm64/include/asm/kvm_host.h             |   15 +
>  arch/arm64/include/asm/sysreg.h               |    2 +
>  arch/arm64/kvm/Kconfig                        |   11 +
>  arch/arm64/kvm/arm.c                          |   12 +-
>  arch/arm64/kvm/debug.c                        |   13 +-
>  arch/arm64/kvm/hyp/vhe/switch.c               |   14 +-
>  arch/arm64/kvm/reset.c                        |    4 +
>  arch/arm64/kvm/sys_regs.c                     | 1265 +++++++++++++++--
>  arch/arm64/kvm/sys_regs_test.c                | 1109 +++++++++++++++
>  include/uapi/linux/kvm.h                      |    1 +
>  tools/arch/arm64/include/asm/sysreg.h         |    1 +
>  tools/testing/selftests/kvm/.gitignore        |    1 +
>  tools/testing/selftests/kvm/Makefile          |    1 +
>  .../selftests/kvm/aarch64/id_reg_test.c       | 1128 +++++++++++++++
>  17 files changed, 3488 insertions(+), 131 deletions(-)
>  create mode 100644 arch/arm64/kvm/sys_regs_test.c
>  create mode 100644 tools/testing/selftests/kvm/aarch64/id_reg_test.c
> 
> -- 
> 2.34.0.rc1.387.gb447b232ab-goog
> 

Re: [RFC PATCH v3 02/29] KVM: arm64: Save ID registers' sanitized value per vCPU

[Reiji Watanabe]

On Tue, Nov 23, 2021 at 2:03 AM Marc Zyngier <maz@kernel.org> wrote:
>
> On Tue, 23 Nov 2021 04:39:27 +0000,
> Reiji Watanabe <reijiw@google.com> wrote:
> >
> > On Sun, Nov 21, 2021 at 4:37 AM Marc Zyngier <maz@kernel.org> wrote:
> > >
> > > On Wed, 17 Nov 2021 06:43:32 +0000,
> > > Reiji Watanabe <reijiw@google.com> wrote:
> > > >
> > > > Extend sys_regs[] of kvm_cpu_context for ID registers and save ID
> > > > registers' sanitized value in the array for the vCPU at the first
> > > > vCPU reset. Use the saved ones when ID registers are read by
> > > > userspace (via KVM_GET_ONE_REG) or the guest.
> > > >
> > > > Signed-off-by: Reiji Watanabe <reijiw@google.com>
> > > > ---
> > > >  arch/arm64/include/asm/kvm_host.h | 10 +++++++
> > > >  arch/arm64/kvm/sys_regs.c         | 43 +++++++++++++++++++------------
> > > >  2 files changed, 37 insertions(+), 16 deletions(-)
> > > >
> > > > diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
> > > > index edbe2cb21947..72db73c79403 100644
> > > > --- a/arch/arm64/include/asm/kvm_host.h
> > > > +++ b/arch/arm64/include/asm/kvm_host.h
> > > > @@ -146,6 +146,14 @@ struct kvm_vcpu_fault_info {
> > > >       u64 disr_el1;           /* Deferred [SError] Status Register */
> > > >  };
> > > >
> > > > +/*
> > > > + * (Op0, Op1, CRn, CRm, Op2) of ID registers is (3, 0, 0, crm, op2),
> > > > + * where 0<=crm<8, 0<=op2<8.
> > > > + */
> > > > +#define KVM_ARM_ID_REG_MAX_NUM 64
> > > > +#define IDREG_IDX(id)                ((sys_reg_CRm(id) << 3) | sys_reg_Op2(id))
> > > > +#define IDREG_SYS_IDX(id)    (ID_REG_BASE + IDREG_IDX(id))
> > > > +
> > > >  enum vcpu_sysreg {
> > > >       __INVALID_SYSREG__,   /* 0 is reserved as an invalid value */
> > > >       MPIDR_EL1,      /* MultiProcessor Affinity Register */
> > > > @@ -210,6 +218,8 @@ enum vcpu_sysreg {
> > > >       CNTP_CVAL_EL0,
> > > >       CNTP_CTL_EL0,
> > > >
> > > > +     ID_REG_BASE,
> > > > +     ID_REG_END = ID_REG_BASE + KVM_ARM_ID_REG_MAX_NUM - 1,
> > >
> > > It is rather unclear to me why we want these registers to be
> > > replicated on a per-CPU basis. Yes, this fits the architecture, but
> > > that's also a total waste of memory if you have more than a single
> > > CPU, because we make a point in only exposing homogeneous properties
> > > to the VM (I don't think anyone intends to support vcpu asymmetry in a
> > > VM, and 64 registers per vcpu is not an insignificant memory usage).
> > >
> > > If there are no reasons for this to be per-CPU, please move it to be
> > > global to the VM. This also mean that once a vcpu has reset, it
> > > shouldn't be possible to affect the registers. This shouldn't affect
> > > the userspace API though.
> >
> >
> > Currently, userspace can configure different CPU features for each vCPU
> > with KVM_ARM_VCPU_INIT, which indirectly affect ID registers.
> > I'm not sure if anyone actually does that though.
>
> But the way the ID regs are affected is always global AFAICT. For
> example, if you instantiate a PMU, you do so on all vcpus, even of the
> architecture allows you to build something completely asymmetric.
>
> > Since I personally thought having ID registers per vCPU more naturally
> > fits the behavior of KVM_ARM_VCPU_INIT and makes more straightforward
> > behavior of KVM_SET_ONE_REG, I chose that.
>
> I agree that this is the logical approach from an architectural PoV.
>
> > (That would be also better in terms of vCPUs scalability for live migration
> >  considering a case where userspace tries to restore ID registers for
> >  many vCPUs in parallel during live migration.  Userspace could avoid that,
> >  and there are ways for KVM to mitigate that though.)
>
> I think these are orthogonal things. We can expose a per-vcpu view,
> but there is no need to have per-vcpu storage and to allow asymmetric
> VMs. If I have anything to say about the future of KVM/arm64, it will
> be that I don't want to support this at all.
>
> > Having ID registers per-VM is of course feasible even while maintaining
> > the current behavior of KVM_ARM_VCPU_INIT though.
>
> Exactly. per-VM storage, and per-vcpu visibility. It will prevent all
> sort of odd behaviours.

Thank you so much for all your comments.
I will make it per VM storage.

Regards,
Reiji

Re: [RFC PATCH v3 00/29] KVM: arm64: Make CPU ID registers writable by userspace

[Reiji Watanabe]

HI Alex,

On Tue, Nov 23, 2021 at 7:59 AM Alexandru Elisei
<alexandru.elisei@arm.com> wrote:
>
> Hi Reiji,
>
> I started reviewing the series, but I ended up being very confused, see
> below.
>
> On Tue, Nov 16, 2021 at 10:43:30PM -0800, Reiji Watanabe wrote:
> > In KVM/arm64, values of ID registers for a guest are mostly same as
> > its host's values except for bits for feature that KVM doesn't support
> > and for opt-in features that userspace didn't configure.  Userspace
> > can use KVM_SET_ONE_REG to a set ID register value, but it fails
> > if userspace attempts to modify the register value.
> >
> > This patch series adds support to allow userspace to modify a value of
> > ID registers (as long as KVM can support features that are indicated
> > in the registers) so userspace can have more control of configuring
> > and unconfiguring features for guests.
>
> What not use VCPU features? Isn't that why the field
> kvm_vcpu_init->features exists in the first place? This cover letter does
> nothing to explaing why any changes are needed.
>
> Do you require finer grained control over certain feature that you cannot
> get with the 32 * 7 = 224 feature flag bits from kvm_vcpu_init? Does using
> the ID registers simplify certain aspects of the implementation?

Since some features are not binary in nature (e.g. AA64DFR0_EL1.BRPs
fields indicate number of breakpoints minus 1), using
kvm_vcpu_init->features to configure such features is inconvenient.

One of the reasons why we want the finer grained control is that
we want to expose a uniform set/level of features for a group of
guests on systems with different ARM CPUs.

I will update the cover letter.

Thanks,
Reiji

>
> Thanks,
> Alex
>
> >
> > The patch series is for both VHE and non-VHE, except for protected VMs,
> > which have a different way of configuring ID registers based on its
> > different requirements [1].
> > There was a patch series that tried to achieve the same thing [2].
> > A few snippets of codes in this series were inspired by or came from [2].
> >
> > The initial value of ID registers for a vCPU will be the host's value
> > with bits cleared for unsupported features and for opt-in features that
> > were not configured. So, the initial value userspace can see (via
> > KVM_GET_ONE_REG) is the upper limit that can be set for the register.
> > Any requests to change the value that conflicts with opt-in features'
> > configuration will fail.
> >
> > When a guest tries to use a CPU feature that is not exposed to the guest,
> > trapping it (to emulate a real CPU's behavior) would generally be a
> > desirable behavior (when it is possible with no or little side effects).
> > The later patches in the series add codes for this.  Only features that
> > can be trapped independently will be trapped by this series though.
> >
> > This series adds kunit tests for new functions in sys_regs.c (except for
> > trivial ones), and these tests are enabled with a new configuration
> > option 'CONFIG_KVM_KUNIT_TEST'.
> >
> > The series is based on v5.16-rc1.
> >
> > v3:
> >   - Remove ID register consistency checking across vCPUs [Oliver]
> >   - Change KVM_CAP_ARM_ID_REG_WRITABLE to
> >     KVM_CAP_ARM_ID_REG_CONFIGURABLE [Oliver]
> >   - Add KUnit testing for ID register validation and trap initialization.
> >   - Change read_id_reg() to take care of ID_AA64PFR0_EL1.GIC
> >   - Add a helper of read_id_reg() (__read_id_reg()) and use the helper
> >     instead of directly using __vcpu_sys_reg()
> >   - Change not to run kvm_id_regs_consistency_check() and
> >     kvm_vcpu_init_traps() for protected VMs.
> >   - Update selftest to remove test cases for ID register consistency
> >     checking across vCPUs and to add test cases for ID_AA64PFR0_EL1.GIC.
> >
> > v2: https://lore.kernel.org/all/20211103062520.1445832-1-reijiw@google.com/
> >   - Remove unnecessary line breaks. [Andrew]
> >   - Use @params for comments. [Andrew]
> >   - Move arm64_check_features to arch/arm64/kvm/sys_regs.c and
> >     change that KVM specific feature check function.  [Andrew]
> >   - Remove unnecessary raz handling from __set_id_reg. [Andrew]
> >   - Remove sys_val field from the initial id_reg_info and add it
> >     in the later patch. [Andrew]
> >   - Call id_reg->init() from id_reg_info_init(). [Andrew]
> >   - Fix cpuid_feature_cap_perfmon_field() to convert 0xf to 0x0
> >     (and use it in the following patches).
> >   - Change kvm_vcpu_first_run_init to set has_run_once to false
> >     when kvm_id_regs_consistency_check() fails.
> >   - Add a patch to introduce id_reg_info for ID_AA64MMFR0_EL1,
> >     which requires special validity checking for TGran*_2 fields.
> >   - Add patches to introduce id_reg_info for ID_DFR1_EL1 and
> >     ID_MMFR0_EL1, which are required due to arm64_check_features
> >     implementation change.
> >   - Add a new argument, which is a pointer to id_reg_info, for
> >     id_reg_info's validate()
> >
> > v1: https://lore.kernel.org/all/20211012043535.500493-1-reijiw@google.com/
> >
> > [1] https://lore.kernel.org/kvmarm/20211010145636.1950948-1-tabba@google.com/
> > [2] https://lore.kernel.org/kvm/20201102033422.657391-1-liangpeng10@huawei.com/
> >
> > Reiji Watanabe (29):
> >   KVM: arm64: Add has_reset_once flag for vcpu
> >   KVM: arm64: Save ID registers' sanitized value per vCPU
> >   KVM: arm64: Introduce struct id_reg_info
> >   KVM: arm64: Make ID_AA64PFR0_EL1 writable
> >   KVM: arm64: Make ID_AA64PFR1_EL1 writable
> >   KVM: arm64: Make ID_AA64ISAR0_EL1 writable
> >   KVM: arm64: Make ID_AA64ISAR1_EL1 writable
> >   KVM: arm64: Make ID_AA64MMFR0_EL1 writable
> >   KVM: arm64: Hide IMPLEMENTATION DEFINED PMU support for the guest
> >   KVM: arm64: Make ID_AA64DFR0_EL1 writable
> >   KVM: arm64: Make ID_DFR0_EL1 writable
> >   KVM: arm64: Make ID_DFR1_EL1 writable
> >   KVM: arm64: Make ID_MMFR0_EL1 writable
> >   KVM: arm64: Make MVFR1_EL1 writable
> >   KVM: arm64: Make ID registers without id_reg_info writable
> >   KVM: arm64: Add consistency checking for frac fields of ID registers
> >   KVM: arm64: Introduce KVM_CAP_ARM_ID_REG_CONFIGURABLE capability
> >   KVM: arm64: Add kunit test for ID register validation
> >   KVM: arm64: Use vcpu->arch cptr_el2 to track value of cptr_el2 for VHE
> >   KVM: arm64: Use vcpu->arch.mdcr_el2 to track value of mdcr_el2
> >   KVM: arm64: Introduce framework to trap disabled features
> >   KVM: arm64: Trap disabled features of ID_AA64PFR0_EL1
> >   KVM: arm64: Trap disabled features of ID_AA64PFR1_EL1
> >   KVM: arm64: Trap disabled features of ID_AA64DFR0_EL1
> >   KVM: arm64: Trap disabled features of ID_AA64MMFR1_EL1
> >   KVM: arm64: Trap disabled features of ID_AA64ISAR1_EL1
> >   KVM: arm64: Initialize trapping of disabled CPU features for the guest
> >   KVM: arm64: Add kunit test for trap initialization
> >   KVM: arm64: selftests: Introduce id_reg_test
> >
> >  Documentation/virt/kvm/api.rst                |    8 +
> >  arch/arm64/include/asm/cpufeature.h           |    2 +-
> >  arch/arm64/include/asm/kvm_arm.h              |   32 +
> >  arch/arm64/include/asm/kvm_host.h             |   15 +
> >  arch/arm64/include/asm/sysreg.h               |    2 +
> >  arch/arm64/kvm/Kconfig                        |   11 +
> >  arch/arm64/kvm/arm.c                          |   12 +-
> >  arch/arm64/kvm/debug.c                        |   13 +-
> >  arch/arm64/kvm/hyp/vhe/switch.c               |   14 +-
> >  arch/arm64/kvm/reset.c                        |    4 +
> >  arch/arm64/kvm/sys_regs.c                     | 1265 +++++++++++++++--
> >  arch/arm64/kvm/sys_regs_test.c                | 1109 +++++++++++++++
> >  include/uapi/linux/kvm.h                      |    1 +
> >  tools/arch/arm64/include/asm/sysreg.h         |    1 +
> >  tools/testing/selftests/kvm/.gitignore        |    1 +
> >  tools/testing/selftests/kvm/Makefile          |    1 +
> >  .../selftests/kvm/aarch64/id_reg_test.c       | 1128 +++++++++++++++
> >  17 files changed, 3488 insertions(+), 131 deletions(-)
> >  create mode 100644 arch/arm64/kvm/sys_regs_test.c
> >  create mode 100644 tools/testing/selftests/kvm/aarch64/id_reg_test.c
> >
> > --
> > 2.34.0.rc1.387.gb447b232ab-goog
> >

Re: [RFC PATCH v3 00/29] KVM: arm64: Make CPU ID registers writable by userspace

[Reiji Watanabe]

Hi Alex,

On Tue, Nov 23, 2021 at 8:25 AM Alexandru Elisei
<alexandru.elisei@arm.com> wrote:
>
> Hi Reiji,
>
> The API documentation for KVM_ARM_VCPU_INIT states:
>
> "Userspace can call this function multiple times for a given vcpu,
> including after the vcpu has been run. This will reset the vcpu to its
> initial state. All calls to this function after the initial call must use
> the same target and same set of feature flags, otherwise EINVAL will be
> returned."
>
> The consequences of that, according to my understanding:
>
> 1. Any changes to the VCPU features made by KVM are observable by
> userspace.
>
> 2. The features in KVM weren't designed and implemented to be disabled
> after being enabled.
>
> With that in mind, I have two questions:
>
> 1. What happens when userspace disables a feature via the ID registers
> which is set in vcpu->arch.features? Does the feature bit get cleared from
> vcpu->arch.features? Does it stay set? If it gets cleared, is it now
> possible for userspace to call KVM_ARM_VCPU_INIT again with a different set
> of VCPU features (it doesn't look possible to me after looking at the
> code). If it stays set, what does it mean when userspace calls
> KVM_ARM_VCPU_INIT with a different set of features enabled than what is
> present in the ID registers? Should the ID registers be changed to match
> the features that userspace set in the last KVM_ARM_VCPU_INIT call (it
> looks to me that the ID registers are not changed)?

KVM will not allow userspace to set the ID register value that conflicts
with CPU features that are configured by the initial KVM_ARM_VCPU_INIT
(or there are a few more APIs).
KVM_SET_ONE_REG for such requests will fail.


> 2. What happens to vcpu->arch.features when userspace enables a feature via
> the ID registers which is not present in the bitmap?

The answer for this question is basically the same as above.
Userspace is not allowed to enable a feature via the ID registers
which is not present in the bit map.

The cover lever included a brief explanation of this, but I will
try to improve the explanation:-)

Regards,
Reiji

>
> Thanks,
> Alex
>
> On Tue, Nov 16, 2021 at 10:43:30PM -0800, Reiji Watanabe wrote:
> > In KVM/arm64, values of ID registers for a guest are mostly same as
> > its host's values except for bits for feature that KVM doesn't support
> > and for opt-in features that userspace didn't configure.  Userspace
> > can use KVM_SET_ONE_REG to a set ID register value, but it fails
> > if userspace attempts to modify the register value.
> >
> > This patch series adds support to allow userspace to modify a value of
> > ID registers (as long as KVM can support features that are indicated
> > in the registers) so userspace can have more control of configuring
> > and unconfiguring features for guests.
> >
> > The patch series is for both VHE and non-VHE, except for protected VMs,
> > which have a different way of configuring ID registers based on its
> > different requirements [1].
> > There was a patch series that tried to achieve the same thing [2].
> > A few snippets of codes in this series were inspired by or came from [2].
> >
> > The initial value of ID registers for a vCPU will be the host's value
> > with bits cleared for unsupported features and for opt-in features that
> > were not configured. So, the initial value userspace can see (via
> > KVM_GET_ONE_REG) is the upper limit that can be set for the register.
> > Any requests to change the value that conflicts with opt-in features'
> > configuration will fail.
> >
> > When a guest tries to use a CPU feature that is not exposed to the guest,
> > trapping it (to emulate a real CPU's behavior) would generally be a
> > desirable behavior (when it is possible with no or little side effects).
> > The later patches in the series add codes for this.  Only features that
> > can be trapped independently will be trapped by this series though.
> >
> > This series adds kunit tests for new functions in sys_regs.c (except for
> > trivial ones), and these tests are enabled with a new configuration
> > option 'CONFIG_KVM_KUNIT_TEST'.
> >
> > The series is based on v5.16-rc1.
> >
> > v3:
> >   - Remove ID register consistency checking across vCPUs [Oliver]
> >   - Change KVM_CAP_ARM_ID_REG_WRITABLE to
> >     KVM_CAP_ARM_ID_REG_CONFIGURABLE [Oliver]
> >   - Add KUnit testing for ID register validation and trap initialization.
> >   - Change read_id_reg() to take care of ID_AA64PFR0_EL1.GIC
> >   - Add a helper of read_id_reg() (__read_id_reg()) and use the helper
> >     instead of directly using __vcpu_sys_reg()
> >   - Change not to run kvm_id_regs_consistency_check() and
> >     kvm_vcpu_init_traps() for protected VMs.
> >   - Update selftest to remove test cases for ID register consistency
> >     checking across vCPUs and to add test cases for ID_AA64PFR0_EL1.GIC.
> >
> > v2: https://lore.kernel.org/all/20211103062520.1445832-1-reijiw@google.com/
> >   - Remove unnecessary line breaks. [Andrew]
> >   - Use @params for comments. [Andrew]
> >   - Move arm64_check_features to arch/arm64/kvm/sys_regs.c and
> >     change that KVM specific feature check function.  [Andrew]
> >   - Remove unnecessary raz handling from __set_id_reg. [Andrew]
> >   - Remove sys_val field from the initial id_reg_info and add it
> >     in the later patch. [Andrew]
> >   - Call id_reg->init() from id_reg_info_init(). [Andrew]
> >   - Fix cpuid_feature_cap_perfmon_field() to convert 0xf to 0x0
> >     (and use it in the following patches).
> >   - Change kvm_vcpu_first_run_init to set has_run_once to false
> >     when kvm_id_regs_consistency_check() fails.
> >   - Add a patch to introduce id_reg_info for ID_AA64MMFR0_EL1,
> >     which requires special validity checking for TGran*_2 fields.
> >   - Add patches to introduce id_reg_info for ID_DFR1_EL1 and
> >     ID_MMFR0_EL1, which are required due to arm64_check_features
> >     implementation change.
> >   - Add a new argument, which is a pointer to id_reg_info, for
> >     id_reg_info's validate()
> >
> > v1: https://lore.kernel.org/all/20211012043535.500493-1-reijiw@google.com/
> >
> > [1] https://lore.kernel.org/kvmarm/20211010145636.1950948-1-tabba@google.com/
> > [2] https://lore.kernel.org/kvm/20201102033422.657391-1-liangpeng10@huawei.com/
> >
> > Reiji Watanabe (29):
> >   KVM: arm64: Add has_reset_once flag for vcpu
> >   KVM: arm64: Save ID registers' sanitized value per vCPU
> >   KVM: arm64: Introduce struct id_reg_info
> >   KVM: arm64: Make ID_AA64PFR0_EL1 writable
> >   KVM: arm64: Make ID_AA64PFR1_EL1 writable
> >   KVM: arm64: Make ID_AA64ISAR0_EL1 writable
> >   KVM: arm64: Make ID_AA64ISAR1_EL1 writable
> >   KVM: arm64: Make ID_AA64MMFR0_EL1 writable
> >   KVM: arm64: Hide IMPLEMENTATION DEFINED PMU support for the guest
> >   KVM: arm64: Make ID_AA64DFR0_EL1 writable
> >   KVM: arm64: Make ID_DFR0_EL1 writable
> >   KVM: arm64: Make ID_DFR1_EL1 writable
> >   KVM: arm64: Make ID_MMFR0_EL1 writable
> >   KVM: arm64: Make MVFR1_EL1 writable
> >   KVM: arm64: Make ID registers without id_reg_info writable
> >   KVM: arm64: Add consistency checking for frac fields of ID registers
> >   KVM: arm64: Introduce KVM_CAP_ARM_ID_REG_CONFIGURABLE capability
> >   KVM: arm64: Add kunit test for ID register validation
> >   KVM: arm64: Use vcpu->arch cptr_el2 to track value of cptr_el2 for VHE
> >   KVM: arm64: Use vcpu->arch.mdcr_el2 to track value of mdcr_el2
> >   KVM: arm64: Introduce framework to trap disabled features
> >   KVM: arm64: Trap disabled features of ID_AA64PFR0_EL1
> >   KVM: arm64: Trap disabled features of ID_AA64PFR1_EL1
> >   KVM: arm64: Trap disabled features of ID_AA64DFR0_EL1
> >   KVM: arm64: Trap disabled features of ID_AA64MMFR1_EL1
> >   KVM: arm64: Trap disabled features of ID_AA64ISAR1_EL1
> >   KVM: arm64: Initialize trapping of disabled CPU features for the guest
> >   KVM: arm64: Add kunit test for trap initialization
> >   KVM: arm64: selftests: Introduce id_reg_test
> >
> >  Documentation/virt/kvm/api.rst                |    8 +
> >  arch/arm64/include/asm/cpufeature.h           |    2 +-
> >  arch/arm64/include/asm/kvm_arm.h              |   32 +
> >  arch/arm64/include/asm/kvm_host.h             |   15 +
> >  arch/arm64/include/asm/sysreg.h               |    2 +
> >  arch/arm64/kvm/Kconfig                        |   11 +
> >  arch/arm64/kvm/arm.c                          |   12 +-
> >  arch/arm64/kvm/debug.c                        |   13 +-
> >  arch/arm64/kvm/hyp/vhe/switch.c               |   14 +-
> >  arch/arm64/kvm/reset.c                        |    4 +
> >  arch/arm64/kvm/sys_regs.c                     | 1265 +++++++++++++++--
> >  arch/arm64/kvm/sys_regs_test.c                | 1109 +++++++++++++++
> >  include/uapi/linux/kvm.h                      |    1 +
> >  tools/arch/arm64/include/asm/sysreg.h         |    1 +
> >  tools/testing/selftests/kvm/.gitignore        |    1 +
> >  tools/testing/selftests/kvm/Makefile          |    1 +
> >  .../selftests/kvm/aarch64/id_reg_test.c       | 1128 +++++++++++++++
> >  17 files changed, 3488 insertions(+), 131 deletions(-)
> >  create mode 100644 arch/arm64/kvm/sys_regs_test.c
> >  create mode 100644 tools/testing/selftests/kvm/aarch64/id_reg_test.c
> >
> > --
> > 2.34.0.rc1.387.gb447b232ab-goog
> >

Re: [RFC PATCH v3 04/29] KVM: arm64: Make ID_AA64PFR0_EL1 writable

[Reiji Watanabe]

On Sun, Nov 21, 2021 at 4:37 AM Marc Zyngier <maz@kernel.org> wrote:
>
> On Wed, 17 Nov 2021 06:43:34 +0000,
> Reiji Watanabe <reijiw@google.com> wrote:
> >
> > This patch adds id_reg_info for ID_AA64PFR0_EL1 to make it writable by
> > userspace.
> >
> > The CSV2/CSV3 fields of the register were already writable and values
> > that were written for them affected all vCPUs before. Now they only
> > affect the vCPU.
> > Return an error if userspace tries to set SVE/GIC field of the register
> > to a value that conflicts with SVE/GIC configuration for the guest.
> > SIMD/FP/SVE fields of the requested value are validated according to
> > Arm ARM.
> >
> > Signed-off-by: Reiji Watanabe <reijiw@google.com>
> > ---
> >  arch/arm64/kvm/sys_regs.c | 159 ++++++++++++++++++++++++--------------
> >  1 file changed, 103 insertions(+), 56 deletions(-)
> >
> > diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
> > index 1552cd5581b7..35400869067a 100644
> > --- a/arch/arm64/kvm/sys_regs.c
> > +++ b/arch/arm64/kvm/sys_regs.c
> > @@ -401,6 +401,92 @@ static void id_reg_info_init(struct id_reg_info *id_reg)
> >               id_reg->init(id_reg);
> >  }
> >
> > +#define      kvm_has_gic3(kvm)               \
> > +     (irqchip_in_kernel(kvm) &&      \
> > +      (kvm)->arch.vgic.vgic_model == KVM_DEV_TYPE_ARM_VGIC_V3)
> > +
> > +static int validate_id_aa64pfr0_el1(struct kvm_vcpu *vcpu,
> > +                                 const struct id_reg_info *id_reg, u64 val)
> > +{
> > +     int fp, simd;
> > +     bool vcpu_has_sve = vcpu_has_sve(vcpu);
> > +     bool pfr0_has_sve = id_aa64pfr0_sve(val);
> > +     int gic;
> > +
> > +     simd = cpuid_feature_extract_signed_field(val, ID_AA64PFR0_ASIMD_SHIFT);
> > +     fp = cpuid_feature_extract_signed_field(val, ID_AA64PFR0_FP_SHIFT);
> > +     if (simd != fp)
> > +             return -EINVAL;
> > +
> > +     /* fp must be supported when sve is supported */
> > +     if (pfr0_has_sve && (fp < 0))
> > +             return -EINVAL;
> > +
> > +     /* Check if there is a conflict with a request via KVM_ARM_VCPU_INIT */
> > +     if (vcpu_has_sve ^ pfr0_has_sve)
> > +             return -EPERM;
> > +
> > +     gic = cpuid_feature_extract_unsigned_field(val, ID_AA64PFR0_GIC_SHIFT);
> > +     if ((gic > 0) ^ kvm_has_gic3(vcpu->kvm))
> > +             return -EPERM;
> > +
> > +     return 0;
> > +}
> > +
> > +static void init_id_aa64pfr0_el1_info(struct id_reg_info *id_reg)
> > +{
> > +     u64 limit = id_reg->vcpu_limit_val;
> > +     unsigned int gic;
> > +
> > +     limit &= ~ARM64_FEATURE_MASK(ID_AA64PFR0_AMU);
> > +     if (!system_supports_sve())
> > +             limit &= ~ARM64_FEATURE_MASK(ID_AA64PFR0_SVE);
> > +
> > +     /*
> > +      * The default is to expose CSV2 == 1 and CSV3 == 1 if the HW
> > +      * isn't affected.  Userspace can override this as long as it
> > +      * doesn't promise the impossible.
> > +      */
> > +     limit &= ~(ARM64_FEATURE_MASK(ID_AA64PFR0_CSV2) |
> > +                ARM64_FEATURE_MASK(ID_AA64PFR0_CSV3));
> > +
> > +     if (arm64_get_spectre_v2_state() == SPECTRE_UNAFFECTED)
> > +             limit |= FIELD_PREP(ARM64_FEATURE_MASK(ID_AA64PFR0_CSV2), 1);
> > +     if (arm64_get_meltdown_state() == SPECTRE_UNAFFECTED)
> > +             limit |= FIELD_PREP(ARM64_FEATURE_MASK(ID_AA64PFR0_CSV3), 1);
> > +
> > +     gic = cpuid_feature_extract_unsigned_field(limit, ID_AA64PFR0_GIC_SHIFT);
> > +     if (gic > 1) {
> > +             /* Limit to GICv3.0/4.0 */
> > +             limit &= ~ARM64_FEATURE_MASK(ID_AA64PFR0_GIC);
> > +             limit |= FIELD_PREP(ARM64_FEATURE_MASK(ID_AA64PFR0_GIC), 1);
> > +     }
> > +     id_reg->vcpu_limit_val = limit;
> > +}
> > +
> > +static u64 get_reset_id_aa64pfr0_el1(struct kvm_vcpu *vcpu,
> > +                                  const struct id_reg_info *idr)
> > +{
> > +     u64 val = idr->vcpu_limit_val;
> > +
> > +     if (!vcpu_has_sve(vcpu))
> > +             val &= ~ARM64_FEATURE_MASK(ID_AA64PFR0_SVE);
> > +
> > +     if (!kvm_has_gic3(vcpu->kvm))
> > +             val &= ~ARM64_FEATURE_MASK(ID_AA64PFR0_GIC);
>
> No. As I said in a previous email, this breaks migration, and
> advertising a GICv3 CPU interface doesn't mean it is usable (the guest
> OS must check that it can actually enable ICC_SRE_EL1.SRE -- see what
> the Linux GICv3 driver does for an example).

Yes, I understand. I will remove that code.

> > +     return val;
> > +}
> > +
> > +static struct id_reg_info id_aa64pfr0_el1_info = {
> > +     .sys_reg = SYS_ID_AA64PFR0_EL1,
> > +     .ftr_check_types = S_FCT(ID_AA64PFR0_ASIMD_SHIFT, FCT_LOWER_SAFE) |
> > +                        S_FCT(ID_AA64PFR0_FP_SHIFT, FCT_LOWER_SAFE),
> > +     .init = init_id_aa64pfr0_el1_info,
> > +     .validate = validate_id_aa64pfr0_el1,
> > +     .get_reset_val = get_reset_id_aa64pfr0_el1,
> > +};
> > +
> >  /*
> >   * An ID register that needs special handling to control the value for the
> >   * guest must have its own id_reg_info in id_reg_info_table.
> > @@ -409,7 +495,9 @@ static void id_reg_info_init(struct id_reg_info *id_reg)
> >   * validation, etc.)
> >   */
> >  #define      GET_ID_REG_INFO(id)     (id_reg_info_table[IDREG_IDX(id)])
> > -static struct id_reg_info *id_reg_info_table[KVM_ARM_ID_REG_MAX_NUM] = {};
> > +static struct id_reg_info *id_reg_info_table[KVM_ARM_ID_REG_MAX_NUM] = {
> > +     [IDREG_IDX(SYS_ID_AA64PFR0_EL1)] = &id_aa64pfr0_el1_info,
> > +};
> >
> >  static int validate_id_reg(struct kvm_vcpu *vcpu,
> >                          const struct sys_reg_desc *rd, u64 val)
> > @@ -1239,20 +1327,22 @@ static bool access_arch_timer(struct kvm_vcpu *vcpu,
> >  static u64 __read_id_reg(const struct kvm_vcpu *vcpu, u32 id)
> >  {
> >       u64 val = __vcpu_sys_reg(vcpu, IDREG_SYS_IDX(id));
> > +     u64 lim, gic, gic_lim;
> > +     const struct id_reg_info *id_reg;
> >
> >       switch (id) {
> >       case SYS_ID_AA64PFR0_EL1:
> > -             if (!vcpu_has_sve(vcpu))
> > -                     val &= ~ARM64_FEATURE_MASK(ID_AA64PFR0_SVE);
> > -             val &= ~ARM64_FEATURE_MASK(ID_AA64PFR0_AMU);
> > -             val &= ~ARM64_FEATURE_MASK(ID_AA64PFR0_CSV2);
> > -             val |= FIELD_PREP(ARM64_FEATURE_MASK(ID_AA64PFR0_CSV2), (u64)vcpu->kvm->arch.pfr0_csv2);
> > -             val &= ~ARM64_FEATURE_MASK(ID_AA64PFR0_CSV3);
> > -             val |= FIELD_PREP(ARM64_FEATURE_MASK(ID_AA64PFR0_CSV3), (u64)vcpu->kvm->arch.pfr0_csv3);
> > -             if (irqchip_in_kernel(vcpu->kvm) &&
> > -                 vcpu->kvm->arch.vgic.vgic_model == KVM_DEV_TYPE_ARM_VGIC_V3) {
> > -                     val &= ~ARM64_FEATURE_MASK(ID_AA64PFR0_GIC);
> > -                     val |= FIELD_PREP(ARM64_FEATURE_MASK(ID_AA64PFR0_GIC), 1);
> > +             gic = cpuid_feature_extract_unsigned_field(val, ID_AA64PFR0_GIC_SHIFT);
> > +             if (kvm_has_gic3(vcpu->kvm) && (gic == 0)) {
> > +                     /*
> > +                      * This is a case where userspace configured gic3 after
> > +                      * the vcpu was created, and then it didn't set
> > +                      * ID_AA64PFR0_EL1.
> > +                      */
>
> Shouldn't that be done at the point where a GICv3 is created, rather
> than after the fact?

I will look into having it done at the point where a GICv3 is created.
(I originally chose this way because I wanted to avoid access to
other vCPUs' ID registers if possible)

Thanks,
Reiji

Re: [RFC PATCH v3 00/29] KVM: arm64: Make CPU ID registers writable by userspace

[Alexandru Elisei]

Hi Reiji,

On Tue, Nov 23, 2021 at 09:49:02PM -0800, Reiji Watanabe wrote:
> Hi Alex,
> 
> On Tue, Nov 23, 2021 at 8:25 AM Alexandru Elisei
> <alexandru.elisei@arm.com> wrote:
> >
> > Hi Reiji,
> >
> > The API documentation for KVM_ARM_VCPU_INIT states:
> >
> > "Userspace can call this function multiple times for a given vcpu,
> > including after the vcpu has been run. This will reset the vcpu to its
> > initial state. All calls to this function after the initial call must use
> > the same target and same set of feature flags, otherwise EINVAL will be
> > returned."
> >
> > The consequences of that, according to my understanding:
> >
> > 1. Any changes to the VCPU features made by KVM are observable by
> > userspace.
> >
> > 2. The features in KVM weren't designed and implemented to be disabled
> > after being enabled.
> >
> > With that in mind, I have two questions:
> >
> > 1. What happens when userspace disables a feature via the ID registers
> > which is set in vcpu->arch.features? Does the feature bit get cleared from
> > vcpu->arch.features? Does it stay set? If it gets cleared, is it now
> > possible for userspace to call KVM_ARM_VCPU_INIT again with a different set
> > of VCPU features (it doesn't look possible to me after looking at the
> > code). If it stays set, what does it mean when userspace calls
> > KVM_ARM_VCPU_INIT with a different set of features enabled than what is
> > present in the ID registers? Should the ID registers be changed to match
> > the features that userspace set in the last KVM_ARM_VCPU_INIT call (it
> > looks to me that the ID registers are not changed)?
> 
> KVM will not allow userspace to set the ID register value that conflicts
> with CPU features that are configured by the initial KVM_ARM_VCPU_INIT
> (or there are a few more APIs).
> KVM_SET_ONE_REG for such requests will fail.
> 
> 
> > 2. What happens to vcpu->arch.features when userspace enables a feature via
> > the ID registers which is not present in the bitmap?
> 
> The answer for this question is basically the same as above.
> Userspace is not allowed to enable a feature via the ID registers
> which is not present in the bit map.
> 
> The cover lever included a brief explanation of this, but I will
> try to improve the explanation:-)

So the ID registers are used to set the version of a feature which is present in
the VCPU features bitmap, not to enable or a disable a VCPU feature. Thank you
for the explanation!

Thanks,
Alex

> 
> Regards,
> Reiji
> 
> >
> > Thanks,
> > Alex
> >
> > On Tue, Nov 16, 2021 at 10:43:30PM -0800, Reiji Watanabe wrote:
> > > In KVM/arm64, values of ID registers for a guest are mostly same as
> > > its host's values except for bits for feature that KVM doesn't support
> > > and for opt-in features that userspace didn't configure.  Userspace
> > > can use KVM_SET_ONE_REG to a set ID register value, but it fails
> > > if userspace attempts to modify the register value.
> > >
> > > This patch series adds support to allow userspace to modify a value of
> > > ID registers (as long as KVM can support features that are indicated
> > > in the registers) so userspace can have more control of configuring
> > > and unconfiguring features for guests.
> > >
> > > The patch series is for both VHE and non-VHE, except for protected VMs,
> > > which have a different way of configuring ID registers based on its
> > > different requirements [1].
> > > There was a patch series that tried to achieve the same thing [2].
> > > A few snippets of codes in this series were inspired by or came from [2].
> > >
> > > The initial value of ID registers for a vCPU will be the host's value
> > > with bits cleared for unsupported features and for opt-in features that
> > > were not configured. So, the initial value userspace can see (via
> > > KVM_GET_ONE_REG) is the upper limit that can be set for the register.
> > > Any requests to change the value that conflicts with opt-in features'
> > > configuration will fail.
> > >
> > > When a guest tries to use a CPU feature that is not exposed to the guest,
> > > trapping it (to emulate a real CPU's behavior) would generally be a
> > > desirable behavior (when it is possible with no or little side effects).
> > > The later patches in the series add codes for this.  Only features that
> > > can be trapped independently will be trapped by this series though.
> > >
> > > This series adds kunit tests for new functions in sys_regs.c (except for
> > > trivial ones), and these tests are enabled with a new configuration
> > > option 'CONFIG_KVM_KUNIT_TEST'.
> > >
> > > The series is based on v5.16-rc1.
> > >
> > > v3:
> > >   - Remove ID register consistency checking across vCPUs [Oliver]
> > >   - Change KVM_CAP_ARM_ID_REG_WRITABLE to
> > >     KVM_CAP_ARM_ID_REG_CONFIGURABLE [Oliver]
> > >   - Add KUnit testing for ID register validation and trap initialization.
> > >   - Change read_id_reg() to take care of ID_AA64PFR0_EL1.GIC
> > >   - Add a helper of read_id_reg() (__read_id_reg()) and use the helper
> > >     instead of directly using __vcpu_sys_reg()
> > >   - Change not to run kvm_id_regs_consistency_check() and
> > >     kvm_vcpu_init_traps() for protected VMs.
> > >   - Update selftest to remove test cases for ID register consistency
> > >     checking across vCPUs and to add test cases for ID_AA64PFR0_EL1.GIC.
> > >
> > > v2: https://lore.kernel.org/all/20211103062520.1445832-1-reijiw@google.com/
> > >   - Remove unnecessary line breaks. [Andrew]
> > >   - Use @params for comments. [Andrew]
> > >   - Move arm64_check_features to arch/arm64/kvm/sys_regs.c and
> > >     change that KVM specific feature check function.  [Andrew]
> > >   - Remove unnecessary raz handling from __set_id_reg. [Andrew]
> > >   - Remove sys_val field from the initial id_reg_info and add it
> > >     in the later patch. [Andrew]
> > >   - Call id_reg->init() from id_reg_info_init(). [Andrew]
> > >   - Fix cpuid_feature_cap_perfmon_field() to convert 0xf to 0x0
> > >     (and use it in the following patches).
> > >   - Change kvm_vcpu_first_run_init to set has_run_once to false
> > >     when kvm_id_regs_consistency_check() fails.
> > >   - Add a patch to introduce id_reg_info for ID_AA64MMFR0_EL1,
> > >     which requires special validity checking for TGran*_2 fields.
> > >   - Add patches to introduce id_reg_info for ID_DFR1_EL1 and
> > >     ID_MMFR0_EL1, which are required due to arm64_check_features
> > >     implementation change.
> > >   - Add a new argument, which is a pointer to id_reg_info, for
> > >     id_reg_info's validate()
> > >
> > > v1: https://lore.kernel.org/all/20211012043535.500493-1-reijiw@google.com/
> > >
> > > [1] https://lore.kernel.org/kvmarm/20211010145636.1950948-1-tabba@google.com/
> > > [2] https://lore.kernel.org/kvm/20201102033422.657391-1-liangpeng10@huawei.com/
> > >
> > > Reiji Watanabe (29):
> > >   KVM: arm64: Add has_reset_once flag for vcpu
> > >   KVM: arm64: Save ID registers' sanitized value per vCPU
> > >   KVM: arm64: Introduce struct id_reg_info
> > >   KVM: arm64: Make ID_AA64PFR0_EL1 writable
> > >   KVM: arm64: Make ID_AA64PFR1_EL1 writable
> > >   KVM: arm64: Make ID_AA64ISAR0_EL1 writable
> > >   KVM: arm64: Make ID_AA64ISAR1_EL1 writable
> > >   KVM: arm64: Make ID_AA64MMFR0_EL1 writable
> > >   KVM: arm64: Hide IMPLEMENTATION DEFINED PMU support for the guest
> > >   KVM: arm64: Make ID_AA64DFR0_EL1 writable
> > >   KVM: arm64: Make ID_DFR0_EL1 writable
> > >   KVM: arm64: Make ID_DFR1_EL1 writable
> > >   KVM: arm64: Make ID_MMFR0_EL1 writable
> > >   KVM: arm64: Make MVFR1_EL1 writable
> > >   KVM: arm64: Make ID registers without id_reg_info writable
> > >   KVM: arm64: Add consistency checking for frac fields of ID registers
> > >   KVM: arm64: Introduce KVM_CAP_ARM_ID_REG_CONFIGURABLE capability
> > >   KVM: arm64: Add kunit test for ID register validation
> > >   KVM: arm64: Use vcpu->arch cptr_el2 to track value of cptr_el2 for VHE
> > >   KVM: arm64: Use vcpu->arch.mdcr_el2 to track value of mdcr_el2
> > >   KVM: arm64: Introduce framework to trap disabled features
> > >   KVM: arm64: Trap disabled features of ID_AA64PFR0_EL1
> > >   KVM: arm64: Trap disabled features of ID_AA64PFR1_EL1
> > >   KVM: arm64: Trap disabled features of ID_AA64DFR0_EL1
> > >   KVM: arm64: Trap disabled features of ID_AA64MMFR1_EL1
> > >   KVM: arm64: Trap disabled features of ID_AA64ISAR1_EL1
> > >   KVM: arm64: Initialize trapping of disabled CPU features for the guest
> > >   KVM: arm64: Add kunit test for trap initialization
> > >   KVM: arm64: selftests: Introduce id_reg_test
> > >
> > >  Documentation/virt/kvm/api.rst                |    8 +
> > >  arch/arm64/include/asm/cpufeature.h           |    2 +-
> > >  arch/arm64/include/asm/kvm_arm.h              |   32 +
> > >  arch/arm64/include/asm/kvm_host.h             |   15 +
> > >  arch/arm64/include/asm/sysreg.h               |    2 +
> > >  arch/arm64/kvm/Kconfig                        |   11 +
> > >  arch/arm64/kvm/arm.c                          |   12 +-
> > >  arch/arm64/kvm/debug.c                        |   13 +-
> > >  arch/arm64/kvm/hyp/vhe/switch.c               |   14 +-
> > >  arch/arm64/kvm/reset.c                        |    4 +
> > >  arch/arm64/kvm/sys_regs.c                     | 1265 +++++++++++++++--
> > >  arch/arm64/kvm/sys_regs_test.c                | 1109 +++++++++++++++
> > >  include/uapi/linux/kvm.h                      |    1 +
> > >  tools/arch/arm64/include/asm/sysreg.h         |    1 +
> > >  tools/testing/selftests/kvm/.gitignore        |    1 +
> > >  tools/testing/selftests/kvm/Makefile          |    1 +
> > >  .../selftests/kvm/aarch64/id_reg_test.c       | 1128 +++++++++++++++
> > >  17 files changed, 3488 insertions(+), 131 deletions(-)
> > >  create mode 100644 arch/arm64/kvm/sys_regs_test.c
> > >  create mode 100644 tools/testing/selftests/kvm/aarch64/id_reg_test.c
> > >
> > > --
> > > 2.34.0.rc1.387.gb447b232ab-goog
> > >

Re: [RFC PATCH v3 00/29] KVM: arm64: Make CPU ID registers writable by userspace

[Alexandru Elisei]

Hi Reiji,

On Tue, Nov 23, 2021 at 09:13:27PM -0800, Reiji Watanabe wrote:
> HI Alex,
> 
> On Tue, Nov 23, 2021 at 7:59 AM Alexandru Elisei
> <alexandru.elisei@arm.com> wrote:
> >
> > Hi Reiji,
> >
> > I started reviewing the series, but I ended up being very confused, see
> > below.
> >
> > On Tue, Nov 16, 2021 at 10:43:30PM -0800, Reiji Watanabe wrote:
> > > In KVM/arm64, values of ID registers for a guest are mostly same as
> > > its host's values except for bits for feature that KVM doesn't support
> > > and for opt-in features that userspace didn't configure.  Userspace
> > > can use KVM_SET_ONE_REG to a set ID register value, but it fails
> > > if userspace attempts to modify the register value.
> > >
> > > This patch series adds support to allow userspace to modify a value of
> > > ID registers (as long as KVM can support features that are indicated
> > > in the registers) so userspace can have more control of configuring
> > > and unconfiguring features for guests.
> >
> > What not use VCPU features? Isn't that why the field
> > kvm_vcpu_init->features exists in the first place? This cover letter does
> > nothing to explaing why any changes are needed.
> >
> > Do you require finer grained control over certain feature that you cannot
> > get with the 32 * 7 = 224 feature flag bits from kvm_vcpu_init? Does using
> > the ID registers simplify certain aspects of the implementation?
> 
> Since some features are not binary in nature (e.g. AA64DFR0_EL1.BRPs
> fields indicate number of breakpoints minus 1), using
> kvm_vcpu_init->features to configure such features is inconvenient.

I see, this makes a lot of sense and this looks like a nice solution to
that problem.

> 
> One of the reasons why we want the finer grained control is that
> we want to expose a uniform set/level of features for a group of
> guests on systems with different ARM CPUs.

So here you are talking specifically about KVM not checking that all VCPUs
have the same feature bits set in vcpu->arch.features, which makes it
possible for userspace to set different features for different VCPUs,
right?

Thanks,
Alex

> 
> I will update the cover letter.
> 
> Thanks,
> Reiji
> 
> >
> > Thanks,
> > Alex
> >
> > >
> > > The patch series is for both VHE and non-VHE, except for protected VMs,
> > > which have a different way of configuring ID registers based on its
> > > different requirements [1].
> > > There was a patch series that tried to achieve the same thing [2].
> > > A few snippets of codes in this series were inspired by or came from [2].
> > >
> > > The initial value of ID registers for a vCPU will be the host's value
> > > with bits cleared for unsupported features and for opt-in features that
> > > were not configured. So, the initial value userspace can see (via
> > > KVM_GET_ONE_REG) is the upper limit that can be set for the register.
> > > Any requests to change the value that conflicts with opt-in features'
> > > configuration will fail.
> > >
> > > When a guest tries to use a CPU feature that is not exposed to the guest,
> > > trapping it (to emulate a real CPU's behavior) would generally be a
> > > desirable behavior (when it is possible with no or little side effects).
> > > The later patches in the series add codes for this.  Only features that
> > > can be trapped independently will be trapped by this series though.
> > >
> > > This series adds kunit tests for new functions in sys_regs.c (except for
> > > trivial ones), and these tests are enabled with a new configuration
> > > option 'CONFIG_KVM_KUNIT_TEST'.
> > >
> > > The series is based on v5.16-rc1.
> > >
> > > v3:
> > >   - Remove ID register consistency checking across vCPUs [Oliver]
> > >   - Change KVM_CAP_ARM_ID_REG_WRITABLE to
> > >     KVM_CAP_ARM_ID_REG_CONFIGURABLE [Oliver]
> > >   - Add KUnit testing for ID register validation and trap initialization.
> > >   - Change read_id_reg() to take care of ID_AA64PFR0_EL1.GIC
> > >   - Add a helper of read_id_reg() (__read_id_reg()) and use the helper
> > >     instead of directly using __vcpu_sys_reg()
> > >   - Change not to run kvm_id_regs_consistency_check() and
> > >     kvm_vcpu_init_traps() for protected VMs.
> > >   - Update selftest to remove test cases for ID register consistency
> > >     checking across vCPUs and to add test cases for ID_AA64PFR0_EL1.GIC.
> > >
> > > v2: https://lore.kernel.org/all/20211103062520.1445832-1-reijiw@google.com/
> > >   - Remove unnecessary line breaks. [Andrew]
> > >   - Use @params for comments. [Andrew]
> > >   - Move arm64_check_features to arch/arm64/kvm/sys_regs.c and
> > >     change that KVM specific feature check function.  [Andrew]
> > >   - Remove unnecessary raz handling from __set_id_reg. [Andrew]
> > >   - Remove sys_val field from the initial id_reg_info and add it
> > >     in the later patch. [Andrew]
> > >   - Call id_reg->init() from id_reg_info_init(). [Andrew]
> > >   - Fix cpuid_feature_cap_perfmon_field() to convert 0xf to 0x0
> > >     (and use it in the following patches).
> > >   - Change kvm_vcpu_first_run_init to set has_run_once to false
> > >     when kvm_id_regs_consistency_check() fails.
> > >   - Add a patch to introduce id_reg_info for ID_AA64MMFR0_EL1,
> > >     which requires special validity checking for TGran*_2 fields.
> > >   - Add patches to introduce id_reg_info for ID_DFR1_EL1 and
> > >     ID_MMFR0_EL1, which are required due to arm64_check_features
> > >     implementation change.
> > >   - Add a new argument, which is a pointer to id_reg_info, for
> > >     id_reg_info's validate()
> > >
> > > v1: https://lore.kernel.org/all/20211012043535.500493-1-reijiw@google.com/
> > >
> > > [1] https://lore.kernel.org/kvmarm/20211010145636.1950948-1-tabba@google.com/
> > > [2] https://lore.kernel.org/kvm/20201102033422.657391-1-liangpeng10@huawei.com/
> > >
> > > Reiji Watanabe (29):
> > >   KVM: arm64: Add has_reset_once flag for vcpu
> > >   KVM: arm64: Save ID registers' sanitized value per vCPU
> > >   KVM: arm64: Introduce struct id_reg_info
> > >   KVM: arm64: Make ID_AA64PFR0_EL1 writable
> > >   KVM: arm64: Make ID_AA64PFR1_EL1 writable
> > >   KVM: arm64: Make ID_AA64ISAR0_EL1 writable
> > >   KVM: arm64: Make ID_AA64ISAR1_EL1 writable
> > >   KVM: arm64: Make ID_AA64MMFR0_EL1 writable
> > >   KVM: arm64: Hide IMPLEMENTATION DEFINED PMU support for the guest
> > >   KVM: arm64: Make ID_AA64DFR0_EL1 writable
> > >   KVM: arm64: Make ID_DFR0_EL1 writable
> > >   KVM: arm64: Make ID_DFR1_EL1 writable
> > >   KVM: arm64: Make ID_MMFR0_EL1 writable
> > >   KVM: arm64: Make MVFR1_EL1 writable
> > >   KVM: arm64: Make ID registers without id_reg_info writable
> > >   KVM: arm64: Add consistency checking for frac fields of ID registers
> > >   KVM: arm64: Introduce KVM_CAP_ARM_ID_REG_CONFIGURABLE capability
> > >   KVM: arm64: Add kunit test for ID register validation
> > >   KVM: arm64: Use vcpu->arch cptr_el2 to track value of cptr_el2 for VHE
> > >   KVM: arm64: Use vcpu->arch.mdcr_el2 to track value of mdcr_el2
> > >   KVM: arm64: Introduce framework to trap disabled features
> > >   KVM: arm64: Trap disabled features of ID_AA64PFR0_EL1
> > >   KVM: arm64: Trap disabled features of ID_AA64PFR1_EL1
> > >   KVM: arm64: Trap disabled features of ID_AA64DFR0_EL1
> > >   KVM: arm64: Trap disabled features of ID_AA64MMFR1_EL1
> > >   KVM: arm64: Trap disabled features of ID_AA64ISAR1_EL1
> > >   KVM: arm64: Initialize trapping of disabled CPU features for the guest
> > >   KVM: arm64: Add kunit test for trap initialization
> > >   KVM: arm64: selftests: Introduce id_reg_test
> > >
> > >  Documentation/virt/kvm/api.rst                |    8 +
> > >  arch/arm64/include/asm/cpufeature.h           |    2 +-
> > >  arch/arm64/include/asm/kvm_arm.h              |   32 +
> > >  arch/arm64/include/asm/kvm_host.h             |   15 +
> > >  arch/arm64/include/asm/sysreg.h               |    2 +
> > >  arch/arm64/kvm/Kconfig                        |   11 +
> > >  arch/arm64/kvm/arm.c                          |   12 +-
> > >  arch/arm64/kvm/debug.c                        |   13 +-
> > >  arch/arm64/kvm/hyp/vhe/switch.c               |   14 +-
> > >  arch/arm64/kvm/reset.c                        |    4 +
> > >  arch/arm64/kvm/sys_regs.c                     | 1265 +++++++++++++++--
> > >  arch/arm64/kvm/sys_regs_test.c                | 1109 +++++++++++++++
> > >  include/uapi/linux/kvm.h                      |    1 +
> > >  tools/arch/arm64/include/asm/sysreg.h         |    1 +
> > >  tools/testing/selftests/kvm/.gitignore        |    1 +
> > >  tools/testing/selftests/kvm/Makefile          |    1 +
> > >  .../selftests/kvm/aarch64/id_reg_test.c       | 1128 +++++++++++++++
> > >  17 files changed, 3488 insertions(+), 131 deletions(-)
> > >  create mode 100644 arch/arm64/kvm/sys_regs_test.c
> > >  create mode 100644 tools/testing/selftests/kvm/aarch64/id_reg_test.c
> > >
> > > --
> > > 2.34.0.rc1.387.gb447b232ab-goog
> > >

Re: [RFC PATCH v3 11/29] KVM: arm64: Make ID_DFR0_EL1 writable

[Eric Auger]

Hi Reiji,

On 11/17/21 7:43 AM, Reiji Watanabe wrote:
> This patch adds id_reg_info for ID_DFR0_EL1 to make it writable
> by userspace.
> 
> Return an error if userspace tries to set PerfMon field of the
> register to a value that conflicts with the PMU configuration.
> 
> Signed-off-by: Reiji Watanabe <reijiw@google.com>
> ---
>  arch/arm64/kvm/sys_regs.c | 52 ++++++++++++++++++++++++++++++++++-----
>  1 file changed, 46 insertions(+), 6 deletions(-)
> 
> diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
> index 0faf458b0efb..fbd335ac5e6b 100644
> --- a/arch/arm64/kvm/sys_regs.c
> +++ b/arch/arm64/kvm/sys_regs.c
> @@ -665,6 +665,27 @@ static int validate_id_aa64dfr0_el1(struct kvm_vcpu *vcpu,
>  	return 0;
>  }
>  
> +static int validate_id_dfr0_el1(struct kvm_vcpu *vcpu,
> +				const struct id_reg_info *id_reg, u64 val)
> +{
> +	bool vcpu_pmu, dfr0_pmu;
> +	unsigned int perfmon;
> +
> +	perfmon = cpuid_feature_extract_unsigned_field(val, ID_DFR0_PERFMON_SHIFT);
> +	if (perfmon == 1 || perfmon == 2)
> +		/* PMUv1 or PMUv2 is not allowed on ARMv8. */
> +		return -EINVAL;
> +
> +	vcpu_pmu = kvm_vcpu_has_pmu(vcpu);
> +	dfr0_pmu = id_reg_has_pmu(val, ID_DFR0_PERFMON_SHIFT, ID_DFR0_PERFMON_8_0);
> +
> +	/* Check if there is a conflict with a request via KVM_ARM_VCPU_INIT */
> +	if (vcpu_pmu ^ dfr0_pmu)
> +		return -EPERM;
This breaks the migration on ThunderX v2 as vcpu_pmu == true and
dfr0_pmu == false

Thanks

Eric
> +
> +	return 0;
> +}
> +
>  static void init_id_aa64pfr0_el1_info(struct id_reg_info *id_reg)
>  {
>  	u64 limit = id_reg->vcpu_limit_val;
> @@ -725,6 +746,15 @@ static void init_id_aa64dfr0_el1_info(struct id_reg_info *id_reg)
>  	id_reg->vcpu_limit_val = limit;
>  }
>  
> +static void init_id_dfr0_el1_info(struct id_reg_info *id_reg)
> +{
> +	/* Limit guests to PMUv3 for ARMv8.4 */
> +	id_reg->vcpu_limit_val =
> +		cpuid_feature_cap_perfmon_field(id_reg->vcpu_limit_val,
> +						ID_DFR0_PERFMON_SHIFT,
> +						ID_DFR0_PERFMON_8_4);
> +}
> +
>  static u64 get_reset_id_aa64pfr0_el1(struct kvm_vcpu *vcpu,
>  				     const struct id_reg_info *idr)
>  {
> @@ -762,6 +792,14 @@ static u64 get_reset_id_aa64dfr0_el1(struct kvm_vcpu *vcpu,
>  	       (idr->vcpu_limit_val & ~(ARM64_FEATURE_MASK(ID_AA64DFR0_PMUVER)));
>  }
>  
> +static u64 get_reset_id_dfr0_el1(struct kvm_vcpu *vcpu,
> +				 const struct id_reg_info *idr)
> +{
> +	return kvm_vcpu_has_pmu(vcpu) ?
> +	       idr->vcpu_limit_val :
> +	       (idr->vcpu_limit_val & ~(ARM64_FEATURE_MASK(ID_DFR0_PERFMON)));
> +}
> +
>  static struct id_reg_info id_aa64pfr0_el1_info = {
>  	.sys_reg = SYS_ID_AA64PFR0_EL1,
>  	.ftr_check_types = S_FCT(ID_AA64PFR0_ASIMD_SHIFT, FCT_LOWER_SAFE) |
> @@ -814,6 +852,13 @@ static struct id_reg_info id_aa64dfr0_el1_info = {
>  	.get_reset_val = get_reset_id_aa64dfr0_el1,
>  };
>  
> +static struct id_reg_info id_dfr0_el1_info = {
> +	.sys_reg = SYS_ID_DFR0_EL1,
> +	.init = init_id_dfr0_el1_info,
> +	.validate = validate_id_dfr0_el1,
> +	.get_reset_val = get_reset_id_dfr0_el1,
> +};
> +
>  /*
>   * An ID register that needs special handling to control the value for the
>   * guest must have its own id_reg_info in id_reg_info_table.
> @@ -823,6 +868,7 @@ static struct id_reg_info id_aa64dfr0_el1_info = {
>   */
>  #define	GET_ID_REG_INFO(id)	(id_reg_info_table[IDREG_IDX(id)])
>  static struct id_reg_info *id_reg_info_table[KVM_ARM_ID_REG_MAX_NUM] = {
> +	[IDREG_IDX(SYS_ID_DFR0_EL1)] = &id_dfr0_el1_info,
>  	[IDREG_IDX(SYS_ID_AA64PFR0_EL1)] = &id_aa64pfr0_el1_info,
>  	[IDREG_IDX(SYS_ID_AA64PFR1_EL1)] = &id_aa64pfr1_el1_info,
>  	[IDREG_IDX(SYS_ID_AA64DFR0_EL1)] = &id_aa64dfr0_el1_info,
> @@ -1677,12 +1723,6 @@ static u64 __read_id_reg(const struct kvm_vcpu *vcpu, u32 id)
>  			val |= FIELD_PREP(ARM64_FEATURE_MASK(ID_AA64PFR0_GIC), gic_lim);
>  		}
>  		break;
> -	case SYS_ID_DFR0_EL1:
> -		/* Limit guests to PMUv3 for ARMv8.4 */
> -		val = cpuid_feature_cap_perfmon_field(val,
> -						      ID_DFR0_PERFMON_SHIFT,
> -						      kvm_vcpu_has_pmu(vcpu) ? ID_DFR0_PERFMON_8_4 : 0);
> -		break;
>  	}
>  
>  	return val;
> 

Re: [RFC PATCH v3 00/29] KVM: arm64: Make CPU ID registers writable by userspace

[Reiji Watanabe]

Hi Alex,

> > > The API documentation for KVM_ARM_VCPU_INIT states:
> > >
> > > "Userspace can call this function multiple times for a given vcpu,
> > > including after the vcpu has been run. This will reset the vcpu to its
> > > initial state. All calls to this function after the initial call must use
> > > the same target and same set of feature flags, otherwise EINVAL will be
> > > returned."
> > >
> > > The consequences of that, according to my understanding:
> > >
> > > 1. Any changes to the VCPU features made by KVM are observable by
> > > userspace.
> > >
> > > 2. The features in KVM weren't designed and implemented to be disabled
> > > after being enabled.
> > >
> > > With that in mind, I have two questions:
> > >
> > > 1. What happens when userspace disables a feature via the ID registers
> > > which is set in vcpu->arch.features? Does the feature bit get cleared from
> > > vcpu->arch.features? Does it stay set? If it gets cleared, is it now
> > > possible for userspace to call KVM_ARM_VCPU_INIT again with a different set
> > > of VCPU features (it doesn't look possible to me after looking at the
> > > code). If it stays set, what does it mean when userspace calls
> > > KVM_ARM_VCPU_INIT with a different set of features enabled than what is
> > > present in the ID registers? Should the ID registers be changed to match
> > > the features that userspace set in the last KVM_ARM_VCPU_INIT call (it
> > > looks to me that the ID registers are not changed)?
> >
> > KVM will not allow userspace to set the ID register value that conflicts
> > with CPU features that are configured by the initial KVM_ARM_VCPU_INIT
> > (or there are a few more APIs).
> > KVM_SET_ONE_REG for such requests will fail.
> >
> >
> > > 2. What happens to vcpu->arch.features when userspace enables a feature via
> > > the ID registers which is not present in the bitmap?
> >
> > The answer for this question is basically the same as above.
> > Userspace is not allowed to enable a feature via the ID registers
> > which is not present in the bit map.
> >
> > The cover lever included a brief explanation of this, but I will
> > try to improve the explanation:-)
>
> So the ID registers are used to set the version of a feature which is present in
> the VCPU features bitmap, not to enable or a disable a VCPU feature. Thank you
> for the explanation!

Yes, that is correct for the CPU features that can be enabled by
KVM_ARM_VCPU_INIT (or KVM_ENABLE_CAP).

For the other CPU features that are indicated in ID registers on
the host, userspace can disable (hide) the features (or lower the
level of features) for the guest by updating its ID registers.

Thanks,
Reiji

Re: [RFC PATCH v3 00/29] KVM: arm64: Make CPU ID registers writable by userspace

[Reiji Watanabe]

Hi Alex,

> > > I started reviewing the series, but I ended up being very confused, see
> > > below.
> > >
> > > On Tue, Nov 16, 2021 at 10:43:30PM -0800, Reiji Watanabe wrote:
> > > > In KVM/arm64, values of ID registers for a guest are mostly same as
> > > > its host's values except for bits for feature that KVM doesn't support
> > > > and for opt-in features that userspace didn't configure.  Userspace
> > > > can use KVM_SET_ONE_REG to a set ID register value, but it fails
> > > > if userspace attempts to modify the register value.
> > > >
> > > > This patch series adds support to allow userspace to modify a value of
> > > > ID registers (as long as KVM can support features that are indicated
> > > > in the registers) so userspace can have more control of configuring
> > > > and unconfiguring features for guests.
> > >
> > > What not use VCPU features? Isn't that why the field
> > > kvm_vcpu_init->features exists in the first place? This cover letter does
> > > nothing to explaing why any changes are needed.
> > >
> > > Do you require finer grained control over certain feature that you cannot
> > > get with the 32 * 7 = 224 feature flag bits from kvm_vcpu_init? Does using
> > > the ID registers simplify certain aspects of the implementation?
> >
> > Since some features are not binary in nature (e.g. AA64DFR0_EL1.BRPs
> > fields indicate number of breakpoints minus 1), using
> > kvm_vcpu_init->features to configure such features is inconvenient.
>
> I see, this makes a lot of sense and this looks like a nice solution to
> that problem.
>
> >
> > One of the reasons why we want the finer grained control is that
> > we want to expose a uniform set/level of features for a group of
> > guests on systems with different ARM CPUs.
>
> So here you are talking specifically about KVM not checking that all VCPUs
> have the same feature bits set in vcpu->arch.features, which makes it
> possible for userspace to set different features for different VCPUs,
> right?

Yes, that is correct.  For features that can be configured by
KVM_ARM_VCPU_INIT, userspace can configure different features
for different vCPUs by using KVM_ARM_VCPU_INIT as before.

Thanks,
Reiji

Re: [RFC PATCH v3 02/29] KVM: arm64: Save ID registers' sanitized value per vCPU

[Eric Auger]

Hi Reiji,

On 11/18/21 11:00 PM, Reiji Watanabe wrote:
> Hi Eric,
> 
> On Thu, Nov 18, 2021 at 12:37 PM Eric Auger <eauger@redhat.com> wrote:
>>
>> Hi Reiji,
>>
>> On 11/17/21 7:43 AM, Reiji Watanabe wrote:
>>> Extend sys_regs[] of kvm_cpu_context for ID registers and save ID
>>> registers' sanitized value in the array for the vCPU at the first
>>> vCPU reset. Use the saved ones when ID registers are read by
>>> userspace (via KVM_GET_ONE_REG) or the guest.
>>>
>>> Signed-off-by: Reiji Watanabe <reijiw@google.com>
>>> ---
>>>  arch/arm64/include/asm/kvm_host.h | 10 +++++++
>>>  arch/arm64/kvm/sys_regs.c         | 43 +++++++++++++++++++------------
>>>  2 files changed, 37 insertions(+), 16 deletions(-)
>>>
>>> diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
>>> index edbe2cb21947..72db73c79403 100644
>>> --- a/arch/arm64/include/asm/kvm_host.h
>>> +++ b/arch/arm64/include/asm/kvm_host.h
>>> @@ -146,6 +146,14 @@ struct kvm_vcpu_fault_info {
>>>       u64 disr_el1;           /* Deferred [SError] Status Register */
>>>  };
>>>
>>> +/*
>>> + * (Op0, Op1, CRn, CRm, Op2) of ID registers is (3, 0, 0, crm, op2),
>>> + * where 0<=crm<8, 0<=op2<8.
>>> + */
>>> +#define KVM_ARM_ID_REG_MAX_NUM 64
>>> +#define IDREG_IDX(id)                ((sys_reg_CRm(id) << 3) | sys_reg_Op2(id))
>>> +#define IDREG_SYS_IDX(id)    (ID_REG_BASE + IDREG_IDX(id))
>>> +
>>>  enum vcpu_sysreg {
>>>       __INVALID_SYSREG__,   /* 0 is reserved as an invalid value */
>>>       MPIDR_EL1,      /* MultiProcessor Affinity Register */
>>> @@ -210,6 +218,8 @@ enum vcpu_sysreg {
>>>       CNTP_CVAL_EL0,
>>>       CNTP_CTL_EL0,
>>>
>>> +     ID_REG_BASE,
>>> +     ID_REG_END = ID_REG_BASE + KVM_ARM_ID_REG_MAX_NUM - 1,
>>>       /* Memory Tagging Extension registers */
>>>       RGSR_EL1,       /* Random Allocation Tag Seed Register */
>>>       GCR_EL1,        /* Tag Control Register */
>>> diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
>>> index e3ec1a44f94d..5608d3410660 100644
>>> --- a/arch/arm64/kvm/sys_regs.c
>>> +++ b/arch/arm64/kvm/sys_regs.c
>>> @@ -33,6 +33,8 @@
>>>
>>>  #include "trace.h"
>>>
>>> +static u64 __read_id_reg(const struct kvm_vcpu *vcpu, u32 id);
>>> +
>>>  /*
>>>   * All of this file is extremely similar to the ARM coproc.c, but the
>>>   * types are different. My gut feeling is that it should be pretty
>>> @@ -273,7 +275,7 @@ static bool trap_loregion(struct kvm_vcpu *vcpu,
>>>                         struct sys_reg_params *p,
>>>                         const struct sys_reg_desc *r)
>>>  {
>>> -     u64 val = read_sanitised_ftr_reg(SYS_ID_AA64MMFR1_EL1);
>>> +     u64 val = __read_id_reg(vcpu, SYS_ID_AA64MMFR1_EL1);
>>>       u32 sr = reg_to_encoding(r);
>>>
>>>       if (!(val & (0xfUL << ID_AA64MMFR1_LOR_SHIFT))) {
>>> @@ -1059,17 +1061,9 @@ static bool access_arch_timer(struct kvm_vcpu *vcpu,
>>>       return true;
>>>  }
>>>
>>> -/* Read a sanitised cpufeature ID register by sys_reg_desc */
>>> -static u64 read_id_reg(const struct kvm_vcpu *vcpu,
>>> -             struct sys_reg_desc const *r, bool raz)
>>> +static u64 __read_id_reg(const struct kvm_vcpu *vcpu, u32 id)
>>>  {
>>> -     u32 id = reg_to_encoding(r);
>>> -     u64 val;
>>> -
>>> -     if (raz)
>>> -             return 0;
>>> -
>>> -     val = read_sanitised_ftr_reg(id);
>>> +     u64 val = __vcpu_sys_reg(vcpu, IDREG_SYS_IDX(id));
>>>
>>>       switch (id) {
>>>       case SYS_ID_AA64PFR0_EL1:
>>> @@ -1119,6 +1113,14 @@ static u64 read_id_reg(const struct kvm_vcpu *vcpu,
>>>       return val;
>>>  }
>>>
>>> +static u64 read_id_reg(const struct kvm_vcpu *vcpu,
>>> +                    struct sys_reg_desc const *r, bool raz)
>>> +{
>>> +     u32 id = reg_to_encoding(r);
>>> +
>>> +     return raz ? 0 : __read_id_reg(vcpu, id);
>>> +}
>>> +
>>>  static unsigned int id_visibility(const struct kvm_vcpu *vcpu,
>>>                                 const struct sys_reg_desc *r)
>>>  {
>>> @@ -1178,6 +1180,16 @@ static unsigned int sve_visibility(const struct kvm_vcpu *vcpu,
>>>       return REG_HIDDEN;
>>>  }
>>>
>>> +static void reset_id_reg(struct kvm_vcpu *vcpu, const struct sys_reg_desc *rd)
>>> +{
>>> +     u32 id = reg_to_encoding(rd);
>>> +
>>> +     if (vcpu_has_reset_once(vcpu))
>>> +             return;
>>> +
>>> +     __vcpu_sys_reg(vcpu, IDREG_SYS_IDX(id)) = read_sanitised_ftr_reg(id);
>>> +}
>>> +
>>>  static int set_id_aa64pfr0_el1(struct kvm_vcpu *vcpu,
>>>                              const struct sys_reg_desc *rd,
>>>                              const struct kvm_one_reg *reg, void __user *uaddr)
>>> @@ -1223,9 +1235,7 @@ static int set_id_aa64pfr0_el1(struct kvm_vcpu *vcpu,
>>>  /*
>>>   * cpufeature ID register user accessors
>>>   *
>>> - * For now, these registers are immutable for userspace, so no values
>>> - * are stored, and for set_id_reg() we don't allow the effective value
>>> - * to be changed.
>>> + * We don't allow the effective value to be changed.
>> This change may be moved to a subsequent patch as this patch does not
>> change the behavior yet.
> 
> Thank you for the review.
> 
> There are three main parts in the original comments.
> 
>  (A) these registers are immutable for userspace
>  (B) no values are stored
>  (C) we don't allow the effective value to be changed
> 
> This patch stores ID register values in sys_regs[].
> So, I don't think (B) should be there, and I removed (B).
> Since (A) is essentially the same as (C), I removed (A)
> (and left (C)).
> 
> Do you think it is better to leave (A) in this patch, too ?
yes I think I would leave 'for now,  these registers are immutable for
userspace'

Eric
> 
> Thanks,
> Reiji
> 
> 
>>>   */
>>>  static int __get_id_reg(const struct kvm_vcpu *vcpu,
>>>                       const struct sys_reg_desc *rd, void __user *uaddr,
>>> @@ -1382,6 +1392,7 @@ static unsigned int mte_visibility(const struct kvm_vcpu *vcpu,
>>>  #define ID_SANITISED(name) {                 \
>>>       SYS_DESC(SYS_##name),                   \
>>>       .access = access_id_reg,                \
>>> +     .reset  = reset_id_reg,                 \
>>>       .get_user = get_id_reg,                 \
>>>       .set_user = set_id_reg,                 \
>>>       .visibility = id_visibility,            \
>>> @@ -1837,8 +1848,8 @@ static bool trap_dbgdidr(struct kvm_vcpu *vcpu,
>>>       if (p->is_write) {
>>>               return ignore_write(vcpu, p);
>>>       } else {
>>> -             u64 dfr = read_sanitised_ftr_reg(SYS_ID_AA64DFR0_EL1);
>>> -             u64 pfr = read_sanitised_ftr_reg(SYS_ID_AA64PFR0_EL1);
>>> +             u64 dfr = __read_id_reg(vcpu, SYS_ID_AA64DFR0_EL1);
>>> +             u64 pfr = __read_id_reg(vcpu, SYS_ID_AA64PFR0_EL1);
>>>               u32 el3 = !!cpuid_feature_extract_unsigned_field(pfr, ID_AA64PFR0_EL3_SHIFT);
>>>
>>>               p->regval = ((((dfr >> ID_AA64DFR0_WRPS_SHIFT) & 0xf) << 28) |
>>>
>> Thanks
>>
>> Eric
>>
> 

Re: [RFC PATCH v3 03/29] KVM: arm64: Introduce struct id_reg_info

[Eric Auger]

Hi Reiji,

On 11/19/21 5:47 AM, Reiji Watanabe wrote:
> Hi Eric,
> 
> On Thu, Nov 18, 2021 at 12:36 PM Eric Auger <eauger@redhat.com> wrote:
>>
>> Hi Reiji,
>>
>> On 11/17/21 7:43 AM, Reiji Watanabe wrote:
>>> This patch lays the groundwork to make ID registers writable.
>>>
>>> Introduce struct id_reg_info for an ID register to manage the
>>> register specific control of its value for the guest, and provide set
>>> of functions commonly used for ID registers to make them writable.
>>>
>>> The id_reg_info is used to do register specific initialization,
>>> validation of the ID register and etc.  Not all ID registers must
>>> have the id_reg_info. ID registers that don't have the id_reg_info
>>> are handled in a common way that is applied to all ID registers.
>>>
>>> At present, changing an ID register from userspace is allowed only
>>> if the ID register has the id_reg_info, but that will be changed
>>> by the following patches.
>>>
>>> No ID register has the structure yet and the following patches
>>> will add the id_reg_info for some ID registers.
>>>
>>> Signed-off-by: Reiji Watanabe <reijiw@google.com>
>>> ---
>>>  arch/arm64/include/asm/sysreg.h |   1 +
>>>  arch/arm64/kvm/sys_regs.c       | 226 ++++++++++++++++++++++++++++++--
>>>  2 files changed, 218 insertions(+), 9 deletions(-)
>>>
>>> diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h
>>> index 16b3f1a1d468..597609f26331 100644
>>> --- a/arch/arm64/include/asm/sysreg.h
>>> +++ b/arch/arm64/include/asm/sysreg.h
>>> @@ -1197,6 +1197,7 @@
>>>  #define ICH_VTR_TDS_MASK     (1 << ICH_VTR_TDS_SHIFT)
>>>
>>>  #define ARM64_FEATURE_FIELD_BITS     4
>>> +#define ARM64_FEATURE_FIELD_MASK     ((1ull << ARM64_FEATURE_FIELD_BITS) - 1)
>>>
>>>  /* Create a mask for the feature bits of the specified feature. */
>>>  #define ARM64_FEATURE_MASK(x)        (GENMASK_ULL(x##_SHIFT + ARM64_FEATURE_FIELD_BITS - 1, x##_SHIFT))
>>> diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
>>> index 5608d3410660..1552cd5581b7 100644
>>> --- a/arch/arm64/kvm/sys_regs.c
>>> +++ b/arch/arm64/kvm/sys_regs.c
>>> @@ -265,6 +265,181 @@ static bool trap_raz_wi(struct kvm_vcpu *vcpu,
>>>               return read_zero(vcpu, p);
>>>  }
>>>
>>> +/*
>>> + * A value for FCT_LOWER_SAFE must be zero and changing that will affect
>>> + * ftr_check_types of id_reg_info.
>>> + */
>>> +enum feature_check_type {
>>> +     FCT_LOWER_SAFE = 0,
>>> +     FCT_HIGHER_SAFE,
>>> +     FCT_HIGHER_OR_ZERO_SAFE,
>>> +     FCT_EXACT,
>>> +     FCT_EXACT_OR_ZERO_SAFE,
>>> +     FCT_IGNORE,     /* Don't check (any value is fine) */
>> Maybe you can remove the _SAFE suffix (EXACT does not have it).
> 
> I am inclined to keep 'SAFE' (otherwise, I am likely to forget
> if lower is safe or not).
> 
>> s/EXACT/EQUAL ?
> 
> I will fix that FCT_EXACT to FCT_EQUAL_SAFE.
> 
>>> +};
>>> +
>>> +static int arm64_check_feature_one(enum feature_check_type type, int val,
>>> +                                int limit)
>>> +{
>>> +     bool is_safe = false;
>>> +
>>> +     if (val == limit)
>>> +             return 0;
>> even if the type is unexpected?
> 
> I will remove it.
then you need to modify the handling of FCT_EXACT*.
> 
>>> +
>>> +     switch (type) {
>>> +     case FCT_LOWER_SAFE:
>>> +             is_safe = (val <= limit);
>>> +             break;
>>> +     case FCT_HIGHER_OR_ZERO_SAFE:
>>> +             if (val == 0) {
>>> +                     is_safe = true;
>>> +                     break;
>>> +             }
>>> +             fallthrough;
>>> +     case FCT_HIGHER_SAFE:
>>> +             is_safe = (val >= limit);
>>> +             break;
>>> +     case FCT_EXACT:
>>> +             break;
>>> +     case FCT_EXACT_OR_ZERO_SAFE:
>>> +             is_safe = (val == 0);
>>> +             break;
>>> +     case FCT_IGNORE:
>>> +             is_safe = true;
>>> +             break;
>>> +     default:
>>> +             WARN_ONCE(1, "Unexpected feature_check_type (%d)\n", type);
>>> +             break;
>>> +     }
>>> +
>>> +     return is_safe ? 0 : -1;
>>> +}
>>> +
>>> +#define      FCT_TYPE_MASK           0x7
>>> +#define      FCT_TYPE_SHIFT          1
>>> +#define      FCT_SIGN_MASK           0x1
>>> +#define      FCT_SIGN_SHIFT          0
>>> +#define      FCT_TYPE(val)   ((val >> FCT_TYPE_SHIFT) & FCT_TYPE_MASK)
>>> +#define      FCT_SIGN(val)   ((val >> FCT_SIGN_SHIFT) & FCT_SIGN_MASK)
>>> +
>>> +#define      MAKE_FCT(shift, type, sign)                             \
>>> +     ((u64)((((type) & FCT_TYPE_MASK) << FCT_TYPE_SHIFT) |   \
>>> +            (((sign) & FCT_SIGN_MASK) << FCT_SIGN_SHIFT)) << (shift))
>>> +
>>> +/* For signed field */
>>> +#define      S_FCT(shift, type)      MAKE_FCT(shift, type, 1)
>>> +/* For unigned field */
>>> +#define      U_FCT(shift, type)      MAKE_FCT(shift, type, 0)
>>> +
>>> +/*
>>> + * @val and @lim are both a value of the ID register. The function checks
>>> + * if all features indicated in @val can be supported for guests on the host,
>>> + * which supports features indicated in @lim. @check_types indicates how> + * features in the ID register needs to be checked.
>>> + * See comments for id_reg_info's ftr_check_types field for more detail.
>> What about RES0 fields which may exist? add a comment to reassure about
>> the fact they are properly handled if there are?
> 
> Any fields including RES0 should be checked based on check_types.
> I will explicitly state that in the comment.
> 
>>> + */
>>> +static int arm64_check_features(u64 check_types, u64 val, u64 lim)
>>> +{
>>> +     int i;
>>> +
>>> +     for (i = 0; i < 64; i += ARM64_FEATURE_FIELD_BITS) {
>>> +             u8 ftr_check = (check_types >> i) & ARM64_FEATURE_FIELD_MASK;
>>> +             bool is_sign = FCT_SIGN(ftr_check);
>>> +             enum feature_check_type fctype = FCT_TYPE(ftr_check);
>>> +             int fval, flim, ret;
>>> +
>>> +             fval = cpuid_feature_extract_field(val, i, is_sign);
>>> +             flim = cpuid_feature_extract_field(lim, i, is_sign);
>>> +
>>> +             ret = arm64_check_feature_one(fctype, fval, flim);
>>> +             if (ret)
>>> +                     return -E2BIG;
>>> +     }
>>> +     return 0;
>>> +}
>>> +
>>> +struct id_reg_info {
>>> +     u32     sys_reg;        /* Register ID */
>> use struct kernel-doc comments instead?
>>> +
>>> +     /*
>>> +      * Limit value of the register for a vcpu. The value is the sanitized
>>> +      * system value with bits cleared for unsupported features for the
>>> +      * guest.
>>> +      */
>>> +     u64     vcpu_limit_val;
>>> +
>>> +     /*
>>> +      * The ftr_check_types is comprised of a set of 4 bits fields.
>> nit: s/bits field/bit field here and below
> 
> I will fix them.
> 
>>> +      * Each 4 bits field is for a feature indicated by the same bits
>>> +      * field of the ID register and indicates how the feature support
>>> +      * for guests needs to be checked.
>>> +      * The bit 0 indicates that the corresponding ID register field
>>> +      * is signed(1) or unsigned(0).
>>> +      * The bits [3:1] hold feature_check_type for the field.
>>> +      * If all zero, all features in the ID register are treated as unsigned
>>> +      * fields and checked based on Principles of the ID scheme for fields
>>> +      * in ID registers (FCT_LOWER_SAFE of feature_check_type).
>> values set by the guest are checked against host ID field values
>> according to FCT_LOWER_SAFE test? You do not actually explicitly explain
>> what the check is about although this may be obvious for you?
> 
> How about this ?
> 
>         /*
>          * The ftr_check_types is comprised of a set of 4 bit fields.
>          * Each 4 bit field is for a feature indicated by the same bit field
>          * of the ID register and indicates how the field needs to be checked
>          * (by arm64_check_feature_one) against the host's ID field when
>          * userspace tries to set the register.
>          * The bit 0 indicates that the corresponding ID register field is
>          * signed(1) or unsigned(0). The bits [3:1] hold feature_check_type
>          * for the field (FCT_LOWER_SAFE == 0, etc).
>          * e.g. for ID_AA64PFR0_EL1.SVE(bits [35:32]), bits[35:32] of
>          * ftr_check_types for the register should be 0. It means the SVE
>          * field is treated as an unsigned field, and userspace can set the
>          * field to a equal or lower value than the host's ID field value.
>          */
yep sounds clearer to me.
> 
>>> +      */
>>> +     u64     ftr_check_types;
>>> +
>>> +     /* Initialization function of the id_reg_info */
>>> +     void (*init)(struct id_reg_info *id_reg);
>>> +
>>> +     /* Register specific validation function */
>> validation callback? it does not register anything. We have check
>> customization means already in ftr_check_types so it is difficult to
>> guess at that point why this cb is needed, all the more so it applies
>> after the ftr_checks.
> 
> I am going to add the following comment. Does it look clear enough for you ?
> 
>         /*
>          * This is an optional ID register specific validation function.
>          * When userspace tries to set the ID register, arm64_check_features()
>          * will check if the requested value indicates any features that cannot
>          * be supported by KVM on the host.  But, some ID register fields need
>          * a special checking and this function can be used for such fields.
>          * e.g. KVM_CREATE_DEVICE must be used to configure GICv3 for a guest.
>          * ID_AA64PFR0_EL1.GIC shouldn't be set to 1 unless GICv3 is configured.
>          * The validation function for ID_AA64PFR0_EL1 could be used to check
>          * the field is consistent with GICv3 configuration.
>          */
> 
>>> +     int (*validate)(struct kvm_vcpu *vcpu, const struct id_reg_info *id_reg,
>>> +                     u64 val);
>>> +
>>> +     /* Return the reset value of the register for the vCPU */
>>> +     u64 (*get_reset_val)(struct kvm_vcpu *vcpu,
>>> +                          const struct id_reg_info *id_reg);
>>> +};
>>> +
>>> +static void id_reg_info_init(struct id_reg_info *id_reg)
>>> +{
>>> +     id_reg->vcpu_limit_val = read_sanitised_ftr_reg(id_reg->sys_reg);
>>> +     if (id_reg->init)
>>> +             id_reg->init(id_reg);
>>> +}
>>> +
>>> +/*
>>> + * An ID register that needs special handling to control the value for the
>>> + * guest must have its own id_reg_info in id_reg_info_table.
>>> + * (i.e. the reset value is different from the host's sanitized value,
>>> + * the value is affected by opt-in features, some fields needs specific
>> s/needs/need
> 
> I will fix it.
> 
> Thank you for your review !
> 
> Regards
> Reiji
> 

Thanks

Eric

Re: [RFC PATCH v3 03/29] KVM: arm64: Introduce struct id_reg_info

[Eric Auger]

Hi Reiji,

On 11/17/21 7:43 AM, Reiji Watanabe wrote:
> This patch lays the groundwork to make ID registers writable.
> 
> Introduce struct id_reg_info for an ID register to manage the
> register specific control of its value for the guest, and provide set
> of functions commonly used for ID registers to make them writable.
> 
> The id_reg_info is used to do register specific initialization,
> validation of the ID register and etc.  Not all ID registers must
> have the id_reg_info. ID registers that don't have the id_reg_info
> are handled in a common way that is applied to all ID registers.
> 
> At present, changing an ID register from userspace is allowed only
> if the ID register has the id_reg_info, but that will be changed
> by the following patches.
> 
> No ID register has the structure yet and the following patches
> will add the id_reg_info for some ID registers.
> 
> Signed-off-by: Reiji Watanabe <reijiw@google.com>
> ---
>  arch/arm64/include/asm/sysreg.h |   1 +
>  arch/arm64/kvm/sys_regs.c       | 226 ++++++++++++++++++++++++++++++--
>  2 files changed, 218 insertions(+), 9 deletions(-)
> 
> diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h
> index 16b3f1a1d468..597609f26331 100644
> --- a/arch/arm64/include/asm/sysreg.h
> +++ b/arch/arm64/include/asm/sysreg.h
> @@ -1197,6 +1197,7 @@
>  #define ICH_VTR_TDS_MASK	(1 << ICH_VTR_TDS_SHIFT)
>  
>  #define ARM64_FEATURE_FIELD_BITS	4
> +#define ARM64_FEATURE_FIELD_MASK	((1ull << ARM64_FEATURE_FIELD_BITS) - 1)
>  
>  /* Create a mask for the feature bits of the specified feature. */
>  #define ARM64_FEATURE_MASK(x)	(GENMASK_ULL(x##_SHIFT + ARM64_FEATURE_FIELD_BITS - 1, x##_SHIFT))
> diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
> index 5608d3410660..1552cd5581b7 100644
> --- a/arch/arm64/kvm/sys_regs.c
> +++ b/arch/arm64/kvm/sys_regs.c
> @@ -265,6 +265,181 @@ static bool trap_raz_wi(struct kvm_vcpu *vcpu,
>  		return read_zero(vcpu, p);
>  }
>  
> +/*
> + * A value for FCT_LOWER_SAFE must be zero and changing that will affect
> + * ftr_check_types of id_reg_info.
> + */
> +enum feature_check_type {
> +	FCT_LOWER_SAFE = 0,
> +	FCT_HIGHER_SAFE,
> +	FCT_HIGHER_OR_ZERO_SAFE,
> +	FCT_EXACT,
> +	FCT_EXACT_OR_ZERO_SAFE,
> +	FCT_IGNORE,	/* Don't check (any value is fine) */
> +};
> +
> +static int arm64_check_feature_one(enum feature_check_type type, int val,
> +				   int limit)
> +{
> +	bool is_safe = false;
> +
> +	if (val == limit)
> +		return 0;
> +
> +	switch (type) {
> +	case FCT_LOWER_SAFE:
> +		is_safe = (val <= limit);
> +		break;
> +	case FCT_HIGHER_OR_ZERO_SAFE:
> +		if (val == 0) {
> +			is_safe = true;
> +			break;
> +		}
> +		fallthrough;
> +	case FCT_HIGHER_SAFE:
> +		is_safe = (val >= limit);
> +		break;
> +	case FCT_EXACT:
> +		break;
> +	case FCT_EXACT_OR_ZERO_SAFE:
> +		is_safe = (val == 0);
> +		break;
> +	case FCT_IGNORE:
> +		is_safe = true;
> +		break;
> +	default:
> +		WARN_ONCE(1, "Unexpected feature_check_type (%d)\n", type);
> +		break;
> +	}
> +
> +	return is_safe ? 0 : -1;
> +}
> +
> +#define	FCT_TYPE_MASK		0x7
> +#define	FCT_TYPE_SHIFT		1
> +#define	FCT_SIGN_MASK		0x1
> +#define	FCT_SIGN_SHIFT		0
> +#define	FCT_TYPE(val)	((val >> FCT_TYPE_SHIFT) & FCT_TYPE_MASK)
> +#define	FCT_SIGN(val)	((val >> FCT_SIGN_SHIFT) & FCT_SIGN_MASK)
> +
> +#define	MAKE_FCT(shift, type, sign)				\
> +	((u64)((((type) & FCT_TYPE_MASK) << FCT_TYPE_SHIFT) |	\
> +	       (((sign) & FCT_SIGN_MASK) << FCT_SIGN_SHIFT)) << (shift))
> +
> +/* For signed field */
> +#define	S_FCT(shift, type)	MAKE_FCT(shift, type, 1)
> +/* For unigned field */
> +#define	U_FCT(shift, type)	MAKE_FCT(shift, type, 0)
> +
> +/*
> + * @val and @lim are both a value of the ID register. The function checks
> + * if all features indicated in @val can be supported for guests on the host,
> + * which supports features indicated in @lim. @check_types indicates how
> + * features in the ID register needs to be checked.
> + * See comments for id_reg_info's ftr_check_types field for more detail.
> + */
> +static int arm64_check_features(u64 check_types, u64 val, u64 lim)
> +{
> +	int i;
> +
> +	for (i = 0; i < 64; i += ARM64_FEATURE_FIELD_BITS) {
> +		u8 ftr_check = (check_types >> i) & ARM64_FEATURE_FIELD_MASK;
> +		bool is_sign = FCT_SIGN(ftr_check);
> +		enum feature_check_type fctype = FCT_TYPE(ftr_check);
> +		int fval, flim, ret;
> +
> +		fval = cpuid_feature_extract_field(val, i, is_sign);
> +		flim = cpuid_feature_extract_field(lim, i, is_sign);
> +
> +		ret = arm64_check_feature_one(fctype, fval, flim);
> +		if (ret)
> +			return -E2BIG;
nit: -EINVAL may be better because depending on the check type this may
not mean too big.

Eric
> +	}
> +	return 0;
> +}
> +
> +struct id_reg_info {
> +	u32	sys_reg;	/* Register ID */
> +
> +	/*
> +	 * Limit value of the register for a vcpu. The value is the sanitized
> +	 * system value with bits cleared for unsupported features for the
> +	 * guest.
> +	 */
> +	u64	vcpu_limit_val;
> +
> +	/*
> +	 * The ftr_check_types is comprised of a set of 4 bits fields.
> +	 * Each 4 bits field is for a feature indicated by the same bits
> +	 * field of the ID register and indicates how the feature support
> +	 * for guests needs to be checked.
> +	 * The bit 0 indicates that the corresponding ID register field
> +	 * is signed(1) or unsigned(0).
> +	 * The bits [3:1] hold feature_check_type for the field.
> +	 * If all zero, all features in the ID register are treated as unsigned
> +	 * fields and checked based on Principles of the ID scheme for fields
> +	 * in ID registers (FCT_LOWER_SAFE of feature_check_type).
> +	 */
> +	u64	ftr_check_types;
> +
> +	/* Initialization function of the id_reg_info */
> +	void (*init)(struct id_reg_info *id_reg);
> +
> +	/* Register specific validation function */
> +	int (*validate)(struct kvm_vcpu *vcpu, const struct id_reg_info *id_reg,
> +			u64 val);
> +
> +	/* Return the reset value of the register for the vCPU */
> +	u64 (*get_reset_val)(struct kvm_vcpu *vcpu,
> +			     const struct id_reg_info *id_reg);
> +};
> +
> +static void id_reg_info_init(struct id_reg_info *id_reg)
> +{
> +	id_reg->vcpu_limit_val = read_sanitised_ftr_reg(id_reg->sys_reg);
> +	if (id_reg->init)
> +		id_reg->init(id_reg);
> +}
> +
> +/*
> + * An ID register that needs special handling to control the value for the
> + * guest must have its own id_reg_info in id_reg_info_table.
> + * (i.e. the reset value is different from the host's sanitized value,
> + * the value is affected by opt-in features, some fields needs specific
> + * validation, etc.)
> + */
> +#define	GET_ID_REG_INFO(id)	(id_reg_info_table[IDREG_IDX(id)])
> +static struct id_reg_info *id_reg_info_table[KVM_ARM_ID_REG_MAX_NUM] = {};
> +
> +static int validate_id_reg(struct kvm_vcpu *vcpu,
> +			   const struct sys_reg_desc *rd, u64 val)
> +{
> +	u32 id = reg_to_encoding(rd);
> +	const struct id_reg_info *id_reg = GET_ID_REG_INFO(id);
> +	u64 limit, check_types;
> +	int err;
> +
> +	if (id_reg) {
> +		check_types = id_reg->ftr_check_types;
> +		limit = id_reg->vcpu_limit_val;
> +	} else {
> +		/* All fields are treated as unsigned and FCT_LOWER_SAFE */
> +		check_types = 0;
> +		limit = read_sanitised_ftr_reg(id);
> +	}
> +
> +	/* Check if the value indicates any feature that is not in the limit. */
> +	err = arm64_check_features(check_types, val, limit);
> +	if (err)
> +		return err;
> +
> +	if (id_reg && id_reg->validate)
> +		/* Run the ID register specific validity check. */
> +		err = id_reg->validate(vcpu, id_reg, val);
> +
> +	return err;
> +}
> +
>  /*
>   * ARMv8.1 mandates at least a trivial LORegion implementation, where all the
>   * RW registers are RES0 (which we can implement as RAZ/WI). On an ARMv8.0
> @@ -1183,11 +1358,19 @@ static unsigned int sve_visibility(const struct kvm_vcpu *vcpu,
>  static void reset_id_reg(struct kvm_vcpu *vcpu, const struct sys_reg_desc *rd)
>  {
>  	u32 id = reg_to_encoding(rd);
> +	struct id_reg_info *id_reg;
> +	u64 val;
>  
>  	if (vcpu_has_reset_once(vcpu))
>  		return;
>  
> -	__vcpu_sys_reg(vcpu, IDREG_SYS_IDX(id)) = read_sanitised_ftr_reg(id);
> +	id_reg = GET_ID_REG_INFO(id);
> +	if (id_reg && id_reg->get_reset_val)
> +		val = id_reg->get_reset_val(vcpu, id_reg);
> +	else
> +		val = read_sanitised_ftr_reg(id);
> +
> +	__vcpu_sys_reg(vcpu, IDREG_SYS_IDX(id)) = val;
>  }
>  
>  static int set_id_aa64pfr0_el1(struct kvm_vcpu *vcpu,
> @@ -1232,11 +1415,7 @@ static int set_id_aa64pfr0_el1(struct kvm_vcpu *vcpu,
>  	return 0;
>  }
>  
> -/*
> - * cpufeature ID register user accessors
> - *
> - * We don't allow the effective value to be changed.
> - */
> +/* cpufeature ID register user accessors */
>  static int __get_id_reg(const struct kvm_vcpu *vcpu,
>  			const struct sys_reg_desc *rd, void __user *uaddr,
>  			bool raz)
> @@ -1247,11 +1426,12 @@ static int __get_id_reg(const struct kvm_vcpu *vcpu,
>  	return reg_to_user(uaddr, &val, id);
>  }
>  
> -static int __set_id_reg(const struct kvm_vcpu *vcpu,
> +static int __set_id_reg(struct kvm_vcpu *vcpu,
>  			const struct sys_reg_desc *rd, void __user *uaddr,
>  			bool raz)
>  {
>  	const u64 id = sys_reg_to_index(rd);
> +	u32 encoding = reg_to_encoding(rd);
>  	int err;
>  	u64 val;
>  
> @@ -1259,10 +1439,22 @@ static int __set_id_reg(const struct kvm_vcpu *vcpu,
>  	if (err)
>  		return err;
>  
> -	/* This is what we mean by invariant: you can't change it. */
> -	if (val != read_id_reg(vcpu, rd, raz))
> +	/* Don't allow to change the reg unless the reg has id_reg_info */
> +	if (val != read_id_reg(vcpu, rd, raz) && !GET_ID_REG_INFO(encoding))
>  		return -EINVAL;
>  
> +	if (raz)
> +		return 0;
> +
> +	/* Don't allow to change the reg after the first KVM_RUN. */
> +	if (vcpu->arch.has_run_once)> +		return -EINVAL;
> +
> +	err = validate_id_reg(vcpu, rd, val);
> +	if (err)
> +		return err;
> +
> +	__vcpu_sys_reg(vcpu, IDREG_SYS_IDX(encoding)) = val;
>  	return 0;
>  }
>  
> @@ -2826,6 +3018,20 @@ int kvm_arm_copy_sys_reg_indices(struct kvm_vcpu *vcpu, u64 __user *uindices)
>  	return write_demux_regids(uindices);
>  }
>  
> +static void id_reg_info_init_all(void)
> +{
> +	int i;
> +	struct id_reg_info *id_reg;
> +
> +	for (i = 0; i < ARRAY_SIZE(id_reg_info_table); i++) {
> +		id_reg = (struct id_reg_info *)id_reg_info_table[i];
> +		if (!id_reg)
> +			continue;
> +
> +		id_reg_info_init(id_reg);
> +	}
> +}
> +
>  void kvm_sys_reg_table_init(void)
>  {
>  	unsigned int i;
> @@ -2860,4 +3066,6 @@ void kvm_sys_reg_table_init(void)
>  			break;
>  	/* Clear all higher bits. */
>  	cache_levels &= (1 << (i*3))-1;
> +
> +	id_reg_info_init_all();
>  }
> 

Re: [RFC PATCH v3 03/29] KVM: arm64: Introduce struct id_reg_info

[Reiji Watanabe]

On Sun, Nov 21, 2021 at 4:37 AM Marc Zyngier <maz@kernel.org> wrote:
>
> On Wed, 17 Nov 2021 06:43:33 +0000,
> Reiji Watanabe <reijiw@google.com> wrote:
> >
> > This patch lays the groundwork to make ID registers writable.
> >
> > Introduce struct id_reg_info for an ID register to manage the
> > register specific control of its value for the guest, and provide set
> > of functions commonly used for ID registers to make them writable.
> >
> > The id_reg_info is used to do register specific initialization,
> > validation of the ID register and etc.  Not all ID registers must
> > have the id_reg_info. ID registers that don't have the id_reg_info
> > are handled in a common way that is applied to all ID registers.
> >
> > At present, changing an ID register from userspace is allowed only
> > if the ID register has the id_reg_info, but that will be changed
> > by the following patches.
> >
> > No ID register has the structure yet and the following patches
> > will add the id_reg_info for some ID registers.
> >
> > Signed-off-by: Reiji Watanabe <reijiw@google.com>
> > ---
> >  arch/arm64/include/asm/sysreg.h |   1 +
> >  arch/arm64/kvm/sys_regs.c       | 226 ++++++++++++++++++++++++++++++--
> >  2 files changed, 218 insertions(+), 9 deletions(-)
> >
> > diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h
> > index 16b3f1a1d468..597609f26331 100644
> > --- a/arch/arm64/include/asm/sysreg.h
> > +++ b/arch/arm64/include/asm/sysreg.h
> > @@ -1197,6 +1197,7 @@
> >  #define ICH_VTR_TDS_MASK     (1 << ICH_VTR_TDS_SHIFT)
> >
> >  #define ARM64_FEATURE_FIELD_BITS     4
> > +#define ARM64_FEATURE_FIELD_MASK     ((1ull << ARM64_FEATURE_FIELD_BITS) - 1)
> >
> >  /* Create a mask for the feature bits of the specified feature. */
> >  #define ARM64_FEATURE_MASK(x)        (GENMASK_ULL(x##_SHIFT + ARM64_FEATURE_FIELD_BITS - 1, x##_SHIFT))
> > diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
> > index 5608d3410660..1552cd5581b7 100644
> > --- a/arch/arm64/kvm/sys_regs.c
> > +++ b/arch/arm64/kvm/sys_regs.c
> > @@ -265,6 +265,181 @@ static bool trap_raz_wi(struct kvm_vcpu *vcpu,
> >               return read_zero(vcpu, p);
> >  }
> >
> > +/*
> > + * A value for FCT_LOWER_SAFE must be zero and changing that will affect
> > + * ftr_check_types of id_reg_info.
> > + */
> > +enum feature_check_type {
> > +     FCT_LOWER_SAFE = 0,
> > +     FCT_HIGHER_SAFE,
> > +     FCT_HIGHER_OR_ZERO_SAFE,
> > +     FCT_EXACT,
> > +     FCT_EXACT_OR_ZERO_SAFE,
> > +     FCT_IGNORE,     /* Don't check (any value is fine) */
> > +};
> > +
> > +static int arm64_check_feature_one(enum feature_check_type type, int val,
> > +                                int limit)
> > +{
> > +     bool is_safe = false;
> > +
> > +     if (val == limit)
> > +             return 0;
> > +
> > +     switch (type) {
> > +     case FCT_LOWER_SAFE:
> > +             is_safe = (val <= limit);
> > +             break;
> > +     case FCT_HIGHER_OR_ZERO_SAFE:
> > +             if (val == 0) {
> > +                     is_safe = true;
> > +                     break;
> > +             }
> > +             fallthrough;
> > +     case FCT_HIGHER_SAFE:
> > +             is_safe = (val >= limit);
> > +             break;
> > +     case FCT_EXACT:
> > +             break;
> > +     case FCT_EXACT_OR_ZERO_SAFE:
> > +             is_safe = (val == 0);
> > +             break;
> > +     case FCT_IGNORE:
> > +             is_safe = true;
> > +             break;
> > +     default:
> > +             WARN_ONCE(1, "Unexpected feature_check_type (%d)\n", type);
> > +             break;
> > +     }
> > +
> > +     return is_safe ? 0 : -1;
> > +}
> > +
> > +#define      FCT_TYPE_MASK           0x7
> > +#define      FCT_TYPE_SHIFT          1
> > +#define      FCT_SIGN_MASK           0x1
> > +#define      FCT_SIGN_SHIFT          0
> > +#define      FCT_TYPE(val)   ((val >> FCT_TYPE_SHIFT) & FCT_TYPE_MASK)
> > +#define      FCT_SIGN(val)   ((val >> FCT_SIGN_SHIFT) & FCT_SIGN_MASK)
> > +
> > +#define      MAKE_FCT(shift, type, sign)                             \
> > +     ((u64)((((type) & FCT_TYPE_MASK) << FCT_TYPE_SHIFT) |   \
> > +            (((sign) & FCT_SIGN_MASK) << FCT_SIGN_SHIFT)) << (shift))
> > +
> > +/* For signed field */
> > +#define      S_FCT(shift, type)      MAKE_FCT(shift, type, 1)
> > +/* For unigned field */
> > +#define      U_FCT(shift, type)      MAKE_FCT(shift, type, 0)
> > +
> > +/*
> > + * @val and @lim are both a value of the ID register. The function checks
> > + * if all features indicated in @val can be supported for guests on the host,
> > + * which supports features indicated in @lim. @check_types indicates how
> > + * features in the ID register needs to be checked.
> > + * See comments for id_reg_info's ftr_check_types field for more detail.
> > + */
> > +static int arm64_check_features(u64 check_types, u64 val, u64 lim)
> > +{
> > +     int i;
> > +
> > +     for (i = 0; i < 64; i += ARM64_FEATURE_FIELD_BITS) {
> > +             u8 ftr_check = (check_types >> i) & ARM64_FEATURE_FIELD_MASK;
> > +             bool is_sign = FCT_SIGN(ftr_check);
> > +             enum feature_check_type fctype = FCT_TYPE(ftr_check);
> > +             int fval, flim, ret;
> > +
> > +             fval = cpuid_feature_extract_field(val, i, is_sign);
> > +             flim = cpuid_feature_extract_field(lim, i, is_sign);
> > +
> > +             ret = arm64_check_feature_one(fctype, fval, flim);
> > +             if (ret)
> > +                     return -E2BIG;
> > +     }
> > +     return 0;
> > +}
>
> All this logic seems to reinvent what we already have in
> arch/arm64/kernel/cpufeature.c. I'd rather we rely on it and maintain
> a single idreg handling library.
>
> Could you outline what is missing in the cpufeature code that requires
> you to invent your own? I'm sure Suzuki could help here to make it
> directly usable.

The issue is that there are some fields whose arm64_ftr_bits don't
match what (I think) I need.  However, looking into that option again,
it seems that the number of such fields are fewer than I originally
thought (I misunderstood some earlier).

They are just three fields below.  The common checking process can be
skipped for those fields (will restore ignore_mask field in id_reg_info
as I had in v1 patch, which is treated like FCT_IGNORE in the v3 patch),
and I will have their ID register specific validation function do
what I want to check into the fields.

 - AA64DFR0.DEBUGVER:
   Its .type is FTR_EXACT.
   I want to treat its .type as FTR_LOWER_SAFE for the check.

 - AA64DFR0.PMUVER:
   Its .sign is FTR_SIGNED and .type is FTR_EXACT.
   I want to treat its .sign as FTR_UNSIGNED and .type as
   FTR_LOWER_SAFE for the check.

 - DFR0.PERFMON:
   Its .sign is FTR_SIGNED (Its .type is FTR_LOWER_SAFE).
   I want to treat its .sign field as FTR_UNSIGNED for the check.

   (NOTE: For PMUVER and PERFMON, Arm ARM says "if the field value
    is not 0xf the field is treated as an unsigned value")

Thanks,
Reiji

Re: [RFC PATCH v3 11/29] KVM: arm64: Make ID_DFR0_EL1 writable

[Reiji Watanabe]

Hi Eric,

On Wed, Nov 24, 2021 at 5:46 AM Eric Auger <eauger@redhat.com> wrote:
>
> Hi Reiji,
>
> On 11/17/21 7:43 AM, Reiji Watanabe wrote:
> > This patch adds id_reg_info for ID_DFR0_EL1 to make it writable
> > by userspace.
> >
> > Return an error if userspace tries to set PerfMon field of the
> > register to a value that conflicts with the PMU configuration.
> >
> > Signed-off-by: Reiji Watanabe <reijiw@google.com>
> > ---
> >  arch/arm64/kvm/sys_regs.c | 52 ++++++++++++++++++++++++++++++++++-----
> >  1 file changed, 46 insertions(+), 6 deletions(-)
> >
> > diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
> > index 0faf458b0efb..fbd335ac5e6b 100644
> > --- a/arch/arm64/kvm/sys_regs.c
> > +++ b/arch/arm64/kvm/sys_regs.c
> > @@ -665,6 +665,27 @@ static int validate_id_aa64dfr0_el1(struct kvm_vcpu *vcpu,
> >       return 0;
> >  }
> >
> > +static int validate_id_dfr0_el1(struct kvm_vcpu *vcpu,
> > +                             const struct id_reg_info *id_reg, u64 val)
> > +{
> > +     bool vcpu_pmu, dfr0_pmu;
> > +     unsigned int perfmon;
> > +
> > +     perfmon = cpuid_feature_extract_unsigned_field(val, ID_DFR0_PERFMON_SHIFT);
> > +     if (perfmon == 1 || perfmon == 2)
> > +             /* PMUv1 or PMUv2 is not allowed on ARMv8. */
> > +             return -EINVAL;
> > +
> > +     vcpu_pmu = kvm_vcpu_has_pmu(vcpu);
> > +     dfr0_pmu = id_reg_has_pmu(val, ID_DFR0_PERFMON_SHIFT, ID_DFR0_PERFMON_8_0);
> > +
> > +     /* Check if there is a conflict with a request via KVM_ARM_VCPU_INIT */
> > +     if (vcpu_pmu ^ dfr0_pmu)
> > +             return -EPERM;
> This breaks the migration on ThunderX v2 as vcpu_pmu == true and
> dfr0_pmu == false

Yes, this is the same (incorrect) assumption as the selftest.
I will fix this as well.

Regards,
Reiji

Re: [RFC PATCH v3 03/29] KVM: arm64: Introduce struct id_reg_info

[Reiji Watanabe]

Hi Eric,

On Wed, Nov 24, 2021 at 10:22 AM Eric Auger <eauger@redhat.com> wrote:
>
> Hi Reiji,
>
> On 11/19/21 5:47 AM, Reiji Watanabe wrote:
> > Hi Eric,
> >
> > On Thu, Nov 18, 2021 at 12:36 PM Eric Auger <eauger@redhat.com> wrote:
> >>
> >> Hi Reiji,
> >>
> >> On 11/17/21 7:43 AM, Reiji Watanabe wrote:
> >>> This patch lays the groundwork to make ID registers writable.
> >>>
> >>> Introduce struct id_reg_info for an ID register to manage the
> >>> register specific control of its value for the guest, and provide set
> >>> of functions commonly used for ID registers to make them writable.
> >>>
> >>> The id_reg_info is used to do register specific initialization,
> >>> validation of the ID register and etc.  Not all ID registers must
> >>> have the id_reg_info. ID registers that don't have the id_reg_info
> >>> are handled in a common way that is applied to all ID registers.
> >>>
> >>> At present, changing an ID register from userspace is allowed only
> >>> if the ID register has the id_reg_info, but that will be changed
> >>> by the following patches.
> >>>
> >>> No ID register has the structure yet and the following patches
> >>> will add the id_reg_info for some ID registers.
> >>>
> >>> Signed-off-by: Reiji Watanabe <reijiw@google.com>
> >>> ---
> >>>  arch/arm64/include/asm/sysreg.h |   1 +
> >>>  arch/arm64/kvm/sys_regs.c       | 226 ++++++++++++++++++++++++++++++--
> >>>  2 files changed, 218 insertions(+), 9 deletions(-)
> >>>
> >>> diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h
> >>> index 16b3f1a1d468..597609f26331 100644
> >>> --- a/arch/arm64/include/asm/sysreg.h
> >>> +++ b/arch/arm64/include/asm/sysreg.h
> >>> @@ -1197,6 +1197,7 @@
> >>>  #define ICH_VTR_TDS_MASK     (1 << ICH_VTR_TDS_SHIFT)
> >>>
> >>>  #define ARM64_FEATURE_FIELD_BITS     4
> >>> +#define ARM64_FEATURE_FIELD_MASK     ((1ull << ARM64_FEATURE_FIELD_BITS) - 1)
> >>>
> >>>  /* Create a mask for the feature bits of the specified feature. */
> >>>  #define ARM64_FEATURE_MASK(x)        (GENMASK_ULL(x##_SHIFT + ARM64_FEATURE_FIELD_BITS - 1, x##_SHIFT))
> >>> diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
> >>> index 5608d3410660..1552cd5581b7 100644
> >>> --- a/arch/arm64/kvm/sys_regs.c
> >>> +++ b/arch/arm64/kvm/sys_regs.c
> >>> @@ -265,6 +265,181 @@ static bool trap_raz_wi(struct kvm_vcpu *vcpu,
> >>>               return read_zero(vcpu, p);
> >>>  }
> >>>
> >>> +/*
> >>> + * A value for FCT_LOWER_SAFE must be zero and changing that will affect
> >>> + * ftr_check_types of id_reg_info.
> >>> + */
> >>> +enum feature_check_type {
> >>> +     FCT_LOWER_SAFE = 0,
> >>> +     FCT_HIGHER_SAFE,
> >>> +     FCT_HIGHER_OR_ZERO_SAFE,
> >>> +     FCT_EXACT,
> >>> +     FCT_EXACT_OR_ZERO_SAFE,
> >>> +     FCT_IGNORE,     /* Don't check (any value is fine) */
> >> Maybe you can remove the _SAFE suffix (EXACT does not have it).
> >
> > I am inclined to keep 'SAFE' (otherwise, I am likely to forget
> > if lower is safe or not).
> >
> >> s/EXACT/EQUAL ?
> >
> > I will fix that FCT_EXACT to FCT_EQUAL_SAFE.
> >
> >>> +};
> >>> +
> >>> +static int arm64_check_feature_one(enum feature_check_type type, int val,
> >>> +                                int limit)
> >>> +{
> >>> +     bool is_safe = false;
> >>> +
> >>> +     if (val == limit)
> >>> +             return 0;
> >> even if the type is unexpected?
> >
> > I will remove it.
> then you need to modify the handling of FCT_EXACT*.

Thank you for the comment. Yes, I understand.
That being said, I might probably make "val == limit" safe
unconditionally as I will move those implementations
to arch/arm64/kernel/cpufeature.c and utilize existing things
as much as I can.
(https://lore.kernel.org/linux-arm-kernel/CAAeT=FxwzRF0YZmmoEmq3xRHnhun-BCx_FeEQrOVLgzwseSy4w@mail.gmail.com/)


> >>> +
> >>> +     switch (type) {
> >>> +     case FCT_LOWER_SAFE:
> >>> +             is_safe = (val <= limit);
> >>> +             break;
> >>> +     case FCT_HIGHER_OR_ZERO_SAFE:
> >>> +             if (val == 0) {
> >>> +                     is_safe = true;
> >>> +                     break;
> >>> +             }
> >>> +             fallthrough;
> >>> +     case FCT_HIGHER_SAFE:
> >>> +             is_safe = (val >= limit);
> >>> +             break;
> >>> +     case FCT_EXACT:
> >>> +             break;
> >>> +     case FCT_EXACT_OR_ZERO_SAFE:
> >>> +             is_safe = (val == 0);
> >>> +             break;
> >>> +     case FCT_IGNORE:
> >>> +             is_safe = true;
> >>> +             break;
> >>> +     default:
> >>> +             WARN_ONCE(1, "Unexpected feature_check_type (%d)\n", type);
> >>> +             break;
> >>> +     }
> >>> +
> >>> +     return is_safe ? 0 : -1;
> >>> +}
> >>> +
> >>> +#define      FCT_TYPE_MASK           0x7
> >>> +#define      FCT_TYPE_SHIFT          1
> >>> +#define      FCT_SIGN_MASK           0x1
> >>> +#define      FCT_SIGN_SHIFT          0
> >>> +#define      FCT_TYPE(val)   ((val >> FCT_TYPE_SHIFT) & FCT_TYPE_MASK)
> >>> +#define      FCT_SIGN(val)   ((val >> FCT_SIGN_SHIFT) & FCT_SIGN_MASK)
> >>> +
> >>> +#define      MAKE_FCT(shift, type, sign)                             \
> >>> +     ((u64)((((type) & FCT_TYPE_MASK) << FCT_TYPE_SHIFT) |   \
> >>> +            (((sign) & FCT_SIGN_MASK) << FCT_SIGN_SHIFT)) << (shift))
> >>> +
> >>> +/* For signed field */
> >>> +#define      S_FCT(shift, type)      MAKE_FCT(shift, type, 1)
> >>> +/* For unigned field */
> >>> +#define      U_FCT(shift, type)      MAKE_FCT(shift, type, 0)
> >>> +
> >>> +/*
> >>> + * @val and @lim are both a value of the ID register. The function checks
> >>> + * if all features indicated in @val can be supported for guests on the host,
> >>> + * which supports features indicated in @lim. @check_types indicates how> + * features in the ID register needs to be checked.
> >>> + * See comments for id_reg_info's ftr_check_types field for more detail.
> >> What about RES0 fields which may exist? add a comment to reassure about
> >> the fact they are properly handled if there are?
> >
> > Any fields including RES0 should be checked based on check_types.
> > I will explicitly state that in the comment.
> >
> >>> + */
> >>> +static int arm64_check_features(u64 check_types, u64 val, u64 lim)
> >>> +{
> >>> +     int i;
> >>> +
> >>> +     for (i = 0; i < 64; i += ARM64_FEATURE_FIELD_BITS) {
> >>> +             u8 ftr_check = (check_types >> i) & ARM64_FEATURE_FIELD_MASK;
> >>> +             bool is_sign = FCT_SIGN(ftr_check);
> >>> +             enum feature_check_type fctype = FCT_TYPE(ftr_check);
> >>> +             int fval, flim, ret;
> >>> +
> >>> +             fval = cpuid_feature_extract_field(val, i, is_sign);
> >>> +             flim = cpuid_feature_extract_field(lim, i, is_sign);
> >>> +
> >>> +             ret = arm64_check_feature_one(fctype, fval, flim);
> >>> +             if (ret)
> >>> +                     return -E2BIG;
> >>> +     }
> >>> +     return 0;
> >>> +}
> >>> +
> >>> +struct id_reg_info {
> >>> +     u32     sys_reg;        /* Register ID */
> >> use struct kernel-doc comments instead?
> >>> +
> >>> +     /*
> >>> +      * Limit value of the register for a vcpu. The value is the sanitized
> >>> +      * system value with bits cleared for unsupported features for the
> >>> +      * guest.
> >>> +      */
> >>> +     u64     vcpu_limit_val;
> >>> +
> >>> +     /*
> >>> +      * The ftr_check_types is comprised of a set of 4 bits fields.
> >> nit: s/bits field/bit field here and below
> >
> > I will fix them.
> >
> >>> +      * Each 4 bits field is for a feature indicated by the same bits
> >>> +      * field of the ID register and indicates how the feature support
> >>> +      * for guests needs to be checked.
> >>> +      * The bit 0 indicates that the corresponding ID register field
> >>> +      * is signed(1) or unsigned(0).
> >>> +      * The bits [3:1] hold feature_check_type for the field.
> >>> +      * If all zero, all features in the ID register are treated as unsigned
> >>> +      * fields and checked based on Principles of the ID scheme for fields
> >>> +      * in ID registers (FCT_LOWER_SAFE of feature_check_type).
> >> values set by the guest are checked against host ID field values
> >> according to FCT_LOWER_SAFE test? You do not actually explicitly explain
> >> what the check is about although this may be obvious for you?
> >
> > How about this ?
> >
> >         /*
> >          * The ftr_check_types is comprised of a set of 4 bit fields.
> >          * Each 4 bit field is for a feature indicated by the same bit field
> >          * of the ID register and indicates how the field needs to be checked
> >          * (by arm64_check_feature_one) against the host's ID field when
> >          * userspace tries to set the register.
> >          * The bit 0 indicates that the corresponding ID register field is
> >          * signed(1) or unsigned(0). The bits [3:1] hold feature_check_type
> >          * for the field (FCT_LOWER_SAFE == 0, etc).
> >          * e.g. for ID_AA64PFR0_EL1.SVE(bits [35:32]), bits[35:32] of
> >          * ftr_check_types for the register should be 0. It means the SVE
> >          * field is treated as an unsigned field, and userspace can set the
> >          * field to a equal or lower value than the host's ID field value.
> >          */
> yep sounds clearer to me.

Thank you for the review !
(ftr_check_types will be gone due to the same reason above though...)

Regards,
Reiji

Re: [RFC PATCH v3 03/29] KVM: arm64: Introduce struct id_reg_info

[Reiji Watanabe]

Hi Eric,

On Wed, Nov 24, 2021 at 1:07 PM Eric Auger <eauger@redhat.com> wrote:
>
> Hi Reiji,
>
> On 11/17/21 7:43 AM, Reiji Watanabe wrote:
> > This patch lays the groundwork to make ID registers writable.
> >
> > Introduce struct id_reg_info for an ID register to manage the
> > register specific control of its value for the guest, and provide set
> > of functions commonly used for ID registers to make them writable.
> >
> > The id_reg_info is used to do register specific initialization,
> > validation of the ID register and etc.  Not all ID registers must
> > have the id_reg_info. ID registers that don't have the id_reg_info
> > are handled in a common way that is applied to all ID registers.
> >
> > At present, changing an ID register from userspace is allowed only
> > if the ID register has the id_reg_info, but that will be changed
> > by the following patches.
> >
> > No ID register has the structure yet and the following patches
> > will add the id_reg_info for some ID registers.
> >
> > Signed-off-by: Reiji Watanabe <reijiw@google.com>
> > ---
> >  arch/arm64/include/asm/sysreg.h |   1 +
> >  arch/arm64/kvm/sys_regs.c       | 226 ++++++++++++++++++++++++++++++--
> >  2 files changed, 218 insertions(+), 9 deletions(-)
> >
> > diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h
> > index 16b3f1a1d468..597609f26331 100644
> > --- a/arch/arm64/include/asm/sysreg.h
> > +++ b/arch/arm64/include/asm/sysreg.h
> > @@ -1197,6 +1197,7 @@
> >  #define ICH_VTR_TDS_MASK     (1 << ICH_VTR_TDS_SHIFT)
> >
> >  #define ARM64_FEATURE_FIELD_BITS     4
> > +#define ARM64_FEATURE_FIELD_MASK     ((1ull << ARM64_FEATURE_FIELD_BITS) - 1)
> >
> >  /* Create a mask for the feature bits of the specified feature. */
> >  #define ARM64_FEATURE_MASK(x)        (GENMASK_ULL(x##_SHIFT + ARM64_FEATURE_FIELD_BITS - 1, x##_SHIFT))
> > diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
> > index 5608d3410660..1552cd5581b7 100644
> > --- a/arch/arm64/kvm/sys_regs.c
> > +++ b/arch/arm64/kvm/sys_regs.c
> > @@ -265,6 +265,181 @@ static bool trap_raz_wi(struct kvm_vcpu *vcpu,
> >               return read_zero(vcpu, p);
> >  }
> >
> > +/*
> > + * A value for FCT_LOWER_SAFE must be zero and changing that will affect
> > + * ftr_check_types of id_reg_info.
> > + */
> > +enum feature_check_type {
> > +     FCT_LOWER_SAFE = 0,
> > +     FCT_HIGHER_SAFE,
> > +     FCT_HIGHER_OR_ZERO_SAFE,
> > +     FCT_EXACT,
> > +     FCT_EXACT_OR_ZERO_SAFE,
> > +     FCT_IGNORE,     /* Don't check (any value is fine) */
> > +};
> > +
> > +static int arm64_check_feature_one(enum feature_check_type type, int val,
> > +                                int limit)
> > +{
> > +     bool is_safe = false;
> > +
> > +     if (val == limit)
> > +             return 0;
> > +
> > +     switch (type) {
> > +     case FCT_LOWER_SAFE:
> > +             is_safe = (val <= limit);
> > +             break;
> > +     case FCT_HIGHER_OR_ZERO_SAFE:
> > +             if (val == 0) {
> > +                     is_safe = true;
> > +                     break;
> > +             }
> > +             fallthrough;
> > +     case FCT_HIGHER_SAFE:
> > +             is_safe = (val >= limit);
> > +             break;
> > +     case FCT_EXACT:
> > +             break;
> > +     case FCT_EXACT_OR_ZERO_SAFE:
> > +             is_safe = (val == 0);
> > +             break;
> > +     case FCT_IGNORE:
> > +             is_safe = true;
> > +             break;
> > +     default:
> > +             WARN_ONCE(1, "Unexpected feature_check_type (%d)\n", type);
> > +             break;
> > +     }
> > +
> > +     return is_safe ? 0 : -1;
> > +}
> > +
> > +#define      FCT_TYPE_MASK           0x7
> > +#define      FCT_TYPE_SHIFT          1
> > +#define      FCT_SIGN_MASK           0x1
> > +#define      FCT_SIGN_SHIFT          0
> > +#define      FCT_TYPE(val)   ((val >> FCT_TYPE_SHIFT) & FCT_TYPE_MASK)
> > +#define      FCT_SIGN(val)   ((val >> FCT_SIGN_SHIFT) & FCT_SIGN_MASK)
> > +
> > +#define      MAKE_FCT(shift, type, sign)                             \
> > +     ((u64)((((type) & FCT_TYPE_MASK) << FCT_TYPE_SHIFT) |   \
> > +            (((sign) & FCT_SIGN_MASK) << FCT_SIGN_SHIFT)) << (shift))
> > +
> > +/* For signed field */
> > +#define      S_FCT(shift, type)      MAKE_FCT(shift, type, 1)
> > +/* For unigned field */
> > +#define      U_FCT(shift, type)      MAKE_FCT(shift, type, 0)
> > +
> > +/*
> > + * @val and @lim are both a value of the ID register. The function checks
> > + * if all features indicated in @val can be supported for guests on the host,
> > + * which supports features indicated in @lim. @check_types indicates how
> > + * features in the ID register needs to be checked.
> > + * See comments for id_reg_info's ftr_check_types field for more detail.
> > + */
> > +static int arm64_check_features(u64 check_types, u64 val, u64 lim)
> > +{
> > +     int i;
> > +
> > +     for (i = 0; i < 64; i += ARM64_FEATURE_FIELD_BITS) {
> > +             u8 ftr_check = (check_types >> i) & ARM64_FEATURE_FIELD_MASK;
> > +             bool is_sign = FCT_SIGN(ftr_check);
> > +             enum feature_check_type fctype = FCT_TYPE(ftr_check);
> > +             int fval, flim, ret;
> > +
> > +             fval = cpuid_feature_extract_field(val, i, is_sign);
> > +             flim = cpuid_feature_extract_field(lim, i, is_sign);
> > +
> > +             ret = arm64_check_feature_one(fctype, fval, flim);
> > +             if (ret)
> > +                     return -E2BIG;
> nit: -EINVAL may be better because depending on the check type this may
> not mean too big.

Yes, that is correct.

This error case means that userspace tried to configure features
or a higher level of features that were not supported on the host.
In that sense, I chose -E2BIG.

I wanted to use an error code specific to this particular case, which
I think makes debugging userspace issue easier when KVM_SET_ONE_REG
fails, and I couldn't find other error codes that fit this case better.
So, I'm trying to avoid using -EINVAL, which is used for other failure
cases.

If you have any other suggested error code for this,
that would be very helpful:)

Thanks,
Reiji

Re: [RFC PATCH v3 08/29] KVM: arm64: Make ID_AA64MMFR0_EL1 writable

[Eric Auger]

On 11/17/21 7:43 AM, Reiji Watanabe wrote:
> This patch adds id_reg_info for ID_AA64MMFR0_EL1 to make it
> writable by userspace.
> 
> Since ID_AA64MMFR0_EL1 stage 2 granule size fields don't follow the
> standard ID scheme, we need a special handling to validate those fields.
> 
> Signed-off-by: Reiji Watanabe <reijiw@google.com>
> ---
>  arch/arm64/kvm/sys_regs.c | 118 ++++++++++++++++++++++++++++++++++++++
>  1 file changed, 118 insertions(+)
> 
> diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
> index 5812e39602fe..772e3d3067b2 100644
> --- a/arch/arm64/kvm/sys_regs.c
> +++ b/arch/arm64/kvm/sys_regs.c
> @@ -519,6 +519,113 @@ static int validate_id_aa64isar1_el1(struct kvm_vcpu *vcpu,
>  	return 0;
>  }
>  
> +/*
> + * Check if the requested stage2 translation granule size indicated in
> + * @mmfr0 is also indicated in @mmfr0_lim.  This function assumes that
> + * the stage1 granule size indicated in @mmfr0 has been validated already.
I would suggest: relies on the fact TGranX fields are validated before
through the arm64_check_features lookup
> + */
> +static int aa64mmfr0_tgran2_check(int field, u64 mmfr0, u64 mmfr0_lim)
> +{
> +	s64 tgran2, lim_tgran2, rtgran1;
> +	int f1;
> +	bool is_signed = true;
> +
> +	tgran2 = cpuid_feature_extract_unsigned_field(mmfr0, field);
> +	lim_tgran2 = cpuid_feature_extract_unsigned_field(mmfr0_lim, field);
> +	if (tgran2 == lim_tgran2)
> +		return 0;
> +
> +	if (tgran2 && lim_tgran2)
> +		return (tgran2 > lim_tgran2) ? -E2BIG : 0;
> +
> +	/*
> +	 * Either tgran2 or lim_tgran2 is zero.
> +	 * Need stage1 granule size to validate tgran2.
> +	 */
> +	switch (field) {
> +	case ID_AA64MMFR0_TGRAN4_2_SHIFT:
> +		f1 = ID_AA64MMFR0_TGRAN4_SHIFT;
> +		break;
> +	case ID_AA64MMFR0_TGRAN64_2_SHIFT:
> +		f1 = ID_AA64MMFR0_TGRAN64_SHIFT;
> +		break;
> +	case ID_AA64MMFR0_TGRAN16_2_SHIFT:
> +		f1 = ID_AA64MMFR0_TGRAN16_SHIFT;
> +		is_signed = false;
I don't get the is_signed setting. Don't the TGRAN_x have the same
format? Beside you can get the shift by substracting 12 to @field.

can't you directly compute if the granule is supported

> +		break;
> +	default:
> +		/* Should never happen */
> +		WARN_ONCE(1, "Unexpected stage2 granule field (%d)\n", field);
> +		return 0;
> +	}
> +
> +	/*
> +	 * If tgran2 == 0 (&& lim_tgran2 != 0), the requested stage2 granule
> +	 * size is indicated in the stage1 granule size field of @mmfr0.
> +	 * So, validate the stage1 granule size against the stage2 limit
> +	 * granule size.
> +	 * If lim_tgran2 == 0 (&& tgran2 != 0), the stage2 limit granule size
> +	 * is indicated in the stage1 granule size field of @mmfr0_lim.
> +	 * So, validate the requested stage2 granule size against the stage1
> +	 * limit granule size.
> +	 */
> +
> +	 /* Get the relevant stage1 granule size to validate tgran2 */
> +	if (tgran2 == 0)
> +		/* The requested stage1 granule size */
> +		rtgran1 = cpuid_feature_extract_field(mmfr0, f1, is_signed);
> +	else /* lim_tgran2 == 0 */
> +		/* The stage1 limit granule size */
> +		rtgran1 = cpuid_feature_extract_field(mmfr0_lim, f1, is_signed);
> +
> +	/*
> +	 * Adjust the value of rtgran1 to compare with stage2 granule size,
> +	 * which indicates: 1: Not supported, 2: Supported, etc.
> +	 */
> +	if (is_signed)
> +		/* For signed, -1: Not supported, 0: Supported, etc. */
> +		rtgran1 += 0x2;
> +	else
> +		/* For unsigned, 0: Not supported, 1: Supported, etc. */
> +		rtgran1 += 0x1;
> +
> +	if ((tgran2 == 0) && (rtgran1 > lim_tgran2))
> +		/*
> +		 * The requested stage1 granule size (== the requested stage2
> +		 * granule size) is larger than the stage2 limit granule size.
> +		 */
> +		return -E2BIG;
> +	else if ((lim_tgran2 == 0) && (tgran2 > rtgran1))
> +		/*
> +		 * The requested stage2 granule size is larger than the stage1
> +		 * limit granulze size (== the stage2 limit granule size).
> +		 */
> +		return -E2BIG;
> +
> +	return 0;
> +}
> +
> +static int validate_id_aa64mmfr0_el1(struct kvm_vcpu *vcpu,
> +				     const struct id_reg_info *id_reg, u64 val)
> +{
> +	u64 limit = id_reg->vcpu_limit_val;
> +	int ret;

shouldn't you forbid reserved values for TGran4, 64?
> +
> +	ret = aa64mmfr0_tgran2_check(ID_AA64MMFR0_TGRAN4_2_SHIFT, val, limit);
> +	if (ret)
> +		return ret;
> +
> +	ret = aa64mmfr0_tgran2_check(ID_AA64MMFR0_TGRAN64_2_SHIFT, val, limit);
> +	if (ret)
> +		return ret;
> +
> +	ret = aa64mmfr0_tgran2_check(ID_AA64MMFR0_TGRAN16_2_SHIFT, val, limit);
> +	if (ret)
> +		return ret;
> +
> +	return 0;
> +}
> +
>  static void init_id_aa64pfr0_el1_info(struct id_reg_info *id_reg)
>  {
>  	u64 limit = id_reg->vcpu_limit_val;
> @@ -625,6 +732,16 @@ static struct id_reg_info id_aa64isar1_el1_info = {
>  	.get_reset_val = get_reset_id_aa64isar1_el1,
>  };
>  
> +static struct id_reg_info id_aa64mmfr0_el1_info = {
> +	.sys_reg = SYS_ID_AA64MMFR0_EL1,
> +	.ftr_check_types = S_FCT(ID_AA64MMFR0_TGRAN4_SHIFT, FCT_LOWER_SAFE) |
> +			   S_FCT(ID_AA64MMFR0_TGRAN64_SHIFT, FCT_LOWER_SAFE) |
the default?
> +			   U_FCT(ID_AA64MMFR0_TGRAN4_2_SHIFT, FCT_IGNORE) |
> +			   U_FCT(ID_AA64MMFR0_TGRAN64_2_SHIFT, FCT_IGNORE) |
> +			   U_FCT(ID_AA64MMFR0_TGRAN16_2_SHIFT, FCT_IGNORE),
maybe add comment telling the actual check is handled in the validate cb
> +	.validate = validate_id_aa64mmfr0_el1,
> +};
> +
>  /*
>   * An ID register that needs special handling to control the value for the
>   * guest must have its own id_reg_info in id_reg_info_table.
> @@ -638,6 +755,7 @@ static struct id_reg_info *id_reg_info_table[KVM_ARM_ID_REG_MAX_NUM] = {
>  	[IDREG_IDX(SYS_ID_AA64PFR1_EL1)] = &id_aa64pfr1_el1_info,
>  	[IDREG_IDX(SYS_ID_AA64ISAR0_EL1)] = &id_aa64isar0_el1_info,
>  	[IDREG_IDX(SYS_ID_AA64ISAR1_EL1)] = &id_aa64isar1_el1_info,
> +	[IDREG_IDX(SYS_ID_AA64MMFR0_EL1)] = &id_aa64mmfr0_el1_info,
>  };
>  
>  static int validate_id_reg(struct kvm_vcpu *vcpu,
> 

Re: [RFC PATCH v3 04/29] KVM: arm64: Make ID_AA64PFR0_EL1 writable

[Eric Auger]

Hi Reiji,

On 11/17/21 7:43 AM, Reiji Watanabe wrote:
> This patch adds id_reg_info for ID_AA64PFR0_EL1 to make it writable by
> userspace.
> 
> The CSV2/CSV3 fields of the register were already writable and values
> that were written for them affected all vCPUs before. Now they only
> affect the vCPU.
> Return an error if userspace tries to set SVE/GIC field of the register
> to a value that conflicts with SVE/GIC configuration for the guest.
> SIMD/FP/SVE fields of the requested value are validated according to
> Arm ARM.
> 
> Signed-off-by: Reiji Watanabe <reijiw@google.com>
> ---
>  arch/arm64/kvm/sys_regs.c | 159 ++++++++++++++++++++++++--------------
>  1 file changed, 103 insertions(+), 56 deletions(-)
> 
> diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
> index 1552cd5581b7..35400869067a 100644
> --- a/arch/arm64/kvm/sys_regs.c
> +++ b/arch/arm64/kvm/sys_regs.c
> @@ -401,6 +401,92 @@ static void id_reg_info_init(struct id_reg_info *id_reg)
>  		id_reg->init(id_reg);
>  }
>  
> +#define	kvm_has_gic3(kvm)		\
> +	(irqchip_in_kernel(kvm) &&	\
> +	 (kvm)->arch.vgic.vgic_model == KVM_DEV_TYPE_ARM_VGIC_V3)
you may move this macro to kvm/arm_vgic.h as this may be used in
vgic/vgic-v3.c too
> +
> +static int validate_id_aa64pfr0_el1(struct kvm_vcpu *vcpu,
> +				    const struct id_reg_info *id_reg, u64 val)
> +{
> +	int fp, simd;
> +	bool vcpu_has_sve = vcpu_has_sve(vcpu);
> +	bool pfr0_has_sve = id_aa64pfr0_sve(val);
> +	int gic;
> +
> +	simd = cpuid_feature_extract_signed_field(val, ID_AA64PFR0_ASIMD_SHIFT);
> +	fp = cpuid_feature_extract_signed_field(val, ID_AA64PFR0_FP_SHIFT);
> +	if (simd != fp)
> +		return -EINVAL;
> +
> +	/* fp must be supported when sve is supported */
> +	if (pfr0_has_sve && (fp < 0))
> +		return -EINVAL;
> +
> +	/* Check if there is a conflict with a request via KVM_ARM_VCPU_INIT */
> +	if (vcpu_has_sve ^ pfr0_has_sve)
> +		return -EPERM;
> +
> +	gic = cpuid_feature_extract_unsigned_field(val, ID_AA64PFR0_GIC_SHIFT);
> +	if ((gic > 0) ^ kvm_has_gic3(vcpu->kvm))
> +		return -EPERM;

Sometimes from a given architecture version, some lower values are not
allowed. For instance from ARMv8.5 onlt 1 is permitted for CSV3.
Shouldn't we handle that kind of check?
> +
> +	return 0;
> +}
> +
> +static void init_id_aa64pfr0_el1_info(struct id_reg_info *id_reg)
> +{
> +	u64 limit = id_reg->vcpu_limit_val;
> +	unsigned int gic;
> +
> +	limit &= ~ARM64_FEATURE_MASK(ID_AA64PFR0_AMU);
> +	if (!system_supports_sve())
> +		limit &= ~ARM64_FEATURE_MASK(ID_AA64PFR0_SVE);
> +
> +	/*
> +	 * The default is to expose CSV2 == 1 and CSV3 == 1 if the HW
> +	 * isn't affected.  Userspace can override this as long as it
> +	 * doesn't promise the impossible.
> +	 */
> +	limit &= ~(ARM64_FEATURE_MASK(ID_AA64PFR0_CSV2) |
> +		   ARM64_FEATURE_MASK(ID_AA64PFR0_CSV3));
> +
> +	if (arm64_get_spectre_v2_state() == SPECTRE_UNAFFECTED)
> +		limit |= FIELD_PREP(ARM64_FEATURE_MASK(ID_AA64PFR0_CSV2), 1);
> +	if (arm64_get_meltdown_state() == SPECTRE_UNAFFECTED)
> +		limit |= FIELD_PREP(ARM64_FEATURE_MASK(ID_AA64PFR0_CSV3), 1);
> +
> +	gic = cpuid_feature_extract_unsigned_field(limit, ID_AA64PFR0_GIC_SHIFT);
> +	if (gic > 1) {
> +		/* Limit to GICv3.0/4.0 */
> +		limit &= ~ARM64_FEATURE_MASK(ID_AA64PFR0_GIC);
> +		limit |= FIELD_PREP(ARM64_FEATURE_MASK(ID_AA64PFR0_GIC), 1);
> +	}
> +	id_reg->vcpu_limit_val = limit;
> +}
> +
> +static u64 get_reset_id_aa64pfr0_el1(struct kvm_vcpu *vcpu,
> +				     const struct id_reg_info *idr)
> +{
> +	u64 val = idr->vcpu_limit_val;
> +
> +	if (!vcpu_has_sve(vcpu))
> +		val &= ~ARM64_FEATURE_MASK(ID_AA64PFR0_SVE);
> +
> +	if (!kvm_has_gic3(vcpu->kvm))
> +		val &= ~ARM64_FEATURE_MASK(ID_AA64PFR0_GIC);
> +
> +	return val;
> +}
> +
> +static struct id_reg_info id_aa64pfr0_el1_info = {
> +	.sys_reg = SYS_ID_AA64PFR0_EL1,
> +	.ftr_check_types = S_FCT(ID_AA64PFR0_ASIMD_SHIFT, FCT_LOWER_SAFE) |
> +			   S_FCT(ID_AA64PFR0_FP_SHIFT, FCT_LOWER_SAFE),
is it needed as it is the default?
> +	.init = init_id_aa64pfr0_el1_info,
> +	.validate = validate_id_aa64pfr0_el1,
> +	.get_reset_val = get_reset_id_aa64pfr0_el1,
> +};
> +
>  /*
>   * An ID register that needs special handling to control the value for the
>   * guest must have its own id_reg_info in id_reg_info_table.
> @@ -409,7 +495,9 @@ static void id_reg_info_init(struct id_reg_info *id_reg)
>   * validation, etc.)
>   */
>  #define	GET_ID_REG_INFO(id)	(id_reg_info_table[IDREG_IDX(id)])
> -static struct id_reg_info *id_reg_info_table[KVM_ARM_ID_REG_MAX_NUM] = {};
> +static struct id_reg_info *id_reg_info_table[KVM_ARM_ID_REG_MAX_NUM] = {
> +	[IDREG_IDX(SYS_ID_AA64PFR0_EL1)] = &id_aa64pfr0_el1_info,
> +};
>  
>  static int validate_id_reg(struct kvm_vcpu *vcpu,
>  			   const struct sys_reg_desc *rd, u64 val)
> @@ -1239,20 +1327,22 @@ static bool access_arch_timer(struct kvm_vcpu *vcpu,
>  static u64 __read_id_reg(const struct kvm_vcpu *vcpu, u32 id)
>  {
>  	u64 val = __vcpu_sys_reg(vcpu, IDREG_SYS_IDX(id));
> +	u64 lim, gic, gic_lim;
> +	const struct id_reg_info *id_reg;
>  
>  	switch (id) {
>  	case SYS_ID_AA64PFR0_EL1:
> -		if (!vcpu_has_sve(vcpu))
> -			val &= ~ARM64_FEATURE_MASK(ID_AA64PFR0_SVE);
> -		val &= ~ARM64_FEATURE_MASK(ID_AA64PFR0_AMU);
> -		val &= ~ARM64_FEATURE_MASK(ID_AA64PFR0_CSV2);
> -		val |= FIELD_PREP(ARM64_FEATURE_MASK(ID_AA64PFR0_CSV2), (u64)vcpu->kvm->arch.pfr0_csv2);
> -		val &= ~ARM64_FEATURE_MASK(ID_AA64PFR0_CSV3);
> -		val |= FIELD_PREP(ARM64_FEATURE_MASK(ID_AA64PFR0_CSV3), (u64)vcpu->kvm->arch.pfr0_csv3);
> -		if (irqchip_in_kernel(vcpu->kvm) &&
> -		    vcpu->kvm->arch.vgic.vgic_model == KVM_DEV_TYPE_ARM_VGIC_V3) {
> -			val &= ~ARM64_FEATURE_MASK(ID_AA64PFR0_GIC);
> -			val |= FIELD_PREP(ARM64_FEATURE_MASK(ID_AA64PFR0_GIC), 1);
> +		gic = cpuid_feature_extract_unsigned_field(val, ID_AA64PFR0_GIC_SHIFT);
> +		if (kvm_has_gic3(vcpu->kvm) && (gic == 0)) {
> +			/*
> +			 * This is a case where userspace configured gic3 after
> +			 * the vcpu was created, and then it didn't set
> +			 * ID_AA64PFR0_EL1.
> +			 */
> +			id_reg = GET_ID_REG_INFO(id);
> +			lim = id_reg->vcpu_limit_val;
> +			gic_lim = cpuid_feature_extract_unsigned_field(lim, ID_AA64PFR0_GIC_SHIFT);
> +			val |= FIELD_PREP(ARM64_FEATURE_MASK(ID_AA64PFR0_GIC), gic_lim);
>  		}
>  		break;
>  	case SYS_ID_AA64PFR1_EL1:
> @@ -1373,48 +1463,6 @@ static void reset_id_reg(struct kvm_vcpu *vcpu, const struct sys_reg_desc *rd)
>  	__vcpu_sys_reg(vcpu, IDREG_SYS_IDX(id)) = val;
>  }
>  
> -static int set_id_aa64pfr0_el1(struct kvm_vcpu *vcpu,
> -			       const struct sys_reg_desc *rd,
> -			       const struct kvm_one_reg *reg, void __user *uaddr)
> -{
> -	const u64 id = sys_reg_to_index(rd);
> -	u8 csv2, csv3;
> -	int err;
> -	u64 val;
> -
> -	err = reg_from_user(&val, uaddr, id);
> -	if (err)
> -		return err;
> -
> -	/*
> -	 * Allow AA64PFR0_EL1.CSV2 to be set from userspace as long as
> -	 * it doesn't promise more than what is actually provided (the
> -	 * guest could otherwise be covered in ectoplasmic residue).
> -	 */
> -	csv2 = cpuid_feature_extract_unsigned_field(val, ID_AA64PFR0_CSV2_SHIFT);
> -	if (csv2 > 1 ||
> -	    (csv2 && arm64_get_spectre_v2_state() != SPECTRE_UNAFFECTED))
> -		return -EINVAL;
> -
> -	/* Same thing for CSV3 */
> -	csv3 = cpuid_feature_extract_unsigned_field(val, ID_AA64PFR0_CSV3_SHIFT);
> -	if (csv3 > 1 ||
> -	    (csv3 && arm64_get_meltdown_state() != SPECTRE_UNAFFECTED))
> -		return -EINVAL;
> -
> -	/* We can only differ with CSV[23], and anything else is an error */
> -	val ^= read_id_reg(vcpu, rd, false);
> -	val &= ~((0xFUL << ID_AA64PFR0_CSV2_SHIFT) |
> -		 (0xFUL << ID_AA64PFR0_CSV3_SHIFT));
> -	if (val)
> -		return -EINVAL;
> -
> -	vcpu->kvm->arch.pfr0_csv2 = csv2;
> -	vcpu->kvm->arch.pfr0_csv3 = csv3 ;
> -
> -	return 0;
> -}
> -
>  /* cpufeature ID register user accessors */
>  static int __get_id_reg(const struct kvm_vcpu *vcpu,
>  			const struct sys_reg_desc *rd, void __user *uaddr,
> @@ -1705,8 +1753,7 @@ static const struct sys_reg_desc sys_reg_descs[] = {
>  
>  	/* AArch64 ID registers */
>  	/* CRm=4 */
> -	{ SYS_DESC(SYS_ID_AA64PFR0_EL1), .access = access_id_reg,
> -	  .get_user = get_id_reg, .set_user = set_id_aa64pfr0_el1, },
> +	ID_SANITISED(ID_AA64PFR0_EL1),
>  	ID_SANITISED(ID_AA64PFR1_EL1),
>  	ID_UNALLOCATED(4,2),
>  	ID_UNALLOCATED(4,3),
> 

Thanks

Eric

Re: [RFC PATCH v3 08/29] KVM: arm64: Make ID_AA64MMFR0_EL1 writable

[Eric Auger]

Hi Reiji,
On 11/17/21 7:43 AM, Reiji Watanabe wrote:
> This patch adds id_reg_info for ID_AA64MMFR0_EL1 to make it
> writable by userspace.
> 
> Since ID_AA64MMFR0_EL1 stage 2 granule size fields don't follow the
> standard ID scheme, we need a special handling to validate those fields.
> 
> Signed-off-by: Reiji Watanabe <reijiw@google.com>
> ---
>  arch/arm64/kvm/sys_regs.c | 118 ++++++++++++++++++++++++++++++++++++++
>  1 file changed, 118 insertions(+)
> 
> diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
> index 5812e39602fe..772e3d3067b2 100644
> --- a/arch/arm64/kvm/sys_regs.c
> +++ b/arch/arm64/kvm/sys_regs.c
> @@ -519,6 +519,113 @@ static int validate_id_aa64isar1_el1(struct kvm_vcpu *vcpu,
>  	return 0;
>  }
>  
> +/*
> + * Check if the requested stage2 translation granule size indicated in
> + * @mmfr0 is also indicated in @mmfr0_lim.  This function assumes that
> + * the stage1 granule size indicated in @mmfr0 has been validated already.
> + */
> +static int aa64mmfr0_tgran2_check(int field, u64 mmfr0, u64 mmfr0_lim)
> +{
> +	s64 tgran2, lim_tgran2, rtgran1;
> +	int f1;
> +	bool is_signed = true;
> +
> +	tgran2 = cpuid_feature_extract_unsigned_field(mmfr0, field);
> +	lim_tgran2 = cpuid_feature_extract_unsigned_field(mmfr0_lim, field);
> +	if (tgran2 == lim_tgran2)
> +		return 0;
> +
> +	if (tgran2 && lim_tgran2)
> +		return (tgran2 > lim_tgran2) ? -E2BIG : 0;
> +
> +	/*
> +	 * Either tgran2 or lim_tgran2 is zero.
> +	 * Need stage1 granule size to validate tgran2.
> +	 */
> +	switch (field) {
> +	case ID_AA64MMFR0_TGRAN4_2_SHIFT:
> +		f1 = ID_AA64MMFR0_TGRAN4_SHIFT;
> +		break;
> +	case ID_AA64MMFR0_TGRAN64_2_SHIFT:
> +		f1 = ID_AA64MMFR0_TGRAN64_SHIFT;
> +		break;
> +	case ID_AA64MMFR0_TGRAN16_2_SHIFT:
> +		f1 = ID_AA64MMFR0_TGRAN16_SHIFT;
> +		is_signed = false;
> +		break;
> +	default:
> +		/* Should never happen */
> +		WARN_ONCE(1, "Unexpected stage2 granule field (%d)\n", field);
> +		return 0;
> +	}

sorry my previous message was sent while I haven't finished :-(

if I understand correctly you forbid setting a granule that is not set
in the lim. So I would first compute whether the granule is set, would
it be through the TGranX (if _2 == 0) or though TGranX_2 if this latter
is not not. Do those computations both on val and lim and eventually
check if gran_val > gran_lim. The current code looks overly complicated
but maybe I miss the actual reason.

Eric
> +
> +	/*
> +	 * If tgran2 == 0 (&& lim_tgran2 != 0), the requested stage2 granule
> +	 * size is indicated in the stage1 granule size field of @mmfr0.
> +	 * So, validate the stage1 granule size against the stage2 limit
> +	 * granule size.
> +	 * If lim_tgran2 == 0 (&& tgran2 != 0), the stage2 limit granule size
> +	 * is indicated in the stage1 granule size field of @mmfr0_lim.
> +	 * So, validate the requested stage2 granule size against the stage1
> +	 * limit granule size.
> +	 */
> +
> +	 /* Get the relevant stage1 granule size to validate tgran2 */
> +	if (tgran2 == 0)
> +		/* The requested stage1 granule size */
> +		rtgran1 = cpuid_feature_extract_field(mmfr0, f1, is_signed);
> +	else /* lim_tgran2 == 0 */
> +		/* The stage1 limit granule size */
> +		rtgran1 = cpuid_feature_extract_field(mmfr0_lim, f1, is_signed);
> +
> +	/*
> +	 * Adjust the value of rtgran1 to compare with stage2 granule size,
> +	 * which indicates: 1: Not supported, 2: Supported, etc.
> +	 */
> +	if (is_signed)
> +		/* For signed, -1: Not supported, 0: Supported, etc. */
> +		rtgran1 += 0x2;
> +	else
> +		/* For unsigned, 0: Not supported, 1: Supported, etc. */
> +		rtgran1 += 0x1;
> +
> +	if ((tgran2 == 0) && (rtgran1 > lim_tgran2))
> +		/*
> +		 * The requested stage1 granule size (== the requested stage2
> +		 * granule size) is larger than the stage2 limit granule size.
> +		 */
> +		return -E2BIG;
> +	else if ((lim_tgran2 == 0) && (tgran2 > rtgran1))
> +		/*
> +		 * The requested stage2 granule size is larger than the stage1
> +		 * limit granulze size (== the stage2 limit granule size).
> +		 */
> +		return -E2BIG;
> +
> +	return 0;
> +}
> +
> +static int validate_id_aa64mmfr0_el1(struct kvm_vcpu *vcpu,
> +				     const struct id_reg_info *id_reg, u64 val)
> +{
> +	u64 limit = id_reg->vcpu_limit_val;
> +	int ret;
> +
> +	ret = aa64mmfr0_tgran2_check(ID_AA64MMFR0_TGRAN4_2_SHIFT, val, limit);
> +	if (ret)
> +		return ret;
> +
> +	ret = aa64mmfr0_tgran2_check(ID_AA64MMFR0_TGRAN64_2_SHIFT, val, limit);
> +	if (ret)
> +		return ret;
> +
> +	ret = aa64mmfr0_tgran2_check(ID_AA64MMFR0_TGRAN16_2_SHIFT, val, limit);
> +	if (ret)
> +		return ret;
> +
> +	return 0;
> +}
> +
>  static void init_id_aa64pfr0_el1_info(struct id_reg_info *id_reg)
>  {
>  	u64 limit = id_reg->vcpu_limit_val;
> @@ -625,6 +732,16 @@ static struct id_reg_info id_aa64isar1_el1_info = {
>  	.get_reset_val = get_reset_id_aa64isar1_el1,
>  };
>  
> +static struct id_reg_info id_aa64mmfr0_el1_info = {
> +	.sys_reg = SYS_ID_AA64MMFR0_EL1,
> +	.ftr_check_types = S_FCT(ID_AA64MMFR0_TGRAN4_SHIFT, FCT_LOWER_SAFE) |
> +			   S_FCT(ID_AA64MMFR0_TGRAN64_SHIFT, FCT_LOWER_SAFE) |
> +			   U_FCT(ID_AA64MMFR0_TGRAN4_2_SHIFT, FCT_IGNORE) |
> +			   U_FCT(ID_AA64MMFR0_TGRAN64_2_SHIFT, FCT_IGNORE) |
> +			   U_FCT(ID_AA64MMFR0_TGRAN16_2_SHIFT, FCT_IGNORE),
> +	.validate = validate_id_aa64mmfr0_el1,
> +};
> +
>  /*
>   * An ID register that needs special handling to control the value for the
>   * guest must have its own id_reg_info in id_reg_info_table.
> @@ -638,6 +755,7 @@ static struct id_reg_info *id_reg_info_table[KVM_ARM_ID_REG_MAX_NUM] = {
>  	[IDREG_IDX(SYS_ID_AA64PFR1_EL1)] = &id_aa64pfr1_el1_info,
>  	[IDREG_IDX(SYS_ID_AA64ISAR0_EL1)] = &id_aa64isar0_el1_info,
>  	[IDREG_IDX(SYS_ID_AA64ISAR1_EL1)] = &id_aa64isar1_el1_info,
> +	[IDREG_IDX(SYS_ID_AA64MMFR0_EL1)] = &id_aa64mmfr0_el1_info,
>  };
>  
>  static int validate_id_reg(struct kvm_vcpu *vcpu,
> 

Re: [RFC PATCH v3 10/29] KVM: arm64: Make ID_AA64DFR0_EL1 writable

[Eric Auger]

Hi Reiji,

On 11/17/21 7:43 AM, Reiji Watanabe wrote:
> This patch adds id_reg_info for ID_AA64DFR0_EL1 to make it writable
> by userspace.
> 
> Return an error if userspace tries to set PMUVER field of the
> register to a value that conflicts with the PMU configuration.
> 
> Since number of context-aware breakpoints must be no more than number
> of supported breakpoints according to Arm ARM, return an error
> if userspace tries to set CTX_CMPS field to such value.
> 
> Signed-off-by: Reiji Watanabe <reijiw@google.com>
> ---
>  arch/arm64/kvm/sys_regs.c | 84 ++++++++++++++++++++++++++++++++++-----
>  1 file changed, 73 insertions(+), 11 deletions(-)
> 
> diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
> index 772e3d3067b2..0faf458b0efb 100644
> --- a/arch/arm64/kvm/sys_regs.c
> +++ b/arch/arm64/kvm/sys_regs.c
> @@ -626,6 +626,45 @@ static int validate_id_aa64mmfr0_el1(struct kvm_vcpu *vcpu,
>  	return 0;
>  }
>  
> +static bool id_reg_has_pmu(u64 val, u64 shift, unsigned int min)
I would rename the function as the name currently is misleading. The
function validate the val filed @shift againt @min
> +{
> +	unsigned int pmu = cpuid_feature_extract_unsigned_field(val, shift);
> +
> +	/*
> +	 * Treat IMPLEMENTATION DEFINED functionality as unimplemented for
> +	 * ID_AA64DFR0_EL1.PMUVer/ID_DFR0_EL1.PerfMon.
> +	 */
> +	if (pmu == 0xf)
> +		pmu = 0;
Shouldn't we simply forbid the userspace to set 0xF?
> +
> +	return (pmu >= min);
> +}
> +
> +static int validate_id_aa64dfr0_el1(struct kvm_vcpu *vcpu,
> +				    const struct id_reg_info *id_reg, u64 val)
> +{
> +	unsigned int brps, ctx_cmps;
> +	bool vcpu_pmu, dfr0_pmu;
> +
> +	brps = cpuid_feature_extract_unsigned_field(val, ID_AA64DFR0_BRPS_SHIFT);
> +	ctx_cmps = cpuid_feature_extract_unsigned_field(val, ID_AA64DFR0_CTX_CMPS_SHIFT);
> +
> +	/*
> +	 * Number of context-aware breakpoints can be no more than number of
> +	 * supported breakpoints.
> +	 */
> +	if (ctx_cmps > brps)
> +		return -EINVAL;
> +
> +	vcpu_pmu = kvm_vcpu_has_pmu(vcpu);
> +	dfr0_pmu = id_reg_has_pmu(val, ID_AA64DFR0_PMUVER_SHIFT, ID_AA64DFR0_PMUVER_8_0);
> +	/* Check if there is a conflict with a request via KVM_ARM_VCPU_INIT */
> +	if (vcpu_pmu ^ dfr0_pmu)
> +		return -EPERM;
> +
> +	return 0;
> +}
> +
>  static void init_id_aa64pfr0_el1_info(struct id_reg_info *id_reg)
>  {
>  	u64 limit = id_reg->vcpu_limit_val;
> @@ -669,6 +708,23 @@ static void init_id_aa64isar1_el1_info(struct id_reg_info *id_reg)
>  		id_reg->vcpu_limit_val &= ~PTRAUTH_MASK;
>  }
>  
> +static void init_id_aa64dfr0_el1_info(struct id_reg_info *id_reg)
> +{
> +	u64 limit = id_reg->vcpu_limit_val;
> +
> +	/* Limit guests to PMUv3 for ARMv8.4 */
> +	limit = cpuid_feature_cap_perfmon_field(limit, ID_AA64DFR0_PMUVER_SHIFT,
> +						ID_AA64DFR0_PMUVER_8_4);
> +	/* Limit debug to ARMv8.0 */
> +	limit &= ~ARM64_FEATURE_MASK(ID_AA64DFR0_DEBUGVER);
> +	limit |= (FIELD_PREP(ARM64_FEATURE_MASK(ID_AA64DFR0_DEBUGVER), 6));
> +
> +	/* Hide SPE from guests */
> +	limit &= ~ARM64_FEATURE_MASK(ID_AA64DFR0_PMSVER);
> +
> +	id_reg->vcpu_limit_val = limit;
> +}
> +
>  static u64 get_reset_id_aa64pfr0_el1(struct kvm_vcpu *vcpu,
>  				     const struct id_reg_info *idr)
>  {
> @@ -698,6 +754,14 @@ static u64 get_reset_id_aa64isar1_el1(struct kvm_vcpu *vcpu,
>  	       idr->vcpu_limit_val : (idr->vcpu_limit_val & ~PTRAUTH_MASK);
>  }
>  
> +static u64 get_reset_id_aa64dfr0_el1(struct kvm_vcpu *vcpu,
> +				     const struct id_reg_info *idr)
> +{
> +	return kvm_vcpu_has_pmu(vcpu) ?
> +	       idr->vcpu_limit_val :
> +	       (idr->vcpu_limit_val & ~(ARM64_FEATURE_MASK(ID_AA64DFR0_PMUVER)));
> +}
> +
>  static struct id_reg_info id_aa64pfr0_el1_info = {
>  	.sys_reg = SYS_ID_AA64PFR0_EL1,
>  	.ftr_check_types = S_FCT(ID_AA64PFR0_ASIMD_SHIFT, FCT_LOWER_SAFE) |
> @@ -742,6 +806,14 @@ static struct id_reg_info id_aa64mmfr0_el1_info = {
>  	.validate = validate_id_aa64mmfr0_el1,
>  };
>  
> +static struct id_reg_info id_aa64dfr0_el1_info = {
> +	.sys_reg = SYS_ID_AA64DFR0_EL1,
> +	.ftr_check_types = S_FCT(ID_AA64DFR0_DOUBLELOCK_SHIFT, FCT_LOWER_SAFE),
> +	.init = init_id_aa64dfr0_el1_info,
> +	.validate = validate_id_aa64dfr0_el1,
> +	.get_reset_val = get_reset_id_aa64dfr0_el1,
> +};
> +
>  /*
>   * An ID register that needs special handling to control the value for the
>   * guest must have its own id_reg_info in id_reg_info_table.
> @@ -753,6 +825,7 @@ static struct id_reg_info id_aa64mmfr0_el1_info = {
>  static struct id_reg_info *id_reg_info_table[KVM_ARM_ID_REG_MAX_NUM] = {
>  	[IDREG_IDX(SYS_ID_AA64PFR0_EL1)] = &id_aa64pfr0_el1_info,
>  	[IDREG_IDX(SYS_ID_AA64PFR1_EL1)] = &id_aa64pfr1_el1_info,
> +	[IDREG_IDX(SYS_ID_AA64DFR0_EL1)] = &id_aa64dfr0_el1_info,
>  	[IDREG_IDX(SYS_ID_AA64ISAR0_EL1)] = &id_aa64isar0_el1_info,
>  	[IDREG_IDX(SYS_ID_AA64ISAR1_EL1)] = &id_aa64isar1_el1_info,
>  	[IDREG_IDX(SYS_ID_AA64MMFR0_EL1)] = &id_aa64mmfr0_el1_info,
> @@ -1604,17 +1677,6 @@ static u64 __read_id_reg(const struct kvm_vcpu *vcpu, u32 id)
>  			val |= FIELD_PREP(ARM64_FEATURE_MASK(ID_AA64PFR0_GIC), gic_lim);
>  		}
>  		break;
> -	case SYS_ID_AA64DFR0_EL1:
> -		/* Limit debug to ARMv8.0 */
> -		val &= ~ARM64_FEATURE_MASK(ID_AA64DFR0_DEBUGVER);
> -		val |= FIELD_PREP(ARM64_FEATURE_MASK(ID_AA64DFR0_DEBUGVER), 6);
> -		/* Limit guests to PMUv3 for ARMv8.4 */
> -		val = cpuid_feature_cap_perfmon_field(val,
> -						      ID_AA64DFR0_PMUVER_SHIFT,
> -						      kvm_vcpu_has_pmu(vcpu) ? ID_AA64DFR0_PMUVER_8_4 : 0);
> -		/* Hide SPE from guests */
> -		val &= ~ARM64_FEATURE_MASK(ID_AA64DFR0_PMSVER);
> -		break;
>  	case SYS_ID_DFR0_EL1:
>  		/* Limit guests to PMUv3 for ARMv8.4 */
>  		val = cpuid_feature_cap_perfmon_field(val,
> 
Eric

Re: [RFC PATCH v3 12/29] KVM: arm64: Make ID_DFR1_EL1 writable

[Eric Auger]

Hi Reiji,

On 11/17/21 7:43 AM, Reiji Watanabe wrote:
> This patch adds id_reg_info for ID_DFR1_EL1 to make it writable
> by userspace.
> 
> Signed-off-by: Reiji Watanabe <reijiw@google.com>
> ---
>  arch/arm64/kvm/sys_regs.c | 6 ++++++
>  1 file changed, 6 insertions(+)
> 
> diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
> index fbd335ac5e6b..dda7001959f6 100644
> --- a/arch/arm64/kvm/sys_regs.c
> +++ b/arch/arm64/kvm/sys_regs.c
> @@ -859,6 +859,11 @@ static struct id_reg_info id_dfr0_el1_info = {
>  	.get_reset_val = get_reset_id_dfr0_el1,
>  };
>  
> +static struct id_reg_info id_dfr1_el1_info = {
> +	.sys_reg = SYS_ID_DFR1_EL1,
> +	.ftr_check_types = S_FCT(ID_DFR1_MTPMU_SHIFT, FCT_LOWER_SAFE),
what about the 0xF value which indicates the MTPMU is not implemented?

Eric
> +};
> +
>  /*
>   * An ID register that needs special handling to control the value for the
>   * guest must have its own id_reg_info in id_reg_info_table.
> @@ -869,6 +874,7 @@ static struct id_reg_info id_dfr0_el1_info = {
>  #define	GET_ID_REG_INFO(id)	(id_reg_info_table[IDREG_IDX(id)])
>  static struct id_reg_info *id_reg_info_table[KVM_ARM_ID_REG_MAX_NUM] = {
>  	[IDREG_IDX(SYS_ID_DFR0_EL1)] = &id_dfr0_el1_info,
> +	[IDREG_IDX(SYS_ID_DFR1_EL1)] = &id_dfr1_el1_info,
>  	[IDREG_IDX(SYS_ID_AA64PFR0_EL1)] = &id_aa64pfr0_el1_info,
>  	[IDREG_IDX(SYS_ID_AA64PFR1_EL1)] = &id_aa64pfr1_el1_info,
>  	[IDREG_IDX(SYS_ID_AA64DFR0_EL1)] = &id_aa64dfr0_el1_info,
> 

Re: [RFC PATCH v3 09/29] KVM: arm64: Hide IMPLEMENTATION DEFINED PMU support for the guest

[Eric Auger]

Hi Reiji,

On 11/17/21 7:43 AM, Reiji Watanabe wrote:
> When ID_AA64DFR0_EL1.PMUVER or ID_DFR0_EL1.PERFMON is 0xf, which
> means IMPLEMENTATION DEFINED PMU supported, KVM unconditionally
> expose the value for the guest as it is.  Since KVM doesn't support
> IMPLEMENTATION DEFINED PMU for the guest, in that case KVM should
> exopse 0x0 (PMU is not implemented) instead.
s/exopse/expose
> 
> Change cpuid_feature_cap_perfmon_field() to update the field value
> to 0x0 when it is 0xf.
is it wrong to expose the guest with a Perfmon value of 0xF? Then the
guest should not use it as a PMUv3?

Eric
> 
> Fixes: 8e35aa642ee4 ("arm64: cpufeature: Extract capped perfmon fields")
> Signed-off-by: Reiji Watanabe <reijiw@google.com>
> ---
>  arch/arm64/include/asm/cpufeature.h | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
> index ef6be92b1921..fd7ad8193827 100644
> --- a/arch/arm64/include/asm/cpufeature.h
> +++ b/arch/arm64/include/asm/cpufeature.h
> @@ -553,7 +553,7 @@ cpuid_feature_cap_perfmon_field(u64 features, int field, u64 cap)
>  
>  	/* Treat IMPLEMENTATION DEFINED functionality as unimplemented */
>  	if (val == ID_AA64DFR0_PMUVER_IMP_DEF)
> -		val = 0;
> +		return (features & ~mask);
>  
>  	if (val > cap) {
>  		features &= ~mask;
> 

Re: [RFC PATCH v3 04/29] KVM: arm64: Make ID_AA64PFR0_EL1 writable

[Reiji Watanabe]

Hi Eric,

On Thu, Nov 25, 2021 at 7:35 AM Eric Auger <eauger@redhat.com> wrote:
>
> Hi Reiji,
>
> On 11/17/21 7:43 AM, Reiji Watanabe wrote:
> > This patch adds id_reg_info for ID_AA64PFR0_EL1 to make it writable by
> > userspace.
> >
> > The CSV2/CSV3 fields of the register were already writable and values
> > that were written for them affected all vCPUs before. Now they only
> > affect the vCPU.
> > Return an error if userspace tries to set SVE/GIC field of the register
> > to a value that conflicts with SVE/GIC configuration for the guest.
> > SIMD/FP/SVE fields of the requested value are validated according to
> > Arm ARM.
> >
> > Signed-off-by: Reiji Watanabe <reijiw@google.com>
> > ---
> >  arch/arm64/kvm/sys_regs.c | 159 ++++++++++++++++++++++++--------------
> >  1 file changed, 103 insertions(+), 56 deletions(-)
> >
> > diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
> > index 1552cd5581b7..35400869067a 100644
> > --- a/arch/arm64/kvm/sys_regs.c
> > +++ b/arch/arm64/kvm/sys_regs.c
> > @@ -401,6 +401,92 @@ static void id_reg_info_init(struct id_reg_info *id_reg)
> >               id_reg->init(id_reg);
> >  }
> >
> > +#define      kvm_has_gic3(kvm)               \
> > +     (irqchip_in_kernel(kvm) &&      \
> > +      (kvm)->arch.vgic.vgic_model == KVM_DEV_TYPE_ARM_VGIC_V3)
> you may move this macro to kvm/arm_vgic.h as this may be used in
> vgic/vgic-v3.c too

Thank you for the suggestion. I will move that to kvm/arm_vgic.h.


> > +
> > +static int validate_id_aa64pfr0_el1(struct kvm_vcpu *vcpu,
> > +                                 const struct id_reg_info *id_reg, u64 val)
> > +{
> > +     int fp, simd;
> > +     bool vcpu_has_sve = vcpu_has_sve(vcpu);
> > +     bool pfr0_has_sve = id_aa64pfr0_sve(val);
> > +     int gic;
> > +
> > +     simd = cpuid_feature_extract_signed_field(val, ID_AA64PFR0_ASIMD_SHIFT);
> > +     fp = cpuid_feature_extract_signed_field(val, ID_AA64PFR0_FP_SHIFT);
> > +     if (simd != fp)
> > +             return -EINVAL;
> > +
> > +     /* fp must be supported when sve is supported */
> > +     if (pfr0_has_sve && (fp < 0))
> > +             return -EINVAL;
> > +
> > +     /* Check if there is a conflict with a request via KVM_ARM_VCPU_INIT */
> > +     if (vcpu_has_sve ^ pfr0_has_sve)
> > +             return -EPERM;
> > +
> > +     gic = cpuid_feature_extract_unsigned_field(val, ID_AA64PFR0_GIC_SHIFT);
> > +     if ((gic > 0) ^ kvm_has_gic3(vcpu->kvm))
> > +             return -EPERM;
>
> Sometimes from a given architecture version, some lower values are not
> allowed. For instance from ARMv8.5 onlt 1 is permitted for CSV3.
> Shouldn't we handle that kind of check?

As far as I know, there is no way for guests to identify the
architecture revision (e.g. v8.1, v8.2, etc).  It might be able
to indirectly infer the revision though (from features that are
available or etc).


> > +
> > +     return 0;
> > +}
> > +
> > +static void init_id_aa64pfr0_el1_info(struct id_reg_info *id_reg)
> > +{
> > +     u64 limit = id_reg->vcpu_limit_val;
> > +     unsigned int gic;
> > +
> > +     limit &= ~ARM64_FEATURE_MASK(ID_AA64PFR0_AMU);
> > +     if (!system_supports_sve())
> > +             limit &= ~ARM64_FEATURE_MASK(ID_AA64PFR0_SVE);
> > +
> > +     /*
> > +      * The default is to expose CSV2 == 1 and CSV3 == 1 if the HW
> > +      * isn't affected.  Userspace can override this as long as it
> > +      * doesn't promise the impossible.
> > +      */
> > +     limit &= ~(ARM64_FEATURE_MASK(ID_AA64PFR0_CSV2) |
> > +                ARM64_FEATURE_MASK(ID_AA64PFR0_CSV3));
> > +
> > +     if (arm64_get_spectre_v2_state() == SPECTRE_UNAFFECTED)
> > +             limit |= FIELD_PREP(ARM64_FEATURE_MASK(ID_AA64PFR0_CSV2), 1);
> > +     if (arm64_get_meltdown_state() == SPECTRE_UNAFFECTED)
> > +             limit |= FIELD_PREP(ARM64_FEATURE_MASK(ID_AA64PFR0_CSV3), 1);
> > +
> > +     gic = cpuid_feature_extract_unsigned_field(limit, ID_AA64PFR0_GIC_SHIFT);
> > +     if (gic > 1) {
> > +             /* Limit to GICv3.0/4.0 */
> > +             limit &= ~ARM64_FEATURE_MASK(ID_AA64PFR0_GIC);
> > +             limit |= FIELD_PREP(ARM64_FEATURE_MASK(ID_AA64PFR0_GIC), 1);
> > +     }
> > +     id_reg->vcpu_limit_val = limit;
> > +}
> > +
> > +static u64 get_reset_id_aa64pfr0_el1(struct kvm_vcpu *vcpu,
> > +                                  const struct id_reg_info *idr)
> > +{
> > +     u64 val = idr->vcpu_limit_val;
> > +
> > +     if (!vcpu_has_sve(vcpu))
> > +             val &= ~ARM64_FEATURE_MASK(ID_AA64PFR0_SVE);
> > +
> > +     if (!kvm_has_gic3(vcpu->kvm))
> > +             val &= ~ARM64_FEATURE_MASK(ID_AA64PFR0_GIC);
> > +
> > +     return val;
> > +}
> > +
> > +static struct id_reg_info id_aa64pfr0_el1_info = {
> > +     .sys_reg = SYS_ID_AA64PFR0_EL1,
> > +     .ftr_check_types = S_FCT(ID_AA64PFR0_ASIMD_SHIFT, FCT_LOWER_SAFE) |
> > +                        S_FCT(ID_AA64PFR0_FP_SHIFT, FCT_LOWER_SAFE),
> is it needed as it is the default?

> > +     .ftr_check_types = S_FCT(ID_AA64PFR0_ASIMD_SHIFT, FCT_LOWER_SAFE) |
> > +                        S_FCT(ID_AA64PFR0_FP_SHIFT, FCT_LOWER_SAFE),
> is it needed as it is the default?

They are needed because they are signed fields (the default is unsigned
and FCT_LOWER_SAFE).  Having said that, ftr_check_types itself will be
gone in the next version (as arm64_ftr_bits will be used instead).

Thanks,
Reiji

Re: [RFC PATCH v3 08/29] KVM: arm64: Make ID_AA64MMFR0_EL1 writable

[Reiji Watanabe]

 Hi Eric,

On Thu, Nov 25, 2021 at 7:31 AM Eric Auger <eauger@redhat.com> wrote:
>
>
>
> On 11/17/21 7:43 AM, Reiji Watanabe wrote:
> > This patch adds id_reg_info for ID_AA64MMFR0_EL1 to make it
> > writable by userspace.
> >
> > Since ID_AA64MMFR0_EL1 stage 2 granule size fields don't follow the
> > standard ID scheme, we need a special handling to validate those fields.
> >
> > Signed-off-by: Reiji Watanabe <reijiw@google.com>
> > ---
> >  arch/arm64/kvm/sys_regs.c | 118 ++++++++++++++++++++++++++++++++++++++
> >  1 file changed, 118 insertions(+)
> >
> > diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
> > index 5812e39602fe..772e3d3067b2 100644
> > --- a/arch/arm64/kvm/sys_regs.c
> > +++ b/arch/arm64/kvm/sys_regs.c
> > @@ -519,6 +519,113 @@ static int validate_id_aa64isar1_el1(struct kvm_vcpu *vcpu,
> >       return 0;
> >  }
> >
> > +/*
> > + * Check if the requested stage2 translation granule size indicated in
> > + * @mmfr0 is also indicated in @mmfr0_lim.  This function assumes that
> > + * the stage1 granule size indicated in @mmfr0 has been validated already.
> I would suggest: relies on the fact TGranX fields are validated before
> through the arm64_check_features lookup

Thank you for the suggestion.  I will fix the comment as you suggested.

> > + */
> > +static int aa64mmfr0_tgran2_check(int field, u64 mmfr0, u64 mmfr0_lim)
> > +{
> > +     s64 tgran2, lim_tgran2, rtgran1;
> > +     int f1;
> > +     bool is_signed = true;
> > +
> > +     tgran2 = cpuid_feature_extract_unsigned_field(mmfr0, field);
> > +     lim_tgran2 = cpuid_feature_extract_unsigned_field(mmfr0_lim, field);
> > +     if (tgran2 == lim_tgran2)
> > +             return 0;
> > +
> > +     if (tgran2 && lim_tgran2)
> > +             return (tgran2 > lim_tgran2) ? -E2BIG : 0;
> > +
> > +     /*
> > +      * Either tgran2 or lim_tgran2 is zero.
> > +      * Need stage1 granule size to validate tgran2.
> > +      */
> > +     switch (field) {
> > +     case ID_AA64MMFR0_TGRAN4_2_SHIFT:
> > +             f1 = ID_AA64MMFR0_TGRAN4_SHIFT;
> > +             break;
> > +     case ID_AA64MMFR0_TGRAN64_2_SHIFT:
> > +             f1 = ID_AA64MMFR0_TGRAN64_SHIFT;
> > +             break;
> > +     case ID_AA64MMFR0_TGRAN16_2_SHIFT:
> > +             f1 = ID_AA64MMFR0_TGRAN16_SHIFT;
> > +             is_signed = false;
> I don't get the is_signed setting. Don't the TGRAN_x have the same
> format? Beside you can get the shift by substracting 12 to @field.

Yes, TGran16 is different from TGran64/TGran4.
 https://developer.arm.com/documentation/ddi0595/2021-09/AArch64-Registers/ID-AA64MMFR0-EL1--AArch64-Memory-Model-Feature-Register-0?lang=en

I didn't realize that we could get the shift by substracting 12 from
@field.  Thank you for the information.  I will use that.


> can't you directly compute if the granule is supported

No, I don't think we can because the value 0 in the TGranx_2 means
that its feature support is identified by another field (TGranx).


> > +             break;
> > +     default:
> > +             /* Should never happen */
> > +             WARN_ONCE(1, "Unexpected stage2 granule field (%d)\n", field);
> > +             return 0;
> > +     }
> > +
> > +     /*
> > +      * If tgran2 == 0 (&& lim_tgran2 != 0), the requested stage2 granule
> > +      * size is indicated in the stage1 granule size field of @mmfr0.
> > +      * So, validate the stage1 granule size against the stage2 limit
> > +      * granule size.
> > +      * If lim_tgran2 == 0 (&& tgran2 != 0), the stage2 limit granule size
> > +      * is indicated in the stage1 granule size field of @mmfr0_lim.
> > +      * So, validate the requested stage2 granule size against the stage1
> > +      * limit granule size.
> > +      */
> > +
> > +      /* Get the relevant stage1 granule size to validate tgran2 */
> > +     if (tgran2 == 0)
> > +             /* The requested stage1 granule size */
> > +             rtgran1 = cpuid_feature_extract_field(mmfr0, f1, is_signed);
> > +     else /* lim_tgran2 == 0 */
> > +             /* The stage1 limit granule size */
> > +             rtgran1 = cpuid_feature_extract_field(mmfr0_lim, f1, is_signed);
> > +
> > +     /*
> > +      * Adjust the value of rtgran1 to compare with stage2 granule size,
> > +      * which indicates: 1: Not supported, 2: Supported, etc.
> > +      */
> > +     if (is_signed)
> > +             /* For signed, -1: Not supported, 0: Supported, etc. */
> > +             rtgran1 += 0x2;
> > +     else
> > +             /* For unsigned, 0: Not supported, 1: Supported, etc. */
> > +             rtgran1 += 0x1;
> > +
> > +     if ((tgran2 == 0) && (rtgran1 > lim_tgran2))
> > +             /*
> > +              * The requested stage1 granule size (== the requested stage2
> > +              * granule size) is larger than the stage2 limit granule size.
> > +              */
> > +             return -E2BIG;
> > +     else if ((lim_tgran2 == 0) && (tgran2 > rtgran1))
> > +             /*
> > +              * The requested stage2 granule size is larger than the stage1
> > +              * limit granulze size (== the stage2 limit granule size).
> > +              */
> > +             return -E2BIG;
> > +
> > +     return 0;
> > +}
> > +
> > +static int validate_id_aa64mmfr0_el1(struct kvm_vcpu *vcpu,
> > +                                  const struct id_reg_info *id_reg, u64 val)
> > +{
> > +     u64 limit = id_reg->vcpu_limit_val;
> > +     int ret;
>
> shouldn't you forbid reserved values for TGran4, 64?

I think what you meant is applied to all signed feature fields on
any ID registers.  Considering "Principles of the ID scheme for fields
in ID registers" that is described in Arm ARM, lower value than -1
should not indicate more or higher level of features than KVM on the
host can support.  In that sense, it doesn't matter (I would think
it won't cause any problems for guests).  So, I rather chose not to do
that check.


> > +
> > +     ret = aa64mmfr0_tgran2_check(ID_AA64MMFR0_TGRAN4_2_SHIFT, val, limit);
> > +     if (ret)
> > +             return ret;
> > +
> > +     ret = aa64mmfr0_tgran2_check(ID_AA64MMFR0_TGRAN64_2_SHIFT, val, limit);
> > +     if (ret)
> > +             return ret;
> > +
> > +     ret = aa64mmfr0_tgran2_check(ID_AA64MMFR0_TGRAN16_2_SHIFT, val, limit);
> > +     if (ret)
> > +             return ret;
> > +
> > +     return 0;
> > +}
> > +
> >  static void init_id_aa64pfr0_el1_info(struct id_reg_info *id_reg)
> >  {
> >       u64 limit = id_reg->vcpu_limit_val;
> > @@ -625,6 +732,16 @@ static struct id_reg_info id_aa64isar1_el1_info = {
> >       .get_reset_val = get_reset_id_aa64isar1_el1,
> >  };
> >
> > +static struct id_reg_info id_aa64mmfr0_el1_info = {
> > +     .sys_reg = SYS_ID_AA64MMFR0_EL1,
> > +     .ftr_check_types = S_FCT(ID_AA64MMFR0_TGRAN4_SHIFT, FCT_LOWER_SAFE) |
> > +                        S_FCT(ID_AA64MMFR0_TGRAN64_SHIFT, FCT_LOWER_SAFE) |
> the default?

They are signed fields, which are not a default.

> > +                        U_FCT(ID_AA64MMFR0_TGRAN4_2_SHIFT, FCT_IGNORE) |
> > +                        U_FCT(ID_AA64MMFR0_TGRAN64_2_SHIFT, FCT_IGNORE) |
> > +                        U_FCT(ID_AA64MMFR0_TGRAN16_2_SHIFT, FCT_IGNORE),
> maybe add comment telling the actual check is handled in the validate cb

That's a good point.
I will add a brief explanation about the validate cb.

Thanks,
Reiji

Re: [RFC PATCH v3 10/29] KVM: arm64: Make ID_AA64DFR0_EL1 writable

[Reiji Watanabe]

Hi Eric,

On Thu, Nov 25, 2021 at 12:30 PM Eric Auger <eauger@redhat.com> wrote:
>
> Hi Reiji,
>
> On 11/17/21 7:43 AM, Reiji Watanabe wrote:
> > This patch adds id_reg_info for ID_AA64DFR0_EL1 to make it writable
> > by userspace.
> >
> > Return an error if userspace tries to set PMUVER field of the
> > register to a value that conflicts with the PMU configuration.
> >
> > Since number of context-aware breakpoints must be no more than number
> > of supported breakpoints according to Arm ARM, return an error
> > if userspace tries to set CTX_CMPS field to such value.
> >
> > Signed-off-by: Reiji Watanabe <reijiw@google.com>
> > ---
> >  arch/arm64/kvm/sys_regs.c | 84 ++++++++++++++++++++++++++++++++++-----
> >  1 file changed, 73 insertions(+), 11 deletions(-)
> >
> > diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
> > index 772e3d3067b2..0faf458b0efb 100644
> > --- a/arch/arm64/kvm/sys_regs.c
> > +++ b/arch/arm64/kvm/sys_regs.c
> > @@ -626,6 +626,45 @@ static int validate_id_aa64mmfr0_el1(struct kvm_vcpu *vcpu,
> >       return 0;
> >  }
> >
> > +static bool id_reg_has_pmu(u64 val, u64 shift, unsigned int min)
> I would rename the function as the name currently is misleading. The
> function validate the val filed @shift againt @min

Thank you for the comment.

The @min is the minimum value that indicates PMUv3 support.
So, if the field value is >= @min, it means PMUv3 is supported.
I want the function to check whether or not @val indicates PMUv3 support,
and that's how the function is used.
I can see what you meant focusing on the function though.
But, if we renaming it to xxx_validate, that would be misleading in the
codes that use the function.

> > +{
> > +     unsigned int pmu = cpuid_feature_extract_unsigned_field(val, shift);
> > +
> > +     /*
> > +      * Treat IMPLEMENTATION DEFINED functionality as unimplemented for
> > +      * ID_AA64DFR0_EL1.PMUVer/ID_DFR0_EL1.PerfMon.
> > +      */
> > +     if (pmu == 0xf)
> > +             pmu = 0;
> Shouldn't we simply forbid the userspace to set 0xF?

This function is to check whether or not the field value indicates PMUv3.
Setting the field to 0xf is forbidden by arm64_check_features().
Having said that, since arm64_check_features() will be implemented by
using arm64_ftr_bits, which treats AA64DFR0.PMUVER and DFR0.PERFMON
as signed fields.
So, it will be forbidden in a different way in the next version.

Thanks,
Reiji

> > +
> > +     return (pmu >= min);
> > +}

Re: [RFC PATCH v3 09/29] KVM: arm64: Hide IMPLEMENTATION DEFINED PMU support for the guest

[Reiji Watanabe]

Hi Eric,

On Thu, Nov 25, 2021 at 12:30 PM Eric Auger <eauger@redhat.com> wrote:
>
> Hi Reiji,
>
> On 11/17/21 7:43 AM, Reiji Watanabe wrote:
> > When ID_AA64DFR0_EL1.PMUVER or ID_DFR0_EL1.PERFMON is 0xf, which
> > means IMPLEMENTATION DEFINED PMU supported, KVM unconditionally
> > expose the value for the guest as it is.  Since KVM doesn't support
> > IMPLEMENTATION DEFINED PMU for the guest, in that case KVM should
> > exopse 0x0 (PMU is not implemented) instead.
> s/exopse/expose
> >
> > Change cpuid_feature_cap_perfmon_field() to update the field value
> > to 0x0 when it is 0xf.
> is it wrong to expose the guest with a Perfmon value of 0xF? Then the
> guest should not use it as a PMUv3?

> is it wrong to expose the guest with a Perfmon value of 0xF? Then the
> guest should not use it as a PMUv3?

For the value 0xf in ID_AA64DFR0_EL1.PMUVER and ID_DFR0_EL1.PERFMON,
Arm ARM says:
  "IMPLEMENTATION DEFINED form of performance monitors supported,
   PMUv3 not supported."

Since the PMU that KVM supports for guests is PMUv3, 0xf shouldn't
be exposed to guests (And this patch series doesn't allow userspace
to set the fields to 0xf for guests).

Thanks,
Reiji

>
> Eric
> >
> > Fixes: 8e35aa642ee4 ("arm64: cpufeature: Extract capped perfmon fields")
> > Signed-off-by: Reiji Watanabe <reijiw@google.com>
> > ---
> >  arch/arm64/include/asm/cpufeature.h | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
> > index ef6be92b1921..fd7ad8193827 100644
> > --- a/arch/arm64/include/asm/cpufeature.h
> > +++ b/arch/arm64/include/asm/cpufeature.h
> > @@ -553,7 +553,7 @@ cpuid_feature_cap_perfmon_field(u64 features, int field, u64 cap)
> >
> >       /* Treat IMPLEMENTATION DEFINED functionality as unimplemented */
> >       if (val == ID_AA64DFR0_PMUVER_IMP_DEF)
> > -             val = 0;
> > +             return (features & ~mask);
> >
> >       if (val > cap) {
> >               features &= ~mask;
> >
>

Re: [RFC PATCH v3 12/29] KVM: arm64: Make ID_DFR1_EL1 writable

[Reiji Watanabe]

Hi Eric,

On Thu, Nov 25, 2021 at 12:30 PM Eric Auger <eauger@redhat.com> wrote:
>
> Hi Reiji,
>
> On 11/17/21 7:43 AM, Reiji Watanabe wrote:
> > This patch adds id_reg_info for ID_DFR1_EL1 to make it writable
> > by userspace.
> >
> > Signed-off-by: Reiji Watanabe <reijiw@google.com>
> > ---
> >  arch/arm64/kvm/sys_regs.c | 6 ++++++
> >  1 file changed, 6 insertions(+)
> >
> > diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
> > index fbd335ac5e6b..dda7001959f6 100644
> > --- a/arch/arm64/kvm/sys_regs.c
> > +++ b/arch/arm64/kvm/sys_regs.c
> > @@ -859,6 +859,11 @@ static struct id_reg_info id_dfr0_el1_info = {
> >       .get_reset_val = get_reset_id_dfr0_el1,
> >  };
> >
> > +static struct id_reg_info id_dfr1_el1_info = {
> > +     .sys_reg = SYS_ID_DFR1_EL1,
> > +     .ftr_check_types = S_FCT(ID_DFR1_MTPMU_SHIFT, FCT_LOWER_SAFE),
> what about the 0xF value which indicates the MTPMU is not implemented?

The field is treated as a signed field.
So, 0xf(== -1) is handled correctly.
(Does it answer your question?)

Thanks,
Reiji

>
> Eric
> > +};
> > +
> >  /*
> >   * An ID register that needs special handling to control the value for the
> >   * guest must have its own id_reg_info in id_reg_info_table.
> > @@ -869,6 +874,7 @@ static struct id_reg_info id_dfr0_el1_info = {
> >  #define      GET_ID_REG_INFO(id)     (id_reg_info_table[IDREG_IDX(id)])
> >  static struct id_reg_info *id_reg_info_table[KVM_ARM_ID_REG_MAX_NUM] = {
> >       [IDREG_IDX(SYS_ID_DFR0_EL1)] = &id_dfr0_el1_info,
> > +     [IDREG_IDX(SYS_ID_DFR1_EL1)] = &id_dfr1_el1_info,
> >       [IDREG_IDX(SYS_ID_AA64PFR0_EL1)] = &id_aa64pfr0_el1_info,
> >       [IDREG_IDX(SYS_ID_AA64PFR1_EL1)] = &id_aa64pfr1_el1_info,
> >       [IDREG_IDX(SYS_ID_AA64DFR0_EL1)] = &id_aa64dfr0_el1_info,
> >
>

Re: [RFC PATCH v3 03/29] KVM: arm64: Introduce struct id_reg_info

[Alexandru Elisei]

Hi Reiji,

On Tue, Nov 16, 2021 at 10:43:33PM -0800, Reiji Watanabe wrote:
> This patch lays the groundwork to make ID registers writable.
> 
> Introduce struct id_reg_info for an ID register to manage the
> register specific control of its value for the guest, and provide set
> of functions commonly used for ID registers to make them writable.
> 
> The id_reg_info is used to do register specific initialization,
> validation of the ID register and etc.  Not all ID registers must
> have the id_reg_info. ID registers that don't have the id_reg_info
> are handled in a common way that is applied to all ID registers.
> 
> At present, changing an ID register from userspace is allowed only
> if the ID register has the id_reg_info, but that will be changed
> by the following patches.
> 
> No ID register has the structure yet and the following patches
> will add the id_reg_info for some ID registers.
> 
> Signed-off-by: Reiji Watanabe <reijiw@google.com>
> ---
>  arch/arm64/include/asm/sysreg.h |   1 +
>  arch/arm64/kvm/sys_regs.c       | 226 ++++++++++++++++++++++++++++++--
>  2 files changed, 218 insertions(+), 9 deletions(-)
> 
> diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h
> index 16b3f1a1d468..597609f26331 100644
> --- a/arch/arm64/include/asm/sysreg.h
> +++ b/arch/arm64/include/asm/sysreg.h
> @@ -1197,6 +1197,7 @@
>  #define ICH_VTR_TDS_MASK	(1 << ICH_VTR_TDS_SHIFT)
>  
>  #define ARM64_FEATURE_FIELD_BITS	4
> +#define ARM64_FEATURE_FIELD_MASK	((1ull << ARM64_FEATURE_FIELD_BITS) - 1)
>  
>  /* Create a mask for the feature bits of the specified feature. */
>  #define ARM64_FEATURE_MASK(x)	(GENMASK_ULL(x##_SHIFT + ARM64_FEATURE_FIELD_BITS - 1, x##_SHIFT))
> diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
> index 5608d3410660..1552cd5581b7 100644
> --- a/arch/arm64/kvm/sys_regs.c
> +++ b/arch/arm64/kvm/sys_regs.c
> @@ -265,6 +265,181 @@ static bool trap_raz_wi(struct kvm_vcpu *vcpu,
>  		return read_zero(vcpu, p);
>  }
>  
> +/*
> + * A value for FCT_LOWER_SAFE must be zero and changing that will affect
> + * ftr_check_types of id_reg_info.
> + */
> +enum feature_check_type {
> +	FCT_LOWER_SAFE = 0,
> +	FCT_HIGHER_SAFE,
> +	FCT_HIGHER_OR_ZERO_SAFE,
> +	FCT_EXACT,
> +	FCT_EXACT_OR_ZERO_SAFE,
> +	FCT_IGNORE,	/* Don't check (any value is fine) */
> +};
> +
> +static int arm64_check_feature_one(enum feature_check_type type, int val,
> +				   int limit)
> +{
> +	bool is_safe = false;
> +
> +	if (val == limit)
> +		return 0;
> +
> +	switch (type) {
> +	case FCT_LOWER_SAFE:
> +		is_safe = (val <= limit);
> +		break;
> +	case FCT_HIGHER_OR_ZERO_SAFE:
> +		if (val == 0) {
> +			is_safe = true;
> +			break;
> +		}
> +		fallthrough;
> +	case FCT_HIGHER_SAFE:
> +		is_safe = (val >= limit);
> +		break;
> +	case FCT_EXACT:
> +		break;
> +	case FCT_EXACT_OR_ZERO_SAFE:
> +		is_safe = (val == 0);
> +		break;
> +	case FCT_IGNORE:

What happens if the a new feature is added and the field has a particular
meaning? How are you going to deal with old userspace implementations that
use a value here which now is not allowed or it affects the guest?

Thanks,
Alex

Re: [RFC PATCH v3 03/29] KVM: arm64: Introduce struct id_reg_info

[Alexandru Elisei]

Hi Reiji,

On Wed, Nov 24, 2021 at 09:27:32PM -0800, Reiji Watanabe wrote:
> On Sun, Nov 21, 2021 at 4:37 AM Marc Zyngier <maz@kernel.org> wrote:
> >
> > On Wed, 17 Nov 2021 06:43:33 +0000,
> > Reiji Watanabe <reijiw@google.com> wrote:
> > >
> > > This patch lays the groundwork to make ID registers writable.
> > >
> > > Introduce struct id_reg_info for an ID register to manage the
> > > register specific control of its value for the guest, and provide set
> > > of functions commonly used for ID registers to make them writable.
> > >
> > > The id_reg_info is used to do register specific initialization,
> > > validation of the ID register and etc.  Not all ID registers must
> > > have the id_reg_info. ID registers that don't have the id_reg_info
> > > are handled in a common way that is applied to all ID registers.
> > >
> > > At present, changing an ID register from userspace is allowed only
> > > if the ID register has the id_reg_info, but that will be changed
> > > by the following patches.
> > >
> > > No ID register has the structure yet and the following patches
> > > will add the id_reg_info for some ID registers.
> > >
> > > Signed-off-by: Reiji Watanabe <reijiw@google.com>
> > > ---
> > >  arch/arm64/include/asm/sysreg.h |   1 +
> > >  arch/arm64/kvm/sys_regs.c       | 226 ++++++++++++++++++++++++++++++--
> > >  2 files changed, 218 insertions(+), 9 deletions(-)
> > >
> > > diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h
> > > index 16b3f1a1d468..597609f26331 100644
> > > --- a/arch/arm64/include/asm/sysreg.h
> > > +++ b/arch/arm64/include/asm/sysreg.h
> > > @@ -1197,6 +1197,7 @@
> > >  #define ICH_VTR_TDS_MASK     (1 << ICH_VTR_TDS_SHIFT)
> > >
> > >  #define ARM64_FEATURE_FIELD_BITS     4
> > > +#define ARM64_FEATURE_FIELD_MASK     ((1ull << ARM64_FEATURE_FIELD_BITS) - 1)
> > >
> > >  /* Create a mask for the feature bits of the specified feature. */
> > >  #define ARM64_FEATURE_MASK(x)        (GENMASK_ULL(x##_SHIFT + ARM64_FEATURE_FIELD_BITS - 1, x##_SHIFT))
> > > diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
> > > index 5608d3410660..1552cd5581b7 100644
> > > --- a/arch/arm64/kvm/sys_regs.c
> > > +++ b/arch/arm64/kvm/sys_regs.c
> > > @@ -265,6 +265,181 @@ static bool trap_raz_wi(struct kvm_vcpu *vcpu,
> > >               return read_zero(vcpu, p);
> > >  }
> > >
> > > +/*
> > > + * A value for FCT_LOWER_SAFE must be zero and changing that will affect
> > > + * ftr_check_types of id_reg_info.
> > > + */
> > > +enum feature_check_type {
> > > +     FCT_LOWER_SAFE = 0,
> > > +     FCT_HIGHER_SAFE,
> > > +     FCT_HIGHER_OR_ZERO_SAFE,
> > > +     FCT_EXACT,
> > > +     FCT_EXACT_OR_ZERO_SAFE,
> > > +     FCT_IGNORE,     /* Don't check (any value is fine) */
> > > +};
> > > +
> > > +static int arm64_check_feature_one(enum feature_check_type type, int val,
> > > +                                int limit)
> > > +{
> > > +     bool is_safe = false;
> > > +
> > > +     if (val == limit)
> > > +             return 0;
> > > +
> > > +     switch (type) {
> > > +     case FCT_LOWER_SAFE:
> > > +             is_safe = (val <= limit);
> > > +             break;
> > > +     case FCT_HIGHER_OR_ZERO_SAFE:
> > > +             if (val == 0) {
> > > +                     is_safe = true;
> > > +                     break;
> > > +             }
> > > +             fallthrough;
> > > +     case FCT_HIGHER_SAFE:
> > > +             is_safe = (val >= limit);
> > > +             break;
> > > +     case FCT_EXACT:
> > > +             break;
> > > +     case FCT_EXACT_OR_ZERO_SAFE:
> > > +             is_safe = (val == 0);
> > > +             break;
> > > +     case FCT_IGNORE:
> > > +             is_safe = true;
> > > +             break;
> > > +     default:
> > > +             WARN_ONCE(1, "Unexpected feature_check_type (%d)\n", type);
> > > +             break;
> > > +     }
> > > +
> > > +     return is_safe ? 0 : -1;
> > > +}
> > > +
> > > +#define      FCT_TYPE_MASK           0x7
> > > +#define      FCT_TYPE_SHIFT          1
> > > +#define      FCT_SIGN_MASK           0x1
> > > +#define      FCT_SIGN_SHIFT          0
> > > +#define      FCT_TYPE(val)   ((val >> FCT_TYPE_SHIFT) & FCT_TYPE_MASK)
> > > +#define      FCT_SIGN(val)   ((val >> FCT_SIGN_SHIFT) & FCT_SIGN_MASK)
> > > +
> > > +#define      MAKE_FCT(shift, type, sign)                             \
> > > +     ((u64)((((type) & FCT_TYPE_MASK) << FCT_TYPE_SHIFT) |   \
> > > +            (((sign) & FCT_SIGN_MASK) << FCT_SIGN_SHIFT)) << (shift))
> > > +
> > > +/* For signed field */
> > > +#define      S_FCT(shift, type)      MAKE_FCT(shift, type, 1)
> > > +/* For unigned field */
> > > +#define      U_FCT(shift, type)      MAKE_FCT(shift, type, 0)
> > > +
> > > +/*
> > > + * @val and @lim are both a value of the ID register. The function checks
> > > + * if all features indicated in @val can be supported for guests on the host,
> > > + * which supports features indicated in @lim. @check_types indicates how
> > > + * features in the ID register needs to be checked.
> > > + * See comments for id_reg_info's ftr_check_types field for more detail.
> > > + */
> > > +static int arm64_check_features(u64 check_types, u64 val, u64 lim)
> > > +{
> > > +     int i;
> > > +
> > > +     for (i = 0; i < 64; i += ARM64_FEATURE_FIELD_BITS) {
> > > +             u8 ftr_check = (check_types >> i) & ARM64_FEATURE_FIELD_MASK;
> > > +             bool is_sign = FCT_SIGN(ftr_check);
> > > +             enum feature_check_type fctype = FCT_TYPE(ftr_check);
> > > +             int fval, flim, ret;
> > > +
> > > +             fval = cpuid_feature_extract_field(val, i, is_sign);
> > > +             flim = cpuid_feature_extract_field(lim, i, is_sign);
> > > +
> > > +             ret = arm64_check_feature_one(fctype, fval, flim);
> > > +             if (ret)
> > > +                     return -E2BIG;
> > > +     }
> > > +     return 0;
> > > +}
> >
> > All this logic seems to reinvent what we already have in
> > arch/arm64/kernel/cpufeature.c. I'd rather we rely on it and maintain
> > a single idreg handling library.
> >
> > Could you outline what is missing in the cpufeature code that requires
> > you to invent your own? I'm sure Suzuki could help here to make it
> > directly usable.
> 
> The issue is that there are some fields whose arm64_ftr_bits don't
> match what (I think) I need.  However, looking into that option again,
> it seems that the number of such fields are fewer than I originally
> thought (I misunderstood some earlier).
> 
> They are just three fields below.  The common checking process can be
> skipped for those fields (will restore ignore_mask field in id_reg_info
> as I had in v1 patch, which is treated like FCT_IGNORE in the v3 patch),
> and I will have their ID register specific validation function do
> what I want to check into the fields.
> 
>  - AA64DFR0.DEBUGVER:
>    Its .type is FTR_EXACT.
>    I want to treat its .type as FTR_LOWER_SAFE for the check.
> 
>  - AA64DFR0.PMUVER:
>    Its .sign is FTR_SIGNED and .type is FTR_EXACT.
>    I want to treat its .sign as FTR_UNSIGNED and .type as
>    FTR_LOWER_SAFE for the check.
> 
>  - DFR0.PERFMON:
>    Its .sign is FTR_SIGNED (Its .type is FTR_LOWER_SAFE).
>    I want to treat its .sign field as FTR_UNSIGNED for the check.
> 
>    (NOTE: For PMUVER and PERFMON, Arm ARM says "if the field value
>     is not 0xf the field is treated as an unsigned value")
> 

I don't think it's required that you use the same ID register field
definitions from cpufeature.c, you can create your own field definitions
for the KVM registers if they are different. But if you use the same
structs and field attributes from cpufeature.h, then you can reuse the
functions from cpufeature.c. I think that's what Marc was suggesting
(someone please correct me if I'm wrong).

The way Linux handles cpu features is already complicated, I think reusing
the same mechanism would be preferable from a maintenance and correctness
perspective. Unless you need something that is missing from cpu features
for which is unreasonable or impossible to add support.

Thanks,
Alex

Re: [RFC PATCH v3 09/29] KVM: arm64: Hide IMPLEMENTATION DEFINED PMU support for the guest

[Alexandru Elisei]

Hi Reiji,

On Mon, Nov 29, 2021 at 09:32:02PM -0800, Reiji Watanabe wrote:
> Hi Eric,
> 
> On Thu, Nov 25, 2021 at 12:30 PM Eric Auger <eauger@redhat.com> wrote:
> >
> > Hi Reiji,
> >
> > On 11/17/21 7:43 AM, Reiji Watanabe wrote:
> > > When ID_AA64DFR0_EL1.PMUVER or ID_DFR0_EL1.PERFMON is 0xf, which
> > > means IMPLEMENTATION DEFINED PMU supported, KVM unconditionally
> > > expose the value for the guest as it is.  Since KVM doesn't support
> > > IMPLEMENTATION DEFINED PMU for the guest, in that case KVM should
> > > exopse 0x0 (PMU is not implemented) instead.
> > s/exopse/expose
> > >
> > > Change cpuid_feature_cap_perfmon_field() to update the field value
> > > to 0x0 when it is 0xf.
> > is it wrong to expose the guest with a Perfmon value of 0xF? Then the
> > guest should not use it as a PMUv3?
> 
> > is it wrong to expose the guest with a Perfmon value of 0xF? Then the
> > guest should not use it as a PMUv3?
> 
> For the value 0xf in ID_AA64DFR0_EL1.PMUVER and ID_DFR0_EL1.PERFMON,
> Arm ARM says:
>   "IMPLEMENTATION DEFINED form of performance monitors supported,
>    PMUv3 not supported."
> 
> Since the PMU that KVM supports for guests is PMUv3, 0xf shouldn't
> be exposed to guests (And this patch series doesn't allow userspace
> to set the fields to 0xf for guests).

While it's true that a value of 0xf means that PMUv3 is not present (both
KVM and the PMU driver handle it this way) this is an userspace visible
change.

Are you sure there isn't software in the wild that relies on this value
being 0xf to detect that some non-Arm architected hardware is present?

Since both 0 and 0xf are valid values that mean that PMUv3 is not present,
I think it's best that both are kept.

Thanks,
Alex

> 
> Thanks,
> Reiji
> 
> >
> > Eric
> > >
> > > Fixes: 8e35aa642ee4 ("arm64: cpufeature: Extract capped perfmon fields")
> > > Signed-off-by: Reiji Watanabe <reijiw@google.com>
> > > ---
> > >  arch/arm64/include/asm/cpufeature.h | 2 +-
> > >  1 file changed, 1 insertion(+), 1 deletion(-)
> > >
> > > diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
> > > index ef6be92b1921..fd7ad8193827 100644
> > > --- a/arch/arm64/include/asm/cpufeature.h
> > > +++ b/arch/arm64/include/asm/cpufeature.h
> > > @@ -553,7 +553,7 @@ cpuid_feature_cap_perfmon_field(u64 features, int field, u64 cap)
> > >
> > >       /* Treat IMPLEMENTATION DEFINED functionality as unimplemented */
> > >       if (val == ID_AA64DFR0_PMUVER_IMP_DEF)
> > > -             val = 0;
> > > +             return (features & ~mask);
> > >
> > >       if (val > cap) {
> > >               features &= ~mask;
> > >
> >

Re: [RFC PATCH v3 09/29] KVM: arm64: Hide IMPLEMENTATION DEFINED PMU support for the guest

[Alexandru Elisei]

Hi Reiji,

On Wed, Dec 01, 2021 at 03:53:18PM +0000, Alexandru Elisei wrote:
> Hi Reiji,
> 
> On Mon, Nov 29, 2021 at 09:32:02PM -0800, Reiji Watanabe wrote:
> > Hi Eric,
> > 
> > On Thu, Nov 25, 2021 at 12:30 PM Eric Auger <eauger@redhat.com> wrote:
> > >
> > > Hi Reiji,
> > >
> > > On 11/17/21 7:43 AM, Reiji Watanabe wrote:
> > > > When ID_AA64DFR0_EL1.PMUVER or ID_DFR0_EL1.PERFMON is 0xf, which
> > > > means IMPLEMENTATION DEFINED PMU supported, KVM unconditionally
> > > > expose the value for the guest as it is.  Since KVM doesn't support
> > > > IMPLEMENTATION DEFINED PMU for the guest, in that case KVM should
> > > > exopse 0x0 (PMU is not implemented) instead.
> > > s/exopse/expose
> > > >
> > > > Change cpuid_feature_cap_perfmon_field() to update the field value
> > > > to 0x0 when it is 0xf.
> > > is it wrong to expose the guest with a Perfmon value of 0xF? Then the
> > > guest should not use it as a PMUv3?
> > 
> > > is it wrong to expose the guest with a Perfmon value of 0xF? Then the
> > > guest should not use it as a PMUv3?
> > 
> > For the value 0xf in ID_AA64DFR0_EL1.PMUVER and ID_DFR0_EL1.PERFMON,
> > Arm ARM says:
> >   "IMPLEMENTATION DEFINED form of performance monitors supported,
> >    PMUv3 not supported."
> > 
> > Since the PMU that KVM supports for guests is PMUv3, 0xf shouldn't
> > be exposed to guests (And this patch series doesn't allow userspace
> > to set the fields to 0xf for guests).
> 
> While it's true that a value of 0xf means that PMUv3 is not present (both
> KVM and the PMU driver handle it this way) this is an userspace visible
> change.
> 
> Are you sure there isn't software in the wild that relies on this value
> being 0xf to detect that some non-Arm architected hardware is present?
> 
> Since both 0 and 0xf are valid values that mean that PMUv3 is not present,
> I think it's best that both are kept.

Sorry, somehow I managed to get myself confused and didn't realize that
this is only used by KVM.

What I said above about the possibility of software existing that pokes IMP
DEF registers when PMUVer = 0xf is in fact a good argument for this patch,
because KVM injects an undefined exception when a guest tries to access
such registers.

Thanks,
Alex

> 
> Thanks,
> Alex
> 
> > 
> > Thanks,
> > Reiji
> > 
> > >
> > > Eric
> > > >
> > > > Fixes: 8e35aa642ee4 ("arm64: cpufeature: Extract capped perfmon fields")
> > > > Signed-off-by: Reiji Watanabe <reijiw@google.com>
> > > > ---
> > > >  arch/arm64/include/asm/cpufeature.h | 2 +-
> > > >  1 file changed, 1 insertion(+), 1 deletion(-)
> > > >
> > > > diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
> > > > index ef6be92b1921..fd7ad8193827 100644
> > > > --- a/arch/arm64/include/asm/cpufeature.h
> > > > +++ b/arch/arm64/include/asm/cpufeature.h
> > > > @@ -553,7 +553,7 @@ cpuid_feature_cap_perfmon_field(u64 features, int field, u64 cap)
> > > >
> > > >       /* Treat IMPLEMENTATION DEFINED functionality as unimplemented */
> > > >       if (val == ID_AA64DFR0_PMUVER_IMP_DEF)
> > > > -             val = 0;
> > > > +             return (features & ~mask);
> > > >
> > > >       if (val > cap) {
> > > >               features &= ~mask;
> > > >
> > >

Re: [RFC PATCH v3 03/29] KVM: arm64: Introduce struct id_reg_info

[Reiji Watanabe]

Hi Alex,

On Wed, Dec 1, 2021 at 7:24 AM Alexandru Elisei
<alexandru.elisei@arm.com> wrote:
>
> Hi Reiji,
>
> On Tue, Nov 16, 2021 at 10:43:33PM -0800, Reiji Watanabe wrote:
> > This patch lays the groundwork to make ID registers writable.
> >
> > Introduce struct id_reg_info for an ID register to manage the
> > register specific control of its value for the guest, and provide set
> > of functions commonly used for ID registers to make them writable.
> >
> > The id_reg_info is used to do register specific initialization,
> > validation of the ID register and etc.  Not all ID registers must
> > have the id_reg_info. ID registers that don't have the id_reg_info
> > are handled in a common way that is applied to all ID registers.
> >
> > At present, changing an ID register from userspace is allowed only
> > if the ID register has the id_reg_info, but that will be changed
> > by the following patches.
> >
> > No ID register has the structure yet and the following patches
> > will add the id_reg_info for some ID registers.
> >
> > Signed-off-by: Reiji Watanabe <reijiw@google.com>
> > ---
> >  arch/arm64/include/asm/sysreg.h |   1 +
> >  arch/arm64/kvm/sys_regs.c       | 226 ++++++++++++++++++++++++++++++--
> >  2 files changed, 218 insertions(+), 9 deletions(-)
> >
> > diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h
> > index 16b3f1a1d468..597609f26331 100644
> > --- a/arch/arm64/include/asm/sysreg.h
> > +++ b/arch/arm64/include/asm/sysreg.h
> > @@ -1197,6 +1197,7 @@
> >  #define ICH_VTR_TDS_MASK     (1 << ICH_VTR_TDS_SHIFT)
> >
> >  #define ARM64_FEATURE_FIELD_BITS     4
> > +#define ARM64_FEATURE_FIELD_MASK     ((1ull << ARM64_FEATURE_FIELD_BITS) - 1)
> >
> >  /* Create a mask for the feature bits of the specified feature. */
> >  #define ARM64_FEATURE_MASK(x)        (GENMASK_ULL(x##_SHIFT + ARM64_FEATURE_FIELD_BITS - 1, x##_SHIFT))
> > diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
> > index 5608d3410660..1552cd5581b7 100644
> > --- a/arch/arm64/kvm/sys_regs.c
> > +++ b/arch/arm64/kvm/sys_regs.c
> > @@ -265,6 +265,181 @@ static bool trap_raz_wi(struct kvm_vcpu *vcpu,
> >               return read_zero(vcpu, p);
> >  }
> >
> > +/*
> > + * A value for FCT_LOWER_SAFE must be zero and changing that will affect
> > + * ftr_check_types of id_reg_info.
> > + */
> > +enum feature_check_type {
> > +     FCT_LOWER_SAFE = 0,
> > +     FCT_HIGHER_SAFE,
> > +     FCT_HIGHER_OR_ZERO_SAFE,
> > +     FCT_EXACT,
> > +     FCT_EXACT_OR_ZERO_SAFE,
> > +     FCT_IGNORE,     /* Don't check (any value is fine) */
> > +};
> > +
> > +static int arm64_check_feature_one(enum feature_check_type type, int val,
> > +                                int limit)
> > +{
> > +     bool is_safe = false;
> > +
> > +     if (val == limit)
> > +             return 0;
> > +
> > +     switch (type) {
> > +     case FCT_LOWER_SAFE:
> > +             is_safe = (val <= limit);
> > +             break;
> > +     case FCT_HIGHER_OR_ZERO_SAFE:
> > +             if (val == 0) {
> > +                     is_safe = true;
> > +                     break;
> > +             }
> > +             fallthrough;
> > +     case FCT_HIGHER_SAFE:
> > +             is_safe = (val >= limit);
> > +             break;
> > +     case FCT_EXACT:
> > +             break;
> > +     case FCT_EXACT_OR_ZERO_SAFE:
> > +             is_safe = (val == 0);
> > +             break;
> > +     case FCT_IGNORE:
>
> What happens if the a new feature is added and the field has a particular
> meaning? How are you going to deal with old userspace implementations that
> use a value here which now is not allowed or it affects the guest?

With this v3 series, unless KVM is changed for the new field,
a new feature will be treated as lower safe (that's the default).
If the field won't fit any of those cases, FCT_IGNORE needs to be
used for the field, and the ID register specific validation function,
which will be registered in id_reg_info, needs to validate the field.

Old userspace implementation shouldn't be affected because the default
values (the values right after the first KVM_ARM_VCPU_INIT) for
ID registers won't be changed by this series (patch-9 changes
AA64DFR0.PMUVER/DFR0.PERFMON but it is due to a bug fix), and the
default value, which is basically same as @limit (or indicates
less or smaller level of features than @limit for features that
can be configured by KVM_ARM_VCPU_INIT, etc), is always allowed
by arm64_check_feature_one().

Having said that, arm64_check_feature_one() will be gone from the next
version, and the similar checking will be done by a new function in
arch/arm64/kernel/cpufeature.c that will use arm64_ftr_bits instead.

  https://lore.kernel.org/all/CAAeT=FxwzRF0YZmmoEmq3xRHnhun-BCx_FeEQrOVLgzwseSy4w@mail.gmail.com/

Unless KVM is changed for the new field, it will be validated based
on arm64_ftr_bits for the field.  If KVM needs to handle the field
differently, then we will have the new function ignore the field,
and will have the ID register specific validation function handle
the field.

Thanks,
Reiji

Re: [RFC PATCH v3 03/29] KVM: arm64: Introduce struct id_reg_info

[Reiji Watanabe]

Hi Alex,

On Wed, Dec 1, 2021 at 7:39 AM Alexandru Elisei
<alexandru.elisei@arm.com> wrote:
>
> Hi Reiji,
>
> On Wed, Nov 24, 2021 at 09:27:32PM -0800, Reiji Watanabe wrote:
> > On Sun, Nov 21, 2021 at 4:37 AM Marc Zyngier <maz@kernel.org> wrote:
> > >
> > > On Wed, 17 Nov 2021 06:43:33 +0000,
> > > Reiji Watanabe <reijiw@google.com> wrote:
> > > >
> > > > This patch lays the groundwork to make ID registers writable.
> > > >
> > > > Introduce struct id_reg_info for an ID register to manage the
> > > > register specific control of its value for the guest, and provide set
> > > > of functions commonly used for ID registers to make them writable.
> > > >
> > > > The id_reg_info is used to do register specific initialization,
> > > > validation of the ID register and etc.  Not all ID registers must
> > > > have the id_reg_info. ID registers that don't have the id_reg_info
> > > > are handled in a common way that is applied to all ID registers.
> > > >
> > > > At present, changing an ID register from userspace is allowed only
> > > > if the ID register has the id_reg_info, but that will be changed
> > > > by the following patches.
> > > >
> > > > No ID register has the structure yet and the following patches
> > > > will add the id_reg_info for some ID registers.
> > > >
> > > > Signed-off-by: Reiji Watanabe <reijiw@google.com>
> > > > ---
> > > >  arch/arm64/include/asm/sysreg.h |   1 +
> > > >  arch/arm64/kvm/sys_regs.c       | 226 ++++++++++++++++++++++++++++++--
> > > >  2 files changed, 218 insertions(+), 9 deletions(-)
> > > >
> > > > diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h
> > > > index 16b3f1a1d468..597609f26331 100644
> > > > --- a/arch/arm64/include/asm/sysreg.h
> > > > +++ b/arch/arm64/include/asm/sysreg.h
> > > > @@ -1197,6 +1197,7 @@
> > > >  #define ICH_VTR_TDS_MASK     (1 << ICH_VTR_TDS_SHIFT)
> > > >
> > > >  #define ARM64_FEATURE_FIELD_BITS     4
> > > > +#define ARM64_FEATURE_FIELD_MASK     ((1ull << ARM64_FEATURE_FIELD_BITS) - 1)
> > > >
> > > >  /* Create a mask for the feature bits of the specified feature. */
> > > >  #define ARM64_FEATURE_MASK(x)        (GENMASK_ULL(x##_SHIFT + ARM64_FEATURE_FIELD_BITS - 1, x##_SHIFT))
> > > > diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
> > > > index 5608d3410660..1552cd5581b7 100644
> > > > --- a/arch/arm64/kvm/sys_regs.c
> > > > +++ b/arch/arm64/kvm/sys_regs.c
> > > > @@ -265,6 +265,181 @@ static bool trap_raz_wi(struct kvm_vcpu *vcpu,
> > > >               return read_zero(vcpu, p);
> > > >  }
> > > >
> > > > +/*
> > > > + * A value for FCT_LOWER_SAFE must be zero and changing that will affect
> > > > + * ftr_check_types of id_reg_info.
> > > > + */
> > > > +enum feature_check_type {
> > > > +     FCT_LOWER_SAFE = 0,
> > > > +     FCT_HIGHER_SAFE,
> > > > +     FCT_HIGHER_OR_ZERO_SAFE,
> > > > +     FCT_EXACT,
> > > > +     FCT_EXACT_OR_ZERO_SAFE,
> > > > +     FCT_IGNORE,     /* Don't check (any value is fine) */
> > > > +};
> > > > +
> > > > +static int arm64_check_feature_one(enum feature_check_type type, int val,
> > > > +                                int limit)
> > > > +{
> > > > +     bool is_safe = false;
> > > > +
> > > > +     if (val == limit)
> > > > +             return 0;
> > > > +
> > > > +     switch (type) {
> > > > +     case FCT_LOWER_SAFE:
> > > > +             is_safe = (val <= limit);
> > > > +             break;
> > > > +     case FCT_HIGHER_OR_ZERO_SAFE:
> > > > +             if (val == 0) {
> > > > +                     is_safe = true;
> > > > +                     break;
> > > > +             }
> > > > +             fallthrough;
> > > > +     case FCT_HIGHER_SAFE:
> > > > +             is_safe = (val >= limit);
> > > > +             break;
> > > > +     case FCT_EXACT:
> > > > +             break;
> > > > +     case FCT_EXACT_OR_ZERO_SAFE:
> > > > +             is_safe = (val == 0);
> > > > +             break;
> > > > +     case FCT_IGNORE:
> > > > +             is_safe = true;
> > > > +             break;
> > > > +     default:
> > > > +             WARN_ONCE(1, "Unexpected feature_check_type (%d)\n", type);
> > > > +             break;
> > > > +     }
> > > > +
> > > > +     return is_safe ? 0 : -1;
> > > > +}
> > > > +
> > > > +#define      FCT_TYPE_MASK           0x7
> > > > +#define      FCT_TYPE_SHIFT          1
> > > > +#define      FCT_SIGN_MASK           0x1
> > > > +#define      FCT_SIGN_SHIFT          0
> > > > +#define      FCT_TYPE(val)   ((val >> FCT_TYPE_SHIFT) & FCT_TYPE_MASK)
> > > > +#define      FCT_SIGN(val)   ((val >> FCT_SIGN_SHIFT) & FCT_SIGN_MASK)
> > > > +
> > > > +#define      MAKE_FCT(shift, type, sign)                             \
> > > > +     ((u64)((((type) & FCT_TYPE_MASK) << FCT_TYPE_SHIFT) |   \
> > > > +            (((sign) & FCT_SIGN_MASK) << FCT_SIGN_SHIFT)) << (shift))
> > > > +
> > > > +/* For signed field */
> > > > +#define      S_FCT(shift, type)      MAKE_FCT(shift, type, 1)
> > > > +/* For unigned field */
> > > > +#define      U_FCT(shift, type)      MAKE_FCT(shift, type, 0)
> > > > +
> > > > +/*
> > > > + * @val and @lim are both a value of the ID register. The function checks
> > > > + * if all features indicated in @val can be supported for guests on the host,
> > > > + * which supports features indicated in @lim. @check_types indicates how
> > > > + * features in the ID register needs to be checked.
> > > > + * See comments for id_reg_info's ftr_check_types field for more detail.
> > > > + */
> > > > +static int arm64_check_features(u64 check_types, u64 val, u64 lim)
> > > > +{
> > > > +     int i;
> > > > +
> > > > +     for (i = 0; i < 64; i += ARM64_FEATURE_FIELD_BITS) {
> > > > +             u8 ftr_check = (check_types >> i) & ARM64_FEATURE_FIELD_MASK;
> > > > +             bool is_sign = FCT_SIGN(ftr_check);
> > > > +             enum feature_check_type fctype = FCT_TYPE(ftr_check);
> > > > +             int fval, flim, ret;
> > > > +
> > > > +             fval = cpuid_feature_extract_field(val, i, is_sign);
> > > > +             flim = cpuid_feature_extract_field(lim, i, is_sign);
> > > > +
> > > > +             ret = arm64_check_feature_one(fctype, fval, flim);
> > > > +             if (ret)
> > > > +                     return -E2BIG;
> > > > +     }
> > > > +     return 0;
> > > > +}
> > >
> > > All this logic seems to reinvent what we already have in
> > > arch/arm64/kernel/cpufeature.c. I'd rather we rely on it and maintain
> > > a single idreg handling library.
> > >
> > > Could you outline what is missing in the cpufeature code that requires
> > > you to invent your own? I'm sure Suzuki could help here to make it
> > > directly usable.
> >
> > The issue is that there are some fields whose arm64_ftr_bits don't
> > match what (I think) I need.  However, looking into that option again,
> > it seems that the number of such fields are fewer than I originally
> > thought (I misunderstood some earlier).
> >
> > They are just three fields below.  The common checking process can be
> > skipped for those fields (will restore ignore_mask field in id_reg_info
> > as I had in v1 patch, which is treated like FCT_IGNORE in the v3 patch),
> > and I will have their ID register specific validation function do
> > what I want to check into the fields.
> >
> >  - AA64DFR0.DEBUGVER:
> >    Its .type is FTR_EXACT.
> >    I want to treat its .type as FTR_LOWER_SAFE for the check.
> >
> >  - AA64DFR0.PMUVER:
> >    Its .sign is FTR_SIGNED and .type is FTR_EXACT.
> >    I want to treat its .sign as FTR_UNSIGNED and .type as
> >    FTR_LOWER_SAFE for the check.
> >
> >  - DFR0.PERFMON:
> >    Its .sign is FTR_SIGNED (Its .type is FTR_LOWER_SAFE).
> >    I want to treat its .sign field as FTR_UNSIGNED for the check.
> >
> >    (NOTE: For PMUVER and PERFMON, Arm ARM says "if the field value
> >     is not 0xf the field is treated as an unsigned value")
> >
>
> I don't think it's required that you use the same ID register field
> definitions from cpufeature.c, you can create your own field definitions
> for the KVM registers if they are different. But if you use the same
> structs and field attributes from cpufeature.h, then you can reuse the
> functions from cpufeature.c. I think that's what Marc was suggesting
> (someone please correct me if I'm wrong).
>
> The way Linux handles cpu features is already complicated, I think reusing
> the same mechanism would be preferable from a maintenance and correctness
> perspective. Unless you need something that is missing from cpu features
> for which is unreasonable or impossible to add support.

Thank you for your comments ! I will explore a way that is easy to
maintain for both cpufeature.c and KVM.  Since almost all ID register
fields can be handled in the same way as cpufeature.c, I will probably
use the same ID register fields though.

Thanks,
Reiji

Re: [RFC PATCH v3 09/29] KVM: arm64: Hide IMPLEMENTATION DEFINED PMU support for the guest

[Reiji Watanabe]

Hi Alex,

On Wed, Dec 1, 2021 at 8:09 AM Alexandru Elisei
<alexandru.elisei@arm.com> wrote:
>
> Hi Reiji,
>
> On Wed, Dec 01, 2021 at 03:53:18PM +0000, Alexandru Elisei wrote:
> > Hi Reiji,
> >
> > On Mon, Nov 29, 2021 at 09:32:02PM -0800, Reiji Watanabe wrote:
> > > Hi Eric,
> > >
> > > On Thu, Nov 25, 2021 at 12:30 PM Eric Auger <eauger@redhat.com> wrote:
> > > >
> > > > Hi Reiji,
> > > >
> > > > On 11/17/21 7:43 AM, Reiji Watanabe wrote:
> > > > > When ID_AA64DFR0_EL1.PMUVER or ID_DFR0_EL1.PERFMON is 0xf, which
> > > > > means IMPLEMENTATION DEFINED PMU supported, KVM unconditionally
> > > > > expose the value for the guest as it is.  Since KVM doesn't support
> > > > > IMPLEMENTATION DEFINED PMU for the guest, in that case KVM should
> > > > > exopse 0x0 (PMU is not implemented) instead.
> > > > s/exopse/expose
> > > > >
> > > > > Change cpuid_feature_cap_perfmon_field() to update the field value
> > > > > to 0x0 when it is 0xf.
> > > > is it wrong to expose the guest with a Perfmon value of 0xF? Then the
> > > > guest should not use it as a PMUv3?
> > >
> > > > is it wrong to expose the guest with a Perfmon value of 0xF? Then the
> > > > guest should not use it as a PMUv3?
> > >
> > > For the value 0xf in ID_AA64DFR0_EL1.PMUVER and ID_DFR0_EL1.PERFMON,
> > > Arm ARM says:
> > >   "IMPLEMENTATION DEFINED form of performance monitors supported,
> > >    PMUv3 not supported."
> > >
> > > Since the PMU that KVM supports for guests is PMUv3, 0xf shouldn't
> > > be exposed to guests (And this patch series doesn't allow userspace
> > > to set the fields to 0xf for guests).
> >
> > While it's true that a value of 0xf means that PMUv3 is not present (both
> > KVM and the PMU driver handle it this way) this is an userspace visible
> > change.
> >
> > Are you sure there isn't software in the wild that relies on this value
> > being 0xf to detect that some non-Arm architected hardware is present?
> >
> > Since both 0 and 0xf are valid values that mean that PMUv3 is not present,
> > I think it's best that both are kept.
>
> Sorry, somehow I managed to get myself confused and didn't realize that
> this is only used by KVM.
>
> What I said above about the possibility of software existing that pokes IMP
> DEF registers when PMUVer = 0xf is in fact a good argument for this patch,
> because KVM injects an undefined exception when a guest tries to access
> such registers.

Thank you for your review.  My understanding is the same as yours.

Thanks,
Reiji

Re: [RFC PATCH v3 09/29] KVM: arm64: Hide IMPLEMENTATION DEFINED PMU support for the guest

[Eric Auger]

Hi Reiji,

On 11/30/21 6:32 AM, Reiji Watanabe wrote:
> Hi Eric,
> 
> On Thu, Nov 25, 2021 at 12:30 PM Eric Auger <eauger@redhat.com> wrote:
>>
>> Hi Reiji,
>>
>> On 11/17/21 7:43 AM, Reiji Watanabe wrote:
>>> When ID_AA64DFR0_EL1.PMUVER or ID_DFR0_EL1.PERFMON is 0xf, which
>>> means IMPLEMENTATION DEFINED PMU supported, KVM unconditionally
>>> expose the value for the guest as it is.  Since KVM doesn't support
>>> IMPLEMENTATION DEFINED PMU for the guest, in that case KVM should
>>> exopse 0x0 (PMU is not implemented) instead.
>> s/exopse/expose
>>>
>>> Change cpuid_feature_cap_perfmon_field() to update the field value
>>> to 0x0 when it is 0xf.
>> is it wrong to expose the guest with a Perfmon value of 0xF? Then the
>> guest should not use it as a PMUv3?
> 
>> is it wrong to expose the guest with a Perfmon value of 0xF? Then the
>> guest should not use it as a PMUv3?
> 
> For the value 0xf in ID_AA64DFR0_EL1.PMUVER and ID_DFR0_EL1.PERFMON,
> Arm ARM says:
>   "IMPLEMENTATION DEFINED form of performance monitors supported,
>    PMUv3 not supported."
> 
> Since the PMU that KVM supports for guests is PMUv3, 0xf shouldn't
> be exposed to guests (And this patch series doesn't allow userspace
> to set the fields to 0xf for guests).
What I don't get is why this isn't detected before (in kvm_reset_vcpu).
if the VCPU was initialized with KVM_ARM_VCPU_PMU_V3 can we honor this
init request if the host pmu is implementation defined?

Thanks

Eric
> 
> Thanks,
> Reiji
> 
>>
>> Eric
>>>
>>> Fixes: 8e35aa642ee4 ("arm64: cpufeature: Extract capped perfmon fields")
>>> Signed-off-by: Reiji Watanabe <reijiw@google.com>
>>> ---
>>>  arch/arm64/include/asm/cpufeature.h | 2 +-
>>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>>
>>> diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
>>> index ef6be92b1921..fd7ad8193827 100644
>>> --- a/arch/arm64/include/asm/cpufeature.h
>>> +++ b/arch/arm64/include/asm/cpufeature.h
>>> @@ -553,7 +553,7 @@ cpuid_feature_cap_perfmon_field(u64 features, int field, u64 cap)
>>>
>>>       /* Treat IMPLEMENTATION DEFINED functionality as unimplemented */
>>>       if (val == ID_AA64DFR0_PMUVER_IMP_DEF)
>>> -             val = 0;
>>> +             return (features & ~mask);
>>>
>>>       if (val > cap) {
>>>               features &= ~mask;
>>>
>>
> _______________________________________________
> kvmarm mailing list
> kvmarm@lists.cs.columbia.edu
> https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
> 

Re: [RFC PATCH v3 02/29] KVM: arm64: Save ID registers' sanitized value per vCPU

[Eric Auger]

Hi Reiji,

On 11/17/21 7:43 AM, Reiji Watanabe wrote:
> Extend sys_regs[] of kvm_cpu_context for ID registers and save ID
> registers' sanitized value in the array for the vCPU at the first
> vCPU reset. Use the saved ones when ID registers are read by
> userspace (via KVM_GET_ONE_REG) or the guest.
> 
> Signed-off-by: Reiji Watanabe <reijiw@google.com>
> ---
>  arch/arm64/include/asm/kvm_host.h | 10 +++++++
>  arch/arm64/kvm/sys_regs.c         | 43 +++++++++++++++++++------------
>  2 files changed, 37 insertions(+), 16 deletions(-)
> 
> diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
> index edbe2cb21947..72db73c79403 100644
> --- a/arch/arm64/include/asm/kvm_host.h
> +++ b/arch/arm64/include/asm/kvm_host.h
> @@ -146,6 +146,14 @@ struct kvm_vcpu_fault_info {
>  	u64 disr_el1;		/* Deferred [SError] Status Register */
>  };
>  
> +/*
> + * (Op0, Op1, CRn, CRm, Op2) of ID registers is (3, 0, 0, crm, op2),
> + * where 0<=crm<8, 0<=op2<8.
> + */
> +#define KVM_ARM_ID_REG_MAX_NUM 64
> +#define IDREG_IDX(id)		((sys_reg_CRm(id) << 3) | sys_reg_Op2(id))
> +#define IDREG_SYS_IDX(id)	(ID_REG_BASE + IDREG_IDX(id))
> +
>  enum vcpu_sysreg {
>  	__INVALID_SYSREG__,   /* 0 is reserved as an invalid value */
>  	MPIDR_EL1,	/* MultiProcessor Affinity Register */
> @@ -210,6 +218,8 @@ enum vcpu_sysreg {
>  	CNTP_CVAL_EL0,
>  	CNTP_CTL_EL0,
>  
> +	ID_REG_BASE,
> +	ID_REG_END = ID_REG_BASE + KVM_ARM_ID_REG_MAX_NUM - 1,
>  	/* Memory Tagging Extension registers */
>  	RGSR_EL1,	/* Random Allocation Tag Seed Register */
>  	GCR_EL1,	/* Tag Control Register */
> diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
> index e3ec1a44f94d..5608d3410660 100644
> --- a/arch/arm64/kvm/sys_regs.c
> +++ b/arch/arm64/kvm/sys_regs.c
> @@ -33,6 +33,8 @@
>  
>  #include "trace.h"
>  
> +static u64 __read_id_reg(const struct kvm_vcpu *vcpu, u32 id);
> +
>  /*
>   * All of this file is extremely similar to the ARM coproc.c, but the
>   * types are different. My gut feeling is that it should be pretty
> @@ -273,7 +275,7 @@ static bool trap_loregion(struct kvm_vcpu *vcpu,
>  			  struct sys_reg_params *p,
>  			  const struct sys_reg_desc *r)
>  {
> -	u64 val = read_sanitised_ftr_reg(SYS_ID_AA64MMFR1_EL1);
> +	u64 val = __read_id_reg(vcpu, SYS_ID_AA64MMFR1_EL1);
>  	u32 sr = reg_to_encoding(r);
>  
>  	if (!(val & (0xfUL << ID_AA64MMFR1_LOR_SHIFT))) {
> @@ -1059,17 +1061,9 @@ static bool access_arch_timer(struct kvm_vcpu *vcpu,
>  	return true;
>  }
>  
> -/* Read a sanitised cpufeature ID register by sys_reg_desc */
> -static u64 read_id_reg(const struct kvm_vcpu *vcpu,
> -		struct sys_reg_desc const *r, bool raz)
> +static u64 __read_id_reg(const struct kvm_vcpu *vcpu, u32 id)
>  {
> -	u32 id = reg_to_encoding(r);
> -	u64 val;
> -
> -	if (raz)
> -		return 0;
> -
> -	val = read_sanitised_ftr_reg(id);
> +	u64 val = __vcpu_sys_reg(vcpu, IDREG_SYS_IDX(id));
>  
>  	switch (id) {
>  	case SYS_ID_AA64PFR0_EL1:
> @@ -1119,6 +1113,14 @@ static u64 read_id_reg(const struct kvm_vcpu *vcpu,
>  	return val;
>  }
>  
> +static u64 read_id_reg(const struct kvm_vcpu *vcpu,
> +		       struct sys_reg_desc const *r, bool raz)
> +{
> +	u32 id = reg_to_encoding(r);
> +
> +	return raz ? 0 : __read_id_reg(vcpu, id);
> +}
> +
>  static unsigned int id_visibility(const struct kvm_vcpu *vcpu,
>  				  const struct sys_reg_desc *r)
>  {
> @@ -1178,6 +1180,16 @@ static unsigned int sve_visibility(const struct kvm_vcpu *vcpu,
>  	return REG_HIDDEN;
>  }
>  
> +static void reset_id_reg(struct kvm_vcpu *vcpu, const struct sys_reg_desc *rd)
> +{
> +	u32 id = reg_to_encoding(rd);
> +
> +	if (vcpu_has_reset_once(vcpu))
> +		return;
The KVM API allows to call VCPU_INIT several times (with same
target/feature). With above check on the second call the ID_REGS won't
be reset. Somehow this is aligned with target/feature behavior. However
if this is what we want, I think we would need to document it in the KVM
API doc.

Thanks

Eric
> +
> +	__vcpu_sys_reg(vcpu, IDREG_SYS_IDX(id)) = read_sanitised_ftr_reg(id);
> +}
> +
>  static int set_id_aa64pfr0_el1(struct kvm_vcpu *vcpu,
>  			       const struct sys_reg_desc *rd,
>  			       const struct kvm_one_reg *reg, void __user *uaddr)
> @@ -1223,9 +1235,7 @@ static int set_id_aa64pfr0_el1(struct kvm_vcpu *vcpu,
>  /*
>   * cpufeature ID register user accessors
>   *
> - * For now, these registers are immutable for userspace, so no values
> - * are stored, and for set_id_reg() we don't allow the effective value
> - * to be changed.
> + * We don't allow the effective value to be changed.
>   */
>  static int __get_id_reg(const struct kvm_vcpu *vcpu,
>  			const struct sys_reg_desc *rd, void __user *uaddr,
> @@ -1382,6 +1392,7 @@ static unsigned int mte_visibility(const struct kvm_vcpu *vcpu,
>  #define ID_SANITISED(name) {			\
>  	SYS_DESC(SYS_##name),			\
>  	.access	= access_id_reg,		\
> +	.reset	= reset_id_reg,			\
>  	.get_user = get_id_reg,			\
>  	.set_user = set_id_reg,			\
>  	.visibility = id_visibility,		\
> @@ -1837,8 +1848,8 @@ static bool trap_dbgdidr(struct kvm_vcpu *vcpu,
>  	if (p->is_write) {
>  		return ignore_write(vcpu, p);
>  	} else {
> -		u64 dfr = read_sanitised_ftr_reg(SYS_ID_AA64DFR0_EL1);
> -		u64 pfr = read_sanitised_ftr_reg(SYS_ID_AA64PFR0_EL1);
> +		u64 dfr = __read_id_reg(vcpu, SYS_ID_AA64DFR0_EL1);
> +		u64 pfr = __read_id_reg(vcpu, SYS_ID_AA64PFR0_EL1);
>  		u32 el3 = !!cpuid_feature_extract_unsigned_field(pfr, ID_AA64PFR0_EL3_SHIFT);
>  
>  		p->regval = ((((dfr >> ID_AA64DFR0_WRPS_SHIFT) & 0xf) << 28) |
> 

Re: [RFC PATCH v3 03/29] KVM: arm64: Introduce struct id_reg_info

[Eric Auger]

On 11/25/21 7:40 AM, Reiji Watanabe wrote:
> Hi Eric,
> 
> On Wed, Nov 24, 2021 at 1:07 PM Eric Auger <eauger@redhat.com> wrote:
>>
>> Hi Reiji,
>>
>> On 11/17/21 7:43 AM, Reiji Watanabe wrote:
>>> This patch lays the groundwork to make ID registers writable.
>>>
>>> Introduce struct id_reg_info for an ID register to manage the
>>> register specific control of its value for the guest, and provide set
>>> of functions commonly used for ID registers to make them writable.
>>>
>>> The id_reg_info is used to do register specific initialization,
>>> validation of the ID register and etc.  Not all ID registers must
>>> have the id_reg_info. ID registers that don't have the id_reg_info
>>> are handled in a common way that is applied to all ID registers.
>>>
>>> At present, changing an ID register from userspace is allowed only
>>> if the ID register has the id_reg_info, but that will be changed
>>> by the following patches.
>>>
>>> No ID register has the structure yet and the following patches
>>> will add the id_reg_info for some ID registers.
>>>
>>> Signed-off-by: Reiji Watanabe <reijiw@google.com>
>>> ---
>>>  arch/arm64/include/asm/sysreg.h |   1 +
>>>  arch/arm64/kvm/sys_regs.c       | 226 ++++++++++++++++++++++++++++++--
>>>  2 files changed, 218 insertions(+), 9 deletions(-)
>>>
>>> diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h
>>> index 16b3f1a1d468..597609f26331 100644
>>> --- a/arch/arm64/include/asm/sysreg.h
>>> +++ b/arch/arm64/include/asm/sysreg.h
>>> @@ -1197,6 +1197,7 @@
>>>  #define ICH_VTR_TDS_MASK     (1 << ICH_VTR_TDS_SHIFT)
>>>
>>>  #define ARM64_FEATURE_FIELD_BITS     4
>>> +#define ARM64_FEATURE_FIELD_MASK     ((1ull << ARM64_FEATURE_FIELD_BITS) - 1)
>>>
>>>  /* Create a mask for the feature bits of the specified feature. */
>>>  #define ARM64_FEATURE_MASK(x)        (GENMASK_ULL(x##_SHIFT + ARM64_FEATURE_FIELD_BITS - 1, x##_SHIFT))
>>> diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
>>> index 5608d3410660..1552cd5581b7 100644
>>> --- a/arch/arm64/kvm/sys_regs.c
>>> +++ b/arch/arm64/kvm/sys_regs.c
>>> @@ -265,6 +265,181 @@ static bool trap_raz_wi(struct kvm_vcpu *vcpu,
>>>               return read_zero(vcpu, p);
>>>  }
>>>
>>> +/*
>>> + * A value for FCT_LOWER_SAFE must be zero and changing that will affect
>>> + * ftr_check_types of id_reg_info.
>>> + */
>>> +enum feature_check_type {
>>> +     FCT_LOWER_SAFE = 0,
>>> +     FCT_HIGHER_SAFE,
>>> +     FCT_HIGHER_OR_ZERO_SAFE,
>>> +     FCT_EXACT,
>>> +     FCT_EXACT_OR_ZERO_SAFE,
>>> +     FCT_IGNORE,     /* Don't check (any value is fine) */
>>> +};
>>> +
>>> +static int arm64_check_feature_one(enum feature_check_type type, int val,
>>> +                                int limit)
>>> +{
>>> +     bool is_safe = false;
>>> +
>>> +     if (val == limit)
>>> +             return 0;
>>> +
>>> +     switch (type) {
>>> +     case FCT_LOWER_SAFE:
>>> +             is_safe = (val <= limit);
>>> +             break;
>>> +     case FCT_HIGHER_OR_ZERO_SAFE:
>>> +             if (val == 0) {
>>> +                     is_safe = true;
>>> +                     break;
>>> +             }
>>> +             fallthrough;
>>> +     case FCT_HIGHER_SAFE:
>>> +             is_safe = (val >= limit);
>>> +             break;
>>> +     case FCT_EXACT:
>>> +             break;
>>> +     case FCT_EXACT_OR_ZERO_SAFE:
>>> +             is_safe = (val == 0);
>>> +             break;
>>> +     case FCT_IGNORE:
>>> +             is_safe = true;
>>> +             break;
>>> +     default:
>>> +             WARN_ONCE(1, "Unexpected feature_check_type (%d)\n", type);
>>> +             break;
>>> +     }
>>> +
>>> +     return is_safe ? 0 : -1;
>>> +}
>>> +
>>> +#define      FCT_TYPE_MASK           0x7
>>> +#define      FCT_TYPE_SHIFT          1
>>> +#define      FCT_SIGN_MASK           0x1
>>> +#define      FCT_SIGN_SHIFT          0
>>> +#define      FCT_TYPE(val)   ((val >> FCT_TYPE_SHIFT) & FCT_TYPE_MASK)
>>> +#define      FCT_SIGN(val)   ((val >> FCT_SIGN_SHIFT) & FCT_SIGN_MASK)
>>> +
>>> +#define      MAKE_FCT(shift, type, sign)                             \
>>> +     ((u64)((((type) & FCT_TYPE_MASK) << FCT_TYPE_SHIFT) |   \
>>> +            (((sign) & FCT_SIGN_MASK) << FCT_SIGN_SHIFT)) << (shift))
>>> +
>>> +/* For signed field */
>>> +#define      S_FCT(shift, type)      MAKE_FCT(shift, type, 1)
>>> +/* For unigned field */
>>> +#define      U_FCT(shift, type)      MAKE_FCT(shift, type, 0)
>>> +
>>> +/*
>>> + * @val and @lim are both a value of the ID register. The function checks
>>> + * if all features indicated in @val can be supported for guests on the host,
>>> + * which supports features indicated in @lim. @check_types indicates how
>>> + * features in the ID register needs to be checked.
>>> + * See comments for id_reg_info's ftr_check_types field for more detail.
>>> + */
>>> +static int arm64_check_features(u64 check_types, u64 val, u64 lim)
>>> +{
>>> +     int i;
>>> +
>>> +     for (i = 0; i < 64; i += ARM64_FEATURE_FIELD_BITS) {
>>> +             u8 ftr_check = (check_types >> i) & ARM64_FEATURE_FIELD_MASK;
>>> +             bool is_sign = FCT_SIGN(ftr_check);
>>> +             enum feature_check_type fctype = FCT_TYPE(ftr_check);
>>> +             int fval, flim, ret;
>>> +
>>> +             fval = cpuid_feature_extract_field(val, i, is_sign);
>>> +             flim = cpuid_feature_extract_field(lim, i, is_sign);
>>> +
>>> +             ret = arm64_check_feature_one(fctype, fval, flim);
>>> +             if (ret)
>>> +                     return -E2BIG;
>> nit: -EINVAL may be better because depending on the check type this may
>> not mean too big.
> 
> Yes, that is correct.
> 
> This error case means that userspace tried to configure features
> or a higher level of features that were not supported on the host.
> In that sense, I chose -E2BIG.
> 
> I wanted to use an error code specific to this particular case, which
> I think makes debugging userspace issue easier when KVM_SET_ONE_REG
> fails, and I couldn't find other error codes that fit this case better.
> So, I'm trying to avoid using -EINVAL, which is used for other failure
> cases.
> 
> If you have any other suggested error code for this,
> that would be very helpful:)

OK faire enought, that's a nit anyway

Eric
> 
> Thanks,
> Reiji
> 

Re: [RFC PATCH v3 03/29] KVM: arm64: Introduce struct id_reg_info

[Eric Auger]

Hi Reiji,

On 11/17/21 7:43 AM, Reiji Watanabe wrote:
> This patch lays the groundwork to make ID registers writable.
> 
> Introduce struct id_reg_info for an ID register to manage the
> register specific control of its value for the guest, and provide set
> of functions commonly used for ID registers to make them writable.
> 
> The id_reg_info is used to do register specific initialization,
> validation of the ID register and etc.  Not all ID registers must
> have the id_reg_info. ID registers that don't have the id_reg_info
> are handled in a common way that is applied to all ID registers.
> 
> At present, changing an ID register from userspace is allowed only
> if the ID register has the id_reg_info, but that will be changed
> by the following patches.
> 
> No ID register has the structure yet and the following patches
> will add the id_reg_info for some ID registers.
> 
> Signed-off-by: Reiji Watanabe <reijiw@google.com>
> ---
>  arch/arm64/include/asm/sysreg.h |   1 +
>  arch/arm64/kvm/sys_regs.c       | 226 ++++++++++++++++++++++++++++++--
>  2 files changed, 218 insertions(+), 9 deletions(-)
> 
> diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h
> index 16b3f1a1d468..597609f26331 100644
> --- a/arch/arm64/include/asm/sysreg.h
> +++ b/arch/arm64/include/asm/sysreg.h
> @@ -1197,6 +1197,7 @@
>  #define ICH_VTR_TDS_MASK	(1 << ICH_VTR_TDS_SHIFT)
>  
>  #define ARM64_FEATURE_FIELD_BITS	4
> +#define ARM64_FEATURE_FIELD_MASK	((1ull << ARM64_FEATURE_FIELD_BITS) - 1)
>  
>  /* Create a mask for the feature bits of the specified feature. */
>  #define ARM64_FEATURE_MASK(x)	(GENMASK_ULL(x##_SHIFT + ARM64_FEATURE_FIELD_BITS - 1, x##_SHIFT))
> diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
> index 5608d3410660..1552cd5581b7 100644
> --- a/arch/arm64/kvm/sys_regs.c
> +++ b/arch/arm64/kvm/sys_regs.c
> @@ -265,6 +265,181 @@ static bool trap_raz_wi(struct kvm_vcpu *vcpu,
>  		return read_zero(vcpu, p);
>  }
>  
> +/*
> + * A value for FCT_LOWER_SAFE must be zero and changing that will affect
> + * ftr_check_types of id_reg_info.
> + */
> +enum feature_check_type {
> +	FCT_LOWER_SAFE = 0,
> +	FCT_HIGHER_SAFE,
> +	FCT_HIGHER_OR_ZERO_SAFE,
> +	FCT_EXACT,
> +	FCT_EXACT_OR_ZERO_SAFE,
> +	FCT_IGNORE,	/* Don't check (any value is fine) */
> +};
> +
> +static int arm64_check_feature_one(enum feature_check_type type, int val,
> +				   int limit)
> +{
> +	bool is_safe = false;
> +
> +	if (val == limit)
> +		return 0;
> +
> +	switch (type) {
> +	case FCT_LOWER_SAFE:
> +		is_safe = (val <= limit);
> +		break;
> +	case FCT_HIGHER_OR_ZERO_SAFE:
> +		if (val == 0) {
> +			is_safe = true;
> +			break;
> +		}
> +		fallthrough;
> +	case FCT_HIGHER_SAFE:
> +		is_safe = (val >= limit);
> +		break;
> +	case FCT_EXACT:
> +		break;
> +	case FCT_EXACT_OR_ZERO_SAFE:
> +		is_safe = (val == 0);
> +		break;
> +	case FCT_IGNORE:
> +		is_safe = true;
> +		break;
> +	default:
> +		WARN_ONCE(1, "Unexpected feature_check_type (%d)\n", type);
> +		break;
> +	}
> +
> +	return is_safe ? 0 : -1;
> +}
> +
> +#define	FCT_TYPE_MASK		0x7
> +#define	FCT_TYPE_SHIFT		1
> +#define	FCT_SIGN_MASK		0x1
> +#define	FCT_SIGN_SHIFT		0
> +#define	FCT_TYPE(val)	((val >> FCT_TYPE_SHIFT) & FCT_TYPE_MASK)
> +#define	FCT_SIGN(val)	((val >> FCT_SIGN_SHIFT) & FCT_SIGN_MASK)
> +
> +#define	MAKE_FCT(shift, type, sign)				\
> +	((u64)((((type) & FCT_TYPE_MASK) << FCT_TYPE_SHIFT) |	\
> +	       (((sign) & FCT_SIGN_MASK) << FCT_SIGN_SHIFT)) << (shift))
> +
> +/* For signed field */
> +#define	S_FCT(shift, type)	MAKE_FCT(shift, type, 1)
> +/* For unigned field */
> +#define	U_FCT(shift, type)	MAKE_FCT(shift, type, 0)
> +
> +/*
> + * @val and @lim are both a value of the ID register. The function checks
> + * if all features indicated in @val can be supported for guests on the host,
> + * which supports features indicated in @lim. @check_types indicates how
> + * features in the ID register needs to be checked.
> + * See comments for id_reg_info's ftr_check_types field for more detail.
> + */
> +static int arm64_check_features(u64 check_types, u64 val, u64 lim)
> +{
> +	int i;
> +
> +	for (i = 0; i < 64; i += ARM64_FEATURE_FIELD_BITS) {
> +		u8 ftr_check = (check_types >> i) & ARM64_FEATURE_FIELD_MASK;
> +		bool is_sign = FCT_SIGN(ftr_check);
> +		enum feature_check_type fctype = FCT_TYPE(ftr_check);
> +		int fval, flim, ret;
> +
> +		fval = cpuid_feature_extract_field(val, i, is_sign);
> +		flim = cpuid_feature_extract_field(lim, i, is_sign);
> +
> +		ret = arm64_check_feature_one(fctype, fval, flim);
> +		if (ret)
> +			return -E2BIG;
> +	}
> +	return 0;
> +}
> +
> +struct id_reg_info {
> +	u32	sys_reg;	/* Register ID */
> +
> +	/*
> +	 * Limit value of the register for a vcpu. The value is the sanitized
> +	 * system value with bits cleared for unsupported features for the
> +	 * guest.
> +	 */
> +	u64	vcpu_limit_val;
> +
> +	/*
> +	 * The ftr_check_types is comprised of a set of 4 bits fields.
> +	 * Each 4 bits field is for a feature indicated by the same bits
> +	 * field of the ID register and indicates how the feature support
> +	 * for guests needs to be checked.
> +	 * The bit 0 indicates that the corresponding ID register field
> +	 * is signed(1) or unsigned(0).
> +	 * The bits [3:1] hold feature_check_type for the field.
> +	 * If all zero, all features in the ID register are treated as unsigned
> +	 * fields and checked based on Principles of the ID scheme for fields
> +	 * in ID registers (FCT_LOWER_SAFE of feature_check_type).
> +	 */
> +	u64	ftr_check_types;
> +
> +	/* Initialization function of the id_reg_info */
> +	void (*init)(struct id_reg_info *id_reg);
> +
> +	/* Register specific validation function */
> +	int (*validate)(struct kvm_vcpu *vcpu, const struct id_reg_info *id_reg,
> +			u64 val);
> +
> +	/* Return the reset value of the register for the vCPU */
> +	u64 (*get_reset_val)(struct kvm_vcpu *vcpu,
> +			     const struct id_reg_info *id_reg);
It is unclear to me why we need 2 different callbacks, ie. init and
get_reset_val. ID_REGS can only be accessed from user space after the
vcpu reset, right? So couldn't we have a single cb instead of this
overwrite mechanism?

Thanks

Eric
> +};
> +
> +static void id_reg_info_init(struct id_reg_info *id_reg)
> +{
> +	id_reg->vcpu_limit_val = read_sanitised_ftr_reg(id_reg->sys_reg);
> +	if (id_reg->init)
> +		id_reg->init(id_reg);
> +}
> +
> +/*
> + * An ID register that needs special handling to control the value for the
> + * guest must have its own id_reg_info in id_reg_info_table.
> + * (i.e. the reset value is different from the host's sanitized value,
> + * the value is affected by opt-in features, some fields needs specific
> + * validation, etc.)
> + */
> +#define	GET_ID_REG_INFO(id)	(id_reg_info_table[IDREG_IDX(id)])
> +static struct id_reg_info *id_reg_info_table[KVM_ARM_ID_REG_MAX_NUM] = {};
> +
> +static int validate_id_reg(struct kvm_vcpu *vcpu,
> +			   const struct sys_reg_desc *rd, u64 val)
> +{
> +	u32 id = reg_to_encoding(rd);
> +	const struct id_reg_info *id_reg = GET_ID_REG_INFO(id);
> +	u64 limit, check_types;
> +	int err;
> +
> +	if (id_reg) {
> +		check_types = id_reg->ftr_check_types;
> +		limit = id_reg->vcpu_limit_val;
> +	} else {
> +		/* All fields are treated as unsigned and FCT_LOWER_SAFE */
> +		check_types = 0;
> +		limit = read_sanitised_ftr_reg(id);
> +	}
> +
> +	/* Check if the value indicates any feature that is not in the limit. */
> +	err = arm64_check_features(check_types, val, limit);
> +	if (err)
> +		return err;
> +
> +	if (id_reg && id_reg->validate)
> +		/* Run the ID register specific validity check. */
> +		err = id_reg->validate(vcpu, id_reg, val);
> +
> +	return err;
> +}
> +
>  /*
>   * ARMv8.1 mandates at least a trivial LORegion implementation, where all the
>   * RW registers are RES0 (which we can implement as RAZ/WI). On an ARMv8.0
> @@ -1183,11 +1358,19 @@ static unsigned int sve_visibility(const struct kvm_vcpu *vcpu,
>  static void reset_id_reg(struct kvm_vcpu *vcpu, const struct sys_reg_desc *rd)
>  {
>  	u32 id = reg_to_encoding(rd);
> +	struct id_reg_info *id_reg;
> +	u64 val;
>  
>  	if (vcpu_has_reset_once(vcpu))
>  		return;
>  
> -	__vcpu_sys_reg(vcpu, IDREG_SYS_IDX(id)) = read_sanitised_ftr_reg(id);
> +	id_reg = GET_ID_REG_INFO(id);
> +	if (id_reg && id_reg->get_reset_val)
> +		val = id_reg->get_reset_val(vcpu, id_reg);
> +	else
> +		val = read_sanitised_ftr_reg(id);
> +
> +	__vcpu_sys_reg(vcpu, IDREG_SYS_IDX(id)) = val;
>  }
>  
>  static int set_id_aa64pfr0_el1(struct kvm_vcpu *vcpu,
> @@ -1232,11 +1415,7 @@ static int set_id_aa64pfr0_el1(struct kvm_vcpu *vcpu,
>  	return 0;
>  }
>  
> -/*
> - * cpufeature ID register user accessors
> - *
> - * We don't allow the effective value to be changed.
> - */
> +/* cpufeature ID register user accessors */
>  static int __get_id_reg(const struct kvm_vcpu *vcpu,
>  			const struct sys_reg_desc *rd, void __user *uaddr,
>  			bool raz)
> @@ -1247,11 +1426,12 @@ static int __get_id_reg(const struct kvm_vcpu *vcpu,
>  	return reg_to_user(uaddr, &val, id);
>  }
>  
> -static int __set_id_reg(const struct kvm_vcpu *vcpu,
> +static int __set_id_reg(struct kvm_vcpu *vcpu,
>  			const struct sys_reg_desc *rd, void __user *uaddr,
>  			bool raz)
>  {
>  	const u64 id = sys_reg_to_index(rd);
> +	u32 encoding = reg_to_encoding(rd);
>  	int err;
>  	u64 val;
>  
> @@ -1259,10 +1439,22 @@ static int __set_id_reg(const struct kvm_vcpu *vcpu,
>  	if (err)
>  		return err;
>  
> -	/* This is what we mean by invariant: you can't change it. */
> -	if (val != read_id_reg(vcpu, rd, raz))
> +	/* Don't allow to change the reg unless the reg has id_reg_info */
> +	if (val != read_id_reg(vcpu, rd, raz) && !GET_ID_REG_INFO(encoding))
>  		return -EINVAL;
>  
> +	if (raz)
> +		return 0;
> +
> +	/* Don't allow to change the reg after the first KVM_RUN. */
> +	if (vcpu->arch.has_run_once)
> +		return -EINVAL;
> +
> +	err = validate_id_reg(vcpu, rd, val);
> +	if (err)
> +		return err;
> +
> +	__vcpu_sys_reg(vcpu, IDREG_SYS_IDX(encoding)) = val;
>  	return 0;
>  }
>  
> @@ -2826,6 +3018,20 @@ int kvm_arm_copy_sys_reg_indices(struct kvm_vcpu *vcpu, u64 __user *uindices)
>  	return write_demux_regids(uindices);
>  }
>  
> +static void id_reg_info_init_all(void)
> +{
> +	int i;
> +	struct id_reg_info *id_reg;
> +
> +	for (i = 0; i < ARRAY_SIZE(id_reg_info_table); i++) {
> +		id_reg = (struct id_reg_info *)id_reg_info_table[i];
> +		if (!id_reg)
> +			continue;
> +
> +		id_reg_info_init(id_reg);
> +	}
> +}
> +
>  void kvm_sys_reg_table_init(void)
>  {
>  	unsigned int i;
> @@ -2860,4 +3066,6 @@ void kvm_sys_reg_table_init(void)
>  			break;
>  	/* Clear all higher bits. */
>  	cache_levels &= (1 << (i*3))-1;
> +
> +	id_reg_info_init_all();
>  }
> 

Re: [RFC PATCH v3 04/29] KVM: arm64: Make ID_AA64PFR0_EL1 writable

[Eric Auger]

Hi Reiji,

On 11/30/21 2:29 AM, Reiji Watanabe wrote:
> Hi Eric,
> 
> On Thu, Nov 25, 2021 at 7:35 AM Eric Auger <eauger@redhat.com> wrote:
>>
>> Hi Reiji,
>>
>> On 11/17/21 7:43 AM, Reiji Watanabe wrote:
>>> This patch adds id_reg_info for ID_AA64PFR0_EL1 to make it writable by
>>> userspace.
>>>
>>> The CSV2/CSV3 fields of the register were already writable and values
>>> that were written for them affected all vCPUs before. Now they only
>>> affect the vCPU.
>>> Return an error if userspace tries to set SVE/GIC field of the register
>>> to a value that conflicts with SVE/GIC configuration for the guest.
>>> SIMD/FP/SVE fields of the requested value are validated according to
>>> Arm ARM.
>>>
>>> Signed-off-by: Reiji Watanabe <reijiw@google.com>
>>> ---
>>>  arch/arm64/kvm/sys_regs.c | 159 ++++++++++++++++++++++++--------------
>>>  1 file changed, 103 insertions(+), 56 deletions(-)
>>>
>>> diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
>>> index 1552cd5581b7..35400869067a 100644
>>> --- a/arch/arm64/kvm/sys_regs.c
>>> +++ b/arch/arm64/kvm/sys_regs.c
>>> @@ -401,6 +401,92 @@ static void id_reg_info_init(struct id_reg_info *id_reg)
>>>               id_reg->init(id_reg);
>>>  }
>>>
>>> +#define      kvm_has_gic3(kvm)               \
>>> +     (irqchip_in_kernel(kvm) &&      \
>>> +      (kvm)->arch.vgic.vgic_model == KVM_DEV_TYPE_ARM_VGIC_V3)
>> you may move this macro to kvm/arm_vgic.h as this may be used in
>> vgic/vgic-v3.c too
> 
> Thank you for the suggestion. I will move that to kvm/arm_vgic.h.
> 
> 
>>> +
>>> +static int validate_id_aa64pfr0_el1(struct kvm_vcpu *vcpu,
>>> +                                 const struct id_reg_info *id_reg, u64 val)
>>> +{
>>> +     int fp, simd;
>>> +     bool vcpu_has_sve = vcpu_has_sve(vcpu);
>>> +     bool pfr0_has_sve = id_aa64pfr0_sve(val);
>>> +     int gic;
>>> +
>>> +     simd = cpuid_feature_extract_signed_field(val, ID_AA64PFR0_ASIMD_SHIFT);
>>> +     fp = cpuid_feature_extract_signed_field(val, ID_AA64PFR0_FP_SHIFT);
>>> +     if (simd != fp)
>>> +             return -EINVAL;
>>> +
>>> +     /* fp must be supported when sve is supported */
>>> +     if (pfr0_has_sve && (fp < 0))
>>> +             return -EINVAL;
>>> +
>>> +     /* Check if there is a conflict with a request via KVM_ARM_VCPU_INIT */
>>> +     if (vcpu_has_sve ^ pfr0_has_sve)
>>> +             return -EPERM;
>>> +
>>> +     gic = cpuid_feature_extract_unsigned_field(val, ID_AA64PFR0_GIC_SHIFT);
>>> +     if ((gic > 0) ^ kvm_has_gic3(vcpu->kvm))
>>> +             return -EPERM;
>>
>> Sometimes from a given architecture version, some lower values are not
>> allowed. For instance from ARMv8.5 onlt 1 is permitted for CSV3.
>> Shouldn't we handle that kind of check?
> 
> As far as I know, there is no way for guests to identify the
> architecture revision (e.g. v8.1, v8.2, etc).  It might be able
> to indirectly infer the revision though (from features that are
> available or etc).

OK. That sounds weird to me as we do many checks accross different IDREG
settings but we may eventually have a wrong "CPU model" exposed by the
user space violating those spec revision minima. Shouldn't we introduce
some way for the userspace to provide his requirements? via new VCPU
targets for instance?

Thanks

Eric
> 
> 
>>> +
>>> +     return 0;
>>> +}
>>> +
>>> +static void init_id_aa64pfr0_el1_info(struct id_reg_info *id_reg)
>>> +{
>>> +     u64 limit = id_reg->vcpu_limit_val;
>>> +     unsigned int gic;
>>> +
>>> +     limit &= ~ARM64_FEATURE_MASK(ID_AA64PFR0_AMU);
>>> +     if (!system_supports_sve())
>>> +             limit &= ~ARM64_FEATURE_MASK(ID_AA64PFR0_SVE);
>>> +
>>> +     /*
>>> +      * The default is to expose CSV2 == 1 and CSV3 == 1 if the HW
>>> +      * isn't affected.  Userspace can override this as long as it
>>> +      * doesn't promise the impossible.
>>> +      */
>>> +     limit &= ~(ARM64_FEATURE_MASK(ID_AA64PFR0_CSV2) |
>>> +                ARM64_FEATURE_MASK(ID_AA64PFR0_CSV3));
>>> +
>>> +     if (arm64_get_spectre_v2_state() == SPECTRE_UNAFFECTED)
>>> +             limit |= FIELD_PREP(ARM64_FEATURE_MASK(ID_AA64PFR0_CSV2), 1);
>>> +     if (arm64_get_meltdown_state() == SPECTRE_UNAFFECTED)
>>> +             limit |= FIELD_PREP(ARM64_FEATURE_MASK(ID_AA64PFR0_CSV3), 1);
>>> +
>>> +     gic = cpuid_feature_extract_unsigned_field(limit, ID_AA64PFR0_GIC_SHIFT);
>>> +     if (gic > 1) {
>>> +             /* Limit to GICv3.0/4.0 */
>>> +             limit &= ~ARM64_FEATURE_MASK(ID_AA64PFR0_GIC);
>>> +             limit |= FIELD_PREP(ARM64_FEATURE_MASK(ID_AA64PFR0_GIC), 1);
>>> +     }
>>> +     id_reg->vcpu_limit_val = limit;
>>> +}
>>> +
>>> +static u64 get_reset_id_aa64pfr0_el1(struct kvm_vcpu *vcpu,
>>> +                                  const struct id_reg_info *idr)
>>> +{
>>> +     u64 val = idr->vcpu_limit_val;
>>> +
>>> +     if (!vcpu_has_sve(vcpu))
>>> +             val &= ~ARM64_FEATURE_MASK(ID_AA64PFR0_SVE);
>>> +
>>> +     if (!kvm_has_gic3(vcpu->kvm))
>>> +             val &= ~ARM64_FEATURE_MASK(ID_AA64PFR0_GIC);
>>> +
>>> +     return val;
>>> +}
>>> +
>>> +static struct id_reg_info id_aa64pfr0_el1_info = {
>>> +     .sys_reg = SYS_ID_AA64PFR0_EL1,
>>> +     .ftr_check_types = S_FCT(ID_AA64PFR0_ASIMD_SHIFT, FCT_LOWER_SAFE) |
>>> +                        S_FCT(ID_AA64PFR0_FP_SHIFT, FCT_LOWER_SAFE),
>> is it needed as it is the default?
> 
>>> +     .ftr_check_types = S_FCT(ID_AA64PFR0_ASIMD_SHIFT, FCT_LOWER_SAFE) |
>>> +                        S_FCT(ID_AA64PFR0_FP_SHIFT, FCT_LOWER_SAFE),
>> is it needed as it is the default?
> 
> They are needed because they are signed fields (the default is unsigned

Ah OK, I did not catch it at first glance while looking at the ARM ARM.

Thanks

Eric

> and FCT_LOWER_SAFE).  Having said that, ftr_check_types itself will be
> gone in the next version (as arm64_ftr_bits will be used instead).
> 
> Thanks,
> Reiji
> 

Re: [RFC PATCH v3 12/29] KVM: arm64: Make ID_DFR1_EL1 writable

[Eric Auger]

On 11/30/21 6:39 AM, Reiji Watanabe wrote:
> Hi Eric,
> 
> On Thu, Nov 25, 2021 at 12:30 PM Eric Auger <eauger@redhat.com> wrote:
>>
>> Hi Reiji,
>>
>> On 11/17/21 7:43 AM, Reiji Watanabe wrote:
>>> This patch adds id_reg_info for ID_DFR1_EL1 to make it writable
>>> by userspace.
>>>
>>> Signed-off-by: Reiji Watanabe <reijiw@google.com>
>>> ---
>>>  arch/arm64/kvm/sys_regs.c | 6 ++++++
>>>  1 file changed, 6 insertions(+)
>>>
>>> diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
>>> index fbd335ac5e6b..dda7001959f6 100644
>>> --- a/arch/arm64/kvm/sys_regs.c
>>> +++ b/arch/arm64/kvm/sys_regs.c
>>> @@ -859,6 +859,11 @@ static struct id_reg_info id_dfr0_el1_info = {
>>>       .get_reset_val = get_reset_id_dfr0_el1,
>>>  };
>>>
>>> +static struct id_reg_info id_dfr1_el1_info = {
>>> +     .sys_reg = SYS_ID_DFR1_EL1,
>>> +     .ftr_check_types = S_FCT(ID_DFR1_MTPMU_SHIFT, FCT_LOWER_SAFE),
>> what about the 0xF value which indicates the MTPMU is not implemented?
> 
> The field is treated as a signed field.
> So, 0xf(== -1) is handled correctly.
> (Does it answer your question?)

yes thanks

Eric
> 
> Thanks,
> Reiji
> 
>>
>> Eric
>>> +};
>>> +
>>>  /*
>>>   * An ID register that needs special handling to control the value for the
>>>   * guest must have its own id_reg_info in id_reg_info_table.
>>> @@ -869,6 +874,7 @@ static struct id_reg_info id_dfr0_el1_info = {
>>>  #define      GET_ID_REG_INFO(id)     (id_reg_info_table[IDREG_IDX(id)])
>>>  static struct id_reg_info *id_reg_info_table[KVM_ARM_ID_REG_MAX_NUM] = {
>>>       [IDREG_IDX(SYS_ID_DFR0_EL1)] = &id_dfr0_el1_info,
>>> +     [IDREG_IDX(SYS_ID_DFR1_EL1)] = &id_dfr1_el1_info,
>>>       [IDREG_IDX(SYS_ID_AA64PFR0_EL1)] = &id_aa64pfr0_el1_info,
>>>       [IDREG_IDX(SYS_ID_AA64PFR1_EL1)] = &id_aa64pfr1_el1_info,
>>>       [IDREG_IDX(SYS_ID_AA64DFR0_EL1)] = &id_aa64dfr0_el1_info,
>>>
>>
> 

Re: [RFC PATCH v3 09/29] KVM: arm64: Hide IMPLEMENTATION DEFINED PMU support for the guest

[Reiji Watanabe]

Hi Eric,

On Thu, Dec 2, 2021 at 2:57 AM Eric Auger <eauger@redhat.com> wrote:
>
> Hi Reiji,
>
> On 11/30/21 6:32 AM, Reiji Watanabe wrote:
> > Hi Eric,
> >
> > On Thu, Nov 25, 2021 at 12:30 PM Eric Auger <eauger@redhat.com> wrote:
> >>
> >> Hi Reiji,
> >>
> >> On 11/17/21 7:43 AM, Reiji Watanabe wrote:
> >>> When ID_AA64DFR0_EL1.PMUVER or ID_DFR0_EL1.PERFMON is 0xf, which
> >>> means IMPLEMENTATION DEFINED PMU supported, KVM unconditionally
> >>> expose the value for the guest as it is.  Since KVM doesn't support
> >>> IMPLEMENTATION DEFINED PMU for the guest, in that case KVM should
> >>> exopse 0x0 (PMU is not implemented) instead.
> >> s/exopse/expose
> >>>
> >>> Change cpuid_feature_cap_perfmon_field() to update the field value
> >>> to 0x0 when it is 0xf.
> >> is it wrong to expose the guest with a Perfmon value of 0xF? Then the
> >> guest should not use it as a PMUv3?
> >
> >> is it wrong to expose the guest with a Perfmon value of 0xF? Then the
> >> guest should not use it as a PMUv3?
> >
> > For the value 0xf in ID_AA64DFR0_EL1.PMUVER and ID_DFR0_EL1.PERFMON,
> > Arm ARM says:
> >   "IMPLEMENTATION DEFINED form of performance monitors supported,
> >    PMUv3 not supported."
> >
> > Since the PMU that KVM supports for guests is PMUv3, 0xf shouldn't
> > be exposed to guests (And this patch series doesn't allow userspace
> > to set the fields to 0xf for guests).
> What I don't get is why this isn't detected before (in kvm_reset_vcpu).
> if the VCPU was initialized with KVM_ARM_VCPU_PMU_V3 can we honor this
> init request if the host pmu is implementation defined?

KVM_ARM_VCPU_INIT with KVM_ARM_VCPU_PMU_V3 will fail in
kvm_reset_vcpu() if the host PMU is implementation defined.

The AA64DFR0 and DFR0 registers for a vCPU without KVM_ARM_VCPU_PMU_V3
indicates IMPLEMENTATION DEFINED PMU support, which is not correct.

Thanks,
Reiji

Re: [RFC PATCH v3 02/29] KVM: arm64: Save ID registers' sanitized value per vCPU

[Reiji Watanabe]

Hi Eric,

On Thu, Dec 2, 2021 at 2:58 AM Eric Auger <eauger@redhat.com> wrote:
>
> Hi Reiji,
>
> On 11/17/21 7:43 AM, Reiji Watanabe wrote:
> > Extend sys_regs[] of kvm_cpu_context for ID registers and save ID
> > registers' sanitized value in the array for the vCPU at the first
> > vCPU reset. Use the saved ones when ID registers are read by
> > userspace (via KVM_GET_ONE_REG) or the guest.
> >
> > Signed-off-by: Reiji Watanabe <reijiw@google.com>
> > ---
> >  arch/arm64/include/asm/kvm_host.h | 10 +++++++
> >  arch/arm64/kvm/sys_regs.c         | 43 +++++++++++++++++++------------
> >  2 files changed, 37 insertions(+), 16 deletions(-)
> >
> > diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
> > index edbe2cb21947..72db73c79403 100644
> > --- a/arch/arm64/include/asm/kvm_host.h
> > +++ b/arch/arm64/include/asm/kvm_host.h
> > @@ -146,6 +146,14 @@ struct kvm_vcpu_fault_info {
> >       u64 disr_el1;           /* Deferred [SError] Status Register */
> >  };
> >
> > +/*
> > + * (Op0, Op1, CRn, CRm, Op2) of ID registers is (3, 0, 0, crm, op2),
> > + * where 0<=crm<8, 0<=op2<8.
> > + */
> > +#define KVM_ARM_ID_REG_MAX_NUM 64
> > +#define IDREG_IDX(id)                ((sys_reg_CRm(id) << 3) | sys_reg_Op2(id))
> > +#define IDREG_SYS_IDX(id)    (ID_REG_BASE + IDREG_IDX(id))
> > +
> >  enum vcpu_sysreg {
> >       __INVALID_SYSREG__,   /* 0 is reserved as an invalid value */
> >       MPIDR_EL1,      /* MultiProcessor Affinity Register */
> > @@ -210,6 +218,8 @@ enum vcpu_sysreg {
> >       CNTP_CVAL_EL0,
> >       CNTP_CTL_EL0,
> >
> > +     ID_REG_BASE,
> > +     ID_REG_END = ID_REG_BASE + KVM_ARM_ID_REG_MAX_NUM - 1,
> >       /* Memory Tagging Extension registers */
> >       RGSR_EL1,       /* Random Allocation Tag Seed Register */
> >       GCR_EL1,        /* Tag Control Register */
> > diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
> > index e3ec1a44f94d..5608d3410660 100644
> > --- a/arch/arm64/kvm/sys_regs.c
> > +++ b/arch/arm64/kvm/sys_regs.c
> > @@ -33,6 +33,8 @@
> >
> >  #include "trace.h"
> >
> > +static u64 __read_id_reg(const struct kvm_vcpu *vcpu, u32 id);
> > +
> >  /*
> >   * All of this file is extremely similar to the ARM coproc.c, but the
> >   * types are different. My gut feeling is that it should be pretty
> > @@ -273,7 +275,7 @@ static bool trap_loregion(struct kvm_vcpu *vcpu,
> >                         struct sys_reg_params *p,
> >                         const struct sys_reg_desc *r)
> >  {
> > -     u64 val = read_sanitised_ftr_reg(SYS_ID_AA64MMFR1_EL1);
> > +     u64 val = __read_id_reg(vcpu, SYS_ID_AA64MMFR1_EL1);
> >       u32 sr = reg_to_encoding(r);
> >
> >       if (!(val & (0xfUL << ID_AA64MMFR1_LOR_SHIFT))) {
> > @@ -1059,17 +1061,9 @@ static bool access_arch_timer(struct kvm_vcpu *vcpu,
> >       return true;
> >  }
> >
> > -/* Read a sanitised cpufeature ID register by sys_reg_desc */
> > -static u64 read_id_reg(const struct kvm_vcpu *vcpu,
> > -             struct sys_reg_desc const *r, bool raz)
> > +static u64 __read_id_reg(const struct kvm_vcpu *vcpu, u32 id)
> >  {
> > -     u32 id = reg_to_encoding(r);
> > -     u64 val;
> > -
> > -     if (raz)
> > -             return 0;
> > -
> > -     val = read_sanitised_ftr_reg(id);
> > +     u64 val = __vcpu_sys_reg(vcpu, IDREG_SYS_IDX(id));
> >
> >       switch (id) {
> >       case SYS_ID_AA64PFR0_EL1:
> > @@ -1119,6 +1113,14 @@ static u64 read_id_reg(const struct kvm_vcpu *vcpu,
> >       return val;
> >  }
> >
> > +static u64 read_id_reg(const struct kvm_vcpu *vcpu,
> > +                    struct sys_reg_desc const *r, bool raz)
> > +{
> > +     u32 id = reg_to_encoding(r);
> > +
> > +     return raz ? 0 : __read_id_reg(vcpu, id);
> > +}
> > +
> >  static unsigned int id_visibility(const struct kvm_vcpu *vcpu,
> >                                 const struct sys_reg_desc *r)
> >  {
> > @@ -1178,6 +1180,16 @@ static unsigned int sve_visibility(const struct kvm_vcpu *vcpu,
> >       return REG_HIDDEN;
> >  }
> >
> > +static void reset_id_reg(struct kvm_vcpu *vcpu, const struct sys_reg_desc *rd)
> > +{
> > +     u32 id = reg_to_encoding(rd);
> > +
> > +     if (vcpu_has_reset_once(vcpu))
> > +             return;
> The KVM API allows to call VCPU_INIT several times (with same
> target/feature). With above check on the second call the ID_REGS won't
> be reset. Somehow this is aligned with target/feature behavior. However
> if this is what we want, I think we would need to document it in the KVM
> API doc.

Thank you for the comment.

That is what we want.  Since ID registers are read only registers,
their values must not change across the reset.

'4.82 KVM_ARM_VCPU_INIT' in api.rst says:

  System registers: Reset to their architecturally defined
  values as for a warm reset to EL1 (resp. SVC)

Since this reset behavior for the ID registers follows what is
described above, I'm not sure if we need to document the reset
behavior of the ID registers specifically.
If KVM changes the values across the resets, I would think it
rather needs to be documented though.

Thanks,
Reiji

Re: [RFC PATCH v3 03/29] KVM: arm64: Introduce struct id_reg_info

[Reiji Watanabe]

Hi Eric,

On Thu, Dec 2, 2021 at 4:51 AM Eric Auger <eauger@redhat.com> wrote:
>
> Hi Reiji,
>
> On 11/17/21 7:43 AM, Reiji Watanabe wrote:
> > This patch lays the groundwork to make ID registers writable.
> >
> > Introduce struct id_reg_info for an ID register to manage the
> > register specific control of its value for the guest, and provide set
> > of functions commonly used for ID registers to make them writable.
> >
> > The id_reg_info is used to do register specific initialization,
> > validation of the ID register and etc.  Not all ID registers must
> > have the id_reg_info. ID registers that don't have the id_reg_info
> > are handled in a common way that is applied to all ID registers.
> >
> > At present, changing an ID register from userspace is allowed only
> > if the ID register has the id_reg_info, but that will be changed
> > by the following patches.
> >
> > No ID register has the structure yet and the following patches
> > will add the id_reg_info for some ID registers.
> >
> > Signed-off-by: Reiji Watanabe <reijiw@google.com>
> > ---
> >  arch/arm64/include/asm/sysreg.h |   1 +
> >  arch/arm64/kvm/sys_regs.c       | 226 ++++++++++++++++++++++++++++++--
> >  2 files changed, 218 insertions(+), 9 deletions(-)
> >
> > diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h
> > index 16b3f1a1d468..597609f26331 100644
> > --- a/arch/arm64/include/asm/sysreg.h
> > +++ b/arch/arm64/include/asm/sysreg.h
> > @@ -1197,6 +1197,7 @@
> >  #define ICH_VTR_TDS_MASK     (1 << ICH_VTR_TDS_SHIFT)
> >
> >  #define ARM64_FEATURE_FIELD_BITS     4
> > +#define ARM64_FEATURE_FIELD_MASK     ((1ull << ARM64_FEATURE_FIELD_BITS) - 1)
> >
> >  /* Create a mask for the feature bits of the specified feature. */
> >  #define ARM64_FEATURE_MASK(x)        (GENMASK_ULL(x##_SHIFT + ARM64_FEATURE_FIELD_BITS - 1, x##_SHIFT))
> > diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
> > index 5608d3410660..1552cd5581b7 100644
> > --- a/arch/arm64/kvm/sys_regs.c
> > +++ b/arch/arm64/kvm/sys_regs.c
> > @@ -265,6 +265,181 @@ static bool trap_raz_wi(struct kvm_vcpu *vcpu,
> >               return read_zero(vcpu, p);
> >  }
> >
> > +/*
> > + * A value for FCT_LOWER_SAFE must be zero and changing that will affect
> > + * ftr_check_types of id_reg_info.
> > + */
> > +enum feature_check_type {
> > +     FCT_LOWER_SAFE = 0,
> > +     FCT_HIGHER_SAFE,
> > +     FCT_HIGHER_OR_ZERO_SAFE,
> > +     FCT_EXACT,
> > +     FCT_EXACT_OR_ZERO_SAFE,
> > +     FCT_IGNORE,     /* Don't check (any value is fine) */
> > +};
> > +
> > +static int arm64_check_feature_one(enum feature_check_type type, int val,
> > +                                int limit)
> > +{
> > +     bool is_safe = false;
> > +
> > +     if (val == limit)
> > +             return 0;
> > +
> > +     switch (type) {
> > +     case FCT_LOWER_SAFE:
> > +             is_safe = (val <= limit);
> > +             break;
> > +     case FCT_HIGHER_OR_ZERO_SAFE:
> > +             if (val == 0) {
> > +                     is_safe = true;
> > +                     break;
> > +             }
> > +             fallthrough;
> > +     case FCT_HIGHER_SAFE:
> > +             is_safe = (val >= limit);
> > +             break;
> > +     case FCT_EXACT:
> > +             break;
> > +     case FCT_EXACT_OR_ZERO_SAFE:
> > +             is_safe = (val == 0);
> > +             break;
> > +     case FCT_IGNORE:
> > +             is_safe = true;
> > +             break;
> > +     default:
> > +             WARN_ONCE(1, "Unexpected feature_check_type (%d)\n", type);
> > +             break;
> > +     }
> > +
> > +     return is_safe ? 0 : -1;
> > +}
> > +
> > +#define      FCT_TYPE_MASK           0x7
> > +#define      FCT_TYPE_SHIFT          1
> > +#define      FCT_SIGN_MASK           0x1
> > +#define      FCT_SIGN_SHIFT          0
> > +#define      FCT_TYPE(val)   ((val >> FCT_TYPE_SHIFT) & FCT_TYPE_MASK)
> > +#define      FCT_SIGN(val)   ((val >> FCT_SIGN_SHIFT) & FCT_SIGN_MASK)
> > +
> > +#define      MAKE_FCT(shift, type, sign)                             \
> > +     ((u64)((((type) & FCT_TYPE_MASK) << FCT_TYPE_SHIFT) |   \
> > +            (((sign) & FCT_SIGN_MASK) << FCT_SIGN_SHIFT)) << (shift))
> > +
> > +/* For signed field */
> > +#define      S_FCT(shift, type)      MAKE_FCT(shift, type, 1)
> > +/* For unigned field */
> > +#define      U_FCT(shift, type)      MAKE_FCT(shift, type, 0)
> > +
> > +/*
> > + * @val and @lim are both a value of the ID register. The function checks
> > + * if all features indicated in @val can be supported for guests on the host,
> > + * which supports features indicated in @lim. @check_types indicates how
> > + * features in the ID register needs to be checked.
> > + * See comments for id_reg_info's ftr_check_types field for more detail.
> > + */
> > +static int arm64_check_features(u64 check_types, u64 val, u64 lim)
> > +{
> > +     int i;
> > +
> > +     for (i = 0; i < 64; i += ARM64_FEATURE_FIELD_BITS) {
> > +             u8 ftr_check = (check_types >> i) & ARM64_FEATURE_FIELD_MASK;
> > +             bool is_sign = FCT_SIGN(ftr_check);
> > +             enum feature_check_type fctype = FCT_TYPE(ftr_check);
> > +             int fval, flim, ret;
> > +
> > +             fval = cpuid_feature_extract_field(val, i, is_sign);
> > +             flim = cpuid_feature_extract_field(lim, i, is_sign);
> > +
> > +             ret = arm64_check_feature_one(fctype, fval, flim);
> > +             if (ret)
> > +                     return -E2BIG;
> > +     }
> > +     return 0;
> > +}
> > +
> > +struct id_reg_info {
> > +     u32     sys_reg;        /* Register ID */
> > +
> > +     /*
> > +      * Limit value of the register for a vcpu. The value is the sanitized
> > +      * system value with bits cleared for unsupported features for the
> > +      * guest.
> > +      */
> > +     u64     vcpu_limit_val;
> > +
> > +     /*
> > +      * The ftr_check_types is comprised of a set of 4 bits fields.
> > +      * Each 4 bits field is for a feature indicated by the same bits
> > +      * field of the ID register and indicates how the feature support
> > +      * for guests needs to be checked.
> > +      * The bit 0 indicates that the corresponding ID register field
> > +      * is signed(1) or unsigned(0).
> > +      * The bits [3:1] hold feature_check_type for the field.
> > +      * If all zero, all features in the ID register are treated as unsigned
> > +      * fields and checked based on Principles of the ID scheme for fields
> > +      * in ID registers (FCT_LOWER_SAFE of feature_check_type).
> > +      */
> > +     u64     ftr_check_types;
> > +
> > +     /* Initialization function of the id_reg_info */
> > +     void (*init)(struct id_reg_info *id_reg);
> > +
> > +     /* Register specific validation function */
> > +     int (*validate)(struct kvm_vcpu *vcpu, const struct id_reg_info *id_reg,
> > +                     u64 val);
> > +
> > +     /* Return the reset value of the register for the vCPU */
> > +     u64 (*get_reset_val)(struct kvm_vcpu *vcpu,
> > +                          const struct id_reg_info *id_reg);
> It is unclear to me why we need 2 different callbacks, ie. init and
> get_reset_val. ID_REGS can only be accessed from user space after the
> vcpu reset, right? So couldn't we have a single cb instead of this
> overwrite mechanism?

Thank you for the comment.

What the init() does needs to be done just once.
It initializes the id_reg_info itself (not for the ID register of vCPU).
And the data initialized by the init() is used not just for the
overwrite mechanism at the vcpu reset but for other purposes as well.

What the get_reset_val does needs to be done for every initial vCPU reset.
It provides the initial value for the vCPU, which depends on its feature
configuration that is configured by KVM_ARM_VCPU_INIT (or other APIs).

Of course there are other ways to achieve the same, and it's entirely
possible to have a single function though.  I just chose to use a
separate function for each of those two different purposes.

Thanks,
Reiji

Re: [RFC PATCH v3 04/29] KVM: arm64: Make ID_AA64PFR0_EL1 writable

[Reiji Watanabe]

Hi Eric,

On Thu, Dec 2, 2021 at 5:02 AM Eric Auger <eauger@redhat.com> wrote:
>
> Hi Reiji,
>
> On 11/30/21 2:29 AM, Reiji Watanabe wrote:
> > Hi Eric,
> >
> > On Thu, Nov 25, 2021 at 7:35 AM Eric Auger <eauger@redhat.com> wrote:
> >>
> >> Hi Reiji,
> >>
> >> On 11/17/21 7:43 AM, Reiji Watanabe wrote:
> >>> This patch adds id_reg_info for ID_AA64PFR0_EL1 to make it writable by
> >>> userspace.
> >>>
> >>> The CSV2/CSV3 fields of the register were already writable and values
> >>> that were written for them affected all vCPUs before. Now they only
> >>> affect the vCPU.
> >>> Return an error if userspace tries to set SVE/GIC field of the register
> >>> to a value that conflicts with SVE/GIC configuration for the guest.
> >>> SIMD/FP/SVE fields of the requested value are validated according to
> >>> Arm ARM.
> >>>
> >>> Signed-off-by: Reiji Watanabe <reijiw@google.com>
> >>> ---
> >>>  arch/arm64/kvm/sys_regs.c | 159 ++++++++++++++++++++++++--------------
> >>>  1 file changed, 103 insertions(+), 56 deletions(-)
> >>>
> >>> diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
> >>> index 1552cd5581b7..35400869067a 100644
> >>> --- a/arch/arm64/kvm/sys_regs.c
> >>> +++ b/arch/arm64/kvm/sys_regs.c
> >>> @@ -401,6 +401,92 @@ static void id_reg_info_init(struct id_reg_info *id_reg)
> >>>               id_reg->init(id_reg);
> >>>  }
> >>>
> >>> +#define      kvm_has_gic3(kvm)               \
> >>> +     (irqchip_in_kernel(kvm) &&      \
> >>> +      (kvm)->arch.vgic.vgic_model == KVM_DEV_TYPE_ARM_VGIC_V3)
> >> you may move this macro to kvm/arm_vgic.h as this may be used in
> >> vgic/vgic-v3.c too
> >
> > Thank you for the suggestion. I will move that to kvm/arm_vgic.h.
> >
> >
> >>> +
> >>> +static int validate_id_aa64pfr0_el1(struct kvm_vcpu *vcpu,
> >>> +                                 const struct id_reg_info *id_reg, u64 val)
> >>> +{
> >>> +     int fp, simd;
> >>> +     bool vcpu_has_sve = vcpu_has_sve(vcpu);
> >>> +     bool pfr0_has_sve = id_aa64pfr0_sve(val);
> >>> +     int gic;
> >>> +
> >>> +     simd = cpuid_feature_extract_signed_field(val, ID_AA64PFR0_ASIMD_SHIFT);
> >>> +     fp = cpuid_feature_extract_signed_field(val, ID_AA64PFR0_FP_SHIFT);
> >>> +     if (simd != fp)
> >>> +             return -EINVAL;
> >>> +
> >>> +     /* fp must be supported when sve is supported */
> >>> +     if (pfr0_has_sve && (fp < 0))
> >>> +             return -EINVAL;
> >>> +
> >>> +     /* Check if there is a conflict with a request via KVM_ARM_VCPU_INIT */
> >>> +     if (vcpu_has_sve ^ pfr0_has_sve)
> >>> +             return -EPERM;
> >>> +
> >>> +     gic = cpuid_feature_extract_unsigned_field(val, ID_AA64PFR0_GIC_SHIFT);
> >>> +     if ((gic > 0) ^ kvm_has_gic3(vcpu->kvm))
> >>> +             return -EPERM;
> >>
> >> Sometimes from a given architecture version, some lower values are not
> >> allowed. For instance from ARMv8.5 onlt 1 is permitted for CSV3.
> >> Shouldn't we handle that kind of check?
> >
> > As far as I know, there is no way for guests to identify the
> > architecture revision (e.g. v8.1, v8.2, etc).  It might be able
> > to indirectly infer the revision though (from features that are
> > available or etc).
>
> OK. That sounds weird to me as we do many checks accross different IDREG
> settings but we may eventually have a wrong "CPU model" exposed by the
> user space violating those spec revision minima. Shouldn't we introduce
> some way for the userspace to provide his requirements? via new VCPU
> targets for instance?

Thank you for sharing your thoughts and providing the suggestion !

Does the "new vCPU targets" mean Armv8.0, armv8.1, and so on ?

The ID registers' consistency checking in the series is to not
promise more to userspace than what KVM (on the host) can provide,
and to not expose ID register values that are not supported on
any ARM v8 architecture for guests (I think those are what the
current KVM is trying to assure).  I'm not trying to have KVM
provide full consistency checking of ID registers to completely
prevent userspace's bugs in setting ID registers.

I agree that it's quite possible that userspace exposes such wrong
CPU models, and KVM's providing more consistency checking would be
nicer in general.  But should it be KVM's responsibility to completely
prevent such ID register issues due to userspace bugs ?

Honestly, I'm a bit reluctant to do that so far yet:)
If that is something useful that userspace or we (KVM developers)
really want or need, or such userspace issue could affect KVM,
I would be happy to add such extra consistency checking though.

Thanks,
Reiji

Re: [RFC PATCH v3 09/29] KVM: arm64: Hide IMPLEMENTATION DEFINED PMU support for the guest

[Eric Auger]

Hi Reiji,

On 12/4/21 2:04 AM, Reiji Watanabe wrote:
> Hi Eric,
> 
> On Thu, Dec 2, 2021 at 2:57 AM Eric Auger <eauger@redhat.com> wrote:
>>
>> Hi Reiji,
>>
>> On 11/30/21 6:32 AM, Reiji Watanabe wrote:
>>> Hi Eric,
>>>
>>> On Thu, Nov 25, 2021 at 12:30 PM Eric Auger <eauger@redhat.com> wrote:
>>>>
>>>> Hi Reiji,
>>>>
>>>> On 11/17/21 7:43 AM, Reiji Watanabe wrote:
>>>>> When ID_AA64DFR0_EL1.PMUVER or ID_DFR0_EL1.PERFMON is 0xf, which
>>>>> means IMPLEMENTATION DEFINED PMU supported, KVM unconditionally
>>>>> expose the value for the guest as it is.  Since KVM doesn't support
>>>>> IMPLEMENTATION DEFINED PMU for the guest, in that case KVM should
>>>>> exopse 0x0 (PMU is not implemented) instead.
>>>> s/exopse/expose
>>>>>
>>>>> Change cpuid_feature_cap_perfmon_field() to update the field value
>>>>> to 0x0 when it is 0xf.
>>>> is it wrong to expose the guest with a Perfmon value of 0xF? Then the
>>>> guest should not use it as a PMUv3?
>>>
>>>> is it wrong to expose the guest with a Perfmon value of 0xF? Then the
>>>> guest should not use it as a PMUv3?
>>>
>>> For the value 0xf in ID_AA64DFR0_EL1.PMUVER and ID_DFR0_EL1.PERFMON,
>>> Arm ARM says:
>>>   "IMPLEMENTATION DEFINED form of performance monitors supported,
>>>    PMUv3 not supported."
>>>
>>> Since the PMU that KVM supports for guests is PMUv3, 0xf shouldn't
>>> be exposed to guests (And this patch series doesn't allow userspace
>>> to set the fields to 0xf for guests).
>> What I don't get is why this isn't detected before (in kvm_reset_vcpu).
>> if the VCPU was initialized with KVM_ARM_VCPU_PMU_V3 can we honor this
>> init request if the host pmu is implementation defined?
> 
> KVM_ARM_VCPU_INIT with KVM_ARM_VCPU_PMU_V3 will fail in
> kvm_reset_vcpu() if the host PMU is implementation defined.

OK. This was not obvsious to me.

                if (kvm_vcpu_has_pmu(vcpu) && !kvm_arm_support_pmu_v3()) {
                        ret = -EINVAL;
                        goto out;
                }

kvm_perf_init
+	if (perf_num_counters() > 0)
+		static_branch_enable(&kvm_arm_pmu_available);

But I believe you ;-), sorry for the noise

Eric

> 
> The AA64DFR0 and DFR0 registers for a vCPU without KVM_ARM_VCPU_PMU_V3
> indicates IMPLEMENTATION DEFINED PMU support, which is not correct.


> 
> Thanks,
> Reiji
> 

Re: [RFC PATCH v3 09/29] KVM: arm64: Hide IMPLEMENTATION DEFINED PMU support for the guest

[Reiji Watanabe]

Hi Eric,

On Sat, Dec 4, 2021 at 6:14 AM Eric Auger <eauger@redhat.com> wrote:
>
> Hi Reiji,
>
> On 12/4/21 2:04 AM, Reiji Watanabe wrote:
> > Hi Eric,
> >
> > On Thu, Dec 2, 2021 at 2:57 AM Eric Auger <eauger@redhat.com> wrote:
> >>
> >> Hi Reiji,
> >>
> >> On 11/30/21 6:32 AM, Reiji Watanabe wrote:
> >>> Hi Eric,
> >>>
> >>> On Thu, Nov 25, 2021 at 12:30 PM Eric Auger <eauger@redhat.com> wrote:
> >>>>
> >>>> Hi Reiji,
> >>>>
> >>>> On 11/17/21 7:43 AM, Reiji Watanabe wrote:
> >>>>> When ID_AA64DFR0_EL1.PMUVER or ID_DFR0_EL1.PERFMON is 0xf, which
> >>>>> means IMPLEMENTATION DEFINED PMU supported, KVM unconditionally
> >>>>> expose the value for the guest as it is.  Since KVM doesn't support
> >>>>> IMPLEMENTATION DEFINED PMU for the guest, in that case KVM should
> >>>>> exopse 0x0 (PMU is not implemented) instead.
> >>>> s/exopse/expose
> >>>>>
> >>>>> Change cpuid_feature_cap_perfmon_field() to update the field value
> >>>>> to 0x0 when it is 0xf.
> >>>> is it wrong to expose the guest with a Perfmon value of 0xF? Then the
> >>>> guest should not use it as a PMUv3?
> >>>
> >>>> is it wrong to expose the guest with a Perfmon value of 0xF? Then the
> >>>> guest should not use it as a PMUv3?
> >>>
> >>> For the value 0xf in ID_AA64DFR0_EL1.PMUVER and ID_DFR0_EL1.PERFMON,
> >>> Arm ARM says:
> >>>   "IMPLEMENTATION DEFINED form of performance monitors supported,
> >>>    PMUv3 not supported."
> >>>
> >>> Since the PMU that KVM supports for guests is PMUv3, 0xf shouldn't
> >>> be exposed to guests (And this patch series doesn't allow userspace
> >>> to set the fields to 0xf for guests).
> >> What I don't get is why this isn't detected before (in kvm_reset_vcpu).
> >> if the VCPU was initialized with KVM_ARM_VCPU_PMU_V3 can we honor this
> >> init request if the host pmu is implementation defined?
> >
> > KVM_ARM_VCPU_INIT with KVM_ARM_VCPU_PMU_V3 will fail in
> > kvm_reset_vcpu() if the host PMU is implementation defined.
>
> OK. This was not obvsious to me.
>
>                 if (kvm_vcpu_has_pmu(vcpu) && !kvm_arm_support_pmu_v3()) {
>                         ret = -EINVAL;
>                         goto out;
>                 }
>
> kvm_perf_init
> +       if (perf_num_counters() > 0)
> +               static_branch_enable(&kvm_arm_pmu_available);
>
> But I believe you ;-), sorry for the noise

Thank you for the review !

I didn't find the code above in v5.16-rc3, which is the base code of
this series.  So, I'm not sure where the code came from (any kvmarm
repository branch ??).

What I see in v5.16-rc3 is:
----
int kvm_perf_init(void)
{
        return perf_register_guest_info_callbacks(&kvm_guest_cbs);
}

void kvm_host_pmu_init(struct arm_pmu *pmu)
{
        if (pmu->pmuver != 0 && pmu->pmuver != ID_AA64DFR0_PMUVER_IMP_DEF &&
            !kvm_arm_support_pmu_v3() && !is_protected_kvm_enabled())
                static_branch_enable(&kvm_arm_pmu_available);
}
----

And I don't find any other code that enables kvm_arm_pmu_available.

Looking at the KVM's PMUV3 support code for guests in v5.16-rc3,
if KVM allows userspace to configure KVM_ARM_VCPU_PMU_V3 even with
ID_AA64DFR0_PMUVER_IMP_DEF on the host (, which I don't think it does),
I think we should fix that to not allow that.
(I'm not sure how KVM's PMUV3 support code is implemented in the
code that you are looking at though)

Thanks,
Reiji

Re: [RFC PATCH v3 09/29] KVM: arm64: Hide IMPLEMENTATION DEFINED PMU support for the guest

[Itaru Kitayama]

Reiji,
GMail keeps marking your email as spam, any ideas?

On Sun, Dec 5, 2021 at 2:43 AM Reiji Watanabe <reijiw@google.com> wrote:
>
> Hi Eric,
>
> On Sat, Dec 4, 2021 at 6:14 AM Eric Auger <eauger@redhat.com> wrote:
> >
> > Hi Reiji,
> >
> > On 12/4/21 2:04 AM, Reiji Watanabe wrote:
> > > Hi Eric,
> > >
> > > On Thu, Dec 2, 2021 at 2:57 AM Eric Auger <eauger@redhat.com> wrote:
> > >>
> > >> Hi Reiji,
> > >>
> > >> On 11/30/21 6:32 AM, Reiji Watanabe wrote:
> > >>> Hi Eric,
> > >>>
> > >>> On Thu, Nov 25, 2021 at 12:30 PM Eric Auger <eauger@redhat.com> wrote:
> > >>>>
> > >>>> Hi Reiji,
> > >>>>
> > >>>> On 11/17/21 7:43 AM, Reiji Watanabe wrote:
> > >>>>> When ID_AA64DFR0_EL1.PMUVER or ID_DFR0_EL1.PERFMON is 0xf, which
> > >>>>> means IMPLEMENTATION DEFINED PMU supported, KVM unconditionally
> > >>>>> expose the value for the guest as it is.  Since KVM doesn't support
> > >>>>> IMPLEMENTATION DEFINED PMU for the guest, in that case KVM should
> > >>>>> exopse 0x0 (PMU is not implemented) instead.
> > >>>> s/exopse/expose
> > >>>>>
> > >>>>> Change cpuid_feature_cap_perfmon_field() to update the field value
> > >>>>> to 0x0 when it is 0xf.
> > >>>> is it wrong to expose the guest with a Perfmon value of 0xF? Then the
> > >>>> guest should not use it as a PMUv3?
> > >>>
> > >>>> is it wrong to expose the guest with a Perfmon value of 0xF? Then the
> > >>>> guest should not use it as a PMUv3?
> > >>>
> > >>> For the value 0xf in ID_AA64DFR0_EL1.PMUVER and ID_DFR0_EL1.PERFMON,
> > >>> Arm ARM says:
> > >>>   "IMPLEMENTATION DEFINED form of performance monitors supported,
> > >>>    PMUv3 not supported."
> > >>>
> > >>> Since the PMU that KVM supports for guests is PMUv3, 0xf shouldn't
> > >>> be exposed to guests (And this patch series doesn't allow userspace
> > >>> to set the fields to 0xf for guests).
> > >> What I don't get is why this isn't detected before (in kvm_reset_vcpu).
> > >> if the VCPU was initialized with KVM_ARM_VCPU_PMU_V3 can we honor this
> > >> init request if the host pmu is implementation defined?
> > >
> > > KVM_ARM_VCPU_INIT with KVM_ARM_VCPU_PMU_V3 will fail in
> > > kvm_reset_vcpu() if the host PMU is implementation defined.
> >
> > OK. This was not obvsious to me.
> >
> >                 if (kvm_vcpu_has_pmu(vcpu) && !kvm_arm_support_pmu_v3()) {
> >                         ret = -EINVAL;
> >                         goto out;
> >                 }
> >
> > kvm_perf_init
> > +       if (perf_num_counters() > 0)
> > +               static_branch_enable(&kvm_arm_pmu_available);
> >
> > But I believe you ;-), sorry for the noise
>
> Thank you for the review !
>
> I didn't find the code above in v5.16-rc3, which is the base code of
> this series.  So, I'm not sure where the code came from (any kvmarm
> repository branch ??).
>
> What I see in v5.16-rc3 is:
> ----
> int kvm_perf_init(void)
> {
>         return perf_register_guest_info_callbacks(&kvm_guest_cbs);
> }
>
> void kvm_host_pmu_init(struct arm_pmu *pmu)
> {
>         if (pmu->pmuver != 0 && pmu->pmuver != ID_AA64DFR0_PMUVER_IMP_DEF &&
>             !kvm_arm_support_pmu_v3() && !is_protected_kvm_enabled())
>                 static_branch_enable(&kvm_arm_pmu_available);
> }
> ----
>
> And I don't find any other code that enables kvm_arm_pmu_available.
>
> Looking at the KVM's PMUV3 support code for guests in v5.16-rc3,
> if KVM allows userspace to configure KVM_ARM_VCPU_PMU_V3 even with
> ID_AA64DFR0_PMUVER_IMP_DEF on the host (, which I don't think it does),
> I think we should fix that to not allow that.
> (I'm not sure how KVM's PMUV3 support code is implemented in the
> code that you are looking at though)
>
> Thanks,
> Reiji
>
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [RFC PATCH v3 09/29] KVM: arm64: Hide IMPLEMENTATION DEFINED PMU support for the guest

[Reiji Watanabe]

Hi Itaru,

No, I have no idea...
(I just noticed that I also had a lot of kvmarm emails that were
treated as spam...)

Thanks,
Reiji




On Sat, Dec 4, 2021 at 3:38 PM Itaru Kitayama <itaru.kitayama@gmail.com> wrote:
>
> Reiji,
> GMail keeps marking your email as spam, any ideas?
>
> On Sun, Dec 5, 2021 at 2:43 AM Reiji Watanabe <reijiw@google.com> wrote:
> >
> > Hi Eric,
> >
> > On Sat, Dec 4, 2021 at 6:14 AM Eric Auger <eauger@redhat.com> wrote:
> > >
> > > Hi Reiji,
> > >
> > > On 12/4/21 2:04 AM, Reiji Watanabe wrote:
> > > > Hi Eric,
> > > >
> > > > On Thu, Dec 2, 2021 at 2:57 AM Eric Auger <eauger@redhat.com> wrote:
> > > >>
> > > >> Hi Reiji,
> > > >>
> > > >> On 11/30/21 6:32 AM, Reiji Watanabe wrote:
> > > >>> Hi Eric,
> > > >>>
> > > >>> On Thu, Nov 25, 2021 at 12:30 PM Eric Auger <eauger@redhat.com> wrote:
> > > >>>>
> > > >>>> Hi Reiji,
> > > >>>>
> > > >>>> On 11/17/21 7:43 AM, Reiji Watanabe wrote:
> > > >>>>> When ID_AA64DFR0_EL1.PMUVER or ID_DFR0_EL1.PERFMON is 0xf, which
> > > >>>>> means IMPLEMENTATION DEFINED PMU supported, KVM unconditionally
> > > >>>>> expose the value for the guest as it is.  Since KVM doesn't support
> > > >>>>> IMPLEMENTATION DEFINED PMU for the guest, in that case KVM should
> > > >>>>> exopse 0x0 (PMU is not implemented) instead.
> > > >>>> s/exopse/expose
> > > >>>>>
> > > >>>>> Change cpuid_feature_cap_perfmon_field() to update the field value
> > > >>>>> to 0x0 when it is 0xf.
> > > >>>> is it wrong to expose the guest with a Perfmon value of 0xF? Then the
> > > >>>> guest should not use it as a PMUv3?
> > > >>>
> > > >>>> is it wrong to expose the guest with a Perfmon value of 0xF? Then the
> > > >>>> guest should not use it as a PMUv3?
> > > >>>
> > > >>> For the value 0xf in ID_AA64DFR0_EL1.PMUVER and ID_DFR0_EL1.PERFMON,
> > > >>> Arm ARM says:
> > > >>>   "IMPLEMENTATION DEFINED form of performance monitors supported,
> > > >>>    PMUv3 not supported."
> > > >>>
> > > >>> Since the PMU that KVM supports for guests is PMUv3, 0xf shouldn't
> > > >>> be exposed to guests (And this patch series doesn't allow userspace
> > > >>> to set the fields to 0xf for guests).
> > > >> What I don't get is why this isn't detected before (in kvm_reset_vcpu).
> > > >> if the VCPU was initialized with KVM_ARM_VCPU_PMU_V3 can we honor this
> > > >> init request if the host pmu is implementation defined?
> > > >
> > > > KVM_ARM_VCPU_INIT with KVM_ARM_VCPU_PMU_V3 will fail in
> > > > kvm_reset_vcpu() if the host PMU is implementation defined.
> > >
> > > OK. This was not obvsious to me.
> > >
> > >                 if (kvm_vcpu_has_pmu(vcpu) && !kvm_arm_support_pmu_v3()) {
> > >                         ret = -EINVAL;
> > >                         goto out;
> > >                 }
> > >
> > > kvm_perf_init
> > > +       if (perf_num_counters() > 0)
> > > +               static_branch_enable(&kvm_arm_pmu_available);
> > >
> > > But I believe you ;-), sorry for the noise
> >
> > Thank you for the review !
> >
> > I didn't find the code above in v5.16-rc3, which is the base code of
> > this series.  So, I'm not sure where the code came from (any kvmarm
> > repository branch ??).
> >
> > What I see in v5.16-rc3 is:
> > ----
> > int kvm_perf_init(void)
> > {
> >         return perf_register_guest_info_callbacks(&kvm_guest_cbs);
> > }
> >
> > void kvm_host_pmu_init(struct arm_pmu *pmu)
> > {
> >         if (pmu->pmuver != 0 && pmu->pmuver != ID_AA64DFR0_PMUVER_IMP_DEF &&
> >             !kvm_arm_support_pmu_v3() && !is_protected_kvm_enabled())
> >                 static_branch_enable(&kvm_arm_pmu_available);
> > }
> > ----
> >
> > And I don't find any other code that enables kvm_arm_pmu_available.
> >
> > Looking at the KVM's PMUV3 support code for guests in v5.16-rc3,
> > if KVM allows userspace to configure KVM_ARM_VCPU_PMU_V3 even with
> > ID_AA64DFR0_PMUVER_IMP_DEF on the host (, which I don't think it does),
> > I think we should fix that to not allow that.
> > (I'm not sure how KVM's PMUV3 support code is implemented in the
> > code that you are looking at though)
> >
> > Thanks,
> > Reiji
> >
> > _______________________________________________
> > linux-arm-kernel mailing list
> > linux-arm-kernel@lists.infradead.org
> > http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [RFC PATCH v3 09/29] KVM: arm64: Hide IMPLEMENTATION DEFINED PMU support for the guest

[Alexandru Elisei]

Hi,

On Sat, Dec 04, 2021 at 09:39:59AM -0800, Reiji Watanabe wrote:
> Hi Eric,
> 
> On Sat, Dec 4, 2021 at 6:14 AM Eric Auger <eauger@redhat.com> wrote:
> >
> > Hi Reiji,
> >
> > On 12/4/21 2:04 AM, Reiji Watanabe wrote:
> > > Hi Eric,
> > >
> > > On Thu, Dec 2, 2021 at 2:57 AM Eric Auger <eauger@redhat.com> wrote:
> > >>
> > >> Hi Reiji,
> > >>
> > >> On 11/30/21 6:32 AM, Reiji Watanabe wrote:
> > >>> Hi Eric,
> > >>>
> > >>> On Thu, Nov 25, 2021 at 12:30 PM Eric Auger <eauger@redhat.com> wrote:
> > >>>>
> > >>>> Hi Reiji,
> > >>>>
> > >>>> On 11/17/21 7:43 AM, Reiji Watanabe wrote:
> > >>>>> When ID_AA64DFR0_EL1.PMUVER or ID_DFR0_EL1.PERFMON is 0xf, which
> > >>>>> means IMPLEMENTATION DEFINED PMU supported, KVM unconditionally
> > >>>>> expose the value for the guest as it is.  Since KVM doesn't support
> > >>>>> IMPLEMENTATION DEFINED PMU for the guest, in that case KVM should
> > >>>>> exopse 0x0 (PMU is not implemented) instead.
> > >>>> s/exopse/expose
> > >>>>>
> > >>>>> Change cpuid_feature_cap_perfmon_field() to update the field value
> > >>>>> to 0x0 when it is 0xf.
> > >>>> is it wrong to expose the guest with a Perfmon value of 0xF? Then the
> > >>>> guest should not use it as a PMUv3?
> > >>>
> > >>>> is it wrong to expose the guest with a Perfmon value of 0xF? Then the
> > >>>> guest should not use it as a PMUv3?
> > >>>
> > >>> For the value 0xf in ID_AA64DFR0_EL1.PMUVER and ID_DFR0_EL1.PERFMON,
> > >>> Arm ARM says:
> > >>>   "IMPLEMENTATION DEFINED form of performance monitors supported,
> > >>>    PMUv3 not supported."
> > >>>
> > >>> Since the PMU that KVM supports for guests is PMUv3, 0xf shouldn't
> > >>> be exposed to guests (And this patch series doesn't allow userspace
> > >>> to set the fields to 0xf for guests).
> > >> What I don't get is why this isn't detected before (in kvm_reset_vcpu).
> > >> if the VCPU was initialized with KVM_ARM_VCPU_PMU_V3 can we honor this
> > >> init request if the host pmu is implementation defined?
> > >
> > > KVM_ARM_VCPU_INIT with KVM_ARM_VCPU_PMU_V3 will fail in
> > > kvm_reset_vcpu() if the host PMU is implementation defined.
> >
> > OK. This was not obvsious to me.
> >
> >                 if (kvm_vcpu_has_pmu(vcpu) && !kvm_arm_support_pmu_v3()) {
> >                         ret = -EINVAL;
> >                         goto out;
> >                 }
> >
> > kvm_perf_init
> > +       if (perf_num_counters() > 0)
> > +               static_branch_enable(&kvm_arm_pmu_available);
> >
> > But I believe you ;-), sorry for the noise
> 
> Thank you for the review !
> 
> I didn't find the code above in v5.16-rc3, which is the base code of
> this series.  So, I'm not sure where the code came from (any kvmarm
> repository branch ??).
> 
> What I see in v5.16-rc3 is:
> ----
> int kvm_perf_init(void)
> {
>         return perf_register_guest_info_callbacks(&kvm_guest_cbs);
> }
> 
> void kvm_host_pmu_init(struct arm_pmu *pmu)
> {
>         if (pmu->pmuver != 0 && pmu->pmuver != ID_AA64DFR0_PMUVER_IMP_DEF &&
>             !kvm_arm_support_pmu_v3() && !is_protected_kvm_enabled())
>                 static_branch_enable(&kvm_arm_pmu_available);
> }
> ----
> 
> And I don't find any other code that enables kvm_arm_pmu_available.

The code was recently changed (in v5.15 I think), I think Eric is looking
at an older version.

> 
> Looking at the KVM's PMUV3 support code for guests in v5.16-rc3,
> if KVM allows userspace to configure KVM_ARM_VCPU_PMU_V3 even with
> ID_AA64DFR0_PMUVER_IMP_DEF on the host (, which I don't think it does),
> I think we should fix that to not allow that.

I recently started looking into that too. If there's only one PMU, then the
guest won't see the value IMP DEF for PMUVer (userspace cannot set the PMU
feature because !kvm_arm_support_pmu_v3()).

On heterogeneous systems with multiple PMUs, it gets complicated. I don't
have any such hardware, but what I think will happen is that KVM will
enable the static branch if there is at least one PMU with
PMUVer != IMP_DEF, even if there are other PMUs with PMUVer = IMP_DEF. But
read_sanitised_ftr_reg() will always return 0 for the
PMUVer field because the field is defined as FTR_EXACT with a safe value of
0 in cpufeature.c. So the guest ends up seeing PMUVer = 0.

I'm not sure if this is the case because I'm not familiar with the cpu
features code, but I planning to investigate further.

Thanks,
Alex

> (I'm not sure how KVM's PMUV3 support code is implemented in the
> code that you are looking at though)
> 
> Thanks,
> Reiji

Re: [RFC PATCH v3 09/29] KVM: arm64: Hide IMPLEMENTATION DEFINED PMU support for the guest

[Eric Auger]

Hi

On 12/6/21 10:52 AM, Alexandru Elisei wrote:
> Hi,
> 
> On Sat, Dec 04, 2021 at 09:39:59AM -0800, Reiji Watanabe wrote:
>> Hi Eric,
>>
>> On Sat, Dec 4, 2021 at 6:14 AM Eric Auger <eauger@redhat.com> wrote:
>>>
>>> Hi Reiji,
>>>
>>> On 12/4/21 2:04 AM, Reiji Watanabe wrote:
>>>> Hi Eric,
>>>>
>>>> On Thu, Dec 2, 2021 at 2:57 AM Eric Auger <eauger@redhat.com> wrote:
>>>>>
>>>>> Hi Reiji,
>>>>>
>>>>> On 11/30/21 6:32 AM, Reiji Watanabe wrote:
>>>>>> Hi Eric,
>>>>>>
>>>>>> On Thu, Nov 25, 2021 at 12:30 PM Eric Auger <eauger@redhat.com> wrote:
>>>>>>>
>>>>>>> Hi Reiji,
>>>>>>>
>>>>>>> On 11/17/21 7:43 AM, Reiji Watanabe wrote:
>>>>>>>> When ID_AA64DFR0_EL1.PMUVER or ID_DFR0_EL1.PERFMON is 0xf, which
>>>>>>>> means IMPLEMENTATION DEFINED PMU supported, KVM unconditionally
>>>>>>>> expose the value for the guest as it is.  Since KVM doesn't support
>>>>>>>> IMPLEMENTATION DEFINED PMU for the guest, in that case KVM should
>>>>>>>> exopse 0x0 (PMU is not implemented) instead.
>>>>>>> s/exopse/expose
>>>>>>>>
>>>>>>>> Change cpuid_feature_cap_perfmon_field() to update the field value
>>>>>>>> to 0x0 when it is 0xf.
>>>>>>> is it wrong to expose the guest with a Perfmon value of 0xF? Then the
>>>>>>> guest should not use it as a PMUv3?
>>>>>>
>>>>>>> is it wrong to expose the guest with a Perfmon value of 0xF? Then the
>>>>>>> guest should not use it as a PMUv3?
>>>>>>
>>>>>> For the value 0xf in ID_AA64DFR0_EL1.PMUVER and ID_DFR0_EL1.PERFMON,
>>>>>> Arm ARM says:
>>>>>>   "IMPLEMENTATION DEFINED form of performance monitors supported,
>>>>>>    PMUv3 not supported."
>>>>>>
>>>>>> Since the PMU that KVM supports for guests is PMUv3, 0xf shouldn't
>>>>>> be exposed to guests (And this patch series doesn't allow userspace
>>>>>> to set the fields to 0xf for guests).
>>>>> What I don't get is why this isn't detected before (in kvm_reset_vcpu).
>>>>> if the VCPU was initialized with KVM_ARM_VCPU_PMU_V3 can we honor this
>>>>> init request if the host pmu is implementation defined?
>>>>
>>>> KVM_ARM_VCPU_INIT with KVM_ARM_VCPU_PMU_V3 will fail in
>>>> kvm_reset_vcpu() if the host PMU is implementation defined.
>>>
>>> OK. This was not obvsious to me.
>>>
>>>                 if (kvm_vcpu_has_pmu(vcpu) && !kvm_arm_support_pmu_v3()) {
>>>                         ret = -EINVAL;
>>>                         goto out;
>>>                 }
>>>
>>> kvm_perf_init
>>> +       if (perf_num_counters() > 0)
>>> +               static_branch_enable(&kvm_arm_pmu_available);
>>>
>>> But I believe you ;-), sorry for the noise
>>
>> Thank you for the review !
>>
>> I didn't find the code above in v5.16-rc3, which is the base code of
>> this series.  So, I'm not sure where the code came from (any kvmarm
>> repository branch ??).
>>
>> What I see in v5.16-rc3 is:
>> ----
>> int kvm_perf_init(void)
>> {
>>         return perf_register_guest_info_callbacks(&kvm_guest_cbs);
>> }
>>
>> void kvm_host_pmu_init(struct arm_pmu *pmu)
>> {
>>         if (pmu->pmuver != 0 && pmu->pmuver != ID_AA64DFR0_PMUVER_IMP_DEF &&
>>             !kvm_arm_support_pmu_v3() && !is_protected_kvm_enabled())
>>                 static_branch_enable(&kvm_arm_pmu_available);
>> }
>> ----
>>
>> And I don't find any other code that enables kvm_arm_pmu_available.
> 
> The code was recently changed (in v5.15 I think), I think Eric is looking
> at an older version.

Yes I was "googling" kvm_arm_pmu_available enablement and I missed the
kvm_pmu_probe_pmuver() != ID_AA64DFR0_PMUVER_IMP_DEF check addition. So
except the heterogenous case reported by Alexandru below, we should be
fine. Sorry for the noise.

Thanks

Eric
> 
>>
>> Looking at the KVM's PMUV3 support code for guests in v5.16-rc3,
>> if KVM allows userspace to configure KVM_ARM_VCPU_PMU_V3 even with
>> ID_AA64DFR0_PMUVER_IMP_DEF on the host (, which I don't think it does),
>> I think we should fix that to not allow that.
> 
> I recently started looking into that too. If there's only one PMU, then the
> guest won't see the value IMP DEF for PMUVer (userspace cannot set the PMU
> feature because !kvm_arm_support_pmu_v3()).
> 
> On heterogeneous systems with multiple PMUs, it gets complicated. I don't
> have any such hardware, but what I think will happen is that KVM will
> enable the static branch if there is at least one PMU with
> PMUVer != IMP_DEF, even if there are other PMUs with PMUVer = IMP_DEF. But
> read_sanitised_ftr_reg() will always return 0 for the
> PMUVer field because the field is defined as FTR_EXACT with a safe value of
> 0 in cpufeature.c. So the guest ends up seeing PMUVer = 0.
> 
> I'm not sure if this is the case because I'm not familiar with the cpu
> features code, but I planning to investigate further.
> 
> Thanks,
> Alex
> 
>> (I'm not sure how KVM's PMUV3 support code is implemented in the
>> code that you are looking at though)
>>
>> Thanks,
>> Reiji
> 

Re: [RFC PATCH v3 09/29] KVM: arm64: Hide IMPLEMENTATION DEFINED PMU support for the guest

[Reiji Watanabe]

Hi Eric,

On Mon, Dec 6, 2021 at 2:25 AM Eric Auger <eauger@redhat.com> wrote:
>
> Hi
>
> On 12/6/21 10:52 AM, Alexandru Elisei wrote:
> > Hi,
> >
> > On Sat, Dec 04, 2021 at 09:39:59AM -0800, Reiji Watanabe wrote:
> >> Hi Eric,
> >>
> >> On Sat, Dec 4, 2021 at 6:14 AM Eric Auger <eauger@redhat.com> wrote:
> >>>
> >>> Hi Reiji,
> >>>
> >>> On 12/4/21 2:04 AM, Reiji Watanabe wrote:
> >>>> Hi Eric,
> >>>>
> >>>> On Thu, Dec 2, 2021 at 2:57 AM Eric Auger <eauger@redhat.com> wrote:
> >>>>>
> >>>>> Hi Reiji,
> >>>>>
> >>>>> On 11/30/21 6:32 AM, Reiji Watanabe wrote:
> >>>>>> Hi Eric,
> >>>>>>
> >>>>>> On Thu, Nov 25, 2021 at 12:30 PM Eric Auger <eauger@redhat.com> wrote:
> >>>>>>>
> >>>>>>> Hi Reiji,
> >>>>>>>
> >>>>>>> On 11/17/21 7:43 AM, Reiji Watanabe wrote:
> >>>>>>>> When ID_AA64DFR0_EL1.PMUVER or ID_DFR0_EL1.PERFMON is 0xf, which
> >>>>>>>> means IMPLEMENTATION DEFINED PMU supported, KVM unconditionally
> >>>>>>>> expose the value for the guest as it is.  Since KVM doesn't support
> >>>>>>>> IMPLEMENTATION DEFINED PMU for the guest, in that case KVM should
> >>>>>>>> exopse 0x0 (PMU is not implemented) instead.
> >>>>>>> s/exopse/expose
> >>>>>>>>
> >>>>>>>> Change cpuid_feature_cap_perfmon_field() to update the field value
> >>>>>>>> to 0x0 when it is 0xf.
> >>>>>>> is it wrong to expose the guest with a Perfmon value of 0xF? Then the
> >>>>>>> guest should not use it as a PMUv3?
> >>>>>>
> >>>>>>> is it wrong to expose the guest with a Perfmon value of 0xF? Then the
> >>>>>>> guest should not use it as a PMUv3?
> >>>>>>
> >>>>>> For the value 0xf in ID_AA64DFR0_EL1.PMUVER and ID_DFR0_EL1.PERFMON,
> >>>>>> Arm ARM says:
> >>>>>>   "IMPLEMENTATION DEFINED form of performance monitors supported,
> >>>>>>    PMUv3 not supported."
> >>>>>>
> >>>>>> Since the PMU that KVM supports for guests is PMUv3, 0xf shouldn't
> >>>>>> be exposed to guests (And this patch series doesn't allow userspace
> >>>>>> to set the fields to 0xf for guests).
> >>>>> What I don't get is why this isn't detected before (in kvm_reset_vcpu).
> >>>>> if the VCPU was initialized with KVM_ARM_VCPU_PMU_V3 can we honor this
> >>>>> init request if the host pmu is implementation defined?
> >>>>
> >>>> KVM_ARM_VCPU_INIT with KVM_ARM_VCPU_PMU_V3 will fail in
> >>>> kvm_reset_vcpu() if the host PMU is implementation defined.
> >>>
> >>> OK. This was not obvsious to me.
> >>>
> >>>                 if (kvm_vcpu_has_pmu(vcpu) && !kvm_arm_support_pmu_v3()) {
> >>>                         ret = -EINVAL;
> >>>                         goto out;
> >>>                 }
> >>>
> >>> kvm_perf_init
> >>> +       if (perf_num_counters() > 0)
> >>> +               static_branch_enable(&kvm_arm_pmu_available);
> >>>
> >>> But I believe you ;-), sorry for the noise
> >>
> >> Thank you for the review !
> >>
> >> I didn't find the code above in v5.16-rc3, which is the base code of
> >> this series.  So, I'm not sure where the code came from (any kvmarm
> >> repository branch ??).
> >>
> >> What I see in v5.16-rc3 is:
> >> ----
> >> int kvm_perf_init(void)
> >> {
> >>         return perf_register_guest_info_callbacks(&kvm_guest_cbs);
> >> }
> >>
> >> void kvm_host_pmu_init(struct arm_pmu *pmu)
> >> {
> >>         if (pmu->pmuver != 0 && pmu->pmuver != ID_AA64DFR0_PMUVER_IMP_DEF &&
> >>             !kvm_arm_support_pmu_v3() && !is_protected_kvm_enabled())
> >>                 static_branch_enable(&kvm_arm_pmu_available);
> >> }
> >> ----
> >>
> >> And I don't find any other code that enables kvm_arm_pmu_available.
> >
> > The code was recently changed (in v5.15 I think), I think Eric is looking
> > at an older version.
>
> Yes I was "googling" kvm_arm_pmu_available enablement and I missed the
> kvm_pmu_probe_pmuver() != ID_AA64DFR0_PMUVER_IMP_DEF check addition. So
> except the heterogenous case reported by Alexandru below, we should be
> fine. Sorry for the noise.

Understood. Thank you for the confirmation.

Regards,
Reiji


> >>
> >> Looking at the KVM's PMUV3 support code for guests in v5.16-rc3,
> >> if KVM allows userspace to configure KVM_ARM_VCPU_PMU_V3 even with
> >> ID_AA64DFR0_PMUVER_IMP_DEF on the host (, which I don't think it does),
> >> I think we should fix that to not allow that.
> >
> > I recently started looking into that too. If there's only one PMU, then the
> > guest won't see the value IMP DEF for PMUVer (userspace cannot set the PMU
> > feature because !kvm_arm_support_pmu_v3()).
> >
> > On heterogeneous systems with multiple PMUs, it gets complicated. I don't
> > have any such hardware, but what I think will happen is that KVM will
> > enable the static branch if there is at least one PMU with
> > PMUVer != IMP_DEF, even if there are other PMUs with PMUVer = IMP_DEF. But
> > read_sanitised_ftr_reg() will always return 0 for the
> > PMUVer field because the field is defined as FTR_EXACT with a safe value of
> > 0 in cpufeature.c. So the guest ends up seeing PMUVer = 0.
> >
> > I'm not sure if this is the case because I'm not familiar with the cpu
> > features code, but I planning to investigate further.
> >
> > Thanks,
> > Alex
> >
> >> (I'm not sure how KVM's PMUV3 support code is implemented in the
> >> code that you are looking at though)
> >>
> >> Thanks,
> >> Reiji
> >
>

Re: [RFC PATCH v3 09/29] KVM: arm64: Hide IMPLEMENTATION DEFINED PMU support for the guest

[Reiji Watanabe]

Hi Alex,

On Mon, Dec 6, 2021 at 1:52 AM Alexandru Elisei
<alexandru.elisei@arm.com> wrote:
>
> Hi,
>
> On Sat, Dec 04, 2021 at 09:39:59AM -0800, Reiji Watanabe wrote:
> > Hi Eric,
> >
> > On Sat, Dec 4, 2021 at 6:14 AM Eric Auger <eauger@redhat.com> wrote:
> > >
> > > Hi Reiji,
> > >
> > > On 12/4/21 2:04 AM, Reiji Watanabe wrote:
> > > > Hi Eric,
> > > >
> > > > On Thu, Dec 2, 2021 at 2:57 AM Eric Auger <eauger@redhat.com> wrote:
> > > >>
> > > >> Hi Reiji,
> > > >>
> > > >> On 11/30/21 6:32 AM, Reiji Watanabe wrote:
> > > >>> Hi Eric,
> > > >>>
> > > >>> On Thu, Nov 25, 2021 at 12:30 PM Eric Auger <eauger@redhat.com> wrote:
> > > >>>>
> > > >>>> Hi Reiji,
> > > >>>>
> > > >>>> On 11/17/21 7:43 AM, Reiji Watanabe wrote:
> > > >>>>> When ID_AA64DFR0_EL1.PMUVER or ID_DFR0_EL1.PERFMON is 0xf, which
> > > >>>>> means IMPLEMENTATION DEFINED PMU supported, KVM unconditionally
> > > >>>>> expose the value for the guest as it is.  Since KVM doesn't support
> > > >>>>> IMPLEMENTATION DEFINED PMU for the guest, in that case KVM should
> > > >>>>> exopse 0x0 (PMU is not implemented) instead.
> > > >>>> s/exopse/expose
> > > >>>>>
> > > >>>>> Change cpuid_feature_cap_perfmon_field() to update the field value
> > > >>>>> to 0x0 when it is 0xf.
> > > >>>> is it wrong to expose the guest with a Perfmon value of 0xF? Then the
> > > >>>> guest should not use it as a PMUv3?
> > > >>>
> > > >>>> is it wrong to expose the guest with a Perfmon value of 0xF? Then the
> > > >>>> guest should not use it as a PMUv3?
> > > >>>
> > > >>> For the value 0xf in ID_AA64DFR0_EL1.PMUVER and ID_DFR0_EL1.PERFMON,
> > > >>> Arm ARM says:
> > > >>>   "IMPLEMENTATION DEFINED form of performance monitors supported,
> > > >>>    PMUv3 not supported."
> > > >>>
> > > >>> Since the PMU that KVM supports for guests is PMUv3, 0xf shouldn't
> > > >>> be exposed to guests (And this patch series doesn't allow userspace
> > > >>> to set the fields to 0xf for guests).
> > > >> What I don't get is why this isn't detected before (in kvm_reset_vcpu).
> > > >> if the VCPU was initialized with KVM_ARM_VCPU_PMU_V3 can we honor this
> > > >> init request if the host pmu is implementation defined?
> > > >
> > > > KVM_ARM_VCPU_INIT with KVM_ARM_VCPU_PMU_V3 will fail in
> > > > kvm_reset_vcpu() if the host PMU is implementation defined.
> > >
> > > OK. This was not obvsious to me.
> > >
> > >                 if (kvm_vcpu_has_pmu(vcpu) && !kvm_arm_support_pmu_v3()) {
> > >                         ret = -EINVAL;
> > >                         goto out;
> > >                 }
> > >
> > > kvm_perf_init
> > > +       if (perf_num_counters() > 0)
> > > +               static_branch_enable(&kvm_arm_pmu_available);
> > >
> > > But I believe you ;-), sorry for the noise
> >
> > Thank you for the review !
> >
> > I didn't find the code above in v5.16-rc3, which is the base code of
> > this series.  So, I'm not sure where the code came from (any kvmarm
> > repository branch ??).
> >
> > What I see in v5.16-rc3 is:
> > ----
> > int kvm_perf_init(void)
> > {
> >         return perf_register_guest_info_callbacks(&kvm_guest_cbs);
> > }
> >
> > void kvm_host_pmu_init(struct arm_pmu *pmu)
> > {
> >         if (pmu->pmuver != 0 && pmu->pmuver != ID_AA64DFR0_PMUVER_IMP_DEF &&
> >             !kvm_arm_support_pmu_v3() && !is_protected_kvm_enabled())
> >                 static_branch_enable(&kvm_arm_pmu_available);
> > }
> > ----
> >
> > And I don't find any other code that enables kvm_arm_pmu_available.
>
> The code was recently changed (in v5.15 I think), I think Eric is looking
> at an older version.
>
> >
> > Looking at the KVM's PMUV3 support code for guests in v5.16-rc3,
> > if KVM allows userspace to configure KVM_ARM_VCPU_PMU_V3 even with
> > ID_AA64DFR0_PMUVER_IMP_DEF on the host (, which I don't think it does),
> > I think we should fix that to not allow that.
>
> I recently started looking into that too. If there's only one PMU, then the
> guest won't see the value IMP DEF for PMUVer (userspace cannot set the PMU
> feature because !kvm_arm_support_pmu_v3()).
>
> On heterogeneous systems with multiple PMUs, it gets complicated. I don't
> have any such hardware, but what I think will happen is that KVM will
> enable the static branch if there is at least one PMU with
> PMUVer != IMP_DEF, even if there are other PMUs with PMUVer = IMP_DEF. But
> read_sanitised_ftr_reg() will always return 0 for the
> PMUVer field because the field is defined as FTR_EXACT with a safe value of
> 0 in cpufeature.c. So the guest ends up seeing PMUVer = 0.
>
> I'm not sure if this is the case because I'm not familiar with the cpu
> features code, but I planning to investigate further.

Thank you for the comment !

Yes, it looks like that KVM will enable the static branch if there
is at least one PMU with PMUVer != 0 && PMUVer != IMP_DEF.
(then, yes, AA64DFR0.PMUVER will be 0 even for a vCPU that
 KVM_ARM_VCPU_PMU_V3 is successfully configured for in the case)

I will look into it some more.

Thanks,
Reiji

Re: [RFC PATCH v3 02/29] KVM: arm64: Save ID registers' sanitized value per vCPU

[Eric Auger]

Hi Reiji,

On 12/4/21 2:45 AM, Reiji Watanabe wrote:
> Hi Eric,
> 
> On Thu, Dec 2, 2021 at 2:58 AM Eric Auger <eauger@redhat.com> wrote:
>>
>> Hi Reiji,
>>
>> On 11/17/21 7:43 AM, Reiji Watanabe wrote:
>>> Extend sys_regs[] of kvm_cpu_context for ID registers and save ID
>>> registers' sanitized value in the array for the vCPU at the first
>>> vCPU reset. Use the saved ones when ID registers are read by
>>> userspace (via KVM_GET_ONE_REG) or the guest.
>>>
>>> Signed-off-by: Reiji Watanabe <reijiw@google.com>
>>> ---
>>>  arch/arm64/include/asm/kvm_host.h | 10 +++++++
>>>  arch/arm64/kvm/sys_regs.c         | 43 +++++++++++++++++++------------
>>>  2 files changed, 37 insertions(+), 16 deletions(-)
>>>
>>> diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
>>> index edbe2cb21947..72db73c79403 100644
>>> --- a/arch/arm64/include/asm/kvm_host.h
>>> +++ b/arch/arm64/include/asm/kvm_host.h
>>> @@ -146,6 +146,14 @@ struct kvm_vcpu_fault_info {
>>>       u64 disr_el1;           /* Deferred [SError] Status Register */
>>>  };
>>>
>>> +/*
>>> + * (Op0, Op1, CRn, CRm, Op2) of ID registers is (3, 0, 0, crm, op2),
>>> + * where 0<=crm<8, 0<=op2<8.
>>> + */
>>> +#define KVM_ARM_ID_REG_MAX_NUM 64
>>> +#define IDREG_IDX(id)                ((sys_reg_CRm(id) << 3) | sys_reg_Op2(id))
>>> +#define IDREG_SYS_IDX(id)    (ID_REG_BASE + IDREG_IDX(id))
>>> +
>>>  enum vcpu_sysreg {
>>>       __INVALID_SYSREG__,   /* 0 is reserved as an invalid value */
>>>       MPIDR_EL1,      /* MultiProcessor Affinity Register */
>>> @@ -210,6 +218,8 @@ enum vcpu_sysreg {
>>>       CNTP_CVAL_EL0,
>>>       CNTP_CTL_EL0,
>>>
>>> +     ID_REG_BASE,
>>> +     ID_REG_END = ID_REG_BASE + KVM_ARM_ID_REG_MAX_NUM - 1,
>>>       /* Memory Tagging Extension registers */
>>>       RGSR_EL1,       /* Random Allocation Tag Seed Register */
>>>       GCR_EL1,        /* Tag Control Register */
>>> diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
>>> index e3ec1a44f94d..5608d3410660 100644
>>> --- a/arch/arm64/kvm/sys_regs.c
>>> +++ b/arch/arm64/kvm/sys_regs.c
>>> @@ -33,6 +33,8 @@
>>>
>>>  #include "trace.h"
>>>
>>> +static u64 __read_id_reg(const struct kvm_vcpu *vcpu, u32 id);
>>> +
>>>  /*
>>>   * All of this file is extremely similar to the ARM coproc.c, but the
>>>   * types are different. My gut feeling is that it should be pretty
>>> @@ -273,7 +275,7 @@ static bool trap_loregion(struct kvm_vcpu *vcpu,
>>>                         struct sys_reg_params *p,
>>>                         const struct sys_reg_desc *r)
>>>  {
>>> -     u64 val = read_sanitised_ftr_reg(SYS_ID_AA64MMFR1_EL1);
>>> +     u64 val = __read_id_reg(vcpu, SYS_ID_AA64MMFR1_EL1);
>>>       u32 sr = reg_to_encoding(r);
>>>
>>>       if (!(val & (0xfUL << ID_AA64MMFR1_LOR_SHIFT))) {
>>> @@ -1059,17 +1061,9 @@ static bool access_arch_timer(struct kvm_vcpu *vcpu,
>>>       return true;
>>>  }
>>>
>>> -/* Read a sanitised cpufeature ID register by sys_reg_desc */
>>> -static u64 read_id_reg(const struct kvm_vcpu *vcpu,
>>> -             struct sys_reg_desc const *r, bool raz)
>>> +static u64 __read_id_reg(const struct kvm_vcpu *vcpu, u32 id)
>>>  {
>>> -     u32 id = reg_to_encoding(r);
>>> -     u64 val;
>>> -
>>> -     if (raz)
>>> -             return 0;
>>> -
>>> -     val = read_sanitised_ftr_reg(id);
>>> +     u64 val = __vcpu_sys_reg(vcpu, IDREG_SYS_IDX(id));
>>>
>>>       switch (id) {
>>>       case SYS_ID_AA64PFR0_EL1:
>>> @@ -1119,6 +1113,14 @@ static u64 read_id_reg(const struct kvm_vcpu *vcpu,
>>>       return val;
>>>  }
>>>
>>> +static u64 read_id_reg(const struct kvm_vcpu *vcpu,
>>> +                    struct sys_reg_desc const *r, bool raz)
>>> +{
>>> +     u32 id = reg_to_encoding(r);
>>> +
>>> +     return raz ? 0 : __read_id_reg(vcpu, id);
>>> +}
>>> +
>>>  static unsigned int id_visibility(const struct kvm_vcpu *vcpu,
>>>                                 const struct sys_reg_desc *r)
>>>  {
>>> @@ -1178,6 +1180,16 @@ static unsigned int sve_visibility(const struct kvm_vcpu *vcpu,
>>>       return REG_HIDDEN;
>>>  }
>>>
>>> +static void reset_id_reg(struct kvm_vcpu *vcpu, const struct sys_reg_desc *rd)
>>> +{
>>> +     u32 id = reg_to_encoding(rd);
>>> +
>>> +     if (vcpu_has_reset_once(vcpu))
>>> +             return;
>> The KVM API allows to call VCPU_INIT several times (with same
>> target/feature). With above check on the second call the ID_REGS won't
>> be reset. Somehow this is aligned with target/feature behavior. However
>> if this is what we want, I think we would need to document it in the KVM
>> API doc.
> 
> Thank you for the comment.
> 
> That is what we want.  Since ID registers are read only registers,
> their values must not change across the reset.
> 
> '4.82 KVM_ARM_VCPU_INIT' in api.rst says:
> 
>   System registers: Reset to their architecturally defined
>   values as for a warm reset to EL1 (resp. SVC)
> 
> Since this reset behavior for the ID registers follows what is
> described above, I'm not sure if we need to document the reset
> behavior of the ID registers specifically.
> If KVM changes the values across the resets, I would think it
> rather needs to be documented though.

Makes sense to freeze the ID REGs on the 1st reset. Was just wondering
if we shouldn't add that the ID REG values are immutable after the 1st
VCPU_INIT.

Thanks

Eric
> 
> Thanks,
> Reiji
> 

Re: [RFC PATCH v3 03/29] KVM: arm64: Introduce struct id_reg_info

[Eric Auger]

Hi Reiji,

On 12/4/21 5:35 AM, Reiji Watanabe wrote:
> Hi Eric,
> 
> On Thu, Dec 2, 2021 at 4:51 AM Eric Auger <eauger@redhat.com> wrote:
>>
>> Hi Reiji,
>>
>> On 11/17/21 7:43 AM, Reiji Watanabe wrote:
>>> This patch lays the groundwork to make ID registers writable.
>>>
>>> Introduce struct id_reg_info for an ID register to manage the
>>> register specific control of its value for the guest, and provide set
>>> of functions commonly used for ID registers to make them writable.
>>>
>>> The id_reg_info is used to do register specific initialization,
>>> validation of the ID register and etc.  Not all ID registers must
>>> have the id_reg_info. ID registers that don't have the id_reg_info
>>> are handled in a common way that is applied to all ID registers.
>>>
>>> At present, changing an ID register from userspace is allowed only
>>> if the ID register has the id_reg_info, but that will be changed
>>> by the following patches.
>>>
>>> No ID register has the structure yet and the following patches
>>> will add the id_reg_info for some ID registers.
>>>
>>> Signed-off-by: Reiji Watanabe <reijiw@google.com>
>>> ---
>>>  arch/arm64/include/asm/sysreg.h |   1 +
>>>  arch/arm64/kvm/sys_regs.c       | 226 ++++++++++++++++++++++++++++++--
>>>  2 files changed, 218 insertions(+), 9 deletions(-)
>>>
>>> diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h
>>> index 16b3f1a1d468..597609f26331 100644
>>> --- a/arch/arm64/include/asm/sysreg.h
>>> +++ b/arch/arm64/include/asm/sysreg.h
>>> @@ -1197,6 +1197,7 @@
>>>  #define ICH_VTR_TDS_MASK     (1 << ICH_VTR_TDS_SHIFT)
>>>
>>>  #define ARM64_FEATURE_FIELD_BITS     4
>>> +#define ARM64_FEATURE_FIELD_MASK     ((1ull << ARM64_FEATURE_FIELD_BITS) - 1)
>>>
>>>  /* Create a mask for the feature bits of the specified feature. */
>>>  #define ARM64_FEATURE_MASK(x)        (GENMASK_ULL(x##_SHIFT + ARM64_FEATURE_FIELD_BITS - 1, x##_SHIFT))
>>> diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
>>> index 5608d3410660..1552cd5581b7 100644
>>> --- a/arch/arm64/kvm/sys_regs.c
>>> +++ b/arch/arm64/kvm/sys_regs.c
>>> @@ -265,6 +265,181 @@ static bool trap_raz_wi(struct kvm_vcpu *vcpu,
>>>               return read_zero(vcpu, p);
>>>  }
>>>
>>> +/*
>>> + * A value for FCT_LOWER_SAFE must be zero and changing that will affect
>>> + * ftr_check_types of id_reg_info.
>>> + */
>>> +enum feature_check_type {
>>> +     FCT_LOWER_SAFE = 0,
>>> +     FCT_HIGHER_SAFE,
>>> +     FCT_HIGHER_OR_ZERO_SAFE,
>>> +     FCT_EXACT,
>>> +     FCT_EXACT_OR_ZERO_SAFE,
>>> +     FCT_IGNORE,     /* Don't check (any value is fine) */
>>> +};
>>> +
>>> +static int arm64_check_feature_one(enum feature_check_type type, int val,
>>> +                                int limit)
>>> +{
>>> +     bool is_safe = false;
>>> +
>>> +     if (val == limit)
>>> +             return 0;
>>> +
>>> +     switch (type) {
>>> +     case FCT_LOWER_SAFE:
>>> +             is_safe = (val <= limit);
>>> +             break;
>>> +     case FCT_HIGHER_OR_ZERO_SAFE:
>>> +             if (val == 0) {
>>> +                     is_safe = true;
>>> +                     break;
>>> +             }
>>> +             fallthrough;
>>> +     case FCT_HIGHER_SAFE:
>>> +             is_safe = (val >= limit);
>>> +             break;
>>> +     case FCT_EXACT:
>>> +             break;
>>> +     case FCT_EXACT_OR_ZERO_SAFE:
>>> +             is_safe = (val == 0);
>>> +             break;
>>> +     case FCT_IGNORE:
>>> +             is_safe = true;
>>> +             break;
>>> +     default:
>>> +             WARN_ONCE(1, "Unexpected feature_check_type (%d)\n", type);
>>> +             break;
>>> +     }
>>> +
>>> +     return is_safe ? 0 : -1;
>>> +}
>>> +
>>> +#define      FCT_TYPE_MASK           0x7
>>> +#define      FCT_TYPE_SHIFT          1
>>> +#define      FCT_SIGN_MASK           0x1
>>> +#define      FCT_SIGN_SHIFT          0
>>> +#define      FCT_TYPE(val)   ((val >> FCT_TYPE_SHIFT) & FCT_TYPE_MASK)
>>> +#define      FCT_SIGN(val)   ((val >> FCT_SIGN_SHIFT) & FCT_SIGN_MASK)
>>> +
>>> +#define      MAKE_FCT(shift, type, sign)                             \
>>> +     ((u64)((((type) & FCT_TYPE_MASK) << FCT_TYPE_SHIFT) |   \
>>> +            (((sign) & FCT_SIGN_MASK) << FCT_SIGN_SHIFT)) << (shift))
>>> +
>>> +/* For signed field */
>>> +#define      S_FCT(shift, type)      MAKE_FCT(shift, type, 1)
>>> +/* For unigned field */
>>> +#define      U_FCT(shift, type)      MAKE_FCT(shift, type, 0)
>>> +
>>> +/*
>>> + * @val and @lim are both a value of the ID register. The function checks
>>> + * if all features indicated in @val can be supported for guests on the host,
>>> + * which supports features indicated in @lim. @check_types indicates how
>>> + * features in the ID register needs to be checked.
>>> + * See comments for id_reg_info's ftr_check_types field for more detail.
>>> + */
>>> +static int arm64_check_features(u64 check_types, u64 val, u64 lim)
>>> +{
>>> +     int i;
>>> +
>>> +     for (i = 0; i < 64; i += ARM64_FEATURE_FIELD_BITS) {
>>> +             u8 ftr_check = (check_types >> i) & ARM64_FEATURE_FIELD_MASK;
>>> +             bool is_sign = FCT_SIGN(ftr_check);
>>> +             enum feature_check_type fctype = FCT_TYPE(ftr_check);
>>> +             int fval, flim, ret;
>>> +
>>> +             fval = cpuid_feature_extract_field(val, i, is_sign);
>>> +             flim = cpuid_feature_extract_field(lim, i, is_sign);
>>> +
>>> +             ret = arm64_check_feature_one(fctype, fval, flim);
>>> +             if (ret)
>>> +                     return -E2BIG;
>>> +     }
>>> +     return 0;
>>> +}
>>> +
>>> +struct id_reg_info {
>>> +     u32     sys_reg;        /* Register ID */
>>> +
>>> +     /*
>>> +      * Limit value of the register for a vcpu. The value is the sanitized
>>> +      * system value with bits cleared for unsupported features for the
>>> +      * guest.
>>> +      */
>>> +     u64     vcpu_limit_val;
>>> +
>>> +     /*
>>> +      * The ftr_check_types is comprised of a set of 4 bits fields.
>>> +      * Each 4 bits field is for a feature indicated by the same bits
>>> +      * field of the ID register and indicates how the feature support
>>> +      * for guests needs to be checked.
>>> +      * The bit 0 indicates that the corresponding ID register field
>>> +      * is signed(1) or unsigned(0).
>>> +      * The bits [3:1] hold feature_check_type for the field.
>>> +      * If all zero, all features in the ID register are treated as unsigned
>>> +      * fields and checked based on Principles of the ID scheme for fields
>>> +      * in ID registers (FCT_LOWER_SAFE of feature_check_type).
>>> +      */
>>> +     u64     ftr_check_types;
>>> +
>>> +     /* Initialization function of the id_reg_info */
>>> +     void (*init)(struct id_reg_info *id_reg);
>>> +
>>> +     /* Register specific validation function */
>>> +     int (*validate)(struct kvm_vcpu *vcpu, const struct id_reg_info *id_reg,
>>> +                     u64 val);
>>> +
>>> +     /* Return the reset value of the register for the vCPU */
>>> +     u64 (*get_reset_val)(struct kvm_vcpu *vcpu,
>>> +                          const struct id_reg_info *id_reg);
>> It is unclear to me why we need 2 different callbacks, ie. init and
>> get_reset_val. ID_REGS can only be accessed from user space after the
>> vcpu reset, right? So couldn't we have a single cb instead of this
>> overwrite mechanism?
> 
> Thank you for the comment.
> 
> What the init() does needs to be done just once.
> It initializes the id_reg_info itself (not for the ID register of vCPU).
> And the data initialized by the init() is used not just for the
> overwrite mechanism at the vcpu reset but for other purposes as well.
> 
> What the get_reset_val does needs to be done for every initial vCPU reset.
> It provides the initial value for the vCPU, which depends on its feature
> configuration that is configured by KVM_ARM_VCPU_INIT (or other APIs).
> 
> Of course there are other ways to achieve the same, and it's entirely
> possible to have a single function though.  I just chose to use a
> separate function for each of those two different purposes.

OK fair enough. Was thinking that maybe it would simplify the code if we
had a single 'reset" cb but up to you.

Thanks

Eric
> 
> Thanks,
> Reiji
> 

Re: [RFC PATCH v3 04/29] KVM: arm64: Make ID_AA64PFR0_EL1 writable

[Eric Auger]

Hi Reiji,

On 12/4/21 8:59 AM, Reiji Watanabe wrote:
> Hi Eric,
> 
> On Thu, Dec 2, 2021 at 5:02 AM Eric Auger <eauger@redhat.com> wrote:
>>
>> Hi Reiji,
>>
>> On 11/30/21 2:29 AM, Reiji Watanabe wrote:
>>> Hi Eric,
>>>
>>> On Thu, Nov 25, 2021 at 7:35 AM Eric Auger <eauger@redhat.com> wrote:
>>>>
>>>> Hi Reiji,
>>>>
>>>> On 11/17/21 7:43 AM, Reiji Watanabe wrote:
>>>>> This patch adds id_reg_info for ID_AA64PFR0_EL1 to make it writable by
>>>>> userspace.
>>>>>
>>>>> The CSV2/CSV3 fields of the register were already writable and values
>>>>> that were written for them affected all vCPUs before. Now they only
>>>>> affect the vCPU.
>>>>> Return an error if userspace tries to set SVE/GIC field of the register
>>>>> to a value that conflicts with SVE/GIC configuration for the guest.
>>>>> SIMD/FP/SVE fields of the requested value are validated according to
>>>>> Arm ARM.
>>>>>
>>>>> Signed-off-by: Reiji Watanabe <reijiw@google.com>
>>>>> ---
>>>>>  arch/arm64/kvm/sys_regs.c | 159 ++++++++++++++++++++++++--------------
>>>>>  1 file changed, 103 insertions(+), 56 deletions(-)
>>>>>
>>>>> diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
>>>>> index 1552cd5581b7..35400869067a 100644
>>>>> --- a/arch/arm64/kvm/sys_regs.c
>>>>> +++ b/arch/arm64/kvm/sys_regs.c
>>>>> @@ -401,6 +401,92 @@ static void id_reg_info_init(struct id_reg_info *id_reg)
>>>>>               id_reg->init(id_reg);
>>>>>  }
>>>>>
>>>>> +#define      kvm_has_gic3(kvm)               \
>>>>> +     (irqchip_in_kernel(kvm) &&      \
>>>>> +      (kvm)->arch.vgic.vgic_model == KVM_DEV_TYPE_ARM_VGIC_V3)
>>>> you may move this macro to kvm/arm_vgic.h as this may be used in
>>>> vgic/vgic-v3.c too
>>>
>>> Thank you for the suggestion. I will move that to kvm/arm_vgic.h.
>>>
>>>
>>>>> +
>>>>> +static int validate_id_aa64pfr0_el1(struct kvm_vcpu *vcpu,
>>>>> +                                 const struct id_reg_info *id_reg, u64 val)
>>>>> +{
>>>>> +     int fp, simd;
>>>>> +     bool vcpu_has_sve = vcpu_has_sve(vcpu);
>>>>> +     bool pfr0_has_sve = id_aa64pfr0_sve(val);
>>>>> +     int gic;
>>>>> +
>>>>> +     simd = cpuid_feature_extract_signed_field(val, ID_AA64PFR0_ASIMD_SHIFT);
>>>>> +     fp = cpuid_feature_extract_signed_field(val, ID_AA64PFR0_FP_SHIFT);
>>>>> +     if (simd != fp)
>>>>> +             return -EINVAL;
>>>>> +
>>>>> +     /* fp must be supported when sve is supported */
>>>>> +     if (pfr0_has_sve && (fp < 0))
>>>>> +             return -EINVAL;
>>>>> +
>>>>> +     /* Check if there is a conflict with a request via KVM_ARM_VCPU_INIT */
>>>>> +     if (vcpu_has_sve ^ pfr0_has_sve)
>>>>> +             return -EPERM;
>>>>> +
>>>>> +     gic = cpuid_feature_extract_unsigned_field(val, ID_AA64PFR0_GIC_SHIFT);
>>>>> +     if ((gic > 0) ^ kvm_has_gic3(vcpu->kvm))
>>>>> +             return -EPERM;
>>>>
>>>> Sometimes from a given architecture version, some lower values are not
>>>> allowed. For instance from ARMv8.5 onlt 1 is permitted for CSV3.
>>>> Shouldn't we handle that kind of check?
>>>
>>> As far as I know, there is no way for guests to identify the
>>> architecture revision (e.g. v8.1, v8.2, etc).  It might be able
>>> to indirectly infer the revision though (from features that are
>>> available or etc).
>>
>> OK. That sounds weird to me as we do many checks accross different IDREG
>> settings but we may eventually have a wrong "CPU model" exposed by the
>> user space violating those spec revision minima. Shouldn't we introduce
>> some way for the userspace to provide his requirements? via new VCPU
>> targets for instance?
> 
> Thank you for sharing your thoughts and providing the suggestion !
> 
> Does the "new vCPU targets" mean Armv8.0, armv8.1, and so on ?

Yeah my suggestion probably is not a good idea, ie. introducing such
VCPU targets. I was simply confused by the fact we introduce in this
series quite intricate consistency checks but given the fact we miss the
spec rev information we are not exhaustive in terms of checking. So it
is sometimes difficult to review against the spec.

> 
> The ID registers' consistency checking in the series is to not
> promise more to userspace than what KVM (on the host) can provide,
> and to not expose ID register values that are not supported on
> any ARM v8 architecture for guests (I think those are what the
> current KVM is trying to assure).  I'm not trying to have KVM
> provide full consistency checking of ID registers to completely
> prevent userspace's bugs in setting ID registers.
> 
> I agree that it's quite possible that userspace exposes such wrong
> CPU models, and KVM's providing more consistency checking would be
> nicer in general.  But should it be KVM's responsibility to completely
> prevent such ID register issues due to userspace bugs ?
> 
> Honestly, I'm a bit reluctant to do that so far yet:)

understood. I will look at the spec in more details on my next review
cycle. Looking forward to reviewing the next version ;-)

Thanks

Eric
> If that is something useful that userspace or we (KVM developers)
> really want or need, or such userspace issue could affect KVM,
> I would be happy to add such extra consistency checking though.
> 
> Thanks,
> Reiji
> 

Re: [RFC PATCH v3 02/29] KVM: arm64: Save ID registers' sanitized value per vCPU

[Reiji Watanabe]

Hi Eric,

On Tue, Dec 7, 2021 at 1:34 AM Eric Auger <eauger@redhat.com> wrote:
>
> Hi Reiji,
>
> On 12/4/21 2:45 AM, Reiji Watanabe wrote:
> > Hi Eric,
> >
> > On Thu, Dec 2, 2021 at 2:58 AM Eric Auger <eauger@redhat.com> wrote:
> >>
> >> Hi Reiji,
> >>
> >> On 11/17/21 7:43 AM, Reiji Watanabe wrote:
> >>> Extend sys_regs[] of kvm_cpu_context for ID registers and save ID
> >>> registers' sanitized value in the array for the vCPU at the first
> >>> vCPU reset. Use the saved ones when ID registers are read by
> >>> userspace (via KVM_GET_ONE_REG) or the guest.
> >>>
> >>> Signed-off-by: Reiji Watanabe <reijiw@google.com>
> >>> ---
> >>>  arch/arm64/include/asm/kvm_host.h | 10 +++++++
> >>>  arch/arm64/kvm/sys_regs.c         | 43 +++++++++++++++++++------------
> >>>  2 files changed, 37 insertions(+), 16 deletions(-)
> >>>
> >>> diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
> >>> index edbe2cb21947..72db73c79403 100644
> >>> --- a/arch/arm64/include/asm/kvm_host.h
> >>> +++ b/arch/arm64/include/asm/kvm_host.h
> >>> @@ -146,6 +146,14 @@ struct kvm_vcpu_fault_info {
> >>>       u64 disr_el1;           /* Deferred [SError] Status Register */
> >>>  };
> >>>
> >>> +/*
> >>> + * (Op0, Op1, CRn, CRm, Op2) of ID registers is (3, 0, 0, crm, op2),
> >>> + * where 0<=crm<8, 0<=op2<8.
> >>> + */
> >>> +#define KVM_ARM_ID_REG_MAX_NUM 64
> >>> +#define IDREG_IDX(id)                ((sys_reg_CRm(id) << 3) | sys_reg_Op2(id))
> >>> +#define IDREG_SYS_IDX(id)    (ID_REG_BASE + IDREG_IDX(id))
> >>> +
> >>>  enum vcpu_sysreg {
> >>>       __INVALID_SYSREG__,   /* 0 is reserved as an invalid value */
> >>>       MPIDR_EL1,      /* MultiProcessor Affinity Register */
> >>> @@ -210,6 +218,8 @@ enum vcpu_sysreg {
> >>>       CNTP_CVAL_EL0,
> >>>       CNTP_CTL_EL0,
> >>>
> >>> +     ID_REG_BASE,
> >>> +     ID_REG_END = ID_REG_BASE + KVM_ARM_ID_REG_MAX_NUM - 1,
> >>>       /* Memory Tagging Extension registers */
> >>>       RGSR_EL1,       /* Random Allocation Tag Seed Register */
> >>>       GCR_EL1,        /* Tag Control Register */
> >>> diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
> >>> index e3ec1a44f94d..5608d3410660 100644
> >>> --- a/arch/arm64/kvm/sys_regs.c
> >>> +++ b/arch/arm64/kvm/sys_regs.c
> >>> @@ -33,6 +33,8 @@
> >>>
> >>>  #include "trace.h"
> >>>
> >>> +static u64 __read_id_reg(const struct kvm_vcpu *vcpu, u32 id);
> >>> +
> >>>  /*
> >>>   * All of this file is extremely similar to the ARM coproc.c, but the
> >>>   * types are different. My gut feeling is that it should be pretty
> >>> @@ -273,7 +275,7 @@ static bool trap_loregion(struct kvm_vcpu *vcpu,
> >>>                         struct sys_reg_params *p,
> >>>                         const struct sys_reg_desc *r)
> >>>  {
> >>> -     u64 val = read_sanitised_ftr_reg(SYS_ID_AA64MMFR1_EL1);
> >>> +     u64 val = __read_id_reg(vcpu, SYS_ID_AA64MMFR1_EL1);
> >>>       u32 sr = reg_to_encoding(r);
> >>>
> >>>       if (!(val & (0xfUL << ID_AA64MMFR1_LOR_SHIFT))) {
> >>> @@ -1059,17 +1061,9 @@ static bool access_arch_timer(struct kvm_vcpu *vcpu,
> >>>       return true;
> >>>  }
> >>>
> >>> -/* Read a sanitised cpufeature ID register by sys_reg_desc */
> >>> -static u64 read_id_reg(const struct kvm_vcpu *vcpu,
> >>> -             struct sys_reg_desc const *r, bool raz)
> >>> +static u64 __read_id_reg(const struct kvm_vcpu *vcpu, u32 id)
> >>>  {
> >>> -     u32 id = reg_to_encoding(r);
> >>> -     u64 val;
> >>> -
> >>> -     if (raz)
> >>> -             return 0;
> >>> -
> >>> -     val = read_sanitised_ftr_reg(id);
> >>> +     u64 val = __vcpu_sys_reg(vcpu, IDREG_SYS_IDX(id));
> >>>
> >>>       switch (id) {
> >>>       case SYS_ID_AA64PFR0_EL1:
> >>> @@ -1119,6 +1113,14 @@ static u64 read_id_reg(const struct kvm_vcpu *vcpu,
> >>>       return val;
> >>>  }
> >>>
> >>> +static u64 read_id_reg(const struct kvm_vcpu *vcpu,
> >>> +                    struct sys_reg_desc const *r, bool raz)
> >>> +{
> >>> +     u32 id = reg_to_encoding(r);
> >>> +
> >>> +     return raz ? 0 : __read_id_reg(vcpu, id);
> >>> +}
> >>> +
> >>>  static unsigned int id_visibility(const struct kvm_vcpu *vcpu,
> >>>                                 const struct sys_reg_desc *r)
> >>>  {
> >>> @@ -1178,6 +1180,16 @@ static unsigned int sve_visibility(const struct kvm_vcpu *vcpu,
> >>>       return REG_HIDDEN;
> >>>  }
> >>>
> >>> +static void reset_id_reg(struct kvm_vcpu *vcpu, const struct sys_reg_desc *rd)
> >>> +{
> >>> +     u32 id = reg_to_encoding(rd);
> >>> +
> >>> +     if (vcpu_has_reset_once(vcpu))
> >>> +             return;
> >> The KVM API allows to call VCPU_INIT several times (with same
> >> target/feature). With above check on the second call the ID_REGS won't
> >> be reset. Somehow this is aligned with target/feature behavior. However
> >> if this is what we want, I think we would need to document it in the KVM
> >> API doc.
> >
> > Thank you for the comment.
> >
> > That is what we want.  Since ID registers are read only registers,
> > their values must not change across the reset.
> >
> > '4.82 KVM_ARM_VCPU_INIT' in api.rst says:
> >
> >   System registers: Reset to their architecturally defined
> >   values as for a warm reset to EL1 (resp. SVC)
> >
> > Since this reset behavior for the ID registers follows what is
> > described above, I'm not sure if we need to document the reset
> > behavior of the ID registers specifically.
> > If KVM changes the values across the resets, I would think it
> > rather needs to be documented though.
>
> Makes sense to freeze the ID REGs on the 1st reset. Was just wondering
> if we shouldn't add that the ID REG values are immutable after the 1st
> VCPU_INIT.

> Makes sense to freeze the ID REGs on the 1st reset. Was just wondering
> if we shouldn't add that the ID REG values are immutable after the 1st
> VCPU_INIT.

Even after the 1st VCPU_INIT, ID REG values can be changed by
KVM_SET_ONE_REG (KVM_SET_ONE_REG/KVM_GET_ONE_REG are allowed
only after the 1st VCPU_INIT).

The ID REG values are immutable after the 1st KVM_RUN,
and I think we should document that.  Is that what you meant ?

Thanks,
Reiji

Re: [RFC PATCH v3 02/29] KVM: arm64: Save ID registers' sanitized value per vCPU

[Eric Auger]

Hi Reiji,

On 12/8/21 6:57 AM, Reiji Watanabe wrote:
> Hi Eric,
> 
> On Tue, Dec 7, 2021 at 1:34 AM Eric Auger <eauger@redhat.com> wrote:
>>
>> Hi Reiji,
>>
>> On 12/4/21 2:45 AM, Reiji Watanabe wrote:
>>> Hi Eric,
>>>
>>> On Thu, Dec 2, 2021 at 2:58 AM Eric Auger <eauger@redhat.com> wrote:
>>>>
>>>> Hi Reiji,
>>>>
>>>> On 11/17/21 7:43 AM, Reiji Watanabe wrote:
>>>>> Extend sys_regs[] of kvm_cpu_context for ID registers and save ID
>>>>> registers' sanitized value in the array for the vCPU at the first
>>>>> vCPU reset. Use the saved ones when ID registers are read by
>>>>> userspace (via KVM_GET_ONE_REG) or the guest.
>>>>>
>>>>> Signed-off-by: Reiji Watanabe <reijiw@google.com>
>>>>> ---
>>>>>  arch/arm64/include/asm/kvm_host.h | 10 +++++++
>>>>>  arch/arm64/kvm/sys_regs.c         | 43 +++++++++++++++++++------------
>>>>>  2 files changed, 37 insertions(+), 16 deletions(-)
>>>>>
>>>>> diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
>>>>> index edbe2cb21947..72db73c79403 100644
>>>>> --- a/arch/arm64/include/asm/kvm_host.h
>>>>> +++ b/arch/arm64/include/asm/kvm_host.h
>>>>> @@ -146,6 +146,14 @@ struct kvm_vcpu_fault_info {
>>>>>       u64 disr_el1;           /* Deferred [SError] Status Register */
>>>>>  };
>>>>>
>>>>> +/*
>>>>> + * (Op0, Op1, CRn, CRm, Op2) of ID registers is (3, 0, 0, crm, op2),
>>>>> + * where 0<=crm<8, 0<=op2<8.
>>>>> + */
>>>>> +#define KVM_ARM_ID_REG_MAX_NUM 64
>>>>> +#define IDREG_IDX(id)                ((sys_reg_CRm(id) << 3) | sys_reg_Op2(id))
>>>>> +#define IDREG_SYS_IDX(id)    (ID_REG_BASE + IDREG_IDX(id))
>>>>> +
>>>>>  enum vcpu_sysreg {
>>>>>       __INVALID_SYSREG__,   /* 0 is reserved as an invalid value */
>>>>>       MPIDR_EL1,      /* MultiProcessor Affinity Register */
>>>>> @@ -210,6 +218,8 @@ enum vcpu_sysreg {
>>>>>       CNTP_CVAL_EL0,
>>>>>       CNTP_CTL_EL0,
>>>>>
>>>>> +     ID_REG_BASE,
>>>>> +     ID_REG_END = ID_REG_BASE + KVM_ARM_ID_REG_MAX_NUM - 1,
>>>>>       /* Memory Tagging Extension registers */
>>>>>       RGSR_EL1,       /* Random Allocation Tag Seed Register */
>>>>>       GCR_EL1,        /* Tag Control Register */
>>>>> diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
>>>>> index e3ec1a44f94d..5608d3410660 100644
>>>>> --- a/arch/arm64/kvm/sys_regs.c
>>>>> +++ b/arch/arm64/kvm/sys_regs.c
>>>>> @@ -33,6 +33,8 @@
>>>>>
>>>>>  #include "trace.h"
>>>>>
>>>>> +static u64 __read_id_reg(const struct kvm_vcpu *vcpu, u32 id);
>>>>> +
>>>>>  /*
>>>>>   * All of this file is extremely similar to the ARM coproc.c, but the
>>>>>   * types are different. My gut feeling is that it should be pretty
>>>>> @@ -273,7 +275,7 @@ static bool trap_loregion(struct kvm_vcpu *vcpu,
>>>>>                         struct sys_reg_params *p,
>>>>>                         const struct sys_reg_desc *r)
>>>>>  {
>>>>> -     u64 val = read_sanitised_ftr_reg(SYS_ID_AA64MMFR1_EL1);
>>>>> +     u64 val = __read_id_reg(vcpu, SYS_ID_AA64MMFR1_EL1);
>>>>>       u32 sr = reg_to_encoding(r);
>>>>>
>>>>>       if (!(val & (0xfUL << ID_AA64MMFR1_LOR_SHIFT))) {
>>>>> @@ -1059,17 +1061,9 @@ static bool access_arch_timer(struct kvm_vcpu *vcpu,
>>>>>       return true;
>>>>>  }
>>>>>
>>>>> -/* Read a sanitised cpufeature ID register by sys_reg_desc */
>>>>> -static u64 read_id_reg(const struct kvm_vcpu *vcpu,
>>>>> -             struct sys_reg_desc const *r, bool raz)
>>>>> +static u64 __read_id_reg(const struct kvm_vcpu *vcpu, u32 id)
>>>>>  {
>>>>> -     u32 id = reg_to_encoding(r);
>>>>> -     u64 val;
>>>>> -
>>>>> -     if (raz)
>>>>> -             return 0;
>>>>> -
>>>>> -     val = read_sanitised_ftr_reg(id);
>>>>> +     u64 val = __vcpu_sys_reg(vcpu, IDREG_SYS_IDX(id));
>>>>>
>>>>>       switch (id) {
>>>>>       case SYS_ID_AA64PFR0_EL1:
>>>>> @@ -1119,6 +1113,14 @@ static u64 read_id_reg(const struct kvm_vcpu *vcpu,
>>>>>       return val;
>>>>>  }
>>>>>
>>>>> +static u64 read_id_reg(const struct kvm_vcpu *vcpu,
>>>>> +                    struct sys_reg_desc const *r, bool raz)
>>>>> +{
>>>>> +     u32 id = reg_to_encoding(r);
>>>>> +
>>>>> +     return raz ? 0 : __read_id_reg(vcpu, id);
>>>>> +}
>>>>> +
>>>>>  static unsigned int id_visibility(const struct kvm_vcpu *vcpu,
>>>>>                                 const struct sys_reg_desc *r)
>>>>>  {
>>>>> @@ -1178,6 +1180,16 @@ static unsigned int sve_visibility(const struct kvm_vcpu *vcpu,
>>>>>       return REG_HIDDEN;
>>>>>  }
>>>>>
>>>>> +static void reset_id_reg(struct kvm_vcpu *vcpu, const struct sys_reg_desc *rd)
>>>>> +{
>>>>> +     u32 id = reg_to_encoding(rd);
>>>>> +
>>>>> +     if (vcpu_has_reset_once(vcpu))
>>>>> +             return;
>>>> The KVM API allows to call VCPU_INIT several times (with same
>>>> target/feature). With above check on the second call the ID_REGS won't
>>>> be reset. Somehow this is aligned with target/feature behavior. However
>>>> if this is what we want, I think we would need to document it in the KVM
>>>> API doc.
>>>
>>> Thank you for the comment.
>>>
>>> That is what we want.  Since ID registers are read only registers,
>>> their values must not change across the reset.
>>>
>>> '4.82 KVM_ARM_VCPU_INIT' in api.rst says:
>>>
>>>   System registers: Reset to their architecturally defined
>>>   values as for a warm reset to EL1 (resp. SVC)
>>>
>>> Since this reset behavior for the ID registers follows what is
>>> described above, I'm not sure if we need to document the reset
>>> behavior of the ID registers specifically.
>>> If KVM changes the values across the resets, I would think it
>>> rather needs to be documented though.
>>
>> Makes sense to freeze the ID REGs on the 1st reset. Was just wondering
>> if we shouldn't add that the ID REG values are immutable after the 1st
>> VCPU_INIT.
> 
>> Makes sense to freeze the ID REGs on the 1st reset. Was just wondering
>> if we shouldn't add that the ID REG values are immutable after the 1st
>> VCPU_INIT.
> 
> Even after the 1st VCPU_INIT, ID REG values can be changed by
> KVM_SET_ONE_REG (KVM_SET_ONE_REG/KVM_GET_ONE_REG are allowed
> only after the 1st VCPU_INIT).
> 
> The ID REG values are immutable after the 1st KVM_RUN,
> and I think we should document that.  Is that what you meant ?
Yes that's what I meant sorry.

Eric
> 
> Thanks,
> Reiji
> 

Re: [RFC PATCH v3 02/29] KVM: arm64: Save ID registers' sanitized value per vCPU

[Reiji Watanabe]

Hi Eric,

On Tue, Dec 7, 2021 at 11:09 PM Eric Auger <eauger@redhat.com> wrote:
>
> Hi Reiji,
>
> On 12/8/21 6:57 AM, Reiji Watanabe wrote:
> > Hi Eric,
> >
> > On Tue, Dec 7, 2021 at 1:34 AM Eric Auger <eauger@redhat.com> wrote:
> >>
> >> Hi Reiji,
> >>
> >> On 12/4/21 2:45 AM, Reiji Watanabe wrote:
> >>> Hi Eric,
> >>>
> >>> On Thu, Dec 2, 2021 at 2:58 AM Eric Auger <eauger@redhat.com> wrote:
> >>>>
> >>>> Hi Reiji,
> >>>>
> >>>> On 11/17/21 7:43 AM, Reiji Watanabe wrote:
> >>>>> Extend sys_regs[] of kvm_cpu_context for ID registers and save ID
> >>>>> registers' sanitized value in the array for the vCPU at the first
> >>>>> vCPU reset. Use the saved ones when ID registers are read by
> >>>>> userspace (via KVM_GET_ONE_REG) or the guest.
> >>>>>
> >>>>> Signed-off-by: Reiji Watanabe <reijiw@google.com>
> >>>>> ---
> >>>>>  arch/arm64/include/asm/kvm_host.h | 10 +++++++
> >>>>>  arch/arm64/kvm/sys_regs.c         | 43 +++++++++++++++++++------------
> >>>>>  2 files changed, 37 insertions(+), 16 deletions(-)
> >>>>>
> >>>>> diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
> >>>>> index edbe2cb21947..72db73c79403 100644
> >>>>> --- a/arch/arm64/include/asm/kvm_host.h
> >>>>> +++ b/arch/arm64/include/asm/kvm_host.h
> >>>>> @@ -146,6 +146,14 @@ struct kvm_vcpu_fault_info {
> >>>>>       u64 disr_el1;           /* Deferred [SError] Status Register */
> >>>>>  };
> >>>>>
> >>>>> +/*
> >>>>> + * (Op0, Op1, CRn, CRm, Op2) of ID registers is (3, 0, 0, crm, op2),
> >>>>> + * where 0<=crm<8, 0<=op2<8.
> >>>>> + */
> >>>>> +#define KVM_ARM_ID_REG_MAX_NUM 64
> >>>>> +#define IDREG_IDX(id)                ((sys_reg_CRm(id) << 3) | sys_reg_Op2(id))
> >>>>> +#define IDREG_SYS_IDX(id)    (ID_REG_BASE + IDREG_IDX(id))
> >>>>> +
> >>>>>  enum vcpu_sysreg {
> >>>>>       __INVALID_SYSREG__,   /* 0 is reserved as an invalid value */
> >>>>>       MPIDR_EL1,      /* MultiProcessor Affinity Register */
> >>>>> @@ -210,6 +218,8 @@ enum vcpu_sysreg {
> >>>>>       CNTP_CVAL_EL0,
> >>>>>       CNTP_CTL_EL0,
> >>>>>
> >>>>> +     ID_REG_BASE,
> >>>>> +     ID_REG_END = ID_REG_BASE + KVM_ARM_ID_REG_MAX_NUM - 1,
> >>>>>       /* Memory Tagging Extension registers */
> >>>>>       RGSR_EL1,       /* Random Allocation Tag Seed Register */
> >>>>>       GCR_EL1,        /* Tag Control Register */
> >>>>> diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
> >>>>> index e3ec1a44f94d..5608d3410660 100644
> >>>>> --- a/arch/arm64/kvm/sys_regs.c
> >>>>> +++ b/arch/arm64/kvm/sys_regs.c
> >>>>> @@ -33,6 +33,8 @@
> >>>>>
> >>>>>  #include "trace.h"
> >>>>>
> >>>>> +static u64 __read_id_reg(const struct kvm_vcpu *vcpu, u32 id);
> >>>>> +
> >>>>>  /*
> >>>>>   * All of this file is extremely similar to the ARM coproc.c, but the
> >>>>>   * types are different. My gut feeling is that it should be pretty
> >>>>> @@ -273,7 +275,7 @@ static bool trap_loregion(struct kvm_vcpu *vcpu,
> >>>>>                         struct sys_reg_params *p,
> >>>>>                         const struct sys_reg_desc *r)
> >>>>>  {
> >>>>> -     u64 val = read_sanitised_ftr_reg(SYS_ID_AA64MMFR1_EL1);
> >>>>> +     u64 val = __read_id_reg(vcpu, SYS_ID_AA64MMFR1_EL1);
> >>>>>       u32 sr = reg_to_encoding(r);
> >>>>>
> >>>>>       if (!(val & (0xfUL << ID_AA64MMFR1_LOR_SHIFT))) {
> >>>>> @@ -1059,17 +1061,9 @@ static bool access_arch_timer(struct kvm_vcpu *vcpu,
> >>>>>       return true;
> >>>>>  }
> >>>>>
> >>>>> -/* Read a sanitised cpufeature ID register by sys_reg_desc */
> >>>>> -static u64 read_id_reg(const struct kvm_vcpu *vcpu,
> >>>>> -             struct sys_reg_desc const *r, bool raz)
> >>>>> +static u64 __read_id_reg(const struct kvm_vcpu *vcpu, u32 id)
> >>>>>  {
> >>>>> -     u32 id = reg_to_encoding(r);
> >>>>> -     u64 val;
> >>>>> -
> >>>>> -     if (raz)
> >>>>> -             return 0;
> >>>>> -
> >>>>> -     val = read_sanitised_ftr_reg(id);
> >>>>> +     u64 val = __vcpu_sys_reg(vcpu, IDREG_SYS_IDX(id));
> >>>>>
> >>>>>       switch (id) {
> >>>>>       case SYS_ID_AA64PFR0_EL1:
> >>>>> @@ -1119,6 +1113,14 @@ static u64 read_id_reg(const struct kvm_vcpu *vcpu,
> >>>>>       return val;
> >>>>>  }
> >>>>>
> >>>>> +static u64 read_id_reg(const struct kvm_vcpu *vcpu,
> >>>>> +                    struct sys_reg_desc const *r, bool raz)
> >>>>> +{
> >>>>> +     u32 id = reg_to_encoding(r);
> >>>>> +
> >>>>> +     return raz ? 0 : __read_id_reg(vcpu, id);
> >>>>> +}
> >>>>> +
> >>>>>  static unsigned int id_visibility(const struct kvm_vcpu *vcpu,
> >>>>>                                 const struct sys_reg_desc *r)
> >>>>>  {
> >>>>> @@ -1178,6 +1180,16 @@ static unsigned int sve_visibility(const struct kvm_vcpu *vcpu,
> >>>>>       return REG_HIDDEN;
> >>>>>  }
> >>>>>
> >>>>> +static void reset_id_reg(struct kvm_vcpu *vcpu, const struct sys_reg_desc *rd)
> >>>>> +{
> >>>>> +     u32 id = reg_to_encoding(rd);
> >>>>> +
> >>>>> +     if (vcpu_has_reset_once(vcpu))
> >>>>> +             return;
> >>>> The KVM API allows to call VCPU_INIT several times (with same
> >>>> target/feature). With above check on the second call the ID_REGS won't
> >>>> be reset. Somehow this is aligned with target/feature behavior. However
> >>>> if this is what we want, I think we would need to document it in the KVM
> >>>> API doc.
> >>>
> >>> Thank you for the comment.
> >>>
> >>> That is what we want.  Since ID registers are read only registers,
> >>> their values must not change across the reset.
> >>>
> >>> '4.82 KVM_ARM_VCPU_INIT' in api.rst says:
> >>>
> >>>   System registers: Reset to their architecturally defined
> >>>   values as for a warm reset to EL1 (resp. SVC)
> >>>
> >>> Since this reset behavior for the ID registers follows what is
> >>> described above, I'm not sure if we need to document the reset
> >>> behavior of the ID registers specifically.
> >>> If KVM changes the values across the resets, I would think it
> >>> rather needs to be documented though.
> >>
> >> Makes sense to freeze the ID REGs on the 1st reset. Was just wondering
> >> if we shouldn't add that the ID REG values are immutable after the 1st
> >> VCPU_INIT.
> >
> >> Makes sense to freeze the ID REGs on the 1st reset. Was just wondering
> >> if we shouldn't add that the ID REG values are immutable after the 1st
> >> VCPU_INIT.
> >
> > Even after the 1st VCPU_INIT, ID REG values can be changed by
> > KVM_SET_ONE_REG (KVM_SET_ONE_REG/KVM_GET_ONE_REG are allowed
> > only after the 1st VCPU_INIT).
> >
> > The ID REG values are immutable after the 1st KVM_RUN,
> > and I think we should document that.  Is that what you meant ?
> Yes that's what I meant sorry.

Thank you for the clarification ! I will document that.

Thanks,
Reiji