[PATCH v3 0/6] target/i386: add support for FRED

[PATCH v3 0/6] target/i386: add support for FRED

[Xin Li]

This patch set adds support for the Intel flexible return and event delivery
(FRED) architecture to allow Qemu to run KVM guests with FRED enabled.

The FRED architecture defines simple new transitions that change privilege
level (ring transitions). The FRED architecture was designed with the
following goals:
1) Improve overall performance and response time by replacing event delivery
   through the interrupt descriptor table (IDT event delivery) and event
   return by the IRET instruction with lower latency transitions.
2) Improve software robustness by ensuring that event delivery establishes
   the full supervisor context and that event return establishes the full
   user context.

Intel VMX architecture is extended to run FRED guests, and the changes
are majorly:
1) New VMCS fields for FRED context management, which includes two new
   event data VMCS fields, eight new guest FRED context VMCS fields and
   eight new host FRED context VMCS fields.
2) VMX nested-Exception support for proper virtualization of stack
   levels introduced with FRED architecture.

Search for the latest FRED spec in most search engines with this search pattern:

  site:intel.com FRED (flexible return and event delivery) specification

The counterpart KVM patch set is at:
https://lore.kernel.org/kvm/20231108183003.5981-1-xin3.li@intel.com/T/#m77876e22876f41c5ec677c0834a46113a4987d31


---
Changelog
v3:
- Add WRMSRNS as a baseline feature for FRED.
- Add the secondary VM exit controls MSR.
- Add FRED VMX controls to VM exit/entry feature words and
  scripts/kvm/vmxcap.
- Do not set/get FRED SSP0 MSR, i.e. PL0_SSP MSR, with FRED, leave it to
  KVM CET.

v2:
- Add VMX nested-exception support to scripts/kvm/vmxcap (Paolo Bonzini).
- Move FRED MSRs from basic x86_cpu part to .subsections part (Weijiang Yang).


Xin Li (6):
  target/i386: add support for FRED in CPUID enumeration
  target/i386: mark CR4.FRED not reserved
  target/i386: add the secondary VM exit controls MSR
  target/i386: add support for VMX FRED controls
  target/i386: enumerate VMX nested-exception support
  target/i386: Add get/set/migrate support for FRED MSRs

 scripts/kvm/vmxcap    | 13 ++++++++++++
 target/i386/cpu.c     | 15 ++++++++++---
 target/i386/cpu.h     | 42 ++++++++++++++++++++++++++++++++++++-
 target/i386/kvm/kvm.c | 49 +++++++++++++++++++++++++++++++++++++++++++
 target/i386/machine.c | 28 +++++++++++++++++++++++++
 5 files changed, 143 insertions(+), 4 deletions(-)


base-commit: a3c3aaa846ad61b801e7196482dcf4afb8ba34e4
-- 
2.42.0

[PATCH v3 1/6] target/i386: add support for FRED in CPUID enumeration

[Xin Li]

FRED, i.e., the Intel flexible return and event delivery architecture,
defines simple new transitions that change privilege level (ring
transitions).

The new transitions defined by the FRED architecture are FRED event
delivery and, for returning from events, two FRED return instructions.
FRED event delivery can effect a transition from ring 3 to ring 0, but
it is used also to deliver events incident to ring 0.  One FRED
instruction (ERETU) effects a return from ring 0 to ring 3, while the
other (ERETS) returns while remaining in ring 0.  Collectively, FRED
event delivery and the FRED return instructions are FRED transitions.

In addition to these transitions, the FRED architecture defines a new
instruction (LKGS) for managing the state of the GS segment register.
The LKGS instruction can be used by 64-bit operating systems that do
not use the new FRED transitions.

WRMSRNS is an instruction that behaves exactly like WRMSR, with the
only difference being that it is not a serializing instruction by
default.  Under certain conditions, WRMSRNS may replace WRMSR to improve
performance.  FRED uses it to switch RSP0 in a faster manner.

Search for the latest FRED spec in most search engines with this search
pattern:

  site:intel.com FRED (flexible return and event delivery) specification

The CPUID feature flag CPUID.(EAX=7,ECX=1):EAX[17] enumerates FRED, and
the CPUID feature flag CPUID.(EAX=7,ECX=1):EAX[18] enumerates LKGS, and
the CPUID feature flag CPUID.(EAX=7,ECX=1):EAX[19] enumerates WRMSRNS.

Add CPUID definitions for FRED/LKGS/WRMSRNS, and expose them to KVM guests.

Because FRED relies on LKGS and WRMSRNS, add that to feature dependency
map.

Tested-by: Shan Kang <shan.kang@intel.com>
Signed-off-by: Xin Li <xin3.li@intel.com>
---
 target/i386/cpu.c | 10 +++++++++-
 target/i386/cpu.h |  6 ++++++
 2 files changed, 15 insertions(+), 1 deletion(-)

diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index 358d9c0a65..403c84177a 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -965,7 +965,7 @@ FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
             "avx-vnni", "avx512-bf16", NULL, "cmpccxadd",
             NULL, NULL, "fzrm", "fsrs",
             "fsrc", NULL, NULL, NULL,
-            NULL, NULL, NULL, NULL,
+            NULL, "fred", "lkgs", "wrmsrns",
             NULL, "amx-fp16", NULL, "avx-ifma",
             NULL, NULL, NULL, NULL,
             NULL, NULL, NULL, NULL,
@@ -1552,6 +1552,14 @@ static FeatureDep feature_dependencies[] = {
         .from = { FEAT_VMX_SECONDARY_CTLS,  VMX_SECONDARY_EXEC_ENABLE_USER_WAIT_PAUSE },
         .to = { FEAT_7_0_ECX,               CPUID_7_0_ECX_WAITPKG },
     },
+    {
+        .from = { FEAT_7_1_EAX,             CPUID_7_1_EAX_FRED },
+        .to = { FEAT_7_1_EAX,               CPUID_7_1_EAX_LKGS },
+    },
+    {
+        .from = { FEAT_7_1_EAX,             CPUID_7_1_EAX_FRED },
+        .to = { FEAT_7_1_EAX,               CPUID_7_1_EAX_WRMSRNS },
+    },
 };
 
 typedef struct X86RegisterInfo32 {
diff --git a/target/i386/cpu.h b/target/i386/cpu.h
index cd2e295bd6..5faf00551d 100644
--- a/target/i386/cpu.h
+++ b/target/i386/cpu.h
@@ -934,6 +934,12 @@ uint64_t x86_cpu_get_supported_feature_word(FeatureWord w,
 #define CPUID_7_1_EDX_AMX_COMPLEX       (1U << 8)
 /* PREFETCHIT0/1 Instructions */
 #define CPUID_7_1_EDX_PREFETCHITI       (1U << 14)
+/* Flexible return and event delivery (FRED) */
+#define CPUID_7_1_EAX_FRED              (1U << 17)
+/* Load into IA32_KERNEL_GS_BASE (LKGS) */
+#define CPUID_7_1_EAX_LKGS              (1U << 18)
+/* Non-Serializing Write to Model Specific Register (WRMSRNS) */
+#define CPUID_7_1_EAX_WRMSRNS           (1U << 19)
 
 /* Do not exhibit MXCSR Configuration Dependent Timing (MCDT) behavior */
 #define CPUID_7_2_EDX_MCDT_NO           (1U << 5)
-- 
2.42.0

[PATCH v3 2/6] target/i386: mark CR4.FRED not reserved

[Xin Li]

The CR4.FRED bit, i.e., CR4[32], is no longer a reserved bit when FRED
is exposed to guests, otherwise it is still a reserved bit.

Tested-by: Shan Kang <shan.kang@intel.com>
Signed-off-by: Xin Li <xin3.li@intel.com>
---
 target/i386/cpu.h | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)

diff --git a/target/i386/cpu.h b/target/i386/cpu.h
index 5faf00551d..e210957cba 100644
--- a/target/i386/cpu.h
+++ b/target/i386/cpu.h
@@ -262,6 +262,12 @@ typedef enum X86Seg {
 #define CR4_PKE_MASK   (1U << 22)
 #define CR4_PKS_MASK   (1U << 24)
 
+#ifdef TARGET_X86_64
+#define CR4_FRED_MASK   (1ULL << 32)
+#else
+#define CR4_FRED_MASK   0
+#endif
+
 #define CR4_RESERVED_MASK \
 (~(target_ulong)(CR4_VME_MASK | CR4_PVI_MASK | CR4_TSD_MASK \
                 | CR4_DE_MASK | CR4_PSE_MASK | CR4_PAE_MASK \
@@ -269,7 +275,8 @@ typedef enum X86Seg {
                 | CR4_OSFXSR_MASK | CR4_OSXMMEXCPT_MASK | CR4_UMIP_MASK \
                 | CR4_LA57_MASK \
                 | CR4_FSGSBASE_MASK | CR4_PCIDE_MASK | CR4_OSXSAVE_MASK \
-                | CR4_SMEP_MASK | CR4_SMAP_MASK | CR4_PKE_MASK | CR4_PKS_MASK))
+                | CR4_SMEP_MASK | CR4_SMAP_MASK | CR4_PKE_MASK | CR4_PKS_MASK \
+                | CR4_FRED_MASK))
 
 #define DR6_BD          (1 << 13)
 #define DR6_BS          (1 << 14)
@@ -2520,6 +2527,9 @@ static inline uint64_t cr4_reserved_bits(CPUX86State *env)
     if (!(env->features[FEAT_7_0_ECX] & CPUID_7_0_ECX_PKS)) {
         reserved_bits |= CR4_PKS_MASK;
     }
+    if (!(env->features[FEAT_7_1_EAX] & CPUID_7_1_EAX_FRED)) {
+        reserved_bits |= CR4_FRED_MASK;
+    }
     return reserved_bits;
 }
 
-- 
2.42.0

[PATCH v3 3/6] target/i386: add the secondary VM exit controls MSR

[Xin Li]

Add the secondary VM exit controls MSR to prepare for KVM FRED enabling.

Tested-by: Shan Kang <shan.kang@intel.com>
Signed-off-by: Xin Li <xin3.li@intel.com>
---
 scripts/kvm/vmxcap | 9 +++++++++
 target/i386/cpu.c  | 2 +-
 target/i386/cpu.h  | 1 +
 3 files changed, 11 insertions(+), 1 deletion(-)

diff --git a/scripts/kvm/vmxcap b/scripts/kvm/vmxcap
index 3fb4d5b342..7da1e00ca8 100755
--- a/scripts/kvm/vmxcap
+++ b/scripts/kvm/vmxcap
@@ -24,6 +24,7 @@ MSR_IA32_VMX_TRUE_EXIT_CTLS = 0x48F
 MSR_IA32_VMX_TRUE_ENTRY_CTLS = 0x490
 MSR_IA32_VMX_VMFUNC = 0x491
 MSR_IA32_VMX_PROCBASED_CTLS3 = 0x492
+MSR_IA32_VMX_EXIT_CTLS2 = 0x493
 
 class msr(object):
     def __init__(self):
@@ -219,11 +220,19 @@ controls = [
             23: 'Clear IA32_BNDCFGS',
             24: 'Conceal VM exits from PT',
             25: 'Clear IA32_RTIT_CTL',
+            31: 'Activate secondary VM-exit controls',
             },
         cap_msr = MSR_IA32_VMX_EXIT_CTLS,
         true_cap_msr = MSR_IA32_VMX_TRUE_EXIT_CTLS,
         ),
 
+    Allowed1Control(
+        name = 'secondary VM-Exit controls',
+        bits = {
+            },
+        cap_msr = MSR_IA32_VMX_EXIT_CTLS2,
+        ),
+
     Control(
         name = 'VM-Entry controls',
         bits = {
diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index 403c84177a..227ee1c759 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -1270,7 +1270,7 @@ FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
             "vmx-exit-save-efer", "vmx-exit-load-efer",
                 "vmx-exit-save-preemption-timer", "vmx-exit-clear-bndcfgs",
             NULL, "vmx-exit-clear-rtit-ctl", NULL, NULL,
-            NULL, "vmx-exit-load-pkrs", NULL, NULL,
+            NULL, "vmx-exit-load-pkrs", NULL, "vmx-exit-secondary-ctls",
         },
         .msr = {
             .index = MSR_IA32_VMX_TRUE_EXIT_CTLS,
diff --git a/target/i386/cpu.h b/target/i386/cpu.h
index e210957cba..a4d3702621 100644
--- a/target/i386/cpu.h
+++ b/target/i386/cpu.h
@@ -551,6 +551,7 @@ typedef enum X86Seg {
 #define MSR_IA32_VMX_TRUE_EXIT_CTLS      0x0000048f
 #define MSR_IA32_VMX_TRUE_ENTRY_CTLS     0x00000490
 #define MSR_IA32_VMX_VMFUNC             0x00000491
+#define MSR_IA32_VMX_EXIT_CTLS2         0x00000493
 
 #define XSTATE_FP_BIT                   0
 #define XSTATE_SSE_BIT                  1
-- 
2.42.0

[PATCH v3 4/6] target/i386: add support for VMX FRED controls

[Xin Li]

Add VMX FRED controls used to enable save/load of FRED MSRs.

Tested-by: Shan Kang <shan.kang@intel.com>
Signed-off-by: Xin Li <xin3.li@intel.com>
---
 scripts/kvm/vmxcap | 3 +++
 target/i386/cpu.c  | 2 +-
 2 files changed, 4 insertions(+), 1 deletion(-)

diff --git a/scripts/kvm/vmxcap b/scripts/kvm/vmxcap
index 7da1e00ca8..44898d73c2 100755
--- a/scripts/kvm/vmxcap
+++ b/scripts/kvm/vmxcap
@@ -229,6 +229,8 @@ controls = [
     Allowed1Control(
         name = 'secondary VM-Exit controls',
         bits = {
+            0: 'Save IA32 FRED MSRs',
+            1: 'Load IA32 FRED MSRs',
             },
         cap_msr = MSR_IA32_VMX_EXIT_CTLS2,
         ),
@@ -246,6 +248,7 @@ controls = [
             16: 'Load IA32_BNDCFGS',
             17: 'Conceal VM entries from PT',
             18: 'Load IA32_RTIT_CTL',
+            23: 'Load IA32 FRED MSRs',
             },
         cap_msr = MSR_IA32_VMX_ENTRY_CTLS,
         true_cap_msr = MSR_IA32_VMX_TRUE_ENTRY_CTLS,
diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index 227ee1c759..dcf914a7ec 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -1285,7 +1285,7 @@ FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
             NULL, "vmx-entry-ia32e-mode", NULL, NULL,
             NULL, "vmx-entry-load-perf-global-ctrl", "vmx-entry-load-pat", "vmx-entry-load-efer",
             "vmx-entry-load-bndcfgs", NULL, "vmx-entry-load-rtit-ctl", NULL,
-            NULL, NULL, "vmx-entry-load-pkrs", NULL,
+            NULL, NULL, "vmx-entry-load-pkrs", "vmx-entry-load-fred",
             NULL, NULL, NULL, NULL,
             NULL, NULL, NULL, NULL,
         },
-- 
2.42.0

[PATCH v3 5/6] target/i386: enumerate VMX nested-exception support

[Xin Li]

Allow VMX nested-exception support to be exposed in KVM guests, thus
nested KVM guests can enumerate it.

Tested-by: Shan Kang <shan.kang@intel.com>
Signed-off-by: Xin Li <xin3.li@intel.com>
---
 scripts/kvm/vmxcap | 1 +
 target/i386/cpu.c  | 1 +
 target/i386/cpu.h  | 1 +
 3 files changed, 3 insertions(+)

diff --git a/scripts/kvm/vmxcap b/scripts/kvm/vmxcap
index 44898d73c2..508be19c75 100755
--- a/scripts/kvm/vmxcap
+++ b/scripts/kvm/vmxcap
@@ -117,6 +117,7 @@ controls = [
             54: 'INS/OUTS instruction information',
             55: 'IA32_VMX_TRUE_*_CTLS support',
             56: 'Skip checks on event error code',
+            58: 'VMX nested exception support',
             },
         msr = MSR_IA32_VMX_BASIC,
         ),
diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index dcf914a7ec..f7556621a5 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -1343,6 +1343,7 @@ FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
             [54] = "vmx-ins-outs",
             [55] = "vmx-true-ctls",
             [56] = "vmx-any-errcode",
+            [58] = "vmx-nested-exception",
         },
         .msr = {
             .index = MSR_IA32_VMX_BASIC,
diff --git a/target/i386/cpu.h b/target/i386/cpu.h
index a4d3702621..cc3b4fefb8 100644
--- a/target/i386/cpu.h
+++ b/target/i386/cpu.h
@@ -1054,6 +1054,7 @@ uint64_t x86_cpu_get_supported_feature_word(FeatureWord w,
 #define MSR_VMX_BASIC_INS_OUTS                       (1ULL << 54)
 #define MSR_VMX_BASIC_TRUE_CTLS                      (1ULL << 55)
 #define MSR_VMX_BASIC_ANY_ERRCODE                    (1ULL << 56)
+#define MSR_VMX_BASIC_NESTED_EXCEPTION               (1ULL << 58)
 
 #define MSR_VMX_MISC_PREEMPTION_TIMER_SHIFT_MASK     0x1Full
 #define MSR_VMX_MISC_STORE_LMA                       (1ULL << 5)
-- 
2.42.0

[PATCH v3 6/6] target/i386: Add get/set/migrate support for FRED MSRs

[Xin Li]

FRED CPU states are managed in 9 new FRED MSRs, in addtion to a few
existing CPU registers and MSRs, e.g., CR4.FRED and MSR_IA32_PL0_SSP.

Save/restore/migrate FRED MSRs if FRED is exposed to the guest.

Tested-by: Shan Kang <shan.kang@intel.com>
Signed-off-by: Xin Li <xin3.li@intel.com>
---
 target/i386/cpu.h     | 22 +++++++++++++++++++
 target/i386/kvm/kvm.c | 49 +++++++++++++++++++++++++++++++++++++++++++
 target/i386/machine.c | 28 +++++++++++++++++++++++++
 3 files changed, 99 insertions(+)

diff --git a/target/i386/cpu.h b/target/i386/cpu.h
index cc3b4fefb8..3b13eceffe 100644
--- a/target/i386/cpu.h
+++ b/target/i386/cpu.h
@@ -529,6 +529,17 @@ typedef enum X86Seg {
 #define MSR_IA32_XFD                    0x000001c4
 #define MSR_IA32_XFD_ERR                0x000001c5
 
+/* FRED MSRs */
+#define MSR_IA32_FRED_RSP0              0x000001cc       /* Stack level 0 regular stack pointer */
+#define MSR_IA32_FRED_RSP1              0x000001cd       /* Stack level 1 regular stack pointer */
+#define MSR_IA32_FRED_RSP2              0x000001ce       /* Stack level 2 regular stack pointer */
+#define MSR_IA32_FRED_RSP3              0x000001cf       /* Stack level 3 regular stack pointer */
+#define MSR_IA32_FRED_STKLVLS           0x000001d0       /* FRED exception stack levels */
+#define MSR_IA32_FRED_SSP1              0x000001d1       /* Stack level 1 shadow stack pointer in ring 0 */
+#define MSR_IA32_FRED_SSP2              0x000001d2       /* Stack level 2 shadow stack pointer in ring 0 */
+#define MSR_IA32_FRED_SSP3              0x000001d3       /* Stack level 3 shadow stack pointer in ring 0 */
+#define MSR_IA32_FRED_CONFIG            0x000001d4       /* FRED Entrypoint and interrupt stack level */
+
 #define MSR_IA32_BNDCFGS                0x00000d90
 #define MSR_IA32_XSS                    0x00000da0
 #define MSR_IA32_UMWAIT_CONTROL         0xe1
@@ -1687,6 +1698,17 @@ typedef struct CPUArchState {
     target_ulong cstar;
     target_ulong fmask;
     target_ulong kernelgsbase;
+
+    /* FRED MSRs */
+    uint64_t fred_rsp0;
+    uint64_t fred_rsp1;
+    uint64_t fred_rsp2;
+    uint64_t fred_rsp3;
+    uint64_t fred_stklvls;
+    uint64_t fred_ssp1;
+    uint64_t fred_ssp2;
+    uint64_t fred_ssp3;
+    uint64_t fred_config;
 #endif
 
     uint64_t tsc_adjust;
diff --git a/target/i386/kvm/kvm.c b/target/i386/kvm/kvm.c
index 11b8177eff..101ff63805 100644
--- a/target/i386/kvm/kvm.c
+++ b/target/i386/kvm/kvm.c
@@ -3309,6 +3309,17 @@ static int kvm_put_msrs(X86CPU *cpu, int level)
         kvm_msr_entry_add(cpu, MSR_KERNELGSBASE, env->kernelgsbase);
         kvm_msr_entry_add(cpu, MSR_FMASK, env->fmask);
         kvm_msr_entry_add(cpu, MSR_LSTAR, env->lstar);
+        if (env->features[FEAT_7_1_EAX] & CPUID_7_1_EAX_FRED) {
+            kvm_msr_entry_add(cpu, MSR_IA32_FRED_RSP0, env->fred_rsp0);
+            kvm_msr_entry_add(cpu, MSR_IA32_FRED_RSP1, env->fred_rsp1);
+            kvm_msr_entry_add(cpu, MSR_IA32_FRED_RSP2, env->fred_rsp2);
+            kvm_msr_entry_add(cpu, MSR_IA32_FRED_RSP3, env->fred_rsp3);
+            kvm_msr_entry_add(cpu, MSR_IA32_FRED_STKLVLS, env->fred_stklvls);
+            kvm_msr_entry_add(cpu, MSR_IA32_FRED_SSP1, env->fred_ssp1);
+            kvm_msr_entry_add(cpu, MSR_IA32_FRED_SSP2, env->fred_ssp2);
+            kvm_msr_entry_add(cpu, MSR_IA32_FRED_SSP3, env->fred_ssp3);
+            kvm_msr_entry_add(cpu, MSR_IA32_FRED_CONFIG, env->fred_config);
+        }
     }
 #endif
 
@@ -3773,6 +3784,17 @@ static int kvm_get_msrs(X86CPU *cpu)
         kvm_msr_entry_add(cpu, MSR_KERNELGSBASE, 0);
         kvm_msr_entry_add(cpu, MSR_FMASK, 0);
         kvm_msr_entry_add(cpu, MSR_LSTAR, 0);
+        if (env->features[FEAT_7_1_EAX] & CPUID_7_1_EAX_FRED) {
+            kvm_msr_entry_add(cpu, MSR_IA32_FRED_RSP0, 0);
+            kvm_msr_entry_add(cpu, MSR_IA32_FRED_RSP1, 0);
+            kvm_msr_entry_add(cpu, MSR_IA32_FRED_RSP2, 0);
+            kvm_msr_entry_add(cpu, MSR_IA32_FRED_RSP3, 0);
+            kvm_msr_entry_add(cpu, MSR_IA32_FRED_STKLVLS, 0);
+            kvm_msr_entry_add(cpu, MSR_IA32_FRED_SSP1, 0);
+            kvm_msr_entry_add(cpu, MSR_IA32_FRED_SSP2, 0);
+            kvm_msr_entry_add(cpu, MSR_IA32_FRED_SSP3, 0);
+            kvm_msr_entry_add(cpu, MSR_IA32_FRED_CONFIG, 0);
+        }
     }
 #endif
     kvm_msr_entry_add(cpu, MSR_KVM_SYSTEM_TIME, 0);
@@ -3994,6 +4016,33 @@ static int kvm_get_msrs(X86CPU *cpu)
         case MSR_LSTAR:
             env->lstar = msrs[i].data;
             break;
+        case MSR_IA32_FRED_RSP0:
+            env->fred_rsp0 = msrs[i].data;
+            break;
+        case MSR_IA32_FRED_RSP1:
+            env->fred_rsp1 = msrs[i].data;
+            break;
+        case MSR_IA32_FRED_RSP2:
+            env->fred_rsp2 = msrs[i].data;
+            break;
+        case MSR_IA32_FRED_RSP3:
+            env->fred_rsp3 = msrs[i].data;
+            break;
+        case MSR_IA32_FRED_STKLVLS:
+            env->fred_stklvls = msrs[i].data;
+            break;
+        case MSR_IA32_FRED_SSP1:
+            env->fred_ssp1 = msrs[i].data;
+            break;
+        case MSR_IA32_FRED_SSP2:
+            env->fred_ssp2 = msrs[i].data;
+            break;
+        case MSR_IA32_FRED_SSP3:
+            env->fred_ssp3 = msrs[i].data;
+            break;
+        case MSR_IA32_FRED_CONFIG:
+            env->fred_config = msrs[i].data;
+            break;
 #endif
         case MSR_IA32_TSC:
             env->tsc = msrs[i].data;
diff --git a/target/i386/machine.c b/target/i386/machine.c
index a1041ef828..850a19cb8e 100644
--- a/target/i386/machine.c
+++ b/target/i386/machine.c
@@ -1544,6 +1544,33 @@ static const VMStateDescription vmstate_msr_xfd = {
 };
 
 #ifdef TARGET_X86_64
+static bool intel_fred_msrs_needed(void *opaque)
+{
+    X86CPU *cpu = opaque;
+    CPUX86State *env = &cpu->env;
+
+    return !!(env->features[FEAT_7_1_EAX] & CPUID_7_1_EAX_FRED);
+}
+
+static const VMStateDescription vmstate_msr_fred = {
+    .name = "cpu/fred",
+    .version_id = 1,
+    .minimum_version_id = 1,
+    .needed = intel_fred_msrs_needed,
+    .fields = (VMStateField[]) {
+        VMSTATE_UINT64(env.fred_rsp0, X86CPU),
+        VMSTATE_UINT64(env.fred_rsp1, X86CPU),
+        VMSTATE_UINT64(env.fred_rsp2, X86CPU),
+        VMSTATE_UINT64(env.fred_rsp3, X86CPU),
+        VMSTATE_UINT64(env.fred_stklvls, X86CPU),
+        VMSTATE_UINT64(env.fred_ssp1, X86CPU),
+        VMSTATE_UINT64(env.fred_ssp2, X86CPU),
+        VMSTATE_UINT64(env.fred_ssp3, X86CPU),
+        VMSTATE_UINT64(env.fred_config, X86CPU),
+        VMSTATE_END_OF_LIST()
+            }
+        };
+
 static bool amx_xtile_needed(void *opaque)
 {
     X86CPU *cpu = opaque;
@@ -1747,6 +1774,7 @@ const VMStateDescription vmstate_x86_cpu = {
         &vmstate_pdptrs,
         &vmstate_msr_xfd,
 #ifdef TARGET_X86_64
+        &vmstate_msr_fred,
         &vmstate_amx_xtile,
 #endif
         &vmstate_arch_lbr,
-- 
2.42.0

Re: [PATCH v3 1/6] target/i386: add support for FRED in CPUID enumeration

[Wu, Dan1]

[-- Attachment #1: Type: text/plain, Size: 4400 bytes --]


On 11/9/2023 3:20 PM, Xin Li wrote:
> FRED, i.e., the Intel flexible return and event delivery architecture,
> defines simple new transitions that change privilege level (ring
> transitions).
>
> The new transitions defined by the FRED architecture are FRED event
> delivery and, for returning from events, two FRED return instructions.
> FRED event delivery can effect a transition from ring 3 to ring 0, but
> it is used also to deliver events incident to ring 0.  One FRED
> instruction (ERETU) effects a return from ring 0 to ring 3, while the
> other (ERETS) returns while remaining in ring 0.  Collectively, FRED
> event delivery and the FRED return instructions are FRED transitions.
>
> In addition to these transitions, the FRED architecture defines a new
> instruction (LKGS) for managing the state of the GS segment register.
> The LKGS instruction can be used by 64-bit operating systems that do
> not use the new FRED transitions.
>
> WRMSRNS is an instruction that behaves exactly like WRMSR, with the
> only difference being that it is not a serializing instruction by
> default.  Under certain conditions, WRMSRNS may replace WRMSR to improve
> performance.  FRED uses it to switch RSP0 in a faster manner.
>
> Search for the latest FRED spec in most search engines with this search
> pattern:
>
>    site:intel.com FRED (flexible return and event delivery) specification
>
> The CPUID feature flag CPUID.(EAX=7,ECX=1):EAX[17] enumerates FRED, and
> the CPUID feature flag CPUID.(EAX=7,ECX=1):EAX[18] enumerates LKGS, and
> the CPUID feature flag CPUID.(EAX=7,ECX=1):EAX[19] enumerates WRMSRNS.
>
> Add CPUID definitions for FRED/LKGS/WRMSRNS, and expose them to KVM guests.
>
> Because FRED relies on LKGS and WRMSRNS, add that to feature dependency
> map.
>
> Tested-by: Shan Kang<shan.kang@intel.com>
> Signed-off-by: Xin Li<xin3.li@intel.com>
> ---
>   target/i386/cpu.c | 10 +++++++++-
>   target/i386/cpu.h |  6 ++++++
>   2 files changed, 15 insertions(+), 1 deletion(-)
>
> diff --git a/target/i386/cpu.c b/target/i386/cpu.c
> index 358d9c0a65..403c84177a 100644
> --- a/target/i386/cpu.c
> +++ b/target/i386/cpu.c
> @@ -965,7 +965,7 @@ FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
>               "avx-vnni", "avx512-bf16", NULL, "cmpccxadd",
>               NULL, NULL, "fzrm", "fsrs",
>               "fsrc", NULL, NULL, NULL,
> -            NULL, NULL, NULL, NULL,
> +            NULL, "fred", "lkgs", "wrmsrns",
>               NULL, "amx-fp16", NULL, "avx-ifma",
>               NULL, NULL, NULL, NULL,
>               NULL, NULL, NULL, NULL,
> @@ -1552,6 +1552,14 @@ static FeatureDep feature_dependencies[] = {
>           .from = { FEAT_VMX_SECONDARY_CTLS,  VMX_SECONDARY_EXEC_ENABLE_USER_WAIT_PAUSE },
>           .to = { FEAT_7_0_ECX,               CPUID_7_0_ECX_WAITPKG },
>       },
> +    {
> +        .from = { FEAT_7_1_EAX,             CPUID_7_1_EAX_FRED },
> +        .to = { FEAT_7_1_EAX,               CPUID_7_1_EAX_LKGS },
> +    },
> +    {
> +        .from = { FEAT_7_1_EAX,             CPUID_7_1_EAX_FRED },
> +        .to = { FEAT_7_1_EAX,               CPUID_7_1_EAX_WRMSRNS },
> +    },

since FRED relies on LKGS and WRMSRNS, here should be as follow:

+    {
+        .from = { FEAT_7_1_EAX,             CPUID_7_1_EAX_LKGS },
+        .to = { FEAT_7_1_EAX,               CPUID_7_1_EAX_FRED },
+    },
+    {
+        .from = { FEAT_7_1_EAX,             CPUID_7_1_EAX_WRMSRNS },
+        .to = { FEAT_7_1_EAX,               CPUID_7_1_EAX_FRED },
+    },

>   };
>   
>   typedef struct X86RegisterInfo32 {
> diff --git a/target/i386/cpu.h b/target/i386/cpu.h
> index cd2e295bd6..5faf00551d 100644
> --- a/target/i386/cpu.h
> +++ b/target/i386/cpu.h
> @@ -934,6 +934,12 @@ uint64_t x86_cpu_get_supported_feature_word(FeatureWord w,
>   #define CPUID_7_1_EDX_AMX_COMPLEX       (1U << 8)
>   /* PREFETCHIT0/1 Instructions */
>   #define CPUID_7_1_EDX_PREFETCHITI       (1U << 14)
> +/* Flexible return and event delivery (FRED) */
> +#define CPUID_7_1_EAX_FRED              (1U << 17)
> +/* Load into IA32_KERNEL_GS_BASE (LKGS) */
> +#define CPUID_7_1_EAX_LKGS              (1U << 18)
> +/* Non-Serializing Write to Model Specific Register (WRMSRNS) */
> +#define CPUID_7_1_EAX_WRMSRNS           (1U << 19)
>   
>   /* Do not exhibit MXCSR Configuration Dependent Timing (MCDT) behavior */
>   #define CPUID_7_2_EDX_MCDT_NO           (1U << 5)

[-- Attachment #2: Type: text/html, Size: 5285 bytes --]

RE: [PATCH v3 1/6] target/i386: add support for FRED in CPUID enumeration

[Li, Xin3]

>> @@ -1552,6 +1552,14 @@ static FeatureDep feature_dependencies[] = {
>>           .from = { FEAT_VMX_SECONDARY_CTLS,  VMX_SECONDARY_EXEC_ENABLE_USER_WAIT_PAUSE },
>>           .to = { FEAT_7_0_ECX,               CPUID_7_0_ECX_WAITPKG },
>>       },
>> +    {
>> +        .from = { FEAT_7_1_EAX,             CPUID_7_1_EAX_FRED },
>> +        .to = { FEAT_7_1_EAX,               CPUID_7_1_EAX_LKGS },
>> +    },
>> +    {
>> +        .from = { FEAT_7_1_EAX,             CPUID_7_1_EAX_FRED },
>> +        .to = { FEAT_7_1_EAX,               CPUID_7_1_EAX_WRMSRNS },
>> +    },
> 
> since FRED relies on LKGS and WRMSRNS, here should be as follow:
>

You're right, I messed it up in this version.
Thank!
    Xin

> +    {
> +        .from = { FEAT_7_1_EAX,             CPUID_7_1_EAX_LKGS },
> +        .to = { FEAT_7_1_EAX,               CPUID_7_1_EAX_FRED },
> +    },
> +    {
> +        .from = { FEAT_7_1_EAX,             CPUID_7_1_EAX_WRMSRNS },
> +        .to = { FEAT_7_1_EAX,               CPUID_7_1_EAX_FRED },
> +    },

[PATCH v3A 1/6] target/i386: add support for FRED in CPUID enumeration

[Xin Li]

FRED, i.e., the Intel flexible return and event delivery architecture,
defines simple new transitions that change privilege level (ring
transitions).

The new transitions defined by the FRED architecture are FRED event
delivery and, for returning from events, two FRED return instructions.
FRED event delivery can effect a transition from ring 3 to ring 0, but
it is used also to deliver events incident to ring 0.  One FRED
instruction (ERETU) effects a return from ring 0 to ring 3, while the
other (ERETS) returns while remaining in ring 0.  Collectively, FRED
event delivery and the FRED return instructions are FRED transitions.

In addition to these transitions, the FRED architecture defines a new
instruction (LKGS) for managing the state of the GS segment register.
The LKGS instruction can be used by 64-bit operating systems that do
not use the new FRED transitions.

WRMSRNS is an instruction that behaves exactly like WRMSR, with the
only difference being that it is not a serializing instruction by
default.  Under certain conditions, WRMSRNS may replace WRMSR to improve
performance.  FRED uses it to switch RSP0 in a faster manner.

Search for the latest FRED spec in most search engines with this search
pattern:

  site:intel.com FRED (flexible return and event delivery) specification

The CPUID feature flag CPUID.(EAX=7,ECX=1):EAX[17] enumerates FRED, and
the CPUID feature flag CPUID.(EAX=7,ECX=1):EAX[18] enumerates LKGS, and
the CPUID feature flag CPUID.(EAX=7,ECX=1):EAX[19] enumerates WRMSRNS.

Add CPUID definitions for FRED/LKGS/WRMSRNS, and expose them to KVM guests.

Because FRED relies on LKGS and WRMSRNS, add that to feature dependency
map.

Tested-by: Shan Kang <shan.kang@intel.com>
Signed-off-by: Xin Li <xin3.li@intel.com>
---

Changelog
v3A:
- Fix reversed dependency (Wu Dan1).
---
 target/i386/cpu.c | 10 +++++++++-
 target/i386/cpu.h |  6 ++++++
 2 files changed, 15 insertions(+), 1 deletion(-)

diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index 358d9c0a65..66551c7eae 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -965,7 +965,7 @@ FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
             "avx-vnni", "avx512-bf16", NULL, "cmpccxadd",
             NULL, NULL, "fzrm", "fsrs",
             "fsrc", NULL, NULL, NULL,
-            NULL, NULL, NULL, NULL,
+            NULL, "fred", "lkgs", "wrmsrns",
             NULL, "amx-fp16", NULL, "avx-ifma",
             NULL, NULL, NULL, NULL,
             NULL, NULL, NULL, NULL,
@@ -1552,6 +1552,14 @@ static FeatureDep feature_dependencies[] = {
         .from = { FEAT_VMX_SECONDARY_CTLS,  VMX_SECONDARY_EXEC_ENABLE_USER_WAIT_PAUSE },
         .to = { FEAT_7_0_ECX,               CPUID_7_0_ECX_WAITPKG },
     },
+    {
+        .from = { FEAT_7_1_EAX,             CPUID_7_1_EAX_LKGS },
+        .to = { FEAT_7_1_EAX,               CPUID_7_1_EAX_FRED },
+    },
+    {
+        .from = { FEAT_7_1_EAX,             CPUID_7_1_EAX_WRMSRNS },
+        .to = { FEAT_7_1_EAX,               CPUID_7_1_EAX_FRED },
+    },
 };
 
 typedef struct X86RegisterInfo32 {
diff --git a/target/i386/cpu.h b/target/i386/cpu.h
index cd2e295bd6..5faf00551d 100644
--- a/target/i386/cpu.h
+++ b/target/i386/cpu.h
@@ -934,6 +934,12 @@ uint64_t x86_cpu_get_supported_feature_word(FeatureWord w,
 #define CPUID_7_1_EDX_AMX_COMPLEX       (1U << 8)
 /* PREFETCHIT0/1 Instructions */
 #define CPUID_7_1_EDX_PREFETCHITI       (1U << 14)
+/* Flexible return and event delivery (FRED) */
+#define CPUID_7_1_EAX_FRED              (1U << 17)
+/* Load into IA32_KERNEL_GS_BASE (LKGS) */
+#define CPUID_7_1_EAX_LKGS              (1U << 18)
+/* Non-Serializing Write to Model Specific Register (WRMSRNS) */
+#define CPUID_7_1_EAX_WRMSRNS           (1U << 19)
 
 /* Do not exhibit MXCSR Configuration Dependent Timing (MCDT) behavior */
 #define CPUID_7_2_EDX_MCDT_NO           (1U << 5)
-- 
2.43.0

Re: [PATCH v3A 1/6] target/i386: add support for FRED in CPUID enumeration

[Zhao Liu]

On Thu, Dec 21, 2023 at 07:03:36PM -0800, Xin Li wrote:
> Date: Thu, 21 Dec 2023 19:03:36 -0800
> From: Xin Li <xin3.li@intel.com>
> Subject: [PATCH v3A 1/6] target/i386: add support for FRED in CPUID
>  enumeration
> X-Mailer: git-send-email 2.43.0
> 
> FRED, i.e., the Intel flexible return and event delivery architecture,
> defines simple new transitions that change privilege level (ring
> transitions).
> 
> The new transitions defined by the FRED architecture are FRED event
> delivery and, for returning from events, two FRED return instructions.
> FRED event delivery can effect a transition from ring 3 to ring 0, but
> it is used also to deliver events incident to ring 0.  One FRED
> instruction (ERETU) effects a return from ring 0 to ring 3, while the
> other (ERETS) returns while remaining in ring 0.  Collectively, FRED
> event delivery and the FRED return instructions are FRED transitions.
> 
> In addition to these transitions, the FRED architecture defines a new
> instruction (LKGS) for managing the state of the GS segment register.
> The LKGS instruction can be used by 64-bit operating systems that do
> not use the new FRED transitions.
> 
> WRMSRNS is an instruction that behaves exactly like WRMSR, with the
> only difference being that it is not a serializing instruction by
> default.  Under certain conditions, WRMSRNS may replace WRMSR to improve
> performance.  FRED uses it to switch RSP0 in a faster manner.
> 
> Search for the latest FRED spec in most search engines with this search
> pattern:
> 
>   site:intel.com FRED (flexible return and event delivery) specification
> 
> The CPUID feature flag CPUID.(EAX=7,ECX=1):EAX[17] enumerates FRED, and
> the CPUID feature flag CPUID.(EAX=7,ECX=1):EAX[18] enumerates LKGS, and
> the CPUID feature flag CPUID.(EAX=7,ECX=1):EAX[19] enumerates WRMSRNS.
> 
> Add CPUID definitions for FRED/LKGS/WRMSRNS, and expose them to KVM guests.
> 
> Because FRED relies on LKGS and WRMSRNS, add that to feature dependency
> map.
> 
> Tested-by: Shan Kang <shan.kang@intel.com>
> Signed-off-by: Xin Li <xin3.li@intel.com>
> ---

Reviewed-by: Zhao Liu <zhao1.liu@intel.com>

> 
> Changelog
> v3A:
> - Fix reversed dependency (Wu Dan1).
> ---
>  target/i386/cpu.c | 10 +++++++++-
>  target/i386/cpu.h |  6 ++++++
>  2 files changed, 15 insertions(+), 1 deletion(-)
> 
> diff --git a/target/i386/cpu.c b/target/i386/cpu.c
> index 358d9c0a65..66551c7eae 100644
> --- a/target/i386/cpu.c
> +++ b/target/i386/cpu.c
> @@ -965,7 +965,7 @@ FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
>              "avx-vnni", "avx512-bf16", NULL, "cmpccxadd",
>              NULL, NULL, "fzrm", "fsrs",
>              "fsrc", NULL, NULL, NULL,
> -            NULL, NULL, NULL, NULL,
> +            NULL, "fred", "lkgs", "wrmsrns",
>              NULL, "amx-fp16", NULL, "avx-ifma",
>              NULL, NULL, NULL, NULL,
>              NULL, NULL, NULL, NULL,
> @@ -1552,6 +1552,14 @@ static FeatureDep feature_dependencies[] = {
>          .from = { FEAT_VMX_SECONDARY_CTLS,  VMX_SECONDARY_EXEC_ENABLE_USER_WAIT_PAUSE },
>          .to = { FEAT_7_0_ECX,               CPUID_7_0_ECX_WAITPKG },
>      },
> +    {
> +        .from = { FEAT_7_1_EAX,             CPUID_7_1_EAX_LKGS },
> +        .to = { FEAT_7_1_EAX,               CPUID_7_1_EAX_FRED },
> +    },
> +    {
> +        .from = { FEAT_7_1_EAX,             CPUID_7_1_EAX_WRMSRNS },
> +        .to = { FEAT_7_1_EAX,               CPUID_7_1_EAX_FRED },
> +    },
>  };
>  
>  typedef struct X86RegisterInfo32 {
> diff --git a/target/i386/cpu.h b/target/i386/cpu.h
> index cd2e295bd6..5faf00551d 100644
> --- a/target/i386/cpu.h
> +++ b/target/i386/cpu.h
> @@ -934,6 +934,12 @@ uint64_t x86_cpu_get_supported_feature_word(FeatureWord w,
>  #define CPUID_7_1_EDX_AMX_COMPLEX       (1U << 8)
>  /* PREFETCHIT0/1 Instructions */
>  #define CPUID_7_1_EDX_PREFETCHITI       (1U << 14)
> +/* Flexible return and event delivery (FRED) */
> +#define CPUID_7_1_EAX_FRED              (1U << 17)
> +/* Load into IA32_KERNEL_GS_BASE (LKGS) */
> +#define CPUID_7_1_EAX_LKGS              (1U << 18)
> +/* Non-Serializing Write to Model Specific Register (WRMSRNS) */
> +#define CPUID_7_1_EAX_WRMSRNS           (1U << 19)
>  
>  /* Do not exhibit MXCSR Configuration Dependent Timing (MCDT) behavior */
>  #define CPUID_7_2_EDX_MCDT_NO           (1U << 5)
> -- 
> 2.43.0
> 
> 

Re: [PATCH v3A 1/6] target/i386: add support for FRED in CPUID enumeration

[Zhao Liu]

On Fri, Dec 22, 2023 at 03:34:02PM +0800, Zhao Liu wrote:
> Date: Fri, 22 Dec 2023 15:34:02 +0800
> From: Zhao Liu <zhao1.liu@intel.com>
> Subject: Re: [PATCH v3A 1/6] target/i386: add support for FRED in CPUID
>  enumeration
> 
> On Thu, Dec 21, 2023 at 07:03:36PM -0800, Xin Li wrote:
> > Date: Thu, 21 Dec 2023 19:03:36 -0800
> > From: Xin Li <xin3.li@intel.com>
> > Subject: [PATCH v3A 1/6] target/i386: add support for FRED in CPUID
> >  enumeration
> > X-Mailer: git-send-email 2.43.0
> > 
> > FRED, i.e., the Intel flexible return and event delivery architecture,
> > defines simple new transitions that change privilege level (ring
> > transitions).
> > 
> > The new transitions defined by the FRED architecture are FRED event
> > delivery and, for returning from events, two FRED return instructions.
> > FRED event delivery can effect a transition from ring 3 to ring 0, but
> > it is used also to deliver events incident to ring 0.  One FRED
> > instruction (ERETU) effects a return from ring 0 to ring 3, while the
> > other (ERETS) returns while remaining in ring 0.  Collectively, FRED
> > event delivery and the FRED return instructions are FRED transitions.
> > 
> > In addition to these transitions, the FRED architecture defines a new
> > instruction (LKGS) for managing the state of the GS segment register.
> > The LKGS instruction can be used by 64-bit operating systems that do
> > not use the new FRED transitions.
> > 
> > WRMSRNS is an instruction that behaves exactly like WRMSR, with the
> > only difference being that it is not a serializing instruction by
> > default.  Under certain conditions, WRMSRNS may replace WRMSR to improve
> > performance.  FRED uses it to switch RSP0 in a faster manner.
> > 
> > Search for the latest FRED spec in most search engines with this search
> > pattern:
> > 
> >   site:intel.com FRED (flexible return and event delivery) specification
> > 
> > The CPUID feature flag CPUID.(EAX=7,ECX=1):EAX[17] enumerates FRED, and
> > the CPUID feature flag CPUID.(EAX=7,ECX=1):EAX[18] enumerates LKGS, and
> > the CPUID feature flag CPUID.(EAX=7,ECX=1):EAX[19] enumerates WRMSRNS.
> > 
> > Add CPUID definitions for FRED/LKGS/WRMSRNS, and expose them to KVM guests.
> > 
> > Because FRED relies on LKGS and WRMSRNS, add that to feature dependency
> > map.
> > 
> > Tested-by: Shan Kang <shan.kang@intel.com>
> > Signed-off-by: Xin Li <xin3.li@intel.com>
> > ---
> 
> Reviewed-by: Zhao Liu <zhao1.liu@intel.com>
> 
> > 
> > Changelog
> > v3A:
> > - Fix reversed dependency (Wu Dan1).
> > ---
> >  target/i386/cpu.c | 10 +++++++++-
> >  target/i386/cpu.h |  6 ++++++
> >  2 files changed, 15 insertions(+), 1 deletion(-)
> > 
> > diff --git a/target/i386/cpu.c b/target/i386/cpu.c
> > index 358d9c0a65..66551c7eae 100644
> > --- a/target/i386/cpu.c
> > +++ b/target/i386/cpu.c
> > @@ -965,7 +965,7 @@ FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
> >              "avx-vnni", "avx512-bf16", NULL, "cmpccxadd",
> >              NULL, NULL, "fzrm", "fsrs",
> >              "fsrc", NULL, NULL, NULL,
> > -            NULL, NULL, NULL, NULL,
> > +            NULL, "fred", "lkgs", "wrmsrns",
> >              NULL, "amx-fp16", NULL, "avx-ifma",
> >              NULL, NULL, NULL, NULL,
> >              NULL, NULL, NULL, NULL,
> > @@ -1552,6 +1552,14 @@ static FeatureDep feature_dependencies[] = {
> >          .from = { FEAT_VMX_SECONDARY_CTLS,  VMX_SECONDARY_EXEC_ENABLE_USER_WAIT_PAUSE },
> >          .to = { FEAT_7_0_ECX,               CPUID_7_0_ECX_WAITPKG },
> >      },
> > +    {
> > +        .from = { FEAT_7_1_EAX,             CPUID_7_1_EAX_LKGS },
> > +        .to = { FEAT_7_1_EAX,               CPUID_7_1_EAX_FRED },
> > +    },
> > +    {
> > +        .from = { FEAT_7_1_EAX,             CPUID_7_1_EAX_WRMSRNS },
> > +        .to = { FEAT_7_1_EAX,               CPUID_7_1_EAX_FRED },
> > +    },

Oh, sorry, one thing that comes to mind, is this dependency required?
Since the FRED spec (v3.0) is all about WRMSR as the example, without
mentioning WRMSRNS, could there be other implementations that depend on
WRMSR instead of WRMSRNS?

The dependencies of LKGS are clearly defined in spec.

-Zhao

> >  };
> >  
> >  typedef struct X86RegisterInfo32 {
> > diff --git a/target/i386/cpu.h b/target/i386/cpu.h
> > index cd2e295bd6..5faf00551d 100644
> > --- a/target/i386/cpu.h
> > +++ b/target/i386/cpu.h
> > @@ -934,6 +934,12 @@ uint64_t x86_cpu_get_supported_feature_word(FeatureWord w,
> >  #define CPUID_7_1_EDX_AMX_COMPLEX       (1U << 8)
> >  /* PREFETCHIT0/1 Instructions */
> >  #define CPUID_7_1_EDX_PREFETCHITI       (1U << 14)
> > +/* Flexible return and event delivery (FRED) */
> > +#define CPUID_7_1_EAX_FRED              (1U << 17)
> > +/* Load into IA32_KERNEL_GS_BASE (LKGS) */
> > +#define CPUID_7_1_EAX_LKGS              (1U << 18)
> > +/* Non-Serializing Write to Model Specific Register (WRMSRNS) */
> > +#define CPUID_7_1_EAX_WRMSRNS           (1U << 19)
> >  
> >  /* Do not exhibit MXCSR Configuration Dependent Timing (MCDT) behavior */
> >  #define CPUID_7_2_EDX_MCDT_NO           (1U << 5)
> > -- 
> > 2.43.0
> > 
> > 
> 

RE: [PATCH v3A 1/6] target/i386: add support for FRED in CPUID enumeration

[Li, Xin3]

> > >              NULL, NULL, NULL, NULL, @@ -1552,6 +1552,14 @@ static
> > > FeatureDep feature_dependencies[] = {
> > >          .from = { FEAT_VMX_SECONDARY_CTLS,
> VMX_SECONDARY_EXEC_ENABLE_USER_WAIT_PAUSE },
> > >          .to = { FEAT_7_0_ECX,               CPUID_7_0_ECX_WAITPKG },
> > >      },
> > > +    {
> > > +        .from = { FEAT_7_1_EAX,             CPUID_7_1_EAX_LKGS },
> > > +        .to = { FEAT_7_1_EAX,               CPUID_7_1_EAX_FRED },
> > > +    },
> > > +    {
> > > +        .from = { FEAT_7_1_EAX,             CPUID_7_1_EAX_WRMSRNS },
> > > +        .to = { FEAT_7_1_EAX,               CPUID_7_1_EAX_FRED },
> > > +    },
> 
> Oh, sorry, one thing that comes to mind, is this dependency required?
> Since the FRED spec (v3.0) is all about WRMSR as the example, without
> mentioning WRMSRNS, could there be other implementations that depend on
> WRMSR instead of WRMSRNS?

This is a community ask from tglx:
https://lkml.kernel.org/kvm/87y1h81ht4.ffs@tglx/

Boris had the same question:
https://lore.kernel.org/lkml/20231114050201.GAZVL%2FSd%2FyLIdON9la@fat_crate.local/

But it needs to go through a formal approach, which takes time, to reach
the FRED public spec.

Thanks!
    Xin

Re: [PATCH v3A 1/6] target/i386: add support for FRED in CPUID enumeration

[Liu, Zhao1]

On Fri, Dec 22, 2023 at 08:24:52AM +0000, Li, Xin3 wrote:
> Date: Fri, 22 Dec 2023 08:24:52 +0000
> From: "Li, Xin3" <xin3.li@intel.com>
> Subject: RE: [PATCH v3A 1/6] target/i386: add support for FRED in CPUID
>  enumeration
> 
> 
> > > >              NULL, NULL, NULL, NULL, @@ -1552,6 +1552,14 @@ static
> > > > FeatureDep feature_dependencies[] = {
> > > >          .from = { FEAT_VMX_SECONDARY_CTLS,
> > VMX_SECONDARY_EXEC_ENABLE_USER_WAIT_PAUSE },
> > > >          .to = { FEAT_7_0_ECX,               CPUID_7_0_ECX_WAITPKG },
> > > >      },
> > > > +    {
> > > > +        .from = { FEAT_7_1_EAX,             CPUID_7_1_EAX_LKGS },
> > > > +        .to = { FEAT_7_1_EAX,               CPUID_7_1_EAX_FRED },
> > > > +    },
> > > > +    {
> > > > +        .from = { FEAT_7_1_EAX,             CPUID_7_1_EAX_WRMSRNS },
> > > > +        .to = { FEAT_7_1_EAX,               CPUID_7_1_EAX_FRED },
> > > > +    },
> > 
> > Oh, sorry, one thing that comes to mind, is this dependency required?
> > Since the FRED spec (v3.0) is all about WRMSR as the example, without
> > mentioning WRMSRNS, could there be other implementations that depend on
> > WRMSR instead of WRMSRNS?
> 
> This is a community ask from tglx:
> https://lkml.kernel.org/kvm/87y1h81ht4.ffs@tglx/
> 
> Boris had the same question:
> https://lore.kernel.org/lkml/20231114050201.GAZVL%2FSd%2FyLIdON9la@fat_crate.local/
> 
> But it needs to go through a formal approach, which takes time, to reach
> the FRED public spec.
> 

Thanks Xin! You can add a simple note in the commit message, such as
FRED's dependency on WRMSRNS will be documented, to avoid confusion
for later reviewers interested in FRED.

Regards,
Zhao

Re: [PATCH v3 2/6] target/i386: mark CR4.FRED not reserved

[Zhao Liu]

On Wed, Nov 08, 2023 at 11:20:08PM -0800, Xin Li wrote:
> Date: Wed,  8 Nov 2023 23:20:08 -0800
> From: Xin Li <xin3.li@intel.com>
> Subject: [PATCH v3 2/6] target/i386: mark CR4.FRED not reserved
> X-Mailer: git-send-email 2.42.0
> 
> The CR4.FRED bit, i.e., CR4[32], is no longer a reserved bit when FRED
> is exposed to guests, otherwise it is still a reserved bit.
> 
> Tested-by: Shan Kang <shan.kang@intel.com>
> Signed-off-by: Xin Li <xin3.li@intel.com>
> ---

Reviewed-by: Zhao Liu <zhao1.liu@intel.com>

>  target/i386/cpu.h | 12 +++++++++++-
>  1 file changed, 11 insertions(+), 1 deletion(-)
> 
> diff --git a/target/i386/cpu.h b/target/i386/cpu.h
> index 5faf00551d..e210957cba 100644
> --- a/target/i386/cpu.h
> +++ b/target/i386/cpu.h
> @@ -262,6 +262,12 @@ typedef enum X86Seg {
>  #define CR4_PKE_MASK   (1U << 22)
>  #define CR4_PKS_MASK   (1U << 24)
>  
> +#ifdef TARGET_X86_64
> +#define CR4_FRED_MASK   (1ULL << 32)
> +#else
> +#define CR4_FRED_MASK   0
> +#endif
> +
>  #define CR4_RESERVED_MASK \
>  (~(target_ulong)(CR4_VME_MASK | CR4_PVI_MASK | CR4_TSD_MASK \
>                  | CR4_DE_MASK | CR4_PSE_MASK | CR4_PAE_MASK \
> @@ -269,7 +275,8 @@ typedef enum X86Seg {
>                  | CR4_OSFXSR_MASK | CR4_OSXMMEXCPT_MASK | CR4_UMIP_MASK \
>                  | CR4_LA57_MASK \
>                  | CR4_FSGSBASE_MASK | CR4_PCIDE_MASK | CR4_OSXSAVE_MASK \
> -                | CR4_SMEP_MASK | CR4_SMAP_MASK | CR4_PKE_MASK | CR4_PKS_MASK))
> +                | CR4_SMEP_MASK | CR4_SMAP_MASK | CR4_PKE_MASK | CR4_PKS_MASK \
> +                | CR4_FRED_MASK))
>  
>  #define DR6_BD          (1 << 13)
>  #define DR6_BS          (1 << 14)
> @@ -2520,6 +2527,9 @@ static inline uint64_t cr4_reserved_bits(CPUX86State *env)
>      if (!(env->features[FEAT_7_0_ECX] & CPUID_7_0_ECX_PKS)) {
>          reserved_bits |= CR4_PKS_MASK;
>      }
> +    if (!(env->features[FEAT_7_1_EAX] & CPUID_7_1_EAX_FRED)) {
> +        reserved_bits |= CR4_FRED_MASK;
> +    }
>      return reserved_bits;
>  }
>  
> -- 
> 2.42.0
> 
>