[PATCH v2 0/2] x86: Enable user wait instructions

[PATCH v2 0/2] x86: Enable user wait instructions

[Tao Xu]

UMONITOR, UMWAIT and TPAUSE are a set of user wait instructions.

UMONITOR arms address monitoring hardware using an address. A store
to an address within the specified address range triggers the
monitoring hardware to wake up the processor waiting in umwait.

UMWAIT instructs the processor to enter an implementation-dependent
optimized state while monitoring a range of addresses. The optimized
state may be either a light-weight power/performance optimized state
(c0.1 state) or an improved power/performance optimized state
(c0.2 state).

TPAUSE instructs the processor to enter an implementation-dependent
optimized state c0.1 or c0.2 state and wake up when time-stamp counter
reaches specified timeout.

Availability of the user wait instructions is indicated by the presence
of the CPUID feature flag WAITPKG CPUID.0x07.0x0:ECX[5].

The patches enable the umonitor, umwait and tpause features in KVM.
Because umwait and tpause can put a (psysical) CPU into a power saving
state, by default we dont't expose it in kvm and provide a capability to
enable it. Use kvm capability to enable UMONITOR, UMWAIT and TPAUSE when
QEMU use "-overcommit cpu-pm=on, a VM can use UMONITOR, UMWAIT and TPAUSE
instructions. If the instruction causes a delay, the amount of time
delayed is called here the physical delay. The physical delay is first
computed by determining the virtual delay (the time to delay relative to
the VM’s timestamp counter). Otherwise, UMONITOR, UMWAIT and TPAUSE cause
an invalid-opcode exception(#UD).

The dependency KVM patch link:
https://lkml.org/lkml/2019/5/24/138

The release document ref below link:
https://software.intel.com/sites/default/files/\
managed/39/c5/325462-sdm-vol-1-2abcd-3abcd.pdf

Changelog:
v2:
	Separated from the series
	https://www.mail-archive.com/qemu-devel@nongnu.org/msg549526.html
	Use kvm capability to enable UMONITOR, UMWAIT and TPAUSE when
	QEMU use "-overcommit cpu-pm=on"
	
v1:
	Sent out with MOVDIRI/MOVDIR64B instructions patches

Tao Xu (2):
  x86/cpu: Add support for UMONITOR/UMWAIT/TPAUSE
  target/i386: Add support for save/load IA32_UMWAIT_CONTROL MSR

 linux-headers/linux/kvm.h |  1 +
 target/i386/cpu.c         |  3 ++-
 target/i386/cpu.h         |  3 +++
 target/i386/kvm.c         | 28 ++++++++++++++++++++++++++++
 target/i386/machine.c     | 20 ++++++++++++++++++++
 5 files changed, 54 insertions(+), 1 deletion(-)

-- 
2.20.1

[Qemu-devel] [PATCH v2 0/2] x86: Enable user wait instructions

[Tao Xu]

UMONITOR, UMWAIT and TPAUSE are a set of user wait instructions.

UMONITOR arms address monitoring hardware using an address. A store
to an address within the specified address range triggers the
monitoring hardware to wake up the processor waiting in umwait.

UMWAIT instructs the processor to enter an implementation-dependent
optimized state while monitoring a range of addresses. The optimized
state may be either a light-weight power/performance optimized state
(c0.1 state) or an improved power/performance optimized state
(c0.2 state).

TPAUSE instructs the processor to enter an implementation-dependent
optimized state c0.1 or c0.2 state and wake up when time-stamp counter
reaches specified timeout.

Availability of the user wait instructions is indicated by the presence
of the CPUID feature flag WAITPKG CPUID.0x07.0x0:ECX[5].

The patches enable the umonitor, umwait and tpause features in KVM.
Because umwait and tpause can put a (psysical) CPU into a power saving
state, by default we dont't expose it in kvm and provide a capability to
enable it. Use kvm capability to enable UMONITOR, UMWAIT and TPAUSE when
QEMU use "-overcommit cpu-pm=on, a VM can use UMONITOR, UMWAIT and TPAUSE
instructions. If the instruction causes a delay, the amount of time
delayed is called here the physical delay. The physical delay is first
computed by determining the virtual delay (the time to delay relative to
the VM’s timestamp counter). Otherwise, UMONITOR, UMWAIT and TPAUSE cause
an invalid-opcode exception(#UD).

The dependency KVM patch link:
https://lkml.org/lkml/2019/5/24/138

The release document ref below link:
https://software.intel.com/sites/default/files/\
managed/39/c5/325462-sdm-vol-1-2abcd-3abcd.pdf

Changelog:
v2:
	Separated from the series
	https://www.mail-archive.com/qemu-devel@nongnu.org/msg549526.html
	Use kvm capability to enable UMONITOR, UMWAIT and TPAUSE when
	QEMU use "-overcommit cpu-pm=on"
	
v1:
	Sent out with MOVDIRI/MOVDIR64B instructions patches

Tao Xu (2):
  x86/cpu: Add support for UMONITOR/UMWAIT/TPAUSE
  target/i386: Add support for save/load IA32_UMWAIT_CONTROL MSR

 linux-headers/linux/kvm.h |  1 +
 target/i386/cpu.c         |  3 ++-
 target/i386/cpu.h         |  3 +++
 target/i386/kvm.c         | 28 ++++++++++++++++++++++++++++
 target/i386/machine.c     | 20 ++++++++++++++++++++
 5 files changed, 54 insertions(+), 1 deletion(-)

-- 
2.20.1

[PATCH v2 1/2] x86/cpu: Add support for UMONITOR/UMWAIT/TPAUSE

[Tao Xu]

UMONITOR, UMWAIT and TPAUSE are a set of user wait instructions.
Availability of the user wait instructions is indicated by the presence
of the CPUID feature flag WAITPKG CPUID.0x07.0x0:ECX[5].

The patch enable the umonitor, umwait and tpause features in KVM.
Because umwait and tpause can put a (psysical) CPU into a power saving
state, by default we dont't expose it in kvm and provide a capability to
enable it. Use kvm capability to enable UMONITOR, UMWAIT and TPAUSE when
QEMU use "-overcommit cpu-pm=on, a VM can use UMONITOR, UMWAIT and TPAUSE
instructions. If the instruction causes a delay, the amount of time
delayed is called here the physical delay. The physical delay is first
computed by determining the virtual delay (the time to delay relative to
the VM’s timestamp counter). Otherwise, UMONITOR, UMWAIT and TPAUSE cause
an invalid-opcode exception(#UD).

The release document ref below link:
https://software.intel.com/sites/default/files/\
managed/39/c5/325462-sdm-vol-1-2abcd-3abcd.pdf

Co-developed-by: Jingqi Liu <jingqi.liu@intel.com>
Signed-off-by: Jingqi Liu <jingqi.liu@intel.com>
Signed-off-by: Tao Xu <tao3.xu@intel.com>
---
 linux-headers/linux/kvm.h |  1 +
 target/i386/cpu.c         |  3 ++-
 target/i386/cpu.h         |  1 +
 target/i386/kvm.c         | 13 +++++++++++++
 4 files changed, 17 insertions(+), 1 deletion(-)

diff --git a/linux-headers/linux/kvm.h b/linux-headers/linux/kvm.h
index c8423e760c..86cc2dbdd0 100644
--- a/linux-headers/linux/kvm.h
+++ b/linux-headers/linux/kvm.h
@@ -993,6 +993,7 @@ struct kvm_ppc_resize_hpt {
 #define KVM_CAP_ARM_SVE 170
 #define KVM_CAP_ARM_PTRAUTH_ADDRESS 171
 #define KVM_CAP_ARM_PTRAUTH_GENERIC 172
+#define KVM_CAP_ENABLE_USR_WAIT_PAUSE 173
 
 #ifdef KVM_CAP_IRQ_ROUTING
 
diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index 2df56fa977..c2ad7866a5 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -1057,7 +1057,7 @@ static FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
         .type = CPUID_FEATURE_WORD,
         .feat_names = {
             NULL, "avx512vbmi", "umip", "pku",
-            NULL /* ospke */, NULL, "avx512vbmi2", NULL,
+            NULL /* ospke */, "waitpkg", "avx512vbmi2", NULL,
             "gfni", "vaes", "vpclmulqdq", "avx512vnni",
             "avx512bitalg", NULL, "avx512-vpopcntdq", NULL,
             "la57", NULL, NULL, NULL,
@@ -5196,6 +5196,7 @@ static void x86_cpu_realizefn(DeviceState *dev, Error **errp)
             host_cpuid(5, 0, &cpu->mwait.eax, &cpu->mwait.ebx,
                        &cpu->mwait.ecx, &cpu->mwait.edx);
             env->features[FEAT_1_ECX] |= CPUID_EXT_MONITOR;
+            env->features[FEAT_7_0_ECX] |= CPUID_7_0_ECX_WAITPKG;
         }
     }
 
diff --git a/target/i386/cpu.h b/target/i386/cpu.h
index fce6660bac..66fa61f02b 100644
--- a/target/i386/cpu.h
+++ b/target/i386/cpu.h
@@ -679,6 +679,7 @@ typedef uint32_t FeatureWordArray[FEATURE_WORDS];
 #define CPUID_7_0_ECX_UMIP     (1U << 2)
 #define CPUID_7_0_ECX_PKU      (1U << 3)
 #define CPUID_7_0_ECX_OSPKE    (1U << 4)
+#define CPUID_7_0_ECX_WAITPKG  (1U << 5) /* UMONITOR/UMWAIT/TPAUSE Instructions */
 #define CPUID_7_0_ECX_VBMI2    (1U << 6) /* Additional VBMI Instrs */
 #define CPUID_7_0_ECX_GFNI     (1U << 8)
 #define CPUID_7_0_ECX_VAES     (1U << 9)
diff --git a/target/i386/kvm.c b/target/i386/kvm.c
index 3b29ce5c0d..b58fc7ab3a 100644
--- a/target/i386/kvm.c
+++ b/target/i386/kvm.c
@@ -389,6 +389,10 @@ uint32_t kvm_arch_get_supported_cpuid(KVMState *s, uint32_t function,
         if (host_tsx_blacklisted()) {
             ret &= ~(CPUID_7_0_EBX_RTM | CPUID_7_0_EBX_HLE);
         }
+    } else if (function == 7 && index == 0 && reg == R_ECX) {
+        if (enable_cpu_pm) {
+            ret |= CPUID_7_0_ECX_WAITPKG;
+        }
     } else if (function == 7 && index == 0 && reg == R_EDX) {
         /*
          * Linux v4.17-v4.20 incorrectly return ARCH_CAPABILITIES on SVM hosts.
@@ -1654,6 +1658,15 @@ int kvm_arch_init(MachineState *ms, KVMState *s)
             error_report("kvm: guest stopping CPU not supported: %s",
                          strerror(-ret));
         }
+
+        if (kvm_check_extension(s, KVM_CAP_ENABLE_USR_WAIT_PAUSE)) {
+            ret = kvm_vm_enable_cap(s, KVM_CAP_ENABLE_USR_WAIT_PAUSE, 0);
+            if (ret < 0) {
+                error_report("kvm: guest can't enable user-level"
+                             " wait and pause: %s",
+                             strerror(-ret));
+            }
+        }
     }
 
     return 0;
-- 
2.20.1

[Qemu-devel] [PATCH v2 1/2] x86/cpu: Add support for UMONITOR/UMWAIT/TPAUSE

[Tao Xu]

UMONITOR, UMWAIT and TPAUSE are a set of user wait instructions.
Availability of the user wait instructions is indicated by the presence
of the CPUID feature flag WAITPKG CPUID.0x07.0x0:ECX[5].

The patch enable the umonitor, umwait and tpause features in KVM.
Because umwait and tpause can put a (psysical) CPU into a power saving
state, by default we dont't expose it in kvm and provide a capability to
enable it. Use kvm capability to enable UMONITOR, UMWAIT and TPAUSE when
QEMU use "-overcommit cpu-pm=on, a VM can use UMONITOR, UMWAIT and TPAUSE
instructions. If the instruction causes a delay, the amount of time
delayed is called here the physical delay. The physical delay is first
computed by determining the virtual delay (the time to delay relative to
the VM’s timestamp counter). Otherwise, UMONITOR, UMWAIT and TPAUSE cause
an invalid-opcode exception(#UD).

The release document ref below link:
https://software.intel.com/sites/default/files/\
managed/39/c5/325462-sdm-vol-1-2abcd-3abcd.pdf

Co-developed-by: Jingqi Liu <jingqi.liu@intel.com>
Signed-off-by: Jingqi Liu <jingqi.liu@intel.com>
Signed-off-by: Tao Xu <tao3.xu@intel.com>
---
 linux-headers/linux/kvm.h |  1 +
 target/i386/cpu.c         |  3 ++-
 target/i386/cpu.h         |  1 +
 target/i386/kvm.c         | 13 +++++++++++++
 4 files changed, 17 insertions(+), 1 deletion(-)

diff --git a/linux-headers/linux/kvm.h b/linux-headers/linux/kvm.h
index c8423e760c..86cc2dbdd0 100644
--- a/linux-headers/linux/kvm.h
+++ b/linux-headers/linux/kvm.h
@@ -993,6 +993,7 @@ struct kvm_ppc_resize_hpt {
 #define KVM_CAP_ARM_SVE 170
 #define KVM_CAP_ARM_PTRAUTH_ADDRESS 171
 #define KVM_CAP_ARM_PTRAUTH_GENERIC 172
+#define KVM_CAP_ENABLE_USR_WAIT_PAUSE 173
 
 #ifdef KVM_CAP_IRQ_ROUTING
 
diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index 2df56fa977..c2ad7866a5 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -1057,7 +1057,7 @@ static FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
         .type = CPUID_FEATURE_WORD,
         .feat_names = {
             NULL, "avx512vbmi", "umip", "pku",
-            NULL /* ospke */, NULL, "avx512vbmi2", NULL,
+            NULL /* ospke */, "waitpkg", "avx512vbmi2", NULL,
             "gfni", "vaes", "vpclmulqdq", "avx512vnni",
             "avx512bitalg", NULL, "avx512-vpopcntdq", NULL,
             "la57", NULL, NULL, NULL,
@@ -5196,6 +5196,7 @@ static void x86_cpu_realizefn(DeviceState *dev, Error **errp)
             host_cpuid(5, 0, &cpu->mwait.eax, &cpu->mwait.ebx,
                        &cpu->mwait.ecx, &cpu->mwait.edx);
             env->features[FEAT_1_ECX] |= CPUID_EXT_MONITOR;
+            env->features[FEAT_7_0_ECX] |= CPUID_7_0_ECX_WAITPKG;
         }
     }
 
diff --git a/target/i386/cpu.h b/target/i386/cpu.h
index fce6660bac..66fa61f02b 100644
--- a/target/i386/cpu.h
+++ b/target/i386/cpu.h
@@ -679,6 +679,7 @@ typedef uint32_t FeatureWordArray[FEATURE_WORDS];
 #define CPUID_7_0_ECX_UMIP     (1U << 2)
 #define CPUID_7_0_ECX_PKU      (1U << 3)
 #define CPUID_7_0_ECX_OSPKE    (1U << 4)
+#define CPUID_7_0_ECX_WAITPKG  (1U << 5) /* UMONITOR/UMWAIT/TPAUSE Instructions */
 #define CPUID_7_0_ECX_VBMI2    (1U << 6) /* Additional VBMI Instrs */
 #define CPUID_7_0_ECX_GFNI     (1U << 8)
 #define CPUID_7_0_ECX_VAES     (1U << 9)
diff --git a/target/i386/kvm.c b/target/i386/kvm.c
index 3b29ce5c0d..b58fc7ab3a 100644
--- a/target/i386/kvm.c
+++ b/target/i386/kvm.c
@@ -389,6 +389,10 @@ uint32_t kvm_arch_get_supported_cpuid(KVMState *s, uint32_t function,
         if (host_tsx_blacklisted()) {
             ret &= ~(CPUID_7_0_EBX_RTM | CPUID_7_0_EBX_HLE);
         }
+    } else if (function == 7 && index == 0 && reg == R_ECX) {
+        if (enable_cpu_pm) {
+            ret |= CPUID_7_0_ECX_WAITPKG;
+        }
     } else if (function == 7 && index == 0 && reg == R_EDX) {
         /*
          * Linux v4.17-v4.20 incorrectly return ARCH_CAPABILITIES on SVM hosts.
@@ -1654,6 +1658,15 @@ int kvm_arch_init(MachineState *ms, KVMState *s)
             error_report("kvm: guest stopping CPU not supported: %s",
                          strerror(-ret));
         }
+
+        if (kvm_check_extension(s, KVM_CAP_ENABLE_USR_WAIT_PAUSE)) {
+            ret = kvm_vm_enable_cap(s, KVM_CAP_ENABLE_USR_WAIT_PAUSE, 0);
+            if (ret < 0) {
+                error_report("kvm: guest can't enable user-level"
+                             " wait and pause: %s",
+                             strerror(-ret));
+            }
+        }
     }
 
     return 0;
-- 
2.20.1

[PATCH v2 2/2] target/i386: Add support for save/load IA32_UMWAIT_CONTROL MSR

[Tao Xu]

UMWAIT and TPAUSE instructions use IA32_UMWAIT_CONTROL at MSR index
E1H to determines the maximum time in TSC-quanta that the processor
can reside in either C0.1 or C0.2.

This patch is to Add support for save/load IA32_UMWAIT_CONTROL MSR in
guest.

Co-developed-by: Jingqi Liu <jingqi.liu@intel.com>
Signed-off-by: Jingqi Liu <jingqi.liu@intel.com>
Signed-off-by: Tao Xu <tao3.xu@intel.com>
---
 target/i386/cpu.h     |  2 ++
 target/i386/kvm.c     | 15 +++++++++++++++
 target/i386/machine.c | 20 ++++++++++++++++++++
 3 files changed, 37 insertions(+)

diff --git a/target/i386/cpu.h b/target/i386/cpu.h
index 66fa61f02b..9d1c8edc1f 100644
--- a/target/i386/cpu.h
+++ b/target/i386/cpu.h
@@ -459,6 +459,7 @@ typedef enum X86Seg {
 
 #define MSR_IA32_BNDCFGS                0x00000d90
 #define MSR_IA32_XSS                    0x00000da0
+#define MSR_IA32_UMWAIT_CONTROL         0xe1
 
 #define XSTATE_FP_BIT                   0
 #define XSTATE_SSE_BIT                  1
@@ -1360,6 +1361,7 @@ typedef struct CPUX86State {
     uint16_t fpregs_format_vmstate;
 
     uint64_t xss;
+    uint64_t umwait;
 
     TPRAccess tpr_access_type;
 } CPUX86State;
diff --git a/target/i386/kvm.c b/target/i386/kvm.c
index b58fc7ab3a..7845f684fc 100644
--- a/target/i386/kvm.c
+++ b/target/i386/kvm.c
@@ -92,6 +92,7 @@ static bool has_msr_hv_stimer;
 static bool has_msr_hv_frequencies;
 static bool has_msr_hv_reenlightenment;
 static bool has_msr_xss;
+static bool has_msr_umwait;
 static bool has_msr_spec_ctrl;
 static bool has_msr_virt_ssbd;
 static bool has_msr_smi_count;
@@ -1487,6 +1488,9 @@ static int kvm_get_supported_msrs(KVMState *s)
                 case MSR_IA32_XSS:
                     has_msr_xss = true;
                     break;
+                case MSR_IA32_UMWAIT_CONTROL:
+                    has_msr_umwait = true;
+                    break;
                 case HV_X64_MSR_CRASH_CTL:
                     has_msr_hv_crash = true;
                     break;
@@ -1667,6 +1671,8 @@ int kvm_arch_init(MachineState *ms, KVMState *s)
                              strerror(-ret));
             }
         }
+    } else {
+        has_msr_umwait = false;
     }
 
     return 0;
@@ -2033,6 +2039,9 @@ static int kvm_put_msrs(X86CPU *cpu, int level)
     if (has_msr_xss) {
         kvm_msr_entry_add(cpu, MSR_IA32_XSS, env->xss);
     }
+    if (has_msr_umwait) {
+        kvm_msr_entry_add(cpu, MSR_IA32_UMWAIT_CONTROL, env->umwait);
+    }
     if (has_msr_spec_ctrl) {
         kvm_msr_entry_add(cpu, MSR_IA32_SPEC_CTRL, env->spec_ctrl);
     }
@@ -2426,6 +2435,9 @@ static int kvm_get_msrs(X86CPU *cpu)
     if (has_msr_xss) {
         kvm_msr_entry_add(cpu, MSR_IA32_XSS, 0);
     }
+    if (has_msr_umwait) {
+        kvm_msr_entry_add(cpu, MSR_IA32_UMWAIT_CONTROL, 0);
+    }
     if (has_msr_spec_ctrl) {
         kvm_msr_entry_add(cpu, MSR_IA32_SPEC_CTRL, 0);
     }
@@ -2675,6 +2687,9 @@ static int kvm_get_msrs(X86CPU *cpu)
         case MSR_IA32_XSS:
             env->xss = msrs[i].data;
             break;
+        case MSR_IA32_UMWAIT_CONTROL:
+            env->umwait = msrs[i].data;
+            break;
         default:
             if (msrs[i].index >= MSR_MC0_CTL &&
                 msrs[i].index < MSR_MC0_CTL + (env->mcg_cap & 0xff) * 4) {
diff --git a/target/i386/machine.c b/target/i386/machine.c
index 225b5d433b..e51285a583 100644
--- a/target/i386/machine.c
+++ b/target/i386/machine.c
@@ -810,6 +810,25 @@ static const VMStateDescription vmstate_xss = {
     }
 };
 
+static bool umwait_needed(void *opaque)
+{
+    X86CPU *cpu = opaque;
+    CPUX86State *env = &cpu->env;
+
+    return env->umwait != 0;
+}
+
+static const VMStateDescription vmstate_umwait = {
+    .name = "cpu/umwait",
+    .version_id = 1,
+    .minimum_version_id = 1,
+    .needed = umwait_needed,
+    .fields = (VMStateField[]) {
+        VMSTATE_UINT64(env.umwait, X86CPU),
+        VMSTATE_END_OF_LIST()
+    }
+};
+
 #ifdef TARGET_X86_64
 static bool pkru_needed(void *opaque)
 {
@@ -1079,6 +1098,7 @@ VMStateDescription vmstate_x86_cpu = {
         &vmstate_msr_hyperv_reenlightenment,
         &vmstate_avx512,
         &vmstate_xss,
+        &vmstate_umwait,
         &vmstate_tsc_khz,
         &vmstate_msr_smi_count,
 #ifdef TARGET_X86_64
-- 
2.20.1

[Qemu-devel] [PATCH v2 2/2] target/i386: Add support for save/load IA32_UMWAIT_CONTROL MSR

[Tao Xu]

UMWAIT and TPAUSE instructions use IA32_UMWAIT_CONTROL at MSR index
E1H to determines the maximum time in TSC-quanta that the processor
can reside in either C0.1 or C0.2.

This patch is to Add support for save/load IA32_UMWAIT_CONTROL MSR in
guest.

Co-developed-by: Jingqi Liu <jingqi.liu@intel.com>
Signed-off-by: Jingqi Liu <jingqi.liu@intel.com>
Signed-off-by: Tao Xu <tao3.xu@intel.com>
---
 target/i386/cpu.h     |  2 ++
 target/i386/kvm.c     | 15 +++++++++++++++
 target/i386/machine.c | 20 ++++++++++++++++++++
 3 files changed, 37 insertions(+)

diff --git a/target/i386/cpu.h b/target/i386/cpu.h
index 66fa61f02b..9d1c8edc1f 100644
--- a/target/i386/cpu.h
+++ b/target/i386/cpu.h
@@ -459,6 +459,7 @@ typedef enum X86Seg {
 
 #define MSR_IA32_BNDCFGS                0x00000d90
 #define MSR_IA32_XSS                    0x00000da0
+#define MSR_IA32_UMWAIT_CONTROL         0xe1
 
 #define XSTATE_FP_BIT                   0
 #define XSTATE_SSE_BIT                  1
@@ -1360,6 +1361,7 @@ typedef struct CPUX86State {
     uint16_t fpregs_format_vmstate;
 
     uint64_t xss;
+    uint64_t umwait;
 
     TPRAccess tpr_access_type;
 } CPUX86State;
diff --git a/target/i386/kvm.c b/target/i386/kvm.c
index b58fc7ab3a..7845f684fc 100644
--- a/target/i386/kvm.c
+++ b/target/i386/kvm.c
@@ -92,6 +92,7 @@ static bool has_msr_hv_stimer;
 static bool has_msr_hv_frequencies;
 static bool has_msr_hv_reenlightenment;
 static bool has_msr_xss;
+static bool has_msr_umwait;
 static bool has_msr_spec_ctrl;
 static bool has_msr_virt_ssbd;
 static bool has_msr_smi_count;
@@ -1487,6 +1488,9 @@ static int kvm_get_supported_msrs(KVMState *s)
                 case MSR_IA32_XSS:
                     has_msr_xss = true;
                     break;
+                case MSR_IA32_UMWAIT_CONTROL:
+                    has_msr_umwait = true;
+                    break;
                 case HV_X64_MSR_CRASH_CTL:
                     has_msr_hv_crash = true;
                     break;
@@ -1667,6 +1671,8 @@ int kvm_arch_init(MachineState *ms, KVMState *s)
                              strerror(-ret));
             }
         }
+    } else {
+        has_msr_umwait = false;
     }
 
     return 0;
@@ -2033,6 +2039,9 @@ static int kvm_put_msrs(X86CPU *cpu, int level)
     if (has_msr_xss) {
         kvm_msr_entry_add(cpu, MSR_IA32_XSS, env->xss);
     }
+    if (has_msr_umwait) {
+        kvm_msr_entry_add(cpu, MSR_IA32_UMWAIT_CONTROL, env->umwait);
+    }
     if (has_msr_spec_ctrl) {
         kvm_msr_entry_add(cpu, MSR_IA32_SPEC_CTRL, env->spec_ctrl);
     }
@@ -2426,6 +2435,9 @@ static int kvm_get_msrs(X86CPU *cpu)
     if (has_msr_xss) {
         kvm_msr_entry_add(cpu, MSR_IA32_XSS, 0);
     }
+    if (has_msr_umwait) {
+        kvm_msr_entry_add(cpu, MSR_IA32_UMWAIT_CONTROL, 0);
+    }
     if (has_msr_spec_ctrl) {
         kvm_msr_entry_add(cpu, MSR_IA32_SPEC_CTRL, 0);
     }
@@ -2675,6 +2687,9 @@ static int kvm_get_msrs(X86CPU *cpu)
         case MSR_IA32_XSS:
             env->xss = msrs[i].data;
             break;
+        case MSR_IA32_UMWAIT_CONTROL:
+            env->umwait = msrs[i].data;
+            break;
         default:
             if (msrs[i].index >= MSR_MC0_CTL &&
                 msrs[i].index < MSR_MC0_CTL + (env->mcg_cap & 0xff) * 4) {
diff --git a/target/i386/machine.c b/target/i386/machine.c
index 225b5d433b..e51285a583 100644
--- a/target/i386/machine.c
+++ b/target/i386/machine.c
@@ -810,6 +810,25 @@ static const VMStateDescription vmstate_xss = {
     }
 };
 
+static bool umwait_needed(void *opaque)
+{
+    X86CPU *cpu = opaque;
+    CPUX86State *env = &cpu->env;
+
+    return env->umwait != 0;
+}
+
+static const VMStateDescription vmstate_umwait = {
+    .name = "cpu/umwait",
+    .version_id = 1,
+    .minimum_version_id = 1,
+    .needed = umwait_needed,
+    .fields = (VMStateField[]) {
+        VMSTATE_UINT64(env.umwait, X86CPU),
+        VMSTATE_END_OF_LIST()
+    }
+};
+
 #ifdef TARGET_X86_64
 static bool pkru_needed(void *opaque)
 {
@@ -1079,6 +1098,7 @@ VMStateDescription vmstate_x86_cpu = {
         &vmstate_msr_hyperv_reenlightenment,
         &vmstate_avx512,
         &vmstate_xss,
+        &vmstate_umwait,
         &vmstate_tsc_khz,
         &vmstate_msr_smi_count,
 #ifdef TARGET_X86_64
-- 
2.20.1

Re: [PATCH v2 1/2] x86/cpu: Add support for UMONITOR/UMWAIT/TPAUSE

[Cornelia Huck]

On Fri, 24 May 2019 16:18:38 +0800
Tao Xu <tao3.xu@intel.com> wrote:

> UMONITOR, UMWAIT and TPAUSE are a set of user wait instructions.
> Availability of the user wait instructions is indicated by the presence
> of the CPUID feature flag WAITPKG CPUID.0x07.0x0:ECX[5].
> 
> The patch enable the umonitor, umwait and tpause features in KVM.
> Because umwait and tpause can put a (psysical) CPU into a power saving
> state, by default we dont't expose it in kvm and provide a capability to
> enable it. Use kvm capability to enable UMONITOR, UMWAIT and TPAUSE when
> QEMU use "-overcommit cpu-pm=on, a VM can use UMONITOR, UMWAIT and TPAUSE
> instructions. If the instruction causes a delay, the amount of time
> delayed is called here the physical delay. The physical delay is first
> computed by determining the virtual delay (the time to delay relative to
> the VM’s timestamp counter). Otherwise, UMONITOR, UMWAIT and TPAUSE cause
> an invalid-opcode exception(#UD).
> 
> The release document ref below link:
> https://software.intel.com/sites/default/files/\
> managed/39/c5/325462-sdm-vol-1-2abcd-3abcd.pdf
> 
> Co-developed-by: Jingqi Liu <jingqi.liu@intel.com>
> Signed-off-by: Jingqi Liu <jingqi.liu@intel.com>
> Signed-off-by: Tao Xu <tao3.xu@intel.com>
> ---
>  linux-headers/linux/kvm.h |  1 +
>  target/i386/cpu.c         |  3 ++-
>  target/i386/cpu.h         |  1 +
>  target/i386/kvm.c         | 13 +++++++++++++
>  4 files changed, 17 insertions(+), 1 deletion(-)
> 
> diff --git a/linux-headers/linux/kvm.h b/linux-headers/linux/kvm.h
> index c8423e760c..86cc2dbdd0 100644
> --- a/linux-headers/linux/kvm.h
> +++ b/linux-headers/linux/kvm.h
> @@ -993,6 +993,7 @@ struct kvm_ppc_resize_hpt {
>  #define KVM_CAP_ARM_SVE 170
>  #define KVM_CAP_ARM_PTRAUTH_ADDRESS 171
>  #define KVM_CAP_ARM_PTRAUTH_GENERIC 172
> +#define KVM_CAP_ENABLE_USR_WAIT_PAUSE 173
>  
>  #ifdef KVM_CAP_IRQ_ROUTING
>  

No comment on the actual change, but please split out any linux-header
changes so they can be replaced with a proper headers update when the
code is merged.

Re: [Qemu-devel] [PATCH v2 1/2] x86/cpu: Add support for UMONITOR/UMWAIT/TPAUSE

[Cornelia Huck]

On Fri, 24 May 2019 16:18:38 +0800
Tao Xu <tao3.xu@intel.com> wrote:

> UMONITOR, UMWAIT and TPAUSE are a set of user wait instructions.
> Availability of the user wait instructions is indicated by the presence
> of the CPUID feature flag WAITPKG CPUID.0x07.0x0:ECX[5].
> 
> The patch enable the umonitor, umwait and tpause features in KVM.
> Because umwait and tpause can put a (psysical) CPU into a power saving
> state, by default we dont't expose it in kvm and provide a capability to
> enable it. Use kvm capability to enable UMONITOR, UMWAIT and TPAUSE when
> QEMU use "-overcommit cpu-pm=on, a VM can use UMONITOR, UMWAIT and TPAUSE
> instructions. If the instruction causes a delay, the amount of time
> delayed is called here the physical delay. The physical delay is first
> computed by determining the virtual delay (the time to delay relative to
> the VM’s timestamp counter). Otherwise, UMONITOR, UMWAIT and TPAUSE cause
> an invalid-opcode exception(#UD).
> 
> The release document ref below link:
> https://software.intel.com/sites/default/files/\
> managed/39/c5/325462-sdm-vol-1-2abcd-3abcd.pdf
> 
> Co-developed-by: Jingqi Liu <jingqi.liu@intel.com>
> Signed-off-by: Jingqi Liu <jingqi.liu@intel.com>
> Signed-off-by: Tao Xu <tao3.xu@intel.com>
> ---
>  linux-headers/linux/kvm.h |  1 +
>  target/i386/cpu.c         |  3 ++-
>  target/i386/cpu.h         |  1 +
>  target/i386/kvm.c         | 13 +++++++++++++
>  4 files changed, 17 insertions(+), 1 deletion(-)
> 
> diff --git a/linux-headers/linux/kvm.h b/linux-headers/linux/kvm.h
> index c8423e760c..86cc2dbdd0 100644
> --- a/linux-headers/linux/kvm.h
> +++ b/linux-headers/linux/kvm.h
> @@ -993,6 +993,7 @@ struct kvm_ppc_resize_hpt {
>  #define KVM_CAP_ARM_SVE 170
>  #define KVM_CAP_ARM_PTRAUTH_ADDRESS 171
>  #define KVM_CAP_ARM_PTRAUTH_GENERIC 172
> +#define KVM_CAP_ENABLE_USR_WAIT_PAUSE 173
>  
>  #ifdef KVM_CAP_IRQ_ROUTING
>  

No comment on the actual change, but please split out any linux-header
changes so they can be replaced with a proper headers update when the
code is merged.

Re: [PATCH v2 1/2] x86/cpu: Add support for UMONITOR/UMWAIT/TPAUSE

[Tao Xu]

On 6/4/2019 10:34 PM, Cornelia Huck wrote:
> On Fri, 24 May 2019 16:18:38 +0800
> Tao Xu <tao3.xu@intel.com> wrote:
> 
>> UMONITOR, UMWAIT and TPAUSE are a set of user wait instructions.
>> Availability of the user wait instructions is indicated by the presence
>> of the CPUID feature flag WAITPKG CPUID.0x07.0x0:ECX[5].
>>
>> The patch enable the umonitor, umwait and tpause features in KVM.
>> Because umwait and tpause can put a (psysical) CPU into a power saving
>> state, by default we dont't expose it in kvm and provide a capability to
>> enable it. Use kvm capability to enable UMONITOR, UMWAIT and TPAUSE when
>> QEMU use "-overcommit cpu-pm=on, a VM can use UMONITOR, UMWAIT and TPAUSE
>> instructions. If the instruction causes a delay, the amount of time
>> delayed is called here the physical delay. The physical delay is first
>> computed by determining the virtual delay (the time to delay relative to
>> the VM’s timestamp counter). Otherwise, UMONITOR, UMWAIT and TPAUSE cause
>> an invalid-opcode exception(#UD).
>>
>> The release document ref below link:
>> https://software.intel.com/sites/default/files/\
>> managed/39/c5/325462-sdm-vol-1-2abcd-3abcd.pdf
>>
>> Co-developed-by: Jingqi Liu <jingqi.liu@intel.com>
>> Signed-off-by: Jingqi Liu <jingqi.liu@intel.com>
>> Signed-off-by: Tao Xu <tao3.xu@intel.com>
>> ---
>>   linux-headers/linux/kvm.h |  1 +
>>   target/i386/cpu.c         |  3 ++-
>>   target/i386/cpu.h         |  1 +
>>   target/i386/kvm.c         | 13 +++++++++++++
>>   4 files changed, 17 insertions(+), 1 deletion(-)
>>
>> diff --git a/linux-headers/linux/kvm.h b/linux-headers/linux/kvm.h
>> index c8423e760c..86cc2dbdd0 100644
>> --- a/linux-headers/linux/kvm.h
>> +++ b/linux-headers/linux/kvm.h
>> @@ -993,6 +993,7 @@ struct kvm_ppc_resize_hpt {
>>   #define KVM_CAP_ARM_SVE 170
>>   #define KVM_CAP_ARM_PTRAUTH_ADDRESS 171
>>   #define KVM_CAP_ARM_PTRAUTH_GENERIC 172
>> +#define KVM_CAP_ENABLE_USR_WAIT_PAUSE 173
>>   
>>   #ifdef KVM_CAP_IRQ_ROUTING
>>   
> 
> No comment on the actual change, but please split out any linux-header
> changes so they can be replaced with a proper headers update when the
> code is merged.
> 
Thank you for your review, I will update this patch when the kvm patch 
change a lot.

Re: [Qemu-devel] [PATCH v2 1/2] x86/cpu: Add support for UMONITOR/UMWAIT/TPAUSE

[Tao Xu]

On 6/4/2019 10:34 PM, Cornelia Huck wrote:
> On Fri, 24 May 2019 16:18:38 +0800
> Tao Xu <tao3.xu@intel.com> wrote:
> 
>> UMONITOR, UMWAIT and TPAUSE are a set of user wait instructions.
>> Availability of the user wait instructions is indicated by the presence
>> of the CPUID feature flag WAITPKG CPUID.0x07.0x0:ECX[5].
>>
>> The patch enable the umonitor, umwait and tpause features in KVM.
>> Because umwait and tpause can put a (psysical) CPU into a power saving
>> state, by default we dont't expose it in kvm and provide a capability to
>> enable it. Use kvm capability to enable UMONITOR, UMWAIT and TPAUSE when
>> QEMU use "-overcommit cpu-pm=on, a VM can use UMONITOR, UMWAIT and TPAUSE
>> instructions. If the instruction causes a delay, the amount of time
>> delayed is called here the physical delay. The physical delay is first
>> computed by determining the virtual delay (the time to delay relative to
>> the VM’s timestamp counter). Otherwise, UMONITOR, UMWAIT and TPAUSE cause
>> an invalid-opcode exception(#UD).
>>
>> The release document ref below link:
>> https://software.intel.com/sites/default/files/\
>> managed/39/c5/325462-sdm-vol-1-2abcd-3abcd.pdf
>>
>> Co-developed-by: Jingqi Liu <jingqi.liu@intel.com>
>> Signed-off-by: Jingqi Liu <jingqi.liu@intel.com>
>> Signed-off-by: Tao Xu <tao3.xu@intel.com>
>> ---
>>   linux-headers/linux/kvm.h |  1 +
>>   target/i386/cpu.c         |  3 ++-
>>   target/i386/cpu.h         |  1 +
>>   target/i386/kvm.c         | 13 +++++++++++++
>>   4 files changed, 17 insertions(+), 1 deletion(-)
>>
>> diff --git a/linux-headers/linux/kvm.h b/linux-headers/linux/kvm.h
>> index c8423e760c..86cc2dbdd0 100644
>> --- a/linux-headers/linux/kvm.h
>> +++ b/linux-headers/linux/kvm.h
>> @@ -993,6 +993,7 @@ struct kvm_ppc_resize_hpt {
>>   #define KVM_CAP_ARM_SVE 170
>>   #define KVM_CAP_ARM_PTRAUTH_ADDRESS 171
>>   #define KVM_CAP_ARM_PTRAUTH_GENERIC 172
>> +#define KVM_CAP_ENABLE_USR_WAIT_PAUSE 173
>>   
>>   #ifdef KVM_CAP_IRQ_ROUTING
>>   
> 
> No comment on the actual change, but please split out any linux-header
> changes so they can be replaced with a proper headers update when the
> code is merged.
> 
Thank you for your review, I will update this patch when the kvm patch 
change a lot.