kvm.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: "Xu, Like" <like.xu@intel.com>
To: "Liuxiangdong (Aven,
	Cloud Infrastructure Service Product Dept.)" 
	<liuxiangdong5@huawei.com>
Cc: "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	Xiexiangyou <xiexiangyou@huawei.com>,
	Wei Wang <wei.w.wang@intel.com>, kvm <kvm@vger.kernel.org>,
	Like Xu <like.xu@linux.intel.com>,
	"Fangyi (Eric)" <eric.fangyi@huawei.com>
Subject: Re: [PATCH v3 00/17] KVM: x86/pmu: Add support to enable Guest PEBS via DS
Date: Tue, 26 Jan 2021 15:08:09 +0800	[thread overview]
Message-ID: <f4dcb068-2ddf-428f-50ad-39f65cad3710@intel.com> (raw)
In-Reply-To: <600ED9F7.1060503@huawei.com>

[-- Attachment #1: Type: text/plain, Size: 6844 bytes --]

On 2021/1/25 22:47, Liuxiangdong (Aven, Cloud Infrastructure Service 
Product Dept.) wrote:
> Thanks for replying,
>
> On 2021/1/25 10:41, Like Xu wrote:
>> + kvm@vger.kernel.org
>>
>> Hi Liuxiangdong,
>>
>> On 2021/1/22 18:02, Liuxiangdong (Aven, Cloud Infrastructure Service 
>> Product Dept.) wrote:
>>> Hi Like,
>>>
>>> Some questions about 
>>> https://lore.kernel.org/kvm/20210104131542.495413-1-like.xu@linux.intel.com/ 
>>> <https://lore.kernel.org/kvm/20210104131542.495413-1-like.xu@linux.intel.com/> 
>>>
>>
>> Thanks for trying the PEBS feature in the guest,
>> and I assume you have correctly applied the QEMU patches for guest PEBS.
>>
> Is there any other patch that needs to be apply? I use qemu 5.2.0. 
> (download from github on January 14th)

Two qemu patches are attached against qemu tree
(commit 31ee895047bdcf7387e3570cbd2a473c6f744b08)
and then run the guest with "-cpu,pebs=true".

Note, this two patch are just for test and not finalized for qemu upstream.

>
>>> 1)Test in IceLake
>>
>> In the [PATCH v3 10/17] KVM: x86/pmu: Expose CPUIDs feature bits PDCM, 
>> DS, DTES64, we only support Ice Lake with the following x86_model(s):
>>
>> #define INTEL_FAM6_ICELAKE_X        0x6A
>> #define INTEL_FAM6_ICELAKE_D        0x6C
>>
>> you can check the eax output of "cpuid -l 1 -1 -r",
>> for example "0x000606a4" meets this requirement.
> It's INTEL_FAM6_ICELAKE_X

Yes, it's the target hardware.

> cpuid -l 1 -1 -r
>
> CPU:
>    0x00000001 0x00: eax=0x000606a6 ebx=0xb4800800 ecx=0x7ffefbf7 
> edx=0xbfebfbff
>
>>>
>>> HOST:
>>>
>>> CPU family:                      6
>>>
>>> Model:                           106
>>>
>>> Model name:                      Intel(R) Xeon(R) Platinum 8378A CPU $@ $@
>>>
>>> microcode: sig=0x606a6, pf=0x1, revision=0xd000122
>>
>> As long as you get the latest BIOS from the provider,
>> you may check 'cat /proc/cpuinfo | grep code | uniq' with the latest one.
> OK. I'll do it later.
>>
>>>
>>> Guest:  linux kernel 5.11.0-rc2
>>
>> I assume it's the "upstream tag v5.11-rc2" which is fine.
> Yes.
>>
>>>
>>> We can find pebs/intel_pt flag in guest cpuinfo, but there still exists 
>>> error when we use perf
>>
>> Just a note, intel_pt and pebs are two features and we can write
>> pebs records to intel_pt buffer with extra hardware support.
>> (by default, pebs records are written to the pebs buffer)
>>
>> You may check the output of "dmesg | grep PEBS" in the guest
>> to see if the guest PEBS cpuinfo is exposed and use "perf record
>> –e cycles:pp" to see if PEBS feature actually  works in the guest.
>
> I apply only pebs patch set to linux kernel 5.11.0-rc2, test perf in 
> guest and dump stack when return -EOPNOTSUPP

Yes, you may apply the qemu patches and try it again.

>
> (1)
> # perf record -e instructions:pp
> Error:
> instructions:pp: PMU Hardware doesn't support 
> sampling/overflow-interrupts. Try 'perf stat'
>
> [  117.793266] Call Trace:
> [  117.793270]  dump_stack+0x57/0x6a
> [  117.793275]  intel_pmu_setup_lbr_filter+0x137/0x190
> [  117.793280]  intel_pmu_hw_config+0x18b/0x320
> [  117.793288]  hsw_hw_config+0xe/0xa0
> [  117.793290]  x86_pmu_event_init+0x8e/0x210
> [  117.793293]  perf_try_init_event+0x40/0x130
> [  117.793297]  perf_event_alloc.part.22+0x611/0xde0
> [  117.793299]  ? alloc_fd+0xba/0x180
> [  117.793302]  __do_sys_perf_event_open+0x1bd/0xd90
> [  117.793305]  do_syscall_64+0x33/0x40
> [  117.793308]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
>
> Do we need lbr when we use pebs?

No, lbr ane pebs are two features and we enable it separately.

>
> I tried to apply lbr patch 
> set(https://lore.kernel.org/kvm/911adb63-ba05-ea93-c038-1c09cff15eda@intel.com/) 
> to kernel and qemu, but there is still other problem.
> Error:
> The sys_perf_event_open() syscall returned with 22 (Invalid argument) for 
> event
> ...

We don't need that patch for PEBS feature.

>
> (2)
> # perf record -e instructions:ppp
> Error:
> instructions:ppp: PMU Hardware doesn't support 
> sampling/overflow-interrupts. Try 'perf stat'
>
> [  115.188498] Call Trace:
> [  115.188503]  dump_stack+0x57/0x6a
> [  115.188509]  x86_pmu_hw_config+0x1eb/0x220
> [  115.188515]  intel_pmu_hw_config+0x13/0x320
> [  115.188519]  hsw_hw_config+0xe/0xa0
> [  115.188521]  x86_pmu_event_init+0x8e/0x210
> [  115.188524]  perf_try_init_event+0x40/0x130
> [  115.188528]  perf_event_alloc.part.22+0x611/0xde0
> [  115.188530]  ? alloc_fd+0xba/0x180
> [  115.188534]  __do_sys_perf_event_open+0x1bd/0xd90
> [  115.188538]  do_syscall_64+0x33/0x40
> [  115.188541]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
>
> This is beacuse x86_pmu.intel_cap.pebs_format is always 0 in 
> x86_pmu_max_precise().
>
> We rdmsr MSR_IA32_PERF_CAPABILITIES(0x00000345)  from HOST, it's f4c5.
> From guest, it's 2000
>
>>>
>>> # perf record –e cycles:pp
>>>
>>> Error:
>>>
>>> cycles:pp: PMU Hardware doesn’t support sampling/overflow-interrupts. 
>>> Try ‘perf stat’
>>>
>>> Could you give some advice?
>>
>> If you have more specific comments or any concerns, just let me know.
>>
>>>
>>> 2)Test in Skylake
>>>
>>> HOST:
>>>
>>> CPU family:                      6
>>>
>>> Model:                           85
>>>
>>> Model name:                      Intel(R) Xeon(R) Gold 6146 CPU @
>>>
>>>                                    3.20GHz
>>>
>>> microcode        : 0x2000064
>>>
>>> Guest: linux 4.18
>>>
>>> we cannot find intel_pt flag in guest cpuinfo because 
>>> cpu_has_vmx_intel_pt() return false.
>>
>> You may check vmx_pebs_supported().
> It's true.
>>
>>>
>>> SECONDARY_EXEC_PT_USE_GPA/VM_EXIT_CLEAR_IA32_RTIT_CTL/VM_ENTRY_LOAD_IA32_RTIT_CTL 
>>> are both disable.
>>>
>>> Is it because microcode is not supported?
>>>
>>> And, isthere a new macrocode which can support these bits? How can we 
>>> get this?
>>
>> Currently, this patch set doesn't support guest PEBS on the Skylake
>> platforms, and if we choose to support it, we will let you know.
>>
> And now, we want to use pebs in skylake. If we develop based on pebs 
> patch set, do you have any suggestions?

- At least you need to pin guest memory such as "-overcommit mem-lock=true" 
for qemu
- You may rewrite the patches 13 - 17 for Skylake specific because the 
records format is different with Ice Lake.

> I think microcode requirements need to be satisfied.  Can we use 
> https://github.com/intel/Intel-Linux-Processor-Microcode-Data-Files ?

You may try it at your risk and again,
this patch set doesn't support guest PEBS on the Skylake platforms currently.

>
>> ---
>> thx,likexu
>>
>>>
>>> Thanks,
>>>
>>> Liuxiangdong
>>>
>>
> Thanks. Liuxiangdong
>


[-- Attachment #2: 0001-target-i386-Expose-PEBS-capabilities-in-the-FEAT_PER.patch --]
[-- Type: text/plain, Size: 1742 bytes --]

From 24a04b800d24e3b493e5094f88649402923147a2 Mon Sep 17 00:00:00 2001
From: Like Xu <like.xu@linux.intel.com>
Date: Fri, 4 Sep 2020 10:19:27 +0800
Subject: [PATCH 1/2] target/i386: Expose PEBS capabilities in the
 FEAT_PERF_CAPABILITIES

The IA32_PERF_CAPABILITIES MSR provides enumeration of a variety of
PEBS feature interfaces:

- PEBSTrap[6]: Trap/Fault-like indicator of PEBS recording assist;
- PEBSArchRegs[7]: Indicator of PEBS assist save architectural registers;
- PEBS_FMT[bits 11:8]: Specifies the encoding of the layout of PEBS records;
- PEBS_BASELINE [bit 14]: If set, the following is true:
    (1) Extended PEBS is supported. All counters support the PEBS facility,
    and all events can generate PEBS records when PEBS is enabled.
    (2) Adaptive PEBS is supported. The PEBS_DATA_CFG MSR and adaptive record
    enable bits are supported.

Signed-off-by: Like Xu <like.xu@linux.intel.com>
---
 target/i386/cpu.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index 72a79e6019..14262c7bf7 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -1136,9 +1136,9 @@ static FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
         .type = MSR_FEATURE_WORD,
         .feat_names = {
             NULL, NULL, NULL, NULL,
-            NULL, NULL, NULL, NULL,
-            NULL, NULL, NULL, NULL,
-            NULL, "full-width-write", NULL, NULL,
+            NULL, NULL, "pebs-trap", "pebs-arch-reg",
+            "pebs-fmt-0", "pebs-fmt-1", "pebs-fmt-2", "pebs-fmt-3",
+            NULL, "full-width-write", "pebs-baseline", NULL,
             NULL, NULL, NULL, NULL,
             NULL, NULL, NULL, NULL,
             NULL, NULL, NULL, NULL,
-- 
2.29.2


[-- Attachment #3: 0002-target-i386-add-cpu-pebs-true-support-to-enable-gues.patch --]
[-- Type: text/plain, Size: 5489 bytes --]

From be5246694aaf2132396ee0b907e679f5c9ccd089 Mon Sep 17 00:00:00 2001
From: Like Xu <like.xu@linux.intel.com>
Date: Fri, 4 Sep 2020 10:42:28 +0800
Subject: [PATCH 2/2] target/i386: add -cpu,pebs=true support to enable guest
 PEBS

The PEBS feature would be enabled on the guest if:
- the KVM is enabled and the PMU is enabled and,
- the msr-based-feature IA32_PERF_CAPABILITIES is supporterd and,
- the supported returned value for PEBS from this msr is not zero.

The PEBS feature would be disabled on the guest if:
- the msr-based-feature IA32_PERF_CAPABILITIES is unsupporterd OR,
- qemu set the IA32_PERF_CAPABILITIES msr feature without pebs_fmt values OR,
- the requested guest vcpu model doesn't support PDCM.

Signed-off-by: Like Xu <like.xu@linux.intel.com>
---
 hw/i386/pc.c          |  1 +
 target/i386/cpu.c     | 20 ++++++++++++++++++++
 target/i386/cpu.h     |  7 +++++++
 target/i386/kvm/kvm.c | 10 ++++++++++
 4 files changed, 38 insertions(+)

diff --git a/hw/i386/pc.c b/hw/i386/pc.c
index 5458f61d10..8e9c1b7545 100644
--- a/hw/i386/pc.c
+++ b/hw/i386/pc.c
@@ -330,6 +330,7 @@ GlobalProperty pc_compat_1_5[] = {
     { "Nehalem-" TYPE_X86_CPU, "min-level", "2" },
     { "virtio-net-pci", "any_layout", "off" },
     { TYPE_X86_CPU, "pmu", "on" },
+    { TYPE_X86_CPU, "pebs", "on" },
     { "i440FX-pcihost", "short_root_bus", "0" },
     { "q35-pcihost", "short_root_bus", "0" },
 };
diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index 14262c7bf7..9dffc85542 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -4228,6 +4228,12 @@ static bool lmce_supported(void)
     return !!(mce_cap & MCG_LMCE_P);
 }
 
+static inline bool lbr_supported(void)
+{
+    return kvm_enabled() && (kvm_arch_get_supported_msr_feature(kvm_state,
+        MSR_IA32_PERF_CAPABILITIES) & PERF_CAP_PEBS_FORMAT);
+}
+
 #define CPUID_MODEL_ID_SZ 48
 
 /**
@@ -4332,6 +4338,9 @@ static void max_x86_cpu_initfn(Object *obj)
     }
 
     object_property_set_bool(OBJECT(cpu), "pmu", true, &error_abort);
+    if (lbr_supported()) {
+        object_property_set_bool(OBJECT(cpu), "pebs", true, &error_abort);
+    }
 }
 
 static const TypeInfo max_x86_cpu_type_info = {
@@ -5545,6 +5554,10 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,
         }
         if (!cpu->enable_pmu) {
             *ecx &= ~CPUID_EXT_PDCM;
+            if (cpu->enable_pebs) {
+                warn_report("PEBS is unsupported since guest PMU is disabled.");
+                exit(1);
+            }
         }
         break;
     case 2:
@@ -6610,6 +6623,12 @@ static void x86_cpu_realizefn(DeviceState *dev, Error **errp)
         }
     }
 
+    if (!cpu->max_features && cpu->enable_pebs &&
+        !(env->features[FEAT_1_ECX] & CPUID_EXT_PDCM)) {
+        warn_report("requested vcpu model doesn't support PDCM for PEBS.");
+        exit(1);
+    }
+
     if (cpu->ucode_rev == 0) {
         /* The default is the same as KVM's.  */
         if (IS_AMD_CPU(env)) {
@@ -7192,6 +7211,7 @@ static Property x86_cpu_properties[] = {
 #endif
     DEFINE_PROP_INT32("node-id", X86CPU, node_id, CPU_UNSET_NUMA_NODE_ID),
     DEFINE_PROP_BOOL("pmu", X86CPU, enable_pmu, false),
+    DEFINE_PROP_BOOL("pebs", X86CPU, enable_pebs, false),
 
     DEFINE_PROP_UINT32("hv-spinlocks", X86CPU, hyperv_spinlock_attempts,
                        HYPERV_SPINLOCK_NEVER_NOTIFY),
diff --git a/target/i386/cpu.h b/target/i386/cpu.h
index d23a5b340a..eac8d8c68e 100644
--- a/target/i386/cpu.h
+++ b/target/i386/cpu.h
@@ -354,6 +354,12 @@ typedef enum X86Seg {
 #define ARCH_CAP_TSX_CTRL_MSR		(1<<7)
 
 #define MSR_IA32_PERF_CAPABILITIES      0x345
+#define PERF_CAP_PEBS_TRAP             BIT_ULL(6)
+#define PERF_CAP_ARCH_REG              BIT_ULL(7)
+#define PERF_CAP_PEBS_FORMAT           0xf00
+#define PERF_CAP_PEBS_BASELINE         BIT_ULL(14)
+#define PERF_CAP_PEBS_MASK	(PERF_CAP_PEBS_TRAP | PERF_CAP_ARCH_REG | \
+	PERF_CAP_PEBS_FORMAT | PERF_CAP_PEBS_BASELINE)
 
 #define MSR_IA32_TSX_CTRL		0x122
 #define MSR_IA32_TSCDEADLINE            0x6e0
@@ -1708,6 +1714,7 @@ struct X86CPU {
      * capabilities) directly to the guest.
      */
     bool enable_pmu;
+    bool enable_pebs;
 
     /* LMCE support can be enabled/disabled via cpu option 'lmce=on/off'. It is
      * disabled by default to avoid breaking migration between QEMU with
diff --git a/target/i386/kvm/kvm.c b/target/i386/kvm/kvm.c
index 6dc1ee052d..8fe1d2feea 100644
--- a/target/i386/kvm/kvm.c
+++ b/target/i386/kvm/kvm.c
@@ -2705,6 +2705,13 @@ static void kvm_msr_entry_add_perf(X86CPU *cpu, FeatureWordArray f)
                                            MSR_IA32_PERF_CAPABILITIES);
 
     if (kvm_perf_cap) {
+        if (!cpu->enable_pebs) {
+            kvm_perf_cap &= ~PERF_CAP_PEBS_MASK;
+        }
+        if (!(kvm_perf_cap & PERF_CAP_PEBS_MASK) && cpu->enable_pebs) {
+            warn_report("MSR_IA32_PERF_CAPABILITIES reported by KVM does not support PEBS.");
+            exit(1);
+        }
         kvm_msr_entry_add(cpu, MSR_IA32_PERF_CAPABILITIES,
                         kvm_perf_cap & f[FEAT_PERF_CAPABILITIES]);
     }
@@ -2744,6 +2751,9 @@ static void kvm_init_msrs(X86CPU *cpu)
 
     if (has_msr_perf_capabs && cpu->enable_pmu) {
         kvm_msr_entry_add_perf(cpu, env->features);
+    } else if (!has_msr_perf_capabs && cpu->enable_pebs) {
+        warn_report("KVM doesn't support MSR_IA32_PERF_CAPABILITIES for PEBS.");
+        exit(1);
     }
 
     if (has_msr_ucode_rev) {
-- 
2.29.2


  reply	other threads:[~2021-01-26 17:09 UTC|newest]

Thread overview: 18+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
     [not found] <EEC2A80E7137D84ABF791B01D40FA9A601EC200E@DGGEMM506-MBX.china.huawei.com>
2021-01-25  2:41 ` Like Xu
2021-01-25 14:47   ` Liuxiangdong (Aven, Cloud Infrastructure Service Product Dept.)
2021-01-26  7:08     ` Xu, Like [this message]
2021-01-29  2:52       ` Liuxiangdong (Aven, Cloud Infrastructure Service Product Dept.)
2021-02-01  8:43         ` Xu, Like
2021-01-04 13:15 Like Xu
2021-01-14 19:10 ` Sean Christopherson
2021-01-15  2:02   ` Xu, Like
2021-01-15 17:57     ` Sean Christopherson
2021-01-15 18:27       ` Andi Kleen
2021-01-15 18:51         ` Sean Christopherson
2021-01-15 19:11           ` Andi Kleen
2021-01-22  9:56           ` Peter Zijlstra
2021-01-25  8:08             ` Like Xu
2021-01-25 11:13               ` Peter Zijlstra
2021-01-25 12:07                 ` Xu, Like
2021-01-25 12:18                   ` Peter Zijlstra
2021-01-25 12:53                     ` Xu, Like

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=f4dcb068-2ddf-428f-50ad-39f65cad3710@intel.com \
    --to=like.xu@intel.com \
    --cc=eric.fangyi@huawei.com \
    --cc=kvm@vger.kernel.org \
    --cc=like.xu@linux.intel.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=liuxiangdong5@huawei.com \
    --cc=wei.w.wang@intel.com \
    --cc=xiexiangyou@huawei.com \
    --subject='Re: [PATCH v3 00/17] KVM: x86/pmu: Add support to enable Guest PEBS via DS' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
on how to clone and mirror all data and code used for this inbox