From: Liuxiangdong <liuxiangdong5@huawei.com>
To: "Xu, Like" <like.xu@intel.com>
Cc: Borislav Petkov <bp@alien8.de>,
Sean Christopherson <seanjc@google.com>,
Vitaly Kuznetsov <vkuznets@redhat.com>,
Wanpeng Li <wanpengli@tencent.com>,
Jim Mattson <jmattson@google.com>, Joerg Roedel <joro@8bytes.org>,
<weijiang.yang@intel.com>, Kan Liang <kan.liang@linux.intel.com>,
<ak@linux.intel.com>, <wei.w.wang@intel.com>,
<eranian@google.com>, <linux-kernel@vger.kernel.org>,
<x86@kernel.org>, <kvm@vger.kernel.org>,
"Fangyi (Eric)" <eric.fangyi@huawei.com>,
Xiexiangyou <xiexiangyou@huawei.com>,
Peter Zijlstra <peterz@infradead.org>,
"Paolo Bonzini" <pbonzini@redhat.com>,
Like Xu <like.xu@linux.intel.com>
Subject: Re: [PATCH v6 00/16] KVM: x86/pmu: Add *basic* support to enable guest PEBS via DS
Date: Tue, 18 May 2021 21:15:26 +0800 [thread overview]
Message-ID: <60A3BDEE.5020203@huawei.com> (raw)
In-Reply-To: <a65c8556-4eac-b8db-8aa4-98229f47fc8d@intel.com>
On 2021/5/18 20:40, Xu, Like wrote:
> On 2021/5/18 20:23, Liuxiangdong wrote:
>>
>>
>> On 2021/5/17 14:38, Like Xu wrote:
>>> Hi xiangdong,
>>>
>>> On 2021/5/15 18:30, Liuxiangdong wrote:
>>>>
>>>>
>>>> On 2021/5/11 10:41, Like Xu wrote:
>>>>> A new kernel cycle has begun, and this version looks promising.
>>>>>
>>>>> The guest Precise Event Based Sampling (PEBS) feature can provide
>>>>> an architectural state of the instruction executed after the guest
>>>>> instruction that exactly caused the event. It needs new hardware
>>>>> facility only available on Intel Ice Lake Server platforms. This
>>>>> patch set enables the basic PEBS feature for KVM guests on ICX.
>>>>>
>>>>> We can use PEBS feature on the Linux guest like native:
>>>>>
>>>>> # perf record -e instructions:ppp ./br_instr a
>>>>> # perf record -c 100000 -e instructions:pp ./br_instr a
>>>>
>>>> Hi, Like.
>>>> Has the qemu patch been modified?
>>>>
>>>> https://lore.kernel.org/kvm/f4dcb068-2ddf-428f-50ad-39f65cad3710@intel.com/
>>>> ?
>>>
>>> I think the qemu part still works based on
>>> 609d7596524ab204ccd71ef42c9eee4c7c338ea4 (tag: v6.0.0).
>>>
>>
>> Yes. I applied these two qemu patches to qemu v6.0.0 and this kvm
>> patches set to latest kvm tree.
>>
>> I can see pebs flags in Guest(linux 5.11) on the IceLake( Model: 106
>> Model name: Intel(R) Xeon(R) Platinum 8378A CPU),
>> and i can use PEBS like this.
>>
>> #perf record -e instructions:pp
>>
>> It can work normally.
>>
>> But there is no sampling when i use "perf record -e events:pp" or
>> just "perf record" in guest
>> unless i delete patch 09 and patch 13 from this kvm patches set.
>>
>>
>
> With patch 9 and 13, does the basic counter sampling still work ?
> You may retry w/ "echo 0 > /proc/sys/kernel/watchdog" on the host and
> guest.
>
Yes. It works! Thanks!
>> Have you tried "perf record -e events:pp" in this patches set? Does
>> it work normally?
>
> All my PEBS testcases passed. You may dump guest msr traces from your
> testcase with me.
>
>>
>>
>>
>> Thanks!
>> Xiangdong Liu
>>
>>
>>
>>> When the LBR qemu patch receives the ACK from the maintainer,
>>> I will submit PBES qemu support because their changes are very similar.
>>>
>>> Please help review this version and
>>> feel free to add your comments or "Reviewed-by".
>>>
>>> Thanks,
>>> Like Xu
>>>
>>>>
>>>>
>>>>> To emulate guest PEBS facility for the above perf usages,
>>>>> we need to implement 2 code paths:
>>>>>
>>>>> 1) Fast path
>>>>>
>>>>> This is when the host assigned physical PMC has an identical index as
>>>>> the virtual PMC (e.g. using physical PMC0 to emulate virtual PMC0).
>>>>> This path is used in most common use cases.
>>>>>
>>>>> 2) Slow path
>>>>>
>>>>> This is when the host assigned physical PMC has a different index
>>>>> from the virtual PMC (e.g. using physical PMC1 to emulate virtual
>>>>> PMC0)
>>>>> In this case, KVM needs to rewrite the PEBS records to change the
>>>>> applicable counter indexes to the virtual PMC indexes, which would
>>>>> otherwise contain the physical counter index written by PEBS
>>>>> facility,
>>>>> and switch the counter reset values to the offset corresponding to
>>>>> the physical counter indexes in the DS data structure.
>>>>>
>>>>> The previous version [0] enables both fast path and slow path, which
>>>>> seems a bit more complex as the first step. In this patchset, we want
>>>>> to start with the fast path to get the basic guest PEBS enabled while
>>>>> keeping the slow path disabled. More focused discussion on the slow
>>>>> path [1] is planned to be put to another patchset in the next step.
>>>>>
>>>>> Compared to later versions in subsequent steps, the functionality
>>>>> to support host-guest PEBS both enabled and the functionality to
>>>>> emulate guest PEBS when the counter is cross-mapped are missing
>>>>> in this patch set (neither of these are typical scenarios).
>>>>>
>>>>> With the basic support, the guest can retrieve the correct PEBS
>>>>> information from its own PEBS records on the Ice Lake servers.
>>>>> And we expect it should work when migrating to another Ice Lake
>>>>> and no regression about host perf is expected.
>>>>>
>>>>> Here are the results of pebs test from guest/host for same workload:
>>>>>
>>>>> perf report on guest:
>>>>> # Samples: 2K of event 'instructions:ppp', # Event count
>>>>> (approx.): 1473377250
>>>>> # Overhead Command Shared Object Symbol
>>>>> 57.74% br_instr br_instr [.] lfsr_cond
>>>>> 41.40% br_instr br_instr [.] cmp_end
>>>>> 0.21% br_instr [kernel.kallsyms] [k] __lock_acquire
>>>>>
>>>>> perf report on host:
>>>>> # Samples: 2K of event 'instructions:ppp', # Event count
>>>>> (approx.): 1462721386
>>>>> # Overhead Command Shared Object Symbol
>>>>> 57.90% br_instr br_instr [.] lfsr_cond
>>>>> 41.95% br_instr br_instr [.] cmp_end
>>>>> 0.05% br_instr [kernel.vmlinux] [k] lock_acquire
>>>>> Conclusion: the profiling results on the guest are similar
>>>>> tothat on the host.
>>>>>
>>>>> A minimum guest kernel version may be v5.4 or a backport version
>>>>> support Icelake server PEBS.
>>>>>
>>>>> Please check more details in each commit and feel free to comment.
>>>>>
>>>>> Previous:
>>>>> https://lore.kernel.org/kvm/20210415032016.166201-1-like.xu@linux.intel.com/
>>>>>
>>>>>
>>>>> [0]
>>>>> https://lore.kernel.org/kvm/20210104131542.495413-1-like.xu@linux.intel.com/
>>>>> [1]
>>>>> https://lore.kernel.org/kvm/20210115191113.nktlnmivc3edstiv@two.firstfloor.org/
>>>>>
>>>>>
>>>>> V5 -> V6 Changelog:
>>>>> - Rebased on the latest kvm/queue tree;
>>>>> - Fix a git rebase issue (Liuxiangdong);
>>>>> - Adjust the patch sequence 06/07 for bisection (Liuxiangdong);
>>>>>
>>>>> Like Xu (16):
>>>>> perf/x86/intel: Add EPT-Friendly PEBS for Ice Lake Server
>>>>> perf/x86/intel: Handle guest PEBS overflow PMI for KVM guest
>>>>> perf/x86/core: Pass "struct kvm_pmu *" to determine the guest
>>>>> values
>>>>> KVM: x86/pmu: Set MSR_IA32_MISC_ENABLE_EMON bit when vPMU is
>>>>> enabled
>>>>> KVM: x86/pmu: Introduce the ctrl_mask value for fixed counter
>>>>> KVM: x86/pmu: Add IA32_PEBS_ENABLE MSR emulation for extended PEBS
>>>>> KVM: x86/pmu: Reprogram PEBS event to emulate guest PEBS counter
>>>>> KVM: x86/pmu: Add IA32_DS_AREA MSR emulation to support guest DS
>>>>> KVM: x86/pmu: Add PEBS_DATA_CFG MSR emulation to support
>>>>> adaptive PEBS
>>>>> KVM: x86: Set PEBS_UNAVAIL in IA32_MISC_ENABLE when PEBS is
>>>>> enabled
>>>>> KVM: x86/pmu: Adjust precise_ip to emulate Ice Lake guest PDIR
>>>>> counter
>>>>> KVM: x86/pmu: Move pmc_speculative_in_use() to arch/x86/kvm/pmu.h
>>>>> KVM: x86/pmu: Disable guest PEBS temporarily in two rare
>>>>> situations
>>>>> KVM: x86/pmu: Add kvm_pmu_cap to optimize
>>>>> perf_get_x86_pmu_capability
>>>>> KVM: x86/cpuid: Refactor host/guest CPU model consistency check
>>>>> KVM: x86/pmu: Expose CPUIDs feature bits PDCM, DS, DTES64
>>>>>
>>>>> arch/x86/events/core.c | 5 +-
>>>>> arch/x86/events/intel/core.c | 129
>>>>> ++++++++++++++++++++++++------
>>>>> arch/x86/events/perf_event.h | 5 +-
>>>>> arch/x86/include/asm/kvm_host.h | 16 ++++
>>>>> arch/x86/include/asm/msr-index.h | 6 ++
>>>>> arch/x86/include/asm/perf_event.h | 5 +-
>>>>> arch/x86/kvm/cpuid.c | 24 ++----
>>>>> arch/x86/kvm/cpuid.h | 5 ++
>>>>> arch/x86/kvm/pmu.c | 50 +++++++++---
>>>>> arch/x86/kvm/pmu.h | 38 +++++++++
>>>>> arch/x86/kvm/vmx/capabilities.h | 26 ++++--
>>>>> arch/x86/kvm/vmx/pmu_intel.c | 115 +++++++++++++++++++++-----
>>>>> arch/x86/kvm/vmx/vmx.c | 24 +++++-
>>>>> arch/x86/kvm/vmx/vmx.h | 2 +-
>>>>> arch/x86/kvm/x86.c | 14 ++--
>>>>> 15 files changed, 368 insertions(+), 96 deletions(-)
>>>>>
>>>
>>
>
next prev parent reply other threads:[~2021-05-18 13:15 UTC|newest]
Thread overview: 56+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-05-11 2:41 [PATCH v6 00/16] KVM: x86/pmu: Add *basic* support to enable guest PEBS via DS Like Xu
2021-05-11 2:41 ` [PATCH v6 01/16] perf/x86/intel: Add EPT-Friendly PEBS for Ice Lake Server Like Xu
2021-05-11 2:42 ` [PATCH v6 02/16] perf/x86/intel: Handle guest PEBS overflow PMI for KVM guest Like Xu
2021-05-17 8:16 ` Peter Zijlstra
2021-05-18 7:38 ` Xu, Like
2021-05-18 8:37 ` Peter Zijlstra
2021-05-11 2:42 ` [PATCH v6 03/16] perf/x86/core: Pass "struct kvm_pmu *" to determine the guest values Like Xu
2021-05-11 2:42 ` [PATCH v6 04/16] KVM: x86/pmu: Set MSR_IA32_MISC_ENABLE_EMON bit when vPMU is enabled Like Xu
2021-05-12 1:58 ` Venkatesh Srinivas
2021-05-12 5:00 ` Xu, Like
2021-05-12 15:18 ` Sean Christopherson
2021-05-13 2:50 ` Xu, Like
2021-05-17 18:43 ` Venkatesh Srinivas
2021-05-17 21:19 ` Sean Christopherson
2021-05-17 21:16 ` Sean Christopherson
2021-05-17 23:51 ` Sean Christopherson
2021-05-18 7:49 ` Xu, Like
2021-05-11 2:42 ` [PATCH v6 05/16] KVM: x86/pmu: Introduce the ctrl_mask value for fixed counter Like Xu
2021-05-17 8:18 ` Peter Zijlstra
2021-05-18 7:55 ` Xu, Like
2021-05-18 8:35 ` Peter Zijlstra
2021-05-11 2:42 ` [PATCH v6 06/16] KVM: x86/pmu: Add IA32_PEBS_ENABLE MSR emulation for extended PEBS Like Xu
2021-05-17 8:32 ` Peter Zijlstra
2021-05-18 8:44 ` Xu, Like
2021-05-18 13:42 ` Peter Zijlstra
2021-05-17 8:33 ` Peter Zijlstra
2021-05-18 8:13 ` Xu, Like
2021-05-11 2:42 ` [PATCH v6 07/16] KVM: x86/pmu: Reprogram PEBS event to emulate guest PEBS counter Like Xu
2021-05-17 8:39 ` Peter Zijlstra
2021-05-17 14:44 ` Andi Kleen
2021-05-18 8:47 ` Peter Zijlstra
2021-05-18 13:15 ` Xu, Like
2021-05-18 15:58 ` Andi Kleen
2021-05-17 9:14 ` Peter Zijlstra
2021-05-18 13:28 ` Xu, Like
2021-05-18 13:36 ` Peter Zijlstra
2021-05-18 14:05 ` Xu, Like
2021-05-11 2:42 ` [PATCH v6 08/16] KVM: x86/pmu: Add IA32_DS_AREA MSR emulation to support guest DS Like Xu
2021-05-12 5:16 ` Xu, Like
2021-05-17 13:26 ` Peter Zijlstra
2021-05-17 14:50 ` Andi Kleen
2021-05-11 2:42 ` [PATCH v6 09/16] KVM: x86/pmu: Add PEBS_DATA_CFG MSR emulation to support adaptive PEBS Like Xu
2021-05-11 2:42 ` [PATCH v6 10/16] KVM: x86: Set PEBS_UNAVAIL in IA32_MISC_ENABLE when PEBS is enabled Like Xu
2021-05-11 2:42 ` [PATCH v6 11/16] KVM: x86/pmu: Adjust precise_ip to emulate Ice Lake guest PDIR counter Like Xu
2021-05-11 2:42 ` [PATCH v6 12/16] KVM: x86/pmu: Move pmc_speculative_in_use() to arch/x86/kvm/pmu.h Like Xu
2021-05-11 2:42 ` [PATCH v6 13/16] KVM: x86/pmu: Disable guest PEBS temporarily in two rare situations Like Xu
2021-05-11 2:42 ` [PATCH v6 14/16] KVM: x86/pmu: Add kvm_pmu_cap to optimize perf_get_x86_pmu_capability Like Xu
2021-05-11 2:42 ` [PATCH v6 15/16] KVM: x86/cpuid: Refactor host/guest CPU model consistency check Like Xu
2021-05-11 2:42 ` [PATCH v6 16/16] KVM: x86/pmu: Expose CPUIDs feature bits PDCM, DS, DTES64 Like Xu
2021-05-15 10:30 ` [PATCH v6 00/16] KVM: x86/pmu: Add *basic* support to enable guest PEBS via DS Liuxiangdong
2021-05-17 6:38 ` Like Xu
2021-05-18 12:23 ` Liuxiangdong
2021-05-18 12:40 ` Xu, Like
2021-05-18 13:15 ` Liuxiangdong [this message]
2021-05-19 1:44 ` Liuxiangdong
2021-05-21 1:37 ` Like Xu
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=60A3BDEE.5020203@huawei.com \
--to=liuxiangdong5@huawei.com \
--cc=ak@linux.intel.com \
--cc=bp@alien8.de \
--cc=eranian@google.com \
--cc=eric.fangyi@huawei.com \
--cc=jmattson@google.com \
--cc=joro@8bytes.org \
--cc=kan.liang@linux.intel.com \
--cc=kvm@vger.kernel.org \
--cc=like.xu@intel.com \
--cc=like.xu@linux.intel.com \
--cc=linux-kernel@vger.kernel.org \
--cc=pbonzini@redhat.com \
--cc=peterz@infradead.org \
--cc=seanjc@google.com \
--cc=vkuznets@redhat.com \
--cc=wanpengli@tencent.com \
--cc=wei.w.wang@intel.com \
--cc=weijiang.yang@intel.com \
--cc=x86@kernel.org \
--cc=xiexiangyou@huawei.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).