kvm.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: "Xu, Like" <like.xu@intel.com>
To: Sean Christopherson <seanjc@google.com>,
	Like Xu <like.xu@linux.intel.com>,
	Peter Zijlstra <peterz@infradead.org>,
	Kan Liang <kan.liang@linux.intel.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>,
	eranian@google.com, kvm@vger.kernel.org,
	Ingo Molnar <mingo@redhat.com>,
	Thomas Gleixner <tglx@linutronix.de>,
	Vitaly Kuznetsov <vkuznets@redhat.com>,
	Wanpeng Li <wanpengli@tencent.com>,
	Jim Mattson <jmattson@google.com>, Joerg Roedel <joro@8bytes.org>,
	Andi Kleen <andi@firstfloor.org>,
	wei.w.wang@intel.com, luwei.kang@intel.com,
	linux-kernel@vger.kernel.org
Subject: Re: [PATCH v3 04/17] perf: x86/ds: Handle guest PEBS overflow PMI and inject it to guest
Date: Fri, 15 Jan 2021 10:49:49 +0800	[thread overview]
Message-ID: <a0b5dc29-e63f-6ec9-a03f-6435cb3373c6@intel.com> (raw)
In-Reply-To: <YACTnkdi1rxfrRCg@google.com>

On 2021/1/15 2:55, Sean Christopherson wrote:
> On Mon, Jan 04, 2021, Like Xu wrote:
>> ---
>>   arch/x86/events/intel/ds.c | 62 ++++++++++++++++++++++++++++++++++++++
>>   1 file changed, 62 insertions(+)
>>
>> diff --git a/arch/x86/events/intel/ds.c b/arch/x86/events/intel/ds.c
>> index b47cc4226934..c499bdb58373 100644
>> --- a/arch/x86/events/intel/ds.c
>> +++ b/arch/x86/events/intel/ds.c
>> @@ -1721,6 +1721,65 @@ intel_pmu_save_and_restart_reload(struct perf_event *event, int count)
>>   	return 0;
>>   }
>>   
>> +/*
>> + * We may be running with guest PEBS events created by KVM, and the
>> + * PEBS records are logged into the guest's DS and invisible to host.
>> + *
>> + * In the case of guest PEBS overflow, we only trigger a fake event
>> + * to emulate the PEBS overflow PMI for guest PBES counters in KVM.
>> + * The guest will then vm-entry and check the guest DS area to read
>> + * the guest PEBS records.
>> + *
>> + * The guest PEBS overflow PMI may be dropped when both the guest and
>> + * the host use PEBS. Therefore, KVM will not enable guest PEBS once
>> + * the host PEBS is enabled since it may bring a confused unknown NMI.
>> + *
>> + * The contents and other behavior of the guest event do not matter.
>> + */
>> +static int intel_pmu_handle_guest_pebs(struct cpu_hw_events *cpuc,
>> +				       struct pt_regs *iregs,
>> +				       struct debug_store *ds)
>> +{
>> +	struct perf_sample_data data;
>> +	struct perf_event *event = NULL;
>> +	u64 guest_pebs_idxs = cpuc->pebs_enabled & ~cpuc->intel_ctrl_host_mask;
>> +	int bit;
>> +
>> +	/*
>> +	 * Ideally, we should check guest DS to understand if it's
>> +	 * a guest PEBS overflow PMI from guest PEBS counters.
>> +	 * However, it brings high overhead to retrieve guest DS in host.
>> +	 * So we check host DS instead for performance.
>> +	 *
>> +	 * If PEBS interrupt threshold on host is not exceeded in a NMI, there
>> +	 * must be a PEBS overflow PMI generated from the guest PEBS counters.
>> +	 * There is no ambiguity since the reported event in the PMI is guest
>> +	 * only. It gets handled correctly on a case by case base for each event.
>> +	 *
>> +	 * Note: KVM disables the co-existence of guest PEBS and host PEBS.
> By "KVM", do you mean KVM's loading of the MSRs provided by intel_guest_get_msrs()?
> Because the PMU should really be the entity that controls guest vs. host.  KVM
> should just be a dumb pipe that handles the mechanics of how values are context
> switch.

The intel_guest_get_msrs() and atomic_switch_perf_msrs()
will work together to disable the co-existence of guest PEBS and host PEBS:

https://lore.kernel.org/kvm/961e6135-ff6d-86d1-3b7b-a1846ad0e4c4@intel.com/

+

static void atomic_switch_perf_msrs(struct vcpu_vmx *vmx)
...
     if (nr_msrs > 2 && (msrs[1].guest & msrs[0].guest)) {
         msrs[2].guest = pmu->ds_area;
         if (nr_msrs > 3)
             msrs[3].guest = pmu->pebs_data_cfg;
     }

    for (i = 0; i < nr_msrs; i++)
...

>
> For example, commit 7099e2e1f4d9 ("KVM: VMX: disable PEBS before a guest entry"),
> where KVM does an explicit WRMSR(PEBS_ENABLE) to (attempt to) force PEBS
> quiescence, is flawed in that the PMU can re-enable PEBS after the WRMSR if a
> PMI arrives between the WRMSR and VM-Enter (because VMX can't block NMIs).  The
> PMU really needs to be involved in the WRMSR workaround.

Thanks, I will carefully confirm the PEBS quiescent behavior on the ICX.
But we're fine to keep "wrmsrl(MSR_IA32_PEBS_ENABLE, 0);" here
since we will load a new guest value (if any) for this msr later.

>
>> +	 */
>> +	if (!guest_pebs_idxs || !in_nmi() ||
> Are PEBS updates guaranteed to be isolated in both directions on relevant
> hardware?

I think it's true on the ICX.

> By that I mean, will host updates be fully processed before VM-Enter
> compeletes, and guest updates before VM-Exit completes?

The situation is more complicated.

> If that's the case,
> then this path could be optimized to change the KVM invocation of the NMI
> handler so that the "is this a guest PEBS PMI" check is done if and only if the
> PMI originated from with the guest.

When we have a PEBS PMI due to guest workload and vm-exits,
the code path from vm-exit to the host PEBS PMI handler may also
bring PEBS PMI and mark the status bit. The current PMI handler
can't distinguish them and would treat the later one as a suspicious
PMI and output a warning.

This is the main reason why we choose to disable the co-existence
of guest PEBS and host PEBS, and future hardware enhancements
may break this limitation.

---
thx, likexu
>
>> +		ds->pebs_index >= ds->pebs_interrupt_threshold)
>> +		return 0;
>> +
>> +	for_each_set_bit(bit, (unsigned long *)&guest_pebs_idxs,
>> +			INTEL_PMC_IDX_FIXED + x86_pmu.num_counters_fixed) {
>> +
>> +		event = cpuc->events[bit];
>> +		if (!event->attr.precise_ip)
>> +			continue;
>> +
>> +		perf_sample_data_init(&data, 0, event->hw.last_period);
>> +		if (perf_event_overflow(event, &data, iregs))
>> +			x86_pmu_stop(event, 0);
>> +
>> +		/* Inject one fake event is enough. */
>> +		return 1;
>> +	}
>> +
>> +	return 0;
>> +}


  reply	other threads:[~2021-01-15  2:50 UTC|newest]

Thread overview: 54+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-01-04 13:15 [PATCH v3 00/17] KVM: x86/pmu: Add support to enable Guest PEBS via DS Like Xu
2021-01-04 13:15 ` [PATCH v3 01/17] KVM: x86/pmu: Set MSR_IA32_MISC_ENABLE_EMON bit when vPMU is enabled Like Xu
2021-01-04 13:15 ` [PATCH v3 02/17] KVM: x86/pmu: Use IA32_PERF_CAPABILITIES to adjust features visibility Like Xu
2021-01-04 13:15 ` [PATCH v3 03/17] KVM: x86/pmu: Introduce the ctrl_mask value for fixed counter Like Xu
2021-01-13 18:06   ` Peter Zijlstra
2021-01-14  1:58     ` Xu, Like
2021-01-04 13:15 ` [PATCH v3 04/17] perf: x86/ds: Handle guest PEBS overflow PMI and inject it to guest Like Xu
2021-01-13 18:22   ` Peter Zijlstra
2021-01-13 18:27     ` Peter Zijlstra
2021-01-14  3:39     ` Xu, Like
2021-01-15 12:01       ` Peter Zijlstra
2021-01-15 14:30         ` Xu, Like
2021-01-15 14:44           ` Peter Zijlstra
2021-01-15 15:12             ` Xu, Like
2021-01-25  8:26             ` Like Xu
2021-01-25 11:47               ` Peter Zijlstra
2021-02-02  6:31                 ` Xu, Like
2021-01-14 18:55   ` Sean Christopherson
2021-01-15  2:49     ` Xu, Like [this message]
2021-01-15 17:42       ` Sean Christopherson
2021-01-22  5:30         ` Like Xu
2021-01-04 13:15 ` [PATCH v3 05/17] KVM: x86/pmu: Reprogram guest PEBS event to emulate guest PEBS counter Like Xu
2021-01-15 11:33   ` Peter Zijlstra
2021-01-15 13:53     ` Xu, Like
2021-01-04 13:15 ` [PATCH v3 06/17] KVM: x86/pmu: Add IA32_PEBS_ENABLE MSR emulation for extended PEBS Like Xu
2021-01-05 21:11   ` Sean Christopherson
2021-01-07 12:38     ` Xu, Like
2021-01-15 14:46   ` Peter Zijlstra
2021-01-15 15:29     ` Xu, Like
2021-01-04 13:15 ` [PATCH v3 07/17] KVM: x86/pmu: Add IA32_DS_AREA MSR emulation to manage guest DS buffer Like Xu
2021-01-05 21:16   ` Sean Christopherson
2021-01-08  3:05     ` Xu, Like
2021-01-04 13:15 ` [PATCH v3 08/17] KVM: x86/pmu: Add PEBS_DATA_CFG MSR emulation to support adaptive PEBS Like Xu
2021-01-04 13:15 ` [PATCH v3 09/17] KVM: x86: Set PEBS_UNAVAIL in IA32_MISC_ENABLE when PEBS is enabled Like Xu
2021-01-04 13:15 ` [PATCH v3 10/17] KVM: x86/pmu: Expose CPUIDs feature bits PDCM, DS, DTES64 Like Xu
2021-01-04 13:15 ` [PATCH v3 11/17] KVM: x86/pmu: Adjust precise_ip to emulate Ice Lake guest PDIR counter Like Xu
2021-01-04 13:15 ` [PATCH v3 12/17] KVM: x86/pmu: Disable guest PEBS when counters are cross-mapped Like Xu
2021-01-04 13:15 ` [PATCH v3 13/17] KVM: x86/pmu: Add hook to emulate pebs for cross-mapped counters Like Xu
2021-01-04 13:15 ` [PATCH v3 14/17] KVM: vmx/pmu: Limit pebs_interrupt_threshold in the guest DS area Like Xu
2021-01-04 13:15 ` [PATCH v3 15/17] KVM: vmx/pmu: Rewrite applicable_counters field in guest PEBS records Like Xu
2021-01-04 13:15 ` [PATCH v3 16/17] KVM: x86/pmu: Save guest pebs reset values when pebs is configured Like Xu
2021-01-04 13:15 ` [PATCH v3 17/17] KVM: x86/pmu: Adjust guest pebs reset values for crpss-mapped counters Like Xu
2021-01-14 19:10 ` [PATCH v3 00/17] KVM: x86/pmu: Add support to enable Guest PEBS via DS Sean Christopherson
2021-01-15  2:02   ` Xu, Like
2021-01-15 17:57     ` Sean Christopherson
2021-01-15 18:27       ` Andi Kleen
2021-01-15 18:51         ` Sean Christopherson
2021-01-15 19:11           ` Andi Kleen
2021-01-22  9:56           ` Peter Zijlstra
2021-01-25  8:08             ` Like Xu
2021-01-25 11:13               ` Peter Zijlstra
2021-01-25 12:07                 ` Xu, Like
2021-01-25 12:18                   ` Peter Zijlstra
2021-01-25 12:53                     ` Xu, Like

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=a0b5dc29-e63f-6ec9-a03f-6435cb3373c6@intel.com \
    --to=like.xu@intel.com \
    --cc=andi@firstfloor.org \
    --cc=eranian@google.com \
    --cc=jmattson@google.com \
    --cc=joro@8bytes.org \
    --cc=kan.liang@linux.intel.com \
    --cc=kvm@vger.kernel.org \
    --cc=like.xu@linux.intel.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=luwei.kang@intel.com \
    --cc=mingo@redhat.com \
    --cc=pbonzini@redhat.com \
    --cc=peterz@infradead.org \
    --cc=seanjc@google.com \
    --cc=tglx@linutronix.de \
    --cc=vkuznets@redhat.com \
    --cc=wanpengli@tencent.com \
    --cc=wei.w.wang@intel.com \
    --subject='Re: [PATCH v3 04/17] perf: x86/ds: Handle guest PEBS overflow PMI and inject it to guest' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
on how to clone and mirror all data and code used for this inbox