Non-exiting rdpmc on KVM guests?

Non-exiting rdpmc on KVM guests?

[Andy Lutomirski]

I just wrote a little perf self-monitoring tool that uses rdpmc to
count cycles.  Performance sucks under KVM (VMX).

How hard would it be to avoid rdpmc exits in cases where the host and
guest pmu configurations are compatible as seen by rdpmc?  I'm mostly
ignorant of how the PMU counter offsets and such work.

(Also, grr, Intel, couldn't you come up with a better interface for this?)

--Andy

Re: Non-exiting rdpmc on KVM guests?

[Peter Zijlstra]

On Mon, Apr 20, 2015 at 05:47:23PM -0700, Andy Lutomirski wrote:
> I just wrote a little perf self-monitoring tool that uses rdpmc to
> count cycles.  Performance sucks under KVM (VMX).
> 
> How hard would it be to avoid rdpmc exits in cases where the host and
> guest pmu configurations are compatible as seen by rdpmc?  I'm mostly
> ignorant of how the PMU counter offsets and such work.

I'm not sure how all the VMX stuff works. how is rdpmc seem? It it at
all possible to not trap that?

But as it stands there is not even a hint from the emulated pmu events
on which counters they 'ought' to run so they landing the same way the
guest thinks they ought to is rather small.

'someone' would have to go look at the event scheduling and see how to
implement it, my first suggestion would be to set hwc->idx to the
desired number and hope x86_schedule_events() takes the fast path.

Of course, the moment there's host events along with the guest events
the whole thing comes tumbling down, but there's nothing much one can do
about that.

Re: Non-exiting rdpmc on KVM guests?

[Paolo Bonzini]

On 21/04/2015 17:05, Peter Zijlstra wrote:
> On Mon, Apr 20, 2015 at 05:47:23PM -0700, Andy Lutomirski wrote:
>> I just wrote a little perf self-monitoring tool that uses rdpmc to
>> count cycles.  Performance sucks under KVM (VMX).
>>
>> How hard would it be to avoid rdpmc exits in cases where the host and
>> guest pmu configurations are compatible as seen by rdpmc?  I'm mostly
>> ignorant of how the PMU counter offsets and such work.
> 
> I'm not sure how all the VMX stuff works. how is rdpmc seem? It it at
> all possible to not trap that?

Yes, it is.  RDPMC is just yet another "exit reason" and you can avoid
exiting.  However, if you take into account that RDPMC can also be used
to read an inactive counter, and that multiple guests "fight" for the
same host counters, it's even harder to ensure that the guest counter
indices match those on the host.

Paolo

Re: Non-exiting rdpmc on KVM guests?

[Peter Zijlstra]

On Tue, Apr 21, 2015 at 06:32:54PM +0200, Paolo Bonzini wrote:

> However, if you take into account that RDPMC can also be used
> to read an inactive counter, and that multiple guests "fight" for the
> same host counters, it's even harder to ensure that the guest counter
> indices match those on the host.

That doesn't make sense, only a single vcpu task will ever run at any
one time.

There cannot be inter guest counter contention.  Only the host can
compete for the same resources.

Re: Non-exiting rdpmc on KVM guests?

[Andy Lutomirski]

On Tue, Apr 21, 2015 at 1:51 PM, Peter Zijlstra <peterz@infradead.org> wrote:
> On Tue, Apr 21, 2015 at 06:32:54PM +0200, Paolo Bonzini wrote:
>
>> However, if you take into account that RDPMC can also be used
>> to read an inactive counter, and that multiple guests "fight" for the
>> same host counters, it's even harder to ensure that the guest counter
>> indices match those on the host.
>
> That doesn't make sense, only a single vcpu task will ever run at any
> one time.
>
> There cannot be inter guest counter contention.  Only the host can
> compete for the same resources.

One question is whether we care if we leak unrelated counters to the
guest.  (We already leak them to unrelated user tasks, so this is
hopefully not a big deal.  OTOH, the API is different for guests.)

Another question is whether it's even worth trying to optimize this.

--Andy

Re: Non-exiting rdpmc on KVM guests?

[Peter Zijlstra]

On Tue, Apr 21, 2015 at 02:10:53PM -0700, Andy Lutomirski wrote:

> One question is whether we care if we leak unrelated counters to the
> guest.  (We already leak them to unrelated user tasks, so this is
> hopefully not a big deal.  OTOH, the API is different for guests.)

Good question indeed. I really do not know.

> Another question is whether it's even worth trying to optimize this.

I think I just ran into a bunch of people who think virt pmu stuff is
important, but we'll have to see if they follow up with the effort of
actually doing the work.

My only concern is that they'll not make a mess of things ;-)

Re: Non-exiting rdpmc on KVM guests?

[Paolo Bonzini]

On 21/04/2015 22:51, Peter Zijlstra wrote:
> > However, if you take into account that RDPMC can also be used
> > to read an inactive counter, and that multiple guests "fight" for the
> > same host counters, it's even harder to ensure that the guest counter
> > indices match those on the host.
>
> That doesn't make sense, only a single vcpu task will ever run at any
> one time.

Right, but it puts more pressure on the scheduler which could end up
going more often through the slow path.

Paolo