Re: [PATCH RFC] KVM: VMX: drop vmm_exclusive module parameter

From: "Radim Krčmář" <rkrcmar@redhat.com>
To: Arnaldo Carvalho de Melo <acme@kernel.org>
Cc: David Hildenbrand <david@redhat.com>,
	kvm@vger.kernel.org, Paolo Bonzini <pbonzini@redhat.com>,
	dvyukov@google.com,
	Alexander Shishkin <alexander.shishkin@linux.intel.com>,
	linux-kernel@vger.kernel.org
Subject: Re: [PATCH RFC] KVM: VMX: drop vmm_exclusive module parameter
Date: Wed, 21 Jun 2017 20:24:01 +0200	[thread overview]
Message-ID: <20170621182401.GC27032@potion> (raw)
In-Reply-To: <20170621174822.GR13640@kernel.org>

2017-06-21 14:48-0300, Arnaldo Carvalho de Melo:
> Em Fri, Mar 10, 2017 at 12:47:13PM +0100, David Hildenbrand escreveu:
> > vmm_exclusive=0 leads to KVM setting X86_CR4_VMXE always and calling
> > VMXON only when the vcpu is loaded. X86_CR4_VMXE is used as an
> > indication in cpu_emergency_vmxoff() (called on kdump) if VMXOFF has to be
> > called. This is obviously not the case if both are used independtly.
> > Calling VMXOFF without a previous VMXON will result in an exception.
> > 
> > In addition, X86_CR4_VMXE is used as a mean to test if VMX is already in
> > use by another VMM in hardware_enable(). So there can't really be
> > co-existance. If the other VMM is prepared for co-existance and does a
> > similar check, only one VMM can exist. If the other VMM is not prepared
> > and blindly sets/clears X86_CR4_VMXE, we will get inconsistencies with
> > X86_CR4_VMXE.
> > 
> > As we also had bug reports related to clearing of vmcs with vmm_exclusive=0
> > this seems to be pretty much untested. So let's better drop it.
> > 
> > While at it, directly move setting/clearing X86_CR4_VMXE into
> > kvm_cpu_vmxon/off.
> 
> Oh well, I was using, as suggested by Alexander, this parameter to be
> able to use Intel PT on the host on a Broadwell machine, i.e.:
> 
>   perf record -e intel_pt// usleep 1
>   perf script

We thought that blacklisting the KVM module was a good solution ...
Were you using KVM virtual machines with vmm_exclusive=0?

> would show decoded Intel PT records, no more :-\ But I'm clueless about
> KVM internals, so just reporting the change in behaviour for this very
> specific use case.
> 
> Now I don't know if this is something that would make Intel PT be usable
> on Broadwell machines but wouldn't be required with newer chips, will
> test with a Kaby Lake i5 7500 when back at my home office...

Most likely, SDM 35.2.8.2 says:

 Initial implementations of Intel Processor Trace do not support tracing
 in VMX operation. Such processors indicate this by returning 0 for
 IA32_VMX_MISC[bit 14].

so something akin to vmm_exclusive is about the only option there.

Please try if Kaby Lake is already an advanced implementation, because
we might need to disable PT when entering VMX non-root mode
(so the tracing packets are not be written into guest's memory, just
 like with PEBS).

Thanks.