Re: [PATCH v1 7/7] x86/vmx: switch IPT MSRs on vmentry/vmexit

[Andrew Cooper <andrew.cooper3@citrix.com>]

On 18/06/2020 00:30, Kang, Luwei wrote:
>>> On Wed, Jun 17, 2020 at 01:54:45PM +0200, Michał Leszczyński wrote:
>>>> ----- 17 cze 2020 o 11:09, Roger Pau Monné roger.pau@citrix.com napisał(a):
>>>>
>>>>> 24 Virtual Machine Control Structures -> 24.8 VM-entry Control
>>>>> Fields -> 24.8.1 VM-Entry Controls Software should consult the VMX
>> capability MSRs IA32_VMX_ENTRY_CTLS to determine how it should set the
>> reserved bits.
>>>> Please look at bit position 18 "Load IA32_RTIT_CTL".
>>> I think this is something different from what I was referring to.
>>> Those options you refer to (load/clear IA32_RTIT_CTL) deal with
>>> loading/storing a specific field on the vmcs that maps to the guest
>>> IA32_RTIT_CTL.
>>>
>>> OTOH MSR load lists can be used to load and store any arbitrary MSR on
>>> vmentry/vmexit, see section 26.4 LOADING MSRS on the SDM. There's
>>> already infrastructure on Xen to do so, see vmx_{add/del/find}_msr.
>> If I remember the historic roadmaps correctly, there are 3 cases.
>>
>> The first hardware to support PT (Broadwell?) prohibited its use completely in
>> VMX operations.  In this case, we can use it to trace PV guests iff we don't
>> enable VMX in hardware to begin with.
>>
>> This was relaxed in later hardware (Skylake?) to permit use within VMX
>> operations, but without any help in the VMCS.  (i.e. manual context switching
>> per this patch, or MSR load lists as noted in the SDM.)
>>
>> Subsequent support for "virtualised PT" was added (IceLake?) which adds the
>> load/save controls, and the ability to translate the output buffer under EPT.
>>
>>
>> All of this is from memory so I'm quite possibly wrong with details, but I believe
>> this is why the current complexity exists.
> Yes, It include 3 cases.
> 1. Before IA32_VMX_MISC[bit 14]:
>      Intel PT doesn't support tracing in VMX operation. Execution of the VMXON instruction clears IA32_RTIT_CTL.TraceEn and any attempt to write IA32_RTIT_CTL in VMX operation causes a general-protection exception (#GP)
> 2. Support IA32_VMX_MISC[bit 14] but no EPT to direct PT output:
>     Intel PT can be enabled across VMX but the address of Intel PT buffer is always HPA from HW point of view. There is not VMCS support in this stage. The MSR load list can be used for Intel PT context switch(VM-Entry/Exit).
> 3. Intel PT VM improvements (start from Icelake):
>     Add a new guest IA32_RTIT_CTL field in VMCS, and HW treat the PT output addresses as GPA and translate them using EPT.

Thanks for the details, and confirming.  I think for now we can ignore
case 1 for simplicity, as I don't think it is likely that we'll have
someone on Broadwell hardware intending to run without VMX.  (If people
really want it, we can retrofit it, but I don't think the effort is
worth it for now)

~Andrew