From: "Roger Pau Monné" <roger.pau@citrix.com>
To: "Michał Leszczyński" <michal.leszczynski@cert.pl>
Cc: Kevin Tian <kevin.tian@intel.com>,
luwei kang <luwei.kang@intel.com>,
Jun Nakajima <jun.nakajima@intel.com>, Wei Liu <wl@xen.org>,
Andrew Cooper <andrew.cooper3@citrix.com>,
Jan Beulich <jbeulich@suse.com>,
Xen-devel <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH v1 7/7] x86/vmx: switch IPT MSRs on vmentry/vmexit
Date: Thu, 18 Jun 2020 13:49:33 +0200 [thread overview]
Message-ID: <20200618114933.GK735@Air-de-Roger> (raw)
In-Reply-To: <633218159.9539851.1592478453009.JavaMail.zimbra@cert.pl>
On Thu, Jun 18, 2020 at 01:07:33PM +0200, Michał Leszczyński wrote:
> ----- 18 cze 2020 o 10:52, Roger Pau Monné roger.pau@citrix.com napisał(a):
>
> > On Wed, Jun 17, 2020 at 08:56:57PM +0200, Michał Leszczyński wrote:
> >> ----- 17 cze 2020 o 17:14, Andrew Cooper andrew.cooper3@citrix.com napisał(a):
> >>
> >> > On 17/06/2020 13:51, Roger Pau Monné wrote:
> >> >> On Wed, Jun 17, 2020 at 01:54:45PM +0200, Michał Leszczyński wrote:
> >> >>> ----- 17 cze 2020 o 11:09, Roger Pau Monné roger.pau@citrix.com napisał(a):
> >> >>>
> >> >>>> 24 Virtual Machine Control Structures -> 24.8 VM-entry Control Fields -> 24.8.1
> >> >>>> VM-Entry Controls
> >> >>>> Software should consult the VMX capability MSRs IA32_VMX_ENTRY_CTLS to determine
> >> >>>> how it should set the reserved bits.
> >> >>> Please look at bit position 18 "Load IA32_RTIT_CTL".
> >> >> I think this is something different from what I was referring to.
> >> >> Those options you refer to (load/clear IA32_RTIT_CTL) deal with
> >> >> loading/storing a specific field on the vmcs that maps to the guest
> >> >> IA32_RTIT_CTL.
> >> >>
> >> >> OTOH MSR load lists can be used to load and store any arbitrary MSR on
> >> >> vmentry/vmexit, see section 26.4 LOADING MSRS on the SDM. There's
> >> >> already infrastructure on Xen to do so, see vmx_{add/del/find}_msr.
> >> >
> >> > If I remember the historic roadmaps correctly, there are 3 cases.
> >> >
> >> > The first hardware to support PT (Broadwell?) prohibited its use
> >> > completely in VMX operations. In this case, we can use it to trace PV
> >> > guests iff we don't enable VMX in hardware to begin with.
> >> >
> >> > This was relaxed in later hardware (Skylake?) to permit use within VMX
> >> > operations, but without any help in the VMCS. (i.e. manual context
> >> > switching per this patch, or MSR load lists as noted in the SDM.)
> >> >
> >> > Subsequent support for "virtualised PT" was added (IceLake?) which adds
> >> > the load/save controls, and the ability to translate the output buffer
> >> > under EPT.
> >> >
> >> >
> >> > All of this is from memory so I'm quite possibly wrong with details, but
> >> > I believe this is why the current complexity exists.
> >> >
> >> > ~Andrew
> >>
> >>
> >> I've managed to toggle MSR_IA32_RTIT_CTL values using MSR load lists, as in:
> >>
> >> > 35.5.2.2 Guest-Only Tracing
> >> > "For this usage, VM-entry is programmed to enable trace packet generation, while
> >> > VM-exit is programmed to clear MSR_IA32_RTIT_CTL.TraceEn so as to disable
> >> > trace-packet generation in the host."
> >>
> >> it actually helped a bit. With patch v1 there were parts of hypervisor recorded
> >> in the trace (i.e. the moment between TRACE_EN being set and actual vmenter,
> >> and the moment between vmexit and TRACE_EN being unset). Using MSR load list
> >> this was eliminated. This change will be reflected in patch v2.
> >>
> >>
> >> I can't however implement any working scenario in which all these MSRs are
> >> managed using MSR load lists. As in "35.3.3 Flushing Trace Output": packets are
> >> buffered internally and are flushed only when TRACE_EN bit in MSR_IA32_RTIT_CTL
> >> is set to 0. The values of remaining registers will be stable after everything
> >> is serialized. I think this is too complex for the load lists alone. I belive
> >> that currently SDM instructs to use load lists only for toggling this single
> >> bit on-or-off.
> >
> > I think that's exactly what we want: handling TraceEn at
> > vmentry/vmexit, so that no hypervisor packets are recorded. The rest
> > of the MSRs can be handled in VMM mode without issues. Switching those
> > on every vmentry/vmexit would also add more overhead that needed,
> > since I assume they don't need to be modified on every entry/exit?
>
>
> Assuming that there is a single DomU per pcpu and they are never migrated between pcpus then you never need to modify the remaining MSRs.
>
> In case DomUs are floating or there are multiple DomUs per pcpu, we need to read out a few MSRs on vm-exit and restore them on vm-entry. Right now I'm always using this approach as I'm pretty not sure how to optimize it without introducing additional bugs. I will show the implementation in patch v2.
I think you might likely only need to modify the MSRs when doing
context switches of domains, instead of doing it on every
vmentry/vmexit?
Roger.
next prev parent reply other threads:[~2020-06-18 11:49 UTC|newest]
Thread overview: 59+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-06-16 15:16 [PATCH v1 0/7] Implement support for external IPT monitoring Michał Leszczyński
2020-06-16 15:19 ` [PATCH v1 1/7] x86/vmx: add Intel PT MSR definitions Michał Leszczyński
2020-06-18 13:31 ` Jan Beulich
2020-06-16 15:20 ` [PATCH v1 2/7] x86/vmx: add IPT cpu feature Michał Leszczyński
2020-06-16 16:30 ` Roger Pau Monné
2020-06-17 11:34 ` Jan Beulich
2020-06-16 15:21 ` [PATCH v1 3/7] x86/vmx: add ipt_state as part of vCPU state Michał Leszczyński
2020-06-16 16:33 ` Roger Pau Monné
2020-06-16 15:22 ` [PATCH v1 4/7] x86/vmx: add do_vmtrace_op Michał Leszczyński
2020-06-16 17:23 ` Roger Pau Monné
2020-06-17 19:13 ` Michał Leszczyński
2020-06-18 3:20 ` Tamas K Lengyel
2020-06-18 11:01 ` Michał Leszczyński
2020-06-18 11:55 ` Roger Pau Monné
2020-06-18 12:51 ` Jan Beulich
2020-06-18 13:09 ` Michał Leszczyński
2020-06-18 13:24 ` Jan Beulich
2020-06-18 13:40 ` Roger Pau Monné
2020-06-18 8:46 ` Roger Pau Monné
2020-06-18 15:25 ` Michał Leszczyński
2020-06-18 15:39 ` Jan Beulich
2020-06-18 15:47 ` Tamas K Lengyel
2020-06-18 15:49 ` Tamas K Lengyel
2020-06-16 15:22 ` [PATCH v1 5/7] tools/libxc: add xc_ptbuf_* functions Michał Leszczyński
2020-06-16 15:23 ` [PATCH v1 6/7] tools/proctrace: add proctrace tool Michał Leszczyński
2020-06-16 15:24 ` [PATCH v1 7/7] x86/vmx: switch IPT MSRs on vmentry/vmexit Michał Leszczyński
2020-06-16 17:38 ` Roger Pau Monné
2020-06-16 17:47 ` Michał Leszczyński
2020-06-17 9:09 ` Roger Pau Monné
2020-06-17 11:54 ` Michał Leszczyński
2020-06-17 12:51 ` Roger Pau Monné
2020-06-17 15:14 ` Andrew Cooper
2020-06-17 18:56 ` Michał Leszczyński
2020-06-18 8:52 ` Roger Pau Monné
2020-06-18 11:07 ` Michał Leszczyński
2020-06-18 11:49 ` Roger Pau Monné [this message]
2020-06-17 23:30 ` Kang, Luwei
2020-06-18 10:02 ` Andrew Cooper
2020-06-18 17:38 ` Andrew Cooper
2020-06-16 18:17 ` [PATCH v1 0/7] Implement support for external IPT monitoring Andrew Cooper
2020-06-16 18:47 ` Michał Leszczyński
2020-06-16 20:16 ` Andrew Cooper
2020-06-17 3:02 ` Tamas K Lengyel
2020-06-17 16:19 ` Andrew Cooper
2020-06-17 16:27 ` Tamas K Lengyel
2020-06-17 17:23 ` Andrew Cooper
2020-06-17 19:31 ` Tamas K Lengyel
2020-06-17 19:30 ` Michał Leszczyński
2020-06-17 20:20 ` Michał Leszczyński
2020-06-18 8:25 ` Roger Pau Monné
2020-06-18 14:59 ` Michał Leszczyński
2020-06-17 1:35 ` Tian, Kevin
2020-06-17 6:45 ` Kang, Luwei
2020-06-17 9:21 ` Roger Pau Monné
2020-06-17 12:37 ` Kang, Luwei
2020-06-17 12:53 ` Roger Pau Monné
2020-06-17 23:29 ` Kang, Luwei
2020-06-18 0:56 ` Michał Leszczyński
2020-06-18 7:00 ` Roger Pau Monné
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200618114933.GK735@Air-de-Roger \
--to=roger.pau@citrix.com \
--cc=andrew.cooper3@citrix.com \
--cc=jbeulich@suse.com \
--cc=jun.nakajima@intel.com \
--cc=kevin.tian@intel.com \
--cc=luwei.kang@intel.com \
--cc=michal.leszczynski@cert.pl \
--cc=wl@xen.org \
--cc=xen-devel@lists.xenproject.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).