Re: [RFC PATCH 08/21] KVM: x86: Add kvm_x86_ops hook to short circuit emulation

From: Sean Christopherson <sean.j.christopherson@intel.com>
To: Andy Lutomirski <luto@amacapital.net>
Cc: "Andy Lutomirski" <luto@kernel.org>,
	"Paolo Bonzini" <pbonzini@redhat.com>,
	"Radim Krčmář" <rkrcmar@redhat.com>,
	"Thomas Gleixner" <tglx@linutronix.de>,
	"Ingo Molnar" <mingo@redhat.com>,
	"Borislav Petkov" <bp@alien8.de>, "X86 ML" <x86@kernel.org>,
	"Jarkko Sakkinen" <jarkko.sakkinen@linux.intel.com>,
	"Joerg Roedel" <joro@8bytes.org>,
	"H. Peter Anvin" <hpa@zytor.com>,
	"kvm list" <kvm@vger.kernel.org>,
	LKML <linux-kernel@vger.kernel.org>,
	linux-sgx@vger.kernel.org
Subject: Re: [RFC PATCH 08/21] KVM: x86: Add kvm_x86_ops hook to short circuit emulation
Date: Mon, 19 Aug 2019 15:01:50 -0700	[thread overview]
Message-ID: <20190819220150.GE1916@linux.intel.com> (raw)
In-Reply-To: <25BBDA64-1253-4429-95AF-5D578684F6CC@amacapital.net>

On Thu, Aug 15, 2019 at 05:47:12PM -0700, Andy Lutomirski wrote:
> 
> 
> >> On Jul 29, 2019, at 7:49 PM, Sean Christopherson <sean.j.christopherson@intel.com> wrote:
> >> 
> >> On Sat, Jul 27, 2019 at 10:38:03AM -0700, Andy Lutomirski wrote:
> >> On Fri, Jul 26, 2019 at 10:52 PM Sean Christopherson
> >> <sean.j.christopherson@intel.com> wrote:
> >>> 
> >>> Similar to the existing AMD #NPF case where emulation of the current
> >>> instruction is not possible due to lack of information, virtualization
> >>> of Intel SGX will introduce a scenario where emulation is not possible
> >>> due to the VMExit occurring in an SGX enclave.  And again similar to
> >>> the AMD case, emulation can be initiated by kvm_mmu_page_fault(), i.e.
> >>> outside of the control of the vendor-specific code.
> >>> 
> >>> While the cause and architecturally visible behavior of the two cases
> >>> is different,  e.g. Intel SGX will inject a #UD whereas AMD #NPF is a
> >>> clean resume or complete shutdown, the impact on the common emulation
> >>> code is identical: KVM must stop emulation immediately and resume the
> >>> guest.
> >>> 
> >>> Replace the exisiting need_emulation_on_page_fault() with a more generic
> >>> is_emulatable() kvm_x86_ops callback, which is called unconditionally
> >>> by x86_emulate_instruction().
> >> 
> >> Having recently noticed that emulate_ud() is broken when the guest's
> >> TF is set, I suppose I should ask: does your new code function
> >> sensibly when TF is set?
> > 
> > Barring a VMX fault injection interaction I'm not thinking of, yes.  The
> > SGX reaction to the #UD VM-Exit is to inject a #UD and resume the guest,
> > pending breakpoints shouldn't be affected in any way (unless some other
> > part of KVM mucks with them, e.g. when guest single-stepping is enabled).
> 
> What I mean is: does the code actually do what you think it does if TF is
> set?  Right now, as I understand it, the KVM emulation code has a bug in
> which some emulated faults also inject #DB despite the fact that the
> instruction faulted, and the #DB seems to take precedence over the original
> fault.  This confuses the guest.

Yes.  The proposed change is to inject the #UD instead of calling into the
emulator, and by inspection I've verified that all code that injects a #DB
is either contained within the emulator or is mutually exclusive with an
intercepted #UD.  It's a qualified yes because I don't have an actual
testcase to verify my literacy.  I'll look into adding a test, either to
the selftest/x86/sgx or to kvm-unit-tests.