From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Huang, Kai" Subject: Re: [intel-sgx-kernel-dev] [PATCH 08/10] kvm: vmx: add guest's IA32_SGXLEPUBKEYHASHn runtime switch support Date: Tue, 23 May 2017 17:43:45 +1200 Message-ID: <24a2198f-fd0f-6db3-274d-3392a9037265@linux.intel.com> References: <20170508052434.3627-1-kai.huang@linux.intel.com> <20170508052434.3627-9-kai.huang@linux.intel.com> <58dcdb2d-6894-b0a3-8d6f-2ab752fd6d22@linux.intel.com> <20170515124622.piupyk57vjdoppl5@intel.com> <478d9303-00b7-4f29-6124-0c1433851952@linux.intel.com> <1495030889.23465.13.camel@intel.com> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Cc: Sean Christopherson , Jarkko Sakkinen , Paolo Bonzini , haim.cohen@intel.com, "intel-sgx-kernel-dev@lists.01.org" , kvm list , Radim Krcmar To: Andy Lutomirski Return-path: Received: from mga14.intel.com ([192.55.52.115]:41094 "EHLO mga14.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S933895AbdEWFnx (ORCPT ); Tue, 23 May 2017 01:43:53 -0400 In-Reply-To: Sender: kvm-owner@vger.kernel.org List-ID: On 5/21/2017 9:55 AM, Andy Lutomirski wrote: > On Thu, May 18, 2017 at 1:14 AM, Huang, Kai wrote: >> You are making assumption that KVM will run ENCLS on behalf of guest. :) >> >> If we don't need to look into guest's SIGSTRUCT, EINITTOKEN, etc, then I >> actually prefer to using MTF, as with MTF we don't have to do all the >> remapping guest's virtual address to KVM's virtual address thing, if we >> don't need to look into guest's ENCLS parameter. But if we need to look into >> guest's ENCLS parameters, for example, to locate physical SECS page, or to >> update physical EPC page's info (that KVM needs to maintain), maybe we can >> choose running ENCLS on behalf of guest. > > After thinking about this a bit, I don't see how MTF helps. > Currently, KVM works kind of like this: > > local_irq_disable(); > set up stuff; > VMRESUME; > restore some host state; > local_irq_enable(); > > If the guest is going to run with the EINIT-exiting bit clear, the > only way I see this working is to modify KVM along the lines of: > > local_irq_disable(); > set up stuff; > if (condition here) { > WRMSR to SGXLEPUBKEYHASH; > update percpu shadow copyl > clear EINIT-exiting bit; > } else { > set EINIT-exiting bit; > } > VMRESUME; > restore some host state; > local_irq_enable(); > > where "condition here" might be something like "the last VMRESUME > exited due to EINIT". > > I don't see how MTF helps much. And if I were the KVM maintainer, I > would probably prefer to trap EINIT instead of adding a special case > to the main vm entry code. > Hi Andy, Thanks for your comments. However I didn't intend to use MTF in your way. The idea of using MTF (along with ENCLS VMEXIT) is, by turning on MTF VMEXIT upon ENCLS VMEXIT, we are able to mark a single step VMEXIT after ENCLS so that ENCLS can run in guest as single step. Let me explain how the two approaches work below in general, so that we can decide which is better. Only trapping EINIT in order to update IA32_SGXLEPUBKEYHASHn is relatively simpler but I'd compare the two in more general way, assuming we may want to trap more ENCLS in order to, ex, track EPC/Enclave status/info, in the future to support, ex, EPC oversubscription between KVM guests. Below diagram shows the basic idea of the two approaches. -------------------------------------------------------------- | ENCLS | -------------------------------------------------------------- | /|\ ENCLS VMEXIT | | VMENTRY | | \|/ | 1) identify which ENCLS leaf (RAX) 2) reconstruct/remap guest's ENCLS parameters, ex: - remap any guest VA (virtual address) to KVM VA - reconstruct PAGEINFO 3) do whatever needed before ENCLS, ex: - updating MSRs before EINIT 4) run ENCLS on behalf of guest, and skip ENCLS 5) emulate ENCLS result (succeeded or not) - update guest's RAX-RDX. - and/or inject #GP (or #UD). 6) do whatever needed after ENCLS, ex: - updating EPC/Enclave status/info 1) Run ENCLS on behalf of guest -------------------------------------------------------------- | ENCLS | -------------------------------------------------------------- |/|\ |/|\ ENCLS VMEXIT | | VMENTRY MTF VMEXIT | | VMENTRY | | | | \|/| \|/| 1) Turn off EMCLS VMEXIT 1) Turn off MTF VMEXIT 2) turn on MTF VMEXIT 2) Turn on ENCLS VMEXIT 3) cache ENCLS parameters 3) check whether ENCLS has run (ENCLS changes RAX) 4) check whether ENCLS succeeded 4) do whatever needed before or not. ENCLS 5) do whatever needed after ENCLS 2) Using MTF The concern of running ENCLS on behalf of guest is emulating ENCLS error. KVM needs to *correctly* emulate ENCLS error to guest so that the error we inject to guest can reflect the right behavior as if ENCLS run in guest. Running ENCLS in root-mode may be potentially different running ENCLS in non-root mode, therefore we have to go through all possible error codes to make sure we can emulate. And for some error code, ex, SGX_LOCKFAIL, we can handle it in KVM and don't have to inject error to guest. So the point is we have to go through all error code to make sure KVM can emulate ENCLS error code correctly for guest. Another argument is Intel may add new error codes in the future when more SGX functionalities are introduced, so emulating error code may be a burden. Using MTF is also a little bit tricky, as when we turn on MTF VMEXIT upon ENCLS VMEXIT, the MTF won't be absolutely pending at end of that ENCLS. For example, MTF may be pending at end of interrupt (cannot recall exactly) if event is pending during VMENTRY from ENCLS VMEXIT. Therefore we have to do additional thing to check whether this MTF VMEXIT really happens after ENCLS run (step 3 above). And depending on what we need to do, we may need to check whether ENCLS succeeded or not in guest, which is also tricky, as ENCLS can fail in either setting error code in RAX, or generating #GP or #UD (step 4 above). We may still need to do gva->gpa->hpa, ex, in order to locate EPC/SECS page and update status, depending on the purpose of trapping ENCLS. But by using MTF, we don't have to worry about ENCLS error emulation, as ENCLS runs in guest, thus we don't need to worry about this root-mode and non-root mode difference. I think this is the major reason that we want to use MTF.