From: Jarkko Sakkinen <jarkko@kernel.org>
To: Kai Huang <kai.huang@intel.com>
Cc: Sean Christopherson <seanjc@google.com>,
kvm@vger.kernel.org, x86@kernel.org, linux-sgx@vger.kernel.org,
linux-kernel@vger.kernel.org, luto@kernel.org,
dave.hansen@intel.com, rick.p.edgecombe@intel.com,
haitao.huang@intel.com, pbonzini@redhat.com, bp@alien8.de,
tglx@linutronix.de, mingo@redhat.com, hpa@zytor.com
Subject: Re: [PATCH v3 03/25] x86/sgx: Wipe out EREMOVE from sgx_free_epc_page()
Date: Tue, 16 Mar 2021 01:11:34 +0200 [thread overview]
Message-ID: <YE/ppohYu9AC68L+@kernel.org> (raw)
In-Reply-To: <20210316092934.d4dd7f2e65f507c3856341bc@intel.com>
On Tue, Mar 16, 2021 at 09:29:34AM +1300, Kai Huang wrote:
> On Mon, 15 Mar 2021 15:19:32 +0200 Jarkko Sakkinen wrote:
> > On Mon, Mar 15, 2021 at 03:18:16PM +0200, Jarkko Sakkinen wrote:
> > > On Mon, Mar 15, 2021 at 08:12:36PM +1300, Kai Huang wrote:
> > > > On Sat, 13 Mar 2021 12:45:53 +0200 Jarkko Sakkinen wrote:
> > > > > On Fri, Mar 12, 2021 at 01:21:54PM -0800, Sean Christopherson wrote:
> > > > > > On Thu, Mar 11, 2021, Kai Huang wrote:
> > > > > > > From: Jarkko Sakkinen <jarkko@kernel.org>
> > > > > > >
> > > > > > > EREMOVE takes a page and removes any association between that page and
> > > > > > > an enclave. It must be run on a page before it can be added into
> > > > > > > another enclave. Currently, EREMOVE is run as part of pages being freed
> > > > > > > into the SGX page allocator. It is not expected to fail.
> > > > > > >
> > > > > > > KVM does not track how guest pages are used, which means that SGX
> > > > > > > virtualization use of EREMOVE might fail.
> > > > > > >
> > > > > > > Break out the EREMOVE call from the SGX page allocator. This will allow
> > > > > > > the SGX virtualization code to use the allocator directly. (SGX/KVM
> > > > > > > will also introduce a more permissive EREMOVE helper).
> > > > > > >
> > > > > > > Implement original sgx_free_epc_page() as sgx_encl_free_epc_page() to be
> > > > > > > more specific that it is used to free EPC page assigned to one enclave.
> > > > > > > Print an error message when EREMOVE fails to explicitly call out EPC
> > > > > > > page is leaked, and requires machine reboot to get leaked pages back.
> > > > > > >
> > > > > > > Signed-off-by: Jarkko Sakkinen <jarkko@kernel.org>
> > > > > > > Co-developed-by: Kai Huang <kai.huang@intel.com>
> > > > > > > Acked-by: Jarkko Sakkinen <jarkko@kernel.org>
> > > > > > > Signed-off-by: Kai Huang <kai.huang@intel.com>
> > > > > > > ---
> > > > > > > v2->v3:
> > > > > > >
> > > > > > > - Fixed bug during copy/paste which results in SECS page and va pages are not
> > > > > > > correctly freed in sgx_encl_release() (sorry for the mistake).
> > > > > > > - Added Jarkko's Acked-by.
> > > > > >
> > > > > > That Acked-by should either be dropped or moved above Co-developed-by to make
> > > > > > checkpatch happy.
> > > > > >
> > > > > > Reviewed-by: Sean Christopherson <seanjc@google.com>
> > > > >
> > > > > Oops, my bad. Yup, ack should be removed.
> > > > >
> > > > > /Jarkko
> > > >
> > > > Hi Jarkko,
> > > >
> > > > Your reply of your concern of this patch to the cover-letter
> > > >
> > > > https://lore.kernel.org/lkml/YEkJXu262YDa8ZaK@kernel.org/
> > > >
> > > > reminds me to do more sanity check of whether removing EREMOVE in
> > > > sgx_free_epc_page() will impact other code path or not, and I think
> > > > sgx_encl_release() is not the only place should be changed:
> > > >
> > > > - sgx_encl_shrink() needs to call sgx_encl_free_epc_page(), since when this is
> > > > called, the VA page can be already valid -- there are other failures can
> > > > trigger sgx_encl_shrink().
> > >
> > > You right about this, good catch.
> > >
> > > Shrink needs to always do EREMOVE as grow has done EPA, which changes
> > > EPC page state.
> > >
> > > > - sgx_encl_add_page() should call sgx_encl_free_epc_page() in "err_out_free:"
> > > > label, since the EPC page can be already valid when error happened, i.e. when
> > > > EEXTEND fails.
> > >
> > > Yes, correct, good work!
> > >
> > > > Other places should be OK per my check, but I'd prefer to just replacing all
> > > > sgx_free_epc_page() call sites in driver with sgx_encl_free_epc_page(), with
> > > > one exception: sgx_alloc_va_page(), which calls sgx_free_epc_page() when EPA
> > > > fails, in which case EREMOVE is not required for sure.
> > >
> > > I would not unless they require it.
> > >
> > > > Your idea, please?
> > > >
> > > > Btw, introducing a driver wrapper of sgx_free_epc_page() does make sense to me,
> > > > because virtualization has a counterpart in sgx/virt.c too.
> > >
> > > It does make sense to use sgx_free_epc_page() everywhere where it's
> > > the right thing to call and here's why.
> > >
> > > If there is some unrelated regression that causes EPC page not get
> > > uninitialized when it actually should, doing extra EREMOVE could mask
> > > those bugs. I.e. it can postpone a failure, which can make a bug harder
> > > to backtrace.
> > >
> >
> > I.e. even though it is true that for correctly working code extra EREMOVE
> > is nil functionality, it could change semantics for buggy code.
>
> Thanks for feedback. Sorry I am not sure if I understand you. So if we don't
> want to bring functionality change, we need to replace sgx_free_epc_page() in
> all call sites with sgx_encl_free_epc_page(). To me for this patch only, it's
> better not to bring any functional change, so I intend to replace all (I now
> consider even leaving sgx_alloc_va_page() out is not good idea in *this*
> patch).
>
> Or do you just want to replace sgx_free_epc_page() with
> sgx_encl_free_epc_page() in sgx_encl_shrink() and sgx_encl_add_page(), as I
> pointed above? In this way there will be functional change in this patch, and
> we need to explicitly explain why leaving others out is OK in commit message.
>
> To me I prefer the former.
But yes, I'm cool with your preference and I do get your argument, I just
need to review it, and do not consider it as my patch :-)
/Jarkko
next prev parent reply other threads:[~2021-03-15 23:12 UTC|newest]
Thread overview: 105+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-03-09 1:38 [PATCH v2 00/25] KVM SGX virtualization support Kai Huang
2021-03-09 1:38 ` [PATCH v2 01/25] x86/cpufeatures: Make SGX_LC feature bit depend on SGX bit Kai Huang
2021-03-12 21:05 ` Sean Christopherson
2021-03-09 1:39 ` [PATCH v2 02/25] x86/cpufeatures: Add SGX1 and SGX2 sub-features Kai Huang
2021-03-12 21:10 ` Sean Christopherson
2021-03-09 1:39 ` [PATCH v2 03/25] x86/sgx: Wipe out EREMOVE from sgx_free_epc_page() Kai Huang
2021-03-10 9:35 ` Kai Huang
2021-03-11 2:01 ` [PATCH v3 " Kai Huang
2021-03-12 21:21 ` Sean Christopherson
2021-03-13 10:45 ` Jarkko Sakkinen
2021-03-15 7:12 ` Kai Huang
2021-03-15 13:18 ` Jarkko Sakkinen
2021-03-15 13:19 ` Jarkko Sakkinen
2021-03-15 20:29 ` Kai Huang
2021-03-15 22:59 ` Jarkko Sakkinen
2021-03-15 23:50 ` Kai Huang
2021-03-15 23:11 ` Jarkko Sakkinen [this message]
2021-03-09 1:39 ` [PATCH v2 04/25] x86/sgx: Add SGX_CHILD_PRESENT hardware error code Kai Huang
2021-03-09 1:39 ` [PATCH v2 05/25] x86/sgx: Introduce virtual EPC for use by KVM guests Kai Huang
2021-03-09 1:39 ` [PATCH v2 06/25] x86/cpu/intel: Allow SGX virtualization without Launch Control support Kai Huang
2021-03-12 21:33 ` Sean Christopherson
2021-03-09 1:39 ` [PATCH v2 07/25] x86/sgx: Initialize virtual EPC driver even when SGX driver is disabled Kai Huang
2021-03-12 21:44 ` Sean Christopherson
2021-03-13 19:05 ` Jarkko Sakkinen
2021-03-13 19:07 ` Jarkko Sakkinen
2021-03-14 15:25 ` Jarkko Sakkinen
2021-03-14 15:27 ` Jarkko Sakkinen
2021-03-15 3:13 ` Kai Huang
2021-03-15 13:04 ` Jarkko Sakkinen
2021-03-15 13:51 ` Jarkko Sakkinen
2021-03-15 20:48 ` Kai Huang
2021-03-15 23:05 ` Jarkko Sakkinen
2021-03-15 23:08 ` Jarkko Sakkinen
2021-03-15 23:49 ` Kai Huang
2021-03-16 12:44 ` Jarkko Sakkinen
2021-03-16 1:13 ` Sean Christopherson
2021-03-16 12:46 ` Jarkko Sakkinen
2021-03-18 0:04 ` Kai Huang
2021-03-09 1:39 ` [PATCH v2 08/25] x86/sgx: Expose SGX architectural definitions to the kernel Kai Huang
2021-03-12 21:58 ` Sean Christopherson
2021-03-15 3:36 ` Kai Huang
2021-03-09 1:39 ` [PATCH v2 09/25] x86/sgx: Move ENCLS leaf definitions to sgx.h Kai Huang
2021-03-09 1:39 ` [PATCH v2 10/25] x86/sgx: Add SGX2 ENCLS leaf definitions (EAUG, EMODPR and EMODT) Kai Huang
2021-03-09 1:39 ` [PATCH v2 11/25] x86/sgx: Add encls_faulted() helper Kai Huang
2021-03-09 1:39 ` [PATCH v2 12/25] x86/sgx: Add helper to update SGX_LEPUBKEYHASHn MSRs Kai Huang
2021-03-09 1:39 ` [PATCH v2 13/25] x86/sgx: Add helpers to expose ECREATE and EINIT to KVM Kai Huang
2021-03-15 4:08 ` Kai Huang
2021-03-09 1:39 ` [PATCH v2 14/25] x86/sgx: Move provisioning device creation out of SGX driver Kai Huang
2021-03-09 1:40 ` [PATCH v2 15/25] KVM: x86: Export kvm_mmu_gva_to_gpa_{read,write}() for SGX (VMX) Kai Huang
2021-03-09 1:40 ` [PATCH v2 16/25] KVM: x86: Define new #PF SGX error code bit Kai Huang
2021-03-09 1:40 ` [PATCH v2 17/25] KVM: x86: Add support for reverse CPUID lookup of scattered features Kai Huang
2021-03-09 1:40 ` [PATCH v2 18/25] KVM: x86: Add reverse-CPUID lookup support for scattered SGX features Kai Huang
2021-03-09 1:40 ` [PATCH v2 19/25] KVM: VMX: Add basic handling of VM-Exit from SGX enclave Kai Huang
2021-03-09 1:40 ` [PATCH v2 20/25] KVM: VMX: Frame in ENCLS handler for SGX virtualization Kai Huang
2021-03-09 1:40 ` [PATCH v2 21/25] KVM: VMX: Add SGX ENCLS[ECREATE] handler to enforce CPUID restrictions Kai Huang
2021-03-09 1:40 ` [PATCH v2 22/25] KVM: VMX: Add emulation of SGX Launch Control LE hash MSRs Kai Huang
2021-03-09 1:40 ` [PATCH v2 23/25] KVM: VMX: Add ENCLS[EINIT] handler to support SGX Launch Control (LC) Kai Huang
2021-03-09 1:40 ` [PATCH v2 24/25] KVM: VMX: Enable SGX virtualization for SGX1, SGX2 and LC Kai Huang
2021-03-09 1:41 ` [PATCH v2 25/25] KVM: x86: Add capability to grant VM access to privileged SGX attribute Kai Huang
2021-03-09 9:30 ` [PATCH v2 00/25] KVM SGX virtualization support Borislav Petkov
2021-03-09 18:08 ` Kai Huang
2021-03-09 18:49 ` Paolo Bonzini
2021-03-12 22:04 ` Sean Christopherson
2021-03-13 4:30 ` Kai Huang
2021-03-10 9:27 ` Kai Huang
2021-03-10 13:29 ` Borislav Petkov
2021-03-11 2:05 ` Kai Huang
2021-03-10 18:01 ` Jarkko Sakkinen
2021-03-10 20:44 ` Kai Huang
2021-03-19 7:29 [PATCH v3 " Kai Huang
2021-03-19 7:22 ` [PATCH v3 03/25] x86/sgx: Wipe out EREMOVE from sgx_free_epc_page() Kai Huang
2021-03-22 18:16 ` Borislav Petkov
2021-03-22 18:56 ` Sean Christopherson
2021-03-22 19:11 ` Paolo Bonzini
2021-03-22 20:43 ` Kai Huang
2021-03-23 16:40 ` Paolo Bonzini
2021-03-22 19:15 ` Borislav Petkov
2021-03-22 19:37 ` Sean Christopherson
2021-03-22 20:36 ` Kai Huang
2021-03-22 21:06 ` Borislav Petkov
2021-03-22 22:06 ` Kai Huang
2021-03-22 22:37 ` Borislav Petkov
2021-03-22 23:16 ` Kai Huang
2021-03-23 15:45 ` Sean Christopherson
2021-03-23 16:06 ` Borislav Petkov
2021-03-23 16:21 ` Sean Christopherson
2021-03-23 16:32 ` Borislav Petkov
2021-03-23 16:51 ` Sean Christopherson
2021-03-24 9:38 ` Kai Huang
2021-03-24 10:09 ` Paolo Bonzini
2021-03-24 10:48 ` Kai Huang
2021-03-24 11:24 ` Paolo Bonzini
2021-03-24 23:23 ` Kai Huang
2021-03-24 23:39 ` Paolo Bonzini
2021-03-24 23:46 ` Kai Huang
2021-03-25 8:42 ` Borislav Petkov
2021-03-25 9:38 ` Kai Huang
2021-03-25 16:52 ` Borislav Petkov
2021-03-24 9:28 ` Jarkko Sakkinen
2021-03-23 16:38 ` Paolo Bonzini
2021-03-23 17:02 ` Sean Christopherson
2021-03-23 17:06 ` Paolo Bonzini
2021-03-23 17:16 ` Sean Christopherson
2021-03-23 18:16 ` Borislav Petkov
2021-03-24 9:26 ` Jarkko Sakkinen
2021-03-22 22:23 ` Kai Huang
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=YE/ppohYu9AC68L+@kernel.org \
--to=jarkko@kernel.org \
--cc=bp@alien8.de \
--cc=dave.hansen@intel.com \
--cc=haitao.huang@intel.com \
--cc=hpa@zytor.com \
--cc=kai.huang@intel.com \
--cc=kvm@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-sgx@vger.kernel.org \
--cc=luto@kernel.org \
--cc=mingo@redhat.com \
--cc=pbonzini@redhat.com \
--cc=rick.p.edgecombe@intel.com \
--cc=seanjc@google.com \
--cc=tglx@linutronix.de \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).