* [PATCH v2 0/2] Take away EREMOVE from sgx_free_epc_page() @ 2021-01-12 23:33 jarkko 2021-01-12 23:33 ` [PATCH v2 1/2] x86/sgx: Remove a warn " jarkko 2021-01-12 23:33 ` [PATCH v2 2/2] x86/sgx: Wipe out EREMOVE " jarkko 0 siblings, 2 replies; 6+ messages in thread From: jarkko @ 2021-01-12 23:33 UTC (permalink / raw) To: linux-sgx; +Cc: dave.hansen, kai.huang, haitao.huang, jarkko From: "jarkko@kernel.org" <jarkko@kernel.org> Refactor EREMOVE out of sgx_free_epc_page() since it is only used in one function. I re-organized my Git a bit in order to keep track of things: - 'master' bleading edge and 'next' ready PR ready branch - 'numa' has nearly finished stuff, including the patches I'm posting now. jarkko@kernel.org (2): x86/sgx: Remove a warn from sgx_free_epc_page() x86/sgx: Wipe out EREMOVE from sgx_free_epc_page() arch/x86/kernel/cpu/sgx/encl.c | 15 ++++++++++++++- arch/x86/kernel/cpu/sgx/main.c | 12 ++++-------- 2 files changed, 18 insertions(+), 9 deletions(-) -- 2.29.2 ^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH v2 1/2] x86/sgx: Remove a warn from sgx_free_epc_page() 2021-01-12 23:33 [PATCH v2 0/2] Take away EREMOVE from sgx_free_epc_page() jarkko @ 2021-01-12 23:33 ` jarkko 2021-01-12 23:33 ` [PATCH v2 2/2] x86/sgx: Wipe out EREMOVE " jarkko 1 sibling, 0 replies; 6+ messages in thread From: jarkko @ 2021-01-12 23:33 UTC (permalink / raw) To: linux-sgx Cc: dave.hansen, kai.huang, haitao.huang, jarkko, Thomas Gleixner, Ingo Molnar, Borislav Petkov, x86, H. Peter Anvin From: "jarkko@kernel.org" <jarkko@kernel.org> Remove SGX_EPC_PAGE_RECLAIMER_TRACKED check and warning. This cannot happen, as enclave pages are freed only at the time when encl->refcount triggers, i.e. when both VFS and the page reclaimer have given up on their references. Signed-off-by: jarkko@kernel.org <jarkko@kernel.org> --- arch/x86/kernel/cpu/sgx/main.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/arch/x86/kernel/cpu/sgx/main.c b/arch/x86/kernel/cpu/sgx/main.c index 0da510763c47..b2996cb63a8a 100644 --- a/arch/x86/kernel/cpu/sgx/main.c +++ b/arch/x86/kernel/cpu/sgx/main.c @@ -636,8 +636,6 @@ void sgx_free_epc_page(struct sgx_epc_page *page) struct sgx_epc_section *section = &sgx_epc_sections[page->section]; int ret; - WARN_ON_ONCE(page->flags & SGX_EPC_PAGE_RECLAIMER_TRACKED); - ret = __eremove(sgx_get_epc_virt_addr(page)); if (WARN_ONCE(ret, "EREMOVE returned %d (0x%x)", ret, ret)) return; -- 2.29.2 ^ permalink raw reply related [flat|nested] 6+ messages in thread
* [PATCH v2 2/2] x86/sgx: Wipe out EREMOVE from sgx_free_epc_page() 2021-01-12 23:33 [PATCH v2 0/2] Take away EREMOVE from sgx_free_epc_page() jarkko 2021-01-12 23:33 ` [PATCH v2 1/2] x86/sgx: Remove a warn " jarkko @ 2021-01-12 23:33 ` jarkko 2021-01-13 1:09 ` Kai Huang 1 sibling, 1 reply; 6+ messages in thread From: jarkko @ 2021-01-12 23:33 UTC (permalink / raw) To: linux-sgx Cc: dave.hansen, kai.huang, haitao.huang, jarkko, Thomas Gleixner, Ingo Molnar, Borislav Petkov, x86, H. Peter Anvin From: "jarkko@kernel.org" <jarkko@kernel.org> Encapsulate the snippet in sgx_free_epc_page() concerning EREMOVE to sgx_reset_epc_page(), which is a static helper function for sgx_encl_release(). It's the only function existing, which deals with initialized pages. Signed-off-by: jarkko@kernel.org <jarkko@kernel.org> --- v2: Fixed a compilation error s/page/epc_page/ change not amended to the patches. arch/x86/kernel/cpu/sgx/encl.c | 15 ++++++++++++++- arch/x86/kernel/cpu/sgx/main.c | 10 ++++------ 2 files changed, 18 insertions(+), 7 deletions(-) diff --git a/arch/x86/kernel/cpu/sgx/encl.c b/arch/x86/kernel/cpu/sgx/encl.c index fe7256db6e73..511f4bc70699 100644 --- a/arch/x86/kernel/cpu/sgx/encl.c +++ b/arch/x86/kernel/cpu/sgx/encl.c @@ -389,6 +389,16 @@ const struct vm_operations_struct sgx_vm_ops = { .access = sgx_vma_access, }; + +static void sgx_reset_epc_page(struct sgx_epc_page *epc_page) +{ + int ret; + + ret = __eremove(sgx_get_epc_virt_addr(epc_page)); + if (WARN_ONCE(ret, "EREMOVE returned %d (0x%x)", ret, ret)) + return; +} + /** * sgx_encl_release - Destroy an enclave instance * @kref: address of a kref inside &sgx_encl @@ -412,6 +422,7 @@ void sgx_encl_release(struct kref *ref) if (sgx_unmark_page_reclaimable(entry->epc_page)) continue; + sgx_reset_epc_page(entry->epc_page); sgx_free_epc_page(entry->epc_page); encl->secs_child_cnt--; entry->epc_page = NULL; @@ -423,6 +434,7 @@ void sgx_encl_release(struct kref *ref) xa_destroy(&encl->page_array); if (!encl->secs_child_cnt && encl->secs.epc_page) { + sgx_reset_epc_page(encl->secs.epc_page); sgx_free_epc_page(encl->secs.epc_page); encl->secs.epc_page = NULL; } @@ -431,7 +443,8 @@ void sgx_encl_release(struct kref *ref) va_page = list_first_entry(&encl->va_pages, struct sgx_va_page, list); list_del(&va_page->list); - sgx_free_epc_page(va_page->epc_page); + sgx_reset_epc_page(entry->epc_page); + sgx_free_epc_page(entry->epc_page); kfree(va_page); } diff --git a/arch/x86/kernel/cpu/sgx/main.c b/arch/x86/kernel/cpu/sgx/main.c index b2996cb63a8a..89d52fb56d4b 100644 --- a/arch/x86/kernel/cpu/sgx/main.c +++ b/arch/x86/kernel/cpu/sgx/main.c @@ -629,16 +629,14 @@ struct sgx_epc_page *sgx_alloc_epc_page(void *owner, bool reclaim) * sgx_free_epc_page() - Free an EPC page * @page: an EPC page * - * Call EREMOVE for an EPC page and insert it back to the list of free pages. + * Put the EPC page back to the list of free pages. It's the callers + * responsibility to make sure that the page is in uninitialized state In other + * words, do EREMOVE, EWB or whatever operation is necessary before calling + * this function. */ void sgx_free_epc_page(struct sgx_epc_page *page) { struct sgx_epc_section *section = &sgx_epc_sections[page->section]; - int ret; - - ret = __eremove(sgx_get_epc_virt_addr(page)); - if (WARN_ONCE(ret, "EREMOVE returned %d (0x%x)", ret, ret)) - return; spin_lock(§ion->lock); list_add_tail(&page->list, §ion->page_list); -- 2.29.2 ^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCH v2 2/2] x86/sgx: Wipe out EREMOVE from sgx_free_epc_page() 2021-01-12 23:33 ` [PATCH v2 2/2] x86/sgx: Wipe out EREMOVE " jarkko @ 2021-01-13 1:09 ` Kai Huang 2021-01-13 21:08 ` Jarkko Sakkinen 0 siblings, 1 reply; 6+ messages in thread From: Kai Huang @ 2021-01-13 1:09 UTC (permalink / raw) To: jarkko Cc: linux-sgx, dave.hansen, haitao.huang, Thomas Gleixner, Ingo Molnar, Borislav Petkov, x86, H. Peter Anvin On Wed, 13 Jan 2021 01:33:51 +0200 jarkko@kernel.org wrote: > From: "jarkko@kernel.org" <jarkko@kernel.org> > > Encapsulate the snippet in sgx_free_epc_page() concerning EREMOVE to > sgx_reset_epc_page(), which is a static helper function for > sgx_encl_release(). It's the only function existing, which deals with > initialized pages. > > Signed-off-by: jarkko@kernel.org <jarkko@kernel.org> > --- > v2: Fixed a compilation error s/page/epc_page/ change not amended > to the patches. > arch/x86/kernel/cpu/sgx/encl.c | 15 ++++++++++++++- > arch/x86/kernel/cpu/sgx/main.c | 10 ++++------ > 2 files changed, 18 insertions(+), 7 deletions(-) > > diff --git a/arch/x86/kernel/cpu/sgx/encl.c b/arch/x86/kernel/cpu/sgx/encl.c > index fe7256db6e73..511f4bc70699 100644 > --- a/arch/x86/kernel/cpu/sgx/encl.c > +++ b/arch/x86/kernel/cpu/sgx/encl.c > @@ -389,6 +389,16 @@ const struct vm_operations_struct sgx_vm_ops = { > .access = sgx_vma_access, > }; > > + > +static void sgx_reset_epc_page(struct sgx_epc_page *epc_page) > +{ > + int ret; > + > + ret = __eremove(sgx_get_epc_virt_addr(epc_page)); > + if (WARN_ONCE(ret, "EREMOVE returned %d (0x%x)", ret, ret)) > + return; > +} > + > /** > * sgx_encl_release - Destroy an enclave instance > * @kref: address of a kref inside &sgx_encl > @@ -412,6 +422,7 @@ void sgx_encl_release(struct kref *ref) > if (sgx_unmark_page_reclaimable(entry->epc_page)) > continue; > > + sgx_reset_epc_page(entry->epc_page); > sgx_free_epc_page(entry->epc_page); > encl->secs_child_cnt--; > entry->epc_page = NULL; > @@ -423,6 +434,7 @@ void sgx_encl_release(struct kref *ref) > xa_destroy(&encl->page_array); > > if (!encl->secs_child_cnt && encl->secs.epc_page) { > + sgx_reset_epc_page(encl->secs.epc_page); > sgx_free_epc_page(encl->secs.epc_page); > encl->secs.epc_page = NULL; > } > @@ -431,7 +443,8 @@ void sgx_encl_release(struct kref *ref) > va_page = list_first_entry(&encl->va_pages, struct sgx_va_page, > list); > list_del(&va_page->list); > - sgx_free_epc_page(va_page->epc_page); > + sgx_reset_epc_page(entry->epc_page); > + sgx_free_epc_page(entry->epc_page); Reply so that Sean can see, since he just subscribed to linux-sgx list. As Sean pointed out in KVM SGX virtualization thread, entry should be va_page. > kfree(va_page); > } > > diff --git a/arch/x86/kernel/cpu/sgx/main.c b/arch/x86/kernel/cpu/sgx/main.c > index b2996cb63a8a..89d52fb56d4b 100644 > --- a/arch/x86/kernel/cpu/sgx/main.c > +++ b/arch/x86/kernel/cpu/sgx/main.c > @@ -629,16 +629,14 @@ struct sgx_epc_page *sgx_alloc_epc_page(void *owner, bool reclaim) > * sgx_free_epc_page() - Free an EPC page > * @page: an EPC page > * > - * Call EREMOVE for an EPC page and insert it back to the list of free pages. > + * Put the EPC page back to the list of free pages. It's the callers > + * responsibility to make sure that the page is in uninitialized state In other > + * words, do EREMOVE, EWB or whatever operation is necessary before calling > + * this function. > */ > void sgx_free_epc_page(struct sgx_epc_page *page) > { > struct sgx_epc_section *section = &sgx_epc_sections[page->section]; > - int ret; > - > - ret = __eremove(sgx_get_epc_virt_addr(page)); > - if (WARN_ONCE(ret, "EREMOVE returned %d (0x%x)", ret, ret)) > - return; > > spin_lock(§ion->lock); > list_add_tail(&page->list, §ion->page_list); > -- > 2.29.2 > ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH v2 2/2] x86/sgx: Wipe out EREMOVE from sgx_free_epc_page() 2021-01-13 1:09 ` Kai Huang @ 2021-01-13 21:08 ` Jarkko Sakkinen 2021-01-13 21:40 ` Kai Huang 0 siblings, 1 reply; 6+ messages in thread From: Jarkko Sakkinen @ 2021-01-13 21:08 UTC (permalink / raw) To: Kai Huang Cc: linux-sgx, dave.hansen, haitao.huang, Thomas Gleixner, Ingo Molnar, Borislav Petkov, x86, H. Peter Anvin On Wed, Jan 13, 2021 at 02:09:17PM +1300, Kai Huang wrote: > On Wed, 13 Jan 2021 01:33:51 +0200 jarkko@kernel.org wrote: > > From: "jarkko@kernel.org" <jarkko@kernel.org> > > > > Encapsulate the snippet in sgx_free_epc_page() concerning EREMOVE to > > sgx_reset_epc_page(), which is a static helper function for > > sgx_encl_release(). It's the only function existing, which deals with > > initialized pages. > > > > Signed-off-by: jarkko@kernel.org <jarkko@kernel.org> > > --- > > v2: Fixed a compilation error s/page/epc_page/ change not amended > > to the patches. > > arch/x86/kernel/cpu/sgx/encl.c | 15 ++++++++++++++- > > arch/x86/kernel/cpu/sgx/main.c | 10 ++++------ > > 2 files changed, 18 insertions(+), 7 deletions(-) > > > > diff --git a/arch/x86/kernel/cpu/sgx/encl.c b/arch/x86/kernel/cpu/sgx/encl.c > > index fe7256db6e73..511f4bc70699 100644 > > --- a/arch/x86/kernel/cpu/sgx/encl.c > > +++ b/arch/x86/kernel/cpu/sgx/encl.c > > @@ -389,6 +389,16 @@ const struct vm_operations_struct sgx_vm_ops = { > > .access = sgx_vma_access, > > }; > > > > + > > +static void sgx_reset_epc_page(struct sgx_epc_page *epc_page) > > +{ > > + int ret; > > + > > + ret = __eremove(sgx_get_epc_virt_addr(epc_page)); > > + if (WARN_ONCE(ret, "EREMOVE returned %d (0x%x)", ret, ret)) > > + return; > > +} > > + > > /** > > * sgx_encl_release - Destroy an enclave instance > > * @kref: address of a kref inside &sgx_encl > > @@ -412,6 +422,7 @@ void sgx_encl_release(struct kref *ref) > > if (sgx_unmark_page_reclaimable(entry->epc_page)) > > continue; > > > > + sgx_reset_epc_page(entry->epc_page); > > sgx_free_epc_page(entry->epc_page); > > encl->secs_child_cnt--; > > entry->epc_page = NULL; > > @@ -423,6 +434,7 @@ void sgx_encl_release(struct kref *ref) > > xa_destroy(&encl->page_array); > > > > if (!encl->secs_child_cnt && encl->secs.epc_page) { > > + sgx_reset_epc_page(encl->secs.epc_page); > > sgx_free_epc_page(encl->secs.epc_page); > > encl->secs.epc_page = NULL; > > } > > @@ -431,7 +443,8 @@ void sgx_encl_release(struct kref *ref) > > va_page = list_first_entry(&encl->va_pages, struct sgx_va_page, > > list); > > list_del(&va_page->list); > > - sgx_free_epc_page(va_page->epc_page); > > + sgx_reset_epc_page(entry->epc_page); > > + sgx_free_epc_page(entry->epc_page); > > Reply so that Sean can see, since he just subscribed to linux-sgx list. > > As Sean pointed out in KVM SGX virtualization thread, entry should be va_page. Thank, noticed, and responded. I'm sending v3. I think it would be best to then include that version to KVM patch set, as there is no such a rush to get my patches to upstream. I already dropped them from my bleeding edge numa branch. /Jarkko ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH v2 2/2] x86/sgx: Wipe out EREMOVE from sgx_free_epc_page() 2021-01-13 21:08 ` Jarkko Sakkinen @ 2021-01-13 21:40 ` Kai Huang 0 siblings, 0 replies; 6+ messages in thread From: Kai Huang @ 2021-01-13 21:40 UTC (permalink / raw) To: Jarkko Sakkinen Cc: linux-sgx, dave.hansen, haitao.huang, Thomas Gleixner, Ingo Molnar, Borislav Petkov, x86, H. Peter Anvin On Wed, 13 Jan 2021 23:08:08 +0200 Jarkko Sakkinen wrote: > On Wed, Jan 13, 2021 at 02:09:17PM +1300, Kai Huang wrote: > > On Wed, 13 Jan 2021 01:33:51 +0200 jarkko@kernel.org wrote: > > > From: "jarkko@kernel.org" <jarkko@kernel.org> > > > > > > Encapsulate the snippet in sgx_free_epc_page() concerning EREMOVE to > > > sgx_reset_epc_page(), which is a static helper function for > > > sgx_encl_release(). It's the only function existing, which deals with > > > initialized pages. > > > > > > Signed-off-by: jarkko@kernel.org <jarkko@kernel.org> > > > --- > > > v2: Fixed a compilation error s/page/epc_page/ change not amended > > > to the patches. > > > arch/x86/kernel/cpu/sgx/encl.c | 15 ++++++++++++++- > > > arch/x86/kernel/cpu/sgx/main.c | 10 ++++------ > > > 2 files changed, 18 insertions(+), 7 deletions(-) > > > > > > diff --git a/arch/x86/kernel/cpu/sgx/encl.c b/arch/x86/kernel/cpu/sgx/encl.c > > > index fe7256db6e73..511f4bc70699 100644 > > > --- a/arch/x86/kernel/cpu/sgx/encl.c > > > +++ b/arch/x86/kernel/cpu/sgx/encl.c > > > @@ -389,6 +389,16 @@ const struct vm_operations_struct sgx_vm_ops = { > > > .access = sgx_vma_access, > > > }; > > > > > > + > > > +static void sgx_reset_epc_page(struct sgx_epc_page *epc_page) > > > +{ > > > + int ret; > > > + > > > + ret = __eremove(sgx_get_epc_virt_addr(epc_page)); > > > + if (WARN_ONCE(ret, "EREMOVE returned %d (0x%x)", ret, ret)) > > > + return; > > > +} > > > + > > > /** > > > * sgx_encl_release - Destroy an enclave instance > > > * @kref: address of a kref inside &sgx_encl > > > @@ -412,6 +422,7 @@ void sgx_encl_release(struct kref *ref) > > > if (sgx_unmark_page_reclaimable(entry->epc_page)) > > > continue; > > > > > > + sgx_reset_epc_page(entry->epc_page); > > > sgx_free_epc_page(entry->epc_page); > > > encl->secs_child_cnt--; > > > entry->epc_page = NULL; > > > @@ -423,6 +434,7 @@ void sgx_encl_release(struct kref *ref) > > > xa_destroy(&encl->page_array); > > > > > > if (!encl->secs_child_cnt && encl->secs.epc_page) { > > > + sgx_reset_epc_page(encl->secs.epc_page); > > > sgx_free_epc_page(encl->secs.epc_page); > > > encl->secs.epc_page = NULL; > > > } > > > @@ -431,7 +443,8 @@ void sgx_encl_release(struct kref *ref) > > > va_page = list_first_entry(&encl->va_pages, struct sgx_va_page, > > > list); > > > list_del(&va_page->list); > > > - sgx_free_epc_page(va_page->epc_page); > > > + sgx_reset_epc_page(entry->epc_page); > > > + sgx_free_epc_page(entry->epc_page); > > > > Reply so that Sean can see, since he just subscribed to linux-sgx list. > > > > As Sean pointed out in KVM SGX virtualization thread, entry should be va_page. > > Thank, noticed, and responded. I'm sending v3. I think it would be best to > then include that version to KVM patch set, as there is no such a rush to > get my patches to upstream. I already dropped them from my bleeding edge > numa branch. Sure. I'll take your patch once I see it. Thanks Jarkko. ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2021-01-14 2:18 UTC | newest] Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2021-01-12 23:33 [PATCH v2 0/2] Take away EREMOVE from sgx_free_epc_page() jarkko 2021-01-12 23:33 ` [PATCH v2 1/2] x86/sgx: Remove a warn " jarkko 2021-01-12 23:33 ` [PATCH v2 2/2] x86/sgx: Wipe out EREMOVE " jarkko 2021-01-13 1:09 ` Kai Huang 2021-01-13 21:08 ` Jarkko Sakkinen 2021-01-13 21:40 ` Kai Huang
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).