From: Jarkko Sakkinen <jarkko@kernel.org>
To: Tony Luck <tony.luck@intel.com>
Cc: Dave Hansen <dave.hansen@linux.intel.com>,
x86@kernel.org, linux-sgx@vger.kernel.org,
linux-kernel@vger.kernel.org
Subject: Re: [PATCH 1/4] x86/sgx: Track phase and type of SGX EPC pages
Date: Fri, 9 Jul 2021 21:09:44 +0300 [thread overview]
Message-ID: <20210709180944.46fr74atwkghmw7a@kernel.org> (raw)
In-Reply-To: <20210709180800.kpjlrvbljaknuncq@kernel.org>
On Fri, Jul 09, 2021 at 09:08:03PM +0300, Jarkko Sakkinen wrote:
> On Thu, Jul 08, 2021 at 11:14:20AM -0700, Tony Luck wrote:
> > Memory errors can be reported either synchronously as memory is accessed,
> > or asynchronously by speculative access or by a memory controller page
> > scrubber. The life cycle of an EPC page takes it through:
> > dirty -> free -> in-use -> free.
> >
> > Memory errors are reported using physical addresses. It is a simple
> > matter to find which sgx_epc_page structure maps a given address.
> > But then recovery code needs to be able to determine the current use of
> > the page to take the appropriate recovery action. Within the "in-use"
> > phase different actions are needed based on how the page is used in
> > the enclave.
> >
> > Add new flags bits to describe the phase (with an extra bit for the new
> > phase of "poisoned"). Drop pages marked as poisoned instead of adding
> > them to a free list to make sure they are not re-used.
> >
> > Add a type field to struct epc_page for how an in-use page has been
> > allocated. Re-use "enum sgx_page_type" for this type, with a couple
> > of additions for s/w types.
> >
> > Signed-off-by: Tony Luck <tony.luck@intel.com>
> > ---
> > arch/x86/include/asm/sgx.h | 6 ++++++
> > arch/x86/kernel/cpu/sgx/encl.c | 4 ++--
> > arch/x86/kernel/cpu/sgx/ioctl.c | 4 ++--
> > arch/x86/kernel/cpu/sgx/main.c | 21 +++++++++++++++++++--
> > arch/x86/kernel/cpu/sgx/sgx.h | 14 ++++++++++++--
> > arch/x86/kernel/cpu/sgx/virt.c | 2 +-
> > 6 files changed, 42 insertions(+), 9 deletions(-)
> >
> > diff --git a/arch/x86/include/asm/sgx.h b/arch/x86/include/asm/sgx.h
> > index 9c31e0ebc55b..9619a6d77a83 100644
> > --- a/arch/x86/include/asm/sgx.h
> > +++ b/arch/x86/include/asm/sgx.h
> > @@ -216,6 +216,8 @@ struct sgx_pageinfo {
> > * %SGX_PAGE_TYPE_REG: a regular page
> > * %SGX_PAGE_TYPE_VA: a VA page
> > * %SGX_PAGE_TYPE_TRIM: a page in trimmed state
> > + *
> > + * Also used to track current use of &struct sgx_epc_page
> > */
> > enum sgx_page_type {
> > SGX_PAGE_TYPE_SECS,
> > @@ -223,6 +225,10 @@ enum sgx_page_type {
> > SGX_PAGE_TYPE_REG,
> > SGX_PAGE_TYPE_VA,
> > SGX_PAGE_TYPE_TRIM,
> > +
> > + /* sgx_epc_page.type */
> > + SGX_PAGE_TYPE_FREE = 100,
> > + SGX_PAGE_TYPE_KVM = 101,
> > };
> >
> > #define SGX_NR_PAGE_TYPES 5
> > diff --git a/arch/x86/kernel/cpu/sgx/encl.c b/arch/x86/kernel/cpu/sgx/encl.c
> > index 3be203297988..abf6e1a704c0 100644
> > --- a/arch/x86/kernel/cpu/sgx/encl.c
> > +++ b/arch/x86/kernel/cpu/sgx/encl.c
> > @@ -72,7 +72,7 @@ static struct sgx_epc_page *sgx_encl_eldu(struct sgx_encl_page *encl_page,
> > struct sgx_epc_page *epc_page;
> > int ret;
> >
> > - epc_page = sgx_alloc_epc_page(encl_page, false);
> > + epc_page = sgx_alloc_epc_page(encl_page, SGX_PAGE_TYPE_REG, false);
> > if (IS_ERR(epc_page))
> > return epc_page;
> >
> > @@ -679,7 +679,7 @@ struct sgx_epc_page *sgx_alloc_va_page(void)
> > struct sgx_epc_page *epc_page;
> > int ret;
> >
> > - epc_page = sgx_alloc_epc_page(NULL, true);
> > + epc_page = sgx_alloc_epc_page(NULL, SGX_PAGE_TYPE_VA, true);
> > if (IS_ERR(epc_page))
> > return ERR_CAST(epc_page);
> >
> > diff --git a/arch/x86/kernel/cpu/sgx/ioctl.c b/arch/x86/kernel/cpu/sgx/ioctl.c
> > index 83df20e3e633..a74ae00194cc 100644
> > --- a/arch/x86/kernel/cpu/sgx/ioctl.c
> > +++ b/arch/x86/kernel/cpu/sgx/ioctl.c
> > @@ -83,7 +83,7 @@ static int sgx_encl_create(struct sgx_encl *encl, struct sgx_secs *secs)
> >
> > encl->backing = backing;
> >
> > - secs_epc = sgx_alloc_epc_page(&encl->secs, true);
> > + secs_epc = sgx_alloc_epc_page(&encl->secs, SGX_PAGE_TYPE_SECS, true);
> > if (IS_ERR(secs_epc)) {
> > ret = PTR_ERR(secs_epc);
> > goto err_out_backing;
> > @@ -300,7 +300,7 @@ static int sgx_encl_add_page(struct sgx_encl *encl, unsigned long src,
> > if (IS_ERR(encl_page))
> > return PTR_ERR(encl_page);
> >
> > - epc_page = sgx_alloc_epc_page(encl_page, true);
> > + epc_page = sgx_alloc_epc_page(encl_page, SGX_PAGE_TYPE_REG, true);
> > if (IS_ERR(epc_page)) {
> > kfree(encl_page);
> > return PTR_ERR(epc_page);
> > diff --git a/arch/x86/kernel/cpu/sgx/main.c b/arch/x86/kernel/cpu/sgx/main.c
> > index 63d3de02bbcc..643df87b3e01 100644
> > --- a/arch/x86/kernel/cpu/sgx/main.c
> > +++ b/arch/x86/kernel/cpu/sgx/main.c
> > @@ -401,7 +401,12 @@ static void sgx_reclaim_pages(void)
> > section = &sgx_epc_sections[epc_page->section];
> > node = section->node;
> >
> > + /* drop poison pages instead of adding to free list */
> > + if (epc_page->flags & SGX_EPC_PAGE_POISON)
> > + continue;
> > +
> > spin_lock(&node->lock);
> > + epc_page->flags = SGX_EPC_PAGE_FREE;
> > list_add_tail(&epc_page->list, &node->free_page_list);
> > sgx_nr_free_pages++;
> > spin_unlock(&node->lock);
> > @@ -560,6 +565,7 @@ int sgx_unmark_page_reclaimable(struct sgx_epc_page *page)
> > /**
> > * sgx_alloc_epc_page() - Allocate an EPC page
> > * @owner: the owner of the EPC page
> > + * @type: type of page being allocated
> > * @reclaim: reclaim pages if necessary
> > *
> > * Iterate through EPC sections and borrow a free EPC page to the caller. When a
> > @@ -574,7 +580,7 @@ int sgx_unmark_page_reclaimable(struct sgx_epc_page *page)
> > * an EPC page,
> > * -errno on error
> > */
> > -struct sgx_epc_page *sgx_alloc_epc_page(void *owner, bool reclaim)
> > +struct sgx_epc_page *sgx_alloc_epc_page(void *owner, enum sgx_page_type type, bool reclaim)
> > {
> > struct sgx_epc_page *page;
> >
> > @@ -582,6 +588,8 @@ struct sgx_epc_page *sgx_alloc_epc_page(void *owner, bool reclaim)
> > page = __sgx_alloc_epc_page();
> > if (!IS_ERR(page)) {
> > page->owner = owner;
> > + page->type = type;
> > + page->flags = 0;
> > break;
> > }
> >
> > @@ -616,14 +624,22 @@ struct sgx_epc_page *sgx_alloc_epc_page(void *owner, bool reclaim)
> > * responsibility to make sure that the page is in uninitialized state. In other
> > * words, do EREMOVE, EWB or whatever operation is necessary before calling
> > * this function.
> > + *
> > + * Note that if the page has been tagged as poisoned, it is simply
> > + * dropped on the floor instead of added to the free list to make
> > + * sure we do not re-use it.
> > */
> > void sgx_free_epc_page(struct sgx_epc_page *page)
> > {
> > struct sgx_epc_section *section = &sgx_epc_sections[page->section];
> > struct sgx_numa_node *node = section->node;
> >
> > + if (page->flags & SGX_EPC_PAGE_POISON)
> > + return;
>
> I tend to think that it would be nice to collect them somewhere instead
> purposely leaking. E.g. this gives possibility to examine list with
> debugging tools.
I'm not also sure why free and dirty pages need to be tagged. Why a
poison flag is enough? This could be better explained in the commit
message.
next prev parent reply other threads:[~2021-07-09 18:09 UTC|newest]
Thread overview: 99+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-07-08 18:14 [PATCH 0/4] Basic recovery for machine checks inside SGX Tony Luck
2021-07-08 18:14 ` [PATCH 1/4] x86/sgx: Track phase and type of SGX EPC pages Tony Luck
2021-07-09 18:08 ` Jarkko Sakkinen
2021-07-09 18:09 ` Jarkko Sakkinen [this message]
2021-07-14 20:42 ` Reinette Chatre
2021-07-14 20:59 ` Luck, Tony
2021-07-14 21:21 ` Reinette Chatre
2021-07-14 23:08 ` Sean Christopherson
2021-07-14 23:39 ` Luck, Tony
2021-07-15 15:33 ` Sean Christopherson
2021-07-08 18:14 ` [PATCH 2/4] x86/sgx: Add basic infrastructure to recover from errors in SGX memory Tony Luck
2021-07-08 18:14 ` [PATCH 3/4] x86/sgx: Hook sgx_memory_failure() into mainline code Tony Luck
2021-07-08 18:14 ` [PATCH 4/4] x86/sgx: Add hook to error injection address validation Tony Luck
2021-07-19 18:20 ` [PATCH v2 0/6] Basic recovery for machine checks inside SGX Tony Luck
2021-07-19 18:20 ` [PATCH v2 1/6] x86/sgx: Provide indication of life-cycle of EPC pages Tony Luck
2021-07-19 18:28 ` Dave Hansen
2021-07-27 2:04 ` Sakkinen, Jarkko
2021-07-19 18:20 ` [PATCH v2 2/6] x86/sgx: Add infrastructure to identify SGX " Tony Luck
2021-07-19 18:20 ` [PATCH v2 3/6] x86/sgx: Initial poison handling for dirty and free pages Tony Luck
2021-07-27 2:08 ` Sakkinen, Jarkko
2021-07-19 18:20 ` [PATCH v2 4/6] x86/sgx: Add SGX infrastructure to recover from poison Tony Luck
2021-07-19 18:20 ` [PATCH v2 5/6] x86/sgx: Hook sgx_memory_failure() into mainline code Tony Luck
2021-07-19 18:20 ` [PATCH v2 6/6] x86/sgx: Add hook to error injection address validation Tony Luck
2021-07-27 1:54 ` [PATCH v2 0/6] Basic recovery for machine checks inside SGX Sakkinen, Jarkko
2021-07-28 20:46 ` [PATCH v3 0/7] " Tony Luck
2021-07-28 20:46 ` [PATCH v3 1/7] x86/sgx: Provide indication of life-cycle of EPC pages Tony Luck
2021-07-28 22:12 ` Dave Hansen
2021-07-28 22:57 ` Luck, Tony
2021-07-28 23:12 ` Dave Hansen
2021-07-28 23:32 ` Sean Christopherson
2021-07-28 23:48 ` Luck, Tony
2021-07-29 0:07 ` Sean Christopherson
2021-07-29 0:42 ` Luck, Tony
2021-07-30 0:34 ` Jarkko Sakkinen
2021-07-30 0:33 ` Jarkko Sakkinen
2021-07-28 20:46 ` [PATCH v3 2/7] x86/sgx: Add infrastructure to identify SGX " Tony Luck
2021-07-28 22:19 ` Dave Hansen
2021-07-30 0:38 ` Jarkko Sakkinen
2021-07-30 16:46 ` Sean Christopherson
2021-07-30 16:50 ` Dave Hansen
2021-07-30 18:44 ` Luck, Tony
2021-07-30 20:35 ` Dave Hansen
2021-07-30 23:35 ` Luck, Tony
2021-08-03 21:34 ` Matthew Wilcox
2021-08-03 23:49 ` Luck, Tony
2021-08-02 8:52 ` Jarkko Sakkinen
2021-08-02 8:51 ` Jarkko Sakkinen
2021-08-02 8:48 ` Jarkko Sakkinen
2021-07-28 20:46 ` [PATCH v3 3/7] x86/sgx: Initial poison handling for dirty and free pages Tony Luck
2021-07-30 0:42 ` Jarkko Sakkinen
2021-07-28 20:46 ` [PATCH v3 4/7] x86/sgx: Add SGX infrastructure to recover from poison Tony Luck
2021-07-28 22:29 ` Dave Hansen
2021-07-28 23:00 ` Sean Christopherson
2021-07-28 20:46 ` [PATCH v3 5/7] x86/sgx: Hook sgx_memory_failure() into mainline code Tony Luck
2021-07-28 20:46 ` [PATCH v3 6/7] x86/sgx: Add hook to error injection address validation Tony Luck
2021-07-28 20:46 ` [PATCH v3 7/7] x86/sgx: Add documentation for SGX memory errors Tony Luck
2021-08-27 19:55 ` [PATCH v4 0/6] Basic recovery for machine checks inside SGX Tony Luck
2021-08-27 19:55 ` [PATCH v4 1/6] x86/sgx: Provide indication of life-cycle of EPC pages Tony Luck
2021-09-01 3:55 ` Jarkko Sakkinen
2021-08-27 19:55 ` [PATCH v4 2/6] x86/sgx: Add infrastructure to identify SGX " Tony Luck
2021-09-01 4:30 ` Jarkko Sakkinen
2021-08-27 19:55 ` [PATCH v4 3/6] x86/sgx: Initial poison handling for dirty and free pages Tony Luck
2021-08-27 19:55 ` [PATCH v4 4/6] x86/sgx: Add SGX infrastructure to recover from poison Tony Luck
2021-08-27 19:55 ` [PATCH v4 5/6] x86/sgx: Hook sgx_memory_failure() into mainline code Tony Luck
2021-09-03 6:12 ` Jarkko Sakkinen
2021-09-03 6:56 ` Jarkko Sakkinen
2021-09-06 18:51 ` Luck, Tony
2021-09-07 14:07 ` Jarkko Sakkinen
2021-09-07 14:13 ` Dave Hansen
2021-09-07 15:07 ` Luck, Tony
2021-09-07 15:03 ` Luck, Tony
2021-09-07 15:08 ` Jarkko Sakkinen
2021-09-07 17:46 ` Luck, Tony
2021-09-08 0:59 ` Luck, Tony
2021-09-08 16:49 ` Dave Hansen
2021-09-08 2:29 ` Jarkko Sakkinen
2021-08-27 19:55 ` [PATCH v4 6/6] x86/sgx: Add hook to error injection address validation Tony Luck
2021-08-27 20:28 ` [PATCH v4 0/6] Basic recovery for machine checks inside SGX Borislav Petkov
2021-08-27 20:43 ` Sean Christopherson
2021-09-01 2:06 ` Jarkko Sakkinen
2021-09-01 14:48 ` Luck, Tony
2021-09-17 21:38 ` [PATCH v5 0/7] " Tony Luck
2021-09-17 21:38 ` [PATCH v5 1/7] x86/sgx: Provide indication of life-cycle of EPC pages Tony Luck
2021-09-21 21:28 ` Jarkko Sakkinen
2021-09-21 21:34 ` Luck, Tony
2021-09-22 5:17 ` Jarkko Sakkinen
2021-09-21 22:15 ` Dave Hansen
2021-09-22 5:27 ` Jarkko Sakkinen
2021-09-17 21:38 ` [PATCH v5 2/7] x86/sgx: Add infrastructure to identify SGX " Tony Luck
2021-09-21 20:23 ` Dave Hansen
2021-09-21 20:50 ` Luck, Tony
2021-09-21 22:32 ` Dave Hansen
2021-09-21 23:48 ` Luck, Tony
2021-09-21 23:50 ` Dave Hansen
2021-09-17 21:38 ` [PATCH v5 3/7] x86/sgx: Initial poison handling for dirty and free pages Tony Luck
2021-09-17 21:38 ` [PATCH v5 4/7] x86/sgx: Add SGX infrastructure to recover from poison Tony Luck
2021-09-17 21:38 ` [PATCH v5 5/7] x86/sgx: Hook arch_memory_failure() into mainline code Tony Luck
2021-09-17 21:38 ` [PATCH v5 6/7] x86/sgx: Add hook to error injection address validation Tony Luck
2021-09-17 21:38 ` [PATCH v5 7/7] x86/sgx: Add check for SGX pages to ghes_do_memory_failure() Tony Luck
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210709180944.46fr74atwkghmw7a@kernel.org \
--to=jarkko@kernel.org \
--cc=dave.hansen@linux.intel.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-sgx@vger.kernel.org \
--cc=tony.luck@intel.com \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).