From: Jarkko Sakkinen <jarkko@kernel.org>
To: Kai Huang <kai.huang@intel.com>
Cc: Dave Hansen <dave.hansen@intel.com>,
linux-sgx@vger.kernel.org,
Dave Hansen <dave.hansen@linux.intel.com>,
Thomas Gleixner <tglx@linutronix.de>,
Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>,
x86@kernel.org, "H. Peter Anvin" <hpa@zytor.com>,
linux-kernel@vger.kernel.org
Subject: Re: [PATCH v3 2/5] x86/sgx: Use sgx_free_epc_page() in sgx_reclaim_pages()
Date: Thu, 11 Mar 2021 00:12:17 +0200 [thread overview]
Message-ID: <YElEQXZlzdrElovv@kernel.org> (raw)
In-Reply-To: <YElD7orORGElfMdZ@kernel.org>
On Thu, Mar 11, 2021 at 12:10:56AM +0200, Jarkko Sakkinen wrote:
> On Thu, Mar 11, 2021 at 09:36:15AM +1300, Kai Huang wrote:
> > On Wed, 2021-03-10 at 17:11 +0200, Jarkko Sakkinen wrote:
> > > On Wed, Mar 03, 2021 at 08:59:17AM -0800, Dave Hansen wrote:
> > > > On 3/3/21 7:03 AM, Jarkko Sakkinen wrote:
> > > > > diff --git a/arch/x86/kernel/cpu/sgx/main.c b/arch/x86/kernel/cpu/sgx/main.c
> > > > > index 52d070fb4c9a..ed99c60024dc 100644
> > > > > --- a/arch/x86/kernel/cpu/sgx/main.c
> > > > > +++ b/arch/x86/kernel/cpu/sgx/main.c
> > > > > @@ -305,7 +305,6 @@ static void sgx_reclaim_pages(void)
> > > > > {
> > > > > struct sgx_epc_page *chunk[SGX_NR_TO_SCAN];
> > > > > struct sgx_backing backing[SGX_NR_TO_SCAN];
> > > > > - struct sgx_epc_section *section;
> > > > > struct sgx_encl_page *encl_page;
> > > > > struct sgx_epc_page *epc_page;
> > > > > pgoff_t page_index;
> > > > > @@ -378,11 +377,7 @@ static void sgx_reclaim_pages(void)
> > > > > kref_put(&encl_page->encl->refcount, sgx_encl_release);
> > > > > epc_page->flags &= ~SGX_EPC_PAGE_RECLAIMER_TRACKED;
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > - section = &sgx_epc_sections[epc_page->section];
> > > > > - spin_lock(§ion->lock);
> > > > > - list_add_tail(&epc_page->list, §ion->page_list);
> > > > > - section->free_cnt++;
> > > > > - spin_unlock(§ion->lock);
> > > > > + sgx_free_epc_page(epc_page);
> > > > > }
> > > > > }
> > > >
> > > > In current upstream (3fb6d0e00e), sgx_free_epc_page() calls __eremove().
> > > > This code does not call __eremove(). That seems to be changing
> > > > behavior where none was intended.
> > >
> > > EREMOVE does not matter here, as it doesn't in almost all most of the sites
> > > where sgx_free_epc_page() is used in the driver. It does nothing to an
> > > uninitialized pages.
> >
> > Right. EREMOVE on uninitialized pages does nothing, so a more reasonable way is to
> > just NOT call EREMOVE (your original code), since it is absolutely unnecessary.
> >
> > I don't see ANY reason we should call EREMOVE here.
> >
> > Actually w/o my patch to split EREMOVE out of sgx_free_epc_page(), it then makes
> > perfect sense to have new sgx_free_epc_page() here.
> >
> > >
> > > The two patches that I posted originally for Kai's series took EREMOVE out
> > > of sgx_free_epc_page() and put an explicit EREMOVE where it is actually
> > > needed, but for reasons unknown to me, that change is gone.
> > >
> >
> > It's not gone. It goes into a new sgx_encl_free_epc_page(), which is exactly the same
> > as current sgx_free_epc_page() which as EREMOVE, instead of putting EREMOVE into a
> > dedicated sgx_reset_epc_page(), as you did in your series:
> >
> > https://lore.kernel.org/linux-sgx/20210113233541.17669-1-jarkko@kernel.org/
> >
> > However, your change has side effort: it always put page back into free pool, even
> > EREMOVE fails. To make your change w/o having any functional change, it has to be:
> >
> > if(!sgx_reset_epc_page())
> > sgx_free_epc_page();
>
> OK, great, your patch set uses the wrapper only in the necessary call
> sites. Sorry, I overlooked this part.
>
> Anyway, it knowingly does that. I considered either as equally harmful
> side-ffects when I implemented. Either can only trigger, when there is a
> bug in the kernel code.
>
> It *could* do what that snippet suggest but it's like "out of the frying pan,
> into the fire" kind of change.
>
> Since NUMA patch set anyway requires to have a global dirty list, I think
> the better way to deal with this, would be to declare a new global in the
> patch under discussion:
>
> static struct list_head sgx_dirty_list;
sgx_dirty_page_list
>
> And sgx_encl_free_epc_page() could simply put the pages in this list. In
> some cases you could possibly even reset the system state using kexec for
> debugging purposes, so it could potentially bring a tiny bit of value.
>
> I can rebase then my NUMA patches on top of SGX specific KVM patches, once
> Boris have applied them.
>
> > And for this, Dave raised one concern we should add a WARN() to let user know EPC
> > page is leaked, and reboot is requied to get them back.
> >
> > However with sgx_reset_epc_page(), there's no place to add such WARN(), and
> > implementing original sgx_free_epc_page() as sgx_encl_free_epc_page() looks very
> > reasonable to me:
> >
> > https://www.spinics.net/lists/linux-sgx/msg04631.html
>
> /Jarkko
/Jarkko
next prev parent reply other threads:[~2021-03-10 22:13 UTC|newest]
Thread overview: 27+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <20210303150323.433207-1-jarkko@kernel.org>
2021-03-03 15:03 ` [PATCH v3 1/5] x86/sgx: Fix a resource leak in sgx_init() Jarkko Sakkinen
2021-03-03 16:56 ` Dave Hansen
2021-03-10 15:00 ` Jarkko Sakkinen
2021-03-10 15:49 ` Sean Christopherson
2021-03-10 21:52 ` Jarkko Sakkinen
2021-03-03 15:03 ` [PATCH v3 2/5] x86/sgx: Use sgx_free_epc_page() in sgx_reclaim_pages() Jarkko Sakkinen
2021-03-03 16:59 ` Dave Hansen
2021-03-10 15:11 ` Jarkko Sakkinen
2021-03-10 15:55 ` Dave Hansen
2021-03-10 21:56 ` Jarkko Sakkinen
2021-03-10 20:36 ` Kai Huang
2021-03-10 22:10 ` Jarkko Sakkinen
2021-03-10 22:12 ` Jarkko Sakkinen [this message]
2021-03-10 22:35 ` Jarkko Sakkinen
2021-03-10 22:43 ` Kai Huang
2021-03-10 22:52 ` Kai Huang
2021-03-03 15:03 ` [PATCH v3 3/5] x86/sgx: Replace section->init_laundry_list with a temp list Jarkko Sakkinen
2021-03-03 18:02 ` Dave Hansen
2021-03-10 14:50 ` Jarkko Sakkinen
2021-03-03 15:03 ` [PATCH v3 4/5] x86/sgx: Replace section->page_list with a global free page list Jarkko Sakkinen
2021-03-03 23:48 ` Dave Hansen
2021-03-10 10:54 ` Jarkko Sakkinen
2021-03-03 15:03 ` [PATCH v3 5/5] x86/sgx: Add a basic NUMA allocation scheme to sgx_alloc_epc_page() Jarkko Sakkinen
2021-03-04 0:20 ` Dave Hansen
2021-03-10 11:30 ` Jarkko Sakkinen
2021-03-10 15:44 ` Dave Hansen
2021-03-10 21:48 ` Jarkko Sakkinen
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=YElEQXZlzdrElovv@kernel.org \
--to=jarkko@kernel.org \
--cc=bp@alien8.de \
--cc=dave.hansen@intel.com \
--cc=dave.hansen@linux.intel.com \
--cc=hpa@zytor.com \
--cc=kai.huang@intel.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-sgx@vger.kernel.org \
--cc=mingo@redhat.com \
--cc=tglx@linutronix.de \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).