From: Jarkko Sakkinen <jarkko.sakkinen@linux.intel.com>
To: Greg KH <gregkh@linuxfoundation.org>
Cc: x86@kernel.org, linux-sgx@vger.kernel.org,
linux-kernel@vger.kernel.org,
linux-security-module@vger.kernel.org, linux-mm@kvack.org,
Andrew Morton <akpm@linux-foundation.org>,
Matthew Wilcox <willy@infradead.org>,
Jethro Beekman <jethro@fortanix.com>,
Haitao Huang <haitao.huang@linux.intel.com>,
Chunyang Hui <sanqian.hcy@antfin.com>,
Jordan Hand <jorhand@linux.microsoft.com>,
Nathaniel McCallum <npmccallum@redhat.com>,
Seth Moore <sethmo@google.com>,
Darren Kenny <darren.kenny@oracle.com>,
Sean Christopherson <sean.j.christopherson@intel.com>,
Suresh Siddha <suresh.b.siddha@intel.com>,
andriy.shevchenko@linux.intel.com, asapek@google.com,
bp@alien8.de, cedric.xing@intel.com, chenalexchen@google.com,
conradparker@google.com, cyhanish@google.com,
dave.hansen@intel.com, haitao.huang@intel.com,
kai.huang@intel.com, kai.svahn@intel.com, kmoy@google.com,
ludloff@google.com, luto@kernel.org, nhorman@redhat.com,
puiterwijk@redhat.com, rientjes@google.com, tglx@linutronix.de,
yaozhangx@google.com, mikko.ylinen@intel.com
Subject: Re: [PATCH v39 11/24] x86/sgx: Add SGX enclave driver
Date: Sun, 4 Oct 2020 17:32:46 +0300 [thread overview]
Message-ID: <20201004143246.GA3561@linux.intel.com> (raw)
In-Reply-To: <20201003143925.GB800720@kroah.com>
On Sat, Oct 03, 2020 at 04:39:25PM +0200, Greg KH wrote:
> On Sat, Oct 03, 2020 at 07:50:46AM +0300, Jarkko Sakkinen wrote:
> > Intel Software Guard eXtensions (SGX) is a set of CPU instructions that can
> > be used by applications to set aside private regions of code and data. The
> > code outside the SGX hosted software entity is prevented from accessing the
> > memory inside the enclave by the CPU. We call these entities enclaves.
> >
> > Add a driver that provides an ioctl API to construct and run enclaves.
> > Enclaves are constructed from pages residing in reserved physical memory
> > areas. The contents of these pages can only be accessed when they are
> > mapped as part of an enclave, by a hardware thread running inside the
> > enclave.
> >
> > The starting state of an enclave consists of a fixed measured set of
> > pages that are copied to the EPC during the construction process by
> > using the opcode ENCLS leaf functions and Software Enclave Control
> > Structure (SECS) that defines the enclave properties.
> >
> > Enclaves are constructed by using ENCLS leaf functions ECREATE, EADD and
> > EINIT. ECREATE initializes SECS, EADD copies pages from system memory to
> > the EPC and EINIT checks a given signed measurement and moves the enclave
> > into a state ready for execution.
> >
> > An initialized enclave can only be accessed through special Thread Control
> > Structure (TCS) pages by using ENCLU (ring-3 only) leaf EENTER. This leaf
> > function converts a thread into enclave mode and continues the execution in
> > the offset defined by the TCS provided to EENTER. An enclave is exited
> > through syscall, exception, interrupts or by explicitly calling another
> > ENCLU leaf EEXIT.
> >
> > The mmap() permissions are capped by the contained enclave page
> > permissions. The mapped areas must also be populated, i.e. each page
> > address must contain a page. This logic is implemented in
> > sgx_encl_may_map().
> >
> > Cc: linux-security-module@vger.kernel.org
> > Cc: linux-mm@kvack.org
> > Cc: Andrew Morton <akpm@linux-foundation.org>
> > Cc: Matthew Wilcox <willy@infradead.org>
> > Acked-by: Jethro Beekman <jethro@fortanix.com>
> > Tested-by: Jethro Beekman <jethro@fortanix.com>
> > Tested-by: Haitao Huang <haitao.huang@linux.intel.com>
> > Tested-by: Chunyang Hui <sanqian.hcy@antfin.com>
> > Tested-by: Jordan Hand <jorhand@linux.microsoft.com>
> > Tested-by: Nathaniel McCallum <npmccallum@redhat.com>
> > Tested-by: Seth Moore <sethmo@google.com>
> > Tested-by: Darren Kenny <darren.kenny@oracle.com>
> > Reviewed-by: Darren Kenny <darren.kenny@oracle.com>
> > Co-developed-by: Sean Christopherson <sean.j.christopherson@intel.com>
> > Signed-off-by: Sean Christopherson <sean.j.christopherson@intel.com>
> > Co-developed-by: Suresh Siddha <suresh.b.siddha@intel.com>
> > Signed-off-by: Suresh Siddha <suresh.b.siddha@intel.com>
> > Signed-off-by: Jarkko Sakkinen <jarkko.sakkinen@linux.intel.com>
> > ---
> > arch/x86/kernel/cpu/sgx/Makefile | 2 +
> > arch/x86/kernel/cpu/sgx/driver.c | 173 ++++++++++++++++
> > arch/x86/kernel/cpu/sgx/driver.h | 29 +++
> > arch/x86/kernel/cpu/sgx/encl.c | 331 +++++++++++++++++++++++++++++++
> > arch/x86/kernel/cpu/sgx/encl.h | 85 ++++++++
> > arch/x86/kernel/cpu/sgx/main.c | 11 +
> > 6 files changed, 631 insertions(+)
> > create mode 100644 arch/x86/kernel/cpu/sgx/driver.c
> > create mode 100644 arch/x86/kernel/cpu/sgx/driver.h
> > create mode 100644 arch/x86/kernel/cpu/sgx/encl.c
> > create mode 100644 arch/x86/kernel/cpu/sgx/encl.h
> >
> > diff --git a/arch/x86/kernel/cpu/sgx/Makefile b/arch/x86/kernel/cpu/sgx/Makefile
> > index 79510ce01b3b..3fc451120735 100644
> > --- a/arch/x86/kernel/cpu/sgx/Makefile
> > +++ b/arch/x86/kernel/cpu/sgx/Makefile
> > @@ -1,2 +1,4 @@
> > obj-y += \
> > + driver.o \
> > + encl.o \
> > main.o
> > diff --git a/arch/x86/kernel/cpu/sgx/driver.c b/arch/x86/kernel/cpu/sgx/driver.c
> > new file mode 100644
> > index 000000000000..f54da5f19c2b
> > --- /dev/null
> > +++ b/arch/x86/kernel/cpu/sgx/driver.c
> > @@ -0,0 +1,173 @@
> > +// SPDX-License-Identifier: (GPL-2.0 OR BSD-3-Clause)
>
> You use gpl-only header files in this file, so how in the world can it
> be bsd-3 licensed?
>
> Please get your legal department to agree with this, after you explain
> to them how you are mixing gpl2-only code in with this file.
I'll do what I already stated that I will do. Should I do something
more?
> > +// Copyright(c) 2016-18 Intel Corporation.
>
> Dates are hard to get right :(
Will fix.
>
> > +
> > +#include <linux/acpi.h>
> > +#include <linux/miscdevice.h>
> > +#include <linux/mman.h>
> > +#include <linux/security.h>
> > +#include <linux/suspend.h>
> > +#include <asm/traps.h>
> > +#include "driver.h"
> > +#include "encl.h"
> > +
> > +u64 sgx_encl_size_max_32;
> > +u64 sgx_encl_size_max_64;
> > +u32 sgx_misc_reserved_mask;
> > +u64 sgx_attributes_reserved_mask;
> > +u64 sgx_xfrm_reserved_mask = ~0x3;
> > +u32 sgx_xsave_size_tbl[64];
> > +
> > +static int sgx_open(struct inode *inode, struct file *file)
> > +{
> > + struct sgx_encl *encl;
> > + int ret;
> > +
> > + encl = kzalloc(sizeof(*encl), GFP_KERNEL);
> > + if (!encl)
> > + return -ENOMEM;
> > +
> > + atomic_set(&encl->flags, 0);
> > + kref_init(&encl->refcount);
> > + xa_init(&encl->page_array);
> > + mutex_init(&encl->lock);
> > + INIT_LIST_HEAD(&encl->mm_list);
> > + spin_lock_init(&encl->mm_lock);
> > +
> > + ret = init_srcu_struct(&encl->srcu);
> > + if (ret) {
> > + kfree(encl);
> > + return ret;
> > + }
> > +
> > + file->private_data = encl;
> > +
> > + return 0;
> > +}
> > +
> > +static int sgx_release(struct inode *inode, struct file *file)
> > +{
> > + struct sgx_encl *encl = file->private_data;
> > + struct sgx_encl_mm *encl_mm;
> > +
> > + for ( ; ; ) {
> > + spin_lock(&encl->mm_lock);
> > +
> > + if (list_empty(&encl->mm_list)) {
> > + encl_mm = NULL;
> > + } else {
> > + encl_mm = list_first_entry(&encl->mm_list,
> > + struct sgx_encl_mm, list);
> > + list_del_rcu(&encl_mm->list);
> > + }
> > +
> > + spin_unlock(&encl->mm_lock);
> > +
> > + /* The list is empty, ready to go. */
> > + if (!encl_mm)
> > + break;
> > +
> > + synchronize_srcu(&encl->srcu);
> > + mmu_notifier_unregister(&encl_mm->mmu_notifier, encl_mm->mm);
> > + kfree(encl_mm);
> > + }
> > +
> > + mutex_lock(&encl->lock);
> > + atomic_or(SGX_ENCL_DEAD, &encl->flags);
>
> So you set a flag that this is dead, and then instantly delete it? Why
> does that matter? I see you check for this flag elsewhere, but as you
> are just about to delete this structure, how can this be an issue?
It matters because ksgxswapd (sgx_reclaimer_*) might be processing it.
It will use the flag to skip the operations that it would do to a victim
page, when the enclave is still alive.
>
> > + mutex_unlock(&encl->lock);
> > +
> > + kref_put(&encl->refcount, sgx_encl_release);
>
> Don't you need to hold the lock across the put? If not, what is
> serializing this?
>
> But an even larger comment, why is this reference count needed at all?
>
> You never grab it except at init time, and you free it at close time.
> Why not rely on the reference counting that the vfs ensures you?
Because ksgxswapd needs the alive enclave instance while it is in the
process of swapping a victim page. The reason for this is the
hierarchical nature of the enclave pages.
As an example, a write operation to main memory, EWB (SDM vol 3D 40-79)
needs to access SGX Enclave Control Structure (SECS) page, which is
contains global data for an enclave, like the unswapped child count.
> > + return 0;
> > +}
> > +
> > +static int sgx_mmap(struct file *file, struct vm_area_struct *vma)
> > +{
> > + struct sgx_encl *encl = file->private_data;
> > + int ret;
> > +
> > + ret = sgx_encl_may_map(encl, vma->vm_start, vma->vm_end, vma->vm_flags);
> > + if (ret)
> > + return ret;
> > +
> > + ret = sgx_encl_mm_add(encl, vma->vm_mm);
> > + if (ret)
> > + return ret;
> > +
> > + vma->vm_ops = &sgx_vm_ops;
> > + vma->vm_flags |= VM_PFNMAP | VM_DONTEXPAND | VM_DONTDUMP | VM_IO;
> > + vma->vm_private_data = encl;
> > +
> > + return 0;
> > +}
> > +
> > +static unsigned long sgx_get_unmapped_area(struct file *file,
> > + unsigned long addr,
> > + unsigned long len,
> > + unsigned long pgoff,
> > + unsigned long flags)
> > +{
> > + if ((flags & MAP_TYPE) == MAP_PRIVATE)
> > + return -EINVAL;
> > +
> > + if (flags & MAP_FIXED)
> > + return addr;
> > +
> > + return current->mm->get_unmapped_area(file, addr, len, pgoff, flags);
> > +}
> > +
> > +static const struct file_operations sgx_encl_fops = {
> > + .owner = THIS_MODULE,
> > + .open = sgx_open,
> > + .release = sgx_release,
> > + .mmap = sgx_mmap,
> > + .get_unmapped_area = sgx_get_unmapped_area,
> > +};
> > +
> > +static struct miscdevice sgx_dev_enclave = {
> > + .minor = MISC_DYNAMIC_MINOR,
> > + .name = "enclave",
> > + .nodename = "sgx/enclave",
>
> A subdir for a single device node? Ok, odd, but why not just
> "sgx_enclave"? How "special" is this device node?
There is a patch that adds "sgx/provision".
Either works for me. Should I flatten them to "sgx_enclave" and
"sgx_provision", or keep them as they are?
> thanks,
>
> greg k-h
/Jarkko
next prev parent reply other threads:[~2020-10-04 14:33 UTC|newest]
Thread overview: 117+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-10-03 4:50 [PATCH v39 00/24] Intel SGX foundations Jarkko Sakkinen
2020-10-03 4:50 ` [PATCH v39 01/24] x86/cpufeatures: x86/msr: Add Intel SGX hardware bits Jarkko Sakkinen
2020-10-19 14:10 ` Dave Hansen
2020-10-19 17:49 ` Sean Christopherson
2020-10-03 4:50 ` [PATCH v39 02/24] x86/cpufeatures: x86/msr: Add Intel SGX Launch Control " Jarkko Sakkinen
2020-10-03 4:50 ` [PATCH v39 03/24] x86/mm: x86/sgx: Signal SIGSEGV with PF_SGX Jarkko Sakkinen
2020-10-03 4:50 ` [PATCH v39 04/24] x86/sgx: Add SGX microarchitectural data structures Jarkko Sakkinen
2020-10-03 4:50 ` [PATCH v39 05/24] x86/sgx: Add wrappers for ENCLS leaf functions Jarkko Sakkinen
2020-10-19 14:30 ` Dave Hansen
2020-10-19 17:38 ` Sean Christopherson
2020-10-19 17:48 ` Dave Hansen
2020-10-19 17:53 ` Sean Christopherson
2020-10-19 17:58 ` Dave Hansen
2020-10-03 4:50 ` [PATCH v39 06/24] x86/cpu/intel: Detect SGX support Jarkko Sakkinen
2020-10-03 4:50 ` [PATCH v39 07/24] x86/cpu/intel: Add nosgx kernel parameter Jarkko Sakkinen
2020-10-03 4:50 ` [PATCH v39 08/24] x86/sgx: Initialize metadata for Enclave Page Cache (EPC) sections Jarkko Sakkinen
2020-10-19 8:45 ` Jarkko Sakkinen
2020-10-19 12:39 ` Borislav Petkov
2020-10-23 9:01 ` Jarkko Sakkinen
2020-10-19 13:40 ` Dave Hansen
2020-10-23 9:03 ` Jarkko Sakkinen
2020-10-03 4:50 ` [PATCH v39 09/24] x86/sgx: Add __sgx_alloc_epc_page() and sgx_free_epc_page() Jarkko Sakkinen
2020-10-03 4:50 ` [PATCH v39 10/24] mm: Add 'mprotect' hook to struct vm_operations_struct Jarkko Sakkinen
2020-10-03 4:50 ` [PATCH v39 11/24] x86/sgx: Add SGX enclave driver Jarkko Sakkinen
2020-10-03 14:39 ` Greg KH
2020-10-04 14:32 ` Jarkko Sakkinen [this message]
2020-10-04 15:01 ` Jarkko Sakkinen
2020-10-05 9:42 ` Greg KH
2020-10-05 12:42 ` Jarkko Sakkinen
2020-10-07 18:09 ` Haitao Huang
2020-10-07 19:26 ` Greg KH
2020-10-09 6:44 ` Jarkko Sakkinen
2020-10-14 20:16 ` Dave Hansen
2020-10-05 8:45 ` Christoph Hellwig
2020-10-05 11:42 ` Jarkko Sakkinen
2020-10-05 11:50 ` Greg KH
2020-10-05 14:23 ` Jarkko Sakkinen
2020-10-05 15:02 ` Greg KH
2020-10-05 16:40 ` Dave Hansen
2020-10-05 20:02 ` Jarkko Sakkinen
2020-10-09 7:10 ` Pavel Machek
2020-10-09 7:21 ` Greg KH
2020-10-09 8:21 ` Pavel Machek
2020-10-03 19:54 ` Matthew Wilcox
2020-10-04 21:50 ` Jarkko Sakkinen
2020-10-04 22:02 ` Jarkko Sakkinen
2020-10-04 22:27 ` Matthew Wilcox
2020-10-04 23:41 ` Jarkko Sakkinen
2020-10-05 1:30 ` Matthew Wilcox
2020-10-05 3:06 ` Jarkko Sakkinen
2020-10-03 4:50 ` [PATCH v39 12/24] x86/sgx: Add SGX_IOC_ENCLAVE_CREATE Jarkko Sakkinen
2020-10-16 17:07 ` Dave Hansen
2020-10-18 4:26 ` Jarkko Sakkinen
2020-10-19 20:21 ` Dave Hansen
2020-10-19 20:48 ` Sean Christopherson
2020-10-03 4:50 ` [PATCH v39 13/24] x86/sgx: Add SGX_IOC_ENCLAVE_ADD_PAGES Jarkko Sakkinen
2020-10-16 21:25 ` Dave Hansen
2020-10-18 5:03 ` Jarkko Sakkinen
2020-10-19 7:03 ` Jarkko Sakkinen
2020-10-19 20:48 ` Dave Hansen
2020-10-19 21:15 ` Sean Christopherson
2020-10-19 21:44 ` Dave Hansen
2020-10-23 10:11 ` Jarkko Sakkinen
2020-10-03 4:50 ` [PATCH v39 14/24] x86/sgx: Add SGX_IOC_ENCLAVE_INIT Jarkko Sakkinen
2020-10-03 4:50 ` [PATCH v39 15/24] x86/sgx: Add SGX_IOC_ENCLAVE_PROVISION Jarkko Sakkinen
2020-10-20 15:48 ` Dave Hansen
2020-10-23 10:14 ` Jarkko Sakkinen
2020-10-20 21:19 ` Dave Hansen
2020-10-23 10:17 ` Jarkko Sakkinen
2020-10-23 14:19 ` Dave Hansen
2020-10-24 11:34 ` Jarkko Sakkinen
2020-10-24 15:47 ` Andy Lutomirski
2020-10-24 20:23 ` Jarkko Sakkinen
2020-10-27 10:38 ` Dr. Greg
2020-10-23 14:23 ` Jethro Beekman
2020-10-24 11:40 ` Jarkko Sakkinen
2020-10-03 4:50 ` [PATCH v39 16/24] x86/sgx: Add a page reclaimer Jarkko Sakkinen
2020-10-03 5:22 ` Haitao Huang
2020-10-03 13:32 ` Jarkko Sakkinen
2020-10-03 18:23 ` Haitao Huang
2020-10-04 22:39 ` Jarkko Sakkinen
2020-10-07 17:25 ` Jarkko Sakkinen
2020-10-03 4:50 ` [PATCH v39 17/24] x86/sgx: Add ptrace() support for the SGX driver Jarkko Sakkinen
2020-10-03 4:50 ` [PATCH v39 18/24] x86/vdso: Add support for exception fixup in vDSO functions Jarkko Sakkinen
2020-10-03 4:50 ` [PATCH v39 19/24] x86/fault: Add helper function to sanitize error code Jarkko Sakkinen
2020-10-03 4:50 ` [PATCH v39 20/24] x86/traps: Attempt to fixup exceptions in vDSO before signaling Jarkko Sakkinen
2020-10-03 4:50 ` [PATCH v39 21/24] x86/vdso: Implement a vDSO for Intel SGX enclave call Jarkko Sakkinen
2020-10-06 2:57 ` Sean Christopherson
2020-10-06 8:30 ` Jethro Beekman
2020-10-06 15:15 ` Sean Christopherson
2020-10-06 17:28 ` Jarkko Sakkinen
2020-10-06 23:21 ` Sean Christopherson
2020-10-07 0:22 ` Jarkko Sakkinen
2020-10-07 1:17 ` Sean Christopherson
2020-10-07 3:14 ` Jarkko Sakkinen
2020-10-07 4:34 ` Sean Christopherson
2020-10-07 7:39 ` Jarkko Sakkinen
2020-10-07 8:04 ` Jarkko Sakkinen
2020-10-07 15:25 ` Sean Christopherson
2020-10-07 17:08 ` Jarkko Sakkinen
2020-10-07 17:13 ` Jarkko Sakkinen
2020-10-06 15:49 ` Jarkko Sakkinen
2020-10-06 15:36 ` Jarkko Sakkinen
2020-10-06 21:39 ` Jarkko Sakkinen
2020-10-07 0:23 ` Jarkko Sakkinen
2020-10-17 1:48 ` Andy Lutomirski
2020-10-17 21:02 ` Jarkko Sakkinen
2020-10-03 4:50 ` [PATCH v39 22/24] selftests/x86: Add a selftest for SGX Jarkko Sakkinen
2020-10-12 16:50 ` Jarkko Sakkinen
2020-10-03 4:50 ` [PATCH v39 23/24] docs: x86/sgx: Document SGX micro architecture and kernel internals Jarkko Sakkinen
2020-10-03 4:50 ` [PATCH v39 24/24] x86/sgx: Update MAINTAINERS Jarkko Sakkinen
2020-10-16 21:04 ` Dave Hansen
2020-10-18 4:27 ` Jarkko Sakkinen
2020-10-03 14:32 ` [PATCH v39 00/24] Intel SGX foundations Greg KH
2020-10-03 14:53 ` Jarkko Sakkinen
2020-10-15 19:06 ` Dave Hansen
2020-10-17 20:43 ` Jarkko Sakkinen
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20201004143246.GA3561@linux.intel.com \
--to=jarkko.sakkinen@linux.intel.com \
--cc=akpm@linux-foundation.org \
--cc=andriy.shevchenko@linux.intel.com \
--cc=asapek@google.com \
--cc=bp@alien8.de \
--cc=cedric.xing@intel.com \
--cc=chenalexchen@google.com \
--cc=conradparker@google.com \
--cc=cyhanish@google.com \
--cc=darren.kenny@oracle.com \
--cc=dave.hansen@intel.com \
--cc=gregkh@linuxfoundation.org \
--cc=haitao.huang@intel.com \
--cc=haitao.huang@linux.intel.com \
--cc=jethro@fortanix.com \
--cc=jorhand@linux.microsoft.com \
--cc=kai.huang@intel.com \
--cc=kai.svahn@intel.com \
--cc=kmoy@google.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=linux-security-module@vger.kernel.org \
--cc=linux-sgx@vger.kernel.org \
--cc=ludloff@google.com \
--cc=luto@kernel.org \
--cc=mikko.ylinen@intel.com \
--cc=nhorman@redhat.com \
--cc=npmccallum@redhat.com \
--cc=puiterwijk@redhat.com \
--cc=rientjes@google.com \
--cc=sanqian.hcy@antfin.com \
--cc=sean.j.christopherson@intel.com \
--cc=sethmo@google.com \
--cc=suresh.b.siddha@intel.com \
--cc=tglx@linutronix.de \
--cc=willy@infradead.org \
--cc=x86@kernel.org \
--cc=yaozhangx@google.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).