Re: x86/sgx: uapi change proposal

From: Sean Christopherson <sean.j.christopherson@intel.com>
To: Andy Lutomirski <luto@kernel.org>
Cc: Jethro Beekman <jethro@fortanix.com>,
	Jarkko Sakkinen <jarkko.sakkinen@linux.intel.com>,
	Thomas Gleixner <tglx@linutronix.de>,
	Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>,
	"x86@kernel.org" <x86@kernel.org>,
	Dave Hansen <dave.hansen@linux.intel.com>,
	Peter Zijlstra <peterz@infradead.org>,
	"H. Peter Anvin" <hpa@zytor.com>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"linux-sgx@vger.kernel.org" <linux-sgx@vger.kernel.org>,
	Josh Triplett <josh@joshtriplett.org>,
	Haitao Huang <haitao.huang@linux.intel.com>,
	"Dr . Greg Wettstein" <greg@enjellic.com>
Subject: Re: x86/sgx: uapi change proposal
Date: Fri, 21 Dec 2018 10:24:54 -0800	[thread overview]
Message-ID: <20181221182454.GA27371@linux.intel.com> (raw)
In-Reply-To: <CALCETrX+KisMCbptrnPSO79-YF4E3nR1XHt+a7hCs1GXsxAbtw@mail.gmail.com>

On Fri, Dec 21, 2018 at 09:12:46AM -0800, Andy Lutomirski wrote:
> > On Dec 21, 2018, at 9:28 AM, Sean Christopherson <sean.j.christopherson@intel.com> wrote:
> >
> > On Wed, Dec 19, 2018 at 06:58:48PM -0800, Andy Lutomirski wrote:
> >>> On Dec 19, 2018, at 6:45 AM, Sean Christopherson <sean.j.christopherson@intel.com> wrote:
> >>>
> >>>>> On Wed, Dec 19, 2018 at 09:36:16AM +0000, Jethro Beekman wrote:
> >>>
> >>> I agree with Jethro, passing the enclave_fd as a param is obnoxious.
> >>> And it means the user needs to open /dev/sgx to do anything with an
> >>> enclave fd, e.g. the enclave fd might be passed to a builder thread,
> >>> it shouldn't also need the device fd.
> >>>
> >>> E.g.:
> >>>
> >>>   sgx_fd = open("/dev/sgx", O_RDWR);
> >>>   BUG_ON(sgx_fd < 0);
> >>>
> >>>   enclave_fd = ioctl(sgx_fd, SGX_ENCLAVE_CREATE, &ecreate);
> >>>   BUG_ON(enclave_fd < 0);
> >>>
> >>>   ret = ioctl(enclave_fd, SGX_ENCLAVE_ADD_PAGE, &eadd);
> >>>   BUG_ON(ret);
> >>>
> >>>   ...
> >>>
> >>>   ret = ioctl(enclave_fd, SGX_ENCLAVE_INIT, &einit);
> >>>   BUG_ON(ret);
> >>>
> >>>   ...
> >>>
> >>>   close(enclave_fd);
> >>>   close(sgx_fd);
> >>>
> >>>
> >>> Take a look at virt/kvm/kvm_main.c to see how KVM manages anon inodes
> >>> and ioctls for VMs and vCPUs.
> >>
> >> Can one of you explain why SGX_ENCLAVE_CREATE is better than just
> >> opening a new instance of /dev/sgx for each encalve?
> >
> > Directly associating /dev/sgx with an enclave means /dev/sgx can't be
> > used to provide ioctl()'s for other SGX-related needs, e.g. to mmap()
> > raw EPC and expose it a VM.  Proposed layout in the link below.  I'll
> > also respond to Jarkko's question about exposing EPC through /dev/sgx
> > instead of having KVM allocate it on behalf of the VM.
> 
> Hmm. I guess this makes some sense.  My instinct would be to do it a
> little differently and have:
> 
> /dev/sgx/enclave: Each instance is an enclave.
> 
> /dev/sgx/epc: Used to get raw EPC for KVM. Might have different
> permissions, perhaps 0660 and group kvm.
> 
> /dev/sgx/something_else: For when SGX v3 adds something else :)

Mmmm, I like this approach a lot.  It would allow userspace to easily
manage permissions for each "feature", e.g. give all users access to
/dev/sgx/epc but restrict /dev/sgx/enclave.

And we could add e.g. /dev/sgx/admin if we wanted to exposed ioctls()
that apply to all aspects of SGX.

Do you know if /dev/sgx/epc could be dynamically created, e.g. by
KVM when the kvm_intel module is loaded?  That would seal the deal for
me as it'd keep open the option of having KVM handle oversubscription
of guest EPC while exposing EPC through /dev/sgx instead of /dev/kvm.  