Re: [PATCH v41 12/24] x86/sgx: Add SGX_IOC_ENCLAVE_CREATE

From: Jarkko Sakkinen <jarkko@kernel.org>
To: Dave Hansen <dave.hansen@intel.com>
Cc: Hillf Danton <hdanton@sina.com>,
	linux-kernel@vger.kernel.org,
	Jethro Beekman <jethro@fortanix.com>,
	akpm@linux-foundation.org, andriy.shevchenko@linux.intel.com,
	asapek@google.com, bp@alien8.de, cedric.xing@intel.com,
	chenalexchen@google.com, conradparker@google.com,
	cyhanish@google.com, haitao.huang@intel.com, kai.huang@intel.com,
	kai.svahn@intel.com, kmoy@google.com, ludloff@google.com,
	luto@kernel.org, nhorman@redhat.com, npmccallum@redhat.com,
	puiterwijk@redhat.com, rientjes@google.com, tglx@linutronix.de,
	yaozhangx@google.com, mikko.ylinen@intel.com
Subject: Re: [PATCH v41 12/24] x86/sgx: Add SGX_IOC_ENCLAVE_CREATE
Date: Tue, 17 Nov 2020 19:40:27 +0200	[thread overview]
Message-ID: <20201117174027.GC8524@kernel.org> (raw)
In-Reply-To: <610d665e-564e-621d-4ecb-fbe5bfdf7133@intel.com>

On Mon, Nov 16, 2020 at 04:34:23PM -0800, Dave Hansen wrote:
> On 11/16/20 9:54 AM, Dave Hansen wrote:
> >> ENCLS instructions must be serialized for a given enclave, but holding
> >> encl->lock for an entire ioctl() will result in deadlock due to an enclave
> >> triggering reclaim on itself.
> >>
> >> Building an enclave must also be serialized, i.e. userspace can't queue up
> >> EADD on multiple threads, because the order in which pages are added to an
> >> enclave affects the measurement.  In other words, rejecting the ioctl() as
> >> opposed to waiting on a lock is also desirable.
> > Sounds like we need should follow up with an add-on patch to get some of
> > that into a comment.
> 
> Jarkko, first of all, let's rename:
> 
> 	SGX_ENCL_IOCTL -> SGX_ENCL_IOCTL_LOCK
> 
> If it walks like a duck and quacks like a duck...
> 
> Sean had a good example of examples of how EADD could go wrong with
> multiple threads.  Were there more good examples we could stick in a
> changelog?  I seem to recall that there are more than a few SGX
> instructions don't even work in parallel and require software
> synchronization.  Could we get a list or at least a few more good examples?

It does not need a specific example. A cryptographic measurement
requires always a strict ordering. Any parallel use is a good example.

> I also think we should be much more assertive about multiple ioctl()
> callers:
> 
> 	/* Multi-threaded enclave management is invalid and unsafe: */
> 	if (test_and_set_bit(SGX_ENCL_IOCTL_LOCK, &encl->flags))
> 		return -EINVAL;
> 
> -EBUSY is saying "everything is OK, just busy, please try again later."
>  -EINVAL is saying, "userspace, you screwed up".

Yeah, it is invalid use of the API. I'll change it to -EINVAL.

> Also, does SGX_ENCL_IOCTL_LOCK provide serialization for anything other
> than the *hardware* instructions?  I couldn't find much, although:
> 
>         encl->attributes_mask |= SGX_ATTR_PROVISIONKEY;
> 
> seems to be lacking any other serialization.
> 
> sgx_encl_create() also seems like it has no other locking and relies on
> SGX_ENCL_IOCTL_LOCK for sanity.

Yeah, those ioctl's take advantage of the flag.

/Jarkko