Re: [PATCH v19 16/27] x86/sgx: Add the Linux SGX Enclave Driver

From: Sean Christopherson <sean.j.christopherson@intel.com>
To: Jethro Beekman <jethro@fortanix.com>
Cc: Jarkko Sakkinen <jarkko.sakkinen@linux.intel.com>,
	"x86@kernel.org" <x86@kernel.org>,
	"linux-sgx@vger.kernel.org" <linux-sgx@vger.kernel.org>,
	"akpm@linux-foundation.org" <akpm@linux-foundation.org>,
	"dave.hansen@intel.com" <dave.hansen@intel.com>,
	"nhorman@redhat.com" <nhorman@redhat.com>,
	"npmccallum@redhat.com" <npmccallum@redhat.com>,
	"serge.ayoun@intel.com" <serge.ayoun@intel.com>,
	"shay.katz-zamir@intel.com" <shay.katz-zamir@intel.com>,
	"haitao.huang@intel.com" <haitao.huang@intel.com>,
	"andriy.shevchenko@linux.intel.com" 
	<andriy.shevchenko@linux.intel.com>,
	"tglx@linutronix.de" <tglx@linutronix.de>,
	"kai.svahn@intel.com" <kai.svahn@intel.com>,
	"bp@alien8.de" <bp@alien8.de>,
	"josh@joshtriplett.org" <josh@joshtriplett.org>,
	"luto@kernel.org" <luto@kernel.org>,
	"kai.huang@intel.com" <kai.huang@intel.com>,
	"rientjes@google.com" <rientjes@google.com>,
	Suresh Siddha <suresh.b.siddha@intel.com>
Subject: Re: [PATCH v19 16/27] x86/sgx: Add the Linux SGX Enclave Driver
Date: Wed, 27 Mar 2019 13:06:11 -0700	[thread overview]
Message-ID: <20190327200610.GF9310@linux.intel.com> (raw)
In-Reply-To: <825f5477-c90a-b54b-efeb-a1dc43ccc9d2@fortanix.com>

On Wed, Mar 27, 2019 at 06:38:57PM +0000, Jethro Beekman wrote:
> On 2019-03-26 22:28, Jarkko Sakkinen wrote:
> >On Tue, Mar 26, 2019 at 04:58:52PM -0700, Sean Christopherson wrote:
> >>On Tue, Mar 26, 2019 at 03:26:50PM +0200, Jarkko Sakkinen wrote:
> >>>On Thu, Mar 21, 2019 at 05:51:11PM +0200, Jarkko Sakkinen wrote:
> >>>>>Yuck.  If we remove the driver specific Makefile then we can eliminate
> >>>>>the "../" prefix here.  E.g. in the main SGX Makefile:
> >>>>>
> >>>>>obj-$(CONFIG_INTEL_SGX_DRIVER) += driver/main.o driver/ioctl.o
> >>>>
> >>>>I think this is a great idea.
> >>>
> >>>On a 2nd thought not gonna do anything to that because it would
> >>>require to move driver.h and it is cleaner to keep all the driver
> >>>files in the same directory (and separated from the core).
> >>
> >>What about collapsing driver/*.c into driver.c and moving driver.{c,h}
> >>to the root sgx directory?  The bulk of driver/main.c is securityfs
> >>and platform driver code, e.g. has a good chance of going away entirely
> >>or being moved out of the "driver".  At that point there probably isn't
> >>a strong reason to have driver/main.c and driver/ioctl.c.
> >
> >I think doing anything major would require to lock in whether to have
> >the LKM for the driver at all. If we wipe out the driver, then this is
> >just matter of moving dev management part to lets say dev.c.
> >
> >Unless there is some real production use I can wipe it away. For v19
> >I wanted to fix it namely because in v18 LKM was just broken. It is
> >always good to make decisions based on working code.
> 
> It should be a module because things should be modules when possible. I'm
> not sure what the "Linux policy" is here but this seems obvious to me.
> 
> For example:
> 
> * Modules allow users to easily disable functionality that they don't use/is
> buggy for them/other reasons using blacklisting.
> * Modules allow users to customize their functionality without having to
> rebuild the entire kernel.
> * Modules allow developers to customize their modules without having to
> rebuild the entire kernel.

I agree with all of the above, but unfortunately blacklisting is really
the only benefit that would be realized by modularizing the driver.  The
"driver" at this point is just the device and its ioctls, the meat of
the functionality has been moved into the subsystem proper.  And the few
remaining tidbits of functionality, e.g. sgx_encl_page_alloc() and
sgx_encl_alloc(), probably should be moved out of ioctl.c as well.

Theoretically, we could revert to the old model, i.e. move a big pile of
code back into the driver, but that approach has a variety of technical
issues that IMO outweigh the benefits of having a module.

Tying into the your comment of "things should be modules when possible",
we've gradually come to the realization that truly modularizing the SGX
driver isn't possible, at least not without compromising other parts of
the design.

For example, relying on the EPC ACPI entry to autoprobe the driver falls
apart when virtualization support also wants to add an SGX device (or we
end up with a weird split model where the uapi driver is autoprobed via
ACPI and the virtualization device is probed by the SGX subsystem).

Relying on the EPC ACPI entry really shows its warts when systems
with multiple EPC sections show up.  The SGS BIOS writer's guide
(allegedly, I haven't personally read it) says that one and only one
EPC entry should be created in the ACPI tables, i.e. software must
use CPUID to enumerate the base+size of EPC sections.

In other words, the whole ACPI entry and platform device approach was a
hack purely to allow SGX to be implemented as an out-of-kernel driver.
If the darn ACPI hack had never been added in the first place, i.e.
CPUID is the only way to enumerate/probe SGX, then odds are we wouldn't
even be having this dicsussion and no one would bat an eye at SGX being
implemented as an Intel-specific feature that is baked into the kernel.

> Specifically for SGX I can think of the following reasons as well:
> 
> * Module-based hypervisors may want to make EPC allocations for their
> guests.
> * Easy experimentation with different EPC interfaces
> * Easy experimentation with in-kernel LE
> 
> --
> Jethro Beekman | Fortanix
> 
> >
> >/Jarkko
> >
> 