Re: [RFC PATCH 00/23] KVM SGX virtualization support

From: Sean Christopherson <seanjc@google.com>
To: Jarkko Sakkinen <jarkko@kernel.org>
Cc: Kai Huang <kai.huang@intel.com>,
	linux-sgx@vger.kernel.org, kvm@vger.kernel.org, x86@kernel.org,
	luto@kernel.org, dave.hansen@intel.com, haitao.huang@intel.com,
	pbonzini@redhat.com, bp@alien8.de, tglx@linutronix.de,
	mingo@redhat.com, hpa@zytor.com, jethro@fortanix.com,
	b.thiel@posteo.de, mattson@google.com, joro@8bytes.org,
	vkuznets@redhat.com, wanpengli@tencent.com, corbet@lwn.net
Subject: Re: [RFC PATCH 00/23] KVM SGX virtualization support
Date: Mon, 11 Jan 2021 10:37:05 -0800	[thread overview]
Message-ID: <X/ya0XnsQn4xb/1L@google.com> (raw)
In-Reply-To: <2422737f6b0cddf6ff1be9cf90e287dd00d6a6a3.camel@kernel.org>

On Mon, Jan 11, 2021, Jarkko Sakkinen wrote:
> On Wed, 2021-01-06 at 14:55 +1300, Kai Huang wrote:
> >   - Does not require changes to KVM's uAPI, e.g. EPC gets handled as
> >     just another memory backend for guests.
> 
> Why this an advantage? No objection, just a question.

There are zero KVM changes required to support exposing EPC to a guest.  KVM's
MMU is completely ignorant of what physical backing is used for any given host
virtual address.  KVM has to be aware of various VM_* flags, e.g. VM_PFNMAP and
VM_IO, but that code is arch agnostic and is quite isolated.

> >   - EPC management is wholly contained in the SGX subsystem, e.g. SGX
> >     does not have to export any symbols, changes to reclaim flows don't
> >     need to be routed through KVM, SGX's dirty laundry doesn't have to
> >     get aired out for the world to see, and so on and so forth.
> 
> No comments to this before understanding code changes better.
> 
> > The virtual EPC allocated to guests is currently not reclaimable, due to
> > reclaiming EPC from KVM guests is not currently supported. Due to the
> > complications of handling reclaim conflicts between guest and host, KVM
> > EPC oversubscription, which allows total virtual EPC size greater than
> > physical EPC by being able to reclaiming guests' EPC, is significantly more
> > complex than basic support for SGX virtualization.
> 
> I think it should be really in the center of the patch set description that
> this patch set implements segmentation of EPC, not oversubscription. It should
> be clear immediately. It's a core part of knowing "what I'm looking at".

Technically, it doesn't implement EPC segmentation of EPC.  It implements
non-reclaimable EPC allocation.  Even that is somewhat untrue as the EPC can be
forcefully reclaimed, but doing so will destroy the guest contents.

Userspace can oversubscribe the EPC to KVM guests, but it would need to kill,
migrate, or pause one or more VMs if the pool of physical EPC were exhausted.

> > - Support SGX virtualization without SGX Launch Control unlocked mode
> > 
> > Although SGX driver requires SGX Launch Control unlocked mode to work, SGX
> > virtualization doesn't, since how enclave is created is completely controlled
> > by guest SGX software, which is not necessarily linux. Therefore, this series
> > allows KVM to expose SGX to guest even SGX Launch Control is in locked mode,
> > or is not present at all. The reason is the goal of SGX virtualization, or
> > virtualization in general, is to expose hardware feature to guest, but not to
> > make assumption how guest will use it. Therefore, KVM should support SGX guest
> > as long as hardware is able to, to have chance to support more potential use
> > cases in cloud environment.
> 
> AFAIK the convergence point with the FLC was, and is that Linux never enables
> SGX with locked MSRs.
> 
> And I don't understand, if it is not fine to allow locked SGX for a *process*,
> why is it fine for a *virtual machine*? They have a lot same.

Because it's a completely different OS/kernel.  If the user has a kernel that
supports locked SGX, then so be it.  There's no novel circumvention of the
kernel policy, e.g. the user could simply boot the non-upstream kernel directly,
and running an upstream kernel in the guest will not cause the kernel to support
SGX.

There are any number of things that are allowed in a KVM guest that are not
allowed in a bare metal process.

> I cannot remember out of top of my head, could the Intel SHA256 be read when
> booted with unlocked MSRs. If that is the case, then you can still support
> guests with that configuration.

No, it's not guaranteed to be readable as firmware could have already changed
the values in the MSRs.

> Context-dependent guidelines tend to also trash code big time. Also, for the
> sake of a sane kernel code base, I would consider only supporting unlocked
> MSRs.

It's one line of a code to teach the kernel driver not to load if the MSRs are
locked.  And IMO, that one line of code is a net positive as it makes it clear
in the driver itself that it chooses not support locked MSRs, even if SGX itself
is fully enabled.