Re: [RFC PATCH 00/23] KVM SGX virtualization support

From: Sean Christopherson <seanjc@google.com>
To: Kai Huang <kai.huang@intel.com>
Cc: Dave Hansen <dave.hansen@intel.com>,
	linux-sgx@vger.kernel.org, kvm@vger.kernel.org, x86@kernel.org,
	jarkko@kernel.org, luto@kernel.org, haitao.huang@intel.com,
	pbonzini@redhat.com, bp@alien8.de, tglx@linutronix.de,
	mingo@redhat.com, hpa@zytor.com, jethro@fortanix.com,
	b.thiel@posteo.de, jmattson@google.com, joro@8bytes.org,
	vkuznets@redhat.com, wanpengli@tencent.com, corbet@lwn.net
Subject: Re: [RFC PATCH 00/23] KVM SGX virtualization support
Date: Thu, 7 Jan 2021 08:14:44 -0800	[thread overview]
Message-ID: <X/czdL6TmbuBdZ/b@google.com> (raw)
In-Reply-To: <20210107145026.a8f593323ab9ae34874250ed@intel.com>

On Thu, Jan 07, 2021, Kai Huang wrote:
> On Wed, 6 Jan 2021 16:48:58 -0800 Dave Hansen wrote:
> > Your series adds some new interfaces, including /dev/sgx_virt_epc.  If
> > the kernel wants to add oversubscription in the future, will old binary
> > application users of /dev/sgx_virt_epc be able to support
> > oversubscription?  Or, would users of /dev/sgx_virt_epc need to change
> > to support oversubscription?
> 
> Oversubscription will be completely done in kernel/kvm, and will be
> transparent to userspace, so it will not impact ABI.

It's not transparent to userpsace, odds are very good that userspace would want
to opt in/out of EPC reclaim for its VMs.  E.g. for cases where it would be
preferable to fail to launch a VM than degrade performance.

That being said, there are no anticipated /dev/sgx_virt_epc ABI changes to
support reclaim, as the ABI changes will be in KVM.  In the KVM oversubscription
POC, I added a KVM ioctl to allow enabling EPC reclaim/oversubscription.  That
ioctl took a fd for a /dev/sgx_virt_epc instance.

The reason for routing through KVM was to solve two dependencies issues:

  - KVM needs a reference to the virt_epc instance to handle SGX_CONFLICT VM-Exits

  - The SGX subsystem needs to be able to translate GPAs to HVAs to retrieve the
    SECS for a page it is reclaiming.  That requires a KVM instance and helper
    function.

Routing the ioctl through KVM allows KVM to hand over a pointer of itself along
with a GPA->HVA helper, and the SGX subsystem in turn can hand back the virt_epc
instance resolved from the fd.

It would be possible to invert the flow, e.g. pass in a KVM fd to a new
/dev/sgx_virt_epc ioctl, but I suspect that would be kludgier, and in any case
it would be a new ioctl and so would not break existing users.