Re: [PATCH 13/25] x86/sgx: Support adding of pages to initialized enclave

[Jarkko Sakkinen <jarkko@kernel.org>]

On Mon, 2021-12-06 at 13:44 -0800, Reinette Chatre wrote:
> Hi Jarkko,
> 
> On 12/4/2021 3:13 PM, Jarkko Sakkinen wrote:
> > "to initialize" -> "to an initialized"
> 
> Will do.
> 
> 
> > 
> > On Wed, Dec 01, 2021 at 11:23:11AM -0800, Reinette Chatre wrote:
> > > With SGX1 an enclave needs to be created with its maximum memory demands
> > > allocated. Pages cannot be added to an enclave after it is initialized.
> > > SGX2 introduces a new function, ENCLS[EAUG], that can be used to add
> > > pages to an initialized enclave. With SGX2 the enclave still needs to
> > > set aside address space for its maximum memory demands during enclave
> > > creation, but all pages need not be added before enclave initialization.
> > > Pages can be added during enclave runtime.
> > > 
> > > Add support for dynamically adding pages to an initialized enclave,
> > > architecturally limited to RW permission. Add pages via the page fault
> > > handler at the time an enclave address without a backing enclave page
> > > is accessed, potentially directly reclaiming pages if no free pages
> > > are available.
> > > 
> > > The enclave is still required to run ENCLU[EACCEPT] on the page before
> > > it can be used. A useful flow is for the enclave to run ENCLU[EACCEPT]
> > > on an uninitialized address. This will trigger the page fault handler
> > > that will add the enclave page and return execution to the enclave to
> > > repeat the ENCLU[EACCEPT] instruction, this time successful.
> > > 
> > > If the enclave accesses an uninitialized address in another way, for
> > > example by expanding the enclave stack to a page that has not yet been
> > > added, then the page fault handler would add the page on the first
> > > write but upon returning to the enclave the instruction that triggered
> > > the page fault would be repeated and since ENCLU[EACCEPT] was not run
> > > yet it would trigger a second page fault, this time with the SGX flag
> > > set in the page fault error code. This can only be recovered by entering
> > > the enclave again and directly running the ENCLU[EACCEPT] instruction on
> > > the now initialized address.
> > > 
> > > Accessing an uninitialized address from outside the enclave also triggers
> > > this flow but the page will remain in PENDING state until accepted from
> > > within the enclave.
> > 
> > What does it mean being in PENDING state, and more imporantly, what is
> > PENDING state? What does a memory access within enclave cause when it
> > touch a page within this state?
> 
> The PENDING state is the enclave page state from the SGX hardware's 
> perspective. The OS uses the ENCLS[EAUG] SGX2 function to add a new page 
> to the enclave but from the SGX hardware's perspective it would be in a 
> PENDING state until the enclave accepts the page. An access to the page 
> in PENDING state would result in a page fault.
> 
> 
> > I see a lot of text in the commit message but zero mentions about EPCM
> > expect this one sudden mention about PENDING field without attaching
> > it to anything concrete.
> 
> My apologies - I will add this to this changelog. This matches your 
> request to describe the __eaug() wrapper introduced in patch 02/25. 
> Would you like me to duplicate this information here and in that patch 
> (a new patch dedicated to the __eaug() wrapper) or would you be ok if I 
> introduce the wrappers all together briefly as in the example you 
> provide and then detail the flows where the wrappers are used - like 
> this patch?

I think it would be a good place to describe these details in 02/25,
and skip them in rest of the patches.

/Jarkko