Re: [PATCH 13/25] x86/sgx: Support adding of pages to initialized enclave

[Reinette Chatre <reinette.chatre@intel.com>]

Hi Jarkko,

On 12/4/2021 3:13 PM, Jarkko Sakkinen wrote:
> "to initialize" -> "to an initialized"

Will do.


> 
> On Wed, Dec 01, 2021 at 11:23:11AM -0800, Reinette Chatre wrote:
>> With SGX1 an enclave needs to be created with its maximum memory demands
>> allocated. Pages cannot be added to an enclave after it is initialized.
>> SGX2 introduces a new function, ENCLS[EAUG], that can be used to add
>> pages to an initialized enclave. With SGX2 the enclave still needs to
>> set aside address space for its maximum memory demands during enclave
>> creation, but all pages need not be added before enclave initialization.
>> Pages can be added during enclave runtime.
>>
>> Add support for dynamically adding pages to an initialized enclave,
>> architecturally limited to RW permission. Add pages via the page fault
>> handler at the time an enclave address without a backing enclave page
>> is accessed, potentially directly reclaiming pages if no free pages
>> are available.
>>
>> The enclave is still required to run ENCLU[EACCEPT] on the page before
>> it can be used. A useful flow is for the enclave to run ENCLU[EACCEPT]
>> on an uninitialized address. This will trigger the page fault handler
>> that will add the enclave page and return execution to the enclave to
>> repeat the ENCLU[EACCEPT] instruction, this time successful.
>>
>> If the enclave accesses an uninitialized address in another way, for
>> example by expanding the enclave stack to a page that has not yet been
>> added, then the page fault handler would add the page on the first
>> write but upon returning to the enclave the instruction that triggered
>> the page fault would be repeated and since ENCLU[EACCEPT] was not run
>> yet it would trigger a second page fault, this time with the SGX flag
>> set in the page fault error code. This can only be recovered by entering
>> the enclave again and directly running the ENCLU[EACCEPT] instruction on
>> the now initialized address.
>>
>> Accessing an uninitialized address from outside the enclave also triggers
>> this flow but the page will remain in PENDING state until accepted from
>> within the enclave.
> 
> What does it mean being in PENDING state, and more imporantly, what is
> PENDING state? What does a memory access within enclave cause when it
> touch a page within this state?

The PENDING state is the enclave page state from the SGX hardware's 
perspective. The OS uses the ENCLS[EAUG] SGX2 function to add a new page 
to the enclave but from the SGX hardware's perspective it would be in a 
PENDING state until the enclave accepts the page. An access to the page 
in PENDING state would result in a page fault.


> I see a lot of text in the commit message but zero mentions about EPCM
> expect this one sudden mention about PENDING field without attaching
> it to anything concrete.

My apologies - I will add this to this changelog. This matches your 
request to describe the __eaug() wrapper introduced in patch 02/25. 
Would you like me to duplicate this information here and in that patch 
(a new patch dedicated to the __eaug() wrapper) or would you be ok if I 
introduce the wrappers all together briefly as in the example you 
provide and then detail the flows where the wrappers are used - like 
this patch?

Reinette