From: Jarkko Sakkinen <jarkko@kernel.org>
To: Reinette Chatre <reinette.chatre@intel.com>,
dave.hansen@linux.intel.com, tglx@linutronix.de, bp@alien8.de,
luto@kernel.org, mingo@redhat.com, linux-sgx@vger.kernel.org,
x86@kernel.org
Cc: seanjc@google.com, kai.huang@intel.com, cathy.zhang@intel.com,
cedric.xing@intel.com, haitao.huang@intel.com,
mark.shanahan@intel.com, hpa@zytor.com,
linux-kernel@vger.kernel.org
Subject: Re: [PATCH V3 15/30] x86/sgx: Support adding of pages to an initialized enclave
Date: Tue, 05 Apr 2022 08:05:54 +0300 [thread overview]
Message-ID: <cd2fc405d541eec1cc80e6fbd511ab978c37aa94.camel@kernel.org> (raw)
In-Reply-To: <50b9da1f491eb31d9b038afa0d75871965f474eb.1648847675.git.reinette.chatre@intel.com>
On Mon, 2022-04-04 at 09:49 -0700, Reinette Chatre wrote:
> With SGX1 an enclave needs to be created with its maximum memory demands
> allocated. Pages cannot be added to an enclave after it is initialized.
> SGX2 introduces a new function, ENCLS[EAUG], that can be used to add
> pages to an initialized enclave. With SGX2 the enclave still needs to
> set aside address space for its maximum memory demands during enclave
> creation, but all pages need not be added before enclave initialization.
> Pages can be added during enclave runtime.
>
> Add support for dynamically adding pages to an initialized enclave,
> architecturally limited to RW permission at creation but allowed to
> obtain RWX permissions after enclave runs EMODPE. Add pages via the
> page fault handler at the time an enclave address without a backing
> enclave page is accessed, potentially directly reclaiming pages if
> no free pages are available.
>
> The enclave is still required to run ENCLU[EACCEPT] on the page before
> it can be used. A useful flow is for the enclave to run ENCLU[EACCEPT]
> on an uninitialized address. This will trigger the page fault handler
> that will add the enclave page and return execution to the enclave to
> repeat the ENCLU[EACCEPT] instruction, this time successful.
>
> If the enclave accesses an uninitialized address in another way, for
> example by expanding the enclave stack to a page that has not yet been
> added, then the page fault handler would add the page on the first
> write but upon returning to the enclave the instruction that triggered
> the page fault would be repeated and since ENCLU[EACCEPT] was not run
> yet it would trigger a second page fault, this time with the SGX flag
> set in the page fault error code. This can only be recovered by entering
> the enclave again and directly running the ENCLU[EACCEPT] instruction on
> the now initialized address.
>
> Accessing an uninitialized address from outside the enclave also
> triggers this flow but the page will remain inaccessible (access will
> result in #PF) until accepted from within the enclave via
> ENCLU[EACCEPT].
>
> Signed-off-by: Reinette Chatre <reinette.chatre@intel.com>
> ---
> Changes since V2:
> - Remove runtime tracking of EPCM permissions
> (sgx_encl_page->vm_run_prot_bits) (Jarkko).
> - Move export of sgx_encl_{grow,shrink}() to separate patch. (Jarkko)
> - Use sgx_encl_page_alloc(). (Jarkko)
> - Set max allowed permissions to be RWX (Jarkko). Update changelog
> to indicate the change and use comment in code as
> created by Jarkko in:
> https://lore.kernel.org/linux-sgx/20220306053211.135762-4-jarkko@kernel.org
> - Do not set protection bits but let it be inherited by VMA (Jarkko)
>
> Changes since V1:
> - Fix subject line "to initialized" -> "to an initialized" (Jarkko).
> - Move text about hardware's PENDING state to the patch that introduces
> the ENCLS[EAUG] wrapper (Jarkko).
> - Ensure kernel-doc uses brackets when referring to function.
>
> arch/x86/kernel/cpu/sgx/encl.c | 124 +++++++++++++++++++++++++++++++++
> 1 file changed, 124 insertions(+)
>
> diff --git a/arch/x86/kernel/cpu/sgx/encl.c b/arch/x86/kernel/cpu/sgx/encl.c
> index 546423753e4c..fa4f947f8496 100644
> --- a/arch/x86/kernel/cpu/sgx/encl.c
> +++ b/arch/x86/kernel/cpu/sgx/encl.c
> @@ -194,6 +194,119 @@ struct sgx_encl_page *sgx_encl_load_page(struct sgx_encl *encl,
> return __sgx_encl_load_page(encl, entry);
> }
>
> +/**
> + * sgx_encl_eaug_page() - Dynamically add page to initialized enclave
> + * @vma: VMA obtained from fault info from where page is accessed
> + * @encl: enclave accessing the page
> + * @addr: address that triggered the page fault
> + *
> + * When an initialized enclave accesses a page with no backing EPC page
> + * on a SGX2 system then the EPC can be added dynamically via the SGX2
> + * ENCLS[EAUG] instruction.
> + *
> + * Returns: Appropriate vm_fault_t: VM_FAULT_NOPAGE when PTE was installed
> + * successfully, VM_FAULT_SIGBUS or VM_FAULT_OOM as error otherwise.
> + */
> +static vm_fault_t sgx_encl_eaug_page(struct vm_area_struct *vma,
> + struct sgx_encl *encl, unsigned long addr)
> +{
> + struct sgx_pageinfo pginfo = {0};
> + struct sgx_encl_page *encl_page;
> + struct sgx_epc_page *epc_page;
> + struct sgx_va_page *va_page;
> + unsigned long phys_addr;
> + u64 secinfo_flags;
> + vm_fault_t vmret;
> + int ret;
> +
> + if (!test_bit(SGX_ENCL_INITIALIZED, &encl->flags))
> + return VM_FAULT_SIGBUS;
> +
> + /*
> + * Ignore internal permission checking for dynamically added pages.
> + * They matter only for data added during the pre-initialization
> + * phase. The enclave decides the permissions by the means of
> + * EACCEPT, EACCEPTCOPY and EMODPE.
> + */
> + secinfo_flags = SGX_SECINFO_R | SGX_SECINFO_W | SGX_SECINFO_X;
> + encl_page = sgx_encl_page_alloc(encl, addr - encl->base, secinfo_flags);
> + if (IS_ERR(encl_page))
> + return VM_FAULT_OOM;
> +
> + epc_page = sgx_alloc_epc_page(encl_page, true);
> + if (IS_ERR(epc_page)) {
> + kfree(encl_page);
> + return VM_FAULT_SIGBUS;
> + }
> +
> + va_page = sgx_encl_grow(encl);
> + if (IS_ERR(va_page)) {
> + ret = PTR_ERR(va_page);
> + goto err_out_free;
> + }
> +
> + mutex_lock(&encl->lock);
> +
> + /*
> + * Copy comment from sgx_encl_add_page() to maintain guidance in
> + * this similar flow:
> + * Adding to encl->va_pages must be done under encl->lock. Ditto for
> + * deleting (via sgx_encl_shrink()) in the error path.
> + */
> + if (va_page)
> + list_add(&va_page->list, &encl->va_pages);
> +
> + ret = xa_insert(&encl->page_array, PFN_DOWN(encl_page->desc),
> + encl_page, GFP_KERNEL);
> + /*
> + * If ret == -EBUSY then page was created in another flow while
> + * running without encl->lock
> + */
> + if (ret)
> + goto err_out_unlock;
> +
> + pginfo.secs = (unsigned long)sgx_get_epc_virt_addr(encl->secs.epc_page);
> + pginfo.addr = encl_page->desc & PAGE_MASK;
> + pginfo.metadata = 0;
> +
> + ret = __eaug(&pginfo, sgx_get_epc_virt_addr(epc_page));
> + if (ret)
> + goto err_out;
> +
> + encl_page->encl = encl;
> + encl_page->epc_page = epc_page;
> + encl_page->type = SGX_PAGE_TYPE_REG;
> + encl->secs_child_cnt++;
> +
> + sgx_mark_page_reclaimable(encl_page->epc_page);
> +
> + phys_addr = sgx_get_epc_phys_addr(epc_page);
> + /*
> + * Do not undo everything when creating PTE entry fails - next #PF
> + * would find page ready for a PTE.
> + */
> + vmret = vmf_insert_pfn(vma, addr, PFN_DOWN(phys_addr));
> + if (vmret != VM_FAULT_NOPAGE) {
> + mutex_unlock(&encl->lock);
> + return VM_FAULT_SIGBUS;
> + }
> + mutex_unlock(&encl->lock);
> + return VM_FAULT_NOPAGE;
> +
> +err_out:
> + xa_erase(&encl->page_array, PFN_DOWN(encl_page->desc));
> +
> +err_out_unlock:
> + sgx_encl_shrink(encl, va_page);
> + mutex_unlock(&encl->lock);
> +
> +err_out_free:
> + sgx_encl_free_epc_page(epc_page);
> + kfree(encl_page);
> +
> + return VM_FAULT_SIGBUS;
> +}
> +
> static vm_fault_t sgx_vma_fault(struct vm_fault *vmf)
> {
> unsigned long addr = (unsigned long)vmf->address;
> @@ -213,6 +326,17 @@ static vm_fault_t sgx_vma_fault(struct vm_fault *vmf)
> if (unlikely(!encl))
> return VM_FAULT_SIGBUS;
>
> + /*
> + * The page_array keeps track of all enclave pages, whether they
> + * are swapped out or not. If there is no entry for this page and
> + * the system supports SGX2 then it is possible to dynamically add
> + * a new enclave page. This is only possible for an initialized
> + * enclave that will be checked for right away.
> + */
> + if (cpu_feature_enabled(X86_FEATURE_SGX2) &&
> + (!xa_load(&encl->page_array, PFN_DOWN(addr))))
> + return sgx_encl_eaug_page(vma, encl, addr);
> +
> mutex_lock(&encl->lock);
>
> entry = sgx_encl_load_page_in_vma(encl, addr, vma->vm_flags);
Reviewed-by: Jarkko Sakkinen <jarkko@kernel.org>
BR, Jarkko
next prev parent reply other threads:[~2022-04-05 5:04 UTC|newest]
Thread overview: 79+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-04-04 16:49 [PATCH V3 00/30] x86/sgx and selftests/sgx: Support SGX2 Reinette Chatre
2022-04-04 16:49 ` [PATCH V3 01/30] x86/sgx: Add short descriptions to ENCLS wrappers Reinette Chatre
2022-04-05 6:52 ` Jarkko Sakkinen
2022-04-04 16:49 ` [PATCH V3 02/30] x86/sgx: Add wrapper for SGX2 EMODPR function Reinette Chatre
2022-04-05 6:53 ` Jarkko Sakkinen
2022-04-04 16:49 ` [PATCH V3 03/30] x86/sgx: Add wrapper for SGX2 EMODT function Reinette Chatre
2022-04-05 6:53 ` Jarkko Sakkinen
2022-04-04 16:49 ` [PATCH V3 04/30] x86/sgx: Add wrapper for SGX2 EAUG function Reinette Chatre
2022-04-05 6:54 ` Jarkko Sakkinen
2022-04-04 16:49 ` [PATCH V3 05/30] x86/sgx: Support loading enclave page without VMA permissions check Reinette Chatre
2022-04-05 6:56 ` Jarkko Sakkinen
2022-04-04 16:49 ` [PATCH V3 06/30] x86/sgx: Export sgx_encl_ewb_cpumask() Reinette Chatre
2022-04-05 6:56 ` Jarkko Sakkinen
2022-04-04 16:49 ` [PATCH V3 07/30] x86/sgx: Rename sgx_encl_ewb_cpumask() as sgx_encl_cpumask() Reinette Chatre
2022-04-05 6:57 ` Jarkko Sakkinen
2022-04-04 16:49 ` [PATCH V3 08/30] x86/sgx: Move PTE zap code to new sgx_zap_enclave_ptes() Reinette Chatre
2022-04-05 6:59 ` Jarkko Sakkinen
2022-04-04 16:49 ` [PATCH V3 09/30] x86/sgx: Make sgx_ipi_cb() available internally Reinette Chatre
2022-04-05 6:59 ` Jarkko Sakkinen
2022-04-04 16:49 ` [PATCH V3 10/30] x86/sgx: Create utility to validate user provided offset and length Reinette Chatre
2022-04-05 7:00 ` Jarkko Sakkinen
2022-04-04 16:49 ` [PATCH V3 11/30] x86/sgx: Keep record of SGX page type Reinette Chatre
2022-04-05 7:00 ` Jarkko Sakkinen
2022-04-04 16:49 ` [PATCH V3 12/30] x86/sgx: Export sgx_encl_{grow,shrink}() Reinette Chatre
2022-04-05 7:04 ` Jarkko Sakkinen
2022-04-04 16:49 ` [PATCH V3 13/30] x86/sgx: Export sgx_encl_page_alloc() Reinette Chatre
2022-04-04 16:49 ` [PATCH V3 14/30] x86/sgx: Support restricting of enclave page permissions Reinette Chatre
2022-04-05 5:03 ` Jarkko Sakkinen
2022-04-05 5:07 ` Jarkko Sakkinen
2022-04-05 13:40 ` Jarkko Sakkinen
2022-04-05 14:19 ` Jarkko Sakkinen
2022-04-05 14:27 ` Jarkko Sakkinen
2022-04-05 14:52 ` Jarkko Sakkinen
2022-04-05 16:49 ` Reinette Chatre
2022-04-05 18:39 ` Jarkko Sakkinen
2022-04-05 18:59 ` Reinette Chatre
2022-04-06 7:30 ` Jarkko Sakkinen
2022-04-06 17:51 ` Reinette Chatre
2022-04-05 16:40 ` Reinette Chatre
2022-04-04 16:49 ` [PATCH V3 15/30] x86/sgx: Support adding of pages to an initialized enclave Reinette Chatre
2022-04-05 5:05 ` Jarkko Sakkinen [this message]
2022-04-05 10:03 ` Jarkko Sakkinen
2022-04-06 7:37 ` Jarkko Sakkinen
2022-04-06 22:42 ` Reinette Chatre
2022-04-04 16:49 ` [PATCH V3 16/30] x86/sgx: Tighten accessible memory range after enclave initialization Reinette Chatre
2022-04-05 7:05 ` Jarkko Sakkinen
2022-04-04 16:49 ` [PATCH V3 17/30] x86/sgx: Support modifying SGX page type Reinette Chatre
2022-04-05 7:06 ` Jarkko Sakkinen
2022-04-05 15:34 ` Jarkko Sakkinen
2022-04-05 17:05 ` Reinette Chatre
2022-04-05 18:41 ` Jarkko Sakkinen
2022-04-05 18:59 ` Reinette Chatre
2022-04-06 7:32 ` Jarkko Sakkinen
2022-04-06 17:50 ` Reinette Chatre
2022-04-04 16:49 ` [PATCH V3 18/30] x86/sgx: Support complete page removal Reinette Chatre
2022-04-05 7:08 ` Jarkko Sakkinen
2022-04-04 16:49 ` [PATCH V3 19/30] x86/sgx: Free up EPC pages directly to support large page ranges Reinette Chatre
2022-04-05 7:11 ` Jarkko Sakkinen
2022-04-05 17:13 ` Reinette Chatre
2022-04-05 17:25 ` Dave Hansen
2022-04-06 6:35 ` Jarkko Sakkinen
2022-04-06 17:50 ` Reinette Chatre
2022-04-05 18:42 ` Jarkko Sakkinen
2022-04-05 19:56 ` Reinette Chatre
2022-04-04 16:49 ` [PATCH V3 20/30] Documentation/x86: Introduce enclave runtime management section Reinette Chatre
2022-04-04 16:49 ` [PATCH V3 21/30] selftests/sgx: Add test for EPCM permission changes Reinette Chatre
2022-04-05 7:02 ` Jarkko Sakkinen
2022-04-05 7:03 ` Jarkko Sakkinen
2022-04-05 17:28 ` Reinette Chatre
2022-04-05 18:43 ` Jarkko Sakkinen
2022-04-04 16:49 ` [PATCH V3 22/30] selftests/sgx: Add test for TCS page " Reinette Chatre
2022-04-04 16:49 ` [PATCH V3 23/30] selftests/sgx: Test two different SGX2 EAUG flows Reinette Chatre
2022-04-04 16:49 ` [PATCH V3 24/30] selftests/sgx: Introduce dynamic entry point Reinette Chatre
2022-04-04 16:49 ` [PATCH V3 25/30] selftests/sgx: Introduce TCS initialization enclave operation Reinette Chatre
2022-04-04 16:49 ` [PATCH V3 26/30] selftests/sgx: Test complete changing of page type flow Reinette Chatre
2022-04-04 16:49 ` [PATCH V3 27/30] selftests/sgx: Test faulty enclave behavior Reinette Chatre
2022-04-04 16:49 ` [PATCH V3 28/30] selftests/sgx: Test invalid access to removed enclave page Reinette Chatre
2022-04-04 16:49 ` [PATCH V3 29/30] selftests/sgx: Test reclaiming of untouched page Reinette Chatre
2022-04-04 16:49 ` [PATCH V3 30/30] selftests/sgx: Page removal stress test Reinette Chatre
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=cd2fc405d541eec1cc80e6fbd511ab978c37aa94.camel@kernel.org \
--to=jarkko@kernel.org \
--cc=bp@alien8.de \
--cc=cathy.zhang@intel.com \
--cc=cedric.xing@intel.com \
--cc=dave.hansen@linux.intel.com \
--cc=haitao.huang@intel.com \
--cc=hpa@zytor.com \
--cc=kai.huang@intel.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-sgx@vger.kernel.org \
--cc=luto@kernel.org \
--cc=mark.shanahan@intel.com \
--cc=mingo@redhat.com \
--cc=reinette.chatre@intel.com \
--cc=seanjc@google.com \
--cc=tglx@linutronix.de \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).