From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 01CE4C433EF for ; Wed, 9 Mar 2022 23:43:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238156AbiCIXoe (ORCPT ); Wed, 9 Mar 2022 18:44:34 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40558 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229897AbiCIXo2 (ORCPT ); Wed, 9 Mar 2022 18:44:28 -0500 Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9AE89119415; Wed, 9 Mar 2022 15:43:28 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 48820B82447; Wed, 9 Mar 2022 23:43:27 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 825B8C340EE; Wed, 9 Mar 2022 23:43:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1646869405; bh=UdPDKptnwZ1SgF/Ief1UzCmFBhIpSC7zBQ8ZwnyvOcA=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=Z3e+uabSlkJEgLA2lsBn9GohSDanxFS41c7vT+rNkmUEB4IOoA/6y0NNktr19ZuDN Ff10o22zXImsGfFiBGJzKMfTWViGCBI+hGOv6XnThXjMKIZIF2iCoVZ01JnhblaLjt T5LetExE7wBjvC7Cc/B3V3Op8iUK6s+FOWMdRN/JctpqLfVKYAdAG0MoV0Ejwj5YaZ 6EVnYmXRkjRAYmlsz9RBYtPwnlcLhpJvqGF4Ygs2ABMqH1l8jjJ2DfKQ4IUhHyCUyP WZ5U+ZPloB3pcFfcKSz9mTQZM686I/k1wI8K/CpRfjZmL3LitdiWtw64W3JTpDMzOz VoRpx3Wjw9xCQ== Date: Thu, 10 Mar 2022 01:42:41 +0200 From: Jarkko Sakkinen To: Reinette Chatre Cc: linux-sgx@vger.kernel.org, Nathaniel McCallum , Dave Hansen , Thomas Gleixner , Ingo Molnar , Borislav Petkov , "maintainer:X86 ARCHITECTURE (32-BIT AND 64-BIT)" , "H. Peter Anvin" , "open list:X86 ARCHITECTURE (32-BIT AND 64-BIT)" Subject: Re: [RFC PATCH v2.1 14/30] x86/sgx: Support restricting of enclave page permissions Message-ID: References: <20220304093524.397485-1-jarkko@kernel.org> <20220304093524.397485-14-jarkko@kernel.org> <5e89a3be-0760-b1b4-7693-2f3d9ac5066b@intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Thu, Mar 10, 2022 at 01:35:53AM +0200, Jarkko Sakkinen wrote: > On Wed, Mar 09, 2022 at 08:59:42AM -0800, Reinette Chatre wrote: > > Hi Jarkko, > > > > On 3/9/2022 1:35 AM, Jarkko Sakkinen wrote: > > > On Wed, Mar 09, 2022 at 10:52:22AM +0200, Jarkko Sakkinen wrote: > > >> On Fri, Mar 04, 2022 at 11:35:08AM +0200, Jarkko Sakkinen wrote: > > >>> +#define SGX_IOC_ENCLAVE_RESTRICT_PERMISSIONS \ > > >>> + _IOWR(SGX_MAGIC, 0x05, struct sgx_enclave_restrict_perm) > > >> > > >> What if this was replaced with just SGX_IOC_ENCLAVE_RESET_PAGES, which > > >> would simply do EMODPR with PROT_NONE? The main ingredient of EMODPR is to > > >> flush out the TLB's, and move a page to pending state, which cannot be done > > >> from inside the enclave. > > > > I see the main ingredient as running EMODPR to restrict the EPCM permissions. If > > the user wants to use SGX_IOC_ENCLAVE_RESTRICT_PERMISSIONS just to flush TLB it is > > already possible since attempting to use EMODPR to relax permissions does not > > change any permissions (although it still sets EPCM.PR) but yet will still > > flush the TLB. > > It's not just to flush the TLB. It also resets permissions to zero from > which it is easy to set the exact permissions with EMODPE. > > > Even so, you have a very good point that removing SGX_IOC_ENCLAVE_RELAX_PERMISSIONS > > removes the ability for users to flush the TLB after an EMODPE. If there are > > thus PTEs present at the time the user runs EMODPE the pages would not be > > accessible with the new permissions. > > > > Repurposing SGX_IOC_ENCLAVE_RESTRICT_PERMISSIONS with PROT_NONE to accomplish > > this is not efficient because: > > - For the OS to flush the TLB the enclave pages need not be in the EPC but > > in order to run EMODPR the enclave page needs to be in the EPC. In an > > oversubscribed environment running EMODPR unnecessarily can thus introduce > > a significant delay. Please see the performance comparison I did in > > https://lore.kernel.org/linux-sgx/77e81306-6b03-4b09-2df2-48e09e2e79d5@intel.com/ > > The test shows that running EMODPR unnecessarily can be orders of magnitude slower. > > - Running EMODPR on an enclave page sets the EPCM.PR bin in the enclave page > > that needs to be cleared with an EACCEPT from within the enclave. > > If the user just wants to reset the TLB after running EMODPE then it should > > not be necessary to run EACCEPT again to reset EPCM.PR. > > > > Resetting the TLB is exactly what SGX_IOC_ENCLAVE_RELAX_PERMISSIONS did in an > > efficient way - it is quick (no need to load pages into EPC) and it does not > > require EACCEPT to clear EPCM.PR. > > > > It looks like we need SGX_IOC_ENCLAVE_RELAX_PERMISSIONS back. We could > > rename it to SGX_IOC_ENCLAVE_RESET_PAGES if you prefer. > > Please do not add it. We do not have any use for it. It's not only used > to flush TLB's so it would not do any good. I just use it with fixed > PROT_NONE permissions. > > > >> It's there because of microarchitecture constraints, and less so to work as > > >> a reasonable permission control mechanism (actually it does terrible job on > > >> that side and only confuses). > > >> > > >> Once you have this magic TLB reset button in place you can just do one > > >> EACCEPT and EMODPE inside the enclave and you're done. > > >> > > >> This is also kind of atomic in the sense that EACCEPT free's a page with no > > >> rights so no misuse can happend before EMODPE has tuned EPCM. > > > > > > I wonder if this type of pattern could be made work out for Graphene: > > > > > > 1. SGX_IOC_ENCLAVE_RESET_PAGES > > > 2. EACCEPT + EMODPE > > > > > > This kind of delivers EMODP that everyone has been looking for. > > > > EACCEPT will result in page table entries created for the enclave page. EMODPE > > will be able to relax the permissions but TLB flush would be required to > > access the page with the new permissions. SGX_IOC_ENCLAVE_RELAX_PERMISSIONS > > (renamed to SGX_IOC_ENCLAVE_RESET_PAGES?) that does just a TLB flush is > > required to be after EMODPE. > > For EMODPE TLB flush is not required. I even verified this from Mark > Shanahan. And since access rights are zero, the page cannot be > deferenced by threads before EMODPE. I'm fine of course keeping SGX_IOC_ENCLAVE_RESTRICT_PERMISSIONS as it is too. It's at least future-proof that way. BR, Jarkko