From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 57095C433F5 for ; Mon, 7 Mar 2022 13:27:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240564AbiCGN14 (ORCPT ); Mon, 7 Mar 2022 08:27:56 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58766 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S242853AbiCGN1r (ORCPT ); Mon, 7 Mar 2022 08:27:47 -0500 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 646818BE17; Mon, 7 Mar 2022 05:26:53 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 0090D6121B; Mon, 7 Mar 2022 13:26:53 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id B4D7FC340E9; Mon, 7 Mar 2022 13:26:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1646659612; bh=qX81leV8bx9GvO9cEY2kcfusSYlXGTfvKVJTjhs6u1k=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=EC5A4Xv6QJzgcsYzCWZkTnri1S1yMA6bREkS5X2l9zN2F1BasfM3p9xw1HQ5LfMcP Nwtut30wUo1NePVEIVCcgq6s0teloneCRYm3xr9DLbDZUahN5RvHxiqHLLp2j1kgWE SnJUZO1FhecuEYalCzLn+5OK2E+6s07YlZ1YRsy9+eY7jdeC76jyH/Yz3Ie2bmIRIR GblNgqmdg0SQr5GjqMSzE8Sc8CpWRwT4T0PJp+RCi9fKYhZYMb6/ZlLkVIA1MtvFCj 2kVaAxYDvdR3laIZVSikufQ4LXJsMAB/Spv2b3PZVzr+4T2DD0EVIelk6vWCBOcCid aVATXTZ8VVvmQ== Date: Mon, 7 Mar 2022 15:26:11 +0200 From: Jarkko Sakkinen To: Matthew Wilcox Cc: Greg Kroah-Hartman , linux-mm@kvack.org, Dave Hansen , Nathaniel McCallum , Reinette Chatre , Andrew Morton , linux-sgx@vger.kernel.org, linux-kernel@vger.kernel.org, Florian Fainelli , Thomas Bogendoerfer , Matthew Auld , Thomas =?iso-8859-1?Q?Hellstr=F6m?= , Daniel Vetter , Jason Ekstrand , Chris Wilson , Maarten Lankhorst , Tvrtko Ursulin , Shakeel Butt , Vasily Averin , zhangyiru , Alexander Mikhalitsyn , Alexey Gladkov , linux-mips@vger.kernel.org, intel-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org, codalist@coda.cs.cmu.edu, linux-unionfs@vger.kernel.org, linux-fsdevel@vger.kernel.org Subject: Re: [PATCH RFC 1/3] mm: Add f_ops->populate() Message-ID: References: <20220306053211.135762-1-jarkko@kernel.org> <20220306053211.135762-2-jarkko@kernel.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: linux-unionfs@vger.kernel.org On Mon, Mar 07, 2022 at 03:16:57PM +0200, Jarkko Sakkinen wrote: > On Sun, Mar 06, 2022 at 10:43:31PM +0000, Matthew Wilcox wrote: > > On Sun, Mar 06, 2022 at 07:02:57PM +0200, Jarkko Sakkinen wrote: > > > So can I conclude from this that in general having populate available for > > > device memory is something horrid, or just the implementation path? > > > > You haven't even attempted to explain what the problem is you're trying > > to solve. You've shown up with some terrible code and said "Hey, is > > this a good idea". No, no, it's not. > > The problem is that in order to include memory to enclave, which is > essentially a reserved address range processes virtual address space > there's two steps into it: > > 1. Host side (kernel) does ENCLS[EAUG] to request a new page to be > added to the enclave. > 2. Enclave accepts request with ENCLU[EACCEPT] or ENCLU[EACCEPTCOPY]. > > In the current SGX2 patch set this taken care by the page fault > handler. I.e. the enclave calls ENCLU[EACCEPT] for an empty address > and the #PF handler then does EAUG for a single page. > > So if you want to process a batch of pages this generates O(n) > round-trips. > > So if there was a way pre-do a batch of EAUG's, that would allow > to load data to the enclave without causing page faults happening > constantly. > > One solution for this simply add ioctl: > > https://lore.kernel.org/linux-sgx/YiLRBglTEbu8cHP9@iki.fi/T/#m195ec84bf85614a140abeee245c5118c22ace8f3 > > But in practice when you wanted to use it, you would setup the > parameters so that they match the mmap() range. So for pratical > user space API having mmap() take care of this would be much more > lean option. For something like Graphene [1] the lazy #PF based option is probably a way to go. For wasm runtime that we're doing in Enarx [2] we get better performance by having something like this. I.e. we most of the time take as much as we use. [1] https://github.com/gramineproject/graphene [2] https://enarx.dev/ BR, Jarkko From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 978B9C433F5 for ; Mon, 7 Mar 2022 13:26:55 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id AF47D10F8BB; Mon, 7 Mar 2022 13:26:54 +0000 (UTC) Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by gabe.freedesktop.org (Postfix) with ESMTPS id 9DA1D10F8BB; Mon, 7 Mar 2022 13:26:53 +0000 (UTC) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 043D96122C; Mon, 7 Mar 2022 13:26:53 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id B4D7FC340E9; Mon, 7 Mar 2022 13:26:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1646659612; bh=qX81leV8bx9GvO9cEY2kcfusSYlXGTfvKVJTjhs6u1k=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=EC5A4Xv6QJzgcsYzCWZkTnri1S1yMA6bREkS5X2l9zN2F1BasfM3p9xw1HQ5LfMcP Nwtut30wUo1NePVEIVCcgq6s0teloneCRYm3xr9DLbDZUahN5RvHxiqHLLp2j1kgWE SnJUZO1FhecuEYalCzLn+5OK2E+6s07YlZ1YRsy9+eY7jdeC76jyH/Yz3Ie2bmIRIR GblNgqmdg0SQr5GjqMSzE8Sc8CpWRwT4T0PJp+RCi9fKYhZYMb6/ZlLkVIA1MtvFCj 2kVaAxYDvdR3laIZVSikufQ4LXJsMAB/Spv2b3PZVzr+4T2DD0EVIelk6vWCBOcCid aVATXTZ8VVvmQ== Date: Mon, 7 Mar 2022 15:26:11 +0200 From: Jarkko Sakkinen To: Matthew Wilcox Subject: Re: [PATCH RFC 1/3] mm: Add f_ops->populate() Message-ID: References: <20220306053211.135762-1-jarkko@kernel.org> <20220306053211.135762-2-jarkko@kernel.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: zhangyiru , Daniel Vetter , Dave Hansen , dri-devel@lists.freedesktop.org, Chris Wilson , linux-mm@kvack.org, Jason Ekstrand , Alexander Mikhalitsyn , Florian Fainelli , linux-unionfs@vger.kernel.org, codalist@coda.cs.cmu.edu, Matthew Auld , Vasily Averin , Thomas =?iso-8859-1?Q?Hellstr=F6m?= , intel-gfx@lists.freedesktop.org, linux-mips@vger.kernel.org, Shakeel Butt , Reinette Chatre , linux-sgx@vger.kernel.org, Thomas Bogendoerfer , Nathaniel McCallum , Greg Kroah-Hartman , Tvrtko Ursulin , linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, Andrew Morton , Alexey Gladkov Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" On Mon, Mar 07, 2022 at 03:16:57PM +0200, Jarkko Sakkinen wrote: > On Sun, Mar 06, 2022 at 10:43:31PM +0000, Matthew Wilcox wrote: > > On Sun, Mar 06, 2022 at 07:02:57PM +0200, Jarkko Sakkinen wrote: > > > So can I conclude from this that in general having populate available for > > > device memory is something horrid, or just the implementation path? > > > > You haven't even attempted to explain what the problem is you're trying > > to solve. You've shown up with some terrible code and said "Hey, is > > this a good idea". No, no, it's not. > > The problem is that in order to include memory to enclave, which is > essentially a reserved address range processes virtual address space > there's two steps into it: > > 1. Host side (kernel) does ENCLS[EAUG] to request a new page to be > added to the enclave. > 2. Enclave accepts request with ENCLU[EACCEPT] or ENCLU[EACCEPTCOPY]. > > In the current SGX2 patch set this taken care by the page fault > handler. I.e. the enclave calls ENCLU[EACCEPT] for an empty address > and the #PF handler then does EAUG for a single page. > > So if you want to process a batch of pages this generates O(n) > round-trips. > > So if there was a way pre-do a batch of EAUG's, that would allow > to load data to the enclave without causing page faults happening > constantly. > > One solution for this simply add ioctl: > > https://lore.kernel.org/linux-sgx/YiLRBglTEbu8cHP9@iki.fi/T/#m195ec84bf85614a140abeee245c5118c22ace8f3 > > But in practice when you wanted to use it, you would setup the > parameters so that they match the mmap() range. So for pratical > user space API having mmap() take care of this would be much more > lean option. For something like Graphene [1] the lazy #PF based option is probably a way to go. For wasm runtime that we're doing in Enarx [2] we get better performance by having something like this. I.e. we most of the time take as much as we use. [1] https://github.com/gramineproject/graphene [2] https://enarx.dev/ BR, Jarkko From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C367CC433F5 for ; Tue, 8 Mar 2022 12:51:01 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id DE6E710E511; Tue, 8 Mar 2022 12:50:53 +0000 (UTC) Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by gabe.freedesktop.org (Postfix) with ESMTPS id 9DA1D10F8BB; Mon, 7 Mar 2022 13:26:53 +0000 (UTC) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 043D96122C; Mon, 7 Mar 2022 13:26:53 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id B4D7FC340E9; Mon, 7 Mar 2022 13:26:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1646659612; bh=qX81leV8bx9GvO9cEY2kcfusSYlXGTfvKVJTjhs6u1k=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=EC5A4Xv6QJzgcsYzCWZkTnri1S1yMA6bREkS5X2l9zN2F1BasfM3p9xw1HQ5LfMcP Nwtut30wUo1NePVEIVCcgq6s0teloneCRYm3xr9DLbDZUahN5RvHxiqHLLp2j1kgWE SnJUZO1FhecuEYalCzLn+5OK2E+6s07YlZ1YRsy9+eY7jdeC76jyH/Yz3Ie2bmIRIR GblNgqmdg0SQr5GjqMSzE8Sc8CpWRwT4T0PJp+RCi9fKYhZYMb6/ZlLkVIA1MtvFCj 2kVaAxYDvdR3laIZVSikufQ4LXJsMAB/Spv2b3PZVzr+4T2DD0EVIelk6vWCBOcCid aVATXTZ8VVvmQ== Date: Mon, 7 Mar 2022 15:26:11 +0200 From: Jarkko Sakkinen To: Matthew Wilcox Message-ID: References: <20220306053211.135762-1-jarkko@kernel.org> <20220306053211.135762-2-jarkko@kernel.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Mailman-Approved-At: Tue, 08 Mar 2022 12:50:52 +0000 Subject: Re: [Intel-gfx] [PATCH RFC 1/3] mm: Add f_ops->populate() X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: zhangyiru , Daniel Vetter , Dave Hansen , dri-devel@lists.freedesktop.org, Chris Wilson , linux-mm@kvack.org, Alexander Mikhalitsyn , Florian Fainelli , linux-unionfs@vger.kernel.org, codalist@coda.cs.cmu.edu, Matthew Auld , Vasily Averin , Thomas =?iso-8859-1?Q?Hellstr=F6m?= , intel-gfx@lists.freedesktop.org, linux-mips@vger.kernel.org, Shakeel Butt , Reinette Chatre , linux-sgx@vger.kernel.org, Thomas Bogendoerfer , Nathaniel McCallum , Greg Kroah-Hartman , linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, Andrew Morton , Alexey Gladkov Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" On Mon, Mar 07, 2022 at 03:16:57PM +0200, Jarkko Sakkinen wrote: > On Sun, Mar 06, 2022 at 10:43:31PM +0000, Matthew Wilcox wrote: > > On Sun, Mar 06, 2022 at 07:02:57PM +0200, Jarkko Sakkinen wrote: > > > So can I conclude from this that in general having populate available for > > > device memory is something horrid, or just the implementation path? > > > > You haven't even attempted to explain what the problem is you're trying > > to solve. You've shown up with some terrible code and said "Hey, is > > this a good idea". No, no, it's not. > > The problem is that in order to include memory to enclave, which is > essentially a reserved address range processes virtual address space > there's two steps into it: > > 1. Host side (kernel) does ENCLS[EAUG] to request a new page to be > added to the enclave. > 2. Enclave accepts request with ENCLU[EACCEPT] or ENCLU[EACCEPTCOPY]. > > In the current SGX2 patch set this taken care by the page fault > handler. I.e. the enclave calls ENCLU[EACCEPT] for an empty address > and the #PF handler then does EAUG for a single page. > > So if you want to process a batch of pages this generates O(n) > round-trips. > > So if there was a way pre-do a batch of EAUG's, that would allow > to load data to the enclave without causing page faults happening > constantly. > > One solution for this simply add ioctl: > > https://lore.kernel.org/linux-sgx/YiLRBglTEbu8cHP9@iki.fi/T/#m195ec84bf85614a140abeee245c5118c22ace8f3 > > But in practice when you wanted to use it, you would setup the > parameters so that they match the mmap() range. So for pratical > user space API having mmap() take care of this would be much more > lean option. For something like Graphene [1] the lazy #PF based option is probably a way to go. For wasm runtime that we're doing in Enarx [2] we get better performance by having something like this. I.e. we most of the time take as much as we use. [1] https://github.com/gramineproject/graphene [2] https://enarx.dev/ BR, Jarkko