From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 91892C4332F for ; Tue, 8 Mar 2022 10:11:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1345734AbiCHKMj (ORCPT ); Tue, 8 Mar 2022 05:12:39 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44038 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238695AbiCHKMh (ORCPT ); Tue, 8 Mar 2022 05:12:37 -0500 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 48ABB42EEE; Tue, 8 Mar 2022 02:11:27 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id C517161512; Tue, 8 Mar 2022 10:11:26 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 7F770C340EB; Tue, 8 Mar 2022 10:11:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1646734286; bh=fYSUhe3wecNdmSDhFtYlrEjcl8plDtdT7/KRcR78mg8=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=u4nsxISBKCbp0kWFZQFzaUS+y+q3Vc8eNVHhG0Dmv4Nb/JhF26ncQ7Echex+2TBWw HzESBnHl4TLrNzabSQGNB7ZbF8RfKTG8YIJq2SApT5oCxEWRfJr2l9JoJPXihK3vfQ duy0ns8W9IpIrx2oeLNdvolKx2mHVixqS4CJNaiMHhOoWN2qQZmcekG4CxoSZIyJX4 EJo+hN5kIQOs94WI2C6ml3xu/HoLZzZhctA1CyAepjY5M6e1IrYFTj5uS071+txQfv E2z0+8a/Amxg9QK2hYAnt6jFaeTl8IVajwPvjbBMhdwUfexozvzQDosql+YtSoO6QB ViNC/V8kidVXQ== Date: Tue, 8 Mar 2022 12:10:45 +0200 From: Jarkko Sakkinen To: David Laight Cc: 'Christoph Hellwig' , Matthew Wilcox , "linux-mm@kvack.org" , Dave Hansen , Nathaniel McCallum , Reinette Chatre , Andrew Morton , "linux-sgx@vger.kernel.org" , "linux-kernel@vger.kernel.org" , Florian Fainelli , Thomas Bogendoerfer , Matthew Auld , Thomas =?iso-8859-1?Q?Hellstr=F6m?= , Daniel Vetter , Jason Ekstrand , Chris Wilson , "G@iki.fi" , Maarten Lankhorst , Greg Kroah-Hartman , Tvrtko Ursulin , Vasily Averin , Shakeel Butt , Michal Hocko , zhangyiru , Alexey Gladkov , Alexander Mikhalitsyn , "linux-mips@vger.kernel.org" , "intel-gfx@lists.freedesktop.org" , "dri-devel@lists.freedesktop.org" , "codalist@coda.cs.cmu.edu" , "linux-unionfs@vger.kernel.org" , "linux-fsdevel@vger.kernel.org" Subject: Re: [PATCH RFC 0/3] MAP_POPULATE for device memory Message-ID: References: <20220306053211.135762-1-jarkko@kernel.org> <5729d03d6a174da6b66d1534ebdb1127@AcuMS.aculab.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <5729d03d6a174da6b66d1534ebdb1127@AcuMS.aculab.com> Precedence: bulk List-ID: X-Mailing-List: linux-unionfs@vger.kernel.org On Mon, Mar 07, 2022 at 10:11:19PM +0000, David Laight wrote: > From: Christoph Hellwig > > Sent: 07 March 2022 15:57 > > > > On Mon, Mar 07, 2022 at 03:29:35PM +0200, Jarkko Sakkinen wrote: > > > So what would you suggest to sort out the issue? I'm happy to go with > > > ioctl if nothing else is acceptable. > > > > PLenty of drivers treat all mmaps as if MAP_POPULATE was specified, > > typically by using (io_)remap_pfn_range. If there any reason to only > > optionally have the pre-fault semantics for sgx? If not this should > > be really simple. And if we have a real need for it to be optional > > we'll just need to find a sane way to pass that information to ->mmap. > > Is there any space in vma->vm_flags ? > > That would be better than an extra argument or function. It's very dense but I'll give a shot for callback route based on Dave's comments in this thread. I.e. use it as filter inside __mm_populate() and populate_vma_page_range(). For Enarx, which we are implementing being able to use MAP_POPULATE and get the full range EAUG'd would be best way to optimize the performance of wasm JIT (Enarx is a wasm run-time capable of running inside an SGX enclave, AMD SEV-SNP VM etc.). More so than any predictor (ra_state, madvice etc.) inside #PF handler, which have been suggested in this thread. After some research on how we implement user space, I'd rather keep the #PF handler working on a single page (EAUG a single page) and have either ioctl or MAP_POPULATE to do the batch fill. We can still "not trust the user space" i.e. the populate does not have to guarantee to do the full length since the #PF handler will then fill the holes. This was one concern in this thread but it is not hard to address. BR, Jarkko From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C58D2C433EF for ; Tue, 8 Mar 2022 10:11:30 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id C8CA110E498; Tue, 8 Mar 2022 10:11:29 +0000 (UTC) Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by gabe.freedesktop.org (Postfix) with ESMTPS id 9C1AA10E488; Tue, 8 Mar 2022 10:11:27 +0000 (UTC) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id C392F61501; Tue, 8 Mar 2022 10:11:26 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 7F770C340EB; Tue, 8 Mar 2022 10:11:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1646734286; bh=fYSUhe3wecNdmSDhFtYlrEjcl8plDtdT7/KRcR78mg8=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=u4nsxISBKCbp0kWFZQFzaUS+y+q3Vc8eNVHhG0Dmv4Nb/JhF26ncQ7Echex+2TBWw HzESBnHl4TLrNzabSQGNB7ZbF8RfKTG8YIJq2SApT5oCxEWRfJr2l9JoJPXihK3vfQ duy0ns8W9IpIrx2oeLNdvolKx2mHVixqS4CJNaiMHhOoWN2qQZmcekG4CxoSZIyJX4 EJo+hN5kIQOs94WI2C6ml3xu/HoLZzZhctA1CyAepjY5M6e1IrYFTj5uS071+txQfv E2z0+8a/Amxg9QK2hYAnt6jFaeTl8IVajwPvjbBMhdwUfexozvzQDosql+YtSoO6QB ViNC/V8kidVXQ== Date: Tue, 8 Mar 2022 12:10:45 +0200 From: Jarkko Sakkinen To: David Laight Subject: Re: [PATCH RFC 0/3] MAP_POPULATE for device memory Message-ID: References: <20220306053211.135762-1-jarkko@kernel.org> <5729d03d6a174da6b66d1534ebdb1127@AcuMS.aculab.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <5729d03d6a174da6b66d1534ebdb1127@AcuMS.aculab.com> X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Michal Hocko , zhangyiru , Daniel Vetter , Dave Hansen , "dri-devel@lists.freedesktop.org" , Chris Wilson , "linux-mm@kvack.org" , Jason Ekstrand , Alexander Mikhalitsyn , Florian Fainelli , "linux-unionfs@vger.kernel.org" , Matthew Wilcox , "codalist@coda.cs.cmu.edu" , 'Christoph Hellwig' , Matthew Auld , Vasily Averin , Thomas =?iso-8859-1?Q?Hellstr=F6m?= , "intel-gfx@lists.freedesktop.org" , "G@iki.fi" , "linux-mips@vger.kernel.org" , Shakeel Butt , Reinette Chatre , "linux-sgx@vger.kernel.org" , Thomas Bogendoerfer , Nathaniel McCallum , Greg Kroah-Hartman , Tvrtko Ursulin , "linux-kernel@vger.kernel.org" , "linux-fsdevel@vger.kernel.org" , Andrew Morton , Alexey Gladkov Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" On Mon, Mar 07, 2022 at 10:11:19PM +0000, David Laight wrote: > From: Christoph Hellwig > > Sent: 07 March 2022 15:57 > > > > On Mon, Mar 07, 2022 at 03:29:35PM +0200, Jarkko Sakkinen wrote: > > > So what would you suggest to sort out the issue? I'm happy to go with > > > ioctl if nothing else is acceptable. > > > > PLenty of drivers treat all mmaps as if MAP_POPULATE was specified, > > typically by using (io_)remap_pfn_range. If there any reason to only > > optionally have the pre-fault semantics for sgx? If not this should > > be really simple. And if we have a real need for it to be optional > > we'll just need to find a sane way to pass that information to ->mmap. > > Is there any space in vma->vm_flags ? > > That would be better than an extra argument or function. It's very dense but I'll give a shot for callback route based on Dave's comments in this thread. I.e. use it as filter inside __mm_populate() and populate_vma_page_range(). For Enarx, which we are implementing being able to use MAP_POPULATE and get the full range EAUG'd would be best way to optimize the performance of wasm JIT (Enarx is a wasm run-time capable of running inside an SGX enclave, AMD SEV-SNP VM etc.). More so than any predictor (ra_state, madvice etc.) inside #PF handler, which have been suggested in this thread. After some research on how we implement user space, I'd rather keep the #PF handler working on a single page (EAUG a single page) and have either ioctl or MAP_POPULATE to do the batch fill. We can still "not trust the user space" i.e. the populate does not have to guarantee to do the full length since the #PF handler will then fill the holes. This was one concern in this thread but it is not hard to address. BR, Jarkko From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 2E820C433F5 for ; Tue, 8 Mar 2022 12:51:05 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 6CAF010E5B4; Tue, 8 Mar 2022 12:50:55 +0000 (UTC) Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by gabe.freedesktop.org (Postfix) with ESMTPS id 9C1AA10E488; Tue, 8 Mar 2022 10:11:27 +0000 (UTC) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id C392F61501; Tue, 8 Mar 2022 10:11:26 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 7F770C340EB; Tue, 8 Mar 2022 10:11:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1646734286; bh=fYSUhe3wecNdmSDhFtYlrEjcl8plDtdT7/KRcR78mg8=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=u4nsxISBKCbp0kWFZQFzaUS+y+q3Vc8eNVHhG0Dmv4Nb/JhF26ncQ7Echex+2TBWw HzESBnHl4TLrNzabSQGNB7ZbF8RfKTG8YIJq2SApT5oCxEWRfJr2l9JoJPXihK3vfQ duy0ns8W9IpIrx2oeLNdvolKx2mHVixqS4CJNaiMHhOoWN2qQZmcekG4CxoSZIyJX4 EJo+hN5kIQOs94WI2C6ml3xu/HoLZzZhctA1CyAepjY5M6e1IrYFTj5uS071+txQfv E2z0+8a/Amxg9QK2hYAnt6jFaeTl8IVajwPvjbBMhdwUfexozvzQDosql+YtSoO6QB ViNC/V8kidVXQ== Date: Tue, 8 Mar 2022 12:10:45 +0200 From: Jarkko Sakkinen To: David Laight Message-ID: References: <20220306053211.135762-1-jarkko@kernel.org> <5729d03d6a174da6b66d1534ebdb1127@AcuMS.aculab.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <5729d03d6a174da6b66d1534ebdb1127@AcuMS.aculab.com> X-Mailman-Approved-At: Tue, 08 Mar 2022 12:50:53 +0000 Subject: Re: [Intel-gfx] [PATCH RFC 0/3] MAP_POPULATE for device memory X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Michal Hocko , zhangyiru , Daniel Vetter , Dave Hansen , "dri-devel@lists.freedesktop.org" , Chris Wilson , "linux-mm@kvack.org" , Alexander Mikhalitsyn , Florian Fainelli , "linux-unionfs@vger.kernel.org" , Matthew Wilcox , "codalist@coda.cs.cmu.edu" , 'Christoph Hellwig' , Matthew Auld , Vasily Averin , Thomas =?iso-8859-1?Q?Hellstr=F6m?= , "intel-gfx@lists.freedesktop.org" , "G@iki.fi" , "linux-mips@vger.kernel.org" , Shakeel Butt , Reinette Chatre , "linux-sgx@vger.kernel.org" , Thomas Bogendoerfer , Nathaniel McCallum , Greg Kroah-Hartman , "linux-kernel@vger.kernel.org" , "linux-fsdevel@vger.kernel.org" , Andrew Morton , Alexey Gladkov Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" On Mon, Mar 07, 2022 at 10:11:19PM +0000, David Laight wrote: > From: Christoph Hellwig > > Sent: 07 March 2022 15:57 > > > > On Mon, Mar 07, 2022 at 03:29:35PM +0200, Jarkko Sakkinen wrote: > > > So what would you suggest to sort out the issue? I'm happy to go with > > > ioctl if nothing else is acceptable. > > > > PLenty of drivers treat all mmaps as if MAP_POPULATE was specified, > > typically by using (io_)remap_pfn_range. If there any reason to only > > optionally have the pre-fault semantics for sgx? If not this should > > be really simple. And if we have a real need for it to be optional > > we'll just need to find a sane way to pass that information to ->mmap. > > Is there any space in vma->vm_flags ? > > That would be better than an extra argument or function. It's very dense but I'll give a shot for callback route based on Dave's comments in this thread. I.e. use it as filter inside __mm_populate() and populate_vma_page_range(). For Enarx, which we are implementing being able to use MAP_POPULATE and get the full range EAUG'd would be best way to optimize the performance of wasm JIT (Enarx is a wasm run-time capable of running inside an SGX enclave, AMD SEV-SNP VM etc.). More so than any predictor (ra_state, madvice etc.) inside #PF handler, which have been suggested in this thread. After some research on how we implement user space, I'd rather keep the #PF handler working on a single page (EAUG a single page) and have either ioctl or MAP_POPULATE to do the batch fill. We can still "not trust the user space" i.e. the populate does not have to guarantee to do the full length since the #PF handler will then fill the holes. This was one concern in this thread but it is not hard to address. BR, Jarkko