From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 82922C6FA89 for ; Wed, 7 Sep 2022 15:02:07 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id A017810E7A0; Wed, 7 Sep 2022 15:02:05 +0000 (UTC) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by gabe.freedesktop.org (Postfix) with ESMTP id A04F510E7A0 for ; Wed, 7 Sep 2022 15:01:59 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 6961313D5; Wed, 7 Sep 2022 08:02:05 -0700 (PDT) Received: from [10.57.15.197] (unknown [10.57.15.197]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id BF12C3F71A; Wed, 7 Sep 2022 08:01:56 -0700 (PDT) Message-ID: Date: Wed, 7 Sep 2022 16:01:54 +0100 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; rv:102.0) Gecko/20100101 Thunderbird/102.2.1 Subject: Re: [PATCH v2 4/4] vfio/pci: Allow MMIO regions to be exported through dma-buf Content-Language: en-GB To: Jason Gunthorpe , Christoph Hellwig References: <0-v2-472615b3877e+28f7-vfio_dma_buf_jgg@nvidia.com> <4-v2-472615b3877e+28f7-vfio_dma_buf_jgg@nvidia.com> From: Robin Murphy In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Leon Romanovsky , kvm@vger.kernel.org, linux-rdma@vger.kernel.org, Daniel Vetter , Oded Gabbay , Cornelia Huck , dri-devel@lists.freedesktop.org, =?UTF-8?Q?Christian_K=c3=b6nig?= , linaro-mm-sig@lists.linaro.org, Alex Williamson , Maor Gottlieb , Sumit Semwal , linux-media@vger.kernel.org Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" On 2022-09-07 13:33, Jason Gunthorpe wrote: > On Wed, Sep 07, 2022 at 05:05:57AM -0700, Christoph Hellwig wrote: >> On Tue, Sep 06, 2022 at 08:48:28AM -0300, Jason Gunthorpe wrote: >>> Right, this whole thing is the "standard" that dmabuf has adopted >>> instead of the struct pages. Once the AMD GPU driver started doing >>> this some time ago other drivers followed. >> >> But it is simple wrong. The scatterlist requires struct page backing. >> In theory a physical address would be enough, but when Dan Williams >> sent patches for that Linus shot them down. > > Yes, you said that, and I said that when the AMD driver first merged > it - but it went in anyhow and now people are using it in a bunch of > places. > > I'm happy that Christian wants to start trying to fix it, and will > help him, but it doesn't really impact this. Whatever fix is cooked up > will apply equally to vfio and habana. We've just added support for P2P segments in scatterlists, can that not be used here? Robin. >> That being said the scatterlist is the wrong interface here (and >> probably for most of it's uses). We really want a lot-level struct >> with just the dma_address and length for the DMA side, and leave it >> separate from that what is used to generate it (in most cases that >> would be a bio_vec). > > Oh definitely > >>> Now we have struct pages, almost, but I'm not sure if their limits are >>> compatible with VFIO? This has to work for small bars as well. >> >> Why would small BARs be problematic for the pages? The pages are more >> a problem for gigantic BARs do the memory overhead. > > How do I get a struct page * for a 4k BAR in vfio? > > The docs say: > > ..hotplug api on memory block boundaries. The implementation relies on > this lack of user-api constraint to allow sub-section sized memory > ranges to be specified to :c:func:`arch_add_memory`, the top-half of > memory hotplug. Sub-section support allows for 2MB as the cross-arch > common alignment granularity for :c:func:`devm_memremap_pages`. > > Jason