From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.8 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 36EE8C43613 for ; Mon, 24 Jun 2019 18:28:48 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 1089120645 for ; Mon, 24 Jun 2019 18:28:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728025AbfFXS2r (ORCPT ); Mon, 24 Jun 2019 14:28:47 -0400 Received: from ale.deltatee.com ([207.54.116.67]:40200 "EHLO ale.deltatee.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726885AbfFXS2r (ORCPT ); Mon, 24 Jun 2019 14:28:47 -0400 Received: from guinness.priv.deltatee.com ([172.16.1.162]) by ale.deltatee.com with esmtp (Exim 4.89) (envelope-from ) id 1hfThm-0001JO-NG; Mon, 24 Jun 2019 12:28:39 -0600 To: Jason Gunthorpe Cc: Christoph Hellwig , Dan Williams , Linux Kernel Mailing List , linux-block@vger.kernel.org, linux-nvme@lists.infradead.org, linux-pci@vger.kernel.org, linux-rdma , Jens Axboe , Bjorn Helgaas , Sagi Grimberg , Keith Busch , Stephen Bates References: <20190620161240.22738-1-logang@deltatee.com> <20190620193353.GF19891@ziepe.ca> <20190624073126.GB3954@lst.de> <20190624134641.GA8268@ziepe.ca> <20190624135024.GA11248@lst.de> <20190624135550.GB8268@ziepe.ca> <7210ba39-c923-79ca-57bb-7cf9afe21d54@deltatee.com> <20190624181632.GC8268@ziepe.ca> From: Logan Gunthorpe Message-ID: Date: Mon, 24 Jun 2019 12:28:33 -0600 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.7.1 MIME-Version: 1.0 In-Reply-To: <20190624181632.GC8268@ziepe.ca> Content-Type: text/plain; charset=utf-8 Content-Language: en-CA Content-Transfer-Encoding: 7bit X-SA-Exim-Connect-IP: 172.16.1.162 X-SA-Exim-Rcpt-To: sbates@raithlin.com, kbusch@kernel.org, sagi@grimberg.me, bhelgaas@google.com, axboe@kernel.dk, linux-rdma@vger.kernel.org, linux-pci@vger.kernel.org, linux-nvme@lists.infradead.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, dan.j.williams@intel.com, hch@lst.de, jgg@ziepe.ca X-SA-Exim-Mail-From: logang@deltatee.com Subject: Re: [RFC PATCH 00/28] Removing struct page from P2PDMA X-SA-Exim-Version: 4.2.1 (built Tue, 02 Aug 2016 21:08:31 +0000) X-SA-Exim-Scanned: Yes (on ale.deltatee.com) Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org On 2019-06-24 12:16 p.m., Jason Gunthorpe wrote: > On Mon, Jun 24, 2019 at 10:53:38AM -0600, Logan Gunthorpe wrote: >>> It is only a very narrow case where you can take shortcuts with >>> dma_addr_t, and I don't think shortcuts like are are appropriate for >>> the mainline kernel.. >> >> I don't think it's that narrow and it opens up a lot of avenues for >> system design that people are wanting to go. If your high speed data >> path can avoid the root complex and CPU, you can design a system which a >> much smaller CPU and fewer lanes directed at the CPU. > > I mean the shortcut that something generates dma_addr_t for Device A > and then passes it to Device B - that is too hacky for mainline. Oh, that's not a shortcut. It's completely invalid and not likely to work in any case. If you're mapping something you have to pass the device that the dma_addr_t is being programmed into. > Sounded like this series does generate the dma_addr for the correct > device.. This series doesn't generate any DMA addresses with dma_map(). The current p2pdma code ensures everything is behind the same root port and only uses the pci bus address. This is valid and correct, but yes it's something to expand upon. I'll be doing some work shortly to add transactions that go through the IOMMU and calls dma_map_* when appropriate. Logan