From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ale.deltatee.com (ale.deltatee.com [207.54.116.67]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 510B32110A3EC for ; Fri, 31 Aug 2018 08:48:52 -0700 (PDT) References: <20180830185352.3369-1-logang@deltatee.com> <20180830185352.3369-2-logang@deltatee.com> <2711b3a7-f582-0dda-8ac2-530be7bda1bb@amd.com> From: Logan Gunthorpe Message-ID: <439181bb-6cd6-0ab2-bca6-fc820845166e@deltatee.com> Date: Fri, 31 Aug 2018 09:48:40 -0600 MIME-Version: 1.0 In-Reply-To: <2711b3a7-f582-0dda-8ac2-530be7bda1bb@amd.com> Content-Language: en-US Subject: Re: [PATCH v5 01/13] PCI/P2PDMA: Support peer-to-peer memory List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Errors-To: linux-nvdimm-bounces@lists.01.org Sender: "Linux-nvdimm" To: =?UTF-8?Q?Christian_K=c3=b6nig?= , linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org, linux-nvme@lists.infradead.org, linux-rdma@vger.kernel.org, linux-nvdimm@lists.01.org, linux-block@vger.kernel.org Cc: Benjamin Herrenschmidt , Alex Williamson , =?UTF-8?B?SsOpcsO0bWUgR2xpc3Nl?= , Jason Gunthorpe , Bjorn Helgaas , Max Gurtovoy , Christoph Hellwig List-ID: CgpPbiAzMS8wOC8xOCAwMjowNCBBTSwgQ2hyaXN0aWFuIEvDtm5pZyB3cm90ZToKPiBBbSAzMC4w OC4yMDE4IHVtIDIwOjUzIHNjaHJpZWIgTG9nYW4gR3VudGhvcnBlOgo+PiBTb21lIFBDSSBkZXZp Y2VzIG1heSBoYXZlIG1lbW9yeSBtYXBwZWQgaW4gYSBCQVIgc3BhY2UgdGhhdCdzCj4+IGludGVu ZGVkIGZvciB1c2UgaW4gcGVlci10by1wZWVyIHRyYW5zYWN0aW9ucy4gSW4gb3JkZXIgdG8gZW5h YmxlCj4+IHN1Y2ggdHJhbnNhY3Rpb25zIHRoZSBtZW1vcnkgbXVzdCBiZSByZWdpc3RlcmVkIHdp dGggWk9ORV9ERVZJQ0UgcGFnZXMKPj4gc28gaXQgY2FuIGJlIHVzZWQgYnkgRE1BIGludGVyZmFj ZXMgaW4gZXhpc3RpbmcgZHJpdmVycy4KPiAKPiBXZSB3YW50IHRvIHVzZSB0aGF0IGZlYXR1cmUg d2l0aG91dCBaT05FX0RFVklDRSBwYWdlcyBmb3IgRE1BLWJ1ZiBhcyB3ZWxsLgo+IAo+IEhvdyBo YXJkIHdvdWxkIGl0IGJlIHRvIHNlcGFyYXRlIGVuYWJsaW5nIFAyUCBkZXRlY3Rpb24gKGUuZy4g ZGlzdGFuY2UgCj4gYmV0d2VlbiB0d28gZGV2aWNlcykgZnJvbSB0aGlzPwoKUHJldHR5IGVhc3ku IFAyUCBkZXRlY3Rpb24gaXMgcHJldHR5IG11Y2gganVzdCBwY2lfcDJwZG1hX2Rpc3RhbmNlKCkg LAp3aGljaCBoYXMgbm90aGluZyB0byBkbyB3aXRoIHRoZSBaT05FX0RFVklDRSBzdXBwb3J0LgoK KEFuZCB0aGUgZGlzdGFuY2UgZnVuY3Rpb24gbWFrZXMgdXNlIG9mIGEgbnVtYmVyIG9mIHN0YXRp YyBmdW5jdGlvbnMKd2hpY2ggY291bGQgYmUgY29tYmluZWQgaW50byBhIHNpbXBsZXIgaW50ZXJm YWNlLCBzaG91bGQgd2UgbmVlZCBpdC4pCgpMb2dhbgpfX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fXwpMaW51eC1udmRpbW0gbWFpbGluZyBsaXN0CkxpbnV4LW52 ZGltbUBsaXN0cy4wMS5vcmcKaHR0cHM6Ly9saXN0cy4wMS5vcmcvbWFpbG1hbi9saXN0aW5mby9s aW51eC1udmRpbW0K From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ale.deltatee.com ([207.54.116.67]:46930 "EHLO ale.deltatee.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727345AbeHaT46 (ORCPT ); Fri, 31 Aug 2018 15:56:58 -0400 To: =?UTF-8?Q?Christian_K=c3=b6nig?= , linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org, linux-nvme@lists.infradead.org, linux-rdma@vger.kernel.org, linux-nvdimm@lists.01.org, linux-block@vger.kernel.org Cc: Stephen Bates , Christoph Hellwig , Keith Busch , Sagi Grimberg , Bjorn Helgaas , Jason Gunthorpe , Max Gurtovoy , Dan Williams , =?UTF-8?B?SsOpcsO0bWUgR2xpc3Nl?= , Benjamin Herrenschmidt , Alex Williamson References: <20180830185352.3369-1-logang@deltatee.com> <20180830185352.3369-2-logang@deltatee.com> <2711b3a7-f582-0dda-8ac2-530be7bda1bb@amd.com> From: Logan Gunthorpe Message-ID: <439181bb-6cd6-0ab2-bca6-fc820845166e@deltatee.com> Date: Fri, 31 Aug 2018 09:48:40 -0600 MIME-Version: 1.0 In-Reply-To: <2711b3a7-f582-0dda-8ac2-530be7bda1bb@amd.com> Content-Type: text/plain; charset=utf-8 Subject: Re: [PATCH v5 01/13] PCI/P2PDMA: Support peer-to-peer memory Sender: linux-block-owner@vger.kernel.org List-Id: linux-block@vger.kernel.org On 31/08/18 02:04 AM, Christian König wrote: > Am 30.08.2018 um 20:53 schrieb Logan Gunthorpe: >> Some PCI devices may have memory mapped in a BAR space that's >> intended for use in peer-to-peer transactions. In order to enable >> such transactions the memory must be registered with ZONE_DEVICE pages >> so it can be used by DMA interfaces in existing drivers. > > We want to use that feature without ZONE_DEVICE pages for DMA-buf as well. > > How hard would it be to separate enabling P2P detection (e.g. distance > between two devices) from this? Pretty easy. P2P detection is pretty much just pci_p2pdma_distance() , which has nothing to do with the ZONE_DEVICE support. (And the distance function makes use of a number of static functions which could be combined into a simpler interface, should we need it.) Logan From mboxrd@z Thu Jan 1 00:00:00 1970 From: Logan Gunthorpe Subject: Re: [PATCH v5 01/13] PCI/P2PDMA: Support peer-to-peer memory Date: Fri, 31 Aug 2018 09:48:40 -0600 Message-ID: <439181bb-6cd6-0ab2-bca6-fc820845166e@deltatee.com> References: <20180830185352.3369-1-logang@deltatee.com> <20180830185352.3369-2-logang@deltatee.com> <2711b3a7-f582-0dda-8ac2-530be7bda1bb@amd.com> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Return-path: In-Reply-To: <2711b3a7-f582-0dda-8ac2-530be7bda1bb-5C7GfCeVMHo@public.gmane.org> Content-Language: en-US List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: linux-nvdimm-bounces-hn68Rpc1hR1g9hUCZPvPmw@public.gmane.org Sender: "Linux-nvdimm" To: =?UTF-8?Q?Christian_K=c3=b6nig?= , linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, linux-pci-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, linux-nvme-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org, linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, linux-nvdimm-hn68Rpc1hR1g9hUCZPvPmw@public.gmane.org, linux-block-u79uwXL29TY76Z2rM5mHXA@public.gmane.org Cc: Benjamin Herrenschmidt , Alex Williamson , =?UTF-8?B?SsOpcsO0bWUgR2xpc3Nl?= , Jason Gunthorpe , Bjorn Helgaas , Max Gurtovoy , Christoph Hellwig List-Id: linux-rdma@vger.kernel.org CgpPbiAzMS8wOC8xOCAwMjowNCBBTSwgQ2hyaXN0aWFuIEvDtm5pZyB3cm90ZToKPiBBbSAzMC4w OC4yMDE4IHVtIDIwOjUzIHNjaHJpZWIgTG9nYW4gR3VudGhvcnBlOgo+PiBTb21lIFBDSSBkZXZp Y2VzIG1heSBoYXZlIG1lbW9yeSBtYXBwZWQgaW4gYSBCQVIgc3BhY2UgdGhhdCdzCj4+IGludGVu ZGVkIGZvciB1c2UgaW4gcGVlci10by1wZWVyIHRyYW5zYWN0aW9ucy4gSW4gb3JkZXIgdG8gZW5h YmxlCj4+IHN1Y2ggdHJhbnNhY3Rpb25zIHRoZSBtZW1vcnkgbXVzdCBiZSByZWdpc3RlcmVkIHdp dGggWk9ORV9ERVZJQ0UgcGFnZXMKPj4gc28gaXQgY2FuIGJlIHVzZWQgYnkgRE1BIGludGVyZmFj ZXMgaW4gZXhpc3RpbmcgZHJpdmVycy4KPiAKPiBXZSB3YW50IHRvIHVzZSB0aGF0IGZlYXR1cmUg d2l0aG91dCBaT05FX0RFVklDRSBwYWdlcyBmb3IgRE1BLWJ1ZiBhcyB3ZWxsLgo+IAo+IEhvdyBo YXJkIHdvdWxkIGl0IGJlIHRvIHNlcGFyYXRlIGVuYWJsaW5nIFAyUCBkZXRlY3Rpb24gKGUuZy4g ZGlzdGFuY2UgCj4gYmV0d2VlbiB0d28gZGV2aWNlcykgZnJvbSB0aGlzPwoKUHJldHR5IGVhc3ku IFAyUCBkZXRlY3Rpb24gaXMgcHJldHR5IG11Y2gganVzdCBwY2lfcDJwZG1hX2Rpc3RhbmNlKCkg LAp3aGljaCBoYXMgbm90aGluZyB0byBkbyB3aXRoIHRoZSBaT05FX0RFVklDRSBzdXBwb3J0LgoK KEFuZCB0aGUgZGlzdGFuY2UgZnVuY3Rpb24gbWFrZXMgdXNlIG9mIGEgbnVtYmVyIG9mIHN0YXRp YyBmdW5jdGlvbnMKd2hpY2ggY291bGQgYmUgY29tYmluZWQgaW50byBhIHNpbXBsZXIgaW50ZXJm YWNlLCBzaG91bGQgd2UgbmVlZCBpdC4pCgpMb2dhbgpfX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fXwpMaW51eC1udmRpbW0gbWFpbGluZyBsaXN0CkxpbnV4LW52 ZGltbUBsaXN0cy4wMS5vcmcKaHR0cHM6Ly9saXN0cy4wMS5vcmcvbWFpbG1hbi9saXN0aW5mby9s aW51eC1udmRpbW0K From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Return-Path: To: =?UTF-8?Q?Christian_K=c3=b6nig?= , linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org, linux-nvme@lists.infradead.org, linux-rdma@vger.kernel.org, linux-nvdimm@lists.01.org, linux-block@vger.kernel.org References: <20180830185352.3369-1-logang@deltatee.com> <20180830185352.3369-2-logang@deltatee.com> <2711b3a7-f582-0dda-8ac2-530be7bda1bb@amd.com> From: Logan Gunthorpe Message-ID: <439181bb-6cd6-0ab2-bca6-fc820845166e@deltatee.com> Date: Fri, 31 Aug 2018 09:48:40 -0600 MIME-Version: 1.0 In-Reply-To: <2711b3a7-f582-0dda-8ac2-530be7bda1bb@amd.com> Subject: Re: [PATCH v5 01/13] PCI/P2PDMA: Support peer-to-peer memory List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Sagi Grimberg , Benjamin Herrenschmidt , Alex Williamson , Stephen Bates , Keith Busch , =?UTF-8?B?SsOpcsO0bWUgR2xpc3Nl?= , Jason Gunthorpe , Bjorn Helgaas , Max Gurtovoy , Dan Williams , Christoph Hellwig Content-Type: text/plain; charset="utf-8" Sender: "Linux-nvme" Errors-To: linux-nvme-bounces+bjorn=helgaas.com@lists.infradead.org List-ID: CgpPbiAzMS8wOC8xOCAwMjowNCBBTSwgQ2hyaXN0aWFuIEvDtm5pZyB3cm90ZToKPiBBbSAzMC4w OC4yMDE4IHVtIDIwOjUzIHNjaHJpZWIgTG9nYW4gR3VudGhvcnBlOgo+PiBTb21lIFBDSSBkZXZp Y2VzIG1heSBoYXZlIG1lbW9yeSBtYXBwZWQgaW4gYSBCQVIgc3BhY2UgdGhhdCdzCj4+IGludGVu ZGVkIGZvciB1c2UgaW4gcGVlci10by1wZWVyIHRyYW5zYWN0aW9ucy4gSW4gb3JkZXIgdG8gZW5h YmxlCj4+IHN1Y2ggdHJhbnNhY3Rpb25zIHRoZSBtZW1vcnkgbXVzdCBiZSByZWdpc3RlcmVkIHdp dGggWk9ORV9ERVZJQ0UgcGFnZXMKPj4gc28gaXQgY2FuIGJlIHVzZWQgYnkgRE1BIGludGVyZmFj ZXMgaW4gZXhpc3RpbmcgZHJpdmVycy4KPiAKPiBXZSB3YW50IHRvIHVzZSB0aGF0IGZlYXR1cmUg d2l0aG91dCBaT05FX0RFVklDRSBwYWdlcyBmb3IgRE1BLWJ1ZiBhcyB3ZWxsLgo+IAo+IEhvdyBo YXJkIHdvdWxkIGl0IGJlIHRvIHNlcGFyYXRlIGVuYWJsaW5nIFAyUCBkZXRlY3Rpb24gKGUuZy4g ZGlzdGFuY2UgCj4gYmV0d2VlbiB0d28gZGV2aWNlcykgZnJvbSB0aGlzPwoKUHJldHR5IGVhc3ku IFAyUCBkZXRlY3Rpb24gaXMgcHJldHR5IG11Y2gganVzdCBwY2lfcDJwZG1hX2Rpc3RhbmNlKCkg LAp3aGljaCBoYXMgbm90aGluZyB0byBkbyB3aXRoIHRoZSBaT05FX0RFVklDRSBzdXBwb3J0LgoK KEFuZCB0aGUgZGlzdGFuY2UgZnVuY3Rpb24gbWFrZXMgdXNlIG9mIGEgbnVtYmVyIG9mIHN0YXRp YyBmdW5jdGlvbnMKd2hpY2ggY291bGQgYmUgY29tYmluZWQgaW50byBhIHNpbXBsZXIgaW50ZXJm YWNlLCBzaG91bGQgd2UgbmVlZCBpdC4pCgpMb2dhbgoKX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX18KTGludXgtbnZtZSBtYWlsaW5nIGxpc3QKTGludXgtbnZt ZUBsaXN0cy5pbmZyYWRlYWQub3JnCmh0dHA6Ly9saXN0cy5pbmZyYWRlYWQub3JnL21haWxtYW4v bGlzdGluZm8vbGludXgtbnZtZQo= From mboxrd@z Thu Jan 1 00:00:00 1970 From: logang@deltatee.com (Logan Gunthorpe) Date: Fri, 31 Aug 2018 09:48:40 -0600 Subject: [PATCH v5 01/13] PCI/P2PDMA: Support peer-to-peer memory In-Reply-To: <2711b3a7-f582-0dda-8ac2-530be7bda1bb@amd.com> References: <20180830185352.3369-1-logang@deltatee.com> <20180830185352.3369-2-logang@deltatee.com> <2711b3a7-f582-0dda-8ac2-530be7bda1bb@amd.com> Message-ID: <439181bb-6cd6-0ab2-bca6-fc820845166e@deltatee.com> On 31/08/18 02:04 AM, Christian K?nig wrote: > Am 30.08.2018 um 20:53 schrieb Logan Gunthorpe: >> Some PCI devices may have memory mapped in a BAR space that's >> intended for use in peer-to-peer transactions. In order to enable >> such transactions the memory must be registered with ZONE_DEVICE pages >> so it can be used by DMA interfaces in existing drivers. > > We want to use that feature without ZONE_DEVICE pages for DMA-buf as well. > > How hard would it be to separate enabling P2P detection (e.g. distance > between two devices) from this? Pretty easy. P2P detection is pretty much just pci_p2pdma_distance() , which has nothing to do with the ZONE_DEVICE support. (And the distance function makes use of a number of static functions which could be combined into a simpler interface, should we need it.) Logan