From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751330AbeDYGYo (ORCPT ); Wed, 25 Apr 2018 02:24:44 -0400 Received: from mail-wm0-f49.google.com ([74.125.82.49]:40366 "EHLO mail-wm0-f49.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750929AbeDYGYi (ORCPT ); Wed, 25 Apr 2018 02:24:38 -0400 X-Google-Smtp-Source: AIpwx4/5euK55kozCfr+KQ/2I4K7ZJsrXk8BiPtsKICha/VvfdNvlM8Xnb4ICJf66V1AXG+boPo82dkKIrCY+gT2VkE= MIME-Version: 1.0 In-Reply-To: References: <20180419081657.GA16735@infradead.org> <20180420071312.GF31310@phenom.ffwll.local> <3e17afc5-7d6c-5795-07bd-f23e34cf8d4b@gmail.com> <20180420101755.GA11400@infradead.org> <20180420124625.GA31078@infradead.org> <20180420152111.GR31310@phenom.ffwll.local> <20180424184847.GA3247@infradead.org> <20180425054855.GA17038@infradead.org> From: Alex Deucher Date: Wed, 25 Apr 2018 02:24:36 -0400 Message-ID: Subject: Re: [Linaro-mm-sig] [PATCH 4/8] dma-buf: add peer2peer flag To: Daniel Vetter Cc: Christoph Hellwig , Linux Kernel Mailing List , dri-devel , "moderated list:DMA BUFFER SHARING FRAMEWORK" , Jerome Glisse , amd-gfx list , Dan Williams , Logan Gunthorpe , =?UTF-8?Q?Christian_K=C3=B6nig?= , "open list:DMA BUFFER SHARING FRAMEWORK" Content-Type: text/plain; charset="UTF-8" Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by mail.home.local id w3P6OoQB003176 On Wed, Apr 25, 2018 at 2:13 AM, Daniel Vetter wrote: > On Wed, Apr 25, 2018 at 7:48 AM, Christoph Hellwig wrote: >> On Tue, Apr 24, 2018 at 09:32:20PM +0200, Daniel Vetter wrote: >>> Out of curiosity, how much virtual flushing stuff is there still out >>> there? At least in drm we've pretty much ignore this, and seem to be >>> getting away without a huge uproar (at least from driver developers >>> and users, core folks are less amused about that). >> >> As I've just been wading through the code, the following architectures >> have non-coherent dma that flushes by virtual address for at least some >> platforms: >> >> - arm [1], arm64, hexagon, nds32, nios2, parisc, sh, xtensa, mips, >> powerpc >> >> These have non-coherent dma ops that flush by physical address: >> >> - arc, arm [1], c6x, m68k, microblaze, openrisc, sparc >> >> And these do not have non-coherent dma ops at all: >> >> - alpha, h8300, riscv, unicore32, x86 >> >> [1] arm Ń•eems to do both virtually and physically based ops, further >> audit needed. >> >> Note that using virtual addresses in the cache flushing interface >> doesn't mean that the cache actually is virtually indexed, but it at >> least allows for the possibility. >> >>> > I think the most important thing about such a buffer object is that >>> > it can distinguish the underlying mapping types. While >>> > dma_alloc_coherent, dma_alloc_attrs with DMA_ATTR_NON_CONSISTENT, >>> > dma_map_page/dma_map_single/dma_map_sg and dma_map_resource all give >>> > back a dma_addr_t they are in now way interchangable. And trying to >>> > stuff them all into a structure like struct scatterlist that has >>> > no indication what kind of mapping you are dealing with is just >>> > asking for trouble. >>> >>> Well the idea was to have 1 interface to allow all drivers to share >>> buffers with anything else, no matter how exactly they're allocated. >> >> Isn't that interface supposed to be dmabuf? Currently dma_map leaks >> a scatterlist through the sg_table in dma_buf_map_attachment / >> ->map_dma_buf, but looking at a few of the callers it seems like they >> really do not even want a scatterlist to start with, but check that >> is contains a physically contiguous range first. So kicking the >> scatterlist our there will probably improve the interface in general. > > I think by number most drm drivers require contiguous memory (or an > iommu that makes it look contiguous). But there's plenty others who > have another set of pagetables on the gpu itself and can > scatter-gather. Usually it's the former for display/video blocks, and > the latter for rendering. > >>> dma-buf has all the functions for flushing, so you can have coherent >>> mappings, non-coherent mappings and pretty much anything else. Or well >>> could, because in practice people hack up layering violations until it >>> works for the 2-3 drivers they care about. On top of that there's the >>> small issue that x86 insists that dma is coherent (and that's true for >>> most devices, including v4l drivers you might want to share stuff >>> with), and gpus really, really really do want to make almost >>> everything incoherent. >> >> How do discrete GPUs manage to be incoherent when attached over PCIe? > > It has a non-coherent transaction mode (which the chipset can opt to > not implement and still flush), to make sure the AGP horror show > doesn't happen again and GPU folks are happy with PCIe. That's at > least my understanding from digging around in amd the last time we had > coherency issues between intel and amd gpus. GPUs have some bits > somewhere (in the pagetables, or in the buffer object description > table created by userspace) to control that stuff. Right. We have a bit in the GPU page table entries that determines whether we snoop the CPU's cache or not. Alex > > For anything on the SoC it's presented as pci device, but that's > extremely fake, and we can definitely do non-snooped transactions on > drm/i915. Again, controlled by a mix of pagetables and > userspace-provided buffer object description tables. > -Daniel > -- > Daniel Vetter > Software Engineer, Intel Corporation > +41 (0) 79 365 57 48 - http://blog.ffwll.ch > _______________________________________________ > amd-gfx mailing list > amd-gfx@lists.freedesktop.org > https://lists.freedesktop.org/mailman/listinfo/amd-gfx From mboxrd@z Thu Jan 1 00:00:00 1970 From: Alex Deucher Subject: Re: [Linaro-mm-sig] [PATCH 4/8] dma-buf: add peer2peer flag Date: Wed, 25 Apr 2018 02:24:36 -0400 Message-ID: References: <20180419081657.GA16735@infradead.org> <20180420071312.GF31310@phenom.ffwll.local> <3e17afc5-7d6c-5795-07bd-f23e34cf8d4b@gmail.com> <20180420101755.GA11400@infradead.org> <20180420124625.GA31078@infradead.org> <20180420152111.GR31310@phenom.ffwll.local> <20180424184847.GA3247@infradead.org> <20180425054855.GA17038@infradead.org> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Return-path: In-Reply-To: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: amd-gfx-bounces-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW@public.gmane.org Sender: "amd-gfx" To: Daniel Vetter Cc: "moderated list:DMA BUFFER SHARING FRAMEWORK" , Linux Kernel Mailing List , amd-gfx list , Christoph Hellwig , Jerome Glisse , dri-devel , Dan Williams , Logan Gunthorpe , =?UTF-8?Q?Christian_K=C3=B6nig?= , "open list:DMA BUFFER SHARING FRAMEWORK" List-Id: dri-devel@lists.freedesktop.org T24gV2VkLCBBcHIgMjUsIDIwMTggYXQgMjoxMyBBTSwgRGFuaWVsIFZldHRlciA8ZGFuaWVsQGZm d2xsLmNoPiB3cm90ZToKPiBPbiBXZWQsIEFwciAyNSwgMjAxOCBhdCA3OjQ4IEFNLCBDaHJpc3Rv cGggSGVsbHdpZyA8aGNoQGluZnJhZGVhZC5vcmc+IHdyb3RlOgo+PiBPbiBUdWUsIEFwciAyNCwg MjAxOCBhdCAwOTozMjoyMFBNICswMjAwLCBEYW5pZWwgVmV0dGVyIHdyb3RlOgo+Pj4gT3V0IG9m IGN1cmlvc2l0eSwgaG93IG11Y2ggdmlydHVhbCBmbHVzaGluZyBzdHVmZiBpcyB0aGVyZSBzdGls bCBvdXQKPj4+IHRoZXJlPyBBdCBsZWFzdCBpbiBkcm0gd2UndmUgcHJldHR5IG11Y2ggaWdub3Jl IHRoaXMsIGFuZCBzZWVtIHRvIGJlCj4+PiBnZXR0aW5nIGF3YXkgd2l0aG91dCBhIGh1Z2UgdXBy b2FyIChhdCBsZWFzdCBmcm9tIGRyaXZlciBkZXZlbG9wZXJzCj4+PiBhbmQgdXNlcnMsIGNvcmUg Zm9sa3MgYXJlIGxlc3MgYW11c2VkIGFib3V0IHRoYXQpLgo+Pgo+PiBBcyBJJ3ZlIGp1c3QgYmVl biB3YWRpbmcgdGhyb3VnaCB0aGUgY29kZSwgdGhlIGZvbGxvd2luZyBhcmNoaXRlY3R1cmVzCj4+ IGhhdmUgbm9uLWNvaGVyZW50IGRtYSB0aGF0IGZsdXNoZXMgYnkgdmlydHVhbCBhZGRyZXNzIGZv ciBhdCBsZWFzdCBzb21lCj4+IHBsYXRmb3JtczoKPj4KPj4gIC0gYXJtIFsxXSwgYXJtNjQsIGhl eGFnb24sIG5kczMyLCBuaW9zMiwgcGFyaXNjLCBzaCwgeHRlbnNhLCBtaXBzLAo+PiAgICBwb3dl cnBjCj4+Cj4+IFRoZXNlIGhhdmUgbm9uLWNvaGVyZW50IGRtYSBvcHMgdGhhdCBmbHVzaCBieSBw aHlzaWNhbCBhZGRyZXNzOgo+Pgo+PiAgLSBhcmMsIGFybSBbMV0sIGM2eCwgbTY4aywgbWljcm9i bGF6ZSwgb3BlbnJpc2MsIHNwYXJjCj4+Cj4+IEFuZCB0aGVzZSBkbyBub3QgaGF2ZSBub24tY29o ZXJlbnQgZG1hIG9wcyBhdCBhbGw6Cj4+Cj4+ICAtIGFscGhhLCBoODMwMCwgcmlzY3YsIHVuaWNv cmUzMiwgeDg2Cj4+Cj4+IFsxXSBhcm0g0ZVlZW1zIHRvIGRvIGJvdGggdmlydHVhbGx5IGFuZCBw aHlzaWNhbGx5IGJhc2VkIG9wcywgZnVydGhlcgo+PiBhdWRpdCBuZWVkZWQuCj4+Cj4+IE5vdGUg dGhhdCB1c2luZyB2aXJ0dWFsIGFkZHJlc3NlcyBpbiB0aGUgY2FjaGUgZmx1c2hpbmcgaW50ZXJm YWNlCj4+IGRvZXNuJ3QgbWVhbiB0aGF0IHRoZSBjYWNoZSBhY3R1YWxseSBpcyB2aXJ0dWFsbHkg aW5kZXhlZCwgYnV0IGl0IGF0Cj4+IGxlYXN0IGFsbG93cyBmb3IgdGhlIHBvc3NpYmlsaXR5Lgo+ Pgo+Pj4gPiBJIHRoaW5rIHRoZSBtb3N0IGltcG9ydGFudCB0aGluZyBhYm91dCBzdWNoIGEgYnVm ZmVyIG9iamVjdCBpcyB0aGF0Cj4+PiA+IGl0IGNhbiBkaXN0aW5ndWlzaCB0aGUgdW5kZXJseWlu ZyBtYXBwaW5nIHR5cGVzLiAgV2hpbGUKPj4+ID4gZG1hX2FsbG9jX2NvaGVyZW50LCBkbWFfYWxs b2NfYXR0cnMgd2l0aCBETUFfQVRUUl9OT05fQ09OU0lTVEVOVCwKPj4+ID4gZG1hX21hcF9wYWdl L2RtYV9tYXBfc2luZ2xlL2RtYV9tYXBfc2cgYW5kIGRtYV9tYXBfcmVzb3VyY2UgYWxsIGdpdmUK Pj4+ID4gYmFjayBhIGRtYV9hZGRyX3QgdGhleSBhcmUgaW4gbm93IHdheSBpbnRlcmNoYW5nYWJs ZS4gIEFuZCB0cnlpbmcgdG8KPj4+ID4gc3R1ZmYgdGhlbSBhbGwgaW50byBhIHN0cnVjdHVyZSBs aWtlIHN0cnVjdCBzY2F0dGVybGlzdCB0aGF0IGhhcwo+Pj4gPiBubyBpbmRpY2F0aW9uIHdoYXQg a2luZCBvZiBtYXBwaW5nIHlvdSBhcmUgZGVhbGluZyB3aXRoIGlzIGp1c3QKPj4+ID4gYXNraW5n IGZvciB0cm91YmxlLgo+Pj4KPj4+IFdlbGwgdGhlIGlkZWEgd2FzIHRvIGhhdmUgMSBpbnRlcmZh Y2UgdG8gYWxsb3cgYWxsIGRyaXZlcnMgdG8gc2hhcmUKPj4+IGJ1ZmZlcnMgd2l0aCBhbnl0aGlu ZyBlbHNlLCBubyBtYXR0ZXIgaG93IGV4YWN0bHkgdGhleSdyZSBhbGxvY2F0ZWQuCj4+Cj4+IElz bid0IHRoYXQgaW50ZXJmYWNlIHN1cHBvc2VkIHRvIGJlIGRtYWJ1Zj8gIEN1cnJlbnRseSBkbWFf bWFwIGxlYWtzCj4+IGEgc2NhdHRlcmxpc3QgdGhyb3VnaCB0aGUgc2dfdGFibGUgaW4gZG1hX2J1 Zl9tYXBfYXR0YWNobWVudCAvCj4+IC0+bWFwX2RtYV9idWYsIGJ1dCBsb29raW5nIGF0IGEgZmV3 IG9mIHRoZSBjYWxsZXJzIGl0IHNlZW1zIGxpa2UgdGhleQo+PiByZWFsbHkgZG8gbm90IGV2ZW4g d2FudCBhIHNjYXR0ZXJsaXN0IHRvIHN0YXJ0IHdpdGgsIGJ1dCBjaGVjayB0aGF0Cj4+IGlzIGNv bnRhaW5zIGEgcGh5c2ljYWxseSBjb250aWd1b3VzIHJhbmdlIGZpcnN0LiAgU28ga2lja2luZyB0 aGUKPj4gc2NhdHRlcmxpc3Qgb3VyIHRoZXJlIHdpbGwgcHJvYmFibHkgaW1wcm92ZSB0aGUgaW50 ZXJmYWNlIGluIGdlbmVyYWwuCj4KPiBJIHRoaW5rIGJ5IG51bWJlciBtb3N0IGRybSBkcml2ZXJz IHJlcXVpcmUgY29udGlndW91cyBtZW1vcnkgKG9yIGFuCj4gaW9tbXUgdGhhdCBtYWtlcyBpdCBs b29rIGNvbnRpZ3VvdXMpLiBCdXQgdGhlcmUncyBwbGVudHkgb3RoZXJzIHdobwo+IGhhdmUgYW5v dGhlciBzZXQgb2YgcGFnZXRhYmxlcyBvbiB0aGUgZ3B1IGl0c2VsZiBhbmQgY2FuCj4gc2NhdHRl ci1nYXRoZXIuIFVzdWFsbHkgaXQncyB0aGUgZm9ybWVyIGZvciBkaXNwbGF5L3ZpZGVvIGJsb2Nr cywgYW5kCj4gdGhlIGxhdHRlciBmb3IgcmVuZGVyaW5nLgo+Cj4+PiBkbWEtYnVmIGhhcyBhbGwg dGhlIGZ1bmN0aW9ucyBmb3IgZmx1c2hpbmcsIHNvIHlvdSBjYW4gaGF2ZSBjb2hlcmVudAo+Pj4g bWFwcGluZ3MsIG5vbi1jb2hlcmVudCBtYXBwaW5ncyBhbmQgcHJldHR5IG11Y2ggYW55dGhpbmcg ZWxzZS4gT3Igd2VsbAo+Pj4gY291bGQsIGJlY2F1c2UgaW4gcHJhY3RpY2UgcGVvcGxlIGhhY2sg dXAgbGF5ZXJpbmcgdmlvbGF0aW9ucyB1bnRpbCBpdAo+Pj4gd29ya3MgZm9yIHRoZSAyLTMgZHJp dmVycyB0aGV5IGNhcmUgYWJvdXQuIE9uIHRvcCBvZiB0aGF0IHRoZXJlJ3MgdGhlCj4+PiBzbWFs bCBpc3N1ZSB0aGF0IHg4NiBpbnNpc3RzIHRoYXQgZG1hIGlzIGNvaGVyZW50IChhbmQgdGhhdCdz IHRydWUgZm9yCj4+PiBtb3N0IGRldmljZXMsIGluY2x1ZGluZyB2NGwgZHJpdmVycyB5b3UgbWln aHQgd2FudCB0byBzaGFyZSBzdHVmZgo+Pj4gd2l0aCksIGFuZCBncHVzIHJlYWxseSwgcmVhbGx5 IHJlYWxseSBkbyB3YW50IHRvIG1ha2UgYWxtb3N0Cj4+PiBldmVyeXRoaW5nIGluY29oZXJlbnQu Cj4+Cj4+IEhvdyBkbyBkaXNjcmV0ZSBHUFVzIG1hbmFnZSB0byBiZSBpbmNvaGVyZW50IHdoZW4g YXR0YWNoZWQgb3ZlciBQQ0llPwo+Cj4gSXQgaGFzIGEgbm9uLWNvaGVyZW50IHRyYW5zYWN0aW9u IG1vZGUgKHdoaWNoIHRoZSBjaGlwc2V0IGNhbiBvcHQgdG8KPiBub3QgaW1wbGVtZW50IGFuZCBz dGlsbCBmbHVzaCksIHRvIG1ha2Ugc3VyZSB0aGUgQUdQIGhvcnJvciBzaG93Cj4gZG9lc24ndCBo YXBwZW4gYWdhaW4gYW5kIEdQVSBmb2xrcyBhcmUgaGFwcHkgd2l0aCBQQ0llLiBUaGF0J3MgYXQK PiBsZWFzdCBteSB1bmRlcnN0YW5kaW5nIGZyb20gZGlnZ2luZyBhcm91bmQgaW4gYW1kIHRoZSBs YXN0IHRpbWUgd2UgaGFkCj4gY29oZXJlbmN5IGlzc3VlcyBiZXR3ZWVuIGludGVsIGFuZCBhbWQg Z3B1cy4gR1BVcyBoYXZlIHNvbWUgYml0cwo+IHNvbWV3aGVyZSAoaW4gdGhlIHBhZ2V0YWJsZXMs IG9yIGluIHRoZSBidWZmZXIgb2JqZWN0IGRlc2NyaXB0aW9uCj4gdGFibGUgY3JlYXRlZCBieSB1 c2Vyc3BhY2UpIHRvIGNvbnRyb2wgdGhhdCBzdHVmZi4KClJpZ2h0LiAgV2UgaGF2ZSBhIGJpdCBp biB0aGUgR1BVIHBhZ2UgdGFibGUgZW50cmllcyB0aGF0IGRldGVybWluZXMKd2hldGhlciB3ZSBz bm9vcCB0aGUgQ1BVJ3MgY2FjaGUgb3Igbm90LgoKQWxleAoKPgo+IEZvciBhbnl0aGluZyBvbiB0 aGUgU29DIGl0J3MgcHJlc2VudGVkIGFzIHBjaSBkZXZpY2UsIGJ1dCB0aGF0J3MKPiBleHRyZW1l bHkgZmFrZSwgYW5kIHdlIGNhbiBkZWZpbml0ZWx5IGRvIG5vbi1zbm9vcGVkIHRyYW5zYWN0aW9u cyBvbgo+IGRybS9pOTE1LiBBZ2FpbiwgY29udHJvbGxlZCBieSBhIG1peCBvZiBwYWdldGFibGVz IGFuZAo+IHVzZXJzcGFjZS1wcm92aWRlZCBidWZmZXIgb2JqZWN0IGRlc2NyaXB0aW9uIHRhYmxl cy4KPiAtRGFuaWVsCj4gLS0KPiBEYW5pZWwgVmV0dGVyCj4gU29mdHdhcmUgRW5naW5lZXIsIElu dGVsIENvcnBvcmF0aW9uCj4gKzQxICgwKSA3OSAzNjUgNTcgNDggLSBodHRwOi8vYmxvZy5mZnds bC5jaAo+IF9fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fCj4g YW1kLWdmeCBtYWlsaW5nIGxpc3QKPiBhbWQtZ2Z4QGxpc3RzLmZyZWVkZXNrdG9wLm9yZwo+IGh0 dHBzOi8vbGlzdHMuZnJlZWRlc2t0b3Aub3JnL21haWxtYW4vbGlzdGluZm8vYW1kLWdmeApfX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fXwphbWQtZ2Z4IG1haWxp bmcgbGlzdAphbWQtZ2Z4QGxpc3RzLmZyZWVkZXNrdG9wLm9yZwpodHRwczovL2xpc3RzLmZyZWVk ZXNrdG9wLm9yZy9tYWlsbWFuL2xpc3RpbmZvL2FtZC1nZngK