From mboxrd@z Thu Jan 1 00:00:00 1970 Return-path: Received: from mail-wm0-f53.google.com ([74.125.82.53]:51256 "EHLO mail-wm0-f53.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754166AbeCSOJk (ORCPT ); Mon, 19 Mar 2018 10:09:40 -0400 Received: by mail-wm0-f53.google.com with SMTP id h21so15400206wmd.1 for ; Mon, 19 Mar 2018 07:09:40 -0700 (PDT) Date: Mon, 19 Mar 2018 15:09:36 +0100 From: Daniel Vetter To: Christian =?iso-8859-1?Q?K=F6nig?= Cc: linaro-mm-sig@lists.linaro.org, linux-media@vger.kernel.org, dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org Subject: Re: RFC: unpinned DMA-buf exporting v2 Message-ID: <20180319140936.GO14155@phenom.ffwll.local> References: <20180316132049.1748-1-christian.koenig@amd.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20180316132049.1748-1-christian.koenig@amd.com> Sender: linux-media-owner@vger.kernel.org List-ID: On Fri, Mar 16, 2018 at 02:20:44PM +0100, Christian König wrote: > Hi everybody, > > since I've got positive feedback from Daniel I continued working on this approach. > > A few issues are still open: > 1. Daniel suggested that I make the invalidate_mappings callback a parameter of dma_buf_attach(). > > This approach unfortunately won't work because when the attachment is > created the importer is not necessarily ready to handle invalidation > events. Why do you have this constraint? This sounds a bit like inverted create/teardown sequence troubles, where you make an object "life" before the thing is fully set up. Can't we fix this by creating the entire ttm scaffolding you'll need for a dma-buf upfront, and only once you have everything we grab the dma_buf attachment? At that point you really should be able to evict buffers again. Not requiring invalidate_mapping to be set together with the attachment means we can't ever require importers to support it (e.g. to address your concern with the userspace dma-buf userptr magic). > E.g. in the amdgpu example we first need to setup the imported GEM/TMM > objects and install that in the attachment. > > My solution is to introduce a separate function to grab the locks and > set the callback, this function could then be used to pin the buffer > later on if that turns out to be necessary after all. > > 2. With my example setup this currently results in a ping/pong situation > because the exporter prefers a VRAM placement while the importer prefers > a GTT placement. > > This results in quite a performance drop, but can be fixed by a simple > mesa patch which allows shred BOs to be placed in both VRAM and GTT. > > Question is what should we do in the meantime? Accept the performance > drop or only allow unpinned sharing with new Mesa? Maybe the exporter should not try to move stuff back into VRAM as long as there's an active dma-buf? I mean it's really cool that it works, but maybe let's just do this for a tech demo :-) Of course if it then runs out of TT then it could still try to move it back in. And "let's not move it when it's imported" is probably too stupid too, and will need to be improved again with more heuristics, but would at least get it off the ground. Long term you might want to move perhaps once per 10 seconds or so, to get idle importers to detach. Adjust 10s to match whatever benchmark/workload you care about. -Daniel -- Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch From mboxrd@z Thu Jan 1 00:00:00 1970 From: Daniel Vetter Subject: Re: RFC: unpinned DMA-buf exporting v2 Date: Mon, 19 Mar 2018 15:09:36 +0100 Message-ID: <20180319140936.GO14155@phenom.ffwll.local> References: <20180316132049.1748-1-christian.koenig@amd.com> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Return-path: Content-Disposition: inline In-Reply-To: <20180316132049.1748-1-christian.koenig-5C7GfCeVMHo@public.gmane.org> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: amd-gfx-bounces-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW@public.gmane.org Sender: "amd-gfx" To: Christian =?iso-8859-1?Q?K=F6nig?= Cc: linaro-mm-sig-cunTk1MwBs8s++Sfvej+rw@public.gmane.org, amd-gfx-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW@public.gmane.org, dri-devel-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW@public.gmane.org, linux-media-u79uwXL29TY76Z2rM5mHXA@public.gmane.org List-Id: dri-devel@lists.freedesktop.org T24gRnJpLCBNYXIgMTYsIDIwMTggYXQgMDI6MjA6NDRQTSArMDEwMCwgQ2hyaXN0aWFuIEvDtm5p ZyB3cm90ZToKPiBIaSBldmVyeWJvZHksCj4gCj4gc2luY2UgSSd2ZSBnb3QgcG9zaXRpdmUgZmVl ZGJhY2sgZnJvbSBEYW5pZWwgSSBjb250aW51ZWQgd29ya2luZyBvbiB0aGlzIGFwcHJvYWNoLgo+ IAo+IEEgZmV3IGlzc3VlcyBhcmUgc3RpbGwgb3BlbjoKPiAxLiBEYW5pZWwgc3VnZ2VzdGVkIHRo YXQgSSBtYWtlIHRoZSBpbnZhbGlkYXRlX21hcHBpbmdzIGNhbGxiYWNrIGEgcGFyYW1ldGVyIG9m IGRtYV9idWZfYXR0YWNoKCkuCj4gCj4gVGhpcyBhcHByb2FjaCB1bmZvcnR1bmF0ZWx5IHdvbid0 IHdvcmsgYmVjYXVzZSB3aGVuIHRoZSBhdHRhY2htZW50IGlzCj4gY3JlYXRlZCB0aGUgaW1wb3J0 ZXIgaXMgbm90IG5lY2Vzc2FyaWx5IHJlYWR5IHRvIGhhbmRsZSBpbnZhbGlkYXRpb24KPiBldmVu dHMuCgpXaHkgZG8geW91IGhhdmUgdGhpcyBjb25zdHJhaW50PyBUaGlzIHNvdW5kcyBhIGJpdCBs aWtlIGludmVydGVkCmNyZWF0ZS90ZWFyZG93biBzZXF1ZW5jZSB0cm91Ymxlcywgd2hlcmUgeW91 IG1ha2UgYW4gb2JqZWN0ICJsaWZlIiBiZWZvcmUKdGhlIHRoaW5nIGlzIGZ1bGx5IHNldCB1cC4K CkNhbid0IHdlIGZpeCB0aGlzIGJ5IGNyZWF0aW5nIHRoZSBlbnRpcmUgdHRtIHNjYWZmb2xkaW5n IHlvdSdsbCBuZWVkIGZvciBhCmRtYS1idWYgdXBmcm9udCwgYW5kIG9ubHkgb25jZSB5b3UgaGF2 ZSBldmVyeXRoaW5nIHdlIGdyYWIgdGhlIGRtYV9idWYKYXR0YWNobWVudD8gQXQgdGhhdCBwb2lu dCB5b3UgcmVhbGx5IHNob3VsZCBiZSBhYmxlIHRvIGV2aWN0IGJ1ZmZlcnMKYWdhaW4uCgpOb3Qg cmVxdWlyaW5nIGludmFsaWRhdGVfbWFwcGluZyB0byBiZSBzZXQgdG9nZXRoZXIgd2l0aCB0aGUg YXR0YWNobWVudAptZWFucyB3ZSBjYW4ndCBldmVyIHJlcXVpcmUgaW1wb3J0ZXJzIHRvIHN1cHBv cnQgaXQgKGUuZy4gdG8gYWRkcmVzcyB5b3VyCmNvbmNlcm4gd2l0aCB0aGUgdXNlcnNwYWNlIGRt YS1idWYgdXNlcnB0ciBtYWdpYykuCgo+IEUuZy4gaW4gdGhlIGFtZGdwdSBleGFtcGxlIHdlIGZp cnN0IG5lZWQgdG8gc2V0dXAgdGhlIGltcG9ydGVkIEdFTS9UTU0KPiBvYmplY3RzIGFuZCBpbnN0 YWxsIHRoYXQgaW4gdGhlIGF0dGFjaG1lbnQuCj4gCj4gTXkgc29sdXRpb24gaXMgdG8gaW50cm9k dWNlIGEgc2VwYXJhdGUgZnVuY3Rpb24gdG8gZ3JhYiB0aGUgbG9ja3MgYW5kCj4gc2V0IHRoZSBj YWxsYmFjaywgdGhpcyBmdW5jdGlvbiBjb3VsZCB0aGVuIGJlIHVzZWQgdG8gcGluIHRoZSBidWZm ZXIKPiBsYXRlciBvbiBpZiB0aGF0IHR1cm5zIG91dCB0byBiZSBuZWNlc3NhcnkgYWZ0ZXIgYWxs Lgo+IAo+IDIuIFdpdGggbXkgZXhhbXBsZSBzZXR1cCB0aGlzIGN1cnJlbnRseSByZXN1bHRzIGlu IGEgcGluZy9wb25nIHNpdHVhdGlvbgo+IGJlY2F1c2UgdGhlIGV4cG9ydGVyIHByZWZlcnMgYSBW UkFNIHBsYWNlbWVudCB3aGlsZSB0aGUgaW1wb3J0ZXIgcHJlZmVycwo+IGEgR1RUIHBsYWNlbWVu dC4KPiAKPiBUaGlzIHJlc3VsdHMgaW4gcXVpdGUgYSBwZXJmb3JtYW5jZSBkcm9wLCBidXQgY2Fu IGJlIGZpeGVkIGJ5IGEgc2ltcGxlCj4gbWVzYSBwYXRjaCB3aGljaCBhbGxvd3Mgc2hyZWQgQk9z IHRvIGJlIHBsYWNlZCBpbiBib3RoIFZSQU0gYW5kIEdUVC4KPiAKPiBRdWVzdGlvbiBpcyB3aGF0 IHNob3VsZCB3ZSBkbyBpbiB0aGUgbWVhbnRpbWU/IEFjY2VwdCB0aGUgcGVyZm9ybWFuY2UKPiBk cm9wIG9yIG9ubHkgYWxsb3cgdW5waW5uZWQgc2hhcmluZyB3aXRoIG5ldyBNZXNhPwoKTWF5YmUg dGhlIGV4cG9ydGVyIHNob3VsZCBub3QgdHJ5IHRvIG1vdmUgc3R1ZmYgYmFjayBpbnRvIFZSQU0g YXMgbG9uZyBhcwp0aGVyZSdzIGFuIGFjdGl2ZSBkbWEtYnVmPyBJIG1lYW4gaXQncyByZWFsbHkg Y29vbCB0aGF0IGl0IHdvcmtzLCBidXQKbWF5YmUgbGV0J3MganVzdCBkbyB0aGlzIGZvciBhIHRl Y2ggZGVtbyA6LSkKCk9mIGNvdXJzZSBpZiBpdCB0aGVuIHJ1bnMgb3V0IG9mIFRUIHRoZW4gaXQg Y291bGQgc3RpbGwgdHJ5IHRvIG1vdmUgaXQKYmFjayBpbi4gQW5kICJsZXQncyBub3QgbW92ZSBp dCB3aGVuIGl0J3MgaW1wb3J0ZWQiIGlzIHByb2JhYmx5IHRvbyBzdHVwaWQKdG9vLCBhbmQgd2ls bCBuZWVkIHRvIGJlIGltcHJvdmVkIGFnYWluIHdpdGggbW9yZSBoZXVyaXN0aWNzLCBidXQgd291 bGQgYXQKbGVhc3QgZ2V0IGl0IG9mZiB0aGUgZ3JvdW5kLgoKTG9uZyB0ZXJtIHlvdSBtaWdodCB3 YW50IHRvIG1vdmUgcGVyaGFwcyBvbmNlIHBlciAxMCBzZWNvbmRzIG9yIHNvLCB0byBnZXQKaWRs ZSBpbXBvcnRlcnMgdG8gZGV0YWNoLiBBZGp1c3QgMTBzIHRvIG1hdGNoIHdoYXRldmVyIGJlbmNo bWFyay93b3JrbG9hZAp5b3UgY2FyZSBhYm91dC4KLURhbmllbAotLSAKRGFuaWVsIFZldHRlcgpT b2Z0d2FyZSBFbmdpbmVlciwgSW50ZWwgQ29ycG9yYXRpb24KaHR0cDovL2Jsb2cuZmZ3bGwuY2gK X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KYW1kLWdmeCBt YWlsaW5nIGxpc3QKYW1kLWdmeEBsaXN0cy5mcmVlZGVza3RvcC5vcmcKaHR0cHM6Ly9saXN0cy5m cmVlZGVza3RvcC5vcmcvbWFpbG1hbi9saXN0aW5mby9hbWQtZ2Z4Cg==