From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jordan Crouse Subject: Re: [PATCH v2 1/5] drm/nouveau: tegra: Detach from ARM DMA/IOMMU mapping Date: Wed, 25 Apr 2018 09:28:49 -0600 Message-ID: <20180425152849.GA2447@jcrouse-lnx.qualcomm.com> References: <20180425101051.15349-1-thierry.reding@gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Return-path: Content-Disposition: inline In-Reply-To: <20180425101051.15349-1-thierry.reding@gmail.com> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" To: Thierry Reding Cc: nouveau@lists.freedesktop.org, Russell King , dri-devel@lists.freedesktop.org, Christoph Hellwig , iommu@lists.linux-foundation.org, linux-tegra@vger.kernel.org, linux-arm-kernel@lists.infradead.org List-Id: linux-tegra@vger.kernel.org T24gV2VkLCBBcHIgMjUsIDIwMTggYXQgMTI6MTA6NDdQTSArMDIwMCwgVGhpZXJyeSBSZWRpbmcg d3JvdGU6Cj4gRnJvbTogVGhpZXJyeSBSZWRpbmcgPHRyZWRpbmdAbnZpZGlhLmNvbT4KPiAKPiBE ZXBlbmRpbmcgb24gdGhlIGtlcm5lbCBjb25maWd1cmF0aW9uLCBlYXJseSBBUk0gYXJjaGl0ZWN0 dXJlIHNldHVwIGNvZGUKPiBtYXkgaGF2ZSBhdHRhY2hlZCB0aGUgR1BVIHRvIGEgRE1BL0lPTU1V IG1hcHBpbmcgdGhhdCB0cmFuc3BhcmVudGx5IHVzZXMKPiB0aGUgSU9NTVUgdG8gYmFjayB0aGUg RE1BIEFQSS4gVGVncmEgcmVxdWlyZXMgc3BlY2lhbCBoYW5kbGluZyBmb3IgSU9NTVUKPiBiYWNr ZWQgYnVmZmVycyAoYSBzcGVjaWFsIGJpdCBpbiB0aGUgR1BVJ3MgTU1VIHBhZ2UgdGFibGVzIGlu ZGljYXRlcyB0aGUKPiBtZW1vcnkgcGF0aCB0byB0YWtlOiB2aWEgdGhlIFNNTVUgb3IgZGlyZWN0 bHkgdG8gdGhlIG1lbW9yeSBjb250cm9sbGVyKS4KPiBUcmFuc3BhcmVudGx5IGJhY2tpbmcgRE1B IG1lbW9yeSB3aXRoIGFuIElPTU1VIHByZXZlbnRzIE5vdXZlYXUgZnJvbQo+IHByb3Blcmx5IGhh bmRsaW5nIHN1Y2ggbWVtb3J5IGFjY2Vzc2VzIGFuZCBjYXVzZXMgbWVtb3J5IGFjY2VzcyBmYXVs dHMuCj4gCj4gQXMgYSBzaWRlLW5vdGU6IGJ1ZmZlcnMgb3RoZXIgdGhhbiB0aG9zZSBhbGxvY2F0 ZWQgaW4gaW5zdGFuY2UgbWVtb3J5Cj4gZG9uJ3QgbmVlZCB0byBiZSBwaHlzaWNhbGx5IGNvbnRp Z3VvdXMgZnJvbSB0aGUgR1BVJ3MgcGVyc3BlY3RpdmUgc2luY2UKPiB0aGUgR1BVIGNhbiBtYXAg dGhlbSBpbnRvIGNvbnRpZ3VvdXMgYnVmZmVycyB1c2luZyBpdHMgb3duIE1NVS4gTWFwcGluZwo+ IHRoZXNlIGJ1ZmZlcnMgdGhyb3VnaCB0aGUgSU9NTVUgaXMgdW5uZWNlc3NhcnkgYW5kIHdpbGwg ZXZlbiBsZWFkIHRvCj4gcGVyZm9ybWFuY2UgZGVncmFkYXRpb24gYmVjYXVzZSBvZiB0aGUgYWRk aXRpb25hbCB0cmFuc2xhdGlvbi4KPiAKPiBTaWduZWQtb2ZmLWJ5OiBUaGllcnJ5IFJlZGluZyA8 dHJlZGluZ0BudmlkaWEuY29tPgo+IC0tLQo+IEkgaGFkIGFscmVhZHkgc2VudCB0aGlzIG91dCBp bmRlcGVuZGVudGx5IHRvIGZpeCBhIHJlZ3Jlc3Npb24gdGhhdCB3YXMKPiBpbnRyb2R1Y2VkIGlu IHY0LjE2LCBidXQgdGhlbiBDaHJpc3RvcGggcG9pbnRlZCBvdXQgdGhhdCBpdCBzaG91bGQndmUK PiBiZWVuIHNlbnQgdG8gYSB3aWRlciBhdWRpZW5jZSBhbmQgc2hvdWxkIHVzZSBhIGNvcmUgQVBJ IHJhdGhlciB0aGFuCj4gY2FsbGluZyBpbnRvIGFyY2hpdGVjdHVyZSBjb2RlIGRpcmVjdGx5Lgo+ IAo+IEkndmUgYWRkZWQgaXQgdG8gdGhpcyBzZXJpZXMgZm9yIGVhc2llciByZWZlcmVuY2UgYW5k IHRvIHNob3cgdGhlIG5lZWQKPiBmb3IgdGhlIG5ldyBBUEkuCgpUaGlzIGlzIGdvb2Qgc3R1ZmYs IEkgYW0gc3RydWdnbGluZyB3aXRoIHNvbWV0aGluZyBzaW1pbGFyIG9uIEFSTTY0LiBPbmUKcHJv YmxlbSB0aGF0IEkgd2Fzbid0IGFibGUgdG8gZnVsbHkgc29sdmUgY2xlYW5seSB3YXMgdGhhdCBm b3IgYXJtLXNtbXUgCnRoZSBTTU1VIEhXIHJlc291cmNlcyBhcmUgbm90IHJlbGVhc2VkIHVudGls IHRoZSBkb21haW4gaXRzZWxmIGlzIGRlc3Ryb3llZAphbmQgSSBuZXZlciBxdWl0ZSBmaWd1cmVk IG91dCBhIHdheSB0byBzd2FwIHRoZSBkZWZhdWx0IGRvbWFpbiBjbGVhbmx5LgoKVGhpcyBpcyBh IHByb2JsZW0gZm9yIHRoZSBNU00gR1BVIGJlY2F1c2Ugbm90IG9ubHkgZG8gd2UgcnVuIG91ciBv d24gSU9NTVUgYXMKeW91IGRvIHdlIGFsc28gaGF2ZSBhIGhhcmR3YXJlIGRlcGVuZGVuY3kgdG8g dXNlIGNvbnRleHQgYmFuayAwIHRvCmFzeW5jaHJvbm91c2x5IHN3aXRjaCB0aGUgcGFnZXRhYmxl IGR1cmluZyByZW5kZXJpbmcuCgpJJ20gbm90IHN1cmUgaWYgdGhpcyBpcyBhIHByb2JsZW0geW91 IGhhdmUgZW5jb3VudGVyZWQuIEluIGFueSBldmVudCwgdGhpcyBjb2RlCmdldHMgdXMgYSBsaXR0 bGUgYml0IGZ1cnRoZXIgZG93biB0aGUgcGF0aCBhbmQgYXQgbGVhc3QgdGhlcmUgaXMgc29tZWJv ZHkgZWxzZQpvdXQgdGhlcmUgaW4gdGhlIGNvbGQgZGFyayB3b3JsZCB0aGF0IHVuZGVyc3RhbmRz IG15IHBhaW4uIDopCgpGb3IgeW91ciBpbnRlcmVzdCwgaGVyZSB3YXMgbXkgaGFsZi1oZWFydGVk IGF0dGVtcHQgdG8gYXZvaWQgY3JlYXRpbmcgRE1BCmRvbWFpbnMgaW4gdGhlIGZpcnN0IHBsYWNl IGJhc2VkIG9uIGEgYmxhY2tsaXN0IHRvIHRyeSB0byBzcHVyIGEgYml0IG9mCmRpc2N1c3Npb246 IGh0dHBzOi8vcGF0Y2h3b3JrLmZyZWVkZXNrdG9wLm9yZy9zZXJpZXMvNDE1NzMvCgpKb3JkYW4K Ci0tIApUaGUgUXVhbGNvbW0gSW5ub3ZhdGlvbiBDZW50ZXIsIEluYy4gaXMgYSBtZW1iZXIgb2Yg Q29kZSBBdXJvcmEgRm9ydW0sCmEgTGludXggRm91bmRhdGlvbiBDb2xsYWJvcmF0aXZlIFByb2pl Y3QKX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KZHJpLWRl dmVsIG1haWxpbmcgbGlzdApkcmktZGV2ZWxAbGlzdHMuZnJlZWRlc2t0b3Aub3JnCmh0dHBzOi8v bGlzdHMuZnJlZWRlc2t0b3Aub3JnL21haWxtYW4vbGlzdGluZm8vZHJpLWRldmVsCg== From mboxrd@z Thu Jan 1 00:00:00 1970 From: jcrouse@codeaurora.org (Jordan Crouse) Date: Wed, 25 Apr 2018 09:28:49 -0600 Subject: [PATCH v2 1/5] drm/nouveau: tegra: Detach from ARM DMA/IOMMU mapping In-Reply-To: <20180425101051.15349-1-thierry.reding@gmail.com> References: <20180425101051.15349-1-thierry.reding@gmail.com> Message-ID: <20180425152849.GA2447@jcrouse-lnx.qualcomm.com> To: linux-arm-kernel@lists.infradead.org List-Id: linux-arm-kernel.lists.infradead.org On Wed, Apr 25, 2018 at 12:10:47PM +0200, Thierry Reding wrote: > From: Thierry Reding > > Depending on the kernel configuration, early ARM architecture setup code > may have attached the GPU to a DMA/IOMMU mapping that transparently uses > the IOMMU to back the DMA API. Tegra requires special handling for IOMMU > backed buffers (a special bit in the GPU's MMU page tables indicates the > memory path to take: via the SMMU or directly to the memory controller). > Transparently backing DMA memory with an IOMMU prevents Nouveau from > properly handling such memory accesses and causes memory access faults. > > As a side-note: buffers other than those allocated in instance memory > don't need to be physically contiguous from the GPU's perspective since > the GPU can map them into contiguous buffers using its own MMU. Mapping > these buffers through the IOMMU is unnecessary and will even lead to > performance degradation because of the additional translation. > > Signed-off-by: Thierry Reding > --- > I had already sent this out independently to fix a regression that was > introduced in v4.16, but then Christoph pointed out that it should've > been sent to a wider audience and should use a core API rather than > calling into architecture code directly. > > I've added it to this series for easier reference and to show the need > for the new API. This is good stuff, I am struggling with something similar on ARM64. One problem that I wasn't able to fully solve cleanly was that for arm-smmu the SMMU HW resources are not released until the domain itself is destroyed and I never quite figured out a way to swap the default domain cleanly. This is a problem for the MSM GPU because not only do we run our own IOMMU as you do we also have a hardware dependency to use context bank 0 to asynchronously switch the pagetable during rendering. I'm not sure if this is a problem you have encountered. In any event, this code gets us a little bit further down the path and at least there is somebody else out there in the cold dark world that understands my pain. :) For your interest, here was my half-hearted attempt to avoid creating DMA domains in the first place based on a blacklist to try to spur a bit of discussion: https://patchwork.freedesktop.org/series/41573/ Jordan -- The Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux Foundation Collaborative Project