From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.3 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 79DD6ECE587 for ; Mon, 14 Oct 2019 16:28:31 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 57CCD2133F for ; Mon, 14 Oct 2019 16:28:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2387647AbfJNQ2a (ORCPT ); Mon, 14 Oct 2019 12:28:30 -0400 Received: from foss.arm.com ([217.140.110.172]:48448 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729930AbfJNQ23 (ORCPT ); Mon, 14 Oct 2019 12:28:29 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 3C9BF28; Mon, 14 Oct 2019 09:28:29 -0700 (PDT) Received: from [10.1.194.43] (e112269-lin.cambridge.arm.com [10.1.194.43]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 391423F718; Mon, 14 Oct 2019 09:28:28 -0700 (PDT) From: Steven Price Subject: Re: [PATCH] drm/panfrost: DMA map all pages shared with the GPU To: Robin Murphy , Daniel Vetter , David Airlie , Rob Herring , Tomeu Vizoso Cc: linux-kernel@vger.kernel.org, dri-devel@lists.freedesktop.org, Alyssa Rosenzweig References: <20191014151616.14099-1-steven.price@arm.com> <99f279c5-e93d-ade6-cd97-56b3078da755@arm.com> <8f8bd089-102f-9b8b-335b-6be06687d0ac@arm.com> Message-ID: <0cfd8582-b4e1-d429-7db8-23814b063403@arm.com> Date: Mon, 14 Oct 2019 17:28:27 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.9.0 MIME-Version: 1.0 In-Reply-To: <8f8bd089-102f-9b8b-335b-6be06687d0ac@arm.com> Content-Type: text/plain; charset=utf-8 Content-Language: en-GB Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 14/10/2019 16:55, Steven Price wrote: > On 14/10/2019 16:46, Robin Murphy wrote: >> On 14/10/2019 16:16, Steven Price wrote: >>> Pages shared with the GPU are (often) not cache coherent with the CPU so >>> cache maintenance is required to flush the CPU's caches. This was >>> already done when mapping pages on fault, but wasn't previously done >>> when mapping a freshly allocated page. >>> >>> Fix this by moving the call to dma_map_sg() into mmu_map_sg() ensuring >>> that it is always called when pages are mapped onto the GPU. Since >>> mmu_map_sg() can now fail the code also now has to handle an error >>> return. >>> >>> Not performing this cache maintenance can cause errors in the GPU output >>> (CPU caches are later flushed and overwrite the GPU output). In theory >>> it also allows the GPU (and by extension user space) to observe the >>> memory contents prior to sanitization. >> >> For the non-heap case, aren't the pages already supposed to be mapped by >> drm_gem_shmem_get_pages_sgt()? > > Hmm, well yes - it looks like it *should* work - but I was definitely > seeing cache artefacts until I did this change... let me do some more > testing. It's possible that this is actually only affecting buffers > imported from another driver. Perhaps it's > drm_gem_shmem_prime_import_sg_table() that needs fixing. Yes this does appear to only affect imported buffers from what I can tell. Looking through the code there is something suspicious in drm_gem_map_dma_buf(). The following "fixes" the problem. But I'm not sure the reasoning behind DMA_ATTR_SKIP_CPU_SYNC being specified - presumably someone though it was a good idea! I'm not sure which driver's responsibility it is to ensure the caches are flushed. ---8<---- diff --git a/drivers/gpu/drm/drm_prime.c b/drivers/gpu/drm/drm_prime.c index 0a2316e0e812..1f7353abd255 100644 --- a/drivers/gpu/drm/drm_prime.c +++ b/drivers/gpu/drm/drm_prime.c @@ -625,7 +625,7 @@ struct sg_table *drm_gem_map_dma_buf(struct dma_buf_attachment *attach, sgt = obj->dev->driver->gem_prime_get_sg_table(obj); if (!dma_map_sg_attrs(attach->dev, sgt->sgl, sgt->nents, dir, - DMA_ATTR_SKIP_CPU_SYNC)) { + 0)) { sg_free_table(sgt); kfree(sgt); sgt = ERR_PTR(-ENOMEM); From mboxrd@z Thu Jan 1 00:00:00 1970 From: Steven Price Subject: Re: [PATCH] drm/panfrost: DMA map all pages shared with the GPU Date: Mon, 14 Oct 2019 17:28:27 +0100 Message-ID: <0cfd8582-b4e1-d429-7db8-23814b063403@arm.com> References: <20191014151616.14099-1-steven.price@arm.com> <99f279c5-e93d-ade6-cd97-56b3078da755@arm.com> <8f8bd089-102f-9b8b-335b-6be06687d0ac@arm.com> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Return-path: Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by gabe.freedesktop.org (Postfix) with ESMTP id C41E86E544 for ; Mon, 14 Oct 2019 16:28:29 +0000 (UTC) In-Reply-To: <8f8bd089-102f-9b8b-335b-6be06687d0ac@arm.com> Content-Language: en-GB List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" To: Robin Murphy , Daniel Vetter , David Airlie , Rob Herring , Tomeu Vizoso Cc: linux-kernel@vger.kernel.org, dri-devel@lists.freedesktop.org, Alyssa Rosenzweig List-Id: dri-devel@lists.freedesktop.org T24gMTQvMTAvMjAxOSAxNjo1NSwgU3RldmVuIFByaWNlIHdyb3RlOgo+IE9uIDE0LzEwLzIwMTkg MTY6NDYsIFJvYmluIE11cnBoeSB3cm90ZToKPj4gT24gMTQvMTAvMjAxOSAxNjoxNiwgU3RldmVu IFByaWNlIHdyb3RlOgo+Pj4gUGFnZXMgc2hhcmVkIHdpdGggdGhlIEdQVSBhcmUgKG9mdGVuKSBu b3QgY2FjaGUgY29oZXJlbnQgd2l0aCB0aGUgQ1BVIHNvCj4+PiBjYWNoZSBtYWludGVuYW5jZSBp cyByZXF1aXJlZCB0byBmbHVzaCB0aGUgQ1BVJ3MgY2FjaGVzLiBUaGlzIHdhcwo+Pj4gYWxyZWFk eSBkb25lIHdoZW4gbWFwcGluZyBwYWdlcyBvbiBmYXVsdCwgYnV0IHdhc24ndCBwcmV2aW91c2x5 IGRvbmUKPj4+IHdoZW4gbWFwcGluZyBhIGZyZXNobHkgYWxsb2NhdGVkIHBhZ2UuCj4+Pgo+Pj4g Rml4IHRoaXMgYnkgbW92aW5nIHRoZSBjYWxsIHRvIGRtYV9tYXBfc2coKSBpbnRvIG1tdV9tYXBf c2coKSBlbnN1cmluZwo+Pj4gdGhhdCBpdCBpcyBhbHdheXMgY2FsbGVkIHdoZW4gcGFnZXMgYXJl IG1hcHBlZCBvbnRvIHRoZSBHUFUuIFNpbmNlCj4+PiBtbXVfbWFwX3NnKCkgY2FuIG5vdyBmYWls IHRoZSBjb2RlIGFsc28gbm93IGhhcyB0byBoYW5kbGUgYW4gZXJyb3IKPj4+IHJldHVybi4KPj4+ Cj4+PiBOb3QgcGVyZm9ybWluZyB0aGlzIGNhY2hlIG1haW50ZW5hbmNlIGNhbiBjYXVzZSBlcnJv cnMgaW4gdGhlIEdQVSBvdXRwdXQKPj4+IChDUFUgY2FjaGVzIGFyZSBsYXRlciBmbHVzaGVkIGFu ZCBvdmVyd3JpdGUgdGhlIEdQVSBvdXRwdXQpLiBJbiB0aGVvcnkKPj4+IGl0IGFsc28gYWxsb3dz IHRoZSBHUFUgKGFuZCBieSBleHRlbnNpb24gdXNlciBzcGFjZSkgdG8gb2JzZXJ2ZSB0aGUKPj4+ IG1lbW9yeSBjb250ZW50cyBwcmlvciB0byBzYW5pdGl6YXRpb24uCj4+Cj4+IEZvciB0aGUgbm9u LWhlYXAgY2FzZSwgYXJlbid0IHRoZSBwYWdlcyBhbHJlYWR5IHN1cHBvc2VkIHRvIGJlIG1hcHBl ZCBieQo+PiBkcm1fZ2VtX3NobWVtX2dldF9wYWdlc19zZ3QoKT8KPiAKPiBIbW0sIHdlbGwgeWVz IC0gaXQgbG9va3MgbGlrZSBpdCAqc2hvdWxkKiB3b3JrIC0gYnV0IEkgd2FzIGRlZmluaXRlbHkK PiBzZWVpbmcgY2FjaGUgYXJ0ZWZhY3RzIHVudGlsIEkgZGlkIHRoaXMgY2hhbmdlLi4uIGxldCBt ZSBkbyBzb21lIG1vcmUKPiB0ZXN0aW5nLiBJdCdzIHBvc3NpYmxlIHRoYXQgdGhpcyBpcyBhY3R1 YWxseSBvbmx5IGFmZmVjdGluZyBidWZmZXJzCj4gaW1wb3J0ZWQgZnJvbSBhbm90aGVyIGRyaXZl ci4gUGVyaGFwcyBpdCdzCj4gZHJtX2dlbV9zaG1lbV9wcmltZV9pbXBvcnRfc2dfdGFibGUoKSB0 aGF0IG5lZWRzIGZpeGluZy4KClllcyB0aGlzIGRvZXMgYXBwZWFyIHRvIG9ubHkgYWZmZWN0IGlt cG9ydGVkIGJ1ZmZlcnMgZnJvbSB3aGF0IEkgY2FuCnRlbGwuIExvb2tpbmcgdGhyb3VnaCB0aGUg Y29kZSB0aGVyZSBpcyBzb21ldGhpbmcgc3VzcGljaW91cyBpbgpkcm1fZ2VtX21hcF9kbWFfYnVm KCkuIFRoZSBmb2xsb3dpbmcgImZpeGVzIiB0aGUgcHJvYmxlbS4gQnV0IEknbSBub3QKc3VyZSB0 aGUgcmVhc29uaW5nIGJlaGluZCBETUFfQVRUUl9TS0lQX0NQVV9TWU5DIGJlaW5nIHNwZWNpZmll ZCAtCnByZXN1bWFibHkgc29tZW9uZSB0aG91Z2ggaXQgd2FzIGEgZ29vZCBpZGVhISBJJ20gbm90 IHN1cmUgd2hpY2ggZHJpdmVyJ3MKcmVzcG9uc2liaWxpdHkgaXQgaXMgdG8gZW5zdXJlIHRoZSBj YWNoZXMgYXJlIGZsdXNoZWQuCgotLS04PC0tLS0KZGlmZiAtLWdpdCBhL2RyaXZlcnMvZ3B1L2Ry bS9kcm1fcHJpbWUuYyBiL2RyaXZlcnMvZ3B1L2RybS9kcm1fcHJpbWUuYwppbmRleCAwYTIzMTZl MGU4MTIuLjFmNzM1M2FiZDI1NSAxMDA2NDQKLS0tIGEvZHJpdmVycy9ncHUvZHJtL2RybV9wcmlt ZS5jCisrKyBiL2RyaXZlcnMvZ3B1L2RybS9kcm1fcHJpbWUuYwpAQCAtNjI1LDcgKzYyNSw3IEBA IHN0cnVjdCBzZ190YWJsZSAqZHJtX2dlbV9tYXBfZG1hX2J1ZihzdHJ1Y3QgZG1hX2J1Zl9hdHRh Y2htZW50ICphdHRhY2gsCiAJCXNndCA9IG9iai0+ZGV2LT5kcml2ZXItPmdlbV9wcmltZV9nZXRf c2dfdGFibGUob2JqKTsKIAogCWlmICghZG1hX21hcF9zZ19hdHRycyhhdHRhY2gtPmRldiwgc2d0 LT5zZ2wsIHNndC0+bmVudHMsIGRpciwKLQkJCSAgICAgIERNQV9BVFRSX1NLSVBfQ1BVX1NZTkMp KSB7CisJCQkgICAgICAwKSkgewogCQlzZ19mcmVlX3RhYmxlKHNndCk7CiAJCWtmcmVlKHNndCk7 CiAJCXNndCA9IEVSUl9QVFIoLUVOT01FTSk7Cl9fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fCmRyaS1kZXZlbCBtYWlsaW5nIGxpc3QKZHJpLWRldmVsQGxpc3Rz LmZyZWVkZXNrdG9wLm9yZwpodHRwczovL2xpc3RzLmZyZWVkZXNrdG9wLm9yZy9tYWlsbWFuL2xp c3RpbmZvL2RyaS1kZXZlbA==