From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.2 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_2 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 07370C432C0 for ; Mon, 2 Dec 2019 09:13:27 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id D5FF2215E5 for ; Mon, 2 Dec 2019 09:13:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726060AbfLBJN0 (ORCPT ); Mon, 2 Dec 2019 04:13:26 -0500 Received: from bhuna.collabora.co.uk ([46.235.227.227]:58340 "EHLO bhuna.collabora.co.uk" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726057AbfLBJN0 (ORCPT ); Mon, 2 Dec 2019 04:13:26 -0500 Received: from localhost (unknown [IPv6:2a01:e0a:2c:6930:5cf4:84a1:2763:fe0d]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: bbrezillon) by bhuna.collabora.co.uk (Postfix) with ESMTPSA id 76EC8283CA2; Mon, 2 Dec 2019 09:13:24 +0000 (GMT) Date: Mon, 2 Dec 2019 10:13:21 +0100 From: Boris Brezillon To: Daniel Vetter Cc: Rob Herring , Tomeu Vizoso , Alyssa Rosenzweig , Steven Price , stable@vger.kernel.org, dri-devel@lists.freedesktop.org Subject: Re: [PATCH 7/8] drm/panfrost: Add the panfrost_gem_mapping concept Message-ID: <20191202101321.5a053f32@collabora.com> In-Reply-To: <20191202085532.GY624164@phenom.ffwll.local> References: <20191129135908.2439529-1-boris.brezillon@collabora.com> <20191129135908.2439529-8-boris.brezillon@collabora.com> <20191129201459.GS624164@phenom.ffwll.local> <20191129223629.3aaab761@collabora.com> <20191202085532.GY624164@phenom.ffwll.local> Organization: Collabora X-Mailer: Claws Mail 3.17.4 (GTK+ 2.24.32; x86_64-redhat-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: stable-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org On Mon, 2 Dec 2019 09:55:32 +0100 Daniel Vetter wrote: > On Fri, Nov 29, 2019 at 10:36:29PM +0100, Boris Brezillon wrote: > > On Fri, 29 Nov 2019 21:14:59 +0100 > > Daniel Vetter wrote: > > > > > On Fri, Nov 29, 2019 at 02:59:07PM +0100, Boris Brezillon wrote: > > > > With the introduction of per-FD address space, the same BO can be mapped > > > > in different address space if the BO is globally visible (GEM_FLINK) > > > > > > Also dma-buf self-imports for wayland/dri3 ... > > > > Indeed, I'll extend the commit message to mention that case. > > > > > > > > > and opened in different context. The current implementation does not > > > > take case into account, and attaches the mapping directly to the > > > > panfrost_gem_object. > > > > > > > > Let's create a panfrost_gem_mapping struct and allow multiple mappings > > > > per BO. > > > > > > > > The mappings are refcounted, which helps solve another problem where > > > > mappings were teared down (GEM handle closed by userspace) while GPU > > > > jobs accessing those BOs were still in-flight. Jobs now keep a > > > > reference on the mappings they use. > > > > > > uh what. > > > > > > tbh this sounds bad enough (as in how did a desktop on panfrost ever work) > > > > Well, we didn't discover this problem until recently because: > > > > 1/ We have a BO cache in mesa, and until recently, this cache could > > only grow (no entry eviction and no MADVISE support), meaning that BOs > > were staying around forever until the app was killed. > > Uh, so where was the userspace when we merged this? Well, userspace was there, it's just that we probably didn't stress the implementation as it should have been when doing the changes described in #1, #2 and 3. > > > 2/ Mappings were teared down at BO destruction time before commit > > a5efb4c9a562 ("drm/panfrost: Restructure the GEM object creation"), and > > jobs are retaining references to all the BO they access. > > > > 3/ The mesa driver was serializing GPU jobs, and only releasing the BO > > reference when the job was done (wait on the completion fence). This > > has recently been changed, and now BOs are returned to the cache as > > soon as the job has been submitted to the kernel. When that > > happens,those BOs are marked purgeable which means the kernel can > > reclaim them when it's under memory pressure. > > > > So yes, kernel 5.4 with a recent mesa version is currently subject to > > GPU page-fault storms when the system starts reclaiming memory. > > > > > that I think you really want a few igts to test this stuff. > > > > I'll see what I can come up with (not sure how to easily detect > > pagefaults from userspace). > > The dumb approach we do is just thrash memory and check nothing has blown > up (which the runner does by looking at the dmesg and a few proc files). > If you run that on a kernel with all debugging enabled, it's pretty good > at catching issues. We could also check the fence state (assuming it's signaled with an error, which I'm not sure is the case right now). > > For added nastiness lots of interrupts to check error paths/syscall > restarting, and at the end of the testcase, some sanity check that all the > bo still contain what you think they should contain. Okay, but that requires a GPU job (vertex or fragment shader) touching a BO. Apparently we haven't done that for panfrost IGT tests yet, and I'm not sure how to approach that. Should we manually forge a cmdstream and submit it? From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.2 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_2 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2189CC43215 for ; Mon, 2 Dec 2019 09:13:28 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 01E50215E5 for ; Mon, 2 Dec 2019 09:13:27 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 01E50215E5 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=collabora.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 3B1D989F6B; Mon, 2 Dec 2019 09:13:27 +0000 (UTC) Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [46.235.227.227]) by gabe.freedesktop.org (Postfix) with ESMTPS id 1C76289F6B for ; Mon, 2 Dec 2019 09:13:26 +0000 (UTC) Received: from localhost (unknown [IPv6:2a01:e0a:2c:6930:5cf4:84a1:2763:fe0d]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: bbrezillon) by bhuna.collabora.co.uk (Postfix) with ESMTPSA id 76EC8283CA2; Mon, 2 Dec 2019 09:13:24 +0000 (GMT) Date: Mon, 2 Dec 2019 10:13:21 +0100 From: Boris Brezillon To: Daniel Vetter Subject: Re: [PATCH 7/8] drm/panfrost: Add the panfrost_gem_mapping concept Message-ID: <20191202101321.5a053f32@collabora.com> In-Reply-To: <20191202085532.GY624164@phenom.ffwll.local> References: <20191129135908.2439529-1-boris.brezillon@collabora.com> <20191129135908.2439529-8-boris.brezillon@collabora.com> <20191129201459.GS624164@phenom.ffwll.local> <20191129223629.3aaab761@collabora.com> <20191202085532.GY624164@phenom.ffwll.local> Organization: Collabora X-Mailer: Claws Mail 3.17.4 (GTK+ 2.24.32; x86_64-redhat-linux-gnu) MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: stable@vger.kernel.org, Steven Price , Rob Herring , dri-devel@lists.freedesktop.org, Alyssa Rosenzweig Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Message-ID: <20191202091321._8mR1xKa7uiKYjnTm03ci9KlwrD2Nxlcx4NEmc0TjEc@z> T24gTW9uLCAyIERlYyAyMDE5IDA5OjU1OjMyICswMTAwCkRhbmllbCBWZXR0ZXIgPGRhbmllbEBm ZndsbC5jaD4gd3JvdGU6Cgo+IE9uIEZyaSwgTm92IDI5LCAyMDE5IGF0IDEwOjM2OjI5UE0gKzAx MDAsIEJvcmlzIEJyZXppbGxvbiB3cm90ZToKPiA+IE9uIEZyaSwgMjkgTm92IDIwMTkgMjE6MTQ6 NTkgKzAxMDAKPiA+IERhbmllbCBWZXR0ZXIgPGRhbmllbEBmZndsbC5jaD4gd3JvdGU6Cj4gPiAg IAo+ID4gPiBPbiBGcmksIE5vdiAyOSwgMjAxOSBhdCAwMjo1OTowN1BNICswMTAwLCBCb3JpcyBC cmV6aWxsb24gd3JvdGU6ICAKPiA+ID4gPiBXaXRoIHRoZSBpbnRyb2R1Y3Rpb24gb2YgcGVyLUZE IGFkZHJlc3Mgc3BhY2UsIHRoZSBzYW1lIEJPIGNhbiBiZSBtYXBwZWQKPiA+ID4gPiBpbiBkaWZm ZXJlbnQgYWRkcmVzcyBzcGFjZSBpZiB0aGUgQk8gaXMgZ2xvYmFsbHkgdmlzaWJsZSAoR0VNX0ZM SU5LKSAgICAKPiA+ID4gCj4gPiA+IEFsc28gZG1hLWJ1ZiBzZWxmLWltcG9ydHMgZm9yIHdheWxh bmQvZHJpMyAuLi4gIAo+ID4gCj4gPiBJbmRlZWQsIEknbGwgZXh0ZW5kIHRoZSBjb21taXQgbWVz c2FnZSB0byBtZW50aW9uIHRoYXQgY2FzZS4KPiA+ICAgCj4gPiA+ICAgCj4gPiA+ID4gYW5kIG9w ZW5lZCBpbiBkaWZmZXJlbnQgY29udGV4dC4gVGhlIGN1cnJlbnQgaW1wbGVtZW50YXRpb24gZG9l cyBub3QKPiA+ID4gPiB0YWtlIGNhc2UgaW50byBhY2NvdW50LCBhbmQgYXR0YWNoZXMgdGhlIG1h cHBpbmcgZGlyZWN0bHkgdG8gdGhlCj4gPiA+ID4gcGFuZnJvc3RfZ2VtX29iamVjdC4KPiA+ID4g PiAKPiA+ID4gPiBMZXQncyBjcmVhdGUgYSBwYW5mcm9zdF9nZW1fbWFwcGluZyBzdHJ1Y3QgYW5k IGFsbG93IG11bHRpcGxlIG1hcHBpbmdzCj4gPiA+ID4gcGVyIEJPLgo+ID4gPiA+IAo+ID4gPiA+ IFRoZSBtYXBwaW5ncyBhcmUgcmVmY291bnRlZCwgd2hpY2ggaGVscHMgc29sdmUgYW5vdGhlciBw cm9ibGVtIHdoZXJlCj4gPiA+ID4gbWFwcGluZ3Mgd2VyZSB0ZWFyZWQgZG93biAoR0VNIGhhbmRs ZSBjbG9zZWQgYnkgdXNlcnNwYWNlKSB3aGlsZSBHUFUKPiA+ID4gPiBqb2JzIGFjY2Vzc2luZyB0 aG9zZSBCT3Mgd2VyZSBzdGlsbCBpbi1mbGlnaHQuIEpvYnMgbm93IGtlZXAgYQo+ID4gPiA+IHJl ZmVyZW5jZSBvbiB0aGUgbWFwcGluZ3MgdGhleSB1c2UuICAgIAo+ID4gPiAKPiA+ID4gdWggd2hh dC4KPiA+ID4gCj4gPiA+IHRiaCB0aGlzIHNvdW5kcyBiYWQgZW5vdWdoIChhcyBpbiBob3cgZGlk IGEgZGVza3RvcCBvbiBwYW5mcm9zdCBldmVyIHdvcmspICAKPiA+IAo+ID4gV2VsbCwgd2UgZGlk bid0IGRpc2NvdmVyIHRoaXMgcHJvYmxlbSB1bnRpbCByZWNlbnRseSBiZWNhdXNlOgo+ID4gCj4g PiAxLyBXZSBoYXZlIGEgQk8gY2FjaGUgaW4gbWVzYSwgYW5kIHVudGlsIHJlY2VudGx5LCB0aGlz IGNhY2hlIGNvdWxkCj4gPiBvbmx5IGdyb3cgKG5vIGVudHJ5IGV2aWN0aW9uIGFuZCBubyBNQURW SVNFIHN1cHBvcnQpLCBtZWFuaW5nIHRoYXQgQk9zCj4gPiB3ZXJlIHN0YXlpbmcgYXJvdW5kIGZv cmV2ZXIgdW50aWwgdGhlIGFwcCB3YXMga2lsbGVkLiAgCj4gCj4gVWgsIHNvIHdoZXJlIHdhcyB0 aGUgdXNlcnNwYWNlIHdoZW4gd2UgbWVyZ2VkIHRoaXM/CgpXZWxsLCB1c2Vyc3BhY2Ugd2FzIHRo ZXJlLCBpdCdzIGp1c3QgdGhhdCB3ZSBwcm9iYWJseSBkaWRuJ3Qgc3RyZXNzCnRoZSBpbXBsZW1l bnRhdGlvbiBhcyBpdCBzaG91bGQgaGF2ZSBiZWVuIHdoZW4gZG9pbmcgdGhlIGNoYW5nZXMKZGVz Y3JpYmVkIGluICMxLCAjMiBhbmQgMy4gCgo+IAo+ID4gMi8gTWFwcGluZ3Mgd2VyZSB0ZWFyZWQg ZG93biBhdCBCTyBkZXN0cnVjdGlvbiB0aW1lIGJlZm9yZSBjb21taXQKPiA+IGE1ZWZiNGM5YTU2 MiAoImRybS9wYW5mcm9zdDogUmVzdHJ1Y3R1cmUgdGhlIEdFTSBvYmplY3QgY3JlYXRpb24iKSwg YW5kCj4gPiBqb2JzIGFyZSByZXRhaW5pbmcgcmVmZXJlbmNlcyB0byBhbGwgdGhlIEJPIHRoZXkg YWNjZXNzLgo+ID4gCj4gPiAzLyBUaGUgbWVzYSBkcml2ZXIgd2FzIHNlcmlhbGl6aW5nIEdQVSBq b2JzLCBhbmQgb25seSByZWxlYXNpbmcgdGhlIEJPCj4gPiByZWZlcmVuY2Ugd2hlbiB0aGUgam9i IHdhcyBkb25lICh3YWl0IG9uIHRoZSBjb21wbGV0aW9uIGZlbmNlKS4gVGhpcwo+ID4gaGFzIHJl Y2VudGx5IGJlZW4gY2hhbmdlZCwgYW5kIG5vdyBCT3MgYXJlIHJldHVybmVkIHRvIHRoZSBjYWNo ZSBhcwo+ID4gc29vbiBhcyB0aGUgam9iIGhhcyBiZWVuIHN1Ym1pdHRlZCB0byB0aGUga2VybmVs LiBXaGVuIHRoYXQKPiA+IGhhcHBlbnMsdGhvc2UgQk9zIGFyZSBtYXJrZWQgcHVyZ2VhYmxlIHdo aWNoIG1lYW5zIHRoZSBrZXJuZWwgY2FuCj4gPiByZWNsYWltIHRoZW0gd2hlbiBpdCdzIHVuZGVy IG1lbW9yeSBwcmVzc3VyZS4KPiA+IAo+ID4gU28geWVzLCBrZXJuZWwgNS40IHdpdGggYSByZWNl bnQgbWVzYSB2ZXJzaW9uIGlzIGN1cnJlbnRseSBzdWJqZWN0IHRvCj4gPiBHUFUgcGFnZS1mYXVs dCBzdG9ybXMgd2hlbiB0aGUgc3lzdGVtIHN0YXJ0cyByZWNsYWltaW5nIG1lbW9yeS4KPiA+ICAg Cj4gPiA+IHRoYXQgSSB0aGluayB5b3UgcmVhbGx5IHdhbnQgYSBmZXcgaWd0cyB0byB0ZXN0IHRo aXMgc3R1ZmYuICAKPiA+IAo+ID4gSSdsbCBzZWUgd2hhdCBJIGNhbiBjb21lIHVwIHdpdGggKG5v dCBzdXJlIGhvdyB0byBlYXNpbHkgZGV0ZWN0Cj4gPiBwYWdlZmF1bHRzIGZyb20gdXNlcnNwYWNl KS4gIAo+IAo+IFRoZSBkdW1iIGFwcHJvYWNoIHdlIGRvIGlzIGp1c3QgdGhyYXNoIG1lbW9yeSBh bmQgY2hlY2sgbm90aGluZyBoYXMgYmxvd24KPiB1cCAod2hpY2ggdGhlIHJ1bm5lciBkb2VzIGJ5 IGxvb2tpbmcgYXQgdGhlIGRtZXNnIGFuZCBhIGZldyBwcm9jIGZpbGVzKS4KPiBJZiB5b3UgcnVu IHRoYXQgb24gYSBrZXJuZWwgd2l0aCBhbGwgZGVidWdnaW5nIGVuYWJsZWQsIGl0J3MgcHJldHR5 IGdvb2QKPiBhdCBjYXRjaGluZyBpc3N1ZXMuCgpXZSBjb3VsZCBhbHNvIGNoZWNrIHRoZSBmZW5j ZSBzdGF0ZSAoYXNzdW1pbmcgaXQncyBzaWduYWxlZCB3aXRoIGFuCmVycm9yLCB3aGljaCBJJ20g bm90IHN1cmUgaXMgdGhlIGNhc2UgcmlnaHQgbm93KS4KCj4gCj4gRm9yIGFkZGVkIG5hc3RpbmVz cyBsb3RzIG9mIGludGVycnVwdHMgdG8gY2hlY2sgZXJyb3IgcGF0aHMvc3lzY2FsbAo+IHJlc3Rh cnRpbmcsIGFuZCBhdCB0aGUgZW5kIG9mIHRoZSB0ZXN0Y2FzZSwgc29tZSBzYW5pdHkgY2hlY2sg dGhhdCBhbGwgdGhlCj4gYm8gc3RpbGwgY29udGFpbiB3aGF0IHlvdSB0aGluayB0aGV5IHNob3Vs ZCBjb250YWluLgoKT2theSwgYnV0IHRoYXQgcmVxdWlyZXMgYSBHUFUgam9iICh2ZXJ0ZXggb3Ig ZnJhZ21lbnQgc2hhZGVyKSB0b3VjaGluZwphIEJPLiBBcHBhcmVudGx5IHdlIGhhdmVuJ3QgZG9u ZSB0aGF0IGZvciBwYW5mcm9zdCBJR1QgdGVzdHMgeWV0LCBhbmQKSSdtIG5vdCBzdXJlIGhvdyB0 byBhcHByb2FjaCB0aGF0LiBTaG91bGQgd2UgbWFudWFsbHkgZm9yZ2UgYSBjbWRzdHJlYW0KYW5k IHN1Ym1pdCBpdD8KX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X18KZHJpLWRldmVsIG1haWxpbmcgbGlzdApkcmktZGV2ZWxAbGlzdHMuZnJlZWRlc2t0b3Aub3Jn Cmh0dHBzOi8vbGlzdHMuZnJlZWRlc2t0b3Aub3JnL21haWxtYW4vbGlzdGluZm8vZHJpLWRldmVs