From: Daniel Vetter <daniel@ffwll.ch> To: Boris Brezillon <boris.brezillon@collabora.com> Cc: Daniel Vetter <daniel@ffwll.ch>, Rob Herring <robh+dt@kernel.org>, Tomeu Vizoso <tomeu@tomeuvizoso.net>, Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>, Steven Price <steven.price@arm.com>, stable@vger.kernel.org, dri-devel@lists.freedesktop.org Subject: Re: [PATCH 7/8] drm/panfrost: Add the panfrost_gem_mapping concept Date: Mon, 2 Dec 2019 09:55:32 +0100 [thread overview] Message-ID: <20191202085532.GY624164@phenom.ffwll.local> (raw) In-Reply-To: <20191129223629.3aaab761@collabora.com> On Fri, Nov 29, 2019 at 10:36:29PM +0100, Boris Brezillon wrote: > On Fri, 29 Nov 2019 21:14:59 +0100 > Daniel Vetter <daniel@ffwll.ch> wrote: > > > On Fri, Nov 29, 2019 at 02:59:07PM +0100, Boris Brezillon wrote: > > > With the introduction of per-FD address space, the same BO can be mapped > > > in different address space if the BO is globally visible (GEM_FLINK) > > > > Also dma-buf self-imports for wayland/dri3 ... > > Indeed, I'll extend the commit message to mention that case. > > > > > > and opened in different context. The current implementation does not > > > take case into account, and attaches the mapping directly to the > > > panfrost_gem_object. > > > > > > Let's create a panfrost_gem_mapping struct and allow multiple mappings > > > per BO. > > > > > > The mappings are refcounted, which helps solve another problem where > > > mappings were teared down (GEM handle closed by userspace) while GPU > > > jobs accessing those BOs were still in-flight. Jobs now keep a > > > reference on the mappings they use. > > > > uh what. > > > > tbh this sounds bad enough (as in how did a desktop on panfrost ever work) > > Well, we didn't discover this problem until recently because: > > 1/ We have a BO cache in mesa, and until recently, this cache could > only grow (no entry eviction and no MADVISE support), meaning that BOs > were staying around forever until the app was killed. Uh, so where was the userspace when we merged this? > 2/ Mappings were teared down at BO destruction time before commit > a5efb4c9a562 ("drm/panfrost: Restructure the GEM object creation"), and > jobs are retaining references to all the BO they access. > > 3/ The mesa driver was serializing GPU jobs, and only releasing the BO > reference when the job was done (wait on the completion fence). This > has recently been changed, and now BOs are returned to the cache as > soon as the job has been submitted to the kernel. When that > happens,those BOs are marked purgeable which means the kernel can > reclaim them when it's under memory pressure. > > So yes, kernel 5.4 with a recent mesa version is currently subject to > GPU page-fault storms when the system starts reclaiming memory. > > > that I think you really want a few igts to test this stuff. > > I'll see what I can come up with (not sure how to easily detect > pagefaults from userspace). The dumb approach we do is just thrash memory and check nothing has blown up (which the runner does by looking at the dmesg and a few proc files). If you run that on a kernel with all debugging enabled, it's pretty good at catching issues. For added nastiness lots of interrupts to check error paths/syscall restarting, and at the end of the testcase, some sanity check that all the bo still contain what you think they should contain. -Daniel -- Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch
WARNING: multiple messages have this Message-ID (diff)
From: Daniel Vetter <daniel@ffwll.ch> To: Boris Brezillon <boris.brezillon@collabora.com> Cc: dri-devel@lists.freedesktop.org, stable@vger.kernel.org, Steven Price <steven.price@arm.com>, Rob Herring <robh+dt@kernel.org>, Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com> Subject: Re: [PATCH 7/8] drm/panfrost: Add the panfrost_gem_mapping concept Date: Mon, 2 Dec 2019 09:55:32 +0100 [thread overview] Message-ID: <20191202085532.GY624164@phenom.ffwll.local> (raw) In-Reply-To: <20191129223629.3aaab761@collabora.com> On Fri, Nov 29, 2019 at 10:36:29PM +0100, Boris Brezillon wrote: > On Fri, 29 Nov 2019 21:14:59 +0100 > Daniel Vetter <daniel@ffwll.ch> wrote: > > > On Fri, Nov 29, 2019 at 02:59:07PM +0100, Boris Brezillon wrote: > > > With the introduction of per-FD address space, the same BO can be mapped > > > in different address space if the BO is globally visible (GEM_FLINK) > > > > Also dma-buf self-imports for wayland/dri3 ... > > Indeed, I'll extend the commit message to mention that case. > > > > > > and opened in different context. The current implementation does not > > > take case into account, and attaches the mapping directly to the > > > panfrost_gem_object. > > > > > > Let's create a panfrost_gem_mapping struct and allow multiple mappings > > > per BO. > > > > > > The mappings are refcounted, which helps solve another problem where > > > mappings were teared down (GEM handle closed by userspace) while GPU > > > jobs accessing those BOs were still in-flight. Jobs now keep a > > > reference on the mappings they use. > > > > uh what. > > > > tbh this sounds bad enough (as in how did a desktop on panfrost ever work) > > Well, we didn't discover this problem until recently because: > > 1/ We have a BO cache in mesa, and until recently, this cache could > only grow (no entry eviction and no MADVISE support), meaning that BOs > were staying around forever until the app was killed. Uh, so where was the userspace when we merged this? > 2/ Mappings were teared down at BO destruction time before commit > a5efb4c9a562 ("drm/panfrost: Restructure the GEM object creation"), and > jobs are retaining references to all the BO they access. > > 3/ The mesa driver was serializing GPU jobs, and only releasing the BO > reference when the job was done (wait on the completion fence). This > has recently been changed, and now BOs are returned to the cache as > soon as the job has been submitted to the kernel. When that > happens,those BOs are marked purgeable which means the kernel can > reclaim them when it's under memory pressure. > > So yes, kernel 5.4 with a recent mesa version is currently subject to > GPU page-fault storms when the system starts reclaiming memory. > > > that I think you really want a few igts to test this stuff. > > I'll see what I can come up with (not sure how to easily detect > pagefaults from userspace). The dumb approach we do is just thrash memory and check nothing has blown up (which the runner does by looking at the dmesg and a few proc files). If you run that on a kernel with all debugging enabled, it's pretty good at catching issues. For added nastiness lots of interrupts to check error paths/syscall restarting, and at the end of the testcase, some sanity check that all the bo still contain what you think they should contain. -Daniel -- Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch _______________________________________________ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel
next prev parent reply other threads:[~2019-12-02 8:55 UTC|newest] Thread overview: 87+ messages / expand[flat|nested] mbox.gz Atom feed top 2019-11-29 13:59 [PATCH 0/8] panfrost: Fixes for 5.4 Boris Brezillon 2019-11-29 13:59 ` [PATCH 1/8] drm/panfrost: Make panfrost_job_run() return an ERR_PTR() instead of NULL Boris Brezillon 2019-11-29 13:59 ` Boris Brezillon 2019-11-29 14:19 ` Steven Price 2019-11-29 14:19 ` Steven Price 2019-11-29 14:31 ` Boris Brezillon 2019-11-29 14:31 ` Boris Brezillon 2019-11-29 14:38 ` Steven Price 2019-11-29 14:38 ` Steven Price 2019-11-29 19:32 ` Boris Brezillon 2019-11-29 19:32 ` Boris Brezillon 2019-11-29 13:59 ` [PATCH 2/8] drm/panfrost: Fix a race in panfrost_ioctl_madvise() Boris Brezillon 2019-11-29 13:59 ` Boris Brezillon 2019-11-29 14:24 ` Steven Price 2019-11-29 14:24 ` Steven Price 2019-11-29 14:33 ` Boris Brezillon 2019-11-29 14:33 ` Boris Brezillon 2019-11-29 14:40 ` Steven Price 2019-11-29 14:40 ` Steven Price 2019-11-29 20:07 ` Daniel Vetter 2019-11-29 20:07 ` Daniel Vetter 2019-11-29 21:45 ` Boris Brezillon 2019-11-29 21:45 ` Boris Brezillon 2019-12-05 23:08 ` Rob Herring 2019-12-05 23:08 ` Rob Herring 2019-12-06 7:53 ` Boris Brezillon 2019-12-06 7:53 ` Boris Brezillon 2019-12-06 8:08 ` Boris Brezillon 2019-12-06 8:08 ` Boris Brezillon 2019-11-29 13:59 ` [PATCH 3/8] drm/panfrost: Fix a BO leak in panfrost_ioctl_mmap_bo() Boris Brezillon 2019-11-29 13:59 ` Boris Brezillon 2019-11-29 14:26 ` Steven Price 2019-11-29 14:26 ` Steven Price 2019-11-29 13:59 ` [PATCH 4/8] drm/panfrost: Fix a race in panfrost_gem_free_object() Boris Brezillon 2019-11-29 13:59 ` Boris Brezillon 2019-11-29 14:28 ` Steven Price 2019-11-29 14:28 ` Steven Price 2019-11-29 13:59 ` [PATCH 5/8] drm/panfrost: Open/close the perfcnt BO Boris Brezillon 2019-11-29 13:59 ` Boris Brezillon 2019-11-29 14:34 ` Steven Price 2019-11-29 14:34 ` Steven Price 2019-11-29 13:59 ` [PATCH 6/8] drm/panfrost: Make sure imported/exported BOs are never purged Boris Brezillon 2019-11-29 13:59 ` Boris Brezillon 2019-11-29 14:14 ` Boris Brezillon 2019-11-29 14:14 ` Boris Brezillon 2019-11-29 14:45 ` Steven Price 2019-11-29 14:45 ` Steven Price 2019-11-29 14:52 ` Boris Brezillon 2019-11-29 14:52 ` Boris Brezillon 2019-11-29 20:12 ` Daniel Vetter 2019-11-29 20:12 ` Daniel Vetter 2019-11-29 21:09 ` Boris Brezillon 2019-11-29 21:09 ` Boris Brezillon 2019-12-02 8:52 ` Daniel Vetter 2019-12-02 8:52 ` Daniel Vetter 2019-12-02 9:50 ` Boris Brezillon 2019-12-02 9:50 ` Boris Brezillon 2019-11-29 13:59 ` [PATCH 7/8] drm/panfrost: Add the panfrost_gem_mapping concept Boris Brezillon 2019-11-29 13:59 ` Boris Brezillon 2019-11-29 15:37 ` Steven Price 2019-11-29 15:37 ` Steven Price 2019-11-29 20:14 ` Daniel Vetter 2019-11-29 20:14 ` Daniel Vetter 2019-11-29 21:36 ` Boris Brezillon 2019-11-29 21:36 ` Boris Brezillon 2019-12-02 8:55 ` Daniel Vetter [this message] 2019-12-02 8:55 ` Daniel Vetter 2019-12-02 9:13 ` Boris Brezillon 2019-12-02 9:13 ` Boris Brezillon 2019-12-02 9:44 ` Daniel Vetter 2019-12-02 9:44 ` Daniel Vetter 2019-12-04 11:41 ` Steven Price 2019-12-04 11:41 ` Steven Price 2019-11-29 13:59 ` [PATCH 8/8] drm/panfrost: Make sure the shrinker does not reclaim referenced BOs Boris Brezillon 2019-11-29 13:59 ` Boris Brezillon 2019-11-29 15:48 ` Steven Price 2019-11-29 15:48 ` Steven Price 2019-11-29 16:07 ` Boris Brezillon 2019-11-29 16:07 ` Boris Brezillon 2019-11-29 16:12 ` Steven Price 2019-11-29 16:12 ` Steven Price 2019-12-02 12:50 ` Robin Murphy 2019-12-02 12:50 ` Robin Murphy 2019-12-02 13:32 ` Boris Brezillon 2019-12-02 13:32 ` Boris Brezillon 2019-11-29 14:23 ` [PATCH 0/8] panfrost: Fixes for 5.4 Alyssa Rosenzweig 2019-12-06 17:16 ` Rob Herring
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20191202085532.GY624164@phenom.ffwll.local \ --to=daniel@ffwll.ch \ --cc=alyssa.rosenzweig@collabora.com \ --cc=boris.brezillon@collabora.com \ --cc=dri-devel@lists.freedesktop.org \ --cc=robh+dt@kernel.org \ --cc=stable@vger.kernel.org \ --cc=steven.price@arm.com \ --cc=tomeu@tomeuvizoso.net \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.