* [early pull] drm/msm: drm-msm-next-2021-07-28 for v5.15
@ 2021-07-28 20:52 ` Rob Clark
0 siblings, 0 replies; 12+ messages in thread
From: Rob Clark @ 2021-07-28 20:52 UTC (permalink / raw)
To: Dave Airlie, Daniel Vetter
Cc: freedreno, dri-devel, linux-arm-msm, Jordan Crouse,
Abhinav Kumar, Dmitry Baryshkov
Hi Dave & Daniel,
An early pull for v5.15 (there'll be more coming in a week or two),
consisting of the drm/scheduler conversion and a couple other small
series that one was based one. Mostly sending this now because IIUC
danvet wanted it in drm-next so he could rebase on it. (Daniel, if
you disagree then speak up, and I'll instead include this in the main
pull request once that is ready.)
This also has a core patch to drop drm_gem_object_put_locked() now
that the last use of it is removed.
The following changes since commit ff1176468d368232b684f75e82563369208bc371:
Linux 5.14-rc3 (2021-07-25 15:35:14 -0700)
are available in the Git repository at:
https://gitlab.freedesktop.org/drm/msm.git drm-msm-next-2021-07-28
for you to fetch changes up to 4541e4f2225c30b0e9442be9eb2fb8b7086cdd1f:
drm/msm/gem: Mark active before pinning (2021-07-28 09:19:00 -0700)
----------------------------------------------------------------
Rob Clark (18):
drm/msm: Let fences read directly from memptrs
drm/msm: Signal fences sooner
drm/msm: Split out devfreq handling
drm/msm: Split out get_freq() helper
drm/msm: Devfreq tuning
drm/msm: Docs and misc cleanup
drm/msm: Small submitqueue creation cleanup
drm/msm: drop drm_gem_object_put_locked()
drm: Drop drm_gem_object_put_locked()
drm/msm/submit: Simplify out-fence-fd handling
drm/msm: Consolidate submit bo state
drm/msm: Track "seqno" fences by idr
drm/msm: Return ERR_PTR() from submit_create()
drm/msm: Conversion to drm scheduler
drm/msm: Drop submit bo_list
drm/msm: Drop struct_mutex in submit path
drm/msm: Utilize gpu scheduler priorities
drm/msm/gem: Mark active before pinning
drivers/gpu/drm/drm_gem.c | 22 --
drivers/gpu/drm/msm/Kconfig | 1 +
drivers/gpu/drm/msm/Makefile | 1 +
drivers/gpu/drm/msm/adreno/a5xx_debugfs.c | 4 +-
drivers/gpu/drm/msm/adreno/a5xx_gpu.c | 6 +-
drivers/gpu/drm/msm/adreno/a5xx_power.c | 2 +-
drivers/gpu/drm/msm/adreno/a5xx_preempt.c | 7 +-
drivers/gpu/drm/msm/adreno/a6xx_gmu.c | 12 +-
drivers/gpu/drm/msm/adreno/a6xx_gpu.c | 6 +-
drivers/gpu/drm/msm/adreno/a6xx_gpu_state.c | 4 +-
drivers/gpu/drm/msm/adreno/adreno_gpu.c | 6 +-
drivers/gpu/drm/msm/msm_drv.c | 30 ++-
drivers/gpu/drm/msm/msm_fence.c | 53 +----
drivers/gpu/drm/msm/msm_fence.h | 44 +++-
drivers/gpu/drm/msm/msm_gem.c | 94 +-------
drivers/gpu/drm/msm/msm_gem.h | 47 ++--
drivers/gpu/drm/msm/msm_gem_submit.c | 344 +++++++++++++++++-----------
drivers/gpu/drm/msm/msm_gpu.c | 220 ++++--------------
drivers/gpu/drm/msm/msm_gpu.h | 139 ++++++++++-
drivers/gpu/drm/msm/msm_gpu_devfreq.c | 203 ++++++++++++++++
drivers/gpu/drm/msm/msm_rd.c | 6 +-
drivers/gpu/drm/msm/msm_ringbuffer.c | 69 +++++-
drivers/gpu/drm/msm/msm_ringbuffer.h | 12 +
drivers/gpu/drm/msm/msm_submitqueue.c | 53 +++--
include/drm/drm_gem.h | 2 -
include/uapi/drm/msm_drm.h | 14 +-
26 files changed, 865 insertions(+), 536 deletions(-)
create mode 100644 drivers/gpu/drm/msm/msm_gpu_devfreq.c
^ permalink raw reply [flat|nested] 12+ messages in thread
* [early pull] drm/msm: drm-msm-next-2021-07-28 for v5.15
@ 2021-07-28 20:52 ` Rob Clark
0 siblings, 0 replies; 12+ messages in thread
From: Rob Clark @ 2021-07-28 20:52 UTC (permalink / raw)
To: Dave Airlie, Daniel Vetter
Cc: linux-arm-msm, Abhinav Kumar, Jordan Crouse, dri-devel,
Dmitry Baryshkov, freedreno
Hi Dave & Daniel,
An early pull for v5.15 (there'll be more coming in a week or two),
consisting of the drm/scheduler conversion and a couple other small
series that one was based one. Mostly sending this now because IIUC
danvet wanted it in drm-next so he could rebase on it. (Daniel, if
you disagree then speak up, and I'll instead include this in the main
pull request once that is ready.)
This also has a core patch to drop drm_gem_object_put_locked() now
that the last use of it is removed.
The following changes since commit ff1176468d368232b684f75e82563369208bc371:
Linux 5.14-rc3 (2021-07-25 15:35:14 -0700)
are available in the Git repository at:
https://gitlab.freedesktop.org/drm/msm.git drm-msm-next-2021-07-28
for you to fetch changes up to 4541e4f2225c30b0e9442be9eb2fb8b7086cdd1f:
drm/msm/gem: Mark active before pinning (2021-07-28 09:19:00 -0700)
----------------------------------------------------------------
Rob Clark (18):
drm/msm: Let fences read directly from memptrs
drm/msm: Signal fences sooner
drm/msm: Split out devfreq handling
drm/msm: Split out get_freq() helper
drm/msm: Devfreq tuning
drm/msm: Docs and misc cleanup
drm/msm: Small submitqueue creation cleanup
drm/msm: drop drm_gem_object_put_locked()
drm: Drop drm_gem_object_put_locked()
drm/msm/submit: Simplify out-fence-fd handling
drm/msm: Consolidate submit bo state
drm/msm: Track "seqno" fences by idr
drm/msm: Return ERR_PTR() from submit_create()
drm/msm: Conversion to drm scheduler
drm/msm: Drop submit bo_list
drm/msm: Drop struct_mutex in submit path
drm/msm: Utilize gpu scheduler priorities
drm/msm/gem: Mark active before pinning
drivers/gpu/drm/drm_gem.c | 22 --
drivers/gpu/drm/msm/Kconfig | 1 +
drivers/gpu/drm/msm/Makefile | 1 +
drivers/gpu/drm/msm/adreno/a5xx_debugfs.c | 4 +-
drivers/gpu/drm/msm/adreno/a5xx_gpu.c | 6 +-
drivers/gpu/drm/msm/adreno/a5xx_power.c | 2 +-
drivers/gpu/drm/msm/adreno/a5xx_preempt.c | 7 +-
drivers/gpu/drm/msm/adreno/a6xx_gmu.c | 12 +-
drivers/gpu/drm/msm/adreno/a6xx_gpu.c | 6 +-
drivers/gpu/drm/msm/adreno/a6xx_gpu_state.c | 4 +-
drivers/gpu/drm/msm/adreno/adreno_gpu.c | 6 +-
drivers/gpu/drm/msm/msm_drv.c | 30 ++-
drivers/gpu/drm/msm/msm_fence.c | 53 +----
drivers/gpu/drm/msm/msm_fence.h | 44 +++-
drivers/gpu/drm/msm/msm_gem.c | 94 +-------
drivers/gpu/drm/msm/msm_gem.h | 47 ++--
drivers/gpu/drm/msm/msm_gem_submit.c | 344 +++++++++++++++++-----------
drivers/gpu/drm/msm/msm_gpu.c | 220 ++++--------------
drivers/gpu/drm/msm/msm_gpu.h | 139 ++++++++++-
drivers/gpu/drm/msm/msm_gpu_devfreq.c | 203 ++++++++++++++++
drivers/gpu/drm/msm/msm_rd.c | 6 +-
drivers/gpu/drm/msm/msm_ringbuffer.c | 69 +++++-
drivers/gpu/drm/msm/msm_ringbuffer.h | 12 +
drivers/gpu/drm/msm/msm_submitqueue.c | 53 +++--
include/drm/drm_gem.h | 2 -
include/uapi/drm/msm_drm.h | 14 +-
26 files changed, 865 insertions(+), 536 deletions(-)
create mode 100644 drivers/gpu/drm/msm/msm_gpu_devfreq.c
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [early pull] drm/msm: drm-msm-next-2021-07-28 for v5.15
2021-07-28 20:52 ` Rob Clark
@ 2021-07-29 0:35 ` Caleb Connolly
-1 siblings, 0 replies; 12+ messages in thread
From: Caleb Connolly @ 2021-07-29 0:35 UTC (permalink / raw)
To: Rob Clark, Dave Airlie, Daniel Vetter
Cc: freedreno, dri-devel, linux-arm-msm, Jordan Crouse,
Abhinav Kumar, Dmitry Baryshkov
Hi Rob,
This series causes a fatal crash on my Oneplus 6, the device goes to
Qualcomm crashdump mode shortly after reaching UI with the following errors:
https://paste.ubuntu.com/p/HvjmzZYtgw/
I did a git bisect and the patch ("drm/msm: Devfreq tuning") seems to be
the cause of the crash, reverting it resolves the issue.
On 28/07/2021 21:52, Rob Clark wrote:
> Hi Dave & Daniel,
>
> An early pull for v5.15 (there'll be more coming in a week or two),
> consisting of the drm/scheduler conversion and a couple other small
> series that one was based one. Mostly sending this now because IIUC
> danvet wanted it in drm-next so he could rebase on it. (Daniel, if
> you disagree then speak up, and I'll instead include this in the main
> pull request once that is ready.)
>
> This also has a core patch to drop drm_gem_object_put_locked() now
> that the last use of it is removed.
>
> The following changes since commit ff1176468d368232b684f75e82563369208bc371:
>
> Linux 5.14-rc3 (2021-07-25 15:35:14 -0700)
>
> are available in the Git repository at:
>
> https://gitlab.freedesktop.org/drm/msm.git drm-msm-next-2021-07-28
>
> for you to fetch changes up to 4541e4f2225c30b0e9442be9eb2fb8b7086cdd1f:
>
> drm/msm/gem: Mark active before pinning (2021-07-28 09:19:00 -0700)
>
> ----------------------------------------------------------------
> Rob Clark (18):
> drm/msm: Let fences read directly from memptrs
> drm/msm: Signal fences sooner
> drm/msm: Split out devfreq handling
> drm/msm: Split out get_freq() helper
> drm/msm: Devfreq tuning
> drm/msm: Docs and misc cleanup
> drm/msm: Small submitqueue creation cleanup
> drm/msm: drop drm_gem_object_put_locked()
> drm: Drop drm_gem_object_put_locked()
> drm/msm/submit: Simplify out-fence-fd handling
> drm/msm: Consolidate submit bo state
> drm/msm: Track "seqno" fences by idr
> drm/msm: Return ERR_PTR() from submit_create()
> drm/msm: Conversion to drm scheduler
> drm/msm: Drop submit bo_list
> drm/msm: Drop struct_mutex in submit path
> drm/msm: Utilize gpu scheduler priorities
> drm/msm/gem: Mark active before pinning
>
> drivers/gpu/drm/drm_gem.c | 22 --
> drivers/gpu/drm/msm/Kconfig | 1 +
> drivers/gpu/drm/msm/Makefile | 1 +
> drivers/gpu/drm/msm/adreno/a5xx_debugfs.c | 4 +-
> drivers/gpu/drm/msm/adreno/a5xx_gpu.c | 6 +-
> drivers/gpu/drm/msm/adreno/a5xx_power.c | 2 +-
> drivers/gpu/drm/msm/adreno/a5xx_preempt.c | 7 +-
> drivers/gpu/drm/msm/adreno/a6xx_gmu.c | 12 +-
> drivers/gpu/drm/msm/adreno/a6xx_gpu.c | 6 +-
> drivers/gpu/drm/msm/adreno/a6xx_gpu_state.c | 4 +-
> drivers/gpu/drm/msm/adreno/adreno_gpu.c | 6 +-
> drivers/gpu/drm/msm/msm_drv.c | 30 ++-
> drivers/gpu/drm/msm/msm_fence.c | 53 +----
> drivers/gpu/drm/msm/msm_fence.h | 44 +++-
> drivers/gpu/drm/msm/msm_gem.c | 94 +-------
> drivers/gpu/drm/msm/msm_gem.h | 47 ++--
> drivers/gpu/drm/msm/msm_gem_submit.c | 344 +++++++++++++++++-----------
> drivers/gpu/drm/msm/msm_gpu.c | 220 ++++--------------
> drivers/gpu/drm/msm/msm_gpu.h | 139 ++++++++++-
> drivers/gpu/drm/msm/msm_gpu_devfreq.c | 203 ++++++++++++++++
> drivers/gpu/drm/msm/msm_rd.c | 6 +-
> drivers/gpu/drm/msm/msm_ringbuffer.c | 69 +++++-
> drivers/gpu/drm/msm/msm_ringbuffer.h | 12 +
> drivers/gpu/drm/msm/msm_submitqueue.c | 53 +++--
> include/drm/drm_gem.h | 2 -
> include/uapi/drm/msm_drm.h | 14 +-
> 26 files changed, 865 insertions(+), 536 deletions(-)
> create mode 100644 drivers/gpu/drm/msm/msm_gpu_devfreq.c
>
--
Kind Regards,
Caleb (they/them)
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [early pull] drm/msm: drm-msm-next-2021-07-28 for v5.15
@ 2021-07-29 0:35 ` Caleb Connolly
0 siblings, 0 replies; 12+ messages in thread
From: Caleb Connolly @ 2021-07-29 0:35 UTC (permalink / raw)
To: Rob Clark, Dave Airlie, Daniel Vetter
Cc: linux-arm-msm, Abhinav Kumar, Jordan Crouse, dri-devel,
Dmitry Baryshkov, freedreno
Hi Rob,
This series causes a fatal crash on my Oneplus 6, the device goes to
Qualcomm crashdump mode shortly after reaching UI with the following errors:
https://paste.ubuntu.com/p/HvjmzZYtgw/
I did a git bisect and the patch ("drm/msm: Devfreq tuning") seems to be
the cause of the crash, reverting it resolves the issue.
On 28/07/2021 21:52, Rob Clark wrote:
> Hi Dave & Daniel,
>
> An early pull for v5.15 (there'll be more coming in a week or two),
> consisting of the drm/scheduler conversion and a couple other small
> series that one was based one. Mostly sending this now because IIUC
> danvet wanted it in drm-next so he could rebase on it. (Daniel, if
> you disagree then speak up, and I'll instead include this in the main
> pull request once that is ready.)
>
> This also has a core patch to drop drm_gem_object_put_locked() now
> that the last use of it is removed.
>
> The following changes since commit ff1176468d368232b684f75e82563369208bc371:
>
> Linux 5.14-rc3 (2021-07-25 15:35:14 -0700)
>
> are available in the Git repository at:
>
> https://gitlab.freedesktop.org/drm/msm.git drm-msm-next-2021-07-28
>
> for you to fetch changes up to 4541e4f2225c30b0e9442be9eb2fb8b7086cdd1f:
>
> drm/msm/gem: Mark active before pinning (2021-07-28 09:19:00 -0700)
>
> ----------------------------------------------------------------
> Rob Clark (18):
> drm/msm: Let fences read directly from memptrs
> drm/msm: Signal fences sooner
> drm/msm: Split out devfreq handling
> drm/msm: Split out get_freq() helper
> drm/msm: Devfreq tuning
> drm/msm: Docs and misc cleanup
> drm/msm: Small submitqueue creation cleanup
> drm/msm: drop drm_gem_object_put_locked()
> drm: Drop drm_gem_object_put_locked()
> drm/msm/submit: Simplify out-fence-fd handling
> drm/msm: Consolidate submit bo state
> drm/msm: Track "seqno" fences by idr
> drm/msm: Return ERR_PTR() from submit_create()
> drm/msm: Conversion to drm scheduler
> drm/msm: Drop submit bo_list
> drm/msm: Drop struct_mutex in submit path
> drm/msm: Utilize gpu scheduler priorities
> drm/msm/gem: Mark active before pinning
>
> drivers/gpu/drm/drm_gem.c | 22 --
> drivers/gpu/drm/msm/Kconfig | 1 +
> drivers/gpu/drm/msm/Makefile | 1 +
> drivers/gpu/drm/msm/adreno/a5xx_debugfs.c | 4 +-
> drivers/gpu/drm/msm/adreno/a5xx_gpu.c | 6 +-
> drivers/gpu/drm/msm/adreno/a5xx_power.c | 2 +-
> drivers/gpu/drm/msm/adreno/a5xx_preempt.c | 7 +-
> drivers/gpu/drm/msm/adreno/a6xx_gmu.c | 12 +-
> drivers/gpu/drm/msm/adreno/a6xx_gpu.c | 6 +-
> drivers/gpu/drm/msm/adreno/a6xx_gpu_state.c | 4 +-
> drivers/gpu/drm/msm/adreno/adreno_gpu.c | 6 +-
> drivers/gpu/drm/msm/msm_drv.c | 30 ++-
> drivers/gpu/drm/msm/msm_fence.c | 53 +----
> drivers/gpu/drm/msm/msm_fence.h | 44 +++-
> drivers/gpu/drm/msm/msm_gem.c | 94 +-------
> drivers/gpu/drm/msm/msm_gem.h | 47 ++--
> drivers/gpu/drm/msm/msm_gem_submit.c | 344 +++++++++++++++++-----------
> drivers/gpu/drm/msm/msm_gpu.c | 220 ++++--------------
> drivers/gpu/drm/msm/msm_gpu.h | 139 ++++++++++-
> drivers/gpu/drm/msm/msm_gpu_devfreq.c | 203 ++++++++++++++++
> drivers/gpu/drm/msm/msm_rd.c | 6 +-
> drivers/gpu/drm/msm/msm_ringbuffer.c | 69 +++++-
> drivers/gpu/drm/msm/msm_ringbuffer.h | 12 +
> drivers/gpu/drm/msm/msm_submitqueue.c | 53 +++--
> include/drm/drm_gem.h | 2 -
> include/uapi/drm/msm_drm.h | 14 +-
> 26 files changed, 865 insertions(+), 536 deletions(-)
> create mode 100644 drivers/gpu/drm/msm/msm_gpu_devfreq.c
>
--
Kind Regards,
Caleb (they/them)
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [early pull] drm/msm: drm-msm-next-2021-07-28 for v5.15
2021-07-29 0:35 ` Caleb Connolly
@ 2021-07-29 1:02 ` Rob Clark
-1 siblings, 0 replies; 12+ messages in thread
From: Rob Clark @ 2021-07-29 1:02 UTC (permalink / raw)
To: Caleb Connolly, Jordan Crouse
Cc: Dave Airlie, Daniel Vetter, freedreno, dri-devel, linux-arm-msm,
Abhinav Kumar, Dmitry Baryshkov
Jordan, any idea if more frequent frequency changes would for some
reason make a630 grumpy? I was expecting it should be somewhat
similar to a618 (same GMU fw, etc). The main result of that patch
should be clamping to min freq when gpu goes idle, and the toggling
back to devfreq provided freq on idle->active transition. So there
might be more frequent freq transitions.
Caleb, I don't suppose you could somehow delay starting UI and get
some traces? Something along the lines of:
localhost ~ # cd /sys/kernel/debug/tracing/
localhost /sys/kernel/debug/tracing # echo 1 > events/drm_msm_gpu/enable
localhost /sys/kernel/debug/tracing # echo 1 > tracing_on
localhost /sys/kernel/debug/tracing # cat trace_pipe
Does adding an 'if (1) return' at the top of msm_devfreq_idle() help?
That should bypass the clamping to min freq when the GPU isn't doing
anything and reduce the # of freq transitions. I suppose we could
opt-in to this behavior on a per-gpu basis..
BR,
-R
On Wed, Jul 28, 2021 at 5:35 PM Caleb Connolly
<caleb.connolly@linaro.org> wrote:
>
> Hi Rob,
>
> This series causes a fatal crash on my Oneplus 6, the device goes to
> Qualcomm crashdump mode shortly after reaching UI with the following errors:
>
> https://paste.ubuntu.com/p/HvjmzZYtgw/
>
> I did a git bisect and the patch ("drm/msm: Devfreq tuning") seems to be
> the cause of the crash, reverting it resolves the issue.
>
>
> On 28/07/2021 21:52, Rob Clark wrote:
> > Hi Dave & Daniel,
> >
> > An early pull for v5.15 (there'll be more coming in a week or two),
> > consisting of the drm/scheduler conversion and a couple other small
> > series that one was based one. Mostly sending this now because IIUC
> > danvet wanted it in drm-next so he could rebase on it. (Daniel, if
> > you disagree then speak up, and I'll instead include this in the main
> > pull request once that is ready.)
> >
> > This also has a core patch to drop drm_gem_object_put_locked() now
> > that the last use of it is removed.
> >
> > The following changes since commit ff1176468d368232b684f75e82563369208bc371:
> >
> > Linux 5.14-rc3 (2021-07-25 15:35:14 -0700)
> >
> > are available in the Git repository at:
> >
> > https://gitlab.freedesktop.org/drm/msm.git drm-msm-next-2021-07-28
> >
> > for you to fetch changes up to 4541e4f2225c30b0e9442be9eb2fb8b7086cdd1f:
> >
> > drm/msm/gem: Mark active before pinning (2021-07-28 09:19:00 -0700)
> >
> > ----------------------------------------------------------------
> > Rob Clark (18):
> > drm/msm: Let fences read directly from memptrs
> > drm/msm: Signal fences sooner
> > drm/msm: Split out devfreq handling
> > drm/msm: Split out get_freq() helper
> > drm/msm: Devfreq tuning
> > drm/msm: Docs and misc cleanup
> > drm/msm: Small submitqueue creation cleanup
> > drm/msm: drop drm_gem_object_put_locked()
> > drm: Drop drm_gem_object_put_locked()
> > drm/msm/submit: Simplify out-fence-fd handling
> > drm/msm: Consolidate submit bo state
> > drm/msm: Track "seqno" fences by idr
> > drm/msm: Return ERR_PTR() from submit_create()
> > drm/msm: Conversion to drm scheduler
> > drm/msm: Drop submit bo_list
> > drm/msm: Drop struct_mutex in submit path
> > drm/msm: Utilize gpu scheduler priorities
> > drm/msm/gem: Mark active before pinning
> >
> > drivers/gpu/drm/drm_gem.c | 22 --
> > drivers/gpu/drm/msm/Kconfig | 1 +
> > drivers/gpu/drm/msm/Makefile | 1 +
> > drivers/gpu/drm/msm/adreno/a5xx_debugfs.c | 4 +-
> > drivers/gpu/drm/msm/adreno/a5xx_gpu.c | 6 +-
> > drivers/gpu/drm/msm/adreno/a5xx_power.c | 2 +-
> > drivers/gpu/drm/msm/adreno/a5xx_preempt.c | 7 +-
> > drivers/gpu/drm/msm/adreno/a6xx_gmu.c | 12 +-
> > drivers/gpu/drm/msm/adreno/a6xx_gpu.c | 6 +-
> > drivers/gpu/drm/msm/adreno/a6xx_gpu_state.c | 4 +-
> > drivers/gpu/drm/msm/adreno/adreno_gpu.c | 6 +-
> > drivers/gpu/drm/msm/msm_drv.c | 30 ++-
> > drivers/gpu/drm/msm/msm_fence.c | 53 +----
> > drivers/gpu/drm/msm/msm_fence.h | 44 +++-
> > drivers/gpu/drm/msm/msm_gem.c | 94 +-------
> > drivers/gpu/drm/msm/msm_gem.h | 47 ++--
> > drivers/gpu/drm/msm/msm_gem_submit.c | 344 +++++++++++++++++-----------
> > drivers/gpu/drm/msm/msm_gpu.c | 220 ++++--------------
> > drivers/gpu/drm/msm/msm_gpu.h | 139 ++++++++++-
> > drivers/gpu/drm/msm/msm_gpu_devfreq.c | 203 ++++++++++++++++
> > drivers/gpu/drm/msm/msm_rd.c | 6 +-
> > drivers/gpu/drm/msm/msm_ringbuffer.c | 69 +++++-
> > drivers/gpu/drm/msm/msm_ringbuffer.h | 12 +
> > drivers/gpu/drm/msm/msm_submitqueue.c | 53 +++--
> > include/drm/drm_gem.h | 2 -
> > include/uapi/drm/msm_drm.h | 14 +-
> > 26 files changed, 865 insertions(+), 536 deletions(-)
> > create mode 100644 drivers/gpu/drm/msm/msm_gpu_devfreq.c
> >
>
> --
> Kind Regards,
> Caleb (they/them)
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [early pull] drm/msm: drm-msm-next-2021-07-28 for v5.15
@ 2021-07-29 1:02 ` Rob Clark
0 siblings, 0 replies; 12+ messages in thread
From: Rob Clark @ 2021-07-29 1:02 UTC (permalink / raw)
To: Caleb Connolly, Jordan Crouse
Cc: Abhinav Kumar, dri-devel, linux-arm-msm, Dmitry Baryshkov, freedreno
Jordan, any idea if more frequent frequency changes would for some
reason make a630 grumpy? I was expecting it should be somewhat
similar to a618 (same GMU fw, etc). The main result of that patch
should be clamping to min freq when gpu goes idle, and the toggling
back to devfreq provided freq on idle->active transition. So there
might be more frequent freq transitions.
Caleb, I don't suppose you could somehow delay starting UI and get
some traces? Something along the lines of:
localhost ~ # cd /sys/kernel/debug/tracing/
localhost /sys/kernel/debug/tracing # echo 1 > events/drm_msm_gpu/enable
localhost /sys/kernel/debug/tracing # echo 1 > tracing_on
localhost /sys/kernel/debug/tracing # cat trace_pipe
Does adding an 'if (1) return' at the top of msm_devfreq_idle() help?
That should bypass the clamping to min freq when the GPU isn't doing
anything and reduce the # of freq transitions. I suppose we could
opt-in to this behavior on a per-gpu basis..
BR,
-R
On Wed, Jul 28, 2021 at 5:35 PM Caleb Connolly
<caleb.connolly@linaro.org> wrote:
>
> Hi Rob,
>
> This series causes a fatal crash on my Oneplus 6, the device goes to
> Qualcomm crashdump mode shortly after reaching UI with the following errors:
>
> https://paste.ubuntu.com/p/HvjmzZYtgw/
>
> I did a git bisect and the patch ("drm/msm: Devfreq tuning") seems to be
> the cause of the crash, reverting it resolves the issue.
>
>
> On 28/07/2021 21:52, Rob Clark wrote:
> > Hi Dave & Daniel,
> >
> > An early pull for v5.15 (there'll be more coming in a week or two),
> > consisting of the drm/scheduler conversion and a couple other small
> > series that one was based one. Mostly sending this now because IIUC
> > danvet wanted it in drm-next so he could rebase on it. (Daniel, if
> > you disagree then speak up, and I'll instead include this in the main
> > pull request once that is ready.)
> >
> > This also has a core patch to drop drm_gem_object_put_locked() now
> > that the last use of it is removed.
> >
> > The following changes since commit ff1176468d368232b684f75e82563369208bc371:
> >
> > Linux 5.14-rc3 (2021-07-25 15:35:14 -0700)
> >
> > are available in the Git repository at:
> >
> > https://gitlab.freedesktop.org/drm/msm.git drm-msm-next-2021-07-28
> >
> > for you to fetch changes up to 4541e4f2225c30b0e9442be9eb2fb8b7086cdd1f:
> >
> > drm/msm/gem: Mark active before pinning (2021-07-28 09:19:00 -0700)
> >
> > ----------------------------------------------------------------
> > Rob Clark (18):
> > drm/msm: Let fences read directly from memptrs
> > drm/msm: Signal fences sooner
> > drm/msm: Split out devfreq handling
> > drm/msm: Split out get_freq() helper
> > drm/msm: Devfreq tuning
> > drm/msm: Docs and misc cleanup
> > drm/msm: Small submitqueue creation cleanup
> > drm/msm: drop drm_gem_object_put_locked()
> > drm: Drop drm_gem_object_put_locked()
> > drm/msm/submit: Simplify out-fence-fd handling
> > drm/msm: Consolidate submit bo state
> > drm/msm: Track "seqno" fences by idr
> > drm/msm: Return ERR_PTR() from submit_create()
> > drm/msm: Conversion to drm scheduler
> > drm/msm: Drop submit bo_list
> > drm/msm: Drop struct_mutex in submit path
> > drm/msm: Utilize gpu scheduler priorities
> > drm/msm/gem: Mark active before pinning
> >
> > drivers/gpu/drm/drm_gem.c | 22 --
> > drivers/gpu/drm/msm/Kconfig | 1 +
> > drivers/gpu/drm/msm/Makefile | 1 +
> > drivers/gpu/drm/msm/adreno/a5xx_debugfs.c | 4 +-
> > drivers/gpu/drm/msm/adreno/a5xx_gpu.c | 6 +-
> > drivers/gpu/drm/msm/adreno/a5xx_power.c | 2 +-
> > drivers/gpu/drm/msm/adreno/a5xx_preempt.c | 7 +-
> > drivers/gpu/drm/msm/adreno/a6xx_gmu.c | 12 +-
> > drivers/gpu/drm/msm/adreno/a6xx_gpu.c | 6 +-
> > drivers/gpu/drm/msm/adreno/a6xx_gpu_state.c | 4 +-
> > drivers/gpu/drm/msm/adreno/adreno_gpu.c | 6 +-
> > drivers/gpu/drm/msm/msm_drv.c | 30 ++-
> > drivers/gpu/drm/msm/msm_fence.c | 53 +----
> > drivers/gpu/drm/msm/msm_fence.h | 44 +++-
> > drivers/gpu/drm/msm/msm_gem.c | 94 +-------
> > drivers/gpu/drm/msm/msm_gem.h | 47 ++--
> > drivers/gpu/drm/msm/msm_gem_submit.c | 344 +++++++++++++++++-----------
> > drivers/gpu/drm/msm/msm_gpu.c | 220 ++++--------------
> > drivers/gpu/drm/msm/msm_gpu.h | 139 ++++++++++-
> > drivers/gpu/drm/msm/msm_gpu_devfreq.c | 203 ++++++++++++++++
> > drivers/gpu/drm/msm/msm_rd.c | 6 +-
> > drivers/gpu/drm/msm/msm_ringbuffer.c | 69 +++++-
> > drivers/gpu/drm/msm/msm_ringbuffer.h | 12 +
> > drivers/gpu/drm/msm/msm_submitqueue.c | 53 +++--
> > include/drm/drm_gem.h | 2 -
> > include/uapi/drm/msm_drm.h | 14 +-
> > 26 files changed, 865 insertions(+), 536 deletions(-)
> > create mode 100644 drivers/gpu/drm/msm/msm_gpu_devfreq.c
> >
>
> --
> Kind Regards,
> Caleb (they/them)
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [early pull] drm/msm: drm-msm-next-2021-07-28 for v5.15
2021-07-29 1:02 ` Rob Clark
@ 2021-07-29 2:18 ` Caleb Connolly
-1 siblings, 0 replies; 12+ messages in thread
From: Caleb Connolly @ 2021-07-29 2:18 UTC (permalink / raw)
To: Rob Clark, Jordan Crouse
Cc: Dave Airlie, Daniel Vetter, freedreno, dri-devel, linux-arm-msm,
Abhinav Kumar, Dmitry Baryshkov
On 29/07/2021 02:02, Rob Clark wrote:
> Jordan, any idea if more frequent frequency changes would for some
> reason make a630 grumpy? I was expecting it should be somewhat
> similar to a618 (same GMU fw, etc). The main result of that patch
> should be clamping to min freq when gpu goes idle, and the toggling
> back to devfreq provided freq on idle->active transition. So there
> might be more frequent freq transitions.
>
> Caleb, I don't suppose you could somehow delay starting UI and get
> some traces? Something along the lines of:
>
> localhost ~ # cd /sys/kernel/debug/tracing/
> localhost /sys/kernel/debug/tracing # echo 1 > events/drm_msm_gpu/enable
> localhost /sys/kernel/debug/tracing # echo 1 > tracing_on
> localhost /sys/kernel/debug/tracing # cat trace_pipe
Sure, here's the last ~1k lines of the trace logs:
https://paste.ubuntu.com/p/XMKjKDWxYg/
And what I managed to get from dmesg before the crash (mostly the same
as before): https://paste.ubuntu.com/p/kGVtRHDWKH/
>
> Does adding an 'if (1) return' at the top of msm_devfreq_idle() help?
> That should bypass the clamping to min freq when the GPU isn't doing
> anything and reduce the # of freq transitions. I suppose we could
> opt-in to this behavior on a per-gpu basis..
Yeah, that seems to resolve the issue, although I got the following
probably unrelated (?) error on rebooting the device:
[ 134.994449] [drm:dpu_encoder_vsync_event_handler:1749] [dpu
error]invalid parameters
I wonder if the PocoPhone F1 has the same problem...
>
> BR,
> -R
>
> On Wed, Jul 28, 2021 at 5:35 PM Caleb Connolly
> <caleb.connolly@linaro.org> wrote:
>>
>> Hi Rob,
>>
>> This series causes a fatal crash on my Oneplus 6, the device goes to
>> Qualcomm crashdump mode shortly after reaching UI with the following errors:
>>
>> https://paste.ubuntu.com/p/HvjmzZYtgw/
>>
>> I did a git bisect and the patch ("drm/msm: Devfreq tuning") seems to be
>> the cause of the crash, reverting it resolves the issue.
>>
>>
>> On 28/07/2021 21:52, Rob Clark wrote:
>>> Hi Dave & Daniel,
>>>
>>> An early pull for v5.15 (there'll be more coming in a week or two),
>>> consisting of the drm/scheduler conversion and a couple other small
>>> series that one was based one. Mostly sending this now because IIUC
>>> danvet wanted it in drm-next so he could rebase on it. (Daniel, if
>>> you disagree then speak up, and I'll instead include this in the main
>>> pull request once that is ready.)
>>>
>>> This also has a core patch to drop drm_gem_object_put_locked() now
>>> that the last use of it is removed.
>>>
>>> The following changes since commit ff1176468d368232b684f75e82563369208bc371:
>>>
>>> Linux 5.14-rc3 (2021-07-25 15:35:14 -0700)
>>>
>>> are available in the Git repository at:
>>>
>>> https://gitlab.freedesktop.org/drm/msm.git drm-msm-next-2021-07-28
>>>
>>> for you to fetch changes up to 4541e4f2225c30b0e9442be9eb2fb8b7086cdd1f:
>>>
>>> drm/msm/gem: Mark active before pinning (2021-07-28 09:19:00 -0700)
>>>
>>> ----------------------------------------------------------------
>>> Rob Clark (18):
>>> drm/msm: Let fences read directly from memptrs
>>> drm/msm: Signal fences sooner
>>> drm/msm: Split out devfreq handling
>>> drm/msm: Split out get_freq() helper
>>> drm/msm: Devfreq tuning
>>> drm/msm: Docs and misc cleanup
>>> drm/msm: Small submitqueue creation cleanup
>>> drm/msm: drop drm_gem_object_put_locked()
>>> drm: Drop drm_gem_object_put_locked()
>>> drm/msm/submit: Simplify out-fence-fd handling
>>> drm/msm: Consolidate submit bo state
>>> drm/msm: Track "seqno" fences by idr
>>> drm/msm: Return ERR_PTR() from submit_create()
>>> drm/msm: Conversion to drm scheduler
>>> drm/msm: Drop submit bo_list
>>> drm/msm: Drop struct_mutex in submit path
>>> drm/msm: Utilize gpu scheduler priorities
>>> drm/msm/gem: Mark active before pinning
>>>
>>> drivers/gpu/drm/drm_gem.c | 22 --
>>> drivers/gpu/drm/msm/Kconfig | 1 +
>>> drivers/gpu/drm/msm/Makefile | 1 +
>>> drivers/gpu/drm/msm/adreno/a5xx_debugfs.c | 4 +-
>>> drivers/gpu/drm/msm/adreno/a5xx_gpu.c | 6 +-
>>> drivers/gpu/drm/msm/adreno/a5xx_power.c | 2 +-
>>> drivers/gpu/drm/msm/adreno/a5xx_preempt.c | 7 +-
>>> drivers/gpu/drm/msm/adreno/a6xx_gmu.c | 12 +-
>>> drivers/gpu/drm/msm/adreno/a6xx_gpu.c | 6 +-
>>> drivers/gpu/drm/msm/adreno/a6xx_gpu_state.c | 4 +-
>>> drivers/gpu/drm/msm/adreno/adreno_gpu.c | 6 +-
>>> drivers/gpu/drm/msm/msm_drv.c | 30 ++-
>>> drivers/gpu/drm/msm/msm_fence.c | 53 +----
>>> drivers/gpu/drm/msm/msm_fence.h | 44 +++-
>>> drivers/gpu/drm/msm/msm_gem.c | 94 +-------
>>> drivers/gpu/drm/msm/msm_gem.h | 47 ++--
>>> drivers/gpu/drm/msm/msm_gem_submit.c | 344 +++++++++++++++++-----------
>>> drivers/gpu/drm/msm/msm_gpu.c | 220 ++++--------------
>>> drivers/gpu/drm/msm/msm_gpu.h | 139 ++++++++++-
>>> drivers/gpu/drm/msm/msm_gpu_devfreq.c | 203 ++++++++++++++++
>>> drivers/gpu/drm/msm/msm_rd.c | 6 +-
>>> drivers/gpu/drm/msm/msm_ringbuffer.c | 69 +++++-
>>> drivers/gpu/drm/msm/msm_ringbuffer.h | 12 +
>>> drivers/gpu/drm/msm/msm_submitqueue.c | 53 +++--
>>> include/drm/drm_gem.h | 2 -
>>> include/uapi/drm/msm_drm.h | 14 +-
>>> 26 files changed, 865 insertions(+), 536 deletions(-)
>>> create mode 100644 drivers/gpu/drm/msm/msm_gpu_devfreq.c
>>>
>>
>> --
>> Kind Regards,
>> Caleb (they/them)
--
Kind Regards,
Caleb (they/them)
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [early pull] drm/msm: drm-msm-next-2021-07-28 for v5.15
@ 2021-07-29 2:18 ` Caleb Connolly
0 siblings, 0 replies; 12+ messages in thread
From: Caleb Connolly @ 2021-07-29 2:18 UTC (permalink / raw)
To: Rob Clark, Jordan Crouse
Cc: Abhinav Kumar, dri-devel, linux-arm-msm, Dmitry Baryshkov, freedreno
On 29/07/2021 02:02, Rob Clark wrote:
> Jordan, any idea if more frequent frequency changes would for some
> reason make a630 grumpy? I was expecting it should be somewhat
> similar to a618 (same GMU fw, etc). The main result of that patch
> should be clamping to min freq when gpu goes idle, and the toggling
> back to devfreq provided freq on idle->active transition. So there
> might be more frequent freq transitions.
>
> Caleb, I don't suppose you could somehow delay starting UI and get
> some traces? Something along the lines of:
>
> localhost ~ # cd /sys/kernel/debug/tracing/
> localhost /sys/kernel/debug/tracing # echo 1 > events/drm_msm_gpu/enable
> localhost /sys/kernel/debug/tracing # echo 1 > tracing_on
> localhost /sys/kernel/debug/tracing # cat trace_pipe
Sure, here's the last ~1k lines of the trace logs:
https://paste.ubuntu.com/p/XMKjKDWxYg/
And what I managed to get from dmesg before the crash (mostly the same
as before): https://paste.ubuntu.com/p/kGVtRHDWKH/
>
> Does adding an 'if (1) return' at the top of msm_devfreq_idle() help?
> That should bypass the clamping to min freq when the GPU isn't doing
> anything and reduce the # of freq transitions. I suppose we could
> opt-in to this behavior on a per-gpu basis..
Yeah, that seems to resolve the issue, although I got the following
probably unrelated (?) error on rebooting the device:
[ 134.994449] [drm:dpu_encoder_vsync_event_handler:1749] [dpu
error]invalid parameters
I wonder if the PocoPhone F1 has the same problem...
>
> BR,
> -R
>
> On Wed, Jul 28, 2021 at 5:35 PM Caleb Connolly
> <caleb.connolly@linaro.org> wrote:
>>
>> Hi Rob,
>>
>> This series causes a fatal crash on my Oneplus 6, the device goes to
>> Qualcomm crashdump mode shortly after reaching UI with the following errors:
>>
>> https://paste.ubuntu.com/p/HvjmzZYtgw/
>>
>> I did a git bisect and the patch ("drm/msm: Devfreq tuning") seems to be
>> the cause of the crash, reverting it resolves the issue.
>>
>>
>> On 28/07/2021 21:52, Rob Clark wrote:
>>> Hi Dave & Daniel,
>>>
>>> An early pull for v5.15 (there'll be more coming in a week or two),
>>> consisting of the drm/scheduler conversion and a couple other small
>>> series that one was based one. Mostly sending this now because IIUC
>>> danvet wanted it in drm-next so he could rebase on it. (Daniel, if
>>> you disagree then speak up, and I'll instead include this in the main
>>> pull request once that is ready.)
>>>
>>> This also has a core patch to drop drm_gem_object_put_locked() now
>>> that the last use of it is removed.
>>>
>>> The following changes since commit ff1176468d368232b684f75e82563369208bc371:
>>>
>>> Linux 5.14-rc3 (2021-07-25 15:35:14 -0700)
>>>
>>> are available in the Git repository at:
>>>
>>> https://gitlab.freedesktop.org/drm/msm.git drm-msm-next-2021-07-28
>>>
>>> for you to fetch changes up to 4541e4f2225c30b0e9442be9eb2fb8b7086cdd1f:
>>>
>>> drm/msm/gem: Mark active before pinning (2021-07-28 09:19:00 -0700)
>>>
>>> ----------------------------------------------------------------
>>> Rob Clark (18):
>>> drm/msm: Let fences read directly from memptrs
>>> drm/msm: Signal fences sooner
>>> drm/msm: Split out devfreq handling
>>> drm/msm: Split out get_freq() helper
>>> drm/msm: Devfreq tuning
>>> drm/msm: Docs and misc cleanup
>>> drm/msm: Small submitqueue creation cleanup
>>> drm/msm: drop drm_gem_object_put_locked()
>>> drm: Drop drm_gem_object_put_locked()
>>> drm/msm/submit: Simplify out-fence-fd handling
>>> drm/msm: Consolidate submit bo state
>>> drm/msm: Track "seqno" fences by idr
>>> drm/msm: Return ERR_PTR() from submit_create()
>>> drm/msm: Conversion to drm scheduler
>>> drm/msm: Drop submit bo_list
>>> drm/msm: Drop struct_mutex in submit path
>>> drm/msm: Utilize gpu scheduler priorities
>>> drm/msm/gem: Mark active before pinning
>>>
>>> drivers/gpu/drm/drm_gem.c | 22 --
>>> drivers/gpu/drm/msm/Kconfig | 1 +
>>> drivers/gpu/drm/msm/Makefile | 1 +
>>> drivers/gpu/drm/msm/adreno/a5xx_debugfs.c | 4 +-
>>> drivers/gpu/drm/msm/adreno/a5xx_gpu.c | 6 +-
>>> drivers/gpu/drm/msm/adreno/a5xx_power.c | 2 +-
>>> drivers/gpu/drm/msm/adreno/a5xx_preempt.c | 7 +-
>>> drivers/gpu/drm/msm/adreno/a6xx_gmu.c | 12 +-
>>> drivers/gpu/drm/msm/adreno/a6xx_gpu.c | 6 +-
>>> drivers/gpu/drm/msm/adreno/a6xx_gpu_state.c | 4 +-
>>> drivers/gpu/drm/msm/adreno/adreno_gpu.c | 6 +-
>>> drivers/gpu/drm/msm/msm_drv.c | 30 ++-
>>> drivers/gpu/drm/msm/msm_fence.c | 53 +----
>>> drivers/gpu/drm/msm/msm_fence.h | 44 +++-
>>> drivers/gpu/drm/msm/msm_gem.c | 94 +-------
>>> drivers/gpu/drm/msm/msm_gem.h | 47 ++--
>>> drivers/gpu/drm/msm/msm_gem_submit.c | 344 +++++++++++++++++-----------
>>> drivers/gpu/drm/msm/msm_gpu.c | 220 ++++--------------
>>> drivers/gpu/drm/msm/msm_gpu.h | 139 ++++++++++-
>>> drivers/gpu/drm/msm/msm_gpu_devfreq.c | 203 ++++++++++++++++
>>> drivers/gpu/drm/msm/msm_rd.c | 6 +-
>>> drivers/gpu/drm/msm/msm_ringbuffer.c | 69 +++++-
>>> drivers/gpu/drm/msm/msm_ringbuffer.h | 12 +
>>> drivers/gpu/drm/msm/msm_submitqueue.c | 53 +++--
>>> include/drm/drm_gem.h | 2 -
>>> include/uapi/drm/msm_drm.h | 14 +-
>>> 26 files changed, 865 insertions(+), 536 deletions(-)
>>> create mode 100644 drivers/gpu/drm/msm/msm_gpu_devfreq.c
>>>
>>
>> --
>> Kind Regards,
>> Caleb (they/them)
--
Kind Regards,
Caleb (they/them)
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [early pull] drm/msm: drm-msm-next-2021-07-28 for v5.15
2021-07-29 2:18 ` Caleb Connolly
@ 2021-07-29 2:50 ` Rob Clark
-1 siblings, 0 replies; 12+ messages in thread
From: Rob Clark @ 2021-07-29 2:50 UTC (permalink / raw)
To: Caleb Connolly
Cc: Jordan Crouse, Dave Airlie, Daniel Vetter, freedreno, dri-devel,
linux-arm-msm, Abhinav Kumar, Dmitry Baryshkov
On Wed, Jul 28, 2021 at 7:18 PM Caleb Connolly
<caleb.connolly@linaro.org> wrote:
>
>
>
> On 29/07/2021 02:02, Rob Clark wrote:
> > Jordan, any idea if more frequent frequency changes would for some
> > reason make a630 grumpy? I was expecting it should be somewhat
> > similar to a618 (same GMU fw, etc). The main result of that patch
> > should be clamping to min freq when gpu goes idle, and the toggling
> > back to devfreq provided freq on idle->active transition. So there
> > might be more frequent freq transitions.
> >
> > Caleb, I don't suppose you could somehow delay starting UI and get
> > some traces? Something along the lines of:
> >
> > localhost ~ # cd /sys/kernel/debug/tracing/
> > localhost /sys/kernel/debug/tracing # echo 1 > events/drm_msm_gpu/enable
> > localhost /sys/kernel/debug/tracing # echo 1 > tracing_on
> > localhost /sys/kernel/debug/tracing # cat trace_pipe
> Sure, here's the last ~1k lines of the trace logs:
> https://paste.ubuntu.com/p/XMKjKDWxYg/
> And what I managed to get from dmesg before the crash (mostly the same
> as before): https://paste.ubuntu.com/p/kGVtRHDWKH/
> >
> > Does adding an 'if (1) return' at the top of msm_devfreq_idle() help?
> > That should bypass the clamping to min freq when the GPU isn't doing
> > anything and reduce the # of freq transitions. I suppose we could
> > opt-in to this behavior on a per-gpu basis..
> Yeah, that seems to resolve the issue, although I got the following
> probably unrelated (?) error on rebooting the device:
> [ 134.994449] [drm:dpu_encoder_vsync_event_handler:1749] [dpu
> error]invalid parameters
I think that should probably be unrelated..
Based on the traces, I'm seeing rapid toggling between idle freq and
non-idle freq.. but no invalid freq's (assuming the dts opp table is
correct) so I *guess* there is maybe some sort of race condition
communicating with GMU or some other issue with rapid freq transition?
Maybe Jordan has some ideas.
The earlier dmesg you posted look like gpu getting cranky about what
looks like a valid opcode, and then it goes off into the weeds.. when
you start seeing things like "0xDEAFBEAF" I think that means the GPU
has lost context (ie. power collapse and back, and now it is reading
bogus power-on default values).
I think I can put together a patch to make the "clamp to min freq when
gpu is idle" opt-in so we can enable it per-gpu once someone has
confirmed that it doesn't cause problems. I guess that would at least
work as a short term solution. But not sure if that is just papering
over some gpu/gmu bug (or maybe gdsc/clk bug), or if it is a legit
workaround for some limitation..
BR,
-R
>
> I wonder if the PocoPhone F1 has the same problem...
> >
> > BR,
> > -R
> >
> > On Wed, Jul 28, 2021 at 5:35 PM Caleb Connolly
> > <caleb.connolly@linaro.org> wrote:
> >>
> >> Hi Rob,
> >>
> >> This series causes a fatal crash on my Oneplus 6, the device goes to
> >> Qualcomm crashdump mode shortly after reaching UI with the following errors:
> >>
> >> https://paste.ubuntu.com/p/HvjmzZYtgw/
> >>
> >> I did a git bisect and the patch ("drm/msm: Devfreq tuning") seems to be
> >> the cause of the crash, reverting it resolves the issue.
> >>
> >>
> >> On 28/07/2021 21:52, Rob Clark wrote:
> >>> Hi Dave & Daniel,
> >>>
> >>> An early pull for v5.15 (there'll be more coming in a week or two),
> >>> consisting of the drm/scheduler conversion and a couple other small
> >>> series that one was based one. Mostly sending this now because IIUC
> >>> danvet wanted it in drm-next so he could rebase on it. (Daniel, if
> >>> you disagree then speak up, and I'll instead include this in the main
> >>> pull request once that is ready.)
> >>>
> >>> This also has a core patch to drop drm_gem_object_put_locked() now
> >>> that the last use of it is removed.
> >>>
> >>> The following changes since commit ff1176468d368232b684f75e82563369208bc371:
> >>>
> >>> Linux 5.14-rc3 (2021-07-25 15:35:14 -0700)
> >>>
> >>> are available in the Git repository at:
> >>>
> >>> https://gitlab.freedesktop.org/drm/msm.git drm-msm-next-2021-07-28
> >>>
> >>> for you to fetch changes up to 4541e4f2225c30b0e9442be9eb2fb8b7086cdd1f:
> >>>
> >>> drm/msm/gem: Mark active before pinning (2021-07-28 09:19:00 -0700)
> >>>
> >>> ----------------------------------------------------------------
> >>> Rob Clark (18):
> >>> drm/msm: Let fences read directly from memptrs
> >>> drm/msm: Signal fences sooner
> >>> drm/msm: Split out devfreq handling
> >>> drm/msm: Split out get_freq() helper
> >>> drm/msm: Devfreq tuning
> >>> drm/msm: Docs and misc cleanup
> >>> drm/msm: Small submitqueue creation cleanup
> >>> drm/msm: drop drm_gem_object_put_locked()
> >>> drm: Drop drm_gem_object_put_locked()
> >>> drm/msm/submit: Simplify out-fence-fd handling
> >>> drm/msm: Consolidate submit bo state
> >>> drm/msm: Track "seqno" fences by idr
> >>> drm/msm: Return ERR_PTR() from submit_create()
> >>> drm/msm: Conversion to drm scheduler
> >>> drm/msm: Drop submit bo_list
> >>> drm/msm: Drop struct_mutex in submit path
> >>> drm/msm: Utilize gpu scheduler priorities
> >>> drm/msm/gem: Mark active before pinning
> >>>
> >>> drivers/gpu/drm/drm_gem.c | 22 --
> >>> drivers/gpu/drm/msm/Kconfig | 1 +
> >>> drivers/gpu/drm/msm/Makefile | 1 +
> >>> drivers/gpu/drm/msm/adreno/a5xx_debugfs.c | 4 +-
> >>> drivers/gpu/drm/msm/adreno/a5xx_gpu.c | 6 +-
> >>> drivers/gpu/drm/msm/adreno/a5xx_power.c | 2 +-
> >>> drivers/gpu/drm/msm/adreno/a5xx_preempt.c | 7 +-
> >>> drivers/gpu/drm/msm/adreno/a6xx_gmu.c | 12 +-
> >>> drivers/gpu/drm/msm/adreno/a6xx_gpu.c | 6 +-
> >>> drivers/gpu/drm/msm/adreno/a6xx_gpu_state.c | 4 +-
> >>> drivers/gpu/drm/msm/adreno/adreno_gpu.c | 6 +-
> >>> drivers/gpu/drm/msm/msm_drv.c | 30 ++-
> >>> drivers/gpu/drm/msm/msm_fence.c | 53 +----
> >>> drivers/gpu/drm/msm/msm_fence.h | 44 +++-
> >>> drivers/gpu/drm/msm/msm_gem.c | 94 +-------
> >>> drivers/gpu/drm/msm/msm_gem.h | 47 ++--
> >>> drivers/gpu/drm/msm/msm_gem_submit.c | 344 +++++++++++++++++-----------
> >>> drivers/gpu/drm/msm/msm_gpu.c | 220 ++++--------------
> >>> drivers/gpu/drm/msm/msm_gpu.h | 139 ++++++++++-
> >>> drivers/gpu/drm/msm/msm_gpu_devfreq.c | 203 ++++++++++++++++
> >>> drivers/gpu/drm/msm/msm_rd.c | 6 +-
> >>> drivers/gpu/drm/msm/msm_ringbuffer.c | 69 +++++-
> >>> drivers/gpu/drm/msm/msm_ringbuffer.h | 12 +
> >>> drivers/gpu/drm/msm/msm_submitqueue.c | 53 +++--
> >>> include/drm/drm_gem.h | 2 -
> >>> include/uapi/drm/msm_drm.h | 14 +-
> >>> 26 files changed, 865 insertions(+), 536 deletions(-)
> >>> create mode 100644 drivers/gpu/drm/msm/msm_gpu_devfreq.c
> >>>
> >>
> >> --
> >> Kind Regards,
> >> Caleb (they/them)
>
> --
> Kind Regards,
> Caleb (they/them)
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [early pull] drm/msm: drm-msm-next-2021-07-28 for v5.15
@ 2021-07-29 2:50 ` Rob Clark
0 siblings, 0 replies; 12+ messages in thread
From: Rob Clark @ 2021-07-29 2:50 UTC (permalink / raw)
To: Caleb Connolly
Cc: linux-arm-msm, dri-devel, Jordan Crouse, Abhinav Kumar,
Dmitry Baryshkov, freedreno
On Wed, Jul 28, 2021 at 7:18 PM Caleb Connolly
<caleb.connolly@linaro.org> wrote:
>
>
>
> On 29/07/2021 02:02, Rob Clark wrote:
> > Jordan, any idea if more frequent frequency changes would for some
> > reason make a630 grumpy? I was expecting it should be somewhat
> > similar to a618 (same GMU fw, etc). The main result of that patch
> > should be clamping to min freq when gpu goes idle, and the toggling
> > back to devfreq provided freq on idle->active transition. So there
> > might be more frequent freq transitions.
> >
> > Caleb, I don't suppose you could somehow delay starting UI and get
> > some traces? Something along the lines of:
> >
> > localhost ~ # cd /sys/kernel/debug/tracing/
> > localhost /sys/kernel/debug/tracing # echo 1 > events/drm_msm_gpu/enable
> > localhost /sys/kernel/debug/tracing # echo 1 > tracing_on
> > localhost /sys/kernel/debug/tracing # cat trace_pipe
> Sure, here's the last ~1k lines of the trace logs:
> https://paste.ubuntu.com/p/XMKjKDWxYg/
> And what I managed to get from dmesg before the crash (mostly the same
> as before): https://paste.ubuntu.com/p/kGVtRHDWKH/
> >
> > Does adding an 'if (1) return' at the top of msm_devfreq_idle() help?
> > That should bypass the clamping to min freq when the GPU isn't doing
> > anything and reduce the # of freq transitions. I suppose we could
> > opt-in to this behavior on a per-gpu basis..
> Yeah, that seems to resolve the issue, although I got the following
> probably unrelated (?) error on rebooting the device:
> [ 134.994449] [drm:dpu_encoder_vsync_event_handler:1749] [dpu
> error]invalid parameters
I think that should probably be unrelated..
Based on the traces, I'm seeing rapid toggling between idle freq and
non-idle freq.. but no invalid freq's (assuming the dts opp table is
correct) so I *guess* there is maybe some sort of race condition
communicating with GMU or some other issue with rapid freq transition?
Maybe Jordan has some ideas.
The earlier dmesg you posted look like gpu getting cranky about what
looks like a valid opcode, and then it goes off into the weeds.. when
you start seeing things like "0xDEAFBEAF" I think that means the GPU
has lost context (ie. power collapse and back, and now it is reading
bogus power-on default values).
I think I can put together a patch to make the "clamp to min freq when
gpu is idle" opt-in so we can enable it per-gpu once someone has
confirmed that it doesn't cause problems. I guess that would at least
work as a short term solution. But not sure if that is just papering
over some gpu/gmu bug (or maybe gdsc/clk bug), or if it is a legit
workaround for some limitation..
BR,
-R
>
> I wonder if the PocoPhone F1 has the same problem...
> >
> > BR,
> > -R
> >
> > On Wed, Jul 28, 2021 at 5:35 PM Caleb Connolly
> > <caleb.connolly@linaro.org> wrote:
> >>
> >> Hi Rob,
> >>
> >> This series causes a fatal crash on my Oneplus 6, the device goes to
> >> Qualcomm crashdump mode shortly after reaching UI with the following errors:
> >>
> >> https://paste.ubuntu.com/p/HvjmzZYtgw/
> >>
> >> I did a git bisect and the patch ("drm/msm: Devfreq tuning") seems to be
> >> the cause of the crash, reverting it resolves the issue.
> >>
> >>
> >> On 28/07/2021 21:52, Rob Clark wrote:
> >>> Hi Dave & Daniel,
> >>>
> >>> An early pull for v5.15 (there'll be more coming in a week or two),
> >>> consisting of the drm/scheduler conversion and a couple other small
> >>> series that one was based one. Mostly sending this now because IIUC
> >>> danvet wanted it in drm-next so he could rebase on it. (Daniel, if
> >>> you disagree then speak up, and I'll instead include this in the main
> >>> pull request once that is ready.)
> >>>
> >>> This also has a core patch to drop drm_gem_object_put_locked() now
> >>> that the last use of it is removed.
> >>>
> >>> The following changes since commit ff1176468d368232b684f75e82563369208bc371:
> >>>
> >>> Linux 5.14-rc3 (2021-07-25 15:35:14 -0700)
> >>>
> >>> are available in the Git repository at:
> >>>
> >>> https://gitlab.freedesktop.org/drm/msm.git drm-msm-next-2021-07-28
> >>>
> >>> for you to fetch changes up to 4541e4f2225c30b0e9442be9eb2fb8b7086cdd1f:
> >>>
> >>> drm/msm/gem: Mark active before pinning (2021-07-28 09:19:00 -0700)
> >>>
> >>> ----------------------------------------------------------------
> >>> Rob Clark (18):
> >>> drm/msm: Let fences read directly from memptrs
> >>> drm/msm: Signal fences sooner
> >>> drm/msm: Split out devfreq handling
> >>> drm/msm: Split out get_freq() helper
> >>> drm/msm: Devfreq tuning
> >>> drm/msm: Docs and misc cleanup
> >>> drm/msm: Small submitqueue creation cleanup
> >>> drm/msm: drop drm_gem_object_put_locked()
> >>> drm: Drop drm_gem_object_put_locked()
> >>> drm/msm/submit: Simplify out-fence-fd handling
> >>> drm/msm: Consolidate submit bo state
> >>> drm/msm: Track "seqno" fences by idr
> >>> drm/msm: Return ERR_PTR() from submit_create()
> >>> drm/msm: Conversion to drm scheduler
> >>> drm/msm: Drop submit bo_list
> >>> drm/msm: Drop struct_mutex in submit path
> >>> drm/msm: Utilize gpu scheduler priorities
> >>> drm/msm/gem: Mark active before pinning
> >>>
> >>> drivers/gpu/drm/drm_gem.c | 22 --
> >>> drivers/gpu/drm/msm/Kconfig | 1 +
> >>> drivers/gpu/drm/msm/Makefile | 1 +
> >>> drivers/gpu/drm/msm/adreno/a5xx_debugfs.c | 4 +-
> >>> drivers/gpu/drm/msm/adreno/a5xx_gpu.c | 6 +-
> >>> drivers/gpu/drm/msm/adreno/a5xx_power.c | 2 +-
> >>> drivers/gpu/drm/msm/adreno/a5xx_preempt.c | 7 +-
> >>> drivers/gpu/drm/msm/adreno/a6xx_gmu.c | 12 +-
> >>> drivers/gpu/drm/msm/adreno/a6xx_gpu.c | 6 +-
> >>> drivers/gpu/drm/msm/adreno/a6xx_gpu_state.c | 4 +-
> >>> drivers/gpu/drm/msm/adreno/adreno_gpu.c | 6 +-
> >>> drivers/gpu/drm/msm/msm_drv.c | 30 ++-
> >>> drivers/gpu/drm/msm/msm_fence.c | 53 +----
> >>> drivers/gpu/drm/msm/msm_fence.h | 44 +++-
> >>> drivers/gpu/drm/msm/msm_gem.c | 94 +-------
> >>> drivers/gpu/drm/msm/msm_gem.h | 47 ++--
> >>> drivers/gpu/drm/msm/msm_gem_submit.c | 344 +++++++++++++++++-----------
> >>> drivers/gpu/drm/msm/msm_gpu.c | 220 ++++--------------
> >>> drivers/gpu/drm/msm/msm_gpu.h | 139 ++++++++++-
> >>> drivers/gpu/drm/msm/msm_gpu_devfreq.c | 203 ++++++++++++++++
> >>> drivers/gpu/drm/msm/msm_rd.c | 6 +-
> >>> drivers/gpu/drm/msm/msm_ringbuffer.c | 69 +++++-
> >>> drivers/gpu/drm/msm/msm_ringbuffer.h | 12 +
> >>> drivers/gpu/drm/msm/msm_submitqueue.c | 53 +++--
> >>> include/drm/drm_gem.h | 2 -
> >>> include/uapi/drm/msm_drm.h | 14 +-
> >>> 26 files changed, 865 insertions(+), 536 deletions(-)
> >>> create mode 100644 drivers/gpu/drm/msm/msm_gpu_devfreq.c
> >>>
> >>
> >> --
> >> Kind Regards,
> >> Caleb (they/them)
>
> --
> Kind Regards,
> Caleb (they/them)
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [early pull] drm/msm: drm-msm-next-2021-07-28 for v5.15
2021-07-28 20:52 ` Rob Clark
@ 2021-07-29 7:13 ` Daniel Vetter
-1 siblings, 0 replies; 12+ messages in thread
From: Daniel Vetter @ 2021-07-29 7:13 UTC (permalink / raw)
To: Rob Clark
Cc: Dave Airlie, Daniel Vetter, freedreno, dri-devel, linux-arm-msm,
Jordan Crouse, Abhinav Kumar, Dmitry Baryshkov
On Wed, Jul 28, 2021 at 01:52:42PM -0700, Rob Clark wrote:
> Hi Dave & Daniel,
>
> An early pull for v5.15 (there'll be more coming in a week or two),
> consisting of the drm/scheduler conversion and a couple other small
> series that one was based one. Mostly sending this now because IIUC
> danvet wanted it in drm-next so he could rebase on it. (Daniel, if
> you disagree then speak up, and I'll instead include this in the main
> pull request once that is ready.)
I like, but might be some time until I managed to rebase it all and adjust
msm too. Thanks for doing the early pull.
-Daniel
>
> This also has a core patch to drop drm_gem_object_put_locked() now
> that the last use of it is removed.
>
> The following changes since commit ff1176468d368232b684f75e82563369208bc371:
>
> Linux 5.14-rc3 (2021-07-25 15:35:14 -0700)
>
> are available in the Git repository at:
>
> https://gitlab.freedesktop.org/drm/msm.git drm-msm-next-2021-07-28
>
> for you to fetch changes up to 4541e4f2225c30b0e9442be9eb2fb8b7086cdd1f:
>
> drm/msm/gem: Mark active before pinning (2021-07-28 09:19:00 -0700)
>
> ----------------------------------------------------------------
> Rob Clark (18):
> drm/msm: Let fences read directly from memptrs
> drm/msm: Signal fences sooner
> drm/msm: Split out devfreq handling
> drm/msm: Split out get_freq() helper
> drm/msm: Devfreq tuning
> drm/msm: Docs and misc cleanup
> drm/msm: Small submitqueue creation cleanup
> drm/msm: drop drm_gem_object_put_locked()
> drm: Drop drm_gem_object_put_locked()
> drm/msm/submit: Simplify out-fence-fd handling
> drm/msm: Consolidate submit bo state
> drm/msm: Track "seqno" fences by idr
> drm/msm: Return ERR_PTR() from submit_create()
> drm/msm: Conversion to drm scheduler
> drm/msm: Drop submit bo_list
> drm/msm: Drop struct_mutex in submit path
> drm/msm: Utilize gpu scheduler priorities
> drm/msm/gem: Mark active before pinning
>
> drivers/gpu/drm/drm_gem.c | 22 --
> drivers/gpu/drm/msm/Kconfig | 1 +
> drivers/gpu/drm/msm/Makefile | 1 +
> drivers/gpu/drm/msm/adreno/a5xx_debugfs.c | 4 +-
> drivers/gpu/drm/msm/adreno/a5xx_gpu.c | 6 +-
> drivers/gpu/drm/msm/adreno/a5xx_power.c | 2 +-
> drivers/gpu/drm/msm/adreno/a5xx_preempt.c | 7 +-
> drivers/gpu/drm/msm/adreno/a6xx_gmu.c | 12 +-
> drivers/gpu/drm/msm/adreno/a6xx_gpu.c | 6 +-
> drivers/gpu/drm/msm/adreno/a6xx_gpu_state.c | 4 +-
> drivers/gpu/drm/msm/adreno/adreno_gpu.c | 6 +-
> drivers/gpu/drm/msm/msm_drv.c | 30 ++-
> drivers/gpu/drm/msm/msm_fence.c | 53 +----
> drivers/gpu/drm/msm/msm_fence.h | 44 +++-
> drivers/gpu/drm/msm/msm_gem.c | 94 +-------
> drivers/gpu/drm/msm/msm_gem.h | 47 ++--
> drivers/gpu/drm/msm/msm_gem_submit.c | 344 +++++++++++++++++-----------
> drivers/gpu/drm/msm/msm_gpu.c | 220 ++++--------------
> drivers/gpu/drm/msm/msm_gpu.h | 139 ++++++++++-
> drivers/gpu/drm/msm/msm_gpu_devfreq.c | 203 ++++++++++++++++
> drivers/gpu/drm/msm/msm_rd.c | 6 +-
> drivers/gpu/drm/msm/msm_ringbuffer.c | 69 +++++-
> drivers/gpu/drm/msm/msm_ringbuffer.h | 12 +
> drivers/gpu/drm/msm/msm_submitqueue.c | 53 +++--
> include/drm/drm_gem.h | 2 -
> include/uapi/drm/msm_drm.h | 14 +-
> 26 files changed, 865 insertions(+), 536 deletions(-)
> create mode 100644 drivers/gpu/drm/msm/msm_gpu_devfreq.c
--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [early pull] drm/msm: drm-msm-next-2021-07-28 for v5.15
@ 2021-07-29 7:13 ` Daniel Vetter
0 siblings, 0 replies; 12+ messages in thread
From: Daniel Vetter @ 2021-07-29 7:13 UTC (permalink / raw)
To: Rob Clark
Cc: dri-devel, Jordan Crouse, Abhinav Kumar, linux-arm-msm,
Dmitry Baryshkov, freedreno
On Wed, Jul 28, 2021 at 01:52:42PM -0700, Rob Clark wrote:
> Hi Dave & Daniel,
>
> An early pull for v5.15 (there'll be more coming in a week or two),
> consisting of the drm/scheduler conversion and a couple other small
> series that one was based one. Mostly sending this now because IIUC
> danvet wanted it in drm-next so he could rebase on it. (Daniel, if
> you disagree then speak up, and I'll instead include this in the main
> pull request once that is ready.)
I like, but might be some time until I managed to rebase it all and adjust
msm too. Thanks for doing the early pull.
-Daniel
>
> This also has a core patch to drop drm_gem_object_put_locked() now
> that the last use of it is removed.
>
> The following changes since commit ff1176468d368232b684f75e82563369208bc371:
>
> Linux 5.14-rc3 (2021-07-25 15:35:14 -0700)
>
> are available in the Git repository at:
>
> https://gitlab.freedesktop.org/drm/msm.git drm-msm-next-2021-07-28
>
> for you to fetch changes up to 4541e4f2225c30b0e9442be9eb2fb8b7086cdd1f:
>
> drm/msm/gem: Mark active before pinning (2021-07-28 09:19:00 -0700)
>
> ----------------------------------------------------------------
> Rob Clark (18):
> drm/msm: Let fences read directly from memptrs
> drm/msm: Signal fences sooner
> drm/msm: Split out devfreq handling
> drm/msm: Split out get_freq() helper
> drm/msm: Devfreq tuning
> drm/msm: Docs and misc cleanup
> drm/msm: Small submitqueue creation cleanup
> drm/msm: drop drm_gem_object_put_locked()
> drm: Drop drm_gem_object_put_locked()
> drm/msm/submit: Simplify out-fence-fd handling
> drm/msm: Consolidate submit bo state
> drm/msm: Track "seqno" fences by idr
> drm/msm: Return ERR_PTR() from submit_create()
> drm/msm: Conversion to drm scheduler
> drm/msm: Drop submit bo_list
> drm/msm: Drop struct_mutex in submit path
> drm/msm: Utilize gpu scheduler priorities
> drm/msm/gem: Mark active before pinning
>
> drivers/gpu/drm/drm_gem.c | 22 --
> drivers/gpu/drm/msm/Kconfig | 1 +
> drivers/gpu/drm/msm/Makefile | 1 +
> drivers/gpu/drm/msm/adreno/a5xx_debugfs.c | 4 +-
> drivers/gpu/drm/msm/adreno/a5xx_gpu.c | 6 +-
> drivers/gpu/drm/msm/adreno/a5xx_power.c | 2 +-
> drivers/gpu/drm/msm/adreno/a5xx_preempt.c | 7 +-
> drivers/gpu/drm/msm/adreno/a6xx_gmu.c | 12 +-
> drivers/gpu/drm/msm/adreno/a6xx_gpu.c | 6 +-
> drivers/gpu/drm/msm/adreno/a6xx_gpu_state.c | 4 +-
> drivers/gpu/drm/msm/adreno/adreno_gpu.c | 6 +-
> drivers/gpu/drm/msm/msm_drv.c | 30 ++-
> drivers/gpu/drm/msm/msm_fence.c | 53 +----
> drivers/gpu/drm/msm/msm_fence.h | 44 +++-
> drivers/gpu/drm/msm/msm_gem.c | 94 +-------
> drivers/gpu/drm/msm/msm_gem.h | 47 ++--
> drivers/gpu/drm/msm/msm_gem_submit.c | 344 +++++++++++++++++-----------
> drivers/gpu/drm/msm/msm_gpu.c | 220 ++++--------------
> drivers/gpu/drm/msm/msm_gpu.h | 139 ++++++++++-
> drivers/gpu/drm/msm/msm_gpu_devfreq.c | 203 ++++++++++++++++
> drivers/gpu/drm/msm/msm_rd.c | 6 +-
> drivers/gpu/drm/msm/msm_ringbuffer.c | 69 +++++-
> drivers/gpu/drm/msm/msm_ringbuffer.h | 12 +
> drivers/gpu/drm/msm/msm_submitqueue.c | 53 +++--
> include/drm/drm_gem.h | 2 -
> include/uapi/drm/msm_drm.h | 14 +-
> 26 files changed, 865 insertions(+), 536 deletions(-)
> create mode 100644 drivers/gpu/drm/msm/msm_gpu_devfreq.c
--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
^ permalink raw reply [flat|nested] 12+ messages in thread
end of thread, other threads:[~2021-07-29 7:13 UTC | newest]
Thread overview: 12+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
[not found] <SKuAxGshCZFzlguCiEJOU0kAFCJ9WDGK_qCmPESnrghqei0-VIp4DD5vL58OEJCq2B-AkvF1az0EedzkGjSNLQ==@protonmail.internalid>
2021-07-28 20:52 ` [early pull] drm/msm: drm-msm-next-2021-07-28 for v5.15 Rob Clark
2021-07-28 20:52 ` Rob Clark
2021-07-29 0:35 ` Caleb Connolly
2021-07-29 0:35 ` Caleb Connolly
2021-07-29 1:02 ` Rob Clark
2021-07-29 1:02 ` Rob Clark
2021-07-29 2:18 ` Caleb Connolly
2021-07-29 2:18 ` Caleb Connolly
2021-07-29 2:50 ` Rob Clark
2021-07-29 2:50 ` Rob Clark
2021-07-29 7:13 ` Daniel Vetter
2021-07-29 7:13 ` Daniel Vetter
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.