* [PATCH 1/4] dma-buf: add some more kerneldoc to dma_resv_add_shared_fence
2021-07-02 11:16 Start fixing the shared to exclusive fence dependencies Christian König
@ 2021-07-02 11:16 ` Christian König
2021-07-02 22:16 ` Daniel Vetter
2021-07-02 11:16 ` [PATCH 2/4] dma-buf: fix dma_resv_test_signaled test_all handling v2 Christian König
` (3 subsequent siblings)
4 siblings, 1 reply; 11+ messages in thread
From: Christian König @ 2021-07-02 11:16 UTC (permalink / raw)
To: dri-devel, daniel.vetter
Explicitly document that code can't assume that shared fences
signal after the exclusive fence.
Signed-off-by: Christian König <christian.koenig@amd.com>
---
drivers/dma-buf/dma-resv.c | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/drivers/dma-buf/dma-resv.c b/drivers/dma-buf/dma-resv.c
index f26c71747d43..4ab02b6c387a 100644
--- a/drivers/dma-buf/dma-resv.c
+++ b/drivers/dma-buf/dma-resv.c
@@ -235,7 +235,10 @@ EXPORT_SYMBOL(dma_resv_reset_shared_max);
* @fence: the shared fence to add
*
* Add a fence to a shared slot, obj->lock must be held, and
- * dma_resv_reserve_shared() has been called.
+ * dma_resv_reserve_shared() has been called. The shared fences can signal in
+ * any order and there is especially no guarantee that shared fences signal
+ * after the exclusive one. Code relying on any signaling order is broken and
+ * needs to be fixed.
*/
void dma_resv_add_shared_fence(struct dma_resv *obj, struct dma_fence *fence)
{
--
2.25.1
^ permalink raw reply related [flat|nested] 11+ messages in thread
* Re: [PATCH 1/4] dma-buf: add some more kerneldoc to dma_resv_add_shared_fence
2021-07-02 11:16 ` [PATCH 1/4] dma-buf: add some more kerneldoc to dma_resv_add_shared_fence Christian König
@ 2021-07-02 22:16 ` Daniel Vetter
0 siblings, 0 replies; 11+ messages in thread
From: Daniel Vetter @ 2021-07-02 22:16 UTC (permalink / raw)
To: Christian König; +Cc: daniel.vetter, dri-devel
On Fri, Jul 02, 2021 at 01:16:39PM +0200, Christian König wrote:
> Explicitly document that code can't assume that shared fences
> signal after the exclusive fence.
>
> Signed-off-by: Christian König <christian.koenig@amd.com>
> ---
> drivers/dma-buf/dma-resv.c | 5 ++++-
> 1 file changed, 4 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/dma-buf/dma-resv.c b/drivers/dma-buf/dma-resv.c
> index f26c71747d43..4ab02b6c387a 100644
> --- a/drivers/dma-buf/dma-resv.c
> +++ b/drivers/dma-buf/dma-resv.c
> @@ -235,7 +235,10 @@ EXPORT_SYMBOL(dma_resv_reset_shared_max);
> * @fence: the shared fence to add
> *
> * Add a fence to a shared slot, obj->lock must be held, and
> - * dma_resv_reserve_shared() has been called.
> + * dma_resv_reserve_shared() has been called. The shared fences can signal in
> + * any order and there is especially no guarantee that shared fences signal
> + * after the exclusive one. Code relying on any signaling order is broken and
> + * needs to be fixed.
This feels like the last place I'd go look for how I should handle
dependencies. It's the function for adding shared fences after all, has
absolutely nothing to do with whether we should wait for them.
I'll type up something else.
-Daniel
> */
> void dma_resv_add_shared_fence(struct dma_resv *obj, struct dma_fence *fence)
> {
> --
> 2.25.1
>
--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
^ permalink raw reply [flat|nested] 11+ messages in thread
* [PATCH 2/4] dma-buf: fix dma_resv_test_signaled test_all handling v2
2021-07-02 11:16 Start fixing the shared to exclusive fence dependencies Christian König
2021-07-02 11:16 ` [PATCH 1/4] dma-buf: add some more kerneldoc to dma_resv_add_shared_fence Christian König
@ 2021-07-02 11:16 ` Christian König
2021-07-02 22:33 ` Daniel Vetter
2021-07-02 11:16 ` [PATCH 3/4] drm/nouveau: always wait for the exclusive fence Christian König
` (2 subsequent siblings)
4 siblings, 1 reply; 11+ messages in thread
From: Christian König @ 2021-07-02 11:16 UTC (permalink / raw)
To: dri-devel, daniel.vetter
As the name implies if testing all fences is requested we
should indeed test all fences and not skip the exclusive
one because we see shared ones.
v2: fix logic once more
Signed-off-by: Christian König <christian.koenig@amd.com>
---
drivers/dma-buf/dma-resv.c | 33 ++++++++++++---------------------
1 file changed, 12 insertions(+), 21 deletions(-)
diff --git a/drivers/dma-buf/dma-resv.c b/drivers/dma-buf/dma-resv.c
index 4ab02b6c387a..18dd5a6ca06c 100644
--- a/drivers/dma-buf/dma-resv.c
+++ b/drivers/dma-buf/dma-resv.c
@@ -618,25 +618,21 @@ static inline int dma_resv_test_signaled_single(struct dma_fence *passed_fence)
*/
bool dma_resv_test_signaled(struct dma_resv *obj, bool test_all)
{
- unsigned int seq, shared_count;
+ struct dma_fence *fence;
+ unsigned int seq;
int ret;
rcu_read_lock();
retry:
ret = true;
- shared_count = 0;
seq = read_seqcount_begin(&obj->seq);
if (test_all) {
struct dma_resv_list *fobj = dma_resv_shared_list(obj);
- unsigned int i;
-
- if (fobj)
- shared_count = fobj->shared_count;
+ unsigned int i, shared_count;
+ shared_count = fobj ? fobj->shared_count : 0;
for (i = 0; i < shared_count; ++i) {
- struct dma_fence *fence;
-
fence = rcu_dereference(fobj->shared[i]);
ret = dma_resv_test_signaled_single(fence);
if (ret < 0)
@@ -644,24 +640,19 @@ bool dma_resv_test_signaled(struct dma_resv *obj, bool test_all)
else if (!ret)
break;
}
-
- if (read_seqcount_retry(&obj->seq, seq))
- goto retry;
}
- if (!shared_count) {
- struct dma_fence *fence_excl = dma_resv_excl_fence(obj);
-
- if (fence_excl) {
- ret = dma_resv_test_signaled_single(fence_excl);
- if (ret < 0)
- goto retry;
+ fence = dma_resv_excl_fence(obj);
+ if (ret && fence) {
+ ret = dma_resv_test_signaled_single(fence);
+ if (ret < 0)
+ goto retry;
- if (read_seqcount_retry(&obj->seq, seq))
- goto retry;
- }
}
+ if (read_seqcount_retry(&obj->seq, seq))
+ goto retry;
+
rcu_read_unlock();
return ret;
}
--
2.25.1
^ permalink raw reply related [flat|nested] 11+ messages in thread
* Re: [PATCH 2/4] dma-buf: fix dma_resv_test_signaled test_all handling v2
2021-07-02 11:16 ` [PATCH 2/4] dma-buf: fix dma_resv_test_signaled test_all handling v2 Christian König
@ 2021-07-02 22:33 ` Daniel Vetter
0 siblings, 0 replies; 11+ messages in thread
From: Daniel Vetter @ 2021-07-02 22:33 UTC (permalink / raw)
To: Christian König; +Cc: daniel.vetter, dri-devel
On Fri, Jul 02, 2021 at 01:16:40PM +0200, Christian König wrote:
> As the name implies if testing all fences is requested we
> should indeed test all fences and not skip the exclusive
> one because we see shared ones.
>
> v2: fix logic once more
>
> Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
> ---
> drivers/dma-buf/dma-resv.c | 33 ++++++++++++---------------------
> 1 file changed, 12 insertions(+), 21 deletions(-)
>
> diff --git a/drivers/dma-buf/dma-resv.c b/drivers/dma-buf/dma-resv.c
> index 4ab02b6c387a..18dd5a6ca06c 100644
> --- a/drivers/dma-buf/dma-resv.c
> +++ b/drivers/dma-buf/dma-resv.c
> @@ -618,25 +618,21 @@ static inline int dma_resv_test_signaled_single(struct dma_fence *passed_fence)
> */
> bool dma_resv_test_signaled(struct dma_resv *obj, bool test_all)
> {
> - unsigned int seq, shared_count;
> + struct dma_fence *fence;
> + unsigned int seq;
> int ret;
>
> rcu_read_lock();
> retry:
> ret = true;
> - shared_count = 0;
> seq = read_seqcount_begin(&obj->seq);
>
> if (test_all) {
> struct dma_resv_list *fobj = dma_resv_shared_list(obj);
> - unsigned int i;
> -
> - if (fobj)
> - shared_count = fobj->shared_count;
> + unsigned int i, shared_count;
>
> + shared_count = fobj ? fobj->shared_count : 0;
> for (i = 0; i < shared_count; ++i) {
> - struct dma_fence *fence;
> -
> fence = rcu_dereference(fobj->shared[i]);
> ret = dma_resv_test_signaled_single(fence);
> if (ret < 0)
> @@ -644,24 +640,19 @@ bool dma_resv_test_signaled(struct dma_resv *obj, bool test_all)
> else if (!ret)
> break;
> }
> -
> - if (read_seqcount_retry(&obj->seq, seq))
> - goto retry;
> }
>
> - if (!shared_count) {
> - struct dma_fence *fence_excl = dma_resv_excl_fence(obj);
> -
> - if (fence_excl) {
> - ret = dma_resv_test_signaled_single(fence_excl);
> - if (ret < 0)
> - goto retry;
> + fence = dma_resv_excl_fence(obj);
> + if (ret && fence) {
> + ret = dma_resv_test_signaled_single(fence);
> + if (ret < 0)
> + goto retry;
>
> - if (read_seqcount_retry(&obj->seq, seq))
> - goto retry;
> - }
> }
>
> + if (read_seqcount_retry(&obj->seq, seq))
> + goto retry;
> +
> rcu_read_unlock();
> return ret;
> }
> --
> 2.25.1
>
--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
^ permalink raw reply [flat|nested] 11+ messages in thread
* [PATCH 3/4] drm/nouveau: always wait for the exclusive fence
2021-07-02 11:16 Start fixing the shared to exclusive fence dependencies Christian König
2021-07-02 11:16 ` [PATCH 1/4] dma-buf: add some more kerneldoc to dma_resv_add_shared_fence Christian König
2021-07-02 11:16 ` [PATCH 2/4] dma-buf: fix dma_resv_test_signaled test_all handling v2 Christian König
@ 2021-07-02 11:16 ` Christian König
2021-07-02 23:00 ` Daniel Vetter
2021-07-02 11:16 ` [PATCH 4/4] drm/msm: " Christian König
2021-07-02 23:53 ` Start fixing the shared to exclusive fence dependencies Daniel Vetter
4 siblings, 1 reply; 11+ messages in thread
From: Christian König @ 2021-07-02 11:16 UTC (permalink / raw)
To: dri-devel, daniel.vetter
Drivers also need to to sync to the exclusive fence when
a shared one is present.
Signed-off-by: Christian König <christian.koenig@amd.com>
---
drivers/gpu/drm/nouveau/nouveau_fence.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/gpu/drm/nouveau/nouveau_fence.c b/drivers/gpu/drm/nouveau/nouveau_fence.c
index 6b43918035df..05d0b3eb3690 100644
--- a/drivers/gpu/drm/nouveau/nouveau_fence.c
+++ b/drivers/gpu/drm/nouveau/nouveau_fence.c
@@ -358,7 +358,7 @@ nouveau_fence_sync(struct nouveau_bo *nvbo, struct nouveau_channel *chan, bool e
fobj = dma_resv_shared_list(resv);
fence = dma_resv_excl_fence(resv);
- if (fence && (!exclusive || !fobj || !fobj->shared_count)) {
+ if (fence) {
struct nouveau_channel *prev = NULL;
bool must_wait = true;
--
2.25.1
^ permalink raw reply related [flat|nested] 11+ messages in thread
* Re: [PATCH 3/4] drm/nouveau: always wait for the exclusive fence
2021-07-02 11:16 ` [PATCH 3/4] drm/nouveau: always wait for the exclusive fence Christian König
@ 2021-07-02 23:00 ` Daniel Vetter
0 siblings, 0 replies; 11+ messages in thread
From: Daniel Vetter @ 2021-07-02 23:00 UTC (permalink / raw)
To: Christian König; +Cc: daniel.vetter, dri-devel
On Fri, Jul 02, 2021 at 01:16:41PM +0200, Christian König wrote:
> Drivers also need to to sync to the exclusive fence when
> a shared one is present.
>
> Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
> ---
> drivers/gpu/drm/nouveau/nouveau_fence.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/drivers/gpu/drm/nouveau/nouveau_fence.c b/drivers/gpu/drm/nouveau/nouveau_fence.c
> index 6b43918035df..05d0b3eb3690 100644
> --- a/drivers/gpu/drm/nouveau/nouveau_fence.c
> +++ b/drivers/gpu/drm/nouveau/nouveau_fence.c
> @@ -358,7 +358,7 @@ nouveau_fence_sync(struct nouveau_bo *nvbo, struct nouveau_channel *chan, bool e
> fobj = dma_resv_shared_list(resv);
> fence = dma_resv_excl_fence(resv);
>
> - if (fence && (!exclusive || !fobj || !fobj->shared_count)) {
> + if (fence) {
> struct nouveau_channel *prev = NULL;
> bool must_wait = true;
>
> --
> 2.25.1
>
--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
^ permalink raw reply [flat|nested] 11+ messages in thread
* [PATCH 4/4] drm/msm: always wait for the exclusive fence
2021-07-02 11:16 Start fixing the shared to exclusive fence dependencies Christian König
` (2 preceding siblings ...)
2021-07-02 11:16 ` [PATCH 3/4] drm/nouveau: always wait for the exclusive fence Christian König
@ 2021-07-02 11:16 ` Christian König
2021-07-02 23:01 ` Daniel Vetter
2021-07-02 23:53 ` Start fixing the shared to exclusive fence dependencies Daniel Vetter
4 siblings, 1 reply; 11+ messages in thread
From: Christian König @ 2021-07-02 11:16 UTC (permalink / raw)
To: dri-devel, daniel.vetter
Drivers also need to to sync to the exclusive fence when
a shared one is present.
Completely untested since the driver won't even compile on !ARM.
Signed-off-by: Christian König <christian.koenig@amd.com>
---
drivers/gpu/drm/msm/msm_gem.c | 16 +++++++---------
1 file changed, 7 insertions(+), 9 deletions(-)
diff --git a/drivers/gpu/drm/msm/msm_gem.c b/drivers/gpu/drm/msm/msm_gem.c
index a94a43de95ef..72a07e311de3 100644
--- a/drivers/gpu/drm/msm/msm_gem.c
+++ b/drivers/gpu/drm/msm/msm_gem.c
@@ -817,17 +817,15 @@ int msm_gem_sync_object(struct drm_gem_object *obj,
struct dma_fence *fence;
int i, ret;
- fobj = dma_resv_shared_list(obj->resv);
- if (!fobj || (fobj->shared_count == 0)) {
- fence = dma_resv_excl_fence(obj->resv);
- /* don't need to wait on our own fences, since ring is fifo */
- if (fence && (fence->context != fctx->context)) {
- ret = dma_fence_wait(fence, true);
- if (ret)
- return ret;
- }
+ fence = dma_resv_excl_fence(obj->resv);
+ /* don't need to wait on our own fences, since ring is fifo */
+ if (fence && (fence->context != fctx->context)) {
+ ret = dma_fence_wait(fence, true);
+ if (ret)
+ return ret;
}
+ fobj = dma_resv_shared_list(obj->resv);
if (!exclusive || !fobj)
return 0;
--
2.25.1
^ permalink raw reply related [flat|nested] 11+ messages in thread
* Re: [PATCH 4/4] drm/msm: always wait for the exclusive fence
2021-07-02 11:16 ` [PATCH 4/4] drm/msm: " Christian König
@ 2021-07-02 23:01 ` Daniel Vetter
2021-07-08 12:52 ` Christian König
0 siblings, 1 reply; 11+ messages in thread
From: Daniel Vetter @ 2021-07-02 23:01 UTC (permalink / raw)
To: Christian König; +Cc: daniel.vetter, dri-devel
On Fri, Jul 02, 2021 at 01:16:42PM +0200, Christian König wrote:
> Drivers also need to to sync to the exclusive fence when
> a shared one is present.
>
> Completely untested since the driver won't even compile on !ARM.
It's really not that hard to set up a cross-compiler, reasonable distros
have that now all packages. Does explain though why you tend to break the
arm build with drm-misc patches.
Please fix this.
> Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
> ---
> drivers/gpu/drm/msm/msm_gem.c | 16 +++++++---------
> 1 file changed, 7 insertions(+), 9 deletions(-)
>
> diff --git a/drivers/gpu/drm/msm/msm_gem.c b/drivers/gpu/drm/msm/msm_gem.c
> index a94a43de95ef..72a07e311de3 100644
> --- a/drivers/gpu/drm/msm/msm_gem.c
> +++ b/drivers/gpu/drm/msm/msm_gem.c
> @@ -817,17 +817,15 @@ int msm_gem_sync_object(struct drm_gem_object *obj,
> struct dma_fence *fence;
> int i, ret;
>
> - fobj = dma_resv_shared_list(obj->resv);
> - if (!fobj || (fobj->shared_count == 0)) {
> - fence = dma_resv_excl_fence(obj->resv);
> - /* don't need to wait on our own fences, since ring is fifo */
> - if (fence && (fence->context != fctx->context)) {
> - ret = dma_fence_wait(fence, true);
> - if (ret)
> - return ret;
> - }
> + fence = dma_resv_excl_fence(obj->resv);
> + /* don't need to wait on our own fences, since ring is fifo */
> + if (fence && (fence->context != fctx->context)) {
> + ret = dma_fence_wait(fence, true);
> + if (ret)
> + return ret;
> }
>
> + fobj = dma_resv_shared_list(obj->resv);
> if (!exclusive || !fobj)
> return 0;
>
> --
> 2.25.1
>
--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH 4/4] drm/msm: always wait for the exclusive fence
2021-07-02 23:01 ` Daniel Vetter
@ 2021-07-08 12:52 ` Christian König
0 siblings, 0 replies; 11+ messages in thread
From: Christian König @ 2021-07-08 12:52 UTC (permalink / raw)
To: Daniel Vetter; +Cc: daniel.vetter, dri-devel
Am 03.07.21 um 01:01 schrieb Daniel Vetter:
> On Fri, Jul 02, 2021 at 01:16:42PM +0200, Christian König wrote:
>> Drivers also need to to sync to the exclusive fence when
>> a shared one is present.
>>
>> Completely untested since the driver won't even compile on !ARM.
> It's really not that hard to set up a cross-compiler, reasonable distros
> have that now all packages. Does explain though why you tend to break the
> arm build with drm-misc patches.
Well having proper COMPILE_TEST handling in kconfig would be even better.
Otherwise everybody needs to cross-compile for ARM, ARM64 (with all the
variants, e.g. BCM, S3C64XX, S5PV210, KEEMBAY, ZYNQMP etc etc), MIPS and
so on.
We have tons of non-x86 drivers, but MSM is the only one which is
painful to get to compile test.
Christian.
>
> Please fix this.
>
>> Signed-off-by: Christian König <christian.koenig@amd.com>
> Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
>> ---
>> drivers/gpu/drm/msm/msm_gem.c | 16 +++++++---------
>> 1 file changed, 7 insertions(+), 9 deletions(-)
>>
>> diff --git a/drivers/gpu/drm/msm/msm_gem.c b/drivers/gpu/drm/msm/msm_gem.c
>> index a94a43de95ef..72a07e311de3 100644
>> --- a/drivers/gpu/drm/msm/msm_gem.c
>> +++ b/drivers/gpu/drm/msm/msm_gem.c
>> @@ -817,17 +817,15 @@ int msm_gem_sync_object(struct drm_gem_object *obj,
>> struct dma_fence *fence;
>> int i, ret;
>>
>> - fobj = dma_resv_shared_list(obj->resv);
>> - if (!fobj || (fobj->shared_count == 0)) {
>> - fence = dma_resv_excl_fence(obj->resv);
>> - /* don't need to wait on our own fences, since ring is fifo */
>> - if (fence && (fence->context != fctx->context)) {
>> - ret = dma_fence_wait(fence, true);
>> - if (ret)
>> - return ret;
>> - }
>> + fence = dma_resv_excl_fence(obj->resv);
>> + /* don't need to wait on our own fences, since ring is fifo */
>> + if (fence && (fence->context != fctx->context)) {
>> + ret = dma_fence_wait(fence, true);
>> + if (ret)
>> + return ret;
>> }
>>
>> + fobj = dma_resv_shared_list(obj->resv);
>> if (!exclusive || !fobj)
>> return 0;
>>
>> --
>> 2.25.1
>>
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Start fixing the shared to exclusive fence dependencies.
2021-07-02 11:16 Start fixing the shared to exclusive fence dependencies Christian König
` (3 preceding siblings ...)
2021-07-02 11:16 ` [PATCH 4/4] drm/msm: " Christian König
@ 2021-07-02 23:53 ` Daniel Vetter
4 siblings, 0 replies; 11+ messages in thread
From: Daniel Vetter @ 2021-07-02 23:53 UTC (permalink / raw)
To: Christian König; +Cc: daniel.vetter, dri-devel
On Fri, Jul 02, 2021 at 01:16:38PM +0200, Christian König wrote:
> Hey Daniel,
>
> even when you are not 100% done with the driver audit I think we should
> push that patch set here to drm-misc-next now so that it can end up in
> 5.15.
So I think I got them all, just need to type up some good docs all over
the place next week and send it out.
-Daniel
>
> Not having any dependency between the exclusive and the shared fence
> signaling order is just way more defensive than the current model.
>
> As discussed I'm holding back any amdgpu and TTM workarounds which could
> be removed for now.
>
> Thoughts?
>
> Thanks,
> Christian.
>
>
--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
^ permalink raw reply [flat|nested] 11+ messages in thread