[PATCH v2 1/3] drm/syncobj: call drm_syncobj_fence_add_wait when WAIT_AVAILABLE flag is set

[PATCH v2 1/3] drm/syncobj: call drm_syncobj_fence_add_wait when WAIT_AVAILABLE flag is set

[Erik Kurzinger]

When waiting for a syncobj timeline point whose fence has not yet been
submitted with the WAIT_FOR_SUBMIT flag, a callback is registered using
drm_syncobj_fence_add_wait and the thread is put to sleep until the
timeout expires. If the fence is submitted before then,
drm_syncobj_add_point will wake up the sleeping thread immediately which
will proceed to wait for the fence to be signaled.

However, if the WAIT_AVAILABLE flag is used instead,
drm_syncobj_fence_add_wait won't get called, meaning the waiting thread
will always sleep for the full timeout duration, even if the fence gets
submitted earlier. If it turns out that the fence *has* been submitted
by the time it eventually wakes up, it will still indicate to userspace
that the wait completed successfully (it won't return -ETIME), but it
will have taken much longer than it should have.

To fix this, we must call drm_syncobj_fence_add_wait if *either* the
WAIT_FOR_SUBMIT flag or the WAIT_AVAILABLE flag is set. The only
difference being that with WAIT_FOR_SUBMIT we will also wait for the
fence to be signaled after it has been submitted while with
WAIT_AVAILABLE we will return immediately.

IGT test patch: https://lists.freedesktop.org/archives/igt-dev/2024-January/067537.html

v1 -> v2: adjust lockdep_assert_none_held_once condition

Fixes: 01d6c3578379 ("drm/syncobj: add support for timeline point wait v8")
Signed-off-by: Erik Kurzinger <ekurzinger@nvidia.com>
---
 drivers/gpu/drm/drm_syncobj.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/drivers/gpu/drm/drm_syncobj.c b/drivers/gpu/drm/drm_syncobj.c
index 94ebc71e5be5..97be8b140599 100644
--- a/drivers/gpu/drm/drm_syncobj.c
+++ b/drivers/gpu/drm/drm_syncobj.c
@@ -1058,7 +1058,8 @@ static signed long drm_syncobj_array_wait_timeout(struct drm_syncobj **syncobjs,
 	uint64_t *points;
 	uint32_t signaled_count, i;
 
-	if (flags & DRM_SYNCOBJ_WAIT_FLAGS_WAIT_FOR_SUBMIT)
+	if (flags & (DRM_SYNCOBJ_WAIT_FLAGS_WAIT_FOR_SUBMIT |
+		     DRM_SYNCOBJ_WAIT_FLAGS_WAIT_AVAILABLE))
 		lockdep_assert_none_held_once();
 
 	points = kmalloc_array(count, sizeof(*points), GFP_KERNEL);
@@ -1127,7 +1128,8 @@ static signed long drm_syncobj_array_wait_timeout(struct drm_syncobj **syncobjs,
 	 * fallthough and try a 0 timeout wait!
 	 */
 
-	if (flags & DRM_SYNCOBJ_WAIT_FLAGS_WAIT_FOR_SUBMIT) {
+	if (flags & (DRM_SYNCOBJ_WAIT_FLAGS_WAIT_FOR_SUBMIT |
+		     DRM_SYNCOBJ_WAIT_FLAGS_WAIT_AVAILABLE)) {
 		for (i = 0; i < count; ++i)
 			drm_syncobj_fence_add_wait(syncobjs[i], &entries[i]);
 	}
-- 
2.43.0

[PATCH v2 2/3] drm/syncobj: reject invalid flags in drm_syncobj_find_fence

[Erik Kurzinger]

The only flag that is meaningful to drm_syncobj_find_fence is
DRM_SYNCOBJ_WAIT_FLAGS_WAIT_FOR_SUBMIT. It should return -EINVAL for any
other flag bits.

Signed-off-by: Erik Kurzinger <ekurzinger@nvidia.com>
---
 drivers/gpu/drm/drm_syncobj.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/drivers/gpu/drm/drm_syncobj.c b/drivers/gpu/drm/drm_syncobj.c
index 97be8b140599..c59bb02e2c07 100644
--- a/drivers/gpu/drm/drm_syncobj.c
+++ b/drivers/gpu/drm/drm_syncobj.c
@@ -448,6 +448,9 @@ int drm_syncobj_find_fence(struct drm_file *file_private,
 	u64 timeout = nsecs_to_jiffies64(DRM_SYNCOBJ_WAIT_FOR_SUBMIT_TIMEOUT);
 	int ret;
 
+	if (flags & ~DRM_SYNCOBJ_WAIT_FLAGS_WAIT_FOR_SUBMIT)
+		return -EINVAL;
+
 	if (!syncobj)
 		return -ENOENT;
 
-- 
2.43.0

[PATCH v2 3/3] drm/syncobj: call might_sleep before waiting for fence submission

[Erik Kurzinger]

If either the DRM_SYNCOBJ_WAIT_FLAGS_WAIT_FOR_SUBMIT or
DRM_SYNCOBJ_WAIT_FLAGS_WAIT_AVAILABLE flags are passed to
drm_syncobj_array_wait_timeout, the function might sleep if the fence at
one of the given timeline points has not yet been submitted. Therefore,
we should call might_sleep in that case to catch potential bugs.

Signed-off-by: Erik Kurzinger <ekurzinger@nvidia.com>
---
 drivers/gpu/drm/drm_syncobj.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/drivers/gpu/drm/drm_syncobj.c b/drivers/gpu/drm/drm_syncobj.c
index c59bb02e2c07..e04965878a08 100644
--- a/drivers/gpu/drm/drm_syncobj.c
+++ b/drivers/gpu/drm/drm_syncobj.c
@@ -1062,8 +1062,10 @@ static signed long drm_syncobj_array_wait_timeout(struct drm_syncobj **syncobjs,
 	uint32_t signaled_count, i;
 
 	if (flags & (DRM_SYNCOBJ_WAIT_FLAGS_WAIT_FOR_SUBMIT |
-		     DRM_SYNCOBJ_WAIT_FLAGS_WAIT_AVAILABLE))
+		     DRM_SYNCOBJ_WAIT_FLAGS_WAIT_AVAILABLE)) {
+		might_sleep();
 		lockdep_assert_none_held_once();
+	}
 
 	points = kmalloc_array(count, sizeof(*points), GFP_KERNEL);
 	if (points == NULL)
-- 
2.43.0

Re: [PATCH v2 1/3] drm/syncobj: call drm_syncobj_fence_add_wait when WAIT_AVAILABLE flag is set

[Daniel Vetter]

On Fri, Jan 19, 2024 at 08:32:06AM -0800, Erik Kurzinger wrote:
> When waiting for a syncobj timeline point whose fence has not yet been
> submitted with the WAIT_FOR_SUBMIT flag, a callback is registered using
> drm_syncobj_fence_add_wait and the thread is put to sleep until the
> timeout expires. If the fence is submitted before then,
> drm_syncobj_add_point will wake up the sleeping thread immediately which
> will proceed to wait for the fence to be signaled.
> 
> However, if the WAIT_AVAILABLE flag is used instead,
> drm_syncobj_fence_add_wait won't get called, meaning the waiting thread
> will always sleep for the full timeout duration, even if the fence gets
> submitted earlier. If it turns out that the fence *has* been submitted
> by the time it eventually wakes up, it will still indicate to userspace
> that the wait completed successfully (it won't return -ETIME), but it
> will have taken much longer than it should have.
> 
> To fix this, we must call drm_syncobj_fence_add_wait if *either* the
> WAIT_FOR_SUBMIT flag or the WAIT_AVAILABLE flag is set. The only
> difference being that with WAIT_FOR_SUBMIT we will also wait for the
> fence to be signaled after it has been submitted while with
> WAIT_AVAILABLE we will return immediately.
> 
> IGT test patch: https://lists.freedesktop.org/archives/igt-dev/2024-January/067537.html
> 
> v1 -> v2: adjust lockdep_assert_none_held_once condition
> 
> Fixes: 01d6c3578379 ("drm/syncobj: add support for timeline point wait v8")
> Signed-off-by: Erik Kurzinger <ekurzinger@nvidia.com>

Yeah I think this series catches now all the corner cases I spotted in v1.
On the series:

Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
> ---
>  drivers/gpu/drm/drm_syncobj.c | 6 ++++--
>  1 file changed, 4 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/gpu/drm/drm_syncobj.c b/drivers/gpu/drm/drm_syncobj.c
> index 94ebc71e5be5..97be8b140599 100644
> --- a/drivers/gpu/drm/drm_syncobj.c
> +++ b/drivers/gpu/drm/drm_syncobj.c
> @@ -1058,7 +1058,8 @@ static signed long drm_syncobj_array_wait_timeout(struct drm_syncobj **syncobjs,
>  	uint64_t *points;
>  	uint32_t signaled_count, i;
>  
> -	if (flags & DRM_SYNCOBJ_WAIT_FLAGS_WAIT_FOR_SUBMIT)
> +	if (flags & (DRM_SYNCOBJ_WAIT_FLAGS_WAIT_FOR_SUBMIT |
> +		     DRM_SYNCOBJ_WAIT_FLAGS_WAIT_AVAILABLE))
>  		lockdep_assert_none_held_once();
>  
>  	points = kmalloc_array(count, sizeof(*points), GFP_KERNEL);
> @@ -1127,7 +1128,8 @@ static signed long drm_syncobj_array_wait_timeout(struct drm_syncobj **syncobjs,
>  	 * fallthough and try a 0 timeout wait!
>  	 */
>  
> -	if (flags & DRM_SYNCOBJ_WAIT_FLAGS_WAIT_FOR_SUBMIT) {
> +	if (flags & (DRM_SYNCOBJ_WAIT_FLAGS_WAIT_FOR_SUBMIT |
> +		     DRM_SYNCOBJ_WAIT_FLAGS_WAIT_AVAILABLE)) {
>  		for (i = 0; i < count; ++i)
>  			drm_syncobj_fence_add_wait(syncobjs[i], &entries[i]);
>  	}
> -- 
> 2.43.0
> 

-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch

Re: [PATCH v2 1/3] drm/syncobj: call drm_syncobj_fence_add_wait when WAIT_AVAILABLE flag is set

[Erik Kurzinger]

It looks these these patches have still not been merged after one month, is there anything more that needs to be done for this to happen?

On 1/25/24 10:12, Daniel Vetter wrote:
> On Fri, Jan 19, 2024 at 08:32:06AM -0800, Erik Kurzinger wrote:
>> When waiting for a syncobj timeline point whose fence has not yet been
>> submitted with the WAIT_FOR_SUBMIT flag, a callback is registered using
>> drm_syncobj_fence_add_wait and the thread is put to sleep until the
>> timeout expires. If the fence is submitted before then,
>> drm_syncobj_add_point will wake up the sleeping thread immediately which
>> will proceed to wait for the fence to be signaled.
>>
>> However, if the WAIT_AVAILABLE flag is used instead,
>> drm_syncobj_fence_add_wait won't get called, meaning the waiting thread
>> will always sleep for the full timeout duration, even if the fence gets
>> submitted earlier. If it turns out that the fence *has* been submitted
>> by the time it eventually wakes up, it will still indicate to userspace
>> that the wait completed successfully (it won't return -ETIME), but it
>> will have taken much longer than it should have.
>>
>> To fix this, we must call drm_syncobj_fence_add_wait if *either* the
>> WAIT_FOR_SUBMIT flag or the WAIT_AVAILABLE flag is set. The only
>> difference being that with WAIT_FOR_SUBMIT we will also wait for the
>> fence to be signaled after it has been submitted while with
>> WAIT_AVAILABLE we will return immediately.
>>
>> IGT test patch: https://lists.freedesktop.org/archives/igt-dev/2024-January/067537.html
>>
>> v1 -> v2: adjust lockdep_assert_none_held_once condition
>>
>> Fixes: 01d6c3578379 ("drm/syncobj: add support for timeline point wait v8")
>> Signed-off-by: Erik Kurzinger <ekurzinger@nvidia.com>
> 
> Yeah I think this series catches now all the corner cases I spotted in v1.
> On the series:
> 
> Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
>> ---
>>  drivers/gpu/drm/drm_syncobj.c | 6 ++++--
>>  1 file changed, 4 insertions(+), 2 deletions(-)
>>
>> diff --git a/drivers/gpu/drm/drm_syncobj.c b/drivers/gpu/drm/drm_syncobj.c
>> index 94ebc71e5be5..97be8b140599 100644
>> --- a/drivers/gpu/drm/drm_syncobj.c
>> +++ b/drivers/gpu/drm/drm_syncobj.c
>> @@ -1058,7 +1058,8 @@ static signed long drm_syncobj_array_wait_timeout(struct drm_syncobj **syncobjs,
>>  	uint64_t *points;
>>  	uint32_t signaled_count, i;
>>  
>> -	if (flags & DRM_SYNCOBJ_WAIT_FLAGS_WAIT_FOR_SUBMIT)
>> +	if (flags & (DRM_SYNCOBJ_WAIT_FLAGS_WAIT_FOR_SUBMIT |
>> +		     DRM_SYNCOBJ_WAIT_FLAGS_WAIT_AVAILABLE))
>>  		lockdep_assert_none_held_once();
>>  
>>  	points = kmalloc_array(count, sizeof(*points), GFP_KERNEL);
>> @@ -1127,7 +1128,8 @@ static signed long drm_syncobj_array_wait_timeout(struct drm_syncobj **syncobjs,
>>  	 * fallthough and try a 0 timeout wait!
>>  	 */
>>  
>> -	if (flags & DRM_SYNCOBJ_WAIT_FLAGS_WAIT_FOR_SUBMIT) {
>> +	if (flags & (DRM_SYNCOBJ_WAIT_FLAGS_WAIT_FOR_SUBMIT |
>> +		     DRM_SYNCOBJ_WAIT_FLAGS_WAIT_AVAILABLE)) {
>>  		for (i = 0; i < count; ++i)
>>  			drm_syncobj_fence_add_wait(syncobjs[i], &entries[i]);
>>  	}
>> -- 
>> 2.43.0
>>
> 

Re: [PATCH v2 1/3] drm/syncobj: call drm_syncobj_fence_add_wait when WAIT_AVAILABLE flag is set

[Dave Airlie]

Can someone pick these up into misc?

Dave.

On Thu, 22 Feb 2024 at 04:48, Erik Kurzinger <ekurzinger@nvidia.com> wrote:
>
> It looks these these patches have still not been merged after one month, is there anything more that needs to be done for this to happen?
>
> On 1/25/24 10:12, Daniel Vetter wrote:
> > On Fri, Jan 19, 2024 at 08:32:06AM -0800, Erik Kurzinger wrote:
> >> When waiting for a syncobj timeline point whose fence has not yet been
> >> submitted with the WAIT_FOR_SUBMIT flag, a callback is registered using
> >> drm_syncobj_fence_add_wait and the thread is put to sleep until the
> >> timeout expires. If the fence is submitted before then,
> >> drm_syncobj_add_point will wake up the sleeping thread immediately which
> >> will proceed to wait for the fence to be signaled.
> >>
> >> However, if the WAIT_AVAILABLE flag is used instead,
> >> drm_syncobj_fence_add_wait won't get called, meaning the waiting thread
> >> will always sleep for the full timeout duration, even if the fence gets
> >> submitted earlier. If it turns out that the fence *has* been submitted
> >> by the time it eventually wakes up, it will still indicate to userspace
> >> that the wait completed successfully (it won't return -ETIME), but it
> >> will have taken much longer than it should have.
> >>
> >> To fix this, we must call drm_syncobj_fence_add_wait if *either* the
> >> WAIT_FOR_SUBMIT flag or the WAIT_AVAILABLE flag is set. The only
> >> difference being that with WAIT_FOR_SUBMIT we will also wait for the
> >> fence to be signaled after it has been submitted while with
> >> WAIT_AVAILABLE we will return immediately.
> >>
> >> IGT test patch: https://lists.freedesktop.org/archives/igt-dev/2024-January/067537.html
> >>
> >> v1 -> v2: adjust lockdep_assert_none_held_once condition
> >>
> >> Fixes: 01d6c3578379 ("drm/syncobj: add support for timeline point wait v8")
> >> Signed-off-by: Erik Kurzinger <ekurzinger@nvidia.com>
> >
> > Yeah I think this series catches now all the corner cases I spotted in v1.
> > On the series:
> >
> > Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
> >> ---
> >>  drivers/gpu/drm/drm_syncobj.c | 6 ++++--
> >>  1 file changed, 4 insertions(+), 2 deletions(-)
> >>
> >> diff --git a/drivers/gpu/drm/drm_syncobj.c b/drivers/gpu/drm/drm_syncobj.c
> >> index 94ebc71e5be5..97be8b140599 100644
> >> --- a/drivers/gpu/drm/drm_syncobj.c
> >> +++ b/drivers/gpu/drm/drm_syncobj.c
> >> @@ -1058,7 +1058,8 @@ static signed long drm_syncobj_array_wait_timeout(struct drm_syncobj **syncobjs,
> >>      uint64_t *points;
> >>      uint32_t signaled_count, i;
> >>
> >> -    if (flags & DRM_SYNCOBJ_WAIT_FLAGS_WAIT_FOR_SUBMIT)
> >> +    if (flags & (DRM_SYNCOBJ_WAIT_FLAGS_WAIT_FOR_SUBMIT |
> >> +                 DRM_SYNCOBJ_WAIT_FLAGS_WAIT_AVAILABLE))
> >>              lockdep_assert_none_held_once();
> >>
> >>      points = kmalloc_array(count, sizeof(*points), GFP_KERNEL);
> >> @@ -1127,7 +1128,8 @@ static signed long drm_syncobj_array_wait_timeout(struct drm_syncobj **syncobjs,
> >>       * fallthough and try a 0 timeout wait!
> >>       */
> >>
> >> -    if (flags & DRM_SYNCOBJ_WAIT_FLAGS_WAIT_FOR_SUBMIT) {
> >> +    if (flags & (DRM_SYNCOBJ_WAIT_FLAGS_WAIT_FOR_SUBMIT |
> >> +                 DRM_SYNCOBJ_WAIT_FLAGS_WAIT_AVAILABLE)) {
> >>              for (i = 0; i < count; ++i)
> >>                      drm_syncobj_fence_add_wait(syncobjs[i], &entries[i]);
> >>      }
> >> --
> >> 2.43.0
> >>
> >
>

Re: [PATCH v2 1/3] drm/syncobj: call drm_syncobj_fence_add_wait when WAIT_AVAILABLE flag is set

[Simon Ser]

Sorry for the delay. The first patch is also Reviewed-by me, the rest
is Acked-by because I only skimmed through them.

Thanks, pushed to drm-misc-next! Please do ping about patches when they
slip through the cracks.

Re: [PATCH v2 1/3] drm/syncobj: call drm_syncobj_fence_add_wait when WAIT_AVAILABLE flag is set

[Simon Ser]

On Thursday, February 22nd, 2024 at 11:32, Simon Ser <contact@emersion.fr> wrote:

> Thanks, pushed to drm-misc-next!

As I write this, I realize I should've pushed the first patch to
drm-misc-fixes instead… Sorry about the fuss…

Sima, Dave, what is the right thing to do here? Push a duplicate commit
to drm-misc-fixes? (I know that's not a great thing to do…)

Re: [PATCH v2 1/3] drm/syncobj: call drm_syncobj_fence_add_wait when WAIT_AVAILABLE flag is set

[Simon Ser]

On Thursday, February 22nd, 2024 at 11:34, Simon Ser <contact@emersion.fr> wrote:

> On Thursday, February 22nd, 2024 at 11:32, Simon Ser contact@emersion.fr wrote:
> 
> > Thanks, pushed to drm-misc-next!
> 
> As I write this, I realize I should've pushed the first patch to
> drm-misc-fixes instead… Sorry about the fuss…
> 
> Sima, Dave, what is the right thing to do here? Push a duplicate commit
> to drm-misc-fixes? (I know that's not a great thing to do…)

I've cherry-picked the fix to drm-misc-fixes with Maxime's ACK.