Re: [Intel-gfx] [PATCH 01/26] dma-buf: add dma_resv_for_each_fence_unlocked

["Christian König" <ckoenig.leichtzumerken@gmail.com>]

Am 14.09.21 um 15:07 schrieb Tvrtko Ursulin:
> On 14/09/2021 12:25, Christian König wrote:
>> Am 14.09.21 um 12:53 schrieb Tvrtko Ursulin:
>>>
>>> On 13/09/2021 14:16, Christian König wrote:
>>>> [SNIP]
>>>> +        if (fence) {
>>>> +            fence = dma_fence_get_rcu(fence);
>>>> +        } else if (all_fences && cursor->fences) {
>>>> +            struct dma_resv_list *fences = cursor->fences;
>>>
>>> If rcu lock is allowed to be dropped while walking the list what 
>>> guarantees list of fences hasn't been freed?
>>
>> Ah, good point! We need to test the sequence number when we enter the 
>> function as well. Going to fix that.
>
> Right, but just to say, I am still on the fence a bit on the concept 
> of the unlocked iterator. So for now I am looking only superficially 
> at the implementation and i915 side of things.

I'm really in favor of taking the lock as well and contain the unlocked 
operation into the dma_resv object code and I think Daniel is on 
absolutely the same side as well.

But the use cases are as they are for now and I think containing the 
internal structure of the dma_resv object is the right next step.

>>> [SNIP]
>>>>   +/**
>>>> + * struct dma_resv_cursor - current position into the dma_resv fences
>>>> + * @seq: sequence number to check
>>>> + * @index: index into the shared fences
>>>> + * @shared: the shared fences
>>>> + * @is_first: true if this is the first returned fence
>>>> + * @is_exclusive: if the current fence is the exclusive one
>>>> + */
>>>> +struct dma_resv_cursor {
>>>> +    unsigned int seq;
>>>> +    unsigned int index;
>>>> +    struct dma_resv_list *fences;
>>>> +    bool is_first;
>>>
>>> Is_first is useful to callers - like they are legitimately allowed 
>>> to look inside this, what could otherwise be private object?
>>
>> Yes, I was pondering on the same question. Key point is that this is 
>> only used by other dma_resv functions which also use cursor.fences 
>> for example.
>>
>> So this is only supposed to be used by code working with other 
>> privates of the dma_resv object as well.
>
> Hmmm and you think external callers have no legitimate case of 
> detecting restarts?

Yes, if somebody needs a snapshot of the current state and can't for 
some reason take the lock they should use dma_resv_get_fences() instead.

On the other hand allocating memory in dma_resv_get_fences() has 
probably more overhead than just grabbing and releasing the lock.

> Or to better say will not have the need of distinguishing between real 
> restarts and just the first iteration? I need to read more of the 
> series to get a more complete opinion here.

Yeah, that's indeed a good point. Off hand I don't see any, but we 
should probably decide for each place individually if we should take the 
lock, allocate memory or use the lockless iterator.

> [SNIP]
>>>
>>>> +};
>>>> +
>>>> +/**
>>>> + * dma_resv_for_each_fence_unlocked - fence iterator
>>>> + * @obj: a dma_resv object pointer
>>>> + * @cursor: a struct dma_resv_cursor pointer
>>>> + * @all_fences: true if all fences should be returned
>>>> + * @fence: the current fence
>>>> + *
>>>> + * Iterate over the fences in a struct dma_resv object without 
>>>> holding the
>>>> + * dma_resv::lock. The RCU read side lock must be hold when using 
>>>> this, but can
>>>> + * be dropped and re-taken as necessary inside the loop. 
>>>> @all_fences controls
>>>> + * if the shared fences are returned as well.
>>>> + */
>>>> +#define dma_resv_for_each_fence_unlocked(obj, cursor, all_fences, 
>>>> fence)    \
>>>> +    for (fence = dma_resv_walk_unlocked(obj, cursor, all_fences, 
>>>> true); \
>>>> +         fence; dma_fence_put(fence),                    \
>>>> +         fence = dma_resv_walk_unlocked(obj, cursor, all_fences, 
>>>> false))
>>>
>>> Has the fact RCU lock can be dropped so there is potential to walk 
>>> over completely different snapshots been discussed?
>>
>> Well that's basically the heart of the functionality. Even without 
>> dropping the RCU lock there can be an restart at any time when the 
>> dma_resv object is modified.
>
> Hm yes.. that's one of the thing which makes me undecided yet whether 
> a generalised helper is desirable. For example i915_gem_busy_ioctl, as 
> converted, is not completely like-for-like. Maybe it is irrelevant for 
> that one, but then the question needs to be answered for all of the 
> replacements.
>
>>
>>> At least if I followed the code correctly - it appears there is 
>>> potential the walk restarts from the start (exclusive slot) at any 
>>> point during the walk.
>>
>> Correct, yes.
>>
>>> Because theoretically I think you could take an atomic snapshot of 
>>> everything (given you have a cursor object) and then release it on 
>>> the end condition.
>>
>> That's what the dma_resv_get_fences() function is good for, yes. This 
>> one returns an array of fences.
>>
>> The key difference is that we need to allocate memory for that which 
>> is at least sometimes not feasible or desired.
>
> Ah true.. dma_resv_list is not reference counted to simply grab it 
> during setup.
>
>> Thanks for the review,
>
> Np, it is intriguing to look at the option of code consolidation. Just 
> need to read more of the series to form a better high level opinion.

Really appreciated, thanks for looking into this.

Regards,
Christian.

>
> Regards,
>
> Tvrtko
>
>> Christian.
>>
>>>
>>> Regards,
>>>
>>> Tvrtko
>>>
>>>> +
>>>>   #define dma_resv_held(obj) lockdep_is_held(&(obj)->lock.base)
>>>>   #define dma_resv_assert_held(obj) 
>>>> lockdep_assert_held(&(obj)->lock.base)
>>>>   @@ -366,6 +399,9 @@ void dma_resv_fini(struct dma_resv *obj);
>>>>   int dma_resv_reserve_shared(struct dma_resv *obj, unsigned int 
>>>> num_fences);
>>>>   void dma_resv_add_shared_fence(struct dma_resv *obj, struct 
>>>> dma_fence *fence);
>>>>   void dma_resv_add_excl_fence(struct dma_resv *obj, struct 
>>>> dma_fence *fence);
>>>> +struct dma_fence *dma_resv_walk_unlocked(struct dma_resv *obj,
>>>> +                     struct dma_resv_cursor *cursor,
>>>> +                     bool first, bool all_fences);
>>>>   int dma_resv_get_fences(struct dma_resv *obj, struct dma_fence 
>>>> **pfence_excl,
>>>>               unsigned *pshared_count, struct dma_fence ***pshared);
>>>>   int dma_resv_copy_fences(struct dma_resv *dst, struct dma_resv 
>>>> *src);
>>>>
>>