Re: [RFC] Host1x/TegraDRM UAPI (sync points)

[Mikko Perttunen <cyndis@kapsi.fi>]

On 7/7/20 2:06 PM, Dmitry Osipenko wrote:
> 02.07.2020 15:10, Mikko Perttunen пишет:
>> Ok, so we would have two kinds of syncpoints for the job; one
>> for kernel job tracking; and one that userspace can
>> manipulate as it wants to.
>>
>> Could we handle the job tracking syncpoint completely inside the kernel,
>> i.e. allocate it in kernel during job submission, and add an increment
>> for it at the end of the job (with condition OP_DONE)? For MLOCKing, the
>> kernel already needs to insert a SYNCPT_INCR(OP_DONE) + WAIT +
>> MLOCK_RELEASE sequence at the end of each job.
> 
> If sync point is allocated within kernel, then we'll need to always
> patch all job's sync point increments with the ID of the allocated sync
> point, regardless of whether firewall enabled or not.

The idea was that the job tracking increment would also be added to the 
pushbuffer in the kernel, so gathers would only have increments for the 
"user syncpoints", if any. I think this should work for THI-based 
engines (e.g. VIC), you probably have better information about 
GR2D/GR3D. On newer Tegras we could use CHANNEL/APPID protection to 
prevent the gathers from incrementing these job tracking syncpoints.

> 
> Secondly, I'm now recalling that only one sync point could be assigned
> to a channel at a time on newer Tegras which support sync point
> protection. So it sounds like we don't really have variants other than
> to allocate one sync point per channel for the jobs usage if we want to
> be able to push multiple jobs into channel's pushbuffer, correct?
> 

The other way around; each syncpoint can be assigned to one channel. One 
channel may have multiple syncpoints.

> ...
>>> Hmm, we actually should be able to have a one sync point per-channel for
>>> the job submission, similarly to what the current driver does!
>>>
>>> I'm keep forgetting about the waitbase existence!
>>
>> Tegra194 doesn't have waitbases, but if we are resubmitting all the jobs
>> anyway, can't we just recalculate wait thresholds at that time?
> 
> Yes, thresholds could be recalculated + job should be re-formed at the
> push-time then.
> 
> It also means that jobs always should be formed only at the push-time if
> wait-command is utilized by cmdstream since the waits always need to be
> patched because we won't know the thresholds until scheduler actually
> runs the job.

Could we keep the job tracking syncpoints entirely within the kernel, 
and have all wait commands and other stuff that userspace does use the 
user syncpoints? Then kernel job tracking and userspace activity would 
be separate from each other.

Alternatively, if we know that jobs can only be removed from the middle 
of pushbuffers, and not added, we could replace any removed jobs with 
syncpoint increments in the pushbuffer and any thresholds would stay intact.

> 
>> Maybe a more detailed sequence list or diagram of what happens during
>> submission and recovery would be useful.
> 
> The textual form + code is already good enough to me. A diagram could be
> nice to have, although it may take a bit too much effort to create +
> maintain it. But I don't mind at all if you'd want to make one :)
> 
> ...
>>>> * We should be able to keep the syncpoint refcounting based on fences.
>>>
>>> The fence doesn't need the sync point itself, it only needs to get a
>>> signal when the threshold is reached or when sync point is ceased.
>>>
>>> Imagine:
>>>
>>>     - Process A creates sync point
>>>     - Process A creates dma-fence from this sync point
>>>     - Process A exports dma-fence to process B
>>>     - Process A dies
>>>
>>> What should happen to process B?
>>>
>>>     - Should dma-fence of the process B get a error signal when process A
>>> dies?
>>>     - Should process B get stuck waiting endlessly for the dma-fence?
>>>
>>> This is one example of why I'm proposing that fence shouldn't be coupled
>>> tightly to a sync point.
>>
>> As a baseline, we should consider process B to get stuck endlessly
>> (until a timeout of its choosing) for the fence. In this case it is
>> avoidable, but if the ID/threshold pair is exported out of the fence and
>> is waited for otherwise, it is unavoidable. I.e. once the ID/threshold
>> are exported out of a fence, the waiter can only see the fence being
>> signaled by the threshold being reached, not by the syncpoint getting
>> freed.
> 
> This is correct. If sync point's FD is exported or once sync point is
> resolved from a dma-fence, then sync point will stay alive until the
> last reference to the sync point is dropped. I.e. if Process A dies
> *after* process B started to wait on the sync point, then it will get stuck.
> 
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel