Re: Stateless Encoding uAPI Discussion and Proposal

[Randy Li <ayaka@soulik.info>]

On 2023/8/30 23:10, Nicolas Dufresne wrote:
> CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe.
>
>
> Le mercredi 23 août 2023 à 11:04 +0800, Hsia-Jun Li a écrit :
>>> Though, if we drop the GOP structure and favour this approach, the latency could
>>> be regain later by introducing fence base streaming. The technique would be for
>>> a video source (like a capture driver) to pass dmabuf that aren't filled yet,
>>> but have a companion fence. This would allow queuing requests ahead of time, and
>>> all we need is enough pre-allocation to accommodate the desired look ahead. Only
>>> issue is that perhaps this violates the fundamental of "short term" delivery of
>>> fences. But fences can also fail I think, in case the capture was stopped.
>>>
>> I don't think it would help. Fence is a thing for DRM/GPU without a queue.
>> Even with a fence, would the video sink tell us the motion delta here?
> It helps with the latency since the encoder can start its search and analyzes as
> soon as frames are available, instead of until you have all N frames available
> (refer to the MIN_BUFFER_FOR controls used when lookahead is needed).

I think the fence in GPU is something attached to per frame 
buffer(IN_FENCE) or completing the render(OUT_FENCE).

So when we enqueue a buffer, what are expecting from the fence?

I think in KMS, you can't enqueue two buffers for the same plane, you 
have to wait the OUT_FENCE.

>
>>> We can certainly move forward with this as a future solution, or just don't
>>> implement future aware RC algorithm in term to avoid the huge task this involves
>>> (and possibly patents?)
>>>
>> I think we should not restrict how the userspace(vendor) operate the
>> hardware.
> Omitting is not restricting. Vendors have to learn to be community members and
> propose/add the tools and APIs they need to support their features. We cannot
> fix vendors in this regard, those who jumps over that fence are wining.

That is not about what vendor would do. I was thinking we are planning 
how we manage the lifetime of the reconstruction buffer, reference 
selecting based on a simple GOP model.
What was designed here would become a barrier for a vendor whose 
hardware has a little capability than this.

All I want to do here is offer my ideas about how we could achieve an 
open interfaces that could cover the future need.

Especially it is hard to expand the V4L2 uAPIs.

>
> Nicolas