From: Nicolas Dufresne <nicolas@ndufresne.ca>
To: Stanimir Varbanov <stanimir.varbanov@linaro.org>,
Tomasz Figa <tfiga@chromium.org>,
Hans Verkuil <hverkuil@xs4all.nl>
Cc: Linux Media Mailing List <linux-media@vger.kernel.org>,
Mauro Carvalho Chehab <mchehab@kernel.org>,
Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
linux-arm-msm <linux-arm-msm@vger.kernel.org>,
Vikash Garodia <vgarodia@codeaurora.org>,
Alexandre Courbot <acourbot@chromium.org>,
Malathi Gottam <mgottam@codeaurora.org>
Subject: Re: [PATCH 10/10] venus: dec: make decoder compliant with stateful codec API
Date: Fri, 31 May 2019 09:53:12 -0400 [thread overview]
Message-ID: <d325988a749b5084471c7f8198f851af58f2ac01.camel@ndufresne.ca> (raw)
In-Reply-To: <ee62fdb0-d8fa-0695-9088-5aaef48e485f@linaro.org>
[-- Attachment #1: Type: text/plain, Size: 7499 bytes --]
Le vendredi 31 mai 2019 à 11:01 +0300, Stanimir Varbanov a écrit :
> Hi,
>
> On 5/27/19 11:18 AM, Tomasz Figa wrote:
> > On Mon, May 27, 2019 at 4:39 PM Hans Verkuil <hverkuil@xs4all.nl> wrote:
> > > On 5/27/19 5:51 AM, Tomasz Figa wrote:
> > > > On Tue, May 21, 2019 at 9:27 PM Hans Verkuil <hverkuil@xs4all.nl> wrote:
> > > > > On 5/21/19 11:09 AM, Tomasz Figa wrote:
> > > > > > Hi Stan,
> > > > > >
> > > > > > On Mon, May 20, 2019 at 11:47 PM Stanimir Varbanov
> > > > > > <stanimir.varbanov@linaro.org> wrote:
> > > > > > > Hi Tomasz,
> > > > > > >
> > > > > > > On 4/24/19 3:39 PM, Tomasz Figa wrote:
> > > > > > > > On Wed, Apr 24, 2019 at 9:15 PM Stanimir Varbanov
> > > > > > > > <stanimir.varbanov@linaro.org> wrote:
> > > > > > > > > Hi Hans,
> > > > > > > > >
> > > > > > > > > On 2/15/19 3:44 PM, Hans Verkuil wrote:
> > > > > > > > > > Hi Stanimir,
> > > > > > > > > >
> > > > > > > > > > I never paid much attention to this patch series since others were busy
> > > > > > > > > > discussing it and I had a lot of other things on my plate, but then I heard
> > > > > > > > > > that this patch made G_FMT blocking.
> > > > > > > > >
> > > > > > > > > OK, another option could be to block REQBUF(CAPTURE) until event from hw
> > > > > > > > > is received that the stream is parsed and the resolution is correctly
> > > > > > > > > set by application. Just to note that I'd think to this like a temporal
> > > > > > > > > solution until gstreamer implements v4l events.
> > > > > > > > >
> > > > > > > > > Is that looks good to you?
> > > > > > > >
> > > > > > > > Hmm, I thought we concluded that gstreamer sets the width and height
> > > > > > > > in OUTPUT queue before querying the CAPTURE queue and so making the
> > > > > > > > driver calculate the CAPTURE format based on what's set on OUTPUT
> > > > > > > > would work fine. Did I miss something?
> > > > > > >
> > > > > > > Nobody is miss something.
> > > > > > >
> > > > > > > First some background about how Venus implements stateful codec API.
> > > > > > >
> > > > > > > The Venus firmware can generate two events "sufficient" and
> > > > > > > "insufficient" buffer requirements (this includes decoder output buffer
> > > > > > > size and internal/scratch buffer sizes). Presently I always set minimum
> > > > > > > possible decoder resolution no matter what the user said, and by that
> > > > > > > way I'm sure that "insufficient" event will always be triggered by the
> > > > > > > firmware (the other reason to take this path is because this is the
> > > > > > > least-common-divider for all supported Venus hw/fw versions thus common
> > > > > > > code in the driver). The reconfiguration (during codec Initialization
> > > > > > > sequence) is made from STREAMON(CAPTURE) context. Now, to make that
> > > > > > > re-configuration happen I need to wait for "insufficient" event from
> > > > > > > firmware in order to know the real coded resolution.
> > > > > > >
> > > > > > > In the case of gstreamer where v4l2_events support is missing I have to
> > > > > > > block (wait for firmware event) REQBUF(CAPTURE) (vb2::queue_setup) or
> > > > > > > STREAMON(CAPTURE) (vb2::start_streaming).
> > > > > > >
> > > > > > > I tried to set the coded resolution to the firmware as-is it set by
> > > > > > > gstreamer but then I cannot receive the "sufficient" event for VP8 and
> > > > > > > VP9 codecs. So I return back to the solution with minimum resolution above.
> > > > > > >
> > > > > > > I'm open for suggestions.
> > > > > >
> > > > > > I think you could still keep setting the minimum size and wait for the
> > > > > > "insufficient" event. At the same time, you could speculatively
> > > > > > advertise the expected "sufficient" size on the CAPTURE queue before
> > > > > > the hardware signals those. Even if you mispredict them, you'll get
> > > > > > the event, update the CAPTURE resolution and send the source change
> > > > > > event to the application, which would then give you the correct
> > > > > > buffers. Would that work for you?
> > > > >
> > > > > As I understand it this still would require event support, which gstreamer
> > > > > doesn't have.
> > > >
> > > > I don't think it matches what I remember from the earlier discussion.
> > > > As long as Gstreamer sets the visible resolution (from the container
> > > > AFAIR) on OUTPUT, the driver would adjust it to something that is
> > > > expected to be the right framebuffer resolution and so Gstreamer would
> > > > be able to continue. Of course if the expected value doesn't match, it
> > > > wouldn't work, but it's the same as currently for Coda AFAICT.
> > > >
> > > > > I think it is OK to have REQBUFS sleep in this case. However, I would only
> > > >
> > > > Why REQBUFS? While that could possibly allow us to allocate the right
> > > > buffers, Gstreamer wouldn't be able to know the right format, because
> > > > it would query it before REQBUFS, wouldn't it?
> > >
> > > Oops, you are right. It's got to be in G_FMT(CAPTURE), but *only* if
> > > nobody subscribed to the SOURCE_CHANGE event.
> > >
> > > > For this reason, s5p-mfc makes G_FMT(CAPTURE) blocking and if we
> > > > decide to forcefully keep the compatibility, even with in drivers, we
> > > > should probably do the same here.
> > > >
> > > > > enable this behavior if the application didn't subscribe to the SOURCE_CHANGE
> > > > > event. That's easy enough to check in the driver. And that means that if the
> > > > > application is well written, then the driver will behave in a completely
> > > > > standard way that the compliance test can check.
> > > >
> > > > I guess one could have some helpers for this. They would listen to the
> > > > source change events internally and block / wake-up appropriate ioctls
> > > > whenever necessary.
> > >
> > > I really do not want this for new drivers. gstreamer should be fixed.
> > > A blocking G_FMT is just plain bad. Only those drivers that do this, can
> > > still block if nobody subscribed to EVENT_SOURCE_CHANGE.
> >
> > Yeah and that's why I just suggested to mimic coda, which doesn't
> > block, but apparently gstreamer still works with it.
>
> Unfortunately in Venus case that is not an easy task (as I tried to
> explain why above).
>
> To have an unified and common code for all different SoCs and
> firmware/hardware versions I decided to set the minimum supported
> resolution for the decoder (no matter what the user said) and trigger
> the reconfiguration event always. Something more, I need the event also
> to retrieve the minimum capture buffers
> (V4L2_CID_MIN_BUFFERS_FOR_CAPTURE) and sizes for capture and
> internal/scratch buffers as well, thus I really need to wait for that
> event.
And to add to this, CODA driver is special, since it has a third buffer
queue, the queue expose on the CAPTURE device does not need to be
representative of the DPB depth.
>
> So, just to confirm - you are fine with blocking G_FMT (not REQBUF) when
> the user doesn't subscribe for v4l2 events?
>
> > > > Another question: If we intend this to be implemented in new drivers
> > > > too, should it be documented in the spec?
> > >
> > > We most certainly do NOT want to implement this in new drivers.
> > >
> >
> > Makes sense.
> >
> > When venus was merged initially, did it already have a blocking G_FMT?
>
> No it isn't.
>
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 195 bytes --]
next prev parent reply other threads:[~2019-05-31 13:53 UTC|newest]
Thread overview: 71+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-01-17 16:19 [PATCH 00/10] Venus stateful Codec API Stanimir Varbanov
2019-01-17 16:19 ` [PATCH 01/10] venus: hfi_cmds: add more not-implemented properties Stanimir Varbanov
2019-01-17 16:20 ` [PATCH 02/10] venus: helpers: fix dynamic buffer mode for v4 Stanimir Varbanov
2019-01-17 16:20 ` [PATCH 03/10] venus: helpers: export few helper functions Stanimir Varbanov
2019-01-24 8:43 ` Alexandre Courbot
2019-01-24 8:48 ` Stanimir Varbanov
2019-01-17 16:20 ` [PATCH 04/10] venus: hfi: add type argument to hfi flush function Stanimir Varbanov
2019-01-17 16:20 ` [PATCH 05/10] venus: hfi: export few HFI functions Stanimir Varbanov
2019-01-17 16:20 ` [PATCH 06/10] venus: hfi: return an error if session_init is already called Stanimir Varbanov
2019-01-17 16:20 ` [PATCH 07/10] venus: helpers: add three more helper functions Stanimir Varbanov
2019-01-24 8:43 ` Alexandre Courbot
2019-01-24 8:54 ` Stanimir Varbanov
2019-01-25 5:47 ` Alexandre Courbot
2019-01-17 16:20 ` [PATCH 08/10] venus: vdec_ctrls: get real minimum buffers for capture Stanimir Varbanov
2019-01-24 8:43 ` Alexandre Courbot
2019-01-24 9:59 ` Stanimir Varbanov
2019-01-17 16:20 ` [PATCH 09/10] venus: vdec: allow bigger sizeimage set by clients Stanimir Varbanov
2019-01-24 8:43 ` Alexandre Courbot
2019-01-24 10:05 ` Stanimir Varbanov
2019-01-25 5:36 ` Alexandre Courbot
2019-01-17 16:20 ` [PATCH 10/10] venus: dec: make decoder compliant with stateful codec API Stanimir Varbanov
2019-01-21 11:20 ` mgottam
2019-01-21 11:29 ` mgottam
2019-01-24 10:24 ` Stanimir Varbanov
2019-01-24 8:44 ` Alexandre Courbot
2019-01-24 12:34 ` Stanimir Varbanov
2019-01-25 5:44 ` Alexandre Courbot
2019-01-25 7:59 ` Tomasz Figa
2019-01-25 10:25 ` Stanimir Varbanov
2019-01-28 7:38 ` Tomasz Figa
2019-01-28 16:28 ` Stanimir Varbanov
2019-01-29 8:24 ` Tomasz Figa
2019-01-30 3:18 ` Nicolas Dufresne
2019-01-30 3:38 ` Tomasz Figa
2019-01-30 4:21 ` Nicolas Dufresne
2019-01-30 6:17 ` Tomasz Figa
2019-01-30 15:32 ` Nicolas Dufresne
2019-01-31 12:42 ` Philipp Zabel
2019-01-31 13:34 ` Tomasz Figa
2019-01-31 15:18 ` Nicolas Dufresne
2019-02-05 6:26 ` Tomasz Figa
2019-02-05 9:00 ` Hans Verkuil
2019-02-05 9:31 ` Tomasz Figa
2019-02-05 10:35 ` Hans Verkuil
2019-02-07 7:33 ` Tomasz Figa
2019-02-14 2:43 ` Tomasz Figa
2019-02-14 16:19 ` Nicolas Dufresne
2019-04-09 9:59 ` Tomasz Figa
2019-04-09 16:30 ` Nicolas Dufresne
2019-04-10 2:32 ` Tomasz Figa
2019-02-15 13:44 ` Hans Verkuil
2019-02-15 16:27 ` Nicolas Dufresne
2019-02-15 16:40 ` Hans Verkuil
2019-04-24 12:15 ` Stanimir Varbanov
2019-04-24 12:39 ` Tomasz Figa
2019-05-20 14:47 ` Stanimir Varbanov
2019-05-21 9:09 ` Tomasz Figa
2019-05-21 12:27 ` Hans Verkuil
2019-05-27 3:51 ` Tomasz Figa
2019-05-27 7:39 ` Hans Verkuil
2019-05-27 8:18 ` Tomasz Figa
2019-05-31 8:01 ` Stanimir Varbanov
2019-05-31 13:53 ` Nicolas Dufresne [this message]
2019-06-03 7:25 ` Hans Verkuil
2019-06-03 7:30 ` Tomasz Figa
2019-01-24 8:43 ` [PATCH 00/10] Venus stateful Codec API Alexandre Courbot
2019-01-24 10:12 ` Stanimir Varbanov
2019-01-25 5:34 ` Alexandre Courbot
2019-01-25 10:35 ` Stanimir Varbanov
2019-01-28 4:16 ` Alexandre Courbot
2019-01-28 4:30 ` Tomasz Figa
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=d325988a749b5084471c7f8198f851af58f2ac01.camel@ndufresne.ca \
--to=nicolas@ndufresne.ca \
--cc=acourbot@chromium.org \
--cc=hverkuil@xs4all.nl \
--cc=linux-arm-msm@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-media@vger.kernel.org \
--cc=mchehab@kernel.org \
--cc=mgottam@codeaurora.org \
--cc=stanimir.varbanov@linaro.org \
--cc=tfiga@chromium.org \
--cc=vgarodia@codeaurora.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).