From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.0 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BC711C07542 for ; Mon, 27 May 2019 07:39:45 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 8ED472173C for ; Mon, 27 May 2019 07:39:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1725940AbfE0Hjp (ORCPT ); Mon, 27 May 2019 03:39:45 -0400 Received: from lb3-smtp-cloud9.xs4all.net ([194.109.24.30]:54193 "EHLO lb3-smtp-cloud9.xs4all.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725869AbfE0Hjp (ORCPT ); Mon, 27 May 2019 03:39:45 -0400 Received: from [IPv6:2001:983:e9a7:1:f4bd:6355:63eb:2e52] ([IPv6:2001:983:e9a7:1:f4bd:6355:63eb:2e52]) by smtp-cloud9.xs4all.net with ESMTPA id VAEOhZ8u3sDWyVAEPhXnKi; Mon, 27 May 2019 09:39:42 +0200 Subject: Re: [PATCH 10/10] venus: dec: make decoder compliant with stateful codec API To: Tomasz Figa Cc: Stanimir Varbanov , Linux Media Mailing List , Mauro Carvalho Chehab , Linux Kernel Mailing List , linux-arm-msm , Vikash Garodia , Alexandre Courbot , Malathi Gottam References: <20190117162008.25217-1-stanimir.varbanov@linaro.org> <20190117162008.25217-11-stanimir.varbanov@linaro.org> <60b3efff-31c1-bc04-8af9-deebb8bc013a@xs4all.nl> <01b6683f-9378-e6f2-501f-e2213e6c690d@xs4all.nl> From: Hans Verkuil Message-ID: Date: Mon, 27 May 2019 09:39:39 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.6.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-CMAE-Envelope: MS4wfP/KlNMUpI8R5BYs77TLAQQpb7XPrT/Q7FQW/9jjzWxU0gK9jSERXE2vWgoHtc8gKV48bA83Qt+62pLbBCyHlf2cvHrjyp5CqYS4fFJGLXiLBwq/jI9c hDq9Idy8JzJI+TCm9X9vX8wxHPMEL6lKNgy+iy0TLmkFwWX/0jDE6KvfDJvi+5bhUPQdA7VjcJzONb7wRB+Pi62MCK/GiCKLLtJW1QuVyIXXQh0Yz3ayCMM3 /9G4xg2jLk6kZMq25trnWTnSUZiLoGr6Ln25TSWe4qZLw/iHQ/mpxfPHWHIkIQw5IWkJcOJ/WRkjcscZxz7AD3WCSA1wgkRvkf6oZLBzDl9DZf82qdO94+K0 nVx9GiUbhAFS+oVCSink8KPdAY2wkqDQ2csnvHoaiJc0a1fzNRPo/psWIzxGj+7fgnV2bw6WWx5I/dbNoremu6MqgDyKIqD18Vbn/NX7pwBZBdmvfDxfIi2b ZTDdE8WOzge1lP4qDPRR/GHpUmUnMi0dJ5VgLdTbJ+9EcaLCuuM0X/rWuT7pGzfUXyFrfentz+yHsfwlumZV5Vpak5oFPIwyhzgKog== Sender: linux-arm-msm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org On 5/27/19 5:51 AM, Tomasz Figa wrote: > On Tue, May 21, 2019 at 9:27 PM Hans Verkuil wrote: >> >> On 5/21/19 11:09 AM, Tomasz Figa wrote: >>> Hi Stan, >>> >>> On Mon, May 20, 2019 at 11:47 PM Stanimir Varbanov >>> wrote: >>>> >>>> Hi Tomasz, >>>> >>>> On 4/24/19 3:39 PM, Tomasz Figa wrote: >>>>> On Wed, Apr 24, 2019 at 9:15 PM Stanimir Varbanov >>>>> wrote: >>>>>> >>>>>> Hi Hans, >>>>>> >>>>>> On 2/15/19 3:44 PM, Hans Verkuil wrote: >>>>>>> Hi Stanimir, >>>>>>> >>>>>>> I never paid much attention to this patch series since others were busy >>>>>>> discussing it and I had a lot of other things on my plate, but then I heard >>>>>>> that this patch made G_FMT blocking. >>>>>> >>>>>> OK, another option could be to block REQBUF(CAPTURE) until event from hw >>>>>> is received that the stream is parsed and the resolution is correctly >>>>>> set by application. Just to note that I'd think to this like a temporal >>>>>> solution until gstreamer implements v4l events. >>>>>> >>>>>> Is that looks good to you? >>>>> >>>>> Hmm, I thought we concluded that gstreamer sets the width and height >>>>> in OUTPUT queue before querying the CAPTURE queue and so making the >>>>> driver calculate the CAPTURE format based on what's set on OUTPUT >>>>> would work fine. Did I miss something? >>>> >>>> Nobody is miss something. >>>> >>>> First some background about how Venus implements stateful codec API. >>>> >>>> The Venus firmware can generate two events "sufficient" and >>>> "insufficient" buffer requirements (this includes decoder output buffer >>>> size and internal/scratch buffer sizes). Presently I always set minimum >>>> possible decoder resolution no matter what the user said, and by that >>>> way I'm sure that "insufficient" event will always be triggered by the >>>> firmware (the other reason to take this path is because this is the >>>> least-common-divider for all supported Venus hw/fw versions thus common >>>> code in the driver). The reconfiguration (during codec Initialization >>>> sequence) is made from STREAMON(CAPTURE) context. Now, to make that >>>> re-configuration happen I need to wait for "insufficient" event from >>>> firmware in order to know the real coded resolution. >>>> >>>> In the case of gstreamer where v4l2_events support is missing I have to >>>> block (wait for firmware event) REQBUF(CAPTURE) (vb2::queue_setup) or >>>> STREAMON(CAPTURE) (vb2::start_streaming). >>>> >>>> I tried to set the coded resolution to the firmware as-is it set by >>>> gstreamer but then I cannot receive the "sufficient" event for VP8 and >>>> VP9 codecs. So I return back to the solution with minimum resolution above. >>>> >>>> I'm open for suggestions. >>> >>> I think you could still keep setting the minimum size and wait for the >>> "insufficient" event. At the same time, you could speculatively >>> advertise the expected "sufficient" size on the CAPTURE queue before >>> the hardware signals those. Even if you mispredict them, you'll get >>> the event, update the CAPTURE resolution and send the source change >>> event to the application, which would then give you the correct >>> buffers. Would that work for you? >> >> As I understand it this still would require event support, which gstreamer >> doesn't have. > > I don't think it matches what I remember from the earlier discussion. > As long as Gstreamer sets the visible resolution (from the container > AFAIR) on OUTPUT, the driver would adjust it to something that is > expected to be the right framebuffer resolution and so Gstreamer would > be able to continue. Of course if the expected value doesn't match, it > wouldn't work, but it's the same as currently for Coda AFAICT. > >> >> I think it is OK to have REQBUFS sleep in this case. However, I would only > > Why REQBUFS? While that could possibly allow us to allocate the right > buffers, Gstreamer wouldn't be able to know the right format, because > it would query it before REQBUFS, wouldn't it? Oops, you are right. It's got to be in G_FMT(CAPTURE), but *only* if nobody subscribed to the SOURCE_CHANGE event. > > For this reason, s5p-mfc makes G_FMT(CAPTURE) blocking and if we > decide to forcefully keep the compatibility, even with in drivers, we > should probably do the same here. > >> enable this behavior if the application didn't subscribe to the SOURCE_CHANGE >> event. That's easy enough to check in the driver. And that means that if the >> application is well written, then the driver will behave in a completely >> standard way that the compliance test can check. > > I guess one could have some helpers for this. They would listen to the > source change events internally and block / wake-up appropriate ioctls > whenever necessary. I really do not want this for new drivers. gstreamer should be fixed. A blocking G_FMT is just plain bad. Only those drivers that do this, can still block if nobody subscribed to EVENT_SOURCE_CHANGE. > Another question: If we intend this to be implemented in new drivers > too, should it be documented in the spec? We most certainly do NOT want to implement this in new drivers. Regards, Hans > > Best regards, > Tomasz >