From mboxrd@z Thu Jan 1 00:00:00 1970 From: Hans Verkuil Subject: Re: [PATCH v4 3/5] staging: Introduce NVIDIA Tegra video decoder driver Date: Tue, 12 Dec 2017 13:54:34 +0100 Message-ID: References: <1a3798f337c0097e67d70226ae3ba665fd9156c2.1508448293.git.digetx@gmail.com> <3ac6a087-def2-014f-673d-1be9d5094635@gmail.com> <93c98569-0282-80d9-78ad-c8ab8fd9db92@xs4all.nl> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: driverdev-devel-bounces@linuxdriverproject.org Sender: "devel" To: Dmitry Osipenko , Thierry Reding , Jonathan Hunter , Greg Kroah-Hartman , Rob Herring , Mauro Carvalho Chehab , Stephen Warren , Vladimir Zapolskiy Cc: devel@driverdev.osuosl.org, devicetree@vger.kernel.org, linux-kernel@vger.kernel.org, linux-tegra@vger.kernel.org, Maxime Ripard , Giulio Benetti , Dan Carpenter , linux-media@vger.kernel.org List-Id: linux-tegra@vger.kernel.org On 10/12/17 19:56, Dmitry Osipenko wrote: > On 05.12.2017 16:03, Hans Verkuil wrote: >> On 12/05/17 13:17, Dmitry Osipenko wrote: >>> Hi Hans, >>> >>> On 04.12.2017 17:04, Hans Verkuil wrote: >>>> Hi Dmitry, >>>> >>>> As you already mention in the TODO, this should become a v4l2 codec driver. >>>> >>>> Good existing examples are the coda, qcom/venus and mtk-vcodec drivers. >>>> >>>> One thing that is not clear from this code is if the tegra hardware is a >>>> stateful or stateless codec, i.e. does it keep track of the decoder state >>>> in the hardware, or does the application have to keep track of the state and >>>> provide the state information together with the video data? >>>> >>>> I ask because at the moment only stateful codecs are supported. Work is ongoing >>>> to support stateless codecs, but we don't support that for now. >>>> >>> >>> It is stateless. Is there anything ready to try out? If yes, could you please >>> give a reference to that work? >> >> I rebased my two year old 'requests2' branch to the latest mainline version and >> gave it the imaginative name 'requests3': >> >> https://git.linuxtv.org/hverkuil/media_tree.git/log/?h=requests3 >> >> (Note: only compile tested!) > > Thank you very much. > >> This is what ChromeOS has been using (actually they use a slightly older version) >> and the new version that is currently being developed will be similar, so any work >> you do on top of this will carry over to the final version without too much effort. >> >> At least, that's the intention :-) >> >> I've CC-ed Maxime and Giulio as well: they are looking into adding support for >> the stateless allwinner codec based on this code as well. There may well be >> opportunities for you to work together, esp. on the userspace side. Note that >> Rockchip has the same issue, they too have a stateless HW codec. > > IIUC, we will have to define video decoder parameters in V4L API and then make a > V4L driver / userspace prototype (ffmpeg for example) that will use the requests > API for video decoding in order to upstream the requests API. Does it sound good? Correct. Hugues Fruchet made an example bit parser for mpeg: https://www.spinics.net/lists/linux-media/msg115017.html So something similar would work. My recommendation would be to make a separate library that can be shared among different implementations (i.e. in gstreamer, in vdpau, using a libv4l2 plugin, etc., etc.). That can easily be hosted as part of v4l-utils to keep it in sync with the kernel drivers. If it's independent of the various drivers, then it can be hosted anywhere of course. BTW, we as V4L2 core developers have no plans on working on such a library :-) Regards, Hans > >>> >>>> Anyway, I'm OK with merging this in staging. Although I think it should go >>>> to staging/media since we want to keep track of it. >>>> >>> >>> Awesome, I'll move driver to staging/media in V5. Thanks! >> >> Nice, thanks! From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754565AbdLLNgT (ORCPT ); Tue, 12 Dec 2017 08:36:19 -0500 Received: from lb1-smtp-cloud8.xs4all.net ([194.109.24.21]:38541 "EHLO lb1-smtp-cloud8.xs4all.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753752AbdLLMyi (ORCPT ); Tue, 12 Dec 2017 07:54:38 -0500 Subject: Re: [PATCH v4 3/5] staging: Introduce NVIDIA Tegra video decoder driver To: Dmitry Osipenko , Thierry Reding , Jonathan Hunter , Greg Kroah-Hartman , Rob Herring , Mauro Carvalho Chehab , Stephen Warren , Vladimir Zapolskiy References: <1a3798f337c0097e67d70226ae3ba665fd9156c2.1508448293.git.digetx@gmail.com> <3ac6a087-def2-014f-673d-1be9d5094635@gmail.com> <93c98569-0282-80d9-78ad-c8ab8fd9db92@xs4all.nl> Cc: Dan Carpenter , linux-media@vger.kernel.org, linux-tegra@vger.kernel.org, devel@driverdev.osuosl.org, devicetree@vger.kernel.org, linux-kernel@vger.kernel.org, Maxime Ripard , Giulio Benetti From: Hans Verkuil Message-ID: Date: Tue, 12 Dec 2017 13:54:34 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.7.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-CMAE-Envelope: MS4wfG9Wyc0cnzuG3DFhYNvqVQZiDf9sFfQDzSMZ84mpbDqChM8ka3TDL5YI+Kvr7z4HKlYOY78Dj1Dr/dTy+AOwr7lxZwkA8XcQ66xlukyLG1Go0iKCNZAN +VW6FetMF2/qngXfqOV7PIekaszN5+r5gOTuDyBTrh0RCVqmHQXAXa+OR+fz0QrvxTA8lgpxxgyFI+ZI0MnCtXPVWjOzgLMLYMFWGjFR5/fdSCcHGDpi+kZC GKJ9wQxx6ljiSuT1tYfhdX7WIyW1TjWS67P/rWEGdYeuBCdvH72Q32jHR6x2RWNGAA1bD55Qn2TvkGTU2DJrYFYliCNYCj1kK0ndmapVwwj/EoaA7Tar1k5v XN+wbmWgz+pHULF7hgFNCu/fiFbxOLG7JQ2YUdHaQOouzpjil95DckGbFw7YLzDW/oDZeQqsWP+yy+SOlih0BJ7kCZwaOa2DKhWYlKdXtwIkRwvpOJbEepju n9NasxeNbVZl02LAhrid72hWSc4R3iHCZwJqX3DXUwPclddiiTrbQXajioljHKyTdX6I0ozkku86l9Vxme3LNflVA3JgIjgsnWt4PA== Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 10/12/17 19:56, Dmitry Osipenko wrote: > On 05.12.2017 16:03, Hans Verkuil wrote: >> On 12/05/17 13:17, Dmitry Osipenko wrote: >>> Hi Hans, >>> >>> On 04.12.2017 17:04, Hans Verkuil wrote: >>>> Hi Dmitry, >>>> >>>> As you already mention in the TODO, this should become a v4l2 codec driver. >>>> >>>> Good existing examples are the coda, qcom/venus and mtk-vcodec drivers. >>>> >>>> One thing that is not clear from this code is if the tegra hardware is a >>>> stateful or stateless codec, i.e. does it keep track of the decoder state >>>> in the hardware, or does the application have to keep track of the state and >>>> provide the state information together with the video data? >>>> >>>> I ask because at the moment only stateful codecs are supported. Work is ongoing >>>> to support stateless codecs, but we don't support that for now. >>>> >>> >>> It is stateless. Is there anything ready to try out? If yes, could you please >>> give a reference to that work? >> >> I rebased my two year old 'requests2' branch to the latest mainline version and >> gave it the imaginative name 'requests3': >> >> https://git.linuxtv.org/hverkuil/media_tree.git/log/?h=requests3 >> >> (Note: only compile tested!) > > Thank you very much. > >> This is what ChromeOS has been using (actually they use a slightly older version) >> and the new version that is currently being developed will be similar, so any work >> you do on top of this will carry over to the final version without too much effort. >> >> At least, that's the intention :-) >> >> I've CC-ed Maxime and Giulio as well: they are looking into adding support for >> the stateless allwinner codec based on this code as well. There may well be >> opportunities for you to work together, esp. on the userspace side. Note that >> Rockchip has the same issue, they too have a stateless HW codec. > > IIUC, we will have to define video decoder parameters in V4L API and then make a > V4L driver / userspace prototype (ffmpeg for example) that will use the requests > API for video decoding in order to upstream the requests API. Does it sound good? Correct. Hugues Fruchet made an example bit parser for mpeg: https://www.spinics.net/lists/linux-media/msg115017.html So something similar would work. My recommendation would be to make a separate library that can be shared among different implementations (i.e. in gstreamer, in vdpau, using a libv4l2 plugin, etc., etc.). That can easily be hosted as part of v4l-utils to keep it in sync with the kernel drivers. If it's independent of the various drivers, then it can be hosted anywhere of course. BTW, we as V4L2 core developers have no plans on working on such a library :-) Regards, Hans > >>> >>>> Anyway, I'm OK with merging this in staging. Although I think it should go >>>> to staging/media since we want to keep track of it. >>>> >>> >>> Awesome, I'll move driver to staging/media in V5. Thanks! >> >> Nice, thanks!