From: tiffany lin <tiffany.lin@mediatek.com> To: Nicolas Dufresne <nicolas@ndufresne.ca> Cc: "Wu-Cheng Li (李務誠)" <wuchengli@chromium.org>, "Hans Verkuil" <hverkuil@xs4all.nl>, "Hans Verkuil" <hans.verkuil@cisco.com>, "Daniel Thompson" <daniel.thompson@linaro.org>, "Rob Herring" <robh+dt@kernel.org>, "Mauro Carvalho Chehab" <mchehab@osg.samsung.com>, "Matthias Brugger" <matthias.bgg@gmail.com>, "Daniel Kurtz" <djkurtz@chromium.org>, "Pawel Osciak" <posciak@chromium.org>, "Eddie Huang" <eddie.huang@mediatek.com>, "Yingjoe Chen" <yingjoe.chen@mediatek.com>, devicetree@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-media@vger.kernel.org, linux-mediatek@lists.infradead.org, "Lin PoChun" <PoChun.Lin@mediatek.com> Subject: Re: [PATCH v3 3/9] DocBook/v4l: Add compressed video formats used on MT8173 codec driver Date: Thu, 14 Jul 2016 15:30:10 +0800 [thread overview] Message-ID: <1468481410.27199.17.camel@mtksdaap41> (raw) In-Reply-To: <1468418157.2867.12.camel@ndufresne.ca> Hi Nicolas, On Wed, 2016-07-13 at 09:55 -0400, Nicolas Dufresne wrote: > Le mercredi 13 juillet 2016 à 10:00 +0800, tiffany lin a écrit : > > Hi Nicolas, > > > > On Tue, 2016-07-12 at 15:14 -0400, Nicolas Dufresne wrote: > > > Le mardi 12 juillet 2016 à 15:08 -0400, Nicolas Dufresne a écrit : > > > > Le mardi 12 juillet 2016 à 16:16 +0800, Wu-Cheng Li (李務誠) a écrit > > : > > > > > Decoder hardware produces MT21 (compressed). Image processor > > can > > > > > convert it to a format that can be input of display driver. > > > > > Tiffany. > > > > > When do you plan to upstream image processor (mtk-mdp)? > > > > > > > > > > > > It can be as input format for encoder, MDP and display > > drivers in > > > > > our > > > > > > platform. > > > > > I remember display driver can only accept uncompressed MT21. > > Right? > > > > > Basically V4L2_PIX_FMT_MT21 is compressed and is like an opaque > > > > > format. It's not usable until it's decompressed and converted > > by > > > > > image > > > > > processor. > > > > > > > > Previously it was described as MediaTek block mode, and now as a > > > > MediaTek compressed format. It makes me think you have no idea > > what > > > > this pixel format really is. Is that right ? > > > > > > > > The main reason why I keep asking, is that we often find > > similarities > > > > between what vendor like to call their proprietary formats. Doing > > the > > > > proper research helps not creating a mess like in Android where > > you > > > > have a lot of formats that all point to the same format. I > > believe > > > > there was the same concern when Samsung wanted to introduce their > > Z- > > > > flip-Z NV12 tile format. In the end they simply provided > > sufficient > > > > documentation so we could document it and implement software > > > > converters > > > > for test and validation purpose. > > > > > > Here's the kind of information we want in the documentation. > > > > > > https://chromium.googlesource.com/chromium/src/media/+/master/base/ > > vide > > > o_types.h#40 > > > > > > // MediaTek proprietary format. MT21 is similar to NV21 except > > the memory > > > // layout and pixel layout (swizzles). 12bpp with Y plane > > followed by a 2x2 > > > // interleaved VU plane. Each image contains two buffers -- Y > > plane and VU > > > // plane. Two planes can be non-contiguous in memory. The > > starting addresses > > > // of Y plane and VU plane are 4KB alignment. > > > // Suppose image dimension is (width, height). For both Y plane > > and VU plane: > > > // Row pitch = ((width+15)/16) * 16. > > > // Plane size = Row pitch * (((height+31)/32)*32) > > > > > > Now obviously this is incomplete, as the swizzling need to be > > documented of course. > > > > > Because it's finally a compressed format from our codec hw, we cannot > > describe its swizzling. > > Can you clarify why it cannot be described ? I don't buy any argument > that it's impossible to convert without hardware. Intel did document > their compressed formats. > What's the Intel compressed formats? Could you provide link so we could reference how they provide this information. > Debugging an integration with such format that we have literally > decided to no make public is a pain. A lot of decoder got abandoned or > never used because of that. > > It would be nice to explain the architecture you are suggesting to make > this decoder useful. How will userspace application be able to use your > decoder. Will the image processor usable outside the display path ? > People might want to do transcoding, or other relatively common task > that does not imply a display. > In MT8173 platform, it has MDP driver, it could convert V4L2_PIX_FMT_MT21 to V4L2_PIX_FMT_NV12M or V4L2_PIX_FMT_YUV420M. MDP is stand alone driver, it is usable outside the display path. NV12M/YUV420M/MT21 -> MDP -> NV12M/YUV420M NV12M/NV21M/YUV420M/YVU420M -> mt8173 Encoder -> H264/VP8 H264/VP8/VP9 -> mtk8173 Decoder -> MT21 When encode with MT21 source, the pipeline will be: MT21 -> MDP driver-> NV12M/NV21M/YUV420M/YVU420M -> Encoder -> H264/VP8 When playback, the pipeline will be: H264/VP8/VP9 -> Decoder driver -> MT21 -> MDP Driver -> DRM best regards, Tiffany > regards, > Nicolas > > p.s. > A small note to Wu-Cheng, the definition of the DRM format in your > Chromium branch is not correct. DRM uses format modifier, such that the > format is the family, in your case it's most likely NV21, and the > modifier is the compression or tiling algorithm in place (or both as > needed).
WARNING: multiple messages have this Message-ID (diff)
From: tiffany.lin@mediatek.com (tiffany lin) To: linux-arm-kernel@lists.infradead.org Subject: [PATCH v3 3/9] DocBook/v4l: Add compressed video formats used on MT8173 codec driver Date: Thu, 14 Jul 2016 15:30:10 +0800 [thread overview] Message-ID: <1468481410.27199.17.camel@mtksdaap41> (raw) In-Reply-To: <1468418157.2867.12.camel@ndufresne.ca> Hi Nicolas, On Wed, 2016-07-13 at 09:55 -0400, Nicolas Dufresne wrote: > Le mercredi 13 juillet 2016 ? 10:00 +0800, tiffany lin a ?crit : > > Hi Nicolas, > > > > On Tue, 2016-07-12 at 15:14 -0400, Nicolas Dufresne wrote: > > > Le mardi 12 juillet 2016 ? 15:08 -0400, Nicolas Dufresne a ?crit : > > > > Le mardi 12 juillet 2016 ? 16:16 +0800, Wu-Cheng Li (???) a ?crit > > : > > > > > Decoder hardware produces MT21 (compressed). Image processor > > can > > > > > convert it to a format that can be input of display driver. > > > > > Tiffany. > > > > > When do you plan to upstream image processor (mtk-mdp)? > > > > > > > > > > > > It can be as input format for encoder, MDP and display > > drivers in > > > > > our > > > > > > platform. > > > > > I remember display driver can only accept uncompressed MT21. > > Right? > > > > > Basically V4L2_PIX_FMT_MT21 is compressed and is like an opaque > > > > > format. It's not usable until it's decompressed and converted > > by > > > > > image > > > > > processor. > > > > > > > > Previously it was described as MediaTek block mode, and now as a > > > > MediaTek compressed format. It makes me think you have no idea > > what > > > > this pixel format really is. Is that right ? > > > > > > > > The main reason why I keep asking, is that we often find > > similarities > > > > between what vendor like to call their proprietary formats. Doing > > the > > > > proper research helps not creating a mess like in Android where > > you > > > > have a lot of formats that all point to the same format. I > > believe > > > > there was the same concern when Samsung wanted to introduce their > > Z- > > > > flip-Z NV12 tile format. In the end they simply provided > > sufficient > > > > documentation so we could document it and implement software > > > > converters > > > > for test and validation purpose. > > > > > > Here's the kind of information we want in the documentation. > > > > > > https://chromium.googlesource.com/chromium/src/media/+/master/base/ > > vide > > > o_types.h#40 > > > > > > // MediaTek proprietary format. MT21 is similar to NV21 except > > the memory > > > // layout and pixel layout (swizzles). 12bpp with Y plane > > followed by a 2x2 > > > // interleaved VU plane. Each image contains two buffers -- Y > > plane and VU > > > // plane. Two planes can be non-contiguous in memory. The > > starting addresses > > > // of Y plane and VU plane are 4KB alignment. > > > // Suppose image dimension is (width, height). For both Y plane > > and VU plane: > > > // Row pitch = ((width+15)/16) * 16. > > > // Plane size = Row pitch * (((height+31)/32)*32) > > > > > > Now obviously this is incomplete, as the swizzling need to be > > documented of course. > > > > > Because it's finally a compressed format from our codec hw, we cannot > > describe its swizzling. > > Can you clarify why it cannot be described ? I don't buy any argument > that it's impossible to convert without hardware. Intel did document > their compressed formats. > What's the Intel compressed formats? Could you provide link so we could reference how they provide this information. > Debugging an integration with such format that we have literally > decided to no make public is a pain. A lot of decoder got abandoned or > never used because of that. > > It would be nice to explain the architecture you are suggesting to make > this decoder useful. How will userspace application be able to use your > decoder. Will the image processor usable outside the display path ? > People might want to do transcoding, or other relatively common task > that does not imply a display. > In MT8173 platform, it has MDP driver, it could convert V4L2_PIX_FMT_MT21 to V4L2_PIX_FMT_NV12M or V4L2_PIX_FMT_YUV420M. MDP is stand alone driver, it is usable outside the display path. NV12M/YUV420M/MT21 -> MDP -> NV12M/YUV420M NV12M/NV21M/YUV420M/YVU420M -> mt8173 Encoder -> H264/VP8 H264/VP8/VP9 -> mtk8173 Decoder -> MT21 When encode with MT21 source, the pipeline will be: MT21 -> MDP driver-> NV12M/NV21M/YUV420M/YVU420M -> Encoder -> H264/VP8 When playback, the pipeline will be: H264/VP8/VP9 -> Decoder driver -> MT21 -> MDP Driver -> DRM best regards, Tiffany > regards, > Nicolas > > p.s. > A small note to Wu-Cheng, the definition of the DRM format in your > Chromium branch is not correct. DRM uses format modifier, such that the > format is the family, in your case it's most likely NV21, and the > modifier is the compression or tiling algorithm in place (or both as > needed).
next prev parent reply other threads:[~2016-07-14 7:30 UTC|newest] Thread overview: 125+ messages / expand[flat|nested] mbox.gz Atom feed top 2016-05-30 12:29 [PATCH v3 0/9] Add MT8173 Video Decoder Driver Tiffany Lin 2016-05-30 12:29 ` Tiffany Lin 2016-05-30 12:29 ` Tiffany Lin 2016-05-30 12:29 ` [PATCH v3 1/9] VPU: mediatek: Add decode support Tiffany Lin 2016-05-30 12:29 ` Tiffany Lin 2016-05-30 12:29 ` Tiffany Lin 2016-05-30 12:29 ` [PATCH v3 2/9] v4l: add Mediatek compressed video block format Tiffany Lin 2016-05-30 12:29 ` Tiffany Lin 2016-05-30 12:29 ` Tiffany Lin 2016-05-30 12:29 ` [PATCH v3 3/9] DocBook/v4l: Add compressed video formats used on MT8173 codec driver Tiffany Lin 2016-05-30 12:29 ` Tiffany Lin 2016-05-30 12:29 ` Tiffany Lin 2016-05-30 12:29 ` [PATCH v3 4/9] dt-bindings: Add a binding for Mediatek Video Decoder Tiffany Lin 2016-05-30 12:29 ` Tiffany Lin 2016-05-30 12:29 ` Tiffany Lin 2016-05-30 12:29 ` [PATCH v3 5/9] vcodec: mediatek: Add Mediatek V4L2 Video Decoder Driver Tiffany Lin 2016-05-30 12:29 ` Tiffany Lin 2016-05-30 12:29 ` Tiffany Lin 2016-05-30 12:29 ` [PATCH v3 6/9] vcodec: mediatek: Add Mediatek H264 " Tiffany Lin 2016-05-30 12:29 ` Tiffany Lin 2016-05-30 12:29 ` Tiffany Lin 2016-05-30 12:29 ` [PATCH v3 7/9] vcodec: mediatek: Add Mediatek VP8 " Tiffany Lin 2016-05-30 12:29 ` Tiffany Lin 2016-05-30 12:29 ` Tiffany Lin 2016-05-30 12:29 ` [PATCH v3 8/9] vcodec: mediatek: Add Mediatek VP9 " Tiffany Lin 2016-05-30 12:29 ` Tiffany Lin 2016-05-30 12:29 ` Tiffany Lin 2016-05-30 12:29 ` [PATCH v3 9/9] arm64: dts: mediatek: Add Video Decoder for MT8173 Tiffany Lin 2016-05-30 12:29 ` Tiffany Lin 2016-05-30 12:29 ` Tiffany Lin 2016-07-08 11:06 ` [PATCH v3 5/9] vcodec: mediatek: Add Mediatek V4L2 Video Decoder Driver Hans Verkuil 2016-07-08 11:06 ` Hans Verkuil 2016-08-01 9:38 ` Tiffany Lin 2016-08-01 9:38 ` Tiffany Lin 2016-08-01 9:38 ` Tiffany Lin 2016-08-01 10:39 ` Hans Verkuil 2016-08-01 10:39 ` Hans Verkuil 2016-08-01 11:28 ` Tiffany Lin 2016-08-01 11:28 ` Tiffany Lin 2016-08-01 11:28 ` Tiffany Lin 2016-07-08 10:23 ` [PATCH v3 3/9] DocBook/v4l: Add compressed video formats used on MT8173 codec driver Hans Verkuil 2016-07-08 10:23 ` Hans Verkuil 2016-07-08 10:23 ` Hans Verkuil 2016-07-11 2:56 ` tiffany lin 2016-07-11 2:56 ` tiffany lin 2016-07-11 2:56 ` tiffany lin 2016-07-12 8:16 ` Wu-Cheng Li (李務誠) 2016-07-12 8:16 ` Wu-Cheng Li (李務誠) 2016-07-12 19:08 ` Nicolas Dufresne 2016-07-12 19:08 ` Nicolas Dufresne 2016-07-12 19:14 ` Nicolas Dufresne 2016-07-12 19:14 ` Nicolas Dufresne 2016-07-12 19:31 ` Ian Arkver 2016-07-12 19:31 ` Ian Arkver 2016-07-13 2:00 ` tiffany lin 2016-07-13 2:00 ` tiffany lin 2016-07-13 2:00 ` tiffany lin 2016-07-13 13:55 ` Nicolas Dufresne 2016-07-13 13:55 ` Nicolas Dufresne 2016-07-13 13:55 ` Nicolas Dufresne 2016-07-14 7:30 ` tiffany lin [this message] 2016-07-14 7:30 ` tiffany lin 2016-07-13 2:22 ` Wu-Cheng Li (李務誠) 2016-07-13 2:22 ` Wu-Cheng Li (李務誠) 2016-07-13 2:22 ` Wu-Cheng Li (李務誠) 2016-07-13 1:50 ` tiffany lin 2016-07-13 1:50 ` tiffany lin 2016-07-13 1:50 ` tiffany lin 2016-07-13 2:03 ` tiffany lin 2016-07-13 2:03 ` tiffany lin 2016-07-13 2:03 ` tiffany lin 2016-06-07 14:22 ` [PATCH v3 0/9] Add MT8173 Video Decoder Driver Mauro Carvalho Chehab 2016-06-07 14:22 ` Mauro Carvalho Chehab 2016-06-07 14:22 ` Mauro Carvalho Chehab 2016-06-07 22:13 ` Hans Verkuil 2016-06-07 22:13 ` Hans Verkuil 2016-06-07 22:13 ` Hans Verkuil 2016-06-14 8:44 ` Wu-Cheng Li (李務誠) 2016-06-14 8:44 ` Wu-Cheng Li (李務誠) 2016-06-14 8:44 ` Wu-Cheng Li (李務誠) 2016-06-14 11:08 ` tiffany lin 2016-06-14 11:08 ` tiffany lin 2016-06-14 11:08 ` tiffany lin 2016-06-16 10:54 ` Mauro Carvalho Chehab 2016-06-16 10:54 ` Mauro Carvalho Chehab 2016-07-01 10:11 ` Hans Verkuil 2016-07-01 10:11 ` Hans Verkuil 2016-07-01 10:11 ` Hans Verkuil 2016-07-01 10:47 ` Wu-Cheng Li (李務誠) 2016-07-01 10:47 ` Wu-Cheng Li (李務誠) 2016-07-01 10:47 ` Wu-Cheng Li (李務誠) 2016-07-01 11:53 ` andrew-ct chen 2016-07-01 11:53 ` andrew-ct chen 2016-07-01 11:53 ` andrew-ct chen 2016-07-01 12:00 ` Hans Verkuil 2016-07-01 12:00 ` Hans Verkuil 2016-07-01 12:00 ` Hans Verkuil 2016-07-04 11:09 ` andrew-ct chen 2016-07-04 11:09 ` andrew-ct chen 2016-07-04 11:09 ` andrew-ct chen 2016-07-06 13:19 ` Hans Verkuil 2016-07-06 13:19 ` Hans Verkuil 2016-07-07 10:16 ` tiffany lin 2016-07-07 10:16 ` tiffany lin 2016-07-07 10:16 ` tiffany lin 2016-07-07 11:13 ` Hans Verkuil 2016-07-07 11:13 ` Hans Verkuil 2016-07-08 11:44 ` Hans Verkuil 2016-07-08 11:44 ` Hans Verkuil 2016-07-11 3:58 ` tiffany lin 2016-07-11 3:58 ` tiffany lin 2016-07-11 3:58 ` tiffany lin 2016-07-21 9:12 ` tiffany lin 2016-07-21 9:12 ` tiffany lin 2016-07-21 9:12 ` tiffany lin 2016-08-15 8:30 ` Hans Verkuil 2016-08-15 8:30 ` Hans Verkuil 2016-08-15 9:03 ` Tiffany Lin 2016-08-15 9:03 ` Tiffany Lin 2016-08-15 9:03 ` Tiffany Lin 2016-08-15 9:07 ` Hans Verkuil 2016-08-15 9:07 ` Hans Verkuil 2016-08-15 9:10 ` Tiffany Lin 2016-08-15 9:10 ` Tiffany Lin 2016-08-15 9:10 ` Tiffany Lin
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=1468481410.27199.17.camel@mtksdaap41 \ --to=tiffany.lin@mediatek.com \ --cc=PoChun.Lin@mediatek.com \ --cc=daniel.thompson@linaro.org \ --cc=devicetree@vger.kernel.org \ --cc=djkurtz@chromium.org \ --cc=eddie.huang@mediatek.com \ --cc=hans.verkuil@cisco.com \ --cc=hverkuil@xs4all.nl \ --cc=linux-arm-kernel@lists.infradead.org \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-media@vger.kernel.org \ --cc=linux-mediatek@lists.infradead.org \ --cc=matthias.bgg@gmail.com \ --cc=mchehab@osg.samsung.com \ --cc=nicolas@ndufresne.ca \ --cc=posciak@chromium.org \ --cc=robh+dt@kernel.org \ --cc=wuchengli@chromium.org \ --cc=yingjoe.chen@mediatek.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.