[PATCH v3 0/9] Add MT8173 Video Decoder Driver

[PATCH v3 0/9] Add MT8173 Video Decoder Driver

[Tiffany Lin]

==============
 Introduction
==============

The purpose of this series is to add the driver for video codec hw embedded in the Mediatek's MT8173 SoCs.
Mediatek Video Codec is able to handle video decoding of in a range of formats.

This patch series add Mediatek block format V4L2_PIX_FMT_MT21, the decoder driver will decoded bitstream to
V4L2_PIX_FMT_MT21 format.

This patch series rely on MTK VPU driver in patch series "Add MT8173 Video Encoder Driver and VPU Driver"[1]
and patch "CHROMIUM: v4l: Add V4L2_PIX_FMT_VP9 definition"[2] for VP9 support.
Mediatek Video Decoder driver rely on VPU driver to load, communicate with VPU.

Internally the driver uses videobuf2 framework and MTK IOMMU and MTK SMI both have been merged in v4.6-rc1.

[1]https://patchwork.linuxtv.org/patch/33734/
[2]https://chromium-review.googlesource.com/#/c/245241/

==================
 Device interface
==================

In principle the driver bases on v4l2 memory-to-memory framework:
it provides a single video node and each opened file handle gets its own private context with separate
buffer queues. Each context consist of 2 buffer queues: OUTPUT (for source buffers, i.e. bitstream)
and CAPTURE (for destination buffers, i.e. decoded video frames).
OUTPUT and CAPTURE buffer could be MMAP or DMABUF memory type.
VIDIOC_G_CTRL and VIDIOC_G_EXT_CTRLS return V4L2_CID_MIN_BUFFERS_FOR_CAPTURE only when dirver in MTK_STATE_HEADER
state, or it will return EAGAIN.
Driver do not support subscribe event for control 'User Controls' for now.
And it default support export DMABUF for other display drivers.

Change in v3:
1. Refine vdec hw clock setting
2. Refine vp9 codec driver
3. Refine v4l2 codec driver

Change in v2:
1. Add documentation for V4L2_PIX_FMT_MT21
2. Remove DRM_FORMAT_MT21
2. Refine code according to review comments

v4l2-compliance test output:
localhost Encode # ./v4l2-compliance -d /dev/video0
Driver Info:
        Driver name   : mtk-vcodec-dec
        Card type     : platform:mt8173
        Bus info      : platform:mt8173
        Driver version: 4.4.0
        Capabilities  : 0x84204000
                Video Memory-to-Memory Multiplanar
                Streaming
                Extended Pix Format
                Device Capabilities
        Device Caps   : 0x04204000
                Video Memory-to-Memory Multiplanar
                Streaming
                Extended Pix Format

Compliance test for device /dev/video0 (not using libv4l2):

Required ioctls:
        test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
        test second video open: OK
        test VIDIOC_QUERYCAP: OK
        test VIDIOC_G/S_PRIORITY: OK

Debug ioctls:
        test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
        test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
        test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
        test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
        test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
        test VIDIOC_ENUMAUDIO: OK (Not Supported)
        test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
        test VIDIOC_G/S_AUDIO: OK (Not Supported)
        Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
        test VIDIOC_G/S_MODULATOR: OK (Not Supported)
        test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
        test VIDIOC_ENUMAUDOUT: OK (Not Supported)
        test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
        test VIDIOC_G/S_AUDOUT: OK (Not Supported)
        Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
        test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
        test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
        test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
        test VIDIOC_G/S_EDID: OK (Not Supported)

        Control ioctls:
                test VIDIOC_QUERYCTRL/MENU: OK
                fail: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-controls.cpp(357): g_ctrl returned an error (11)
                test VIDIOC_G/S_CTRL: FAIL
                fail: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-controls.cpp(579): g_ext_ctrls returned an error (11)
                test VIDIOC_G/S/TRY_EXT_CTRLS: FAIL
                fail: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-controls.cpp(721): subscribe event for control 'User Controls' failed
                test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: FAIL
                test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
                Standard Controls: 2 Private Controls: 0

        Format ioctls:
                test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
                test VIDIOC_G/S_PARM: OK (Not Supported)
                test VIDIOC_G_FBUF: OK (Not Supported)
                fail: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-formats.cpp(405): expected EINVAL, but got 11 when getting format for buftype 9
                test VIDIOC_G_FMT: FAIL
                test VIDIOC_TRY_FMT: OK (Not Supported)
                test VIDIOC_S_FMT: OK (Not Supported)
                test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)

        Codec ioctls:
                test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
                test VIDIOC_G_ENC_INDEX: OK (Not Supported)
                test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

        Buffer ioctls:
                test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
                fail: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-buffers.cpp(500): q.has_expbuf(node)
                test VIDIOC_EXPBUF: FAIL


Total: 38, Succeeded: 33, Failed: 5, Warnings: 0


Andrew-CT Chen (1):
  VPU: mediatek: Add decode support

Tiffany Lin (8):
  v4l: add Mediatek compressed video block format
  DocBook/v4l: Add compressed video formats used on MT8173 codec driver
  dt-bindings: Add a binding for Mediatek Video Decoder
  vcodec: mediatek: Add Mediatek V4L2 Video Decoder Driver
  vcodec: mediatek: Add Mediatek H264 Video Decoder Driver
  vcodec: mediatek: Add Mediatek VP8 Video Decoder Driver
  vcodec: mediatek: Add Mediatek VP9 Video Decoder Driver
  arm64: dts: mediatek: Add Video Decoder for MT8173

 Documentation/DocBook/media/v4l/pixfmt.xml         |    6 +
 .../devicetree/bindings/media/mediatek-vcodec.txt  |   57 +-
 arch/arm64/boot/dts/mediatek/mt8173.dtsi           |   44 +
 drivers/media/platform/mtk-vcodec/Makefile         |   15 +-
 drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c | 1348 ++++++++++++++++++++
 drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.h |   88 ++
 .../media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c |  408 ++++++
 .../media/platform/mtk-vcodec/mtk_vcodec_dec_pm.c  |  206 +++
 .../media/platform/mtk-vcodec/mtk_vcodec_dec_pm.h  |   28 +
 drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h |   88 +-
 .../media/platform/mtk-vcodec/mtk_vcodec_intr.c    |    4 +-
 .../media/platform/mtk-vcodec/mtk_vcodec_intr.h    |    2 +-
 .../media/platform/mtk-vcodec/mtk_vcodec_util.c    |   31 +-
 .../media/platform/mtk-vcodec/mtk_vcodec_util.h    |    7 +-
 .../media/platform/mtk-vcodec/vdec/vdec_h264_if.c  |  503 ++++++++
 .../media/platform/mtk-vcodec/vdec/vdec_vp8_if.c   |  630 +++++++++
 .../media/platform/mtk-vcodec/vdec/vdec_vp9_if.c   |  967 ++++++++++++++
 drivers/media/platform/mtk-vcodec/vdec_drv_base.h  |   56 +
 drivers/media/platform/mtk-vcodec/vdec_drv_if.c    |  117 ++
 drivers/media/platform/mtk-vcodec/vdec_drv_if.h    |  101 ++
 drivers/media/platform/mtk-vcodec/vdec_ipi_msg.h   |  103 ++
 drivers/media/platform/mtk-vcodec/vdec_vpu_if.c    |  168 +++
 drivers/media/platform/mtk-vcodec/vdec_vpu_if.h    |   96 ++
 drivers/media/platform/mtk-vpu/mtk_vpu.c           |   12 +
 drivers/media/platform/mtk-vpu/mtk_vpu.h           |   27 +
 include/uapi/linux/videodev2.h                     |    1 +
 26 files changed, 5090 insertions(+), 23 deletions(-)
 create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c
 create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.h
 create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c
 create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_pm.c
 create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_pm.h
 create mode 100644 drivers/media/platform/mtk-vcodec/vdec/vdec_h264_if.c
 create mode 100644 drivers/media/platform/mtk-vcodec/vdec/vdec_vp8_if.c
 create mode 100644 drivers/media/platform/mtk-vcodec/vdec/vdec_vp9_if.c
 create mode 100644 drivers/media/platform/mtk-vcodec/vdec_drv_base.h
 create mode 100644 drivers/media/platform/mtk-vcodec/vdec_drv_if.c
 create mode 100644 drivers/media/platform/mtk-vcodec/vdec_drv_if.h
 create mode 100644 drivers/media/platform/mtk-vcodec/vdec_ipi_msg.h
 create mode 100644 drivers/media/platform/mtk-vcodec/vdec_vpu_if.c
 create mode 100644 drivers/media/platform/mtk-vcodec/vdec_vpu_if.h

-- 
1.7.9.5

[PATCH v3 1/9] VPU: mediatek: Add decode support

[Tiffany Lin]

From: Andrew-CT Chen <andrew-ct.chen@mediatek.com>

VPU driver add decode support

Signed-off-by: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
---
 drivers/media/platform/mtk-vpu/mtk_vpu.c |   12 ++++++++++++
 drivers/media/platform/mtk-vpu/mtk_vpu.h |   27 +++++++++++++++++++++++++++
 2 files changed, 39 insertions(+)

diff --git a/drivers/media/platform/mtk-vpu/mtk_vpu.c b/drivers/media/platform/mtk-vpu/mtk_vpu.c
index b60d02c..ca23b1f 100644
--- a/drivers/media/platform/mtk-vpu/mtk_vpu.c
+++ b/drivers/media/platform/mtk-vpu/mtk_vpu.c
@@ -134,6 +134,8 @@ struct vpu_wdt {
  *
  * @signaled:		the signal of vpu initialization completed
  * @fw_ver:		VPU firmware version
+ * @dec_capability:	decoder capability which is not used for now and
+ *			the value is reserved for future use
  * @enc_capability:	encoder capability which is not used for now and
  *			the value is reserved for future use
  * @wq:			wait queue for VPU initialization status
@@ -141,6 +143,7 @@ struct vpu_wdt {
 struct vpu_run {
 	u32 signaled;
 	char fw_ver[VPU_FW_VER_LEN];
+	unsigned int	dec_capability;
 	unsigned int	enc_capability;
 	wait_queue_head_t wq;
 };
@@ -415,6 +418,14 @@ int vpu_wdt_reg_handler(struct platform_device *pdev,
 }
 EXPORT_SYMBOL_GPL(vpu_wdt_reg_handler);
 
+unsigned int vpu_get_vdec_hw_capa(struct platform_device *pdev)
+{
+	struct mtk_vpu *vpu = platform_get_drvdata(pdev);
+
+	return vpu->run.dec_capability;
+}
+EXPORT_SYMBOL_GPL(vpu_get_vdec_hw_capa);
+
 unsigned int vpu_get_venc_hw_capa(struct platform_device *pdev)
 {
 	struct mtk_vpu *vpu = platform_get_drvdata(pdev);
@@ -600,6 +611,7 @@ static void vpu_init_ipi_handler(void *data, unsigned int len, void *priv)
 
 	vpu->run.signaled = run->signaled;
 	strncpy(vpu->run.fw_ver, run->fw_ver, VPU_FW_VER_LEN);
+	vpu->run.dec_capability = run->dec_capability;
 	vpu->run.enc_capability = run->enc_capability;
 	wake_up_interruptible(&vpu->run.wq);
 }
diff --git a/drivers/media/platform/mtk-vpu/mtk_vpu.h b/drivers/media/platform/mtk-vpu/mtk_vpu.h
index 5ab37f0..f457479 100644
--- a/drivers/media/platform/mtk-vpu/mtk_vpu.h
+++ b/drivers/media/platform/mtk-vpu/mtk_vpu.h
@@ -37,6 +37,18 @@ typedef void (*ipi_handler_t) (void *data,
 			 command to VPU.
 			 For other IPI below, AP should send the request
 			 to VPU to trigger the interrupt.
+ * @IPI_VDEC_H264:	 The interrupt from vpu is to notify kernel to
+			 handle H264 vidoe decoder job, and vice versa.
+			 Decode output format is always MT21 no matter what
+			 the input format is.
+ * @IPI_VDEC_VP8:	 The interrupt from is to notify kernel to
+			 handle VP8 video decoder job, and vice versa.
+			 Decode output format is always MT21 no matter what
+			 the input format is.
+ * @IPI_VDEC_VP9:	 The interrupt from vpu is to notify kernel to
+			 handle VP9 video decoder job, and vice versa.
+			 Decode output format is always MT21 no matter what
+			 the input format is.
  * @IPI_VENC_H264:	 The interrupt from vpu is to notify kernel to
 			 handle H264 video encoder job, and vice versa.
  * @IPI_VENC_VP8:	 The interrupt fro vpu is to notify kernel to
@@ -46,6 +58,9 @@ typedef void (*ipi_handler_t) (void *data,
 
 enum ipi_id {
 	IPI_VPU_INIT = 0,
+	IPI_VDEC_H264,
+	IPI_VDEC_VP8,
+	IPI_VDEC_VP9,
 	IPI_VENC_H264,
 	IPI_VENC_VP8,
 	IPI_MAX,
@@ -55,10 +70,12 @@ enum ipi_id {
  * enum rst_id - reset id to register reset function for VPU watchdog timeout
  *
  * @VPU_RST_ENC: encoder reset id
+ * @VPU_RST_DEC: decoder reset id
  * @VPU_RST_MAX: maximum reset id
  */
 enum rst_id {
 	VPU_RST_ENC,
+	VPU_RST_DEC,
 	VPU_RST_MAX,
 };
 
@@ -125,6 +142,16 @@ struct platform_device *vpu_get_plat_device(struct platform_device *pdev);
 int vpu_wdt_reg_handler(struct platform_device *pdev,
 			void vpu_wdt_reset_func(void *),
 			void *priv, enum rst_id id);
+
+/**
+ * vpu_get_vdec_hw_capa - get video decoder hardware capability
+ *
+ * @pdev:	VPU platform device
+ *
+ * Return: video decoder hardware capability
+ **/
+unsigned int vpu_get_vdec_hw_capa(struct platform_device *pdev);
+
 /**
  * vpu_get_venc_hw_capa - get video encoder hardware capability
  *
-- 
1.7.9.5

[PATCH v3 2/9] v4l: add Mediatek compressed video block format

[Tiffany Lin]

Add V4L2_PIX_FMT_MT21 format used on MT8173 driver.
It is compressed format and need MT8173 MDP driver to transfer to other
standard format.

Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
---
 include/uapi/linux/videodev2.h |    1 +
 1 file changed, 1 insertion(+)

diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 8f95191..52feea6 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -625,6 +625,7 @@ struct v4l2_pix_format {
 #define V4L2_PIX_FMT_Y8I      v4l2_fourcc('Y', '8', 'I', ' ') /* Greyscale 8-bit L/R interleaved */
 #define V4L2_PIX_FMT_Y12I     v4l2_fourcc('Y', '1', '2', 'I') /* Greyscale 12-bit L/R interleaved */
 #define V4L2_PIX_FMT_Z16      v4l2_fourcc('Z', '1', '6', ' ') /* Depth data 16-bit */
+#define V4L2_PIX_FMT_MT21     v4l2_fourcc('M', 'T', '2', '1') /* Mediatek compressed block mode  */
 
 /* SDR formats - used only for Software Defined Radio devices */
 #define V4L2_SDR_FMT_CU8          v4l2_fourcc('C', 'U', '0', '8') /* IQ u8 */
-- 
1.7.9.5

[PATCH v3 3/9] DocBook/v4l: Add compressed video formats used on MT8173 codec driver

[Tiffany Lin]

Add V4L2_PIX_FMT_MT21 documentation

Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
---
 Documentation/DocBook/media/v4l/pixfmt.xml |    6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/Documentation/DocBook/media/v4l/pixfmt.xml b/Documentation/DocBook/media/v4l/pixfmt.xml
index 5a08aee..d40e0ce 100644
--- a/Documentation/DocBook/media/v4l/pixfmt.xml
+++ b/Documentation/DocBook/media/v4l/pixfmt.xml
@@ -1980,6 +1980,12 @@ array. Anything what's in between the UYVY lines is JPEG data and should be
 concatenated to form the JPEG stream. </para>
 </entry>
 	  </row>
+	  <row id="V4L2_PIX_FMT_MT21">
+	    <entry><constant>V4L2_PIX_FMT_MT21</constant></entry>
+	    <entry>'MT21'</entry>
+	    <entry>Compressed two-planar YVU420 format used by Mediatek MT8173
+	    codec driver.</entry>
+	  </row>
 	</tbody>
       </tgroup>
     </table>
-- 
1.7.9.5

[PATCH v3 4/9] dt-bindings: Add a binding for Mediatek Video Decoder

[Tiffany Lin]

Add a DT binding documentation of Video Decoder for the
MT8173 SoC from Mediatek.

Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
Acked-by: Rob Herring <rob@kernel.org>
---
 .../devicetree/bindings/media/mediatek-vcodec.txt  |   57 ++++++++++++++++++--
 1 file changed, 53 insertions(+), 4 deletions(-)

diff --git a/Documentation/devicetree/bindings/media/mediatek-vcodec.txt b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
index 59a47a5..46c15c5 100644
--- a/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
+++ b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
@@ -1,25 +1,74 @@
 Mediatek Video Codec
 
 Mediatek Video Codec is the video codec hw present in Mediatek SoCs which
-supports high resolution encoding functionalities.
+supports high resolution encoding and decoding functionalities.
 
 Required properties:
 - compatible : "mediatek,mt8173-vcodec-enc" for encoder
+  "mediatek,mt8173-vcodec-dec" for decoder.
 - reg : Physical base address of the video codec registers and length of
   memory mapped region.
 - interrupts : interrupt number to the cpu.
 - mediatek,larb : must contain the local arbiters in the current Socs.
 - clocks : list of clock specifiers, corresponding to entries in
   the clock-names property.
-- clock-names: encoder must contain "venc_sel_src", "venc_sel",
-- "venc_lt_sel_src", "venc_lt_sel".
+- clock-names: encoder must contain "venc_sel_src", "venc_sel",,
+  "venc_lt_sel_src", "venc_lt_sel", decoder must contain "vcodecpll",
+  "univpll_d2", "clk_cci400_sel", "vdec_sel", "vdecpll", "vencpll",
+  "venc_lt_sel", "vdec_bus_clk_src".
 - iommus : should point to the respective IOMMU block with master port as
   argument, see Documentation/devicetree/bindings/iommu/mediatek,iommu.txt
   for details.
 - mediatek,vpu : the node of video processor unit
 
+
 Example:
-vcodec_enc: vcodec at 0x18002000 {
+
+vcodec_dec: vcodec at 16000000 {
+    compatible = "mediatek,mt8173-vcodec-dec";
+    reg = <0 0x16000000 0 0x100>,   /*VDEC_SYS*/
+          <0 0x16020000 0 0x1000>,  /*VDEC_MISC*/
+          <0 0x16021000 0 0x800>,   /*VDEC_LD*/
+          <0 0x16021800 0 0x800>,   /*VDEC_TOP*/
+          <0 0x16022000 0 0x1000>,  /*VDEC_CM*/
+          <0 0x16023000 0 0x1000>,  /*VDEC_AD*/
+          <0 0x16024000 0 0x1000>,  /*VDEC_AV*/
+          <0 0x16025000 0 0x1000>,  /*VDEC_PP*/
+          <0 0x16026800 0 0x800>,   /*VP8_VD*/
+          <0 0x16027000 0 0x800>,   /*VP6_VD*/
+          <0 0x16027800 0 0x800>,   /*VP8_VL*/
+          <0 0x16028400 0 0x400>;   /*VP9_VD*/
+    interrupts = <GIC_SPI 204 IRQ_TYPE_LEVEL_LOW>;
+    mediatek,larb = <&larb1>;
+    iommus = <&iommu M4U_PORT_HW_VDEC_MC_EXT>,
+             <&iommu M4U_PORT_HW_VDEC_PP_EXT>,
+             <&iommu M4U_PORT_HW_VDEC_AVC_MV_EXT>,
+             <&iommu M4U_PORT_HW_VDEC_PRED_RD_EXT>,
+             <&iommu M4U_PORT_HW_VDEC_PRED_WR_EXT>,
+             <&iommu M4U_PORT_HW_VDEC_UFO_EXT>,
+             <&iommu M4U_PORT_HW_VDEC_VLD_EXT>,
+             <&iommu M4U_PORT_HW_VDEC_VLD2_EXT>;
+    mediatek,vpu = <&vpu>;
+    power-domains = <&scpsys MT8173_POWER_DOMAIN_VDEC>;
+    clocks = <&apmixedsys CLK_APMIXED_VCODECPLL>,
+             <&topckgen CLK_TOP_UNIVPLL_D2>,
+             <&topckgen CLK_TOP_CCI400_SEL>,
+             <&topckgen CLK_TOP_VDEC_SEL>,
+             <&topckgen CLK_TOP_VCODECPLL>,
+             <&apmixedsys CLK_APMIXED_VENCPLL>,
+             <&topckgen CLK_TOP_VENC_LT_SEL>,
+             <&topckgen CLK_TOP_VCODECPLL_370P5>;
+    clock-names = "vcodecpll",
+                  "univpll_d2",
+                  "clk_cci400_sel",
+                  "vdec_sel",
+                  "vdecpll",
+                  "vencpll",
+                  "venc_lt_sel",
+                  "vdec_bus_clk_src";
+  };
+
+  vcodec_enc: vcodec at 0x18002000 {
     compatible = "mediatek,mt8173-vcodec-enc";
     reg = <0 0x18002000 0 0x1000>,    /*VENC_SYS*/
           <0 0x19002000 0 0x1000>;    /*VENC_LT_SYS*/
-- 
1.7.9.5

[PATCH v3 5/9] vcodec: mediatek: Add Mediatek V4L2 Video Decoder Driver

[Tiffany Lin]

Add v4l2 layer decoder driver for MT8173

Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
---
 drivers/media/platform/mtk-vcodec/Makefile         |   10 +-
 drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c | 1348 ++++++++++++++++++++
 drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.h |   88 ++
 .../media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c |  408 ++++++
 .../media/platform/mtk-vcodec/mtk_vcodec_dec_pm.c  |  206 +++
 .../media/platform/mtk-vcodec/mtk_vcodec_dec_pm.h  |   28 +
 drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h |   88 +-
 .../media/platform/mtk-vcodec/mtk_vcodec_intr.c    |    4 +-
 .../media/platform/mtk-vcodec/mtk_vcodec_intr.h    |    2 +-
 .../media/platform/mtk-vcodec/mtk_vcodec_util.c    |   31 +-
 .../media/platform/mtk-vcodec/mtk_vcodec_util.h    |    7 +-
 drivers/media/platform/mtk-vcodec/vdec_drv_base.h  |   56 +
 drivers/media/platform/mtk-vcodec/vdec_drv_if.c    |  107 ++
 drivers/media/platform/mtk-vcodec/vdec_drv_if.h    |  101 ++
 drivers/media/platform/mtk-vcodec/vdec_ipi_msg.h   |  103 ++
 15 files changed, 2569 insertions(+), 18 deletions(-)
 create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c
 create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.h
 create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c
 create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_pm.c
 create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_pm.h
 create mode 100644 drivers/media/platform/mtk-vcodec/vdec_drv_base.h
 create mode 100644 drivers/media/platform/mtk-vcodec/vdec_drv_if.c
 create mode 100644 drivers/media/platform/mtk-vcodec/vdec_drv_if.h
 create mode 100644 drivers/media/platform/mtk-vcodec/vdec_ipi_msg.h

diff --git a/drivers/media/platform/mtk-vcodec/Makefile b/drivers/media/platform/mtk-vcodec/Makefile
index dc5cb00..4c8ed2f 100644
--- a/drivers/media/platform/mtk-vcodec/Makefile
+++ b/drivers/media/platform/mtk-vcodec/Makefile
@@ -1,7 +1,13 @@
 
 
-obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += mtk-vcodec-enc.o mtk-vcodec-common.o
-
+obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += mtk-vcodec-dec.o \
+				       mtk-vcodec-enc.o \
+				       mtk-vcodec-common.o
+
+mtk-vcodec-dec-y := mtk_vcodec_dec_drv.o \
+		vdec_drv_if.o \
+		mtk_vcodec_dec.o \
+		mtk_vcodec_dec_pm.o \
 
 
 mtk-vcodec-enc-y := venc/venc_vp8_if.o \
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c
new file mode 100644
index 0000000..792858a
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c
@@ -0,0 +1,1348 @@
+/*
+* Copyright (c) 2016 MediaTek Inc.
+* Author: PC Chen <pc.chen@mediatek.com>
+*         Tiffany Lin <tiffany.lin@mediatek.com>
+*
+* This program is free software; you can redistribute it and/or modify
+* it under the terms of the GNU General Public License version 2 as
+* published by the Free Software Foundation.
+*
+* This program is distributed in the hope that it will be useful,
+* but WITHOUT ANY WARRANTY; without even the implied warranty of
+* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+* GNU General Public License for more details.
+*/
+
+#include <media/v4l2-event.h>
+#include <media/v4l2-mem2mem.h>
+#include <media/videobuf2-dma-contig.h>
+
+#include "mtk_vcodec_drv.h"
+#include "mtk_vcodec_dec.h"
+#include "mtk_vcodec_intr.h"
+#include "mtk_vcodec_util.h"
+#include "vdec_drv_if.h"
+#include "mtk_vcodec_dec_pm.h"
+
+#define OUT_FMT_IDX	0
+#define CAP_FMT_IDX	3
+
+#define MTK_VDEC_MIN_W	64U
+#define MTK_VDEC_MIN_H	64U
+#define DFT_CFG_WIDTH	MTK_VDEC_MIN_W
+#define DFT_CFG_HEIGHT	MTK_VDEC_MIN_H
+
+static struct mtk_video_fmt mtk_video_formats[] = {
+	{
+		.fourcc = V4L2_PIX_FMT_H264,
+		.type = MTK_FMT_DEC,
+		.num_planes = 1,
+	},
+	{
+		.fourcc = V4L2_PIX_FMT_VP8,
+		.type = MTK_FMT_DEC,
+		.num_planes = 1,
+	},
+	{
+		.fourcc = V4L2_PIX_FMT_VP9,
+		.type = MTK_FMT_DEC,
+		.num_planes = 1,
+	},
+	{
+		.fourcc = V4L2_PIX_FMT_MT21,
+		.type = MTK_FMT_FRAME,
+		.num_planes = 2,
+	},
+};
+
+static const struct mtk_codec_framesizes mtk_vdec_framesizes[] = {
+	{
+		.fourcc	= V4L2_PIX_FMT_H264,
+		.stepwise = {  MTK_VDEC_MIN_W, MTK_VDEC_MAX_W, 16,
+				MTK_VDEC_MIN_H, MTK_VDEC_MAX_H, 16 },
+	},
+	{
+		.fourcc	= V4L2_PIX_FMT_VP8,
+		.stepwise = {  MTK_VDEC_MIN_W, MTK_VDEC_MAX_W, 16,
+				MTK_VDEC_MIN_H, MTK_VDEC_MAX_H, 16 },
+	},
+	{
+		.fourcc = V4L2_PIX_FMT_VP9,
+		.stepwise = {  MTK_VDEC_MIN_W, MTK_VDEC_MAX_W, 16,
+				MTK_VDEC_MIN_H, MTK_VDEC_MAX_H, 16 },
+	},
+};
+
+#define NUM_SUPPORTED_FRAMESIZE ARRAY_SIZE(mtk_vdec_framesizes)
+#define NUM_FORMATS ARRAY_SIZE(mtk_video_formats)
+
+static struct mtk_video_fmt *mtk_vdec_find_format(struct v4l2_format *f)
+{
+	struct mtk_video_fmt *fmt;
+	unsigned int k;
+
+	for (k = 0; k < NUM_FORMATS; k++) {
+		fmt = &mtk_video_formats[k];
+		if (fmt->fourcc == f->fmt.pix_mp.pixelformat)
+			return fmt;
+	}
+
+	return NULL;
+}
+
+static struct mtk_q_data *mtk_vdec_get_q_data(struct mtk_vcodec_ctx *ctx,
+					      enum v4l2_buf_type type)
+{
+	if (V4L2_TYPE_IS_OUTPUT(type))
+		return &ctx->q_data[MTK_Q_DATA_SRC];
+
+	return &ctx->q_data[MTK_Q_DATA_DST];
+}
+
+/* This function is try to clean all display buffer, the buffer will return in
+ * display order.
+ * Note the buffer return from codec driver may still in driver's reference
+ * list.
+ */
+static struct vb2_buffer *get_display_buffer(struct mtk_vcodec_ctx *ctx)
+{
+	struct vdec_fb *disp_frame_buffer = NULL;
+	struct mtk_video_dec_buf *dstbuf;
+
+	mtk_v4l2_debug(3, "[%d]", ctx->id);
+	if (vdec_if_get_param(ctx,
+			GET_PARAM_DISP_FRAME_BUFFER,
+			&disp_frame_buffer)) {
+		mtk_v4l2_err("[%d]Cannot get param : GET_PARAM_DISP_FRAME_BUFFER",
+			ctx->id);
+		return NULL;
+	}
+
+	if (disp_frame_buffer == NULL) {
+		mtk_v4l2_debug(3, "No display frame buffer");
+		return NULL;
+	}
+
+	dstbuf = container_of(disp_frame_buffer, struct mtk_video_dec_buf,
+				frame_buffer);
+	mutex_lock(&ctx->lock);
+	if (dstbuf->used) {
+		vb2_set_plane_payload(&dstbuf->vb.vb2_buf, 0,
+					ctx->picinfo.y_bs_sz);
+		vb2_set_plane_payload(&dstbuf->vb.vb2_buf, 1,
+					ctx->picinfo.c_bs_sz);
+
+		dstbuf->ready_to_display = true;
+
+		mtk_v4l2_debug(2,
+				"[%d]status=%x queue id=%d to done_list %d",
+				ctx->id, disp_frame_buffer->status,
+				dstbuf->vb.vb2_buf.index,
+				dstbuf->queued_in_vb2);
+
+		v4l2_m2m_buf_done(&dstbuf->vb, VB2_BUF_STATE_DONE);
+		ctx->decoded_frame_cnt++;
+	}
+	mutex_unlock(&ctx->lock);
+	return &dstbuf->vb.vb2_buf;
+}
+
+/* This function is try to clean all capture buffers that is not used as
+ * reference buffer by codec driver any more
+ * In this case, we need re-queue buffer to vb2 buffer if user space
+ * already return this buffer to v4l2 or this buffer is just the output of
+ * previous sps/pps/resolution change decode, or do nothing if user
+ * space still own this buffer
+ */
+static struct vb2_buffer *get_free_buffer(struct mtk_vcodec_ctx *ctx)
+{
+	struct mtk_video_dec_buf *dstbuf;
+	struct vdec_fb *free_frame_buffer = NULL;
+
+	if (vdec_if_get_param(ctx,
+				GET_PARAM_FREE_FRAME_BUFFER,
+				&free_frame_buffer)) {
+		mtk_v4l2_err("[%d] Error!! Cannot get param", ctx->id);
+		return NULL;
+	}
+	if (free_frame_buffer == NULL) {
+		mtk_v4l2_debug(3, " No free frame buffer");
+		return NULL;
+	}
+
+	mtk_v4l2_debug(3, "[%d] tmp_frame_addr = 0x%p",
+			ctx->id, free_frame_buffer);
+
+	dstbuf = container_of(free_frame_buffer, struct mtk_video_dec_buf,
+				frame_buffer);
+
+	mutex_lock(&ctx->lock);
+	if (dstbuf->used) {
+		if ((dstbuf->queued_in_vb2) &&
+		    (dstbuf->queued_in_v4l2) &&
+		    (free_frame_buffer->status == FB_ST_FREE)) {
+			/* After decode sps/pps or non-display buffer, we dont
+			 * need to return capture buffer to user space, but
+			 * just re-queue this capture buffer to vb2 queue.
+			 * This reduce overheads that dq/q unused capture
+			 * buffer We use queued_in_vb2 to mark this buffer
+			 * is from decode output, not q from user space.
+			 */
+			mtk_v4l2_debug(2,
+				"[%d]status=%x queue id=%d to rdy_queue %d",
+				ctx->id, free_frame_buffer->status,
+				dstbuf->vb.vb2_buf.index,
+				dstbuf->queued_in_vb2);
+			v4l2_m2m_buf_queue(ctx->m2m_ctx, &dstbuf->vb);
+		} else if ((dstbuf->queued_in_vb2 == false) &&
+			   (dstbuf->queued_in_v4l2 == true)) {
+			/* If buffer in v4l2 driver but not in vb2 queue yet,
+			 * and we get this buffer from free_list, it means
+			 * that codec driver do not use this buffer as
+			 * reference buffer anymore. We should q buffer to vb2
+			 * queue, so later work thread could get this buffer
+			 * for decode In this sate, queued_in_vb2 = false
+			 * menas this buffer is not from previous decode output
+			 */
+			mtk_v4l2_debug(2,
+					"[%d]status=%x queue id=%d to rdy_queue",
+					ctx->id, free_frame_buffer->status,
+					dstbuf->vb.vb2_buf.index);
+			v4l2_m2m_buf_queue(ctx->m2m_ctx, &dstbuf->vb);
+			dstbuf->queued_in_vb2 = true;
+		} else {
+			/* Codec driver do not need to reference this capture
+			 * buffer and this buffer is not in v4l2 driver.
+			 * Then we dont need to do any thing, just add log when
+			 * we need to debug buffer flow.
+			 * When this buffer q from user space, it could
+			 * directly q to vb2 buffer
+			 */
+			mtk_v4l2_debug(3, "[%d]status=%x err queue id=%d %d %d",
+					ctx->id, free_frame_buffer->status,
+					dstbuf->vb.vb2_buf.index,
+					dstbuf->queued_in_vb2,
+					dstbuf->queued_in_v4l2);
+		}
+		dstbuf->used = false;
+	}
+	mutex_unlock(&ctx->lock);
+	return &dstbuf->vb.vb2_buf;
+}
+
+static void clean_display_buffer(struct mtk_vcodec_ctx *ctx)
+{
+	struct vb2_buffer *framptr;
+
+	do {
+		framptr = get_display_buffer(ctx);
+	} while (framptr);
+}
+
+static void clean_free_buffer(struct mtk_vcodec_ctx *ctx)
+{
+	struct vb2_buffer *framptr;
+
+	do {
+		framptr = get_free_buffer(ctx);
+	} while (framptr);
+}
+
+static int mtk_vdec_queue_res_chg_event(struct mtk_vcodec_ctx *ctx)
+{
+	static const struct v4l2_event ev_src_ch = {
+		.type = V4L2_EVENT_SOURCE_CHANGE,
+		.u.src_change.changes =
+		V4L2_EVENT_SRC_CH_RESOLUTION,
+	};
+
+	mtk_v4l2_debug(1, "[%d]", ctx->id);
+	v4l2_event_queue_fh(&ctx->fh, &ev_src_ch);
+
+	return 0;
+}
+
+static void mtk_vdec_flush_decoder(struct mtk_vcodec_ctx *ctx)
+{
+	bool res_chg;
+	int ret = 0;
+
+	ret = vdec_if_decode(ctx, NULL, NULL, &res_chg);
+	if (ret)
+		mtk_v4l2_err("DecodeFinal failed, ret=%d", ret);
+
+	clean_display_buffer(ctx);
+	clean_free_buffer(ctx);
+}
+
+static void mtk_vdec_pic_info_update(struct mtk_vcodec_ctx *ctx)
+{
+	unsigned int dpbsize = 0;
+	int ret;
+
+	if (vdec_if_get_param(ctx,
+				GET_PARAM_PIC_INFO,
+				&ctx->last_decoded_picinfo)) {
+		mtk_v4l2_err("[%d]Error!! Cannot get param : GET_PARAM_PICTURE_INFO ERR",
+				ctx->id);
+		return;
+	}
+
+	if (ctx->last_decoded_picinfo.pic_w == 0 ||
+		ctx->last_decoded_picinfo.pic_h == 0 ||
+		ctx->last_decoded_picinfo.buf_w == 0 ||
+		ctx->last_decoded_picinfo.buf_h == 0) {
+		mtk_v4l2_err("Cannot get correct pic info");
+		return;
+	}
+
+	if ((ctx->last_decoded_picinfo.pic_w == ctx->picinfo.pic_w) ||
+	    (ctx->last_decoded_picinfo.pic_h == ctx->picinfo.pic_h))
+		return;
+
+	mtk_v4l2_debug(1,
+			"[%d]-> new(%d,%d), old(%d,%d), real(%d,%d)",
+			ctx->id, ctx->last_decoded_picinfo.pic_w,
+			ctx->last_decoded_picinfo.pic_h,
+			ctx->picinfo.pic_w, ctx->picinfo.pic_h,
+			ctx->last_decoded_picinfo.buf_w,
+			ctx->last_decoded_picinfo.buf_h);
+
+	ret = vdec_if_get_param(ctx, GET_PARAM_DPB_SIZE, &dpbsize);
+	if (dpbsize == 0)
+		mtk_v4l2_err("Incorrect dpb size, ret=%d", ret);
+
+	ctx->dpb_size = dpbsize;
+}
+
+static void mtk_vdec_worker(struct work_struct *work)
+{
+	struct mtk_vcodec_ctx *ctx = container_of(work, struct mtk_vcodec_ctx,
+				decode_work);
+	struct mtk_vcodec_dev *dev = ctx->dev;
+	struct vb2_buffer *src_buf, *dst_buf;
+	struct mtk_vcodec_mem buf;
+	struct vdec_fb *pfb;
+	bool res_chg = false;
+	int ret;
+	struct mtk_video_dec_buf *dst_buf_info, *src_buf_info;
+	struct vb2_v4l2_buffer *dst_vb2_v4l2, *src_vb2_v4l2;
+
+	src_buf = v4l2_m2m_next_src_buf(ctx->m2m_ctx);
+	if (src_buf == NULL) {
+		v4l2_m2m_job_finish(dev->m2m_dev_dec, ctx->m2m_ctx);
+		mtk_v4l2_debug(1, "[%d] src_buf empty!!", ctx->id);
+		return;
+	}
+
+	dst_buf = v4l2_m2m_next_dst_buf(ctx->m2m_ctx);
+	if (dst_buf == NULL) {
+		v4l2_m2m_job_finish(dev->m2m_dev_dec, ctx->m2m_ctx);
+		mtk_v4l2_debug(1, "[%d] dst_buf empty!!", ctx->id);
+		return;
+	}
+
+	src_vb2_v4l2 = container_of(src_buf, struct vb2_v4l2_buffer, vb2_buf);
+	src_buf_info = container_of(src_vb2_v4l2, struct mtk_video_dec_buf, vb);
+
+	dst_vb2_v4l2 = container_of(dst_buf, struct vb2_v4l2_buffer, vb2_buf);
+	dst_buf_info = container_of(dst_vb2_v4l2, struct mtk_video_dec_buf, vb);
+
+	buf.va = vb2_plane_vaddr(src_buf, 0);
+	buf.dma_addr = vb2_dma_contig_plane_dma_addr(src_buf, 0);
+	buf.size = (size_t)src_buf->planes[0].bytesused;
+	if (!buf.va) {
+		v4l2_m2m_job_finish(dev->m2m_dev_dec, ctx->m2m_ctx);
+		mtk_v4l2_err("[%d] id=%d src_addr is NULL!!",
+				ctx->id, src_buf->index);
+		return;
+	}
+
+	pfb = &dst_buf_info->frame_buffer;
+	pfb->base_y.va = vb2_plane_vaddr(dst_buf, 0);
+	pfb->base_y.dma_addr = vb2_dma_contig_plane_dma_addr(dst_buf, 0);
+	pfb->base_y.size = ctx->picinfo.y_bs_sz + ctx->picinfo.y_len_sz;
+
+	pfb->base_c.va = vb2_plane_vaddr(dst_buf, 1);
+	pfb->base_c.dma_addr = vb2_dma_contig_plane_dma_addr(dst_buf, 1);
+	pfb->base_c.size = ctx->picinfo.c_bs_sz + ctx->picinfo.c_len_sz;
+	pfb->status = 0;
+	mtk_v4l2_debug(3, "===>[%d] vdec_if_decode() ===>", ctx->id);
+	mtk_v4l2_debug(3, "[%d] Bitstream VA=%p DMA=%llx Size=0x%lx vb=%p",
+			ctx->id, buf.va, (u64)buf.dma_addr, buf.size, src_buf);
+
+	mtk_v4l2_debug(3,
+			"id=%d Framebuf  pfb=%p VA=%p Y_DMA=%llx C_DMA=%llx Size=0x%lx",
+			dst_buf->index, pfb,
+			pfb->base_y.va, (u64)pfb->base_y.dma_addr,
+			(u64)pfb->base_c.dma_addr, pfb->base_y.size);
+
+	if (src_buf_info->lastframe) {
+		/* update src buf status */
+		src_buf = v4l2_m2m_src_buf_remove(ctx->m2m_ctx);
+		src_buf_info->lastframe = false;
+		v4l2_m2m_buf_done(&src_buf_info->vb, VB2_BUF_STATE_DONE);
+
+		/* update dst buf status */
+		dst_buf = v4l2_m2m_dst_buf_remove(ctx->m2m_ctx);
+		dst_buf_info->used = false;
+
+		vdec_if_decode(ctx, NULL, NULL, &res_chg);
+		clean_display_buffer(ctx);
+		vb2_set_plane_payload(&dst_buf_info->vb.vb2_buf, 0, 0);
+		vb2_set_plane_payload(&dst_buf_info->vb.vb2_buf, 1, 0);
+		v4l2_m2m_buf_done(&dst_buf_info->vb, VB2_BUF_STATE_DONE);
+		clean_free_buffer(ctx);
+	} else {
+		dst_buf_info->vb.vb2_buf.timestamp
+				= src_buf_info->vb.vb2_buf.timestamp;
+		dst_buf_info->vb.timecode
+				= src_buf_info->vb.timecode;
+		mutex_lock(&ctx->lock);
+		dst_buf_info->used = true;
+		mutex_unlock(&ctx->lock);
+		src_buf_info->used = true;
+
+		ret = vdec_if_decode(ctx, &buf, pfb, &res_chg);
+
+		if (ret) {
+			mtk_v4l2_err(
+				" <===[%d], src_buf[%d]%d sz=0x%zx pts=%llu dst_buf[%d] vdec_if_decode() ret=%d res_chg=%d===>",
+				ctx->id,
+				src_buf->index,
+				src_buf_info->lastframe,
+				buf.size,
+				src_buf_info->vb.vb2_buf.timestamp,
+				dst_buf->index,
+				ret, res_chg);
+			src_buf = v4l2_m2m_src_buf_remove(ctx->m2m_ctx);
+			v4l2_m2m_buf_done(&src_buf_info->vb,
+					VB2_BUF_STATE_ERROR);
+		} else if (res_chg == false) {
+			/* we only return src buffer with VB2_BUF_STATE_DONE
+			  * when decode success without resolution change
+			  */
+			src_buf = v4l2_m2m_src_buf_remove(ctx->m2m_ctx);
+			v4l2_m2m_buf_done(&src_buf_info->vb,
+					VB2_BUF_STATE_DONE);
+		}
+
+		dst_buf = v4l2_m2m_dst_buf_remove(ctx->m2m_ctx);
+		clean_display_buffer(ctx);
+		clean_free_buffer(ctx);
+
+		if (!ret && res_chg) {
+			mtk_vdec_pic_info_update(ctx);
+			/*
+			 * On encountering a resolution change in the stream.
+			 * The driver must first process and decode all
+			 * remaining buffers from before the resolution change
+			 * point, so call flush decode here
+			 */
+			mtk_vdec_flush_decoder(ctx);
+			/*
+			 * After all buffers containing decoded frames from
+			 * before the resolution change point ready to be
+			 * dequeued on the CAPTURE queue, the driver sends a
+			 * V4L2_EVENT_SOURCE_CHANGE event for source change
+			 * type V4L2_EVENT_SRC_CH_RESOLUTION
+			 */
+			mtk_vdec_queue_res_chg_event(ctx);
+		}
+	}
+
+	v4l2_m2m_job_finish(dev->m2m_dev_dec, ctx->m2m_ctx);
+}
+
+void mtk_vdec_unlock(struct mtk_vcodec_ctx *ctx)
+{
+	mutex_unlock(&ctx->dev->dec_mutex);
+}
+
+void mtk_vdec_lock(struct mtk_vcodec_ctx *ctx)
+{
+	mutex_lock(&ctx->dev->dec_mutex);
+}
+
+void mtk_vcodec_vdec_release(struct mtk_vcodec_ctx *ctx)
+{
+	vdec_if_deinit(ctx);
+	ctx->state = MTK_STATE_FREE;
+}
+
+void mtk_vcodec_dec_set_default_params(struct mtk_vcodec_ctx *ctx)
+{
+	struct mtk_q_data *q_data;
+
+	ctx->m2m_ctx->q_lock = &ctx->dev->dev_mutex;
+	ctx->fh.m2m_ctx = ctx->m2m_ctx;
+	ctx->fh.ctrl_handler = &ctx->ctrl_hdl;
+	INIT_WORK(&ctx->decode_work, mtk_vdec_worker);
+	ctx->colorspace = V4L2_COLORSPACE_REC709;
+	ctx->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
+	ctx->quantization = V4L2_QUANTIZATION_DEFAULT;
+	ctx->xfer_func = V4L2_XFER_FUNC_DEFAULT;
+
+	q_data = &ctx->q_data[MTK_Q_DATA_SRC];
+	memset(q_data, 0, sizeof(struct mtk_q_data));
+	q_data->visible_width = DFT_CFG_WIDTH;
+	q_data->visible_height = DFT_CFG_HEIGHT;
+	q_data->fmt = &mtk_video_formats[OUT_FMT_IDX];
+	q_data->field = V4L2_FIELD_NONE;
+
+	ctx->q_data[MTK_Q_DATA_DST].sizeimage[0] =
+		DFT_CFG_WIDTH * DFT_CFG_HEIGHT;
+	ctx->q_data[MTK_Q_DATA_DST].bytesperline[0] = 0;
+
+	q_data = &ctx->q_data[MTK_Q_DATA_DST];
+	memset(q_data, 0, sizeof(struct mtk_q_data));
+	q_data->visible_width = DFT_CFG_WIDTH;
+	q_data->visible_height = DFT_CFG_HEIGHT;
+	q_data->coded_width = DFT_CFG_WIDTH;
+	q_data->coded_height = DFT_CFG_HEIGHT;
+	q_data->fmt = &mtk_video_formats[CAP_FMT_IDX];
+	q_data->field = V4L2_FIELD_NONE;
+
+	v4l_bound_align_image(&q_data->coded_width,
+				MTK_VDEC_MIN_W,
+				MTK_VDEC_MAX_W, 4,
+				&q_data->coded_height,
+				MTK_VDEC_MIN_H,
+				MTK_VDEC_MAX_H, 5, 6);
+
+	q_data->sizeimage[0] = q_data->coded_width * q_data->coded_height;
+	q_data->bytesperline[0] = q_data->coded_width;
+	q_data->sizeimage[1] = q_data->sizeimage[0] / 2;
+	q_data->bytesperline[1] = q_data->coded_width;
+}
+
+static int vidioc_vdec_qbuf(struct file *file, void *priv,
+			    struct v4l2_buffer *buf)
+{
+	struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
+	struct vb2_queue *vq;
+	struct vb2_buffer *vb;
+	struct mtk_video_dec_buf *mtkbuf;
+	struct vb2_v4l2_buffer	*vb2_v4l2;
+
+	if (ctx->state == MTK_STATE_ABORT) {
+		mtk_v4l2_err("[%d] Call on QBUF after unrecoverable error",
+				ctx->id);
+		return -EIO;
+	}
+
+	vq = v4l2_m2m_get_vq(ctx->m2m_ctx, buf->type);
+	vb = vq->bufs[buf->index];
+	vb2_v4l2 = container_of(vb, struct vb2_v4l2_buffer, vb2_buf);
+	mtkbuf = container_of(vb2_v4l2, struct mtk_video_dec_buf, vb);
+
+	if ((buf->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) &&
+	    (buf->m.planes[0].bytesused == 0)) {
+		mtkbuf->lastframe = true;
+		mtk_v4l2_debug(1, "[%d] (%d) id=%d lastframe=%d (%d,%d, %d) vb=%p",
+			 ctx->id, buf->type, buf->index,
+			 mtkbuf->lastframe, buf->bytesused,
+			 buf->m.planes[0].bytesused, buf->length,
+			 vb);
+	}
+
+	return v4l2_m2m_qbuf(file, ctx->m2m_ctx, buf);
+}
+
+static int vidioc_vdec_dqbuf(struct file *file, void *priv,
+			     struct v4l2_buffer *buf)
+{
+	struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
+
+	if (ctx->state == MTK_STATE_ABORT) {
+		mtk_v4l2_err("[%d] Call on DQBUF after unrecoverable error",
+				ctx->id);
+		return -EIO;
+	}
+
+	return v4l2_m2m_dqbuf(file, ctx->m2m_ctx, buf);
+}
+
+static int vidioc_vdec_querycap(struct file *file, void *priv,
+				struct v4l2_capability *cap)
+{
+	strlcpy(cap->driver, MTK_VCODEC_DEC_NAME, sizeof(cap->driver));
+	strlcpy(cap->bus_info, MTK_PLATFORM_STR, sizeof(cap->bus_info));
+	strlcpy(cap->card, MTK_PLATFORM_STR, sizeof(cap->card));
+
+	return 0;
+}
+
+static int vidioc_vdec_subscribe_evt(struct v4l2_fh *fh,
+				     const struct v4l2_event_subscription *sub)
+{
+	switch (sub->type) {
+	case V4L2_EVENT_EOS:
+		return v4l2_event_subscribe(fh, sub, 2, NULL);
+	case V4L2_EVENT_SOURCE_CHANGE:
+		return v4l2_src_change_event_subscribe(fh, sub);
+	default:
+		return v4l2_ctrl_subscribe_event(fh, sub);
+	}
+}
+
+static int vidioc_try_fmt_vid_cap_mplane(struct file *file, void *priv,
+				struct v4l2_format *f)
+{
+	struct mtk_video_fmt *fmt;
+
+	fmt = mtk_vdec_find_format(f);
+	if (!fmt) {
+		f->fmt.pix.pixelformat = mtk_video_formats[CAP_FMT_IDX].fourcc;
+		fmt = mtk_vdec_find_format(f);
+	}
+
+	return 0;
+}
+
+static int vidioc_try_fmt_vid_out_mplane(struct file *file, void *priv,
+				struct v4l2_format *f)
+{
+	struct v4l2_pix_format_mplane *pix_fmt_mp = &f->fmt.pix_mp;
+	struct mtk_video_fmt *fmt;
+
+	fmt = mtk_vdec_find_format(f);
+	if (!fmt) {
+		f->fmt.pix.pixelformat = mtk_video_formats[OUT_FMT_IDX].fourcc;
+		fmt = mtk_vdec_find_format(f);
+	}
+
+	if (pix_fmt_mp->plane_fmt[0].sizeimage == 0) {
+		mtk_v4l2_err("sizeimage of output format must be given");
+		return -EINVAL;
+	}
+
+	return 0;
+}
+
+static int vidioc_vdec_g_selection(struct file *file, void *priv,
+			struct v4l2_selection *s)
+{
+	struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
+
+	if (V4L2_TYPE_IS_OUTPUT(s->type))
+		return -EINVAL;
+
+	if (s->target != V4L2_SEL_TGT_CROP)
+		return -EINVAL;
+
+	if (ctx->state < MTK_STATE_HEADER)
+		return -EINVAL;
+
+	if ((ctx->q_data[MTK_Q_DATA_SRC].fmt->fourcc == V4L2_PIX_FMT_H264) ||
+	    (ctx->q_data[MTK_Q_DATA_SRC].fmt->fourcc == V4L2_PIX_FMT_VP8) ||
+	    (ctx->q_data[MTK_Q_DATA_SRC].fmt->fourcc == V4L2_PIX_FMT_VP9)) {
+
+		if (vdec_if_get_param(ctx, GET_PARAM_CROP_INFO, &(s->r))) {
+			mtk_v4l2_debug(2,
+					"[%d]Error!! Cannot get param : GET_PARAM_CROP_INFO ERR",
+					ctx->id);
+			s->r.left = 0;
+			s->r.top = 0;
+			s->r.width = ctx->picinfo.buf_w;
+			s->r.height = ctx->picinfo.buf_h;
+		}
+		mtk_v4l2_debug(2, "Cropping info: l=%d t=%d w=%d h=%d",
+				s->r.left, s->r.top, s->r.width,
+				s->r.height);
+	} else {
+		s->r.left = 0;
+		s->r.top = 0;
+		s->r.width = ctx->picinfo.pic_w;
+		s->r.height = ctx->picinfo.pic_h;
+		mtk_v4l2_debug(2, "Cropping info: w=%d h=%d fw=%d fh=%d",
+				s->r.width, s->r.height, ctx->picinfo.buf_w,
+				ctx->picinfo.buf_h);
+	}
+	return 0;
+}
+
+static int vidioc_vdec_s_fmt(struct file *file, void *priv,
+			     struct v4l2_format *f)
+{
+	struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
+	struct v4l2_pix_format_mplane *pix_mp;
+	struct mtk_q_data *q_data;
+	int ret = 0;
+	struct mtk_video_fmt *fmt;
+	struct vb2_queue *vq;
+
+	mtk_v4l2_debug(3, "[%d]", ctx->id);
+
+	fmt = mtk_vdec_find_format(f);
+	if (fmt == NULL) {
+		mtk_v4l2_err("vidioc_vdec_s_fmt fail");
+		return ret;
+	}
+
+	pix_mp = &f->fmt.pix_mp;
+	if ((f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) &&
+	    vb2_is_busy(&ctx->m2m_ctx->out_q_ctx.q)) {
+		mtk_v4l2_err("out_q_ctx buffers already requested");
+		ret = -EBUSY;
+	}
+
+	if ((f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) &&
+	    vb2_is_busy(&ctx->m2m_ctx->cap_q_ctx.q)) {
+		mtk_v4l2_err("cap_q_ctx buffers already requested");
+		ret = -EBUSY;
+	}
+
+	vq = v4l2_m2m_get_vq(ctx->m2m_ctx, f->type);
+
+	q_data = mtk_vdec_get_q_data(ctx, f->type);
+	if (!q_data)
+		return -EINVAL;
+
+	q_data->fmt = fmt;
+	if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
+		q_data->sizeimage[0] = pix_mp->plane_fmt[0].sizeimage;
+		pix_mp->plane_fmt[0].bytesperline = 0;
+		pix_mp->width = 0;
+		pix_mp->height = 0;
+
+		if (ctx->state == MTK_STATE_FREE) {
+			ret = vdec_if_init(ctx, q_data->fmt->fourcc);
+			if (ret) {
+				mtk_v4l2_err("[%d]: vdec_if_init() fail ret=%d",
+					ctx->id, ret);
+				return -EINVAL;
+			}
+			ctx->state = MTK_STATE_INIT;
+		}
+	}
+
+	return 0;
+}
+
+static int vidioc_enum_framesizes(struct file *file, void *priv,
+				struct v4l2_frmsizeenum *fsize)
+{
+	int i = 0;
+	struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
+
+	if (fsize->index != 0)
+		return -EINVAL;
+
+	for (i = 0; i < NUM_SUPPORTED_FRAMESIZE; ++i) {
+		if (fsize->pixel_format != mtk_vdec_framesizes[i].fourcc)
+			continue;
+
+		fsize->type = V4L2_FRMSIZE_TYPE_STEPWISE;
+		fsize->stepwise = mtk_vdec_framesizes[i].stepwise;
+		if (!(ctx->dev->dec_capability &
+				VCODEC_CAPABILITY_4K_DISABLED)) {
+			mtk_v4l2_debug(3, "4K is enabled");
+			fsize->stepwise.max_width =
+					VCODEC_DEC_4K_CODED_WIDTH;
+			fsize->stepwise.max_height =
+					VCODEC_DEC_4K_CODED_HEIGHT;
+		}
+		mtk_v4l2_debug(1, "%x, %d %d %d %d %d %d",
+				ctx->dev->dec_capability,
+				fsize->stepwise.min_width,
+				fsize->stepwise.max_width,
+				fsize->stepwise.step_width,
+				fsize->stepwise.min_height,
+				fsize->stepwise.max_height,
+				fsize->stepwise.step_height);
+		return 0;
+	}
+
+	return -EINVAL;
+}
+
+static int vidioc_enum_fmt(struct v4l2_fmtdesc *f, bool output_queue)
+{
+	struct mtk_video_fmt *fmt;
+	int i, j = 0;
+
+	for (i = 0; i < NUM_FORMATS; i++) {
+		if (output_queue && (mtk_video_formats[i].type != MTK_FMT_DEC))
+			continue;
+		if (!output_queue &&
+			(mtk_video_formats[i].type != MTK_FMT_FRAME))
+			continue;
+
+		if (j == f->index)
+			break;
+		++j;
+	}
+
+	if (i == NUM_FORMATS)
+		return -EINVAL;
+
+	fmt = &mtk_video_formats[i];
+	f->pixelformat = fmt->fourcc;
+
+	return 0;
+}
+
+static int vidioc_vdec_enum_fmt_vid_cap_mplane(struct file *file, void *pirv,
+					       struct v4l2_fmtdesc *f)
+{
+	return vidioc_enum_fmt(f, false);
+}
+
+static int vidioc_vdec_enum_fmt_vid_out_mplane(struct file *file, void *priv,
+					       struct v4l2_fmtdesc *f)
+{
+	return vidioc_enum_fmt(f, true);
+}
+
+static int vidioc_vdec_g_fmt(struct file *file, void *priv,
+			     struct v4l2_format *f)
+{
+	struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
+	struct v4l2_pix_format_mplane *pix_mp = &f->fmt.pix_mp;
+	struct vb2_queue *vq;
+	struct mtk_q_data *q_data;
+
+	vq = v4l2_m2m_get_vq(ctx->m2m_ctx, f->type);
+	if (!vq) {
+		mtk_v4l2_err("no vb2 queue for type=%d",
+			f->type);
+		return -EINVAL;
+	}
+
+	q_data = mtk_vdec_get_q_data(ctx, f->type);
+
+	if ((f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) &&
+	    (ctx->state >= MTK_STATE_HEADER)) {
+		/* Until STREAMOFF is called on the CAPTURE queue
+		 * (acknowledging the event), the driver operates as if
+		 * the resolution hasn't changed yet.
+		 * So we just return picinfo yet, and update picinfo in
+		 * stop_streaming hook function
+		 */
+		q_data->sizeimage[0] = ctx->picinfo.y_bs_sz +
+					ctx->picinfo.y_len_sz;
+		q_data->sizeimage[1] = ctx->picinfo.c_bs_sz +
+					ctx->picinfo.c_len_sz;
+		q_data->bytesperline[0] = ctx->last_decoded_picinfo.buf_w;
+		q_data->bytesperline[1] = ctx->last_decoded_picinfo.buf_w;
+		q_data->coded_width = ctx->picinfo.buf_w;
+		q_data->coded_height = ctx->picinfo.buf_h;
+
+		/*
+		 * Width and height are set to the dimensions
+		 * of the movie, the buffer is bigger and
+		 * further processing stages should crop to this
+		 * rectangle.
+		 */
+		pix_mp->width = q_data->coded_width;
+		pix_mp->height = q_data->coded_height;
+		pix_mp->field = V4L2_FIELD_NONE;
+
+		/*
+		 * Set pixelformat to the format in which mt vcodec
+		 * outputs the decoded frame
+		 */
+		pix_mp->num_planes = q_data->fmt->num_planes;
+		pix_mp->pixelformat = q_data->fmt->fourcc;
+		pix_mp->plane_fmt[0].bytesperline = q_data->bytesperline[0];
+		pix_mp->plane_fmt[0].sizeimage = q_data->sizeimage[0];
+		pix_mp->plane_fmt[1].bytesperline = q_data->bytesperline[1];
+		pix_mp->plane_fmt[1].sizeimage = q_data->sizeimage[1];
+
+	} else if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
+		/*
+		 * This is run on OUTPUT
+		 * The buffer contains compressed image
+		 * so width and height have no meaning.
+		 * Assign value here to pass v4l2-compliance test
+		 */
+		pix_mp->width = q_data->visible_width;
+		pix_mp->height = q_data->visible_height;
+		pix_mp->field = V4L2_FIELD_NONE;
+		pix_mp->plane_fmt[0].bytesperline = q_data->bytesperline[0];
+		pix_mp->plane_fmt[0].sizeimage = q_data->sizeimage[0];
+		pix_mp->pixelformat = q_data->fmt->fourcc;
+		pix_mp->num_planes = q_data->fmt->num_planes;
+	} else {
+		pix_mp->width = q_data->coded_width;
+		pix_mp->height = q_data->coded_height;
+		pix_mp->field = V4L2_FIELD_NONE;
+		pix_mp->num_planes = q_data->fmt->num_planes;
+		pix_mp->pixelformat = q_data->fmt->fourcc;
+		pix_mp->plane_fmt[0].bytesperline = q_data->bytesperline[0];
+		pix_mp->plane_fmt[0].sizeimage = q_data->sizeimage[0];
+		pix_mp->plane_fmt[1].bytesperline = q_data->bytesperline[1];
+		pix_mp->plane_fmt[1].sizeimage = q_data->sizeimage[1];
+
+		mtk_v4l2_debug(1, "[%d] state=%d Format could not be read %d, not ready yet!",
+				ctx->id, ctx->state, f->type);
+		return -EINVAL;
+	}
+
+	return 0;
+}
+
+static int vb2ops_vdec_queue_setup(struct vb2_queue *vq,
+				unsigned int *nbuffers,
+				unsigned int *nplanes,
+				unsigned int sizes[], void *alloc_ctxs[])
+{
+	struct mtk_vcodec_ctx *ctx = vb2_get_drv_priv(vq);
+	struct mtk_q_data *q_data;
+	unsigned int i;
+
+	q_data = mtk_vdec_get_q_data(ctx, vq->type);
+
+	if (q_data == NULL) {
+		mtk_v4l2_err("vq->type=%d err\n", vq->type);
+		return -EINVAL;
+	}
+
+	if (*nplanes) {
+		for (i = 0; i < *nplanes; i++) {
+			if (sizes[i] < q_data->sizeimage[i])
+				return -EINVAL;
+			alloc_ctxs[i] = ctx->dev->alloc_ctx;
+		}
+	} else {
+
+		if (vq->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
+			if (ctx->state < MTK_STATE_HEADER) {
+				mtk_v4l2_err("State not ready");
+				return -EINVAL;
+			}
+			/*
+			 * Output plane count is 2 - one for Y and one for CbCr
+			 */
+			*nplanes = 2;
+		} else {
+			*nplanes = 1;
+		}
+
+		for (i = 0; i < *nplanes; i++) {
+			sizes[i] = q_data->sizeimage[i];
+			alloc_ctxs[i] = ctx->dev->alloc_ctx;
+		}
+	}
+
+	mtk_v4l2_debug(1,
+			"[%d]\t type = %d, get %d plane(s), %d buffer(s) of size 0x%x 0x%x ",
+			ctx->id, vq->type, *nplanes, *nbuffers,
+			sizes[0], sizes[1]);
+
+	return 0;
+}
+
+static int vb2ops_vdec_buf_prepare(struct vb2_buffer *vb)
+{
+	struct mtk_vcodec_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue);
+	struct mtk_q_data *q_data;
+	int i;
+
+	mtk_v4l2_debug(3, "[%d] (%d) id=%d",
+			ctx->id, vb->vb2_queue->type, vb->index);
+
+	q_data = mtk_vdec_get_q_data(ctx, vb->vb2_queue->type);
+
+	for (i = 0; i < q_data->fmt->num_planes; i++) {
+		if (vb2_plane_size(vb, i) > q_data->sizeimage[i]) {
+			mtk_v4l2_err("data will not fit into plane %d (%lu < %d)",
+				i, vb2_plane_size(vb, i),
+				q_data->sizeimage[i]);
+		}
+	}
+
+	return 0;
+}
+
+static void vb2ops_vdec_buf_queue(struct vb2_buffer *vb)
+{
+	struct mtk_vcodec_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue);
+	struct vb2_v4l2_buffer *vb2_v4l2 = container_of(vb,
+				struct vb2_v4l2_buffer, vb2_buf);
+	struct mtk_video_dec_buf *buf = container_of(vb2_v4l2,
+				struct mtk_video_dec_buf, vb);
+
+	mtk_v4l2_debug(3, "[%d] (%d) id=%d, vb=%p",
+			ctx->id, vb->vb2_queue->type,
+			vb->index, vb);
+	/*
+	 * check if this buffer is ready to be used after decode
+	 */
+	if (vb->vb2_queue->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
+		struct vb2_buffer *src_buf;
+		struct mtk_vcodec_mem src_mem;
+		bool res_chg = false;
+		int ret = 0;
+		unsigned int dpbsize = 1;
+
+		v4l2_m2m_buf_queue(ctx->m2m_ctx, vb2_v4l2);
+
+		if (ctx->state == MTK_STATE_INIT) {
+			src_buf = v4l2_m2m_next_src_buf(ctx->m2m_ctx);
+			if (!src_buf) {
+				mtk_v4l2_err("No src buffer");
+				return;
+			}
+
+			src_mem.va	= vb2_plane_vaddr(src_buf, 0);
+			src_mem.dma_addr = vb2_dma_contig_plane_dma_addr(
+						src_buf,
+						0);
+			src_mem.size = (size_t)src_buf->planes[0].bytesused;
+			mtk_v4l2_debug(2,
+					"[%d] buf id=%d va=%p dma=%llx size=%zu",
+					ctx->id, src_buf->index,
+					src_mem.va, (u64)src_mem.dma_addr,
+					src_mem.size);
+
+			ret = vdec_if_decode(ctx, &src_mem, NULL, &res_chg);
+			if (ret) {
+				/* fb == NULL menas to parse SPS/PPS header or
+				 * resolution info in src_mem. Decode can fail
+				 * if there is no SPS header or picture info
+				 * in bs
+				 */
+				src_buf = v4l2_m2m_src_buf_remove(ctx->m2m_ctx);
+				v4l2_m2m_buf_done(to_vb2_v4l2_buffer(src_buf),
+						VB2_BUF_STATE_DONE);
+				mtk_v4l2_debug(0, "[%d] vdec_if_decode() src_buf=%d, size=%zu, fail=%d",
+						ctx->id, src_buf->index,
+						src_mem.size, ret);
+				return;
+			}
+
+			if (vdec_if_get_param(ctx,
+						GET_PARAM_PIC_INFO,
+						&ctx->picinfo)) {
+				mtk_v4l2_err("[%d]Error!! Cannot get param : GET_PARAM_PICTURE_INFO ERR",
+						ctx->id);
+				return;
+			}
+
+			ctx->last_decoded_picinfo = ctx->picinfo;
+			ctx->q_data[MTK_Q_DATA_DST].sizeimage[0] =
+						ctx->picinfo.y_bs_sz +
+						ctx->picinfo.y_len_sz;
+			ctx->q_data[MTK_Q_DATA_DST].bytesperline[0] =
+						ctx->picinfo.buf_w;
+			ctx->q_data[MTK_Q_DATA_DST].sizeimage[1] =
+						ctx->picinfo.c_bs_sz +
+						ctx->picinfo.c_len_sz;
+			ctx->q_data[MTK_Q_DATA_DST].bytesperline[1] =
+						ctx->picinfo.buf_w;
+			mtk_v4l2_debug(2, "[%d] vdec_if_init() OK wxh=%dx%d pic wxh=%dx%d sz[0]=0x%x sz[1]=0x%x",
+				ctx->id,
+				ctx->picinfo.buf_w, ctx->picinfo.buf_h,
+				ctx->picinfo.pic_w, ctx->picinfo.pic_h,
+				ctx->q_data[MTK_Q_DATA_DST].sizeimage[0],
+				ctx->q_data[MTK_Q_DATA_DST].sizeimage[1]);
+
+			ret = vdec_if_get_param(ctx, GET_PARAM_DPB_SIZE,
+						&dpbsize);
+			if (dpbsize == 0) {
+				mtk_v4l2_err("[%d] vdec_if_get_param()  GET_PARAM_DPB_SIZE fail=%d",
+						ctx->id, ret);
+			}
+			ctx->dpb_size = dpbsize;
+			ctx->state = MTK_STATE_HEADER;
+			mtk_v4l2_debug(1, "[%d] dpbsize=%d", ctx->id,
+					ctx->dpb_size);
+		} else {
+			mtk_v4l2_debug(3, "[%d] already init driver %d",
+					ctx->id, ctx->state);
+		}
+	} else {
+		mutex_lock(&ctx->lock);
+		if (buf->used == false) {
+			v4l2_m2m_buf_queue(ctx->m2m_ctx,
+					to_vb2_v4l2_buffer(vb));
+			buf->queued_in_vb2 = true;
+			buf->queued_in_v4l2 = true;
+			buf->ready_to_display = false;
+		} else {
+			buf->queued_in_vb2 = false;
+			buf->queued_in_v4l2 = true;
+			buf->ready_to_display = false;
+		}
+		mutex_unlock(&ctx->lock);
+	}
+}
+
+static void vb2ops_vdec_buf_finish(struct vb2_buffer *vb)
+{
+	struct mtk_vcodec_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue);
+
+	if (vb->vb2_queue->type ==
+		V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
+		struct vb2_v4l2_buffer *vb2_v4l2;
+		struct mtk_video_dec_buf *buf;
+
+		vb2_v4l2 = container_of(vb, struct vb2_v4l2_buffer, vb2_buf);
+		buf = container_of(vb2_v4l2, struct mtk_video_dec_buf, vb);
+		mutex_lock(&ctx->lock);
+		buf->queued_in_v4l2 = false;
+		buf->queued_in_vb2 = false;
+		mutex_unlock(&ctx->lock);
+	}
+}
+
+static int vb2ops_vdec_buf_init(struct vb2_buffer *vb)
+{
+	struct vb2_v4l2_buffer *vb2_v4l2 = container_of(vb,
+					struct vb2_v4l2_buffer, vb2_buf);
+	struct mtk_video_dec_buf *buf = container_of(vb2_v4l2,
+					struct mtk_video_dec_buf, vb);
+
+	if (vb->vb2_queue->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
+		buf->used = false;
+		buf->ready_to_display = false;
+		buf->queued_in_v4l2 = false;
+	} else {
+		buf->lastframe = false;
+	}
+
+	return 0;
+}
+
+static int vb2ops_vdec_start_streaming(struct vb2_queue *q, unsigned int count)
+{
+	struct mtk_vcodec_ctx *ctx = vb2_get_drv_priv(q);
+
+	if (ctx->state == MTK_STATE_FLUSH)
+		ctx->state = MTK_STATE_HEADER;
+
+	return 0;
+}
+
+static void vb2ops_vdec_stop_streaming(struct vb2_queue *q)
+{
+	struct vb2_buffer *src_buf = NULL, *dst_buf = NULL;
+	struct mtk_vcodec_ctx *ctx = vb2_get_drv_priv(q);
+
+	mtk_v4l2_debug(3, "[%d] (%d) state=(%x) ctx->decoded_frame_cnt=%d",
+			ctx->id, q->type, ctx->state, ctx->decoded_frame_cnt);
+
+	if (q->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
+		while ((src_buf = v4l2_m2m_src_buf_remove(ctx->m2m_ctx)))
+			v4l2_m2m_buf_done(to_vb2_v4l2_buffer(src_buf),
+					VB2_BUF_STATE_ERROR);
+
+	} else if (q->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
+		if (ctx->state >= MTK_STATE_HEADER) {
+
+			/* Until STREAMOFF is called on the CAPTURE queue
+			 * (acknowledging the event), the driver operates
+			 * as if the resolution hasn't changed yet, i.e.
+			 * VIDIOC_G_FMT< etc. return previous resolution.
+			 * So we update picinfo here
+			 */
+			ctx->picinfo = ctx->last_decoded_picinfo;
+
+			mtk_v4l2_debug(2,
+				"[%d]-> new(%d,%d), old(%d,%d), real(%d,%d)",
+				ctx->id, ctx->last_decoded_picinfo.pic_w,
+				ctx->last_decoded_picinfo.pic_h,
+				ctx->picinfo.pic_w, ctx->picinfo.pic_h,
+				ctx->last_decoded_picinfo.buf_w,
+				ctx->last_decoded_picinfo.buf_h);
+
+			mtk_vdec_flush_decoder(ctx);
+		}
+		ctx->state = MTK_STATE_FLUSH;
+
+		while ((dst_buf = v4l2_m2m_dst_buf_remove(ctx->m2m_ctx))) {
+			vb2_set_plane_payload(dst_buf, 0, 0);
+			vb2_set_plane_payload(dst_buf, 1, 0);
+			v4l2_m2m_buf_done(to_vb2_v4l2_buffer(dst_buf),
+					VB2_BUF_STATE_ERROR);
+		}
+	}
+}
+
+static void m2mops_vdec_device_run(void *priv)
+{
+	struct mtk_vcodec_ctx *ctx = priv;
+	struct mtk_vcodec_dev *dev = ctx->dev;
+
+	queue_work(dev->decode_workqueue, &ctx->decode_work);
+}
+
+static int m2mops_vdec_job_ready(void *m2m_priv)
+{
+	struct mtk_vcodec_ctx *ctx = m2m_priv;
+
+	mtk_v4l2_debug(3, "[%d]", ctx->id);
+
+	if (ctx->state == MTK_STATE_ABORT)
+		return 0;
+
+	if ((ctx->last_decoded_picinfo.pic_w != ctx->picinfo.pic_w) ||
+	    (ctx->last_decoded_picinfo.pic_h != ctx->picinfo.pic_h))
+		return 0;
+
+	if (ctx->state != MTK_STATE_HEADER)
+		return 0;
+
+	return 1;
+}
+
+static void m2mops_vdec_job_abort(void *priv)
+{
+	struct mtk_vcodec_ctx *ctx = priv;
+
+	ctx->state = MTK_STATE_ABORT;
+}
+
+static int mtk_vdec_g_v_ctrl(struct v4l2_ctrl *ctrl)
+{
+	struct mtk_vcodec_ctx *ctx = ctrl_to_ctx(ctrl);
+	int ret = 0;
+
+	switch (ctrl->id) {
+	case V4L2_CID_MIN_BUFFERS_FOR_CAPTURE:
+		if (ctx->state >= MTK_STATE_HEADER) {
+			ctrl->val = ctx->dpb_size;
+		} else {
+			mtk_v4l2_err("Seqinfo not ready");
+			ctrl->val = 1;
+			ret = -EAGAIN;
+		}
+		break;
+	default:
+		ret = -EINVAL;
+	}
+	return ret;
+}
+
+static const struct v4l2_ctrl_ops mtk_vcodec_dec_ctrl_ops = {
+	.g_volatile_ctrl = mtk_vdec_g_v_ctrl,
+};
+
+int mtk_vcodec_dec_ctrls_setup(struct mtk_vcodec_ctx *ctx)
+{
+	struct v4l2_ctrl *ctrl;
+
+	v4l2_ctrl_handler_init(&ctx->ctrl_hdl, 1);
+
+	ctrl = v4l2_ctrl_new_std(&ctx->ctrl_hdl,
+				&mtk_vcodec_dec_ctrl_ops,
+				V4L2_CID_MIN_BUFFERS_FOR_CAPTURE,
+				1, 32, 1, 1);
+	ctrl->flags |= V4L2_CTRL_FLAG_VOLATILE;
+
+	if (ctx->ctrl_hdl.error) {
+		mtk_v4l2_err("Adding control failed %d",
+				ctx->ctrl_hdl.error);
+		return ctx->ctrl_hdl.error;
+	}
+
+	v4l2_ctrl_handler_setup(&ctx->ctrl_hdl);
+	return 0;
+}
+
+static void m2mops_vdec_lock(void *m2m_priv)
+{
+	struct mtk_vcodec_ctx *ctx = m2m_priv;
+
+	mtk_v4l2_debug(3, "[%d]", ctx->id);
+	mutex_lock(&ctx->dev->dev_mutex);
+}
+
+static void m2mops_vdec_unlock(void *m2m_priv)
+{
+	struct mtk_vcodec_ctx *ctx = m2m_priv;
+
+	mtk_v4l2_debug(3, "[%d]", ctx->id);
+	mutex_unlock(&ctx->dev->dev_mutex);
+}
+
+const struct v4l2_m2m_ops mtk_vdec_m2m_ops = {
+	.device_run	= m2mops_vdec_device_run,
+	.job_ready	= m2mops_vdec_job_ready,
+	.job_abort	= m2mops_vdec_job_abort,
+	.lock		= m2mops_vdec_lock,
+	.unlock		= m2mops_vdec_unlock,
+};
+
+static const struct vb2_ops mtk_vdec_vb2_ops = {
+	.queue_setup	= vb2ops_vdec_queue_setup,
+	.buf_prepare	= vb2ops_vdec_buf_prepare,
+	.buf_queue	= vb2ops_vdec_buf_queue,
+	.wait_prepare	= vb2_ops_wait_prepare,
+	.wait_finish	= vb2_ops_wait_finish,
+	.buf_init	= vb2ops_vdec_buf_init,
+	.buf_finish	= vb2ops_vdec_buf_finish,
+	.start_streaming	= vb2ops_vdec_start_streaming,
+	.stop_streaming	= vb2ops_vdec_stop_streaming,
+};
+
+const struct v4l2_ioctl_ops mtk_vdec_ioctl_ops = {
+	.vidioc_streamon	= v4l2_m2m_ioctl_streamon,
+	.vidioc_streamoff	= v4l2_m2m_ioctl_streamoff,
+	.vidioc_reqbufs		= v4l2_m2m_ioctl_reqbufs,
+	.vidioc_querybuf	= v4l2_m2m_ioctl_querybuf,
+	.vidioc_expbuf		= v4l2_m2m_ioctl_expbuf,
+
+	.vidioc_qbuf		= vidioc_vdec_qbuf,
+	.vidioc_dqbuf		= vidioc_vdec_dqbuf,
+
+	.vidioc_try_fmt_vid_cap_mplane	= vidioc_try_fmt_vid_cap_mplane,
+	.vidioc_try_fmt_vid_out_mplane	= vidioc_try_fmt_vid_out_mplane,
+
+	.vidioc_s_fmt_vid_cap_mplane	= vidioc_vdec_s_fmt,
+	.vidioc_s_fmt_vid_out_mplane	= vidioc_vdec_s_fmt,
+	.vidioc_g_fmt_vid_cap_mplane	= vidioc_vdec_g_fmt,
+	.vidioc_g_fmt_vid_out_mplane	= vidioc_vdec_g_fmt,
+
+	.vidioc_enum_fmt_vid_cap_mplane	= vidioc_vdec_enum_fmt_vid_cap_mplane,
+	.vidioc_enum_fmt_vid_out_mplane	= vidioc_vdec_enum_fmt_vid_out_mplane,
+	.vidioc_enum_framesizes	= vidioc_enum_framesizes,
+
+	.vidioc_querycap		= vidioc_vdec_querycap,
+	.vidioc_subscribe_event		= vidioc_vdec_subscribe_evt,
+	.vidioc_unsubscribe_event	= v4l2_event_unsubscribe,
+	.vidioc_g_selection             = vidioc_vdec_g_selection,
+};
+
+int mtk_vcodec_dec_queue_init(void *priv, struct vb2_queue *src_vq,
+			   struct vb2_queue *dst_vq)
+{
+	struct mtk_vcodec_ctx *ctx = priv;
+	int ret = 0;
+
+	mtk_v4l2_debug(3, "[%d]", ctx->id);
+
+	src_vq->type		= V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE;
+	src_vq->io_modes	= VB2_DMABUF | VB2_MMAP;
+	src_vq->drv_priv	= ctx;
+	src_vq->buf_struct_size = sizeof(struct mtk_video_dec_buf);
+	src_vq->ops		= &mtk_vdec_vb2_ops;
+	src_vq->mem_ops		= &vb2_dma_contig_memops;
+	src_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
+	src_vq->lock		= &ctx->dev->dev_mutex;
+
+	ret = vb2_queue_init(src_vq);
+	if (ret) {
+		mtk_v4l2_err("Failed to initialize videobuf2 queue(output)");
+		return ret;
+	}
+	dst_vq->type		= V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
+	dst_vq->io_modes	= VB2_DMABUF | VB2_MMAP;
+	dst_vq->drv_priv	= ctx;
+	dst_vq->buf_struct_size = sizeof(struct mtk_video_dec_buf);
+	dst_vq->ops		= &mtk_vdec_vb2_ops;
+	dst_vq->mem_ops		= &vb2_dma_contig_memops;
+	dst_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
+	dst_vq->lock		= &ctx->dev->dev_mutex;
+
+	ret = vb2_queue_init(dst_vq);
+	if (ret) {
+		vb2_queue_release(src_vq);
+		mtk_v4l2_err("Failed to initialize videobuf2 queue(capture)");
+	}
+
+	return ret;
+}
+
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.h
new file mode 100644
index 0000000..595c244
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.h
@@ -0,0 +1,88 @@
+/*
+* Copyright (c) 2016 MediaTek Inc.
+* Author: PC Chen <pc.chen@mediatek.com>
+*         Tiffany Lin <tiffany.lin@mediatek.com>
+*
+* This program is free software; you can redistribute it and/or modify
+* it under the terms of the GNU General Public License version 2 as
+* published by the Free Software Foundation.
+*
+* This program is distributed in the hope that it will be useful,
+* but WITHOUT ANY WARRANTY; without even the implied warranty of
+* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+* GNU General Public License for more details.
+*/
+
+#ifndef _MTK_VCODEC_DEC_H_
+#define _MTK_VCODEC_DEC_H_
+
+#include <media/videobuf2-core.h>
+#include <media/videobuf2-v4l2.h>
+
+#define VCODEC_CAPABILITY_4K_DISABLED	0x10
+#define VCODEC_DEC_4K_CODED_WIDTH	4096U
+#define VCODEC_DEC_4K_CODED_HEIGHT	2304U
+#define MTK_VDEC_MAX_W	2048U
+#define MTK_VDEC_MAX_H	1088U
+
+#define MTK_VDEC_IRQ_STATUS_DEC_SUCCESS        0x10000
+
+/**
+ * struct vdec_fb  - decoder frame buffer
+ * @base_y	: Y plane memory info
+ * @base_c	: C plane memory info
+ * @status      : frame buffer status (vdec_fb_status)
+ */
+struct vdec_fb {
+	struct mtk_vcodec_mem	base_y;
+	struct mtk_vcodec_mem	base_c;
+	unsigned int	status;
+};
+
+/**
+ * struct mtk_video_dec_buf - Private data related to each VB2 buffer.
+ * @b:		VB2 buffer
+ * @list:	link list
+ * @used:	Capture buffer contain decoded frame data and keep in
+ *			codec data structure
+ * @ready_to_display:	Capture buffer not display yet
+ * @queued_in_vb2:	Capture buffer is queue in vb2
+ * @queued_in_v4l2:	Capture buffer is in v4l2 driver, but not in vb2
+ *			queue yet
+ * @lastframe:		Intput buffer is last buffer - EOS
+ * @frame_buffer:	Decode status, and buffer information of Capture buffer
+ *
+ * Note : These status information help us track and debug buffer state
+ */
+struct mtk_video_dec_buf {
+	struct vb2_v4l2_buffer	vb;
+	struct list_head	list;
+
+	bool	used;
+	bool	ready_to_display;
+	bool	queued_in_vb2;
+	bool	queued_in_v4l2;
+	bool	lastframe;
+	struct vdec_fb	frame_buffer;
+};
+
+extern const struct v4l2_ioctl_ops mtk_vdec_ioctl_ops;
+extern const struct v4l2_m2m_ops mtk_vdec_m2m_ops;
+
+
+/*
+ * mtk_vdec_lock/mtk_vdec_unlock are for ctx instance to
+ * get/release lock before/after access decoder hw.
+ * mtk_vdec_lock get decoder hw lock and set curr_ctx
+ * to ctx instance that get lock
+ */
+void mtk_vdec_unlock(struct mtk_vcodec_ctx *ctx);
+void mtk_vdec_lock(struct mtk_vcodec_ctx *ctx);
+int mtk_vcodec_dec_queue_init(void *priv, struct vb2_queue *src_vq,
+			   struct vb2_queue *dst_vq);
+void mtk_vcodec_dec_set_default_params(struct mtk_vcodec_ctx *ctx);
+void mtk_vcodec_vdec_release(struct mtk_vcodec_ctx *ctx);
+int mtk_vcodec_dec_ctrls_setup(struct mtk_vcodec_ctx *ctx);
+
+
+#endif /* _MTK_VCODEC_DEC_H_ */
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c
new file mode 100644
index 0000000..725d1e5
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c
@@ -0,0 +1,408 @@
+/*
+* Copyright (c) 2016 MediaTek Inc.
+* Author: PC Chen <pc.chen@mediatek.com>
+*         Tiffany Lin <tiffany.lin@mediatek.com>
+*
+* This program is free software; you can redistribute it and/or modify
+* it under the terms of the GNU General Public License version 2 as
+* published by the Free Software Foundation.
+*
+* This program is distributed in the hope that it will be useful,
+* but WITHOUT ANY WARRANTY; without even the implied warranty of
+* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+* GNU General Public License for more details.
+*/
+
+#include <linux/slab.h>
+#include <linux/interrupt.h>
+#include <linux/irq.h>
+#include <linux/module.h>
+#include <linux/of_device.h>
+#include <linux/of.h>
+#include <media/v4l2-event.h>
+#include <media/v4l2-mem2mem.h>
+#include <media/videobuf2-dma-contig.h>
+
+#include "mtk_vcodec_drv.h"
+#include "mtk_vcodec_dec.h"
+#include "mtk_vcodec_dec_pm.h"
+#include "mtk_vcodec_intr.h"
+#include "mtk_vcodec_util.h"
+#include "mtk_vpu.h"
+
+#define VDEC_HW_ACTIVE	0x10
+#define VDEC_IRQ_CFG	0x11
+#define VDEC_IRQ_CLR	0x10
+#define VDEC_IRQ_CFG_REG	0xa4
+
+module_param(mtk_v4l2_dbg_level, int, S_IRUGO | S_IWUSR);
+module_param(mtk_vcodec_dbg, bool, S_IRUGO | S_IWUSR);
+
+/* Wake up context wait_queue */
+static void wake_up_ctx(struct mtk_vcodec_ctx *ctx)
+{
+	ctx->int_cond = 1;
+	wake_up_interruptible(&ctx->queue);
+}
+
+static irqreturn_t mtk_vcodec_dec_irq_handler(int irq, void *priv)
+{
+	struct mtk_vcodec_dev *dev = priv;
+	struct mtk_vcodec_ctx *ctx;
+	u32 cg_status = 0;
+	unsigned int dec_done_status = 0;
+	void __iomem *vdec_misc_addr = dev->reg_base[VDEC_MISC] +
+					VDEC_IRQ_CFG_REG;
+
+	ctx = mtk_vcodec_get_curr_ctx(dev);
+
+	/* check if HW active or not */
+	cg_status = readl(dev->reg_base[0]);
+	if ((cg_status & VDEC_HW_ACTIVE) != 0) {
+		mtk_v4l2_err("DEC ISR, VDEC active is not 0x0 (0x%08x)",
+			     cg_status);
+		return IRQ_HANDLED;
+	}
+
+	dec_done_status = readl(vdec_misc_addr);
+	ctx->irq_status = dec_done_status;
+	if ((dec_done_status & MTK_VDEC_IRQ_STATUS_DEC_SUCCESS) !=
+		MTK_VDEC_IRQ_STATUS_DEC_SUCCESS)
+		return IRQ_HANDLED;
+
+	/* clear interrupt */
+	writel((readl(vdec_misc_addr) | VDEC_IRQ_CFG),
+		dev->reg_base[VDEC_MISC] + VDEC_IRQ_CFG_REG);
+	writel((readl(vdec_misc_addr) & ~VDEC_IRQ_CLR),
+		dev->reg_base[VDEC_MISC] + VDEC_IRQ_CFG_REG);
+
+	wake_up_ctx(ctx);
+
+	mtk_v4l2_debug(3,
+			"mtk_vcodec_dec_irq_handler :wake up ctx %d, dec_done_status=%x",
+			ctx->id, dec_done_status);
+
+	return IRQ_HANDLED;
+}
+
+static void mtk_vcodec_dec_reset_handler(void *priv)
+{
+	struct mtk_vcodec_dev *dev = priv;
+	struct mtk_vcodec_ctx *ctx;
+
+	mtk_v4l2_err("Watchdog timeout!!");
+
+	mutex_lock(&dev->dev_mutex);
+	list_for_each_entry(ctx, &dev->ctx_list, list) {
+		ctx->state = MTK_STATE_ABORT;
+		mtk_v4l2_debug(0, "[%d] Change to state MTK_STATE_ERROR",
+				ctx->id);
+	}
+	mutex_unlock(&dev->dev_mutex);
+}
+
+static int fops_vcodec_open(struct file *file)
+{
+	struct mtk_vcodec_dev *dev = video_drvdata(file);
+	struct mtk_vcodec_ctx *ctx = NULL;
+	int ret = 0;
+
+	ctx = kzalloc(sizeof(*ctx), GFP_KERNEL);
+	if (!ctx)
+		return -ENOMEM;
+
+	mutex_lock(&dev->dev_mutex);
+	ctx->id = dev->id_counter++;
+	v4l2_fh_init(&ctx->fh, video_devdata(file));
+	file->private_data = &ctx->fh;
+	v4l2_fh_add(&ctx->fh);
+	INIT_LIST_HEAD(&ctx->list);
+	ctx->dev = dev;
+	init_waitqueue_head(&ctx->queue);
+	mutex_init(&ctx->lock);
+
+	ctx->type = MTK_INST_DECODER;
+	ret = mtk_vcodec_dec_ctrls_setup(ctx);
+	if (ret) {
+		mtk_v4l2_err("Failed to setup mt vcodec controls");
+		goto err_ctrls_setup;
+	}
+	ctx->m2m_ctx = v4l2_m2m_ctx_init(dev->m2m_dev_dec, ctx,
+		&mtk_vcodec_dec_queue_init);
+	if (IS_ERR((__force void *)ctx->m2m_ctx)) {
+		ret = PTR_ERR((__force void *)ctx->m2m_ctx);
+		mtk_v4l2_err("Failed to v4l2_m2m_ctx_init() (%d)",
+			ret);
+		goto err_m2m_ctx_init;
+	}
+	mtk_vcodec_dec_set_default_params(ctx);
+
+	if (v4l2_fh_is_singular(&ctx->fh)) {
+		mtk_vcodec_dec_pw_on(&dev->pm);
+		/*
+		 * vpu_load_firmware checks if it was loaded already and
+		 * does nothing in that case
+		 */
+		ret = vpu_load_firmware(dev->vpu_plat_dev);
+		if (ret < 0) {
+			/*
+			 * Return 0 if downloading firmware successfully,
+			 * otherwise it is failed
+			 */
+			mtk_v4l2_err("vpu_load_firmware failed!");
+			goto err_load_fw;
+		}
+
+		dev->dec_capability =
+			vpu_get_vdec_hw_capa(dev->vpu_plat_dev);
+		mtk_v4l2_debug(0, "decoder capability %x", dev->dec_capability);
+	}
+
+	dev->num_instances++;
+	list_add(&ctx->list, &dev->ctx_list);
+
+	mutex_unlock(&dev->dev_mutex);
+	mtk_v4l2_debug(0, "%s decoder [%d]", dev_name(&dev->plat_dev->dev),
+			ctx->id);
+	return ret;
+
+	/* Deinit when failure occurred */
+err_load_fw:
+	v4l2_m2m_ctx_release(ctx->m2m_ctx);
+err_m2m_ctx_init:
+	v4l2_ctrl_handler_free(&ctx->ctrl_hdl);
+err_ctrls_setup:
+	v4l2_fh_del(&ctx->fh);
+	v4l2_fh_exit(&ctx->fh);
+	kfree(ctx);
+	mutex_unlock(&dev->dev_mutex);
+
+	return ret;
+}
+
+static int fops_vcodec_release(struct file *file)
+{
+	struct mtk_vcodec_dev *dev = video_drvdata(file);
+	struct mtk_vcodec_ctx *ctx = fh_to_ctx(file->private_data);
+
+	mtk_v4l2_debug(0, "[%d] decoder", ctx->id);
+	mutex_lock(&dev->dev_mutex);
+
+	mtk_vcodec_vdec_release(ctx);
+	if (v4l2_fh_is_singular(&ctx->fh))
+		mtk_vcodec_dec_pw_off(&dev->pm);
+
+	v4l2_fh_del(&ctx->fh);
+	v4l2_fh_exit(&ctx->fh);
+	v4l2_ctrl_handler_free(&ctx->ctrl_hdl);
+	v4l2_m2m_ctx_release(ctx->m2m_ctx);
+
+	list_del_init(&ctx->list);
+	dev->num_instances--;
+	kfree(ctx);
+	mutex_unlock(&dev->dev_mutex);
+	return 0;
+}
+
+static const struct v4l2_file_operations mtk_vcodec_fops = {
+	.owner		= THIS_MODULE,
+	.open		= fops_vcodec_open,
+	.release	= fops_vcodec_release,
+	.poll		= v4l2_m2m_fop_poll,
+	.unlocked_ioctl	= video_ioctl2,
+	.mmap		= v4l2_m2m_fop_mmap,
+};
+
+static int mtk_vcodec_probe(struct platform_device *pdev)
+{
+	struct mtk_vcodec_dev *dev;
+	struct video_device *vfd_dec;
+	struct resource *res;
+	int i, ret;
+	DEFINE_DMA_ATTRS(attrs);
+
+	dev = devm_kzalloc(&pdev->dev, sizeof(*dev), GFP_KERNEL);
+	if (!dev)
+		return -ENOMEM;
+
+	INIT_LIST_HEAD(&dev->ctx_list);
+	dev->plat_dev = pdev;
+
+	dev->vpu_plat_dev = vpu_get_plat_device(dev->plat_dev);
+	if (dev->vpu_plat_dev == NULL) {
+		mtk_v4l2_err("[VPU] vpu device in not ready");
+		return -EPROBE_DEFER;
+	}
+
+	vpu_wdt_reg_handler(dev->vpu_plat_dev, mtk_vcodec_dec_reset_handler,
+			dev, VPU_RST_DEC);
+
+	ret = mtk_vcodec_init_dec_pm(dev);
+	if (ret < 0) {
+		dev_err(&pdev->dev, "Failed to get mt vcodec clock source");
+		return ret;
+	}
+
+	for (i = 0; i < NUM_MAX_VDEC_REG_BASE; i++) {
+		res = platform_get_resource(pdev, IORESOURCE_MEM, i);
+		if (res == NULL) {
+			dev_err(&pdev->dev, "get memory resource failed.");
+			ret = -ENXIO;
+			goto err_res;
+		}
+		dev->reg_base[i] = devm_ioremap_resource(&pdev->dev, res);
+		if (IS_ERR((__force void *)dev->reg_base[i])) {
+			dev_err(&pdev->dev,
+				"devm_ioremap_resource %d failed.", i);
+			ret = PTR_ERR((__force void *)dev->reg_base);
+			goto err_res;
+		}
+		mtk_v4l2_debug(2, "reg[%d] base=%p", i, dev->reg_base[i]);
+	}
+
+	res = platform_get_resource(pdev, IORESOURCE_IRQ, 0);
+	if (res == NULL) {
+		dev_err(&pdev->dev, "failed to get irq resource");
+		ret = -ENOENT;
+		goto err_res;
+	}
+
+	dev->dec_irq = platform_get_irq(pdev, 0);
+	ret = devm_request_irq(&pdev->dev, dev->dec_irq,
+			mtk_vcodec_dec_irq_handler, 0, pdev->name, dev);
+	if (ret) {
+		dev_err(&pdev->dev, "Failed to install dev->dec_irq %d (%d)",
+			dev->dec_irq,
+			ret);
+		goto err_res;
+	}
+
+	disable_irq(dev->dec_irq);
+	mutex_init(&dev->dec_mutex);
+	mutex_init(&dev->dev_mutex);
+	spin_lock_init(&dev->irqlock);
+
+	snprintf(dev->v4l2_dev.name, sizeof(dev->v4l2_dev.name), "%s",
+		"[/MTK_V4L2_VDEC]");
+
+	ret = v4l2_device_register(&pdev->dev, &dev->v4l2_dev);
+	if (ret) {
+		mtk_v4l2_err("v4l2_device_register err=%d", ret);
+		goto err_res;
+	}
+
+	init_waitqueue_head(&dev->queue);
+
+	vfd_dec = video_device_alloc();
+	if (!vfd_dec) {
+		mtk_v4l2_err("Failed to allocate video device");
+		ret = -ENOMEM;
+		goto err_dec_alloc;
+	}
+	vfd_dec->fops		= &mtk_vcodec_fops;
+	vfd_dec->ioctl_ops	= &mtk_vdec_ioctl_ops;
+	vfd_dec->release	= video_device_release;
+	vfd_dec->lock		= &dev->dev_mutex;
+	vfd_dec->v4l2_dev	= &dev->v4l2_dev;
+	vfd_dec->vfl_dir	= VFL_DIR_M2M;
+	vfd_dec->device_caps	= V4L2_CAP_VIDEO_M2M_MPLANE |
+			V4L2_CAP_STREAMING;
+
+	snprintf(vfd_dec->name, sizeof(vfd_dec->name), "%s",
+		MTK_VCODEC_DEC_NAME);
+	video_set_drvdata(vfd_dec, dev);
+	dev->vfd_dec = vfd_dec;
+	platform_set_drvdata(pdev, dev);
+
+	dev->alloc_ctx = vb2_dma_contig_init_ctx(&pdev->dev);
+	if (IS_ERR((__force void *)dev->alloc_ctx)) {
+		mtk_v4l2_err("Failed to alloc vb2 dma context 0");
+		ret = PTR_ERR((__force void *)dev->alloc_ctx);
+		goto err_vb2_ctx_init;
+	}
+
+	dev->m2m_dev_dec = v4l2_m2m_init(&mtk_vdec_m2m_ops);
+	if (IS_ERR((__force void *)dev->m2m_dev_dec)) {
+		mtk_v4l2_err("Failed to init mem2mem dec device");
+		ret = PTR_ERR((__force void *)dev->m2m_dev_dec);
+		goto err_dec_mem_init;
+	}
+
+	dev->decode_workqueue =
+		alloc_ordered_workqueue(MTK_VCODEC_DEC_NAME,
+			WQ_MEM_RECLAIM | WQ_FREEZABLE);
+	if (!dev->decode_workqueue) {
+		mtk_v4l2_err("Failed to create decode workqueue");
+		ret = -EINVAL;
+		goto err_event_workq;
+	}
+
+	ret = video_register_device(vfd_dec, VFL_TYPE_GRABBER, 0);
+	if (ret) {
+		mtk_v4l2_err("Failed to register video device");
+		goto err_dec_reg;
+	}
+
+	/* Avoid the iommu eat big hunks */
+	dma_set_attr(DMA_ATTR_ALLOC_SINGLE_PAGES, &attrs);
+
+	mtk_v4l2_debug(0, "decoder registered as /dev/video%d",
+		vfd_dec->num);
+
+	return 0;
+
+err_dec_reg:
+	destroy_workqueue(dev->decode_workqueue);
+err_event_workq:
+	v4l2_m2m_release(dev->m2m_dev_dec);
+err_dec_mem_init:
+	vb2_dma_contig_cleanup_ctx(dev->alloc_ctx);
+err_vb2_ctx_init:
+	video_unregister_device(vfd_dec);
+err_dec_alloc:
+	v4l2_device_unregister(&dev->v4l2_dev);
+err_res:
+	mtk_vcodec_release_dec_pm(dev);
+	return ret;
+}
+
+static const struct of_device_id mtk_vcodec_match[] = {
+	{.compatible = "mediatek,mt8173-vcodec-dec",},
+	{},
+};
+
+MODULE_DEVICE_TABLE(of, mtk_vcodec_match);
+
+static int mtk_vcodec_dec_remove(struct platform_device *pdev)
+{
+	struct mtk_vcodec_dev *dev = platform_get_drvdata(pdev);
+
+	flush_workqueue(dev->decode_workqueue);
+	destroy_workqueue(dev->decode_workqueue);
+	if (dev->m2m_dev_dec)
+		v4l2_m2m_release(dev->m2m_dev_dec);
+	if (dev->alloc_ctx)
+		vb2_dma_contig_cleanup_ctx(dev->alloc_ctx);
+
+	if (dev->vfd_dec)
+		video_unregister_device(dev->vfd_dec);
+
+	v4l2_device_unregister(&dev->v4l2_dev);
+	mtk_vcodec_release_dec_pm(dev);
+	return 0;
+}
+
+static struct platform_driver mtk_vcodec_dec_driver = {
+	.probe	= mtk_vcodec_probe,
+	.remove	= mtk_vcodec_dec_remove,
+	.driver	= {
+		.name	= MTK_VCODEC_DEC_NAME,
+		.owner	= THIS_MODULE,
+		.of_match_table = mtk_vcodec_match,
+	},
+};
+
+module_platform_driver(mtk_vcodec_dec_driver);
+
+MODULE_LICENSE("GPL v2");
+MODULE_DESCRIPTION("Mediatek video codec V4L2 decoder driver");
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_pm.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_pm.c
new file mode 100644
index 0000000..364b756
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_pm.c
@@ -0,0 +1,206 @@
+/*
+* Copyright (c) 2016 MediaTek Inc.
+* Author: Tiffany Lin <tiffany.lin@mediatek.com>
+*
+* This program is free software; you can redistribute it and/or modify
+* it under the terms of the GNU General Public License version 2 as
+* published by the Free Software Foundation.
+*
+* This program is distributed in the hope that it will be useful,
+* but WITHOUT ANY WARRANTY; without even the implied warranty of
+* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+* GNU General Public License for more details.
+*/
+
+#include <linux/clk.h>
+#include <linux/of_address.h>
+#include <linux/of_platform.h>
+#include <linux/pm_runtime.h>
+#include <soc/mediatek/smi.h>
+
+#include "mtk_vcodec_dec_pm.h"
+#include "mtk_vcodec_util.h"
+#include "mtk_vpu.h"
+
+int mtk_vcodec_init_dec_pm(struct mtk_vcodec_dev *mtkdev)
+{
+	struct device_node *node;
+	struct platform_device *pdev;
+	struct device *dev;
+	struct mtk_vcodec_pm *pm;
+	int ret = 0;
+
+	pdev = mtkdev->plat_dev;
+	pm = &mtkdev->pm;
+	pm->mtkdev = mtkdev;
+	dev = &pdev->dev;
+	node = of_parse_phandle(pdev->dev.of_node, "mediatek,larb", 0);
+	if (!node) {
+		mtk_v4l2_err("of_parse_phandle mediatek,larb fail!");
+		return -1;
+	}
+
+	pdev = of_find_device_by_node(node);
+	if (WARN_ON(!pdev)) {
+		of_node_put(node);
+		return -1;
+	}
+	pm->larbvdec = &pdev->dev;
+
+	pdev = mtkdev->plat_dev;
+	pm->dev = &pdev->dev;
+
+	pm->vcodecpll = devm_clk_get(&pdev->dev, "vcodecpll");
+	if (pm->vcodecpll == NULL) {
+		mtk_v4l2_err("devm_clk_get vcodecpll fail");
+		ret = -1;
+	}
+
+	pm->univpll_d2 = devm_clk_get(&pdev->dev, "univpll_d2");
+	if (pm->univpll_d2 == NULL) {
+		mtk_v4l2_err("devm_clk_get univpll_d2 fail");
+		ret = -1;
+	}
+
+	pm->clk_cci400_sel = devm_clk_get(&pdev->dev, "clk_cci400_sel");
+	if (pm->clk_cci400_sel == NULL) {
+		mtk_v4l2_err("devm_clk_get clk_cci400_sel fail");
+		ret = -1;
+	}
+
+	pm->vdec_sel = devm_clk_get(&pdev->dev, "vdec_sel");
+	if (pm->vdec_sel == NULL) {
+		mtk_v4l2_err("devm_clk_get vdec_sel fail");
+		ret = -1;
+	}
+
+	pm->vdecpll = devm_clk_get(&pdev->dev, "vdecpll");
+	if (pm->vdecpll == NULL) {
+		mtk_v4l2_err("devm_clk_get vdecpll fail");
+		ret = -1;
+	}
+
+	pm->vencpll = devm_clk_get(&pdev->dev, "vencpll");
+	if (pm->vencpll == NULL) {
+		mtk_v4l2_err("devm_clk_get vencpll fail");
+		ret = -1;
+	}
+
+	pm->venc_lt_sel = devm_clk_get(&pdev->dev, "venc_lt_sel");
+	if (pm->venc_lt_sel == NULL) {
+		mtk_v4l2_err("devm_clk_get venc_lt_sel fail");
+		ret = -1;
+	}
+
+	pm->vdec_bus_clk_src = devm_clk_get(&pdev->dev, "vdec_bus_clk_src");
+	if (pm->vdec_bus_clk_src == NULL) {
+		mtk_v4l2_err("devm_clk_get vdec_bus_clk_src");
+		ret = -1;
+	}
+
+	pm_runtime_enable(&pdev->dev);
+
+	return ret;
+}
+
+void mtk_vcodec_release_dec_pm(struct mtk_vcodec_dev *dev)
+{
+	pm_runtime_disable(dev->pm.dev);
+}
+
+void mtk_vcodec_dec_pw_on(struct mtk_vcodec_pm *pm)
+{
+	int ret;
+
+	ret = pm_runtime_get_sync(pm->dev);
+	if (ret)
+		mtk_v4l2_err("pm_runtime_get_sync fail %d", ret);
+}
+
+void mtk_vcodec_dec_pw_off(struct mtk_vcodec_pm *pm)
+{
+	int ret;
+
+	ret = pm_runtime_put_sync(pm->dev);
+	if (ret)
+		mtk_v4l2_err("pm_runtime_put_sync fail %d", ret);
+}
+
+void mtk_vcodec_dec_clock_on(struct mtk_vcodec_pm *pm)
+{
+	int ret;
+
+	ret = clk_set_rate(pm->vcodecpll, 1482 * 1000000);
+	if (ret)
+		mtk_v4l2_err("clk_set_rate vcodecpll fail %d", ret);
+
+	ret = clk_set_rate(pm->vencpll, 800 * 1000000);
+	if (ret)
+		mtk_v4l2_err("clk_set_rate vencpll fail %d", ret);
+
+	ret = clk_prepare_enable(pm->vcodecpll);
+	if (ret)
+		mtk_v4l2_err("clk_prepare_enable vcodecpll fail %d", ret);
+
+	ret = clk_prepare_enable(pm->vencpll);
+	if (ret)
+		mtk_v4l2_err("clk_prepare_enable vencpll fail %d", ret);
+
+	ret = clk_prepare_enable(pm->vdec_bus_clk_src);
+	if (ret)
+		mtk_v4l2_err("clk_prepare_enable vdec_bus_clk_src fail %d",
+				ret);
+
+	ret = clk_prepare_enable(pm->venc_lt_sel);
+	if (ret)
+		mtk_v4l2_err("clk_prepare_enable venc_lt_sel fail %d", ret);
+
+	ret = clk_set_parent(pm->venc_lt_sel, pm->vdec_bus_clk_src);
+	if (ret)
+		mtk_v4l2_err("clk_set_parent venc_lt_sel vdec_bus_clk_src fail %d",
+				ret);
+
+	ret = clk_prepare_enable(pm->univpll_d2);
+	if (ret)
+		mtk_v4l2_err("clk_prepare_enable univpll_d2 fail %d", ret);
+
+	ret = clk_prepare_enable(pm->clk_cci400_sel);
+	if (ret)
+		mtk_v4l2_err("clk_prepare_enable clk_cci400_sel fail %d", ret);
+
+	ret = clk_set_parent(pm->clk_cci400_sel, pm->univpll_d2);
+	if (ret)
+		mtk_v4l2_err("clk_set_parent clk_cci400_sel univpll_d2 fail %d",
+				ret);
+
+	ret = clk_prepare_enable(pm->vdecpll);
+	if (ret)
+		mtk_v4l2_err("clk_prepare_enable vdecpll fail %d", ret);
+
+	ret = clk_prepare_enable(pm->vdec_sel);
+	if (ret)
+		mtk_v4l2_err("clk_prepare_enable vdec_sel fail %d", ret);
+
+	ret = clk_set_parent(pm->vdec_sel, pm->vdecpll);
+	if (ret)
+		mtk_v4l2_err("clk_set_parent vdec_sel vdecpll fail %d", ret);
+
+	ret = mtk_smi_larb_get(pm->larbvdec);
+	if (ret)
+		mtk_v4l2_err("mtk_smi_larb_get larbvdec fail %d", ret);
+
+}
+
+void mtk_vcodec_dec_clock_off(struct mtk_vcodec_pm *pm)
+{
+	mtk_smi_larb_put(pm->larbvdec);
+	clk_disable_unprepare(pm->vdec_sel);
+	clk_disable_unprepare(pm->vdecpll);
+	clk_disable_unprepare(pm->univpll_d2);
+	clk_disable_unprepare(pm->clk_cci400_sel);
+	clk_disable_unprepare(pm->venc_lt_sel);
+	clk_disable_unprepare(pm->vdec_bus_clk_src);
+	clk_disable_unprepare(pm->vencpll);
+	clk_disable_unprepare(pm->vcodecpll);
+}
+
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_pm.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_pm.h
new file mode 100644
index 0000000..b8bf56f
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_pm.h
@@ -0,0 +1,28 @@
+/*
+* Copyright (c) 2016 MediaTek Inc.
+* Author: Tiffany Lin <tiffany.lin@mediatek.com>
+*
+* This program is free software; you can redistribute it and/or modify
+* it under the terms of the GNU General Public License version 2 as
+* published by the Free Software Foundation.
+*
+* This program is distributed in the hope that it will be useful,
+* but WITHOUT ANY WARRANTY; without even the implied warranty of
+* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+* GNU General Public License for more details.
+*/
+
+#ifndef _MTK_VCODEC_DEC_PM_H_
+#define _MTK_VCODEC_DEC_PM_H_
+
+#include "mtk_vcodec_drv.h"
+
+int mtk_vcodec_init_dec_pm(struct mtk_vcodec_dev *dev);
+void mtk_vcodec_release_dec_pm(struct mtk_vcodec_dev *dev);
+
+void mtk_vcodec_dec_pw_on(struct mtk_vcodec_pm *pm);
+void mtk_vcodec_dec_pw_off(struct mtk_vcodec_pm *pm);
+void mtk_vcodec_dec_clock_on(struct mtk_vcodec_pm *pm);
+void mtk_vcodec_dec_clock_off(struct mtk_vcodec_pm *pm);
+
+#endif /* _MTK_VCODEC_DEC_PM_H_ */
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
index 78eee50..a11dfec 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
@@ -26,10 +26,10 @@
 #include "mtk_vcodec_util.h"
 
 #define MTK_VCODEC_DRV_NAME	"mtk_vcodec_drv"
+#define MTK_VCODEC_DEC_NAME	"mtk-vcodec-dec"
 #define MTK_VCODEC_ENC_NAME	"mtk-vcodec-enc"
 #define MTK_PLATFORM_STR	"platform:mt8173"
 
-
 #define MTK_VCODEC_MAX_PLANES	3
 #define MTK_V4L2_BENCHMARK	0
 #define WAIT_INTR_TIMEOUT_MS	1000
@@ -180,6 +180,9 @@ struct mtk_enc_params {
  * struct mtk_vcodec_pm - Power management data structure
  */
 struct mtk_vcodec_pm {
+	struct clk	*vdec_bus_clk_src;
+	struct clk	*vencpll;
+
 	struct clk	*vcodecpll;
 	struct clk	*univpll_d2;
 	struct clk	*clk_cci400_sel;
@@ -197,6 +200,32 @@ struct mtk_vcodec_pm {
 };
 
 /**
+ * struct vdec_pic_info  - picture size information
+ * @pic_w: picture width
+ * @pic_h: picture height
+ * @buf_w: picture buffer width (64 aligned up from pic_w)
+ * @buf_h: picture buffer heiht (64 aligned up from pic_h)
+ * @y_bs_sz: Y bitstream size
+ * @c_bs_sz: CbCr bitstream size
+ * @y_len_sz: additional size required to store decompress information for y
+ *		plane
+ * @c_len_sz: additional size required to store decompress information for cbcr
+ *		plane
+ * E.g. suppose picture size is 176x144,
+ *      buffer size will be aligned to 176x160.
+ */
+struct vdec_pic_info {
+	unsigned int pic_w;
+	unsigned int pic_h;
+	unsigned int buf_w;
+	unsigned int buf_h;
+	unsigned int y_bs_sz;
+	unsigned int c_bs_sz;
+	unsigned int y_len_sz;
+	unsigned int c_len_sz;
+};
+
+/**
  * struct mtk_vcodec_ctx - Context (instance) private data.
  *
  * @type: type of the instance - decoder or encoder
@@ -210,9 +239,12 @@ struct mtk_vcodec_pm {
  * @state: state of the context
  * @param_change: indicate encode parameter type
  * @enc_params: encoding parameters
- * @enc_if: hoooked encoder driver interface
+ * @dec_if: hooked decoder driver interface
+ * @enc_if: hooked encoder driver interface
  * @drv_handle: driver handle for specific decode/encode instance
  *
+ * @picinfo: store picture info after header parsing
+ * @dpb_size: store dpb count after header parsing
  * @int_cond: variable used by the waitqueue
  * @int_type: type of the last interrupt
  * @queue: waitqueue that can be used to wait for this context to
@@ -220,12 +252,16 @@ struct mtk_vcodec_pm {
  * @irq_status: irq status
  *
  * @ctrl_hdl: handler for v4l2 framework
+ * @decode_work: worker for the decoding
  * @encode_work: worker for the encoding
+ * @last_decoded_picinfo: pic information get from latest decode
  *
  * @colorspace: enum v4l2_colorspace; supplemental to pixelformat
  * @ycbcr_enc: enum v4l2_ycbcr_encoding, Y'CbCr encoding
  * @quantization: enum v4l2_quantization, colorspace quantization
  * @xfer_func: enum v4l2_xfer_func, colorspace transfer function
+ * @lock: protect variables accessed by V4L2 threads and worker thread such as
+ *	  mtk_video_dec_buf.
  */
 struct mtk_vcodec_ctx {
 	enum mtk_instance_type type;
@@ -240,21 +276,31 @@ struct mtk_vcodec_ctx {
 	enum mtk_encode_param param_change;
 	struct mtk_enc_params enc_params;
 
+	struct vdec_common_if *dec_if;
 	struct venc_common_if *enc_if;
 	unsigned long drv_handle;
 
+	struct vdec_pic_info picinfo;
+	int dpb_size;
+
 	int int_cond;
 	int int_type;
 	wait_queue_head_t queue;
 	unsigned int irq_status;
 
 	struct v4l2_ctrl_handler ctrl_hdl;
+	struct work_struct decode_work;
 	struct work_struct encode_work;
+	struct vdec_pic_info last_decoded_picinfo;
 
 	enum v4l2_colorspace colorspace;
 	enum v4l2_ycbcr_encoding ycbcr_enc;
 	enum v4l2_quantization quantization;
 	enum v4l2_xfer_func xfer_func;
+
+	int decoded_frame_cnt;
+	struct mutex lock;
+
 };
 
 /**
@@ -268,25 +314,48 @@ struct mtk_vcodec_ctx {
  * @alloc_ctx: VB2 allocator context
  *	       (for allocations without kernel mapping).
  * @ctx_list: list of struct mtk_vcodec_ctx
- * @irqlock: protect data access by irq handler and work thread
- * @curr_ctx: The context that is waiting for codec hardware
+ * @irqlock: protect data access by irq handler and work thread -- curr_ctx
+ * @curr_ctx: The context that is waiting for codec hardware. Protected by
+ *		irqlock and dec_mutex or enc_mutex.
  *
  * @reg_base: Mapped address of MTK Vcodec registers.
  *
- * @id_counter: used to identify current opened instance
+ * @id_counter:  An integer id given to the next opened instance
  * @num_instances: counter of active MTK Vcodec instances
  *
  * @encode_workqueue: encode work queue
  *
+ * @v4l2_dev: V4L2 device to register video devices for.
+ * @vfd_dec: Video device for decoder
+ * @vfd_enc: Video device for encoder
+ *
+ * @m2m_dev_dec: m2m device for decoder
+ * @m2m_dev_enc: m2m device for encoder
+ * @plat_dev: platform device
+ * @vpu_plat_dev: mtk vpu platform device
+ * @alloc_ctx:VB2 allocator context
+ *	      (for allocations without kernel mapping).
+ * @ctx_list: list of struct mtk_vcodec_ctx
+ * @irqlock: protect data access by irq handler and work thread
+ * @curr_ctx: The context that is waiting for codec hardware
+ * @reg_base: Mapped address of MTK Vcodec registers.
+ *
+ * @instance_mask: used to mark which index id are used
+ * @num_instances: counter of active MTK Vcodec instances
+ *
+ * @decode_workqueue: decode work queue
+ * @encode_workqueue: encode work queue
  * @int_cond: used to identify interrupt condition happen
  * @int_type: used to identify what kind of interrupt condition happen
  * @dev_mutex: video_device lock
  * @queue: waitqueue for waiting for completion of device commands
  *
+ * @dec_irq: decoder irq resource
  * @enc_irq: h264 encoder irq resource
- * @enc_lt_irq: vp8 encoder irq resource
+ * @enc_lt_irq: vp8 encoder lt irq resource
  *
- * @enc_mutex: encoder hardware lock.
+ * @dec_mutex: decoder hardware lock
+ * @enc_mutex: encoder hardware lock
  *
  * @pm: power management control
  * @dec_capability: used to identify decode capability, ex: 4k
@@ -294,8 +363,10 @@ struct mtk_vcodec_ctx {
  */
 struct mtk_vcodec_dev {
 	struct v4l2_device v4l2_dev;
+	struct video_device *vfd_dec;
 	struct video_device *vfd_enc;
 
+	struct v4l2_m2m_dev *m2m_dev_dec;
 	struct v4l2_m2m_dev *m2m_dev_enc;
 	struct platform_device *plat_dev;
 	struct platform_device *vpu_plat_dev;
@@ -308,6 +379,7 @@ struct mtk_vcodec_dev {
 	unsigned long id_counter;
 	int num_instances;
 
+	struct workqueue_struct *decode_workqueue;
 	struct workqueue_struct *encode_workqueue;
 
 	int int_cond;
@@ -315,9 +387,11 @@ struct mtk_vcodec_dev {
 	struct mutex dev_mutex;
 	wait_queue_head_t queue;
 
+	int dec_irq;
 	int enc_irq;
 	int enc_lt_irq;
 
+	struct mutex dec_mutex;
 	struct mutex enc_mutex;
 
 	struct mtk_vcodec_pm pm;
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.c
index 52e7e5c..f0bdb19 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.c
@@ -30,8 +30,7 @@ int mtk_vcodec_wait_for_done_ctx(struct mtk_vcodec_ctx  *ctx, int command,
 	timeout_jiff = msecs_to_jiffies(timeout_ms);
 
 	ret = wait_event_interruptible_timeout(*waitqueue,
-				(ctx->int_cond &&
-				(ctx->int_type == command)),
+				ctx->int_cond,
 				timeout_jiff);
 
 	if (!ret) {
@@ -52,3 +51,4 @@ int mtk_vcodec_wait_for_done_ctx(struct mtk_vcodec_ctx  *ctx, int command,
 	return status;
 }
 EXPORT_SYMBOL(mtk_vcodec_wait_for_done_ctx);
+
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.h
index 33e890f..fbb8b1e 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.h
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.h
@@ -16,7 +16,7 @@
 #define _MTK_VCODEC_INTR_H_
 
 #define MTK_INST_IRQ_RECEIVED		0x1
-#define MTK_INST_WORK_THREAD_ABORT_DONE	0x2
+
 
 struct mtk_vcodec_ctx;
 
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c
index 5e36513..7617305 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c
@@ -81,14 +81,35 @@ void mtk_vcodec_mem_free(struct mtk_vcodec_ctx *data,
 		return;
 	}
 
-	dma_free_coherent(dev, size, mem->va, mem->dma_addr);
-	mem->va = NULL;
-	mem->dma_addr = 0;
-	mem->size = 0;
-
 	mtk_v4l2_debug(3, "[%d]  - va      = %p", ctx->id, mem->va);
 	mtk_v4l2_debug(3, "[%d]  - dma     = 0x%lx", ctx->id,
 		       (unsigned long)mem->dma_addr);
 	mtk_v4l2_debug(3, "[%d]    size = 0x%lx", ctx->id, size);
+
+	dma_free_coherent(dev, size, mem->va, mem->dma_addr);
+	mem->va = NULL;
+	mem->dma_addr = 0;
+	mem->size = 0;
 }
 EXPORT_SYMBOL(mtk_vcodec_mem_free);
+
+void mtk_vcodec_set_curr_ctx(struct mtk_vcodec_dev *dev,
+	struct mtk_vcodec_ctx *ctx)
+{
+	unsigned long flags;
+
+	spin_lock_irqsave(&dev->irqlock, flags);
+	dev->curr_ctx = ctx;
+	spin_unlock_irqrestore(&dev->irqlock, flags);
+}
+
+struct mtk_vcodec_ctx *mtk_vcodec_get_curr_ctx(struct mtk_vcodec_dev *dev)
+{
+	unsigned long flags;
+	struct mtk_vcodec_ctx *ctx;
+
+	spin_lock_irqsave(&dev->irqlock, flags);
+	ctx = dev->curr_ctx;
+	spin_unlock_irqrestore(&dev->irqlock, flags);
+	return ctx;
+}
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
index d6345fc..91a0b1a 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
@@ -1,7 +1,7 @@
 /*
 * Copyright (c) 2016 MediaTek Inc.
 * Author: PC Chen <pc.chen@mediatek.com>
-*	Tiffany Lin <tiffany.lin@mediatek.com>
+*		Tiffany Lin <tiffany.lin@mediatek.com>
 *
 * This program is free software; you can redistribute it and/or modify
 * it under the terms of the GNU General Public License version 2 as
@@ -26,6 +26,7 @@ struct mtk_vcodec_mem {
 };
 
 struct mtk_vcodec_ctx;
+struct mtk_vcodec_dev;
 
 extern int mtk_v4l2_dbg_level;
 extern bool mtk_vcodec_dbg;
@@ -84,4 +85,8 @@ int mtk_vcodec_mem_alloc(struct mtk_vcodec_ctx *data,
 				struct mtk_vcodec_mem *mem);
 void mtk_vcodec_mem_free(struct mtk_vcodec_ctx *data,
 				struct mtk_vcodec_mem *mem);
+void mtk_vcodec_set_curr_ctx(struct mtk_vcodec_dev *dev,
+	struct mtk_vcodec_ctx *ctx);
+struct mtk_vcodec_ctx *mtk_vcodec_get_curr_ctx(struct mtk_vcodec_dev *dev);
+
 #endif /* _MTK_VCODEC_UTIL_H_ */
diff --git a/drivers/media/platform/mtk-vcodec/vdec_drv_base.h b/drivers/media/platform/mtk-vcodec/vdec_drv_base.h
new file mode 100644
index 0000000..7e4c1a9
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/vdec_drv_base.h
@@ -0,0 +1,56 @@
+/*
+ * Copyright (c) 2016 MediaTek Inc.
+ * Author: PC Chen <pc.chen@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef _VDEC_DRV_BASE_
+#define _VDEC_DRV_BASE_
+
+#include "mtk_vcodec_drv.h"
+
+#include "vdec_drv_if.h"
+
+struct vdec_common_if {
+	/**
+	 * (*init)() - initialize decode driver
+	 * @ctx     : [in] mtk v4l2 context
+	 * @h_vdec  : [out] driver handle
+	 */
+	int (*init)(struct mtk_vcodec_ctx *ctx, unsigned long *h_vdec);
+
+	/**
+	 * (*decode)() - trigger decode
+	 * @h_vdec  : [in] driver handle
+	 * @bs      : [in] input bitstream
+	 * @fb      : [in] frame buffer to store decoded frame
+	 * @res_chg : [out] resolution change happen
+	 */
+	int (*decode)(unsigned long h_vdec, struct mtk_vcodec_mem *bs,
+		      struct vdec_fb *fb, bool *res_chg);
+
+	/**
+	 * (*get_param)() - get driver's parameter
+	 * @h_vdec : [in] driver handle
+	 * @type   : [in] input parameter type
+	 * @out    : [out] buffer to store query result
+	 */
+	int (*get_param)(unsigned long h_vdec, enum vdec_get_param_type type,
+			 void *out);
+
+	/**
+	 * (*deinit)() - deinitialize driver.
+	 * @h_vdec : [in] driver handle to be deinit
+	 */
+	void (*deinit)(unsigned long h_vdec);
+};
+
+#endif
diff --git a/drivers/media/platform/mtk-vcodec/vdec_drv_if.c b/drivers/media/platform/mtk-vcodec/vdec_drv_if.c
new file mode 100644
index 0000000..8202b14
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/vdec_drv_if.c
@@ -0,0 +1,107 @@
+/*
+ * Copyright (c) 2016 MediaTek Inc.
+ * Author: PC Chen <pc.chen@mediatek.com>
+ *         Tiffany Lin <tiffany.lin@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/interrupt.h>
+#include <linux/kernel.h>
+#include <linux/slab.h>
+
+#include "vdec_drv_base.h"
+#include "vdec_drv_if.h"
+
+#include "mtk_vcodec_dec_pm.h"
+#include "mtk_vcodec_util.h"
+#include "mtk_vpu.h"
+
+int vdec_if_init(struct mtk_vcodec_ctx *ctx, unsigned int fourcc)
+{
+	int ret = 0;
+
+	switch (fourcc) {
+	case V4L2_PIX_FMT_H264:
+	case V4L2_PIX_FMT_VP8:
+	case V4L2_PIX_FMT_VP9:
+	default:
+		return -EINVAL;
+	}
+
+	mtk_vdec_lock(ctx);
+	mtk_vcodec_dec_clock_on(&ctx->dev->pm);
+	ret = ctx->dec_if->init(ctx, &ctx->drv_handle);
+	mtk_vcodec_dec_clock_off(&ctx->dev->pm);
+	mtk_vdec_unlock(ctx);
+
+	return ret;
+}
+
+int vdec_if_decode(struct mtk_vcodec_ctx *ctx, struct mtk_vcodec_mem *bs,
+		   struct vdec_fb *fb, bool *res_chg)
+{
+	int ret = 0;
+
+	if (bs) {
+		if ((bs->dma_addr & 63) != 0) {
+			mtk_v4l2_err("bs dma_addr should 64 byte align");
+			return -EINVAL;
+		}
+	}
+
+	if (fb) {
+		if (((fb->base_y.dma_addr & 511) != 0) ||
+		    ((fb->base_c.dma_addr & 511) != 0)) {
+			mtk_v4l2_err("frame buffer dma_addr should 512 byte align");
+			return -EINVAL;
+		}
+	}
+
+	mtk_vdec_lock(ctx);
+
+	mtk_vcodec_set_curr_ctx(ctx->dev, ctx);
+	mtk_vcodec_dec_clock_on(&ctx->dev->pm);
+	enable_irq(ctx->dev->dec_irq);
+	ret = ctx->dec_if->decode(ctx->drv_handle, bs, fb, res_chg);
+	disable_irq(ctx->dev->dec_irq);
+	mtk_vcodec_dec_clock_off(&ctx->dev->pm);
+	mtk_vcodec_set_curr_ctx(ctx->dev, NULL);
+
+	mtk_vdec_unlock(ctx);
+
+	return ret;
+}
+
+int vdec_if_get_param(struct mtk_vcodec_ctx *ctx, enum vdec_get_param_type type,
+		      void *out)
+{
+	int ret = 0;
+
+	mtk_vdec_lock(ctx);
+	ret = ctx->dec_if->get_param(ctx->drv_handle, type, out);
+	mtk_vdec_unlock(ctx);
+
+	return ret;
+}
+
+void vdec_if_deinit(struct mtk_vcodec_ctx *ctx)
+{
+	if (ctx->drv_handle == 0)
+		return;
+
+	mtk_vdec_lock(ctx);
+	mtk_vcodec_dec_clock_on(&ctx->dev->pm);
+	ctx->dec_if->deinit(ctx->drv_handle);
+	mtk_vcodec_dec_clock_off(&ctx->dev->pm);
+	mtk_vdec_unlock(ctx);
+
+	ctx->drv_handle = 0;
+}
diff --git a/drivers/media/platform/mtk-vcodec/vdec_drv_if.h b/drivers/media/platform/mtk-vcodec/vdec_drv_if.h
new file mode 100644
index 0000000..db6b520
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/vdec_drv_if.h
@@ -0,0 +1,101 @@
+/*
+ * Copyright (c) 2016 MediaTek Inc.
+ * Author: PC Chen <pc.chen@mediatek.com>
+ *		   Tiffany Lin <tiffany.lin@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef _VDEC_DRV_IF_H_
+#define _VDEC_DRV_IF_H_
+
+#include "mtk_vcodec_drv.h"
+#include "mtk_vcodec_dec.h"
+#include "mtk_vcodec_util.h"
+
+
+/**
+ * struct vdec_fb_status  - decoder frame buffer status
+ * @FB_ST_NORMAL	: initial state
+ * @FB_ST_DISPLAY	: frmae buffer is ready to be displayed
+ * @FB_ST_FREE		: frame buffer is not used by decoder any more
+ */
+enum vdec_fb_status {
+	FB_ST_NORMAL		= 0,
+	FB_ST_DISPLAY		= (1 << 0),
+	FB_ST_FREE		= (1 << 1)
+};
+
+/* For GET_PARAM_DISP_FRAME_BUFFER and GET_PARAM_FREE_FRAME_BUFFER,
+ * the caller does not own the returned buffer. The buffer will not be
+ *				released before vdec_if_deinit.
+ * GET_PARAM_DISP_FRAME_BUFFER	: get next displayable frame buffer,
+ *				struct vdec_fb**
+ * GET_PARAM_FREE_FRAME_BUFFER	: get non-referenced framebuffer, vdec_fb**
+ * GET_PARAM_PIC_INFO		: get picture info, struct vdec_pic_info*
+ * GET_PARAM_CROP_INFO		: get crop info, struct v4l2_crop*
+ * GET_PARAM_DPB_SIZE		: get dpb size, unsigned int*
+ */
+enum vdec_get_param_type {
+	GET_PARAM_DISP_FRAME_BUFFER,
+	GET_PARAM_FREE_FRAME_BUFFER,
+	GET_PARAM_PIC_INFO,
+	GET_PARAM_CROP_INFO,
+	GET_PARAM_DPB_SIZE
+};
+
+/**
+ * struct vdec_fb_node  - decoder frame buffer node
+ * @list	: list to hold this node
+ * @fb	: point to frame buffer (vdec_fb), fb could point to frame buffer and
+ *	working buffer this is for maintain buffers in different state
+ */
+struct vdec_fb_node {
+	struct list_head list;
+	struct vdec_fb *fb;
+};
+
+/**
+ * vdec_if_init() - initialize decode driver
+ * @ctx	: [in] v4l2 context
+ * @fourcc	: [in] video format fourcc, V4L2_PIX_FMT_H264/VP8/VP9..
+ */
+int vdec_if_init(struct mtk_vcodec_ctx *ctx, unsigned int fourcc);
+
+/**
+ * vdec_if_deinit() - deinitialize decode driver
+ * @ctx	: [in] v4l2 context
+ *
+ */
+void vdec_if_deinit(struct mtk_vcodec_ctx *ctx);
+
+/**
+ * vdec_if_decode() - trigger decode
+ * @ctx	: [in] v4l2 context
+ * @bs	: [in] input bitstream
+ * @fb	: [in] frame buffer to store decoded frame, when null menas parse
+ *	header only
+ * @res_chg	: [out] resolution change happens if current bs have different
+ *	picture width/height
+ * Note: To flush the decoder when reaching EOF, set input bitstream as NULL.
+ */
+int vdec_if_decode(struct mtk_vcodec_ctx *ctx, struct mtk_vcodec_mem *bs,
+		   struct vdec_fb *fb, bool *res_chg);
+
+/**
+ * vdec_if_get_param() - get driver's parameter
+ * @ctx	: [in] v4l2 context
+ * @type	: [in] input parameter type
+ * @out	: [out] buffer to store query result
+ */
+int vdec_if_get_param(struct mtk_vcodec_ctx *ctx, enum vdec_get_param_type type,
+		      void *out);
+
+#endif
diff --git a/drivers/media/platform/mtk-vcodec/vdec_ipi_msg.h b/drivers/media/platform/mtk-vcodec/vdec_ipi_msg.h
new file mode 100644
index 0000000..5a8a629
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/vdec_ipi_msg.h
@@ -0,0 +1,103 @@
+/*
+ * Copyright (c) 2016 MediaTek Inc.
+ * Author: PC Chen <pc.chen@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef _VDEC_IPI_MSG_H_
+#define _VDEC_IPI_MSG_H_
+
+/**
+ * enum vdec_ipi_msgid - message id between AP and VPU
+ * @AP_IPIMSG_XXX	: AP to VPU cmd message id
+ * @VPU_IPIMSG_XXX_ACK	: VPU ack AP cmd message id
+ */
+enum vdec_ipi_msgid {
+	AP_IPIMSG_DEC_INIT = 0xA000,
+	AP_IPIMSG_DEC_START = 0xA001,
+	AP_IPIMSG_DEC_END = 0xA002,
+	AP_IPIMSG_DEC_DEINIT = 0xA003,
+	AP_IPIMSG_DEC_RESET = 0xA004,
+
+	VPU_IPIMSG_DEC_INIT_ACK = 0xB000,
+	VPU_IPIMSG_DEC_START_ACK = 0xB001,
+	VPU_IPIMSG_DEC_END_ACK = 0xB002,
+	VPU_IPIMSG_DEC_DEINIT_ACK = 0xB003,
+	VPU_IPIMSG_DEC_RESET_ACK = 0xB004,
+};
+
+/**
+ * struct vdec_ap_ipi_cmd - generic AP to VPU ipi command format
+ * @msg_id	: vdec_ipi_msgid
+ * @vpu_inst_addr	: VPU decoder instance address
+ */
+struct vdec_ap_ipi_cmd {
+	uint32_t msg_id;
+	uint32_t vpu_inst_addr;
+};
+
+/**
+ * struct vdec_vpu_ipi_ack - generic VPU to AP ipi command format
+ * @msg_id	: vdec_ipi_msgid
+ * @status	: VPU exeuction result
+ * @ap_inst_addr	: AP video decoder instance address
+ */
+struct vdec_vpu_ipi_ack {
+	uint32_t msg_id;
+	int32_t status;
+	uint64_t ap_inst_addr;
+};
+
+/**
+ * struct vdec_ap_ipi_init - for AP_IPIMSG_DEC_INIT
+ * @msg_id	: AP_IPIMSG_DEC_INIT
+ * @reserved	: Reserved field
+ * @ap_inst_addr	: AP video decoder instance address
+ */
+struct vdec_ap_ipi_init {
+	uint32_t msg_id;
+	uint32_t reserved;
+	uint64_t ap_inst_addr;
+};
+
+/**
+ * struct vdec_ap_ipi_dec_start - for AP_IPIMSG_DEC_START
+ * @msg_id	: AP_IPIMSG_DEC_START
+ * @vpu_inst_addr	: VPU decoder instance address
+ * @data	: Header info
+ *	H264 decoder [0]:buf_sz [1]:nal_start
+ *	VP8 decoder  [0]:width/height
+ *	VP9 decoder  [0]:profile, [1][2] width/height
+ * @reserved	: Reserved field
+ */
+struct vdec_ap_ipi_dec_start {
+	uint32_t msg_id;
+	uint32_t vpu_inst_addr;
+	uint32_t data[3];
+	uint32_t reserved;
+};
+
+/**
+ * struct vdec_vpu_ipi_init_ack - for VPU_IPIMSG_DEC_INIT_ACK
+ * @msg_id	: VPU_IPIMSG_DEC_INIT_ACK
+ * @status	: VPU exeuction result
+ * @ap_inst_addr	: AP vcodec_vpu_inst instance address
+ * @vpu_inst_addr	: VPU decoder instance address
+ */
+struct vdec_vpu_ipi_init_ack {
+	uint32_t msg_id;
+	int32_t status;
+	uint64_t ap_inst_addr;
+	uint32_t vpu_inst_addr;
+};
+
+#endif
-- 
1.7.9.5

[PATCH v3 6/9] vcodec: mediatek: Add Mediatek H264 Video Decoder Driver

[Tiffany Lin]

Add h264 decoder driver for MT8173

Signed-off-by: PC Chen <pc.chen@mediatek.com>
Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
---
 drivers/media/platform/mtk-vcodec/Makefile         |    5 +-
 .../media/platform/mtk-vcodec/vdec/vdec_h264_if.c  |  503 ++++++++++++++++++++
 drivers/media/platform/mtk-vcodec/vdec_drv_if.c    |    4 +
 drivers/media/platform/mtk-vcodec/vdec_vpu_if.c    |  168 +++++++
 drivers/media/platform/mtk-vcodec/vdec_vpu_if.h    |   96 ++++
 5 files changed, 774 insertions(+), 2 deletions(-)
 create mode 100644 drivers/media/platform/mtk-vcodec/vdec/vdec_h264_if.c
 create mode 100644 drivers/media/platform/mtk-vcodec/vdec_vpu_if.c
 create mode 100644 drivers/media/platform/mtk-vcodec/vdec_vpu_if.h

diff --git a/drivers/media/platform/mtk-vcodec/Makefile b/drivers/media/platform/mtk-vcodec/Makefile
index 4c8ed2f..58243ed 100644
--- a/drivers/media/platform/mtk-vcodec/Makefile
+++ b/drivers/media/platform/mtk-vcodec/Makefile
@@ -1,11 +1,12 @@
 
-
 obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += mtk-vcodec-dec.o \
 				       mtk-vcodec-enc.o \
 				       mtk-vcodec-common.o
 
-mtk-vcodec-dec-y := mtk_vcodec_dec_drv.o \
+mtk-vcodec-dec-y := vdec/vdec_h264_if.o \
+		mtk_vcodec_dec_drv.o \
 		vdec_drv_if.o \
+		vdec_vpu_if.o \
 		mtk_vcodec_dec.o \
 		mtk_vcodec_dec_pm.o \
 
diff --git a/drivers/media/platform/mtk-vcodec/vdec/vdec_h264_if.c b/drivers/media/platform/mtk-vcodec/vdec/vdec_h264_if.c
new file mode 100644
index 0000000..8de0b53
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/vdec/vdec_h264_if.c
@@ -0,0 +1,503 @@
+/*
+ * Copyright (c) 2016 MediaTek Inc.
+ * Author: PC Chen <pc.chen@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/module.h>
+#include <linux/slab.h>
+
+#include "../mtk_vcodec_intr.h"
+#include "../vdec_vpu_if.h"
+#include "../vdec_drv_base.h"
+
+#define NAL_NON_IDR_SLICE			0x01
+#define NAL_IDR_SLICE				0x05
+#define NAL_H264_PPS				0x08
+#define NAL_TYPE(value)				((value) & 0x1F)
+
+#define BUF_PREDICTION_SZ			(32 * 1024)
+
+#define MB_UNIT_LEN				16
+
+/* motion vector size (bytes) for every macro block */
+#define HW_MB_STORE_SZ				64
+
+#define H264_MAX_FB_NUM				17
+#define HDR_PARSING_BUF_SZ			1024
+
+/**
+ * struct h264_fb - h264 decode frame buffer information
+ * @vdec_fb_va  : virtual address of struct vdec_fb
+ * @y_fb_dma    : dma address of Y frame buffer (luma)
+ * @c_fb_dma    : dma address of C frame buffer (chroma)
+ * @poc         : picture order count of frame buffer
+ * @reserved    : for 8 bytes alignment
+ */
+struct h264_fb {
+	uint64_t vdec_fb_va;
+	uint64_t y_fb_dma;
+	uint64_t c_fb_dma;
+	int32_t poc;
+	uint32_t reserved;
+};
+
+/**
+ * struct h264_ring_fb_list - ring frame buffer list
+ * @fb_list   : frame buffer arrary
+ * @read_idx  : read index
+ * @write_idx : write index
+ * @count     : buffer count in list
+ */
+struct h264_ring_fb_list {
+	struct h264_fb fb_list[H264_MAX_FB_NUM];
+	unsigned int read_idx;
+	unsigned int write_idx;
+	unsigned int count;
+	unsigned int reserved;
+};
+
+/**
+ * struct vdec_h264_dec_info - decode information
+ * @dpb_sz		: decoding picture buffer size
+ * @resolution_changed  : resoltion change happen
+ * @realloc_mv_buf	: flag to notify driver to re-allocate mv buffer
+ * @reserved		: for 8 bytes alignment
+ * @bs_dma		: Input bit-stream buffer dma address
+ * @y_fb_dma		: Y frame buffer dma address
+ * @c_fb_dma		: C frame buffer dma address
+ * @vdec_fb_va		: VDEC frame buffer struct virtual address
+ */
+struct vdec_h264_dec_info {
+	uint32_t dpb_sz;
+	uint32_t resolution_changed;
+	uint32_t realloc_mv_buf;
+	uint32_t reserved;
+	uint64_t bs_dma;
+	uint64_t y_fb_dma;
+	uint64_t c_fb_dma;
+	uint64_t vdec_fb_va;
+};
+
+/**
+ * struct vdec_h264_vsi - shared memory for decode information exchange
+ *                        between VPU and Host.
+ *                        The memory is allocated by VPU then mapping to Host
+ *                        in vpu_dec_init() and freed in vpu_dec_deinit()
+ *                        by VPU.
+ *                        AP-W/R : AP is writer/reader on this item
+ *                        VPU-W/R: VPU is write/reader on this item
+ * @hdr_buf      : Header parsing buffer (AP-W, VPU-R)
+ * @pred_buf_dma : HW working predication buffer dma address (AP-W, VPU-R)
+ * @mv_buf_dma   : HW working motion vector buffer dma address (AP-W, VPU-R)
+ * @list_free    : free frame buffer ring list (AP-W/R, VPU-W)
+ * @list_disp    : display frame buffer ring list (AP-R, VPU-W)
+ * @dec          : decode information (AP-R, VPU-W)
+ * @pic          : picture information (AP-R, VPU-W)
+ * @crop         : crop information (AP-R, VPU-W)
+ */
+struct vdec_h264_vsi {
+	unsigned char hdr_buf[HDR_PARSING_BUF_SZ];
+	uint64_t pred_buf_dma;
+	uint64_t mv_buf_dma[H264_MAX_FB_NUM];
+	struct h264_ring_fb_list list_free;
+	struct h264_ring_fb_list list_disp;
+	struct vdec_h264_dec_info dec;
+	struct vdec_pic_info pic;
+	struct v4l2_rect crop;
+};
+
+/**
+ * struct vdec_h264_inst - h264 decoder instance
+ * @num_nalu : how many nalus be decoded
+ * @ctx      : point to mtk_vcodec_ctx
+ * @pred_buf : HW working predication buffer
+ * @mv_buf   : HW working motion vector buffer
+ * @vpu      : VPU instance
+ * @vsi      : VPU shared information
+ */
+struct vdec_h264_inst {
+	unsigned int num_nalu;
+	struct mtk_vcodec_ctx *ctx;
+	struct mtk_vcodec_mem pred_buf;
+	struct mtk_vcodec_mem mv_buf[H264_MAX_FB_NUM];
+	struct vdec_vpu_inst vpu;
+	struct vdec_h264_vsi *vsi;
+};
+
+static unsigned int get_mv_buf_size(unsigned int width, unsigned int height)
+{
+	return HW_MB_STORE_SZ * (width/MB_UNIT_LEN) * (height/MB_UNIT_LEN);
+}
+
+static int allocate_predication_buf(struct vdec_h264_inst *inst)
+{
+	int err = 0;
+
+	inst->pred_buf.size = BUF_PREDICTION_SZ;
+	err = mtk_vcodec_mem_alloc(inst->ctx, &inst->pred_buf);
+	if (err) {
+		mtk_vcodec_err(inst, "failed to allocate ppl buf");
+		return err;
+	}
+
+	inst->vsi->pred_buf_dma = inst->pred_buf.dma_addr;
+	return 0;
+}
+
+static void free_predication_buf(struct vdec_h264_inst *inst)
+{
+	struct mtk_vcodec_mem *mem = NULL;
+
+	mtk_vcodec_debug_enter(inst);
+
+	inst->vsi->pred_buf_dma = 0;
+	mem = &inst->pred_buf;
+	if (mem->va)
+		mtk_vcodec_mem_free(inst->ctx, mem);
+}
+
+static int alloc_mv_buf(struct vdec_h264_inst *inst, struct vdec_pic_info *pic)
+{
+	int i;
+	int err;
+	struct mtk_vcodec_mem *mem = NULL;
+	unsigned int buf_sz = get_mv_buf_size(pic->buf_w, pic->buf_h);
+
+	for (i = 0; i < H264_MAX_FB_NUM; i++) {
+		mem = &inst->mv_buf[i];
+		if (mem->va)
+			mtk_vcodec_mem_free(inst->ctx, mem);
+		mem->size = buf_sz;
+		err = mtk_vcodec_mem_alloc(inst->ctx, mem);
+		if (err) {
+			mtk_vcodec_err(inst, "failed to allocate mv buf");
+			return err;
+		}
+		inst->vsi->mv_buf_dma[i] = mem->dma_addr;
+	}
+
+	return 0;
+}
+
+static void free_mv_buf(struct vdec_h264_inst *inst)
+{
+	int i;
+	struct mtk_vcodec_mem *mem = NULL;
+
+	for (i = 0; i < H264_MAX_FB_NUM; i++) {
+		inst->vsi->mv_buf_dma[i] = 0;
+		mem = &inst->mv_buf[i];
+		if (mem->va)
+			mtk_vcodec_mem_free(inst->ctx, mem);
+	}
+}
+
+static int check_list_validity(struct vdec_h264_inst *inst, bool disp_list)
+{
+	struct h264_ring_fb_list *list;
+
+	list = disp_list ? &inst->vsi->list_disp : &inst->vsi->list_free;
+
+	if (list->count > H264_MAX_FB_NUM ||
+	    list->read_idx >= H264_MAX_FB_NUM ||
+	    list->write_idx >= H264_MAX_FB_NUM) {
+		mtk_vcodec_err(inst, "%s list err: cnt=%d r_idx=%d w_idx=%d",
+			       disp_list ? "disp" : "free", list->count,
+			       list->read_idx, list->write_idx);
+		return -EINVAL;
+	}
+
+	return 0;
+}
+
+static void put_fb_to_free(struct vdec_h264_inst *inst, struct vdec_fb *fb)
+{
+	struct h264_ring_fb_list *list;
+
+	if (fb) {
+		if (check_list_validity(inst, false))
+			return;
+
+		list = &inst->vsi->list_free;
+		if (list->count == H264_MAX_FB_NUM) {
+			mtk_vcodec_err(inst, "[FB] put fb free_list full");
+			return;
+		}
+
+		mtk_vcodec_debug(inst, "[FB] put fb into free_list @(%p, %llx)",
+				 fb->base_y.va, (u64)fb->base_y.dma_addr);
+
+		list->fb_list[list->write_idx].vdec_fb_va = (u64)fb;
+		list->write_idx = (list->write_idx == H264_MAX_FB_NUM - 1) ?
+				  0 : list->write_idx + 1;
+		list->count++;
+	}
+}
+
+static void get_pic_info(struct vdec_h264_inst *inst,
+			 struct vdec_pic_info *pic)
+{
+	*pic = inst->vsi->pic;
+	mtk_vcodec_debug(inst, "pic(%d, %d), buf(%d, %d)",
+			 pic->pic_w, pic->pic_h, pic->buf_w, pic->buf_h);
+	mtk_vcodec_debug(inst, "Y(%d, %d), C(%d, %d)", pic->y_bs_sz,
+			 pic->y_len_sz, pic->c_bs_sz, pic->c_len_sz);
+}
+
+static void get_crop_info(struct vdec_h264_inst *inst, struct v4l2_rect *cr)
+{
+	cr->left = inst->vsi->crop.left;
+	cr->top = inst->vsi->crop.top;
+	cr->width = inst->vsi->crop.width;
+	cr->height = inst->vsi->crop.height;
+
+	mtk_vcodec_debug(inst, "l=%d, t=%d, w=%d, h=%d",
+			 cr->left, cr->top, cr->width, cr->height);
+}
+
+static void get_dpb_size(struct vdec_h264_inst *inst, unsigned int *dpb_sz)
+{
+	*dpb_sz = inst->vsi->dec.dpb_sz;
+	mtk_vcodec_debug(inst, "sz=%d", *dpb_sz);
+}
+
+static int vdec_h264_init(struct mtk_vcodec_ctx *ctx, unsigned long *h_vdec)
+{
+	struct vdec_h264_inst *inst = NULL;
+	int err;
+
+	inst = kzalloc(sizeof(*inst), GFP_KERNEL);
+	if (!inst)
+		return -ENOMEM;
+
+	inst->ctx = ctx;
+
+	inst->vpu.id = IPI_VDEC_H264;
+	inst->vpu.dev = ctx->dev->vpu_plat_dev;
+	inst->vpu.ctx = ctx;
+	inst->vpu.handler = vpu_dec_ipi_handler;
+
+	err = vpu_dec_init(&inst->vpu);
+	if (err) {
+		mtk_vcodec_err(inst, "vdec_h264 init err=%d", err);
+		goto error_free_inst;
+	}
+
+	inst->vsi = (struct vdec_h264_vsi *)inst->vpu.vsi;
+	err = allocate_predication_buf(inst);
+	if (err)
+		goto error_deinit;
+
+	mtk_vcodec_debug(inst, "H264 Instance >> %p", inst);
+
+	*h_vdec = (unsigned long)inst;
+	return 0;
+
+error_deinit:
+	vpu_dec_deinit(&inst->vpu);
+
+error_free_inst:
+	kfree(inst);
+	return err;
+}
+
+static void vdec_h264_deinit(unsigned long h_vdec)
+{
+	struct vdec_h264_inst *inst = (struct vdec_h264_inst *)h_vdec;
+
+	mtk_vcodec_debug_enter(inst);
+
+	vpu_dec_deinit(&inst->vpu);
+	free_predication_buf(inst);
+	free_mv_buf(inst);
+
+	kfree(inst);
+}
+
+static int find_start_code(unsigned char *data, unsigned int data_sz)
+{
+	if (data_sz > 3 && data[0] == 0 && data[1] == 0 && data[2] == 1)
+		return 3;
+
+	if (data_sz > 4 && data[0] == 0 && data[1] == 0 && data[2] == 0 &&
+	    data[3] == 1)
+		return 4;
+
+	return -1;
+}
+
+static int vdec_h264_decode(unsigned long h_vdec, struct mtk_vcodec_mem *bs,
+			    struct vdec_fb *fb, bool *res_chg)
+{
+	struct vdec_h264_inst *inst = (struct vdec_h264_inst *)h_vdec;
+	struct vdec_vpu_inst *vpu = &inst->vpu;
+	int nal_start_idx = 0;
+	int err = 0;
+	unsigned int nal_start;
+	unsigned int nal_type;
+	unsigned char *buf;
+	unsigned int buf_sz;
+	unsigned int data[2];
+	uint64_t vdec_fb_va = (u64)fb;
+	uint64_t y_fb_dma = fb ? (u64)fb->base_y.dma_addr : 0;
+	uint64_t c_fb_dma = fb ? (u64)fb->base_c.dma_addr : 0;
+
+	mtk_vcodec_debug(inst, "+ [%d] FB y_dma=%llx c_dma=%llx va=%p",
+			 ++inst->num_nalu, y_fb_dma, c_fb_dma, fb);
+
+	/* bs NULL means flush decoder */
+	if (bs == NULL)
+		return vpu_dec_reset(vpu);
+
+	buf = (unsigned char *)bs->va;
+	buf_sz = bs->size;
+	nal_start_idx = find_start_code(buf, buf_sz);
+	if (nal_start_idx < 0)
+		goto err_free_fb_out;
+
+	nal_start = buf[nal_start_idx];
+	nal_type = NAL_TYPE(buf[nal_start_idx]);
+	mtk_vcodec_debug(inst, "\n + NALU[%d] type %d +\n", inst->num_nalu,
+			 nal_type);
+
+	if (nal_type == NAL_H264_PPS) {
+		buf_sz -= nal_start_idx;
+		if (buf_sz > HDR_PARSING_BUF_SZ) {
+			err = -EILSEQ;
+			goto err_free_fb_out;
+		}
+		memcpy(inst->vsi->hdr_buf, buf + nal_start_idx, buf_sz);
+	}
+
+	inst->vsi->dec.bs_dma = (uint64_t)bs->dma_addr;
+	inst->vsi->dec.y_fb_dma = y_fb_dma;
+	inst->vsi->dec.c_fb_dma = c_fb_dma;
+	inst->vsi->dec.vdec_fb_va = vdec_fb_va;
+
+	data[0] = buf_sz;
+	data[1] = nal_start;
+	err = vpu_dec_start(vpu, data, 2);
+	if (err)
+		goto err_free_fb_out;
+
+	*res_chg = inst->vsi->dec.resolution_changed;
+	if (*res_chg) {
+		struct vdec_pic_info pic;
+
+		mtk_vcodec_debug(inst, "- resolution changed -");
+		get_pic_info(inst, &pic);
+
+		if (inst->vsi->dec.realloc_mv_buf) {
+			err = alloc_mv_buf(inst, &pic);
+			if (err)
+				goto err_free_fb_out;
+		}
+	}
+
+	if (nal_type == NAL_NON_IDR_SLICE || nal_type == NAL_IDR_SLICE) {
+		/* wait decoder done interrupt */
+		err = mtk_vcodec_wait_for_done_ctx(inst->ctx,
+						   MTK_INST_IRQ_RECEIVED,
+						   WAIT_INTR_TIMEOUT_MS);
+		if (err)
+			goto err_free_fb_out;
+
+		vpu_dec_end(vpu);
+	}
+
+	mtk_vcodec_debug(inst, "\n - NALU[%d] type=%d -\n", inst->num_nalu,
+			 nal_type);
+	return 0;
+
+err_free_fb_out:
+	put_fb_to_free(inst, fb);
+	mtk_vcodec_err(inst, "\n - NALU[%d] err=%d -\n", inst->num_nalu, err);
+	return err;
+}
+
+static void vdec_h264_get_fb(struct vdec_h264_inst *inst,
+			     struct h264_ring_fb_list *list,
+			     bool disp_list, struct vdec_fb **out_fb)
+{
+	struct vdec_fb *fb;
+
+	if (check_list_validity(inst, disp_list))
+		return;
+
+	if (list->count == 0) {
+		mtk_vcodec_debug(inst, "[FB] there is no %s fb",
+				 disp_list ? "disp" : "free");
+		*out_fb = NULL;
+		return;
+	}
+
+	fb = (struct vdec_fb *)list->fb_list[list->read_idx].vdec_fb_va;
+	fb->status |= (disp_list ? FB_ST_DISPLAY : FB_ST_FREE);
+
+	*out_fb = fb;
+	mtk_vcodec_debug(inst, "[FB] get %s fb st=%d poc=%d %llx",
+			 disp_list ? "disp" : "free",
+			 fb->status, list->fb_list[list->read_idx].poc,
+			 list->fb_list[list->read_idx].vdec_fb_va);
+
+	list->read_idx = (list->read_idx == H264_MAX_FB_NUM - 1) ?
+			 0 : list->read_idx + 1;
+	list->count--;
+}
+
+static int vdec_h264_get_param(unsigned long h_vdec,
+			       enum vdec_get_param_type type, void *out)
+{
+	struct vdec_h264_inst *inst = (struct vdec_h264_inst *)h_vdec;
+
+	switch (type) {
+	case GET_PARAM_DISP_FRAME_BUFFER:
+		vdec_h264_get_fb(inst, &inst->vsi->list_disp, true, out);
+		break;
+
+	case GET_PARAM_FREE_FRAME_BUFFER:
+		vdec_h264_get_fb(inst, &inst->vsi->list_free, false, out);
+		break;
+
+	case GET_PARAM_PIC_INFO:
+		get_pic_info(inst, out);
+		break;
+
+	case GET_PARAM_DPB_SIZE:
+		get_dpb_size(inst, out);
+		break;
+
+	case GET_PARAM_CROP_INFO:
+		get_crop_info(inst, out);
+		break;
+
+	default:
+		mtk_vcodec_err(inst, "invalid get parameter type=%d", type);
+		return -EINVAL;
+	}
+
+	return 0;
+}
+
+static struct vdec_common_if vdec_h264_if = {
+	vdec_h264_init,
+	vdec_h264_decode,
+	vdec_h264_get_param,
+	vdec_h264_deinit,
+};
+
+struct vdec_common_if *get_h264_dec_comm_if(void);
+
+struct vdec_common_if *get_h264_dec_comm_if(void)
+{
+	return &vdec_h264_if;
+}
diff --git a/drivers/media/platform/mtk-vcodec/vdec_drv_if.c b/drivers/media/platform/mtk-vcodec/vdec_drv_if.c
index 8202b14..e61f541 100644
--- a/drivers/media/platform/mtk-vcodec/vdec_drv_if.c
+++ b/drivers/media/platform/mtk-vcodec/vdec_drv_if.c
@@ -24,12 +24,16 @@
 #include "mtk_vcodec_util.h"
 #include "mtk_vpu.h"
 
+struct vdec_common_if *get_h264_dec_comm_if(void);
+
 int vdec_if_init(struct mtk_vcodec_ctx *ctx, unsigned int fourcc)
 {
 	int ret = 0;
 
 	switch (fourcc) {
 	case V4L2_PIX_FMT_H264:
+		ctx->dec_if = get_h264_dec_comm_if();
+		break;
 	case V4L2_PIX_FMT_VP8:
 	case V4L2_PIX_FMT_VP9:
 	default:
diff --git a/drivers/media/platform/mtk-vcodec/vdec_vpu_if.c b/drivers/media/platform/mtk-vcodec/vdec_vpu_if.c
new file mode 100644
index 0000000..0798a6b
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/vdec_vpu_if.c
@@ -0,0 +1,168 @@
+/*
+ * Copyright (c) 2016 MediaTek Inc.
+ * Author: PC Chen <pc.chen@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#include "mtk_vcodec_drv.h"
+#include "mtk_vcodec_util.h"
+#include "vdec_ipi_msg.h"
+#include "vdec_vpu_if.h"
+
+static void handle_init_ack_msg(struct vdec_vpu_ipi_init_ack *msg)
+{
+	struct vdec_vpu_inst *vpu = (struct vdec_vpu_inst *)msg->ap_inst_addr;
+
+	mtk_vcodec_debug(vpu, "+ ap_inst_addr = 0x%llx", msg->ap_inst_addr);
+
+	/* mapping VPU address to kernel virtual address */
+	/* the content in vsi is initialized to 0 in VPU */
+	vpu->vsi = vpu_mapping_dm_addr(vpu->dev, msg->vpu_inst_addr);
+	vpu->inst_addr = msg->vpu_inst_addr;
+
+	mtk_vcodec_debug(vpu, "- vpu_inst_addr = 0x%x", vpu->inst_addr);
+}
+
+/*
+ * This function runs in interrupt context and it means there's an IPI MSG
+ * from VPU.
+ */
+void vpu_dec_ipi_handler(void *data, unsigned int len, void *priv)
+{
+	struct vdec_vpu_ipi_ack *msg = data;
+	struct vdec_vpu_inst *vpu = (struct vdec_vpu_inst *)msg->ap_inst_addr;
+
+	mtk_vcodec_debug(vpu, "+ id=%X", msg->msg_id);
+
+	if (msg->status == 0) {
+		switch (msg->msg_id) {
+		case VPU_IPIMSG_DEC_INIT_ACK:
+			handle_init_ack_msg(data);
+			break;
+
+		case VPU_IPIMSG_DEC_START_ACK:
+		case VPU_IPIMSG_DEC_END_ACK:
+		case VPU_IPIMSG_DEC_DEINIT_ACK:
+		case VPU_IPIMSG_DEC_RESET_ACK:
+			break;
+
+		default:
+			mtk_vcodec_err(vpu, "invalid msg=%X", msg->msg_id);
+			break;
+		}
+	}
+
+	mtk_vcodec_debug(vpu, "- id=%X", msg->msg_id);
+	vpu->failure = msg->status;
+	vpu->signaled = 1;
+}
+
+static int vcodec_vpu_send_msg(struct vdec_vpu_inst *vpu, void *msg, int len)
+{
+	int err;
+	uint32_t msg_id = *(uint32_t *)msg;
+
+	mtk_vcodec_debug(vpu, "id=%X", msg_id);
+
+	vpu->failure = 0;
+	vpu->signaled = 0;
+
+	err = vpu_ipi_send(vpu->dev, vpu->id, msg, len);
+	if (err) {
+		mtk_vcodec_err(vpu, "send fail vpu_id=%d msg_id=%X status=%d",
+			       vpu->id, msg_id, err);
+		return err;
+	}
+
+	return vpu->failure;
+}
+
+static int vcodec_send_ap_ipi(struct vdec_vpu_inst *vpu, unsigned int msg_id)
+{
+	struct vdec_ap_ipi_cmd msg;
+	int err = 0;
+
+	mtk_vcodec_debug(vpu, "+ id=%X", msg_id);
+
+	memset(&msg, 0, sizeof(msg));
+	msg.msg_id = msg_id;
+	msg.vpu_inst_addr = vpu->inst_addr;
+
+	err = vcodec_vpu_send_msg(vpu, &msg, sizeof(msg));
+	mtk_vcodec_debug(vpu, "- id=%X ret=%d", msg_id, err);
+	return err;
+}
+
+int vpu_dec_init(struct vdec_vpu_inst *vpu)
+{
+	struct vdec_ap_ipi_init msg;
+	int err;
+
+	mtk_vcodec_debug_enter(vpu);
+
+	init_waitqueue_head(&vpu->wq);
+
+	err = vpu_ipi_register(vpu->dev, vpu->id, vpu->handler, "vdec", NULL);
+	if (err != 0) {
+		mtk_vcodec_err(vpu, "vpu_ipi_register fail status=%d", err);
+		return err;
+	}
+
+	memset(&msg, 0, sizeof(msg));
+	msg.msg_id = AP_IPIMSG_DEC_INIT;
+	msg.ap_inst_addr = (unsigned long)vpu;
+
+	mtk_vcodec_debug(vpu, "vdec_inst=%p", vpu);
+
+	err = vcodec_vpu_send_msg(vpu, (void *)&msg, sizeof(msg));
+	mtk_vcodec_debug(vpu, "- ret=%d", err);
+	return err;
+}
+
+int vpu_dec_start(struct vdec_vpu_inst *vpu, uint32_t *data, unsigned int len)
+{
+	struct vdec_ap_ipi_dec_start msg;
+	int i;
+	int err = 0;
+
+	mtk_vcodec_debug_enter(vpu);
+
+	if (len > ARRAY_SIZE(msg.data)) {
+		mtk_vcodec_err(vpu, "invalid len = %d\n", len);
+		return -EINVAL;
+	}
+
+	memset(&msg, 0, sizeof(msg));
+	msg.msg_id = AP_IPIMSG_DEC_START;
+	msg.vpu_inst_addr = vpu->inst_addr;
+
+	for (i = 0; i < len; i++)
+		msg.data[i] = data[i];
+
+	err = vcodec_vpu_send_msg(vpu, (void *)&msg, sizeof(msg));
+	mtk_vcodec_debug(vpu, "- ret=%d", err);
+	return err;
+}
+
+int vpu_dec_end(struct vdec_vpu_inst *vpu)
+{
+	return vcodec_send_ap_ipi(vpu, AP_IPIMSG_DEC_END);
+}
+
+int vpu_dec_deinit(struct vdec_vpu_inst *vpu)
+{
+	return vcodec_send_ap_ipi(vpu, AP_IPIMSG_DEC_DEINIT);
+}
+
+int vpu_dec_reset(struct vdec_vpu_inst *vpu)
+{
+	return vcodec_send_ap_ipi(vpu, AP_IPIMSG_DEC_RESET);
+}
diff --git a/drivers/media/platform/mtk-vcodec/vdec_vpu_if.h b/drivers/media/platform/mtk-vcodec/vdec_vpu_if.h
new file mode 100644
index 0000000..8dcc1cb
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/vdec_vpu_if.h
@@ -0,0 +1,96 @@
+/*
+ * Copyright (c) 2016 MediaTek Inc.
+ * Author: PC Chen <pc.chen@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef _VDEC_VPU_IF_H_
+#define _VDEC_VPU_IF_H_
+
+#include "mtk_vpu.h"
+
+/**
+ * struct vdec_vpu_inst - VPU instance for video codec
+ * @ipi_id      : ipi id for each decoder
+ * @vsi         : driver structure allocated by VPU side and shared to AP side
+ *                for control and info share
+ * @failure     : VPU execution result status, 0: success, others: fail
+ * @inst_addr	: VPU decoder instance address
+ * @signaled    : 1 - Host has received ack message from VPU, 0 - not received
+ * @ctx         : context for v4l2 layer integration
+ * @dev	        : platform device of VPU
+ * @wq          : wait queue to wait VPU message ack
+ * @handler     : ipi handler for each decoder
+ */
+struct vdec_vpu_inst {
+	enum ipi_id id;
+	void *vsi;
+	int32_t failure;
+	uint32_t inst_addr;
+	unsigned int signaled;
+	struct mtk_vcodec_ctx *ctx;
+	struct platform_device *dev;
+	wait_queue_head_t wq;
+	ipi_handler_t handler;
+};
+
+/**
+ * vpu_dec_init - init decoder instance and allocate required resource in VPU.
+ *
+ * @vpu: instance for vdec_vpu_inst
+ */
+int vpu_dec_init(struct vdec_vpu_inst *vpu);
+
+/**
+ * vpu_dec_start - start decoding, basically the function will be invoked once
+ *                 every frame.
+ *
+ * @vpu : instance for vdec_vpu_inst
+ * @data: meta data to pass bitstream info to VPU decoder
+ * @len : meta data length
+ */
+int vpu_dec_start(struct vdec_vpu_inst *vpu, uint32_t *data, unsigned int len);
+
+/**
+ * vpu_dec_end - end decoding, basically the function will be invoked once
+ *               when HW decoding done interrupt received successfully. The
+ *               decoder in VPU will continute to do referene frame management
+ *               and check if there is a new decoded frame available to display.
+ *
+ * @vpu : instance for vdec_vpu_inst
+ */
+int vpu_dec_end(struct vdec_vpu_inst *vpu);
+
+/**
+ * vpu_dec_deinit - deinit decoder instance and resource freed in VPU.
+ *
+ * @vpu: instance for vdec_vpu_inst
+ */
+int vpu_dec_deinit(struct vdec_vpu_inst *vpu);
+
+/**
+ * vpu_dec_reset - reset decoder, use for flush decoder when end of stream or
+ *                 seek. Remainig non displayed frame will be pushed to display.
+ *
+ * @vpu: instance for vdec_vpu_inst
+ */
+int vpu_dec_reset(struct vdec_vpu_inst *vpu);
+
+/**
+ * vpu_dec_ipi_handler - Handler for VPU ipi message.
+ *
+ * @data: ipi message
+ * @len : length of ipi message
+ * @priv: callback private data whcih is passed by decoder when register.
+ */
+void vpu_dec_ipi_handler(void *data, unsigned int len, void *priv);
+
+#endif
-- 
1.7.9.5

[PATCH v3 7/9] vcodec: mediatek: Add Mediatek VP8 Video Decoder Driver

[Tiffany Lin]

Add vp8 decoder driver for MT8173

Signed-off-by: PC Chen <pc.chen@mediatek.com>
Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
---
 drivers/media/platform/mtk-vcodec/Makefile         |    1 +
 .../media/platform/mtk-vcodec/vdec/vdec_vp8_if.c   |  630 ++++++++++++++++++++
 drivers/media/platform/mtk-vcodec/vdec_drv_if.c    |    3 +
 3 files changed, 634 insertions(+)
 create mode 100644 drivers/media/platform/mtk-vcodec/vdec/vdec_vp8_if.c

diff --git a/drivers/media/platform/mtk-vcodec/Makefile b/drivers/media/platform/mtk-vcodec/Makefile
index 58243ed..7743c81 100644
--- a/drivers/media/platform/mtk-vcodec/Makefile
+++ b/drivers/media/platform/mtk-vcodec/Makefile
@@ -4,6 +4,7 @@ obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += mtk-vcodec-dec.o \
 				       mtk-vcodec-common.o
 
 mtk-vcodec-dec-y := vdec/vdec_h264_if.o \
+		vdec/vdec_vp8_if.o \
 		mtk_vcodec_dec_drv.o \
 		vdec_drv_if.o \
 		vdec_vpu_if.o \
diff --git a/drivers/media/platform/mtk-vcodec/vdec/vdec_vp8_if.c b/drivers/media/platform/mtk-vcodec/vdec/vdec_vp8_if.c
new file mode 100644
index 0000000..0c86df0
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/vdec/vdec_vp8_if.c
@@ -0,0 +1,630 @@
+/*
+ * Copyright (c) 2016 MediaTek Inc.
+ * Author: Jungchang Tsao <jungchang.tsao@mediatek.com>
+ *	   PC Chen <pc.chen@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#include "../mtk_vcodec_intr.h"
+#include "../vdec_vpu_if.h"
+#include "../vdec_drv_base.h"
+
+/* Decoding picture buffer size (3 reference frames plus current frame) */
+#define VP8_DPB_SIZE			4
+
+/* HW working buffer size (bytes) */
+#define VP8_WORKING_BUF_SZ		(45 * 4096)
+
+/* HW control register address */
+#define VP8_SEGID_DRAM_ADDR		0x3c
+#define VP8_HW_VLD_ADDR			0x93C
+#define VP8_HW_VLD_VALUE		0x940
+#define VP8_BSASET			0x100
+#define VP8_BSDSET			0x104
+#define VP8_RW_CKEN_SET			0x0
+#define VP8_RW_DCM_CON			0x18
+#define VP8_WO_VLD_SRST			0x108
+#define VP8_RW_MISC_SYS_SEL		0x84
+#define VP8_RW_MISC_SPEC_CON		0xC8
+#define VP8_WO_VLD_SRST			0x108
+#define VP8_RW_VP8_CTRL			0xA4
+#define VP8_RW_MISC_DCM_CON		0xEC
+#define VP8_RW_MISC_SRST		0xF4
+#define VP8_RW_MISC_FUNC_CON		0xCC
+
+#define VP8_MAX_FRM_BUF_NUM		5
+#define VP8_MAX_FRM_BUF_NODE_NUM	(VP8_MAX_FRM_BUF_NUM * 2)
+
+/* required buffer size (bytes) to store decode information */
+#define VP8_HW_SEGMENT_DATA_SZ		272
+#define VP8_HW_SEGMENT_UINT		4
+
+#define VP8_DEC_TABLE_PROC_LOOP		96
+#define VP8_DEC_TABLE_UNIT		3
+#define VP8_DEC_TABLE_SZ		300
+#define VP8_DEC_TABLE_OFFSET		2
+#define VP8_DEC_TABLE_RW_UNIT		4
+
+/**
+ * struct vdec_vp8_dec_info - decode misc information
+ * @working_buf_dma   : working buffer dma address
+ * @prev_y_dma        : previous decoded frame buffer Y plane address
+ * @cur_y_fb_dma      : current plane Y frame buffer dma address
+ * @cur_c_fb_dma      : current plane C frame buffer dma address
+ * @bs_dma	      : bitstream dma address
+ * @bs_sz	      : bitstream size
+ * @resolution_changed: resolution change flag 1 - changed,  0 - not change
+ * @show_frame	      : display this frame or not
+ * @wait_key_frame    : wait key frame coming
+ */
+struct vdec_vp8_dec_info {
+	uint64_t working_buf_dma;
+	uint64_t prev_y_dma;
+	uint64_t cur_y_fb_dma;
+	uint64_t cur_c_fb_dma;
+	uint64_t bs_dma;
+	uint32_t bs_sz;
+	uint32_t resolution_changed;
+	uint32_t show_frame;
+	uint32_t wait_key_frame;
+};
+
+/**
+ * struct vdec_vp8_vsi - VPU shared information
+ * @dec			: decoding information
+ * @pic			: picture information
+ * @dec_table		: decoder coefficient table
+ * @segment_buf		: segmentation buffer
+ * @load_data		: flag to indicate reload decode data
+ */
+struct vdec_vp8_vsi {
+	struct vdec_vp8_dec_info dec;
+	struct vdec_pic_info pic;
+	uint32_t dec_table[VP8_DEC_TABLE_SZ];
+	uint32_t segment_buf[VP8_HW_SEGMENT_DATA_SZ][VP8_HW_SEGMENT_UINT];
+	uint32_t load_data;
+};
+
+/**
+ * struct vdec_vp8_hw_reg_base - HW register base
+ * @sys		: base address for sys
+ * @misc	: base address for misc
+ * @ld		: base address for ld
+ * @top		: base address for top
+ * @cm		: base address for cm
+ * @hwd		: base address for hwd
+ * @hwb		: base address for hwb
+*/
+struct vdec_vp8_hw_reg_base {
+	void __iomem *sys;
+	void __iomem *misc;
+	void __iomem *ld;
+	void __iomem *top;
+	void __iomem *cm;
+	void __iomem *hwd;
+	void __iomem *hwb;
+};
+
+/**
+ * struct vdec_vp8_vpu_inst - VPU instance for VP8 decode
+ * @wq_hd	: Wait queue to wait VPU message ack
+ * @signaled	: 1 - Host has received ack message from VPU, 0 - not recevie
+ * @failure	: VPU execution result status 0 - success, others - fail
+ * @inst_addr	: VPU decoder instance address
+ */
+struct vdec_vp8_vpu_inst {
+	wait_queue_head_t wq_hd;
+	int signaled;
+	int failure;
+	uint32_t inst_addr;
+};
+
+/* frame buffer (fb) list
+ * [available_fb_node_list]  - decode fb are initialized to 0 and populated in
+ * [fb_use_list]  - fb is set after decode and is moved to this list
+ * [fb_free_list] - fb is not needed for reference will be moved from
+ *		     [fb_use_list] to [fb_free_list] and
+ *		     once user remove fb from [fb_free_list],
+ *		     it is circulated back to [available_fb_node_list]
+ * [fb_disp_list] - fb is set after decode and is moved to this list
+ *                   once user remove fb from [fb_disp_list] it is
+ *                   circulated back to [available_fb_node_list]
+ */
+
+/**
+ * struct vdec_vp8_inst - VP8 decoder instance
+ * @cur_fb		   : current frame buffer
+ * @dec_fb		   : decode frame buffer node
+ * @available_fb_node_list : list to store available frame buffer node
+ * @fb_use_list		   : list to store frame buffer in use
+ * @fb_free_list	   : list to store free frame buffer
+ * @fb_disp_list	   : list to store display ready frame buffer
+ * @working_buf		   : HW decoder working buffer
+ * @reg_base		   : HW register base address
+ * @frm_cnt		   : decode frame count
+ * @ctx			   : V4L2 context
+ * @dev			   : platform device
+ * @vpu			   : VPU instance for decoder
+ * @vsi			   : VPU share information
+ */
+struct vdec_vp8_inst {
+	struct vdec_fb *cur_fb;
+	struct vdec_fb_node dec_fb[VP8_MAX_FRM_BUF_NODE_NUM];
+	struct list_head available_fb_node_list;
+	struct list_head fb_use_list;
+	struct list_head fb_free_list;
+	struct list_head fb_disp_list;
+	struct mtk_vcodec_mem working_buf;
+	struct vdec_vp8_hw_reg_base reg_base;
+	unsigned int frm_cnt;
+	struct mtk_vcodec_ctx *ctx;
+	struct vdec_vpu_inst vpu;
+	struct vdec_vp8_vsi *vsi;
+};
+
+static void get_hw_reg_base(struct vdec_vp8_inst *inst)
+{
+	inst->reg_base.top = mtk_vcodec_get_reg_addr(inst->ctx, VDEC_TOP);
+	inst->reg_base.cm = mtk_vcodec_get_reg_addr(inst->ctx, VDEC_CM);
+	inst->reg_base.hwd = mtk_vcodec_get_reg_addr(inst->ctx, VDEC_HWD);
+	inst->reg_base.sys = mtk_vcodec_get_reg_addr(inst->ctx, VDEC_SYS);
+	inst->reg_base.misc = mtk_vcodec_get_reg_addr(inst->ctx, VDEC_MISC);
+	inst->reg_base.ld = mtk_vcodec_get_reg_addr(inst->ctx, VDEC_LD);
+	inst->reg_base.hwb = mtk_vcodec_get_reg_addr(inst->ctx, VDEC_HWB);
+}
+
+static void write_hw_segmentation_data(struct vdec_vp8_inst *inst)
+{
+	int i, j;
+	u32 seg_id_addr;
+	u32 val;
+	void __iomem *cm = inst->reg_base.cm;
+	struct vdec_vp8_vsi *vsi = inst->vsi;
+
+	seg_id_addr = readl(inst->reg_base.top + VP8_SEGID_DRAM_ADDR) >> 4;
+
+	for (i = 0; i < ARRAY_SIZE(vsi->segment_buf); i++) {
+		for (j = ARRAY_SIZE(vsi->segment_buf[i]) - 1; j >= 0; j--) {
+			val = (1 << 16) + ((seg_id_addr + i) << 2) + j;
+			writel(val, cm + VP8_HW_VLD_ADDR);
+
+			val = vsi->segment_buf[i][j];
+			writel(val, cm + VP8_HW_VLD_VALUE);
+		}
+	}
+}
+
+static void read_hw_segmentation_data(struct vdec_vp8_inst *inst)
+{
+	int i, j;
+	u32 seg_id_addr;
+	u32 val;
+	void __iomem *cm = inst->reg_base.cm;
+	struct vdec_vp8_vsi *vsi = inst->vsi;
+
+	seg_id_addr = readl(inst->reg_base.top + VP8_SEGID_DRAM_ADDR) >> 4;
+
+	for (i = 0; i < ARRAY_SIZE(vsi->segment_buf); i++) {
+		for (j = ARRAY_SIZE(vsi->segment_buf[i]) - 1; j >= 0; j--) {
+			val = ((seg_id_addr + i) << 2) + j;
+			writel(val, cm + VP8_HW_VLD_ADDR);
+
+			val = readl(cm + VP8_HW_VLD_VALUE);
+			vsi->segment_buf[i][j] = val;
+		}
+	}
+}
+
+/* reset HW and enable HW read/write data function */
+static void enable_hw_rw_function(struct vdec_vp8_inst *inst)
+{
+	u32 val = 0;
+	void __iomem *sys = inst->reg_base.sys;
+	void __iomem *misc = inst->reg_base.misc;
+	void __iomem *ld = inst->reg_base.ld;
+	void __iomem *hwb = inst->reg_base.hwb;
+	void __iomem *hwd = inst->reg_base.hwd;
+
+	writel(0x1, sys + VP8_RW_CKEN_SET);
+	writel(0x101, ld + VP8_WO_VLD_SRST);
+	writel(0x101, hwb + VP8_WO_VLD_SRST);
+
+	writel(1, sys);
+	val = readl(misc + VP8_RW_MISC_SRST);
+	writel((val & 0xFFFFFFFE), misc + VP8_RW_MISC_SRST);
+
+	writel(0x1, misc + VP8_RW_MISC_SYS_SEL);
+	writel(0x17F, misc + VP8_RW_MISC_SPEC_CON);
+	writel(0x71201100, misc + VP8_RW_MISC_FUNC_CON);
+	writel(0x0, ld + VP8_WO_VLD_SRST);
+	writel(0x0, hwb + VP8_WO_VLD_SRST);
+	writel(0x1, sys + VP8_RW_DCM_CON);
+	writel(0x1, misc + VP8_RW_MISC_DCM_CON);
+	writel(0x1, hwd + VP8_RW_VP8_CTRL);
+}
+
+static void store_dec_table(struct vdec_vp8_inst *inst)
+{
+	int i, j;
+	u32 addr = 0, val = 0;
+	void __iomem *hwd = inst->reg_base.hwd;
+	u32 *p = &inst->vsi->dec_table[VP8_DEC_TABLE_OFFSET];
+
+	for (i = 0; i < VP8_DEC_TABLE_PROC_LOOP; i++) {
+		writel(addr, hwd + VP8_BSASET);
+		for (j = 0; j < VP8_DEC_TABLE_UNIT ; j++) {
+			val = *p++;
+			writel(val, hwd + VP8_BSDSET);
+		}
+		addr += VP8_DEC_TABLE_RW_UNIT;
+	}
+}
+
+static void load_dec_table(struct vdec_vp8_inst *inst)
+{
+	int i;
+	u32 addr = 0;
+	u32 *p = &inst->vsi->dec_table[VP8_DEC_TABLE_OFFSET];
+	void __iomem *hwd = inst->reg_base.hwd;
+
+	for (i = 0; i < VP8_DEC_TABLE_PROC_LOOP; i++) {
+		writel(addr, hwd + VP8_BSASET);
+		/* read total 11 bytes */
+		*p++ = readl(hwd + VP8_BSDSET);
+		*p++ = readl(hwd + VP8_BSDSET);
+		*p++ = readl(hwd + VP8_BSDSET) & 0xFFFFFF;
+		addr += VP8_DEC_TABLE_RW_UNIT;
+	}
+}
+
+static void get_pic_info(struct vdec_vp8_inst *inst, struct vdec_pic_info *pic)
+{
+	*pic = inst->vsi->pic;
+
+	mtk_vcodec_debug(inst, "pic(%d, %d), buf(%d, %d)",
+			 pic->pic_w, pic->pic_h, pic->buf_w, pic->buf_h);
+	mtk_vcodec_debug(inst, "Y(%d, %d), C(%d, %d)", pic->y_bs_sz,
+			 pic->y_len_sz, pic->c_bs_sz, pic->c_len_sz);
+}
+
+static void vp8_dec_finish(struct vdec_vp8_inst *inst)
+{
+	struct vdec_fb_node *node;
+	uint64_t prev_y_dma = inst->vsi->dec.prev_y_dma;
+
+	mtk_vcodec_debug(inst, "prev fb base dma=%llx", prev_y_dma);
+
+	/* put last decode ok frame to fb_free_list */
+	if (prev_y_dma != 0) {
+		list_for_each_entry(node, &inst->fb_use_list, list) {
+			struct vdec_fb *fb = (struct vdec_fb *)node->fb;
+
+			if (prev_y_dma == (uint64_t)fb->base_y.dma_addr) {
+				list_move_tail(&node->list,
+					       &inst->fb_free_list);
+				break;
+			}
+		}
+	}
+
+	/* available_fb_node_list -> fb_use_list */
+	node = list_first_entry(&inst->available_fb_node_list,
+				struct vdec_fb_node, list);
+	node->fb = inst->cur_fb;
+	list_move_tail(&node->list, &inst->fb_use_list);
+
+	/* available_fb_node_list -> fb_disp_list */
+	if (inst->vsi->dec.show_frame) {
+		node = list_first_entry(&inst->available_fb_node_list,
+					struct vdec_fb_node, list);
+		node->fb = inst->cur_fb;
+		list_move_tail(&node->list, &inst->fb_disp_list);
+	}
+}
+
+static void move_fb_list_use_to_free(struct vdec_vp8_inst *inst)
+{
+	struct vdec_fb_node *node, *tmp;
+
+	list_for_each_entry_safe(node, tmp, &inst->fb_use_list, list)
+		list_move_tail(&node->list, &inst->fb_free_list);
+}
+
+static void init_list(struct vdec_vp8_inst *inst)
+{
+	int i;
+
+	INIT_LIST_HEAD(&inst->available_fb_node_list);
+	INIT_LIST_HEAD(&inst->fb_use_list);
+	INIT_LIST_HEAD(&inst->fb_free_list);
+	INIT_LIST_HEAD(&inst->fb_disp_list);
+
+	for (i = 0; i < ARRAY_SIZE(inst->dec_fb); i++) {
+		INIT_LIST_HEAD(&inst->dec_fb[i].list);
+		inst->dec_fb[i].fb = NULL;
+		list_add_tail(&inst->dec_fb[i].list,
+			      &inst->available_fb_node_list);
+	}
+}
+
+static void add_fb_to_free_list(struct vdec_vp8_inst *inst, void *fb)
+{
+	struct vdec_fb_node *node;
+
+	if (fb) {
+		node = list_first_entry(&inst->available_fb_node_list,
+					struct vdec_fb_node, list);
+		node->fb = fb;
+		list_move_tail(&node->list, &inst->fb_free_list);
+	}
+}
+
+static int alloc_working_buf(struct vdec_vp8_inst *inst)
+{
+	int err;
+	struct mtk_vcodec_mem *mem = &inst->working_buf;
+
+	mem->size = VP8_WORKING_BUF_SZ;
+	err = mtk_vcodec_mem_alloc(inst->ctx, mem);
+	if (err) {
+		mtk_vcodec_err(inst, "Cannot allocate working buffer");
+		return err;
+	}
+
+	inst->vsi->dec.working_buf_dma = (uint64_t)mem->dma_addr;
+	return 0;
+}
+
+static void free_working_buf(struct vdec_vp8_inst *inst)
+{
+	struct mtk_vcodec_mem *mem = &inst->working_buf;
+
+	if (mem->va)
+		mtk_vcodec_mem_free(inst->ctx, mem);
+
+	inst->vsi->dec.working_buf_dma = 0;
+}
+
+static int vdec_vp8_init(struct mtk_vcodec_ctx *ctx, unsigned long *h_vdec)
+{
+	struct vdec_vp8_inst *inst;
+	int err;
+
+	inst = kzalloc(sizeof(*inst), GFP_KERNEL);
+	if (!inst)
+		return  -ENOMEM;
+
+	inst->ctx = ctx;
+
+	inst->vpu.id = IPI_VDEC_VP8;
+	inst->vpu.dev = ctx->dev->vpu_plat_dev;
+	inst->vpu.ctx = ctx;
+	inst->vpu.handler = vpu_dec_ipi_handler;
+
+	err = vpu_dec_init(&inst->vpu);
+	if (err) {
+		mtk_vcodec_err(inst, "vdec_vp8 init err=%d", err);
+		goto error_free_inst;
+	}
+
+	inst->vsi = (struct vdec_vp8_vsi *)inst->vpu.vsi;
+	init_list(inst);
+	err = alloc_working_buf(inst);
+	if (err)
+		goto error_deinit;
+
+	get_hw_reg_base(inst);
+	mtk_vcodec_debug(inst, "VP8 Instance >> %p", inst);
+
+	*h_vdec = (unsigned long)inst;
+	return 0;
+
+error_deinit:
+	vpu_dec_deinit(&inst->vpu);
+error_free_inst:
+	kfree(inst);
+	return err;
+}
+
+static int vdec_vp8_decode(unsigned long h_vdec, struct mtk_vcodec_mem *bs,
+			   struct vdec_fb *fb, bool *res_chg)
+{
+	struct vdec_vp8_inst *inst = (struct vdec_vp8_inst *)h_vdec;
+	struct vdec_vp8_dec_info *dec = &inst->vsi->dec;
+	struct vdec_vpu_inst *vpu = &inst->vpu;
+	unsigned char *bs_va;
+	unsigned int data;
+	int err = 0;
+	uint64_t y_fb_dma;
+	uint64_t c_fb_dma;
+
+	/* bs NULL means flush decoder */
+	if (bs == NULL) {
+		move_fb_list_use_to_free(inst);
+		return vpu_dec_reset(vpu);
+	}
+
+	y_fb_dma = fb ? (u64)fb->base_y.dma_addr : 0;
+	c_fb_dma = fb ? (u64)fb->base_c.dma_addr : 0;
+
+	mtk_vcodec_debug(inst, "+ [%d] FB y_dma=%llx c_dma=%llx fb=%p",
+			 inst->frm_cnt, y_fb_dma, c_fb_dma, fb);
+
+	inst->cur_fb = fb;
+	dec->bs_dma = (unsigned long)bs->dma_addr;
+	dec->bs_sz = bs->size;
+	dec->cur_y_fb_dma = y_fb_dma;
+	dec->cur_c_fb_dma = c_fb_dma;
+
+	mtk_vcodec_debug(inst, "\n + FRAME[%d] +\n", inst->frm_cnt);
+
+	write_hw_segmentation_data(inst);
+	enable_hw_rw_function(inst);
+	store_dec_table(inst);
+
+	bs_va = (unsigned char *)bs->va;
+
+	/* retrieve width/hight and scale info from header */
+	data = (*(bs_va + 9) << 24) | (*(bs_va + 8) << 16) |
+	       (*(bs_va + 7) << 8) | *(bs_va + 6);
+	err = vpu_dec_start(vpu, &data, 1);
+	if (err) {
+		add_fb_to_free_list(inst, fb);
+		if (dec->wait_key_frame) {
+			mtk_vcodec_debug(inst, "wait key frame !");
+			return 0;
+		}
+
+		goto error;
+	}
+
+	if (dec->resolution_changed) {
+		mtk_vcodec_debug(inst, "- resolution_changed -");
+		*res_chg = true;
+		add_fb_to_free_list(inst, fb);
+		return 0;
+	}
+
+	/* wait decoder done interrupt */
+	mtk_vcodec_wait_for_done_ctx(inst->ctx, MTK_INST_IRQ_RECEIVED,
+				     WAIT_INTR_TIMEOUT_MS);
+
+	if (inst->vsi->load_data)
+		load_dec_table(inst);
+
+	vp8_dec_finish(inst);
+	read_hw_segmentation_data(inst);
+
+	err = vpu_dec_end(vpu);
+	if (err)
+		goto error;
+
+	mtk_vcodec_debug(inst, "\n - FRAME[%d] - show=%d\n", inst->frm_cnt,
+			 dec->show_frame);
+	inst->frm_cnt++;
+	*res_chg = false;
+	return 0;
+
+error:
+	mtk_vcodec_err(inst, "\n - FRAME[%d] - err=%d\n", inst->frm_cnt, err);
+	return err;
+}
+
+static void get_disp_fb(struct vdec_vp8_inst *inst, struct vdec_fb **out_fb)
+{
+	struct vdec_fb_node *node;
+	struct vdec_fb *fb;
+
+	node = list_first_entry_or_null(&inst->fb_disp_list,
+					struct vdec_fb_node, list);
+	if (node) {
+		list_move_tail(&node->list, &inst->available_fb_node_list);
+		fb = (struct vdec_fb *)node->fb;
+		fb->status |= FB_ST_DISPLAY;
+		mtk_vcodec_debug(inst, "[FB] get disp fb %p st=%d",
+				 node->fb, fb->status);
+	} else {
+		fb = NULL;
+		mtk_vcodec_debug(inst, "[FB] there is no disp fb");
+	}
+
+	*out_fb = fb;
+}
+
+static void get_free_fb(struct vdec_vp8_inst *inst, struct vdec_fb **out_fb)
+{
+	struct vdec_fb_node *node;
+	struct vdec_fb *fb;
+
+	node = list_first_entry_or_null(&inst->fb_free_list,
+					struct vdec_fb_node, list);
+	if (node) {
+		list_move_tail(&node->list, &inst->available_fb_node_list);
+		fb = (struct vdec_fb *)node->fb;
+		fb->status |= FB_ST_FREE;
+		mtk_vcodec_debug(inst, "[FB] get free fb %p st=%d",
+				 node->fb, fb->status);
+	} else {
+		fb = NULL;
+		mtk_vcodec_debug(inst, "[FB] there is no free fb");
+	}
+
+	*out_fb = fb;
+}
+
+static void get_crop_info(struct vdec_vp8_inst *inst, struct v4l2_rect *cr)
+{
+	cr->left = 0;
+	cr->top = 0;
+	cr->width = inst->vsi->pic.pic_w;
+	cr->height = inst->vsi->pic.pic_h;
+	mtk_vcodec_debug(inst, "get crop info l=%d, t=%d, w=%d, h=%d",
+			 cr->left, cr->top, cr->width, cr->height);
+}
+
+static int vdec_vp8_get_param(unsigned long h_vdec,
+			      enum vdec_get_param_type type, void *out)
+{
+	struct vdec_vp8_inst *inst = (struct vdec_vp8_inst *)h_vdec;
+
+	switch (type) {
+	case GET_PARAM_DISP_FRAME_BUFFER:
+		get_disp_fb(inst, out);
+		break;
+
+	case GET_PARAM_FREE_FRAME_BUFFER:
+		get_free_fb(inst, out);
+		break;
+
+	case GET_PARAM_PIC_INFO:
+		get_pic_info(inst, out);
+		break;
+
+	case GET_PARAM_CROP_INFO:
+		get_crop_info(inst, out);
+		break;
+
+	case GET_PARAM_DPB_SIZE:
+		*((unsigned int *)out) = VP8_DPB_SIZE;
+		break;
+
+	default:
+		mtk_vcodec_err(inst, "invalid get parameter type=%d", type);
+		return -EINVAL;
+	}
+
+	return 0;
+}
+
+static void vdec_vp8_deinit(unsigned long h_vdec)
+{
+	struct vdec_vp8_inst *inst = (struct vdec_vp8_inst *)h_vdec;
+
+	mtk_vcodec_debug_enter(inst);
+
+	vpu_dec_deinit(&inst->vpu);
+	free_working_buf(inst);
+	kfree(inst);
+}
+
+static struct vdec_common_if vdec_vp8_if = {
+	vdec_vp8_init,
+	vdec_vp8_decode,
+	vdec_vp8_get_param,
+	vdec_vp8_deinit,
+};
+
+struct vdec_common_if *get_vp8_dec_comm_if(void);
+
+struct vdec_common_if *get_vp8_dec_comm_if(void)
+{
+	return &vdec_vp8_if;
+}
diff --git a/drivers/media/platform/mtk-vcodec/vdec_drv_if.c b/drivers/media/platform/mtk-vcodec/vdec_drv_if.c
index e61f541..5338171 100644
--- a/drivers/media/platform/mtk-vcodec/vdec_drv_if.c
+++ b/drivers/media/platform/mtk-vcodec/vdec_drv_if.c
@@ -25,6 +25,7 @@
 #include "mtk_vpu.h"
 
 struct vdec_common_if *get_h264_dec_comm_if(void);
+struct vdec_common_if *get_vp8_dec_comm_if(void);
 
 int vdec_if_init(struct mtk_vcodec_ctx *ctx, unsigned int fourcc)
 {
@@ -35,6 +36,8 @@ int vdec_if_init(struct mtk_vcodec_ctx *ctx, unsigned int fourcc)
 		ctx->dec_if = get_h264_dec_comm_if();
 		break;
 	case V4L2_PIX_FMT_VP8:
+		ctx->dec_if = get_vp8_dec_comm_if();
+		break;
 	case V4L2_PIX_FMT_VP9:
 	default:
 		return -EINVAL;
-- 
1.7.9.5

[PATCH v3 8/9] vcodec: mediatek: Add Mediatek VP9 Video Decoder Driver

[Tiffany Lin]

Add vp9 decoder driver for MT8173

Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
---
 drivers/media/platform/mtk-vcodec/Makefile         |    1 +
 .../media/platform/mtk-vcodec/vdec/vdec_vp9_if.c   |  967 ++++++++++++++++++++
 drivers/media/platform/mtk-vcodec/vdec_drv_if.c    |    3 +
 3 files changed, 971 insertions(+)
 create mode 100644 drivers/media/platform/mtk-vcodec/vdec/vdec_vp9_if.c

diff --git a/drivers/media/platform/mtk-vcodec/Makefile b/drivers/media/platform/mtk-vcodec/Makefile
index 7743c81..852d969 100644
--- a/drivers/media/platform/mtk-vcodec/Makefile
+++ b/drivers/media/platform/mtk-vcodec/Makefile
@@ -5,6 +5,7 @@ obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += mtk-vcodec-dec.o \
 
 mtk-vcodec-dec-y := vdec/vdec_h264_if.o \
 		vdec/vdec_vp8_if.o \
+		vdec/vdec_vp9_if.o \
 		mtk_vcodec_dec_drv.o \
 		vdec_drv_if.o \
 		vdec_vpu_if.o \
diff --git a/drivers/media/platform/mtk-vcodec/vdec/vdec_vp9_if.c b/drivers/media/platform/mtk-vcodec/vdec/vdec_vp9_if.c
new file mode 100644
index 0000000..650b93a
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/vdec/vdec_vp9_if.c
@@ -0,0 +1,967 @@
+/*
+ * Copyright (c) 2016 MediaTek Inc.
+ * Author: Daniel Hsiao <daniel.hsiao@mediatek.com>
+ *	Kai-Sean Yang <kai-sean.yang@mediatek.com>
+ *	Tiffany Lin <tiffany.lin@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * version 2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/fs.h>
+#include <linux/slab.h>
+#include <linux/syscalls.h>
+#include <linux/delay.h>
+#include <linux/time.h>
+
+#include "../mtk_vcodec_intr.h"
+#include "../vdec_drv_base.h"
+#include "../vdec_vpu_if.h"
+
+#define VP9_SUPER_FRAME_BS_SZ 64
+#define MAX_VP9_DPB_SIZE	9
+
+#define REFS_PER_FRAME 3
+#define MAX_NUM_REF_FRAMES 8
+#define VP9_MAX_FRM_BUF_NUM 9
+#define VP9_MAX_FRM_BUF_NODE_NUM (VP9_MAX_FRM_BUF_NUM * 2)
+
+/**
+ * struct vp9_dram_buf - contains buffer info for vpu
+ * @va : cpu address
+ * @pa : iova address
+ * @sz : buffer size
+ * @padding : for 64 bytes alignment
+ */
+struct vp9_dram_buf {
+	unsigned long va;
+	unsigned long pa;
+	unsigned int sz;
+	unsigned int padding;
+};
+
+/**
+ * struct vp9_fb_info - contains frame buffer info
+ * @fb : frmae buffer
+ * @reserved : reserved field used by vpu
+ */
+struct vp9_fb_info {
+	struct vdec_fb *fb;
+	unsigned int reserved[32];
+};
+
+/**
+ * struct vp9_ref_cnt_buf - contains reference buffer information
+ * @buf : referenced frame buffer
+ * @ref_cnt : referenced frame buffer's reference count.
+ *	When reference count=0, remove it from reference list
+ */
+struct vp9_ref_cnt_buf {
+	struct vp9_fb_info buf;
+	unsigned int ref_cnt;
+};
+
+/**
+ * struct vp9_fb_info - contains current frame's reference buffer information
+ * @buf : reference buffer
+ * @idx : reference buffer index to frm_bufs
+ * @reserved : reserved field used by vpu
+ */
+struct vp9_ref_buf {
+	struct vp9_fb_info *buf;
+	unsigned int idx;
+	unsigned int reserved[6];
+};
+
+/**
+ * struct vp9_fb_info - contains frame buffer info
+ * @fb : super frame reference frame buffer
+ * @used : this reference frame info entry is used
+ * @padding : for 64 bytes size align
+ */
+struct vp9_sf_ref_fb {
+	struct vdec_fb fb;
+	int used;
+	int padding;
+};
+
+/*
+ * struct vdec_vp9_vsi - shared buffer between host and VPU firmware
+ *	AP-W/R : AP is writer/reader on this item
+ *	VPU-W/R: VPU is write/reader on this item
+ * @sf_bs_buf : super frame backup buffer (AP-W, VPU-R)
+ * @sf_ref_fb : record supoer frame reference buffer information
+ *	(AP-R/W, VPU-R/W)
+ * @sf_next_ref_fb_idx : next available super frame (AP-W, VPU-R)
+ * @sf_frm_cnt : super frame count, filled by vpu (AP-R, VPU-W)
+ * @sf_frm_offset : super frame offset, filled by vpu (AP-R, VPU-W)
+ * @sf_frm_sz : super frame size, filled by vpu (AP-R, VPU-W)
+ * @sf_frm_idx : current super frame (AP-R, VPU-W)
+ * @sf_init : inform super frame info already parsed by vpu (AP-R, VPU-W)
+ * @fb : capture buffer (AP-W, VPU-R)
+ * @bs : bs buffer (AP-W, VPU-R)
+ * @cur_fb : current show capture buffer (AP-R/W, VPU-R/W)
+ * @pic_w : picture width (AP-R, VPU-W)
+ * @pic_h : picture height (AP-R, VPU-W)
+ * @buf_w : codec width (AP-R, VPU-W)
+ * @buf_h : coded height (AP-R, VPU-W)
+ * @buf_sz_y_bs : ufo compressed y plane size (AP-R, VPU-W)
+ * @buf_sz_c_bs : ufo compressed cbcr plane size (AP-R, VPU-W)
+ * @buf_len_sz_y : size used to store y plane ufo info (AP-R, VPU-W)
+ * @buf_len_sz_c : size used to store cbcr plane ufo info (AP-R, VPU-W)
+
+ * @profile : profile sparsed from vpu (AP-R, VPU-W)
+ * @show_frame : display this frame or not (AP-R, VPU-W)
+ * @show_existing_frame : inform this frame is show existing frame
+ *	(AP-R, VPU-W)
+ * @frm_to_show_idx : index to show frame (AP-R, VPU-W)
+
+ * @refresh_frm_flags : indicate when frame need to refine reference count
+ *	(AP-R, VPU-W)
+ * @resolution_changed : resolution change in this frame (AP-R, VPU-W)
+
+ * @frm_bufs : maintain reference buffer info (AP-R/W, VPU-R/W)
+ * @ref_frm_map : maintain reference buffer map info (AP-R/W, VPU-R/W)
+ * @new_fb_idx : index to frm_bufs array (AP-R, VPU-W)
+ * @frm_num : decoded frame number, include sub-frame count (AP-R, VPU-W)
+ * @mv_buf : motion vector working buffer (AP-W, VPU-R)
+ * @frm_refs : maintain three reference buffer info (AP-R/W, VPU-R/W)
+ */
+struct vdec_vp9_vsi {
+	unsigned char sf_bs_buf[VP9_SUPER_FRAME_BS_SZ];
+	struct vp9_sf_ref_fb sf_ref_fb[VP9_MAX_FRM_BUF_NUM-1];
+	int sf_next_ref_fb_idx;
+	unsigned int sf_frm_cnt;
+	unsigned int sf_frm_offset[VP9_MAX_FRM_BUF_NUM-1];
+	unsigned int sf_frm_sz[VP9_MAX_FRM_BUF_NUM-1];
+	unsigned int sf_frm_idx;
+	unsigned int sf_init;
+	struct vdec_fb fb;
+	struct mtk_vcodec_mem bs;
+	struct vdec_fb cur_fb;
+	unsigned int pic_w;
+	unsigned int pic_h;
+	unsigned int buf_w;
+	unsigned int buf_h;
+	unsigned int buf_sz_y_bs;
+	unsigned int buf_sz_c_bs;
+	unsigned int buf_len_sz_y;
+	unsigned int buf_len_sz_c;
+	unsigned int profile;
+	unsigned int show_frame;
+	unsigned int show_existing_frame;
+	unsigned int frm_to_show_idx;
+	unsigned int refresh_frm_flags;
+	unsigned int resolution_changed;
+
+	struct vp9_ref_cnt_buf frm_bufs[VP9_MAX_FRM_BUF_NUM];
+	int ref_frm_map[MAX_NUM_REF_FRAMES];
+	unsigned int new_fb_idx;
+	unsigned int frm_num;
+	struct vp9_dram_buf mv_buf;
+
+	struct vp9_ref_buf frm_refs[REFS_PER_FRAME];
+};
+
+/*
+ * struct vdec_vp9_inst - vp9 decode instance
+ * @mv_buf : working buffer for mv
+ * @dec_fb : vdec_fb node to link fb to different fb_xxx_list
+ * @available_fb_node_list : current available vdec_fb node
+ * @fb_use_list : current used or referenced vdec_fb
+ * @fb_free_list : current availabe to free vdec_fb
+ * @fb_disp_list : current availabe to display vdec_fb
+ * @cur_fb : current frame buffer
+ * @ctx : current decode context
+ * @vpu : vpu instance information
+ * @vsi : shared buffer between host and VPU firmware
+ * @total_frm_cnt : total frame count, it do not include sub-frames in super
+ *	    frame
+ * @mem : instance memory information
+ */
+struct vdec_vp9_inst {
+	struct mtk_vcodec_mem mv_buf;
+
+	struct vdec_fb_node dec_fb[VP9_MAX_FRM_BUF_NODE_NUM];
+	struct list_head available_fb_node_list;
+	struct list_head fb_use_list;
+	struct list_head fb_free_list;
+	struct list_head fb_disp_list;
+	struct vdec_fb *cur_fb;
+	struct mtk_vcodec_ctx *ctx;
+	struct vdec_vpu_inst vpu;
+	struct vdec_vp9_vsi *vsi;
+	unsigned int total_frm_cnt;
+	struct mtk_vcodec_mem mem;
+};
+
+static bool vp9_is_sf_ref_fb(struct vdec_vp9_inst *inst, struct vdec_fb *fb)
+{
+	int i;
+	struct vdec_vp9_vsi *vsi = inst->vsi;
+
+	for (i = 0; i < ARRAY_SIZE(vsi->sf_ref_fb); i++) {
+		if (fb == &vsi->sf_ref_fb[i].fb)
+			return true;
+	}
+	return false;
+}
+
+static struct vdec_fb *vp9_rm_from_fb_use_list(struct vdec_vp9_inst
+					*inst, void *addr)
+{
+	struct vdec_fb *fb = NULL;
+	struct vdec_fb_node *node;
+
+	list_for_each_entry(node, &inst->fb_use_list, list) {
+		fb = (struct vdec_fb *)node->fb;
+		if (fb->base_y.va == addr) {
+			list_move_tail(&node->list,
+				       &inst->available_fb_node_list);
+			break;
+		}
+	}
+	return fb;
+}
+
+static void vp9_add_to_fb_free_list(struct vdec_vp9_inst *inst,
+			     struct vdec_fb *fb)
+{
+	struct vdec_fb_node *node;
+
+	if (fb) {
+		node = list_first_entry_or_null(&inst->available_fb_node_list,
+					struct vdec_fb_node, list);
+
+		if (node) {
+			node->fb = fb;
+			list_move_tail(&node->list, &inst->fb_free_list);
+		}
+	} else {
+		mtk_vcodec_debug(inst, "No free fb node");
+	}
+}
+
+static void vp9_free_sf_ref_fb(struct vdec_fb *fb)
+{
+	struct vp9_sf_ref_fb *sf_ref_fb =
+		container_of(fb, struct vp9_sf_ref_fb, fb);
+
+	sf_ref_fb->used = 0;
+}
+
+static void vp9_ref_cnt_fb(struct vdec_vp9_inst *inst, int *idx,
+			   int new_idx)
+{
+	struct vdec_vp9_vsi *vsi = inst->vsi;
+	int ref_idx = *idx;
+
+	if (ref_idx >= 0 && vsi->frm_bufs[ref_idx].ref_cnt > 0) {
+		vsi->frm_bufs[ref_idx].ref_cnt--;
+
+		if (vsi->frm_bufs[ref_idx].ref_cnt == 0) {
+			if (!vp9_is_sf_ref_fb(inst,
+					      vsi->frm_bufs[ref_idx].buf.fb)) {
+				struct vdec_fb *fb;
+
+				fb = vp9_rm_from_fb_use_list(inst,
+				     vsi->frm_bufs[ref_idx].buf.fb->base_y.va);
+				vp9_add_to_fb_free_list(inst, fb);
+			} else
+				vp9_free_sf_ref_fb(
+					vsi->frm_bufs[ref_idx].buf.fb);
+		}
+	}
+
+	*idx = new_idx;
+	vsi->frm_bufs[new_idx].ref_cnt++;
+}
+
+static void vp9_free_all_sf_ref_fb(struct vdec_vp9_inst *inst)
+{
+	int i;
+	struct vdec_vp9_vsi *vsi = inst->vsi;
+
+	for (i = 0; i < ARRAY_SIZE(vsi->sf_ref_fb); i++) {
+		if (vsi->sf_ref_fb[i].fb.base_y.va) {
+			mtk_vcodec_mem_free(inst->ctx,
+				&vsi->sf_ref_fb[i].fb.base_y);
+			mtk_vcodec_mem_free(inst->ctx,
+				&vsi->sf_ref_fb[i].fb.base_c);
+			vsi->sf_ref_fb[i].used = 0;
+		}
+	}
+}
+
+/* For each sub-frame except the last one, the driver will dynamically
+ * allocate reference buffer by calling vp9_get_sf_ref_fb()
+ * The last sub-frame will use the original fb provided by the
+ * vp9_dec_decode() interface
+ */
+static int vp9_get_sf_ref_fb(struct vdec_vp9_inst *inst)
+{
+	int idx;
+	struct mtk_vcodec_mem *mem_basy_y;
+	struct mtk_vcodec_mem *mem_basy_c;
+	struct vdec_vp9_vsi *vsi = inst->vsi;
+
+	for (idx = 0;
+		idx < ARRAY_SIZE(vsi->sf_ref_fb);
+		idx++) {
+		if (vsi->sf_ref_fb[idx].fb.base_y.va &&
+		    vsi->sf_ref_fb[idx].used == 0) {
+			return idx;
+		}
+	}
+
+	for (idx = 0;
+		idx < ARRAY_SIZE(vsi->sf_ref_fb);
+		idx++) {
+		if (vsi->sf_ref_fb[idx].fb.base_y.va == NULL)
+			break;
+	}
+
+	if (idx == ARRAY_SIZE(vsi->sf_ref_fb)) {
+		mtk_vcodec_err(inst, "List Full");
+		return -1;
+	}
+
+	mem_basy_y = &vsi->sf_ref_fb[idx].fb.base_y;
+	mem_basy_y->size = vsi->buf_sz_y_bs +
+		vsi->buf_len_sz_y;
+
+	if (mtk_vcodec_mem_alloc(inst->ctx, mem_basy_y)) {
+		mtk_vcodec_err(inst, "Cannot allocate sf_ref_buf y_buf");
+		return -1;
+	}
+
+	mem_basy_c = &vsi->sf_ref_fb[idx].fb.base_c;
+	mem_basy_c->size = vsi->buf_sz_c_bs +
+		vsi->buf_len_sz_c;
+
+	if (mtk_vcodec_mem_alloc(inst->ctx, mem_basy_c)) {
+		mtk_vcodec_err(inst, "Cannot allocate sf_ref_fb c_buf");
+		return -1;
+	}
+	vsi->sf_ref_fb[idx].used = 0;
+
+	return idx;
+}
+
+static bool vp9_alloc_work_buf(struct vdec_vp9_inst *inst)
+{
+	struct vdec_vp9_vsi *vsi = inst->vsi;
+	int result;
+	struct mtk_vcodec_mem *mem;
+
+	unsigned int max_pic_w;
+	unsigned int max_pic_h;
+
+
+	if (!(inst->ctx->dev->dec_capability &
+		VCODEC_CAPABILITY_4K_DISABLED)) {
+		max_pic_w = VCODEC_DEC_4K_CODED_WIDTH;
+		max_pic_h = VCODEC_DEC_4K_CODED_HEIGHT;
+	} else {
+		max_pic_w = MTK_VDEC_MAX_W;
+		max_pic_h = MTK_VDEC_MAX_H;
+	}
+
+	if ((vsi->pic_w > max_pic_w) ||
+		(vsi->pic_h > max_pic_h)) {
+		mtk_vcodec_err(inst, "Invalid w/h %d/%d",
+				vsi->pic_w, vsi->pic_h);
+		return false;
+	}
+
+	mtk_vcodec_debug(inst, "BUF CHG(%d): w/h/sb_w/sb_h=%d/%d/%d/%d",
+			vsi->resolution_changed,
+			vsi->pic_w,
+			vsi->pic_h,
+			vsi->buf_w,
+			vsi->buf_h);
+
+	mem = &inst->mv_buf;
+
+	if (mem->va)
+		mtk_vcodec_mem_free(inst->ctx, mem);
+
+	mem->size = ((vsi->buf_w / 64) *
+		    (vsi->buf_h / 64) + 2) * 36 * 16;
+
+	result = mtk_vcodec_mem_alloc(inst->ctx, mem);
+	if (result) {
+		mem->size = 0;
+		mtk_vcodec_err(inst, "Cannot allocate mv_buf");
+		return false;
+	}
+	/* Set the va again */
+	vsi->mv_buf.va = (unsigned long)mem->va;
+	vsi->mv_buf.pa = (unsigned long)mem->dma_addr;
+	vsi->mv_buf.sz = (unsigned int)mem->size;
+
+	vp9_free_all_sf_ref_fb(inst);
+	vsi->sf_next_ref_fb_idx = vp9_get_sf_ref_fb(inst);
+
+	return true;
+}
+
+static bool vp9_add_to_fb_disp_list(struct vdec_vp9_inst *inst,
+			     struct vdec_fb *fb)
+{
+	struct vdec_fb_node *node;
+
+	if (!fb) {
+		mtk_vcodec_err(inst, "fb == NULL");
+		return false;
+	}
+
+	node = list_first_entry_or_null(&inst->available_fb_node_list,
+					struct vdec_fb_node, list);
+	if (node) {
+		node->fb = fb;
+		list_move_tail(&node->list, &inst->fb_disp_list);
+	} else {
+		mtk_vcodec_err(inst, "No available fb node");
+		return false;
+	}
+
+	return true;
+}
+
+/* If any buffer updating is signaled it should be done here. */
+static void vp9_swap_frm_bufs(struct vdec_vp9_inst *inst)
+{
+	struct vdec_vp9_vsi *vsi = inst->vsi;
+	struct vp9_fb_info *frm_to_show;
+	int ref_index = 0, mask;
+
+	for (mask = vsi->refresh_frm_flags; mask; mask >>= 1) {
+		if (mask & 1)
+			vp9_ref_cnt_fb(inst, &vsi->ref_frm_map[ref_index],
+				       vsi->new_fb_idx);
+		++ref_index;
+	}
+
+	frm_to_show = &vsi->frm_bufs[vsi->new_fb_idx].buf;
+	vsi->frm_bufs[vsi->new_fb_idx].ref_cnt--;
+
+	if (frm_to_show->fb != inst->cur_fb) {
+		/* This frame is show exist frame and no decode output
+		 * copy frame data from frm_to_show to current CAPTURE
+		 * buffer
+		 */
+		if ((frm_to_show->fb != NULL) &&
+			(inst->cur_fb->base_y.size >=
+			frm_to_show->fb->base_y.size)) {
+			memcpy((void *)inst->cur_fb->base_y.va,
+				(void *)frm_to_show->fb->base_y.va,
+				vsi->buf_w *
+				vsi->buf_h);
+			memcpy((void *)inst->cur_fb->base_c.va,
+				(void *)frm_to_show->fb->base_c.va,
+				vsi->buf_w *
+				vsi->buf_h / 2);
+		} else {
+			/* After resolution change case, current CAPTURE buffer
+			 * may have less buffer size than frm_to_show buffer
+			 * size
+			 */
+			if (frm_to_show->fb != NULL)
+				mtk_vcodec_err(inst,
+					"inst->cur_fb->base_y.size=%lx, frm_to_show->fb.base_y.size=%lx",
+					inst->cur_fb->base_y.size,
+					frm_to_show->fb->base_y.size);
+		}
+		if (!vp9_is_sf_ref_fb(inst, inst->cur_fb)) {
+			if (vsi->show_frame)
+				vp9_add_to_fb_disp_list(inst, inst->cur_fb);
+		}
+	} else {
+		if (!vp9_is_sf_ref_fb(inst, inst->cur_fb)) {
+			if (vsi->show_frame)
+				vp9_add_to_fb_disp_list(inst, frm_to_show->fb);
+		}
+	}
+
+	/* when ref_cnt ==0, move this fb to fb_free_list. v4l2 driver will
+	 * clean fb_free_list
+	 */
+	if (vsi->frm_bufs[vsi->new_fb_idx].ref_cnt == 0) {
+		if (!vp9_is_sf_ref_fb(
+			inst, vsi->frm_bufs[vsi->new_fb_idx].buf.fb)) {
+			struct vdec_fb *fb;
+
+			fb = vp9_rm_from_fb_use_list(inst,
+			vsi->frm_bufs[vsi->new_fb_idx].buf.fb->base_y.va);
+
+			vp9_add_to_fb_free_list(inst, fb);
+		} else {
+			vp9_free_sf_ref_fb(
+				vsi->frm_bufs[vsi->new_fb_idx].buf.fb);
+		}
+	}
+
+	/* if this super frame and it is not last sub-frame, get next fb for
+	 * sub-frame decode
+	 */
+	if (vsi->sf_frm_cnt > 0 && vsi->sf_frm_idx != vsi->sf_frm_cnt - 1)
+		vsi->sf_next_ref_fb_idx = vp9_get_sf_ref_fb(inst);
+}
+
+static bool vp9_wait_dec_end(struct vdec_vp9_inst *inst)
+{
+	struct mtk_vcodec_ctx *ctx = inst->ctx;
+
+	mtk_vcodec_wait_for_done_ctx(inst->ctx,
+			MTK_INST_IRQ_RECEIVED,
+			WAIT_INTR_TIMEOUT_MS);
+
+	if (ctx->irq_status & MTK_VDEC_IRQ_STATUS_DEC_SUCCESS)
+		return true;
+	else
+		return false;
+}
+
+static struct vdec_vp9_inst *vp9_alloc_inst(struct mtk_vcodec_ctx *ctx)
+{
+	int result;
+	struct mtk_vcodec_mem mem;
+	struct vdec_vp9_inst *inst;
+
+	memset(&mem, 0, sizeof(mem));
+	mem.size = sizeof(struct vdec_vp9_inst);
+	result = mtk_vcodec_mem_alloc(ctx, &mem);
+	if (result)
+		return NULL;
+
+	inst = mem.va;
+	inst->mem = mem;
+
+	return inst;
+}
+
+static void vp9_free_inst(struct vdec_vp9_inst *inst)
+{
+	struct mtk_vcodec_mem mem;
+
+	mem = inst->mem;
+	if (mem.va)
+		mtk_vcodec_mem_free(inst->ctx, &mem);
+}
+
+static bool vp9_decode_end_proc(struct vdec_vp9_inst *inst)
+{
+	struct vdec_vp9_vsi *vsi = inst->vsi;
+	bool ret = false;
+
+	if (!vsi->show_existing_frame) {
+		ret = vp9_wait_dec_end(inst);
+		if (!ret) {
+			mtk_vcodec_err(inst, "Decode failed, Decode Timeout @[%d]",
+				vsi->frm_num);
+			return false;
+		}
+
+		if (vpu_dec_end(&inst->vpu)) {
+			mtk_vcodec_err(inst, "vp9_dec_vpu_end failed");
+			return false;
+		}
+		mtk_vcodec_debug(inst, "Decode Ok @%d (%d/%d)", vsi->frm_num,
+				vsi->pic_w, vsi->pic_h);
+	} else {
+		mtk_vcodec_debug(inst, "Decode Ok @%d (show_existing_frame)",
+				vsi->frm_num);
+	}
+
+	vp9_swap_frm_bufs(inst);
+	vsi->frm_num++;
+	return true;
+}
+
+static bool vp9_is_last_sub_frm(struct vdec_vp9_inst *inst)
+{
+	struct vdec_vp9_vsi *vsi = inst->vsi;
+
+	if (vsi->sf_frm_cnt <= 0 || vsi->sf_frm_idx == vsi->sf_frm_cnt)
+		return true;
+
+	return false;
+}
+
+static struct vdec_fb *vp9_rm_from_fb_disp_list(struct vdec_vp9_inst *inst)
+{
+	struct vdec_fb_node *node;
+	struct vdec_fb *fb = NULL;
+
+	node = list_first_entry_or_null(&inst->fb_disp_list,
+					struct vdec_fb_node, list);
+	if (node) {
+		fb = (struct vdec_fb *)node->fb;
+		fb->status |= FB_ST_DISPLAY;
+		list_move_tail(&node->list, &inst->available_fb_node_list);
+		mtk_vcodec_debug(inst, "[FB] get disp fb %p st=%d",
+				 node->fb, fb->status);
+	} else
+		mtk_vcodec_debug(inst, "[FB] there is no disp fb");
+
+	return fb;
+}
+
+static bool vp9_add_to_fb_use_list(struct vdec_vp9_inst *inst,
+			    struct vdec_fb *fb)
+{
+	struct vdec_fb_node *node;
+
+	if (!fb) {
+		mtk_vcodec_debug(inst, "fb == NULL");
+		return false;
+	}
+
+	node = list_first_entry_or_null(&inst->available_fb_node_list,
+					struct vdec_fb_node, list);
+	if (node) {
+		node->fb = fb;
+		list_move_tail(&node->list, &inst->fb_use_list);
+	} else {
+		mtk_vcodec_err(inst, "No free fb node");
+		return false;
+	}
+	return true;
+}
+
+static void vp9_reset(struct vdec_vp9_inst *inst)
+{
+	struct vdec_fb_node *node, *tmp;
+
+	list_for_each_entry_safe(node, tmp, &inst->fb_use_list, list)
+		list_move_tail(&node->list, &inst->fb_free_list);
+
+	vp9_free_all_sf_ref_fb(inst);
+	inst->vsi->sf_next_ref_fb_idx = vp9_get_sf_ref_fb(inst);
+
+	if (vpu_dec_reset(&inst->vpu))
+		mtk_vcodec_err(inst, "vp9_dec_vpu_reset failed");
+
+	/* Set the va again, since vpu_dec_reset will clear mv_buf in vpu */
+	inst->vsi->mv_buf.va = (unsigned long)inst->mv_buf.va;
+	inst->vsi->mv_buf.pa = (unsigned long)inst->mv_buf.dma_addr;
+	inst->vsi->mv_buf.sz = (unsigned long)inst->mv_buf.size;
+}
+
+static void init_all_fb_lists(struct vdec_vp9_inst *inst)
+{
+	int i;
+
+	INIT_LIST_HEAD(&inst->available_fb_node_list);
+	INIT_LIST_HEAD(&inst->fb_use_list);
+	INIT_LIST_HEAD(&inst->fb_free_list);
+	INIT_LIST_HEAD(&inst->fb_disp_list);
+
+	for (i = 0; i < ARRAY_SIZE(inst->dec_fb); i++) {
+		INIT_LIST_HEAD(&inst->dec_fb[i].list);
+		inst->dec_fb[i].fb = NULL;
+		list_add_tail(&inst->dec_fb[i].list,
+			      &inst->available_fb_node_list);
+	}
+}
+
+static void get_pic_info(struct vdec_vp9_inst *inst, struct vdec_pic_info *pic)
+{
+	pic->y_bs_sz = inst->vsi->buf_sz_y_bs;
+	pic->c_bs_sz = inst->vsi->buf_sz_c_bs;
+	pic->y_len_sz = inst->vsi->buf_len_sz_y;
+	pic->c_len_sz = inst->vsi->buf_len_sz_c;
+
+	pic->pic_w = inst->vsi->pic_w;
+	pic->pic_h = inst->vsi->pic_h;
+	pic->buf_w = inst->vsi->buf_w;
+	pic->buf_h = inst->vsi->buf_h;
+
+	mtk_vcodec_debug(inst, "pic(%d, %d), buf(%d, %d)",
+		 pic->pic_w, pic->pic_h, pic->buf_w, pic->buf_h);
+	mtk_vcodec_debug(inst, "Y(%d, %d), C(%d, %d)", pic->y_bs_sz,
+		 pic->y_len_sz, pic->c_bs_sz, pic->c_len_sz);
+}
+
+static void get_disp_fb(struct vdec_vp9_inst *inst, struct vdec_fb **out_fb)
+{
+
+	*out_fb = vp9_rm_from_fb_disp_list(inst);
+	if (*out_fb)
+		(*out_fb)->status |= FB_ST_DISPLAY;
+}
+
+static void get_free_fb(struct vdec_vp9_inst *inst, struct vdec_fb **out_fb)
+{
+	struct vdec_fb_node *node;
+	struct vdec_fb *fb = NULL;
+
+	node = list_first_entry_or_null(&inst->fb_free_list,
+					struct vdec_fb_node, list);
+	if (node) {
+		list_move_tail(&node->list, &inst->available_fb_node_list);
+		fb = (struct vdec_fb *)node->fb;
+		fb->status |= FB_ST_FREE;
+		mtk_vcodec_debug(inst, "[FB] get free fb %p st=%d",
+				 node->fb, fb->status);
+	} else {
+		mtk_vcodec_debug(inst, "[FB] there is no free fb");
+	}
+
+	*out_fb = fb;
+}
+
+static void vdec_vp9_deinit(unsigned long h_vdec)
+{
+	struct vdec_vp9_inst *inst = (struct vdec_vp9_inst *)h_vdec;
+	struct mtk_vcodec_mem *mem;
+	int ret = 0;
+
+	ret = vpu_dec_deinit(&inst->vpu);
+	if (ret)
+		mtk_vcodec_err(inst, "vpu_dec_deinit failed");
+
+	mem = &inst->mv_buf;
+	if (mem->va)
+		mtk_vcodec_mem_free(inst->ctx, mem);
+
+	vp9_free_all_sf_ref_fb(inst);
+	vp9_free_inst(inst);
+}
+
+static int vdec_vp9_init(struct mtk_vcodec_ctx *ctx, unsigned long *h_vdec)
+{
+	struct vdec_vp9_inst *inst;
+
+	inst = vp9_alloc_inst(ctx);
+	if (!inst)
+		return -ENOMEM;
+
+	inst->total_frm_cnt = 0;
+	inst->ctx = ctx;
+
+	inst->vpu.id = IPI_VDEC_VP9;
+	inst->vpu.dev = ctx->dev->vpu_plat_dev;
+	inst->vpu.ctx = ctx;
+	inst->vpu.handler = vpu_dec_ipi_handler;
+
+	if (vpu_dec_init(&inst->vpu)) {
+		mtk_vcodec_err(inst, "vp9_dec_vpu_init failed");
+		goto err_deinit_inst;
+	}
+
+	inst->vsi = (struct vdec_vp9_vsi *)inst->vpu.vsi;
+	init_all_fb_lists(inst);
+
+	(*h_vdec) = (unsigned long)inst;
+	return 0;
+
+err_deinit_inst:
+	vp9_free_inst(inst);
+
+	return -EINVAL;
+}
+
+static int vdec_vp9_decode(unsigned long h_vdec, struct mtk_vcodec_mem *bs,
+		   struct vdec_fb *fb, bool *res_chg)
+{
+	int ret = 0;
+	struct vdec_vp9_inst *inst = (struct vdec_vp9_inst *)h_vdec;
+	struct vdec_vp9_vsi *vsi = inst->vsi;
+	u32 data[3];
+	int i;
+
+	*res_chg = false;
+
+	if ((bs == NULL) && (fb == NULL)) {
+		mtk_vcodec_debug(inst, "[EOS]");
+		vp9_reset(inst);
+		return ret;
+	}
+
+	if (bs == NULL) {
+		mtk_vcodec_err(inst, "bs == NULL");
+		return -EINVAL;
+	}
+
+	mtk_vcodec_debug(inst, "Input BS Size = %ld", bs->size);
+
+	while (1) {
+		struct vdec_fb *cur_fb = NULL;
+
+		data[0] = *((unsigned int *)bs->va);
+		data[1] = *((unsigned int *)(bs->va + 4));
+		data[2] = *((unsigned int *)(bs->va + 8));
+
+		vsi->bs = *bs;
+
+		if (fb)
+			vsi->fb = *fb;
+
+		if (!vsi->sf_init) {
+			unsigned int sf_bs_sz;
+			unsigned int sf_bs_off;
+			unsigned char *sf_bs_src;
+			unsigned char *sf_bs_dst;
+
+			sf_bs_sz = bs->size > VP9_SUPER_FRAME_BS_SZ ?
+					VP9_SUPER_FRAME_BS_SZ : bs->size;
+			sf_bs_off = VP9_SUPER_FRAME_BS_SZ - sf_bs_sz;
+			sf_bs_src = bs->va + bs->size - sf_bs_sz;
+			sf_bs_dst = vsi->sf_bs_buf + sf_bs_off;
+			memcpy(sf_bs_dst, sf_bs_src, sf_bs_sz);
+		} else {
+			if ((vsi->sf_frm_cnt > 0) &&
+				(vsi->sf_frm_idx < vsi->sf_frm_cnt)) {
+				unsigned int idx = vsi->sf_frm_idx;
+
+				memcpy((void *)bs->va,
+					(void *)(bs->va +
+					vsi->sf_frm_offset[idx]),
+					vsi->sf_frm_sz[idx]);
+			}
+		}
+		ret = vpu_dec_start(&inst->vpu, data, 3);
+		if (ret) {
+			mtk_vcodec_err(inst, "vpu_dec_start failed");
+			goto DECODE_ERROR;
+		}
+
+		if (vsi->resolution_changed) {
+			if (!vp9_alloc_work_buf(inst)) {
+				ret = -EINVAL;
+				goto DECODE_ERROR;
+			}
+		}
+
+		if (vsi->sf_frm_cnt > 0) {
+			cur_fb = &vsi->sf_ref_fb[vsi->sf_next_ref_fb_idx].fb;
+
+			if (vsi->sf_frm_idx < vsi->sf_frm_cnt)
+				inst->cur_fb = cur_fb;
+			else
+				inst->cur_fb = fb;
+		} else {
+			inst->cur_fb = fb;
+		}
+
+		vsi->frm_bufs[vsi->new_fb_idx].buf.fb = inst->cur_fb;
+		if (!vp9_is_sf_ref_fb(inst, inst->cur_fb))
+			vp9_add_to_fb_use_list(inst, inst->cur_fb);
+
+		mtk_vcodec_debug(inst, "[#pic %d]", vsi->frm_num);
+
+		if (vsi->show_existing_frame)
+			mtk_vcodec_debug(inst,
+				"drv->new_fb_idx=%d, drv->frm_to_show_idx=%d",
+				vsi->new_fb_idx, vsi->frm_to_show_idx);
+
+		if (vsi->show_existing_frame && (vsi->frm_to_show_idx <
+					VP9_MAX_FRM_BUF_NUM)) {
+			mtk_vcodec_err(inst,
+				"Skip Decode drv->new_fb_idx=%d, drv->frm_to_show_idx=%d",
+				vsi->new_fb_idx, vsi->frm_to_show_idx);
+
+			vp9_ref_cnt_fb(inst, &vsi->new_fb_idx,
+					vsi->frm_to_show_idx);
+			ret = -EINVAL;
+			goto DECODE_ERROR;
+		}
+
+		/* VPU assign the buffer pointer in its address space,
+		 * reassign here
+		 */
+		for (i = 0; i < ARRAY_SIZE(vsi->frm_refs); i++) {
+			unsigned int idx = vsi->frm_refs[i].idx;
+
+			vsi->frm_refs[i].buf = &vsi->frm_bufs[idx].buf;
+		}
+
+		if (vsi->resolution_changed) {
+			*res_chg = true;
+			mtk_vcodec_debug(inst, "VDEC_ST_RESOLUTION_CHANGED");
+
+			ret = 0;
+			goto DECODE_ERROR;
+		}
+
+		if (vp9_decode_end_proc(inst) != true) {
+			mtk_vcodec_err(inst, "vp9_decode_end_proc");
+			ret = -EINVAL;
+			goto DECODE_ERROR;
+		}
+
+		if (vp9_is_last_sub_frm(inst))
+			break;
+
+	}
+	inst->total_frm_cnt++;
+
+DECODE_ERROR:
+	if (ret < 0)
+		vp9_add_to_fb_free_list(inst, fb);
+
+	return ret;
+}
+
+static void get_crop_info(struct vdec_vp9_inst *inst, struct v4l2_rect *cr)
+{
+	cr->left = 0;
+	cr->top = 0;
+	cr->width = inst->vsi->pic_w;
+	cr->height = inst->vsi->buf_h;
+	mtk_vcodec_debug(inst, "get crop info l=%d, t=%d, w=%d, h=%d\n",
+			 cr->left, cr->top, cr->width, cr->height);
+}
+
+static int vdec_vp9_get_param(unsigned long h_vdec,
+		enum vdec_get_param_type type, void *out)
+{
+	struct vdec_vp9_inst *inst = (struct vdec_vp9_inst *)h_vdec;
+	int ret = 0;
+
+	switch (type) {
+	case GET_PARAM_DISP_FRAME_BUFFER:
+		get_disp_fb(inst, out);
+		break;
+	case GET_PARAM_FREE_FRAME_BUFFER:
+		get_free_fb(inst, out);
+		break;
+	case GET_PARAM_PIC_INFO:
+		get_pic_info(inst, out);
+		break;
+	case GET_PARAM_DPB_SIZE:
+		*((unsigned int *)out) = MAX_VP9_DPB_SIZE;
+		break;
+	case GET_PARAM_CROP_INFO:
+		get_crop_info(inst, out);
+		break;
+	default:
+		mtk_vcodec_err(inst, "not supported param type %d", type);
+		ret = -EINVAL;
+		break;
+	}
+
+	return ret;
+}
+
+static struct vdec_common_if vdec_vp9_if = {
+	vdec_vp9_init,
+	vdec_vp9_decode,
+	vdec_vp9_get_param,
+	vdec_vp9_deinit,
+};
+
+struct vdec_common_if *get_vp9_dec_comm_if(void);
+
+struct vdec_common_if *get_vp9_dec_comm_if(void)
+{
+	return &vdec_vp9_if;
+}
diff --git a/drivers/media/platform/mtk-vcodec/vdec_drv_if.c b/drivers/media/platform/mtk-vcodec/vdec_drv_if.c
index 5338171..3d7aaab 100644
--- a/drivers/media/platform/mtk-vcodec/vdec_drv_if.c
+++ b/drivers/media/platform/mtk-vcodec/vdec_drv_if.c
@@ -26,6 +26,7 @@
 
 struct vdec_common_if *get_h264_dec_comm_if(void);
 struct vdec_common_if *get_vp8_dec_comm_if(void);
+struct vdec_common_if *get_vp9_dec_comm_if(void);
 
 int vdec_if_init(struct mtk_vcodec_ctx *ctx, unsigned int fourcc)
 {
@@ -39,6 +40,8 @@ int vdec_if_init(struct mtk_vcodec_ctx *ctx, unsigned int fourcc)
 		ctx->dec_if = get_vp8_dec_comm_if();
 		break;
 	case V4L2_PIX_FMT_VP9:
+		ctx->dec_if = get_vp9_dec_comm_if();
+		break;
 	default:
 		return -EINVAL;
 	}
-- 
1.7.9.5

[PATCH v3 9/9] arm64: dts: mediatek: Add Video Decoder for MT8173

[Tiffany Lin]

Add video decoder node for MT8173

Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
---
 arch/arm64/boot/dts/mediatek/mt8173.dtsi |   44 ++++++++++++++++++++++++++++++
 1 file changed, 44 insertions(+)

diff --git a/arch/arm64/boot/dts/mediatek/mt8173.dtsi b/arch/arm64/boot/dts/mediatek/mt8173.dtsi
index 348ce0e..4ac06d3 100644
--- a/arch/arm64/boot/dts/mediatek/mt8173.dtsi
+++ b/arch/arm64/boot/dts/mediatek/mt8173.dtsi
@@ -691,6 +691,50 @@
 			#clock-cells = <1>;
 		};
 
+		vcodec_dec: vcodec at 16000000 {
+			compatible = "mediatek,mt8173-vcodec-dec";
+			reg = <0 0x16000000 0 0x100>,	/* VDEC_SYS */
+			      <0 0x16020000 0 0x1000>,	/* VDEC_MISC */
+			      <0 0x16021000 0 0x800>,	/* VDEC_LD */
+			      <0 0x16021800 0 0x800>,	/* VDEC_TOP */
+			      <0 0x16022000 0 0x1000>,	/* VDEC_CM */
+			      <0 0x16023000 0 0x1000>,	/* VDEC_AD */
+			      <0 0x16024000 0 0x1000>,	/* VDEC_AV */
+			      <0 0x16025000 0 0x1000>,	/* VDEC_PP */
+			      <0 0x16026800 0 0x800>,	/* VDEC_HWD */
+			      <0 0x16027000 0 0x800>,	/* VDEC_HWQ */
+			      <0 0x16027800 0 0x800>,	/* VDEC_HWB */
+			      <0 0x16028400 0 0x400>;	/* VDEC_HWG */
+			interrupts = <GIC_SPI 204 IRQ_TYPE_LEVEL_LOW>;
+			mediatek,larb = <&larb1>;
+			iommus = <&iommu M4U_PORT_HW_VDEC_MC_EXT>,
+				 <&iommu M4U_PORT_HW_VDEC_PP_EXT>,
+				 <&iommu M4U_PORT_HW_VDEC_AVC_MV_EXT>,
+				 <&iommu M4U_PORT_HW_VDEC_PRED_RD_EXT>,
+				 <&iommu M4U_PORT_HW_VDEC_PRED_WR_EXT>,
+				 <&iommu M4U_PORT_HW_VDEC_UFO_EXT>,
+				 <&iommu M4U_PORT_HW_VDEC_VLD_EXT>,
+				 <&iommu M4U_PORT_HW_VDEC_VLD2_EXT>;
+			mediatek,vpu = <&vpu>;
+			power-domains = <&scpsys MT8173_POWER_DOMAIN_VDEC>;
+			clocks = <&apmixedsys CLK_APMIXED_VCODECPLL>,
+				 <&topckgen CLK_TOP_UNIVPLL_D2>,
+				 <&topckgen CLK_TOP_CCI400_SEL>,
+				 <&topckgen CLK_TOP_VDEC_SEL>,
+				 <&topckgen CLK_TOP_VCODECPLL>,
+				 <&apmixedsys CLK_APMIXED_VENCPLL>,
+				 <&topckgen CLK_TOP_VENC_LT_SEL>,
+				 <&topckgen CLK_TOP_VCODECPLL_370P5>;
+			clock-names = "vcodecpll",
+				      "univpll_d2",
+				      "clk_cci400_sel",
+				      "vdec_sel",
+				      "vdecpll",
+				      "vencpll",
+				      "venc_lt_sel",
+				      "vdec_bus_clk_src";
+		};
+
 		larb1: larb at 16010000 {
 			compatible = "mediatek,mt8173-smi-larb";
 			reg = <0 0x16010000 0 0x1000>;
-- 
1.7.9.5

[PATCH v3 5/9] vcodec: mediatek: Add Mediatek V4L2 Video Decoder Driver

[Hans Verkuil]

On 05/30/2016 02:29 PM, Tiffany Lin wrote:
> Add v4l2 layer decoder driver for MT8173
> 
> Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
> ---
>  drivers/media/platform/mtk-vcodec/Makefile         |   10 +-
>  drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c | 1348 ++++++++++++++++++++
>  drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.h |   88 ++
>  .../media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c |  408 ++++++
>  .../media/platform/mtk-vcodec/mtk_vcodec_dec_pm.c  |  206 +++
>  .../media/platform/mtk-vcodec/mtk_vcodec_dec_pm.h  |   28 +
>  drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h |   88 +-
>  .../media/platform/mtk-vcodec/mtk_vcodec_intr.c    |    4 +-
>  .../media/platform/mtk-vcodec/mtk_vcodec_intr.h    |    2 +-
>  .../media/platform/mtk-vcodec/mtk_vcodec_util.c    |   31 +-
>  .../media/platform/mtk-vcodec/mtk_vcodec_util.h    |    7 +-
>  drivers/media/platform/mtk-vcodec/vdec_drv_base.h  |   56 +
>  drivers/media/platform/mtk-vcodec/vdec_drv_if.c    |  107 ++
>  drivers/media/platform/mtk-vcodec/vdec_drv_if.h    |  101 ++
>  drivers/media/platform/mtk-vcodec/vdec_ipi_msg.h   |  103 ++
>  15 files changed, 2569 insertions(+), 18 deletions(-)
>  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c
>  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.h
>  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c
>  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_pm.c
>  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_pm.h
>  create mode 100644 drivers/media/platform/mtk-vcodec/vdec_drv_base.h
>  create mode 100644 drivers/media/platform/mtk-vcodec/vdec_drv_if.c
>  create mode 100644 drivers/media/platform/mtk-vcodec/vdec_drv_if.h
>  create mode 100644 drivers/media/platform/mtk-vcodec/vdec_ipi_msg.h
> 
> diff --git a/drivers/media/platform/mtk-vcodec/Makefile b/drivers/media/platform/mtk-vcodec/Makefile
> index dc5cb00..4c8ed2f 100644
> --- a/drivers/media/platform/mtk-vcodec/Makefile
> +++ b/drivers/media/platform/mtk-vcodec/Makefile
> @@ -1,7 +1,13 @@
>  
>  
> -obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += mtk-vcodec-enc.o mtk-vcodec-common.o
> -
> +obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += mtk-vcodec-dec.o \
> +				       mtk-vcodec-enc.o \
> +				       mtk-vcodec-common.o
> +
> +mtk-vcodec-dec-y := mtk_vcodec_dec_drv.o \
> +		vdec_drv_if.o \
> +		mtk_vcodec_dec.o \
> +		mtk_vcodec_dec_pm.o \
>  
>  
>  mtk-vcodec-enc-y := venc/venc_vp8_if.o \
> diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c
> new file mode 100644
> index 0000000..792858a
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c
> @@ -0,0 +1,1348 @@
> +/*
> +* Copyright (c) 2016 MediaTek Inc.
> +* Author: PC Chen <pc.chen@mediatek.com>
> +*         Tiffany Lin <tiffany.lin@mediatek.com>
> +*
> +* This program is free software; you can redistribute it and/or modify
> +* it under the terms of the GNU General Public License version 2 as
> +* published by the Free Software Foundation.
> +*
> +* This program is distributed in the hope that it will be useful,
> +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +* GNU General Public License for more details.
> +*/
> +
> +#include <media/v4l2-event.h>
> +#include <media/v4l2-mem2mem.h>
> +#include <media/videobuf2-dma-contig.h>
> +
> +#include "mtk_vcodec_drv.h"
> +#include "mtk_vcodec_dec.h"
> +#include "mtk_vcodec_intr.h"
> +#include "mtk_vcodec_util.h"
> +#include "vdec_drv_if.h"
> +#include "mtk_vcodec_dec_pm.h"
> +
> +#define OUT_FMT_IDX	0
> +#define CAP_FMT_IDX	3
> +
> +#define MTK_VDEC_MIN_W	64U
> +#define MTK_VDEC_MIN_H	64U
> +#define DFT_CFG_WIDTH	MTK_VDEC_MIN_W
> +#define DFT_CFG_HEIGHT	MTK_VDEC_MIN_H
> +
> +static struct mtk_video_fmt mtk_video_formats[] = {
> +	{
> +		.fourcc = V4L2_PIX_FMT_H264,
> +		.type = MTK_FMT_DEC,
> +		.num_planes = 1,
> +	},
> +	{
> +		.fourcc = V4L2_PIX_FMT_VP8,
> +		.type = MTK_FMT_DEC,
> +		.num_planes = 1,
> +	},
> +	{
> +		.fourcc = V4L2_PIX_FMT_VP9,
> +		.type = MTK_FMT_DEC,
> +		.num_planes = 1,
> +	},
> +	{
> +		.fourcc = V4L2_PIX_FMT_MT21,
> +		.type = MTK_FMT_FRAME,
> +		.num_planes = 2,
> +	},
> +};
> +
> +static const struct mtk_codec_framesizes mtk_vdec_framesizes[] = {
> +	{
> +		.fourcc	= V4L2_PIX_FMT_H264,
> +		.stepwise = {  MTK_VDEC_MIN_W, MTK_VDEC_MAX_W, 16,
> +				MTK_VDEC_MIN_H, MTK_VDEC_MAX_H, 16 },
> +	},
> +	{
> +		.fourcc	= V4L2_PIX_FMT_VP8,
> +		.stepwise = {  MTK_VDEC_MIN_W, MTK_VDEC_MAX_W, 16,
> +				MTK_VDEC_MIN_H, MTK_VDEC_MAX_H, 16 },
> +	},
> +	{
> +		.fourcc = V4L2_PIX_FMT_VP9,
> +		.stepwise = {  MTK_VDEC_MIN_W, MTK_VDEC_MAX_W, 16,
> +				MTK_VDEC_MIN_H, MTK_VDEC_MAX_H, 16 },
> +	},
> +};
> +
> +#define NUM_SUPPORTED_FRAMESIZE ARRAY_SIZE(mtk_vdec_framesizes)
> +#define NUM_FORMATS ARRAY_SIZE(mtk_video_formats)
> +
> +static struct mtk_video_fmt *mtk_vdec_find_format(struct v4l2_format *f)
> +{
> +	struct mtk_video_fmt *fmt;
> +	unsigned int k;
> +
> +	for (k = 0; k < NUM_FORMATS; k++) {
> +		fmt = &mtk_video_formats[k];
> +		if (fmt->fourcc == f->fmt.pix_mp.pixelformat)
> +			return fmt;
> +	}
> +
> +	return NULL;
> +}
> +
> +static struct mtk_q_data *mtk_vdec_get_q_data(struct mtk_vcodec_ctx *ctx,
> +					      enum v4l2_buf_type type)
> +{
> +	if (V4L2_TYPE_IS_OUTPUT(type))
> +		return &ctx->q_data[MTK_Q_DATA_SRC];
> +
> +	return &ctx->q_data[MTK_Q_DATA_DST];
> +}
> +
> +/* This function is try to clean all display buffer, the buffer will return in

s/is try/tries/
s/buffer/buffers/ (2 times)

checkpatch should complain about this comment block format ('/*' should be on
its own line). Please run checkpatch to double check.

> + * display order.
> + * Note the buffer return from codec driver may still in driver's reference

s/buffer return/buffers returned/
s/still/still be/

> + * list.
> + */
> +static struct vb2_buffer *get_display_buffer(struct mtk_vcodec_ctx *ctx)
> +{
> +	struct vdec_fb *disp_frame_buffer = NULL;
> +	struct mtk_video_dec_buf *dstbuf;
> +
> +	mtk_v4l2_debug(3, "[%d]", ctx->id);
> +	if (vdec_if_get_param(ctx,
> +			GET_PARAM_DISP_FRAME_BUFFER,
> +			&disp_frame_buffer)) {
> +		mtk_v4l2_err("[%d]Cannot get param : GET_PARAM_DISP_FRAME_BUFFER",
> +			ctx->id);
> +		return NULL;
> +	}
> +
> +	if (disp_frame_buffer == NULL) {
> +		mtk_v4l2_debug(3, "No display frame buffer");
> +		return NULL;
> +	}
> +
> +	dstbuf = container_of(disp_frame_buffer, struct mtk_video_dec_buf,
> +				frame_buffer);
> +	mutex_lock(&ctx->lock);
> +	if (dstbuf->used) {
> +		vb2_set_plane_payload(&dstbuf->vb.vb2_buf, 0,
> +					ctx->picinfo.y_bs_sz);
> +		vb2_set_plane_payload(&dstbuf->vb.vb2_buf, 1,
> +					ctx->picinfo.c_bs_sz);
> +
> +		dstbuf->ready_to_display = true;
> +
> +		mtk_v4l2_debug(2,
> +				"[%d]status=%x queue id=%d to done_list %d",
> +				ctx->id, disp_frame_buffer->status,
> +				dstbuf->vb.vb2_buf.index,
> +				dstbuf->queued_in_vb2);
> +
> +		v4l2_m2m_buf_done(&dstbuf->vb, VB2_BUF_STATE_DONE);
> +		ctx->decoded_frame_cnt++;
> +	}
> +	mutex_unlock(&ctx->lock);
> +	return &dstbuf->vb.vb2_buf;
> +}
> +
> +/* This function is try to clean all capture buffers that is not used as

s/is try/tries/
s/is not/are not/

> + * reference buffer by codec driver any more

buffers

> + * In this case, we need re-queue buffer to vb2 buffer if user space
> + * already return this buffer to v4l2 or this buffer is just the output of

returns

> + * previous sps/pps/resolution change decode, or do nothing if user
> + * space still own this buffer

owns

> + */
> +static struct vb2_buffer *get_free_buffer(struct mtk_vcodec_ctx *ctx)
> +{
> +	struct mtk_video_dec_buf *dstbuf;
> +	struct vdec_fb *free_frame_buffer = NULL;
> +
> +	if (vdec_if_get_param(ctx,
> +				GET_PARAM_FREE_FRAME_BUFFER,
> +				&free_frame_buffer)) {
> +		mtk_v4l2_err("[%d] Error!! Cannot get param", ctx->id);
> +		return NULL;
> +	}
> +	if (free_frame_buffer == NULL) {
> +		mtk_v4l2_debug(3, " No free frame buffer");
> +		return NULL;
> +	}
> +
> +	mtk_v4l2_debug(3, "[%d] tmp_frame_addr = 0x%p",
> +			ctx->id, free_frame_buffer);
> +
> +	dstbuf = container_of(free_frame_buffer, struct mtk_video_dec_buf,
> +				frame_buffer);
> +
> +	mutex_lock(&ctx->lock);
> +	if (dstbuf->used) {
> +		if ((dstbuf->queued_in_vb2) &&
> +		    (dstbuf->queued_in_v4l2) &&
> +		    (free_frame_buffer->status == FB_ST_FREE)) {
> +			/* After decode sps/pps or non-display buffer, we dont

don't

> +			 * need to return capture buffer to user space, but
> +			 * just re-queue this capture buffer to vb2 queue.

> +			 * This reduce overheads that dq/q unused capture
> +			 * buffer We use queued_in_vb2 to mark this buffer
> +			 * is from decode output, not q from user space.

garbled comment. Not sure what is meant here.

> +			 */
> +			mtk_v4l2_debug(2,
> +				"[%d]status=%x queue id=%d to rdy_queue %d",
> +				ctx->id, free_frame_buffer->status,
> +				dstbuf->vb.vb2_buf.index,
> +				dstbuf->queued_in_vb2);
> +			v4l2_m2m_buf_queue(ctx->m2m_ctx, &dstbuf->vb);
> +		} else if ((dstbuf->queued_in_vb2 == false) &&
> +			   (dstbuf->queued_in_v4l2 == true)) {
> +			/* If buffer in v4l2 driver but not in vb2 queue yet,
> +			 * and we get this buffer from free_list, it means
> +			 * that codec driver do not use this buffer as
> +			 * reference buffer anymore. We should q buffer to vb2
> +			 * queue, so later work thread could get this buffer
> +			 * for decode In this sate, queued_in_vb2 = false

s/sate/case/

> +			 * menas this buffer is not from previous decode output

means

> +			 */
> +			mtk_v4l2_debug(2,
> +					"[%d]status=%x queue id=%d to rdy_queue",
> +					ctx->id, free_frame_buffer->status,
> +					dstbuf->vb.vb2_buf.index);
> +			v4l2_m2m_buf_queue(ctx->m2m_ctx, &dstbuf->vb);
> +			dstbuf->queued_in_vb2 = true;
> +		} else {
> +			/* Codec driver do not need to reference this capture
> +			 * buffer and this buffer is not in v4l2 driver.
> +			 * Then we dont need to do any thing, just add log when

don't

> +			 * we need to debug buffer flow.
> +			 * When this buffer q from user space, it could
> +			 * directly q to vb2 buffer
> +			 */
> +			mtk_v4l2_debug(3, "[%d]status=%x err queue id=%d %d %d",
> +					ctx->id, free_frame_buffer->status,
> +					dstbuf->vb.vb2_buf.index,
> +					dstbuf->queued_in_vb2,
> +					dstbuf->queued_in_v4l2);
> +		}
> +		dstbuf->used = false;
> +	}
> +	mutex_unlock(&ctx->lock);
> +	return &dstbuf->vb.vb2_buf;
> +}
> +
> +static void clean_display_buffer(struct mtk_vcodec_ctx *ctx)
> +{
> +	struct vb2_buffer *framptr;
> +
> +	do {
> +		framptr = get_display_buffer(ctx);
> +	} while (framptr);
> +}
> +
> +static void clean_free_buffer(struct mtk_vcodec_ctx *ctx)
> +{
> +	struct vb2_buffer *framptr;
> +
> +	do {
> +		framptr = get_free_buffer(ctx);
> +	} while (framptr);
> +}
> +
> +static int mtk_vdec_queue_res_chg_event(struct mtk_vcodec_ctx *ctx)

This can be a void function.

> +{
> +	static const struct v4l2_event ev_src_ch = {
> +		.type = V4L2_EVENT_SOURCE_CHANGE,
> +		.u.src_change.changes =
> +		V4L2_EVENT_SRC_CH_RESOLUTION,
> +	};
> +
> +	mtk_v4l2_debug(1, "[%d]", ctx->id);
> +	v4l2_event_queue_fh(&ctx->fh, &ev_src_ch);
> +
> +	return 0;
> +}
> +
> +static void mtk_vdec_flush_decoder(struct mtk_vcodec_ctx *ctx)
> +{
> +	bool res_chg;
> +	int ret = 0;
> +
> +	ret = vdec_if_decode(ctx, NULL, NULL, &res_chg);
> +	if (ret)
> +		mtk_v4l2_err("DecodeFinal failed, ret=%d", ret);
> +
> +	clean_display_buffer(ctx);
> +	clean_free_buffer(ctx);
> +}
> +
> +static void mtk_vdec_pic_info_update(struct mtk_vcodec_ctx *ctx)
> +{
> +	unsigned int dpbsize = 0;
> +	int ret;
> +
> +	if (vdec_if_get_param(ctx,
> +				GET_PARAM_PIC_INFO,
> +				&ctx->last_decoded_picinfo)) {
> +		mtk_v4l2_err("[%d]Error!! Cannot get param : GET_PARAM_PICTURE_INFO ERR",
> +				ctx->id);
> +		return;
> +	}
> +
> +	if (ctx->last_decoded_picinfo.pic_w == 0 ||
> +		ctx->last_decoded_picinfo.pic_h == 0 ||
> +		ctx->last_decoded_picinfo.buf_w == 0 ||
> +		ctx->last_decoded_picinfo.buf_h == 0) {
> +		mtk_v4l2_err("Cannot get correct pic info");
> +		return;
> +	}
> +
> +	if ((ctx->last_decoded_picinfo.pic_w == ctx->picinfo.pic_w) ||
> +	    (ctx->last_decoded_picinfo.pic_h == ctx->picinfo.pic_h))
> +		return;
> +
> +	mtk_v4l2_debug(1,
> +			"[%d]-> new(%d,%d), old(%d,%d), real(%d,%d)",
> +			ctx->id, ctx->last_decoded_picinfo.pic_w,
> +			ctx->last_decoded_picinfo.pic_h,
> +			ctx->picinfo.pic_w, ctx->picinfo.pic_h,
> +			ctx->last_decoded_picinfo.buf_w,
> +			ctx->last_decoded_picinfo.buf_h);
> +
> +	ret = vdec_if_get_param(ctx, GET_PARAM_DPB_SIZE, &dpbsize);
> +	if (dpbsize == 0)
> +		mtk_v4l2_err("Incorrect dpb size, ret=%d", ret);
> +
> +	ctx->dpb_size = dpbsize;
> +}
> +
> +static void mtk_vdec_worker(struct work_struct *work)
> +{
> +	struct mtk_vcodec_ctx *ctx = container_of(work, struct mtk_vcodec_ctx,
> +				decode_work);
> +	struct mtk_vcodec_dev *dev = ctx->dev;
> +	struct vb2_buffer *src_buf, *dst_buf;
> +	struct mtk_vcodec_mem buf;
> +	struct vdec_fb *pfb;
> +	bool res_chg = false;
> +	int ret;
> +	struct mtk_video_dec_buf *dst_buf_info, *src_buf_info;
> +	struct vb2_v4l2_buffer *dst_vb2_v4l2, *src_vb2_v4l2;
> +
> +	src_buf = v4l2_m2m_next_src_buf(ctx->m2m_ctx);
> +	if (src_buf == NULL) {
> +		v4l2_m2m_job_finish(dev->m2m_dev_dec, ctx->m2m_ctx);
> +		mtk_v4l2_debug(1, "[%d] src_buf empty!!", ctx->id);
> +		return;
> +	}
> +
> +	dst_buf = v4l2_m2m_next_dst_buf(ctx->m2m_ctx);
> +	if (dst_buf == NULL) {
> +		v4l2_m2m_job_finish(dev->m2m_dev_dec, ctx->m2m_ctx);
> +		mtk_v4l2_debug(1, "[%d] dst_buf empty!!", ctx->id);
> +		return;
> +	}
> +
> +	src_vb2_v4l2 = container_of(src_buf, struct vb2_v4l2_buffer, vb2_buf);
> +	src_buf_info = container_of(src_vb2_v4l2, struct mtk_video_dec_buf, vb);
> +
> +	dst_vb2_v4l2 = container_of(dst_buf, struct vb2_v4l2_buffer, vb2_buf);
> +	dst_buf_info = container_of(dst_vb2_v4l2, struct mtk_video_dec_buf, vb);
> +
> +	buf.va = vb2_plane_vaddr(src_buf, 0);
> +	buf.dma_addr = vb2_dma_contig_plane_dma_addr(src_buf, 0);
> +	buf.size = (size_t)src_buf->planes[0].bytesused;
> +	if (!buf.va) {
> +		v4l2_m2m_job_finish(dev->m2m_dev_dec, ctx->m2m_ctx);
> +		mtk_v4l2_err("[%d] id=%d src_addr is NULL!!",
> +				ctx->id, src_buf->index);
> +		return;
> +	}
> +
> +	pfb = &dst_buf_info->frame_buffer;
> +	pfb->base_y.va = vb2_plane_vaddr(dst_buf, 0);
> +	pfb->base_y.dma_addr = vb2_dma_contig_plane_dma_addr(dst_buf, 0);
> +	pfb->base_y.size = ctx->picinfo.y_bs_sz + ctx->picinfo.y_len_sz;
> +
> +	pfb->base_c.va = vb2_plane_vaddr(dst_buf, 1);
> +	pfb->base_c.dma_addr = vb2_dma_contig_plane_dma_addr(dst_buf, 1);
> +	pfb->base_c.size = ctx->picinfo.c_bs_sz + ctx->picinfo.c_len_sz;
> +	pfb->status = 0;
> +	mtk_v4l2_debug(3, "===>[%d] vdec_if_decode() ===>", ctx->id);
> +	mtk_v4l2_debug(3, "[%d] Bitstream VA=%p DMA=%llx Size=0x%lx vb=%p",
> +			ctx->id, buf.va, (u64)buf.dma_addr, buf.size, src_buf);
> +
> +	mtk_v4l2_debug(3,
> +			"id=%d Framebuf  pfb=%p VA=%p Y_DMA=%llx C_DMA=%llx Size=0x%lx",
> +			dst_buf->index, pfb,
> +			pfb->base_y.va, (u64)pfb->base_y.dma_addr,
> +			(u64)pfb->base_c.dma_addr, pfb->base_y.size);
> +
> +	if (src_buf_info->lastframe) {
> +		/* update src buf status */
> +		src_buf = v4l2_m2m_src_buf_remove(ctx->m2m_ctx);
> +		src_buf_info->lastframe = false;
> +		v4l2_m2m_buf_done(&src_buf_info->vb, VB2_BUF_STATE_DONE);
> +
> +		/* update dst buf status */
> +		dst_buf = v4l2_m2m_dst_buf_remove(ctx->m2m_ctx);
> +		dst_buf_info->used = false;
> +
> +		vdec_if_decode(ctx, NULL, NULL, &res_chg);
> +		clean_display_buffer(ctx);
> +		vb2_set_plane_payload(&dst_buf_info->vb.vb2_buf, 0, 0);
> +		vb2_set_plane_payload(&dst_buf_info->vb.vb2_buf, 1, 0);
> +		v4l2_m2m_buf_done(&dst_buf_info->vb, VB2_BUF_STATE_DONE);
> +		clean_free_buffer(ctx);

Just call v4l2_m2m_job_finish(dev->m2m_dev_dec, ctx->m2m_ctx); here and return.


> +	} else {

This can now have one less TAB.

> +		dst_buf_info->vb.vb2_buf.timestamp
> +				= src_buf_info->vb.vb2_buf.timestamp;
> +		dst_buf_info->vb.timecode
> +				= src_buf_info->vb.timecode;
> +		mutex_lock(&ctx->lock);
> +		dst_buf_info->used = true;
> +		mutex_unlock(&ctx->lock);
> +		src_buf_info->used = true;
> +
> +		ret = vdec_if_decode(ctx, &buf, pfb, &res_chg);
> +
> +		if (ret) {
> +			mtk_v4l2_err(
> +				" <===[%d], src_buf[%d]%d sz=0x%zx pts=%llu dst_buf[%d] vdec_if_decode() ret=%d res_chg=%d===>",
> +				ctx->id,
> +				src_buf->index,
> +				src_buf_info->lastframe,
> +				buf.size,
> +				src_buf_info->vb.vb2_buf.timestamp,
> +				dst_buf->index,
> +				ret, res_chg);
> +			src_buf = v4l2_m2m_src_buf_remove(ctx->m2m_ctx);
> +			v4l2_m2m_buf_done(&src_buf_info->vb,
> +					VB2_BUF_STATE_ERROR);
> +		} else if (res_chg == false) {
> +			/* we only return src buffer with VB2_BUF_STATE_DONE
> +			  * when decode success without resolution change
> +			  */
> +			src_buf = v4l2_m2m_src_buf_remove(ctx->m2m_ctx);
> +			v4l2_m2m_buf_done(&src_buf_info->vb,
> +					VB2_BUF_STATE_DONE);
> +		}
> +
> +		dst_buf = v4l2_m2m_dst_buf_remove(ctx->m2m_ctx);
> +		clean_display_buffer(ctx);
> +		clean_free_buffer(ctx);
> +
> +		if (!ret && res_chg) {
> +			mtk_vdec_pic_info_update(ctx);
> +			/*
> +			 * On encountering a resolution change in the stream.
> +			 * The driver must first process and decode all
> +			 * remaining buffers from before the resolution change
> +			 * point, so call flush decode here
> +			 */
> +			mtk_vdec_flush_decoder(ctx);
> +			/*
> +			 * After all buffers containing decoded frames from
> +			 * before the resolution change point ready to be
> +			 * dequeued on the CAPTURE queue, the driver sends a
> +			 * V4L2_EVENT_SOURCE_CHANGE event for source change
> +			 * type V4L2_EVENT_SRC_CH_RESOLUTION
> +			 */
> +			mtk_vdec_queue_res_chg_event(ctx);
> +		}
> +	}
> +
> +	v4l2_m2m_job_finish(dev->m2m_dev_dec, ctx->m2m_ctx);
> +}
> +
> +void mtk_vdec_unlock(struct mtk_vcodec_ctx *ctx)
> +{
> +	mutex_unlock(&ctx->dev->dec_mutex);
> +}
> +
> +void mtk_vdec_lock(struct mtk_vcodec_ctx *ctx)
> +{
> +	mutex_lock(&ctx->dev->dec_mutex);
> +}
> +
> +void mtk_vcodec_vdec_release(struct mtk_vcodec_ctx *ctx)
> +{
> +	vdec_if_deinit(ctx);
> +	ctx->state = MTK_STATE_FREE;
> +}
> +
> +void mtk_vcodec_dec_set_default_params(struct mtk_vcodec_ctx *ctx)
> +{
> +	struct mtk_q_data *q_data;
> +
> +	ctx->m2m_ctx->q_lock = &ctx->dev->dev_mutex;
> +	ctx->fh.m2m_ctx = ctx->m2m_ctx;
> +	ctx->fh.ctrl_handler = &ctx->ctrl_hdl;
> +	INIT_WORK(&ctx->decode_work, mtk_vdec_worker);
> +	ctx->colorspace = V4L2_COLORSPACE_REC709;
> +	ctx->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
> +	ctx->quantization = V4L2_QUANTIZATION_DEFAULT;
> +	ctx->xfer_func = V4L2_XFER_FUNC_DEFAULT;
> +
> +	q_data = &ctx->q_data[MTK_Q_DATA_SRC];
> +	memset(q_data, 0, sizeof(struct mtk_q_data));
> +	q_data->visible_width = DFT_CFG_WIDTH;
> +	q_data->visible_height = DFT_CFG_HEIGHT;
> +	q_data->fmt = &mtk_video_formats[OUT_FMT_IDX];
> +	q_data->field = V4L2_FIELD_NONE;
> +
> +	ctx->q_data[MTK_Q_DATA_DST].sizeimage[0] =
> +		DFT_CFG_WIDTH * DFT_CFG_HEIGHT;
> +	ctx->q_data[MTK_Q_DATA_DST].bytesperline[0] = 0;
> +
> +	q_data = &ctx->q_data[MTK_Q_DATA_DST];
> +	memset(q_data, 0, sizeof(struct mtk_q_data));
> +	q_data->visible_width = DFT_CFG_WIDTH;
> +	q_data->visible_height = DFT_CFG_HEIGHT;
> +	q_data->coded_width = DFT_CFG_WIDTH;
> +	q_data->coded_height = DFT_CFG_HEIGHT;
> +	q_data->fmt = &mtk_video_formats[CAP_FMT_IDX];
> +	q_data->field = V4L2_FIELD_NONE;
> +
> +	v4l_bound_align_image(&q_data->coded_width,
> +				MTK_VDEC_MIN_W,
> +				MTK_VDEC_MAX_W, 4,
> +				&q_data->coded_height,
> +				MTK_VDEC_MIN_H,
> +				MTK_VDEC_MAX_H, 5, 6);
> +
> +	q_data->sizeimage[0] = q_data->coded_width * q_data->coded_height;
> +	q_data->bytesperline[0] = q_data->coded_width;
> +	q_data->sizeimage[1] = q_data->sizeimage[0] / 2;
> +	q_data->bytesperline[1] = q_data->coded_width;
> +}
> +
> +static int vidioc_vdec_qbuf(struct file *file, void *priv,
> +			    struct v4l2_buffer *buf)
> +{
> +	struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
> +	struct vb2_queue *vq;
> +	struct vb2_buffer *vb;
> +	struct mtk_video_dec_buf *mtkbuf;
> +	struct vb2_v4l2_buffer	*vb2_v4l2;
> +
> +	if (ctx->state == MTK_STATE_ABORT) {
> +		mtk_v4l2_err("[%d] Call on QBUF after unrecoverable error",
> +				ctx->id);
> +		return -EIO;
> +	}
> +
> +	vq = v4l2_m2m_get_vq(ctx->m2m_ctx, buf->type);
> +	vb = vq->bufs[buf->index];
> +	vb2_v4l2 = container_of(vb, struct vb2_v4l2_buffer, vb2_buf);
> +	mtkbuf = container_of(vb2_v4l2, struct mtk_video_dec_buf, vb);
> +
> +	if ((buf->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) &&
> +	    (buf->m.planes[0].bytesused == 0)) {
> +		mtkbuf->lastframe = true;
> +		mtk_v4l2_debug(1, "[%d] (%d) id=%d lastframe=%d (%d,%d, %d) vb=%p",
> +			 ctx->id, buf->type, buf->index,
> +			 mtkbuf->lastframe, buf->bytesused,
> +			 buf->m.planes[0].bytesused, buf->length,
> +			 vb);
> +	}
> +
> +	return v4l2_m2m_qbuf(file, ctx->m2m_ctx, buf);
> +}
> +
> +static int vidioc_vdec_dqbuf(struct file *file, void *priv,
> +			     struct v4l2_buffer *buf)
> +{
> +	struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
> +
> +	if (ctx->state == MTK_STATE_ABORT) {
> +		mtk_v4l2_err("[%d] Call on DQBUF after unrecoverable error",
> +				ctx->id);
> +		return -EIO;
> +	}
> +
> +	return v4l2_m2m_dqbuf(file, ctx->m2m_ctx, buf);
> +}
> +
> +static int vidioc_vdec_querycap(struct file *file, void *priv,
> +				struct v4l2_capability *cap)
> +{
> +	strlcpy(cap->driver, MTK_VCODEC_DEC_NAME, sizeof(cap->driver));
> +	strlcpy(cap->bus_info, MTK_PLATFORM_STR, sizeof(cap->bus_info));
> +	strlcpy(cap->card, MTK_PLATFORM_STR, sizeof(cap->card));
> +
> +	return 0;
> +}
> +
> +static int vidioc_vdec_subscribe_evt(struct v4l2_fh *fh,
> +				     const struct v4l2_event_subscription *sub)
> +{
> +	switch (sub->type) {
> +	case V4L2_EVENT_EOS:
> +		return v4l2_event_subscribe(fh, sub, 2, NULL);
> +	case V4L2_EVENT_SOURCE_CHANGE:
> +		return v4l2_src_change_event_subscribe(fh, sub);
> +	default:
> +		return v4l2_ctrl_subscribe_event(fh, sub);
> +	}
> +}
> +
> +static int vidioc_try_fmt_vid_cap_mplane(struct file *file, void *priv,
> +				struct v4l2_format *f)
> +{
> +	struct mtk_video_fmt *fmt;
> +
> +	fmt = mtk_vdec_find_format(f);
> +	if (!fmt) {
> +		f->fmt.pix.pixelformat = mtk_video_formats[CAP_FMT_IDX].fourcc;
> +		fmt = mtk_vdec_find_format(f);
> +	}
> +
> +	return 0;
> +}
> +
> +static int vidioc_try_fmt_vid_out_mplane(struct file *file, void *priv,
> +				struct v4l2_format *f)
> +{
> +	struct v4l2_pix_format_mplane *pix_fmt_mp = &f->fmt.pix_mp;
> +	struct mtk_video_fmt *fmt;
> +
> +	fmt = mtk_vdec_find_format(f);
> +	if (!fmt) {
> +		f->fmt.pix.pixelformat = mtk_video_formats[OUT_FMT_IDX].fourcc;
> +		fmt = mtk_vdec_find_format(f);
> +	}
> +
> +	if (pix_fmt_mp->plane_fmt[0].sizeimage == 0) {
> +		mtk_v4l2_err("sizeimage of output format must be given");
> +		return -EINVAL;
> +	}
> +
> +	return 0;
> +}
> +
> +static int vidioc_vdec_g_selection(struct file *file, void *priv,
> +			struct v4l2_selection *s)
> +{
> +	struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
> +
> +	if (V4L2_TYPE_IS_OUTPUT(s->type))
> +		return -EINVAL;
> +
> +	if (s->target != V4L2_SEL_TGT_CROP)
> +		return -EINVAL;

How does the cropping rectangle relate to the format size? There is no s_selection,
so this doesn't make sense.

Alternatively, it could be that you are really returning V4L2_SEL_TGT_COMPOSE_PADDED.

> +
> +	if (ctx->state < MTK_STATE_HEADER)
> +		return -EINVAL;
> +
> +	if ((ctx->q_data[MTK_Q_DATA_SRC].fmt->fourcc == V4L2_PIX_FMT_H264) ||
> +	    (ctx->q_data[MTK_Q_DATA_SRC].fmt->fourcc == V4L2_PIX_FMT_VP8) ||
> +	    (ctx->q_data[MTK_Q_DATA_SRC].fmt->fourcc == V4L2_PIX_FMT_VP9)) {
> +
> +		if (vdec_if_get_param(ctx, GET_PARAM_CROP_INFO, &(s->r))) {
> +			mtk_v4l2_debug(2,
> +					"[%d]Error!! Cannot get param : GET_PARAM_CROP_INFO ERR",
> +					ctx->id);
> +			s->r.left = 0;
> +			s->r.top = 0;
> +			s->r.width = ctx->picinfo.buf_w;
> +			s->r.height = ctx->picinfo.buf_h;
> +		}
> +		mtk_v4l2_debug(2, "Cropping info: l=%d t=%d w=%d h=%d",
> +				s->r.left, s->r.top, s->r.width,
> +				s->r.height);
> +	} else {
> +		s->r.left = 0;
> +		s->r.top = 0;
> +		s->r.width = ctx->picinfo.pic_w;
> +		s->r.height = ctx->picinfo.pic_h;
> +		mtk_v4l2_debug(2, "Cropping info: w=%d h=%d fw=%d fh=%d",
> +				s->r.width, s->r.height, ctx->picinfo.buf_w,
> +				ctx->picinfo.buf_h);
> +	}
> +	return 0;
> +}
> +
> +static int vidioc_vdec_s_fmt(struct file *file, void *priv,
> +			     struct v4l2_format *f)
> +{
> +	struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
> +	struct v4l2_pix_format_mplane *pix_mp;
> +	struct mtk_q_data *q_data;
> +	int ret = 0;
> +	struct mtk_video_fmt *fmt;
> +	struct vb2_queue *vq;
> +
> +	mtk_v4l2_debug(3, "[%d]", ctx->id);
> +
> +	fmt = mtk_vdec_find_format(f);
> +	if (fmt == NULL) {
> +		mtk_v4l2_err("vidioc_vdec_s_fmt fail");
> +		return ret;
> +	}
> +
> +	pix_mp = &f->fmt.pix_mp;
> +	if ((f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) &&
> +	    vb2_is_busy(&ctx->m2m_ctx->out_q_ctx.q)) {
> +		mtk_v4l2_err("out_q_ctx buffers already requested");
> +		ret = -EBUSY;
> +	}
> +
> +	if ((f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) &&
> +	    vb2_is_busy(&ctx->m2m_ctx->cap_q_ctx.q)) {
> +		mtk_v4l2_err("cap_q_ctx buffers already requested");
> +		ret = -EBUSY;
> +	}
> +
> +	vq = v4l2_m2m_get_vq(ctx->m2m_ctx, f->type);
> +
> +	q_data = mtk_vdec_get_q_data(ctx, f->type);
> +	if (!q_data)
> +		return -EINVAL;
> +
> +	q_data->fmt = fmt;
> +	if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
> +		q_data->sizeimage[0] = pix_mp->plane_fmt[0].sizeimage;
> +		pix_mp->plane_fmt[0].bytesperline = 0;
> +		pix_mp->width = 0;
> +		pix_mp->height = 0;
> +
> +		if (ctx->state == MTK_STATE_FREE) {
> +			ret = vdec_if_init(ctx, q_data->fmt->fourcc);
> +			if (ret) {
> +				mtk_v4l2_err("[%d]: vdec_if_init() fail ret=%d",
> +					ctx->id, ret);
> +				return -EINVAL;
> +			}
> +			ctx->state = MTK_STATE_INIT;
> +		}
> +	}
> +
> +	return 0;
> +}
> +
> +static int vidioc_enum_framesizes(struct file *file, void *priv,
> +				struct v4l2_frmsizeenum *fsize)
> +{
> +	int i = 0;
> +	struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
> +
> +	if (fsize->index != 0)
> +		return -EINVAL;
> +
> +	for (i = 0; i < NUM_SUPPORTED_FRAMESIZE; ++i) {
> +		if (fsize->pixel_format != mtk_vdec_framesizes[i].fourcc)
> +			continue;
> +
> +		fsize->type = V4L2_FRMSIZE_TYPE_STEPWISE;
> +		fsize->stepwise = mtk_vdec_framesizes[i].stepwise;
> +		if (!(ctx->dev->dec_capability &
> +				VCODEC_CAPABILITY_4K_DISABLED)) {
> +			mtk_v4l2_debug(3, "4K is enabled");
> +			fsize->stepwise.max_width =
> +					VCODEC_DEC_4K_CODED_WIDTH;
> +			fsize->stepwise.max_height =
> +					VCODEC_DEC_4K_CODED_HEIGHT;
> +		}
> +		mtk_v4l2_debug(1, "%x, %d %d %d %d %d %d",
> +				ctx->dev->dec_capability,
> +				fsize->stepwise.min_width,
> +				fsize->stepwise.max_width,
> +				fsize->stepwise.step_width,
> +				fsize->stepwise.min_height,
> +				fsize->stepwise.max_height,
> +				fsize->stepwise.step_height);
> +		return 0;
> +	}
> +
> +	return -EINVAL;
> +}
> +
> +static int vidioc_enum_fmt(struct v4l2_fmtdesc *f, bool output_queue)
> +{
> +	struct mtk_video_fmt *fmt;
> +	int i, j = 0;
> +
> +	for (i = 0; i < NUM_FORMATS; i++) {
> +		if (output_queue && (mtk_video_formats[i].type != MTK_FMT_DEC))
> +			continue;
> +		if (!output_queue &&
> +			(mtk_video_formats[i].type != MTK_FMT_FRAME))
> +			continue;
> +
> +		if (j == f->index)
> +			break;
> +		++j;
> +	}
> +
> +	if (i == NUM_FORMATS)
> +		return -EINVAL;
> +
> +	fmt = &mtk_video_formats[i];
> +	f->pixelformat = fmt->fourcc;
> +
> +	return 0;
> +}
> +
> +static int vidioc_vdec_enum_fmt_vid_cap_mplane(struct file *file, void *pirv,
> +					       struct v4l2_fmtdesc *f)
> +{
> +	return vidioc_enum_fmt(f, false);
> +}
> +
> +static int vidioc_vdec_enum_fmt_vid_out_mplane(struct file *file, void *priv,
> +					       struct v4l2_fmtdesc *f)
> +{
> +	return vidioc_enum_fmt(f, true);
> +}
> +
> +static int vidioc_vdec_g_fmt(struct file *file, void *priv,
> +			     struct v4l2_format *f)
> +{
> +	struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
> +	struct v4l2_pix_format_mplane *pix_mp = &f->fmt.pix_mp;
> +	struct vb2_queue *vq;
> +	struct mtk_q_data *q_data;
> +
> +	vq = v4l2_m2m_get_vq(ctx->m2m_ctx, f->type);
> +	if (!vq) {
> +		mtk_v4l2_err("no vb2 queue for type=%d",
> +			f->type);
> +		return -EINVAL;
> +	}
> +
> +	q_data = mtk_vdec_get_q_data(ctx, f->type);
> +
> +	if ((f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) &&
> +	    (ctx->state >= MTK_STATE_HEADER)) {
> +		/* Until STREAMOFF is called on the CAPTURE queue
> +		 * (acknowledging the event), the driver operates as if
> +		 * the resolution hasn't changed yet.
> +		 * So we just return picinfo yet, and update picinfo in
> +		 * stop_streaming hook function
> +		 */
> +		q_data->sizeimage[0] = ctx->picinfo.y_bs_sz +
> +					ctx->picinfo.y_len_sz;
> +		q_data->sizeimage[1] = ctx->picinfo.c_bs_sz +
> +					ctx->picinfo.c_len_sz;
> +		q_data->bytesperline[0] = ctx->last_decoded_picinfo.buf_w;
> +		q_data->bytesperline[1] = ctx->last_decoded_picinfo.buf_w;
> +		q_data->coded_width = ctx->picinfo.buf_w;
> +		q_data->coded_height = ctx->picinfo.buf_h;
> +
> +		/*
> +		 * Width and height are set to the dimensions
> +		 * of the movie, the buffer is bigger and
> +		 * further processing stages should crop to this
> +		 * rectangle.
> +		 */
> +		pix_mp->width = q_data->coded_width;
> +		pix_mp->height = q_data->coded_height;
> +		pix_mp->field = V4L2_FIELD_NONE;
> +
> +		/*
> +		 * Set pixelformat to the format in which mt vcodec
> +		 * outputs the decoded frame
> +		 */
> +		pix_mp->num_planes = q_data->fmt->num_planes;
> +		pix_mp->pixelformat = q_data->fmt->fourcc;
> +		pix_mp->plane_fmt[0].bytesperline = q_data->bytesperline[0];
> +		pix_mp->plane_fmt[0].sizeimage = q_data->sizeimage[0];
> +		pix_mp->plane_fmt[1].bytesperline = q_data->bytesperline[1];
> +		pix_mp->plane_fmt[1].sizeimage = q_data->sizeimage[1];
> +
> +	} else if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
> +		/*
> +		 * This is run on OUTPUT
> +		 * The buffer contains compressed image
> +		 * so width and height have no meaning.
> +		 * Assign value here to pass v4l2-compliance test
> +		 */
> +		pix_mp->width = q_data->visible_width;
> +		pix_mp->height = q_data->visible_height;
> +		pix_mp->field = V4L2_FIELD_NONE;
> +		pix_mp->plane_fmt[0].bytesperline = q_data->bytesperline[0];
> +		pix_mp->plane_fmt[0].sizeimage = q_data->sizeimage[0];
> +		pix_mp->pixelformat = q_data->fmt->fourcc;
> +		pix_mp->num_planes = q_data->fmt->num_planes;
> +	} else {
> +		pix_mp->width = q_data->coded_width;
> +		pix_mp->height = q_data->coded_height;
> +		pix_mp->field = V4L2_FIELD_NONE;
> +		pix_mp->num_planes = q_data->fmt->num_planes;
> +		pix_mp->pixelformat = q_data->fmt->fourcc;
> +		pix_mp->plane_fmt[0].bytesperline = q_data->bytesperline[0];
> +		pix_mp->plane_fmt[0].sizeimage = q_data->sizeimage[0];
> +		pix_mp->plane_fmt[1].bytesperline = q_data->bytesperline[1];
> +		pix_mp->plane_fmt[1].sizeimage = q_data->sizeimage[1];
> +
> +		mtk_v4l2_debug(1, "[%d] state=%d Format could not be read %d, not ready yet!",
> +				ctx->id, ctx->state, f->type);
> +		return -EINVAL;
> +	}
> +
> +	return 0;
> +}
> +
> +static int vb2ops_vdec_queue_setup(struct vb2_queue *vq,
> +				unsigned int *nbuffers,
> +				unsigned int *nplanes,
> +				unsigned int sizes[], void *alloc_ctxs[])
> +{
> +	struct mtk_vcodec_ctx *ctx = vb2_get_drv_priv(vq);
> +	struct mtk_q_data *q_data;
> +	unsigned int i;
> +
> +	q_data = mtk_vdec_get_q_data(ctx, vq->type);
> +
> +	if (q_data == NULL) {
> +		mtk_v4l2_err("vq->type=%d err\n", vq->type);
> +		return -EINVAL;
> +	}
> +
> +	if (*nplanes) {
> +		for (i = 0; i < *nplanes; i++) {
> +			if (sizes[i] < q_data->sizeimage[i])
> +				return -EINVAL;
> +			alloc_ctxs[i] = ctx->dev->alloc_ctx;
> +		}
> +	} else {
> +
> +		if (vq->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
> +			if (ctx->state < MTK_STATE_HEADER) {
> +				mtk_v4l2_err("State not ready");
> +				return -EINVAL;
> +			}
> +			/*
> +			 * Output plane count is 2 - one for Y and one for CbCr
> +			 */
> +			*nplanes = 2;
> +		} else {
> +			*nplanes = 1;
> +		}
> +
> +		for (i = 0; i < *nplanes; i++) {
> +			sizes[i] = q_data->sizeimage[i];
> +			alloc_ctxs[i] = ctx->dev->alloc_ctx;
> +		}
> +	}
> +
> +	mtk_v4l2_debug(1,
> +			"[%d]\t type = %d, get %d plane(s), %d buffer(s) of size 0x%x 0x%x ",
> +			ctx->id, vq->type, *nplanes, *nbuffers,
> +			sizes[0], sizes[1]);
> +
> +	return 0;
> +}
> +
> +static int vb2ops_vdec_buf_prepare(struct vb2_buffer *vb)
> +{
> +	struct mtk_vcodec_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue);
> +	struct mtk_q_data *q_data;
> +	int i;
> +
> +	mtk_v4l2_debug(3, "[%d] (%d) id=%d",
> +			ctx->id, vb->vb2_queue->type, vb->index);
> +
> +	q_data = mtk_vdec_get_q_data(ctx, vb->vb2_queue->type);
> +
> +	for (i = 0; i < q_data->fmt->num_planes; i++) {
> +		if (vb2_plane_size(vb, i) > q_data->sizeimage[i]) {

Shouldn't this be < instead of > ?

> +			mtk_v4l2_err("data will not fit into plane %d (%lu < %d)",
> +				i, vb2_plane_size(vb, i),
> +				q_data->sizeimage[i]);
> +		}
> +	}
> +
> +	return 0;
> +}
> +
> +static void vb2ops_vdec_buf_queue(struct vb2_buffer *vb)
> +{
> +	struct mtk_vcodec_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue);
> +	struct vb2_v4l2_buffer *vb2_v4l2 = container_of(vb,
> +				struct vb2_v4l2_buffer, vb2_buf);
> +	struct mtk_video_dec_buf *buf = container_of(vb2_v4l2,
> +				struct mtk_video_dec_buf, vb);
> +
> +	mtk_v4l2_debug(3, "[%d] (%d) id=%d, vb=%p",
> +			ctx->id, vb->vb2_queue->type,
> +			vb->index, vb);
> +	/*
> +	 * check if this buffer is ready to be used after decode
> +	 */
> +	if (vb->vb2_queue->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {

Do the 'else' part first and return there.

Then this can be shifted one TAB to the left.

> +		struct vb2_buffer *src_buf;
> +		struct mtk_vcodec_mem src_mem;
> +		bool res_chg = false;
> +		int ret = 0;
> +		unsigned int dpbsize = 1;
> +
> +		v4l2_m2m_buf_queue(ctx->m2m_ctx, vb2_v4l2);
> +
> +		if (ctx->state == MTK_STATE_INIT) {

Ditto.

> +			src_buf = v4l2_m2m_next_src_buf(ctx->m2m_ctx);
> +			if (!src_buf) {
> +				mtk_v4l2_err("No src buffer");
> +				return;
> +			}
> +
> +			src_mem.va	= vb2_plane_vaddr(src_buf, 0);
> +			src_mem.dma_addr = vb2_dma_contig_plane_dma_addr(
> +						src_buf,
> +						0);
> +			src_mem.size = (size_t)src_buf->planes[0].bytesused;
> +			mtk_v4l2_debug(2,
> +					"[%d] buf id=%d va=%p dma=%llx size=%zu",
> +					ctx->id, src_buf->index,
> +					src_mem.va, (u64)src_mem.dma_addr,
> +					src_mem.size);
> +
> +			ret = vdec_if_decode(ctx, &src_mem, NULL, &res_chg);
> +			if (ret) {
> +				/* fb == NULL menas to parse SPS/PPS header or
> +				 * resolution info in src_mem. Decode can fail
> +				 * if there is no SPS header or picture info
> +				 * in bs
> +				 */
> +				src_buf = v4l2_m2m_src_buf_remove(ctx->m2m_ctx);
> +				v4l2_m2m_buf_done(to_vb2_v4l2_buffer(src_buf),
> +						VB2_BUF_STATE_DONE);
> +				mtk_v4l2_debug(0, "[%d] vdec_if_decode() src_buf=%d, size=%zu, fail=%d",
> +						ctx->id, src_buf->index,
> +						src_mem.size, ret);
> +				return;
> +			}
> +
> +			if (vdec_if_get_param(ctx,
> +						GET_PARAM_PIC_INFO,
> +						&ctx->picinfo)) {
> +				mtk_v4l2_err("[%d]Error!! Cannot get param : GET_PARAM_PICTURE_INFO ERR",
> +						ctx->id);
> +				return;
> +			}
> +
> +			ctx->last_decoded_picinfo = ctx->picinfo;
> +			ctx->q_data[MTK_Q_DATA_DST].sizeimage[0] =
> +						ctx->picinfo.y_bs_sz +
> +						ctx->picinfo.y_len_sz;
> +			ctx->q_data[MTK_Q_DATA_DST].bytesperline[0] =
> +						ctx->picinfo.buf_w;
> +			ctx->q_data[MTK_Q_DATA_DST].sizeimage[1] =
> +						ctx->picinfo.c_bs_sz +
> +						ctx->picinfo.c_len_sz;
> +			ctx->q_data[MTK_Q_DATA_DST].bytesperline[1] =
> +						ctx->picinfo.buf_w;
> +			mtk_v4l2_debug(2, "[%d] vdec_if_init() OK wxh=%dx%d pic wxh=%dx%d sz[0]=0x%x sz[1]=0x%x",
> +				ctx->id,
> +				ctx->picinfo.buf_w, ctx->picinfo.buf_h,
> +				ctx->picinfo.pic_w, ctx->picinfo.pic_h,
> +				ctx->q_data[MTK_Q_DATA_DST].sizeimage[0],
> +				ctx->q_data[MTK_Q_DATA_DST].sizeimage[1]);
> +
> +			ret = vdec_if_get_param(ctx, GET_PARAM_DPB_SIZE,
> +						&dpbsize);
> +			if (dpbsize == 0) {
> +				mtk_v4l2_err("[%d] vdec_if_get_param()  GET_PARAM_DPB_SIZE fail=%d",
> +						ctx->id, ret);
> +			}
> +			ctx->dpb_size = dpbsize;
> +			ctx->state = MTK_STATE_HEADER;
> +			mtk_v4l2_debug(1, "[%d] dpbsize=%d", ctx->id,
> +					ctx->dpb_size);
> +		} else {
> +			mtk_v4l2_debug(3, "[%d] already init driver %d",
> +					ctx->id, ctx->state);
> +		}
> +	} else {
> +		mutex_lock(&ctx->lock);
> +		if (buf->used == false) {
> +			v4l2_m2m_buf_queue(ctx->m2m_ctx,
> +					to_vb2_v4l2_buffer(vb));
> +			buf->queued_in_vb2 = true;
> +			buf->queued_in_v4l2 = true;
> +			buf->ready_to_display = false;
> +		} else {
> +			buf->queued_in_vb2 = false;
> +			buf->queued_in_v4l2 = true;
> +			buf->ready_to_display = false;
> +		}
> +		mutex_unlock(&ctx->lock);
> +	}
> +}
> +
> +static void vb2ops_vdec_buf_finish(struct vb2_buffer *vb)
> +{
> +	struct mtk_vcodec_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue);
> +
> +	if (vb->vb2_queue->type ==
> +		V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {

Place this on one line.

Actually, just invert the condition and return. The remainder can then be shifted
one TAB to the left.

> +		struct vb2_v4l2_buffer *vb2_v4l2;
> +		struct mtk_video_dec_buf *buf;
> +
> +		vb2_v4l2 = container_of(vb, struct vb2_v4l2_buffer, vb2_buf);
> +		buf = container_of(vb2_v4l2, struct mtk_video_dec_buf, vb);
> +		mutex_lock(&ctx->lock);
> +		buf->queued_in_v4l2 = false;
> +		buf->queued_in_vb2 = false;
> +		mutex_unlock(&ctx->lock);
> +	}
> +}
> +
> +static int vb2ops_vdec_buf_init(struct vb2_buffer *vb)
> +{
> +	struct vb2_v4l2_buffer *vb2_v4l2 = container_of(vb,
> +					struct vb2_v4l2_buffer, vb2_buf);
> +	struct mtk_video_dec_buf *buf = container_of(vb2_v4l2,
> +					struct mtk_video_dec_buf, vb);
> +
> +	if (vb->vb2_queue->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
> +		buf->used = false;
> +		buf->ready_to_display = false;
> +		buf->queued_in_v4l2 = false;
> +	} else {
> +		buf->lastframe = false;
> +	}
> +
> +	return 0;
> +}
> +
> +static int vb2ops_vdec_start_streaming(struct vb2_queue *q, unsigned int count)
> +{
> +	struct mtk_vcodec_ctx *ctx = vb2_get_drv_priv(q);
> +
> +	if (ctx->state == MTK_STATE_FLUSH)
> +		ctx->state = MTK_STATE_HEADER;
> +
> +	return 0;
> +}
> +
> +static void vb2ops_vdec_stop_streaming(struct vb2_queue *q)
> +{
> +	struct vb2_buffer *src_buf = NULL, *dst_buf = NULL;
> +	struct mtk_vcodec_ctx *ctx = vb2_get_drv_priv(q);
> +
> +	mtk_v4l2_debug(3, "[%d] (%d) state=(%x) ctx->decoded_frame_cnt=%d",
> +			ctx->id, q->type, ctx->state, ctx->decoded_frame_cnt);
> +
> +	if (q->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
> +		while ((src_buf = v4l2_m2m_src_buf_remove(ctx->m2m_ctx)))
> +			v4l2_m2m_buf_done(to_vb2_v4l2_buffer(src_buf),
> +					VB2_BUF_STATE_ERROR);
> +

Same trick can be done here.

> +	} else if (q->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
> +		if (ctx->state >= MTK_STATE_HEADER) {
> +
> +			/* Until STREAMOFF is called on the CAPTURE queue
> +			 * (acknowledging the event), the driver operates
> +			 * as if the resolution hasn't changed yet, i.e.
> +			 * VIDIOC_G_FMT< etc. return previous resolution.
> +			 * So we update picinfo here
> +			 */
> +			ctx->picinfo = ctx->last_decoded_picinfo;
> +
> +			mtk_v4l2_debug(2,
> +				"[%d]-> new(%d,%d), old(%d,%d), real(%d,%d)",
> +				ctx->id, ctx->last_decoded_picinfo.pic_w,
> +				ctx->last_decoded_picinfo.pic_h,
> +				ctx->picinfo.pic_w, ctx->picinfo.pic_h,
> +				ctx->last_decoded_picinfo.buf_w,
> +				ctx->last_decoded_picinfo.buf_h);
> +
> +			mtk_vdec_flush_decoder(ctx);
> +		}
> +		ctx->state = MTK_STATE_FLUSH;
> +
> +		while ((dst_buf = v4l2_m2m_dst_buf_remove(ctx->m2m_ctx))) {
> +			vb2_set_plane_payload(dst_buf, 0, 0);
> +			vb2_set_plane_payload(dst_buf, 1, 0);
> +			v4l2_m2m_buf_done(to_vb2_v4l2_buffer(dst_buf),
> +					VB2_BUF_STATE_ERROR);
> +		}
> +	}
> +}
> +
> +static void m2mops_vdec_device_run(void *priv)
> +{
> +	struct mtk_vcodec_ctx *ctx = priv;
> +	struct mtk_vcodec_dev *dev = ctx->dev;
> +
> +	queue_work(dev->decode_workqueue, &ctx->decode_work);
> +}
> +
> +static int m2mops_vdec_job_ready(void *m2m_priv)
> +{
> +	struct mtk_vcodec_ctx *ctx = m2m_priv;
> +
> +	mtk_v4l2_debug(3, "[%d]", ctx->id);
> +
> +	if (ctx->state == MTK_STATE_ABORT)
> +		return 0;
> +
> +	if ((ctx->last_decoded_picinfo.pic_w != ctx->picinfo.pic_w) ||
> +	    (ctx->last_decoded_picinfo.pic_h != ctx->picinfo.pic_h))
> +		return 0;
> +
> +	if (ctx->state != MTK_STATE_HEADER)
> +		return 0;
> +
> +	return 1;
> +}
> +
> +static void m2mops_vdec_job_abort(void *priv)
> +{
> +	struct mtk_vcodec_ctx *ctx = priv;
> +
> +	ctx->state = MTK_STATE_ABORT;
> +}
> +
> +static int mtk_vdec_g_v_ctrl(struct v4l2_ctrl *ctrl)
> +{
> +	struct mtk_vcodec_ctx *ctx = ctrl_to_ctx(ctrl);
> +	int ret = 0;
> +
> +	switch (ctrl->id) {
> +	case V4L2_CID_MIN_BUFFERS_FOR_CAPTURE:
> +		if (ctx->state >= MTK_STATE_HEADER) {
> +			ctrl->val = ctx->dpb_size;
> +		} else {
> +			mtk_v4l2_err("Seqinfo not ready");
> +			ctrl->val = 1;
> +			ret = -EAGAIN;
> +		}
> +		break;
> +	default:
> +		ret = -EINVAL;
> +	}
> +	return ret;
> +}
> +
> +static const struct v4l2_ctrl_ops mtk_vcodec_dec_ctrl_ops = {
> +	.g_volatile_ctrl = mtk_vdec_g_v_ctrl,
> +};
> +
> +int mtk_vcodec_dec_ctrls_setup(struct mtk_vcodec_ctx *ctx)
> +{
> +	struct v4l2_ctrl *ctrl;
> +
> +	v4l2_ctrl_handler_init(&ctx->ctrl_hdl, 1);
> +
> +	ctrl = v4l2_ctrl_new_std(&ctx->ctrl_hdl,
> +				&mtk_vcodec_dec_ctrl_ops,
> +				V4L2_CID_MIN_BUFFERS_FOR_CAPTURE,
> +				1, 32, 1, 1);
> +	ctrl->flags |= V4L2_CTRL_FLAG_VOLATILE;
> +
> +	if (ctx->ctrl_hdl.error) {
> +		mtk_v4l2_err("Adding control failed %d",
> +				ctx->ctrl_hdl.error);
> +		return ctx->ctrl_hdl.error;
> +	}
> +
> +	v4l2_ctrl_handler_setup(&ctx->ctrl_hdl);
> +	return 0;
> +}
> +
> +static void m2mops_vdec_lock(void *m2m_priv)
> +{
> +	struct mtk_vcodec_ctx *ctx = m2m_priv;
> +
> +	mtk_v4l2_debug(3, "[%d]", ctx->id);
> +	mutex_lock(&ctx->dev->dev_mutex);
> +}
> +
> +static void m2mops_vdec_unlock(void *m2m_priv)
> +{
> +	struct mtk_vcodec_ctx *ctx = m2m_priv;
> +
> +	mtk_v4l2_debug(3, "[%d]", ctx->id);
> +	mutex_unlock(&ctx->dev->dev_mutex);
> +}
> +
> +const struct v4l2_m2m_ops mtk_vdec_m2m_ops = {
> +	.device_run	= m2mops_vdec_device_run,
> +	.job_ready	= m2mops_vdec_job_ready,
> +	.job_abort	= m2mops_vdec_job_abort,
> +	.lock		= m2mops_vdec_lock,
> +	.unlock		= m2mops_vdec_unlock,
> +};
> +
> +static const struct vb2_ops mtk_vdec_vb2_ops = {
> +	.queue_setup	= vb2ops_vdec_queue_setup,
> +	.buf_prepare	= vb2ops_vdec_buf_prepare,
> +	.buf_queue	= vb2ops_vdec_buf_queue,
> +	.wait_prepare	= vb2_ops_wait_prepare,
> +	.wait_finish	= vb2_ops_wait_finish,
> +	.buf_init	= vb2ops_vdec_buf_init,
> +	.buf_finish	= vb2ops_vdec_buf_finish,
> +	.start_streaming	= vb2ops_vdec_start_streaming,
> +	.stop_streaming	= vb2ops_vdec_stop_streaming,
> +};
> +
> +const struct v4l2_ioctl_ops mtk_vdec_ioctl_ops = {
> +	.vidioc_streamon	= v4l2_m2m_ioctl_streamon,
> +	.vidioc_streamoff	= v4l2_m2m_ioctl_streamoff,
> +	.vidioc_reqbufs		= v4l2_m2m_ioctl_reqbufs,
> +	.vidioc_querybuf	= v4l2_m2m_ioctl_querybuf,
> +	.vidioc_expbuf		= v4l2_m2m_ioctl_expbuf,
> +
> +	.vidioc_qbuf		= vidioc_vdec_qbuf,
> +	.vidioc_dqbuf		= vidioc_vdec_dqbuf,
> +
> +	.vidioc_try_fmt_vid_cap_mplane	= vidioc_try_fmt_vid_cap_mplane,
> +	.vidioc_try_fmt_vid_out_mplane	= vidioc_try_fmt_vid_out_mplane,
> +
> +	.vidioc_s_fmt_vid_cap_mplane	= vidioc_vdec_s_fmt,
> +	.vidioc_s_fmt_vid_out_mplane	= vidioc_vdec_s_fmt,
> +	.vidioc_g_fmt_vid_cap_mplane	= vidioc_vdec_g_fmt,
> +	.vidioc_g_fmt_vid_out_mplane	= vidioc_vdec_g_fmt,
> +
> +	.vidioc_enum_fmt_vid_cap_mplane	= vidioc_vdec_enum_fmt_vid_cap_mplane,
> +	.vidioc_enum_fmt_vid_out_mplane	= vidioc_vdec_enum_fmt_vid_out_mplane,
> +	.vidioc_enum_framesizes	= vidioc_enum_framesizes,
> +
> +	.vidioc_querycap		= vidioc_vdec_querycap,
> +	.vidioc_subscribe_event		= vidioc_vdec_subscribe_evt,
> +	.vidioc_unsubscribe_event	= v4l2_event_unsubscribe,
> +	.vidioc_g_selection             = vidioc_vdec_g_selection,
> +};
> +
> +int mtk_vcodec_dec_queue_init(void *priv, struct vb2_queue *src_vq,
> +			   struct vb2_queue *dst_vq)
> +{
> +	struct mtk_vcodec_ctx *ctx = priv;
> +	int ret = 0;
> +
> +	mtk_v4l2_debug(3, "[%d]", ctx->id);
> +
> +	src_vq->type		= V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE;
> +	src_vq->io_modes	= VB2_DMABUF | VB2_MMAP;
> +	src_vq->drv_priv	= ctx;
> +	src_vq->buf_struct_size = sizeof(struct mtk_video_dec_buf);
> +	src_vq->ops		= &mtk_vdec_vb2_ops;
> +	src_vq->mem_ops		= &vb2_dma_contig_memops;
> +	src_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
> +	src_vq->lock		= &ctx->dev->dev_mutex;
> +
> +	ret = vb2_queue_init(src_vq);
> +	if (ret) {
> +		mtk_v4l2_err("Failed to initialize videobuf2 queue(output)");
> +		return ret;
> +	}
> +	dst_vq->type		= V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
> +	dst_vq->io_modes	= VB2_DMABUF | VB2_MMAP;
> +	dst_vq->drv_priv	= ctx;
> +	dst_vq->buf_struct_size = sizeof(struct mtk_video_dec_buf);
> +	dst_vq->ops		= &mtk_vdec_vb2_ops;
> +	dst_vq->mem_ops		= &vb2_dma_contig_memops;
> +	dst_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
> +	dst_vq->lock		= &ctx->dev->dev_mutex;
> +
> +	ret = vb2_queue_init(dst_vq);
> +	if (ret) {
> +		vb2_queue_release(src_vq);
> +		mtk_v4l2_err("Failed to initialize videobuf2 queue(capture)");
> +	}
> +
> +	return ret;
> +}
> +
> diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.h
> new file mode 100644
> index 0000000..595c244
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.h
> @@ -0,0 +1,88 @@
> +/*
> +* Copyright (c) 2016 MediaTek Inc.
> +* Author: PC Chen <pc.chen@mediatek.com>
> +*         Tiffany Lin <tiffany.lin@mediatek.com>
> +*
> +* This program is free software; you can redistribute it and/or modify
> +* it under the terms of the GNU General Public License version 2 as
> +* published by the Free Software Foundation.
> +*
> +* This program is distributed in the hope that it will be useful,
> +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +* GNU General Public License for more details.
> +*/
> +
> +#ifndef _MTK_VCODEC_DEC_H_
> +#define _MTK_VCODEC_DEC_H_
> +
> +#include <media/videobuf2-core.h>
> +#include <media/videobuf2-v4l2.h>
> +
> +#define VCODEC_CAPABILITY_4K_DISABLED	0x10
> +#define VCODEC_DEC_4K_CODED_WIDTH	4096U
> +#define VCODEC_DEC_4K_CODED_HEIGHT	2304U
> +#define MTK_VDEC_MAX_W	2048U
> +#define MTK_VDEC_MAX_H	1088U
> +
> +#define MTK_VDEC_IRQ_STATUS_DEC_SUCCESS        0x10000
> +
> +/**
> + * struct vdec_fb  - decoder frame buffer
> + * @base_y	: Y plane memory info
> + * @base_c	: C plane memory info
> + * @status      : frame buffer status (vdec_fb_status)
> + */
> +struct vdec_fb {
> +	struct mtk_vcodec_mem	base_y;
> +	struct mtk_vcodec_mem	base_c;
> +	unsigned int	status;
> +};
> +
> +/**
> + * struct mtk_video_dec_buf - Private data related to each VB2 buffer.
> + * @b:		VB2 buffer
> + * @list:	link list
> + * @used:	Capture buffer contain decoded frame data and keep in
> + *			codec data structure
> + * @ready_to_display:	Capture buffer not display yet
> + * @queued_in_vb2:	Capture buffer is queue in vb2
> + * @queued_in_v4l2:	Capture buffer is in v4l2 driver, but not in vb2
> + *			queue yet
> + * @lastframe:		Intput buffer is last buffer - EOS
> + * @frame_buffer:	Decode status, and buffer information of Capture buffer
> + *
> + * Note : These status information help us track and debug buffer state
> + */
> +struct mtk_video_dec_buf {
> +	struct vb2_v4l2_buffer	vb;
> +	struct list_head	list;
> +
> +	bool	used;
> +	bool	ready_to_display;
> +	bool	queued_in_vb2;
> +	bool	queued_in_v4l2;
> +	bool	lastframe;
> +	struct vdec_fb	frame_buffer;
> +};
> +
> +extern const struct v4l2_ioctl_ops mtk_vdec_ioctl_ops;
> +extern const struct v4l2_m2m_ops mtk_vdec_m2m_ops;
> +
> +
> +/*
> + * mtk_vdec_lock/mtk_vdec_unlock are for ctx instance to
> + * get/release lock before/after access decoder hw.
> + * mtk_vdec_lock get decoder hw lock and set curr_ctx
> + * to ctx instance that get lock
> + */
> +void mtk_vdec_unlock(struct mtk_vcodec_ctx *ctx);
> +void mtk_vdec_lock(struct mtk_vcodec_ctx *ctx);
> +int mtk_vcodec_dec_queue_init(void *priv, struct vb2_queue *src_vq,
> +			   struct vb2_queue *dst_vq);
> +void mtk_vcodec_dec_set_default_params(struct mtk_vcodec_ctx *ctx);
> +void mtk_vcodec_vdec_release(struct mtk_vcodec_ctx *ctx);
> +int mtk_vcodec_dec_ctrls_setup(struct mtk_vcodec_ctx *ctx);
> +
> +
> +#endif /* _MTK_VCODEC_DEC_H_ */
> diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c
> new file mode 100644
> index 0000000..725d1e5
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c
> @@ -0,0 +1,408 @@
> +/*
> +* Copyright (c) 2016 MediaTek Inc.
> +* Author: PC Chen <pc.chen@mediatek.com>
> +*         Tiffany Lin <tiffany.lin@mediatek.com>
> +*
> +* This program is free software; you can redistribute it and/or modify
> +* it under the terms of the GNU General Public License version 2 as
> +* published by the Free Software Foundation.
> +*
> +* This program is distributed in the hope that it will be useful,
> +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +* GNU General Public License for more details.
> +*/
> +
> +#include <linux/slab.h>
> +#include <linux/interrupt.h>
> +#include <linux/irq.h>
> +#include <linux/module.h>
> +#include <linux/of_device.h>
> +#include <linux/of.h>
> +#include <media/v4l2-event.h>
> +#include <media/v4l2-mem2mem.h>
> +#include <media/videobuf2-dma-contig.h>
> +
> +#include "mtk_vcodec_drv.h"
> +#include "mtk_vcodec_dec.h"
> +#include "mtk_vcodec_dec_pm.h"
> +#include "mtk_vcodec_intr.h"
> +#include "mtk_vcodec_util.h"
> +#include "mtk_vpu.h"
> +
> +#define VDEC_HW_ACTIVE	0x10
> +#define VDEC_IRQ_CFG	0x11
> +#define VDEC_IRQ_CLR	0x10
> +#define VDEC_IRQ_CFG_REG	0xa4
> +
> +module_param(mtk_v4l2_dbg_level, int, S_IRUGO | S_IWUSR);
> +module_param(mtk_vcodec_dbg, bool, S_IRUGO | S_IWUSR);
> +
> +/* Wake up context wait_queue */
> +static void wake_up_ctx(struct mtk_vcodec_ctx *ctx)
> +{
> +	ctx->int_cond = 1;
> +	wake_up_interruptible(&ctx->queue);
> +}
> +
> +static irqreturn_t mtk_vcodec_dec_irq_handler(int irq, void *priv)
> +{
> +	struct mtk_vcodec_dev *dev = priv;
> +	struct mtk_vcodec_ctx *ctx;
> +	u32 cg_status = 0;
> +	unsigned int dec_done_status = 0;
> +	void __iomem *vdec_misc_addr = dev->reg_base[VDEC_MISC] +
> +					VDEC_IRQ_CFG_REG;
> +
> +	ctx = mtk_vcodec_get_curr_ctx(dev);
> +
> +	/* check if HW active or not */
> +	cg_status = readl(dev->reg_base[0]);
> +	if ((cg_status & VDEC_HW_ACTIVE) != 0) {
> +		mtk_v4l2_err("DEC ISR, VDEC active is not 0x0 (0x%08x)",
> +			     cg_status);
> +		return IRQ_HANDLED;
> +	}
> +
> +	dec_done_status = readl(vdec_misc_addr);
> +	ctx->irq_status = dec_done_status;
> +	if ((dec_done_status & MTK_VDEC_IRQ_STATUS_DEC_SUCCESS) !=
> +		MTK_VDEC_IRQ_STATUS_DEC_SUCCESS)
> +		return IRQ_HANDLED;
> +
> +	/* clear interrupt */
> +	writel((readl(vdec_misc_addr) | VDEC_IRQ_CFG),
> +		dev->reg_base[VDEC_MISC] + VDEC_IRQ_CFG_REG);
> +	writel((readl(vdec_misc_addr) & ~VDEC_IRQ_CLR),
> +		dev->reg_base[VDEC_MISC] + VDEC_IRQ_CFG_REG);
> +
> +	wake_up_ctx(ctx);
> +
> +	mtk_v4l2_debug(3,
> +			"mtk_vcodec_dec_irq_handler :wake up ctx %d, dec_done_status=%x",
> +			ctx->id, dec_done_status);
> +
> +	return IRQ_HANDLED;
> +}
> +
> +static void mtk_vcodec_dec_reset_handler(void *priv)
> +{
> +	struct mtk_vcodec_dev *dev = priv;
> +	struct mtk_vcodec_ctx *ctx;
> +
> +	mtk_v4l2_err("Watchdog timeout!!");
> +
> +	mutex_lock(&dev->dev_mutex);
> +	list_for_each_entry(ctx, &dev->ctx_list, list) {
> +		ctx->state = MTK_STATE_ABORT;
> +		mtk_v4l2_debug(0, "[%d] Change to state MTK_STATE_ERROR",
> +				ctx->id);
> +	}
> +	mutex_unlock(&dev->dev_mutex);
> +}
> +
> +static int fops_vcodec_open(struct file *file)
> +{
> +	struct mtk_vcodec_dev *dev = video_drvdata(file);
> +	struct mtk_vcodec_ctx *ctx = NULL;
> +	int ret = 0;
> +
> +	ctx = kzalloc(sizeof(*ctx), GFP_KERNEL);
> +	if (!ctx)
> +		return -ENOMEM;
> +
> +	mutex_lock(&dev->dev_mutex);
> +	ctx->id = dev->id_counter++;
> +	v4l2_fh_init(&ctx->fh, video_devdata(file));
> +	file->private_data = &ctx->fh;
> +	v4l2_fh_add(&ctx->fh);
> +	INIT_LIST_HEAD(&ctx->list);
> +	ctx->dev = dev;
> +	init_waitqueue_head(&ctx->queue);
> +	mutex_init(&ctx->lock);
> +
> +	ctx->type = MTK_INST_DECODER;
> +	ret = mtk_vcodec_dec_ctrls_setup(ctx);
> +	if (ret) {
> +		mtk_v4l2_err("Failed to setup mt vcodec controls");
> +		goto err_ctrls_setup;
> +	}
> +	ctx->m2m_ctx = v4l2_m2m_ctx_init(dev->m2m_dev_dec, ctx,
> +		&mtk_vcodec_dec_queue_init);
> +	if (IS_ERR((__force void *)ctx->m2m_ctx)) {
> +		ret = PTR_ERR((__force void *)ctx->m2m_ctx);
> +		mtk_v4l2_err("Failed to v4l2_m2m_ctx_init() (%d)",
> +			ret);
> +		goto err_m2m_ctx_init;
> +	}
> +	mtk_vcodec_dec_set_default_params(ctx);
> +
> +	if (v4l2_fh_is_singular(&ctx->fh)) {
> +		mtk_vcodec_dec_pw_on(&dev->pm);
> +		/*
> +		 * vpu_load_firmware checks if it was loaded already and
> +		 * does nothing in that case
> +		 */
> +		ret = vpu_load_firmware(dev->vpu_plat_dev);
> +		if (ret < 0) {
> +			/*
> +			 * Return 0 if downloading firmware successfully,
> +			 * otherwise it is failed
> +			 */
> +			mtk_v4l2_err("vpu_load_firmware failed!");
> +			goto err_load_fw;
> +		}
> +
> +		dev->dec_capability =
> +			vpu_get_vdec_hw_capa(dev->vpu_plat_dev);
> +		mtk_v4l2_debug(0, "decoder capability %x", dev->dec_capability);
> +	}
> +
> +	dev->num_instances++;

This field can be dropped since it is no longer used.

> +	list_add(&ctx->list, &dev->ctx_list);
> +
> +	mutex_unlock(&dev->dev_mutex);
> +	mtk_v4l2_debug(0, "%s decoder [%d]", dev_name(&dev->plat_dev->dev),
> +			ctx->id);
> +	return ret;
> +
> +	/* Deinit when failure occurred */
> +err_load_fw:
> +	v4l2_m2m_ctx_release(ctx->m2m_ctx);
> +err_m2m_ctx_init:
> +	v4l2_ctrl_handler_free(&ctx->ctrl_hdl);
> +err_ctrls_setup:
> +	v4l2_fh_del(&ctx->fh);
> +	v4l2_fh_exit(&ctx->fh);
> +	kfree(ctx);
> +	mutex_unlock(&dev->dev_mutex);
> +
> +	return ret;
> +}
> +
> +static int fops_vcodec_release(struct file *file)
> +{
> +	struct mtk_vcodec_dev *dev = video_drvdata(file);
> +	struct mtk_vcodec_ctx *ctx = fh_to_ctx(file->private_data);
> +
> +	mtk_v4l2_debug(0, "[%d] decoder", ctx->id);
> +	mutex_lock(&dev->dev_mutex);
> +
> +	mtk_vcodec_vdec_release(ctx);
> +	if (v4l2_fh_is_singular(&ctx->fh))
> +		mtk_vcodec_dec_pw_off(&dev->pm);
> +
> +	v4l2_fh_del(&ctx->fh);
> +	v4l2_fh_exit(&ctx->fh);
> +	v4l2_ctrl_handler_free(&ctx->ctrl_hdl);
> +	v4l2_m2m_ctx_release(ctx->m2m_ctx);
> +
> +	list_del_init(&ctx->list);
> +	dev->num_instances--;
> +	kfree(ctx);
> +	mutex_unlock(&dev->dev_mutex);
> +	return 0;
> +}
> +
> +static const struct v4l2_file_operations mtk_vcodec_fops = {
> +	.owner		= THIS_MODULE,
> +	.open		= fops_vcodec_open,
> +	.release	= fops_vcodec_release,
> +	.poll		= v4l2_m2m_fop_poll,
> +	.unlocked_ioctl	= video_ioctl2,
> +	.mmap		= v4l2_m2m_fop_mmap,
> +};
> +
> +static int mtk_vcodec_probe(struct platform_device *pdev)
> +{
> +	struct mtk_vcodec_dev *dev;
> +	struct video_device *vfd_dec;
> +	struct resource *res;
> +	int i, ret;
> +	DEFINE_DMA_ATTRS(attrs);
> +
> +	dev = devm_kzalloc(&pdev->dev, sizeof(*dev), GFP_KERNEL);
> +	if (!dev)
> +		return -ENOMEM;
> +
> +	INIT_LIST_HEAD(&dev->ctx_list);
> +	dev->plat_dev = pdev;
> +
> +	dev->vpu_plat_dev = vpu_get_plat_device(dev->plat_dev);
> +	if (dev->vpu_plat_dev == NULL) {
> +		mtk_v4l2_err("[VPU] vpu device in not ready");
> +		return -EPROBE_DEFER;
> +	}
> +
> +	vpu_wdt_reg_handler(dev->vpu_plat_dev, mtk_vcodec_dec_reset_handler,
> +			dev, VPU_RST_DEC);
> +
> +	ret = mtk_vcodec_init_dec_pm(dev);
> +	if (ret < 0) {
> +		dev_err(&pdev->dev, "Failed to get mt vcodec clock source");
> +		return ret;
> +	}
> +
> +	for (i = 0; i < NUM_MAX_VDEC_REG_BASE; i++) {
> +		res = platform_get_resource(pdev, IORESOURCE_MEM, i);
> +		if (res == NULL) {
> +			dev_err(&pdev->dev, "get memory resource failed.");
> +			ret = -ENXIO;
> +			goto err_res;
> +		}
> +		dev->reg_base[i] = devm_ioremap_resource(&pdev->dev, res);
> +		if (IS_ERR((__force void *)dev->reg_base[i])) {
> +			dev_err(&pdev->dev,
> +				"devm_ioremap_resource %d failed.", i);
> +			ret = PTR_ERR((__force void *)dev->reg_base);
> +			goto err_res;
> +		}
> +		mtk_v4l2_debug(2, "reg[%d] base=%p", i, dev->reg_base[i]);
> +	}
> +
> +	res = platform_get_resource(pdev, IORESOURCE_IRQ, 0);
> +	if (res == NULL) {
> +		dev_err(&pdev->dev, "failed to get irq resource");
> +		ret = -ENOENT;
> +		goto err_res;
> +	}
> +
> +	dev->dec_irq = platform_get_irq(pdev, 0);
> +	ret = devm_request_irq(&pdev->dev, dev->dec_irq,
> +			mtk_vcodec_dec_irq_handler, 0, pdev->name, dev);
> +	if (ret) {
> +		dev_err(&pdev->dev, "Failed to install dev->dec_irq %d (%d)",
> +			dev->dec_irq,
> +			ret);
> +		goto err_res;
> +	}
> +
> +	disable_irq(dev->dec_irq);
> +	mutex_init(&dev->dec_mutex);
> +	mutex_init(&dev->dev_mutex);
> +	spin_lock_init(&dev->irqlock);
> +
> +	snprintf(dev->v4l2_dev.name, sizeof(dev->v4l2_dev.name), "%s",
> +		"[/MTK_V4L2_VDEC]");
> +
> +	ret = v4l2_device_register(&pdev->dev, &dev->v4l2_dev);
> +	if (ret) {
> +		mtk_v4l2_err("v4l2_device_register err=%d", ret);
> +		goto err_res;
> +	}
> +
> +	init_waitqueue_head(&dev->queue);
> +
> +	vfd_dec = video_device_alloc();
> +	if (!vfd_dec) {
> +		mtk_v4l2_err("Failed to allocate video device");
> +		ret = -ENOMEM;
> +		goto err_dec_alloc;
> +	}
> +	vfd_dec->fops		= &mtk_vcodec_fops;
> +	vfd_dec->ioctl_ops	= &mtk_vdec_ioctl_ops;
> +	vfd_dec->release	= video_device_release;
> +	vfd_dec->lock		= &dev->dev_mutex;
> +	vfd_dec->v4l2_dev	= &dev->v4l2_dev;
> +	vfd_dec->vfl_dir	= VFL_DIR_M2M;
> +	vfd_dec->device_caps	= V4L2_CAP_VIDEO_M2M_MPLANE |
> +			V4L2_CAP_STREAMING;
> +
> +	snprintf(vfd_dec->name, sizeof(vfd_dec->name), "%s",
> +		MTK_VCODEC_DEC_NAME);
> +	video_set_drvdata(vfd_dec, dev);
> +	dev->vfd_dec = vfd_dec;
> +	platform_set_drvdata(pdev, dev);
> +
> +	dev->alloc_ctx = vb2_dma_contig_init_ctx(&pdev->dev);
> +	if (IS_ERR((__force void *)dev->alloc_ctx)) {
> +		mtk_v4l2_err("Failed to alloc vb2 dma context 0");
> +		ret = PTR_ERR((__force void *)dev->alloc_ctx);
> +		goto err_vb2_ctx_init;
> +	}
> +
> +	dev->m2m_dev_dec = v4l2_m2m_init(&mtk_vdec_m2m_ops);
> +	if (IS_ERR((__force void *)dev->m2m_dev_dec)) {
> +		mtk_v4l2_err("Failed to init mem2mem dec device");
> +		ret = PTR_ERR((__force void *)dev->m2m_dev_dec);
> +		goto err_dec_mem_init;
> +	}
> +
> +	dev->decode_workqueue =
> +		alloc_ordered_workqueue(MTK_VCODEC_DEC_NAME,
> +			WQ_MEM_RECLAIM | WQ_FREEZABLE);
> +	if (!dev->decode_workqueue) {
> +		mtk_v4l2_err("Failed to create decode workqueue");
> +		ret = -EINVAL;
> +		goto err_event_workq;
> +	}
> +
> +	ret = video_register_device(vfd_dec, VFL_TYPE_GRABBER, 0);
> +	if (ret) {
> +		mtk_v4l2_err("Failed to register video device");
> +		goto err_dec_reg;
> +	}
> +
> +	/* Avoid the iommu eat big hunks */
> +	dma_set_attr(DMA_ATTR_ALLOC_SINGLE_PAGES, &attrs);
> +
> +	mtk_v4l2_debug(0, "decoder registered as /dev/video%d",
> +		vfd_dec->num);
> +
> +	return 0;
> +
> +err_dec_reg:
> +	destroy_workqueue(dev->decode_workqueue);
> +err_event_workq:
> +	v4l2_m2m_release(dev->m2m_dev_dec);
> +err_dec_mem_init:
> +	vb2_dma_contig_cleanup_ctx(dev->alloc_ctx);
> +err_vb2_ctx_init:
> +	video_unregister_device(vfd_dec);
> +err_dec_alloc:
> +	v4l2_device_unregister(&dev->v4l2_dev);
> +err_res:
> +	mtk_vcodec_release_dec_pm(dev);
> +	return ret;
> +}
> +
> +static const struct of_device_id mtk_vcodec_match[] = {
> +	{.compatible = "mediatek,mt8173-vcodec-dec",},
> +	{},
> +};
> +
> +MODULE_DEVICE_TABLE(of, mtk_vcodec_match);
> +
> +static int mtk_vcodec_dec_remove(struct platform_device *pdev)
> +{
> +	struct mtk_vcodec_dev *dev = platform_get_drvdata(pdev);
> +
> +	flush_workqueue(dev->decode_workqueue);
> +	destroy_workqueue(dev->decode_workqueue);
> +	if (dev->m2m_dev_dec)
> +		v4l2_m2m_release(dev->m2m_dev_dec);
> +	if (dev->alloc_ctx)
> +		vb2_dma_contig_cleanup_ctx(dev->alloc_ctx);
> +
> +	if (dev->vfd_dec)
> +		video_unregister_device(dev->vfd_dec);
> +
> +	v4l2_device_unregister(&dev->v4l2_dev);
> +	mtk_vcodec_release_dec_pm(dev);
> +	return 0;
> +}
> +
> +static struct platform_driver mtk_vcodec_dec_driver = {
> +	.probe	= mtk_vcodec_probe,
> +	.remove	= mtk_vcodec_dec_remove,
> +	.driver	= {
> +		.name	= MTK_VCODEC_DEC_NAME,
> +		.owner	= THIS_MODULE,
> +		.of_match_table = mtk_vcodec_match,
> +	},
> +};
> +
> +module_platform_driver(mtk_vcodec_dec_driver);
> +
> +MODULE_LICENSE("GPL v2");
> +MODULE_DESCRIPTION("Mediatek video codec V4L2 decoder driver");
> diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_pm.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_pm.c
> new file mode 100644
> index 0000000..364b756
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_pm.c
> @@ -0,0 +1,206 @@
> +/*
> +* Copyright (c) 2016 MediaTek Inc.
> +* Author: Tiffany Lin <tiffany.lin@mediatek.com>
> +*
> +* This program is free software; you can redistribute it and/or modify
> +* it under the terms of the GNU General Public License version 2 as
> +* published by the Free Software Foundation.
> +*
> +* This program is distributed in the hope that it will be useful,
> +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +* GNU General Public License for more details.
> +*/
> +
> +#include <linux/clk.h>
> +#include <linux/of_address.h>
> +#include <linux/of_platform.h>
> +#include <linux/pm_runtime.h>
> +#include <soc/mediatek/smi.h>
> +
> +#include "mtk_vcodec_dec_pm.h"
> +#include "mtk_vcodec_util.h"
> +#include "mtk_vpu.h"
> +
> +int mtk_vcodec_init_dec_pm(struct mtk_vcodec_dev *mtkdev)
> +{
> +	struct device_node *node;
> +	struct platform_device *pdev;
> +	struct device *dev;
> +	struct mtk_vcodec_pm *pm;
> +	int ret = 0;
> +
> +	pdev = mtkdev->plat_dev;
> +	pm = &mtkdev->pm;
> +	pm->mtkdev = mtkdev;
> +	dev = &pdev->dev;
> +	node = of_parse_phandle(pdev->dev.of_node, "mediatek,larb", 0);
> +	if (!node) {
> +		mtk_v4l2_err("of_parse_phandle mediatek,larb fail!");
> +		return -1;
> +	}
> +
> +	pdev = of_find_device_by_node(node);
> +	if (WARN_ON(!pdev)) {
> +		of_node_put(node);
> +		return -1;
> +	}
> +	pm->larbvdec = &pdev->dev;
> +
> +	pdev = mtkdev->plat_dev;
> +	pm->dev = &pdev->dev;
> +
> +	pm->vcodecpll = devm_clk_get(&pdev->dev, "vcodecpll");
> +	if (pm->vcodecpll == NULL) {
> +		mtk_v4l2_err("devm_clk_get vcodecpll fail");
> +		ret = -1;
> +	}
> +
> +	pm->univpll_d2 = devm_clk_get(&pdev->dev, "univpll_d2");
> +	if (pm->univpll_d2 == NULL) {
> +		mtk_v4l2_err("devm_clk_get univpll_d2 fail");
> +		ret = -1;
> +	}
> +
> +	pm->clk_cci400_sel = devm_clk_get(&pdev->dev, "clk_cci400_sel");
> +	if (pm->clk_cci400_sel == NULL) {
> +		mtk_v4l2_err("devm_clk_get clk_cci400_sel fail");
> +		ret = -1;
> +	}
> +
> +	pm->vdec_sel = devm_clk_get(&pdev->dev, "vdec_sel");
> +	if (pm->vdec_sel == NULL) {
> +		mtk_v4l2_err("devm_clk_get vdec_sel fail");
> +		ret = -1;
> +	}
> +
> +	pm->vdecpll = devm_clk_get(&pdev->dev, "vdecpll");
> +	if (pm->vdecpll == NULL) {
> +		mtk_v4l2_err("devm_clk_get vdecpll fail");
> +		ret = -1;
> +	}
> +
> +	pm->vencpll = devm_clk_get(&pdev->dev, "vencpll");
> +	if (pm->vencpll == NULL) {
> +		mtk_v4l2_err("devm_clk_get vencpll fail");
> +		ret = -1;
> +	}
> +
> +	pm->venc_lt_sel = devm_clk_get(&pdev->dev, "venc_lt_sel");
> +	if (pm->venc_lt_sel == NULL) {
> +		mtk_v4l2_err("devm_clk_get venc_lt_sel fail");
> +		ret = -1;
> +	}
> +
> +	pm->vdec_bus_clk_src = devm_clk_get(&pdev->dev, "vdec_bus_clk_src");
> +	if (pm->vdec_bus_clk_src == NULL) {
> +		mtk_v4l2_err("devm_clk_get vdec_bus_clk_src");
> +		ret = -1;
> +	}
> +
> +	pm_runtime_enable(&pdev->dev);
> +
> +	return ret;
> +}
> +
> +void mtk_vcodec_release_dec_pm(struct mtk_vcodec_dev *dev)
> +{
> +	pm_runtime_disable(dev->pm.dev);
> +}
> +
> +void mtk_vcodec_dec_pw_on(struct mtk_vcodec_pm *pm)
> +{
> +	int ret;
> +
> +	ret = pm_runtime_get_sync(pm->dev);
> +	if (ret)
> +		mtk_v4l2_err("pm_runtime_get_sync fail %d", ret);
> +}
> +
> +void mtk_vcodec_dec_pw_off(struct mtk_vcodec_pm *pm)
> +{
> +	int ret;
> +
> +	ret = pm_runtime_put_sync(pm->dev);
> +	if (ret)
> +		mtk_v4l2_err("pm_runtime_put_sync fail %d", ret);
> +}
> +
> +void mtk_vcodec_dec_clock_on(struct mtk_vcodec_pm *pm)
> +{
> +	int ret;
> +
> +	ret = clk_set_rate(pm->vcodecpll, 1482 * 1000000);
> +	if (ret)
> +		mtk_v4l2_err("clk_set_rate vcodecpll fail %d", ret);
> +
> +	ret = clk_set_rate(pm->vencpll, 800 * 1000000);
> +	if (ret)
> +		mtk_v4l2_err("clk_set_rate vencpll fail %d", ret);
> +
> +	ret = clk_prepare_enable(pm->vcodecpll);
> +	if (ret)
> +		mtk_v4l2_err("clk_prepare_enable vcodecpll fail %d", ret);
> +
> +	ret = clk_prepare_enable(pm->vencpll);
> +	if (ret)
> +		mtk_v4l2_err("clk_prepare_enable vencpll fail %d", ret);
> +
> +	ret = clk_prepare_enable(pm->vdec_bus_clk_src);
> +	if (ret)
> +		mtk_v4l2_err("clk_prepare_enable vdec_bus_clk_src fail %d",
> +				ret);
> +
> +	ret = clk_prepare_enable(pm->venc_lt_sel);
> +	if (ret)
> +		mtk_v4l2_err("clk_prepare_enable venc_lt_sel fail %d", ret);
> +
> +	ret = clk_set_parent(pm->venc_lt_sel, pm->vdec_bus_clk_src);
> +	if (ret)
> +		mtk_v4l2_err("clk_set_parent venc_lt_sel vdec_bus_clk_src fail %d",
> +				ret);
> +
> +	ret = clk_prepare_enable(pm->univpll_d2);
> +	if (ret)
> +		mtk_v4l2_err("clk_prepare_enable univpll_d2 fail %d", ret);
> +
> +	ret = clk_prepare_enable(pm->clk_cci400_sel);
> +	if (ret)
> +		mtk_v4l2_err("clk_prepare_enable clk_cci400_sel fail %d", ret);
> +
> +	ret = clk_set_parent(pm->clk_cci400_sel, pm->univpll_d2);
> +	if (ret)
> +		mtk_v4l2_err("clk_set_parent clk_cci400_sel univpll_d2 fail %d",
> +				ret);
> +
> +	ret = clk_prepare_enable(pm->vdecpll);
> +	if (ret)
> +		mtk_v4l2_err("clk_prepare_enable vdecpll fail %d", ret);
> +
> +	ret = clk_prepare_enable(pm->vdec_sel);
> +	if (ret)
> +		mtk_v4l2_err("clk_prepare_enable vdec_sel fail %d", ret);
> +
> +	ret = clk_set_parent(pm->vdec_sel, pm->vdecpll);
> +	if (ret)
> +		mtk_v4l2_err("clk_set_parent vdec_sel vdecpll fail %d", ret);
> +
> +	ret = mtk_smi_larb_get(pm->larbvdec);
> +	if (ret)
> +		mtk_v4l2_err("mtk_smi_larb_get larbvdec fail %d", ret);
> +
> +}
> +
> +void mtk_vcodec_dec_clock_off(struct mtk_vcodec_pm *pm)
> +{
> +	mtk_smi_larb_put(pm->larbvdec);
> +	clk_disable_unprepare(pm->vdec_sel);
> +	clk_disable_unprepare(pm->vdecpll);
> +	clk_disable_unprepare(pm->univpll_d2);
> +	clk_disable_unprepare(pm->clk_cci400_sel);
> +	clk_disable_unprepare(pm->venc_lt_sel);
> +	clk_disable_unprepare(pm->vdec_bus_clk_src);
> +	clk_disable_unprepare(pm->vencpll);
> +	clk_disable_unprepare(pm->vcodecpll);
> +}
> +
> diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_pm.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_pm.h
> new file mode 100644
> index 0000000..b8bf56f
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_pm.h
> @@ -0,0 +1,28 @@
> +/*
> +* Copyright (c) 2016 MediaTek Inc.
> +* Author: Tiffany Lin <tiffany.lin@mediatek.com>
> +*
> +* This program is free software; you can redistribute it and/or modify
> +* it under the terms of the GNU General Public License version 2 as
> +* published by the Free Software Foundation.
> +*
> +* This program is distributed in the hope that it will be useful,
> +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +* GNU General Public License for more details.
> +*/
> +
> +#ifndef _MTK_VCODEC_DEC_PM_H_
> +#define _MTK_VCODEC_DEC_PM_H_
> +
> +#include "mtk_vcodec_drv.h"
> +
> +int mtk_vcodec_init_dec_pm(struct mtk_vcodec_dev *dev);
> +void mtk_vcodec_release_dec_pm(struct mtk_vcodec_dev *dev);
> +
> +void mtk_vcodec_dec_pw_on(struct mtk_vcodec_pm *pm);
> +void mtk_vcodec_dec_pw_off(struct mtk_vcodec_pm *pm);
> +void mtk_vcodec_dec_clock_on(struct mtk_vcodec_pm *pm);
> +void mtk_vcodec_dec_clock_off(struct mtk_vcodec_pm *pm);
> +
> +#endif /* _MTK_VCODEC_DEC_PM_H_ */
> diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
> index 78eee50..a11dfec 100644
> --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
> +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
> @@ -26,10 +26,10 @@
>  #include "mtk_vcodec_util.h"
>  
>  #define MTK_VCODEC_DRV_NAME	"mtk_vcodec_drv"
> +#define MTK_VCODEC_DEC_NAME	"mtk-vcodec-dec"
>  #define MTK_VCODEC_ENC_NAME	"mtk-vcodec-enc"
>  #define MTK_PLATFORM_STR	"platform:mt8173"
>  
> -
>  #define MTK_VCODEC_MAX_PLANES	3
>  #define MTK_V4L2_BENCHMARK	0
>  #define WAIT_INTR_TIMEOUT_MS	1000
> @@ -180,6 +180,9 @@ struct mtk_enc_params {
>   * struct mtk_vcodec_pm - Power management data structure
>   */
>  struct mtk_vcodec_pm {
> +	struct clk	*vdec_bus_clk_src;
> +	struct clk	*vencpll;
> +
>  	struct clk	*vcodecpll;
>  	struct clk	*univpll_d2;
>  	struct clk	*clk_cci400_sel;
> @@ -197,6 +200,32 @@ struct mtk_vcodec_pm {
>  };
>  
>  /**
> + * struct vdec_pic_info  - picture size information
> + * @pic_w: picture width
> + * @pic_h: picture height
> + * @buf_w: picture buffer width (64 aligned up from pic_w)
> + * @buf_h: picture buffer heiht (64 aligned up from pic_h)
> + * @y_bs_sz: Y bitstream size
> + * @c_bs_sz: CbCr bitstream size
> + * @y_len_sz: additional size required to store decompress information for y
> + *		plane
> + * @c_len_sz: additional size required to store decompress information for cbcr
> + *		plane
> + * E.g. suppose picture size is 176x144,
> + *      buffer size will be aligned to 176x160.
> + */
> +struct vdec_pic_info {
> +	unsigned int pic_w;
> +	unsigned int pic_h;
> +	unsigned int buf_w;
> +	unsigned int buf_h;
> +	unsigned int y_bs_sz;
> +	unsigned int c_bs_sz;
> +	unsigned int y_len_sz;
> +	unsigned int c_len_sz;
> +};
> +
> +/**
>   * struct mtk_vcodec_ctx - Context (instance) private data.
>   *
>   * @type: type of the instance - decoder or encoder
> @@ -210,9 +239,12 @@ struct mtk_vcodec_pm {
>   * @state: state of the context
>   * @param_change: indicate encode parameter type
>   * @enc_params: encoding parameters
> - * @enc_if: hoooked encoder driver interface
> + * @dec_if: hooked decoder driver interface
> + * @enc_if: hooked encoder driver interface
>   * @drv_handle: driver handle for specific decode/encode instance
>   *
> + * @picinfo: store picture info after header parsing
> + * @dpb_size: store dpb count after header parsing
>   * @int_cond: variable used by the waitqueue
>   * @int_type: type of the last interrupt
>   * @queue: waitqueue that can be used to wait for this context to
> @@ -220,12 +252,16 @@ struct mtk_vcodec_pm {
>   * @irq_status: irq status
>   *
>   * @ctrl_hdl: handler for v4l2 framework
> + * @decode_work: worker for the decoding
>   * @encode_work: worker for the encoding
> + * @last_decoded_picinfo: pic information get from latest decode
>   *
>   * @colorspace: enum v4l2_colorspace; supplemental to pixelformat
>   * @ycbcr_enc: enum v4l2_ycbcr_encoding, Y'CbCr encoding
>   * @quantization: enum v4l2_quantization, colorspace quantization
>   * @xfer_func: enum v4l2_xfer_func, colorspace transfer function
> + * @lock: protect variables accessed by V4L2 threads and worker thread such as
> + *	  mtk_video_dec_buf.
>   */
>  struct mtk_vcodec_ctx {
>  	enum mtk_instance_type type;
> @@ -240,21 +276,31 @@ struct mtk_vcodec_ctx {
>  	enum mtk_encode_param param_change;
>  	struct mtk_enc_params enc_params;
>  
> +	struct vdec_common_if *dec_if;
>  	struct venc_common_if *enc_if;
>  	unsigned long drv_handle;
>  
> +	struct vdec_pic_info picinfo;
> +	int dpb_size;
> +
>  	int int_cond;
>  	int int_type;
>  	wait_queue_head_t queue;
>  	unsigned int irq_status;
>  
>  	struct v4l2_ctrl_handler ctrl_hdl;
> +	struct work_struct decode_work;
>  	struct work_struct encode_work;
> +	struct vdec_pic_info last_decoded_picinfo;
>  
>  	enum v4l2_colorspace colorspace;
>  	enum v4l2_ycbcr_encoding ycbcr_enc;
>  	enum v4l2_quantization quantization;
>  	enum v4l2_xfer_func xfer_func;
> +
> +	int decoded_frame_cnt;
> +	struct mutex lock;
> +
>  };
>  
>  /**
> @@ -268,25 +314,48 @@ struct mtk_vcodec_ctx {
>   * @alloc_ctx: VB2 allocator context
>   *	       (for allocations without kernel mapping).
>   * @ctx_list: list of struct mtk_vcodec_ctx
> - * @irqlock: protect data access by irq handler and work thread
> - * @curr_ctx: The context that is waiting for codec hardware
> + * @irqlock: protect data access by irq handler and work thread -- curr_ctx
> + * @curr_ctx: The context that is waiting for codec hardware. Protected by
> + *		irqlock and dec_mutex or enc_mutex.
>   *
>   * @reg_base: Mapped address of MTK Vcodec registers.
>   *
> - * @id_counter: used to identify current opened instance
> + * @id_counter:  An integer id given to the next opened instance
>   * @num_instances: counter of active MTK Vcodec instances
>   *
>   * @encode_workqueue: encode work queue
>   *
> + * @v4l2_dev: V4L2 device to register video devices for.
> + * @vfd_dec: Video device for decoder
> + * @vfd_enc: Video device for encoder
> + *
> + * @m2m_dev_dec: m2m device for decoder
> + * @m2m_dev_enc: m2m device for encoder
> + * @plat_dev: platform device
> + * @vpu_plat_dev: mtk vpu platform device
> + * @alloc_ctx:VB2 allocator context
> + *	      (for allocations without kernel mapping).
> + * @ctx_list: list of struct mtk_vcodec_ctx
> + * @irqlock: protect data access by irq handler and work thread
> + * @curr_ctx: The context that is waiting for codec hardware
> + * @reg_base: Mapped address of MTK Vcodec registers.
> + *
> + * @instance_mask: used to mark which index id are used
> + * @num_instances: counter of active MTK Vcodec instances
> + *
> + * @decode_workqueue: decode work queue
> + * @encode_workqueue: encode work queue
>   * @int_cond: used to identify interrupt condition happen
>   * @int_type: used to identify what kind of interrupt condition happen
>   * @dev_mutex: video_device lock
>   * @queue: waitqueue for waiting for completion of device commands
>   *
> + * @dec_irq: decoder irq resource
>   * @enc_irq: h264 encoder irq resource
> - * @enc_lt_irq: vp8 encoder irq resource
> + * @enc_lt_irq: vp8 encoder lt irq resource
>   *
> - * @enc_mutex: encoder hardware lock.
> + * @dec_mutex: decoder hardware lock
> + * @enc_mutex: encoder hardware lock
>   *
>   * @pm: power management control
>   * @dec_capability: used to identify decode capability, ex: 4k
> @@ -294,8 +363,10 @@ struct mtk_vcodec_ctx {
>   */
>  struct mtk_vcodec_dev {
>  	struct v4l2_device v4l2_dev;
> +	struct video_device *vfd_dec;
>  	struct video_device *vfd_enc;
>  
> +	struct v4l2_m2m_dev *m2m_dev_dec;
>  	struct v4l2_m2m_dev *m2m_dev_enc;
>  	struct platform_device *plat_dev;
>  	struct platform_device *vpu_plat_dev;
> @@ -308,6 +379,7 @@ struct mtk_vcodec_dev {
>  	unsigned long id_counter;
>  	int num_instances;
>  
> +	struct workqueue_struct *decode_workqueue;
>  	struct workqueue_struct *encode_workqueue;
>  
>  	int int_cond;
> @@ -315,9 +387,11 @@ struct mtk_vcodec_dev {
>  	struct mutex dev_mutex;
>  	wait_queue_head_t queue;
>  
> +	int dec_irq;
>  	int enc_irq;
>  	int enc_lt_irq;
>  
> +	struct mutex dec_mutex;
>  	struct mutex enc_mutex;
>  
>  	struct mtk_vcodec_pm pm;
> diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.c
> index 52e7e5c..f0bdb19 100644
> --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.c
> +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.c
> @@ -30,8 +30,7 @@ int mtk_vcodec_wait_for_done_ctx(struct mtk_vcodec_ctx  *ctx, int command,
>  	timeout_jiff = msecs_to_jiffies(timeout_ms);
>  
>  	ret = wait_event_interruptible_timeout(*waitqueue,
> -				(ctx->int_cond &&
> -				(ctx->int_type == command)),
> +				ctx->int_cond,
>  				timeout_jiff);
>  
>  	if (!ret) {
> @@ -52,3 +51,4 @@ int mtk_vcodec_wait_for_done_ctx(struct mtk_vcodec_ctx  *ctx, int command,
>  	return status;
>  }
>  EXPORT_SYMBOL(mtk_vcodec_wait_for_done_ctx);
> +
> diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.h
> index 33e890f..fbb8b1e 100644
> --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.h
> +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.h
> @@ -16,7 +16,7 @@
>  #define _MTK_VCODEC_INTR_H_
>  
>  #define MTK_INST_IRQ_RECEIVED		0x1
> -#define MTK_INST_WORK_THREAD_ABORT_DONE	0x2
> +
>  
>  struct mtk_vcodec_ctx;
>  
> diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c
> index 5e36513..7617305 100644
> --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c
> +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c
> @@ -81,14 +81,35 @@ void mtk_vcodec_mem_free(struct mtk_vcodec_ctx *data,
>  		return;
>  	}
>  
> -	dma_free_coherent(dev, size, mem->va, mem->dma_addr);
> -	mem->va = NULL;
> -	mem->dma_addr = 0;
> -	mem->size = 0;
> -
>  	mtk_v4l2_debug(3, "[%d]  - va      = %p", ctx->id, mem->va);
>  	mtk_v4l2_debug(3, "[%d]  - dma     = 0x%lx", ctx->id,
>  		       (unsigned long)mem->dma_addr);
>  	mtk_v4l2_debug(3, "[%d]    size = 0x%lx", ctx->id, size);
> +
> +	dma_free_coherent(dev, size, mem->va, mem->dma_addr);
> +	mem->va = NULL;
> +	mem->dma_addr = 0;
> +	mem->size = 0;
>  }
>  EXPORT_SYMBOL(mtk_vcodec_mem_free);
> +
> +void mtk_vcodec_set_curr_ctx(struct mtk_vcodec_dev *dev,
> +	struct mtk_vcodec_ctx *ctx)
> +{
> +	unsigned long flags;
> +
> +	spin_lock_irqsave(&dev->irqlock, flags);
> +	dev->curr_ctx = ctx;
> +	spin_unlock_irqrestore(&dev->irqlock, flags);
> +}
> +
> +struct mtk_vcodec_ctx *mtk_vcodec_get_curr_ctx(struct mtk_vcodec_dev *dev)
> +{
> +	unsigned long flags;
> +	struct mtk_vcodec_ctx *ctx;
> +
> +	spin_lock_irqsave(&dev->irqlock, flags);
> +	ctx = dev->curr_ctx;
> +	spin_unlock_irqrestore(&dev->irqlock, flags);
> +	return ctx;
> +}
> diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
> index d6345fc..91a0b1a 100644
> --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
> +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
> @@ -1,7 +1,7 @@
>  /*
>  * Copyright (c) 2016 MediaTek Inc.
>  * Author: PC Chen <pc.chen@mediatek.com>
> -*	Tiffany Lin <tiffany.lin@mediatek.com>
> +*		Tiffany Lin <tiffany.lin@mediatek.com>
>  *
>  * This program is free software; you can redistribute it and/or modify
>  * it under the terms of the GNU General Public License version 2 as
> @@ -26,6 +26,7 @@ struct mtk_vcodec_mem {
>  };
>  
>  struct mtk_vcodec_ctx;
> +struct mtk_vcodec_dev;
>  
>  extern int mtk_v4l2_dbg_level;
>  extern bool mtk_vcodec_dbg;
> @@ -84,4 +85,8 @@ int mtk_vcodec_mem_alloc(struct mtk_vcodec_ctx *data,
>  				struct mtk_vcodec_mem *mem);
>  void mtk_vcodec_mem_free(struct mtk_vcodec_ctx *data,
>  				struct mtk_vcodec_mem *mem);
> +void mtk_vcodec_set_curr_ctx(struct mtk_vcodec_dev *dev,
> +	struct mtk_vcodec_ctx *ctx);
> +struct mtk_vcodec_ctx *mtk_vcodec_get_curr_ctx(struct mtk_vcodec_dev *dev);
> +
>  #endif /* _MTK_VCODEC_UTIL_H_ */
> diff --git a/drivers/media/platform/mtk-vcodec/vdec_drv_base.h b/drivers/media/platform/mtk-vcodec/vdec_drv_base.h
> new file mode 100644
> index 0000000..7e4c1a9
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vcodec/vdec_drv_base.h
> @@ -0,0 +1,56 @@
> +/*
> + * Copyright (c) 2016 MediaTek Inc.
> + * Author: PC Chen <pc.chen@mediatek.com>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + */
> +
> +#ifndef _VDEC_DRV_BASE_
> +#define _VDEC_DRV_BASE_
> +
> +#include "mtk_vcodec_drv.h"
> +
> +#include "vdec_drv_if.h"
> +
> +struct vdec_common_if {
> +	/**
> +	 * (*init)() - initialize decode driver
> +	 * @ctx     : [in] mtk v4l2 context
> +	 * @h_vdec  : [out] driver handle
> +	 */
> +	int (*init)(struct mtk_vcodec_ctx *ctx, unsigned long *h_vdec);
> +
> +	/**
> +	 * (*decode)() - trigger decode
> +	 * @h_vdec  : [in] driver handle
> +	 * @bs      : [in] input bitstream
> +	 * @fb      : [in] frame buffer to store decoded frame
> +	 * @res_chg : [out] resolution change happen
> +	 */
> +	int (*decode)(unsigned long h_vdec, struct mtk_vcodec_mem *bs,
> +		      struct vdec_fb *fb, bool *res_chg);
> +
> +	/**
> +	 * (*get_param)() - get driver's parameter
> +	 * @h_vdec : [in] driver handle
> +	 * @type   : [in] input parameter type
> +	 * @out    : [out] buffer to store query result
> +	 */
> +	int (*get_param)(unsigned long h_vdec, enum vdec_get_param_type type,
> +			 void *out);
> +
> +	/**
> +	 * (*deinit)() - deinitialize driver.
> +	 * @h_vdec : [in] driver handle to be deinit
> +	 */
> +	void (*deinit)(unsigned long h_vdec);
> +};
> +
> +#endif
> diff --git a/drivers/media/platform/mtk-vcodec/vdec_drv_if.c b/drivers/media/platform/mtk-vcodec/vdec_drv_if.c
> new file mode 100644
> index 0000000..8202b14
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vcodec/vdec_drv_if.c
> @@ -0,0 +1,107 @@
> +/*
> + * Copyright (c) 2016 MediaTek Inc.
> + * Author: PC Chen <pc.chen@mediatek.com>
> + *         Tiffany Lin <tiffany.lin@mediatek.com>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + */
> +
> +#include <linux/interrupt.h>
> +#include <linux/kernel.h>
> +#include <linux/slab.h>
> +
> +#include "vdec_drv_base.h"
> +#include "vdec_drv_if.h"
> +
> +#include "mtk_vcodec_dec_pm.h"
> +#include "mtk_vcodec_util.h"
> +#include "mtk_vpu.h"
> +
> +int vdec_if_init(struct mtk_vcodec_ctx *ctx, unsigned int fourcc)
> +{
> +	int ret = 0;
> +
> +	switch (fourcc) {
> +	case V4L2_PIX_FMT_H264:
> +	case V4L2_PIX_FMT_VP8:
> +	case V4L2_PIX_FMT_VP9:
> +	default:
> +		return -EINVAL;
> +	}
> +
> +	mtk_vdec_lock(ctx);
> +	mtk_vcodec_dec_clock_on(&ctx->dev->pm);
> +	ret = ctx->dec_if->init(ctx, &ctx->drv_handle);
> +	mtk_vcodec_dec_clock_off(&ctx->dev->pm);
> +	mtk_vdec_unlock(ctx);
> +
> +	return ret;
> +}
> +
> +int vdec_if_decode(struct mtk_vcodec_ctx *ctx, struct mtk_vcodec_mem *bs,
> +		   struct vdec_fb *fb, bool *res_chg)
> +{
> +	int ret = 0;
> +
> +	if (bs) {
> +		if ((bs->dma_addr & 63) != 0) {
> +			mtk_v4l2_err("bs dma_addr should 64 byte align");
> +			return -EINVAL;
> +		}
> +	}
> +
> +	if (fb) {
> +		if (((fb->base_y.dma_addr & 511) != 0) ||
> +		    ((fb->base_c.dma_addr & 511) != 0)) {
> +			mtk_v4l2_err("frame buffer dma_addr should 512 byte align");
> +			return -EINVAL;
> +		}
> +	}
> +
> +	mtk_vdec_lock(ctx);
> +
> +	mtk_vcodec_set_curr_ctx(ctx->dev, ctx);
> +	mtk_vcodec_dec_clock_on(&ctx->dev->pm);
> +	enable_irq(ctx->dev->dec_irq);
> +	ret = ctx->dec_if->decode(ctx->drv_handle, bs, fb, res_chg);
> +	disable_irq(ctx->dev->dec_irq);
> +	mtk_vcodec_dec_clock_off(&ctx->dev->pm);
> +	mtk_vcodec_set_curr_ctx(ctx->dev, NULL);
> +
> +	mtk_vdec_unlock(ctx);
> +
> +	return ret;
> +}
> +
> +int vdec_if_get_param(struct mtk_vcodec_ctx *ctx, enum vdec_get_param_type type,
> +		      void *out)
> +{
> +	int ret = 0;
> +
> +	mtk_vdec_lock(ctx);
> +	ret = ctx->dec_if->get_param(ctx->drv_handle, type, out);
> +	mtk_vdec_unlock(ctx);
> +
> +	return ret;
> +}
> +
> +void vdec_if_deinit(struct mtk_vcodec_ctx *ctx)
> +{
> +	if (ctx->drv_handle == 0)
> +		return;
> +
> +	mtk_vdec_lock(ctx);
> +	mtk_vcodec_dec_clock_on(&ctx->dev->pm);
> +	ctx->dec_if->deinit(ctx->drv_handle);
> +	mtk_vcodec_dec_clock_off(&ctx->dev->pm);
> +	mtk_vdec_unlock(ctx);
> +
> +	ctx->drv_handle = 0;
> +}
> diff --git a/drivers/media/platform/mtk-vcodec/vdec_drv_if.h b/drivers/media/platform/mtk-vcodec/vdec_drv_if.h
> new file mode 100644
> index 0000000..db6b520
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vcodec/vdec_drv_if.h
> @@ -0,0 +1,101 @@
> +/*
> + * Copyright (c) 2016 MediaTek Inc.
> + * Author: PC Chen <pc.chen@mediatek.com>
> + *		   Tiffany Lin <tiffany.lin@mediatek.com>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + */
> +
> +#ifndef _VDEC_DRV_IF_H_
> +#define _VDEC_DRV_IF_H_
> +
> +#include "mtk_vcodec_drv.h"
> +#include "mtk_vcodec_dec.h"
> +#include "mtk_vcodec_util.h"
> +
> +
> +/**
> + * struct vdec_fb_status  - decoder frame buffer status
> + * @FB_ST_NORMAL	: initial state
> + * @FB_ST_DISPLAY	: frmae buffer is ready to be displayed
> + * @FB_ST_FREE		: frame buffer is not used by decoder any more
> + */
> +enum vdec_fb_status {
> +	FB_ST_NORMAL		= 0,
> +	FB_ST_DISPLAY		= (1 << 0),
> +	FB_ST_FREE		= (1 << 1)
> +};
> +
> +/* For GET_PARAM_DISP_FRAME_BUFFER and GET_PARAM_FREE_FRAME_BUFFER,
> + * the caller does not own the returned buffer. The buffer will not be
> + *				released before vdec_if_deinit.
> + * GET_PARAM_DISP_FRAME_BUFFER	: get next displayable frame buffer,
> + *				struct vdec_fb**
> + * GET_PARAM_FREE_FRAME_BUFFER	: get non-referenced framebuffer, vdec_fb**
> + * GET_PARAM_PIC_INFO		: get picture info, struct vdec_pic_info*
> + * GET_PARAM_CROP_INFO		: get crop info, struct v4l2_crop*
> + * GET_PARAM_DPB_SIZE		: get dpb size, unsigned int*
> + */
> +enum vdec_get_param_type {
> +	GET_PARAM_DISP_FRAME_BUFFER,
> +	GET_PARAM_FREE_FRAME_BUFFER,
> +	GET_PARAM_PIC_INFO,
> +	GET_PARAM_CROP_INFO,
> +	GET_PARAM_DPB_SIZE
> +};
> +
> +/**
> + * struct vdec_fb_node  - decoder frame buffer node
> + * @list	: list to hold this node
> + * @fb	: point to frame buffer (vdec_fb), fb could point to frame buffer and
> + *	working buffer this is for maintain buffers in different state
> + */
> +struct vdec_fb_node {
> +	struct list_head list;
> +	struct vdec_fb *fb;
> +};
> +
> +/**
> + * vdec_if_init() - initialize decode driver
> + * @ctx	: [in] v4l2 context
> + * @fourcc	: [in] video format fourcc, V4L2_PIX_FMT_H264/VP8/VP9..
> + */
> +int vdec_if_init(struct mtk_vcodec_ctx *ctx, unsigned int fourcc);
> +
> +/**
> + * vdec_if_deinit() - deinitialize decode driver
> + * @ctx	: [in] v4l2 context
> + *
> + */
> +void vdec_if_deinit(struct mtk_vcodec_ctx *ctx);
> +
> +/**
> + * vdec_if_decode() - trigger decode
> + * @ctx	: [in] v4l2 context
> + * @bs	: [in] input bitstream
> + * @fb	: [in] frame buffer to store decoded frame, when null menas parse
> + *	header only
> + * @res_chg	: [out] resolution change happens if current bs have different
> + *	picture width/height
> + * Note: To flush the decoder when reaching EOF, set input bitstream as NULL.
> + */
> +int vdec_if_decode(struct mtk_vcodec_ctx *ctx, struct mtk_vcodec_mem *bs,
> +		   struct vdec_fb *fb, bool *res_chg);
> +
> +/**
> + * vdec_if_get_param() - get driver's parameter
> + * @ctx	: [in] v4l2 context
> + * @type	: [in] input parameter type
> + * @out	: [out] buffer to store query result
> + */
> +int vdec_if_get_param(struct mtk_vcodec_ctx *ctx, enum vdec_get_param_type type,
> +		      void *out);
> +
> +#endif
> diff --git a/drivers/media/platform/mtk-vcodec/vdec_ipi_msg.h b/drivers/media/platform/mtk-vcodec/vdec_ipi_msg.h
> new file mode 100644
> index 0000000..5a8a629
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vcodec/vdec_ipi_msg.h
> @@ -0,0 +1,103 @@
> +/*
> + * Copyright (c) 2016 MediaTek Inc.
> + * Author: PC Chen <pc.chen@mediatek.com>
> + *
> + * This program is free software; you can redistribute it and/or
> + * modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + */
> +
> +#ifndef _VDEC_IPI_MSG_H_
> +#define _VDEC_IPI_MSG_H_
> +
> +/**
> + * enum vdec_ipi_msgid - message id between AP and VPU
> + * @AP_IPIMSG_XXX	: AP to VPU cmd message id
> + * @VPU_IPIMSG_XXX_ACK	: VPU ack AP cmd message id
> + */
> +enum vdec_ipi_msgid {
> +	AP_IPIMSG_DEC_INIT = 0xA000,
> +	AP_IPIMSG_DEC_START = 0xA001,
> +	AP_IPIMSG_DEC_END = 0xA002,
> +	AP_IPIMSG_DEC_DEINIT = 0xA003,
> +	AP_IPIMSG_DEC_RESET = 0xA004,
> +
> +	VPU_IPIMSG_DEC_INIT_ACK = 0xB000,
> +	VPU_IPIMSG_DEC_START_ACK = 0xB001,
> +	VPU_IPIMSG_DEC_END_ACK = 0xB002,
> +	VPU_IPIMSG_DEC_DEINIT_ACK = 0xB003,
> +	VPU_IPIMSG_DEC_RESET_ACK = 0xB004,
> +};
> +
> +/**
> + * struct vdec_ap_ipi_cmd - generic AP to VPU ipi command format
> + * @msg_id	: vdec_ipi_msgid
> + * @vpu_inst_addr	: VPU decoder instance address
> + */
> +struct vdec_ap_ipi_cmd {
> +	uint32_t msg_id;
> +	uint32_t vpu_inst_addr;
> +};
> +
> +/**
> + * struct vdec_vpu_ipi_ack - generic VPU to AP ipi command format
> + * @msg_id	: vdec_ipi_msgid
> + * @status	: VPU exeuction result
> + * @ap_inst_addr	: AP video decoder instance address
> + */
> +struct vdec_vpu_ipi_ack {
> +	uint32_t msg_id;
> +	int32_t status;
> +	uint64_t ap_inst_addr;
> +};
> +
> +/**
> + * struct vdec_ap_ipi_init - for AP_IPIMSG_DEC_INIT
> + * @msg_id	: AP_IPIMSG_DEC_INIT
> + * @reserved	: Reserved field
> + * @ap_inst_addr	: AP video decoder instance address
> + */
> +struct vdec_ap_ipi_init {
> +	uint32_t msg_id;
> +	uint32_t reserved;
> +	uint64_t ap_inst_addr;
> +};
> +
> +/**
> + * struct vdec_ap_ipi_dec_start - for AP_IPIMSG_DEC_START
> + * @msg_id	: AP_IPIMSG_DEC_START
> + * @vpu_inst_addr	: VPU decoder instance address
> + * @data	: Header info
> + *	H264 decoder [0]:buf_sz [1]:nal_start
> + *	VP8 decoder  [0]:width/height
> + *	VP9 decoder  [0]:profile, [1][2] width/height
> + * @reserved	: Reserved field
> + */
> +struct vdec_ap_ipi_dec_start {
> +	uint32_t msg_id;
> +	uint32_t vpu_inst_addr;
> +	uint32_t data[3];
> +	uint32_t reserved;
> +};
> +
> +/**
> + * struct vdec_vpu_ipi_init_ack - for VPU_IPIMSG_DEC_INIT_ACK
> + * @msg_id	: VPU_IPIMSG_DEC_INIT_ACK
> + * @status	: VPU exeuction result
> + * @ap_inst_addr	: AP vcodec_vpu_inst instance address
> + * @vpu_inst_addr	: VPU decoder instance address
> + */
> +struct vdec_vpu_ipi_init_ack {
> +	uint32_t msg_id;
> +	int32_t status;
> +	uint64_t ap_inst_addr;
> +	uint32_t vpu_inst_addr;
> +};
> +
> +#endif
> 

Regards,

	Hans

[PATCH v3 5/9] vcodec: mediatek: Add Mediatek V4L2 Video Decoder Driver

[Tiffany Lin]

Hi Hans,

On Fri, 2016-07-08 at 13:06 +0200, Hans Verkuil wrote:
> On 05/30/2016 02:29 PM, Tiffany Lin wrote:
> > Add v4l2 layer decoder driver for MT8173
> > 
> > Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
> > ---
> >  drivers/media/platform/mtk-vcodec/Makefile         |   10 +-
> >  drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c | 1348 ++++++++++++++++++++
> >  drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.h |   88 ++
> >  .../media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c |  408 ++++++
> >  .../media/platform/mtk-vcodec/mtk_vcodec_dec_pm.c  |  206 +++
> >  .../media/platform/mtk-vcodec/mtk_vcodec_dec_pm.h  |   28 +
> >  drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h |   88 +-
> >  .../media/platform/mtk-vcodec/mtk_vcodec_intr.c    |    4 +-
> >  .../media/platform/mtk-vcodec/mtk_vcodec_intr.h    |    2 +-
> >  .../media/platform/mtk-vcodec/mtk_vcodec_util.c    |   31 +-
> >  .../media/platform/mtk-vcodec/mtk_vcodec_util.h    |    7 +-
> >  drivers/media/platform/mtk-vcodec/vdec_drv_base.h  |   56 +
> >  drivers/media/platform/mtk-vcodec/vdec_drv_if.c    |  107 ++
> >  drivers/media/platform/mtk-vcodec/vdec_drv_if.h    |  101 ++
> >  drivers/media/platform/mtk-vcodec/vdec_ipi_msg.h   |  103 ++
> >  15 files changed, 2569 insertions(+), 18 deletions(-)
> >  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c
> >  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.h
> >  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c
> >  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_pm.c
> >  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_pm.h
> >  create mode 100644 drivers/media/platform/mtk-vcodec/vdec_drv_base.h
> >  create mode 100644 drivers/media/platform/mtk-vcodec/vdec_drv_if.c
> >  create mode 100644 drivers/media/platform/mtk-vcodec/vdec_drv_if.h
> >  create mode 100644 drivers/media/platform/mtk-vcodec/vdec_ipi_msg.h
> > 
> > diff --git a/drivers/media/platform/mtk-vcodec/Makefile b/drivers/media/platform/mtk-vcodec/Makefile
> > index dc5cb00..4c8ed2f 100644
> > --- a/drivers/media/platform/mtk-vcodec/Makefile
> > +++ b/drivers/media/platform/mtk-vcodec/Makefile
> > @@ -1,7 +1,13 @@
> >  
> >  
> > -obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += mtk-vcodec-enc.o mtk-vcodec-common.o
> > -
> > +obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += mtk-vcodec-dec.o \
> > +				       mtk-vcodec-enc.o \
> > +				       mtk-vcodec-common.o
> > +
> > +mtk-vcodec-dec-y := mtk_vcodec_dec_drv.o \
> > +		vdec_drv_if.o \
> > +		mtk_vcodec_dec.o \
> > +		mtk_vcodec_dec_pm.o \
> >  
> >  
> >  mtk-vcodec-enc-y := venc/venc_vp8_if.o \
> > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c
> > new file mode 100644
> > index 0000000..792858a
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c
> > @@ -0,0 +1,1348 @@
> > +/*
> > +* Copyright (c) 2016 MediaTek Inc.
> > +* Author: PC Chen <pc.chen@mediatek.com>
> > +*         Tiffany Lin <tiffany.lin@mediatek.com>
> > +*
> > +* This program is free software; you can redistribute it and/or modify
> > +* it under the terms of the GNU General Public License version 2 as
> > +* published by the Free Software Foundation.
> > +*
> > +* This program is distributed in the hope that it will be useful,
> > +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> > +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > +* GNU General Public License for more details.
> > +*/
> > +
> > +#include <media/v4l2-event.h>
> > +#include <media/v4l2-mem2mem.h>
> > +#include <media/videobuf2-dma-contig.h>
> > +
> > +#include "mtk_vcodec_drv.h"
> > +#include "mtk_vcodec_dec.h"
> > +#include "mtk_vcodec_intr.h"
> > +#include "mtk_vcodec_util.h"
> > +#include "vdec_drv_if.h"
> > +#include "mtk_vcodec_dec_pm.h"
> > +
> > +#define OUT_FMT_IDX	0
> > +#define CAP_FMT_IDX	3
> > +
> > +#define MTK_VDEC_MIN_W	64U
> > +#define MTK_VDEC_MIN_H	64U
> > +#define DFT_CFG_WIDTH	MTK_VDEC_MIN_W
> > +#define DFT_CFG_HEIGHT	MTK_VDEC_MIN_H
> > +
> > +static struct mtk_video_fmt mtk_video_formats[] = {
> > +	{
> > +		.fourcc = V4L2_PIX_FMT_H264,
> > +		.type = MTK_FMT_DEC,
> > +		.num_planes = 1,
> > +	},
> > +	{
> > +		.fourcc = V4L2_PIX_FMT_VP8,
> > +		.type = MTK_FMT_DEC,
> > +		.num_planes = 1,
> > +	},
> > +	{
> > +		.fourcc = V4L2_PIX_FMT_VP9,
> > +		.type = MTK_FMT_DEC,
> > +		.num_planes = 1,
> > +	},
> > +	{
> > +		.fourcc = V4L2_PIX_FMT_MT21,
> > +		.type = MTK_FMT_FRAME,
> > +		.num_planes = 2,
> > +	},
> > +};
> > +
> > +static const struct mtk_codec_framesizes mtk_vdec_framesizes[] = {
> > +	{
> > +		.fourcc	= V4L2_PIX_FMT_H264,
> > +		.stepwise = {  MTK_VDEC_MIN_W, MTK_VDEC_MAX_W, 16,
> > +				MTK_VDEC_MIN_H, MTK_VDEC_MAX_H, 16 },
> > +	},
> > +	{
> > +		.fourcc	= V4L2_PIX_FMT_VP8,
> > +		.stepwise = {  MTK_VDEC_MIN_W, MTK_VDEC_MAX_W, 16,
> > +				MTK_VDEC_MIN_H, MTK_VDEC_MAX_H, 16 },
> > +	},
> > +	{
> > +		.fourcc = V4L2_PIX_FMT_VP9,
> > +		.stepwise = {  MTK_VDEC_MIN_W, MTK_VDEC_MAX_W, 16,
> > +				MTK_VDEC_MIN_H, MTK_VDEC_MAX_H, 16 },
> > +	},
> > +};
> > +
> > +#define NUM_SUPPORTED_FRAMESIZE ARRAY_SIZE(mtk_vdec_framesizes)
> > +#define NUM_FORMATS ARRAY_SIZE(mtk_video_formats)
> > +
> > +static struct mtk_video_fmt *mtk_vdec_find_format(struct v4l2_format *f)
> > +{
> > +	struct mtk_video_fmt *fmt;
> > +	unsigned int k;
> > +
> > +	for (k = 0; k < NUM_FORMATS; k++) {
> > +		fmt = &mtk_video_formats[k];
> > +		if (fmt->fourcc == f->fmt.pix_mp.pixelformat)
> > +			return fmt;
> > +	}
> > +
> > +	return NULL;
> > +}
> > +
> > +static struct mtk_q_data *mtk_vdec_get_q_data(struct mtk_vcodec_ctx *ctx,
> > +					      enum v4l2_buf_type type)
> > +{
> > +	if (V4L2_TYPE_IS_OUTPUT(type))
> > +		return &ctx->q_data[MTK_Q_DATA_SRC];
> > +
> > +	return &ctx->q_data[MTK_Q_DATA_DST];
> > +}
> > +
> > +/* This function is try to clean all display buffer, the buffer will return in
> 
> s/is try/tries/
> s/buffer/buffers/ (2 times)
> 
> checkpatch should complain about this comment block format ('/*' should be on
> its own line). Please run checkpatch to double check.
> 
> > + * display order.
> > + * Note the buffer return from codec driver may still in driver's reference
> 
> s/buffer return/buffers returned/
> s/still/still be/
> 

Will fix this in next version.

> > + * list.
> > + */
> > +static struct vb2_buffer *get_display_buffer(struct mtk_vcodec_ctx *ctx)
> > +{
> > +	struct vdec_fb *disp_frame_buffer = NULL;
> > +	struct mtk_video_dec_buf *dstbuf;
> > +
> > +	mtk_v4l2_debug(3, "[%d]", ctx->id);
> > +	if (vdec_if_get_param(ctx,
> > +			GET_PARAM_DISP_FRAME_BUFFER,
> > +			&disp_frame_buffer)) {
> > +		mtk_v4l2_err("[%d]Cannot get param : GET_PARAM_DISP_FRAME_BUFFER",
> > +			ctx->id);
> > +		return NULL;
> > +	}
> > +
> > +	if (disp_frame_buffer == NULL) {
> > +		mtk_v4l2_debug(3, "No display frame buffer");
> > +		return NULL;
> > +	}
> > +
> > +	dstbuf = container_of(disp_frame_buffer, struct mtk_video_dec_buf,
> > +				frame_buffer);
> > +	mutex_lock(&ctx->lock);
> > +	if (dstbuf->used) {
> > +		vb2_set_plane_payload(&dstbuf->vb.vb2_buf, 0,
> > +					ctx->picinfo.y_bs_sz);
> > +		vb2_set_plane_payload(&dstbuf->vb.vb2_buf, 1,
> > +					ctx->picinfo.c_bs_sz);
> > +
> > +		dstbuf->ready_to_display = true;
> > +
> > +		mtk_v4l2_debug(2,
> > +				"[%d]status=%x queue id=%d to done_list %d",
> > +				ctx->id, disp_frame_buffer->status,
> > +				dstbuf->vb.vb2_buf.index,
> > +				dstbuf->queued_in_vb2);
> > +
> > +		v4l2_m2m_buf_done(&dstbuf->vb, VB2_BUF_STATE_DONE);
> > +		ctx->decoded_frame_cnt++;
> > +	}
> > +	mutex_unlock(&ctx->lock);
> > +	return &dstbuf->vb.vb2_buf;
> > +}
> > +
> > +/* This function is try to clean all capture buffers that is not used as
> 
> s/is try/tries/
> s/is not/are not/
> 
> > + * reference buffer by codec driver any more
> 
> buffers
> 
> > + * In this case, we need re-queue buffer to vb2 buffer if user space
> > + * already return this buffer to v4l2 or this buffer is just the output of
> 
> returns
> 
> > + * previous sps/pps/resolution change decode, or do nothing if user
> > + * space still own this buffer
> 
> owns
> 

Will fix this in next version.

> > + */
> > +static struct vb2_buffer *get_free_buffer(struct mtk_vcodec_ctx *ctx)
> > +{
> > +	struct mtk_video_dec_buf *dstbuf;
> > +	struct vdec_fb *free_frame_buffer = NULL;
> > +
> > +	if (vdec_if_get_param(ctx,
> > +				GET_PARAM_FREE_FRAME_BUFFER,
> > +				&free_frame_buffer)) {
> > +		mtk_v4l2_err("[%d] Error!! Cannot get param", ctx->id);
> > +		return NULL;
> > +	}
> > +	if (free_frame_buffer == NULL) {
> > +		mtk_v4l2_debug(3, " No free frame buffer");
> > +		return NULL;
> > +	}
> > +
> > +	mtk_v4l2_debug(3, "[%d] tmp_frame_addr = 0x%p",
> > +			ctx->id, free_frame_buffer);
> > +
> > +	dstbuf = container_of(free_frame_buffer, struct mtk_video_dec_buf,
> > +				frame_buffer);
> > +
> > +	mutex_lock(&ctx->lock);
> > +	if (dstbuf->used) {
> > +		if ((dstbuf->queued_in_vb2) &&
> > +		    (dstbuf->queued_in_v4l2) &&
> > +		    (free_frame_buffer->status == FB_ST_FREE)) {
> > +			/* After decode sps/pps or non-display buffer, we dont
> 
> don't
> 
> > +			 * need to return capture buffer to user space, but
> > +			 * just re-queue this capture buffer to vb2 queue.
> 
> > +			 * This reduce overheads that dq/q unused capture
> > +			 * buffer We use queued_in_vb2 to mark this buffer
> > +			 * is from decode output, not q from user space.
> 
> garbled comment. Not sure what is meant here.
> 
Will fix and refine comment in next version

> > +			 */
> > +			mtk_v4l2_debug(2,
> > +				"[%d]status=%x queue id=%d to rdy_queue %d",
> > +				ctx->id, free_frame_buffer->status,
> > +				dstbuf->vb.vb2_buf.index,
> > +				dstbuf->queued_in_vb2);
> > +			v4l2_m2m_buf_queue(ctx->m2m_ctx, &dstbuf->vb);
> > +		} else if ((dstbuf->queued_in_vb2 == false) &&
> > +			   (dstbuf->queued_in_v4l2 == true)) {
> > +			/* If buffer in v4l2 driver but not in vb2 queue yet,
> > +			 * and we get this buffer from free_list, it means
> > +			 * that codec driver do not use this buffer as
> > +			 * reference buffer anymore. We should q buffer to vb2
> > +			 * queue, so later work thread could get this buffer
> > +			 * for decode In this sate, queued_in_vb2 = false
> 
> s/sate/case/
> 
> > +			 * menas this buffer is not from previous decode output
> 
> means
> 
Will fix this in next version.

> > +			 */
> > +			mtk_v4l2_debug(2,
> > +					"[%d]status=%x queue id=%d to rdy_queue",
> > +					ctx->id, free_frame_buffer->status,
> > +					dstbuf->vb.vb2_buf.index);
> > +			v4l2_m2m_buf_queue(ctx->m2m_ctx, &dstbuf->vb);
> > +			dstbuf->queued_in_vb2 = true;
> > +		} else {
> > +			/* Codec driver do not need to reference this capture
> > +			 * buffer and this buffer is not in v4l2 driver.
> > +			 * Then we dont need to do any thing, just add log when
> 
> don't
> 
Will fix this in next version.

> > +			 * we need to debug buffer flow.
> > +			 * When this buffer q from user space, it could
> > +			 * directly q to vb2 buffer
> > +			 */
> > +			mtk_v4l2_debug(3, "[%d]status=%x err queue id=%d %d %d",
> > +					ctx->id, free_frame_buffer->status,
> > +					dstbuf->vb.vb2_buf.index,
> > +					dstbuf->queued_in_vb2,
> > +					dstbuf->queued_in_v4l2);
> > +		}
> > +		dstbuf->used = false;
> > +	}
> > +	mutex_unlock(&ctx->lock);
> > +	return &dstbuf->vb.vb2_buf;
> > +}
> > +
> > +static void clean_display_buffer(struct mtk_vcodec_ctx *ctx)
> > +{
> > +	struct vb2_buffer *framptr;
> > +
> > +	do {
> > +		framptr = get_display_buffer(ctx);
> > +	} while (framptr);
> > +}
> > +
> > +static void clean_free_buffer(struct mtk_vcodec_ctx *ctx)
> > +{
> > +	struct vb2_buffer *framptr;
> > +
> > +	do {
> > +		framptr = get_free_buffer(ctx);
> > +	} while (framptr);
> > +}
> > +
> > +static int mtk_vdec_queue_res_chg_event(struct mtk_vcodec_ctx *ctx)
> 
> This can be a void function.
> 
Will fix this in next version.

> > +{
> > +	static const struct v4l2_event ev_src_ch = {
> > +		.type = V4L2_EVENT_SOURCE_CHANGE,
> > +		.u.src_change.changes =
> > +		V4L2_EVENT_SRC_CH_RESOLUTION,
> > +	};
> > +
> > +	mtk_v4l2_debug(1, "[%d]", ctx->id);
> > +	v4l2_event_queue_fh(&ctx->fh, &ev_src_ch);
> > +
> > +	return 0;
> > +}
> > +
> > +static void mtk_vdec_flush_decoder(struct mtk_vcodec_ctx *ctx)
> > +{
> > +	bool res_chg;
> > +	int ret = 0;
> > +
> > +	ret = vdec_if_decode(ctx, NULL, NULL, &res_chg);
> > +	if (ret)
> > +		mtk_v4l2_err("DecodeFinal failed, ret=%d", ret);
> > +
> > +	clean_display_buffer(ctx);
> > +	clean_free_buffer(ctx);
> > +}
> > +
> > +static void mtk_vdec_pic_info_update(struct mtk_vcodec_ctx *ctx)
> > +{
> > +	unsigned int dpbsize = 0;
> > +	int ret;
> > +
> > +	if (vdec_if_get_param(ctx,
> > +				GET_PARAM_PIC_INFO,
> > +				&ctx->last_decoded_picinfo)) {
> > +		mtk_v4l2_err("[%d]Error!! Cannot get param : GET_PARAM_PICTURE_INFO ERR",
> > +				ctx->id);
> > +		return;
> > +	}
> > +
> > +	if (ctx->last_decoded_picinfo.pic_w == 0 ||
> > +		ctx->last_decoded_picinfo.pic_h == 0 ||
> > +		ctx->last_decoded_picinfo.buf_w == 0 ||
> > +		ctx->last_decoded_picinfo.buf_h == 0) {
> > +		mtk_v4l2_err("Cannot get correct pic info");
> > +		return;
> > +	}
> > +
> > +	if ((ctx->last_decoded_picinfo.pic_w == ctx->picinfo.pic_w) ||
> > +	    (ctx->last_decoded_picinfo.pic_h == ctx->picinfo.pic_h))
> > +		return;
> > +
> > +	mtk_v4l2_debug(1,
> > +			"[%d]-> new(%d,%d), old(%d,%d), real(%d,%d)",
> > +			ctx->id, ctx->last_decoded_picinfo.pic_w,
> > +			ctx->last_decoded_picinfo.pic_h,
> > +			ctx->picinfo.pic_w, ctx->picinfo.pic_h,
> > +			ctx->last_decoded_picinfo.buf_w,
> > +			ctx->last_decoded_picinfo.buf_h);
> > +
> > +	ret = vdec_if_get_param(ctx, GET_PARAM_DPB_SIZE, &dpbsize);
> > +	if (dpbsize == 0)
> > +		mtk_v4l2_err("Incorrect dpb size, ret=%d", ret);
> > +
> > +	ctx->dpb_size = dpbsize;
> > +}
> > +
> > +static void mtk_vdec_worker(struct work_struct *work)
> > +{
> > +	struct mtk_vcodec_ctx *ctx = container_of(work, struct mtk_vcodec_ctx,
> > +				decode_work);
> > +	struct mtk_vcodec_dev *dev = ctx->dev;
> > +	struct vb2_buffer *src_buf, *dst_buf;
> > +	struct mtk_vcodec_mem buf;
> > +	struct vdec_fb *pfb;
> > +	bool res_chg = false;
> > +	int ret;
> > +	struct mtk_video_dec_buf *dst_buf_info, *src_buf_info;
> > +	struct vb2_v4l2_buffer *dst_vb2_v4l2, *src_vb2_v4l2;
> > +
> > +	src_buf = v4l2_m2m_next_src_buf(ctx->m2m_ctx);
> > +	if (src_buf == NULL) {
> > +		v4l2_m2m_job_finish(dev->m2m_dev_dec, ctx->m2m_ctx);
> > +		mtk_v4l2_debug(1, "[%d] src_buf empty!!", ctx->id);
> > +		return;
> > +	}
> > +
> > +	dst_buf = v4l2_m2m_next_dst_buf(ctx->m2m_ctx);
> > +	if (dst_buf == NULL) {
> > +		v4l2_m2m_job_finish(dev->m2m_dev_dec, ctx->m2m_ctx);
> > +		mtk_v4l2_debug(1, "[%d] dst_buf empty!!", ctx->id);
> > +		return;
> > +	}
> > +
> > +	src_vb2_v4l2 = container_of(src_buf, struct vb2_v4l2_buffer, vb2_buf);
> > +	src_buf_info = container_of(src_vb2_v4l2, struct mtk_video_dec_buf, vb);
> > +
> > +	dst_vb2_v4l2 = container_of(dst_buf, struct vb2_v4l2_buffer, vb2_buf);
> > +	dst_buf_info = container_of(dst_vb2_v4l2, struct mtk_video_dec_buf, vb);
> > +
> > +	buf.va = vb2_plane_vaddr(src_buf, 0);
> > +	buf.dma_addr = vb2_dma_contig_plane_dma_addr(src_buf, 0);
> > +	buf.size = (size_t)src_buf->planes[0].bytesused;
> > +	if (!buf.va) {
> > +		v4l2_m2m_job_finish(dev->m2m_dev_dec, ctx->m2m_ctx);
> > +		mtk_v4l2_err("[%d] id=%d src_addr is NULL!!",
> > +				ctx->id, src_buf->index);
> > +		return;
> > +	}
> > +
> > +	pfb = &dst_buf_info->frame_buffer;
> > +	pfb->base_y.va = vb2_plane_vaddr(dst_buf, 0);
> > +	pfb->base_y.dma_addr = vb2_dma_contig_plane_dma_addr(dst_buf, 0);
> > +	pfb->base_y.size = ctx->picinfo.y_bs_sz + ctx->picinfo.y_len_sz;
> > +
> > +	pfb->base_c.va = vb2_plane_vaddr(dst_buf, 1);
> > +	pfb->base_c.dma_addr = vb2_dma_contig_plane_dma_addr(dst_buf, 1);
> > +	pfb->base_c.size = ctx->picinfo.c_bs_sz + ctx->picinfo.c_len_sz;
> > +	pfb->status = 0;
> > +	mtk_v4l2_debug(3, "===>[%d] vdec_if_decode() ===>", ctx->id);
> > +	mtk_v4l2_debug(3, "[%d] Bitstream VA=%p DMA=%llx Size=0x%lx vb=%p",
> > +			ctx->id, buf.va, (u64)buf.dma_addr, buf.size, src_buf);
> > +
> > +	mtk_v4l2_debug(3,
> > +			"id=%d Framebuf  pfb=%p VA=%p Y_DMA=%llx C_DMA=%llx Size=0x%lx",
> > +			dst_buf->index, pfb,
> > +			pfb->base_y.va, (u64)pfb->base_y.dma_addr,
> > +			(u64)pfb->base_c.dma_addr, pfb->base_y.size);
> > +
> > +	if (src_buf_info->lastframe) {
> > +		/* update src buf status */
> > +		src_buf = v4l2_m2m_src_buf_remove(ctx->m2m_ctx);
> > +		src_buf_info->lastframe = false;
> > +		v4l2_m2m_buf_done(&src_buf_info->vb, VB2_BUF_STATE_DONE);
> > +
> > +		/* update dst buf status */
> > +		dst_buf = v4l2_m2m_dst_buf_remove(ctx->m2m_ctx);
> > +		dst_buf_info->used = false;
> > +
> > +		vdec_if_decode(ctx, NULL, NULL, &res_chg);
> > +		clean_display_buffer(ctx);
> > +		vb2_set_plane_payload(&dst_buf_info->vb.vb2_buf, 0, 0);
> > +		vb2_set_plane_payload(&dst_buf_info->vb.vb2_buf, 1, 0);
> > +		v4l2_m2m_buf_done(&dst_buf_info->vb, VB2_BUF_STATE_DONE);
> > +		clean_free_buffer(ctx);
> 
> Just call v4l2_m2m_job_finish(dev->m2m_dev_dec, ctx->m2m_ctx); here and return.
> 
> 
> > +	} else {
> 
> This can now have one less TAB.
> 
Got it. Will fix this in next version.

> > +		dst_buf_info->vb.vb2_buf.timestamp
> > +				= src_buf_info->vb.vb2_buf.timestamp;
> > +		dst_buf_info->vb.timecode
> > +				= src_buf_info->vb.timecode;
> > +		mutex_lock(&ctx->lock);
> > +		dst_buf_info->used = true;
> > +		mutex_unlock(&ctx->lock);
> > +		src_buf_info->used = true;
> > +
> > +		ret = vdec_if_decode(ctx, &buf, pfb, &res_chg);
> > +
> > +		if (ret) {
> > +			mtk_v4l2_err(
> > +				" <===[%d], src_buf[%d]%d sz=0x%zx pts=%llu dst_buf[%d] vdec_if_decode() ret=%d res_chg=%d===>",
> > +				ctx->id,
> > +				src_buf->index,
> > +				src_buf_info->lastframe,
> > +				buf.size,
> > +				src_buf_info->vb.vb2_buf.timestamp,
> > +				dst_buf->index,
> > +				ret, res_chg);
> > +			src_buf = v4l2_m2m_src_buf_remove(ctx->m2m_ctx);
> > +			v4l2_m2m_buf_done(&src_buf_info->vb,
> > +					VB2_BUF_STATE_ERROR);
> > +		} else if (res_chg == false) {
> > +			/* we only return src buffer with VB2_BUF_STATE_DONE
> > +			  * when decode success without resolution change
> > +			  */
> > +			src_buf = v4l2_m2m_src_buf_remove(ctx->m2m_ctx);
> > +			v4l2_m2m_buf_done(&src_buf_info->vb,
> > +					VB2_BUF_STATE_DONE);
> > +		}
> > +
> > +		dst_buf = v4l2_m2m_dst_buf_remove(ctx->m2m_ctx);
> > +		clean_display_buffer(ctx);
> > +		clean_free_buffer(ctx);
> > +
> > +		if (!ret && res_chg) {
> > +			mtk_vdec_pic_info_update(ctx);
> > +			/*
> > +			 * On encountering a resolution change in the stream.
> > +			 * The driver must first process and decode all
> > +			 * remaining buffers from before the resolution change
> > +			 * point, so call flush decode here
> > +			 */
> > +			mtk_vdec_flush_decoder(ctx);
> > +			/*
> > +			 * After all buffers containing decoded frames from
> > +			 * before the resolution change point ready to be
> > +			 * dequeued on the CAPTURE queue, the driver sends a
> > +			 * V4L2_EVENT_SOURCE_CHANGE event for source change
> > +			 * type V4L2_EVENT_SRC_CH_RESOLUTION
> > +			 */
> > +			mtk_vdec_queue_res_chg_event(ctx);
> > +		}
> > +	}
> > +
> > +	v4l2_m2m_job_finish(dev->m2m_dev_dec, ctx->m2m_ctx);
> > +}
> > +
> > +void mtk_vdec_unlock(struct mtk_vcodec_ctx *ctx)
> > +{
> > +	mutex_unlock(&ctx->dev->dec_mutex);
> > +}
> > +
> > +void mtk_vdec_lock(struct mtk_vcodec_ctx *ctx)
> > +{
> > +	mutex_lock(&ctx->dev->dec_mutex);
> > +}
> > +
> > +void mtk_vcodec_vdec_release(struct mtk_vcodec_ctx *ctx)
> > +{
> > +	vdec_if_deinit(ctx);
> > +	ctx->state = MTK_STATE_FREE;
> > +}
> > +
> > +void mtk_vcodec_dec_set_default_params(struct mtk_vcodec_ctx *ctx)
> > +{
> > +	struct mtk_q_data *q_data;
> > +
> > +	ctx->m2m_ctx->q_lock = &ctx->dev->dev_mutex;
> > +	ctx->fh.m2m_ctx = ctx->m2m_ctx;
> > +	ctx->fh.ctrl_handler = &ctx->ctrl_hdl;
> > +	INIT_WORK(&ctx->decode_work, mtk_vdec_worker);
> > +	ctx->colorspace = V4L2_COLORSPACE_REC709;
> > +	ctx->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
> > +	ctx->quantization = V4L2_QUANTIZATION_DEFAULT;
> > +	ctx->xfer_func = V4L2_XFER_FUNC_DEFAULT;
> > +
> > +	q_data = &ctx->q_data[MTK_Q_DATA_SRC];
> > +	memset(q_data, 0, sizeof(struct mtk_q_data));
> > +	q_data->visible_width = DFT_CFG_WIDTH;
> > +	q_data->visible_height = DFT_CFG_HEIGHT;
> > +	q_data->fmt = &mtk_video_formats[OUT_FMT_IDX];
> > +	q_data->field = V4L2_FIELD_NONE;
> > +
> > +	ctx->q_data[MTK_Q_DATA_DST].sizeimage[0] =
> > +		DFT_CFG_WIDTH * DFT_CFG_HEIGHT;
> > +	ctx->q_data[MTK_Q_DATA_DST].bytesperline[0] = 0;
> > +
> > +	q_data = &ctx->q_data[MTK_Q_DATA_DST];
> > +	memset(q_data, 0, sizeof(struct mtk_q_data));
> > +	q_data->visible_width = DFT_CFG_WIDTH;
> > +	q_data->visible_height = DFT_CFG_HEIGHT;
> > +	q_data->coded_width = DFT_CFG_WIDTH;
> > +	q_data->coded_height = DFT_CFG_HEIGHT;
> > +	q_data->fmt = &mtk_video_formats[CAP_FMT_IDX];
> > +	q_data->field = V4L2_FIELD_NONE;
> > +
> > +	v4l_bound_align_image(&q_data->coded_width,
> > +				MTK_VDEC_MIN_W,
> > +				MTK_VDEC_MAX_W, 4,
> > +				&q_data->coded_height,
> > +				MTK_VDEC_MIN_H,
> > +				MTK_VDEC_MAX_H, 5, 6);
> > +
> > +	q_data->sizeimage[0] = q_data->coded_width * q_data->coded_height;
> > +	q_data->bytesperline[0] = q_data->coded_width;
> > +	q_data->sizeimage[1] = q_data->sizeimage[0] / 2;
> > +	q_data->bytesperline[1] = q_data->coded_width;
> > +}
> > +
> > +static int vidioc_vdec_qbuf(struct file *file, void *priv,
> > +			    struct v4l2_buffer *buf)
> > +{
> > +	struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
> > +	struct vb2_queue *vq;
> > +	struct vb2_buffer *vb;
> > +	struct mtk_video_dec_buf *mtkbuf;
> > +	struct vb2_v4l2_buffer	*vb2_v4l2;
> > +
> > +	if (ctx->state == MTK_STATE_ABORT) {
> > +		mtk_v4l2_err("[%d] Call on QBUF after unrecoverable error",
> > +				ctx->id);
> > +		return -EIO;
> > +	}
> > +
> > +	vq = v4l2_m2m_get_vq(ctx->m2m_ctx, buf->type);
> > +	vb = vq->bufs[buf->index];
> > +	vb2_v4l2 = container_of(vb, struct vb2_v4l2_buffer, vb2_buf);
> > +	mtkbuf = container_of(vb2_v4l2, struct mtk_video_dec_buf, vb);
> > +
> > +	if ((buf->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) &&
> > +	    (buf->m.planes[0].bytesused == 0)) {
> > +		mtkbuf->lastframe = true;
> > +		mtk_v4l2_debug(1, "[%d] (%d) id=%d lastframe=%d (%d,%d, %d) vb=%p",
> > +			 ctx->id, buf->type, buf->index,
> > +			 mtkbuf->lastframe, buf->bytesused,
> > +			 buf->m.planes[0].bytesused, buf->length,
> > +			 vb);
> > +	}
> > +
> > +	return v4l2_m2m_qbuf(file, ctx->m2m_ctx, buf);
> > +}
> > +
> > +static int vidioc_vdec_dqbuf(struct file *file, void *priv,
> > +			     struct v4l2_buffer *buf)
> > +{
> > +	struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
> > +
> > +	if (ctx->state == MTK_STATE_ABORT) {
> > +		mtk_v4l2_err("[%d] Call on DQBUF after unrecoverable error",
> > +				ctx->id);
> > +		return -EIO;
> > +	}
> > +
> > +	return v4l2_m2m_dqbuf(file, ctx->m2m_ctx, buf);
> > +}
> > +
> > +static int vidioc_vdec_querycap(struct file *file, void *priv,
> > +				struct v4l2_capability *cap)
> > +{
> > +	strlcpy(cap->driver, MTK_VCODEC_DEC_NAME, sizeof(cap->driver));
> > +	strlcpy(cap->bus_info, MTK_PLATFORM_STR, sizeof(cap->bus_info));
> > +	strlcpy(cap->card, MTK_PLATFORM_STR, sizeof(cap->card));
> > +
> > +	return 0;
> > +}
> > +
> > +static int vidioc_vdec_subscribe_evt(struct v4l2_fh *fh,
> > +				     const struct v4l2_event_subscription *sub)
> > +{
> > +	switch (sub->type) {
> > +	case V4L2_EVENT_EOS:
> > +		return v4l2_event_subscribe(fh, sub, 2, NULL);
> > +	case V4L2_EVENT_SOURCE_CHANGE:
> > +		return v4l2_src_change_event_subscribe(fh, sub);
> > +	default:
> > +		return v4l2_ctrl_subscribe_event(fh, sub);
> > +	}
> > +}
> > +
> > +static int vidioc_try_fmt_vid_cap_mplane(struct file *file, void *priv,
> > +				struct v4l2_format *f)
> > +{
> > +	struct mtk_video_fmt *fmt;
> > +
> > +	fmt = mtk_vdec_find_format(f);
> > +	if (!fmt) {
> > +		f->fmt.pix.pixelformat = mtk_video_formats[CAP_FMT_IDX].fourcc;
> > +		fmt = mtk_vdec_find_format(f);
> > +	}
> > +
> > +	return 0;
> > +}
> > +
> > +static int vidioc_try_fmt_vid_out_mplane(struct file *file, void *priv,
> > +				struct v4l2_format *f)
> > +{
> > +	struct v4l2_pix_format_mplane *pix_fmt_mp = &f->fmt.pix_mp;
> > +	struct mtk_video_fmt *fmt;
> > +
> > +	fmt = mtk_vdec_find_format(f);
> > +	if (!fmt) {
> > +		f->fmt.pix.pixelformat = mtk_video_formats[OUT_FMT_IDX].fourcc;
> > +		fmt = mtk_vdec_find_format(f);
> > +	}
> > +
> > +	if (pix_fmt_mp->plane_fmt[0].sizeimage == 0) {
> > +		mtk_v4l2_err("sizeimage of output format must be given");
> > +		return -EINVAL;
> > +	}
> > +
> > +	return 0;
> > +}
> > +
> > +static int vidioc_vdec_g_selection(struct file *file, void *priv,
> > +			struct v4l2_selection *s)
> > +{
> > +	struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
> > +
> > +	if (V4L2_TYPE_IS_OUTPUT(s->type))
> > +		return -EINVAL;
> > +
> > +	if (s->target != V4L2_SEL_TGT_CROP)
> > +		return -EINVAL;
> 
> How does the cropping rectangle relate to the format size? There is no s_selection,
> so this doesn't make sense.
> 
I want to return encoded crop information or real display region that
display driver could know where is padding region.
User space use s_fmt/g_fmt  to set/get coded_width and coded_height, and
use g_crop to get real display region.
That's why I do not add s_selection.

> Alternatively, it could be that you are really returning V4L2_SEL_TGT_COMPOSE_PADDED.
> 

V4L2_SEL_TGT_COMPOSE_PADDED means
"The active area and all padding pixels that are inserted or modified by
hardware."
But I just want to return active area to user space.

> > +
> > +	if (ctx->state < MTK_STATE_HEADER)
> > +		return -EINVAL;
> > +
> > +	if ((ctx->q_data[MTK_Q_DATA_SRC].fmt->fourcc == V4L2_PIX_FMT_H264) ||
> > +	    (ctx->q_data[MTK_Q_DATA_SRC].fmt->fourcc == V4L2_PIX_FMT_VP8) ||
> > +	    (ctx->q_data[MTK_Q_DATA_SRC].fmt->fourcc == V4L2_PIX_FMT_VP9)) {
> > +
> > +		if (vdec_if_get_param(ctx, GET_PARAM_CROP_INFO, &(s->r))) {
> > +			mtk_v4l2_debug(2,
> > +					"[%d]Error!! Cannot get param : GET_PARAM_CROP_INFO ERR",
> > +					ctx->id);
> > +			s->r.left = 0;
> > +			s->r.top = 0;
> > +			s->r.width = ctx->picinfo.buf_w;
> > +			s->r.height = ctx->picinfo.buf_h;
> > +		}
> > +		mtk_v4l2_debug(2, "Cropping info: l=%d t=%d w=%d h=%d",
> > +				s->r.left, s->r.top, s->r.width,
> > +				s->r.height);
> > +	} else {
> > +		s->r.left = 0;
> > +		s->r.top = 0;
> > +		s->r.width = ctx->picinfo.pic_w;
> > +		s->r.height = ctx->picinfo.pic_h;
> > +		mtk_v4l2_debug(2, "Cropping info: w=%d h=%d fw=%d fh=%d",
> > +				s->r.width, s->r.height, ctx->picinfo.buf_w,
> > +				ctx->picinfo.buf_h);
> > +	}
> > +	return 0;
> > +}
> > +
> > +static int vidioc_vdec_s_fmt(struct file *file, void *priv,
> > +			     struct v4l2_format *f)
> > +{
> > +	struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
> > +	struct v4l2_pix_format_mplane *pix_mp;
> > +	struct mtk_q_data *q_data;
> > +	int ret = 0;
> > +	struct mtk_video_fmt *fmt;
> > +	struct vb2_queue *vq;
> > +
> > +	mtk_v4l2_debug(3, "[%d]", ctx->id);
> > +
> > +	fmt = mtk_vdec_find_format(f);
> > +	if (fmt == NULL) {
> > +		mtk_v4l2_err("vidioc_vdec_s_fmt fail");
> > +		return ret;
> > +	}
> > +
> > +	pix_mp = &f->fmt.pix_mp;
> > +	if ((f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) &&
> > +	    vb2_is_busy(&ctx->m2m_ctx->out_q_ctx.q)) {
> > +		mtk_v4l2_err("out_q_ctx buffers already requested");
> > +		ret = -EBUSY;
> > +	}
> > +
> > +	if ((f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) &&
> > +	    vb2_is_busy(&ctx->m2m_ctx->cap_q_ctx.q)) {
> > +		mtk_v4l2_err("cap_q_ctx buffers already requested");
> > +		ret = -EBUSY;
> > +	}
> > +
> > +	vq = v4l2_m2m_get_vq(ctx->m2m_ctx, f->type);
> > +
> > +	q_data = mtk_vdec_get_q_data(ctx, f->type);
> > +	if (!q_data)
> > +		return -EINVAL;
> > +
> > +	q_data->fmt = fmt;
> > +	if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
> > +		q_data->sizeimage[0] = pix_mp->plane_fmt[0].sizeimage;
> > +		pix_mp->plane_fmt[0].bytesperline = 0;
> > +		pix_mp->width = 0;
> > +		pix_mp->height = 0;
> > +
> > +		if (ctx->state == MTK_STATE_FREE) {
> > +			ret = vdec_if_init(ctx, q_data->fmt->fourcc);
> > +			if (ret) {
> > +				mtk_v4l2_err("[%d]: vdec_if_init() fail ret=%d",
> > +					ctx->id, ret);
> > +				return -EINVAL;
> > +			}
> > +			ctx->state = MTK_STATE_INIT;
> > +		}
> > +	}
> > +
> > +	return 0;
> > +}
> > +
> > +static int vidioc_enum_framesizes(struct file *file, void *priv,
> > +				struct v4l2_frmsizeenum *fsize)
> > +{
> > +	int i = 0;
> > +	struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
> > +
> > +	if (fsize->index != 0)
> > +		return -EINVAL;
> > +
> > +	for (i = 0; i < NUM_SUPPORTED_FRAMESIZE; ++i) {
> > +		if (fsize->pixel_format != mtk_vdec_framesizes[i].fourcc)
> > +			continue;
> > +
> > +		fsize->type = V4L2_FRMSIZE_TYPE_STEPWISE;
> > +		fsize->stepwise = mtk_vdec_framesizes[i].stepwise;
> > +		if (!(ctx->dev->dec_capability &
> > +				VCODEC_CAPABILITY_4K_DISABLED)) {
> > +			mtk_v4l2_debug(3, "4K is enabled");
> > +			fsize->stepwise.max_width =
> > +					VCODEC_DEC_4K_CODED_WIDTH;
> > +			fsize->stepwise.max_height =
> > +					VCODEC_DEC_4K_CODED_HEIGHT;
> > +		}
> > +		mtk_v4l2_debug(1, "%x, %d %d %d %d %d %d",
> > +				ctx->dev->dec_capability,
> > +				fsize->stepwise.min_width,
> > +				fsize->stepwise.max_width,
> > +				fsize->stepwise.step_width,
> > +				fsize->stepwise.min_height,
> > +				fsize->stepwise.max_height,
> > +				fsize->stepwise.step_height);
> > +		return 0;
> > +	}
> > +
> > +	return -EINVAL;
> > +}
> > +
> > +static int vidioc_enum_fmt(struct v4l2_fmtdesc *f, bool output_queue)
> > +{
> > +	struct mtk_video_fmt *fmt;
> > +	int i, j = 0;
> > +
> > +	for (i = 0; i < NUM_FORMATS; i++) {
> > +		if (output_queue && (mtk_video_formats[i].type != MTK_FMT_DEC))
> > +			continue;
> > +		if (!output_queue &&
> > +			(mtk_video_formats[i].type != MTK_FMT_FRAME))
> > +			continue;
> > +
> > +		if (j == f->index)
> > +			break;
> > +		++j;
> > +	}
> > +
> > +	if (i == NUM_FORMATS)
> > +		return -EINVAL;
> > +
> > +	fmt = &mtk_video_formats[i];
> > +	f->pixelformat = fmt->fourcc;
> > +
> > +	return 0;
> > +}
> > +
> > +static int vidioc_vdec_enum_fmt_vid_cap_mplane(struct file *file, void *pirv,
> > +					       struct v4l2_fmtdesc *f)
> > +{
> > +	return vidioc_enum_fmt(f, false);
> > +}
> > +
> > +static int vidioc_vdec_enum_fmt_vid_out_mplane(struct file *file, void *priv,
> > +					       struct v4l2_fmtdesc *f)
> > +{
> > +	return vidioc_enum_fmt(f, true);
> > +}
> > +
> > +static int vidioc_vdec_g_fmt(struct file *file, void *priv,
> > +			     struct v4l2_format *f)
> > +{
> > +	struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
> > +	struct v4l2_pix_format_mplane *pix_mp = &f->fmt.pix_mp;
> > +	struct vb2_queue *vq;
> > +	struct mtk_q_data *q_data;
> > +
> > +	vq = v4l2_m2m_get_vq(ctx->m2m_ctx, f->type);
> > +	if (!vq) {
> > +		mtk_v4l2_err("no vb2 queue for type=%d",
> > +			f->type);
> > +		return -EINVAL;
> > +	}
> > +
> > +	q_data = mtk_vdec_get_q_data(ctx, f->type);
> > +
> > +	if ((f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) &&
> > +	    (ctx->state >= MTK_STATE_HEADER)) {
> > +		/* Until STREAMOFF is called on the CAPTURE queue
> > +		 * (acknowledging the event), the driver operates as if
> > +		 * the resolution hasn't changed yet.
> > +		 * So we just return picinfo yet, and update picinfo in
> > +		 * stop_streaming hook function
> > +		 */
> > +		q_data->sizeimage[0] = ctx->picinfo.y_bs_sz +
> > +					ctx->picinfo.y_len_sz;
> > +		q_data->sizeimage[1] = ctx->picinfo.c_bs_sz +
> > +					ctx->picinfo.c_len_sz;
> > +		q_data->bytesperline[0] = ctx->last_decoded_picinfo.buf_w;
> > +		q_data->bytesperline[1] = ctx->last_decoded_picinfo.buf_w;
> > +		q_data->coded_width = ctx->picinfo.buf_w;
> > +		q_data->coded_height = ctx->picinfo.buf_h;
> > +
> > +		/*
> > +		 * Width and height are set to the dimensions
> > +		 * of the movie, the buffer is bigger and
> > +		 * further processing stages should crop to this
> > +		 * rectangle.
> > +		 */
> > +		pix_mp->width = q_data->coded_width;
> > +		pix_mp->height = q_data->coded_height;
> > +		pix_mp->field = V4L2_FIELD_NONE;
> > +
> > +		/*
> > +		 * Set pixelformat to the format in which mt vcodec
> > +		 * outputs the decoded frame
> > +		 */
> > +		pix_mp->num_planes = q_data->fmt->num_planes;
> > +		pix_mp->pixelformat = q_data->fmt->fourcc;
> > +		pix_mp->plane_fmt[0].bytesperline = q_data->bytesperline[0];
> > +		pix_mp->plane_fmt[0].sizeimage = q_data->sizeimage[0];
> > +		pix_mp->plane_fmt[1].bytesperline = q_data->bytesperline[1];
> > +		pix_mp->plane_fmt[1].sizeimage = q_data->sizeimage[1];
> > +
> > +	} else if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
> > +		/*
> > +		 * This is run on OUTPUT
> > +		 * The buffer contains compressed image
> > +		 * so width and height have no meaning.
> > +		 * Assign value here to pass v4l2-compliance test
> > +		 */
> > +		pix_mp->width = q_data->visible_width;
> > +		pix_mp->height = q_data->visible_height;
> > +		pix_mp->field = V4L2_FIELD_NONE;
> > +		pix_mp->plane_fmt[0].bytesperline = q_data->bytesperline[0];
> > +		pix_mp->plane_fmt[0].sizeimage = q_data->sizeimage[0];
> > +		pix_mp->pixelformat = q_data->fmt->fourcc;
> > +		pix_mp->num_planes = q_data->fmt->num_planes;
> > +	} else {
> > +		pix_mp->width = q_data->coded_width;
> > +		pix_mp->height = q_data->coded_height;
> > +		pix_mp->field = V4L2_FIELD_NONE;
> > +		pix_mp->num_planes = q_data->fmt->num_planes;
> > +		pix_mp->pixelformat = q_data->fmt->fourcc;
> > +		pix_mp->plane_fmt[0].bytesperline = q_data->bytesperline[0];
> > +		pix_mp->plane_fmt[0].sizeimage = q_data->sizeimage[0];
> > +		pix_mp->plane_fmt[1].bytesperline = q_data->bytesperline[1];
> > +		pix_mp->plane_fmt[1].sizeimage = q_data->sizeimage[1];
> > +
> > +		mtk_v4l2_debug(1, "[%d] state=%d Format could not be read %d, not ready yet!",
> > +				ctx->id, ctx->state, f->type);
> > +		return -EINVAL;
> > +	}
> > +
> > +	return 0;
> > +}
> > +
> > +static int vb2ops_vdec_queue_setup(struct vb2_queue *vq,
> > +				unsigned int *nbuffers,
> > +				unsigned int *nplanes,
> > +				unsigned int sizes[], void *alloc_ctxs[])
> > +{
> > +	struct mtk_vcodec_ctx *ctx = vb2_get_drv_priv(vq);
> > +	struct mtk_q_data *q_data;
> > +	unsigned int i;
> > +
> > +	q_data = mtk_vdec_get_q_data(ctx, vq->type);
> > +
> > +	if (q_data == NULL) {
> > +		mtk_v4l2_err("vq->type=%d err\n", vq->type);
> > +		return -EINVAL;
> > +	}
> > +
> > +	if (*nplanes) {
> > +		for (i = 0; i < *nplanes; i++) {
> > +			if (sizes[i] < q_data->sizeimage[i])
> > +				return -EINVAL;
> > +			alloc_ctxs[i] = ctx->dev->alloc_ctx;
> > +		}
> > +	} else {
> > +
> > +		if (vq->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
> > +			if (ctx->state < MTK_STATE_HEADER) {
> > +				mtk_v4l2_err("State not ready");
> > +				return -EINVAL;
> > +			}
> > +			/*
> > +			 * Output plane count is 2 - one for Y and one for CbCr
> > +			 */
> > +			*nplanes = 2;
> > +		} else {
> > +			*nplanes = 1;
> > +		}
> > +
> > +		for (i = 0; i < *nplanes; i++) {
> > +			sizes[i] = q_data->sizeimage[i];
> > +			alloc_ctxs[i] = ctx->dev->alloc_ctx;
> > +		}
> > +	}
> > +
> > +	mtk_v4l2_debug(1,
> > +			"[%d]\t type = %d, get %d plane(s), %d buffer(s) of size 0x%x 0x%x ",
> > +			ctx->id, vq->type, *nplanes, *nbuffers,
> > +			sizes[0], sizes[1]);
> > +
> > +	return 0;
> > +}
> > +
> > +static int vb2ops_vdec_buf_prepare(struct vb2_buffer *vb)
> > +{
> > +	struct mtk_vcodec_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue);
> > +	struct mtk_q_data *q_data;
> > +	int i;
> > +
> > +	mtk_v4l2_debug(3, "[%d] (%d) id=%d",
> > +			ctx->id, vb->vb2_queue->type, vb->index);
> > +
> > +	q_data = mtk_vdec_get_q_data(ctx, vb->vb2_queue->type);
> > +
> > +	for (i = 0; i < q_data->fmt->num_planes; i++) {
> > +		if (vb2_plane_size(vb, i) > q_data->sizeimage[i]) {
> 
> Shouldn't this be < instead of > ?
> 
Yes, will fix this in next version.

> > +			mtk_v4l2_err("data will not fit into plane %d (%lu < %d)",
> > +				i, vb2_plane_size(vb, i),
> > +				q_data->sizeimage[i]);
> > +		}
> > +	}
> > +
> > +	return 0;
> > +}
> > +
> > +static void vb2ops_vdec_buf_queue(struct vb2_buffer *vb)
> > +{
> > +	struct mtk_vcodec_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue);
> > +	struct vb2_v4l2_buffer *vb2_v4l2 = container_of(vb,
> > +				struct vb2_v4l2_buffer, vb2_buf);
> > +	struct mtk_video_dec_buf *buf = container_of(vb2_v4l2,
> > +				struct mtk_video_dec_buf, vb);
> > +
> > +	mtk_v4l2_debug(3, "[%d] (%d) id=%d, vb=%p",
> > +			ctx->id, vb->vb2_queue->type,
> > +			vb->index, vb);
> > +	/*
> > +	 * check if this buffer is ready to be used after decode
> > +	 */
> > +	if (vb->vb2_queue->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
> 
> Do the 'else' part first and return there.
> 
> Then this can be shifted one TAB to the left.
> 
Will fix this in next version.

> > +		struct vb2_buffer *src_buf;
> > +		struct mtk_vcodec_mem src_mem;
> > +		bool res_chg = false;
> > +		int ret = 0;
> > +		unsigned int dpbsize = 1;
> > +
> > +		v4l2_m2m_buf_queue(ctx->m2m_ctx, vb2_v4l2);
> > +
> > +		if (ctx->state == MTK_STATE_INIT) {
> 
> Ditto.
> 
Will fix this in next version.

> > +			src_buf = v4l2_m2m_next_src_buf(ctx->m2m_ctx);
> > +			if (!src_buf) {
> > +				mtk_v4l2_err("No src buffer");
> > +				return;
> > +			}
> > +
> > +			src_mem.va	= vb2_plane_vaddr(src_buf, 0);
> > +			src_mem.dma_addr = vb2_dma_contig_plane_dma_addr(
> > +						src_buf,
> > +						0);
> > +			src_mem.size = (size_t)src_buf->planes[0].bytesused;
> > +			mtk_v4l2_debug(2,
> > +					"[%d] buf id=%d va=%p dma=%llx size=%zu",
> > +					ctx->id, src_buf->index,
> > +					src_mem.va, (u64)src_mem.dma_addr,
> > +					src_mem.size);
> > +
> > +			ret = vdec_if_decode(ctx, &src_mem, NULL, &res_chg);
> > +			if (ret) {
> > +				/* fb == NULL menas to parse SPS/PPS header or
> > +				 * resolution info in src_mem. Decode can fail
> > +				 * if there is no SPS header or picture info
> > +				 * in bs
> > +				 */
> > +				src_buf = v4l2_m2m_src_buf_remove(ctx->m2m_ctx);
> > +				v4l2_m2m_buf_done(to_vb2_v4l2_buffer(src_buf),
> > +						VB2_BUF_STATE_DONE);
> > +				mtk_v4l2_debug(0, "[%d] vdec_if_decode() src_buf=%d, size=%zu, fail=%d",
> > +						ctx->id, src_buf->index,
> > +						src_mem.size, ret);
> > +				return;
> > +			}
> > +
> > +			if (vdec_if_get_param(ctx,
> > +						GET_PARAM_PIC_INFO,
> > +						&ctx->picinfo)) {
> > +				mtk_v4l2_err("[%d]Error!! Cannot get param : GET_PARAM_PICTURE_INFO ERR",
> > +						ctx->id);
> > +				return;
> > +			}
> > +
> > +			ctx->last_decoded_picinfo = ctx->picinfo;
> > +			ctx->q_data[MTK_Q_DATA_DST].sizeimage[0] =
> > +						ctx->picinfo.y_bs_sz +
> > +						ctx->picinfo.y_len_sz;
> > +			ctx->q_data[MTK_Q_DATA_DST].bytesperline[0] =
> > +						ctx->picinfo.buf_w;
> > +			ctx->q_data[MTK_Q_DATA_DST].sizeimage[1] =
> > +						ctx->picinfo.c_bs_sz +
> > +						ctx->picinfo.c_len_sz;
> > +			ctx->q_data[MTK_Q_DATA_DST].bytesperline[1] =
> > +						ctx->picinfo.buf_w;
> > +			mtk_v4l2_debug(2, "[%d] vdec_if_init() OK wxh=%dx%d pic wxh=%dx%d sz[0]=0x%x sz[1]=0x%x",
> > +				ctx->id,
> > +				ctx->picinfo.buf_w, ctx->picinfo.buf_h,
> > +				ctx->picinfo.pic_w, ctx->picinfo.pic_h,
> > +				ctx->q_data[MTK_Q_DATA_DST].sizeimage[0],
> > +				ctx->q_data[MTK_Q_DATA_DST].sizeimage[1]);
> > +
> > +			ret = vdec_if_get_param(ctx, GET_PARAM_DPB_SIZE,
> > +						&dpbsize);
> > +			if (dpbsize == 0) {
> > +				mtk_v4l2_err("[%d] vdec_if_get_param()  GET_PARAM_DPB_SIZE fail=%d",
> > +						ctx->id, ret);
> > +			}
> > +			ctx->dpb_size = dpbsize;
> > +			ctx->state = MTK_STATE_HEADER;
> > +			mtk_v4l2_debug(1, "[%d] dpbsize=%d", ctx->id,
> > +					ctx->dpb_size);
> > +		} else {
> > +			mtk_v4l2_debug(3, "[%d] already init driver %d",
> > +					ctx->id, ctx->state);
> > +		}
> > +	} else {
> > +		mutex_lock(&ctx->lock);
> > +		if (buf->used == false) {
> > +			v4l2_m2m_buf_queue(ctx->m2m_ctx,
> > +					to_vb2_v4l2_buffer(vb));
> > +			buf->queued_in_vb2 = true;
> > +			buf->queued_in_v4l2 = true;
> > +			buf->ready_to_display = false;
> > +		} else {
> > +			buf->queued_in_vb2 = false;
> > +			buf->queued_in_v4l2 = true;
> > +			buf->ready_to_display = false;
> > +		}
> > +		mutex_unlock(&ctx->lock);
> > +	}
> > +}
> > +
> > +static void vb2ops_vdec_buf_finish(struct vb2_buffer *vb)
> > +{
> > +	struct mtk_vcodec_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue);
> > +
> > +	if (vb->vb2_queue->type ==
> > +		V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
> 
> Place this on one line.
> 
> Actually, just invert the condition and return. The remainder can then be shifted
> one TAB to the left.
> 
Got it, will fix this in next version.

> > +		struct vb2_v4l2_buffer *vb2_v4l2;
> > +		struct mtk_video_dec_buf *buf;
> > +
> > +		vb2_v4l2 = container_of(vb, struct vb2_v4l2_buffer, vb2_buf);
> > +		buf = container_of(vb2_v4l2, struct mtk_video_dec_buf, vb);
> > +		mutex_lock(&ctx->lock);
> > +		buf->queued_in_v4l2 = false;
> > +		buf->queued_in_vb2 = false;
> > +		mutex_unlock(&ctx->lock);
> > +	}
> > +}
> > +
> > +static int vb2ops_vdec_buf_init(struct vb2_buffer *vb)
> > +{
> > +	struct vb2_v4l2_buffer *vb2_v4l2 = container_of(vb,
> > +					struct vb2_v4l2_buffer, vb2_buf);
> > +	struct mtk_video_dec_buf *buf = container_of(vb2_v4l2,
> > +					struct mtk_video_dec_buf, vb);
> > +
> > +	if (vb->vb2_queue->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
> > +		buf->used = false;
> > +		buf->ready_to_display = false;
> > +		buf->queued_in_v4l2 = false;
> > +	} else {
> > +		buf->lastframe = false;
> > +	}
> > +
> > +	return 0;
> > +}
> > +
> > +static int vb2ops_vdec_start_streaming(struct vb2_queue *q, unsigned int count)
> > +{
> > +	struct mtk_vcodec_ctx *ctx = vb2_get_drv_priv(q);
> > +
> > +	if (ctx->state == MTK_STATE_FLUSH)
> > +		ctx->state = MTK_STATE_HEADER;
> > +
> > +	return 0;
> > +}
> > +
> > +static void vb2ops_vdec_stop_streaming(struct vb2_queue *q)
> > +{
> > +	struct vb2_buffer *src_buf = NULL, *dst_buf = NULL;
> > +	struct mtk_vcodec_ctx *ctx = vb2_get_drv_priv(q);
> > +
> > +	mtk_v4l2_debug(3, "[%d] (%d) state=(%x) ctx->decoded_frame_cnt=%d",
> > +			ctx->id, q->type, ctx->state, ctx->decoded_frame_cnt);
> > +
> > +	if (q->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
> > +		while ((src_buf = v4l2_m2m_src_buf_remove(ctx->m2m_ctx)))
> > +			v4l2_m2m_buf_done(to_vb2_v4l2_buffer(src_buf),
> > +					VB2_BUF_STATE_ERROR);
> > +
> 
> Same trick can be done here.
> 
Will fix this in next version.

> > +	} else if (q->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
> > +		if (ctx->state >= MTK_STATE_HEADER) {
> > +
> > +			/* Until STREAMOFF is called on the CAPTURE queue
> > +			 * (acknowledging the event), the driver operates
> > +			 * as if the resolution hasn't changed yet, i.e.
> > +			 * VIDIOC_G_FMT< etc. return previous resolution.
> > +			 * So we update picinfo here
> > +			 */
> > +			ctx->picinfo = ctx->last_decoded_picinfo;
> > +
> > +			mtk_v4l2_debug(2,
> > +				"[%d]-> new(%d,%d), old(%d,%d), real(%d,%d)",
> > +				ctx->id, ctx->last_decoded_picinfo.pic_w,
> > +				ctx->last_decoded_picinfo.pic_h,
> > +				ctx->picinfo.pic_w, ctx->picinfo.pic_h,
> > +				ctx->last_decoded_picinfo.buf_w,
> > +				ctx->last_decoded_picinfo.buf_h);
> > +
> > +			mtk_vdec_flush_decoder(ctx);
> > +		}
> > +		ctx->state = MTK_STATE_FLUSH;
> > +
> > +		while ((dst_buf = v4l2_m2m_dst_buf_remove(ctx->m2m_ctx))) {
> > +			vb2_set_plane_payload(dst_buf, 0, 0);
> > +			vb2_set_plane_payload(dst_buf, 1, 0);
> > +			v4l2_m2m_buf_done(to_vb2_v4l2_buffer(dst_buf),
> > +					VB2_BUF_STATE_ERROR);
> > +		}
> > +	}
> > +}
> > +
> > +static void m2mops_vdec_device_run(void *priv)
> > +{
> > +	struct mtk_vcodec_ctx *ctx = priv;
> > +	struct mtk_vcodec_dev *dev = ctx->dev;
> > +
> > +	queue_work(dev->decode_workqueue, &ctx->decode_work);
> > +}
> > +
> > +static int m2mops_vdec_job_ready(void *m2m_priv)
> > +{
> > +	struct mtk_vcodec_ctx *ctx = m2m_priv;
> > +
> > +	mtk_v4l2_debug(3, "[%d]", ctx->id);
> > +
> > +	if (ctx->state == MTK_STATE_ABORT)
> > +		return 0;
> > +
> > +	if ((ctx->last_decoded_picinfo.pic_w != ctx->picinfo.pic_w) ||
> > +	    (ctx->last_decoded_picinfo.pic_h != ctx->picinfo.pic_h))
> > +		return 0;
> > +
> > +	if (ctx->state != MTK_STATE_HEADER)
> > +		return 0;
> > +
> > +	return 1;
> > +}
> > +
> > +static void m2mops_vdec_job_abort(void *priv)
> > +{
> > +	struct mtk_vcodec_ctx *ctx = priv;
> > +
> > +	ctx->state = MTK_STATE_ABORT;
> > +}
> > +
> > +static int mtk_vdec_g_v_ctrl(struct v4l2_ctrl *ctrl)
> > +{
> > +	struct mtk_vcodec_ctx *ctx = ctrl_to_ctx(ctrl);
> > +	int ret = 0;
> > +
> > +	switch (ctrl->id) {
> > +	case V4L2_CID_MIN_BUFFERS_FOR_CAPTURE:
> > +		if (ctx->state >= MTK_STATE_HEADER) {
> > +			ctrl->val = ctx->dpb_size;
> > +		} else {
> > +			mtk_v4l2_err("Seqinfo not ready");
> > +			ctrl->val = 1;
> > +			ret = -EAGAIN;
> > +		}
> > +		break;
> > +	default:
> > +		ret = -EINVAL;
> > +	}
> > +	return ret;
> > +}
> > +
> > +static const struct v4l2_ctrl_ops mtk_vcodec_dec_ctrl_ops = {
> > +	.g_volatile_ctrl = mtk_vdec_g_v_ctrl,
> > +};
> > +
> > +int mtk_vcodec_dec_ctrls_setup(struct mtk_vcodec_ctx *ctx)
> > +{
> > +	struct v4l2_ctrl *ctrl;
> > +
> > +	v4l2_ctrl_handler_init(&ctx->ctrl_hdl, 1);
> > +
> > +	ctrl = v4l2_ctrl_new_std(&ctx->ctrl_hdl,
> > +				&mtk_vcodec_dec_ctrl_ops,
> > +				V4L2_CID_MIN_BUFFERS_FOR_CAPTURE,
> > +				1, 32, 1, 1);
> > +	ctrl->flags |= V4L2_CTRL_FLAG_VOLATILE;
> > +
> > +	if (ctx->ctrl_hdl.error) {
> > +		mtk_v4l2_err("Adding control failed %d",
> > +				ctx->ctrl_hdl.error);
> > +		return ctx->ctrl_hdl.error;
> > +	}
> > +
> > +	v4l2_ctrl_handler_setup(&ctx->ctrl_hdl);
> > +	return 0;
> > +}
> > +
> > +static void m2mops_vdec_lock(void *m2m_priv)
> > +{
> > +	struct mtk_vcodec_ctx *ctx = m2m_priv;
> > +
> > +	mtk_v4l2_debug(3, "[%d]", ctx->id);
> > +	mutex_lock(&ctx->dev->dev_mutex);
> > +}
> > +
> > +static void m2mops_vdec_unlock(void *m2m_priv)
> > +{
> > +	struct mtk_vcodec_ctx *ctx = m2m_priv;
> > +
> > +	mtk_v4l2_debug(3, "[%d]", ctx->id);
> > +	mutex_unlock(&ctx->dev->dev_mutex);
> > +}
> > +
> > +const struct v4l2_m2m_ops mtk_vdec_m2m_ops = {
> > +	.device_run	= m2mops_vdec_device_run,
> > +	.job_ready	= m2mops_vdec_job_ready,
> > +	.job_abort	= m2mops_vdec_job_abort,
> > +	.lock		= m2mops_vdec_lock,
> > +	.unlock		= m2mops_vdec_unlock,
> > +};
> > +
> > +static const struct vb2_ops mtk_vdec_vb2_ops = {
> > +	.queue_setup	= vb2ops_vdec_queue_setup,
> > +	.buf_prepare	= vb2ops_vdec_buf_prepare,
> > +	.buf_queue	= vb2ops_vdec_buf_queue,
> > +	.wait_prepare	= vb2_ops_wait_prepare,
> > +	.wait_finish	= vb2_ops_wait_finish,
> > +	.buf_init	= vb2ops_vdec_buf_init,
> > +	.buf_finish	= vb2ops_vdec_buf_finish,
> > +	.start_streaming	= vb2ops_vdec_start_streaming,
> > +	.stop_streaming	= vb2ops_vdec_stop_streaming,
> > +};
> > +
> > +const struct v4l2_ioctl_ops mtk_vdec_ioctl_ops = {
> > +	.vidioc_streamon	= v4l2_m2m_ioctl_streamon,
> > +	.vidioc_streamoff	= v4l2_m2m_ioctl_streamoff,
> > +	.vidioc_reqbufs		= v4l2_m2m_ioctl_reqbufs,
> > +	.vidioc_querybuf	= v4l2_m2m_ioctl_querybuf,
> > +	.vidioc_expbuf		= v4l2_m2m_ioctl_expbuf,
> > +
> > +	.vidioc_qbuf		= vidioc_vdec_qbuf,
> > +	.vidioc_dqbuf		= vidioc_vdec_dqbuf,
> > +
> > +	.vidioc_try_fmt_vid_cap_mplane	= vidioc_try_fmt_vid_cap_mplane,
> > +	.vidioc_try_fmt_vid_out_mplane	= vidioc_try_fmt_vid_out_mplane,
> > +
> > +	.vidioc_s_fmt_vid_cap_mplane	= vidioc_vdec_s_fmt,
> > +	.vidioc_s_fmt_vid_out_mplane	= vidioc_vdec_s_fmt,
> > +	.vidioc_g_fmt_vid_cap_mplane	= vidioc_vdec_g_fmt,
> > +	.vidioc_g_fmt_vid_out_mplane	= vidioc_vdec_g_fmt,
> > +
> > +	.vidioc_enum_fmt_vid_cap_mplane	= vidioc_vdec_enum_fmt_vid_cap_mplane,
> > +	.vidioc_enum_fmt_vid_out_mplane	= vidioc_vdec_enum_fmt_vid_out_mplane,
> > +	.vidioc_enum_framesizes	= vidioc_enum_framesizes,
> > +
> > +	.vidioc_querycap		= vidioc_vdec_querycap,
> > +	.vidioc_subscribe_event		= vidioc_vdec_subscribe_evt,
> > +	.vidioc_unsubscribe_event	= v4l2_event_unsubscribe,
> > +	.vidioc_g_selection             = vidioc_vdec_g_selection,
> > +};
> > +
> > +int mtk_vcodec_dec_queue_init(void *priv, struct vb2_queue *src_vq,
> > +			   struct vb2_queue *dst_vq)
> > +{
> > +	struct mtk_vcodec_ctx *ctx = priv;
> > +	int ret = 0;
> > +
> > +	mtk_v4l2_debug(3, "[%d]", ctx->id);
> > +
> > +	src_vq->type		= V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE;
> > +	src_vq->io_modes	= VB2_DMABUF | VB2_MMAP;
> > +	src_vq->drv_priv	= ctx;
> > +	src_vq->buf_struct_size = sizeof(struct mtk_video_dec_buf);
> > +	src_vq->ops		= &mtk_vdec_vb2_ops;
> > +	src_vq->mem_ops		= &vb2_dma_contig_memops;
> > +	src_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
> > +	src_vq->lock		= &ctx->dev->dev_mutex;
> > +
> > +	ret = vb2_queue_init(src_vq);
> > +	if (ret) {
> > +		mtk_v4l2_err("Failed to initialize videobuf2 queue(output)");
> > +		return ret;
> > +	}
> > +	dst_vq->type		= V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
> > +	dst_vq->io_modes	= VB2_DMABUF | VB2_MMAP;
> > +	dst_vq->drv_priv	= ctx;
> > +	dst_vq->buf_struct_size = sizeof(struct mtk_video_dec_buf);
> > +	dst_vq->ops		= &mtk_vdec_vb2_ops;
> > +	dst_vq->mem_ops		= &vb2_dma_contig_memops;
> > +	dst_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
> > +	dst_vq->lock		= &ctx->dev->dev_mutex;
> > +
> > +	ret = vb2_queue_init(dst_vq);
> > +	if (ret) {
> > +		vb2_queue_release(src_vq);
> > +		mtk_v4l2_err("Failed to initialize videobuf2 queue(capture)");
> > +	}
> > +
> > +	return ret;
> > +}
> > +
> > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.h
> > new file mode 100644
> > index 0000000..595c244
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.h
> > @@ -0,0 +1,88 @@
> > +/*
> > +* Copyright (c) 2016 MediaTek Inc.
> > +* Author: PC Chen <pc.chen@mediatek.com>
> > +*         Tiffany Lin <tiffany.lin@mediatek.com>
> > +*
> > +* This program is free software; you can redistribute it and/or modify
> > +* it under the terms of the GNU General Public License version 2 as
> > +* published by the Free Software Foundation.
> > +*
> > +* This program is distributed in the hope that it will be useful,
> > +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> > +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > +* GNU General Public License for more details.
> > +*/
> > +
> > +#ifndef _MTK_VCODEC_DEC_H_
> > +#define _MTK_VCODEC_DEC_H_
> > +
> > +#include <media/videobuf2-core.h>
> > +#include <media/videobuf2-v4l2.h>
> > +
> > +#define VCODEC_CAPABILITY_4K_DISABLED	0x10
> > +#define VCODEC_DEC_4K_CODED_WIDTH	4096U
> > +#define VCODEC_DEC_4K_CODED_HEIGHT	2304U
> > +#define MTK_VDEC_MAX_W	2048U
> > +#define MTK_VDEC_MAX_H	1088U
> > +
> > +#define MTK_VDEC_IRQ_STATUS_DEC_SUCCESS        0x10000
> > +
> > +/**
> > + * struct vdec_fb  - decoder frame buffer
> > + * @base_y	: Y plane memory info
> > + * @base_c	: C plane memory info
> > + * @status      : frame buffer status (vdec_fb_status)
> > + */
> > +struct vdec_fb {
> > +	struct mtk_vcodec_mem	base_y;
> > +	struct mtk_vcodec_mem	base_c;
> > +	unsigned int	status;
> > +};
> > +
> > +/**
> > + * struct mtk_video_dec_buf - Private data related to each VB2 buffer.
> > + * @b:		VB2 buffer
> > + * @list:	link list
> > + * @used:	Capture buffer contain decoded frame data and keep in
> > + *			codec data structure
> > + * @ready_to_display:	Capture buffer not display yet
> > + * @queued_in_vb2:	Capture buffer is queue in vb2
> > + * @queued_in_v4l2:	Capture buffer is in v4l2 driver, but not in vb2
> > + *			queue yet
> > + * @lastframe:		Intput buffer is last buffer - EOS
> > + * @frame_buffer:	Decode status, and buffer information of Capture buffer
> > + *
> > + * Note : These status information help us track and debug buffer state
> > + */
> > +struct mtk_video_dec_buf {
> > +	struct vb2_v4l2_buffer	vb;
> > +	struct list_head	list;
> > +
> > +	bool	used;
> > +	bool	ready_to_display;
> > +	bool	queued_in_vb2;
> > +	bool	queued_in_v4l2;
> > +	bool	lastframe;
> > +	struct vdec_fb	frame_buffer;
> > +};
> > +
> > +extern const struct v4l2_ioctl_ops mtk_vdec_ioctl_ops;
> > +extern const struct v4l2_m2m_ops mtk_vdec_m2m_ops;
> > +
> > +
> > +/*
> > + * mtk_vdec_lock/mtk_vdec_unlock are for ctx instance to
> > + * get/release lock before/after access decoder hw.
> > + * mtk_vdec_lock get decoder hw lock and set curr_ctx
> > + * to ctx instance that get lock
> > + */
> > +void mtk_vdec_unlock(struct mtk_vcodec_ctx *ctx);
> > +void mtk_vdec_lock(struct mtk_vcodec_ctx *ctx);
> > +int mtk_vcodec_dec_queue_init(void *priv, struct vb2_queue *src_vq,
> > +			   struct vb2_queue *dst_vq);
> > +void mtk_vcodec_dec_set_default_params(struct mtk_vcodec_ctx *ctx);
> > +void mtk_vcodec_vdec_release(struct mtk_vcodec_ctx *ctx);
> > +int mtk_vcodec_dec_ctrls_setup(struct mtk_vcodec_ctx *ctx);
> > +
> > +
> > +#endif /* _MTK_VCODEC_DEC_H_ */
> > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c
> > new file mode 100644
> > index 0000000..725d1e5
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c
> > @@ -0,0 +1,408 @@
> > +/*
> > +* Copyright (c) 2016 MediaTek Inc.
> > +* Author: PC Chen <pc.chen@mediatek.com>
> > +*         Tiffany Lin <tiffany.lin@mediatek.com>
> > +*
> > +* This program is free software; you can redistribute it and/or modify
> > +* it under the terms of the GNU General Public License version 2 as
> > +* published by the Free Software Foundation.
> > +*
> > +* This program is distributed in the hope that it will be useful,
> > +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> > +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > +* GNU General Public License for more details.
> > +*/
> > +
> > +#include <linux/slab.h>
> > +#include <linux/interrupt.h>
> > +#include <linux/irq.h>
> > +#include <linux/module.h>
> > +#include <linux/of_device.h>
> > +#include <linux/of.h>
> > +#include <media/v4l2-event.h>
> > +#include <media/v4l2-mem2mem.h>
> > +#include <media/videobuf2-dma-contig.h>
> > +
> > +#include "mtk_vcodec_drv.h"
> > +#include "mtk_vcodec_dec.h"
> > +#include "mtk_vcodec_dec_pm.h"
> > +#include "mtk_vcodec_intr.h"
> > +#include "mtk_vcodec_util.h"
> > +#include "mtk_vpu.h"
> > +
> > +#define VDEC_HW_ACTIVE	0x10
> > +#define VDEC_IRQ_CFG	0x11
> > +#define VDEC_IRQ_CLR	0x10
> > +#define VDEC_IRQ_CFG_REG	0xa4
> > +
> > +module_param(mtk_v4l2_dbg_level, int, S_IRUGO | S_IWUSR);
> > +module_param(mtk_vcodec_dbg, bool, S_IRUGO | S_IWUSR);
> > +
> > +/* Wake up context wait_queue */
> > +static void wake_up_ctx(struct mtk_vcodec_ctx *ctx)
> > +{
> > +	ctx->int_cond = 1;
> > +	wake_up_interruptible(&ctx->queue);
> > +}
> > +
> > +static irqreturn_t mtk_vcodec_dec_irq_handler(int irq, void *priv)
> > +{
> > +	struct mtk_vcodec_dev *dev = priv;
> > +	struct mtk_vcodec_ctx *ctx;
> > +	u32 cg_status = 0;
> > +	unsigned int dec_done_status = 0;
> > +	void __iomem *vdec_misc_addr = dev->reg_base[VDEC_MISC] +
> > +					VDEC_IRQ_CFG_REG;
> > +
> > +	ctx = mtk_vcodec_get_curr_ctx(dev);
> > +
> > +	/* check if HW active or not */
> > +	cg_status = readl(dev->reg_base[0]);
> > +	if ((cg_status & VDEC_HW_ACTIVE) != 0) {
> > +		mtk_v4l2_err("DEC ISR, VDEC active is not 0x0 (0x%08x)",
> > +			     cg_status);
> > +		return IRQ_HANDLED;
> > +	}
> > +
> > +	dec_done_status = readl(vdec_misc_addr);
> > +	ctx->irq_status = dec_done_status;
> > +	if ((dec_done_status & MTK_VDEC_IRQ_STATUS_DEC_SUCCESS) !=
> > +		MTK_VDEC_IRQ_STATUS_DEC_SUCCESS)
> > +		return IRQ_HANDLED;
> > +
> > +	/* clear interrupt */
> > +	writel((readl(vdec_misc_addr) | VDEC_IRQ_CFG),
> > +		dev->reg_base[VDEC_MISC] + VDEC_IRQ_CFG_REG);
> > +	writel((readl(vdec_misc_addr) & ~VDEC_IRQ_CLR),
> > +		dev->reg_base[VDEC_MISC] + VDEC_IRQ_CFG_REG);
> > +
> > +	wake_up_ctx(ctx);
> > +
> > +	mtk_v4l2_debug(3,
> > +			"mtk_vcodec_dec_irq_handler :wake up ctx %d, dec_done_status=%x",
> > +			ctx->id, dec_done_status);
> > +
> > +	return IRQ_HANDLED;
> > +}
> > +
> > +static void mtk_vcodec_dec_reset_handler(void *priv)
> > +{
> > +	struct mtk_vcodec_dev *dev = priv;
> > +	struct mtk_vcodec_ctx *ctx;
> > +
> > +	mtk_v4l2_err("Watchdog timeout!!");
> > +
> > +	mutex_lock(&dev->dev_mutex);
> > +	list_for_each_entry(ctx, &dev->ctx_list, list) {
> > +		ctx->state = MTK_STATE_ABORT;
> > +		mtk_v4l2_debug(0, "[%d] Change to state MTK_STATE_ERROR",
> > +				ctx->id);
> > +	}
> > +	mutex_unlock(&dev->dev_mutex);
> > +}
> > +
> > +static int fops_vcodec_open(struct file *file)
> > +{
> > +	struct mtk_vcodec_dev *dev = video_drvdata(file);
> > +	struct mtk_vcodec_ctx *ctx = NULL;
> > +	int ret = 0;
> > +
> > +	ctx = kzalloc(sizeof(*ctx), GFP_KERNEL);
> > +	if (!ctx)
> > +		return -ENOMEM;
> > +
> > +	mutex_lock(&dev->dev_mutex);
> > +	ctx->id = dev->id_counter++;
> > +	v4l2_fh_init(&ctx->fh, video_devdata(file));
> > +	file->private_data = &ctx->fh;
> > +	v4l2_fh_add(&ctx->fh);
> > +	INIT_LIST_HEAD(&ctx->list);
> > +	ctx->dev = dev;
> > +	init_waitqueue_head(&ctx->queue);
> > +	mutex_init(&ctx->lock);
> > +
> > +	ctx->type = MTK_INST_DECODER;
> > +	ret = mtk_vcodec_dec_ctrls_setup(ctx);
> > +	if (ret) {
> > +		mtk_v4l2_err("Failed to setup mt vcodec controls");
> > +		goto err_ctrls_setup;
> > +	}
> > +	ctx->m2m_ctx = v4l2_m2m_ctx_init(dev->m2m_dev_dec, ctx,
> > +		&mtk_vcodec_dec_queue_init);
> > +	if (IS_ERR((__force void *)ctx->m2m_ctx)) {
> > +		ret = PTR_ERR((__force void *)ctx->m2m_ctx);
> > +		mtk_v4l2_err("Failed to v4l2_m2m_ctx_init() (%d)",
> > +			ret);
> > +		goto err_m2m_ctx_init;
> > +	}
> > +	mtk_vcodec_dec_set_default_params(ctx);
> > +
> > +	if (v4l2_fh_is_singular(&ctx->fh)) {
> > +		mtk_vcodec_dec_pw_on(&dev->pm);
> > +		/*
> > +		 * vpu_load_firmware checks if it was loaded already and
> > +		 * does nothing in that case
> > +		 */
> > +		ret = vpu_load_firmware(dev->vpu_plat_dev);
> > +		if (ret < 0) {
> > +			/*
> > +			 * Return 0 if downloading firmware successfully,
> > +			 * otherwise it is failed
> > +			 */
> > +			mtk_v4l2_err("vpu_load_firmware failed!");
> > +			goto err_load_fw;
> > +		}
> > +
> > +		dev->dec_capability =
> > +			vpu_get_vdec_hw_capa(dev->vpu_plat_dev);
> > +		mtk_v4l2_debug(0, "decoder capability %x", dev->dec_capability);
> > +	}
> > +
> > +	dev->num_instances++;
> 
> This field can be dropped since it is no longer used.
> 
Will remove this in next version.

best regards,
Tiffany

> > +	list_add(&ctx->list, &dev->ctx_list);
> > +
> > +	mutex_unlock(&dev->dev_mutex);
> > +	mtk_v4l2_debug(0, "%s decoder [%d]", dev_name(&dev->plat_dev->dev),
> > +			ctx->id);
> > +	return ret;
> > +
> > +	/* Deinit when failure occurred */
> > +err_load_fw:
> > +	v4l2_m2m_ctx_release(ctx->m2m_ctx);
> > +err_m2m_ctx_init:
> > +	v4l2_ctrl_handler_free(&ctx->ctrl_hdl);
> > +err_ctrls_setup:
> > +	v4l2_fh_del(&ctx->fh);
> > +	v4l2_fh_exit(&ctx->fh);
> > +	kfree(ctx);
> > +	mutex_unlock(&dev->dev_mutex);
> > +
> > +	return ret;
> > +}
> > +
> > +static int fops_vcodec_release(struct file *file)
> > +{
> > +	struct mtk_vcodec_dev *dev = video_drvdata(file);
> > +	struct mtk_vcodec_ctx *ctx = fh_to_ctx(file->private_data);
> > +
> > +	mtk_v4l2_debug(0, "[%d] decoder", ctx->id);
> > +	mutex_lock(&dev->dev_mutex);
> > +
> > +	mtk_vcodec_vdec_release(ctx);
> > +	if (v4l2_fh_is_singular(&ctx->fh))
> > +		mtk_vcodec_dec_pw_off(&dev->pm);
> > +
> > +	v4l2_fh_del(&ctx->fh);
> > +	v4l2_fh_exit(&ctx->fh);
> > +	v4l2_ctrl_handler_free(&ctx->ctrl_hdl);
> > +	v4l2_m2m_ctx_release(ctx->m2m_ctx);
> > +
> > +	list_del_init(&ctx->list);
> > +	dev->num_instances--;
> > +	kfree(ctx);
> > +	mutex_unlock(&dev->dev_mutex);
> > +	return 0;
> > +}
> > +
> > +static const struct v4l2_file_operations mtk_vcodec_fops = {
> > +	.owner		= THIS_MODULE,
> > +	.open		= fops_vcodec_open,
> > +	.release	= fops_vcodec_release,
> > +	.poll		= v4l2_m2m_fop_poll,
> > +	.unlocked_ioctl	= video_ioctl2,
> > +	.mmap		= v4l2_m2m_fop_mmap,
> > +};
> > +
> > +static int mtk_vcodec_probe(struct platform_device *pdev)
> > +{
> > +	struct mtk_vcodec_dev *dev;
> > +	struct video_device *vfd_dec;
> > +	struct resource *res;
> > +	int i, ret;
> > +	DEFINE_DMA_ATTRS(attrs);
> > +
> > +	dev = devm_kzalloc(&pdev->dev, sizeof(*dev), GFP_KERNEL);
> > +	if (!dev)
> > +		return -ENOMEM;
> > +
> > +	INIT_LIST_HEAD(&dev->ctx_list);
> > +	dev->plat_dev = pdev;
> > +
> > +	dev->vpu_plat_dev = vpu_get_plat_device(dev->plat_dev);
> > +	if (dev->vpu_plat_dev == NULL) {
> > +		mtk_v4l2_err("[VPU] vpu device in not ready");
> > +		return -EPROBE_DEFER;
> > +	}
> > +
> > +	vpu_wdt_reg_handler(dev->vpu_plat_dev, mtk_vcodec_dec_reset_handler,
> > +			dev, VPU_RST_DEC);
> > +
> > +	ret = mtk_vcodec_init_dec_pm(dev);
> > +	if (ret < 0) {
> > +		dev_err(&pdev->dev, "Failed to get mt vcodec clock source");
> > +		return ret;
> > +	}
> > +
> > +	for (i = 0; i < NUM_MAX_VDEC_REG_BASE; i++) {
> > +		res = platform_get_resource(pdev, IORESOURCE_MEM, i);
> > +		if (res == NULL) {
> > +			dev_err(&pdev->dev, "get memory resource failed.");
> > +			ret = -ENXIO;
> > +			goto err_res;
> > +		}
> > +		dev->reg_base[i] = devm_ioremap_resource(&pdev->dev, res);
> > +		if (IS_ERR((__force void *)dev->reg_base[i])) {
> > +			dev_err(&pdev->dev,
> > +				"devm_ioremap_resource %d failed.", i);
> > +			ret = PTR_ERR((__force void *)dev->reg_base);
> > +			goto err_res;
> > +		}
> > +		mtk_v4l2_debug(2, "reg[%d] base=%p", i, dev->reg_base[i]);
> > +	}
> > +
> > +	res = platform_get_resource(pdev, IORESOURCE_IRQ, 0);
> > +	if (res == NULL) {
> > +		dev_err(&pdev->dev, "failed to get irq resource");
> > +		ret = -ENOENT;
> > +		goto err_res;
> > +	}
> > +
> > +	dev->dec_irq = platform_get_irq(pdev, 0);
> > +	ret = devm_request_irq(&pdev->dev, dev->dec_irq,
> > +			mtk_vcodec_dec_irq_handler, 0, pdev->name, dev);
> > +	if (ret) {
> > +		dev_err(&pdev->dev, "Failed to install dev->dec_irq %d (%d)",
> > +			dev->dec_irq,
> > +			ret);
> > +		goto err_res;
> > +	}
> > +
> > +	disable_irq(dev->dec_irq);
> > +	mutex_init(&dev->dec_mutex);
> > +	mutex_init(&dev->dev_mutex);
> > +	spin_lock_init(&dev->irqlock);
> > +
> > +	snprintf(dev->v4l2_dev.name, sizeof(dev->v4l2_dev.name), "%s",
> > +		"[/MTK_V4L2_VDEC]");
> > +
> > +	ret = v4l2_device_register(&pdev->dev, &dev->v4l2_dev);
> > +	if (ret) {
> > +		mtk_v4l2_err("v4l2_device_register err=%d", ret);
> > +		goto err_res;
> > +	}
> > +
> > +	init_waitqueue_head(&dev->queue);
> > +
> > +	vfd_dec = video_device_alloc();
> > +	if (!vfd_dec) {
> > +		mtk_v4l2_err("Failed to allocate video device");
> > +		ret = -ENOMEM;
> > +		goto err_dec_alloc;
> > +	}
> > +	vfd_dec->fops		= &mtk_vcodec_fops;
> > +	vfd_dec->ioctl_ops	= &mtk_vdec_ioctl_ops;
> > +	vfd_dec->release	= video_device_release;
> > +	vfd_dec->lock		= &dev->dev_mutex;
> > +	vfd_dec->v4l2_dev	= &dev->v4l2_dev;
> > +	vfd_dec->vfl_dir	= VFL_DIR_M2M;
> > +	vfd_dec->device_caps	= V4L2_CAP_VIDEO_M2M_MPLANE |
> > +			V4L2_CAP_STREAMING;
> > +
> > +	snprintf(vfd_dec->name, sizeof(vfd_dec->name), "%s",
> > +		MTK_VCODEC_DEC_NAME);
> > +	video_set_drvdata(vfd_dec, dev);
> > +	dev->vfd_dec = vfd_dec;
> > +	platform_set_drvdata(pdev, dev);
> > +
> > +	dev->alloc_ctx = vb2_dma_contig_init_ctx(&pdev->dev);
> > +	if (IS_ERR((__force void *)dev->alloc_ctx)) {
> > +		mtk_v4l2_err("Failed to alloc vb2 dma context 0");
> > +		ret = PTR_ERR((__force void *)dev->alloc_ctx);
> > +		goto err_vb2_ctx_init;
> > +	}
> > +
> > +	dev->m2m_dev_dec = v4l2_m2m_init(&mtk_vdec_m2m_ops);
> > +	if (IS_ERR((__force void *)dev->m2m_dev_dec)) {
> > +		mtk_v4l2_err("Failed to init mem2mem dec device");
> > +		ret = PTR_ERR((__force void *)dev->m2m_dev_dec);
> > +		goto err_dec_mem_init;
> > +	}
> > +
> > +	dev->decode_workqueue =
> > +		alloc_ordered_workqueue(MTK_VCODEC_DEC_NAME,
> > +			WQ_MEM_RECLAIM | WQ_FREEZABLE);
> > +	if (!dev->decode_workqueue) {
> > +		mtk_v4l2_err("Failed to create decode workqueue");
> > +		ret = -EINVAL;
> > +		goto err_event_workq;
> > +	}
> > +
> > +	ret = video_register_device(vfd_dec, VFL_TYPE_GRABBER, 0);
> > +	if (ret) {
> > +		mtk_v4l2_err("Failed to register video device");
> > +		goto err_dec_reg;
> > +	}
> > +
> > +	/* Avoid the iommu eat big hunks */
> > +	dma_set_attr(DMA_ATTR_ALLOC_SINGLE_PAGES, &attrs);
> > +
> > +	mtk_v4l2_debug(0, "decoder registered as /dev/video%d",
> > +		vfd_dec->num);
> > +
> > +	return 0;
> > +
> > +err_dec_reg:
> > +	destroy_workqueue(dev->decode_workqueue);
> > +err_event_workq:
> > +	v4l2_m2m_release(dev->m2m_dev_dec);
> > +err_dec_mem_init:
> > +	vb2_dma_contig_cleanup_ctx(dev->alloc_ctx);
> > +err_vb2_ctx_init:
> > +	video_unregister_device(vfd_dec);
> > +err_dec_alloc:
> > +	v4l2_device_unregister(&dev->v4l2_dev);
> > +err_res:
> > +	mtk_vcodec_release_dec_pm(dev);
> > +	return ret;
> > +}
> > +
> > +static const struct of_device_id mtk_vcodec_match[] = {
> > +	{.compatible = "mediatek,mt8173-vcodec-dec",},
> > +	{},
> > +};
> > +
> > +MODULE_DEVICE_TABLE(of, mtk_vcodec_match);
> > +
> > +static int mtk_vcodec_dec_remove(struct platform_device *pdev)
> > +{
> > +	struct mtk_vcodec_dev *dev = platform_get_drvdata(pdev);
> > +
> > +	flush_workqueue(dev->decode_workqueue);
> > +	destroy_workqueue(dev->decode_workqueue);
> > +	if (dev->m2m_dev_dec)
> > +		v4l2_m2m_release(dev->m2m_dev_dec);
> > +	if (dev->alloc_ctx)
> > +		vb2_dma_contig_cleanup_ctx(dev->alloc_ctx);
> > +
> > +	if (dev->vfd_dec)
> > +		video_unregister_device(dev->vfd_dec);
> > +
> > +	v4l2_device_unregister(&dev->v4l2_dev);
> > +	mtk_vcodec_release_dec_pm(dev);
> > +	return 0;
> > +}
> > +
> > +static struct platform_driver mtk_vcodec_dec_driver = {
> > +	.probe	= mtk_vcodec_probe,
> > +	.remove	= mtk_vcodec_dec_remove,
> > +	.driver	= {
> > +		.name	= MTK_VCODEC_DEC_NAME,
> > +		.owner	= THIS_MODULE,
> > +		.of_match_table = mtk_vcodec_match,
> > +	},
> > +};
> > +
> > +module_platform_driver(mtk_vcodec_dec_driver);
> > +
> > +MODULE_LICENSE("GPL v2");
> > +MODULE_DESCRIPTION("Mediatek video codec V4L2 decoder driver");
> > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_pm.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_pm.c
> > new file mode 100644
> > index 0000000..364b756
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_pm.c
> > @@ -0,0 +1,206 @@
> > +/*
> > +* Copyright (c) 2016 MediaTek Inc.
> > +* Author: Tiffany Lin <tiffany.lin@mediatek.com>
> > +*
> > +* This program is free software; you can redistribute it and/or modify
> > +* it under the terms of the GNU General Public License version 2 as
> > +* published by the Free Software Foundation.
> > +*
> > +* This program is distributed in the hope that it will be useful,
> > +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> > +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > +* GNU General Public License for more details.
> > +*/
> > +
> > +#include <linux/clk.h>
> > +#include <linux/of_address.h>
> > +#include <linux/of_platform.h>
> > +#include <linux/pm_runtime.h>
> > +#include <soc/mediatek/smi.h>
> > +
> > +#include "mtk_vcodec_dec_pm.h"
> > +#include "mtk_vcodec_util.h"
> > +#include "mtk_vpu.h"
> > +
> > +int mtk_vcodec_init_dec_pm(struct mtk_vcodec_dev *mtkdev)
> > +{
> > +	struct device_node *node;
> > +	struct platform_device *pdev;
> > +	struct device *dev;
> > +	struct mtk_vcodec_pm *pm;
> > +	int ret = 0;
> > +
> > +	pdev = mtkdev->plat_dev;
> > +	pm = &mtkdev->pm;
> > +	pm->mtkdev = mtkdev;
> > +	dev = &pdev->dev;
> > +	node = of_parse_phandle(pdev->dev.of_node, "mediatek,larb", 0);
> > +	if (!node) {
> > +		mtk_v4l2_err("of_parse_phandle mediatek,larb fail!");
> > +		return -1;
> > +	}
> > +
> > +	pdev = of_find_device_by_node(node);
> > +	if (WARN_ON(!pdev)) {
> > +		of_node_put(node);
> > +		return -1;
> > +	}
> > +	pm->larbvdec = &pdev->dev;
> > +
> > +	pdev = mtkdev->plat_dev;
> > +	pm->dev = &pdev->dev;
> > +
> > +	pm->vcodecpll = devm_clk_get(&pdev->dev, "vcodecpll");
> > +	if (pm->vcodecpll == NULL) {
> > +		mtk_v4l2_err("devm_clk_get vcodecpll fail");
> > +		ret = -1;
> > +	}
> > +
> > +	pm->univpll_d2 = devm_clk_get(&pdev->dev, "univpll_d2");
> > +	if (pm->univpll_d2 == NULL) {
> > +		mtk_v4l2_err("devm_clk_get univpll_d2 fail");
> > +		ret = -1;
> > +	}
> > +
> > +	pm->clk_cci400_sel = devm_clk_get(&pdev->dev, "clk_cci400_sel");
> > +	if (pm->clk_cci400_sel == NULL) {
> > +		mtk_v4l2_err("devm_clk_get clk_cci400_sel fail");
> > +		ret = -1;
> > +	}
> > +
> > +	pm->vdec_sel = devm_clk_get(&pdev->dev, "vdec_sel");
> > +	if (pm->vdec_sel == NULL) {
> > +		mtk_v4l2_err("devm_clk_get vdec_sel fail");
> > +		ret = -1;
> > +	}
> > +
> > +	pm->vdecpll = devm_clk_get(&pdev->dev, "vdecpll");
> > +	if (pm->vdecpll == NULL) {
> > +		mtk_v4l2_err("devm_clk_get vdecpll fail");
> > +		ret = -1;
> > +	}
> > +
> > +	pm->vencpll = devm_clk_get(&pdev->dev, "vencpll");
> > +	if (pm->vencpll == NULL) {
> > +		mtk_v4l2_err("devm_clk_get vencpll fail");
> > +		ret = -1;
> > +	}
> > +
> > +	pm->venc_lt_sel = devm_clk_get(&pdev->dev, "venc_lt_sel");
> > +	if (pm->venc_lt_sel == NULL) {
> > +		mtk_v4l2_err("devm_clk_get venc_lt_sel fail");
> > +		ret = -1;
> > +	}
> > +
> > +	pm->vdec_bus_clk_src = devm_clk_get(&pdev->dev, "vdec_bus_clk_src");
> > +	if (pm->vdec_bus_clk_src == NULL) {
> > +		mtk_v4l2_err("devm_clk_get vdec_bus_clk_src");
> > +		ret = -1;
> > +	}
> > +
> > +	pm_runtime_enable(&pdev->dev);
> > +
> > +	return ret;
> > +}
> > +
> > +void mtk_vcodec_release_dec_pm(struct mtk_vcodec_dev *dev)
> > +{
> > +	pm_runtime_disable(dev->pm.dev);
> > +}
> > +
> > +void mtk_vcodec_dec_pw_on(struct mtk_vcodec_pm *pm)
> > +{
> > +	int ret;
> > +
> > +	ret = pm_runtime_get_sync(pm->dev);
> > +	if (ret)
> > +		mtk_v4l2_err("pm_runtime_get_sync fail %d", ret);
> > +}
> > +
> > +void mtk_vcodec_dec_pw_off(struct mtk_vcodec_pm *pm)
> > +{
> > +	int ret;
> > +
> > +	ret = pm_runtime_put_sync(pm->dev);
> > +	if (ret)
> > +		mtk_v4l2_err("pm_runtime_put_sync fail %d", ret);
> > +}
> > +
> > +void mtk_vcodec_dec_clock_on(struct mtk_vcodec_pm *pm)
> > +{
> > +	int ret;
> > +
> > +	ret = clk_set_rate(pm->vcodecpll, 1482 * 1000000);
> > +	if (ret)
> > +		mtk_v4l2_err("clk_set_rate vcodecpll fail %d", ret);
> > +
> > +	ret = clk_set_rate(pm->vencpll, 800 * 1000000);
> > +	if (ret)
> > +		mtk_v4l2_err("clk_set_rate vencpll fail %d", ret);
> > +
> > +	ret = clk_prepare_enable(pm->vcodecpll);
> > +	if (ret)
> > +		mtk_v4l2_err("clk_prepare_enable vcodecpll fail %d", ret);
> > +
> > +	ret = clk_prepare_enable(pm->vencpll);
> > +	if (ret)
> > +		mtk_v4l2_err("clk_prepare_enable vencpll fail %d", ret);
> > +
> > +	ret = clk_prepare_enable(pm->vdec_bus_clk_src);
> > +	if (ret)
> > +		mtk_v4l2_err("clk_prepare_enable vdec_bus_clk_src fail %d",
> > +				ret);
> > +
> > +	ret = clk_prepare_enable(pm->venc_lt_sel);
> > +	if (ret)
> > +		mtk_v4l2_err("clk_prepare_enable venc_lt_sel fail %d", ret);
> > +
> > +	ret = clk_set_parent(pm->venc_lt_sel, pm->vdec_bus_clk_src);
> > +	if (ret)
> > +		mtk_v4l2_err("clk_set_parent venc_lt_sel vdec_bus_clk_src fail %d",
> > +				ret);
> > +
> > +	ret = clk_prepare_enable(pm->univpll_d2);
> > +	if (ret)
> > +		mtk_v4l2_err("clk_prepare_enable univpll_d2 fail %d", ret);
> > +
> > +	ret = clk_prepare_enable(pm->clk_cci400_sel);
> > +	if (ret)
> > +		mtk_v4l2_err("clk_prepare_enable clk_cci400_sel fail %d", ret);
> > +
> > +	ret = clk_set_parent(pm->clk_cci400_sel, pm->univpll_d2);
> > +	if (ret)
> > +		mtk_v4l2_err("clk_set_parent clk_cci400_sel univpll_d2 fail %d",
> > +				ret);
> > +
> > +	ret = clk_prepare_enable(pm->vdecpll);
> > +	if (ret)
> > +		mtk_v4l2_err("clk_prepare_enable vdecpll fail %d", ret);
> > +
> > +	ret = clk_prepare_enable(pm->vdec_sel);
> > +	if (ret)
> > +		mtk_v4l2_err("clk_prepare_enable vdec_sel fail %d", ret);
> > +
> > +	ret = clk_set_parent(pm->vdec_sel, pm->vdecpll);
> > +	if (ret)
> > +		mtk_v4l2_err("clk_set_parent vdec_sel vdecpll fail %d", ret);
> > +
> > +	ret = mtk_smi_larb_get(pm->larbvdec);
> > +	if (ret)
> > +		mtk_v4l2_err("mtk_smi_larb_get larbvdec fail %d", ret);
> > +
> > +}
> > +
> > +void mtk_vcodec_dec_clock_off(struct mtk_vcodec_pm *pm)
> > +{
> > +	mtk_smi_larb_put(pm->larbvdec);
> > +	clk_disable_unprepare(pm->vdec_sel);
> > +	clk_disable_unprepare(pm->vdecpll);
> > +	clk_disable_unprepare(pm->univpll_d2);
> > +	clk_disable_unprepare(pm->clk_cci400_sel);
> > +	clk_disable_unprepare(pm->venc_lt_sel);
> > +	clk_disable_unprepare(pm->vdec_bus_clk_src);
> > +	clk_disable_unprepare(pm->vencpll);
> > +	clk_disable_unprepare(pm->vcodecpll);
> > +}
> > +
> > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_pm.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_pm.h
> > new file mode 100644
> > index 0000000..b8bf56f
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_pm.h
> > @@ -0,0 +1,28 @@
> > +/*
> > +* Copyright (c) 2016 MediaTek Inc.
> > +* Author: Tiffany Lin <tiffany.lin@mediatek.com>
> > +*
> > +* This program is free software; you can redistribute it and/or modify
> > +* it under the terms of the GNU General Public License version 2 as
> > +* published by the Free Software Foundation.
> > +*
> > +* This program is distributed in the hope that it will be useful,
> > +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> > +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > +* GNU General Public License for more details.
> > +*/
> > +
> > +#ifndef _MTK_VCODEC_DEC_PM_H_
> > +#define _MTK_VCODEC_DEC_PM_H_
> > +
> > +#include "mtk_vcodec_drv.h"
> > +
> > +int mtk_vcodec_init_dec_pm(struct mtk_vcodec_dev *dev);
> > +void mtk_vcodec_release_dec_pm(struct mtk_vcodec_dev *dev);
> > +
> > +void mtk_vcodec_dec_pw_on(struct mtk_vcodec_pm *pm);
> > +void mtk_vcodec_dec_pw_off(struct mtk_vcodec_pm *pm);
> > +void mtk_vcodec_dec_clock_on(struct mtk_vcodec_pm *pm);
> > +void mtk_vcodec_dec_clock_off(struct mtk_vcodec_pm *pm);
> > +
> > +#endif /* _MTK_VCODEC_DEC_PM_H_ */
> > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
> > index 78eee50..a11dfec 100644
> > --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
> > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
> > @@ -26,10 +26,10 @@
> >  #include "mtk_vcodec_util.h"
> >  
> >  #define MTK_VCODEC_DRV_NAME	"mtk_vcodec_drv"
> > +#define MTK_VCODEC_DEC_NAME	"mtk-vcodec-dec"
> >  #define MTK_VCODEC_ENC_NAME	"mtk-vcodec-enc"
> >  #define MTK_PLATFORM_STR	"platform:mt8173"
> >  
> > -
> >  #define MTK_VCODEC_MAX_PLANES	3
> >  #define MTK_V4L2_BENCHMARK	0
> >  #define WAIT_INTR_TIMEOUT_MS	1000
> > @@ -180,6 +180,9 @@ struct mtk_enc_params {
> >   * struct mtk_vcodec_pm - Power management data structure
> >   */
> >  struct mtk_vcodec_pm {
> > +	struct clk	*vdec_bus_clk_src;
> > +	struct clk	*vencpll;
> > +
> >  	struct clk	*vcodecpll;
> >  	struct clk	*univpll_d2;
> >  	struct clk	*clk_cci400_sel;
> > @@ -197,6 +200,32 @@ struct mtk_vcodec_pm {
> >  };
> >  
> >  /**
> > + * struct vdec_pic_info  - picture size information
> > + * @pic_w: picture width
> > + * @pic_h: picture height
> > + * @buf_w: picture buffer width (64 aligned up from pic_w)
> > + * @buf_h: picture buffer heiht (64 aligned up from pic_h)
> > + * @y_bs_sz: Y bitstream size
> > + * @c_bs_sz: CbCr bitstream size
> > + * @y_len_sz: additional size required to store decompress information for y
> > + *		plane
> > + * @c_len_sz: additional size required to store decompress information for cbcr
> > + *		plane
> > + * E.g. suppose picture size is 176x144,
> > + *      buffer size will be aligned to 176x160.
> > + */
> > +struct vdec_pic_info {
> > +	unsigned int pic_w;
> > +	unsigned int pic_h;
> > +	unsigned int buf_w;
> > +	unsigned int buf_h;
> > +	unsigned int y_bs_sz;
> > +	unsigned int c_bs_sz;
> > +	unsigned int y_len_sz;
> > +	unsigned int c_len_sz;
> > +};
> > +
> > +/**
> >   * struct mtk_vcodec_ctx - Context (instance) private data.
> >   *
> >   * @type: type of the instance - decoder or encoder
> > @@ -210,9 +239,12 @@ struct mtk_vcodec_pm {
> >   * @state: state of the context
> >   * @param_change: indicate encode parameter type
> >   * @enc_params: encoding parameters
> > - * @enc_if: hoooked encoder driver interface
> > + * @dec_if: hooked decoder driver interface
> > + * @enc_if: hooked encoder driver interface
> >   * @drv_handle: driver handle for specific decode/encode instance
> >   *
> > + * @picinfo: store picture info after header parsing
> > + * @dpb_size: store dpb count after header parsing
> >   * @int_cond: variable used by the waitqueue
> >   * @int_type: type of the last interrupt
> >   * @queue: waitqueue that can be used to wait for this context to
> > @@ -220,12 +252,16 @@ struct mtk_vcodec_pm {
> >   * @irq_status: irq status
> >   *
> >   * @ctrl_hdl: handler for v4l2 framework
> > + * @decode_work: worker for the decoding
> >   * @encode_work: worker for the encoding
> > + * @last_decoded_picinfo: pic information get from latest decode
> >   *
> >   * @colorspace: enum v4l2_colorspace; supplemental to pixelformat
> >   * @ycbcr_enc: enum v4l2_ycbcr_encoding, Y'CbCr encoding
> >   * @quantization: enum v4l2_quantization, colorspace quantization
> >   * @xfer_func: enum v4l2_xfer_func, colorspace transfer function
> > + * @lock: protect variables accessed by V4L2 threads and worker thread such as
> > + *	  mtk_video_dec_buf.
> >   */
> >  struct mtk_vcodec_ctx {
> >  	enum mtk_instance_type type;
> > @@ -240,21 +276,31 @@ struct mtk_vcodec_ctx {
> >  	enum mtk_encode_param param_change;
> >  	struct mtk_enc_params enc_params;
> >  
> > +	struct vdec_common_if *dec_if;
> >  	struct venc_common_if *enc_if;
> >  	unsigned long drv_handle;
> >  
> > +	struct vdec_pic_info picinfo;
> > +	int dpb_size;
> > +
> >  	int int_cond;
> >  	int int_type;
> >  	wait_queue_head_t queue;
> >  	unsigned int irq_status;
> >  
> >  	struct v4l2_ctrl_handler ctrl_hdl;
> > +	struct work_struct decode_work;
> >  	struct work_struct encode_work;
> > +	struct vdec_pic_info last_decoded_picinfo;
> >  
> >  	enum v4l2_colorspace colorspace;
> >  	enum v4l2_ycbcr_encoding ycbcr_enc;
> >  	enum v4l2_quantization quantization;
> >  	enum v4l2_xfer_func xfer_func;
> > +
> > +	int decoded_frame_cnt;
> > +	struct mutex lock;
> > +
> >  };
> >  
> >  /**
> > @@ -268,25 +314,48 @@ struct mtk_vcodec_ctx {
> >   * @alloc_ctx: VB2 allocator context
> >   *	       (for allocations without kernel mapping).
> >   * @ctx_list: list of struct mtk_vcodec_ctx
> > - * @irqlock: protect data access by irq handler and work thread
> > - * @curr_ctx: The context that is waiting for codec hardware
> > + * @irqlock: protect data access by irq handler and work thread -- curr_ctx
> > + * @curr_ctx: The context that is waiting for codec hardware. Protected by
> > + *		irqlock and dec_mutex or enc_mutex.
> >   *
> >   * @reg_base: Mapped address of MTK Vcodec registers.
> >   *
> > - * @id_counter: used to identify current opened instance
> > + * @id_counter:  An integer id given to the next opened instance
> >   * @num_instances: counter of active MTK Vcodec instances
> >   *
> >   * @encode_workqueue: encode work queue
> >   *
> > + * @v4l2_dev: V4L2 device to register video devices for.
> > + * @vfd_dec: Video device for decoder
> > + * @vfd_enc: Video device for encoder
> > + *
> > + * @m2m_dev_dec: m2m device for decoder
> > + * @m2m_dev_enc: m2m device for encoder
> > + * @plat_dev: platform device
> > + * @vpu_plat_dev: mtk vpu platform device
> > + * @alloc_ctx:VB2 allocator context
> > + *	      (for allocations without kernel mapping).
> > + * @ctx_list: list of struct mtk_vcodec_ctx
> > + * @irqlock: protect data access by irq handler and work thread
> > + * @curr_ctx: The context that is waiting for codec hardware
> > + * @reg_base: Mapped address of MTK Vcodec registers.
> > + *
> > + * @instance_mask: used to mark which index id are used
> > + * @num_instances: counter of active MTK Vcodec instances
> > + *
> > + * @decode_workqueue: decode work queue
> > + * @encode_workqueue: encode work queue
> >   * @int_cond: used to identify interrupt condition happen
> >   * @int_type: used to identify what kind of interrupt condition happen
> >   * @dev_mutex: video_device lock
> >   * @queue: waitqueue for waiting for completion of device commands
> >   *
> > + * @dec_irq: decoder irq resource
> >   * @enc_irq: h264 encoder irq resource
> > - * @enc_lt_irq: vp8 encoder irq resource
> > + * @enc_lt_irq: vp8 encoder lt irq resource
> >   *
> > - * @enc_mutex: encoder hardware lock.
> > + * @dec_mutex: decoder hardware lock
> > + * @enc_mutex: encoder hardware lock
> >   *
> >   * @pm: power management control
> >   * @dec_capability: used to identify decode capability, ex: 4k
> > @@ -294,8 +363,10 @@ struct mtk_vcodec_ctx {
> >   */
> >  struct mtk_vcodec_dev {
> >  	struct v4l2_device v4l2_dev;
> > +	struct video_device *vfd_dec;
> >  	struct video_device *vfd_enc;
> >  
> > +	struct v4l2_m2m_dev *m2m_dev_dec;
> >  	struct v4l2_m2m_dev *m2m_dev_enc;
> >  	struct platform_device *plat_dev;
> >  	struct platform_device *vpu_plat_dev;
> > @@ -308,6 +379,7 @@ struct mtk_vcodec_dev {
> >  	unsigned long id_counter;
> >  	int num_instances;
> >  
> > +	struct workqueue_struct *decode_workqueue;
> >  	struct workqueue_struct *encode_workqueue;
> >  
> >  	int int_cond;
> > @@ -315,9 +387,11 @@ struct mtk_vcodec_dev {
> >  	struct mutex dev_mutex;
> >  	wait_queue_head_t queue;
> >  
> > +	int dec_irq;
> >  	int enc_irq;
> >  	int enc_lt_irq;
> >  
> > +	struct mutex dec_mutex;
> >  	struct mutex enc_mutex;
> >  
> >  	struct mtk_vcodec_pm pm;
> > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.c
> > index 52e7e5c..f0bdb19 100644
> > --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.c
> > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.c
> > @@ -30,8 +30,7 @@ int mtk_vcodec_wait_for_done_ctx(struct mtk_vcodec_ctx  *ctx, int command,
> >  	timeout_jiff = msecs_to_jiffies(timeout_ms);
> >  
> >  	ret = wait_event_interruptible_timeout(*waitqueue,
> > -				(ctx->int_cond &&
> > -				(ctx->int_type == command)),
> > +				ctx->int_cond,
> >  				timeout_jiff);
> >  
> >  	if (!ret) {
> > @@ -52,3 +51,4 @@ int mtk_vcodec_wait_for_done_ctx(struct mtk_vcodec_ctx  *ctx, int command,
> >  	return status;
> >  }
> >  EXPORT_SYMBOL(mtk_vcodec_wait_for_done_ctx);
> > +
> > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.h
> > index 33e890f..fbb8b1e 100644
> > --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.h
> > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.h
> > @@ -16,7 +16,7 @@
> >  #define _MTK_VCODEC_INTR_H_
> >  
> >  #define MTK_INST_IRQ_RECEIVED		0x1
> > -#define MTK_INST_WORK_THREAD_ABORT_DONE	0x2
> > +
> >  
> >  struct mtk_vcodec_ctx;
> >  
> > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c
> > index 5e36513..7617305 100644
> > --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c
> > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c
> > @@ -81,14 +81,35 @@ void mtk_vcodec_mem_free(struct mtk_vcodec_ctx *data,
> >  		return;
> >  	}
> >  
> > -	dma_free_coherent(dev, size, mem->va, mem->dma_addr);
> > -	mem->va = NULL;
> > -	mem->dma_addr = 0;
> > -	mem->size = 0;
> > -
> >  	mtk_v4l2_debug(3, "[%d]  - va      = %p", ctx->id, mem->va);
> >  	mtk_v4l2_debug(3, "[%d]  - dma     = 0x%lx", ctx->id,
> >  		       (unsigned long)mem->dma_addr);
> >  	mtk_v4l2_debug(3, "[%d]    size = 0x%lx", ctx->id, size);
> > +
> > +	dma_free_coherent(dev, size, mem->va, mem->dma_addr);
> > +	mem->va = NULL;
> > +	mem->dma_addr = 0;
> > +	mem->size = 0;
> >  }
> >  EXPORT_SYMBOL(mtk_vcodec_mem_free);
> > +
> > +void mtk_vcodec_set_curr_ctx(struct mtk_vcodec_dev *dev,
> > +	struct mtk_vcodec_ctx *ctx)
> > +{
> > +	unsigned long flags;
> > +
> > +	spin_lock_irqsave(&dev->irqlock, flags);
> > +	dev->curr_ctx = ctx;
> > +	spin_unlock_irqrestore(&dev->irqlock, flags);
> > +}
> > +
> > +struct mtk_vcodec_ctx *mtk_vcodec_get_curr_ctx(struct mtk_vcodec_dev *dev)
> > +{
> > +	unsigned long flags;
> > +	struct mtk_vcodec_ctx *ctx;
> > +
> > +	spin_lock_irqsave(&dev->irqlock, flags);
> > +	ctx = dev->curr_ctx;
> > +	spin_unlock_irqrestore(&dev->irqlock, flags);
> > +	return ctx;
> > +}
> > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
> > index d6345fc..91a0b1a 100644
> > --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
> > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
> > @@ -1,7 +1,7 @@
> >  /*
> >  * Copyright (c) 2016 MediaTek Inc.
> >  * Author: PC Chen <pc.chen@mediatek.com>
> > -*	Tiffany Lin <tiffany.lin@mediatek.com>
> > +*		Tiffany Lin <tiffany.lin@mediatek.com>
> >  *
> >  * This program is free software; you can redistribute it and/or modify
> >  * it under the terms of the GNU General Public License version 2 as
> > @@ -26,6 +26,7 @@ struct mtk_vcodec_mem {
> >  };
> >  
> >  struct mtk_vcodec_ctx;
> > +struct mtk_vcodec_dev;
> >  
> >  extern int mtk_v4l2_dbg_level;
> >  extern bool mtk_vcodec_dbg;
> > @@ -84,4 +85,8 @@ int mtk_vcodec_mem_alloc(struct mtk_vcodec_ctx *data,
> >  				struct mtk_vcodec_mem *mem);
> >  void mtk_vcodec_mem_free(struct mtk_vcodec_ctx *data,
> >  				struct mtk_vcodec_mem *mem);
> > +void mtk_vcodec_set_curr_ctx(struct mtk_vcodec_dev *dev,
> > +	struct mtk_vcodec_ctx *ctx);
> > +struct mtk_vcodec_ctx *mtk_vcodec_get_curr_ctx(struct mtk_vcodec_dev *dev);
> > +
> >  #endif /* _MTK_VCODEC_UTIL_H_ */
> > diff --git a/drivers/media/platform/mtk-vcodec/vdec_drv_base.h b/drivers/media/platform/mtk-vcodec/vdec_drv_base.h
> > new file mode 100644
> > index 0000000..7e4c1a9
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/vdec_drv_base.h
> > @@ -0,0 +1,56 @@
> > +/*
> > + * Copyright (c) 2016 MediaTek Inc.
> > + * Author: PC Chen <pc.chen@mediatek.com>
> > + *
> > + * This program is free software; you can redistribute it and/or modify
> > + * it under the terms of the GNU General Public License version 2 as
> > + * published by the Free Software Foundation.
> > + *
> > + * This program is distributed in the hope that it will be useful,
> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > + * GNU General Public License for more details.
> > + */
> > +
> > +#ifndef _VDEC_DRV_BASE_
> > +#define _VDEC_DRV_BASE_
> > +
> > +#include "mtk_vcodec_drv.h"
> > +
> > +#include "vdec_drv_if.h"
> > +
> > +struct vdec_common_if {
> > +	/**
> > +	 * (*init)() - initialize decode driver
> > +	 * @ctx     : [in] mtk v4l2 context
> > +	 * @h_vdec  : [out] driver handle
> > +	 */
> > +	int (*init)(struct mtk_vcodec_ctx *ctx, unsigned long *h_vdec);
> > +
> > +	/**
> > +	 * (*decode)() - trigger decode
> > +	 * @h_vdec  : [in] driver handle
> > +	 * @bs      : [in] input bitstream
> > +	 * @fb      : [in] frame buffer to store decoded frame
> > +	 * @res_chg : [out] resolution change happen
> > +	 */
> > +	int (*decode)(unsigned long h_vdec, struct mtk_vcodec_mem *bs,
> > +		      struct vdec_fb *fb, bool *res_chg);
> > +
> > +	/**
> > +	 * (*get_param)() - get driver's parameter
> > +	 * @h_vdec : [in] driver handle
> > +	 * @type   : [in] input parameter type
> > +	 * @out    : [out] buffer to store query result
> > +	 */
> > +	int (*get_param)(unsigned long h_vdec, enum vdec_get_param_type type,
> > +			 void *out);
> > +
> > +	/**
> > +	 * (*deinit)() - deinitialize driver.
> > +	 * @h_vdec : [in] driver handle to be deinit
> > +	 */
> > +	void (*deinit)(unsigned long h_vdec);
> > +};
> > +
> > +#endif
> > diff --git a/drivers/media/platform/mtk-vcodec/vdec_drv_if.c b/drivers/media/platform/mtk-vcodec/vdec_drv_if.c
> > new file mode 100644
> > index 0000000..8202b14
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/vdec_drv_if.c
> > @@ -0,0 +1,107 @@
> > +/*
> > + * Copyright (c) 2016 MediaTek Inc.
> > + * Author: PC Chen <pc.chen@mediatek.com>
> > + *         Tiffany Lin <tiffany.lin@mediatek.com>
> > + *
> > + * This program is free software; you can redistribute it and/or modify
> > + * it under the terms of the GNU General Public License version 2 as
> > + * published by the Free Software Foundation.
> > + *
> > + * This program is distributed in the hope that it will be useful,
> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > + * GNU General Public License for more details.
> > + */
> > +
> > +#include <linux/interrupt.h>
> > +#include <linux/kernel.h>
> > +#include <linux/slab.h>
> > +
> > +#include "vdec_drv_base.h"
> > +#include "vdec_drv_if.h"
> > +
> > +#include "mtk_vcodec_dec_pm.h"
> > +#include "mtk_vcodec_util.h"
> > +#include "mtk_vpu.h"
> > +
> > +int vdec_if_init(struct mtk_vcodec_ctx *ctx, unsigned int fourcc)
> > +{
> > +	int ret = 0;
> > +
> > +	switch (fourcc) {
> > +	case V4L2_PIX_FMT_H264:
> > +	case V4L2_PIX_FMT_VP8:
> > +	case V4L2_PIX_FMT_VP9:
> > +	default:
> > +		return -EINVAL;
> > +	}
> > +
> > +	mtk_vdec_lock(ctx);
> > +	mtk_vcodec_dec_clock_on(&ctx->dev->pm);
> > +	ret = ctx->dec_if->init(ctx, &ctx->drv_handle);
> > +	mtk_vcodec_dec_clock_off(&ctx->dev->pm);
> > +	mtk_vdec_unlock(ctx);
> > +
> > +	return ret;
> > +}
> > +
> > +int vdec_if_decode(struct mtk_vcodec_ctx *ctx, struct mtk_vcodec_mem *bs,
> > +		   struct vdec_fb *fb, bool *res_chg)
> > +{
> > +	int ret = 0;
> > +
> > +	if (bs) {
> > +		if ((bs->dma_addr & 63) != 0) {
> > +			mtk_v4l2_err("bs dma_addr should 64 byte align");
> > +			return -EINVAL;
> > +		}
> > +	}
> > +
> > +	if (fb) {
> > +		if (((fb->base_y.dma_addr & 511) != 0) ||
> > +		    ((fb->base_c.dma_addr & 511) != 0)) {
> > +			mtk_v4l2_err("frame buffer dma_addr should 512 byte align");
> > +			return -EINVAL;
> > +		}
> > +	}
> > +
> > +	mtk_vdec_lock(ctx);
> > +
> > +	mtk_vcodec_set_curr_ctx(ctx->dev, ctx);
> > +	mtk_vcodec_dec_clock_on(&ctx->dev->pm);
> > +	enable_irq(ctx->dev->dec_irq);
> > +	ret = ctx->dec_if->decode(ctx->drv_handle, bs, fb, res_chg);
> > +	disable_irq(ctx->dev->dec_irq);
> > +	mtk_vcodec_dec_clock_off(&ctx->dev->pm);
> > +	mtk_vcodec_set_curr_ctx(ctx->dev, NULL);
> > +
> > +	mtk_vdec_unlock(ctx);
> > +
> > +	return ret;
> > +}
> > +
> > +int vdec_if_get_param(struct mtk_vcodec_ctx *ctx, enum vdec_get_param_type type,
> > +		      void *out)
> > +{
> > +	int ret = 0;
> > +
> > +	mtk_vdec_lock(ctx);
> > +	ret = ctx->dec_if->get_param(ctx->drv_handle, type, out);
> > +	mtk_vdec_unlock(ctx);
> > +
> > +	return ret;
> > +}
> > +
> > +void vdec_if_deinit(struct mtk_vcodec_ctx *ctx)
> > +{
> > +	if (ctx->drv_handle == 0)
> > +		return;
> > +
> > +	mtk_vdec_lock(ctx);
> > +	mtk_vcodec_dec_clock_on(&ctx->dev->pm);
> > +	ctx->dec_if->deinit(ctx->drv_handle);
> > +	mtk_vcodec_dec_clock_off(&ctx->dev->pm);
> > +	mtk_vdec_unlock(ctx);
> > +
> > +	ctx->drv_handle = 0;
> > +}
> > diff --git a/drivers/media/platform/mtk-vcodec/vdec_drv_if.h b/drivers/media/platform/mtk-vcodec/vdec_drv_if.h
> > new file mode 100644
> > index 0000000..db6b520
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/vdec_drv_if.h
> > @@ -0,0 +1,101 @@
> > +/*
> > + * Copyright (c) 2016 MediaTek Inc.
> > + * Author: PC Chen <pc.chen@mediatek.com>
> > + *		   Tiffany Lin <tiffany.lin@mediatek.com>
> > + *
> > + * This program is free software; you can redistribute it and/or modify
> > + * it under the terms of the GNU General Public License version 2 as
> > + * published by the Free Software Foundation.
> > + *
> > + * This program is distributed in the hope that it will be useful,
> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > + * GNU General Public License for more details.
> > + */
> > +
> > +#ifndef _VDEC_DRV_IF_H_
> > +#define _VDEC_DRV_IF_H_
> > +
> > +#include "mtk_vcodec_drv.h"
> > +#include "mtk_vcodec_dec.h"
> > +#include "mtk_vcodec_util.h"
> > +
> > +
> > +/**
> > + * struct vdec_fb_status  - decoder frame buffer status
> > + * @FB_ST_NORMAL	: initial state
> > + * @FB_ST_DISPLAY	: frmae buffer is ready to be displayed
> > + * @FB_ST_FREE		: frame buffer is not used by decoder any more
> > + */
> > +enum vdec_fb_status {
> > +	FB_ST_NORMAL		= 0,
> > +	FB_ST_DISPLAY		= (1 << 0),
> > +	FB_ST_FREE		= (1 << 1)
> > +};
> > +
> > +/* For GET_PARAM_DISP_FRAME_BUFFER and GET_PARAM_FREE_FRAME_BUFFER,
> > + * the caller does not own the returned buffer. The buffer will not be
> > + *				released before vdec_if_deinit.
> > + * GET_PARAM_DISP_FRAME_BUFFER	: get next displayable frame buffer,
> > + *				struct vdec_fb**
> > + * GET_PARAM_FREE_FRAME_BUFFER	: get non-referenced framebuffer, vdec_fb**
> > + * GET_PARAM_PIC_INFO		: get picture info, struct vdec_pic_info*
> > + * GET_PARAM_CROP_INFO		: get crop info, struct v4l2_crop*
> > + * GET_PARAM_DPB_SIZE		: get dpb size, unsigned int*
> > + */
> > +enum vdec_get_param_type {
> > +	GET_PARAM_DISP_FRAME_BUFFER,
> > +	GET_PARAM_FREE_FRAME_BUFFER,
> > +	GET_PARAM_PIC_INFO,
> > +	GET_PARAM_CROP_INFO,
> > +	GET_PARAM_DPB_SIZE
> > +};
> > +
> > +/**
> > + * struct vdec_fb_node  - decoder frame buffer node
> > + * @list	: list to hold this node
> > + * @fb	: point to frame buffer (vdec_fb), fb could point to frame buffer and
> > + *	working buffer this is for maintain buffers in different state
> > + */
> > +struct vdec_fb_node {
> > +	struct list_head list;
> > +	struct vdec_fb *fb;
> > +};
> > +
> > +/**
> > + * vdec_if_init() - initialize decode driver
> > + * @ctx	: [in] v4l2 context
> > + * @fourcc	: [in] video format fourcc, V4L2_PIX_FMT_H264/VP8/VP9..
> > + */
> > +int vdec_if_init(struct mtk_vcodec_ctx *ctx, unsigned int fourcc);
> > +
> > +/**
> > + * vdec_if_deinit() - deinitialize decode driver
> > + * @ctx	: [in] v4l2 context
> > + *
> > + */
> > +void vdec_if_deinit(struct mtk_vcodec_ctx *ctx);
> > +
> > +/**
> > + * vdec_if_decode() - trigger decode
> > + * @ctx	: [in] v4l2 context
> > + * @bs	: [in] input bitstream
> > + * @fb	: [in] frame buffer to store decoded frame, when null menas parse
> > + *	header only
> > + * @res_chg	: [out] resolution change happens if current bs have different
> > + *	picture width/height
> > + * Note: To flush the decoder when reaching EOF, set input bitstream as NULL.
> > + */
> > +int vdec_if_decode(struct mtk_vcodec_ctx *ctx, struct mtk_vcodec_mem *bs,
> > +		   struct vdec_fb *fb, bool *res_chg);
> > +
> > +/**
> > + * vdec_if_get_param() - get driver's parameter
> > + * @ctx	: [in] v4l2 context
> > + * @type	: [in] input parameter type
> > + * @out	: [out] buffer to store query result
> > + */
> > +int vdec_if_get_param(struct mtk_vcodec_ctx *ctx, enum vdec_get_param_type type,
> > +		      void *out);
> > +
> > +#endif
> > diff --git a/drivers/media/platform/mtk-vcodec/vdec_ipi_msg.h b/drivers/media/platform/mtk-vcodec/vdec_ipi_msg.h
> > new file mode 100644
> > index 0000000..5a8a629
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/vdec_ipi_msg.h
> > @@ -0,0 +1,103 @@
> > +/*
> > + * Copyright (c) 2016 MediaTek Inc.
> > + * Author: PC Chen <pc.chen@mediatek.com>
> > + *
> > + * This program is free software; you can redistribute it and/or
> > + * modify
> > + * it under the terms of the GNU General Public License version 2 as
> > + * published by the Free Software Foundation.
> > + *
> > + * This program is distributed in the hope that it will be useful,
> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > + * GNU General Public License for more details.
> > + */
> > +
> > +#ifndef _VDEC_IPI_MSG_H_
> > +#define _VDEC_IPI_MSG_H_
> > +
> > +/**
> > + * enum vdec_ipi_msgid - message id between AP and VPU
> > + * @AP_IPIMSG_XXX	: AP to VPU cmd message id
> > + * @VPU_IPIMSG_XXX_ACK	: VPU ack AP cmd message id
> > + */
> > +enum vdec_ipi_msgid {
> > +	AP_IPIMSG_DEC_INIT = 0xA000,
> > +	AP_IPIMSG_DEC_START = 0xA001,
> > +	AP_IPIMSG_DEC_END = 0xA002,
> > +	AP_IPIMSG_DEC_DEINIT = 0xA003,
> > +	AP_IPIMSG_DEC_RESET = 0xA004,
> > +
> > +	VPU_IPIMSG_DEC_INIT_ACK = 0xB000,
> > +	VPU_IPIMSG_DEC_START_ACK = 0xB001,
> > +	VPU_IPIMSG_DEC_END_ACK = 0xB002,
> > +	VPU_IPIMSG_DEC_DEINIT_ACK = 0xB003,
> > +	VPU_IPIMSG_DEC_RESET_ACK = 0xB004,
> > +};
> > +
> > +/**
> > + * struct vdec_ap_ipi_cmd - generic AP to VPU ipi command format
> > + * @msg_id	: vdec_ipi_msgid
> > + * @vpu_inst_addr	: VPU decoder instance address
> > + */
> > +struct vdec_ap_ipi_cmd {
> > +	uint32_t msg_id;
> > +	uint32_t vpu_inst_addr;
> > +};
> > +
> > +/**
> > + * struct vdec_vpu_ipi_ack - generic VPU to AP ipi command format
> > + * @msg_id	: vdec_ipi_msgid
> > + * @status	: VPU exeuction result
> > + * @ap_inst_addr	: AP video decoder instance address
> > + */
> > +struct vdec_vpu_ipi_ack {
> > +	uint32_t msg_id;
> > +	int32_t status;
> > +	uint64_t ap_inst_addr;
> > +};
> > +
> > +/**
> > + * struct vdec_ap_ipi_init - for AP_IPIMSG_DEC_INIT
> > + * @msg_id	: AP_IPIMSG_DEC_INIT
> > + * @reserved	: Reserved field
> > + * @ap_inst_addr	: AP video decoder instance address
> > + */
> > +struct vdec_ap_ipi_init {
> > +	uint32_t msg_id;
> > +	uint32_t reserved;
> > +	uint64_t ap_inst_addr;
> > +};
> > +
> > +/**
> > + * struct vdec_ap_ipi_dec_start - for AP_IPIMSG_DEC_START
> > + * @msg_id	: AP_IPIMSG_DEC_START
> > + * @vpu_inst_addr	: VPU decoder instance address
> > + * @data	: Header info
> > + *	H264 decoder [0]:buf_sz [1]:nal_start
> > + *	VP8 decoder  [0]:width/height
> > + *	VP9 decoder  [0]:profile, [1][2] width/height
> > + * @reserved	: Reserved field
> > + */
> > +struct vdec_ap_ipi_dec_start {
> > +	uint32_t msg_id;
> > +	uint32_t vpu_inst_addr;
> > +	uint32_t data[3];
> > +	uint32_t reserved;
> > +};
> > +
> > +/**
> > + * struct vdec_vpu_ipi_init_ack - for VPU_IPIMSG_DEC_INIT_ACK
> > + * @msg_id	: VPU_IPIMSG_DEC_INIT_ACK
> > + * @status	: VPU exeuction result
> > + * @ap_inst_addr	: AP vcodec_vpu_inst instance address
> > + * @vpu_inst_addr	: VPU decoder instance address
> > + */
> > +struct vdec_vpu_ipi_init_ack {
> > +	uint32_t msg_id;
> > +	int32_t status;
> > +	uint64_t ap_inst_addr;
> > +	uint32_t vpu_inst_addr;
> > +};
> > +
> > +#endif
> > 
> 
> Regards,
> 
> 	Hans

[PATCH v3 5/9] vcodec: mediatek: Add Mediatek V4L2 Video Decoder Driver

[Hans Verkuil]

Hi Tiffany,

On 08/01/2016 11:38 AM, Tiffany Lin wrote:
> Hi Hans,

<snip>

>>> +static int vidioc_vdec_g_selection(struct file *file, void *priv,
>>> +			struct v4l2_selection *s)
>>> +{
>>> +	struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
>>> +
>>> +	if (V4L2_TYPE_IS_OUTPUT(s->type))
>>> +		return -EINVAL;
>>> +
>>> +	if (s->target != V4L2_SEL_TGT_CROP)
>>> +		return -EINVAL;
>>
>> How does the cropping rectangle relate to the format size? There is no s_selection,
>> so this doesn't make sense.
>>
> I want to return encoded crop information or real display region that
> display driver could know where is padding region.

Sorry, I don't understand this.

This is a decoder, so based on the bitstream it decodes to a certain width
and height. I assume that is what you refer to as coded_width and coded_height?

If so, then what is the 'real display region' and how does it relate to the
coded width/height?

This is probably a terminology issue but I need to understand this before I
can decide what should be done here.

Regards,

	Hans

> User space use s_fmt/g_fmt  to set/get coded_width and coded_height, and
> use g_crop to get real display region.
> That's why I do not add s_selection.
> 
>> Alternatively, it could be that you are really returning V4L2_SEL_TGT_COMPOSE_PADDED.
>>
> 
> V4L2_SEL_TGT_COMPOSE_PADDED means
> "The active area and all padding pixels that are inserted or modified by
> hardware."
> But I just want to return active area to user space.
> 
>>> +
>>> +	if (ctx->state < MTK_STATE_HEADER)
>>> +		return -EINVAL;
>>> +
>>> +	if ((ctx->q_data[MTK_Q_DATA_SRC].fmt->fourcc == V4L2_PIX_FMT_H264) ||
>>> +	    (ctx->q_data[MTK_Q_DATA_SRC].fmt->fourcc == V4L2_PIX_FMT_VP8) ||
>>> +	    (ctx->q_data[MTK_Q_DATA_SRC].fmt->fourcc == V4L2_PIX_FMT_VP9)) {
>>> +
>>> +		if (vdec_if_get_param(ctx, GET_PARAM_CROP_INFO, &(s->r))) {
>>> +			mtk_v4l2_debug(2,
>>> +					"[%d]Error!! Cannot get param : GET_PARAM_CROP_INFO ERR",
>>> +					ctx->id);
>>> +			s->r.left = 0;
>>> +			s->r.top = 0;
>>> +			s->r.width = ctx->picinfo.buf_w;
>>> +			s->r.height = ctx->picinfo.buf_h;
>>> +		}
>>> +		mtk_v4l2_debug(2, "Cropping info: l=%d t=%d w=%d h=%d",
>>> +				s->r.left, s->r.top, s->r.width,
>>> +				s->r.height);
>>> +	} else {
>>> +		s->r.left = 0;
>>> +		s->r.top = 0;
>>> +		s->r.width = ctx->picinfo.pic_w;
>>> +		s->r.height = ctx->picinfo.pic_h;
>>> +		mtk_v4l2_debug(2, "Cropping info: w=%d h=%d fw=%d fh=%d",
>>> +				s->r.width, s->r.height, ctx->picinfo.buf_w,
>>> +				ctx->picinfo.buf_h);
>>> +	}
>>> +	return 0;
>>> +}

[PATCH v3 5/9] vcodec: mediatek: Add Mediatek V4L2 Video Decoder Driver

[Tiffany Lin]

Hi Hans,

On Mon, 2016-08-01 at 12:39 +0200, Hans Verkuil wrote:
> Hi Tiffany,
> 
> On 08/01/2016 11:38 AM, Tiffany Lin wrote:
> > Hi Hans,
> 
> <snip>
> 
> >>> +static int vidioc_vdec_g_selection(struct file *file, void *priv,
> >>> +			struct v4l2_selection *s)
> >>> +{
> >>> +	struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
> >>> +
> >>> +	if (V4L2_TYPE_IS_OUTPUT(s->type))
> >>> +		return -EINVAL;
> >>> +
> >>> +	if (s->target != V4L2_SEL_TGT_CROP)
> >>> +		return -EINVAL;
> >>
> >> How does the cropping rectangle relate to the format size? There is no s_selection,
> >> so this doesn't make sense.
> >>
> > I want to return encoded crop information or real display region that
> > display driver could know where is padding region.
> 
> Sorry, I don't understand this.
> 
> This is a decoder, so based on the bitstream it decodes to a certain width
> and height. I assume that is what you refer to as coded_width and coded_height?
> 
> If so, then what is the 'real display region' and how does it relate to the
> coded width/height?
> 
> This is probably a terminology issue but I need to understand this before I
> can decide what should be done here.
> 
We define coded_width and coded_height as frame buffer width and height.
Coded resolution is stream resolution in pixels aligned to codec format
and hardware requirements.
And we assign bytesperline as coded_width.

g_selection is for getting visible resolution for user space.
visible resolution - stream resolution of the visible picture, in
pixels, to be used for display purposes; must be smaller or equal to
coded resolution;
visible height - height for given visible resolution
visible width - width for given visible resolution

For my understand, g_fmt should return coded width/height in
pix_mp->width, pix_mp->height.
And bytesperline is coded with.
g_selection return visible width/height.
Is this understanding correct?


best regards,
Tiffany
> Regards,
> 
> 	Hans
> 
> > User space use s_fmt/g_fmt  to set/get coded_width and coded_height, and
> > use g_crop to get real display region.
> > That's why I do not add s_selection.
> > 
> >> Alternatively, it could be that you are really returning V4L2_SEL_TGT_COMPOSE_PADDED.
> >>
> > 
> > V4L2_SEL_TGT_COMPOSE_PADDED means
> > "The active area and all padding pixels that are inserted or modified by
> > hardware."
> > But I just want to return active area to user space.
> > 
> >>> +
> >>> +	if (ctx->state < MTK_STATE_HEADER)
> >>> +		return -EINVAL;
> >>> +
> >>> +	if ((ctx->q_data[MTK_Q_DATA_SRC].fmt->fourcc == V4L2_PIX_FMT_H264) ||
> >>> +	    (ctx->q_data[MTK_Q_DATA_SRC].fmt->fourcc == V4L2_PIX_FMT_VP8) ||
> >>> +	    (ctx->q_data[MTK_Q_DATA_SRC].fmt->fourcc == V4L2_PIX_FMT_VP9)) {
> >>> +
> >>> +		if (vdec_if_get_param(ctx, GET_PARAM_CROP_INFO, &(s->r))) {
> >>> +			mtk_v4l2_debug(2,
> >>> +					"[%d]Error!! Cannot get param : GET_PARAM_CROP_INFO ERR",
> >>> +					ctx->id);
> >>> +			s->r.left = 0;
> >>> +			s->r.top = 0;
> >>> +			s->r.width = ctx->picinfo.buf_w;
> >>> +			s->r.height = ctx->picinfo.buf_h;
> >>> +		}
> >>> +		mtk_v4l2_debug(2, "Cropping info: l=%d t=%d w=%d h=%d",
> >>> +				s->r.left, s->r.top, s->r.width,
> >>> +				s->r.height);
> >>> +	} else {
> >>> +		s->r.left = 0;
> >>> +		s->r.top = 0;
> >>> +		s->r.width = ctx->picinfo.pic_w;
> >>> +		s->r.height = ctx->picinfo.pic_h;
> >>> +		mtk_v4l2_debug(2, "Cropping info: w=%d h=%d fw=%d fh=%d",
> >>> +				s->r.width, s->r.height, ctx->picinfo.buf_w,
> >>> +				ctx->picinfo.buf_h);
> >>> +	}
> >>> +	return 0;
> >>> +}

[PATCH v3 3/9] DocBook/v4l: Add compressed video formats used on MT8173 codec driver

[Hans Verkuil]

On 05/30/2016 02:29 PM, Tiffany Lin wrote:
> Add V4L2_PIX_FMT_MT21 documentation
> 
> Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
> ---
>  Documentation/DocBook/media/v4l/pixfmt.xml |    6 ++++++
>  1 file changed, 6 insertions(+)
> 
> diff --git a/Documentation/DocBook/media/v4l/pixfmt.xml b/Documentation/DocBook/media/v4l/pixfmt.xml
> index 5a08aee..d40e0ce 100644
> --- a/Documentation/DocBook/media/v4l/pixfmt.xml
> +++ b/Documentation/DocBook/media/v4l/pixfmt.xml
> @@ -1980,6 +1980,12 @@ array. Anything what's in between the UYVY lines is JPEG data and should be
>  concatenated to form the JPEG stream. </para>
>  </entry>
>  	  </row>
> +	  <row id="V4L2_PIX_FMT_MT21">
> +	    <entry><constant>V4L2_PIX_FMT_MT21</constant></entry>
> +	    <entry>'MT21'</entry>
> +	    <entry>Compressed two-planar YVU420 format used by Mediatek MT8173
> +	    codec driver.</entry>

Can you give a few more details? The encoder driver doesn't seem to produce this
format, so who is creating this? Where is this format documented?

Regards,

	Hans

> +	  </row>
>  	</tbody>
>        </tgroup>
>      </table>
> 

[PATCH v3 3/9] DocBook/v4l: Add compressed video formats used on MT8173 codec driver

[tiffany lin]

Hi Hans,

On Fri, 2016-07-08 at 12:23 +0200, Hans Verkuil wrote:
> On 05/30/2016 02:29 PM, Tiffany Lin wrote:
> > Add V4L2_PIX_FMT_MT21 documentation
> > 
> > Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
> > ---
> >  Documentation/DocBook/media/v4l/pixfmt.xml |    6 ++++++
> >  1 file changed, 6 insertions(+)
> > 
> > diff --git a/Documentation/DocBook/media/v4l/pixfmt.xml b/Documentation/DocBook/media/v4l/pixfmt.xml
> > index 5a08aee..d40e0ce 100644
> > --- a/Documentation/DocBook/media/v4l/pixfmt.xml
> > +++ b/Documentation/DocBook/media/v4l/pixfmt.xml
> > @@ -1980,6 +1980,12 @@ array. Anything what's in between the UYVY lines is JPEG data and should be
> >  concatenated to form the JPEG stream. </para>
> >  </entry>
> >  	  </row>
> > +	  <row id="V4L2_PIX_FMT_MT21">
> > +	    <entry><constant>V4L2_PIX_FMT_MT21</constant></entry>
> > +	    <entry>'MT21'</entry>
> > +	    <entry>Compressed two-planar YVU420 format used by Mediatek MT8173
> > +	    codec driver.</entry>
> 
> Can you give a few more details? The encoder driver doesn't seem to produce this
> format, so who is creating this? Where is this format documented?
> 

It can be as input format for encoder, MDP and display drivers in our
platform.
This private format is only available in our platform.
So I put it in "Reserved Format Identifiers" sections.


best regards,
Tiffany

> Regards,
> 
> 	Hans
> 
> > +	  </row>
> >  	</tbody>
> >        </tgroup>
> >      </table>
> > 

[PATCH v3 3/9] DocBook/v4l: Add compressed video formats used on MT8173 codec driver

[Wu-Cheng Li (李務誠)]

On Mon, Jul 11, 2016 at 10:56 AM, tiffany lin <tiffany.lin@mediatek.com> wrote:
> Hi Hans,
>
> On Fri, 2016-07-08 at 12:23 +0200, Hans Verkuil wrote:
>> On 05/30/2016 02:29 PM, Tiffany Lin wrote:
>> > Add V4L2_PIX_FMT_MT21 documentation
>> >
>> > Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
>> > ---
>> >  Documentation/DocBook/media/v4l/pixfmt.xml |    6 ++++++
>> >  1 file changed, 6 insertions(+)
>> >
>> > diff --git a/Documentation/DocBook/media/v4l/pixfmt.xml b/Documentation/DocBook/media/v4l/pixfmt.xml
>> > index 5a08aee..d40e0ce 100644
>> > --- a/Documentation/DocBook/media/v4l/pixfmt.xml
>> > +++ b/Documentation/DocBook/media/v4l/pixfmt.xml
>> > @@ -1980,6 +1980,12 @@ array. Anything what's in between the UYVY lines is JPEG data and should be
>> >  concatenated to form the JPEG stream. </para>
>> >  </entry>
>> >       </row>
>> > +     <row id="V4L2_PIX_FMT_MT21">
>> > +       <entry><constant>V4L2_PIX_FMT_MT21</constant></entry>
>> > +       <entry>'MT21'</entry>
>> > +       <entry>Compressed two-planar YVU420 format used by Mediatek MT8173
>> > +       codec driver.</entry>
>>
>> Can you give a few more details? The encoder driver doesn't seem to produce this
>> format, so who is creating this? Where is this format documented?
Decoder hardware produces MT21 (compressed). Image processor can
convert it to a format that can be input of display driver. Tiffany.
When do you plan to upstream image processor (mtk-mdp)?
>
> It can be as input format for encoder, MDP and display drivers in our
> platform.
I remember display driver can only accept uncompressed MT21. Right?
Basically V4L2_PIX_FMT_MT21 is compressed and is like an opaque
format. It's not usable until it's decompressed and converted by image
processor.
> This private format is only available in our platform.
> So I put it in "Reserved Format Identifiers" sections.
>
>
> best regards,
> Tiffany
>
>> Regards,
>>
>>       Hans
>>
>> > +     </row>
>> >     </tbody>
>> >        </tgroup>
>> >      </table>
>> >
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-media" in
> the body of a message to majordomo at vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v3 3/9] DocBook/v4l: Add compressed video formats used on MT8173 codec driver

[Nicolas Dufresne]

Le mardi 12 juillet 2016 ? 16:16 +0800, Wu-Cheng Li (???) a ?crit?:
> Decoder hardware produces MT21 (compressed). Image processor can
> convert it to a format that can be input of display driver. Tiffany.
> When do you plan to upstream image processor (mtk-mdp)?
> >
> > It can be as input format for encoder, MDP and display drivers in
> our
> > platform.
> I remember display driver can only accept uncompressed MT21. Right?
> Basically V4L2_PIX_FMT_MT21 is compressed and is like an opaque
> format. It's not usable until it's decompressed and converted by
> image
> processor.

Previously it was described as MediaTek block mode, and now as a
MediaTek compressed format. It makes me think you have no idea what
this pixel format really is. Is that right ?

The main reason why I keep asking, is that we often find similarities
between what vendor like to call their proprietary formats. Doing the
proper research helps not creating a mess like in Android where you
have a lot of formats that all point to the same format. I believe
there was the same concern when Samsung wanted to introduce their Z-
flip-Z NV12 tile format. In the end they simply provided sufficient
documentation so we could document it and implement software converters
for test and validation purpose.

regards,
Nicolas

[PATCH v3 3/9] DocBook/v4l: Add compressed video formats used on MT8173 codec driver

[Nicolas Dufresne]

Le mardi 12 juillet 2016 ? 15:08 -0400, Nicolas Dufresne a ?crit?:
> Le mardi 12 juillet 2016 ? 16:16 +0800, Wu-Cheng Li (???) a ?crit?:
> > Decoder hardware produces MT21 (compressed). Image processor can
> > convert it to a format that can be input of display driver.
> > Tiffany.
> > When do you plan to upstream image processor (mtk-mdp)?
> > > 
> > > It can be as input format for encoder, MDP and display drivers in
> > our
> > > platform.
> > I remember display driver can only accept uncompressed MT21. Right?
> > Basically V4L2_PIX_FMT_MT21 is compressed and is like an opaque
> > format. It's not usable until it's decompressed and converted by
> > image
> > processor.
> 
> Previously it was described as MediaTek block mode, and now as a
> MediaTek compressed format. It makes me think you have no idea what
> this pixel format really is. Is that right ?
> 
> The main reason why I keep asking, is that we often find similarities
> between what vendor like to call their proprietary formats. Doing the
> proper research helps not creating a mess like in Android where you
> have a lot of formats that all point to the same format. I believe
> there was the same concern when Samsung wanted to introduce their Z-
> flip-Z NV12 tile format. In the end they simply provided sufficient
> documentation so we could document it and implement software
> converters
> for test and validation purpose.

Here's the kind of information we want in the documentation.

https://chromium.googlesource.com/chromium/src/media/+/master/base/vide
o_types.h#40

? // MediaTek proprietary format. MT21 is similar to NV21 except the memory
? // layout and pixel layout (swizzles). 12bpp with Y plane followed by a 2x2
? // interleaved VU plane. Each image contains two buffers -- Y plane and VU
? // plane. Two planes can be non-contiguous in memory. The starting addresses
? // of Y plane and VU plane are 4KB alignment.
? // Suppose image dimension is (width, height). For both Y plane and VU plane:
? // Row pitch = ((width+15)/16) * 16.
? // Plane size = Row pitch * (((height+31)/32)*32)

Now obviously this is incomplete, as the swizzling need to be documented of course.

> 
> regards,
> Nicolas

[PATCH v3 3/9] DocBook/v4l: Add compressed video formats used on MT8173 codec driver

[Ian Arkver]

On 12/07/16 20:14, Nicolas Dufresne wrote:
> Le mardi 12 juillet 2016 ? 15:08 -0400, Nicolas Dufresne a ?crit :
>> Le mardi 12 juillet 2016 ? 16:16 +0800, Wu-Cheng Li (???) a ?crit :
>>> Decoder hardware produces MT21 (compressed). Image processor can
>>> convert it to a format that can be input of display driver.
>>> Tiffany.
>>> When do you plan to upstream image processor (mtk-mdp)?
>>>> It can be as input format for encoder, MDP and display drivers in
>>> our
>>>> platform.
>>> I remember display driver can only accept uncompressed MT21. Right?
>>> Basically V4L2_PIX_FMT_MT21 is compressed and is like an opaque
>>> format. It's not usable until it's decompressed and converted by
>>> image
>>> processor.
>> Previously it was described as MediaTek block mode, and now as a
>> MediaTek compressed format. It makes me think you have no idea what
>> this pixel format really is. Is that right ?
>>
>> The main reason why I keep asking, is that we often find similarities
>> between what vendor like to call their proprietary formats. Doing the
>> proper research helps not creating a mess like in Android where you
>> have a lot of formats that all point to the same format. I believe
>> there was the same concern when Samsung wanted to introduce their Z-
>> flip-Z NV12 tile format. In the end they simply provided sufficient
>> documentation so we could document it and implement software
>> converters
>> for test and validation purpose.
> Here's the kind of information we want in the documentation.
>
> https://chromium.googlesource.com/chromium/src/media/+/master/base/vide
> o_types.h#40
>
>    // MediaTek proprietary format. MT21 is similar to NV21 except the memory
>    // layout and pixel layout (swizzles). 12bpp with Y plane followed by a 2x2
>    // interleaved VU plane. Each image contains two buffers -- Y plane and VU
>    // plane. Two planes can be non-contiguous in memory. The starting addresses
>    // of Y plane and VU plane are 4KB alignment.
>    // Suppose image dimension is (width, height). For both Y plane and VU plane:
>    // Row pitch = ((width+15)/16) * 16.
>    // Plane size = Row pitch * (((height+31)/32)*32)
>
> Now obviously this is incomplete, as the swizzling need to be documented of course.
Not sure where that chromium comment came from, but if MT21 really is 
similar to NV21 (a 4:2:0 format) and has 2x2 downsampled chroma, then 
the combined VU plane will be half the size of the Y plane.

Maybe that's not relevant to this discussion.

Regards,
Ian
>
>> regards,
>> Nicolas
> --
> To unsubscribe from this list: send the line "unsubscribe linux-media" in
> the body of a message to majordomo at vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v3 3/9] DocBook/v4l: Add compressed video formats used on MT8173 codec driver

[tiffany lin]

Hi Nicolas,

On Tue, 2016-07-12 at 15:14 -0400, Nicolas Dufresne wrote:
> Le mardi 12 juillet 2016 ? 15:08 -0400, Nicolas Dufresne a ?crit :
> > Le mardi 12 juillet 2016 ? 16:16 +0800, Wu-Cheng Li (???) a ?crit :
> > > Decoder hardware produces MT21 (compressed). Image processor can
> > > convert it to a format that can be input of display driver.
> > > Tiffany.
> > > When do you plan to upstream image processor (mtk-mdp)?
> > > > 
> > > > It can be as input format for encoder, MDP and display drivers in
> > > our
> > > > platform.
> > > I remember display driver can only accept uncompressed MT21. Right?
> > > Basically V4L2_PIX_FMT_MT21 is compressed and is like an opaque
> > > format. It's not usable until it's decompressed and converted by
> > > image
> > > processor.
> > 
> > Previously it was described as MediaTek block mode, and now as a
> > MediaTek compressed format. It makes me think you have no idea what
> > this pixel format really is. Is that right ?
> > 
> > The main reason why I keep asking, is that we often find similarities
> > between what vendor like to call their proprietary formats. Doing the
> > proper research helps not creating a mess like in Android where you
> > have a lot of formats that all point to the same format. I believe
> > there was the same concern when Samsung wanted to introduce their Z-
> > flip-Z NV12 tile format. In the end they simply provided sufficient
> > documentation so we could document it and implement software
> > converters
> > for test and validation purpose.
> 
> Here's the kind of information we want in the documentation.
> 
> https://chromium.googlesource.com/chromium/src/media/+/master/base/vide
> o_types.h#40
> 
>   // MediaTek proprietary format. MT21 is similar to NV21 except the memory
>   // layout and pixel layout (swizzles). 12bpp with Y plane followed by a 2x2
>   // interleaved VU plane. Each image contains two buffers -- Y plane and VU
>   // plane. Two planes can be non-contiguous in memory. The starting addresses
>   // of Y plane and VU plane are 4KB alignment.
>   // Suppose image dimension is (width, height). For both Y plane and VU plane:
>   // Row pitch = ((width+15)/16) * 16.
>   // Plane size = Row pitch * (((height+31)/32)*32)
> 
> Now obviously this is incomplete, as the swizzling need to be documented of course.
> 
Because it's finally a compressed format from our codec hw, we cannot
describe its swizzling.

best regards,
Tiffany

> > 
> > regards,
> > Nicolas

[PATCH v3 3/9] DocBook/v4l: Add compressed video formats used on MT8173 codec driver

[Nicolas Dufresne]

Le mercredi 13 juillet 2016 ? 10:00 +0800, tiffany lin a ?crit?:
> Hi Nicolas,
> 
> On Tue, 2016-07-12 at 15:14 -0400, Nicolas Dufresne wrote:
> > Le mardi 12 juillet 2016 ? 15:08 -0400, Nicolas Dufresne a ?crit :
> > > Le mardi 12 juillet 2016 ? 16:16 +0800, Wu-Cheng Li (???) a ?crit
> :
> > > > Decoder hardware produces MT21 (compressed). Image processor
> can
> > > > convert it to a format that can be input of display driver.
> > > > Tiffany.
> > > > When do you plan to upstream image processor (mtk-mdp)?
> > > > >?
> > > > > It can be as input format for encoder, MDP and display
> drivers in
> > > > our
> > > > > platform.
> > > > I remember display driver can only accept uncompressed MT21.
> Right?
> > > > Basically V4L2_PIX_FMT_MT21 is compressed and is like an opaque
> > > > format. It's not usable until it's decompressed and converted
> by
> > > > image
> > > > processor.
> > >?
> > > Previously it was described as MediaTek block mode, and now as a
> > > MediaTek compressed format. It makes me think you have no idea
> what
> > > this pixel format really is. Is that right ?
> > >?
> > > The main reason why I keep asking, is that we often find
> similarities
> > > between what vendor like to call their proprietary formats. Doing
> the
> > > proper research helps not creating a mess like in Android where
> you
> > > have a lot of formats that all point to the same format. I
> believe
> > > there was the same concern when Samsung wanted to introduce their
> Z-
> > > flip-Z NV12 tile format. In the end they simply provided
> sufficient
> > > documentation so we could document it and implement software
> > > converters
> > > for test and validation purpose.
> >?
> > Here's the kind of information we want in the documentation.
> >?
> > https://chromium.googlesource.com/chromium/src/media/+/master/base/
> vide
> > o_types.h#40
> >?
> >?? // MediaTek proprietary format. MT21 is similar to NV21 except
> the memory
> >?? // layout and pixel layout (swizzles). 12bpp with Y plane
> followed by a 2x2
> >?? // interleaved VU plane. Each image contains two buffers -- Y
> plane and VU
> >?? // plane. Two planes can be non-contiguous in memory. The
> starting addresses
> >?? // of Y plane and VU plane are 4KB alignment.
> >?? // Suppose image dimension is (width, height). For both Y plane
> and VU plane:
> >?? // Row pitch = ((width+15)/16) * 16.
> >?? // Plane size = Row pitch * (((height+31)/32)*32)
> >?
> > Now obviously this is incomplete, as the swizzling need to be
> documented of course.
> >?
> Because it's finally a compressed format from our codec hw, we cannot
> describe its swizzling.

Can you clarify why it cannot be described ? I don't buy any argument
that it's impossible to convert without hardware. Intel did document
their compressed formats.

Debugging an integration with such format that we have literally
decided to no make public is a pain. A lot of decoder got abandoned or
never used because of that.

It would be nice to explain the architecture you are suggesting to make
this decoder useful. How will userspace application be able to use your
decoder. Will the image processor usable outside the display path ?
People might want to do transcoding, or other relatively common task
that does not imply a display.

regards,
Nicolas

p.s.
A small note to Wu-Cheng, the definition of the DRM format in your
Chromium branch is not correct. DRM uses format modifier, such that the
format is the family, in your case it's most likely NV21, and the
modifier is the compression or tiling algorithm in place (or both as
needed).

[PATCH v3 3/9] DocBook/v4l: Add compressed video formats used on MT8173 codec driver

[tiffany lin]

Hi Nicolas,

On Wed, 2016-07-13 at 09:55 -0400, Nicolas Dufresne wrote:
> Le mercredi 13 juillet 2016 ? 10:00 +0800, tiffany lin a ?crit :
> > Hi Nicolas,
> > 
> > On Tue, 2016-07-12 at 15:14 -0400, Nicolas Dufresne wrote:
> > > Le mardi 12 juillet 2016 ? 15:08 -0400, Nicolas Dufresne a ?crit :
> > > > Le mardi 12 juillet 2016 ? 16:16 +0800, Wu-Cheng Li (???) a ?crit
> > :
> > > > > Decoder hardware produces MT21 (compressed). Image processor
> > can
> > > > > convert it to a format that can be input of display driver.
> > > > > Tiffany.
> > > > > When do you plan to upstream image processor (mtk-mdp)?
> > > > > > 
> > > > > > It can be as input format for encoder, MDP and display
> > drivers in
> > > > > our
> > > > > > platform.
> > > > > I remember display driver can only accept uncompressed MT21.
> > Right?
> > > > > Basically V4L2_PIX_FMT_MT21 is compressed and is like an opaque
> > > > > format. It's not usable until it's decompressed and converted
> > by
> > > > > image
> > > > > processor.
> > > > 
> > > > Previously it was described as MediaTek block mode, and now as a
> > > > MediaTek compressed format. It makes me think you have no idea
> > what
> > > > this pixel format really is. Is that right ?
> > > > 
> > > > The main reason why I keep asking, is that we often find
> > similarities
> > > > between what vendor like to call their proprietary formats. Doing
> > the
> > > > proper research helps not creating a mess like in Android where
> > you
> > > > have a lot of formats that all point to the same format. I
> > believe
> > > > there was the same concern when Samsung wanted to introduce their
> > Z-
> > > > flip-Z NV12 tile format. In the end they simply provided
> > sufficient
> > > > documentation so we could document it and implement software
> > > > converters
> > > > for test and validation purpose.
> > > 
> > > Here's the kind of information we want in the documentation.
> > > 
> > > https://chromium.googlesource.com/chromium/src/media/+/master/base/
> > vide
> > > o_types.h#40
> > > 
> > >   // MediaTek proprietary format. MT21 is similar to NV21 except
> > the memory
> > >   // layout and pixel layout (swizzles). 12bpp with Y plane
> > followed by a 2x2
> > >   // interleaved VU plane. Each image contains two buffers -- Y
> > plane and VU
> > >   // plane. Two planes can be non-contiguous in memory. The
> > starting addresses
> > >   // of Y plane and VU plane are 4KB alignment.
> > >   // Suppose image dimension is (width, height). For both Y plane
> > and VU plane:
> > >   // Row pitch = ((width+15)/16) * 16.
> > >   // Plane size = Row pitch * (((height+31)/32)*32)
> > > 
> > > Now obviously this is incomplete, as the swizzling need to be
> > documented of course.
> > > 
> > Because it's finally a compressed format from our codec hw, we cannot
> > describe its swizzling.
> 
> Can you clarify why it cannot be described ? I don't buy any argument
> that it's impossible to convert without hardware. Intel did document
> their compressed formats.
> 
What's the Intel compressed formats? Could you provide link so we could
reference how they provide this information.

> Debugging an integration with such format that we have literally
> decided to no make public is a pain. A lot of decoder got abandoned or
> never used because of that.
> 
> It would be nice to explain the architecture you are suggesting to make
> this decoder useful. How will userspace application be able to use your
> decoder. Will the image processor usable outside the display path ?
> People might want to do transcoding, or other relatively common task
> that does not imply a display.
> 

In MT8173 platform, it has MDP driver, it could convert
V4L2_PIX_FMT_MT21 to V4L2_PIX_FMT_NV12M or V4L2_PIX_FMT_YUV420M.

MDP is stand alone driver, it is usable outside the display path.

NV12M/YUV420M/MT21 -> MDP -> NV12M/YUV420M

NV12M/NV21M/YUV420M/YVU420M -> mt8173 Encoder -> H264/VP8

H264/VP8/VP9 -> mtk8173 Decoder -> MT21

When encode with MT21 source, the pipeline will be:
MT21 -> MDP driver-> NV12M/NV21M/YUV420M/YVU420M -> Encoder -> H264/VP8

When playback, the pipeline will be:
H264/VP8/VP9 -> Decoder driver -> MT21 -> MDP Driver -> DRM


best regards,
Tiffany


> regards,
> Nicolas
> 
> p.s.
> A small note to Wu-Cheng, the definition of the DRM format in your
> Chromium branch is not correct. DRM uses format modifier, such that the
> format is the family, in your case it's most likely NV21, and the
> modifier is the compression or tiling algorithm in place (or both as
> needed).

[PATCH v3 3/9] DocBook/v4l: Add compressed video formats used on MT8173 codec driver

[Wu-Cheng Li (李務誠)]

On Wed, Jul 13, 2016 at 3:14 AM, Nicolas Dufresne
<nicolas.dufresne@gmail.com> wrote:
> Le mardi 12 juillet 2016 ? 15:08 -0400, Nicolas Dufresne a ?crit :
>> Le mardi 12 juillet 2016 ? 16:16 +0800, Wu-Cheng Li (???) a ?crit :
>> > Decoder hardware produces MT21 (compressed). Image processor can
>> > convert it to a format that can be input of display driver.
>> > Tiffany.
>> > When do you plan to upstream image processor (mtk-mdp)?
>> > >
>> > > It can be as input format for encoder, MDP and display drivers in
>> > our
>> > > platform.
>> > I remember display driver can only accept uncompressed MT21. Right?
>> > Basically V4L2_PIX_FMT_MT21 is compressed and is like an opaque
>> > format. It's not usable until it's decompressed and converted by
>> > image
>> > processor.
>>
>> Previously it was described as MediaTek block mode, and now as a
>> MediaTek compressed format. It makes me think you have no idea what
>> this pixel format really is. Is that right ?
>>
>> The main reason why I keep asking, is that we often find similarities
>> between what vendor like to call their proprietary formats. Doing the
>> proper research helps not creating a mess like in Android where you
>> have a lot of formats that all point to the same format. I believe
>> there was the same concern when Samsung wanted to introduce their Z-
>> flip-Z NV12 tile format. In the end they simply provided sufficient
>> documentation so we could document it and implement software
>> converters
>> for test and validation purpose.
>
> Here's the kind of information we want in the documentation.
>
> https://chromium.googlesource.com/chromium/src/media/+/master/base/vide
> o_types.h#40
That is the documentation of decompressed MT21. Originally MT21 was meant
to be a YUV format and we can map it in CPU to use it. The name was changed
to mean a compressed format. The current design is only MTK image processor
can convert it. Software cannot decompress it. I'm not sure if we
should document
the format inside if we cannot decompress in software. For chromium, I'll update
the code to explain MT21 is an opaque compressed format.
>
>   // MediaTek proprietary format. MT21 is similar to NV21 except the memory
>   // layout and pixel layout (swizzles). 12bpp with Y plane followed by a 2x2
>   // interleaved VU plane. Each image contains two buffers -- Y plane and VU
>   // plane. Two planes can be non-contiguous in memory. The starting addresses
>   // of Y plane and VU plane are 4KB alignment.
>   // Suppose image dimension is (width, height). For both Y plane and VU plane:
>   // Row pitch = ((width+15)/16) * 16.
>   // Plane size = Row pitch * (((height+31)/32)*32)
>
> Now obviously this is incomplete, as the swizzling need to be documented of course.
>
>>
>> regards,
>> Nicolas

[PATCH v3 3/9] DocBook/v4l: Add compressed video formats used on MT8173 codec driver

[tiffany lin]

Hi Nicolas,

On Tue, 2016-07-12 at 15:08 -0400, Nicolas Dufresne wrote:
> Le mardi 12 juillet 2016 ? 16:16 +0800, Wu-Cheng Li (???) a ?crit :
> > Decoder hardware produces MT21 (compressed). Image processor can
> > convert it to a format that can be input of display driver. Tiffany.
> > When do you plan to upstream image processor (mtk-mdp)?
> > >
> > > It can be as input format for encoder, MDP and display drivers in
> > our
> > > platform.
> > I remember display driver can only accept uncompressed MT21. Right?
> > Basically V4L2_PIX_FMT_MT21 is compressed and is like an opaque
> > format. It's not usable until it's decompressed and converted by
> > image
> > processor.
> 
> Previously it was described as MediaTek block mode, and now as a
> MediaTek compressed format. It makes me think you have no idea what
> this pixel format really is. Is that right ?
> 
That's not right.
Its a compressed format as I document in "[PATCH v3 3/9] DocBook/v4l:
Add compressed video formats used on MT8173 codec driver."
In MT8173 platform, when using this format, we need Image Processor to
cover it to standard format as wucheng mentioned.
To prevent this ambiguous, I will change it to V4L2_PIX_FMT_M21C, it
means its compressed data. Is it ok?

best regards,
Tiffany

> The main reason why I keep asking, is that we often find similarities
> between what vendor like to call their proprietary formats. Doing the
> proper research helps not creating a mess like in Android where you
> have a lot of formats that all point to the same format. I believe
> there was the same concern when Samsung wanted to introduce their Z-
> flip-Z NV12 tile format. In the end they simply provided sufficient
> documentation so we could document it and implement software converters
> for test and validation purpose.
> 
> regards,
> Nicolas

[PATCH v3 3/9] DocBook/v4l: Add compressed video formats used on MT8173 codec driver

[tiffany lin]

On Tue, 2016-07-12 at 16:16 +0800, Wu-Cheng Li (???) wrote:
> On Mon, Jul 11, 2016 at 10:56 AM, tiffany lin <tiffany.lin@mediatek.com> wrote:
> > Hi Hans,
> >
> > On Fri, 2016-07-08 at 12:23 +0200, Hans Verkuil wrote:
> >> On 05/30/2016 02:29 PM, Tiffany Lin wrote:
> >> > Add V4L2_PIX_FMT_MT21 documentation
> >> >
> >> > Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
> >> > ---
> >> >  Documentation/DocBook/media/v4l/pixfmt.xml |    6 ++++++
> >> >  1 file changed, 6 insertions(+)
> >> >
> >> > diff --git a/Documentation/DocBook/media/v4l/pixfmt.xml b/Documentation/DocBook/media/v4l/pixfmt.xml
> >> > index 5a08aee..d40e0ce 100644
> >> > --- a/Documentation/DocBook/media/v4l/pixfmt.xml
> >> > +++ b/Documentation/DocBook/media/v4l/pixfmt.xml
> >> > @@ -1980,6 +1980,12 @@ array. Anything what's in between the UYVY lines is JPEG data and should be
> >> >  concatenated to form the JPEG stream. </para>
> >> >  </entry>
> >> >       </row>
> >> > +     <row id="V4L2_PIX_FMT_MT21">
> >> > +       <entry><constant>V4L2_PIX_FMT_MT21</constant></entry>
> >> > +       <entry>'MT21'</entry>
> >> > +       <entry>Compressed two-planar YVU420 format used by Mediatek MT8173
> >> > +       codec driver.</entry>
> >>
> >> Can you give a few more details? The encoder driver doesn't seem to produce this
> >> format, so who is creating this? Where is this format documented?
> Decoder hardware produces MT21 (compressed). Image processor can
> convert it to a format that can be input of display driver. Tiffany.
> When do you plan to upstream image processor (mtk-mdp)?
> >
We are working on this. Will upstream soon.

> > It can be as input format for encoder, MDP and display drivers in our
> > platform.
> I remember display driver can only accept uncompressed MT21. Right?
> Basically V4L2_PIX_FMT_MT21 is compressed and is like an opaque
> format. It's not usable until it's decompressed and converted by image
> processor.
That's right in MT8173 platform.

best regards,
Tiffany
> > This private format is only available in our platform.
> > So I put it in "Reserved Format Identifiers" sections.
> >
> >
> > best regards,
> > Tiffany
> >
> >> Regards,
> >>
> >>       Hans
> >>
> >> > +     </row>
> >> >     </tbody>
> >> >        </tgroup>
> >> >      </table>
> >> >
> >
> >
> > --
> > To unsubscribe from this list: send the line "unsubscribe linux-media" in
> > the body of a message to majordomo at vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v3 0/9] Add MT8173 Video Decoder Driver

[Mauro Carvalho Chehab]

Em Mon, 30 May 2016 20:29:14 +0800
Tiffany Lin <tiffany.lin@mediatek.com> escreveu:

> ==============
>  Introduction
> ==============
> 
> The purpose of this series is to add the driver for video codec hw embedded in the Mediatek's MT8173 SoCs.
> Mediatek Video Codec is able to handle video decoding of in a range of formats.
> 
> This patch series add Mediatek block format V4L2_PIX_FMT_MT21, the decoder driver will decoded bitstream to
> V4L2_PIX_FMT_MT21 format.
> 
> This patch series rely on MTK VPU driver in patch series "Add MT8173 Video Encoder Driver and VPU Driver"[1]
> and patch "CHROMIUM: v4l: Add V4L2_PIX_FMT_VP9 definition"[2] for VP9 support.
> Mediatek Video Decoder driver rely on VPU driver to load, communicate with VPU.
> 
> Internally the driver uses videobuf2 framework and MTK IOMMU and MTK SMI both have been merged in v4.6-rc1.
> 
> [1]https://patchwork.linuxtv.org/patch/33734/
> [2]https://chromium-review.googlesource.com/#/c/245241/

Hmm... I'm not seeing the firmware for this driver at the
linux-firmware tree:
	https://git.kernel.org/cgit/linux/kernel/git/firmware/linux-firmware.git/log/

Nor I'm seeing any pull request for them. Did you send it?
I'll only merge the driver upstream after seeing such pull request.

Regards,
Mauro

[PATCH v3 0/9] Add MT8173 Video Decoder Driver

[Hans Verkuil]

On 06/07/2016 11:22 PM, Mauro Carvalho Chehab wrote:
> Em Mon, 30 May 2016 20:29:14 +0800
> Tiffany Lin <tiffany.lin@mediatek.com> escreveu:
>
>> ==============
>>   Introduction
>> ==============
>>
>> The purpose of this series is to add the driver for video codec hw embedded in the Mediatek's MT8173 SoCs.
>> Mediatek Video Codec is able to handle video decoding of in a range of formats.
>>
>> This patch series add Mediatek block format V4L2_PIX_FMT_MT21, the decoder driver will decoded bitstream to
>> V4L2_PIX_FMT_MT21 format.
>>
>> This patch series rely on MTK VPU driver in patch series "Add MT8173 Video Encoder Driver and VPU Driver"[1]
>> and patch "CHROMIUM: v4l: Add V4L2_PIX_FMT_VP9 definition"[2] for VP9 support.
>> Mediatek Video Decoder driver rely on VPU driver to load, communicate with VPU.
>>
>> Internally the driver uses videobuf2 framework and MTK IOMMU and MTK SMI both have been merged in v4.6-rc1.
>>
>> [1]https://patchwork.linuxtv.org/patch/33734/
>> [2]https://chromium-review.googlesource.com/#/c/245241/
>
> Hmm... I'm not seeing the firmware for this driver at the
> linux-firmware tree:
> 	https://git.kernel.org/cgit/linux/kernel/git/firmware/linux-firmware.git/log/
>
> Nor I'm seeing any pull request for them. Did you send it?
> I'll only merge the driver upstream after seeing such pull request.

Mauro, are you confusing the decoder and encoder driver? I haven't thoroughly reviewed the decoder driver
yet, so there is no pull request for the decoder driver.

The only pull request I made was for the encoder driver.

Regards,

	Hans

[PATCH v3 0/9] Add MT8173 Video Decoder Driver

[Wu-Cheng Li (李務誠)]

On Wed, Jun 8, 2016 at 6:13 AM, Hans Verkuil <hansverk@cisco.com> wrote:
>
>
> On 06/07/2016 11:22 PM, Mauro Carvalho Chehab wrote:
>>
>> Em Mon, 30 May 2016 20:29:14 +0800
>> Tiffany Lin <tiffany.lin@mediatek.com> escreveu:
>>
>>> ==============
>>>   Introduction
>>> ==============
>>>
>>> The purpose of this series is to add the driver for video codec hw
>>> embedded in the Mediatek's MT8173 SoCs.
>>> Mediatek Video Codec is able to handle video decoding of in a range of
>>> formats.
>>>
>>> This patch series add Mediatek block format V4L2_PIX_FMT_MT21, the
>>> decoder driver will decoded bitstream to
>>> V4L2_PIX_FMT_MT21 format.
>>>
>>> This patch series rely on MTK VPU driver in patch series "Add MT8173
>>> Video Encoder Driver and VPU Driver"[1]
>>> and patch "CHROMIUM: v4l: Add V4L2_PIX_FMT_VP9 definition"[2] for VP9
>>> support.
>>> Mediatek Video Decoder driver rely on VPU driver to load, communicate
>>> with VPU.
>>>
>>> Internally the driver uses videobuf2 framework and MTK IOMMU and MTK SMI
>>> both have been merged in v4.6-rc1.
>>>
>>> [1]https://patchwork.linuxtv.org/patch/33734/
>>> [2]https://chromium-review.googlesource.com/#/c/245241/
>>
>>
>> Hmm... I'm not seeing the firmware for this driver at the
>> linux-firmware tree:
>>
>> https://git.kernel.org/cgit/linux/kernel/git/firmware/linux-firmware.git/log/
Tiffany. Can you check the license and add the firmware to linux-firmware?

For the information, both encoder and decoder drivers require the
firmware to work.
>>
>> Nor I'm seeing any pull request for them. Did you send it?
>> I'll only merge the driver upstream after seeing such pull request.
>
>
> Mauro, are you confusing the decoder and encoder driver? I haven't
> thoroughly reviewed the decoder driver
> yet, so there is no pull request for the decoder driver.
>
> The only pull request I made was for the encoder driver.
>
> Regards,
>
>         Hans
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-media" in
> the body of a message to majordomo at vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v3 0/9] Add MT8173 Video Decoder Driver

[tiffany lin]

Hi Mauro,


On Wed, 2016-06-08 at 07:13 +0900, Hans Verkuil wrote:
> 
> On 06/07/2016 11:22 PM, Mauro Carvalho Chehab wrote:
> > Em Mon, 30 May 2016 20:29:14 +0800
> > Tiffany Lin <tiffany.lin@mediatek.com> escreveu:
> >
> >> ==============
> >>   Introduction
> >> ==============
> >>
> >> The purpose of this series is to add the driver for video codec hw embedded in the Mediatek's MT8173 SoCs.
> >> Mediatek Video Codec is able to handle video decoding of in a range of formats.
> >>
> >> This patch series add Mediatek block format V4L2_PIX_FMT_MT21, the decoder driver will decoded bitstream to
> >> V4L2_PIX_FMT_MT21 format.
> >>
> >> This patch series rely on MTK VPU driver in patch series "Add MT8173 Video Encoder Driver and VPU Driver"[1]
> >> and patch "CHROMIUM: v4l: Add V4L2_PIX_FMT_VP9 definition"[2] for VP9 support.
> >> Mediatek Video Decoder driver rely on VPU driver to load, communicate with VPU.
> >>
> >> Internally the driver uses videobuf2 framework and MTK IOMMU and MTK SMI both have been merged in v4.6-rc1.
> >>
> >> [1]https://patchwork.linuxtv.org/patch/33734/
> >> [2]https://chromium-review.googlesource.com/#/c/245241/
> >
> > Hmm... I'm not seeing the firmware for this driver at the
> > linux-firmware tree:
> > 	https://git.kernel.org/cgit/linux/kernel/git/firmware/linux-firmware.git/log/
> >
> > Nor I'm seeing any pull request for them. Did you send it?
> > I'll only merge the driver upstream after seeing such pull request.
> 
Sorry, I am not familiar with how to upstream firmware.
Do you mean we need to upstream vpu firmware first before merge encoder
driver upstream?

In
https://git.kernel.org/cgit/linux/kernel/git/firmware/linux-firmware.git/tree/README, it mentions that 
"To submit firmware to this repository, please send either a git binary
diff or preferably a git pull request to: linux-firmware at kernel.org and
also cc: to related mailing lists."

How we made a git pull request to linux-firmware at kernel.org? 
How we find out related mailing lists?

best regards,
Tiffany

> Mauro, are you confusing the decoder and encoder driver? I haven't thoroughly reviewed the decoder driver
> yet, so there is no pull request for the decoder driver.
> 
> The only pull request I made was for the encoder driver.
> 
> Regards,
> 
> 	Hans

[PATCH v3 0/9] Add MT8173 Video Decoder Driver

[Mauro Carvalho Chehab]

Em Tue, 14 Jun 2016 19:08:08 +0800
tiffany lin <tiffany.lin@mediatek.com> escreveu:

> Hi Mauro,
> 
> 
> On Wed, 2016-06-08 at 07:13 +0900, Hans Verkuil wrote:
> > 
> > On 06/07/2016 11:22 PM, Mauro Carvalho Chehab wrote:  
> > > Em Mon, 30 May 2016 20:29:14 +0800
> > > Tiffany Lin <tiffany.lin@mediatek.com> escreveu:
> > >  
> > >> ==============
> > >>   Introduction
> > >> ==============
> > >>
> > >> The purpose of this series is to add the driver for video codec hw embedded in the Mediatek's MT8173 SoCs.
> > >> Mediatek Video Codec is able to handle video decoding of in a range of formats.
> > >>
> > >> This patch series add Mediatek block format V4L2_PIX_FMT_MT21, the decoder driver will decoded bitstream to
> > >> V4L2_PIX_FMT_MT21 format.
> > >>
> > >> This patch series rely on MTK VPU driver in patch series "Add MT8173 Video Encoder Driver and VPU Driver"[1]
> > >> and patch "CHROMIUM: v4l: Add V4L2_PIX_FMT_VP9 definition"[2] for VP9 support.
> > >> Mediatek Video Decoder driver rely on VPU driver to load, communicate with VPU.
> > >>
> > >> Internally the driver uses videobuf2 framework and MTK IOMMU and MTK SMI both have been merged in v4.6-rc1.
> > >>
> > >> [1]https://patchwork.linuxtv.org/patch/33734/
> > >> [2]https://chromium-review.googlesource.com/#/c/245241/  
> > >
> > > Hmm... I'm not seeing the firmware for this driver at the
> > > linux-firmware tree:
> > > 	https://git.kernel.org/cgit/linux/kernel/git/firmware/linux-firmware.git/log/
> > >
> > > Nor I'm seeing any pull request for them. Did you send it?
> > > I'll only merge the driver upstream after seeing such pull request.  
> >   
> Sorry, I am not familiar with how to upstream firmware.
> Do you mean we need to upstream vpu firmware first before merge encoder
> driver upstream?

Please look at this page:
	https://linuxtv.org/wiki/index.php/Development:_How_to_submit_patches#Firmware_submission

The information here can also be useful:
	https://www.kernel.org/doc/readme/firmware-README.AddingFirmware

In summary, you need to provide redistribution rights for the
firmware blob. You can either submit it to me or directly to
linux-firmware. In the latter, please c/c me on such patch.

Thanks!
Mauro

> 
> In
> https://git.kernel.org/cgit/linux/kernel/git/firmware/linux-firmware.git/tree/README, it mentions that 
> "To submit firmware to this repository, please send either a git binary
> diff or preferably a git pull request to: linux-firmware at kernel.org and
> also cc: to related mailing lists."
> 
> How we made a git pull request to linux-firmware at kernel.org? 
> How we find out related mailing lists?
> 
> best regards,
> Tiffany
> 
> > Mauro, are you confusing the decoder and encoder driver? I haven't thoroughly reviewed the decoder driver
> > yet, so there is no pull request for the decoder driver.
> > 
> > The only pull request I made was for the encoder driver.
> > 
> > Regards,
> > 
> > 	Hans  
> 
> 


-- 
Thanks,
Mauro

[PATCH v3 0/9] Add MT8173 Video Decoder Driver

[Hans Verkuil]

On 06/16/2016 12:54 PM, Mauro Carvalho Chehab wrote:
> Em Tue, 14 Jun 2016 19:08:08 +0800
> tiffany lin <tiffany.lin@mediatek.com> escreveu:
> 
>> Hi Mauro,
>>
>>
>> On Wed, 2016-06-08 at 07:13 +0900, Hans Verkuil wrote:
>>>
>>> On 06/07/2016 11:22 PM, Mauro Carvalho Chehab wrote:  
>>>> Em Mon, 30 May 2016 20:29:14 +0800
>>>> Tiffany Lin <tiffany.lin@mediatek.com> escreveu:
>>>>  
>>>>> ==============
>>>>>   Introduction
>>>>> ==============
>>>>>
>>>>> The purpose of this series is to add the driver for video codec hw embedded in the Mediatek's MT8173 SoCs.
>>>>> Mediatek Video Codec is able to handle video decoding of in a range of formats.
>>>>>
>>>>> This patch series add Mediatek block format V4L2_PIX_FMT_MT21, the decoder driver will decoded bitstream to
>>>>> V4L2_PIX_FMT_MT21 format.
>>>>>
>>>>> This patch series rely on MTK VPU driver in patch series "Add MT8173 Video Encoder Driver and VPU Driver"[1]
>>>>> and patch "CHROMIUM: v4l: Add V4L2_PIX_FMT_VP9 definition"[2] for VP9 support.
>>>>> Mediatek Video Decoder driver rely on VPU driver to load, communicate with VPU.
>>>>>
>>>>> Internally the driver uses videobuf2 framework and MTK IOMMU and MTK SMI both have been merged in v4.6-rc1.
>>>>>
>>>>> [1]https://patchwork.linuxtv.org/patch/33734/
>>>>> [2]https://chromium-review.googlesource.com/#/c/245241/  
>>>>
>>>> Hmm... I'm not seeing the firmware for this driver at the
>>>> linux-firmware tree:
>>>> 	https://git.kernel.org/cgit/linux/kernel/git/firmware/linux-firmware.git/log/
>>>>
>>>> Nor I'm seeing any pull request for them. Did you send it?
>>>> I'll only merge the driver upstream after seeing such pull request.  
>>>   
>> Sorry, I am not familiar with how to upstream firmware.
>> Do you mean we need to upstream vpu firmware first before merge encoder
>> driver upstream?
> 
> Please look at this page:
> 	https://linuxtv.org/wiki/index.php/Development:_How_to_submit_patches#Firmware_submission
> 
> The information here can also be useful:
> 	https://www.kernel.org/doc/readme/firmware-README.AddingFirmware
> 
> In summary, you need to provide redistribution rights for the
> firmware blob. You can either submit it to me or directly to
> linux-firmware. In the latter, please c/c me on such patch.

Tiffany, what is the status of the firmware submission?

Regards,

	Hans

[PATCH v3 0/9] Add MT8173 Video Decoder Driver

[Wu-Cheng Li (李務誠)]

+Andrew. Last I heard the license was ready and Andrew was preparing
the VPU firmware. Andrew. Is the firmware ready to submit?

On Fri, Jul 1, 2016 at 6:11 PM, Hans Verkuil <hverkuil@xs4all.nl> wrote:
> On 06/16/2016 12:54 PM, Mauro Carvalho Chehab wrote:
>> Em Tue, 14 Jun 2016 19:08:08 +0800
>> tiffany lin <tiffany.lin@mediatek.com> escreveu:
>>
>>> Hi Mauro,
>>>
>>>
>>> On Wed, 2016-06-08 at 07:13 +0900, Hans Verkuil wrote:
>>>>
>>>> On 06/07/2016 11:22 PM, Mauro Carvalho Chehab wrote:
>>>>> Em Mon, 30 May 2016 20:29:14 +0800
>>>>> Tiffany Lin <tiffany.lin@mediatek.com> escreveu:
>>>>>
>>>>>> ==============
>>>>>>   Introduction
>>>>>> ==============
>>>>>>
>>>>>> The purpose of this series is to add the driver for video codec hw embedded in the Mediatek's MT8173 SoCs.
>>>>>> Mediatek Video Codec is able to handle video decoding of in a range of formats.
>>>>>>
>>>>>> This patch series add Mediatek block format V4L2_PIX_FMT_MT21, the decoder driver will decoded bitstream to
>>>>>> V4L2_PIX_FMT_MT21 format.
>>>>>>
>>>>>> This patch series rely on MTK VPU driver in patch series "Add MT8173 Video Encoder Driver and VPU Driver"[1]
>>>>>> and patch "CHROMIUM: v4l: Add V4L2_PIX_FMT_VP9 definition"[2] for VP9 support.
>>>>>> Mediatek Video Decoder driver rely on VPU driver to load, communicate with VPU.
>>>>>>
>>>>>> Internally the driver uses videobuf2 framework and MTK IOMMU and MTK SMI both have been merged in v4.6-rc1.
>>>>>>
>>>>>> [1]https://patchwork.linuxtv.org/patch/33734/
>>>>>> [2]https://chromium-review.googlesource.com/#/c/245241/
>>>>>
>>>>> Hmm... I'm not seeing the firmware for this driver at the
>>>>> linux-firmware tree:
>>>>>    https://git.kernel.org/cgit/linux/kernel/git/firmware/linux-firmware.git/log/
>>>>>
>>>>> Nor I'm seeing any pull request for them. Did you send it?
>>>>> I'll only merge the driver upstream after seeing such pull request.
>>>>
>>> Sorry, I am not familiar with how to upstream firmware.
>>> Do you mean we need to upstream vpu firmware first before merge encoder
>>> driver upstream?
>>
>> Please look at this page:
>>       https://linuxtv.org/wiki/index.php/Development:_How_to_submit_patches#Firmware_submission
>>
>> The information here can also be useful:
>>       https://www.kernel.org/doc/readme/firmware-README.AddingFirmware
>>
>> In summary, you need to provide redistribution rights for the
>> firmware blob. You can either submit it to me or directly to
>> linux-firmware. In the latter, please c/c me on such patch.
>
> Tiffany, what is the status of the firmware submission?
>
> Regards,
>
>         Hans
> --
> To unsubscribe from this list: send the line "unsubscribe linux-media" in
> the body of a message to majordomo at vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v3 0/9] Add MT8173 Video Decoder Driver

[andrew-ct chen]

On Fri, 2016-07-01 at 12:11 +0200, Hans Verkuil wrote:
> On 06/16/2016 12:54 PM, Mauro Carvalho Chehab wrote:
> > Em Tue, 14 Jun 2016 19:08:08 +0800
> > tiffany lin <tiffany.lin@mediatek.com> escreveu:
> > 
> >> Hi Mauro,
> >>
> >>
> >> On Wed, 2016-06-08 at 07:13 +0900, Hans Verkuil wrote:
> >>>
> >>> On 06/07/2016 11:22 PM, Mauro Carvalho Chehab wrote:  
> >>>> Em Mon, 30 May 2016 20:29:14 +0800
> >>>> Tiffany Lin <tiffany.lin@mediatek.com> escreveu:
> >>>>  
> >>>>> ==============
> >>>>>   Introduction
> >>>>> ==============
> >>>>>
> >>>>> The purpose of this series is to add the driver for video codec hw embedded in the Mediatek's MT8173 SoCs.
> >>>>> Mediatek Video Codec is able to handle video decoding of in a range of formats.
> >>>>>
> >>>>> This patch series add Mediatek block format V4L2_PIX_FMT_MT21, the decoder driver will decoded bitstream to
> >>>>> V4L2_PIX_FMT_MT21 format.
> >>>>>
> >>>>> This patch series rely on MTK VPU driver in patch series "Add MT8173 Video Encoder Driver and VPU Driver"[1]
> >>>>> and patch "CHROMIUM: v4l: Add V4L2_PIX_FMT_VP9 definition"[2] for VP9 support.
> >>>>> Mediatek Video Decoder driver rely on VPU driver to load, communicate with VPU.
> >>>>>
> >>>>> Internally the driver uses videobuf2 framework and MTK IOMMU and MTK SMI both have been merged in v4.6-rc1.
> >>>>>
> >>>>> [1]https://patchwork.linuxtv.org/patch/33734/
> >>>>> [2]https://chromium-review.googlesource.com/#/c/245241/  
> >>>>
> >>>> Hmm... I'm not seeing the firmware for this driver at the
> >>>> linux-firmware tree:
> >>>> 	https://git.kernel.org/cgit/linux/kernel/git/firmware/linux-firmware.git/log/
> >>>>
> >>>> Nor I'm seeing any pull request for them. Did you send it?
> >>>> I'll only merge the driver upstream after seeing such pull request.  
> >>>   
> >> Sorry, I am not familiar with how to upstream firmware.
> >> Do you mean we need to upstream vpu firmware first before merge encoder
> >> driver upstream?
> > 
> > Please look at this page:
> > 	https://linuxtv.org/wiki/index.php/Development:_How_to_submit_patches#Firmware_submission
> > 
> > The information here can also be useful:
> > 	https://www.kernel.org/doc/readme/firmware-README.AddingFirmware
> > 
> > In summary, you need to provide redistribution rights for the
> > firmware blob. You can either submit it to me or directly to
> > linux-firmware. In the latter, please c/c me on such patch.
> 
> Tiffany, what is the status of the firmware submission?
> 
> Regards,
> 
> 	Hans

Hi Hans,
We are working on firmware test to make sure that both decoder and
encoder work well. Hopes it can be ready (firmware submission) on July 4
or July 5.

Thanks.
Andrew

> 
> _______________________________________________
> Linux-mediatek mailing list
> Linux-mediatek at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-mediatek

[PATCH v3 0/9] Add MT8173 Video Decoder Driver

[Hans Verkuil]

On 07/01/2016 01:53 PM, andrew-ct chen wrote:
> On Fri, 2016-07-01 at 12:11 +0200, Hans Verkuil wrote:
>> On 06/16/2016 12:54 PM, Mauro Carvalho Chehab wrote:
>>> Em Tue, 14 Jun 2016 19:08:08 +0800
>>> tiffany lin <tiffany.lin@mediatek.com> escreveu:
>>>
>>>> Hi Mauro,
>>>>
>>>>
>>>> On Wed, 2016-06-08 at 07:13 +0900, Hans Verkuil wrote:
>>>>>
>>>>> On 06/07/2016 11:22 PM, Mauro Carvalho Chehab wrote:  
>>>>>> Em Mon, 30 May 2016 20:29:14 +0800
>>>>>> Tiffany Lin <tiffany.lin@mediatek.com> escreveu:
>>>>>>  
>>>>>>> ==============
>>>>>>>   Introduction
>>>>>>> ==============
>>>>>>>
>>>>>>> The purpose of this series is to add the driver for video codec hw embedded in the Mediatek's MT8173 SoCs.
>>>>>>> Mediatek Video Codec is able to handle video decoding of in a range of formats.
>>>>>>>
>>>>>>> This patch series add Mediatek block format V4L2_PIX_FMT_MT21, the decoder driver will decoded bitstream to
>>>>>>> V4L2_PIX_FMT_MT21 format.
>>>>>>>
>>>>>>> This patch series rely on MTK VPU driver in patch series "Add MT8173 Video Encoder Driver and VPU Driver"[1]
>>>>>>> and patch "CHROMIUM: v4l: Add V4L2_PIX_FMT_VP9 definition"[2] for VP9 support.
>>>>>>> Mediatek Video Decoder driver rely on VPU driver to load, communicate with VPU.
>>>>>>>
>>>>>>> Internally the driver uses videobuf2 framework and MTK IOMMU and MTK SMI both have been merged in v4.6-rc1.
>>>>>>>
>>>>>>> [1]https://patchwork.linuxtv.org/patch/33734/
>>>>>>> [2]https://chromium-review.googlesource.com/#/c/245241/  
>>>>>>
>>>>>> Hmm... I'm not seeing the firmware for this driver at the
>>>>>> linux-firmware tree:
>>>>>> 	https://git.kernel.org/cgit/linux/kernel/git/firmware/linux-firmware.git/log/
>>>>>>
>>>>>> Nor I'm seeing any pull request for them. Did you send it?
>>>>>> I'll only merge the driver upstream after seeing such pull request.  
>>>>>   
>>>> Sorry, I am not familiar with how to upstream firmware.
>>>> Do you mean we need to upstream vpu firmware first before merge encoder
>>>> driver upstream?
>>>
>>> Please look at this page:
>>> 	https://linuxtv.org/wiki/index.php/Development:_How_to_submit_patches#Firmware_submission
>>>
>>> The information here can also be useful:
>>> 	https://www.kernel.org/doc/readme/firmware-README.AddingFirmware
>>>
>>> In summary, you need to provide redistribution rights for the
>>> firmware blob. You can either submit it to me or directly to
>>> linux-firmware. In the latter, please c/c me on such patch.
>>
>> Tiffany, what is the status of the firmware submission?
>>
>> Regards,
>>
>> 	Hans
> 
> Hi Hans,
> We are working on firmware test to make sure that both decoder and
> encoder work well. Hopes it can be ready (firmware submission) on July 4
> or July 5.

OK, great! Just wanted to make sure that this work was progressing and not stalled.

Thanks!

	Hans

[PATCH v3 0/9] Add MT8173 Video Decoder Driver

[andrew-ct chen]

On Fri, 2016-07-01 at 14:00 +0200, Hans Verkuil wrote:
> On 07/01/2016 01:53 PM, andrew-ct chen wrote:
> > On Fri, 2016-07-01 at 12:11 +0200, Hans Verkuil wrote:
> >> On 06/16/2016 12:54 PM, Mauro Carvalho Chehab wrote:
> >>> Em Tue, 14 Jun 2016 19:08:08 +0800
> >>> tiffany lin <tiffany.lin@mediatek.com> escreveu:
> >>>
> >>>> Hi Mauro,
> >>>>
> >>>>
> >>>> On Wed, 2016-06-08 at 07:13 +0900, Hans Verkuil wrote:
> >>>>>
> >>>>> On 06/07/2016 11:22 PM, Mauro Carvalho Chehab wrote:  
> >>>>>> Em Mon, 30 May 2016 20:29:14 +0800
> >>>>>> Tiffany Lin <tiffany.lin@mediatek.com> escreveu:
> >>>>>>  
> >>>>>>> ==============
> >>>>>>>   Introduction
> >>>>>>> ==============
> >>>>>>>
> >>>>>>> The purpose of this series is to add the driver for video codec hw embedded in the Mediatek's MT8173 SoCs.
> >>>>>>> Mediatek Video Codec is able to handle video decoding of in a range of formats.
> >>>>>>>
> >>>>>>> This patch series add Mediatek block format V4L2_PIX_FMT_MT21, the decoder driver will decoded bitstream to
> >>>>>>> V4L2_PIX_FMT_MT21 format.
> >>>>>>>
> >>>>>>> This patch series rely on MTK VPU driver in patch series "Add MT8173 Video Encoder Driver and VPU Driver"[1]
> >>>>>>> and patch "CHROMIUM: v4l: Add V4L2_PIX_FMT_VP9 definition"[2] for VP9 support.
> >>>>>>> Mediatek Video Decoder driver rely on VPU driver to load, communicate with VPU.
> >>>>>>>
> >>>>>>> Internally the driver uses videobuf2 framework and MTK IOMMU and MTK SMI both have been merged in v4.6-rc1.
> >>>>>>>
> >>>>>>> [1]https://patchwork.linuxtv.org/patch/33734/
> >>>>>>> [2]https://chromium-review.googlesource.com/#/c/245241/  
> >>>>>>
> >>>>>> Hmm... I'm not seeing the firmware for this driver at the
> >>>>>> linux-firmware tree:
> >>>>>> 	https://git.kernel.org/cgit/linux/kernel/git/firmware/linux-firmware.git/log/
> >>>>>>
> >>>>>> Nor I'm seeing any pull request for them. Did you send it?
> >>>>>> I'll only merge the driver upstream after seeing such pull request.  
> >>>>>   
> >>>> Sorry, I am not familiar with how to upstream firmware.
> >>>> Do you mean we need to upstream vpu firmware first before merge encoder
> >>>> driver upstream?
> >>>
> >>> Please look at this page:
> >>> 	https://linuxtv.org/wiki/index.php/Development:_How_to_submit_patches#Firmware_submission
> >>>
> >>> The information here can also be useful:
> >>> 	https://www.kernel.org/doc/readme/firmware-README.AddingFirmware
> >>>
> >>> In summary, you need to provide redistribution rights for the
> >>> firmware blob. You can either submit it to me or directly to
> >>> linux-firmware. In the latter, please c/c me on such patch.
> >>
> >> Tiffany, what is the status of the firmware submission?
> >>
> >> Regards,


> >>
> >> 	Hans
> > 
> > Hi Hans,
> > We are working on firmware test to make sure that both decoder and
> > encoder work well. Hopes it can be ready (firmware submission) on July 4
> > or July 5.
> 
> OK, great! Just wanted to make sure that this work was progressing and not stalled.
> 
> Thanks!
> 
> 	Hans

Hi Hans,

We have submitted VPU firmware for encoder as the following link.
http://lists.infradead.org/pipermail/linux-mediatek/2016-July/006082.html
If the series of decoder patches apply, we will update the VPU firmware
for decoder support.

Thanks!.
	
	Andrew

[PATCH v3 0/9] Add MT8173 Video Decoder Driver

[Hans Verkuil]

Hi Tiffany,

I plan to review this patch series on Friday, but one obvious question is
what the reason for these failures is:

> Input/Output configuration ioctls:
>         test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>         test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>         test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>         test VIDIOC_G/S_EDID: OK (Not Supported)
> 
>         Control ioctls:
>                 test VIDIOC_QUERYCTRL/MENU: OK
>                 fail: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-controls.cpp(357): g_ctrl returned an error (11)
>                 test VIDIOC_G/S_CTRL: FAIL
>                 fail: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-controls.cpp(579): g_ext_ctrls returned an error (11)
>                 test VIDIOC_G/S/TRY_EXT_CTRLS: FAIL
>                 fail: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-controls.cpp(721): subscribe event for control 'User Controls' failed
>                 test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: FAIL
>                 test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>                 Standard Controls: 2 Private Controls: 0
> 
>         Format ioctls:
>                 test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
>                 test VIDIOC_G/S_PARM: OK (Not Supported)
>                 test VIDIOC_G_FBUF: OK (Not Supported)
>                 fail: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-formats.cpp(405): expected EINVAL, but got 11 when getting format for buftype 9
>                 test VIDIOC_G_FMT: FAIL
>                 test VIDIOC_TRY_FMT: OK (Not Supported)
>                 test VIDIOC_S_FMT: OK (Not Supported)
>                 test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> 
>         Codec ioctls:
>                 test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>                 test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>                 test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> 
>         Buffer ioctls:
>                 test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
>                 fail: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-buffers.cpp(500): q.has_expbuf(node)
>                 test VIDIOC_EXPBUF: FAIL
> 
> 
> Total: 38, Succeeded: 33, Failed: 5, Warnings: 0

If it is due to a bug in v4l2-compliance, then let me know and I'll fix it. If not,
then it should be fixed in the driver.

Frankly, it was the presence of these failures that made me think this patch series
wasn't final. Before a v4l2 driver can be accepted in the kernel, v4l2-compliance must pass.

Regards,

	Hans

[PATCH v3 0/9] Add MT8173 Video Decoder Driver

[tiffany lin]

Hi Hans,


On Wed, 2016-07-06 at 15:19 +0200, Hans Verkuil wrote:
> Hi Tiffany,
> 
> I plan to review this patch series on Friday, but one obvious question is
> what the reason for these failures is:
> 
> > Input/Output configuration ioctls:
> >         test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> >         test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> >         test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> >         test VIDIOC_G/S_EDID: OK (Not Supported)
> > 
> >         Control ioctls:
> >                 test VIDIOC_QUERYCTRL/MENU: OK
> >                 fail: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-controls.cpp(357): g_ctrl returned an error (11)
> >                 test VIDIOC_G/S_CTRL: FAIL
> >                 fail: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-controls.cpp(579): g_ext_ctrls returned an error (11)
> >                 test VIDIOC_G/S/TRY_EXT_CTRLS: FAIL
These fails are because VIDIOC_G_CTRL and VIDIOC_G_EXT_CTRLS return
V4L2_CID_MIN_BUFFERS_FOR_CAPTURE only when dirver in MTK_STATE_HEADER
state, or it will return EAGAIN.
This could help user space get correct value, not default value that may
changed base on media content.

> >                 fail: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-controls.cpp(721): subscribe event for control 'User Controls' failed
> >                 test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: FAIL
Driver do not support subscribe event for control 'User Controls' for
now.
Do we need to support this?

> >                 test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> >                 Standard Controls: 2 Private Controls: 0
> > 
> >         Format ioctls:
> >                 test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> >                 test VIDIOC_G/S_PARM: OK (Not Supported)
> >                 test VIDIOC_G_FBUF: OK (Not Supported)
> >                 fail: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-formats.cpp(405): expected EINVAL, but got 11 when getting format for buftype 9
> >                 test VIDIOC_G_FMT: FAIL
This is because vidioc_vdec_g_fmt only succeed when context is in
MTK_STATE_HEADER state, or user space cannot get correct format data
using this function.

> >                 test VIDIOC_TRY_FMT: OK (Not Supported)
> >                 test VIDIOC_S_FMT: OK (Not Supported)
> >                 test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> > 
> >         Codec ioctls:
> >                 test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> >                 test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> >                 test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> > 
> >         Buffer ioctls:
> >                 test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
> >                 fail: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-buffers.cpp(500): q.has_expbuf(node)
Our OUTPUT and CAPTURE queue support both VB2_DMABUF and VB2_MMAP, user
space can select which to use in runtime.
So our driver default support v4l2_m2m_ioctl_expbuf functionality.
In v4l2-compliance test, it will check v4l2_m2m_ioctl_expbuf only valid
when node->valid_memorytype is V4L2_MEMORY_MMAP.
So when go through node->valid_memorytype is V4L2_MEMORY_DMABUF, it
fail.


best regards,
Tiffany



> >                 test VIDIOC_EXPBUF: FAIL
> > 
> > 
> > Total: 38, Succeeded: 33, Failed: 5, Warnings: 0
> 
> If it is due to a bug in v4l2-compliance, then let me know and I'll fix it. If not,
> then it should be fixed in the driver.
> 
> Frankly, it was the presence of these failures that made me think this patch series
> wasn't final. Before a v4l2 driver can be accepted in the kernel, v4l2-compliance must pass.
> 
> Regards,
> 
> 	Hans

[PATCH v3 0/9] Add MT8173 Video Decoder Driver

[Hans Verkuil]

Hi Tiffany,

On 07/07/16 12:16, tiffany lin wrote:
> Hi Hans,
> 
> 
> On Wed, 2016-07-06 at 15:19 +0200, Hans Verkuil wrote:
>> Hi Tiffany,
>>
>> I plan to review this patch series on Friday, but one obvious question is
>> what the reason for these failures is:
>>
>>> Input/Output configuration ioctls:
>>>         test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>>>         test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>>>         test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>>>         test VIDIOC_G/S_EDID: OK (Not Supported)
>>>
>>>         Control ioctls:
>>>                 test VIDIOC_QUERYCTRL/MENU: OK
>>>                 fail: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-controls.cpp(357): g_ctrl returned an error (11)
>>>                 test VIDIOC_G/S_CTRL: FAIL
>>>                 fail: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-controls.cpp(579): g_ext_ctrls returned an error (11)
>>>                 test VIDIOC_G/S/TRY_EXT_CTRLS: FAIL
> These fails are because VIDIOC_G_CTRL and VIDIOC_G_EXT_CTRLS return
> V4L2_CID_MIN_BUFFERS_FOR_CAPTURE only when dirver in MTK_STATE_HEADER
> state, or it will return EAGAIN.
> This could help user space get correct value, not default value that may
> changed base on media content.

So this should be improved in v4l2-compliance.

> 
>>>                 fail: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-controls.cpp(721): subscribe event for control 'User Controls' failed
>>>                 test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: FAIL
> Driver do not support subscribe event for control 'User Controls' for
> now.
> Do we need to support this?

Let me check: this might be a knock-on effect from the previous error.

Which controls does your driver have? (v4l2-ctl -l)

> 
>>>                 test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>>>                 Standard Controls: 2 Private Controls: 0
>>>
>>>         Format ioctls:
>>>                 test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
>>>                 test VIDIOC_G/S_PARM: OK (Not Supported)
>>>                 test VIDIOC_G_FBUF: OK (Not Supported)
>>>                 fail: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-formats.cpp(405): expected EINVAL, but got 11 when getting format for buftype 9
>>>                 test VIDIOC_G_FMT: FAIL
> This is because vidioc_vdec_g_fmt only succeed when context is in
> MTK_STATE_HEADER state, or user space cannot get correct format data
> using this function.

I'll take a look at this as well.

> 
>>>                 test VIDIOC_TRY_FMT: OK (Not Supported)
>>>                 test VIDIOC_S_FMT: OK (Not Supported)
>>>                 test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>>>
>>>         Codec ioctls:
>>>                 test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>>>                 test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>>>                 test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>>>
>>>         Buffer ioctls:
>>>                 test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
>>>                 fail: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-buffers.cpp(500): q.has_expbuf(node)

Don't use v4l-utils-1.6: just use the latest master branch. That way you can
be certain that the latest fixes are in. Besides, 1.6.0 is really old (almost
two years!). If you used the same old version for the encoder driver as well,
then please retest that encoder driver!

> Our OUTPUT and CAPTURE queue support both VB2_DMABUF and VB2_MMAP, user
> space can select which to use in runtime.
> So our driver default support v4l2_m2m_ioctl_expbuf functionality.
> In v4l2-compliance test, it will check v4l2_m2m_ioctl_expbuf only valid
> when node->valid_memorytype is V4L2_MEMORY_MMAP.
> So when go through node->valid_memorytype is V4L2_MEMORY_DMABUF, it
> fail.

That sounds like a bug. I'll take a look. For all I know, this is fixed in the
master branch.

Regards,

	Hans

> 
> 
> best regards,
> Tiffany
> 
> 
> 
>>>                 test VIDIOC_EXPBUF: FAIL
>>>
>>>
>>> Total: 38, Succeeded: 33, Failed: 5, Warnings: 0
>>
>> If it is due to a bug in v4l2-compliance, then let me know and I'll fix it. If not,
>> then it should be fixed in the driver.
>>
>> Frankly, it was the presence of these failures that made me think this patch series
>> wasn't final. Before a v4l2 driver can be accepted in the kernel, v4l2-compliance must pass.
>>
>> Regards,
>>
>> 	Hans
> 
> 

[PATCH v3 0/9] Add MT8173 Video Decoder Driver

[Hans Verkuil]

On 07/07/2016 12:16 PM, tiffany lin wrote:
> Hi Hans,
> 
> 
> On Wed, 2016-07-06 at 15:19 +0200, Hans Verkuil wrote:
>> Hi Tiffany,
>>
>> I plan to review this patch series on Friday, but one obvious question is
>> what the reason for these failures is:
>>
>>> Input/Output configuration ioctls:
>>>         test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>>>         test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>>>         test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>>>         test VIDIOC_G/S_EDID: OK (Not Supported)
>>>
>>>         Control ioctls:
>>>                 test VIDIOC_QUERYCTRL/MENU: OK
>>>                 fail: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-controls.cpp(357): g_ctrl returned an error (11)
>>>                 test VIDIOC_G/S_CTRL: FAIL
>>>                 fail: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-controls.cpp(579): g_ext_ctrls returned an error (11)
>>>                 test VIDIOC_G/S/TRY_EXT_CTRLS: FAIL
> These fails are because VIDIOC_G_CTRL and VIDIOC_G_EXT_CTRLS return
> V4L2_CID_MIN_BUFFERS_FOR_CAPTURE only when dirver in MTK_STATE_HEADER
> state, or it will return EAGAIN.
> This could help user space get correct value, not default value that may
> changed base on media content.

OK, I really don't like this. I also looked what the s5p-mfc-dec driver does (the only other
driver currently implementing this), and that returns -EINVAL.

My proposal would be to change this. If this information isn't known yet, why not
just return 0 as the value? The doc would have to be updated and (preferably) also
the s5p-mfc-dec driver. I've added Samsung devs to the Cc list, let me know what you
think.

> 
>>>                 fail: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-controls.cpp(721): subscribe event for control 'User Controls' failed
>>>                 test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: FAIL
> Driver do not support subscribe event for control 'User Controls' for
> now.
> Do we need to support this?

I don't see why this would fail. It's OK to subscribe to such controls, although
you'll never get an event.

>>>                 test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>>>                 Standard Controls: 2 Private Controls: 0
>>>
>>>         Format ioctls:
>>>                 test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
>>>                 test VIDIOC_G/S_PARM: OK (Not Supported)
>>>                 test VIDIOC_G_FBUF: OK (Not Supported)
>>>                 fail: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-formats.cpp(405): expected EINVAL, but got 11 when getting format for buftype 9
>>>                 test VIDIOC_G_FMT: FAIL
> This is because vidioc_vdec_g_fmt only succeed when context is in
> MTK_STATE_HEADER state, or user space cannot get correct format data
> using this function.

Comparing this to s5p-mfc-dec I see that -EINVAL is returned in that case.

I am not opposed to using EAGAIN in s5p-mfc-dec as well. Marek, Kamil, what is
your opinion?

> 
>>>                 test VIDIOC_TRY_FMT: OK (Not Supported)
>>>                 test VIDIOC_S_FMT: OK (Not Supported)
>>>                 test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>>>
>>>         Codec ioctls:
>>>                 test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>>>                 test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>>>                 test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>>>
>>>         Buffer ioctls:
>>>                 test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
>>>                 fail: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-buffers.cpp(500): q.has_expbuf(node)
> Our OUTPUT and CAPTURE queue support both VB2_DMABUF and VB2_MMAP, user
> space can select which to use in runtime.
> So our driver default support v4l2_m2m_ioctl_expbuf functionality.
> In v4l2-compliance test, it will check v4l2_m2m_ioctl_expbuf only valid
> when node->valid_memorytype is V4L2_MEMORY_MMAP.
> So when go through node->valid_memorytype is V4L2_MEMORY_DMABUF, it
> fail.

valid_memorytype should have both MMAP and DMABUF flags.

But v4l-utils-1.6.0 is way too old to be certain it isn't some v4l2-compliance bug
that has since been fixed.

Regards,

	Hans

> 
> 
> best regards,
> Tiffany
> 
> 
> 
>>>                 test VIDIOC_EXPBUF: FAIL
>>>
>>>
>>> Total: 38, Succeeded: 33, Failed: 5, Warnings: 0
>>
>> If it is due to a bug in v4l2-compliance, then let me know and I'll fix it. If not,
>> then it should be fixed in the driver.
>>
>> Frankly, it was the presence of these failures that made me think this patch series
>> wasn't final. Before a v4l2 driver can be accepted in the kernel, v4l2-compliance must pass.
>>
>> Regards,
>>
>> 	Hans
> 
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-media" in
> the body of a message to majordomo at vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

[PATCH v3 0/9] Add MT8173 Video Decoder Driver

[tiffany lin]

Hi Hans,

On Fri, 2016-07-08 at 13:44 +0200, Hans Verkuil wrote:
> On 07/07/2016 12:16 PM, tiffany lin wrote:
> > Hi Hans,
> > 
> > 
> > On Wed, 2016-07-06 at 15:19 +0200, Hans Verkuil wrote:
> >> Hi Tiffany,
> >>
> >> I plan to review this patch series on Friday, but one obvious question is
> >> what the reason for these failures is:
> >>
> >>> Input/Output configuration ioctls:
> >>>         test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> >>>         test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> >>>         test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> >>>         test VIDIOC_G/S_EDID: OK (Not Supported)
> >>>
> >>>         Control ioctls:
> >>>                 test VIDIOC_QUERYCTRL/MENU: OK
> >>>                 fail: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-controls.cpp(357): g_ctrl returned an error (11)
> >>>                 test VIDIOC_G/S_CTRL: FAIL
> >>>                 fail: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-controls.cpp(579): g_ext_ctrls returned an error (11)
> >>>                 test VIDIOC_G/S/TRY_EXT_CTRLS: FAIL
> > These fails are because VIDIOC_G_CTRL and VIDIOC_G_EXT_CTRLS return
> > V4L2_CID_MIN_BUFFERS_FOR_CAPTURE only when dirver in MTK_STATE_HEADER
> > state, or it will return EAGAIN.
> > This could help user space get correct value, not default value that may
> > changed base on media content.
> 
> OK, I really don't like this. I also looked what the s5p-mfc-dec driver does (the only other
> driver currently implementing this), and that returns -EINVAL.
> 
> My proposal would be to change this. If this information isn't known yet, why not
> just return 0 as the value? The doc would have to be updated and (preferably) also
> the s5p-mfc-dec driver. I've added Samsung devs to the Cc list, let me know what you
> think.
> 
Got it. We will return 0 as the value instead EAGAIN.

> > 
> >>>                 fail: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-controls.cpp(721): subscribe event for control 'User Controls' failed
> >>>                 test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: FAIL
> > Driver do not support subscribe event for control 'User Controls' for
> > now.
> > Do we need to support this?
> 
> I don't see why this would fail. It's OK to subscribe to such controls, although
> you'll never get an event.
> 
Got it. I will rerun using latest v4l2-compliance test check this.

> >>>                 test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> >>>                 Standard Controls: 2 Private Controls: 0
> >>>
> >>>         Format ioctls:
> >>>                 test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> >>>                 test VIDIOC_G/S_PARM: OK (Not Supported)
> >>>                 test VIDIOC_G_FBUF: OK (Not Supported)
> >>>                 fail: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-formats.cpp(405): expected EINVAL, but got 11 when getting format for buftype 9
> >>>                 test VIDIOC_G_FMT: FAIL
> > This is because vidioc_vdec_g_fmt only succeed when context is in
> > MTK_STATE_HEADER state, or user space cannot get correct format data
> > using this function.
> 
> Comparing this to s5p-mfc-dec I see that -EINVAL is returned in that case.
> 
> I am not opposed to using EAGAIN in s5p-mfc-dec as well. Marek, Kamil, what is
> your opinion?
> 
> > 
> >>>                 test VIDIOC_TRY_FMT: OK (Not Supported)
> >>>                 test VIDIOC_S_FMT: OK (Not Supported)
> >>>                 test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> >>>
> >>>         Codec ioctls:
> >>>                 test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> >>>                 test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> >>>                 test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> >>>
> >>>         Buffer ioctls:
> >>>                 test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
> >>>                 fail: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-buffers.cpp(500): q.has_expbuf(node)
> > Our OUTPUT and CAPTURE queue support both VB2_DMABUF and VB2_MMAP, user
> > space can select which to use in runtime.
> > So our driver default support v4l2_m2m_ioctl_expbuf functionality.
> > In v4l2-compliance test, it will check v4l2_m2m_ioctl_expbuf only valid
> > when node->valid_memorytype is V4L2_MEMORY_MMAP.
> > So when go through node->valid_memorytype is V4L2_MEMORY_DMABUF, it
> > fail.
> 
> valid_memorytype should have both MMAP and DMABUF flags.
> 
> But v4l-utils-1.6.0 is way too old to be certain it isn't some v4l2-compliance bug
> that has since been fixed.
> 
Got it. I will rerun using master branch to check this.

best regards,
Tiffany

> Regards,
> 
> 	Hans
> 
> > 
> > 
> > best regards,
> > Tiffany
> > 
> > 
> > 
> >>>                 test VIDIOC_EXPBUF: FAIL
> >>>
> >>>
> >>> Total: 38, Succeeded: 33, Failed: 5, Warnings: 0
> >>
> >> If it is due to a bug in v4l2-compliance, then let me know and I'll fix it. If not,
> >> then it should be fixed in the driver.
> >>
> >> Frankly, it was the presence of these failures that made me think this patch series
> >> wasn't final. Before a v4l2 driver can be accepted in the kernel, v4l2-compliance must pass.
> >>
> >> Regards,
> >>
> >> 	Hans
> > 
> > 
> > --
> > To unsubscribe from this list: send the line "unsubscribe linux-media" in
> > the body of a message to majordomo at vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> > 

[PATCH v3 0/9] Add MT8173 Video Decoder Driver

[tiffany lin]

Hi Hans,

On Fri, 2016-07-08 at 13:44 +0200, Hans Verkuil wrote:
> On 07/07/2016 12:16 PM, tiffany lin wrote:
> > Hi Hans,
> > 
> > 
> > On Wed, 2016-07-06 at 15:19 +0200, Hans Verkuil wrote:
> >> Hi Tiffany,
> >>
> >> I plan to review this patch series on Friday, but one obvious question is
> >> what the reason for these failures is:
> >>
> >>> Input/Output configuration ioctls:
> >>>         test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> >>>         test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> >>>         test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> >>>         test VIDIOC_G/S_EDID: OK (Not Supported)
> >>>
> >>>         Control ioctls:
> >>>                 test VIDIOC_QUERYCTRL/MENU: OK
> >>>                 fail: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-controls.cpp(357): g_ctrl returned an error (11)
> >>>                 test VIDIOC_G/S_CTRL: FAIL
> >>>                 fail: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-controls.cpp(579): g_ext_ctrls returned an error (11)
> >>>                 test VIDIOC_G/S/TRY_EXT_CTRLS: FAIL
> > These fails are because VIDIOC_G_CTRL and VIDIOC_G_EXT_CTRLS return
> > V4L2_CID_MIN_BUFFERS_FOR_CAPTURE only when dirver in MTK_STATE_HEADER
> > state, or it will return EAGAIN.
> > This could help user space get correct value, not default value that may
> > changed base on media content.
> 
> OK, I really don't like this. I also looked what the s5p-mfc-dec driver does (the only other
> driver currently implementing this), and that returns -EINVAL.
> 
> My proposal would be to change this. If this information isn't known yet, why not
> just return 0 as the value? The doc would have to be updated and (preferably) also
> the s5p-mfc-dec driver. I've added Samsung devs to the Cc list, let me know what you
> think.
> 
We are ok with just return 0 as the value when header not parsed yet
We will update decoder with this solution.


> > 
> >>>                 fail: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-controls.cpp(721): subscribe event for control 'User Controls' failed
> >>>                 test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: FAIL
> > Driver do not support subscribe event for control 'User Controls' for
> > now.
> > Do we need to support this?
> 
> I don't see why this would fail. It's OK to subscribe to such controls, although
> you'll never get an event.
> 
After upgrade to latest v4l-utils,this test pass.

> >>>                 test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> >>>                 Standard Controls: 2 Private Controls: 0
> >>>
> >>>         Format ioctls:
> >>>                 test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> >>>                 test VIDIOC_G/S_PARM: OK (Not Supported)
> >>>                 test VIDIOC_G_FBUF: OK (Not Supported)
> >>>                 fail: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-formats.cpp(405): expected EINVAL, but got 11 when getting format for buftype 9
> >>>                 test VIDIOC_G_FMT: FAIL
> > This is because vidioc_vdec_g_fmt only succeed when context is in
> > MTK_STATE_HEADER state, or user space cannot get correct format data
> > using this function.
> 
> Comparing this to s5p-mfc-dec I see that -EINVAL is returned in that case.
> 
> I am not opposed to using EAGAIN in s5p-mfc-dec as well. Marek, Kamil, what is
> your opinion?
> 
> > 
> >>>                 test VIDIOC_TRY_FMT: OK (Not Supported)
> >>>                 test VIDIOC_S_FMT: OK (Not Supported)
> >>>                 test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> >>>
> >>>         Codec ioctls:
> >>>                 test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> >>>                 test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> >>>                 test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> >>>
> >>>         Buffer ioctls:
> >>>                 test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
> >>>                 fail: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-buffers.cpp(500): q.has_expbuf(node)
> > Our OUTPUT and CAPTURE queue support both VB2_DMABUF and VB2_MMAP, user
> > space can select which to use in runtime.
> > So our driver default support v4l2_m2m_ioctl_expbuf functionality.
> > In v4l2-compliance test, it will check v4l2_m2m_ioctl_expbuf only valid
> > when node->valid_memorytype is V4L2_MEMORY_MMAP.
> > So when go through node->valid_memorytype is V4L2_MEMORY_DMABUF, it
> > fail.
> 
> valid_memorytype should have both MMAP and DMABUF flags.
> 
> But v4l-utils-1.6.0 is way too old to be certain it isn't some v4l2-compliance bug
> that has since been fixed.
After upgrade to latest v4l-utils, I still see this issue.

test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
fail: v4l2-test-buffers.cpp(571): q.has_expbuf(node)

best regards,
Tiffany

> 
> Regards,
> 
> 	Hans
> 
> > 
> > 
> > best regards,
> > Tiffany
> > 
> > 
> > 
> >>>                 test VIDIOC_EXPBUF: FAIL
> >>>
> >>>
> >>> Total: 38, Succeeded: 33, Failed: 5, Warnings: 0
> >>
> >> If it is due to a bug in v4l2-compliance, then let me know and I'll fix it. If not,
> >> then it should be fixed in the driver.
> >>
> >> Frankly, it was the presence of these failures that made me think this patch series
> >> wasn't final. Before a v4l2 driver can be accepted in the kernel, v4l2-compliance must pass.
> >>
> >> Regards,
> >>
> >> 	Hans
> > 
> > 
> > --
> > To unsubscribe from this list: send the line "unsubscribe linux-media" in
> > the body of a message to majordomo at vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> > 

[PATCH v3 0/9] Add MT8173 Video Decoder Driver

[Hans Verkuil]

Hi Tiffany,

This needs a v4: the DocBook format has been replaced by sphinx, and I also see that
the vb2 queue_setup function is out-of-date (the allocation contexts no longer exist).

There were also a lot of comments about the MT21 format. I recommend splitting off the
code adding MT21 into separate patches at the end of the patch series. That way if there
are more comments it is easy to merge everything but those patches relating to MT21.

Can you also add the patches for the new V4L2_PIX_FMT_VP9 to this patch series? Those
needs to be updated as well due to the move to sphinx.

Thanks!

	Hans

On 05/30/2016 02:29 PM, Tiffany Lin wrote:
> ==============
>  Introduction
> ==============
> 
> The purpose of this series is to add the driver for video codec hw embedded in the Mediatek's MT8173 SoCs.
> Mediatek Video Codec is able to handle video decoding of in a range of formats.
> 
> This patch series add Mediatek block format V4L2_PIX_FMT_MT21, the decoder driver will decoded bitstream to
> V4L2_PIX_FMT_MT21 format.
> 
> This patch series rely on MTK VPU driver in patch series "Add MT8173 Video Encoder Driver and VPU Driver"[1]
> and patch "CHROMIUM: v4l: Add V4L2_PIX_FMT_VP9 definition"[2] for VP9 support.
> Mediatek Video Decoder driver rely on VPU driver to load, communicate with VPU.
> 
> Internally the driver uses videobuf2 framework and MTK IOMMU and MTK SMI both have been merged in v4.6-rc1.
> 
> [1]https://patchwork.linuxtv.org/patch/33734/
> [2]https://chromium-review.googlesource.com/#/c/245241/
> 
> ==================
>  Device interface
> ==================
> 
> In principle the driver bases on v4l2 memory-to-memory framework:
> it provides a single video node and each opened file handle gets its own private context with separate
> buffer queues. Each context consist of 2 buffer queues: OUTPUT (for source buffers, i.e. bitstream)
> and CAPTURE (for destination buffers, i.e. decoded video frames).
> OUTPUT and CAPTURE buffer could be MMAP or DMABUF memory type.
> VIDIOC_G_CTRL and VIDIOC_G_EXT_CTRLS return V4L2_CID_MIN_BUFFERS_FOR_CAPTURE only when dirver in MTK_STATE_HEADER
> state, or it will return EAGAIN.
> Driver do not support subscribe event for control 'User Controls' for now.
> And it default support export DMABUF for other display drivers.
> 
> Change in v3:
> 1. Refine vdec hw clock setting
> 2. Refine vp9 codec driver
> 3. Refine v4l2 codec driver
> 
> Change in v2:
> 1. Add documentation for V4L2_PIX_FMT_MT21
> 2. Remove DRM_FORMAT_MT21
> 2. Refine code according to review comments
> 
> v4l2-compliance test output:
> localhost Encode # ./v4l2-compliance -d /dev/video0
> Driver Info:
>         Driver name   : mtk-vcodec-dec
>         Card type     : platform:mt8173
>         Bus info      : platform:mt8173
>         Driver version: 4.4.0
>         Capabilities  : 0x84204000
>                 Video Memory-to-Memory Multiplanar
>                 Streaming
>                 Extended Pix Format
>                 Device Capabilities
>         Device Caps   : 0x04204000
>                 Video Memory-to-Memory Multiplanar
>                 Streaming
>                 Extended Pix Format
> 
> Compliance test for device /dev/video0 (not using libv4l2):
> 
> Required ioctls:
>         test VIDIOC_QUERYCAP: OK
> 
> Allow for multiple opens:
>         test second video open: OK
>         test VIDIOC_QUERYCAP: OK
>         test VIDIOC_G/S_PRIORITY: OK
> 
> Debug ioctls:
>         test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
>         test VIDIOC_LOG_STATUS: OK (Not Supported)
> 
> Input ioctls:
>         test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>         test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>         test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>         test VIDIOC_ENUMAUDIO: OK (Not Supported)
>         test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>         test VIDIOC_G/S_AUDIO: OK (Not Supported)
>         Inputs: 0 Audio Inputs: 0 Tuners: 0
> 
> Output ioctls:
>         test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>         test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>         test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>         test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>         test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>         Outputs: 0 Audio Outputs: 0 Modulators: 0
> 
> Input/Output configuration ioctls:
>         test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>         test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>         test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>         test VIDIOC_G/S_EDID: OK (Not Supported)
> 
>         Control ioctls:
>                 test VIDIOC_QUERYCTRL/MENU: OK
>                 fail: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-controls.cpp(357): g_ctrl returned an error (11)
>                 test VIDIOC_G/S_CTRL: FAIL
>                 fail: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-controls.cpp(579): g_ext_ctrls returned an error (11)
>                 test VIDIOC_G/S/TRY_EXT_CTRLS: FAIL
>                 fail: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-controls.cpp(721): subscribe event for control 'User Controls' failed
>                 test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: FAIL
>                 test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>                 Standard Controls: 2 Private Controls: 0
> 
>         Format ioctls:
>                 test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
>                 test VIDIOC_G/S_PARM: OK (Not Supported)
>                 test VIDIOC_G_FBUF: OK (Not Supported)
>                 fail: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-formats.cpp(405): expected EINVAL, but got 11 when getting format for buftype 9
>                 test VIDIOC_G_FMT: FAIL
>                 test VIDIOC_TRY_FMT: OK (Not Supported)
>                 test VIDIOC_S_FMT: OK (Not Supported)
>                 test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> 
>         Codec ioctls:
>                 test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>                 test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>                 test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> 
>         Buffer ioctls:
>                 test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
>                 fail: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-buffers.cpp(500): q.has_expbuf(node)
>                 test VIDIOC_EXPBUF: FAIL
> 
> 
> Total: 38, Succeeded: 33, Failed: 5, Warnings: 0
> 
> 
> Andrew-CT Chen (1):
>   VPU: mediatek: Add decode support
> 
> Tiffany Lin (8):
>   v4l: add Mediatek compressed video block format
>   DocBook/v4l: Add compressed video formats used on MT8173 codec driver
>   dt-bindings: Add a binding for Mediatek Video Decoder
>   vcodec: mediatek: Add Mediatek V4L2 Video Decoder Driver
>   vcodec: mediatek: Add Mediatek H264 Video Decoder Driver
>   vcodec: mediatek: Add Mediatek VP8 Video Decoder Driver
>   vcodec: mediatek: Add Mediatek VP9 Video Decoder Driver
>   arm64: dts: mediatek: Add Video Decoder for MT8173
> 
>  Documentation/DocBook/media/v4l/pixfmt.xml         |    6 +
>  .../devicetree/bindings/media/mediatek-vcodec.txt  |   57 +-
>  arch/arm64/boot/dts/mediatek/mt8173.dtsi           |   44 +
>  drivers/media/platform/mtk-vcodec/Makefile         |   15 +-
>  drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c | 1348 ++++++++++++++++++++
>  drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.h |   88 ++
>  .../media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c |  408 ++++++
>  .../media/platform/mtk-vcodec/mtk_vcodec_dec_pm.c  |  206 +++
>  .../media/platform/mtk-vcodec/mtk_vcodec_dec_pm.h  |   28 +
>  drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h |   88 +-
>  .../media/platform/mtk-vcodec/mtk_vcodec_intr.c    |    4 +-
>  .../media/platform/mtk-vcodec/mtk_vcodec_intr.h    |    2 +-
>  .../media/platform/mtk-vcodec/mtk_vcodec_util.c    |   31 +-
>  .../media/platform/mtk-vcodec/mtk_vcodec_util.h    |    7 +-
>  .../media/platform/mtk-vcodec/vdec/vdec_h264_if.c  |  503 ++++++++
>  .../media/platform/mtk-vcodec/vdec/vdec_vp8_if.c   |  630 +++++++++
>  .../media/platform/mtk-vcodec/vdec/vdec_vp9_if.c   |  967 ++++++++++++++
>  drivers/media/platform/mtk-vcodec/vdec_drv_base.h  |   56 +
>  drivers/media/platform/mtk-vcodec/vdec_drv_if.c    |  117 ++
>  drivers/media/platform/mtk-vcodec/vdec_drv_if.h    |  101 ++
>  drivers/media/platform/mtk-vcodec/vdec_ipi_msg.h   |  103 ++
>  drivers/media/platform/mtk-vcodec/vdec_vpu_if.c    |  168 +++
>  drivers/media/platform/mtk-vcodec/vdec_vpu_if.h    |   96 ++
>  drivers/media/platform/mtk-vpu/mtk_vpu.c           |   12 +
>  drivers/media/platform/mtk-vpu/mtk_vpu.h           |   27 +
>  include/uapi/linux/videodev2.h                     |    1 +
>  26 files changed, 5090 insertions(+), 23 deletions(-)
>  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c
>  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.h
>  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c
>  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_pm.c
>  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_pm.h
>  create mode 100644 drivers/media/platform/mtk-vcodec/vdec/vdec_h264_if.c
>  create mode 100644 drivers/media/platform/mtk-vcodec/vdec/vdec_vp8_if.c
>  create mode 100644 drivers/media/platform/mtk-vcodec/vdec/vdec_vp9_if.c
>  create mode 100644 drivers/media/platform/mtk-vcodec/vdec_drv_base.h
>  create mode 100644 drivers/media/platform/mtk-vcodec/vdec_drv_if.c
>  create mode 100644 drivers/media/platform/mtk-vcodec/vdec_drv_if.h
>  create mode 100644 drivers/media/platform/mtk-vcodec/vdec_ipi_msg.h
>  create mode 100644 drivers/media/platform/mtk-vcodec/vdec_vpu_if.c
>  create mode 100644 drivers/media/platform/mtk-vcodec/vdec_vpu_if.h
> 

[PATCH v3 0/9] Add MT8173 Video Decoder Driver

[Tiffany Lin]

Hi Hans,

I upstream v4 on 8/10, I don't know why this version is not shown in
https://patchwork.kernel.org.
But I could see it in other link.
http://www.spinics.net/lists/arm-kernel/msg523095.html
I refine DocBook and vb2 queue_setup function in v4.

For the MT21 format, if I put MT21 into separate patch, this patch
series will build fail, I was confused how to fix this issue?


For VP9, I will merge this three patchs to V4L2 Decoder patch series.
 * linux-media: [v1,1/3] videodev2.h: add V4L2_PIX_FMT_VP9 format.
     - http://patchwork.linuxtv.org/patch/34392/
 * linux-media: [v1,3/3] V4L: add VP9 format documentation
     - http://patchwork.linuxtv.org/patch/34393/
 * linux-media: [v1,2/3] v4l2-ioctl: add VP9 format description.
     - http://patchwork.linuxtv.org/patch/34394/


best regards,
Tiffany


On Mon, 2016-08-15 at 10:30 +0200, Hans Verkuil wrote:
> Hi Tiffany,
> 
> This needs a v4: the DocBook format has been replaced by sphinx, and I also see that
> the vb2 queue_setup function is out-of-date (the allocation contexts no longer exist).
> 
> There were also a lot of comments about the MT21 format. I recommend splitting off the
> code adding MT21 into separate patches at the end of the patch series. That way if there
> are more comments it is easy to merge everything but those patches relating to MT21.
> 
> Can you also add the patches for the new V4L2_PIX_FMT_VP9 to this patch series? Those
> needs to be updated as well due to the move to sphinx.
> 
> Thanks!
> 
> 	Hans
> 
> On 05/30/2016 02:29 PM, Tiffany Lin wrote:
> > ==============
> >  Introduction
> > ==============
> > 
> > The purpose of this series is to add the driver for video codec hw embedded in the Mediatek's MT8173 SoCs.
> > Mediatek Video Codec is able to handle video decoding of in a range of formats.
> > 
> > This patch series add Mediatek block format V4L2_PIX_FMT_MT21, the decoder driver will decoded bitstream to
> > V4L2_PIX_FMT_MT21 format.
> > 
> > This patch series rely on MTK VPU driver in patch series "Add MT8173 Video Encoder Driver and VPU Driver"[1]
> > and patch "CHROMIUM: v4l: Add V4L2_PIX_FMT_VP9 definition"[2] for VP9 support.
> > Mediatek Video Decoder driver rely on VPU driver to load, communicate with VPU.
> > 
> > Internally the driver uses videobuf2 framework and MTK IOMMU and MTK SMI both have been merged in v4.6-rc1.
> > 
> > [1]https://patchwork.linuxtv.org/patch/33734/
> > [2]https://chromium-review.googlesource.com/#/c/245241/
> > 
> > ==================
> >  Device interface
> > ==================
> > 
> > In principle the driver bases on v4l2 memory-to-memory framework:
> > it provides a single video node and each opened file handle gets its own private context with separate
> > buffer queues. Each context consist of 2 buffer queues: OUTPUT (for source buffers, i.e. bitstream)
> > and CAPTURE (for destination buffers, i.e. decoded video frames).
> > OUTPUT and CAPTURE buffer could be MMAP or DMABUF memory type.
> > VIDIOC_G_CTRL and VIDIOC_G_EXT_CTRLS return V4L2_CID_MIN_BUFFERS_FOR_CAPTURE only when dirver in MTK_STATE_HEADER
> > state, or it will return EAGAIN.
> > Driver do not support subscribe event for control 'User Controls' for now.
> > And it default support export DMABUF for other display drivers.
> > 
> > Change in v3:
> > 1. Refine vdec hw clock setting
> > 2. Refine vp9 codec driver
> > 3. Refine v4l2 codec driver
> > 
> > Change in v2:
> > 1. Add documentation for V4L2_PIX_FMT_MT21
> > 2. Remove DRM_FORMAT_MT21
> > 2. Refine code according to review comments
> > 
> > v4l2-compliance test output:
> > localhost Encode # ./v4l2-compliance -d /dev/video0
> > Driver Info:
> >         Driver name   : mtk-vcodec-dec
> >         Card type     : platform:mt8173
> >         Bus info      : platform:mt8173
> >         Driver version: 4.4.0
> >         Capabilities  : 0x84204000
> >                 Video Memory-to-Memory Multiplanar
> >                 Streaming
> >                 Extended Pix Format
> >                 Device Capabilities
> >         Device Caps   : 0x04204000
> >                 Video Memory-to-Memory Multiplanar
> >                 Streaming
> >                 Extended Pix Format
> > 
> > Compliance test for device /dev/video0 (not using libv4l2):
> > 
> > Required ioctls:
> >         test VIDIOC_QUERYCAP: OK
> > 
> > Allow for multiple opens:
> >         test second video open: OK
> >         test VIDIOC_QUERYCAP: OK
> >         test VIDIOC_G/S_PRIORITY: OK
> > 
> > Debug ioctls:
> >         test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> >         test VIDIOC_LOG_STATUS: OK (Not Supported)
> > 
> > Input ioctls:
> >         test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> >         test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >         test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> >         test VIDIOC_ENUMAUDIO: OK (Not Supported)
> >         test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> >         test VIDIOC_G/S_AUDIO: OK (Not Supported)
> >         Inputs: 0 Audio Inputs: 0 Tuners: 0
> > 
> > Output ioctls:
> >         test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> >         test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >         test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> >         test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> >         test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> >         Outputs: 0 Audio Outputs: 0 Modulators: 0
> > 
> > Input/Output configuration ioctls:
> >         test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> >         test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> >         test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> >         test VIDIOC_G/S_EDID: OK (Not Supported)
> > 
> >         Control ioctls:
> >                 test VIDIOC_QUERYCTRL/MENU: OK
> >                 fail: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-controls.cpp(357): g_ctrl returned an error (11)
> >                 test VIDIOC_G/S_CTRL: FAIL
> >                 fail: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-controls.cpp(579): g_ext_ctrls returned an error (11)
> >                 test VIDIOC_G/S/TRY_EXT_CTRLS: FAIL
> >                 fail: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-controls.cpp(721): subscribe event for control 'User Controls' failed
> >                 test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: FAIL
> >                 test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> >                 Standard Controls: 2 Private Controls: 0
> > 
> >         Format ioctls:
> >                 test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> >                 test VIDIOC_G/S_PARM: OK (Not Supported)
> >                 test VIDIOC_G_FBUF: OK (Not Supported)
> >                 fail: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-formats.cpp(405): expected EINVAL, but got 11 when getting format for buftype 9
> >                 test VIDIOC_G_FMT: FAIL
> >                 test VIDIOC_TRY_FMT: OK (Not Supported)
> >                 test VIDIOC_S_FMT: OK (Not Supported)
> >                 test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> > 
> >         Codec ioctls:
> >                 test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> >                 test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> >                 test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> > 
> >         Buffer ioctls:
> >                 test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
> >                 fail: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-buffers.cpp(500): q.has_expbuf(node)
> >                 test VIDIOC_EXPBUF: FAIL
> > 
> > 
> > Total: 38, Succeeded: 33, Failed: 5, Warnings: 0
> > 
> > 
> > Andrew-CT Chen (1):
> >   VPU: mediatek: Add decode support
> > 
> > Tiffany Lin (8):
> >   v4l: add Mediatek compressed video block format
> >   DocBook/v4l: Add compressed video formats used on MT8173 codec driver
> >   dt-bindings: Add a binding for Mediatek Video Decoder
> >   vcodec: mediatek: Add Mediatek V4L2 Video Decoder Driver
> >   vcodec: mediatek: Add Mediatek H264 Video Decoder Driver
> >   vcodec: mediatek: Add Mediatek VP8 Video Decoder Driver
> >   vcodec: mediatek: Add Mediatek VP9 Video Decoder Driver
> >   arm64: dts: mediatek: Add Video Decoder for MT8173
> > 
> >  Documentation/DocBook/media/v4l/pixfmt.xml         |    6 +
> >  .../devicetree/bindings/media/mediatek-vcodec.txt  |   57 +-
> >  arch/arm64/boot/dts/mediatek/mt8173.dtsi           |   44 +
> >  drivers/media/platform/mtk-vcodec/Makefile         |   15 +-
> >  drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c | 1348 ++++++++++++++++++++
> >  drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.h |   88 ++
> >  .../media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c |  408 ++++++
> >  .../media/platform/mtk-vcodec/mtk_vcodec_dec_pm.c  |  206 +++
> >  .../media/platform/mtk-vcodec/mtk_vcodec_dec_pm.h  |   28 +
> >  drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h |   88 +-
> >  .../media/platform/mtk-vcodec/mtk_vcodec_intr.c    |    4 +-
> >  .../media/platform/mtk-vcodec/mtk_vcodec_intr.h    |    2 +-
> >  .../media/platform/mtk-vcodec/mtk_vcodec_util.c    |   31 +-
> >  .../media/platform/mtk-vcodec/mtk_vcodec_util.h    |    7 +-
> >  .../media/platform/mtk-vcodec/vdec/vdec_h264_if.c  |  503 ++++++++
> >  .../media/platform/mtk-vcodec/vdec/vdec_vp8_if.c   |  630 +++++++++
> >  .../media/platform/mtk-vcodec/vdec/vdec_vp9_if.c   |  967 ++++++++++++++
> >  drivers/media/platform/mtk-vcodec/vdec_drv_base.h  |   56 +
> >  drivers/media/platform/mtk-vcodec/vdec_drv_if.c    |  117 ++
> >  drivers/media/platform/mtk-vcodec/vdec_drv_if.h    |  101 ++
> >  drivers/media/platform/mtk-vcodec/vdec_ipi_msg.h   |  103 ++
> >  drivers/media/platform/mtk-vcodec/vdec_vpu_if.c    |  168 +++
> >  drivers/media/platform/mtk-vcodec/vdec_vpu_if.h    |   96 ++
> >  drivers/media/platform/mtk-vpu/mtk_vpu.c           |   12 +
> >  drivers/media/platform/mtk-vpu/mtk_vpu.h           |   27 +
> >  include/uapi/linux/videodev2.h                     |    1 +
> >  26 files changed, 5090 insertions(+), 23 deletions(-)
> >  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c
> >  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.h
> >  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c
> >  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_pm.c
> >  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_pm.h
> >  create mode 100644 drivers/media/platform/mtk-vcodec/vdec/vdec_h264_if.c
> >  create mode 100644 drivers/media/platform/mtk-vcodec/vdec/vdec_vp8_if.c
> >  create mode 100644 drivers/media/platform/mtk-vcodec/vdec/vdec_vp9_if.c
> >  create mode 100644 drivers/media/platform/mtk-vcodec/vdec_drv_base.h
> >  create mode 100644 drivers/media/platform/mtk-vcodec/vdec_drv_if.c
> >  create mode 100644 drivers/media/platform/mtk-vcodec/vdec_drv_if.h
> >  create mode 100644 drivers/media/platform/mtk-vcodec/vdec_ipi_msg.h
> >  create mode 100644 drivers/media/platform/mtk-vcodec/vdec_vpu_if.c
> >  create mode 100644 drivers/media/platform/mtk-vcodec/vdec_vpu_if.h
> > 

[PATCH v3 0/9] Add MT8173 Video Decoder Driver

[Hans Verkuil]

On 08/15/2016 11:03 AM, Tiffany Lin wrote:
> Hi Hans,
> 
> I upstream v4 on 8/10, I don't know why this version is not shown in
> https://patchwork.kernel.org.
> But I could see it in other link.
> http://www.spinics.net/lists/arm-kernel/msg523095.html
> I refine DocBook and vb2 queue_setup function in v4.
> 
> For the MT21 format, if I put MT21 into separate patch, this patch
> series will build fail, I was confused how to fix this issue?

Just don't implement that pixelformat yet. I.e. everything else can be
implemented, just don't add MT21 to the list of pixelformats. Do that in
a separate patch at the end.

If that pixelformat isn't exposed, then nobody will use it either :-)

Regards,

	Hans

[PATCH v3 0/9] Add MT8173 Video Decoder Driver

[Tiffany Lin]

Hi Hans,


On Mon, 2016-08-15 at 11:07 +0200, Hans Verkuil wrote:
> On 08/15/2016 11:03 AM, Tiffany Lin wrote:
> > Hi Hans,
> > 
> > I upstream v4 on 8/10, I don't know why this version is not shown in
> > https://patchwork.kernel.org.
> > But I could see it in other link.
> > http://www.spinics.net/lists/arm-kernel/msg523095.html
> > I refine DocBook and vb2 queue_setup function in v4.
> > 
> > For the MT21 format, if I put MT21 into separate patch, this patch
> > series will build fail, I was confused how to fix this issue?
> 
> Just don't implement that pixelformat yet. I.e. everything else can be
> implemented, just don't add MT21 to the list of pixelformats. Do that in
> a separate patch at the end.
> 
> If that pixelformat isn't exposed, then nobody will use it either :-)
> 
Got it now.
Thanks a lot.


best regards,
Tiffany

> Regards,
> 
> 	Hans