All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH v4 0/8] Add MT8173 Video Encoder Driver and VPU Driver
@ 2016-02-04 11:34 ` Tiffany Lin
  0 siblings, 0 replies; 102+ messages in thread
From: Tiffany Lin @ 2016-02-04 11:34 UTC (permalink / raw)
  To: Hans Verkuil, daniel.thompson, Rob Herring,
	Mauro Carvalho Chehab, Matthias Brugger, Daniel Kurtz,
	Pawel Osciak
  Cc: Eddie Huang, Yingjoe Chen, devicetree, linux-kernel,
	linux-arm-kernel, linux-media, linux-mediatek, PoChun.Lin,
	Tiffany Lin


==============
 Introduction
==============

The purpose of this series is to add the driver for video codec hw embedded in the Mediatek's MT8173 SoCs.
Mediatek Video Codec is able to handle video encoding of in a range of formats.

This patch series also include VPU driver. Mediatek Video Codec driver rely on VPU driver to load, communicate with VPU.

Internally the driver uses videobuf2 framework and MTK IOMMU and MTK SMI.
MTK IOMMU[1] and MTK SMI[2] have not yet been merged, but we wanted to start discussion about the driver earlier so it could be merged sooner.

[1]https://patchwork.kernel.org/patch/8120921/
[2]https://patchwork.kernel.org/patch/8120941/


==================
 Device interface
==================

In principle the driver bases on v4l2 memory-to-memory framework:
it provides a single video node and each opened file handle gets its own private context with separate buffer queues. Each context consist of 2 buffer queues: OUTPUT (for source buffers, i.e. raw video frames) and CAPTURE (for destination buffers, i.e. encoded video frames).

==============================
 VPU (Video Processor Unit)
==============================
The VPU driver for hw video codec embedded in Mediatek's MT8173 SOCs.
It is able to handle video decoding/encoding in a range of formats.
The driver provides with VPU firmware download, memory management and the communication interface between CPU and VPU.
For VPU initialization, it will create virtual memory for CPU access and physical address for VPU hw device access. When a decode/encode instance opens a device node, vpu driver will download vpu firmware to the device.
A decode/encode instant will decode/encode a frame using VPU interface to interrupt vpu to handle decoding/encoding jobs.

Please have a look at the code and comments will be very much appreciated.


Change in v4:
Vcodec Part
1. Remove MTK_ENCODE_PARAM_SKIP_FRAME support
2. Remove MTK_ENCODE_PARAM_FRAME_TYPE and change to use V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME[3]
3. Refine Encoder HW clock source
4. Refine debug log
5. Add watchdog support
6. With patch "media: v4l2-compat-ioctl32: fix missing length copy in put_v4l2_buffer32"[4], v4l2-compliance test passed[5] in v4.4-rc5 

VPU Part
1. These two patches were Acked-by: Rob Herring <robh@kernel.org> in v3
   [PATCH v3 1/8] dt-bindings: Add a binding for Mediatek Video Processor
   [PATCH v3 3/8] arm64: dts: mediatek: Add node for Mediatek Video Processor Unit
   Because we were wrong about how the hardware works, there is no connection between VPU and IOMMU HW
   We remove VPU attaching to IOMMU
2. Support VPU running on 4GB DRAM system
3. Support VPU watchdog reset
4. Refine for coding style 

[3]https://patchwork.linuxtv.org/patch/32670/
[4] https://patchwork.linuxtv.org/patch/32631/
[5]localhost ~ # /usr/bin/v4l2-compliance -d /dev/video1
Driver Info:
        Driver name   : mtk-vcodec-en
        Card type     : platform:mt817
        Bus info      : platform:mt817
        Driver version: 4.4.0
        Capabilities  : 0x84204000
                Video Memory-to-Memory Multiplanar
                Streaming
                Extended Pix Format
                Device Capabilities
        Device Caps   : 0x04204000
                Video Memory-to-Memory Multiplanar
                Streaming
                Extended Pix Format

Compliance test for device /dev/video1 (not using libv4l2):

Required ioctls:
        test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
        test second video open: OK
        test VIDIOC_QUERYCAP: OK
        test VIDIOC_G/S_PRIORITY: OK

Debug ioctls:
        test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
        test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
        test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
        test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
        test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
        test VIDIOC_ENUMAUDIO: OK (Not Supported)
        test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
        test VIDIOC_G/S_AUDIO: OK (Not Supported)
        Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
        test VIDIOC_G/S_MODULATOR: OK (Not Supported)
        test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
        test VIDIOC_ENUMAUDOUT: OK (Not Supported)
        test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
        test VIDIOC_G/S_AUDOUT: OK (Not Supported)
        Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
        test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
        test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
        test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
        test VIDIOC_G/S_EDID: OK (Not Supported)

        Control ioctls:
                test VIDIOC_QUERYCTRL/MENU: OK
                test VIDIOC_G/S_CTRL: OK
                test VIDIOC_G/S/TRY_EXT_CTRLS: OK
                test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
                test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
                Standard Controls: 12 Private Controls: 0

        Format ioctls:
                test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
                test VIDIOC_G/S_PARM: OK (Not Supported)
                test VIDIOC_G_FBUF: OK (Not Supported)
                test VIDIOC_G_FMT: OK
                test VIDIOC_TRY_FMT: OK
                test VIDIOC_S_FMT: OK
                test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)

        Codec ioctls:
                test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
                test VIDIOC_G_ENC_INDEX: OK (Not Supported)
                test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

        Buffer ioctls:
                warn: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-buffers.cpp(475): VIDIOC_CREATE_BUFS not supported
                warn: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-buffers.cpp(475): VIDIOC_CREATE_BUFS not supported
                test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
                test VIDIOC_EXPBUF: OK

Total: 38, Succeeded: 38, Failed: 0, Warnings: 2


Change in v3:
1.Refine code to pass v4l2-compliance test, now it still has 2 issues 2.Refine code according to latest MTK IOMMU patches[1] 3.Remove MFC51 specific CIDs and add MTK specific CIDs for for keyframe and
  skip I-frame
4.Refine code according to review comments

Below is the v1.6 version v4l2-compliance report for the mt8173 encoder driver.
Now there are still 2 test fail in v1.6.
For VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF, we directly use v4l2_m2m_ioctl_* functions, but it still fail. It pass in kernel 3.18 but fail in kernel 4.4.
We will try v1.8 in next version.
VIDIOC_EXPBUF is becuase we support all three memory types VB2_DMABUF, VB2_MMAP and VB2_USERPTR. VIDIOC_EXPBUF only allowed when only VB2_MMAP supported.
localhost ~ # /usr/bin/v4l2-compliance -d /dev/video1 Driver Info:
        Driver name   : mtk-vcodec-en
        Card type     : platform:mt817
        Bus info      : platform:mt817
        Driver version: 4.4.0
        Capabilities  : 0x84204000
                Video Memory-to-Memory Multiplanar
                Streaming
                Extended Pix Format
                Device Capabilities
        Device Caps   : 0x04204000
                Video Memory-to-Memory Multiplanar
                Streaming
                Extended Pix Format

Compliance test for device /dev/video1 (not using libv4l2):

Required ioctls:
        test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
        test second video open: OK
        test VIDIOC_QUERYCAP: OK
        test VIDIOC_G/S_PRIORITY: OK

Debug ioctls:
        test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
        test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
        test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
        test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
        test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
        test VIDIOC_ENUMAUDIO: OK (Not Supported)
        test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
        test VIDIOC_G/S_AUDIO: OK (Not Supported)
        Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
        test VIDIOC_G/S_MODULATOR: OK (Not Supported)
        test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
        test VIDIOC_ENUMAUDOUT: OK (Not Supported)
        test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
        test VIDIOC_G/S_AUDOUT: OK (Not Supported)
        Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
        test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
        test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
        test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
        test VIDIOC_G/S_EDID: OK (Not Supported)

        Control ioctls:
                test VIDIOC_QUERYCTRL/MENU: OK
                test VIDIOC_G/S_CTRL: OK
                test VIDIOC_G/S/TRY_EXT_CTRLS: OK
                test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
                test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
                Standard Controls: 11 Private Controls: 2

        Format ioctls:
                test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
                test VIDIOC_G/S_PARM: OK (Not Supported)
                test VIDIOC_G_FBUF: OK (Not Supported)
                test VIDIOC_G_FMT: OK
                test VIDIOC_TRY_FMT: OK
                test VIDIOC_S_FMT: OK
                test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)

        Codec ioctls:
                test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
                test VIDIOC_G_ENC_INDEX: OK (Not Supported)
                test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

        Buffer ioctls:
                fail: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-buffers.cpp(266): vp->length == 0
                fail: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-buffers.cpp(335): buf.check(Unqueued, i)
                fail: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-buffers.cpp(420): testQueryBuf(node, i, q.g_buffers())
                test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
                fail: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-buffers.cpp(500): q.has_expbuf(node)
                test VIDIOC_EXPBUF: FAIL
Total: 38, Succeeded: 36, Failed: 2, Warnings: 0


Change in v2:
Vcodec Part
1.Remove common and include directory in mtk-vcodec 2.Refine vb2ops_venc_start_streaming and vb2ops_venc_stop_streaming and state machine 3.Remove venc_if_init and venc_if_deinit 4.Refine debug message 5.Refine lab and vpu decription in mediatek-vcodec.txt

VPU Part
1. Modify VPU Kconfig
2. Move encoder header files to other patch sets 3. Remove marcos for extended virtual/iova address 4. Change register and variable names 5. Add a reference counter for VPU watchdog 6. Remove one busy waiting in function vpu_ipi_send 7. Operate VPU clock in VPU driver (not called by encoder drivers) 8. Refine memory mapping, firmware download and extended memory allocation/free functions 9. Release more allocated resources in driver remove function



Andrew-CT Chen (2):
  dt-bindings: Add a binding for Mediatek Video Processor
  [Media] vcodec: mediatek: Add Mediatek V4L2 Video Encoder Driver

Tiffany Lin (6):
  [media] VPU: mediatek: support Mediatek VPU
  arm64: dts: mediatek: Add node for Mediatek Video Processor Unit
  dt-bindings: Add a binding for Mediatek Video Encoder
  [media] vcodec: mediatek: Add Mediatek VP8 Video Encoder Driver
  [media] vcodec: mediatek: Add Mediatek H264 Video Encoder Driver
  arm64: dts: mediatek: Add Video Encoder for MT8173

 .../devicetree/bindings/media/mediatek-vcodec.txt  |   59 +
 .../devicetree/bindings/media/mediatek-vpu.txt     |   31 +
 arch/arm64/boot/dts/mediatek/mt8173.dtsi           |   62 +
 drivers/media/platform/Kconfig                     |   20 +
 drivers/media/platform/Makefile                    |    4 +
 drivers/media/platform/mtk-vcodec/Makefile         |   11 +
 .../media/platform/mtk-vcodec/h264_enc/Makefile    |    6 +
 .../platform/mtk-vcodec/h264_enc/venc_h264_if.c    |  540 ++++++++
 .../platform/mtk-vcodec/h264_enc/venc_h264_if.h    |  165 +++
 .../platform/mtk-vcodec/h264_enc/venc_h264_vpu.c   |  309 +++++
 .../platform/mtk-vcodec/h264_enc/venc_h264_vpu.h   |   30 +
 drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h |  388 ++++++
 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c | 1380 ++++++++++++++++++++
 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h |   46 +
 .../media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c |  476 +++++++
 .../media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c  |  132 ++
 .../media/platform/mtk-vcodec/mtk_vcodec_intr.c    |  102 ++
 .../media/platform/mtk-vcodec/mtk_vcodec_intr.h    |   29 +
 drivers/media/platform/mtk-vcodec/mtk_vcodec_pm.h  |   26 +
 .../media/platform/mtk-vcodec/mtk_vcodec_util.c    |  106 ++
 .../media/platform/mtk-vcodec/mtk_vcodec_util.h    |   85 ++
 drivers/media/platform/mtk-vcodec/venc_drv_base.h  |   62 +
 drivers/media/platform/mtk-vcodec/venc_drv_if.c    |  106 ++
 drivers/media/platform/mtk-vcodec/venc_drv_if.h    |  175 +++
 drivers/media/platform/mtk-vcodec/venc_ipi_msg.h   |  212 +++
 drivers/media/platform/mtk-vcodec/vp8_enc/Makefile |    6 +
 .../platform/mtk-vcodec/vp8_enc/venc_vp8_if.c      |  422 ++++++
 .../platform/mtk-vcodec/vp8_enc/venc_vp8_if.h      |  149 +++
 .../platform/mtk-vcodec/vp8_enc/venc_vp8_vpu.c     |  240 ++++
 .../platform/mtk-vcodec/vp8_enc/venc_vp8_vpu.h     |   28 +
 drivers/media/platform/mtk-vpu/Makefile            |    1 +
 drivers/media/platform/mtk-vpu/mtk_vpu.c           |  994 ++++++++++++++
 drivers/media/platform/mtk-vpu/mtk_vpu.h           |  167 +++
 include/uapi/linux/v4l2-controls.h                 |    4 +
 34 files changed, 6573 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/media/mediatek-vcodec.txt
 create mode 100644 Documentation/devicetree/bindings/media/mediatek-vpu.txt
 create mode 100644 drivers/media/platform/mtk-vcodec/Makefile
 create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/Makefile
 create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.c
 create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.h
 create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.c
 create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.h
 create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
 create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
 create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h
 create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
 create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
 create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.c
 create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.h
 create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_pm.h
 create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c
 create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
 create mode 100644 drivers/media/platform/mtk-vcodec/venc_drv_base.h
 create mode 100644 drivers/media/platform/mtk-vcodec/venc_drv_if.c
 create mode 100644 drivers/media/platform/mtk-vcodec/venc_drv_if.h
 create mode 100644 drivers/media/platform/mtk-vcodec/venc_ipi_msg.h
 create mode 100644 drivers/media/platform/mtk-vcodec/vp8_enc/Makefile
 create mode 100644 drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_if.c
 create mode 100644 drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_if.h
 create mode 100644 drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_vpu.c
 create mode 100644 drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_vpu.h
 create mode 100644 drivers/media/platform/mtk-vpu/Makefile
 create mode 100644 drivers/media/platform/mtk-vpu/mtk_vpu.c
 create mode 100644 drivers/media/platform/mtk-vpu/mtk_vpu.h
 mode change 100644 => 100755 include/uapi/linux/v4l2-controls.h

-- 
1.7.9.5

^ permalink raw reply	[flat|nested] 102+ messages in thread

* [PATCH v4 0/8] Add MT8173 Video Encoder Driver and VPU Driver
@ 2016-02-04 11:34 ` Tiffany Lin
  0 siblings, 0 replies; 102+ messages in thread
From: Tiffany Lin @ 2016-02-04 11:34 UTC (permalink / raw)
  To: Hans Verkuil, daniel.thompson, Rob Herring,
	Mauro Carvalho Chehab, Matthias Brugger, Daniel Kurtz,
	Pawel Osciak
  Cc: Eddie Huang, Yingjoe Chen, devicetree, linux-kernel,
	linux-arm-kernel, linux-media, linux-mediatek, PoChun.Lin,
	Tiffany Lin


==============
 Introduction
==============

The purpose of this series is to add the driver for video codec hw embedded in the Mediatek's MT8173 SoCs.
Mediatek Video Codec is able to handle video encoding of in a range of formats.

This patch series also include VPU driver. Mediatek Video Codec driver rely on VPU driver to load, communicate with VPU.

Internally the driver uses videobuf2 framework and MTK IOMMU and MTK SMI.
MTK IOMMU[1] and MTK SMI[2] have not yet been merged, but we wanted to start discussion about the driver earlier so it could be merged sooner.

[1]https://patchwork.kernel.org/patch/8120921/
[2]https://patchwork.kernel.org/patch/8120941/


==================
 Device interface
==================

In principle the driver bases on v4l2 memory-to-memory framework:
it provides a single video node and each opened file handle gets its own private context with separate buffer queues. Each context consist of 2 buffer queues: OUTPUT (for source buffers, i.e. raw video frames) and CAPTURE (for destination buffers, i.e. encoded video frames).

==============================
 VPU (Video Processor Unit)
==============================
The VPU driver for hw video codec embedded in Mediatek's MT8173 SOCs.
It is able to handle video decoding/encoding in a range of formats.
The driver provides with VPU firmware download, memory management and the communication interface between CPU and VPU.
For VPU initialization, it will create virtual memory for CPU access and physical address for VPU hw device access. When a decode/encode instance opens a device node, vpu driver will download vpu firmware to the device.
A decode/encode instant will decode/encode a frame using VPU interface to interrupt vpu to handle decoding/encoding jobs.

Please have a look at the code and comments will be very much appreciated.


Change in v4:
Vcodec Part
1. Remove MTK_ENCODE_PARAM_SKIP_FRAME support
2. Remove MTK_ENCODE_PARAM_FRAME_TYPE and change to use V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME[3]
3. Refine Encoder HW clock source
4. Refine debug log
5. Add watchdog support
6. With patch "media: v4l2-compat-ioctl32: fix missing length copy in put_v4l2_buffer32"[4], v4l2-compliance test passed[5] in v4.4-rc5 

VPU Part
1. These two patches were Acked-by: Rob Herring <robh@kernel.org> in v3
   [PATCH v3 1/8] dt-bindings: Add a binding for Mediatek Video Processor
   [PATCH v3 3/8] arm64: dts: mediatek: Add node for Mediatek Video Processor Unit
   Because we were wrong about how the hardware works, there is no connection between VPU and IOMMU HW
   We remove VPU attaching to IOMMU
2. Support VPU running on 4GB DRAM system
3. Support VPU watchdog reset
4. Refine for coding style 

[3]https://patchwork.linuxtv.org/patch/32670/
[4] https://patchwork.linuxtv.org/patch/32631/
[5]localhost ~ # /usr/bin/v4l2-compliance -d /dev/video1
Driver Info:
        Driver name   : mtk-vcodec-en
        Card type     : platform:mt817
        Bus info      : platform:mt817
        Driver version: 4.4.0
        Capabilities  : 0x84204000
                Video Memory-to-Memory Multiplanar
                Streaming
                Extended Pix Format
                Device Capabilities
        Device Caps   : 0x04204000
                Video Memory-to-Memory Multiplanar
                Streaming
                Extended Pix Format

Compliance test for device /dev/video1 (not using libv4l2):

Required ioctls:
        test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
        test second video open: OK
        test VIDIOC_QUERYCAP: OK
        test VIDIOC_G/S_PRIORITY: OK

Debug ioctls:
        test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
        test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
        test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
        test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
        test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
        test VIDIOC_ENUMAUDIO: OK (Not Supported)
        test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
        test VIDIOC_G/S_AUDIO: OK (Not Supported)
        Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
        test VIDIOC_G/S_MODULATOR: OK (Not Supported)
        test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
        test VIDIOC_ENUMAUDOUT: OK (Not Supported)
        test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
        test VIDIOC_G/S_AUDOUT: OK (Not Supported)
        Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
        test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
        test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
        test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
        test VIDIOC_G/S_EDID: OK (Not Supported)

        Control ioctls:
                test VIDIOC_QUERYCTRL/MENU: OK
                test VIDIOC_G/S_CTRL: OK
                test VIDIOC_G/S/TRY_EXT_CTRLS: OK
                test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
                test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
                Standard Controls: 12 Private Controls: 0

        Format ioctls:
                test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
                test VIDIOC_G/S_PARM: OK (Not Supported)
                test VIDIOC_G_FBUF: OK (Not Supported)
                test VIDIOC_G_FMT: OK
                test VIDIOC_TRY_FMT: OK
                test VIDIOC_S_FMT: OK
                test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)

        Codec ioctls:
                test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
                test VIDIOC_G_ENC_INDEX: OK (Not Supported)
                test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

        Buffer ioctls:
                warn: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-buffers.cpp(475): VIDIOC_CREATE_BUFS not supported
                warn: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-buffers.cpp(475): VIDIOC_CREATE_BUFS not supported
                test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
                test VIDIOC_EXPBUF: OK

Total: 38, Succeeded: 38, Failed: 0, Warnings: 2


Change in v3:
1.Refine code to pass v4l2-compliance test, now it still has 2 issues 2.Refine code according to latest MTK IOMMU patches[1] 3.Remove MFC51 specific CIDs and add MTK specific CIDs for for keyframe and
  skip I-frame
4.Refine code according to review comments

Below is the v1.6 version v4l2-compliance report for the mt8173 encoder driver.
Now there are still 2 test fail in v1.6.
For VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF, we directly use v4l2_m2m_ioctl_* functions, but it still fail. It pass in kernel 3.18 but fail in kernel 4.4.
We will try v1.8 in next version.
VIDIOC_EXPBUF is becuase we support all three memory types VB2_DMABUF, VB2_MMAP and VB2_USERPTR. VIDIOC_EXPBUF only allowed when only VB2_MMAP supported.
localhost ~ # /usr/bin/v4l2-compliance -d /dev/video1 Driver Info:
        Driver name   : mtk-vcodec-en
        Card type     : platform:mt817
        Bus info      : platform:mt817
        Driver version: 4.4.0
        Capabilities  : 0x84204000
                Video Memory-to-Memory Multiplanar
                Streaming
                Extended Pix Format
                Device Capabilities
        Device Caps   : 0x04204000
                Video Memory-to-Memory Multiplanar
                Streaming
                Extended Pix Format

Compliance test for device /dev/video1 (not using libv4l2):

Required ioctls:
        test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
        test second video open: OK
        test VIDIOC_QUERYCAP: OK
        test VIDIOC_G/S_PRIORITY: OK

Debug ioctls:
        test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
        test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
        test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
        test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
        test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
        test VIDIOC_ENUMAUDIO: OK (Not Supported)
        test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
        test VIDIOC_G/S_AUDIO: OK (Not Supported)
        Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
        test VIDIOC_G/S_MODULATOR: OK (Not Supported)
        test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
        test VIDIOC_ENUMAUDOUT: OK (Not Supported)
        test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
        test VIDIOC_G/S_AUDOUT: OK (Not Supported)
        Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
        test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
        test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
        test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
        test VIDIOC_G/S_EDID: OK (Not Supported)

        Control ioctls:
                test VIDIOC_QUERYCTRL/MENU: OK
                test VIDIOC_G/S_CTRL: OK
                test VIDIOC_G/S/TRY_EXT_CTRLS: OK
                test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
                test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
                Standard Controls: 11 Private Controls: 2

        Format ioctls:
                test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
                test VIDIOC_G/S_PARM: OK (Not Supported)
                test VIDIOC_G_FBUF: OK (Not Supported)
                test VIDIOC_G_FMT: OK
                test VIDIOC_TRY_FMT: OK
                test VIDIOC_S_FMT: OK
                test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)

        Codec ioctls:
                test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
                test VIDIOC_G_ENC_INDEX: OK (Not Supported)
                test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

        Buffer ioctls:
                fail: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-buffers.cpp(266): vp->length == 0
                fail: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-buffers.cpp(335): buf.check(Unqueued, i)
                fail: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-buffers.cpp(420): testQueryBuf(node, i, q.g_buffers())
                test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
                fail: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-buffers.cpp(500): q.has_expbuf(node)
                test VIDIOC_EXPBUF: FAIL
Total: 38, Succeeded: 36, Failed: 2, Warnings: 0


Change in v2:
Vcodec Part
1.Remove common and include directory in mtk-vcodec 2.Refine vb2ops_venc_start_streaming and vb2ops_venc_stop_streaming and state machine 3.Remove venc_if_init and venc_if_deinit 4.Refine debug message 5.Refine lab and vpu decription in mediatek-vcodec.txt

VPU Part
1. Modify VPU Kconfig
2. Move encoder header files to other patch sets 3. Remove marcos for extended virtual/iova address 4. Change register and variable names 5. Add a reference counter for VPU watchdog 6. Remove one busy waiting in function vpu_ipi_send 7. Operate VPU clock in VPU driver (not called by encoder drivers) 8. Refine memory mapping, firmware download and extended memory allocation/free functions 9. Release more allocated resources in driver remove function



Andrew-CT Chen (2):
  dt-bindings: Add a binding for Mediatek Video Processor
  [Media] vcodec: mediatek: Add Mediatek V4L2 Video Encoder Driver

Tiffany Lin (6):
  [media] VPU: mediatek: support Mediatek VPU
  arm64: dts: mediatek: Add node for Mediatek Video Processor Unit
  dt-bindings: Add a binding for Mediatek Video Encoder
  [media] vcodec: mediatek: Add Mediatek VP8 Video Encoder Driver
  [media] vcodec: mediatek: Add Mediatek H264 Video Encoder Driver
  arm64: dts: mediatek: Add Video Encoder for MT8173

 .../devicetree/bindings/media/mediatek-vcodec.txt  |   59 +
 .../devicetree/bindings/media/mediatek-vpu.txt     |   31 +
 arch/arm64/boot/dts/mediatek/mt8173.dtsi           |   62 +
 drivers/media/platform/Kconfig                     |   20 +
 drivers/media/platform/Makefile                    |    4 +
 drivers/media/platform/mtk-vcodec/Makefile         |   11 +
 .../media/platform/mtk-vcodec/h264_enc/Makefile    |    6 +
 .../platform/mtk-vcodec/h264_enc/venc_h264_if.c    |  540 ++++++++
 .../platform/mtk-vcodec/h264_enc/venc_h264_if.h    |  165 +++
 .../platform/mtk-vcodec/h264_enc/venc_h264_vpu.c   |  309 +++++
 .../platform/mtk-vcodec/h264_enc/venc_h264_vpu.h   |   30 +
 drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h |  388 ++++++
 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c | 1380 ++++++++++++++++++++
 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h |   46 +
 .../media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c |  476 +++++++
 .../media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c  |  132 ++
 .../media/platform/mtk-vcodec/mtk_vcodec_intr.c    |  102 ++
 .../media/platform/mtk-vcodec/mtk_vcodec_intr.h    |   29 +
 drivers/media/platform/mtk-vcodec/mtk_vcodec_pm.h  |   26 +
 .../media/platform/mtk-vcodec/mtk_vcodec_util.c    |  106 ++
 .../media/platform/mtk-vcodec/mtk_vcodec_util.h    |   85 ++
 drivers/media/platform/mtk-vcodec/venc_drv_base.h  |   62 +
 drivers/media/platform/mtk-vcodec/venc_drv_if.c    |  106 ++
 drivers/media/platform/mtk-vcodec/venc_drv_if.h    |  175 +++
 drivers/media/platform/mtk-vcodec/venc_ipi_msg.h   |  212 +++
 drivers/media/platform/mtk-vcodec/vp8_enc/Makefile |    6 +
 .../platform/mtk-vcodec/vp8_enc/venc_vp8_if.c      |  422 ++++++
 .../platform/mtk-vcodec/vp8_enc/venc_vp8_if.h      |  149 +++
 .../platform/mtk-vcodec/vp8_enc/venc_vp8_vpu.c     |  240 ++++
 .../platform/mtk-vcodec/vp8_enc/venc_vp8_vpu.h     |   28 +
 drivers/media/platform/mtk-vpu/Makefile            |    1 +
 drivers/media/platform/mtk-vpu/mtk_vpu.c           |  994 ++++++++++++++
 drivers/media/platform/mtk-vpu/mtk_vpu.h           |  167 +++
 include/uapi/linux/v4l2-controls.h                 |    4 +
 34 files changed, 6573 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/media/mediatek-vcodec.txt
 create mode 100644 Documentation/devicetree/bindings/media/mediatek-vpu.txt
 create mode 100644 drivers/media/platform/mtk-vcodec/Makefile
 create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/Makefile
 create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.c
 create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.h
 create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.c
 create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.h
 create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
 create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
 create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h
 create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
 create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
 create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.c
 create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.h
 create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_pm.h
 create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c
 create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
 create mode 100644 drivers/media/platform/mtk-vcodec/venc_drv_base.h
 create mode 100644 drivers/media/platform/mtk-vcodec/venc_drv_if.c
 create mode 100644 drivers/media/platform/mtk-vcodec/venc_drv_if.h
 create mode 100644 drivers/media/platform/mtk-vcodec/venc_ipi_msg.h
 create mode 100644 drivers/media/platform/mtk-vcodec/vp8_enc/Makefile
 create mode 100644 drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_if.c
 create mode 100644 drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_if.h
 create mode 100644 drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_vpu.c
 create mode 100644 drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_vpu.h
 create mode 100644 drivers/media/platform/mtk-vpu/Makefile
 create mode 100644 drivers/media/platform/mtk-vpu/mtk_vpu.c
 create mode 100644 drivers/media/platform/mtk-vpu/mtk_vpu.h
 mode change 100644 => 100755 include/uapi/linux/v4l2-controls.h

-- 
1.7.9.5

^ permalink raw reply	[flat|nested] 102+ messages in thread

* [PATCH v4 0/8] Add MT8173 Video Encoder Driver and VPU Driver
@ 2016-02-04 11:34 ` Tiffany Lin
  0 siblings, 0 replies; 102+ messages in thread
From: Tiffany Lin @ 2016-02-04 11:34 UTC (permalink / raw)
  To: linux-arm-kernel


==============
 Introduction
==============

The purpose of this series is to add the driver for video codec hw embedded in the Mediatek's MT8173 SoCs.
Mediatek Video Codec is able to handle video encoding of in a range of formats.

This patch series also include VPU driver. Mediatek Video Codec driver rely on VPU driver to load, communicate with VPU.

Internally the driver uses videobuf2 framework and MTK IOMMU and MTK SMI.
MTK IOMMU[1] and MTK SMI[2] have not yet been merged, but we wanted to start discussion about the driver earlier so it could be merged sooner.

[1]https://patchwork.kernel.org/patch/8120921/
[2]https://patchwork.kernel.org/patch/8120941/


==================
 Device interface
==================

In principle the driver bases on v4l2 memory-to-memory framework:
it provides a single video node and each opened file handle gets its own private context with separate buffer queues. Each context consist of 2 buffer queues: OUTPUT (for source buffers, i.e. raw video frames) and CAPTURE (for destination buffers, i.e. encoded video frames).

==============================
 VPU (Video Processor Unit)
==============================
The VPU driver for hw video codec embedded in Mediatek's MT8173 SOCs.
It is able to handle video decoding/encoding in a range of formats.
The driver provides with VPU firmware download, memory management and the communication interface between CPU and VPU.
For VPU initialization, it will create virtual memory for CPU access and physical address for VPU hw device access. When a decode/encode instance opens a device node, vpu driver will download vpu firmware to the device.
A decode/encode instant will decode/encode a frame using VPU interface to interrupt vpu to handle decoding/encoding jobs.

Please have a look at the code and comments will be very much appreciated.


Change in v4:
Vcodec Part
1. Remove MTK_ENCODE_PARAM_SKIP_FRAME support
2. Remove MTK_ENCODE_PARAM_FRAME_TYPE and change to use V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME[3]
3. Refine Encoder HW clock source
4. Refine debug log
5. Add watchdog support
6. With patch "media: v4l2-compat-ioctl32: fix missing length copy in put_v4l2_buffer32"[4], v4l2-compliance test passed[5] in v4.4-rc5 

VPU Part
1. These two patches were Acked-by: Rob Herring <robh@kernel.org> in v3
   [PATCH v3 1/8] dt-bindings: Add a binding for Mediatek Video Processor
   [PATCH v3 3/8] arm64: dts: mediatek: Add node for Mediatek Video Processor Unit
   Because we were wrong about how the hardware works, there is no connection between VPU and IOMMU HW
   We remove VPU attaching to IOMMU
2. Support VPU running on 4GB DRAM system
3. Support VPU watchdog reset
4. Refine for coding style 

[3]https://patchwork.linuxtv.org/patch/32670/
[4] https://patchwork.linuxtv.org/patch/32631/
[5]localhost ~ # /usr/bin/v4l2-compliance -d /dev/video1
Driver Info:
        Driver name   : mtk-vcodec-en
        Card type     : platform:mt817
        Bus info      : platform:mt817
        Driver version: 4.4.0
        Capabilities  : 0x84204000
                Video Memory-to-Memory Multiplanar
                Streaming
                Extended Pix Format
                Device Capabilities
        Device Caps   : 0x04204000
                Video Memory-to-Memory Multiplanar
                Streaming
                Extended Pix Format

Compliance test for device /dev/video1 (not using libv4l2):

Required ioctls:
        test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
        test second video open: OK
        test VIDIOC_QUERYCAP: OK
        test VIDIOC_G/S_PRIORITY: OK

Debug ioctls:
        test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
        test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
        test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
        test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
        test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
        test VIDIOC_ENUMAUDIO: OK (Not Supported)
        test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
        test VIDIOC_G/S_AUDIO: OK (Not Supported)
        Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
        test VIDIOC_G/S_MODULATOR: OK (Not Supported)
        test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
        test VIDIOC_ENUMAUDOUT: OK (Not Supported)
        test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
        test VIDIOC_G/S_AUDOUT: OK (Not Supported)
        Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
        test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
        test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
        test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
        test VIDIOC_G/S_EDID: OK (Not Supported)

        Control ioctls:
                test VIDIOC_QUERYCTRL/MENU: OK
                test VIDIOC_G/S_CTRL: OK
                test VIDIOC_G/S/TRY_EXT_CTRLS: OK
                test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
                test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
                Standard Controls: 12 Private Controls: 0

        Format ioctls:
                test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
                test VIDIOC_G/S_PARM: OK (Not Supported)
                test VIDIOC_G_FBUF: OK (Not Supported)
                test VIDIOC_G_FMT: OK
                test VIDIOC_TRY_FMT: OK
                test VIDIOC_S_FMT: OK
                test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)

        Codec ioctls:
                test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
                test VIDIOC_G_ENC_INDEX: OK (Not Supported)
                test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

        Buffer ioctls:
                warn: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-buffers.cpp(475): VIDIOC_CREATE_BUFS not supported
                warn: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-buffers.cpp(475): VIDIOC_CREATE_BUFS not supported
                test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
                test VIDIOC_EXPBUF: OK

Total: 38, Succeeded: 38, Failed: 0, Warnings: 2


Change in v3:
1.Refine code to pass v4l2-compliance test, now it still has 2 issues 2.Refine code according to latest MTK IOMMU patches[1] 3.Remove MFC51 specific CIDs and add MTK specific CIDs for for keyframe and
  skip I-frame
4.Refine code according to review comments

Below is the v1.6 version v4l2-compliance report for the mt8173 encoder driver.
Now there are still 2 test fail in v1.6.
For VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF, we directly use v4l2_m2m_ioctl_* functions, but it still fail. It pass in kernel 3.18 but fail in kernel 4.4.
We will try v1.8 in next version.
VIDIOC_EXPBUF is becuase we support all three memory types VB2_DMABUF, VB2_MMAP and VB2_USERPTR. VIDIOC_EXPBUF only allowed when only VB2_MMAP supported.
localhost ~ # /usr/bin/v4l2-compliance -d /dev/video1 Driver Info:
        Driver name   : mtk-vcodec-en
        Card type     : platform:mt817
        Bus info      : platform:mt817
        Driver version: 4.4.0
        Capabilities  : 0x84204000
                Video Memory-to-Memory Multiplanar
                Streaming
                Extended Pix Format
                Device Capabilities
        Device Caps   : 0x04204000
                Video Memory-to-Memory Multiplanar
                Streaming
                Extended Pix Format

Compliance test for device /dev/video1 (not using libv4l2):

Required ioctls:
        test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
        test second video open: OK
        test VIDIOC_QUERYCAP: OK
        test VIDIOC_G/S_PRIORITY: OK

Debug ioctls:
        test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
        test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
        test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
        test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
        test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
        test VIDIOC_ENUMAUDIO: OK (Not Supported)
        test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
        test VIDIOC_G/S_AUDIO: OK (Not Supported)
        Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
        test VIDIOC_G/S_MODULATOR: OK (Not Supported)
        test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
        test VIDIOC_ENUMAUDOUT: OK (Not Supported)
        test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
        test VIDIOC_G/S_AUDOUT: OK (Not Supported)
        Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
        test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
        test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
        test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
        test VIDIOC_G/S_EDID: OK (Not Supported)

        Control ioctls:
                test VIDIOC_QUERYCTRL/MENU: OK
                test VIDIOC_G/S_CTRL: OK
                test VIDIOC_G/S/TRY_EXT_CTRLS: OK
                test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
                test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
                Standard Controls: 11 Private Controls: 2

        Format ioctls:
                test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
                test VIDIOC_G/S_PARM: OK (Not Supported)
                test VIDIOC_G_FBUF: OK (Not Supported)
                test VIDIOC_G_FMT: OK
                test VIDIOC_TRY_FMT: OK
                test VIDIOC_S_FMT: OK
                test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)

        Codec ioctls:
                test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
                test VIDIOC_G_ENC_INDEX: OK (Not Supported)
                test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

        Buffer ioctls:
                fail: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-buffers.cpp(266): vp->length == 0
                fail: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-buffers.cpp(335): buf.check(Unqueued, i)
                fail: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-buffers.cpp(420): testQueryBuf(node, i, q.g_buffers())
                test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
                fail: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-buffers.cpp(500): q.has_expbuf(node)
                test VIDIOC_EXPBUF: FAIL
Total: 38, Succeeded: 36, Failed: 2, Warnings: 0


Change in v2:
Vcodec Part
1.Remove common and include directory in mtk-vcodec 2.Refine vb2ops_venc_start_streaming and vb2ops_venc_stop_streaming and state machine 3.Remove venc_if_init and venc_if_deinit 4.Refine debug message 5.Refine lab and vpu decription in mediatek-vcodec.txt

VPU Part
1. Modify VPU Kconfig
2. Move encoder header files to other patch sets 3. Remove marcos for extended virtual/iova address 4. Change register and variable names 5. Add a reference counter for VPU watchdog 6. Remove one busy waiting in function vpu_ipi_send 7. Operate VPU clock in VPU driver (not called by encoder drivers) 8. Refine memory mapping, firmware download and extended memory allocation/free functions 9. Release more allocated resources in driver remove function



Andrew-CT Chen (2):
  dt-bindings: Add a binding for Mediatek Video Processor
  [Media] vcodec: mediatek: Add Mediatek V4L2 Video Encoder Driver

Tiffany Lin (6):
  [media] VPU: mediatek: support Mediatek VPU
  arm64: dts: mediatek: Add node for Mediatek Video Processor Unit
  dt-bindings: Add a binding for Mediatek Video Encoder
  [media] vcodec: mediatek: Add Mediatek VP8 Video Encoder Driver
  [media] vcodec: mediatek: Add Mediatek H264 Video Encoder Driver
  arm64: dts: mediatek: Add Video Encoder for MT8173

 .../devicetree/bindings/media/mediatek-vcodec.txt  |   59 +
 .../devicetree/bindings/media/mediatek-vpu.txt     |   31 +
 arch/arm64/boot/dts/mediatek/mt8173.dtsi           |   62 +
 drivers/media/platform/Kconfig                     |   20 +
 drivers/media/platform/Makefile                    |    4 +
 drivers/media/platform/mtk-vcodec/Makefile         |   11 +
 .../media/platform/mtk-vcodec/h264_enc/Makefile    |    6 +
 .../platform/mtk-vcodec/h264_enc/venc_h264_if.c    |  540 ++++++++
 .../platform/mtk-vcodec/h264_enc/venc_h264_if.h    |  165 +++
 .../platform/mtk-vcodec/h264_enc/venc_h264_vpu.c   |  309 +++++
 .../platform/mtk-vcodec/h264_enc/venc_h264_vpu.h   |   30 +
 drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h |  388 ++++++
 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c | 1380 ++++++++++++++++++++
 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h |   46 +
 .../media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c |  476 +++++++
 .../media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c  |  132 ++
 .../media/platform/mtk-vcodec/mtk_vcodec_intr.c    |  102 ++
 .../media/platform/mtk-vcodec/mtk_vcodec_intr.h    |   29 +
 drivers/media/platform/mtk-vcodec/mtk_vcodec_pm.h  |   26 +
 .../media/platform/mtk-vcodec/mtk_vcodec_util.c    |  106 ++
 .../media/platform/mtk-vcodec/mtk_vcodec_util.h    |   85 ++
 drivers/media/platform/mtk-vcodec/venc_drv_base.h  |   62 +
 drivers/media/platform/mtk-vcodec/venc_drv_if.c    |  106 ++
 drivers/media/platform/mtk-vcodec/venc_drv_if.h    |  175 +++
 drivers/media/platform/mtk-vcodec/venc_ipi_msg.h   |  212 +++
 drivers/media/platform/mtk-vcodec/vp8_enc/Makefile |    6 +
 .../platform/mtk-vcodec/vp8_enc/venc_vp8_if.c      |  422 ++++++
 .../platform/mtk-vcodec/vp8_enc/venc_vp8_if.h      |  149 +++
 .../platform/mtk-vcodec/vp8_enc/venc_vp8_vpu.c     |  240 ++++
 .../platform/mtk-vcodec/vp8_enc/venc_vp8_vpu.h     |   28 +
 drivers/media/platform/mtk-vpu/Makefile            |    1 +
 drivers/media/platform/mtk-vpu/mtk_vpu.c           |  994 ++++++++++++++
 drivers/media/platform/mtk-vpu/mtk_vpu.h           |  167 +++
 include/uapi/linux/v4l2-controls.h                 |    4 +
 34 files changed, 6573 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/media/mediatek-vcodec.txt
 create mode 100644 Documentation/devicetree/bindings/media/mediatek-vpu.txt
 create mode 100644 drivers/media/platform/mtk-vcodec/Makefile
 create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/Makefile
 create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.c
 create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.h
 create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.c
 create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.h
 create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
 create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
 create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h
 create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
 create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
 create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.c
 create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.h
 create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_pm.h
 create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c
 create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
 create mode 100644 drivers/media/platform/mtk-vcodec/venc_drv_base.h
 create mode 100644 drivers/media/platform/mtk-vcodec/venc_drv_if.c
 create mode 100644 drivers/media/platform/mtk-vcodec/venc_drv_if.h
 create mode 100644 drivers/media/platform/mtk-vcodec/venc_ipi_msg.h
 create mode 100644 drivers/media/platform/mtk-vcodec/vp8_enc/Makefile
 create mode 100644 drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_if.c
 create mode 100644 drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_if.h
 create mode 100644 drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_vpu.c
 create mode 100644 drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_vpu.h
 create mode 100644 drivers/media/platform/mtk-vpu/Makefile
 create mode 100644 drivers/media/platform/mtk-vpu/mtk_vpu.c
 create mode 100644 drivers/media/platform/mtk-vpu/mtk_vpu.h
 mode change 100644 => 100755 include/uapi/linux/v4l2-controls.h

-- 
1.7.9.5

^ permalink raw reply	[flat|nested] 102+ messages in thread

* [PATCH v4 1/8] dt-bindings: Add a binding for Mediatek Video Processor
@ 2016-02-04 11:34   ` Tiffany Lin
  0 siblings, 0 replies; 102+ messages in thread
From: Tiffany Lin @ 2016-02-04 11:34 UTC (permalink / raw)
  To: Hans Verkuil, daniel.thompson, Rob Herring,
	Mauro Carvalho Chehab, Matthias Brugger, Daniel Kurtz,
	Pawel Osciak
  Cc: Eddie Huang, Yingjoe Chen, devicetree, linux-kernel,
	linux-arm-kernel, linux-media, linux-mediatek, PoChun.Lin,
	Andrew-CT Chen, Tiffany Lin

From: Andrew-CT Chen <andrew-ct.chen@mediatek.com>

Add a DT binding documentation of Video Processor Unit for the
MT8173 SoC from Mediatek.


Signed-off-by: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
---
 .../devicetree/bindings/media/mediatek-vpu.txt     |   31 ++++++++++++++++++++
 1 file changed, 31 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/media/mediatek-vpu.txt

diff --git a/Documentation/devicetree/bindings/media/mediatek-vpu.txt b/Documentation/devicetree/bindings/media/mediatek-vpu.txt
new file mode 100644
index 0000000..2a5bac3
--- /dev/null
+++ b/Documentation/devicetree/bindings/media/mediatek-vpu.txt
@@ -0,0 +1,31 @@
+* Mediatek Video Processor Unit
+
+Video Processor Unit is a HW video controller. It controls HW Codec including
+H.264/VP8/VP9 Decode, H.264/VP8 Encode and Image Processor (scale/rotate/color convert).
+
+Required properties:
+  - compatible: "mediatek,mt8173-vpu"
+  - reg: Must contain an entry for each entry in reg-names.
+  - reg-names: Must include the following entries:
+    "tcm": tcm base
+    "cfg_reg": Main configuration registers base
+  - interrupts: interrupt number to the cpu.
+  - clocks : clock name from clock manager
+  - clock-names: must be main. It is the main clock of VPU
+
+Optional properties:
+  - memory-region: phandle to a node describing memory (see
+    Documentation/devicetree/bindings/reserved-memory/reserved-memory.txt)
+    to be used for VPU extended memory; if not present, VPU may be located
+    anywhere in the memory
+
+Example:
+	vpu: vpu@10020000 {
+		compatible = "mediatek,mt8173-vpu";
+		reg = <0 0x10020000 0 0x30000>,
+		      <0 0x10050000 0 0x100>;
+		reg-names = "tcm", "cfg_reg";
+		interrupts = <GIC_SPI 166 IRQ_TYPE_LEVEL_HIGH>;
+		clocks = <&topckgen TOP_SCP_SEL>;
+		clock-names = "main";
+	};
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 102+ messages in thread

* [PATCH v4 1/8] dt-bindings: Add a binding for Mediatek Video Processor
@ 2016-02-04 11:34   ` Tiffany Lin
  0 siblings, 0 replies; 102+ messages in thread
From: Tiffany Lin @ 2016-02-04 11:34 UTC (permalink / raw)
  To: Hans Verkuil, daniel.thompson-QSEj5FYQhm4dnm+yROfE0A,
	Rob Herring, Mauro Carvalho Chehab, Matthias Brugger,
	Daniel Kurtz, Pawel Osciak
  Cc: Eddie Huang, Yingjoe Chen, devicetree-u79uwXL29TY76Z2rM5mHXA,
	linux-kernel-u79uwXL29TY76Z2rM5mHXA,
	linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
	linux-media-u79uwXL29TY76Z2rM5mHXA,
	linux-mediatek-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
	PoChun.Lin-NuS5LvNUpcJWk0Htik3J/w, Andrew-CT Chen, Tiffany Lin

From: Andrew-CT Chen <andrew-ct.chen-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>

Add a DT binding documentation of Video Processor Unit for the
MT8173 SoC from Mediatek.


Signed-off-by: Andrew-CT Chen <andrew-ct.chen-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
Signed-off-by: Tiffany Lin <tiffany.lin-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
---
 .../devicetree/bindings/media/mediatek-vpu.txt     |   31 ++++++++++++++++++++
 1 file changed, 31 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/media/mediatek-vpu.txt

diff --git a/Documentation/devicetree/bindings/media/mediatek-vpu.txt b/Documentation/devicetree/bindings/media/mediatek-vpu.txt
new file mode 100644
index 0000000..2a5bac3
--- /dev/null
+++ b/Documentation/devicetree/bindings/media/mediatek-vpu.txt
@@ -0,0 +1,31 @@
+* Mediatek Video Processor Unit
+
+Video Processor Unit is a HW video controller. It controls HW Codec including
+H.264/VP8/VP9 Decode, H.264/VP8 Encode and Image Processor (scale/rotate/color convert).
+
+Required properties:
+  - compatible: "mediatek,mt8173-vpu"
+  - reg: Must contain an entry for each entry in reg-names.
+  - reg-names: Must include the following entries:
+    "tcm": tcm base
+    "cfg_reg": Main configuration registers base
+  - interrupts: interrupt number to the cpu.
+  - clocks : clock name from clock manager
+  - clock-names: must be main. It is the main clock of VPU
+
+Optional properties:
+  - memory-region: phandle to a node describing memory (see
+    Documentation/devicetree/bindings/reserved-memory/reserved-memory.txt)
+    to be used for VPU extended memory; if not present, VPU may be located
+    anywhere in the memory
+
+Example:
+	vpu: vpu@10020000 {
+		compatible = "mediatek,mt8173-vpu";
+		reg = <0 0x10020000 0 0x30000>,
+		      <0 0x10050000 0 0x100>;
+		reg-names = "tcm", "cfg_reg";
+		interrupts = <GIC_SPI 166 IRQ_TYPE_LEVEL_HIGH>;
+		clocks = <&topckgen TOP_SCP_SEL>;
+		clock-names = "main";
+	};
-- 
1.7.9.5

--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply related	[flat|nested] 102+ messages in thread

* [PATCH v4 1/8] dt-bindings: Add a binding for Mediatek Video Processor
@ 2016-02-04 11:34   ` Tiffany Lin
  0 siblings, 0 replies; 102+ messages in thread
From: Tiffany Lin @ 2016-02-04 11:34 UTC (permalink / raw)
  To: linux-arm-kernel

From: Andrew-CT Chen <andrew-ct.chen@mediatek.com>

Add a DT binding documentation of Video Processor Unit for the
MT8173 SoC from Mediatek.


Signed-off-by: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
---
 .../devicetree/bindings/media/mediatek-vpu.txt     |   31 ++++++++++++++++++++
 1 file changed, 31 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/media/mediatek-vpu.txt

diff --git a/Documentation/devicetree/bindings/media/mediatek-vpu.txt b/Documentation/devicetree/bindings/media/mediatek-vpu.txt
new file mode 100644
index 0000000..2a5bac3
--- /dev/null
+++ b/Documentation/devicetree/bindings/media/mediatek-vpu.txt
@@ -0,0 +1,31 @@
+* Mediatek Video Processor Unit
+
+Video Processor Unit is a HW video controller. It controls HW Codec including
+H.264/VP8/VP9 Decode, H.264/VP8 Encode and Image Processor (scale/rotate/color convert).
+
+Required properties:
+  - compatible: "mediatek,mt8173-vpu"
+  - reg: Must contain an entry for each entry in reg-names.
+  - reg-names: Must include the following entries:
+    "tcm": tcm base
+    "cfg_reg": Main configuration registers base
+  - interrupts: interrupt number to the cpu.
+  - clocks : clock name from clock manager
+  - clock-names: must be main. It is the main clock of VPU
+
+Optional properties:
+  - memory-region: phandle to a node describing memory (see
+    Documentation/devicetree/bindings/reserved-memory/reserved-memory.txt)
+    to be used for VPU extended memory; if not present, VPU may be located
+    anywhere in the memory
+
+Example:
+	vpu: vpu at 10020000 {
+		compatible = "mediatek,mt8173-vpu";
+		reg = <0 0x10020000 0 0x30000>,
+		      <0 0x10050000 0 0x100>;
+		reg-names = "tcm", "cfg_reg";
+		interrupts = <GIC_SPI 166 IRQ_TYPE_LEVEL_HIGH>;
+		clocks = <&topckgen TOP_SCP_SEL>;
+		clock-names = "main";
+	};
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 102+ messages in thread

* [PATCH v4 2/8] [media] VPU: mediatek: support Mediatek VPU
  2016-02-04 11:34   ` Tiffany Lin
  (?)
@ 2016-02-04 11:34     ` Tiffany Lin
  -1 siblings, 0 replies; 102+ messages in thread
From: Tiffany Lin @ 2016-02-04 11:34 UTC (permalink / raw)
  To: Hans Verkuil, daniel.thompson, Rob Herring,
	Mauro Carvalho Chehab, Matthias Brugger, Daniel Kurtz,
	Pawel Osciak
  Cc: Eddie Huang, Yingjoe Chen, devicetree, linux-kernel,
	linux-arm-kernel, linux-media, linux-mediatek, PoChun.Lin,
	Tiffany Lin, Andrew-CT Chen

The VPU driver for hw video codec embedded in Mediatek's MT8173 SOCs.
It is able to handle video decoding/encoding of in a range of formats.
The driver provides with VPU firmware download, memory management and
the communication interface between CPU and VPU.
For VPU initialization, it will create virtual memory for CPU access and
IOMMU address for vcodec hw device access. When a decode/encode instance
opens a device node, vpu driver will download vpu firmware to the device.
A decode/encode instant will decode/encode a frame using VPU
interface to interrupt vpu to handle decoding/encoding jobs.

Signed-off-by: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
---
 drivers/media/platform/Kconfig           |    9 +
 drivers/media/platform/Makefile          |    2 +
 drivers/media/platform/mtk-vpu/Makefile  |    1 +
 drivers/media/platform/mtk-vpu/mtk_vpu.c |  994 ++++++++++++++++++++++++++++++
 drivers/media/platform/mtk-vpu/mtk_vpu.h |  167 +++++
 5 files changed, 1173 insertions(+)
 create mode 100644 drivers/media/platform/mtk-vpu/Makefile
 create mode 100644 drivers/media/platform/mtk-vpu/mtk_vpu.c
 create mode 100644 drivers/media/platform/mtk-vpu/mtk_vpu.h

diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
index ccbc974..ba812d6 100644
--- a/drivers/media/platform/Kconfig
+++ b/drivers/media/platform/Kconfig
@@ -148,6 +148,15 @@ config VIDEO_CODA
 	   Coda is a range of video codec IPs that supports
 	   H.264, MPEG-4, and other video formats.
 
+config VIDEO_MEDIATEK_VPU
+	tristate "Mediatek Video Processor Unit"
+	depends on VIDEO_DEV && VIDEO_V4L2 && ARCH_MEDIATEK
+	---help---
+	    This driver provides downloading VPU firmware and
+	    communicating with VPU. This driver for hw video
+	    codec embedded in new Mediatek's SOCs. It is able
+	    to handle video decoding/encoding in a range of formats.
+
 config VIDEO_MEM2MEM_DEINTERLACE
 	tristate "Deinterlace support"
 	depends on VIDEO_DEV && VIDEO_V4L2 && DMA_ENGINE
diff --git a/drivers/media/platform/Makefile b/drivers/media/platform/Makefile
index efa0295..e5b19c6 100644
--- a/drivers/media/platform/Makefile
+++ b/drivers/media/platform/Makefile
@@ -55,3 +55,5 @@ obj-$(CONFIG_VIDEO_AM437X_VPFE)		+= am437x/
 obj-$(CONFIG_VIDEO_XILINX)		+= xilinx/
 
 ccflags-y += -I$(srctree)/drivers/media/i2c
+
+obj-$(CONFIG_VIDEO_MEDIATEK_VPU)	+= mtk-vpu/
diff --git a/drivers/media/platform/mtk-vpu/Makefile b/drivers/media/platform/mtk-vpu/Makefile
new file mode 100644
index 0000000..d890a66
--- /dev/null
+++ b/drivers/media/platform/mtk-vpu/Makefile
@@ -0,0 +1 @@
+obj-y += mtk_vpu.o
diff --git a/drivers/media/platform/mtk-vpu/mtk_vpu.c b/drivers/media/platform/mtk-vpu/mtk_vpu.c
new file mode 100644
index 0000000..f54fd89
--- /dev/null
+++ b/drivers/media/platform/mtk-vpu/mtk_vpu.c
@@ -0,0 +1,994 @@
+/*
+* Copyright (c) 2015 MediaTek Inc.
+* Author: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
+*
+* This program is free software; you can redistribute it and/or modify
+* it under the terms of the GNU General Public License version 2 as
+* published by the Free Software Foundation.
+*
+* This program is distributed in the hope that it will be useful,
+* but WITHOUT ANY WARRANTY; without even the implied warranty of
+* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+* GNU General Public License for more details.
+*/
+#include <linux/bootmem.h>
+#include <linux/clk.h>
+#include <linux/debugfs.h>
+#include <linux/firmware.h>
+#include <linux/interrupt.h>
+#include <linux/iommu.h>
+#include <linux/module.h>
+#include <linux/of_address.h>
+#include <linux/of_irq.h>
+#include <linux/of_platform.h>
+#include <linux/of_reserved_mem.h>
+#include <linux/sched.h>
+#include <linux/sizes.h>
+
+#include "mtk_vpu.h"
+
+/**
+ * VPU (video processor unit) is a tiny processor controlling video hardware
+ * related to video codec, scaling and color format converting.
+ * VPU interfaces with other blocks by share memory and interrupt.
+ **/
+
+#define INIT_TIMEOUT_MS		2000U
+#define IPI_TIMEOUT_MS		2000U
+#define VPU_FW_VER_LEN		16
+
+/* maximum program/data TCM (Tightly-Coupled Memory) size */
+#define VPU_PTCM_SIZE		(96 * SZ_1K)
+#define VPU_DTCM_SIZE		(32 * SZ_1K)
+/* the offset to get data tcm address */
+#define VPU_DTCM_OFFSET		0x18000UL
+/* daynamic allocated maximum extended memory size */
+#define VPU_EXT_P_SIZE		SZ_1M
+#define VPU_EXT_D_SIZE		SZ_4M
+/* maximum binary firmware size */
+#define VPU_P_FW_SIZE		(VPU_PTCM_SIZE + VPU_EXT_P_SIZE)
+#define VPU_D_FW_SIZE		(VPU_DTCM_SIZE + VPU_EXT_D_SIZE)
+/* the size of share buffer between Host and  VPU */
+#define SHARE_BUF_SIZE		48
+
+/* binary firmware name */
+#define VPU_P_FW		"vpu_p.bin"
+#define VPU_D_FW		"vpu_d.bin"
+
+#define VPU_RESET		0x0
+#define VPU_TCM_CFG		0x0008
+#define VPU_PMEM_EXT0_ADDR	0x000C
+#define VPU_PMEM_EXT1_ADDR	0x0010
+#define VPU_TO_HOST		0x001C
+#define VPU_DMEM_EXT0_ADDR	0x0014
+#define VPU_DMEM_EXT1_ADDR	0x0018
+#define HOST_TO_VPU		0x0024
+#define VPU_PC_REG		0x0060
+#define VPU_WDT_REG		0x0084
+
+/* vpu inter-processor communication interrupt */
+#define VPU_IPC_INT		BIT(8)
+
+/**
+ * enum vpu_fw_type - VPU firmware type
+ *
+ * @P_FW: program firmware
+ * @D_FW: data firmware
+ *
+ */
+enum vpu_fw_type {
+	P_FW,
+	D_FW,
+};
+
+/**
+ * struct vpu_mem - VPU extended program/data memory information
+ *
+ * @va:		the kernel virtual memory address of VPU extended memory
+ * @pa:		the physical memory address of VPU extended memory
+ *
+ */
+struct vpu_mem {
+	void *va;
+	dma_addr_t pa;
+};
+
+/**
+ * struct vpu_regs - VPU TCM and configuration registers
+ *
+ * @tcm:	the register for VPU Tightly-Coupled Memory
+ * @cfg:	the register for VPU configuration
+ * @irq:	the irq number for VPU interrupt
+ */
+struct vpu_regs {
+	void __iomem *tcm;
+	void __iomem *cfg;
+	int irq;
+};
+
+/**
+ * struct vpu_wdt_handler - VPU watchdog reset handler
+ *
+ * @reset_func:	reset handler
+ * @priv:	private data
+ */
+struct vpu_wdt_handler {
+	void (*reset_func)(void *);
+	void *priv;
+};
+
+/**
+ * struct vpu_wdt - VPU watchdog workqueue
+ *
+ * @handler:	VPU watchdog reset handler
+ * @ws:		workstruct for VPU watchdog
+ * @wq:		workqueue for VPU watchdog
+ */
+struct vpu_wdt {
+	struct vpu_wdt_handler handler[VPU_RST_MAX];
+	struct work_struct ws;
+	struct workqueue_struct *wq;
+};
+
+/**
+ * struct vpu_run - VPU initialization status
+ *
+ * @signaled:		the signal of vpu initialization completed
+ * @fw_ver:		VPU firmware version
+ * @enc_capability:	encoder capability
+ * @wq:			wait queue for VPU initialization status
+ */
+struct vpu_run {
+	u32 signaled;
+	char fw_ver[VPU_FW_VER_LEN];
+	unsigned int	enc_capability;
+	wait_queue_head_t wq;
+};
+
+/**
+ * struct vpu_ipi_desc - VPU IPI descriptor
+ *
+ * @handler:	IPI handler
+ * @name:	the name of IPI handler
+ * @priv:	the private data of IPI handler
+ */
+struct vpu_ipi_desc {
+	ipi_handler_t handler;
+	const char *name;
+	void *priv;
+};
+
+/**
+ * struct share_obj - DTCM (Data Tightly-Coupled Memory) buffer shared with
+ *		      AP and VPU
+ *
+ * @id:		IPI id
+ * @len:	share buffer length
+ * @share_buf:	share buffer data
+ */
+struct share_obj {
+	s32 id;
+	u32 len;
+	unsigned char share_buf[SHARE_BUF_SIZE];
+};
+
+/**
+ * struct mtk_vpu - vpu driver data
+ * @extmem:		VPU extended memory information
+ * @reg:		VPU TCM and configuration registers
+ * @run:		VPU initialization status
+ * @ipi_desc:		VPU IPI descriptor
+ * @recv_buf:		VPU DTCM share buffer for receiving. The
+ *			receive buffer is only accessed in interrupt context.
+ * @send_buf:		VPU DTCM share buffer for sending
+ * @dev:		VPU struct device
+ * @clk:		VPU clock on/off
+ * @enable_4GB:		VPU 4GB mode on/off
+ * @vpu_mutex:		protect mtk_vpu (except recv_buf) and ensure only
+ *			one client to use VPU service at a time. For example,
+ *			suppose a client is using VPU to decode VP8.
+ *			If the other client wants to encode VP8,
+ *			it has to wait until VP8 decode completes.
+ * @wdt_refcnt		WDT reference count to make sure the watchdog can be
+ *			disabled if no other client is using VPU service
+ * @ipi_ack_signaled:	The ACKs for registered IPI function sending
+ *			interrupt to VPU
+ * @ack_wq:		The wait queue for each codec and mdp. When sleeping
+ *			processes wake up, they will check the condition
+ *			"ipi_ack_signaled" to run the corresponding action or
+ *			go back to sleep.
+ *
+ */
+struct mtk_vpu {
+	struct vpu_mem extmem[2];
+	struct vpu_regs reg;
+	struct vpu_run run;
+	struct vpu_wdt wdt;
+	struct vpu_ipi_desc ipi_desc[IPI_MAX];
+	struct share_obj *recv_buf;
+	struct share_obj *send_buf;
+	struct device *dev;
+	struct clk *clk;
+	bool enable_4GB;
+	struct mutex vpu_mutex; /* for protecting vpu data data structure */
+	atomic_t wdt_refcnt;
+	wait_queue_head_t ack_wq;
+	bool ipi_id_ack[IPI_MAX];
+};
+
+static inline void vpu_cfg_writel(struct mtk_vpu *vpu, u32 val, u32 offset)
+{
+	writel(val, vpu->reg.cfg + offset);
+}
+
+static inline u32 vpu_cfg_readl(struct mtk_vpu *vpu, u32 offset)
+{
+	return readl(vpu->reg.cfg + offset);
+}
+
+static inline bool vpu_running(struct mtk_vpu *vpu)
+{
+	return vpu_cfg_readl(vpu, VPU_RESET) & BIT(0);
+}
+
+void vpu_clock_disable(struct mtk_vpu *vpu)
+{
+	/* Disable VPU watchdog */
+	if (atomic_dec_and_test(&vpu->wdt_refcnt))
+		vpu_cfg_writel(vpu,
+			       vpu_cfg_readl(vpu, VPU_WDT_REG) & ~(1L << 31),
+			       VPU_WDT_REG);
+
+	clk_disable(vpu->clk);
+}
+
+int vpu_clock_enable(struct mtk_vpu *vpu)
+{
+	int ret;
+
+	ret = clk_enable(vpu->clk);
+	if (ret)
+		return ret;
+	/* Enable VPU watchdog */
+	if (!atomic_read(&vpu->wdt_refcnt))
+		vpu_cfg_writel(vpu,
+			       vpu_cfg_readl(vpu, VPU_WDT_REG) | (1L << 31),
+			       VPU_WDT_REG);
+
+	atomic_inc(&vpu->wdt_refcnt);
+
+	return ret;
+}
+
+int vpu_ipi_register(struct platform_device *pdev,
+		     enum ipi_id id, ipi_handler_t handler,
+		     const char *name, void *priv)
+{
+	struct mtk_vpu *vpu = platform_get_drvdata(pdev);
+	struct vpu_ipi_desc *ipi_desc;
+
+	if (!vpu) {
+		dev_err(&pdev->dev, "vpu device in not ready\n");
+		return -EPROBE_DEFER;
+	}
+
+	if (id < IPI_MAX && handler) {
+		ipi_desc = vpu->ipi_desc;
+		ipi_desc[id].name = name;
+		ipi_desc[id].handler = handler;
+		ipi_desc[id].priv = priv;
+		return 0;
+	}
+
+	dev_err(&pdev->dev, "register vpu ipi with invalid arguments\n");
+	return -EINVAL;
+}
+
+int vpu_ipi_send(struct platform_device *pdev,
+		 enum ipi_id id, void *buf,
+		 unsigned int len)
+{
+	struct mtk_vpu *vpu = platform_get_drvdata(pdev);
+	struct share_obj *send_obj = vpu->send_buf;
+	unsigned long timeout;
+	int ret = 0;
+
+	if (id <= IPI_VPU_INIT || id >= IPI_MAX ||
+	    len > sizeof(send_obj->share_buf) || !buf) {
+		dev_err(vpu->dev, "failed to send ipi message\n");
+		return -EINVAL;
+	}
+
+	ret = vpu_clock_enable(vpu);
+	if (ret) {
+		dev_err(vpu->dev, "failed to enable vpu clock\n");
+		return ret;
+	}
+	if (!vpu_running(vpu)) {
+		dev_err(vpu->dev, "vpu_ipi_send: VPU is not running\n");
+		ret = -EINVAL;
+		goto clock_disable;
+	}
+
+	mutex_lock(&vpu->vpu_mutex);
+
+	 /* Wait until VPU receives the last command */
+	timeout = jiffies + msecs_to_jiffies(IPI_TIMEOUT_MS);
+	do {
+		if (time_after(jiffies, timeout)) {
+			dev_err(vpu->dev, "vpu_ipi_send: IPI timeout!\n");
+			ret = -EIO;
+			goto mut_unlock;
+		}
+	} while (vpu_cfg_readl(vpu, HOST_TO_VPU));
+
+	memcpy((void *)send_obj->share_buf, buf, len);
+	send_obj->len = len;
+	send_obj->id = id;
+
+	vpu->ipi_id_ack[id] = false;
+	/* send the command to VPU */
+	vpu_cfg_writel(vpu, 0x1, HOST_TO_VPU);
+
+	mutex_unlock(&vpu->vpu_mutex);
+
+	/* wait for VPU's ACK */
+	timeout = msecs_to_jiffies(IPI_TIMEOUT_MS);
+	ret = wait_event_interruptible_timeout(vpu->ack_wq,
+					       vpu->ipi_id_ack[id], timeout);
+	vpu->ipi_id_ack[id] = false;
+	if (ret == 0) {
+		dev_err(vpu->dev, "vpu ipi %d ack time out !", id);
+		ret = -EIO;
+		goto clock_disable;
+	} else if (-ERESTARTSYS == ret) {
+		dev_err(vpu->dev, "vpu ipi %d ack wait interrupted by a signal",
+			id);
+		ret = -ERESTARTSYS;
+		goto clock_disable;
+	}
+	vpu_clock_disable(vpu);
+
+	return 0;
+
+mut_unlock:
+	vpu->ipi_id_ack[id] = false;
+	mutex_unlock(&vpu->vpu_mutex);
+clock_disable:
+	vpu_clock_disable(vpu);
+
+	return ret;
+}
+
+static void vpu_wdt_reset_func(struct work_struct *ws)
+{
+	struct vpu_wdt *wdt = container_of(ws, struct vpu_wdt, ws);
+	struct mtk_vpu *vpu = container_of(wdt, struct mtk_vpu, wdt);
+	struct vpu_wdt_handler *handler = wdt->handler;
+	int index, ret;
+
+	dev_info(vpu->dev, "vpu reset\n");
+	mutex_lock(&vpu->vpu_mutex);
+	ret = vpu_clock_enable(vpu);
+	if (ret) {
+		dev_err(vpu->dev, "[VPU] wdt enables clock failed %d\n", ret);
+		return;
+	}
+	vpu_cfg_writel(vpu, 0x0, VPU_RESET);
+	vpu_clock_disable(vpu);
+	mutex_unlock(&vpu->vpu_mutex);
+
+	for (index = 0; index < VPU_RST_MAX; index++) {
+		if (handler[index].reset_func) {
+			handler[index].reset_func(handler[index].priv);
+			dev_dbg(vpu->dev, "wdt handler func %d\n", index);
+		}
+	}
+}
+
+int vpu_wdt_reg_handler(struct platform_device *pdev,
+			void wdt_reset(void *),
+			void *priv, enum rst_id id)
+{
+	struct mtk_vpu *vpu = platform_get_drvdata(pdev);
+	struct vpu_wdt_handler *handler = vpu->wdt.handler;
+
+	if (!vpu) {
+		dev_err(vpu->dev, "vpu device in not ready\n");
+		return -EPROBE_DEFER;
+	}
+
+	if (id < VPU_RST_MAX && wdt_reset != NULL) {
+		dev_dbg(vpu->dev, "wdt register id %d\n", id);
+		mutex_lock(&vpu->vpu_mutex);
+		handler[id].reset_func = wdt_reset;
+		handler[id].priv = priv;
+		mutex_unlock(&vpu->vpu_mutex);
+		return 0;
+	}
+
+	dev_err(vpu->dev, "register vpu wdt handler failed\n");
+	return -EINVAL;
+}
+
+unsigned int vpu_get_venc_hw_capa(struct platform_device *pdev)
+{
+	struct mtk_vpu *vpu = platform_get_drvdata(pdev);
+
+	return vpu->run.enc_capability;
+}
+
+void *vpu_mapping_dm_addr(struct platform_device *pdev,
+			  u32 dtcm_dmem_addr)
+{
+	struct mtk_vpu *vpu = platform_get_drvdata(pdev);
+
+	if (!dtcm_dmem_addr ||
+	    (dtcm_dmem_addr > (VPU_DTCM_SIZE + VPU_EXT_D_SIZE))) {
+		dev_err(vpu->dev, "invalid virtual data memory address\n");
+		return ERR_PTR(-EINVAL);
+	}
+
+	if (dtcm_dmem_addr < VPU_DTCM_SIZE)
+		return dtcm_dmem_addr + vpu->reg.tcm + VPU_DTCM_OFFSET;
+
+	return vpu->extmem[D_FW].va + (dtcm_dmem_addr - VPU_DTCM_SIZE);
+}
+
+struct platform_device *vpu_get_plat_device(struct platform_device *pdev)
+{
+	struct device *dev = &pdev->dev;
+	struct device_node *vpu_node;
+	struct platform_device *vpu_pdev;
+
+	vpu_node = of_parse_phandle(dev->of_node, "mediatek,vpu", 0);
+	if (!vpu_node) {
+		dev_err(dev, "can't get vpu node\n");
+		return NULL;
+	}
+
+	vpu_pdev = of_find_device_by_node(vpu_node);
+	if (WARN_ON(!vpu_pdev)) {
+		dev_err(dev, "vpu pdev failed\n");
+		of_node_put(vpu_node);
+		return NULL;
+	}
+
+	return vpu_pdev;
+}
+
+/* load vpu program/data memory */
+static int load_requested_vpu(struct mtk_vpu *vpu,
+			      const struct firmware *vpu_fw,
+			      u8 fw_type)
+{
+	size_t tcm_size = fw_type ? VPU_DTCM_SIZE : VPU_PTCM_SIZE;
+	size_t fw_size = fw_type ? VPU_D_FW_SIZE : VPU_P_FW_SIZE;
+	char *fw_name = fw_type ? VPU_D_FW : VPU_P_FW;
+	size_t dl_size = 0;
+	size_t extra_fw_size = 0;
+	void *dest;
+	int ret;
+
+	ret = request_firmware(&vpu_fw, fw_name, vpu->dev);
+	if (ret < 0) {
+		dev_err(vpu->dev, "Failed to load %s, %d\n", fw_name, ret);
+		return ret;
+	}
+	dl_size = vpu_fw->size;
+	if (dl_size > fw_size) {
+		dev_err(vpu->dev, "fw %s size %zu is abnormal\n", fw_name,
+			dl_size);
+		release_firmware(vpu_fw);
+		return  -EFBIG;
+	}
+	dev_dbg(vpu->dev, "Downloaded fw %s size: %zu.\n",
+		fw_name,
+		dl_size);
+	/* reset VPU */
+	vpu_cfg_writel(vpu, 0x0, VPU_RESET);
+
+	/* handle extended firmware size */
+	if (dl_size > tcm_size) {
+		dev_dbg(vpu->dev, "fw size %lx > limited fw size %lx\n",
+			dl_size, tcm_size);
+		extra_fw_size = dl_size - tcm_size;
+		dev_dbg(vpu->dev, "extra_fw_size %lx\n", extra_fw_size);
+		dl_size = tcm_size;
+	}
+	dest = vpu->reg.tcm;
+	if (fw_type == D_FW)
+		dest += VPU_DTCM_OFFSET;
+	memcpy(dest, vpu_fw->data, dl_size);
+	/* download to extended memory if need */
+	if (extra_fw_size > 0) {
+		dest = vpu->extmem[fw_type].va;
+		dev_dbg(vpu->dev, "download extended memory type %x\n",
+			fw_type);
+		memcpy(dest, vpu_fw->data + tcm_size, extra_fw_size);
+	}
+
+	release_firmware(vpu_fw);
+
+	return 0;
+}
+
+int vpu_load_firmware(struct platform_device *pdev)
+{
+	struct mtk_vpu *vpu = platform_get_drvdata(pdev);
+	struct device *dev = &pdev->dev;
+	struct vpu_run *run = &vpu->run;
+	const struct firmware *vpu_fw;
+	int ret;
+
+	if (!pdev) {
+		dev_err(dev, "VPU platform device is invalid\n");
+		return -EINVAL;
+	}
+
+	mutex_lock(&vpu->vpu_mutex);
+
+	ret = vpu_clock_enable(vpu);
+	if (ret) {
+		dev_err(dev, "enable clock failed %d\n", ret);
+		goto OUT_LOAD_FW;
+	}
+
+	if (vpu_running(vpu)) {
+		vpu_clock_disable(vpu);
+		mutex_unlock(&vpu->vpu_mutex);
+		dev_warn(dev, "vpu is running already\n");
+		return 0;
+	}
+
+	run->signaled = false;
+	dev_dbg(vpu->dev, "firmware request\n");
+	/* Downloading program firmware to device*/
+	ret = load_requested_vpu(vpu, vpu_fw, P_FW);
+	if (ret < 0) {
+		dev_err(dev, "Failed to request %s, %d\n", VPU_P_FW, ret);
+		goto OUT_LOAD_FW;
+	}
+
+	/* Downloading data firmware to device */
+	ret = load_requested_vpu(vpu, vpu_fw, D_FW);
+	if (ret < 0) {
+		dev_err(dev, "Failed to request %s, %d\n", VPU_D_FW, ret);
+		goto OUT_LOAD_FW;
+	}
+
+	/* boot up vpu */
+	vpu_cfg_writel(vpu, 0x1, VPU_RESET);
+
+	ret = wait_event_interruptible_timeout(run->wq,
+					       run->signaled,
+					       msecs_to_jiffies(INIT_TIMEOUT_MS)
+					       );
+	if (ret == 0) {
+		ret = -ETIME;
+		dev_err(dev, "wait vpu initialization timout!\n");
+		goto OUT_LOAD_FW;
+	} else if (-ERESTARTSYS == ret) {
+		dev_err(dev, "wait vpu interrupted by a signal!\n");
+		goto OUT_LOAD_FW;
+	}
+
+	ret = 0;
+	dev_info(dev, "vpu is ready. Fw version %s\n", run->fw_ver);
+
+OUT_LOAD_FW:
+	vpu_clock_disable(vpu);
+	mutex_unlock(&vpu->vpu_mutex);
+
+	return ret;
+}
+
+int vpu_compare_version(struct platform_device *pdev,
+			const char *expected_version)
+{
+	struct mtk_vpu *vpu = platform_get_drvdata(pdev);
+	int cur_major, cur_minor, cur_build, cur_rel, cur_ver_num;
+	int major, minor, build, rel, ver_num;
+	char *cur_version = vpu->run.fw_ver;
+
+	cur_ver_num = sscanf(cur_version, "%d.%d.%d-rc%d",
+			     &cur_major, &cur_minor, &cur_build, &cur_rel);
+	if (cur_ver_num < 3)
+		return -1;
+	ver_num = sscanf(expected_version, "%d.%d.%d-rc%d",
+			 &major, &minor, &build, &rel);
+	if (ver_num < 3)
+		return -1;
+
+	if (cur_major < major)
+		return -1;
+	if (cur_major > major)
+		return 1;
+
+	if (cur_minor < minor)
+		return -1;
+	if (cur_minor > minor)
+		return 1;
+
+	if (cur_build < build)
+		return -1;
+	if (cur_build > build)
+		return 1;
+
+	if (cur_ver_num < ver_num)
+		return -1;
+	if (cur_ver_num > ver_num)
+		return 1;
+
+	if (ver_num > 3) {
+		if (cur_rel < rel)
+			return -1;
+		if (cur_rel > rel)
+			return 1;
+	}
+
+	return 0;
+}
+
+static void vpu_init_ipi_handler(void *data, unsigned int len, void *priv)
+{
+	struct mtk_vpu *vpu = (struct mtk_vpu *)priv;
+	struct vpu_run *run = (struct vpu_run *)data;
+
+	vpu->run.signaled = run->signaled;
+	strncpy(vpu->run.fw_ver, run->fw_ver, VPU_FW_VER_LEN);
+	vpu->run.enc_capability = run->enc_capability;
+	wake_up_interruptible(&vpu->run.wq);
+}
+
+#ifdef CONFIG_DEBUG_FS
+static int vpu_debug_open(struct inode *inode, struct file *file)
+{
+	file->private_data = inode->i_private;
+	return 0;
+}
+
+static ssize_t vpu_debug_read(struct file *file, char __user *user_buf,
+			      size_t count, loff_t *ppos)
+{
+	char buf[256];
+	unsigned int len;
+	unsigned int running, pc, vpu_to_host, host_to_vpu, wdt;
+	int ret;
+	struct device *dev = file->private_data;
+	struct mtk_vpu *vpu = dev_get_drvdata(dev);
+
+	ret = vpu_clock_enable(vpu);
+	if (ret) {
+		dev_err(vpu->dev, "[VPU] enable clock failed %d\n", ret);
+		return 0;
+	}
+
+	/* vpu register status */
+	running = vpu_running(vpu);
+	pc = vpu_cfg_readl(vpu, VPU_PC_REG);
+	wdt = vpu_cfg_readl(vpu, VPU_WDT_REG);
+	host_to_vpu = vpu_cfg_readl(vpu, HOST_TO_VPU);
+	vpu_to_host = vpu_cfg_readl(vpu, VPU_TO_HOST);
+	vpu_clock_disable(vpu);
+
+	if (running) {
+		len = sprintf(buf, "VPU is running\n\n"
+		"FW Version: %s\n"
+		"PC: 0x%x\n"
+		"WDT: 0x%x\n"
+		"Host to VPU: 0x%x\n"
+		"VPU to Host: 0x%x\n",
+		vpu->run.fw_ver, pc, wdt,
+		host_to_vpu, vpu_to_host);
+	} else {
+		len = sprintf(buf, "VPU not running\n");
+	}
+
+	return simple_read_from_buffer(user_buf, count, ppos, buf, len);
+}
+
+static const struct file_operations vpu_debug_fops = {
+	.open = vpu_debug_open,
+	.read = vpu_debug_read,
+};
+#endif /* CONFIG_DEBUG_FS */
+
+static void vpu_free_ext_mem(struct mtk_vpu *vpu, u8 fw_type)
+{
+	struct device *dev = vpu->dev;
+	size_t fw_ext_size = fw_type ? VPU_EXT_D_SIZE : VPU_EXT_P_SIZE;
+
+	dma_free_coherent(dev, fw_ext_size, vpu->extmem[fw_type].va,
+			  vpu->extmem[fw_type].pa);
+}
+
+static int vpu_alloc_ext_mem(struct mtk_vpu *vpu, u32 fw_type)
+{
+	struct device *dev = vpu->dev;
+	size_t fw_ext_size = fw_type ? VPU_EXT_D_SIZE : VPU_EXT_P_SIZE;
+	u32 vpu_ext_mem0 = fw_type ? VPU_DMEM_EXT0_ADDR : VPU_PMEM_EXT0_ADDR;
+	u32 vpu_ext_mem1 = fw_type ? VPU_DMEM_EXT1_ADDR : VPU_PMEM_EXT1_ADDR;
+	u32 offset_4gb = vpu->enable_4GB ? 0x40000000 : 0;
+
+	vpu->extmem[fw_type].va = dma_alloc_coherent(dev,
+					       fw_ext_size,
+					       &vpu->extmem[fw_type].pa,
+					       GFP_KERNEL);
+	if (!vpu->extmem[fw_type].va) {
+		dev_err(dev, "Failed to allocate the extended program memory\n");
+		return PTR_ERR(vpu->extmem[fw_type].va);
+	}
+
+	/* Disable extend0. Enable extend1 */
+	vpu_cfg_writel(vpu, 0x1, vpu_ext_mem0);
+	vpu_cfg_writel(vpu, (vpu->extmem[fw_type].pa & 0xFFFFF000) + offset_4gb,
+		       vpu_ext_mem1);
+
+	dev_info(dev, "%s extend memory phy=0x%llx virt=0x%p\n",
+		 fw_type ? "Data" : "Program",
+		 (unsigned long long)vpu->extmem[fw_type].pa,
+		 vpu->extmem[fw_type].va);
+
+	return 0;
+}
+
+static void vpu_ipi_handler(struct mtk_vpu *vpu)
+{
+	struct share_obj *rcv_obj = vpu->recv_buf;
+	struct vpu_ipi_desc *ipi_desc = vpu->ipi_desc;
+
+	if (rcv_obj->id < IPI_MAX && ipi_desc[rcv_obj->id].handler) {
+		ipi_desc[rcv_obj->id].handler(rcv_obj->share_buf,
+					      rcv_obj->len,
+					      ipi_desc[rcv_obj->id].priv);
+		if (rcv_obj->id > IPI_VPU_INIT) {
+			vpu->ipi_id_ack[rcv_obj->id] = true;
+			wake_up_interruptible(&vpu->ack_wq);
+		}
+	} else {
+		dev_err(vpu->dev, "No such ipi id = %d\n", rcv_obj->id);
+	}
+}
+
+static int vpu_ipi_init(struct mtk_vpu *vpu)
+{
+	/* Disable VPU to host interrupt */
+	vpu_cfg_writel(vpu, 0x0, VPU_TO_HOST);
+
+	/* shared buffer initialization */
+	vpu->recv_buf = (struct share_obj *)(vpu->reg.tcm + VPU_DTCM_OFFSET);
+	vpu->send_buf = vpu->recv_buf + 1;
+	memset(vpu->recv_buf, 0, sizeof(struct share_obj));
+	memset(vpu->send_buf, 0, sizeof(struct share_obj));
+	mutex_init(&vpu->vpu_mutex);
+
+	return 0;
+}
+
+static irqreturn_t vpu_irq_handler(int irq, void *priv)
+{
+	struct mtk_vpu *vpu = priv;
+	u32 vpu_to_host;
+	int ret;
+
+	/*
+	 * Clock should have been enabled already.
+	 * Enable again in case vpu_ipi_send times out
+	 * and has disabled the clock.
+	 */
+	ret = clk_enable(vpu->clk);
+	if (ret) {
+		dev_err(vpu->dev, "[VPU] enable clock failed %d\n", ret);
+		return IRQ_NONE;
+	}
+	vpu_to_host = vpu_cfg_readl(vpu, VPU_TO_HOST);
+	if (vpu_to_host & VPU_IPC_INT) {
+		vpu_ipi_handler(vpu);
+	} else {
+		dev_err(vpu->dev, "vpu watchdog timeout! 0x%x", vpu_to_host);
+		if (vpu->wdt.wq)
+			queue_work(vpu->wdt.wq, &vpu->wdt.ws);
+	}
+
+	/* VPU won't send another interrupt until we set VPU_TO_HOST to 0. */
+	vpu_cfg_writel(vpu, 0x0, VPU_TO_HOST);
+	clk_disable(vpu->clk);
+
+	return IRQ_HANDLED;
+}
+
+#ifdef CONFIG_DEBUG_FS
+static struct dentry *vpu_debugfs;
+#endif
+static int mtk_vpu_probe(struct platform_device *pdev)
+{
+	struct mtk_vpu *vpu;
+	struct device *dev;
+	struct resource *res;
+	int ret = 0;
+
+	dev_dbg(&pdev->dev, "initialization\n");
+
+	dev = &pdev->dev;
+	vpu = devm_kzalloc(dev, sizeof(*vpu), GFP_KERNEL);
+	if (!vpu)
+		return -ENOMEM;
+
+	vpu->dev = &pdev->dev;
+	res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "tcm");
+	vpu->reg.tcm = devm_ioremap_resource(dev, res);
+	if (IS_ERR(vpu->reg.tcm)) {
+		dev_err(dev, "devm_ioremap_resource vpu tcm failed.\n");
+		return PTR_ERR(vpu->reg.tcm);
+	}
+
+	res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "cfg_reg");
+	vpu->reg.cfg = devm_ioremap_resource(dev, res);
+	if (IS_ERR(vpu->reg.cfg)) {
+		dev_err(dev, "devm_ioremap_resource vpu cfg failed.\n");
+		return PTR_ERR(vpu->reg.cfg);
+	}
+
+	/* Get VPU clock */
+	vpu->clk = devm_clk_get(dev, "main");
+	if (!vpu->clk) {
+		dev_err(dev, "get vpu clock failed\n");
+		return -EINVAL;
+	}
+
+	platform_set_drvdata(pdev, vpu);
+
+	ret = clk_prepare(vpu->clk);
+	if (ret) {
+		dev_err(dev, "prepare vpu clock failed\n");
+		return ret;
+	}
+
+	/* VPU watchdog */
+	vpu->wdt.wq = create_singlethread_workqueue("vpu_wdt");
+	if (!vpu->wdt.wq) {
+		dev_err(dev, "initialize wdt workqueue failed\n");
+		return -ENOMEM;
+	}
+	INIT_WORK(&vpu->wdt.ws, vpu_wdt_reset_func);
+
+	ret = vpu_clock_enable(vpu);
+	if (ret) {
+		dev_err(dev, "enable vpu clock failed\n");
+		goto workqueue_destroy;
+	}
+
+	dev_dbg(dev, "vpu ipi init\n");
+	ret = vpu_ipi_init(vpu);
+	if (ret) {
+		dev_err(dev, "Failed to init ipi\n");
+		goto disable_vpu_clk;
+	}
+
+	/* register vpu initialization IPI */
+	ret = vpu_ipi_register(pdev, IPI_VPU_INIT, vpu_init_ipi_handler,
+			       "vpu_init", vpu);
+	if (ret) {
+		dev_err(dev, "Failed to register IPI_VPU_INIT\n");
+		goto vpu_mutex_destroy;
+	}
+
+#ifdef CONFIG_DEBUG_FS
+	vpu_debugfs = debugfs_create_file("mtk_vpu", S_IRUGO, NULL, (void *)dev,
+					  &vpu_debug_fops);
+	if (!vpu_debugfs) {
+		ret = -ENOMEM;
+		goto cleanup_ipi;
+	}
+#endif
+
+	/* Set PTCM to 96K and DTCM to 32K */
+	vpu_cfg_writel(vpu, 0x2, VPU_TCM_CFG);
+
+	vpu->enable_4GB = !!(max_pfn > (0xffffffffUL >> PAGE_SHIFT));
+	dev_dbg(dev, "4GB mode %u\n", vpu->enable_4GB);
+
+	if (vpu->enable_4GB) {
+		ret = of_reserved_mem_device_init(dev);
+		if (ret)
+			dev_info(dev, "init reserved memory failed\n");
+			/* continue to use dynamic allocation if failed */
+	}
+
+	ret = vpu_alloc_ext_mem(vpu, D_FW);
+	if (ret) {
+		dev_err(dev, "Allocate DM failed\n");
+		goto remove_debugfs;
+	}
+
+	ret = vpu_alloc_ext_mem(vpu, P_FW);
+	if (ret) {
+		dev_err(dev, "Allocate PM failed\n");
+		goto free_d_mem;
+	}
+
+	init_waitqueue_head(&vpu->run.wq);
+	init_waitqueue_head(&vpu->ack_wq);
+
+	res = platform_get_resource(pdev, IORESOURCE_IRQ, 0);
+	if (!res) {
+		dev_err(dev, "get IRQ resource failed.\n");
+		ret = -ENXIO;
+		goto free_p_mem;
+	}
+	vpu->reg.irq = platform_get_irq(pdev, 0);
+	ret = devm_request_irq(dev, vpu->reg.irq, vpu_irq_handler, 0,
+			       pdev->name, vpu);
+	if (ret) {
+		dev_err(dev, "failed to request irq\n");
+		goto free_p_mem;
+	}
+
+	vpu_clock_disable(vpu);
+	dev_dbg(dev, "initialization completed\n");
+
+	return 0;
+
+free_p_mem:
+	vpu_free_ext_mem(vpu, P_FW);
+free_d_mem:
+	vpu_free_ext_mem(vpu, D_FW);
+remove_debugfs:
+	of_reserved_mem_device_release(dev);
+#ifdef CONFIG_DEBUG_FS
+	debugfs_remove(vpu_debugfs);
+cleanup_ipi:
+#endif
+	memset(vpu->ipi_desc, 0, sizeof(struct vpu_ipi_desc) * IPI_MAX);
+vpu_mutex_destroy:
+	mutex_destroy(&vpu->vpu_mutex);
+disable_vpu_clk:
+	vpu_clock_disable(vpu);
+workqueue_destroy:
+	destroy_workqueue(vpu->wdt.wq);
+
+	return ret;
+}
+
+static const struct of_device_id mtk_vpu_match[] = {
+	{
+		.compatible = "mediatek,mt8173-vpu",
+	},
+	{},
+};
+MODULE_DEVICE_TABLE(of, mtk_vpu_match);
+
+static int mtk_vpu_remove(struct platform_device *pdev)
+{
+	struct mtk_vpu *vpu = platform_get_drvdata(pdev);
+
+#ifdef CONFIG_DEBUG_FS
+	debugfs_remove(vpu_debugfs);
+#endif
+	if (vpu->wdt.wq) {
+		flush_workqueue(vpu->wdt.wq);
+		destroy_workqueue(vpu->wdt.wq);
+	}
+	vpu_free_ext_mem(vpu, P_FW);
+	vpu_free_ext_mem(vpu, D_FW);
+	mutex_destroy(&vpu->vpu_mutex);
+	clk_unprepare(vpu->clk);
+
+	return 0;
+}
+
+static struct platform_driver mtk_vpu_driver = {
+	.probe	= mtk_vpu_probe,
+	.remove	= mtk_vpu_remove,
+	.driver	= {
+		.name	= "mtk_vpu",
+		.owner	= THIS_MODULE,
+		.of_match_table = mtk_vpu_match,
+	},
+};
+
+module_platform_driver(mtk_vpu_driver);
+
+MODULE_LICENSE("GPL v2");
+MODULE_DESCRIPTION("Mediatek Video Prosessor Unit driver");
diff --git a/drivers/media/platform/mtk-vpu/mtk_vpu.h b/drivers/media/platform/mtk-vpu/mtk_vpu.h
new file mode 100644
index 0000000..d9c3cde
--- /dev/null
+++ b/drivers/media/platform/mtk-vpu/mtk_vpu.h
@@ -0,0 +1,167 @@
+/*
+* Copyright (c) 2015 MediaTek Inc.
+* Author: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
+*
+* This program is free software; you can redistribute it and/or modify
+* it under the terms of the GNU General Public License version 2 as
+* published by the Free Software Foundation.
+*
+* This program is distributed in the hope that it will be useful,
+* but WITHOUT ANY WARRANTY; without even the implied warranty of
+* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+* GNU General Public License for more details.
+*/
+
+#ifndef _MTK_VPU_H
+#define _MTK_VPU_H
+
+#include <linux/platform_device.h>
+
+/**
+ * VPU (video processor unit) is a tiny processor controlling video hardware
+ * related to video codec, scaling and color format converting.
+ * VPU interfaces with other blocks by share memory and interrupt.
+ **/
+
+typedef void (*ipi_handler_t) (void *data,
+			       unsigned int len,
+			       void *priv);
+
+/**
+ * enum ipi_id - the id of inter-processor interrupt
+ *
+ * @IPI_VPU_INIT:	 The interrupt from vpu is to notfiy kernel
+			 VPU initialization completed.
+			 IPI_VPU_INIT is sent from VPU when firmware is
+			 loaded. AP doesn't need to send IPI_VPU_INIT
+			 command to VPU.
+			 For other IPI below, AP should send the request
+			 to VPU to trigger the interrupt.
+ * @IPI_VENC_H264:	 The interrupt from vpu is to notify kernel to
+			 handle H264 video encoder job, and vice versa.
+ * @IPI_VENC_VP8:	 The interrupt fro vpu is to notify kernel to
+			 handle VP8 video encoder job,, and vice versa.
+ * @IPI_MAX:		 The maximum IPI number
+ */
+
+enum ipi_id {
+	IPI_VPU_INIT = 0,
+	IPI_VENC_H264,
+	IPI_VENC_VP8,
+	IPI_MAX,
+};
+
+/**
+ * enum rst_id - reset id to register reset function for VPU watchdog timeout
+ *
+ * @VPU_RST_DEC: decoder reset id
+ * @VPU_RST_ENC: encoder reset id
+ * @VPU_RST_MDP: MDP (Media Data Path) reset id
+ * @VPU_RST_MAX: maximum reset id
+ */
+enum rst_id {
+	VPU_RST_ENC,
+	VPU_RST_DEC,
+	VPU_RST_MDP,
+	VPU_RST_MAX,
+};
+
+/**
+ * vpu_ipi_register - register an ipi function
+ *
+ * @pdev:	VPU platform device
+ * @id:		IPI ID
+ * @handler:	IPI handler
+ * @name:	IPI name
+ * @priv:	private data for IPI handler
+ *
+ * Register an ipi function to receive ipi interrupt from VPU.
+ *
+ * Return: Return 0 if ipi registers successfully, otherwise it is failed.
+ */
+int vpu_ipi_register(struct platform_device *pdev, enum ipi_id id,
+		     ipi_handler_t handler, const char *name, void *priv);
+
+/**
+ * vpu_ipi_send - send data from AP to vpu.
+ *
+ * @pdev:	VPU platform device
+ * @id:		IPI ID
+ * @buf:	the data buffer
+ * @len:	the data buffer length
+ *
+ * This function is thread-safe. When this function returns,
+ * VPU has received the data and starts the processing.
+ * When the processing completes, IPI handler registered
+ * by vpu_ipi_register will be called in interrupt context.
+ *
+ * Return: Return 0 if sending data successfully, otherwise it is failed.
+ **/
+int vpu_ipi_send(struct platform_device *pdev,
+		 enum ipi_id id, void *buf,
+		 unsigned int len);
+
+/**
+ * vpu_get_plat_device - get VPU's platform device
+ *
+ * @pdev:	the platform device of the module requesting VPU platform
+ *		device for using VPU API.
+ *
+ * Return: Return NULL if it is failed.
+ * otherwise it is VPU's platform device
+ **/
+struct platform_device *vpu_get_plat_device(struct platform_device *pdev);
+
+/**
+ * vpu_wdt_reg_handler - register a VPU watchdog handler
+ *
+ * @pdev:               VPU platform device
+ * @vpu_wdt_reset_func:	the callback reset function
+ * @private_data:       the private data for reset function
+ * @rst_id:		reset id
+ *
+ * Register a handler performing own tasks when vpu reset by watchdog
+ *
+ * Return: Return 0 if the handler is added successfully,
+ * otherwise it is failed.
+ *
+ **/
+int vpu_wdt_reg_handler(struct platform_device *pdev,
+			void vpu_wdt_reset_func(void *),
+			void *priv, enum rst_id id);
+
+/**
+ * vpu_get_venc_hw_capa - get video encoder hardware capability
+ *
+ * @pdev:	VPU platform device
+ *
+ * Return: video encoder hardware capability
+ **/
+unsigned int vpu_get_venc_hw_capa(struct platform_device *pdev);
+
+/**
+ * vpu_load_firmware - download VPU firmware and boot it
+ *
+ * @pdev:	VPU platform device
+ *
+ * Return: Return 0 if downloading firmware successfully,
+ * otherwise it is failed
+ **/
+int vpu_load_firmware(struct platform_device *pdev);
+
+/**
+ * vpu_mapping_dm_addr - Mapping DTCM/DMEM to kernel virtual address
+ *
+ * @pdev:	VPU platform device
+ * @dmem_addr:	VPU's data memory address
+ *
+ * Mapping the VPU's DTCM (Data Tightly-Coupled Memory) /
+ * DMEM (Data Extended Memory) memory address to
+ * kernel virtual address.
+ *
+ * Return: Return ERR_PTR(-EINVAL) if mapping failed,
+ * otherwise the mapped kernel virtual address
+ **/
+void *vpu_mapping_dm_addr(struct platform_device *pdev,
+			  u32 dtcm_dmem_addr);
+#endif /* _MTK_VPU_H */
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 102+ messages in thread

* [PATCH v4 2/8] [media] VPU: mediatek: support Mediatek VPU
@ 2016-02-04 11:34     ` Tiffany Lin
  0 siblings, 0 replies; 102+ messages in thread
From: Tiffany Lin @ 2016-02-04 11:34 UTC (permalink / raw)
  To: Hans Verkuil, daniel.thompson, Rob Herring,
	Mauro Carvalho Chehab, Matthias Brugger, Daniel Kurtz,
	Pawel Osciak
  Cc: devicetree, Tiffany Lin, Andrew-CT Chen, linux-kernel,
	linux-mediatek, PoChun.Lin, Yingjoe Chen, Eddie Huang,
	linux-arm-kernel, linux-media

The VPU driver for hw video codec embedded in Mediatek's MT8173 SOCs.
It is able to handle video decoding/encoding of in a range of formats.
The driver provides with VPU firmware download, memory management and
the communication interface between CPU and VPU.
For VPU initialization, it will create virtual memory for CPU access and
IOMMU address for vcodec hw device access. When a decode/encode instance
opens a device node, vpu driver will download vpu firmware to the device.
A decode/encode instant will decode/encode a frame using VPU
interface to interrupt vpu to handle decoding/encoding jobs.

Signed-off-by: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
---
 drivers/media/platform/Kconfig           |    9 +
 drivers/media/platform/Makefile          |    2 +
 drivers/media/platform/mtk-vpu/Makefile  |    1 +
 drivers/media/platform/mtk-vpu/mtk_vpu.c |  994 ++++++++++++++++++++++++++++++
 drivers/media/platform/mtk-vpu/mtk_vpu.h |  167 +++++
 5 files changed, 1173 insertions(+)
 create mode 100644 drivers/media/platform/mtk-vpu/Makefile
 create mode 100644 drivers/media/platform/mtk-vpu/mtk_vpu.c
 create mode 100644 drivers/media/platform/mtk-vpu/mtk_vpu.h

diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
index ccbc974..ba812d6 100644
--- a/drivers/media/platform/Kconfig
+++ b/drivers/media/platform/Kconfig
@@ -148,6 +148,15 @@ config VIDEO_CODA
 	   Coda is a range of video codec IPs that supports
 	   H.264, MPEG-4, and other video formats.
 
+config VIDEO_MEDIATEK_VPU
+	tristate "Mediatek Video Processor Unit"
+	depends on VIDEO_DEV && VIDEO_V4L2 && ARCH_MEDIATEK
+	---help---
+	    This driver provides downloading VPU firmware and
+	    communicating with VPU. This driver for hw video
+	    codec embedded in new Mediatek's SOCs. It is able
+	    to handle video decoding/encoding in a range of formats.
+
 config VIDEO_MEM2MEM_DEINTERLACE
 	tristate "Deinterlace support"
 	depends on VIDEO_DEV && VIDEO_V4L2 && DMA_ENGINE
diff --git a/drivers/media/platform/Makefile b/drivers/media/platform/Makefile
index efa0295..e5b19c6 100644
--- a/drivers/media/platform/Makefile
+++ b/drivers/media/platform/Makefile
@@ -55,3 +55,5 @@ obj-$(CONFIG_VIDEO_AM437X_VPFE)		+= am437x/
 obj-$(CONFIG_VIDEO_XILINX)		+= xilinx/
 
 ccflags-y += -I$(srctree)/drivers/media/i2c
+
+obj-$(CONFIG_VIDEO_MEDIATEK_VPU)	+= mtk-vpu/
diff --git a/drivers/media/platform/mtk-vpu/Makefile b/drivers/media/platform/mtk-vpu/Makefile
new file mode 100644
index 0000000..d890a66
--- /dev/null
+++ b/drivers/media/platform/mtk-vpu/Makefile
@@ -0,0 +1 @@
+obj-y += mtk_vpu.o
diff --git a/drivers/media/platform/mtk-vpu/mtk_vpu.c b/drivers/media/platform/mtk-vpu/mtk_vpu.c
new file mode 100644
index 0000000..f54fd89
--- /dev/null
+++ b/drivers/media/platform/mtk-vpu/mtk_vpu.c
@@ -0,0 +1,994 @@
+/*
+* Copyright (c) 2015 MediaTek Inc.
+* Author: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
+*
+* This program is free software; you can redistribute it and/or modify
+* it under the terms of the GNU General Public License version 2 as
+* published by the Free Software Foundation.
+*
+* This program is distributed in the hope that it will be useful,
+* but WITHOUT ANY WARRANTY; without even the implied warranty of
+* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+* GNU General Public License for more details.
+*/
+#include <linux/bootmem.h>
+#include <linux/clk.h>
+#include <linux/debugfs.h>
+#include <linux/firmware.h>
+#include <linux/interrupt.h>
+#include <linux/iommu.h>
+#include <linux/module.h>
+#include <linux/of_address.h>
+#include <linux/of_irq.h>
+#include <linux/of_platform.h>
+#include <linux/of_reserved_mem.h>
+#include <linux/sched.h>
+#include <linux/sizes.h>
+
+#include "mtk_vpu.h"
+
+/**
+ * VPU (video processor unit) is a tiny processor controlling video hardware
+ * related to video codec, scaling and color format converting.
+ * VPU interfaces with other blocks by share memory and interrupt.
+ **/
+
+#define INIT_TIMEOUT_MS		2000U
+#define IPI_TIMEOUT_MS		2000U
+#define VPU_FW_VER_LEN		16
+
+/* maximum program/data TCM (Tightly-Coupled Memory) size */
+#define VPU_PTCM_SIZE		(96 * SZ_1K)
+#define VPU_DTCM_SIZE		(32 * SZ_1K)
+/* the offset to get data tcm address */
+#define VPU_DTCM_OFFSET		0x18000UL
+/* daynamic allocated maximum extended memory size */
+#define VPU_EXT_P_SIZE		SZ_1M
+#define VPU_EXT_D_SIZE		SZ_4M
+/* maximum binary firmware size */
+#define VPU_P_FW_SIZE		(VPU_PTCM_SIZE + VPU_EXT_P_SIZE)
+#define VPU_D_FW_SIZE		(VPU_DTCM_SIZE + VPU_EXT_D_SIZE)
+/* the size of share buffer between Host and  VPU */
+#define SHARE_BUF_SIZE		48
+
+/* binary firmware name */
+#define VPU_P_FW		"vpu_p.bin"
+#define VPU_D_FW		"vpu_d.bin"
+
+#define VPU_RESET		0x0
+#define VPU_TCM_CFG		0x0008
+#define VPU_PMEM_EXT0_ADDR	0x000C
+#define VPU_PMEM_EXT1_ADDR	0x0010
+#define VPU_TO_HOST		0x001C
+#define VPU_DMEM_EXT0_ADDR	0x0014
+#define VPU_DMEM_EXT1_ADDR	0x0018
+#define HOST_TO_VPU		0x0024
+#define VPU_PC_REG		0x0060
+#define VPU_WDT_REG		0x0084
+
+/* vpu inter-processor communication interrupt */
+#define VPU_IPC_INT		BIT(8)
+
+/**
+ * enum vpu_fw_type - VPU firmware type
+ *
+ * @P_FW: program firmware
+ * @D_FW: data firmware
+ *
+ */
+enum vpu_fw_type {
+	P_FW,
+	D_FW,
+};
+
+/**
+ * struct vpu_mem - VPU extended program/data memory information
+ *
+ * @va:		the kernel virtual memory address of VPU extended memory
+ * @pa:		the physical memory address of VPU extended memory
+ *
+ */
+struct vpu_mem {
+	void *va;
+	dma_addr_t pa;
+};
+
+/**
+ * struct vpu_regs - VPU TCM and configuration registers
+ *
+ * @tcm:	the register for VPU Tightly-Coupled Memory
+ * @cfg:	the register for VPU configuration
+ * @irq:	the irq number for VPU interrupt
+ */
+struct vpu_regs {
+	void __iomem *tcm;
+	void __iomem *cfg;
+	int irq;
+};
+
+/**
+ * struct vpu_wdt_handler - VPU watchdog reset handler
+ *
+ * @reset_func:	reset handler
+ * @priv:	private data
+ */
+struct vpu_wdt_handler {
+	void (*reset_func)(void *);
+	void *priv;
+};
+
+/**
+ * struct vpu_wdt - VPU watchdog workqueue
+ *
+ * @handler:	VPU watchdog reset handler
+ * @ws:		workstruct for VPU watchdog
+ * @wq:		workqueue for VPU watchdog
+ */
+struct vpu_wdt {
+	struct vpu_wdt_handler handler[VPU_RST_MAX];
+	struct work_struct ws;
+	struct workqueue_struct *wq;
+};
+
+/**
+ * struct vpu_run - VPU initialization status
+ *
+ * @signaled:		the signal of vpu initialization completed
+ * @fw_ver:		VPU firmware version
+ * @enc_capability:	encoder capability
+ * @wq:			wait queue for VPU initialization status
+ */
+struct vpu_run {
+	u32 signaled;
+	char fw_ver[VPU_FW_VER_LEN];
+	unsigned int	enc_capability;
+	wait_queue_head_t wq;
+};
+
+/**
+ * struct vpu_ipi_desc - VPU IPI descriptor
+ *
+ * @handler:	IPI handler
+ * @name:	the name of IPI handler
+ * @priv:	the private data of IPI handler
+ */
+struct vpu_ipi_desc {
+	ipi_handler_t handler;
+	const char *name;
+	void *priv;
+};
+
+/**
+ * struct share_obj - DTCM (Data Tightly-Coupled Memory) buffer shared with
+ *		      AP and VPU
+ *
+ * @id:		IPI id
+ * @len:	share buffer length
+ * @share_buf:	share buffer data
+ */
+struct share_obj {
+	s32 id;
+	u32 len;
+	unsigned char share_buf[SHARE_BUF_SIZE];
+};
+
+/**
+ * struct mtk_vpu - vpu driver data
+ * @extmem:		VPU extended memory information
+ * @reg:		VPU TCM and configuration registers
+ * @run:		VPU initialization status
+ * @ipi_desc:		VPU IPI descriptor
+ * @recv_buf:		VPU DTCM share buffer for receiving. The
+ *			receive buffer is only accessed in interrupt context.
+ * @send_buf:		VPU DTCM share buffer for sending
+ * @dev:		VPU struct device
+ * @clk:		VPU clock on/off
+ * @enable_4GB:		VPU 4GB mode on/off
+ * @vpu_mutex:		protect mtk_vpu (except recv_buf) and ensure only
+ *			one client to use VPU service at a time. For example,
+ *			suppose a client is using VPU to decode VP8.
+ *			If the other client wants to encode VP8,
+ *			it has to wait until VP8 decode completes.
+ * @wdt_refcnt		WDT reference count to make sure the watchdog can be
+ *			disabled if no other client is using VPU service
+ * @ipi_ack_signaled:	The ACKs for registered IPI function sending
+ *			interrupt to VPU
+ * @ack_wq:		The wait queue for each codec and mdp. When sleeping
+ *			processes wake up, they will check the condition
+ *			"ipi_ack_signaled" to run the corresponding action or
+ *			go back to sleep.
+ *
+ */
+struct mtk_vpu {
+	struct vpu_mem extmem[2];
+	struct vpu_regs reg;
+	struct vpu_run run;
+	struct vpu_wdt wdt;
+	struct vpu_ipi_desc ipi_desc[IPI_MAX];
+	struct share_obj *recv_buf;
+	struct share_obj *send_buf;
+	struct device *dev;
+	struct clk *clk;
+	bool enable_4GB;
+	struct mutex vpu_mutex; /* for protecting vpu data data structure */
+	atomic_t wdt_refcnt;
+	wait_queue_head_t ack_wq;
+	bool ipi_id_ack[IPI_MAX];
+};
+
+static inline void vpu_cfg_writel(struct mtk_vpu *vpu, u32 val, u32 offset)
+{
+	writel(val, vpu->reg.cfg + offset);
+}
+
+static inline u32 vpu_cfg_readl(struct mtk_vpu *vpu, u32 offset)
+{
+	return readl(vpu->reg.cfg + offset);
+}
+
+static inline bool vpu_running(struct mtk_vpu *vpu)
+{
+	return vpu_cfg_readl(vpu, VPU_RESET) & BIT(0);
+}
+
+void vpu_clock_disable(struct mtk_vpu *vpu)
+{
+	/* Disable VPU watchdog */
+	if (atomic_dec_and_test(&vpu->wdt_refcnt))
+		vpu_cfg_writel(vpu,
+			       vpu_cfg_readl(vpu, VPU_WDT_REG) & ~(1L << 31),
+			       VPU_WDT_REG);
+
+	clk_disable(vpu->clk);
+}
+
+int vpu_clock_enable(struct mtk_vpu *vpu)
+{
+	int ret;
+
+	ret = clk_enable(vpu->clk);
+	if (ret)
+		return ret;
+	/* Enable VPU watchdog */
+	if (!atomic_read(&vpu->wdt_refcnt))
+		vpu_cfg_writel(vpu,
+			       vpu_cfg_readl(vpu, VPU_WDT_REG) | (1L << 31),
+			       VPU_WDT_REG);
+
+	atomic_inc(&vpu->wdt_refcnt);
+
+	return ret;
+}
+
+int vpu_ipi_register(struct platform_device *pdev,
+		     enum ipi_id id, ipi_handler_t handler,
+		     const char *name, void *priv)
+{
+	struct mtk_vpu *vpu = platform_get_drvdata(pdev);
+	struct vpu_ipi_desc *ipi_desc;
+
+	if (!vpu) {
+		dev_err(&pdev->dev, "vpu device in not ready\n");
+		return -EPROBE_DEFER;
+	}
+
+	if (id < IPI_MAX && handler) {
+		ipi_desc = vpu->ipi_desc;
+		ipi_desc[id].name = name;
+		ipi_desc[id].handler = handler;
+		ipi_desc[id].priv = priv;
+		return 0;
+	}
+
+	dev_err(&pdev->dev, "register vpu ipi with invalid arguments\n");
+	return -EINVAL;
+}
+
+int vpu_ipi_send(struct platform_device *pdev,
+		 enum ipi_id id, void *buf,
+		 unsigned int len)
+{
+	struct mtk_vpu *vpu = platform_get_drvdata(pdev);
+	struct share_obj *send_obj = vpu->send_buf;
+	unsigned long timeout;
+	int ret = 0;
+
+	if (id <= IPI_VPU_INIT || id >= IPI_MAX ||
+	    len > sizeof(send_obj->share_buf) || !buf) {
+		dev_err(vpu->dev, "failed to send ipi message\n");
+		return -EINVAL;
+	}
+
+	ret = vpu_clock_enable(vpu);
+	if (ret) {
+		dev_err(vpu->dev, "failed to enable vpu clock\n");
+		return ret;
+	}
+	if (!vpu_running(vpu)) {
+		dev_err(vpu->dev, "vpu_ipi_send: VPU is not running\n");
+		ret = -EINVAL;
+		goto clock_disable;
+	}
+
+	mutex_lock(&vpu->vpu_mutex);
+
+	 /* Wait until VPU receives the last command */
+	timeout = jiffies + msecs_to_jiffies(IPI_TIMEOUT_MS);
+	do {
+		if (time_after(jiffies, timeout)) {
+			dev_err(vpu->dev, "vpu_ipi_send: IPI timeout!\n");
+			ret = -EIO;
+			goto mut_unlock;
+		}
+	} while (vpu_cfg_readl(vpu, HOST_TO_VPU));
+
+	memcpy((void *)send_obj->share_buf, buf, len);
+	send_obj->len = len;
+	send_obj->id = id;
+
+	vpu->ipi_id_ack[id] = false;
+	/* send the command to VPU */
+	vpu_cfg_writel(vpu, 0x1, HOST_TO_VPU);
+
+	mutex_unlock(&vpu->vpu_mutex);
+
+	/* wait for VPU's ACK */
+	timeout = msecs_to_jiffies(IPI_TIMEOUT_MS);
+	ret = wait_event_interruptible_timeout(vpu->ack_wq,
+					       vpu->ipi_id_ack[id], timeout);
+	vpu->ipi_id_ack[id] = false;
+	if (ret == 0) {
+		dev_err(vpu->dev, "vpu ipi %d ack time out !", id);
+		ret = -EIO;
+		goto clock_disable;
+	} else if (-ERESTARTSYS == ret) {
+		dev_err(vpu->dev, "vpu ipi %d ack wait interrupted by a signal",
+			id);
+		ret = -ERESTARTSYS;
+		goto clock_disable;
+	}
+	vpu_clock_disable(vpu);
+
+	return 0;
+
+mut_unlock:
+	vpu->ipi_id_ack[id] = false;
+	mutex_unlock(&vpu->vpu_mutex);
+clock_disable:
+	vpu_clock_disable(vpu);
+
+	return ret;
+}
+
+static void vpu_wdt_reset_func(struct work_struct *ws)
+{
+	struct vpu_wdt *wdt = container_of(ws, struct vpu_wdt, ws);
+	struct mtk_vpu *vpu = container_of(wdt, struct mtk_vpu, wdt);
+	struct vpu_wdt_handler *handler = wdt->handler;
+	int index, ret;
+
+	dev_info(vpu->dev, "vpu reset\n");
+	mutex_lock(&vpu->vpu_mutex);
+	ret = vpu_clock_enable(vpu);
+	if (ret) {
+		dev_err(vpu->dev, "[VPU] wdt enables clock failed %d\n", ret);
+		return;
+	}
+	vpu_cfg_writel(vpu, 0x0, VPU_RESET);
+	vpu_clock_disable(vpu);
+	mutex_unlock(&vpu->vpu_mutex);
+
+	for (index = 0; index < VPU_RST_MAX; index++) {
+		if (handler[index].reset_func) {
+			handler[index].reset_func(handler[index].priv);
+			dev_dbg(vpu->dev, "wdt handler func %d\n", index);
+		}
+	}
+}
+
+int vpu_wdt_reg_handler(struct platform_device *pdev,
+			void wdt_reset(void *),
+			void *priv, enum rst_id id)
+{
+	struct mtk_vpu *vpu = platform_get_drvdata(pdev);
+	struct vpu_wdt_handler *handler = vpu->wdt.handler;
+
+	if (!vpu) {
+		dev_err(vpu->dev, "vpu device in not ready\n");
+		return -EPROBE_DEFER;
+	}
+
+	if (id < VPU_RST_MAX && wdt_reset != NULL) {
+		dev_dbg(vpu->dev, "wdt register id %d\n", id);
+		mutex_lock(&vpu->vpu_mutex);
+		handler[id].reset_func = wdt_reset;
+		handler[id].priv = priv;
+		mutex_unlock(&vpu->vpu_mutex);
+		return 0;
+	}
+
+	dev_err(vpu->dev, "register vpu wdt handler failed\n");
+	return -EINVAL;
+}
+
+unsigned int vpu_get_venc_hw_capa(struct platform_device *pdev)
+{
+	struct mtk_vpu *vpu = platform_get_drvdata(pdev);
+
+	return vpu->run.enc_capability;
+}
+
+void *vpu_mapping_dm_addr(struct platform_device *pdev,
+			  u32 dtcm_dmem_addr)
+{
+	struct mtk_vpu *vpu = platform_get_drvdata(pdev);
+
+	if (!dtcm_dmem_addr ||
+	    (dtcm_dmem_addr > (VPU_DTCM_SIZE + VPU_EXT_D_SIZE))) {
+		dev_err(vpu->dev, "invalid virtual data memory address\n");
+		return ERR_PTR(-EINVAL);
+	}
+
+	if (dtcm_dmem_addr < VPU_DTCM_SIZE)
+		return dtcm_dmem_addr + vpu->reg.tcm + VPU_DTCM_OFFSET;
+
+	return vpu->extmem[D_FW].va + (dtcm_dmem_addr - VPU_DTCM_SIZE);
+}
+
+struct platform_device *vpu_get_plat_device(struct platform_device *pdev)
+{
+	struct device *dev = &pdev->dev;
+	struct device_node *vpu_node;
+	struct platform_device *vpu_pdev;
+
+	vpu_node = of_parse_phandle(dev->of_node, "mediatek,vpu", 0);
+	if (!vpu_node) {
+		dev_err(dev, "can't get vpu node\n");
+		return NULL;
+	}
+
+	vpu_pdev = of_find_device_by_node(vpu_node);
+	if (WARN_ON(!vpu_pdev)) {
+		dev_err(dev, "vpu pdev failed\n");
+		of_node_put(vpu_node);
+		return NULL;
+	}
+
+	return vpu_pdev;
+}
+
+/* load vpu program/data memory */
+static int load_requested_vpu(struct mtk_vpu *vpu,
+			      const struct firmware *vpu_fw,
+			      u8 fw_type)
+{
+	size_t tcm_size = fw_type ? VPU_DTCM_SIZE : VPU_PTCM_SIZE;
+	size_t fw_size = fw_type ? VPU_D_FW_SIZE : VPU_P_FW_SIZE;
+	char *fw_name = fw_type ? VPU_D_FW : VPU_P_FW;
+	size_t dl_size = 0;
+	size_t extra_fw_size = 0;
+	void *dest;
+	int ret;
+
+	ret = request_firmware(&vpu_fw, fw_name, vpu->dev);
+	if (ret < 0) {
+		dev_err(vpu->dev, "Failed to load %s, %d\n", fw_name, ret);
+		return ret;
+	}
+	dl_size = vpu_fw->size;
+	if (dl_size > fw_size) {
+		dev_err(vpu->dev, "fw %s size %zu is abnormal\n", fw_name,
+			dl_size);
+		release_firmware(vpu_fw);
+		return  -EFBIG;
+	}
+	dev_dbg(vpu->dev, "Downloaded fw %s size: %zu.\n",
+		fw_name,
+		dl_size);
+	/* reset VPU */
+	vpu_cfg_writel(vpu, 0x0, VPU_RESET);
+
+	/* handle extended firmware size */
+	if (dl_size > tcm_size) {
+		dev_dbg(vpu->dev, "fw size %lx > limited fw size %lx\n",
+			dl_size, tcm_size);
+		extra_fw_size = dl_size - tcm_size;
+		dev_dbg(vpu->dev, "extra_fw_size %lx\n", extra_fw_size);
+		dl_size = tcm_size;
+	}
+	dest = vpu->reg.tcm;
+	if (fw_type == D_FW)
+		dest += VPU_DTCM_OFFSET;
+	memcpy(dest, vpu_fw->data, dl_size);
+	/* download to extended memory if need */
+	if (extra_fw_size > 0) {
+		dest = vpu->extmem[fw_type].va;
+		dev_dbg(vpu->dev, "download extended memory type %x\n",
+			fw_type);
+		memcpy(dest, vpu_fw->data + tcm_size, extra_fw_size);
+	}
+
+	release_firmware(vpu_fw);
+
+	return 0;
+}
+
+int vpu_load_firmware(struct platform_device *pdev)
+{
+	struct mtk_vpu *vpu = platform_get_drvdata(pdev);
+	struct device *dev = &pdev->dev;
+	struct vpu_run *run = &vpu->run;
+	const struct firmware *vpu_fw;
+	int ret;
+
+	if (!pdev) {
+		dev_err(dev, "VPU platform device is invalid\n");
+		return -EINVAL;
+	}
+
+	mutex_lock(&vpu->vpu_mutex);
+
+	ret = vpu_clock_enable(vpu);
+	if (ret) {
+		dev_err(dev, "enable clock failed %d\n", ret);
+		goto OUT_LOAD_FW;
+	}
+
+	if (vpu_running(vpu)) {
+		vpu_clock_disable(vpu);
+		mutex_unlock(&vpu->vpu_mutex);
+		dev_warn(dev, "vpu is running already\n");
+		return 0;
+	}
+
+	run->signaled = false;
+	dev_dbg(vpu->dev, "firmware request\n");
+	/* Downloading program firmware to device*/
+	ret = load_requested_vpu(vpu, vpu_fw, P_FW);
+	if (ret < 0) {
+		dev_err(dev, "Failed to request %s, %d\n", VPU_P_FW, ret);
+		goto OUT_LOAD_FW;
+	}
+
+	/* Downloading data firmware to device */
+	ret = load_requested_vpu(vpu, vpu_fw, D_FW);
+	if (ret < 0) {
+		dev_err(dev, "Failed to request %s, %d\n", VPU_D_FW, ret);
+		goto OUT_LOAD_FW;
+	}
+
+	/* boot up vpu */
+	vpu_cfg_writel(vpu, 0x1, VPU_RESET);
+
+	ret = wait_event_interruptible_timeout(run->wq,
+					       run->signaled,
+					       msecs_to_jiffies(INIT_TIMEOUT_MS)
+					       );
+	if (ret == 0) {
+		ret = -ETIME;
+		dev_err(dev, "wait vpu initialization timout!\n");
+		goto OUT_LOAD_FW;
+	} else if (-ERESTARTSYS == ret) {
+		dev_err(dev, "wait vpu interrupted by a signal!\n");
+		goto OUT_LOAD_FW;
+	}
+
+	ret = 0;
+	dev_info(dev, "vpu is ready. Fw version %s\n", run->fw_ver);
+
+OUT_LOAD_FW:
+	vpu_clock_disable(vpu);
+	mutex_unlock(&vpu->vpu_mutex);
+
+	return ret;
+}
+
+int vpu_compare_version(struct platform_device *pdev,
+			const char *expected_version)
+{
+	struct mtk_vpu *vpu = platform_get_drvdata(pdev);
+	int cur_major, cur_minor, cur_build, cur_rel, cur_ver_num;
+	int major, minor, build, rel, ver_num;
+	char *cur_version = vpu->run.fw_ver;
+
+	cur_ver_num = sscanf(cur_version, "%d.%d.%d-rc%d",
+			     &cur_major, &cur_minor, &cur_build, &cur_rel);
+	if (cur_ver_num < 3)
+		return -1;
+	ver_num = sscanf(expected_version, "%d.%d.%d-rc%d",
+			 &major, &minor, &build, &rel);
+	if (ver_num < 3)
+		return -1;
+
+	if (cur_major < major)
+		return -1;
+	if (cur_major > major)
+		return 1;
+
+	if (cur_minor < minor)
+		return -1;
+	if (cur_minor > minor)
+		return 1;
+
+	if (cur_build < build)
+		return -1;
+	if (cur_build > build)
+		return 1;
+
+	if (cur_ver_num < ver_num)
+		return -1;
+	if (cur_ver_num > ver_num)
+		return 1;
+
+	if (ver_num > 3) {
+		if (cur_rel < rel)
+			return -1;
+		if (cur_rel > rel)
+			return 1;
+	}
+
+	return 0;
+}
+
+static void vpu_init_ipi_handler(void *data, unsigned int len, void *priv)
+{
+	struct mtk_vpu *vpu = (struct mtk_vpu *)priv;
+	struct vpu_run *run = (struct vpu_run *)data;
+
+	vpu->run.signaled = run->signaled;
+	strncpy(vpu->run.fw_ver, run->fw_ver, VPU_FW_VER_LEN);
+	vpu->run.enc_capability = run->enc_capability;
+	wake_up_interruptible(&vpu->run.wq);
+}
+
+#ifdef CONFIG_DEBUG_FS
+static int vpu_debug_open(struct inode *inode, struct file *file)
+{
+	file->private_data = inode->i_private;
+	return 0;
+}
+
+static ssize_t vpu_debug_read(struct file *file, char __user *user_buf,
+			      size_t count, loff_t *ppos)
+{
+	char buf[256];
+	unsigned int len;
+	unsigned int running, pc, vpu_to_host, host_to_vpu, wdt;
+	int ret;
+	struct device *dev = file->private_data;
+	struct mtk_vpu *vpu = dev_get_drvdata(dev);
+
+	ret = vpu_clock_enable(vpu);
+	if (ret) {
+		dev_err(vpu->dev, "[VPU] enable clock failed %d\n", ret);
+		return 0;
+	}
+
+	/* vpu register status */
+	running = vpu_running(vpu);
+	pc = vpu_cfg_readl(vpu, VPU_PC_REG);
+	wdt = vpu_cfg_readl(vpu, VPU_WDT_REG);
+	host_to_vpu = vpu_cfg_readl(vpu, HOST_TO_VPU);
+	vpu_to_host = vpu_cfg_readl(vpu, VPU_TO_HOST);
+	vpu_clock_disable(vpu);
+
+	if (running) {
+		len = sprintf(buf, "VPU is running\n\n"
+		"FW Version: %s\n"
+		"PC: 0x%x\n"
+		"WDT: 0x%x\n"
+		"Host to VPU: 0x%x\n"
+		"VPU to Host: 0x%x\n",
+		vpu->run.fw_ver, pc, wdt,
+		host_to_vpu, vpu_to_host);
+	} else {
+		len = sprintf(buf, "VPU not running\n");
+	}
+
+	return simple_read_from_buffer(user_buf, count, ppos, buf, len);
+}
+
+static const struct file_operations vpu_debug_fops = {
+	.open = vpu_debug_open,
+	.read = vpu_debug_read,
+};
+#endif /* CONFIG_DEBUG_FS */
+
+static void vpu_free_ext_mem(struct mtk_vpu *vpu, u8 fw_type)
+{
+	struct device *dev = vpu->dev;
+	size_t fw_ext_size = fw_type ? VPU_EXT_D_SIZE : VPU_EXT_P_SIZE;
+
+	dma_free_coherent(dev, fw_ext_size, vpu->extmem[fw_type].va,
+			  vpu->extmem[fw_type].pa);
+}
+
+static int vpu_alloc_ext_mem(struct mtk_vpu *vpu, u32 fw_type)
+{
+	struct device *dev = vpu->dev;
+	size_t fw_ext_size = fw_type ? VPU_EXT_D_SIZE : VPU_EXT_P_SIZE;
+	u32 vpu_ext_mem0 = fw_type ? VPU_DMEM_EXT0_ADDR : VPU_PMEM_EXT0_ADDR;
+	u32 vpu_ext_mem1 = fw_type ? VPU_DMEM_EXT1_ADDR : VPU_PMEM_EXT1_ADDR;
+	u32 offset_4gb = vpu->enable_4GB ? 0x40000000 : 0;
+
+	vpu->extmem[fw_type].va = dma_alloc_coherent(dev,
+					       fw_ext_size,
+					       &vpu->extmem[fw_type].pa,
+					       GFP_KERNEL);
+	if (!vpu->extmem[fw_type].va) {
+		dev_err(dev, "Failed to allocate the extended program memory\n");
+		return PTR_ERR(vpu->extmem[fw_type].va);
+	}
+
+	/* Disable extend0. Enable extend1 */
+	vpu_cfg_writel(vpu, 0x1, vpu_ext_mem0);
+	vpu_cfg_writel(vpu, (vpu->extmem[fw_type].pa & 0xFFFFF000) + offset_4gb,
+		       vpu_ext_mem1);
+
+	dev_info(dev, "%s extend memory phy=0x%llx virt=0x%p\n",
+		 fw_type ? "Data" : "Program",
+		 (unsigned long long)vpu->extmem[fw_type].pa,
+		 vpu->extmem[fw_type].va);
+
+	return 0;
+}
+
+static void vpu_ipi_handler(struct mtk_vpu *vpu)
+{
+	struct share_obj *rcv_obj = vpu->recv_buf;
+	struct vpu_ipi_desc *ipi_desc = vpu->ipi_desc;
+
+	if (rcv_obj->id < IPI_MAX && ipi_desc[rcv_obj->id].handler) {
+		ipi_desc[rcv_obj->id].handler(rcv_obj->share_buf,
+					      rcv_obj->len,
+					      ipi_desc[rcv_obj->id].priv);
+		if (rcv_obj->id > IPI_VPU_INIT) {
+			vpu->ipi_id_ack[rcv_obj->id] = true;
+			wake_up_interruptible(&vpu->ack_wq);
+		}
+	} else {
+		dev_err(vpu->dev, "No such ipi id = %d\n", rcv_obj->id);
+	}
+}
+
+static int vpu_ipi_init(struct mtk_vpu *vpu)
+{
+	/* Disable VPU to host interrupt */
+	vpu_cfg_writel(vpu, 0x0, VPU_TO_HOST);
+
+	/* shared buffer initialization */
+	vpu->recv_buf = (struct share_obj *)(vpu->reg.tcm + VPU_DTCM_OFFSET);
+	vpu->send_buf = vpu->recv_buf + 1;
+	memset(vpu->recv_buf, 0, sizeof(struct share_obj));
+	memset(vpu->send_buf, 0, sizeof(struct share_obj));
+	mutex_init(&vpu->vpu_mutex);
+
+	return 0;
+}
+
+static irqreturn_t vpu_irq_handler(int irq, void *priv)
+{
+	struct mtk_vpu *vpu = priv;
+	u32 vpu_to_host;
+	int ret;
+
+	/*
+	 * Clock should have been enabled already.
+	 * Enable again in case vpu_ipi_send times out
+	 * and has disabled the clock.
+	 */
+	ret = clk_enable(vpu->clk);
+	if (ret) {
+		dev_err(vpu->dev, "[VPU] enable clock failed %d\n", ret);
+		return IRQ_NONE;
+	}
+	vpu_to_host = vpu_cfg_readl(vpu, VPU_TO_HOST);
+	if (vpu_to_host & VPU_IPC_INT) {
+		vpu_ipi_handler(vpu);
+	} else {
+		dev_err(vpu->dev, "vpu watchdog timeout! 0x%x", vpu_to_host);
+		if (vpu->wdt.wq)
+			queue_work(vpu->wdt.wq, &vpu->wdt.ws);
+	}
+
+	/* VPU won't send another interrupt until we set VPU_TO_HOST to 0. */
+	vpu_cfg_writel(vpu, 0x0, VPU_TO_HOST);
+	clk_disable(vpu->clk);
+
+	return IRQ_HANDLED;
+}
+
+#ifdef CONFIG_DEBUG_FS
+static struct dentry *vpu_debugfs;
+#endif
+static int mtk_vpu_probe(struct platform_device *pdev)
+{
+	struct mtk_vpu *vpu;
+	struct device *dev;
+	struct resource *res;
+	int ret = 0;
+
+	dev_dbg(&pdev->dev, "initialization\n");
+
+	dev = &pdev->dev;
+	vpu = devm_kzalloc(dev, sizeof(*vpu), GFP_KERNEL);
+	if (!vpu)
+		return -ENOMEM;
+
+	vpu->dev = &pdev->dev;
+	res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "tcm");
+	vpu->reg.tcm = devm_ioremap_resource(dev, res);
+	if (IS_ERR(vpu->reg.tcm)) {
+		dev_err(dev, "devm_ioremap_resource vpu tcm failed.\n");
+		return PTR_ERR(vpu->reg.tcm);
+	}
+
+	res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "cfg_reg");
+	vpu->reg.cfg = devm_ioremap_resource(dev, res);
+	if (IS_ERR(vpu->reg.cfg)) {
+		dev_err(dev, "devm_ioremap_resource vpu cfg failed.\n");
+		return PTR_ERR(vpu->reg.cfg);
+	}
+
+	/* Get VPU clock */
+	vpu->clk = devm_clk_get(dev, "main");
+	if (!vpu->clk) {
+		dev_err(dev, "get vpu clock failed\n");
+		return -EINVAL;
+	}
+
+	platform_set_drvdata(pdev, vpu);
+
+	ret = clk_prepare(vpu->clk);
+	if (ret) {
+		dev_err(dev, "prepare vpu clock failed\n");
+		return ret;
+	}
+
+	/* VPU watchdog */
+	vpu->wdt.wq = create_singlethread_workqueue("vpu_wdt");
+	if (!vpu->wdt.wq) {
+		dev_err(dev, "initialize wdt workqueue failed\n");
+		return -ENOMEM;
+	}
+	INIT_WORK(&vpu->wdt.ws, vpu_wdt_reset_func);
+
+	ret = vpu_clock_enable(vpu);
+	if (ret) {
+		dev_err(dev, "enable vpu clock failed\n");
+		goto workqueue_destroy;
+	}
+
+	dev_dbg(dev, "vpu ipi init\n");
+	ret = vpu_ipi_init(vpu);
+	if (ret) {
+		dev_err(dev, "Failed to init ipi\n");
+		goto disable_vpu_clk;
+	}
+
+	/* register vpu initialization IPI */
+	ret = vpu_ipi_register(pdev, IPI_VPU_INIT, vpu_init_ipi_handler,
+			       "vpu_init", vpu);
+	if (ret) {
+		dev_err(dev, "Failed to register IPI_VPU_INIT\n");
+		goto vpu_mutex_destroy;
+	}
+
+#ifdef CONFIG_DEBUG_FS
+	vpu_debugfs = debugfs_create_file("mtk_vpu", S_IRUGO, NULL, (void *)dev,
+					  &vpu_debug_fops);
+	if (!vpu_debugfs) {
+		ret = -ENOMEM;
+		goto cleanup_ipi;
+	}
+#endif
+
+	/* Set PTCM to 96K and DTCM to 32K */
+	vpu_cfg_writel(vpu, 0x2, VPU_TCM_CFG);
+
+	vpu->enable_4GB = !!(max_pfn > (0xffffffffUL >> PAGE_SHIFT));
+	dev_dbg(dev, "4GB mode %u\n", vpu->enable_4GB);
+
+	if (vpu->enable_4GB) {
+		ret = of_reserved_mem_device_init(dev);
+		if (ret)
+			dev_info(dev, "init reserved memory failed\n");
+			/* continue to use dynamic allocation if failed */
+	}
+
+	ret = vpu_alloc_ext_mem(vpu, D_FW);
+	if (ret) {
+		dev_err(dev, "Allocate DM failed\n");
+		goto remove_debugfs;
+	}
+
+	ret = vpu_alloc_ext_mem(vpu, P_FW);
+	if (ret) {
+		dev_err(dev, "Allocate PM failed\n");
+		goto free_d_mem;
+	}
+
+	init_waitqueue_head(&vpu->run.wq);
+	init_waitqueue_head(&vpu->ack_wq);
+
+	res = platform_get_resource(pdev, IORESOURCE_IRQ, 0);
+	if (!res) {
+		dev_err(dev, "get IRQ resource failed.\n");
+		ret = -ENXIO;
+		goto free_p_mem;
+	}
+	vpu->reg.irq = platform_get_irq(pdev, 0);
+	ret = devm_request_irq(dev, vpu->reg.irq, vpu_irq_handler, 0,
+			       pdev->name, vpu);
+	if (ret) {
+		dev_err(dev, "failed to request irq\n");
+		goto free_p_mem;
+	}
+
+	vpu_clock_disable(vpu);
+	dev_dbg(dev, "initialization completed\n");
+
+	return 0;
+
+free_p_mem:
+	vpu_free_ext_mem(vpu, P_FW);
+free_d_mem:
+	vpu_free_ext_mem(vpu, D_FW);
+remove_debugfs:
+	of_reserved_mem_device_release(dev);
+#ifdef CONFIG_DEBUG_FS
+	debugfs_remove(vpu_debugfs);
+cleanup_ipi:
+#endif
+	memset(vpu->ipi_desc, 0, sizeof(struct vpu_ipi_desc) * IPI_MAX);
+vpu_mutex_destroy:
+	mutex_destroy(&vpu->vpu_mutex);
+disable_vpu_clk:
+	vpu_clock_disable(vpu);
+workqueue_destroy:
+	destroy_workqueue(vpu->wdt.wq);
+
+	return ret;
+}
+
+static const struct of_device_id mtk_vpu_match[] = {
+	{
+		.compatible = "mediatek,mt8173-vpu",
+	},
+	{},
+};
+MODULE_DEVICE_TABLE(of, mtk_vpu_match);
+
+static int mtk_vpu_remove(struct platform_device *pdev)
+{
+	struct mtk_vpu *vpu = platform_get_drvdata(pdev);
+
+#ifdef CONFIG_DEBUG_FS
+	debugfs_remove(vpu_debugfs);
+#endif
+	if (vpu->wdt.wq) {
+		flush_workqueue(vpu->wdt.wq);
+		destroy_workqueue(vpu->wdt.wq);
+	}
+	vpu_free_ext_mem(vpu, P_FW);
+	vpu_free_ext_mem(vpu, D_FW);
+	mutex_destroy(&vpu->vpu_mutex);
+	clk_unprepare(vpu->clk);
+
+	return 0;
+}
+
+static struct platform_driver mtk_vpu_driver = {
+	.probe	= mtk_vpu_probe,
+	.remove	= mtk_vpu_remove,
+	.driver	= {
+		.name	= "mtk_vpu",
+		.owner	= THIS_MODULE,
+		.of_match_table = mtk_vpu_match,
+	},
+};
+
+module_platform_driver(mtk_vpu_driver);
+
+MODULE_LICENSE("GPL v2");
+MODULE_DESCRIPTION("Mediatek Video Prosessor Unit driver");
diff --git a/drivers/media/platform/mtk-vpu/mtk_vpu.h b/drivers/media/platform/mtk-vpu/mtk_vpu.h
new file mode 100644
index 0000000..d9c3cde
--- /dev/null
+++ b/drivers/media/platform/mtk-vpu/mtk_vpu.h
@@ -0,0 +1,167 @@
+/*
+* Copyright (c) 2015 MediaTek Inc.
+* Author: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
+*
+* This program is free software; you can redistribute it and/or modify
+* it under the terms of the GNU General Public License version 2 as
+* published by the Free Software Foundation.
+*
+* This program is distributed in the hope that it will be useful,
+* but WITHOUT ANY WARRANTY; without even the implied warranty of
+* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+* GNU General Public License for more details.
+*/
+
+#ifndef _MTK_VPU_H
+#define _MTK_VPU_H
+
+#include <linux/platform_device.h>
+
+/**
+ * VPU (video processor unit) is a tiny processor controlling video hardware
+ * related to video codec, scaling and color format converting.
+ * VPU interfaces with other blocks by share memory and interrupt.
+ **/
+
+typedef void (*ipi_handler_t) (void *data,
+			       unsigned int len,
+			       void *priv);
+
+/**
+ * enum ipi_id - the id of inter-processor interrupt
+ *
+ * @IPI_VPU_INIT:	 The interrupt from vpu is to notfiy kernel
+			 VPU initialization completed.
+			 IPI_VPU_INIT is sent from VPU when firmware is
+			 loaded. AP doesn't need to send IPI_VPU_INIT
+			 command to VPU.
+			 For other IPI below, AP should send the request
+			 to VPU to trigger the interrupt.
+ * @IPI_VENC_H264:	 The interrupt from vpu is to notify kernel to
+			 handle H264 video encoder job, and vice versa.
+ * @IPI_VENC_VP8:	 The interrupt fro vpu is to notify kernel to
+			 handle VP8 video encoder job,, and vice versa.
+ * @IPI_MAX:		 The maximum IPI number
+ */
+
+enum ipi_id {
+	IPI_VPU_INIT = 0,
+	IPI_VENC_H264,
+	IPI_VENC_VP8,
+	IPI_MAX,
+};
+
+/**
+ * enum rst_id - reset id to register reset function for VPU watchdog timeout
+ *
+ * @VPU_RST_DEC: decoder reset id
+ * @VPU_RST_ENC: encoder reset id
+ * @VPU_RST_MDP: MDP (Media Data Path) reset id
+ * @VPU_RST_MAX: maximum reset id
+ */
+enum rst_id {
+	VPU_RST_ENC,
+	VPU_RST_DEC,
+	VPU_RST_MDP,
+	VPU_RST_MAX,
+};
+
+/**
+ * vpu_ipi_register - register an ipi function
+ *
+ * @pdev:	VPU platform device
+ * @id:		IPI ID
+ * @handler:	IPI handler
+ * @name:	IPI name
+ * @priv:	private data for IPI handler
+ *
+ * Register an ipi function to receive ipi interrupt from VPU.
+ *
+ * Return: Return 0 if ipi registers successfully, otherwise it is failed.
+ */
+int vpu_ipi_register(struct platform_device *pdev, enum ipi_id id,
+		     ipi_handler_t handler, const char *name, void *priv);
+
+/**
+ * vpu_ipi_send - send data from AP to vpu.
+ *
+ * @pdev:	VPU platform device
+ * @id:		IPI ID
+ * @buf:	the data buffer
+ * @len:	the data buffer length
+ *
+ * This function is thread-safe. When this function returns,
+ * VPU has received the data and starts the processing.
+ * When the processing completes, IPI handler registered
+ * by vpu_ipi_register will be called in interrupt context.
+ *
+ * Return: Return 0 if sending data successfully, otherwise it is failed.
+ **/
+int vpu_ipi_send(struct platform_device *pdev,
+		 enum ipi_id id, void *buf,
+		 unsigned int len);
+
+/**
+ * vpu_get_plat_device - get VPU's platform device
+ *
+ * @pdev:	the platform device of the module requesting VPU platform
+ *		device for using VPU API.
+ *
+ * Return: Return NULL if it is failed.
+ * otherwise it is VPU's platform device
+ **/
+struct platform_device *vpu_get_plat_device(struct platform_device *pdev);
+
+/**
+ * vpu_wdt_reg_handler - register a VPU watchdog handler
+ *
+ * @pdev:               VPU platform device
+ * @vpu_wdt_reset_func:	the callback reset function
+ * @private_data:       the private data for reset function
+ * @rst_id:		reset id
+ *
+ * Register a handler performing own tasks when vpu reset by watchdog
+ *
+ * Return: Return 0 if the handler is added successfully,
+ * otherwise it is failed.
+ *
+ **/
+int vpu_wdt_reg_handler(struct platform_device *pdev,
+			void vpu_wdt_reset_func(void *),
+			void *priv, enum rst_id id);
+
+/**
+ * vpu_get_venc_hw_capa - get video encoder hardware capability
+ *
+ * @pdev:	VPU platform device
+ *
+ * Return: video encoder hardware capability
+ **/
+unsigned int vpu_get_venc_hw_capa(struct platform_device *pdev);
+
+/**
+ * vpu_load_firmware - download VPU firmware and boot it
+ *
+ * @pdev:	VPU platform device
+ *
+ * Return: Return 0 if downloading firmware successfully,
+ * otherwise it is failed
+ **/
+int vpu_load_firmware(struct platform_device *pdev);
+
+/**
+ * vpu_mapping_dm_addr - Mapping DTCM/DMEM to kernel virtual address
+ *
+ * @pdev:	VPU platform device
+ * @dmem_addr:	VPU's data memory address
+ *
+ * Mapping the VPU's DTCM (Data Tightly-Coupled Memory) /
+ * DMEM (Data Extended Memory) memory address to
+ * kernel virtual address.
+ *
+ * Return: Return ERR_PTR(-EINVAL) if mapping failed,
+ * otherwise the mapped kernel virtual address
+ **/
+void *vpu_mapping_dm_addr(struct platform_device *pdev,
+			  u32 dtcm_dmem_addr);
+#endif /* _MTK_VPU_H */
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 102+ messages in thread

* [PATCH v4 2/8] [media] VPU: mediatek: support Mediatek VPU
@ 2016-02-04 11:34     ` Tiffany Lin
  0 siblings, 0 replies; 102+ messages in thread
From: Tiffany Lin @ 2016-02-04 11:34 UTC (permalink / raw)
  To: linux-arm-kernel

The VPU driver for hw video codec embedded in Mediatek's MT8173 SOCs.
It is able to handle video decoding/encoding of in a range of formats.
The driver provides with VPU firmware download, memory management and
the communication interface between CPU and VPU.
For VPU initialization, it will create virtual memory for CPU access and
IOMMU address for vcodec hw device access. When a decode/encode instance
opens a device node, vpu driver will download vpu firmware to the device.
A decode/encode instant will decode/encode a frame using VPU
interface to interrupt vpu to handle decoding/encoding jobs.

Signed-off-by: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
---
 drivers/media/platform/Kconfig           |    9 +
 drivers/media/platform/Makefile          |    2 +
 drivers/media/platform/mtk-vpu/Makefile  |    1 +
 drivers/media/platform/mtk-vpu/mtk_vpu.c |  994 ++++++++++++++++++++++++++++++
 drivers/media/platform/mtk-vpu/mtk_vpu.h |  167 +++++
 5 files changed, 1173 insertions(+)
 create mode 100644 drivers/media/platform/mtk-vpu/Makefile
 create mode 100644 drivers/media/platform/mtk-vpu/mtk_vpu.c
 create mode 100644 drivers/media/platform/mtk-vpu/mtk_vpu.h

diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
index ccbc974..ba812d6 100644
--- a/drivers/media/platform/Kconfig
+++ b/drivers/media/platform/Kconfig
@@ -148,6 +148,15 @@ config VIDEO_CODA
 	   Coda is a range of video codec IPs that supports
 	   H.264, MPEG-4, and other video formats.
 
+config VIDEO_MEDIATEK_VPU
+	tristate "Mediatek Video Processor Unit"
+	depends on VIDEO_DEV && VIDEO_V4L2 && ARCH_MEDIATEK
+	---help---
+	    This driver provides downloading VPU firmware and
+	    communicating with VPU. This driver for hw video
+	    codec embedded in new Mediatek's SOCs. It is able
+	    to handle video decoding/encoding in a range of formats.
+
 config VIDEO_MEM2MEM_DEINTERLACE
 	tristate "Deinterlace support"
 	depends on VIDEO_DEV && VIDEO_V4L2 && DMA_ENGINE
diff --git a/drivers/media/platform/Makefile b/drivers/media/platform/Makefile
index efa0295..e5b19c6 100644
--- a/drivers/media/platform/Makefile
+++ b/drivers/media/platform/Makefile
@@ -55,3 +55,5 @@ obj-$(CONFIG_VIDEO_AM437X_VPFE)		+= am437x/
 obj-$(CONFIG_VIDEO_XILINX)		+= xilinx/
 
 ccflags-y += -I$(srctree)/drivers/media/i2c
+
+obj-$(CONFIG_VIDEO_MEDIATEK_VPU)	+= mtk-vpu/
diff --git a/drivers/media/platform/mtk-vpu/Makefile b/drivers/media/platform/mtk-vpu/Makefile
new file mode 100644
index 0000000..d890a66
--- /dev/null
+++ b/drivers/media/platform/mtk-vpu/Makefile
@@ -0,0 +1 @@
+obj-y += mtk_vpu.o
diff --git a/drivers/media/platform/mtk-vpu/mtk_vpu.c b/drivers/media/platform/mtk-vpu/mtk_vpu.c
new file mode 100644
index 0000000..f54fd89
--- /dev/null
+++ b/drivers/media/platform/mtk-vpu/mtk_vpu.c
@@ -0,0 +1,994 @@
+/*
+* Copyright (c) 2015 MediaTek Inc.
+* Author: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
+*
+* This program is free software; you can redistribute it and/or modify
+* it under the terms of the GNU General Public License version 2 as
+* published by the Free Software Foundation.
+*
+* This program is distributed in the hope that it will be useful,
+* but WITHOUT ANY WARRANTY; without even the implied warranty of
+* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+* GNU General Public License for more details.
+*/
+#include <linux/bootmem.h>
+#include <linux/clk.h>
+#include <linux/debugfs.h>
+#include <linux/firmware.h>
+#include <linux/interrupt.h>
+#include <linux/iommu.h>
+#include <linux/module.h>
+#include <linux/of_address.h>
+#include <linux/of_irq.h>
+#include <linux/of_platform.h>
+#include <linux/of_reserved_mem.h>
+#include <linux/sched.h>
+#include <linux/sizes.h>
+
+#include "mtk_vpu.h"
+
+/**
+ * VPU (video processor unit) is a tiny processor controlling video hardware
+ * related to video codec, scaling and color format converting.
+ * VPU interfaces with other blocks by share memory and interrupt.
+ **/
+
+#define INIT_TIMEOUT_MS		2000U
+#define IPI_TIMEOUT_MS		2000U
+#define VPU_FW_VER_LEN		16
+
+/* maximum program/data TCM (Tightly-Coupled Memory) size */
+#define VPU_PTCM_SIZE		(96 * SZ_1K)
+#define VPU_DTCM_SIZE		(32 * SZ_1K)
+/* the offset to get data tcm address */
+#define VPU_DTCM_OFFSET		0x18000UL
+/* daynamic allocated maximum extended memory size */
+#define VPU_EXT_P_SIZE		SZ_1M
+#define VPU_EXT_D_SIZE		SZ_4M
+/* maximum binary firmware size */
+#define VPU_P_FW_SIZE		(VPU_PTCM_SIZE + VPU_EXT_P_SIZE)
+#define VPU_D_FW_SIZE		(VPU_DTCM_SIZE + VPU_EXT_D_SIZE)
+/* the size of share buffer between Host and  VPU */
+#define SHARE_BUF_SIZE		48
+
+/* binary firmware name */
+#define VPU_P_FW		"vpu_p.bin"
+#define VPU_D_FW		"vpu_d.bin"
+
+#define VPU_RESET		0x0
+#define VPU_TCM_CFG		0x0008
+#define VPU_PMEM_EXT0_ADDR	0x000C
+#define VPU_PMEM_EXT1_ADDR	0x0010
+#define VPU_TO_HOST		0x001C
+#define VPU_DMEM_EXT0_ADDR	0x0014
+#define VPU_DMEM_EXT1_ADDR	0x0018
+#define HOST_TO_VPU		0x0024
+#define VPU_PC_REG		0x0060
+#define VPU_WDT_REG		0x0084
+
+/* vpu inter-processor communication interrupt */
+#define VPU_IPC_INT		BIT(8)
+
+/**
+ * enum vpu_fw_type - VPU firmware type
+ *
+ * @P_FW: program firmware
+ * @D_FW: data firmware
+ *
+ */
+enum vpu_fw_type {
+	P_FW,
+	D_FW,
+};
+
+/**
+ * struct vpu_mem - VPU extended program/data memory information
+ *
+ * @va:		the kernel virtual memory address of VPU extended memory
+ * @pa:		the physical memory address of VPU extended memory
+ *
+ */
+struct vpu_mem {
+	void *va;
+	dma_addr_t pa;
+};
+
+/**
+ * struct vpu_regs - VPU TCM and configuration registers
+ *
+ * @tcm:	the register for VPU Tightly-Coupled Memory
+ * @cfg:	the register for VPU configuration
+ * @irq:	the irq number for VPU interrupt
+ */
+struct vpu_regs {
+	void __iomem *tcm;
+	void __iomem *cfg;
+	int irq;
+};
+
+/**
+ * struct vpu_wdt_handler - VPU watchdog reset handler
+ *
+ * @reset_func:	reset handler
+ * @priv:	private data
+ */
+struct vpu_wdt_handler {
+	void (*reset_func)(void *);
+	void *priv;
+};
+
+/**
+ * struct vpu_wdt - VPU watchdog workqueue
+ *
+ * @handler:	VPU watchdog reset handler
+ * @ws:		workstruct for VPU watchdog
+ * @wq:		workqueue for VPU watchdog
+ */
+struct vpu_wdt {
+	struct vpu_wdt_handler handler[VPU_RST_MAX];
+	struct work_struct ws;
+	struct workqueue_struct *wq;
+};
+
+/**
+ * struct vpu_run - VPU initialization status
+ *
+ * @signaled:		the signal of vpu initialization completed
+ * @fw_ver:		VPU firmware version
+ * @enc_capability:	encoder capability
+ * @wq:			wait queue for VPU initialization status
+ */
+struct vpu_run {
+	u32 signaled;
+	char fw_ver[VPU_FW_VER_LEN];
+	unsigned int	enc_capability;
+	wait_queue_head_t wq;
+};
+
+/**
+ * struct vpu_ipi_desc - VPU IPI descriptor
+ *
+ * @handler:	IPI handler
+ * @name:	the name of IPI handler
+ * @priv:	the private data of IPI handler
+ */
+struct vpu_ipi_desc {
+	ipi_handler_t handler;
+	const char *name;
+	void *priv;
+};
+
+/**
+ * struct share_obj - DTCM (Data Tightly-Coupled Memory) buffer shared with
+ *		      AP and VPU
+ *
+ * @id:		IPI id
+ * @len:	share buffer length
+ * @share_buf:	share buffer data
+ */
+struct share_obj {
+	s32 id;
+	u32 len;
+	unsigned char share_buf[SHARE_BUF_SIZE];
+};
+
+/**
+ * struct mtk_vpu - vpu driver data
+ * @extmem:		VPU extended memory information
+ * @reg:		VPU TCM and configuration registers
+ * @run:		VPU initialization status
+ * @ipi_desc:		VPU IPI descriptor
+ * @recv_buf:		VPU DTCM share buffer for receiving. The
+ *			receive buffer is only accessed in interrupt context.
+ * @send_buf:		VPU DTCM share buffer for sending
+ * @dev:		VPU struct device
+ * @clk:		VPU clock on/off
+ * @enable_4GB:		VPU 4GB mode on/off
+ * @vpu_mutex:		protect mtk_vpu (except recv_buf) and ensure only
+ *			one client to use VPU service at a time. For example,
+ *			suppose a client is using VPU to decode VP8.
+ *			If the other client wants to encode VP8,
+ *			it has to wait until VP8 decode completes.
+ * @wdt_refcnt		WDT reference count to make sure the watchdog can be
+ *			disabled if no other client is using VPU service
+ * @ipi_ack_signaled:	The ACKs for registered IPI function sending
+ *			interrupt to VPU
+ * @ack_wq:		The wait queue for each codec and mdp. When sleeping
+ *			processes wake up, they will check the condition
+ *			"ipi_ack_signaled" to run the corresponding action or
+ *			go back to sleep.
+ *
+ */
+struct mtk_vpu {
+	struct vpu_mem extmem[2];
+	struct vpu_regs reg;
+	struct vpu_run run;
+	struct vpu_wdt wdt;
+	struct vpu_ipi_desc ipi_desc[IPI_MAX];
+	struct share_obj *recv_buf;
+	struct share_obj *send_buf;
+	struct device *dev;
+	struct clk *clk;
+	bool enable_4GB;
+	struct mutex vpu_mutex; /* for protecting vpu data data structure */
+	atomic_t wdt_refcnt;
+	wait_queue_head_t ack_wq;
+	bool ipi_id_ack[IPI_MAX];
+};
+
+static inline void vpu_cfg_writel(struct mtk_vpu *vpu, u32 val, u32 offset)
+{
+	writel(val, vpu->reg.cfg + offset);
+}
+
+static inline u32 vpu_cfg_readl(struct mtk_vpu *vpu, u32 offset)
+{
+	return readl(vpu->reg.cfg + offset);
+}
+
+static inline bool vpu_running(struct mtk_vpu *vpu)
+{
+	return vpu_cfg_readl(vpu, VPU_RESET) & BIT(0);
+}
+
+void vpu_clock_disable(struct mtk_vpu *vpu)
+{
+	/* Disable VPU watchdog */
+	if (atomic_dec_and_test(&vpu->wdt_refcnt))
+		vpu_cfg_writel(vpu,
+			       vpu_cfg_readl(vpu, VPU_WDT_REG) & ~(1L << 31),
+			       VPU_WDT_REG);
+
+	clk_disable(vpu->clk);
+}
+
+int vpu_clock_enable(struct mtk_vpu *vpu)
+{
+	int ret;
+
+	ret = clk_enable(vpu->clk);
+	if (ret)
+		return ret;
+	/* Enable VPU watchdog */
+	if (!atomic_read(&vpu->wdt_refcnt))
+		vpu_cfg_writel(vpu,
+			       vpu_cfg_readl(vpu, VPU_WDT_REG) | (1L << 31),
+			       VPU_WDT_REG);
+
+	atomic_inc(&vpu->wdt_refcnt);
+
+	return ret;
+}
+
+int vpu_ipi_register(struct platform_device *pdev,
+		     enum ipi_id id, ipi_handler_t handler,
+		     const char *name, void *priv)
+{
+	struct mtk_vpu *vpu = platform_get_drvdata(pdev);
+	struct vpu_ipi_desc *ipi_desc;
+
+	if (!vpu) {
+		dev_err(&pdev->dev, "vpu device in not ready\n");
+		return -EPROBE_DEFER;
+	}
+
+	if (id < IPI_MAX && handler) {
+		ipi_desc = vpu->ipi_desc;
+		ipi_desc[id].name = name;
+		ipi_desc[id].handler = handler;
+		ipi_desc[id].priv = priv;
+		return 0;
+	}
+
+	dev_err(&pdev->dev, "register vpu ipi with invalid arguments\n");
+	return -EINVAL;
+}
+
+int vpu_ipi_send(struct platform_device *pdev,
+		 enum ipi_id id, void *buf,
+		 unsigned int len)
+{
+	struct mtk_vpu *vpu = platform_get_drvdata(pdev);
+	struct share_obj *send_obj = vpu->send_buf;
+	unsigned long timeout;
+	int ret = 0;
+
+	if (id <= IPI_VPU_INIT || id >= IPI_MAX ||
+	    len > sizeof(send_obj->share_buf) || !buf) {
+		dev_err(vpu->dev, "failed to send ipi message\n");
+		return -EINVAL;
+	}
+
+	ret = vpu_clock_enable(vpu);
+	if (ret) {
+		dev_err(vpu->dev, "failed to enable vpu clock\n");
+		return ret;
+	}
+	if (!vpu_running(vpu)) {
+		dev_err(vpu->dev, "vpu_ipi_send: VPU is not running\n");
+		ret = -EINVAL;
+		goto clock_disable;
+	}
+
+	mutex_lock(&vpu->vpu_mutex);
+
+	 /* Wait until VPU receives the last command */
+	timeout = jiffies + msecs_to_jiffies(IPI_TIMEOUT_MS);
+	do {
+		if (time_after(jiffies, timeout)) {
+			dev_err(vpu->dev, "vpu_ipi_send: IPI timeout!\n");
+			ret = -EIO;
+			goto mut_unlock;
+		}
+	} while (vpu_cfg_readl(vpu, HOST_TO_VPU));
+
+	memcpy((void *)send_obj->share_buf, buf, len);
+	send_obj->len = len;
+	send_obj->id = id;
+
+	vpu->ipi_id_ack[id] = false;
+	/* send the command to VPU */
+	vpu_cfg_writel(vpu, 0x1, HOST_TO_VPU);
+
+	mutex_unlock(&vpu->vpu_mutex);
+
+	/* wait for VPU's ACK */
+	timeout = msecs_to_jiffies(IPI_TIMEOUT_MS);
+	ret = wait_event_interruptible_timeout(vpu->ack_wq,
+					       vpu->ipi_id_ack[id], timeout);
+	vpu->ipi_id_ack[id] = false;
+	if (ret == 0) {
+		dev_err(vpu->dev, "vpu ipi %d ack time out !", id);
+		ret = -EIO;
+		goto clock_disable;
+	} else if (-ERESTARTSYS == ret) {
+		dev_err(vpu->dev, "vpu ipi %d ack wait interrupted by a signal",
+			id);
+		ret = -ERESTARTSYS;
+		goto clock_disable;
+	}
+	vpu_clock_disable(vpu);
+
+	return 0;
+
+mut_unlock:
+	vpu->ipi_id_ack[id] = false;
+	mutex_unlock(&vpu->vpu_mutex);
+clock_disable:
+	vpu_clock_disable(vpu);
+
+	return ret;
+}
+
+static void vpu_wdt_reset_func(struct work_struct *ws)
+{
+	struct vpu_wdt *wdt = container_of(ws, struct vpu_wdt, ws);
+	struct mtk_vpu *vpu = container_of(wdt, struct mtk_vpu, wdt);
+	struct vpu_wdt_handler *handler = wdt->handler;
+	int index, ret;
+
+	dev_info(vpu->dev, "vpu reset\n");
+	mutex_lock(&vpu->vpu_mutex);
+	ret = vpu_clock_enable(vpu);
+	if (ret) {
+		dev_err(vpu->dev, "[VPU] wdt enables clock failed %d\n", ret);
+		return;
+	}
+	vpu_cfg_writel(vpu, 0x0, VPU_RESET);
+	vpu_clock_disable(vpu);
+	mutex_unlock(&vpu->vpu_mutex);
+
+	for (index = 0; index < VPU_RST_MAX; index++) {
+		if (handler[index].reset_func) {
+			handler[index].reset_func(handler[index].priv);
+			dev_dbg(vpu->dev, "wdt handler func %d\n", index);
+		}
+	}
+}
+
+int vpu_wdt_reg_handler(struct platform_device *pdev,
+			void wdt_reset(void *),
+			void *priv, enum rst_id id)
+{
+	struct mtk_vpu *vpu = platform_get_drvdata(pdev);
+	struct vpu_wdt_handler *handler = vpu->wdt.handler;
+
+	if (!vpu) {
+		dev_err(vpu->dev, "vpu device in not ready\n");
+		return -EPROBE_DEFER;
+	}
+
+	if (id < VPU_RST_MAX && wdt_reset != NULL) {
+		dev_dbg(vpu->dev, "wdt register id %d\n", id);
+		mutex_lock(&vpu->vpu_mutex);
+		handler[id].reset_func = wdt_reset;
+		handler[id].priv = priv;
+		mutex_unlock(&vpu->vpu_mutex);
+		return 0;
+	}
+
+	dev_err(vpu->dev, "register vpu wdt handler failed\n");
+	return -EINVAL;
+}
+
+unsigned int vpu_get_venc_hw_capa(struct platform_device *pdev)
+{
+	struct mtk_vpu *vpu = platform_get_drvdata(pdev);
+
+	return vpu->run.enc_capability;
+}
+
+void *vpu_mapping_dm_addr(struct platform_device *pdev,
+			  u32 dtcm_dmem_addr)
+{
+	struct mtk_vpu *vpu = platform_get_drvdata(pdev);
+
+	if (!dtcm_dmem_addr ||
+	    (dtcm_dmem_addr > (VPU_DTCM_SIZE + VPU_EXT_D_SIZE))) {
+		dev_err(vpu->dev, "invalid virtual data memory address\n");
+		return ERR_PTR(-EINVAL);
+	}
+
+	if (dtcm_dmem_addr < VPU_DTCM_SIZE)
+		return dtcm_dmem_addr + vpu->reg.tcm + VPU_DTCM_OFFSET;
+
+	return vpu->extmem[D_FW].va + (dtcm_dmem_addr - VPU_DTCM_SIZE);
+}
+
+struct platform_device *vpu_get_plat_device(struct platform_device *pdev)
+{
+	struct device *dev = &pdev->dev;
+	struct device_node *vpu_node;
+	struct platform_device *vpu_pdev;
+
+	vpu_node = of_parse_phandle(dev->of_node, "mediatek,vpu", 0);
+	if (!vpu_node) {
+		dev_err(dev, "can't get vpu node\n");
+		return NULL;
+	}
+
+	vpu_pdev = of_find_device_by_node(vpu_node);
+	if (WARN_ON(!vpu_pdev)) {
+		dev_err(dev, "vpu pdev failed\n");
+		of_node_put(vpu_node);
+		return NULL;
+	}
+
+	return vpu_pdev;
+}
+
+/* load vpu program/data memory */
+static int load_requested_vpu(struct mtk_vpu *vpu,
+			      const struct firmware *vpu_fw,
+			      u8 fw_type)
+{
+	size_t tcm_size = fw_type ? VPU_DTCM_SIZE : VPU_PTCM_SIZE;
+	size_t fw_size = fw_type ? VPU_D_FW_SIZE : VPU_P_FW_SIZE;
+	char *fw_name = fw_type ? VPU_D_FW : VPU_P_FW;
+	size_t dl_size = 0;
+	size_t extra_fw_size = 0;
+	void *dest;
+	int ret;
+
+	ret = request_firmware(&vpu_fw, fw_name, vpu->dev);
+	if (ret < 0) {
+		dev_err(vpu->dev, "Failed to load %s, %d\n", fw_name, ret);
+		return ret;
+	}
+	dl_size = vpu_fw->size;
+	if (dl_size > fw_size) {
+		dev_err(vpu->dev, "fw %s size %zu is abnormal\n", fw_name,
+			dl_size);
+		release_firmware(vpu_fw);
+		return  -EFBIG;
+	}
+	dev_dbg(vpu->dev, "Downloaded fw %s size: %zu.\n",
+		fw_name,
+		dl_size);
+	/* reset VPU */
+	vpu_cfg_writel(vpu, 0x0, VPU_RESET);
+
+	/* handle extended firmware size */
+	if (dl_size > tcm_size) {
+		dev_dbg(vpu->dev, "fw size %lx > limited fw size %lx\n",
+			dl_size, tcm_size);
+		extra_fw_size = dl_size - tcm_size;
+		dev_dbg(vpu->dev, "extra_fw_size %lx\n", extra_fw_size);
+		dl_size = tcm_size;
+	}
+	dest = vpu->reg.tcm;
+	if (fw_type == D_FW)
+		dest += VPU_DTCM_OFFSET;
+	memcpy(dest, vpu_fw->data, dl_size);
+	/* download to extended memory if need */
+	if (extra_fw_size > 0) {
+		dest = vpu->extmem[fw_type].va;
+		dev_dbg(vpu->dev, "download extended memory type %x\n",
+			fw_type);
+		memcpy(dest, vpu_fw->data + tcm_size, extra_fw_size);
+	}
+
+	release_firmware(vpu_fw);
+
+	return 0;
+}
+
+int vpu_load_firmware(struct platform_device *pdev)
+{
+	struct mtk_vpu *vpu = platform_get_drvdata(pdev);
+	struct device *dev = &pdev->dev;
+	struct vpu_run *run = &vpu->run;
+	const struct firmware *vpu_fw;
+	int ret;
+
+	if (!pdev) {
+		dev_err(dev, "VPU platform device is invalid\n");
+		return -EINVAL;
+	}
+
+	mutex_lock(&vpu->vpu_mutex);
+
+	ret = vpu_clock_enable(vpu);
+	if (ret) {
+		dev_err(dev, "enable clock failed %d\n", ret);
+		goto OUT_LOAD_FW;
+	}
+
+	if (vpu_running(vpu)) {
+		vpu_clock_disable(vpu);
+		mutex_unlock(&vpu->vpu_mutex);
+		dev_warn(dev, "vpu is running already\n");
+		return 0;
+	}
+
+	run->signaled = false;
+	dev_dbg(vpu->dev, "firmware request\n");
+	/* Downloading program firmware to device*/
+	ret = load_requested_vpu(vpu, vpu_fw, P_FW);
+	if (ret < 0) {
+		dev_err(dev, "Failed to request %s, %d\n", VPU_P_FW, ret);
+		goto OUT_LOAD_FW;
+	}
+
+	/* Downloading data firmware to device */
+	ret = load_requested_vpu(vpu, vpu_fw, D_FW);
+	if (ret < 0) {
+		dev_err(dev, "Failed to request %s, %d\n", VPU_D_FW, ret);
+		goto OUT_LOAD_FW;
+	}
+
+	/* boot up vpu */
+	vpu_cfg_writel(vpu, 0x1, VPU_RESET);
+
+	ret = wait_event_interruptible_timeout(run->wq,
+					       run->signaled,
+					       msecs_to_jiffies(INIT_TIMEOUT_MS)
+					       );
+	if (ret == 0) {
+		ret = -ETIME;
+		dev_err(dev, "wait vpu initialization timout!\n");
+		goto OUT_LOAD_FW;
+	} else if (-ERESTARTSYS == ret) {
+		dev_err(dev, "wait vpu interrupted by a signal!\n");
+		goto OUT_LOAD_FW;
+	}
+
+	ret = 0;
+	dev_info(dev, "vpu is ready. Fw version %s\n", run->fw_ver);
+
+OUT_LOAD_FW:
+	vpu_clock_disable(vpu);
+	mutex_unlock(&vpu->vpu_mutex);
+
+	return ret;
+}
+
+int vpu_compare_version(struct platform_device *pdev,
+			const char *expected_version)
+{
+	struct mtk_vpu *vpu = platform_get_drvdata(pdev);
+	int cur_major, cur_minor, cur_build, cur_rel, cur_ver_num;
+	int major, minor, build, rel, ver_num;
+	char *cur_version = vpu->run.fw_ver;
+
+	cur_ver_num = sscanf(cur_version, "%d.%d.%d-rc%d",
+			     &cur_major, &cur_minor, &cur_build, &cur_rel);
+	if (cur_ver_num < 3)
+		return -1;
+	ver_num = sscanf(expected_version, "%d.%d.%d-rc%d",
+			 &major, &minor, &build, &rel);
+	if (ver_num < 3)
+		return -1;
+
+	if (cur_major < major)
+		return -1;
+	if (cur_major > major)
+		return 1;
+
+	if (cur_minor < minor)
+		return -1;
+	if (cur_minor > minor)
+		return 1;
+
+	if (cur_build < build)
+		return -1;
+	if (cur_build > build)
+		return 1;
+
+	if (cur_ver_num < ver_num)
+		return -1;
+	if (cur_ver_num > ver_num)
+		return 1;
+
+	if (ver_num > 3) {
+		if (cur_rel < rel)
+			return -1;
+		if (cur_rel > rel)
+			return 1;
+	}
+
+	return 0;
+}
+
+static void vpu_init_ipi_handler(void *data, unsigned int len, void *priv)
+{
+	struct mtk_vpu *vpu = (struct mtk_vpu *)priv;
+	struct vpu_run *run = (struct vpu_run *)data;
+
+	vpu->run.signaled = run->signaled;
+	strncpy(vpu->run.fw_ver, run->fw_ver, VPU_FW_VER_LEN);
+	vpu->run.enc_capability = run->enc_capability;
+	wake_up_interruptible(&vpu->run.wq);
+}
+
+#ifdef CONFIG_DEBUG_FS
+static int vpu_debug_open(struct inode *inode, struct file *file)
+{
+	file->private_data = inode->i_private;
+	return 0;
+}
+
+static ssize_t vpu_debug_read(struct file *file, char __user *user_buf,
+			      size_t count, loff_t *ppos)
+{
+	char buf[256];
+	unsigned int len;
+	unsigned int running, pc, vpu_to_host, host_to_vpu, wdt;
+	int ret;
+	struct device *dev = file->private_data;
+	struct mtk_vpu *vpu = dev_get_drvdata(dev);
+
+	ret = vpu_clock_enable(vpu);
+	if (ret) {
+		dev_err(vpu->dev, "[VPU] enable clock failed %d\n", ret);
+		return 0;
+	}
+
+	/* vpu register status */
+	running = vpu_running(vpu);
+	pc = vpu_cfg_readl(vpu, VPU_PC_REG);
+	wdt = vpu_cfg_readl(vpu, VPU_WDT_REG);
+	host_to_vpu = vpu_cfg_readl(vpu, HOST_TO_VPU);
+	vpu_to_host = vpu_cfg_readl(vpu, VPU_TO_HOST);
+	vpu_clock_disable(vpu);
+
+	if (running) {
+		len = sprintf(buf, "VPU is running\n\n"
+		"FW Version: %s\n"
+		"PC: 0x%x\n"
+		"WDT: 0x%x\n"
+		"Host to VPU: 0x%x\n"
+		"VPU to Host: 0x%x\n",
+		vpu->run.fw_ver, pc, wdt,
+		host_to_vpu, vpu_to_host);
+	} else {
+		len = sprintf(buf, "VPU not running\n");
+	}
+
+	return simple_read_from_buffer(user_buf, count, ppos, buf, len);
+}
+
+static const struct file_operations vpu_debug_fops = {
+	.open = vpu_debug_open,
+	.read = vpu_debug_read,
+};
+#endif /* CONFIG_DEBUG_FS */
+
+static void vpu_free_ext_mem(struct mtk_vpu *vpu, u8 fw_type)
+{
+	struct device *dev = vpu->dev;
+	size_t fw_ext_size = fw_type ? VPU_EXT_D_SIZE : VPU_EXT_P_SIZE;
+
+	dma_free_coherent(dev, fw_ext_size, vpu->extmem[fw_type].va,
+			  vpu->extmem[fw_type].pa);
+}
+
+static int vpu_alloc_ext_mem(struct mtk_vpu *vpu, u32 fw_type)
+{
+	struct device *dev = vpu->dev;
+	size_t fw_ext_size = fw_type ? VPU_EXT_D_SIZE : VPU_EXT_P_SIZE;
+	u32 vpu_ext_mem0 = fw_type ? VPU_DMEM_EXT0_ADDR : VPU_PMEM_EXT0_ADDR;
+	u32 vpu_ext_mem1 = fw_type ? VPU_DMEM_EXT1_ADDR : VPU_PMEM_EXT1_ADDR;
+	u32 offset_4gb = vpu->enable_4GB ? 0x40000000 : 0;
+
+	vpu->extmem[fw_type].va = dma_alloc_coherent(dev,
+					       fw_ext_size,
+					       &vpu->extmem[fw_type].pa,
+					       GFP_KERNEL);
+	if (!vpu->extmem[fw_type].va) {
+		dev_err(dev, "Failed to allocate the extended program memory\n");
+		return PTR_ERR(vpu->extmem[fw_type].va);
+	}
+
+	/* Disable extend0. Enable extend1 */
+	vpu_cfg_writel(vpu, 0x1, vpu_ext_mem0);
+	vpu_cfg_writel(vpu, (vpu->extmem[fw_type].pa & 0xFFFFF000) + offset_4gb,
+		       vpu_ext_mem1);
+
+	dev_info(dev, "%s extend memory phy=0x%llx virt=0x%p\n",
+		 fw_type ? "Data" : "Program",
+		 (unsigned long long)vpu->extmem[fw_type].pa,
+		 vpu->extmem[fw_type].va);
+
+	return 0;
+}
+
+static void vpu_ipi_handler(struct mtk_vpu *vpu)
+{
+	struct share_obj *rcv_obj = vpu->recv_buf;
+	struct vpu_ipi_desc *ipi_desc = vpu->ipi_desc;
+
+	if (rcv_obj->id < IPI_MAX && ipi_desc[rcv_obj->id].handler) {
+		ipi_desc[rcv_obj->id].handler(rcv_obj->share_buf,
+					      rcv_obj->len,
+					      ipi_desc[rcv_obj->id].priv);
+		if (rcv_obj->id > IPI_VPU_INIT) {
+			vpu->ipi_id_ack[rcv_obj->id] = true;
+			wake_up_interruptible(&vpu->ack_wq);
+		}
+	} else {
+		dev_err(vpu->dev, "No such ipi id = %d\n", rcv_obj->id);
+	}
+}
+
+static int vpu_ipi_init(struct mtk_vpu *vpu)
+{
+	/* Disable VPU to host interrupt */
+	vpu_cfg_writel(vpu, 0x0, VPU_TO_HOST);
+
+	/* shared buffer initialization */
+	vpu->recv_buf = (struct share_obj *)(vpu->reg.tcm + VPU_DTCM_OFFSET);
+	vpu->send_buf = vpu->recv_buf + 1;
+	memset(vpu->recv_buf, 0, sizeof(struct share_obj));
+	memset(vpu->send_buf, 0, sizeof(struct share_obj));
+	mutex_init(&vpu->vpu_mutex);
+
+	return 0;
+}
+
+static irqreturn_t vpu_irq_handler(int irq, void *priv)
+{
+	struct mtk_vpu *vpu = priv;
+	u32 vpu_to_host;
+	int ret;
+
+	/*
+	 * Clock should have been enabled already.
+	 * Enable again in case vpu_ipi_send times out
+	 * and has disabled the clock.
+	 */
+	ret = clk_enable(vpu->clk);
+	if (ret) {
+		dev_err(vpu->dev, "[VPU] enable clock failed %d\n", ret);
+		return IRQ_NONE;
+	}
+	vpu_to_host = vpu_cfg_readl(vpu, VPU_TO_HOST);
+	if (vpu_to_host & VPU_IPC_INT) {
+		vpu_ipi_handler(vpu);
+	} else {
+		dev_err(vpu->dev, "vpu watchdog timeout! 0x%x", vpu_to_host);
+		if (vpu->wdt.wq)
+			queue_work(vpu->wdt.wq, &vpu->wdt.ws);
+	}
+
+	/* VPU won't send another interrupt until we set VPU_TO_HOST to 0. */
+	vpu_cfg_writel(vpu, 0x0, VPU_TO_HOST);
+	clk_disable(vpu->clk);
+
+	return IRQ_HANDLED;
+}
+
+#ifdef CONFIG_DEBUG_FS
+static struct dentry *vpu_debugfs;
+#endif
+static int mtk_vpu_probe(struct platform_device *pdev)
+{
+	struct mtk_vpu *vpu;
+	struct device *dev;
+	struct resource *res;
+	int ret = 0;
+
+	dev_dbg(&pdev->dev, "initialization\n");
+
+	dev = &pdev->dev;
+	vpu = devm_kzalloc(dev, sizeof(*vpu), GFP_KERNEL);
+	if (!vpu)
+		return -ENOMEM;
+
+	vpu->dev = &pdev->dev;
+	res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "tcm");
+	vpu->reg.tcm = devm_ioremap_resource(dev, res);
+	if (IS_ERR(vpu->reg.tcm)) {
+		dev_err(dev, "devm_ioremap_resource vpu tcm failed.\n");
+		return PTR_ERR(vpu->reg.tcm);
+	}
+
+	res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "cfg_reg");
+	vpu->reg.cfg = devm_ioremap_resource(dev, res);
+	if (IS_ERR(vpu->reg.cfg)) {
+		dev_err(dev, "devm_ioremap_resource vpu cfg failed.\n");
+		return PTR_ERR(vpu->reg.cfg);
+	}
+
+	/* Get VPU clock */
+	vpu->clk = devm_clk_get(dev, "main");
+	if (!vpu->clk) {
+		dev_err(dev, "get vpu clock failed\n");
+		return -EINVAL;
+	}
+
+	platform_set_drvdata(pdev, vpu);
+
+	ret = clk_prepare(vpu->clk);
+	if (ret) {
+		dev_err(dev, "prepare vpu clock failed\n");
+		return ret;
+	}
+
+	/* VPU watchdog */
+	vpu->wdt.wq = create_singlethread_workqueue("vpu_wdt");
+	if (!vpu->wdt.wq) {
+		dev_err(dev, "initialize wdt workqueue failed\n");
+		return -ENOMEM;
+	}
+	INIT_WORK(&vpu->wdt.ws, vpu_wdt_reset_func);
+
+	ret = vpu_clock_enable(vpu);
+	if (ret) {
+		dev_err(dev, "enable vpu clock failed\n");
+		goto workqueue_destroy;
+	}
+
+	dev_dbg(dev, "vpu ipi init\n");
+	ret = vpu_ipi_init(vpu);
+	if (ret) {
+		dev_err(dev, "Failed to init ipi\n");
+		goto disable_vpu_clk;
+	}
+
+	/* register vpu initialization IPI */
+	ret = vpu_ipi_register(pdev, IPI_VPU_INIT, vpu_init_ipi_handler,
+			       "vpu_init", vpu);
+	if (ret) {
+		dev_err(dev, "Failed to register IPI_VPU_INIT\n");
+		goto vpu_mutex_destroy;
+	}
+
+#ifdef CONFIG_DEBUG_FS
+	vpu_debugfs = debugfs_create_file("mtk_vpu", S_IRUGO, NULL, (void *)dev,
+					  &vpu_debug_fops);
+	if (!vpu_debugfs) {
+		ret = -ENOMEM;
+		goto cleanup_ipi;
+	}
+#endif
+
+	/* Set PTCM to 96K and DTCM to 32K */
+	vpu_cfg_writel(vpu, 0x2, VPU_TCM_CFG);
+
+	vpu->enable_4GB = !!(max_pfn > (0xffffffffUL >> PAGE_SHIFT));
+	dev_dbg(dev, "4GB mode %u\n", vpu->enable_4GB);
+
+	if (vpu->enable_4GB) {
+		ret = of_reserved_mem_device_init(dev);
+		if (ret)
+			dev_info(dev, "init reserved memory failed\n");
+			/* continue to use dynamic allocation if failed */
+	}
+
+	ret = vpu_alloc_ext_mem(vpu, D_FW);
+	if (ret) {
+		dev_err(dev, "Allocate DM failed\n");
+		goto remove_debugfs;
+	}
+
+	ret = vpu_alloc_ext_mem(vpu, P_FW);
+	if (ret) {
+		dev_err(dev, "Allocate PM failed\n");
+		goto free_d_mem;
+	}
+
+	init_waitqueue_head(&vpu->run.wq);
+	init_waitqueue_head(&vpu->ack_wq);
+
+	res = platform_get_resource(pdev, IORESOURCE_IRQ, 0);
+	if (!res) {
+		dev_err(dev, "get IRQ resource failed.\n");
+		ret = -ENXIO;
+		goto free_p_mem;
+	}
+	vpu->reg.irq = platform_get_irq(pdev, 0);
+	ret = devm_request_irq(dev, vpu->reg.irq, vpu_irq_handler, 0,
+			       pdev->name, vpu);
+	if (ret) {
+		dev_err(dev, "failed to request irq\n");
+		goto free_p_mem;
+	}
+
+	vpu_clock_disable(vpu);
+	dev_dbg(dev, "initialization completed\n");
+
+	return 0;
+
+free_p_mem:
+	vpu_free_ext_mem(vpu, P_FW);
+free_d_mem:
+	vpu_free_ext_mem(vpu, D_FW);
+remove_debugfs:
+	of_reserved_mem_device_release(dev);
+#ifdef CONFIG_DEBUG_FS
+	debugfs_remove(vpu_debugfs);
+cleanup_ipi:
+#endif
+	memset(vpu->ipi_desc, 0, sizeof(struct vpu_ipi_desc) * IPI_MAX);
+vpu_mutex_destroy:
+	mutex_destroy(&vpu->vpu_mutex);
+disable_vpu_clk:
+	vpu_clock_disable(vpu);
+workqueue_destroy:
+	destroy_workqueue(vpu->wdt.wq);
+
+	return ret;
+}
+
+static const struct of_device_id mtk_vpu_match[] = {
+	{
+		.compatible = "mediatek,mt8173-vpu",
+	},
+	{},
+};
+MODULE_DEVICE_TABLE(of, mtk_vpu_match);
+
+static int mtk_vpu_remove(struct platform_device *pdev)
+{
+	struct mtk_vpu *vpu = platform_get_drvdata(pdev);
+
+#ifdef CONFIG_DEBUG_FS
+	debugfs_remove(vpu_debugfs);
+#endif
+	if (vpu->wdt.wq) {
+		flush_workqueue(vpu->wdt.wq);
+		destroy_workqueue(vpu->wdt.wq);
+	}
+	vpu_free_ext_mem(vpu, P_FW);
+	vpu_free_ext_mem(vpu, D_FW);
+	mutex_destroy(&vpu->vpu_mutex);
+	clk_unprepare(vpu->clk);
+
+	return 0;
+}
+
+static struct platform_driver mtk_vpu_driver = {
+	.probe	= mtk_vpu_probe,
+	.remove	= mtk_vpu_remove,
+	.driver	= {
+		.name	= "mtk_vpu",
+		.owner	= THIS_MODULE,
+		.of_match_table = mtk_vpu_match,
+	},
+};
+
+module_platform_driver(mtk_vpu_driver);
+
+MODULE_LICENSE("GPL v2");
+MODULE_DESCRIPTION("Mediatek Video Prosessor Unit driver");
diff --git a/drivers/media/platform/mtk-vpu/mtk_vpu.h b/drivers/media/platform/mtk-vpu/mtk_vpu.h
new file mode 100644
index 0000000..d9c3cde
--- /dev/null
+++ b/drivers/media/platform/mtk-vpu/mtk_vpu.h
@@ -0,0 +1,167 @@
+/*
+* Copyright (c) 2015 MediaTek Inc.
+* Author: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
+*
+* This program is free software; you can redistribute it and/or modify
+* it under the terms of the GNU General Public License version 2 as
+* published by the Free Software Foundation.
+*
+* This program is distributed in the hope that it will be useful,
+* but WITHOUT ANY WARRANTY; without even the implied warranty of
+* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+* GNU General Public License for more details.
+*/
+
+#ifndef _MTK_VPU_H
+#define _MTK_VPU_H
+
+#include <linux/platform_device.h>
+
+/**
+ * VPU (video processor unit) is a tiny processor controlling video hardware
+ * related to video codec, scaling and color format converting.
+ * VPU interfaces with other blocks by share memory and interrupt.
+ **/
+
+typedef void (*ipi_handler_t) (void *data,
+			       unsigned int len,
+			       void *priv);
+
+/**
+ * enum ipi_id - the id of inter-processor interrupt
+ *
+ * @IPI_VPU_INIT:	 The interrupt from vpu is to notfiy kernel
+			 VPU initialization completed.
+			 IPI_VPU_INIT is sent from VPU when firmware is
+			 loaded. AP doesn't need to send IPI_VPU_INIT
+			 command to VPU.
+			 For other IPI below, AP should send the request
+			 to VPU to trigger the interrupt.
+ * @IPI_VENC_H264:	 The interrupt from vpu is to notify kernel to
+			 handle H264 video encoder job, and vice versa.
+ * @IPI_VENC_VP8:	 The interrupt fro vpu is to notify kernel to
+			 handle VP8 video encoder job,, and vice versa.
+ * @IPI_MAX:		 The maximum IPI number
+ */
+
+enum ipi_id {
+	IPI_VPU_INIT = 0,
+	IPI_VENC_H264,
+	IPI_VENC_VP8,
+	IPI_MAX,
+};
+
+/**
+ * enum rst_id - reset id to register reset function for VPU watchdog timeout
+ *
+ * @VPU_RST_DEC: decoder reset id
+ * @VPU_RST_ENC: encoder reset id
+ * @VPU_RST_MDP: MDP (Media Data Path) reset id
+ * @VPU_RST_MAX: maximum reset id
+ */
+enum rst_id {
+	VPU_RST_ENC,
+	VPU_RST_DEC,
+	VPU_RST_MDP,
+	VPU_RST_MAX,
+};
+
+/**
+ * vpu_ipi_register - register an ipi function
+ *
+ * @pdev:	VPU platform device
+ * @id:		IPI ID
+ * @handler:	IPI handler
+ * @name:	IPI name
+ * @priv:	private data for IPI handler
+ *
+ * Register an ipi function to receive ipi interrupt from VPU.
+ *
+ * Return: Return 0 if ipi registers successfully, otherwise it is failed.
+ */
+int vpu_ipi_register(struct platform_device *pdev, enum ipi_id id,
+		     ipi_handler_t handler, const char *name, void *priv);
+
+/**
+ * vpu_ipi_send - send data from AP to vpu.
+ *
+ * @pdev:	VPU platform device
+ * @id:		IPI ID
+ * @buf:	the data buffer
+ * @len:	the data buffer length
+ *
+ * This function is thread-safe. When this function returns,
+ * VPU has received the data and starts the processing.
+ * When the processing completes, IPI handler registered
+ * by vpu_ipi_register will be called in interrupt context.
+ *
+ * Return: Return 0 if sending data successfully, otherwise it is failed.
+ **/
+int vpu_ipi_send(struct platform_device *pdev,
+		 enum ipi_id id, void *buf,
+		 unsigned int len);
+
+/**
+ * vpu_get_plat_device - get VPU's platform device
+ *
+ * @pdev:	the platform device of the module requesting VPU platform
+ *		device for using VPU API.
+ *
+ * Return: Return NULL if it is failed.
+ * otherwise it is VPU's platform device
+ **/
+struct platform_device *vpu_get_plat_device(struct platform_device *pdev);
+
+/**
+ * vpu_wdt_reg_handler - register a VPU watchdog handler
+ *
+ * @pdev:               VPU platform device
+ * @vpu_wdt_reset_func:	the callback reset function
+ * @private_data:       the private data for reset function
+ * @rst_id:		reset id
+ *
+ * Register a handler performing own tasks when vpu reset by watchdog
+ *
+ * Return: Return 0 if the handler is added successfully,
+ * otherwise it is failed.
+ *
+ **/
+int vpu_wdt_reg_handler(struct platform_device *pdev,
+			void vpu_wdt_reset_func(void *),
+			void *priv, enum rst_id id);
+
+/**
+ * vpu_get_venc_hw_capa - get video encoder hardware capability
+ *
+ * @pdev:	VPU platform device
+ *
+ * Return: video encoder hardware capability
+ **/
+unsigned int vpu_get_venc_hw_capa(struct platform_device *pdev);
+
+/**
+ * vpu_load_firmware - download VPU firmware and boot it
+ *
+ * @pdev:	VPU platform device
+ *
+ * Return: Return 0 if downloading firmware successfully,
+ * otherwise it is failed
+ **/
+int vpu_load_firmware(struct platform_device *pdev);
+
+/**
+ * vpu_mapping_dm_addr - Mapping DTCM/DMEM to kernel virtual address
+ *
+ * @pdev:	VPU platform device
+ * @dmem_addr:	VPU's data memory address
+ *
+ * Mapping the VPU's DTCM (Data Tightly-Coupled Memory) /
+ * DMEM (Data Extended Memory) memory address to
+ * kernel virtual address.
+ *
+ * Return: Return ERR_PTR(-EINVAL) if mapping failed,
+ * otherwise the mapped kernel virtual address
+ **/
+void *vpu_mapping_dm_addr(struct platform_device *pdev,
+			  u32 dtcm_dmem_addr);
+#endif /* _MTK_VPU_H */
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 102+ messages in thread

* [PATCH v4 3/8] arm64: dts: mediatek: Add node for Mediatek Video Processor Unit
  2016-02-04 11:34     ` Tiffany Lin
  (?)
@ 2016-02-04 11:34       ` Tiffany Lin
  -1 siblings, 0 replies; 102+ messages in thread
From: Tiffany Lin @ 2016-02-04 11:34 UTC (permalink / raw)
  To: Hans Verkuil, daniel.thompson, Rob Herring,
	Mauro Carvalho Chehab, Matthias Brugger, Daniel Kurtz,
	Pawel Osciak
  Cc: Eddie Huang, Yingjoe Chen, devicetree, linux-kernel,
	linux-arm-kernel, linux-media, linux-mediatek, PoChun.Lin,
	Tiffany Lin, Andrew-CT Chen

Add VPU drivers for MT8173

Signed-off-by: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
---
 arch/arm64/boot/dts/mediatek/mt8173.dtsi |   23 +++++++++++++++++++++++
 1 file changed, 23 insertions(+)

diff --git a/arch/arm64/boot/dts/mediatek/mt8173.dtsi b/arch/arm64/boot/dts/mediatek/mt8173.dtsi
index 60a1284..5b0b38a 100644
--- a/arch/arm64/boot/dts/mediatek/mt8173.dtsi
+++ b/arch/arm64/boot/dts/mediatek/mt8173.dtsi
@@ -200,6 +200,18 @@
 		clock-output-names = "cpum_ck";
 	};
 
+	reserved-memory {
+		#address-cells = <2>;
+		#size-cells = <2>;
+		ranges;
+		vpu_dma_reserved: vpu_dma_mem_region {
+			compatible = "shared-dma-pool";
+			reg = <0 0xb7000000 0 0x500000>;
+			alignment = <0x1000>;
+			no-map;
+		};
+	};
+
 	thermal-zones {
 		cpu_thermal: cpu_thermal {
 			polling-delay-passive = <1000>; /* milliseconds */
@@ -422,6 +434,17 @@
 			clocks = <&infracfg CLK_INFRA_CEC>;
 		};
 
+		vpu: vpu@10020000 {
+			compatible = "mediatek,mt8173-vpu";
+			reg = <0 0x10020000 0 0x30000>,
+			      <0 0x10050000 0 0x100>;
+			reg-names = "tcm", "cfg_reg";
+			interrupts = <GIC_SPI 166 IRQ_TYPE_LEVEL_HIGH>;
+			clocks = <&topckgen CLK_TOP_SCP_SEL>;
+			clock-names = "main";
+			memory-region = <&vpu_dma_reserved>;
+		};
+
 		sysirq: intpol-controller@10200620 {
 			compatible = "mediatek,mt8173-sysirq",
 				     "mediatek,mt6577-sysirq";
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 102+ messages in thread

* [PATCH v4 3/8] arm64: dts: mediatek: Add node for Mediatek Video Processor Unit
@ 2016-02-04 11:34       ` Tiffany Lin
  0 siblings, 0 replies; 102+ messages in thread
From: Tiffany Lin @ 2016-02-04 11:34 UTC (permalink / raw)
  To: Hans Verkuil, daniel.thompson, Rob Herring,
	Mauro Carvalho Chehab, Matthias Brugger, Daniel Kurtz,
	Pawel Osciak
  Cc: Eddie Huang, Yingjoe Chen, devicetree, linux-kernel,
	linux-arm-kernel, linux-media, linux-mediatek, PoChun.Lin,
	Tiffany Lin, Andrew-CT Chen

Add VPU drivers for MT8173

Signed-off-by: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
---
 arch/arm64/boot/dts/mediatek/mt8173.dtsi |   23 +++++++++++++++++++++++
 1 file changed, 23 insertions(+)

diff --git a/arch/arm64/boot/dts/mediatek/mt8173.dtsi b/arch/arm64/boot/dts/mediatek/mt8173.dtsi
index 60a1284..5b0b38a 100644
--- a/arch/arm64/boot/dts/mediatek/mt8173.dtsi
+++ b/arch/arm64/boot/dts/mediatek/mt8173.dtsi
@@ -200,6 +200,18 @@
 		clock-output-names = "cpum_ck";
 	};
 
+	reserved-memory {
+		#address-cells = <2>;
+		#size-cells = <2>;
+		ranges;
+		vpu_dma_reserved: vpu_dma_mem_region {
+			compatible = "shared-dma-pool";
+			reg = <0 0xb7000000 0 0x500000>;
+			alignment = <0x1000>;
+			no-map;
+		};
+	};
+
 	thermal-zones {
 		cpu_thermal: cpu_thermal {
 			polling-delay-passive = <1000>; /* milliseconds */
@@ -422,6 +434,17 @@
 			clocks = <&infracfg CLK_INFRA_CEC>;
 		};
 
+		vpu: vpu@10020000 {
+			compatible = "mediatek,mt8173-vpu";
+			reg = <0 0x10020000 0 0x30000>,
+			      <0 0x10050000 0 0x100>;
+			reg-names = "tcm", "cfg_reg";
+			interrupts = <GIC_SPI 166 IRQ_TYPE_LEVEL_HIGH>;
+			clocks = <&topckgen CLK_TOP_SCP_SEL>;
+			clock-names = "main";
+			memory-region = <&vpu_dma_reserved>;
+		};
+
 		sysirq: intpol-controller@10200620 {
 			compatible = "mediatek,mt8173-sysirq",
 				     "mediatek,mt6577-sysirq";
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 102+ messages in thread

* [PATCH v4 3/8] arm64: dts: mediatek: Add node for Mediatek Video Processor Unit
@ 2016-02-04 11:34       ` Tiffany Lin
  0 siblings, 0 replies; 102+ messages in thread
From: Tiffany Lin @ 2016-02-04 11:34 UTC (permalink / raw)
  To: linux-arm-kernel

Add VPU drivers for MT8173

Signed-off-by: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
---
 arch/arm64/boot/dts/mediatek/mt8173.dtsi |   23 +++++++++++++++++++++++
 1 file changed, 23 insertions(+)

diff --git a/arch/arm64/boot/dts/mediatek/mt8173.dtsi b/arch/arm64/boot/dts/mediatek/mt8173.dtsi
index 60a1284..5b0b38a 100644
--- a/arch/arm64/boot/dts/mediatek/mt8173.dtsi
+++ b/arch/arm64/boot/dts/mediatek/mt8173.dtsi
@@ -200,6 +200,18 @@
 		clock-output-names = "cpum_ck";
 	};
 
+	reserved-memory {
+		#address-cells = <2>;
+		#size-cells = <2>;
+		ranges;
+		vpu_dma_reserved: vpu_dma_mem_region {
+			compatible = "shared-dma-pool";
+			reg = <0 0xb7000000 0 0x500000>;
+			alignment = <0x1000>;
+			no-map;
+		};
+	};
+
 	thermal-zones {
 		cpu_thermal: cpu_thermal {
 			polling-delay-passive = <1000>; /* milliseconds */
@@ -422,6 +434,17 @@
 			clocks = <&infracfg CLK_INFRA_CEC>;
 		};
 
+		vpu: vpu at 10020000 {
+			compatible = "mediatek,mt8173-vpu";
+			reg = <0 0x10020000 0 0x30000>,
+			      <0 0x10050000 0 0x100>;
+			reg-names = "tcm", "cfg_reg";
+			interrupts = <GIC_SPI 166 IRQ_TYPE_LEVEL_HIGH>;
+			clocks = <&topckgen CLK_TOP_SCP_SEL>;
+			clock-names = "main";
+			memory-region = <&vpu_dma_reserved>;
+		};
+
 		sysirq: intpol-controller at 10200620 {
 			compatible = "mediatek,mt8173-sysirq",
 				     "mediatek,mt6577-sysirq";
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 102+ messages in thread

* [PATCH v4 4/8] dt-bindings: Add a binding for Mediatek Video Encoder
  2016-02-04 11:34       ` Tiffany Lin
  (?)
@ 2016-02-04 11:34         ` Tiffany Lin
  -1 siblings, 0 replies; 102+ messages in thread
From: Tiffany Lin @ 2016-02-04 11:34 UTC (permalink / raw)
  To: Hans Verkuil, daniel.thompson, Rob Herring,
	Mauro Carvalho Chehab, Matthias Brugger, Daniel Kurtz,
	Pawel Osciak
  Cc: Eddie Huang, Yingjoe Chen, devicetree, linux-kernel,
	linux-arm-kernel, linux-media, linux-mediatek, PoChun.Lin,
	Tiffany Lin

Add a DT binding documentation of Video Encoder for the
MT8173 SoC from Mediatek.

Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
---
 .../devicetree/bindings/media/mediatek-vcodec.txt  |   59 ++++++++++++++++++++
 1 file changed, 59 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/media/mediatek-vcodec.txt

diff --git a/Documentation/devicetree/bindings/media/mediatek-vcodec.txt b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
new file mode 100644
index 0000000..572bfdd
--- /dev/null
+++ b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
@@ -0,0 +1,59 @@
+Mediatek Video Codec
+
+Mediatek Video Codec is the video codec hw present in Mediatek SoCs which
+supports high resolution encoding functionalities.
+
+Required properties:
+- compatible : "mediatek,mt8173-vcodec-enc" for encoder
+- reg : Physical base address of the video codec registers and length of
+  memory mapped region.
+- interrupts : interrupt number to the cpu.
+- mediatek,larb : must contain the local arbiters in the current Socs.
+- clocks : list of clock specifiers, corresponding to entries in
+  the clock-names property.
+- clock-names: encoder must contain "vencpll_d2", "venc_sel", "univpll1_d2",
+  "venc_lt_sel".
+- iommus : should point to the respective IOMMU block with master port as
+  argument, see Documentation/devicetree/bindings/iommu/mediatek,iommu.txt
+  for details.
+- mediatek,vpu : the node of video processor unit
+
+Example:
+vcodec_enc: vcodec@0x18002000 {
+    compatible = "mediatek,mt8173-vcodec-enc";
+    reg = <0 0x18002000 0 0x1000>,    /*VENC_SYS*/
+          <0 0x19002000 0 0x1000>;    /*VENC_LT_SYS*/
+    interrupts = <GIC_SPI 198 IRQ_TYPE_LEVEL_LOW>,
+           <GIC_SPI 202 IRQ_TYPE_LEVEL_LOW>;
+    mediatek,larb = <&larb3>,
+		    <&larb5>;
+    iommus = <&iommu M4U_PORT_VENC_RCPU>,
+             <&iommu M4U_PORT_VENC_REC>,
+             <&iommu M4U_PORT_VENC_BSDMA>,
+             <&iommu M4U_PORT_VENC_SV_COMV>,
+             <&iommu M4U_PORT_VENC_RD_COMV>,
+             <&iommu M4U_PORT_VENC_CUR_LUMA>,
+             <&iommu M4U_PORT_VENC_CUR_CHROMA>,
+             <&iommu M4U_PORT_VENC_REF_LUMA>,
+             <&iommu M4U_PORT_VENC_REF_CHROMA>,
+             <&iommu M4U_PORT_VENC_NBM_RDMA>,
+             <&iommu M4U_PORT_VENC_NBM_WDMA>,
+             <&iommu M4U_PORT_VENC_RCPU_SET2>,
+             <&iommu M4U_PORT_VENC_REC_FRM_SET2>,
+             <&iommu M4U_PORT_VENC_BSDMA_SET2>,
+             <&iommu M4U_PORT_VENC_SV_COMA_SET2>,
+             <&iommu M4U_PORT_VENC_RD_COMA_SET2>,
+             <&iommu M4U_PORT_VENC_CUR_LUMA_SET2>,
+             <&iommu M4U_PORT_VENC_CUR_CHROMA_SET2>,
+             <&iommu M4U_PORT_VENC_REF_LUMA_SET2>,
+             <&iommu M4U_PORT_VENC_REC_CHROMA_SET2>;
+    mediatek,vpu = <&vpu>;
+    clocks = <&topckgen CLK_TOP_VENCPLL_D2>,
+             <&topckgen CLK_TOP_VENC_SEL>,
+             <&topckgen CLK_TOP_UNIVPLL1_D2>,
+             <&topckgen CLK_TOP_VENC_LT_SEL>;
+    clock-names = "vencpll_d2",
+                  "venc_sel",
+                  "univpll1_d2",
+                  "venc_lt_sel";
+  };
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 102+ messages in thread

* [PATCH v4 4/8] dt-bindings: Add a binding for Mediatek Video Encoder
@ 2016-02-04 11:34         ` Tiffany Lin
  0 siblings, 0 replies; 102+ messages in thread
From: Tiffany Lin @ 2016-02-04 11:34 UTC (permalink / raw)
  To: Hans Verkuil, daniel.thompson, Rob Herring,
	Mauro Carvalho Chehab, Matthias Brugger, Daniel Kurtz,
	Pawel Osciak
  Cc: devicetree, Tiffany Lin, linux-kernel, linux-mediatek,
	PoChun.Lin, Yingjoe Chen, Eddie Huang, linux-arm-kernel,
	linux-media

Add a DT binding documentation of Video Encoder for the
MT8173 SoC from Mediatek.

Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
---
 .../devicetree/bindings/media/mediatek-vcodec.txt  |   59 ++++++++++++++++++++
 1 file changed, 59 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/media/mediatek-vcodec.txt

diff --git a/Documentation/devicetree/bindings/media/mediatek-vcodec.txt b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
new file mode 100644
index 0000000..572bfdd
--- /dev/null
+++ b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
@@ -0,0 +1,59 @@
+Mediatek Video Codec
+
+Mediatek Video Codec is the video codec hw present in Mediatek SoCs which
+supports high resolution encoding functionalities.
+
+Required properties:
+- compatible : "mediatek,mt8173-vcodec-enc" for encoder
+- reg : Physical base address of the video codec registers and length of
+  memory mapped region.
+- interrupts : interrupt number to the cpu.
+- mediatek,larb : must contain the local arbiters in the current Socs.
+- clocks : list of clock specifiers, corresponding to entries in
+  the clock-names property.
+- clock-names: encoder must contain "vencpll_d2", "venc_sel", "univpll1_d2",
+  "venc_lt_sel".
+- iommus : should point to the respective IOMMU block with master port as
+  argument, see Documentation/devicetree/bindings/iommu/mediatek,iommu.txt
+  for details.
+- mediatek,vpu : the node of video processor unit
+
+Example:
+vcodec_enc: vcodec@0x18002000 {
+    compatible = "mediatek,mt8173-vcodec-enc";
+    reg = <0 0x18002000 0 0x1000>,    /*VENC_SYS*/
+          <0 0x19002000 0 0x1000>;    /*VENC_LT_SYS*/
+    interrupts = <GIC_SPI 198 IRQ_TYPE_LEVEL_LOW>,
+           <GIC_SPI 202 IRQ_TYPE_LEVEL_LOW>;
+    mediatek,larb = <&larb3>,
+		    <&larb5>;
+    iommus = <&iommu M4U_PORT_VENC_RCPU>,
+             <&iommu M4U_PORT_VENC_REC>,
+             <&iommu M4U_PORT_VENC_BSDMA>,
+             <&iommu M4U_PORT_VENC_SV_COMV>,
+             <&iommu M4U_PORT_VENC_RD_COMV>,
+             <&iommu M4U_PORT_VENC_CUR_LUMA>,
+             <&iommu M4U_PORT_VENC_CUR_CHROMA>,
+             <&iommu M4U_PORT_VENC_REF_LUMA>,
+             <&iommu M4U_PORT_VENC_REF_CHROMA>,
+             <&iommu M4U_PORT_VENC_NBM_RDMA>,
+             <&iommu M4U_PORT_VENC_NBM_WDMA>,
+             <&iommu M4U_PORT_VENC_RCPU_SET2>,
+             <&iommu M4U_PORT_VENC_REC_FRM_SET2>,
+             <&iommu M4U_PORT_VENC_BSDMA_SET2>,
+             <&iommu M4U_PORT_VENC_SV_COMA_SET2>,
+             <&iommu M4U_PORT_VENC_RD_COMA_SET2>,
+             <&iommu M4U_PORT_VENC_CUR_LUMA_SET2>,
+             <&iommu M4U_PORT_VENC_CUR_CHROMA_SET2>,
+             <&iommu M4U_PORT_VENC_REF_LUMA_SET2>,
+             <&iommu M4U_PORT_VENC_REC_CHROMA_SET2>;
+    mediatek,vpu = <&vpu>;
+    clocks = <&topckgen CLK_TOP_VENCPLL_D2>,
+             <&topckgen CLK_TOP_VENC_SEL>,
+             <&topckgen CLK_TOP_UNIVPLL1_D2>,
+             <&topckgen CLK_TOP_VENC_LT_SEL>;
+    clock-names = "vencpll_d2",
+                  "venc_sel",
+                  "univpll1_d2",
+                  "venc_lt_sel";
+  };
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 102+ messages in thread

* [PATCH v4 4/8] dt-bindings: Add a binding for Mediatek Video Encoder
@ 2016-02-04 11:34         ` Tiffany Lin
  0 siblings, 0 replies; 102+ messages in thread
From: Tiffany Lin @ 2016-02-04 11:34 UTC (permalink / raw)
  To: linux-arm-kernel

Add a DT binding documentation of Video Encoder for the
MT8173 SoC from Mediatek.

Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
---
 .../devicetree/bindings/media/mediatek-vcodec.txt  |   59 ++++++++++++++++++++
 1 file changed, 59 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/media/mediatek-vcodec.txt

diff --git a/Documentation/devicetree/bindings/media/mediatek-vcodec.txt b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
new file mode 100644
index 0000000..572bfdd
--- /dev/null
+++ b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
@@ -0,0 +1,59 @@
+Mediatek Video Codec
+
+Mediatek Video Codec is the video codec hw present in Mediatek SoCs which
+supports high resolution encoding functionalities.
+
+Required properties:
+- compatible : "mediatek,mt8173-vcodec-enc" for encoder
+- reg : Physical base address of the video codec registers and length of
+  memory mapped region.
+- interrupts : interrupt number to the cpu.
+- mediatek,larb : must contain the local arbiters in the current Socs.
+- clocks : list of clock specifiers, corresponding to entries in
+  the clock-names property.
+- clock-names: encoder must contain "vencpll_d2", "venc_sel", "univpll1_d2",
+  "venc_lt_sel".
+- iommus : should point to the respective IOMMU block with master port as
+  argument, see Documentation/devicetree/bindings/iommu/mediatek,iommu.txt
+  for details.
+- mediatek,vpu : the node of video processor unit
+
+Example:
+vcodec_enc: vcodec at 0x18002000 {
+    compatible = "mediatek,mt8173-vcodec-enc";
+    reg = <0 0x18002000 0 0x1000>,    /*VENC_SYS*/
+          <0 0x19002000 0 0x1000>;    /*VENC_LT_SYS*/
+    interrupts = <GIC_SPI 198 IRQ_TYPE_LEVEL_LOW>,
+           <GIC_SPI 202 IRQ_TYPE_LEVEL_LOW>;
+    mediatek,larb = <&larb3>,
+		    <&larb5>;
+    iommus = <&iommu M4U_PORT_VENC_RCPU>,
+             <&iommu M4U_PORT_VENC_REC>,
+             <&iommu M4U_PORT_VENC_BSDMA>,
+             <&iommu M4U_PORT_VENC_SV_COMV>,
+             <&iommu M4U_PORT_VENC_RD_COMV>,
+             <&iommu M4U_PORT_VENC_CUR_LUMA>,
+             <&iommu M4U_PORT_VENC_CUR_CHROMA>,
+             <&iommu M4U_PORT_VENC_REF_LUMA>,
+             <&iommu M4U_PORT_VENC_REF_CHROMA>,
+             <&iommu M4U_PORT_VENC_NBM_RDMA>,
+             <&iommu M4U_PORT_VENC_NBM_WDMA>,
+             <&iommu M4U_PORT_VENC_RCPU_SET2>,
+             <&iommu M4U_PORT_VENC_REC_FRM_SET2>,
+             <&iommu M4U_PORT_VENC_BSDMA_SET2>,
+             <&iommu M4U_PORT_VENC_SV_COMA_SET2>,
+             <&iommu M4U_PORT_VENC_RD_COMA_SET2>,
+             <&iommu M4U_PORT_VENC_CUR_LUMA_SET2>,
+             <&iommu M4U_PORT_VENC_CUR_CHROMA_SET2>,
+             <&iommu M4U_PORT_VENC_REF_LUMA_SET2>,
+             <&iommu M4U_PORT_VENC_REC_CHROMA_SET2>;
+    mediatek,vpu = <&vpu>;
+    clocks = <&topckgen CLK_TOP_VENCPLL_D2>,
+             <&topckgen CLK_TOP_VENC_SEL>,
+             <&topckgen CLK_TOP_UNIVPLL1_D2>,
+             <&topckgen CLK_TOP_VENC_LT_SEL>;
+    clock-names = "vencpll_d2",
+                  "venc_sel",
+                  "univpll1_d2",
+                  "venc_lt_sel";
+  };
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 102+ messages in thread

* [PATCH v4 5/8] [Media] vcodec: mediatek: Add Mediatek V4L2 Video Encoder Driver
@ 2016-02-04 11:35           ` Tiffany Lin
  0 siblings, 0 replies; 102+ messages in thread
From: Tiffany Lin @ 2016-02-04 11:35 UTC (permalink / raw)
  To: Hans Verkuil, daniel.thompson, Rob Herring,
	Mauro Carvalho Chehab, Matthias Brugger, Daniel Kurtz,
	Pawel Osciak
  Cc: Eddie Huang, Yingjoe Chen, devicetree, linux-kernel,
	linux-arm-kernel, linux-media, linux-mediatek, PoChun.Lin,
	Andrew-CT Chen, Tiffany Lin

From: Andrew-CT Chen <andrew-ct.chen@mediatek.com>

Add v4l2 layer encoder driver for MT8173

Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
---
 drivers/media/platform/Kconfig                     |   11 +
 drivers/media/platform/Makefile                    |    2 +
 drivers/media/platform/mtk-vcodec/Makefile         |    8 +
 drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h |  388 ++++++
 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c | 1380 ++++++++++++++++++++
 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h |   46 +
 .../media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c |  476 +++++++
 .../media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c  |  132 ++
 .../media/platform/mtk-vcodec/mtk_vcodec_intr.c    |  102 ++
 .../media/platform/mtk-vcodec/mtk_vcodec_intr.h    |   29 +
 drivers/media/platform/mtk-vcodec/mtk_vcodec_pm.h  |   26 +
 .../media/platform/mtk-vcodec/mtk_vcodec_util.c    |  106 ++
 .../media/platform/mtk-vcodec/mtk_vcodec_util.h    |   85 ++
 drivers/media/platform/mtk-vcodec/venc_drv_base.h  |   62 +
 drivers/media/platform/mtk-vcodec/venc_drv_if.c    |  100 ++
 drivers/media/platform/mtk-vcodec/venc_drv_if.h    |  175 +++
 drivers/media/platform/mtk-vcodec/venc_ipi_msg.h   |  212 +++
 include/uapi/linux/v4l2-controls.h                 |    4 +
 18 files changed, 3344 insertions(+)
 create mode 100644 drivers/media/platform/mtk-vcodec/Makefile
 create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
 create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
 create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h
 create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
 create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
 create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.c
 create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.h
 create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_pm.h
 create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c
 create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
 create mode 100644 drivers/media/platform/mtk-vcodec/venc_drv_base.h
 create mode 100644 drivers/media/platform/mtk-vcodec/venc_drv_if.c
 create mode 100644 drivers/media/platform/mtk-vcodec/venc_drv_if.h
 create mode 100644 drivers/media/platform/mtk-vcodec/venc_ipi_msg.h
 mode change 100644 => 100755 include/uapi/linux/v4l2-controls.h

diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
index ba812d6..3e831c5 100644
--- a/drivers/media/platform/Kconfig
+++ b/drivers/media/platform/Kconfig
@@ -157,6 +157,17 @@ config VIDEO_MEDIATEK_VPU
 	    codec embedded in new Mediatek's SOCs. It is able
 	    to handle video decoding/encoding in a range of formats.
 
+config VIDEO_MEDIATEK_VCODEC
+        tristate "Mediatek Video Codec driver"
+        depends on VIDEO_DEV && VIDEO_V4L2
+        depends on ARCH_MEDIATEK || COMPILE_TEST
+        select VIDEOBUF2_DMA_CONTIG
+        select V4L2_MEM2MEM_DEV
+        select MEDIATEK_VPU
+        default n
+        ---help---
+            Mediatek video codec driver for V4L2
+
 config VIDEO_MEM2MEM_DEINTERLACE
 	tristate "Deinterlace support"
 	depends on VIDEO_DEV && VIDEO_V4L2 && DMA_ENGINE
diff --git a/drivers/media/platform/Makefile b/drivers/media/platform/Makefile
index e5b19c6..510e06b 100644
--- a/drivers/media/platform/Makefile
+++ b/drivers/media/platform/Makefile
@@ -57,3 +57,5 @@ obj-$(CONFIG_VIDEO_XILINX)		+= xilinx/
 ccflags-y += -I$(srctree)/drivers/media/i2c
 
 obj-$(CONFIG_VIDEO_MEDIATEK_VPU)	+= mtk-vpu/
+
+obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC)	+= mtk-vcodec/
diff --git a/drivers/media/platform/mtk-vcodec/Makefile b/drivers/media/platform/mtk-vcodec/Makefile
new file mode 100644
index 0000000..ce38689
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/Makefile
@@ -0,0 +1,8 @@
+obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += mtk_vcodec_intr.o \
+				       mtk_vcodec_util.o \
+				       mtk_vcodec_enc_drv.o \
+				       mtk_vcodec_enc.o \
+				       mtk_vcodec_enc_pm.o \
+				       venc_drv_if.o
+
+ccflags-y += -I$(srctree)/drivers/media/platform/mtk-vpu
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
new file mode 100644
index 0000000..9da2818
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
@@ -0,0 +1,388 @@
+/*
+* Copyright (c) 2015 MediaTek Inc.
+* Author: PC Chen <pc.chen@mediatek.com>
+*         Tiffany Lin <tiffany.lin@mediatek.com>
+*
+* This program is free software; you can redistribute it and/or modify
+* it under the terms of the GNU General Public License version 2 as
+* published by the Free Software Foundation.
+*
+* This program is distributed in the hope that it will be useful,
+* but WITHOUT ANY WARRANTY; without even the implied warranty of
+* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+* GNU General Public License for more details.
+*/
+
+#ifndef _MTK_VCODEC_DRV_H_
+#define _MTK_VCODEC_DRV_H_
+
+#include <linux/platform_device.h>
+#include <linux/videodev2.h>
+#include <media/v4l2-ctrls.h>
+#include <media/v4l2-device.h>
+#include <media/v4l2-ioctl.h>
+#include <media/videobuf2-core.h>
+
+#include "mtk_vcodec_util.h"
+
+#define MTK_VCODEC_MAX_INSTANCES		10
+#define MTK_VCODEC_MAX_ENCODER_INSTANCES	3
+
+#define MTK_VCODEC_MAX_FRAME_SIZE	0x800000
+#define MTK_VIDEO_MAX_FRAME		32
+#define MTK_MAX_CTRLS			20
+
+#define MTK_VCODEC_DRV_NAME		"mtk_vcodec_drv"
+#define MTK_VCODEC_ENC_NAME		"mtk-vcodec-enc"
+#define MTK_PLATFORM_STR		"platform:mt8173"
+
+#define MTK_VENC_IRQ_STATUS_SPS          0x1
+#define MTK_VENC_IRQ_STATUS_PPS          0x2
+#define MTK_VENC_IRQ_STATUS_FRM          0x4
+#define MTK_VENC_IRQ_STATUS_DRAM         0x8
+#define MTK_VENC_IRQ_STATUS_PAUSE        0x10
+#define MTK_VENC_IRQ_STATUS_SWITCH       0x20
+
+#define MTK_VENC_IRQ_STATUS_OFFSET       0x05C
+#define MTK_VENC_IRQ_ACK_OFFSET          0x060
+
+#define MTK_VCODEC_MAX_PLANES		3
+
+#define VDEC_HW_ACTIVE	0x10
+#define VDEC_IRQ_CFG    0x11
+#define VDEC_IRQ_CLR    0x10
+
+#define VDEC_IRQ_CFG_REG	0xa4
+#define WAIT_INTR_TIMEOUT	1000
+
+/**
+ * enum mtk_hw_reg_idx - MTK hw register base index
+ */
+enum mtk_hw_reg_idx {
+	VDEC_SYS,
+	VDEC_MISC,
+	VDEC_LD,
+	VDEC_TOP,
+	VDEC_CM,
+	VDEC_AD,
+	VDEC_AV,
+	VDEC_PP,
+	VDEC_HWD,
+	VDEC_HWQ,
+	VDEC_HWB,
+	VDEC_HWG,
+	NUM_MAX_VDEC_REG_BASE,
+	VENC_SYS = NUM_MAX_VDEC_REG_BASE,
+	VENC_LT_SYS,
+	NUM_MAX_VCODEC_REG_BASE
+};
+
+/**
+ * enum mtk_instance_type - The type of an MTK Vcodec instance.
+ */
+enum mtk_instance_type {
+	MTK_INST_DECODER		= 0,
+	MTK_INST_ENCODER		= 1,
+};
+
+/**
+ * enum mtk_instance_state - The state of an MTK Vcodec instance.
+ * @MTK_STATE_FREE - default state when instance is created
+ * @MTK_STATE_INIT - vcodec instance is initialized
+ * @MTK_STATE_HEADER - vdec had sps/pps header parsed or venc
+ *			had sps/pps header encoded
+ * @MTK_STATE_FLUSH - vdec is flushing. Only used by decoder
+ * @MTK_STATE_RES_CHANGE - vdec detect resolution change.
+ * 			Only used by decoder
+ * @MTK_STATE_ABORT - vcodec should be aborted
+ */
+enum mtk_instance_state {
+	MTK_STATE_FREE = 0,
+	MTK_STATE_INIT = 1,
+	MTK_STATE_HEADER = 2,
+	MTK_STATE_FLUSH = 3,
+	MTK_STATE_RES_CHANGE = 4,
+	MTK_STATE_ABORT = 5,
+};
+
+/**
+ * struct mtk_param_change - General encoding parameters type
+ */
+enum mtk_encode_param {
+	MTK_ENCODE_PARAM_NONE = 0,
+	MTK_ENCODE_PARAM_BITRATE = (1 << 0),
+	MTK_ENCODE_PARAM_FRAMERATE = (1 << 1),
+	MTK_ENCODE_PARAM_INTRA_PERIOD = (1 << 2),
+	MTK_ENCODE_PARAM_FRAME_TYPE = (1 << 3),
+};
+
+/**
+ * enum mtk_fmt_type - Type of the pixelformat
+ * @MTK_FMT_FRAME - mtk vcodec raw frame
+ */
+enum mtk_fmt_type {
+	MTK_FMT_DEC		= 0,
+	MTK_FMT_ENC		= 1,
+	MTK_FMT_FRAME		= 2,
+};
+
+/**
+ * struct mtk_video_fmt - Structure used to store information about pixelformats
+ */
+struct mtk_video_fmt {
+	u32 fourcc;
+	enum mtk_fmt_type type;
+	u32 num_planes;
+};
+
+/**
+ * struct mtk_codec_framesizes - Structure used to store information about framesizes
+ */
+struct mtk_codec_framesizes {
+	u32 fourcc;
+	struct	v4l2_frmsize_stepwise	stepwise;
+};
+
+/**
+ * struct mtk_q_type - Type of queue
+ */
+enum mtk_q_type {
+	MTK_Q_DATA_SRC		= 0,
+	MTK_Q_DATA_DST		= 1,
+};
+
+/**
+ * struct mtk_q_data - Structure used to store information about queue
+ * @colorspace	reserved for encoder
+ * @field		reserved for encoder
+ */
+struct mtk_q_data {
+	unsigned int		width;
+	unsigned int		height;
+	enum v4l2_field		field;
+	enum v4l2_colorspace	colorspace;
+	unsigned int		bytesperline[MTK_VCODEC_MAX_PLANES];
+	unsigned int		sizeimage[MTK_VCODEC_MAX_PLANES];
+	struct mtk_video_fmt	*fmt;
+};
+
+/**
+ * struct mtk_enc_params - General encoding parameters
+ * @bitrate - target bitrate
+ * @num_b_frame - number of b frames between p-frame
+ * @rc_frame - frame based rate control
+ * @rc_mb - macroblock based rate control
+ * @seq_hdr_mode - H.264 sequence header is encoded separately or joined with the first frame
+ * @gop_size - group of picture size, it's used as the intra frame period
+ * @framerate_num - frame rate numerator
+ * @framerate_denom - frame rate denominator
+ * @h264_max_qp - Max value for H.264 quantization parameter
+ * @h264_profile - V4L2 defined H.264 profile
+ * @h264_level - V4L2 defined H.264 level
+ * @force_intra - force/insert intra frame
+ * @skip_frame - encode in skip frame mode that use minimum number of bits
+ */
+struct mtk_enc_params {
+	unsigned int	bitrate;
+	unsigned int	num_b_frame;
+	unsigned int	rc_frame;
+	unsigned int	rc_mb;
+	unsigned int	seq_hdr_mode;
+	unsigned int	gop_size;
+	unsigned int	framerate_num;
+	unsigned int	framerate_denom;
+	unsigned int	h264_max_qp;
+	unsigned int	h264_profile;
+	unsigned int	h264_level;
+	unsigned int	force_intra;
+};
+
+/**
+ * struct mtk_vcodec_pm - Power management data structure
+ */
+struct mtk_vcodec_pm {
+	struct clk	*vcodecpll;
+	struct clk	*univpll_d2;
+	struct clk	*clk_cci400_sel;
+	struct clk	*vdecpll;
+	struct clk	*vdec_sel;
+	struct clk	*vencpll_d2;
+	struct clk	*venc_sel;
+	struct clk	*univpll1_d2;
+	struct clk	*venc_lt_sel;
+	struct device	*larbvdec;
+	struct device	*larbvenc;
+	struct device	*larbvenclt;
+	struct device	*dev;
+	struct mtk_vcodec_dev *mtkdev;
+};
+
+
+/**
+ * struct mtk_vcodec_ctx - Context (instance) private data.
+ *
+ * @type:		type of the instance - decoder or encoder
+ * @dev:		pointer to the mtk_vcodec_dev of the device
+ * @fh:			struct v4l2_fh
+ * @m2m_ctx:		pointer to the v4l2_m2m_ctx of the context
+ * @q_data:		store information of input and output queue
+ *			of the context
+ * @idx:		index of the context that this structure describes
+ * @state:		state of the context
+ * @param_change:	encode parameters
+ * @enc_params:		encoding parameters
+ * @colorspace:
+ * @enc_if:		hoooked encoder driver interface
+ * @drv_handle:		driver handle for specific decode/encode instance
+ *
+ * @picinfo:		store width/height of image and buffer and planes' size for decoder
+ *			and encoder
+ * @dpb_count:		count of the DPB buffers required by MTK Vcodec hw
+ *
+ * @int_cond:		variable used by the waitqueue
+ * @int_type:		type of the last interrupt
+ * @queue:		waitqueue that can be used to wait for this context to
+ *			finish
+ * @irq_status:		irq status
+ *
+ * @ctrl_hdl:		handler for v4l2 framework
+ * @ctrls:		array of controls, used when adding controls to the
+ *			v4l2 control framework
+ *
+ * @encode_work:	worker for the encoding
+ */
+struct mtk_vcodec_ctx {
+	enum mtk_instance_type type;
+	struct mtk_vcodec_dev *dev;
+	struct v4l2_fh fh;
+	struct v4l2_m2m_ctx *m2m_ctx;
+	struct mtk_q_data q_data[2];
+	int idx;
+	enum mtk_instance_state state;
+	enum mtk_encode_param param_change;
+	struct mtk_enc_params enc_params;
+
+	struct venc_common_if *enc_if;
+	unsigned long drv_handle;
+
+
+	int int_cond;
+	int int_type;
+	wait_queue_head_t queue;
+	unsigned int irq_status;
+
+	struct v4l2_ctrl_handler ctrl_hdl;
+	struct v4l2_ctrl *ctrls[MTK_MAX_CTRLS];
+
+	struct work_struct encode_work;
+
+};
+
+/**
+ * struct mtk_vcodec_dev - driver data
+ * @v4l2_dev:		V4L2 device to register video devices for.
+ * @vfd_enc:		Video device for encoder.
+ *
+ * @m2m_dev_enc:	m2m device for encoder.
+ * @plat_dev:		platform device
+ * @alloc_ctx:		VB2 allocator context
+ *			(for allocations without kernel mapping).
+ * @ctx:		array of driver contexts
+ *
+ * @curr_ctx:		The context that is waiting for codec hardware
+ *
+ * @reg_base:		Mapped address of MTK Vcodec registers.
+ *
+ * @instance_mask:	used to mark which contexts are opened
+ * @num_instances:	counter of active MTK Vcodec instances
+ *
+ * @encode_workqueue:	encode work queue
+ *
+ * @int_cond:		used to identify interrupt condition happen
+ * @int_type:		used to identify what kind of interrupt condition happen
+ * @dev_mutex:		video_device lock
+ * @queue:		waitqueue for waiting for completion of device commands
+ *
+ * @enc_irq:		h264 encoder irq resource
+ * @enc_lt_irq:		vp8 encoder irq resource
+ *
+ * @enc_mutex:		encoder hardware lock.
+ *
+ * @pm:			power management control
+ * @dec_capability:	used to identify decode capability, ex: 4k
+ * @enc_capability:     used to identify encode capability
+ */
+struct mtk_vcodec_dev {
+	struct v4l2_device	v4l2_dev;
+	struct video_device	*vfd_enc;
+
+	struct v4l2_m2m_dev	*m2m_dev_enc;
+	struct platform_device	*plat_dev;
+	struct platform_device 	*vpu_plat_dev;
+	struct vb2_alloc_ctx	*alloc_ctx;
+	struct mtk_vcodec_ctx	*ctx[MTK_VCODEC_MAX_INSTANCES];
+	int curr_ctx;
+	void __iomem		*reg_base[NUM_MAX_VCODEC_REG_BASE];
+
+	unsigned long	instance_mask[BITS_TO_LONGS(MTK_VCODEC_MAX_INSTANCES)];
+	int			num_instances;
+
+	struct workqueue_struct *encode_workqueue;
+
+	int			int_cond;
+	int			int_type;
+	struct mutex		dev_mutex;
+	wait_queue_head_t	queue;
+
+	int			enc_irq;
+	int			enc_lt_irq;
+
+	struct mutex		enc_mutex;
+
+	struct mtk_vcodec_pm	pm;
+	unsigned int		dec_capability;
+	unsigned int		enc_capability;
+};
+
+/**
+ * struct mtk_vcodec_ctrl - information about controls to be registered.
+ * @id:			Control ID.
+ * @type:		Type of the control.
+ * @name:		Human readable name of the control.
+ * @minimum:		Minimum value of the control.
+ * @maximum:		Maximum value of the control.
+ * @step:		Control value increase step.
+ * @menu_skip_mask:	Mask of invalid menu positions.
+ * @default_value:	Initial value of the control.
+ * @is_volatile:	Control is volatile.
+ *
+ * See also struct v4l2_ctrl_config.
+ */
+struct mtk_vcodec_ctrl {
+	u32			id;
+	enum v4l2_ctrl_type	type;
+	u8			name[32];
+	s32			minimum;
+	s32			maximum;
+	s32			step;
+	u32			menu_skip_mask;
+	s32			default_value;
+	u8			is_volatile;
+};
+
+static inline struct mtk_vcodec_ctx *fh_to_ctx(struct v4l2_fh *fh)
+{
+	return container_of(fh, struct mtk_vcodec_ctx, fh);
+}
+
+static inline struct mtk_vcodec_ctx *ctrl_to_ctx(struct v4l2_ctrl *ctrl)
+{
+	return container_of(ctrl->handler, struct mtk_vcodec_ctx, ctrl_hdl);
+}
+
+extern const struct v4l2_ioctl_ops mtk_vdec_ioctl_ops;
+extern const struct v4l2_m2m_ops mtk_vdec_m2m_ops;
+extern const struct v4l2_ioctl_ops mtk_venc_ioctl_ops;
+extern const struct v4l2_m2m_ops mtk_venc_m2m_ops;
+
+#endif /* _MTK_VCODEC_DRV_H_ */
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
new file mode 100644
index 0000000..ee602fe
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
@@ -0,0 +1,1380 @@
+/*
+* Copyright (c) 2015 MediaTek Inc.
+* Author: PC Chen <pc.chen@mediatek.com>
+*         Tiffany Lin <tiffany.lin@mediatek.com>
+*
+* This program is free software; you can redistribute it and/or modify
+* it under the terms of the GNU General Public License version 2 as
+* published by the Free Software Foundation.
+*
+* This program is distributed in the hope that it will be useful,
+* but WITHOUT ANY WARRANTY; without even the implied warranty of
+* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+* GNU General Public License for more details.
+*/
+
+#include <media/v4l2-event.h>
+#include <media/v4l2-mem2mem.h>
+#include <media/videobuf2-dma-contig.h>
+
+#include "mtk_vcodec_drv.h"
+#include "mtk_vcodec_enc.h"
+#include "mtk_vcodec_intr.h"
+#include "mtk_vcodec_util.h"
+#include "venc_drv_if.h"
+
+#define MTK_VENC_MIN_W	32
+#define MTK_VENC_MIN_H	32
+#define MTK_VENC_MAX_W	1920
+#define MTK_VENC_MAX_H	1080
+#define DFT_CFG_WIDTH	MTK_VENC_MIN_W
+#define DFT_CFG_HEIGHT	MTK_VENC_MIN_H
+
+static void mtk_venc_worker(struct work_struct *work);
+
+static struct mtk_video_fmt mtk_video_formats[] = {
+	{
+		.fourcc		= V4L2_PIX_FMT_YUV420,
+		.type		= MTK_FMT_FRAME,
+		.num_planes	= 3,
+	},
+	{
+		.fourcc		= V4L2_PIX_FMT_YVU420,
+		.type		= MTK_FMT_FRAME,
+		.num_planes	= 3,
+	},
+	{
+		.fourcc		= V4L2_PIX_FMT_NV12,
+		.type		= MTK_FMT_FRAME,
+		.num_planes	= 2,
+	},
+	{
+		.fourcc		= V4L2_PIX_FMT_NV21,
+		.type		= MTK_FMT_FRAME,
+		.num_planes	= 2,
+	},
+	{
+		.fourcc		= V4L2_PIX_FMT_YUV420M,
+		.type		= MTK_FMT_FRAME,
+		.num_planes	= 3,
+	},
+	{
+		.fourcc		= V4L2_PIX_FMT_YVU420M,
+		.type		= MTK_FMT_FRAME,
+		.num_planes	= 3,
+	},
+	{
+		.fourcc 	= V4L2_PIX_FMT_NV12M,
+		.type		= MTK_FMT_FRAME,
+		.num_planes	= 2,
+	},
+	{
+		.fourcc		= V4L2_PIX_FMT_NV21M,
+		.type		= MTK_FMT_FRAME,
+		.num_planes	= 2,
+	},
+	{
+		.fourcc		= V4L2_PIX_FMT_H264,
+		.type		= MTK_FMT_ENC,
+		.num_planes	= 1,
+	},
+	{
+		.fourcc		= V4L2_PIX_FMT_VP8,
+		.type		= MTK_FMT_ENC,
+		.num_planes	= 1,
+	},
+};
+
+#define NUM_FORMATS ARRAY_SIZE(mtk_video_formats)
+
+static const struct mtk_codec_framesizes mtk_venc_framesizes[] = {
+	{
+		.fourcc	= V4L2_PIX_FMT_H264,
+		.stepwise = {  160, 1920, 16, 128, 1088, 16 },
+	},
+	{
+		.fourcc = V4L2_PIX_FMT_VP8,
+		.stepwise = {  160, 1920, 16, 128, 1088, 16 },
+	},
+};
+
+#define NUM_SUPPORTED_FRAMESIZE ARRAY_SIZE(mtk_venc_framesizes)
+
+static int vidioc_venc_s_ctrl(struct v4l2_ctrl *ctrl)
+{
+	struct mtk_vcodec_ctx *ctx = ctrl_to_ctx(ctrl);
+	struct mtk_enc_params *p = &ctx->enc_params;
+	int ret = 0;
+
+	switch (ctrl->id) {
+	case V4L2_CID_MPEG_VIDEO_BITRATE:
+		mtk_v4l2_debug(2, "V4L2_CID_MPEG_VIDEO_BITRATE val = %d",
+			ctrl->val);
+		p->bitrate = ctrl->val;
+		ctx->param_change |= MTK_ENCODE_PARAM_BITRATE;
+		break;
+	case V4L2_CID_MPEG_VIDEO_B_FRAMES:
+		mtk_v4l2_debug(2, "V4L2_CID_MPEG_VIDEO_B_FRAMES val = %d",
+			ctrl->val);
+		p->num_b_frame = ctrl->val;
+		break;
+	case V4L2_CID_MPEG_VIDEO_FRAME_RC_ENABLE:
+		mtk_v4l2_debug(2, "V4L2_CID_MPEG_VIDEO_FRAME_RC_ENABLE val = %d",
+			ctrl->val);
+		p->rc_frame = ctrl->val;
+		break;
+	case V4L2_CID_MPEG_VIDEO_H264_MAX_QP:
+		mtk_v4l2_debug(2, "V4L2_CID_MPEG_VIDEO_H264_MAX_QP val = %d",
+			ctrl->val);
+		p->h264_max_qp = ctrl->val;
+		break;
+	case V4L2_CID_MPEG_VIDEO_HEADER_MODE:
+		mtk_v4l2_debug(2, "V4L2_CID_MPEG_VIDEO_HEADER_MODE val = %d",
+			ctrl->val);
+		p->seq_hdr_mode = ctrl->val;
+		break;
+	case V4L2_CID_MPEG_VIDEO_MB_RC_ENABLE:
+		mtk_v4l2_debug(2, "V4L2_CID_MPEG_VIDEO_MB_RC_ENABLE val = %d",
+			ctrl->val);
+		p->rc_mb = ctrl->val;
+		break;
+	case V4L2_CID_MPEG_VIDEO_H264_PROFILE:
+		mtk_v4l2_debug(2, "V4L2_CID_MPEG_VIDEO_H264_PROFILE val = %d",
+			ctrl->val);
+		p->h264_profile = ctrl->val;
+		break;
+	case V4L2_CID_MPEG_VIDEO_H264_LEVEL:
+		mtk_v4l2_debug(2, "V4L2_CID_MPEG_VIDEO_H264_LEVEL val = %d",
+			ctrl->val);
+		p->h264_level = ctrl->val;
+		break;
+	case V4L2_CID_MPEG_VIDEO_H264_I_PERIOD:
+		mtk_v4l2_debug(2, "V4L2_CID_MPEG_VIDEO_H264_I_PERIOD val = %d",
+			ctrl->val);
+	case V4L2_CID_MPEG_VIDEO_GOP_SIZE:
+		mtk_v4l2_debug(2, "V4L2_CID_MPEG_VIDEO_GOP_SIZE val = %d",
+			ctrl->val);
+		p->gop_size = ctrl->val;
+		ctx->param_change |= MTK_ENCODE_PARAM_INTRA_PERIOD;
+		break;
+	case V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME:
+		mtk_v4l2_debug(2, "V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME");
+		p->force_intra = 1;
+		ctx->param_change |= MTK_ENCODE_PARAM_FRAME_TYPE;
+		break;
+	default:
+		ret = -EINVAL;
+		break;
+	}
+
+	return ret;
+}
+
+static const struct v4l2_ctrl_ops mtk_vcodec_enc_ctrl_ops = {
+	.s_ctrl = vidioc_venc_s_ctrl,
+};
+
+static int vidioc_enum_fmt(struct file *file, struct v4l2_fmtdesc *f,
+			   bool out)
+{
+	struct mtk_video_fmt *fmt;
+	int i, j = 0;
+
+	for (i = 0; i < NUM_FORMATS; ++i) {
+		if (out && mtk_video_formats[i].type != MTK_FMT_FRAME)
+			continue;
+		else if (!out && mtk_video_formats[i].type != MTK_FMT_ENC)
+			continue;
+
+		if (j == f->index) {
+			fmt = &mtk_video_formats[i];
+			f->pixelformat = fmt->fourcc;
+			return 0;
+		}
+		++j;
+	}
+
+	return -EINVAL;
+}
+
+static int vidioc_enum_framesizes(struct file *file, void *fh,
+				  struct v4l2_frmsizeenum *fsize)
+{
+	int i = 0;
+
+	for (i = 0; i < NUM_SUPPORTED_FRAMESIZE; ++i) {
+		if (fsize->pixel_format != mtk_venc_framesizes[i].fourcc)
+			continue;
+
+		if (!fsize->index) {
+			fsize->type = V4L2_FRMSIZE_TYPE_STEPWISE;
+			fsize->stepwise = mtk_venc_framesizes[i].stepwise;
+			return 0;
+		}
+	}
+
+	return -EINVAL;
+}
+
+static int vidioc_enum_fmt_vid_cap_mplane(struct file *file, void *pirv,
+					  struct v4l2_fmtdesc *f)
+{
+	return vidioc_enum_fmt(file, f, false);
+}
+
+static int vidioc_enum_fmt_vid_out_mplane(struct file *file, void *prov,
+					  struct v4l2_fmtdesc *f)
+{
+	return vidioc_enum_fmt(file, f, true);
+}
+
+static int vidioc_venc_querycap(struct file *file, void *priv,
+				struct v4l2_capability *cap)
+{
+        strlcpy(cap->driver, MTK_VCODEC_ENC_NAME, strlen(MTK_VCODEC_ENC_NAME));
+        cap->driver[strlen(MTK_VCODEC_ENC_NAME)]=0;
+        strlcpy(cap->bus_info, MTK_PLATFORM_STR, strlen(MTK_PLATFORM_STR));
+        cap->bus_info[strlen(MTK_PLATFORM_STR)]=0;
+        strlcpy(cap->card, MTK_PLATFORM_STR, strlen(MTK_PLATFORM_STR));
+        cap->card[strlen(MTK_PLATFORM_STR)]=0;
+
+	/*
+	 * This is only a mem-to-mem video device. The capture and output
+	 * device capability flags are left only for backward compatibility
+	 * and are scheduled for removal.
+	 */
+	cap->device_caps  = V4L2_CAP_VIDEO_M2M_MPLANE | V4L2_CAP_STREAMING;
+	cap->capabilities = cap->device_caps | V4L2_CAP_DEVICE_CAPS;
+
+	return 0;
+}
+static int vidioc_venc_s_parm(struct file *file, void *priv,
+			      struct v4l2_streamparm *a)
+{
+	struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
+
+	if (a->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
+		ctx->enc_params.framerate_num =
+			a->parm.output.timeperframe.denominator;
+		ctx->enc_params.framerate_denom =
+			a->parm.output.timeperframe.numerator;
+		ctx->param_change |= MTK_ENCODE_PARAM_FRAMERATE;
+	} else {
+		return -EINVAL;
+	}
+	return 0;
+}
+
+static struct mtk_q_data *mtk_venc_get_q_data(struct mtk_vcodec_ctx *ctx,
+					      enum v4l2_buf_type type)
+{
+	if (V4L2_TYPE_IS_OUTPUT(type))
+		return &ctx->q_data[MTK_Q_DATA_SRC];
+
+	return &ctx->q_data[MTK_Q_DATA_DST];
+}
+
+static struct mtk_video_fmt *mtk_venc_find_format(struct v4l2_format *f)
+{
+	struct mtk_video_fmt *fmt;
+	unsigned int k;
+
+	for (k = 0; k < NUM_FORMATS; k++) {
+		fmt = &mtk_video_formats[k];
+		if (fmt->fourcc == f->fmt.pix.pixelformat)
+			return fmt;
+	}
+
+	return NULL;
+}
+
+static void mtk_vcodec_enc_calc_src_size(
+	unsigned int num_planes, unsigned int pic_width,
+	unsigned int pic_height, unsigned int sizeimage[],
+	unsigned int bytesperline[])
+{
+	unsigned int y_pitch_w_div16;
+	unsigned int c_pitch_w_div16;
+
+	y_pitch_w_div16 = ALIGN(pic_width, 16) >> 4;
+	c_pitch_w_div16 = ALIGN(pic_width, 16) >> 4;
+
+	if (num_planes == 2) {
+		sizeimage[0] =
+			(y_pitch_w_div16) * (((pic_height + 31) / 32) * 2) * 256 +
+			((y_pitch_w_div16 % 8 == 0) ? 0 : ((ALIGN(pic_width, 16) * 2) * 16));
+
+		sizeimage[1] =
+			(c_pitch_w_div16) * (((pic_height + 31) / 32) * 2) * 128 +
+			((c_pitch_w_div16 % 8 == 0) ? 0 : (ALIGN(pic_width, 16) * 16));
+
+		sizeimage[2] = 0;
+
+		bytesperline[0] = ALIGN(pic_width, 16);
+		bytesperline[1] = ALIGN(pic_width, 16);
+		bytesperline[2] = 0;
+
+	} else {
+		sizeimage[0] =
+			(y_pitch_w_div16) * (((pic_height + 31) / 32) * 2) * 256 +
+			((y_pitch_w_div16 % 8 == 0) ? 0 : ((ALIGN(pic_width, 16) * 2) * 16));
+
+		sizeimage[1] =
+			(c_pitch_w_div16) * (((pic_height + 31) / 32) * 2) * 64 +
+			((c_pitch_w_div16 % 8 == 0) ? 0 : ((ALIGN(pic_width, 16) / 2) * 16));
+
+		sizeimage[2] =
+			(c_pitch_w_div16) * (((pic_height + 31) / 32) * 2) * 64 +
+			((c_pitch_w_div16 % 8 == 0) ? 0 : ((ALIGN(pic_width, 16) / 2) * 16));
+
+		bytesperline[0] = ALIGN(pic_width, 16);
+		bytesperline[1] = ALIGN(pic_width, 16) / 2;
+		bytesperline[2] = ALIGN(pic_width, 16) / 2;
+	}
+}
+
+static int vidioc_try_fmt(struct v4l2_format *f, struct mtk_video_fmt *fmt)
+{
+	struct v4l2_pix_format_mplane *pix_fmt_mp = &f->fmt.pix_mp;
+
+	/* V4L2 specification suggests the driver corrects the format struct
+	  * if any of the dimensions is unsupported */
+        if (pix_fmt_mp->height < MTK_VENC_MIN_H)
+                pix_fmt_mp->height = MTK_VENC_MIN_H;
+        else if (pix_fmt_mp->height > MTK_VENC_MAX_H)
+                pix_fmt_mp->height = MTK_VENC_MAX_H;
+
+        if (pix_fmt_mp->width < MTK_VENC_MIN_W)
+                pix_fmt_mp->width = MTK_VENC_MIN_W;
+        else if (pix_fmt_mp->width > MTK_VENC_MAX_W)
+                pix_fmt_mp->width = MTK_VENC_MAX_W;
+
+        pix_fmt_mp->field = V4L2_FIELD_NONE;
+
+	if (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
+		int size = pix_fmt_mp->height * pix_fmt_mp->width;
+		if (fmt->num_planes != pix_fmt_mp->num_planes)
+			pix_fmt_mp->num_planes = fmt->num_planes;
+
+		if(pix_fmt_mp->plane_fmt[0].sizeimage != size)
+			pix_fmt_mp->plane_fmt[0].sizeimage = size;
+ 		pix_fmt_mp->plane_fmt[0].bytesperline = 0;
+		memset(&(pix_fmt_mp->plane_fmt[0].reserved[0]), 0x0,
+			sizeof(pix_fmt_mp->plane_fmt[0].reserved));
+	} else if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
+		int i;
+		unsigned int sizeimage[VIDEO_MAX_PLANES];
+		unsigned int bytesperline[VIDEO_MAX_PLANES];
+
+		v4l_bound_align_image(&pix_fmt_mp->width, 8, 1920, 1,
+				      &pix_fmt_mp->height, 4, 1080, 1, 0);
+
+		if (fmt->num_planes != pix_fmt_mp->num_planes)
+			pix_fmt_mp->num_planes = fmt->num_planes;
+
+		mtk_vcodec_enc_calc_src_size(pix_fmt_mp->num_planes,
+					pix_fmt_mp->width,
+					pix_fmt_mp->height,
+					sizeimage, bytesperline);
+
+		for (i=0; i < pix_fmt_mp->num_planes; i++) {
+			pix_fmt_mp->plane_fmt[i].sizeimage = sizeimage[i];
+			pix_fmt_mp->plane_fmt[i].bytesperline = bytesperline[i];
+			memset(&(pix_fmt_mp->plane_fmt[i].reserved[0]), 0x0,
+				sizeof(pix_fmt_mp->plane_fmt[0].reserved));
+		}
+	} else {
+		return -EINVAL;
+	}
+
+	pix_fmt_mp->flags = 0;
+	pix_fmt_mp->ycbcr_enc = 0;
+	pix_fmt_mp->quantization = 0;
+	pix_fmt_mp->xfer_func = 0;
+	memset(&pix_fmt_mp->reserved[0], 0x0, sizeof(pix_fmt_mp->reserved));
+
+	return 0;
+}
+
+static void mtk_venc_set_param(struct mtk_vcodec_ctx *ctx, void *param)
+{
+	struct venc_enc_prm *p = (struct venc_enc_prm *)param;
+	struct mtk_q_data *q_data_src = &ctx->q_data[MTK_Q_DATA_SRC];
+	struct mtk_enc_params *enc_params = &ctx->enc_params;
+	unsigned int frame_rate;
+
+	frame_rate = enc_params->framerate_num / enc_params->framerate_denom;
+
+	switch (q_data_src->fmt->fourcc) {
+	case V4L2_PIX_FMT_YUV420:
+	case V4L2_PIX_FMT_YUV420M:
+		p->input_fourcc = VENC_YUV_FORMAT_420;
+		break;
+	case V4L2_PIX_FMT_YVU420:
+	case V4L2_PIX_FMT_YVU420M:
+		p->input_fourcc = VENC_YUV_FORMAT_YV12;
+		break;
+	case V4L2_PIX_FMT_NV12:
+	case V4L2_PIX_FMT_NV12M:
+		p->input_fourcc = VENC_YUV_FORMAT_NV12;
+		break;
+	case V4L2_PIX_FMT_NV21:
+	case V4L2_PIX_FMT_NV21M:
+		p->input_fourcc = VENC_YUV_FORMAT_NV21;
+		break;
+	}
+	p->h264_profile = enc_params->h264_profile;
+	p->h264_level = enc_params->h264_level;
+	p->width = q_data_src->width;
+	p->height = q_data_src->height;
+	p->buf_width = q_data_src->bytesperline[0];
+	p->buf_height = ((q_data_src->height + 0xf) & (~0xf));
+	p->frm_rate = frame_rate;
+	p->intra_period = enc_params->gop_size;
+	p->bitrate = enc_params->bitrate;
+
+	ctx->param_change = MTK_ENCODE_PARAM_NONE;
+
+	mtk_v4l2_debug(1, "fmt 0x%x, P/L %d/%d, w/h %d/%d, buf %d/%d, fps/bps %d/%d, gop %d",
+		       p->input_fourcc, p->h264_profile, p->h264_level,
+		       p->width, p->height, p->buf_width, p->buf_height,
+		       p->frm_rate, p->bitrate, p->intra_period);
+}
+
+static int vidioc_venc_s_fmt(struct file *file, void *priv,
+			     struct v4l2_format *f)
+{
+	struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
+	struct v4l2_pix_format_mplane *pix_fmt_mp = &f->fmt.pix_mp;
+	struct vb2_queue *vq;
+	struct mtk_q_data *q_data;
+	struct venc_enc_prm param;
+	int i, ret;
+	struct mtk_video_fmt *fmt;
+
+	vq = v4l2_m2m_get_vq(ctx->m2m_ctx, f->type);
+	if (!vq) {
+		mtk_v4l2_err("fail to get vq\n");
+		return -EINVAL;
+	}
+
+	if (vb2_is_busy(vq)) {
+		mtk_v4l2_err("queue busy\n");
+		return -EBUSY;
+	}
+
+	q_data = mtk_venc_get_q_data(ctx, f->type);
+	if (!q_data) {
+		mtk_v4l2_err("fail to get q data\n");
+		return -EINVAL;
+	}
+
+	fmt = mtk_venc_find_format(f);
+	if (!fmt) {
+		if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
+			f->fmt.pix.pixelformat = mtk_video_formats[0].fourcc;
+			fmt = mtk_venc_find_format(f);
+		} else {
+			f->fmt.pix.pixelformat = mtk_video_formats[8].fourcc;
+			fmt = mtk_venc_find_format(f);
+		}
+	}
+
+	q_data->fmt = fmt;
+	ret = vidioc_try_fmt(f, q_data->fmt);
+	if (ret)
+		return ret;
+
+	q_data->width		= f->fmt.pix_mp.width;
+	q_data->height		= f->fmt.pix_mp.height;
+	q_data->field		= f->fmt.pix_mp.field;
+
+	if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
+		q_data->colorspace = f->fmt.pix_mp.colorspace;
+		ctx->q_data[MTK_Q_DATA_SRC].bytesperline[0] =
+			ALIGN(q_data->width, 16);
+
+		if (q_data->fmt->num_planes == 2) {
+			ctx->q_data[MTK_Q_DATA_SRC].bytesperline[1] =
+				ALIGN(q_data->width, 16);
+			ctx->q_data[MTK_Q_DATA_SRC].bytesperline[2] = 0;
+		} else {
+			ctx->q_data[MTK_Q_DATA_SRC].bytesperline[1] =
+				ALIGN(q_data->width, 16) / 2;
+			ctx->q_data[MTK_Q_DATA_SRC].bytesperline[2] =
+				ALIGN(q_data->width, 16) / 2;
+		}
+
+		memset(&param, 0, sizeof(param));
+		mtk_venc_set_param(ctx, &param);
+		if (ctx->state == MTK_STATE_INIT) {
+			ret = venc_if_set_param(ctx,
+						VENC_SET_PARAM_ENC,
+						&param);
+			if (ret)
+				mtk_v4l2_err("venc_if_set_param failed=%d\n",
+						ret);
+
+			/* Get codec driver advice sizeimage from vpu */
+			for (i = 0; i < MTK_VCODEC_MAX_PLANES; i++) {
+				q_data->sizeimage[i] = param.sizeimage[i];
+				pix_fmt_mp->plane_fmt[i].sizeimage =
+					param.sizeimage[i];
+			}
+			q_data->bytesperline[0] =
+				pix_fmt_mp->plane_fmt[0].bytesperline;
+			q_data->bytesperline[1] =
+				pix_fmt_mp->plane_fmt[1].bytesperline;
+			q_data->bytesperline[2] =
+				pix_fmt_mp->plane_fmt[2].bytesperline;
+		} else {
+			for (i = 0; i < MTK_VCODEC_MAX_PLANES; i++) {
+				q_data->sizeimage[i] =
+					pix_fmt_mp->plane_fmt[i].sizeimage;
+				q_data->bytesperline[i] =
+					pix_fmt_mp->plane_fmt[i].bytesperline;
+			}
+		}
+	} else if (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE){
+		for (i = 0; i < f->fmt.pix_mp.num_planes; i++) {
+			struct v4l2_plane_pix_format	*plane_fmt;
+
+			plane_fmt = &f->fmt.pix_mp.plane_fmt[i];
+			q_data->bytesperline[i]	= plane_fmt->bytesperline;
+			q_data->sizeimage[i]	= plane_fmt->sizeimage;
+		}
+
+		if (ctx->state == MTK_STATE_FREE) {
+			ret = venc_if_create(ctx, q_data->fmt->fourcc);
+			if (ret) {
+				mtk_v4l2_err("venc_if_create failed=%d, codec type=%x\n",
+					ret, q_data->fmt->fourcc);
+				return 0;
+			}
+
+			ctx->state = MTK_STATE_INIT;
+		}
+	}
+
+	return 0;
+}
+
+static int vidioc_venc_g_fmt(struct file *file, void *priv,
+			     struct v4l2_format *f)
+{
+	struct v4l2_pix_format_mplane *pix = &f->fmt.pix_mp;
+	struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
+	struct vb2_queue *vq;
+	struct mtk_q_data *q_data;
+	int i;
+
+	vq = v4l2_m2m_get_vq(ctx->m2m_ctx, f->type);
+	if (!vq)
+		return -EINVAL;
+
+	q_data = mtk_venc_get_q_data(ctx, f->type);
+
+	pix->width = q_data->width;
+	pix->height = q_data->height;
+	pix->pixelformat = q_data->fmt->fourcc;
+	pix->field = q_data->field;
+	pix->colorspace = q_data->colorspace;
+	pix->num_planes = q_data->fmt->num_planes;
+	for (i = 0; i < pix->num_planes; i++) {
+		pix->plane_fmt[i].bytesperline = q_data->bytesperline[i];
+		pix->plane_fmt[i].sizeimage = q_data->sizeimage[i];
+		memset(&(pix->plane_fmt[i].reserved[0]), 0x0,
+			sizeof(pix->plane_fmt[i].reserved));
+	}
+	pix->flags = 0;
+	pix->ycbcr_enc = 0;
+	pix->quantization = 0;
+	pix->xfer_func = 0;
+	memset(&pix->reserved[0], 0x0, sizeof(pix->reserved));
+
+	return 0;
+}
+
+static int vidioc_try_fmt_vid_cap_mplane(struct file *file, void *priv,
+                                  struct v4l2_format *f)
+{
+        struct mtk_video_fmt *fmt;
+	struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
+
+        fmt = mtk_venc_find_format(f);
+        if (!fmt) {
+                f->fmt.pix.pixelformat = mtk_video_formats[8].fourcc;
+                fmt = mtk_venc_find_format(f);
+        }
+        if (fmt->type != MTK_FMT_ENC) {
+		mtk_v4l2_err("Fourcc format (0x%08x) invalid.\n",
+			     f->fmt.pix.pixelformat);
+		return -EINVAL;
+        }
+        f->fmt.pix_mp.colorspace = ctx->q_data[MTK_Q_DATA_SRC].colorspace;
+
+        return vidioc_try_fmt(f, fmt);
+}
+
+static int vidioc_try_fmt_vid_out_mplane(struct file *file, void *priv,
+                                  struct v4l2_format *f)
+{
+        struct mtk_video_fmt *fmt;
+
+        fmt = mtk_venc_find_format(f);
+        if (!fmt) {
+                f->fmt.pix.pixelformat = mtk_video_formats[0].fourcc;
+                fmt = mtk_venc_find_format(f);
+        }
+        if (!(fmt->type & MTK_FMT_FRAME)) {
+		mtk_v4l2_err("Fourcc format (0x%08x) invalid.\n",
+			     f->fmt.pix.pixelformat);
+		return -EINVAL;
+        }
+        if (!f->fmt.pix_mp.colorspace)
+                f->fmt.pix_mp.colorspace = V4L2_COLORSPACE_REC709;
+
+        return vidioc_try_fmt(f, fmt);
+}
+
+static int vidioc_venc_g_s_selection(struct file *file, void *priv,
+                                struct v4l2_selection *s)
+{
+	struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
+	struct mtk_q_data *q_data;
+
+	if (V4L2_TYPE_IS_OUTPUT(s->type)) {
+		if (s->target !=  V4L2_SEL_TGT_COMPOSE)
+			return -EINVAL;
+	} else {
+		if (s->target != V4L2_SEL_TGT_CROP)
+			return -EINVAL;
+	}
+
+	if (s->r.left || s->r.top)
+		return -EINVAL;
+
+	q_data = mtk_venc_get_q_data(ctx, s->type);
+	if (!q_data)
+		return -EINVAL;
+
+	s->r.width = q_data->width;
+	s->r.height = q_data->height;
+
+	return 0;
+}
+
+
+static int vidioc_venc_qbuf(struct file *file, void *priv,
+			    struct v4l2_buffer *buf)
+{
+
+	struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
+
+	if (ctx->state == MTK_STATE_ABORT) {
+		mtk_v4l2_err("[%d] Call on QBUF after unrecoverable error\n", ctx->idx);
+		return -EIO;
+	}
+
+	return v4l2_m2m_qbuf(file, ctx->m2m_ctx, buf);
+}
+
+static int vidioc_venc_dqbuf(struct file *file, void *priv,
+			     struct v4l2_buffer *buf)
+{
+	struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
+	if (ctx->state == MTK_STATE_ABORT) {
+		mtk_v4l2_err("[%d] Call on QBUF after unrecoverable error\n", ctx->idx);
+		return -EIO;
+	}
+
+	return v4l2_m2m_dqbuf(file, ctx->m2m_ctx, buf);
+}
+
+
+const struct v4l2_ioctl_ops mtk_venc_ioctl_ops = {
+	.vidioc_streamon		= v4l2_m2m_ioctl_streamon,
+	.vidioc_streamoff		= v4l2_m2m_ioctl_streamoff,
+
+	.vidioc_reqbufs			= v4l2_m2m_ioctl_reqbufs,
+	.vidioc_querybuf		= v4l2_m2m_ioctl_querybuf,
+	.vidioc_qbuf			= vidioc_venc_qbuf,
+	.vidioc_dqbuf			= vidioc_venc_dqbuf,
+
+	.vidioc_querycap		= vidioc_venc_querycap,
+	.vidioc_enum_fmt_vid_cap_mplane = vidioc_enum_fmt_vid_cap_mplane,
+	.vidioc_enum_fmt_vid_out_mplane = vidioc_enum_fmt_vid_out_mplane,
+	.vidioc_enum_framesizes		= vidioc_enum_framesizes,
+
+	.vidioc_try_fmt_vid_cap_mplane	= vidioc_try_fmt_vid_cap_mplane,
+	.vidioc_try_fmt_vid_out_mplane	= vidioc_try_fmt_vid_out_mplane,
+	.vidioc_expbuf			= v4l2_m2m_ioctl_expbuf,
+	.vidioc_subscribe_event		= v4l2_ctrl_subscribe_event,
+	.vidioc_unsubscribe_event	= v4l2_event_unsubscribe,
+
+	.vidioc_s_parm			= vidioc_venc_s_parm,
+
+	.vidioc_s_fmt_vid_cap_mplane	= vidioc_venc_s_fmt,
+	.vidioc_s_fmt_vid_out_mplane	= vidioc_venc_s_fmt,
+
+	.vidioc_g_fmt_vid_cap_mplane	= vidioc_venc_g_fmt,
+	.vidioc_g_fmt_vid_out_mplane	= vidioc_venc_g_fmt,
+
+	.vidioc_g_selection		= vidioc_venc_g_s_selection,
+	.vidioc_s_selection		= vidioc_venc_g_s_selection,
+};
+
+static int vb2ops_venc_queue_setup(struct vb2_queue *vq,
+				   const void *parg,
+				   unsigned int *nbuffers,
+				   unsigned int *nplanes,
+				   unsigned int sizes[], void *alloc_ctxs[])
+{
+	struct mtk_vcodec_ctx *ctx = vb2_get_drv_priv(vq);
+	struct mtk_q_data *q_data;
+
+	q_data = mtk_venc_get_q_data(ctx, vq->type);
+
+	if (*nbuffers < 1)
+		*nbuffers = 1;
+	if (*nbuffers > MTK_VIDEO_MAX_FRAME)
+		*nbuffers = MTK_VIDEO_MAX_FRAME;
+
+	*nplanes = q_data->fmt->num_planes;
+
+	if (vq->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
+		unsigned int i;
+
+		for (i = 0; i < *nplanes; i++) {
+			sizes[i] = q_data->sizeimage[i];
+			alloc_ctxs[i] = ctx->dev->alloc_ctx;
+		}
+	} else if (vq->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
+		sizes[0] = q_data->sizeimage[0];
+		alloc_ctxs[0] = ctx->dev->alloc_ctx;
+	} else {
+		return -EINVAL;
+	}
+
+	mtk_v4l2_debug(2, "[%d]get %d buffer(s) of size 0x%x each, vq->memory=%d",
+		       ctx->idx, *nbuffers, sizes[0], vq->memory);
+
+	return 0;
+}
+
+static int vb2ops_venc_buf_prepare(struct vb2_buffer *vb)
+{
+	struct mtk_vcodec_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue);
+	struct mtk_q_data *q_data;
+	int i;
+
+	q_data = mtk_venc_get_q_data(ctx, vb->vb2_queue->type);
+
+	for (i = 0; i < q_data->fmt->num_planes; i++) {
+		if (vb2_plane_size(vb, i) < q_data->sizeimage[i]) {
+			mtk_v4l2_debug(2, "data will not fit into plane %d (%lu < %d)",
+				       i, vb2_plane_size(vb, i),
+				       q_data->sizeimage[i]);
+			return -EINVAL;
+		}
+	}
+
+	return 0;
+}
+
+static void vb2ops_venc_buf_queue(struct vb2_buffer *vb)
+{
+	struct mtk_vcodec_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue);
+	struct vb2_v4l2_buffer *vb2_v4l2 =
+			container_of(vb, struct vb2_v4l2_buffer, vb2_buf);
+	struct mtk_video_enc_buf *mtk_buf =
+			container_of(vb2_v4l2, struct mtk_video_enc_buf, vb);
+
+	if ((vb->vb2_queue->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) &&
+		(ctx->param_change != MTK_ENCODE_PARAM_NONE)) {
+		mtk_v4l2_debug(1,
+				"[%d] Before id=%d encode parameter change %x",
+				ctx->idx, vb->index,
+				ctx->param_change);
+		mtk_buf->param_change = ctx->param_change;
+		if (mtk_buf->param_change & MTK_ENCODE_PARAM_BITRATE) {
+			mtk_buf->enc_params.bitrate = ctx->enc_params.bitrate;
+			mtk_v4l2_debug(1, "[%d] idx=%d change param br=%d",
+				ctx->idx,
+				mtk_buf->vb.vb2_buf.index,
+				mtk_buf->enc_params.bitrate);
+		}
+		if (ctx->param_change & MTK_ENCODE_PARAM_FRAMERATE) {
+			mtk_buf->enc_params.framerate_num =
+				ctx->enc_params.framerate_num;
+			mtk_buf->enc_params.framerate_denom =
+				ctx->enc_params.framerate_denom;
+			mtk_v4l2_debug(1, "[%d] idx=%d, change param fr=%d/%d",
+					ctx->idx,
+					mtk_buf->vb.vb2_buf.index,
+					mtk_buf->enc_params.framerate_num,
+					mtk_buf->enc_params.framerate_denom);
+		}
+		if (ctx->param_change & MTK_ENCODE_PARAM_INTRA_PERIOD) {
+			mtk_buf->enc_params.gop_size = ctx->enc_params.gop_size;
+			mtk_v4l2_debug(1, "[%d] idx=%d, change param intra period=%d",
+					ctx->idx,
+					mtk_buf->vb.vb2_buf.index,
+					mtk_buf->enc_params.gop_size);
+		}
+		if (ctx->param_change & MTK_ENCODE_PARAM_FRAME_TYPE) {
+			mtk_buf->enc_params.force_intra =
+				ctx->enc_params.force_intra;
+			mtk_v4l2_debug(1, "[%d] idx=%d, change param force I=%d",
+					ctx->idx,
+					mtk_buf->vb.vb2_buf.index,
+					mtk_buf->enc_params.force_intra);
+		}
+		ctx->param_change = MTK_ENCODE_PARAM_NONE;
+	}
+
+	v4l2_m2m_buf_queue(ctx->m2m_ctx, to_vb2_v4l2_buffer(vb));
+}
+
+static int vb2ops_venc_start_streaming(struct vb2_queue *q, unsigned int count)
+{
+	struct mtk_vcodec_ctx *ctx = vb2_get_drv_priv(q);
+	struct venc_enc_prm param;
+	int ret;
+	int i;
+
+	/* Once state turn into MTK_STATE_ABORT, we need stop_streaming to clear it */
+	if ((ctx->state == MTK_STATE_ABORT) || (ctx->state == MTK_STATE_FREE))
+		goto err_set_param;
+
+	if (!(vb2_start_streaming_called(&ctx->m2m_ctx->out_q_ctx.q) &
+	      vb2_start_streaming_called(&ctx->m2m_ctx->cap_q_ctx.q))) {
+		mtk_v4l2_debug(1, "[%d]-> out=%d cap=%d",
+		 ctx->idx,
+		 vb2_start_streaming_called(&ctx->m2m_ctx->out_q_ctx.q),
+		 vb2_start_streaming_called(&ctx->m2m_ctx->cap_q_ctx.q));
+		return 0;
+	}
+
+	mtk_venc_set_param(ctx, &param);
+	ret = venc_if_set_param(ctx,
+				VENC_SET_PARAM_ENC,
+				&param);
+	if (ret) {
+		mtk_v4l2_err("venc_if_set_param failed=%d\n", ret);
+		ctx->state = MTK_STATE_ABORT;
+		goto err_set_param;
+	}
+
+	if ((ctx->q_data[MTK_Q_DATA_DST].fmt->fourcc == V4L2_PIX_FMT_H264) &&
+	    (ctx->enc_params.seq_hdr_mode != V4L2_MPEG_VIDEO_HEADER_MODE_SEPARATE)) {
+		ret = venc_if_set_param(ctx,
+					VENC_SET_PARAM_PREPEND_HEADER,
+					0);
+		if (ret) {
+			mtk_v4l2_err("venc_if_set_param failed=%d\n", ret);
+			ctx->state = MTK_STATE_ABORT;
+			goto err_set_param;
+		}
+		ctx->state = MTK_STATE_HEADER;
+	}
+
+	return 0;
+
+err_set_param:
+	for (i = 0; i < q->num_buffers; ++i) {
+		if (q->bufs[i]->state == VB2_BUF_STATE_ACTIVE) {
+			mtk_v4l2_debug(0, "[%d] idx=%d, type=%d, %d -> VB2_BUF_STATE_QUEUED",
+					ctx->idx, i, q->type,
+					(int)q->bufs[i]->state );
+			v4l2_m2m_buf_done(to_vb2_v4l2_buffer(q->bufs[i]), VB2_BUF_STATE_QUEUED);
+		}
+	}
+
+	return -EINVAL;
+}
+
+static void vb2ops_venc_stop_streaming(struct vb2_queue *q)
+{
+	struct mtk_vcodec_ctx *ctx = vb2_get_drv_priv(q);
+	struct vb2_buffer *src_buf, *dst_buf;
+	int ret;
+
+	mtk_v4l2_debug(2, "[%d]-> type=%d", ctx->idx, q->type);
+
+	if (q->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
+		while ((dst_buf = v4l2_m2m_dst_buf_remove(ctx->m2m_ctx))) {
+			dst_buf->planes[0].bytesused = 0;
+			v4l2_m2m_buf_done(to_vb2_v4l2_buffer(dst_buf),
+						VB2_BUF_STATE_ERROR);
+		}
+	} else {
+		while ((src_buf = v4l2_m2m_src_buf_remove(ctx->m2m_ctx)))
+			v4l2_m2m_buf_done(to_vb2_v4l2_buffer(src_buf),
+						VB2_BUF_STATE_ERROR);
+	}
+
+	if ((q->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE &&
+	     vb2_is_streaming(&ctx->m2m_ctx->out_q_ctx.q)) ||
+	    (q->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE &&
+	     vb2_is_streaming(&ctx->m2m_ctx->cap_q_ctx.q))) {
+		mtk_v4l2_debug(1, "[%d]-> q type %d out=%d cap=%d",
+			       ctx->idx, q->type,
+			       vb2_is_streaming(&ctx->m2m_ctx->out_q_ctx.q),
+			       vb2_is_streaming(&ctx->m2m_ctx->cap_q_ctx.q));
+		return;
+	}
+
+	ret = venc_if_release(ctx);
+	if (ret)
+		mtk_v4l2_err("venc_if_release failed=%d\n", ret);
+
+	ctx->state = MTK_STATE_FREE;
+}
+
+static struct vb2_ops mtk_venc_vb2_ops = {
+	.queue_setup			= vb2ops_venc_queue_setup,
+	.buf_prepare			= vb2ops_venc_buf_prepare,
+	.buf_queue			= vb2ops_venc_buf_queue,
+	.wait_prepare			= vb2_ops_wait_prepare,
+	.wait_finish			= vb2_ops_wait_finish,
+	.start_streaming		= vb2ops_venc_start_streaming,
+	.stop_streaming			= vb2ops_venc_stop_streaming,
+};
+
+static int mtk_venc_encode_header(void *priv)
+{
+	struct mtk_vcodec_ctx *ctx = priv;
+	int ret;
+	struct vb2_buffer *dst_buf;
+	struct mtk_vcodec_mem bs_buf;
+	struct venc_done_result enc_result;
+
+	dst_buf = v4l2_m2m_dst_buf_remove(ctx->m2m_ctx);
+	if (!dst_buf) {
+		mtk_v4l2_debug(1, "No dst buffer");
+		return -EINVAL;
+	}
+
+	bs_buf.va = vb2_plane_vaddr(dst_buf, 0);
+	bs_buf.dma_addr = vb2_dma_contig_plane_dma_addr(dst_buf, 0);
+	bs_buf.size = (unsigned int)dst_buf->planes[0].length;
+
+	mtk_v4l2_debug(1,
+			"[%d] buf idx=%d va=0x%p dma_addr=0x%llx size=0x%lx",
+			ctx->idx,
+			dst_buf->index, bs_buf.va,
+			(u64)bs_buf.dma_addr,
+			bs_buf.size);
+
+	ret = venc_if_encode(ctx,
+			VENC_START_OPT_ENCODE_SEQUENCE_HEADER,
+			0, &bs_buf, &enc_result);
+
+	if (ret) {
+		dst_buf->planes[0].bytesused = 0;
+		ctx->state = MTK_STATE_ABORT;
+		v4l2_m2m_buf_done(to_vb2_v4l2_buffer(dst_buf), VB2_BUF_STATE_ERROR);
+		mtk_v4l2_err("venc_if_encode failed=%d", ret);
+		return -EINVAL;
+	}
+
+	ctx->state = MTK_STATE_HEADER;
+	dst_buf->planes[0].bytesused = enc_result.bs_size;
+
+#if defined(DEBUG)
+{
+	int i;
+	mtk_v4l2_debug(1, "[%d] venc_if_encode header len=%d",
+			ctx->idx,
+			enc_result.bs_size);
+	for (i = 0; i < enc_result.bs_size; i++) {
+		unsigned char *p = (unsigned char *)bs_buf.va;
+
+		mtk_v4l2_debug(1, "[%d] buf[%d]=0x%2x", ctx->idx, i, p[i]);
+	}
+}
+#endif
+	v4l2_m2m_buf_done(to_vb2_v4l2_buffer(dst_buf), VB2_BUF_STATE_DONE);
+
+	return 0;
+}
+
+static int mtk_venc_param_change(struct mtk_vcodec_ctx *ctx, void *priv)
+{
+	struct vb2_buffer *vb = priv;
+	struct vb2_v4l2_buffer *vb2_v4l2 =
+			container_of(vb, struct vb2_v4l2_buffer, vb2_buf);
+	struct mtk_video_enc_buf *mtk_buf =
+			container_of(vb2_v4l2, struct mtk_video_enc_buf, vb);
+	int ret = 0;
+
+	if (mtk_buf->param_change == MTK_ENCODE_PARAM_NONE)
+		return 0;
+
+	mtk_v4l2_debug(1, "encode parameters change id=%d", vb->index);
+	if (mtk_buf->param_change & MTK_ENCODE_PARAM_BITRATE) {
+		struct venc_enc_prm enc_prm;
+
+		enc_prm.bitrate = mtk_buf->enc_params.bitrate;
+		mtk_v4l2_debug(1, "[%d] idx=%d, change param br=%d",
+				ctx->idx,
+				mtk_buf->vb.vb2_buf.index,
+				enc_prm.bitrate);
+		ret |= venc_if_set_param(ctx,
+					 VENC_SET_PARAM_ADJUST_BITRATE,
+					 &enc_prm);
+	}
+	if (mtk_buf->param_change & MTK_ENCODE_PARAM_FRAMERATE) {
+		struct venc_enc_prm enc_prm;
+
+		enc_prm.frm_rate = mtk_buf->enc_params.framerate_num /
+				   mtk_buf->enc_params.framerate_denom;
+		mtk_v4l2_debug(1, "[%d] idx=%d, change param fr=%d",
+			       ctx->idx,
+			       mtk_buf->vb.vb2_buf.index,
+			       enc_prm.frm_rate);
+		ret |= venc_if_set_param(ctx,
+					 VENC_SET_PARAM_ADJUST_FRAMERATE,
+					 &enc_prm);
+	}
+	if (mtk_buf->param_change & MTK_ENCODE_PARAM_INTRA_PERIOD) {
+		mtk_v4l2_debug(1, "change param intra period=%d",
+				 mtk_buf->enc_params.gop_size);
+		ret |= venc_if_set_param(ctx,
+					 VENC_SET_PARAM_I_FRAME_INTERVAL,
+					 &mtk_buf->enc_params.gop_size);
+	}
+	if (mtk_buf->param_change & MTK_ENCODE_PARAM_FRAME_TYPE) {
+		mtk_v4l2_debug(1, "[%d] idx=%d, change param force I=%d",
+				ctx->idx,
+				mtk_buf->vb.vb2_buf.index,
+				mtk_buf->enc_params.force_intra);
+		if (mtk_buf->enc_params.force_intra)
+			ret |= venc_if_set_param(ctx,
+						 VENC_SET_PARAM_FORCE_INTRA,
+						 0);
+	}
+
+	mtk_buf->param_change = MTK_ENCODE_PARAM_NONE;
+
+	if (ret) {
+		ctx->state = MTK_STATE_ABORT;
+		mtk_v4l2_err("venc_if_set_param %d failed=%d\n",
+			MTK_ENCODE_PARAM_FRAME_TYPE, ret);
+		return -1;
+	}
+
+	return 0;
+}
+
+static void mtk_venc_worker(struct work_struct *work)
+{
+	struct mtk_vcodec_ctx *ctx = container_of(work, struct mtk_vcodec_ctx,
+				    encode_work);
+	struct vb2_buffer *src_buf, *dst_buf;
+	struct venc_frm_buf frm_buf;
+	struct mtk_vcodec_mem bs_buf;
+	struct venc_done_result enc_result;
+	int ret;
+	struct vb2_v4l2_buffer *v4l2_vb;
+
+	if ((ctx->q_data[MTK_Q_DATA_DST].fmt->fourcc == V4L2_PIX_FMT_H264) &&
+	    (ctx->state != MTK_STATE_HEADER)) {
+		/* encode h264 sps/pps header */
+		mtk_venc_encode_header(ctx);
+		v4l2_m2m_job_finish(ctx->dev->m2m_dev_enc, ctx->m2m_ctx);
+		return;
+	}
+
+	src_buf = v4l2_m2m_next_src_buf(ctx->m2m_ctx);
+	if (!src_buf) {
+		v4l2_m2m_job_finish(ctx->dev->m2m_dev_enc, ctx->m2m_ctx);
+		return;
+	}
+
+	dst_buf = v4l2_m2m_dst_buf_remove(ctx->m2m_ctx);
+	if (!dst_buf) {
+		v4l2_m2m_job_finish(ctx->dev->m2m_dev_enc, ctx->m2m_ctx);
+		return;
+	}
+
+	mtk_venc_param_change(ctx, src_buf);
+
+	frm_buf.fb_addr.va = vb2_plane_vaddr(src_buf, 0);
+	frm_buf.fb_addr.dma_addr = vb2_dma_contig_plane_dma_addr(src_buf, 0);
+	frm_buf.fb_addr.size = (unsigned int)src_buf->planes[0].length;
+	frm_buf.fb_addr1.va = vb2_plane_vaddr(src_buf, 1);
+	frm_buf.fb_addr1.dma_addr = vb2_dma_contig_plane_dma_addr(src_buf, 1);
+	frm_buf.fb_addr1.size = (unsigned int)src_buf->planes[1].length;
+	if (src_buf->num_planes == 3) {
+		frm_buf.fb_addr2.va = vb2_plane_vaddr(src_buf, 2);
+		frm_buf.fb_addr2.dma_addr =
+			vb2_dma_contig_plane_dma_addr(src_buf, 2);
+		frm_buf.fb_addr2.size =
+			(unsigned int)src_buf->planes[2].length;
+	} else {
+		frm_buf.fb_addr2.va = NULL;
+		frm_buf.fb_addr2.dma_addr = 0;
+		frm_buf.fb_addr2.size = 0;
+	}
+	bs_buf.va = vb2_plane_vaddr(dst_buf, 0);
+	bs_buf.dma_addr = vb2_dma_contig_plane_dma_addr(dst_buf, 0);
+	bs_buf.size = (unsigned int)dst_buf->planes[0].length;
+
+	mtk_v4l2_debug(2,
+			"Framebuf VA=%p PA=%llx Size=0x%lx;VA=%p PA=0x%llx Size=0x%lx;VA=%p PA=0x%llx Size=0x%lx",
+			frm_buf.fb_addr.va,
+			(u64)frm_buf.fb_addr.dma_addr,
+			frm_buf.fb_addr.size,
+			frm_buf.fb_addr1.va,
+			(u64)frm_buf.fb_addr1.dma_addr,
+			frm_buf.fb_addr1.size,
+			frm_buf.fb_addr2.va,
+			(u64)frm_buf.fb_addr2.dma_addr,
+			frm_buf.fb_addr2.size);
+
+	ret = venc_if_encode(ctx, VENC_START_OPT_ENCODE_FRAME,
+			     &frm_buf, &bs_buf, &enc_result);
+
+	src_buf = v4l2_m2m_src_buf_remove(ctx->m2m_ctx);
+	if (enc_result.msg == VENC_MESSAGE_OK)
+		v4l2_m2m_buf_done(to_vb2_v4l2_buffer(src_buf), VB2_BUF_STATE_DONE);
+	else
+		v4l2_m2m_buf_done(to_vb2_v4l2_buffer(src_buf), VB2_BUF_STATE_ERROR);
+
+	if (enc_result.is_key_frm) {
+		v4l2_vb = to_vb2_v4l2_buffer(dst_buf);
+		v4l2_vb->flags |= V4L2_BUF_FLAG_KEYFRAME;
+	}
+
+	if (ret) {
+		dst_buf->planes[0].bytesused = 0;
+		v4l2_m2m_buf_done(to_vb2_v4l2_buffer(dst_buf), VB2_BUF_STATE_ERROR);
+		mtk_v4l2_err("venc_if_encode failed=%d", ret);
+	} else {
+		dst_buf->planes[0].bytesused = enc_result.bs_size;
+		v4l2_m2m_buf_done(to_vb2_v4l2_buffer(dst_buf), VB2_BUF_STATE_DONE);
+		mtk_v4l2_debug(2, "venc_if_encode bs size=%d",
+				 enc_result.bs_size);
+	}
+
+	v4l2_m2m_job_finish(ctx->dev->m2m_dev_enc, ctx->m2m_ctx);
+
+	mtk_v4l2_debug(1, "<=== src_buf[%d] dst_buf[%d] venc_if_encode ret=%d Size=%u===>",
+			src_buf->index, dst_buf->index, ret,
+			enc_result.bs_size);
+}
+
+static void m2mops_venc_device_run(void *priv)
+{
+	struct mtk_vcodec_ctx *ctx = priv;
+	queue_work(ctx->dev->encode_workqueue, &ctx->encode_work);
+}
+
+static int m2mops_venc_job_ready(void *m2m_priv)
+{
+	struct mtk_vcodec_ctx *ctx = m2m_priv;
+
+	if (!v4l2_m2m_num_dst_bufs_ready(ctx->m2m_ctx)) {
+		mtk_v4l2_debug(3, "[%d]Not ready: not enough video dst buffers.",
+			       ctx->idx);
+		return 0;
+	}
+
+	if (!v4l2_m2m_num_src_bufs_ready(ctx->m2m_ctx)) {
+		mtk_v4l2_debug(3, "[%d]Not ready: not enough video src buffers.",
+			       ctx->idx);
+			return 0;
+		}
+
+	if (ctx->state == MTK_STATE_ABORT) {
+		mtk_v4l2_debug(3, "[%d]Not ready: state=0x%x.",
+			       ctx->idx, ctx->state);
+		return 0;
+	}
+
+	if (ctx->state == MTK_STATE_FREE) {
+		mtk_v4l2_debug(3, "[%d]Not ready: state=0x%x.",
+			       ctx->idx, ctx->state);
+		return 0;
+	}
+
+	return 1;
+}
+
+static void m2mops_venc_job_abort(void *priv)
+{
+	struct mtk_vcodec_ctx *ctx = priv;
+
+	ctx->state = MTK_STATE_ABORT;
+}
+
+static void m2mops_venc_lock(void *m2m_priv)
+{
+	struct mtk_vcodec_ctx *ctx = m2m_priv;
+
+	mutex_lock(&ctx->dev->dev_mutex);
+}
+
+static void m2mops_venc_unlock(void *m2m_priv)
+{
+	struct mtk_vcodec_ctx *ctx = m2m_priv;
+
+	mutex_unlock(&ctx->dev->dev_mutex);
+}
+
+const struct v4l2_m2m_ops mtk_venc_m2m_ops = {
+	.device_run			= m2mops_venc_device_run,
+	.job_ready			= m2mops_venc_job_ready,
+	.job_abort			= m2mops_venc_job_abort,
+	.lock				= m2mops_venc_lock,
+	.unlock				= m2mops_venc_unlock,
+};
+
+#define IS_MTK_VENC_PRIV(x) ((V4L2_CTRL_ID2CLASS(x) == V4L2_CTRL_CLASS_MPEG) &&\
+			     V4L2_CTRL_DRIVER_PRIV(x))
+
+void mtk_vcodec_enc_ctx_params_setup(struct mtk_vcodec_ctx *ctx)
+{
+	struct mtk_q_data *q_data;
+	struct mtk_video_fmt *fmt;
+
+	ctx->m2m_ctx->q_lock = &ctx->dev->dev_mutex;
+	ctx->fh.m2m_ctx = ctx->m2m_ctx;
+	ctx->fh.ctrl_handler = &ctx->ctrl_hdl;
+	INIT_WORK(&ctx->encode_work, mtk_venc_worker);
+
+	ctx->q_data[MTK_Q_DATA_SRC].width = DFT_CFG_WIDTH;
+	ctx->q_data[MTK_Q_DATA_SRC].height = DFT_CFG_HEIGHT;
+	ctx->q_data[MTK_Q_DATA_SRC].fmt = &mtk_video_formats[0];
+	ctx->q_data[MTK_Q_DATA_SRC].colorspace = V4L2_COLORSPACE_REC709;
+	ctx->q_data[MTK_Q_DATA_SRC].field = V4L2_FIELD_NONE;
+
+	q_data = &ctx->q_data[MTK_Q_DATA_SRC];
+	fmt = ctx->q_data[MTK_Q_DATA_SRC].fmt;
+	mtk_vcodec_enc_calc_src_size(fmt->num_planes, q_data->width,
+			q_data->height,
+			ctx->q_data[MTK_Q_DATA_SRC].sizeimage,
+			ctx->q_data[MTK_Q_DATA_SRC].bytesperline);
+
+	ctx->q_data[MTK_Q_DATA_DST].width = DFT_CFG_WIDTH;
+	ctx->q_data[MTK_Q_DATA_DST].height = DFT_CFG_HEIGHT;
+	ctx->q_data[MTK_Q_DATA_DST].fmt = &mtk_video_formats[9];
+	ctx->q_data[MTK_Q_DATA_DST].colorspace = V4L2_COLORSPACE_REC709;
+	ctx->q_data[MTK_Q_DATA_DST].field = V4L2_FIELD_NONE;
+
+	q_data = &ctx->q_data[MTK_Q_DATA_DST];
+	fmt = ctx->q_data[MTK_Q_DATA_DST].fmt;
+	ctx->q_data[MTK_Q_DATA_DST].sizeimage[0] = q_data->width * q_data->height;
+	ctx->q_data[MTK_Q_DATA_DST].bytesperline[0] = 0;
+
+}
+
+int mtk_vcodec_enc_ctrls_setup(struct mtk_vcodec_ctx *ctx)
+{
+	const struct v4l2_ctrl_ops *ops = &mtk_vcodec_enc_ctrl_ops;
+	struct v4l2_ctrl_handler *handler = &ctx->ctrl_hdl;
+	struct v4l2_ctrl_config cfg;
+
+	v4l2_ctrl_handler_init(handler, MTK_MAX_CTRLS);
+	if (handler->error) {
+		mtk_v4l2_err("Init control handler fail %d\n",
+				handler->error);
+		return handler->error;
+	}
+
+	ctx->ctrls[0] = v4l2_ctrl_new_std(handler, ops,
+					V4L2_CID_MPEG_VIDEO_BITRATE,
+					1, 4000000, 1, 4000000);
+	ctx->ctrls[1] = v4l2_ctrl_new_std(handler, ops,
+					V4L2_CID_MPEG_VIDEO_B_FRAMES,
+					0, 2, 1, 0);
+	ctx->ctrls[2] = v4l2_ctrl_new_std(handler, ops,
+					V4L2_CID_MPEG_VIDEO_FRAME_RC_ENABLE,
+					0, 1, 1, 1);
+	ctx->ctrls[3] = v4l2_ctrl_new_std(handler, ops,
+					V4L2_CID_MPEG_VIDEO_H264_MAX_QP,
+					0, 51, 1, 51);
+	ctx->ctrls[4] = v4l2_ctrl_new_std(handler, ops,
+					V4L2_CID_MPEG_VIDEO_H264_I_PERIOD,
+					0, 65535, 1, 30);
+	ctx->ctrls[5] = v4l2_ctrl_new_std(handler, ops,
+					V4L2_CID_MPEG_VIDEO_GOP_SIZE,
+					0, 65535, 1, 30);
+	ctx->ctrls[6] = v4l2_ctrl_new_std(handler, ops,
+					V4L2_CID_MPEG_VIDEO_MB_RC_ENABLE,
+					0, 1, 1, 0);
+	ctx->ctrls[7] = v4l2_ctrl_new_std_menu(handler, ops,
+					V4L2_CID_MPEG_VIDEO_HEADER_MODE,
+					V4L2_MPEG_VIDEO_HEADER_MODE_JOINED_WITH_1ST_FRAME,
+					0, V4L2_MPEG_VIDEO_HEADER_MODE_SEPARATE);
+	ctx->ctrls[8] = v4l2_ctrl_new_std_menu(handler, ops,
+					V4L2_CID_MPEG_VIDEO_H264_PROFILE,
+					V4L2_MPEG_VIDEO_H264_PROFILE_HIGH,
+					0, V4L2_MPEG_VIDEO_H264_PROFILE_MAIN);
+	ctx->ctrls[9] = v4l2_ctrl_new_std_menu(handler, ops,
+					V4L2_CID_MPEG_VIDEO_H264_LEVEL,
+					V4L2_MPEG_VIDEO_H264_LEVEL_4_2,
+					0, V4L2_MPEG_VIDEO_H264_LEVEL_4_0);
+	ctx->ctrls[6] = v4l2_ctrl_new_std(handler, ops,
+					V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME,
+					0, 0, 0, 0);
+
+	v4l2_ctrl_handler_setup(&ctx->ctrl_hdl);
+
+	return 0;
+}
+
+int mtk_vcodec_enc_queue_init(void *priv, struct vb2_queue *src_vq,
+			   struct vb2_queue *dst_vq)
+{
+	struct mtk_vcodec_ctx *ctx = priv;
+	int ret;
+
+	src_vq->type		= V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE;
+	src_vq->io_modes	= VB2_DMABUF | VB2_MMAP | VB2_USERPTR;
+	src_vq->drv_priv	= ctx;
+	src_vq->buf_struct_size = sizeof(struct mtk_video_enc_buf);
+	src_vq->ops		= &mtk_venc_vb2_ops;
+	src_vq->mem_ops		= &vb2_dma_contig_memops;
+	src_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
+	src_vq->lock = &ctx->dev->dev_mutex;
+
+	ret = vb2_queue_init(src_vq);
+	if (ret)
+		return ret;
+
+	dst_vq->type		= V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
+	dst_vq->io_modes	= VB2_DMABUF | VB2_MMAP | VB2_USERPTR;
+	dst_vq->drv_priv	= ctx;
+	dst_vq->buf_struct_size = sizeof(struct v4l2_m2m_buffer);
+	dst_vq->ops		= &mtk_venc_vb2_ops;
+	dst_vq->mem_ops		= &vb2_dma_contig_memops;
+	dst_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
+	dst_vq->lock = &ctx->dev->dev_mutex;
+
+	return vb2_queue_init(dst_vq);
+}
+
+int mtk_venc_unlock(struct mtk_vcodec_ctx *ctx)
+{
+	struct mtk_vcodec_dev *dev = ctx->dev;
+	dev->curr_ctx = -1;
+	mutex_unlock(&dev->enc_mutex);
+	return 0;
+}
+
+int mtk_venc_lock(struct mtk_vcodec_ctx *ctx)
+{
+	struct mtk_vcodec_dev *dev = ctx->dev;
+
+	mutex_lock(&dev->enc_mutex);
+	dev->curr_ctx = ctx->idx;
+	return 0;
+}
+
+void mtk_vcodec_enc_release(struct mtk_vcodec_ctx *ctx)
+{
+	venc_if_release(ctx);
+}
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h
new file mode 100644
index 0000000..e09524b
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h
@@ -0,0 +1,46 @@
+/*
+* Copyright (c) 2015 MediaTek Inc.
+* Author: PC Chen <pc.chen@mediatek.com>
+*         Tiffany Lin <tiffany.lin@mediatek.com>
+*
+* This program is free software; you can redistribute it and/or modify
+* it under the terms of the GNU General Public License version 2 as
+* published by the Free Software Foundation.
+*
+* This program is distributed in the hope that it will be useful,
+* but WITHOUT ANY WARRANTY; without even the implied warranty of
+* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+* GNU General Public License for more details.
+*/
+
+
+#ifndef _MTK_VCODEC_ENC_H_
+#define _MTK_VCODEC_ENC_H_
+
+#include <media/videobuf2-core.h>
+#include <media/videobuf2-v4l2.h>
+
+/**
+ * struct mtk_video_enc_buf - Private data related to each VB2 buffer.
+ * @b:			Pointer to related VB2 buffer.
+ * @param_change:	Types of encode parameter change before encode this
+ *			buffer
+ * @enc_params		Encode parameters changed before encode this buffer
+ */
+struct mtk_video_enc_buf {
+	struct vb2_v4l2_buffer	vb;
+	struct list_head	list;
+
+	enum mtk_encode_param param_change;
+	struct mtk_enc_params enc_params;
+};
+
+int mtk_venc_unlock(struct mtk_vcodec_ctx *ctx);
+int mtk_venc_lock(struct mtk_vcodec_ctx *ctx);
+int mtk_vcodec_enc_queue_init(void *priv, struct vb2_queue *src_vq,
+					struct vb2_queue *dst_vq);
+void mtk_vcodec_enc_release(struct mtk_vcodec_ctx *ctx);
+int mtk_vcodec_enc_ctrls_setup(struct mtk_vcodec_ctx *ctx);
+void mtk_vcodec_enc_ctx_params_setup(struct mtk_vcodec_ctx *ctx);
+
+#endif /* _MTK_VCODEC_ENC_H_ */
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
new file mode 100644
index 0000000..e7ab14a
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
@@ -0,0 +1,476 @@
+/*
+* Copyright (c) 2015 MediaTek Inc.
+* Author: PC Chen <pc.chen@mediatek.com>
+*         Tiffany Lin <tiffany.lin@mediatek.com>
+*
+* This program is free software; you can redistribute it and/or modify
+* it under the terms of the GNU General Public License version 2 as
+* published by the Free Software Foundation.
+*
+* This program is distributed in the hope that it will be useful,
+* but WITHOUT ANY WARRANTY; without even the implied warranty of
+* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+* GNU General Public License for more details.
+*/
+
+#include <linux/interrupt.h>
+#include <linux/irq.h>
+#include <linux/module.h>
+#include <linux/of_device.h>
+#include <linux/of.h>
+#include <media/v4l2-event.h>
+#include <media/v4l2-mem2mem.h>
+#include <media/videobuf2-dma-contig.h>
+#include <linux/pm_runtime.h>
+
+#include "mtk_vcodec_drv.h"
+#include "mtk_vcodec_enc.h"
+#include "mtk_vcodec_pm.h"
+#include "mtk_vcodec_intr.h"
+#include "mtk_vcodec_util.h"
+#include "mtk_vpu.h"
+
+
+/* Wake up context wait_queue */
+static void wake_up_ctx(struct mtk_vcodec_ctx *ctx, unsigned int reason)
+{
+	ctx->int_cond = 1;
+	ctx->int_type = reason;
+	wake_up_interruptible(&ctx->queue);
+}
+
+static irqreturn_t mtk_vcodec_enc_irq_handler(int irq, void *priv)
+{
+	struct mtk_vcodec_dev *dev = priv;
+	struct mtk_vcodec_ctx *ctx;
+	unsigned int irq_status;
+
+	if (dev->curr_ctx == -1) {
+		mtk_v4l2_err("curr_ctx = -1");
+		return IRQ_HANDLED;
+	}
+
+	ctx = dev->ctx[dev->curr_ctx];
+	if (ctx == NULL) {
+		mtk_v4l2_err("curr_ctx==NULL");
+		return IRQ_HANDLED;
+	}
+	mtk_v4l2_debug(1, "idx=%d", ctx->idx);
+	irq_status = readl(dev->reg_base[VENC_SYS] +
+				(MTK_VENC_IRQ_STATUS_OFFSET));
+	if (irq_status & MTK_VENC_IRQ_STATUS_PAUSE)
+		writel((MTK_VENC_IRQ_STATUS_PAUSE),
+		       dev->reg_base[VENC_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
+
+	if (irq_status & MTK_VENC_IRQ_STATUS_SWITCH)
+		writel((MTK_VENC_IRQ_STATUS_SWITCH),
+		       dev->reg_base[VENC_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
+
+	if (irq_status & MTK_VENC_IRQ_STATUS_DRAM)
+		writel((MTK_VENC_IRQ_STATUS_DRAM),
+		       dev->reg_base[VENC_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
+
+	if (irq_status & MTK_VENC_IRQ_STATUS_SPS)
+		writel((MTK_VENC_IRQ_STATUS_SPS),
+		       dev->reg_base[VENC_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
+
+	if (irq_status & MTK_VENC_IRQ_STATUS_PPS)
+		writel((MTK_VENC_IRQ_STATUS_PPS),
+		       dev->reg_base[VENC_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
+
+	if (irq_status & MTK_VENC_IRQ_STATUS_FRM)
+		writel((MTK_VENC_IRQ_STATUS_FRM),
+		       dev->reg_base[VENC_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
+
+	ctx->irq_status = irq_status;
+	wake_up_ctx(ctx, MTK_INST_IRQ_RECEIVED);
+	return IRQ_HANDLED;
+}
+
+#if 1 /* VENC_LT */
+static irqreturn_t mtk_vcodec_enc_irq_handler2(int irq, void *priv)
+{
+	struct mtk_vcodec_dev *dev = priv;
+	struct mtk_vcodec_ctx *ctx;
+	unsigned int irq_status;
+
+	ctx = dev->ctx[dev->curr_ctx];
+	if (ctx == NULL) {
+		mtk_v4l2_err("ctx==NULL");
+		return IRQ_HANDLED;
+	}
+	mtk_v4l2_debug(1, "idx=%d", ctx->idx);
+	irq_status = readl(dev->reg_base[VENC_LT_SYS] +
+				(MTK_VENC_IRQ_STATUS_OFFSET));
+	if (irq_status & MTK_VENC_IRQ_STATUS_PAUSE)
+		writel((MTK_VENC_IRQ_STATUS_PAUSE),
+		       dev->reg_base[VENC_LT_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
+
+	if (irq_status & MTK_VENC_IRQ_STATUS_SWITCH)
+		writel((MTK_VENC_IRQ_STATUS_SWITCH),
+		       dev->reg_base[VENC_LT_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
+
+	if (irq_status & MTK_VENC_IRQ_STATUS_DRAM)
+		writel((MTK_VENC_IRQ_STATUS_DRAM),
+		       dev->reg_base[VENC_LT_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
+
+	if (irq_status & MTK_VENC_IRQ_STATUS_SPS)
+		writel((MTK_VENC_IRQ_STATUS_SPS),
+		       dev->reg_base[VENC_LT_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
+
+	if (irq_status & MTK_VENC_IRQ_STATUS_PPS)
+		writel((MTK_VENC_IRQ_STATUS_PPS),
+		       dev->reg_base[VENC_LT_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
+
+	if (irq_status & MTK_VENC_IRQ_STATUS_FRM)
+		writel((MTK_VENC_IRQ_STATUS_FRM),
+		       dev->reg_base[VENC_LT_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
+
+	ctx->irq_status = irq_status;
+	wake_up_ctx(ctx, MTK_INST_IRQ_RECEIVED);
+	return IRQ_HANDLED;
+}
+#endif
+
+static void mtk_vcodec_enc_reset_handler(void *priv)
+{
+	int i;
+	struct mtk_vcodec_dev *dev = priv;
+	struct mtk_vcodec_ctx *ctx;
+
+	mtk_v4l2_debug(0, "Watchdog timeout!!");
+
+	mutex_lock(&dev->dev_mutex);
+	for(i = 0; i < MTK_VCODEC_MAX_ENCODER_INSTANCES; i++) {
+		ctx = dev->ctx[i];
+		if (ctx) {
+			ctx->state = MTK_STATE_ABORT;
+			mtk_v4l2_debug(0, "[%d] Change to state MTK_STATE_ERROR", ctx->idx);
+		}
+
+	}
+	mutex_unlock(&dev->dev_mutex);
+}
+
+static int fops_vcodec_open(struct file *file)
+{
+	struct video_device *vfd = video_devdata(file);
+	struct mtk_vcodec_dev *dev = video_drvdata(file);
+	struct mtk_vcodec_ctx *ctx = NULL;
+	int ret = 0;
+
+	mutex_lock(&dev->dev_mutex);
+
+	ctx = devm_kzalloc(&dev->plat_dev->dev, sizeof(*ctx), GFP_KERNEL);
+	if (!ctx) {
+		ret = -ENOMEM;
+		goto err_alloc;
+	}
+
+	if (dev->num_instances >= MTK_VCODEC_MAX_ENCODER_INSTANCES) {
+		mtk_v4l2_err("Too many open contexts\n");
+		ret = -EBUSY;
+		goto err_no_ctx;
+	}
+
+	ctx->idx = ffz(dev->instance_mask[0]);
+	v4l2_fh_init(&ctx->fh, video_devdata(file));
+	file->private_data = &ctx->fh;
+	v4l2_fh_add(&ctx->fh);
+	ctx->dev = dev;
+
+	if (vfd == dev->vfd_enc) {
+		ctx->type = MTK_INST_ENCODER;
+
+		ret = mtk_vcodec_enc_ctrls_setup(ctx);
+		if (ret) {
+			mtk_v4l2_err("Failed to setup controls() (%d)\n",
+				       ret);
+			goto err_ctrls_setup;
+		}
+		ctx->m2m_ctx = v4l2_m2m_ctx_init(dev->m2m_dev_enc, ctx,
+						 &mtk_vcodec_enc_queue_init);
+		if (IS_ERR(ctx->m2m_ctx)) {
+			ret = PTR_ERR(ctx->m2m_ctx);
+			mtk_v4l2_err("Failed to v4l2_m2m_ctx_init() (%d)\n",
+				       ret);
+			goto err_ctx_init;
+		}
+		mtk_vcodec_enc_ctx_params_setup(ctx);
+	} else {
+		mtk_v4l2_err("Invalid vfd !\n");
+		ret = -ENOENT;
+		goto err_ctx_init;
+	}
+
+	init_waitqueue_head(&ctx->queue);
+	dev->num_instances++;
+
+	if (dev->num_instances == 1) {
+		ret = vpu_load_firmware(dev->vpu_plat_dev);
+		if (ret < 0) {
+				mtk_v4l2_err("vpu_load_firmware failed!\n");
+			goto err_load_fw;
+		}
+
+		dev->enc_capability =
+			vpu_get_venc_hw_capa(dev->vpu_plat_dev);
+		mtk_v4l2_debug(0, "encoder capability %x", dev->enc_capability);
+	}
+
+	mtk_v4l2_debug(2, "Create instance [%d]@%p m2m_ctx=%p type=%d\n",
+			 ctx->idx, ctx, ctx->m2m_ctx, ctx->type);
+	set_bit(ctx->idx, &dev->instance_mask[0]);
+	dev->ctx[ctx->idx] = ctx;
+
+	mutex_unlock(&dev->dev_mutex);
+	mtk_v4l2_debug(0, "%s encoder [%d]", dev_name(&dev->plat_dev->dev), ctx->idx);
+	return ret;
+
+	/* Deinit when failure occurred */
+err_load_fw:
+	v4l2_m2m_ctx_release(ctx->m2m_ctx);
+	v4l2_fh_del(&ctx->fh);
+	v4l2_fh_exit(&ctx->fh);
+	dev->num_instances--;
+err_ctx_init:
+err_ctrls_setup:
+	v4l2_ctrl_handler_free(&ctx->ctrl_hdl);
+err_no_ctx:
+	devm_kfree(&dev->plat_dev->dev, ctx);
+err_alloc:
+	mutex_unlock(&dev->dev_mutex);
+	return ret;
+}
+
+static int fops_vcodec_release(struct file *file)
+{
+	struct mtk_vcodec_dev *dev = video_drvdata(file);
+	struct mtk_vcodec_ctx *ctx = fh_to_ctx(file->private_data);
+
+	mtk_v4l2_debug(0, "[%d] encoder\n", ctx->idx);
+	mutex_lock(&dev->dev_mutex);
+
+	mtk_vcodec_enc_release(ctx);
+	v4l2_fh_del(&ctx->fh);
+	v4l2_fh_exit(&ctx->fh);
+	v4l2_ctrl_handler_free(&ctx->ctrl_hdl);
+	v4l2_m2m_ctx_release(ctx->m2m_ctx);
+
+	dev->ctx[ctx->idx] = NULL;
+	dev->num_instances--;
+	clear_bit(ctx->idx, &dev->instance_mask[0]);
+	devm_kfree(&dev->plat_dev->dev, ctx);
+	mutex_unlock(&dev->dev_mutex);
+	return 0;
+}
+
+static const struct v4l2_file_operations mtk_vcodec_fops = {
+	.owner				= THIS_MODULE,
+	.open				= fops_vcodec_open,
+	.release			= fops_vcodec_release,
+	.poll				= v4l2_m2m_fop_poll,
+	.unlocked_ioctl			= video_ioctl2,
+	.mmap				= v4l2_m2m_fop_mmap,
+};
+
+static int mtk_vcodec_probe(struct platform_device *pdev)
+{
+	struct mtk_vcodec_dev *dev;
+	struct video_device *vfd_enc;
+	struct resource *res;
+	int i, j, ret;
+
+	dev = devm_kzalloc(&pdev->dev, sizeof(*dev), GFP_KERNEL);
+	if (!dev)
+		return -ENOMEM;
+
+	dev->plat_dev = pdev;
+
+	dev->vpu_plat_dev = vpu_get_plat_device(dev->plat_dev);
+	if (dev->vpu_plat_dev == NULL) {
+		mtk_v4l2_err("[VPU] vpu device in not ready\n");
+		return -EPROBE_DEFER;
+	}
+
+	vpu_wdt_reg_handler(dev->vpu_plat_dev, mtk_vcodec_enc_reset_handler, dev,
+			    VPU_RST_ENC);
+
+	ret = mtk_vcodec_init_enc_pm(dev);
+	if (ret < 0) {
+		dev_err(&pdev->dev, "Failed to get mt vcodec clock source!\n");
+		return ret;
+	}
+
+	for (i = VENC_SYS, j = 0; i < NUM_MAX_VCODEC_REG_BASE; i++, j++) {
+		res = platform_get_resource(pdev, IORESOURCE_MEM, j);
+		if (res == NULL) {
+			dev_err(&pdev->dev, "get memory resource failed.\n");
+			ret = -ENXIO;
+			goto err_res;
+		}
+		dev->reg_base[i] = devm_ioremap_resource(&pdev->dev, res);
+		if (IS_ERR(dev->reg_base[i])) {
+			dev_err(&pdev->dev,
+				"devm_ioremap_resource %d failed.\n", i);
+			ret = PTR_ERR(dev->reg_base);
+			goto err_res;
+		}
+		mtk_v4l2_debug(2, "reg[%d] base=0x%p\n", i, dev->reg_base[i]);
+	}
+
+	res = platform_get_resource(pdev, IORESOURCE_IRQ, 0);
+	if (res == NULL) {
+		dev_err(&pdev->dev, "failed to get irq resource\n");
+		ret = -ENOENT;
+		goto err_res;
+	}
+
+	dev->enc_irq = platform_get_irq(pdev, 0);
+	ret = devm_request_irq(&pdev->dev, dev->enc_irq,
+			       mtk_vcodec_enc_irq_handler,
+			       0, pdev->name, dev);
+	if (ret) {
+		dev_err(&pdev->dev, "Failed to install dev->enc_irq %d (%d)\n",
+			dev->enc_irq,
+			ret);
+		ret = -EINVAL;
+		goto err_res;
+	}
+
+	dev->enc_lt_irq = platform_get_irq(pdev, 1);
+	ret = devm_request_irq(&pdev->dev,
+			       dev->enc_lt_irq, mtk_vcodec_enc_irq_handler2,
+			       0, pdev->name, dev);
+	if (ret) {
+		dev_err(&pdev->dev,
+			"Failed to install dev->enc_lt_irq %d (%d)\n",
+			dev->enc_lt_irq, ret);
+		ret = -EINVAL;
+		goto err_res;
+	}
+
+	disable_irq(dev->enc_irq);
+	disable_irq(dev->enc_lt_irq); /* VENC_LT */
+	mutex_init(&dev->enc_mutex);
+	mutex_init(&dev->dev_mutex);
+
+	snprintf(dev->v4l2_dev.name, sizeof(dev->v4l2_dev.name), "%s",
+		 "[MTK_V4L2_VENC]");
+
+	ret = v4l2_device_register(&pdev->dev, &dev->v4l2_dev);
+	if (ret) {
+		mtk_v4l2_err("v4l2_device_register err=%d\n", ret);
+		return ret;
+	}
+
+	init_waitqueue_head(&dev->queue);
+
+	/* allocate video device for encoder and register it */
+	vfd_enc = video_device_alloc();
+	if (!vfd_enc) {
+		mtk_v4l2_err("Failed to allocate video device\n");
+		ret = -ENOMEM;
+		goto err_enc_alloc;
+	}
+	vfd_enc->fops           = &mtk_vcodec_fops;
+	vfd_enc->ioctl_ops      = &mtk_venc_ioctl_ops;
+	vfd_enc->release        = video_device_release;
+	vfd_enc->lock           = &dev->dev_mutex;
+	vfd_enc->v4l2_dev       = &dev->v4l2_dev;
+	vfd_enc->vfl_dir        = VFL_DIR_M2M;
+
+	snprintf(vfd_enc->name, sizeof(vfd_enc->name), "%s",
+		 MTK_VCODEC_ENC_NAME);
+	video_set_drvdata(vfd_enc, dev);
+	dev->vfd_enc = vfd_enc;
+	platform_set_drvdata(pdev, dev);
+	ret = video_register_device(vfd_enc, VFL_TYPE_GRABBER, 1);
+	if (ret) {
+		mtk_v4l2_err("Failed to register video device\n");
+		goto err_enc_reg;
+	}
+	mtk_v4l2_debug(0, "encoder registered as /dev/video%d\n",
+			 vfd_enc->num);
+
+	dev->alloc_ctx = vb2_dma_contig_init_ctx(&pdev->dev);
+	if (IS_ERR(dev->alloc_ctx)) {
+		mtk_v4l2_err("Failed to alloc vb2 dma context 0\n");
+		ret = PTR_ERR(dev->alloc_ctx);
+		goto err_vb2_ctx_init;
+	}
+
+	dev->m2m_dev_enc = v4l2_m2m_init(&mtk_venc_m2m_ops);
+	if (IS_ERR(dev->m2m_dev_enc)) {
+		mtk_v4l2_err("Failed to init mem2mem enc device\n");
+		ret = PTR_ERR(dev->m2m_dev_enc);
+		goto err_enc_mem_init;
+	}
+
+	dev->encode_workqueue =
+			alloc_ordered_workqueue(MTK_VCODEC_ENC_NAME, WQ_MEM_RECLAIM | WQ_FREEZABLE);
+	if (!dev->encode_workqueue) {
+		mtk_v4l2_err("Failed to create encode workqueue\n");
+		ret = -EINVAL;
+		goto err_event_workq;
+	}
+
+	return 0;
+
+err_event_workq:
+	v4l2_m2m_release(dev->m2m_dev_enc);
+err_enc_mem_init:
+	vb2_dma_contig_cleanup_ctx(dev->alloc_ctx);
+err_vb2_ctx_init:
+	video_unregister_device(vfd_enc);
+err_enc_reg:
+	video_device_release(vfd_enc);
+err_enc_alloc:
+	v4l2_device_unregister(&dev->v4l2_dev);
+err_res:
+	mtk_vcodec_release_enc_pm(dev);
+	return ret;
+}
+
+static const struct of_device_id mtk_vcodec_match[] = {
+	{.compatible = "mediatek,mt8173-vcodec-enc",},
+	{},
+};
+MODULE_DEVICE_TABLE(of, mtk_vcodec_match);
+
+static int mtk_vcodec_remove(struct platform_device *pdev)
+{
+	struct mtk_vcodec_dev *dev = platform_get_drvdata(pdev);
+
+	mtk_v4l2_debug_enter();
+	flush_workqueue(dev->encode_workqueue);
+	destroy_workqueue(dev->encode_workqueue);
+	if (dev->m2m_dev_enc)
+		v4l2_m2m_release(dev->m2m_dev_enc);
+	if (dev->alloc_ctx)
+		vb2_dma_contig_cleanup_ctx(dev->alloc_ctx);
+
+	if (dev->vfd_enc) {
+		video_unregister_device(dev->vfd_enc);
+		video_device_release(dev->vfd_enc);
+	}
+	v4l2_device_unregister(&dev->v4l2_dev);
+	mtk_vcodec_release_enc_pm(dev);
+	return 0;
+}
+
+static struct platform_driver mtk_vcodec_driver = {
+	.probe	= mtk_vcodec_probe,
+	.remove	= mtk_vcodec_remove,
+	.driver	= {
+		.name	= MTK_VCODEC_ENC_NAME,
+		.owner	= THIS_MODULE,
+		.of_match_table = mtk_vcodec_match,
+	},
+};
+
+module_platform_driver(mtk_vcodec_driver);
+
+
+MODULE_LICENSE("GPL v2");
+MODULE_DESCRIPTION("Mediatek video codec V4L2 driver");
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
new file mode 100644
index 0000000..518fba7
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
@@ -0,0 +1,132 @@
+/*
+* Copyright (c) 2015 MediaTek Inc.
+* Author: Tiffany Lin <tiffany.lin@mediatek.com>
+*
+* This program is free software; you can redistribute it and/or modify
+* it under the terms of the GNU General Public License version 2 as
+* published by the Free Software Foundation.
+*
+* This program is distributed in the hope that it will be useful,
+* but WITHOUT ANY WARRANTY; without even the implied warranty of
+* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+* GNU General Public License for more details.
+*/
+
+#include <linux/clk.h>
+#include <linux/of_address.h>
+#include <linux/of_platform.h>
+#include <linux/pm_runtime.h>
+#include <soc/mediatek/smi.h>
+
+#include "mtk_vcodec_pm.h"
+#include "mtk_vcodec_util.h"
+#include "mtk_vpu.h"
+
+
+int mtk_vcodec_init_enc_pm(struct mtk_vcodec_dev *mtkdev)
+{
+	struct device_node *node;
+	struct platform_device *pdev;
+	struct device *dev;
+	struct mtk_vcodec_pm *pm;
+	int ret = 0;
+
+	pdev = mtkdev->plat_dev;
+	pm = &mtkdev->pm;
+	memset(pm, 0, sizeof(struct mtk_vcodec_pm));
+	pm->mtkdev = mtkdev;
+	dev = &pdev->dev;
+
+	node = of_parse_phandle(dev->of_node, "mediatek,larb", 0);
+	if (!node)
+		return -1;
+	pdev = of_find_device_by_node(node);
+	if (WARN_ON(!pdev)) {
+		of_node_put(node);
+		return -1;
+	}
+	pm->larbvenc = &pdev->dev;
+
+	node = of_parse_phandle(dev->of_node, "mediatek,larb", 1);
+	if (!node)
+		return -1;
+
+	pdev = of_find_device_by_node(node);
+	if (WARN_ON(!pdev)) {
+		of_node_put(node);
+		return -EINVAL;
+	}
+	pm->larbvenclt = &pdev->dev;
+
+	pdev = mtkdev->plat_dev;
+	pm->dev = &pdev->dev;
+
+	pm->vencpll_d2 = devm_clk_get(&pdev->dev, "vencpll_d2");
+	if (pm->vencpll_d2 == NULL) {
+		mtk_v4l2_err("devm_clk_get vencpll_d2 fail");
+		ret = -1;
+	}
+
+	pm->venc_sel = devm_clk_get(&pdev->dev, "venc_sel");
+	if (pm->venc_sel == NULL) {
+		mtk_v4l2_err("devm_clk_get venc_sel fail");
+		ret = -1;
+	}
+
+	pm->univpll1_d2 = devm_clk_get(&pdev->dev, "univpll1_d2");
+	if (pm->univpll1_d2 == NULL) {
+		mtk_v4l2_err("devm_clk_get univpll1_d2 fail");
+		ret = -1;
+	}
+
+	pm->venc_lt_sel = devm_clk_get(&pdev->dev, "venc_lt_sel");
+	if (pm->venc_lt_sel == NULL) {
+		mtk_v4l2_err("devm_clk_get venc_lt_sel fail");
+		ret = -1;
+	}
+
+	return ret;
+}
+
+void mtk_vcodec_release_enc_pm(struct mtk_vcodec_dev *mtkdev)
+{
+}
+
+
+void mtk_vcodec_enc_clock_on(struct mtk_vcodec_pm *pm)
+{
+	int ret;
+
+	ret = clk_prepare_enable(pm->venc_sel);
+	if (ret)
+		mtk_v4l2_err("venc_sel fail %d", ret);
+
+	ret = clk_set_parent(pm->venc_sel, pm->vencpll_d2);
+	if (ret)
+		mtk_v4l2_err("clk_set_parent fail %d", ret);
+
+	ret = clk_prepare_enable(pm->venc_lt_sel);
+	if (ret)
+		mtk_v4l2_err("venc_lt_sel fail %d", ret);
+
+	ret = clk_set_parent(pm->venc_lt_sel, pm->univpll1_d2);
+	if (ret)
+		mtk_v4l2_err("clk_set_parent fail %d", ret);
+
+	ret = mtk_smi_larb_get(pm->larbvenc);
+	if (ret)
+		mtk_v4l2_err("mtk_smi_larb_get larb3 fail %d\n", ret);
+
+	ret = mtk_smi_larb_get(pm->larbvenclt);
+	if (ret)
+		mtk_v4l2_err("mtk_smi_larb_get larb4 fail %d\n", ret);
+
+}
+
+void mtk_vcodec_enc_clock_off(struct mtk_vcodec_pm *pm)
+{
+	mtk_smi_larb_put(pm->larbvenc);
+	mtk_smi_larb_put(pm->larbvenclt);
+	clk_disable_unprepare(pm->venc_lt_sel);
+	clk_disable_unprepare(pm->venc_sel);
+}
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.c
new file mode 100644
index 0000000..919b949
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.c
@@ -0,0 +1,102 @@
+/*
+* Copyright (c) 2015 MediaTek Inc.
+* Author: Tiffany Lin <tiffany.lin@mediatek.com>
+*
+* This program is free software; you can redistribute it and/or modify
+* it under the terms of the GNU General Public License version 2 as
+* published by the Free Software Foundation.
+*
+* This program is distributed in the hope that it will be useful,
+* but WITHOUT ANY WARRANTY; without even the implied warranty of
+* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+* GNU General Public License for more details.
+*/
+
+#include <linux/errno.h>
+#include <linux/wait.h>
+
+#include "mtk_vcodec_drv.h"
+#include "mtk_vcodec_intr.h"
+#include "mtk_vcodec_util.h"
+
+void mtk_vcodec_clean_dev_int_flags(void *data)
+{
+	struct mtk_vcodec_dev *dev = (struct mtk_vcodec_dev *)data;
+
+	dev->int_cond = 0;
+	dev->int_type = 0;
+}
+
+int mtk_vcodec_wait_for_done_ctx(void *data, int command,
+				 unsigned int timeout_ms, int interrupt)
+{
+	wait_queue_head_t *waitqueue;
+	long timeout_jiff, ret;
+	int status = 0;
+	struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)data;
+
+	waitqueue = (wait_queue_head_t *)&ctx->queue;
+	timeout_jiff = msecs_to_jiffies(timeout_ms);
+	if (interrupt) {
+		ret = wait_event_interruptible_timeout(*waitqueue,
+				(ctx->int_cond &&
+				(ctx->int_type == command)),
+				timeout_jiff);
+	} else {
+		ret = wait_event_timeout(*waitqueue,
+				(ctx->int_cond &&
+				(ctx->int_type == command)),
+				 timeout_jiff);
+	}
+	if (0 == ret) {
+		status = -1;	/* timeout */
+		mtk_v4l2_err("[%d] cmd=%d, ctx->type=%d, wait_event_interruptible_timeout time=%ums out %d %d!",
+				ctx->idx, ctx->type, command, timeout_ms,
+				ctx->int_cond, ctx->int_type);
+	} else if (-ERESTARTSYS == ret) {
+		mtk_v4l2_err("[%d] cmd=%d, ctx->type=%d, wait_event_interruptible_timeout interrupted by a signal %d %d",
+				ctx->idx, ctx->type, command, ctx->int_cond,
+				ctx->int_type);
+		status = -1;
+	}
+
+	ctx->int_cond = 0;
+	ctx->int_type = 0;
+
+	return status;
+}
+
+int mtk_vcodec_wait_for_done_dev(void *data, int command,
+				 unsigned int timeout, int interrupt)
+{
+	wait_queue_head_t *waitqueue;
+	long timeout_jiff, ret;
+	int status = 0;
+	struct mtk_vcodec_dev *dev = (struct mtk_vcodec_dev *)data;
+
+	waitqueue = (wait_queue_head_t *)&dev->queue;
+	timeout_jiff = msecs_to_jiffies(timeout);
+	if (interrupt) {
+		ret = wait_event_interruptible_timeout(*waitqueue,
+				(dev->int_cond &&
+				(dev->int_type == command)),
+				timeout_jiff);
+	} else {
+		ret = wait_event_timeout(*waitqueue,
+				(dev->int_cond &&
+				(dev->int_type == command)),
+				timeout_jiff);
+	}
+	if (0 == ret) {
+		status = -1;	/* timeout */
+		mtk_v4l2_err("wait_event_interruptible_timeout time=%lu out %d %d!",
+				timeout_jiff, dev->int_cond, dev->int_type);
+	} else if (-ERESTARTSYS == ret) {
+		mtk_v4l2_err("wait_event_interruptible_timeout interrupted by a signal %d %d",
+				dev->int_cond, dev->int_type);
+		status = -1;
+	}
+	dev->int_cond = 0;
+	dev->int_type = 0;
+	return status;
+}
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.h
new file mode 100644
index 0000000..e9b7f94
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.h
@@ -0,0 +1,29 @@
+/*
+* Copyright (c) 2015 MediaTek Inc.
+* Author: Tiffany Lin <tiffany.lin@mediatek.com>
+*
+* This program is free software; you can redistribute it and/or modify
+* it under the terms of the GNU General Public License version 2 as
+* published by the Free Software Foundation.
+*
+* This program is distributed in the hope that it will be useful,
+* but WITHOUT ANY WARRANTY; without even the implied warranty of
+* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+* GNU General Public License for more details.
+*/
+
+#ifndef _MTK_VCODEC_INTR_H_
+#define _MTK_VCODEC_INTR_H_
+
+#define MTK_INST_IRQ_RECEIVED		0x1
+#define MTK_INST_WORK_THREAD_ABORT_DONE	0x2
+
+/* timeout is ms */
+int mtk_vcodec_wait_for_done_ctx(void *data, int command, unsigned int timeout,
+				 int interrupt);
+int mtk_vcodec_wait_for_done_dev(void *data, int command, unsigned int timeout,
+				 int interrupt);
+
+void mtk_vcodec_clean_dev_int_flags(void *data);
+
+#endif /* _MTK_VCODEC_INTR_H_ */
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_pm.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_pm.h
new file mode 100644
index 0000000..fdadec9
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_pm.h
@@ -0,0 +1,26 @@
+/*
+* Copyright (c) 2015 MediaTek Inc.
+* Author: Tiffany Lin <tiffany.lin@mediatek.com>
+*
+* This program is free software; you can redistribute it and/or modify
+* it under the terms of the GNU General Public License version 2 as
+* published by the Free Software Foundation.
+*
+* This program is distributed in the hope that it will be useful,
+* but WITHOUT ANY WARRANTY; without even the implied warranty of
+* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+* GNU General Public License for more details.
+*/
+
+#ifndef _MTK_VCODEC_PM_H_
+#define _MTK_VCODEC_PM_H_
+
+#include "mtk_vcodec_drv.h"
+
+int mtk_vcodec_init_enc_pm(struct mtk_vcodec_dev *dev);
+void mtk_vcodec_release_enc_pm(struct mtk_vcodec_dev *dev);
+
+void mtk_vcodec_enc_clock_on(struct mtk_vcodec_pm *pm);
+void mtk_vcodec_enc_clock_off(struct mtk_vcodec_pm *pm);
+
+#endif /* _MTK_VCODEC_PM_H_ */
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c
new file mode 100644
index 0000000..3fede8d
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c
@@ -0,0 +1,106 @@
+/*
+* Copyright (c) 2015 MediaTek Inc.
+* Author: PC Chen <pc.chen@mediatek.com>
+*         Tiffany Lin <tiffany.lin@mediatek.com>
+*
+* This program is free software; you can redistribute it and/or modify
+* it under the terms of the GNU General Public License version 2 as
+* published by the Free Software Foundation.
+*
+* This program is distributed in the hope that it will be useful,
+* but WITHOUT ANY WARRANTY; without even the implied warranty of
+* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+* GNU General Public License for more details.
+*/
+
+#include <linux/module.h>
+
+#include "mtk_vcodec_drv.h"
+#include "mtk_vcodec_util.h"
+#include "mtk_vpu.h"
+
+bool mtk_vcodec_dbg = false;
+int mtk_v4l2_dbg_level = 0;
+
+module_param(mtk_v4l2_dbg_level, int, S_IRUGO | S_IWUSR);
+module_param(mtk_vcodec_dbg, bool, S_IRUGO | S_IWUSR);
+
+void __iomem *mtk_vcodec_get_reg_addr(void *data, unsigned int reg_idx)
+{
+	struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)data;
+
+	if (!data || reg_idx >= NUM_MAX_VCODEC_REG_BASE) {
+		mtk_v4l2_err("Invalid arguments");
+		return NULL;
+	}
+	return ctx->dev->reg_base[reg_idx];
+}
+
+int mtk_vcodec_mem_alloc(void *data, struct mtk_vcodec_mem *mem)
+{
+	unsigned long size = mem->size;
+	struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)data;
+	struct device *dev = &ctx->dev->plat_dev->dev;
+
+	mem->va = dma_alloc_coherent(dev, size, &mem->dma_addr, GFP_KERNEL);
+
+	if (!mem->va) {
+		mtk_v4l2_err("%s dma_alloc size=%ld failed!", dev_name(dev),
+			       size);
+		return -ENOMEM;
+	}
+
+	memset(mem->va, 0, size);
+
+	mtk_v4l2_debug(3, "[%d]  - va      = %p", ctx->idx, mem->va);
+	mtk_v4l2_debug(3, "[%d]  - dma     = 0x%lx", ctx->idx,
+			 (unsigned long)mem->dma_addr);
+	mtk_v4l2_debug(3, "[%d]    size = 0x%lx", ctx->idx, size);
+
+	return 0;
+}
+
+void mtk_vcodec_mem_free(void *data, struct mtk_vcodec_mem *mem)
+{
+	unsigned long size = mem->size;
+	struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)data;
+	struct device *dev = &ctx->dev->plat_dev->dev;
+
+	dma_free_coherent(dev, size, mem->va, mem->dma_addr);
+	mem->va = NULL;
+
+	mtk_v4l2_debug(3, "[%d]  - va      = %p", ctx->idx, mem->va);
+	mtk_v4l2_debug(3, "[%d]  - dma     = 0x%lx", ctx->idx,
+			 (unsigned long)mem->dma_addr);
+	mtk_v4l2_debug(3, "[%d]    size = 0x%lx", ctx->idx, size);
+}
+
+int mtk_vcodec_get_ctx_id(void *data)
+{
+	struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)data;
+
+	if (!ctx)
+		return -1;
+
+	return ctx->idx;
+}
+
+struct platform_device *mtk_vcodec_get_plat_dev(void *data)
+{
+	struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)data;
+
+	if (!ctx)
+		return NULL;
+
+	return vpu_get_plat_device(ctx->dev->plat_dev);
+}
+
+void mtk_vcodec_fmt2str(u32 fmt, char *str)
+{
+	char a = fmt & 0xFF;
+	char b = (fmt >> 8) & 0xFF;
+	char c = (fmt >> 16) & 0xFF;
+	char d = (fmt >> 24) & 0xFF;
+
+	sprintf(str, "%c%c%c%c", a, b, c, d);
+}
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
new file mode 100644
index 0000000..47016ae
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
@@ -0,0 +1,85 @@
+/*
+* Copyright (c) 2015 MediaTek Inc.
+* Author: PC Chen <pc.chen@mediatek.com>
+*         Tiffany Lin <tiffany.lin@mediatek.com>
+*
+* This program is free software; you can redistribute it and/or modify
+* it under the terms of the GNU General Public License version 2 as
+* published by the Free Software Foundation.
+*
+* This program is distributed in the hope that it will be useful,
+* but WITHOUT ANY WARRANTY; without even the implied warranty of
+* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+* GNU General Public License for more details.
+*/
+
+#ifndef _MTK_VCODEC_UTIL_H_
+#define _MTK_VCODEC_UTIL_H_
+
+#include <linux/types.h>
+#include <linux/dma-direction.h>
+
+struct mtk_vcodec_mem {
+	size_t size;
+	void *va;
+	dma_addr_t dma_addr;
+};
+
+extern int mtk_v4l2_dbg_level;
+extern bool mtk_vcodec_dbg;
+
+#define DEBUG 	1
+
+#if defined(DEBUG)
+
+#define mtk_v4l2_debug(level, fmt, args...)				 \
+	do {								 \
+		if (mtk_v4l2_dbg_level >= level)			 \
+			pr_info("[MTK_V4L2] level=%d %s(),%d: " fmt "\n",\
+				level, __func__, __LINE__, ##args);	 \
+	} while (0)
+
+#define mtk_v4l2_err(fmt, args...)                \
+	pr_err("[MTK_V4L2][ERROR] %s:%d: " fmt "\n", __func__, __LINE__, \
+	       ##args)
+
+
+#define mtk_v4l2_debug_enter()  mtk_v4l2_debug(3, "+\n")
+#define mtk_v4l2_debug_leave()  mtk_v4l2_debug(3, "-\n")
+
+#define mtk_vcodec_debug(h, fmt, args...)				\
+	do {								\
+		if (mtk_vcodec_dbg)					\
+			pr_info("[MTK_VCODEC][%d]: %s() " fmt "\n",	\
+				((struct mtk_vcodec_ctx *)h->ctx)->idx, \
+				__func__, ##args);			\
+	} while (0)
+
+#define mtk_vcodec_err(h, fmt, args...)					\
+	pr_err("[MTK_VCODEC][ERROR][%d]: %s() " fmt "\n",		\
+	       ((struct mtk_vcodec_ctx *)h->ctx)->idx, __func__, ##args)
+
+#define mtk_vcodec_debug_enter(h)  mtk_vcodec_debug(h, "+\n")
+#define mtk_vcodec_debug_leave(h)  mtk_vcodec_debug(h, "-\n")
+
+#else
+
+#define mtk_v4l2_debug(level, fmt, args...)
+#define mtk_v4l2_err(fmt, args...)
+#define mtk_v4l2_debug_enter()
+#define mtk_v4l2_debug_leave()
+
+#define mtk_vcodec_debug(h, fmt, args...)
+#define mtk_vcodec_err(h, fmt, args...)
+#define mtk_vcodec_debug_enter(h)
+#define mtk_vcodec_debug_leave(h)
+
+#endif
+
+void __iomem *mtk_vcodec_get_reg_addr(void *data, unsigned int reg_idx);
+int mtk_vcodec_mem_alloc(void *data, struct mtk_vcodec_mem *mem);
+void mtk_vcodec_mem_free(void *data, struct mtk_vcodec_mem *mem);
+int mtk_vcodec_get_ctx_id(void *data);
+struct platform_device *mtk_vcodec_get_plat_dev(void *data);
+void mtk_vcodec_fmt2str(u32 fmt, char *str);
+#endif /* _MTK_VCODEC_UTIL_H_ */
diff --git a/drivers/media/platform/mtk-vcodec/venc_drv_base.h b/drivers/media/platform/mtk-vcodec/venc_drv_base.h
new file mode 100644
index 0000000..ed9cbf0
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/venc_drv_base.h
@@ -0,0 +1,62 @@
+/*
+ * Copyright (c) 2015 MediaTek Inc.
+ * Author: Daniel Hsiao <daniel.hsiao@mediatek.com>
+ *         Jungchang Tsao <jungchang.tsao@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef _VENC_DRV_BASE_
+#define _VENC_DRV_BASE_
+
+#include "mtk_vcodec_drv.h"
+
+#include "venc_drv_if.h"
+
+struct venc_common_if {
+	/**
+	 * (*init)() - initialize driver
+	 * @ctx:	[in] mtk v4l2 context
+	 * @handle: [out] driver handle
+	 */
+	int (*init)(struct mtk_vcodec_ctx *ctx, unsigned long *handle);
+
+	/**
+	 * (*encode)() - trigger encode
+	 * @handle: [in] driver handle
+	 * @opt: [in] encode option
+	 * @frm_buf: [in] frame buffer to store input frame
+	 * @bs_buf: [in] bitstream buffer to store output bitstream
+	 * @result: [out] encode result
+	 */
+	int (*encode)(unsigned long handle, enum venc_start_opt opt,
+		      struct venc_frm_buf *frm_buf,
+		      struct mtk_vcodec_mem *bs_buf,
+		      struct venc_done_result *result);
+
+	/**
+	 * (*set_param)() - set driver's parameter
+	 * @handle: [in] driver handle
+	 * @type: [in] parameter type
+	 * @in: [in] buffer to store the parameter
+	 */
+	int (*set_param)(unsigned long handle, enum venc_set_param_type type,
+			 void *in);
+
+	/**
+	 * (*deinit)() - deinitialize driver.
+	 * @handle: [in] driver handle
+	 */
+	int (*deinit)(unsigned long handle);
+};
+
+
+#endif
diff --git a/drivers/media/platform/mtk-vcodec/venc_drv_if.c b/drivers/media/platform/mtk-vcodec/venc_drv_if.c
new file mode 100644
index 0000000..daa8e93
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/venc_drv_if.c
@@ -0,0 +1,100 @@
+/*
+ * Copyright (c) 2015 MediaTek Inc.
+ * Author: Daniel Hsiao <daniel.hsiao@mediatek.com>
+ *         Jungchang Tsao <jungchang.tsao@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/interrupt.h>
+#include <linux/kernel.h>
+#include <linux/slab.h>
+
+#include "venc_drv_if.h"
+#include "mtk_vcodec_enc.h"
+#include "mtk_vcodec_pm.h"
+#include "mtk_vpu.h"
+
+#include "venc_drv_base.h"
+
+int venc_if_create(struct mtk_vcodec_ctx *ctx, unsigned int fourcc)
+{
+	char str[10];
+	int ret = 0;
+
+	mtk_vcodec_fmt2str(fourcc, str);
+
+	switch (fourcc) {
+	case V4L2_PIX_FMT_VP8:
+	case V4L2_PIX_FMT_H264:
+	default:
+		return -EINVAL;
+	}
+
+	mtk_venc_lock(ctx);
+	mtk_vcodec_enc_clock_on(&ctx->dev->pm);
+	ret = ctx->enc_if->init(ctx, (unsigned long *)&ctx->drv_handle);
+	mtk_vcodec_enc_clock_off(&ctx->dev->pm);
+	mtk_venc_unlock(ctx);
+
+	return ret;
+
+}
+
+
+int venc_if_set_param(struct mtk_vcodec_ctx *ctx,
+		      enum venc_set_param_type type, void *in)
+{
+	int ret = 0;
+
+	mtk_venc_lock(ctx);
+	mtk_vcodec_enc_clock_on(&ctx->dev->pm);
+	ret = ctx->enc_if->set_param(ctx->drv_handle, type, in);
+	mtk_vcodec_enc_clock_off(&ctx->dev->pm);
+	mtk_venc_unlock(ctx);
+
+	return ret;
+}
+
+int venc_if_encode(struct mtk_vcodec_ctx *ctx,
+		   enum venc_start_opt opt, struct venc_frm_buf *frm_buf,
+		   struct mtk_vcodec_mem *bs_buf,
+		   struct venc_done_result *result)
+{
+	int ret = 0;
+	
+	mtk_venc_lock(ctx);
+	mtk_vcodec_enc_clock_on(&ctx->dev->pm);
+	ret = ctx->enc_if->encode(ctx->drv_handle, opt, frm_buf, bs_buf, result);
+	mtk_vcodec_enc_clock_off(&ctx->dev->pm);
+	mtk_venc_unlock(ctx);
+	
+	return ret;
+}
+
+int venc_if_release(struct mtk_vcodec_ctx *ctx)
+{
+	int ret = 0;
+
+	if(ctx->drv_handle == 0)
+		return 0;
+
+	mtk_venc_lock(ctx);
+	mtk_vcodec_enc_clock_on(&ctx->dev->pm);
+	ret = ctx->enc_if->deinit(ctx->drv_handle);
+	mtk_vcodec_enc_clock_off(&ctx->dev->pm);
+	mtk_venc_unlock(ctx);
+
+	ctx->drv_handle = 0;
+
+	return ret;
+}
+
diff --git a/drivers/media/platform/mtk-vcodec/venc_drv_if.h b/drivers/media/platform/mtk-vcodec/venc_drv_if.h
new file mode 100644
index 0000000..a387011
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/venc_drv_if.h
@@ -0,0 +1,175 @@
+/*
+ * Copyright (c) 2015 MediaTek Inc.
+ * Author: Daniel Hsiao <daniel.hsiao@mediatek.com>
+ *         Jungchang Tsao <jungchang.tsao@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef _VENC_DRV_IF_H_
+#define _VENC_DRV_IF_H_
+
+#include "mtk_vcodec_drv.h"
+#include "mtk_vcodec_util.h"
+
+/*
+ * enum venc_yuv_fmt - The type of input yuv format
+ * (VPU related: If you change the order, you must also update the VPU codes.)
+ * @VENC_YUV_FORMAT_420: 420 YUV format
+ * @VENC_YUV_FORMAT_YV12: YV12 YUV format
+ * @VENC_YUV_FORMAT_NV12: NV12 YUV format
+ * @VENC_YUV_FORMAT_NV21: NV21 YUV format
+ */
+enum venc_yuv_fmt {
+	VENC_YUV_FORMAT_420 = 3,
+	VENC_YUV_FORMAT_YV12 = 5,
+	VENC_YUV_FORMAT_NV12 = 6,
+	VENC_YUV_FORMAT_NV21 = 7,
+};
+
+/*
+ * enum venc_start_opt - encode frame option used in venc_if_encode()
+ * @VENC_START_OPT_ENCODE_SEQUENCE_HEADER: encode SPS/PPS for H264
+ * @VENC_START_OPT_ENCODE_FRAME: encode normal frame
+ */
+enum venc_start_opt {
+	VENC_START_OPT_ENCODE_SEQUENCE_HEADER,
+	VENC_START_OPT_ENCODE_FRAME,
+};
+
+/*
+ * enum venc_drv_msg - The type of encode frame status used in venc_if_encode()
+ * @VENC_MESSAGE_OK: encode ok
+ * @VENC_MESSAGE_ERR: encode error
+ */
+enum venc_drv_msg {
+	VENC_MESSAGE_OK,
+	VENC_MESSAGE_ERR,
+};
+
+/*
+ * enum venc_set_param_type - The type of set parameter used in venc_if_set_param()
+ * (VPU related: If you change the order, you must also update the VPU codes.)
+ * @VENC_SET_PARAM_ENC: set encoder parameters
+ * @VENC_SET_PARAM_FORCE_INTRA: set force intra frame
+ * @VENC_SET_PARAM_ADJUST_BITRATE: set to adjust bitrate (in bps)
+ * @VENC_SET_PARAM_ADJUST_FRAMERATE: set frame rate
+ * @VENC_SET_PARAM_I_FRAME_INTERVAL: set I frame interval
+ * @VENC_SET_PARAM_SKIP_FRAME: set H264 skip one frame
+ * @VENC_SET_PARAM_PREPEND_HEADER: set H264 prepend SPS/PPS before IDR
+ * @VENC_SET_PARAM_TS_MODE: set VP8 temporal scalability mode
+ */
+enum venc_set_param_type {
+	VENC_SET_PARAM_ENC,
+	VENC_SET_PARAM_FORCE_INTRA,
+	VENC_SET_PARAM_ADJUST_BITRATE,
+	VENC_SET_PARAM_ADJUST_FRAMERATE,
+	VENC_SET_PARAM_I_FRAME_INTERVAL,
+	VENC_SET_PARAM_SKIP_FRAME,
+	VENC_SET_PARAM_PREPEND_HEADER,
+	VENC_SET_PARAM_TS_MODE,
+};
+
+/*
+ * struct venc_enc_prm - encoder settings for VENC_SET_PARAM_ENC used in venc_if_set_param()
+ * @input_fourcc: input fourcc
+ * @h264_profile: V4L2 defined H.264 profile
+ * @h264_level: V4L2 defined H.264 level
+ * @width: image width
+ * @height: image height
+ * @buf_width: buffer width
+ * @buf_height: buffer height
+ * @frm_rate: frame rate
+ * @intra_period: intra frame period
+ * @bitrate: target bitrate in kbps
+ */
+struct venc_enc_prm {
+	enum venc_yuv_fmt input_fourcc;
+	unsigned int h264_profile;
+	unsigned int h264_level;
+	unsigned int width;
+	unsigned int height;
+	unsigned int buf_width;
+	unsigned int buf_height;
+	unsigned int frm_rate;
+	unsigned int intra_period;
+	unsigned int bitrate;
+	unsigned int sizeimage[MTK_VCODEC_MAX_PLANES];
+};
+
+/*
+ * struct venc_frm_buf - frame buffer information used in venc_if_encode()
+ * @fb_addr: plane 0 frame buffer address
+ * @fb_addr1: plane 1 frame buffer address
+ * @fb_addr2: plane 2 frame buffer address
+ */
+struct venc_frm_buf {
+	struct mtk_vcodec_mem fb_addr;
+	struct mtk_vcodec_mem fb_addr1;
+	struct mtk_vcodec_mem fb_addr2;
+};
+
+/*
+ * struct venc_done_result - This is return information used in venc_if_encode()
+ * @msg: message, such as success or error code
+ * @bs_size: output bitstream size
+ * @is_key_frm: output is key frame or not
+ */
+struct venc_done_result {
+	enum venc_drv_msg msg;
+	unsigned int bs_size;
+	bool is_key_frm;
+};
+
+/*
+ * venc_if_create - Create the driver handle
+ * @ctx: device context
+ * @fourcc: encoder output format
+ * @handle: driver handle
+ * Return: 0 if creating handle successfully, otherwise it is failed.
+ */
+int venc_if_create(struct mtk_vcodec_ctx *ctx, unsigned int fourcc);
+
+/*
+ * venc_if_release - Release the driver handle
+ * @handle: driver handle
+ * Return: 0 if releasing handle successfully, otherwise it is failed.
+ */
+int venc_if_release(struct mtk_vcodec_ctx *ctx);
+
+/*
+ * venc_if_set_param - Set parameter to driver
+ * @handle: driver handle
+ * @type: set type
+ * @in: input parameter
+ * @out: output parameter
+ * Return: 0 if setting param successfully, otherwise it is failed.
+ */
+int venc_if_set_param(struct mtk_vcodec_ctx *ctx,
+		      enum venc_set_param_type type,
+		      void *in);
+
+/*
+ * venc_if_encode - Encode frame
+ * @handle: driver handle
+ * @opt: encode frame option
+ * @frm_buf: input frame buffer information
+ * @bs_buf: output bitstream buffer infomraiton
+ * @result: encode result
+ * Return: 0 if encoding frame successfully, otherwise it is failed.
+ */
+int venc_if_encode(struct mtk_vcodec_ctx *ctx,
+		   enum venc_start_opt opt,
+		   struct venc_frm_buf *frm_buf,
+		   struct mtk_vcodec_mem *bs_buf,
+		   struct venc_done_result *result);
+
+#endif /* _VENC_DRV_IF_H_ */
diff --git a/drivers/media/platform/mtk-vcodec/venc_ipi_msg.h b/drivers/media/platform/mtk-vcodec/venc_ipi_msg.h
new file mode 100644
index 0000000..a345b98
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/venc_ipi_msg.h
@@ -0,0 +1,212 @@
+/*
+ * Copyright (c) 2015 MediaTek Inc.
+ * Author: Jungchang Tsao <jungchang.tsao@mediatek.com>
+ *         Daniel Hsiao <daniel.hsiao@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef _VENC_IPI_MSG_H_
+#define _VENC_IPI_MSG_H_
+
+#define IPIMSG_H264_ENC_ID 0x100
+#define IPIMSG_VP8_ENC_ID 0x200
+
+#define AP_IPIMSG_VENC_BASE 0x20000
+#define VPU_IPIMSG_VENC_BASE 0x30000
+
+/**
+ * enum venc_ipi_msg_id - message id between AP and VPU
+ * (ipi stands for inter-processor interrupt)
+ * @AP_IPIMSG_XXX:		AP to VPU cmd message id
+ * @VPU_IPIMSG_XXX_DONE:	VPU ack AP cmd message id
+ */
+enum venc_ipi_msg_id {
+	AP_IPIMSG_H264_ENC_INIT = AP_IPIMSG_VENC_BASE +
+				  IPIMSG_H264_ENC_ID,
+	AP_IPIMSG_H264_ENC_SET_PARAM,
+	AP_IPIMSG_H264_ENC_ENCODE,
+	AP_IPIMSG_H264_ENC_DEINIT,
+
+	AP_IPIMSG_VP8_ENC_INIT = AP_IPIMSG_VENC_BASE +
+				 IPIMSG_VP8_ENC_ID,
+	AP_IPIMSG_VP8_ENC_SET_PARAM,
+	AP_IPIMSG_VP8_ENC_ENCODE,
+	AP_IPIMSG_VP8_ENC_DEINIT,
+
+	VPU_IPIMSG_H264_ENC_INIT_DONE = VPU_IPIMSG_VENC_BASE +
+					IPIMSG_H264_ENC_ID,
+	VPU_IPIMSG_H264_ENC_SET_PARAM_DONE,
+	VPU_IPIMSG_H264_ENC_ENCODE_DONE,
+	VPU_IPIMSG_H264_ENC_DEINIT_DONE,
+
+	VPU_IPIMSG_VP8_ENC_INIT_DONE = VPU_IPIMSG_VENC_BASE +
+				       IPIMSG_VP8_ENC_ID,
+	VPU_IPIMSG_VP8_ENC_SET_PARAM_DONE,
+	VPU_IPIMSG_VP8_ENC_ENCODE_DONE,
+	VPU_IPIMSG_VP8_ENC_DEINIT_DONE,
+};
+
+/**
+ * struct venc_ap_ipi_msg_init - AP to VPU init cmd structure
+ * @msg_id:	message id (AP_IPIMSG_XXX_ENC_INIT)
+ * @venc_inst:	AP encoder instance (struct venc_vp8_handle/venc_h264_handle *)
+ */
+struct venc_ap_ipi_msg_init {
+	uint32_t msg_id;
+	uint32_t reserved;
+	uint64_t venc_inst;
+};
+
+/**
+ * struct venc_ap_ipi_msg_set_param - AP to VPU set_param cmd structure
+ * @msg_id:	message id (AP_IPIMSG_XXX_ENC_SET_PARAM)
+ * @inst_id:	VPU encoder instance id (struct venc_vp8_vpu_drv/venc_h264_vpu_drv *)
+ * @param_id:	parameter id (venc_set_param_type)
+ * @data_item:	number of items in the data array
+ * @data[8]:	data array to store the set parameters
+ */
+struct venc_ap_ipi_msg_set_param {
+	uint32_t msg_id;
+	uint32_t inst_id;
+	uint32_t param_id;
+	uint32_t data_item;
+	uint32_t data[8];
+};
+
+/**
+ * struct venc_ap_ipi_msg_enc - AP to VPU enc cmd structure
+ * @msg_id:	message id (AP_IPIMSG_XXX_ENC_ENCODE)
+ * @inst_id:	VPU encoder instance id (struct venc_vp8_vpu_drv/venc_h264_vpu_drv *)
+ * @bs_mode:	bitstream mode for h264
+ *		(H264_BS_MODE_SPS/H264_BS_MODE_PPS/H264_BS_MODE_FRAME)
+ * @input_addr:	pointer to input image buffer plane
+ * @bs_addr:	pointer to output bit stream buffer
+ * @bs_size:	bit stream buffer size
+ */
+struct venc_ap_ipi_msg_enc {
+	uint32_t msg_id;
+	uint32_t inst_id;
+	uint32_t bs_mode;
+	uint32_t input_addr[3];
+	uint32_t bs_addr;
+	uint32_t bs_size;
+};
+
+/**
+ * struct venc_ap_ipi_msg_deinit - AP to VPU deinit cmd structure
+ * @msg_id:	message id (AP_IPIMSG_XXX_ENC_DEINIT)
+ * @inst_id:	VPU encoder instance id (struct venc_vp8_vpu_drv/venc_h264_vpu_drv *)
+ */
+struct venc_ap_ipi_msg_deinit {
+	uint32_t msg_id;
+	uint32_t inst_id;
+};
+
+/**
+ * enum venc_ipi_msg_status - VPU ack AP cmd status
+ */
+enum venc_ipi_msg_status {
+	VENC_IPI_MSG_STATUS_OK,
+	VENC_IPI_MSG_STATUS_FAIL,
+};
+
+/**
+ * struct venc_vpu_ipi_msg_common - VPU ack AP cmd common structure
+ * @msg_id:	message id (VPU_IPIMSG_XXX_DONE)
+ * @status:	cmd status (venc_ipi_msg_status)
+ * @venc_inst:	AP encoder instance (struct venc_vp8_handle/venc_h264_handle *)
+ */
+struct venc_vpu_ipi_msg_common {
+	uint32_t msg_id;
+	uint32_t status;
+	uint64_t venc_inst;
+};
+
+/**
+ * struct venc_vpu_ipi_msg_init - VPU ack AP init cmd structure
+ * @msg_id:	message id (VPU_IPIMSG_XXX_ENC_SET_PARAM_DONE)
+ * @status:	cmd status (venc_ipi_msg_status)
+ * @venc_inst:	AP encoder instance (struct venc_vp8_handle/venc_h264_handle *)
+ * @inst_id:	VPU encoder instance id (struct venc_vp8_vpu_drv/venc_h264_vpu_drv *)
+ */
+struct venc_vpu_ipi_msg_init {
+	uint32_t msg_id;
+	uint32_t status;
+	uint64_t venc_inst;
+	uint32_t inst_id;
+	uint32_t reserved;
+};
+
+/**
+ * struct venc_vpu_ipi_msg_set_param - VPU ack AP set_param cmd structure
+ * @msg_id:	message id (VPU_IPIMSG_XXX_ENC_SET_PARAM_DONE)
+ * @status:	cmd status (venc_ipi_msg_status)
+ * @venc_inst:	AP encoder instance (struct venc_vp8_handle/venc_h264_handle *)
+ * @param_id:	parameter id (venc_set_param_type)
+ * @data_item:	number of items in the data array
+ * @data[6]:	data array to store the return result
+ */
+struct venc_vpu_ipi_msg_set_param {
+	uint32_t msg_id;
+	uint32_t status;
+	uint64_t venc_inst;
+	uint32_t param_id;
+	uint32_t data_item;
+	uint32_t data[6];
+};
+
+/**
+ * enum venc_ipi_msg_enc_state - Type of encode state
+ * VEN_IPI_MSG_ENC_STATE_FRAME:	one frame being encoded
+ * VEN_IPI_MSG_ENC_STATE_PART:	bit stream buffer full
+ * VEN_IPI_MSG_ENC_STATE_SKIP:	encoded skip frame
+ * VEN_IPI_MSG_ENC_STATE_ERROR:	encounter error
+ */
+enum venc_ipi_msg_enc_state {
+	VEN_IPI_MSG_ENC_STATE_FRAME,
+	VEN_IPI_MSG_ENC_STATE_PART,
+	VEN_IPI_MSG_ENC_STATE_SKIP,
+	VEN_IPI_MSG_ENC_STATE_ERROR,
+};
+
+/**
+ * struct venc_vpu_ipi_msg_enc - VPU ack AP enc cmd structure
+ * @msg_id:	message id (VPU_IPIMSG_XXX_ENC_ENCODE_DONE)
+ * @status:	cmd status (venc_ipi_msg_status)
+ * @venc_inst:	AP encoder instance (struct venc_vp8_handle/venc_h264_handle *)
+ * @state:	encode state (venc_ipi_msg_enc_state)
+ * @key_frame:	whether the encoded frame is key frame
+ * @bs_size:	encoded bitstream size
+ */
+struct venc_vpu_ipi_msg_enc {
+	uint32_t msg_id;
+	uint32_t status;
+	uint64_t venc_inst;
+	uint32_t state;
+	uint32_t key_frame;
+	uint32_t bs_size;
+	uint32_t reserved;
+};
+
+/**
+ * struct venc_vpu_ipi_msg_deinit - VPU ack AP deinit cmd structure
+ * @msg_id:   message id (VPU_IPIMSG_XXX_ENC_DEINIT_DONE)
+ * @status:   cmd status (venc_ipi_msg_status)
+ * @venc_inst:	AP encoder instance (struct venc_vp8_handle/venc_h264_handle *)
+ */
+struct venc_vpu_ipi_msg_deinit {
+	uint32_t msg_id;
+	uint32_t status;
+	uint64_t venc_inst;
+};
+
+#endif /* _VENC_IPI_MSG_H_ */
diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
old mode 100644
new mode 100755
index ee9d530..3ac35c4
--- a/include/uapi/linux/v4l2-controls.h
+++ b/include/uapi/linux/v4l2-controls.h
@@ -646,6 +646,10 @@ enum v4l2_mpeg_mfc51_video_force_frame_type {
 #define V4L2_CID_MPEG_MFC51_VIDEO_H264_NUM_REF_PIC_FOR_P		(V4L2_CID_MPEG_MFC51_BASE+54)
 
 
+#define V4L2_CID_MPEG_MTK_BASE					(V4L2_CTRL_CLASS_MPEG | 0x5500)
+#define V4L2_CID_MPEG_MTK_VIDEO_FRAME_SKIP_ENABLE		(V4L2_CID_MPEG_MTK_BASE+0)
+#define V4L2_CID_MPEG_MTK_VIDEO_FORCE_FRAME_TYPE_I_FRAME	(V4L2_CID_MPEG_MTK_BASE+1)
+
 /*  Camera class control IDs */
 
 #define V4L2_CID_CAMERA_CLASS_BASE 	(V4L2_CTRL_CLASS_CAMERA | 0x900)
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 102+ messages in thread

* [PATCH v4 5/8] [Media] vcodec: mediatek: Add Mediatek V4L2 Video Encoder Driver
@ 2016-02-04 11:35           ` Tiffany Lin
  0 siblings, 0 replies; 102+ messages in thread
From: Tiffany Lin @ 2016-02-04 11:35 UTC (permalink / raw)
  To: Hans Verkuil, daniel.thompson-QSEj5FYQhm4dnm+yROfE0A,
	Rob Herring, Mauro Carvalho Chehab, Matthias Brugger,
	Daniel Kurtz, Pawel Osciak
  Cc: Eddie Huang, Yingjoe Chen, devicetree-u79uwXL29TY76Z2rM5mHXA,
	linux-kernel-u79uwXL29TY76Z2rM5mHXA,
	linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
	linux-media-u79uwXL29TY76Z2rM5mHXA,
	linux-mediatek-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
	PoChun.Lin-NuS5LvNUpcJWk0Htik3J/w, Andrew-CT Chen, Tiffany Lin

From: Andrew-CT Chen <andrew-ct.chen-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>

Add v4l2 layer encoder driver for MT8173

Signed-off-by: Tiffany Lin <tiffany.lin-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
---
 drivers/media/platform/Kconfig                     |   11 +
 drivers/media/platform/Makefile                    |    2 +
 drivers/media/platform/mtk-vcodec/Makefile         |    8 +
 drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h |  388 ++++++
 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c | 1380 ++++++++++++++++++++
 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h |   46 +
 .../media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c |  476 +++++++
 .../media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c  |  132 ++
 .../media/platform/mtk-vcodec/mtk_vcodec_intr.c    |  102 ++
 .../media/platform/mtk-vcodec/mtk_vcodec_intr.h    |   29 +
 drivers/media/platform/mtk-vcodec/mtk_vcodec_pm.h  |   26 +
 .../media/platform/mtk-vcodec/mtk_vcodec_util.c    |  106 ++
 .../media/platform/mtk-vcodec/mtk_vcodec_util.h    |   85 ++
 drivers/media/platform/mtk-vcodec/venc_drv_base.h  |   62 +
 drivers/media/platform/mtk-vcodec/venc_drv_if.c    |  100 ++
 drivers/media/platform/mtk-vcodec/venc_drv_if.h    |  175 +++
 drivers/media/platform/mtk-vcodec/venc_ipi_msg.h   |  212 +++
 include/uapi/linux/v4l2-controls.h                 |    4 +
 18 files changed, 3344 insertions(+)
 create mode 100644 drivers/media/platform/mtk-vcodec/Makefile
 create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
 create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
 create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h
 create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
 create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
 create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.c
 create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.h
 create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_pm.h
 create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c
 create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
 create mode 100644 drivers/media/platform/mtk-vcodec/venc_drv_base.h
 create mode 100644 drivers/media/platform/mtk-vcodec/venc_drv_if.c
 create mode 100644 drivers/media/platform/mtk-vcodec/venc_drv_if.h
 create mode 100644 drivers/media/platform/mtk-vcodec/venc_ipi_msg.h
 mode change 100644 => 100755 include/uapi/linux/v4l2-controls.h

diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
index ba812d6..3e831c5 100644
--- a/drivers/media/platform/Kconfig
+++ b/drivers/media/platform/Kconfig
@@ -157,6 +157,17 @@ config VIDEO_MEDIATEK_VPU
 	    codec embedded in new Mediatek's SOCs. It is able
 	    to handle video decoding/encoding in a range of formats.
 
+config VIDEO_MEDIATEK_VCODEC
+        tristate "Mediatek Video Codec driver"
+        depends on VIDEO_DEV && VIDEO_V4L2
+        depends on ARCH_MEDIATEK || COMPILE_TEST
+        select VIDEOBUF2_DMA_CONTIG
+        select V4L2_MEM2MEM_DEV
+        select MEDIATEK_VPU
+        default n
+        ---help---
+            Mediatek video codec driver for V4L2
+
 config VIDEO_MEM2MEM_DEINTERLACE
 	tristate "Deinterlace support"
 	depends on VIDEO_DEV && VIDEO_V4L2 && DMA_ENGINE
diff --git a/drivers/media/platform/Makefile b/drivers/media/platform/Makefile
index e5b19c6..510e06b 100644
--- a/drivers/media/platform/Makefile
+++ b/drivers/media/platform/Makefile
@@ -57,3 +57,5 @@ obj-$(CONFIG_VIDEO_XILINX)		+= xilinx/
 ccflags-y += -I$(srctree)/drivers/media/i2c
 
 obj-$(CONFIG_VIDEO_MEDIATEK_VPU)	+= mtk-vpu/
+
+obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC)	+= mtk-vcodec/
diff --git a/drivers/media/platform/mtk-vcodec/Makefile b/drivers/media/platform/mtk-vcodec/Makefile
new file mode 100644
index 0000000..ce38689
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/Makefile
@@ -0,0 +1,8 @@
+obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += mtk_vcodec_intr.o \
+				       mtk_vcodec_util.o \
+				       mtk_vcodec_enc_drv.o \
+				       mtk_vcodec_enc.o \
+				       mtk_vcodec_enc_pm.o \
+				       venc_drv_if.o
+
+ccflags-y += -I$(srctree)/drivers/media/platform/mtk-vpu
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
new file mode 100644
index 0000000..9da2818
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
@@ -0,0 +1,388 @@
+/*
+* Copyright (c) 2015 MediaTek Inc.
+* Author: PC Chen <pc.chen-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
+*         Tiffany Lin <tiffany.lin-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
+*
+* This program is free software; you can redistribute it and/or modify
+* it under the terms of the GNU General Public License version 2 as
+* published by the Free Software Foundation.
+*
+* This program is distributed in the hope that it will be useful,
+* but WITHOUT ANY WARRANTY; without even the implied warranty of
+* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+* GNU General Public License for more details.
+*/
+
+#ifndef _MTK_VCODEC_DRV_H_
+#define _MTK_VCODEC_DRV_H_
+
+#include <linux/platform_device.h>
+#include <linux/videodev2.h>
+#include <media/v4l2-ctrls.h>
+#include <media/v4l2-device.h>
+#include <media/v4l2-ioctl.h>
+#include <media/videobuf2-core.h>
+
+#include "mtk_vcodec_util.h"
+
+#define MTK_VCODEC_MAX_INSTANCES		10
+#define MTK_VCODEC_MAX_ENCODER_INSTANCES	3
+
+#define MTK_VCODEC_MAX_FRAME_SIZE	0x800000
+#define MTK_VIDEO_MAX_FRAME		32
+#define MTK_MAX_CTRLS			20
+
+#define MTK_VCODEC_DRV_NAME		"mtk_vcodec_drv"
+#define MTK_VCODEC_ENC_NAME		"mtk-vcodec-enc"
+#define MTK_PLATFORM_STR		"platform:mt8173"
+
+#define MTK_VENC_IRQ_STATUS_SPS          0x1
+#define MTK_VENC_IRQ_STATUS_PPS          0x2
+#define MTK_VENC_IRQ_STATUS_FRM          0x4
+#define MTK_VENC_IRQ_STATUS_DRAM         0x8
+#define MTK_VENC_IRQ_STATUS_PAUSE        0x10
+#define MTK_VENC_IRQ_STATUS_SWITCH       0x20
+
+#define MTK_VENC_IRQ_STATUS_OFFSET       0x05C
+#define MTK_VENC_IRQ_ACK_OFFSET          0x060
+
+#define MTK_VCODEC_MAX_PLANES		3
+
+#define VDEC_HW_ACTIVE	0x10
+#define VDEC_IRQ_CFG    0x11
+#define VDEC_IRQ_CLR    0x10
+
+#define VDEC_IRQ_CFG_REG	0xa4
+#define WAIT_INTR_TIMEOUT	1000
+
+/**
+ * enum mtk_hw_reg_idx - MTK hw register base index
+ */
+enum mtk_hw_reg_idx {
+	VDEC_SYS,
+	VDEC_MISC,
+	VDEC_LD,
+	VDEC_TOP,
+	VDEC_CM,
+	VDEC_AD,
+	VDEC_AV,
+	VDEC_PP,
+	VDEC_HWD,
+	VDEC_HWQ,
+	VDEC_HWB,
+	VDEC_HWG,
+	NUM_MAX_VDEC_REG_BASE,
+	VENC_SYS = NUM_MAX_VDEC_REG_BASE,
+	VENC_LT_SYS,
+	NUM_MAX_VCODEC_REG_BASE
+};
+
+/**
+ * enum mtk_instance_type - The type of an MTK Vcodec instance.
+ */
+enum mtk_instance_type {
+	MTK_INST_DECODER		= 0,
+	MTK_INST_ENCODER		= 1,
+};
+
+/**
+ * enum mtk_instance_state - The state of an MTK Vcodec instance.
+ * @MTK_STATE_FREE - default state when instance is created
+ * @MTK_STATE_INIT - vcodec instance is initialized
+ * @MTK_STATE_HEADER - vdec had sps/pps header parsed or venc
+ *			had sps/pps header encoded
+ * @MTK_STATE_FLUSH - vdec is flushing. Only used by decoder
+ * @MTK_STATE_RES_CHANGE - vdec detect resolution change.
+ * 			Only used by decoder
+ * @MTK_STATE_ABORT - vcodec should be aborted
+ */
+enum mtk_instance_state {
+	MTK_STATE_FREE = 0,
+	MTK_STATE_INIT = 1,
+	MTK_STATE_HEADER = 2,
+	MTK_STATE_FLUSH = 3,
+	MTK_STATE_RES_CHANGE = 4,
+	MTK_STATE_ABORT = 5,
+};
+
+/**
+ * struct mtk_param_change - General encoding parameters type
+ */
+enum mtk_encode_param {
+	MTK_ENCODE_PARAM_NONE = 0,
+	MTK_ENCODE_PARAM_BITRATE = (1 << 0),
+	MTK_ENCODE_PARAM_FRAMERATE = (1 << 1),
+	MTK_ENCODE_PARAM_INTRA_PERIOD = (1 << 2),
+	MTK_ENCODE_PARAM_FRAME_TYPE = (1 << 3),
+};
+
+/**
+ * enum mtk_fmt_type - Type of the pixelformat
+ * @MTK_FMT_FRAME - mtk vcodec raw frame
+ */
+enum mtk_fmt_type {
+	MTK_FMT_DEC		= 0,
+	MTK_FMT_ENC		= 1,
+	MTK_FMT_FRAME		= 2,
+};
+
+/**
+ * struct mtk_video_fmt - Structure used to store information about pixelformats
+ */
+struct mtk_video_fmt {
+	u32 fourcc;
+	enum mtk_fmt_type type;
+	u32 num_planes;
+};
+
+/**
+ * struct mtk_codec_framesizes - Structure used to store information about framesizes
+ */
+struct mtk_codec_framesizes {
+	u32 fourcc;
+	struct	v4l2_frmsize_stepwise	stepwise;
+};
+
+/**
+ * struct mtk_q_type - Type of queue
+ */
+enum mtk_q_type {
+	MTK_Q_DATA_SRC		= 0,
+	MTK_Q_DATA_DST		= 1,
+};
+
+/**
+ * struct mtk_q_data - Structure used to store information about queue
+ * @colorspace	reserved for encoder
+ * @field		reserved for encoder
+ */
+struct mtk_q_data {
+	unsigned int		width;
+	unsigned int		height;
+	enum v4l2_field		field;
+	enum v4l2_colorspace	colorspace;
+	unsigned int		bytesperline[MTK_VCODEC_MAX_PLANES];
+	unsigned int		sizeimage[MTK_VCODEC_MAX_PLANES];
+	struct mtk_video_fmt	*fmt;
+};
+
+/**
+ * struct mtk_enc_params - General encoding parameters
+ * @bitrate - target bitrate
+ * @num_b_frame - number of b frames between p-frame
+ * @rc_frame - frame based rate control
+ * @rc_mb - macroblock based rate control
+ * @seq_hdr_mode - H.264 sequence header is encoded separately or joined with the first frame
+ * @gop_size - group of picture size, it's used as the intra frame period
+ * @framerate_num - frame rate numerator
+ * @framerate_denom - frame rate denominator
+ * @h264_max_qp - Max value for H.264 quantization parameter
+ * @h264_profile - V4L2 defined H.264 profile
+ * @h264_level - V4L2 defined H.264 level
+ * @force_intra - force/insert intra frame
+ * @skip_frame - encode in skip frame mode that use minimum number of bits
+ */
+struct mtk_enc_params {
+	unsigned int	bitrate;
+	unsigned int	num_b_frame;
+	unsigned int	rc_frame;
+	unsigned int	rc_mb;
+	unsigned int	seq_hdr_mode;
+	unsigned int	gop_size;
+	unsigned int	framerate_num;
+	unsigned int	framerate_denom;
+	unsigned int	h264_max_qp;
+	unsigned int	h264_profile;
+	unsigned int	h264_level;
+	unsigned int	force_intra;
+};
+
+/**
+ * struct mtk_vcodec_pm - Power management data structure
+ */
+struct mtk_vcodec_pm {
+	struct clk	*vcodecpll;
+	struct clk	*univpll_d2;
+	struct clk	*clk_cci400_sel;
+	struct clk	*vdecpll;
+	struct clk	*vdec_sel;
+	struct clk	*vencpll_d2;
+	struct clk	*venc_sel;
+	struct clk	*univpll1_d2;
+	struct clk	*venc_lt_sel;
+	struct device	*larbvdec;
+	struct device	*larbvenc;
+	struct device	*larbvenclt;
+	struct device	*dev;
+	struct mtk_vcodec_dev *mtkdev;
+};
+
+
+/**
+ * struct mtk_vcodec_ctx - Context (instance) private data.
+ *
+ * @type:		type of the instance - decoder or encoder
+ * @dev:		pointer to the mtk_vcodec_dev of the device
+ * @fh:			struct v4l2_fh
+ * @m2m_ctx:		pointer to the v4l2_m2m_ctx of the context
+ * @q_data:		store information of input and output queue
+ *			of the context
+ * @idx:		index of the context that this structure describes
+ * @state:		state of the context
+ * @param_change:	encode parameters
+ * @enc_params:		encoding parameters
+ * @colorspace:
+ * @enc_if:		hoooked encoder driver interface
+ * @drv_handle:		driver handle for specific decode/encode instance
+ *
+ * @picinfo:		store width/height of image and buffer and planes' size for decoder
+ *			and encoder
+ * @dpb_count:		count of the DPB buffers required by MTK Vcodec hw
+ *
+ * @int_cond:		variable used by the waitqueue
+ * @int_type:		type of the last interrupt
+ * @queue:		waitqueue that can be used to wait for this context to
+ *			finish
+ * @irq_status:		irq status
+ *
+ * @ctrl_hdl:		handler for v4l2 framework
+ * @ctrls:		array of controls, used when adding controls to the
+ *			v4l2 control framework
+ *
+ * @encode_work:	worker for the encoding
+ */
+struct mtk_vcodec_ctx {
+	enum mtk_instance_type type;
+	struct mtk_vcodec_dev *dev;
+	struct v4l2_fh fh;
+	struct v4l2_m2m_ctx *m2m_ctx;
+	struct mtk_q_data q_data[2];
+	int idx;
+	enum mtk_instance_state state;
+	enum mtk_encode_param param_change;
+	struct mtk_enc_params enc_params;
+
+	struct venc_common_if *enc_if;
+	unsigned long drv_handle;
+
+
+	int int_cond;
+	int int_type;
+	wait_queue_head_t queue;
+	unsigned int irq_status;
+
+	struct v4l2_ctrl_handler ctrl_hdl;
+	struct v4l2_ctrl *ctrls[MTK_MAX_CTRLS];
+
+	struct work_struct encode_work;
+
+};
+
+/**
+ * struct mtk_vcodec_dev - driver data
+ * @v4l2_dev:		V4L2 device to register video devices for.
+ * @vfd_enc:		Video device for encoder.
+ *
+ * @m2m_dev_enc:	m2m device for encoder.
+ * @plat_dev:		platform device
+ * @alloc_ctx:		VB2 allocator context
+ *			(for allocations without kernel mapping).
+ * @ctx:		array of driver contexts
+ *
+ * @curr_ctx:		The context that is waiting for codec hardware
+ *
+ * @reg_base:		Mapped address of MTK Vcodec registers.
+ *
+ * @instance_mask:	used to mark which contexts are opened
+ * @num_instances:	counter of active MTK Vcodec instances
+ *
+ * @encode_workqueue:	encode work queue
+ *
+ * @int_cond:		used to identify interrupt condition happen
+ * @int_type:		used to identify what kind of interrupt condition happen
+ * @dev_mutex:		video_device lock
+ * @queue:		waitqueue for waiting for completion of device commands
+ *
+ * @enc_irq:		h264 encoder irq resource
+ * @enc_lt_irq:		vp8 encoder irq resource
+ *
+ * @enc_mutex:		encoder hardware lock.
+ *
+ * @pm:			power management control
+ * @dec_capability:	used to identify decode capability, ex: 4k
+ * @enc_capability:     used to identify encode capability
+ */
+struct mtk_vcodec_dev {
+	struct v4l2_device	v4l2_dev;
+	struct video_device	*vfd_enc;
+
+	struct v4l2_m2m_dev	*m2m_dev_enc;
+	struct platform_device	*plat_dev;
+	struct platform_device 	*vpu_plat_dev;
+	struct vb2_alloc_ctx	*alloc_ctx;
+	struct mtk_vcodec_ctx	*ctx[MTK_VCODEC_MAX_INSTANCES];
+	int curr_ctx;
+	void __iomem		*reg_base[NUM_MAX_VCODEC_REG_BASE];
+
+	unsigned long	instance_mask[BITS_TO_LONGS(MTK_VCODEC_MAX_INSTANCES)];
+	int			num_instances;
+
+	struct workqueue_struct *encode_workqueue;
+
+	int			int_cond;
+	int			int_type;
+	struct mutex		dev_mutex;
+	wait_queue_head_t	queue;
+
+	int			enc_irq;
+	int			enc_lt_irq;
+
+	struct mutex		enc_mutex;
+
+	struct mtk_vcodec_pm	pm;
+	unsigned int		dec_capability;
+	unsigned int		enc_capability;
+};
+
+/**
+ * struct mtk_vcodec_ctrl - information about controls to be registered.
+ * @id:			Control ID.
+ * @type:		Type of the control.
+ * @name:		Human readable name of the control.
+ * @minimum:		Minimum value of the control.
+ * @maximum:		Maximum value of the control.
+ * @step:		Control value increase step.
+ * @menu_skip_mask:	Mask of invalid menu positions.
+ * @default_value:	Initial value of the control.
+ * @is_volatile:	Control is volatile.
+ *
+ * See also struct v4l2_ctrl_config.
+ */
+struct mtk_vcodec_ctrl {
+	u32			id;
+	enum v4l2_ctrl_type	type;
+	u8			name[32];
+	s32			minimum;
+	s32			maximum;
+	s32			step;
+	u32			menu_skip_mask;
+	s32			default_value;
+	u8			is_volatile;
+};
+
+static inline struct mtk_vcodec_ctx *fh_to_ctx(struct v4l2_fh *fh)
+{
+	return container_of(fh, struct mtk_vcodec_ctx, fh);
+}
+
+static inline struct mtk_vcodec_ctx *ctrl_to_ctx(struct v4l2_ctrl *ctrl)
+{
+	return container_of(ctrl->handler, struct mtk_vcodec_ctx, ctrl_hdl);
+}
+
+extern const struct v4l2_ioctl_ops mtk_vdec_ioctl_ops;
+extern const struct v4l2_m2m_ops mtk_vdec_m2m_ops;
+extern const struct v4l2_ioctl_ops mtk_venc_ioctl_ops;
+extern const struct v4l2_m2m_ops mtk_venc_m2m_ops;
+
+#endif /* _MTK_VCODEC_DRV_H_ */
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
new file mode 100644
index 0000000..ee602fe
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
@@ -0,0 +1,1380 @@
+/*
+* Copyright (c) 2015 MediaTek Inc.
+* Author: PC Chen <pc.chen-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
+*         Tiffany Lin <tiffany.lin-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
+*
+* This program is free software; you can redistribute it and/or modify
+* it under the terms of the GNU General Public License version 2 as
+* published by the Free Software Foundation.
+*
+* This program is distributed in the hope that it will be useful,
+* but WITHOUT ANY WARRANTY; without even the implied warranty of
+* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+* GNU General Public License for more details.
+*/
+
+#include <media/v4l2-event.h>
+#include <media/v4l2-mem2mem.h>
+#include <media/videobuf2-dma-contig.h>
+
+#include "mtk_vcodec_drv.h"
+#include "mtk_vcodec_enc.h"
+#include "mtk_vcodec_intr.h"
+#include "mtk_vcodec_util.h"
+#include "venc_drv_if.h"
+
+#define MTK_VENC_MIN_W	32
+#define MTK_VENC_MIN_H	32
+#define MTK_VENC_MAX_W	1920
+#define MTK_VENC_MAX_H	1080
+#define DFT_CFG_WIDTH	MTK_VENC_MIN_W
+#define DFT_CFG_HEIGHT	MTK_VENC_MIN_H
+
+static void mtk_venc_worker(struct work_struct *work);
+
+static struct mtk_video_fmt mtk_video_formats[] = {
+	{
+		.fourcc		= V4L2_PIX_FMT_YUV420,
+		.type		= MTK_FMT_FRAME,
+		.num_planes	= 3,
+	},
+	{
+		.fourcc		= V4L2_PIX_FMT_YVU420,
+		.type		= MTK_FMT_FRAME,
+		.num_planes	= 3,
+	},
+	{
+		.fourcc		= V4L2_PIX_FMT_NV12,
+		.type		= MTK_FMT_FRAME,
+		.num_planes	= 2,
+	},
+	{
+		.fourcc		= V4L2_PIX_FMT_NV21,
+		.type		= MTK_FMT_FRAME,
+		.num_planes	= 2,
+	},
+	{
+		.fourcc		= V4L2_PIX_FMT_YUV420M,
+		.type		= MTK_FMT_FRAME,
+		.num_planes	= 3,
+	},
+	{
+		.fourcc		= V4L2_PIX_FMT_YVU420M,
+		.type		= MTK_FMT_FRAME,
+		.num_planes	= 3,
+	},
+	{
+		.fourcc 	= V4L2_PIX_FMT_NV12M,
+		.type		= MTK_FMT_FRAME,
+		.num_planes	= 2,
+	},
+	{
+		.fourcc		= V4L2_PIX_FMT_NV21M,
+		.type		= MTK_FMT_FRAME,
+		.num_planes	= 2,
+	},
+	{
+		.fourcc		= V4L2_PIX_FMT_H264,
+		.type		= MTK_FMT_ENC,
+		.num_planes	= 1,
+	},
+	{
+		.fourcc		= V4L2_PIX_FMT_VP8,
+		.type		= MTK_FMT_ENC,
+		.num_planes	= 1,
+	},
+};
+
+#define NUM_FORMATS ARRAY_SIZE(mtk_video_formats)
+
+static const struct mtk_codec_framesizes mtk_venc_framesizes[] = {
+	{
+		.fourcc	= V4L2_PIX_FMT_H264,
+		.stepwise = {  160, 1920, 16, 128, 1088, 16 },
+	},
+	{
+		.fourcc = V4L2_PIX_FMT_VP8,
+		.stepwise = {  160, 1920, 16, 128, 1088, 16 },
+	},
+};
+
+#define NUM_SUPPORTED_FRAMESIZE ARRAY_SIZE(mtk_venc_framesizes)
+
+static int vidioc_venc_s_ctrl(struct v4l2_ctrl *ctrl)
+{
+	struct mtk_vcodec_ctx *ctx = ctrl_to_ctx(ctrl);
+	struct mtk_enc_params *p = &ctx->enc_params;
+	int ret = 0;
+
+	switch (ctrl->id) {
+	case V4L2_CID_MPEG_VIDEO_BITRATE:
+		mtk_v4l2_debug(2, "V4L2_CID_MPEG_VIDEO_BITRATE val = %d",
+			ctrl->val);
+		p->bitrate = ctrl->val;
+		ctx->param_change |= MTK_ENCODE_PARAM_BITRATE;
+		break;
+	case V4L2_CID_MPEG_VIDEO_B_FRAMES:
+		mtk_v4l2_debug(2, "V4L2_CID_MPEG_VIDEO_B_FRAMES val = %d",
+			ctrl->val);
+		p->num_b_frame = ctrl->val;
+		break;
+	case V4L2_CID_MPEG_VIDEO_FRAME_RC_ENABLE:
+		mtk_v4l2_debug(2, "V4L2_CID_MPEG_VIDEO_FRAME_RC_ENABLE val = %d",
+			ctrl->val);
+		p->rc_frame = ctrl->val;
+		break;
+	case V4L2_CID_MPEG_VIDEO_H264_MAX_QP:
+		mtk_v4l2_debug(2, "V4L2_CID_MPEG_VIDEO_H264_MAX_QP val = %d",
+			ctrl->val);
+		p->h264_max_qp = ctrl->val;
+		break;
+	case V4L2_CID_MPEG_VIDEO_HEADER_MODE:
+		mtk_v4l2_debug(2, "V4L2_CID_MPEG_VIDEO_HEADER_MODE val = %d",
+			ctrl->val);
+		p->seq_hdr_mode = ctrl->val;
+		break;
+	case V4L2_CID_MPEG_VIDEO_MB_RC_ENABLE:
+		mtk_v4l2_debug(2, "V4L2_CID_MPEG_VIDEO_MB_RC_ENABLE val = %d",
+			ctrl->val);
+		p->rc_mb = ctrl->val;
+		break;
+	case V4L2_CID_MPEG_VIDEO_H264_PROFILE:
+		mtk_v4l2_debug(2, "V4L2_CID_MPEG_VIDEO_H264_PROFILE val = %d",
+			ctrl->val);
+		p->h264_profile = ctrl->val;
+		break;
+	case V4L2_CID_MPEG_VIDEO_H264_LEVEL:
+		mtk_v4l2_debug(2, "V4L2_CID_MPEG_VIDEO_H264_LEVEL val = %d",
+			ctrl->val);
+		p->h264_level = ctrl->val;
+		break;
+	case V4L2_CID_MPEG_VIDEO_H264_I_PERIOD:
+		mtk_v4l2_debug(2, "V4L2_CID_MPEG_VIDEO_H264_I_PERIOD val = %d",
+			ctrl->val);
+	case V4L2_CID_MPEG_VIDEO_GOP_SIZE:
+		mtk_v4l2_debug(2, "V4L2_CID_MPEG_VIDEO_GOP_SIZE val = %d",
+			ctrl->val);
+		p->gop_size = ctrl->val;
+		ctx->param_change |= MTK_ENCODE_PARAM_INTRA_PERIOD;
+		break;
+	case V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME:
+		mtk_v4l2_debug(2, "V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME");
+		p->force_intra = 1;
+		ctx->param_change |= MTK_ENCODE_PARAM_FRAME_TYPE;
+		break;
+	default:
+		ret = -EINVAL;
+		break;
+	}
+
+	return ret;
+}
+
+static const struct v4l2_ctrl_ops mtk_vcodec_enc_ctrl_ops = {
+	.s_ctrl = vidioc_venc_s_ctrl,
+};
+
+static int vidioc_enum_fmt(struct file *file, struct v4l2_fmtdesc *f,
+			   bool out)
+{
+	struct mtk_video_fmt *fmt;
+	int i, j = 0;
+
+	for (i = 0; i < NUM_FORMATS; ++i) {
+		if (out && mtk_video_formats[i].type != MTK_FMT_FRAME)
+			continue;
+		else if (!out && mtk_video_formats[i].type != MTK_FMT_ENC)
+			continue;
+
+		if (j == f->index) {
+			fmt = &mtk_video_formats[i];
+			f->pixelformat = fmt->fourcc;
+			return 0;
+		}
+		++j;
+	}
+
+	return -EINVAL;
+}
+
+static int vidioc_enum_framesizes(struct file *file, void *fh,
+				  struct v4l2_frmsizeenum *fsize)
+{
+	int i = 0;
+
+	for (i = 0; i < NUM_SUPPORTED_FRAMESIZE; ++i) {
+		if (fsize->pixel_format != mtk_venc_framesizes[i].fourcc)
+			continue;
+
+		if (!fsize->index) {
+			fsize->type = V4L2_FRMSIZE_TYPE_STEPWISE;
+			fsize->stepwise = mtk_venc_framesizes[i].stepwise;
+			return 0;
+		}
+	}
+
+	return -EINVAL;
+}
+
+static int vidioc_enum_fmt_vid_cap_mplane(struct file *file, void *pirv,
+					  struct v4l2_fmtdesc *f)
+{
+	return vidioc_enum_fmt(file, f, false);
+}
+
+static int vidioc_enum_fmt_vid_out_mplane(struct file *file, void *prov,
+					  struct v4l2_fmtdesc *f)
+{
+	return vidioc_enum_fmt(file, f, true);
+}
+
+static int vidioc_venc_querycap(struct file *file, void *priv,
+				struct v4l2_capability *cap)
+{
+        strlcpy(cap->driver, MTK_VCODEC_ENC_NAME, strlen(MTK_VCODEC_ENC_NAME));
+        cap->driver[strlen(MTK_VCODEC_ENC_NAME)]=0;
+        strlcpy(cap->bus_info, MTK_PLATFORM_STR, strlen(MTK_PLATFORM_STR));
+        cap->bus_info[strlen(MTK_PLATFORM_STR)]=0;
+        strlcpy(cap->card, MTK_PLATFORM_STR, strlen(MTK_PLATFORM_STR));
+        cap->card[strlen(MTK_PLATFORM_STR)]=0;
+
+	/*
+	 * This is only a mem-to-mem video device. The capture and output
+	 * device capability flags are left only for backward compatibility
+	 * and are scheduled for removal.
+	 */
+	cap->device_caps  = V4L2_CAP_VIDEO_M2M_MPLANE | V4L2_CAP_STREAMING;
+	cap->capabilities = cap->device_caps | V4L2_CAP_DEVICE_CAPS;
+
+	return 0;
+}
+static int vidioc_venc_s_parm(struct file *file, void *priv,
+			      struct v4l2_streamparm *a)
+{
+	struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
+
+	if (a->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
+		ctx->enc_params.framerate_num =
+			a->parm.output.timeperframe.denominator;
+		ctx->enc_params.framerate_denom =
+			a->parm.output.timeperframe.numerator;
+		ctx->param_change |= MTK_ENCODE_PARAM_FRAMERATE;
+	} else {
+		return -EINVAL;
+	}
+	return 0;
+}
+
+static struct mtk_q_data *mtk_venc_get_q_data(struct mtk_vcodec_ctx *ctx,
+					      enum v4l2_buf_type type)
+{
+	if (V4L2_TYPE_IS_OUTPUT(type))
+		return &ctx->q_data[MTK_Q_DATA_SRC];
+
+	return &ctx->q_data[MTK_Q_DATA_DST];
+}
+
+static struct mtk_video_fmt *mtk_venc_find_format(struct v4l2_format *f)
+{
+	struct mtk_video_fmt *fmt;
+	unsigned int k;
+
+	for (k = 0; k < NUM_FORMATS; k++) {
+		fmt = &mtk_video_formats[k];
+		if (fmt->fourcc == f->fmt.pix.pixelformat)
+			return fmt;
+	}
+
+	return NULL;
+}
+
+static void mtk_vcodec_enc_calc_src_size(
+	unsigned int num_planes, unsigned int pic_width,
+	unsigned int pic_height, unsigned int sizeimage[],
+	unsigned int bytesperline[])
+{
+	unsigned int y_pitch_w_div16;
+	unsigned int c_pitch_w_div16;
+
+	y_pitch_w_div16 = ALIGN(pic_width, 16) >> 4;
+	c_pitch_w_div16 = ALIGN(pic_width, 16) >> 4;
+
+	if (num_planes == 2) {
+		sizeimage[0] =
+			(y_pitch_w_div16) * (((pic_height + 31) / 32) * 2) * 256 +
+			((y_pitch_w_div16 % 8 == 0) ? 0 : ((ALIGN(pic_width, 16) * 2) * 16));
+
+		sizeimage[1] =
+			(c_pitch_w_div16) * (((pic_height + 31) / 32) * 2) * 128 +
+			((c_pitch_w_div16 % 8 == 0) ? 0 : (ALIGN(pic_width, 16) * 16));
+
+		sizeimage[2] = 0;
+
+		bytesperline[0] = ALIGN(pic_width, 16);
+		bytesperline[1] = ALIGN(pic_width, 16);
+		bytesperline[2] = 0;
+
+	} else {
+		sizeimage[0] =
+			(y_pitch_w_div16) * (((pic_height + 31) / 32) * 2) * 256 +
+			((y_pitch_w_div16 % 8 == 0) ? 0 : ((ALIGN(pic_width, 16) * 2) * 16));
+
+		sizeimage[1] =
+			(c_pitch_w_div16) * (((pic_height + 31) / 32) * 2) * 64 +
+			((c_pitch_w_div16 % 8 == 0) ? 0 : ((ALIGN(pic_width, 16) / 2) * 16));
+
+		sizeimage[2] =
+			(c_pitch_w_div16) * (((pic_height + 31) / 32) * 2) * 64 +
+			((c_pitch_w_div16 % 8 == 0) ? 0 : ((ALIGN(pic_width, 16) / 2) * 16));
+
+		bytesperline[0] = ALIGN(pic_width, 16);
+		bytesperline[1] = ALIGN(pic_width, 16) / 2;
+		bytesperline[2] = ALIGN(pic_width, 16) / 2;
+	}
+}
+
+static int vidioc_try_fmt(struct v4l2_format *f, struct mtk_video_fmt *fmt)
+{
+	struct v4l2_pix_format_mplane *pix_fmt_mp = &f->fmt.pix_mp;
+
+	/* V4L2 specification suggests the driver corrects the format struct
+	  * if any of the dimensions is unsupported */
+        if (pix_fmt_mp->height < MTK_VENC_MIN_H)
+                pix_fmt_mp->height = MTK_VENC_MIN_H;
+        else if (pix_fmt_mp->height > MTK_VENC_MAX_H)
+                pix_fmt_mp->height = MTK_VENC_MAX_H;
+
+        if (pix_fmt_mp->width < MTK_VENC_MIN_W)
+                pix_fmt_mp->width = MTK_VENC_MIN_W;
+        else if (pix_fmt_mp->width > MTK_VENC_MAX_W)
+                pix_fmt_mp->width = MTK_VENC_MAX_W;
+
+        pix_fmt_mp->field = V4L2_FIELD_NONE;
+
+	if (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
+		int size = pix_fmt_mp->height * pix_fmt_mp->width;
+		if (fmt->num_planes != pix_fmt_mp->num_planes)
+			pix_fmt_mp->num_planes = fmt->num_planes;
+
+		if(pix_fmt_mp->plane_fmt[0].sizeimage != size)
+			pix_fmt_mp->plane_fmt[0].sizeimage = size;
+ 		pix_fmt_mp->plane_fmt[0].bytesperline = 0;
+		memset(&(pix_fmt_mp->plane_fmt[0].reserved[0]), 0x0,
+			sizeof(pix_fmt_mp->plane_fmt[0].reserved));
+	} else if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
+		int i;
+		unsigned int sizeimage[VIDEO_MAX_PLANES];
+		unsigned int bytesperline[VIDEO_MAX_PLANES];
+
+		v4l_bound_align_image(&pix_fmt_mp->width, 8, 1920, 1,
+				      &pix_fmt_mp->height, 4, 1080, 1, 0);
+
+		if (fmt->num_planes != pix_fmt_mp->num_planes)
+			pix_fmt_mp->num_planes = fmt->num_planes;
+
+		mtk_vcodec_enc_calc_src_size(pix_fmt_mp->num_planes,
+					pix_fmt_mp->width,
+					pix_fmt_mp->height,
+					sizeimage, bytesperline);
+
+		for (i=0; i < pix_fmt_mp->num_planes; i++) {
+			pix_fmt_mp->plane_fmt[i].sizeimage = sizeimage[i];
+			pix_fmt_mp->plane_fmt[i].bytesperline = bytesperline[i];
+			memset(&(pix_fmt_mp->plane_fmt[i].reserved[0]), 0x0,
+				sizeof(pix_fmt_mp->plane_fmt[0].reserved));
+		}
+	} else {
+		return -EINVAL;
+	}
+
+	pix_fmt_mp->flags = 0;
+	pix_fmt_mp->ycbcr_enc = 0;
+	pix_fmt_mp->quantization = 0;
+	pix_fmt_mp->xfer_func = 0;
+	memset(&pix_fmt_mp->reserved[0], 0x0, sizeof(pix_fmt_mp->reserved));
+
+	return 0;
+}
+
+static void mtk_venc_set_param(struct mtk_vcodec_ctx *ctx, void *param)
+{
+	struct venc_enc_prm *p = (struct venc_enc_prm *)param;
+	struct mtk_q_data *q_data_src = &ctx->q_data[MTK_Q_DATA_SRC];
+	struct mtk_enc_params *enc_params = &ctx->enc_params;
+	unsigned int frame_rate;
+
+	frame_rate = enc_params->framerate_num / enc_params->framerate_denom;
+
+	switch (q_data_src->fmt->fourcc) {
+	case V4L2_PIX_FMT_YUV420:
+	case V4L2_PIX_FMT_YUV420M:
+		p->input_fourcc = VENC_YUV_FORMAT_420;
+		break;
+	case V4L2_PIX_FMT_YVU420:
+	case V4L2_PIX_FMT_YVU420M:
+		p->input_fourcc = VENC_YUV_FORMAT_YV12;
+		break;
+	case V4L2_PIX_FMT_NV12:
+	case V4L2_PIX_FMT_NV12M:
+		p->input_fourcc = VENC_YUV_FORMAT_NV12;
+		break;
+	case V4L2_PIX_FMT_NV21:
+	case V4L2_PIX_FMT_NV21M:
+		p->input_fourcc = VENC_YUV_FORMAT_NV21;
+		break;
+	}
+	p->h264_profile = enc_params->h264_profile;
+	p->h264_level = enc_params->h264_level;
+	p->width = q_data_src->width;
+	p->height = q_data_src->height;
+	p->buf_width = q_data_src->bytesperline[0];
+	p->buf_height = ((q_data_src->height + 0xf) & (~0xf));
+	p->frm_rate = frame_rate;
+	p->intra_period = enc_params->gop_size;
+	p->bitrate = enc_params->bitrate;
+
+	ctx->param_change = MTK_ENCODE_PARAM_NONE;
+
+	mtk_v4l2_debug(1, "fmt 0x%x, P/L %d/%d, w/h %d/%d, buf %d/%d, fps/bps %d/%d, gop %d",
+		       p->input_fourcc, p->h264_profile, p->h264_level,
+		       p->width, p->height, p->buf_width, p->buf_height,
+		       p->frm_rate, p->bitrate, p->intra_period);
+}
+
+static int vidioc_venc_s_fmt(struct file *file, void *priv,
+			     struct v4l2_format *f)
+{
+	struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
+	struct v4l2_pix_format_mplane *pix_fmt_mp = &f->fmt.pix_mp;
+	struct vb2_queue *vq;
+	struct mtk_q_data *q_data;
+	struct venc_enc_prm param;
+	int i, ret;
+	struct mtk_video_fmt *fmt;
+
+	vq = v4l2_m2m_get_vq(ctx->m2m_ctx, f->type);
+	if (!vq) {
+		mtk_v4l2_err("fail to get vq\n");
+		return -EINVAL;
+	}
+
+	if (vb2_is_busy(vq)) {
+		mtk_v4l2_err("queue busy\n");
+		return -EBUSY;
+	}
+
+	q_data = mtk_venc_get_q_data(ctx, f->type);
+	if (!q_data) {
+		mtk_v4l2_err("fail to get q data\n");
+		return -EINVAL;
+	}
+
+	fmt = mtk_venc_find_format(f);
+	if (!fmt) {
+		if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
+			f->fmt.pix.pixelformat = mtk_video_formats[0].fourcc;
+			fmt = mtk_venc_find_format(f);
+		} else {
+			f->fmt.pix.pixelformat = mtk_video_formats[8].fourcc;
+			fmt = mtk_venc_find_format(f);
+		}
+	}
+
+	q_data->fmt = fmt;
+	ret = vidioc_try_fmt(f, q_data->fmt);
+	if (ret)
+		return ret;
+
+	q_data->width		= f->fmt.pix_mp.width;
+	q_data->height		= f->fmt.pix_mp.height;
+	q_data->field		= f->fmt.pix_mp.field;
+
+	if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
+		q_data->colorspace = f->fmt.pix_mp.colorspace;
+		ctx->q_data[MTK_Q_DATA_SRC].bytesperline[0] =
+			ALIGN(q_data->width, 16);
+
+		if (q_data->fmt->num_planes == 2) {
+			ctx->q_data[MTK_Q_DATA_SRC].bytesperline[1] =
+				ALIGN(q_data->width, 16);
+			ctx->q_data[MTK_Q_DATA_SRC].bytesperline[2] = 0;
+		} else {
+			ctx->q_data[MTK_Q_DATA_SRC].bytesperline[1] =
+				ALIGN(q_data->width, 16) / 2;
+			ctx->q_data[MTK_Q_DATA_SRC].bytesperline[2] =
+				ALIGN(q_data->width, 16) / 2;
+		}
+
+		memset(&param, 0, sizeof(param));
+		mtk_venc_set_param(ctx, &param);
+		if (ctx->state == MTK_STATE_INIT) {
+			ret = venc_if_set_param(ctx,
+						VENC_SET_PARAM_ENC,
+						&param);
+			if (ret)
+				mtk_v4l2_err("venc_if_set_param failed=%d\n",
+						ret);
+
+			/* Get codec driver advice sizeimage from vpu */
+			for (i = 0; i < MTK_VCODEC_MAX_PLANES; i++) {
+				q_data->sizeimage[i] = param.sizeimage[i];
+				pix_fmt_mp->plane_fmt[i].sizeimage =
+					param.sizeimage[i];
+			}
+			q_data->bytesperline[0] =
+				pix_fmt_mp->plane_fmt[0].bytesperline;
+			q_data->bytesperline[1] =
+				pix_fmt_mp->plane_fmt[1].bytesperline;
+			q_data->bytesperline[2] =
+				pix_fmt_mp->plane_fmt[2].bytesperline;
+		} else {
+			for (i = 0; i < MTK_VCODEC_MAX_PLANES; i++) {
+				q_data->sizeimage[i] =
+					pix_fmt_mp->plane_fmt[i].sizeimage;
+				q_data->bytesperline[i] =
+					pix_fmt_mp->plane_fmt[i].bytesperline;
+			}
+		}
+	} else if (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE){
+		for (i = 0; i < f->fmt.pix_mp.num_planes; i++) {
+			struct v4l2_plane_pix_format	*plane_fmt;
+
+			plane_fmt = &f->fmt.pix_mp.plane_fmt[i];
+			q_data->bytesperline[i]	= plane_fmt->bytesperline;
+			q_data->sizeimage[i]	= plane_fmt->sizeimage;
+		}
+
+		if (ctx->state == MTK_STATE_FREE) {
+			ret = venc_if_create(ctx, q_data->fmt->fourcc);
+			if (ret) {
+				mtk_v4l2_err("venc_if_create failed=%d, codec type=%x\n",
+					ret, q_data->fmt->fourcc);
+				return 0;
+			}
+
+			ctx->state = MTK_STATE_INIT;
+		}
+	}
+
+	return 0;
+}
+
+static int vidioc_venc_g_fmt(struct file *file, void *priv,
+			     struct v4l2_format *f)
+{
+	struct v4l2_pix_format_mplane *pix = &f->fmt.pix_mp;
+	struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
+	struct vb2_queue *vq;
+	struct mtk_q_data *q_data;
+	int i;
+
+	vq = v4l2_m2m_get_vq(ctx->m2m_ctx, f->type);
+	if (!vq)
+		return -EINVAL;
+
+	q_data = mtk_venc_get_q_data(ctx, f->type);
+
+	pix->width = q_data->width;
+	pix->height = q_data->height;
+	pix->pixelformat = q_data->fmt->fourcc;
+	pix->field = q_data->field;
+	pix->colorspace = q_data->colorspace;
+	pix->num_planes = q_data->fmt->num_planes;
+	for (i = 0; i < pix->num_planes; i++) {
+		pix->plane_fmt[i].bytesperline = q_data->bytesperline[i];
+		pix->plane_fmt[i].sizeimage = q_data->sizeimage[i];
+		memset(&(pix->plane_fmt[i].reserved[0]), 0x0,
+			sizeof(pix->plane_fmt[i].reserved));
+	}
+	pix->flags = 0;
+	pix->ycbcr_enc = 0;
+	pix->quantization = 0;
+	pix->xfer_func = 0;
+	memset(&pix->reserved[0], 0x0, sizeof(pix->reserved));
+
+	return 0;
+}
+
+static int vidioc_try_fmt_vid_cap_mplane(struct file *file, void *priv,
+                                  struct v4l2_format *f)
+{
+        struct mtk_video_fmt *fmt;
+	struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
+
+        fmt = mtk_venc_find_format(f);
+        if (!fmt) {
+                f->fmt.pix.pixelformat = mtk_video_formats[8].fourcc;
+                fmt = mtk_venc_find_format(f);
+        }
+        if (fmt->type != MTK_FMT_ENC) {
+		mtk_v4l2_err("Fourcc format (0x%08x) invalid.\n",
+			     f->fmt.pix.pixelformat);
+		return -EINVAL;
+        }
+        f->fmt.pix_mp.colorspace = ctx->q_data[MTK_Q_DATA_SRC].colorspace;
+
+        return vidioc_try_fmt(f, fmt);
+}
+
+static int vidioc_try_fmt_vid_out_mplane(struct file *file, void *priv,
+                                  struct v4l2_format *f)
+{
+        struct mtk_video_fmt *fmt;
+
+        fmt = mtk_venc_find_format(f);
+        if (!fmt) {
+                f->fmt.pix.pixelformat = mtk_video_formats[0].fourcc;
+                fmt = mtk_venc_find_format(f);
+        }
+        if (!(fmt->type & MTK_FMT_FRAME)) {
+		mtk_v4l2_err("Fourcc format (0x%08x) invalid.\n",
+			     f->fmt.pix.pixelformat);
+		return -EINVAL;
+        }
+        if (!f->fmt.pix_mp.colorspace)
+                f->fmt.pix_mp.colorspace = V4L2_COLORSPACE_REC709;
+
+        return vidioc_try_fmt(f, fmt);
+}
+
+static int vidioc_venc_g_s_selection(struct file *file, void *priv,
+                                struct v4l2_selection *s)
+{
+	struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
+	struct mtk_q_data *q_data;
+
+	if (V4L2_TYPE_IS_OUTPUT(s->type)) {
+		if (s->target !=  V4L2_SEL_TGT_COMPOSE)
+			return -EINVAL;
+	} else {
+		if (s->target != V4L2_SEL_TGT_CROP)
+			return -EINVAL;
+	}
+
+	if (s->r.left || s->r.top)
+		return -EINVAL;
+
+	q_data = mtk_venc_get_q_data(ctx, s->type);
+	if (!q_data)
+		return -EINVAL;
+
+	s->r.width = q_data->width;
+	s->r.height = q_data->height;
+
+	return 0;
+}
+
+
+static int vidioc_venc_qbuf(struct file *file, void *priv,
+			    struct v4l2_buffer *buf)
+{
+
+	struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
+
+	if (ctx->state == MTK_STATE_ABORT) {
+		mtk_v4l2_err("[%d] Call on QBUF after unrecoverable error\n", ctx->idx);
+		return -EIO;
+	}
+
+	return v4l2_m2m_qbuf(file, ctx->m2m_ctx, buf);
+}
+
+static int vidioc_venc_dqbuf(struct file *file, void *priv,
+			     struct v4l2_buffer *buf)
+{
+	struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
+	if (ctx->state == MTK_STATE_ABORT) {
+		mtk_v4l2_err("[%d] Call on QBUF after unrecoverable error\n", ctx->idx);
+		return -EIO;
+	}
+
+	return v4l2_m2m_dqbuf(file, ctx->m2m_ctx, buf);
+}
+
+
+const struct v4l2_ioctl_ops mtk_venc_ioctl_ops = {
+	.vidioc_streamon		= v4l2_m2m_ioctl_streamon,
+	.vidioc_streamoff		= v4l2_m2m_ioctl_streamoff,
+
+	.vidioc_reqbufs			= v4l2_m2m_ioctl_reqbufs,
+	.vidioc_querybuf		= v4l2_m2m_ioctl_querybuf,
+	.vidioc_qbuf			= vidioc_venc_qbuf,
+	.vidioc_dqbuf			= vidioc_venc_dqbuf,
+
+	.vidioc_querycap		= vidioc_venc_querycap,
+	.vidioc_enum_fmt_vid_cap_mplane = vidioc_enum_fmt_vid_cap_mplane,
+	.vidioc_enum_fmt_vid_out_mplane = vidioc_enum_fmt_vid_out_mplane,
+	.vidioc_enum_framesizes		= vidioc_enum_framesizes,
+
+	.vidioc_try_fmt_vid_cap_mplane	= vidioc_try_fmt_vid_cap_mplane,
+	.vidioc_try_fmt_vid_out_mplane	= vidioc_try_fmt_vid_out_mplane,
+	.vidioc_expbuf			= v4l2_m2m_ioctl_expbuf,
+	.vidioc_subscribe_event		= v4l2_ctrl_subscribe_event,
+	.vidioc_unsubscribe_event	= v4l2_event_unsubscribe,
+
+	.vidioc_s_parm			= vidioc_venc_s_parm,
+
+	.vidioc_s_fmt_vid_cap_mplane	= vidioc_venc_s_fmt,
+	.vidioc_s_fmt_vid_out_mplane	= vidioc_venc_s_fmt,
+
+	.vidioc_g_fmt_vid_cap_mplane	= vidioc_venc_g_fmt,
+	.vidioc_g_fmt_vid_out_mplane	= vidioc_venc_g_fmt,
+
+	.vidioc_g_selection		= vidioc_venc_g_s_selection,
+	.vidioc_s_selection		= vidioc_venc_g_s_selection,
+};
+
+static int vb2ops_venc_queue_setup(struct vb2_queue *vq,
+				   const void *parg,
+				   unsigned int *nbuffers,
+				   unsigned int *nplanes,
+				   unsigned int sizes[], void *alloc_ctxs[])
+{
+	struct mtk_vcodec_ctx *ctx = vb2_get_drv_priv(vq);
+	struct mtk_q_data *q_data;
+
+	q_data = mtk_venc_get_q_data(ctx, vq->type);
+
+	if (*nbuffers < 1)
+		*nbuffers = 1;
+	if (*nbuffers > MTK_VIDEO_MAX_FRAME)
+		*nbuffers = MTK_VIDEO_MAX_FRAME;
+
+	*nplanes = q_data->fmt->num_planes;
+
+	if (vq->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
+		unsigned int i;
+
+		for (i = 0; i < *nplanes; i++) {
+			sizes[i] = q_data->sizeimage[i];
+			alloc_ctxs[i] = ctx->dev->alloc_ctx;
+		}
+	} else if (vq->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
+		sizes[0] = q_data->sizeimage[0];
+		alloc_ctxs[0] = ctx->dev->alloc_ctx;
+	} else {
+		return -EINVAL;
+	}
+
+	mtk_v4l2_debug(2, "[%d]get %d buffer(s) of size 0x%x each, vq->memory=%d",
+		       ctx->idx, *nbuffers, sizes[0], vq->memory);
+
+	return 0;
+}
+
+static int vb2ops_venc_buf_prepare(struct vb2_buffer *vb)
+{
+	struct mtk_vcodec_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue);
+	struct mtk_q_data *q_data;
+	int i;
+
+	q_data = mtk_venc_get_q_data(ctx, vb->vb2_queue->type);
+
+	for (i = 0; i < q_data->fmt->num_planes; i++) {
+		if (vb2_plane_size(vb, i) < q_data->sizeimage[i]) {
+			mtk_v4l2_debug(2, "data will not fit into plane %d (%lu < %d)",
+				       i, vb2_plane_size(vb, i),
+				       q_data->sizeimage[i]);
+			return -EINVAL;
+		}
+	}
+
+	return 0;
+}
+
+static void vb2ops_venc_buf_queue(struct vb2_buffer *vb)
+{
+	struct mtk_vcodec_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue);
+	struct vb2_v4l2_buffer *vb2_v4l2 =
+			container_of(vb, struct vb2_v4l2_buffer, vb2_buf);
+	struct mtk_video_enc_buf *mtk_buf =
+			container_of(vb2_v4l2, struct mtk_video_enc_buf, vb);
+
+	if ((vb->vb2_queue->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) &&
+		(ctx->param_change != MTK_ENCODE_PARAM_NONE)) {
+		mtk_v4l2_debug(1,
+				"[%d] Before id=%d encode parameter change %x",
+				ctx->idx, vb->index,
+				ctx->param_change);
+		mtk_buf->param_change = ctx->param_change;
+		if (mtk_buf->param_change & MTK_ENCODE_PARAM_BITRATE) {
+			mtk_buf->enc_params.bitrate = ctx->enc_params.bitrate;
+			mtk_v4l2_debug(1, "[%d] idx=%d change param br=%d",
+				ctx->idx,
+				mtk_buf->vb.vb2_buf.index,
+				mtk_buf->enc_params.bitrate);
+		}
+		if (ctx->param_change & MTK_ENCODE_PARAM_FRAMERATE) {
+			mtk_buf->enc_params.framerate_num =
+				ctx->enc_params.framerate_num;
+			mtk_buf->enc_params.framerate_denom =
+				ctx->enc_params.framerate_denom;
+			mtk_v4l2_debug(1, "[%d] idx=%d, change param fr=%d/%d",
+					ctx->idx,
+					mtk_buf->vb.vb2_buf.index,
+					mtk_buf->enc_params.framerate_num,
+					mtk_buf->enc_params.framerate_denom);
+		}
+		if (ctx->param_change & MTK_ENCODE_PARAM_INTRA_PERIOD) {
+			mtk_buf->enc_params.gop_size = ctx->enc_params.gop_size;
+			mtk_v4l2_debug(1, "[%d] idx=%d, change param intra period=%d",
+					ctx->idx,
+					mtk_buf->vb.vb2_buf.index,
+					mtk_buf->enc_params.gop_size);
+		}
+		if (ctx->param_change & MTK_ENCODE_PARAM_FRAME_TYPE) {
+			mtk_buf->enc_params.force_intra =
+				ctx->enc_params.force_intra;
+			mtk_v4l2_debug(1, "[%d] idx=%d, change param force I=%d",
+					ctx->idx,
+					mtk_buf->vb.vb2_buf.index,
+					mtk_buf->enc_params.force_intra);
+		}
+		ctx->param_change = MTK_ENCODE_PARAM_NONE;
+	}
+
+	v4l2_m2m_buf_queue(ctx->m2m_ctx, to_vb2_v4l2_buffer(vb));
+}
+
+static int vb2ops_venc_start_streaming(struct vb2_queue *q, unsigned int count)
+{
+	struct mtk_vcodec_ctx *ctx = vb2_get_drv_priv(q);
+	struct venc_enc_prm param;
+	int ret;
+	int i;
+
+	/* Once state turn into MTK_STATE_ABORT, we need stop_streaming to clear it */
+	if ((ctx->state == MTK_STATE_ABORT) || (ctx->state == MTK_STATE_FREE))
+		goto err_set_param;
+
+	if (!(vb2_start_streaming_called(&ctx->m2m_ctx->out_q_ctx.q) &
+	      vb2_start_streaming_called(&ctx->m2m_ctx->cap_q_ctx.q))) {
+		mtk_v4l2_debug(1, "[%d]-> out=%d cap=%d",
+		 ctx->idx,
+		 vb2_start_streaming_called(&ctx->m2m_ctx->out_q_ctx.q),
+		 vb2_start_streaming_called(&ctx->m2m_ctx->cap_q_ctx.q));
+		return 0;
+	}
+
+	mtk_venc_set_param(ctx, &param);
+	ret = venc_if_set_param(ctx,
+				VENC_SET_PARAM_ENC,
+				&param);
+	if (ret) {
+		mtk_v4l2_err("venc_if_set_param failed=%d\n", ret);
+		ctx->state = MTK_STATE_ABORT;
+		goto err_set_param;
+	}
+
+	if ((ctx->q_data[MTK_Q_DATA_DST].fmt->fourcc == V4L2_PIX_FMT_H264) &&
+	    (ctx->enc_params.seq_hdr_mode != V4L2_MPEG_VIDEO_HEADER_MODE_SEPARATE)) {
+		ret = venc_if_set_param(ctx,
+					VENC_SET_PARAM_PREPEND_HEADER,
+					0);
+		if (ret) {
+			mtk_v4l2_err("venc_if_set_param failed=%d\n", ret);
+			ctx->state = MTK_STATE_ABORT;
+			goto err_set_param;
+		}
+		ctx->state = MTK_STATE_HEADER;
+	}
+
+	return 0;
+
+err_set_param:
+	for (i = 0; i < q->num_buffers; ++i) {
+		if (q->bufs[i]->state == VB2_BUF_STATE_ACTIVE) {
+			mtk_v4l2_debug(0, "[%d] idx=%d, type=%d, %d -> VB2_BUF_STATE_QUEUED",
+					ctx->idx, i, q->type,
+					(int)q->bufs[i]->state );
+			v4l2_m2m_buf_done(to_vb2_v4l2_buffer(q->bufs[i]), VB2_BUF_STATE_QUEUED);
+		}
+	}
+
+	return -EINVAL;
+}
+
+static void vb2ops_venc_stop_streaming(struct vb2_queue *q)
+{
+	struct mtk_vcodec_ctx *ctx = vb2_get_drv_priv(q);
+	struct vb2_buffer *src_buf, *dst_buf;
+	int ret;
+
+	mtk_v4l2_debug(2, "[%d]-> type=%d", ctx->idx, q->type);
+
+	if (q->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
+		while ((dst_buf = v4l2_m2m_dst_buf_remove(ctx->m2m_ctx))) {
+			dst_buf->planes[0].bytesused = 0;
+			v4l2_m2m_buf_done(to_vb2_v4l2_buffer(dst_buf),
+						VB2_BUF_STATE_ERROR);
+		}
+	} else {
+		while ((src_buf = v4l2_m2m_src_buf_remove(ctx->m2m_ctx)))
+			v4l2_m2m_buf_done(to_vb2_v4l2_buffer(src_buf),
+						VB2_BUF_STATE_ERROR);
+	}
+
+	if ((q->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE &&
+	     vb2_is_streaming(&ctx->m2m_ctx->out_q_ctx.q)) ||
+	    (q->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE &&
+	     vb2_is_streaming(&ctx->m2m_ctx->cap_q_ctx.q))) {
+		mtk_v4l2_debug(1, "[%d]-> q type %d out=%d cap=%d",
+			       ctx->idx, q->type,
+			       vb2_is_streaming(&ctx->m2m_ctx->out_q_ctx.q),
+			       vb2_is_streaming(&ctx->m2m_ctx->cap_q_ctx.q));
+		return;
+	}
+
+	ret = venc_if_release(ctx);
+	if (ret)
+		mtk_v4l2_err("venc_if_release failed=%d\n", ret);
+
+	ctx->state = MTK_STATE_FREE;
+}
+
+static struct vb2_ops mtk_venc_vb2_ops = {
+	.queue_setup			= vb2ops_venc_queue_setup,
+	.buf_prepare			= vb2ops_venc_buf_prepare,
+	.buf_queue			= vb2ops_venc_buf_queue,
+	.wait_prepare			= vb2_ops_wait_prepare,
+	.wait_finish			= vb2_ops_wait_finish,
+	.start_streaming		= vb2ops_venc_start_streaming,
+	.stop_streaming			= vb2ops_venc_stop_streaming,
+};
+
+static int mtk_venc_encode_header(void *priv)
+{
+	struct mtk_vcodec_ctx *ctx = priv;
+	int ret;
+	struct vb2_buffer *dst_buf;
+	struct mtk_vcodec_mem bs_buf;
+	struct venc_done_result enc_result;
+
+	dst_buf = v4l2_m2m_dst_buf_remove(ctx->m2m_ctx);
+	if (!dst_buf) {
+		mtk_v4l2_debug(1, "No dst buffer");
+		return -EINVAL;
+	}
+
+	bs_buf.va = vb2_plane_vaddr(dst_buf, 0);
+	bs_buf.dma_addr = vb2_dma_contig_plane_dma_addr(dst_buf, 0);
+	bs_buf.size = (unsigned int)dst_buf->planes[0].length;
+
+	mtk_v4l2_debug(1,
+			"[%d] buf idx=%d va=0x%p dma_addr=0x%llx size=0x%lx",
+			ctx->idx,
+			dst_buf->index, bs_buf.va,
+			(u64)bs_buf.dma_addr,
+			bs_buf.size);
+
+	ret = venc_if_encode(ctx,
+			VENC_START_OPT_ENCODE_SEQUENCE_HEADER,
+			0, &bs_buf, &enc_result);
+
+	if (ret) {
+		dst_buf->planes[0].bytesused = 0;
+		ctx->state = MTK_STATE_ABORT;
+		v4l2_m2m_buf_done(to_vb2_v4l2_buffer(dst_buf), VB2_BUF_STATE_ERROR);
+		mtk_v4l2_err("venc_if_encode failed=%d", ret);
+		return -EINVAL;
+	}
+
+	ctx->state = MTK_STATE_HEADER;
+	dst_buf->planes[0].bytesused = enc_result.bs_size;
+
+#if defined(DEBUG)
+{
+	int i;
+	mtk_v4l2_debug(1, "[%d] venc_if_encode header len=%d",
+			ctx->idx,
+			enc_result.bs_size);
+	for (i = 0; i < enc_result.bs_size; i++) {
+		unsigned char *p = (unsigned char *)bs_buf.va;
+
+		mtk_v4l2_debug(1, "[%d] buf[%d]=0x%2x", ctx->idx, i, p[i]);
+	}
+}
+#endif
+	v4l2_m2m_buf_done(to_vb2_v4l2_buffer(dst_buf), VB2_BUF_STATE_DONE);
+
+	return 0;
+}
+
+static int mtk_venc_param_change(struct mtk_vcodec_ctx *ctx, void *priv)
+{
+	struct vb2_buffer *vb = priv;
+	struct vb2_v4l2_buffer *vb2_v4l2 =
+			container_of(vb, struct vb2_v4l2_buffer, vb2_buf);
+	struct mtk_video_enc_buf *mtk_buf =
+			container_of(vb2_v4l2, struct mtk_video_enc_buf, vb);
+	int ret = 0;
+
+	if (mtk_buf->param_change == MTK_ENCODE_PARAM_NONE)
+		return 0;
+
+	mtk_v4l2_debug(1, "encode parameters change id=%d", vb->index);
+	if (mtk_buf->param_change & MTK_ENCODE_PARAM_BITRATE) {
+		struct venc_enc_prm enc_prm;
+
+		enc_prm.bitrate = mtk_buf->enc_params.bitrate;
+		mtk_v4l2_debug(1, "[%d] idx=%d, change param br=%d",
+				ctx->idx,
+				mtk_buf->vb.vb2_buf.index,
+				enc_prm.bitrate);
+		ret |= venc_if_set_param(ctx,
+					 VENC_SET_PARAM_ADJUST_BITRATE,
+					 &enc_prm);
+	}
+	if (mtk_buf->param_change & MTK_ENCODE_PARAM_FRAMERATE) {
+		struct venc_enc_prm enc_prm;
+
+		enc_prm.frm_rate = mtk_buf->enc_params.framerate_num /
+				   mtk_buf->enc_params.framerate_denom;
+		mtk_v4l2_debug(1, "[%d] idx=%d, change param fr=%d",
+			       ctx->idx,
+			       mtk_buf->vb.vb2_buf.index,
+			       enc_prm.frm_rate);
+		ret |= venc_if_set_param(ctx,
+					 VENC_SET_PARAM_ADJUST_FRAMERATE,
+					 &enc_prm);
+	}
+	if (mtk_buf->param_change & MTK_ENCODE_PARAM_INTRA_PERIOD) {
+		mtk_v4l2_debug(1, "change param intra period=%d",
+				 mtk_buf->enc_params.gop_size);
+		ret |= venc_if_set_param(ctx,
+					 VENC_SET_PARAM_I_FRAME_INTERVAL,
+					 &mtk_buf->enc_params.gop_size);
+	}
+	if (mtk_buf->param_change & MTK_ENCODE_PARAM_FRAME_TYPE) {
+		mtk_v4l2_debug(1, "[%d] idx=%d, change param force I=%d",
+				ctx->idx,
+				mtk_buf->vb.vb2_buf.index,
+				mtk_buf->enc_params.force_intra);
+		if (mtk_buf->enc_params.force_intra)
+			ret |= venc_if_set_param(ctx,
+						 VENC_SET_PARAM_FORCE_INTRA,
+						 0);
+	}
+
+	mtk_buf->param_change = MTK_ENCODE_PARAM_NONE;
+
+	if (ret) {
+		ctx->state = MTK_STATE_ABORT;
+		mtk_v4l2_err("venc_if_set_param %d failed=%d\n",
+			MTK_ENCODE_PARAM_FRAME_TYPE, ret);
+		return -1;
+	}
+
+	return 0;
+}
+
+static void mtk_venc_worker(struct work_struct *work)
+{
+	struct mtk_vcodec_ctx *ctx = container_of(work, struct mtk_vcodec_ctx,
+				    encode_work);
+	struct vb2_buffer *src_buf, *dst_buf;
+	struct venc_frm_buf frm_buf;
+	struct mtk_vcodec_mem bs_buf;
+	struct venc_done_result enc_result;
+	int ret;
+	struct vb2_v4l2_buffer *v4l2_vb;
+
+	if ((ctx->q_data[MTK_Q_DATA_DST].fmt->fourcc == V4L2_PIX_FMT_H264) &&
+	    (ctx->state != MTK_STATE_HEADER)) {
+		/* encode h264 sps/pps header */
+		mtk_venc_encode_header(ctx);
+		v4l2_m2m_job_finish(ctx->dev->m2m_dev_enc, ctx->m2m_ctx);
+		return;
+	}
+
+	src_buf = v4l2_m2m_next_src_buf(ctx->m2m_ctx);
+	if (!src_buf) {
+		v4l2_m2m_job_finish(ctx->dev->m2m_dev_enc, ctx->m2m_ctx);
+		return;
+	}
+
+	dst_buf = v4l2_m2m_dst_buf_remove(ctx->m2m_ctx);
+	if (!dst_buf) {
+		v4l2_m2m_job_finish(ctx->dev->m2m_dev_enc, ctx->m2m_ctx);
+		return;
+	}
+
+	mtk_venc_param_change(ctx, src_buf);
+
+	frm_buf.fb_addr.va = vb2_plane_vaddr(src_buf, 0);
+	frm_buf.fb_addr.dma_addr = vb2_dma_contig_plane_dma_addr(src_buf, 0);
+	frm_buf.fb_addr.size = (unsigned int)src_buf->planes[0].length;
+	frm_buf.fb_addr1.va = vb2_plane_vaddr(src_buf, 1);
+	frm_buf.fb_addr1.dma_addr = vb2_dma_contig_plane_dma_addr(src_buf, 1);
+	frm_buf.fb_addr1.size = (unsigned int)src_buf->planes[1].length;
+	if (src_buf->num_planes == 3) {
+		frm_buf.fb_addr2.va = vb2_plane_vaddr(src_buf, 2);
+		frm_buf.fb_addr2.dma_addr =
+			vb2_dma_contig_plane_dma_addr(src_buf, 2);
+		frm_buf.fb_addr2.size =
+			(unsigned int)src_buf->planes[2].length;
+	} else {
+		frm_buf.fb_addr2.va = NULL;
+		frm_buf.fb_addr2.dma_addr = 0;
+		frm_buf.fb_addr2.size = 0;
+	}
+	bs_buf.va = vb2_plane_vaddr(dst_buf, 0);
+	bs_buf.dma_addr = vb2_dma_contig_plane_dma_addr(dst_buf, 0);
+	bs_buf.size = (unsigned int)dst_buf->planes[0].length;
+
+	mtk_v4l2_debug(2,
+			"Framebuf VA=%p PA=%llx Size=0x%lx;VA=%p PA=0x%llx Size=0x%lx;VA=%p PA=0x%llx Size=0x%lx",
+			frm_buf.fb_addr.va,
+			(u64)frm_buf.fb_addr.dma_addr,
+			frm_buf.fb_addr.size,
+			frm_buf.fb_addr1.va,
+			(u64)frm_buf.fb_addr1.dma_addr,
+			frm_buf.fb_addr1.size,
+			frm_buf.fb_addr2.va,
+			(u64)frm_buf.fb_addr2.dma_addr,
+			frm_buf.fb_addr2.size);
+
+	ret = venc_if_encode(ctx, VENC_START_OPT_ENCODE_FRAME,
+			     &frm_buf, &bs_buf, &enc_result);
+
+	src_buf = v4l2_m2m_src_buf_remove(ctx->m2m_ctx);
+	if (enc_result.msg == VENC_MESSAGE_OK)
+		v4l2_m2m_buf_done(to_vb2_v4l2_buffer(src_buf), VB2_BUF_STATE_DONE);
+	else
+		v4l2_m2m_buf_done(to_vb2_v4l2_buffer(src_buf), VB2_BUF_STATE_ERROR);
+
+	if (enc_result.is_key_frm) {
+		v4l2_vb = to_vb2_v4l2_buffer(dst_buf);
+		v4l2_vb->flags |= V4L2_BUF_FLAG_KEYFRAME;
+	}
+
+	if (ret) {
+		dst_buf->planes[0].bytesused = 0;
+		v4l2_m2m_buf_done(to_vb2_v4l2_buffer(dst_buf), VB2_BUF_STATE_ERROR);
+		mtk_v4l2_err("venc_if_encode failed=%d", ret);
+	} else {
+		dst_buf->planes[0].bytesused = enc_result.bs_size;
+		v4l2_m2m_buf_done(to_vb2_v4l2_buffer(dst_buf), VB2_BUF_STATE_DONE);
+		mtk_v4l2_debug(2, "venc_if_encode bs size=%d",
+				 enc_result.bs_size);
+	}
+
+	v4l2_m2m_job_finish(ctx->dev->m2m_dev_enc, ctx->m2m_ctx);
+
+	mtk_v4l2_debug(1, "<=== src_buf[%d] dst_buf[%d] venc_if_encode ret=%d Size=%u===>",
+			src_buf->index, dst_buf->index, ret,
+			enc_result.bs_size);
+}
+
+static void m2mops_venc_device_run(void *priv)
+{
+	struct mtk_vcodec_ctx *ctx = priv;
+	queue_work(ctx->dev->encode_workqueue, &ctx->encode_work);
+}
+
+static int m2mops_venc_job_ready(void *m2m_priv)
+{
+	struct mtk_vcodec_ctx *ctx = m2m_priv;
+
+	if (!v4l2_m2m_num_dst_bufs_ready(ctx->m2m_ctx)) {
+		mtk_v4l2_debug(3, "[%d]Not ready: not enough video dst buffers.",
+			       ctx->idx);
+		return 0;
+	}
+
+	if (!v4l2_m2m_num_src_bufs_ready(ctx->m2m_ctx)) {
+		mtk_v4l2_debug(3, "[%d]Not ready: not enough video src buffers.",
+			       ctx->idx);
+			return 0;
+		}
+
+	if (ctx->state == MTK_STATE_ABORT) {
+		mtk_v4l2_debug(3, "[%d]Not ready: state=0x%x.",
+			       ctx->idx, ctx->state);
+		return 0;
+	}
+
+	if (ctx->state == MTK_STATE_FREE) {
+		mtk_v4l2_debug(3, "[%d]Not ready: state=0x%x.",
+			       ctx->idx, ctx->state);
+		return 0;
+	}
+
+	return 1;
+}
+
+static void m2mops_venc_job_abort(void *priv)
+{
+	struct mtk_vcodec_ctx *ctx = priv;
+
+	ctx->state = MTK_STATE_ABORT;
+}
+
+static void m2mops_venc_lock(void *m2m_priv)
+{
+	struct mtk_vcodec_ctx *ctx = m2m_priv;
+
+	mutex_lock(&ctx->dev->dev_mutex);
+}
+
+static void m2mops_venc_unlock(void *m2m_priv)
+{
+	struct mtk_vcodec_ctx *ctx = m2m_priv;
+
+	mutex_unlock(&ctx->dev->dev_mutex);
+}
+
+const struct v4l2_m2m_ops mtk_venc_m2m_ops = {
+	.device_run			= m2mops_venc_device_run,
+	.job_ready			= m2mops_venc_job_ready,
+	.job_abort			= m2mops_venc_job_abort,
+	.lock				= m2mops_venc_lock,
+	.unlock				= m2mops_venc_unlock,
+};
+
+#define IS_MTK_VENC_PRIV(x) ((V4L2_CTRL_ID2CLASS(x) == V4L2_CTRL_CLASS_MPEG) &&\
+			     V4L2_CTRL_DRIVER_PRIV(x))
+
+void mtk_vcodec_enc_ctx_params_setup(struct mtk_vcodec_ctx *ctx)
+{
+	struct mtk_q_data *q_data;
+	struct mtk_video_fmt *fmt;
+
+	ctx->m2m_ctx->q_lock = &ctx->dev->dev_mutex;
+	ctx->fh.m2m_ctx = ctx->m2m_ctx;
+	ctx->fh.ctrl_handler = &ctx->ctrl_hdl;
+	INIT_WORK(&ctx->encode_work, mtk_venc_worker);
+
+	ctx->q_data[MTK_Q_DATA_SRC].width = DFT_CFG_WIDTH;
+	ctx->q_data[MTK_Q_DATA_SRC].height = DFT_CFG_HEIGHT;
+	ctx->q_data[MTK_Q_DATA_SRC].fmt = &mtk_video_formats[0];
+	ctx->q_data[MTK_Q_DATA_SRC].colorspace = V4L2_COLORSPACE_REC709;
+	ctx->q_data[MTK_Q_DATA_SRC].field = V4L2_FIELD_NONE;
+
+	q_data = &ctx->q_data[MTK_Q_DATA_SRC];
+	fmt = ctx->q_data[MTK_Q_DATA_SRC].fmt;
+	mtk_vcodec_enc_calc_src_size(fmt->num_planes, q_data->width,
+			q_data->height,
+			ctx->q_data[MTK_Q_DATA_SRC].sizeimage,
+			ctx->q_data[MTK_Q_DATA_SRC].bytesperline);
+
+	ctx->q_data[MTK_Q_DATA_DST].width = DFT_CFG_WIDTH;
+	ctx->q_data[MTK_Q_DATA_DST].height = DFT_CFG_HEIGHT;
+	ctx->q_data[MTK_Q_DATA_DST].fmt = &mtk_video_formats[9];
+	ctx->q_data[MTK_Q_DATA_DST].colorspace = V4L2_COLORSPACE_REC709;
+	ctx->q_data[MTK_Q_DATA_DST].field = V4L2_FIELD_NONE;
+
+	q_data = &ctx->q_data[MTK_Q_DATA_DST];
+	fmt = ctx->q_data[MTK_Q_DATA_DST].fmt;
+	ctx->q_data[MTK_Q_DATA_DST].sizeimage[0] = q_data->width * q_data->height;
+	ctx->q_data[MTK_Q_DATA_DST].bytesperline[0] = 0;
+
+}
+
+int mtk_vcodec_enc_ctrls_setup(struct mtk_vcodec_ctx *ctx)
+{
+	const struct v4l2_ctrl_ops *ops = &mtk_vcodec_enc_ctrl_ops;
+	struct v4l2_ctrl_handler *handler = &ctx->ctrl_hdl;
+	struct v4l2_ctrl_config cfg;
+
+	v4l2_ctrl_handler_init(handler, MTK_MAX_CTRLS);
+	if (handler->error) {
+		mtk_v4l2_err("Init control handler fail %d\n",
+				handler->error);
+		return handler->error;
+	}
+
+	ctx->ctrls[0] = v4l2_ctrl_new_std(handler, ops,
+					V4L2_CID_MPEG_VIDEO_BITRATE,
+					1, 4000000, 1, 4000000);
+	ctx->ctrls[1] = v4l2_ctrl_new_std(handler, ops,
+					V4L2_CID_MPEG_VIDEO_B_FRAMES,
+					0, 2, 1, 0);
+	ctx->ctrls[2] = v4l2_ctrl_new_std(handler, ops,
+					V4L2_CID_MPEG_VIDEO_FRAME_RC_ENABLE,
+					0, 1, 1, 1);
+	ctx->ctrls[3] = v4l2_ctrl_new_std(handler, ops,
+					V4L2_CID_MPEG_VIDEO_H264_MAX_QP,
+					0, 51, 1, 51);
+	ctx->ctrls[4] = v4l2_ctrl_new_std(handler, ops,
+					V4L2_CID_MPEG_VIDEO_H264_I_PERIOD,
+					0, 65535, 1, 30);
+	ctx->ctrls[5] = v4l2_ctrl_new_std(handler, ops,
+					V4L2_CID_MPEG_VIDEO_GOP_SIZE,
+					0, 65535, 1, 30);
+	ctx->ctrls[6] = v4l2_ctrl_new_std(handler, ops,
+					V4L2_CID_MPEG_VIDEO_MB_RC_ENABLE,
+					0, 1, 1, 0);
+	ctx->ctrls[7] = v4l2_ctrl_new_std_menu(handler, ops,
+					V4L2_CID_MPEG_VIDEO_HEADER_MODE,
+					V4L2_MPEG_VIDEO_HEADER_MODE_JOINED_WITH_1ST_FRAME,
+					0, V4L2_MPEG_VIDEO_HEADER_MODE_SEPARATE);
+	ctx->ctrls[8] = v4l2_ctrl_new_std_menu(handler, ops,
+					V4L2_CID_MPEG_VIDEO_H264_PROFILE,
+					V4L2_MPEG_VIDEO_H264_PROFILE_HIGH,
+					0, V4L2_MPEG_VIDEO_H264_PROFILE_MAIN);
+	ctx->ctrls[9] = v4l2_ctrl_new_std_menu(handler, ops,
+					V4L2_CID_MPEG_VIDEO_H264_LEVEL,
+					V4L2_MPEG_VIDEO_H264_LEVEL_4_2,
+					0, V4L2_MPEG_VIDEO_H264_LEVEL_4_0);
+	ctx->ctrls[6] = v4l2_ctrl_new_std(handler, ops,
+					V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME,
+					0, 0, 0, 0);
+
+	v4l2_ctrl_handler_setup(&ctx->ctrl_hdl);
+
+	return 0;
+}
+
+int mtk_vcodec_enc_queue_init(void *priv, struct vb2_queue *src_vq,
+			   struct vb2_queue *dst_vq)
+{
+	struct mtk_vcodec_ctx *ctx = priv;
+	int ret;
+
+	src_vq->type		= V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE;
+	src_vq->io_modes	= VB2_DMABUF | VB2_MMAP | VB2_USERPTR;
+	src_vq->drv_priv	= ctx;
+	src_vq->buf_struct_size = sizeof(struct mtk_video_enc_buf);
+	src_vq->ops		= &mtk_venc_vb2_ops;
+	src_vq->mem_ops		= &vb2_dma_contig_memops;
+	src_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
+	src_vq->lock = &ctx->dev->dev_mutex;
+
+	ret = vb2_queue_init(src_vq);
+	if (ret)
+		return ret;
+
+	dst_vq->type		= V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
+	dst_vq->io_modes	= VB2_DMABUF | VB2_MMAP | VB2_USERPTR;
+	dst_vq->drv_priv	= ctx;
+	dst_vq->buf_struct_size = sizeof(struct v4l2_m2m_buffer);
+	dst_vq->ops		= &mtk_venc_vb2_ops;
+	dst_vq->mem_ops		= &vb2_dma_contig_memops;
+	dst_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
+	dst_vq->lock = &ctx->dev->dev_mutex;
+
+	return vb2_queue_init(dst_vq);
+}
+
+int mtk_venc_unlock(struct mtk_vcodec_ctx *ctx)
+{
+	struct mtk_vcodec_dev *dev = ctx->dev;
+	dev->curr_ctx = -1;
+	mutex_unlock(&dev->enc_mutex);
+	return 0;
+}
+
+int mtk_venc_lock(struct mtk_vcodec_ctx *ctx)
+{
+	struct mtk_vcodec_dev *dev = ctx->dev;
+
+	mutex_lock(&dev->enc_mutex);
+	dev->curr_ctx = ctx->idx;
+	return 0;
+}
+
+void mtk_vcodec_enc_release(struct mtk_vcodec_ctx *ctx)
+{
+	venc_if_release(ctx);
+}
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h
new file mode 100644
index 0000000..e09524b
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h
@@ -0,0 +1,46 @@
+/*
+* Copyright (c) 2015 MediaTek Inc.
+* Author: PC Chen <pc.chen-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
+*         Tiffany Lin <tiffany.lin-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
+*
+* This program is free software; you can redistribute it and/or modify
+* it under the terms of the GNU General Public License version 2 as
+* published by the Free Software Foundation.
+*
+* This program is distributed in the hope that it will be useful,
+* but WITHOUT ANY WARRANTY; without even the implied warranty of
+* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+* GNU General Public License for more details.
+*/
+
+
+#ifndef _MTK_VCODEC_ENC_H_
+#define _MTK_VCODEC_ENC_H_
+
+#include <media/videobuf2-core.h>
+#include <media/videobuf2-v4l2.h>
+
+/**
+ * struct mtk_video_enc_buf - Private data related to each VB2 buffer.
+ * @b:			Pointer to related VB2 buffer.
+ * @param_change:	Types of encode parameter change before encode this
+ *			buffer
+ * @enc_params		Encode parameters changed before encode this buffer
+ */
+struct mtk_video_enc_buf {
+	struct vb2_v4l2_buffer	vb;
+	struct list_head	list;
+
+	enum mtk_encode_param param_change;
+	struct mtk_enc_params enc_params;
+};
+
+int mtk_venc_unlock(struct mtk_vcodec_ctx *ctx);
+int mtk_venc_lock(struct mtk_vcodec_ctx *ctx);
+int mtk_vcodec_enc_queue_init(void *priv, struct vb2_queue *src_vq,
+					struct vb2_queue *dst_vq);
+void mtk_vcodec_enc_release(struct mtk_vcodec_ctx *ctx);
+int mtk_vcodec_enc_ctrls_setup(struct mtk_vcodec_ctx *ctx);
+void mtk_vcodec_enc_ctx_params_setup(struct mtk_vcodec_ctx *ctx);
+
+#endif /* _MTK_VCODEC_ENC_H_ */
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
new file mode 100644
index 0000000..e7ab14a
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
@@ -0,0 +1,476 @@
+/*
+* Copyright (c) 2015 MediaTek Inc.
+* Author: PC Chen <pc.chen-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
+*         Tiffany Lin <tiffany.lin-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
+*
+* This program is free software; you can redistribute it and/or modify
+* it under the terms of the GNU General Public License version 2 as
+* published by the Free Software Foundation.
+*
+* This program is distributed in the hope that it will be useful,
+* but WITHOUT ANY WARRANTY; without even the implied warranty of
+* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+* GNU General Public License for more details.
+*/
+
+#include <linux/interrupt.h>
+#include <linux/irq.h>
+#include <linux/module.h>
+#include <linux/of_device.h>
+#include <linux/of.h>
+#include <media/v4l2-event.h>
+#include <media/v4l2-mem2mem.h>
+#include <media/videobuf2-dma-contig.h>
+#include <linux/pm_runtime.h>
+
+#include "mtk_vcodec_drv.h"
+#include "mtk_vcodec_enc.h"
+#include "mtk_vcodec_pm.h"
+#include "mtk_vcodec_intr.h"
+#include "mtk_vcodec_util.h"
+#include "mtk_vpu.h"
+
+
+/* Wake up context wait_queue */
+static void wake_up_ctx(struct mtk_vcodec_ctx *ctx, unsigned int reason)
+{
+	ctx->int_cond = 1;
+	ctx->int_type = reason;
+	wake_up_interruptible(&ctx->queue);
+}
+
+static irqreturn_t mtk_vcodec_enc_irq_handler(int irq, void *priv)
+{
+	struct mtk_vcodec_dev *dev = priv;
+	struct mtk_vcodec_ctx *ctx;
+	unsigned int irq_status;
+
+	if (dev->curr_ctx == -1) {
+		mtk_v4l2_err("curr_ctx = -1");
+		return IRQ_HANDLED;
+	}
+
+	ctx = dev->ctx[dev->curr_ctx];
+	if (ctx == NULL) {
+		mtk_v4l2_err("curr_ctx==NULL");
+		return IRQ_HANDLED;
+	}
+	mtk_v4l2_debug(1, "idx=%d", ctx->idx);
+	irq_status = readl(dev->reg_base[VENC_SYS] +
+				(MTK_VENC_IRQ_STATUS_OFFSET));
+	if (irq_status & MTK_VENC_IRQ_STATUS_PAUSE)
+		writel((MTK_VENC_IRQ_STATUS_PAUSE),
+		       dev->reg_base[VENC_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
+
+	if (irq_status & MTK_VENC_IRQ_STATUS_SWITCH)
+		writel((MTK_VENC_IRQ_STATUS_SWITCH),
+		       dev->reg_base[VENC_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
+
+	if (irq_status & MTK_VENC_IRQ_STATUS_DRAM)
+		writel((MTK_VENC_IRQ_STATUS_DRAM),
+		       dev->reg_base[VENC_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
+
+	if (irq_status & MTK_VENC_IRQ_STATUS_SPS)
+		writel((MTK_VENC_IRQ_STATUS_SPS),
+		       dev->reg_base[VENC_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
+
+	if (irq_status & MTK_VENC_IRQ_STATUS_PPS)
+		writel((MTK_VENC_IRQ_STATUS_PPS),
+		       dev->reg_base[VENC_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
+
+	if (irq_status & MTK_VENC_IRQ_STATUS_FRM)
+		writel((MTK_VENC_IRQ_STATUS_FRM),
+		       dev->reg_base[VENC_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
+
+	ctx->irq_status = irq_status;
+	wake_up_ctx(ctx, MTK_INST_IRQ_RECEIVED);
+	return IRQ_HANDLED;
+}
+
+#if 1 /* VENC_LT */
+static irqreturn_t mtk_vcodec_enc_irq_handler2(int irq, void *priv)
+{
+	struct mtk_vcodec_dev *dev = priv;
+	struct mtk_vcodec_ctx *ctx;
+	unsigned int irq_status;
+
+	ctx = dev->ctx[dev->curr_ctx];
+	if (ctx == NULL) {
+		mtk_v4l2_err("ctx==NULL");
+		return IRQ_HANDLED;
+	}
+	mtk_v4l2_debug(1, "idx=%d", ctx->idx);
+	irq_status = readl(dev->reg_base[VENC_LT_SYS] +
+				(MTK_VENC_IRQ_STATUS_OFFSET));
+	if (irq_status & MTK_VENC_IRQ_STATUS_PAUSE)
+		writel((MTK_VENC_IRQ_STATUS_PAUSE),
+		       dev->reg_base[VENC_LT_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
+
+	if (irq_status & MTK_VENC_IRQ_STATUS_SWITCH)
+		writel((MTK_VENC_IRQ_STATUS_SWITCH),
+		       dev->reg_base[VENC_LT_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
+
+	if (irq_status & MTK_VENC_IRQ_STATUS_DRAM)
+		writel((MTK_VENC_IRQ_STATUS_DRAM),
+		       dev->reg_base[VENC_LT_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
+
+	if (irq_status & MTK_VENC_IRQ_STATUS_SPS)
+		writel((MTK_VENC_IRQ_STATUS_SPS),
+		       dev->reg_base[VENC_LT_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
+
+	if (irq_status & MTK_VENC_IRQ_STATUS_PPS)
+		writel((MTK_VENC_IRQ_STATUS_PPS),
+		       dev->reg_base[VENC_LT_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
+
+	if (irq_status & MTK_VENC_IRQ_STATUS_FRM)
+		writel((MTK_VENC_IRQ_STATUS_FRM),
+		       dev->reg_base[VENC_LT_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
+
+	ctx->irq_status = irq_status;
+	wake_up_ctx(ctx, MTK_INST_IRQ_RECEIVED);
+	return IRQ_HANDLED;
+}
+#endif
+
+static void mtk_vcodec_enc_reset_handler(void *priv)
+{
+	int i;
+	struct mtk_vcodec_dev *dev = priv;
+	struct mtk_vcodec_ctx *ctx;
+
+	mtk_v4l2_debug(0, "Watchdog timeout!!");
+
+	mutex_lock(&dev->dev_mutex);
+	for(i = 0; i < MTK_VCODEC_MAX_ENCODER_INSTANCES; i++) {
+		ctx = dev->ctx[i];
+		if (ctx) {
+			ctx->state = MTK_STATE_ABORT;
+			mtk_v4l2_debug(0, "[%d] Change to state MTK_STATE_ERROR", ctx->idx);
+		}
+
+	}
+	mutex_unlock(&dev->dev_mutex);
+}
+
+static int fops_vcodec_open(struct file *file)
+{
+	struct video_device *vfd = video_devdata(file);
+	struct mtk_vcodec_dev *dev = video_drvdata(file);
+	struct mtk_vcodec_ctx *ctx = NULL;
+	int ret = 0;
+
+	mutex_lock(&dev->dev_mutex);
+
+	ctx = devm_kzalloc(&dev->plat_dev->dev, sizeof(*ctx), GFP_KERNEL);
+	if (!ctx) {
+		ret = -ENOMEM;
+		goto err_alloc;
+	}
+
+	if (dev->num_instances >= MTK_VCODEC_MAX_ENCODER_INSTANCES) {
+		mtk_v4l2_err("Too many open contexts\n");
+		ret = -EBUSY;
+		goto err_no_ctx;
+	}
+
+	ctx->idx = ffz(dev->instance_mask[0]);
+	v4l2_fh_init(&ctx->fh, video_devdata(file));
+	file->private_data = &ctx->fh;
+	v4l2_fh_add(&ctx->fh);
+	ctx->dev = dev;
+
+	if (vfd == dev->vfd_enc) {
+		ctx->type = MTK_INST_ENCODER;
+
+		ret = mtk_vcodec_enc_ctrls_setup(ctx);
+		if (ret) {
+			mtk_v4l2_err("Failed to setup controls() (%d)\n",
+				       ret);
+			goto err_ctrls_setup;
+		}
+		ctx->m2m_ctx = v4l2_m2m_ctx_init(dev->m2m_dev_enc, ctx,
+						 &mtk_vcodec_enc_queue_init);
+		if (IS_ERR(ctx->m2m_ctx)) {
+			ret = PTR_ERR(ctx->m2m_ctx);
+			mtk_v4l2_err("Failed to v4l2_m2m_ctx_init() (%d)\n",
+				       ret);
+			goto err_ctx_init;
+		}
+		mtk_vcodec_enc_ctx_params_setup(ctx);
+	} else {
+		mtk_v4l2_err("Invalid vfd !\n");
+		ret = -ENOENT;
+		goto err_ctx_init;
+	}
+
+	init_waitqueue_head(&ctx->queue);
+	dev->num_instances++;
+
+	if (dev->num_instances == 1) {
+		ret = vpu_load_firmware(dev->vpu_plat_dev);
+		if (ret < 0) {
+				mtk_v4l2_err("vpu_load_firmware failed!\n");
+			goto err_load_fw;
+		}
+
+		dev->enc_capability =
+			vpu_get_venc_hw_capa(dev->vpu_plat_dev);
+		mtk_v4l2_debug(0, "encoder capability %x", dev->enc_capability);
+	}
+
+	mtk_v4l2_debug(2, "Create instance [%d]@%p m2m_ctx=%p type=%d\n",
+			 ctx->idx, ctx, ctx->m2m_ctx, ctx->type);
+	set_bit(ctx->idx, &dev->instance_mask[0]);
+	dev->ctx[ctx->idx] = ctx;
+
+	mutex_unlock(&dev->dev_mutex);
+	mtk_v4l2_debug(0, "%s encoder [%d]", dev_name(&dev->plat_dev->dev), ctx->idx);
+	return ret;
+
+	/* Deinit when failure occurred */
+err_load_fw:
+	v4l2_m2m_ctx_release(ctx->m2m_ctx);
+	v4l2_fh_del(&ctx->fh);
+	v4l2_fh_exit(&ctx->fh);
+	dev->num_instances--;
+err_ctx_init:
+err_ctrls_setup:
+	v4l2_ctrl_handler_free(&ctx->ctrl_hdl);
+err_no_ctx:
+	devm_kfree(&dev->plat_dev->dev, ctx);
+err_alloc:
+	mutex_unlock(&dev->dev_mutex);
+	return ret;
+}
+
+static int fops_vcodec_release(struct file *file)
+{
+	struct mtk_vcodec_dev *dev = video_drvdata(file);
+	struct mtk_vcodec_ctx *ctx = fh_to_ctx(file->private_data);
+
+	mtk_v4l2_debug(0, "[%d] encoder\n", ctx->idx);
+	mutex_lock(&dev->dev_mutex);
+
+	mtk_vcodec_enc_release(ctx);
+	v4l2_fh_del(&ctx->fh);
+	v4l2_fh_exit(&ctx->fh);
+	v4l2_ctrl_handler_free(&ctx->ctrl_hdl);
+	v4l2_m2m_ctx_release(ctx->m2m_ctx);
+
+	dev->ctx[ctx->idx] = NULL;
+	dev->num_instances--;
+	clear_bit(ctx->idx, &dev->instance_mask[0]);
+	devm_kfree(&dev->plat_dev->dev, ctx);
+	mutex_unlock(&dev->dev_mutex);
+	return 0;
+}
+
+static const struct v4l2_file_operations mtk_vcodec_fops = {
+	.owner				= THIS_MODULE,
+	.open				= fops_vcodec_open,
+	.release			= fops_vcodec_release,
+	.poll				= v4l2_m2m_fop_poll,
+	.unlocked_ioctl			= video_ioctl2,
+	.mmap				= v4l2_m2m_fop_mmap,
+};
+
+static int mtk_vcodec_probe(struct platform_device *pdev)
+{
+	struct mtk_vcodec_dev *dev;
+	struct video_device *vfd_enc;
+	struct resource *res;
+	int i, j, ret;
+
+	dev = devm_kzalloc(&pdev->dev, sizeof(*dev), GFP_KERNEL);
+	if (!dev)
+		return -ENOMEM;
+
+	dev->plat_dev = pdev;
+
+	dev->vpu_plat_dev = vpu_get_plat_device(dev->plat_dev);
+	if (dev->vpu_plat_dev == NULL) {
+		mtk_v4l2_err("[VPU] vpu device in not ready\n");
+		return -EPROBE_DEFER;
+	}
+
+	vpu_wdt_reg_handler(dev->vpu_plat_dev, mtk_vcodec_enc_reset_handler, dev,
+			    VPU_RST_ENC);
+
+	ret = mtk_vcodec_init_enc_pm(dev);
+	if (ret < 0) {
+		dev_err(&pdev->dev, "Failed to get mt vcodec clock source!\n");
+		return ret;
+	}
+
+	for (i = VENC_SYS, j = 0; i < NUM_MAX_VCODEC_REG_BASE; i++, j++) {
+		res = platform_get_resource(pdev, IORESOURCE_MEM, j);
+		if (res == NULL) {
+			dev_err(&pdev->dev, "get memory resource failed.\n");
+			ret = -ENXIO;
+			goto err_res;
+		}
+		dev->reg_base[i] = devm_ioremap_resource(&pdev->dev, res);
+		if (IS_ERR(dev->reg_base[i])) {
+			dev_err(&pdev->dev,
+				"devm_ioremap_resource %d failed.\n", i);
+			ret = PTR_ERR(dev->reg_base);
+			goto err_res;
+		}
+		mtk_v4l2_debug(2, "reg[%d] base=0x%p\n", i, dev->reg_base[i]);
+	}
+
+	res = platform_get_resource(pdev, IORESOURCE_IRQ, 0);
+	if (res == NULL) {
+		dev_err(&pdev->dev, "failed to get irq resource\n");
+		ret = -ENOENT;
+		goto err_res;
+	}
+
+	dev->enc_irq = platform_get_irq(pdev, 0);
+	ret = devm_request_irq(&pdev->dev, dev->enc_irq,
+			       mtk_vcodec_enc_irq_handler,
+			       0, pdev->name, dev);
+	if (ret) {
+		dev_err(&pdev->dev, "Failed to install dev->enc_irq %d (%d)\n",
+			dev->enc_irq,
+			ret);
+		ret = -EINVAL;
+		goto err_res;
+	}
+
+	dev->enc_lt_irq = platform_get_irq(pdev, 1);
+	ret = devm_request_irq(&pdev->dev,
+			       dev->enc_lt_irq, mtk_vcodec_enc_irq_handler2,
+			       0, pdev->name, dev);
+	if (ret) {
+		dev_err(&pdev->dev,
+			"Failed to install dev->enc_lt_irq %d (%d)\n",
+			dev->enc_lt_irq, ret);
+		ret = -EINVAL;
+		goto err_res;
+	}
+
+	disable_irq(dev->enc_irq);
+	disable_irq(dev->enc_lt_irq); /* VENC_LT */
+	mutex_init(&dev->enc_mutex);
+	mutex_init(&dev->dev_mutex);
+
+	snprintf(dev->v4l2_dev.name, sizeof(dev->v4l2_dev.name), "%s",
+		 "[MTK_V4L2_VENC]");
+
+	ret = v4l2_device_register(&pdev->dev, &dev->v4l2_dev);
+	if (ret) {
+		mtk_v4l2_err("v4l2_device_register err=%d\n", ret);
+		return ret;
+	}
+
+	init_waitqueue_head(&dev->queue);
+
+	/* allocate video device for encoder and register it */
+	vfd_enc = video_device_alloc();
+	if (!vfd_enc) {
+		mtk_v4l2_err("Failed to allocate video device\n");
+		ret = -ENOMEM;
+		goto err_enc_alloc;
+	}
+	vfd_enc->fops           = &mtk_vcodec_fops;
+	vfd_enc->ioctl_ops      = &mtk_venc_ioctl_ops;
+	vfd_enc->release        = video_device_release;
+	vfd_enc->lock           = &dev->dev_mutex;
+	vfd_enc->v4l2_dev       = &dev->v4l2_dev;
+	vfd_enc->vfl_dir        = VFL_DIR_M2M;
+
+	snprintf(vfd_enc->name, sizeof(vfd_enc->name), "%s",
+		 MTK_VCODEC_ENC_NAME);
+	video_set_drvdata(vfd_enc, dev);
+	dev->vfd_enc = vfd_enc;
+	platform_set_drvdata(pdev, dev);
+	ret = video_register_device(vfd_enc, VFL_TYPE_GRABBER, 1);
+	if (ret) {
+		mtk_v4l2_err("Failed to register video device\n");
+		goto err_enc_reg;
+	}
+	mtk_v4l2_debug(0, "encoder registered as /dev/video%d\n",
+			 vfd_enc->num);
+
+	dev->alloc_ctx = vb2_dma_contig_init_ctx(&pdev->dev);
+	if (IS_ERR(dev->alloc_ctx)) {
+		mtk_v4l2_err("Failed to alloc vb2 dma context 0\n");
+		ret = PTR_ERR(dev->alloc_ctx);
+		goto err_vb2_ctx_init;
+	}
+
+	dev->m2m_dev_enc = v4l2_m2m_init(&mtk_venc_m2m_ops);
+	if (IS_ERR(dev->m2m_dev_enc)) {
+		mtk_v4l2_err("Failed to init mem2mem enc device\n");
+		ret = PTR_ERR(dev->m2m_dev_enc);
+		goto err_enc_mem_init;
+	}
+
+	dev->encode_workqueue =
+			alloc_ordered_workqueue(MTK_VCODEC_ENC_NAME, WQ_MEM_RECLAIM | WQ_FREEZABLE);
+	if (!dev->encode_workqueue) {
+		mtk_v4l2_err("Failed to create encode workqueue\n");
+		ret = -EINVAL;
+		goto err_event_workq;
+	}
+
+	return 0;
+
+err_event_workq:
+	v4l2_m2m_release(dev->m2m_dev_enc);
+err_enc_mem_init:
+	vb2_dma_contig_cleanup_ctx(dev->alloc_ctx);
+err_vb2_ctx_init:
+	video_unregister_device(vfd_enc);
+err_enc_reg:
+	video_device_release(vfd_enc);
+err_enc_alloc:
+	v4l2_device_unregister(&dev->v4l2_dev);
+err_res:
+	mtk_vcodec_release_enc_pm(dev);
+	return ret;
+}
+
+static const struct of_device_id mtk_vcodec_match[] = {
+	{.compatible = "mediatek,mt8173-vcodec-enc",},
+	{},
+};
+MODULE_DEVICE_TABLE(of, mtk_vcodec_match);
+
+static int mtk_vcodec_remove(struct platform_device *pdev)
+{
+	struct mtk_vcodec_dev *dev = platform_get_drvdata(pdev);
+
+	mtk_v4l2_debug_enter();
+	flush_workqueue(dev->encode_workqueue);
+	destroy_workqueue(dev->encode_workqueue);
+	if (dev->m2m_dev_enc)
+		v4l2_m2m_release(dev->m2m_dev_enc);
+	if (dev->alloc_ctx)
+		vb2_dma_contig_cleanup_ctx(dev->alloc_ctx);
+
+	if (dev->vfd_enc) {
+		video_unregister_device(dev->vfd_enc);
+		video_device_release(dev->vfd_enc);
+	}
+	v4l2_device_unregister(&dev->v4l2_dev);
+	mtk_vcodec_release_enc_pm(dev);
+	return 0;
+}
+
+static struct platform_driver mtk_vcodec_driver = {
+	.probe	= mtk_vcodec_probe,
+	.remove	= mtk_vcodec_remove,
+	.driver	= {
+		.name	= MTK_VCODEC_ENC_NAME,
+		.owner	= THIS_MODULE,
+		.of_match_table = mtk_vcodec_match,
+	},
+};
+
+module_platform_driver(mtk_vcodec_driver);
+
+
+MODULE_LICENSE("GPL v2");
+MODULE_DESCRIPTION("Mediatek video codec V4L2 driver");
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
new file mode 100644
index 0000000..518fba7
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
@@ -0,0 +1,132 @@
+/*
+* Copyright (c) 2015 MediaTek Inc.
+* Author: Tiffany Lin <tiffany.lin-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
+*
+* This program is free software; you can redistribute it and/or modify
+* it under the terms of the GNU General Public License version 2 as
+* published by the Free Software Foundation.
+*
+* This program is distributed in the hope that it will be useful,
+* but WITHOUT ANY WARRANTY; without even the implied warranty of
+* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+* GNU General Public License for more details.
+*/
+
+#include <linux/clk.h>
+#include <linux/of_address.h>
+#include <linux/of_platform.h>
+#include <linux/pm_runtime.h>
+#include <soc/mediatek/smi.h>
+
+#include "mtk_vcodec_pm.h"
+#include "mtk_vcodec_util.h"
+#include "mtk_vpu.h"
+
+
+int mtk_vcodec_init_enc_pm(struct mtk_vcodec_dev *mtkdev)
+{
+	struct device_node *node;
+	struct platform_device *pdev;
+	struct device *dev;
+	struct mtk_vcodec_pm *pm;
+	int ret = 0;
+
+	pdev = mtkdev->plat_dev;
+	pm = &mtkdev->pm;
+	memset(pm, 0, sizeof(struct mtk_vcodec_pm));
+	pm->mtkdev = mtkdev;
+	dev = &pdev->dev;
+
+	node = of_parse_phandle(dev->of_node, "mediatek,larb", 0);
+	if (!node)
+		return -1;
+	pdev = of_find_device_by_node(node);
+	if (WARN_ON(!pdev)) {
+		of_node_put(node);
+		return -1;
+	}
+	pm->larbvenc = &pdev->dev;
+
+	node = of_parse_phandle(dev->of_node, "mediatek,larb", 1);
+	if (!node)
+		return -1;
+
+	pdev = of_find_device_by_node(node);
+	if (WARN_ON(!pdev)) {
+		of_node_put(node);
+		return -EINVAL;
+	}
+	pm->larbvenclt = &pdev->dev;
+
+	pdev = mtkdev->plat_dev;
+	pm->dev = &pdev->dev;
+
+	pm->vencpll_d2 = devm_clk_get(&pdev->dev, "vencpll_d2");
+	if (pm->vencpll_d2 == NULL) {
+		mtk_v4l2_err("devm_clk_get vencpll_d2 fail");
+		ret = -1;
+	}
+
+	pm->venc_sel = devm_clk_get(&pdev->dev, "venc_sel");
+	if (pm->venc_sel == NULL) {
+		mtk_v4l2_err("devm_clk_get venc_sel fail");
+		ret = -1;
+	}
+
+	pm->univpll1_d2 = devm_clk_get(&pdev->dev, "univpll1_d2");
+	if (pm->univpll1_d2 == NULL) {
+		mtk_v4l2_err("devm_clk_get univpll1_d2 fail");
+		ret = -1;
+	}
+
+	pm->venc_lt_sel = devm_clk_get(&pdev->dev, "venc_lt_sel");
+	if (pm->venc_lt_sel == NULL) {
+		mtk_v4l2_err("devm_clk_get venc_lt_sel fail");
+		ret = -1;
+	}
+
+	return ret;
+}
+
+void mtk_vcodec_release_enc_pm(struct mtk_vcodec_dev *mtkdev)
+{
+}
+
+
+void mtk_vcodec_enc_clock_on(struct mtk_vcodec_pm *pm)
+{
+	int ret;
+
+	ret = clk_prepare_enable(pm->venc_sel);
+	if (ret)
+		mtk_v4l2_err("venc_sel fail %d", ret);
+
+	ret = clk_set_parent(pm->venc_sel, pm->vencpll_d2);
+	if (ret)
+		mtk_v4l2_err("clk_set_parent fail %d", ret);
+
+	ret = clk_prepare_enable(pm->venc_lt_sel);
+	if (ret)
+		mtk_v4l2_err("venc_lt_sel fail %d", ret);
+
+	ret = clk_set_parent(pm->venc_lt_sel, pm->univpll1_d2);
+	if (ret)
+		mtk_v4l2_err("clk_set_parent fail %d", ret);
+
+	ret = mtk_smi_larb_get(pm->larbvenc);
+	if (ret)
+		mtk_v4l2_err("mtk_smi_larb_get larb3 fail %d\n", ret);
+
+	ret = mtk_smi_larb_get(pm->larbvenclt);
+	if (ret)
+		mtk_v4l2_err("mtk_smi_larb_get larb4 fail %d\n", ret);
+
+}
+
+void mtk_vcodec_enc_clock_off(struct mtk_vcodec_pm *pm)
+{
+	mtk_smi_larb_put(pm->larbvenc);
+	mtk_smi_larb_put(pm->larbvenclt);
+	clk_disable_unprepare(pm->venc_lt_sel);
+	clk_disable_unprepare(pm->venc_sel);
+}
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.c
new file mode 100644
index 0000000..919b949
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.c
@@ -0,0 +1,102 @@
+/*
+* Copyright (c) 2015 MediaTek Inc.
+* Author: Tiffany Lin <tiffany.lin-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
+*
+* This program is free software; you can redistribute it and/or modify
+* it under the terms of the GNU General Public License version 2 as
+* published by the Free Software Foundation.
+*
+* This program is distributed in the hope that it will be useful,
+* but WITHOUT ANY WARRANTY; without even the implied warranty of
+* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+* GNU General Public License for more details.
+*/
+
+#include <linux/errno.h>
+#include <linux/wait.h>
+
+#include "mtk_vcodec_drv.h"
+#include "mtk_vcodec_intr.h"
+#include "mtk_vcodec_util.h"
+
+void mtk_vcodec_clean_dev_int_flags(void *data)
+{
+	struct mtk_vcodec_dev *dev = (struct mtk_vcodec_dev *)data;
+
+	dev->int_cond = 0;
+	dev->int_type = 0;
+}
+
+int mtk_vcodec_wait_for_done_ctx(void *data, int command,
+				 unsigned int timeout_ms, int interrupt)
+{
+	wait_queue_head_t *waitqueue;
+	long timeout_jiff, ret;
+	int status = 0;
+	struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)data;
+
+	waitqueue = (wait_queue_head_t *)&ctx->queue;
+	timeout_jiff = msecs_to_jiffies(timeout_ms);
+	if (interrupt) {
+		ret = wait_event_interruptible_timeout(*waitqueue,
+				(ctx->int_cond &&
+				(ctx->int_type == command)),
+				timeout_jiff);
+	} else {
+		ret = wait_event_timeout(*waitqueue,
+				(ctx->int_cond &&
+				(ctx->int_type == command)),
+				 timeout_jiff);
+	}
+	if (0 == ret) {
+		status = -1;	/* timeout */
+		mtk_v4l2_err("[%d] cmd=%d, ctx->type=%d, wait_event_interruptible_timeout time=%ums out %d %d!",
+				ctx->idx, ctx->type, command, timeout_ms,
+				ctx->int_cond, ctx->int_type);
+	} else if (-ERESTARTSYS == ret) {
+		mtk_v4l2_err("[%d] cmd=%d, ctx->type=%d, wait_event_interruptible_timeout interrupted by a signal %d %d",
+				ctx->idx, ctx->type, command, ctx->int_cond,
+				ctx->int_type);
+		status = -1;
+	}
+
+	ctx->int_cond = 0;
+	ctx->int_type = 0;
+
+	return status;
+}
+
+int mtk_vcodec_wait_for_done_dev(void *data, int command,
+				 unsigned int timeout, int interrupt)
+{
+	wait_queue_head_t *waitqueue;
+	long timeout_jiff, ret;
+	int status = 0;
+	struct mtk_vcodec_dev *dev = (struct mtk_vcodec_dev *)data;
+
+	waitqueue = (wait_queue_head_t *)&dev->queue;
+	timeout_jiff = msecs_to_jiffies(timeout);
+	if (interrupt) {
+		ret = wait_event_interruptible_timeout(*waitqueue,
+				(dev->int_cond &&
+				(dev->int_type == command)),
+				timeout_jiff);
+	} else {
+		ret = wait_event_timeout(*waitqueue,
+				(dev->int_cond &&
+				(dev->int_type == command)),
+				timeout_jiff);
+	}
+	if (0 == ret) {
+		status = -1;	/* timeout */
+		mtk_v4l2_err("wait_event_interruptible_timeout time=%lu out %d %d!",
+				timeout_jiff, dev->int_cond, dev->int_type);
+	} else if (-ERESTARTSYS == ret) {
+		mtk_v4l2_err("wait_event_interruptible_timeout interrupted by a signal %d %d",
+				dev->int_cond, dev->int_type);
+		status = -1;
+	}
+	dev->int_cond = 0;
+	dev->int_type = 0;
+	return status;
+}
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.h
new file mode 100644
index 0000000..e9b7f94
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.h
@@ -0,0 +1,29 @@
+/*
+* Copyright (c) 2015 MediaTek Inc.
+* Author: Tiffany Lin <tiffany.lin-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
+*
+* This program is free software; you can redistribute it and/or modify
+* it under the terms of the GNU General Public License version 2 as
+* published by the Free Software Foundation.
+*
+* This program is distributed in the hope that it will be useful,
+* but WITHOUT ANY WARRANTY; without even the implied warranty of
+* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+* GNU General Public License for more details.
+*/
+
+#ifndef _MTK_VCODEC_INTR_H_
+#define _MTK_VCODEC_INTR_H_
+
+#define MTK_INST_IRQ_RECEIVED		0x1
+#define MTK_INST_WORK_THREAD_ABORT_DONE	0x2
+
+/* timeout is ms */
+int mtk_vcodec_wait_for_done_ctx(void *data, int command, unsigned int timeout,
+				 int interrupt);
+int mtk_vcodec_wait_for_done_dev(void *data, int command, unsigned int timeout,
+				 int interrupt);
+
+void mtk_vcodec_clean_dev_int_flags(void *data);
+
+#endif /* _MTK_VCODEC_INTR_H_ */
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_pm.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_pm.h
new file mode 100644
index 0000000..fdadec9
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_pm.h
@@ -0,0 +1,26 @@
+/*
+* Copyright (c) 2015 MediaTek Inc.
+* Author: Tiffany Lin <tiffany.lin-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
+*
+* This program is free software; you can redistribute it and/or modify
+* it under the terms of the GNU General Public License version 2 as
+* published by the Free Software Foundation.
+*
+* This program is distributed in the hope that it will be useful,
+* but WITHOUT ANY WARRANTY; without even the implied warranty of
+* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+* GNU General Public License for more details.
+*/
+
+#ifndef _MTK_VCODEC_PM_H_
+#define _MTK_VCODEC_PM_H_
+
+#include "mtk_vcodec_drv.h"
+
+int mtk_vcodec_init_enc_pm(struct mtk_vcodec_dev *dev);
+void mtk_vcodec_release_enc_pm(struct mtk_vcodec_dev *dev);
+
+void mtk_vcodec_enc_clock_on(struct mtk_vcodec_pm *pm);
+void mtk_vcodec_enc_clock_off(struct mtk_vcodec_pm *pm);
+
+#endif /* _MTK_VCODEC_PM_H_ */
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c
new file mode 100644
index 0000000..3fede8d
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c
@@ -0,0 +1,106 @@
+/*
+* Copyright (c) 2015 MediaTek Inc.
+* Author: PC Chen <pc.chen-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
+*         Tiffany Lin <tiffany.lin-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
+*
+* This program is free software; you can redistribute it and/or modify
+* it under the terms of the GNU General Public License version 2 as
+* published by the Free Software Foundation.
+*
+* This program is distributed in the hope that it will be useful,
+* but WITHOUT ANY WARRANTY; without even the implied warranty of
+* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+* GNU General Public License for more details.
+*/
+
+#include <linux/module.h>
+
+#include "mtk_vcodec_drv.h"
+#include "mtk_vcodec_util.h"
+#include "mtk_vpu.h"
+
+bool mtk_vcodec_dbg = false;
+int mtk_v4l2_dbg_level = 0;
+
+module_param(mtk_v4l2_dbg_level, int, S_IRUGO | S_IWUSR);
+module_param(mtk_vcodec_dbg, bool, S_IRUGO | S_IWUSR);
+
+void __iomem *mtk_vcodec_get_reg_addr(void *data, unsigned int reg_idx)
+{
+	struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)data;
+
+	if (!data || reg_idx >= NUM_MAX_VCODEC_REG_BASE) {
+		mtk_v4l2_err("Invalid arguments");
+		return NULL;
+	}
+	return ctx->dev->reg_base[reg_idx];
+}
+
+int mtk_vcodec_mem_alloc(void *data, struct mtk_vcodec_mem *mem)
+{
+	unsigned long size = mem->size;
+	struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)data;
+	struct device *dev = &ctx->dev->plat_dev->dev;
+
+	mem->va = dma_alloc_coherent(dev, size, &mem->dma_addr, GFP_KERNEL);
+
+	if (!mem->va) {
+		mtk_v4l2_err("%s dma_alloc size=%ld failed!", dev_name(dev),
+			       size);
+		return -ENOMEM;
+	}
+
+	memset(mem->va, 0, size);
+
+	mtk_v4l2_debug(3, "[%d]  - va      = %p", ctx->idx, mem->va);
+	mtk_v4l2_debug(3, "[%d]  - dma     = 0x%lx", ctx->idx,
+			 (unsigned long)mem->dma_addr);
+	mtk_v4l2_debug(3, "[%d]    size = 0x%lx", ctx->idx, size);
+
+	return 0;
+}
+
+void mtk_vcodec_mem_free(void *data, struct mtk_vcodec_mem *mem)
+{
+	unsigned long size = mem->size;
+	struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)data;
+	struct device *dev = &ctx->dev->plat_dev->dev;
+
+	dma_free_coherent(dev, size, mem->va, mem->dma_addr);
+	mem->va = NULL;
+
+	mtk_v4l2_debug(3, "[%d]  - va      = %p", ctx->idx, mem->va);
+	mtk_v4l2_debug(3, "[%d]  - dma     = 0x%lx", ctx->idx,
+			 (unsigned long)mem->dma_addr);
+	mtk_v4l2_debug(3, "[%d]    size = 0x%lx", ctx->idx, size);
+}
+
+int mtk_vcodec_get_ctx_id(void *data)
+{
+	struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)data;
+
+	if (!ctx)
+		return -1;
+
+	return ctx->idx;
+}
+
+struct platform_device *mtk_vcodec_get_plat_dev(void *data)
+{
+	struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)data;
+
+	if (!ctx)
+		return NULL;
+
+	return vpu_get_plat_device(ctx->dev->plat_dev);
+}
+
+void mtk_vcodec_fmt2str(u32 fmt, char *str)
+{
+	char a = fmt & 0xFF;
+	char b = (fmt >> 8) & 0xFF;
+	char c = (fmt >> 16) & 0xFF;
+	char d = (fmt >> 24) & 0xFF;
+
+	sprintf(str, "%c%c%c%c", a, b, c, d);
+}
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
new file mode 100644
index 0000000..47016ae
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
@@ -0,0 +1,85 @@
+/*
+* Copyright (c) 2015 MediaTek Inc.
+* Author: PC Chen <pc.chen-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
+*         Tiffany Lin <tiffany.lin-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
+*
+* This program is free software; you can redistribute it and/or modify
+* it under the terms of the GNU General Public License version 2 as
+* published by the Free Software Foundation.
+*
+* This program is distributed in the hope that it will be useful,
+* but WITHOUT ANY WARRANTY; without even the implied warranty of
+* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+* GNU General Public License for more details.
+*/
+
+#ifndef _MTK_VCODEC_UTIL_H_
+#define _MTK_VCODEC_UTIL_H_
+
+#include <linux/types.h>
+#include <linux/dma-direction.h>
+
+struct mtk_vcodec_mem {
+	size_t size;
+	void *va;
+	dma_addr_t dma_addr;
+};
+
+extern int mtk_v4l2_dbg_level;
+extern bool mtk_vcodec_dbg;
+
+#define DEBUG 	1
+
+#if defined(DEBUG)
+
+#define mtk_v4l2_debug(level, fmt, args...)				 \
+	do {								 \
+		if (mtk_v4l2_dbg_level >= level)			 \
+			pr_info("[MTK_V4L2] level=%d %s(),%d: " fmt "\n",\
+				level, __func__, __LINE__, ##args);	 \
+	} while (0)
+
+#define mtk_v4l2_err(fmt, args...)                \
+	pr_err("[MTK_V4L2][ERROR] %s:%d: " fmt "\n", __func__, __LINE__, \
+	       ##args)
+
+
+#define mtk_v4l2_debug_enter()  mtk_v4l2_debug(3, "+\n")
+#define mtk_v4l2_debug_leave()  mtk_v4l2_debug(3, "-\n")
+
+#define mtk_vcodec_debug(h, fmt, args...)				\
+	do {								\
+		if (mtk_vcodec_dbg)					\
+			pr_info("[MTK_VCODEC][%d]: %s() " fmt "\n",	\
+				((struct mtk_vcodec_ctx *)h->ctx)->idx, \
+				__func__, ##args);			\
+	} while (0)
+
+#define mtk_vcodec_err(h, fmt, args...)					\
+	pr_err("[MTK_VCODEC][ERROR][%d]: %s() " fmt "\n",		\
+	       ((struct mtk_vcodec_ctx *)h->ctx)->idx, __func__, ##args)
+
+#define mtk_vcodec_debug_enter(h)  mtk_vcodec_debug(h, "+\n")
+#define mtk_vcodec_debug_leave(h)  mtk_vcodec_debug(h, "-\n")
+
+#else
+
+#define mtk_v4l2_debug(level, fmt, args...)
+#define mtk_v4l2_err(fmt, args...)
+#define mtk_v4l2_debug_enter()
+#define mtk_v4l2_debug_leave()
+
+#define mtk_vcodec_debug(h, fmt, args...)
+#define mtk_vcodec_err(h, fmt, args...)
+#define mtk_vcodec_debug_enter(h)
+#define mtk_vcodec_debug_leave(h)
+
+#endif
+
+void __iomem *mtk_vcodec_get_reg_addr(void *data, unsigned int reg_idx);
+int mtk_vcodec_mem_alloc(void *data, struct mtk_vcodec_mem *mem);
+void mtk_vcodec_mem_free(void *data, struct mtk_vcodec_mem *mem);
+int mtk_vcodec_get_ctx_id(void *data);
+struct platform_device *mtk_vcodec_get_plat_dev(void *data);
+void mtk_vcodec_fmt2str(u32 fmt, char *str);
+#endif /* _MTK_VCODEC_UTIL_H_ */
diff --git a/drivers/media/platform/mtk-vcodec/venc_drv_base.h b/drivers/media/platform/mtk-vcodec/venc_drv_base.h
new file mode 100644
index 0000000..ed9cbf0
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/venc_drv_base.h
@@ -0,0 +1,62 @@
+/*
+ * Copyright (c) 2015 MediaTek Inc.
+ * Author: Daniel Hsiao <daniel.hsiao-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
+ *         Jungchang Tsao <jungchang.tsao-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef _VENC_DRV_BASE_
+#define _VENC_DRV_BASE_
+
+#include "mtk_vcodec_drv.h"
+
+#include "venc_drv_if.h"
+
+struct venc_common_if {
+	/**
+	 * (*init)() - initialize driver
+	 * @ctx:	[in] mtk v4l2 context
+	 * @handle: [out] driver handle
+	 */
+	int (*init)(struct mtk_vcodec_ctx *ctx, unsigned long *handle);
+
+	/**
+	 * (*encode)() - trigger encode
+	 * @handle: [in] driver handle
+	 * @opt: [in] encode option
+	 * @frm_buf: [in] frame buffer to store input frame
+	 * @bs_buf: [in] bitstream buffer to store output bitstream
+	 * @result: [out] encode result
+	 */
+	int (*encode)(unsigned long handle, enum venc_start_opt opt,
+		      struct venc_frm_buf *frm_buf,
+		      struct mtk_vcodec_mem *bs_buf,
+		      struct venc_done_result *result);
+
+	/**
+	 * (*set_param)() - set driver's parameter
+	 * @handle: [in] driver handle
+	 * @type: [in] parameter type
+	 * @in: [in] buffer to store the parameter
+	 */
+	int (*set_param)(unsigned long handle, enum venc_set_param_type type,
+			 void *in);
+
+	/**
+	 * (*deinit)() - deinitialize driver.
+	 * @handle: [in] driver handle
+	 */
+	int (*deinit)(unsigned long handle);
+};
+
+
+#endif
diff --git a/drivers/media/platform/mtk-vcodec/venc_drv_if.c b/drivers/media/platform/mtk-vcodec/venc_drv_if.c
new file mode 100644
index 0000000..daa8e93
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/venc_drv_if.c
@@ -0,0 +1,100 @@
+/*
+ * Copyright (c) 2015 MediaTek Inc.
+ * Author: Daniel Hsiao <daniel.hsiao-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
+ *         Jungchang Tsao <jungchang.tsao-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/interrupt.h>
+#include <linux/kernel.h>
+#include <linux/slab.h>
+
+#include "venc_drv_if.h"
+#include "mtk_vcodec_enc.h"
+#include "mtk_vcodec_pm.h"
+#include "mtk_vpu.h"
+
+#include "venc_drv_base.h"
+
+int venc_if_create(struct mtk_vcodec_ctx *ctx, unsigned int fourcc)
+{
+	char str[10];
+	int ret = 0;
+
+	mtk_vcodec_fmt2str(fourcc, str);
+
+	switch (fourcc) {
+	case V4L2_PIX_FMT_VP8:
+	case V4L2_PIX_FMT_H264:
+	default:
+		return -EINVAL;
+	}
+
+	mtk_venc_lock(ctx);
+	mtk_vcodec_enc_clock_on(&ctx->dev->pm);
+	ret = ctx->enc_if->init(ctx, (unsigned long *)&ctx->drv_handle);
+	mtk_vcodec_enc_clock_off(&ctx->dev->pm);
+	mtk_venc_unlock(ctx);
+
+	return ret;
+
+}
+
+
+int venc_if_set_param(struct mtk_vcodec_ctx *ctx,
+		      enum venc_set_param_type type, void *in)
+{
+	int ret = 0;
+
+	mtk_venc_lock(ctx);
+	mtk_vcodec_enc_clock_on(&ctx->dev->pm);
+	ret = ctx->enc_if->set_param(ctx->drv_handle, type, in);
+	mtk_vcodec_enc_clock_off(&ctx->dev->pm);
+	mtk_venc_unlock(ctx);
+
+	return ret;
+}
+
+int venc_if_encode(struct mtk_vcodec_ctx *ctx,
+		   enum venc_start_opt opt, struct venc_frm_buf *frm_buf,
+		   struct mtk_vcodec_mem *bs_buf,
+		   struct venc_done_result *result)
+{
+	int ret = 0;
+	
+	mtk_venc_lock(ctx);
+	mtk_vcodec_enc_clock_on(&ctx->dev->pm);
+	ret = ctx->enc_if->encode(ctx->drv_handle, opt, frm_buf, bs_buf, result);
+	mtk_vcodec_enc_clock_off(&ctx->dev->pm);
+	mtk_venc_unlock(ctx);
+	
+	return ret;
+}
+
+int venc_if_release(struct mtk_vcodec_ctx *ctx)
+{
+	int ret = 0;
+
+	if(ctx->drv_handle == 0)
+		return 0;
+
+	mtk_venc_lock(ctx);
+	mtk_vcodec_enc_clock_on(&ctx->dev->pm);
+	ret = ctx->enc_if->deinit(ctx->drv_handle);
+	mtk_vcodec_enc_clock_off(&ctx->dev->pm);
+	mtk_venc_unlock(ctx);
+
+	ctx->drv_handle = 0;
+
+	return ret;
+}
+
diff --git a/drivers/media/platform/mtk-vcodec/venc_drv_if.h b/drivers/media/platform/mtk-vcodec/venc_drv_if.h
new file mode 100644
index 0000000..a387011
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/venc_drv_if.h
@@ -0,0 +1,175 @@
+/*
+ * Copyright (c) 2015 MediaTek Inc.
+ * Author: Daniel Hsiao <daniel.hsiao-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
+ *         Jungchang Tsao <jungchang.tsao-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef _VENC_DRV_IF_H_
+#define _VENC_DRV_IF_H_
+
+#include "mtk_vcodec_drv.h"
+#include "mtk_vcodec_util.h"
+
+/*
+ * enum venc_yuv_fmt - The type of input yuv format
+ * (VPU related: If you change the order, you must also update the VPU codes.)
+ * @VENC_YUV_FORMAT_420: 420 YUV format
+ * @VENC_YUV_FORMAT_YV12: YV12 YUV format
+ * @VENC_YUV_FORMAT_NV12: NV12 YUV format
+ * @VENC_YUV_FORMAT_NV21: NV21 YUV format
+ */
+enum venc_yuv_fmt {
+	VENC_YUV_FORMAT_420 = 3,
+	VENC_YUV_FORMAT_YV12 = 5,
+	VENC_YUV_FORMAT_NV12 = 6,
+	VENC_YUV_FORMAT_NV21 = 7,
+};
+
+/*
+ * enum venc_start_opt - encode frame option used in venc_if_encode()
+ * @VENC_START_OPT_ENCODE_SEQUENCE_HEADER: encode SPS/PPS for H264
+ * @VENC_START_OPT_ENCODE_FRAME: encode normal frame
+ */
+enum venc_start_opt {
+	VENC_START_OPT_ENCODE_SEQUENCE_HEADER,
+	VENC_START_OPT_ENCODE_FRAME,
+};
+
+/*
+ * enum venc_drv_msg - The type of encode frame status used in venc_if_encode()
+ * @VENC_MESSAGE_OK: encode ok
+ * @VENC_MESSAGE_ERR: encode error
+ */
+enum venc_drv_msg {
+	VENC_MESSAGE_OK,
+	VENC_MESSAGE_ERR,
+};
+
+/*
+ * enum venc_set_param_type - The type of set parameter used in venc_if_set_param()
+ * (VPU related: If you change the order, you must also update the VPU codes.)
+ * @VENC_SET_PARAM_ENC: set encoder parameters
+ * @VENC_SET_PARAM_FORCE_INTRA: set force intra frame
+ * @VENC_SET_PARAM_ADJUST_BITRATE: set to adjust bitrate (in bps)
+ * @VENC_SET_PARAM_ADJUST_FRAMERATE: set frame rate
+ * @VENC_SET_PARAM_I_FRAME_INTERVAL: set I frame interval
+ * @VENC_SET_PARAM_SKIP_FRAME: set H264 skip one frame
+ * @VENC_SET_PARAM_PREPEND_HEADER: set H264 prepend SPS/PPS before IDR
+ * @VENC_SET_PARAM_TS_MODE: set VP8 temporal scalability mode
+ */
+enum venc_set_param_type {
+	VENC_SET_PARAM_ENC,
+	VENC_SET_PARAM_FORCE_INTRA,
+	VENC_SET_PARAM_ADJUST_BITRATE,
+	VENC_SET_PARAM_ADJUST_FRAMERATE,
+	VENC_SET_PARAM_I_FRAME_INTERVAL,
+	VENC_SET_PARAM_SKIP_FRAME,
+	VENC_SET_PARAM_PREPEND_HEADER,
+	VENC_SET_PARAM_TS_MODE,
+};
+
+/*
+ * struct venc_enc_prm - encoder settings for VENC_SET_PARAM_ENC used in venc_if_set_param()
+ * @input_fourcc: input fourcc
+ * @h264_profile: V4L2 defined H.264 profile
+ * @h264_level: V4L2 defined H.264 level
+ * @width: image width
+ * @height: image height
+ * @buf_width: buffer width
+ * @buf_height: buffer height
+ * @frm_rate: frame rate
+ * @intra_period: intra frame period
+ * @bitrate: target bitrate in kbps
+ */
+struct venc_enc_prm {
+	enum venc_yuv_fmt input_fourcc;
+	unsigned int h264_profile;
+	unsigned int h264_level;
+	unsigned int width;
+	unsigned int height;
+	unsigned int buf_width;
+	unsigned int buf_height;
+	unsigned int frm_rate;
+	unsigned int intra_period;
+	unsigned int bitrate;
+	unsigned int sizeimage[MTK_VCODEC_MAX_PLANES];
+};
+
+/*
+ * struct venc_frm_buf - frame buffer information used in venc_if_encode()
+ * @fb_addr: plane 0 frame buffer address
+ * @fb_addr1: plane 1 frame buffer address
+ * @fb_addr2: plane 2 frame buffer address
+ */
+struct venc_frm_buf {
+	struct mtk_vcodec_mem fb_addr;
+	struct mtk_vcodec_mem fb_addr1;
+	struct mtk_vcodec_mem fb_addr2;
+};
+
+/*
+ * struct venc_done_result - This is return information used in venc_if_encode()
+ * @msg: message, such as success or error code
+ * @bs_size: output bitstream size
+ * @is_key_frm: output is key frame or not
+ */
+struct venc_done_result {
+	enum venc_drv_msg msg;
+	unsigned int bs_size;
+	bool is_key_frm;
+};
+
+/*
+ * venc_if_create - Create the driver handle
+ * @ctx: device context
+ * @fourcc: encoder output format
+ * @handle: driver handle
+ * Return: 0 if creating handle successfully, otherwise it is failed.
+ */
+int venc_if_create(struct mtk_vcodec_ctx *ctx, unsigned int fourcc);
+
+/*
+ * venc_if_release - Release the driver handle
+ * @handle: driver handle
+ * Return: 0 if releasing handle successfully, otherwise it is failed.
+ */
+int venc_if_release(struct mtk_vcodec_ctx *ctx);
+
+/*
+ * venc_if_set_param - Set parameter to driver
+ * @handle: driver handle
+ * @type: set type
+ * @in: input parameter
+ * @out: output parameter
+ * Return: 0 if setting param successfully, otherwise it is failed.
+ */
+int venc_if_set_param(struct mtk_vcodec_ctx *ctx,
+		      enum venc_set_param_type type,
+		      void *in);
+
+/*
+ * venc_if_encode - Encode frame
+ * @handle: driver handle
+ * @opt: encode frame option
+ * @frm_buf: input frame buffer information
+ * @bs_buf: output bitstream buffer infomraiton
+ * @result: encode result
+ * Return: 0 if encoding frame successfully, otherwise it is failed.
+ */
+int venc_if_encode(struct mtk_vcodec_ctx *ctx,
+		   enum venc_start_opt opt,
+		   struct venc_frm_buf *frm_buf,
+		   struct mtk_vcodec_mem *bs_buf,
+		   struct venc_done_result *result);
+
+#endif /* _VENC_DRV_IF_H_ */
diff --git a/drivers/media/platform/mtk-vcodec/venc_ipi_msg.h b/drivers/media/platform/mtk-vcodec/venc_ipi_msg.h
new file mode 100644
index 0000000..a345b98
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/venc_ipi_msg.h
@@ -0,0 +1,212 @@
+/*
+ * Copyright (c) 2015 MediaTek Inc.
+ * Author: Jungchang Tsao <jungchang.tsao-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
+ *         Daniel Hsiao <daniel.hsiao-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef _VENC_IPI_MSG_H_
+#define _VENC_IPI_MSG_H_
+
+#define IPIMSG_H264_ENC_ID 0x100
+#define IPIMSG_VP8_ENC_ID 0x200
+
+#define AP_IPIMSG_VENC_BASE 0x20000
+#define VPU_IPIMSG_VENC_BASE 0x30000
+
+/**
+ * enum venc_ipi_msg_id - message id between AP and VPU
+ * (ipi stands for inter-processor interrupt)
+ * @AP_IPIMSG_XXX:		AP to VPU cmd message id
+ * @VPU_IPIMSG_XXX_DONE:	VPU ack AP cmd message id
+ */
+enum venc_ipi_msg_id {
+	AP_IPIMSG_H264_ENC_INIT = AP_IPIMSG_VENC_BASE +
+				  IPIMSG_H264_ENC_ID,
+	AP_IPIMSG_H264_ENC_SET_PARAM,
+	AP_IPIMSG_H264_ENC_ENCODE,
+	AP_IPIMSG_H264_ENC_DEINIT,
+
+	AP_IPIMSG_VP8_ENC_INIT = AP_IPIMSG_VENC_BASE +
+				 IPIMSG_VP8_ENC_ID,
+	AP_IPIMSG_VP8_ENC_SET_PARAM,
+	AP_IPIMSG_VP8_ENC_ENCODE,
+	AP_IPIMSG_VP8_ENC_DEINIT,
+
+	VPU_IPIMSG_H264_ENC_INIT_DONE = VPU_IPIMSG_VENC_BASE +
+					IPIMSG_H264_ENC_ID,
+	VPU_IPIMSG_H264_ENC_SET_PARAM_DONE,
+	VPU_IPIMSG_H264_ENC_ENCODE_DONE,
+	VPU_IPIMSG_H264_ENC_DEINIT_DONE,
+
+	VPU_IPIMSG_VP8_ENC_INIT_DONE = VPU_IPIMSG_VENC_BASE +
+				       IPIMSG_VP8_ENC_ID,
+	VPU_IPIMSG_VP8_ENC_SET_PARAM_DONE,
+	VPU_IPIMSG_VP8_ENC_ENCODE_DONE,
+	VPU_IPIMSG_VP8_ENC_DEINIT_DONE,
+};
+
+/**
+ * struct venc_ap_ipi_msg_init - AP to VPU init cmd structure
+ * @msg_id:	message id (AP_IPIMSG_XXX_ENC_INIT)
+ * @venc_inst:	AP encoder instance (struct venc_vp8_handle/venc_h264_handle *)
+ */
+struct venc_ap_ipi_msg_init {
+	uint32_t msg_id;
+	uint32_t reserved;
+	uint64_t venc_inst;
+};
+
+/**
+ * struct venc_ap_ipi_msg_set_param - AP to VPU set_param cmd structure
+ * @msg_id:	message id (AP_IPIMSG_XXX_ENC_SET_PARAM)
+ * @inst_id:	VPU encoder instance id (struct venc_vp8_vpu_drv/venc_h264_vpu_drv *)
+ * @param_id:	parameter id (venc_set_param_type)
+ * @data_item:	number of items in the data array
+ * @data[8]:	data array to store the set parameters
+ */
+struct venc_ap_ipi_msg_set_param {
+	uint32_t msg_id;
+	uint32_t inst_id;
+	uint32_t param_id;
+	uint32_t data_item;
+	uint32_t data[8];
+};
+
+/**
+ * struct venc_ap_ipi_msg_enc - AP to VPU enc cmd structure
+ * @msg_id:	message id (AP_IPIMSG_XXX_ENC_ENCODE)
+ * @inst_id:	VPU encoder instance id (struct venc_vp8_vpu_drv/venc_h264_vpu_drv *)
+ * @bs_mode:	bitstream mode for h264
+ *		(H264_BS_MODE_SPS/H264_BS_MODE_PPS/H264_BS_MODE_FRAME)
+ * @input_addr:	pointer to input image buffer plane
+ * @bs_addr:	pointer to output bit stream buffer
+ * @bs_size:	bit stream buffer size
+ */
+struct venc_ap_ipi_msg_enc {
+	uint32_t msg_id;
+	uint32_t inst_id;
+	uint32_t bs_mode;
+	uint32_t input_addr[3];
+	uint32_t bs_addr;
+	uint32_t bs_size;
+};
+
+/**
+ * struct venc_ap_ipi_msg_deinit - AP to VPU deinit cmd structure
+ * @msg_id:	message id (AP_IPIMSG_XXX_ENC_DEINIT)
+ * @inst_id:	VPU encoder instance id (struct venc_vp8_vpu_drv/venc_h264_vpu_drv *)
+ */
+struct venc_ap_ipi_msg_deinit {
+	uint32_t msg_id;
+	uint32_t inst_id;
+};
+
+/**
+ * enum venc_ipi_msg_status - VPU ack AP cmd status
+ */
+enum venc_ipi_msg_status {
+	VENC_IPI_MSG_STATUS_OK,
+	VENC_IPI_MSG_STATUS_FAIL,
+};
+
+/**
+ * struct venc_vpu_ipi_msg_common - VPU ack AP cmd common structure
+ * @msg_id:	message id (VPU_IPIMSG_XXX_DONE)
+ * @status:	cmd status (venc_ipi_msg_status)
+ * @venc_inst:	AP encoder instance (struct venc_vp8_handle/venc_h264_handle *)
+ */
+struct venc_vpu_ipi_msg_common {
+	uint32_t msg_id;
+	uint32_t status;
+	uint64_t venc_inst;
+};
+
+/**
+ * struct venc_vpu_ipi_msg_init - VPU ack AP init cmd structure
+ * @msg_id:	message id (VPU_IPIMSG_XXX_ENC_SET_PARAM_DONE)
+ * @status:	cmd status (venc_ipi_msg_status)
+ * @venc_inst:	AP encoder instance (struct venc_vp8_handle/venc_h264_handle *)
+ * @inst_id:	VPU encoder instance id (struct venc_vp8_vpu_drv/venc_h264_vpu_drv *)
+ */
+struct venc_vpu_ipi_msg_init {
+	uint32_t msg_id;
+	uint32_t status;
+	uint64_t venc_inst;
+	uint32_t inst_id;
+	uint32_t reserved;
+};
+
+/**
+ * struct venc_vpu_ipi_msg_set_param - VPU ack AP set_param cmd structure
+ * @msg_id:	message id (VPU_IPIMSG_XXX_ENC_SET_PARAM_DONE)
+ * @status:	cmd status (venc_ipi_msg_status)
+ * @venc_inst:	AP encoder instance (struct venc_vp8_handle/venc_h264_handle *)
+ * @param_id:	parameter id (venc_set_param_type)
+ * @data_item:	number of items in the data array
+ * @data[6]:	data array to store the return result
+ */
+struct venc_vpu_ipi_msg_set_param {
+	uint32_t msg_id;
+	uint32_t status;
+	uint64_t venc_inst;
+	uint32_t param_id;
+	uint32_t data_item;
+	uint32_t data[6];
+};
+
+/**
+ * enum venc_ipi_msg_enc_state - Type of encode state
+ * VEN_IPI_MSG_ENC_STATE_FRAME:	one frame being encoded
+ * VEN_IPI_MSG_ENC_STATE_PART:	bit stream buffer full
+ * VEN_IPI_MSG_ENC_STATE_SKIP:	encoded skip frame
+ * VEN_IPI_MSG_ENC_STATE_ERROR:	encounter error
+ */
+enum venc_ipi_msg_enc_state {
+	VEN_IPI_MSG_ENC_STATE_FRAME,
+	VEN_IPI_MSG_ENC_STATE_PART,
+	VEN_IPI_MSG_ENC_STATE_SKIP,
+	VEN_IPI_MSG_ENC_STATE_ERROR,
+};
+
+/**
+ * struct venc_vpu_ipi_msg_enc - VPU ack AP enc cmd structure
+ * @msg_id:	message id (VPU_IPIMSG_XXX_ENC_ENCODE_DONE)
+ * @status:	cmd status (venc_ipi_msg_status)
+ * @venc_inst:	AP encoder instance (struct venc_vp8_handle/venc_h264_handle *)
+ * @state:	encode state (venc_ipi_msg_enc_state)
+ * @key_frame:	whether the encoded frame is key frame
+ * @bs_size:	encoded bitstream size
+ */
+struct venc_vpu_ipi_msg_enc {
+	uint32_t msg_id;
+	uint32_t status;
+	uint64_t venc_inst;
+	uint32_t state;
+	uint32_t key_frame;
+	uint32_t bs_size;
+	uint32_t reserved;
+};
+
+/**
+ * struct venc_vpu_ipi_msg_deinit - VPU ack AP deinit cmd structure
+ * @msg_id:   message id (VPU_IPIMSG_XXX_ENC_DEINIT_DONE)
+ * @status:   cmd status (venc_ipi_msg_status)
+ * @venc_inst:	AP encoder instance (struct venc_vp8_handle/venc_h264_handle *)
+ */
+struct venc_vpu_ipi_msg_deinit {
+	uint32_t msg_id;
+	uint32_t status;
+	uint64_t venc_inst;
+};
+
+#endif /* _VENC_IPI_MSG_H_ */
diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
old mode 100644
new mode 100755
index ee9d530..3ac35c4
--- a/include/uapi/linux/v4l2-controls.h
+++ b/include/uapi/linux/v4l2-controls.h
@@ -646,6 +646,10 @@ enum v4l2_mpeg_mfc51_video_force_frame_type {
 #define V4L2_CID_MPEG_MFC51_VIDEO_H264_NUM_REF_PIC_FOR_P		(V4L2_CID_MPEG_MFC51_BASE+54)
 
 
+#define V4L2_CID_MPEG_MTK_BASE					(V4L2_CTRL_CLASS_MPEG | 0x5500)
+#define V4L2_CID_MPEG_MTK_VIDEO_FRAME_SKIP_ENABLE		(V4L2_CID_MPEG_MTK_BASE+0)
+#define V4L2_CID_MPEG_MTK_VIDEO_FORCE_FRAME_TYPE_I_FRAME	(V4L2_CID_MPEG_MTK_BASE+1)
+
 /*  Camera class control IDs */
 
 #define V4L2_CID_CAMERA_CLASS_BASE 	(V4L2_CTRL_CLASS_CAMERA | 0x900)
-- 
1.7.9.5

--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply related	[flat|nested] 102+ messages in thread

* [PATCH v4 6/8] [media] vcodec: mediatek: Add Mediatek VP8 Video Encoder Driver
  2016-02-04 11:35           ` Tiffany Lin
  (?)
@ 2016-02-04 11:35             ` Tiffany Lin
  -1 siblings, 0 replies; 102+ messages in thread
From: Tiffany Lin @ 2016-02-04 11:35 UTC (permalink / raw)
  To: Hans Verkuil, daniel.thompson, Rob Herring,
	Mauro Carvalho Chehab, Matthias Brugger, Daniel Kurtz,
	Pawel Osciak
  Cc: Eddie Huang, Yingjoe Chen, devicetree, linux-kernel,
	linux-arm-kernel, linux-media, linux-mediatek, PoChun.Lin,
	Tiffany Lin, Daniel Hsiao

Add vp8 encoder driver for MT8173

Signed-off-by: Daniel Hsiao <daniel.hsiao@mediatek.com>
Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
---
 drivers/media/platform/mtk-vcodec/Makefile         |    2 +
 drivers/media/platform/mtk-vcodec/venc_drv_if.c    |    3 +
 drivers/media/platform/mtk-vcodec/vp8_enc/Makefile |    6 +
 .../platform/mtk-vcodec/vp8_enc/venc_vp8_if.c      |  422 ++++++++++++++++++++
 .../platform/mtk-vcodec/vp8_enc/venc_vp8_if.h      |  149 +++++++
 .../platform/mtk-vcodec/vp8_enc/venc_vp8_vpu.c     |  240 +++++++++++
 .../platform/mtk-vcodec/vp8_enc/venc_vp8_vpu.h     |   28 ++
 7 files changed, 850 insertions(+)
 create mode 100644 drivers/media/platform/mtk-vcodec/vp8_enc/Makefile
 create mode 100644 drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_if.c
 create mode 100644 drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_if.h
 create mode 100644 drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_vpu.c
 create mode 100644 drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_vpu.h

diff --git a/drivers/media/platform/mtk-vcodec/Makefile b/drivers/media/platform/mtk-vcodec/Makefile
index ce38689..f4ef502 100644
--- a/drivers/media/platform/mtk-vcodec/Makefile
+++ b/drivers/media/platform/mtk-vcodec/Makefile
@@ -5,4 +5,6 @@ obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += mtk_vcodec_intr.o \
 				       mtk_vcodec_enc_pm.o \
 				       venc_drv_if.o
 
+obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += vp8_enc/
+
 ccflags-y += -I$(srctree)/drivers/media/platform/mtk-vpu
diff --git a/drivers/media/platform/mtk-vcodec/venc_drv_if.c b/drivers/media/platform/mtk-vcodec/venc_drv_if.c
index daa8e93..d293f2c 100644
--- a/drivers/media/platform/mtk-vcodec/venc_drv_if.c
+++ b/drivers/media/platform/mtk-vcodec/venc_drv_if.c
@@ -24,6 +24,7 @@
 #include "mtk_vpu.h"
 
 #include "venc_drv_base.h"
+#include "vp8_enc/venc_vp8_if.h"
 
 int venc_if_create(struct mtk_vcodec_ctx *ctx, unsigned int fourcc)
 {
@@ -34,6 +35,8 @@ int venc_if_create(struct mtk_vcodec_ctx *ctx, unsigned int fourcc)
 
 	switch (fourcc) {
 	case V4L2_PIX_FMT_VP8:
+                ctx->enc_if = get_vp8_enc_comm_if();
+                break;
 	case V4L2_PIX_FMT_H264:
 	default:
 		return -EINVAL;
diff --git a/drivers/media/platform/mtk-vcodec/vp8_enc/Makefile b/drivers/media/platform/mtk-vcodec/vp8_enc/Makefile
new file mode 100644
index 0000000..0f515c7
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/vp8_enc/Makefile
@@ -0,0 +1,6 @@
+obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += venc_vp8_if.o venc_vp8_vpu.o
+
+ccflags-y += \
+    -I$(srctree)/drivers/media/platform/mtk-vcodec/ \
+    -I$(srctree)/drivers/media/platform/mtk-vcodec/vp8_enc \
+    -I$(srctree)/drivers/media/platform/mtk-vpu
diff --git a/drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_if.c b/drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_if.c
new file mode 100644
index 0000000..4253317
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_if.c
@@ -0,0 +1,422 @@
+/*
+ * Copyright (c) 2015 MediaTek Inc.
+ * Author: Daniel Hsiao <daniel.hsiao@mediatek.com>
+ *         PoChun Lin <pochun.lin@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/interrupt.h>
+#include <linux/kernel.h>
+#include <linux/slab.h>
+
+#include "mtk_vcodec_drv.h"
+#include "mtk_vcodec_util.h"
+#include "mtk_vcodec_intr.h"
+#include "mtk_vcodec_enc.h"
+#include "mtk_vcodec_pm.h"
+#include "mtk_vpu.h"
+
+#include "venc_vp8_if.h"
+#include "venc_vp8_vpu.h"
+
+#define VENC_PIC_BITSTREAM_BYTE_CNT 0x0098
+#define VENC_PIC_BITSTREAM_BYTE_CNT1 0x00e8
+#define VENC_IRQ_STATUS_ENC_FRM_INT 0x04
+
+#define MAX_AC_TAG_SZ 10
+
+static inline void vp8_enc_write_reg(struct venc_vp8_inst *inst, u32 addr,
+				     u32 val)
+{
+	writel(val, inst->hw_base + addr);
+}
+
+static inline u32 vp8_enc_read_reg(struct venc_vp8_inst *inst, u32 addr)
+{
+	return readl(inst->hw_base + addr);
+}
+
+static void vp8_enc_free_work_buf(struct venc_vp8_inst *inst)
+{
+	int i;
+
+	mtk_vcodec_debug_enter(inst);
+
+	/* Buffers need to be freed by AP. */
+	for (i = 0; i < VENC_VP8_VPU_WORK_BUF_MAX; i++)
+		if (inst->work_bufs[i].va != NULL)
+			mtk_vcodec_mem_free(inst->ctx,
+					    &inst->work_bufs[i]);
+
+	mtk_vcodec_debug_leave(inst);
+}
+
+static int vp8_enc_alloc_work_buf(struct venc_vp8_inst *inst, void *param)
+{
+	int i;
+	int ret = 0;
+	struct venc_vp8_vpu_buf *wb = inst->vpu_inst.drv->work_bufs;
+	struct venc_enc_prm *enc_param = param;
+
+	mtk_vcodec_debug_enter(inst);
+
+	for (i = 0; i < VENC_VP8_VPU_WORK_BUF_MAX; i++) {
+		/*
+		 * Only temporal scalability mode will use RC_CODE2 & RC_CODE3
+		 * Each three temporal layer has its own rate control code.
+		 */
+		if ((i == VENC_VP8_VPU_WORK_BUF_RC_CODE2 ||
+		     i == VENC_VP8_VPU_WORK_BUF_RC_CODE3) && !inst->ts_mode)
+			continue;
+
+		/*
+		 * This 'wb' structure is set by VPU side and shared to AP for
+		 * buffer allocation and IO virtual addr mapping. For most of
+		 * the buffers, AP will allocate the buffer according to 'size'
+		 * field and store the IO virtual addr in 'iova' field. For the
+		 * RC_CODEx buffers, they are pre-allocated in the VPU side
+		 * because they are inside VPU SRAM, and save the VPU addr in
+		 * the 'vpua' field. The AP will translate the VPU addr to the
+		 * corresponding IO virtual addr and store in 'iova' field.
+		 */
+		if (i < VENC_VP8_VPU_WORK_BUF_RC_CODE) {
+			inst->work_bufs[i].size = wb[i].size;
+			ret = mtk_vcodec_mem_alloc(inst->ctx,
+						   &inst->work_bufs[i]);
+			if (ret) {
+				mtk_vcodec_err(inst,
+					       "cannot alloc work_bufs[%d]", i);
+				goto err_alloc;
+			}
+			wb[i].iova = inst->work_bufs[i].dma_addr;
+		} else if (i == VENC_VP8_VPU_WORK_BUF_SRC_LUMA ||
+			   i == VENC_VP8_VPU_WORK_BUF_SRC_CHROMA ||
+			   i == VENC_VP8_VPU_WORK_BUF_SRC_CHROMA_CB ||
+			   i == VENC_VP8_VPU_WORK_BUF_SRC_CHROMA_CR) {
+			inst->work_bufs[i].size = wb[i].size;
+			inst->work_bufs[i].dma_addr = 0;
+			inst->work_bufs[i].va = NULL;
+			wb[i].iova = inst->work_bufs[i].dma_addr;
+		} else {
+			void *tmp_va;
+
+			tmp_va = vpu_mapping_dm_addr(inst->dev, wb[i].vpua);
+			inst->work_bufs[i].size = wb[i].size;
+			ret = mtk_vcodec_mem_alloc(inst->ctx,
+						   &inst->work_bufs[i]);
+			if (ret) {
+				mtk_vcodec_err(inst,
+					       "cannot alloc work_bufs[%d]", i);
+				goto err_alloc;
+			}
+			memcpy(inst->work_bufs[i].va, tmp_va, wb[i].size);
+			wb[i].iova = inst->work_bufs[i].dma_addr;
+		}
+		mtk_vcodec_debug(inst,
+				 "work_bufs[%d] va=0x%p,iova=0x%p,size=0x%lx",
+				 i, inst->work_bufs[i].va,
+				 (void *)inst->work_bufs[i].dma_addr,
+				 inst->work_bufs[i].size);
+	}
+
+	if (enc_param->input_fourcc == VENC_YUV_FORMAT_NV12 ||
+	    enc_param->input_fourcc == VENC_YUV_FORMAT_NV21) {
+		enc_param->sizeimage[0] =
+			inst->work_bufs[VENC_VP8_VPU_WORK_BUF_SRC_LUMA].size;
+		enc_param->sizeimage[1] =
+			inst->work_bufs[VENC_VP8_VPU_WORK_BUF_SRC_CHROMA].size;
+		enc_param->sizeimage[2] = 0;
+	} else {
+		enc_param->sizeimage[0] =
+			inst->work_bufs[VENC_VP8_VPU_WORK_BUF_SRC_LUMA].size;
+		enc_param->sizeimage[1] =
+			inst->work_bufs[VENC_VP8_VPU_WORK_BUF_SRC_CHROMA_CB].size;
+		enc_param->sizeimage[2] =
+			inst->work_bufs[VENC_VP8_VPU_WORK_BUF_SRC_CHROMA_CR].size;
+	}
+	mtk_vcodec_debug_leave(inst);
+
+	return ret;
+
+err_alloc:
+	vp8_enc_free_work_buf(inst);
+
+	return ret;
+}
+
+static unsigned int vp8_enc_wait_venc_done(struct venc_vp8_inst *inst)
+{
+	unsigned int irq_status = 0;
+	struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)inst->ctx;
+
+	mtk_vcodec_wait_for_done_ctx(ctx, MTK_INST_IRQ_RECEIVED,
+				     WAIT_INTR_TIMEOUT, true);
+	irq_status = ctx->irq_status;
+	mtk_vcodec_debug(inst, "isr return %x", irq_status);
+
+	return irq_status;
+}
+
+/*
+ * Compose ac_tag, bitstream header and bitstream payload into
+ * one bitstream buffer.
+ */
+static int vp8_enc_compose_one_frame(struct venc_vp8_inst *inst,
+				     struct mtk_vcodec_mem *bs_buf,
+				     unsigned int *bs_size)
+{
+	unsigned int is_key;
+	u32 bs_size_frm;
+	u32 bs_hdr_len;
+	unsigned int ac_tag_sz;
+	u8 ac_tag[MAX_AC_TAG_SZ];
+
+	bs_size_frm = vp8_enc_read_reg(inst,
+				       VENC_PIC_BITSTREAM_BYTE_CNT);
+	bs_hdr_len = vp8_enc_read_reg(inst,
+				      VENC_PIC_BITSTREAM_BYTE_CNT1);
+
+	/* if a frame is key frame, is_key is 0 */
+	is_key = !inst->is_key_frm;
+	*(u32 *)ac_tag = __cpu_to_le32((bs_hdr_len << 5) | 0x10 | is_key);
+	/* key frame */
+	if (is_key == 0) {
+		ac_tag[3] = 0x9d;
+		ac_tag[4] = 0x01;
+		ac_tag[5] = 0x2a;
+		ac_tag[6] = inst->vpu_inst.drv->config.pic_w;
+		ac_tag[7] = inst->vpu_inst.drv->config.pic_w >> 8;
+		ac_tag[8] = inst->vpu_inst.drv->config.pic_h;
+		ac_tag[9] = inst->vpu_inst.drv->config.pic_h >> 8;
+	}
+
+	if (is_key == 0)
+		ac_tag_sz = MAX_AC_TAG_SZ;
+	else
+		ac_tag_sz = 3;
+
+	if (bs_buf->size <= bs_hdr_len + bs_size_frm + ac_tag_sz) {
+		mtk_vcodec_err(inst, "bitstream buf size is too small(%ld)",
+			       bs_buf->size);
+		return -EINVAL;
+	}
+
+	/*
+	* (1) The vp8 bitstream header and body are generated by the HW vp8
+	* encoder separately at the same time. We cannot know the bitstream
+	* header length in advance.
+	* (2) From the vp8 spec, there is no stuffing byte allowed between the
+	* ac tag, bitstream header and bitstream body.
+	*/
+	memmove(bs_buf->va + bs_hdr_len + ac_tag_sz,
+		bs_buf->va, bs_size_frm);
+	memcpy(bs_buf->va + ac_tag_sz,
+	       inst->work_bufs[VENC_VP8_VPU_WORK_BUF_BS_HD].va,
+	       bs_hdr_len);
+	memcpy(bs_buf->va, ac_tag, ac_tag_sz);
+	*bs_size = bs_size_frm + bs_hdr_len + ac_tag_sz;
+
+	return 0;
+}
+
+static int vp8_enc_encode_frame(struct venc_vp8_inst *inst,
+				struct venc_frm_buf *frm_buf,
+				struct mtk_vcodec_mem *bs_buf,
+				unsigned int *bs_size)
+{
+	int ret = 0;
+	unsigned int irq_status;
+
+	mtk_vcodec_debug(inst, "->frm_cnt=%d", inst->frm_cnt);
+
+	ret = vp8_enc_vpu_encode(inst, frm_buf, bs_buf);
+	if (ret)
+		return ret;
+
+	irq_status = vp8_enc_wait_venc_done(inst);
+	if (irq_status != VENC_IRQ_STATUS_ENC_FRM_INT) {
+		mtk_vcodec_err(inst, "irq_status=%d failed", irq_status);
+		return -EINVAL;
+	}
+
+	if (vp8_enc_compose_one_frame(inst, bs_buf, bs_size)) {
+		mtk_vcodec_err(inst, "vp8_enc_compose_one_frame failed");
+		return -EINVAL;
+	}
+
+	inst->frm_cnt++;
+	mtk_vcodec_debug(inst, "<-size=%d key_frm=%d", *bs_size,
+			 inst->is_key_frm);
+
+	return ret;
+}
+
+static int vp8_enc_init(struct mtk_vcodec_ctx *ctx, unsigned long *handle)
+{
+	int ret = 0;
+	struct venc_vp8_inst *inst;
+
+	inst = kzalloc(sizeof(*inst), GFP_KERNEL);
+	if (!inst)
+		return -ENOMEM;
+
+	inst->ctx = ctx;
+	inst->dev = mtk_vcodec_get_plat_dev(ctx);
+	inst->hw_base = mtk_vcodec_get_reg_addr(inst->ctx, VENC_LT_SYS);
+
+	mtk_vcodec_debug_enter(inst);
+	ret = vp8_enc_vpu_init(inst);
+	if (ret)
+		kfree(inst);
+	else
+		(*handle) = (unsigned long)inst;
+
+	mtk_vcodec_debug_leave(inst);
+
+	return ret;
+}
+
+static int vp8_enc_encode(unsigned long handle,
+			  enum venc_start_opt opt,
+			  struct venc_frm_buf *frm_buf,
+			  struct mtk_vcodec_mem *bs_buf,
+			  struct venc_done_result *result)
+{
+	int ret = 0;
+	struct venc_vp8_inst *inst = (struct venc_vp8_inst *)handle;
+	struct mtk_vcodec_ctx *ctx = inst->ctx;
+
+	mtk_vcodec_debug_enter(inst);
+
+	enable_irq(ctx->dev->enc_lt_irq);
+
+	switch (opt) {
+	case VENC_START_OPT_ENCODE_FRAME:
+		ret = vp8_enc_encode_frame(inst, frm_buf, bs_buf,
+					   &result->bs_size);
+		if (ret)
+			goto encode_err;
+		result->is_key_frm = inst->is_key_frm;
+		break;
+
+	default:
+		mtk_vcodec_err(inst, "opt not support:%d", opt);
+		ret = -EINVAL;
+		break;
+	}
+
+encode_err:
+	if (ret)
+		result->msg = VENC_MESSAGE_ERR;
+	else
+		result->msg = VENC_MESSAGE_OK;
+
+	disable_irq(ctx->dev->enc_lt_irq);
+	mtk_vcodec_debug_leave(inst);
+
+	return ret;
+}
+
+static int vp8_enc_set_param(unsigned long handle,
+			     enum venc_set_param_type type, void *in)
+{
+	int ret = 0;
+	struct venc_vp8_inst *inst = (struct venc_vp8_inst *)handle;
+	struct venc_enc_prm *enc_prm;
+
+	mtk_vcodec_debug(inst, "->type=%d", type);
+
+	switch (type) {
+	case VENC_SET_PARAM_ENC:
+		enc_prm = in;
+		ret = vp8_enc_vpu_set_param(inst, type, enc_prm);
+		if (ret)
+			break;
+		if (inst->work_buf_allocated == 1) {
+			vp8_enc_free_work_buf(inst);
+			inst->work_buf_allocated = 0;
+		}
+		if (inst->work_buf_allocated == 0) {
+			ret = vp8_enc_alloc_work_buf(inst, enc_prm);
+			if (ret)
+				break;
+			inst->work_buf_allocated = 1;
+		}
+		break;
+
+	case VENC_SET_PARAM_FORCE_INTRA:
+		ret = vp8_enc_vpu_set_param(inst, type, 0);
+		break;
+
+	case VENC_SET_PARAM_ADJUST_BITRATE:
+		enc_prm = in;
+		ret = vp8_enc_vpu_set_param(inst, type, &enc_prm->bitrate);
+		break;
+
+	case VENC_SET_PARAM_ADJUST_FRAMERATE:
+		enc_prm = in;
+		ret = vp8_enc_vpu_set_param(inst, type, &enc_prm->frm_rate);
+		break;
+
+	case VENC_SET_PARAM_I_FRAME_INTERVAL:
+		ret = vp8_enc_vpu_set_param(inst, type, in);
+		break;
+
+	/*
+	 * VENC_SET_PARAM_TS_MODE must be called before
+	 * VENC_SET_PARAM_ENC
+	 */
+	case VENC_SET_PARAM_TS_MODE:
+		inst->ts_mode = 1;
+		mtk_vcodec_debug(inst, "set ts_mode");
+		break;
+
+	default:
+		mtk_vcodec_err(inst, "type not support:%d", type);
+		ret = -EINVAL;
+		break;
+	}
+
+	mtk_vcodec_debug_leave(inst);
+
+	return ret;
+}
+
+static int vp8_enc_deinit(unsigned long handle)
+{
+	int ret = 0;
+	struct venc_vp8_inst *inst = (struct venc_vp8_inst *)handle;
+
+	mtk_vcodec_debug_enter(inst);
+
+	ret = vp8_enc_vpu_deinit(inst);
+
+	if (inst->work_buf_allocated)
+		vp8_enc_free_work_buf(inst);
+
+	mtk_vcodec_debug_leave(inst);
+	kfree(inst);
+
+	return ret;
+}
+
+static struct venc_common_if venc_vp8_if = {
+	vp8_enc_init,
+	vp8_enc_encode,
+	vp8_enc_set_param,
+	vp8_enc_deinit,
+};
+
+struct venc_common_if *get_vp8_enc_comm_if(void)
+{
+	return &venc_vp8_if;
+}
diff --git a/drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_if.h b/drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_if.h
new file mode 100644
index 0000000..b32220e
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_if.h
@@ -0,0 +1,149 @@
+/*
+ * Copyright (c) 2015 MediaTek Inc.
+ * Author: Daniel Hsiao <daniel.hsiao@mediatek.com>
+ *         PoChun Lin <pochun.lin@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef _VENC_VP8_IF_H_
+#define _VENC_VP8_IF_H_
+
+#include "venc_drv_base.h"
+
+/**
+ * enum venc_vp8_vpu_work_buf - vp8 encoder buffer index
+ */
+enum venc_vp8_vpu_work_buf {
+	VENC_VP8_VPU_WORK_BUF_LUMA,
+	VENC_VP8_VPU_WORK_BUF_LUMA2,
+	VENC_VP8_VPU_WORK_BUF_LUMA3,
+	VENC_VP8_VPU_WORK_BUF_CHROMA,
+	VENC_VP8_VPU_WORK_BUF_CHROMA2,
+	VENC_VP8_VPU_WORK_BUF_CHROMA3,
+	VENC_VP8_VPU_WORK_BUF_MV_INFO,
+	VENC_VP8_VPU_WORK_BUF_BS_HD,
+	VENC_VP8_VPU_WORK_BUF_PROB_BUF,
+	VENC_VP8_VPU_WORK_BUF_RC_INFO,
+	VENC_VP8_VPU_WORK_BUF_RC_CODE,
+	VENC_VP8_VPU_WORK_BUF_RC_CODE2,
+	VENC_VP8_VPU_WORK_BUF_RC_CODE3,
+	VENC_VP8_VPU_WORK_BUF_SRC_LUMA,
+	VENC_VP8_VPU_WORK_BUF_SRC_CHROMA,
+	VENC_VP8_VPU_WORK_BUF_SRC_CHROMA_CB,
+	VENC_VP8_VPU_WORK_BUF_SRC_CHROMA_CR,
+	VENC_VP8_VPU_WORK_BUF_MAX,
+};
+
+/*
+ * struct venc_vp8_vpu_config - Structure for vp8 encoder configuration
+ * @input_fourcc: input fourcc
+ * @bitrate: target bitrate (in bps)
+ * @pic_w: picture width
+ * @pic_h: picture height
+ * @buf_w: buffer width (with 16 alignment)
+ * @buf_h: buffer height (with 16 alignment)
+ * @intra_period: intra frame period
+ * @framerate: frame rate
+ * @ts_mode: temporal scalability mode (0: disable, 1: enable)
+ *	     support three temporal layers - 0: 7.5fps 1: 7.5fps 2: 15fps.
+ */
+struct venc_vp8_vpu_config {
+	u32 input_fourcc;
+	u32 bitrate;
+	u32 pic_w;
+	u32 pic_h;
+	u32 buf_w;
+	u32 buf_h;
+	u32 intra_period;
+	u32 framerate;
+	u32 ts_mode;
+};
+
+/*
+ * struct venc_vp8_vpu_buf -Structure for buffer information
+ * @align: buffer alignment (in bytes)
+ * @iova: IO virtual address
+ * @vpua: VPU side memory addr which is used by RC_CODE
+ * @size: buffer size (in bytes)
+ */
+struct venc_vp8_vpu_buf {
+	u32 align;
+	u32 iova;
+	u32 vpua;
+	u32 size;
+};
+
+/*
+ * struct venc_vp8_vpu_drv - Structure for VPU driver control and info share
+ * This structure is allocated in VPU side and shared to AP side.
+ * @config: vp8 encoder configuration
+ * @work_bufs: working buffer information in VPU side
+ * The work_bufs here is for storing the 'size' info shared to AP side.
+ * The similar item in struct venc_vp8_inst is for memory allocation
+ * in AP side. The AP driver will copy the 'size' from here to the one in
+ * struct mtk_vcodec_mem, then invoke mtk_vcodec_mem_alloc to allocate
+ * the buffer. After that, bypass the 'dma_addr' to the 'iova' field here for
+ * register setting in VPU side.
+ */
+struct venc_vp8_vpu_drv {
+	struct venc_vp8_vpu_config config;
+	struct venc_vp8_vpu_buf work_bufs[VENC_VP8_VPU_WORK_BUF_MAX];
+};
+
+/*
+ * struct venc_vp8_vpu_inst - vp8 encoder VPU driver instance
+ * @wq_hd: wait queue used for vpu cmd trigger then wait vpu interrupt done
+ * @signaled: flag used for checking vpu interrupt done
+ * @failure: flag to show vpu cmd succeeds or not
+ * @state: enum venc_ipi_msg_enc_state
+ * @id: VPU instance id
+ * @drv: driver structure allocated by VPU side and shared to AP side for
+ *	 control and info share
+ */
+struct venc_vp8_vpu_inst {
+	wait_queue_head_t wq_hd;
+	int signaled;
+	int failure;
+	int state;
+	unsigned int id;
+	struct venc_vp8_vpu_drv *drv;
+};
+
+/*
+ * struct venc_vp8_inst - vp8 encoder AP driver instance
+ * @hw_base: vp8 encoder hardware register base
+ * @work_bufs: working buffer
+ * @work_buf_allocated: working buffer allocated flag
+ * @frm_cnt: encoded frame count, it's used for I-frame judgement and
+ *	     reset when force intra cmd received.
+ * @ts_mode: temporal scalability mode (0: disable, 1: enable)
+ *	     support three temporal layers - 0: 7.5fps 1: 7.5fps 2: 15fps.
+ * @is_key_frm: key frame flag
+ * @vpu_inst: VPU instance to exchange information between AP and VPU
+ * @ctx: context for v4l2 layer integration
+ * @dev: device for v4l2 layer integration
+ */
+struct venc_vp8_inst {
+	void __iomem *hw_base;
+	struct mtk_vcodec_mem work_bufs[VENC_VP8_VPU_WORK_BUF_MAX];
+	bool work_buf_allocated;
+	unsigned int frm_cnt;
+	unsigned int ts_mode;
+	unsigned int is_key_frm;
+	struct venc_vp8_vpu_inst vpu_inst;
+	void *ctx;
+	struct platform_device *dev;
+};
+
+struct venc_common_if *get_vp8_enc_comm_if(void);
+
+#endif
diff --git a/drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_vpu.c b/drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_vpu.c
new file mode 100644
index 0000000..5f797c9
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_vpu.c
@@ -0,0 +1,240 @@
+/*
+ * Copyright (c) 2015 MediaTek Inc.
+ * Author: Daniel Hsiao <daniel.hsiao@mediatek.com>
+ *         PoChun Lin <pochun.lin@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#include "mtk_vpu.h"
+
+#include "venc_vp8_if.h"
+#include "venc_vp8_vpu.h"
+#include "venc_ipi_msg.h"
+
+static void handle_vp8_enc_init_msg(struct venc_vp8_inst *inst, void *data)
+{
+	struct venc_vpu_ipi_msg_init *msg = data;
+
+	inst->vpu_inst.id = msg->inst_id;
+	inst->vpu_inst.drv = (struct venc_vp8_vpu_drv *)
+		vpu_mapping_dm_addr(inst->dev, msg->inst_id);
+}
+
+static void handle_vp8_enc_encode_msg(struct venc_vp8_inst *inst, void *data)
+{
+	struct venc_vpu_ipi_msg_enc *msg = data;
+
+	inst->vpu_inst.state = msg->state;
+	inst->is_key_frm = msg->key_frame;
+}
+
+static void vp8_enc_vpu_ipi_handler(void *data, unsigned int len, void *priv)
+{
+	struct venc_vpu_ipi_msg_common *msg = data;
+	struct venc_vp8_inst *inst = (struct venc_vp8_inst *)msg->venc_inst;
+
+	mtk_vcodec_debug(inst, "->msg_id=%x status=%d",
+			 msg->msg_id, msg->status);
+
+	switch (msg->msg_id) {
+	case VPU_IPIMSG_VP8_ENC_INIT_DONE:
+		handle_vp8_enc_init_msg(inst, data);
+		break;
+	case VPU_IPIMSG_VP8_ENC_SET_PARAM_DONE:
+		break;
+	case VPU_IPIMSG_VP8_ENC_ENCODE_DONE:
+		handle_vp8_enc_encode_msg(inst, data);
+		break;
+	case VPU_IPIMSG_VP8_ENC_DEINIT_DONE:
+		break;
+	default:
+		mtk_vcodec_err(inst, "unknown msg id=%x", msg->msg_id);
+		break;
+	}
+
+	inst->vpu_inst.signaled = 1;
+	inst->vpu_inst.failure = (msg->status != VENC_IPI_MSG_STATUS_OK);
+
+	mtk_vcodec_debug_leave(inst);
+}
+
+static int vp8_enc_vpu_send_msg(struct venc_vp8_inst *inst, void *msg,
+				int len)
+{
+	int status;
+
+	mtk_vcodec_debug_enter(inst);
+	status = vpu_ipi_send(inst->dev, IPI_VENC_VP8, (void *)msg, len);
+	if (status) {
+		mtk_vcodec_err(inst,
+			       "vpu_ipi_send msg_id=%x len=%d failed status=%d",
+			       *(unsigned int *)msg, len, status);
+		return -EINVAL;
+	}
+	mtk_vcodec_debug_leave(inst);
+
+	return 0;
+}
+
+int vp8_enc_vpu_init(struct venc_vp8_inst *inst)
+{
+	int status;
+	struct venc_ap_ipi_msg_init out;
+
+	mtk_vcodec_debug_enter(inst);
+
+	init_waitqueue_head(&inst->vpu_inst.wq_hd);
+	inst->vpu_inst.signaled = 0;
+	inst->vpu_inst.failure = 0;
+
+	status = vpu_ipi_register(inst->dev, IPI_VENC_VP8,
+				  vp8_enc_vpu_ipi_handler,
+				  "vp8_enc", NULL);
+	if (status) {
+		mtk_vcodec_err(inst,
+			       "vpu_ipi_register failed status=%d", status);
+		return -EINVAL;
+	}
+
+	out.msg_id = AP_IPIMSG_VP8_ENC_INIT;
+	out.venc_inst = (unsigned long)inst;
+	if (vp8_enc_vpu_send_msg(inst, &out, sizeof(out)) ||
+	    inst->vpu_inst.failure) {
+		mtk_vcodec_err(inst, "failed");
+		return -EINVAL;
+	}
+
+	mtk_vcodec_debug_leave(inst);
+
+	return 0;
+}
+
+int vp8_enc_vpu_set_param(struct venc_vp8_inst *inst, unsigned int id,
+			  void *param)
+{
+	struct venc_ap_ipi_msg_set_param out;
+
+	mtk_vcodec_debug_enter(inst);
+
+	out.msg_id = AP_IPIMSG_VP8_ENC_SET_PARAM;
+	out.inst_id = inst->vpu_inst.id;
+	out.param_id = id;
+	switch (id) {
+	case VENC_SET_PARAM_ENC: {
+		struct venc_enc_prm *enc_param = (struct venc_enc_prm *)param;
+
+		inst->vpu_inst.drv->config.input_fourcc =
+			enc_param->input_fourcc;
+		inst->vpu_inst.drv->config.bitrate = enc_param->bitrate;
+		inst->vpu_inst.drv->config.pic_w = enc_param->width;
+		inst->vpu_inst.drv->config.pic_h = enc_param->height;
+		inst->vpu_inst.drv->config.buf_w = enc_param->buf_width;
+		inst->vpu_inst.drv->config.buf_h = enc_param->buf_height;
+		inst->vpu_inst.drv->config.intra_period =
+			enc_param->intra_period;
+		inst->vpu_inst.drv->config.framerate = enc_param->frm_rate;
+		inst->vpu_inst.drv->config.ts_mode = inst->ts_mode;
+		out.data_item = 0;
+		break;
+	}
+	case VENC_SET_PARAM_FORCE_INTRA:
+		out.data_item = 0;
+		break;
+	case VENC_SET_PARAM_ADJUST_BITRATE:
+		out.data_item = 1;
+		out.data[0] = *(unsigned int *)param;
+		break;
+	case VENC_SET_PARAM_ADJUST_FRAMERATE:
+		out.data_item = 1;
+		out.data[0] = *(unsigned int *)param;
+		break;
+	case VENC_SET_PARAM_I_FRAME_INTERVAL:
+		out.data_item = 1;
+		out.data[0] = *(unsigned int *)param;
+		break;
+	}
+	if (vp8_enc_vpu_send_msg(inst, &out, sizeof(out)) ||
+	    inst->vpu_inst.failure) {
+		mtk_vcodec_err(inst, "failed");
+		return -EINVAL;
+	}
+
+	mtk_vcodec_debug_leave(inst);
+
+	return 0;
+}
+
+int vp8_enc_vpu_encode(struct venc_vp8_inst *inst,
+		       struct venc_frm_buf *frm_buf,
+		       struct mtk_vcodec_mem *bs_buf)
+{
+	struct venc_ap_ipi_msg_enc out;
+
+	mtk_vcodec_debug_enter(inst);
+
+	out.msg_id = AP_IPIMSG_VP8_ENC_ENCODE;
+	out.inst_id = inst->vpu_inst.id;
+	if (frm_buf) {
+		if ((frm_buf->fb_addr.dma_addr % 16 == 0) &&
+		    (frm_buf->fb_addr1.dma_addr % 16 == 0) &&
+		    (frm_buf->fb_addr2.dma_addr % 16 == 0)) {
+			out.input_addr[0] = frm_buf->fb_addr.dma_addr;
+			out.input_addr[1] = frm_buf->fb_addr1.dma_addr;
+			out.input_addr[2] = frm_buf->fb_addr2.dma_addr;
+		} else {
+			mtk_vcodec_err(inst, "dma_addr not align to 16");
+			return -EINVAL;
+		}
+	} else {
+		out.input_addr[0] = 0;
+		out.input_addr[1] = 0;
+		out.input_addr[2] = 0;
+	}
+	if (bs_buf) {
+		out.bs_addr = bs_buf->dma_addr;
+		out.bs_size = bs_buf->size;
+	} else {
+		out.bs_addr = 0;
+		out.bs_size = 0;
+	}
+	if (vp8_enc_vpu_send_msg(inst, &out, sizeof(out)) ||
+	    inst->vpu_inst.failure) {
+		mtk_vcodec_err(inst, "failed");
+		return -EINVAL;
+	}
+
+	mtk_vcodec_debug(inst, "state=%d key_frm=%d",
+			 inst->vpu_inst.state, inst->is_key_frm);
+
+	mtk_vcodec_debug_leave(inst);
+
+	return 0;
+}
+
+int vp8_enc_vpu_deinit(struct venc_vp8_inst *inst)
+{
+	struct venc_ap_ipi_msg_deinit out;
+
+	mtk_vcodec_debug_enter(inst);
+
+	out.msg_id = AP_IPIMSG_VP8_ENC_DEINIT;
+	out.inst_id = inst->vpu_inst.id;
+	if (vp8_enc_vpu_send_msg(inst, &out, sizeof(out)) ||
+	    inst->vpu_inst.failure) {
+		mtk_vcodec_err(inst, "failed");
+		return -EINVAL;
+	}
+
+	mtk_vcodec_debug_leave(inst);
+
+	return 0;
+}
diff --git a/drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_vpu.h b/drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_vpu.h
new file mode 100644
index 0000000..d7d79a8
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_vpu.h
@@ -0,0 +1,28 @@
+/*
+ * Copyright (c) 2015 MediaTek Inc.
+ * Author: Daniel Hsiao <daniel.hsiao@mediatek.com>
+ *         PoChun Lin <pochun.lin@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef _VENC_VP8_VPU_H_
+#define _VENC_VP8_VPU_H_
+
+int vp8_enc_vpu_init(struct venc_vp8_inst *inst);
+int vp8_enc_vpu_set_param(struct venc_vp8_inst *inst, unsigned int id,
+			  void *param);
+int vp8_enc_vpu_encode(struct venc_vp8_inst *inst,
+		       struct venc_frm_buf *frm_buf,
+		       struct mtk_vcodec_mem *bs_buf);
+int vp8_enc_vpu_deinit(struct venc_vp8_inst *inst);
+
+#endif
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 102+ messages in thread

* [PATCH v4 6/8] [media] vcodec: mediatek: Add Mediatek VP8 Video Encoder Driver
@ 2016-02-04 11:35             ` Tiffany Lin
  0 siblings, 0 replies; 102+ messages in thread
From: Tiffany Lin @ 2016-02-04 11:35 UTC (permalink / raw)
  To: Hans Verkuil, daniel.thompson, Rob Herring,
	Mauro Carvalho Chehab, Matthias Brugger, Daniel Kurtz,
	Pawel Osciak
  Cc: Eddie Huang, Yingjoe Chen, devicetree, linux-kernel,
	linux-arm-kernel, linux-media, linux-mediatek, PoChun.Lin,
	Tiffany Lin, Daniel Hsiao

Add vp8 encoder driver for MT8173

Signed-off-by: Daniel Hsiao <daniel.hsiao@mediatek.com>
Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
---
 drivers/media/platform/mtk-vcodec/Makefile         |    2 +
 drivers/media/platform/mtk-vcodec/venc_drv_if.c    |    3 +
 drivers/media/platform/mtk-vcodec/vp8_enc/Makefile |    6 +
 .../platform/mtk-vcodec/vp8_enc/venc_vp8_if.c      |  422 ++++++++++++++++++++
 .../platform/mtk-vcodec/vp8_enc/venc_vp8_if.h      |  149 +++++++
 .../platform/mtk-vcodec/vp8_enc/venc_vp8_vpu.c     |  240 +++++++++++
 .../platform/mtk-vcodec/vp8_enc/venc_vp8_vpu.h     |   28 ++
 7 files changed, 850 insertions(+)
 create mode 100644 drivers/media/platform/mtk-vcodec/vp8_enc/Makefile
 create mode 100644 drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_if.c
 create mode 100644 drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_if.h
 create mode 100644 drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_vpu.c
 create mode 100644 drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_vpu.h

diff --git a/drivers/media/platform/mtk-vcodec/Makefile b/drivers/media/platform/mtk-vcodec/Makefile
index ce38689..f4ef502 100644
--- a/drivers/media/platform/mtk-vcodec/Makefile
+++ b/drivers/media/platform/mtk-vcodec/Makefile
@@ -5,4 +5,6 @@ obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += mtk_vcodec_intr.o \
 				       mtk_vcodec_enc_pm.o \
 				       venc_drv_if.o
 
+obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += vp8_enc/
+
 ccflags-y += -I$(srctree)/drivers/media/platform/mtk-vpu
diff --git a/drivers/media/platform/mtk-vcodec/venc_drv_if.c b/drivers/media/platform/mtk-vcodec/venc_drv_if.c
index daa8e93..d293f2c 100644
--- a/drivers/media/platform/mtk-vcodec/venc_drv_if.c
+++ b/drivers/media/platform/mtk-vcodec/venc_drv_if.c
@@ -24,6 +24,7 @@
 #include "mtk_vpu.h"
 
 #include "venc_drv_base.h"
+#include "vp8_enc/venc_vp8_if.h"
 
 int venc_if_create(struct mtk_vcodec_ctx *ctx, unsigned int fourcc)
 {
@@ -34,6 +35,8 @@ int venc_if_create(struct mtk_vcodec_ctx *ctx, unsigned int fourcc)
 
 	switch (fourcc) {
 	case V4L2_PIX_FMT_VP8:
+                ctx->enc_if = get_vp8_enc_comm_if();
+                break;
 	case V4L2_PIX_FMT_H264:
 	default:
 		return -EINVAL;
diff --git a/drivers/media/platform/mtk-vcodec/vp8_enc/Makefile b/drivers/media/platform/mtk-vcodec/vp8_enc/Makefile
new file mode 100644
index 0000000..0f515c7
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/vp8_enc/Makefile
@@ -0,0 +1,6 @@
+obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += venc_vp8_if.o venc_vp8_vpu.o
+
+ccflags-y += \
+    -I$(srctree)/drivers/media/platform/mtk-vcodec/ \
+    -I$(srctree)/drivers/media/platform/mtk-vcodec/vp8_enc \
+    -I$(srctree)/drivers/media/platform/mtk-vpu
diff --git a/drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_if.c b/drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_if.c
new file mode 100644
index 0000000..4253317
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_if.c
@@ -0,0 +1,422 @@
+/*
+ * Copyright (c) 2015 MediaTek Inc.
+ * Author: Daniel Hsiao <daniel.hsiao@mediatek.com>
+ *         PoChun Lin <pochun.lin@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/interrupt.h>
+#include <linux/kernel.h>
+#include <linux/slab.h>
+
+#include "mtk_vcodec_drv.h"
+#include "mtk_vcodec_util.h"
+#include "mtk_vcodec_intr.h"
+#include "mtk_vcodec_enc.h"
+#include "mtk_vcodec_pm.h"
+#include "mtk_vpu.h"
+
+#include "venc_vp8_if.h"
+#include "venc_vp8_vpu.h"
+
+#define VENC_PIC_BITSTREAM_BYTE_CNT 0x0098
+#define VENC_PIC_BITSTREAM_BYTE_CNT1 0x00e8
+#define VENC_IRQ_STATUS_ENC_FRM_INT 0x04
+
+#define MAX_AC_TAG_SZ 10
+
+static inline void vp8_enc_write_reg(struct venc_vp8_inst *inst, u32 addr,
+				     u32 val)
+{
+	writel(val, inst->hw_base + addr);
+}
+
+static inline u32 vp8_enc_read_reg(struct venc_vp8_inst *inst, u32 addr)
+{
+	return readl(inst->hw_base + addr);
+}
+
+static void vp8_enc_free_work_buf(struct venc_vp8_inst *inst)
+{
+	int i;
+
+	mtk_vcodec_debug_enter(inst);
+
+	/* Buffers need to be freed by AP. */
+	for (i = 0; i < VENC_VP8_VPU_WORK_BUF_MAX; i++)
+		if (inst->work_bufs[i].va != NULL)
+			mtk_vcodec_mem_free(inst->ctx,
+					    &inst->work_bufs[i]);
+
+	mtk_vcodec_debug_leave(inst);
+}
+
+static int vp8_enc_alloc_work_buf(struct venc_vp8_inst *inst, void *param)
+{
+	int i;
+	int ret = 0;
+	struct venc_vp8_vpu_buf *wb = inst->vpu_inst.drv->work_bufs;
+	struct venc_enc_prm *enc_param = param;
+
+	mtk_vcodec_debug_enter(inst);
+
+	for (i = 0; i < VENC_VP8_VPU_WORK_BUF_MAX; i++) {
+		/*
+		 * Only temporal scalability mode will use RC_CODE2 & RC_CODE3
+		 * Each three temporal layer has its own rate control code.
+		 */
+		if ((i == VENC_VP8_VPU_WORK_BUF_RC_CODE2 ||
+		     i == VENC_VP8_VPU_WORK_BUF_RC_CODE3) && !inst->ts_mode)
+			continue;
+
+		/*
+		 * This 'wb' structure is set by VPU side and shared to AP for
+		 * buffer allocation and IO virtual addr mapping. For most of
+		 * the buffers, AP will allocate the buffer according to 'size'
+		 * field and store the IO virtual addr in 'iova' field. For the
+		 * RC_CODEx buffers, they are pre-allocated in the VPU side
+		 * because they are inside VPU SRAM, and save the VPU addr in
+		 * the 'vpua' field. The AP will translate the VPU addr to the
+		 * corresponding IO virtual addr and store in 'iova' field.
+		 */
+		if (i < VENC_VP8_VPU_WORK_BUF_RC_CODE) {
+			inst->work_bufs[i].size = wb[i].size;
+			ret = mtk_vcodec_mem_alloc(inst->ctx,
+						   &inst->work_bufs[i]);
+			if (ret) {
+				mtk_vcodec_err(inst,
+					       "cannot alloc work_bufs[%d]", i);
+				goto err_alloc;
+			}
+			wb[i].iova = inst->work_bufs[i].dma_addr;
+		} else if (i == VENC_VP8_VPU_WORK_BUF_SRC_LUMA ||
+			   i == VENC_VP8_VPU_WORK_BUF_SRC_CHROMA ||
+			   i == VENC_VP8_VPU_WORK_BUF_SRC_CHROMA_CB ||
+			   i == VENC_VP8_VPU_WORK_BUF_SRC_CHROMA_CR) {
+			inst->work_bufs[i].size = wb[i].size;
+			inst->work_bufs[i].dma_addr = 0;
+			inst->work_bufs[i].va = NULL;
+			wb[i].iova = inst->work_bufs[i].dma_addr;
+		} else {
+			void *tmp_va;
+
+			tmp_va = vpu_mapping_dm_addr(inst->dev, wb[i].vpua);
+			inst->work_bufs[i].size = wb[i].size;
+			ret = mtk_vcodec_mem_alloc(inst->ctx,
+						   &inst->work_bufs[i]);
+			if (ret) {
+				mtk_vcodec_err(inst,
+					       "cannot alloc work_bufs[%d]", i);
+				goto err_alloc;
+			}
+			memcpy(inst->work_bufs[i].va, tmp_va, wb[i].size);
+			wb[i].iova = inst->work_bufs[i].dma_addr;
+		}
+		mtk_vcodec_debug(inst,
+				 "work_bufs[%d] va=0x%p,iova=0x%p,size=0x%lx",
+				 i, inst->work_bufs[i].va,
+				 (void *)inst->work_bufs[i].dma_addr,
+				 inst->work_bufs[i].size);
+	}
+
+	if (enc_param->input_fourcc == VENC_YUV_FORMAT_NV12 ||
+	    enc_param->input_fourcc == VENC_YUV_FORMAT_NV21) {
+		enc_param->sizeimage[0] =
+			inst->work_bufs[VENC_VP8_VPU_WORK_BUF_SRC_LUMA].size;
+		enc_param->sizeimage[1] =
+			inst->work_bufs[VENC_VP8_VPU_WORK_BUF_SRC_CHROMA].size;
+		enc_param->sizeimage[2] = 0;
+	} else {
+		enc_param->sizeimage[0] =
+			inst->work_bufs[VENC_VP8_VPU_WORK_BUF_SRC_LUMA].size;
+		enc_param->sizeimage[1] =
+			inst->work_bufs[VENC_VP8_VPU_WORK_BUF_SRC_CHROMA_CB].size;
+		enc_param->sizeimage[2] =
+			inst->work_bufs[VENC_VP8_VPU_WORK_BUF_SRC_CHROMA_CR].size;
+	}
+	mtk_vcodec_debug_leave(inst);
+
+	return ret;
+
+err_alloc:
+	vp8_enc_free_work_buf(inst);
+
+	return ret;
+}
+
+static unsigned int vp8_enc_wait_venc_done(struct venc_vp8_inst *inst)
+{
+	unsigned int irq_status = 0;
+	struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)inst->ctx;
+
+	mtk_vcodec_wait_for_done_ctx(ctx, MTK_INST_IRQ_RECEIVED,
+				     WAIT_INTR_TIMEOUT, true);
+	irq_status = ctx->irq_status;
+	mtk_vcodec_debug(inst, "isr return %x", irq_status);
+
+	return irq_status;
+}
+
+/*
+ * Compose ac_tag, bitstream header and bitstream payload into
+ * one bitstream buffer.
+ */
+static int vp8_enc_compose_one_frame(struct venc_vp8_inst *inst,
+				     struct mtk_vcodec_mem *bs_buf,
+				     unsigned int *bs_size)
+{
+	unsigned int is_key;
+	u32 bs_size_frm;
+	u32 bs_hdr_len;
+	unsigned int ac_tag_sz;
+	u8 ac_tag[MAX_AC_TAG_SZ];
+
+	bs_size_frm = vp8_enc_read_reg(inst,
+				       VENC_PIC_BITSTREAM_BYTE_CNT);
+	bs_hdr_len = vp8_enc_read_reg(inst,
+				      VENC_PIC_BITSTREAM_BYTE_CNT1);
+
+	/* if a frame is key frame, is_key is 0 */
+	is_key = !inst->is_key_frm;
+	*(u32 *)ac_tag = __cpu_to_le32((bs_hdr_len << 5) | 0x10 | is_key);
+	/* key frame */
+	if (is_key == 0) {
+		ac_tag[3] = 0x9d;
+		ac_tag[4] = 0x01;
+		ac_tag[5] = 0x2a;
+		ac_tag[6] = inst->vpu_inst.drv->config.pic_w;
+		ac_tag[7] = inst->vpu_inst.drv->config.pic_w >> 8;
+		ac_tag[8] = inst->vpu_inst.drv->config.pic_h;
+		ac_tag[9] = inst->vpu_inst.drv->config.pic_h >> 8;
+	}
+
+	if (is_key == 0)
+		ac_tag_sz = MAX_AC_TAG_SZ;
+	else
+		ac_tag_sz = 3;
+
+	if (bs_buf->size <= bs_hdr_len + bs_size_frm + ac_tag_sz) {
+		mtk_vcodec_err(inst, "bitstream buf size is too small(%ld)",
+			       bs_buf->size);
+		return -EINVAL;
+	}
+
+	/*
+	* (1) The vp8 bitstream header and body are generated by the HW vp8
+	* encoder separately at the same time. We cannot know the bitstream
+	* header length in advance.
+	* (2) From the vp8 spec, there is no stuffing byte allowed between the
+	* ac tag, bitstream header and bitstream body.
+	*/
+	memmove(bs_buf->va + bs_hdr_len + ac_tag_sz,
+		bs_buf->va, bs_size_frm);
+	memcpy(bs_buf->va + ac_tag_sz,
+	       inst->work_bufs[VENC_VP8_VPU_WORK_BUF_BS_HD].va,
+	       bs_hdr_len);
+	memcpy(bs_buf->va, ac_tag, ac_tag_sz);
+	*bs_size = bs_size_frm + bs_hdr_len + ac_tag_sz;
+
+	return 0;
+}
+
+static int vp8_enc_encode_frame(struct venc_vp8_inst *inst,
+				struct venc_frm_buf *frm_buf,
+				struct mtk_vcodec_mem *bs_buf,
+				unsigned int *bs_size)
+{
+	int ret = 0;
+	unsigned int irq_status;
+
+	mtk_vcodec_debug(inst, "->frm_cnt=%d", inst->frm_cnt);
+
+	ret = vp8_enc_vpu_encode(inst, frm_buf, bs_buf);
+	if (ret)
+		return ret;
+
+	irq_status = vp8_enc_wait_venc_done(inst);
+	if (irq_status != VENC_IRQ_STATUS_ENC_FRM_INT) {
+		mtk_vcodec_err(inst, "irq_status=%d failed", irq_status);
+		return -EINVAL;
+	}
+
+	if (vp8_enc_compose_one_frame(inst, bs_buf, bs_size)) {
+		mtk_vcodec_err(inst, "vp8_enc_compose_one_frame failed");
+		return -EINVAL;
+	}
+
+	inst->frm_cnt++;
+	mtk_vcodec_debug(inst, "<-size=%d key_frm=%d", *bs_size,
+			 inst->is_key_frm);
+
+	return ret;
+}
+
+static int vp8_enc_init(struct mtk_vcodec_ctx *ctx, unsigned long *handle)
+{
+	int ret = 0;
+	struct venc_vp8_inst *inst;
+
+	inst = kzalloc(sizeof(*inst), GFP_KERNEL);
+	if (!inst)
+		return -ENOMEM;
+
+	inst->ctx = ctx;
+	inst->dev = mtk_vcodec_get_plat_dev(ctx);
+	inst->hw_base = mtk_vcodec_get_reg_addr(inst->ctx, VENC_LT_SYS);
+
+	mtk_vcodec_debug_enter(inst);
+	ret = vp8_enc_vpu_init(inst);
+	if (ret)
+		kfree(inst);
+	else
+		(*handle) = (unsigned long)inst;
+
+	mtk_vcodec_debug_leave(inst);
+
+	return ret;
+}
+
+static int vp8_enc_encode(unsigned long handle,
+			  enum venc_start_opt opt,
+			  struct venc_frm_buf *frm_buf,
+			  struct mtk_vcodec_mem *bs_buf,
+			  struct venc_done_result *result)
+{
+	int ret = 0;
+	struct venc_vp8_inst *inst = (struct venc_vp8_inst *)handle;
+	struct mtk_vcodec_ctx *ctx = inst->ctx;
+
+	mtk_vcodec_debug_enter(inst);
+
+	enable_irq(ctx->dev->enc_lt_irq);
+
+	switch (opt) {
+	case VENC_START_OPT_ENCODE_FRAME:
+		ret = vp8_enc_encode_frame(inst, frm_buf, bs_buf,
+					   &result->bs_size);
+		if (ret)
+			goto encode_err;
+		result->is_key_frm = inst->is_key_frm;
+		break;
+
+	default:
+		mtk_vcodec_err(inst, "opt not support:%d", opt);
+		ret = -EINVAL;
+		break;
+	}
+
+encode_err:
+	if (ret)
+		result->msg = VENC_MESSAGE_ERR;
+	else
+		result->msg = VENC_MESSAGE_OK;
+
+	disable_irq(ctx->dev->enc_lt_irq);
+	mtk_vcodec_debug_leave(inst);
+
+	return ret;
+}
+
+static int vp8_enc_set_param(unsigned long handle,
+			     enum venc_set_param_type type, void *in)
+{
+	int ret = 0;
+	struct venc_vp8_inst *inst = (struct venc_vp8_inst *)handle;
+	struct venc_enc_prm *enc_prm;
+
+	mtk_vcodec_debug(inst, "->type=%d", type);
+
+	switch (type) {
+	case VENC_SET_PARAM_ENC:
+		enc_prm = in;
+		ret = vp8_enc_vpu_set_param(inst, type, enc_prm);
+		if (ret)
+			break;
+		if (inst->work_buf_allocated == 1) {
+			vp8_enc_free_work_buf(inst);
+			inst->work_buf_allocated = 0;
+		}
+		if (inst->work_buf_allocated == 0) {
+			ret = vp8_enc_alloc_work_buf(inst, enc_prm);
+			if (ret)
+				break;
+			inst->work_buf_allocated = 1;
+		}
+		break;
+
+	case VENC_SET_PARAM_FORCE_INTRA:
+		ret = vp8_enc_vpu_set_param(inst, type, 0);
+		break;
+
+	case VENC_SET_PARAM_ADJUST_BITRATE:
+		enc_prm = in;
+		ret = vp8_enc_vpu_set_param(inst, type, &enc_prm->bitrate);
+		break;
+
+	case VENC_SET_PARAM_ADJUST_FRAMERATE:
+		enc_prm = in;
+		ret = vp8_enc_vpu_set_param(inst, type, &enc_prm->frm_rate);
+		break;
+
+	case VENC_SET_PARAM_I_FRAME_INTERVAL:
+		ret = vp8_enc_vpu_set_param(inst, type, in);
+		break;
+
+	/*
+	 * VENC_SET_PARAM_TS_MODE must be called before
+	 * VENC_SET_PARAM_ENC
+	 */
+	case VENC_SET_PARAM_TS_MODE:
+		inst->ts_mode = 1;
+		mtk_vcodec_debug(inst, "set ts_mode");
+		break;
+
+	default:
+		mtk_vcodec_err(inst, "type not support:%d", type);
+		ret = -EINVAL;
+		break;
+	}
+
+	mtk_vcodec_debug_leave(inst);
+
+	return ret;
+}
+
+static int vp8_enc_deinit(unsigned long handle)
+{
+	int ret = 0;
+	struct venc_vp8_inst *inst = (struct venc_vp8_inst *)handle;
+
+	mtk_vcodec_debug_enter(inst);
+
+	ret = vp8_enc_vpu_deinit(inst);
+
+	if (inst->work_buf_allocated)
+		vp8_enc_free_work_buf(inst);
+
+	mtk_vcodec_debug_leave(inst);
+	kfree(inst);
+
+	return ret;
+}
+
+static struct venc_common_if venc_vp8_if = {
+	vp8_enc_init,
+	vp8_enc_encode,
+	vp8_enc_set_param,
+	vp8_enc_deinit,
+};
+
+struct venc_common_if *get_vp8_enc_comm_if(void)
+{
+	return &venc_vp8_if;
+}
diff --git a/drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_if.h b/drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_if.h
new file mode 100644
index 0000000..b32220e
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_if.h
@@ -0,0 +1,149 @@
+/*
+ * Copyright (c) 2015 MediaTek Inc.
+ * Author: Daniel Hsiao <daniel.hsiao@mediatek.com>
+ *         PoChun Lin <pochun.lin@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef _VENC_VP8_IF_H_
+#define _VENC_VP8_IF_H_
+
+#include "venc_drv_base.h"
+
+/**
+ * enum venc_vp8_vpu_work_buf - vp8 encoder buffer index
+ */
+enum venc_vp8_vpu_work_buf {
+	VENC_VP8_VPU_WORK_BUF_LUMA,
+	VENC_VP8_VPU_WORK_BUF_LUMA2,
+	VENC_VP8_VPU_WORK_BUF_LUMA3,
+	VENC_VP8_VPU_WORK_BUF_CHROMA,
+	VENC_VP8_VPU_WORK_BUF_CHROMA2,
+	VENC_VP8_VPU_WORK_BUF_CHROMA3,
+	VENC_VP8_VPU_WORK_BUF_MV_INFO,
+	VENC_VP8_VPU_WORK_BUF_BS_HD,
+	VENC_VP8_VPU_WORK_BUF_PROB_BUF,
+	VENC_VP8_VPU_WORK_BUF_RC_INFO,
+	VENC_VP8_VPU_WORK_BUF_RC_CODE,
+	VENC_VP8_VPU_WORK_BUF_RC_CODE2,
+	VENC_VP8_VPU_WORK_BUF_RC_CODE3,
+	VENC_VP8_VPU_WORK_BUF_SRC_LUMA,
+	VENC_VP8_VPU_WORK_BUF_SRC_CHROMA,
+	VENC_VP8_VPU_WORK_BUF_SRC_CHROMA_CB,
+	VENC_VP8_VPU_WORK_BUF_SRC_CHROMA_CR,
+	VENC_VP8_VPU_WORK_BUF_MAX,
+};
+
+/*
+ * struct venc_vp8_vpu_config - Structure for vp8 encoder configuration
+ * @input_fourcc: input fourcc
+ * @bitrate: target bitrate (in bps)
+ * @pic_w: picture width
+ * @pic_h: picture height
+ * @buf_w: buffer width (with 16 alignment)
+ * @buf_h: buffer height (with 16 alignment)
+ * @intra_period: intra frame period
+ * @framerate: frame rate
+ * @ts_mode: temporal scalability mode (0: disable, 1: enable)
+ *	     support three temporal layers - 0: 7.5fps 1: 7.5fps 2: 15fps.
+ */
+struct venc_vp8_vpu_config {
+	u32 input_fourcc;
+	u32 bitrate;
+	u32 pic_w;
+	u32 pic_h;
+	u32 buf_w;
+	u32 buf_h;
+	u32 intra_period;
+	u32 framerate;
+	u32 ts_mode;
+};
+
+/*
+ * struct venc_vp8_vpu_buf -Structure for buffer information
+ * @align: buffer alignment (in bytes)
+ * @iova: IO virtual address
+ * @vpua: VPU side memory addr which is used by RC_CODE
+ * @size: buffer size (in bytes)
+ */
+struct venc_vp8_vpu_buf {
+	u32 align;
+	u32 iova;
+	u32 vpua;
+	u32 size;
+};
+
+/*
+ * struct venc_vp8_vpu_drv - Structure for VPU driver control and info share
+ * This structure is allocated in VPU side and shared to AP side.
+ * @config: vp8 encoder configuration
+ * @work_bufs: working buffer information in VPU side
+ * The work_bufs here is for storing the 'size' info shared to AP side.
+ * The similar item in struct venc_vp8_inst is for memory allocation
+ * in AP side. The AP driver will copy the 'size' from here to the one in
+ * struct mtk_vcodec_mem, then invoke mtk_vcodec_mem_alloc to allocate
+ * the buffer. After that, bypass the 'dma_addr' to the 'iova' field here for
+ * register setting in VPU side.
+ */
+struct venc_vp8_vpu_drv {
+	struct venc_vp8_vpu_config config;
+	struct venc_vp8_vpu_buf work_bufs[VENC_VP8_VPU_WORK_BUF_MAX];
+};
+
+/*
+ * struct venc_vp8_vpu_inst - vp8 encoder VPU driver instance
+ * @wq_hd: wait queue used for vpu cmd trigger then wait vpu interrupt done
+ * @signaled: flag used for checking vpu interrupt done
+ * @failure: flag to show vpu cmd succeeds or not
+ * @state: enum venc_ipi_msg_enc_state
+ * @id: VPU instance id
+ * @drv: driver structure allocated by VPU side and shared to AP side for
+ *	 control and info share
+ */
+struct venc_vp8_vpu_inst {
+	wait_queue_head_t wq_hd;
+	int signaled;
+	int failure;
+	int state;
+	unsigned int id;
+	struct venc_vp8_vpu_drv *drv;
+};
+
+/*
+ * struct venc_vp8_inst - vp8 encoder AP driver instance
+ * @hw_base: vp8 encoder hardware register base
+ * @work_bufs: working buffer
+ * @work_buf_allocated: working buffer allocated flag
+ * @frm_cnt: encoded frame count, it's used for I-frame judgement and
+ *	     reset when force intra cmd received.
+ * @ts_mode: temporal scalability mode (0: disable, 1: enable)
+ *	     support three temporal layers - 0: 7.5fps 1: 7.5fps 2: 15fps.
+ * @is_key_frm: key frame flag
+ * @vpu_inst: VPU instance to exchange information between AP and VPU
+ * @ctx: context for v4l2 layer integration
+ * @dev: device for v4l2 layer integration
+ */
+struct venc_vp8_inst {
+	void __iomem *hw_base;
+	struct mtk_vcodec_mem work_bufs[VENC_VP8_VPU_WORK_BUF_MAX];
+	bool work_buf_allocated;
+	unsigned int frm_cnt;
+	unsigned int ts_mode;
+	unsigned int is_key_frm;
+	struct venc_vp8_vpu_inst vpu_inst;
+	void *ctx;
+	struct platform_device *dev;
+};
+
+struct venc_common_if *get_vp8_enc_comm_if(void);
+
+#endif
diff --git a/drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_vpu.c b/drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_vpu.c
new file mode 100644
index 0000000..5f797c9
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_vpu.c
@@ -0,0 +1,240 @@
+/*
+ * Copyright (c) 2015 MediaTek Inc.
+ * Author: Daniel Hsiao <daniel.hsiao@mediatek.com>
+ *         PoChun Lin <pochun.lin@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#include "mtk_vpu.h"
+
+#include "venc_vp8_if.h"
+#include "venc_vp8_vpu.h"
+#include "venc_ipi_msg.h"
+
+static void handle_vp8_enc_init_msg(struct venc_vp8_inst *inst, void *data)
+{
+	struct venc_vpu_ipi_msg_init *msg = data;
+
+	inst->vpu_inst.id = msg->inst_id;
+	inst->vpu_inst.drv = (struct venc_vp8_vpu_drv *)
+		vpu_mapping_dm_addr(inst->dev, msg->inst_id);
+}
+
+static void handle_vp8_enc_encode_msg(struct venc_vp8_inst *inst, void *data)
+{
+	struct venc_vpu_ipi_msg_enc *msg = data;
+
+	inst->vpu_inst.state = msg->state;
+	inst->is_key_frm = msg->key_frame;
+}
+
+static void vp8_enc_vpu_ipi_handler(void *data, unsigned int len, void *priv)
+{
+	struct venc_vpu_ipi_msg_common *msg = data;
+	struct venc_vp8_inst *inst = (struct venc_vp8_inst *)msg->venc_inst;
+
+	mtk_vcodec_debug(inst, "->msg_id=%x status=%d",
+			 msg->msg_id, msg->status);
+
+	switch (msg->msg_id) {
+	case VPU_IPIMSG_VP8_ENC_INIT_DONE:
+		handle_vp8_enc_init_msg(inst, data);
+		break;
+	case VPU_IPIMSG_VP8_ENC_SET_PARAM_DONE:
+		break;
+	case VPU_IPIMSG_VP8_ENC_ENCODE_DONE:
+		handle_vp8_enc_encode_msg(inst, data);
+		break;
+	case VPU_IPIMSG_VP8_ENC_DEINIT_DONE:
+		break;
+	default:
+		mtk_vcodec_err(inst, "unknown msg id=%x", msg->msg_id);
+		break;
+	}
+
+	inst->vpu_inst.signaled = 1;
+	inst->vpu_inst.failure = (msg->status != VENC_IPI_MSG_STATUS_OK);
+
+	mtk_vcodec_debug_leave(inst);
+}
+
+static int vp8_enc_vpu_send_msg(struct venc_vp8_inst *inst, void *msg,
+				int len)
+{
+	int status;
+
+	mtk_vcodec_debug_enter(inst);
+	status = vpu_ipi_send(inst->dev, IPI_VENC_VP8, (void *)msg, len);
+	if (status) {
+		mtk_vcodec_err(inst,
+			       "vpu_ipi_send msg_id=%x len=%d failed status=%d",
+			       *(unsigned int *)msg, len, status);
+		return -EINVAL;
+	}
+	mtk_vcodec_debug_leave(inst);
+
+	return 0;
+}
+
+int vp8_enc_vpu_init(struct venc_vp8_inst *inst)
+{
+	int status;
+	struct venc_ap_ipi_msg_init out;
+
+	mtk_vcodec_debug_enter(inst);
+
+	init_waitqueue_head(&inst->vpu_inst.wq_hd);
+	inst->vpu_inst.signaled = 0;
+	inst->vpu_inst.failure = 0;
+
+	status = vpu_ipi_register(inst->dev, IPI_VENC_VP8,
+				  vp8_enc_vpu_ipi_handler,
+				  "vp8_enc", NULL);
+	if (status) {
+		mtk_vcodec_err(inst,
+			       "vpu_ipi_register failed status=%d", status);
+		return -EINVAL;
+	}
+
+	out.msg_id = AP_IPIMSG_VP8_ENC_INIT;
+	out.venc_inst = (unsigned long)inst;
+	if (vp8_enc_vpu_send_msg(inst, &out, sizeof(out)) ||
+	    inst->vpu_inst.failure) {
+		mtk_vcodec_err(inst, "failed");
+		return -EINVAL;
+	}
+
+	mtk_vcodec_debug_leave(inst);
+
+	return 0;
+}
+
+int vp8_enc_vpu_set_param(struct venc_vp8_inst *inst, unsigned int id,
+			  void *param)
+{
+	struct venc_ap_ipi_msg_set_param out;
+
+	mtk_vcodec_debug_enter(inst);
+
+	out.msg_id = AP_IPIMSG_VP8_ENC_SET_PARAM;
+	out.inst_id = inst->vpu_inst.id;
+	out.param_id = id;
+	switch (id) {
+	case VENC_SET_PARAM_ENC: {
+		struct venc_enc_prm *enc_param = (struct venc_enc_prm *)param;
+
+		inst->vpu_inst.drv->config.input_fourcc =
+			enc_param->input_fourcc;
+		inst->vpu_inst.drv->config.bitrate = enc_param->bitrate;
+		inst->vpu_inst.drv->config.pic_w = enc_param->width;
+		inst->vpu_inst.drv->config.pic_h = enc_param->height;
+		inst->vpu_inst.drv->config.buf_w = enc_param->buf_width;
+		inst->vpu_inst.drv->config.buf_h = enc_param->buf_height;
+		inst->vpu_inst.drv->config.intra_period =
+			enc_param->intra_period;
+		inst->vpu_inst.drv->config.framerate = enc_param->frm_rate;
+		inst->vpu_inst.drv->config.ts_mode = inst->ts_mode;
+		out.data_item = 0;
+		break;
+	}
+	case VENC_SET_PARAM_FORCE_INTRA:
+		out.data_item = 0;
+		break;
+	case VENC_SET_PARAM_ADJUST_BITRATE:
+		out.data_item = 1;
+		out.data[0] = *(unsigned int *)param;
+		break;
+	case VENC_SET_PARAM_ADJUST_FRAMERATE:
+		out.data_item = 1;
+		out.data[0] = *(unsigned int *)param;
+		break;
+	case VENC_SET_PARAM_I_FRAME_INTERVAL:
+		out.data_item = 1;
+		out.data[0] = *(unsigned int *)param;
+		break;
+	}
+	if (vp8_enc_vpu_send_msg(inst, &out, sizeof(out)) ||
+	    inst->vpu_inst.failure) {
+		mtk_vcodec_err(inst, "failed");
+		return -EINVAL;
+	}
+
+	mtk_vcodec_debug_leave(inst);
+
+	return 0;
+}
+
+int vp8_enc_vpu_encode(struct venc_vp8_inst *inst,
+		       struct venc_frm_buf *frm_buf,
+		       struct mtk_vcodec_mem *bs_buf)
+{
+	struct venc_ap_ipi_msg_enc out;
+
+	mtk_vcodec_debug_enter(inst);
+
+	out.msg_id = AP_IPIMSG_VP8_ENC_ENCODE;
+	out.inst_id = inst->vpu_inst.id;
+	if (frm_buf) {
+		if ((frm_buf->fb_addr.dma_addr % 16 == 0) &&
+		    (frm_buf->fb_addr1.dma_addr % 16 == 0) &&
+		    (frm_buf->fb_addr2.dma_addr % 16 == 0)) {
+			out.input_addr[0] = frm_buf->fb_addr.dma_addr;
+			out.input_addr[1] = frm_buf->fb_addr1.dma_addr;
+			out.input_addr[2] = frm_buf->fb_addr2.dma_addr;
+		} else {
+			mtk_vcodec_err(inst, "dma_addr not align to 16");
+			return -EINVAL;
+		}
+	} else {
+		out.input_addr[0] = 0;
+		out.input_addr[1] = 0;
+		out.input_addr[2] = 0;
+	}
+	if (bs_buf) {
+		out.bs_addr = bs_buf->dma_addr;
+		out.bs_size = bs_buf->size;
+	} else {
+		out.bs_addr = 0;
+		out.bs_size = 0;
+	}
+	if (vp8_enc_vpu_send_msg(inst, &out, sizeof(out)) ||
+	    inst->vpu_inst.failure) {
+		mtk_vcodec_err(inst, "failed");
+		return -EINVAL;
+	}
+
+	mtk_vcodec_debug(inst, "state=%d key_frm=%d",
+			 inst->vpu_inst.state, inst->is_key_frm);
+
+	mtk_vcodec_debug_leave(inst);
+
+	return 0;
+}
+
+int vp8_enc_vpu_deinit(struct venc_vp8_inst *inst)
+{
+	struct venc_ap_ipi_msg_deinit out;
+
+	mtk_vcodec_debug_enter(inst);
+
+	out.msg_id = AP_IPIMSG_VP8_ENC_DEINIT;
+	out.inst_id = inst->vpu_inst.id;
+	if (vp8_enc_vpu_send_msg(inst, &out, sizeof(out)) ||
+	    inst->vpu_inst.failure) {
+		mtk_vcodec_err(inst, "failed");
+		return -EINVAL;
+	}
+
+	mtk_vcodec_debug_leave(inst);
+
+	return 0;
+}
diff --git a/drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_vpu.h b/drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_vpu.h
new file mode 100644
index 0000000..d7d79a8
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_vpu.h
@@ -0,0 +1,28 @@
+/*
+ * Copyright (c) 2015 MediaTek Inc.
+ * Author: Daniel Hsiao <daniel.hsiao@mediatek.com>
+ *         PoChun Lin <pochun.lin@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef _VENC_VP8_VPU_H_
+#define _VENC_VP8_VPU_H_
+
+int vp8_enc_vpu_init(struct venc_vp8_inst *inst);
+int vp8_enc_vpu_set_param(struct venc_vp8_inst *inst, unsigned int id,
+			  void *param);
+int vp8_enc_vpu_encode(struct venc_vp8_inst *inst,
+		       struct venc_frm_buf *frm_buf,
+		       struct mtk_vcodec_mem *bs_buf);
+int vp8_enc_vpu_deinit(struct venc_vp8_inst *inst);
+
+#endif
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 102+ messages in thread

* [PATCH v4 6/8] [media] vcodec: mediatek: Add Mediatek VP8 Video Encoder Driver
@ 2016-02-04 11:35             ` Tiffany Lin
  0 siblings, 0 replies; 102+ messages in thread
From: Tiffany Lin @ 2016-02-04 11:35 UTC (permalink / raw)
  To: linux-arm-kernel

Add vp8 encoder driver for MT8173

Signed-off-by: Daniel Hsiao <daniel.hsiao@mediatek.com>
Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
---
 drivers/media/platform/mtk-vcodec/Makefile         |    2 +
 drivers/media/platform/mtk-vcodec/venc_drv_if.c    |    3 +
 drivers/media/platform/mtk-vcodec/vp8_enc/Makefile |    6 +
 .../platform/mtk-vcodec/vp8_enc/venc_vp8_if.c      |  422 ++++++++++++++++++++
 .../platform/mtk-vcodec/vp8_enc/venc_vp8_if.h      |  149 +++++++
 .../platform/mtk-vcodec/vp8_enc/venc_vp8_vpu.c     |  240 +++++++++++
 .../platform/mtk-vcodec/vp8_enc/venc_vp8_vpu.h     |   28 ++
 7 files changed, 850 insertions(+)
 create mode 100644 drivers/media/platform/mtk-vcodec/vp8_enc/Makefile
 create mode 100644 drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_if.c
 create mode 100644 drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_if.h
 create mode 100644 drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_vpu.c
 create mode 100644 drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_vpu.h

diff --git a/drivers/media/platform/mtk-vcodec/Makefile b/drivers/media/platform/mtk-vcodec/Makefile
index ce38689..f4ef502 100644
--- a/drivers/media/platform/mtk-vcodec/Makefile
+++ b/drivers/media/platform/mtk-vcodec/Makefile
@@ -5,4 +5,6 @@ obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += mtk_vcodec_intr.o \
 				       mtk_vcodec_enc_pm.o \
 				       venc_drv_if.o
 
+obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += vp8_enc/
+
 ccflags-y += -I$(srctree)/drivers/media/platform/mtk-vpu
diff --git a/drivers/media/platform/mtk-vcodec/venc_drv_if.c b/drivers/media/platform/mtk-vcodec/venc_drv_if.c
index daa8e93..d293f2c 100644
--- a/drivers/media/platform/mtk-vcodec/venc_drv_if.c
+++ b/drivers/media/platform/mtk-vcodec/venc_drv_if.c
@@ -24,6 +24,7 @@
 #include "mtk_vpu.h"
 
 #include "venc_drv_base.h"
+#include "vp8_enc/venc_vp8_if.h"
 
 int venc_if_create(struct mtk_vcodec_ctx *ctx, unsigned int fourcc)
 {
@@ -34,6 +35,8 @@ int venc_if_create(struct mtk_vcodec_ctx *ctx, unsigned int fourcc)
 
 	switch (fourcc) {
 	case V4L2_PIX_FMT_VP8:
+                ctx->enc_if = get_vp8_enc_comm_if();
+                break;
 	case V4L2_PIX_FMT_H264:
 	default:
 		return -EINVAL;
diff --git a/drivers/media/platform/mtk-vcodec/vp8_enc/Makefile b/drivers/media/platform/mtk-vcodec/vp8_enc/Makefile
new file mode 100644
index 0000000..0f515c7
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/vp8_enc/Makefile
@@ -0,0 +1,6 @@
+obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += venc_vp8_if.o venc_vp8_vpu.o
+
+ccflags-y += \
+    -I$(srctree)/drivers/media/platform/mtk-vcodec/ \
+    -I$(srctree)/drivers/media/platform/mtk-vcodec/vp8_enc \
+    -I$(srctree)/drivers/media/platform/mtk-vpu
diff --git a/drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_if.c b/drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_if.c
new file mode 100644
index 0000000..4253317
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_if.c
@@ -0,0 +1,422 @@
+/*
+ * Copyright (c) 2015 MediaTek Inc.
+ * Author: Daniel Hsiao <daniel.hsiao@mediatek.com>
+ *         PoChun Lin <pochun.lin@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/interrupt.h>
+#include <linux/kernel.h>
+#include <linux/slab.h>
+
+#include "mtk_vcodec_drv.h"
+#include "mtk_vcodec_util.h"
+#include "mtk_vcodec_intr.h"
+#include "mtk_vcodec_enc.h"
+#include "mtk_vcodec_pm.h"
+#include "mtk_vpu.h"
+
+#include "venc_vp8_if.h"
+#include "venc_vp8_vpu.h"
+
+#define VENC_PIC_BITSTREAM_BYTE_CNT 0x0098
+#define VENC_PIC_BITSTREAM_BYTE_CNT1 0x00e8
+#define VENC_IRQ_STATUS_ENC_FRM_INT 0x04
+
+#define MAX_AC_TAG_SZ 10
+
+static inline void vp8_enc_write_reg(struct venc_vp8_inst *inst, u32 addr,
+				     u32 val)
+{
+	writel(val, inst->hw_base + addr);
+}
+
+static inline u32 vp8_enc_read_reg(struct venc_vp8_inst *inst, u32 addr)
+{
+	return readl(inst->hw_base + addr);
+}
+
+static void vp8_enc_free_work_buf(struct venc_vp8_inst *inst)
+{
+	int i;
+
+	mtk_vcodec_debug_enter(inst);
+
+	/* Buffers need to be freed by AP. */
+	for (i = 0; i < VENC_VP8_VPU_WORK_BUF_MAX; i++)
+		if (inst->work_bufs[i].va != NULL)
+			mtk_vcodec_mem_free(inst->ctx,
+					    &inst->work_bufs[i]);
+
+	mtk_vcodec_debug_leave(inst);
+}
+
+static int vp8_enc_alloc_work_buf(struct venc_vp8_inst *inst, void *param)
+{
+	int i;
+	int ret = 0;
+	struct venc_vp8_vpu_buf *wb = inst->vpu_inst.drv->work_bufs;
+	struct venc_enc_prm *enc_param = param;
+
+	mtk_vcodec_debug_enter(inst);
+
+	for (i = 0; i < VENC_VP8_VPU_WORK_BUF_MAX; i++) {
+		/*
+		 * Only temporal scalability mode will use RC_CODE2 & RC_CODE3
+		 * Each three temporal layer has its own rate control code.
+		 */
+		if ((i == VENC_VP8_VPU_WORK_BUF_RC_CODE2 ||
+		     i == VENC_VP8_VPU_WORK_BUF_RC_CODE3) && !inst->ts_mode)
+			continue;
+
+		/*
+		 * This 'wb' structure is set by VPU side and shared to AP for
+		 * buffer allocation and IO virtual addr mapping. For most of
+		 * the buffers, AP will allocate the buffer according to 'size'
+		 * field and store the IO virtual addr in 'iova' field. For the
+		 * RC_CODEx buffers, they are pre-allocated in the VPU side
+		 * because they are inside VPU SRAM, and save the VPU addr in
+		 * the 'vpua' field. The AP will translate the VPU addr to the
+		 * corresponding IO virtual addr and store in 'iova' field.
+		 */
+		if (i < VENC_VP8_VPU_WORK_BUF_RC_CODE) {
+			inst->work_bufs[i].size = wb[i].size;
+			ret = mtk_vcodec_mem_alloc(inst->ctx,
+						   &inst->work_bufs[i]);
+			if (ret) {
+				mtk_vcodec_err(inst,
+					       "cannot alloc work_bufs[%d]", i);
+				goto err_alloc;
+			}
+			wb[i].iova = inst->work_bufs[i].dma_addr;
+		} else if (i == VENC_VP8_VPU_WORK_BUF_SRC_LUMA ||
+			   i == VENC_VP8_VPU_WORK_BUF_SRC_CHROMA ||
+			   i == VENC_VP8_VPU_WORK_BUF_SRC_CHROMA_CB ||
+			   i == VENC_VP8_VPU_WORK_BUF_SRC_CHROMA_CR) {
+			inst->work_bufs[i].size = wb[i].size;
+			inst->work_bufs[i].dma_addr = 0;
+			inst->work_bufs[i].va = NULL;
+			wb[i].iova = inst->work_bufs[i].dma_addr;
+		} else {
+			void *tmp_va;
+
+			tmp_va = vpu_mapping_dm_addr(inst->dev, wb[i].vpua);
+			inst->work_bufs[i].size = wb[i].size;
+			ret = mtk_vcodec_mem_alloc(inst->ctx,
+						   &inst->work_bufs[i]);
+			if (ret) {
+				mtk_vcodec_err(inst,
+					       "cannot alloc work_bufs[%d]", i);
+				goto err_alloc;
+			}
+			memcpy(inst->work_bufs[i].va, tmp_va, wb[i].size);
+			wb[i].iova = inst->work_bufs[i].dma_addr;
+		}
+		mtk_vcodec_debug(inst,
+				 "work_bufs[%d] va=0x%p,iova=0x%p,size=0x%lx",
+				 i, inst->work_bufs[i].va,
+				 (void *)inst->work_bufs[i].dma_addr,
+				 inst->work_bufs[i].size);
+	}
+
+	if (enc_param->input_fourcc == VENC_YUV_FORMAT_NV12 ||
+	    enc_param->input_fourcc == VENC_YUV_FORMAT_NV21) {
+		enc_param->sizeimage[0] =
+			inst->work_bufs[VENC_VP8_VPU_WORK_BUF_SRC_LUMA].size;
+		enc_param->sizeimage[1] =
+			inst->work_bufs[VENC_VP8_VPU_WORK_BUF_SRC_CHROMA].size;
+		enc_param->sizeimage[2] = 0;
+	} else {
+		enc_param->sizeimage[0] =
+			inst->work_bufs[VENC_VP8_VPU_WORK_BUF_SRC_LUMA].size;
+		enc_param->sizeimage[1] =
+			inst->work_bufs[VENC_VP8_VPU_WORK_BUF_SRC_CHROMA_CB].size;
+		enc_param->sizeimage[2] =
+			inst->work_bufs[VENC_VP8_VPU_WORK_BUF_SRC_CHROMA_CR].size;
+	}
+	mtk_vcodec_debug_leave(inst);
+
+	return ret;
+
+err_alloc:
+	vp8_enc_free_work_buf(inst);
+
+	return ret;
+}
+
+static unsigned int vp8_enc_wait_venc_done(struct venc_vp8_inst *inst)
+{
+	unsigned int irq_status = 0;
+	struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)inst->ctx;
+
+	mtk_vcodec_wait_for_done_ctx(ctx, MTK_INST_IRQ_RECEIVED,
+				     WAIT_INTR_TIMEOUT, true);
+	irq_status = ctx->irq_status;
+	mtk_vcodec_debug(inst, "isr return %x", irq_status);
+
+	return irq_status;
+}
+
+/*
+ * Compose ac_tag, bitstream header and bitstream payload into
+ * one bitstream buffer.
+ */
+static int vp8_enc_compose_one_frame(struct venc_vp8_inst *inst,
+				     struct mtk_vcodec_mem *bs_buf,
+				     unsigned int *bs_size)
+{
+	unsigned int is_key;
+	u32 bs_size_frm;
+	u32 bs_hdr_len;
+	unsigned int ac_tag_sz;
+	u8 ac_tag[MAX_AC_TAG_SZ];
+
+	bs_size_frm = vp8_enc_read_reg(inst,
+				       VENC_PIC_BITSTREAM_BYTE_CNT);
+	bs_hdr_len = vp8_enc_read_reg(inst,
+				      VENC_PIC_BITSTREAM_BYTE_CNT1);
+
+	/* if a frame is key frame, is_key is 0 */
+	is_key = !inst->is_key_frm;
+	*(u32 *)ac_tag = __cpu_to_le32((bs_hdr_len << 5) | 0x10 | is_key);
+	/* key frame */
+	if (is_key == 0) {
+		ac_tag[3] = 0x9d;
+		ac_tag[4] = 0x01;
+		ac_tag[5] = 0x2a;
+		ac_tag[6] = inst->vpu_inst.drv->config.pic_w;
+		ac_tag[7] = inst->vpu_inst.drv->config.pic_w >> 8;
+		ac_tag[8] = inst->vpu_inst.drv->config.pic_h;
+		ac_tag[9] = inst->vpu_inst.drv->config.pic_h >> 8;
+	}
+
+	if (is_key == 0)
+		ac_tag_sz = MAX_AC_TAG_SZ;
+	else
+		ac_tag_sz = 3;
+
+	if (bs_buf->size <= bs_hdr_len + bs_size_frm + ac_tag_sz) {
+		mtk_vcodec_err(inst, "bitstream buf size is too small(%ld)",
+			       bs_buf->size);
+		return -EINVAL;
+	}
+
+	/*
+	* (1) The vp8 bitstream header and body are generated by the HW vp8
+	* encoder separately at the same time. We cannot know the bitstream
+	* header length in advance.
+	* (2) From the vp8 spec, there is no stuffing byte allowed between the
+	* ac tag, bitstream header and bitstream body.
+	*/
+	memmove(bs_buf->va + bs_hdr_len + ac_tag_sz,
+		bs_buf->va, bs_size_frm);
+	memcpy(bs_buf->va + ac_tag_sz,
+	       inst->work_bufs[VENC_VP8_VPU_WORK_BUF_BS_HD].va,
+	       bs_hdr_len);
+	memcpy(bs_buf->va, ac_tag, ac_tag_sz);
+	*bs_size = bs_size_frm + bs_hdr_len + ac_tag_sz;
+
+	return 0;
+}
+
+static int vp8_enc_encode_frame(struct venc_vp8_inst *inst,
+				struct venc_frm_buf *frm_buf,
+				struct mtk_vcodec_mem *bs_buf,
+				unsigned int *bs_size)
+{
+	int ret = 0;
+	unsigned int irq_status;
+
+	mtk_vcodec_debug(inst, "->frm_cnt=%d", inst->frm_cnt);
+
+	ret = vp8_enc_vpu_encode(inst, frm_buf, bs_buf);
+	if (ret)
+		return ret;
+
+	irq_status = vp8_enc_wait_venc_done(inst);
+	if (irq_status != VENC_IRQ_STATUS_ENC_FRM_INT) {
+		mtk_vcodec_err(inst, "irq_status=%d failed", irq_status);
+		return -EINVAL;
+	}
+
+	if (vp8_enc_compose_one_frame(inst, bs_buf, bs_size)) {
+		mtk_vcodec_err(inst, "vp8_enc_compose_one_frame failed");
+		return -EINVAL;
+	}
+
+	inst->frm_cnt++;
+	mtk_vcodec_debug(inst, "<-size=%d key_frm=%d", *bs_size,
+			 inst->is_key_frm);
+
+	return ret;
+}
+
+static int vp8_enc_init(struct mtk_vcodec_ctx *ctx, unsigned long *handle)
+{
+	int ret = 0;
+	struct venc_vp8_inst *inst;
+
+	inst = kzalloc(sizeof(*inst), GFP_KERNEL);
+	if (!inst)
+		return -ENOMEM;
+
+	inst->ctx = ctx;
+	inst->dev = mtk_vcodec_get_plat_dev(ctx);
+	inst->hw_base = mtk_vcodec_get_reg_addr(inst->ctx, VENC_LT_SYS);
+
+	mtk_vcodec_debug_enter(inst);
+	ret = vp8_enc_vpu_init(inst);
+	if (ret)
+		kfree(inst);
+	else
+		(*handle) = (unsigned long)inst;
+
+	mtk_vcodec_debug_leave(inst);
+
+	return ret;
+}
+
+static int vp8_enc_encode(unsigned long handle,
+			  enum venc_start_opt opt,
+			  struct venc_frm_buf *frm_buf,
+			  struct mtk_vcodec_mem *bs_buf,
+			  struct venc_done_result *result)
+{
+	int ret = 0;
+	struct venc_vp8_inst *inst = (struct venc_vp8_inst *)handle;
+	struct mtk_vcodec_ctx *ctx = inst->ctx;
+
+	mtk_vcodec_debug_enter(inst);
+
+	enable_irq(ctx->dev->enc_lt_irq);
+
+	switch (opt) {
+	case VENC_START_OPT_ENCODE_FRAME:
+		ret = vp8_enc_encode_frame(inst, frm_buf, bs_buf,
+					   &result->bs_size);
+		if (ret)
+			goto encode_err;
+		result->is_key_frm = inst->is_key_frm;
+		break;
+
+	default:
+		mtk_vcodec_err(inst, "opt not support:%d", opt);
+		ret = -EINVAL;
+		break;
+	}
+
+encode_err:
+	if (ret)
+		result->msg = VENC_MESSAGE_ERR;
+	else
+		result->msg = VENC_MESSAGE_OK;
+
+	disable_irq(ctx->dev->enc_lt_irq);
+	mtk_vcodec_debug_leave(inst);
+
+	return ret;
+}
+
+static int vp8_enc_set_param(unsigned long handle,
+			     enum venc_set_param_type type, void *in)
+{
+	int ret = 0;
+	struct venc_vp8_inst *inst = (struct venc_vp8_inst *)handle;
+	struct venc_enc_prm *enc_prm;
+
+	mtk_vcodec_debug(inst, "->type=%d", type);
+
+	switch (type) {
+	case VENC_SET_PARAM_ENC:
+		enc_prm = in;
+		ret = vp8_enc_vpu_set_param(inst, type, enc_prm);
+		if (ret)
+			break;
+		if (inst->work_buf_allocated == 1) {
+			vp8_enc_free_work_buf(inst);
+			inst->work_buf_allocated = 0;
+		}
+		if (inst->work_buf_allocated == 0) {
+			ret = vp8_enc_alloc_work_buf(inst, enc_prm);
+			if (ret)
+				break;
+			inst->work_buf_allocated = 1;
+		}
+		break;
+
+	case VENC_SET_PARAM_FORCE_INTRA:
+		ret = vp8_enc_vpu_set_param(inst, type, 0);
+		break;
+
+	case VENC_SET_PARAM_ADJUST_BITRATE:
+		enc_prm = in;
+		ret = vp8_enc_vpu_set_param(inst, type, &enc_prm->bitrate);
+		break;
+
+	case VENC_SET_PARAM_ADJUST_FRAMERATE:
+		enc_prm = in;
+		ret = vp8_enc_vpu_set_param(inst, type, &enc_prm->frm_rate);
+		break;
+
+	case VENC_SET_PARAM_I_FRAME_INTERVAL:
+		ret = vp8_enc_vpu_set_param(inst, type, in);
+		break;
+
+	/*
+	 * VENC_SET_PARAM_TS_MODE must be called before
+	 * VENC_SET_PARAM_ENC
+	 */
+	case VENC_SET_PARAM_TS_MODE:
+		inst->ts_mode = 1;
+		mtk_vcodec_debug(inst, "set ts_mode");
+		break;
+
+	default:
+		mtk_vcodec_err(inst, "type not support:%d", type);
+		ret = -EINVAL;
+		break;
+	}
+
+	mtk_vcodec_debug_leave(inst);
+
+	return ret;
+}
+
+static int vp8_enc_deinit(unsigned long handle)
+{
+	int ret = 0;
+	struct venc_vp8_inst *inst = (struct venc_vp8_inst *)handle;
+
+	mtk_vcodec_debug_enter(inst);
+
+	ret = vp8_enc_vpu_deinit(inst);
+
+	if (inst->work_buf_allocated)
+		vp8_enc_free_work_buf(inst);
+
+	mtk_vcodec_debug_leave(inst);
+	kfree(inst);
+
+	return ret;
+}
+
+static struct venc_common_if venc_vp8_if = {
+	vp8_enc_init,
+	vp8_enc_encode,
+	vp8_enc_set_param,
+	vp8_enc_deinit,
+};
+
+struct venc_common_if *get_vp8_enc_comm_if(void)
+{
+	return &venc_vp8_if;
+}
diff --git a/drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_if.h b/drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_if.h
new file mode 100644
index 0000000..b32220e
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_if.h
@@ -0,0 +1,149 @@
+/*
+ * Copyright (c) 2015 MediaTek Inc.
+ * Author: Daniel Hsiao <daniel.hsiao@mediatek.com>
+ *         PoChun Lin <pochun.lin@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef _VENC_VP8_IF_H_
+#define _VENC_VP8_IF_H_
+
+#include "venc_drv_base.h"
+
+/**
+ * enum venc_vp8_vpu_work_buf - vp8 encoder buffer index
+ */
+enum venc_vp8_vpu_work_buf {
+	VENC_VP8_VPU_WORK_BUF_LUMA,
+	VENC_VP8_VPU_WORK_BUF_LUMA2,
+	VENC_VP8_VPU_WORK_BUF_LUMA3,
+	VENC_VP8_VPU_WORK_BUF_CHROMA,
+	VENC_VP8_VPU_WORK_BUF_CHROMA2,
+	VENC_VP8_VPU_WORK_BUF_CHROMA3,
+	VENC_VP8_VPU_WORK_BUF_MV_INFO,
+	VENC_VP8_VPU_WORK_BUF_BS_HD,
+	VENC_VP8_VPU_WORK_BUF_PROB_BUF,
+	VENC_VP8_VPU_WORK_BUF_RC_INFO,
+	VENC_VP8_VPU_WORK_BUF_RC_CODE,
+	VENC_VP8_VPU_WORK_BUF_RC_CODE2,
+	VENC_VP8_VPU_WORK_BUF_RC_CODE3,
+	VENC_VP8_VPU_WORK_BUF_SRC_LUMA,
+	VENC_VP8_VPU_WORK_BUF_SRC_CHROMA,
+	VENC_VP8_VPU_WORK_BUF_SRC_CHROMA_CB,
+	VENC_VP8_VPU_WORK_BUF_SRC_CHROMA_CR,
+	VENC_VP8_VPU_WORK_BUF_MAX,
+};
+
+/*
+ * struct venc_vp8_vpu_config - Structure for vp8 encoder configuration
+ * @input_fourcc: input fourcc
+ * @bitrate: target bitrate (in bps)
+ * @pic_w: picture width
+ * @pic_h: picture height
+ * @buf_w: buffer width (with 16 alignment)
+ * @buf_h: buffer height (with 16 alignment)
+ * @intra_period: intra frame period
+ * @framerate: frame rate
+ * @ts_mode: temporal scalability mode (0: disable, 1: enable)
+ *	     support three temporal layers - 0: 7.5fps 1: 7.5fps 2: 15fps.
+ */
+struct venc_vp8_vpu_config {
+	u32 input_fourcc;
+	u32 bitrate;
+	u32 pic_w;
+	u32 pic_h;
+	u32 buf_w;
+	u32 buf_h;
+	u32 intra_period;
+	u32 framerate;
+	u32 ts_mode;
+};
+
+/*
+ * struct venc_vp8_vpu_buf -Structure for buffer information
+ * @align: buffer alignment (in bytes)
+ * @iova: IO virtual address
+ * @vpua: VPU side memory addr which is used by RC_CODE
+ * @size: buffer size (in bytes)
+ */
+struct venc_vp8_vpu_buf {
+	u32 align;
+	u32 iova;
+	u32 vpua;
+	u32 size;
+};
+
+/*
+ * struct venc_vp8_vpu_drv - Structure for VPU driver control and info share
+ * This structure is allocated in VPU side and shared to AP side.
+ * @config: vp8 encoder configuration
+ * @work_bufs: working buffer information in VPU side
+ * The work_bufs here is for storing the 'size' info shared to AP side.
+ * The similar item in struct venc_vp8_inst is for memory allocation
+ * in AP side. The AP driver will copy the 'size' from here to the one in
+ * struct mtk_vcodec_mem, then invoke mtk_vcodec_mem_alloc to allocate
+ * the buffer. After that, bypass the 'dma_addr' to the 'iova' field here for
+ * register setting in VPU side.
+ */
+struct venc_vp8_vpu_drv {
+	struct venc_vp8_vpu_config config;
+	struct venc_vp8_vpu_buf work_bufs[VENC_VP8_VPU_WORK_BUF_MAX];
+};
+
+/*
+ * struct venc_vp8_vpu_inst - vp8 encoder VPU driver instance
+ * @wq_hd: wait queue used for vpu cmd trigger then wait vpu interrupt done
+ * @signaled: flag used for checking vpu interrupt done
+ * @failure: flag to show vpu cmd succeeds or not
+ * @state: enum venc_ipi_msg_enc_state
+ * @id: VPU instance id
+ * @drv: driver structure allocated by VPU side and shared to AP side for
+ *	 control and info share
+ */
+struct venc_vp8_vpu_inst {
+	wait_queue_head_t wq_hd;
+	int signaled;
+	int failure;
+	int state;
+	unsigned int id;
+	struct venc_vp8_vpu_drv *drv;
+};
+
+/*
+ * struct venc_vp8_inst - vp8 encoder AP driver instance
+ * @hw_base: vp8 encoder hardware register base
+ * @work_bufs: working buffer
+ * @work_buf_allocated: working buffer allocated flag
+ * @frm_cnt: encoded frame count, it's used for I-frame judgement and
+ *	     reset when force intra cmd received.
+ * @ts_mode: temporal scalability mode (0: disable, 1: enable)
+ *	     support three temporal layers - 0: 7.5fps 1: 7.5fps 2: 15fps.
+ * @is_key_frm: key frame flag
+ * @vpu_inst: VPU instance to exchange information between AP and VPU
+ * @ctx: context for v4l2 layer integration
+ * @dev: device for v4l2 layer integration
+ */
+struct venc_vp8_inst {
+	void __iomem *hw_base;
+	struct mtk_vcodec_mem work_bufs[VENC_VP8_VPU_WORK_BUF_MAX];
+	bool work_buf_allocated;
+	unsigned int frm_cnt;
+	unsigned int ts_mode;
+	unsigned int is_key_frm;
+	struct venc_vp8_vpu_inst vpu_inst;
+	void *ctx;
+	struct platform_device *dev;
+};
+
+struct venc_common_if *get_vp8_enc_comm_if(void);
+
+#endif
diff --git a/drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_vpu.c b/drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_vpu.c
new file mode 100644
index 0000000..5f797c9
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_vpu.c
@@ -0,0 +1,240 @@
+/*
+ * Copyright (c) 2015 MediaTek Inc.
+ * Author: Daniel Hsiao <daniel.hsiao@mediatek.com>
+ *         PoChun Lin <pochun.lin@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#include "mtk_vpu.h"
+
+#include "venc_vp8_if.h"
+#include "venc_vp8_vpu.h"
+#include "venc_ipi_msg.h"
+
+static void handle_vp8_enc_init_msg(struct venc_vp8_inst *inst, void *data)
+{
+	struct venc_vpu_ipi_msg_init *msg = data;
+
+	inst->vpu_inst.id = msg->inst_id;
+	inst->vpu_inst.drv = (struct venc_vp8_vpu_drv *)
+		vpu_mapping_dm_addr(inst->dev, msg->inst_id);
+}
+
+static void handle_vp8_enc_encode_msg(struct venc_vp8_inst *inst, void *data)
+{
+	struct venc_vpu_ipi_msg_enc *msg = data;
+
+	inst->vpu_inst.state = msg->state;
+	inst->is_key_frm = msg->key_frame;
+}
+
+static void vp8_enc_vpu_ipi_handler(void *data, unsigned int len, void *priv)
+{
+	struct venc_vpu_ipi_msg_common *msg = data;
+	struct venc_vp8_inst *inst = (struct venc_vp8_inst *)msg->venc_inst;
+
+	mtk_vcodec_debug(inst, "->msg_id=%x status=%d",
+			 msg->msg_id, msg->status);
+
+	switch (msg->msg_id) {
+	case VPU_IPIMSG_VP8_ENC_INIT_DONE:
+		handle_vp8_enc_init_msg(inst, data);
+		break;
+	case VPU_IPIMSG_VP8_ENC_SET_PARAM_DONE:
+		break;
+	case VPU_IPIMSG_VP8_ENC_ENCODE_DONE:
+		handle_vp8_enc_encode_msg(inst, data);
+		break;
+	case VPU_IPIMSG_VP8_ENC_DEINIT_DONE:
+		break;
+	default:
+		mtk_vcodec_err(inst, "unknown msg id=%x", msg->msg_id);
+		break;
+	}
+
+	inst->vpu_inst.signaled = 1;
+	inst->vpu_inst.failure = (msg->status != VENC_IPI_MSG_STATUS_OK);
+
+	mtk_vcodec_debug_leave(inst);
+}
+
+static int vp8_enc_vpu_send_msg(struct venc_vp8_inst *inst, void *msg,
+				int len)
+{
+	int status;
+
+	mtk_vcodec_debug_enter(inst);
+	status = vpu_ipi_send(inst->dev, IPI_VENC_VP8, (void *)msg, len);
+	if (status) {
+		mtk_vcodec_err(inst,
+			       "vpu_ipi_send msg_id=%x len=%d failed status=%d",
+			       *(unsigned int *)msg, len, status);
+		return -EINVAL;
+	}
+	mtk_vcodec_debug_leave(inst);
+
+	return 0;
+}
+
+int vp8_enc_vpu_init(struct venc_vp8_inst *inst)
+{
+	int status;
+	struct venc_ap_ipi_msg_init out;
+
+	mtk_vcodec_debug_enter(inst);
+
+	init_waitqueue_head(&inst->vpu_inst.wq_hd);
+	inst->vpu_inst.signaled = 0;
+	inst->vpu_inst.failure = 0;
+
+	status = vpu_ipi_register(inst->dev, IPI_VENC_VP8,
+				  vp8_enc_vpu_ipi_handler,
+				  "vp8_enc", NULL);
+	if (status) {
+		mtk_vcodec_err(inst,
+			       "vpu_ipi_register failed status=%d", status);
+		return -EINVAL;
+	}
+
+	out.msg_id = AP_IPIMSG_VP8_ENC_INIT;
+	out.venc_inst = (unsigned long)inst;
+	if (vp8_enc_vpu_send_msg(inst, &out, sizeof(out)) ||
+	    inst->vpu_inst.failure) {
+		mtk_vcodec_err(inst, "failed");
+		return -EINVAL;
+	}
+
+	mtk_vcodec_debug_leave(inst);
+
+	return 0;
+}
+
+int vp8_enc_vpu_set_param(struct venc_vp8_inst *inst, unsigned int id,
+			  void *param)
+{
+	struct venc_ap_ipi_msg_set_param out;
+
+	mtk_vcodec_debug_enter(inst);
+
+	out.msg_id = AP_IPIMSG_VP8_ENC_SET_PARAM;
+	out.inst_id = inst->vpu_inst.id;
+	out.param_id = id;
+	switch (id) {
+	case VENC_SET_PARAM_ENC: {
+		struct venc_enc_prm *enc_param = (struct venc_enc_prm *)param;
+
+		inst->vpu_inst.drv->config.input_fourcc =
+			enc_param->input_fourcc;
+		inst->vpu_inst.drv->config.bitrate = enc_param->bitrate;
+		inst->vpu_inst.drv->config.pic_w = enc_param->width;
+		inst->vpu_inst.drv->config.pic_h = enc_param->height;
+		inst->vpu_inst.drv->config.buf_w = enc_param->buf_width;
+		inst->vpu_inst.drv->config.buf_h = enc_param->buf_height;
+		inst->vpu_inst.drv->config.intra_period =
+			enc_param->intra_period;
+		inst->vpu_inst.drv->config.framerate = enc_param->frm_rate;
+		inst->vpu_inst.drv->config.ts_mode = inst->ts_mode;
+		out.data_item = 0;
+		break;
+	}
+	case VENC_SET_PARAM_FORCE_INTRA:
+		out.data_item = 0;
+		break;
+	case VENC_SET_PARAM_ADJUST_BITRATE:
+		out.data_item = 1;
+		out.data[0] = *(unsigned int *)param;
+		break;
+	case VENC_SET_PARAM_ADJUST_FRAMERATE:
+		out.data_item = 1;
+		out.data[0] = *(unsigned int *)param;
+		break;
+	case VENC_SET_PARAM_I_FRAME_INTERVAL:
+		out.data_item = 1;
+		out.data[0] = *(unsigned int *)param;
+		break;
+	}
+	if (vp8_enc_vpu_send_msg(inst, &out, sizeof(out)) ||
+	    inst->vpu_inst.failure) {
+		mtk_vcodec_err(inst, "failed");
+		return -EINVAL;
+	}
+
+	mtk_vcodec_debug_leave(inst);
+
+	return 0;
+}
+
+int vp8_enc_vpu_encode(struct venc_vp8_inst *inst,
+		       struct venc_frm_buf *frm_buf,
+		       struct mtk_vcodec_mem *bs_buf)
+{
+	struct venc_ap_ipi_msg_enc out;
+
+	mtk_vcodec_debug_enter(inst);
+
+	out.msg_id = AP_IPIMSG_VP8_ENC_ENCODE;
+	out.inst_id = inst->vpu_inst.id;
+	if (frm_buf) {
+		if ((frm_buf->fb_addr.dma_addr % 16 == 0) &&
+		    (frm_buf->fb_addr1.dma_addr % 16 == 0) &&
+		    (frm_buf->fb_addr2.dma_addr % 16 == 0)) {
+			out.input_addr[0] = frm_buf->fb_addr.dma_addr;
+			out.input_addr[1] = frm_buf->fb_addr1.dma_addr;
+			out.input_addr[2] = frm_buf->fb_addr2.dma_addr;
+		} else {
+			mtk_vcodec_err(inst, "dma_addr not align to 16");
+			return -EINVAL;
+		}
+	} else {
+		out.input_addr[0] = 0;
+		out.input_addr[1] = 0;
+		out.input_addr[2] = 0;
+	}
+	if (bs_buf) {
+		out.bs_addr = bs_buf->dma_addr;
+		out.bs_size = bs_buf->size;
+	} else {
+		out.bs_addr = 0;
+		out.bs_size = 0;
+	}
+	if (vp8_enc_vpu_send_msg(inst, &out, sizeof(out)) ||
+	    inst->vpu_inst.failure) {
+		mtk_vcodec_err(inst, "failed");
+		return -EINVAL;
+	}
+
+	mtk_vcodec_debug(inst, "state=%d key_frm=%d",
+			 inst->vpu_inst.state, inst->is_key_frm);
+
+	mtk_vcodec_debug_leave(inst);
+
+	return 0;
+}
+
+int vp8_enc_vpu_deinit(struct venc_vp8_inst *inst)
+{
+	struct venc_ap_ipi_msg_deinit out;
+
+	mtk_vcodec_debug_enter(inst);
+
+	out.msg_id = AP_IPIMSG_VP8_ENC_DEINIT;
+	out.inst_id = inst->vpu_inst.id;
+	if (vp8_enc_vpu_send_msg(inst, &out, sizeof(out)) ||
+	    inst->vpu_inst.failure) {
+		mtk_vcodec_err(inst, "failed");
+		return -EINVAL;
+	}
+
+	mtk_vcodec_debug_leave(inst);
+
+	return 0;
+}
diff --git a/drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_vpu.h b/drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_vpu.h
new file mode 100644
index 0000000..d7d79a8
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_vpu.h
@@ -0,0 +1,28 @@
+/*
+ * Copyright (c) 2015 MediaTek Inc.
+ * Author: Daniel Hsiao <daniel.hsiao@mediatek.com>
+ *         PoChun Lin <pochun.lin@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef _VENC_VP8_VPU_H_
+#define _VENC_VP8_VPU_H_
+
+int vp8_enc_vpu_init(struct venc_vp8_inst *inst);
+int vp8_enc_vpu_set_param(struct venc_vp8_inst *inst, unsigned int id,
+			  void *param);
+int vp8_enc_vpu_encode(struct venc_vp8_inst *inst,
+		       struct venc_frm_buf *frm_buf,
+		       struct mtk_vcodec_mem *bs_buf);
+int vp8_enc_vpu_deinit(struct venc_vp8_inst *inst);
+
+#endif
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 102+ messages in thread

* [PATCH v4 7/8] [media] vcodec: mediatek: Add Mediatek H264 Video Encoder Driver
  2016-02-04 11:35             ` Tiffany Lin
  (?)
@ 2016-02-04 11:35               ` Tiffany Lin
  -1 siblings, 0 replies; 102+ messages in thread
From: Tiffany Lin @ 2016-02-04 11:35 UTC (permalink / raw)
  To: Hans Verkuil, daniel.thompson, Rob Herring,
	Mauro Carvalho Chehab, Matthias Brugger, Daniel Kurtz,
	Pawel Osciak
  Cc: Eddie Huang, Yingjoe Chen, devicetree, linux-kernel,
	linux-arm-kernel, linux-media, linux-mediatek, PoChun.Lin,
	Tiffany Lin, Daniel Hsiao

Add h264 encoder driver for MT8173

Signed-off-by: Daniel Hsiao <daniel.hsiao@mediatek.com>
Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
---
 drivers/media/platform/mtk-vcodec/Makefile         |    3 +-
 .../media/platform/mtk-vcodec/h264_enc/Makefile    |    6 +
 .../platform/mtk-vcodec/h264_enc/venc_h264_if.c    |  540 ++++++++++++++++++++
 .../platform/mtk-vcodec/h264_enc/venc_h264_if.h    |  165 ++++++
 .../platform/mtk-vcodec/h264_enc/venc_h264_vpu.c   |  309 +++++++++++
 .../platform/mtk-vcodec/h264_enc/venc_h264_vpu.h   |   30 ++
 drivers/media/platform/mtk-vcodec/venc_drv_if.c    |    3 +
 7 files changed, 1055 insertions(+), 1 deletion(-)
 create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/Makefile
 create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.c
 create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.h
 create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.c
 create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.h

diff --git a/drivers/media/platform/mtk-vcodec/Makefile b/drivers/media/platform/mtk-vcodec/Makefile
index f4ef502..f47dfc7 100644
--- a/drivers/media/platform/mtk-vcodec/Makefile
+++ b/drivers/media/platform/mtk-vcodec/Makefile
@@ -5,6 +5,7 @@ obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += mtk_vcodec_intr.o \
 				       mtk_vcodec_enc_pm.o \
 				       venc_drv_if.o
 
-obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += vp8_enc/
+
+obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += vp8_enc/ h264_enc/
 
 ccflags-y += -I$(srctree)/drivers/media/platform/mtk-vpu
diff --git a/drivers/media/platform/mtk-vcodec/h264_enc/Makefile b/drivers/media/platform/mtk-vcodec/h264_enc/Makefile
new file mode 100644
index 0000000..765b45f
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/h264_enc/Makefile
@@ -0,0 +1,6 @@
+obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += venc_h264_if.o venc_h264_vpu.o
+
+ccflags-y += \
+    -I$(srctree)/drivers/media/platform/mtk-vcodec/ \
+    -I$(srctree)/drivers/media/platform/mtk-vcodec/h264_enc \
+    -I$(srctree)/drivers/media/platform/mtk-vpu
diff --git a/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.c b/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.c
new file mode 100644
index 0000000..c35fb26
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.c
@@ -0,0 +1,540 @@
+/*
+ * Copyright (c) 2015 MediaTek Inc.
+ * Author: Jungchang Tsao <jungchang.tsao@mediatek.com>
+ *         Daniel Hsiao <daniel.hsiao@mediatek.com>
+ *         PoChun Lin <pochun.lin@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/interrupt.h>
+#include <linux/kernel.h>
+#include <linux/slab.h>
+
+#include "mtk_vcodec_drv.h"
+#include "mtk_vcodec_util.h"
+#include "mtk_vcodec_intr.h"
+#include "mtk_vcodec_enc.h"
+#include "mtk_vcodec_pm.h"
+#include "mtk_vpu.h"
+
+#include "venc_h264_if.h"
+#include "venc_h264_vpu.h"
+
+#define VENC_PIC_BITSTREAM_BYTE_CNT 0x0098
+
+enum venc_h264_irq_status {
+	H264_IRQ_STATUS_ENC_SPS_INT = (1 << 0),
+	H264_IRQ_STATUS_ENC_PPS_INT = (1 << 1),
+	H264_IRQ_STATUS_ENC_FRM_INT = (1 << 2),
+};
+
+static inline void h264_write_reg(struct venc_h264_inst *inst, u32 addr,
+				  u32 val)
+{
+	writel(val, inst->hw_base + addr);
+}
+
+static inline u32 h264_read_reg(struct venc_h264_inst *inst, u32 addr)
+{
+	return readl(inst->hw_base + addr);
+}
+
+static void h264_enc_free_work_buf(struct venc_h264_inst *inst)
+{
+	int i;
+
+	mtk_vcodec_debug_enter(inst);
+
+	/* Except the SKIP_FRAME buffers,
+	 * other buffers need to be freed by AP.
+	 */
+	for (i = 0; i < VENC_H264_VPU_WORK_BUF_MAX; i++) {
+		if (i != VENC_H264_VPU_WORK_BUF_SKIP_FRAME)
+			if (inst->work_bufs[i].va != NULL)
+				mtk_vcodec_mem_free(inst->ctx,
+						    &inst->work_bufs[i]);
+	}
+
+	if (inst->pps_buf.va != NULL)
+		mtk_vcodec_mem_free(inst->ctx, &inst->pps_buf);
+
+	mtk_vcodec_debug_leave(inst);
+}
+
+static int h264_enc_alloc_work_buf(struct venc_h264_inst *inst, void *param)
+{
+	int i;
+	int ret = 0;
+	struct venc_h264_vpu_buf *wb = inst->vpu_inst.drv->work_bufs;
+	struct venc_enc_prm *enc_param = param;
+
+	mtk_vcodec_debug_enter(inst);
+
+	for (i = 0; i < VENC_H264_VPU_WORK_BUF_MAX; i++) {
+		/*
+		 * This 'wb' structure is set by VPU side and shared to AP for
+		 * buffer allocation and IO virtual addr mapping. For most of
+		 * the buffers, AP will allocate the buffer according to 'size'
+		 * field and store the IO virtual addr in 'iova' field. There
+		 * are two exceptions:
+		 * (1) RC_CODE buffer, it's pre-allocated in the VPU side, and
+		 * save the VPU addr in the 'vpua' field. The AP will translate
+		 * the VPU addr to the corresponding IO virtual addr and store
+		 * in 'iova' field for reg setting in VPU side.
+		 * (2) SKIP_FRAME buffer, it's pre-allocated in the VPU side,
+		 * and save the VPU addr in the 'vpua' field. The AP will
+		 * translate the VPU addr to the corresponding AP side virtual
+		 * address and do some memcpy access to move to bitstream buffer
+		 * assigned by v4l2 layer.
+		 */
+		if (i == VENC_H264_VPU_WORK_BUF_RC_CODE) {
+			void *tmp_va;
+
+			tmp_va = vpu_mapping_dm_addr(inst->dev, wb[i].vpua);
+			inst->work_bufs[i].size = wb[i].size;
+			if (mtk_vcodec_mem_alloc(inst->ctx,
+						 &inst->work_bufs[i])) {
+				mtk_vcodec_err(inst,
+					       "cannot allocate buf %d", i);
+				ret = -ENOMEM;
+				goto err_alloc;
+			}
+			memcpy(inst->work_bufs[i].va, tmp_va, wb[i].size);
+			wb[i].iova = inst->work_bufs[i].dma_addr;
+		} else if (i == VENC_H264_VPU_WORK_BUF_SKIP_FRAME) {
+			inst->work_bufs[i].size = wb[i].size;
+			inst->work_bufs[i].va = vpu_mapping_dm_addr(
+				inst->dev, wb[i].vpua);
+			inst->work_bufs[i].dma_addr = 0;
+			wb[i].iova = inst->work_bufs[i].dma_addr;
+		} else if (i == VENC_H264_VPU_WORK_BUF_SRC_LUMA ||
+			   i == VENC_H264_VPU_WORK_BUF_SRC_CHROMA ||
+			   i == VENC_H264_VPU_WORK_BUF_SRC_CHROMA_CB ||
+			   i == VENC_H264_VPU_WORK_BUF_SRC_CHROMA_CR) {
+			inst->work_bufs[i].size = wb[i].size;
+			inst->work_bufs[i].dma_addr = 0;
+			inst->work_bufs[i].va = NULL;
+			wb[i].iova = inst->work_bufs[i].dma_addr;
+		} else {
+			inst->work_bufs[i].size = wb[i].size;
+			if (mtk_vcodec_mem_alloc(inst->ctx,
+						 &inst->work_bufs[i])) {
+				mtk_vcodec_err(inst,
+					       "cannot allocate buf %d", i);
+				ret = -ENOMEM;
+				goto err_alloc;
+			}
+			wb[i].iova = inst->work_bufs[i].dma_addr;
+		}
+		mtk_vcodec_debug(inst, "buf[%d] va=0x%p iova=0x%p size=0x%lx",
+				 i, inst->work_bufs[i].va,
+				 (void *)inst->work_bufs[i].dma_addr,
+				 inst->work_bufs[i].size);
+	}
+
+	if (enc_param->input_fourcc == VENC_YUV_FORMAT_NV12 ||
+	    enc_param->input_fourcc == VENC_YUV_FORMAT_NV21) {
+		enc_param->sizeimage[0] =
+			inst->work_bufs[VENC_H264_VPU_WORK_BUF_SRC_LUMA].size;
+		enc_param->sizeimage[1] =
+			inst->work_bufs[VENC_H264_VPU_WORK_BUF_SRC_CHROMA].size;
+		enc_param->sizeimage[2] = 0;
+	} else {
+		enc_param->sizeimage[0] =
+			inst->work_bufs[VENC_H264_VPU_WORK_BUF_SRC_LUMA].size;
+		enc_param->sizeimage[1] =
+			inst->work_bufs[VENC_H264_VPU_WORK_BUF_SRC_CHROMA_CB].size;
+		enc_param->sizeimage[2] =
+			inst->work_bufs[VENC_H264_VPU_WORK_BUF_SRC_CHROMA_CR].size;
+	}
+
+	/* the pps_buf is used by AP side only */
+	inst->pps_buf.size = 128;
+	if (mtk_vcodec_mem_alloc(inst->ctx,
+				 &inst->pps_buf)) {
+		mtk_vcodec_err(inst, "cannot allocate pps_buf");
+		ret = -ENOMEM;
+		goto err_alloc;
+	}
+	mtk_vcodec_debug_leave(inst);
+
+	return ret;
+
+err_alloc:
+	h264_enc_free_work_buf(inst);
+
+	return ret;
+}
+
+static unsigned int h264_enc_wait_venc_done(struct venc_h264_inst *inst)
+{
+	unsigned int irq_status = 0;
+	struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)inst->ctx;
+
+	mtk_vcodec_wait_for_done_ctx(ctx, MTK_INST_IRQ_RECEIVED,
+				     WAIT_INTR_TIMEOUT, true);
+	irq_status = ctx->irq_status;
+	mtk_vcodec_debug(inst, "irq_status %x <-", irq_status);
+
+	return irq_status;
+}
+
+static int h264_encode_sps(struct venc_h264_inst *inst,
+			   struct mtk_vcodec_mem *bs_buf,
+			   unsigned int *bs_size)
+{
+	int ret = 0;
+	unsigned int irq_status;
+
+	mtk_vcodec_debug_enter(inst);
+
+	ret = h264_enc_vpu_encode(inst, H264_BS_MODE_SPS, NULL,
+				  bs_buf, bs_size);
+	if (ret)
+		return ret;
+
+	irq_status = h264_enc_wait_venc_done(inst);
+	if (irq_status != H264_IRQ_STATUS_ENC_SPS_INT) {
+		mtk_vcodec_err(inst, "expect irq status %d",
+			       H264_IRQ_STATUS_ENC_SPS_INT);
+		return -EINVAL;
+	}
+
+	*bs_size = h264_read_reg(inst, VENC_PIC_BITSTREAM_BYTE_CNT);
+	mtk_vcodec_debug(inst, "bs size %d <-", *bs_size);
+
+	return ret;
+}
+
+static int h264_encode_pps(struct venc_h264_inst *inst,
+			   struct mtk_vcodec_mem *bs_buf,
+			   unsigned int *bs_size)
+{
+	int ret = 0;
+	unsigned int irq_status;
+
+	mtk_vcodec_debug_enter(inst);
+
+	ret = h264_enc_vpu_encode(inst, H264_BS_MODE_PPS, NULL,
+				  bs_buf, bs_size);
+	if (ret)
+		return ret;
+
+	irq_status = h264_enc_wait_venc_done(inst);
+	if (irq_status != H264_IRQ_STATUS_ENC_PPS_INT) {
+		mtk_vcodec_err(inst, "expect irq status %d",
+			       H264_IRQ_STATUS_ENC_PPS_INT);
+		return -EINVAL;
+	}
+
+	*bs_size = h264_read_reg(inst, VENC_PIC_BITSTREAM_BYTE_CNT);
+	mtk_vcodec_debug(inst, "bs size %d <-", *bs_size);
+
+	return ret;
+}
+
+static int h264_encode_frame(struct venc_h264_inst *inst,
+			     struct venc_frm_buf *frm_buf,
+			     struct mtk_vcodec_mem *bs_buf,
+			     unsigned int *bs_size)
+{
+	int ret = 0;
+	unsigned int irq_status;
+
+	mtk_vcodec_debug_enter(inst);
+
+	ret = h264_enc_vpu_encode(inst, H264_BS_MODE_FRAME, frm_buf,
+				  bs_buf, bs_size);
+	if (ret)
+		return ret;
+
+	/*
+	 * skip frame case: The skip frame buffer is composed by vpu side only,
+	 * it does not trigger the hw, so skip the wait interrupt operation.
+	 */
+	if (!inst->vpu_inst.wait_int) {
+		++inst->frm_cnt;
+		return ret;
+	}
+
+	irq_status = h264_enc_wait_venc_done(inst);
+	if (irq_status != H264_IRQ_STATUS_ENC_FRM_INT) {
+		mtk_vcodec_err(inst, "irq_status=%d failed", irq_status);
+		return -EINVAL;
+	}
+
+	*bs_size = h264_read_reg(inst,
+				 VENC_PIC_BITSTREAM_BYTE_CNT);
+	++inst->frm_cnt;
+	mtk_vcodec_debug(inst, "frm %d bs size %d key_frm %d <-",
+			 inst->frm_cnt,
+			 *bs_size, inst->is_key_frm);
+
+	return ret;
+}
+
+static void h264_encode_filler(struct venc_h264_inst *inst, void *buf,
+			       int size)
+{
+	unsigned char *p = buf;
+
+	*p++ = 0x0;
+	*p++ = 0x0;
+	*p++ = 0x0;
+	*p++ = 0x1;
+	*p++ = 0xc;
+	size -= 5;
+	while (size) {
+		*p++ = 0xff;
+		size -= 1;
+	}
+}
+
+static int h264_enc_init(struct mtk_vcodec_ctx *ctx, unsigned long *handle)
+{
+	int ret = 0;
+	struct venc_h264_inst *inst;
+
+	inst = kzalloc(sizeof(*inst), GFP_KERNEL);
+	if (!inst)
+		return -ENOMEM;
+
+	inst->ctx = ctx;
+	inst->dev = mtk_vcodec_get_plat_dev(ctx);
+	inst->hw_base = mtk_vcodec_get_reg_addr(inst->ctx, VENC_SYS);
+
+	mtk_vcodec_debug_enter(inst);
+	ret = h264_enc_vpu_init(inst);
+	if (ret)
+		kfree(inst);
+	else
+		(*handle) = (unsigned long)inst;
+
+	mtk_vcodec_debug_leave(inst);
+
+	return ret;
+}
+
+static int h264_enc_encode(unsigned long handle,
+			   enum venc_start_opt opt,
+			   struct venc_frm_buf *frm_buf,
+			   struct mtk_vcodec_mem *bs_buf,
+			   struct venc_done_result *result)
+{
+	int ret = 0;
+	struct venc_h264_inst *inst = (struct venc_h264_inst *)handle;
+	struct mtk_vcodec_ctx *ctx = inst->ctx;
+
+	mtk_vcodec_debug(inst, "opt %d ->", opt);
+
+	enable_irq(ctx->dev->enc_irq);
+
+	switch (opt) {
+	case VENC_START_OPT_ENCODE_SEQUENCE_HEADER: {
+		unsigned int bs_size_sps;
+		unsigned int bs_size_pps;
+
+		memset(bs_buf->va, 0x38, 20);
+		ret = h264_encode_sps(inst, bs_buf, &bs_size_sps);
+		if (ret)
+			goto encode_err;
+
+		memset(inst->pps_buf.va, 0x49, 20);
+		ret = h264_encode_pps(inst, &inst->pps_buf, &bs_size_pps);
+		if (ret)
+			goto encode_err;
+
+		memcpy(bs_buf->va + bs_size_sps,
+		       inst->pps_buf.va,
+		       bs_size_pps);
+		result->bs_size = bs_size_sps + bs_size_pps;
+		result->is_key_frm = false;
+	}
+	break;
+
+	case VENC_START_OPT_ENCODE_FRAME:
+		if (inst->prepend_hdr) {
+			int hdr_sz;
+			int hdr_sz_ext;
+			int bs_alignment = 128;
+			int filler_sz = 0;
+			struct mtk_vcodec_mem tmp_bs_buf;
+			unsigned int bs_size_sps;
+			unsigned int bs_size_pps;
+			unsigned int bs_size_frm;
+
+			mtk_vcodec_debug(inst,
+					 "h264_encode_frame prepend SPS/PPS");
+			ret = h264_encode_sps(inst, bs_buf, &bs_size_sps);
+			if (ret)
+				goto encode_err;
+
+			ret = h264_encode_pps(inst, &inst->pps_buf,
+					      &bs_size_pps);
+			if (ret)
+				goto encode_err;
+
+			memcpy(bs_buf->va + bs_size_sps,
+			       inst->pps_buf.va,
+			       bs_size_pps);
+
+			hdr_sz = bs_size_sps + bs_size_pps;
+			hdr_sz_ext = (hdr_sz & (bs_alignment - 1));
+			if (hdr_sz_ext) {
+				filler_sz = bs_alignment - hdr_sz_ext;
+				if (hdr_sz_ext + 5 > bs_alignment)
+					filler_sz += bs_alignment;
+				h264_encode_filler(
+					inst, bs_buf->va + hdr_sz,
+					filler_sz);
+			}
+
+			tmp_bs_buf.va = bs_buf->va + hdr_sz +
+				filler_sz;
+			tmp_bs_buf.dma_addr = bs_buf->dma_addr + hdr_sz +
+				filler_sz;
+			tmp_bs_buf.size = bs_buf->size -
+				(hdr_sz + filler_sz);
+
+			ret = h264_encode_frame(inst, frm_buf, &tmp_bs_buf,
+						&bs_size_frm);
+			if (ret)
+				goto encode_err;
+
+			result->bs_size = hdr_sz + filler_sz + bs_size_frm;
+			mtk_vcodec_debug(inst,
+					 "hdr %d filler %d frame %d bs %d",
+					 hdr_sz, filler_sz, bs_size_frm,
+					 result->bs_size);
+
+			inst->prepend_hdr = 0;
+		} else {
+			ret = h264_encode_frame(inst, frm_buf, bs_buf,
+						&result->bs_size);
+			if (ret)
+				goto encode_err;
+		}
+		result->is_key_frm = inst->is_key_frm;
+		break;
+
+	default:
+		mtk_vcodec_err(inst, "venc_start_opt %d not supported", opt);
+		ret = -EINVAL;
+		break;
+	}
+
+encode_err:
+	if (ret)
+		result->msg = VENC_MESSAGE_ERR;
+	else
+		result->msg = VENC_MESSAGE_OK;
+
+	disable_irq(ctx->dev->enc_irq);
+	mtk_vcodec_debug(inst, "opt %d <-", opt);
+
+	return ret;
+}
+
+static int h264_enc_set_param(unsigned long handle,
+			      enum venc_set_param_type type, void *in)
+{
+	int ret = 0;
+	struct venc_h264_inst *inst = (struct venc_h264_inst *)handle;
+	struct venc_enc_prm *enc_prm;
+
+	mtk_vcodec_debug(inst, "->type=%d", type);
+
+	switch (type) {
+	case VENC_SET_PARAM_ENC:
+		enc_prm = in;
+		ret = h264_enc_vpu_set_param(inst, type, enc_prm);
+		if (ret)
+			break;
+		if (inst->work_buf_allocated == 1) {
+			h264_enc_free_work_buf(inst);
+			inst->work_buf_allocated = 0;
+		}
+		if (inst->work_buf_allocated == 0) {
+			ret = h264_enc_alloc_work_buf(inst, enc_prm);
+			if (ret)
+				break;
+			inst->work_buf_allocated = 1;
+		}
+		break;
+
+	case VENC_SET_PARAM_FORCE_INTRA:
+		ret = h264_enc_vpu_set_param(inst, type, 0);
+		break;
+
+	case VENC_SET_PARAM_ADJUST_BITRATE:
+		enc_prm = in;
+		ret = h264_enc_vpu_set_param(inst, type, &enc_prm->bitrate);
+		break;
+
+	case VENC_SET_PARAM_ADJUST_FRAMERATE:
+		enc_prm = in;
+		ret = h264_enc_vpu_set_param(inst, type, &enc_prm->frm_rate);
+		break;
+
+	case VENC_SET_PARAM_I_FRAME_INTERVAL:
+		ret = h264_enc_vpu_set_param(inst, type, in);
+		break;
+
+	case VENC_SET_PARAM_SKIP_FRAME:
+		ret = h264_enc_vpu_set_param(inst, type, 0);
+		break;
+
+	case VENC_SET_PARAM_PREPEND_HEADER:
+		inst->prepend_hdr = 1;
+		mtk_vcodec_debug(inst, "set prepend header mode");
+		break;
+
+	default:
+		mtk_vcodec_err(inst, "type %d not supported", type);
+		ret = -EINVAL;
+		break;
+	}
+
+	mtk_vcodec_debug_leave(inst);
+
+	return ret;
+}
+
+static int h264_enc_deinit(unsigned long handle)
+{
+	int ret = 0;
+	struct venc_h264_inst *inst = (struct venc_h264_inst *)handle;
+
+	mtk_vcodec_debug_enter(inst);
+
+	ret = h264_enc_vpu_deinit(inst);
+
+	if (inst->work_buf_allocated)
+		h264_enc_free_work_buf(inst);
+
+	mtk_vcodec_debug_leave(inst);
+	kfree(inst);
+
+	return ret;
+}
+
+static struct venc_common_if venc_h264_if = {
+	h264_enc_init,
+	h264_enc_encode,
+	h264_enc_set_param,
+	h264_enc_deinit,
+};
+
+struct venc_common_if *get_h264_enc_comm_if(void)
+{
+	return &venc_h264_if;
+}
diff --git a/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.h b/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.h
new file mode 100644
index 0000000..9ac317a
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.h
@@ -0,0 +1,165 @@
+/*
+ * Copyright (c) 2015 MediaTek Inc.
+ * Author: Jungchang Tsao <jungchang.tsao@mediatek.com>
+ *         Daniel Hsiao <daniel.hsiao@mediatek.com>
+ *         PoChun Lin <pochun.lin@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef _VENC_H264_IF_H_
+#define _VENC_H264_IF_H_
+
+#include "venc_drv_base.h"
+
+/**
+ * enum venc_h264_vpu_work_buf - h264 encoder buffer index
+ */
+enum venc_h264_vpu_work_buf {
+	VENC_H264_VPU_WORK_BUF_RC_INFO,
+	VENC_H264_VPU_WORK_BUF_RC_CODE,
+	VENC_H264_VPU_WORK_BUF_REC_LUMA,
+	VENC_H264_VPU_WORK_BUF_REC_CHROMA,
+	VENC_H264_VPU_WORK_BUF_REF_LUMA,
+	VENC_H264_VPU_WORK_BUF_REF_CHROMA,
+	VENC_H264_VPU_WORK_BUF_MV_INFO_1,
+	VENC_H264_VPU_WORK_BUF_MV_INFO_2,
+	VENC_H264_VPU_WORK_BUF_SKIP_FRAME,
+	VENC_H264_VPU_WORK_BUF_SRC_LUMA,
+	VENC_H264_VPU_WORK_BUF_SRC_CHROMA,
+	VENC_H264_VPU_WORK_BUF_SRC_CHROMA_CB,
+	VENC_H264_VPU_WORK_BUF_SRC_CHROMA_CR,
+	VENC_H264_VPU_WORK_BUF_MAX,
+};
+
+/**
+ * enum venc_h264_bs_mode - for bs_mode argument in h264_enc_vpu_encode
+ */
+enum venc_h264_bs_mode {
+	H264_BS_MODE_SPS,
+	H264_BS_MODE_PPS,
+	H264_BS_MODE_FRAME,
+};
+
+/*
+ * struct venc_h264_vpu_config - Structure for h264 encoder configuration
+ * @input_fourcc: input fourcc
+ * @bitrate: target bitrate (in bps)
+ * @pic_w: picture width
+ * @pic_h: picture height
+ * @buf_w: buffer width
+ * @buf_h: buffer height
+ * @intra_period: intra frame period
+ * @framerate: frame rate
+ * @profile: as specified in standard
+ * @level: as specified in standard
+ * @wfd: WFD mode 1:on, 0:off
+ */
+struct venc_h264_vpu_config {
+	u32 input_fourcc;
+	u32 bitrate;
+	u32 pic_w;
+	u32 pic_h;
+	u32 buf_w;
+	u32 buf_h;
+	u32 intra_period;
+	u32 framerate;
+	u32 profile;
+	u32 level;
+	u32 wfd;
+};
+
+/*
+ * struct venc_h264_vpu_buf - Structure for buffer information
+ * @align: buffer alignment (in bytes)
+ * @iova: IO virtual address
+ * @vpua: VPU side memory addr which is used by RC_CODE
+ * @size: buffer size (in bytes)
+ */
+struct venc_h264_vpu_buf {
+	u32 align;
+	u32 iova;
+	u32 vpua;
+	u32 size;
+};
+
+/*
+ * struct venc_h264_vpu_drv - Structure for VPU driver control and info share
+ * This structure is allocated in VPU side and shared to AP side.
+ * @config: h264 encoder configuration
+ * @work_bufs: working buffer information in VPU side
+ * The work_bufs here is for storing the 'size' info shared to AP side.
+ * The similar item in struct venc_h264_inst is for memory allocation
+ * in AP side. The AP driver will copy the 'size' from here to the one in
+ * struct mtk_vcodec_mem, then invoke mtk_vcodec_mem_alloc to allocate
+ * the buffer. After that, bypass the 'dma_addr' to the 'iova' field here for
+ * register setting in VPU side.
+ */
+struct venc_h264_vpu_drv {
+	struct venc_h264_vpu_config config;
+	struct venc_h264_vpu_buf work_bufs[VENC_H264_VPU_WORK_BUF_MAX];
+};
+
+/*
+ * struct venc_h264_vpu_inst - h264 encoder VPU driver instance
+ * @wq_hd: wait queue used for vpu cmd trigger then wait vpu interrupt done
+ * @signaled: flag used for checking vpu interrupt done
+ * @failure: flag to show vpu cmd succeeds or not
+ * @state: enum venc_ipi_msg_enc_state
+ * @bs_size: bitstream size for skip frame case usage
+ * @wait_int: flag to wait interrupt done (0: for skip frame case, 1: normal
+ *	      case)
+ * @id: VPU instance id
+ * @drv: driver structure allocated by VPU side and shared to AP side for
+ *	 control and info share
+ */
+struct venc_h264_vpu_inst {
+	wait_queue_head_t wq_hd;
+	int signaled;
+	int failure;
+	int state;
+	int bs_size;
+	int wait_int;
+	unsigned int id;
+	struct venc_h264_vpu_drv *drv;
+};
+
+/*
+ * struct venc_h264_inst - h264 encoder AP driver instance
+ * @hw_base: h264 encoder hardware register base
+ * @work_bufs: working buffer
+ * @pps_buf: buffer to store the pps bitstream
+ * @work_buf_allocated: working buffer allocated flag
+ * @frm_cnt: encoded frame count
+ * @prepend_hdr: when the v4l2 layer send VENC_SET_PARAM_PREPEND_HEADER cmd
+ *  through h264_enc_set_param interface, it will set this flag and prepend the
+ *  sps/pps in h264_enc_encode function.
+ * @is_key_frm: key frame flag
+ * @vpu_inst: VPU instance to exchange information between AP and VPU
+ * @ctx: context for v4l2 layer integration
+ * @dev: device for v4l2 layer integration
+ */
+struct venc_h264_inst {
+	void __iomem *hw_base;
+	struct mtk_vcodec_mem work_bufs[VENC_H264_VPU_WORK_BUF_MAX];
+	struct mtk_vcodec_mem pps_buf;
+	bool work_buf_allocated;
+	unsigned int frm_cnt;
+	unsigned int prepend_hdr;
+	unsigned int is_key_frm;
+	struct venc_h264_vpu_inst vpu_inst;
+	void *ctx;
+	struct platform_device *dev;
+};
+
+struct venc_common_if *get_h264_enc_comm_if(void);
+
+#endif
diff --git a/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.c b/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.c
new file mode 100644
index 0000000..450be45
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.c
@@ -0,0 +1,309 @@
+/*
+ * Copyright (c) 2015 MediaTek Inc.
+ * Author: Jungchang Tsao <jungchang.tsao@mediatek.com>
+ *         Daniel Hsiao <daniel.hsiao@mediatek.com>
+ *         PoChun Lin <pochun.lin@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#include "mtk_vpu.h"
+
+#include "venc_h264_if.h"
+#include "venc_h264_vpu.h"
+#include "venc_ipi_msg.h"
+
+static unsigned int h264_get_profile(unsigned int profile)
+{
+	/* (Baseline=66, Main=77, High=100) */
+	switch (profile) {
+	case V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE:
+		return 66;
+	case V4L2_MPEG_VIDEO_H264_PROFILE_MAIN:
+		return 77;
+	case V4L2_MPEG_VIDEO_H264_PROFILE_HIGH:
+		return 100;
+	default:
+		return 100;
+	}
+}
+
+static unsigned int h264_get_level(unsigned int level)
+{
+	/* (UpTo4.1(HighProfile)) */
+	switch (level) {
+	case V4L2_MPEG_VIDEO_H264_LEVEL_1_0:
+		return 10;
+	case V4L2_MPEG_VIDEO_H264_LEVEL_1_1:
+		return 11;
+	case V4L2_MPEG_VIDEO_H264_LEVEL_1_2:
+		return 12;
+	case V4L2_MPEG_VIDEO_H264_LEVEL_1_3:
+		return 13;
+	case V4L2_MPEG_VIDEO_H264_LEVEL_2_0:
+		return 20;
+	case V4L2_MPEG_VIDEO_H264_LEVEL_2_1:
+		return 21;
+	case V4L2_MPEG_VIDEO_H264_LEVEL_2_2:
+		return 22;
+	case V4L2_MPEG_VIDEO_H264_LEVEL_3_0:
+		return 30;
+	case V4L2_MPEG_VIDEO_H264_LEVEL_3_1:
+		return 31;
+	case V4L2_MPEG_VIDEO_H264_LEVEL_3_2:
+		return 32;
+	case V4L2_MPEG_VIDEO_H264_LEVEL_4_0:
+		return 40;
+	case V4L2_MPEG_VIDEO_H264_LEVEL_4_1:
+		return 41;
+	default:
+		return 31;
+	}
+}
+
+static void handle_h264_enc_init_msg(struct venc_h264_inst *inst, void *data)
+{
+	struct venc_vpu_ipi_msg_init *msg = data;
+
+	inst->vpu_inst.id = msg->inst_id;
+	inst->vpu_inst.drv = (struct venc_h264_vpu_drv *)vpu_mapping_dm_addr(
+		inst->dev, msg->inst_id);
+}
+
+static void handle_h264_enc_encode_msg(struct venc_h264_inst *inst, void *data)
+{
+	struct venc_vpu_ipi_msg_enc *msg = data;
+
+	inst->vpu_inst.state = msg->state;
+	inst->vpu_inst.bs_size = msg->bs_size;
+	inst->is_key_frm = msg->key_frame;
+}
+
+static void h264_enc_vpu_ipi_handler(void *data, unsigned int len, void *priv)
+{
+	struct venc_vpu_ipi_msg_common *msg = data;
+	struct venc_h264_inst *inst = (struct venc_h264_inst *)msg->venc_inst;
+
+	mtk_vcodec_debug(inst, "msg_id %x inst %p status %d",
+			 msg->msg_id, inst, msg->status);
+
+	switch (msg->msg_id) {
+	case VPU_IPIMSG_H264_ENC_INIT_DONE:
+		handle_h264_enc_init_msg(inst, data);
+		break;
+	case VPU_IPIMSG_H264_ENC_SET_PARAM_DONE:
+		break;
+	case VPU_IPIMSG_H264_ENC_ENCODE_DONE:
+		handle_h264_enc_encode_msg(inst, data);
+		break;
+	case VPU_IPIMSG_H264_ENC_DEINIT_DONE:
+		break;
+	default:
+		mtk_vcodec_err(inst, "unknown msg id %x", msg->msg_id);
+		break;
+	}
+
+	inst->vpu_inst.signaled = 1;
+	inst->vpu_inst.failure = (msg->status != VENC_IPI_MSG_STATUS_OK);
+
+	mtk_vcodec_debug_leave(inst);
+}
+
+static int h264_enc_vpu_send_msg(struct venc_h264_inst *inst, void *msg,
+				 int len)
+{
+	int status;
+
+	mtk_vcodec_debug_enter(inst);
+	status = vpu_ipi_send(inst->dev, IPI_VENC_H264, msg, len);
+	if (status) {
+		mtk_vcodec_err(inst, "vpu_ipi_send msg %x len %d fail %d",
+			       *(unsigned int *)msg, len, status);
+		return -EINVAL;
+	}
+	mtk_vcodec_debug_leave(inst);
+
+	return 0;
+}
+
+int h264_enc_vpu_init(struct venc_h264_inst *inst)
+{
+	int status;
+	struct venc_ap_ipi_msg_init out;
+
+	mtk_vcodec_debug_enter(inst);
+
+	init_waitqueue_head(&inst->vpu_inst.wq_hd);
+	inst->vpu_inst.signaled = 0;
+	inst->vpu_inst.failure = 0;
+
+	status = vpu_ipi_register(inst->dev, IPI_VENC_H264,
+				  h264_enc_vpu_ipi_handler,
+				  "h264_enc", NULL);
+	if (status) {
+		mtk_vcodec_err(inst, "vpu_ipi_register fail %d", status);
+		return -EINVAL;
+	}
+
+	out.msg_id = AP_IPIMSG_H264_ENC_INIT;
+	out.venc_inst = (unsigned long)inst;
+	if (h264_enc_vpu_send_msg(inst, &out, sizeof(out)) ||
+	    inst->vpu_inst.failure) {
+		mtk_vcodec_err(inst, "AP_IPIMSG_H264_ENC_INIT failed");
+		return -EINVAL;
+	}
+
+	mtk_vcodec_debug_leave(inst);
+
+	return 0;
+}
+
+int h264_enc_vpu_set_param(struct venc_h264_inst *inst, unsigned int id,
+			   void *param)
+{
+	struct venc_ap_ipi_msg_set_param out;
+
+	mtk_vcodec_debug(inst, "id %d ->", id);
+
+	out.msg_id = AP_IPIMSG_H264_ENC_SET_PARAM;
+	out.inst_id = inst->vpu_inst.id;
+	out.param_id = id;
+	switch (id) {
+	case VENC_SET_PARAM_ENC: {
+		struct venc_enc_prm *enc_param = (struct venc_enc_prm *)param;
+
+		inst->vpu_inst.drv->config.input_fourcc =
+			enc_param->input_fourcc;
+		inst->vpu_inst.drv->config.bitrate = enc_param->bitrate;
+		inst->vpu_inst.drv->config.pic_w = enc_param->width;
+		inst->vpu_inst.drv->config.pic_h = enc_param->height;
+		inst->vpu_inst.drv->config.buf_w = enc_param->buf_width;
+		inst->vpu_inst.drv->config.buf_h = enc_param->buf_height;
+		inst->vpu_inst.drv->config.intra_period =
+			enc_param->intra_period;
+		inst->vpu_inst.drv->config.framerate = enc_param->frm_rate;
+		inst->vpu_inst.drv->config.profile =
+			h264_get_profile(enc_param->h264_profile);
+		inst->vpu_inst.drv->config.level =
+			h264_get_level(enc_param->h264_level);
+		inst->vpu_inst.drv->config.wfd = 0;
+		out.data_item = 0;
+		break;
+	}
+	case VENC_SET_PARAM_FORCE_INTRA:
+		out.data_item = 0;
+		break;
+	case VENC_SET_PARAM_ADJUST_BITRATE:
+		out.data_item = 1;
+		out.data[0] = *(unsigned int *)param;
+		break;
+	case VENC_SET_PARAM_ADJUST_FRAMERATE:
+		out.data_item = 1;
+		out.data[0] = *(unsigned int *)param;
+		break;
+	case VENC_SET_PARAM_I_FRAME_INTERVAL:
+		out.data_item = 1;
+		out.data[0] = *(unsigned int *)param;
+		break;
+	case VENC_SET_PARAM_SKIP_FRAME:
+		out.data_item = 0;
+		break;
+	}
+	if (h264_enc_vpu_send_msg(inst, &out, sizeof(out)) ||
+	    inst->vpu_inst.failure) {
+		mtk_vcodec_err(inst,
+			       "AP_IPIMSG_H264_ENC_SET_PARAM %d fail", id);
+		return -EINVAL;
+	}
+
+	mtk_vcodec_debug(inst, "id %d <-", id);
+
+	return 0;
+}
+
+int h264_enc_vpu_encode(struct venc_h264_inst *inst, unsigned int bs_mode,
+			struct venc_frm_buf *frm_buf,
+			struct mtk_vcodec_mem *bs_buf,
+			unsigned int *bs_size)
+{
+	struct venc_ap_ipi_msg_enc out;
+
+	mtk_vcodec_debug(inst, "bs_mode %d ->", bs_mode);
+
+	out.msg_id = AP_IPIMSG_H264_ENC_ENCODE;
+	out.inst_id = inst->vpu_inst.id;
+	out.bs_mode = bs_mode;
+	if (frm_buf) {
+		if ((frm_buf->fb_addr.dma_addr % 16 == 0) &&
+		    (frm_buf->fb_addr1.dma_addr % 16 == 0) &&
+		    (frm_buf->fb_addr2.dma_addr % 16 == 0)) {
+			out.input_addr[0] = frm_buf->fb_addr.dma_addr;
+			out.input_addr[1] = frm_buf->fb_addr1.dma_addr;
+			out.input_addr[2] = frm_buf->fb_addr2.dma_addr;
+		} else {
+			mtk_vcodec_err(inst, "dma_addr not align to 16");
+			return -EINVAL;
+		}
+	} else {
+		out.input_addr[0] = 0;
+		out.input_addr[1] = 0;
+		out.input_addr[2] = 0;
+	}
+	if (bs_buf) {
+		out.bs_addr = bs_buf->dma_addr;
+		out.bs_size = bs_buf->size;
+	} else {
+		out.bs_addr = 0;
+		out.bs_size = 0;
+	}
+	if (h264_enc_vpu_send_msg(inst, &out, sizeof(out)) ||
+	    inst->vpu_inst.failure) {
+		mtk_vcodec_err(inst, "AP_IPIMSG_H264_ENC_ENCODE %d fail",
+			       bs_mode);
+		return -EINVAL;
+	}
+
+	mtk_vcodec_debug(inst, "state %d size %d key_frm %d",
+			 inst->vpu_inst.state, inst->vpu_inst.bs_size,
+			 inst->is_key_frm);
+
+	inst->vpu_inst.wait_int = 1;
+	if (inst->vpu_inst.state == VEN_IPI_MSG_ENC_STATE_SKIP) {
+		*bs_size = inst->vpu_inst.bs_size;
+		memcpy(bs_buf->va,
+		       inst->work_bufs[VENC_H264_VPU_WORK_BUF_SKIP_FRAME].va,
+		       *bs_size);
+		inst->vpu_inst.wait_int = 0;
+	}
+
+	mtk_vcodec_debug(inst, "bs_mode %d <-", bs_mode);
+
+	return 0;
+}
+
+int h264_enc_vpu_deinit(struct venc_h264_inst *inst)
+{
+	struct venc_ap_ipi_msg_deinit out;
+
+	mtk_vcodec_debug_enter(inst);
+
+	out.msg_id = AP_IPIMSG_H264_ENC_DEINIT;
+	out.inst_id = inst->vpu_inst.id;
+	if (h264_enc_vpu_send_msg(inst, &out, sizeof(out)) ||
+	    inst->vpu_inst.failure) {
+		mtk_vcodec_err(inst, "AP_IPIMSG_H264_ENC_DEINIT fail");
+		return -EINVAL;
+	}
+
+	mtk_vcodec_debug_leave(inst);
+
+	return 0;
+}
diff --git a/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.h b/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.h
new file mode 100644
index 0000000..deccc6f
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.h
@@ -0,0 +1,30 @@
+/*
+ * Copyright (c) 2015 MediaTek Inc.
+ * Author: Jungchang Tsao <jungchang.tsao@mediatek.com>
+ *         Daniel Hsiao <daniel.hsiao@mediatek.com>
+ *         PoChun Lin <pochun.lin@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef _VENC_H264_VPU_H_
+#define _VENC_H264_VPU_H_
+
+int h264_enc_vpu_init(struct venc_h264_inst *inst);
+int h264_enc_vpu_set_param(struct venc_h264_inst *inst, unsigned int id,
+			   void *param);
+int h264_enc_vpu_encode(struct venc_h264_inst *inst, unsigned int bs_mode,
+			struct venc_frm_buf *frm_buf,
+			struct mtk_vcodec_mem *bs_buf,
+			unsigned int *bs_size);
+int h264_enc_vpu_deinit(struct venc_h264_inst *inst);
+
+#endif
diff --git a/drivers/media/platform/mtk-vcodec/venc_drv_if.c b/drivers/media/platform/mtk-vcodec/venc_drv_if.c
index d293f2c..28ef4a7 100644
--- a/drivers/media/platform/mtk-vcodec/venc_drv_if.c
+++ b/drivers/media/platform/mtk-vcodec/venc_drv_if.c
@@ -25,6 +25,7 @@
 
 #include "venc_drv_base.h"
 #include "vp8_enc/venc_vp8_if.h"
+#include "h264_enc/venc_h264_if.h"
 
 int venc_if_create(struct mtk_vcodec_ctx *ctx, unsigned int fourcc)
 {
@@ -38,6 +39,8 @@ int venc_if_create(struct mtk_vcodec_ctx *ctx, unsigned int fourcc)
                 ctx->enc_if = get_vp8_enc_comm_if();
                 break;
 	case V4L2_PIX_FMT_H264:
+	        ctx->enc_if = get_h264_enc_comm_if();
+	        break;
 	default:
 		return -EINVAL;
 	}
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 102+ messages in thread

* [PATCH v4 7/8] [media] vcodec: mediatek: Add Mediatek H264 Video Encoder Driver
@ 2016-02-04 11:35               ` Tiffany Lin
  0 siblings, 0 replies; 102+ messages in thread
From: Tiffany Lin @ 2016-02-04 11:35 UTC (permalink / raw)
  To: Hans Verkuil, daniel.thompson, Rob Herring,
	Mauro Carvalho Chehab, Matthias Brugger, Daniel Kurtz,
	Pawel Osciak
  Cc: Eddie Huang, Yingjoe Chen, devicetree, linux-kernel,
	linux-arm-kernel, linux-media, linux-mediatek, PoChun.Lin,
	Tiffany Lin, Daniel Hsiao

Add h264 encoder driver for MT8173

Signed-off-by: Daniel Hsiao <daniel.hsiao@mediatek.com>
Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
---
 drivers/media/platform/mtk-vcodec/Makefile         |    3 +-
 .../media/platform/mtk-vcodec/h264_enc/Makefile    |    6 +
 .../platform/mtk-vcodec/h264_enc/venc_h264_if.c    |  540 ++++++++++++++++++++
 .../platform/mtk-vcodec/h264_enc/venc_h264_if.h    |  165 ++++++
 .../platform/mtk-vcodec/h264_enc/venc_h264_vpu.c   |  309 +++++++++++
 .../platform/mtk-vcodec/h264_enc/venc_h264_vpu.h   |   30 ++
 drivers/media/platform/mtk-vcodec/venc_drv_if.c    |    3 +
 7 files changed, 1055 insertions(+), 1 deletion(-)
 create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/Makefile
 create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.c
 create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.h
 create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.c
 create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.h

diff --git a/drivers/media/platform/mtk-vcodec/Makefile b/drivers/media/platform/mtk-vcodec/Makefile
index f4ef502..f47dfc7 100644
--- a/drivers/media/platform/mtk-vcodec/Makefile
+++ b/drivers/media/platform/mtk-vcodec/Makefile
@@ -5,6 +5,7 @@ obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += mtk_vcodec_intr.o \
 				       mtk_vcodec_enc_pm.o \
 				       venc_drv_if.o
 
-obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += vp8_enc/
+
+obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += vp8_enc/ h264_enc/
 
 ccflags-y += -I$(srctree)/drivers/media/platform/mtk-vpu
diff --git a/drivers/media/platform/mtk-vcodec/h264_enc/Makefile b/drivers/media/platform/mtk-vcodec/h264_enc/Makefile
new file mode 100644
index 0000000..765b45f
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/h264_enc/Makefile
@@ -0,0 +1,6 @@
+obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += venc_h264_if.o venc_h264_vpu.o
+
+ccflags-y += \
+    -I$(srctree)/drivers/media/platform/mtk-vcodec/ \
+    -I$(srctree)/drivers/media/platform/mtk-vcodec/h264_enc \
+    -I$(srctree)/drivers/media/platform/mtk-vpu
diff --git a/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.c b/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.c
new file mode 100644
index 0000000..c35fb26
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.c
@@ -0,0 +1,540 @@
+/*
+ * Copyright (c) 2015 MediaTek Inc.
+ * Author: Jungchang Tsao <jungchang.tsao@mediatek.com>
+ *         Daniel Hsiao <daniel.hsiao@mediatek.com>
+ *         PoChun Lin <pochun.lin@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/interrupt.h>
+#include <linux/kernel.h>
+#include <linux/slab.h>
+
+#include "mtk_vcodec_drv.h"
+#include "mtk_vcodec_util.h"
+#include "mtk_vcodec_intr.h"
+#include "mtk_vcodec_enc.h"
+#include "mtk_vcodec_pm.h"
+#include "mtk_vpu.h"
+
+#include "venc_h264_if.h"
+#include "venc_h264_vpu.h"
+
+#define VENC_PIC_BITSTREAM_BYTE_CNT 0x0098
+
+enum venc_h264_irq_status {
+	H264_IRQ_STATUS_ENC_SPS_INT = (1 << 0),
+	H264_IRQ_STATUS_ENC_PPS_INT = (1 << 1),
+	H264_IRQ_STATUS_ENC_FRM_INT = (1 << 2),
+};
+
+static inline void h264_write_reg(struct venc_h264_inst *inst, u32 addr,
+				  u32 val)
+{
+	writel(val, inst->hw_base + addr);
+}
+
+static inline u32 h264_read_reg(struct venc_h264_inst *inst, u32 addr)
+{
+	return readl(inst->hw_base + addr);
+}
+
+static void h264_enc_free_work_buf(struct venc_h264_inst *inst)
+{
+	int i;
+
+	mtk_vcodec_debug_enter(inst);
+
+	/* Except the SKIP_FRAME buffers,
+	 * other buffers need to be freed by AP.
+	 */
+	for (i = 0; i < VENC_H264_VPU_WORK_BUF_MAX; i++) {
+		if (i != VENC_H264_VPU_WORK_BUF_SKIP_FRAME)
+			if (inst->work_bufs[i].va != NULL)
+				mtk_vcodec_mem_free(inst->ctx,
+						    &inst->work_bufs[i]);
+	}
+
+	if (inst->pps_buf.va != NULL)
+		mtk_vcodec_mem_free(inst->ctx, &inst->pps_buf);
+
+	mtk_vcodec_debug_leave(inst);
+}
+
+static int h264_enc_alloc_work_buf(struct venc_h264_inst *inst, void *param)
+{
+	int i;
+	int ret = 0;
+	struct venc_h264_vpu_buf *wb = inst->vpu_inst.drv->work_bufs;
+	struct venc_enc_prm *enc_param = param;
+
+	mtk_vcodec_debug_enter(inst);
+
+	for (i = 0; i < VENC_H264_VPU_WORK_BUF_MAX; i++) {
+		/*
+		 * This 'wb' structure is set by VPU side and shared to AP for
+		 * buffer allocation and IO virtual addr mapping. For most of
+		 * the buffers, AP will allocate the buffer according to 'size'
+		 * field and store the IO virtual addr in 'iova' field. There
+		 * are two exceptions:
+		 * (1) RC_CODE buffer, it's pre-allocated in the VPU side, and
+		 * save the VPU addr in the 'vpua' field. The AP will translate
+		 * the VPU addr to the corresponding IO virtual addr and store
+		 * in 'iova' field for reg setting in VPU side.
+		 * (2) SKIP_FRAME buffer, it's pre-allocated in the VPU side,
+		 * and save the VPU addr in the 'vpua' field. The AP will
+		 * translate the VPU addr to the corresponding AP side virtual
+		 * address and do some memcpy access to move to bitstream buffer
+		 * assigned by v4l2 layer.
+		 */
+		if (i == VENC_H264_VPU_WORK_BUF_RC_CODE) {
+			void *tmp_va;
+
+			tmp_va = vpu_mapping_dm_addr(inst->dev, wb[i].vpua);
+			inst->work_bufs[i].size = wb[i].size;
+			if (mtk_vcodec_mem_alloc(inst->ctx,
+						 &inst->work_bufs[i])) {
+				mtk_vcodec_err(inst,
+					       "cannot allocate buf %d", i);
+				ret = -ENOMEM;
+				goto err_alloc;
+			}
+			memcpy(inst->work_bufs[i].va, tmp_va, wb[i].size);
+			wb[i].iova = inst->work_bufs[i].dma_addr;
+		} else if (i == VENC_H264_VPU_WORK_BUF_SKIP_FRAME) {
+			inst->work_bufs[i].size = wb[i].size;
+			inst->work_bufs[i].va = vpu_mapping_dm_addr(
+				inst->dev, wb[i].vpua);
+			inst->work_bufs[i].dma_addr = 0;
+			wb[i].iova = inst->work_bufs[i].dma_addr;
+		} else if (i == VENC_H264_VPU_WORK_BUF_SRC_LUMA ||
+			   i == VENC_H264_VPU_WORK_BUF_SRC_CHROMA ||
+			   i == VENC_H264_VPU_WORK_BUF_SRC_CHROMA_CB ||
+			   i == VENC_H264_VPU_WORK_BUF_SRC_CHROMA_CR) {
+			inst->work_bufs[i].size = wb[i].size;
+			inst->work_bufs[i].dma_addr = 0;
+			inst->work_bufs[i].va = NULL;
+			wb[i].iova = inst->work_bufs[i].dma_addr;
+		} else {
+			inst->work_bufs[i].size = wb[i].size;
+			if (mtk_vcodec_mem_alloc(inst->ctx,
+						 &inst->work_bufs[i])) {
+				mtk_vcodec_err(inst,
+					       "cannot allocate buf %d", i);
+				ret = -ENOMEM;
+				goto err_alloc;
+			}
+			wb[i].iova = inst->work_bufs[i].dma_addr;
+		}
+		mtk_vcodec_debug(inst, "buf[%d] va=0x%p iova=0x%p size=0x%lx",
+				 i, inst->work_bufs[i].va,
+				 (void *)inst->work_bufs[i].dma_addr,
+				 inst->work_bufs[i].size);
+	}
+
+	if (enc_param->input_fourcc == VENC_YUV_FORMAT_NV12 ||
+	    enc_param->input_fourcc == VENC_YUV_FORMAT_NV21) {
+		enc_param->sizeimage[0] =
+			inst->work_bufs[VENC_H264_VPU_WORK_BUF_SRC_LUMA].size;
+		enc_param->sizeimage[1] =
+			inst->work_bufs[VENC_H264_VPU_WORK_BUF_SRC_CHROMA].size;
+		enc_param->sizeimage[2] = 0;
+	} else {
+		enc_param->sizeimage[0] =
+			inst->work_bufs[VENC_H264_VPU_WORK_BUF_SRC_LUMA].size;
+		enc_param->sizeimage[1] =
+			inst->work_bufs[VENC_H264_VPU_WORK_BUF_SRC_CHROMA_CB].size;
+		enc_param->sizeimage[2] =
+			inst->work_bufs[VENC_H264_VPU_WORK_BUF_SRC_CHROMA_CR].size;
+	}
+
+	/* the pps_buf is used by AP side only */
+	inst->pps_buf.size = 128;
+	if (mtk_vcodec_mem_alloc(inst->ctx,
+				 &inst->pps_buf)) {
+		mtk_vcodec_err(inst, "cannot allocate pps_buf");
+		ret = -ENOMEM;
+		goto err_alloc;
+	}
+	mtk_vcodec_debug_leave(inst);
+
+	return ret;
+
+err_alloc:
+	h264_enc_free_work_buf(inst);
+
+	return ret;
+}
+
+static unsigned int h264_enc_wait_venc_done(struct venc_h264_inst *inst)
+{
+	unsigned int irq_status = 0;
+	struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)inst->ctx;
+
+	mtk_vcodec_wait_for_done_ctx(ctx, MTK_INST_IRQ_RECEIVED,
+				     WAIT_INTR_TIMEOUT, true);
+	irq_status = ctx->irq_status;
+	mtk_vcodec_debug(inst, "irq_status %x <-", irq_status);
+
+	return irq_status;
+}
+
+static int h264_encode_sps(struct venc_h264_inst *inst,
+			   struct mtk_vcodec_mem *bs_buf,
+			   unsigned int *bs_size)
+{
+	int ret = 0;
+	unsigned int irq_status;
+
+	mtk_vcodec_debug_enter(inst);
+
+	ret = h264_enc_vpu_encode(inst, H264_BS_MODE_SPS, NULL,
+				  bs_buf, bs_size);
+	if (ret)
+		return ret;
+
+	irq_status = h264_enc_wait_venc_done(inst);
+	if (irq_status != H264_IRQ_STATUS_ENC_SPS_INT) {
+		mtk_vcodec_err(inst, "expect irq status %d",
+			       H264_IRQ_STATUS_ENC_SPS_INT);
+		return -EINVAL;
+	}
+
+	*bs_size = h264_read_reg(inst, VENC_PIC_BITSTREAM_BYTE_CNT);
+	mtk_vcodec_debug(inst, "bs size %d <-", *bs_size);
+
+	return ret;
+}
+
+static int h264_encode_pps(struct venc_h264_inst *inst,
+			   struct mtk_vcodec_mem *bs_buf,
+			   unsigned int *bs_size)
+{
+	int ret = 0;
+	unsigned int irq_status;
+
+	mtk_vcodec_debug_enter(inst);
+
+	ret = h264_enc_vpu_encode(inst, H264_BS_MODE_PPS, NULL,
+				  bs_buf, bs_size);
+	if (ret)
+		return ret;
+
+	irq_status = h264_enc_wait_venc_done(inst);
+	if (irq_status != H264_IRQ_STATUS_ENC_PPS_INT) {
+		mtk_vcodec_err(inst, "expect irq status %d",
+			       H264_IRQ_STATUS_ENC_PPS_INT);
+		return -EINVAL;
+	}
+
+	*bs_size = h264_read_reg(inst, VENC_PIC_BITSTREAM_BYTE_CNT);
+	mtk_vcodec_debug(inst, "bs size %d <-", *bs_size);
+
+	return ret;
+}
+
+static int h264_encode_frame(struct venc_h264_inst *inst,
+			     struct venc_frm_buf *frm_buf,
+			     struct mtk_vcodec_mem *bs_buf,
+			     unsigned int *bs_size)
+{
+	int ret = 0;
+	unsigned int irq_status;
+
+	mtk_vcodec_debug_enter(inst);
+
+	ret = h264_enc_vpu_encode(inst, H264_BS_MODE_FRAME, frm_buf,
+				  bs_buf, bs_size);
+	if (ret)
+		return ret;
+
+	/*
+	 * skip frame case: The skip frame buffer is composed by vpu side only,
+	 * it does not trigger the hw, so skip the wait interrupt operation.
+	 */
+	if (!inst->vpu_inst.wait_int) {
+		++inst->frm_cnt;
+		return ret;
+	}
+
+	irq_status = h264_enc_wait_venc_done(inst);
+	if (irq_status != H264_IRQ_STATUS_ENC_FRM_INT) {
+		mtk_vcodec_err(inst, "irq_status=%d failed", irq_status);
+		return -EINVAL;
+	}
+
+	*bs_size = h264_read_reg(inst,
+				 VENC_PIC_BITSTREAM_BYTE_CNT);
+	++inst->frm_cnt;
+	mtk_vcodec_debug(inst, "frm %d bs size %d key_frm %d <-",
+			 inst->frm_cnt,
+			 *bs_size, inst->is_key_frm);
+
+	return ret;
+}
+
+static void h264_encode_filler(struct venc_h264_inst *inst, void *buf,
+			       int size)
+{
+	unsigned char *p = buf;
+
+	*p++ = 0x0;
+	*p++ = 0x0;
+	*p++ = 0x0;
+	*p++ = 0x1;
+	*p++ = 0xc;
+	size -= 5;
+	while (size) {
+		*p++ = 0xff;
+		size -= 1;
+	}
+}
+
+static int h264_enc_init(struct mtk_vcodec_ctx *ctx, unsigned long *handle)
+{
+	int ret = 0;
+	struct venc_h264_inst *inst;
+
+	inst = kzalloc(sizeof(*inst), GFP_KERNEL);
+	if (!inst)
+		return -ENOMEM;
+
+	inst->ctx = ctx;
+	inst->dev = mtk_vcodec_get_plat_dev(ctx);
+	inst->hw_base = mtk_vcodec_get_reg_addr(inst->ctx, VENC_SYS);
+
+	mtk_vcodec_debug_enter(inst);
+	ret = h264_enc_vpu_init(inst);
+	if (ret)
+		kfree(inst);
+	else
+		(*handle) = (unsigned long)inst;
+
+	mtk_vcodec_debug_leave(inst);
+
+	return ret;
+}
+
+static int h264_enc_encode(unsigned long handle,
+			   enum venc_start_opt opt,
+			   struct venc_frm_buf *frm_buf,
+			   struct mtk_vcodec_mem *bs_buf,
+			   struct venc_done_result *result)
+{
+	int ret = 0;
+	struct venc_h264_inst *inst = (struct venc_h264_inst *)handle;
+	struct mtk_vcodec_ctx *ctx = inst->ctx;
+
+	mtk_vcodec_debug(inst, "opt %d ->", opt);
+
+	enable_irq(ctx->dev->enc_irq);
+
+	switch (opt) {
+	case VENC_START_OPT_ENCODE_SEQUENCE_HEADER: {
+		unsigned int bs_size_sps;
+		unsigned int bs_size_pps;
+
+		memset(bs_buf->va, 0x38, 20);
+		ret = h264_encode_sps(inst, bs_buf, &bs_size_sps);
+		if (ret)
+			goto encode_err;
+
+		memset(inst->pps_buf.va, 0x49, 20);
+		ret = h264_encode_pps(inst, &inst->pps_buf, &bs_size_pps);
+		if (ret)
+			goto encode_err;
+
+		memcpy(bs_buf->va + bs_size_sps,
+		       inst->pps_buf.va,
+		       bs_size_pps);
+		result->bs_size = bs_size_sps + bs_size_pps;
+		result->is_key_frm = false;
+	}
+	break;
+
+	case VENC_START_OPT_ENCODE_FRAME:
+		if (inst->prepend_hdr) {
+			int hdr_sz;
+			int hdr_sz_ext;
+			int bs_alignment = 128;
+			int filler_sz = 0;
+			struct mtk_vcodec_mem tmp_bs_buf;
+			unsigned int bs_size_sps;
+			unsigned int bs_size_pps;
+			unsigned int bs_size_frm;
+
+			mtk_vcodec_debug(inst,
+					 "h264_encode_frame prepend SPS/PPS");
+			ret = h264_encode_sps(inst, bs_buf, &bs_size_sps);
+			if (ret)
+				goto encode_err;
+
+			ret = h264_encode_pps(inst, &inst->pps_buf,
+					      &bs_size_pps);
+			if (ret)
+				goto encode_err;
+
+			memcpy(bs_buf->va + bs_size_sps,
+			       inst->pps_buf.va,
+			       bs_size_pps);
+
+			hdr_sz = bs_size_sps + bs_size_pps;
+			hdr_sz_ext = (hdr_sz & (bs_alignment - 1));
+			if (hdr_sz_ext) {
+				filler_sz = bs_alignment - hdr_sz_ext;
+				if (hdr_sz_ext + 5 > bs_alignment)
+					filler_sz += bs_alignment;
+				h264_encode_filler(
+					inst, bs_buf->va + hdr_sz,
+					filler_sz);
+			}
+
+			tmp_bs_buf.va = bs_buf->va + hdr_sz +
+				filler_sz;
+			tmp_bs_buf.dma_addr = bs_buf->dma_addr + hdr_sz +
+				filler_sz;
+			tmp_bs_buf.size = bs_buf->size -
+				(hdr_sz + filler_sz);
+
+			ret = h264_encode_frame(inst, frm_buf, &tmp_bs_buf,
+						&bs_size_frm);
+			if (ret)
+				goto encode_err;
+
+			result->bs_size = hdr_sz + filler_sz + bs_size_frm;
+			mtk_vcodec_debug(inst,
+					 "hdr %d filler %d frame %d bs %d",
+					 hdr_sz, filler_sz, bs_size_frm,
+					 result->bs_size);
+
+			inst->prepend_hdr = 0;
+		} else {
+			ret = h264_encode_frame(inst, frm_buf, bs_buf,
+						&result->bs_size);
+			if (ret)
+				goto encode_err;
+		}
+		result->is_key_frm = inst->is_key_frm;
+		break;
+
+	default:
+		mtk_vcodec_err(inst, "venc_start_opt %d not supported", opt);
+		ret = -EINVAL;
+		break;
+	}
+
+encode_err:
+	if (ret)
+		result->msg = VENC_MESSAGE_ERR;
+	else
+		result->msg = VENC_MESSAGE_OK;
+
+	disable_irq(ctx->dev->enc_irq);
+	mtk_vcodec_debug(inst, "opt %d <-", opt);
+
+	return ret;
+}
+
+static int h264_enc_set_param(unsigned long handle,
+			      enum venc_set_param_type type, void *in)
+{
+	int ret = 0;
+	struct venc_h264_inst *inst = (struct venc_h264_inst *)handle;
+	struct venc_enc_prm *enc_prm;
+
+	mtk_vcodec_debug(inst, "->type=%d", type);
+
+	switch (type) {
+	case VENC_SET_PARAM_ENC:
+		enc_prm = in;
+		ret = h264_enc_vpu_set_param(inst, type, enc_prm);
+		if (ret)
+			break;
+		if (inst->work_buf_allocated == 1) {
+			h264_enc_free_work_buf(inst);
+			inst->work_buf_allocated = 0;
+		}
+		if (inst->work_buf_allocated == 0) {
+			ret = h264_enc_alloc_work_buf(inst, enc_prm);
+			if (ret)
+				break;
+			inst->work_buf_allocated = 1;
+		}
+		break;
+
+	case VENC_SET_PARAM_FORCE_INTRA:
+		ret = h264_enc_vpu_set_param(inst, type, 0);
+		break;
+
+	case VENC_SET_PARAM_ADJUST_BITRATE:
+		enc_prm = in;
+		ret = h264_enc_vpu_set_param(inst, type, &enc_prm->bitrate);
+		break;
+
+	case VENC_SET_PARAM_ADJUST_FRAMERATE:
+		enc_prm = in;
+		ret = h264_enc_vpu_set_param(inst, type, &enc_prm->frm_rate);
+		break;
+
+	case VENC_SET_PARAM_I_FRAME_INTERVAL:
+		ret = h264_enc_vpu_set_param(inst, type, in);
+		break;
+
+	case VENC_SET_PARAM_SKIP_FRAME:
+		ret = h264_enc_vpu_set_param(inst, type, 0);
+		break;
+
+	case VENC_SET_PARAM_PREPEND_HEADER:
+		inst->prepend_hdr = 1;
+		mtk_vcodec_debug(inst, "set prepend header mode");
+		break;
+
+	default:
+		mtk_vcodec_err(inst, "type %d not supported", type);
+		ret = -EINVAL;
+		break;
+	}
+
+	mtk_vcodec_debug_leave(inst);
+
+	return ret;
+}
+
+static int h264_enc_deinit(unsigned long handle)
+{
+	int ret = 0;
+	struct venc_h264_inst *inst = (struct venc_h264_inst *)handle;
+
+	mtk_vcodec_debug_enter(inst);
+
+	ret = h264_enc_vpu_deinit(inst);
+
+	if (inst->work_buf_allocated)
+		h264_enc_free_work_buf(inst);
+
+	mtk_vcodec_debug_leave(inst);
+	kfree(inst);
+
+	return ret;
+}
+
+static struct venc_common_if venc_h264_if = {
+	h264_enc_init,
+	h264_enc_encode,
+	h264_enc_set_param,
+	h264_enc_deinit,
+};
+
+struct venc_common_if *get_h264_enc_comm_if(void)
+{
+	return &venc_h264_if;
+}
diff --git a/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.h b/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.h
new file mode 100644
index 0000000..9ac317a
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.h
@@ -0,0 +1,165 @@
+/*
+ * Copyright (c) 2015 MediaTek Inc.
+ * Author: Jungchang Tsao <jungchang.tsao@mediatek.com>
+ *         Daniel Hsiao <daniel.hsiao@mediatek.com>
+ *         PoChun Lin <pochun.lin@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef _VENC_H264_IF_H_
+#define _VENC_H264_IF_H_
+
+#include "venc_drv_base.h"
+
+/**
+ * enum venc_h264_vpu_work_buf - h264 encoder buffer index
+ */
+enum venc_h264_vpu_work_buf {
+	VENC_H264_VPU_WORK_BUF_RC_INFO,
+	VENC_H264_VPU_WORK_BUF_RC_CODE,
+	VENC_H264_VPU_WORK_BUF_REC_LUMA,
+	VENC_H264_VPU_WORK_BUF_REC_CHROMA,
+	VENC_H264_VPU_WORK_BUF_REF_LUMA,
+	VENC_H264_VPU_WORK_BUF_REF_CHROMA,
+	VENC_H264_VPU_WORK_BUF_MV_INFO_1,
+	VENC_H264_VPU_WORK_BUF_MV_INFO_2,
+	VENC_H264_VPU_WORK_BUF_SKIP_FRAME,
+	VENC_H264_VPU_WORK_BUF_SRC_LUMA,
+	VENC_H264_VPU_WORK_BUF_SRC_CHROMA,
+	VENC_H264_VPU_WORK_BUF_SRC_CHROMA_CB,
+	VENC_H264_VPU_WORK_BUF_SRC_CHROMA_CR,
+	VENC_H264_VPU_WORK_BUF_MAX,
+};
+
+/**
+ * enum venc_h264_bs_mode - for bs_mode argument in h264_enc_vpu_encode
+ */
+enum venc_h264_bs_mode {
+	H264_BS_MODE_SPS,
+	H264_BS_MODE_PPS,
+	H264_BS_MODE_FRAME,
+};
+
+/*
+ * struct venc_h264_vpu_config - Structure for h264 encoder configuration
+ * @input_fourcc: input fourcc
+ * @bitrate: target bitrate (in bps)
+ * @pic_w: picture width
+ * @pic_h: picture height
+ * @buf_w: buffer width
+ * @buf_h: buffer height
+ * @intra_period: intra frame period
+ * @framerate: frame rate
+ * @profile: as specified in standard
+ * @level: as specified in standard
+ * @wfd: WFD mode 1:on, 0:off
+ */
+struct venc_h264_vpu_config {
+	u32 input_fourcc;
+	u32 bitrate;
+	u32 pic_w;
+	u32 pic_h;
+	u32 buf_w;
+	u32 buf_h;
+	u32 intra_period;
+	u32 framerate;
+	u32 profile;
+	u32 level;
+	u32 wfd;
+};
+
+/*
+ * struct venc_h264_vpu_buf - Structure for buffer information
+ * @align: buffer alignment (in bytes)
+ * @iova: IO virtual address
+ * @vpua: VPU side memory addr which is used by RC_CODE
+ * @size: buffer size (in bytes)
+ */
+struct venc_h264_vpu_buf {
+	u32 align;
+	u32 iova;
+	u32 vpua;
+	u32 size;
+};
+
+/*
+ * struct venc_h264_vpu_drv - Structure for VPU driver control and info share
+ * This structure is allocated in VPU side and shared to AP side.
+ * @config: h264 encoder configuration
+ * @work_bufs: working buffer information in VPU side
+ * The work_bufs here is for storing the 'size' info shared to AP side.
+ * The similar item in struct venc_h264_inst is for memory allocation
+ * in AP side. The AP driver will copy the 'size' from here to the one in
+ * struct mtk_vcodec_mem, then invoke mtk_vcodec_mem_alloc to allocate
+ * the buffer. After that, bypass the 'dma_addr' to the 'iova' field here for
+ * register setting in VPU side.
+ */
+struct venc_h264_vpu_drv {
+	struct venc_h264_vpu_config config;
+	struct venc_h264_vpu_buf work_bufs[VENC_H264_VPU_WORK_BUF_MAX];
+};
+
+/*
+ * struct venc_h264_vpu_inst - h264 encoder VPU driver instance
+ * @wq_hd: wait queue used for vpu cmd trigger then wait vpu interrupt done
+ * @signaled: flag used for checking vpu interrupt done
+ * @failure: flag to show vpu cmd succeeds or not
+ * @state: enum venc_ipi_msg_enc_state
+ * @bs_size: bitstream size for skip frame case usage
+ * @wait_int: flag to wait interrupt done (0: for skip frame case, 1: normal
+ *	      case)
+ * @id: VPU instance id
+ * @drv: driver structure allocated by VPU side and shared to AP side for
+ *	 control and info share
+ */
+struct venc_h264_vpu_inst {
+	wait_queue_head_t wq_hd;
+	int signaled;
+	int failure;
+	int state;
+	int bs_size;
+	int wait_int;
+	unsigned int id;
+	struct venc_h264_vpu_drv *drv;
+};
+
+/*
+ * struct venc_h264_inst - h264 encoder AP driver instance
+ * @hw_base: h264 encoder hardware register base
+ * @work_bufs: working buffer
+ * @pps_buf: buffer to store the pps bitstream
+ * @work_buf_allocated: working buffer allocated flag
+ * @frm_cnt: encoded frame count
+ * @prepend_hdr: when the v4l2 layer send VENC_SET_PARAM_PREPEND_HEADER cmd
+ *  through h264_enc_set_param interface, it will set this flag and prepend the
+ *  sps/pps in h264_enc_encode function.
+ * @is_key_frm: key frame flag
+ * @vpu_inst: VPU instance to exchange information between AP and VPU
+ * @ctx: context for v4l2 layer integration
+ * @dev: device for v4l2 layer integration
+ */
+struct venc_h264_inst {
+	void __iomem *hw_base;
+	struct mtk_vcodec_mem work_bufs[VENC_H264_VPU_WORK_BUF_MAX];
+	struct mtk_vcodec_mem pps_buf;
+	bool work_buf_allocated;
+	unsigned int frm_cnt;
+	unsigned int prepend_hdr;
+	unsigned int is_key_frm;
+	struct venc_h264_vpu_inst vpu_inst;
+	void *ctx;
+	struct platform_device *dev;
+};
+
+struct venc_common_if *get_h264_enc_comm_if(void);
+
+#endif
diff --git a/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.c b/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.c
new file mode 100644
index 0000000..450be45
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.c
@@ -0,0 +1,309 @@
+/*
+ * Copyright (c) 2015 MediaTek Inc.
+ * Author: Jungchang Tsao <jungchang.tsao@mediatek.com>
+ *         Daniel Hsiao <daniel.hsiao@mediatek.com>
+ *         PoChun Lin <pochun.lin@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#include "mtk_vpu.h"
+
+#include "venc_h264_if.h"
+#include "venc_h264_vpu.h"
+#include "venc_ipi_msg.h"
+
+static unsigned int h264_get_profile(unsigned int profile)
+{
+	/* (Baseline=66, Main=77, High=100) */
+	switch (profile) {
+	case V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE:
+		return 66;
+	case V4L2_MPEG_VIDEO_H264_PROFILE_MAIN:
+		return 77;
+	case V4L2_MPEG_VIDEO_H264_PROFILE_HIGH:
+		return 100;
+	default:
+		return 100;
+	}
+}
+
+static unsigned int h264_get_level(unsigned int level)
+{
+	/* (UpTo4.1(HighProfile)) */
+	switch (level) {
+	case V4L2_MPEG_VIDEO_H264_LEVEL_1_0:
+		return 10;
+	case V4L2_MPEG_VIDEO_H264_LEVEL_1_1:
+		return 11;
+	case V4L2_MPEG_VIDEO_H264_LEVEL_1_2:
+		return 12;
+	case V4L2_MPEG_VIDEO_H264_LEVEL_1_3:
+		return 13;
+	case V4L2_MPEG_VIDEO_H264_LEVEL_2_0:
+		return 20;
+	case V4L2_MPEG_VIDEO_H264_LEVEL_2_1:
+		return 21;
+	case V4L2_MPEG_VIDEO_H264_LEVEL_2_2:
+		return 22;
+	case V4L2_MPEG_VIDEO_H264_LEVEL_3_0:
+		return 30;
+	case V4L2_MPEG_VIDEO_H264_LEVEL_3_1:
+		return 31;
+	case V4L2_MPEG_VIDEO_H264_LEVEL_3_2:
+		return 32;
+	case V4L2_MPEG_VIDEO_H264_LEVEL_4_0:
+		return 40;
+	case V4L2_MPEG_VIDEO_H264_LEVEL_4_1:
+		return 41;
+	default:
+		return 31;
+	}
+}
+
+static void handle_h264_enc_init_msg(struct venc_h264_inst *inst, void *data)
+{
+	struct venc_vpu_ipi_msg_init *msg = data;
+
+	inst->vpu_inst.id = msg->inst_id;
+	inst->vpu_inst.drv = (struct venc_h264_vpu_drv *)vpu_mapping_dm_addr(
+		inst->dev, msg->inst_id);
+}
+
+static void handle_h264_enc_encode_msg(struct venc_h264_inst *inst, void *data)
+{
+	struct venc_vpu_ipi_msg_enc *msg = data;
+
+	inst->vpu_inst.state = msg->state;
+	inst->vpu_inst.bs_size = msg->bs_size;
+	inst->is_key_frm = msg->key_frame;
+}
+
+static void h264_enc_vpu_ipi_handler(void *data, unsigned int len, void *priv)
+{
+	struct venc_vpu_ipi_msg_common *msg = data;
+	struct venc_h264_inst *inst = (struct venc_h264_inst *)msg->venc_inst;
+
+	mtk_vcodec_debug(inst, "msg_id %x inst %p status %d",
+			 msg->msg_id, inst, msg->status);
+
+	switch (msg->msg_id) {
+	case VPU_IPIMSG_H264_ENC_INIT_DONE:
+		handle_h264_enc_init_msg(inst, data);
+		break;
+	case VPU_IPIMSG_H264_ENC_SET_PARAM_DONE:
+		break;
+	case VPU_IPIMSG_H264_ENC_ENCODE_DONE:
+		handle_h264_enc_encode_msg(inst, data);
+		break;
+	case VPU_IPIMSG_H264_ENC_DEINIT_DONE:
+		break;
+	default:
+		mtk_vcodec_err(inst, "unknown msg id %x", msg->msg_id);
+		break;
+	}
+
+	inst->vpu_inst.signaled = 1;
+	inst->vpu_inst.failure = (msg->status != VENC_IPI_MSG_STATUS_OK);
+
+	mtk_vcodec_debug_leave(inst);
+}
+
+static int h264_enc_vpu_send_msg(struct venc_h264_inst *inst, void *msg,
+				 int len)
+{
+	int status;
+
+	mtk_vcodec_debug_enter(inst);
+	status = vpu_ipi_send(inst->dev, IPI_VENC_H264, msg, len);
+	if (status) {
+		mtk_vcodec_err(inst, "vpu_ipi_send msg %x len %d fail %d",
+			       *(unsigned int *)msg, len, status);
+		return -EINVAL;
+	}
+	mtk_vcodec_debug_leave(inst);
+
+	return 0;
+}
+
+int h264_enc_vpu_init(struct venc_h264_inst *inst)
+{
+	int status;
+	struct venc_ap_ipi_msg_init out;
+
+	mtk_vcodec_debug_enter(inst);
+
+	init_waitqueue_head(&inst->vpu_inst.wq_hd);
+	inst->vpu_inst.signaled = 0;
+	inst->vpu_inst.failure = 0;
+
+	status = vpu_ipi_register(inst->dev, IPI_VENC_H264,
+				  h264_enc_vpu_ipi_handler,
+				  "h264_enc", NULL);
+	if (status) {
+		mtk_vcodec_err(inst, "vpu_ipi_register fail %d", status);
+		return -EINVAL;
+	}
+
+	out.msg_id = AP_IPIMSG_H264_ENC_INIT;
+	out.venc_inst = (unsigned long)inst;
+	if (h264_enc_vpu_send_msg(inst, &out, sizeof(out)) ||
+	    inst->vpu_inst.failure) {
+		mtk_vcodec_err(inst, "AP_IPIMSG_H264_ENC_INIT failed");
+		return -EINVAL;
+	}
+
+	mtk_vcodec_debug_leave(inst);
+
+	return 0;
+}
+
+int h264_enc_vpu_set_param(struct venc_h264_inst *inst, unsigned int id,
+			   void *param)
+{
+	struct venc_ap_ipi_msg_set_param out;
+
+	mtk_vcodec_debug(inst, "id %d ->", id);
+
+	out.msg_id = AP_IPIMSG_H264_ENC_SET_PARAM;
+	out.inst_id = inst->vpu_inst.id;
+	out.param_id = id;
+	switch (id) {
+	case VENC_SET_PARAM_ENC: {
+		struct venc_enc_prm *enc_param = (struct venc_enc_prm *)param;
+
+		inst->vpu_inst.drv->config.input_fourcc =
+			enc_param->input_fourcc;
+		inst->vpu_inst.drv->config.bitrate = enc_param->bitrate;
+		inst->vpu_inst.drv->config.pic_w = enc_param->width;
+		inst->vpu_inst.drv->config.pic_h = enc_param->height;
+		inst->vpu_inst.drv->config.buf_w = enc_param->buf_width;
+		inst->vpu_inst.drv->config.buf_h = enc_param->buf_height;
+		inst->vpu_inst.drv->config.intra_period =
+			enc_param->intra_period;
+		inst->vpu_inst.drv->config.framerate = enc_param->frm_rate;
+		inst->vpu_inst.drv->config.profile =
+			h264_get_profile(enc_param->h264_profile);
+		inst->vpu_inst.drv->config.level =
+			h264_get_level(enc_param->h264_level);
+		inst->vpu_inst.drv->config.wfd = 0;
+		out.data_item = 0;
+		break;
+	}
+	case VENC_SET_PARAM_FORCE_INTRA:
+		out.data_item = 0;
+		break;
+	case VENC_SET_PARAM_ADJUST_BITRATE:
+		out.data_item = 1;
+		out.data[0] = *(unsigned int *)param;
+		break;
+	case VENC_SET_PARAM_ADJUST_FRAMERATE:
+		out.data_item = 1;
+		out.data[0] = *(unsigned int *)param;
+		break;
+	case VENC_SET_PARAM_I_FRAME_INTERVAL:
+		out.data_item = 1;
+		out.data[0] = *(unsigned int *)param;
+		break;
+	case VENC_SET_PARAM_SKIP_FRAME:
+		out.data_item = 0;
+		break;
+	}
+	if (h264_enc_vpu_send_msg(inst, &out, sizeof(out)) ||
+	    inst->vpu_inst.failure) {
+		mtk_vcodec_err(inst,
+			       "AP_IPIMSG_H264_ENC_SET_PARAM %d fail", id);
+		return -EINVAL;
+	}
+
+	mtk_vcodec_debug(inst, "id %d <-", id);
+
+	return 0;
+}
+
+int h264_enc_vpu_encode(struct venc_h264_inst *inst, unsigned int bs_mode,
+			struct venc_frm_buf *frm_buf,
+			struct mtk_vcodec_mem *bs_buf,
+			unsigned int *bs_size)
+{
+	struct venc_ap_ipi_msg_enc out;
+
+	mtk_vcodec_debug(inst, "bs_mode %d ->", bs_mode);
+
+	out.msg_id = AP_IPIMSG_H264_ENC_ENCODE;
+	out.inst_id = inst->vpu_inst.id;
+	out.bs_mode = bs_mode;
+	if (frm_buf) {
+		if ((frm_buf->fb_addr.dma_addr % 16 == 0) &&
+		    (frm_buf->fb_addr1.dma_addr % 16 == 0) &&
+		    (frm_buf->fb_addr2.dma_addr % 16 == 0)) {
+			out.input_addr[0] = frm_buf->fb_addr.dma_addr;
+			out.input_addr[1] = frm_buf->fb_addr1.dma_addr;
+			out.input_addr[2] = frm_buf->fb_addr2.dma_addr;
+		} else {
+			mtk_vcodec_err(inst, "dma_addr not align to 16");
+			return -EINVAL;
+		}
+	} else {
+		out.input_addr[0] = 0;
+		out.input_addr[1] = 0;
+		out.input_addr[2] = 0;
+	}
+	if (bs_buf) {
+		out.bs_addr = bs_buf->dma_addr;
+		out.bs_size = bs_buf->size;
+	} else {
+		out.bs_addr = 0;
+		out.bs_size = 0;
+	}
+	if (h264_enc_vpu_send_msg(inst, &out, sizeof(out)) ||
+	    inst->vpu_inst.failure) {
+		mtk_vcodec_err(inst, "AP_IPIMSG_H264_ENC_ENCODE %d fail",
+			       bs_mode);
+		return -EINVAL;
+	}
+
+	mtk_vcodec_debug(inst, "state %d size %d key_frm %d",
+			 inst->vpu_inst.state, inst->vpu_inst.bs_size,
+			 inst->is_key_frm);
+
+	inst->vpu_inst.wait_int = 1;
+	if (inst->vpu_inst.state == VEN_IPI_MSG_ENC_STATE_SKIP) {
+		*bs_size = inst->vpu_inst.bs_size;
+		memcpy(bs_buf->va,
+		       inst->work_bufs[VENC_H264_VPU_WORK_BUF_SKIP_FRAME].va,
+		       *bs_size);
+		inst->vpu_inst.wait_int = 0;
+	}
+
+	mtk_vcodec_debug(inst, "bs_mode %d <-", bs_mode);
+
+	return 0;
+}
+
+int h264_enc_vpu_deinit(struct venc_h264_inst *inst)
+{
+	struct venc_ap_ipi_msg_deinit out;
+
+	mtk_vcodec_debug_enter(inst);
+
+	out.msg_id = AP_IPIMSG_H264_ENC_DEINIT;
+	out.inst_id = inst->vpu_inst.id;
+	if (h264_enc_vpu_send_msg(inst, &out, sizeof(out)) ||
+	    inst->vpu_inst.failure) {
+		mtk_vcodec_err(inst, "AP_IPIMSG_H264_ENC_DEINIT fail");
+		return -EINVAL;
+	}
+
+	mtk_vcodec_debug_leave(inst);
+
+	return 0;
+}
diff --git a/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.h b/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.h
new file mode 100644
index 0000000..deccc6f
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.h
@@ -0,0 +1,30 @@
+/*
+ * Copyright (c) 2015 MediaTek Inc.
+ * Author: Jungchang Tsao <jungchang.tsao@mediatek.com>
+ *         Daniel Hsiao <daniel.hsiao@mediatek.com>
+ *         PoChun Lin <pochun.lin@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef _VENC_H264_VPU_H_
+#define _VENC_H264_VPU_H_
+
+int h264_enc_vpu_init(struct venc_h264_inst *inst);
+int h264_enc_vpu_set_param(struct venc_h264_inst *inst, unsigned int id,
+			   void *param);
+int h264_enc_vpu_encode(struct venc_h264_inst *inst, unsigned int bs_mode,
+			struct venc_frm_buf *frm_buf,
+			struct mtk_vcodec_mem *bs_buf,
+			unsigned int *bs_size);
+int h264_enc_vpu_deinit(struct venc_h264_inst *inst);
+
+#endif
diff --git a/drivers/media/platform/mtk-vcodec/venc_drv_if.c b/drivers/media/platform/mtk-vcodec/venc_drv_if.c
index d293f2c..28ef4a7 100644
--- a/drivers/media/platform/mtk-vcodec/venc_drv_if.c
+++ b/drivers/media/platform/mtk-vcodec/venc_drv_if.c
@@ -25,6 +25,7 @@
 
 #include "venc_drv_base.h"
 #include "vp8_enc/venc_vp8_if.h"
+#include "h264_enc/venc_h264_if.h"
 
 int venc_if_create(struct mtk_vcodec_ctx *ctx, unsigned int fourcc)
 {
@@ -38,6 +39,8 @@ int venc_if_create(struct mtk_vcodec_ctx *ctx, unsigned int fourcc)
                 ctx->enc_if = get_vp8_enc_comm_if();
                 break;
 	case V4L2_PIX_FMT_H264:
+	        ctx->enc_if = get_h264_enc_comm_if();
+	        break;
 	default:
 		return -EINVAL;
 	}
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 102+ messages in thread

* [PATCH v4 7/8] [media] vcodec: mediatek: Add Mediatek H264 Video Encoder Driver
@ 2016-02-04 11:35               ` Tiffany Lin
  0 siblings, 0 replies; 102+ messages in thread
From: Tiffany Lin @ 2016-02-04 11:35 UTC (permalink / raw)
  To: linux-arm-kernel

Add h264 encoder driver for MT8173

Signed-off-by: Daniel Hsiao <daniel.hsiao@mediatek.com>
Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
---
 drivers/media/platform/mtk-vcodec/Makefile         |    3 +-
 .../media/platform/mtk-vcodec/h264_enc/Makefile    |    6 +
 .../platform/mtk-vcodec/h264_enc/venc_h264_if.c    |  540 ++++++++++++++++++++
 .../platform/mtk-vcodec/h264_enc/venc_h264_if.h    |  165 ++++++
 .../platform/mtk-vcodec/h264_enc/venc_h264_vpu.c   |  309 +++++++++++
 .../platform/mtk-vcodec/h264_enc/venc_h264_vpu.h   |   30 ++
 drivers/media/platform/mtk-vcodec/venc_drv_if.c    |    3 +
 7 files changed, 1055 insertions(+), 1 deletion(-)
 create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/Makefile
 create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.c
 create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.h
 create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.c
 create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.h

diff --git a/drivers/media/platform/mtk-vcodec/Makefile b/drivers/media/platform/mtk-vcodec/Makefile
index f4ef502..f47dfc7 100644
--- a/drivers/media/platform/mtk-vcodec/Makefile
+++ b/drivers/media/platform/mtk-vcodec/Makefile
@@ -5,6 +5,7 @@ obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += mtk_vcodec_intr.o \
 				       mtk_vcodec_enc_pm.o \
 				       venc_drv_if.o
 
-obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += vp8_enc/
+
+obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += vp8_enc/ h264_enc/
 
 ccflags-y += -I$(srctree)/drivers/media/platform/mtk-vpu
diff --git a/drivers/media/platform/mtk-vcodec/h264_enc/Makefile b/drivers/media/platform/mtk-vcodec/h264_enc/Makefile
new file mode 100644
index 0000000..765b45f
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/h264_enc/Makefile
@@ -0,0 +1,6 @@
+obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += venc_h264_if.o venc_h264_vpu.o
+
+ccflags-y += \
+    -I$(srctree)/drivers/media/platform/mtk-vcodec/ \
+    -I$(srctree)/drivers/media/platform/mtk-vcodec/h264_enc \
+    -I$(srctree)/drivers/media/platform/mtk-vpu
diff --git a/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.c b/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.c
new file mode 100644
index 0000000..c35fb26
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.c
@@ -0,0 +1,540 @@
+/*
+ * Copyright (c) 2015 MediaTek Inc.
+ * Author: Jungchang Tsao <jungchang.tsao@mediatek.com>
+ *         Daniel Hsiao <daniel.hsiao@mediatek.com>
+ *         PoChun Lin <pochun.lin@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/interrupt.h>
+#include <linux/kernel.h>
+#include <linux/slab.h>
+
+#include "mtk_vcodec_drv.h"
+#include "mtk_vcodec_util.h"
+#include "mtk_vcodec_intr.h"
+#include "mtk_vcodec_enc.h"
+#include "mtk_vcodec_pm.h"
+#include "mtk_vpu.h"
+
+#include "venc_h264_if.h"
+#include "venc_h264_vpu.h"
+
+#define VENC_PIC_BITSTREAM_BYTE_CNT 0x0098
+
+enum venc_h264_irq_status {
+	H264_IRQ_STATUS_ENC_SPS_INT = (1 << 0),
+	H264_IRQ_STATUS_ENC_PPS_INT = (1 << 1),
+	H264_IRQ_STATUS_ENC_FRM_INT = (1 << 2),
+};
+
+static inline void h264_write_reg(struct venc_h264_inst *inst, u32 addr,
+				  u32 val)
+{
+	writel(val, inst->hw_base + addr);
+}
+
+static inline u32 h264_read_reg(struct venc_h264_inst *inst, u32 addr)
+{
+	return readl(inst->hw_base + addr);
+}
+
+static void h264_enc_free_work_buf(struct venc_h264_inst *inst)
+{
+	int i;
+
+	mtk_vcodec_debug_enter(inst);
+
+	/* Except the SKIP_FRAME buffers,
+	 * other buffers need to be freed by AP.
+	 */
+	for (i = 0; i < VENC_H264_VPU_WORK_BUF_MAX; i++) {
+		if (i != VENC_H264_VPU_WORK_BUF_SKIP_FRAME)
+			if (inst->work_bufs[i].va != NULL)
+				mtk_vcodec_mem_free(inst->ctx,
+						    &inst->work_bufs[i]);
+	}
+
+	if (inst->pps_buf.va != NULL)
+		mtk_vcodec_mem_free(inst->ctx, &inst->pps_buf);
+
+	mtk_vcodec_debug_leave(inst);
+}
+
+static int h264_enc_alloc_work_buf(struct venc_h264_inst *inst, void *param)
+{
+	int i;
+	int ret = 0;
+	struct venc_h264_vpu_buf *wb = inst->vpu_inst.drv->work_bufs;
+	struct venc_enc_prm *enc_param = param;
+
+	mtk_vcodec_debug_enter(inst);
+
+	for (i = 0; i < VENC_H264_VPU_WORK_BUF_MAX; i++) {
+		/*
+		 * This 'wb' structure is set by VPU side and shared to AP for
+		 * buffer allocation and IO virtual addr mapping. For most of
+		 * the buffers, AP will allocate the buffer according to 'size'
+		 * field and store the IO virtual addr in 'iova' field. There
+		 * are two exceptions:
+		 * (1) RC_CODE buffer, it's pre-allocated in the VPU side, and
+		 * save the VPU addr in the 'vpua' field. The AP will translate
+		 * the VPU addr to the corresponding IO virtual addr and store
+		 * in 'iova' field for reg setting in VPU side.
+		 * (2) SKIP_FRAME buffer, it's pre-allocated in the VPU side,
+		 * and save the VPU addr in the 'vpua' field. The AP will
+		 * translate the VPU addr to the corresponding AP side virtual
+		 * address and do some memcpy access to move to bitstream buffer
+		 * assigned by v4l2 layer.
+		 */
+		if (i == VENC_H264_VPU_WORK_BUF_RC_CODE) {
+			void *tmp_va;
+
+			tmp_va = vpu_mapping_dm_addr(inst->dev, wb[i].vpua);
+			inst->work_bufs[i].size = wb[i].size;
+			if (mtk_vcodec_mem_alloc(inst->ctx,
+						 &inst->work_bufs[i])) {
+				mtk_vcodec_err(inst,
+					       "cannot allocate buf %d", i);
+				ret = -ENOMEM;
+				goto err_alloc;
+			}
+			memcpy(inst->work_bufs[i].va, tmp_va, wb[i].size);
+			wb[i].iova = inst->work_bufs[i].dma_addr;
+		} else if (i == VENC_H264_VPU_WORK_BUF_SKIP_FRAME) {
+			inst->work_bufs[i].size = wb[i].size;
+			inst->work_bufs[i].va = vpu_mapping_dm_addr(
+				inst->dev, wb[i].vpua);
+			inst->work_bufs[i].dma_addr = 0;
+			wb[i].iova = inst->work_bufs[i].dma_addr;
+		} else if (i == VENC_H264_VPU_WORK_BUF_SRC_LUMA ||
+			   i == VENC_H264_VPU_WORK_BUF_SRC_CHROMA ||
+			   i == VENC_H264_VPU_WORK_BUF_SRC_CHROMA_CB ||
+			   i == VENC_H264_VPU_WORK_BUF_SRC_CHROMA_CR) {
+			inst->work_bufs[i].size = wb[i].size;
+			inst->work_bufs[i].dma_addr = 0;
+			inst->work_bufs[i].va = NULL;
+			wb[i].iova = inst->work_bufs[i].dma_addr;
+		} else {
+			inst->work_bufs[i].size = wb[i].size;
+			if (mtk_vcodec_mem_alloc(inst->ctx,
+						 &inst->work_bufs[i])) {
+				mtk_vcodec_err(inst,
+					       "cannot allocate buf %d", i);
+				ret = -ENOMEM;
+				goto err_alloc;
+			}
+			wb[i].iova = inst->work_bufs[i].dma_addr;
+		}
+		mtk_vcodec_debug(inst, "buf[%d] va=0x%p iova=0x%p size=0x%lx",
+				 i, inst->work_bufs[i].va,
+				 (void *)inst->work_bufs[i].dma_addr,
+				 inst->work_bufs[i].size);
+	}
+
+	if (enc_param->input_fourcc == VENC_YUV_FORMAT_NV12 ||
+	    enc_param->input_fourcc == VENC_YUV_FORMAT_NV21) {
+		enc_param->sizeimage[0] =
+			inst->work_bufs[VENC_H264_VPU_WORK_BUF_SRC_LUMA].size;
+		enc_param->sizeimage[1] =
+			inst->work_bufs[VENC_H264_VPU_WORK_BUF_SRC_CHROMA].size;
+		enc_param->sizeimage[2] = 0;
+	} else {
+		enc_param->sizeimage[0] =
+			inst->work_bufs[VENC_H264_VPU_WORK_BUF_SRC_LUMA].size;
+		enc_param->sizeimage[1] =
+			inst->work_bufs[VENC_H264_VPU_WORK_BUF_SRC_CHROMA_CB].size;
+		enc_param->sizeimage[2] =
+			inst->work_bufs[VENC_H264_VPU_WORK_BUF_SRC_CHROMA_CR].size;
+	}
+
+	/* the pps_buf is used by AP side only */
+	inst->pps_buf.size = 128;
+	if (mtk_vcodec_mem_alloc(inst->ctx,
+				 &inst->pps_buf)) {
+		mtk_vcodec_err(inst, "cannot allocate pps_buf");
+		ret = -ENOMEM;
+		goto err_alloc;
+	}
+	mtk_vcodec_debug_leave(inst);
+
+	return ret;
+
+err_alloc:
+	h264_enc_free_work_buf(inst);
+
+	return ret;
+}
+
+static unsigned int h264_enc_wait_venc_done(struct venc_h264_inst *inst)
+{
+	unsigned int irq_status = 0;
+	struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)inst->ctx;
+
+	mtk_vcodec_wait_for_done_ctx(ctx, MTK_INST_IRQ_RECEIVED,
+				     WAIT_INTR_TIMEOUT, true);
+	irq_status = ctx->irq_status;
+	mtk_vcodec_debug(inst, "irq_status %x <-", irq_status);
+
+	return irq_status;
+}
+
+static int h264_encode_sps(struct venc_h264_inst *inst,
+			   struct mtk_vcodec_mem *bs_buf,
+			   unsigned int *bs_size)
+{
+	int ret = 0;
+	unsigned int irq_status;
+
+	mtk_vcodec_debug_enter(inst);
+
+	ret = h264_enc_vpu_encode(inst, H264_BS_MODE_SPS, NULL,
+				  bs_buf, bs_size);
+	if (ret)
+		return ret;
+
+	irq_status = h264_enc_wait_venc_done(inst);
+	if (irq_status != H264_IRQ_STATUS_ENC_SPS_INT) {
+		mtk_vcodec_err(inst, "expect irq status %d",
+			       H264_IRQ_STATUS_ENC_SPS_INT);
+		return -EINVAL;
+	}
+
+	*bs_size = h264_read_reg(inst, VENC_PIC_BITSTREAM_BYTE_CNT);
+	mtk_vcodec_debug(inst, "bs size %d <-", *bs_size);
+
+	return ret;
+}
+
+static int h264_encode_pps(struct venc_h264_inst *inst,
+			   struct mtk_vcodec_mem *bs_buf,
+			   unsigned int *bs_size)
+{
+	int ret = 0;
+	unsigned int irq_status;
+
+	mtk_vcodec_debug_enter(inst);
+
+	ret = h264_enc_vpu_encode(inst, H264_BS_MODE_PPS, NULL,
+				  bs_buf, bs_size);
+	if (ret)
+		return ret;
+
+	irq_status = h264_enc_wait_venc_done(inst);
+	if (irq_status != H264_IRQ_STATUS_ENC_PPS_INT) {
+		mtk_vcodec_err(inst, "expect irq status %d",
+			       H264_IRQ_STATUS_ENC_PPS_INT);
+		return -EINVAL;
+	}
+
+	*bs_size = h264_read_reg(inst, VENC_PIC_BITSTREAM_BYTE_CNT);
+	mtk_vcodec_debug(inst, "bs size %d <-", *bs_size);
+
+	return ret;
+}
+
+static int h264_encode_frame(struct venc_h264_inst *inst,
+			     struct venc_frm_buf *frm_buf,
+			     struct mtk_vcodec_mem *bs_buf,
+			     unsigned int *bs_size)
+{
+	int ret = 0;
+	unsigned int irq_status;
+
+	mtk_vcodec_debug_enter(inst);
+
+	ret = h264_enc_vpu_encode(inst, H264_BS_MODE_FRAME, frm_buf,
+				  bs_buf, bs_size);
+	if (ret)
+		return ret;
+
+	/*
+	 * skip frame case: The skip frame buffer is composed by vpu side only,
+	 * it does not trigger the hw, so skip the wait interrupt operation.
+	 */
+	if (!inst->vpu_inst.wait_int) {
+		++inst->frm_cnt;
+		return ret;
+	}
+
+	irq_status = h264_enc_wait_venc_done(inst);
+	if (irq_status != H264_IRQ_STATUS_ENC_FRM_INT) {
+		mtk_vcodec_err(inst, "irq_status=%d failed", irq_status);
+		return -EINVAL;
+	}
+
+	*bs_size = h264_read_reg(inst,
+				 VENC_PIC_BITSTREAM_BYTE_CNT);
+	++inst->frm_cnt;
+	mtk_vcodec_debug(inst, "frm %d bs size %d key_frm %d <-",
+			 inst->frm_cnt,
+			 *bs_size, inst->is_key_frm);
+
+	return ret;
+}
+
+static void h264_encode_filler(struct venc_h264_inst *inst, void *buf,
+			       int size)
+{
+	unsigned char *p = buf;
+
+	*p++ = 0x0;
+	*p++ = 0x0;
+	*p++ = 0x0;
+	*p++ = 0x1;
+	*p++ = 0xc;
+	size -= 5;
+	while (size) {
+		*p++ = 0xff;
+		size -= 1;
+	}
+}
+
+static int h264_enc_init(struct mtk_vcodec_ctx *ctx, unsigned long *handle)
+{
+	int ret = 0;
+	struct venc_h264_inst *inst;
+
+	inst = kzalloc(sizeof(*inst), GFP_KERNEL);
+	if (!inst)
+		return -ENOMEM;
+
+	inst->ctx = ctx;
+	inst->dev = mtk_vcodec_get_plat_dev(ctx);
+	inst->hw_base = mtk_vcodec_get_reg_addr(inst->ctx, VENC_SYS);
+
+	mtk_vcodec_debug_enter(inst);
+	ret = h264_enc_vpu_init(inst);
+	if (ret)
+		kfree(inst);
+	else
+		(*handle) = (unsigned long)inst;
+
+	mtk_vcodec_debug_leave(inst);
+
+	return ret;
+}
+
+static int h264_enc_encode(unsigned long handle,
+			   enum venc_start_opt opt,
+			   struct venc_frm_buf *frm_buf,
+			   struct mtk_vcodec_mem *bs_buf,
+			   struct venc_done_result *result)
+{
+	int ret = 0;
+	struct venc_h264_inst *inst = (struct venc_h264_inst *)handle;
+	struct mtk_vcodec_ctx *ctx = inst->ctx;
+
+	mtk_vcodec_debug(inst, "opt %d ->", opt);
+
+	enable_irq(ctx->dev->enc_irq);
+
+	switch (opt) {
+	case VENC_START_OPT_ENCODE_SEQUENCE_HEADER: {
+		unsigned int bs_size_sps;
+		unsigned int bs_size_pps;
+
+		memset(bs_buf->va, 0x38, 20);
+		ret = h264_encode_sps(inst, bs_buf, &bs_size_sps);
+		if (ret)
+			goto encode_err;
+
+		memset(inst->pps_buf.va, 0x49, 20);
+		ret = h264_encode_pps(inst, &inst->pps_buf, &bs_size_pps);
+		if (ret)
+			goto encode_err;
+
+		memcpy(bs_buf->va + bs_size_sps,
+		       inst->pps_buf.va,
+		       bs_size_pps);
+		result->bs_size = bs_size_sps + bs_size_pps;
+		result->is_key_frm = false;
+	}
+	break;
+
+	case VENC_START_OPT_ENCODE_FRAME:
+		if (inst->prepend_hdr) {
+			int hdr_sz;
+			int hdr_sz_ext;
+			int bs_alignment = 128;
+			int filler_sz = 0;
+			struct mtk_vcodec_mem tmp_bs_buf;
+			unsigned int bs_size_sps;
+			unsigned int bs_size_pps;
+			unsigned int bs_size_frm;
+
+			mtk_vcodec_debug(inst,
+					 "h264_encode_frame prepend SPS/PPS");
+			ret = h264_encode_sps(inst, bs_buf, &bs_size_sps);
+			if (ret)
+				goto encode_err;
+
+			ret = h264_encode_pps(inst, &inst->pps_buf,
+					      &bs_size_pps);
+			if (ret)
+				goto encode_err;
+
+			memcpy(bs_buf->va + bs_size_sps,
+			       inst->pps_buf.va,
+			       bs_size_pps);
+
+			hdr_sz = bs_size_sps + bs_size_pps;
+			hdr_sz_ext = (hdr_sz & (bs_alignment - 1));
+			if (hdr_sz_ext) {
+				filler_sz = bs_alignment - hdr_sz_ext;
+				if (hdr_sz_ext + 5 > bs_alignment)
+					filler_sz += bs_alignment;
+				h264_encode_filler(
+					inst, bs_buf->va + hdr_sz,
+					filler_sz);
+			}
+
+			tmp_bs_buf.va = bs_buf->va + hdr_sz +
+				filler_sz;
+			tmp_bs_buf.dma_addr = bs_buf->dma_addr + hdr_sz +
+				filler_sz;
+			tmp_bs_buf.size = bs_buf->size -
+				(hdr_sz + filler_sz);
+
+			ret = h264_encode_frame(inst, frm_buf, &tmp_bs_buf,
+						&bs_size_frm);
+			if (ret)
+				goto encode_err;
+
+			result->bs_size = hdr_sz + filler_sz + bs_size_frm;
+			mtk_vcodec_debug(inst,
+					 "hdr %d filler %d frame %d bs %d",
+					 hdr_sz, filler_sz, bs_size_frm,
+					 result->bs_size);
+
+			inst->prepend_hdr = 0;
+		} else {
+			ret = h264_encode_frame(inst, frm_buf, bs_buf,
+						&result->bs_size);
+			if (ret)
+				goto encode_err;
+		}
+		result->is_key_frm = inst->is_key_frm;
+		break;
+
+	default:
+		mtk_vcodec_err(inst, "venc_start_opt %d not supported", opt);
+		ret = -EINVAL;
+		break;
+	}
+
+encode_err:
+	if (ret)
+		result->msg = VENC_MESSAGE_ERR;
+	else
+		result->msg = VENC_MESSAGE_OK;
+
+	disable_irq(ctx->dev->enc_irq);
+	mtk_vcodec_debug(inst, "opt %d <-", opt);
+
+	return ret;
+}
+
+static int h264_enc_set_param(unsigned long handle,
+			      enum venc_set_param_type type, void *in)
+{
+	int ret = 0;
+	struct venc_h264_inst *inst = (struct venc_h264_inst *)handle;
+	struct venc_enc_prm *enc_prm;
+
+	mtk_vcodec_debug(inst, "->type=%d", type);
+
+	switch (type) {
+	case VENC_SET_PARAM_ENC:
+		enc_prm = in;
+		ret = h264_enc_vpu_set_param(inst, type, enc_prm);
+		if (ret)
+			break;
+		if (inst->work_buf_allocated == 1) {
+			h264_enc_free_work_buf(inst);
+			inst->work_buf_allocated = 0;
+		}
+		if (inst->work_buf_allocated == 0) {
+			ret = h264_enc_alloc_work_buf(inst, enc_prm);
+			if (ret)
+				break;
+			inst->work_buf_allocated = 1;
+		}
+		break;
+
+	case VENC_SET_PARAM_FORCE_INTRA:
+		ret = h264_enc_vpu_set_param(inst, type, 0);
+		break;
+
+	case VENC_SET_PARAM_ADJUST_BITRATE:
+		enc_prm = in;
+		ret = h264_enc_vpu_set_param(inst, type, &enc_prm->bitrate);
+		break;
+
+	case VENC_SET_PARAM_ADJUST_FRAMERATE:
+		enc_prm = in;
+		ret = h264_enc_vpu_set_param(inst, type, &enc_prm->frm_rate);
+		break;
+
+	case VENC_SET_PARAM_I_FRAME_INTERVAL:
+		ret = h264_enc_vpu_set_param(inst, type, in);
+		break;
+
+	case VENC_SET_PARAM_SKIP_FRAME:
+		ret = h264_enc_vpu_set_param(inst, type, 0);
+		break;
+
+	case VENC_SET_PARAM_PREPEND_HEADER:
+		inst->prepend_hdr = 1;
+		mtk_vcodec_debug(inst, "set prepend header mode");
+		break;
+
+	default:
+		mtk_vcodec_err(inst, "type %d not supported", type);
+		ret = -EINVAL;
+		break;
+	}
+
+	mtk_vcodec_debug_leave(inst);
+
+	return ret;
+}
+
+static int h264_enc_deinit(unsigned long handle)
+{
+	int ret = 0;
+	struct venc_h264_inst *inst = (struct venc_h264_inst *)handle;
+
+	mtk_vcodec_debug_enter(inst);
+
+	ret = h264_enc_vpu_deinit(inst);
+
+	if (inst->work_buf_allocated)
+		h264_enc_free_work_buf(inst);
+
+	mtk_vcodec_debug_leave(inst);
+	kfree(inst);
+
+	return ret;
+}
+
+static struct venc_common_if venc_h264_if = {
+	h264_enc_init,
+	h264_enc_encode,
+	h264_enc_set_param,
+	h264_enc_deinit,
+};
+
+struct venc_common_if *get_h264_enc_comm_if(void)
+{
+	return &venc_h264_if;
+}
diff --git a/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.h b/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.h
new file mode 100644
index 0000000..9ac317a
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.h
@@ -0,0 +1,165 @@
+/*
+ * Copyright (c) 2015 MediaTek Inc.
+ * Author: Jungchang Tsao <jungchang.tsao@mediatek.com>
+ *         Daniel Hsiao <daniel.hsiao@mediatek.com>
+ *         PoChun Lin <pochun.lin@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef _VENC_H264_IF_H_
+#define _VENC_H264_IF_H_
+
+#include "venc_drv_base.h"
+
+/**
+ * enum venc_h264_vpu_work_buf - h264 encoder buffer index
+ */
+enum venc_h264_vpu_work_buf {
+	VENC_H264_VPU_WORK_BUF_RC_INFO,
+	VENC_H264_VPU_WORK_BUF_RC_CODE,
+	VENC_H264_VPU_WORK_BUF_REC_LUMA,
+	VENC_H264_VPU_WORK_BUF_REC_CHROMA,
+	VENC_H264_VPU_WORK_BUF_REF_LUMA,
+	VENC_H264_VPU_WORK_BUF_REF_CHROMA,
+	VENC_H264_VPU_WORK_BUF_MV_INFO_1,
+	VENC_H264_VPU_WORK_BUF_MV_INFO_2,
+	VENC_H264_VPU_WORK_BUF_SKIP_FRAME,
+	VENC_H264_VPU_WORK_BUF_SRC_LUMA,
+	VENC_H264_VPU_WORK_BUF_SRC_CHROMA,
+	VENC_H264_VPU_WORK_BUF_SRC_CHROMA_CB,
+	VENC_H264_VPU_WORK_BUF_SRC_CHROMA_CR,
+	VENC_H264_VPU_WORK_BUF_MAX,
+};
+
+/**
+ * enum venc_h264_bs_mode - for bs_mode argument in h264_enc_vpu_encode
+ */
+enum venc_h264_bs_mode {
+	H264_BS_MODE_SPS,
+	H264_BS_MODE_PPS,
+	H264_BS_MODE_FRAME,
+};
+
+/*
+ * struct venc_h264_vpu_config - Structure for h264 encoder configuration
+ * @input_fourcc: input fourcc
+ * @bitrate: target bitrate (in bps)
+ * @pic_w: picture width
+ * @pic_h: picture height
+ * @buf_w: buffer width
+ * @buf_h: buffer height
+ * @intra_period: intra frame period
+ * @framerate: frame rate
+ * @profile: as specified in standard
+ * @level: as specified in standard
+ * @wfd: WFD mode 1:on, 0:off
+ */
+struct venc_h264_vpu_config {
+	u32 input_fourcc;
+	u32 bitrate;
+	u32 pic_w;
+	u32 pic_h;
+	u32 buf_w;
+	u32 buf_h;
+	u32 intra_period;
+	u32 framerate;
+	u32 profile;
+	u32 level;
+	u32 wfd;
+};
+
+/*
+ * struct venc_h264_vpu_buf - Structure for buffer information
+ * @align: buffer alignment (in bytes)
+ * @iova: IO virtual address
+ * @vpua: VPU side memory addr which is used by RC_CODE
+ * @size: buffer size (in bytes)
+ */
+struct venc_h264_vpu_buf {
+	u32 align;
+	u32 iova;
+	u32 vpua;
+	u32 size;
+};
+
+/*
+ * struct venc_h264_vpu_drv - Structure for VPU driver control and info share
+ * This structure is allocated in VPU side and shared to AP side.
+ * @config: h264 encoder configuration
+ * @work_bufs: working buffer information in VPU side
+ * The work_bufs here is for storing the 'size' info shared to AP side.
+ * The similar item in struct venc_h264_inst is for memory allocation
+ * in AP side. The AP driver will copy the 'size' from here to the one in
+ * struct mtk_vcodec_mem, then invoke mtk_vcodec_mem_alloc to allocate
+ * the buffer. After that, bypass the 'dma_addr' to the 'iova' field here for
+ * register setting in VPU side.
+ */
+struct venc_h264_vpu_drv {
+	struct venc_h264_vpu_config config;
+	struct venc_h264_vpu_buf work_bufs[VENC_H264_VPU_WORK_BUF_MAX];
+};
+
+/*
+ * struct venc_h264_vpu_inst - h264 encoder VPU driver instance
+ * @wq_hd: wait queue used for vpu cmd trigger then wait vpu interrupt done
+ * @signaled: flag used for checking vpu interrupt done
+ * @failure: flag to show vpu cmd succeeds or not
+ * @state: enum venc_ipi_msg_enc_state
+ * @bs_size: bitstream size for skip frame case usage
+ * @wait_int: flag to wait interrupt done (0: for skip frame case, 1: normal
+ *	      case)
+ * @id: VPU instance id
+ * @drv: driver structure allocated by VPU side and shared to AP side for
+ *	 control and info share
+ */
+struct venc_h264_vpu_inst {
+	wait_queue_head_t wq_hd;
+	int signaled;
+	int failure;
+	int state;
+	int bs_size;
+	int wait_int;
+	unsigned int id;
+	struct venc_h264_vpu_drv *drv;
+};
+
+/*
+ * struct venc_h264_inst - h264 encoder AP driver instance
+ * @hw_base: h264 encoder hardware register base
+ * @work_bufs: working buffer
+ * @pps_buf: buffer to store the pps bitstream
+ * @work_buf_allocated: working buffer allocated flag
+ * @frm_cnt: encoded frame count
+ * @prepend_hdr: when the v4l2 layer send VENC_SET_PARAM_PREPEND_HEADER cmd
+ *  through h264_enc_set_param interface, it will set this flag and prepend the
+ *  sps/pps in h264_enc_encode function.
+ * @is_key_frm: key frame flag
+ * @vpu_inst: VPU instance to exchange information between AP and VPU
+ * @ctx: context for v4l2 layer integration
+ * @dev: device for v4l2 layer integration
+ */
+struct venc_h264_inst {
+	void __iomem *hw_base;
+	struct mtk_vcodec_mem work_bufs[VENC_H264_VPU_WORK_BUF_MAX];
+	struct mtk_vcodec_mem pps_buf;
+	bool work_buf_allocated;
+	unsigned int frm_cnt;
+	unsigned int prepend_hdr;
+	unsigned int is_key_frm;
+	struct venc_h264_vpu_inst vpu_inst;
+	void *ctx;
+	struct platform_device *dev;
+};
+
+struct venc_common_if *get_h264_enc_comm_if(void);
+
+#endif
diff --git a/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.c b/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.c
new file mode 100644
index 0000000..450be45
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.c
@@ -0,0 +1,309 @@
+/*
+ * Copyright (c) 2015 MediaTek Inc.
+ * Author: Jungchang Tsao <jungchang.tsao@mediatek.com>
+ *         Daniel Hsiao <daniel.hsiao@mediatek.com>
+ *         PoChun Lin <pochun.lin@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#include "mtk_vpu.h"
+
+#include "venc_h264_if.h"
+#include "venc_h264_vpu.h"
+#include "venc_ipi_msg.h"
+
+static unsigned int h264_get_profile(unsigned int profile)
+{
+	/* (Baseline=66, Main=77, High=100) */
+	switch (profile) {
+	case V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE:
+		return 66;
+	case V4L2_MPEG_VIDEO_H264_PROFILE_MAIN:
+		return 77;
+	case V4L2_MPEG_VIDEO_H264_PROFILE_HIGH:
+		return 100;
+	default:
+		return 100;
+	}
+}
+
+static unsigned int h264_get_level(unsigned int level)
+{
+	/* (UpTo4.1(HighProfile)) */
+	switch (level) {
+	case V4L2_MPEG_VIDEO_H264_LEVEL_1_0:
+		return 10;
+	case V4L2_MPEG_VIDEO_H264_LEVEL_1_1:
+		return 11;
+	case V4L2_MPEG_VIDEO_H264_LEVEL_1_2:
+		return 12;
+	case V4L2_MPEG_VIDEO_H264_LEVEL_1_3:
+		return 13;
+	case V4L2_MPEG_VIDEO_H264_LEVEL_2_0:
+		return 20;
+	case V4L2_MPEG_VIDEO_H264_LEVEL_2_1:
+		return 21;
+	case V4L2_MPEG_VIDEO_H264_LEVEL_2_2:
+		return 22;
+	case V4L2_MPEG_VIDEO_H264_LEVEL_3_0:
+		return 30;
+	case V4L2_MPEG_VIDEO_H264_LEVEL_3_1:
+		return 31;
+	case V4L2_MPEG_VIDEO_H264_LEVEL_3_2:
+		return 32;
+	case V4L2_MPEG_VIDEO_H264_LEVEL_4_0:
+		return 40;
+	case V4L2_MPEG_VIDEO_H264_LEVEL_4_1:
+		return 41;
+	default:
+		return 31;
+	}
+}
+
+static void handle_h264_enc_init_msg(struct venc_h264_inst *inst, void *data)
+{
+	struct venc_vpu_ipi_msg_init *msg = data;
+
+	inst->vpu_inst.id = msg->inst_id;
+	inst->vpu_inst.drv = (struct venc_h264_vpu_drv *)vpu_mapping_dm_addr(
+		inst->dev, msg->inst_id);
+}
+
+static void handle_h264_enc_encode_msg(struct venc_h264_inst *inst, void *data)
+{
+	struct venc_vpu_ipi_msg_enc *msg = data;
+
+	inst->vpu_inst.state = msg->state;
+	inst->vpu_inst.bs_size = msg->bs_size;
+	inst->is_key_frm = msg->key_frame;
+}
+
+static void h264_enc_vpu_ipi_handler(void *data, unsigned int len, void *priv)
+{
+	struct venc_vpu_ipi_msg_common *msg = data;
+	struct venc_h264_inst *inst = (struct venc_h264_inst *)msg->venc_inst;
+
+	mtk_vcodec_debug(inst, "msg_id %x inst %p status %d",
+			 msg->msg_id, inst, msg->status);
+
+	switch (msg->msg_id) {
+	case VPU_IPIMSG_H264_ENC_INIT_DONE:
+		handle_h264_enc_init_msg(inst, data);
+		break;
+	case VPU_IPIMSG_H264_ENC_SET_PARAM_DONE:
+		break;
+	case VPU_IPIMSG_H264_ENC_ENCODE_DONE:
+		handle_h264_enc_encode_msg(inst, data);
+		break;
+	case VPU_IPIMSG_H264_ENC_DEINIT_DONE:
+		break;
+	default:
+		mtk_vcodec_err(inst, "unknown msg id %x", msg->msg_id);
+		break;
+	}
+
+	inst->vpu_inst.signaled = 1;
+	inst->vpu_inst.failure = (msg->status != VENC_IPI_MSG_STATUS_OK);
+
+	mtk_vcodec_debug_leave(inst);
+}
+
+static int h264_enc_vpu_send_msg(struct venc_h264_inst *inst, void *msg,
+				 int len)
+{
+	int status;
+
+	mtk_vcodec_debug_enter(inst);
+	status = vpu_ipi_send(inst->dev, IPI_VENC_H264, msg, len);
+	if (status) {
+		mtk_vcodec_err(inst, "vpu_ipi_send msg %x len %d fail %d",
+			       *(unsigned int *)msg, len, status);
+		return -EINVAL;
+	}
+	mtk_vcodec_debug_leave(inst);
+
+	return 0;
+}
+
+int h264_enc_vpu_init(struct venc_h264_inst *inst)
+{
+	int status;
+	struct venc_ap_ipi_msg_init out;
+
+	mtk_vcodec_debug_enter(inst);
+
+	init_waitqueue_head(&inst->vpu_inst.wq_hd);
+	inst->vpu_inst.signaled = 0;
+	inst->vpu_inst.failure = 0;
+
+	status = vpu_ipi_register(inst->dev, IPI_VENC_H264,
+				  h264_enc_vpu_ipi_handler,
+				  "h264_enc", NULL);
+	if (status) {
+		mtk_vcodec_err(inst, "vpu_ipi_register fail %d", status);
+		return -EINVAL;
+	}
+
+	out.msg_id = AP_IPIMSG_H264_ENC_INIT;
+	out.venc_inst = (unsigned long)inst;
+	if (h264_enc_vpu_send_msg(inst, &out, sizeof(out)) ||
+	    inst->vpu_inst.failure) {
+		mtk_vcodec_err(inst, "AP_IPIMSG_H264_ENC_INIT failed");
+		return -EINVAL;
+	}
+
+	mtk_vcodec_debug_leave(inst);
+
+	return 0;
+}
+
+int h264_enc_vpu_set_param(struct venc_h264_inst *inst, unsigned int id,
+			   void *param)
+{
+	struct venc_ap_ipi_msg_set_param out;
+
+	mtk_vcodec_debug(inst, "id %d ->", id);
+
+	out.msg_id = AP_IPIMSG_H264_ENC_SET_PARAM;
+	out.inst_id = inst->vpu_inst.id;
+	out.param_id = id;
+	switch (id) {
+	case VENC_SET_PARAM_ENC: {
+		struct venc_enc_prm *enc_param = (struct venc_enc_prm *)param;
+
+		inst->vpu_inst.drv->config.input_fourcc =
+			enc_param->input_fourcc;
+		inst->vpu_inst.drv->config.bitrate = enc_param->bitrate;
+		inst->vpu_inst.drv->config.pic_w = enc_param->width;
+		inst->vpu_inst.drv->config.pic_h = enc_param->height;
+		inst->vpu_inst.drv->config.buf_w = enc_param->buf_width;
+		inst->vpu_inst.drv->config.buf_h = enc_param->buf_height;
+		inst->vpu_inst.drv->config.intra_period =
+			enc_param->intra_period;
+		inst->vpu_inst.drv->config.framerate = enc_param->frm_rate;
+		inst->vpu_inst.drv->config.profile =
+			h264_get_profile(enc_param->h264_profile);
+		inst->vpu_inst.drv->config.level =
+			h264_get_level(enc_param->h264_level);
+		inst->vpu_inst.drv->config.wfd = 0;
+		out.data_item = 0;
+		break;
+	}
+	case VENC_SET_PARAM_FORCE_INTRA:
+		out.data_item = 0;
+		break;
+	case VENC_SET_PARAM_ADJUST_BITRATE:
+		out.data_item = 1;
+		out.data[0] = *(unsigned int *)param;
+		break;
+	case VENC_SET_PARAM_ADJUST_FRAMERATE:
+		out.data_item = 1;
+		out.data[0] = *(unsigned int *)param;
+		break;
+	case VENC_SET_PARAM_I_FRAME_INTERVAL:
+		out.data_item = 1;
+		out.data[0] = *(unsigned int *)param;
+		break;
+	case VENC_SET_PARAM_SKIP_FRAME:
+		out.data_item = 0;
+		break;
+	}
+	if (h264_enc_vpu_send_msg(inst, &out, sizeof(out)) ||
+	    inst->vpu_inst.failure) {
+		mtk_vcodec_err(inst,
+			       "AP_IPIMSG_H264_ENC_SET_PARAM %d fail", id);
+		return -EINVAL;
+	}
+
+	mtk_vcodec_debug(inst, "id %d <-", id);
+
+	return 0;
+}
+
+int h264_enc_vpu_encode(struct venc_h264_inst *inst, unsigned int bs_mode,
+			struct venc_frm_buf *frm_buf,
+			struct mtk_vcodec_mem *bs_buf,
+			unsigned int *bs_size)
+{
+	struct venc_ap_ipi_msg_enc out;
+
+	mtk_vcodec_debug(inst, "bs_mode %d ->", bs_mode);
+
+	out.msg_id = AP_IPIMSG_H264_ENC_ENCODE;
+	out.inst_id = inst->vpu_inst.id;
+	out.bs_mode = bs_mode;
+	if (frm_buf) {
+		if ((frm_buf->fb_addr.dma_addr % 16 == 0) &&
+		    (frm_buf->fb_addr1.dma_addr % 16 == 0) &&
+		    (frm_buf->fb_addr2.dma_addr % 16 == 0)) {
+			out.input_addr[0] = frm_buf->fb_addr.dma_addr;
+			out.input_addr[1] = frm_buf->fb_addr1.dma_addr;
+			out.input_addr[2] = frm_buf->fb_addr2.dma_addr;
+		} else {
+			mtk_vcodec_err(inst, "dma_addr not align to 16");
+			return -EINVAL;
+		}
+	} else {
+		out.input_addr[0] = 0;
+		out.input_addr[1] = 0;
+		out.input_addr[2] = 0;
+	}
+	if (bs_buf) {
+		out.bs_addr = bs_buf->dma_addr;
+		out.bs_size = bs_buf->size;
+	} else {
+		out.bs_addr = 0;
+		out.bs_size = 0;
+	}
+	if (h264_enc_vpu_send_msg(inst, &out, sizeof(out)) ||
+	    inst->vpu_inst.failure) {
+		mtk_vcodec_err(inst, "AP_IPIMSG_H264_ENC_ENCODE %d fail",
+			       bs_mode);
+		return -EINVAL;
+	}
+
+	mtk_vcodec_debug(inst, "state %d size %d key_frm %d",
+			 inst->vpu_inst.state, inst->vpu_inst.bs_size,
+			 inst->is_key_frm);
+
+	inst->vpu_inst.wait_int = 1;
+	if (inst->vpu_inst.state == VEN_IPI_MSG_ENC_STATE_SKIP) {
+		*bs_size = inst->vpu_inst.bs_size;
+		memcpy(bs_buf->va,
+		       inst->work_bufs[VENC_H264_VPU_WORK_BUF_SKIP_FRAME].va,
+		       *bs_size);
+		inst->vpu_inst.wait_int = 0;
+	}
+
+	mtk_vcodec_debug(inst, "bs_mode %d <-", bs_mode);
+
+	return 0;
+}
+
+int h264_enc_vpu_deinit(struct venc_h264_inst *inst)
+{
+	struct venc_ap_ipi_msg_deinit out;
+
+	mtk_vcodec_debug_enter(inst);
+
+	out.msg_id = AP_IPIMSG_H264_ENC_DEINIT;
+	out.inst_id = inst->vpu_inst.id;
+	if (h264_enc_vpu_send_msg(inst, &out, sizeof(out)) ||
+	    inst->vpu_inst.failure) {
+		mtk_vcodec_err(inst, "AP_IPIMSG_H264_ENC_DEINIT fail");
+		return -EINVAL;
+	}
+
+	mtk_vcodec_debug_leave(inst);
+
+	return 0;
+}
diff --git a/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.h b/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.h
new file mode 100644
index 0000000..deccc6f
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.h
@@ -0,0 +1,30 @@
+/*
+ * Copyright (c) 2015 MediaTek Inc.
+ * Author: Jungchang Tsao <jungchang.tsao@mediatek.com>
+ *         Daniel Hsiao <daniel.hsiao@mediatek.com>
+ *         PoChun Lin <pochun.lin@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef _VENC_H264_VPU_H_
+#define _VENC_H264_VPU_H_
+
+int h264_enc_vpu_init(struct venc_h264_inst *inst);
+int h264_enc_vpu_set_param(struct venc_h264_inst *inst, unsigned int id,
+			   void *param);
+int h264_enc_vpu_encode(struct venc_h264_inst *inst, unsigned int bs_mode,
+			struct venc_frm_buf *frm_buf,
+			struct mtk_vcodec_mem *bs_buf,
+			unsigned int *bs_size);
+int h264_enc_vpu_deinit(struct venc_h264_inst *inst);
+
+#endif
diff --git a/drivers/media/platform/mtk-vcodec/venc_drv_if.c b/drivers/media/platform/mtk-vcodec/venc_drv_if.c
index d293f2c..28ef4a7 100644
--- a/drivers/media/platform/mtk-vcodec/venc_drv_if.c
+++ b/drivers/media/platform/mtk-vcodec/venc_drv_if.c
@@ -25,6 +25,7 @@
 
 #include "venc_drv_base.h"
 #include "vp8_enc/venc_vp8_if.h"
+#include "h264_enc/venc_h264_if.h"
 
 int venc_if_create(struct mtk_vcodec_ctx *ctx, unsigned int fourcc)
 {
@@ -38,6 +39,8 @@ int venc_if_create(struct mtk_vcodec_ctx *ctx, unsigned int fourcc)
                 ctx->enc_if = get_vp8_enc_comm_if();
                 break;
 	case V4L2_PIX_FMT_H264:
+	        ctx->enc_if = get_h264_enc_comm_if();
+	        break;
 	default:
 		return -EINVAL;
 	}
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 102+ messages in thread

* [PATCH v4 8/8] arm64: dts: mediatek: Add Video Encoder for MT8173
@ 2016-02-04 11:35                 ` Tiffany Lin
  0 siblings, 0 replies; 102+ messages in thread
From: Tiffany Lin @ 2016-02-04 11:35 UTC (permalink / raw)
  To: Hans Verkuil, daniel.thompson, Rob Herring,
	Mauro Carvalho Chehab, Matthias Brugger, Daniel Kurtz,
	Pawel Osciak
  Cc: Eddie Huang, Yingjoe Chen, devicetree, linux-kernel,
	linux-arm-kernel, linux-media, linux-mediatek, PoChun.Lin,
	Tiffany Lin

Add video encoder node for MT8173

Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
---
 arch/arm64/boot/dts/mediatek/mt8173.dtsi |   39 ++++++++++++++++++++++++++++++
 1 file changed, 39 insertions(+)

diff --git a/arch/arm64/boot/dts/mediatek/mt8173.dtsi b/arch/arm64/boot/dts/mediatek/mt8173.dtsi
index 5b0b38a..f61669d 100644
--- a/arch/arm64/boot/dts/mediatek/mt8173.dtsi
+++ b/arch/arm64/boot/dts/mediatek/mt8173.dtsi
@@ -1150,6 +1150,45 @@
 			clock-names = "apb", "smi";
 		};
 
+		vcodec_enc: vcodec@18002000 {
+			compatible = "mediatek,mt8173-vcodec-enc";
+			reg = <0 0x18002000 0 0x1000>,	/* VENC_SYS */
+			      <0 0x19002000 0 0x1000>;	/* VENC_LT_SYS */
+			interrupts = <GIC_SPI 198 IRQ_TYPE_LEVEL_LOW>,
+				     <GIC_SPI 202 IRQ_TYPE_LEVEL_LOW>;
+			mediatek,larb = <&larb3>,
+					<&larb5>;
+			iommus = <&iommu M4U_PORT_VENC_RCPU>,
+				 <&iommu M4U_PORT_VENC_REC>,
+				 <&iommu M4U_PORT_VENC_BSDMA>,
+				 <&iommu M4U_PORT_VENC_SV_COMV>,
+				 <&iommu M4U_PORT_VENC_RD_COMV>,
+				 <&iommu M4U_PORT_VENC_CUR_LUMA>,
+				 <&iommu M4U_PORT_VENC_CUR_CHROMA>,
+				 <&iommu M4U_PORT_VENC_REF_LUMA>,
+				 <&iommu M4U_PORT_VENC_REF_CHROMA>,
+				 <&iommu M4U_PORT_VENC_NBM_RDMA>,
+				 <&iommu M4U_PORT_VENC_NBM_WDMA>,
+				 <&iommu M4U_PORT_VENC_RCPU_SET2>,
+				 <&iommu M4U_PORT_VENC_REC_FRM_SET2>,
+				 <&iommu M4U_PORT_VENC_BSDMA_SET2>,
+				 <&iommu M4U_PORT_VENC_SV_COMA_SET2>,
+				 <&iommu M4U_PORT_VENC_RD_COMA_SET2>,
+				 <&iommu M4U_PORT_VENC_CUR_LUMA_SET2>,
+				 <&iommu M4U_PORT_VENC_CUR_CHROMA_SET2>,
+				 <&iommu M4U_PORT_VENC_REF_LUMA_SET2>,
+				 <&iommu M4U_PORT_VENC_REC_CHROMA_SET2>;
+			mediatek,vpu = <&vpu>;
+			clocks = <&topckgen CLK_TOP_VENCPLL_D2>,
+				 <&topckgen CLK_TOP_VENC_SEL>,
+				 <&topckgen CLK_TOP_UNIVPLL1_D2>,
+				 <&topckgen CLK_TOP_VENC_LT_SEL>;
+			clock-names = "vencpll_d2",
+				      "venc_sel",
+				      "univpll1_d2",
+				      "venc_lt_sel";
+		};
+
 		vencltsys: clock-controller@19000000 {
 			compatible = "mediatek,mt8173-vencltsys", "syscon";
 			reg = <0 0x19000000 0 0x1000>;
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 102+ messages in thread

* [PATCH v4 8/8] arm64: dts: mediatek: Add Video Encoder for MT8173
@ 2016-02-04 11:35                 ` Tiffany Lin
  0 siblings, 0 replies; 102+ messages in thread
From: Tiffany Lin @ 2016-02-04 11:35 UTC (permalink / raw)
  To: Hans Verkuil, daniel.thompson-QSEj5FYQhm4dnm+yROfE0A,
	Rob Herring, Mauro Carvalho Chehab, Matthias Brugger,
	Daniel Kurtz, Pawel Osciak
  Cc: Eddie Huang, Yingjoe Chen, devicetree-u79uwXL29TY76Z2rM5mHXA,
	linux-kernel-u79uwXL29TY76Z2rM5mHXA,
	linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
	linux-media-u79uwXL29TY76Z2rM5mHXA,
	linux-mediatek-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
	PoChun.Lin-NuS5LvNUpcJWk0Htik3J/w, Tiffany Lin

Add video encoder node for MT8173

Signed-off-by: Tiffany Lin <tiffany.lin-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
---
 arch/arm64/boot/dts/mediatek/mt8173.dtsi |   39 ++++++++++++++++++++++++++++++
 1 file changed, 39 insertions(+)

diff --git a/arch/arm64/boot/dts/mediatek/mt8173.dtsi b/arch/arm64/boot/dts/mediatek/mt8173.dtsi
index 5b0b38a..f61669d 100644
--- a/arch/arm64/boot/dts/mediatek/mt8173.dtsi
+++ b/arch/arm64/boot/dts/mediatek/mt8173.dtsi
@@ -1150,6 +1150,45 @@
 			clock-names = "apb", "smi";
 		};
 
+		vcodec_enc: vcodec@18002000 {
+			compatible = "mediatek,mt8173-vcodec-enc";
+			reg = <0 0x18002000 0 0x1000>,	/* VENC_SYS */
+			      <0 0x19002000 0 0x1000>;	/* VENC_LT_SYS */
+			interrupts = <GIC_SPI 198 IRQ_TYPE_LEVEL_LOW>,
+				     <GIC_SPI 202 IRQ_TYPE_LEVEL_LOW>;
+			mediatek,larb = <&larb3>,
+					<&larb5>;
+			iommus = <&iommu M4U_PORT_VENC_RCPU>,
+				 <&iommu M4U_PORT_VENC_REC>,
+				 <&iommu M4U_PORT_VENC_BSDMA>,
+				 <&iommu M4U_PORT_VENC_SV_COMV>,
+				 <&iommu M4U_PORT_VENC_RD_COMV>,
+				 <&iommu M4U_PORT_VENC_CUR_LUMA>,
+				 <&iommu M4U_PORT_VENC_CUR_CHROMA>,
+				 <&iommu M4U_PORT_VENC_REF_LUMA>,
+				 <&iommu M4U_PORT_VENC_REF_CHROMA>,
+				 <&iommu M4U_PORT_VENC_NBM_RDMA>,
+				 <&iommu M4U_PORT_VENC_NBM_WDMA>,
+				 <&iommu M4U_PORT_VENC_RCPU_SET2>,
+				 <&iommu M4U_PORT_VENC_REC_FRM_SET2>,
+				 <&iommu M4U_PORT_VENC_BSDMA_SET2>,
+				 <&iommu M4U_PORT_VENC_SV_COMA_SET2>,
+				 <&iommu M4U_PORT_VENC_RD_COMA_SET2>,
+				 <&iommu M4U_PORT_VENC_CUR_LUMA_SET2>,
+				 <&iommu M4U_PORT_VENC_CUR_CHROMA_SET2>,
+				 <&iommu M4U_PORT_VENC_REF_LUMA_SET2>,
+				 <&iommu M4U_PORT_VENC_REC_CHROMA_SET2>;
+			mediatek,vpu = <&vpu>;
+			clocks = <&topckgen CLK_TOP_VENCPLL_D2>,
+				 <&topckgen CLK_TOP_VENC_SEL>,
+				 <&topckgen CLK_TOP_UNIVPLL1_D2>,
+				 <&topckgen CLK_TOP_VENC_LT_SEL>;
+			clock-names = "vencpll_d2",
+				      "venc_sel",
+				      "univpll1_d2",
+				      "venc_lt_sel";
+		};
+
 		vencltsys: clock-controller@19000000 {
 			compatible = "mediatek,mt8173-vencltsys", "syscon";
 			reg = <0 0x19000000 0 0x1000>;
-- 
1.7.9.5

--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply related	[flat|nested] 102+ messages in thread

* [PATCH v4 8/8] arm64: dts: mediatek: Add Video Encoder for MT8173
@ 2016-02-04 11:35                 ` Tiffany Lin
  0 siblings, 0 replies; 102+ messages in thread
From: Tiffany Lin @ 2016-02-04 11:35 UTC (permalink / raw)
  To: linux-arm-kernel

Add video encoder node for MT8173

Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
---
 arch/arm64/boot/dts/mediatek/mt8173.dtsi |   39 ++++++++++++++++++++++++++++++
 1 file changed, 39 insertions(+)

diff --git a/arch/arm64/boot/dts/mediatek/mt8173.dtsi b/arch/arm64/boot/dts/mediatek/mt8173.dtsi
index 5b0b38a..f61669d 100644
--- a/arch/arm64/boot/dts/mediatek/mt8173.dtsi
+++ b/arch/arm64/boot/dts/mediatek/mt8173.dtsi
@@ -1150,6 +1150,45 @@
 			clock-names = "apb", "smi";
 		};
 
+		vcodec_enc: vcodec at 18002000 {
+			compatible = "mediatek,mt8173-vcodec-enc";
+			reg = <0 0x18002000 0 0x1000>,	/* VENC_SYS */
+			      <0 0x19002000 0 0x1000>;	/* VENC_LT_SYS */
+			interrupts = <GIC_SPI 198 IRQ_TYPE_LEVEL_LOW>,
+				     <GIC_SPI 202 IRQ_TYPE_LEVEL_LOW>;
+			mediatek,larb = <&larb3>,
+					<&larb5>;
+			iommus = <&iommu M4U_PORT_VENC_RCPU>,
+				 <&iommu M4U_PORT_VENC_REC>,
+				 <&iommu M4U_PORT_VENC_BSDMA>,
+				 <&iommu M4U_PORT_VENC_SV_COMV>,
+				 <&iommu M4U_PORT_VENC_RD_COMV>,
+				 <&iommu M4U_PORT_VENC_CUR_LUMA>,
+				 <&iommu M4U_PORT_VENC_CUR_CHROMA>,
+				 <&iommu M4U_PORT_VENC_REF_LUMA>,
+				 <&iommu M4U_PORT_VENC_REF_CHROMA>,
+				 <&iommu M4U_PORT_VENC_NBM_RDMA>,
+				 <&iommu M4U_PORT_VENC_NBM_WDMA>,
+				 <&iommu M4U_PORT_VENC_RCPU_SET2>,
+				 <&iommu M4U_PORT_VENC_REC_FRM_SET2>,
+				 <&iommu M4U_PORT_VENC_BSDMA_SET2>,
+				 <&iommu M4U_PORT_VENC_SV_COMA_SET2>,
+				 <&iommu M4U_PORT_VENC_RD_COMA_SET2>,
+				 <&iommu M4U_PORT_VENC_CUR_LUMA_SET2>,
+				 <&iommu M4U_PORT_VENC_CUR_CHROMA_SET2>,
+				 <&iommu M4U_PORT_VENC_REF_LUMA_SET2>,
+				 <&iommu M4U_PORT_VENC_REC_CHROMA_SET2>;
+			mediatek,vpu = <&vpu>;
+			clocks = <&topckgen CLK_TOP_VENCPLL_D2>,
+				 <&topckgen CLK_TOP_VENC_SEL>,
+				 <&topckgen CLK_TOP_UNIVPLL1_D2>,
+				 <&topckgen CLK_TOP_VENC_LT_SEL>;
+			clock-names = "vencpll_d2",
+				      "venc_sel",
+				      "univpll1_d2",
+				      "venc_lt_sel";
+		};
+
 		vencltsys: clock-controller at 19000000 {
 			compatible = "mediatek,mt8173-vencltsys", "syscon";
 			reg = <0 0x19000000 0 0x1000>;
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 102+ messages in thread

* Re: [PATCH v4 1/8] dt-bindings: Add a binding for Mediatek Video Processor
  2016-02-04 11:34   ` Tiffany Lin
@ 2016-02-08 18:54     ` Rob Herring
  -1 siblings, 0 replies; 102+ messages in thread
From: Rob Herring @ 2016-02-08 18:54 UTC (permalink / raw)
  To: Tiffany Lin
  Cc: Hans Verkuil, daniel.thompson, Mauro Carvalho Chehab,
	Matthias Brugger, Daniel Kurtz, Pawel Osciak, Eddie Huang,
	Yingjoe Chen, devicetree, linux-kernel, linux-arm-kernel,
	linux-media, linux-mediatek, PoChun.Lin, Andrew-CT Chen

On Thu, Feb 04, 2016 at 07:34:56PM +0800, Tiffany Lin wrote:
> From: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
> 
> Add a DT binding documentation of Video Processor Unit for the
> MT8173 SoC from Mediatek.
> 
> 
> Signed-off-by: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
> Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
> ---
>  .../devicetree/bindings/media/mediatek-vpu.txt     |   31 ++++++++++++++++++++
>  1 file changed, 31 insertions(+)
>  create mode 100644 Documentation/devicetree/bindings/media/mediatek-vpu.txt

Acked-by: Rob Herring <robh@kernel.org>

^ permalink raw reply	[flat|nested] 102+ messages in thread

* [PATCH v4 1/8] dt-bindings: Add a binding for Mediatek Video Processor
@ 2016-02-08 18:54     ` Rob Herring
  0 siblings, 0 replies; 102+ messages in thread
From: Rob Herring @ 2016-02-08 18:54 UTC (permalink / raw)
  To: linux-arm-kernel

On Thu, Feb 04, 2016 at 07:34:56PM +0800, Tiffany Lin wrote:
> From: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
> 
> Add a DT binding documentation of Video Processor Unit for the
> MT8173 SoC from Mediatek.
> 
> 
> Signed-off-by: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
> Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
> ---
>  .../devicetree/bindings/media/mediatek-vpu.txt     |   31 ++++++++++++++++++++
>  1 file changed, 31 insertions(+)
>  create mode 100644 Documentation/devicetree/bindings/media/mediatek-vpu.txt

Acked-by: Rob Herring <robh@kernel.org>

^ permalink raw reply	[flat|nested] 102+ messages in thread

* Re: [PATCH v4 4/8] dt-bindings: Add a binding for Mediatek Video Encoder
@ 2016-02-08 18:56           ` Rob Herring
  0 siblings, 0 replies; 102+ messages in thread
From: Rob Herring @ 2016-02-08 18:56 UTC (permalink / raw)
  To: Tiffany Lin
  Cc: Hans Verkuil, daniel.thompson, Mauro Carvalho Chehab,
	Matthias Brugger, Daniel Kurtz, Pawel Osciak, Eddie Huang,
	Yingjoe Chen, devicetree, linux-kernel, linux-arm-kernel,
	linux-media, linux-mediatek, PoChun.Lin

On Thu, Feb 04, 2016 at 07:34:59PM +0800, Tiffany Lin wrote:
> Add a DT binding documentation of Video Encoder for the
> MT8173 SoC from Mediatek.
> 
> Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
> ---
>  .../devicetree/bindings/media/mediatek-vcodec.txt  |   59 ++++++++++++++++++++
>  1 file changed, 59 insertions(+)
>  create mode 100644 Documentation/devicetree/bindings/media/mediatek-vcodec.txt

Acked-by: Rob Herring <robh@kernel.org>

^ permalink raw reply	[flat|nested] 102+ messages in thread

* Re: [PATCH v4 4/8] dt-bindings: Add a binding for Mediatek Video Encoder
@ 2016-02-08 18:56           ` Rob Herring
  0 siblings, 0 replies; 102+ messages in thread
From: Rob Herring @ 2016-02-08 18:56 UTC (permalink / raw)
  To: Tiffany Lin
  Cc: Hans Verkuil, daniel.thompson-QSEj5FYQhm4dnm+yROfE0A,
	Mauro Carvalho Chehab, Matthias Brugger, Daniel Kurtz,
	Pawel Osciak, Eddie Huang, Yingjoe Chen,
	devicetree-u79uwXL29TY76Z2rM5mHXA,
	linux-kernel-u79uwXL29TY76Z2rM5mHXA,
	linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
	linux-media-u79uwXL29TY76Z2rM5mHXA,
	linux-mediatek-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
	PoChun.Lin-NuS5LvNUpcJWk0Htik3J/w

On Thu, Feb 04, 2016 at 07:34:59PM +0800, Tiffany Lin wrote:
> Add a DT binding documentation of Video Encoder for the
> MT8173 SoC from Mediatek.
> 
> Signed-off-by: Tiffany Lin <tiffany.lin-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
> ---
>  .../devicetree/bindings/media/mediatek-vcodec.txt  |   59 ++++++++++++++++++++
>  1 file changed, 59 insertions(+)
>  create mode 100644 Documentation/devicetree/bindings/media/mediatek-vcodec.txt

Acked-by: Rob Herring <robh-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>

--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 102+ messages in thread

* [PATCH v4 4/8] dt-bindings: Add a binding for Mediatek Video Encoder
@ 2016-02-08 18:56           ` Rob Herring
  0 siblings, 0 replies; 102+ messages in thread
From: Rob Herring @ 2016-02-08 18:56 UTC (permalink / raw)
  To: linux-arm-kernel

On Thu, Feb 04, 2016 at 07:34:59PM +0800, Tiffany Lin wrote:
> Add a DT binding documentation of Video Encoder for the
> MT8173 SoC from Mediatek.
> 
> Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
> ---
>  .../devicetree/bindings/media/mediatek-vcodec.txt  |   59 ++++++++++++++++++++
>  1 file changed, 59 insertions(+)
>  create mode 100644 Documentation/devicetree/bindings/media/mediatek-vcodec.txt

Acked-by: Rob Herring <robh@kernel.org>

^ permalink raw reply	[flat|nested] 102+ messages in thread

* Re: [PATCH v4 4/8] dt-bindings: Add a binding for Mediatek Video Encoder
  2016-02-04 11:34         ` Tiffany Lin
  (?)
@ 2016-02-09 11:29           ` Daniel Kurtz
  -1 siblings, 0 replies; 102+ messages in thread
From: Daniel Kurtz @ 2016-02-09 11:29 UTC (permalink / raw)
  To: Tiffany Lin
  Cc: Hans Verkuil, daniel.thompson, Rob Herring,
	Mauro Carvalho Chehab, Matthias Brugger, Pawel Osciak,
	Eddie Huang, Yingjoe Chen, open list:OPEN FIRMWARE AND...,
	linux-kernel, linux-arm-kernel, linux-media,
	moderated list:ARM/Mediatek SoC support, Lin PoChun

Hi Tiffany,

On Thu, Feb 4, 2016 at 7:34 PM, Tiffany Lin <tiffany.lin@mediatek.com> wrote:
> Add a DT binding documentation of Video Encoder for the
> MT8173 SoC from Mediatek.
>
> Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
> ---
>  .../devicetree/bindings/media/mediatek-vcodec.txt  |   59 ++++++++++++++++++++
>  1 file changed, 59 insertions(+)
>  create mode 100644 Documentation/devicetree/bindings/media/mediatek-vcodec.txt
>
> diff --git a/Documentation/devicetree/bindings/media/mediatek-vcodec.txt b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
> new file mode 100644
> index 0000000..572bfdd
> --- /dev/null
> +++ b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
> @@ -0,0 +1,59 @@
> +Mediatek Video Codec
> +
> +Mediatek Video Codec is the video codec hw present in Mediatek SoCs which
> +supports high resolution encoding functionalities.
> +
> +Required properties:
> +- compatible : "mediatek,mt8173-vcodec-enc" for encoder
> +- reg : Physical base address of the video codec registers and length of
> +  memory mapped region.
> +- interrupts : interrupt number to the cpu.
> +- mediatek,larb : must contain the local arbiters in the current Socs.
> +- clocks : list of clock specifiers, corresponding to entries in
> +  the clock-names property.
> +- clock-names: encoder must contain "vencpll_d2", "venc_sel", "univpll1_d2",
> +  "venc_lt_sel".
> +- iommus : should point to the respective IOMMU block with master port as
> +  argument, see Documentation/devicetree/bindings/iommu/mediatek,iommu.txt
> +  for details.
> +- mediatek,vpu : the node of video processor unit
> +
> +Example:
> +vcodec_enc: vcodec@0x18002000 {
> +    compatible = "mediatek,mt8173-vcodec-enc";
> +    reg = <0 0x18002000 0 0x1000>,    /*VENC_SYS*/
> +          <0 0x19002000 0 0x1000>;    /*VENC_LT_SYS*/

This really looks like two encoder devices combined into a single
device tree node.
There are two register sets, two irqs, two sets of iommus, and two
sets of clocks.

If possible, please split this node into two, one for each encoder.


> +    interrupts = <GIC_SPI 198 IRQ_TYPE_LEVEL_LOW>,
> +           <GIC_SPI 202 IRQ_TYPE_LEVEL_LOW>;
> +    mediatek,larb = <&larb3>,
> +                   <&larb5>;
> +    iommus = <&iommu M4U_PORT_VENC_RCPU>,
> +             <&iommu M4U_PORT_VENC_REC>,
> +             <&iommu M4U_PORT_VENC_BSDMA>,
> +             <&iommu M4U_PORT_VENC_SV_COMV>,
> +             <&iommu M4U_PORT_VENC_RD_COMV>,
> +             <&iommu M4U_PORT_VENC_CUR_LUMA>,
> +             <&iommu M4U_PORT_VENC_CUR_CHROMA>,
> +             <&iommu M4U_PORT_VENC_REF_LUMA>,
> +             <&iommu M4U_PORT_VENC_REF_CHROMA>,
> +             <&iommu M4U_PORT_VENC_NBM_RDMA>,
> +             <&iommu M4U_PORT_VENC_NBM_WDMA>,
> +             <&iommu M4U_PORT_VENC_RCPU_SET2>,
> +             <&iommu M4U_PORT_VENC_REC_FRM_SET2>,
> +             <&iommu M4U_PORT_VENC_BSDMA_SET2>,
> +             <&iommu M4U_PORT_VENC_SV_COMA_SET2>,
> +             <&iommu M4U_PORT_VENC_RD_COMA_SET2>,
> +             <&iommu M4U_PORT_VENC_CUR_LUMA_SET2>,
> +             <&iommu M4U_PORT_VENC_CUR_CHROMA_SET2>,
> +             <&iommu M4U_PORT_VENC_REF_LUMA_SET2>,
> +             <&iommu M4U_PORT_VENC_REC_CHROMA_SET2>;
> +    mediatek,vpu = <&vpu>;
> +    clocks = <&topckgen CLK_TOP_VENCPLL_D2>,
> +             <&topckgen CLK_TOP_VENC_SEL>,
> +             <&topckgen CLK_TOP_UNIVPLL1_D2>,
> +             <&topckgen CLK_TOP_VENC_LT_SEL>;
> +    clock-names = "vencpll_d2",
> +                  "venc_sel",
> +                  "univpll1_d2",
> +                  "venc_lt_sel";

The names of these clocks should be from the perspective of the
encoder, not the clock provider.

-Dan

> +  };
> --
> 1.7.9.5
>

^ permalink raw reply	[flat|nested] 102+ messages in thread

* Re: [PATCH v4 4/8] dt-bindings: Add a binding for Mediatek Video Encoder
@ 2016-02-09 11:29           ` Daniel Kurtz
  0 siblings, 0 replies; 102+ messages in thread
From: Daniel Kurtz @ 2016-02-09 11:29 UTC (permalink / raw)
  To: Tiffany Lin
  Cc: Hans Verkuil, daniel.thompson, Rob Herring,
	Mauro Carvalho Chehab, Matthias Brugger, Pawel Osciak,
	Eddie Huang, Yingjoe Chen, open list:OPEN FIRMWARE AND...,
	linux-kernel, linux-arm-kernel, linux-media,
	moderated list:ARM/Mediatek SoC support, Lin PoChun

Hi Tiffany,

On Thu, Feb 4, 2016 at 7:34 PM, Tiffany Lin <tiffany.lin@mediatek.com> wrote:
> Add a DT binding documentation of Video Encoder for the
> MT8173 SoC from Mediatek.
>
> Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
> ---
>  .../devicetree/bindings/media/mediatek-vcodec.txt  |   59 ++++++++++++++++++++
>  1 file changed, 59 insertions(+)
>  create mode 100644 Documentation/devicetree/bindings/media/mediatek-vcodec.txt
>
> diff --git a/Documentation/devicetree/bindings/media/mediatek-vcodec.txt b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
> new file mode 100644
> index 0000000..572bfdd
> --- /dev/null
> +++ b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
> @@ -0,0 +1,59 @@
> +Mediatek Video Codec
> +
> +Mediatek Video Codec is the video codec hw present in Mediatek SoCs which
> +supports high resolution encoding functionalities.
> +
> +Required properties:
> +- compatible : "mediatek,mt8173-vcodec-enc" for encoder
> +- reg : Physical base address of the video codec registers and length of
> +  memory mapped region.
> +- interrupts : interrupt number to the cpu.
> +- mediatek,larb : must contain the local arbiters in the current Socs.
> +- clocks : list of clock specifiers, corresponding to entries in
> +  the clock-names property.
> +- clock-names: encoder must contain "vencpll_d2", "venc_sel", "univpll1_d2",
> +  "venc_lt_sel".
> +- iommus : should point to the respective IOMMU block with master port as
> +  argument, see Documentation/devicetree/bindings/iommu/mediatek,iommu.txt
> +  for details.
> +- mediatek,vpu : the node of video processor unit
> +
> +Example:
> +vcodec_enc: vcodec@0x18002000 {
> +    compatible = "mediatek,mt8173-vcodec-enc";
> +    reg = <0 0x18002000 0 0x1000>,    /*VENC_SYS*/
> +          <0 0x19002000 0 0x1000>;    /*VENC_LT_SYS*/

This really looks like two encoder devices combined into a single
device tree node.
There are two register sets, two irqs, two sets of iommus, and two
sets of clocks.

If possible, please split this node into two, one for each encoder.


> +    interrupts = <GIC_SPI 198 IRQ_TYPE_LEVEL_LOW>,
> +           <GIC_SPI 202 IRQ_TYPE_LEVEL_LOW>;
> +    mediatek,larb = <&larb3>,
> +                   <&larb5>;
> +    iommus = <&iommu M4U_PORT_VENC_RCPU>,
> +             <&iommu M4U_PORT_VENC_REC>,
> +             <&iommu M4U_PORT_VENC_BSDMA>,
> +             <&iommu M4U_PORT_VENC_SV_COMV>,
> +             <&iommu M4U_PORT_VENC_RD_COMV>,
> +             <&iommu M4U_PORT_VENC_CUR_LUMA>,
> +             <&iommu M4U_PORT_VENC_CUR_CHROMA>,
> +             <&iommu M4U_PORT_VENC_REF_LUMA>,
> +             <&iommu M4U_PORT_VENC_REF_CHROMA>,
> +             <&iommu M4U_PORT_VENC_NBM_RDMA>,
> +             <&iommu M4U_PORT_VENC_NBM_WDMA>,
> +             <&iommu M4U_PORT_VENC_RCPU_SET2>,
> +             <&iommu M4U_PORT_VENC_REC_FRM_SET2>,
> +             <&iommu M4U_PORT_VENC_BSDMA_SET2>,
> +             <&iommu M4U_PORT_VENC_SV_COMA_SET2>,
> +             <&iommu M4U_PORT_VENC_RD_COMA_SET2>,
> +             <&iommu M4U_PORT_VENC_CUR_LUMA_SET2>,
> +             <&iommu M4U_PORT_VENC_CUR_CHROMA_SET2>,
> +             <&iommu M4U_PORT_VENC_REF_LUMA_SET2>,
> +             <&iommu M4U_PORT_VENC_REC_CHROMA_SET2>;
> +    mediatek,vpu = <&vpu>;
> +    clocks = <&topckgen CLK_TOP_VENCPLL_D2>,
> +             <&topckgen CLK_TOP_VENC_SEL>,
> +             <&topckgen CLK_TOP_UNIVPLL1_D2>,
> +             <&topckgen CLK_TOP_VENC_LT_SEL>;
> +    clock-names = "vencpll_d2",
> +                  "venc_sel",
> +                  "univpll1_d2",
> +                  "venc_lt_sel";

The names of these clocks should be from the perspective of the
encoder, not the clock provider.

-Dan

> +  };
> --
> 1.7.9.5
>

^ permalink raw reply	[flat|nested] 102+ messages in thread

* [PATCH v4 4/8] dt-bindings: Add a binding for Mediatek Video Encoder
@ 2016-02-09 11:29           ` Daniel Kurtz
  0 siblings, 0 replies; 102+ messages in thread
From: Daniel Kurtz @ 2016-02-09 11:29 UTC (permalink / raw)
  To: linux-arm-kernel

Hi Tiffany,

On Thu, Feb 4, 2016 at 7:34 PM, Tiffany Lin <tiffany.lin@mediatek.com> wrote:
> Add a DT binding documentation of Video Encoder for the
> MT8173 SoC from Mediatek.
>
> Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
> ---
>  .../devicetree/bindings/media/mediatek-vcodec.txt  |   59 ++++++++++++++++++++
>  1 file changed, 59 insertions(+)
>  create mode 100644 Documentation/devicetree/bindings/media/mediatek-vcodec.txt
>
> diff --git a/Documentation/devicetree/bindings/media/mediatek-vcodec.txt b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
> new file mode 100644
> index 0000000..572bfdd
> --- /dev/null
> +++ b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
> @@ -0,0 +1,59 @@
> +Mediatek Video Codec
> +
> +Mediatek Video Codec is the video codec hw present in Mediatek SoCs which
> +supports high resolution encoding functionalities.
> +
> +Required properties:
> +- compatible : "mediatek,mt8173-vcodec-enc" for encoder
> +- reg : Physical base address of the video codec registers and length of
> +  memory mapped region.
> +- interrupts : interrupt number to the cpu.
> +- mediatek,larb : must contain the local arbiters in the current Socs.
> +- clocks : list of clock specifiers, corresponding to entries in
> +  the clock-names property.
> +- clock-names: encoder must contain "vencpll_d2", "venc_sel", "univpll1_d2",
> +  "venc_lt_sel".
> +- iommus : should point to the respective IOMMU block with master port as
> +  argument, see Documentation/devicetree/bindings/iommu/mediatek,iommu.txt
> +  for details.
> +- mediatek,vpu : the node of video processor unit
> +
> +Example:
> +vcodec_enc: vcodec at 0x18002000 {
> +    compatible = "mediatek,mt8173-vcodec-enc";
> +    reg = <0 0x18002000 0 0x1000>,    /*VENC_SYS*/
> +          <0 0x19002000 0 0x1000>;    /*VENC_LT_SYS*/

This really looks like two encoder devices combined into a single
device tree node.
There are two register sets, two irqs, two sets of iommus, and two
sets of clocks.

If possible, please split this node into two, one for each encoder.


> +    interrupts = <GIC_SPI 198 IRQ_TYPE_LEVEL_LOW>,
> +           <GIC_SPI 202 IRQ_TYPE_LEVEL_LOW>;
> +    mediatek,larb = <&larb3>,
> +                   <&larb5>;
> +    iommus = <&iommu M4U_PORT_VENC_RCPU>,
> +             <&iommu M4U_PORT_VENC_REC>,
> +             <&iommu M4U_PORT_VENC_BSDMA>,
> +             <&iommu M4U_PORT_VENC_SV_COMV>,
> +             <&iommu M4U_PORT_VENC_RD_COMV>,
> +             <&iommu M4U_PORT_VENC_CUR_LUMA>,
> +             <&iommu M4U_PORT_VENC_CUR_CHROMA>,
> +             <&iommu M4U_PORT_VENC_REF_LUMA>,
> +             <&iommu M4U_PORT_VENC_REF_CHROMA>,
> +             <&iommu M4U_PORT_VENC_NBM_RDMA>,
> +             <&iommu M4U_PORT_VENC_NBM_WDMA>,
> +             <&iommu M4U_PORT_VENC_RCPU_SET2>,
> +             <&iommu M4U_PORT_VENC_REC_FRM_SET2>,
> +             <&iommu M4U_PORT_VENC_BSDMA_SET2>,
> +             <&iommu M4U_PORT_VENC_SV_COMA_SET2>,
> +             <&iommu M4U_PORT_VENC_RD_COMA_SET2>,
> +             <&iommu M4U_PORT_VENC_CUR_LUMA_SET2>,
> +             <&iommu M4U_PORT_VENC_CUR_CHROMA_SET2>,
> +             <&iommu M4U_PORT_VENC_REF_LUMA_SET2>,
> +             <&iommu M4U_PORT_VENC_REC_CHROMA_SET2>;
> +    mediatek,vpu = <&vpu>;
> +    clocks = <&topckgen CLK_TOP_VENCPLL_D2>,
> +             <&topckgen CLK_TOP_VENC_SEL>,
> +             <&topckgen CLK_TOP_UNIVPLL1_D2>,
> +             <&topckgen CLK_TOP_VENC_LT_SEL>;
> +    clock-names = "vencpll_d2",
> +                  "venc_sel",
> +                  "univpll1_d2",
> +                  "venc_lt_sel";

The names of these clocks should be from the perspective of the
encoder, not the clock provider.

-Dan

> +  };
> --
> 1.7.9.5
>

^ permalink raw reply	[flat|nested] 102+ messages in thread

* Re: [PATCH v4 2/8] [media] VPU: mediatek: support Mediatek VPU
  2016-02-04 11:34     ` Tiffany Lin
@ 2016-02-15 10:07       ` Hans Verkuil
  -1 siblings, 0 replies; 102+ messages in thread
From: Hans Verkuil @ 2016-02-15 10:07 UTC (permalink / raw)
  To: Tiffany Lin, Hans Verkuil, daniel.thompson, Rob Herring,
	Mauro Carvalho Chehab, Matthias Brugger, Daniel Kurtz,
	Pawel Osciak
  Cc: Eddie Huang, Yingjoe Chen, devicetree, linux-kernel,
	linux-arm-kernel, linux-media, linux-mediatek, PoChun.Lin,
	Andrew-CT Chen

Hi Tiffany,

A small review comment below:

On 02/04/2016 12:34 PM, Tiffany Lin wrote:
> The VPU driver for hw video codec embedded in Mediatek's MT8173 SOCs.
> It is able to handle video decoding/encoding of in a range of formats.
> The driver provides with VPU firmware download, memory management and
> the communication interface between CPU and VPU.
> For VPU initialization, it will create virtual memory for CPU access and
> IOMMU address for vcodec hw device access. When a decode/encode instance
> opens a device node, vpu driver will download vpu firmware to the device.
> A decode/encode instant will decode/encode a frame using VPU
> interface to interrupt vpu to handle decoding/encoding jobs.
> 
> Signed-off-by: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
> Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
> ---
>  drivers/media/platform/Kconfig           |    9 +
>  drivers/media/platform/Makefile          |    2 +
>  drivers/media/platform/mtk-vpu/Makefile  |    1 +
>  drivers/media/platform/mtk-vpu/mtk_vpu.c |  994 ++++++++++++++++++++++++++++++
>  drivers/media/platform/mtk-vpu/mtk_vpu.h |  167 +++++
>  5 files changed, 1173 insertions(+)
>  create mode 100644 drivers/media/platform/mtk-vpu/Makefile
>  create mode 100644 drivers/media/platform/mtk-vpu/mtk_vpu.c
>  create mode 100644 drivers/media/platform/mtk-vpu/mtk_vpu.h
> 

<snip>

> diff --git a/drivers/media/platform/mtk-vpu/mtk_vpu.c b/drivers/media/platform/mtk-vpu/mtk_vpu.c
> new file mode 100644
> index 0000000..f54fd89
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vpu/mtk_vpu.c
> @@ -0,0 +1,994 @@

<snip>

> +static ssize_t vpu_debug_read(struct file *file, char __user *user_buf,
> +			      size_t count, loff_t *ppos)
> +{
> +	char buf[256];
> +	unsigned int len;
> +	unsigned int running, pc, vpu_to_host, host_to_vpu, wdt;
> +	int ret;
> +	struct device *dev = file->private_data;
> +	struct mtk_vpu *vpu = dev_get_drvdata(dev);
> +
> +	ret = vpu_clock_enable(vpu);
> +	if (ret) {
> +		dev_err(vpu->dev, "[VPU] enable clock failed %d\n", ret);
> +		return 0;
> +	}
> +
> +	/* vpu register status */
> +	running = vpu_running(vpu);
> +	pc = vpu_cfg_readl(vpu, VPU_PC_REG);
> +	wdt = vpu_cfg_readl(vpu, VPU_WDT_REG);
> +	host_to_vpu = vpu_cfg_readl(vpu, HOST_TO_VPU);
> +	vpu_to_host = vpu_cfg_readl(vpu, VPU_TO_HOST);
> +	vpu_clock_disable(vpu);
> +
> +	if (running) {
> +		len = sprintf(buf, "VPU is running\n\n"

Please use snprintf.

> +		"FW Version: %s\n"
> +		"PC: 0x%x\n"
> +		"WDT: 0x%x\n"
> +		"Host to VPU: 0x%x\n"
> +		"VPU to Host: 0x%x\n",
> +		vpu->run.fw_ver, pc, wdt,
> +		host_to_vpu, vpu_to_host);
> +	} else {
> +		len = sprintf(buf, "VPU not running\n");

Ditto.

sprintf always makes me feel uncomfortable :-)

> +	}
> +
> +	return simple_read_from_buffer(user_buf, count, ppos, buf, len);
> +}

Regards,

	Hans

^ permalink raw reply	[flat|nested] 102+ messages in thread

* [PATCH v4 2/8] [media] VPU: mediatek: support Mediatek VPU
@ 2016-02-15 10:07       ` Hans Verkuil
  0 siblings, 0 replies; 102+ messages in thread
From: Hans Verkuil @ 2016-02-15 10:07 UTC (permalink / raw)
  To: linux-arm-kernel

Hi Tiffany,

A small review comment below:

On 02/04/2016 12:34 PM, Tiffany Lin wrote:
> The VPU driver for hw video codec embedded in Mediatek's MT8173 SOCs.
> It is able to handle video decoding/encoding of in a range of formats.
> The driver provides with VPU firmware download, memory management and
> the communication interface between CPU and VPU.
> For VPU initialization, it will create virtual memory for CPU access and
> IOMMU address for vcodec hw device access. When a decode/encode instance
> opens a device node, vpu driver will download vpu firmware to the device.
> A decode/encode instant will decode/encode a frame using VPU
> interface to interrupt vpu to handle decoding/encoding jobs.
> 
> Signed-off-by: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
> Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
> ---
>  drivers/media/platform/Kconfig           |    9 +
>  drivers/media/platform/Makefile          |    2 +
>  drivers/media/platform/mtk-vpu/Makefile  |    1 +
>  drivers/media/platform/mtk-vpu/mtk_vpu.c |  994 ++++++++++++++++++++++++++++++
>  drivers/media/platform/mtk-vpu/mtk_vpu.h |  167 +++++
>  5 files changed, 1173 insertions(+)
>  create mode 100644 drivers/media/platform/mtk-vpu/Makefile
>  create mode 100644 drivers/media/platform/mtk-vpu/mtk_vpu.c
>  create mode 100644 drivers/media/platform/mtk-vpu/mtk_vpu.h
> 

<snip>

> diff --git a/drivers/media/platform/mtk-vpu/mtk_vpu.c b/drivers/media/platform/mtk-vpu/mtk_vpu.c
> new file mode 100644
> index 0000000..f54fd89
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vpu/mtk_vpu.c
> @@ -0,0 +1,994 @@

<snip>

> +static ssize_t vpu_debug_read(struct file *file, char __user *user_buf,
> +			      size_t count, loff_t *ppos)
> +{
> +	char buf[256];
> +	unsigned int len;
> +	unsigned int running, pc, vpu_to_host, host_to_vpu, wdt;
> +	int ret;
> +	struct device *dev = file->private_data;
> +	struct mtk_vpu *vpu = dev_get_drvdata(dev);
> +
> +	ret = vpu_clock_enable(vpu);
> +	if (ret) {
> +		dev_err(vpu->dev, "[VPU] enable clock failed %d\n", ret);
> +		return 0;
> +	}
> +
> +	/* vpu register status */
> +	running = vpu_running(vpu);
> +	pc = vpu_cfg_readl(vpu, VPU_PC_REG);
> +	wdt = vpu_cfg_readl(vpu, VPU_WDT_REG);
> +	host_to_vpu = vpu_cfg_readl(vpu, HOST_TO_VPU);
> +	vpu_to_host = vpu_cfg_readl(vpu, VPU_TO_HOST);
> +	vpu_clock_disable(vpu);
> +
> +	if (running) {
> +		len = sprintf(buf, "VPU is running\n\n"

Please use snprintf.

> +		"FW Version: %s\n"
> +		"PC: 0x%x\n"
> +		"WDT: 0x%x\n"
> +		"Host to VPU: 0x%x\n"
> +		"VPU to Host: 0x%x\n",
> +		vpu->run.fw_ver, pc, wdt,
> +		host_to_vpu, vpu_to_host);
> +	} else {
> +		len = sprintf(buf, "VPU not running\n");

Ditto.

sprintf always makes me feel uncomfortable :-)

> +	}
> +
> +	return simple_read_from_buffer(user_buf, count, ppos, buf, len);
> +}

Regards,

	Hans

^ permalink raw reply	[flat|nested] 102+ messages in thread

* Re: [PATCH v4 2/8] [media] VPU: mediatek: support Mediatek VPU
  2016-02-04 11:34     ` Tiffany Lin
@ 2016-02-15 10:13       ` Hans Verkuil
  -1 siblings, 0 replies; 102+ messages in thread
From: Hans Verkuil @ 2016-02-15 10:13 UTC (permalink / raw)
  To: Tiffany Lin, Hans Verkuil, daniel.thompson, Rob Herring,
	Mauro Carvalho Chehab, Matthias Brugger, Daniel Kurtz,
	Pawel Osciak
  Cc: Eddie Huang, Yingjoe Chen, devicetree, linux-kernel,
	linux-arm-kernel, linux-media, linux-mediatek, PoChun.Lin,
	Andrew-CT Chen

On 02/04/2016 12:34 PM, Tiffany Lin wrote:
> The VPU driver for hw video codec embedded in Mediatek's MT8173 SOCs.
> It is able to handle video decoding/encoding of in a range of formats.
> The driver provides with VPU firmware download, memory management and
> the communication interface between CPU and VPU.
> For VPU initialization, it will create virtual memory for CPU access and
> IOMMU address for vcodec hw device access. When a decode/encode instance
> opens a device node, vpu driver will download vpu firmware to the device.
> A decode/encode instant will decode/encode a frame using VPU
> interface to interrupt vpu to handle decoding/encoding jobs.
> 
> Signed-off-by: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
> Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
> ---
>  drivers/media/platform/Kconfig           |    9 +
>  drivers/media/platform/Makefile          |    2 +
>  drivers/media/platform/mtk-vpu/Makefile  |    1 +
>  drivers/media/platform/mtk-vpu/mtk_vpu.c |  994 ++++++++++++++++++++++++++++++
>  drivers/media/platform/mtk-vpu/mtk_vpu.h |  167 +++++
>  5 files changed, 1173 insertions(+)
>  create mode 100644 drivers/media/platform/mtk-vpu/Makefile
>  create mode 100644 drivers/media/platform/mtk-vpu/mtk_vpu.c
>  create mode 100644 drivers/media/platform/mtk-vpu/mtk_vpu.h
> 
> diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
> index ccbc974..ba812d6 100644
> --- a/drivers/media/platform/Kconfig
> +++ b/drivers/media/platform/Kconfig
> @@ -148,6 +148,15 @@ config VIDEO_CODA
>  	   Coda is a range of video codec IPs that supports
>  	   H.264, MPEG-4, and other video formats.
>  
> +config VIDEO_MEDIATEK_VPU
> +	tristate "Mediatek Video Processor Unit"
> +	depends on VIDEO_DEV && VIDEO_V4L2 && ARCH_MEDIATEK
> +	---help---
> +	    This driver provides downloading VPU firmware and
> +	    communicating with VPU. This driver for hw video
> +	    codec embedded in new Mediatek's SOCs. It is able
> +	    to handle video decoding/encoding in a range of formats.

Can you be more specific in this text and mention for which Mediatek SoCs
this driver is for? Just like you did in the commit log.

Also add something like this:

          To compile this driver as a module, choose M here: the module
          will be called mtk-vpu.

I always find it useful if the Kconfig text mentions the module name.

Regards,

	Hans

> +
>  config VIDEO_MEM2MEM_DEINTERLACE
>  	tristate "Deinterlace support"
>  	depends on VIDEO_DEV && VIDEO_V4L2 && DMA_ENGINE

^ permalink raw reply	[flat|nested] 102+ messages in thread

* [PATCH v4 2/8] [media] VPU: mediatek: support Mediatek VPU
@ 2016-02-15 10:13       ` Hans Verkuil
  0 siblings, 0 replies; 102+ messages in thread
From: Hans Verkuil @ 2016-02-15 10:13 UTC (permalink / raw)
  To: linux-arm-kernel

On 02/04/2016 12:34 PM, Tiffany Lin wrote:
> The VPU driver for hw video codec embedded in Mediatek's MT8173 SOCs.
> It is able to handle video decoding/encoding of in a range of formats.
> The driver provides with VPU firmware download, memory management and
> the communication interface between CPU and VPU.
> For VPU initialization, it will create virtual memory for CPU access and
> IOMMU address for vcodec hw device access. When a decode/encode instance
> opens a device node, vpu driver will download vpu firmware to the device.
> A decode/encode instant will decode/encode a frame using VPU
> interface to interrupt vpu to handle decoding/encoding jobs.
> 
> Signed-off-by: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
> Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
> ---
>  drivers/media/platform/Kconfig           |    9 +
>  drivers/media/platform/Makefile          |    2 +
>  drivers/media/platform/mtk-vpu/Makefile  |    1 +
>  drivers/media/platform/mtk-vpu/mtk_vpu.c |  994 ++++++++++++++++++++++++++++++
>  drivers/media/platform/mtk-vpu/mtk_vpu.h |  167 +++++
>  5 files changed, 1173 insertions(+)
>  create mode 100644 drivers/media/platform/mtk-vpu/Makefile
>  create mode 100644 drivers/media/platform/mtk-vpu/mtk_vpu.c
>  create mode 100644 drivers/media/platform/mtk-vpu/mtk_vpu.h
> 
> diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
> index ccbc974..ba812d6 100644
> --- a/drivers/media/platform/Kconfig
> +++ b/drivers/media/platform/Kconfig
> @@ -148,6 +148,15 @@ config VIDEO_CODA
>  	   Coda is a range of video codec IPs that supports
>  	   H.264, MPEG-4, and other video formats.
>  
> +config VIDEO_MEDIATEK_VPU
> +	tristate "Mediatek Video Processor Unit"
> +	depends on VIDEO_DEV && VIDEO_V4L2 && ARCH_MEDIATEK
> +	---help---
> +	    This driver provides downloading VPU firmware and
> +	    communicating with VPU. This driver for hw video
> +	    codec embedded in new Mediatek's SOCs. It is able
> +	    to handle video decoding/encoding in a range of formats.

Can you be more specific in this text and mention for which Mediatek SoCs
this driver is for? Just like you did in the commit log.

Also add something like this:

          To compile this driver as a module, choose M here: the module
          will be called mtk-vpu.

I always find it useful if the Kconfig text mentions the module name.

Regards,

	Hans

> +
>  config VIDEO_MEM2MEM_DEINTERLACE
>  	tristate "Deinterlace support"
>  	depends on VIDEO_DEV && VIDEO_V4L2 && DMA_ENGINE

^ permalink raw reply	[flat|nested] 102+ messages in thread

* Re: [PATCH v4 4/8] dt-bindings: Add a binding for Mediatek Video Encoder
  2016-02-09 11:29           ` Daniel Kurtz
  (?)
@ 2016-02-15 10:42             ` Daniel Kurtz
  -1 siblings, 0 replies; 102+ messages in thread
From: Daniel Kurtz @ 2016-02-15 10:42 UTC (permalink / raw)
  To: Tiffany Lin
  Cc: Hans Verkuil, Daniel Thompson, Rob Herring,
	Mauro Carvalho Chehab, Matthias Brugger, Pawel Osciak,
	Eddie Huang, Yingjoe Chen, open list:OPEN FIRMWARE AND...,
	linux-kernel, linux-arm-kernel, linux-media,
	moderated list:ARM/Mediatek SoC support, Lin PoChun, Tomasz Figa

On Tue, Feb 9, 2016 at 7:29 PM, Daniel Kurtz <djkurtz@chromium.org> wrote:
> Hi Tiffany,
>
> On Thu, Feb 4, 2016 at 7:34 PM, Tiffany Lin <tiffany.lin@mediatek.com> wrote:
>> Add a DT binding documentation of Video Encoder for the
>> MT8173 SoC from Mediatek.
>>
>> Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
>> ---
>>  .../devicetree/bindings/media/mediatek-vcodec.txt  |   59 ++++++++++++++++++++
>>  1 file changed, 59 insertions(+)
>>  create mode 100644 Documentation/devicetree/bindings/media/mediatek-vcodec.txt
>>
>> diff --git a/Documentation/devicetree/bindings/media/mediatek-vcodec.txt b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
>> new file mode 100644
>> index 0000000..572bfdd
>> --- /dev/null
>> +++ b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
>> @@ -0,0 +1,59 @@
>> +Mediatek Video Codec
>> +
>> +Mediatek Video Codec is the video codec hw present in Mediatek SoCs which
>> +supports high resolution encoding functionalities.
>> +
>> +Required properties:
>> +- compatible : "mediatek,mt8173-vcodec-enc" for encoder
>> +- reg : Physical base address of the video codec registers and length of
>> +  memory mapped region.
>> +- interrupts : interrupt number to the cpu.
>> +- mediatek,larb : must contain the local arbiters in the current Socs.
>> +- clocks : list of clock specifiers, corresponding to entries in
>> +  the clock-names property.
>> +- clock-names: encoder must contain "vencpll_d2", "venc_sel", "univpll1_d2",
>> +  "venc_lt_sel".
>> +- iommus : should point to the respective IOMMU block with master port as
>> +  argument, see Documentation/devicetree/bindings/iommu/mediatek,iommu.txt
>> +  for details.
>> +- mediatek,vpu : the node of video processor unit
>> +
>> +Example:
>> +vcodec_enc: vcodec@0x18002000 {
>> +    compatible = "mediatek,mt8173-vcodec-enc";
>> +    reg = <0 0x18002000 0 0x1000>,    /*VENC_SYS*/
>> +          <0 0x19002000 0 0x1000>;    /*VENC_LT_SYS*/
>
> This really looks like two encoder devices combined into a single
> device tree node.
> There are two register sets, two irqs, two sets of iommus, and two
> sets of clocks.
>
> If possible, please split this node into two, one for each encoder.

I chatted offline with Mediatek.  They explained that there really is
just one encoder hardware, that happens to support multiple formats.
The encoder cannot encode with both formats at the same time.  The
Mediatek HW designers added a new format to an existing encoder by
adding a second interface (register set, irq, iommus, clocks) without
modifying the original interface.  However in the hardware itself
there is really just one encoder device.

So, although this node looks like it is for two encoder devices (one
for each format), really there is just one device that supports each
format through its large interface.

So, I'm fine with this being a single device node.

>> +    interrupts = <GIC_SPI 198 IRQ_TYPE_LEVEL_LOW>,
>> +           <GIC_SPI 202 IRQ_TYPE_LEVEL_LOW>;
>> +    mediatek,larb = <&larb3>,
>> +                   <&larb5>;
>> +    iommus = <&iommu M4U_PORT_VENC_RCPU>,
>> +             <&iommu M4U_PORT_VENC_REC>,
>> +             <&iommu M4U_PORT_VENC_BSDMA>,
>> +             <&iommu M4U_PORT_VENC_SV_COMV>,
>> +             <&iommu M4U_PORT_VENC_RD_COMV>,
>> +             <&iommu M4U_PORT_VENC_CUR_LUMA>,
>> +             <&iommu M4U_PORT_VENC_CUR_CHROMA>,
>> +             <&iommu M4U_PORT_VENC_REF_LUMA>,
>> +             <&iommu M4U_PORT_VENC_REF_CHROMA>,
>> +             <&iommu M4U_PORT_VENC_NBM_RDMA>,
>> +             <&iommu M4U_PORT_VENC_NBM_WDMA>,
>> +             <&iommu M4U_PORT_VENC_RCPU_SET2>,
>> +             <&iommu M4U_PORT_VENC_REC_FRM_SET2>,
>> +             <&iommu M4U_PORT_VENC_BSDMA_SET2>,
>> +             <&iommu M4U_PORT_VENC_SV_COMA_SET2>,
>> +             <&iommu M4U_PORT_VENC_RD_COMA_SET2>,
>> +             <&iommu M4U_PORT_VENC_CUR_LUMA_SET2>,
>> +             <&iommu M4U_PORT_VENC_CUR_CHROMA_SET2>,
>> +             <&iommu M4U_PORT_VENC_REF_LUMA_SET2>,
>> +             <&iommu M4U_PORT_VENC_REC_CHROMA_SET2>;
>> +    mediatek,vpu = <&vpu>;
>> +    clocks = <&topckgen CLK_TOP_VENCPLL_D2>,
>> +             <&topckgen CLK_TOP_VENC_SEL>,
>> +             <&topckgen CLK_TOP_UNIVPLL1_D2>,
>> +             <&topckgen CLK_TOP_VENC_LT_SEL>;
>> +    clock-names = "vencpll_d2",
>> +                  "venc_sel",
>> +                  "univpll1_d2",
>> +                  "venc_lt_sel";
>
> The names of these clocks should be from the perspective of the
> encoder, not the clock provider.

I still think these clock names should be updated, however.

-Dan

^ permalink raw reply	[flat|nested] 102+ messages in thread

* Re: [PATCH v4 4/8] dt-bindings: Add a binding for Mediatek Video Encoder
@ 2016-02-15 10:42             ` Daniel Kurtz
  0 siblings, 0 replies; 102+ messages in thread
From: Daniel Kurtz @ 2016-02-15 10:42 UTC (permalink / raw)
  To: Tiffany Lin
  Cc: Hans Verkuil, Daniel Thompson, Rob Herring,
	Mauro Carvalho Chehab, Matthias Brugger, Pawel Osciak,
	Eddie Huang, Yingjoe Chen, open list:OPEN FIRMWARE AND...,
	linux-kernel, linux-arm-kernel, linux-media,
	moderated list:ARM/Mediatek SoC support, Lin PoChun, Tomasz Figa

On Tue, Feb 9, 2016 at 7:29 PM, Daniel Kurtz <djkurtz@chromium.org> wrote:
> Hi Tiffany,
>
> On Thu, Feb 4, 2016 at 7:34 PM, Tiffany Lin <tiffany.lin@mediatek.com> wrote:
>> Add a DT binding documentation of Video Encoder for the
>> MT8173 SoC from Mediatek.
>>
>> Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
>> ---
>>  .../devicetree/bindings/media/mediatek-vcodec.txt  |   59 ++++++++++++++++++++
>>  1 file changed, 59 insertions(+)
>>  create mode 100644 Documentation/devicetree/bindings/media/mediatek-vcodec.txt
>>
>> diff --git a/Documentation/devicetree/bindings/media/mediatek-vcodec.txt b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
>> new file mode 100644
>> index 0000000..572bfdd
>> --- /dev/null
>> +++ b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
>> @@ -0,0 +1,59 @@
>> +Mediatek Video Codec
>> +
>> +Mediatek Video Codec is the video codec hw present in Mediatek SoCs which
>> +supports high resolution encoding functionalities.
>> +
>> +Required properties:
>> +- compatible : "mediatek,mt8173-vcodec-enc" for encoder
>> +- reg : Physical base address of the video codec registers and length of
>> +  memory mapped region.
>> +- interrupts : interrupt number to the cpu.
>> +- mediatek,larb : must contain the local arbiters in the current Socs.
>> +- clocks : list of clock specifiers, corresponding to entries in
>> +  the clock-names property.
>> +- clock-names: encoder must contain "vencpll_d2", "venc_sel", "univpll1_d2",
>> +  "venc_lt_sel".
>> +- iommus : should point to the respective IOMMU block with master port as
>> +  argument, see Documentation/devicetree/bindings/iommu/mediatek,iommu.txt
>> +  for details.
>> +- mediatek,vpu : the node of video processor unit
>> +
>> +Example:
>> +vcodec_enc: vcodec@0x18002000 {
>> +    compatible = "mediatek,mt8173-vcodec-enc";
>> +    reg = <0 0x18002000 0 0x1000>,    /*VENC_SYS*/
>> +          <0 0x19002000 0 0x1000>;    /*VENC_LT_SYS*/
>
> This really looks like two encoder devices combined into a single
> device tree node.
> There are two register sets, two irqs, two sets of iommus, and two
> sets of clocks.
>
> If possible, please split this node into two, one for each encoder.

I chatted offline with Mediatek.  They explained that there really is
just one encoder hardware, that happens to support multiple formats.
The encoder cannot encode with both formats at the same time.  The
Mediatek HW designers added a new format to an existing encoder by
adding a second interface (register set, irq, iommus, clocks) without
modifying the original interface.  However in the hardware itself
there is really just one encoder device.

So, although this node looks like it is for two encoder devices (one
for each format), really there is just one device that supports each
format through its large interface.

So, I'm fine with this being a single device node.

>> +    interrupts = <GIC_SPI 198 IRQ_TYPE_LEVEL_LOW>,
>> +           <GIC_SPI 202 IRQ_TYPE_LEVEL_LOW>;
>> +    mediatek,larb = <&larb3>,
>> +                   <&larb5>;
>> +    iommus = <&iommu M4U_PORT_VENC_RCPU>,
>> +             <&iommu M4U_PORT_VENC_REC>,
>> +             <&iommu M4U_PORT_VENC_BSDMA>,
>> +             <&iommu M4U_PORT_VENC_SV_COMV>,
>> +             <&iommu M4U_PORT_VENC_RD_COMV>,
>> +             <&iommu M4U_PORT_VENC_CUR_LUMA>,
>> +             <&iommu M4U_PORT_VENC_CUR_CHROMA>,
>> +             <&iommu M4U_PORT_VENC_REF_LUMA>,
>> +             <&iommu M4U_PORT_VENC_REF_CHROMA>,
>> +             <&iommu M4U_PORT_VENC_NBM_RDMA>,
>> +             <&iommu M4U_PORT_VENC_NBM_WDMA>,
>> +             <&iommu M4U_PORT_VENC_RCPU_SET2>,
>> +             <&iommu M4U_PORT_VENC_REC_FRM_SET2>,
>> +             <&iommu M4U_PORT_VENC_BSDMA_SET2>,
>> +             <&iommu M4U_PORT_VENC_SV_COMA_SET2>,
>> +             <&iommu M4U_PORT_VENC_RD_COMA_SET2>,
>> +             <&iommu M4U_PORT_VENC_CUR_LUMA_SET2>,
>> +             <&iommu M4U_PORT_VENC_CUR_CHROMA_SET2>,
>> +             <&iommu M4U_PORT_VENC_REF_LUMA_SET2>,
>> +             <&iommu M4U_PORT_VENC_REC_CHROMA_SET2>;
>> +    mediatek,vpu = <&vpu>;
>> +    clocks = <&topckgen CLK_TOP_VENCPLL_D2>,
>> +             <&topckgen CLK_TOP_VENC_SEL>,
>> +             <&topckgen CLK_TOP_UNIVPLL1_D2>,
>> +             <&topckgen CLK_TOP_VENC_LT_SEL>;
>> +    clock-names = "vencpll_d2",
>> +                  "venc_sel",
>> +                  "univpll1_d2",
>> +                  "venc_lt_sel";
>
> The names of these clocks should be from the perspective of the
> encoder, not the clock provider.

I still think these clock names should be updated, however.

-Dan

^ permalink raw reply	[flat|nested] 102+ messages in thread

* [PATCH v4 4/8] dt-bindings: Add a binding for Mediatek Video Encoder
@ 2016-02-15 10:42             ` Daniel Kurtz
  0 siblings, 0 replies; 102+ messages in thread
From: Daniel Kurtz @ 2016-02-15 10:42 UTC (permalink / raw)
  To: linux-arm-kernel

On Tue, Feb 9, 2016 at 7:29 PM, Daniel Kurtz <djkurtz@chromium.org> wrote:
> Hi Tiffany,
>
> On Thu, Feb 4, 2016 at 7:34 PM, Tiffany Lin <tiffany.lin@mediatek.com> wrote:
>> Add a DT binding documentation of Video Encoder for the
>> MT8173 SoC from Mediatek.
>>
>> Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
>> ---
>>  .../devicetree/bindings/media/mediatek-vcodec.txt  |   59 ++++++++++++++++++++
>>  1 file changed, 59 insertions(+)
>>  create mode 100644 Documentation/devicetree/bindings/media/mediatek-vcodec.txt
>>
>> diff --git a/Documentation/devicetree/bindings/media/mediatek-vcodec.txt b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
>> new file mode 100644
>> index 0000000..572bfdd
>> --- /dev/null
>> +++ b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
>> @@ -0,0 +1,59 @@
>> +Mediatek Video Codec
>> +
>> +Mediatek Video Codec is the video codec hw present in Mediatek SoCs which
>> +supports high resolution encoding functionalities.
>> +
>> +Required properties:
>> +- compatible : "mediatek,mt8173-vcodec-enc" for encoder
>> +- reg : Physical base address of the video codec registers and length of
>> +  memory mapped region.
>> +- interrupts : interrupt number to the cpu.
>> +- mediatek,larb : must contain the local arbiters in the current Socs.
>> +- clocks : list of clock specifiers, corresponding to entries in
>> +  the clock-names property.
>> +- clock-names: encoder must contain "vencpll_d2", "venc_sel", "univpll1_d2",
>> +  "venc_lt_sel".
>> +- iommus : should point to the respective IOMMU block with master port as
>> +  argument, see Documentation/devicetree/bindings/iommu/mediatek,iommu.txt
>> +  for details.
>> +- mediatek,vpu : the node of video processor unit
>> +
>> +Example:
>> +vcodec_enc: vcodec at 0x18002000 {
>> +    compatible = "mediatek,mt8173-vcodec-enc";
>> +    reg = <0 0x18002000 0 0x1000>,    /*VENC_SYS*/
>> +          <0 0x19002000 0 0x1000>;    /*VENC_LT_SYS*/
>
> This really looks like two encoder devices combined into a single
> device tree node.
> There are two register sets, two irqs, two sets of iommus, and two
> sets of clocks.
>
> If possible, please split this node into two, one for each encoder.

I chatted offline with Mediatek.  They explained that there really is
just one encoder hardware, that happens to support multiple formats.
The encoder cannot encode with both formats at the same time.  The
Mediatek HW designers added a new format to an existing encoder by
adding a second interface (register set, irq, iommus, clocks) without
modifying the original interface.  However in the hardware itself
there is really just one encoder device.

So, although this node looks like it is for two encoder devices (one
for each format), really there is just one device that supports each
format through its large interface.

So, I'm fine with this being a single device node.

>> +    interrupts = <GIC_SPI 198 IRQ_TYPE_LEVEL_LOW>,
>> +           <GIC_SPI 202 IRQ_TYPE_LEVEL_LOW>;
>> +    mediatek,larb = <&larb3>,
>> +                   <&larb5>;
>> +    iommus = <&iommu M4U_PORT_VENC_RCPU>,
>> +             <&iommu M4U_PORT_VENC_REC>,
>> +             <&iommu M4U_PORT_VENC_BSDMA>,
>> +             <&iommu M4U_PORT_VENC_SV_COMV>,
>> +             <&iommu M4U_PORT_VENC_RD_COMV>,
>> +             <&iommu M4U_PORT_VENC_CUR_LUMA>,
>> +             <&iommu M4U_PORT_VENC_CUR_CHROMA>,
>> +             <&iommu M4U_PORT_VENC_REF_LUMA>,
>> +             <&iommu M4U_PORT_VENC_REF_CHROMA>,
>> +             <&iommu M4U_PORT_VENC_NBM_RDMA>,
>> +             <&iommu M4U_PORT_VENC_NBM_WDMA>,
>> +             <&iommu M4U_PORT_VENC_RCPU_SET2>,
>> +             <&iommu M4U_PORT_VENC_REC_FRM_SET2>,
>> +             <&iommu M4U_PORT_VENC_BSDMA_SET2>,
>> +             <&iommu M4U_PORT_VENC_SV_COMA_SET2>,
>> +             <&iommu M4U_PORT_VENC_RD_COMA_SET2>,
>> +             <&iommu M4U_PORT_VENC_CUR_LUMA_SET2>,
>> +             <&iommu M4U_PORT_VENC_CUR_CHROMA_SET2>,
>> +             <&iommu M4U_PORT_VENC_REF_LUMA_SET2>,
>> +             <&iommu M4U_PORT_VENC_REC_CHROMA_SET2>;
>> +    mediatek,vpu = <&vpu>;
>> +    clocks = <&topckgen CLK_TOP_VENCPLL_D2>,
>> +             <&topckgen CLK_TOP_VENC_SEL>,
>> +             <&topckgen CLK_TOP_UNIVPLL1_D2>,
>> +             <&topckgen CLK_TOP_VENC_LT_SEL>;
>> +    clock-names = "vencpll_d2",
>> +                  "venc_sel",
>> +                  "univpll1_d2",
>> +                  "venc_lt_sel";
>
> The names of these clocks should be from the perspective of the
> encoder, not the clock provider.

I still think these clock names should be updated, however.

-Dan

^ permalink raw reply	[flat|nested] 102+ messages in thread

* Re: [PATCH v4 2/8] [media] VPU: mediatek: support Mediatek VPU
  2016-02-15 10:07       ` Hans Verkuil
  (?)
@ 2016-02-15 11:20         ` tiffany lin
  -1 siblings, 0 replies; 102+ messages in thread
From: tiffany lin @ 2016-02-15 11:20 UTC (permalink / raw)
  To: Hans Verkuil
  Cc: Hans Verkuil, daniel.thompson, Rob Herring,
	Mauro Carvalho Chehab, Matthias Brugger, Daniel Kurtz,
	Pawel Osciak, Eddie Huang, Yingjoe Chen, devicetree,
	linux-kernel, linux-arm-kernel, linux-media, linux-mediatek,
	PoChun.Lin, Andrew-CT Chen

Hi Hans,

On Mon, 2016-02-15 at 11:07 +0100, Hans Verkuil wrote:
> Hi Tiffany,
> 
> A small review comment below:
> 
> On 02/04/2016 12:34 PM, Tiffany Lin wrote:
> > The VPU driver for hw video codec embedded in Mediatek's MT8173 SOCs.
> > It is able to handle video decoding/encoding of in a range of formats.
> > The driver provides with VPU firmware download, memory management and
> > the communication interface between CPU and VPU.
> > For VPU initialization, it will create virtual memory for CPU access and
> > IOMMU address for vcodec hw device access. When a decode/encode instance
> > opens a device node, vpu driver will download vpu firmware to the device.
> > A decode/encode instant will decode/encode a frame using VPU
> > interface to interrupt vpu to handle decoding/encoding jobs.
> > 
> > Signed-off-by: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
> > Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
> > ---
> >  drivers/media/platform/Kconfig           |    9 +
> >  drivers/media/platform/Makefile          |    2 +
> >  drivers/media/platform/mtk-vpu/Makefile  |    1 +
> >  drivers/media/platform/mtk-vpu/mtk_vpu.c |  994 ++++++++++++++++++++++++++++++
> >  drivers/media/platform/mtk-vpu/mtk_vpu.h |  167 +++++
> >  5 files changed, 1173 insertions(+)
> >  create mode 100644 drivers/media/platform/mtk-vpu/Makefile
> >  create mode 100644 drivers/media/platform/mtk-vpu/mtk_vpu.c
> >  create mode 100644 drivers/media/platform/mtk-vpu/mtk_vpu.h
> > 
> 
> <snip>
> 
> > diff --git a/drivers/media/platform/mtk-vpu/mtk_vpu.c b/drivers/media/platform/mtk-vpu/mtk_vpu.c
> > new file mode 100644
> > index 0000000..f54fd89
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vpu/mtk_vpu.c
> > @@ -0,0 +1,994 @@
> 
> <snip>
> 
> > +static ssize_t vpu_debug_read(struct file *file, char __user *user_buf,
> > +			      size_t count, loff_t *ppos)
> > +{
> > +	char buf[256];
> > +	unsigned int len;
> > +	unsigned int running, pc, vpu_to_host, host_to_vpu, wdt;
> > +	int ret;
> > +	struct device *dev = file->private_data;
> > +	struct mtk_vpu *vpu = dev_get_drvdata(dev);
> > +
> > +	ret = vpu_clock_enable(vpu);
> > +	if (ret) {
> > +		dev_err(vpu->dev, "[VPU] enable clock failed %d\n", ret);
> > +		return 0;
> > +	}
> > +
> > +	/* vpu register status */
> > +	running = vpu_running(vpu);
> > +	pc = vpu_cfg_readl(vpu, VPU_PC_REG);
> > +	wdt = vpu_cfg_readl(vpu, VPU_WDT_REG);
> > +	host_to_vpu = vpu_cfg_readl(vpu, HOST_TO_VPU);
> > +	vpu_to_host = vpu_cfg_readl(vpu, VPU_TO_HOST);
> > +	vpu_clock_disable(vpu);
> > +
> > +	if (running) {
> > +		len = sprintf(buf, "VPU is running\n\n"
> 
> Please use snprintf.
> 
Got it, we will fix this in next version.

> > +		"FW Version: %s\n"
> > +		"PC: 0x%x\n"
> > +		"WDT: 0x%x\n"
> > +		"Host to VPU: 0x%x\n"
> > +		"VPU to Host: 0x%x\n",
> > +		vpu->run.fw_ver, pc, wdt,
> > +		host_to_vpu, vpu_to_host);
> > +	} else {
> > +		len = sprintf(buf, "VPU not running\n");
> 
> Ditto.
> 
> sprintf always makes me feel uncomfortable :-)
> 
Sorry about that, we will check patch series and remove all sprintf in
next version.

> > +	}
> > +
> > +	return simple_read_from_buffer(user_buf, count, ppos, buf, len);
> > +}
> 
> Regards,
> 
> 	Hans
> 

^ permalink raw reply	[flat|nested] 102+ messages in thread

* Re: [PATCH v4 2/8] [media] VPU: mediatek: support Mediatek VPU
@ 2016-02-15 11:20         ` tiffany lin
  0 siblings, 0 replies; 102+ messages in thread
From: tiffany lin @ 2016-02-15 11:20 UTC (permalink / raw)
  To: Hans Verkuil
  Cc: Hans Verkuil, daniel.thompson, Rob Herring,
	Mauro Carvalho Chehab, Matthias Brugger, Daniel Kurtz,
	Pawel Osciak, Eddie Huang, Yingjoe Chen, devicetree,
	linux-kernel, linux-arm-kernel, linux-media, linux-mediatek,
	PoChun.Lin, Andrew-CT Chen

Hi Hans,

On Mon, 2016-02-15 at 11:07 +0100, Hans Verkuil wrote:
> Hi Tiffany,
> 
> A small review comment below:
> 
> On 02/04/2016 12:34 PM, Tiffany Lin wrote:
> > The VPU driver for hw video codec embedded in Mediatek's MT8173 SOCs.
> > It is able to handle video decoding/encoding of in a range of formats.
> > The driver provides with VPU firmware download, memory management and
> > the communication interface between CPU and VPU.
> > For VPU initialization, it will create virtual memory for CPU access and
> > IOMMU address for vcodec hw device access. When a decode/encode instance
> > opens a device node, vpu driver will download vpu firmware to the device.
> > A decode/encode instant will decode/encode a frame using VPU
> > interface to interrupt vpu to handle decoding/encoding jobs.
> > 
> > Signed-off-by: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
> > Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
> > ---
> >  drivers/media/platform/Kconfig           |    9 +
> >  drivers/media/platform/Makefile          |    2 +
> >  drivers/media/platform/mtk-vpu/Makefile  |    1 +
> >  drivers/media/platform/mtk-vpu/mtk_vpu.c |  994 ++++++++++++++++++++++++++++++
> >  drivers/media/platform/mtk-vpu/mtk_vpu.h |  167 +++++
> >  5 files changed, 1173 insertions(+)
> >  create mode 100644 drivers/media/platform/mtk-vpu/Makefile
> >  create mode 100644 drivers/media/platform/mtk-vpu/mtk_vpu.c
> >  create mode 100644 drivers/media/platform/mtk-vpu/mtk_vpu.h
> > 
> 
> <snip>
> 
> > diff --git a/drivers/media/platform/mtk-vpu/mtk_vpu.c b/drivers/media/platform/mtk-vpu/mtk_vpu.c
> > new file mode 100644
> > index 0000000..f54fd89
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vpu/mtk_vpu.c
> > @@ -0,0 +1,994 @@
> 
> <snip>
> 
> > +static ssize_t vpu_debug_read(struct file *file, char __user *user_buf,
> > +			      size_t count, loff_t *ppos)
> > +{
> > +	char buf[256];
> > +	unsigned int len;
> > +	unsigned int running, pc, vpu_to_host, host_to_vpu, wdt;
> > +	int ret;
> > +	struct device *dev = file->private_data;
> > +	struct mtk_vpu *vpu = dev_get_drvdata(dev);
> > +
> > +	ret = vpu_clock_enable(vpu);
> > +	if (ret) {
> > +		dev_err(vpu->dev, "[VPU] enable clock failed %d\n", ret);
> > +		return 0;
> > +	}
> > +
> > +	/* vpu register status */
> > +	running = vpu_running(vpu);
> > +	pc = vpu_cfg_readl(vpu, VPU_PC_REG);
> > +	wdt = vpu_cfg_readl(vpu, VPU_WDT_REG);
> > +	host_to_vpu = vpu_cfg_readl(vpu, HOST_TO_VPU);
> > +	vpu_to_host = vpu_cfg_readl(vpu, VPU_TO_HOST);
> > +	vpu_clock_disable(vpu);
> > +
> > +	if (running) {
> > +		len = sprintf(buf, "VPU is running\n\n"
> 
> Please use snprintf.
> 
Got it, we will fix this in next version.

> > +		"FW Version: %s\n"
> > +		"PC: 0x%x\n"
> > +		"WDT: 0x%x\n"
> > +		"Host to VPU: 0x%x\n"
> > +		"VPU to Host: 0x%x\n",
> > +		vpu->run.fw_ver, pc, wdt,
> > +		host_to_vpu, vpu_to_host);
> > +	} else {
> > +		len = sprintf(buf, "VPU not running\n");
> 
> Ditto.
> 
> sprintf always makes me feel uncomfortable :-)
> 
Sorry about that, we will check patch series and remove all sprintf in
next version.

> > +	}
> > +
> > +	return simple_read_from_buffer(user_buf, count, ppos, buf, len);
> > +}
> 
> Regards,
> 
> 	Hans
> 

^ permalink raw reply	[flat|nested] 102+ messages in thread

* [PATCH v4 2/8] [media] VPU: mediatek: support Mediatek VPU
@ 2016-02-15 11:20         ` tiffany lin
  0 siblings, 0 replies; 102+ messages in thread
From: tiffany lin @ 2016-02-15 11:20 UTC (permalink / raw)
  To: linux-arm-kernel

Hi Hans,

On Mon, 2016-02-15 at 11:07 +0100, Hans Verkuil wrote:
> Hi Tiffany,
> 
> A small review comment below:
> 
> On 02/04/2016 12:34 PM, Tiffany Lin wrote:
> > The VPU driver for hw video codec embedded in Mediatek's MT8173 SOCs.
> > It is able to handle video decoding/encoding of in a range of formats.
> > The driver provides with VPU firmware download, memory management and
> > the communication interface between CPU and VPU.
> > For VPU initialization, it will create virtual memory for CPU access and
> > IOMMU address for vcodec hw device access. When a decode/encode instance
> > opens a device node, vpu driver will download vpu firmware to the device.
> > A decode/encode instant will decode/encode a frame using VPU
> > interface to interrupt vpu to handle decoding/encoding jobs.
> > 
> > Signed-off-by: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
> > Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
> > ---
> >  drivers/media/platform/Kconfig           |    9 +
> >  drivers/media/platform/Makefile          |    2 +
> >  drivers/media/platform/mtk-vpu/Makefile  |    1 +
> >  drivers/media/platform/mtk-vpu/mtk_vpu.c |  994 ++++++++++++++++++++++++++++++
> >  drivers/media/platform/mtk-vpu/mtk_vpu.h |  167 +++++
> >  5 files changed, 1173 insertions(+)
> >  create mode 100644 drivers/media/platform/mtk-vpu/Makefile
> >  create mode 100644 drivers/media/platform/mtk-vpu/mtk_vpu.c
> >  create mode 100644 drivers/media/platform/mtk-vpu/mtk_vpu.h
> > 
> 
> <snip>
> 
> > diff --git a/drivers/media/platform/mtk-vpu/mtk_vpu.c b/drivers/media/platform/mtk-vpu/mtk_vpu.c
> > new file mode 100644
> > index 0000000..f54fd89
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vpu/mtk_vpu.c
> > @@ -0,0 +1,994 @@
> 
> <snip>
> 
> > +static ssize_t vpu_debug_read(struct file *file, char __user *user_buf,
> > +			      size_t count, loff_t *ppos)
> > +{
> > +	char buf[256];
> > +	unsigned int len;
> > +	unsigned int running, pc, vpu_to_host, host_to_vpu, wdt;
> > +	int ret;
> > +	struct device *dev = file->private_data;
> > +	struct mtk_vpu *vpu = dev_get_drvdata(dev);
> > +
> > +	ret = vpu_clock_enable(vpu);
> > +	if (ret) {
> > +		dev_err(vpu->dev, "[VPU] enable clock failed %d\n", ret);
> > +		return 0;
> > +	}
> > +
> > +	/* vpu register status */
> > +	running = vpu_running(vpu);
> > +	pc = vpu_cfg_readl(vpu, VPU_PC_REG);
> > +	wdt = vpu_cfg_readl(vpu, VPU_WDT_REG);
> > +	host_to_vpu = vpu_cfg_readl(vpu, HOST_TO_VPU);
> > +	vpu_to_host = vpu_cfg_readl(vpu, VPU_TO_HOST);
> > +	vpu_clock_disable(vpu);
> > +
> > +	if (running) {
> > +		len = sprintf(buf, "VPU is running\n\n"
> 
> Please use snprintf.
> 
Got it, we will fix this in next version.

> > +		"FW Version: %s\n"
> > +		"PC: 0x%x\n"
> > +		"WDT: 0x%x\n"
> > +		"Host to VPU: 0x%x\n"
> > +		"VPU to Host: 0x%x\n",
> > +		vpu->run.fw_ver, pc, wdt,
> > +		host_to_vpu, vpu_to_host);
> > +	} else {
> > +		len = sprintf(buf, "VPU not running\n");
> 
> Ditto.
> 
> sprintf always makes me feel uncomfortable :-)
> 
Sorry about that, we will check patch series and remove all sprintf in
next version.

> > +	}
> > +
> > +	return simple_read_from_buffer(user_buf, count, ppos, buf, len);
> > +}
> 
> Regards,
> 
> 	Hans
> 

^ permalink raw reply	[flat|nested] 102+ messages in thread

* Re: [PATCH v4 5/8] [Media] vcodec: mediatek: Add Mediatek V4L2 Video Encoder Driver
  2016-02-04 11:35           ` Tiffany Lin
  (?)
  (?)
@ 2016-02-15 11:21           ` Hans Verkuil
  2016-02-16  6:37               ` tiffany lin
  -1 siblings, 1 reply; 102+ messages in thread
From: Hans Verkuil @ 2016-02-15 11:21 UTC (permalink / raw)
  To: Tiffany Lin, Hans Verkuil, daniel.thompson, Rob Herring,
	Mauro Carvalho Chehab, Matthias Brugger, Daniel Kurtz,
	Pawel Osciak
  Cc: Eddie Huang, Yingjoe Chen, devicetree, linux-kernel,
	linux-arm-kernel, linux-media, linux-mediatek, PoChun.Lin,
	Andrew-CT Chen

On 02/04/2016 12:35 PM, Tiffany Lin wrote:
> From: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
> 
> Add v4l2 layer encoder driver for MT8173
> 
> Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>

If Andrew is the author, shouldn't there be a Signed-off-by from him as well?

And in copyright notices (might want to update the year to 2016 BTW) PC Chen
is mentioned among others. It might be useful to update the Signed-off-by lines.

> ---
>  drivers/media/platform/Kconfig                     |   11 +
>  drivers/media/platform/Makefile                    |    2 +
>  drivers/media/platform/mtk-vcodec/Makefile         |    8 +
>  drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h |  388 ++++++
>  drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c | 1380 ++++++++++++++++++++
>  drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h |   46 +
>  .../media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c |  476 +++++++
>  .../media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c  |  132 ++
>  .../media/platform/mtk-vcodec/mtk_vcodec_intr.c    |  102 ++
>  .../media/platform/mtk-vcodec/mtk_vcodec_intr.h    |   29 +
>  drivers/media/platform/mtk-vcodec/mtk_vcodec_pm.h  |   26 +
>  .../media/platform/mtk-vcodec/mtk_vcodec_util.c    |  106 ++
>  .../media/platform/mtk-vcodec/mtk_vcodec_util.h    |   85 ++
>  drivers/media/platform/mtk-vcodec/venc_drv_base.h  |   62 +
>  drivers/media/platform/mtk-vcodec/venc_drv_if.c    |  100 ++
>  drivers/media/platform/mtk-vcodec/venc_drv_if.h    |  175 +++
>  drivers/media/platform/mtk-vcodec/venc_ipi_msg.h   |  212 +++
>  include/uapi/linux/v4l2-controls.h                 |    4 +
>  18 files changed, 3344 insertions(+)
>  create mode 100644 drivers/media/platform/mtk-vcodec/Makefile
>  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
>  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
>  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h
>  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
>  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
>  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.c
>  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.h
>  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_pm.h
>  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c
>  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
>  create mode 100644 drivers/media/platform/mtk-vcodec/venc_drv_base.h
>  create mode 100644 drivers/media/platform/mtk-vcodec/venc_drv_if.c
>  create mode 100644 drivers/media/platform/mtk-vcodec/venc_drv_if.h
>  create mode 100644 drivers/media/platform/mtk-vcodec/venc_ipi_msg.h
>  mode change 100644 => 100755 include/uapi/linux/v4l2-controls.h
> 
> diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
> index ba812d6..3e831c5 100644
> --- a/drivers/media/platform/Kconfig
> +++ b/drivers/media/platform/Kconfig
> @@ -157,6 +157,17 @@ config VIDEO_MEDIATEK_VPU
>  	    codec embedded in new Mediatek's SOCs. It is able
>  	    to handle video decoding/encoding in a range of formats.
>  
> +config VIDEO_MEDIATEK_VCODEC
> +        tristate "Mediatek Video Codec driver"
> +        depends on VIDEO_DEV && VIDEO_V4L2
> +        depends on ARCH_MEDIATEK || COMPILE_TEST
> +        select VIDEOBUF2_DMA_CONTIG
> +        select V4L2_MEM2MEM_DEV
> +        select MEDIATEK_VPU
> +        default n
> +        ---help---
> +            Mediatek video codec driver for V4L2
> +
>  config VIDEO_MEM2MEM_DEINTERLACE
>  	tristate "Deinterlace support"
>  	depends on VIDEO_DEV && VIDEO_V4L2 && DMA_ENGINE
> diff --git a/drivers/media/platform/Makefile b/drivers/media/platform/Makefile
> index e5b19c6..510e06b 100644
> --- a/drivers/media/platform/Makefile
> +++ b/drivers/media/platform/Makefile
> @@ -57,3 +57,5 @@ obj-$(CONFIG_VIDEO_XILINX)		+= xilinx/
>  ccflags-y += -I$(srctree)/drivers/media/i2c
>  
>  obj-$(CONFIG_VIDEO_MEDIATEK_VPU)	+= mtk-vpu/
> +
> +obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC)	+= mtk-vcodec/
> diff --git a/drivers/media/platform/mtk-vcodec/Makefile b/drivers/media/platform/mtk-vcodec/Makefile
> new file mode 100644
> index 0000000..ce38689
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vcodec/Makefile
> @@ -0,0 +1,8 @@
> +obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += mtk_vcodec_intr.o \
> +				       mtk_vcodec_util.o \
> +				       mtk_vcodec_enc_drv.o \
> +				       mtk_vcodec_enc.o \
> +				       mtk_vcodec_enc_pm.o \
> +				       venc_drv_if.o
> +
> +ccflags-y += -I$(srctree)/drivers/media/platform/mtk-vpu
> diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
> new file mode 100644
> index 0000000..9da2818
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
> @@ -0,0 +1,388 @@
> +/*
> +* Copyright (c) 2015 MediaTek Inc.
> +* Author: PC Chen <pc.chen@mediatek.com>
> +*         Tiffany Lin <tiffany.lin@mediatek.com>
> +*
> +* This program is free software; you can redistribute it and/or modify
> +* it under the terms of the GNU General Public License version 2 as
> +* published by the Free Software Foundation.
> +*
> +* This program is distributed in the hope that it will be useful,
> +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +* GNU General Public License for more details.
> +*/
> +
> +#ifndef _MTK_VCODEC_DRV_H_
> +#define _MTK_VCODEC_DRV_H_
> +
> +#include <linux/platform_device.h>
> +#include <linux/videodev2.h>
> +#include <media/v4l2-ctrls.h>
> +#include <media/v4l2-device.h>
> +#include <media/v4l2-ioctl.h>
> +#include <media/videobuf2-core.h>
> +
> +#include "mtk_vcodec_util.h"
> +
> +#define MTK_VCODEC_MAX_INSTANCES		10
> +#define MTK_VCODEC_MAX_ENCODER_INSTANCES	3
> +
> +#define MTK_VCODEC_MAX_FRAME_SIZE	0x800000
> +#define MTK_VIDEO_MAX_FRAME		32
> +#define MTK_MAX_CTRLS			20
> +
> +#define MTK_VCODEC_DRV_NAME		"mtk_vcodec_drv"
> +#define MTK_VCODEC_ENC_NAME		"mtk-vcodec-enc"
> +#define MTK_PLATFORM_STR		"platform:mt8173"
> +
> +#define MTK_VENC_IRQ_STATUS_SPS          0x1
> +#define MTK_VENC_IRQ_STATUS_PPS          0x2
> +#define MTK_VENC_IRQ_STATUS_FRM          0x4
> +#define MTK_VENC_IRQ_STATUS_DRAM         0x8
> +#define MTK_VENC_IRQ_STATUS_PAUSE        0x10
> +#define MTK_VENC_IRQ_STATUS_SWITCH       0x20
> +
> +#define MTK_VENC_IRQ_STATUS_OFFSET       0x05C
> +#define MTK_VENC_IRQ_ACK_OFFSET          0x060
> +
> +#define MTK_VCODEC_MAX_PLANES		3
> +
> +#define VDEC_HW_ACTIVE	0x10
> +#define VDEC_IRQ_CFG    0x11
> +#define VDEC_IRQ_CLR    0x10
> +
> +#define VDEC_IRQ_CFG_REG	0xa4
> +#define WAIT_INTR_TIMEOUT	1000
> +
> +/**
> + * enum mtk_hw_reg_idx - MTK hw register base index
> + */
> +enum mtk_hw_reg_idx {
> +	VDEC_SYS,
> +	VDEC_MISC,
> +	VDEC_LD,
> +	VDEC_TOP,
> +	VDEC_CM,
> +	VDEC_AD,
> +	VDEC_AV,
> +	VDEC_PP,
> +	VDEC_HWD,
> +	VDEC_HWQ,
> +	VDEC_HWB,
> +	VDEC_HWG,
> +	NUM_MAX_VDEC_REG_BASE,
> +	VENC_SYS = NUM_MAX_VDEC_REG_BASE,
> +	VENC_LT_SYS,
> +	NUM_MAX_VCODEC_REG_BASE
> +};
> +
> +/**
> + * enum mtk_instance_type - The type of an MTK Vcodec instance.
> + */
> +enum mtk_instance_type {
> +	MTK_INST_DECODER		= 0,
> +	MTK_INST_ENCODER		= 1,
> +};
> +
> +/**
> + * enum mtk_instance_state - The state of an MTK Vcodec instance.
> + * @MTK_STATE_FREE - default state when instance is created
> + * @MTK_STATE_INIT - vcodec instance is initialized
> + * @MTK_STATE_HEADER - vdec had sps/pps header parsed or venc
> + *			had sps/pps header encoded
> + * @MTK_STATE_FLUSH - vdec is flushing. Only used by decoder
> + * @MTK_STATE_RES_CHANGE - vdec detect resolution change.
> + * 			Only used by decoder
> + * @MTK_STATE_ABORT - vcodec should be aborted
> + */
> +enum mtk_instance_state {
> +	MTK_STATE_FREE = 0,
> +	MTK_STATE_INIT = 1,
> +	MTK_STATE_HEADER = 2,
> +	MTK_STATE_FLUSH = 3,
> +	MTK_STATE_RES_CHANGE = 4,
> +	MTK_STATE_ABORT = 5,
> +};
> +
> +/**
> + * struct mtk_param_change - General encoding parameters type
> + */
> +enum mtk_encode_param {
> +	MTK_ENCODE_PARAM_NONE = 0,
> +	MTK_ENCODE_PARAM_BITRATE = (1 << 0),
> +	MTK_ENCODE_PARAM_FRAMERATE = (1 << 1),
> +	MTK_ENCODE_PARAM_INTRA_PERIOD = (1 << 2),
> +	MTK_ENCODE_PARAM_FRAME_TYPE = (1 << 3),
> +};
> +
> +/**
> + * enum mtk_fmt_type - Type of the pixelformat
> + * @MTK_FMT_FRAME - mtk vcodec raw frame
> + */
> +enum mtk_fmt_type {
> +	MTK_FMT_DEC		= 0,
> +	MTK_FMT_ENC		= 1,
> +	MTK_FMT_FRAME		= 2,
> +};
> +
> +/**
> + * struct mtk_video_fmt - Structure used to store information about pixelformats
> + */
> +struct mtk_video_fmt {
> +	u32 fourcc;
> +	enum mtk_fmt_type type;
> +	u32 num_planes;
> +};
> +
> +/**
> + * struct mtk_codec_framesizes - Structure used to store information about framesizes
> + */
> +struct mtk_codec_framesizes {
> +	u32 fourcc;
> +	struct	v4l2_frmsize_stepwise	stepwise;
> +};
> +
> +/**
> + * struct mtk_q_type - Type of queue
> + */
> +enum mtk_q_type {
> +	MTK_Q_DATA_SRC		= 0,
> +	MTK_Q_DATA_DST		= 1,
> +};
> +
> +/**
> + * struct mtk_q_data - Structure used to store information about queue
> + * @colorspace	reserved for encoder
> + * @field		reserved for encoder

struct is only partially documented here.

> + */
> +struct mtk_q_data {
> +	unsigned int		width;
> +	unsigned int		height;
> +	enum v4l2_field		field;
> +	enum v4l2_colorspace	colorspace;
> +	unsigned int		bytesperline[MTK_VCODEC_MAX_PLANES];
> +	unsigned int		sizeimage[MTK_VCODEC_MAX_PLANES];
> +	struct mtk_video_fmt	*fmt;
> +};
> +
> +/**
> + * struct mtk_enc_params - General encoding parameters
> + * @bitrate - target bitrate
> + * @num_b_frame - number of b frames between p-frame
> + * @rc_frame - frame based rate control
> + * @rc_mb - macroblock based rate control
> + * @seq_hdr_mode - H.264 sequence header is encoded separately or joined with the first frame
> + * @gop_size - group of picture size, it's used as the intra frame period
> + * @framerate_num - frame rate numerator
> + * @framerate_denom - frame rate denominator
> + * @h264_max_qp - Max value for H.264 quantization parameter
> + * @h264_profile - V4L2 defined H.264 profile
> + * @h264_level - V4L2 defined H.264 level
> + * @force_intra - force/insert intra frame
> + * @skip_frame - encode in skip frame mode that use minimum number of bits

skip_frame isn't in the struct anymore.

> + */
> +struct mtk_enc_params {
> +	unsigned int	bitrate;
> +	unsigned int	num_b_frame;
> +	unsigned int	rc_frame;
> +	unsigned int	rc_mb;
> +	unsigned int	seq_hdr_mode;
> +	unsigned int	gop_size;
> +	unsigned int	framerate_num;
> +	unsigned int	framerate_denom;
> +	unsigned int	h264_max_qp;
> +	unsigned int	h264_profile;
> +	unsigned int	h264_level;
> +	unsigned int	force_intra;
> +};
> +
> +/**
> + * struct mtk_vcodec_pm - Power management data structure
> + */
> +struct mtk_vcodec_pm {
> +	struct clk	*vcodecpll;
> +	struct clk	*univpll_d2;
> +	struct clk	*clk_cci400_sel;
> +	struct clk	*vdecpll;
> +	struct clk	*vdec_sel;
> +	struct clk	*vencpll_d2;
> +	struct clk	*venc_sel;
> +	struct clk	*univpll1_d2;
> +	struct clk	*venc_lt_sel;
> +	struct device	*larbvdec;
> +	struct device	*larbvenc;
> +	struct device	*larbvenclt;
> +	struct device	*dev;
> +	struct mtk_vcodec_dev *mtkdev;
> +};
> +
> +
> +/**
> + * struct mtk_vcodec_ctx - Context (instance) private data.
> + *
> + * @type:		type of the instance - decoder or encoder
> + * @dev:		pointer to the mtk_vcodec_dev of the device
> + * @fh:			struct v4l2_fh
> + * @m2m_ctx:		pointer to the v4l2_m2m_ctx of the context
> + * @q_data:		store information of input and output queue
> + *			of the context
> + * @idx:		index of the context that this structure describes
> + * @state:		state of the context
> + * @param_change:	encode parameters
> + * @enc_params:		encoding parameters
> + * @colorspace:
> + * @enc_if:		hoooked encoder driver interface
> + * @drv_handle:		driver handle for specific decode/encode instance
> + *
> + * @picinfo:		store width/height of image and buffer and planes' size for decoder
> + *			and encoder
> + * @dpb_count:		count of the DPB buffers required by MTK Vcodec hw
> + *
> + * @int_cond:		variable used by the waitqueue
> + * @int_type:		type of the last interrupt
> + * @queue:		waitqueue that can be used to wait for this context to
> + *			finish
> + * @irq_status:		irq status
> + *
> + * @ctrl_hdl:		handler for v4l2 framework
> + * @ctrls:		array of controls, used when adding controls to the
> + *			v4l2 control framework
> + *
> + * @encode_work:	worker for the encoding
> + */

These comments are out of date with the actual struct!

> +struct mtk_vcodec_ctx {
> +	enum mtk_instance_type type;
> +	struct mtk_vcodec_dev *dev;
> +	struct v4l2_fh fh;
> +	struct v4l2_m2m_ctx *m2m_ctx;
> +	struct mtk_q_data q_data[2];
> +	int idx;
> +	enum mtk_instance_state state;
> +	enum mtk_encode_param param_change;
> +	struct mtk_enc_params enc_params;
> +
> +	struct venc_common_if *enc_if;
> +	unsigned long drv_handle;
> +
> +
> +	int int_cond;
> +	int int_type;
> +	wait_queue_head_t queue;
> +	unsigned int irq_status;
> +
> +	struct v4l2_ctrl_handler ctrl_hdl;
> +	struct v4l2_ctrl *ctrls[MTK_MAX_CTRLS];
> +
> +	struct work_struct encode_work;
> +
> +};
> +
> +/**
> + * struct mtk_vcodec_dev - driver data
> + * @v4l2_dev:		V4L2 device to register video devices for.
> + * @vfd_enc:		Video device for encoder.
> + *
> + * @m2m_dev_enc:	m2m device for encoder.
> + * @plat_dev:		platform device
> + * @alloc_ctx:		VB2 allocator context
> + *			(for allocations without kernel mapping).
> + * @ctx:		array of driver contexts
> + *
> + * @curr_ctx:		The context that is waiting for codec hardware
> + *
> + * @reg_base:		Mapped address of MTK Vcodec registers.
> + *
> + * @instance_mask:	used to mark which contexts are opened
> + * @num_instances:	counter of active MTK Vcodec instances
> + *
> + * @encode_workqueue:	encode work queue
> + *
> + * @int_cond:		used to identify interrupt condition happen
> + * @int_type:		used to identify what kind of interrupt condition happen
> + * @dev_mutex:		video_device lock
> + * @queue:		waitqueue for waiting for completion of device commands
> + *
> + * @enc_irq:		h264 encoder irq resource
> + * @enc_lt_irq:		vp8 encoder irq resource
> + *
> + * @enc_mutex:		encoder hardware lock.
> + *
> + * @pm:			power management control
> + * @dec_capability:	used to identify decode capability, ex: 4k
> + * @enc_capability:     used to identify encode capability
> + */
> +struct mtk_vcodec_dev {
> +	struct v4l2_device	v4l2_dev;
> +	struct video_device	*vfd_enc;
> +
> +	struct v4l2_m2m_dev	*m2m_dev_enc;
> +	struct platform_device	*plat_dev;
> +	struct platform_device 	*vpu_plat_dev;
> +	struct vb2_alloc_ctx	*alloc_ctx;
> +	struct mtk_vcodec_ctx	*ctx[MTK_VCODEC_MAX_INSTANCES];
> +	int curr_ctx;
> +	void __iomem		*reg_base[NUM_MAX_VCODEC_REG_BASE];
> +
> +	unsigned long	instance_mask[BITS_TO_LONGS(MTK_VCODEC_MAX_INSTANCES)];
> +	int			num_instances;
> +
> +	struct workqueue_struct *encode_workqueue;
> +
> +	int			int_cond;
> +	int			int_type;
> +	struct mutex		dev_mutex;
> +	wait_queue_head_t	queue;
> +
> +	int			enc_irq;
> +	int			enc_lt_irq;
> +
> +	struct mutex		enc_mutex;
> +
> +	struct mtk_vcodec_pm	pm;
> +	unsigned int		dec_capability;
> +	unsigned int		enc_capability;
> +};
> +
> +/**
> + * struct mtk_vcodec_ctrl - information about controls to be registered.
> + * @id:			Control ID.
> + * @type:		Type of the control.
> + * @name:		Human readable name of the control.
> + * @minimum:		Minimum value of the control.
> + * @maximum:		Maximum value of the control.
> + * @step:		Control value increase step.
> + * @menu_skip_mask:	Mask of invalid menu positions.
> + * @default_value:	Initial value of the control.
> + * @is_volatile:	Control is volatile.
> + *
> + * See also struct v4l2_ctrl_config.
> + */
> +struct mtk_vcodec_ctrl {
> +	u32			id;
> +	enum v4l2_ctrl_type	type;
> +	u8			name[32];
> +	s32			minimum;
> +	s32			maximum;
> +	s32			step;
> +	u32			menu_skip_mask;
> +	s32			default_value;
> +	u8			is_volatile;
> +};
> +
> +static inline struct mtk_vcodec_ctx *fh_to_ctx(struct v4l2_fh *fh)
> +{
> +	return container_of(fh, struct mtk_vcodec_ctx, fh);
> +}
> +
> +static inline struct mtk_vcodec_ctx *ctrl_to_ctx(struct v4l2_ctrl *ctrl)
> +{
> +	return container_of(ctrl->handler, struct mtk_vcodec_ctx, ctrl_hdl);
> +}
> +
> +extern const struct v4l2_ioctl_ops mtk_vdec_ioctl_ops;
> +extern const struct v4l2_m2m_ops mtk_vdec_m2m_ops;
> +extern const struct v4l2_ioctl_ops mtk_venc_ioctl_ops;
> +extern const struct v4l2_m2m_ops mtk_venc_m2m_ops;
> +
> +#endif /* _MTK_VCODEC_DRV_H_ */
> diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
> new file mode 100644
> index 0000000..ee602fe
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
> @@ -0,0 +1,1380 @@
> +/*
> +* Copyright (c) 2015 MediaTek Inc.
> +* Author: PC Chen <pc.chen@mediatek.com>
> +*         Tiffany Lin <tiffany.lin@mediatek.com>
> +*
> +* This program is free software; you can redistribute it and/or modify
> +* it under the terms of the GNU General Public License version 2 as
> +* published by the Free Software Foundation.
> +*
> +* This program is distributed in the hope that it will be useful,
> +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +* GNU General Public License for more details.
> +*/
> +
> +#include <media/v4l2-event.h>
> +#include <media/v4l2-mem2mem.h>
> +#include <media/videobuf2-dma-contig.h>
> +
> +#include "mtk_vcodec_drv.h"
> +#include "mtk_vcodec_enc.h"
> +#include "mtk_vcodec_intr.h"
> +#include "mtk_vcodec_util.h"
> +#include "venc_drv_if.h"
> +
> +#define MTK_VENC_MIN_W	32
> +#define MTK_VENC_MIN_H	32
> +#define MTK_VENC_MAX_W	1920
> +#define MTK_VENC_MAX_H	1080
> +#define DFT_CFG_WIDTH	MTK_VENC_MIN_W
> +#define DFT_CFG_HEIGHT	MTK_VENC_MIN_H
> +
> +static void mtk_venc_worker(struct work_struct *work);
> +
> +static struct mtk_video_fmt mtk_video_formats[] = {
> +	{
> +		.fourcc		= V4L2_PIX_FMT_YUV420,
> +		.type		= MTK_FMT_FRAME,
> +		.num_planes	= 3,
> +	},
> +	{
> +		.fourcc		= V4L2_PIX_FMT_YVU420,
> +		.type		= MTK_FMT_FRAME,
> +		.num_planes	= 3,
> +	},
> +	{
> +		.fourcc		= V4L2_PIX_FMT_NV12,
> +		.type		= MTK_FMT_FRAME,
> +		.num_planes	= 2,
> +	},
> +	{
> +		.fourcc		= V4L2_PIX_FMT_NV21,
> +		.type		= MTK_FMT_FRAME,
> +		.num_planes	= 2,
> +	},
> +	{
> +		.fourcc		= V4L2_PIX_FMT_YUV420M,
> +		.type		= MTK_FMT_FRAME,
> +		.num_planes	= 3,
> +	},
> +	{
> +		.fourcc		= V4L2_PIX_FMT_YVU420M,
> +		.type		= MTK_FMT_FRAME,
> +		.num_planes	= 3,
> +	},
> +	{
> +		.fourcc 	= V4L2_PIX_FMT_NV12M,
> +		.type		= MTK_FMT_FRAME,
> +		.num_planes	= 2,
> +	},
> +	{
> +		.fourcc		= V4L2_PIX_FMT_NV21M,
> +		.type		= MTK_FMT_FRAME,
> +		.num_planes	= 2,
> +	},
> +	{
> +		.fourcc		= V4L2_PIX_FMT_H264,
> +		.type		= MTK_FMT_ENC,
> +		.num_planes	= 1,
> +	},
> +	{
> +		.fourcc		= V4L2_PIX_FMT_VP8,
> +		.type		= MTK_FMT_ENC,
> +		.num_planes	= 1,
> +	},
> +};
> +
> +#define NUM_FORMATS ARRAY_SIZE(mtk_video_formats)
> +
> +static const struct mtk_codec_framesizes mtk_venc_framesizes[] = {
> +	{
> +		.fourcc	= V4L2_PIX_FMT_H264,
> +		.stepwise = {  160, 1920, 16, 128, 1088, 16 },
> +	},
> +	{
> +		.fourcc = V4L2_PIX_FMT_VP8,
> +		.stepwise = {  160, 1920, 16, 128, 1088, 16 },
> +	},
> +};
> +
> +#define NUM_SUPPORTED_FRAMESIZE ARRAY_SIZE(mtk_venc_framesizes)
> +
> +static int vidioc_venc_s_ctrl(struct v4l2_ctrl *ctrl)
> +{
> +	struct mtk_vcodec_ctx *ctx = ctrl_to_ctx(ctrl);
> +	struct mtk_enc_params *p = &ctx->enc_params;
> +	int ret = 0;
> +
> +	switch (ctrl->id) {
> +	case V4L2_CID_MPEG_VIDEO_BITRATE:
> +		mtk_v4l2_debug(2, "V4L2_CID_MPEG_VIDEO_BITRATE val = %d",
> +			ctrl->val);
> +		p->bitrate = ctrl->val;
> +		ctx->param_change |= MTK_ENCODE_PARAM_BITRATE;
> +		break;
> +	case V4L2_CID_MPEG_VIDEO_B_FRAMES:
> +		mtk_v4l2_debug(2, "V4L2_CID_MPEG_VIDEO_B_FRAMES val = %d",
> +			ctrl->val);
> +		p->num_b_frame = ctrl->val;
> +		break;
> +	case V4L2_CID_MPEG_VIDEO_FRAME_RC_ENABLE:
> +		mtk_v4l2_debug(2, "V4L2_CID_MPEG_VIDEO_FRAME_RC_ENABLE val = %d",
> +			ctrl->val);
> +		p->rc_frame = ctrl->val;
> +		break;
> +	case V4L2_CID_MPEG_VIDEO_H264_MAX_QP:
> +		mtk_v4l2_debug(2, "V4L2_CID_MPEG_VIDEO_H264_MAX_QP val = %d",
> +			ctrl->val);
> +		p->h264_max_qp = ctrl->val;
> +		break;
> +	case V4L2_CID_MPEG_VIDEO_HEADER_MODE:
> +		mtk_v4l2_debug(2, "V4L2_CID_MPEG_VIDEO_HEADER_MODE val = %d",
> +			ctrl->val);
> +		p->seq_hdr_mode = ctrl->val;
> +		break;
> +	case V4L2_CID_MPEG_VIDEO_MB_RC_ENABLE:
> +		mtk_v4l2_debug(2, "V4L2_CID_MPEG_VIDEO_MB_RC_ENABLE val = %d",
> +			ctrl->val);
> +		p->rc_mb = ctrl->val;
> +		break;
> +	case V4L2_CID_MPEG_VIDEO_H264_PROFILE:
> +		mtk_v4l2_debug(2, "V4L2_CID_MPEG_VIDEO_H264_PROFILE val = %d",
> +			ctrl->val);
> +		p->h264_profile = ctrl->val;
> +		break;
> +	case V4L2_CID_MPEG_VIDEO_H264_LEVEL:
> +		mtk_v4l2_debug(2, "V4L2_CID_MPEG_VIDEO_H264_LEVEL val = %d",
> +			ctrl->val);
> +		p->h264_level = ctrl->val;
> +		break;
> +	case V4L2_CID_MPEG_VIDEO_H264_I_PERIOD:
> +		mtk_v4l2_debug(2, "V4L2_CID_MPEG_VIDEO_H264_I_PERIOD val = %d",
> +			ctrl->val);
> +	case V4L2_CID_MPEG_VIDEO_GOP_SIZE:
> +		mtk_v4l2_debug(2, "V4L2_CID_MPEG_VIDEO_GOP_SIZE val = %d",
> +			ctrl->val);
> +		p->gop_size = ctrl->val;
> +		ctx->param_change |= MTK_ENCODE_PARAM_INTRA_PERIOD;
> +		break;
> +	case V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME:
> +		mtk_v4l2_debug(2, "V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME");
> +		p->force_intra = 1;
> +		ctx->param_change |= MTK_ENCODE_PARAM_FRAME_TYPE;
> +		break;
> +	default:
> +		ret = -EINVAL;
> +		break;
> +	}
> +
> +	return ret;
> +}
> +
> +static const struct v4l2_ctrl_ops mtk_vcodec_enc_ctrl_ops = {
> +	.s_ctrl = vidioc_venc_s_ctrl,
> +};
> +
> +static int vidioc_enum_fmt(struct file *file, struct v4l2_fmtdesc *f,
> +			   bool out)
> +{
> +	struct mtk_video_fmt *fmt;
> +	int i, j = 0;
> +
> +	for (i = 0; i < NUM_FORMATS; ++i) {
> +		if (out && mtk_video_formats[i].type != MTK_FMT_FRAME)
> +			continue;
> +		else if (!out && mtk_video_formats[i].type != MTK_FMT_ENC)

No 'else' needed here.

> +			continue;
> +
> +		if (j == f->index) {
> +			fmt = &mtk_video_formats[i];
> +			f->pixelformat = fmt->fourcc;
> +			return 0;
> +		}
> +		++j;
> +	}
> +
> +	return -EINVAL;
> +}
> +
> +static int vidioc_enum_framesizes(struct file *file, void *fh,
> +				  struct v4l2_frmsizeenum *fsize)
> +{
> +	int i = 0;
> +
> +	for (i = 0; i < NUM_SUPPORTED_FRAMESIZE; ++i) {
> +		if (fsize->pixel_format != mtk_venc_framesizes[i].fourcc)
> +			continue;
> +
> +		if (!fsize->index) {
> +			fsize->type = V4L2_FRMSIZE_TYPE_STEPWISE;
> +			fsize->stepwise = mtk_venc_framesizes[i].stepwise;
> +			return 0;
> +		}
> +	}
> +
> +	return -EINVAL;
> +}
> +
> +static int vidioc_enum_fmt_vid_cap_mplane(struct file *file, void *pirv,
> +					  struct v4l2_fmtdesc *f)
> +{
> +	return vidioc_enum_fmt(file, f, false);
> +}
> +
> +static int vidioc_enum_fmt_vid_out_mplane(struct file *file, void *prov,
> +					  struct v4l2_fmtdesc *f)
> +{
> +	return vidioc_enum_fmt(file, f, true);
> +}
> +
> +static int vidioc_venc_querycap(struct file *file, void *priv,
> +				struct v4l2_capability *cap)
> +{
> +        strlcpy(cap->driver, MTK_VCODEC_ENC_NAME, strlen(MTK_VCODEC_ENC_NAME));

This should be:

	strlcpy(cap->driver, MTK_VCODEC_ENC_NAME, sizeof(cap->driver));

> +        cap->driver[strlen(MTK_VCODEC_ENC_NAME)]=0;

No need to terminate with 0, strlcpy does that already.

> +        strlcpy(cap->bus_info, MTK_PLATFORM_STR, strlen(MTK_PLATFORM_STR));
> +        cap->bus_info[strlen(MTK_PLATFORM_STR)]=0;
> +        strlcpy(cap->card, MTK_PLATFORM_STR, strlen(MTK_PLATFORM_STR));
> +        cap->card[strlen(MTK_PLATFORM_STR)]=0;

Ditto for these two fields. Note that use use spaces instead of a tab here as
well. Please fix. (checkpatch should have complained about that!)

> +
> +	/*
> +	 * This is only a mem-to-mem video device. The capture and output
> +	 * device capability flags are left only for backward compatibility
> +	 * and are scheduled for removal.
> +	 */

Comment is out of date.

> +	cap->device_caps  = V4L2_CAP_VIDEO_M2M_MPLANE | V4L2_CAP_STREAMING;
> +	cap->capabilities = cap->device_caps | V4L2_CAP_DEVICE_CAPS;
> +
> +	return 0;
> +}
> +static int vidioc_venc_s_parm(struct file *file, void *priv,
> +			      struct v4l2_streamparm *a)
> +{
> +	struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
> +
> +	if (a->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
> +		ctx->enc_params.framerate_num =
> +			a->parm.output.timeperframe.denominator;
> +		ctx->enc_params.framerate_denom =
> +			a->parm.output.timeperframe.numerator;
> +		ctx->param_change |= MTK_ENCODE_PARAM_FRAMERATE;
> +	} else {
> +		return -EINVAL;
> +	}

I'd invert the test:

	if (a->type != V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE)
		return -EINVAL;

and now you can just set ctx->enc_params.

> +	return 0;
> +}

And if there is an s_parm, then there should be a g_parm as well!

> +
> +static struct mtk_q_data *mtk_venc_get_q_data(struct mtk_vcodec_ctx *ctx,
> +					      enum v4l2_buf_type type)
> +{
> +	if (V4L2_TYPE_IS_OUTPUT(type))
> +		return &ctx->q_data[MTK_Q_DATA_SRC];
> +
> +	return &ctx->q_data[MTK_Q_DATA_DST];
> +}
> +
> +static struct mtk_video_fmt *mtk_venc_find_format(struct v4l2_format *f)
> +{
> +	struct mtk_video_fmt *fmt;
> +	unsigned int k;
> +
> +	for (k = 0; k < NUM_FORMATS; k++) {
> +		fmt = &mtk_video_formats[k];
> +		if (fmt->fourcc == f->fmt.pix.pixelformat)
> +			return fmt;
> +	}
> +
> +	return NULL;
> +}
> +
> +static void mtk_vcodec_enc_calc_src_size(
> +	unsigned int num_planes, unsigned int pic_width,
> +	unsigned int pic_height, unsigned int sizeimage[],
> +	unsigned int bytesperline[])
> +{
> +	unsigned int y_pitch_w_div16;
> +	unsigned int c_pitch_w_div16;
> +
> +	y_pitch_w_div16 = ALIGN(pic_width, 16) >> 4;
> +	c_pitch_w_div16 = ALIGN(pic_width, 16) >> 4;
> +
> +	if (num_planes == 2) {
> +		sizeimage[0] =
> +			(y_pitch_w_div16) * (((pic_height + 31) / 32) * 2) * 256 +
> +			((y_pitch_w_div16 % 8 == 0) ? 0 : ((ALIGN(pic_width, 16) * 2) * 16));
> +
> +		sizeimage[1] =
> +			(c_pitch_w_div16) * (((pic_height + 31) / 32) * 2) * 128 +
> +			((c_pitch_w_div16 % 8 == 0) ? 0 : (ALIGN(pic_width, 16) * 16));
> +
> +		sizeimage[2] = 0;
> +
> +		bytesperline[0] = ALIGN(pic_width, 16);
> +		bytesperline[1] = ALIGN(pic_width, 16);
> +		bytesperline[2] = 0;
> +
> +	} else {
> +		sizeimage[0] =
> +			(y_pitch_w_div16) * (((pic_height + 31) / 32) * 2) * 256 +
> +			((y_pitch_w_div16 % 8 == 0) ? 0 : ((ALIGN(pic_width, 16) * 2) * 16));
> +
> +		sizeimage[1] =
> +			(c_pitch_w_div16) * (((pic_height + 31) / 32) * 2) * 64 +
> +			((c_pitch_w_div16 % 8 == 0) ? 0 : ((ALIGN(pic_width, 16) / 2) * 16));
> +
> +		sizeimage[2] =
> +			(c_pitch_w_div16) * (((pic_height + 31) / 32) * 2) * 64 +
> +			((c_pitch_w_div16 % 8 == 0) ? 0 : ((ALIGN(pic_width, 16) / 2) * 16));
> +
> +		bytesperline[0] = ALIGN(pic_width, 16);
> +		bytesperline[1] = ALIGN(pic_width, 16) / 2;
> +		bytesperline[2] = ALIGN(pic_width, 16) / 2;
> +	}
> +}
> +
> +static int vidioc_try_fmt(struct v4l2_format *f, struct mtk_video_fmt *fmt)
> +{
> +	struct v4l2_pix_format_mplane *pix_fmt_mp = &f->fmt.pix_mp;
> +
> +	/* V4L2 specification suggests the driver corrects the format struct
> +	  * if any of the dimensions is unsupported */
> +        if (pix_fmt_mp->height < MTK_VENC_MIN_H)
> +                pix_fmt_mp->height = MTK_VENC_MIN_H;
> +        else if (pix_fmt_mp->height > MTK_VENC_MAX_H)
> +                pix_fmt_mp->height = MTK_VENC_MAX_H;
> +
> +        if (pix_fmt_mp->width < MTK_VENC_MIN_W)
> +                pix_fmt_mp->width = MTK_VENC_MIN_W;
> +        else if (pix_fmt_mp->width > MTK_VENC_MAX_W)
> +                pix_fmt_mp->width = MTK_VENC_MAX_W;

Use the clamp macro for this.

> +
> +        pix_fmt_mp->field = V4L2_FIELD_NONE;
> +
> +	if (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
> +		int size = pix_fmt_mp->height * pix_fmt_mp->width;

Add an empty line.

> +		if (fmt->num_planes != pix_fmt_mp->num_planes)
> +			pix_fmt_mp->num_planes = fmt->num_planes;
> +
> +		if(pix_fmt_mp->plane_fmt[0].sizeimage != size)

Add space after if. Again, checkpatch should have complained about that.
Please run checkpatch first before posting the next version.

> +			pix_fmt_mp->plane_fmt[0].sizeimage = size;
> + 		pix_fmt_mp->plane_fmt[0].bytesperline = 0;
> +		memset(&(pix_fmt_mp->plane_fmt[0].reserved[0]), 0x0,
> +			sizeof(pix_fmt_mp->plane_fmt[0].reserved));
> +	} else if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
> +		int i;
> +		unsigned int sizeimage[VIDEO_MAX_PLANES];
> +		unsigned int bytesperline[VIDEO_MAX_PLANES];
> +
> +		v4l_bound_align_image(&pix_fmt_mp->width, 8, 1920, 1,
> +				      &pix_fmt_mp->height, 4, 1080, 1, 0);
> +
> +		if (fmt->num_planes != pix_fmt_mp->num_planes)
> +			pix_fmt_mp->num_planes = fmt->num_planes;
> +
> +		mtk_vcodec_enc_calc_src_size(pix_fmt_mp->num_planes,
> +					pix_fmt_mp->width,
> +					pix_fmt_mp->height,
> +					sizeimage, bytesperline);
> +
> +		for (i=0; i < pix_fmt_mp->num_planes; i++) {

Spaces around '='.

> +			pix_fmt_mp->plane_fmt[i].sizeimage = sizeimage[i];
> +			pix_fmt_mp->plane_fmt[i].bytesperline = bytesperline[i];
> +			memset(&(pix_fmt_mp->plane_fmt[i].reserved[0]), 0x0,
> +				sizeof(pix_fmt_mp->plane_fmt[0].reserved));
> +		}
> +	} else {
> +		return -EINVAL;
> +	}
> +
> +	pix_fmt_mp->flags = 0;
> +	pix_fmt_mp->ycbcr_enc = 0;
> +	pix_fmt_mp->quantization = 0;
> +	pix_fmt_mp->xfer_func = 0;
> +	memset(&pix_fmt_mp->reserved[0], 0x0, sizeof(pix_fmt_mp->reserved));
> +
> +	return 0;
> +}
> +
> +static void mtk_venc_set_param(struct mtk_vcodec_ctx *ctx, void *param)
> +{
> +	struct venc_enc_prm *p = (struct venc_enc_prm *)param;
> +	struct mtk_q_data *q_data_src = &ctx->q_data[MTK_Q_DATA_SRC];
> +	struct mtk_enc_params *enc_params = &ctx->enc_params;
> +	unsigned int frame_rate;
> +
> +	frame_rate = enc_params->framerate_num / enc_params->framerate_denom;
> +
> +	switch (q_data_src->fmt->fourcc) {
> +	case V4L2_PIX_FMT_YUV420:
> +	case V4L2_PIX_FMT_YUV420M:
> +		p->input_fourcc = VENC_YUV_FORMAT_420;
> +		break;
> +	case V4L2_PIX_FMT_YVU420:
> +	case V4L2_PIX_FMT_YVU420M:
> +		p->input_fourcc = VENC_YUV_FORMAT_YV12;
> +		break;
> +	case V4L2_PIX_FMT_NV12:
> +	case V4L2_PIX_FMT_NV12M:
> +		p->input_fourcc = VENC_YUV_FORMAT_NV12;
> +		break;
> +	case V4L2_PIX_FMT_NV21:
> +	case V4L2_PIX_FMT_NV21M:
> +		p->input_fourcc = VENC_YUV_FORMAT_NV21;
> +		break;
> +	}
> +	p->h264_profile = enc_params->h264_profile;
> +	p->h264_level = enc_params->h264_level;
> +	p->width = q_data_src->width;
> +	p->height = q_data_src->height;
> +	p->buf_width = q_data_src->bytesperline[0];
> +	p->buf_height = ((q_data_src->height + 0xf) & (~0xf));
> +	p->frm_rate = frame_rate;
> +	p->intra_period = enc_params->gop_size;
> +	p->bitrate = enc_params->bitrate;
> +
> +	ctx->param_change = MTK_ENCODE_PARAM_NONE;
> +
> +	mtk_v4l2_debug(1, "fmt 0x%x, P/L %d/%d, w/h %d/%d, buf %d/%d, fps/bps %d/%d, gop %d",
> +		       p->input_fourcc, p->h264_profile, p->h264_level,
> +		       p->width, p->height, p->buf_width, p->buf_height,
> +		       p->frm_rate, p->bitrate, p->intra_period);
> +}
> +
> +static int vidioc_venc_s_fmt(struct file *file, void *priv,
> +			     struct v4l2_format *f)

I am not convinced that combining capture and output in one function is the
most readable. I think you are better off with separate functions. Too much is
different between the two.

> +{
> +	struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
> +	struct v4l2_pix_format_mplane *pix_fmt_mp = &f->fmt.pix_mp;
> +	struct vb2_queue *vq;
> +	struct mtk_q_data *q_data;
> +	struct venc_enc_prm param;
> +	int i, ret;
> +	struct mtk_video_fmt *fmt;
> +
> +	vq = v4l2_m2m_get_vq(ctx->m2m_ctx, f->type);
> +	if (!vq) {
> +		mtk_v4l2_err("fail to get vq\n");
> +		return -EINVAL;
> +	}
> +
> +	if (vb2_is_busy(vq)) {
> +		mtk_v4l2_err("queue busy\n");
> +		return -EBUSY;
> +	}
> +
> +	q_data = mtk_venc_get_q_data(ctx, f->type);
> +	if (!q_data) {
> +		mtk_v4l2_err("fail to get q data\n");
> +		return -EINVAL;
> +	}
> +
> +	fmt = mtk_venc_find_format(f);
> +	if (!fmt) {
> +		if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
> +			f->fmt.pix.pixelformat = mtk_video_formats[0].fourcc;
> +			fmt = mtk_venc_find_format(f);
> +		} else {
> +			f->fmt.pix.pixelformat = mtk_video_formats[8].fourcc;
> +			fmt = mtk_venc_find_format(f);
> +		}
> +	}
> +
> +	q_data->fmt = fmt;
> +	ret = vidioc_try_fmt(f, q_data->fmt);
> +	if (ret)
> +		return ret;
> +
> +	q_data->width		= f->fmt.pix_mp.width;
> +	q_data->height		= f->fmt.pix_mp.height;
> +	q_data->field		= f->fmt.pix_mp.field;
> +
> +	if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
> +		q_data->colorspace = f->fmt.pix_mp.colorspace;
> +		ctx->q_data[MTK_Q_DATA_SRC].bytesperline[0] =
> +			ALIGN(q_data->width, 16);
> +
> +		if (q_data->fmt->num_planes == 2) {
> +			ctx->q_data[MTK_Q_DATA_SRC].bytesperline[1] =
> +				ALIGN(q_data->width, 16);
> +			ctx->q_data[MTK_Q_DATA_SRC].bytesperline[2] = 0;
> +		} else {
> +			ctx->q_data[MTK_Q_DATA_SRC].bytesperline[1] =
> +				ALIGN(q_data->width, 16) / 2;
> +			ctx->q_data[MTK_Q_DATA_SRC].bytesperline[2] =
> +				ALIGN(q_data->width, 16) / 2;
> +		}
> +
> +		memset(&param, 0, sizeof(param));
> +		mtk_venc_set_param(ctx, &param);
> +		if (ctx->state == MTK_STATE_INIT) {
> +			ret = venc_if_set_param(ctx,
> +						VENC_SET_PARAM_ENC,
> +						&param);
> +			if (ret)
> +				mtk_v4l2_err("venc_if_set_param failed=%d\n",
> +						ret);
> +
> +			/* Get codec driver advice sizeimage from vpu */
> +			for (i = 0; i < MTK_VCODEC_MAX_PLANES; i++) {
> +				q_data->sizeimage[i] = param.sizeimage[i];
> +				pix_fmt_mp->plane_fmt[i].sizeimage =
> +					param.sizeimage[i];
> +			}
> +			q_data->bytesperline[0] =
> +				pix_fmt_mp->plane_fmt[0].bytesperline;
> +			q_data->bytesperline[1] =
> +				pix_fmt_mp->plane_fmt[1].bytesperline;
> +			q_data->bytesperline[2] =
> +				pix_fmt_mp->plane_fmt[2].bytesperline;
> +		} else {
> +			for (i = 0; i < MTK_VCODEC_MAX_PLANES; i++) {
> +				q_data->sizeimage[i] =
> +					pix_fmt_mp->plane_fmt[i].sizeimage;
> +				q_data->bytesperline[i] =
> +					pix_fmt_mp->plane_fmt[i].bytesperline;
> +			}
> +		}
> +	} else if (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE){
> +		for (i = 0; i < f->fmt.pix_mp.num_planes; i++) {
> +			struct v4l2_plane_pix_format	*plane_fmt;
> +
> +			plane_fmt = &f->fmt.pix_mp.plane_fmt[i];
> +			q_data->bytesperline[i]	= plane_fmt->bytesperline;
> +			q_data->sizeimage[i]	= plane_fmt->sizeimage;
> +		}
> +
> +		if (ctx->state == MTK_STATE_FREE) {
> +			ret = venc_if_create(ctx, q_data->fmt->fourcc);
> +			if (ret) {
> +				mtk_v4l2_err("venc_if_create failed=%d, codec type=%x\n",
> +					ret, q_data->fmt->fourcc);
> +				return 0;
> +			}
> +
> +			ctx->state = MTK_STATE_INIT;
> +		}
> +	}
> +
> +	return 0;
> +}
> +
> +static int vidioc_venc_g_fmt(struct file *file, void *priv,
> +			     struct v4l2_format *f)
> +{
> +	struct v4l2_pix_format_mplane *pix = &f->fmt.pix_mp;
> +	struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
> +	struct vb2_queue *vq;
> +	struct mtk_q_data *q_data;
> +	int i;
> +
> +	vq = v4l2_m2m_get_vq(ctx->m2m_ctx, f->type);
> +	if (!vq)
> +		return -EINVAL;
> +
> +	q_data = mtk_venc_get_q_data(ctx, f->type);
> +
> +	pix->width = q_data->width;
> +	pix->height = q_data->height;
> +	pix->pixelformat = q_data->fmt->fourcc;
> +	pix->field = q_data->field;
> +	pix->colorspace = q_data->colorspace;
> +	pix->num_planes = q_data->fmt->num_planes;
> +	for (i = 0; i < pix->num_planes; i++) {
> +		pix->plane_fmt[i].bytesperline = q_data->bytesperline[i];
> +		pix->plane_fmt[i].sizeimage = q_data->sizeimage[i];
> +		memset(&(pix->plane_fmt[i].reserved[0]), 0x0,
> +			sizeof(pix->plane_fmt[i].reserved));
> +	}
> +	pix->flags = 0;
> +	pix->ycbcr_enc = 0;
> +	pix->quantization = 0;
> +	pix->xfer_func = 0;
> +	memset(&pix->reserved[0], 0x0, sizeof(pix->reserved));
> +
> +	return 0;
> +}
> +
> +static int vidioc_try_fmt_vid_cap_mplane(struct file *file, void *priv,
> +                                  struct v4l2_format *f)
> +{
> +        struct mtk_video_fmt *fmt;
> +	struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
> +
> +        fmt = mtk_venc_find_format(f);
> +        if (!fmt) {
> +                f->fmt.pix.pixelformat = mtk_video_formats[8].fourcc;
> +                fmt = mtk_venc_find_format(f);
> +        }
> +        if (fmt->type != MTK_FMT_ENC) {
> +		mtk_v4l2_err("Fourcc format (0x%08x) invalid.\n",
> +			     f->fmt.pix.pixelformat);
> +		return -EINVAL;
> +        }
> +        f->fmt.pix_mp.colorspace = ctx->q_data[MTK_Q_DATA_SRC].colorspace;
> +
> +        return vidioc_try_fmt(f, fmt);
> +}
> +
> +static int vidioc_try_fmt_vid_out_mplane(struct file *file, void *priv,
> +                                  struct v4l2_format *f)
> +{
> +        struct mtk_video_fmt *fmt;
> +
> +        fmt = mtk_venc_find_format(f);
> +        if (!fmt) {
> +                f->fmt.pix.pixelformat = mtk_video_formats[0].fourcc;
> +                fmt = mtk_venc_find_format(f);
> +        }
> +        if (!(fmt->type & MTK_FMT_FRAME)) {
> +		mtk_v4l2_err("Fourcc format (0x%08x) invalid.\n",
> +			     f->fmt.pix.pixelformat);
> +		return -EINVAL;
> +        }
> +        if (!f->fmt.pix_mp.colorspace)
> +                f->fmt.pix_mp.colorspace = V4L2_COLORSPACE_REC709;
> +
> +        return vidioc_try_fmt(f, fmt);
> +}
> +
> +static int vidioc_venc_g_s_selection(struct file *file, void *priv,
> +                                struct v4l2_selection *s)

Why support s_selection if you can only return the current width and height?
And why support g_selection if you can't change the selection?

In other words, why implement this at all?

Unless I am missing something here, I would just drop this.

> +{
> +	struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
> +	struct mtk_q_data *q_data;
> +
> +	if (V4L2_TYPE_IS_OUTPUT(s->type)) {
> +		if (s->target !=  V4L2_SEL_TGT_COMPOSE)
> +			return -EINVAL;
> +	} else {
> +		if (s->target != V4L2_SEL_TGT_CROP)
> +			return -EINVAL;
> +	}
> +
> +	if (s->r.left || s->r.top)
> +		return -EINVAL;
> +
> +	q_data = mtk_venc_get_q_data(ctx, s->type);
> +	if (!q_data)
> +		return -EINVAL;
> +
> +	s->r.width = q_data->width;
> +	s->r.height = q_data->height;
> +
> +	return 0;
> +}
> +
> +
> +static int vidioc_venc_qbuf(struct file *file, void *priv,
> +			    struct v4l2_buffer *buf)
> +{
> +
> +	struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
> +
> +	if (ctx->state == MTK_STATE_ABORT) {
> +		mtk_v4l2_err("[%d] Call on QBUF after unrecoverable error\n", ctx->idx);
> +		return -EIO;
> +	}
> +
> +	return v4l2_m2m_qbuf(file, ctx->m2m_ctx, buf);
> +}
> +
> +static int vidioc_venc_dqbuf(struct file *file, void *priv,
> +			     struct v4l2_buffer *buf)
> +{
> +	struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
> +	if (ctx->state == MTK_STATE_ABORT) {
> +		mtk_v4l2_err("[%d] Call on QBUF after unrecoverable error\n", ctx->idx);
> +		return -EIO;
> +	}
> +
> +	return v4l2_m2m_dqbuf(file, ctx->m2m_ctx, buf);
> +}
> +
> +
> +const struct v4l2_ioctl_ops mtk_venc_ioctl_ops = {
> +	.vidioc_streamon		= v4l2_m2m_ioctl_streamon,
> +	.vidioc_streamoff		= v4l2_m2m_ioctl_streamoff,
> +
> +	.vidioc_reqbufs			= v4l2_m2m_ioctl_reqbufs,
> +	.vidioc_querybuf		= v4l2_m2m_ioctl_querybuf,
> +	.vidioc_qbuf			= vidioc_venc_qbuf,
> +	.vidioc_dqbuf			= vidioc_venc_dqbuf,
> +
> +	.vidioc_querycap		= vidioc_venc_querycap,
> +	.vidioc_enum_fmt_vid_cap_mplane = vidioc_enum_fmt_vid_cap_mplane,
> +	.vidioc_enum_fmt_vid_out_mplane = vidioc_enum_fmt_vid_out_mplane,
> +	.vidioc_enum_framesizes		= vidioc_enum_framesizes,
> +
> +	.vidioc_try_fmt_vid_cap_mplane	= vidioc_try_fmt_vid_cap_mplane,
> +	.vidioc_try_fmt_vid_out_mplane	= vidioc_try_fmt_vid_out_mplane,
> +	.vidioc_expbuf			= v4l2_m2m_ioctl_expbuf,

Please add vidioc_create_bufs and vidioc_prepare_buf as well.

> +	.vidioc_subscribe_event		= v4l2_ctrl_subscribe_event,
> +	.vidioc_unsubscribe_event	= v4l2_event_unsubscribe,
> +
> +	.vidioc_s_parm			= vidioc_venc_s_parm,
> +
> +	.vidioc_s_fmt_vid_cap_mplane	= vidioc_venc_s_fmt,
> +	.vidioc_s_fmt_vid_out_mplane	= vidioc_venc_s_fmt,
> +
> +	.vidioc_g_fmt_vid_cap_mplane	= vidioc_venc_g_fmt,
> +	.vidioc_g_fmt_vid_out_mplane	= vidioc_venc_g_fmt,
> +
> +	.vidioc_g_selection		= vidioc_venc_g_s_selection,
> +	.vidioc_s_selection		= vidioc_venc_g_s_selection,
> +};
> +
> +static int vb2ops_venc_queue_setup(struct vb2_queue *vq,
> +				   const void *parg,
> +				   unsigned int *nbuffers,
> +				   unsigned int *nplanes,
> +				   unsigned int sizes[], void *alloc_ctxs[])
> +{
> +	struct mtk_vcodec_ctx *ctx = vb2_get_drv_priv(vq);
> +	struct mtk_q_data *q_data;
> +
> +	q_data = mtk_venc_get_q_data(ctx, vq->type);
> +
> +	if (*nbuffers < 1)
> +		*nbuffers = 1;

*nbuffers can never be 0, so no need to check.

> +	if (*nbuffers > MTK_VIDEO_MAX_FRAME)

That should be (q->num_buffers + *nbuffers > MTK_VIDEO_MAX_FRAME)

> +		*nbuffers = MTK_VIDEO_MAX_FRAME;

and *nbuffers = MTK_VIDEO_MAX_FRAME - q->num_buffers;

And you will need a check if q->num_buffers == MTK_VIDEO_MAX_FRAME) too
(otherwise *nbuffers could become <= 0).

In order to correctly handle create_bufs you will need to make a few changes
here. Read the queue_setup description in include/media/videobuf2-core.h and
look at queue_setup() in Documentation/video4linux/v4l2-pci-skeleton.c.

> +
> +	*nplanes = q_data->fmt->num_planes;
> +
> +	if (vq->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
> +		unsigned int i;
> +
> +		for (i = 0; i < *nplanes; i++) {
> +			sizes[i] = q_data->sizeimage[i];
> +			alloc_ctxs[i] = ctx->dev->alloc_ctx;
> +		}
> +	} else if (vq->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
> +		sizes[0] = q_data->sizeimage[0];
> +		alloc_ctxs[0] = ctx->dev->alloc_ctx;
> +	} else {
> +		return -EINVAL;
> +	}
> +
> +	mtk_v4l2_debug(2, "[%d]get %d buffer(s) of size 0x%x each, vq->memory=%d",
> +		       ctx->idx, *nbuffers, sizes[0], vq->memory);
> +
> +	return 0;
> +}
> +
> +static int vb2ops_venc_buf_prepare(struct vb2_buffer *vb)
> +{
> +	struct mtk_vcodec_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue);
> +	struct mtk_q_data *q_data;
> +	int i;
> +
> +	q_data = mtk_venc_get_q_data(ctx, vb->vb2_queue->type);
> +
> +	for (i = 0; i < q_data->fmt->num_planes; i++) {
> +		if (vb2_plane_size(vb, i) < q_data->sizeimage[i]) {
> +			mtk_v4l2_debug(2, "data will not fit into plane %d (%lu < %d)",
> +				       i, vb2_plane_size(vb, i),
> +				       q_data->sizeimage[i]);
> +			return -EINVAL;
> +		}
> +	}
> +
> +	return 0;
> +}
> +
> +static void vb2ops_venc_buf_queue(struct vb2_buffer *vb)
> +{
> +	struct mtk_vcodec_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue);
> +	struct vb2_v4l2_buffer *vb2_v4l2 =
> +			container_of(vb, struct vb2_v4l2_buffer, vb2_buf);
> +	struct mtk_video_enc_buf *mtk_buf =
> +			container_of(vb2_v4l2, struct mtk_video_enc_buf, vb);
> +
> +	if ((vb->vb2_queue->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) &&
> +		(ctx->param_change != MTK_ENCODE_PARAM_NONE)) {
> +		mtk_v4l2_debug(1,
> +				"[%d] Before id=%d encode parameter change %x",
> +				ctx->idx, vb->index,
> +				ctx->param_change);
> +		mtk_buf->param_change = ctx->param_change;
> +		if (mtk_buf->param_change & MTK_ENCODE_PARAM_BITRATE) {
> +			mtk_buf->enc_params.bitrate = ctx->enc_params.bitrate;
> +			mtk_v4l2_debug(1, "[%d] idx=%d change param br=%d",
> +				ctx->idx,
> +				mtk_buf->vb.vb2_buf.index,
> +				mtk_buf->enc_params.bitrate);
> +		}
> +		if (ctx->param_change & MTK_ENCODE_PARAM_FRAMERATE) {
> +			mtk_buf->enc_params.framerate_num =
> +				ctx->enc_params.framerate_num;
> +			mtk_buf->enc_params.framerate_denom =
> +				ctx->enc_params.framerate_denom;
> +			mtk_v4l2_debug(1, "[%d] idx=%d, change param fr=%d/%d",
> +					ctx->idx,
> +					mtk_buf->vb.vb2_buf.index,
> +					mtk_buf->enc_params.framerate_num,
> +					mtk_buf->enc_params.framerate_denom);
> +		}
> +		if (ctx->param_change & MTK_ENCODE_PARAM_INTRA_PERIOD) {
> +			mtk_buf->enc_params.gop_size = ctx->enc_params.gop_size;
> +			mtk_v4l2_debug(1, "[%d] idx=%d, change param intra period=%d",
> +					ctx->idx,
> +					mtk_buf->vb.vb2_buf.index,
> +					mtk_buf->enc_params.gop_size);
> +		}
> +		if (ctx->param_change & MTK_ENCODE_PARAM_FRAME_TYPE) {
> +			mtk_buf->enc_params.force_intra =
> +				ctx->enc_params.force_intra;
> +			mtk_v4l2_debug(1, "[%d] idx=%d, change param force I=%d",
> +					ctx->idx,
> +					mtk_buf->vb.vb2_buf.index,
> +					mtk_buf->enc_params.force_intra);
> +		}
> +		ctx->param_change = MTK_ENCODE_PARAM_NONE;
> +	}
> +
> +	v4l2_m2m_buf_queue(ctx->m2m_ctx, to_vb2_v4l2_buffer(vb));
> +}
> +
> +static int vb2ops_venc_start_streaming(struct vb2_queue *q, unsigned int count)
> +{
> +	struct mtk_vcodec_ctx *ctx = vb2_get_drv_priv(q);
> +	struct venc_enc_prm param;
> +	int ret;
> +	int i;
> +
> +	/* Once state turn into MTK_STATE_ABORT, we need stop_streaming to clear it */
> +	if ((ctx->state == MTK_STATE_ABORT) || (ctx->state == MTK_STATE_FREE))
> +		goto err_set_param;
> +
> +	if (!(vb2_start_streaming_called(&ctx->m2m_ctx->out_q_ctx.q) &
> +	      vb2_start_streaming_called(&ctx->m2m_ctx->cap_q_ctx.q))) {
> +		mtk_v4l2_debug(1, "[%d]-> out=%d cap=%d",
> +		 ctx->idx,
> +		 vb2_start_streaming_called(&ctx->m2m_ctx->out_q_ctx.q),
> +		 vb2_start_streaming_called(&ctx->m2m_ctx->cap_q_ctx.q));
> +		return 0;
> +	}
> +
> +	mtk_venc_set_param(ctx, &param);
> +	ret = venc_if_set_param(ctx,
> +				VENC_SET_PARAM_ENC,
> +				&param);
> +	if (ret) {
> +		mtk_v4l2_err("venc_if_set_param failed=%d\n", ret);
> +		ctx->state = MTK_STATE_ABORT;
> +		goto err_set_param;
> +	}
> +
> +	if ((ctx->q_data[MTK_Q_DATA_DST].fmt->fourcc == V4L2_PIX_FMT_H264) &&
> +	    (ctx->enc_params.seq_hdr_mode != V4L2_MPEG_VIDEO_HEADER_MODE_SEPARATE)) {
> +		ret = venc_if_set_param(ctx,
> +					VENC_SET_PARAM_PREPEND_HEADER,
> +					0);
> +		if (ret) {
> +			mtk_v4l2_err("venc_if_set_param failed=%d\n", ret);
> +			ctx->state = MTK_STATE_ABORT;
> +			goto err_set_param;
> +		}
> +		ctx->state = MTK_STATE_HEADER;
> +	}
> +
> +	return 0;
> +
> +err_set_param:
> +	for (i = 0; i < q->num_buffers; ++i) {
> +		if (q->bufs[i]->state == VB2_BUF_STATE_ACTIVE) {
> +			mtk_v4l2_debug(0, "[%d] idx=%d, type=%d, %d -> VB2_BUF_STATE_QUEUED",
> +					ctx->idx, i, q->type,
> +					(int)q->bufs[i]->state );
> +			v4l2_m2m_buf_done(to_vb2_v4l2_buffer(q->bufs[i]), VB2_BUF_STATE_QUEUED);
> +		}
> +	}
> +
> +	return -EINVAL;
> +}
> +
> +static void vb2ops_venc_stop_streaming(struct vb2_queue *q)
> +{
> +	struct mtk_vcodec_ctx *ctx = vb2_get_drv_priv(q);
> +	struct vb2_buffer *src_buf, *dst_buf;
> +	int ret;
> +
> +	mtk_v4l2_debug(2, "[%d]-> type=%d", ctx->idx, q->type);
> +
> +	if (q->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
> +		while ((dst_buf = v4l2_m2m_dst_buf_remove(ctx->m2m_ctx))) {
> +			dst_buf->planes[0].bytesused = 0;
> +			v4l2_m2m_buf_done(to_vb2_v4l2_buffer(dst_buf),
> +						VB2_BUF_STATE_ERROR);
> +		}
> +	} else {
> +		while ((src_buf = v4l2_m2m_src_buf_remove(ctx->m2m_ctx)))
> +			v4l2_m2m_buf_done(to_vb2_v4l2_buffer(src_buf),
> +						VB2_BUF_STATE_ERROR);
> +	}
> +
> +	if ((q->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE &&
> +	     vb2_is_streaming(&ctx->m2m_ctx->out_q_ctx.q)) ||
> +	    (q->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE &&
> +	     vb2_is_streaming(&ctx->m2m_ctx->cap_q_ctx.q))) {
> +		mtk_v4l2_debug(1, "[%d]-> q type %d out=%d cap=%d",
> +			       ctx->idx, q->type,
> +			       vb2_is_streaming(&ctx->m2m_ctx->out_q_ctx.q),
> +			       vb2_is_streaming(&ctx->m2m_ctx->cap_q_ctx.q));
> +		return;
> +	}
> +
> +	ret = venc_if_release(ctx);
> +	if (ret)
> +		mtk_v4l2_err("venc_if_release failed=%d\n", ret);
> +
> +	ctx->state = MTK_STATE_FREE;
> +}
> +
> +static struct vb2_ops mtk_venc_vb2_ops = {
> +	.queue_setup			= vb2ops_venc_queue_setup,
> +	.buf_prepare			= vb2ops_venc_buf_prepare,
> +	.buf_queue			= vb2ops_venc_buf_queue,
> +	.wait_prepare			= vb2_ops_wait_prepare,
> +	.wait_finish			= vb2_ops_wait_finish,
> +	.start_streaming		= vb2ops_venc_start_streaming,
> +	.stop_streaming			= vb2ops_venc_stop_streaming,
> +};
> +
> +static int mtk_venc_encode_header(void *priv)
> +{
> +	struct mtk_vcodec_ctx *ctx = priv;
> +	int ret;
> +	struct vb2_buffer *dst_buf;
> +	struct mtk_vcodec_mem bs_buf;
> +	struct venc_done_result enc_result;
> +
> +	dst_buf = v4l2_m2m_dst_buf_remove(ctx->m2m_ctx);
> +	if (!dst_buf) {
> +		mtk_v4l2_debug(1, "No dst buffer");
> +		return -EINVAL;
> +	}
> +
> +	bs_buf.va = vb2_plane_vaddr(dst_buf, 0);
> +	bs_buf.dma_addr = vb2_dma_contig_plane_dma_addr(dst_buf, 0);
> +	bs_buf.size = (unsigned int)dst_buf->planes[0].length;
> +
> +	mtk_v4l2_debug(1,
> +			"[%d] buf idx=%d va=0x%p dma_addr=0x%llx size=0x%lx",
> +			ctx->idx,
> +			dst_buf->index, bs_buf.va,
> +			(u64)bs_buf.dma_addr,
> +			bs_buf.size);
> +
> +	ret = venc_if_encode(ctx,
> +			VENC_START_OPT_ENCODE_SEQUENCE_HEADER,
> +			0, &bs_buf, &enc_result);
> +
> +	if (ret) {
> +		dst_buf->planes[0].bytesused = 0;
> +		ctx->state = MTK_STATE_ABORT;
> +		v4l2_m2m_buf_done(to_vb2_v4l2_buffer(dst_buf), VB2_BUF_STATE_ERROR);
> +		mtk_v4l2_err("venc_if_encode failed=%d", ret);
> +		return -EINVAL;
> +	}
> +
> +	ctx->state = MTK_STATE_HEADER;
> +	dst_buf->planes[0].bytesused = enc_result.bs_size;
> +
> +#if defined(DEBUG)
> +{
> +	int i;
> +	mtk_v4l2_debug(1, "[%d] venc_if_encode header len=%d",
> +			ctx->idx,
> +			enc_result.bs_size);
> +	for (i = 0; i < enc_result.bs_size; i++) {
> +		unsigned char *p = (unsigned char *)bs_buf.va;
> +
> +		mtk_v4l2_debug(1, "[%d] buf[%d]=0x%2x", ctx->idx, i, p[i]);
> +	}
> +}
> +#endif
> +	v4l2_m2m_buf_done(to_vb2_v4l2_buffer(dst_buf), VB2_BUF_STATE_DONE);
> +
> +	return 0;
> +}
> +
> +static int mtk_venc_param_change(struct mtk_vcodec_ctx *ctx, void *priv)
> +{
> +	struct vb2_buffer *vb = priv;
> +	struct vb2_v4l2_buffer *vb2_v4l2 =
> +			container_of(vb, struct vb2_v4l2_buffer, vb2_buf);
> +	struct mtk_video_enc_buf *mtk_buf =
> +			container_of(vb2_v4l2, struct mtk_video_enc_buf, vb);
> +	int ret = 0;
> +
> +	if (mtk_buf->param_change == MTK_ENCODE_PARAM_NONE)
> +		return 0;
> +
> +	mtk_v4l2_debug(1, "encode parameters change id=%d", vb->index);
> +	if (mtk_buf->param_change & MTK_ENCODE_PARAM_BITRATE) {
> +		struct venc_enc_prm enc_prm;
> +
> +		enc_prm.bitrate = mtk_buf->enc_params.bitrate;
> +		mtk_v4l2_debug(1, "[%d] idx=%d, change param br=%d",
> +				ctx->idx,
> +				mtk_buf->vb.vb2_buf.index,
> +				enc_prm.bitrate);
> +		ret |= venc_if_set_param(ctx,
> +					 VENC_SET_PARAM_ADJUST_BITRATE,
> +					 &enc_prm);
> +	}
> +	if (mtk_buf->param_change & MTK_ENCODE_PARAM_FRAMERATE) {
> +		struct venc_enc_prm enc_prm;
> +
> +		enc_prm.frm_rate = mtk_buf->enc_params.framerate_num /
> +				   mtk_buf->enc_params.framerate_denom;
> +		mtk_v4l2_debug(1, "[%d] idx=%d, change param fr=%d",
> +			       ctx->idx,
> +			       mtk_buf->vb.vb2_buf.index,
> +			       enc_prm.frm_rate);
> +		ret |= venc_if_set_param(ctx,
> +					 VENC_SET_PARAM_ADJUST_FRAMERATE,
> +					 &enc_prm);
> +	}
> +	if (mtk_buf->param_change & MTK_ENCODE_PARAM_INTRA_PERIOD) {
> +		mtk_v4l2_debug(1, "change param intra period=%d",
> +				 mtk_buf->enc_params.gop_size);
> +		ret |= venc_if_set_param(ctx,
> +					 VENC_SET_PARAM_I_FRAME_INTERVAL,
> +					 &mtk_buf->enc_params.gop_size);
> +	}
> +	if (mtk_buf->param_change & MTK_ENCODE_PARAM_FRAME_TYPE) {
> +		mtk_v4l2_debug(1, "[%d] idx=%d, change param force I=%d",
> +				ctx->idx,
> +				mtk_buf->vb.vb2_buf.index,
> +				mtk_buf->enc_params.force_intra);
> +		if (mtk_buf->enc_params.force_intra)
> +			ret |= venc_if_set_param(ctx,
> +						 VENC_SET_PARAM_FORCE_INTRA,
> +						 0);
> +	}
> +
> +	mtk_buf->param_change = MTK_ENCODE_PARAM_NONE;
> +
> +	if (ret) {
> +		ctx->state = MTK_STATE_ABORT;
> +		mtk_v4l2_err("venc_if_set_param %d failed=%d\n",
> +			MTK_ENCODE_PARAM_FRAME_TYPE, ret);
> +		return -1;
> +	}
> +
> +	return 0;
> +}
> +
> +static void mtk_venc_worker(struct work_struct *work)
> +{
> +	struct mtk_vcodec_ctx *ctx = container_of(work, struct mtk_vcodec_ctx,
> +				    encode_work);
> +	struct vb2_buffer *src_buf, *dst_buf;
> +	struct venc_frm_buf frm_buf;
> +	struct mtk_vcodec_mem bs_buf;
> +	struct venc_done_result enc_result;
> +	int ret;
> +	struct vb2_v4l2_buffer *v4l2_vb;
> +
> +	if ((ctx->q_data[MTK_Q_DATA_DST].fmt->fourcc == V4L2_PIX_FMT_H264) &&
> +	    (ctx->state != MTK_STATE_HEADER)) {
> +		/* encode h264 sps/pps header */
> +		mtk_venc_encode_header(ctx);
> +		v4l2_m2m_job_finish(ctx->dev->m2m_dev_enc, ctx->m2m_ctx);
> +		return;
> +	}
> +
> +	src_buf = v4l2_m2m_next_src_buf(ctx->m2m_ctx);
> +	if (!src_buf) {
> +		v4l2_m2m_job_finish(ctx->dev->m2m_dev_enc, ctx->m2m_ctx);
> +		return;
> +	}
> +
> +	dst_buf = v4l2_m2m_dst_buf_remove(ctx->m2m_ctx);
> +	if (!dst_buf) {
> +		v4l2_m2m_job_finish(ctx->dev->m2m_dev_enc, ctx->m2m_ctx);
> +		return;
> +	}
> +
> +	mtk_venc_param_change(ctx, src_buf);
> +
> +	frm_buf.fb_addr.va = vb2_plane_vaddr(src_buf, 0);
> +	frm_buf.fb_addr.dma_addr = vb2_dma_contig_plane_dma_addr(src_buf, 0);
> +	frm_buf.fb_addr.size = (unsigned int)src_buf->planes[0].length;
> +	frm_buf.fb_addr1.va = vb2_plane_vaddr(src_buf, 1);
> +	frm_buf.fb_addr1.dma_addr = vb2_dma_contig_plane_dma_addr(src_buf, 1);
> +	frm_buf.fb_addr1.size = (unsigned int)src_buf->planes[1].length;
> +	if (src_buf->num_planes == 3) {
> +		frm_buf.fb_addr2.va = vb2_plane_vaddr(src_buf, 2);
> +		frm_buf.fb_addr2.dma_addr =
> +			vb2_dma_contig_plane_dma_addr(src_buf, 2);
> +		frm_buf.fb_addr2.size =
> +			(unsigned int)src_buf->planes[2].length;
> +	} else {
> +		frm_buf.fb_addr2.va = NULL;
> +		frm_buf.fb_addr2.dma_addr = 0;
> +		frm_buf.fb_addr2.size = 0;
> +	}
> +	bs_buf.va = vb2_plane_vaddr(dst_buf, 0);
> +	bs_buf.dma_addr = vb2_dma_contig_plane_dma_addr(dst_buf, 0);
> +	bs_buf.size = (unsigned int)dst_buf->planes[0].length;
> +
> +	mtk_v4l2_debug(2,
> +			"Framebuf VA=%p PA=%llx Size=0x%lx;VA=%p PA=0x%llx Size=0x%lx;VA=%p PA=0x%llx Size=0x%lx",
> +			frm_buf.fb_addr.va,
> +			(u64)frm_buf.fb_addr.dma_addr,
> +			frm_buf.fb_addr.size,
> +			frm_buf.fb_addr1.va,
> +			(u64)frm_buf.fb_addr1.dma_addr,
> +			frm_buf.fb_addr1.size,
> +			frm_buf.fb_addr2.va,
> +			(u64)frm_buf.fb_addr2.dma_addr,
> +			frm_buf.fb_addr2.size);
> +
> +	ret = venc_if_encode(ctx, VENC_START_OPT_ENCODE_FRAME,
> +			     &frm_buf, &bs_buf, &enc_result);
> +
> +	src_buf = v4l2_m2m_src_buf_remove(ctx->m2m_ctx);
> +	if (enc_result.msg == VENC_MESSAGE_OK)
> +		v4l2_m2m_buf_done(to_vb2_v4l2_buffer(src_buf), VB2_BUF_STATE_DONE);
> +	else
> +		v4l2_m2m_buf_done(to_vb2_v4l2_buffer(src_buf), VB2_BUF_STATE_ERROR);
> +
> +	if (enc_result.is_key_frm) {
> +		v4l2_vb = to_vb2_v4l2_buffer(dst_buf);
> +		v4l2_vb->flags |= V4L2_BUF_FLAG_KEYFRAME;
> +	}
> +
> +	if (ret) {
> +		dst_buf->planes[0].bytesused = 0;
> +		v4l2_m2m_buf_done(to_vb2_v4l2_buffer(dst_buf), VB2_BUF_STATE_ERROR);
> +		mtk_v4l2_err("venc_if_encode failed=%d", ret);
> +	} else {
> +		dst_buf->planes[0].bytesused = enc_result.bs_size;
> +		v4l2_m2m_buf_done(to_vb2_v4l2_buffer(dst_buf), VB2_BUF_STATE_DONE);
> +		mtk_v4l2_debug(2, "venc_if_encode bs size=%d",
> +				 enc_result.bs_size);
> +	}
> +
> +	v4l2_m2m_job_finish(ctx->dev->m2m_dev_enc, ctx->m2m_ctx);
> +
> +	mtk_v4l2_debug(1, "<=== src_buf[%d] dst_buf[%d] venc_if_encode ret=%d Size=%u===>",
> +			src_buf->index, dst_buf->index, ret,
> +			enc_result.bs_size);
> +}
> +
> +static void m2mops_venc_device_run(void *priv)
> +{
> +	struct mtk_vcodec_ctx *ctx = priv;

Add empty line.

> +	queue_work(ctx->dev->encode_workqueue, &ctx->encode_work);
> +}
> +
> +static int m2mops_venc_job_ready(void *m2m_priv)
> +{
> +	struct mtk_vcodec_ctx *ctx = m2m_priv;
> +
> +	if (!v4l2_m2m_num_dst_bufs_ready(ctx->m2m_ctx)) {
> +		mtk_v4l2_debug(3, "[%d]Not ready: not enough video dst buffers.",
> +			       ctx->idx);
> +		return 0;
> +	}
> +
> +	if (!v4l2_m2m_num_src_bufs_ready(ctx->m2m_ctx)) {
> +		mtk_v4l2_debug(3, "[%d]Not ready: not enough video src buffers.",
> +			       ctx->idx);
> +			return 0;
> +		}

Broken indentation.

> +
> +	if (ctx->state == MTK_STATE_ABORT) {
> +		mtk_v4l2_debug(3, "[%d]Not ready: state=0x%x.",
> +			       ctx->idx, ctx->state);
> +		return 0;
> +	}
> +
> +	if (ctx->state == MTK_STATE_FREE) {
> +		mtk_v4l2_debug(3, "[%d]Not ready: state=0x%x.",
> +			       ctx->idx, ctx->state);
> +		return 0;
> +	}
> +
> +	return 1;
> +}
> +
> +static void m2mops_venc_job_abort(void *priv)
> +{
> +	struct mtk_vcodec_ctx *ctx = priv;
> +
> +	ctx->state = MTK_STATE_ABORT;
> +}
> +
> +static void m2mops_venc_lock(void *m2m_priv)
> +{
> +	struct mtk_vcodec_ctx *ctx = m2m_priv;
> +
> +	mutex_lock(&ctx->dev->dev_mutex);
> +}
> +
> +static void m2mops_venc_unlock(void *m2m_priv)
> +{
> +	struct mtk_vcodec_ctx *ctx = m2m_priv;
> +
> +	mutex_unlock(&ctx->dev->dev_mutex);
> +}
> +
> +const struct v4l2_m2m_ops mtk_venc_m2m_ops = {
> +	.device_run			= m2mops_venc_device_run,
> +	.job_ready			= m2mops_venc_job_ready,
> +	.job_abort			= m2mops_venc_job_abort,
> +	.lock				= m2mops_venc_lock,
> +	.unlock				= m2mops_venc_unlock,
> +};
> +
> +#define IS_MTK_VENC_PRIV(x) ((V4L2_CTRL_ID2CLASS(x) == V4L2_CTRL_CLASS_MPEG) &&\
> +			     V4L2_CTRL_DRIVER_PRIV(x))
> +
> +void mtk_vcodec_enc_ctx_params_setup(struct mtk_vcodec_ctx *ctx)
> +{
> +	struct mtk_q_data *q_data;
> +	struct mtk_video_fmt *fmt;
> +
> +	ctx->m2m_ctx->q_lock = &ctx->dev->dev_mutex;
> +	ctx->fh.m2m_ctx = ctx->m2m_ctx;
> +	ctx->fh.ctrl_handler = &ctx->ctrl_hdl;
> +	INIT_WORK(&ctx->encode_work, mtk_venc_worker);
> +
> +	ctx->q_data[MTK_Q_DATA_SRC].width = DFT_CFG_WIDTH;
> +	ctx->q_data[MTK_Q_DATA_SRC].height = DFT_CFG_HEIGHT;
> +	ctx->q_data[MTK_Q_DATA_SRC].fmt = &mtk_video_formats[0];
> +	ctx->q_data[MTK_Q_DATA_SRC].colorspace = V4L2_COLORSPACE_REC709;
> +	ctx->q_data[MTK_Q_DATA_SRC].field = V4L2_FIELD_NONE;
> +
> +	q_data = &ctx->q_data[MTK_Q_DATA_SRC];
> +	fmt = ctx->q_data[MTK_Q_DATA_SRC].fmt;
> +	mtk_vcodec_enc_calc_src_size(fmt->num_planes, q_data->width,
> +			q_data->height,
> +			ctx->q_data[MTK_Q_DATA_SRC].sizeimage,
> +			ctx->q_data[MTK_Q_DATA_SRC].bytesperline);
> +
> +	ctx->q_data[MTK_Q_DATA_DST].width = DFT_CFG_WIDTH;
> +	ctx->q_data[MTK_Q_DATA_DST].height = DFT_CFG_HEIGHT;
> +	ctx->q_data[MTK_Q_DATA_DST].fmt = &mtk_video_formats[9];
> +	ctx->q_data[MTK_Q_DATA_DST].colorspace = V4L2_COLORSPACE_REC709;
> +	ctx->q_data[MTK_Q_DATA_DST].field = V4L2_FIELD_NONE;
> +
> +	q_data = &ctx->q_data[MTK_Q_DATA_DST];
> +	fmt = ctx->q_data[MTK_Q_DATA_DST].fmt;
> +	ctx->q_data[MTK_Q_DATA_DST].sizeimage[0] = q_data->width * q_data->height;
> +	ctx->q_data[MTK_Q_DATA_DST].bytesperline[0] = 0;
> +
> +}
> +
> +int mtk_vcodec_enc_ctrls_setup(struct mtk_vcodec_ctx *ctx)
> +{
> +	const struct v4l2_ctrl_ops *ops = &mtk_vcodec_enc_ctrl_ops;
> +	struct v4l2_ctrl_handler *handler = &ctx->ctrl_hdl;
> +	struct v4l2_ctrl_config cfg;
> +
> +	v4l2_ctrl_handler_init(handler, MTK_MAX_CTRLS);
> +	if (handler->error) {
> +		mtk_v4l2_err("Init control handler fail %d\n",
> +				handler->error);
> +		return handler->error;
> +	}

Move this check to just before the v4l2_ctrl_handler_setup(). Any of the
v4l2_ctrl_new_std* functions can set handler->error, which is why it should
be check after all controls are added.

> +
> +	ctx->ctrls[0] = v4l2_ctrl_new_std(handler, ops,
> +					V4L2_CID_MPEG_VIDEO_BITRATE,
> +					1, 4000000, 1, 4000000);
> +	ctx->ctrls[1] = v4l2_ctrl_new_std(handler, ops,
> +					V4L2_CID_MPEG_VIDEO_B_FRAMES,
> +					0, 2, 1, 0);
> +	ctx->ctrls[2] = v4l2_ctrl_new_std(handler, ops,
> +					V4L2_CID_MPEG_VIDEO_FRAME_RC_ENABLE,
> +					0, 1, 1, 1);
> +	ctx->ctrls[3] = v4l2_ctrl_new_std(handler, ops,
> +					V4L2_CID_MPEG_VIDEO_H264_MAX_QP,
> +					0, 51, 1, 51);
> +	ctx->ctrls[4] = v4l2_ctrl_new_std(handler, ops,
> +					V4L2_CID_MPEG_VIDEO_H264_I_PERIOD,
> +					0, 65535, 1, 30);
> +	ctx->ctrls[5] = v4l2_ctrl_new_std(handler, ops,
> +					V4L2_CID_MPEG_VIDEO_GOP_SIZE,
> +					0, 65535, 1, 30);
> +	ctx->ctrls[6] = v4l2_ctrl_new_std(handler, ops,
> +					V4L2_CID_MPEG_VIDEO_MB_RC_ENABLE,
> +					0, 1, 1, 0);
> +	ctx->ctrls[7] = v4l2_ctrl_new_std_menu(handler, ops,
> +					V4L2_CID_MPEG_VIDEO_HEADER_MODE,
> +					V4L2_MPEG_VIDEO_HEADER_MODE_JOINED_WITH_1ST_FRAME,
> +					0, V4L2_MPEG_VIDEO_HEADER_MODE_SEPARATE);
> +	ctx->ctrls[8] = v4l2_ctrl_new_std_menu(handler, ops,
> +					V4L2_CID_MPEG_VIDEO_H264_PROFILE,
> +					V4L2_MPEG_VIDEO_H264_PROFILE_HIGH,
> +					0, V4L2_MPEG_VIDEO_H264_PROFILE_MAIN);
> +	ctx->ctrls[9] = v4l2_ctrl_new_std_menu(handler, ops,
> +					V4L2_CID_MPEG_VIDEO_H264_LEVEL,
> +					V4L2_MPEG_VIDEO_H264_LEVEL_4_2,
> +					0, V4L2_MPEG_VIDEO_H264_LEVEL_4_0);
> +	ctx->ctrls[6] = v4l2_ctrl_new_std(handler, ops,
> +					V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME,
> +					0, 0, 0, 0);

You are overwriting V4L2_CID_MPEG_VIDEO_MB_RC_ENABLE! Why are you assigning it
anyway? Unless you are using ctx->ctrls[] you don't need to assign the result
of v4l2_ctrl_new_std to anything.

> +
> +	v4l2_ctrl_handler_setup(&ctx->ctrl_hdl);
> +
> +	return 0;
> +}
> +
> +int mtk_vcodec_enc_queue_init(void *priv, struct vb2_queue *src_vq,
> +			   struct vb2_queue *dst_vq)
> +{
> +	struct mtk_vcodec_ctx *ctx = priv;
> +	int ret;
> +
> +	src_vq->type		= V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE;
> +	src_vq->io_modes	= VB2_DMABUF | VB2_MMAP | VB2_USERPTR;

I recomment dropping VB2_USERPTR. That only makes sense for scatter-gather dma,
and you use physically contiguous DMA.

> +	src_vq->drv_priv	= ctx;
> +	src_vq->buf_struct_size = sizeof(struct mtk_video_enc_buf);
> +	src_vq->ops		= &mtk_venc_vb2_ops;
> +	src_vq->mem_ops		= &vb2_dma_contig_memops;
> +	src_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
> +	src_vq->lock = &ctx->dev->dev_mutex;
> +
> +	ret = vb2_queue_init(src_vq);
> +	if (ret)
> +		return ret;
> +
> +	dst_vq->type		= V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
> +	dst_vq->io_modes	= VB2_DMABUF | VB2_MMAP | VB2_USERPTR;
> +	dst_vq->drv_priv	= ctx;
> +	dst_vq->buf_struct_size = sizeof(struct v4l2_m2m_buffer);
> +	dst_vq->ops		= &mtk_venc_vb2_ops;
> +	dst_vq->mem_ops		= &vb2_dma_contig_memops;
> +	dst_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
> +	dst_vq->lock = &ctx->dev->dev_mutex;
> +
> +	return vb2_queue_init(dst_vq);
> +}
> +
> +int mtk_venc_unlock(struct mtk_vcodec_ctx *ctx)
> +{
> +	struct mtk_vcodec_dev *dev = ctx->dev;

Add empty line.

> +	dev->curr_ctx = -1;
> +	mutex_unlock(&dev->enc_mutex);
> +	return 0;
> +}
> +
> +int mtk_venc_lock(struct mtk_vcodec_ctx *ctx)
> +{
> +	struct mtk_vcodec_dev *dev = ctx->dev;
> +
> +	mutex_lock(&dev->enc_mutex);
> +	dev->curr_ctx = ctx->idx;
> +	return 0;
> +}
> +
> +void mtk_vcodec_enc_release(struct mtk_vcodec_ctx *ctx)
> +{
> +	venc_if_release(ctx);
> +}
> diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h
> new file mode 100644
> index 0000000..e09524b
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h
> @@ -0,0 +1,46 @@
> +/*
> +* Copyright (c) 2015 MediaTek Inc.
> +* Author: PC Chen <pc.chen@mediatek.com>
> +*         Tiffany Lin <tiffany.lin@mediatek.com>
> +*
> +* This program is free software; you can redistribute it and/or modify
> +* it under the terms of the GNU General Public License version 2 as
> +* published by the Free Software Foundation.
> +*
> +* This program is distributed in the hope that it will be useful,
> +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +* GNU General Public License for more details.
> +*/
> +
> +
> +#ifndef _MTK_VCODEC_ENC_H_
> +#define _MTK_VCODEC_ENC_H_
> +
> +#include <media/videobuf2-core.h>
> +#include <media/videobuf2-v4l2.h>
> +
> +/**
> + * struct mtk_video_enc_buf - Private data related to each VB2 buffer.
> + * @b:			Pointer to related VB2 buffer.
> + * @param_change:	Types of encode parameter change before encode this
> + *			buffer
> + * @enc_params		Encode parameters changed before encode this buffer
> + */
> +struct mtk_video_enc_buf {
> +	struct vb2_v4l2_buffer	vb;
> +	struct list_head	list;
> +
> +	enum mtk_encode_param param_change;
> +	struct mtk_enc_params enc_params;
> +};
> +
> +int mtk_venc_unlock(struct mtk_vcodec_ctx *ctx);
> +int mtk_venc_lock(struct mtk_vcodec_ctx *ctx);
> +int mtk_vcodec_enc_queue_init(void *priv, struct vb2_queue *src_vq,
> +					struct vb2_queue *dst_vq);
> +void mtk_vcodec_enc_release(struct mtk_vcodec_ctx *ctx);
> +int mtk_vcodec_enc_ctrls_setup(struct mtk_vcodec_ctx *ctx);
> +void mtk_vcodec_enc_ctx_params_setup(struct mtk_vcodec_ctx *ctx);
> +
> +#endif /* _MTK_VCODEC_ENC_H_ */
> diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
> new file mode 100644
> index 0000000..e7ab14a
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
> @@ -0,0 +1,476 @@
> +/*
> +* Copyright (c) 2015 MediaTek Inc.
> +* Author: PC Chen <pc.chen@mediatek.com>
> +*         Tiffany Lin <tiffany.lin@mediatek.com>
> +*
> +* This program is free software; you can redistribute it and/or modify
> +* it under the terms of the GNU General Public License version 2 as
> +* published by the Free Software Foundation.
> +*
> +* This program is distributed in the hope that it will be useful,
> +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +* GNU General Public License for more details.
> +*/
> +
> +#include <linux/interrupt.h>
> +#include <linux/irq.h>
> +#include <linux/module.h>
> +#include <linux/of_device.h>
> +#include <linux/of.h>
> +#include <media/v4l2-event.h>
> +#include <media/v4l2-mem2mem.h>
> +#include <media/videobuf2-dma-contig.h>
> +#include <linux/pm_runtime.h>
> +
> +#include "mtk_vcodec_drv.h"
> +#include "mtk_vcodec_enc.h"
> +#include "mtk_vcodec_pm.h"
> +#include "mtk_vcodec_intr.h"
> +#include "mtk_vcodec_util.h"
> +#include "mtk_vpu.h"
> +
> +
> +/* Wake up context wait_queue */
> +static void wake_up_ctx(struct mtk_vcodec_ctx *ctx, unsigned int reason)
> +{
> +	ctx->int_cond = 1;
> +	ctx->int_type = reason;
> +	wake_up_interruptible(&ctx->queue);
> +}
> +
> +static irqreturn_t mtk_vcodec_enc_irq_handler(int irq, void *priv)
> +{
> +	struct mtk_vcodec_dev *dev = priv;
> +	struct mtk_vcodec_ctx *ctx;
> +	unsigned int irq_status;
> +
> +	if (dev->curr_ctx == -1) {
> +		mtk_v4l2_err("curr_ctx = -1");
> +		return IRQ_HANDLED;
> +	}
> +
> +	ctx = dev->ctx[dev->curr_ctx];
> +	if (ctx == NULL) {
> +		mtk_v4l2_err("curr_ctx==NULL");
> +		return IRQ_HANDLED;
> +	}
> +	mtk_v4l2_debug(1, "idx=%d", ctx->idx);
> +	irq_status = readl(dev->reg_base[VENC_SYS] +
> +				(MTK_VENC_IRQ_STATUS_OFFSET));
> +	if (irq_status & MTK_VENC_IRQ_STATUS_PAUSE)
> +		writel((MTK_VENC_IRQ_STATUS_PAUSE),
> +		       dev->reg_base[VENC_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
> +
> +	if (irq_status & MTK_VENC_IRQ_STATUS_SWITCH)
> +		writel((MTK_VENC_IRQ_STATUS_SWITCH),
> +		       dev->reg_base[VENC_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
> +
> +	if (irq_status & MTK_VENC_IRQ_STATUS_DRAM)
> +		writel((MTK_VENC_IRQ_STATUS_DRAM),
> +		       dev->reg_base[VENC_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
> +
> +	if (irq_status & MTK_VENC_IRQ_STATUS_SPS)
> +		writel((MTK_VENC_IRQ_STATUS_SPS),
> +		       dev->reg_base[VENC_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
> +
> +	if (irq_status & MTK_VENC_IRQ_STATUS_PPS)
> +		writel((MTK_VENC_IRQ_STATUS_PPS),
> +		       dev->reg_base[VENC_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
> +
> +	if (irq_status & MTK_VENC_IRQ_STATUS_FRM)
> +		writel((MTK_VENC_IRQ_STATUS_FRM),
> +		       dev->reg_base[VENC_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
> +
> +	ctx->irq_status = irq_status;
> +	wake_up_ctx(ctx, MTK_INST_IRQ_RECEIVED);
> +	return IRQ_HANDLED;
> +}
> +
> +#if 1 /* VENC_LT */
> +static irqreturn_t mtk_vcodec_enc_irq_handler2(int irq, void *priv)
> +{
> +	struct mtk_vcodec_dev *dev = priv;
> +	struct mtk_vcodec_ctx *ctx;
> +	unsigned int irq_status;
> +
> +	ctx = dev->ctx[dev->curr_ctx];
> +	if (ctx == NULL) {
> +		mtk_v4l2_err("ctx==NULL");
> +		return IRQ_HANDLED;
> +	}
> +	mtk_v4l2_debug(1, "idx=%d", ctx->idx);
> +	irq_status = readl(dev->reg_base[VENC_LT_SYS] +
> +				(MTK_VENC_IRQ_STATUS_OFFSET));
> +	if (irq_status & MTK_VENC_IRQ_STATUS_PAUSE)
> +		writel((MTK_VENC_IRQ_STATUS_PAUSE),
> +		       dev->reg_base[VENC_LT_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
> +
> +	if (irq_status & MTK_VENC_IRQ_STATUS_SWITCH)
> +		writel((MTK_VENC_IRQ_STATUS_SWITCH),
> +		       dev->reg_base[VENC_LT_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
> +
> +	if (irq_status & MTK_VENC_IRQ_STATUS_DRAM)
> +		writel((MTK_VENC_IRQ_STATUS_DRAM),
> +		       dev->reg_base[VENC_LT_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
> +
> +	if (irq_status & MTK_VENC_IRQ_STATUS_SPS)
> +		writel((MTK_VENC_IRQ_STATUS_SPS),
> +		       dev->reg_base[VENC_LT_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
> +
> +	if (irq_status & MTK_VENC_IRQ_STATUS_PPS)
> +		writel((MTK_VENC_IRQ_STATUS_PPS),
> +		       dev->reg_base[VENC_LT_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
> +
> +	if (irq_status & MTK_VENC_IRQ_STATUS_FRM)
> +		writel((MTK_VENC_IRQ_STATUS_FRM),
> +		       dev->reg_base[VENC_LT_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
> +
> +	ctx->irq_status = irq_status;
> +	wake_up_ctx(ctx, MTK_INST_IRQ_RECEIVED);
> +	return IRQ_HANDLED;
> +}
> +#endif
> +
> +static void mtk_vcodec_enc_reset_handler(void *priv)
> +{
> +	int i;
> +	struct mtk_vcodec_dev *dev = priv;
> +	struct mtk_vcodec_ctx *ctx;
> +
> +	mtk_v4l2_debug(0, "Watchdog timeout!!");
> +
> +	mutex_lock(&dev->dev_mutex);
> +	for(i = 0; i < MTK_VCODEC_MAX_ENCODER_INSTANCES; i++) {
> +		ctx = dev->ctx[i];
> +		if (ctx) {
> +			ctx->state = MTK_STATE_ABORT;
> +			mtk_v4l2_debug(0, "[%d] Change to state MTK_STATE_ERROR", ctx->idx);
> +		}
> +
> +	}
> +	mutex_unlock(&dev->dev_mutex);
> +}
> +
> +static int fops_vcodec_open(struct file *file)
> +{
> +	struct video_device *vfd = video_devdata(file);
> +	struct mtk_vcodec_dev *dev = video_drvdata(file);
> +	struct mtk_vcodec_ctx *ctx = NULL;
> +	int ret = 0;
> +
> +	mutex_lock(&dev->dev_mutex);
> +
> +	ctx = devm_kzalloc(&dev->plat_dev->dev, sizeof(*ctx), GFP_KERNEL);
> +	if (!ctx) {
> +		ret = -ENOMEM;
> +		goto err_alloc;
> +	}
> +
> +	if (dev->num_instances >= MTK_VCODEC_MAX_ENCODER_INSTANCES) {
> +		mtk_v4l2_err("Too many open contexts\n");
> +		ret = -EBUSY;
> +		goto err_no_ctx;

Hmm. I never like it if you can't open a video node because of a reason like this.

I.e. a simple 'v4l2-ctl -D' (i.e. calling QUERYCAP) should never fail.

If there are hardware limitation that prevent more than X instances from running at
the same time, then those limitations typically kick in when you start to stream
(or possibly when calling REQBUFS). But before that it should always be possible to
open the device.

Having this check at open() is an indication of a poor design.

Is this is a hardware limitation at all?

> +	}
> +
> +	ctx->idx = ffz(dev->instance_mask[0]);
> +	v4l2_fh_init(&ctx->fh, video_devdata(file));
> +	file->private_data = &ctx->fh;
> +	v4l2_fh_add(&ctx->fh);
> +	ctx->dev = dev;
> +
> +	if (vfd == dev->vfd_enc) {
> +		ctx->type = MTK_INST_ENCODER;
> +
> +		ret = mtk_vcodec_enc_ctrls_setup(ctx);
> +		if (ret) {
> +			mtk_v4l2_err("Failed to setup controls() (%d)\n",
> +				       ret);
> +			goto err_ctrls_setup;
> +		}
> +		ctx->m2m_ctx = v4l2_m2m_ctx_init(dev->m2m_dev_enc, ctx,
> +						 &mtk_vcodec_enc_queue_init);
> +		if (IS_ERR(ctx->m2m_ctx)) {
> +			ret = PTR_ERR(ctx->m2m_ctx);
> +			mtk_v4l2_err("Failed to v4l2_m2m_ctx_init() (%d)\n",
> +				       ret);
> +			goto err_ctx_init;
> +		}
> +		mtk_vcodec_enc_ctx_params_setup(ctx);
> +	} else {
> +		mtk_v4l2_err("Invalid vfd !\n");
> +		ret = -ENOENT;
> +		goto err_ctx_init;
> +	}
> +
> +	init_waitqueue_head(&ctx->queue);
> +	dev->num_instances++;
> +
> +	if (dev->num_instances == 1) {

Having a counter is also not needed. You can use v4l2_fh_is_singular() to check if this
is the first open.

> +		ret = vpu_load_firmware(dev->vpu_plat_dev);
> +		if (ret < 0) {
> +				mtk_v4l2_err("vpu_load_firmware failed!\n");
> +			goto err_load_fw;
> +		}
> +
> +		dev->enc_capability =
> +			vpu_get_venc_hw_capa(dev->vpu_plat_dev);
> +		mtk_v4l2_debug(0, "encoder capability %x", dev->enc_capability);
> +	}
> +
> +	mtk_v4l2_debug(2, "Create instance [%d]@%p m2m_ctx=%p type=%d\n",
> +			 ctx->idx, ctx, ctx->m2m_ctx, ctx->type);
> +	set_bit(ctx->idx, &dev->instance_mask[0]);
> +	dev->ctx[ctx->idx] = ctx;
> +
> +	mutex_unlock(&dev->dev_mutex);
> +	mtk_v4l2_debug(0, "%s encoder [%d]", dev_name(&dev->plat_dev->dev), ctx->idx);
> +	return ret;
> +
> +	/* Deinit when failure occurred */
> +err_load_fw:
> +	v4l2_m2m_ctx_release(ctx->m2m_ctx);
> +	v4l2_fh_del(&ctx->fh);
> +	v4l2_fh_exit(&ctx->fh);
> +	dev->num_instances--;
> +err_ctx_init:
> +err_ctrls_setup:
> +	v4l2_ctrl_handler_free(&ctx->ctrl_hdl);
> +err_no_ctx:
> +	devm_kfree(&dev->plat_dev->dev, ctx);
> +err_alloc:
> +	mutex_unlock(&dev->dev_mutex);
> +	return ret;
> +}
> +
> +static int fops_vcodec_release(struct file *file)
> +{
> +	struct mtk_vcodec_dev *dev = video_drvdata(file);
> +	struct mtk_vcodec_ctx *ctx = fh_to_ctx(file->private_data);
> +
> +	mtk_v4l2_debug(0, "[%d] encoder\n", ctx->idx);
> +	mutex_lock(&dev->dev_mutex);
> +
> +	mtk_vcodec_enc_release(ctx);
> +	v4l2_fh_del(&ctx->fh);
> +	v4l2_fh_exit(&ctx->fh);
> +	v4l2_ctrl_handler_free(&ctx->ctrl_hdl);
> +	v4l2_m2m_ctx_release(ctx->m2m_ctx);
> +
> +	dev->ctx[ctx->idx] = NULL;
> +	dev->num_instances--;
> +	clear_bit(ctx->idx, &dev->instance_mask[0]);
> +	devm_kfree(&dev->plat_dev->dev, ctx);
> +	mutex_unlock(&dev->dev_mutex);
> +	return 0;
> +}
> +
> +static const struct v4l2_file_operations mtk_vcodec_fops = {
> +	.owner				= THIS_MODULE,
> +	.open				= fops_vcodec_open,
> +	.release			= fops_vcodec_release,
> +	.poll				= v4l2_m2m_fop_poll,
> +	.unlocked_ioctl			= video_ioctl2,
> +	.mmap				= v4l2_m2m_fop_mmap,
> +};
> +
> +static int mtk_vcodec_probe(struct platform_device *pdev)
> +{
> +	struct mtk_vcodec_dev *dev;
> +	struct video_device *vfd_enc;
> +	struct resource *res;
> +	int i, j, ret;
> +
> +	dev = devm_kzalloc(&pdev->dev, sizeof(*dev), GFP_KERNEL);
> +	if (!dev)
> +		return -ENOMEM;
> +
> +	dev->plat_dev = pdev;
> +
> +	dev->vpu_plat_dev = vpu_get_plat_device(dev->plat_dev);
> +	if (dev->vpu_plat_dev == NULL) {
> +		mtk_v4l2_err("[VPU] vpu device in not ready\n");
> +		return -EPROBE_DEFER;
> +	}
> +
> +	vpu_wdt_reg_handler(dev->vpu_plat_dev, mtk_vcodec_enc_reset_handler, dev,
> +			    VPU_RST_ENC);
> +
> +	ret = mtk_vcodec_init_enc_pm(dev);
> +	if (ret < 0) {
> +		dev_err(&pdev->dev, "Failed to get mt vcodec clock source!\n");
> +		return ret;
> +	}
> +
> +	for (i = VENC_SYS, j = 0; i < NUM_MAX_VCODEC_REG_BASE; i++, j++) {
> +		res = platform_get_resource(pdev, IORESOURCE_MEM, j);
> +		if (res == NULL) {
> +			dev_err(&pdev->dev, "get memory resource failed.\n");
> +			ret = -ENXIO;
> +			goto err_res;
> +		}
> +		dev->reg_base[i] = devm_ioremap_resource(&pdev->dev, res);
> +		if (IS_ERR(dev->reg_base[i])) {
> +			dev_err(&pdev->dev,
> +				"devm_ioremap_resource %d failed.\n", i);
> +			ret = PTR_ERR(dev->reg_base);
> +			goto err_res;
> +		}
> +		mtk_v4l2_debug(2, "reg[%d] base=0x%p\n", i, dev->reg_base[i]);
> +	}
> +
> +	res = platform_get_resource(pdev, IORESOURCE_IRQ, 0);
> +	if (res == NULL) {
> +		dev_err(&pdev->dev, "failed to get irq resource\n");
> +		ret = -ENOENT;
> +		goto err_res;
> +	}
> +
> +	dev->enc_irq = platform_get_irq(pdev, 0);
> +	ret = devm_request_irq(&pdev->dev, dev->enc_irq,
> +			       mtk_vcodec_enc_irq_handler,
> +			       0, pdev->name, dev);
> +	if (ret) {
> +		dev_err(&pdev->dev, "Failed to install dev->enc_irq %d (%d)\n",
> +			dev->enc_irq,
> +			ret);
> +		ret = -EINVAL;
> +		goto err_res;
> +	}
> +
> +	dev->enc_lt_irq = platform_get_irq(pdev, 1);
> +	ret = devm_request_irq(&pdev->dev,
> +			       dev->enc_lt_irq, mtk_vcodec_enc_irq_handler2,
> +			       0, pdev->name, dev);
> +	if (ret) {
> +		dev_err(&pdev->dev,
> +			"Failed to install dev->enc_lt_irq %d (%d)\n",
> +			dev->enc_lt_irq, ret);
> +		ret = -EINVAL;
> +		goto err_res;
> +	}
> +
> +	disable_irq(dev->enc_irq);
> +	disable_irq(dev->enc_lt_irq); /* VENC_LT */
> +	mutex_init(&dev->enc_mutex);
> +	mutex_init(&dev->dev_mutex);
> +
> +	snprintf(dev->v4l2_dev.name, sizeof(dev->v4l2_dev.name), "%s",
> +		 "[MTK_V4L2_VENC]");
> +
> +	ret = v4l2_device_register(&pdev->dev, &dev->v4l2_dev);
> +	if (ret) {
> +		mtk_v4l2_err("v4l2_device_register err=%d\n", ret);
> +		return ret;
> +	}
> +
> +	init_waitqueue_head(&dev->queue);
> +
> +	/* allocate video device for encoder and register it */
> +	vfd_enc = video_device_alloc();
> +	if (!vfd_enc) {
> +		mtk_v4l2_err("Failed to allocate video device\n");
> +		ret = -ENOMEM;
> +		goto err_enc_alloc;
> +	}
> +	vfd_enc->fops           = &mtk_vcodec_fops;
> +	vfd_enc->ioctl_ops      = &mtk_venc_ioctl_ops;
> +	vfd_enc->release        = video_device_release;
> +	vfd_enc->lock           = &dev->dev_mutex;
> +	vfd_enc->v4l2_dev       = &dev->v4l2_dev;
> +	vfd_enc->vfl_dir        = VFL_DIR_M2M;
> +
> +	snprintf(vfd_enc->name, sizeof(vfd_enc->name), "%s",
> +		 MTK_VCODEC_ENC_NAME);
> +	video_set_drvdata(vfd_enc, dev);
> +	dev->vfd_enc = vfd_enc;
> +	platform_set_drvdata(pdev, dev);
> +	ret = video_register_device(vfd_enc, VFL_TYPE_GRABBER, 1);
> +	if (ret) {
> +		mtk_v4l2_err("Failed to register video device\n");
> +		goto err_enc_reg;
> +	}

Do the register last. After registering the device anyone can start using it,
but since not everything is initialized that might cause crashes.

> +	mtk_v4l2_debug(0, "encoder registered as /dev/video%d\n",
> +			 vfd_enc->num);
> +
> +	dev->alloc_ctx = vb2_dma_contig_init_ctx(&pdev->dev);
> +	if (IS_ERR(dev->alloc_ctx)) {
> +		mtk_v4l2_err("Failed to alloc vb2 dma context 0\n");
> +		ret = PTR_ERR(dev->alloc_ctx);
> +		goto err_vb2_ctx_init;
> +	}
> +
> +	dev->m2m_dev_enc = v4l2_m2m_init(&mtk_venc_m2m_ops);
> +	if (IS_ERR(dev->m2m_dev_enc)) {
> +		mtk_v4l2_err("Failed to init mem2mem enc device\n");
> +		ret = PTR_ERR(dev->m2m_dev_enc);
> +		goto err_enc_mem_init;
> +	}
> +
> +	dev->encode_workqueue =
> +			alloc_ordered_workqueue(MTK_VCODEC_ENC_NAME, WQ_MEM_RECLAIM | WQ_FREEZABLE);
> +	if (!dev->encode_workqueue) {
> +		mtk_v4l2_err("Failed to create encode workqueue\n");
> +		ret = -EINVAL;
> +		goto err_event_workq;
> +	}
> +
> +	return 0;
> +
> +err_event_workq:
> +	v4l2_m2m_release(dev->m2m_dev_enc);
> +err_enc_mem_init:
> +	vb2_dma_contig_cleanup_ctx(dev->alloc_ctx);
> +err_vb2_ctx_init:
> +	video_unregister_device(vfd_enc);
> +err_enc_reg:
> +	video_device_release(vfd_enc);
> +err_enc_alloc:
> +	v4l2_device_unregister(&dev->v4l2_dev);
> +err_res:
> +	mtk_vcodec_release_enc_pm(dev);
> +	return ret;
> +}
> +
> +static const struct of_device_id mtk_vcodec_match[] = {
> +	{.compatible = "mediatek,mt8173-vcodec-enc",},
> +	{},
> +};
> +MODULE_DEVICE_TABLE(of, mtk_vcodec_match);
> +
> +static int mtk_vcodec_remove(struct platform_device *pdev)
> +{
> +	struct mtk_vcodec_dev *dev = platform_get_drvdata(pdev);
> +
> +	mtk_v4l2_debug_enter();
> +	flush_workqueue(dev->encode_workqueue);
> +	destroy_workqueue(dev->encode_workqueue);
> +	if (dev->m2m_dev_enc)
> +		v4l2_m2m_release(dev->m2m_dev_enc);
> +	if (dev->alloc_ctx)
> +		vb2_dma_contig_cleanup_ctx(dev->alloc_ctx);
> +
> +	if (dev->vfd_enc) {
> +		video_unregister_device(dev->vfd_enc);
> +		video_device_release(dev->vfd_enc);

Don't call video_device_release! That will happen automatically once the
last filehandle is closed.

> +	}
> +	v4l2_device_unregister(&dev->v4l2_dev);
> +	mtk_vcodec_release_enc_pm(dev);
> +	return 0;
> +}
> +
> +static struct platform_driver mtk_vcodec_driver = {
> +	.probe	= mtk_vcodec_probe,
> +	.remove	= mtk_vcodec_remove,
> +	.driver	= {
> +		.name	= MTK_VCODEC_ENC_NAME,
> +		.owner	= THIS_MODULE,
> +		.of_match_table = mtk_vcodec_match,
> +	},
> +};
> +
> +module_platform_driver(mtk_vcodec_driver);
> +
> +
> +MODULE_LICENSE("GPL v2");
> +MODULE_DESCRIPTION("Mediatek video codec V4L2 driver");
> diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
> new file mode 100644
> index 0000000..518fba7
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
> @@ -0,0 +1,132 @@
> +/*
> +* Copyright (c) 2015 MediaTek Inc.
> +* Author: Tiffany Lin <tiffany.lin@mediatek.com>
> +*
> +* This program is free software; you can redistribute it and/or modify
> +* it under the terms of the GNU General Public License version 2 as
> +* published by the Free Software Foundation.
> +*
> +* This program is distributed in the hope that it will be useful,
> +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +* GNU General Public License for more details.
> +*/
> +
> +#include <linux/clk.h>
> +#include <linux/of_address.h>
> +#include <linux/of_platform.h>
> +#include <linux/pm_runtime.h>
> +#include <soc/mediatek/smi.h>
> +
> +#include "mtk_vcodec_pm.h"
> +#include "mtk_vcodec_util.h"
> +#include "mtk_vpu.h"
> +
> +
> +int mtk_vcodec_init_enc_pm(struct mtk_vcodec_dev *mtkdev)
> +{
> +	struct device_node *node;
> +	struct platform_device *pdev;
> +	struct device *dev;
> +	struct mtk_vcodec_pm *pm;
> +	int ret = 0;
> +
> +	pdev = mtkdev->plat_dev;
> +	pm = &mtkdev->pm;
> +	memset(pm, 0, sizeof(struct mtk_vcodec_pm));
> +	pm->mtkdev = mtkdev;
> +	dev = &pdev->dev;
> +
> +	node = of_parse_phandle(dev->of_node, "mediatek,larb", 0);
> +	if (!node)
> +		return -1;
> +	pdev = of_find_device_by_node(node);
> +	if (WARN_ON(!pdev)) {
> +		of_node_put(node);
> +		return -1;
> +	}
> +	pm->larbvenc = &pdev->dev;
> +
> +	node = of_parse_phandle(dev->of_node, "mediatek,larb", 1);
> +	if (!node)
> +		return -1;
> +
> +	pdev = of_find_device_by_node(node);
> +	if (WARN_ON(!pdev)) {
> +		of_node_put(node);
> +		return -EINVAL;
> +	}
> +	pm->larbvenclt = &pdev->dev;
> +
> +	pdev = mtkdev->plat_dev;
> +	pm->dev = &pdev->dev;
> +
> +	pm->vencpll_d2 = devm_clk_get(&pdev->dev, "vencpll_d2");
> +	if (pm->vencpll_d2 == NULL) {
> +		mtk_v4l2_err("devm_clk_get vencpll_d2 fail");
> +		ret = -1;
> +	}
> +
> +	pm->venc_sel = devm_clk_get(&pdev->dev, "venc_sel");
> +	if (pm->venc_sel == NULL) {
> +		mtk_v4l2_err("devm_clk_get venc_sel fail");
> +		ret = -1;
> +	}
> +
> +	pm->univpll1_d2 = devm_clk_get(&pdev->dev, "univpll1_d2");
> +	if (pm->univpll1_d2 == NULL) {
> +		mtk_v4l2_err("devm_clk_get univpll1_d2 fail");
> +		ret = -1;
> +	}
> +
> +	pm->venc_lt_sel = devm_clk_get(&pdev->dev, "venc_lt_sel");
> +	if (pm->venc_lt_sel == NULL) {
> +		mtk_v4l2_err("devm_clk_get venc_lt_sel fail");
> +		ret = -1;
> +	}
> +
> +	return ret;
> +}
> +
> +void mtk_vcodec_release_enc_pm(struct mtk_vcodec_dev *mtkdev)
> +{
> +}
> +
> +
> +void mtk_vcodec_enc_clock_on(struct mtk_vcodec_pm *pm)
> +{
> +	int ret;
> +
> +	ret = clk_prepare_enable(pm->venc_sel);
> +	if (ret)
> +		mtk_v4l2_err("venc_sel fail %d", ret);
> +
> +	ret = clk_set_parent(pm->venc_sel, pm->vencpll_d2);
> +	if (ret)
> +		mtk_v4l2_err("clk_set_parent fail %d", ret);
> +
> +	ret = clk_prepare_enable(pm->venc_lt_sel);
> +	if (ret)
> +		mtk_v4l2_err("venc_lt_sel fail %d", ret);
> +
> +	ret = clk_set_parent(pm->venc_lt_sel, pm->univpll1_d2);
> +	if (ret)
> +		mtk_v4l2_err("clk_set_parent fail %d", ret);
> +
> +	ret = mtk_smi_larb_get(pm->larbvenc);
> +	if (ret)
> +		mtk_v4l2_err("mtk_smi_larb_get larb3 fail %d\n", ret);
> +
> +	ret = mtk_smi_larb_get(pm->larbvenclt);
> +	if (ret)
> +		mtk_v4l2_err("mtk_smi_larb_get larb4 fail %d\n", ret);
> +
> +}
> +
> +void mtk_vcodec_enc_clock_off(struct mtk_vcodec_pm *pm)
> +{
> +	mtk_smi_larb_put(pm->larbvenc);
> +	mtk_smi_larb_put(pm->larbvenclt);
> +	clk_disable_unprepare(pm->venc_lt_sel);
> +	clk_disable_unprepare(pm->venc_sel);
> +}
> diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.c
> new file mode 100644
> index 0000000..919b949
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.c
> @@ -0,0 +1,102 @@
> +/*
> +* Copyright (c) 2015 MediaTek Inc.
> +* Author: Tiffany Lin <tiffany.lin@mediatek.com>
> +*
> +* This program is free software; you can redistribute it and/or modify
> +* it under the terms of the GNU General Public License version 2 as
> +* published by the Free Software Foundation.
> +*
> +* This program is distributed in the hope that it will be useful,
> +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +* GNU General Public License for more details.
> +*/
> +
> +#include <linux/errno.h>
> +#include <linux/wait.h>
> +
> +#include "mtk_vcodec_drv.h"
> +#include "mtk_vcodec_intr.h"
> +#include "mtk_vcodec_util.h"
> +
> +void mtk_vcodec_clean_dev_int_flags(void *data)
> +{
> +	struct mtk_vcodec_dev *dev = (struct mtk_vcodec_dev *)data;
> +
> +	dev->int_cond = 0;
> +	dev->int_type = 0;
> +}
> +
> +int mtk_vcodec_wait_for_done_ctx(void *data, int command,
> +				 unsigned int timeout_ms, int interrupt)
> +{
> +	wait_queue_head_t *waitqueue;
> +	long timeout_jiff, ret;
> +	int status = 0;
> +	struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)data;
> +
> +	waitqueue = (wait_queue_head_t *)&ctx->queue;
> +	timeout_jiff = msecs_to_jiffies(timeout_ms);
> +	if (interrupt) {
> +		ret = wait_event_interruptible_timeout(*waitqueue,
> +				(ctx->int_cond &&
> +				(ctx->int_type == command)),
> +				timeout_jiff);
> +	} else {
> +		ret = wait_event_timeout(*waitqueue,
> +				(ctx->int_cond &&
> +				(ctx->int_type == command)),
> +				 timeout_jiff);
> +	}
> +	if (0 == ret) {
> +		status = -1;	/* timeout */
> +		mtk_v4l2_err("[%d] cmd=%d, ctx->type=%d, wait_event_interruptible_timeout time=%ums out %d %d!",
> +				ctx->idx, ctx->type, command, timeout_ms,
> +				ctx->int_cond, ctx->int_type);
> +	} else if (-ERESTARTSYS == ret) {
> +		mtk_v4l2_err("[%d] cmd=%d, ctx->type=%d, wait_event_interruptible_timeout interrupted by a signal %d %d",
> +				ctx->idx, ctx->type, command, ctx->int_cond,
> +				ctx->int_type);
> +		status = -1;
> +	}
> +
> +	ctx->int_cond = 0;
> +	ctx->int_type = 0;
> +
> +	return status;
> +}
> +
> +int mtk_vcodec_wait_for_done_dev(void *data, int command,
> +				 unsigned int timeout, int interrupt)
> +{
> +	wait_queue_head_t *waitqueue;
> +	long timeout_jiff, ret;
> +	int status = 0;
> +	struct mtk_vcodec_dev *dev = (struct mtk_vcodec_dev *)data;
> +
> +	waitqueue = (wait_queue_head_t *)&dev->queue;
> +	timeout_jiff = msecs_to_jiffies(timeout);
> +	if (interrupt) {
> +		ret = wait_event_interruptible_timeout(*waitqueue,
> +				(dev->int_cond &&
> +				(dev->int_type == command)),
> +				timeout_jiff);
> +	} else {
> +		ret = wait_event_timeout(*waitqueue,
> +				(dev->int_cond &&
> +				(dev->int_type == command)),
> +				timeout_jiff);
> +	}
> +	if (0 == ret) {
> +		status = -1;	/* timeout */
> +		mtk_v4l2_err("wait_event_interruptible_timeout time=%lu out %d %d!",
> +				timeout_jiff, dev->int_cond, dev->int_type);
> +	} else if (-ERESTARTSYS == ret) {
> +		mtk_v4l2_err("wait_event_interruptible_timeout interrupted by a signal %d %d",
> +				dev->int_cond, dev->int_type);
> +		status = -1;
> +	}
> +	dev->int_cond = 0;
> +	dev->int_type = 0;
> +	return status;
> +}
> diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.h
> new file mode 100644
> index 0000000..e9b7f94
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.h
> @@ -0,0 +1,29 @@
> +/*
> +* Copyright (c) 2015 MediaTek Inc.
> +* Author: Tiffany Lin <tiffany.lin@mediatek.com>
> +*
> +* This program is free software; you can redistribute it and/or modify
> +* it under the terms of the GNU General Public License version 2 as
> +* published by the Free Software Foundation.
> +*
> +* This program is distributed in the hope that it will be useful,
> +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +* GNU General Public License for more details.
> +*/
> +
> +#ifndef _MTK_VCODEC_INTR_H_
> +#define _MTK_VCODEC_INTR_H_
> +
> +#define MTK_INST_IRQ_RECEIVED		0x1
> +#define MTK_INST_WORK_THREAD_ABORT_DONE	0x2
> +
> +/* timeout is ms */
> +int mtk_vcodec_wait_for_done_ctx(void *data, int command, unsigned int timeout,
> +				 int interrupt);
> +int mtk_vcodec_wait_for_done_dev(void *data, int command, unsigned int timeout,
> +				 int interrupt);
> +
> +void mtk_vcodec_clean_dev_int_flags(void *data);
> +
> +#endif /* _MTK_VCODEC_INTR_H_ */
> diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_pm.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_pm.h
> new file mode 100644
> index 0000000..fdadec9
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_pm.h
> @@ -0,0 +1,26 @@
> +/*
> +* Copyright (c) 2015 MediaTek Inc.
> +* Author: Tiffany Lin <tiffany.lin@mediatek.com>
> +*
> +* This program is free software; you can redistribute it and/or modify
> +* it under the terms of the GNU General Public License version 2 as
> +* published by the Free Software Foundation.
> +*
> +* This program is distributed in the hope that it will be useful,
> +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +* GNU General Public License for more details.
> +*/
> +
> +#ifndef _MTK_VCODEC_PM_H_
> +#define _MTK_VCODEC_PM_H_
> +
> +#include "mtk_vcodec_drv.h"
> +
> +int mtk_vcodec_init_enc_pm(struct mtk_vcodec_dev *dev);
> +void mtk_vcodec_release_enc_pm(struct mtk_vcodec_dev *dev);
> +
> +void mtk_vcodec_enc_clock_on(struct mtk_vcodec_pm *pm);
> +void mtk_vcodec_enc_clock_off(struct mtk_vcodec_pm *pm);
> +
> +#endif /* _MTK_VCODEC_PM_H_ */
> diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c
> new file mode 100644
> index 0000000..3fede8d
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c
> @@ -0,0 +1,106 @@
> +/*
> +* Copyright (c) 2015 MediaTek Inc.
> +* Author: PC Chen <pc.chen@mediatek.com>
> +*         Tiffany Lin <tiffany.lin@mediatek.com>
> +*
> +* This program is free software; you can redistribute it and/or modify
> +* it under the terms of the GNU General Public License version 2 as
> +* published by the Free Software Foundation.
> +*
> +* This program is distributed in the hope that it will be useful,
> +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +* GNU General Public License for more details.
> +*/
> +
> +#include <linux/module.h>
> +
> +#include "mtk_vcodec_drv.h"
> +#include "mtk_vcodec_util.h"
> +#include "mtk_vpu.h"
> +
> +bool mtk_vcodec_dbg = false;
> +int mtk_v4l2_dbg_level = 0;
> +
> +module_param(mtk_v4l2_dbg_level, int, S_IRUGO | S_IWUSR);
> +module_param(mtk_vcodec_dbg, bool, S_IRUGO | S_IWUSR);
> +
> +void __iomem *mtk_vcodec_get_reg_addr(void *data, unsigned int reg_idx)
> +{
> +	struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)data;
> +
> +	if (!data || reg_idx >= NUM_MAX_VCODEC_REG_BASE) {
> +		mtk_v4l2_err("Invalid arguments");
> +		return NULL;
> +	}
> +	return ctx->dev->reg_base[reg_idx];
> +}
> +
> +int mtk_vcodec_mem_alloc(void *data, struct mtk_vcodec_mem *mem)
> +{
> +	unsigned long size = mem->size;
> +	struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)data;
> +	struct device *dev = &ctx->dev->plat_dev->dev;
> +
> +	mem->va = dma_alloc_coherent(dev, size, &mem->dma_addr, GFP_KERNEL);
> +
> +	if (!mem->va) {
> +		mtk_v4l2_err("%s dma_alloc size=%ld failed!", dev_name(dev),
> +			       size);
> +		return -ENOMEM;
> +	}
> +
> +	memset(mem->va, 0, size);
> +
> +	mtk_v4l2_debug(3, "[%d]  - va      = %p", ctx->idx, mem->va);
> +	mtk_v4l2_debug(3, "[%d]  - dma     = 0x%lx", ctx->idx,
> +			 (unsigned long)mem->dma_addr);
> +	mtk_v4l2_debug(3, "[%d]    size = 0x%lx", ctx->idx, size);
> +
> +	return 0;
> +}
> +
> +void mtk_vcodec_mem_free(void *data, struct mtk_vcodec_mem *mem)
> +{
> +	unsigned long size = mem->size;
> +	struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)data;
> +	struct device *dev = &ctx->dev->plat_dev->dev;
> +
> +	dma_free_coherent(dev, size, mem->va, mem->dma_addr);
> +	mem->va = NULL;
> +
> +	mtk_v4l2_debug(3, "[%d]  - va      = %p", ctx->idx, mem->va);
> +	mtk_v4l2_debug(3, "[%d]  - dma     = 0x%lx", ctx->idx,
> +			 (unsigned long)mem->dma_addr);
> +	mtk_v4l2_debug(3, "[%d]    size = 0x%lx", ctx->idx, size);
> +}
> +
> +int mtk_vcodec_get_ctx_id(void *data)
> +{
> +	struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)data;
> +
> +	if (!ctx)
> +		return -1;
> +
> +	return ctx->idx;
> +}
> +
> +struct platform_device *mtk_vcodec_get_plat_dev(void *data)
> +{
> +	struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)data;
> +
> +	if (!ctx)
> +		return NULL;
> +
> +	return vpu_get_plat_device(ctx->dev->plat_dev);
> +}
> +
> +void mtk_vcodec_fmt2str(u32 fmt, char *str)
> +{
> +	char a = fmt & 0xFF;
> +	char b = (fmt >> 8) & 0xFF;
> +	char c = (fmt >> 16) & 0xFF;
> +	char d = (fmt >> 24) & 0xFF;
> +
> +	sprintf(str, "%c%c%c%c", a, b, c, d);
> +}
> diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
> new file mode 100644
> index 0000000..47016ae
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
> @@ -0,0 +1,85 @@
> +/*
> +* Copyright (c) 2015 MediaTek Inc.
> +* Author: PC Chen <pc.chen@mediatek.com>
> +*         Tiffany Lin <tiffany.lin@mediatek.com>
> +*
> +* This program is free software; you can redistribute it and/or modify
> +* it under the terms of the GNU General Public License version 2 as
> +* published by the Free Software Foundation.
> +*
> +* This program is distributed in the hope that it will be useful,
> +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +* GNU General Public License for more details.
> +*/
> +
> +#ifndef _MTK_VCODEC_UTIL_H_
> +#define _MTK_VCODEC_UTIL_H_
> +
> +#include <linux/types.h>
> +#include <linux/dma-direction.h>
> +
> +struct mtk_vcodec_mem {
> +	size_t size;
> +	void *va;
> +	dma_addr_t dma_addr;
> +};
> +
> +extern int mtk_v4l2_dbg_level;
> +extern bool mtk_vcodec_dbg;
> +
> +#define DEBUG 	1
> +
> +#if defined(DEBUG)
> +
> +#define mtk_v4l2_debug(level, fmt, args...)				 \
> +	do {								 \
> +		if (mtk_v4l2_dbg_level >= level)			 \
> +			pr_info("[MTK_V4L2] level=%d %s(),%d: " fmt "\n",\
> +				level, __func__, __LINE__, ##args);	 \
> +	} while (0)
> +
> +#define mtk_v4l2_err(fmt, args...)                \
> +	pr_err("[MTK_V4L2][ERROR] %s:%d: " fmt "\n", __func__, __LINE__, \
> +	       ##args)
> +
> +
> +#define mtk_v4l2_debug_enter()  mtk_v4l2_debug(3, "+\n")
> +#define mtk_v4l2_debug_leave()  mtk_v4l2_debug(3, "-\n")
> +
> +#define mtk_vcodec_debug(h, fmt, args...)				\
> +	do {								\
> +		if (mtk_vcodec_dbg)					\
> +			pr_info("[MTK_VCODEC][%d]: %s() " fmt "\n",	\
> +				((struct mtk_vcodec_ctx *)h->ctx)->idx, \
> +				__func__, ##args);			\
> +	} while (0)
> +
> +#define mtk_vcodec_err(h, fmt, args...)					\
> +	pr_err("[MTK_VCODEC][ERROR][%d]: %s() " fmt "\n",		\
> +	       ((struct mtk_vcodec_ctx *)h->ctx)->idx, __func__, ##args)
> +
> +#define mtk_vcodec_debug_enter(h)  mtk_vcodec_debug(h, "+\n")
> +#define mtk_vcodec_debug_leave(h)  mtk_vcodec_debug(h, "-\n")
> +
> +#else
> +
> +#define mtk_v4l2_debug(level, fmt, args...)
> +#define mtk_v4l2_err(fmt, args...)
> +#define mtk_v4l2_debug_enter()
> +#define mtk_v4l2_debug_leave()
> +
> +#define mtk_vcodec_debug(h, fmt, args...)
> +#define mtk_vcodec_err(h, fmt, args...)
> +#define mtk_vcodec_debug_enter(h)
> +#define mtk_vcodec_debug_leave(h)
> +
> +#endif
> +
> +void __iomem *mtk_vcodec_get_reg_addr(void *data, unsigned int reg_idx);
> +int mtk_vcodec_mem_alloc(void *data, struct mtk_vcodec_mem *mem);
> +void mtk_vcodec_mem_free(void *data, struct mtk_vcodec_mem *mem);
> +int mtk_vcodec_get_ctx_id(void *data);
> +struct platform_device *mtk_vcodec_get_plat_dev(void *data);
> +void mtk_vcodec_fmt2str(u32 fmt, char *str);
> +#endif /* _MTK_VCODEC_UTIL_H_ */
> diff --git a/drivers/media/platform/mtk-vcodec/venc_drv_base.h b/drivers/media/platform/mtk-vcodec/venc_drv_base.h
> new file mode 100644
> index 0000000..ed9cbf0
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vcodec/venc_drv_base.h
> @@ -0,0 +1,62 @@
> +/*
> + * Copyright (c) 2015 MediaTek Inc.
> + * Author: Daniel Hsiao <daniel.hsiao@mediatek.com>
> + *         Jungchang Tsao <jungchang.tsao@mediatek.com>
> + *
> + * This program is free software; you can redistribute it and/or
> + * modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + */
> +
> +#ifndef _VENC_DRV_BASE_
> +#define _VENC_DRV_BASE_
> +
> +#include "mtk_vcodec_drv.h"
> +
> +#include "venc_drv_if.h"
> +
> +struct venc_common_if {
> +	/**
> +	 * (*init)() - initialize driver
> +	 * @ctx:	[in] mtk v4l2 context
> +	 * @handle: [out] driver handle
> +	 */
> +	int (*init)(struct mtk_vcodec_ctx *ctx, unsigned long *handle);
> +
> +	/**
> +	 * (*encode)() - trigger encode
> +	 * @handle: [in] driver handle
> +	 * @opt: [in] encode option
> +	 * @frm_buf: [in] frame buffer to store input frame
> +	 * @bs_buf: [in] bitstream buffer to store output bitstream
> +	 * @result: [out] encode result
> +	 */
> +	int (*encode)(unsigned long handle, enum venc_start_opt opt,
> +		      struct venc_frm_buf *frm_buf,
> +		      struct mtk_vcodec_mem *bs_buf,
> +		      struct venc_done_result *result);
> +
> +	/**
> +	 * (*set_param)() - set driver's parameter
> +	 * @handle: [in] driver handle
> +	 * @type: [in] parameter type
> +	 * @in: [in] buffer to store the parameter
> +	 */
> +	int (*set_param)(unsigned long handle, enum venc_set_param_type type,
> +			 void *in);
> +
> +	/**
> +	 * (*deinit)() - deinitialize driver.
> +	 * @handle: [in] driver handle
> +	 */
> +	int (*deinit)(unsigned long handle);
> +};
> +
> +
> +#endif
> diff --git a/drivers/media/platform/mtk-vcodec/venc_drv_if.c b/drivers/media/platform/mtk-vcodec/venc_drv_if.c
> new file mode 100644
> index 0000000..daa8e93
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vcodec/venc_drv_if.c
> @@ -0,0 +1,100 @@
> +/*
> + * Copyright (c) 2015 MediaTek Inc.
> + * Author: Daniel Hsiao <daniel.hsiao@mediatek.com>
> + *         Jungchang Tsao <jungchang.tsao@mediatek.com>
> + *
> + * This program is free software; you can redistribute it and/or
> + * modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + */
> +
> +#include <linux/interrupt.h>
> +#include <linux/kernel.h>
> +#include <linux/slab.h>
> +
> +#include "venc_drv_if.h"
> +#include "mtk_vcodec_enc.h"
> +#include "mtk_vcodec_pm.h"
> +#include "mtk_vpu.h"
> +
> +#include "venc_drv_base.h"
> +
> +int venc_if_create(struct mtk_vcodec_ctx *ctx, unsigned int fourcc)
> +{
> +	char str[10];
> +	int ret = 0;
> +
> +	mtk_vcodec_fmt2str(fourcc, str);
> +
> +	switch (fourcc) {
> +	case V4L2_PIX_FMT_VP8:
> +	case V4L2_PIX_FMT_H264:
> +	default:
> +		return -EINVAL;
> +	}
> +
> +	mtk_venc_lock(ctx);
> +	mtk_vcodec_enc_clock_on(&ctx->dev->pm);
> +	ret = ctx->enc_if->init(ctx, (unsigned long *)&ctx->drv_handle);
> +	mtk_vcodec_enc_clock_off(&ctx->dev->pm);
> +	mtk_venc_unlock(ctx);
> +
> +	return ret;
> +
> +}
> +
> +
> +int venc_if_set_param(struct mtk_vcodec_ctx *ctx,
> +		      enum venc_set_param_type type, void *in)
> +{
> +	int ret = 0;
> +
> +	mtk_venc_lock(ctx);
> +	mtk_vcodec_enc_clock_on(&ctx->dev->pm);
> +	ret = ctx->enc_if->set_param(ctx->drv_handle, type, in);
> +	mtk_vcodec_enc_clock_off(&ctx->dev->pm);
> +	mtk_venc_unlock(ctx);
> +
> +	return ret;
> +}
> +
> +int venc_if_encode(struct mtk_vcodec_ctx *ctx,
> +		   enum venc_start_opt opt, struct venc_frm_buf *frm_buf,
> +		   struct mtk_vcodec_mem *bs_buf,
> +		   struct venc_done_result *result)
> +{
> +	int ret = 0;
> +	
> +	mtk_venc_lock(ctx);
> +	mtk_vcodec_enc_clock_on(&ctx->dev->pm);
> +	ret = ctx->enc_if->encode(ctx->drv_handle, opt, frm_buf, bs_buf, result);
> +	mtk_vcodec_enc_clock_off(&ctx->dev->pm);
> +	mtk_venc_unlock(ctx);
> +	
> +	return ret;
> +}
> +
> +int venc_if_release(struct mtk_vcodec_ctx *ctx)
> +{
> +	int ret = 0;
> +
> +	if(ctx->drv_handle == 0)
> +		return 0;
> +
> +	mtk_venc_lock(ctx);
> +	mtk_vcodec_enc_clock_on(&ctx->dev->pm);
> +	ret = ctx->enc_if->deinit(ctx->drv_handle);
> +	mtk_vcodec_enc_clock_off(&ctx->dev->pm);
> +	mtk_venc_unlock(ctx);
> +
> +	ctx->drv_handle = 0;
> +
> +	return ret;
> +}
> +
> diff --git a/drivers/media/platform/mtk-vcodec/venc_drv_if.h b/drivers/media/platform/mtk-vcodec/venc_drv_if.h
> new file mode 100644
> index 0000000..a387011
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vcodec/venc_drv_if.h
> @@ -0,0 +1,175 @@
> +/*
> + * Copyright (c) 2015 MediaTek Inc.
> + * Author: Daniel Hsiao <daniel.hsiao@mediatek.com>
> + *         Jungchang Tsao <jungchang.tsao@mediatek.com>
> + *
> + * This program is free software; you can redistribute it and/or
> + * modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + */
> +
> +#ifndef _VENC_DRV_IF_H_
> +#define _VENC_DRV_IF_H_
> +
> +#include "mtk_vcodec_drv.h"
> +#include "mtk_vcodec_util.h"
> +
> +/*
> + * enum venc_yuv_fmt - The type of input yuv format
> + * (VPU related: If you change the order, you must also update the VPU codes.)
> + * @VENC_YUV_FORMAT_420: 420 YUV format
> + * @VENC_YUV_FORMAT_YV12: YV12 YUV format
> + * @VENC_YUV_FORMAT_NV12: NV12 YUV format
> + * @VENC_YUV_FORMAT_NV21: NV21 YUV format
> + */
> +enum venc_yuv_fmt {
> +	VENC_YUV_FORMAT_420 = 3,
> +	VENC_YUV_FORMAT_YV12 = 5,
> +	VENC_YUV_FORMAT_NV12 = 6,
> +	VENC_YUV_FORMAT_NV21 = 7,
> +};
> +
> +/*
> + * enum venc_start_opt - encode frame option used in venc_if_encode()
> + * @VENC_START_OPT_ENCODE_SEQUENCE_HEADER: encode SPS/PPS for H264
> + * @VENC_START_OPT_ENCODE_FRAME: encode normal frame
> + */
> +enum venc_start_opt {
> +	VENC_START_OPT_ENCODE_SEQUENCE_HEADER,
> +	VENC_START_OPT_ENCODE_FRAME,
> +};
> +
> +/*
> + * enum venc_drv_msg - The type of encode frame status used in venc_if_encode()
> + * @VENC_MESSAGE_OK: encode ok
> + * @VENC_MESSAGE_ERR: encode error
> + */
> +enum venc_drv_msg {
> +	VENC_MESSAGE_OK,
> +	VENC_MESSAGE_ERR,
> +};
> +
> +/*
> + * enum venc_set_param_type - The type of set parameter used in venc_if_set_param()
> + * (VPU related: If you change the order, you must also update the VPU codes.)
> + * @VENC_SET_PARAM_ENC: set encoder parameters
> + * @VENC_SET_PARAM_FORCE_INTRA: set force intra frame
> + * @VENC_SET_PARAM_ADJUST_BITRATE: set to adjust bitrate (in bps)
> + * @VENC_SET_PARAM_ADJUST_FRAMERATE: set frame rate
> + * @VENC_SET_PARAM_I_FRAME_INTERVAL: set I frame interval
> + * @VENC_SET_PARAM_SKIP_FRAME: set H264 skip one frame
> + * @VENC_SET_PARAM_PREPEND_HEADER: set H264 prepend SPS/PPS before IDR
> + * @VENC_SET_PARAM_TS_MODE: set VP8 temporal scalability mode
> + */
> +enum venc_set_param_type {
> +	VENC_SET_PARAM_ENC,
> +	VENC_SET_PARAM_FORCE_INTRA,
> +	VENC_SET_PARAM_ADJUST_BITRATE,
> +	VENC_SET_PARAM_ADJUST_FRAMERATE,
> +	VENC_SET_PARAM_I_FRAME_INTERVAL,
> +	VENC_SET_PARAM_SKIP_FRAME,
> +	VENC_SET_PARAM_PREPEND_HEADER,
> +	VENC_SET_PARAM_TS_MODE,
> +};
> +
> +/*
> + * struct venc_enc_prm - encoder settings for VENC_SET_PARAM_ENC used in venc_if_set_param()
> + * @input_fourcc: input fourcc
> + * @h264_profile: V4L2 defined H.264 profile
> + * @h264_level: V4L2 defined H.264 level
> + * @width: image width
> + * @height: image height
> + * @buf_width: buffer width
> + * @buf_height: buffer height
> + * @frm_rate: frame rate
> + * @intra_period: intra frame period
> + * @bitrate: target bitrate in kbps
> + */
> +struct venc_enc_prm {
> +	enum venc_yuv_fmt input_fourcc;
> +	unsigned int h264_profile;
> +	unsigned int h264_level;
> +	unsigned int width;
> +	unsigned int height;
> +	unsigned int buf_width;
> +	unsigned int buf_height;
> +	unsigned int frm_rate;
> +	unsigned int intra_period;
> +	unsigned int bitrate;
> +	unsigned int sizeimage[MTK_VCODEC_MAX_PLANES];
> +};
> +
> +/*
> + * struct venc_frm_buf - frame buffer information used in venc_if_encode()
> + * @fb_addr: plane 0 frame buffer address
> + * @fb_addr1: plane 1 frame buffer address
> + * @fb_addr2: plane 2 frame buffer address
> + */
> +struct venc_frm_buf {
> +	struct mtk_vcodec_mem fb_addr;
> +	struct mtk_vcodec_mem fb_addr1;
> +	struct mtk_vcodec_mem fb_addr2;
> +};
> +
> +/*
> + * struct venc_done_result - This is return information used in venc_if_encode()
> + * @msg: message, such as success or error code
> + * @bs_size: output bitstream size
> + * @is_key_frm: output is key frame or not
> + */
> +struct venc_done_result {
> +	enum venc_drv_msg msg;
> +	unsigned int bs_size;
> +	bool is_key_frm;
> +};
> +
> +/*
> + * venc_if_create - Create the driver handle
> + * @ctx: device context
> + * @fourcc: encoder output format
> + * @handle: driver handle
> + * Return: 0 if creating handle successfully, otherwise it is failed.
> + */
> +int venc_if_create(struct mtk_vcodec_ctx *ctx, unsigned int fourcc);
> +
> +/*
> + * venc_if_release - Release the driver handle
> + * @handle: driver handle
> + * Return: 0 if releasing handle successfully, otherwise it is failed.
> + */
> +int venc_if_release(struct mtk_vcodec_ctx *ctx);
> +
> +/*
> + * venc_if_set_param - Set parameter to driver
> + * @handle: driver handle
> + * @type: set type
> + * @in: input parameter
> + * @out: output parameter
> + * Return: 0 if setting param successfully, otherwise it is failed.
> + */
> +int venc_if_set_param(struct mtk_vcodec_ctx *ctx,
> +		      enum venc_set_param_type type,
> +		      void *in);
> +
> +/*
> + * venc_if_encode - Encode frame
> + * @handle: driver handle
> + * @opt: encode frame option
> + * @frm_buf: input frame buffer information
> + * @bs_buf: output bitstream buffer infomraiton
> + * @result: encode result
> + * Return: 0 if encoding frame successfully, otherwise it is failed.
> + */
> +int venc_if_encode(struct mtk_vcodec_ctx *ctx,
> +		   enum venc_start_opt opt,
> +		   struct venc_frm_buf *frm_buf,
> +		   struct mtk_vcodec_mem *bs_buf,
> +		   struct venc_done_result *result);
> +
> +#endif /* _VENC_DRV_IF_H_ */
> diff --git a/drivers/media/platform/mtk-vcodec/venc_ipi_msg.h b/drivers/media/platform/mtk-vcodec/venc_ipi_msg.h
> new file mode 100644
> index 0000000..a345b98
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vcodec/venc_ipi_msg.h
> @@ -0,0 +1,212 @@
> +/*
> + * Copyright (c) 2015 MediaTek Inc.
> + * Author: Jungchang Tsao <jungchang.tsao@mediatek.com>
> + *         Daniel Hsiao <daniel.hsiao@mediatek.com>
> + *
> + * This program is free software; you can redistribute it and/or
> + * modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + */
> +
> +#ifndef _VENC_IPI_MSG_H_
> +#define _VENC_IPI_MSG_H_
> +
> +#define IPIMSG_H264_ENC_ID 0x100
> +#define IPIMSG_VP8_ENC_ID 0x200
> +
> +#define AP_IPIMSG_VENC_BASE 0x20000
> +#define VPU_IPIMSG_VENC_BASE 0x30000
> +
> +/**
> + * enum venc_ipi_msg_id - message id between AP and VPU
> + * (ipi stands for inter-processor interrupt)
> + * @AP_IPIMSG_XXX:		AP to VPU cmd message id
> + * @VPU_IPIMSG_XXX_DONE:	VPU ack AP cmd message id
> + */
> +enum venc_ipi_msg_id {
> +	AP_IPIMSG_H264_ENC_INIT = AP_IPIMSG_VENC_BASE +
> +				  IPIMSG_H264_ENC_ID,
> +	AP_IPIMSG_H264_ENC_SET_PARAM,
> +	AP_IPIMSG_H264_ENC_ENCODE,
> +	AP_IPIMSG_H264_ENC_DEINIT,
> +
> +	AP_IPIMSG_VP8_ENC_INIT = AP_IPIMSG_VENC_BASE +
> +				 IPIMSG_VP8_ENC_ID,
> +	AP_IPIMSG_VP8_ENC_SET_PARAM,
> +	AP_IPIMSG_VP8_ENC_ENCODE,
> +	AP_IPIMSG_VP8_ENC_DEINIT,
> +
> +	VPU_IPIMSG_H264_ENC_INIT_DONE = VPU_IPIMSG_VENC_BASE +
> +					IPIMSG_H264_ENC_ID,
> +	VPU_IPIMSG_H264_ENC_SET_PARAM_DONE,
> +	VPU_IPIMSG_H264_ENC_ENCODE_DONE,
> +	VPU_IPIMSG_H264_ENC_DEINIT_DONE,
> +
> +	VPU_IPIMSG_VP8_ENC_INIT_DONE = VPU_IPIMSG_VENC_BASE +
> +				       IPIMSG_VP8_ENC_ID,
> +	VPU_IPIMSG_VP8_ENC_SET_PARAM_DONE,
> +	VPU_IPIMSG_VP8_ENC_ENCODE_DONE,
> +	VPU_IPIMSG_VP8_ENC_DEINIT_DONE,
> +};
> +
> +/**
> + * struct venc_ap_ipi_msg_init - AP to VPU init cmd structure
> + * @msg_id:	message id (AP_IPIMSG_XXX_ENC_INIT)
> + * @venc_inst:	AP encoder instance (struct venc_vp8_handle/venc_h264_handle *)
> + */
> +struct venc_ap_ipi_msg_init {
> +	uint32_t msg_id;
> +	uint32_t reserved;
> +	uint64_t venc_inst;
> +};
> +
> +/**
> + * struct venc_ap_ipi_msg_set_param - AP to VPU set_param cmd structure
> + * @msg_id:	message id (AP_IPIMSG_XXX_ENC_SET_PARAM)
> + * @inst_id:	VPU encoder instance id (struct venc_vp8_vpu_drv/venc_h264_vpu_drv *)
> + * @param_id:	parameter id (venc_set_param_type)
> + * @data_item:	number of items in the data array
> + * @data[8]:	data array to store the set parameters
> + */
> +struct venc_ap_ipi_msg_set_param {
> +	uint32_t msg_id;
> +	uint32_t inst_id;
> +	uint32_t param_id;
> +	uint32_t data_item;
> +	uint32_t data[8];
> +};
> +
> +/**
> + * struct venc_ap_ipi_msg_enc - AP to VPU enc cmd structure
> + * @msg_id:	message id (AP_IPIMSG_XXX_ENC_ENCODE)
> + * @inst_id:	VPU encoder instance id (struct venc_vp8_vpu_drv/venc_h264_vpu_drv *)
> + * @bs_mode:	bitstream mode for h264
> + *		(H264_BS_MODE_SPS/H264_BS_MODE_PPS/H264_BS_MODE_FRAME)
> + * @input_addr:	pointer to input image buffer plane
> + * @bs_addr:	pointer to output bit stream buffer
> + * @bs_size:	bit stream buffer size
> + */
> +struct venc_ap_ipi_msg_enc {
> +	uint32_t msg_id;
> +	uint32_t inst_id;
> +	uint32_t bs_mode;
> +	uint32_t input_addr[3];
> +	uint32_t bs_addr;
> +	uint32_t bs_size;
> +};
> +
> +/**
> + * struct venc_ap_ipi_msg_deinit - AP to VPU deinit cmd structure
> + * @msg_id:	message id (AP_IPIMSG_XXX_ENC_DEINIT)
> + * @inst_id:	VPU encoder instance id (struct venc_vp8_vpu_drv/venc_h264_vpu_drv *)
> + */
> +struct venc_ap_ipi_msg_deinit {
> +	uint32_t msg_id;
> +	uint32_t inst_id;
> +};
> +
> +/**
> + * enum venc_ipi_msg_status - VPU ack AP cmd status
> + */
> +enum venc_ipi_msg_status {
> +	VENC_IPI_MSG_STATUS_OK,
> +	VENC_IPI_MSG_STATUS_FAIL,
> +};
> +
> +/**
> + * struct venc_vpu_ipi_msg_common - VPU ack AP cmd common structure
> + * @msg_id:	message id (VPU_IPIMSG_XXX_DONE)
> + * @status:	cmd status (venc_ipi_msg_status)
> + * @venc_inst:	AP encoder instance (struct venc_vp8_handle/venc_h264_handle *)
> + */
> +struct venc_vpu_ipi_msg_common {
> +	uint32_t msg_id;
> +	uint32_t status;
> +	uint64_t venc_inst;
> +};
> +
> +/**
> + * struct venc_vpu_ipi_msg_init - VPU ack AP init cmd structure
> + * @msg_id:	message id (VPU_IPIMSG_XXX_ENC_SET_PARAM_DONE)
> + * @status:	cmd status (venc_ipi_msg_status)
> + * @venc_inst:	AP encoder instance (struct venc_vp8_handle/venc_h264_handle *)
> + * @inst_id:	VPU encoder instance id (struct venc_vp8_vpu_drv/venc_h264_vpu_drv *)
> + */
> +struct venc_vpu_ipi_msg_init {
> +	uint32_t msg_id;
> +	uint32_t status;
> +	uint64_t venc_inst;
> +	uint32_t inst_id;
> +	uint32_t reserved;
> +};
> +
> +/**
> + * struct venc_vpu_ipi_msg_set_param - VPU ack AP set_param cmd structure
> + * @msg_id:	message id (VPU_IPIMSG_XXX_ENC_SET_PARAM_DONE)
> + * @status:	cmd status (venc_ipi_msg_status)
> + * @venc_inst:	AP encoder instance (struct venc_vp8_handle/venc_h264_handle *)
> + * @param_id:	parameter id (venc_set_param_type)
> + * @data_item:	number of items in the data array
> + * @data[6]:	data array to store the return result
> + */
> +struct venc_vpu_ipi_msg_set_param {
> +	uint32_t msg_id;
> +	uint32_t status;
> +	uint64_t venc_inst;
> +	uint32_t param_id;
> +	uint32_t data_item;
> +	uint32_t data[6];
> +};
> +
> +/**
> + * enum venc_ipi_msg_enc_state - Type of encode state
> + * VEN_IPI_MSG_ENC_STATE_FRAME:	one frame being encoded
> + * VEN_IPI_MSG_ENC_STATE_PART:	bit stream buffer full
> + * VEN_IPI_MSG_ENC_STATE_SKIP:	encoded skip frame
> + * VEN_IPI_MSG_ENC_STATE_ERROR:	encounter error
> + */
> +enum venc_ipi_msg_enc_state {
> +	VEN_IPI_MSG_ENC_STATE_FRAME,
> +	VEN_IPI_MSG_ENC_STATE_PART,
> +	VEN_IPI_MSG_ENC_STATE_SKIP,
> +	VEN_IPI_MSG_ENC_STATE_ERROR,
> +};
> +
> +/**
> + * struct venc_vpu_ipi_msg_enc - VPU ack AP enc cmd structure
> + * @msg_id:	message id (VPU_IPIMSG_XXX_ENC_ENCODE_DONE)
> + * @status:	cmd status (venc_ipi_msg_status)
> + * @venc_inst:	AP encoder instance (struct venc_vp8_handle/venc_h264_handle *)
> + * @state:	encode state (venc_ipi_msg_enc_state)
> + * @key_frame:	whether the encoded frame is key frame
> + * @bs_size:	encoded bitstream size

'reserved' isn't documented. I assume this should be set to 0?

> + */
> +struct venc_vpu_ipi_msg_enc {
> +	uint32_t msg_id;
> +	uint32_t status;
> +	uint64_t venc_inst;
> +	uint32_t state;
> +	uint32_t key_frame;
> +	uint32_t bs_size;
> +	uint32_t reserved;
> +};
> +
> +/**
> + * struct venc_vpu_ipi_msg_deinit - VPU ack AP deinit cmd structure
> + * @msg_id:   message id (VPU_IPIMSG_XXX_ENC_DEINIT_DONE)
> + * @status:   cmd status (venc_ipi_msg_status)
> + * @venc_inst:	AP encoder instance (struct venc_vp8_handle/venc_h264_handle *)
> + */
> +struct venc_vpu_ipi_msg_deinit {
> +	uint32_t msg_id;
> +	uint32_t status;
> +	uint64_t venc_inst;
> +};
> +
> +#endif /* _VENC_IPI_MSG_H_ */
> diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
> old mode 100644
> new mode 100755
> index ee9d530..3ac35c4
> --- a/include/uapi/linux/v4l2-controls.h
> +++ b/include/uapi/linux/v4l2-controls.h
> @@ -646,6 +646,10 @@ enum v4l2_mpeg_mfc51_video_force_frame_type {
>  #define V4L2_CID_MPEG_MFC51_VIDEO_H264_NUM_REF_PIC_FOR_P		(V4L2_CID_MPEG_MFC51_BASE+54)
>  
>  
> +#define V4L2_CID_MPEG_MTK_BASE					(V4L2_CTRL_CLASS_MPEG | 0x5500)
> +#define V4L2_CID_MPEG_MTK_VIDEO_FRAME_SKIP_ENABLE		(V4L2_CID_MPEG_MTK_BASE+0)
> +#define V4L2_CID_MPEG_MTK_VIDEO_FORCE_FRAME_TYPE_I_FRAME	(V4L2_CID_MPEG_MTK_BASE+1)

Please note that these controls should be documented in DocBook.

Regards,

	Hans

> +
>  /*  Camera class control IDs */
>  
>  #define V4L2_CID_CAMERA_CLASS_BASE 	(V4L2_CTRL_CLASS_CAMERA | 0x900)
> 

^ permalink raw reply	[flat|nested] 102+ messages in thread

* Re: [PATCH v4 2/8] [media] VPU: mediatek: support Mediatek VPU
  2016-02-15 10:13       ` Hans Verkuil
  (?)
@ 2016-02-15 11:27         ` tiffany lin
  -1 siblings, 0 replies; 102+ messages in thread
From: tiffany lin @ 2016-02-15 11:27 UTC (permalink / raw)
  To: Hans Verkuil
  Cc: Hans Verkuil, daniel.thompson, Rob Herring,
	Mauro Carvalho Chehab, Matthias Brugger, Daniel Kurtz,
	Pawel Osciak, Eddie Huang, Yingjoe Chen, devicetree,
	linux-kernel, linux-arm-kernel, linux-media, linux-mediatek,
	PoChun.Lin, Andrew-CT Chen

On Mon, 2016-02-15 at 11:13 +0100, Hans Verkuil wrote:
> On 02/04/2016 12:34 PM, Tiffany Lin wrote:
> > The VPU driver for hw video codec embedded in Mediatek's MT8173 SOCs.
> > It is able to handle video decoding/encoding of in a range of formats.
> > The driver provides with VPU firmware download, memory management and
> > the communication interface between CPU and VPU.
> > For VPU initialization, it will create virtual memory for CPU access and
> > IOMMU address for vcodec hw device access. When a decode/encode instance
> > opens a device node, vpu driver will download vpu firmware to the device.
> > A decode/encode instant will decode/encode a frame using VPU
> > interface to interrupt vpu to handle decoding/encoding jobs.
> > 
> > Signed-off-by: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
> > Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
> > ---
> >  drivers/media/platform/Kconfig           |    9 +
> >  drivers/media/platform/Makefile          |    2 +
> >  drivers/media/platform/mtk-vpu/Makefile  |    1 +
> >  drivers/media/platform/mtk-vpu/mtk_vpu.c |  994 ++++++++++++++++++++++++++++++
> >  drivers/media/platform/mtk-vpu/mtk_vpu.h |  167 +++++
> >  5 files changed, 1173 insertions(+)
> >  create mode 100644 drivers/media/platform/mtk-vpu/Makefile
> >  create mode 100644 drivers/media/platform/mtk-vpu/mtk_vpu.c
> >  create mode 100644 drivers/media/platform/mtk-vpu/mtk_vpu.h
> > 
> > diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
> > index ccbc974..ba812d6 100644
> > --- a/drivers/media/platform/Kconfig
> > +++ b/drivers/media/platform/Kconfig
> > @@ -148,6 +148,15 @@ config VIDEO_CODA
> >  	   Coda is a range of video codec IPs that supports
> >  	   H.264, MPEG-4, and other video formats.
> >  
> > +config VIDEO_MEDIATEK_VPU
> > +	tristate "Mediatek Video Processor Unit"
> > +	depends on VIDEO_DEV && VIDEO_V4L2 && ARCH_MEDIATEK
> > +	---help---
> > +	    This driver provides downloading VPU firmware and
> > +	    communicating with VPU. This driver for hw video
> > +	    codec embedded in new Mediatek's SOCs. It is able
> > +	    to handle video decoding/encoding in a range of formats.
> 
> Can you be more specific in this text and mention for which Mediatek SoCs
> this driver is for? Just like you did in the commit log.
> 
Got it. We will add more specific description in Kconfig in next
version.

> Also add something like this:
> 
>           To compile this driver as a module, choose M here: the module
>           will be called mtk-vpu.
> 
> I always find it useful if the Kconfig text mentions the module name.
> 
Got it. We will fix this. Thanks for sharing experience.


best regards,
Tiffany

> Regards,
> 
> 	Hans
> 
> > +
> >  config VIDEO_MEM2MEM_DEINTERLACE
> >  	tristate "Deinterlace support"
> >  	depends on VIDEO_DEV && VIDEO_V4L2 && DMA_ENGINE
> 

^ permalink raw reply	[flat|nested] 102+ messages in thread

* Re: [PATCH v4 2/8] [media] VPU: mediatek: support Mediatek VPU
@ 2016-02-15 11:27         ` tiffany lin
  0 siblings, 0 replies; 102+ messages in thread
From: tiffany lin @ 2016-02-15 11:27 UTC (permalink / raw)
  To: Hans Verkuil
  Cc: Hans Verkuil, daniel.thompson, Rob Herring,
	Mauro Carvalho Chehab, Matthias Brugger, Daniel Kurtz,
	Pawel Osciak, Eddie Huang, Yingjoe Chen, devicetree,
	linux-kernel, linux-arm-kernel, linux-media, linux-mediatek,
	PoChun.Lin, Andrew-CT Chen

On Mon, 2016-02-15 at 11:13 +0100, Hans Verkuil wrote:
> On 02/04/2016 12:34 PM, Tiffany Lin wrote:
> > The VPU driver for hw video codec embedded in Mediatek's MT8173 SOCs.
> > It is able to handle video decoding/encoding of in a range of formats.
> > The driver provides with VPU firmware download, memory management and
> > the communication interface between CPU and VPU.
> > For VPU initialization, it will create virtual memory for CPU access and
> > IOMMU address for vcodec hw device access. When a decode/encode instance
> > opens a device node, vpu driver will download vpu firmware to the device.
> > A decode/encode instant will decode/encode a frame using VPU
> > interface to interrupt vpu to handle decoding/encoding jobs.
> > 
> > Signed-off-by: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
> > Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
> > ---
> >  drivers/media/platform/Kconfig           |    9 +
> >  drivers/media/platform/Makefile          |    2 +
> >  drivers/media/platform/mtk-vpu/Makefile  |    1 +
> >  drivers/media/platform/mtk-vpu/mtk_vpu.c |  994 ++++++++++++++++++++++++++++++
> >  drivers/media/platform/mtk-vpu/mtk_vpu.h |  167 +++++
> >  5 files changed, 1173 insertions(+)
> >  create mode 100644 drivers/media/platform/mtk-vpu/Makefile
> >  create mode 100644 drivers/media/platform/mtk-vpu/mtk_vpu.c
> >  create mode 100644 drivers/media/platform/mtk-vpu/mtk_vpu.h
> > 
> > diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
> > index ccbc974..ba812d6 100644
> > --- a/drivers/media/platform/Kconfig
> > +++ b/drivers/media/platform/Kconfig
> > @@ -148,6 +148,15 @@ config VIDEO_CODA
> >  	   Coda is a range of video codec IPs that supports
> >  	   H.264, MPEG-4, and other video formats.
> >  
> > +config VIDEO_MEDIATEK_VPU
> > +	tristate "Mediatek Video Processor Unit"
> > +	depends on VIDEO_DEV && VIDEO_V4L2 && ARCH_MEDIATEK
> > +	---help---
> > +	    This driver provides downloading VPU firmware and
> > +	    communicating with VPU. This driver for hw video
> > +	    codec embedded in new Mediatek's SOCs. It is able
> > +	    to handle video decoding/encoding in a range of formats.
> 
> Can you be more specific in this text and mention for which Mediatek SoCs
> this driver is for? Just like you did in the commit log.
> 
Got it. We will add more specific description in Kconfig in next
version.

> Also add something like this:
> 
>           To compile this driver as a module, choose M here: the module
>           will be called mtk-vpu.
> 
> I always find it useful if the Kconfig text mentions the module name.
> 
Got it. We will fix this. Thanks for sharing experience.


best regards,
Tiffany

> Regards,
> 
> 	Hans
> 
> > +
> >  config VIDEO_MEM2MEM_DEINTERLACE
> >  	tristate "Deinterlace support"
> >  	depends on VIDEO_DEV && VIDEO_V4L2 && DMA_ENGINE
> 

^ permalink raw reply	[flat|nested] 102+ messages in thread

* [PATCH v4 2/8] [media] VPU: mediatek: support Mediatek VPU
@ 2016-02-15 11:27         ` tiffany lin
  0 siblings, 0 replies; 102+ messages in thread
From: tiffany lin @ 2016-02-15 11:27 UTC (permalink / raw)
  To: linux-arm-kernel

On Mon, 2016-02-15 at 11:13 +0100, Hans Verkuil wrote:
> On 02/04/2016 12:34 PM, Tiffany Lin wrote:
> > The VPU driver for hw video codec embedded in Mediatek's MT8173 SOCs.
> > It is able to handle video decoding/encoding of in a range of formats.
> > The driver provides with VPU firmware download, memory management and
> > the communication interface between CPU and VPU.
> > For VPU initialization, it will create virtual memory for CPU access and
> > IOMMU address for vcodec hw device access. When a decode/encode instance
> > opens a device node, vpu driver will download vpu firmware to the device.
> > A decode/encode instant will decode/encode a frame using VPU
> > interface to interrupt vpu to handle decoding/encoding jobs.
> > 
> > Signed-off-by: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
> > Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
> > ---
> >  drivers/media/platform/Kconfig           |    9 +
> >  drivers/media/platform/Makefile          |    2 +
> >  drivers/media/platform/mtk-vpu/Makefile  |    1 +
> >  drivers/media/platform/mtk-vpu/mtk_vpu.c |  994 ++++++++++++++++++++++++++++++
> >  drivers/media/platform/mtk-vpu/mtk_vpu.h |  167 +++++
> >  5 files changed, 1173 insertions(+)
> >  create mode 100644 drivers/media/platform/mtk-vpu/Makefile
> >  create mode 100644 drivers/media/platform/mtk-vpu/mtk_vpu.c
> >  create mode 100644 drivers/media/platform/mtk-vpu/mtk_vpu.h
> > 
> > diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
> > index ccbc974..ba812d6 100644
> > --- a/drivers/media/platform/Kconfig
> > +++ b/drivers/media/platform/Kconfig
> > @@ -148,6 +148,15 @@ config VIDEO_CODA
> >  	   Coda is a range of video codec IPs that supports
> >  	   H.264, MPEG-4, and other video formats.
> >  
> > +config VIDEO_MEDIATEK_VPU
> > +	tristate "Mediatek Video Processor Unit"
> > +	depends on VIDEO_DEV && VIDEO_V4L2 && ARCH_MEDIATEK
> > +	---help---
> > +	    This driver provides downloading VPU firmware and
> > +	    communicating with VPU. This driver for hw video
> > +	    codec embedded in new Mediatek's SOCs. It is able
> > +	    to handle video decoding/encoding in a range of formats.
> 
> Can you be more specific in this text and mention for which Mediatek SoCs
> this driver is for? Just like you did in the commit log.
> 
Got it. We will add more specific description in Kconfig in next
version.

> Also add something like this:
> 
>           To compile this driver as a module, choose M here: the module
>           will be called mtk-vpu.
> 
> I always find it useful if the Kconfig text mentions the module name.
> 
Got it. We will fix this. Thanks for sharing experience.


best regards,
Tiffany

> Regards,
> 
> 	Hans
> 
> > +
> >  config VIDEO_MEM2MEM_DEINTERLACE
> >  	tristate "Deinterlace support"
> >  	depends on VIDEO_DEV && VIDEO_V4L2 && DMA_ENGINE
> 

^ permalink raw reply	[flat|nested] 102+ messages in thread

* Re: [PATCH v4 7/8] [media] vcodec: mediatek: Add Mediatek H264 Video Encoder Driver
  2016-02-04 11:35               ` Tiffany Lin
@ 2016-02-15 11:33                 ` Hans Verkuil
  -1 siblings, 0 replies; 102+ messages in thread
From: Hans Verkuil @ 2016-02-15 11:33 UTC (permalink / raw)
  To: Tiffany Lin, Hans Verkuil, daniel.thompson, Rob Herring,
	Mauro Carvalho Chehab, Matthias Brugger, Daniel Kurtz,
	Pawel Osciak
  Cc: Eddie Huang, Yingjoe Chen, devicetree, linux-kernel,
	linux-arm-kernel, linux-media, linux-mediatek, PoChun.Lin,
	Daniel Hsiao

On 02/04/2016 12:35 PM, Tiffany Lin wrote:
> Add h264 encoder driver for MT8173
> 
> Signed-off-by: Daniel Hsiao <daniel.hsiao@mediatek.com>
> Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
> ---
>  drivers/media/platform/mtk-vcodec/Makefile         |    3 +-
>  .../media/platform/mtk-vcodec/h264_enc/Makefile    |    6 +
>  .../platform/mtk-vcodec/h264_enc/venc_h264_if.c    |  540 ++++++++++++++++++++
>  .../platform/mtk-vcodec/h264_enc/venc_h264_if.h    |  165 ++++++
>  .../platform/mtk-vcodec/h264_enc/venc_h264_vpu.c   |  309 +++++++++++
>  .../platform/mtk-vcodec/h264_enc/venc_h264_vpu.h   |   30 ++
>  drivers/media/platform/mtk-vcodec/venc_drv_if.c    |    3 +
>  7 files changed, 1055 insertions(+), 1 deletion(-)
>  create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/Makefile
>  create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.c
>  create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.h
>  create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.c
>  create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.h
> 
> diff --git a/drivers/media/platform/mtk-vcodec/Makefile b/drivers/media/platform/mtk-vcodec/Makefile
> index f4ef502..f47dfc7 100644
> --- a/drivers/media/platform/mtk-vcodec/Makefile
> +++ b/drivers/media/platform/mtk-vcodec/Makefile
> @@ -5,6 +5,7 @@ obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += mtk_vcodec_intr.o \
>  				       mtk_vcodec_enc_pm.o \
>  				       venc_drv_if.o
>  
> -obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += vp8_enc/
> +
> +obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += vp8_enc/ h264_enc/
>  
>  ccflags-y += -I$(srctree)/drivers/media/platform/mtk-vpu
> diff --git a/drivers/media/platform/mtk-vcodec/h264_enc/Makefile b/drivers/media/platform/mtk-vcodec/h264_enc/Makefile
> new file mode 100644
> index 0000000..765b45f
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vcodec/h264_enc/Makefile
> @@ -0,0 +1,6 @@
> +obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += venc_h264_if.o venc_h264_vpu.o
> +
> +ccflags-y += \
> +    -I$(srctree)/drivers/media/platform/mtk-vcodec/ \
> +    -I$(srctree)/drivers/media/platform/mtk-vcodec/h264_enc \
> +    -I$(srctree)/drivers/media/platform/mtk-vpu
> diff --git a/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.c b/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.c
> new file mode 100644
> index 0000000..c35fb26
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.c
> @@ -0,0 +1,540 @@
> +/*
> + * Copyright (c) 2015 MediaTek Inc.
> + * Author: Jungchang Tsao <jungchang.tsao@mediatek.com>
> + *         Daniel Hsiao <daniel.hsiao@mediatek.com>
> + *         PoChun Lin <pochun.lin@mediatek.com>
> + *
> + * This program is free software; you can redistribute it and/or
> + * modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + */
> +
> +#include <linux/interrupt.h>
> +#include <linux/kernel.h>
> +#include <linux/slab.h>
> +
> +#include "mtk_vcodec_drv.h"
> +#include "mtk_vcodec_util.h"
> +#include "mtk_vcodec_intr.h"
> +#include "mtk_vcodec_enc.h"
> +#include "mtk_vcodec_pm.h"
> +#include "mtk_vpu.h"
> +
> +#include "venc_h264_if.h"
> +#include "venc_h264_vpu.h"
> +
> +#define VENC_PIC_BITSTREAM_BYTE_CNT 0x0098
> +
> +enum venc_h264_irq_status {
> +	H264_IRQ_STATUS_ENC_SPS_INT = (1 << 0),
> +	H264_IRQ_STATUS_ENC_PPS_INT = (1 << 1),
> +	H264_IRQ_STATUS_ENC_FRM_INT = (1 << 2),
> +};
> +
> +static inline void h264_write_reg(struct venc_h264_inst *inst, u32 addr,
> +				  u32 val)
> +{
> +	writel(val, inst->hw_base + addr);
> +}
> +
> +static inline u32 h264_read_reg(struct venc_h264_inst *inst, u32 addr)
> +{
> +	return readl(inst->hw_base + addr);
> +}
> +
> +static void h264_enc_free_work_buf(struct venc_h264_inst *inst)
> +{
> +	int i;
> +
> +	mtk_vcodec_debug_enter(inst);
> +
> +	/* Except the SKIP_FRAME buffers,
> +	 * other buffers need to be freed by AP.
> +	 */
> +	for (i = 0; i < VENC_H264_VPU_WORK_BUF_MAX; i++) {
> +		if (i != VENC_H264_VPU_WORK_BUF_SKIP_FRAME)
> +			if (inst->work_bufs[i].va != NULL)
> +				mtk_vcodec_mem_free(inst->ctx,
> +						    &inst->work_bufs[i]);
> +	}
> +
> +	if (inst->pps_buf.va != NULL)
> +		mtk_vcodec_mem_free(inst->ctx, &inst->pps_buf);
> +
> +	mtk_vcodec_debug_leave(inst);
> +}
> +
> +static int h264_enc_alloc_work_buf(struct venc_h264_inst *inst, void *param)
> +{
> +	int i;
> +	int ret = 0;
> +	struct venc_h264_vpu_buf *wb = inst->vpu_inst.drv->work_bufs;
> +	struct venc_enc_prm *enc_param = param;
> +
> +	mtk_vcodec_debug_enter(inst);
> +
> +	for (i = 0; i < VENC_H264_VPU_WORK_BUF_MAX; i++) {
> +		/*
> +		 * This 'wb' structure is set by VPU side and shared to AP for
> +		 * buffer allocation and IO virtual addr mapping. For most of
> +		 * the buffers, AP will allocate the buffer according to 'size'
> +		 * field and store the IO virtual addr in 'iova' field. There
> +		 * are two exceptions:
> +		 * (1) RC_CODE buffer, it's pre-allocated in the VPU side, and
> +		 * save the VPU addr in the 'vpua' field. The AP will translate
> +		 * the VPU addr to the corresponding IO virtual addr and store
> +		 * in 'iova' field for reg setting in VPU side.
> +		 * (2) SKIP_FRAME buffer, it's pre-allocated in the VPU side,
> +		 * and save the VPU addr in the 'vpua' field. The AP will
> +		 * translate the VPU addr to the corresponding AP side virtual
> +		 * address and do some memcpy access to move to bitstream buffer
> +		 * assigned by v4l2 layer.
> +		 */
> +		if (i == VENC_H264_VPU_WORK_BUF_RC_CODE) {
> +			void *tmp_va;
> +
> +			tmp_va = vpu_mapping_dm_addr(inst->dev, wb[i].vpua);
> +			inst->work_bufs[i].size = wb[i].size;
> +			if (mtk_vcodec_mem_alloc(inst->ctx,
> +						 &inst->work_bufs[i])) {
> +				mtk_vcodec_err(inst,
> +					       "cannot allocate buf %d", i);
> +				ret = -ENOMEM;
> +				goto err_alloc;
> +			}
> +			memcpy(inst->work_bufs[i].va, tmp_va, wb[i].size);
> +			wb[i].iova = inst->work_bufs[i].dma_addr;
> +		} else if (i == VENC_H264_VPU_WORK_BUF_SKIP_FRAME) {
> +			inst->work_bufs[i].size = wb[i].size;
> +			inst->work_bufs[i].va = vpu_mapping_dm_addr(
> +				inst->dev, wb[i].vpua);
> +			inst->work_bufs[i].dma_addr = 0;
> +			wb[i].iova = inst->work_bufs[i].dma_addr;
> +		} else if (i == VENC_H264_VPU_WORK_BUF_SRC_LUMA ||
> +			   i == VENC_H264_VPU_WORK_BUF_SRC_CHROMA ||
> +			   i == VENC_H264_VPU_WORK_BUF_SRC_CHROMA_CB ||
> +			   i == VENC_H264_VPU_WORK_BUF_SRC_CHROMA_CR) {
> +			inst->work_bufs[i].size = wb[i].size;
> +			inst->work_bufs[i].dma_addr = 0;
> +			inst->work_bufs[i].va = NULL;
> +			wb[i].iova = inst->work_bufs[i].dma_addr;
> +		} else {
> +			inst->work_bufs[i].size = wb[i].size;
> +			if (mtk_vcodec_mem_alloc(inst->ctx,
> +						 &inst->work_bufs[i])) {
> +				mtk_vcodec_err(inst,
> +					       "cannot allocate buf %d", i);
> +				ret = -ENOMEM;
> +				goto err_alloc;
> +			}
> +			wb[i].iova = inst->work_bufs[i].dma_addr;
> +		}
> +		mtk_vcodec_debug(inst, "buf[%d] va=0x%p iova=0x%p size=0x%lx",
> +				 i, inst->work_bufs[i].va,
> +				 (void *)inst->work_bufs[i].dma_addr,
> +				 inst->work_bufs[i].size);
> +	}
> +
> +	if (enc_param->input_fourcc == VENC_YUV_FORMAT_NV12 ||
> +	    enc_param->input_fourcc == VENC_YUV_FORMAT_NV21) {
> +		enc_param->sizeimage[0] =
> +			inst->work_bufs[VENC_H264_VPU_WORK_BUF_SRC_LUMA].size;
> +		enc_param->sizeimage[1] =
> +			inst->work_bufs[VENC_H264_VPU_WORK_BUF_SRC_CHROMA].size;
> +		enc_param->sizeimage[2] = 0;
> +	} else {
> +		enc_param->sizeimage[0] =
> +			inst->work_bufs[VENC_H264_VPU_WORK_BUF_SRC_LUMA].size;
> +		enc_param->sizeimage[1] =
> +			inst->work_bufs[VENC_H264_VPU_WORK_BUF_SRC_CHROMA_CB].size;
> +		enc_param->sizeimage[2] =
> +			inst->work_bufs[VENC_H264_VPU_WORK_BUF_SRC_CHROMA_CR].size;
> +	}
> +
> +	/* the pps_buf is used by AP side only */
> +	inst->pps_buf.size = 128;
> +	if (mtk_vcodec_mem_alloc(inst->ctx,
> +				 &inst->pps_buf)) {
> +		mtk_vcodec_err(inst, "cannot allocate pps_buf");
> +		ret = -ENOMEM;
> +		goto err_alloc;
> +	}
> +	mtk_vcodec_debug_leave(inst);
> +
> +	return ret;
> +
> +err_alloc:
> +	h264_enc_free_work_buf(inst);
> +
> +	return ret;
> +}
> +
> +static unsigned int h264_enc_wait_venc_done(struct venc_h264_inst *inst)
> +{
> +	unsigned int irq_status = 0;
> +	struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)inst->ctx;
> +
> +	mtk_vcodec_wait_for_done_ctx(ctx, MTK_INST_IRQ_RECEIVED,
> +				     WAIT_INTR_TIMEOUT, true);
> +	irq_status = ctx->irq_status;
> +	mtk_vcodec_debug(inst, "irq_status %x <-", irq_status);
> +
> +	return irq_status;
> +}
> +
> +static int h264_encode_sps(struct venc_h264_inst *inst,
> +			   struct mtk_vcodec_mem *bs_buf,
> +			   unsigned int *bs_size)
> +{
> +	int ret = 0;
> +	unsigned int irq_status;
> +
> +	mtk_vcodec_debug_enter(inst);
> +
> +	ret = h264_enc_vpu_encode(inst, H264_BS_MODE_SPS, NULL,
> +				  bs_buf, bs_size);
> +	if (ret)
> +		return ret;
> +
> +	irq_status = h264_enc_wait_venc_done(inst);
> +	if (irq_status != H264_IRQ_STATUS_ENC_SPS_INT) {
> +		mtk_vcodec_err(inst, "expect irq status %d",
> +			       H264_IRQ_STATUS_ENC_SPS_INT);
> +		return -EINVAL;
> +	}
> +
> +	*bs_size = h264_read_reg(inst, VENC_PIC_BITSTREAM_BYTE_CNT);
> +	mtk_vcodec_debug(inst, "bs size %d <-", *bs_size);
> +
> +	return ret;
> +}
> +
> +static int h264_encode_pps(struct venc_h264_inst *inst,
> +			   struct mtk_vcodec_mem *bs_buf,
> +			   unsigned int *bs_size)
> +{
> +	int ret = 0;
> +	unsigned int irq_status;
> +
> +	mtk_vcodec_debug_enter(inst);
> +
> +	ret = h264_enc_vpu_encode(inst, H264_BS_MODE_PPS, NULL,
> +				  bs_buf, bs_size);
> +	if (ret)
> +		return ret;
> +
> +	irq_status = h264_enc_wait_venc_done(inst);
> +	if (irq_status != H264_IRQ_STATUS_ENC_PPS_INT) {
> +		mtk_vcodec_err(inst, "expect irq status %d",
> +			       H264_IRQ_STATUS_ENC_PPS_INT);
> +		return -EINVAL;
> +	}
> +
> +	*bs_size = h264_read_reg(inst, VENC_PIC_BITSTREAM_BYTE_CNT);
> +	mtk_vcodec_debug(inst, "bs size %d <-", *bs_size);
> +
> +	return ret;
> +}
> +
> +static int h264_encode_frame(struct venc_h264_inst *inst,
> +			     struct venc_frm_buf *frm_buf,
> +			     struct mtk_vcodec_mem *bs_buf,
> +			     unsigned int *bs_size)
> +{
> +	int ret = 0;
> +	unsigned int irq_status;
> +
> +	mtk_vcodec_debug_enter(inst);
> +
> +	ret = h264_enc_vpu_encode(inst, H264_BS_MODE_FRAME, frm_buf,
> +				  bs_buf, bs_size);
> +	if (ret)
> +		return ret;
> +
> +	/*
> +	 * skip frame case: The skip frame buffer is composed by vpu side only,
> +	 * it does not trigger the hw, so skip the wait interrupt operation.
> +	 */
> +	if (!inst->vpu_inst.wait_int) {
> +		++inst->frm_cnt;
> +		return ret;
> +	}
> +
> +	irq_status = h264_enc_wait_venc_done(inst);
> +	if (irq_status != H264_IRQ_STATUS_ENC_FRM_INT) {
> +		mtk_vcodec_err(inst, "irq_status=%d failed", irq_status);
> +		return -EINVAL;
> +	}
> +
> +	*bs_size = h264_read_reg(inst,
> +				 VENC_PIC_BITSTREAM_BYTE_CNT);
> +	++inst->frm_cnt;
> +	mtk_vcodec_debug(inst, "frm %d bs size %d key_frm %d <-",
> +			 inst->frm_cnt,
> +			 *bs_size, inst->is_key_frm);
> +
> +	return ret;
> +}
> +
> +static void h264_encode_filler(struct venc_h264_inst *inst, void *buf,
> +			       int size)
> +{
> +	unsigned char *p = buf;
> +
> +	*p++ = 0x0;
> +	*p++ = 0x0;
> +	*p++ = 0x0;
> +	*p++ = 0x1;
> +	*p++ = 0xc;
> +	size -= 5;
> +	while (size) {
> +		*p++ = 0xff;
> +		size -= 1;
> +	}
> +}
> +
> +static int h264_enc_init(struct mtk_vcodec_ctx *ctx, unsigned long *handle)
> +{
> +	int ret = 0;
> +	struct venc_h264_inst *inst;
> +
> +	inst = kzalloc(sizeof(*inst), GFP_KERNEL);
> +	if (!inst)
> +		return -ENOMEM;
> +
> +	inst->ctx = ctx;
> +	inst->dev = mtk_vcodec_get_plat_dev(ctx);
> +	inst->hw_base = mtk_vcodec_get_reg_addr(inst->ctx, VENC_SYS);
> +
> +	mtk_vcodec_debug_enter(inst);
> +	ret = h264_enc_vpu_init(inst);
> +	if (ret)
> +		kfree(inst);
> +	else
> +		(*handle) = (unsigned long)inst;
> +
> +	mtk_vcodec_debug_leave(inst);
> +
> +	return ret;
> +}
> +
> +static int h264_enc_encode(unsigned long handle,
> +			   enum venc_start_opt opt,
> +			   struct venc_frm_buf *frm_buf,
> +			   struct mtk_vcodec_mem *bs_buf,
> +			   struct venc_done_result *result)
> +{
> +	int ret = 0;
> +	struct venc_h264_inst *inst = (struct venc_h264_inst *)handle;
> +	struct mtk_vcodec_ctx *ctx = inst->ctx;
> +
> +	mtk_vcodec_debug(inst, "opt %d ->", opt);
> +
> +	enable_irq(ctx->dev->enc_irq);
> +
> +	switch (opt) {
> +	case VENC_START_OPT_ENCODE_SEQUENCE_HEADER: {
> +		unsigned int bs_size_sps;
> +		unsigned int bs_size_pps;
> +
> +		memset(bs_buf->va, 0x38, 20);
> +		ret = h264_encode_sps(inst, bs_buf, &bs_size_sps);
> +		if (ret)
> +			goto encode_err;
> +
> +		memset(inst->pps_buf.va, 0x49, 20);
> +		ret = h264_encode_pps(inst, &inst->pps_buf, &bs_size_pps);
> +		if (ret)
> +			goto encode_err;
> +
> +		memcpy(bs_buf->va + bs_size_sps,
> +		       inst->pps_buf.va,
> +		       bs_size_pps);
> +		result->bs_size = bs_size_sps + bs_size_pps;
> +		result->is_key_frm = false;
> +	}
> +	break;

Wrong indentation for break, put it before the '}'.

> +
> +	case VENC_START_OPT_ENCODE_FRAME:
> +		if (inst->prepend_hdr) {

Invert the condition to make it more readable:

		if (!inst->prepend_hdr) {
			ret = h264_encode_frame(inst, frm_buf, bs_buf,
						&result->bs_size);
			if (ret)
				goto encode_err;
			result->is_key_frm = inst->is_key_frm;
			break;
		}

Now you can shift the remainder one tab to the left and join lines that
are now broken up (probably because they exceed 80 chars).

> +			int hdr_sz;
> +			int hdr_sz_ext;
> +			int bs_alignment = 128;
> +			int filler_sz = 0;
> +			struct mtk_vcodec_mem tmp_bs_buf;
> +			unsigned int bs_size_sps;
> +			unsigned int bs_size_pps;
> +			unsigned int bs_size_frm;
> +
> +			mtk_vcodec_debug(inst,
> +					 "h264_encode_frame prepend SPS/PPS");
> +			ret = h264_encode_sps(inst, bs_buf, &bs_size_sps);
> +			if (ret)
> +				goto encode_err;
> +
> +			ret = h264_encode_pps(inst, &inst->pps_buf,
> +					      &bs_size_pps);
> +			if (ret)
> +				goto encode_err;
> +
> +			memcpy(bs_buf->va + bs_size_sps,
> +			       inst->pps_buf.va,
> +			       bs_size_pps);
> +
> +			hdr_sz = bs_size_sps + bs_size_pps;
> +			hdr_sz_ext = (hdr_sz & (bs_alignment - 1));
> +			if (hdr_sz_ext) {
> +				filler_sz = bs_alignment - hdr_sz_ext;
> +				if (hdr_sz_ext + 5 > bs_alignment)
> +					filler_sz += bs_alignment;
> +				h264_encode_filler(
> +					inst, bs_buf->va + hdr_sz,
> +					filler_sz);
> +			}
> +
> +			tmp_bs_buf.va = bs_buf->va + hdr_sz +
> +				filler_sz;
> +			tmp_bs_buf.dma_addr = bs_buf->dma_addr + hdr_sz +
> +				filler_sz;
> +			tmp_bs_buf.size = bs_buf->size -
> +				(hdr_sz + filler_sz);
> +
> +			ret = h264_encode_frame(inst, frm_buf, &tmp_bs_buf,
> +						&bs_size_frm);
> +			if (ret)
> +				goto encode_err;
> +
> +			result->bs_size = hdr_sz + filler_sz + bs_size_frm;
> +			mtk_vcodec_debug(inst,
> +					 "hdr %d filler %d frame %d bs %d",
> +					 hdr_sz, filler_sz, bs_size_frm,
> +					 result->bs_size);
> +
> +			inst->prepend_hdr = 0;
> +		} else {
> +			ret = h264_encode_frame(inst, frm_buf, bs_buf,
> +						&result->bs_size);
> +			if (ret)
> +				goto encode_err;
> +		}
> +		result->is_key_frm = inst->is_key_frm;
> +		break;
> +
> +	default:
> +		mtk_vcodec_err(inst, "venc_start_opt %d not supported", opt);
> +		ret = -EINVAL;
> +		break;
> +	}
> +
> +encode_err:
> +	if (ret)
> +		result->msg = VENC_MESSAGE_ERR;
> +	else
> +		result->msg = VENC_MESSAGE_OK;
> +
> +	disable_irq(ctx->dev->enc_irq);
> +	mtk_vcodec_debug(inst, "opt %d <-", opt);
> +
> +	return ret;
> +}
> +
> +static int h264_enc_set_param(unsigned long handle,
> +			      enum venc_set_param_type type, void *in)
> +{
> +	int ret = 0;
> +	struct venc_h264_inst *inst = (struct venc_h264_inst *)handle;
> +	struct venc_enc_prm *enc_prm;
> +
> +	mtk_vcodec_debug(inst, "->type=%d", type);
> +
> +	switch (type) {
> +	case VENC_SET_PARAM_ENC:
> +		enc_prm = in;
> +		ret = h264_enc_vpu_set_param(inst, type, enc_prm);
> +		if (ret)
> +			break;
> +		if (inst->work_buf_allocated == 1) {
> +			h264_enc_free_work_buf(inst);
> +			inst->work_buf_allocated = 0;
> +		}
> +		if (inst->work_buf_allocated == 0) {
> +			ret = h264_enc_alloc_work_buf(inst, enc_prm);
> +			if (ret)
> +				break;
> +			inst->work_buf_allocated = 1;
> +		}
> +		break;
> +
> +	case VENC_SET_PARAM_FORCE_INTRA:
> +		ret = h264_enc_vpu_set_param(inst, type, 0);
> +		break;
> +
> +	case VENC_SET_PARAM_ADJUST_BITRATE:
> +		enc_prm = in;
> +		ret = h264_enc_vpu_set_param(inst, type, &enc_prm->bitrate);
> +		break;
> +
> +	case VENC_SET_PARAM_ADJUST_FRAMERATE:
> +		enc_prm = in;
> +		ret = h264_enc_vpu_set_param(inst, type, &enc_prm->frm_rate);
> +		break;
> +
> +	case VENC_SET_PARAM_I_FRAME_INTERVAL:
> +		ret = h264_enc_vpu_set_param(inst, type, in);
> +		break;
> +
> +	case VENC_SET_PARAM_SKIP_FRAME:
> +		ret = h264_enc_vpu_set_param(inst, type, 0);
> +		break;
> +
> +	case VENC_SET_PARAM_PREPEND_HEADER:
> +		inst->prepend_hdr = 1;
> +		mtk_vcodec_debug(inst, "set prepend header mode");
> +		break;
> +
> +	default:
> +		mtk_vcodec_err(inst, "type %d not supported", type);
> +		ret = -EINVAL;
> +		break;
> +	}
> +
> +	mtk_vcodec_debug_leave(inst);
> +
> +	return ret;
> +}
> +
> +static int h264_enc_deinit(unsigned long handle)
> +{
> +	int ret = 0;
> +	struct venc_h264_inst *inst = (struct venc_h264_inst *)handle;
> +
> +	mtk_vcodec_debug_enter(inst);
> +
> +	ret = h264_enc_vpu_deinit(inst);
> +
> +	if (inst->work_buf_allocated)
> +		h264_enc_free_work_buf(inst);
> +
> +	mtk_vcodec_debug_leave(inst);
> +	kfree(inst);
> +
> +	return ret;
> +}
> +
> +static struct venc_common_if venc_h264_if = {
> +	h264_enc_init,
> +	h264_enc_encode,
> +	h264_enc_set_param,
> +	h264_enc_deinit,
> +};
> +
> +struct venc_common_if *get_h264_enc_comm_if(void)
> +{
> +	return &venc_h264_if;
> +}
> diff --git a/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.h b/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.h
> new file mode 100644
> index 0000000..9ac317a
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.h
> @@ -0,0 +1,165 @@
> +/*
> + * Copyright (c) 2015 MediaTek Inc.
> + * Author: Jungchang Tsao <jungchang.tsao@mediatek.com>
> + *         Daniel Hsiao <daniel.hsiao@mediatek.com>
> + *         PoChun Lin <pochun.lin@mediatek.com>
> + *
> + * This program is free software; you can redistribute it and/or
> + * modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + */
> +
> +#ifndef _VENC_H264_IF_H_
> +#define _VENC_H264_IF_H_
> +
> +#include "venc_drv_base.h"
> +
> +/**
> + * enum venc_h264_vpu_work_buf - h264 encoder buffer index
> + */
> +enum venc_h264_vpu_work_buf {
> +	VENC_H264_VPU_WORK_BUF_RC_INFO,
> +	VENC_H264_VPU_WORK_BUF_RC_CODE,
> +	VENC_H264_VPU_WORK_BUF_REC_LUMA,
> +	VENC_H264_VPU_WORK_BUF_REC_CHROMA,
> +	VENC_H264_VPU_WORK_BUF_REF_LUMA,
> +	VENC_H264_VPU_WORK_BUF_REF_CHROMA,
> +	VENC_H264_VPU_WORK_BUF_MV_INFO_1,
> +	VENC_H264_VPU_WORK_BUF_MV_INFO_2,
> +	VENC_H264_VPU_WORK_BUF_SKIP_FRAME,
> +	VENC_H264_VPU_WORK_BUF_SRC_LUMA,
> +	VENC_H264_VPU_WORK_BUF_SRC_CHROMA,
> +	VENC_H264_VPU_WORK_BUF_SRC_CHROMA_CB,
> +	VENC_H264_VPU_WORK_BUF_SRC_CHROMA_CR,
> +	VENC_H264_VPU_WORK_BUF_MAX,
> +};
> +
> +/**
> + * enum venc_h264_bs_mode - for bs_mode argument in h264_enc_vpu_encode
> + */
> +enum venc_h264_bs_mode {
> +	H264_BS_MODE_SPS,
> +	H264_BS_MODE_PPS,
> +	H264_BS_MODE_FRAME,
> +};
> +
> +/*
> + * struct venc_h264_vpu_config - Structure for h264 encoder configuration
> + * @input_fourcc: input fourcc
> + * @bitrate: target bitrate (in bps)
> + * @pic_w: picture width
> + * @pic_h: picture height
> + * @buf_w: buffer width
> + * @buf_h: buffer height
> + * @intra_period: intra frame period
> + * @framerate: frame rate
> + * @profile: as specified in standard
> + * @level: as specified in standard
> + * @wfd: WFD mode 1:on, 0:off
> + */
> +struct venc_h264_vpu_config {
> +	u32 input_fourcc;
> +	u32 bitrate;
> +	u32 pic_w;
> +	u32 pic_h;
> +	u32 buf_w;
> +	u32 buf_h;
> +	u32 intra_period;
> +	u32 framerate;
> +	u32 profile;
> +	u32 level;
> +	u32 wfd;
> +};
> +
> +/*
> + * struct venc_h264_vpu_buf - Structure for buffer information
> + * @align: buffer alignment (in bytes)
> + * @iova: IO virtual address
> + * @vpua: VPU side memory addr which is used by RC_CODE
> + * @size: buffer size (in bytes)
> + */
> +struct venc_h264_vpu_buf {
> +	u32 align;
> +	u32 iova;
> +	u32 vpua;
> +	u32 size;
> +};
> +
> +/*
> + * struct venc_h264_vpu_drv - Structure for VPU driver control and info share
> + * This structure is allocated in VPU side and shared to AP side.
> + * @config: h264 encoder configuration
> + * @work_bufs: working buffer information in VPU side
> + * The work_bufs here is for storing the 'size' info shared to AP side.
> + * The similar item in struct venc_h264_inst is for memory allocation
> + * in AP side. The AP driver will copy the 'size' from here to the one in
> + * struct mtk_vcodec_mem, then invoke mtk_vcodec_mem_alloc to allocate
> + * the buffer. After that, bypass the 'dma_addr' to the 'iova' field here for
> + * register setting in VPU side.
> + */
> +struct venc_h264_vpu_drv {
> +	struct venc_h264_vpu_config config;
> +	struct venc_h264_vpu_buf work_bufs[VENC_H264_VPU_WORK_BUF_MAX];
> +};
> +
> +/*
> + * struct venc_h264_vpu_inst - h264 encoder VPU driver instance
> + * @wq_hd: wait queue used for vpu cmd trigger then wait vpu interrupt done
> + * @signaled: flag used for checking vpu interrupt done
> + * @failure: flag to show vpu cmd succeeds or not
> + * @state: enum venc_ipi_msg_enc_state
> + * @bs_size: bitstream size for skip frame case usage
> + * @wait_int: flag to wait interrupt done (0: for skip frame case, 1: normal
> + *	      case)
> + * @id: VPU instance id
> + * @drv: driver structure allocated by VPU side and shared to AP side for
> + *	 control and info share
> + */
> +struct venc_h264_vpu_inst {
> +	wait_queue_head_t wq_hd;
> +	int signaled;
> +	int failure;
> +	int state;
> +	int bs_size;
> +	int wait_int;
> +	unsigned int id;
> +	struct venc_h264_vpu_drv *drv;
> +};
> +
> +/*
> + * struct venc_h264_inst - h264 encoder AP driver instance
> + * @hw_base: h264 encoder hardware register base
> + * @work_bufs: working buffer
> + * @pps_buf: buffer to store the pps bitstream
> + * @work_buf_allocated: working buffer allocated flag
> + * @frm_cnt: encoded frame count
> + * @prepend_hdr: when the v4l2 layer send VENC_SET_PARAM_PREPEND_HEADER cmd
> + *  through h264_enc_set_param interface, it will set this flag and prepend the
> + *  sps/pps in h264_enc_encode function.
> + * @is_key_frm: key frame flag
> + * @vpu_inst: VPU instance to exchange information between AP and VPU
> + * @ctx: context for v4l2 layer integration
> + * @dev: device for v4l2 layer integration
> + */
> +struct venc_h264_inst {
> +	void __iomem *hw_base;
> +	struct mtk_vcodec_mem work_bufs[VENC_H264_VPU_WORK_BUF_MAX];
> +	struct mtk_vcodec_mem pps_buf;
> +	bool work_buf_allocated;
> +	unsigned int frm_cnt;
> +	unsigned int prepend_hdr;
> +	unsigned int is_key_frm;
> +	struct venc_h264_vpu_inst vpu_inst;
> +	void *ctx;
> +	struct platform_device *dev;
> +};
> +
> +struct venc_common_if *get_h264_enc_comm_if(void);
> +
> +#endif
> diff --git a/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.c b/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.c
> new file mode 100644
> index 0000000..450be45
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.c
> @@ -0,0 +1,309 @@
> +/*
> + * Copyright (c) 2015 MediaTek Inc.
> + * Author: Jungchang Tsao <jungchang.tsao@mediatek.com>
> + *         Daniel Hsiao <daniel.hsiao@mediatek.com>
> + *         PoChun Lin <pochun.lin@mediatek.com>
> + *
> + * This program is free software; you can redistribute it and/or
> + * modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + */
> +
> +#include "mtk_vpu.h"
> +
> +#include "venc_h264_if.h"
> +#include "venc_h264_vpu.h"
> +#include "venc_ipi_msg.h"
> +
> +static unsigned int h264_get_profile(unsigned int profile)
> +{
> +	/* (Baseline=66, Main=77, High=100) */
> +	switch (profile) {
> +	case V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE:
> +		return 66;
> +	case V4L2_MPEG_VIDEO_H264_PROFILE_MAIN:
> +		return 77;
> +	case V4L2_MPEG_VIDEO_H264_PROFILE_HIGH:
> +		return 100;
> +	default:
> +		return 100;
> +	}
> +}
> +
> +static unsigned int h264_get_level(unsigned int level)
> +{
> +	/* (UpTo4.1(HighProfile)) */
> +	switch (level) {
> +	case V4L2_MPEG_VIDEO_H264_LEVEL_1_0:
> +		return 10;
> +	case V4L2_MPEG_VIDEO_H264_LEVEL_1_1:
> +		return 11;
> +	case V4L2_MPEG_VIDEO_H264_LEVEL_1_2:
> +		return 12;
> +	case V4L2_MPEG_VIDEO_H264_LEVEL_1_3:
> +		return 13;
> +	case V4L2_MPEG_VIDEO_H264_LEVEL_2_0:
> +		return 20;
> +	case V4L2_MPEG_VIDEO_H264_LEVEL_2_1:
> +		return 21;
> +	case V4L2_MPEG_VIDEO_H264_LEVEL_2_2:
> +		return 22;
> +	case V4L2_MPEG_VIDEO_H264_LEVEL_3_0:
> +		return 30;
> +	case V4L2_MPEG_VIDEO_H264_LEVEL_3_1:
> +		return 31;
> +	case V4L2_MPEG_VIDEO_H264_LEVEL_3_2:
> +		return 32;
> +	case V4L2_MPEG_VIDEO_H264_LEVEL_4_0:
> +		return 40;
> +	case V4L2_MPEG_VIDEO_H264_LEVEL_4_1:
> +		return 41;
> +	default:
> +		return 31;
> +	}
> +}
> +
> +static void handle_h264_enc_init_msg(struct venc_h264_inst *inst, void *data)
> +{
> +	struct venc_vpu_ipi_msg_init *msg = data;
> +
> +	inst->vpu_inst.id = msg->inst_id;
> +	inst->vpu_inst.drv = (struct venc_h264_vpu_drv *)vpu_mapping_dm_addr(
> +		inst->dev, msg->inst_id);
> +}
> +
> +static void handle_h264_enc_encode_msg(struct venc_h264_inst *inst, void *data)
> +{
> +	struct venc_vpu_ipi_msg_enc *msg = data;
> +
> +	inst->vpu_inst.state = msg->state;
> +	inst->vpu_inst.bs_size = msg->bs_size;
> +	inst->is_key_frm = msg->key_frame;
> +}
> +
> +static void h264_enc_vpu_ipi_handler(void *data, unsigned int len, void *priv)
> +{
> +	struct venc_vpu_ipi_msg_common *msg = data;
> +	struct venc_h264_inst *inst = (struct venc_h264_inst *)msg->venc_inst;
> +
> +	mtk_vcodec_debug(inst, "msg_id %x inst %p status %d",
> +			 msg->msg_id, inst, msg->status);
> +
> +	switch (msg->msg_id) {
> +	case VPU_IPIMSG_H264_ENC_INIT_DONE:
> +		handle_h264_enc_init_msg(inst, data);
> +		break;
> +	case VPU_IPIMSG_H264_ENC_SET_PARAM_DONE:
> +		break;
> +	case VPU_IPIMSG_H264_ENC_ENCODE_DONE:
> +		handle_h264_enc_encode_msg(inst, data);
> +		break;
> +	case VPU_IPIMSG_H264_ENC_DEINIT_DONE:
> +		break;
> +	default:
> +		mtk_vcodec_err(inst, "unknown msg id %x", msg->msg_id);
> +		break;
> +	}
> +
> +	inst->vpu_inst.signaled = 1;
> +	inst->vpu_inst.failure = (msg->status != VENC_IPI_MSG_STATUS_OK);
> +
> +	mtk_vcodec_debug_leave(inst);
> +}
> +
> +static int h264_enc_vpu_send_msg(struct venc_h264_inst *inst, void *msg,
> +				 int len)
> +{
> +	int status;
> +
> +	mtk_vcodec_debug_enter(inst);
> +	status = vpu_ipi_send(inst->dev, IPI_VENC_H264, msg, len);
> +	if (status) {
> +		mtk_vcodec_err(inst, "vpu_ipi_send msg %x len %d fail %d",
> +			       *(unsigned int *)msg, len, status);
> +		return -EINVAL;
> +	}
> +	mtk_vcodec_debug_leave(inst);
> +
> +	return 0;
> +}
> +
> +int h264_enc_vpu_init(struct venc_h264_inst *inst)
> +{
> +	int status;
> +	struct venc_ap_ipi_msg_init out;
> +
> +	mtk_vcodec_debug_enter(inst);
> +
> +	init_waitqueue_head(&inst->vpu_inst.wq_hd);
> +	inst->vpu_inst.signaled = 0;
> +	inst->vpu_inst.failure = 0;
> +
> +	status = vpu_ipi_register(inst->dev, IPI_VENC_H264,
> +				  h264_enc_vpu_ipi_handler,
> +				  "h264_enc", NULL);
> +	if (status) {
> +		mtk_vcodec_err(inst, "vpu_ipi_register fail %d", status);
> +		return -EINVAL;
> +	}
> +
> +	out.msg_id = AP_IPIMSG_H264_ENC_INIT;
> +	out.venc_inst = (unsigned long)inst;
> +	if (h264_enc_vpu_send_msg(inst, &out, sizeof(out)) ||
> +	    inst->vpu_inst.failure) {
> +		mtk_vcodec_err(inst, "AP_IPIMSG_H264_ENC_INIT failed");
> +		return -EINVAL;
> +	}
> +
> +	mtk_vcodec_debug_leave(inst);
> +
> +	return 0;
> +}
> +
> +int h264_enc_vpu_set_param(struct venc_h264_inst *inst, unsigned int id,
> +			   void *param)
> +{
> +	struct venc_ap_ipi_msg_set_param out;
> +
> +	mtk_vcodec_debug(inst, "id %d ->", id);
> +
> +	out.msg_id = AP_IPIMSG_H264_ENC_SET_PARAM;
> +	out.inst_id = inst->vpu_inst.id;
> +	out.param_id = id;
> +	switch (id) {
> +	case VENC_SET_PARAM_ENC: {
> +		struct venc_enc_prm *enc_param = (struct venc_enc_prm *)param;
> +
> +		inst->vpu_inst.drv->config.input_fourcc =
> +			enc_param->input_fourcc;
> +		inst->vpu_inst.drv->config.bitrate = enc_param->bitrate;
> +		inst->vpu_inst.drv->config.pic_w = enc_param->width;
> +		inst->vpu_inst.drv->config.pic_h = enc_param->height;
> +		inst->vpu_inst.drv->config.buf_w = enc_param->buf_width;
> +		inst->vpu_inst.drv->config.buf_h = enc_param->buf_height;
> +		inst->vpu_inst.drv->config.intra_period =
> +			enc_param->intra_period;
> +		inst->vpu_inst.drv->config.framerate = enc_param->frm_rate;
> +		inst->vpu_inst.drv->config.profile =
> +			h264_get_profile(enc_param->h264_profile);
> +		inst->vpu_inst.drv->config.level =
> +			h264_get_level(enc_param->h264_level);
> +		inst->vpu_inst.drv->config.wfd = 0;
> +		out.data_item = 0;
> +		break;
> +	}
> +	case VENC_SET_PARAM_FORCE_INTRA:
> +		out.data_item = 0;
> +		break;
> +	case VENC_SET_PARAM_ADJUST_BITRATE:
> +		out.data_item = 1;
> +		out.data[0] = *(unsigned int *)param;
> +		break;
> +	case VENC_SET_PARAM_ADJUST_FRAMERATE:
> +		out.data_item = 1;
> +		out.data[0] = *(unsigned int *)param;
> +		break;
> +	case VENC_SET_PARAM_I_FRAME_INTERVAL:
> +		out.data_item = 1;
> +		out.data[0] = *(unsigned int *)param;
> +		break;
> +	case VENC_SET_PARAM_SKIP_FRAME:
> +		out.data_item = 0;
> +		break;
> +	}
> +	if (h264_enc_vpu_send_msg(inst, &out, sizeof(out)) ||
> +	    inst->vpu_inst.failure) {
> +		mtk_vcodec_err(inst,
> +			       "AP_IPIMSG_H264_ENC_SET_PARAM %d fail", id);
> +		return -EINVAL;
> +	}
> +
> +	mtk_vcodec_debug(inst, "id %d <-", id);
> +
> +	return 0;
> +}
> +
> +int h264_enc_vpu_encode(struct venc_h264_inst *inst, unsigned int bs_mode,
> +			struct venc_frm_buf *frm_buf,
> +			struct mtk_vcodec_mem *bs_buf,
> +			unsigned int *bs_size)
> +{
> +	struct venc_ap_ipi_msg_enc out;
> +
> +	mtk_vcodec_debug(inst, "bs_mode %d ->", bs_mode);
> +
> +	out.msg_id = AP_IPIMSG_H264_ENC_ENCODE;
> +	out.inst_id = inst->vpu_inst.id;
> +	out.bs_mode = bs_mode;
> +	if (frm_buf) {
> +		if ((frm_buf->fb_addr.dma_addr % 16 == 0) &&
> +		    (frm_buf->fb_addr1.dma_addr % 16 == 0) &&
> +		    (frm_buf->fb_addr2.dma_addr % 16 == 0)) {
> +			out.input_addr[0] = frm_buf->fb_addr.dma_addr;
> +			out.input_addr[1] = frm_buf->fb_addr1.dma_addr;
> +			out.input_addr[2] = frm_buf->fb_addr2.dma_addr;
> +		} else {
> +			mtk_vcodec_err(inst, "dma_addr not align to 16");
> +			return -EINVAL;
> +		}
> +	} else {
> +		out.input_addr[0] = 0;
> +		out.input_addr[1] = 0;
> +		out.input_addr[2] = 0;
> +	}
> +	if (bs_buf) {
> +		out.bs_addr = bs_buf->dma_addr;
> +		out.bs_size = bs_buf->size;
> +	} else {
> +		out.bs_addr = 0;
> +		out.bs_size = 0;
> +	}
> +	if (h264_enc_vpu_send_msg(inst, &out, sizeof(out)) ||
> +	    inst->vpu_inst.failure) {
> +		mtk_vcodec_err(inst, "AP_IPIMSG_H264_ENC_ENCODE %d fail",
> +			       bs_mode);
> +		return -EINVAL;
> +	}
> +
> +	mtk_vcodec_debug(inst, "state %d size %d key_frm %d",
> +			 inst->vpu_inst.state, inst->vpu_inst.bs_size,
> +			 inst->is_key_frm);
> +
> +	inst->vpu_inst.wait_int = 1;
> +	if (inst->vpu_inst.state == VEN_IPI_MSG_ENC_STATE_SKIP) {
> +		*bs_size = inst->vpu_inst.bs_size;
> +		memcpy(bs_buf->va,
> +		       inst->work_bufs[VENC_H264_VPU_WORK_BUF_SKIP_FRAME].va,
> +		       *bs_size);
> +		inst->vpu_inst.wait_int = 0;
> +	}
> +
> +	mtk_vcodec_debug(inst, "bs_mode %d <-", bs_mode);
> +
> +	return 0;
> +}
> +
> +int h264_enc_vpu_deinit(struct venc_h264_inst *inst)
> +{
> +	struct venc_ap_ipi_msg_deinit out;
> +
> +	mtk_vcodec_debug_enter(inst);
> +
> +	out.msg_id = AP_IPIMSG_H264_ENC_DEINIT;
> +	out.inst_id = inst->vpu_inst.id;
> +	if (h264_enc_vpu_send_msg(inst, &out, sizeof(out)) ||
> +	    inst->vpu_inst.failure) {
> +		mtk_vcodec_err(inst, "AP_IPIMSG_H264_ENC_DEINIT fail");
> +		return -EINVAL;
> +	}
> +
> +	mtk_vcodec_debug_leave(inst);
> +
> +	return 0;
> +}
> diff --git a/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.h b/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.h
> new file mode 100644
> index 0000000..deccc6f
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.h
> @@ -0,0 +1,30 @@
> +/*
> + * Copyright (c) 2015 MediaTek Inc.
> + * Author: Jungchang Tsao <jungchang.tsao@mediatek.com>
> + *         Daniel Hsiao <daniel.hsiao@mediatek.com>
> + *         PoChun Lin <pochun.lin@mediatek.com>
> + *
> + * This program is free software; you can redistribute it and/or
> + * modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + */
> +
> +#ifndef _VENC_H264_VPU_H_
> +#define _VENC_H264_VPU_H_
> +
> +int h264_enc_vpu_init(struct venc_h264_inst *inst);
> +int h264_enc_vpu_set_param(struct venc_h264_inst *inst, unsigned int id,
> +			   void *param);
> +int h264_enc_vpu_encode(struct venc_h264_inst *inst, unsigned int bs_mode,
> +			struct venc_frm_buf *frm_buf,
> +			struct mtk_vcodec_mem *bs_buf,
> +			unsigned int *bs_size);
> +int h264_enc_vpu_deinit(struct venc_h264_inst *inst);
> +
> +#endif
> diff --git a/drivers/media/platform/mtk-vcodec/venc_drv_if.c b/drivers/media/platform/mtk-vcodec/venc_drv_if.c
> index d293f2c..28ef4a7 100644
> --- a/drivers/media/platform/mtk-vcodec/venc_drv_if.c
> +++ b/drivers/media/platform/mtk-vcodec/venc_drv_if.c
> @@ -25,6 +25,7 @@
>  
>  #include "venc_drv_base.h"
>  #include "vp8_enc/venc_vp8_if.h"
> +#include "h264_enc/venc_h264_if.h"
>  
>  int venc_if_create(struct mtk_vcodec_ctx *ctx, unsigned int fourcc)
>  {
> @@ -38,6 +39,8 @@ int venc_if_create(struct mtk_vcodec_ctx *ctx, unsigned int fourcc)
>                  ctx->enc_if = get_vp8_enc_comm_if();
>                  break;
>  	case V4L2_PIX_FMT_H264:
> +	        ctx->enc_if = get_h264_enc_comm_if();
> +	        break;
>  	default:
>  		return -EINVAL;
>  	}
> 

Regards,

	Hans

^ permalink raw reply	[flat|nested] 102+ messages in thread

* [PATCH v4 7/8] [media] vcodec: mediatek: Add Mediatek H264 Video Encoder Driver
@ 2016-02-15 11:33                 ` Hans Verkuil
  0 siblings, 0 replies; 102+ messages in thread
From: Hans Verkuil @ 2016-02-15 11:33 UTC (permalink / raw)
  To: linux-arm-kernel

On 02/04/2016 12:35 PM, Tiffany Lin wrote:
> Add h264 encoder driver for MT8173
> 
> Signed-off-by: Daniel Hsiao <daniel.hsiao@mediatek.com>
> Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
> ---
>  drivers/media/platform/mtk-vcodec/Makefile         |    3 +-
>  .../media/platform/mtk-vcodec/h264_enc/Makefile    |    6 +
>  .../platform/mtk-vcodec/h264_enc/venc_h264_if.c    |  540 ++++++++++++++++++++
>  .../platform/mtk-vcodec/h264_enc/venc_h264_if.h    |  165 ++++++
>  .../platform/mtk-vcodec/h264_enc/venc_h264_vpu.c   |  309 +++++++++++
>  .../platform/mtk-vcodec/h264_enc/venc_h264_vpu.h   |   30 ++
>  drivers/media/platform/mtk-vcodec/venc_drv_if.c    |    3 +
>  7 files changed, 1055 insertions(+), 1 deletion(-)
>  create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/Makefile
>  create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.c
>  create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.h
>  create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.c
>  create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.h
> 
> diff --git a/drivers/media/platform/mtk-vcodec/Makefile b/drivers/media/platform/mtk-vcodec/Makefile
> index f4ef502..f47dfc7 100644
> --- a/drivers/media/platform/mtk-vcodec/Makefile
> +++ b/drivers/media/platform/mtk-vcodec/Makefile
> @@ -5,6 +5,7 @@ obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += mtk_vcodec_intr.o \
>  				       mtk_vcodec_enc_pm.o \
>  				       venc_drv_if.o
>  
> -obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += vp8_enc/
> +
> +obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += vp8_enc/ h264_enc/
>  
>  ccflags-y += -I$(srctree)/drivers/media/platform/mtk-vpu
> diff --git a/drivers/media/platform/mtk-vcodec/h264_enc/Makefile b/drivers/media/platform/mtk-vcodec/h264_enc/Makefile
> new file mode 100644
> index 0000000..765b45f
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vcodec/h264_enc/Makefile
> @@ -0,0 +1,6 @@
> +obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += venc_h264_if.o venc_h264_vpu.o
> +
> +ccflags-y += \
> +    -I$(srctree)/drivers/media/platform/mtk-vcodec/ \
> +    -I$(srctree)/drivers/media/platform/mtk-vcodec/h264_enc \
> +    -I$(srctree)/drivers/media/platform/mtk-vpu
> diff --git a/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.c b/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.c
> new file mode 100644
> index 0000000..c35fb26
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.c
> @@ -0,0 +1,540 @@
> +/*
> + * Copyright (c) 2015 MediaTek Inc.
> + * Author: Jungchang Tsao <jungchang.tsao@mediatek.com>
> + *         Daniel Hsiao <daniel.hsiao@mediatek.com>
> + *         PoChun Lin <pochun.lin@mediatek.com>
> + *
> + * This program is free software; you can redistribute it and/or
> + * modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + */
> +
> +#include <linux/interrupt.h>
> +#include <linux/kernel.h>
> +#include <linux/slab.h>
> +
> +#include "mtk_vcodec_drv.h"
> +#include "mtk_vcodec_util.h"
> +#include "mtk_vcodec_intr.h"
> +#include "mtk_vcodec_enc.h"
> +#include "mtk_vcodec_pm.h"
> +#include "mtk_vpu.h"
> +
> +#include "venc_h264_if.h"
> +#include "venc_h264_vpu.h"
> +
> +#define VENC_PIC_BITSTREAM_BYTE_CNT 0x0098
> +
> +enum venc_h264_irq_status {
> +	H264_IRQ_STATUS_ENC_SPS_INT = (1 << 0),
> +	H264_IRQ_STATUS_ENC_PPS_INT = (1 << 1),
> +	H264_IRQ_STATUS_ENC_FRM_INT = (1 << 2),
> +};
> +
> +static inline void h264_write_reg(struct venc_h264_inst *inst, u32 addr,
> +				  u32 val)
> +{
> +	writel(val, inst->hw_base + addr);
> +}
> +
> +static inline u32 h264_read_reg(struct venc_h264_inst *inst, u32 addr)
> +{
> +	return readl(inst->hw_base + addr);
> +}
> +
> +static void h264_enc_free_work_buf(struct venc_h264_inst *inst)
> +{
> +	int i;
> +
> +	mtk_vcodec_debug_enter(inst);
> +
> +	/* Except the SKIP_FRAME buffers,
> +	 * other buffers need to be freed by AP.
> +	 */
> +	for (i = 0; i < VENC_H264_VPU_WORK_BUF_MAX; i++) {
> +		if (i != VENC_H264_VPU_WORK_BUF_SKIP_FRAME)
> +			if (inst->work_bufs[i].va != NULL)
> +				mtk_vcodec_mem_free(inst->ctx,
> +						    &inst->work_bufs[i]);
> +	}
> +
> +	if (inst->pps_buf.va != NULL)
> +		mtk_vcodec_mem_free(inst->ctx, &inst->pps_buf);
> +
> +	mtk_vcodec_debug_leave(inst);
> +}
> +
> +static int h264_enc_alloc_work_buf(struct venc_h264_inst *inst, void *param)
> +{
> +	int i;
> +	int ret = 0;
> +	struct venc_h264_vpu_buf *wb = inst->vpu_inst.drv->work_bufs;
> +	struct venc_enc_prm *enc_param = param;
> +
> +	mtk_vcodec_debug_enter(inst);
> +
> +	for (i = 0; i < VENC_H264_VPU_WORK_BUF_MAX; i++) {
> +		/*
> +		 * This 'wb' structure is set by VPU side and shared to AP for
> +		 * buffer allocation and IO virtual addr mapping. For most of
> +		 * the buffers, AP will allocate the buffer according to 'size'
> +		 * field and store the IO virtual addr in 'iova' field. There
> +		 * are two exceptions:
> +		 * (1) RC_CODE buffer, it's pre-allocated in the VPU side, and
> +		 * save the VPU addr in the 'vpua' field. The AP will translate
> +		 * the VPU addr to the corresponding IO virtual addr and store
> +		 * in 'iova' field for reg setting in VPU side.
> +		 * (2) SKIP_FRAME buffer, it's pre-allocated in the VPU side,
> +		 * and save the VPU addr in the 'vpua' field. The AP will
> +		 * translate the VPU addr to the corresponding AP side virtual
> +		 * address and do some memcpy access to move to bitstream buffer
> +		 * assigned by v4l2 layer.
> +		 */
> +		if (i == VENC_H264_VPU_WORK_BUF_RC_CODE) {
> +			void *tmp_va;
> +
> +			tmp_va = vpu_mapping_dm_addr(inst->dev, wb[i].vpua);
> +			inst->work_bufs[i].size = wb[i].size;
> +			if (mtk_vcodec_mem_alloc(inst->ctx,
> +						 &inst->work_bufs[i])) {
> +				mtk_vcodec_err(inst,
> +					       "cannot allocate buf %d", i);
> +				ret = -ENOMEM;
> +				goto err_alloc;
> +			}
> +			memcpy(inst->work_bufs[i].va, tmp_va, wb[i].size);
> +			wb[i].iova = inst->work_bufs[i].dma_addr;
> +		} else if (i == VENC_H264_VPU_WORK_BUF_SKIP_FRAME) {
> +			inst->work_bufs[i].size = wb[i].size;
> +			inst->work_bufs[i].va = vpu_mapping_dm_addr(
> +				inst->dev, wb[i].vpua);
> +			inst->work_bufs[i].dma_addr = 0;
> +			wb[i].iova = inst->work_bufs[i].dma_addr;
> +		} else if (i == VENC_H264_VPU_WORK_BUF_SRC_LUMA ||
> +			   i == VENC_H264_VPU_WORK_BUF_SRC_CHROMA ||
> +			   i == VENC_H264_VPU_WORK_BUF_SRC_CHROMA_CB ||
> +			   i == VENC_H264_VPU_WORK_BUF_SRC_CHROMA_CR) {
> +			inst->work_bufs[i].size = wb[i].size;
> +			inst->work_bufs[i].dma_addr = 0;
> +			inst->work_bufs[i].va = NULL;
> +			wb[i].iova = inst->work_bufs[i].dma_addr;
> +		} else {
> +			inst->work_bufs[i].size = wb[i].size;
> +			if (mtk_vcodec_mem_alloc(inst->ctx,
> +						 &inst->work_bufs[i])) {
> +				mtk_vcodec_err(inst,
> +					       "cannot allocate buf %d", i);
> +				ret = -ENOMEM;
> +				goto err_alloc;
> +			}
> +			wb[i].iova = inst->work_bufs[i].dma_addr;
> +		}
> +		mtk_vcodec_debug(inst, "buf[%d] va=0x%p iova=0x%p size=0x%lx",
> +				 i, inst->work_bufs[i].va,
> +				 (void *)inst->work_bufs[i].dma_addr,
> +				 inst->work_bufs[i].size);
> +	}
> +
> +	if (enc_param->input_fourcc == VENC_YUV_FORMAT_NV12 ||
> +	    enc_param->input_fourcc == VENC_YUV_FORMAT_NV21) {
> +		enc_param->sizeimage[0] =
> +			inst->work_bufs[VENC_H264_VPU_WORK_BUF_SRC_LUMA].size;
> +		enc_param->sizeimage[1] =
> +			inst->work_bufs[VENC_H264_VPU_WORK_BUF_SRC_CHROMA].size;
> +		enc_param->sizeimage[2] = 0;
> +	} else {
> +		enc_param->sizeimage[0] =
> +			inst->work_bufs[VENC_H264_VPU_WORK_BUF_SRC_LUMA].size;
> +		enc_param->sizeimage[1] =
> +			inst->work_bufs[VENC_H264_VPU_WORK_BUF_SRC_CHROMA_CB].size;
> +		enc_param->sizeimage[2] =
> +			inst->work_bufs[VENC_H264_VPU_WORK_BUF_SRC_CHROMA_CR].size;
> +	}
> +
> +	/* the pps_buf is used by AP side only */
> +	inst->pps_buf.size = 128;
> +	if (mtk_vcodec_mem_alloc(inst->ctx,
> +				 &inst->pps_buf)) {
> +		mtk_vcodec_err(inst, "cannot allocate pps_buf");
> +		ret = -ENOMEM;
> +		goto err_alloc;
> +	}
> +	mtk_vcodec_debug_leave(inst);
> +
> +	return ret;
> +
> +err_alloc:
> +	h264_enc_free_work_buf(inst);
> +
> +	return ret;
> +}
> +
> +static unsigned int h264_enc_wait_venc_done(struct venc_h264_inst *inst)
> +{
> +	unsigned int irq_status = 0;
> +	struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)inst->ctx;
> +
> +	mtk_vcodec_wait_for_done_ctx(ctx, MTK_INST_IRQ_RECEIVED,
> +				     WAIT_INTR_TIMEOUT, true);
> +	irq_status = ctx->irq_status;
> +	mtk_vcodec_debug(inst, "irq_status %x <-", irq_status);
> +
> +	return irq_status;
> +}
> +
> +static int h264_encode_sps(struct venc_h264_inst *inst,
> +			   struct mtk_vcodec_mem *bs_buf,
> +			   unsigned int *bs_size)
> +{
> +	int ret = 0;
> +	unsigned int irq_status;
> +
> +	mtk_vcodec_debug_enter(inst);
> +
> +	ret = h264_enc_vpu_encode(inst, H264_BS_MODE_SPS, NULL,
> +				  bs_buf, bs_size);
> +	if (ret)
> +		return ret;
> +
> +	irq_status = h264_enc_wait_venc_done(inst);
> +	if (irq_status != H264_IRQ_STATUS_ENC_SPS_INT) {
> +		mtk_vcodec_err(inst, "expect irq status %d",
> +			       H264_IRQ_STATUS_ENC_SPS_INT);
> +		return -EINVAL;
> +	}
> +
> +	*bs_size = h264_read_reg(inst, VENC_PIC_BITSTREAM_BYTE_CNT);
> +	mtk_vcodec_debug(inst, "bs size %d <-", *bs_size);
> +
> +	return ret;
> +}
> +
> +static int h264_encode_pps(struct venc_h264_inst *inst,
> +			   struct mtk_vcodec_mem *bs_buf,
> +			   unsigned int *bs_size)
> +{
> +	int ret = 0;
> +	unsigned int irq_status;
> +
> +	mtk_vcodec_debug_enter(inst);
> +
> +	ret = h264_enc_vpu_encode(inst, H264_BS_MODE_PPS, NULL,
> +				  bs_buf, bs_size);
> +	if (ret)
> +		return ret;
> +
> +	irq_status = h264_enc_wait_venc_done(inst);
> +	if (irq_status != H264_IRQ_STATUS_ENC_PPS_INT) {
> +		mtk_vcodec_err(inst, "expect irq status %d",
> +			       H264_IRQ_STATUS_ENC_PPS_INT);
> +		return -EINVAL;
> +	}
> +
> +	*bs_size = h264_read_reg(inst, VENC_PIC_BITSTREAM_BYTE_CNT);
> +	mtk_vcodec_debug(inst, "bs size %d <-", *bs_size);
> +
> +	return ret;
> +}
> +
> +static int h264_encode_frame(struct venc_h264_inst *inst,
> +			     struct venc_frm_buf *frm_buf,
> +			     struct mtk_vcodec_mem *bs_buf,
> +			     unsigned int *bs_size)
> +{
> +	int ret = 0;
> +	unsigned int irq_status;
> +
> +	mtk_vcodec_debug_enter(inst);
> +
> +	ret = h264_enc_vpu_encode(inst, H264_BS_MODE_FRAME, frm_buf,
> +				  bs_buf, bs_size);
> +	if (ret)
> +		return ret;
> +
> +	/*
> +	 * skip frame case: The skip frame buffer is composed by vpu side only,
> +	 * it does not trigger the hw, so skip the wait interrupt operation.
> +	 */
> +	if (!inst->vpu_inst.wait_int) {
> +		++inst->frm_cnt;
> +		return ret;
> +	}
> +
> +	irq_status = h264_enc_wait_venc_done(inst);
> +	if (irq_status != H264_IRQ_STATUS_ENC_FRM_INT) {
> +		mtk_vcodec_err(inst, "irq_status=%d failed", irq_status);
> +		return -EINVAL;
> +	}
> +
> +	*bs_size = h264_read_reg(inst,
> +				 VENC_PIC_BITSTREAM_BYTE_CNT);
> +	++inst->frm_cnt;
> +	mtk_vcodec_debug(inst, "frm %d bs size %d key_frm %d <-",
> +			 inst->frm_cnt,
> +			 *bs_size, inst->is_key_frm);
> +
> +	return ret;
> +}
> +
> +static void h264_encode_filler(struct venc_h264_inst *inst, void *buf,
> +			       int size)
> +{
> +	unsigned char *p = buf;
> +
> +	*p++ = 0x0;
> +	*p++ = 0x0;
> +	*p++ = 0x0;
> +	*p++ = 0x1;
> +	*p++ = 0xc;
> +	size -= 5;
> +	while (size) {
> +		*p++ = 0xff;
> +		size -= 1;
> +	}
> +}
> +
> +static int h264_enc_init(struct mtk_vcodec_ctx *ctx, unsigned long *handle)
> +{
> +	int ret = 0;
> +	struct venc_h264_inst *inst;
> +
> +	inst = kzalloc(sizeof(*inst), GFP_KERNEL);
> +	if (!inst)
> +		return -ENOMEM;
> +
> +	inst->ctx = ctx;
> +	inst->dev = mtk_vcodec_get_plat_dev(ctx);
> +	inst->hw_base = mtk_vcodec_get_reg_addr(inst->ctx, VENC_SYS);
> +
> +	mtk_vcodec_debug_enter(inst);
> +	ret = h264_enc_vpu_init(inst);
> +	if (ret)
> +		kfree(inst);
> +	else
> +		(*handle) = (unsigned long)inst;
> +
> +	mtk_vcodec_debug_leave(inst);
> +
> +	return ret;
> +}
> +
> +static int h264_enc_encode(unsigned long handle,
> +			   enum venc_start_opt opt,
> +			   struct venc_frm_buf *frm_buf,
> +			   struct mtk_vcodec_mem *bs_buf,
> +			   struct venc_done_result *result)
> +{
> +	int ret = 0;
> +	struct venc_h264_inst *inst = (struct venc_h264_inst *)handle;
> +	struct mtk_vcodec_ctx *ctx = inst->ctx;
> +
> +	mtk_vcodec_debug(inst, "opt %d ->", opt);
> +
> +	enable_irq(ctx->dev->enc_irq);
> +
> +	switch (opt) {
> +	case VENC_START_OPT_ENCODE_SEQUENCE_HEADER: {
> +		unsigned int bs_size_sps;
> +		unsigned int bs_size_pps;
> +
> +		memset(bs_buf->va, 0x38, 20);
> +		ret = h264_encode_sps(inst, bs_buf, &bs_size_sps);
> +		if (ret)
> +			goto encode_err;
> +
> +		memset(inst->pps_buf.va, 0x49, 20);
> +		ret = h264_encode_pps(inst, &inst->pps_buf, &bs_size_pps);
> +		if (ret)
> +			goto encode_err;
> +
> +		memcpy(bs_buf->va + bs_size_sps,
> +		       inst->pps_buf.va,
> +		       bs_size_pps);
> +		result->bs_size = bs_size_sps + bs_size_pps;
> +		result->is_key_frm = false;
> +	}
> +	break;

Wrong indentation for break, put it before the '}'.

> +
> +	case VENC_START_OPT_ENCODE_FRAME:
> +		if (inst->prepend_hdr) {

Invert the condition to make it more readable:

		if (!inst->prepend_hdr) {
			ret = h264_encode_frame(inst, frm_buf, bs_buf,
						&result->bs_size);
			if (ret)
				goto encode_err;
			result->is_key_frm = inst->is_key_frm;
			break;
		}

Now you can shift the remainder one tab to the left and join lines that
are now broken up (probably because they exceed 80 chars).

> +			int hdr_sz;
> +			int hdr_sz_ext;
> +			int bs_alignment = 128;
> +			int filler_sz = 0;
> +			struct mtk_vcodec_mem tmp_bs_buf;
> +			unsigned int bs_size_sps;
> +			unsigned int bs_size_pps;
> +			unsigned int bs_size_frm;
> +
> +			mtk_vcodec_debug(inst,
> +					 "h264_encode_frame prepend SPS/PPS");
> +			ret = h264_encode_sps(inst, bs_buf, &bs_size_sps);
> +			if (ret)
> +				goto encode_err;
> +
> +			ret = h264_encode_pps(inst, &inst->pps_buf,
> +					      &bs_size_pps);
> +			if (ret)
> +				goto encode_err;
> +
> +			memcpy(bs_buf->va + bs_size_sps,
> +			       inst->pps_buf.va,
> +			       bs_size_pps);
> +
> +			hdr_sz = bs_size_sps + bs_size_pps;
> +			hdr_sz_ext = (hdr_sz & (bs_alignment - 1));
> +			if (hdr_sz_ext) {
> +				filler_sz = bs_alignment - hdr_sz_ext;
> +				if (hdr_sz_ext + 5 > bs_alignment)
> +					filler_sz += bs_alignment;
> +				h264_encode_filler(
> +					inst, bs_buf->va + hdr_sz,
> +					filler_sz);
> +			}
> +
> +			tmp_bs_buf.va = bs_buf->va + hdr_sz +
> +				filler_sz;
> +			tmp_bs_buf.dma_addr = bs_buf->dma_addr + hdr_sz +
> +				filler_sz;
> +			tmp_bs_buf.size = bs_buf->size -
> +				(hdr_sz + filler_sz);
> +
> +			ret = h264_encode_frame(inst, frm_buf, &tmp_bs_buf,
> +						&bs_size_frm);
> +			if (ret)
> +				goto encode_err;
> +
> +			result->bs_size = hdr_sz + filler_sz + bs_size_frm;
> +			mtk_vcodec_debug(inst,
> +					 "hdr %d filler %d frame %d bs %d",
> +					 hdr_sz, filler_sz, bs_size_frm,
> +					 result->bs_size);
> +
> +			inst->prepend_hdr = 0;
> +		} else {
> +			ret = h264_encode_frame(inst, frm_buf, bs_buf,
> +						&result->bs_size);
> +			if (ret)
> +				goto encode_err;
> +		}
> +		result->is_key_frm = inst->is_key_frm;
> +		break;
> +
> +	default:
> +		mtk_vcodec_err(inst, "venc_start_opt %d not supported", opt);
> +		ret = -EINVAL;
> +		break;
> +	}
> +
> +encode_err:
> +	if (ret)
> +		result->msg = VENC_MESSAGE_ERR;
> +	else
> +		result->msg = VENC_MESSAGE_OK;
> +
> +	disable_irq(ctx->dev->enc_irq);
> +	mtk_vcodec_debug(inst, "opt %d <-", opt);
> +
> +	return ret;
> +}
> +
> +static int h264_enc_set_param(unsigned long handle,
> +			      enum venc_set_param_type type, void *in)
> +{
> +	int ret = 0;
> +	struct venc_h264_inst *inst = (struct venc_h264_inst *)handle;
> +	struct venc_enc_prm *enc_prm;
> +
> +	mtk_vcodec_debug(inst, "->type=%d", type);
> +
> +	switch (type) {
> +	case VENC_SET_PARAM_ENC:
> +		enc_prm = in;
> +		ret = h264_enc_vpu_set_param(inst, type, enc_prm);
> +		if (ret)
> +			break;
> +		if (inst->work_buf_allocated == 1) {
> +			h264_enc_free_work_buf(inst);
> +			inst->work_buf_allocated = 0;
> +		}
> +		if (inst->work_buf_allocated == 0) {
> +			ret = h264_enc_alloc_work_buf(inst, enc_prm);
> +			if (ret)
> +				break;
> +			inst->work_buf_allocated = 1;
> +		}
> +		break;
> +
> +	case VENC_SET_PARAM_FORCE_INTRA:
> +		ret = h264_enc_vpu_set_param(inst, type, 0);
> +		break;
> +
> +	case VENC_SET_PARAM_ADJUST_BITRATE:
> +		enc_prm = in;
> +		ret = h264_enc_vpu_set_param(inst, type, &enc_prm->bitrate);
> +		break;
> +
> +	case VENC_SET_PARAM_ADJUST_FRAMERATE:
> +		enc_prm = in;
> +		ret = h264_enc_vpu_set_param(inst, type, &enc_prm->frm_rate);
> +		break;
> +
> +	case VENC_SET_PARAM_I_FRAME_INTERVAL:
> +		ret = h264_enc_vpu_set_param(inst, type, in);
> +		break;
> +
> +	case VENC_SET_PARAM_SKIP_FRAME:
> +		ret = h264_enc_vpu_set_param(inst, type, 0);
> +		break;
> +
> +	case VENC_SET_PARAM_PREPEND_HEADER:
> +		inst->prepend_hdr = 1;
> +		mtk_vcodec_debug(inst, "set prepend header mode");
> +		break;
> +
> +	default:
> +		mtk_vcodec_err(inst, "type %d not supported", type);
> +		ret = -EINVAL;
> +		break;
> +	}
> +
> +	mtk_vcodec_debug_leave(inst);
> +
> +	return ret;
> +}
> +
> +static int h264_enc_deinit(unsigned long handle)
> +{
> +	int ret = 0;
> +	struct venc_h264_inst *inst = (struct venc_h264_inst *)handle;
> +
> +	mtk_vcodec_debug_enter(inst);
> +
> +	ret = h264_enc_vpu_deinit(inst);
> +
> +	if (inst->work_buf_allocated)
> +		h264_enc_free_work_buf(inst);
> +
> +	mtk_vcodec_debug_leave(inst);
> +	kfree(inst);
> +
> +	return ret;
> +}
> +
> +static struct venc_common_if venc_h264_if = {
> +	h264_enc_init,
> +	h264_enc_encode,
> +	h264_enc_set_param,
> +	h264_enc_deinit,
> +};
> +
> +struct venc_common_if *get_h264_enc_comm_if(void)
> +{
> +	return &venc_h264_if;
> +}
> diff --git a/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.h b/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.h
> new file mode 100644
> index 0000000..9ac317a
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.h
> @@ -0,0 +1,165 @@
> +/*
> + * Copyright (c) 2015 MediaTek Inc.
> + * Author: Jungchang Tsao <jungchang.tsao@mediatek.com>
> + *         Daniel Hsiao <daniel.hsiao@mediatek.com>
> + *         PoChun Lin <pochun.lin@mediatek.com>
> + *
> + * This program is free software; you can redistribute it and/or
> + * modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + */
> +
> +#ifndef _VENC_H264_IF_H_
> +#define _VENC_H264_IF_H_
> +
> +#include "venc_drv_base.h"
> +
> +/**
> + * enum venc_h264_vpu_work_buf - h264 encoder buffer index
> + */
> +enum venc_h264_vpu_work_buf {
> +	VENC_H264_VPU_WORK_BUF_RC_INFO,
> +	VENC_H264_VPU_WORK_BUF_RC_CODE,
> +	VENC_H264_VPU_WORK_BUF_REC_LUMA,
> +	VENC_H264_VPU_WORK_BUF_REC_CHROMA,
> +	VENC_H264_VPU_WORK_BUF_REF_LUMA,
> +	VENC_H264_VPU_WORK_BUF_REF_CHROMA,
> +	VENC_H264_VPU_WORK_BUF_MV_INFO_1,
> +	VENC_H264_VPU_WORK_BUF_MV_INFO_2,
> +	VENC_H264_VPU_WORK_BUF_SKIP_FRAME,
> +	VENC_H264_VPU_WORK_BUF_SRC_LUMA,
> +	VENC_H264_VPU_WORK_BUF_SRC_CHROMA,
> +	VENC_H264_VPU_WORK_BUF_SRC_CHROMA_CB,
> +	VENC_H264_VPU_WORK_BUF_SRC_CHROMA_CR,
> +	VENC_H264_VPU_WORK_BUF_MAX,
> +};
> +
> +/**
> + * enum venc_h264_bs_mode - for bs_mode argument in h264_enc_vpu_encode
> + */
> +enum venc_h264_bs_mode {
> +	H264_BS_MODE_SPS,
> +	H264_BS_MODE_PPS,
> +	H264_BS_MODE_FRAME,
> +};
> +
> +/*
> + * struct venc_h264_vpu_config - Structure for h264 encoder configuration
> + * @input_fourcc: input fourcc
> + * @bitrate: target bitrate (in bps)
> + * @pic_w: picture width
> + * @pic_h: picture height
> + * @buf_w: buffer width
> + * @buf_h: buffer height
> + * @intra_period: intra frame period
> + * @framerate: frame rate
> + * @profile: as specified in standard
> + * @level: as specified in standard
> + * @wfd: WFD mode 1:on, 0:off
> + */
> +struct venc_h264_vpu_config {
> +	u32 input_fourcc;
> +	u32 bitrate;
> +	u32 pic_w;
> +	u32 pic_h;
> +	u32 buf_w;
> +	u32 buf_h;
> +	u32 intra_period;
> +	u32 framerate;
> +	u32 profile;
> +	u32 level;
> +	u32 wfd;
> +};
> +
> +/*
> + * struct venc_h264_vpu_buf - Structure for buffer information
> + * @align: buffer alignment (in bytes)
> + * @iova: IO virtual address
> + * @vpua: VPU side memory addr which is used by RC_CODE
> + * @size: buffer size (in bytes)
> + */
> +struct venc_h264_vpu_buf {
> +	u32 align;
> +	u32 iova;
> +	u32 vpua;
> +	u32 size;
> +};
> +
> +/*
> + * struct venc_h264_vpu_drv - Structure for VPU driver control and info share
> + * This structure is allocated in VPU side and shared to AP side.
> + * @config: h264 encoder configuration
> + * @work_bufs: working buffer information in VPU side
> + * The work_bufs here is for storing the 'size' info shared to AP side.
> + * The similar item in struct venc_h264_inst is for memory allocation
> + * in AP side. The AP driver will copy the 'size' from here to the one in
> + * struct mtk_vcodec_mem, then invoke mtk_vcodec_mem_alloc to allocate
> + * the buffer. After that, bypass the 'dma_addr' to the 'iova' field here for
> + * register setting in VPU side.
> + */
> +struct venc_h264_vpu_drv {
> +	struct venc_h264_vpu_config config;
> +	struct venc_h264_vpu_buf work_bufs[VENC_H264_VPU_WORK_BUF_MAX];
> +};
> +
> +/*
> + * struct venc_h264_vpu_inst - h264 encoder VPU driver instance
> + * @wq_hd: wait queue used for vpu cmd trigger then wait vpu interrupt done
> + * @signaled: flag used for checking vpu interrupt done
> + * @failure: flag to show vpu cmd succeeds or not
> + * @state: enum venc_ipi_msg_enc_state
> + * @bs_size: bitstream size for skip frame case usage
> + * @wait_int: flag to wait interrupt done (0: for skip frame case, 1: normal
> + *	      case)
> + * @id: VPU instance id
> + * @drv: driver structure allocated by VPU side and shared to AP side for
> + *	 control and info share
> + */
> +struct venc_h264_vpu_inst {
> +	wait_queue_head_t wq_hd;
> +	int signaled;
> +	int failure;
> +	int state;
> +	int bs_size;
> +	int wait_int;
> +	unsigned int id;
> +	struct venc_h264_vpu_drv *drv;
> +};
> +
> +/*
> + * struct venc_h264_inst - h264 encoder AP driver instance
> + * @hw_base: h264 encoder hardware register base
> + * @work_bufs: working buffer
> + * @pps_buf: buffer to store the pps bitstream
> + * @work_buf_allocated: working buffer allocated flag
> + * @frm_cnt: encoded frame count
> + * @prepend_hdr: when the v4l2 layer send VENC_SET_PARAM_PREPEND_HEADER cmd
> + *  through h264_enc_set_param interface, it will set this flag and prepend the
> + *  sps/pps in h264_enc_encode function.
> + * @is_key_frm: key frame flag
> + * @vpu_inst: VPU instance to exchange information between AP and VPU
> + * @ctx: context for v4l2 layer integration
> + * @dev: device for v4l2 layer integration
> + */
> +struct venc_h264_inst {
> +	void __iomem *hw_base;
> +	struct mtk_vcodec_mem work_bufs[VENC_H264_VPU_WORK_BUF_MAX];
> +	struct mtk_vcodec_mem pps_buf;
> +	bool work_buf_allocated;
> +	unsigned int frm_cnt;
> +	unsigned int prepend_hdr;
> +	unsigned int is_key_frm;
> +	struct venc_h264_vpu_inst vpu_inst;
> +	void *ctx;
> +	struct platform_device *dev;
> +};
> +
> +struct venc_common_if *get_h264_enc_comm_if(void);
> +
> +#endif
> diff --git a/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.c b/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.c
> new file mode 100644
> index 0000000..450be45
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.c
> @@ -0,0 +1,309 @@
> +/*
> + * Copyright (c) 2015 MediaTek Inc.
> + * Author: Jungchang Tsao <jungchang.tsao@mediatek.com>
> + *         Daniel Hsiao <daniel.hsiao@mediatek.com>
> + *         PoChun Lin <pochun.lin@mediatek.com>
> + *
> + * This program is free software; you can redistribute it and/or
> + * modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + */
> +
> +#include "mtk_vpu.h"
> +
> +#include "venc_h264_if.h"
> +#include "venc_h264_vpu.h"
> +#include "venc_ipi_msg.h"
> +
> +static unsigned int h264_get_profile(unsigned int profile)
> +{
> +	/* (Baseline=66, Main=77, High=100) */
> +	switch (profile) {
> +	case V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE:
> +		return 66;
> +	case V4L2_MPEG_VIDEO_H264_PROFILE_MAIN:
> +		return 77;
> +	case V4L2_MPEG_VIDEO_H264_PROFILE_HIGH:
> +		return 100;
> +	default:
> +		return 100;
> +	}
> +}
> +
> +static unsigned int h264_get_level(unsigned int level)
> +{
> +	/* (UpTo4.1(HighProfile)) */
> +	switch (level) {
> +	case V4L2_MPEG_VIDEO_H264_LEVEL_1_0:
> +		return 10;
> +	case V4L2_MPEG_VIDEO_H264_LEVEL_1_1:
> +		return 11;
> +	case V4L2_MPEG_VIDEO_H264_LEVEL_1_2:
> +		return 12;
> +	case V4L2_MPEG_VIDEO_H264_LEVEL_1_3:
> +		return 13;
> +	case V4L2_MPEG_VIDEO_H264_LEVEL_2_0:
> +		return 20;
> +	case V4L2_MPEG_VIDEO_H264_LEVEL_2_1:
> +		return 21;
> +	case V4L2_MPEG_VIDEO_H264_LEVEL_2_2:
> +		return 22;
> +	case V4L2_MPEG_VIDEO_H264_LEVEL_3_0:
> +		return 30;
> +	case V4L2_MPEG_VIDEO_H264_LEVEL_3_1:
> +		return 31;
> +	case V4L2_MPEG_VIDEO_H264_LEVEL_3_2:
> +		return 32;
> +	case V4L2_MPEG_VIDEO_H264_LEVEL_4_0:
> +		return 40;
> +	case V4L2_MPEG_VIDEO_H264_LEVEL_4_1:
> +		return 41;
> +	default:
> +		return 31;
> +	}
> +}
> +
> +static void handle_h264_enc_init_msg(struct venc_h264_inst *inst, void *data)
> +{
> +	struct venc_vpu_ipi_msg_init *msg = data;
> +
> +	inst->vpu_inst.id = msg->inst_id;
> +	inst->vpu_inst.drv = (struct venc_h264_vpu_drv *)vpu_mapping_dm_addr(
> +		inst->dev, msg->inst_id);
> +}
> +
> +static void handle_h264_enc_encode_msg(struct venc_h264_inst *inst, void *data)
> +{
> +	struct venc_vpu_ipi_msg_enc *msg = data;
> +
> +	inst->vpu_inst.state = msg->state;
> +	inst->vpu_inst.bs_size = msg->bs_size;
> +	inst->is_key_frm = msg->key_frame;
> +}
> +
> +static void h264_enc_vpu_ipi_handler(void *data, unsigned int len, void *priv)
> +{
> +	struct venc_vpu_ipi_msg_common *msg = data;
> +	struct venc_h264_inst *inst = (struct venc_h264_inst *)msg->venc_inst;
> +
> +	mtk_vcodec_debug(inst, "msg_id %x inst %p status %d",
> +			 msg->msg_id, inst, msg->status);
> +
> +	switch (msg->msg_id) {
> +	case VPU_IPIMSG_H264_ENC_INIT_DONE:
> +		handle_h264_enc_init_msg(inst, data);
> +		break;
> +	case VPU_IPIMSG_H264_ENC_SET_PARAM_DONE:
> +		break;
> +	case VPU_IPIMSG_H264_ENC_ENCODE_DONE:
> +		handle_h264_enc_encode_msg(inst, data);
> +		break;
> +	case VPU_IPIMSG_H264_ENC_DEINIT_DONE:
> +		break;
> +	default:
> +		mtk_vcodec_err(inst, "unknown msg id %x", msg->msg_id);
> +		break;
> +	}
> +
> +	inst->vpu_inst.signaled = 1;
> +	inst->vpu_inst.failure = (msg->status != VENC_IPI_MSG_STATUS_OK);
> +
> +	mtk_vcodec_debug_leave(inst);
> +}
> +
> +static int h264_enc_vpu_send_msg(struct venc_h264_inst *inst, void *msg,
> +				 int len)
> +{
> +	int status;
> +
> +	mtk_vcodec_debug_enter(inst);
> +	status = vpu_ipi_send(inst->dev, IPI_VENC_H264, msg, len);
> +	if (status) {
> +		mtk_vcodec_err(inst, "vpu_ipi_send msg %x len %d fail %d",
> +			       *(unsigned int *)msg, len, status);
> +		return -EINVAL;
> +	}
> +	mtk_vcodec_debug_leave(inst);
> +
> +	return 0;
> +}
> +
> +int h264_enc_vpu_init(struct venc_h264_inst *inst)
> +{
> +	int status;
> +	struct venc_ap_ipi_msg_init out;
> +
> +	mtk_vcodec_debug_enter(inst);
> +
> +	init_waitqueue_head(&inst->vpu_inst.wq_hd);
> +	inst->vpu_inst.signaled = 0;
> +	inst->vpu_inst.failure = 0;
> +
> +	status = vpu_ipi_register(inst->dev, IPI_VENC_H264,
> +				  h264_enc_vpu_ipi_handler,
> +				  "h264_enc", NULL);
> +	if (status) {
> +		mtk_vcodec_err(inst, "vpu_ipi_register fail %d", status);
> +		return -EINVAL;
> +	}
> +
> +	out.msg_id = AP_IPIMSG_H264_ENC_INIT;
> +	out.venc_inst = (unsigned long)inst;
> +	if (h264_enc_vpu_send_msg(inst, &out, sizeof(out)) ||
> +	    inst->vpu_inst.failure) {
> +		mtk_vcodec_err(inst, "AP_IPIMSG_H264_ENC_INIT failed");
> +		return -EINVAL;
> +	}
> +
> +	mtk_vcodec_debug_leave(inst);
> +
> +	return 0;
> +}
> +
> +int h264_enc_vpu_set_param(struct venc_h264_inst *inst, unsigned int id,
> +			   void *param)
> +{
> +	struct venc_ap_ipi_msg_set_param out;
> +
> +	mtk_vcodec_debug(inst, "id %d ->", id);
> +
> +	out.msg_id = AP_IPIMSG_H264_ENC_SET_PARAM;
> +	out.inst_id = inst->vpu_inst.id;
> +	out.param_id = id;
> +	switch (id) {
> +	case VENC_SET_PARAM_ENC: {
> +		struct venc_enc_prm *enc_param = (struct venc_enc_prm *)param;
> +
> +		inst->vpu_inst.drv->config.input_fourcc =
> +			enc_param->input_fourcc;
> +		inst->vpu_inst.drv->config.bitrate = enc_param->bitrate;
> +		inst->vpu_inst.drv->config.pic_w = enc_param->width;
> +		inst->vpu_inst.drv->config.pic_h = enc_param->height;
> +		inst->vpu_inst.drv->config.buf_w = enc_param->buf_width;
> +		inst->vpu_inst.drv->config.buf_h = enc_param->buf_height;
> +		inst->vpu_inst.drv->config.intra_period =
> +			enc_param->intra_period;
> +		inst->vpu_inst.drv->config.framerate = enc_param->frm_rate;
> +		inst->vpu_inst.drv->config.profile =
> +			h264_get_profile(enc_param->h264_profile);
> +		inst->vpu_inst.drv->config.level =
> +			h264_get_level(enc_param->h264_level);
> +		inst->vpu_inst.drv->config.wfd = 0;
> +		out.data_item = 0;
> +		break;
> +	}
> +	case VENC_SET_PARAM_FORCE_INTRA:
> +		out.data_item = 0;
> +		break;
> +	case VENC_SET_PARAM_ADJUST_BITRATE:
> +		out.data_item = 1;
> +		out.data[0] = *(unsigned int *)param;
> +		break;
> +	case VENC_SET_PARAM_ADJUST_FRAMERATE:
> +		out.data_item = 1;
> +		out.data[0] = *(unsigned int *)param;
> +		break;
> +	case VENC_SET_PARAM_I_FRAME_INTERVAL:
> +		out.data_item = 1;
> +		out.data[0] = *(unsigned int *)param;
> +		break;
> +	case VENC_SET_PARAM_SKIP_FRAME:
> +		out.data_item = 0;
> +		break;
> +	}
> +	if (h264_enc_vpu_send_msg(inst, &out, sizeof(out)) ||
> +	    inst->vpu_inst.failure) {
> +		mtk_vcodec_err(inst,
> +			       "AP_IPIMSG_H264_ENC_SET_PARAM %d fail", id);
> +		return -EINVAL;
> +	}
> +
> +	mtk_vcodec_debug(inst, "id %d <-", id);
> +
> +	return 0;
> +}
> +
> +int h264_enc_vpu_encode(struct venc_h264_inst *inst, unsigned int bs_mode,
> +			struct venc_frm_buf *frm_buf,
> +			struct mtk_vcodec_mem *bs_buf,
> +			unsigned int *bs_size)
> +{
> +	struct venc_ap_ipi_msg_enc out;
> +
> +	mtk_vcodec_debug(inst, "bs_mode %d ->", bs_mode);
> +
> +	out.msg_id = AP_IPIMSG_H264_ENC_ENCODE;
> +	out.inst_id = inst->vpu_inst.id;
> +	out.bs_mode = bs_mode;
> +	if (frm_buf) {
> +		if ((frm_buf->fb_addr.dma_addr % 16 == 0) &&
> +		    (frm_buf->fb_addr1.dma_addr % 16 == 0) &&
> +		    (frm_buf->fb_addr2.dma_addr % 16 == 0)) {
> +			out.input_addr[0] = frm_buf->fb_addr.dma_addr;
> +			out.input_addr[1] = frm_buf->fb_addr1.dma_addr;
> +			out.input_addr[2] = frm_buf->fb_addr2.dma_addr;
> +		} else {
> +			mtk_vcodec_err(inst, "dma_addr not align to 16");
> +			return -EINVAL;
> +		}
> +	} else {
> +		out.input_addr[0] = 0;
> +		out.input_addr[1] = 0;
> +		out.input_addr[2] = 0;
> +	}
> +	if (bs_buf) {
> +		out.bs_addr = bs_buf->dma_addr;
> +		out.bs_size = bs_buf->size;
> +	} else {
> +		out.bs_addr = 0;
> +		out.bs_size = 0;
> +	}
> +	if (h264_enc_vpu_send_msg(inst, &out, sizeof(out)) ||
> +	    inst->vpu_inst.failure) {
> +		mtk_vcodec_err(inst, "AP_IPIMSG_H264_ENC_ENCODE %d fail",
> +			       bs_mode);
> +		return -EINVAL;
> +	}
> +
> +	mtk_vcodec_debug(inst, "state %d size %d key_frm %d",
> +			 inst->vpu_inst.state, inst->vpu_inst.bs_size,
> +			 inst->is_key_frm);
> +
> +	inst->vpu_inst.wait_int = 1;
> +	if (inst->vpu_inst.state == VEN_IPI_MSG_ENC_STATE_SKIP) {
> +		*bs_size = inst->vpu_inst.bs_size;
> +		memcpy(bs_buf->va,
> +		       inst->work_bufs[VENC_H264_VPU_WORK_BUF_SKIP_FRAME].va,
> +		       *bs_size);
> +		inst->vpu_inst.wait_int = 0;
> +	}
> +
> +	mtk_vcodec_debug(inst, "bs_mode %d <-", bs_mode);
> +
> +	return 0;
> +}
> +
> +int h264_enc_vpu_deinit(struct venc_h264_inst *inst)
> +{
> +	struct venc_ap_ipi_msg_deinit out;
> +
> +	mtk_vcodec_debug_enter(inst);
> +
> +	out.msg_id = AP_IPIMSG_H264_ENC_DEINIT;
> +	out.inst_id = inst->vpu_inst.id;
> +	if (h264_enc_vpu_send_msg(inst, &out, sizeof(out)) ||
> +	    inst->vpu_inst.failure) {
> +		mtk_vcodec_err(inst, "AP_IPIMSG_H264_ENC_DEINIT fail");
> +		return -EINVAL;
> +	}
> +
> +	mtk_vcodec_debug_leave(inst);
> +
> +	return 0;
> +}
> diff --git a/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.h b/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.h
> new file mode 100644
> index 0000000..deccc6f
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.h
> @@ -0,0 +1,30 @@
> +/*
> + * Copyright (c) 2015 MediaTek Inc.
> + * Author: Jungchang Tsao <jungchang.tsao@mediatek.com>
> + *         Daniel Hsiao <daniel.hsiao@mediatek.com>
> + *         PoChun Lin <pochun.lin@mediatek.com>
> + *
> + * This program is free software; you can redistribute it and/or
> + * modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + */
> +
> +#ifndef _VENC_H264_VPU_H_
> +#define _VENC_H264_VPU_H_
> +
> +int h264_enc_vpu_init(struct venc_h264_inst *inst);
> +int h264_enc_vpu_set_param(struct venc_h264_inst *inst, unsigned int id,
> +			   void *param);
> +int h264_enc_vpu_encode(struct venc_h264_inst *inst, unsigned int bs_mode,
> +			struct venc_frm_buf *frm_buf,
> +			struct mtk_vcodec_mem *bs_buf,
> +			unsigned int *bs_size);
> +int h264_enc_vpu_deinit(struct venc_h264_inst *inst);
> +
> +#endif
> diff --git a/drivers/media/platform/mtk-vcodec/venc_drv_if.c b/drivers/media/platform/mtk-vcodec/venc_drv_if.c
> index d293f2c..28ef4a7 100644
> --- a/drivers/media/platform/mtk-vcodec/venc_drv_if.c
> +++ b/drivers/media/platform/mtk-vcodec/venc_drv_if.c
> @@ -25,6 +25,7 @@
>  
>  #include "venc_drv_base.h"
>  #include "vp8_enc/venc_vp8_if.h"
> +#include "h264_enc/venc_h264_if.h"
>  
>  int venc_if_create(struct mtk_vcodec_ctx *ctx, unsigned int fourcc)
>  {
> @@ -38,6 +39,8 @@ int venc_if_create(struct mtk_vcodec_ctx *ctx, unsigned int fourcc)
>                  ctx->enc_if = get_vp8_enc_comm_if();
>                  break;
>  	case V4L2_PIX_FMT_H264:
> +	        ctx->enc_if = get_h264_enc_comm_if();
> +	        break;
>  	default:
>  		return -EINVAL;
>  	}
> 

Regards,

	Hans

^ permalink raw reply	[flat|nested] 102+ messages in thread

* Re: [PATCH v4 0/8] Add MT8173 Video Encoder Driver and VPU Driver
  2016-02-04 11:34 ` Tiffany Lin
@ 2016-02-15 12:04   ` Hans Verkuil
  -1 siblings, 0 replies; 102+ messages in thread
From: Hans Verkuil @ 2016-02-15 12:04 UTC (permalink / raw)
  To: Tiffany Lin, Hans Verkuil, daniel.thompson, Rob Herring,
	Mauro Carvalho Chehab, Matthias Brugger, Daniel Kurtz,
	Pawel Osciak
  Cc: Eddie Huang, Yingjoe Chen, devicetree, linux-kernel,
	linux-arm-kernel, linux-media, linux-mediatek, PoChun.Lin

Hi Tiffany,

Once again, thanks for your patch series!

I have some more comments below.

On 02/04/2016 12:34 PM, Tiffany Lin wrote:
> 
> ==============
>  Introduction
> ==============
> 
> The purpose of this series is to add the driver for video codec hw embedded in the Mediatek's MT8173 SoCs.
> Mediatek Video Codec is able to handle video encoding of in a range of formats.
> 
> This patch series also include VPU driver. Mediatek Video Codec driver rely on VPU driver to load, communicate with VPU.
> 
> Internally the driver uses videobuf2 framework and MTK IOMMU and MTK SMI.
> MTK IOMMU[1] and MTK SMI[2] have not yet been merged, but we wanted to start discussion about the driver earlier so it could be merged sooner.
> 
> [1]https://patchwork.kernel.org/patch/8120921/
> [2]https://patchwork.kernel.org/patch/8120941/
> 
> 
> ==================
>  Device interface
> ==================
> 
> In principle the driver bases on v4l2 memory-to-memory framework:
> it provides a single video node and each opened file handle gets its own private context with separate buffer queues. Each context consist of 2 buffer queues: OUTPUT (for source buffers, i.e. raw video frames) and CAPTURE (for destination buffers, i.e. encoded video frames).
> 
> ==============================
>  VPU (Video Processor Unit)
> ==============================
> The VPU driver for hw video codec embedded in Mediatek's MT8173 SOCs.
> It is able to handle video decoding/encoding in a range of formats.
> The driver provides with VPU firmware download, memory management and the communication interface between CPU and VPU.
> For VPU initialization, it will create virtual memory for CPU access and physical address for VPU hw device access. When a decode/encode instance opens a device node, vpu driver will download vpu firmware to the device.
> A decode/encode instant will decode/encode a frame using VPU interface to interrupt vpu to handle decoding/encoding jobs.
> 
> Please have a look at the code and comments will be very much appreciated.
> 
> 
> Change in v4:
> Vcodec Part
> 1. Remove MTK_ENCODE_PARAM_SKIP_FRAME support
> 2. Remove MTK_ENCODE_PARAM_FRAME_TYPE and change to use V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME[3]
> 3. Refine Encoder HW clock source
> 4. Refine debug log
> 5. Add watchdog support
> 6. With patch "media: v4l2-compat-ioctl32: fix missing length copy in put_v4l2_buffer32"[4], v4l2-compliance test passed[5] in v4.4-rc5 
> 
> VPU Part
> 1. These two patches were Acked-by: Rob Herring <robh@kernel.org> in v3
>    [PATCH v3 1/8] dt-bindings: Add a binding for Mediatek Video Processor
>    [PATCH v3 3/8] arm64: dts: mediatek: Add node for Mediatek Video Processor Unit
>    Because we were wrong about how the hardware works, there is no connection between VPU and IOMMU HW
>    We remove VPU attaching to IOMMU
> 2. Support VPU running on 4GB DRAM system
> 3. Support VPU watchdog reset
> 4. Refine for coding style 
> 
> [3]https://patchwork.linuxtv.org/patch/32670/
> [4] https://patchwork.linuxtv.org/patch/32631/
> [5]localhost ~ # /usr/bin/v4l2-compliance -d /dev/video1
> Driver Info:
>         Driver name   : mtk-vcodec-en
>         Card type     : platform:mt817
>         Bus info      : platform:mt817

Perhaps this should be mt817x?

>         Driver version: 4.4.0
>         Capabilities  : 0x84204000
>                 Video Memory-to-Memory Multiplanar
>                 Streaming
>                 Extended Pix Format
>                 Device Capabilities
>         Device Caps   : 0x04204000
>                 Video Memory-to-Memory Multiplanar
>                 Streaming
>                 Extended Pix Format
> 
> Compliance test for device /dev/video1 (not using libv4l2):
> 
> Required ioctls:
>         test VIDIOC_QUERYCAP: OK
> 
> Allow for multiple opens:
>         test second video open: OK
>         test VIDIOC_QUERYCAP: OK
>         test VIDIOC_G/S_PRIORITY: OK
> 
> Debug ioctls:
>         test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
>         test VIDIOC_LOG_STATUS: OK (Not Supported)
> 
> Input ioctls:
>         test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>         test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>         test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>         test VIDIOC_ENUMAUDIO: OK (Not Supported)
>         test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>         test VIDIOC_G/S_AUDIO: OK (Not Supported)
>         Inputs: 0 Audio Inputs: 0 Tuners: 0
> 
> Output ioctls:
>         test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>         test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>         test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>         test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>         test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>         Outputs: 0 Audio Outputs: 0 Modulators: 0
> 
> Input/Output configuration ioctls:
>         test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>         test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>         test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>         test VIDIOC_G/S_EDID: OK (Not Supported)
> 
>         Control ioctls:
>                 test VIDIOC_QUERYCTRL/MENU: OK
>                 test VIDIOC_G/S_CTRL: OK
>                 test VIDIOC_G/S/TRY_EXT_CTRLS: OK
>                 test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
>                 test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>                 Standard Controls: 12 Private Controls: 0
> 
>         Format ioctls:
>                 test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
>                 test VIDIOC_G/S_PARM: OK (Not Supported)
>                 test VIDIOC_G_FBUF: OK (Not Supported)
>                 test VIDIOC_G_FMT: OK
>                 test VIDIOC_TRY_FMT: OK
>                 test VIDIOC_S_FMT: OK
>                 test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> 
>         Codec ioctls:
>                 test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>                 test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>                 test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> 
>         Buffer ioctls:
>                 warn: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-buffers.cpp(475): VIDIOC_CREATE_BUFS not supported
>                 warn: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-buffers.cpp(475): VIDIOC_CREATE_BUFS not supported
>                 test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
>                 test VIDIOC_EXPBUF: OK
> 
> Total: 38, Succeeded: 38, Failed: 0, Warnings: 2

Looks great!

Having reviewed the v4 patch series I'd say it is close to being ready (at least
w.r.t. the V4L2 parts). Most of my comments are minor, but fixing the open()
limitation is probably a bigger change on your side.

It should be well worth the effort, though. In my experience limiting the number
of time you can call open() is always a sign of a design flaw and the code improves
if this is handled in the right place (usually start_streaming, which is when you
actually start to use the hardware).

And besides, limiting the number of open()s is out-of-spec.

Regards,

	Hans

> 
> 
> Change in v3:
> 1.Refine code to pass v4l2-compliance test, now it still has 2 issues 2.Refine code according to latest MTK IOMMU patches[1] 3.Remove MFC51 specific CIDs and add MTK specific CIDs for for keyframe and
>   skip I-frame
> 4.Refine code according to review comments
> 
> Below is the v1.6 version v4l2-compliance report for the mt8173 encoder driver.
> Now there are still 2 test fail in v1.6.
> For VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF, we directly use v4l2_m2m_ioctl_* functions, but it still fail. It pass in kernel 3.18 but fail in kernel 4.4.
> We will try v1.8 in next version.
> VIDIOC_EXPBUF is becuase we support all three memory types VB2_DMABUF, VB2_MMAP and VB2_USERPTR. VIDIOC_EXPBUF only allowed when only VB2_MMAP supported.
> localhost ~ # /usr/bin/v4l2-compliance -d /dev/video1 Driver Info:
>         Driver name   : mtk-vcodec-en
>         Card type     : platform:mt817
>         Bus info      : platform:mt817
>         Driver version: 4.4.0
>         Capabilities  : 0x84204000
>                 Video Memory-to-Memory Multiplanar
>                 Streaming
>                 Extended Pix Format
>                 Device Capabilities
>         Device Caps   : 0x04204000
>                 Video Memory-to-Memory Multiplanar
>                 Streaming
>                 Extended Pix Format
> 
> Compliance test for device /dev/video1 (not using libv4l2):
> 
> Required ioctls:
>         test VIDIOC_QUERYCAP: OK
> 
> Allow for multiple opens:
>         test second video open: OK
>         test VIDIOC_QUERYCAP: OK
>         test VIDIOC_G/S_PRIORITY: OK
> 
> Debug ioctls:
>         test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
>         test VIDIOC_LOG_STATUS: OK (Not Supported)
> 
> Input ioctls:
>         test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>         test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>         test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>         test VIDIOC_ENUMAUDIO: OK (Not Supported)
>         test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>         test VIDIOC_G/S_AUDIO: OK (Not Supported)
>         Inputs: 0 Audio Inputs: 0 Tuners: 0
> 
> Output ioctls:
>         test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>         test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>         test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>         test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>         test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>         Outputs: 0 Audio Outputs: 0 Modulators: 0
> 
> Input/Output configuration ioctls:
>         test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>         test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>         test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>         test VIDIOC_G/S_EDID: OK (Not Supported)
> 
>         Control ioctls:
>                 test VIDIOC_QUERYCTRL/MENU: OK
>                 test VIDIOC_G/S_CTRL: OK
>                 test VIDIOC_G/S/TRY_EXT_CTRLS: OK
>                 test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
>                 test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>                 Standard Controls: 11 Private Controls: 2
> 
>         Format ioctls:
>                 test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
>                 test VIDIOC_G/S_PARM: OK (Not Supported)
>                 test VIDIOC_G_FBUF: OK (Not Supported)
>                 test VIDIOC_G_FMT: OK
>                 test VIDIOC_TRY_FMT: OK
>                 test VIDIOC_S_FMT: OK
>                 test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> 
>         Codec ioctls:
>                 test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>                 test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>                 test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> 
>         Buffer ioctls:
>                 fail: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-buffers.cpp(266): vp->length == 0
>                 fail: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-buffers.cpp(335): buf.check(Unqueued, i)
>                 fail: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-buffers.cpp(420): testQueryBuf(node, i, q.g_buffers())
>                 test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
>                 fail: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-buffers.cpp(500): q.has_expbuf(node)
>                 test VIDIOC_EXPBUF: FAIL
> Total: 38, Succeeded: 36, Failed: 2, Warnings: 0
> 
> 
> Change in v2:
> Vcodec Part
> 1.Remove common and include directory in mtk-vcodec 2.Refine vb2ops_venc_start_streaming and vb2ops_venc_stop_streaming and state machine 3.Remove venc_if_init and venc_if_deinit 4.Refine debug message 5.Refine lab and vpu decription in mediatek-vcodec.txt
> 
> VPU Part
> 1. Modify VPU Kconfig
> 2. Move encoder header files to other patch sets 3. Remove marcos for extended virtual/iova address 4. Change register and variable names 5. Add a reference counter for VPU watchdog 6. Remove one busy waiting in function vpu_ipi_send 7. Operate VPU clock in VPU driver (not called by encoder drivers) 8. Refine memory mapping, firmware download and extended memory allocation/free functions 9. Release more allocated resources in driver remove function
> 
> 
> 
> Andrew-CT Chen (2):
>   dt-bindings: Add a binding for Mediatek Video Processor
>   [Media] vcodec: mediatek: Add Mediatek V4L2 Video Encoder Driver
> 
> Tiffany Lin (6):
>   [media] VPU: mediatek: support Mediatek VPU
>   arm64: dts: mediatek: Add node for Mediatek Video Processor Unit
>   dt-bindings: Add a binding for Mediatek Video Encoder
>   [media] vcodec: mediatek: Add Mediatek VP8 Video Encoder Driver
>   [media] vcodec: mediatek: Add Mediatek H264 Video Encoder Driver
>   arm64: dts: mediatek: Add Video Encoder for MT8173
> 
>  .../devicetree/bindings/media/mediatek-vcodec.txt  |   59 +
>  .../devicetree/bindings/media/mediatek-vpu.txt     |   31 +
>  arch/arm64/boot/dts/mediatek/mt8173.dtsi           |   62 +
>  drivers/media/platform/Kconfig                     |   20 +
>  drivers/media/platform/Makefile                    |    4 +
>  drivers/media/platform/mtk-vcodec/Makefile         |   11 +
>  .../media/platform/mtk-vcodec/h264_enc/Makefile    |    6 +
>  .../platform/mtk-vcodec/h264_enc/venc_h264_if.c    |  540 ++++++++
>  .../platform/mtk-vcodec/h264_enc/venc_h264_if.h    |  165 +++
>  .../platform/mtk-vcodec/h264_enc/venc_h264_vpu.c   |  309 +++++
>  .../platform/mtk-vcodec/h264_enc/venc_h264_vpu.h   |   30 +
>  drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h |  388 ++++++
>  drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c | 1380 ++++++++++++++++++++
>  drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h |   46 +
>  .../media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c |  476 +++++++
>  .../media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c  |  132 ++
>  .../media/platform/mtk-vcodec/mtk_vcodec_intr.c    |  102 ++
>  .../media/platform/mtk-vcodec/mtk_vcodec_intr.h    |   29 +
>  drivers/media/platform/mtk-vcodec/mtk_vcodec_pm.h  |   26 +
>  .../media/platform/mtk-vcodec/mtk_vcodec_util.c    |  106 ++
>  .../media/platform/mtk-vcodec/mtk_vcodec_util.h    |   85 ++
>  drivers/media/platform/mtk-vcodec/venc_drv_base.h  |   62 +
>  drivers/media/platform/mtk-vcodec/venc_drv_if.c    |  106 ++
>  drivers/media/platform/mtk-vcodec/venc_drv_if.h    |  175 +++
>  drivers/media/platform/mtk-vcodec/venc_ipi_msg.h   |  212 +++
>  drivers/media/platform/mtk-vcodec/vp8_enc/Makefile |    6 +
>  .../platform/mtk-vcodec/vp8_enc/venc_vp8_if.c      |  422 ++++++
>  .../platform/mtk-vcodec/vp8_enc/venc_vp8_if.h      |  149 +++
>  .../platform/mtk-vcodec/vp8_enc/venc_vp8_vpu.c     |  240 ++++
>  .../platform/mtk-vcodec/vp8_enc/venc_vp8_vpu.h     |   28 +
>  drivers/media/platform/mtk-vpu/Makefile            |    1 +
>  drivers/media/platform/mtk-vpu/mtk_vpu.c           |  994 ++++++++++++++
>  drivers/media/platform/mtk-vpu/mtk_vpu.h           |  167 +++
>  include/uapi/linux/v4l2-controls.h                 |    4 +
>  34 files changed, 6573 insertions(+)
>  create mode 100644 Documentation/devicetree/bindings/media/mediatek-vcodec.txt
>  create mode 100644 Documentation/devicetree/bindings/media/mediatek-vpu.txt
>  create mode 100644 drivers/media/platform/mtk-vcodec/Makefile
>  create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/Makefile
>  create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.c
>  create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.h
>  create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.c
>  create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.h
>  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
>  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
>  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h
>  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
>  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
>  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.c
>  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.h
>  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_pm.h
>  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c
>  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
>  create mode 100644 drivers/media/platform/mtk-vcodec/venc_drv_base.h
>  create mode 100644 drivers/media/platform/mtk-vcodec/venc_drv_if.c
>  create mode 100644 drivers/media/platform/mtk-vcodec/venc_drv_if.h
>  create mode 100644 drivers/media/platform/mtk-vcodec/venc_ipi_msg.h
>  create mode 100644 drivers/media/platform/mtk-vcodec/vp8_enc/Makefile
>  create mode 100644 drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_if.c
>  create mode 100644 drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_if.h
>  create mode 100644 drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_vpu.c
>  create mode 100644 drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_vpu.h
>  create mode 100644 drivers/media/platform/mtk-vpu/Makefile
>  create mode 100644 drivers/media/platform/mtk-vpu/mtk_vpu.c
>  create mode 100644 drivers/media/platform/mtk-vpu/mtk_vpu.h
>  mode change 100644 => 100755 include/uapi/linux/v4l2-controls.h
> 

^ permalink raw reply	[flat|nested] 102+ messages in thread

* [PATCH v4 0/8] Add MT8173 Video Encoder Driver and VPU Driver
@ 2016-02-15 12:04   ` Hans Verkuil
  0 siblings, 0 replies; 102+ messages in thread
From: Hans Verkuil @ 2016-02-15 12:04 UTC (permalink / raw)
  To: linux-arm-kernel

Hi Tiffany,

Once again, thanks for your patch series!

I have some more comments below.

On 02/04/2016 12:34 PM, Tiffany Lin wrote:
> 
> ==============
>  Introduction
> ==============
> 
> The purpose of this series is to add the driver for video codec hw embedded in the Mediatek's MT8173 SoCs.
> Mediatek Video Codec is able to handle video encoding of in a range of formats.
> 
> This patch series also include VPU driver. Mediatek Video Codec driver rely on VPU driver to load, communicate with VPU.
> 
> Internally the driver uses videobuf2 framework and MTK IOMMU and MTK SMI.
> MTK IOMMU[1] and MTK SMI[2] have not yet been merged, but we wanted to start discussion about the driver earlier so it could be merged sooner.
> 
> [1]https://patchwork.kernel.org/patch/8120921/
> [2]https://patchwork.kernel.org/patch/8120941/
> 
> 
> ==================
>  Device interface
> ==================
> 
> In principle the driver bases on v4l2 memory-to-memory framework:
> it provides a single video node and each opened file handle gets its own private context with separate buffer queues. Each context consist of 2 buffer queues: OUTPUT (for source buffers, i.e. raw video frames) and CAPTURE (for destination buffers, i.e. encoded video frames).
> 
> ==============================
>  VPU (Video Processor Unit)
> ==============================
> The VPU driver for hw video codec embedded in Mediatek's MT8173 SOCs.
> It is able to handle video decoding/encoding in a range of formats.
> The driver provides with VPU firmware download, memory management and the communication interface between CPU and VPU.
> For VPU initialization, it will create virtual memory for CPU access and physical address for VPU hw device access. When a decode/encode instance opens a device node, vpu driver will download vpu firmware to the device.
> A decode/encode instant will decode/encode a frame using VPU interface to interrupt vpu to handle decoding/encoding jobs.
> 
> Please have a look at the code and comments will be very much appreciated.
> 
> 
> Change in v4:
> Vcodec Part
> 1. Remove MTK_ENCODE_PARAM_SKIP_FRAME support
> 2. Remove MTK_ENCODE_PARAM_FRAME_TYPE and change to use V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME[3]
> 3. Refine Encoder HW clock source
> 4. Refine debug log
> 5. Add watchdog support
> 6. With patch "media: v4l2-compat-ioctl32: fix missing length copy in put_v4l2_buffer32"[4], v4l2-compliance test passed[5] in v4.4-rc5 
> 
> VPU Part
> 1. These two patches were Acked-by: Rob Herring <robh@kernel.org> in v3
>    [PATCH v3 1/8] dt-bindings: Add a binding for Mediatek Video Processor
>    [PATCH v3 3/8] arm64: dts: mediatek: Add node for Mediatek Video Processor Unit
>    Because we were wrong about how the hardware works, there is no connection between VPU and IOMMU HW
>    We remove VPU attaching to IOMMU
> 2. Support VPU running on 4GB DRAM system
> 3. Support VPU watchdog reset
> 4. Refine for coding style 
> 
> [3]https://patchwork.linuxtv.org/patch/32670/
> [4] https://patchwork.linuxtv.org/patch/32631/
> [5]localhost ~ # /usr/bin/v4l2-compliance -d /dev/video1
> Driver Info:
>         Driver name   : mtk-vcodec-en
>         Card type     : platform:mt817
>         Bus info      : platform:mt817

Perhaps this should be mt817x?

>         Driver version: 4.4.0
>         Capabilities  : 0x84204000
>                 Video Memory-to-Memory Multiplanar
>                 Streaming
>                 Extended Pix Format
>                 Device Capabilities
>         Device Caps   : 0x04204000
>                 Video Memory-to-Memory Multiplanar
>                 Streaming
>                 Extended Pix Format
> 
> Compliance test for device /dev/video1 (not using libv4l2):
> 
> Required ioctls:
>         test VIDIOC_QUERYCAP: OK
> 
> Allow for multiple opens:
>         test second video open: OK
>         test VIDIOC_QUERYCAP: OK
>         test VIDIOC_G/S_PRIORITY: OK
> 
> Debug ioctls:
>         test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
>         test VIDIOC_LOG_STATUS: OK (Not Supported)
> 
> Input ioctls:
>         test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>         test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>         test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>         test VIDIOC_ENUMAUDIO: OK (Not Supported)
>         test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>         test VIDIOC_G/S_AUDIO: OK (Not Supported)
>         Inputs: 0 Audio Inputs: 0 Tuners: 0
> 
> Output ioctls:
>         test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>         test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>         test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>         test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>         test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>         Outputs: 0 Audio Outputs: 0 Modulators: 0
> 
> Input/Output configuration ioctls:
>         test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>         test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>         test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>         test VIDIOC_G/S_EDID: OK (Not Supported)
> 
>         Control ioctls:
>                 test VIDIOC_QUERYCTRL/MENU: OK
>                 test VIDIOC_G/S_CTRL: OK
>                 test VIDIOC_G/S/TRY_EXT_CTRLS: OK
>                 test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
>                 test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>                 Standard Controls: 12 Private Controls: 0
> 
>         Format ioctls:
>                 test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
>                 test VIDIOC_G/S_PARM: OK (Not Supported)
>                 test VIDIOC_G_FBUF: OK (Not Supported)
>                 test VIDIOC_G_FMT: OK
>                 test VIDIOC_TRY_FMT: OK
>                 test VIDIOC_S_FMT: OK
>                 test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> 
>         Codec ioctls:
>                 test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>                 test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>                 test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> 
>         Buffer ioctls:
>                 warn: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-buffers.cpp(475): VIDIOC_CREATE_BUFS not supported
>                 warn: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-buffers.cpp(475): VIDIOC_CREATE_BUFS not supported
>                 test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
>                 test VIDIOC_EXPBUF: OK
> 
> Total: 38, Succeeded: 38, Failed: 0, Warnings: 2

Looks great!

Having reviewed the v4 patch series I'd say it is close to being ready (at least
w.r.t. the V4L2 parts). Most of my comments are minor, but fixing the open()
limitation is probably a bigger change on your side.

It should be well worth the effort, though. In my experience limiting the number
of time you can call open() is always a sign of a design flaw and the code improves
if this is handled in the right place (usually start_streaming, which is when you
actually start to use the hardware).

And besides, limiting the number of open()s is out-of-spec.

Regards,

	Hans

> 
> 
> Change in v3:
> 1.Refine code to pass v4l2-compliance test, now it still has 2 issues 2.Refine code according to latest MTK IOMMU patches[1] 3.Remove MFC51 specific CIDs and add MTK specific CIDs for for keyframe and
>   skip I-frame
> 4.Refine code according to review comments
> 
> Below is the v1.6 version v4l2-compliance report for the mt8173 encoder driver.
> Now there are still 2 test fail in v1.6.
> For VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF, we directly use v4l2_m2m_ioctl_* functions, but it still fail. It pass in kernel 3.18 but fail in kernel 4.4.
> We will try v1.8 in next version.
> VIDIOC_EXPBUF is becuase we support all three memory types VB2_DMABUF, VB2_MMAP and VB2_USERPTR. VIDIOC_EXPBUF only allowed when only VB2_MMAP supported.
> localhost ~ # /usr/bin/v4l2-compliance -d /dev/video1 Driver Info:
>         Driver name   : mtk-vcodec-en
>         Card type     : platform:mt817
>         Bus info      : platform:mt817
>         Driver version: 4.4.0
>         Capabilities  : 0x84204000
>                 Video Memory-to-Memory Multiplanar
>                 Streaming
>                 Extended Pix Format
>                 Device Capabilities
>         Device Caps   : 0x04204000
>                 Video Memory-to-Memory Multiplanar
>                 Streaming
>                 Extended Pix Format
> 
> Compliance test for device /dev/video1 (not using libv4l2):
> 
> Required ioctls:
>         test VIDIOC_QUERYCAP: OK
> 
> Allow for multiple opens:
>         test second video open: OK
>         test VIDIOC_QUERYCAP: OK
>         test VIDIOC_G/S_PRIORITY: OK
> 
> Debug ioctls:
>         test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
>         test VIDIOC_LOG_STATUS: OK (Not Supported)
> 
> Input ioctls:
>         test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>         test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>         test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>         test VIDIOC_ENUMAUDIO: OK (Not Supported)
>         test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>         test VIDIOC_G/S_AUDIO: OK (Not Supported)
>         Inputs: 0 Audio Inputs: 0 Tuners: 0
> 
> Output ioctls:
>         test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>         test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>         test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>         test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>         test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>         Outputs: 0 Audio Outputs: 0 Modulators: 0
> 
> Input/Output configuration ioctls:
>         test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>         test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>         test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>         test VIDIOC_G/S_EDID: OK (Not Supported)
> 
>         Control ioctls:
>                 test VIDIOC_QUERYCTRL/MENU: OK
>                 test VIDIOC_G/S_CTRL: OK
>                 test VIDIOC_G/S/TRY_EXT_CTRLS: OK
>                 test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
>                 test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>                 Standard Controls: 11 Private Controls: 2
> 
>         Format ioctls:
>                 test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
>                 test VIDIOC_G/S_PARM: OK (Not Supported)
>                 test VIDIOC_G_FBUF: OK (Not Supported)
>                 test VIDIOC_G_FMT: OK
>                 test VIDIOC_TRY_FMT: OK
>                 test VIDIOC_S_FMT: OK
>                 test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> 
>         Codec ioctls:
>                 test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>                 test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>                 test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> 
>         Buffer ioctls:
>                 fail: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-buffers.cpp(266): vp->length == 0
>                 fail: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-buffers.cpp(335): buf.check(Unqueued, i)
>                 fail: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-buffers.cpp(420): testQueryBuf(node, i, q.g_buffers())
>                 test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
>                 fail: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-buffers.cpp(500): q.has_expbuf(node)
>                 test VIDIOC_EXPBUF: FAIL
> Total: 38, Succeeded: 36, Failed: 2, Warnings: 0
> 
> 
> Change in v2:
> Vcodec Part
> 1.Remove common and include directory in mtk-vcodec 2.Refine vb2ops_venc_start_streaming and vb2ops_venc_stop_streaming and state machine 3.Remove venc_if_init and venc_if_deinit 4.Refine debug message 5.Refine lab and vpu decription in mediatek-vcodec.txt
> 
> VPU Part
> 1. Modify VPU Kconfig
> 2. Move encoder header files to other patch sets 3. Remove marcos for extended virtual/iova address 4. Change register and variable names 5. Add a reference counter for VPU watchdog 6. Remove one busy waiting in function vpu_ipi_send 7. Operate VPU clock in VPU driver (not called by encoder drivers) 8. Refine memory mapping, firmware download and extended memory allocation/free functions 9. Release more allocated resources in driver remove function
> 
> 
> 
> Andrew-CT Chen (2):
>   dt-bindings: Add a binding for Mediatek Video Processor
>   [Media] vcodec: mediatek: Add Mediatek V4L2 Video Encoder Driver
> 
> Tiffany Lin (6):
>   [media] VPU: mediatek: support Mediatek VPU
>   arm64: dts: mediatek: Add node for Mediatek Video Processor Unit
>   dt-bindings: Add a binding for Mediatek Video Encoder
>   [media] vcodec: mediatek: Add Mediatek VP8 Video Encoder Driver
>   [media] vcodec: mediatek: Add Mediatek H264 Video Encoder Driver
>   arm64: dts: mediatek: Add Video Encoder for MT8173
> 
>  .../devicetree/bindings/media/mediatek-vcodec.txt  |   59 +
>  .../devicetree/bindings/media/mediatek-vpu.txt     |   31 +
>  arch/arm64/boot/dts/mediatek/mt8173.dtsi           |   62 +
>  drivers/media/platform/Kconfig                     |   20 +
>  drivers/media/platform/Makefile                    |    4 +
>  drivers/media/platform/mtk-vcodec/Makefile         |   11 +
>  .../media/platform/mtk-vcodec/h264_enc/Makefile    |    6 +
>  .../platform/mtk-vcodec/h264_enc/venc_h264_if.c    |  540 ++++++++
>  .../platform/mtk-vcodec/h264_enc/venc_h264_if.h    |  165 +++
>  .../platform/mtk-vcodec/h264_enc/venc_h264_vpu.c   |  309 +++++
>  .../platform/mtk-vcodec/h264_enc/venc_h264_vpu.h   |   30 +
>  drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h |  388 ++++++
>  drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c | 1380 ++++++++++++++++++++
>  drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h |   46 +
>  .../media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c |  476 +++++++
>  .../media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c  |  132 ++
>  .../media/platform/mtk-vcodec/mtk_vcodec_intr.c    |  102 ++
>  .../media/platform/mtk-vcodec/mtk_vcodec_intr.h    |   29 +
>  drivers/media/platform/mtk-vcodec/mtk_vcodec_pm.h  |   26 +
>  .../media/platform/mtk-vcodec/mtk_vcodec_util.c    |  106 ++
>  .../media/platform/mtk-vcodec/mtk_vcodec_util.h    |   85 ++
>  drivers/media/platform/mtk-vcodec/venc_drv_base.h  |   62 +
>  drivers/media/platform/mtk-vcodec/venc_drv_if.c    |  106 ++
>  drivers/media/platform/mtk-vcodec/venc_drv_if.h    |  175 +++
>  drivers/media/platform/mtk-vcodec/venc_ipi_msg.h   |  212 +++
>  drivers/media/platform/mtk-vcodec/vp8_enc/Makefile |    6 +
>  .../platform/mtk-vcodec/vp8_enc/venc_vp8_if.c      |  422 ++++++
>  .../platform/mtk-vcodec/vp8_enc/venc_vp8_if.h      |  149 +++
>  .../platform/mtk-vcodec/vp8_enc/venc_vp8_vpu.c     |  240 ++++
>  .../platform/mtk-vcodec/vp8_enc/venc_vp8_vpu.h     |   28 +
>  drivers/media/platform/mtk-vpu/Makefile            |    1 +
>  drivers/media/platform/mtk-vpu/mtk_vpu.c           |  994 ++++++++++++++
>  drivers/media/platform/mtk-vpu/mtk_vpu.h           |  167 +++
>  include/uapi/linux/v4l2-controls.h                 |    4 +
>  34 files changed, 6573 insertions(+)
>  create mode 100644 Documentation/devicetree/bindings/media/mediatek-vcodec.txt
>  create mode 100644 Documentation/devicetree/bindings/media/mediatek-vpu.txt
>  create mode 100644 drivers/media/platform/mtk-vcodec/Makefile
>  create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/Makefile
>  create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.c
>  create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.h
>  create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.c
>  create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.h
>  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
>  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
>  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h
>  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
>  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
>  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.c
>  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.h
>  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_pm.h
>  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c
>  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
>  create mode 100644 drivers/media/platform/mtk-vcodec/venc_drv_base.h
>  create mode 100644 drivers/media/platform/mtk-vcodec/venc_drv_if.c
>  create mode 100644 drivers/media/platform/mtk-vcodec/venc_drv_if.h
>  create mode 100644 drivers/media/platform/mtk-vcodec/venc_ipi_msg.h
>  create mode 100644 drivers/media/platform/mtk-vcodec/vp8_enc/Makefile
>  create mode 100644 drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_if.c
>  create mode 100644 drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_if.h
>  create mode 100644 drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_vpu.c
>  create mode 100644 drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_vpu.h
>  create mode 100644 drivers/media/platform/mtk-vpu/Makefile
>  create mode 100644 drivers/media/platform/mtk-vpu/mtk_vpu.c
>  create mode 100644 drivers/media/platform/mtk-vpu/mtk_vpu.h
>  mode change 100644 => 100755 include/uapi/linux/v4l2-controls.h
> 

^ permalink raw reply	[flat|nested] 102+ messages in thread

* Re: [PATCH v4 2/8] [media] VPU: mediatek: support Mediatek VPU
  2016-02-04 11:34     ` Tiffany Lin
@ 2016-02-15 13:59       ` Wu-Cheng Li (李務誠)
  -1 siblings, 0 replies; 102+ messages in thread
From: Wu-Cheng Li (李務誠) @ 2016-02-15 13:59 UTC (permalink / raw)
  To: Tiffany Lin
  Cc: Hans Verkuil, daniel.thompson, Rob Herring,
	Mauro Carvalho Chehab, Matthias Brugger, Daniel Kurtz,
	Pawel Osciak, Eddie Huang, Yingjoe Chen, devicetree,
	linux-kernel, linux-arm-kernel, linux-media, linux-mediatek,
	Lin PoChun, Andrew-CT Chen

On Thu, Feb 4, 2016 at 7:34 PM, Tiffany Lin <tiffany.lin@mediatek.com> wrote:
> The VPU driver for hw video codec embedded in Mediatek's MT8173 SOCs.
> It is able to handle video decoding/encoding of in a range of formats.
> The driver provides with VPU firmware download, memory management and
> the communication interface between CPU and VPU.
> For VPU initialization, it will create virtual memory for CPU access and
> IOMMU address for vcodec hw device access. When a decode/encode instance
> opens a device node, vpu driver will download vpu firmware to the device.
> A decode/encode instant will decode/encode a frame using VPU
> interface to interrupt vpu to handle decoding/encoding jobs.
>
> Signed-off-by: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
> Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
> ---
>  drivers/media/platform/Kconfig           |    9 +
>  drivers/media/platform/Makefile          |    2 +
>  drivers/media/platform/mtk-vpu/Makefile  |    1 +
>  drivers/media/platform/mtk-vpu/mtk_vpu.c |  994 ++++++++++++++++++++++++++++++
>  drivers/media/platform/mtk-vpu/mtk_vpu.h |  167 +++++
>  5 files changed, 1173 insertions(+)
>  create mode 100644 drivers/media/platform/mtk-vpu/Makefile
>  create mode 100644 drivers/media/platform/mtk-vpu/mtk_vpu.c
>  create mode 100644 drivers/media/platform/mtk-vpu/mtk_vpu.h
>
> diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
> index ccbc974..ba812d6 100644
> --- a/drivers/media/platform/Kconfig
> +++ b/drivers/media/platform/Kconfig
> @@ -148,6 +148,15 @@ config VIDEO_CODA
>            Coda is a range of video codec IPs that supports
>            H.264, MPEG-4, and other video formats.
>
> +config VIDEO_MEDIATEK_VPU
> +       tristate "Mediatek Video Processor Unit"
> +       depends on VIDEO_DEV && VIDEO_V4L2 && ARCH_MEDIATEK
> +       ---help---
> +           This driver provides downloading VPU firmware and
> +           communicating with VPU. This driver for hw video
> +           codec embedded in new Mediatek's SOCs. It is able
> +           to handle video decoding/encoding in a range of formats.
> +
>  config VIDEO_MEM2MEM_DEINTERLACE
>         tristate "Deinterlace support"
>         depends on VIDEO_DEV && VIDEO_V4L2 && DMA_ENGINE
> diff --git a/drivers/media/platform/Makefile b/drivers/media/platform/Makefile
> index efa0295..e5b19c6 100644
> --- a/drivers/media/platform/Makefile
> +++ b/drivers/media/platform/Makefile
> @@ -55,3 +55,5 @@ obj-$(CONFIG_VIDEO_AM437X_VPFE)               += am437x/
>  obj-$(CONFIG_VIDEO_XILINX)             += xilinx/
>
>  ccflags-y += -I$(srctree)/drivers/media/i2c
> +
> +obj-$(CONFIG_VIDEO_MEDIATEK_VPU)       += mtk-vpu/
> diff --git a/drivers/media/platform/mtk-vpu/Makefile b/drivers/media/platform/mtk-vpu/Makefile
> new file mode 100644
> index 0000000..d890a66
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vpu/Makefile
> @@ -0,0 +1 @@
> +obj-y += mtk_vpu.o
> diff --git a/drivers/media/platform/mtk-vpu/mtk_vpu.c b/drivers/media/platform/mtk-vpu/mtk_vpu.c
> new file mode 100644
> index 0000000..f54fd89
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vpu/mtk_vpu.c
> @@ -0,0 +1,994 @@
> +/*
> +* Copyright (c) 2015 MediaTek Inc.
> +* Author: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
> +*
> +* This program is free software; you can redistribute it and/or modify
> +* it under the terms of the GNU General Public License version 2 as
> +* published by the Free Software Foundation.
> +*
> +* This program is distributed in the hope that it will be useful,
> +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +* GNU General Public License for more details.
> +*/
> +#include <linux/bootmem.h>
> +#include <linux/clk.h>
> +#include <linux/debugfs.h>
> +#include <linux/firmware.h>
> +#include <linux/interrupt.h>
> +#include <linux/iommu.h>
> +#include <linux/module.h>
> +#include <linux/of_address.h>
> +#include <linux/of_irq.h>
> +#include <linux/of_platform.h>
> +#include <linux/of_reserved_mem.h>
> +#include <linux/sched.h>
> +#include <linux/sizes.h>
> +
> +#include "mtk_vpu.h"
> +
> +/**
> + * VPU (video processor unit) is a tiny processor controlling video hardware
> + * related to video codec, scaling and color format converting.
> + * VPU interfaces with other blocks by share memory and interrupt.
> + **/
> +
> +#define INIT_TIMEOUT_MS                2000U
> +#define IPI_TIMEOUT_MS         2000U
> +#define VPU_FW_VER_LEN         16
> +
> +/* maximum program/data TCM (Tightly-Coupled Memory) size */
> +#define VPU_PTCM_SIZE          (96 * SZ_1K)
> +#define VPU_DTCM_SIZE          (32 * SZ_1K)
> +/* the offset to get data tcm address */
> +#define VPU_DTCM_OFFSET                0x18000UL
> +/* daynamic allocated maximum extended memory size */
> +#define VPU_EXT_P_SIZE         SZ_1M
> +#define VPU_EXT_D_SIZE         SZ_4M
> +/* maximum binary firmware size */
> +#define VPU_P_FW_SIZE          (VPU_PTCM_SIZE + VPU_EXT_P_SIZE)
> +#define VPU_D_FW_SIZE          (VPU_DTCM_SIZE + VPU_EXT_D_SIZE)
> +/* the size of share buffer between Host and  VPU */
> +#define SHARE_BUF_SIZE         48
> +
> +/* binary firmware name */
> +#define VPU_P_FW               "vpu_p.bin"
> +#define VPU_D_FW               "vpu_d.bin"
> +
> +#define VPU_RESET              0x0
> +#define VPU_TCM_CFG            0x0008
> +#define VPU_PMEM_EXT0_ADDR     0x000C
> +#define VPU_PMEM_EXT1_ADDR     0x0010
> +#define VPU_TO_HOST            0x001C
> +#define VPU_DMEM_EXT0_ADDR     0x0014
> +#define VPU_DMEM_EXT1_ADDR     0x0018
> +#define HOST_TO_VPU            0x0024
> +#define VPU_PC_REG             0x0060
> +#define VPU_WDT_REG            0x0084
> +
> +/* vpu inter-processor communication interrupt */
> +#define VPU_IPC_INT            BIT(8)
> +
> +/**
> + * enum vpu_fw_type - VPU firmware type
> + *
> + * @P_FW: program firmware
> + * @D_FW: data firmware
> + *
> + */
> +enum vpu_fw_type {
> +       P_FW,
> +       D_FW,
> +};
> +
> +/**
> + * struct vpu_mem - VPU extended program/data memory information
> + *
> + * @va:                the kernel virtual memory address of VPU extended memory
> + * @pa:                the physical memory address of VPU extended memory
This should be device memory address? If yes, s/physical/device/ and
s/pa/dma_addr/.
> + *
> + */
> +struct vpu_mem {
> +       void *va;
> +       dma_addr_t pa;
> +};
> +
> +/**
> + * struct vpu_regs - VPU TCM and configuration registers
> + *
> + * @tcm:       the register for VPU Tightly-Coupled Memory
> + * @cfg:       the register for VPU configuration
> + * @irq:       the irq number for VPU interrupt
> + */
> +struct vpu_regs {
> +       void __iomem *tcm;
> +       void __iomem *cfg;
> +       int irq;
> +};
> +
> +/**
> + * struct vpu_wdt_handler - VPU watchdog reset handler
> + *
> + * @reset_func:        reset handler
> + * @priv:      private data
> + */
> +struct vpu_wdt_handler {
> +       void (*reset_func)(void *);
> +       void *priv;
> +};
> +
> +/**
> + * struct vpu_wdt - VPU watchdog workqueue
> + *
> + * @handler:   VPU watchdog reset handler
> + * @ws:                workstruct for VPU watchdog
> + * @wq:                workqueue for VPU watchdog
> + */
> +struct vpu_wdt {
> +       struct vpu_wdt_handler handler[VPU_RST_MAX];
> +       struct work_struct ws;
> +       struct workqueue_struct *wq;
> +};
> +
> +/**
> + * struct vpu_run - VPU initialization status
> + *
> + * @signaled:          the signal of vpu initialization completed
> + * @fw_ver:            VPU firmware version
> + * @enc_capability:    encoder capability
We should document the meaning of the bit used. Since no bit is used
by the codec driver, document this is not used for now and the value
is reserved for future use.
> + * @wq:                        wait queue for VPU initialization status
> + */
> +struct vpu_run {
> +       u32 signaled;
> +       char fw_ver[VPU_FW_VER_LEN];
> +       unsigned int    enc_capability;
> +       wait_queue_head_t wq;
> +};
> +
> +/**
> + * struct vpu_ipi_desc - VPU IPI descriptor
> + *
> + * @handler:   IPI handler
> + * @name:      the name of IPI handler
> + * @priv:      the private data of IPI handler
> + */
> +struct vpu_ipi_desc {
> +       ipi_handler_t handler;
> +       const char *name;
> +       void *priv;
> +};
> +
> +/**
> + * struct share_obj - DTCM (Data Tightly-Coupled Memory) buffer shared with
> + *                   AP and VPU
> + *
> + * @id:                IPI id
> + * @len:       share buffer length
> + * @share_buf: share buffer data
> + */
> +struct share_obj {
> +       s32 id;
> +       u32 len;
> +       unsigned char share_buf[SHARE_BUF_SIZE];
> +};
> +
> +/**
> + * struct mtk_vpu - vpu driver data
> + * @extmem:            VPU extended memory information
> + * @reg:               VPU TCM and configuration registers
> + * @run:               VPU initialization status
> + * @ipi_desc:          VPU IPI descriptor
> + * @recv_buf:          VPU DTCM share buffer for receiving. The
> + *                     receive buffer is only accessed in interrupt context.
> + * @send_buf:          VPU DTCM share buffer for sending
> + * @dev:               VPU struct device
> + * @clk:               VPU clock on/off
> + * @enable_4GB:                VPU 4GB mode on/off
> + * @vpu_mutex:         protect mtk_vpu (except recv_buf) and ensure only
> + *                     one client to use VPU service at a time. For example,
> + *                     suppose a client is using VPU to decode VP8.
> + *                     If the other client wants to encode VP8,
> + *                     it has to wait until VP8 decode completes.
> + * @wdt_refcnt         WDT reference count to make sure the watchdog can be
> + *                     disabled if no other client is using VPU service
> + * @ipi_ack_signaled:  The ACKs for registered IPI function sending
> + *                     interrupt to VPU
s/ipi_ack_signaled/ipi_id_ack/. Move to after |ack_wq| to be
consistent with the order of variable declaration.
> + * @ack_wq:            The wait queue for each codec and mdp. When sleeping
> + *                     processes wake up, they will check the condition
> + *                     "ipi_ack_signaled" to run the corresponding action or
> + *                     go back to sleep.
> + *
> + */
> +struct mtk_vpu {
> +       struct vpu_mem extmem[2];
> +       struct vpu_regs reg;
> +       struct vpu_run run;
> +       struct vpu_wdt wdt;
> +       struct vpu_ipi_desc ipi_desc[IPI_MAX];
> +       struct share_obj *recv_buf;
> +       struct share_obj *send_buf;
> +       struct device *dev;
> +       struct clk *clk;
> +       bool enable_4GB;
> +       struct mutex vpu_mutex; /* for protecting vpu data data structure */
Remove the comment here. It should be documented in function comment above.
> +       atomic_t wdt_refcnt;
> +       wait_queue_head_t ack_wq;
> +       bool ipi_id_ack[IPI_MAX];
> +};
> +
> +static inline void vpu_cfg_writel(struct mtk_vpu *vpu, u32 val, u32 offset)
> +{
> +       writel(val, vpu->reg.cfg + offset);
> +}
> +
> +static inline u32 vpu_cfg_readl(struct mtk_vpu *vpu, u32 offset)
> +{
> +       return readl(vpu->reg.cfg + offset);
> +}
> +
> +static inline bool vpu_running(struct mtk_vpu *vpu)
> +{
> +       return vpu_cfg_readl(vpu, VPU_RESET) & BIT(0);
> +}
> +
> +void vpu_clock_disable(struct mtk_vpu *vpu)
> +{
> +       /* Disable VPU watchdog */
> +       if (atomic_dec_and_test(&vpu->wdt_refcnt))
Checking wdt_refcnt and doing vpu_cfg_writel should be done
atomically. For example, if vpu_clock_enable is called after
atomic_dec_and_test(&vpu->wdt_refcnt) and before vpu_cfg_writel(...),
the state of watchdog could be wrong. Adding a spinlock to protect
wdt_refcnt and vpu_cfg_writel together. Then wdt_refcnt doesn't need
to be atomic_t.
> +               vpu_cfg_writel(vpu,
> +                              vpu_cfg_readl(vpu, VPU_WDT_REG) & ~(1L << 31),
> +                              VPU_WDT_REG);
> +
> +       clk_disable(vpu->clk);
> +}
> +
> +int vpu_clock_enable(struct mtk_vpu *vpu)
> +{
> +       int ret;
> +
> +       ret = clk_enable(vpu->clk);
> +       if (ret)
> +               return ret;
> +       /* Enable VPU watchdog */
> +       if (!atomic_read(&vpu->wdt_refcnt))
Do we need to enable the watchdog other than vpu_ipi_send? The only
place that AP waits for VPU is in vpu_ipi_send. Right? If yes, can we
just enable and disable the watchdog in vpu_ipi_send?
> +               vpu_cfg_writel(vpu,
> +                              vpu_cfg_readl(vpu, VPU_WDT_REG) | (1L << 31),
> +                              VPU_WDT_REG);
> +
> +       atomic_inc(&vpu->wdt_refcnt);
Same above. atomic_read, vpu_cfg_writel, and atomic_inc should be done
atomically.
> +
> +       return ret;
> +}
> +
> +int vpu_ipi_register(struct platform_device *pdev,
> +                    enum ipi_id id, ipi_handler_t handler,
> +                    const char *name, void *priv)
> +{
> +       struct mtk_vpu *vpu = platform_get_drvdata(pdev);
> +       struct vpu_ipi_desc *ipi_desc;
> +
> +       if (!vpu) {
> +               dev_err(&pdev->dev, "vpu device in not ready\n");
> +               return -EPROBE_DEFER;
> +       }
> +
> +       if (id < IPI_MAX && handler) {
Check if id >= 0 because type of enum is implementation dependent.
> +               ipi_desc = vpu->ipi_desc;
> +               ipi_desc[id].name = name;
> +               ipi_desc[id].handler = handler;
> +               ipi_desc[id].priv = priv;
> +               return 0;
> +       }
> +
> +       dev_err(&pdev->dev, "register vpu ipi with invalid arguments\n");
print id
> +       return -EINVAL;
> +}
> +
> +int vpu_ipi_send(struct platform_device *pdev,
> +                enum ipi_id id, void *buf,
> +                unsigned int len)
> +{
> +       struct mtk_vpu *vpu = platform_get_drvdata(pdev);
> +       struct share_obj *send_obj = vpu->send_buf;
> +       unsigned long timeout;
> +       int ret = 0;
> +
> +       if (id <= IPI_VPU_INIT || id >= IPI_MAX ||
> +           len > sizeof(send_obj->share_buf) || !buf) {
> +               dev_err(vpu->dev, "failed to send ipi message\n");
> +               return -EINVAL;
> +       }
> +
> +       ret = vpu_clock_enable(vpu);
> +       if (ret) {
> +               dev_err(vpu->dev, "failed to enable vpu clock\n");
> +               return ret;
> +       }
> +       if (!vpu_running(vpu)) {
> +               dev_err(vpu->dev, "vpu_ipi_send: VPU is not running\n");
> +               ret = -EINVAL;
> +               goto clock_disable;
> +       }
> +
> +       mutex_lock(&vpu->vpu_mutex);
> +
> +        /* Wait until VPU receives the last command */
> +       timeout = jiffies + msecs_to_jiffies(IPI_TIMEOUT_MS);
> +       do {
> +               if (time_after(jiffies, timeout)) {
> +                       dev_err(vpu->dev, "vpu_ipi_send: IPI timeout!\n");
> +                       ret = -EIO;
> +                       goto mut_unlock;
> +               }
> +       } while (vpu_cfg_readl(vpu, HOST_TO_VPU));
> +
> +       memcpy((void *)send_obj->share_buf, buf, len);
> +       send_obj->len = len;
> +       send_obj->id = id;
> +
> +       vpu->ipi_id_ack[id] = false;
> +       /* send the command to VPU */
> +       vpu_cfg_writel(vpu, 0x1, HOST_TO_VPU);
> +
> +       mutex_unlock(&vpu->vpu_mutex);
> +
> +       /* wait for VPU's ACK */
> +       timeout = msecs_to_jiffies(IPI_TIMEOUT_MS);
> +       ret = wait_event_interruptible_timeout(vpu->ack_wq,
> +                                              vpu->ipi_id_ack[id], timeout);
> +       vpu->ipi_id_ack[id] = false;
> +       if (ret == 0) {
> +               dev_err(vpu->dev, "vpu ipi %d ack time out !", id);
> +               ret = -EIO;
> +               goto clock_disable;
> +       } else if (-ERESTARTSYS == ret) {
> +               dev_err(vpu->dev, "vpu ipi %d ack wait interrupted by a signal",
> +                       id);
> +               ret = -ERESTARTSYS;
> +               goto clock_disable;
> +       }
> +       vpu_clock_disable(vpu);
> +
> +       return 0;
> +
> +mut_unlock:
> +       vpu->ipi_id_ack[id] = false;
I don't see why we need to set it to false here. Remove if not needed.
> +       mutex_unlock(&vpu->vpu_mutex);
> +clock_disable:
> +       vpu_clock_disable(vpu);
> +
> +       return ret;
> +}
> +
> +static void vpu_wdt_reset_func(struct work_struct *ws)
> +{
> +       struct vpu_wdt *wdt = container_of(ws, struct vpu_wdt, ws);
> +       struct mtk_vpu *vpu = container_of(wdt, struct mtk_vpu, wdt);
> +       struct vpu_wdt_handler *handler = wdt->handler;
> +       int index, ret;
> +
> +       dev_info(vpu->dev, "vpu reset\n");
> +       mutex_lock(&vpu->vpu_mutex);
> +       ret = vpu_clock_enable(vpu);
> +       if (ret) {
This is missing mutex_unlock. Move mutex_lock right before
vpu_cfg_writel(vpu, 0x0, VPU_RESET);. So we don't need to unlock when
vpu_clock_enable fails.
> +               dev_err(vpu->dev, "[VPU] wdt enables clock failed %d\n", ret);
> +               return;
> +       }
> +       vpu_cfg_writel(vpu, 0x0, VPU_RESET);
> +       vpu_clock_disable(vpu);
> +       mutex_unlock(&vpu->vpu_mutex);
> +
> +       for (index = 0; index < VPU_RST_MAX; index++) {
> +               if (handler[index].reset_func) {
> +                       handler[index].reset_func(handler[index].priv);
> +                       dev_dbg(vpu->dev, "wdt handler func %d\n", index);
> +               }
> +       }
> +}
> +
> +int vpu_wdt_reg_handler(struct platform_device *pdev,
> +                       void wdt_reset(void *),
> +                       void *priv, enum rst_id id)
> +{
> +       struct mtk_vpu *vpu = platform_get_drvdata(pdev);
> +       struct vpu_wdt_handler *handler = vpu->wdt.handler;
> +
> +       if (!vpu) {
> +               dev_err(vpu->dev, "vpu device in not ready\n");
> +               return -EPROBE_DEFER;
> +       }
> +
> +       if (id < VPU_RST_MAX && wdt_reset != NULL) {
Check id >= 0
> +               dev_dbg(vpu->dev, "wdt register id %d\n", id);
> +               mutex_lock(&vpu->vpu_mutex);
> +               handler[id].reset_func = wdt_reset;
> +               handler[id].priv = priv;
> +               mutex_unlock(&vpu->vpu_mutex);
> +               return 0;
> +       }
> +
> +       dev_err(vpu->dev, "register vpu wdt handler failed\n");
> +       return -EINVAL;
> +}
> +
> +unsigned int vpu_get_venc_hw_capa(struct platform_device *pdev)
> +{
> +       struct mtk_vpu *vpu = platform_get_drvdata(pdev);
> +
> +       return vpu->run.enc_capability;
> +}
> +
> +void *vpu_mapping_dm_addr(struct platform_device *pdev,
> +                         u32 dtcm_dmem_addr)
> +{
> +       struct mtk_vpu *vpu = platform_get_drvdata(pdev);
> +
> +       if (!dtcm_dmem_addr ||
> +           (dtcm_dmem_addr > (VPU_DTCM_SIZE + VPU_EXT_D_SIZE))) {
> +               dev_err(vpu->dev, "invalid virtual data memory address\n");
> +               return ERR_PTR(-EINVAL);
> +       }
> +
> +       if (dtcm_dmem_addr < VPU_DTCM_SIZE)
> +               return dtcm_dmem_addr + vpu->reg.tcm + VPU_DTCM_OFFSET;
> +
> +       return vpu->extmem[D_FW].va + (dtcm_dmem_addr - VPU_DTCM_SIZE);
> +}
> +
> +struct platform_device *vpu_get_plat_device(struct platform_device *pdev)
> +{
> +       struct device *dev = &pdev->dev;
> +       struct device_node *vpu_node;
> +       struct platform_device *vpu_pdev;
> +
> +       vpu_node = of_parse_phandle(dev->of_node, "mediatek,vpu", 0);
> +       if (!vpu_node) {
> +               dev_err(dev, "can't get vpu node\n");
> +               return NULL;
> +       }
> +
> +       vpu_pdev = of_find_device_by_node(vpu_node);
> +       if (WARN_ON(!vpu_pdev)) {
> +               dev_err(dev, "vpu pdev failed\n");
> +               of_node_put(vpu_node);
> +               return NULL;
> +       }
> +
> +       return vpu_pdev;
> +}
> +
> +/* load vpu program/data memory */
> +static int load_requested_vpu(struct mtk_vpu *vpu,
> +                             const struct firmware *vpu_fw,
> +                             u8 fw_type)
> +{
> +       size_t tcm_size = fw_type ? VPU_DTCM_SIZE : VPU_PTCM_SIZE;
> +       size_t fw_size = fw_type ? VPU_D_FW_SIZE : VPU_P_FW_SIZE;
> +       char *fw_name = fw_type ? VPU_D_FW : VPU_P_FW;
> +       size_t dl_size = 0;
> +       size_t extra_fw_size = 0;
> +       void *dest;
> +       int ret;
> +
> +       ret = request_firmware(&vpu_fw, fw_name, vpu->dev);
> +       if (ret < 0) {
> +               dev_err(vpu->dev, "Failed to load %s, %d\n", fw_name, ret);
> +               return ret;
> +       }
> +       dl_size = vpu_fw->size;
> +       if (dl_size > fw_size) {
> +               dev_err(vpu->dev, "fw %s size %zu is abnormal\n", fw_name,
> +                       dl_size);
> +               release_firmware(vpu_fw);
> +               return  -EFBIG;
> +       }
> +       dev_dbg(vpu->dev, "Downloaded fw %s size: %zu.\n",
> +               fw_name,
> +               dl_size);
> +       /* reset VPU */
> +       vpu_cfg_writel(vpu, 0x0, VPU_RESET);
> +
> +       /* handle extended firmware size */
> +       if (dl_size > tcm_size) {
> +               dev_dbg(vpu->dev, "fw size %lx > limited fw size %lx\n",
> +                       dl_size, tcm_size);
> +               extra_fw_size = dl_size - tcm_size;
> +               dev_dbg(vpu->dev, "extra_fw_size %lx\n", extra_fw_size);
> +               dl_size = tcm_size;
> +       }
> +       dest = vpu->reg.tcm;
> +       if (fw_type == D_FW)
> +               dest += VPU_DTCM_OFFSET;
> +       memcpy(dest, vpu_fw->data, dl_size);
> +       /* download to extended memory if need */
> +       if (extra_fw_size > 0) {
> +               dest = vpu->extmem[fw_type].va;
> +               dev_dbg(vpu->dev, "download extended memory type %x\n",
> +                       fw_type);
> +               memcpy(dest, vpu_fw->data + tcm_size, extra_fw_size);
> +       }
> +
> +       release_firmware(vpu_fw);
> +
> +       return 0;
> +}
> +
> +int vpu_load_firmware(struct platform_device *pdev)
> +{
> +       struct mtk_vpu *vpu = platform_get_drvdata(pdev);
> +       struct device *dev = &pdev->dev;
> +       struct vpu_run *run = &vpu->run;
> +       const struct firmware *vpu_fw;
> +       int ret;
> +
> +       if (!pdev) {
> +               dev_err(dev, "VPU platform device is invalid\n");
> +               return -EINVAL;
> +       }
> +
> +       mutex_lock(&vpu->vpu_mutex);
> +
> +       ret = vpu_clock_enable(vpu);
> +       if (ret) {
> +               dev_err(dev, "enable clock failed %d\n", ret);
> +               goto OUT_LOAD_FW;
> +       }
> +
> +       if (vpu_running(vpu)) {
> +               vpu_clock_disable(vpu);
> +               mutex_unlock(&vpu->vpu_mutex);
> +               dev_warn(dev, "vpu is running already\n");
> +               return 0;
> +       }
> +
> +       run->signaled = false;
> +       dev_dbg(vpu->dev, "firmware request\n");
> +       /* Downloading program firmware to device*/
> +       ret = load_requested_vpu(vpu, vpu_fw, P_FW);
> +       if (ret < 0) {
> +               dev_err(dev, "Failed to request %s, %d\n", VPU_P_FW, ret);
> +               goto OUT_LOAD_FW;
> +       }
> +
> +       /* Downloading data firmware to device */
> +       ret = load_requested_vpu(vpu, vpu_fw, D_FW);
> +       if (ret < 0) {
> +               dev_err(dev, "Failed to request %s, %d\n", VPU_D_FW, ret);
> +               goto OUT_LOAD_FW;
> +       }
> +
> +       /* boot up vpu */
> +       vpu_cfg_writel(vpu, 0x1, VPU_RESET);
> +
> +       ret = wait_event_interruptible_timeout(run->wq,
> +                                              run->signaled,
> +                                              msecs_to_jiffies(INIT_TIMEOUT_MS)
> +                                              );
> +       if (ret == 0) {
> +               ret = -ETIME;
> +               dev_err(dev, "wait vpu initialization timout!\n");
> +               goto OUT_LOAD_FW;
> +       } else if (-ERESTARTSYS == ret) {
> +               dev_err(dev, "wait vpu interrupted by a signal!\n");
> +               goto OUT_LOAD_FW;
> +       }
> +
> +       ret = 0;
> +       dev_info(dev, "vpu is ready. Fw version %s\n", run->fw_ver);
> +
> +OUT_LOAD_FW:
> +       vpu_clock_disable(vpu);
> +       mutex_unlock(&vpu->vpu_mutex);
> +
> +       return ret;
> +}
> +
> +int vpu_compare_version(struct platform_device *pdev,
> +                       const char *expected_version)
> +{
> +       struct mtk_vpu *vpu = platform_get_drvdata(pdev);
> +       int cur_major, cur_minor, cur_build, cur_rel, cur_ver_num;
> +       int major, minor, build, rel, ver_num;
> +       char *cur_version = vpu->run.fw_ver;
> +
> +       cur_ver_num = sscanf(cur_version, "%d.%d.%d-rc%d",
> +                            &cur_major, &cur_minor, &cur_build, &cur_rel);
> +       if (cur_ver_num < 3)
> +               return -1;
> +       ver_num = sscanf(expected_version, "%d.%d.%d-rc%d",
> +                        &major, &minor, &build, &rel);
> +       if (ver_num < 3)
> +               return -1;
> +
> +       if (cur_major < major)
> +               return -1;
> +       if (cur_major > major)
> +               return 1;
> +
> +       if (cur_minor < minor)
> +               return -1;
> +       if (cur_minor > minor)
> +               return 1;
> +
> +       if (cur_build < build)
> +               return -1;
> +       if (cur_build > build)
> +               return 1;
> +
> +       if (cur_ver_num < ver_num)
> +               return -1;
> +       if (cur_ver_num > ver_num)
> +               return 1;
> +
> +       if (ver_num > 3) {
> +               if (cur_rel < rel)
> +                       return -1;
> +               if (cur_rel > rel)
> +                       return 1;
> +       }
> +
> +       return 0;
> +}
> +
> +static void vpu_init_ipi_handler(void *data, unsigned int len, void *priv)
> +{
> +       struct mtk_vpu *vpu = (struct mtk_vpu *)priv;
> +       struct vpu_run *run = (struct vpu_run *)data;
> +
> +       vpu->run.signaled = run->signaled;
> +       strncpy(vpu->run.fw_ver, run->fw_ver, VPU_FW_VER_LEN);
> +       vpu->run.enc_capability = run->enc_capability;
> +       wake_up_interruptible(&vpu->run.wq);
> +}
> +
> +#ifdef CONFIG_DEBUG_FS
> +static int vpu_debug_open(struct inode *inode, struct file *file)
> +{
> +       file->private_data = inode->i_private;
> +       return 0;
> +}
> +
> +static ssize_t vpu_debug_read(struct file *file, char __user *user_buf,
> +                             size_t count, loff_t *ppos)
> +{
> +       char buf[256];
> +       unsigned int len;
> +       unsigned int running, pc, vpu_to_host, host_to_vpu, wdt;
> +       int ret;
> +       struct device *dev = file->private_data;
> +       struct mtk_vpu *vpu = dev_get_drvdata(dev);
> +
> +       ret = vpu_clock_enable(vpu);
> +       if (ret) {
> +               dev_err(vpu->dev, "[VPU] enable clock failed %d\n", ret);
> +               return 0;
> +       }
> +
> +       /* vpu register status */
> +       running = vpu_running(vpu);
> +       pc = vpu_cfg_readl(vpu, VPU_PC_REG);
> +       wdt = vpu_cfg_readl(vpu, VPU_WDT_REG);
> +       host_to_vpu = vpu_cfg_readl(vpu, HOST_TO_VPU);
> +       vpu_to_host = vpu_cfg_readl(vpu, VPU_TO_HOST);
> +       vpu_clock_disable(vpu);
> +
> +       if (running) {
> +               len = sprintf(buf, "VPU is running\n\n"
> +               "FW Version: %s\n"
> +               "PC: 0x%x\n"
> +               "WDT: 0x%x\n"
> +               "Host to VPU: 0x%x\n"
> +               "VPU to Host: 0x%x\n",
> +               vpu->run.fw_ver, pc, wdt,
> +               host_to_vpu, vpu_to_host);
> +       } else {
> +               len = sprintf(buf, "VPU not running\n");
> +       }
> +
> +       return simple_read_from_buffer(user_buf, count, ppos, buf, len);
> +}
> +
> +static const struct file_operations vpu_debug_fops = {
> +       .open = vpu_debug_open,
> +       .read = vpu_debug_read,
> +};
> +#endif /* CONFIG_DEBUG_FS */
> +
> +static void vpu_free_ext_mem(struct mtk_vpu *vpu, u8 fw_type)
> +{
> +       struct device *dev = vpu->dev;
> +       size_t fw_ext_size = fw_type ? VPU_EXT_D_SIZE : VPU_EXT_P_SIZE;
> +
> +       dma_free_coherent(dev, fw_ext_size, vpu->extmem[fw_type].va,
> +                         vpu->extmem[fw_type].pa);
> +}
> +
> +static int vpu_alloc_ext_mem(struct mtk_vpu *vpu, u32 fw_type)
> +{
> +       struct device *dev = vpu->dev;
> +       size_t fw_ext_size = fw_type ? VPU_EXT_D_SIZE : VPU_EXT_P_SIZE;
> +       u32 vpu_ext_mem0 = fw_type ? VPU_DMEM_EXT0_ADDR : VPU_PMEM_EXT0_ADDR;
> +       u32 vpu_ext_mem1 = fw_type ? VPU_DMEM_EXT1_ADDR : VPU_PMEM_EXT1_ADDR;
> +       u32 offset_4gb = vpu->enable_4GB ? 0x40000000 : 0;
> +
> +       vpu->extmem[fw_type].va = dma_alloc_coherent(dev,
> +                                              fw_ext_size,
> +                                              &vpu->extmem[fw_type].pa,
> +                                              GFP_KERNEL);
> +       if (!vpu->extmem[fw_type].va) {
> +               dev_err(dev, "Failed to allocate the extended program memory\n");
> +               return PTR_ERR(vpu->extmem[fw_type].va);
> +       }
> +
> +       /* Disable extend0. Enable extend1 */
> +       vpu_cfg_writel(vpu, 0x1, vpu_ext_mem0);
> +       vpu_cfg_writel(vpu, (vpu->extmem[fw_type].pa & 0xFFFFF000) + offset_4gb,
> +                      vpu_ext_mem1);
> +
> +       dev_info(dev, "%s extend memory phy=0x%llx virt=0x%p\n",
> +                fw_type ? "Data" : "Program",
> +                (unsigned long long)vpu->extmem[fw_type].pa,
> +                vpu->extmem[fw_type].va);
> +
> +       return 0;
> +}
> +
> +static void vpu_ipi_handler(struct mtk_vpu *vpu)
> +{
> +       struct share_obj *rcv_obj = vpu->recv_buf;
> +       struct vpu_ipi_desc *ipi_desc = vpu->ipi_desc;
> +
> +       if (rcv_obj->id < IPI_MAX && ipi_desc[rcv_obj->id].handler) {
check rcv_obj->id >= 0
> +               ipi_desc[rcv_obj->id].handler(rcv_obj->share_buf,
> +                                             rcv_obj->len,
> +                                             ipi_desc[rcv_obj->id].priv);
> +               if (rcv_obj->id > IPI_VPU_INIT) {
> +                       vpu->ipi_id_ack[rcv_obj->id] = true;
> +                       wake_up_interruptible(&vpu->ack_wq);
> +               }
> +       } else {
> +               dev_err(vpu->dev, "No such ipi id = %d\n", rcv_obj->id);
> +       }
> +}
> +
> +static int vpu_ipi_init(struct mtk_vpu *vpu)
> +{
> +       /* Disable VPU to host interrupt */
> +       vpu_cfg_writel(vpu, 0x0, VPU_TO_HOST);
> +
> +       /* shared buffer initialization */
> +       vpu->recv_buf = (struct share_obj *)(vpu->reg.tcm + VPU_DTCM_OFFSET);
> +       vpu->send_buf = vpu->recv_buf + 1;
> +       memset(vpu->recv_buf, 0, sizeof(struct share_obj));
> +       memset(vpu->send_buf, 0, sizeof(struct share_obj));
> +       mutex_init(&vpu->vpu_mutex);
> +
> +       return 0;
> +}
> +
> +static irqreturn_t vpu_irq_handler(int irq, void *priv)
> +{
> +       struct mtk_vpu *vpu = priv;
> +       u32 vpu_to_host;
> +       int ret;
> +
> +       /*
> +        * Clock should have been enabled already.
> +        * Enable again in case vpu_ipi_send times out
> +        * and has disabled the clock.
> +        */
> +       ret = clk_enable(vpu->clk);
> +       if (ret) {
> +               dev_err(vpu->dev, "[VPU] enable clock failed %d\n", ret);
> +               return IRQ_NONE;
> +       }
> +       vpu_to_host = vpu_cfg_readl(vpu, VPU_TO_HOST);
> +       if (vpu_to_host & VPU_IPC_INT) {
> +               vpu_ipi_handler(vpu);
> +       } else {
> +               dev_err(vpu->dev, "vpu watchdog timeout! 0x%x", vpu_to_host);
> +               if (vpu->wdt.wq)
Remove. mtk_vpu_probe makes sure wdt.wq is not NULL.
> +                       queue_work(vpu->wdt.wq, &vpu->wdt.ws);
> +       }
> +
> +       /* VPU won't send another interrupt until we set VPU_TO_HOST to 0. */
> +       vpu_cfg_writel(vpu, 0x0, VPU_TO_HOST);
> +       clk_disable(vpu->clk);
> +
> +       return IRQ_HANDLED;
> +}
> +
> +#ifdef CONFIG_DEBUG_FS
> +static struct dentry *vpu_debugfs;
> +#endif
> +static int mtk_vpu_probe(struct platform_device *pdev)
> +{
> +       struct mtk_vpu *vpu;
> +       struct device *dev;
> +       struct resource *res;
> +       int ret = 0;
> +
> +       dev_dbg(&pdev->dev, "initialization\n");
> +
> +       dev = &pdev->dev;
> +       vpu = devm_kzalloc(dev, sizeof(*vpu), GFP_KERNEL);
> +       if (!vpu)
> +               return -ENOMEM;
> +
> +       vpu->dev = &pdev->dev;
> +       res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "tcm");
> +       vpu->reg.tcm = devm_ioremap_resource(dev, res);
> +       if (IS_ERR(vpu->reg.tcm)) {
> +               dev_err(dev, "devm_ioremap_resource vpu tcm failed.\n");
> +               return PTR_ERR(vpu->reg.tcm);
> +       }
> +
> +       res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "cfg_reg");
> +       vpu->reg.cfg = devm_ioremap_resource(dev, res);
> +       if (IS_ERR(vpu->reg.cfg)) {
> +               dev_err(dev, "devm_ioremap_resource vpu cfg failed.\n");
> +               return PTR_ERR(vpu->reg.cfg);
> +       }
> +
> +       /* Get VPU clock */
> +       vpu->clk = devm_clk_get(dev, "main");
> +       if (!vpu->clk) {
> +               dev_err(dev, "get vpu clock failed\n");
> +               return -EINVAL;
> +       }
> +
> +       platform_set_drvdata(pdev, vpu);
> +
> +       ret = clk_prepare(vpu->clk);
> +       if (ret) {
> +               dev_err(dev, "prepare vpu clock failed\n");
> +               return ret;
> +       }
> +
> +       /* VPU watchdog */
> +       vpu->wdt.wq = create_singlethread_workqueue("vpu_wdt");
> +       if (!vpu->wdt.wq) {
> +               dev_err(dev, "initialize wdt workqueue failed\n");
> +               return -ENOMEM;
> +       }
> +       INIT_WORK(&vpu->wdt.ws, vpu_wdt_reset_func);
> +
> +       ret = vpu_clock_enable(vpu);
> +       if (ret) {
> +               dev_err(dev, "enable vpu clock failed\n");
> +               goto workqueue_destroy;
> +       }
> +
> +       dev_dbg(dev, "vpu ipi init\n");
> +       ret = vpu_ipi_init(vpu);
> +       if (ret) {
> +               dev_err(dev, "Failed to init ipi\n");
> +               goto disable_vpu_clk;
> +       }
> +
> +       /* register vpu initialization IPI */
> +       ret = vpu_ipi_register(pdev, IPI_VPU_INIT, vpu_init_ipi_handler,
> +                              "vpu_init", vpu);
> +       if (ret) {
> +               dev_err(dev, "Failed to register IPI_VPU_INIT\n");
> +               goto vpu_mutex_destroy;
> +       }
> +
> +#ifdef CONFIG_DEBUG_FS
> +       vpu_debugfs = debugfs_create_file("mtk_vpu", S_IRUGO, NULL, (void *)dev,
> +                                         &vpu_debug_fops);
> +       if (!vpu_debugfs) {
> +               ret = -ENOMEM;
> +               goto cleanup_ipi;
> +       }
> +#endif
> +
> +       /* Set PTCM to 96K and DTCM to 32K */
> +       vpu_cfg_writel(vpu, 0x2, VPU_TCM_CFG);
> +
> +       vpu->enable_4GB = !!(max_pfn > (0xffffffffUL >> PAGE_SHIFT));
> +       dev_dbg(dev, "4GB mode %u\n", vpu->enable_4GB);
> +
> +       if (vpu->enable_4GB) {
> +               ret = of_reserved_mem_device_init(dev);
> +               if (ret)
> +                       dev_info(dev, "init reserved memory failed\n");
> +                       /* continue to use dynamic allocation if failed */
> +       }
> +
> +       ret = vpu_alloc_ext_mem(vpu, D_FW);
> +       if (ret) {
> +               dev_err(dev, "Allocate DM failed\n");
> +               goto remove_debugfs;
> +       }
> +
> +       ret = vpu_alloc_ext_mem(vpu, P_FW);
> +       if (ret) {
> +               dev_err(dev, "Allocate PM failed\n");
> +               goto free_d_mem;
> +       }
> +
> +       init_waitqueue_head(&vpu->run.wq);
> +       init_waitqueue_head(&vpu->ack_wq);
> +
> +       res = platform_get_resource(pdev, IORESOURCE_IRQ, 0);
> +       if (!res) {
> +               dev_err(dev, "get IRQ resource failed.\n");
> +               ret = -ENXIO;
> +               goto free_p_mem;
> +       }
> +       vpu->reg.irq = platform_get_irq(pdev, 0);
> +       ret = devm_request_irq(dev, vpu->reg.irq, vpu_irq_handler, 0,
> +                              pdev->name, vpu);
> +       if (ret) {
> +               dev_err(dev, "failed to request irq\n");
> +               goto free_p_mem;
> +       }
> +
> +       vpu_clock_disable(vpu);
> +       dev_dbg(dev, "initialization completed\n");
> +
> +       return 0;
> +
> +free_p_mem:
> +       vpu_free_ext_mem(vpu, P_FW);
> +free_d_mem:
> +       vpu_free_ext_mem(vpu, D_FW);
> +remove_debugfs:
> +       of_reserved_mem_device_release(dev);
> +#ifdef CONFIG_DEBUG_FS
> +       debugfs_remove(vpu_debugfs);
> +cleanup_ipi:
> +#endif
> +       memset(vpu->ipi_desc, 0, sizeof(struct vpu_ipi_desc) * IPI_MAX);
> +vpu_mutex_destroy:
> +       mutex_destroy(&vpu->vpu_mutex);
> +disable_vpu_clk:
> +       vpu_clock_disable(vpu);
> +workqueue_destroy:
> +       destroy_workqueue(vpu->wdt.wq);
> +
> +       return ret;
> +}
> +
> +static const struct of_device_id mtk_vpu_match[] = {
> +       {
> +               .compatible = "mediatek,mt8173-vpu",
> +       },
> +       {},
> +};
> +MODULE_DEVICE_TABLE(of, mtk_vpu_match);
> +
> +static int mtk_vpu_remove(struct platform_device *pdev)
> +{
> +       struct mtk_vpu *vpu = platform_get_drvdata(pdev);
> +
> +#ifdef CONFIG_DEBUG_FS
> +       debugfs_remove(vpu_debugfs);
> +#endif
> +       if (vpu->wdt.wq) {
> +               flush_workqueue(vpu->wdt.wq);
> +               destroy_workqueue(vpu->wdt.wq);
> +       }
> +       vpu_free_ext_mem(vpu, P_FW);
> +       vpu_free_ext_mem(vpu, D_FW);
> +       mutex_destroy(&vpu->vpu_mutex);
> +       clk_unprepare(vpu->clk);
> +
> +       return 0;
> +}
> +
> +static struct platform_driver mtk_vpu_driver = {
> +       .probe  = mtk_vpu_probe,
> +       .remove = mtk_vpu_remove,
> +       .driver = {
> +               .name   = "mtk_vpu",
> +               .owner  = THIS_MODULE,
> +               .of_match_table = mtk_vpu_match,
> +       },
> +};
> +
> +module_platform_driver(mtk_vpu_driver);
> +
> +MODULE_LICENSE("GPL v2");
> +MODULE_DESCRIPTION("Mediatek Video Prosessor Unit driver");
> diff --git a/drivers/media/platform/mtk-vpu/mtk_vpu.h b/drivers/media/platform/mtk-vpu/mtk_vpu.h
> new file mode 100644
> index 0000000..d9c3cde
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vpu/mtk_vpu.h
> @@ -0,0 +1,167 @@
> +/*
> +* Copyright (c) 2015 MediaTek Inc.
> +* Author: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
> +*
> +* This program is free software; you can redistribute it and/or modify
> +* it under the terms of the GNU General Public License version 2 as
> +* published by the Free Software Foundation.
> +*
> +* This program is distributed in the hope that it will be useful,
> +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +* GNU General Public License for more details.
> +*/
> +
> +#ifndef _MTK_VPU_H
> +#define _MTK_VPU_H
> +
> +#include <linux/platform_device.h>
> +
> +/**
> + * VPU (video processor unit) is a tiny processor controlling video hardware
> + * related to video codec, scaling and color format converting.
> + * VPU interfaces with other blocks by share memory and interrupt.
> + **/
> +
> +typedef void (*ipi_handler_t) (void *data,
> +                              unsigned int len,
> +                              void *priv);
> +
> +/**
> + * enum ipi_id - the id of inter-processor interrupt
> + *
> + * @IPI_VPU_INIT:       The interrupt from vpu is to notfiy kernel
> +                        VPU initialization completed.
> +                        IPI_VPU_INIT is sent from VPU when firmware is
> +                        loaded. AP doesn't need to send IPI_VPU_INIT
> +                        command to VPU.
> +                        For other IPI below, AP should send the request
> +                        to VPU to trigger the interrupt.
> + * @IPI_VENC_H264:      The interrupt from vpu is to notify kernel to
> +                        handle H264 video encoder job, and vice versa.
> + * @IPI_VENC_VP8:       The interrupt fro vpu is to notify kernel to
> +                        handle VP8 video encoder job,, and vice versa.
> + * @IPI_MAX:            The maximum IPI number
> + */
> +
> +enum ipi_id {
> +       IPI_VPU_INIT = 0,
> +       IPI_VENC_H264,
> +       IPI_VENC_VP8,
> +       IPI_MAX,
> +};
> +
> +/**
> + * enum rst_id - reset id to register reset function for VPU watchdog timeout
> + *
> + * @VPU_RST_DEC: decoder reset id
> + * @VPU_RST_ENC: encoder reset id
> + * @VPU_RST_MDP: MDP (Media Data Path) reset id
> + * @VPU_RST_MAX: maximum reset id
> + */
> +enum rst_id {
> +       VPU_RST_ENC,
> +       VPU_RST_DEC,
> +       VPU_RST_MDP,
> +       VPU_RST_MAX,
> +};
> +
> +/**
> + * vpu_ipi_register - register an ipi function
> + *
> + * @pdev:      VPU platform device
> + * @id:                IPI ID
> + * @handler:   IPI handler
> + * @name:      IPI name
> + * @priv:      private data for IPI handler
> + *
> + * Register an ipi function to receive ipi interrupt from VPU.
> + *
> + * Return: Return 0 if ipi registers successfully, otherwise it is failed.
> + */
> +int vpu_ipi_register(struct platform_device *pdev, enum ipi_id id,
> +                    ipi_handler_t handler, const char *name, void *priv);
> +
> +/**
> + * vpu_ipi_send - send data from AP to vpu.
> + *
> + * @pdev:      VPU platform device
> + * @id:                IPI ID
> + * @buf:       the data buffer
> + * @len:       the data buffer length
> + *
> + * This function is thread-safe. When this function returns,
> + * VPU has received the data and starts the processing.
> + * When the processing completes, IPI handler registered
> + * by vpu_ipi_register will be called in interrupt context.
> + *
> + * Return: Return 0 if sending data successfully, otherwise it is failed.
> + **/
> +int vpu_ipi_send(struct platform_device *pdev,
> +                enum ipi_id id, void *buf,
> +                unsigned int len);
> +
> +/**
> + * vpu_get_plat_device - get VPU's platform device
> + *
> + * @pdev:      the platform device of the module requesting VPU platform
> + *             device for using VPU API.
> + *
> + * Return: Return NULL if it is failed.
> + * otherwise it is VPU's platform device
> + **/
> +struct platform_device *vpu_get_plat_device(struct platform_device *pdev);
> +
> +/**
> + * vpu_wdt_reg_handler - register a VPU watchdog handler
> + *
> + * @pdev:               VPU platform device
> + * @vpu_wdt_reset_func:        the callback reset function
> + * @private_data:       the private data for reset function
> + * @rst_id:            reset id
> + *
> + * Register a handler performing own tasks when vpu reset by watchdog
> + *
> + * Return: Return 0 if the handler is added successfully,
> + * otherwise it is failed.
> + *
> + **/
> +int vpu_wdt_reg_handler(struct platform_device *pdev,
> +                       void vpu_wdt_reset_func(void *),
> +                       void *priv, enum rst_id id);
> +
> +/**
> + * vpu_get_venc_hw_capa - get video encoder hardware capability
> + *
> + * @pdev:      VPU platform device
> + *
> + * Return: video encoder hardware capability
> + **/
> +unsigned int vpu_get_venc_hw_capa(struct platform_device *pdev);
> +
> +/**
> + * vpu_load_firmware - download VPU firmware and boot it
> + *
> + * @pdev:      VPU platform device
> + *
> + * Return: Return 0 if downloading firmware successfully,
> + * otherwise it is failed
> + **/
> +int vpu_load_firmware(struct platform_device *pdev);
> +
> +/**
> + * vpu_mapping_dm_addr - Mapping DTCM/DMEM to kernel virtual address
> + *
> + * @pdev:      VPU platform device
> + * @dmem_addr: VPU's data memory address
> + *
> + * Mapping the VPU's DTCM (Data Tightly-Coupled Memory) /
> + * DMEM (Data Extended Memory) memory address to
> + * kernel virtual address.
> + *
> + * Return: Return ERR_PTR(-EINVAL) if mapping failed,
> + * otherwise the mapped kernel virtual address
> + **/
> +void *vpu_mapping_dm_addr(struct platform_device *pdev,
> +                         u32 dtcm_dmem_addr);
> +#endif /* _MTK_VPU_H */
> --
> 1.7.9.5
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-media" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 102+ messages in thread

* [PATCH v4 2/8] [media] VPU: mediatek: support Mediatek VPU
@ 2016-02-15 13:59       ` Wu-Cheng Li (李務誠)
  0 siblings, 0 replies; 102+ messages in thread
From: Wu-Cheng Li (李務誠) @ 2016-02-15 13:59 UTC (permalink / raw)
  To: linux-arm-kernel

On Thu, Feb 4, 2016 at 7:34 PM, Tiffany Lin <tiffany.lin@mediatek.com> wrote:
> The VPU driver for hw video codec embedded in Mediatek's MT8173 SOCs.
> It is able to handle video decoding/encoding of in a range of formats.
> The driver provides with VPU firmware download, memory management and
> the communication interface between CPU and VPU.
> For VPU initialization, it will create virtual memory for CPU access and
> IOMMU address for vcodec hw device access. When a decode/encode instance
> opens a device node, vpu driver will download vpu firmware to the device.
> A decode/encode instant will decode/encode a frame using VPU
> interface to interrupt vpu to handle decoding/encoding jobs.
>
> Signed-off-by: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
> Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
> ---
>  drivers/media/platform/Kconfig           |    9 +
>  drivers/media/platform/Makefile          |    2 +
>  drivers/media/platform/mtk-vpu/Makefile  |    1 +
>  drivers/media/platform/mtk-vpu/mtk_vpu.c |  994 ++++++++++++++++++++++++++++++
>  drivers/media/platform/mtk-vpu/mtk_vpu.h |  167 +++++
>  5 files changed, 1173 insertions(+)
>  create mode 100644 drivers/media/platform/mtk-vpu/Makefile
>  create mode 100644 drivers/media/platform/mtk-vpu/mtk_vpu.c
>  create mode 100644 drivers/media/platform/mtk-vpu/mtk_vpu.h
>
> diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
> index ccbc974..ba812d6 100644
> --- a/drivers/media/platform/Kconfig
> +++ b/drivers/media/platform/Kconfig
> @@ -148,6 +148,15 @@ config VIDEO_CODA
>            Coda is a range of video codec IPs that supports
>            H.264, MPEG-4, and other video formats.
>
> +config VIDEO_MEDIATEK_VPU
> +       tristate "Mediatek Video Processor Unit"
> +       depends on VIDEO_DEV && VIDEO_V4L2 && ARCH_MEDIATEK
> +       ---help---
> +           This driver provides downloading VPU firmware and
> +           communicating with VPU. This driver for hw video
> +           codec embedded in new Mediatek's SOCs. It is able
> +           to handle video decoding/encoding in a range of formats.
> +
>  config VIDEO_MEM2MEM_DEINTERLACE
>         tristate "Deinterlace support"
>         depends on VIDEO_DEV && VIDEO_V4L2 && DMA_ENGINE
> diff --git a/drivers/media/platform/Makefile b/drivers/media/platform/Makefile
> index efa0295..e5b19c6 100644
> --- a/drivers/media/platform/Makefile
> +++ b/drivers/media/platform/Makefile
> @@ -55,3 +55,5 @@ obj-$(CONFIG_VIDEO_AM437X_VPFE)               += am437x/
>  obj-$(CONFIG_VIDEO_XILINX)             += xilinx/
>
>  ccflags-y += -I$(srctree)/drivers/media/i2c
> +
> +obj-$(CONFIG_VIDEO_MEDIATEK_VPU)       += mtk-vpu/
> diff --git a/drivers/media/platform/mtk-vpu/Makefile b/drivers/media/platform/mtk-vpu/Makefile
> new file mode 100644
> index 0000000..d890a66
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vpu/Makefile
> @@ -0,0 +1 @@
> +obj-y += mtk_vpu.o
> diff --git a/drivers/media/platform/mtk-vpu/mtk_vpu.c b/drivers/media/platform/mtk-vpu/mtk_vpu.c
> new file mode 100644
> index 0000000..f54fd89
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vpu/mtk_vpu.c
> @@ -0,0 +1,994 @@
> +/*
> +* Copyright (c) 2015 MediaTek Inc.
> +* Author: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
> +*
> +* This program is free software; you can redistribute it and/or modify
> +* it under the terms of the GNU General Public License version 2 as
> +* published by the Free Software Foundation.
> +*
> +* This program is distributed in the hope that it will be useful,
> +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +* GNU General Public License for more details.
> +*/
> +#include <linux/bootmem.h>
> +#include <linux/clk.h>
> +#include <linux/debugfs.h>
> +#include <linux/firmware.h>
> +#include <linux/interrupt.h>
> +#include <linux/iommu.h>
> +#include <linux/module.h>
> +#include <linux/of_address.h>
> +#include <linux/of_irq.h>
> +#include <linux/of_platform.h>
> +#include <linux/of_reserved_mem.h>
> +#include <linux/sched.h>
> +#include <linux/sizes.h>
> +
> +#include "mtk_vpu.h"
> +
> +/**
> + * VPU (video processor unit) is a tiny processor controlling video hardware
> + * related to video codec, scaling and color format converting.
> + * VPU interfaces with other blocks by share memory and interrupt.
> + **/
> +
> +#define INIT_TIMEOUT_MS                2000U
> +#define IPI_TIMEOUT_MS         2000U
> +#define VPU_FW_VER_LEN         16
> +
> +/* maximum program/data TCM (Tightly-Coupled Memory) size */
> +#define VPU_PTCM_SIZE          (96 * SZ_1K)
> +#define VPU_DTCM_SIZE          (32 * SZ_1K)
> +/* the offset to get data tcm address */
> +#define VPU_DTCM_OFFSET                0x18000UL
> +/* daynamic allocated maximum extended memory size */
> +#define VPU_EXT_P_SIZE         SZ_1M
> +#define VPU_EXT_D_SIZE         SZ_4M
> +/* maximum binary firmware size */
> +#define VPU_P_FW_SIZE          (VPU_PTCM_SIZE + VPU_EXT_P_SIZE)
> +#define VPU_D_FW_SIZE          (VPU_DTCM_SIZE + VPU_EXT_D_SIZE)
> +/* the size of share buffer between Host and  VPU */
> +#define SHARE_BUF_SIZE         48
> +
> +/* binary firmware name */
> +#define VPU_P_FW               "vpu_p.bin"
> +#define VPU_D_FW               "vpu_d.bin"
> +
> +#define VPU_RESET              0x0
> +#define VPU_TCM_CFG            0x0008
> +#define VPU_PMEM_EXT0_ADDR     0x000C
> +#define VPU_PMEM_EXT1_ADDR     0x0010
> +#define VPU_TO_HOST            0x001C
> +#define VPU_DMEM_EXT0_ADDR     0x0014
> +#define VPU_DMEM_EXT1_ADDR     0x0018
> +#define HOST_TO_VPU            0x0024
> +#define VPU_PC_REG             0x0060
> +#define VPU_WDT_REG            0x0084
> +
> +/* vpu inter-processor communication interrupt */
> +#define VPU_IPC_INT            BIT(8)
> +
> +/**
> + * enum vpu_fw_type - VPU firmware type
> + *
> + * @P_FW: program firmware
> + * @D_FW: data firmware
> + *
> + */
> +enum vpu_fw_type {
> +       P_FW,
> +       D_FW,
> +};
> +
> +/**
> + * struct vpu_mem - VPU extended program/data memory information
> + *
> + * @va:                the kernel virtual memory address of VPU extended memory
> + * @pa:                the physical memory address of VPU extended memory
This should be device memory address? If yes, s/physical/device/ and
s/pa/dma_addr/.
> + *
> + */
> +struct vpu_mem {
> +       void *va;
> +       dma_addr_t pa;
> +};
> +
> +/**
> + * struct vpu_regs - VPU TCM and configuration registers
> + *
> + * @tcm:       the register for VPU Tightly-Coupled Memory
> + * @cfg:       the register for VPU configuration
> + * @irq:       the irq number for VPU interrupt
> + */
> +struct vpu_regs {
> +       void __iomem *tcm;
> +       void __iomem *cfg;
> +       int irq;
> +};
> +
> +/**
> + * struct vpu_wdt_handler - VPU watchdog reset handler
> + *
> + * @reset_func:        reset handler
> + * @priv:      private data
> + */
> +struct vpu_wdt_handler {
> +       void (*reset_func)(void *);
> +       void *priv;
> +};
> +
> +/**
> + * struct vpu_wdt - VPU watchdog workqueue
> + *
> + * @handler:   VPU watchdog reset handler
> + * @ws:                workstruct for VPU watchdog
> + * @wq:                workqueue for VPU watchdog
> + */
> +struct vpu_wdt {
> +       struct vpu_wdt_handler handler[VPU_RST_MAX];
> +       struct work_struct ws;
> +       struct workqueue_struct *wq;
> +};
> +
> +/**
> + * struct vpu_run - VPU initialization status
> + *
> + * @signaled:          the signal of vpu initialization completed
> + * @fw_ver:            VPU firmware version
> + * @enc_capability:    encoder capability
We should document the meaning of the bit used. Since no bit is used
by the codec driver, document this is not used for now and the value
is reserved for future use.
> + * @wq:                        wait queue for VPU initialization status
> + */
> +struct vpu_run {
> +       u32 signaled;
> +       char fw_ver[VPU_FW_VER_LEN];
> +       unsigned int    enc_capability;
> +       wait_queue_head_t wq;
> +};
> +
> +/**
> + * struct vpu_ipi_desc - VPU IPI descriptor
> + *
> + * @handler:   IPI handler
> + * @name:      the name of IPI handler
> + * @priv:      the private data of IPI handler
> + */
> +struct vpu_ipi_desc {
> +       ipi_handler_t handler;
> +       const char *name;
> +       void *priv;
> +};
> +
> +/**
> + * struct share_obj - DTCM (Data Tightly-Coupled Memory) buffer shared with
> + *                   AP and VPU
> + *
> + * @id:                IPI id
> + * @len:       share buffer length
> + * @share_buf: share buffer data
> + */
> +struct share_obj {
> +       s32 id;
> +       u32 len;
> +       unsigned char share_buf[SHARE_BUF_SIZE];
> +};
> +
> +/**
> + * struct mtk_vpu - vpu driver data
> + * @extmem:            VPU extended memory information
> + * @reg:               VPU TCM and configuration registers
> + * @run:               VPU initialization status
> + * @ipi_desc:          VPU IPI descriptor
> + * @recv_buf:          VPU DTCM share buffer for receiving. The
> + *                     receive buffer is only accessed in interrupt context.
> + * @send_buf:          VPU DTCM share buffer for sending
> + * @dev:               VPU struct device
> + * @clk:               VPU clock on/off
> + * @enable_4GB:                VPU 4GB mode on/off
> + * @vpu_mutex:         protect mtk_vpu (except recv_buf) and ensure only
> + *                     one client to use VPU service at a time. For example,
> + *                     suppose a client is using VPU to decode VP8.
> + *                     If the other client wants to encode VP8,
> + *                     it has to wait until VP8 decode completes.
> + * @wdt_refcnt         WDT reference count to make sure the watchdog can be
> + *                     disabled if no other client is using VPU service
> + * @ipi_ack_signaled:  The ACKs for registered IPI function sending
> + *                     interrupt to VPU
s/ipi_ack_signaled/ipi_id_ack/. Move to after |ack_wq| to be
consistent with the order of variable declaration.
> + * @ack_wq:            The wait queue for each codec and mdp. When sleeping
> + *                     processes wake up, they will check the condition
> + *                     "ipi_ack_signaled" to run the corresponding action or
> + *                     go back to sleep.
> + *
> + */
> +struct mtk_vpu {
> +       struct vpu_mem extmem[2];
> +       struct vpu_regs reg;
> +       struct vpu_run run;
> +       struct vpu_wdt wdt;
> +       struct vpu_ipi_desc ipi_desc[IPI_MAX];
> +       struct share_obj *recv_buf;
> +       struct share_obj *send_buf;
> +       struct device *dev;
> +       struct clk *clk;
> +       bool enable_4GB;
> +       struct mutex vpu_mutex; /* for protecting vpu data data structure */
Remove the comment here. It should be documented in function comment above.
> +       atomic_t wdt_refcnt;
> +       wait_queue_head_t ack_wq;
> +       bool ipi_id_ack[IPI_MAX];
> +};
> +
> +static inline void vpu_cfg_writel(struct mtk_vpu *vpu, u32 val, u32 offset)
> +{
> +       writel(val, vpu->reg.cfg + offset);
> +}
> +
> +static inline u32 vpu_cfg_readl(struct mtk_vpu *vpu, u32 offset)
> +{
> +       return readl(vpu->reg.cfg + offset);
> +}
> +
> +static inline bool vpu_running(struct mtk_vpu *vpu)
> +{
> +       return vpu_cfg_readl(vpu, VPU_RESET) & BIT(0);
> +}
> +
> +void vpu_clock_disable(struct mtk_vpu *vpu)
> +{
> +       /* Disable VPU watchdog */
> +       if (atomic_dec_and_test(&vpu->wdt_refcnt))
Checking wdt_refcnt and doing vpu_cfg_writel should be done
atomically. For example, if vpu_clock_enable is called after
atomic_dec_and_test(&vpu->wdt_refcnt) and before vpu_cfg_writel(...),
the state of watchdog could be wrong. Adding a spinlock to protect
wdt_refcnt and vpu_cfg_writel together. Then wdt_refcnt doesn't need
to be atomic_t.
> +               vpu_cfg_writel(vpu,
> +                              vpu_cfg_readl(vpu, VPU_WDT_REG) & ~(1L << 31),
> +                              VPU_WDT_REG);
> +
> +       clk_disable(vpu->clk);
> +}
> +
> +int vpu_clock_enable(struct mtk_vpu *vpu)
> +{
> +       int ret;
> +
> +       ret = clk_enable(vpu->clk);
> +       if (ret)
> +               return ret;
> +       /* Enable VPU watchdog */
> +       if (!atomic_read(&vpu->wdt_refcnt))
Do we need to enable the watchdog other than vpu_ipi_send? The only
place that AP waits for VPU is in vpu_ipi_send. Right? If yes, can we
just enable and disable the watchdog in vpu_ipi_send?
> +               vpu_cfg_writel(vpu,
> +                              vpu_cfg_readl(vpu, VPU_WDT_REG) | (1L << 31),
> +                              VPU_WDT_REG);
> +
> +       atomic_inc(&vpu->wdt_refcnt);
Same above. atomic_read, vpu_cfg_writel, and atomic_inc should be done
atomically.
> +
> +       return ret;
> +}
> +
> +int vpu_ipi_register(struct platform_device *pdev,
> +                    enum ipi_id id, ipi_handler_t handler,
> +                    const char *name, void *priv)
> +{
> +       struct mtk_vpu *vpu = platform_get_drvdata(pdev);
> +       struct vpu_ipi_desc *ipi_desc;
> +
> +       if (!vpu) {
> +               dev_err(&pdev->dev, "vpu device in not ready\n");
> +               return -EPROBE_DEFER;
> +       }
> +
> +       if (id < IPI_MAX && handler) {
Check if id >= 0 because type of enum is implementation dependent.
> +               ipi_desc = vpu->ipi_desc;
> +               ipi_desc[id].name = name;
> +               ipi_desc[id].handler = handler;
> +               ipi_desc[id].priv = priv;
> +               return 0;
> +       }
> +
> +       dev_err(&pdev->dev, "register vpu ipi with invalid arguments\n");
print id
> +       return -EINVAL;
> +}
> +
> +int vpu_ipi_send(struct platform_device *pdev,
> +                enum ipi_id id, void *buf,
> +                unsigned int len)
> +{
> +       struct mtk_vpu *vpu = platform_get_drvdata(pdev);
> +       struct share_obj *send_obj = vpu->send_buf;
> +       unsigned long timeout;
> +       int ret = 0;
> +
> +       if (id <= IPI_VPU_INIT || id >= IPI_MAX ||
> +           len > sizeof(send_obj->share_buf) || !buf) {
> +               dev_err(vpu->dev, "failed to send ipi message\n");
> +               return -EINVAL;
> +       }
> +
> +       ret = vpu_clock_enable(vpu);
> +       if (ret) {
> +               dev_err(vpu->dev, "failed to enable vpu clock\n");
> +               return ret;
> +       }
> +       if (!vpu_running(vpu)) {
> +               dev_err(vpu->dev, "vpu_ipi_send: VPU is not running\n");
> +               ret = -EINVAL;
> +               goto clock_disable;
> +       }
> +
> +       mutex_lock(&vpu->vpu_mutex);
> +
> +        /* Wait until VPU receives the last command */
> +       timeout = jiffies + msecs_to_jiffies(IPI_TIMEOUT_MS);
> +       do {
> +               if (time_after(jiffies, timeout)) {
> +                       dev_err(vpu->dev, "vpu_ipi_send: IPI timeout!\n");
> +                       ret = -EIO;
> +                       goto mut_unlock;
> +               }
> +       } while (vpu_cfg_readl(vpu, HOST_TO_VPU));
> +
> +       memcpy((void *)send_obj->share_buf, buf, len);
> +       send_obj->len = len;
> +       send_obj->id = id;
> +
> +       vpu->ipi_id_ack[id] = false;
> +       /* send the command to VPU */
> +       vpu_cfg_writel(vpu, 0x1, HOST_TO_VPU);
> +
> +       mutex_unlock(&vpu->vpu_mutex);
> +
> +       /* wait for VPU's ACK */
> +       timeout = msecs_to_jiffies(IPI_TIMEOUT_MS);
> +       ret = wait_event_interruptible_timeout(vpu->ack_wq,
> +                                              vpu->ipi_id_ack[id], timeout);
> +       vpu->ipi_id_ack[id] = false;
> +       if (ret == 0) {
> +               dev_err(vpu->dev, "vpu ipi %d ack time out !", id);
> +               ret = -EIO;
> +               goto clock_disable;
> +       } else if (-ERESTARTSYS == ret) {
> +               dev_err(vpu->dev, "vpu ipi %d ack wait interrupted by a signal",
> +                       id);
> +               ret = -ERESTARTSYS;
> +               goto clock_disable;
> +       }
> +       vpu_clock_disable(vpu);
> +
> +       return 0;
> +
> +mut_unlock:
> +       vpu->ipi_id_ack[id] = false;
I don't see why we need to set it to false here. Remove if not needed.
> +       mutex_unlock(&vpu->vpu_mutex);
> +clock_disable:
> +       vpu_clock_disable(vpu);
> +
> +       return ret;
> +}
> +
> +static void vpu_wdt_reset_func(struct work_struct *ws)
> +{
> +       struct vpu_wdt *wdt = container_of(ws, struct vpu_wdt, ws);
> +       struct mtk_vpu *vpu = container_of(wdt, struct mtk_vpu, wdt);
> +       struct vpu_wdt_handler *handler = wdt->handler;
> +       int index, ret;
> +
> +       dev_info(vpu->dev, "vpu reset\n");
> +       mutex_lock(&vpu->vpu_mutex);
> +       ret = vpu_clock_enable(vpu);
> +       if (ret) {
This is missing mutex_unlock. Move mutex_lock right before
vpu_cfg_writel(vpu, 0x0, VPU_RESET);. So we don't need to unlock when
vpu_clock_enable fails.
> +               dev_err(vpu->dev, "[VPU] wdt enables clock failed %d\n", ret);
> +               return;
> +       }
> +       vpu_cfg_writel(vpu, 0x0, VPU_RESET);
> +       vpu_clock_disable(vpu);
> +       mutex_unlock(&vpu->vpu_mutex);
> +
> +       for (index = 0; index < VPU_RST_MAX; index++) {
> +               if (handler[index].reset_func) {
> +                       handler[index].reset_func(handler[index].priv);
> +                       dev_dbg(vpu->dev, "wdt handler func %d\n", index);
> +               }
> +       }
> +}
> +
> +int vpu_wdt_reg_handler(struct platform_device *pdev,
> +                       void wdt_reset(void *),
> +                       void *priv, enum rst_id id)
> +{
> +       struct mtk_vpu *vpu = platform_get_drvdata(pdev);
> +       struct vpu_wdt_handler *handler = vpu->wdt.handler;
> +
> +       if (!vpu) {
> +               dev_err(vpu->dev, "vpu device in not ready\n");
> +               return -EPROBE_DEFER;
> +       }
> +
> +       if (id < VPU_RST_MAX && wdt_reset != NULL) {
Check id >= 0
> +               dev_dbg(vpu->dev, "wdt register id %d\n", id);
> +               mutex_lock(&vpu->vpu_mutex);
> +               handler[id].reset_func = wdt_reset;
> +               handler[id].priv = priv;
> +               mutex_unlock(&vpu->vpu_mutex);
> +               return 0;
> +       }
> +
> +       dev_err(vpu->dev, "register vpu wdt handler failed\n");
> +       return -EINVAL;
> +}
> +
> +unsigned int vpu_get_venc_hw_capa(struct platform_device *pdev)
> +{
> +       struct mtk_vpu *vpu = platform_get_drvdata(pdev);
> +
> +       return vpu->run.enc_capability;
> +}
> +
> +void *vpu_mapping_dm_addr(struct platform_device *pdev,
> +                         u32 dtcm_dmem_addr)
> +{
> +       struct mtk_vpu *vpu = platform_get_drvdata(pdev);
> +
> +       if (!dtcm_dmem_addr ||
> +           (dtcm_dmem_addr > (VPU_DTCM_SIZE + VPU_EXT_D_SIZE))) {
> +               dev_err(vpu->dev, "invalid virtual data memory address\n");
> +               return ERR_PTR(-EINVAL);
> +       }
> +
> +       if (dtcm_dmem_addr < VPU_DTCM_SIZE)
> +               return dtcm_dmem_addr + vpu->reg.tcm + VPU_DTCM_OFFSET;
> +
> +       return vpu->extmem[D_FW].va + (dtcm_dmem_addr - VPU_DTCM_SIZE);
> +}
> +
> +struct platform_device *vpu_get_plat_device(struct platform_device *pdev)
> +{
> +       struct device *dev = &pdev->dev;
> +       struct device_node *vpu_node;
> +       struct platform_device *vpu_pdev;
> +
> +       vpu_node = of_parse_phandle(dev->of_node, "mediatek,vpu", 0);
> +       if (!vpu_node) {
> +               dev_err(dev, "can't get vpu node\n");
> +               return NULL;
> +       }
> +
> +       vpu_pdev = of_find_device_by_node(vpu_node);
> +       if (WARN_ON(!vpu_pdev)) {
> +               dev_err(dev, "vpu pdev failed\n");
> +               of_node_put(vpu_node);
> +               return NULL;
> +       }
> +
> +       return vpu_pdev;
> +}
> +
> +/* load vpu program/data memory */
> +static int load_requested_vpu(struct mtk_vpu *vpu,
> +                             const struct firmware *vpu_fw,
> +                             u8 fw_type)
> +{
> +       size_t tcm_size = fw_type ? VPU_DTCM_SIZE : VPU_PTCM_SIZE;
> +       size_t fw_size = fw_type ? VPU_D_FW_SIZE : VPU_P_FW_SIZE;
> +       char *fw_name = fw_type ? VPU_D_FW : VPU_P_FW;
> +       size_t dl_size = 0;
> +       size_t extra_fw_size = 0;
> +       void *dest;
> +       int ret;
> +
> +       ret = request_firmware(&vpu_fw, fw_name, vpu->dev);
> +       if (ret < 0) {
> +               dev_err(vpu->dev, "Failed to load %s, %d\n", fw_name, ret);
> +               return ret;
> +       }
> +       dl_size = vpu_fw->size;
> +       if (dl_size > fw_size) {
> +               dev_err(vpu->dev, "fw %s size %zu is abnormal\n", fw_name,
> +                       dl_size);
> +               release_firmware(vpu_fw);
> +               return  -EFBIG;
> +       }
> +       dev_dbg(vpu->dev, "Downloaded fw %s size: %zu.\n",
> +               fw_name,
> +               dl_size);
> +       /* reset VPU */
> +       vpu_cfg_writel(vpu, 0x0, VPU_RESET);
> +
> +       /* handle extended firmware size */
> +       if (dl_size > tcm_size) {
> +               dev_dbg(vpu->dev, "fw size %lx > limited fw size %lx\n",
> +                       dl_size, tcm_size);
> +               extra_fw_size = dl_size - tcm_size;
> +               dev_dbg(vpu->dev, "extra_fw_size %lx\n", extra_fw_size);
> +               dl_size = tcm_size;
> +       }
> +       dest = vpu->reg.tcm;
> +       if (fw_type == D_FW)
> +               dest += VPU_DTCM_OFFSET;
> +       memcpy(dest, vpu_fw->data, dl_size);
> +       /* download to extended memory if need */
> +       if (extra_fw_size > 0) {
> +               dest = vpu->extmem[fw_type].va;
> +               dev_dbg(vpu->dev, "download extended memory type %x\n",
> +                       fw_type);
> +               memcpy(dest, vpu_fw->data + tcm_size, extra_fw_size);
> +       }
> +
> +       release_firmware(vpu_fw);
> +
> +       return 0;
> +}
> +
> +int vpu_load_firmware(struct platform_device *pdev)
> +{
> +       struct mtk_vpu *vpu = platform_get_drvdata(pdev);
> +       struct device *dev = &pdev->dev;
> +       struct vpu_run *run = &vpu->run;
> +       const struct firmware *vpu_fw;
> +       int ret;
> +
> +       if (!pdev) {
> +               dev_err(dev, "VPU platform device is invalid\n");
> +               return -EINVAL;
> +       }
> +
> +       mutex_lock(&vpu->vpu_mutex);
> +
> +       ret = vpu_clock_enable(vpu);
> +       if (ret) {
> +               dev_err(dev, "enable clock failed %d\n", ret);
> +               goto OUT_LOAD_FW;
> +       }
> +
> +       if (vpu_running(vpu)) {
> +               vpu_clock_disable(vpu);
> +               mutex_unlock(&vpu->vpu_mutex);
> +               dev_warn(dev, "vpu is running already\n");
> +               return 0;
> +       }
> +
> +       run->signaled = false;
> +       dev_dbg(vpu->dev, "firmware request\n");
> +       /* Downloading program firmware to device*/
> +       ret = load_requested_vpu(vpu, vpu_fw, P_FW);
> +       if (ret < 0) {
> +               dev_err(dev, "Failed to request %s, %d\n", VPU_P_FW, ret);
> +               goto OUT_LOAD_FW;
> +       }
> +
> +       /* Downloading data firmware to device */
> +       ret = load_requested_vpu(vpu, vpu_fw, D_FW);
> +       if (ret < 0) {
> +               dev_err(dev, "Failed to request %s, %d\n", VPU_D_FW, ret);
> +               goto OUT_LOAD_FW;
> +       }
> +
> +       /* boot up vpu */
> +       vpu_cfg_writel(vpu, 0x1, VPU_RESET);
> +
> +       ret = wait_event_interruptible_timeout(run->wq,
> +                                              run->signaled,
> +                                              msecs_to_jiffies(INIT_TIMEOUT_MS)
> +                                              );
> +       if (ret == 0) {
> +               ret = -ETIME;
> +               dev_err(dev, "wait vpu initialization timout!\n");
> +               goto OUT_LOAD_FW;
> +       } else if (-ERESTARTSYS == ret) {
> +               dev_err(dev, "wait vpu interrupted by a signal!\n");
> +               goto OUT_LOAD_FW;
> +       }
> +
> +       ret = 0;
> +       dev_info(dev, "vpu is ready. Fw version %s\n", run->fw_ver);
> +
> +OUT_LOAD_FW:
> +       vpu_clock_disable(vpu);
> +       mutex_unlock(&vpu->vpu_mutex);
> +
> +       return ret;
> +}
> +
> +int vpu_compare_version(struct platform_device *pdev,
> +                       const char *expected_version)
> +{
> +       struct mtk_vpu *vpu = platform_get_drvdata(pdev);
> +       int cur_major, cur_minor, cur_build, cur_rel, cur_ver_num;
> +       int major, minor, build, rel, ver_num;
> +       char *cur_version = vpu->run.fw_ver;
> +
> +       cur_ver_num = sscanf(cur_version, "%d.%d.%d-rc%d",
> +                            &cur_major, &cur_minor, &cur_build, &cur_rel);
> +       if (cur_ver_num < 3)
> +               return -1;
> +       ver_num = sscanf(expected_version, "%d.%d.%d-rc%d",
> +                        &major, &minor, &build, &rel);
> +       if (ver_num < 3)
> +               return -1;
> +
> +       if (cur_major < major)
> +               return -1;
> +       if (cur_major > major)
> +               return 1;
> +
> +       if (cur_minor < minor)
> +               return -1;
> +       if (cur_minor > minor)
> +               return 1;
> +
> +       if (cur_build < build)
> +               return -1;
> +       if (cur_build > build)
> +               return 1;
> +
> +       if (cur_ver_num < ver_num)
> +               return -1;
> +       if (cur_ver_num > ver_num)
> +               return 1;
> +
> +       if (ver_num > 3) {
> +               if (cur_rel < rel)
> +                       return -1;
> +               if (cur_rel > rel)
> +                       return 1;
> +       }
> +
> +       return 0;
> +}
> +
> +static void vpu_init_ipi_handler(void *data, unsigned int len, void *priv)
> +{
> +       struct mtk_vpu *vpu = (struct mtk_vpu *)priv;
> +       struct vpu_run *run = (struct vpu_run *)data;
> +
> +       vpu->run.signaled = run->signaled;
> +       strncpy(vpu->run.fw_ver, run->fw_ver, VPU_FW_VER_LEN);
> +       vpu->run.enc_capability = run->enc_capability;
> +       wake_up_interruptible(&vpu->run.wq);
> +}
> +
> +#ifdef CONFIG_DEBUG_FS
> +static int vpu_debug_open(struct inode *inode, struct file *file)
> +{
> +       file->private_data = inode->i_private;
> +       return 0;
> +}
> +
> +static ssize_t vpu_debug_read(struct file *file, char __user *user_buf,
> +                             size_t count, loff_t *ppos)
> +{
> +       char buf[256];
> +       unsigned int len;
> +       unsigned int running, pc, vpu_to_host, host_to_vpu, wdt;
> +       int ret;
> +       struct device *dev = file->private_data;
> +       struct mtk_vpu *vpu = dev_get_drvdata(dev);
> +
> +       ret = vpu_clock_enable(vpu);
> +       if (ret) {
> +               dev_err(vpu->dev, "[VPU] enable clock failed %d\n", ret);
> +               return 0;
> +       }
> +
> +       /* vpu register status */
> +       running = vpu_running(vpu);
> +       pc = vpu_cfg_readl(vpu, VPU_PC_REG);
> +       wdt = vpu_cfg_readl(vpu, VPU_WDT_REG);
> +       host_to_vpu = vpu_cfg_readl(vpu, HOST_TO_VPU);
> +       vpu_to_host = vpu_cfg_readl(vpu, VPU_TO_HOST);
> +       vpu_clock_disable(vpu);
> +
> +       if (running) {
> +               len = sprintf(buf, "VPU is running\n\n"
> +               "FW Version: %s\n"
> +               "PC: 0x%x\n"
> +               "WDT: 0x%x\n"
> +               "Host to VPU: 0x%x\n"
> +               "VPU to Host: 0x%x\n",
> +               vpu->run.fw_ver, pc, wdt,
> +               host_to_vpu, vpu_to_host);
> +       } else {
> +               len = sprintf(buf, "VPU not running\n");
> +       }
> +
> +       return simple_read_from_buffer(user_buf, count, ppos, buf, len);
> +}
> +
> +static const struct file_operations vpu_debug_fops = {
> +       .open = vpu_debug_open,
> +       .read = vpu_debug_read,
> +};
> +#endif /* CONFIG_DEBUG_FS */
> +
> +static void vpu_free_ext_mem(struct mtk_vpu *vpu, u8 fw_type)
> +{
> +       struct device *dev = vpu->dev;
> +       size_t fw_ext_size = fw_type ? VPU_EXT_D_SIZE : VPU_EXT_P_SIZE;
> +
> +       dma_free_coherent(dev, fw_ext_size, vpu->extmem[fw_type].va,
> +                         vpu->extmem[fw_type].pa);
> +}
> +
> +static int vpu_alloc_ext_mem(struct mtk_vpu *vpu, u32 fw_type)
> +{
> +       struct device *dev = vpu->dev;
> +       size_t fw_ext_size = fw_type ? VPU_EXT_D_SIZE : VPU_EXT_P_SIZE;
> +       u32 vpu_ext_mem0 = fw_type ? VPU_DMEM_EXT0_ADDR : VPU_PMEM_EXT0_ADDR;
> +       u32 vpu_ext_mem1 = fw_type ? VPU_DMEM_EXT1_ADDR : VPU_PMEM_EXT1_ADDR;
> +       u32 offset_4gb = vpu->enable_4GB ? 0x40000000 : 0;
> +
> +       vpu->extmem[fw_type].va = dma_alloc_coherent(dev,
> +                                              fw_ext_size,
> +                                              &vpu->extmem[fw_type].pa,
> +                                              GFP_KERNEL);
> +       if (!vpu->extmem[fw_type].va) {
> +               dev_err(dev, "Failed to allocate the extended program memory\n");
> +               return PTR_ERR(vpu->extmem[fw_type].va);
> +       }
> +
> +       /* Disable extend0. Enable extend1 */
> +       vpu_cfg_writel(vpu, 0x1, vpu_ext_mem0);
> +       vpu_cfg_writel(vpu, (vpu->extmem[fw_type].pa & 0xFFFFF000) + offset_4gb,
> +                      vpu_ext_mem1);
> +
> +       dev_info(dev, "%s extend memory phy=0x%llx virt=0x%p\n",
> +                fw_type ? "Data" : "Program",
> +                (unsigned long long)vpu->extmem[fw_type].pa,
> +                vpu->extmem[fw_type].va);
> +
> +       return 0;
> +}
> +
> +static void vpu_ipi_handler(struct mtk_vpu *vpu)
> +{
> +       struct share_obj *rcv_obj = vpu->recv_buf;
> +       struct vpu_ipi_desc *ipi_desc = vpu->ipi_desc;
> +
> +       if (rcv_obj->id < IPI_MAX && ipi_desc[rcv_obj->id].handler) {
check rcv_obj->id >= 0
> +               ipi_desc[rcv_obj->id].handler(rcv_obj->share_buf,
> +                                             rcv_obj->len,
> +                                             ipi_desc[rcv_obj->id].priv);
> +               if (rcv_obj->id > IPI_VPU_INIT) {
> +                       vpu->ipi_id_ack[rcv_obj->id] = true;
> +                       wake_up_interruptible(&vpu->ack_wq);
> +               }
> +       } else {
> +               dev_err(vpu->dev, "No such ipi id = %d\n", rcv_obj->id);
> +       }
> +}
> +
> +static int vpu_ipi_init(struct mtk_vpu *vpu)
> +{
> +       /* Disable VPU to host interrupt */
> +       vpu_cfg_writel(vpu, 0x0, VPU_TO_HOST);
> +
> +       /* shared buffer initialization */
> +       vpu->recv_buf = (struct share_obj *)(vpu->reg.tcm + VPU_DTCM_OFFSET);
> +       vpu->send_buf = vpu->recv_buf + 1;
> +       memset(vpu->recv_buf, 0, sizeof(struct share_obj));
> +       memset(vpu->send_buf, 0, sizeof(struct share_obj));
> +       mutex_init(&vpu->vpu_mutex);
> +
> +       return 0;
> +}
> +
> +static irqreturn_t vpu_irq_handler(int irq, void *priv)
> +{
> +       struct mtk_vpu *vpu = priv;
> +       u32 vpu_to_host;
> +       int ret;
> +
> +       /*
> +        * Clock should have been enabled already.
> +        * Enable again in case vpu_ipi_send times out
> +        * and has disabled the clock.
> +        */
> +       ret = clk_enable(vpu->clk);
> +       if (ret) {
> +               dev_err(vpu->dev, "[VPU] enable clock failed %d\n", ret);
> +               return IRQ_NONE;
> +       }
> +       vpu_to_host = vpu_cfg_readl(vpu, VPU_TO_HOST);
> +       if (vpu_to_host & VPU_IPC_INT) {
> +               vpu_ipi_handler(vpu);
> +       } else {
> +               dev_err(vpu->dev, "vpu watchdog timeout! 0x%x", vpu_to_host);
> +               if (vpu->wdt.wq)
Remove. mtk_vpu_probe makes sure wdt.wq is not NULL.
> +                       queue_work(vpu->wdt.wq, &vpu->wdt.ws);
> +       }
> +
> +       /* VPU won't send another interrupt until we set VPU_TO_HOST to 0. */
> +       vpu_cfg_writel(vpu, 0x0, VPU_TO_HOST);
> +       clk_disable(vpu->clk);
> +
> +       return IRQ_HANDLED;
> +}
> +
> +#ifdef CONFIG_DEBUG_FS
> +static struct dentry *vpu_debugfs;
> +#endif
> +static int mtk_vpu_probe(struct platform_device *pdev)
> +{
> +       struct mtk_vpu *vpu;
> +       struct device *dev;
> +       struct resource *res;
> +       int ret = 0;
> +
> +       dev_dbg(&pdev->dev, "initialization\n");
> +
> +       dev = &pdev->dev;
> +       vpu = devm_kzalloc(dev, sizeof(*vpu), GFP_KERNEL);
> +       if (!vpu)
> +               return -ENOMEM;
> +
> +       vpu->dev = &pdev->dev;
> +       res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "tcm");
> +       vpu->reg.tcm = devm_ioremap_resource(dev, res);
> +       if (IS_ERR(vpu->reg.tcm)) {
> +               dev_err(dev, "devm_ioremap_resource vpu tcm failed.\n");
> +               return PTR_ERR(vpu->reg.tcm);
> +       }
> +
> +       res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "cfg_reg");
> +       vpu->reg.cfg = devm_ioremap_resource(dev, res);
> +       if (IS_ERR(vpu->reg.cfg)) {
> +               dev_err(dev, "devm_ioremap_resource vpu cfg failed.\n");
> +               return PTR_ERR(vpu->reg.cfg);
> +       }
> +
> +       /* Get VPU clock */
> +       vpu->clk = devm_clk_get(dev, "main");
> +       if (!vpu->clk) {
> +               dev_err(dev, "get vpu clock failed\n");
> +               return -EINVAL;
> +       }
> +
> +       platform_set_drvdata(pdev, vpu);
> +
> +       ret = clk_prepare(vpu->clk);
> +       if (ret) {
> +               dev_err(dev, "prepare vpu clock failed\n");
> +               return ret;
> +       }
> +
> +       /* VPU watchdog */
> +       vpu->wdt.wq = create_singlethread_workqueue("vpu_wdt");
> +       if (!vpu->wdt.wq) {
> +               dev_err(dev, "initialize wdt workqueue failed\n");
> +               return -ENOMEM;
> +       }
> +       INIT_WORK(&vpu->wdt.ws, vpu_wdt_reset_func);
> +
> +       ret = vpu_clock_enable(vpu);
> +       if (ret) {
> +               dev_err(dev, "enable vpu clock failed\n");
> +               goto workqueue_destroy;
> +       }
> +
> +       dev_dbg(dev, "vpu ipi init\n");
> +       ret = vpu_ipi_init(vpu);
> +       if (ret) {
> +               dev_err(dev, "Failed to init ipi\n");
> +               goto disable_vpu_clk;
> +       }
> +
> +       /* register vpu initialization IPI */
> +       ret = vpu_ipi_register(pdev, IPI_VPU_INIT, vpu_init_ipi_handler,
> +                              "vpu_init", vpu);
> +       if (ret) {
> +               dev_err(dev, "Failed to register IPI_VPU_INIT\n");
> +               goto vpu_mutex_destroy;
> +       }
> +
> +#ifdef CONFIG_DEBUG_FS
> +       vpu_debugfs = debugfs_create_file("mtk_vpu", S_IRUGO, NULL, (void *)dev,
> +                                         &vpu_debug_fops);
> +       if (!vpu_debugfs) {
> +               ret = -ENOMEM;
> +               goto cleanup_ipi;
> +       }
> +#endif
> +
> +       /* Set PTCM to 96K and DTCM to 32K */
> +       vpu_cfg_writel(vpu, 0x2, VPU_TCM_CFG);
> +
> +       vpu->enable_4GB = !!(max_pfn > (0xffffffffUL >> PAGE_SHIFT));
> +       dev_dbg(dev, "4GB mode %u\n", vpu->enable_4GB);
> +
> +       if (vpu->enable_4GB) {
> +               ret = of_reserved_mem_device_init(dev);
> +               if (ret)
> +                       dev_info(dev, "init reserved memory failed\n");
> +                       /* continue to use dynamic allocation if failed */
> +       }
> +
> +       ret = vpu_alloc_ext_mem(vpu, D_FW);
> +       if (ret) {
> +               dev_err(dev, "Allocate DM failed\n");
> +               goto remove_debugfs;
> +       }
> +
> +       ret = vpu_alloc_ext_mem(vpu, P_FW);
> +       if (ret) {
> +               dev_err(dev, "Allocate PM failed\n");
> +               goto free_d_mem;
> +       }
> +
> +       init_waitqueue_head(&vpu->run.wq);
> +       init_waitqueue_head(&vpu->ack_wq);
> +
> +       res = platform_get_resource(pdev, IORESOURCE_IRQ, 0);
> +       if (!res) {
> +               dev_err(dev, "get IRQ resource failed.\n");
> +               ret = -ENXIO;
> +               goto free_p_mem;
> +       }
> +       vpu->reg.irq = platform_get_irq(pdev, 0);
> +       ret = devm_request_irq(dev, vpu->reg.irq, vpu_irq_handler, 0,
> +                              pdev->name, vpu);
> +       if (ret) {
> +               dev_err(dev, "failed to request irq\n");
> +               goto free_p_mem;
> +       }
> +
> +       vpu_clock_disable(vpu);
> +       dev_dbg(dev, "initialization completed\n");
> +
> +       return 0;
> +
> +free_p_mem:
> +       vpu_free_ext_mem(vpu, P_FW);
> +free_d_mem:
> +       vpu_free_ext_mem(vpu, D_FW);
> +remove_debugfs:
> +       of_reserved_mem_device_release(dev);
> +#ifdef CONFIG_DEBUG_FS
> +       debugfs_remove(vpu_debugfs);
> +cleanup_ipi:
> +#endif
> +       memset(vpu->ipi_desc, 0, sizeof(struct vpu_ipi_desc) * IPI_MAX);
> +vpu_mutex_destroy:
> +       mutex_destroy(&vpu->vpu_mutex);
> +disable_vpu_clk:
> +       vpu_clock_disable(vpu);
> +workqueue_destroy:
> +       destroy_workqueue(vpu->wdt.wq);
> +
> +       return ret;
> +}
> +
> +static const struct of_device_id mtk_vpu_match[] = {
> +       {
> +               .compatible = "mediatek,mt8173-vpu",
> +       },
> +       {},
> +};
> +MODULE_DEVICE_TABLE(of, mtk_vpu_match);
> +
> +static int mtk_vpu_remove(struct platform_device *pdev)
> +{
> +       struct mtk_vpu *vpu = platform_get_drvdata(pdev);
> +
> +#ifdef CONFIG_DEBUG_FS
> +       debugfs_remove(vpu_debugfs);
> +#endif
> +       if (vpu->wdt.wq) {
> +               flush_workqueue(vpu->wdt.wq);
> +               destroy_workqueue(vpu->wdt.wq);
> +       }
> +       vpu_free_ext_mem(vpu, P_FW);
> +       vpu_free_ext_mem(vpu, D_FW);
> +       mutex_destroy(&vpu->vpu_mutex);
> +       clk_unprepare(vpu->clk);
> +
> +       return 0;
> +}
> +
> +static struct platform_driver mtk_vpu_driver = {
> +       .probe  = mtk_vpu_probe,
> +       .remove = mtk_vpu_remove,
> +       .driver = {
> +               .name   = "mtk_vpu",
> +               .owner  = THIS_MODULE,
> +               .of_match_table = mtk_vpu_match,
> +       },
> +};
> +
> +module_platform_driver(mtk_vpu_driver);
> +
> +MODULE_LICENSE("GPL v2");
> +MODULE_DESCRIPTION("Mediatek Video Prosessor Unit driver");
> diff --git a/drivers/media/platform/mtk-vpu/mtk_vpu.h b/drivers/media/platform/mtk-vpu/mtk_vpu.h
> new file mode 100644
> index 0000000..d9c3cde
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vpu/mtk_vpu.h
> @@ -0,0 +1,167 @@
> +/*
> +* Copyright (c) 2015 MediaTek Inc.
> +* Author: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
> +*
> +* This program is free software; you can redistribute it and/or modify
> +* it under the terms of the GNU General Public License version 2 as
> +* published by the Free Software Foundation.
> +*
> +* This program is distributed in the hope that it will be useful,
> +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +* GNU General Public License for more details.
> +*/
> +
> +#ifndef _MTK_VPU_H
> +#define _MTK_VPU_H
> +
> +#include <linux/platform_device.h>
> +
> +/**
> + * VPU (video processor unit) is a tiny processor controlling video hardware
> + * related to video codec, scaling and color format converting.
> + * VPU interfaces with other blocks by share memory and interrupt.
> + **/
> +
> +typedef void (*ipi_handler_t) (void *data,
> +                              unsigned int len,
> +                              void *priv);
> +
> +/**
> + * enum ipi_id - the id of inter-processor interrupt
> + *
> + * @IPI_VPU_INIT:       The interrupt from vpu is to notfiy kernel
> +                        VPU initialization completed.
> +                        IPI_VPU_INIT is sent from VPU when firmware is
> +                        loaded. AP doesn't need to send IPI_VPU_INIT
> +                        command to VPU.
> +                        For other IPI below, AP should send the request
> +                        to VPU to trigger the interrupt.
> + * @IPI_VENC_H264:      The interrupt from vpu is to notify kernel to
> +                        handle H264 video encoder job, and vice versa.
> + * @IPI_VENC_VP8:       The interrupt fro vpu is to notify kernel to
> +                        handle VP8 video encoder job,, and vice versa.
> + * @IPI_MAX:            The maximum IPI number
> + */
> +
> +enum ipi_id {
> +       IPI_VPU_INIT = 0,
> +       IPI_VENC_H264,
> +       IPI_VENC_VP8,
> +       IPI_MAX,
> +};
> +
> +/**
> + * enum rst_id - reset id to register reset function for VPU watchdog timeout
> + *
> + * @VPU_RST_DEC: decoder reset id
> + * @VPU_RST_ENC: encoder reset id
> + * @VPU_RST_MDP: MDP (Media Data Path) reset id
> + * @VPU_RST_MAX: maximum reset id
> + */
> +enum rst_id {
> +       VPU_RST_ENC,
> +       VPU_RST_DEC,
> +       VPU_RST_MDP,
> +       VPU_RST_MAX,
> +};
> +
> +/**
> + * vpu_ipi_register - register an ipi function
> + *
> + * @pdev:      VPU platform device
> + * @id:                IPI ID
> + * @handler:   IPI handler
> + * @name:      IPI name
> + * @priv:      private data for IPI handler
> + *
> + * Register an ipi function to receive ipi interrupt from VPU.
> + *
> + * Return: Return 0 if ipi registers successfully, otherwise it is failed.
> + */
> +int vpu_ipi_register(struct platform_device *pdev, enum ipi_id id,
> +                    ipi_handler_t handler, const char *name, void *priv);
> +
> +/**
> + * vpu_ipi_send - send data from AP to vpu.
> + *
> + * @pdev:      VPU platform device
> + * @id:                IPI ID
> + * @buf:       the data buffer
> + * @len:       the data buffer length
> + *
> + * This function is thread-safe. When this function returns,
> + * VPU has received the data and starts the processing.
> + * When the processing completes, IPI handler registered
> + * by vpu_ipi_register will be called in interrupt context.
> + *
> + * Return: Return 0 if sending data successfully, otherwise it is failed.
> + **/
> +int vpu_ipi_send(struct platform_device *pdev,
> +                enum ipi_id id, void *buf,
> +                unsigned int len);
> +
> +/**
> + * vpu_get_plat_device - get VPU's platform device
> + *
> + * @pdev:      the platform device of the module requesting VPU platform
> + *             device for using VPU API.
> + *
> + * Return: Return NULL if it is failed.
> + * otherwise it is VPU's platform device
> + **/
> +struct platform_device *vpu_get_plat_device(struct platform_device *pdev);
> +
> +/**
> + * vpu_wdt_reg_handler - register a VPU watchdog handler
> + *
> + * @pdev:               VPU platform device
> + * @vpu_wdt_reset_func:        the callback reset function
> + * @private_data:       the private data for reset function
> + * @rst_id:            reset id
> + *
> + * Register a handler performing own tasks when vpu reset by watchdog
> + *
> + * Return: Return 0 if the handler is added successfully,
> + * otherwise it is failed.
> + *
> + **/
> +int vpu_wdt_reg_handler(struct platform_device *pdev,
> +                       void vpu_wdt_reset_func(void *),
> +                       void *priv, enum rst_id id);
> +
> +/**
> + * vpu_get_venc_hw_capa - get video encoder hardware capability
> + *
> + * @pdev:      VPU platform device
> + *
> + * Return: video encoder hardware capability
> + **/
> +unsigned int vpu_get_venc_hw_capa(struct platform_device *pdev);
> +
> +/**
> + * vpu_load_firmware - download VPU firmware and boot it
> + *
> + * @pdev:      VPU platform device
> + *
> + * Return: Return 0 if downloading firmware successfully,
> + * otherwise it is failed
> + **/
> +int vpu_load_firmware(struct platform_device *pdev);
> +
> +/**
> + * vpu_mapping_dm_addr - Mapping DTCM/DMEM to kernel virtual address
> + *
> + * @pdev:      VPU platform device
> + * @dmem_addr: VPU's data memory address
> + *
> + * Mapping the VPU's DTCM (Data Tightly-Coupled Memory) /
> + * DMEM (Data Extended Memory) memory address to
> + * kernel virtual address.
> + *
> + * Return: Return ERR_PTR(-EINVAL) if mapping failed,
> + * otherwise the mapped kernel virtual address
> + **/
> +void *vpu_mapping_dm_addr(struct platform_device *pdev,
> +                         u32 dtcm_dmem_addr);
> +#endif /* _MTK_VPU_H */
> --
> 1.7.9.5
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-media" in
> the body of a message to majordomo at vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 102+ messages in thread

* Re: [PATCH v4 4/8] dt-bindings: Add a binding for Mediatek Video Encoder
  2016-02-15 10:42             ` Daniel Kurtz
  (?)
  (?)
@ 2016-02-16  2:09               ` tiffany lin
  -1 siblings, 0 replies; 102+ messages in thread
From: tiffany lin @ 2016-02-16  2:09 UTC (permalink / raw)
  To: Daniel Kurtz
  Cc: Hans Verkuil, Daniel Thompson, Rob Herring,
	Mauro Carvalho Chehab, Matthias Brugger, Pawel Osciak,
	Eddie Huang, Yingjoe Chen, open list:OPEN FIRMWARE AND...,
	linux-kernel, linux-arm-kernel, linux-media,
	moderated list:ARM/Mediatek SoC support, Lin PoChun, Tomasz Figa

Hi Daniel,

On Mon, 2016-02-15 at 18:42 +0800, Daniel Kurtz wrote:
> On Tue, Feb 9, 2016 at 7:29 PM, Daniel Kurtz <djkurtz@chromium.org> wrote:
> > Hi Tiffany,
> >
> > On Thu, Feb 4, 2016 at 7:34 PM, Tiffany Lin <tiffany.lin@mediatek.com> wrote:
> >> Add a DT binding documentation of Video Encoder for the
> >> MT8173 SoC from Mediatek.
> >>
> >> Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
> >> ---
> >>  .../devicetree/bindings/media/mediatek-vcodec.txt  |   59 ++++++++++++++++++++
> >>  1 file changed, 59 insertions(+)
> >>  create mode 100644 Documentation/devicetree/bindings/media/mediatek-vcodec.txt
> >>
> >> diff --git a/Documentation/devicetree/bindings/media/mediatek-vcodec.txt b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
> >> new file mode 100644
> >> index 0000000..572bfdd
> >> --- /dev/null
> >> +++ b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
> >> @@ -0,0 +1,59 @@
> >> +Mediatek Video Codec
> >> +
> >> +Mediatek Video Codec is the video codec hw present in Mediatek SoCs which
> >> +supports high resolution encoding functionalities.
> >> +
> >> +Required properties:
> >> +- compatible : "mediatek,mt8173-vcodec-enc" for encoder
> >> +- reg : Physical base address of the video codec registers and length of
> >> +  memory mapped region.
> >> +- interrupts : interrupt number to the cpu.
> >> +- mediatek,larb : must contain the local arbiters in the current Socs.
> >> +- clocks : list of clock specifiers, corresponding to entries in
> >> +  the clock-names property.
> >> +- clock-names: encoder must contain "vencpll_d2", "venc_sel", "univpll1_d2",
> >> +  "venc_lt_sel".
> >> +- iommus : should point to the respective IOMMU block with master port as
> >> +  argument, see Documentation/devicetree/bindings/iommu/mediatek,iommu.txt
> >> +  for details.
> >> +- mediatek,vpu : the node of video processor unit
> >> +
> >> +Example:
> >> +vcodec_enc: vcodec@0x18002000 {
> >> +    compatible = "mediatek,mt8173-vcodec-enc";
> >> +    reg = <0 0x18002000 0 0x1000>,    /*VENC_SYS*/
> >> +          <0 0x19002000 0 0x1000>;    /*VENC_LT_SYS*/
> >
> > This really looks like two encoder devices combined into a single
> > device tree node.
> > There are two register sets, two irqs, two sets of iommus, and two
> > sets of clocks.
> >
> > If possible, please split this node into two, one for each encoder.
> 
> I chatted offline with Mediatek.  They explained that there really is
> just one encoder hardware, that happens to support multiple formats.
> The encoder cannot encode with both formats at the same time.  The
> Mediatek HW designers added a new format to an existing encoder by
> adding a second interface (register set, irq, iommus, clocks) without
> modifying the original interface.  However in the hardware itself
> there is really just one encoder device.
> 
> So, although this node looks like it is for two encoder devices (one
> for each format), really there is just one device that supports each
> format through its large interface.
> 
> So, I'm fine with this being a single device node.
> 
> >> +    interrupts = <GIC_SPI 198 IRQ_TYPE_LEVEL_LOW>,
> >> +           <GIC_SPI 202 IRQ_TYPE_LEVEL_LOW>;
> >> +    mediatek,larb = <&larb3>,
> >> +                   <&larb5>;
> >> +    iommus = <&iommu M4U_PORT_VENC_RCPU>,
> >> +             <&iommu M4U_PORT_VENC_REC>,
> >> +             <&iommu M4U_PORT_VENC_BSDMA>,
> >> +             <&iommu M4U_PORT_VENC_SV_COMV>,
> >> +             <&iommu M4U_PORT_VENC_RD_COMV>,
> >> +             <&iommu M4U_PORT_VENC_CUR_LUMA>,
> >> +             <&iommu M4U_PORT_VENC_CUR_CHROMA>,
> >> +             <&iommu M4U_PORT_VENC_REF_LUMA>,
> >> +             <&iommu M4U_PORT_VENC_REF_CHROMA>,
> >> +             <&iommu M4U_PORT_VENC_NBM_RDMA>,
> >> +             <&iommu M4U_PORT_VENC_NBM_WDMA>,
> >> +             <&iommu M4U_PORT_VENC_RCPU_SET2>,
> >> +             <&iommu M4U_PORT_VENC_REC_FRM_SET2>,
> >> +             <&iommu M4U_PORT_VENC_BSDMA_SET2>,
> >> +             <&iommu M4U_PORT_VENC_SV_COMA_SET2>,
> >> +             <&iommu M4U_PORT_VENC_RD_COMA_SET2>,
> >> +             <&iommu M4U_PORT_VENC_CUR_LUMA_SET2>,
> >> +             <&iommu M4U_PORT_VENC_CUR_CHROMA_SET2>,
> >> +             <&iommu M4U_PORT_VENC_REF_LUMA_SET2>,
> >> +             <&iommu M4U_PORT_VENC_REC_CHROMA_SET2>;
> >> +    mediatek,vpu = <&vpu>;
> >> +    clocks = <&topckgen CLK_TOP_VENCPLL_D2>,
> >> +             <&topckgen CLK_TOP_VENC_SEL>,
> >> +             <&topckgen CLK_TOP_UNIVPLL1_D2>,
> >> +             <&topckgen CLK_TOP_VENC_LT_SEL>;
> >> +    clock-names = "vencpll_d2",
> >> +                  "venc_sel",
> >> +                  "univpll1_d2",
> >> +                  "venc_lt_sel";
> >
> > The names of these clocks should be from the perspective of the
> > encoder, not the clock provider.
> 
> I still think these clock names should be updated, however.
> 
Got it. We will fix this in next version.

> -Dan

^ permalink raw reply	[flat|nested] 102+ messages in thread

* Re: [PATCH v4 4/8] dt-bindings: Add a binding for Mediatek Video Encoder
@ 2016-02-16  2:09               ` tiffany lin
  0 siblings, 0 replies; 102+ messages in thread
From: tiffany lin @ 2016-02-16  2:09 UTC (permalink / raw)
  To: Daniel Kurtz
  Cc: Hans Verkuil, Daniel Thompson, Rob Herring,
	Mauro Carvalho Chehab, Matthias Brugger, Pawel Osciak,
	Eddie Huang, Yingjoe Chen, open list:OPEN FIRMWARE AND...,
	linux-kernel, linux-arm-kernel, linux-media,
	moderated list:ARM/Mediatek SoC support, Lin PoChun, Tomasz Figa

Hi Daniel,

On Mon, 2016-02-15 at 18:42 +0800, Daniel Kurtz wrote:
> On Tue, Feb 9, 2016 at 7:29 PM, Daniel Kurtz <djkurtz@chromium.org> wrote:
> > Hi Tiffany,
> >
> > On Thu, Feb 4, 2016 at 7:34 PM, Tiffany Lin <tiffany.lin@mediatek.com> wrote:
> >> Add a DT binding documentation of Video Encoder for the
> >> MT8173 SoC from Mediatek.
> >>
> >> Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
> >> ---
> >>  .../devicetree/bindings/media/mediatek-vcodec.txt  |   59 ++++++++++++++++++++
> >>  1 file changed, 59 insertions(+)
> >>  create mode 100644 Documentation/devicetree/bindings/media/mediatek-vcodec.txt
> >>
> >> diff --git a/Documentation/devicetree/bindings/media/mediatek-vcodec.txt b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
> >> new file mode 100644
> >> index 0000000..572bfdd
> >> --- /dev/null
> >> +++ b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
> >> @@ -0,0 +1,59 @@
> >> +Mediatek Video Codec
> >> +
> >> +Mediatek Video Codec is the video codec hw present in Mediatek SoCs which
> >> +supports high resolution encoding functionalities.
> >> +
> >> +Required properties:
> >> +- compatible : "mediatek,mt8173-vcodec-enc" for encoder
> >> +- reg : Physical base address of the video codec registers and length of
> >> +  memory mapped region.
> >> +- interrupts : interrupt number to the cpu.
> >> +- mediatek,larb : must contain the local arbiters in the current Socs.
> >> +- clocks : list of clock specifiers, corresponding to entries in
> >> +  the clock-names property.
> >> +- clock-names: encoder must contain "vencpll_d2", "venc_sel", "univpll1_d2",
> >> +  "venc_lt_sel".
> >> +- iommus : should point to the respective IOMMU block with master port as
> >> +  argument, see Documentation/devicetree/bindings/iommu/mediatek,iommu.txt
> >> +  for details.
> >> +- mediatek,vpu : the node of video processor unit
> >> +
> >> +Example:
> >> +vcodec_enc: vcodec@0x18002000 {
> >> +    compatible = "mediatek,mt8173-vcodec-enc";
> >> +    reg = <0 0x18002000 0 0x1000>,    /*VENC_SYS*/
> >> +          <0 0x19002000 0 0x1000>;    /*VENC_LT_SYS*/
> >
> > This really looks like two encoder devices combined into a single
> > device tree node.
> > There are two register sets, two irqs, two sets of iommus, and two
> > sets of clocks.
> >
> > If possible, please split this node into two, one for each encoder.
> 
> I chatted offline with Mediatek.  They explained that there really is
> just one encoder hardware, that happens to support multiple formats.
> The encoder cannot encode with both formats at the same time.  The
> Mediatek HW designers added a new format to an existing encoder by
> adding a second interface (register set, irq, iommus, clocks) without
> modifying the original interface.  However in the hardware itself
> there is really just one encoder device.
> 
> So, although this node looks like it is for two encoder devices (one
> for each format), really there is just one device that supports each
> format through its large interface.
> 
> So, I'm fine with this being a single device node.
> 
> >> +    interrupts = <GIC_SPI 198 IRQ_TYPE_LEVEL_LOW>,
> >> +           <GIC_SPI 202 IRQ_TYPE_LEVEL_LOW>;
> >> +    mediatek,larb = <&larb3>,
> >> +                   <&larb5>;
> >> +    iommus = <&iommu M4U_PORT_VENC_RCPU>,
> >> +             <&iommu M4U_PORT_VENC_REC>,
> >> +             <&iommu M4U_PORT_VENC_BSDMA>,
> >> +             <&iommu M4U_PORT_VENC_SV_COMV>,
> >> +             <&iommu M4U_PORT_VENC_RD_COMV>,
> >> +             <&iommu M4U_PORT_VENC_CUR_LUMA>,
> >> +             <&iommu M4U_PORT_VENC_CUR_CHROMA>,
> >> +             <&iommu M4U_PORT_VENC_REF_LUMA>,
> >> +             <&iommu M4U_PORT_VENC_REF_CHROMA>,
> >> +             <&iommu M4U_PORT_VENC_NBM_RDMA>,
> >> +             <&iommu M4U_PORT_VENC_NBM_WDMA>,
> >> +             <&iommu M4U_PORT_VENC_RCPU_SET2>,
> >> +             <&iommu M4U_PORT_VENC_REC_FRM_SET2>,
> >> +             <&iommu M4U_PORT_VENC_BSDMA_SET2>,
> >> +             <&iommu M4U_PORT_VENC_SV_COMA_SET2>,
> >> +             <&iommu M4U_PORT_VENC_RD_COMA_SET2>,
> >> +             <&iommu M4U_PORT_VENC_CUR_LUMA_SET2>,
> >> +             <&iommu M4U_PORT_VENC_CUR_CHROMA_SET2>,
> >> +             <&iommu M4U_PORT_VENC_REF_LUMA_SET2>,
> >> +             <&iommu M4U_PORT_VENC_REC_CHROMA_SET2>;
> >> +    mediatek,vpu = <&vpu>;
> >> +    clocks = <&topckgen CLK_TOP_VENCPLL_D2>,
> >> +             <&topckgen CLK_TOP_VENC_SEL>,
> >> +             <&topckgen CLK_TOP_UNIVPLL1_D2>,
> >> +             <&topckgen CLK_TOP_VENC_LT_SEL>;
> >> +    clock-names = "vencpll_d2",
> >> +                  "venc_sel",
> >> +                  "univpll1_d2",
> >> +                  "venc_lt_sel";
> >
> > The names of these clocks should be from the perspective of the
> > encoder, not the clock provider.
> 
> I still think these clock names should be updated, however.
> 
Got it. We will fix this in next version.

> -Dan

^ permalink raw reply	[flat|nested] 102+ messages in thread

* Re: [PATCH v4 4/8] dt-bindings: Add a binding for Mediatek Video Encoder
@ 2016-02-16  2:09               ` tiffany lin
  0 siblings, 0 replies; 102+ messages in thread
From: tiffany lin @ 2016-02-16  2:09 UTC (permalink / raw)
  To: Daniel Kurtz
  Cc: Hans Verkuil, Daniel Thompson, Rob Herring,
	Mauro Carvalho Chehab, Matthias Brugger, Pawel Osciak,
	Eddie Huang, Yingjoe Chen, open list:OPEN FIRMWARE AND...,
	linux-kernel, linux-arm-kernel, linux-media,
	moderated list:ARM/Mediatek SoC support, Lin PoChun, Tomasz Figa

Hi Daniel,

On Mon, 2016-02-15 at 18:42 +0800, Daniel Kurtz wrote:
> On Tue, Feb 9, 2016 at 7:29 PM, Daniel Kurtz <djkurtz@chromium.org> wrote:
> > Hi Tiffany,
> >
> > On Thu, Feb 4, 2016 at 7:34 PM, Tiffany Lin <tiffany.lin@mediatek.com> wrote:
> >> Add a DT binding documentation of Video Encoder for the
> >> MT8173 SoC from Mediatek.
> >>
> >> Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
> >> ---
> >>  .../devicetree/bindings/media/mediatek-vcodec.txt  |   59 ++++++++++++++++++++
> >>  1 file changed, 59 insertions(+)
> >>  create mode 100644 Documentation/devicetree/bindings/media/mediatek-vcodec.txt
> >>
> >> diff --git a/Documentation/devicetree/bindings/media/mediatek-vcodec.txt b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
> >> new file mode 100644
> >> index 0000000..572bfdd
> >> --- /dev/null
> >> +++ b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
> >> @@ -0,0 +1,59 @@
> >> +Mediatek Video Codec
> >> +
> >> +Mediatek Video Codec is the video codec hw present in Mediatek SoCs which
> >> +supports high resolution encoding functionalities.
> >> +
> >> +Required properties:
> >> +- compatible : "mediatek,mt8173-vcodec-enc" for encoder
> >> +- reg : Physical base address of the video codec registers and length of
> >> +  memory mapped region.
> >> +- interrupts : interrupt number to the cpu.
> >> +- mediatek,larb : must contain the local arbiters in the current Socs.
> >> +- clocks : list of clock specifiers, corresponding to entries in
> >> +  the clock-names property.
> >> +- clock-names: encoder must contain "vencpll_d2", "venc_sel", "univpll1_d2",
> >> +  "venc_lt_sel".
> >> +- iommus : should point to the respective IOMMU block with master port as
> >> +  argument, see Documentation/devicetree/bindings/iommu/mediatek,iommu.txt
> >> +  for details.
> >> +- mediatek,vpu : the node of video processor unit
> >> +
> >> +Example:
> >> +vcodec_enc: vcodec@0x18002000 {
> >> +    compatible = "mediatek,mt8173-vcodec-enc";
> >> +    reg = <0 0x18002000 0 0x1000>,    /*VENC_SYS*/
> >> +          <0 0x19002000 0 0x1000>;    /*VENC_LT_SYS*/
> >
> > This really looks like two encoder devices combined into a single
> > device tree node.
> > There are two register sets, two irqs, two sets of iommus, and two
> > sets of clocks.
> >
> > If possible, please split this node into two, one for each encoder.
> 
> I chatted offline with Mediatek.  They explained that there really is
> just one encoder hardware, that happens to support multiple formats.
> The encoder cannot encode with both formats at the same time.  The
> Mediatek HW designers added a new format to an existing encoder by
> adding a second interface (register set, irq, iommus, clocks) without
> modifying the original interface.  However in the hardware itself
> there is really just one encoder device.
> 
> So, although this node looks like it is for two encoder devices (one
> for each format), really there is just one device that supports each
> format through its large interface.
> 
> So, I'm fine with this being a single device node.
> 
> >> +    interrupts = <GIC_SPI 198 IRQ_TYPE_LEVEL_LOW>,
> >> +           <GIC_SPI 202 IRQ_TYPE_LEVEL_LOW>;
> >> +    mediatek,larb = <&larb3>,
> >> +                   <&larb5>;
> >> +    iommus = <&iommu M4U_PORT_VENC_RCPU>,
> >> +             <&iommu M4U_PORT_VENC_REC>,
> >> +             <&iommu M4U_PORT_VENC_BSDMA>,
> >> +             <&iommu M4U_PORT_VENC_SV_COMV>,
> >> +             <&iommu M4U_PORT_VENC_RD_COMV>,
> >> +             <&iommu M4U_PORT_VENC_CUR_LUMA>,
> >> +             <&iommu M4U_PORT_VENC_CUR_CHROMA>,
> >> +             <&iommu M4U_PORT_VENC_REF_LUMA>,
> >> +             <&iommu M4U_PORT_VENC_REF_CHROMA>,
> >> +             <&iommu M4U_PORT_VENC_NBM_RDMA>,
> >> +             <&iommu M4U_PORT_VENC_NBM_WDMA>,
> >> +             <&iommu M4U_PORT_VENC_RCPU_SET2>,
> >> +             <&iommu M4U_PORT_VENC_REC_FRM_SET2>,
> >> +             <&iommu M4U_PORT_VENC_BSDMA_SET2>,
> >> +             <&iommu M4U_PORT_VENC_SV_COMA_SET2>,
> >> +             <&iommu M4U_PORT_VENC_RD_COMA_SET2>,
> >> +             <&iommu M4U_PORT_VENC_CUR_LUMA_SET2>,
> >> +             <&iommu M4U_PORT_VENC_CUR_CHROMA_SET2>,
> >> +             <&iommu M4U_PORT_VENC_REF_LUMA_SET2>,
> >> +             <&iommu M4U_PORT_VENC_REC_CHROMA_SET2>;
> >> +    mediatek,vpu = <&vpu>;
> >> +    clocks = <&topckgen CLK_TOP_VENCPLL_D2>,
> >> +             <&topckgen CLK_TOP_VENC_SEL>,
> >> +             <&topckgen CLK_TOP_UNIVPLL1_D2>,
> >> +             <&topckgen CLK_TOP_VENC_LT_SEL>;
> >> +    clock-names = "vencpll_d2",
> >> +                  "venc_sel",
> >> +                  "univpll1_d2",
> >> +                  "venc_lt_sel";
> >
> > The names of these clocks should be from the perspective of the
> > encoder, not the clock provider.
> 
> I still think these clock names should be updated, however.
> 
Got it. We will fix this in next version.

> -Dan



^ permalink raw reply	[flat|nested] 102+ messages in thread

* [PATCH v4 4/8] dt-bindings: Add a binding for Mediatek Video Encoder
@ 2016-02-16  2:09               ` tiffany lin
  0 siblings, 0 replies; 102+ messages in thread
From: tiffany lin @ 2016-02-16  2:09 UTC (permalink / raw)
  To: linux-arm-kernel

Hi Daniel,

On Mon, 2016-02-15 at 18:42 +0800, Daniel Kurtz wrote:
> On Tue, Feb 9, 2016 at 7:29 PM, Daniel Kurtz <djkurtz@chromium.org> wrote:
> > Hi Tiffany,
> >
> > On Thu, Feb 4, 2016 at 7:34 PM, Tiffany Lin <tiffany.lin@mediatek.com> wrote:
> >> Add a DT binding documentation of Video Encoder for the
> >> MT8173 SoC from Mediatek.
> >>
> >> Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
> >> ---
> >>  .../devicetree/bindings/media/mediatek-vcodec.txt  |   59 ++++++++++++++++++++
> >>  1 file changed, 59 insertions(+)
> >>  create mode 100644 Documentation/devicetree/bindings/media/mediatek-vcodec.txt
> >>
> >> diff --git a/Documentation/devicetree/bindings/media/mediatek-vcodec.txt b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
> >> new file mode 100644
> >> index 0000000..572bfdd
> >> --- /dev/null
> >> +++ b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
> >> @@ -0,0 +1,59 @@
> >> +Mediatek Video Codec
> >> +
> >> +Mediatek Video Codec is the video codec hw present in Mediatek SoCs which
> >> +supports high resolution encoding functionalities.
> >> +
> >> +Required properties:
> >> +- compatible : "mediatek,mt8173-vcodec-enc" for encoder
> >> +- reg : Physical base address of the video codec registers and length of
> >> +  memory mapped region.
> >> +- interrupts : interrupt number to the cpu.
> >> +- mediatek,larb : must contain the local arbiters in the current Socs.
> >> +- clocks : list of clock specifiers, corresponding to entries in
> >> +  the clock-names property.
> >> +- clock-names: encoder must contain "vencpll_d2", "venc_sel", "univpll1_d2",
> >> +  "venc_lt_sel".
> >> +- iommus : should point to the respective IOMMU block with master port as
> >> +  argument, see Documentation/devicetree/bindings/iommu/mediatek,iommu.txt
> >> +  for details.
> >> +- mediatek,vpu : the node of video processor unit
> >> +
> >> +Example:
> >> +vcodec_enc: vcodec at 0x18002000 {
> >> +    compatible = "mediatek,mt8173-vcodec-enc";
> >> +    reg = <0 0x18002000 0 0x1000>,    /*VENC_SYS*/
> >> +          <0 0x19002000 0 0x1000>;    /*VENC_LT_SYS*/
> >
> > This really looks like two encoder devices combined into a single
> > device tree node.
> > There are two register sets, two irqs, two sets of iommus, and two
> > sets of clocks.
> >
> > If possible, please split this node into two, one for each encoder.
> 
> I chatted offline with Mediatek.  They explained that there really is
> just one encoder hardware, that happens to support multiple formats.
> The encoder cannot encode with both formats at the same time.  The
> Mediatek HW designers added a new format to an existing encoder by
> adding a second interface (register set, irq, iommus, clocks) without
> modifying the original interface.  However in the hardware itself
> there is really just one encoder device.
> 
> So, although this node looks like it is for two encoder devices (one
> for each format), really there is just one device that supports each
> format through its large interface.
> 
> So, I'm fine with this being a single device node.
> 
> >> +    interrupts = <GIC_SPI 198 IRQ_TYPE_LEVEL_LOW>,
> >> +           <GIC_SPI 202 IRQ_TYPE_LEVEL_LOW>;
> >> +    mediatek,larb = <&larb3>,
> >> +                   <&larb5>;
> >> +    iommus = <&iommu M4U_PORT_VENC_RCPU>,
> >> +             <&iommu M4U_PORT_VENC_REC>,
> >> +             <&iommu M4U_PORT_VENC_BSDMA>,
> >> +             <&iommu M4U_PORT_VENC_SV_COMV>,
> >> +             <&iommu M4U_PORT_VENC_RD_COMV>,
> >> +             <&iommu M4U_PORT_VENC_CUR_LUMA>,
> >> +             <&iommu M4U_PORT_VENC_CUR_CHROMA>,
> >> +             <&iommu M4U_PORT_VENC_REF_LUMA>,
> >> +             <&iommu M4U_PORT_VENC_REF_CHROMA>,
> >> +             <&iommu M4U_PORT_VENC_NBM_RDMA>,
> >> +             <&iommu M4U_PORT_VENC_NBM_WDMA>,
> >> +             <&iommu M4U_PORT_VENC_RCPU_SET2>,
> >> +             <&iommu M4U_PORT_VENC_REC_FRM_SET2>,
> >> +             <&iommu M4U_PORT_VENC_BSDMA_SET2>,
> >> +             <&iommu M4U_PORT_VENC_SV_COMA_SET2>,
> >> +             <&iommu M4U_PORT_VENC_RD_COMA_SET2>,
> >> +             <&iommu M4U_PORT_VENC_CUR_LUMA_SET2>,
> >> +             <&iommu M4U_PORT_VENC_CUR_CHROMA_SET2>,
> >> +             <&iommu M4U_PORT_VENC_REF_LUMA_SET2>,
> >> +             <&iommu M4U_PORT_VENC_REC_CHROMA_SET2>;
> >> +    mediatek,vpu = <&vpu>;
> >> +    clocks = <&topckgen CLK_TOP_VENCPLL_D2>,
> >> +             <&topckgen CLK_TOP_VENC_SEL>,
> >> +             <&topckgen CLK_TOP_UNIVPLL1_D2>,
> >> +             <&topckgen CLK_TOP_VENC_LT_SEL>;
> >> +    clock-names = "vencpll_d2",
> >> +                  "venc_sel",
> >> +                  "univpll1_d2",
> >> +                  "venc_lt_sel";
> >
> > The names of these clocks should be from the perspective of the
> > encoder, not the clock provider.
> 
> I still think these clock names should be updated, however.
> 
Got it. We will fix this in next version.

> -Dan

^ permalink raw reply	[flat|nested] 102+ messages in thread

* Re: [PATCH v4 5/8] [Media] vcodec: mediatek: Add Mediatek V4L2 Video Encoder Driver
  2016-02-15 11:21           ` [PATCH v4 5/8] [Media] vcodec: mediatek: Add Mediatek V4L2 " Hans Verkuil
@ 2016-02-16  6:37               ` tiffany lin
  0 siblings, 0 replies; 102+ messages in thread
From: tiffany lin @ 2016-02-16  6:37 UTC (permalink / raw)
  To: Hans Verkuil
  Cc: Hans Verkuil, daniel.thompson, Rob Herring,
	Mauro Carvalho Chehab, Matthias Brugger, Daniel Kurtz,
	Pawel Osciak, Eddie Huang, Yingjoe Chen, devicetree,
	linux-kernel, linux-arm-kernel, linux-media, linux-mediatek,
	PoChun.Lin, Andrew-CT Chen

Hi Hans,

Thanks for your time.
On Mon, 2016-02-15 at 12:21 +0100, Hans Verkuil wrote:
> On 02/04/2016 12:35 PM, Tiffany Lin wrote:
> > From: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
> > 
> > Add v4l2 layer encoder driver for MT8173
> > 
> > Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
> 
> If Andrew is the author, shouldn't there be a Signed-off-by from him as well?
> 
> And in copyright notices (might want to update the year to 2016 BTW) PC Chen
> is mentioned among others. It might be useful to update the Signed-off-by lines.
> 

Author is PC Chen and Tiffany Lin, Andrew-CT Chen is author of mtk-vpu
module.
We will fix copyright and this in next version.

> > ---
> >  drivers/media/platform/Kconfig                     |   11 +
> >  drivers/media/platform/Makefile                    |    2 +
> >  drivers/media/platform/mtk-vcodec/Makefile         |    8 +
> >  drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h |  388 ++++++
> >  drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c | 1380 ++++++++++++++++++++
> >  drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h |   46 +
> >  .../media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c |  476 +++++++
> >  .../media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c  |  132 ++
> >  .../media/platform/mtk-vcodec/mtk_vcodec_intr.c    |  102 ++
> >  .../media/platform/mtk-vcodec/mtk_vcodec_intr.h    |   29 +
> >  drivers/media/platform/mtk-vcodec/mtk_vcodec_pm.h  |   26 +
> >  .../media/platform/mtk-vcodec/mtk_vcodec_util.c    |  106 ++
> >  .../media/platform/mtk-vcodec/mtk_vcodec_util.h    |   85 ++
> >  drivers/media/platform/mtk-vcodec/venc_drv_base.h  |   62 +
> >  drivers/media/platform/mtk-vcodec/venc_drv_if.c    |  100 ++
> >  drivers/media/platform/mtk-vcodec/venc_drv_if.h    |  175 +++
> >  drivers/media/platform/mtk-vcodec/venc_ipi_msg.h   |  212 +++
> >  include/uapi/linux/v4l2-controls.h                 |    4 +
> >  18 files changed, 3344 insertions(+)
> >  create mode 100644 drivers/media/platform/mtk-vcodec/Makefile
> >  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
> >  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
> >  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h
> >  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
> >  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
> >  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.c
> >  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.h
> >  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_pm.h
> >  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c
> >  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
> >  create mode 100644 drivers/media/platform/mtk-vcodec/venc_drv_base.h
> >  create mode 100644 drivers/media/platform/mtk-vcodec/venc_drv_if.c
> >  create mode 100644 drivers/media/platform/mtk-vcodec/venc_drv_if.h
> >  create mode 100644 drivers/media/platform/mtk-vcodec/venc_ipi_msg.h
> >  mode change 100644 => 100755 include/uapi/linux/v4l2-controls.h
> > 
> > diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
> > index ba812d6..3e831c5 100644
> > --- a/drivers/media/platform/Kconfig
> > +++ b/drivers/media/platform/Kconfig
> > @@ -157,6 +157,17 @@ config VIDEO_MEDIATEK_VPU
> >  	    codec embedded in new Mediatek's SOCs. It is able
> >  	    to handle video decoding/encoding in a range of formats.
> >  
> > +config VIDEO_MEDIATEK_VCODEC
> > +        tristate "Mediatek Video Codec driver"
> > +        depends on VIDEO_DEV && VIDEO_V4L2
> > +        depends on ARCH_MEDIATEK || COMPILE_TEST
> > +        select VIDEOBUF2_DMA_CONTIG
> > +        select V4L2_MEM2MEM_DEV
> > +        select MEDIATEK_VPU
> > +        default n
> > +        ---help---
> > +            Mediatek video codec driver for V4L2
> > +
> >  config VIDEO_MEM2MEM_DEINTERLACE
> >  	tristate "Deinterlace support"
> >  	depends on VIDEO_DEV && VIDEO_V4L2 && DMA_ENGINE
> > diff --git a/drivers/media/platform/Makefile b/drivers/media/platform/Makefile
> > index e5b19c6..510e06b 100644
> > --- a/drivers/media/platform/Makefile
> > +++ b/drivers/media/platform/Makefile
> > @@ -57,3 +57,5 @@ obj-$(CONFIG_VIDEO_XILINX)		+= xilinx/
> >  ccflags-y += -I$(srctree)/drivers/media/i2c
> >  
> >  obj-$(CONFIG_VIDEO_MEDIATEK_VPU)	+= mtk-vpu/
> > +
> > +obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC)	+= mtk-vcodec/
> > diff --git a/drivers/media/platform/mtk-vcodec/Makefile b/drivers/media/platform/mtk-vcodec/Makefile
> > new file mode 100644
> > index 0000000..ce38689
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/Makefile
> > @@ -0,0 +1,8 @@
> > +obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += mtk_vcodec_intr.o \
> > +				       mtk_vcodec_util.o \
> > +				       mtk_vcodec_enc_drv.o \
> > +				       mtk_vcodec_enc.o \
> > +				       mtk_vcodec_enc_pm.o \
> > +				       venc_drv_if.o
> > +
> > +ccflags-y += -I$(srctree)/drivers/media/platform/mtk-vpu
> > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
> > new file mode 100644
> > index 0000000..9da2818
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
> > @@ -0,0 +1,388 @@
> > +/*
> > +* Copyright (c) 2015 MediaTek Inc.
> > +* Author: PC Chen <pc.chen@mediatek.com>
> > +*         Tiffany Lin <tiffany.lin@mediatek.com>
> > +*
> > +* This program is free software; you can redistribute it and/or modify
> > +* it under the terms of the GNU General Public License version 2 as
> > +* published by the Free Software Foundation.
> > +*
> > +* This program is distributed in the hope that it will be useful,
> > +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> > +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > +* GNU General Public License for more details.
> > +*/
> > +
> > +#ifndef _MTK_VCODEC_DRV_H_
> > +#define _MTK_VCODEC_DRV_H_
> > +
> > +#include <linux/platform_device.h>
> > +#include <linux/videodev2.h>
> > +#include <media/v4l2-ctrls.h>
> > +#include <media/v4l2-device.h>
> > +#include <media/v4l2-ioctl.h>
> > +#include <media/videobuf2-core.h>
> > +
> > +#include "mtk_vcodec_util.h"
> > +
> > +#define MTK_VCODEC_MAX_INSTANCES		10
> > +#define MTK_VCODEC_MAX_ENCODER_INSTANCES	3
> > +
> > +#define MTK_VCODEC_MAX_FRAME_SIZE	0x800000
> > +#define MTK_VIDEO_MAX_FRAME		32
> > +#define MTK_MAX_CTRLS			20
> > +
> > +#define MTK_VCODEC_DRV_NAME		"mtk_vcodec_drv"
> > +#define MTK_VCODEC_ENC_NAME		"mtk-vcodec-enc"
> > +#define MTK_PLATFORM_STR		"platform:mt8173"
> > +
> > +#define MTK_VENC_IRQ_STATUS_SPS          0x1
> > +#define MTK_VENC_IRQ_STATUS_PPS          0x2
> > +#define MTK_VENC_IRQ_STATUS_FRM          0x4
> > +#define MTK_VENC_IRQ_STATUS_DRAM         0x8
> > +#define MTK_VENC_IRQ_STATUS_PAUSE        0x10
> > +#define MTK_VENC_IRQ_STATUS_SWITCH       0x20
> > +
> > +#define MTK_VENC_IRQ_STATUS_OFFSET       0x05C
> > +#define MTK_VENC_IRQ_ACK_OFFSET          0x060
> > +
> > +#define MTK_VCODEC_MAX_PLANES		3
> > +
> > +#define VDEC_HW_ACTIVE	0x10
> > +#define VDEC_IRQ_CFG    0x11
> > +#define VDEC_IRQ_CLR    0x10
> > +
> > +#define VDEC_IRQ_CFG_REG	0xa4
> > +#define WAIT_INTR_TIMEOUT	1000
> > +
> > +/**
> > + * enum mtk_hw_reg_idx - MTK hw register base index
> > + */
> > +enum mtk_hw_reg_idx {
> > +	VDEC_SYS,
> > +	VDEC_MISC,
> > +	VDEC_LD,
> > +	VDEC_TOP,
> > +	VDEC_CM,
> > +	VDEC_AD,
> > +	VDEC_AV,
> > +	VDEC_PP,
> > +	VDEC_HWD,
> > +	VDEC_HWQ,
> > +	VDEC_HWB,
> > +	VDEC_HWG,
> > +	NUM_MAX_VDEC_REG_BASE,
> > +	VENC_SYS = NUM_MAX_VDEC_REG_BASE,
> > +	VENC_LT_SYS,
> > +	NUM_MAX_VCODEC_REG_BASE
> > +};
> > +
> > +/**
> > + * enum mtk_instance_type - The type of an MTK Vcodec instance.
> > + */
> > +enum mtk_instance_type {
> > +	MTK_INST_DECODER		= 0,
> > +	MTK_INST_ENCODER		= 1,
> > +};
> > +
> > +/**
> > + * enum mtk_instance_state - The state of an MTK Vcodec instance.
> > + * @MTK_STATE_FREE - default state when instance is created
> > + * @MTK_STATE_INIT - vcodec instance is initialized
> > + * @MTK_STATE_HEADER - vdec had sps/pps header parsed or venc
> > + *			had sps/pps header encoded
> > + * @MTK_STATE_FLUSH - vdec is flushing. Only used by decoder
> > + * @MTK_STATE_RES_CHANGE - vdec detect resolution change.
> > + * 			Only used by decoder
> > + * @MTK_STATE_ABORT - vcodec should be aborted
> > + */
> > +enum mtk_instance_state {
> > +	MTK_STATE_FREE = 0,
> > +	MTK_STATE_INIT = 1,
> > +	MTK_STATE_HEADER = 2,
> > +	MTK_STATE_FLUSH = 3,
> > +	MTK_STATE_RES_CHANGE = 4,
> > +	MTK_STATE_ABORT = 5,
> > +};
> > +
> > +/**
> > + * struct mtk_param_change - General encoding parameters type
> > + */
> > +enum mtk_encode_param {
> > +	MTK_ENCODE_PARAM_NONE = 0,
> > +	MTK_ENCODE_PARAM_BITRATE = (1 << 0),
> > +	MTK_ENCODE_PARAM_FRAMERATE = (1 << 1),
> > +	MTK_ENCODE_PARAM_INTRA_PERIOD = (1 << 2),
> > +	MTK_ENCODE_PARAM_FRAME_TYPE = (1 << 3),
> > +};
> > +
> > +/**
> > + * enum mtk_fmt_type - Type of the pixelformat
> > + * @MTK_FMT_FRAME - mtk vcodec raw frame
> > + */
> > +enum mtk_fmt_type {
> > +	MTK_FMT_DEC		= 0,
> > +	MTK_FMT_ENC		= 1,
> > +	MTK_FMT_FRAME		= 2,
> > +};
> > +
> > +/**
> > + * struct mtk_video_fmt - Structure used to store information about pixelformats
> > + */
> > +struct mtk_video_fmt {
> > +	u32 fourcc;
> > +	enum mtk_fmt_type type;
> > +	u32 num_planes;
> > +};
> > +
> > +/**
> > + * struct mtk_codec_framesizes - Structure used to store information about framesizes
> > + */
> > +struct mtk_codec_framesizes {
> > +	u32 fourcc;
> > +	struct	v4l2_frmsize_stepwise	stepwise;
> > +};
> > +
> > +/**
> > + * struct mtk_q_type - Type of queue
> > + */
> > +enum mtk_q_type {
> > +	MTK_Q_DATA_SRC		= 0,
> > +	MTK_Q_DATA_DST		= 1,
> > +};
> > +
> > +/**
> > + * struct mtk_q_data - Structure used to store information about queue
> > + * @colorspace	reserved for encoder
> > + * @field		reserved for encoder
> 
> struct is only partially documented here.
We will fix this in next version.

> 
> > + */
> > +struct mtk_q_data {
> > +	unsigned int		width;
> > +	unsigned int		height;
> > +	enum v4l2_field		field;
> > +	enum v4l2_colorspace	colorspace;
> > +	unsigned int		bytesperline[MTK_VCODEC_MAX_PLANES];
> > +	unsigned int		sizeimage[MTK_VCODEC_MAX_PLANES];
> > +	struct mtk_video_fmt	*fmt;
> > +};
> > +
> > +/**
> > + * struct mtk_enc_params - General encoding parameters
> > + * @bitrate - target bitrate
> > + * @num_b_frame - number of b frames between p-frame
> > + * @rc_frame - frame based rate control
> > + * @rc_mb - macroblock based rate control
> > + * @seq_hdr_mode - H.264 sequence header is encoded separately or joined with the first frame
> > + * @gop_size - group of picture size, it's used as the intra frame period
> > + * @framerate_num - frame rate numerator
> > + * @framerate_denom - frame rate denominator
> > + * @h264_max_qp - Max value for H.264 quantization parameter
> > + * @h264_profile - V4L2 defined H.264 profile
> > + * @h264_level - V4L2 defined H.264 level
> > + * @force_intra - force/insert intra frame
> > + * @skip_frame - encode in skip frame mode that use minimum number of bits
> 
> skip_frame isn't in the struct anymore.
We will remove this in next version.

> 
> > + */
> > +struct mtk_enc_params {
> > +	unsigned int	bitrate;
> > +	unsigned int	num_b_frame;
> > +	unsigned int	rc_frame;
> > +	unsigned int	rc_mb;
> > +	unsigned int	seq_hdr_mode;
> > +	unsigned int	gop_size;
> > +	unsigned int	framerate_num;
> > +	unsigned int	framerate_denom;
> > +	unsigned int	h264_max_qp;
> > +	unsigned int	h264_profile;
> > +	unsigned int	h264_level;
> > +	unsigned int	force_intra;
> > +};
> > +
> > +/**
> > + * struct mtk_vcodec_pm - Power management data structure
> > + */
> > +struct mtk_vcodec_pm {
> > +	struct clk	*vcodecpll;
> > +	struct clk	*univpll_d2;
> > +	struct clk	*clk_cci400_sel;
> > +	struct clk	*vdecpll;
> > +	struct clk	*vdec_sel;
> > +	struct clk	*vencpll_d2;
> > +	struct clk	*venc_sel;
> > +	struct clk	*univpll1_d2;
> > +	struct clk	*venc_lt_sel;
> > +	struct device	*larbvdec;
> > +	struct device	*larbvenc;
> > +	struct device	*larbvenclt;
> > +	struct device	*dev;
> > +	struct mtk_vcodec_dev *mtkdev;
> > +};
> > +
> > +
> > +/**
> > + * struct mtk_vcodec_ctx - Context (instance) private data.
> > + *
> > + * @type:		type of the instance - decoder or encoder
> > + * @dev:		pointer to the mtk_vcodec_dev of the device
> > + * @fh:			struct v4l2_fh
> > + * @m2m_ctx:		pointer to the v4l2_m2m_ctx of the context
> > + * @q_data:		store information of input and output queue
> > + *			of the context
> > + * @idx:		index of the context that this structure describes
> > + * @state:		state of the context
> > + * @param_change:	encode parameters
> > + * @enc_params:		encoding parameters
> > + * @colorspace:
> > + * @enc_if:		hoooked encoder driver interface
> > + * @drv_handle:		driver handle for specific decode/encode instance
> > + *
> > + * @picinfo:		store width/height of image and buffer and planes' size for decoder
> > + *			and encoder
> > + * @dpb_count:		count of the DPB buffers required by MTK Vcodec hw
> > + *
> > + * @int_cond:		variable used by the waitqueue
> > + * @int_type:		type of the last interrupt
> > + * @queue:		waitqueue that can be used to wait for this context to
> > + *			finish
> > + * @irq_status:		irq status
> > + *
> > + * @ctrl_hdl:		handler for v4l2 framework
> > + * @ctrls:		array of controls, used when adding controls to the
> > + *			v4l2 control framework
> > + *
> > + * @encode_work:	worker for the encoding
> > + */
> 
> These comments are out of date with the actual struct!
We will fix these in next version.

> 
> > +struct mtk_vcodec_ctx {
> > +	enum mtk_instance_type type;
> > +	struct mtk_vcodec_dev *dev;
> > +	struct v4l2_fh fh;
> > +	struct v4l2_m2m_ctx *m2m_ctx;
> > +	struct mtk_q_data q_data[2];
> > +	int idx;
> > +	enum mtk_instance_state state;
> > +	enum mtk_encode_param param_change;
> > +	struct mtk_enc_params enc_params;
> > +
> > +	struct venc_common_if *enc_if;
> > +	unsigned long drv_handle;
> > +
> > +
> > +	int int_cond;
> > +	int int_type;
> > +	wait_queue_head_t queue;
> > +	unsigned int irq_status;
> > +
> > +	struct v4l2_ctrl_handler ctrl_hdl;
> > +	struct v4l2_ctrl *ctrls[MTK_MAX_CTRLS];
> > +
> > +	struct work_struct encode_work;
> > +
> > +};
> > +
> > +/**
> > + * struct mtk_vcodec_dev - driver data
> > + * @v4l2_dev:		V4L2 device to register video devices for.
> > + * @vfd_enc:		Video device for encoder.
> > + *
> > + * @m2m_dev_enc:	m2m device for encoder.
> > + * @plat_dev:		platform device
> > + * @alloc_ctx:		VB2 allocator context
> > + *			(for allocations without kernel mapping).
> > + * @ctx:		array of driver contexts
> > + *
> > + * @curr_ctx:		The context that is waiting for codec hardware
> > + *
> > + * @reg_base:		Mapped address of MTK Vcodec registers.
> > + *
> > + * @instance_mask:	used to mark which contexts are opened
> > + * @num_instances:	counter of active MTK Vcodec instances
> > + *
> > + * @encode_workqueue:	encode work queue
> > + *
> > + * @int_cond:		used to identify interrupt condition happen
> > + * @int_type:		used to identify what kind of interrupt condition happen
> > + * @dev_mutex:		video_device lock
> > + * @queue:		waitqueue for waiting for completion of device commands
> > + *
> > + * @enc_irq:		h264 encoder irq resource
> > + * @enc_lt_irq:		vp8 encoder irq resource
> > + *
> > + * @enc_mutex:		encoder hardware lock.
> > + *
> > + * @pm:			power management control
> > + * @dec_capability:	used to identify decode capability, ex: 4k
> > + * @enc_capability:     used to identify encode capability
> > + */
> > +struct mtk_vcodec_dev {
> > +	struct v4l2_device	v4l2_dev;
> > +	struct video_device	*vfd_enc;
> > +
> > +	struct v4l2_m2m_dev	*m2m_dev_enc;
> > +	struct platform_device	*plat_dev;
> > +	struct platform_device 	*vpu_plat_dev;
> > +	struct vb2_alloc_ctx	*alloc_ctx;
> > +	struct mtk_vcodec_ctx	*ctx[MTK_VCODEC_MAX_INSTANCES];
> > +	int curr_ctx;
> > +	void __iomem		*reg_base[NUM_MAX_VCODEC_REG_BASE];
> > +
> > +	unsigned long	instance_mask[BITS_TO_LONGS(MTK_VCODEC_MAX_INSTANCES)];
> > +	int			num_instances;
> > +
> > +	struct workqueue_struct *encode_workqueue;
> > +
> > +	int			int_cond;
> > +	int			int_type;
> > +	struct mutex		dev_mutex;
> > +	wait_queue_head_t	queue;
> > +
> > +	int			enc_irq;
> > +	int			enc_lt_irq;
> > +
> > +	struct mutex		enc_mutex;
> > +
> > +	struct mtk_vcodec_pm	pm;
> > +	unsigned int		dec_capability;
> > +	unsigned int		enc_capability;
> > +};
> > +
> > +/**
> > + * struct mtk_vcodec_ctrl - information about controls to be registered.
> > + * @id:			Control ID.
> > + * @type:		Type of the control.
> > + * @name:		Human readable name of the control.
> > + * @minimum:		Minimum value of the control.
> > + * @maximum:		Maximum value of the control.
> > + * @step:		Control value increase step.
> > + * @menu_skip_mask:	Mask of invalid menu positions.
> > + * @default_value:	Initial value of the control.
> > + * @is_volatile:	Control is volatile.
> > + *
> > + * See also struct v4l2_ctrl_config.
> > + */
> > +struct mtk_vcodec_ctrl {
> > +	u32			id;
> > +	enum v4l2_ctrl_type	type;
> > +	u8			name[32];
> > +	s32			minimum;
> > +	s32			maximum;
> > +	s32			step;
> > +	u32			menu_skip_mask;
> > +	s32			default_value;
> > +	u8			is_volatile;
> > +};
> > +
> > +static inline struct mtk_vcodec_ctx *fh_to_ctx(struct v4l2_fh *fh)
> > +{
> > +	return container_of(fh, struct mtk_vcodec_ctx, fh);
> > +}
> > +
> > +static inline struct mtk_vcodec_ctx *ctrl_to_ctx(struct v4l2_ctrl *ctrl)
> > +{
> > +	return container_of(ctrl->handler, struct mtk_vcodec_ctx, ctrl_hdl);
> > +}
> > +
> > +extern const struct v4l2_ioctl_ops mtk_vdec_ioctl_ops;
> > +extern const struct v4l2_m2m_ops mtk_vdec_m2m_ops;
> > +extern const struct v4l2_ioctl_ops mtk_venc_ioctl_ops;
> > +extern const struct v4l2_m2m_ops mtk_venc_m2m_ops;
> > +
> > +#endif /* _MTK_VCODEC_DRV_H_ */
> > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
> > new file mode 100644
> > index 0000000..ee602fe
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
> > @@ -0,0 +1,1380 @@
> > +/*
> > +* Copyright (c) 2015 MediaTek Inc.
> > +* Author: PC Chen <pc.chen@mediatek.com>
> > +*         Tiffany Lin <tiffany.lin@mediatek.com>
> > +*
> > +* This program is free software; you can redistribute it and/or modify
> > +* it under the terms of the GNU General Public License version 2 as
> > +* published by the Free Software Foundation.
> > +*
> > +* This program is distributed in the hope that it will be useful,
> > +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> > +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > +* GNU General Public License for more details.
> > +*/
> > +
> > +#include <media/v4l2-event.h>
> > +#include <media/v4l2-mem2mem.h>
> > +#include <media/videobuf2-dma-contig.h>
> > +
> > +#include "mtk_vcodec_drv.h"
> > +#include "mtk_vcodec_enc.h"
> > +#include "mtk_vcodec_intr.h"
> > +#include "mtk_vcodec_util.h"
> > +#include "venc_drv_if.h"
> > +
> > +#define MTK_VENC_MIN_W	32
> > +#define MTK_VENC_MIN_H	32
> > +#define MTK_VENC_MAX_W	1920
> > +#define MTK_VENC_MAX_H	1080
> > +#define DFT_CFG_WIDTH	MTK_VENC_MIN_W
> > +#define DFT_CFG_HEIGHT	MTK_VENC_MIN_H
> > +
> > +static void mtk_venc_worker(struct work_struct *work);
> > +
> > +static struct mtk_video_fmt mtk_video_formats[] = {
> > +	{
> > +		.fourcc		= V4L2_PIX_FMT_YUV420,
> > +		.type		= MTK_FMT_FRAME,
> > +		.num_planes	= 3,
> > +	},
> > +	{
> > +		.fourcc		= V4L2_PIX_FMT_YVU420,
> > +		.type		= MTK_FMT_FRAME,
> > +		.num_planes	= 3,
> > +	},
> > +	{
> > +		.fourcc		= V4L2_PIX_FMT_NV12,
> > +		.type		= MTK_FMT_FRAME,
> > +		.num_planes	= 2,
> > +	},
> > +	{
> > +		.fourcc		= V4L2_PIX_FMT_NV21,
> > +		.type		= MTK_FMT_FRAME,
> > +		.num_planes	= 2,
> > +	},
> > +	{
> > +		.fourcc		= V4L2_PIX_FMT_YUV420M,
> > +		.type		= MTK_FMT_FRAME,
> > +		.num_planes	= 3,
> > +	},
> > +	{
> > +		.fourcc		= V4L2_PIX_FMT_YVU420M,
> > +		.type		= MTK_FMT_FRAME,
> > +		.num_planes	= 3,
> > +	},
> > +	{
> > +		.fourcc 	= V4L2_PIX_FMT_NV12M,
> > +		.type		= MTK_FMT_FRAME,
> > +		.num_planes	= 2,
> > +	},
> > +	{
> > +		.fourcc		= V4L2_PIX_FMT_NV21M,
> > +		.type		= MTK_FMT_FRAME,
> > +		.num_planes	= 2,
> > +	},
> > +	{
> > +		.fourcc		= V4L2_PIX_FMT_H264,
> > +		.type		= MTK_FMT_ENC,
> > +		.num_planes	= 1,
> > +	},
> > +	{
> > +		.fourcc		= V4L2_PIX_FMT_VP8,
> > +		.type		= MTK_FMT_ENC,
> > +		.num_planes	= 1,
> > +	},
> > +};
> > +
> > +#define NUM_FORMATS ARRAY_SIZE(mtk_video_formats)
> > +
> > +static const struct mtk_codec_framesizes mtk_venc_framesizes[] = {
> > +	{
> > +		.fourcc	= V4L2_PIX_FMT_H264,
> > +		.stepwise = {  160, 1920, 16, 128, 1088, 16 },
> > +	},
> > +	{
> > +		.fourcc = V4L2_PIX_FMT_VP8,
> > +		.stepwise = {  160, 1920, 16, 128, 1088, 16 },
> > +	},
> > +};
> > +
> > +#define NUM_SUPPORTED_FRAMESIZE ARRAY_SIZE(mtk_venc_framesizes)
> > +
> > +static int vidioc_venc_s_ctrl(struct v4l2_ctrl *ctrl)
> > +{
> > +	struct mtk_vcodec_ctx *ctx = ctrl_to_ctx(ctrl);
> > +	struct mtk_enc_params *p = &ctx->enc_params;
> > +	int ret = 0;
> > +
> > +	switch (ctrl->id) {
> > +	case V4L2_CID_MPEG_VIDEO_BITRATE:
> > +		mtk_v4l2_debug(2, "V4L2_CID_MPEG_VIDEO_BITRATE val = %d",
> > +			ctrl->val);
> > +		p->bitrate = ctrl->val;
> > +		ctx->param_change |= MTK_ENCODE_PARAM_BITRATE;
> > +		break;
> > +	case V4L2_CID_MPEG_VIDEO_B_FRAMES:
> > +		mtk_v4l2_debug(2, "V4L2_CID_MPEG_VIDEO_B_FRAMES val = %d",
> > +			ctrl->val);
> > +		p->num_b_frame = ctrl->val;
> > +		break;
> > +	case V4L2_CID_MPEG_VIDEO_FRAME_RC_ENABLE:
> > +		mtk_v4l2_debug(2, "V4L2_CID_MPEG_VIDEO_FRAME_RC_ENABLE val = %d",
> > +			ctrl->val);
> > +		p->rc_frame = ctrl->val;
> > +		break;
> > +	case V4L2_CID_MPEG_VIDEO_H264_MAX_QP:
> > +		mtk_v4l2_debug(2, "V4L2_CID_MPEG_VIDEO_H264_MAX_QP val = %d",
> > +			ctrl->val);
> > +		p->h264_max_qp = ctrl->val;
> > +		break;
> > +	case V4L2_CID_MPEG_VIDEO_HEADER_MODE:
> > +		mtk_v4l2_debug(2, "V4L2_CID_MPEG_VIDEO_HEADER_MODE val = %d",
> > +			ctrl->val);
> > +		p->seq_hdr_mode = ctrl->val;
> > +		break;
> > +	case V4L2_CID_MPEG_VIDEO_MB_RC_ENABLE:
> > +		mtk_v4l2_debug(2, "V4L2_CID_MPEG_VIDEO_MB_RC_ENABLE val = %d",
> > +			ctrl->val);
> > +		p->rc_mb = ctrl->val;
> > +		break;
> > +	case V4L2_CID_MPEG_VIDEO_H264_PROFILE:
> > +		mtk_v4l2_debug(2, "V4L2_CID_MPEG_VIDEO_H264_PROFILE val = %d",
> > +			ctrl->val);
> > +		p->h264_profile = ctrl->val;
> > +		break;
> > +	case V4L2_CID_MPEG_VIDEO_H264_LEVEL:
> > +		mtk_v4l2_debug(2, "V4L2_CID_MPEG_VIDEO_H264_LEVEL val = %d",
> > +			ctrl->val);
> > +		p->h264_level = ctrl->val;
> > +		break;
> > +	case V4L2_CID_MPEG_VIDEO_H264_I_PERIOD:
> > +		mtk_v4l2_debug(2, "V4L2_CID_MPEG_VIDEO_H264_I_PERIOD val = %d",
> > +			ctrl->val);
> > +	case V4L2_CID_MPEG_VIDEO_GOP_SIZE:
> > +		mtk_v4l2_debug(2, "V4L2_CID_MPEG_VIDEO_GOP_SIZE val = %d",
> > +			ctrl->val);
> > +		p->gop_size = ctrl->val;
> > +		ctx->param_change |= MTK_ENCODE_PARAM_INTRA_PERIOD;
> > +		break;
> > +	case V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME:
> > +		mtk_v4l2_debug(2, "V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME");
> > +		p->force_intra = 1;
> > +		ctx->param_change |= MTK_ENCODE_PARAM_FRAME_TYPE;
> > +		break;
> > +	default:
> > +		ret = -EINVAL;
> > +		break;
> > +	}
> > +
> > +	return ret;
> > +}
> > +
> > +static const struct v4l2_ctrl_ops mtk_vcodec_enc_ctrl_ops = {
> > +	.s_ctrl = vidioc_venc_s_ctrl,
> > +};
> > +
> > +static int vidioc_enum_fmt(struct file *file, struct v4l2_fmtdesc *f,
> > +			   bool out)
> > +{
> > +	struct mtk_video_fmt *fmt;
> > +	int i, j = 0;
> > +
> > +	for (i = 0; i < NUM_FORMATS; ++i) {
> > +		if (out && mtk_video_formats[i].type != MTK_FMT_FRAME)
> > +			continue;
> > +		else if (!out && mtk_video_formats[i].type != MTK_FMT_ENC)
> 
> No 'else' needed here.
> 
We will remote it in next version.

> > +			continue;
> > +
> > +		if (j == f->index) {
> > +			fmt = &mtk_video_formats[i];
> > +			f->pixelformat = fmt->fourcc;
> > +			return 0;
> > +		}
> > +		++j;
> > +	}
> > +
> > +	return -EINVAL;
> > +}
> > +
> > +static int vidioc_enum_framesizes(struct file *file, void *fh,
> > +				  struct v4l2_frmsizeenum *fsize)
> > +{
> > +	int i = 0;
> > +
> > +	for (i = 0; i < NUM_SUPPORTED_FRAMESIZE; ++i) {
> > +		if (fsize->pixel_format != mtk_venc_framesizes[i].fourcc)
> > +			continue;
> > +
> > +		if (!fsize->index) {
> > +			fsize->type = V4L2_FRMSIZE_TYPE_STEPWISE;
> > +			fsize->stepwise = mtk_venc_framesizes[i].stepwise;
> > +			return 0;
> > +		}
> > +	}
> > +
> > +	return -EINVAL;
> > +}
> > +
> > +static int vidioc_enum_fmt_vid_cap_mplane(struct file *file, void *pirv,
> > +					  struct v4l2_fmtdesc *f)
> > +{
> > +	return vidioc_enum_fmt(file, f, false);
> > +}
> > +
> > +static int vidioc_enum_fmt_vid_out_mplane(struct file *file, void *prov,
> > +					  struct v4l2_fmtdesc *f)
> > +{
> > +	return vidioc_enum_fmt(file, f, true);
> > +}
> > +
> > +static int vidioc_venc_querycap(struct file *file, void *priv,
> > +				struct v4l2_capability *cap)
> > +{
> > +        strlcpy(cap->driver, MTK_VCODEC_ENC_NAME, strlen(MTK_VCODEC_ENC_NAME));
> 
> This should be:
> 
> 	strlcpy(cap->driver, MTK_VCODEC_ENC_NAME, sizeof(cap->driver));
> 
We will fix this.

> > +        cap->driver[strlen(MTK_VCODEC_ENC_NAME)]=0;
> 
> No need to terminate with 0, strlcpy does that already.
> 
> > +        strlcpy(cap->bus_info, MTK_PLATFORM_STR, strlen(MTK_PLATFORM_STR));
> > +        cap->bus_info[strlen(MTK_PLATFORM_STR)]=0;
> > +        strlcpy(cap->card, MTK_PLATFORM_STR, strlen(MTK_PLATFORM_STR));
> > +        cap->card[strlen(MTK_PLATFORM_STR)]=0;
> 
> Ditto for these two fields. Note that use use spaces instead of a tab here as
> well. Please fix. (checkpatch should have complained about that!)
> 
We will fix this.

> > +
> > +	/*
> > +	 * This is only a mem-to-mem video device. The capture and output
> > +	 * device capability flags are left only for backward compatibility
> > +	 * and are scheduled for removal.
> > +	 */
> 
> Comment is out of date.
> 
We will remove this.

> > +	cap->device_caps  = V4L2_CAP_VIDEO_M2M_MPLANE | V4L2_CAP_STREAMING;
> > +	cap->capabilities = cap->device_caps | V4L2_CAP_DEVICE_CAPS;
> > +
> > +	return 0;
> > +}
> > +static int vidioc_venc_s_parm(struct file *file, void *priv,
> > +			      struct v4l2_streamparm *a)
> > +{
> > +	struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
> > +
> > +	if (a->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
> > +		ctx->enc_params.framerate_num =
> > +			a->parm.output.timeperframe.denominator;
> > +		ctx->enc_params.framerate_denom =
> > +			a->parm.output.timeperframe.numerator;
> > +		ctx->param_change |= MTK_ENCODE_PARAM_FRAMERATE;
> > +	} else {
> > +		return -EINVAL;
> > +	}
> 
> I'd invert the test:
> 
> 	if (a->type != V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE)
> 		return -EINVAL;
> 
> and now you can just set ctx->enc_params.
> 
We will fix this in next version.


> > +	return 0;
> > +}
> 
> And if there is an s_parm, then there should be a g_parm as well!
> 
Now our driver does not support g_parm, our use cases do not use g_parm
too. 
Do we need to add g_parm at this moment? Or we could add it when we need
g_parm?

> > +
> > +static struct mtk_q_data *mtk_venc_get_q_data(struct mtk_vcodec_ctx *ctx,
> > +					      enum v4l2_buf_type type)
> > +{
> > +	if (V4L2_TYPE_IS_OUTPUT(type))
> > +		return &ctx->q_data[MTK_Q_DATA_SRC];
> > +
> > +	return &ctx->q_data[MTK_Q_DATA_DST];
> > +}
> > +
> > +static struct mtk_video_fmt *mtk_venc_find_format(struct v4l2_format *f)
> > +{
> > +	struct mtk_video_fmt *fmt;
> > +	unsigned int k;
> > +
> > +	for (k = 0; k < NUM_FORMATS; k++) {
> > +		fmt = &mtk_video_formats[k];
> > +		if (fmt->fourcc == f->fmt.pix.pixelformat)
> > +			return fmt;
> > +	}
> > +
> > +	return NULL;
> > +}
> > +
> > +static void mtk_vcodec_enc_calc_src_size(
> > +	unsigned int num_planes, unsigned int pic_width,
> > +	unsigned int pic_height, unsigned int sizeimage[],
> > +	unsigned int bytesperline[])
> > +{
> > +	unsigned int y_pitch_w_div16;
> > +	unsigned int c_pitch_w_div16;
> > +
> > +	y_pitch_w_div16 = ALIGN(pic_width, 16) >> 4;
> > +	c_pitch_w_div16 = ALIGN(pic_width, 16) >> 4;
> > +
> > +	if (num_planes == 2) {
> > +		sizeimage[0] =
> > +			(y_pitch_w_div16) * (((pic_height + 31) / 32) * 2) * 256 +
> > +			((y_pitch_w_div16 % 8 == 0) ? 0 : ((ALIGN(pic_width, 16) * 2) * 16));
> > +
> > +		sizeimage[1] =
> > +			(c_pitch_w_div16) * (((pic_height + 31) / 32) * 2) * 128 +
> > +			((c_pitch_w_div16 % 8 == 0) ? 0 : (ALIGN(pic_width, 16) * 16));
> > +
> > +		sizeimage[2] = 0;
> > +
> > +		bytesperline[0] = ALIGN(pic_width, 16);
> > +		bytesperline[1] = ALIGN(pic_width, 16);
> > +		bytesperline[2] = 0;
> > +
> > +	} else {
> > +		sizeimage[0] =
> > +			(y_pitch_w_div16) * (((pic_height + 31) / 32) * 2) * 256 +
> > +			((y_pitch_w_div16 % 8 == 0) ? 0 : ((ALIGN(pic_width, 16) * 2) * 16));
> > +
> > +		sizeimage[1] =
> > +			(c_pitch_w_div16) * (((pic_height + 31) / 32) * 2) * 64 +
> > +			((c_pitch_w_div16 % 8 == 0) ? 0 : ((ALIGN(pic_width, 16) / 2) * 16));
> > +
> > +		sizeimage[2] =
> > +			(c_pitch_w_div16) * (((pic_height + 31) / 32) * 2) * 64 +
> > +			((c_pitch_w_div16 % 8 == 0) ? 0 : ((ALIGN(pic_width, 16) / 2) * 16));
> > +
> > +		bytesperline[0] = ALIGN(pic_width, 16);
> > +		bytesperline[1] = ALIGN(pic_width, 16) / 2;
> > +		bytesperline[2] = ALIGN(pic_width, 16) / 2;
> > +	}
> > +}
> > +
> > +static int vidioc_try_fmt(struct v4l2_format *f, struct mtk_video_fmt *fmt)
> > +{
> > +	struct v4l2_pix_format_mplane *pix_fmt_mp = &f->fmt.pix_mp;
> > +
> > +	/* V4L2 specification suggests the driver corrects the format struct
> > +	  * if any of the dimensions is unsupported */
> > +        if (pix_fmt_mp->height < MTK_VENC_MIN_H)
> > +                pix_fmt_mp->height = MTK_VENC_MIN_H;
> > +        else if (pix_fmt_mp->height > MTK_VENC_MAX_H)
> > +                pix_fmt_mp->height = MTK_VENC_MAX_H;
> > +
> > +        if (pix_fmt_mp->width < MTK_VENC_MIN_W)
> > +                pix_fmt_mp->width = MTK_VENC_MIN_W;
> > +        else if (pix_fmt_mp->width > MTK_VENC_MAX_W)
> > +                pix_fmt_mp->width = MTK_VENC_MAX_W;
> 
> Use the clamp macro for this.
> 
We will fix this in next version.

> > +
> > +        pix_fmt_mp->field = V4L2_FIELD_NONE;
> > +
> > +	if (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
> > +		int size = pix_fmt_mp->height * pix_fmt_mp->width;
> 
> Add an empty line.
> 
We will fix this in next version.

> > +		if (fmt->num_planes != pix_fmt_mp->num_planes)
> > +			pix_fmt_mp->num_planes = fmt->num_planes;
> > +
> > +		if(pix_fmt_mp->plane_fmt[0].sizeimage != size)
> 
> Add space after if. Again, checkpatch should have complained about that.
> Please run checkpatch first before posting the next version.
> 
We will run checkpatch before posting the next version.

> > +			pix_fmt_mp->plane_fmt[0].sizeimage = size;
> > + 		pix_fmt_mp->plane_fmt[0].bytesperline = 0;
> > +		memset(&(pix_fmt_mp->plane_fmt[0].reserved[0]), 0x0,
> > +			sizeof(pix_fmt_mp->plane_fmt[0].reserved));
> > +	} else if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
> > +		int i;
> > +		unsigned int sizeimage[VIDEO_MAX_PLANES];
> > +		unsigned int bytesperline[VIDEO_MAX_PLANES];
> > +
> > +		v4l_bound_align_image(&pix_fmt_mp->width, 8, 1920, 1,
> > +				      &pix_fmt_mp->height, 4, 1080, 1, 0);
> > +
> > +		if (fmt->num_planes != pix_fmt_mp->num_planes)
> > +			pix_fmt_mp->num_planes = fmt->num_planes;
> > +
> > +		mtk_vcodec_enc_calc_src_size(pix_fmt_mp->num_planes,
> > +					pix_fmt_mp->width,
> > +					pix_fmt_mp->height,
> > +					sizeimage, bytesperline);
> > +
> > +		for (i=0; i < pix_fmt_mp->num_planes; i++) {
> 
> Spaces around '='.
> 
We will fix this.

> > +			pix_fmt_mp->plane_fmt[i].sizeimage = sizeimage[i];
> > +			pix_fmt_mp->plane_fmt[i].bytesperline = bytesperline[i];
> > +			memset(&(pix_fmt_mp->plane_fmt[i].reserved[0]), 0x0,
> > +				sizeof(pix_fmt_mp->plane_fmt[0].reserved));
> > +		}
> > +	} else {
> > +		return -EINVAL;
> > +	}
> > +
> > +	pix_fmt_mp->flags = 0;
> > +	pix_fmt_mp->ycbcr_enc = 0;
> > +	pix_fmt_mp->quantization = 0;
> > +	pix_fmt_mp->xfer_func = 0;
> > +	memset(&pix_fmt_mp->reserved[0], 0x0, sizeof(pix_fmt_mp->reserved));
> > +
> > +	return 0;
> > +}
> > +
> > +static void mtk_venc_set_param(struct mtk_vcodec_ctx *ctx, void *param)
> > +{
> > +	struct venc_enc_prm *p = (struct venc_enc_prm *)param;
> > +	struct mtk_q_data *q_data_src = &ctx->q_data[MTK_Q_DATA_SRC];
> > +	struct mtk_enc_params *enc_params = &ctx->enc_params;
> > +	unsigned int frame_rate;
> > +
> > +	frame_rate = enc_params->framerate_num / enc_params->framerate_denom;
> > +
> > +	switch (q_data_src->fmt->fourcc) {
> > +	case V4L2_PIX_FMT_YUV420:
> > +	case V4L2_PIX_FMT_YUV420M:
> > +		p->input_fourcc = VENC_YUV_FORMAT_420;
> > +		break;
> > +	case V4L2_PIX_FMT_YVU420:
> > +	case V4L2_PIX_FMT_YVU420M:
> > +		p->input_fourcc = VENC_YUV_FORMAT_YV12;
> > +		break;
> > +	case V4L2_PIX_FMT_NV12:
> > +	case V4L2_PIX_FMT_NV12M:
> > +		p->input_fourcc = VENC_YUV_FORMAT_NV12;
> > +		break;
> > +	case V4L2_PIX_FMT_NV21:
> > +	case V4L2_PIX_FMT_NV21M:
> > +		p->input_fourcc = VENC_YUV_FORMAT_NV21;
> > +		break;
> > +	}
> > +	p->h264_profile = enc_params->h264_profile;
> > +	p->h264_level = enc_params->h264_level;
> > +	p->width = q_data_src->width;
> > +	p->height = q_data_src->height;
> > +	p->buf_width = q_data_src->bytesperline[0];
> > +	p->buf_height = ((q_data_src->height + 0xf) & (~0xf));
> > +	p->frm_rate = frame_rate;
> > +	p->intra_period = enc_params->gop_size;
> > +	p->bitrate = enc_params->bitrate;
> > +
> > +	ctx->param_change = MTK_ENCODE_PARAM_NONE;
> > +
> > +	mtk_v4l2_debug(1, "fmt 0x%x, P/L %d/%d, w/h %d/%d, buf %d/%d, fps/bps %d/%d, gop %d",
> > +		       p->input_fourcc, p->h264_profile, p->h264_level,
> > +		       p->width, p->height, p->buf_width, p->buf_height,
> > +		       p->frm_rate, p->bitrate, p->intra_period);
> > +}
> > +
> > +static int vidioc_venc_s_fmt(struct file *file, void *priv,
> > +			     struct v4l2_format *f)
> 
> I am not convinced that combining capture and output in one function is the
> most readable. I think you are better off with separate functions. Too much is
> different between the two.
> 
Got it. We will separate it to vidioc_venc_s_fmt_cap and
vidioc_venc_s_fmt_out.

> > +{
> > +	struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
> > +	struct v4l2_pix_format_mplane *pix_fmt_mp = &f->fmt.pix_mp;
> > +	struct vb2_queue *vq;
> > +	struct mtk_q_data *q_data;
> > +	struct venc_enc_prm param;
> > +	int i, ret;
> > +	struct mtk_video_fmt *fmt;
> > +
> > +	vq = v4l2_m2m_get_vq(ctx->m2m_ctx, f->type);
> > +	if (!vq) {
> > +		mtk_v4l2_err("fail to get vq\n");
> > +		return -EINVAL;
> > +	}
> > +
> > +	if (vb2_is_busy(vq)) {
> > +		mtk_v4l2_err("queue busy\n");
> > +		return -EBUSY;
> > +	}
> > +
> > +	q_data = mtk_venc_get_q_data(ctx, f->type);
> > +	if (!q_data) {
> > +		mtk_v4l2_err("fail to get q data\n");
> > +		return -EINVAL;
> > +	}
> > +
> > +	fmt = mtk_venc_find_format(f);
> > +	if (!fmt) {
> > +		if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
> > +			f->fmt.pix.pixelformat = mtk_video_formats[0].fourcc;
> > +			fmt = mtk_venc_find_format(f);
> > +		} else {
> > +			f->fmt.pix.pixelformat = mtk_video_formats[8].fourcc;
> > +			fmt = mtk_venc_find_format(f);
> > +		}
> > +	}
> > +
> > +	q_data->fmt = fmt;
> > +	ret = vidioc_try_fmt(f, q_data->fmt);
> > +	if (ret)
> > +		return ret;
> > +
> > +	q_data->width		= f->fmt.pix_mp.width;
> > +	q_data->height		= f->fmt.pix_mp.height;
> > +	q_data->field		= f->fmt.pix_mp.field;
> > +
> > +	if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
> > +		q_data->colorspace = f->fmt.pix_mp.colorspace;
> > +		ctx->q_data[MTK_Q_DATA_SRC].bytesperline[0] =
> > +			ALIGN(q_data->width, 16);
> > +
> > +		if (q_data->fmt->num_planes == 2) {
> > +			ctx->q_data[MTK_Q_DATA_SRC].bytesperline[1] =
> > +				ALIGN(q_data->width, 16);
> > +			ctx->q_data[MTK_Q_DATA_SRC].bytesperline[2] = 0;
> > +		} else {
> > +			ctx->q_data[MTK_Q_DATA_SRC].bytesperline[1] =
> > +				ALIGN(q_data->width, 16) / 2;
> > +			ctx->q_data[MTK_Q_DATA_SRC].bytesperline[2] =
> > +				ALIGN(q_data->width, 16) / 2;
> > +		}
> > +
> > +		memset(&param, 0, sizeof(param));
> > +		mtk_venc_set_param(ctx, &param);
> > +		if (ctx->state == MTK_STATE_INIT) {
> > +			ret = venc_if_set_param(ctx,
> > +						VENC_SET_PARAM_ENC,
> > +						&param);
> > +			if (ret)
> > +				mtk_v4l2_err("venc_if_set_param failed=%d\n",
> > +						ret);
> > +
> > +			/* Get codec driver advice sizeimage from vpu */
> > +			for (i = 0; i < MTK_VCODEC_MAX_PLANES; i++) {
> > +				q_data->sizeimage[i] = param.sizeimage[i];
> > +				pix_fmt_mp->plane_fmt[i].sizeimage =
> > +					param.sizeimage[i];
> > +			}
> > +			q_data->bytesperline[0] =
> > +				pix_fmt_mp->plane_fmt[0].bytesperline;
> > +			q_data->bytesperline[1] =
> > +				pix_fmt_mp->plane_fmt[1].bytesperline;
> > +			q_data->bytesperline[2] =
> > +				pix_fmt_mp->plane_fmt[2].bytesperline;
> > +		} else {
> > +			for (i = 0; i < MTK_VCODEC_MAX_PLANES; i++) {
> > +				q_data->sizeimage[i] =
> > +					pix_fmt_mp->plane_fmt[i].sizeimage;
> > +				q_data->bytesperline[i] =
> > +					pix_fmt_mp->plane_fmt[i].bytesperline;
> > +			}
> > +		}
> > +	} else if (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE){
> > +		for (i = 0; i < f->fmt.pix_mp.num_planes; i++) {
> > +			struct v4l2_plane_pix_format	*plane_fmt;
> > +
> > +			plane_fmt = &f->fmt.pix_mp.plane_fmt[i];
> > +			q_data->bytesperline[i]	= plane_fmt->bytesperline;
> > +			q_data->sizeimage[i]	= plane_fmt->sizeimage;
> > +		}
> > +
> > +		if (ctx->state == MTK_STATE_FREE) {
> > +			ret = venc_if_create(ctx, q_data->fmt->fourcc);
> > +			if (ret) {
> > +				mtk_v4l2_err("venc_if_create failed=%d, codec type=%x\n",
> > +					ret, q_data->fmt->fourcc);
> > +				return 0;
> > +			}
> > +
> > +			ctx->state = MTK_STATE_INIT;
> > +		}
> > +	}
> > +
> > +	return 0;
> > +}
> > +
> > +static int vidioc_venc_g_fmt(struct file *file, void *priv,
> > +			     struct v4l2_format *f)
> > +{
> > +	struct v4l2_pix_format_mplane *pix = &f->fmt.pix_mp;
> > +	struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
> > +	struct vb2_queue *vq;
> > +	struct mtk_q_data *q_data;
> > +	int i;
> > +
> > +	vq = v4l2_m2m_get_vq(ctx->m2m_ctx, f->type);
> > +	if (!vq)
> > +		return -EINVAL;
> > +
> > +	q_data = mtk_venc_get_q_data(ctx, f->type);
> > +
> > +	pix->width = q_data->width;
> > +	pix->height = q_data->height;
> > +	pix->pixelformat = q_data->fmt->fourcc;
> > +	pix->field = q_data->field;
> > +	pix->colorspace = q_data->colorspace;
> > +	pix->num_planes = q_data->fmt->num_planes;
> > +	for (i = 0; i < pix->num_planes; i++) {
> > +		pix->plane_fmt[i].bytesperline = q_data->bytesperline[i];
> > +		pix->plane_fmt[i].sizeimage = q_data->sizeimage[i];
> > +		memset(&(pix->plane_fmt[i].reserved[0]), 0x0,
> > +			sizeof(pix->plane_fmt[i].reserved));
> > +	}
> > +	pix->flags = 0;
> > +	pix->ycbcr_enc = 0;
> > +	pix->quantization = 0;
> > +	pix->xfer_func = 0;
> > +	memset(&pix->reserved[0], 0x0, sizeof(pix->reserved));
> > +
> > +	return 0;
> > +}
> > +
> > +static int vidioc_try_fmt_vid_cap_mplane(struct file *file, void *priv,
> > +                                  struct v4l2_format *f)
> > +{
> > +        struct mtk_video_fmt *fmt;
> > +	struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
> > +
> > +        fmt = mtk_venc_find_format(f);
> > +        if (!fmt) {
> > +                f->fmt.pix.pixelformat = mtk_video_formats[8].fourcc;
> > +                fmt = mtk_venc_find_format(f);
> > +        }
> > +        if (fmt->type != MTK_FMT_ENC) {
> > +		mtk_v4l2_err("Fourcc format (0x%08x) invalid.\n",
> > +			     f->fmt.pix.pixelformat);
> > +		return -EINVAL;
> > +        }
> > +        f->fmt.pix_mp.colorspace = ctx->q_data[MTK_Q_DATA_SRC].colorspace;
> > +
> > +        return vidioc_try_fmt(f, fmt);
> > +}
> > +
> > +static int vidioc_try_fmt_vid_out_mplane(struct file *file, void *priv,
> > +                                  struct v4l2_format *f)
> > +{
> > +        struct mtk_video_fmt *fmt;
> > +
> > +        fmt = mtk_venc_find_format(f);
> > +        if (!fmt) {
> > +                f->fmt.pix.pixelformat = mtk_video_formats[0].fourcc;
> > +                fmt = mtk_venc_find_format(f);
> > +        }
> > +        if (!(fmt->type & MTK_FMT_FRAME)) {
> > +		mtk_v4l2_err("Fourcc format (0x%08x) invalid.\n",
> > +			     f->fmt.pix.pixelformat);
> > +		return -EINVAL;
> > +        }
> > +        if (!f->fmt.pix_mp.colorspace)
> > +                f->fmt.pix_mp.colorspace = V4L2_COLORSPACE_REC709;
> > +
> > +        return vidioc_try_fmt(f, fmt);
> > +}
> > +
> > +static int vidioc_venc_g_s_selection(struct file *file, void *priv,
> > +                                struct v4l2_selection *s)
> 
> Why support s_selection if you can only return the current width and height?
> And why support g_selection if you can't change the selection?
> 
> In other words, why implement this at all?
> 
> Unless I am missing something here, I would just drop this.
> 
Now our driver do not support these capabilities, but userspace app will
check whether g/s_crop are implemented when using encoder.
Because g/s_crop are deprecated as you mentioned in previous v2 review
comments. We change to use g_s_selection.
We will check if we could add this capability.

> > +{
> > +	struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
> > +	struct mtk_q_data *q_data;
> > +
> > +	if (V4L2_TYPE_IS_OUTPUT(s->type)) {
> > +		if (s->target !=  V4L2_SEL_TGT_COMPOSE)
> > +			return -EINVAL;
> > +	} else {
> > +		if (s->target != V4L2_SEL_TGT_CROP)
> > +			return -EINVAL;
> > +	}
> > +
> > +	if (s->r.left || s->r.top)
> > +		return -EINVAL;
> > +
> > +	q_data = mtk_venc_get_q_data(ctx, s->type);
> > +	if (!q_data)
> > +		return -EINVAL;
> > +
> > +	s->r.width = q_data->width;
> > +	s->r.height = q_data->height;
> > +
> > +	return 0;
> > +}
> > +
> > +
> > +static int vidioc_venc_qbuf(struct file *file, void *priv,
> > +			    struct v4l2_buffer *buf)
> > +{
> > +
> > +	struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
> > +
> > +	if (ctx->state == MTK_STATE_ABORT) {
> > +		mtk_v4l2_err("[%d] Call on QBUF after unrecoverable error\n", ctx->idx);
> > +		return -EIO;
> > +	}
> > +
> > +	return v4l2_m2m_qbuf(file, ctx->m2m_ctx, buf);
> > +}
> > +
> > +static int vidioc_venc_dqbuf(struct file *file, void *priv,
> > +			     struct v4l2_buffer *buf)
> > +{
> > +	struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
> > +	if (ctx->state == MTK_STATE_ABORT) {
> > +		mtk_v4l2_err("[%d] Call on QBUF after unrecoverable error\n", ctx->idx);
> > +		return -EIO;
> > +	}
> > +
> > +	return v4l2_m2m_dqbuf(file, ctx->m2m_ctx, buf);
> > +}
> > +
> > +
> > +const struct v4l2_ioctl_ops mtk_venc_ioctl_ops = {
> > +	.vidioc_streamon		= v4l2_m2m_ioctl_streamon,
> > +	.vidioc_streamoff		= v4l2_m2m_ioctl_streamoff,
> > +
> > +	.vidioc_reqbufs			= v4l2_m2m_ioctl_reqbufs,
> > +	.vidioc_querybuf		= v4l2_m2m_ioctl_querybuf,
> > +	.vidioc_qbuf			= vidioc_venc_qbuf,
> > +	.vidioc_dqbuf			= vidioc_venc_dqbuf,
> > +
> > +	.vidioc_querycap		= vidioc_venc_querycap,
> > +	.vidioc_enum_fmt_vid_cap_mplane = vidioc_enum_fmt_vid_cap_mplane,
> > +	.vidioc_enum_fmt_vid_out_mplane = vidioc_enum_fmt_vid_out_mplane,
> > +	.vidioc_enum_framesizes		= vidioc_enum_framesizes,
> > +
> > +	.vidioc_try_fmt_vid_cap_mplane	= vidioc_try_fmt_vid_cap_mplane,
> > +	.vidioc_try_fmt_vid_out_mplane	= vidioc_try_fmt_vid_out_mplane,
> > +	.vidioc_expbuf			= v4l2_m2m_ioctl_expbuf,
> 
> Please add vidioc_create_bufs and vidioc_prepare_buf as well.
> 

Currently we do not support these use cases, do we need to add
vidioc_create_bufs and vidioc_prepare_buf now?


> > +	.vidioc_subscribe_event		= v4l2_ctrl_subscribe_event,
> > +	.vidioc_unsubscribe_event	= v4l2_event_unsubscribe,
> > +
> > +	.vidioc_s_parm			= vidioc_venc_s_parm,
> > +
> > +	.vidioc_s_fmt_vid_cap_mplane	= vidioc_venc_s_fmt,
> > +	.vidioc_s_fmt_vid_out_mplane	= vidioc_venc_s_fmt,
> > +
> > +	.vidioc_g_fmt_vid_cap_mplane	= vidioc_venc_g_fmt,
> > +	.vidioc_g_fmt_vid_out_mplane	= vidioc_venc_g_fmt,
> > +
> > +	.vidioc_g_selection		= vidioc_venc_g_s_selection,
> > +	.vidioc_s_selection		= vidioc_venc_g_s_selection,
> > +};
> > +
> > +static int vb2ops_venc_queue_setup(struct vb2_queue *vq,
> > +				   const void *parg,
> > +				   unsigned int *nbuffers,
> > +				   unsigned int *nplanes,
> > +				   unsigned int sizes[], void *alloc_ctxs[])
> > +{
> > +	struct mtk_vcodec_ctx *ctx = vb2_get_drv_priv(vq);
> > +	struct mtk_q_data *q_data;
> > +
> > +	q_data = mtk_venc_get_q_data(ctx, vq->type);
> > +
> > +	if (*nbuffers < 1)
> > +		*nbuffers = 1;
> 
> *nbuffers can never be 0, so no need to check.
> 
We will remove this in next version.

> > +	if (*nbuffers > MTK_VIDEO_MAX_FRAME)
> 
> That should be (q->num_buffers + *nbuffers > MTK_VIDEO_MAX_FRAME)
> 
We will fix this in next version.

> > +		*nbuffers = MTK_VIDEO_MAX_FRAME;
> 
> and *nbuffers = MTK_VIDEO_MAX_FRAME - q->num_buffers;
> 
> And you will need a check if q->num_buffers == MTK_VIDEO_MAX_FRAME) too
> (otherwise *nbuffers could become <= 0).
> 
> In order to correctly handle create_bufs you will need to make a few changes
> here. Read the queue_setup description in include/media/videobuf2-core.h and
> look at queue_setup() in Documentation/video4linux/v4l2-pci-skeleton.c.
> 
Got it. We will check this.


> > +
> > +	*nplanes = q_data->fmt->num_planes;
> > +
> > +	if (vq->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
> > +		unsigned int i;
> > +
> > +		for (i = 0; i < *nplanes; i++) {
> > +			sizes[i] = q_data->sizeimage[i];
> > +			alloc_ctxs[i] = ctx->dev->alloc_ctx;
> > +		}
> > +	} else if (vq->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
> > +		sizes[0] = q_data->sizeimage[0];
> > +		alloc_ctxs[0] = ctx->dev->alloc_ctx;
> > +	} else {
> > +		return -EINVAL;
> > +	}
> > +
> > +	mtk_v4l2_debug(2, "[%d]get %d buffer(s) of size 0x%x each, vq->memory=%d",
> > +		       ctx->idx, *nbuffers, sizes[0], vq->memory);
> > +
> > +	return 0;
> > +}
> > +
> > +static int vb2ops_venc_buf_prepare(struct vb2_buffer *vb)
> > +{
> > +	struct mtk_vcodec_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue);
> > +	struct mtk_q_data *q_data;
> > +	int i;
> > +
> > +	q_data = mtk_venc_get_q_data(ctx, vb->vb2_queue->type);
> > +
> > +	for (i = 0; i < q_data->fmt->num_planes; i++) {
> > +		if (vb2_plane_size(vb, i) < q_data->sizeimage[i]) {
> > +			mtk_v4l2_debug(2, "data will not fit into plane %d (%lu < %d)",
> > +				       i, vb2_plane_size(vb, i),
> > +				       q_data->sizeimage[i]);
> > +			return -EINVAL;
> > +		}
> > +	}
> > +
> > +	return 0;
> > +}
> > +
> > +static void vb2ops_venc_buf_queue(struct vb2_buffer *vb)
> > +{
> > +	struct mtk_vcodec_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue);
> > +	struct vb2_v4l2_buffer *vb2_v4l2 =
> > +			container_of(vb, struct vb2_v4l2_buffer, vb2_buf);
> > +	struct mtk_video_enc_buf *mtk_buf =
> > +			container_of(vb2_v4l2, struct mtk_video_enc_buf, vb);
> > +
> > +	if ((vb->vb2_queue->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) &&
> > +		(ctx->param_change != MTK_ENCODE_PARAM_NONE)) {
> > +		mtk_v4l2_debug(1,
> > +				"[%d] Before id=%d encode parameter change %x",
> > +				ctx->idx, vb->index,
> > +				ctx->param_change);
> > +		mtk_buf->param_change = ctx->param_change;
> > +		if (mtk_buf->param_change & MTK_ENCODE_PARAM_BITRATE) {
> > +			mtk_buf->enc_params.bitrate = ctx->enc_params.bitrate;
> > +			mtk_v4l2_debug(1, "[%d] idx=%d change param br=%d",
> > +				ctx->idx,
> > +				mtk_buf->vb.vb2_buf.index,
> > +				mtk_buf->enc_params.bitrate);
> > +		}
> > +		if (ctx->param_change & MTK_ENCODE_PARAM_FRAMERATE) {
> > +			mtk_buf->enc_params.framerate_num =
> > +				ctx->enc_params.framerate_num;
> > +			mtk_buf->enc_params.framerate_denom =
> > +				ctx->enc_params.framerate_denom;
> > +			mtk_v4l2_debug(1, "[%d] idx=%d, change param fr=%d/%d",
> > +					ctx->idx,
> > +					mtk_buf->vb.vb2_buf.index,
> > +					mtk_buf->enc_params.framerate_num,
> > +					mtk_buf->enc_params.framerate_denom);
> > +		}
> > +		if (ctx->param_change & MTK_ENCODE_PARAM_INTRA_PERIOD) {
> > +			mtk_buf->enc_params.gop_size = ctx->enc_params.gop_size;
> > +			mtk_v4l2_debug(1, "[%d] idx=%d, change param intra period=%d",
> > +					ctx->idx,
> > +					mtk_buf->vb.vb2_buf.index,
> > +					mtk_buf->enc_params.gop_size);
> > +		}
> > +		if (ctx->param_change & MTK_ENCODE_PARAM_FRAME_TYPE) {
> > +			mtk_buf->enc_params.force_intra =
> > +				ctx->enc_params.force_intra;
> > +			mtk_v4l2_debug(1, "[%d] idx=%d, change param force I=%d",
> > +					ctx->idx,
> > +					mtk_buf->vb.vb2_buf.index,
> > +					mtk_buf->enc_params.force_intra);
> > +		}
> > +		ctx->param_change = MTK_ENCODE_PARAM_NONE;
> > +	}
> > +
> > +	v4l2_m2m_buf_queue(ctx->m2m_ctx, to_vb2_v4l2_buffer(vb));
> > +}
> > +
> > +static int vb2ops_venc_start_streaming(struct vb2_queue *q, unsigned int count)
> > +{
> > +	struct mtk_vcodec_ctx *ctx = vb2_get_drv_priv(q);
> > +	struct venc_enc_prm param;
> > +	int ret;
> > +	int i;
> > +
> > +	/* Once state turn into MTK_STATE_ABORT, we need stop_streaming to clear it */
> > +	if ((ctx->state == MTK_STATE_ABORT) || (ctx->state == MTK_STATE_FREE))
> > +		goto err_set_param;
> > +
> > +	if (!(vb2_start_streaming_called(&ctx->m2m_ctx->out_q_ctx.q) &
> > +	      vb2_start_streaming_called(&ctx->m2m_ctx->cap_q_ctx.q))) {
> > +		mtk_v4l2_debug(1, "[%d]-> out=%d cap=%d",
> > +		 ctx->idx,
> > +		 vb2_start_streaming_called(&ctx->m2m_ctx->out_q_ctx.q),
> > +		 vb2_start_streaming_called(&ctx->m2m_ctx->cap_q_ctx.q));
> > +		return 0;
> > +	}
> > +
> > +	mtk_venc_set_param(ctx, &param);
> > +	ret = venc_if_set_param(ctx,
> > +				VENC_SET_PARAM_ENC,
> > +				&param);
> > +	if (ret) {
> > +		mtk_v4l2_err("venc_if_set_param failed=%d\n", ret);
> > +		ctx->state = MTK_STATE_ABORT;
> > +		goto err_set_param;
> > +	}
> > +
> > +	if ((ctx->q_data[MTK_Q_DATA_DST].fmt->fourcc == V4L2_PIX_FMT_H264) &&
> > +	    (ctx->enc_params.seq_hdr_mode != V4L2_MPEG_VIDEO_HEADER_MODE_SEPARATE)) {
> > +		ret = venc_if_set_param(ctx,
> > +					VENC_SET_PARAM_PREPEND_HEADER,
> > +					0);
> > +		if (ret) {
> > +			mtk_v4l2_err("venc_if_set_param failed=%d\n", ret);
> > +			ctx->state = MTK_STATE_ABORT;
> > +			goto err_set_param;
> > +		}
> > +		ctx->state = MTK_STATE_HEADER;
> > +	}
> > +
> > +	return 0;
> > +
> > +err_set_param:
> > +	for (i = 0; i < q->num_buffers; ++i) {
> > +		if (q->bufs[i]->state == VB2_BUF_STATE_ACTIVE) {
> > +			mtk_v4l2_debug(0, "[%d] idx=%d, type=%d, %d -> VB2_BUF_STATE_QUEUED",
> > +					ctx->idx, i, q->type,
> > +					(int)q->bufs[i]->state );
> > +			v4l2_m2m_buf_done(to_vb2_v4l2_buffer(q->bufs[i]), VB2_BUF_STATE_QUEUED);
> > +		}
> > +	}
> > +
> > +	return -EINVAL;
> > +}
> > +
> > +static void vb2ops_venc_stop_streaming(struct vb2_queue *q)
> > +{
> > +	struct mtk_vcodec_ctx *ctx = vb2_get_drv_priv(q);
> > +	struct vb2_buffer *src_buf, *dst_buf;
> > +	int ret;
> > +
> > +	mtk_v4l2_debug(2, "[%d]-> type=%d", ctx->idx, q->type);
> > +
> > +	if (q->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
> > +		while ((dst_buf = v4l2_m2m_dst_buf_remove(ctx->m2m_ctx))) {
> > +			dst_buf->planes[0].bytesused = 0;
> > +			v4l2_m2m_buf_done(to_vb2_v4l2_buffer(dst_buf),
> > +						VB2_BUF_STATE_ERROR);
> > +		}
> > +	} else {
> > +		while ((src_buf = v4l2_m2m_src_buf_remove(ctx->m2m_ctx)))
> > +			v4l2_m2m_buf_done(to_vb2_v4l2_buffer(src_buf),
> > +						VB2_BUF_STATE_ERROR);
> > +	}
> > +
> > +	if ((q->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE &&
> > +	     vb2_is_streaming(&ctx->m2m_ctx->out_q_ctx.q)) ||
> > +	    (q->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE &&
> > +	     vb2_is_streaming(&ctx->m2m_ctx->cap_q_ctx.q))) {
> > +		mtk_v4l2_debug(1, "[%d]-> q type %d out=%d cap=%d",
> > +			       ctx->idx, q->type,
> > +			       vb2_is_streaming(&ctx->m2m_ctx->out_q_ctx.q),
> > +			       vb2_is_streaming(&ctx->m2m_ctx->cap_q_ctx.q));
> > +		return;
> > +	}
> > +
> > +	ret = venc_if_release(ctx);
> > +	if (ret)
> > +		mtk_v4l2_err("venc_if_release failed=%d\n", ret);
> > +
> > +	ctx->state = MTK_STATE_FREE;
> > +}
> > +
> > +static struct vb2_ops mtk_venc_vb2_ops = {
> > +	.queue_setup			= vb2ops_venc_queue_setup,
> > +	.buf_prepare			= vb2ops_venc_buf_prepare,
> > +	.buf_queue			= vb2ops_venc_buf_queue,
> > +	.wait_prepare			= vb2_ops_wait_prepare,
> > +	.wait_finish			= vb2_ops_wait_finish,
> > +	.start_streaming		= vb2ops_venc_start_streaming,
> > +	.stop_streaming			= vb2ops_venc_stop_streaming,
> > +};
> > +
> > +static int mtk_venc_encode_header(void *priv)
> > +{
> > +	struct mtk_vcodec_ctx *ctx = priv;
> > +	int ret;
> > +	struct vb2_buffer *dst_buf;
> > +	struct mtk_vcodec_mem bs_buf;
> > +	struct venc_done_result enc_result;
> > +
> > +	dst_buf = v4l2_m2m_dst_buf_remove(ctx->m2m_ctx);
> > +	if (!dst_buf) {
> > +		mtk_v4l2_debug(1, "No dst buffer");
> > +		return -EINVAL;
> > +	}
> > +
> > +	bs_buf.va = vb2_plane_vaddr(dst_buf, 0);
> > +	bs_buf.dma_addr = vb2_dma_contig_plane_dma_addr(dst_buf, 0);
> > +	bs_buf.size = (unsigned int)dst_buf->planes[0].length;
> > +
> > +	mtk_v4l2_debug(1,
> > +			"[%d] buf idx=%d va=0x%p dma_addr=0x%llx size=0x%lx",
> > +			ctx->idx,
> > +			dst_buf->index, bs_buf.va,
> > +			(u64)bs_buf.dma_addr,
> > +			bs_buf.size);
> > +
> > +	ret = venc_if_encode(ctx,
> > +			VENC_START_OPT_ENCODE_SEQUENCE_HEADER,
> > +			0, &bs_buf, &enc_result);
> > +
> > +	if (ret) {
> > +		dst_buf->planes[0].bytesused = 0;
> > +		ctx->state = MTK_STATE_ABORT;
> > +		v4l2_m2m_buf_done(to_vb2_v4l2_buffer(dst_buf), VB2_BUF_STATE_ERROR);
> > +		mtk_v4l2_err("venc_if_encode failed=%d", ret);
> > +		return -EINVAL;
> > +	}
> > +
> > +	ctx->state = MTK_STATE_HEADER;
> > +	dst_buf->planes[0].bytesused = enc_result.bs_size;
> > +
> > +#if defined(DEBUG)
> > +{
> > +	int i;
> > +	mtk_v4l2_debug(1, "[%d] venc_if_encode header len=%d",
> > +			ctx->idx,
> > +			enc_result.bs_size);
> > +	for (i = 0; i < enc_result.bs_size; i++) {
> > +		unsigned char *p = (unsigned char *)bs_buf.va;
> > +
> > +		mtk_v4l2_debug(1, "[%d] buf[%d]=0x%2x", ctx->idx, i, p[i]);
> > +	}
> > +}
> > +#endif
> > +	v4l2_m2m_buf_done(to_vb2_v4l2_buffer(dst_buf), VB2_BUF_STATE_DONE);
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_venc_param_change(struct mtk_vcodec_ctx *ctx, void *priv)
> > +{
> > +	struct vb2_buffer *vb = priv;
> > +	struct vb2_v4l2_buffer *vb2_v4l2 =
> > +			container_of(vb, struct vb2_v4l2_buffer, vb2_buf);
> > +	struct mtk_video_enc_buf *mtk_buf =
> > +			container_of(vb2_v4l2, struct mtk_video_enc_buf, vb);
> > +	int ret = 0;
> > +
> > +	if (mtk_buf->param_change == MTK_ENCODE_PARAM_NONE)
> > +		return 0;
> > +
> > +	mtk_v4l2_debug(1, "encode parameters change id=%d", vb->index);
> > +	if (mtk_buf->param_change & MTK_ENCODE_PARAM_BITRATE) {
> > +		struct venc_enc_prm enc_prm;
> > +
> > +		enc_prm.bitrate = mtk_buf->enc_params.bitrate;
> > +		mtk_v4l2_debug(1, "[%d] idx=%d, change param br=%d",
> > +				ctx->idx,
> > +				mtk_buf->vb.vb2_buf.index,
> > +				enc_prm.bitrate);
> > +		ret |= venc_if_set_param(ctx,
> > +					 VENC_SET_PARAM_ADJUST_BITRATE,
> > +					 &enc_prm);
> > +	}
> > +	if (mtk_buf->param_change & MTK_ENCODE_PARAM_FRAMERATE) {
> > +		struct venc_enc_prm enc_prm;
> > +
> > +		enc_prm.frm_rate = mtk_buf->enc_params.framerate_num /
> > +				   mtk_buf->enc_params.framerate_denom;
> > +		mtk_v4l2_debug(1, "[%d] idx=%d, change param fr=%d",
> > +			       ctx->idx,
> > +			       mtk_buf->vb.vb2_buf.index,
> > +			       enc_prm.frm_rate);
> > +		ret |= venc_if_set_param(ctx,
> > +					 VENC_SET_PARAM_ADJUST_FRAMERATE,
> > +					 &enc_prm);
> > +	}
> > +	if (mtk_buf->param_change & MTK_ENCODE_PARAM_INTRA_PERIOD) {
> > +		mtk_v4l2_debug(1, "change param intra period=%d",
> > +				 mtk_buf->enc_params.gop_size);
> > +		ret |= venc_if_set_param(ctx,
> > +					 VENC_SET_PARAM_I_FRAME_INTERVAL,
> > +					 &mtk_buf->enc_params.gop_size);
> > +	}
> > +	if (mtk_buf->param_change & MTK_ENCODE_PARAM_FRAME_TYPE) {
> > +		mtk_v4l2_debug(1, "[%d] idx=%d, change param force I=%d",
> > +				ctx->idx,
> > +				mtk_buf->vb.vb2_buf.index,
> > +				mtk_buf->enc_params.force_intra);
> > +		if (mtk_buf->enc_params.force_intra)
> > +			ret |= venc_if_set_param(ctx,
> > +						 VENC_SET_PARAM_FORCE_INTRA,
> > +						 0);
> > +	}
> > +
> > +	mtk_buf->param_change = MTK_ENCODE_PARAM_NONE;
> > +
> > +	if (ret) {
> > +		ctx->state = MTK_STATE_ABORT;
> > +		mtk_v4l2_err("venc_if_set_param %d failed=%d\n",
> > +			MTK_ENCODE_PARAM_FRAME_TYPE, ret);
> > +		return -1;
> > +	}
> > +
> > +	return 0;
> > +}
> > +
> > +static void mtk_venc_worker(struct work_struct *work)
> > +{
> > +	struct mtk_vcodec_ctx *ctx = container_of(work, struct mtk_vcodec_ctx,
> > +				    encode_work);
> > +	struct vb2_buffer *src_buf, *dst_buf;
> > +	struct venc_frm_buf frm_buf;
> > +	struct mtk_vcodec_mem bs_buf;
> > +	struct venc_done_result enc_result;
> > +	int ret;
> > +	struct vb2_v4l2_buffer *v4l2_vb;
> > +
> > +	if ((ctx->q_data[MTK_Q_DATA_DST].fmt->fourcc == V4L2_PIX_FMT_H264) &&
> > +	    (ctx->state != MTK_STATE_HEADER)) {
> > +		/* encode h264 sps/pps header */
> > +		mtk_venc_encode_header(ctx);
> > +		v4l2_m2m_job_finish(ctx->dev->m2m_dev_enc, ctx->m2m_ctx);
> > +		return;
> > +	}
> > +
> > +	src_buf = v4l2_m2m_next_src_buf(ctx->m2m_ctx);
> > +	if (!src_buf) {
> > +		v4l2_m2m_job_finish(ctx->dev->m2m_dev_enc, ctx->m2m_ctx);
> > +		return;
> > +	}
> > +
> > +	dst_buf = v4l2_m2m_dst_buf_remove(ctx->m2m_ctx);
> > +	if (!dst_buf) {
> > +		v4l2_m2m_job_finish(ctx->dev->m2m_dev_enc, ctx->m2m_ctx);
> > +		return;
> > +	}
> > +
> > +	mtk_venc_param_change(ctx, src_buf);
> > +
> > +	frm_buf.fb_addr.va = vb2_plane_vaddr(src_buf, 0);
> > +	frm_buf.fb_addr.dma_addr = vb2_dma_contig_plane_dma_addr(src_buf, 0);
> > +	frm_buf.fb_addr.size = (unsigned int)src_buf->planes[0].length;
> > +	frm_buf.fb_addr1.va = vb2_plane_vaddr(src_buf, 1);
> > +	frm_buf.fb_addr1.dma_addr = vb2_dma_contig_plane_dma_addr(src_buf, 1);
> > +	frm_buf.fb_addr1.size = (unsigned int)src_buf->planes[1].length;
> > +	if (src_buf->num_planes == 3) {
> > +		frm_buf.fb_addr2.va = vb2_plane_vaddr(src_buf, 2);
> > +		frm_buf.fb_addr2.dma_addr =
> > +			vb2_dma_contig_plane_dma_addr(src_buf, 2);
> > +		frm_buf.fb_addr2.size =
> > +			(unsigned int)src_buf->planes[2].length;
> > +	} else {
> > +		frm_buf.fb_addr2.va = NULL;
> > +		frm_buf.fb_addr2.dma_addr = 0;
> > +		frm_buf.fb_addr2.size = 0;
> > +	}
> > +	bs_buf.va = vb2_plane_vaddr(dst_buf, 0);
> > +	bs_buf.dma_addr = vb2_dma_contig_plane_dma_addr(dst_buf, 0);
> > +	bs_buf.size = (unsigned int)dst_buf->planes[0].length;
> > +
> > +	mtk_v4l2_debug(2,
> > +			"Framebuf VA=%p PA=%llx Size=0x%lx;VA=%p PA=0x%llx Size=0x%lx;VA=%p PA=0x%llx Size=0x%lx",
> > +			frm_buf.fb_addr.va,
> > +			(u64)frm_buf.fb_addr.dma_addr,
> > +			frm_buf.fb_addr.size,
> > +			frm_buf.fb_addr1.va,
> > +			(u64)frm_buf.fb_addr1.dma_addr,
> > +			frm_buf.fb_addr1.size,
> > +			frm_buf.fb_addr2.va,
> > +			(u64)frm_buf.fb_addr2.dma_addr,
> > +			frm_buf.fb_addr2.size);
> > +
> > +	ret = venc_if_encode(ctx, VENC_START_OPT_ENCODE_FRAME,
> > +			     &frm_buf, &bs_buf, &enc_result);
> > +
> > +	src_buf = v4l2_m2m_src_buf_remove(ctx->m2m_ctx);
> > +	if (enc_result.msg == VENC_MESSAGE_OK)
> > +		v4l2_m2m_buf_done(to_vb2_v4l2_buffer(src_buf), VB2_BUF_STATE_DONE);
> > +	else
> > +		v4l2_m2m_buf_done(to_vb2_v4l2_buffer(src_buf), VB2_BUF_STATE_ERROR);
> > +
> > +	if (enc_result.is_key_frm) {
> > +		v4l2_vb = to_vb2_v4l2_buffer(dst_buf);
> > +		v4l2_vb->flags |= V4L2_BUF_FLAG_KEYFRAME;
> > +	}
> > +
> > +	if (ret) {
> > +		dst_buf->planes[0].bytesused = 0;
> > +		v4l2_m2m_buf_done(to_vb2_v4l2_buffer(dst_buf), VB2_BUF_STATE_ERROR);
> > +		mtk_v4l2_err("venc_if_encode failed=%d", ret);
> > +	} else {
> > +		dst_buf->planes[0].bytesused = enc_result.bs_size;
> > +		v4l2_m2m_buf_done(to_vb2_v4l2_buffer(dst_buf), VB2_BUF_STATE_DONE);
> > +		mtk_v4l2_debug(2, "venc_if_encode bs size=%d",
> > +				 enc_result.bs_size);
> > +	}
> > +
> > +	v4l2_m2m_job_finish(ctx->dev->m2m_dev_enc, ctx->m2m_ctx);
> > +
> > +	mtk_v4l2_debug(1, "<=== src_buf[%d] dst_buf[%d] venc_if_encode ret=%d Size=%u===>",
> > +			src_buf->index, dst_buf->index, ret,
> > +			enc_result.bs_size);
> > +}
> > +
> > +static void m2mops_venc_device_run(void *priv)
> > +{
> > +	struct mtk_vcodec_ctx *ctx = priv;
> 
> Add empty line.
> 
We will fix this.

> > +	queue_work(ctx->dev->encode_workqueue, &ctx->encode_work);
> > +}
> > +
> > +static int m2mops_venc_job_ready(void *m2m_priv)
> > +{
> > +	struct mtk_vcodec_ctx *ctx = m2m_priv;
> > +
> > +	if (!v4l2_m2m_num_dst_bufs_ready(ctx->m2m_ctx)) {
> > +		mtk_v4l2_debug(3, "[%d]Not ready: not enough video dst buffers.",
> > +			       ctx->idx);
> > +		return 0;
> > +	}
> > +
> > +	if (!v4l2_m2m_num_src_bufs_ready(ctx->m2m_ctx)) {
> > +		mtk_v4l2_debug(3, "[%d]Not ready: not enough video src buffers.",
> > +			       ctx->idx);
> > +			return 0;
> > +		}
> 
> Broken indentation.
> 
We will fix this.

> > +
> > +	if (ctx->state == MTK_STATE_ABORT) {
> > +		mtk_v4l2_debug(3, "[%d]Not ready: state=0x%x.",
> > +			       ctx->idx, ctx->state);
> > +		return 0;
> > +	}
> > +
> > +	if (ctx->state == MTK_STATE_FREE) {
> > +		mtk_v4l2_debug(3, "[%d]Not ready: state=0x%x.",
> > +			       ctx->idx, ctx->state);
> > +		return 0;
> > +	}
> > +
> > +	return 1;
> > +}
> > +
> > +static void m2mops_venc_job_abort(void *priv)
> > +{
> > +	struct mtk_vcodec_ctx *ctx = priv;
> > +
> > +	ctx->state = MTK_STATE_ABORT;
> > +}
> > +
> > +static void m2mops_venc_lock(void *m2m_priv)
> > +{
> > +	struct mtk_vcodec_ctx *ctx = m2m_priv;
> > +
> > +	mutex_lock(&ctx->dev->dev_mutex);
> > +}
> > +
> > +static void m2mops_venc_unlock(void *m2m_priv)
> > +{
> > +	struct mtk_vcodec_ctx *ctx = m2m_priv;
> > +
> > +	mutex_unlock(&ctx->dev->dev_mutex);
> > +}
> > +
> > +const struct v4l2_m2m_ops mtk_venc_m2m_ops = {
> > +	.device_run			= m2mops_venc_device_run,
> > +	.job_ready			= m2mops_venc_job_ready,
> > +	.job_abort			= m2mops_venc_job_abort,
> > +	.lock				= m2mops_venc_lock,
> > +	.unlock				= m2mops_venc_unlock,
> > +};
> > +
> > +#define IS_MTK_VENC_PRIV(x) ((V4L2_CTRL_ID2CLASS(x) == V4L2_CTRL_CLASS_MPEG) &&\
> > +			     V4L2_CTRL_DRIVER_PRIV(x))
> > +
> > +void mtk_vcodec_enc_ctx_params_setup(struct mtk_vcodec_ctx *ctx)
> > +{
> > +	struct mtk_q_data *q_data;
> > +	struct mtk_video_fmt *fmt;
> > +
> > +	ctx->m2m_ctx->q_lock = &ctx->dev->dev_mutex;
> > +	ctx->fh.m2m_ctx = ctx->m2m_ctx;
> > +	ctx->fh.ctrl_handler = &ctx->ctrl_hdl;
> > +	INIT_WORK(&ctx->encode_work, mtk_venc_worker);
> > +
> > +	ctx->q_data[MTK_Q_DATA_SRC].width = DFT_CFG_WIDTH;
> > +	ctx->q_data[MTK_Q_DATA_SRC].height = DFT_CFG_HEIGHT;
> > +	ctx->q_data[MTK_Q_DATA_SRC].fmt = &mtk_video_formats[0];
> > +	ctx->q_data[MTK_Q_DATA_SRC].colorspace = V4L2_COLORSPACE_REC709;
> > +	ctx->q_data[MTK_Q_DATA_SRC].field = V4L2_FIELD_NONE;
> > +
> > +	q_data = &ctx->q_data[MTK_Q_DATA_SRC];
> > +	fmt = ctx->q_data[MTK_Q_DATA_SRC].fmt;
> > +	mtk_vcodec_enc_calc_src_size(fmt->num_planes, q_data->width,
> > +			q_data->height,
> > +			ctx->q_data[MTK_Q_DATA_SRC].sizeimage,
> > +			ctx->q_data[MTK_Q_DATA_SRC].bytesperline);
> > +
> > +	ctx->q_data[MTK_Q_DATA_DST].width = DFT_CFG_WIDTH;
> > +	ctx->q_data[MTK_Q_DATA_DST].height = DFT_CFG_HEIGHT;
> > +	ctx->q_data[MTK_Q_DATA_DST].fmt = &mtk_video_formats[9];
> > +	ctx->q_data[MTK_Q_DATA_DST].colorspace = V4L2_COLORSPACE_REC709;
> > +	ctx->q_data[MTK_Q_DATA_DST].field = V4L2_FIELD_NONE;
> > +
> > +	q_data = &ctx->q_data[MTK_Q_DATA_DST];
> > +	fmt = ctx->q_data[MTK_Q_DATA_DST].fmt;
> > +	ctx->q_data[MTK_Q_DATA_DST].sizeimage[0] = q_data->width * q_data->height;
> > +	ctx->q_data[MTK_Q_DATA_DST].bytesperline[0] = 0;
> > +
> > +}
> > +
> > +int mtk_vcodec_enc_ctrls_setup(struct mtk_vcodec_ctx *ctx)
> > +{
> > +	const struct v4l2_ctrl_ops *ops = &mtk_vcodec_enc_ctrl_ops;
> > +	struct v4l2_ctrl_handler *handler = &ctx->ctrl_hdl;
> > +	struct v4l2_ctrl_config cfg;
> > +
> > +	v4l2_ctrl_handler_init(handler, MTK_MAX_CTRLS);
> > +	if (handler->error) {
> > +		mtk_v4l2_err("Init control handler fail %d\n",
> > +				handler->error);
> > +		return handler->error;
> > +	}
> 
> Move this check to just before the v4l2_ctrl_handler_setup(). Any of the
> v4l2_ctrl_new_std* functions can set handler->error, which is why it should
> be check after all controls are added.
> 
Got it. Will move it just before the v4l2_ctrl_handler_setup().

> > +
> > +	ctx->ctrls[0] = v4l2_ctrl_new_std(handler, ops,
> > +					V4L2_CID_MPEG_VIDEO_BITRATE,
> > +					1, 4000000, 1, 4000000);
> > +	ctx->ctrls[1] = v4l2_ctrl_new_std(handler, ops,
> > +					V4L2_CID_MPEG_VIDEO_B_FRAMES,
> > +					0, 2, 1, 0);
> > +	ctx->ctrls[2] = v4l2_ctrl_new_std(handler, ops,
> > +					V4L2_CID_MPEG_VIDEO_FRAME_RC_ENABLE,
> > +					0, 1, 1, 1);
> > +	ctx->ctrls[3] = v4l2_ctrl_new_std(handler, ops,
> > +					V4L2_CID_MPEG_VIDEO_H264_MAX_QP,
> > +					0, 51, 1, 51);
> > +	ctx->ctrls[4] = v4l2_ctrl_new_std(handler, ops,
> > +					V4L2_CID_MPEG_VIDEO_H264_I_PERIOD,
> > +					0, 65535, 1, 30);
> > +	ctx->ctrls[5] = v4l2_ctrl_new_std(handler, ops,
> > +					V4L2_CID_MPEG_VIDEO_GOP_SIZE,
> > +					0, 65535, 1, 30);
> > +	ctx->ctrls[6] = v4l2_ctrl_new_std(handler, ops,
> > +					V4L2_CID_MPEG_VIDEO_MB_RC_ENABLE,
> > +					0, 1, 1, 0);
> > +	ctx->ctrls[7] = v4l2_ctrl_new_std_menu(handler, ops,
> > +					V4L2_CID_MPEG_VIDEO_HEADER_MODE,
> > +					V4L2_MPEG_VIDEO_HEADER_MODE_JOINED_WITH_1ST_FRAME,
> > +					0, V4L2_MPEG_VIDEO_HEADER_MODE_SEPARATE);
> > +	ctx->ctrls[8] = v4l2_ctrl_new_std_menu(handler, ops,
> > +					V4L2_CID_MPEG_VIDEO_H264_PROFILE,
> > +					V4L2_MPEG_VIDEO_H264_PROFILE_HIGH,
> > +					0, V4L2_MPEG_VIDEO_H264_PROFILE_MAIN);
> > +	ctx->ctrls[9] = v4l2_ctrl_new_std_menu(handler, ops,
> > +					V4L2_CID_MPEG_VIDEO_H264_LEVEL,
> > +					V4L2_MPEG_VIDEO_H264_LEVEL_4_2,
> > +					0, V4L2_MPEG_VIDEO_H264_LEVEL_4_0);
> > +	ctx->ctrls[6] = v4l2_ctrl_new_std(handler, ops,
> > +					V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME,
> > +					0, 0, 0, 0);
> 
> You are overwriting V4L2_CID_MPEG_VIDEO_MB_RC_ENABLE! Why are you assigning it
> anyway? Unless you are using ctx->ctrls[] you don't need to assign the result
> of v4l2_ctrl_new_std to anything.
> 
Got it. We don't use ctx->ctrls. We will remove it in next version.

> > +
> > +	v4l2_ctrl_handler_setup(&ctx->ctrl_hdl);
> > +
> > +	return 0;
> > +}
> > +
> > +int mtk_vcodec_enc_queue_init(void *priv, struct vb2_queue *src_vq,
> > +			   struct vb2_queue *dst_vq)
> > +{
> > +	struct mtk_vcodec_ctx *ctx = priv;
> > +	int ret;
> > +
> > +	src_vq->type		= V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE;
> > +	src_vq->io_modes	= VB2_DMABUF | VB2_MMAP | VB2_USERPTR;
> 
> I recomment dropping VB2_USERPTR. That only makes sense for scatter-gather dma,
> and you use physically contiguous DMA.
> 
Now our userspace app use VB2_USERPTR. I need to check if we could drop
VB2_USERPTR.
We use src_vq->mem_ops = &vb2_dma_contig_memops;
And there are
	.get_userptr	= vb2_dc_get_userptr,
	.put_userptr	= vb2_dc_put_userptr,
I was confused why it only make sense for scatter-gather.
Could you kindly explain more?

> > +	src_vq->drv_priv	= ctx;
> > +	src_vq->buf_struct_size = sizeof(struct mtk_video_enc_buf);
> > +	src_vq->ops		= &mtk_venc_vb2_ops;
> > +	src_vq->mem_ops		= &vb2_dma_contig_memops;
> > +	src_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
> > +	src_vq->lock = &ctx->dev->dev_mutex;
> > +
> > +	ret = vb2_queue_init(src_vq);
> > +	if (ret)
> > +		return ret;
> > +
> > +	dst_vq->type		= V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
> > +	dst_vq->io_modes	= VB2_DMABUF | VB2_MMAP | VB2_USERPTR;
> > +	dst_vq->drv_priv	= ctx;
> > +	dst_vq->buf_struct_size = sizeof(struct v4l2_m2m_buffer);
> > +	dst_vq->ops		= &mtk_venc_vb2_ops;
> > +	dst_vq->mem_ops		= &vb2_dma_contig_memops;
> > +	dst_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
> > +	dst_vq->lock = &ctx->dev->dev_mutex;
> > +
> > +	return vb2_queue_init(dst_vq);
> > +}
> > +
> > +int mtk_venc_unlock(struct mtk_vcodec_ctx *ctx)
> > +{
> > +	struct mtk_vcodec_dev *dev = ctx->dev;
> 
> Add empty line.
> 
Will fix this in next version.

> > +	dev->curr_ctx = -1;
> > +	mutex_unlock(&dev->enc_mutex);
> > +	return 0;
> > +}
> > +
> > +int mtk_venc_lock(struct mtk_vcodec_ctx *ctx)
> > +{
> > +	struct mtk_vcodec_dev *dev = ctx->dev;
> > +
> > +	mutex_lock(&dev->enc_mutex);
> > +	dev->curr_ctx = ctx->idx;
> > +	return 0;
> > +}
> > +
> > +void mtk_vcodec_enc_release(struct mtk_vcodec_ctx *ctx)
> > +{
> > +	venc_if_release(ctx);
> > +}
> > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h
> > new file mode 100644
> > index 0000000..e09524b
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h
> > @@ -0,0 +1,46 @@
> > +/*
> > +* Copyright (c) 2015 MediaTek Inc.
> > +* Author: PC Chen <pc.chen@mediatek.com>
> > +*         Tiffany Lin <tiffany.lin@mediatek.com>
> > +*
> > +* This program is free software; you can redistribute it and/or modify
> > +* it under the terms of the GNU General Public License version 2 as
> > +* published by the Free Software Foundation.
> > +*
> > +* This program is distributed in the hope that it will be useful,
> > +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> > +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > +* GNU General Public License for more details.
> > +*/
> > +
> > +
> > +#ifndef _MTK_VCODEC_ENC_H_
> > +#define _MTK_VCODEC_ENC_H_
> > +
> > +#include <media/videobuf2-core.h>
> > +#include <media/videobuf2-v4l2.h>
> > +
> > +/**
> > + * struct mtk_video_enc_buf - Private data related to each VB2 buffer.
> > + * @b:			Pointer to related VB2 buffer.
> > + * @param_change:	Types of encode parameter change before encode this
> > + *			buffer
> > + * @enc_params		Encode parameters changed before encode this buffer
> > + */
> > +struct mtk_video_enc_buf {
> > +	struct vb2_v4l2_buffer	vb;
> > +	struct list_head	list;
> > +
> > +	enum mtk_encode_param param_change;
> > +	struct mtk_enc_params enc_params;
> > +};
> > +
> > +int mtk_venc_unlock(struct mtk_vcodec_ctx *ctx);
> > +int mtk_venc_lock(struct mtk_vcodec_ctx *ctx);
> > +int mtk_vcodec_enc_queue_init(void *priv, struct vb2_queue *src_vq,
> > +					struct vb2_queue *dst_vq);
> > +void mtk_vcodec_enc_release(struct mtk_vcodec_ctx *ctx);
> > +int mtk_vcodec_enc_ctrls_setup(struct mtk_vcodec_ctx *ctx);
> > +void mtk_vcodec_enc_ctx_params_setup(struct mtk_vcodec_ctx *ctx);
> > +
> > +#endif /* _MTK_VCODEC_ENC_H_ */
> > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
> > new file mode 100644
> > index 0000000..e7ab14a
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
> > @@ -0,0 +1,476 @@
> > +/*
> > +* Copyright (c) 2015 MediaTek Inc.
> > +* Author: PC Chen <pc.chen@mediatek.com>
> > +*         Tiffany Lin <tiffany.lin@mediatek.com>
> > +*
> > +* This program is free software; you can redistribute it and/or modify
> > +* it under the terms of the GNU General Public License version 2 as
> > +* published by the Free Software Foundation.
> > +*
> > +* This program is distributed in the hope that it will be useful,
> > +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> > +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > +* GNU General Public License for more details.
> > +*/
> > +
> > +#include <linux/interrupt.h>
> > +#include <linux/irq.h>
> > +#include <linux/module.h>
> > +#include <linux/of_device.h>
> > +#include <linux/of.h>
> > +#include <media/v4l2-event.h>
> > +#include <media/v4l2-mem2mem.h>
> > +#include <media/videobuf2-dma-contig.h>
> > +#include <linux/pm_runtime.h>
> > +
> > +#include "mtk_vcodec_drv.h"
> > +#include "mtk_vcodec_enc.h"
> > +#include "mtk_vcodec_pm.h"
> > +#include "mtk_vcodec_intr.h"
> > +#include "mtk_vcodec_util.h"
> > +#include "mtk_vpu.h"
> > +
> > +
> > +/* Wake up context wait_queue */
> > +static void wake_up_ctx(struct mtk_vcodec_ctx *ctx, unsigned int reason)
> > +{
> > +	ctx->int_cond = 1;
> > +	ctx->int_type = reason;
> > +	wake_up_interruptible(&ctx->queue);
> > +}
> > +
> > +static irqreturn_t mtk_vcodec_enc_irq_handler(int irq, void *priv)
> > +{
> > +	struct mtk_vcodec_dev *dev = priv;
> > +	struct mtk_vcodec_ctx *ctx;
> > +	unsigned int irq_status;
> > +
> > +	if (dev->curr_ctx == -1) {
> > +		mtk_v4l2_err("curr_ctx = -1");
> > +		return IRQ_HANDLED;
> > +	}
> > +
> > +	ctx = dev->ctx[dev->curr_ctx];
> > +	if (ctx == NULL) {
> > +		mtk_v4l2_err("curr_ctx==NULL");
> > +		return IRQ_HANDLED;
> > +	}
> > +	mtk_v4l2_debug(1, "idx=%d", ctx->idx);
> > +	irq_status = readl(dev->reg_base[VENC_SYS] +
> > +				(MTK_VENC_IRQ_STATUS_OFFSET));
> > +	if (irq_status & MTK_VENC_IRQ_STATUS_PAUSE)
> > +		writel((MTK_VENC_IRQ_STATUS_PAUSE),
> > +		       dev->reg_base[VENC_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
> > +
> > +	if (irq_status & MTK_VENC_IRQ_STATUS_SWITCH)
> > +		writel((MTK_VENC_IRQ_STATUS_SWITCH),
> > +		       dev->reg_base[VENC_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
> > +
> > +	if (irq_status & MTK_VENC_IRQ_STATUS_DRAM)
> > +		writel((MTK_VENC_IRQ_STATUS_DRAM),
> > +		       dev->reg_base[VENC_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
> > +
> > +	if (irq_status & MTK_VENC_IRQ_STATUS_SPS)
> > +		writel((MTK_VENC_IRQ_STATUS_SPS),
> > +		       dev->reg_base[VENC_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
> > +
> > +	if (irq_status & MTK_VENC_IRQ_STATUS_PPS)
> > +		writel((MTK_VENC_IRQ_STATUS_PPS),
> > +		       dev->reg_base[VENC_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
> > +
> > +	if (irq_status & MTK_VENC_IRQ_STATUS_FRM)
> > +		writel((MTK_VENC_IRQ_STATUS_FRM),
> > +		       dev->reg_base[VENC_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
> > +
> > +	ctx->irq_status = irq_status;
> > +	wake_up_ctx(ctx, MTK_INST_IRQ_RECEIVED);
> > +	return IRQ_HANDLED;
> > +}
> > +
> > +#if 1 /* VENC_LT */
> > +static irqreturn_t mtk_vcodec_enc_irq_handler2(int irq, void *priv)
> > +{
> > +	struct mtk_vcodec_dev *dev = priv;
> > +	struct mtk_vcodec_ctx *ctx;
> > +	unsigned int irq_status;
> > +
> > +	ctx = dev->ctx[dev->curr_ctx];
> > +	if (ctx == NULL) {
> > +		mtk_v4l2_err("ctx==NULL");
> > +		return IRQ_HANDLED;
> > +	}
> > +	mtk_v4l2_debug(1, "idx=%d", ctx->idx);
> > +	irq_status = readl(dev->reg_base[VENC_LT_SYS] +
> > +				(MTK_VENC_IRQ_STATUS_OFFSET));
> > +	if (irq_status & MTK_VENC_IRQ_STATUS_PAUSE)
> > +		writel((MTK_VENC_IRQ_STATUS_PAUSE),
> > +		       dev->reg_base[VENC_LT_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
> > +
> > +	if (irq_status & MTK_VENC_IRQ_STATUS_SWITCH)
> > +		writel((MTK_VENC_IRQ_STATUS_SWITCH),
> > +		       dev->reg_base[VENC_LT_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
> > +
> > +	if (irq_status & MTK_VENC_IRQ_STATUS_DRAM)
> > +		writel((MTK_VENC_IRQ_STATUS_DRAM),
> > +		       dev->reg_base[VENC_LT_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
> > +
> > +	if (irq_status & MTK_VENC_IRQ_STATUS_SPS)
> > +		writel((MTK_VENC_IRQ_STATUS_SPS),
> > +		       dev->reg_base[VENC_LT_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
> > +
> > +	if (irq_status & MTK_VENC_IRQ_STATUS_PPS)
> > +		writel((MTK_VENC_IRQ_STATUS_PPS),
> > +		       dev->reg_base[VENC_LT_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
> > +
> > +	if (irq_status & MTK_VENC_IRQ_STATUS_FRM)
> > +		writel((MTK_VENC_IRQ_STATUS_FRM),
> > +		       dev->reg_base[VENC_LT_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
> > +
> > +	ctx->irq_status = irq_status;
> > +	wake_up_ctx(ctx, MTK_INST_IRQ_RECEIVED);
> > +	return IRQ_HANDLED;
> > +}
> > +#endif
> > +
> > +static void mtk_vcodec_enc_reset_handler(void *priv)
> > +{
> > +	int i;
> > +	struct mtk_vcodec_dev *dev = priv;
> > +	struct mtk_vcodec_ctx *ctx;
> > +
> > +	mtk_v4l2_debug(0, "Watchdog timeout!!");
> > +
> > +	mutex_lock(&dev->dev_mutex);
> > +	for(i = 0; i < MTK_VCODEC_MAX_ENCODER_INSTANCES; i++) {
> > +		ctx = dev->ctx[i];
> > +		if (ctx) {
> > +			ctx->state = MTK_STATE_ABORT;
> > +			mtk_v4l2_debug(0, "[%d] Change to state MTK_STATE_ERROR", ctx->idx);
> > +		}
> > +
> > +	}
> > +	mutex_unlock(&dev->dev_mutex);
> > +}
> > +
> > +static int fops_vcodec_open(struct file *file)
> > +{
> > +	struct video_device *vfd = video_devdata(file);
> > +	struct mtk_vcodec_dev *dev = video_drvdata(file);
> > +	struct mtk_vcodec_ctx *ctx = NULL;
> > +	int ret = 0;
> > +
> > +	mutex_lock(&dev->dev_mutex);
> > +
> > +	ctx = devm_kzalloc(&dev->plat_dev->dev, sizeof(*ctx), GFP_KERNEL);
> > +	if (!ctx) {
> > +		ret = -ENOMEM;
> > +		goto err_alloc;
> > +	}
> > +
> > +	if (dev->num_instances >= MTK_VCODEC_MAX_ENCODER_INSTANCES) {
> > +		mtk_v4l2_err("Too many open contexts\n");
> > +		ret = -EBUSY;
> > +		goto err_no_ctx;
> 
> Hmm. I never like it if you can't open a video node because of a reason like this.
> 
> I.e. a simple 'v4l2-ctl -D' (i.e. calling QUERYCAP) should never fail.
> 
> If there are hardware limitation that prevent more than X instances from running at
> the same time, then those limitations typically kick in when you start to stream
> (or possibly when calling REQBUFS). But before that it should always be possible to
> open the device.
> 
> Having this check at open() is an indication of a poor design.
> 
> Is this is a hardware limitation at all?
> 
This is to make sure performance meet requirements, such as bitrate and
framerate.
We got your point. We will remove this and move limitation control to
start_streaming or REQBUFS.
Appreciated for your suggestion.:)


> > +	}
> > +
> > +	ctx->idx = ffz(dev->instance_mask[0]);
> > +	v4l2_fh_init(&ctx->fh, video_devdata(file));
> > +	file->private_data = &ctx->fh;
> > +	v4l2_fh_add(&ctx->fh);
> > +	ctx->dev = dev;
> > +
> > +	if (vfd == dev->vfd_enc) {
> > +		ctx->type = MTK_INST_ENCODER;
> > +
> > +		ret = mtk_vcodec_enc_ctrls_setup(ctx);
> > +		if (ret) {
> > +			mtk_v4l2_err("Failed to setup controls() (%d)\n",
> > +				       ret);
> > +			goto err_ctrls_setup;
> > +		}
> > +		ctx->m2m_ctx = v4l2_m2m_ctx_init(dev->m2m_dev_enc, ctx,
> > +						 &mtk_vcodec_enc_queue_init);
> > +		if (IS_ERR(ctx->m2m_ctx)) {
> > +			ret = PTR_ERR(ctx->m2m_ctx);
> > +			mtk_v4l2_err("Failed to v4l2_m2m_ctx_init() (%d)\n",
> > +				       ret);
> > +			goto err_ctx_init;
> > +		}
> > +		mtk_vcodec_enc_ctx_params_setup(ctx);
> > +	} else {
> > +		mtk_v4l2_err("Invalid vfd !\n");
> > +		ret = -ENOENT;
> > +		goto err_ctx_init;
> > +	}
> > +
> > +	init_waitqueue_head(&ctx->queue);
> > +	dev->num_instances++;
> > +
> > +	if (dev->num_instances == 1) {
> 
> Having a counter is also not needed. You can use v4l2_fh_is_singular() to check if this
> is the first open.
> 
Got it. Thanks for your information.

> > +		ret = vpu_load_firmware(dev->vpu_plat_dev);
> > +		if (ret < 0) {
> > +				mtk_v4l2_err("vpu_load_firmware failed!\n");
> > +			goto err_load_fw;
> > +		}
> > +
> > +		dev->enc_capability =
> > +			vpu_get_venc_hw_capa(dev->vpu_plat_dev);
> > +		mtk_v4l2_debug(0, "encoder capability %x", dev->enc_capability);
> > +	}
> > +
> > +	mtk_v4l2_debug(2, "Create instance [%d]@%p m2m_ctx=%p type=%d\n",
> > +			 ctx->idx, ctx, ctx->m2m_ctx, ctx->type);
> > +	set_bit(ctx->idx, &dev->instance_mask[0]);
> > +	dev->ctx[ctx->idx] = ctx;
> > +
> > +	mutex_unlock(&dev->dev_mutex);
> > +	mtk_v4l2_debug(0, "%s encoder [%d]", dev_name(&dev->plat_dev->dev), ctx->idx);
> > +	return ret;
> > +
> > +	/* Deinit when failure occurred */
> > +err_load_fw:
> > +	v4l2_m2m_ctx_release(ctx->m2m_ctx);
> > +	v4l2_fh_del(&ctx->fh);
> > +	v4l2_fh_exit(&ctx->fh);
> > +	dev->num_instances--;
> > +err_ctx_init:
> > +err_ctrls_setup:
> > +	v4l2_ctrl_handler_free(&ctx->ctrl_hdl);
> > +err_no_ctx:
> > +	devm_kfree(&dev->plat_dev->dev, ctx);
> > +err_alloc:
> > +	mutex_unlock(&dev->dev_mutex);
> > +	return ret;
> > +}
> > +
> > +static int fops_vcodec_release(struct file *file)
> > +{
> > +	struct mtk_vcodec_dev *dev = video_drvdata(file);
> > +	struct mtk_vcodec_ctx *ctx = fh_to_ctx(file->private_data);
> > +
> > +	mtk_v4l2_debug(0, "[%d] encoder\n", ctx->idx);
> > +	mutex_lock(&dev->dev_mutex);
> > +
> > +	mtk_vcodec_enc_release(ctx);
> > +	v4l2_fh_del(&ctx->fh);
> > +	v4l2_fh_exit(&ctx->fh);
> > +	v4l2_ctrl_handler_free(&ctx->ctrl_hdl);
> > +	v4l2_m2m_ctx_release(ctx->m2m_ctx);
> > +
> > +	dev->ctx[ctx->idx] = NULL;
> > +	dev->num_instances--;
> > +	clear_bit(ctx->idx, &dev->instance_mask[0]);
> > +	devm_kfree(&dev->plat_dev->dev, ctx);
> > +	mutex_unlock(&dev->dev_mutex);
> > +	return 0;
> > +}
> > +
> > +static const struct v4l2_file_operations mtk_vcodec_fops = {
> > +	.owner				= THIS_MODULE,
> > +	.open				= fops_vcodec_open,
> > +	.release			= fops_vcodec_release,
> > +	.poll				= v4l2_m2m_fop_poll,
> > +	.unlocked_ioctl			= video_ioctl2,
> > +	.mmap				= v4l2_m2m_fop_mmap,
> > +};
> > +
> > +static int mtk_vcodec_probe(struct platform_device *pdev)
> > +{
> > +	struct mtk_vcodec_dev *dev;
> > +	struct video_device *vfd_enc;
> > +	struct resource *res;
> > +	int i, j, ret;
> > +
> > +	dev = devm_kzalloc(&pdev->dev, sizeof(*dev), GFP_KERNEL);
> > +	if (!dev)
> > +		return -ENOMEM;
> > +
> > +	dev->plat_dev = pdev;
> > +
> > +	dev->vpu_plat_dev = vpu_get_plat_device(dev->plat_dev);
> > +	if (dev->vpu_plat_dev == NULL) {
> > +		mtk_v4l2_err("[VPU] vpu device in not ready\n");
> > +		return -EPROBE_DEFER;
> > +	}
> > +
> > +	vpu_wdt_reg_handler(dev->vpu_plat_dev, mtk_vcodec_enc_reset_handler, dev,
> > +			    VPU_RST_ENC);
> > +
> > +	ret = mtk_vcodec_init_enc_pm(dev);
> > +	if (ret < 0) {
> > +		dev_err(&pdev->dev, "Failed to get mt vcodec clock source!\n");
> > +		return ret;
> > +	}
> > +
> > +	for (i = VENC_SYS, j = 0; i < NUM_MAX_VCODEC_REG_BASE; i++, j++) {
> > +		res = platform_get_resource(pdev, IORESOURCE_MEM, j);
> > +		if (res == NULL) {
> > +			dev_err(&pdev->dev, "get memory resource failed.\n");
> > +			ret = -ENXIO;
> > +			goto err_res;
> > +		}
> > +		dev->reg_base[i] = devm_ioremap_resource(&pdev->dev, res);
> > +		if (IS_ERR(dev->reg_base[i])) {
> > +			dev_err(&pdev->dev,
> > +				"devm_ioremap_resource %d failed.\n", i);
> > +			ret = PTR_ERR(dev->reg_base);
> > +			goto err_res;
> > +		}
> > +		mtk_v4l2_debug(2, "reg[%d] base=0x%p\n", i, dev->reg_base[i]);
> > +	}
> > +
> > +	res = platform_get_resource(pdev, IORESOURCE_IRQ, 0);
> > +	if (res == NULL) {
> > +		dev_err(&pdev->dev, "failed to get irq resource\n");
> > +		ret = -ENOENT;
> > +		goto err_res;
> > +	}
> > +
> > +	dev->enc_irq = platform_get_irq(pdev, 0);
> > +	ret = devm_request_irq(&pdev->dev, dev->enc_irq,
> > +			       mtk_vcodec_enc_irq_handler,
> > +			       0, pdev->name, dev);
> > +	if (ret) {
> > +		dev_err(&pdev->dev, "Failed to install dev->enc_irq %d (%d)\n",
> > +			dev->enc_irq,
> > +			ret);
> > +		ret = -EINVAL;
> > +		goto err_res;
> > +	}
> > +
> > +	dev->enc_lt_irq = platform_get_irq(pdev, 1);
> > +	ret = devm_request_irq(&pdev->dev,
> > +			       dev->enc_lt_irq, mtk_vcodec_enc_irq_handler2,
> > +			       0, pdev->name, dev);
> > +	if (ret) {
> > +		dev_err(&pdev->dev,
> > +			"Failed to install dev->enc_lt_irq %d (%d)\n",
> > +			dev->enc_lt_irq, ret);
> > +		ret = -EINVAL;
> > +		goto err_res;
> > +	}
> > +
> > +	disable_irq(dev->enc_irq);
> > +	disable_irq(dev->enc_lt_irq); /* VENC_LT */
> > +	mutex_init(&dev->enc_mutex);
> > +	mutex_init(&dev->dev_mutex);
> > +
> > +	snprintf(dev->v4l2_dev.name, sizeof(dev->v4l2_dev.name), "%s",
> > +		 "[MTK_V4L2_VENC]");
> > +
> > +	ret = v4l2_device_register(&pdev->dev, &dev->v4l2_dev);
> > +	if (ret) {
> > +		mtk_v4l2_err("v4l2_device_register err=%d\n", ret);
> > +		return ret;
> > +	}
> > +
> > +	init_waitqueue_head(&dev->queue);
> > +
> > +	/* allocate video device for encoder and register it */
> > +	vfd_enc = video_device_alloc();
> > +	if (!vfd_enc) {
> > +		mtk_v4l2_err("Failed to allocate video device\n");
> > +		ret = -ENOMEM;
> > +		goto err_enc_alloc;
> > +	}
> > +	vfd_enc->fops           = &mtk_vcodec_fops;
> > +	vfd_enc->ioctl_ops      = &mtk_venc_ioctl_ops;
> > +	vfd_enc->release        = video_device_release;
> > +	vfd_enc->lock           = &dev->dev_mutex;
> > +	vfd_enc->v4l2_dev       = &dev->v4l2_dev;
> > +	vfd_enc->vfl_dir        = VFL_DIR_M2M;
> > +
> > +	snprintf(vfd_enc->name, sizeof(vfd_enc->name), "%s",
> > +		 MTK_VCODEC_ENC_NAME);
> > +	video_set_drvdata(vfd_enc, dev);
> > +	dev->vfd_enc = vfd_enc;
> > +	platform_set_drvdata(pdev, dev);
> > +	ret = video_register_device(vfd_enc, VFL_TYPE_GRABBER, 1);
> > +	if (ret) {
> > +		mtk_v4l2_err("Failed to register video device\n");
> > +		goto err_enc_reg;
> > +	}
> 
> Do the register last. After registering the device anyone can start using it,
> but since not everything is initialized that might cause crashes.
> 
Got it. We will fix this.

> > +	mtk_v4l2_debug(0, "encoder registered as /dev/video%d\n",
> > +			 vfd_enc->num);
> > +
> > +	dev->alloc_ctx = vb2_dma_contig_init_ctx(&pdev->dev);
> > +	if (IS_ERR(dev->alloc_ctx)) {
> > +		mtk_v4l2_err("Failed to alloc vb2 dma context 0\n");
> > +		ret = PTR_ERR(dev->alloc_ctx);
> > +		goto err_vb2_ctx_init;
> > +	}
> > +
> > +	dev->m2m_dev_enc = v4l2_m2m_init(&mtk_venc_m2m_ops);
> > +	if (IS_ERR(dev->m2m_dev_enc)) {
> > +		mtk_v4l2_err("Failed to init mem2mem enc device\n");
> > +		ret = PTR_ERR(dev->m2m_dev_enc);
> > +		goto err_enc_mem_init;
> > +	}
> > +
> > +	dev->encode_workqueue =
> > +			alloc_ordered_workqueue(MTK_VCODEC_ENC_NAME, WQ_MEM_RECLAIM | WQ_FREEZABLE);
> > +	if (!dev->encode_workqueue) {
> > +		mtk_v4l2_err("Failed to create encode workqueue\n");
> > +		ret = -EINVAL;
> > +		goto err_event_workq;
> > +	}
> > +
> > +	return 0;
> > +
> > +err_event_workq:
> > +	v4l2_m2m_release(dev->m2m_dev_enc);
> > +err_enc_mem_init:
> > +	vb2_dma_contig_cleanup_ctx(dev->alloc_ctx);
> > +err_vb2_ctx_init:
> > +	video_unregister_device(vfd_enc);
> > +err_enc_reg:
> > +	video_device_release(vfd_enc);
> > +err_enc_alloc:
> > +	v4l2_device_unregister(&dev->v4l2_dev);
> > +err_res:
> > +	mtk_vcodec_release_enc_pm(dev);
> > +	return ret;
> > +}
> > +
> > +static const struct of_device_id mtk_vcodec_match[] = {
> > +	{.compatible = "mediatek,mt8173-vcodec-enc",},
> > +	{},
> > +};
> > +MODULE_DEVICE_TABLE(of, mtk_vcodec_match);
> > +
> > +static int mtk_vcodec_remove(struct platform_device *pdev)
> > +{
> > +	struct mtk_vcodec_dev *dev = platform_get_drvdata(pdev);
> > +
> > +	mtk_v4l2_debug_enter();
> > +	flush_workqueue(dev->encode_workqueue);
> > +	destroy_workqueue(dev->encode_workqueue);
> > +	if (dev->m2m_dev_enc)
> > +		v4l2_m2m_release(dev->m2m_dev_enc);
> > +	if (dev->alloc_ctx)
> > +		vb2_dma_contig_cleanup_ctx(dev->alloc_ctx);
> > +
> > +	if (dev->vfd_enc) {
> > +		video_unregister_device(dev->vfd_enc);
> > +		video_device_release(dev->vfd_enc);
> 
> Don't call video_device_release! That will happen automatically once the
> last filehandle is closed.
> 
Got it. We will fix this.

> > +	}
> > +	v4l2_device_unregister(&dev->v4l2_dev);
> > +	mtk_vcodec_release_enc_pm(dev);
> > +	return 0;
> > +}
> > +
> > +static struct platform_driver mtk_vcodec_driver = {
> > +	.probe	= mtk_vcodec_probe,
> > +	.remove	= mtk_vcodec_remove,
> > +	.driver	= {
> > +		.name	= MTK_VCODEC_ENC_NAME,
> > +		.owner	= THIS_MODULE,
> > +		.of_match_table = mtk_vcodec_match,
> > +	},
> > +};
> > +
> > +module_platform_driver(mtk_vcodec_driver);
> > +
> > +
> > +MODULE_LICENSE("GPL v2");
> > +MODULE_DESCRIPTION("Mediatek video codec V4L2 driver");
> > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
> > new file mode 100644
> > index 0000000..518fba7
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
> > @@ -0,0 +1,132 @@
> > +/*
> > +* Copyright (c) 2015 MediaTek Inc.
> > +* Author: Tiffany Lin <tiffany.lin@mediatek.com>
> > +*
> > +* This program is free software; you can redistribute it and/or modify
> > +* it under the terms of the GNU General Public License version 2 as
> > +* published by the Free Software Foundation.
> > +*
> > +* This program is distributed in the hope that it will be useful,
> > +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> > +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > +* GNU General Public License for more details.
> > +*/
> > +
> > +#include <linux/clk.h>
> > +#include <linux/of_address.h>
> > +#include <linux/of_platform.h>
> > +#include <linux/pm_runtime.h>
> > +#include <soc/mediatek/smi.h>
> > +
> > +#include "mtk_vcodec_pm.h"
> > +#include "mtk_vcodec_util.h"
> > +#include "mtk_vpu.h"
> > +
> > +
> > +int mtk_vcodec_init_enc_pm(struct mtk_vcodec_dev *mtkdev)
> > +{
> > +	struct device_node *node;
> > +	struct platform_device *pdev;
> > +	struct device *dev;
> > +	struct mtk_vcodec_pm *pm;
> > +	int ret = 0;
> > +
> > +	pdev = mtkdev->plat_dev;
> > +	pm = &mtkdev->pm;
> > +	memset(pm, 0, sizeof(struct mtk_vcodec_pm));
> > +	pm->mtkdev = mtkdev;
> > +	dev = &pdev->dev;
> > +
> > +	node = of_parse_phandle(dev->of_node, "mediatek,larb", 0);
> > +	if (!node)
> > +		return -1;
> > +	pdev = of_find_device_by_node(node);
> > +	if (WARN_ON(!pdev)) {
> > +		of_node_put(node);
> > +		return -1;
> > +	}
> > +	pm->larbvenc = &pdev->dev;
> > +
> > +	node = of_parse_phandle(dev->of_node, "mediatek,larb", 1);
> > +	if (!node)
> > +		return -1;
> > +
> > +	pdev = of_find_device_by_node(node);
> > +	if (WARN_ON(!pdev)) {
> > +		of_node_put(node);
> > +		return -EINVAL;
> > +	}
> > +	pm->larbvenclt = &pdev->dev;
> > +
> > +	pdev = mtkdev->plat_dev;
> > +	pm->dev = &pdev->dev;
> > +
> > +	pm->vencpll_d2 = devm_clk_get(&pdev->dev, "vencpll_d2");
> > +	if (pm->vencpll_d2 == NULL) {
> > +		mtk_v4l2_err("devm_clk_get vencpll_d2 fail");
> > +		ret = -1;
> > +	}
> > +
> > +	pm->venc_sel = devm_clk_get(&pdev->dev, "venc_sel");
> > +	if (pm->venc_sel == NULL) {
> > +		mtk_v4l2_err("devm_clk_get venc_sel fail");
> > +		ret = -1;
> > +	}
> > +
> > +	pm->univpll1_d2 = devm_clk_get(&pdev->dev, "univpll1_d2");
> > +	if (pm->univpll1_d2 == NULL) {
> > +		mtk_v4l2_err("devm_clk_get univpll1_d2 fail");
> > +		ret = -1;
> > +	}
> > +
> > +	pm->venc_lt_sel = devm_clk_get(&pdev->dev, "venc_lt_sel");
> > +	if (pm->venc_lt_sel == NULL) {
> > +		mtk_v4l2_err("devm_clk_get venc_lt_sel fail");
> > +		ret = -1;
> > +	}
> > +
> > +	return ret;
> > +}
> > +
> > +void mtk_vcodec_release_enc_pm(struct mtk_vcodec_dev *mtkdev)
> > +{
> > +}
> > +
> > +
> > +void mtk_vcodec_enc_clock_on(struct mtk_vcodec_pm *pm)
> > +{
> > +	int ret;
> > +
> > +	ret = clk_prepare_enable(pm->venc_sel);
> > +	if (ret)
> > +		mtk_v4l2_err("venc_sel fail %d", ret);
> > +
> > +	ret = clk_set_parent(pm->venc_sel, pm->vencpll_d2);
> > +	if (ret)
> > +		mtk_v4l2_err("clk_set_parent fail %d", ret);
> > +
> > +	ret = clk_prepare_enable(pm->venc_lt_sel);
> > +	if (ret)
> > +		mtk_v4l2_err("venc_lt_sel fail %d", ret);
> > +
> > +	ret = clk_set_parent(pm->venc_lt_sel, pm->univpll1_d2);
> > +	if (ret)
> > +		mtk_v4l2_err("clk_set_parent fail %d", ret);
> > +
> > +	ret = mtk_smi_larb_get(pm->larbvenc);
> > +	if (ret)
> > +		mtk_v4l2_err("mtk_smi_larb_get larb3 fail %d\n", ret);
> > +
> > +	ret = mtk_smi_larb_get(pm->larbvenclt);
> > +	if (ret)
> > +		mtk_v4l2_err("mtk_smi_larb_get larb4 fail %d\n", ret);
> > +
> > +}
> > +
> > +void mtk_vcodec_enc_clock_off(struct mtk_vcodec_pm *pm)
> > +{
> > +	mtk_smi_larb_put(pm->larbvenc);
> > +	mtk_smi_larb_put(pm->larbvenclt);
> > +	clk_disable_unprepare(pm->venc_lt_sel);
> > +	clk_disable_unprepare(pm->venc_sel);
> > +}
> > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.c
> > new file mode 100644
> > index 0000000..919b949
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.c
> > @@ -0,0 +1,102 @@
> > +/*
> > +* Copyright (c) 2015 MediaTek Inc.
> > +* Author: Tiffany Lin <tiffany.lin@mediatek.com>
> > +*
> > +* This program is free software; you can redistribute it and/or modify
> > +* it under the terms of the GNU General Public License version 2 as
> > +* published by the Free Software Foundation.
> > +*
> > +* This program is distributed in the hope that it will be useful,
> > +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> > +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > +* GNU General Public License for more details.
> > +*/
> > +
> > +#include <linux/errno.h>
> > +#include <linux/wait.h>
> > +
> > +#include "mtk_vcodec_drv.h"
> > +#include "mtk_vcodec_intr.h"
> > +#include "mtk_vcodec_util.h"
> > +
> > +void mtk_vcodec_clean_dev_int_flags(void *data)
> > +{
> > +	struct mtk_vcodec_dev *dev = (struct mtk_vcodec_dev *)data;
> > +
> > +	dev->int_cond = 0;
> > +	dev->int_type = 0;
> > +}
> > +
> > +int mtk_vcodec_wait_for_done_ctx(void *data, int command,
> > +				 unsigned int timeout_ms, int interrupt)
> > +{
> > +	wait_queue_head_t *waitqueue;
> > +	long timeout_jiff, ret;
> > +	int status = 0;
> > +	struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)data;
> > +
> > +	waitqueue = (wait_queue_head_t *)&ctx->queue;
> > +	timeout_jiff = msecs_to_jiffies(timeout_ms);
> > +	if (interrupt) {
> > +		ret = wait_event_interruptible_timeout(*waitqueue,
> > +				(ctx->int_cond &&
> > +				(ctx->int_type == command)),
> > +				timeout_jiff);
> > +	} else {
> > +		ret = wait_event_timeout(*waitqueue,
> > +				(ctx->int_cond &&
> > +				(ctx->int_type == command)),
> > +				 timeout_jiff);
> > +	}
> > +	if (0 == ret) {
> > +		status = -1;	/* timeout */
> > +		mtk_v4l2_err("[%d] cmd=%d, ctx->type=%d, wait_event_interruptible_timeout time=%ums out %d %d!",
> > +				ctx->idx, ctx->type, command, timeout_ms,
> > +				ctx->int_cond, ctx->int_type);
> > +	} else if (-ERESTARTSYS == ret) {
> > +		mtk_v4l2_err("[%d] cmd=%d, ctx->type=%d, wait_event_interruptible_timeout interrupted by a signal %d %d",
> > +				ctx->idx, ctx->type, command, ctx->int_cond,
> > +				ctx->int_type);
> > +		status = -1;
> > +	}
> > +
> > +	ctx->int_cond = 0;
> > +	ctx->int_type = 0;
> > +
> > +	return status;
> > +}
> > +
> > +int mtk_vcodec_wait_for_done_dev(void *data, int command,
> > +				 unsigned int timeout, int interrupt)
> > +{
> > +	wait_queue_head_t *waitqueue;
> > +	long timeout_jiff, ret;
> > +	int status = 0;
> > +	struct mtk_vcodec_dev *dev = (struct mtk_vcodec_dev *)data;
> > +
> > +	waitqueue = (wait_queue_head_t *)&dev->queue;
> > +	timeout_jiff = msecs_to_jiffies(timeout);
> > +	if (interrupt) {
> > +		ret = wait_event_interruptible_timeout(*waitqueue,
> > +				(dev->int_cond &&
> > +				(dev->int_type == command)),
> > +				timeout_jiff);
> > +	} else {
> > +		ret = wait_event_timeout(*waitqueue,
> > +				(dev->int_cond &&
> > +				(dev->int_type == command)),
> > +				timeout_jiff);
> > +	}
> > +	if (0 == ret) {
> > +		status = -1;	/* timeout */
> > +		mtk_v4l2_err("wait_event_interruptible_timeout time=%lu out %d %d!",
> > +				timeout_jiff, dev->int_cond, dev->int_type);
> > +	} else if (-ERESTARTSYS == ret) {
> > +		mtk_v4l2_err("wait_event_interruptible_timeout interrupted by a signal %d %d",
> > +				dev->int_cond, dev->int_type);
> > +		status = -1;
> > +	}
> > +	dev->int_cond = 0;
> > +	dev->int_type = 0;
> > +	return status;
> > +}
> > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.h
> > new file mode 100644
> > index 0000000..e9b7f94
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.h
> > @@ -0,0 +1,29 @@
> > +/*
> > +* Copyright (c) 2015 MediaTek Inc.
> > +* Author: Tiffany Lin <tiffany.lin@mediatek.com>
> > +*
> > +* This program is free software; you can redistribute it and/or modify
> > +* it under the terms of the GNU General Public License version 2 as
> > +* published by the Free Software Foundation.
> > +*
> > +* This program is distributed in the hope that it will be useful,
> > +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> > +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > +* GNU General Public License for more details.
> > +*/
> > +
> > +#ifndef _MTK_VCODEC_INTR_H_
> > +#define _MTK_VCODEC_INTR_H_
> > +
> > +#define MTK_INST_IRQ_RECEIVED		0x1
> > +#define MTK_INST_WORK_THREAD_ABORT_DONE	0x2
> > +
> > +/* timeout is ms */
> > +int mtk_vcodec_wait_for_done_ctx(void *data, int command, unsigned int timeout,
> > +				 int interrupt);
> > +int mtk_vcodec_wait_for_done_dev(void *data, int command, unsigned int timeout,
> > +				 int interrupt);
> > +
> > +void mtk_vcodec_clean_dev_int_flags(void *data);
> > +
> > +#endif /* _MTK_VCODEC_INTR_H_ */
> > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_pm.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_pm.h
> > new file mode 100644
> > index 0000000..fdadec9
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_pm.h
> > @@ -0,0 +1,26 @@
> > +/*
> > +* Copyright (c) 2015 MediaTek Inc.
> > +* Author: Tiffany Lin <tiffany.lin@mediatek.com>
> > +*
> > +* This program is free software; you can redistribute it and/or modify
> > +* it under the terms of the GNU General Public License version 2 as
> > +* published by the Free Software Foundation.
> > +*
> > +* This program is distributed in the hope that it will be useful,
> > +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> > +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > +* GNU General Public License for more details.
> > +*/
> > +
> > +#ifndef _MTK_VCODEC_PM_H_
> > +#define _MTK_VCODEC_PM_H_
> > +
> > +#include "mtk_vcodec_drv.h"
> > +
> > +int mtk_vcodec_init_enc_pm(struct mtk_vcodec_dev *dev);
> > +void mtk_vcodec_release_enc_pm(struct mtk_vcodec_dev *dev);
> > +
> > +void mtk_vcodec_enc_clock_on(struct mtk_vcodec_pm *pm);
> > +void mtk_vcodec_enc_clock_off(struct mtk_vcodec_pm *pm);
> > +
> > +#endif /* _MTK_VCODEC_PM_H_ */
> > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c
> > new file mode 100644
> > index 0000000..3fede8d
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c
> > @@ -0,0 +1,106 @@
> > +/*
> > +* Copyright (c) 2015 MediaTek Inc.
> > +* Author: PC Chen <pc.chen@mediatek.com>
> > +*         Tiffany Lin <tiffany.lin@mediatek.com>
> > +*
> > +* This program is free software; you can redistribute it and/or modify
> > +* it under the terms of the GNU General Public License version 2 as
> > +* published by the Free Software Foundation.
> > +*
> > +* This program is distributed in the hope that it will be useful,
> > +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> > +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > +* GNU General Public License for more details.
> > +*/
> > +
> > +#include <linux/module.h>
> > +
> > +#include "mtk_vcodec_drv.h"
> > +#include "mtk_vcodec_util.h"
> > +#include "mtk_vpu.h"
> > +
> > +bool mtk_vcodec_dbg = false;
> > +int mtk_v4l2_dbg_level = 0;
> > +
> > +module_param(mtk_v4l2_dbg_level, int, S_IRUGO | S_IWUSR);
> > +module_param(mtk_vcodec_dbg, bool, S_IRUGO | S_IWUSR);
> > +
> > +void __iomem *mtk_vcodec_get_reg_addr(void *data, unsigned int reg_idx)
> > +{
> > +	struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)data;
> > +
> > +	if (!data || reg_idx >= NUM_MAX_VCODEC_REG_BASE) {
> > +		mtk_v4l2_err("Invalid arguments");
> > +		return NULL;
> > +	}
> > +	return ctx->dev->reg_base[reg_idx];
> > +}
> > +
> > +int mtk_vcodec_mem_alloc(void *data, struct mtk_vcodec_mem *mem)
> > +{
> > +	unsigned long size = mem->size;
> > +	struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)data;
> > +	struct device *dev = &ctx->dev->plat_dev->dev;
> > +
> > +	mem->va = dma_alloc_coherent(dev, size, &mem->dma_addr, GFP_KERNEL);
> > +
> > +	if (!mem->va) {
> > +		mtk_v4l2_err("%s dma_alloc size=%ld failed!", dev_name(dev),
> > +			       size);
> > +		return -ENOMEM;
> > +	}
> > +
> > +	memset(mem->va, 0, size);
> > +
> > +	mtk_v4l2_debug(3, "[%d]  - va      = %p", ctx->idx, mem->va);
> > +	mtk_v4l2_debug(3, "[%d]  - dma     = 0x%lx", ctx->idx,
> > +			 (unsigned long)mem->dma_addr);
> > +	mtk_v4l2_debug(3, "[%d]    size = 0x%lx", ctx->idx, size);
> > +
> > +	return 0;
> > +}
> > +
> > +void mtk_vcodec_mem_free(void *data, struct mtk_vcodec_mem *mem)
> > +{
> > +	unsigned long size = mem->size;
> > +	struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)data;
> > +	struct device *dev = &ctx->dev->plat_dev->dev;
> > +
> > +	dma_free_coherent(dev, size, mem->va, mem->dma_addr);
> > +	mem->va = NULL;
> > +
> > +	mtk_v4l2_debug(3, "[%d]  - va      = %p", ctx->idx, mem->va);
> > +	mtk_v4l2_debug(3, "[%d]  - dma     = 0x%lx", ctx->idx,
> > +			 (unsigned long)mem->dma_addr);
> > +	mtk_v4l2_debug(3, "[%d]    size = 0x%lx", ctx->idx, size);
> > +}
> > +
> > +int mtk_vcodec_get_ctx_id(void *data)
> > +{
> > +	struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)data;
> > +
> > +	if (!ctx)
> > +		return -1;
> > +
> > +	return ctx->idx;
> > +}
> > +
> > +struct platform_device *mtk_vcodec_get_plat_dev(void *data)
> > +{
> > +	struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)data;
> > +
> > +	if (!ctx)
> > +		return NULL;
> > +
> > +	return vpu_get_plat_device(ctx->dev->plat_dev);
> > +}
> > +
> > +void mtk_vcodec_fmt2str(u32 fmt, char *str)
> > +{
> > +	char a = fmt & 0xFF;
> > +	char b = (fmt >> 8) & 0xFF;
> > +	char c = (fmt >> 16) & 0xFF;
> > +	char d = (fmt >> 24) & 0xFF;
> > +
> > +	sprintf(str, "%c%c%c%c", a, b, c, d);
> > +}
> > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
> > new file mode 100644
> > index 0000000..47016ae
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
> > @@ -0,0 +1,85 @@
> > +/*
> > +* Copyright (c) 2015 MediaTek Inc.
> > +* Author: PC Chen <pc.chen@mediatek.com>
> > +*         Tiffany Lin <tiffany.lin@mediatek.com>
> > +*
> > +* This program is free software; you can redistribute it and/or modify
> > +* it under the terms of the GNU General Public License version 2 as
> > +* published by the Free Software Foundation.
> > +*
> > +* This program is distributed in the hope that it will be useful,
> > +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> > +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > +* GNU General Public License for more details.
> > +*/
> > +
> > +#ifndef _MTK_VCODEC_UTIL_H_
> > +#define _MTK_VCODEC_UTIL_H_
> > +
> > +#include <linux/types.h>
> > +#include <linux/dma-direction.h>
> > +
> > +struct mtk_vcodec_mem {
> > +	size_t size;
> > +	void *va;
> > +	dma_addr_t dma_addr;
> > +};
> > +
> > +extern int mtk_v4l2_dbg_level;
> > +extern bool mtk_vcodec_dbg;
> > +
> > +#define DEBUG 	1
> > +
> > +#if defined(DEBUG)
> > +
> > +#define mtk_v4l2_debug(level, fmt, args...)				 \
> > +	do {								 \
> > +		if (mtk_v4l2_dbg_level >= level)			 \
> > +			pr_info("[MTK_V4L2] level=%d %s(),%d: " fmt "\n",\
> > +				level, __func__, __LINE__, ##args);	 \
> > +	} while (0)
> > +
> > +#define mtk_v4l2_err(fmt, args...)                \
> > +	pr_err("[MTK_V4L2][ERROR] %s:%d: " fmt "\n", __func__, __LINE__, \
> > +	       ##args)
> > +
> > +
> > +#define mtk_v4l2_debug_enter()  mtk_v4l2_debug(3, "+\n")
> > +#define mtk_v4l2_debug_leave()  mtk_v4l2_debug(3, "-\n")
> > +
> > +#define mtk_vcodec_debug(h, fmt, args...)				\
> > +	do {								\
> > +		if (mtk_vcodec_dbg)					\
> > +			pr_info("[MTK_VCODEC][%d]: %s() " fmt "\n",	\
> > +				((struct mtk_vcodec_ctx *)h->ctx)->idx, \
> > +				__func__, ##args);			\
> > +	} while (0)
> > +
> > +#define mtk_vcodec_err(h, fmt, args...)					\
> > +	pr_err("[MTK_VCODEC][ERROR][%d]: %s() " fmt "\n",		\
> > +	       ((struct mtk_vcodec_ctx *)h->ctx)->idx, __func__, ##args)
> > +
> > +#define mtk_vcodec_debug_enter(h)  mtk_vcodec_debug(h, "+\n")
> > +#define mtk_vcodec_debug_leave(h)  mtk_vcodec_debug(h, "-\n")
> > +
> > +#else
> > +
> > +#define mtk_v4l2_debug(level, fmt, args...)
> > +#define mtk_v4l2_err(fmt, args...)
> > +#define mtk_v4l2_debug_enter()
> > +#define mtk_v4l2_debug_leave()
> > +
> > +#define mtk_vcodec_debug(h, fmt, args...)
> > +#define mtk_vcodec_err(h, fmt, args...)
> > +#define mtk_vcodec_debug_enter(h)
> > +#define mtk_vcodec_debug_leave(h)
> > +
> > +#endif
> > +
> > +void __iomem *mtk_vcodec_get_reg_addr(void *data, unsigned int reg_idx);
> > +int mtk_vcodec_mem_alloc(void *data, struct mtk_vcodec_mem *mem);
> > +void mtk_vcodec_mem_free(void *data, struct mtk_vcodec_mem *mem);
> > +int mtk_vcodec_get_ctx_id(void *data);
> > +struct platform_device *mtk_vcodec_get_plat_dev(void *data);
> > +void mtk_vcodec_fmt2str(u32 fmt, char *str);
> > +#endif /* _MTK_VCODEC_UTIL_H_ */
> > diff --git a/drivers/media/platform/mtk-vcodec/venc_drv_base.h b/drivers/media/platform/mtk-vcodec/venc_drv_base.h
> > new file mode 100644
> > index 0000000..ed9cbf0
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/venc_drv_base.h
> > @@ -0,0 +1,62 @@
> > +/*
> > + * Copyright (c) 2015 MediaTek Inc.
> > + * Author: Daniel Hsiao <daniel.hsiao@mediatek.com>
> > + *         Jungchang Tsao <jungchang.tsao@mediatek.com>
> > + *
> > + * This program is free software; you can redistribute it and/or
> > + * modify
> > + * it under the terms of the GNU General Public License version 2 as
> > + * published by the Free Software Foundation.
> > + *
> > + * This program is distributed in the hope that it will be useful,
> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > + * GNU General Public License for more details.
> > + */
> > +
> > +#ifndef _VENC_DRV_BASE_
> > +#define _VENC_DRV_BASE_
> > +
> > +#include "mtk_vcodec_drv.h"
> > +
> > +#include "venc_drv_if.h"
> > +
> > +struct venc_common_if {
> > +	/**
> > +	 * (*init)() - initialize driver
> > +	 * @ctx:	[in] mtk v4l2 context
> > +	 * @handle: [out] driver handle
> > +	 */
> > +	int (*init)(struct mtk_vcodec_ctx *ctx, unsigned long *handle);
> > +
> > +	/**
> > +	 * (*encode)() - trigger encode
> > +	 * @handle: [in] driver handle
> > +	 * @opt: [in] encode option
> > +	 * @frm_buf: [in] frame buffer to store input frame
> > +	 * @bs_buf: [in] bitstream buffer to store output bitstream
> > +	 * @result: [out] encode result
> > +	 */
> > +	int (*encode)(unsigned long handle, enum venc_start_opt opt,
> > +		      struct venc_frm_buf *frm_buf,
> > +		      struct mtk_vcodec_mem *bs_buf,
> > +		      struct venc_done_result *result);
> > +
> > +	/**
> > +	 * (*set_param)() - set driver's parameter
> > +	 * @handle: [in] driver handle
> > +	 * @type: [in] parameter type
> > +	 * @in: [in] buffer to store the parameter
> > +	 */
> > +	int (*set_param)(unsigned long handle, enum venc_set_param_type type,
> > +			 void *in);
> > +
> > +	/**
> > +	 * (*deinit)() - deinitialize driver.
> > +	 * @handle: [in] driver handle
> > +	 */
> > +	int (*deinit)(unsigned long handle);
> > +};
> > +
> > +
> > +#endif
> > diff --git a/drivers/media/platform/mtk-vcodec/venc_drv_if.c b/drivers/media/platform/mtk-vcodec/venc_drv_if.c
> > new file mode 100644
> > index 0000000..daa8e93
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/venc_drv_if.c
> > @@ -0,0 +1,100 @@
> > +/*
> > + * Copyright (c) 2015 MediaTek Inc.
> > + * Author: Daniel Hsiao <daniel.hsiao@mediatek.com>
> > + *         Jungchang Tsao <jungchang.tsao@mediatek.com>
> > + *
> > + * This program is free software; you can redistribute it and/or
> > + * modify
> > + * it under the terms of the GNU General Public License version 2 as
> > + * published by the Free Software Foundation.
> > + *
> > + * This program is distributed in the hope that it will be useful,
> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > + * GNU General Public License for more details.
> > + */
> > +
> > +#include <linux/interrupt.h>
> > +#include <linux/kernel.h>
> > +#include <linux/slab.h>
> > +
> > +#include "venc_drv_if.h"
> > +#include "mtk_vcodec_enc.h"
> > +#include "mtk_vcodec_pm.h"
> > +#include "mtk_vpu.h"
> > +
> > +#include "venc_drv_base.h"
> > +
> > +int venc_if_create(struct mtk_vcodec_ctx *ctx, unsigned int fourcc)
> > +{
> > +	char str[10];
> > +	int ret = 0;
> > +
> > +	mtk_vcodec_fmt2str(fourcc, str);
> > +
> > +	switch (fourcc) {
> > +	case V4L2_PIX_FMT_VP8:
> > +	case V4L2_PIX_FMT_H264:
> > +	default:
> > +		return -EINVAL;
> > +	}
> > +
> > +	mtk_venc_lock(ctx);
> > +	mtk_vcodec_enc_clock_on(&ctx->dev->pm);
> > +	ret = ctx->enc_if->init(ctx, (unsigned long *)&ctx->drv_handle);
> > +	mtk_vcodec_enc_clock_off(&ctx->dev->pm);
> > +	mtk_venc_unlock(ctx);
> > +
> > +	return ret;
> > +
> > +}
> > +
> > +
> > +int venc_if_set_param(struct mtk_vcodec_ctx *ctx,
> > +		      enum venc_set_param_type type, void *in)
> > +{
> > +	int ret = 0;
> > +
> > +	mtk_venc_lock(ctx);
> > +	mtk_vcodec_enc_clock_on(&ctx->dev->pm);
> > +	ret = ctx->enc_if->set_param(ctx->drv_handle, type, in);
> > +	mtk_vcodec_enc_clock_off(&ctx->dev->pm);
> > +	mtk_venc_unlock(ctx);
> > +
> > +	return ret;
> > +}
> > +
> > +int venc_if_encode(struct mtk_vcodec_ctx *ctx,
> > +		   enum venc_start_opt opt, struct venc_frm_buf *frm_buf,
> > +		   struct mtk_vcodec_mem *bs_buf,
> > +		   struct venc_done_result *result)
> > +{
> > +	int ret = 0;
> > +	
> > +	mtk_venc_lock(ctx);
> > +	mtk_vcodec_enc_clock_on(&ctx->dev->pm);
> > +	ret = ctx->enc_if->encode(ctx->drv_handle, opt, frm_buf, bs_buf, result);
> > +	mtk_vcodec_enc_clock_off(&ctx->dev->pm);
> > +	mtk_venc_unlock(ctx);
> > +	
> > +	return ret;
> > +}
> > +
> > +int venc_if_release(struct mtk_vcodec_ctx *ctx)
> > +{
> > +	int ret = 0;
> > +
> > +	if(ctx->drv_handle == 0)
> > +		return 0;
> > +
> > +	mtk_venc_lock(ctx);
> > +	mtk_vcodec_enc_clock_on(&ctx->dev->pm);
> > +	ret = ctx->enc_if->deinit(ctx->drv_handle);
> > +	mtk_vcodec_enc_clock_off(&ctx->dev->pm);
> > +	mtk_venc_unlock(ctx);
> > +
> > +	ctx->drv_handle = 0;
> > +
> > +	return ret;
> > +}
> > +
> > diff --git a/drivers/media/platform/mtk-vcodec/venc_drv_if.h b/drivers/media/platform/mtk-vcodec/venc_drv_if.h
> > new file mode 100644
> > index 0000000..a387011
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/venc_drv_if.h
> > @@ -0,0 +1,175 @@
> > +/*
> > + * Copyright (c) 2015 MediaTek Inc.
> > + * Author: Daniel Hsiao <daniel.hsiao@mediatek.com>
> > + *         Jungchang Tsao <jungchang.tsao@mediatek.com>
> > + *
> > + * This program is free software; you can redistribute it and/or
> > + * modify
> > + * it under the terms of the GNU General Public License version 2 as
> > + * published by the Free Software Foundation.
> > + *
> > + * This program is distributed in the hope that it will be useful,
> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > + * GNU General Public License for more details.
> > + */
> > +
> > +#ifndef _VENC_DRV_IF_H_
> > +#define _VENC_DRV_IF_H_
> > +
> > +#include "mtk_vcodec_drv.h"
> > +#include "mtk_vcodec_util.h"
> > +
> > +/*
> > + * enum venc_yuv_fmt - The type of input yuv format
> > + * (VPU related: If you change the order, you must also update the VPU codes.)
> > + * @VENC_YUV_FORMAT_420: 420 YUV format
> > + * @VENC_YUV_FORMAT_YV12: YV12 YUV format
> > + * @VENC_YUV_FORMAT_NV12: NV12 YUV format
> > + * @VENC_YUV_FORMAT_NV21: NV21 YUV format
> > + */
> > +enum venc_yuv_fmt {
> > +	VENC_YUV_FORMAT_420 = 3,
> > +	VENC_YUV_FORMAT_YV12 = 5,
> > +	VENC_YUV_FORMAT_NV12 = 6,
> > +	VENC_YUV_FORMAT_NV21 = 7,
> > +};
> > +
> > +/*
> > + * enum venc_start_opt - encode frame option used in venc_if_encode()
> > + * @VENC_START_OPT_ENCODE_SEQUENCE_HEADER: encode SPS/PPS for H264
> > + * @VENC_START_OPT_ENCODE_FRAME: encode normal frame
> > + */
> > +enum venc_start_opt {
> > +	VENC_START_OPT_ENCODE_SEQUENCE_HEADER,
> > +	VENC_START_OPT_ENCODE_FRAME,
> > +};
> > +
> > +/*
> > + * enum venc_drv_msg - The type of encode frame status used in venc_if_encode()
> > + * @VENC_MESSAGE_OK: encode ok
> > + * @VENC_MESSAGE_ERR: encode error
> > + */
> > +enum venc_drv_msg {
> > +	VENC_MESSAGE_OK,
> > +	VENC_MESSAGE_ERR,
> > +};
> > +
> > +/*
> > + * enum venc_set_param_type - The type of set parameter used in venc_if_set_param()
> > + * (VPU related: If you change the order, you must also update the VPU codes.)
> > + * @VENC_SET_PARAM_ENC: set encoder parameters
> > + * @VENC_SET_PARAM_FORCE_INTRA: set force intra frame
> > + * @VENC_SET_PARAM_ADJUST_BITRATE: set to adjust bitrate (in bps)
> > + * @VENC_SET_PARAM_ADJUST_FRAMERATE: set frame rate
> > + * @VENC_SET_PARAM_I_FRAME_INTERVAL: set I frame interval
> > + * @VENC_SET_PARAM_SKIP_FRAME: set H264 skip one frame
> > + * @VENC_SET_PARAM_PREPEND_HEADER: set H264 prepend SPS/PPS before IDR
> > + * @VENC_SET_PARAM_TS_MODE: set VP8 temporal scalability mode
> > + */
> > +enum venc_set_param_type {
> > +	VENC_SET_PARAM_ENC,
> > +	VENC_SET_PARAM_FORCE_INTRA,
> > +	VENC_SET_PARAM_ADJUST_BITRATE,
> > +	VENC_SET_PARAM_ADJUST_FRAMERATE,
> > +	VENC_SET_PARAM_I_FRAME_INTERVAL,
> > +	VENC_SET_PARAM_SKIP_FRAME,
> > +	VENC_SET_PARAM_PREPEND_HEADER,
> > +	VENC_SET_PARAM_TS_MODE,
> > +};
> > +
> > +/*
> > + * struct venc_enc_prm - encoder settings for VENC_SET_PARAM_ENC used in venc_if_set_param()
> > + * @input_fourcc: input fourcc
> > + * @h264_profile: V4L2 defined H.264 profile
> > + * @h264_level: V4L2 defined H.264 level
> > + * @width: image width
> > + * @height: image height
> > + * @buf_width: buffer width
> > + * @buf_height: buffer height
> > + * @frm_rate: frame rate
> > + * @intra_period: intra frame period
> > + * @bitrate: target bitrate in kbps
> > + */
> > +struct venc_enc_prm {
> > +	enum venc_yuv_fmt input_fourcc;
> > +	unsigned int h264_profile;
> > +	unsigned int h264_level;
> > +	unsigned int width;
> > +	unsigned int height;
> > +	unsigned int buf_width;
> > +	unsigned int buf_height;
> > +	unsigned int frm_rate;
> > +	unsigned int intra_period;
> > +	unsigned int bitrate;
> > +	unsigned int sizeimage[MTK_VCODEC_MAX_PLANES];
> > +};
> > +
> > +/*
> > + * struct venc_frm_buf - frame buffer information used in venc_if_encode()
> > + * @fb_addr: plane 0 frame buffer address
> > + * @fb_addr1: plane 1 frame buffer address
> > + * @fb_addr2: plane 2 frame buffer address
> > + */
> > +struct venc_frm_buf {
> > +	struct mtk_vcodec_mem fb_addr;
> > +	struct mtk_vcodec_mem fb_addr1;
> > +	struct mtk_vcodec_mem fb_addr2;
> > +};
> > +
> > +/*
> > + * struct venc_done_result - This is return information used in venc_if_encode()
> > + * @msg: message, such as success or error code
> > + * @bs_size: output bitstream size
> > + * @is_key_frm: output is key frame or not
> > + */
> > +struct venc_done_result {
> > +	enum venc_drv_msg msg;
> > +	unsigned int bs_size;
> > +	bool is_key_frm;
> > +};
> > +
> > +/*
> > + * venc_if_create - Create the driver handle
> > + * @ctx: device context
> > + * @fourcc: encoder output format
> > + * @handle: driver handle
> > + * Return: 0 if creating handle successfully, otherwise it is failed.
> > + */
> > +int venc_if_create(struct mtk_vcodec_ctx *ctx, unsigned int fourcc);
> > +
> > +/*
> > + * venc_if_release - Release the driver handle
> > + * @handle: driver handle
> > + * Return: 0 if releasing handle successfully, otherwise it is failed.
> > + */
> > +int venc_if_release(struct mtk_vcodec_ctx *ctx);
> > +
> > +/*
> > + * venc_if_set_param - Set parameter to driver
> > + * @handle: driver handle
> > + * @type: set type
> > + * @in: input parameter
> > + * @out: output parameter
> > + * Return: 0 if setting param successfully, otherwise it is failed.
> > + */
> > +int venc_if_set_param(struct mtk_vcodec_ctx *ctx,
> > +		      enum venc_set_param_type type,
> > +		      void *in);
> > +
> > +/*
> > + * venc_if_encode - Encode frame
> > + * @handle: driver handle
> > + * @opt: encode frame option
> > + * @frm_buf: input frame buffer information
> > + * @bs_buf: output bitstream buffer infomraiton
> > + * @result: encode result
> > + * Return: 0 if encoding frame successfully, otherwise it is failed.
> > + */
> > +int venc_if_encode(struct mtk_vcodec_ctx *ctx,
> > +		   enum venc_start_opt opt,
> > +		   struct venc_frm_buf *frm_buf,
> > +		   struct mtk_vcodec_mem *bs_buf,
> > +		   struct venc_done_result *result);
> > +
> > +#endif /* _VENC_DRV_IF_H_ */
> > diff --git a/drivers/media/platform/mtk-vcodec/venc_ipi_msg.h b/drivers/media/platform/mtk-vcodec/venc_ipi_msg.h
> > new file mode 100644
> > index 0000000..a345b98
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/venc_ipi_msg.h
> > @@ -0,0 +1,212 @@
> > +/*
> > + * Copyright (c) 2015 MediaTek Inc.
> > + * Author: Jungchang Tsao <jungchang.tsao@mediatek.com>
> > + *         Daniel Hsiao <daniel.hsiao@mediatek.com>
> > + *
> > + * This program is free software; you can redistribute it and/or
> > + * modify
> > + * it under the terms of the GNU General Public License version 2 as
> > + * published by the Free Software Foundation.
> > + *
> > + * This program is distributed in the hope that it will be useful,
> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > + * GNU General Public License for more details.
> > + */
> > +
> > +#ifndef _VENC_IPI_MSG_H_
> > +#define _VENC_IPI_MSG_H_
> > +
> > +#define IPIMSG_H264_ENC_ID 0x100
> > +#define IPIMSG_VP8_ENC_ID 0x200
> > +
> > +#define AP_IPIMSG_VENC_BASE 0x20000
> > +#define VPU_IPIMSG_VENC_BASE 0x30000
> > +
> > +/**
> > + * enum venc_ipi_msg_id - message id between AP and VPU
> > + * (ipi stands for inter-processor interrupt)
> > + * @AP_IPIMSG_XXX:		AP to VPU cmd message id
> > + * @VPU_IPIMSG_XXX_DONE:	VPU ack AP cmd message id
> > + */
> > +enum venc_ipi_msg_id {
> > +	AP_IPIMSG_H264_ENC_INIT = AP_IPIMSG_VENC_BASE +
> > +				  IPIMSG_H264_ENC_ID,
> > +	AP_IPIMSG_H264_ENC_SET_PARAM,
> > +	AP_IPIMSG_H264_ENC_ENCODE,
> > +	AP_IPIMSG_H264_ENC_DEINIT,
> > +
> > +	AP_IPIMSG_VP8_ENC_INIT = AP_IPIMSG_VENC_BASE +
> > +				 IPIMSG_VP8_ENC_ID,
> > +	AP_IPIMSG_VP8_ENC_SET_PARAM,
> > +	AP_IPIMSG_VP8_ENC_ENCODE,
> > +	AP_IPIMSG_VP8_ENC_DEINIT,
> > +
> > +	VPU_IPIMSG_H264_ENC_INIT_DONE = VPU_IPIMSG_VENC_BASE +
> > +					IPIMSG_H264_ENC_ID,
> > +	VPU_IPIMSG_H264_ENC_SET_PARAM_DONE,
> > +	VPU_IPIMSG_H264_ENC_ENCODE_DONE,
> > +	VPU_IPIMSG_H264_ENC_DEINIT_DONE,
> > +
> > +	VPU_IPIMSG_VP8_ENC_INIT_DONE = VPU_IPIMSG_VENC_BASE +
> > +				       IPIMSG_VP8_ENC_ID,
> > +	VPU_IPIMSG_VP8_ENC_SET_PARAM_DONE,
> > +	VPU_IPIMSG_VP8_ENC_ENCODE_DONE,
> > +	VPU_IPIMSG_VP8_ENC_DEINIT_DONE,
> > +};
> > +
> > +/**
> > + * struct venc_ap_ipi_msg_init - AP to VPU init cmd structure
> > + * @msg_id:	message id (AP_IPIMSG_XXX_ENC_INIT)
> > + * @venc_inst:	AP encoder instance (struct venc_vp8_handle/venc_h264_handle *)
> > + */
> > +struct venc_ap_ipi_msg_init {
> > +	uint32_t msg_id;
> > +	uint32_t reserved;
> > +	uint64_t venc_inst;
> > +};
> > +
> > +/**
> > + * struct venc_ap_ipi_msg_set_param - AP to VPU set_param cmd structure
> > + * @msg_id:	message id (AP_IPIMSG_XXX_ENC_SET_PARAM)
> > + * @inst_id:	VPU encoder instance id (struct venc_vp8_vpu_drv/venc_h264_vpu_drv *)
> > + * @param_id:	parameter id (venc_set_param_type)
> > + * @data_item:	number of items in the data array
> > + * @data[8]:	data array to store the set parameters
> > + */
> > +struct venc_ap_ipi_msg_set_param {
> > +	uint32_t msg_id;
> > +	uint32_t inst_id;
> > +	uint32_t param_id;
> > +	uint32_t data_item;
> > +	uint32_t data[8];
> > +};
> > +
> > +/**
> > + * struct venc_ap_ipi_msg_enc - AP to VPU enc cmd structure
> > + * @msg_id:	message id (AP_IPIMSG_XXX_ENC_ENCODE)
> > + * @inst_id:	VPU encoder instance id (struct venc_vp8_vpu_drv/venc_h264_vpu_drv *)
> > + * @bs_mode:	bitstream mode for h264
> > + *		(H264_BS_MODE_SPS/H264_BS_MODE_PPS/H264_BS_MODE_FRAME)
> > + * @input_addr:	pointer to input image buffer plane
> > + * @bs_addr:	pointer to output bit stream buffer
> > + * @bs_size:	bit stream buffer size
> > + */
> > +struct venc_ap_ipi_msg_enc {
> > +	uint32_t msg_id;
> > +	uint32_t inst_id;
> > +	uint32_t bs_mode;
> > +	uint32_t input_addr[3];
> > +	uint32_t bs_addr;
> > +	uint32_t bs_size;
> > +};
> > +
> > +/**
> > + * struct venc_ap_ipi_msg_deinit - AP to VPU deinit cmd structure
> > + * @msg_id:	message id (AP_IPIMSG_XXX_ENC_DEINIT)
> > + * @inst_id:	VPU encoder instance id (struct venc_vp8_vpu_drv/venc_h264_vpu_drv *)
> > + */
> > +struct venc_ap_ipi_msg_deinit {
> > +	uint32_t msg_id;
> > +	uint32_t inst_id;
> > +};
> > +
> > +/**
> > + * enum venc_ipi_msg_status - VPU ack AP cmd status
> > + */
> > +enum venc_ipi_msg_status {
> > +	VENC_IPI_MSG_STATUS_OK,
> > +	VENC_IPI_MSG_STATUS_FAIL,
> > +};
> > +
> > +/**
> > + * struct venc_vpu_ipi_msg_common - VPU ack AP cmd common structure
> > + * @msg_id:	message id (VPU_IPIMSG_XXX_DONE)
> > + * @status:	cmd status (venc_ipi_msg_status)
> > + * @venc_inst:	AP encoder instance (struct venc_vp8_handle/venc_h264_handle *)
> > + */
> > +struct venc_vpu_ipi_msg_common {
> > +	uint32_t msg_id;
> > +	uint32_t status;
> > +	uint64_t venc_inst;
> > +};
> > +
> > +/**
> > + * struct venc_vpu_ipi_msg_init - VPU ack AP init cmd structure
> > + * @msg_id:	message id (VPU_IPIMSG_XXX_ENC_SET_PARAM_DONE)
> > + * @status:	cmd status (venc_ipi_msg_status)
> > + * @venc_inst:	AP encoder instance (struct venc_vp8_handle/venc_h264_handle *)
> > + * @inst_id:	VPU encoder instance id (struct venc_vp8_vpu_drv/venc_h264_vpu_drv *)
> > + */
> > +struct venc_vpu_ipi_msg_init {
> > +	uint32_t msg_id;
> > +	uint32_t status;
> > +	uint64_t venc_inst;
> > +	uint32_t inst_id;
> > +	uint32_t reserved;
> > +};
> > +
> > +/**
> > + * struct venc_vpu_ipi_msg_set_param - VPU ack AP set_param cmd structure
> > + * @msg_id:	message id (VPU_IPIMSG_XXX_ENC_SET_PARAM_DONE)
> > + * @status:	cmd status (venc_ipi_msg_status)
> > + * @venc_inst:	AP encoder instance (struct venc_vp8_handle/venc_h264_handle *)
> > + * @param_id:	parameter id (venc_set_param_type)
> > + * @data_item:	number of items in the data array
> > + * @data[6]:	data array to store the return result
> > + */
> > +struct venc_vpu_ipi_msg_set_param {
> > +	uint32_t msg_id;
> > +	uint32_t status;
> > +	uint64_t venc_inst;
> > +	uint32_t param_id;
> > +	uint32_t data_item;
> > +	uint32_t data[6];
> > +};
> > +
> > +/**
> > + * enum venc_ipi_msg_enc_state - Type of encode state
> > + * VEN_IPI_MSG_ENC_STATE_FRAME:	one frame being encoded
> > + * VEN_IPI_MSG_ENC_STATE_PART:	bit stream buffer full
> > + * VEN_IPI_MSG_ENC_STATE_SKIP:	encoded skip frame
> > + * VEN_IPI_MSG_ENC_STATE_ERROR:	encounter error
> > + */
> > +enum venc_ipi_msg_enc_state {
> > +	VEN_IPI_MSG_ENC_STATE_FRAME,
> > +	VEN_IPI_MSG_ENC_STATE_PART,
> > +	VEN_IPI_MSG_ENC_STATE_SKIP,
> > +	VEN_IPI_MSG_ENC_STATE_ERROR,
> > +};
> > +
> > +/**
> > + * struct venc_vpu_ipi_msg_enc - VPU ack AP enc cmd structure
> > + * @msg_id:	message id (VPU_IPIMSG_XXX_ENC_ENCODE_DONE)
> > + * @status:	cmd status (venc_ipi_msg_status)
> > + * @venc_inst:	AP encoder instance (struct venc_vp8_handle/venc_h264_handle *)
> > + * @state:	encode state (venc_ipi_msg_enc_state)
> > + * @key_frame:	whether the encoded frame is key frame
> > + * @bs_size:	encoded bitstream size
> 
> 'reserved' isn't documented. I assume this should be set to 0?
> 
Yes. We will fix this.

> > + */
> > +struct venc_vpu_ipi_msg_enc {
> > +	uint32_t msg_id;
> > +	uint32_t status;
> > +	uint64_t venc_inst;
> > +	uint32_t state;
> > +	uint32_t key_frame;
> > +	uint32_t bs_size;
> > +	uint32_t reserved;
> > +};
> > +
> > +/**
> > + * struct venc_vpu_ipi_msg_deinit - VPU ack AP deinit cmd structure
> > + * @msg_id:   message id (VPU_IPIMSG_XXX_ENC_DEINIT_DONE)
> > + * @status:   cmd status (venc_ipi_msg_status)
> > + * @venc_inst:	AP encoder instance (struct venc_vp8_handle/venc_h264_handle *)
> > + */
> > +struct venc_vpu_ipi_msg_deinit {
> > +	uint32_t msg_id;
> > +	uint32_t status;
> > +	uint64_t venc_inst;
> > +};
> > +
> > +#endif /* _VENC_IPI_MSG_H_ */
> > diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
> > old mode 100644
> > new mode 100755
> > index ee9d530..3ac35c4
> > --- a/include/uapi/linux/v4l2-controls.h
> > +++ b/include/uapi/linux/v4l2-controls.h
> > @@ -646,6 +646,10 @@ enum v4l2_mpeg_mfc51_video_force_frame_type {
> >  #define V4L2_CID_MPEG_MFC51_VIDEO_H264_NUM_REF_PIC_FOR_P		(V4L2_CID_MPEG_MFC51_BASE+54)
> >  
> >  
> > +#define V4L2_CID_MPEG_MTK_BASE					(V4L2_CTRL_CLASS_MPEG | 0x5500)
> > +#define V4L2_CID_MPEG_MTK_VIDEO_FRAME_SKIP_ENABLE		(V4L2_CID_MPEG_MTK_BASE+0)
> > +#define V4L2_CID_MPEG_MTK_VIDEO_FORCE_FRAME_TYPE_I_FRAME	(V4L2_CID_MPEG_MTK_BASE+1)
> 
> Please note that these controls should be documented in DocBook.
> 
Sorry, I do not clean these part.
We plain to remove V4L2_CID_MPEG_MTK_VIDEO_FRAME_SKIP_ENABL and use
V4L2_CID_MPEG_VIDEO_FORCE_I_FRAME in wucheng's patch
https://patchwork.linuxtv.org/patch/32670/.


best regards,
Tiffany

> Regards,
> 
> 	Hans
> 
> > +
> >  /*  Camera class control IDs */
> >  
> >  #define V4L2_CID_CAMERA_CLASS_BASE 	(V4L2_CTRL_CLASS_CAMERA | 0x900)
> > 
> 

^ permalink raw reply	[flat|nested] 102+ messages in thread

* Re: [PATCH v4 5/8] [Media] vcodec: mediatek: Add Mediatek V4L2 Video Encoder Driver
@ 2016-02-16  6:37               ` tiffany lin
  0 siblings, 0 replies; 102+ messages in thread
From: tiffany lin @ 2016-02-16  6:37 UTC (permalink / raw)
  To: Hans Verkuil
  Cc: Hans Verkuil, daniel.thompson, Rob Herring,
	Mauro Carvalho Chehab, Matthias Brugger, Daniel Kurtz,
	Pawel Osciak, Eddie Huang, Yingjoe Chen, devicetree,
	linux-kernel, linux-arm-kernel, linux-media, linux-mediatek,
	PoChun.Lin, Andrew-CT Chen

Hi Hans,

Thanks for your time.
On Mon, 2016-02-15 at 12:21 +0100, Hans Verkuil wrote:
> On 02/04/2016 12:35 PM, Tiffany Lin wrote:
> > From: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
> > 
> > Add v4l2 layer encoder driver for MT8173
> > 
> > Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
> 
> If Andrew is the author, shouldn't there be a Signed-off-by from him as well?
> 
> And in copyright notices (might want to update the year to 2016 BTW) PC Chen
> is mentioned among others. It might be useful to update the Signed-off-by lines.
> 

Author is PC Chen and Tiffany Lin, Andrew-CT Chen is author of mtk-vpu
module.
We will fix copyright and this in next version.

> > ---
> >  drivers/media/platform/Kconfig                     |   11 +
> >  drivers/media/platform/Makefile                    |    2 +
> >  drivers/media/platform/mtk-vcodec/Makefile         |    8 +
> >  drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h |  388 ++++++
> >  drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c | 1380 ++++++++++++++++++++
> >  drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h |   46 +
> >  .../media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c |  476 +++++++
> >  .../media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c  |  132 ++
> >  .../media/platform/mtk-vcodec/mtk_vcodec_intr.c    |  102 ++
> >  .../media/platform/mtk-vcodec/mtk_vcodec_intr.h    |   29 +
> >  drivers/media/platform/mtk-vcodec/mtk_vcodec_pm.h  |   26 +
> >  .../media/platform/mtk-vcodec/mtk_vcodec_util.c    |  106 ++
> >  .../media/platform/mtk-vcodec/mtk_vcodec_util.h    |   85 ++
> >  drivers/media/platform/mtk-vcodec/venc_drv_base.h  |   62 +
> >  drivers/media/platform/mtk-vcodec/venc_drv_if.c    |  100 ++
> >  drivers/media/platform/mtk-vcodec/venc_drv_if.h    |  175 +++
> >  drivers/media/platform/mtk-vcodec/venc_ipi_msg.h   |  212 +++
> >  include/uapi/linux/v4l2-controls.h                 |    4 +
> >  18 files changed, 3344 insertions(+)
> >  create mode 100644 drivers/media/platform/mtk-vcodec/Makefile
> >  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
> >  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
> >  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h
> >  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
> >  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
> >  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.c
> >  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.h
> >  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_pm.h
> >  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c
> >  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
> >  create mode 100644 drivers/media/platform/mtk-vcodec/venc_drv_base.h
> >  create mode 100644 drivers/media/platform/mtk-vcodec/venc_drv_if.c
> >  create mode 100644 drivers/media/platform/mtk-vcodec/venc_drv_if.h
> >  create mode 100644 drivers/media/platform/mtk-vcodec/venc_ipi_msg.h
> >  mode change 100644 => 100755 include/uapi/linux/v4l2-controls.h
> > 
> > diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
> > index ba812d6..3e831c5 100644
> > --- a/drivers/media/platform/Kconfig
> > +++ b/drivers/media/platform/Kconfig
> > @@ -157,6 +157,17 @@ config VIDEO_MEDIATEK_VPU
> >  	    codec embedded in new Mediatek's SOCs. It is able
> >  	    to handle video decoding/encoding in a range of formats.
> >  
> > +config VIDEO_MEDIATEK_VCODEC
> > +        tristate "Mediatek Video Codec driver"
> > +        depends on VIDEO_DEV && VIDEO_V4L2
> > +        depends on ARCH_MEDIATEK || COMPILE_TEST
> > +        select VIDEOBUF2_DMA_CONTIG
> > +        select V4L2_MEM2MEM_DEV
> > +        select MEDIATEK_VPU
> > +        default n
> > +        ---help---
> > +            Mediatek video codec driver for V4L2
> > +
> >  config VIDEO_MEM2MEM_DEINTERLACE
> >  	tristate "Deinterlace support"
> >  	depends on VIDEO_DEV && VIDEO_V4L2 && DMA_ENGINE
> > diff --git a/drivers/media/platform/Makefile b/drivers/media/platform/Makefile
> > index e5b19c6..510e06b 100644
> > --- a/drivers/media/platform/Makefile
> > +++ b/drivers/media/platform/Makefile
> > @@ -57,3 +57,5 @@ obj-$(CONFIG_VIDEO_XILINX)		+= xilinx/
> >  ccflags-y += -I$(srctree)/drivers/media/i2c
> >  
> >  obj-$(CONFIG_VIDEO_MEDIATEK_VPU)	+= mtk-vpu/
> > +
> > +obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC)	+= mtk-vcodec/
> > diff --git a/drivers/media/platform/mtk-vcodec/Makefile b/drivers/media/platform/mtk-vcodec/Makefile
> > new file mode 100644
> > index 0000000..ce38689
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/Makefile
> > @@ -0,0 +1,8 @@
> > +obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += mtk_vcodec_intr.o \
> > +				       mtk_vcodec_util.o \
> > +				       mtk_vcodec_enc_drv.o \
> > +				       mtk_vcodec_enc.o \
> > +				       mtk_vcodec_enc_pm.o \
> > +				       venc_drv_if.o
> > +
> > +ccflags-y += -I$(srctree)/drivers/media/platform/mtk-vpu
> > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
> > new file mode 100644
> > index 0000000..9da2818
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
> > @@ -0,0 +1,388 @@
> > +/*
> > +* Copyright (c) 2015 MediaTek Inc.
> > +* Author: PC Chen <pc.chen@mediatek.com>
> > +*         Tiffany Lin <tiffany.lin@mediatek.com>
> > +*
> > +* This program is free software; you can redistribute it and/or modify
> > +* it under the terms of the GNU General Public License version 2 as
> > +* published by the Free Software Foundation.
> > +*
> > +* This program is distributed in the hope that it will be useful,
> > +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> > +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > +* GNU General Public License for more details.
> > +*/
> > +
> > +#ifndef _MTK_VCODEC_DRV_H_
> > +#define _MTK_VCODEC_DRV_H_
> > +
> > +#include <linux/platform_device.h>
> > +#include <linux/videodev2.h>
> > +#include <media/v4l2-ctrls.h>
> > +#include <media/v4l2-device.h>
> > +#include <media/v4l2-ioctl.h>
> > +#include <media/videobuf2-core.h>
> > +
> > +#include "mtk_vcodec_util.h"
> > +
> > +#define MTK_VCODEC_MAX_INSTANCES		10
> > +#define MTK_VCODEC_MAX_ENCODER_INSTANCES	3
> > +
> > +#define MTK_VCODEC_MAX_FRAME_SIZE	0x800000
> > +#define MTK_VIDEO_MAX_FRAME		32
> > +#define MTK_MAX_CTRLS			20
> > +
> > +#define MTK_VCODEC_DRV_NAME		"mtk_vcodec_drv"
> > +#define MTK_VCODEC_ENC_NAME		"mtk-vcodec-enc"
> > +#define MTK_PLATFORM_STR		"platform:mt8173"
> > +
> > +#define MTK_VENC_IRQ_STATUS_SPS          0x1
> > +#define MTK_VENC_IRQ_STATUS_PPS          0x2
> > +#define MTK_VENC_IRQ_STATUS_FRM          0x4
> > +#define MTK_VENC_IRQ_STATUS_DRAM         0x8
> > +#define MTK_VENC_IRQ_STATUS_PAUSE        0x10
> > +#define MTK_VENC_IRQ_STATUS_SWITCH       0x20
> > +
> > +#define MTK_VENC_IRQ_STATUS_OFFSET       0x05C
> > +#define MTK_VENC_IRQ_ACK_OFFSET          0x060
> > +
> > +#define MTK_VCODEC_MAX_PLANES		3
> > +
> > +#define VDEC_HW_ACTIVE	0x10
> > +#define VDEC_IRQ_CFG    0x11
> > +#define VDEC_IRQ_CLR    0x10
> > +
> > +#define VDEC_IRQ_CFG_REG	0xa4
> > +#define WAIT_INTR_TIMEOUT	1000
> > +
> > +/**
> > + * enum mtk_hw_reg_idx - MTK hw register base index
> > + */
> > +enum mtk_hw_reg_idx {
> > +	VDEC_SYS,
> > +	VDEC_MISC,
> > +	VDEC_LD,
> > +	VDEC_TOP,
> > +	VDEC_CM,
> > +	VDEC_AD,
> > +	VDEC_AV,
> > +	VDEC_PP,
> > +	VDEC_HWD,
> > +	VDEC_HWQ,
> > +	VDEC_HWB,
> > +	VDEC_HWG,
> > +	NUM_MAX_VDEC_REG_BASE,
> > +	VENC_SYS = NUM_MAX_VDEC_REG_BASE,
> > +	VENC_LT_SYS,
> > +	NUM_MAX_VCODEC_REG_BASE
> > +};
> > +
> > +/**
> > + * enum mtk_instance_type - The type of an MTK Vcodec instance.
> > + */
> > +enum mtk_instance_type {
> > +	MTK_INST_DECODER		= 0,
> > +	MTK_INST_ENCODER		= 1,
> > +};
> > +
> > +/**
> > + * enum mtk_instance_state - The state of an MTK Vcodec instance.
> > + * @MTK_STATE_FREE - default state when instance is created
> > + * @MTK_STATE_INIT - vcodec instance is initialized
> > + * @MTK_STATE_HEADER - vdec had sps/pps header parsed or venc
> > + *			had sps/pps header encoded
> > + * @MTK_STATE_FLUSH - vdec is flushing. Only used by decoder
> > + * @MTK_STATE_RES_CHANGE - vdec detect resolution change.
> > + * 			Only used by decoder
> > + * @MTK_STATE_ABORT - vcodec should be aborted
> > + */
> > +enum mtk_instance_state {
> > +	MTK_STATE_FREE = 0,
> > +	MTK_STATE_INIT = 1,
> > +	MTK_STATE_HEADER = 2,
> > +	MTK_STATE_FLUSH = 3,
> > +	MTK_STATE_RES_CHANGE = 4,
> > +	MTK_STATE_ABORT = 5,
> > +};
> > +
> > +/**
> > + * struct mtk_param_change - General encoding parameters type
> > + */
> > +enum mtk_encode_param {
> > +	MTK_ENCODE_PARAM_NONE = 0,
> > +	MTK_ENCODE_PARAM_BITRATE = (1 << 0),
> > +	MTK_ENCODE_PARAM_FRAMERATE = (1 << 1),
> > +	MTK_ENCODE_PARAM_INTRA_PERIOD = (1 << 2),
> > +	MTK_ENCODE_PARAM_FRAME_TYPE = (1 << 3),
> > +};
> > +
> > +/**
> > + * enum mtk_fmt_type - Type of the pixelformat
> > + * @MTK_FMT_FRAME - mtk vcodec raw frame
> > + */
> > +enum mtk_fmt_type {
> > +	MTK_FMT_DEC		= 0,
> > +	MTK_FMT_ENC		= 1,
> > +	MTK_FMT_FRAME		= 2,
> > +};
> > +
> > +/**
> > + * struct mtk_video_fmt - Structure used to store information about pixelformats
> > + */
> > +struct mtk_video_fmt {
> > +	u32 fourcc;
> > +	enum mtk_fmt_type type;
> > +	u32 num_planes;
> > +};
> > +
> > +/**
> > + * struct mtk_codec_framesizes - Structure used to store information about framesizes
> > + */
> > +struct mtk_codec_framesizes {
> > +	u32 fourcc;
> > +	struct	v4l2_frmsize_stepwise	stepwise;
> > +};
> > +
> > +/**
> > + * struct mtk_q_type - Type of queue
> > + */
> > +enum mtk_q_type {
> > +	MTK_Q_DATA_SRC		= 0,
> > +	MTK_Q_DATA_DST		= 1,
> > +};
> > +
> > +/**
> > + * struct mtk_q_data - Structure used to store information about queue
> > + * @colorspace	reserved for encoder
> > + * @field		reserved for encoder
> 
> struct is only partially documented here.
We will fix this in next version.

> 
> > + */
> > +struct mtk_q_data {
> > +	unsigned int		width;
> > +	unsigned int		height;
> > +	enum v4l2_field		field;
> > +	enum v4l2_colorspace	colorspace;
> > +	unsigned int		bytesperline[MTK_VCODEC_MAX_PLANES];
> > +	unsigned int		sizeimage[MTK_VCODEC_MAX_PLANES];
> > +	struct mtk_video_fmt	*fmt;
> > +};
> > +
> > +/**
> > + * struct mtk_enc_params - General encoding parameters
> > + * @bitrate - target bitrate
> > + * @num_b_frame - number of b frames between p-frame
> > + * @rc_frame - frame based rate control
> > + * @rc_mb - macroblock based rate control
> > + * @seq_hdr_mode - H.264 sequence header is encoded separately or joined with the first frame
> > + * @gop_size - group of picture size, it's used as the intra frame period
> > + * @framerate_num - frame rate numerator
> > + * @framerate_denom - frame rate denominator
> > + * @h264_max_qp - Max value for H.264 quantization parameter
> > + * @h264_profile - V4L2 defined H.264 profile
> > + * @h264_level - V4L2 defined H.264 level
> > + * @force_intra - force/insert intra frame
> > + * @skip_frame - encode in skip frame mode that use minimum number of bits
> 
> skip_frame isn't in the struct anymore.
We will remove this in next version.

> 
> > + */
> > +struct mtk_enc_params {
> > +	unsigned int	bitrate;
> > +	unsigned int	num_b_frame;
> > +	unsigned int	rc_frame;
> > +	unsigned int	rc_mb;
> > +	unsigned int	seq_hdr_mode;
> > +	unsigned int	gop_size;
> > +	unsigned int	framerate_num;
> > +	unsigned int	framerate_denom;
> > +	unsigned int	h264_max_qp;
> > +	unsigned int	h264_profile;
> > +	unsigned int	h264_level;
> > +	unsigned int	force_intra;
> > +};
> > +
> > +/**
> > + * struct mtk_vcodec_pm - Power management data structure
> > + */
> > +struct mtk_vcodec_pm {
> > +	struct clk	*vcodecpll;
> > +	struct clk	*univpll_d2;
> > +	struct clk	*clk_cci400_sel;
> > +	struct clk	*vdecpll;
> > +	struct clk	*vdec_sel;
> > +	struct clk	*vencpll_d2;
> > +	struct clk	*venc_sel;
> > +	struct clk	*univpll1_d2;
> > +	struct clk	*venc_lt_sel;
> > +	struct device	*larbvdec;
> > +	struct device	*larbvenc;
> > +	struct device	*larbvenclt;
> > +	struct device	*dev;
> > +	struct mtk_vcodec_dev *mtkdev;
> > +};
> > +
> > +
> > +/**
> > + * struct mtk_vcodec_ctx - Context (instance) private data.
> > + *
> > + * @type:		type of the instance - decoder or encoder
> > + * @dev:		pointer to the mtk_vcodec_dev of the device
> > + * @fh:			struct v4l2_fh
> > + * @m2m_ctx:		pointer to the v4l2_m2m_ctx of the context
> > + * @q_data:		store information of input and output queue
> > + *			of the context
> > + * @idx:		index of the context that this structure describes
> > + * @state:		state of the context
> > + * @param_change:	encode parameters
> > + * @enc_params:		encoding parameters
> > + * @colorspace:
> > + * @enc_if:		hoooked encoder driver interface
> > + * @drv_handle:		driver handle for specific decode/encode instance
> > + *
> > + * @picinfo:		store width/height of image and buffer and planes' size for decoder
> > + *			and encoder
> > + * @dpb_count:		count of the DPB buffers required by MTK Vcodec hw
> > + *
> > + * @int_cond:		variable used by the waitqueue
> > + * @int_type:		type of the last interrupt
> > + * @queue:		waitqueue that can be used to wait for this context to
> > + *			finish
> > + * @irq_status:		irq status
> > + *
> > + * @ctrl_hdl:		handler for v4l2 framework
> > + * @ctrls:		array of controls, used when adding controls to the
> > + *			v4l2 control framework
> > + *
> > + * @encode_work:	worker for the encoding
> > + */
> 
> These comments are out of date with the actual struct!
We will fix these in next version.

> 
> > +struct mtk_vcodec_ctx {
> > +	enum mtk_instance_type type;
> > +	struct mtk_vcodec_dev *dev;
> > +	struct v4l2_fh fh;
> > +	struct v4l2_m2m_ctx *m2m_ctx;
> > +	struct mtk_q_data q_data[2];
> > +	int idx;
> > +	enum mtk_instance_state state;
> > +	enum mtk_encode_param param_change;
> > +	struct mtk_enc_params enc_params;
> > +
> > +	struct venc_common_if *enc_if;
> > +	unsigned long drv_handle;
> > +
> > +
> > +	int int_cond;
> > +	int int_type;
> > +	wait_queue_head_t queue;
> > +	unsigned int irq_status;
> > +
> > +	struct v4l2_ctrl_handler ctrl_hdl;
> > +	struct v4l2_ctrl *ctrls[MTK_MAX_CTRLS];
> > +
> > +	struct work_struct encode_work;
> > +
> > +};
> > +
> > +/**
> > + * struct mtk_vcodec_dev - driver data
> > + * @v4l2_dev:		V4L2 device to register video devices for.
> > + * @vfd_enc:		Video device for encoder.
> > + *
> > + * @m2m_dev_enc:	m2m device for encoder.
> > + * @plat_dev:		platform device
> > + * @alloc_ctx:		VB2 allocator context
> > + *			(for allocations without kernel mapping).
> > + * @ctx:		array of driver contexts
> > + *
> > + * @curr_ctx:		The context that is waiting for codec hardware
> > + *
> > + * @reg_base:		Mapped address of MTK Vcodec registers.
> > + *
> > + * @instance_mask:	used to mark which contexts are opened
> > + * @num_instances:	counter of active MTK Vcodec instances
> > + *
> > + * @encode_workqueue:	encode work queue
> > + *
> > + * @int_cond:		used to identify interrupt condition happen
> > + * @int_type:		used to identify what kind of interrupt condition happen
> > + * @dev_mutex:		video_device lock
> > + * @queue:		waitqueue for waiting for completion of device commands
> > + *
> > + * @enc_irq:		h264 encoder irq resource
> > + * @enc_lt_irq:		vp8 encoder irq resource
> > + *
> > + * @enc_mutex:		encoder hardware lock.
> > + *
> > + * @pm:			power management control
> > + * @dec_capability:	used to identify decode capability, ex: 4k
> > + * @enc_capability:     used to identify encode capability
> > + */
> > +struct mtk_vcodec_dev {
> > +	struct v4l2_device	v4l2_dev;
> > +	struct video_device	*vfd_enc;
> > +
> > +	struct v4l2_m2m_dev	*m2m_dev_enc;
> > +	struct platform_device	*plat_dev;
> > +	struct platform_device 	*vpu_plat_dev;
> > +	struct vb2_alloc_ctx	*alloc_ctx;
> > +	struct mtk_vcodec_ctx	*ctx[MTK_VCODEC_MAX_INSTANCES];
> > +	int curr_ctx;
> > +	void __iomem		*reg_base[NUM_MAX_VCODEC_REG_BASE];
> > +
> > +	unsigned long	instance_mask[BITS_TO_LONGS(MTK_VCODEC_MAX_INSTANCES)];
> > +	int			num_instances;
> > +
> > +	struct workqueue_struct *encode_workqueue;
> > +
> > +	int			int_cond;
> > +	int			int_type;
> > +	struct mutex		dev_mutex;
> > +	wait_queue_head_t	queue;
> > +
> > +	int			enc_irq;
> > +	int			enc_lt_irq;
> > +
> > +	struct mutex		enc_mutex;
> > +
> > +	struct mtk_vcodec_pm	pm;
> > +	unsigned int		dec_capability;
> > +	unsigned int		enc_capability;
> > +};
> > +
> > +/**
> > + * struct mtk_vcodec_ctrl - information about controls to be registered.
> > + * @id:			Control ID.
> > + * @type:		Type of the control.
> > + * @name:		Human readable name of the control.
> > + * @minimum:		Minimum value of the control.
> > + * @maximum:		Maximum value of the control.
> > + * @step:		Control value increase step.
> > + * @menu_skip_mask:	Mask of invalid menu positions.
> > + * @default_value:	Initial value of the control.
> > + * @is_volatile:	Control is volatile.
> > + *
> > + * See also struct v4l2_ctrl_config.
> > + */
> > +struct mtk_vcodec_ctrl {
> > +	u32			id;
> > +	enum v4l2_ctrl_type	type;
> > +	u8			name[32];
> > +	s32			minimum;
> > +	s32			maximum;
> > +	s32			step;
> > +	u32			menu_skip_mask;
> > +	s32			default_value;
> > +	u8			is_volatile;
> > +};
> > +
> > +static inline struct mtk_vcodec_ctx *fh_to_ctx(struct v4l2_fh *fh)
> > +{
> > +	return container_of(fh, struct mtk_vcodec_ctx, fh);
> > +}
> > +
> > +static inline struct mtk_vcodec_ctx *ctrl_to_ctx(struct v4l2_ctrl *ctrl)
> > +{
> > +	return container_of(ctrl->handler, struct mtk_vcodec_ctx, ctrl_hdl);
> > +}
> > +
> > +extern const struct v4l2_ioctl_ops mtk_vdec_ioctl_ops;
> > +extern const struct v4l2_m2m_ops mtk_vdec_m2m_ops;
> > +extern const struct v4l2_ioctl_ops mtk_venc_ioctl_ops;
> > +extern const struct v4l2_m2m_ops mtk_venc_m2m_ops;
> > +
> > +#endif /* _MTK_VCODEC_DRV_H_ */
> > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
> > new file mode 100644
> > index 0000000..ee602fe
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
> > @@ -0,0 +1,1380 @@
> > +/*
> > +* Copyright (c) 2015 MediaTek Inc.
> > +* Author: PC Chen <pc.chen@mediatek.com>
> > +*         Tiffany Lin <tiffany.lin@mediatek.com>
> > +*
> > +* This program is free software; you can redistribute it and/or modify
> > +* it under the terms of the GNU General Public License version 2 as
> > +* published by the Free Software Foundation.
> > +*
> > +* This program is distributed in the hope that it will be useful,
> > +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> > +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > +* GNU General Public License for more details.
> > +*/
> > +
> > +#include <media/v4l2-event.h>
> > +#include <media/v4l2-mem2mem.h>
> > +#include <media/videobuf2-dma-contig.h>
> > +
> > +#include "mtk_vcodec_drv.h"
> > +#include "mtk_vcodec_enc.h"
> > +#include "mtk_vcodec_intr.h"
> > +#include "mtk_vcodec_util.h"
> > +#include "venc_drv_if.h"
> > +
> > +#define MTK_VENC_MIN_W	32
> > +#define MTK_VENC_MIN_H	32
> > +#define MTK_VENC_MAX_W	1920
> > +#define MTK_VENC_MAX_H	1080
> > +#define DFT_CFG_WIDTH	MTK_VENC_MIN_W
> > +#define DFT_CFG_HEIGHT	MTK_VENC_MIN_H
> > +
> > +static void mtk_venc_worker(struct work_struct *work);
> > +
> > +static struct mtk_video_fmt mtk_video_formats[] = {
> > +	{
> > +		.fourcc		= V4L2_PIX_FMT_YUV420,
> > +		.type		= MTK_FMT_FRAME,
> > +		.num_planes	= 3,
> > +	},
> > +	{
> > +		.fourcc		= V4L2_PIX_FMT_YVU420,
> > +		.type		= MTK_FMT_FRAME,
> > +		.num_planes	= 3,
> > +	},
> > +	{
> > +		.fourcc		= V4L2_PIX_FMT_NV12,
> > +		.type		= MTK_FMT_FRAME,
> > +		.num_planes	= 2,
> > +	},
> > +	{
> > +		.fourcc		= V4L2_PIX_FMT_NV21,
> > +		.type		= MTK_FMT_FRAME,
> > +		.num_planes	= 2,
> > +	},
> > +	{
> > +		.fourcc		= V4L2_PIX_FMT_YUV420M,
> > +		.type		= MTK_FMT_FRAME,
> > +		.num_planes	= 3,
> > +	},
> > +	{
> > +		.fourcc		= V4L2_PIX_FMT_YVU420M,
> > +		.type		= MTK_FMT_FRAME,
> > +		.num_planes	= 3,
> > +	},
> > +	{
> > +		.fourcc 	= V4L2_PIX_FMT_NV12M,
> > +		.type		= MTK_FMT_FRAME,
> > +		.num_planes	= 2,
> > +	},
> > +	{
> > +		.fourcc		= V4L2_PIX_FMT_NV21M,
> > +		.type		= MTK_FMT_FRAME,
> > +		.num_planes	= 2,
> > +	},
> > +	{
> > +		.fourcc		= V4L2_PIX_FMT_H264,
> > +		.type		= MTK_FMT_ENC,
> > +		.num_planes	= 1,
> > +	},
> > +	{
> > +		.fourcc		= V4L2_PIX_FMT_VP8,
> > +		.type		= MTK_FMT_ENC,
> > +		.num_planes	= 1,
> > +	},
> > +};
> > +
> > +#define NUM_FORMATS ARRAY_SIZE(mtk_video_formats)
> > +
> > +static const struct mtk_codec_framesizes mtk_venc_framesizes[] = {
> > +	{
> > +		.fourcc	= V4L2_PIX_FMT_H264,
> > +		.stepwise = {  160, 1920, 16, 128, 1088, 16 },
> > +	},
> > +	{
> > +		.fourcc = V4L2_PIX_FMT_VP8,
> > +		.stepwise = {  160, 1920, 16, 128, 1088, 16 },
> > +	},
> > +};
> > +
> > +#define NUM_SUPPORTED_FRAMESIZE ARRAY_SIZE(mtk_venc_framesizes)
> > +
> > +static int vidioc_venc_s_ctrl(struct v4l2_ctrl *ctrl)
> > +{
> > +	struct mtk_vcodec_ctx *ctx = ctrl_to_ctx(ctrl);
> > +	struct mtk_enc_params *p = &ctx->enc_params;
> > +	int ret = 0;
> > +
> > +	switch (ctrl->id) {
> > +	case V4L2_CID_MPEG_VIDEO_BITRATE:
> > +		mtk_v4l2_debug(2, "V4L2_CID_MPEG_VIDEO_BITRATE val = %d",
> > +			ctrl->val);
> > +		p->bitrate = ctrl->val;
> > +		ctx->param_change |= MTK_ENCODE_PARAM_BITRATE;
> > +		break;
> > +	case V4L2_CID_MPEG_VIDEO_B_FRAMES:
> > +		mtk_v4l2_debug(2, "V4L2_CID_MPEG_VIDEO_B_FRAMES val = %d",
> > +			ctrl->val);
> > +		p->num_b_frame = ctrl->val;
> > +		break;
> > +	case V4L2_CID_MPEG_VIDEO_FRAME_RC_ENABLE:
> > +		mtk_v4l2_debug(2, "V4L2_CID_MPEG_VIDEO_FRAME_RC_ENABLE val = %d",
> > +			ctrl->val);
> > +		p->rc_frame = ctrl->val;
> > +		break;
> > +	case V4L2_CID_MPEG_VIDEO_H264_MAX_QP:
> > +		mtk_v4l2_debug(2, "V4L2_CID_MPEG_VIDEO_H264_MAX_QP val = %d",
> > +			ctrl->val);
> > +		p->h264_max_qp = ctrl->val;
> > +		break;
> > +	case V4L2_CID_MPEG_VIDEO_HEADER_MODE:
> > +		mtk_v4l2_debug(2, "V4L2_CID_MPEG_VIDEO_HEADER_MODE val = %d",
> > +			ctrl->val);
> > +		p->seq_hdr_mode = ctrl->val;
> > +		break;
> > +	case V4L2_CID_MPEG_VIDEO_MB_RC_ENABLE:
> > +		mtk_v4l2_debug(2, "V4L2_CID_MPEG_VIDEO_MB_RC_ENABLE val = %d",
> > +			ctrl->val);
> > +		p->rc_mb = ctrl->val;
> > +		break;
> > +	case V4L2_CID_MPEG_VIDEO_H264_PROFILE:
> > +		mtk_v4l2_debug(2, "V4L2_CID_MPEG_VIDEO_H264_PROFILE val = %d",
> > +			ctrl->val);
> > +		p->h264_profile = ctrl->val;
> > +		break;
> > +	case V4L2_CID_MPEG_VIDEO_H264_LEVEL:
> > +		mtk_v4l2_debug(2, "V4L2_CID_MPEG_VIDEO_H264_LEVEL val = %d",
> > +			ctrl->val);
> > +		p->h264_level = ctrl->val;
> > +		break;
> > +	case V4L2_CID_MPEG_VIDEO_H264_I_PERIOD:
> > +		mtk_v4l2_debug(2, "V4L2_CID_MPEG_VIDEO_H264_I_PERIOD val = %d",
> > +			ctrl->val);
> > +	case V4L2_CID_MPEG_VIDEO_GOP_SIZE:
> > +		mtk_v4l2_debug(2, "V4L2_CID_MPEG_VIDEO_GOP_SIZE val = %d",
> > +			ctrl->val);
> > +		p->gop_size = ctrl->val;
> > +		ctx->param_change |= MTK_ENCODE_PARAM_INTRA_PERIOD;
> > +		break;
> > +	case V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME:
> > +		mtk_v4l2_debug(2, "V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME");
> > +		p->force_intra = 1;
> > +		ctx->param_change |= MTK_ENCODE_PARAM_FRAME_TYPE;
> > +		break;
> > +	default:
> > +		ret = -EINVAL;
> > +		break;
> > +	}
> > +
> > +	return ret;
> > +}
> > +
> > +static const struct v4l2_ctrl_ops mtk_vcodec_enc_ctrl_ops = {
> > +	.s_ctrl = vidioc_venc_s_ctrl,
> > +};
> > +
> > +static int vidioc_enum_fmt(struct file *file, struct v4l2_fmtdesc *f,
> > +			   bool out)
> > +{
> > +	struct mtk_video_fmt *fmt;
> > +	int i, j = 0;
> > +
> > +	for (i = 0; i < NUM_FORMATS; ++i) {
> > +		if (out && mtk_video_formats[i].type != MTK_FMT_FRAME)
> > +			continue;
> > +		else if (!out && mtk_video_formats[i].type != MTK_FMT_ENC)
> 
> No 'else' needed here.
> 
We will remote it in next version.

> > +			continue;
> > +
> > +		if (j == f->index) {
> > +			fmt = &mtk_video_formats[i];
> > +			f->pixelformat = fmt->fourcc;
> > +			return 0;
> > +		}
> > +		++j;
> > +	}
> > +
> > +	return -EINVAL;
> > +}
> > +
> > +static int vidioc_enum_framesizes(struct file *file, void *fh,
> > +				  struct v4l2_frmsizeenum *fsize)
> > +{
> > +	int i = 0;
> > +
> > +	for (i = 0; i < NUM_SUPPORTED_FRAMESIZE; ++i) {
> > +		if (fsize->pixel_format != mtk_venc_framesizes[i].fourcc)
> > +			continue;
> > +
> > +		if (!fsize->index) {
> > +			fsize->type = V4L2_FRMSIZE_TYPE_STEPWISE;
> > +			fsize->stepwise = mtk_venc_framesizes[i].stepwise;
> > +			return 0;
> > +		}
> > +	}
> > +
> > +	return -EINVAL;
> > +}
> > +
> > +static int vidioc_enum_fmt_vid_cap_mplane(struct file *file, void *pirv,
> > +					  struct v4l2_fmtdesc *f)
> > +{
> > +	return vidioc_enum_fmt(file, f, false);
> > +}
> > +
> > +static int vidioc_enum_fmt_vid_out_mplane(struct file *file, void *prov,
> > +					  struct v4l2_fmtdesc *f)
> > +{
> > +	return vidioc_enum_fmt(file, f, true);
> > +}
> > +
> > +static int vidioc_venc_querycap(struct file *file, void *priv,
> > +				struct v4l2_capability *cap)
> > +{
> > +        strlcpy(cap->driver, MTK_VCODEC_ENC_NAME, strlen(MTK_VCODEC_ENC_NAME));
> 
> This should be:
> 
> 	strlcpy(cap->driver, MTK_VCODEC_ENC_NAME, sizeof(cap->driver));
> 
We will fix this.

> > +        cap->driver[strlen(MTK_VCODEC_ENC_NAME)]=0;
> 
> No need to terminate with 0, strlcpy does that already.
> 
> > +        strlcpy(cap->bus_info, MTK_PLATFORM_STR, strlen(MTK_PLATFORM_STR));
> > +        cap->bus_info[strlen(MTK_PLATFORM_STR)]=0;
> > +        strlcpy(cap->card, MTK_PLATFORM_STR, strlen(MTK_PLATFORM_STR));
> > +        cap->card[strlen(MTK_PLATFORM_STR)]=0;
> 
> Ditto for these two fields. Note that use use spaces instead of a tab here as
> well. Please fix. (checkpatch should have complained about that!)
> 
We will fix this.

> > +
> > +	/*
> > +	 * This is only a mem-to-mem video device. The capture and output
> > +	 * device capability flags are left only for backward compatibility
> > +	 * and are scheduled for removal.
> > +	 */
> 
> Comment is out of date.
> 
We will remove this.

> > +	cap->device_caps  = V4L2_CAP_VIDEO_M2M_MPLANE | V4L2_CAP_STREAMING;
> > +	cap->capabilities = cap->device_caps | V4L2_CAP_DEVICE_CAPS;
> > +
> > +	return 0;
> > +}
> > +static int vidioc_venc_s_parm(struct file *file, void *priv,
> > +			      struct v4l2_streamparm *a)
> > +{
> > +	struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
> > +
> > +	if (a->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
> > +		ctx->enc_params.framerate_num =
> > +			a->parm.output.timeperframe.denominator;
> > +		ctx->enc_params.framerate_denom =
> > +			a->parm.output.timeperframe.numerator;
> > +		ctx->param_change |= MTK_ENCODE_PARAM_FRAMERATE;
> > +	} else {
> > +		return -EINVAL;
> > +	}
> 
> I'd invert the test:
> 
> 	if (a->type != V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE)
> 		return -EINVAL;
> 
> and now you can just set ctx->enc_params.
> 
We will fix this in next version.


> > +	return 0;
> > +}
> 
> And if there is an s_parm, then there should be a g_parm as well!
> 
Now our driver does not support g_parm, our use cases do not use g_parm
too. 
Do we need to add g_parm at this moment? Or we could add it when we need
g_parm?

> > +
> > +static struct mtk_q_data *mtk_venc_get_q_data(struct mtk_vcodec_ctx *ctx,
> > +					      enum v4l2_buf_type type)
> > +{
> > +	if (V4L2_TYPE_IS_OUTPUT(type))
> > +		return &ctx->q_data[MTK_Q_DATA_SRC];
> > +
> > +	return &ctx->q_data[MTK_Q_DATA_DST];
> > +}
> > +
> > +static struct mtk_video_fmt *mtk_venc_find_format(struct v4l2_format *f)
> > +{
> > +	struct mtk_video_fmt *fmt;
> > +	unsigned int k;
> > +
> > +	for (k = 0; k < NUM_FORMATS; k++) {
> > +		fmt = &mtk_video_formats[k];
> > +		if (fmt->fourcc == f->fmt.pix.pixelformat)
> > +			return fmt;
> > +	}
> > +
> > +	return NULL;
> > +}
> > +
> > +static void mtk_vcodec_enc_calc_src_size(
> > +	unsigned int num_planes, unsigned int pic_width,
> > +	unsigned int pic_height, unsigned int sizeimage[],
> > +	unsigned int bytesperline[])
> > +{
> > +	unsigned int y_pitch_w_div16;
> > +	unsigned int c_pitch_w_div16;
> > +
> > +	y_pitch_w_div16 = ALIGN(pic_width, 16) >> 4;
> > +	c_pitch_w_div16 = ALIGN(pic_width, 16) >> 4;
> > +
> > +	if (num_planes == 2) {
> > +		sizeimage[0] =
> > +			(y_pitch_w_div16) * (((pic_height + 31) / 32) * 2) * 256 +
> > +			((y_pitch_w_div16 % 8 == 0) ? 0 : ((ALIGN(pic_width, 16) * 2) * 16));
> > +
> > +		sizeimage[1] =
> > +			(c_pitch_w_div16) * (((pic_height + 31) / 32) * 2) * 128 +
> > +			((c_pitch_w_div16 % 8 == 0) ? 0 : (ALIGN(pic_width, 16) * 16));
> > +
> > +		sizeimage[2] = 0;
> > +
> > +		bytesperline[0] = ALIGN(pic_width, 16);
> > +		bytesperline[1] = ALIGN(pic_width, 16);
> > +		bytesperline[2] = 0;
> > +
> > +	} else {
> > +		sizeimage[0] =
> > +			(y_pitch_w_div16) * (((pic_height + 31) / 32) * 2) * 256 +
> > +			((y_pitch_w_div16 % 8 == 0) ? 0 : ((ALIGN(pic_width, 16) * 2) * 16));
> > +
> > +		sizeimage[1] =
> > +			(c_pitch_w_div16) * (((pic_height + 31) / 32) * 2) * 64 +
> > +			((c_pitch_w_div16 % 8 == 0) ? 0 : ((ALIGN(pic_width, 16) / 2) * 16));
> > +
> > +		sizeimage[2] =
> > +			(c_pitch_w_div16) * (((pic_height + 31) / 32) * 2) * 64 +
> > +			((c_pitch_w_div16 % 8 == 0) ? 0 : ((ALIGN(pic_width, 16) / 2) * 16));
> > +
> > +		bytesperline[0] = ALIGN(pic_width, 16);
> > +		bytesperline[1] = ALIGN(pic_width, 16) / 2;
> > +		bytesperline[2] = ALIGN(pic_width, 16) / 2;
> > +	}
> > +}
> > +
> > +static int vidioc_try_fmt(struct v4l2_format *f, struct mtk_video_fmt *fmt)
> > +{
> > +	struct v4l2_pix_format_mplane *pix_fmt_mp = &f->fmt.pix_mp;
> > +
> > +	/* V4L2 specification suggests the driver corrects the format struct
> > +	  * if any of the dimensions is unsupported */
> > +        if (pix_fmt_mp->height < MTK_VENC_MIN_H)
> > +                pix_fmt_mp->height = MTK_VENC_MIN_H;
> > +        else if (pix_fmt_mp->height > MTK_VENC_MAX_H)
> > +                pix_fmt_mp->height = MTK_VENC_MAX_H;
> > +
> > +        if (pix_fmt_mp->width < MTK_VENC_MIN_W)
> > +                pix_fmt_mp->width = MTK_VENC_MIN_W;
> > +        else if (pix_fmt_mp->width > MTK_VENC_MAX_W)
> > +                pix_fmt_mp->width = MTK_VENC_MAX_W;
> 
> Use the clamp macro for this.
> 
We will fix this in next version.

> > +
> > +        pix_fmt_mp->field = V4L2_FIELD_NONE;
> > +
> > +	if (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
> > +		int size = pix_fmt_mp->height * pix_fmt_mp->width;
> 
> Add an empty line.
> 
We will fix this in next version.

> > +		if (fmt->num_planes != pix_fmt_mp->num_planes)
> > +			pix_fmt_mp->num_planes = fmt->num_planes;
> > +
> > +		if(pix_fmt_mp->plane_fmt[0].sizeimage != size)
> 
> Add space after if. Again, checkpatch should have complained about that.
> Please run checkpatch first before posting the next version.
> 
We will run checkpatch before posting the next version.

> > +			pix_fmt_mp->plane_fmt[0].sizeimage = size;
> > + 		pix_fmt_mp->plane_fmt[0].bytesperline = 0;
> > +		memset(&(pix_fmt_mp->plane_fmt[0].reserved[0]), 0x0,
> > +			sizeof(pix_fmt_mp->plane_fmt[0].reserved));
> > +	} else if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
> > +		int i;
> > +		unsigned int sizeimage[VIDEO_MAX_PLANES];
> > +		unsigned int bytesperline[VIDEO_MAX_PLANES];
> > +
> > +		v4l_bound_align_image(&pix_fmt_mp->width, 8, 1920, 1,
> > +				      &pix_fmt_mp->height, 4, 1080, 1, 0);
> > +
> > +		if (fmt->num_planes != pix_fmt_mp->num_planes)
> > +			pix_fmt_mp->num_planes = fmt->num_planes;
> > +
> > +		mtk_vcodec_enc_calc_src_size(pix_fmt_mp->num_planes,
> > +					pix_fmt_mp->width,
> > +					pix_fmt_mp->height,
> > +					sizeimage, bytesperline);
> > +
> > +		for (i=0; i < pix_fmt_mp->num_planes; i++) {
> 
> Spaces around '='.
> 
We will fix this.

> > +			pix_fmt_mp->plane_fmt[i].sizeimage = sizeimage[i];
> > +			pix_fmt_mp->plane_fmt[i].bytesperline = bytesperline[i];
> > +			memset(&(pix_fmt_mp->plane_fmt[i].reserved[0]), 0x0,
> > +				sizeof(pix_fmt_mp->plane_fmt[0].reserved));
> > +		}
> > +	} else {
> > +		return -EINVAL;
> > +	}
> > +
> > +	pix_fmt_mp->flags = 0;
> > +	pix_fmt_mp->ycbcr_enc = 0;
> > +	pix_fmt_mp->quantization = 0;
> > +	pix_fmt_mp->xfer_func = 0;
> > +	memset(&pix_fmt_mp->reserved[0], 0x0, sizeof(pix_fmt_mp->reserved));
> > +
> > +	return 0;
> > +}
> > +
> > +static void mtk_venc_set_param(struct mtk_vcodec_ctx *ctx, void *param)
> > +{
> > +	struct venc_enc_prm *p = (struct venc_enc_prm *)param;
> > +	struct mtk_q_data *q_data_src = &ctx->q_data[MTK_Q_DATA_SRC];
> > +	struct mtk_enc_params *enc_params = &ctx->enc_params;
> > +	unsigned int frame_rate;
> > +
> > +	frame_rate = enc_params->framerate_num / enc_params->framerate_denom;
> > +
> > +	switch (q_data_src->fmt->fourcc) {
> > +	case V4L2_PIX_FMT_YUV420:
> > +	case V4L2_PIX_FMT_YUV420M:
> > +		p->input_fourcc = VENC_YUV_FORMAT_420;
> > +		break;
> > +	case V4L2_PIX_FMT_YVU420:
> > +	case V4L2_PIX_FMT_YVU420M:
> > +		p->input_fourcc = VENC_YUV_FORMAT_YV12;
> > +		break;
> > +	case V4L2_PIX_FMT_NV12:
> > +	case V4L2_PIX_FMT_NV12M:
> > +		p->input_fourcc = VENC_YUV_FORMAT_NV12;
> > +		break;
> > +	case V4L2_PIX_FMT_NV21:
> > +	case V4L2_PIX_FMT_NV21M:
> > +		p->input_fourcc = VENC_YUV_FORMAT_NV21;
> > +		break;
> > +	}
> > +	p->h264_profile = enc_params->h264_profile;
> > +	p->h264_level = enc_params->h264_level;
> > +	p->width = q_data_src->width;
> > +	p->height = q_data_src->height;
> > +	p->buf_width = q_data_src->bytesperline[0];
> > +	p->buf_height = ((q_data_src->height + 0xf) & (~0xf));
> > +	p->frm_rate = frame_rate;
> > +	p->intra_period = enc_params->gop_size;
> > +	p->bitrate = enc_params->bitrate;
> > +
> > +	ctx->param_change = MTK_ENCODE_PARAM_NONE;
> > +
> > +	mtk_v4l2_debug(1, "fmt 0x%x, P/L %d/%d, w/h %d/%d, buf %d/%d, fps/bps %d/%d, gop %d",
> > +		       p->input_fourcc, p->h264_profile, p->h264_level,
> > +		       p->width, p->height, p->buf_width, p->buf_height,
> > +		       p->frm_rate, p->bitrate, p->intra_period);
> > +}
> > +
> > +static int vidioc_venc_s_fmt(struct file *file, void *priv,
> > +			     struct v4l2_format *f)
> 
> I am not convinced that combining capture and output in one function is the
> most readable. I think you are better off with separate functions. Too much is
> different between the two.
> 
Got it. We will separate it to vidioc_venc_s_fmt_cap and
vidioc_venc_s_fmt_out.

> > +{
> > +	struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
> > +	struct v4l2_pix_format_mplane *pix_fmt_mp = &f->fmt.pix_mp;
> > +	struct vb2_queue *vq;
> > +	struct mtk_q_data *q_data;
> > +	struct venc_enc_prm param;
> > +	int i, ret;
> > +	struct mtk_video_fmt *fmt;
> > +
> > +	vq = v4l2_m2m_get_vq(ctx->m2m_ctx, f->type);
> > +	if (!vq) {
> > +		mtk_v4l2_err("fail to get vq\n");
> > +		return -EINVAL;
> > +	}
> > +
> > +	if (vb2_is_busy(vq)) {
> > +		mtk_v4l2_err("queue busy\n");
> > +		return -EBUSY;
> > +	}
> > +
> > +	q_data = mtk_venc_get_q_data(ctx, f->type);
> > +	if (!q_data) {
> > +		mtk_v4l2_err("fail to get q data\n");
> > +		return -EINVAL;
> > +	}
> > +
> > +	fmt = mtk_venc_find_format(f);
> > +	if (!fmt) {
> > +		if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
> > +			f->fmt.pix.pixelformat = mtk_video_formats[0].fourcc;
> > +			fmt = mtk_venc_find_format(f);
> > +		} else {
> > +			f->fmt.pix.pixelformat = mtk_video_formats[8].fourcc;
> > +			fmt = mtk_venc_find_format(f);
> > +		}
> > +	}
> > +
> > +	q_data->fmt = fmt;
> > +	ret = vidioc_try_fmt(f, q_data->fmt);
> > +	if (ret)
> > +		return ret;
> > +
> > +	q_data->width		= f->fmt.pix_mp.width;
> > +	q_data->height		= f->fmt.pix_mp.height;
> > +	q_data->field		= f->fmt.pix_mp.field;
> > +
> > +	if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
> > +		q_data->colorspace = f->fmt.pix_mp.colorspace;
> > +		ctx->q_data[MTK_Q_DATA_SRC].bytesperline[0] =
> > +			ALIGN(q_data->width, 16);
> > +
> > +		if (q_data->fmt->num_planes == 2) {
> > +			ctx->q_data[MTK_Q_DATA_SRC].bytesperline[1] =
> > +				ALIGN(q_data->width, 16);
> > +			ctx->q_data[MTK_Q_DATA_SRC].bytesperline[2] = 0;
> > +		} else {
> > +			ctx->q_data[MTK_Q_DATA_SRC].bytesperline[1] =
> > +				ALIGN(q_data->width, 16) / 2;
> > +			ctx->q_data[MTK_Q_DATA_SRC].bytesperline[2] =
> > +				ALIGN(q_data->width, 16) / 2;
> > +		}
> > +
> > +		memset(&param, 0, sizeof(param));
> > +		mtk_venc_set_param(ctx, &param);
> > +		if (ctx->state == MTK_STATE_INIT) {
> > +			ret = venc_if_set_param(ctx,
> > +						VENC_SET_PARAM_ENC,
> > +						&param);
> > +			if (ret)
> > +				mtk_v4l2_err("venc_if_set_param failed=%d\n",
> > +						ret);
> > +
> > +			/* Get codec driver advice sizeimage from vpu */
> > +			for (i = 0; i < MTK_VCODEC_MAX_PLANES; i++) {
> > +				q_data->sizeimage[i] = param.sizeimage[i];
> > +				pix_fmt_mp->plane_fmt[i].sizeimage =
> > +					param.sizeimage[i];
> > +			}
> > +			q_data->bytesperline[0] =
> > +				pix_fmt_mp->plane_fmt[0].bytesperline;
> > +			q_data->bytesperline[1] =
> > +				pix_fmt_mp->plane_fmt[1].bytesperline;
> > +			q_data->bytesperline[2] =
> > +				pix_fmt_mp->plane_fmt[2].bytesperline;
> > +		} else {
> > +			for (i = 0; i < MTK_VCODEC_MAX_PLANES; i++) {
> > +				q_data->sizeimage[i] =
> > +					pix_fmt_mp->plane_fmt[i].sizeimage;
> > +				q_data->bytesperline[i] =
> > +					pix_fmt_mp->plane_fmt[i].bytesperline;
> > +			}
> > +		}
> > +	} else if (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE){
> > +		for (i = 0; i < f->fmt.pix_mp.num_planes; i++) {
> > +			struct v4l2_plane_pix_format	*plane_fmt;
> > +
> > +			plane_fmt = &f->fmt.pix_mp.plane_fmt[i];
> > +			q_data->bytesperline[i]	= plane_fmt->bytesperline;
> > +			q_data->sizeimage[i]	= plane_fmt->sizeimage;
> > +		}
> > +
> > +		if (ctx->state == MTK_STATE_FREE) {
> > +			ret = venc_if_create(ctx, q_data->fmt->fourcc);
> > +			if (ret) {
> > +				mtk_v4l2_err("venc_if_create failed=%d, codec type=%x\n",
> > +					ret, q_data->fmt->fourcc);
> > +				return 0;
> > +			}
> > +
> > +			ctx->state = MTK_STATE_INIT;
> > +		}
> > +	}
> > +
> > +	return 0;
> > +}
> > +
> > +static int vidioc_venc_g_fmt(struct file *file, void *priv,
> > +			     struct v4l2_format *f)
> > +{
> > +	struct v4l2_pix_format_mplane *pix = &f->fmt.pix_mp;
> > +	struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
> > +	struct vb2_queue *vq;
> > +	struct mtk_q_data *q_data;
> > +	int i;
> > +
> > +	vq = v4l2_m2m_get_vq(ctx->m2m_ctx, f->type);
> > +	if (!vq)
> > +		return -EINVAL;
> > +
> > +	q_data = mtk_venc_get_q_data(ctx, f->type);
> > +
> > +	pix->width = q_data->width;
> > +	pix->height = q_data->height;
> > +	pix->pixelformat = q_data->fmt->fourcc;
> > +	pix->field = q_data->field;
> > +	pix->colorspace = q_data->colorspace;
> > +	pix->num_planes = q_data->fmt->num_planes;
> > +	for (i = 0; i < pix->num_planes; i++) {
> > +		pix->plane_fmt[i].bytesperline = q_data->bytesperline[i];
> > +		pix->plane_fmt[i].sizeimage = q_data->sizeimage[i];
> > +		memset(&(pix->plane_fmt[i].reserved[0]), 0x0,
> > +			sizeof(pix->plane_fmt[i].reserved));
> > +	}
> > +	pix->flags = 0;
> > +	pix->ycbcr_enc = 0;
> > +	pix->quantization = 0;
> > +	pix->xfer_func = 0;
> > +	memset(&pix->reserved[0], 0x0, sizeof(pix->reserved));
> > +
> > +	return 0;
> > +}
> > +
> > +static int vidioc_try_fmt_vid_cap_mplane(struct file *file, void *priv,
> > +                                  struct v4l2_format *f)
> > +{
> > +        struct mtk_video_fmt *fmt;
> > +	struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
> > +
> > +        fmt = mtk_venc_find_format(f);
> > +        if (!fmt) {
> > +                f->fmt.pix.pixelformat = mtk_video_formats[8].fourcc;
> > +                fmt = mtk_venc_find_format(f);
> > +        }
> > +        if (fmt->type != MTK_FMT_ENC) {
> > +		mtk_v4l2_err("Fourcc format (0x%08x) invalid.\n",
> > +			     f->fmt.pix.pixelformat);
> > +		return -EINVAL;
> > +        }
> > +        f->fmt.pix_mp.colorspace = ctx->q_data[MTK_Q_DATA_SRC].colorspace;
> > +
> > +        return vidioc_try_fmt(f, fmt);
> > +}
> > +
> > +static int vidioc_try_fmt_vid_out_mplane(struct file *file, void *priv,
> > +                                  struct v4l2_format *f)
> > +{
> > +        struct mtk_video_fmt *fmt;
> > +
> > +        fmt = mtk_venc_find_format(f);
> > +        if (!fmt) {
> > +                f->fmt.pix.pixelformat = mtk_video_formats[0].fourcc;
> > +                fmt = mtk_venc_find_format(f);
> > +        }
> > +        if (!(fmt->type & MTK_FMT_FRAME)) {
> > +		mtk_v4l2_err("Fourcc format (0x%08x) invalid.\n",
> > +			     f->fmt.pix.pixelformat);
> > +		return -EINVAL;
> > +        }
> > +        if (!f->fmt.pix_mp.colorspace)
> > +                f->fmt.pix_mp.colorspace = V4L2_COLORSPACE_REC709;
> > +
> > +        return vidioc_try_fmt(f, fmt);
> > +}
> > +
> > +static int vidioc_venc_g_s_selection(struct file *file, void *priv,
> > +                                struct v4l2_selection *s)
> 
> Why support s_selection if you can only return the current width and height?
> And why support g_selection if you can't change the selection?
> 
> In other words, why implement this at all?
> 
> Unless I am missing something here, I would just drop this.
> 
Now our driver do not support these capabilities, but userspace app will
check whether g/s_crop are implemented when using encoder.
Because g/s_crop are deprecated as you mentioned in previous v2 review
comments. We change to use g_s_selection.
We will check if we could add this capability.

> > +{
> > +	struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
> > +	struct mtk_q_data *q_data;
> > +
> > +	if (V4L2_TYPE_IS_OUTPUT(s->type)) {
> > +		if (s->target !=  V4L2_SEL_TGT_COMPOSE)
> > +			return -EINVAL;
> > +	} else {
> > +		if (s->target != V4L2_SEL_TGT_CROP)
> > +			return -EINVAL;
> > +	}
> > +
> > +	if (s->r.left || s->r.top)
> > +		return -EINVAL;
> > +
> > +	q_data = mtk_venc_get_q_data(ctx, s->type);
> > +	if (!q_data)
> > +		return -EINVAL;
> > +
> > +	s->r.width = q_data->width;
> > +	s->r.height = q_data->height;
> > +
> > +	return 0;
> > +}
> > +
> > +
> > +static int vidioc_venc_qbuf(struct file *file, void *priv,
> > +			    struct v4l2_buffer *buf)
> > +{
> > +
> > +	struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
> > +
> > +	if (ctx->state == MTK_STATE_ABORT) {
> > +		mtk_v4l2_err("[%d] Call on QBUF after unrecoverable error\n", ctx->idx);
> > +		return -EIO;
> > +	}
> > +
> > +	return v4l2_m2m_qbuf(file, ctx->m2m_ctx, buf);
> > +}
> > +
> > +static int vidioc_venc_dqbuf(struct file *file, void *priv,
> > +			     struct v4l2_buffer *buf)
> > +{
> > +	struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
> > +	if (ctx->state == MTK_STATE_ABORT) {
> > +		mtk_v4l2_err("[%d] Call on QBUF after unrecoverable error\n", ctx->idx);
> > +		return -EIO;
> > +	}
> > +
> > +	return v4l2_m2m_dqbuf(file, ctx->m2m_ctx, buf);
> > +}
> > +
> > +
> > +const struct v4l2_ioctl_ops mtk_venc_ioctl_ops = {
> > +	.vidioc_streamon		= v4l2_m2m_ioctl_streamon,
> > +	.vidioc_streamoff		= v4l2_m2m_ioctl_streamoff,
> > +
> > +	.vidioc_reqbufs			= v4l2_m2m_ioctl_reqbufs,
> > +	.vidioc_querybuf		= v4l2_m2m_ioctl_querybuf,
> > +	.vidioc_qbuf			= vidioc_venc_qbuf,
> > +	.vidioc_dqbuf			= vidioc_venc_dqbuf,
> > +
> > +	.vidioc_querycap		= vidioc_venc_querycap,
> > +	.vidioc_enum_fmt_vid_cap_mplane = vidioc_enum_fmt_vid_cap_mplane,
> > +	.vidioc_enum_fmt_vid_out_mplane = vidioc_enum_fmt_vid_out_mplane,
> > +	.vidioc_enum_framesizes		= vidioc_enum_framesizes,
> > +
> > +	.vidioc_try_fmt_vid_cap_mplane	= vidioc_try_fmt_vid_cap_mplane,
> > +	.vidioc_try_fmt_vid_out_mplane	= vidioc_try_fmt_vid_out_mplane,
> > +	.vidioc_expbuf			= v4l2_m2m_ioctl_expbuf,
> 
> Please add vidioc_create_bufs and vidioc_prepare_buf as well.
> 

Currently we do not support these use cases, do we need to add
vidioc_create_bufs and vidioc_prepare_buf now?


> > +	.vidioc_subscribe_event		= v4l2_ctrl_subscribe_event,
> > +	.vidioc_unsubscribe_event	= v4l2_event_unsubscribe,
> > +
> > +	.vidioc_s_parm			= vidioc_venc_s_parm,
> > +
> > +	.vidioc_s_fmt_vid_cap_mplane	= vidioc_venc_s_fmt,
> > +	.vidioc_s_fmt_vid_out_mplane	= vidioc_venc_s_fmt,
> > +
> > +	.vidioc_g_fmt_vid_cap_mplane	= vidioc_venc_g_fmt,
> > +	.vidioc_g_fmt_vid_out_mplane	= vidioc_venc_g_fmt,
> > +
> > +	.vidioc_g_selection		= vidioc_venc_g_s_selection,
> > +	.vidioc_s_selection		= vidioc_venc_g_s_selection,
> > +};
> > +
> > +static int vb2ops_venc_queue_setup(struct vb2_queue *vq,
> > +				   const void *parg,
> > +				   unsigned int *nbuffers,
> > +				   unsigned int *nplanes,
> > +				   unsigned int sizes[], void *alloc_ctxs[])
> > +{
> > +	struct mtk_vcodec_ctx *ctx = vb2_get_drv_priv(vq);
> > +	struct mtk_q_data *q_data;
> > +
> > +	q_data = mtk_venc_get_q_data(ctx, vq->type);
> > +
> > +	if (*nbuffers < 1)
> > +		*nbuffers = 1;
> 
> *nbuffers can never be 0, so no need to check.
> 
We will remove this in next version.

> > +	if (*nbuffers > MTK_VIDEO_MAX_FRAME)
> 
> That should be (q->num_buffers + *nbuffers > MTK_VIDEO_MAX_FRAME)
> 
We will fix this in next version.

> > +		*nbuffers = MTK_VIDEO_MAX_FRAME;
> 
> and *nbuffers = MTK_VIDEO_MAX_FRAME - q->num_buffers;
> 
> And you will need a check if q->num_buffers == MTK_VIDEO_MAX_FRAME) too
> (otherwise *nbuffers could become <= 0).
> 
> In order to correctly handle create_bufs you will need to make a few changes
> here. Read the queue_setup description in include/media/videobuf2-core.h and
> look at queue_setup() in Documentation/video4linux/v4l2-pci-skeleton.c.
> 
Got it. We will check this.


> > +
> > +	*nplanes = q_data->fmt->num_planes;
> > +
> > +	if (vq->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
> > +		unsigned int i;
> > +
> > +		for (i = 0; i < *nplanes; i++) {
> > +			sizes[i] = q_data->sizeimage[i];
> > +			alloc_ctxs[i] = ctx->dev->alloc_ctx;
> > +		}
> > +	} else if (vq->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
> > +		sizes[0] = q_data->sizeimage[0];
> > +		alloc_ctxs[0] = ctx->dev->alloc_ctx;
> > +	} else {
> > +		return -EINVAL;
> > +	}
> > +
> > +	mtk_v4l2_debug(2, "[%d]get %d buffer(s) of size 0x%x each, vq->memory=%d",
> > +		       ctx->idx, *nbuffers, sizes[0], vq->memory);
> > +
> > +	return 0;
> > +}
> > +
> > +static int vb2ops_venc_buf_prepare(struct vb2_buffer *vb)
> > +{
> > +	struct mtk_vcodec_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue);
> > +	struct mtk_q_data *q_data;
> > +	int i;
> > +
> > +	q_data = mtk_venc_get_q_data(ctx, vb->vb2_queue->type);
> > +
> > +	for (i = 0; i < q_data->fmt->num_planes; i++) {
> > +		if (vb2_plane_size(vb, i) < q_data->sizeimage[i]) {
> > +			mtk_v4l2_debug(2, "data will not fit into plane %d (%lu < %d)",
> > +				       i, vb2_plane_size(vb, i),
> > +				       q_data->sizeimage[i]);
> > +			return -EINVAL;
> > +		}
> > +	}
> > +
> > +	return 0;
> > +}
> > +
> > +static void vb2ops_venc_buf_queue(struct vb2_buffer *vb)
> > +{
> > +	struct mtk_vcodec_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue);
> > +	struct vb2_v4l2_buffer *vb2_v4l2 =
> > +			container_of(vb, struct vb2_v4l2_buffer, vb2_buf);
> > +	struct mtk_video_enc_buf *mtk_buf =
> > +			container_of(vb2_v4l2, struct mtk_video_enc_buf, vb);
> > +
> > +	if ((vb->vb2_queue->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) &&
> > +		(ctx->param_change != MTK_ENCODE_PARAM_NONE)) {
> > +		mtk_v4l2_debug(1,
> > +				"[%d] Before id=%d encode parameter change %x",
> > +				ctx->idx, vb->index,
> > +				ctx->param_change);
> > +		mtk_buf->param_change = ctx->param_change;
> > +		if (mtk_buf->param_change & MTK_ENCODE_PARAM_BITRATE) {
> > +			mtk_buf->enc_params.bitrate = ctx->enc_params.bitrate;
> > +			mtk_v4l2_debug(1, "[%d] idx=%d change param br=%d",
> > +				ctx->idx,
> > +				mtk_buf->vb.vb2_buf.index,
> > +				mtk_buf->enc_params.bitrate);
> > +		}
> > +		if (ctx->param_change & MTK_ENCODE_PARAM_FRAMERATE) {
> > +			mtk_buf->enc_params.framerate_num =
> > +				ctx->enc_params.framerate_num;
> > +			mtk_buf->enc_params.framerate_denom =
> > +				ctx->enc_params.framerate_denom;
> > +			mtk_v4l2_debug(1, "[%d] idx=%d, change param fr=%d/%d",
> > +					ctx->idx,
> > +					mtk_buf->vb.vb2_buf.index,
> > +					mtk_buf->enc_params.framerate_num,
> > +					mtk_buf->enc_params.framerate_denom);
> > +		}
> > +		if (ctx->param_change & MTK_ENCODE_PARAM_INTRA_PERIOD) {
> > +			mtk_buf->enc_params.gop_size = ctx->enc_params.gop_size;
> > +			mtk_v4l2_debug(1, "[%d] idx=%d, change param intra period=%d",
> > +					ctx->idx,
> > +					mtk_buf->vb.vb2_buf.index,
> > +					mtk_buf->enc_params.gop_size);
> > +		}
> > +		if (ctx->param_change & MTK_ENCODE_PARAM_FRAME_TYPE) {
> > +			mtk_buf->enc_params.force_intra =
> > +				ctx->enc_params.force_intra;
> > +			mtk_v4l2_debug(1, "[%d] idx=%d, change param force I=%d",
> > +					ctx->idx,
> > +					mtk_buf->vb.vb2_buf.index,
> > +					mtk_buf->enc_params.force_intra);
> > +		}
> > +		ctx->param_change = MTK_ENCODE_PARAM_NONE;
> > +	}
> > +
> > +	v4l2_m2m_buf_queue(ctx->m2m_ctx, to_vb2_v4l2_buffer(vb));
> > +}
> > +
> > +static int vb2ops_venc_start_streaming(struct vb2_queue *q, unsigned int count)
> > +{
> > +	struct mtk_vcodec_ctx *ctx = vb2_get_drv_priv(q);
> > +	struct venc_enc_prm param;
> > +	int ret;
> > +	int i;
> > +
> > +	/* Once state turn into MTK_STATE_ABORT, we need stop_streaming to clear it */
> > +	if ((ctx->state == MTK_STATE_ABORT) || (ctx->state == MTK_STATE_FREE))
> > +		goto err_set_param;
> > +
> > +	if (!(vb2_start_streaming_called(&ctx->m2m_ctx->out_q_ctx.q) &
> > +	      vb2_start_streaming_called(&ctx->m2m_ctx->cap_q_ctx.q))) {
> > +		mtk_v4l2_debug(1, "[%d]-> out=%d cap=%d",
> > +		 ctx->idx,
> > +		 vb2_start_streaming_called(&ctx->m2m_ctx->out_q_ctx.q),
> > +		 vb2_start_streaming_called(&ctx->m2m_ctx->cap_q_ctx.q));
> > +		return 0;
> > +	}
> > +
> > +	mtk_venc_set_param(ctx, &param);
> > +	ret = venc_if_set_param(ctx,
> > +				VENC_SET_PARAM_ENC,
> > +				&param);
> > +	if (ret) {
> > +		mtk_v4l2_err("venc_if_set_param failed=%d\n", ret);
> > +		ctx->state = MTK_STATE_ABORT;
> > +		goto err_set_param;
> > +	}
> > +
> > +	if ((ctx->q_data[MTK_Q_DATA_DST].fmt->fourcc == V4L2_PIX_FMT_H264) &&
> > +	    (ctx->enc_params.seq_hdr_mode != V4L2_MPEG_VIDEO_HEADER_MODE_SEPARATE)) {
> > +		ret = venc_if_set_param(ctx,
> > +					VENC_SET_PARAM_PREPEND_HEADER,
> > +					0);
> > +		if (ret) {
> > +			mtk_v4l2_err("venc_if_set_param failed=%d\n", ret);
> > +			ctx->state = MTK_STATE_ABORT;
> > +			goto err_set_param;
> > +		}
> > +		ctx->state = MTK_STATE_HEADER;
> > +	}
> > +
> > +	return 0;
> > +
> > +err_set_param:
> > +	for (i = 0; i < q->num_buffers; ++i) {
> > +		if (q->bufs[i]->state == VB2_BUF_STATE_ACTIVE) {
> > +			mtk_v4l2_debug(0, "[%d] idx=%d, type=%d, %d -> VB2_BUF_STATE_QUEUED",
> > +					ctx->idx, i, q->type,
> > +					(int)q->bufs[i]->state );
> > +			v4l2_m2m_buf_done(to_vb2_v4l2_buffer(q->bufs[i]), VB2_BUF_STATE_QUEUED);
> > +		}
> > +	}
> > +
> > +	return -EINVAL;
> > +}
> > +
> > +static void vb2ops_venc_stop_streaming(struct vb2_queue *q)
> > +{
> > +	struct mtk_vcodec_ctx *ctx = vb2_get_drv_priv(q);
> > +	struct vb2_buffer *src_buf, *dst_buf;
> > +	int ret;
> > +
> > +	mtk_v4l2_debug(2, "[%d]-> type=%d", ctx->idx, q->type);
> > +
> > +	if (q->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
> > +		while ((dst_buf = v4l2_m2m_dst_buf_remove(ctx->m2m_ctx))) {
> > +			dst_buf->planes[0].bytesused = 0;
> > +			v4l2_m2m_buf_done(to_vb2_v4l2_buffer(dst_buf),
> > +						VB2_BUF_STATE_ERROR);
> > +		}
> > +	} else {
> > +		while ((src_buf = v4l2_m2m_src_buf_remove(ctx->m2m_ctx)))
> > +			v4l2_m2m_buf_done(to_vb2_v4l2_buffer(src_buf),
> > +						VB2_BUF_STATE_ERROR);
> > +	}
> > +
> > +	if ((q->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE &&
> > +	     vb2_is_streaming(&ctx->m2m_ctx->out_q_ctx.q)) ||
> > +	    (q->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE &&
> > +	     vb2_is_streaming(&ctx->m2m_ctx->cap_q_ctx.q))) {
> > +		mtk_v4l2_debug(1, "[%d]-> q type %d out=%d cap=%d",
> > +			       ctx->idx, q->type,
> > +			       vb2_is_streaming(&ctx->m2m_ctx->out_q_ctx.q),
> > +			       vb2_is_streaming(&ctx->m2m_ctx->cap_q_ctx.q));
> > +		return;
> > +	}
> > +
> > +	ret = venc_if_release(ctx);
> > +	if (ret)
> > +		mtk_v4l2_err("venc_if_release failed=%d\n", ret);
> > +
> > +	ctx->state = MTK_STATE_FREE;
> > +}
> > +
> > +static struct vb2_ops mtk_venc_vb2_ops = {
> > +	.queue_setup			= vb2ops_venc_queue_setup,
> > +	.buf_prepare			= vb2ops_venc_buf_prepare,
> > +	.buf_queue			= vb2ops_venc_buf_queue,
> > +	.wait_prepare			= vb2_ops_wait_prepare,
> > +	.wait_finish			= vb2_ops_wait_finish,
> > +	.start_streaming		= vb2ops_venc_start_streaming,
> > +	.stop_streaming			= vb2ops_venc_stop_streaming,
> > +};
> > +
> > +static int mtk_venc_encode_header(void *priv)
> > +{
> > +	struct mtk_vcodec_ctx *ctx = priv;
> > +	int ret;
> > +	struct vb2_buffer *dst_buf;
> > +	struct mtk_vcodec_mem bs_buf;
> > +	struct venc_done_result enc_result;
> > +
> > +	dst_buf = v4l2_m2m_dst_buf_remove(ctx->m2m_ctx);
> > +	if (!dst_buf) {
> > +		mtk_v4l2_debug(1, "No dst buffer");
> > +		return -EINVAL;
> > +	}
> > +
> > +	bs_buf.va = vb2_plane_vaddr(dst_buf, 0);
> > +	bs_buf.dma_addr = vb2_dma_contig_plane_dma_addr(dst_buf, 0);
> > +	bs_buf.size = (unsigned int)dst_buf->planes[0].length;
> > +
> > +	mtk_v4l2_debug(1,
> > +			"[%d] buf idx=%d va=0x%p dma_addr=0x%llx size=0x%lx",
> > +			ctx->idx,
> > +			dst_buf->index, bs_buf.va,
> > +			(u64)bs_buf.dma_addr,
> > +			bs_buf.size);
> > +
> > +	ret = venc_if_encode(ctx,
> > +			VENC_START_OPT_ENCODE_SEQUENCE_HEADER,
> > +			0, &bs_buf, &enc_result);
> > +
> > +	if (ret) {
> > +		dst_buf->planes[0].bytesused = 0;
> > +		ctx->state = MTK_STATE_ABORT;
> > +		v4l2_m2m_buf_done(to_vb2_v4l2_buffer(dst_buf), VB2_BUF_STATE_ERROR);
> > +		mtk_v4l2_err("venc_if_encode failed=%d", ret);
> > +		return -EINVAL;
> > +	}
> > +
> > +	ctx->state = MTK_STATE_HEADER;
> > +	dst_buf->planes[0].bytesused = enc_result.bs_size;
> > +
> > +#if defined(DEBUG)
> > +{
> > +	int i;
> > +	mtk_v4l2_debug(1, "[%d] venc_if_encode header len=%d",
> > +			ctx->idx,
> > +			enc_result.bs_size);
> > +	for (i = 0; i < enc_result.bs_size; i++) {
> > +		unsigned char *p = (unsigned char *)bs_buf.va;
> > +
> > +		mtk_v4l2_debug(1, "[%d] buf[%d]=0x%2x", ctx->idx, i, p[i]);
> > +	}
> > +}
> > +#endif
> > +	v4l2_m2m_buf_done(to_vb2_v4l2_buffer(dst_buf), VB2_BUF_STATE_DONE);
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_venc_param_change(struct mtk_vcodec_ctx *ctx, void *priv)
> > +{
> > +	struct vb2_buffer *vb = priv;
> > +	struct vb2_v4l2_buffer *vb2_v4l2 =
> > +			container_of(vb, struct vb2_v4l2_buffer, vb2_buf);
> > +	struct mtk_video_enc_buf *mtk_buf =
> > +			container_of(vb2_v4l2, struct mtk_video_enc_buf, vb);
> > +	int ret = 0;
> > +
> > +	if (mtk_buf->param_change == MTK_ENCODE_PARAM_NONE)
> > +		return 0;
> > +
> > +	mtk_v4l2_debug(1, "encode parameters change id=%d", vb->index);
> > +	if (mtk_buf->param_change & MTK_ENCODE_PARAM_BITRATE) {
> > +		struct venc_enc_prm enc_prm;
> > +
> > +		enc_prm.bitrate = mtk_buf->enc_params.bitrate;
> > +		mtk_v4l2_debug(1, "[%d] idx=%d, change param br=%d",
> > +				ctx->idx,
> > +				mtk_buf->vb.vb2_buf.index,
> > +				enc_prm.bitrate);
> > +		ret |= venc_if_set_param(ctx,
> > +					 VENC_SET_PARAM_ADJUST_BITRATE,
> > +					 &enc_prm);
> > +	}
> > +	if (mtk_buf->param_change & MTK_ENCODE_PARAM_FRAMERATE) {
> > +		struct venc_enc_prm enc_prm;
> > +
> > +		enc_prm.frm_rate = mtk_buf->enc_params.framerate_num /
> > +				   mtk_buf->enc_params.framerate_denom;
> > +		mtk_v4l2_debug(1, "[%d] idx=%d, change param fr=%d",
> > +			       ctx->idx,
> > +			       mtk_buf->vb.vb2_buf.index,
> > +			       enc_prm.frm_rate);
> > +		ret |= venc_if_set_param(ctx,
> > +					 VENC_SET_PARAM_ADJUST_FRAMERATE,
> > +					 &enc_prm);
> > +	}
> > +	if (mtk_buf->param_change & MTK_ENCODE_PARAM_INTRA_PERIOD) {
> > +		mtk_v4l2_debug(1, "change param intra period=%d",
> > +				 mtk_buf->enc_params.gop_size);
> > +		ret |= venc_if_set_param(ctx,
> > +					 VENC_SET_PARAM_I_FRAME_INTERVAL,
> > +					 &mtk_buf->enc_params.gop_size);
> > +	}
> > +	if (mtk_buf->param_change & MTK_ENCODE_PARAM_FRAME_TYPE) {
> > +		mtk_v4l2_debug(1, "[%d] idx=%d, change param force I=%d",
> > +				ctx->idx,
> > +				mtk_buf->vb.vb2_buf.index,
> > +				mtk_buf->enc_params.force_intra);
> > +		if (mtk_buf->enc_params.force_intra)
> > +			ret |= venc_if_set_param(ctx,
> > +						 VENC_SET_PARAM_FORCE_INTRA,
> > +						 0);
> > +	}
> > +
> > +	mtk_buf->param_change = MTK_ENCODE_PARAM_NONE;
> > +
> > +	if (ret) {
> > +		ctx->state = MTK_STATE_ABORT;
> > +		mtk_v4l2_err("venc_if_set_param %d failed=%d\n",
> > +			MTK_ENCODE_PARAM_FRAME_TYPE, ret);
> > +		return -1;
> > +	}
> > +
> > +	return 0;
> > +}
> > +
> > +static void mtk_venc_worker(struct work_struct *work)
> > +{
> > +	struct mtk_vcodec_ctx *ctx = container_of(work, struct mtk_vcodec_ctx,
> > +				    encode_work);
> > +	struct vb2_buffer *src_buf, *dst_buf;
> > +	struct venc_frm_buf frm_buf;
> > +	struct mtk_vcodec_mem bs_buf;
> > +	struct venc_done_result enc_result;
> > +	int ret;
> > +	struct vb2_v4l2_buffer *v4l2_vb;
> > +
> > +	if ((ctx->q_data[MTK_Q_DATA_DST].fmt->fourcc == V4L2_PIX_FMT_H264) &&
> > +	    (ctx->state != MTK_STATE_HEADER)) {
> > +		/* encode h264 sps/pps header */
> > +		mtk_venc_encode_header(ctx);
> > +		v4l2_m2m_job_finish(ctx->dev->m2m_dev_enc, ctx->m2m_ctx);
> > +		return;
> > +	}
> > +
> > +	src_buf = v4l2_m2m_next_src_buf(ctx->m2m_ctx);
> > +	if (!src_buf) {
> > +		v4l2_m2m_job_finish(ctx->dev->m2m_dev_enc, ctx->m2m_ctx);
> > +		return;
> > +	}
> > +
> > +	dst_buf = v4l2_m2m_dst_buf_remove(ctx->m2m_ctx);
> > +	if (!dst_buf) {
> > +		v4l2_m2m_job_finish(ctx->dev->m2m_dev_enc, ctx->m2m_ctx);
> > +		return;
> > +	}
> > +
> > +	mtk_venc_param_change(ctx, src_buf);
> > +
> > +	frm_buf.fb_addr.va = vb2_plane_vaddr(src_buf, 0);
> > +	frm_buf.fb_addr.dma_addr = vb2_dma_contig_plane_dma_addr(src_buf, 0);
> > +	frm_buf.fb_addr.size = (unsigned int)src_buf->planes[0].length;
> > +	frm_buf.fb_addr1.va = vb2_plane_vaddr(src_buf, 1);
> > +	frm_buf.fb_addr1.dma_addr = vb2_dma_contig_plane_dma_addr(src_buf, 1);
> > +	frm_buf.fb_addr1.size = (unsigned int)src_buf->planes[1].length;
> > +	if (src_buf->num_planes == 3) {
> > +		frm_buf.fb_addr2.va = vb2_plane_vaddr(src_buf, 2);
> > +		frm_buf.fb_addr2.dma_addr =
> > +			vb2_dma_contig_plane_dma_addr(src_buf, 2);
> > +		frm_buf.fb_addr2.size =
> > +			(unsigned int)src_buf->planes[2].length;
> > +	} else {
> > +		frm_buf.fb_addr2.va = NULL;
> > +		frm_buf.fb_addr2.dma_addr = 0;
> > +		frm_buf.fb_addr2.size = 0;
> > +	}
> > +	bs_buf.va = vb2_plane_vaddr(dst_buf, 0);
> > +	bs_buf.dma_addr = vb2_dma_contig_plane_dma_addr(dst_buf, 0);
> > +	bs_buf.size = (unsigned int)dst_buf->planes[0].length;
> > +
> > +	mtk_v4l2_debug(2,
> > +			"Framebuf VA=%p PA=%llx Size=0x%lx;VA=%p PA=0x%llx Size=0x%lx;VA=%p PA=0x%llx Size=0x%lx",
> > +			frm_buf.fb_addr.va,
> > +			(u64)frm_buf.fb_addr.dma_addr,
> > +			frm_buf.fb_addr.size,
> > +			frm_buf.fb_addr1.va,
> > +			(u64)frm_buf.fb_addr1.dma_addr,
> > +			frm_buf.fb_addr1.size,
> > +			frm_buf.fb_addr2.va,
> > +			(u64)frm_buf.fb_addr2.dma_addr,
> > +			frm_buf.fb_addr2.size);
> > +
> > +	ret = venc_if_encode(ctx, VENC_START_OPT_ENCODE_FRAME,
> > +			     &frm_buf, &bs_buf, &enc_result);
> > +
> > +	src_buf = v4l2_m2m_src_buf_remove(ctx->m2m_ctx);
> > +	if (enc_result.msg == VENC_MESSAGE_OK)
> > +		v4l2_m2m_buf_done(to_vb2_v4l2_buffer(src_buf), VB2_BUF_STATE_DONE);
> > +	else
> > +		v4l2_m2m_buf_done(to_vb2_v4l2_buffer(src_buf), VB2_BUF_STATE_ERROR);
> > +
> > +	if (enc_result.is_key_frm) {
> > +		v4l2_vb = to_vb2_v4l2_buffer(dst_buf);
> > +		v4l2_vb->flags |= V4L2_BUF_FLAG_KEYFRAME;
> > +	}
> > +
> > +	if (ret) {
> > +		dst_buf->planes[0].bytesused = 0;
> > +		v4l2_m2m_buf_done(to_vb2_v4l2_buffer(dst_buf), VB2_BUF_STATE_ERROR);
> > +		mtk_v4l2_err("venc_if_encode failed=%d", ret);
> > +	} else {
> > +		dst_buf->planes[0].bytesused = enc_result.bs_size;
> > +		v4l2_m2m_buf_done(to_vb2_v4l2_buffer(dst_buf), VB2_BUF_STATE_DONE);
> > +		mtk_v4l2_debug(2, "venc_if_encode bs size=%d",
> > +				 enc_result.bs_size);
> > +	}
> > +
> > +	v4l2_m2m_job_finish(ctx->dev->m2m_dev_enc, ctx->m2m_ctx);
> > +
> > +	mtk_v4l2_debug(1, "<=== src_buf[%d] dst_buf[%d] venc_if_encode ret=%d Size=%u===>",
> > +			src_buf->index, dst_buf->index, ret,
> > +			enc_result.bs_size);
> > +}
> > +
> > +static void m2mops_venc_device_run(void *priv)
> > +{
> > +	struct mtk_vcodec_ctx *ctx = priv;
> 
> Add empty line.
> 
We will fix this.

> > +	queue_work(ctx->dev->encode_workqueue, &ctx->encode_work);
> > +}
> > +
> > +static int m2mops_venc_job_ready(void *m2m_priv)
> > +{
> > +	struct mtk_vcodec_ctx *ctx = m2m_priv;
> > +
> > +	if (!v4l2_m2m_num_dst_bufs_ready(ctx->m2m_ctx)) {
> > +		mtk_v4l2_debug(3, "[%d]Not ready: not enough video dst buffers.",
> > +			       ctx->idx);
> > +		return 0;
> > +	}
> > +
> > +	if (!v4l2_m2m_num_src_bufs_ready(ctx->m2m_ctx)) {
> > +		mtk_v4l2_debug(3, "[%d]Not ready: not enough video src buffers.",
> > +			       ctx->idx);
> > +			return 0;
> > +		}
> 
> Broken indentation.
> 
We will fix this.

> > +
> > +	if (ctx->state == MTK_STATE_ABORT) {
> > +		mtk_v4l2_debug(3, "[%d]Not ready: state=0x%x.",
> > +			       ctx->idx, ctx->state);
> > +		return 0;
> > +	}
> > +
> > +	if (ctx->state == MTK_STATE_FREE) {
> > +		mtk_v4l2_debug(3, "[%d]Not ready: state=0x%x.",
> > +			       ctx->idx, ctx->state);
> > +		return 0;
> > +	}
> > +
> > +	return 1;
> > +}
> > +
> > +static void m2mops_venc_job_abort(void *priv)
> > +{
> > +	struct mtk_vcodec_ctx *ctx = priv;
> > +
> > +	ctx->state = MTK_STATE_ABORT;
> > +}
> > +
> > +static void m2mops_venc_lock(void *m2m_priv)
> > +{
> > +	struct mtk_vcodec_ctx *ctx = m2m_priv;
> > +
> > +	mutex_lock(&ctx->dev->dev_mutex);
> > +}
> > +
> > +static void m2mops_venc_unlock(void *m2m_priv)
> > +{
> > +	struct mtk_vcodec_ctx *ctx = m2m_priv;
> > +
> > +	mutex_unlock(&ctx->dev->dev_mutex);
> > +}
> > +
> > +const struct v4l2_m2m_ops mtk_venc_m2m_ops = {
> > +	.device_run			= m2mops_venc_device_run,
> > +	.job_ready			= m2mops_venc_job_ready,
> > +	.job_abort			= m2mops_venc_job_abort,
> > +	.lock				= m2mops_venc_lock,
> > +	.unlock				= m2mops_venc_unlock,
> > +};
> > +
> > +#define IS_MTK_VENC_PRIV(x) ((V4L2_CTRL_ID2CLASS(x) == V4L2_CTRL_CLASS_MPEG) &&\
> > +			     V4L2_CTRL_DRIVER_PRIV(x))
> > +
> > +void mtk_vcodec_enc_ctx_params_setup(struct mtk_vcodec_ctx *ctx)
> > +{
> > +	struct mtk_q_data *q_data;
> > +	struct mtk_video_fmt *fmt;
> > +
> > +	ctx->m2m_ctx->q_lock = &ctx->dev->dev_mutex;
> > +	ctx->fh.m2m_ctx = ctx->m2m_ctx;
> > +	ctx->fh.ctrl_handler = &ctx->ctrl_hdl;
> > +	INIT_WORK(&ctx->encode_work, mtk_venc_worker);
> > +
> > +	ctx->q_data[MTK_Q_DATA_SRC].width = DFT_CFG_WIDTH;
> > +	ctx->q_data[MTK_Q_DATA_SRC].height = DFT_CFG_HEIGHT;
> > +	ctx->q_data[MTK_Q_DATA_SRC].fmt = &mtk_video_formats[0];
> > +	ctx->q_data[MTK_Q_DATA_SRC].colorspace = V4L2_COLORSPACE_REC709;
> > +	ctx->q_data[MTK_Q_DATA_SRC].field = V4L2_FIELD_NONE;
> > +
> > +	q_data = &ctx->q_data[MTK_Q_DATA_SRC];
> > +	fmt = ctx->q_data[MTK_Q_DATA_SRC].fmt;
> > +	mtk_vcodec_enc_calc_src_size(fmt->num_planes, q_data->width,
> > +			q_data->height,
> > +			ctx->q_data[MTK_Q_DATA_SRC].sizeimage,
> > +			ctx->q_data[MTK_Q_DATA_SRC].bytesperline);
> > +
> > +	ctx->q_data[MTK_Q_DATA_DST].width = DFT_CFG_WIDTH;
> > +	ctx->q_data[MTK_Q_DATA_DST].height = DFT_CFG_HEIGHT;
> > +	ctx->q_data[MTK_Q_DATA_DST].fmt = &mtk_video_formats[9];
> > +	ctx->q_data[MTK_Q_DATA_DST].colorspace = V4L2_COLORSPACE_REC709;
> > +	ctx->q_data[MTK_Q_DATA_DST].field = V4L2_FIELD_NONE;
> > +
> > +	q_data = &ctx->q_data[MTK_Q_DATA_DST];
> > +	fmt = ctx->q_data[MTK_Q_DATA_DST].fmt;
> > +	ctx->q_data[MTK_Q_DATA_DST].sizeimage[0] = q_data->width * q_data->height;
> > +	ctx->q_data[MTK_Q_DATA_DST].bytesperline[0] = 0;
> > +
> > +}
> > +
> > +int mtk_vcodec_enc_ctrls_setup(struct mtk_vcodec_ctx *ctx)
> > +{
> > +	const struct v4l2_ctrl_ops *ops = &mtk_vcodec_enc_ctrl_ops;
> > +	struct v4l2_ctrl_handler *handler = &ctx->ctrl_hdl;
> > +	struct v4l2_ctrl_config cfg;
> > +
> > +	v4l2_ctrl_handler_init(handler, MTK_MAX_CTRLS);
> > +	if (handler->error) {
> > +		mtk_v4l2_err("Init control handler fail %d\n",
> > +				handler->error);
> > +		return handler->error;
> > +	}
> 
> Move this check to just before the v4l2_ctrl_handler_setup(). Any of the
> v4l2_ctrl_new_std* functions can set handler->error, which is why it should
> be check after all controls are added.
> 
Got it. Will move it just before the v4l2_ctrl_handler_setup().

> > +
> > +	ctx->ctrls[0] = v4l2_ctrl_new_std(handler, ops,
> > +					V4L2_CID_MPEG_VIDEO_BITRATE,
> > +					1, 4000000, 1, 4000000);
> > +	ctx->ctrls[1] = v4l2_ctrl_new_std(handler, ops,
> > +					V4L2_CID_MPEG_VIDEO_B_FRAMES,
> > +					0, 2, 1, 0);
> > +	ctx->ctrls[2] = v4l2_ctrl_new_std(handler, ops,
> > +					V4L2_CID_MPEG_VIDEO_FRAME_RC_ENABLE,
> > +					0, 1, 1, 1);
> > +	ctx->ctrls[3] = v4l2_ctrl_new_std(handler, ops,
> > +					V4L2_CID_MPEG_VIDEO_H264_MAX_QP,
> > +					0, 51, 1, 51);
> > +	ctx->ctrls[4] = v4l2_ctrl_new_std(handler, ops,
> > +					V4L2_CID_MPEG_VIDEO_H264_I_PERIOD,
> > +					0, 65535, 1, 30);
> > +	ctx->ctrls[5] = v4l2_ctrl_new_std(handler, ops,
> > +					V4L2_CID_MPEG_VIDEO_GOP_SIZE,
> > +					0, 65535, 1, 30);
> > +	ctx->ctrls[6] = v4l2_ctrl_new_std(handler, ops,
> > +					V4L2_CID_MPEG_VIDEO_MB_RC_ENABLE,
> > +					0, 1, 1, 0);
> > +	ctx->ctrls[7] = v4l2_ctrl_new_std_menu(handler, ops,
> > +					V4L2_CID_MPEG_VIDEO_HEADER_MODE,
> > +					V4L2_MPEG_VIDEO_HEADER_MODE_JOINED_WITH_1ST_FRAME,
> > +					0, V4L2_MPEG_VIDEO_HEADER_MODE_SEPARATE);
> > +	ctx->ctrls[8] = v4l2_ctrl_new_std_menu(handler, ops,
> > +					V4L2_CID_MPEG_VIDEO_H264_PROFILE,
> > +					V4L2_MPEG_VIDEO_H264_PROFILE_HIGH,
> > +					0, V4L2_MPEG_VIDEO_H264_PROFILE_MAIN);
> > +	ctx->ctrls[9] = v4l2_ctrl_new_std_menu(handler, ops,
> > +					V4L2_CID_MPEG_VIDEO_H264_LEVEL,
> > +					V4L2_MPEG_VIDEO_H264_LEVEL_4_2,
> > +					0, V4L2_MPEG_VIDEO_H264_LEVEL_4_0);
> > +	ctx->ctrls[6] = v4l2_ctrl_new_std(handler, ops,
> > +					V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME,
> > +					0, 0, 0, 0);
> 
> You are overwriting V4L2_CID_MPEG_VIDEO_MB_RC_ENABLE! Why are you assigning it
> anyway? Unless you are using ctx->ctrls[] you don't need to assign the result
> of v4l2_ctrl_new_std to anything.
> 
Got it. We don't use ctx->ctrls. We will remove it in next version.

> > +
> > +	v4l2_ctrl_handler_setup(&ctx->ctrl_hdl);
> > +
> > +	return 0;
> > +}
> > +
> > +int mtk_vcodec_enc_queue_init(void *priv, struct vb2_queue *src_vq,
> > +			   struct vb2_queue *dst_vq)
> > +{
> > +	struct mtk_vcodec_ctx *ctx = priv;
> > +	int ret;
> > +
> > +	src_vq->type		= V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE;
> > +	src_vq->io_modes	= VB2_DMABUF | VB2_MMAP | VB2_USERPTR;
> 
> I recomment dropping VB2_USERPTR. That only makes sense for scatter-gather dma,
> and you use physically contiguous DMA.
> 
Now our userspace app use VB2_USERPTR. I need to check if we could drop
VB2_USERPTR.
We use src_vq->mem_ops = &vb2_dma_contig_memops;
And there are
	.get_userptr	= vb2_dc_get_userptr,
	.put_userptr	= vb2_dc_put_userptr,
I was confused why it only make sense for scatter-gather.
Could you kindly explain more?

> > +	src_vq->drv_priv	= ctx;
> > +	src_vq->buf_struct_size = sizeof(struct mtk_video_enc_buf);
> > +	src_vq->ops		= &mtk_venc_vb2_ops;
> > +	src_vq->mem_ops		= &vb2_dma_contig_memops;
> > +	src_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
> > +	src_vq->lock = &ctx->dev->dev_mutex;
> > +
> > +	ret = vb2_queue_init(src_vq);
> > +	if (ret)
> > +		return ret;
> > +
> > +	dst_vq->type		= V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
> > +	dst_vq->io_modes	= VB2_DMABUF | VB2_MMAP | VB2_USERPTR;
> > +	dst_vq->drv_priv	= ctx;
> > +	dst_vq->buf_struct_size = sizeof(struct v4l2_m2m_buffer);
> > +	dst_vq->ops		= &mtk_venc_vb2_ops;
> > +	dst_vq->mem_ops		= &vb2_dma_contig_memops;
> > +	dst_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
> > +	dst_vq->lock = &ctx->dev->dev_mutex;
> > +
> > +	return vb2_queue_init(dst_vq);
> > +}
> > +
> > +int mtk_venc_unlock(struct mtk_vcodec_ctx *ctx)
> > +{
> > +	struct mtk_vcodec_dev *dev = ctx->dev;
> 
> Add empty line.
> 
Will fix this in next version.

> > +	dev->curr_ctx = -1;
> > +	mutex_unlock(&dev->enc_mutex);
> > +	return 0;
> > +}
> > +
> > +int mtk_venc_lock(struct mtk_vcodec_ctx *ctx)
> > +{
> > +	struct mtk_vcodec_dev *dev = ctx->dev;
> > +
> > +	mutex_lock(&dev->enc_mutex);
> > +	dev->curr_ctx = ctx->idx;
> > +	return 0;
> > +}
> > +
> > +void mtk_vcodec_enc_release(struct mtk_vcodec_ctx *ctx)
> > +{
> > +	venc_if_release(ctx);
> > +}
> > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h
> > new file mode 100644
> > index 0000000..e09524b
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h
> > @@ -0,0 +1,46 @@
> > +/*
> > +* Copyright (c) 2015 MediaTek Inc.
> > +* Author: PC Chen <pc.chen@mediatek.com>
> > +*         Tiffany Lin <tiffany.lin@mediatek.com>
> > +*
> > +* This program is free software; you can redistribute it and/or modify
> > +* it under the terms of the GNU General Public License version 2 as
> > +* published by the Free Software Foundation.
> > +*
> > +* This program is distributed in the hope that it will be useful,
> > +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> > +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > +* GNU General Public License for more details.
> > +*/
> > +
> > +
> > +#ifndef _MTK_VCODEC_ENC_H_
> > +#define _MTK_VCODEC_ENC_H_
> > +
> > +#include <media/videobuf2-core.h>
> > +#include <media/videobuf2-v4l2.h>
> > +
> > +/**
> > + * struct mtk_video_enc_buf - Private data related to each VB2 buffer.
> > + * @b:			Pointer to related VB2 buffer.
> > + * @param_change:	Types of encode parameter change before encode this
> > + *			buffer
> > + * @enc_params		Encode parameters changed before encode this buffer
> > + */
> > +struct mtk_video_enc_buf {
> > +	struct vb2_v4l2_buffer	vb;
> > +	struct list_head	list;
> > +
> > +	enum mtk_encode_param param_change;
> > +	struct mtk_enc_params enc_params;
> > +};
> > +
> > +int mtk_venc_unlock(struct mtk_vcodec_ctx *ctx);
> > +int mtk_venc_lock(struct mtk_vcodec_ctx *ctx);
> > +int mtk_vcodec_enc_queue_init(void *priv, struct vb2_queue *src_vq,
> > +					struct vb2_queue *dst_vq);
> > +void mtk_vcodec_enc_release(struct mtk_vcodec_ctx *ctx);
> > +int mtk_vcodec_enc_ctrls_setup(struct mtk_vcodec_ctx *ctx);
> > +void mtk_vcodec_enc_ctx_params_setup(struct mtk_vcodec_ctx *ctx);
> > +
> > +#endif /* _MTK_VCODEC_ENC_H_ */
> > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
> > new file mode 100644
> > index 0000000..e7ab14a
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
> > @@ -0,0 +1,476 @@
> > +/*
> > +* Copyright (c) 2015 MediaTek Inc.
> > +* Author: PC Chen <pc.chen@mediatek.com>
> > +*         Tiffany Lin <tiffany.lin@mediatek.com>
> > +*
> > +* This program is free software; you can redistribute it and/or modify
> > +* it under the terms of the GNU General Public License version 2 as
> > +* published by the Free Software Foundation.
> > +*
> > +* This program is distributed in the hope that it will be useful,
> > +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> > +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > +* GNU General Public License for more details.
> > +*/
> > +
> > +#include <linux/interrupt.h>
> > +#include <linux/irq.h>
> > +#include <linux/module.h>
> > +#include <linux/of_device.h>
> > +#include <linux/of.h>
> > +#include <media/v4l2-event.h>
> > +#include <media/v4l2-mem2mem.h>
> > +#include <media/videobuf2-dma-contig.h>
> > +#include <linux/pm_runtime.h>
> > +
> > +#include "mtk_vcodec_drv.h"
> > +#include "mtk_vcodec_enc.h"
> > +#include "mtk_vcodec_pm.h"
> > +#include "mtk_vcodec_intr.h"
> > +#include "mtk_vcodec_util.h"
> > +#include "mtk_vpu.h"
> > +
> > +
> > +/* Wake up context wait_queue */
> > +static void wake_up_ctx(struct mtk_vcodec_ctx *ctx, unsigned int reason)
> > +{
> > +	ctx->int_cond = 1;
> > +	ctx->int_type = reason;
> > +	wake_up_interruptible(&ctx->queue);
> > +}
> > +
> > +static irqreturn_t mtk_vcodec_enc_irq_handler(int irq, void *priv)
> > +{
> > +	struct mtk_vcodec_dev *dev = priv;
> > +	struct mtk_vcodec_ctx *ctx;
> > +	unsigned int irq_status;
> > +
> > +	if (dev->curr_ctx == -1) {
> > +		mtk_v4l2_err("curr_ctx = -1");
> > +		return IRQ_HANDLED;
> > +	}
> > +
> > +	ctx = dev->ctx[dev->curr_ctx];
> > +	if (ctx == NULL) {
> > +		mtk_v4l2_err("curr_ctx==NULL");
> > +		return IRQ_HANDLED;
> > +	}
> > +	mtk_v4l2_debug(1, "idx=%d", ctx->idx);
> > +	irq_status = readl(dev->reg_base[VENC_SYS] +
> > +				(MTK_VENC_IRQ_STATUS_OFFSET));
> > +	if (irq_status & MTK_VENC_IRQ_STATUS_PAUSE)
> > +		writel((MTK_VENC_IRQ_STATUS_PAUSE),
> > +		       dev->reg_base[VENC_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
> > +
> > +	if (irq_status & MTK_VENC_IRQ_STATUS_SWITCH)
> > +		writel((MTK_VENC_IRQ_STATUS_SWITCH),
> > +		       dev->reg_base[VENC_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
> > +
> > +	if (irq_status & MTK_VENC_IRQ_STATUS_DRAM)
> > +		writel((MTK_VENC_IRQ_STATUS_DRAM),
> > +		       dev->reg_base[VENC_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
> > +
> > +	if (irq_status & MTK_VENC_IRQ_STATUS_SPS)
> > +		writel((MTK_VENC_IRQ_STATUS_SPS),
> > +		       dev->reg_base[VENC_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
> > +
> > +	if (irq_status & MTK_VENC_IRQ_STATUS_PPS)
> > +		writel((MTK_VENC_IRQ_STATUS_PPS),
> > +		       dev->reg_base[VENC_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
> > +
> > +	if (irq_status & MTK_VENC_IRQ_STATUS_FRM)
> > +		writel((MTK_VENC_IRQ_STATUS_FRM),
> > +		       dev->reg_base[VENC_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
> > +
> > +	ctx->irq_status = irq_status;
> > +	wake_up_ctx(ctx, MTK_INST_IRQ_RECEIVED);
> > +	return IRQ_HANDLED;
> > +}
> > +
> > +#if 1 /* VENC_LT */
> > +static irqreturn_t mtk_vcodec_enc_irq_handler2(int irq, void *priv)
> > +{
> > +	struct mtk_vcodec_dev *dev = priv;
> > +	struct mtk_vcodec_ctx *ctx;
> > +	unsigned int irq_status;
> > +
> > +	ctx = dev->ctx[dev->curr_ctx];
> > +	if (ctx == NULL) {
> > +		mtk_v4l2_err("ctx==NULL");
> > +		return IRQ_HANDLED;
> > +	}
> > +	mtk_v4l2_debug(1, "idx=%d", ctx->idx);
> > +	irq_status = readl(dev->reg_base[VENC_LT_SYS] +
> > +				(MTK_VENC_IRQ_STATUS_OFFSET));
> > +	if (irq_status & MTK_VENC_IRQ_STATUS_PAUSE)
> > +		writel((MTK_VENC_IRQ_STATUS_PAUSE),
> > +		       dev->reg_base[VENC_LT_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
> > +
> > +	if (irq_status & MTK_VENC_IRQ_STATUS_SWITCH)
> > +		writel((MTK_VENC_IRQ_STATUS_SWITCH),
> > +		       dev->reg_base[VENC_LT_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
> > +
> > +	if (irq_status & MTK_VENC_IRQ_STATUS_DRAM)
> > +		writel((MTK_VENC_IRQ_STATUS_DRAM),
> > +		       dev->reg_base[VENC_LT_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
> > +
> > +	if (irq_status & MTK_VENC_IRQ_STATUS_SPS)
> > +		writel((MTK_VENC_IRQ_STATUS_SPS),
> > +		       dev->reg_base[VENC_LT_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
> > +
> > +	if (irq_status & MTK_VENC_IRQ_STATUS_PPS)
> > +		writel((MTK_VENC_IRQ_STATUS_PPS),
> > +		       dev->reg_base[VENC_LT_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
> > +
> > +	if (irq_status & MTK_VENC_IRQ_STATUS_FRM)
> > +		writel((MTK_VENC_IRQ_STATUS_FRM),
> > +		       dev->reg_base[VENC_LT_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
> > +
> > +	ctx->irq_status = irq_status;
> > +	wake_up_ctx(ctx, MTK_INST_IRQ_RECEIVED);
> > +	return IRQ_HANDLED;
> > +}
> > +#endif
> > +
> > +static void mtk_vcodec_enc_reset_handler(void *priv)
> > +{
> > +	int i;
> > +	struct mtk_vcodec_dev *dev = priv;
> > +	struct mtk_vcodec_ctx *ctx;
> > +
> > +	mtk_v4l2_debug(0, "Watchdog timeout!!");
> > +
> > +	mutex_lock(&dev->dev_mutex);
> > +	for(i = 0; i < MTK_VCODEC_MAX_ENCODER_INSTANCES; i++) {
> > +		ctx = dev->ctx[i];
> > +		if (ctx) {
> > +			ctx->state = MTK_STATE_ABORT;
> > +			mtk_v4l2_debug(0, "[%d] Change to state MTK_STATE_ERROR", ctx->idx);
> > +		}
> > +
> > +	}
> > +	mutex_unlock(&dev->dev_mutex);
> > +}
> > +
> > +static int fops_vcodec_open(struct file *file)
> > +{
> > +	struct video_device *vfd = video_devdata(file);
> > +	struct mtk_vcodec_dev *dev = video_drvdata(file);
> > +	struct mtk_vcodec_ctx *ctx = NULL;
> > +	int ret = 0;
> > +
> > +	mutex_lock(&dev->dev_mutex);
> > +
> > +	ctx = devm_kzalloc(&dev->plat_dev->dev, sizeof(*ctx), GFP_KERNEL);
> > +	if (!ctx) {
> > +		ret = -ENOMEM;
> > +		goto err_alloc;
> > +	}
> > +
> > +	if (dev->num_instances >= MTK_VCODEC_MAX_ENCODER_INSTANCES) {
> > +		mtk_v4l2_err("Too many open contexts\n");
> > +		ret = -EBUSY;
> > +		goto err_no_ctx;
> 
> Hmm. I never like it if you can't open a video node because of a reason like this.
> 
> I.e. a simple 'v4l2-ctl -D' (i.e. calling QUERYCAP) should never fail.
> 
> If there are hardware limitation that prevent more than X instances from running at
> the same time, then those limitations typically kick in when you start to stream
> (or possibly when calling REQBUFS). But before that it should always be possible to
> open the device.
> 
> Having this check at open() is an indication of a poor design.
> 
> Is this is a hardware limitation at all?
> 
This is to make sure performance meet requirements, such as bitrate and
framerate.
We got your point. We will remove this and move limitation control to
start_streaming or REQBUFS.
Appreciated for your suggestion.:)


> > +	}
> > +
> > +	ctx->idx = ffz(dev->instance_mask[0]);
> > +	v4l2_fh_init(&ctx->fh, video_devdata(file));
> > +	file->private_data = &ctx->fh;
> > +	v4l2_fh_add(&ctx->fh);
> > +	ctx->dev = dev;
> > +
> > +	if (vfd == dev->vfd_enc) {
> > +		ctx->type = MTK_INST_ENCODER;
> > +
> > +		ret = mtk_vcodec_enc_ctrls_setup(ctx);
> > +		if (ret) {
> > +			mtk_v4l2_err("Failed to setup controls() (%d)\n",
> > +				       ret);
> > +			goto err_ctrls_setup;
> > +		}
> > +		ctx->m2m_ctx = v4l2_m2m_ctx_init(dev->m2m_dev_enc, ctx,
> > +						 &mtk_vcodec_enc_queue_init);
> > +		if (IS_ERR(ctx->m2m_ctx)) {
> > +			ret = PTR_ERR(ctx->m2m_ctx);
> > +			mtk_v4l2_err("Failed to v4l2_m2m_ctx_init() (%d)\n",
> > +				       ret);
> > +			goto err_ctx_init;
> > +		}
> > +		mtk_vcodec_enc_ctx_params_setup(ctx);
> > +	} else {
> > +		mtk_v4l2_err("Invalid vfd !\n");
> > +		ret = -ENOENT;
> > +		goto err_ctx_init;
> > +	}
> > +
> > +	init_waitqueue_head(&ctx->queue);
> > +	dev->num_instances++;
> > +
> > +	if (dev->num_instances == 1) {
> 
> Having a counter is also not needed. You can use v4l2_fh_is_singular() to check if this
> is the first open.
> 
Got it. Thanks for your information.

> > +		ret = vpu_load_firmware(dev->vpu_plat_dev);
> > +		if (ret < 0) {
> > +				mtk_v4l2_err("vpu_load_firmware failed!\n");
> > +			goto err_load_fw;
> > +		}
> > +
> > +		dev->enc_capability =
> > +			vpu_get_venc_hw_capa(dev->vpu_plat_dev);
> > +		mtk_v4l2_debug(0, "encoder capability %x", dev->enc_capability);
> > +	}
> > +
> > +	mtk_v4l2_debug(2, "Create instance [%d]@%p m2m_ctx=%p type=%d\n",
> > +			 ctx->idx, ctx, ctx->m2m_ctx, ctx->type);
> > +	set_bit(ctx->idx, &dev->instance_mask[0]);
> > +	dev->ctx[ctx->idx] = ctx;
> > +
> > +	mutex_unlock(&dev->dev_mutex);
> > +	mtk_v4l2_debug(0, "%s encoder [%d]", dev_name(&dev->plat_dev->dev), ctx->idx);
> > +	return ret;
> > +
> > +	/* Deinit when failure occurred */
> > +err_load_fw:
> > +	v4l2_m2m_ctx_release(ctx->m2m_ctx);
> > +	v4l2_fh_del(&ctx->fh);
> > +	v4l2_fh_exit(&ctx->fh);
> > +	dev->num_instances--;
> > +err_ctx_init:
> > +err_ctrls_setup:
> > +	v4l2_ctrl_handler_free(&ctx->ctrl_hdl);
> > +err_no_ctx:
> > +	devm_kfree(&dev->plat_dev->dev, ctx);
> > +err_alloc:
> > +	mutex_unlock(&dev->dev_mutex);
> > +	return ret;
> > +}
> > +
> > +static int fops_vcodec_release(struct file *file)
> > +{
> > +	struct mtk_vcodec_dev *dev = video_drvdata(file);
> > +	struct mtk_vcodec_ctx *ctx = fh_to_ctx(file->private_data);
> > +
> > +	mtk_v4l2_debug(0, "[%d] encoder\n", ctx->idx);
> > +	mutex_lock(&dev->dev_mutex);
> > +
> > +	mtk_vcodec_enc_release(ctx);
> > +	v4l2_fh_del(&ctx->fh);
> > +	v4l2_fh_exit(&ctx->fh);
> > +	v4l2_ctrl_handler_free(&ctx->ctrl_hdl);
> > +	v4l2_m2m_ctx_release(ctx->m2m_ctx);
> > +
> > +	dev->ctx[ctx->idx] = NULL;
> > +	dev->num_instances--;
> > +	clear_bit(ctx->idx, &dev->instance_mask[0]);
> > +	devm_kfree(&dev->plat_dev->dev, ctx);
> > +	mutex_unlock(&dev->dev_mutex);
> > +	return 0;
> > +}
> > +
> > +static const struct v4l2_file_operations mtk_vcodec_fops = {
> > +	.owner				= THIS_MODULE,
> > +	.open				= fops_vcodec_open,
> > +	.release			= fops_vcodec_release,
> > +	.poll				= v4l2_m2m_fop_poll,
> > +	.unlocked_ioctl			= video_ioctl2,
> > +	.mmap				= v4l2_m2m_fop_mmap,
> > +};
> > +
> > +static int mtk_vcodec_probe(struct platform_device *pdev)
> > +{
> > +	struct mtk_vcodec_dev *dev;
> > +	struct video_device *vfd_enc;
> > +	struct resource *res;
> > +	int i, j, ret;
> > +
> > +	dev = devm_kzalloc(&pdev->dev, sizeof(*dev), GFP_KERNEL);
> > +	if (!dev)
> > +		return -ENOMEM;
> > +
> > +	dev->plat_dev = pdev;
> > +
> > +	dev->vpu_plat_dev = vpu_get_plat_device(dev->plat_dev);
> > +	if (dev->vpu_plat_dev == NULL) {
> > +		mtk_v4l2_err("[VPU] vpu device in not ready\n");
> > +		return -EPROBE_DEFER;
> > +	}
> > +
> > +	vpu_wdt_reg_handler(dev->vpu_plat_dev, mtk_vcodec_enc_reset_handler, dev,
> > +			    VPU_RST_ENC);
> > +
> > +	ret = mtk_vcodec_init_enc_pm(dev);
> > +	if (ret < 0) {
> > +		dev_err(&pdev->dev, "Failed to get mt vcodec clock source!\n");
> > +		return ret;
> > +	}
> > +
> > +	for (i = VENC_SYS, j = 0; i < NUM_MAX_VCODEC_REG_BASE; i++, j++) {
> > +		res = platform_get_resource(pdev, IORESOURCE_MEM, j);
> > +		if (res == NULL) {
> > +			dev_err(&pdev->dev, "get memory resource failed.\n");
> > +			ret = -ENXIO;
> > +			goto err_res;
> > +		}
> > +		dev->reg_base[i] = devm_ioremap_resource(&pdev->dev, res);
> > +		if (IS_ERR(dev->reg_base[i])) {
> > +			dev_err(&pdev->dev,
> > +				"devm_ioremap_resource %d failed.\n", i);
> > +			ret = PTR_ERR(dev->reg_base);
> > +			goto err_res;
> > +		}
> > +		mtk_v4l2_debug(2, "reg[%d] base=0x%p\n", i, dev->reg_base[i]);
> > +	}
> > +
> > +	res = platform_get_resource(pdev, IORESOURCE_IRQ, 0);
> > +	if (res == NULL) {
> > +		dev_err(&pdev->dev, "failed to get irq resource\n");
> > +		ret = -ENOENT;
> > +		goto err_res;
> > +	}
> > +
> > +	dev->enc_irq = platform_get_irq(pdev, 0);
> > +	ret = devm_request_irq(&pdev->dev, dev->enc_irq,
> > +			       mtk_vcodec_enc_irq_handler,
> > +			       0, pdev->name, dev);
> > +	if (ret) {
> > +		dev_err(&pdev->dev, "Failed to install dev->enc_irq %d (%d)\n",
> > +			dev->enc_irq,
> > +			ret);
> > +		ret = -EINVAL;
> > +		goto err_res;
> > +	}
> > +
> > +	dev->enc_lt_irq = platform_get_irq(pdev, 1);
> > +	ret = devm_request_irq(&pdev->dev,
> > +			       dev->enc_lt_irq, mtk_vcodec_enc_irq_handler2,
> > +			       0, pdev->name, dev);
> > +	if (ret) {
> > +		dev_err(&pdev->dev,
> > +			"Failed to install dev->enc_lt_irq %d (%d)\n",
> > +			dev->enc_lt_irq, ret);
> > +		ret = -EINVAL;
> > +		goto err_res;
> > +	}
> > +
> > +	disable_irq(dev->enc_irq);
> > +	disable_irq(dev->enc_lt_irq); /* VENC_LT */
> > +	mutex_init(&dev->enc_mutex);
> > +	mutex_init(&dev->dev_mutex);
> > +
> > +	snprintf(dev->v4l2_dev.name, sizeof(dev->v4l2_dev.name), "%s",
> > +		 "[MTK_V4L2_VENC]");
> > +
> > +	ret = v4l2_device_register(&pdev->dev, &dev->v4l2_dev);
> > +	if (ret) {
> > +		mtk_v4l2_err("v4l2_device_register err=%d\n", ret);
> > +		return ret;
> > +	}
> > +
> > +	init_waitqueue_head(&dev->queue);
> > +
> > +	/* allocate video device for encoder and register it */
> > +	vfd_enc = video_device_alloc();
> > +	if (!vfd_enc) {
> > +		mtk_v4l2_err("Failed to allocate video device\n");
> > +		ret = -ENOMEM;
> > +		goto err_enc_alloc;
> > +	}
> > +	vfd_enc->fops           = &mtk_vcodec_fops;
> > +	vfd_enc->ioctl_ops      = &mtk_venc_ioctl_ops;
> > +	vfd_enc->release        = video_device_release;
> > +	vfd_enc->lock           = &dev->dev_mutex;
> > +	vfd_enc->v4l2_dev       = &dev->v4l2_dev;
> > +	vfd_enc->vfl_dir        = VFL_DIR_M2M;
> > +
> > +	snprintf(vfd_enc->name, sizeof(vfd_enc->name), "%s",
> > +		 MTK_VCODEC_ENC_NAME);
> > +	video_set_drvdata(vfd_enc, dev);
> > +	dev->vfd_enc = vfd_enc;
> > +	platform_set_drvdata(pdev, dev);
> > +	ret = video_register_device(vfd_enc, VFL_TYPE_GRABBER, 1);
> > +	if (ret) {
> > +		mtk_v4l2_err("Failed to register video device\n");
> > +		goto err_enc_reg;
> > +	}
> 
> Do the register last. After registering the device anyone can start using it,
> but since not everything is initialized that might cause crashes.
> 
Got it. We will fix this.

> > +	mtk_v4l2_debug(0, "encoder registered as /dev/video%d\n",
> > +			 vfd_enc->num);
> > +
> > +	dev->alloc_ctx = vb2_dma_contig_init_ctx(&pdev->dev);
> > +	if (IS_ERR(dev->alloc_ctx)) {
> > +		mtk_v4l2_err("Failed to alloc vb2 dma context 0\n");
> > +		ret = PTR_ERR(dev->alloc_ctx);
> > +		goto err_vb2_ctx_init;
> > +	}
> > +
> > +	dev->m2m_dev_enc = v4l2_m2m_init(&mtk_venc_m2m_ops);
> > +	if (IS_ERR(dev->m2m_dev_enc)) {
> > +		mtk_v4l2_err("Failed to init mem2mem enc device\n");
> > +		ret = PTR_ERR(dev->m2m_dev_enc);
> > +		goto err_enc_mem_init;
> > +	}
> > +
> > +	dev->encode_workqueue =
> > +			alloc_ordered_workqueue(MTK_VCODEC_ENC_NAME, WQ_MEM_RECLAIM | WQ_FREEZABLE);
> > +	if (!dev->encode_workqueue) {
> > +		mtk_v4l2_err("Failed to create encode workqueue\n");
> > +		ret = -EINVAL;
> > +		goto err_event_workq;
> > +	}
> > +
> > +	return 0;
> > +
> > +err_event_workq:
> > +	v4l2_m2m_release(dev->m2m_dev_enc);
> > +err_enc_mem_init:
> > +	vb2_dma_contig_cleanup_ctx(dev->alloc_ctx);
> > +err_vb2_ctx_init:
> > +	video_unregister_device(vfd_enc);
> > +err_enc_reg:
> > +	video_device_release(vfd_enc);
> > +err_enc_alloc:
> > +	v4l2_device_unregister(&dev->v4l2_dev);
> > +err_res:
> > +	mtk_vcodec_release_enc_pm(dev);
> > +	return ret;
> > +}
> > +
> > +static const struct of_device_id mtk_vcodec_match[] = {
> > +	{.compatible = "mediatek,mt8173-vcodec-enc",},
> > +	{},
> > +};
> > +MODULE_DEVICE_TABLE(of, mtk_vcodec_match);
> > +
> > +static int mtk_vcodec_remove(struct platform_device *pdev)
> > +{
> > +	struct mtk_vcodec_dev *dev = platform_get_drvdata(pdev);
> > +
> > +	mtk_v4l2_debug_enter();
> > +	flush_workqueue(dev->encode_workqueue);
> > +	destroy_workqueue(dev->encode_workqueue);
> > +	if (dev->m2m_dev_enc)
> > +		v4l2_m2m_release(dev->m2m_dev_enc);
> > +	if (dev->alloc_ctx)
> > +		vb2_dma_contig_cleanup_ctx(dev->alloc_ctx);
> > +
> > +	if (dev->vfd_enc) {
> > +		video_unregister_device(dev->vfd_enc);
> > +		video_device_release(dev->vfd_enc);
> 
> Don't call video_device_release! That will happen automatically once the
> last filehandle is closed.
> 
Got it. We will fix this.

> > +	}
> > +	v4l2_device_unregister(&dev->v4l2_dev);
> > +	mtk_vcodec_release_enc_pm(dev);
> > +	return 0;
> > +}
> > +
> > +static struct platform_driver mtk_vcodec_driver = {
> > +	.probe	= mtk_vcodec_probe,
> > +	.remove	= mtk_vcodec_remove,
> > +	.driver	= {
> > +		.name	= MTK_VCODEC_ENC_NAME,
> > +		.owner	= THIS_MODULE,
> > +		.of_match_table = mtk_vcodec_match,
> > +	},
> > +};
> > +
> > +module_platform_driver(mtk_vcodec_driver);
> > +
> > +
> > +MODULE_LICENSE("GPL v2");
> > +MODULE_DESCRIPTION("Mediatek video codec V4L2 driver");
> > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
> > new file mode 100644
> > index 0000000..518fba7
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
> > @@ -0,0 +1,132 @@
> > +/*
> > +* Copyright (c) 2015 MediaTek Inc.
> > +* Author: Tiffany Lin <tiffany.lin@mediatek.com>
> > +*
> > +* This program is free software; you can redistribute it and/or modify
> > +* it under the terms of the GNU General Public License version 2 as
> > +* published by the Free Software Foundation.
> > +*
> > +* This program is distributed in the hope that it will be useful,
> > +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> > +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > +* GNU General Public License for more details.
> > +*/
> > +
> > +#include <linux/clk.h>
> > +#include <linux/of_address.h>
> > +#include <linux/of_platform.h>
> > +#include <linux/pm_runtime.h>
> > +#include <soc/mediatek/smi.h>
> > +
> > +#include "mtk_vcodec_pm.h"
> > +#include "mtk_vcodec_util.h"
> > +#include "mtk_vpu.h"
> > +
> > +
> > +int mtk_vcodec_init_enc_pm(struct mtk_vcodec_dev *mtkdev)
> > +{
> > +	struct device_node *node;
> > +	struct platform_device *pdev;
> > +	struct device *dev;
> > +	struct mtk_vcodec_pm *pm;
> > +	int ret = 0;
> > +
> > +	pdev = mtkdev->plat_dev;
> > +	pm = &mtkdev->pm;
> > +	memset(pm, 0, sizeof(struct mtk_vcodec_pm));
> > +	pm->mtkdev = mtkdev;
> > +	dev = &pdev->dev;
> > +
> > +	node = of_parse_phandle(dev->of_node, "mediatek,larb", 0);
> > +	if (!node)
> > +		return -1;
> > +	pdev = of_find_device_by_node(node);
> > +	if (WARN_ON(!pdev)) {
> > +		of_node_put(node);
> > +		return -1;
> > +	}
> > +	pm->larbvenc = &pdev->dev;
> > +
> > +	node = of_parse_phandle(dev->of_node, "mediatek,larb", 1);
> > +	if (!node)
> > +		return -1;
> > +
> > +	pdev = of_find_device_by_node(node);
> > +	if (WARN_ON(!pdev)) {
> > +		of_node_put(node);
> > +		return -EINVAL;
> > +	}
> > +	pm->larbvenclt = &pdev->dev;
> > +
> > +	pdev = mtkdev->plat_dev;
> > +	pm->dev = &pdev->dev;
> > +
> > +	pm->vencpll_d2 = devm_clk_get(&pdev->dev, "vencpll_d2");
> > +	if (pm->vencpll_d2 == NULL) {
> > +		mtk_v4l2_err("devm_clk_get vencpll_d2 fail");
> > +		ret = -1;
> > +	}
> > +
> > +	pm->venc_sel = devm_clk_get(&pdev->dev, "venc_sel");
> > +	if (pm->venc_sel == NULL) {
> > +		mtk_v4l2_err("devm_clk_get venc_sel fail");
> > +		ret = -1;
> > +	}
> > +
> > +	pm->univpll1_d2 = devm_clk_get(&pdev->dev, "univpll1_d2");
> > +	if (pm->univpll1_d2 == NULL) {
> > +		mtk_v4l2_err("devm_clk_get univpll1_d2 fail");
> > +		ret = -1;
> > +	}
> > +
> > +	pm->venc_lt_sel = devm_clk_get(&pdev->dev, "venc_lt_sel");
> > +	if (pm->venc_lt_sel == NULL) {
> > +		mtk_v4l2_err("devm_clk_get venc_lt_sel fail");
> > +		ret = -1;
> > +	}
> > +
> > +	return ret;
> > +}
> > +
> > +void mtk_vcodec_release_enc_pm(struct mtk_vcodec_dev *mtkdev)
> > +{
> > +}
> > +
> > +
> > +void mtk_vcodec_enc_clock_on(struct mtk_vcodec_pm *pm)
> > +{
> > +	int ret;
> > +
> > +	ret = clk_prepare_enable(pm->venc_sel);
> > +	if (ret)
> > +		mtk_v4l2_err("venc_sel fail %d", ret);
> > +
> > +	ret = clk_set_parent(pm->venc_sel, pm->vencpll_d2);
> > +	if (ret)
> > +		mtk_v4l2_err("clk_set_parent fail %d", ret);
> > +
> > +	ret = clk_prepare_enable(pm->venc_lt_sel);
> > +	if (ret)
> > +		mtk_v4l2_err("venc_lt_sel fail %d", ret);
> > +
> > +	ret = clk_set_parent(pm->venc_lt_sel, pm->univpll1_d2);
> > +	if (ret)
> > +		mtk_v4l2_err("clk_set_parent fail %d", ret);
> > +
> > +	ret = mtk_smi_larb_get(pm->larbvenc);
> > +	if (ret)
> > +		mtk_v4l2_err("mtk_smi_larb_get larb3 fail %d\n", ret);
> > +
> > +	ret = mtk_smi_larb_get(pm->larbvenclt);
> > +	if (ret)
> > +		mtk_v4l2_err("mtk_smi_larb_get larb4 fail %d\n", ret);
> > +
> > +}
> > +
> > +void mtk_vcodec_enc_clock_off(struct mtk_vcodec_pm *pm)
> > +{
> > +	mtk_smi_larb_put(pm->larbvenc);
> > +	mtk_smi_larb_put(pm->larbvenclt);
> > +	clk_disable_unprepare(pm->venc_lt_sel);
> > +	clk_disable_unprepare(pm->venc_sel);
> > +}
> > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.c
> > new file mode 100644
> > index 0000000..919b949
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.c
> > @@ -0,0 +1,102 @@
> > +/*
> > +* Copyright (c) 2015 MediaTek Inc.
> > +* Author: Tiffany Lin <tiffany.lin@mediatek.com>
> > +*
> > +* This program is free software; you can redistribute it and/or modify
> > +* it under the terms of the GNU General Public License version 2 as
> > +* published by the Free Software Foundation.
> > +*
> > +* This program is distributed in the hope that it will be useful,
> > +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> > +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > +* GNU General Public License for more details.
> > +*/
> > +
> > +#include <linux/errno.h>
> > +#include <linux/wait.h>
> > +
> > +#include "mtk_vcodec_drv.h"
> > +#include "mtk_vcodec_intr.h"
> > +#include "mtk_vcodec_util.h"
> > +
> > +void mtk_vcodec_clean_dev_int_flags(void *data)
> > +{
> > +	struct mtk_vcodec_dev *dev = (struct mtk_vcodec_dev *)data;
> > +
> > +	dev->int_cond = 0;
> > +	dev->int_type = 0;
> > +}
> > +
> > +int mtk_vcodec_wait_for_done_ctx(void *data, int command,
> > +				 unsigned int timeout_ms, int interrupt)
> > +{
> > +	wait_queue_head_t *waitqueue;
> > +	long timeout_jiff, ret;
> > +	int status = 0;
> > +	struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)data;
> > +
> > +	waitqueue = (wait_queue_head_t *)&ctx->queue;
> > +	timeout_jiff = msecs_to_jiffies(timeout_ms);
> > +	if (interrupt) {
> > +		ret = wait_event_interruptible_timeout(*waitqueue,
> > +				(ctx->int_cond &&
> > +				(ctx->int_type == command)),
> > +				timeout_jiff);
> > +	} else {
> > +		ret = wait_event_timeout(*waitqueue,
> > +				(ctx->int_cond &&
> > +				(ctx->int_type == command)),
> > +				 timeout_jiff);
> > +	}
> > +	if (0 == ret) {
> > +		status = -1;	/* timeout */
> > +		mtk_v4l2_err("[%d] cmd=%d, ctx->type=%d, wait_event_interruptible_timeout time=%ums out %d %d!",
> > +				ctx->idx, ctx->type, command, timeout_ms,
> > +				ctx->int_cond, ctx->int_type);
> > +	} else if (-ERESTARTSYS == ret) {
> > +		mtk_v4l2_err("[%d] cmd=%d, ctx->type=%d, wait_event_interruptible_timeout interrupted by a signal %d %d",
> > +				ctx->idx, ctx->type, command, ctx->int_cond,
> > +				ctx->int_type);
> > +		status = -1;
> > +	}
> > +
> > +	ctx->int_cond = 0;
> > +	ctx->int_type = 0;
> > +
> > +	return status;
> > +}
> > +
> > +int mtk_vcodec_wait_for_done_dev(void *data, int command,
> > +				 unsigned int timeout, int interrupt)
> > +{
> > +	wait_queue_head_t *waitqueue;
> > +	long timeout_jiff, ret;
> > +	int status = 0;
> > +	struct mtk_vcodec_dev *dev = (struct mtk_vcodec_dev *)data;
> > +
> > +	waitqueue = (wait_queue_head_t *)&dev->queue;
> > +	timeout_jiff = msecs_to_jiffies(timeout);
> > +	if (interrupt) {
> > +		ret = wait_event_interruptible_timeout(*waitqueue,
> > +				(dev->int_cond &&
> > +				(dev->int_type == command)),
> > +				timeout_jiff);
> > +	} else {
> > +		ret = wait_event_timeout(*waitqueue,
> > +				(dev->int_cond &&
> > +				(dev->int_type == command)),
> > +				timeout_jiff);
> > +	}
> > +	if (0 == ret) {
> > +		status = -1;	/* timeout */
> > +		mtk_v4l2_err("wait_event_interruptible_timeout time=%lu out %d %d!",
> > +				timeout_jiff, dev->int_cond, dev->int_type);
> > +	} else if (-ERESTARTSYS == ret) {
> > +		mtk_v4l2_err("wait_event_interruptible_timeout interrupted by a signal %d %d",
> > +				dev->int_cond, dev->int_type);
> > +		status = -1;
> > +	}
> > +	dev->int_cond = 0;
> > +	dev->int_type = 0;
> > +	return status;
> > +}
> > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.h
> > new file mode 100644
> > index 0000000..e9b7f94
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.h
> > @@ -0,0 +1,29 @@
> > +/*
> > +* Copyright (c) 2015 MediaTek Inc.
> > +* Author: Tiffany Lin <tiffany.lin@mediatek.com>
> > +*
> > +* This program is free software; you can redistribute it and/or modify
> > +* it under the terms of the GNU General Public License version 2 as
> > +* published by the Free Software Foundation.
> > +*
> > +* This program is distributed in the hope that it will be useful,
> > +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> > +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > +* GNU General Public License for more details.
> > +*/
> > +
> > +#ifndef _MTK_VCODEC_INTR_H_
> > +#define _MTK_VCODEC_INTR_H_
> > +
> > +#define MTK_INST_IRQ_RECEIVED		0x1
> > +#define MTK_INST_WORK_THREAD_ABORT_DONE	0x2
> > +
> > +/* timeout is ms */
> > +int mtk_vcodec_wait_for_done_ctx(void *data, int command, unsigned int timeout,
> > +				 int interrupt);
> > +int mtk_vcodec_wait_for_done_dev(void *data, int command, unsigned int timeout,
> > +				 int interrupt);
> > +
> > +void mtk_vcodec_clean_dev_int_flags(void *data);
> > +
> > +#endif /* _MTK_VCODEC_INTR_H_ */
> > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_pm.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_pm.h
> > new file mode 100644
> > index 0000000..fdadec9
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_pm.h
> > @@ -0,0 +1,26 @@
> > +/*
> > +* Copyright (c) 2015 MediaTek Inc.
> > +* Author: Tiffany Lin <tiffany.lin@mediatek.com>
> > +*
> > +* This program is free software; you can redistribute it and/or modify
> > +* it under the terms of the GNU General Public License version 2 as
> > +* published by the Free Software Foundation.
> > +*
> > +* This program is distributed in the hope that it will be useful,
> > +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> > +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > +* GNU General Public License for more details.
> > +*/
> > +
> > +#ifndef _MTK_VCODEC_PM_H_
> > +#define _MTK_VCODEC_PM_H_
> > +
> > +#include "mtk_vcodec_drv.h"
> > +
> > +int mtk_vcodec_init_enc_pm(struct mtk_vcodec_dev *dev);
> > +void mtk_vcodec_release_enc_pm(struct mtk_vcodec_dev *dev);
> > +
> > +void mtk_vcodec_enc_clock_on(struct mtk_vcodec_pm *pm);
> > +void mtk_vcodec_enc_clock_off(struct mtk_vcodec_pm *pm);
> > +
> > +#endif /* _MTK_VCODEC_PM_H_ */
> > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c
> > new file mode 100644
> > index 0000000..3fede8d
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c
> > @@ -0,0 +1,106 @@
> > +/*
> > +* Copyright (c) 2015 MediaTek Inc.
> > +* Author: PC Chen <pc.chen@mediatek.com>
> > +*         Tiffany Lin <tiffany.lin@mediatek.com>
> > +*
> > +* This program is free software; you can redistribute it and/or modify
> > +* it under the terms of the GNU General Public License version 2 as
> > +* published by the Free Software Foundation.
> > +*
> > +* This program is distributed in the hope that it will be useful,
> > +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> > +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > +* GNU General Public License for more details.
> > +*/
> > +
> > +#include <linux/module.h>
> > +
> > +#include "mtk_vcodec_drv.h"
> > +#include "mtk_vcodec_util.h"
> > +#include "mtk_vpu.h"
> > +
> > +bool mtk_vcodec_dbg = false;
> > +int mtk_v4l2_dbg_level = 0;
> > +
> > +module_param(mtk_v4l2_dbg_level, int, S_IRUGO | S_IWUSR);
> > +module_param(mtk_vcodec_dbg, bool, S_IRUGO | S_IWUSR);
> > +
> > +void __iomem *mtk_vcodec_get_reg_addr(void *data, unsigned int reg_idx)
> > +{
> > +	struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)data;
> > +
> > +	if (!data || reg_idx >= NUM_MAX_VCODEC_REG_BASE) {
> > +		mtk_v4l2_err("Invalid arguments");
> > +		return NULL;
> > +	}
> > +	return ctx->dev->reg_base[reg_idx];
> > +}
> > +
> > +int mtk_vcodec_mem_alloc(void *data, struct mtk_vcodec_mem *mem)
> > +{
> > +	unsigned long size = mem->size;
> > +	struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)data;
> > +	struct device *dev = &ctx->dev->plat_dev->dev;
> > +
> > +	mem->va = dma_alloc_coherent(dev, size, &mem->dma_addr, GFP_KERNEL);
> > +
> > +	if (!mem->va) {
> > +		mtk_v4l2_err("%s dma_alloc size=%ld failed!", dev_name(dev),
> > +			       size);
> > +		return -ENOMEM;
> > +	}
> > +
> > +	memset(mem->va, 0, size);
> > +
> > +	mtk_v4l2_debug(3, "[%d]  - va      = %p", ctx->idx, mem->va);
> > +	mtk_v4l2_debug(3, "[%d]  - dma     = 0x%lx", ctx->idx,
> > +			 (unsigned long)mem->dma_addr);
> > +	mtk_v4l2_debug(3, "[%d]    size = 0x%lx", ctx->idx, size);
> > +
> > +	return 0;
> > +}
> > +
> > +void mtk_vcodec_mem_free(void *data, struct mtk_vcodec_mem *mem)
> > +{
> > +	unsigned long size = mem->size;
> > +	struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)data;
> > +	struct device *dev = &ctx->dev->plat_dev->dev;
> > +
> > +	dma_free_coherent(dev, size, mem->va, mem->dma_addr);
> > +	mem->va = NULL;
> > +
> > +	mtk_v4l2_debug(3, "[%d]  - va      = %p", ctx->idx, mem->va);
> > +	mtk_v4l2_debug(3, "[%d]  - dma     = 0x%lx", ctx->idx,
> > +			 (unsigned long)mem->dma_addr);
> > +	mtk_v4l2_debug(3, "[%d]    size = 0x%lx", ctx->idx, size);
> > +}
> > +
> > +int mtk_vcodec_get_ctx_id(void *data)
> > +{
> > +	struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)data;
> > +
> > +	if (!ctx)
> > +		return -1;
> > +
> > +	return ctx->idx;
> > +}
> > +
> > +struct platform_device *mtk_vcodec_get_plat_dev(void *data)
> > +{
> > +	struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)data;
> > +
> > +	if (!ctx)
> > +		return NULL;
> > +
> > +	return vpu_get_plat_device(ctx->dev->plat_dev);
> > +}
> > +
> > +void mtk_vcodec_fmt2str(u32 fmt, char *str)
> > +{
> > +	char a = fmt & 0xFF;
> > +	char b = (fmt >> 8) & 0xFF;
> > +	char c = (fmt >> 16) & 0xFF;
> > +	char d = (fmt >> 24) & 0xFF;
> > +
> > +	sprintf(str, "%c%c%c%c", a, b, c, d);
> > +}
> > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
> > new file mode 100644
> > index 0000000..47016ae
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
> > @@ -0,0 +1,85 @@
> > +/*
> > +* Copyright (c) 2015 MediaTek Inc.
> > +* Author: PC Chen <pc.chen@mediatek.com>
> > +*         Tiffany Lin <tiffany.lin@mediatek.com>
> > +*
> > +* This program is free software; you can redistribute it and/or modify
> > +* it under the terms of the GNU General Public License version 2 as
> > +* published by the Free Software Foundation.
> > +*
> > +* This program is distributed in the hope that it will be useful,
> > +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> > +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > +* GNU General Public License for more details.
> > +*/
> > +
> > +#ifndef _MTK_VCODEC_UTIL_H_
> > +#define _MTK_VCODEC_UTIL_H_
> > +
> > +#include <linux/types.h>
> > +#include <linux/dma-direction.h>
> > +
> > +struct mtk_vcodec_mem {
> > +	size_t size;
> > +	void *va;
> > +	dma_addr_t dma_addr;
> > +};
> > +
> > +extern int mtk_v4l2_dbg_level;
> > +extern bool mtk_vcodec_dbg;
> > +
> > +#define DEBUG 	1
> > +
> > +#if defined(DEBUG)
> > +
> > +#define mtk_v4l2_debug(level, fmt, args...)				 \
> > +	do {								 \
> > +		if (mtk_v4l2_dbg_level >= level)			 \
> > +			pr_info("[MTK_V4L2] level=%d %s(),%d: " fmt "\n",\
> > +				level, __func__, __LINE__, ##args);	 \
> > +	} while (0)
> > +
> > +#define mtk_v4l2_err(fmt, args...)                \
> > +	pr_err("[MTK_V4L2][ERROR] %s:%d: " fmt "\n", __func__, __LINE__, \
> > +	       ##args)
> > +
> > +
> > +#define mtk_v4l2_debug_enter()  mtk_v4l2_debug(3, "+\n")
> > +#define mtk_v4l2_debug_leave()  mtk_v4l2_debug(3, "-\n")
> > +
> > +#define mtk_vcodec_debug(h, fmt, args...)				\
> > +	do {								\
> > +		if (mtk_vcodec_dbg)					\
> > +			pr_info("[MTK_VCODEC][%d]: %s() " fmt "\n",	\
> > +				((struct mtk_vcodec_ctx *)h->ctx)->idx, \
> > +				__func__, ##args);			\
> > +	} while (0)
> > +
> > +#define mtk_vcodec_err(h, fmt, args...)					\
> > +	pr_err("[MTK_VCODEC][ERROR][%d]: %s() " fmt "\n",		\
> > +	       ((struct mtk_vcodec_ctx *)h->ctx)->idx, __func__, ##args)
> > +
> > +#define mtk_vcodec_debug_enter(h)  mtk_vcodec_debug(h, "+\n")
> > +#define mtk_vcodec_debug_leave(h)  mtk_vcodec_debug(h, "-\n")
> > +
> > +#else
> > +
> > +#define mtk_v4l2_debug(level, fmt, args...)
> > +#define mtk_v4l2_err(fmt, args...)
> > +#define mtk_v4l2_debug_enter()
> > +#define mtk_v4l2_debug_leave()
> > +
> > +#define mtk_vcodec_debug(h, fmt, args...)
> > +#define mtk_vcodec_err(h, fmt, args...)
> > +#define mtk_vcodec_debug_enter(h)
> > +#define mtk_vcodec_debug_leave(h)
> > +
> > +#endif
> > +
> > +void __iomem *mtk_vcodec_get_reg_addr(void *data, unsigned int reg_idx);
> > +int mtk_vcodec_mem_alloc(void *data, struct mtk_vcodec_mem *mem);
> > +void mtk_vcodec_mem_free(void *data, struct mtk_vcodec_mem *mem);
> > +int mtk_vcodec_get_ctx_id(void *data);
> > +struct platform_device *mtk_vcodec_get_plat_dev(void *data);
> > +void mtk_vcodec_fmt2str(u32 fmt, char *str);
> > +#endif /* _MTK_VCODEC_UTIL_H_ */
> > diff --git a/drivers/media/platform/mtk-vcodec/venc_drv_base.h b/drivers/media/platform/mtk-vcodec/venc_drv_base.h
> > new file mode 100644
> > index 0000000..ed9cbf0
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/venc_drv_base.h
> > @@ -0,0 +1,62 @@
> > +/*
> > + * Copyright (c) 2015 MediaTek Inc.
> > + * Author: Daniel Hsiao <daniel.hsiao@mediatek.com>
> > + *         Jungchang Tsao <jungchang.tsao@mediatek.com>
> > + *
> > + * This program is free software; you can redistribute it and/or
> > + * modify
> > + * it under the terms of the GNU General Public License version 2 as
> > + * published by the Free Software Foundation.
> > + *
> > + * This program is distributed in the hope that it will be useful,
> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > + * GNU General Public License for more details.
> > + */
> > +
> > +#ifndef _VENC_DRV_BASE_
> > +#define _VENC_DRV_BASE_
> > +
> > +#include "mtk_vcodec_drv.h"
> > +
> > +#include "venc_drv_if.h"
> > +
> > +struct venc_common_if {
> > +	/**
> > +	 * (*init)() - initialize driver
> > +	 * @ctx:	[in] mtk v4l2 context
> > +	 * @handle: [out] driver handle
> > +	 */
> > +	int (*init)(struct mtk_vcodec_ctx *ctx, unsigned long *handle);
> > +
> > +	/**
> > +	 * (*encode)() - trigger encode
> > +	 * @handle: [in] driver handle
> > +	 * @opt: [in] encode option
> > +	 * @frm_buf: [in] frame buffer to store input frame
> > +	 * @bs_buf: [in] bitstream buffer to store output bitstream
> > +	 * @result: [out] encode result
> > +	 */
> > +	int (*encode)(unsigned long handle, enum venc_start_opt opt,
> > +		      struct venc_frm_buf *frm_buf,
> > +		      struct mtk_vcodec_mem *bs_buf,
> > +		      struct venc_done_result *result);
> > +
> > +	/**
> > +	 * (*set_param)() - set driver's parameter
> > +	 * @handle: [in] driver handle
> > +	 * @type: [in] parameter type
> > +	 * @in: [in] buffer to store the parameter
> > +	 */
> > +	int (*set_param)(unsigned long handle, enum venc_set_param_type type,
> > +			 void *in);
> > +
> > +	/**
> > +	 * (*deinit)() - deinitialize driver.
> > +	 * @handle: [in] driver handle
> > +	 */
> > +	int (*deinit)(unsigned long handle);
> > +};
> > +
> > +
> > +#endif
> > diff --git a/drivers/media/platform/mtk-vcodec/venc_drv_if.c b/drivers/media/platform/mtk-vcodec/venc_drv_if.c
> > new file mode 100644
> > index 0000000..daa8e93
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/venc_drv_if.c
> > @@ -0,0 +1,100 @@
> > +/*
> > + * Copyright (c) 2015 MediaTek Inc.
> > + * Author: Daniel Hsiao <daniel.hsiao@mediatek.com>
> > + *         Jungchang Tsao <jungchang.tsao@mediatek.com>
> > + *
> > + * This program is free software; you can redistribute it and/or
> > + * modify
> > + * it under the terms of the GNU General Public License version 2 as
> > + * published by the Free Software Foundation.
> > + *
> > + * This program is distributed in the hope that it will be useful,
> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > + * GNU General Public License for more details.
> > + */
> > +
> > +#include <linux/interrupt.h>
> > +#include <linux/kernel.h>
> > +#include <linux/slab.h>
> > +
> > +#include "venc_drv_if.h"
> > +#include "mtk_vcodec_enc.h"
> > +#include "mtk_vcodec_pm.h"
> > +#include "mtk_vpu.h"
> > +
> > +#include "venc_drv_base.h"
> > +
> > +int venc_if_create(struct mtk_vcodec_ctx *ctx, unsigned int fourcc)
> > +{
> > +	char str[10];
> > +	int ret = 0;
> > +
> > +	mtk_vcodec_fmt2str(fourcc, str);
> > +
> > +	switch (fourcc) {
> > +	case V4L2_PIX_FMT_VP8:
> > +	case V4L2_PIX_FMT_H264:
> > +	default:
> > +		return -EINVAL;
> > +	}
> > +
> > +	mtk_venc_lock(ctx);
> > +	mtk_vcodec_enc_clock_on(&ctx->dev->pm);
> > +	ret = ctx->enc_if->init(ctx, (unsigned long *)&ctx->drv_handle);
> > +	mtk_vcodec_enc_clock_off(&ctx->dev->pm);
> > +	mtk_venc_unlock(ctx);
> > +
> > +	return ret;
> > +
> > +}
> > +
> > +
> > +int venc_if_set_param(struct mtk_vcodec_ctx *ctx,
> > +		      enum venc_set_param_type type, void *in)
> > +{
> > +	int ret = 0;
> > +
> > +	mtk_venc_lock(ctx);
> > +	mtk_vcodec_enc_clock_on(&ctx->dev->pm);
> > +	ret = ctx->enc_if->set_param(ctx->drv_handle, type, in);
> > +	mtk_vcodec_enc_clock_off(&ctx->dev->pm);
> > +	mtk_venc_unlock(ctx);
> > +
> > +	return ret;
> > +}
> > +
> > +int venc_if_encode(struct mtk_vcodec_ctx *ctx,
> > +		   enum venc_start_opt opt, struct venc_frm_buf *frm_buf,
> > +		   struct mtk_vcodec_mem *bs_buf,
> > +		   struct venc_done_result *result)
> > +{
> > +	int ret = 0;
> > +	
> > +	mtk_venc_lock(ctx);
> > +	mtk_vcodec_enc_clock_on(&ctx->dev->pm);
> > +	ret = ctx->enc_if->encode(ctx->drv_handle, opt, frm_buf, bs_buf, result);
> > +	mtk_vcodec_enc_clock_off(&ctx->dev->pm);
> > +	mtk_venc_unlock(ctx);
> > +	
> > +	return ret;
> > +}
> > +
> > +int venc_if_release(struct mtk_vcodec_ctx *ctx)
> > +{
> > +	int ret = 0;
> > +
> > +	if(ctx->drv_handle == 0)
> > +		return 0;
> > +
> > +	mtk_venc_lock(ctx);
> > +	mtk_vcodec_enc_clock_on(&ctx->dev->pm);
> > +	ret = ctx->enc_if->deinit(ctx->drv_handle);
> > +	mtk_vcodec_enc_clock_off(&ctx->dev->pm);
> > +	mtk_venc_unlock(ctx);
> > +
> > +	ctx->drv_handle = 0;
> > +
> > +	return ret;
> > +}
> > +
> > diff --git a/drivers/media/platform/mtk-vcodec/venc_drv_if.h b/drivers/media/platform/mtk-vcodec/venc_drv_if.h
> > new file mode 100644
> > index 0000000..a387011
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/venc_drv_if.h
> > @@ -0,0 +1,175 @@
> > +/*
> > + * Copyright (c) 2015 MediaTek Inc.
> > + * Author: Daniel Hsiao <daniel.hsiao@mediatek.com>
> > + *         Jungchang Tsao <jungchang.tsao@mediatek.com>
> > + *
> > + * This program is free software; you can redistribute it and/or
> > + * modify
> > + * it under the terms of the GNU General Public License version 2 as
> > + * published by the Free Software Foundation.
> > + *
> > + * This program is distributed in the hope that it will be useful,
> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > + * GNU General Public License for more details.
> > + */
> > +
> > +#ifndef _VENC_DRV_IF_H_
> > +#define _VENC_DRV_IF_H_
> > +
> > +#include "mtk_vcodec_drv.h"
> > +#include "mtk_vcodec_util.h"
> > +
> > +/*
> > + * enum venc_yuv_fmt - The type of input yuv format
> > + * (VPU related: If you change the order, you must also update the VPU codes.)
> > + * @VENC_YUV_FORMAT_420: 420 YUV format
> > + * @VENC_YUV_FORMAT_YV12: YV12 YUV format
> > + * @VENC_YUV_FORMAT_NV12: NV12 YUV format
> > + * @VENC_YUV_FORMAT_NV21: NV21 YUV format
> > + */
> > +enum venc_yuv_fmt {
> > +	VENC_YUV_FORMAT_420 = 3,
> > +	VENC_YUV_FORMAT_YV12 = 5,
> > +	VENC_YUV_FORMAT_NV12 = 6,
> > +	VENC_YUV_FORMAT_NV21 = 7,
> > +};
> > +
> > +/*
> > + * enum venc_start_opt - encode frame option used in venc_if_encode()
> > + * @VENC_START_OPT_ENCODE_SEQUENCE_HEADER: encode SPS/PPS for H264
> > + * @VENC_START_OPT_ENCODE_FRAME: encode normal frame
> > + */
> > +enum venc_start_opt {
> > +	VENC_START_OPT_ENCODE_SEQUENCE_HEADER,
> > +	VENC_START_OPT_ENCODE_FRAME,
> > +};
> > +
> > +/*
> > + * enum venc_drv_msg - The type of encode frame status used in venc_if_encode()
> > + * @VENC_MESSAGE_OK: encode ok
> > + * @VENC_MESSAGE_ERR: encode error
> > + */
> > +enum venc_drv_msg {
> > +	VENC_MESSAGE_OK,
> > +	VENC_MESSAGE_ERR,
> > +};
> > +
> > +/*
> > + * enum venc_set_param_type - The type of set parameter used in venc_if_set_param()
> > + * (VPU related: If you change the order, you must also update the VPU codes.)
> > + * @VENC_SET_PARAM_ENC: set encoder parameters
> > + * @VENC_SET_PARAM_FORCE_INTRA: set force intra frame
> > + * @VENC_SET_PARAM_ADJUST_BITRATE: set to adjust bitrate (in bps)
> > + * @VENC_SET_PARAM_ADJUST_FRAMERATE: set frame rate
> > + * @VENC_SET_PARAM_I_FRAME_INTERVAL: set I frame interval
> > + * @VENC_SET_PARAM_SKIP_FRAME: set H264 skip one frame
> > + * @VENC_SET_PARAM_PREPEND_HEADER: set H264 prepend SPS/PPS before IDR
> > + * @VENC_SET_PARAM_TS_MODE: set VP8 temporal scalability mode
> > + */
> > +enum venc_set_param_type {
> > +	VENC_SET_PARAM_ENC,
> > +	VENC_SET_PARAM_FORCE_INTRA,
> > +	VENC_SET_PARAM_ADJUST_BITRATE,
> > +	VENC_SET_PARAM_ADJUST_FRAMERATE,
> > +	VENC_SET_PARAM_I_FRAME_INTERVAL,
> > +	VENC_SET_PARAM_SKIP_FRAME,
> > +	VENC_SET_PARAM_PREPEND_HEADER,
> > +	VENC_SET_PARAM_TS_MODE,
> > +};
> > +
> > +/*
> > + * struct venc_enc_prm - encoder settings for VENC_SET_PARAM_ENC used in venc_if_set_param()
> > + * @input_fourcc: input fourcc
> > + * @h264_profile: V4L2 defined H.264 profile
> > + * @h264_level: V4L2 defined H.264 level
> > + * @width: image width
> > + * @height: image height
> > + * @buf_width: buffer width
> > + * @buf_height: buffer height
> > + * @frm_rate: frame rate
> > + * @intra_period: intra frame period
> > + * @bitrate: target bitrate in kbps
> > + */
> > +struct venc_enc_prm {
> > +	enum venc_yuv_fmt input_fourcc;
> > +	unsigned int h264_profile;
> > +	unsigned int h264_level;
> > +	unsigned int width;
> > +	unsigned int height;
> > +	unsigned int buf_width;
> > +	unsigned int buf_height;
> > +	unsigned int frm_rate;
> > +	unsigned int intra_period;
> > +	unsigned int bitrate;
> > +	unsigned int sizeimage[MTK_VCODEC_MAX_PLANES];
> > +};
> > +
> > +/*
> > + * struct venc_frm_buf - frame buffer information used in venc_if_encode()
> > + * @fb_addr: plane 0 frame buffer address
> > + * @fb_addr1: plane 1 frame buffer address
> > + * @fb_addr2: plane 2 frame buffer address
> > + */
> > +struct venc_frm_buf {
> > +	struct mtk_vcodec_mem fb_addr;
> > +	struct mtk_vcodec_mem fb_addr1;
> > +	struct mtk_vcodec_mem fb_addr2;
> > +};
> > +
> > +/*
> > + * struct venc_done_result - This is return information used in venc_if_encode()
> > + * @msg: message, such as success or error code
> > + * @bs_size: output bitstream size
> > + * @is_key_frm: output is key frame or not
> > + */
> > +struct venc_done_result {
> > +	enum venc_drv_msg msg;
> > +	unsigned int bs_size;
> > +	bool is_key_frm;
> > +};
> > +
> > +/*
> > + * venc_if_create - Create the driver handle
> > + * @ctx: device context
> > + * @fourcc: encoder output format
> > + * @handle: driver handle
> > + * Return: 0 if creating handle successfully, otherwise it is failed.
> > + */
> > +int venc_if_create(struct mtk_vcodec_ctx *ctx, unsigned int fourcc);
> > +
> > +/*
> > + * venc_if_release - Release the driver handle
> > + * @handle: driver handle
> > + * Return: 0 if releasing handle successfully, otherwise it is failed.
> > + */
> > +int venc_if_release(struct mtk_vcodec_ctx *ctx);
> > +
> > +/*
> > + * venc_if_set_param - Set parameter to driver
> > + * @handle: driver handle
> > + * @type: set type
> > + * @in: input parameter
> > + * @out: output parameter
> > + * Return: 0 if setting param successfully, otherwise it is failed.
> > + */
> > +int venc_if_set_param(struct mtk_vcodec_ctx *ctx,
> > +		      enum venc_set_param_type type,
> > +		      void *in);
> > +
> > +/*
> > + * venc_if_encode - Encode frame
> > + * @handle: driver handle
> > + * @opt: encode frame option
> > + * @frm_buf: input frame buffer information
> > + * @bs_buf: output bitstream buffer infomraiton
> > + * @result: encode result
> > + * Return: 0 if encoding frame successfully, otherwise it is failed.
> > + */
> > +int venc_if_encode(struct mtk_vcodec_ctx *ctx,
> > +		   enum venc_start_opt opt,
> > +		   struct venc_frm_buf *frm_buf,
> > +		   struct mtk_vcodec_mem *bs_buf,
> > +		   struct venc_done_result *result);
> > +
> > +#endif /* _VENC_DRV_IF_H_ */
> > diff --git a/drivers/media/platform/mtk-vcodec/venc_ipi_msg.h b/drivers/media/platform/mtk-vcodec/venc_ipi_msg.h
> > new file mode 100644
> > index 0000000..a345b98
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/venc_ipi_msg.h
> > @@ -0,0 +1,212 @@
> > +/*
> > + * Copyright (c) 2015 MediaTek Inc.
> > + * Author: Jungchang Tsao <jungchang.tsao@mediatek.com>
> > + *         Daniel Hsiao <daniel.hsiao@mediatek.com>
> > + *
> > + * This program is free software; you can redistribute it and/or
> > + * modify
> > + * it under the terms of the GNU General Public License version 2 as
> > + * published by the Free Software Foundation.
> > + *
> > + * This program is distributed in the hope that it will be useful,
> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > + * GNU General Public License for more details.
> > + */
> > +
> > +#ifndef _VENC_IPI_MSG_H_
> > +#define _VENC_IPI_MSG_H_
> > +
> > +#define IPIMSG_H264_ENC_ID 0x100
> > +#define IPIMSG_VP8_ENC_ID 0x200
> > +
> > +#define AP_IPIMSG_VENC_BASE 0x20000
> > +#define VPU_IPIMSG_VENC_BASE 0x30000
> > +
> > +/**
> > + * enum venc_ipi_msg_id - message id between AP and VPU
> > + * (ipi stands for inter-processor interrupt)
> > + * @AP_IPIMSG_XXX:		AP to VPU cmd message id
> > + * @VPU_IPIMSG_XXX_DONE:	VPU ack AP cmd message id
> > + */
> > +enum venc_ipi_msg_id {
> > +	AP_IPIMSG_H264_ENC_INIT = AP_IPIMSG_VENC_BASE +
> > +				  IPIMSG_H264_ENC_ID,
> > +	AP_IPIMSG_H264_ENC_SET_PARAM,
> > +	AP_IPIMSG_H264_ENC_ENCODE,
> > +	AP_IPIMSG_H264_ENC_DEINIT,
> > +
> > +	AP_IPIMSG_VP8_ENC_INIT = AP_IPIMSG_VENC_BASE +
> > +				 IPIMSG_VP8_ENC_ID,
> > +	AP_IPIMSG_VP8_ENC_SET_PARAM,
> > +	AP_IPIMSG_VP8_ENC_ENCODE,
> > +	AP_IPIMSG_VP8_ENC_DEINIT,
> > +
> > +	VPU_IPIMSG_H264_ENC_INIT_DONE = VPU_IPIMSG_VENC_BASE +
> > +					IPIMSG_H264_ENC_ID,
> > +	VPU_IPIMSG_H264_ENC_SET_PARAM_DONE,
> > +	VPU_IPIMSG_H264_ENC_ENCODE_DONE,
> > +	VPU_IPIMSG_H264_ENC_DEINIT_DONE,
> > +
> > +	VPU_IPIMSG_VP8_ENC_INIT_DONE = VPU_IPIMSG_VENC_BASE +
> > +				       IPIMSG_VP8_ENC_ID,
> > +	VPU_IPIMSG_VP8_ENC_SET_PARAM_DONE,
> > +	VPU_IPIMSG_VP8_ENC_ENCODE_DONE,
> > +	VPU_IPIMSG_VP8_ENC_DEINIT_DONE,
> > +};
> > +
> > +/**
> > + * struct venc_ap_ipi_msg_init - AP to VPU init cmd structure
> > + * @msg_id:	message id (AP_IPIMSG_XXX_ENC_INIT)
> > + * @venc_inst:	AP encoder instance (struct venc_vp8_handle/venc_h264_handle *)
> > + */
> > +struct venc_ap_ipi_msg_init {
> > +	uint32_t msg_id;
> > +	uint32_t reserved;
> > +	uint64_t venc_inst;
> > +};
> > +
> > +/**
> > + * struct venc_ap_ipi_msg_set_param - AP to VPU set_param cmd structure
> > + * @msg_id:	message id (AP_IPIMSG_XXX_ENC_SET_PARAM)
> > + * @inst_id:	VPU encoder instance id (struct venc_vp8_vpu_drv/venc_h264_vpu_drv *)
> > + * @param_id:	parameter id (venc_set_param_type)
> > + * @data_item:	number of items in the data array
> > + * @data[8]:	data array to store the set parameters
> > + */
> > +struct venc_ap_ipi_msg_set_param {
> > +	uint32_t msg_id;
> > +	uint32_t inst_id;
> > +	uint32_t param_id;
> > +	uint32_t data_item;
> > +	uint32_t data[8];
> > +};
> > +
> > +/**
> > + * struct venc_ap_ipi_msg_enc - AP to VPU enc cmd structure
> > + * @msg_id:	message id (AP_IPIMSG_XXX_ENC_ENCODE)
> > + * @inst_id:	VPU encoder instance id (struct venc_vp8_vpu_drv/venc_h264_vpu_drv *)
> > + * @bs_mode:	bitstream mode for h264
> > + *		(H264_BS_MODE_SPS/H264_BS_MODE_PPS/H264_BS_MODE_FRAME)
> > + * @input_addr:	pointer to input image buffer plane
> > + * @bs_addr:	pointer to output bit stream buffer
> > + * @bs_size:	bit stream buffer size
> > + */
> > +struct venc_ap_ipi_msg_enc {
> > +	uint32_t msg_id;
> > +	uint32_t inst_id;
> > +	uint32_t bs_mode;
> > +	uint32_t input_addr[3];
> > +	uint32_t bs_addr;
> > +	uint32_t bs_size;
> > +};
> > +
> > +/**
> > + * struct venc_ap_ipi_msg_deinit - AP to VPU deinit cmd structure
> > + * @msg_id:	message id (AP_IPIMSG_XXX_ENC_DEINIT)
> > + * @inst_id:	VPU encoder instance id (struct venc_vp8_vpu_drv/venc_h264_vpu_drv *)
> > + */
> > +struct venc_ap_ipi_msg_deinit {
> > +	uint32_t msg_id;
> > +	uint32_t inst_id;
> > +};
> > +
> > +/**
> > + * enum venc_ipi_msg_status - VPU ack AP cmd status
> > + */
> > +enum venc_ipi_msg_status {
> > +	VENC_IPI_MSG_STATUS_OK,
> > +	VENC_IPI_MSG_STATUS_FAIL,
> > +};
> > +
> > +/**
> > + * struct venc_vpu_ipi_msg_common - VPU ack AP cmd common structure
> > + * @msg_id:	message id (VPU_IPIMSG_XXX_DONE)
> > + * @status:	cmd status (venc_ipi_msg_status)
> > + * @venc_inst:	AP encoder instance (struct venc_vp8_handle/venc_h264_handle *)
> > + */
> > +struct venc_vpu_ipi_msg_common {
> > +	uint32_t msg_id;
> > +	uint32_t status;
> > +	uint64_t venc_inst;
> > +};
> > +
> > +/**
> > + * struct venc_vpu_ipi_msg_init - VPU ack AP init cmd structure
> > + * @msg_id:	message id (VPU_IPIMSG_XXX_ENC_SET_PARAM_DONE)
> > + * @status:	cmd status (venc_ipi_msg_status)
> > + * @venc_inst:	AP encoder instance (struct venc_vp8_handle/venc_h264_handle *)
> > + * @inst_id:	VPU encoder instance id (struct venc_vp8_vpu_drv/venc_h264_vpu_drv *)
> > + */
> > +struct venc_vpu_ipi_msg_init {
> > +	uint32_t msg_id;
> > +	uint32_t status;
> > +	uint64_t venc_inst;
> > +	uint32_t inst_id;
> > +	uint32_t reserved;
> > +};
> > +
> > +/**
> > + * struct venc_vpu_ipi_msg_set_param - VPU ack AP set_param cmd structure
> > + * @msg_id:	message id (VPU_IPIMSG_XXX_ENC_SET_PARAM_DONE)
> > + * @status:	cmd status (venc_ipi_msg_status)
> > + * @venc_inst:	AP encoder instance (struct venc_vp8_handle/venc_h264_handle *)
> > + * @param_id:	parameter id (venc_set_param_type)
> > + * @data_item:	number of items in the data array
> > + * @data[6]:	data array to store the return result
> > + */
> > +struct venc_vpu_ipi_msg_set_param {
> > +	uint32_t msg_id;
> > +	uint32_t status;
> > +	uint64_t venc_inst;
> > +	uint32_t param_id;
> > +	uint32_t data_item;
> > +	uint32_t data[6];
> > +};
> > +
> > +/**
> > + * enum venc_ipi_msg_enc_state - Type of encode state
> > + * VEN_IPI_MSG_ENC_STATE_FRAME:	one frame being encoded
> > + * VEN_IPI_MSG_ENC_STATE_PART:	bit stream buffer full
> > + * VEN_IPI_MSG_ENC_STATE_SKIP:	encoded skip frame
> > + * VEN_IPI_MSG_ENC_STATE_ERROR:	encounter error
> > + */
> > +enum venc_ipi_msg_enc_state {
> > +	VEN_IPI_MSG_ENC_STATE_FRAME,
> > +	VEN_IPI_MSG_ENC_STATE_PART,
> > +	VEN_IPI_MSG_ENC_STATE_SKIP,
> > +	VEN_IPI_MSG_ENC_STATE_ERROR,
> > +};
> > +
> > +/**
> > + * struct venc_vpu_ipi_msg_enc - VPU ack AP enc cmd structure
> > + * @msg_id:	message id (VPU_IPIMSG_XXX_ENC_ENCODE_DONE)
> > + * @status:	cmd status (venc_ipi_msg_status)
> > + * @venc_inst:	AP encoder instance (struct venc_vp8_handle/venc_h264_handle *)
> > + * @state:	encode state (venc_ipi_msg_enc_state)
> > + * @key_frame:	whether the encoded frame is key frame
> > + * @bs_size:	encoded bitstream size
> 
> 'reserved' isn't documented. I assume this should be set to 0?
> 
Yes. We will fix this.

> > + */
> > +struct venc_vpu_ipi_msg_enc {
> > +	uint32_t msg_id;
> > +	uint32_t status;
> > +	uint64_t venc_inst;
> > +	uint32_t state;
> > +	uint32_t key_frame;
> > +	uint32_t bs_size;
> > +	uint32_t reserved;
> > +};
> > +
> > +/**
> > + * struct venc_vpu_ipi_msg_deinit - VPU ack AP deinit cmd structure
> > + * @msg_id:   message id (VPU_IPIMSG_XXX_ENC_DEINIT_DONE)
> > + * @status:   cmd status (venc_ipi_msg_status)
> > + * @venc_inst:	AP encoder instance (struct venc_vp8_handle/venc_h264_handle *)
> > + */
> > +struct venc_vpu_ipi_msg_deinit {
> > +	uint32_t msg_id;
> > +	uint32_t status;
> > +	uint64_t venc_inst;
> > +};
> > +
> > +#endif /* _VENC_IPI_MSG_H_ */
> > diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
> > old mode 100644
> > new mode 100755
> > index ee9d530..3ac35c4
> > --- a/include/uapi/linux/v4l2-controls.h
> > +++ b/include/uapi/linux/v4l2-controls.h
> > @@ -646,6 +646,10 @@ enum v4l2_mpeg_mfc51_video_force_frame_type {
> >  #define V4L2_CID_MPEG_MFC51_VIDEO_H264_NUM_REF_PIC_FOR_P		(V4L2_CID_MPEG_MFC51_BASE+54)
> >  
> >  
> > +#define V4L2_CID_MPEG_MTK_BASE					(V4L2_CTRL_CLASS_MPEG | 0x5500)
> > +#define V4L2_CID_MPEG_MTK_VIDEO_FRAME_SKIP_ENABLE		(V4L2_CID_MPEG_MTK_BASE+0)
> > +#define V4L2_CID_MPEG_MTK_VIDEO_FORCE_FRAME_TYPE_I_FRAME	(V4L2_CID_MPEG_MTK_BASE+1)
> 
> Please note that these controls should be documented in DocBook.
> 
Sorry, I do not clean these part.
We plain to remove V4L2_CID_MPEG_MTK_VIDEO_FRAME_SKIP_ENABL and use
V4L2_CID_MPEG_VIDEO_FORCE_I_FRAME in wucheng's patch
https://patchwork.linuxtv.org/patch/32670/.


best regards,
Tiffany

> Regards,
> 
> 	Hans
> 
> > +
> >  /*  Camera class control IDs */
> >  
> >  #define V4L2_CID_CAMERA_CLASS_BASE 	(V4L2_CTRL_CLASS_CAMERA | 0x900)
> > 
> 

^ permalink raw reply	[flat|nested] 102+ messages in thread

* Re: [PATCH v4 0/8] Add MT8173 Video Encoder Driver and VPU Driver
@ 2016-02-16  6:46     ` tiffany lin
  0 siblings, 0 replies; 102+ messages in thread
From: tiffany lin @ 2016-02-16  6:46 UTC (permalink / raw)
  To: Hans Verkuil
  Cc: Hans Verkuil, daniel.thompson, Rob Herring,
	Mauro Carvalho Chehab, Matthias Brugger, Daniel Kurtz,
	Pawel Osciak, Eddie Huang, Yingjoe Chen, devicetree,
	linux-kernel, linux-arm-kernel, linux-media, linux-mediatek,
	PoChun.Lin

Hi Hans,

On Mon, 2016-02-15 at 13:04 +0100, Hans Verkuil wrote:
> Hi Tiffany,
> 
> Once again, thanks for your patch series!
> 
> I have some more comments below.
> 
> On 02/04/2016 12:34 PM, Tiffany Lin wrote:
> > 
> > ==============
> >  Introduction
> > ==============
> > 
> > The purpose of this series is to add the driver for video codec hw embedded in the Mediatek's MT8173 SoCs.
> > Mediatek Video Codec is able to handle video encoding of in a range of formats.
> > 
> > This patch series also include VPU driver. Mediatek Video Codec driver rely on VPU driver to load, communicate with VPU.
> > 
> > Internally the driver uses videobuf2 framework and MTK IOMMU and MTK SMI.
> > MTK IOMMU[1] and MTK SMI[2] have not yet been merged, but we wanted to start discussion about the driver earlier so it could be merged sooner.
> > 
> > [1]https://patchwork.kernel.org/patch/8120921/
> > [2]https://patchwork.kernel.org/patch/8120941/
> > 
> > 
> > ==================
> >  Device interface
> > ==================
> > 
> > In principle the driver bases on v4l2 memory-to-memory framework:
> > it provides a single video node and each opened file handle gets its own private context with separate buffer queues. Each context consist of 2 buffer queues: OUTPUT (for source buffers, i.e. raw video frames) and CAPTURE (for destination buffers, i.e. encoded video frames).
> > 
> > ==============================
> >  VPU (Video Processor Unit)
> > ==============================
> > The VPU driver for hw video codec embedded in Mediatek's MT8173 SOCs.
> > It is able to handle video decoding/encoding in a range of formats.
> > The driver provides with VPU firmware download, memory management and the communication interface between CPU and VPU.
> > For VPU initialization, it will create virtual memory for CPU access and physical address for VPU hw device access. When a decode/encode instance opens a device node, vpu driver will download vpu firmware to the device.
> > A decode/encode instant will decode/encode a frame using VPU interface to interrupt vpu to handle decoding/encoding jobs.
> > 
> > Please have a look at the code and comments will be very much appreciated.
> > 
> > 
> > Change in v4:
> > Vcodec Part
> > 1. Remove MTK_ENCODE_PARAM_SKIP_FRAME support
> > 2. Remove MTK_ENCODE_PARAM_FRAME_TYPE and change to use V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME[3]
> > 3. Refine Encoder HW clock source
> > 4. Refine debug log
> > 5. Add watchdog support
> > 6. With patch "media: v4l2-compat-ioctl32: fix missing length copy in put_v4l2_buffer32"[4], v4l2-compliance test passed[5] in v4.4-rc5 
> > 
> > VPU Part
> > 1. These two patches were Acked-by: Rob Herring <robh@kernel.org> in v3
> >    [PATCH v3 1/8] dt-bindings: Add a binding for Mediatek Video Processor
> >    [PATCH v3 3/8] arm64: dts: mediatek: Add node for Mediatek Video Processor Unit
> >    Because we were wrong about how the hardware works, there is no connection between VPU and IOMMU HW
> >    We remove VPU attaching to IOMMU
> > 2. Support VPU running on 4GB DRAM system
> > 3. Support VPU watchdog reset
> > 4. Refine for coding style 
> > 
> > [3]https://patchwork.linuxtv.org/patch/32670/
> > [4] https://patchwork.linuxtv.org/patch/32631/
> > [5]localhost ~ # /usr/bin/v4l2-compliance -d /dev/video1
> > Driver Info:
> >         Driver name   : mtk-vcodec-en
> >         Card type     : platform:mt817
> >         Bus info      : platform:mt817
> 
> Perhaps this should be mt817x?
> 
> >         Driver version: 4.4.0
> >         Capabilities  : 0x84204000
> >                 Video Memory-to-Memory Multiplanar
> >                 Streaming
> >                 Extended Pix Format
> >                 Device Capabilities
> >         Device Caps   : 0x04204000
> >                 Video Memory-to-Memory Multiplanar
> >                 Streaming
> >                 Extended Pix Format
> > 
> > Compliance test for device /dev/video1 (not using libv4l2):
> > 
> > Required ioctls:
> >         test VIDIOC_QUERYCAP: OK
> > 
> > Allow for multiple opens:
> >         test second video open: OK
> >         test VIDIOC_QUERYCAP: OK
> >         test VIDIOC_G/S_PRIORITY: OK
> > 
> > Debug ioctls:
> >         test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> >         test VIDIOC_LOG_STATUS: OK (Not Supported)
> > 
> > Input ioctls:
> >         test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> >         test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >         test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> >         test VIDIOC_ENUMAUDIO: OK (Not Supported)
> >         test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> >         test VIDIOC_G/S_AUDIO: OK (Not Supported)
> >         Inputs: 0 Audio Inputs: 0 Tuners: 0
> > 
> > Output ioctls:
> >         test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> >         test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >         test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> >         test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> >         test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> >         Outputs: 0 Audio Outputs: 0 Modulators: 0
> > 
> > Input/Output configuration ioctls:
> >         test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> >         test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> >         test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> >         test VIDIOC_G/S_EDID: OK (Not Supported)
> > 
> >         Control ioctls:
> >                 test VIDIOC_QUERYCTRL/MENU: OK
> >                 test VIDIOC_G/S_CTRL: OK
> >                 test VIDIOC_G/S/TRY_EXT_CTRLS: OK
> >                 test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
> >                 test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> >                 Standard Controls: 12 Private Controls: 0
> > 
> >         Format ioctls:
> >                 test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> >                 test VIDIOC_G/S_PARM: OK (Not Supported)
> >                 test VIDIOC_G_FBUF: OK (Not Supported)
> >                 test VIDIOC_G_FMT: OK
> >                 test VIDIOC_TRY_FMT: OK
> >                 test VIDIOC_S_FMT: OK
> >                 test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> > 
> >         Codec ioctls:
> >                 test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> >                 test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> >                 test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> > 
> >         Buffer ioctls:
> >                 warn: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-buffers.cpp(475): VIDIOC_CREATE_BUFS not supported
> >                 warn: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-buffers.cpp(475): VIDIOC_CREATE_BUFS not supported
> >                 test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
> >                 test VIDIOC_EXPBUF: OK
> > 
> > Total: 38, Succeeded: 38, Failed: 0, Warnings: 2
> 
> Looks great!
> 
> Having reviewed the v4 patch series I'd say it is close to being ready (at least
> w.r.t. the V4L2 parts). Most of my comments are minor, but fixing the open()
> limitation is probably a bigger change on your side.
> 
> It should be well worth the effort, though. In my experience limiting the number
> of time you can call open() is always a sign of a design flaw and the code improves
> if this is handled in the right place (usually start_streaming, which is when you
> actually start to use the hardware).
> 
> And besides, limiting the number of open()s is out-of-spec.
> 
You are right. We did not think about this before.
We will remove open() limitation and put limitation control in right
place.
Appreciated your time and suggestions.



> Regards,
> 
> 	Hans
> 
> > 
> > 
> > Change in v3:
> > 1.Refine code to pass v4l2-compliance test, now it still has 2 issues 2.Refine code according to latest MTK IOMMU patches[1] 3.Remove MFC51 specific CIDs and add MTK specific CIDs for for keyframe and
> >   skip I-frame
> > 4.Refine code according to review comments
> > 
> > Below is the v1.6 version v4l2-compliance report for the mt8173 encoder driver.
> > Now there are still 2 test fail in v1.6.
> > For VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF, we directly use v4l2_m2m_ioctl_* functions, but it still fail. It pass in kernel 3.18 but fail in kernel 4.4.
> > We will try v1.8 in next version.
> > VIDIOC_EXPBUF is becuase we support all three memory types VB2_DMABUF, VB2_MMAP and VB2_USERPTR. VIDIOC_EXPBUF only allowed when only VB2_MMAP supported.
> > localhost ~ # /usr/bin/v4l2-compliance -d /dev/video1 Driver Info:
> >         Driver name   : mtk-vcodec-en
> >         Card type     : platform:mt817
> >         Bus info      : platform:mt817
> >         Driver version: 4.4.0
> >         Capabilities  : 0x84204000
> >                 Video Memory-to-Memory Multiplanar
> >                 Streaming
> >                 Extended Pix Format
> >                 Device Capabilities
> >         Device Caps   : 0x04204000
> >                 Video Memory-to-Memory Multiplanar
> >                 Streaming
> >                 Extended Pix Format
> > 
> > Compliance test for device /dev/video1 (not using libv4l2):
> > 
> > Required ioctls:
> >         test VIDIOC_QUERYCAP: OK
> > 
> > Allow for multiple opens:
> >         test second video open: OK
> >         test VIDIOC_QUERYCAP: OK
> >         test VIDIOC_G/S_PRIORITY: OK
> > 
> > Debug ioctls:
> >         test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> >         test VIDIOC_LOG_STATUS: OK (Not Supported)
> > 
> > Input ioctls:
> >         test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> >         test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >         test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> >         test VIDIOC_ENUMAUDIO: OK (Not Supported)
> >         test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> >         test VIDIOC_G/S_AUDIO: OK (Not Supported)
> >         Inputs: 0 Audio Inputs: 0 Tuners: 0
> > 
> > Output ioctls:
> >         test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> >         test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >         test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> >         test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> >         test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> >         Outputs: 0 Audio Outputs: 0 Modulators: 0
> > 
> > Input/Output configuration ioctls:
> >         test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> >         test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> >         test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> >         test VIDIOC_G/S_EDID: OK (Not Supported)
> > 
> >         Control ioctls:
> >                 test VIDIOC_QUERYCTRL/MENU: OK
> >                 test VIDIOC_G/S_CTRL: OK
> >                 test VIDIOC_G/S/TRY_EXT_CTRLS: OK
> >                 test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
> >                 test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> >                 Standard Controls: 11 Private Controls: 2
> > 
> >         Format ioctls:
> >                 test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> >                 test VIDIOC_G/S_PARM: OK (Not Supported)
> >                 test VIDIOC_G_FBUF: OK (Not Supported)
> >                 test VIDIOC_G_FMT: OK
> >                 test VIDIOC_TRY_FMT: OK
> >                 test VIDIOC_S_FMT: OK
> >                 test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> > 
> >         Codec ioctls:
> >                 test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> >                 test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> >                 test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> > 
> >         Buffer ioctls:
> >                 fail: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-buffers.cpp(266): vp->length == 0
> >                 fail: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-buffers.cpp(335): buf.check(Unqueued, i)
> >                 fail: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-buffers.cpp(420): testQueryBuf(node, i, q.g_buffers())
> >                 test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
> >                 fail: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-buffers.cpp(500): q.has_expbuf(node)
> >                 test VIDIOC_EXPBUF: FAIL
> > Total: 38, Succeeded: 36, Failed: 2, Warnings: 0
> > 
> > 
> > Change in v2:
> > Vcodec Part
> > 1.Remove common and include directory in mtk-vcodec 2.Refine vb2ops_venc_start_streaming and vb2ops_venc_stop_streaming and state machine 3.Remove venc_if_init and venc_if_deinit 4.Refine debug message 5.Refine lab and vpu decription in mediatek-vcodec.txt
> > 
> > VPU Part
> > 1. Modify VPU Kconfig
> > 2. Move encoder header files to other patch sets 3. Remove marcos for extended virtual/iova address 4. Change register and variable names 5. Add a reference counter for VPU watchdog 6. Remove one busy waiting in function vpu_ipi_send 7. Operate VPU clock in VPU driver (not called by encoder drivers) 8. Refine memory mapping, firmware download and extended memory allocation/free functions 9. Release more allocated resources in driver remove function
> > 
> > 
> > 
> > Andrew-CT Chen (2):
> >   dt-bindings: Add a binding for Mediatek Video Processor
> >   [Media] vcodec: mediatek: Add Mediatek V4L2 Video Encoder Driver
> > 
> > Tiffany Lin (6):
> >   [media] VPU: mediatek: support Mediatek VPU
> >   arm64: dts: mediatek: Add node for Mediatek Video Processor Unit
> >   dt-bindings: Add a binding for Mediatek Video Encoder
> >   [media] vcodec: mediatek: Add Mediatek VP8 Video Encoder Driver
> >   [media] vcodec: mediatek: Add Mediatek H264 Video Encoder Driver
> >   arm64: dts: mediatek: Add Video Encoder for MT8173
> > 
> >  .../devicetree/bindings/media/mediatek-vcodec.txt  |   59 +
> >  .../devicetree/bindings/media/mediatek-vpu.txt     |   31 +
> >  arch/arm64/boot/dts/mediatek/mt8173.dtsi           |   62 +
> >  drivers/media/platform/Kconfig                     |   20 +
> >  drivers/media/platform/Makefile                    |    4 +
> >  drivers/media/platform/mtk-vcodec/Makefile         |   11 +
> >  .../media/platform/mtk-vcodec/h264_enc/Makefile    |    6 +
> >  .../platform/mtk-vcodec/h264_enc/venc_h264_if.c    |  540 ++++++++
> >  .../platform/mtk-vcodec/h264_enc/venc_h264_if.h    |  165 +++
> >  .../platform/mtk-vcodec/h264_enc/venc_h264_vpu.c   |  309 +++++
> >  .../platform/mtk-vcodec/h264_enc/venc_h264_vpu.h   |   30 +
> >  drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h |  388 ++++++
> >  drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c | 1380 ++++++++++++++++++++
> >  drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h |   46 +
> >  .../media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c |  476 +++++++
> >  .../media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c  |  132 ++
> >  .../media/platform/mtk-vcodec/mtk_vcodec_intr.c    |  102 ++
> >  .../media/platform/mtk-vcodec/mtk_vcodec_intr.h    |   29 +
> >  drivers/media/platform/mtk-vcodec/mtk_vcodec_pm.h  |   26 +
> >  .../media/platform/mtk-vcodec/mtk_vcodec_util.c    |  106 ++
> >  .../media/platform/mtk-vcodec/mtk_vcodec_util.h    |   85 ++
> >  drivers/media/platform/mtk-vcodec/venc_drv_base.h  |   62 +
> >  drivers/media/platform/mtk-vcodec/venc_drv_if.c    |  106 ++
> >  drivers/media/platform/mtk-vcodec/venc_drv_if.h    |  175 +++
> >  drivers/media/platform/mtk-vcodec/venc_ipi_msg.h   |  212 +++
> >  drivers/media/platform/mtk-vcodec/vp8_enc/Makefile |    6 +
> >  .../platform/mtk-vcodec/vp8_enc/venc_vp8_if.c      |  422 ++++++
> >  .../platform/mtk-vcodec/vp8_enc/venc_vp8_if.h      |  149 +++
> >  .../platform/mtk-vcodec/vp8_enc/venc_vp8_vpu.c     |  240 ++++
> >  .../platform/mtk-vcodec/vp8_enc/venc_vp8_vpu.h     |   28 +
> >  drivers/media/platform/mtk-vpu/Makefile            |    1 +
> >  drivers/media/platform/mtk-vpu/mtk_vpu.c           |  994 ++++++++++++++
> >  drivers/media/platform/mtk-vpu/mtk_vpu.h           |  167 +++
> >  include/uapi/linux/v4l2-controls.h                 |    4 +
> >  34 files changed, 6573 insertions(+)
> >  create mode 100644 Documentation/devicetree/bindings/media/mediatek-vcodec.txt
> >  create mode 100644 Documentation/devicetree/bindings/media/mediatek-vpu.txt
> >  create mode 100644 drivers/media/platform/mtk-vcodec/Makefile
> >  create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/Makefile
> >  create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.c
> >  create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.h
> >  create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.c
> >  create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.h
> >  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
> >  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
> >  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h
> >  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
> >  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
> >  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.c
> >  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.h
> >  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_pm.h
> >  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c
> >  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
> >  create mode 100644 drivers/media/platform/mtk-vcodec/venc_drv_base.h
> >  create mode 100644 drivers/media/platform/mtk-vcodec/venc_drv_if.c
> >  create mode 100644 drivers/media/platform/mtk-vcodec/venc_drv_if.h
> >  create mode 100644 drivers/media/platform/mtk-vcodec/venc_ipi_msg.h
> >  create mode 100644 drivers/media/platform/mtk-vcodec/vp8_enc/Makefile
> >  create mode 100644 drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_if.c
> >  create mode 100644 drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_if.h
> >  create mode 100644 drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_vpu.c
> >  create mode 100644 drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_vpu.h
> >  create mode 100644 drivers/media/platform/mtk-vpu/Makefile
> >  create mode 100644 drivers/media/platform/mtk-vpu/mtk_vpu.c
> >  create mode 100644 drivers/media/platform/mtk-vpu/mtk_vpu.h
> >  mode change 100644 => 100755 include/uapi/linux/v4l2-controls.h
> > 
> 

^ permalink raw reply	[flat|nested] 102+ messages in thread

* Re: [PATCH v4 0/8] Add MT8173 Video Encoder Driver and VPU Driver
@ 2016-02-16  6:46     ` tiffany lin
  0 siblings, 0 replies; 102+ messages in thread
From: tiffany lin @ 2016-02-16  6:46 UTC (permalink / raw)
  To: Hans Verkuil
  Cc: Hans Verkuil, daniel.thompson-QSEj5FYQhm4dnm+yROfE0A,
	Rob Herring, Mauro Carvalho Chehab, Matthias Brugger,
	Daniel Kurtz, Pawel Osciak, Eddie Huang, Yingjoe Chen,
	devicetree-u79uwXL29TY76Z2rM5mHXA,
	linux-kernel-u79uwXL29TY76Z2rM5mHXA,
	linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
	linux-media-u79uwXL29TY76Z2rM5mHXA,
	linux-mediatek-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
	PoChun.Lin-NuS5LvNUpcJWk0Htik3J/w

Hi Hans,

On Mon, 2016-02-15 at 13:04 +0100, Hans Verkuil wrote:
> Hi Tiffany,
> 
> Once again, thanks for your patch series!
> 
> I have some more comments below.
> 
> On 02/04/2016 12:34 PM, Tiffany Lin wrote:
> > 
> > ==============
> >  Introduction
> > ==============
> > 
> > The purpose of this series is to add the driver for video codec hw embedded in the Mediatek's MT8173 SoCs.
> > Mediatek Video Codec is able to handle video encoding of in a range of formats.
> > 
> > This patch series also include VPU driver. Mediatek Video Codec driver rely on VPU driver to load, communicate with VPU.
> > 
> > Internally the driver uses videobuf2 framework and MTK IOMMU and MTK SMI.
> > MTK IOMMU[1] and MTK SMI[2] have not yet been merged, but we wanted to start discussion about the driver earlier so it could be merged sooner.
> > 
> > [1]https://patchwork.kernel.org/patch/8120921/
> > [2]https://patchwork.kernel.org/patch/8120941/
> > 
> > 
> > ==================
> >  Device interface
> > ==================
> > 
> > In principle the driver bases on v4l2 memory-to-memory framework:
> > it provides a single video node and each opened file handle gets its own private context with separate buffer queues. Each context consist of 2 buffer queues: OUTPUT (for source buffers, i.e. raw video frames) and CAPTURE (for destination buffers, i.e. encoded video frames).
> > 
> > ==============================
> >  VPU (Video Processor Unit)
> > ==============================
> > The VPU driver for hw video codec embedded in Mediatek's MT8173 SOCs.
> > It is able to handle video decoding/encoding in a range of formats.
> > The driver provides with VPU firmware download, memory management and the communication interface between CPU and VPU.
> > For VPU initialization, it will create virtual memory for CPU access and physical address for VPU hw device access. When a decode/encode instance opens a device node, vpu driver will download vpu firmware to the device.
> > A decode/encode instant will decode/encode a frame using VPU interface to interrupt vpu to handle decoding/encoding jobs.
> > 
> > Please have a look at the code and comments will be very much appreciated.
> > 
> > 
> > Change in v4:
> > Vcodec Part
> > 1. Remove MTK_ENCODE_PARAM_SKIP_FRAME support
> > 2. Remove MTK_ENCODE_PARAM_FRAME_TYPE and change to use V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME[3]
> > 3. Refine Encoder HW clock source
> > 4. Refine debug log
> > 5. Add watchdog support
> > 6. With patch "media: v4l2-compat-ioctl32: fix missing length copy in put_v4l2_buffer32"[4], v4l2-compliance test passed[5] in v4.4-rc5 
> > 
> > VPU Part
> > 1. These two patches were Acked-by: Rob Herring <robh-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org> in v3
> >    [PATCH v3 1/8] dt-bindings: Add a binding for Mediatek Video Processor
> >    [PATCH v3 3/8] arm64: dts: mediatek: Add node for Mediatek Video Processor Unit
> >    Because we were wrong about how the hardware works, there is no connection between VPU and IOMMU HW
> >    We remove VPU attaching to IOMMU
> > 2. Support VPU running on 4GB DRAM system
> > 3. Support VPU watchdog reset
> > 4. Refine for coding style 
> > 
> > [3]https://patchwork.linuxtv.org/patch/32670/
> > [4] https://patchwork.linuxtv.org/patch/32631/
> > [5]localhost ~ # /usr/bin/v4l2-compliance -d /dev/video1
> > Driver Info:
> >         Driver name   : mtk-vcodec-en
> >         Card type     : platform:mt817
> >         Bus info      : platform:mt817
> 
> Perhaps this should be mt817x?
> 
> >         Driver version: 4.4.0
> >         Capabilities  : 0x84204000
> >                 Video Memory-to-Memory Multiplanar
> >                 Streaming
> >                 Extended Pix Format
> >                 Device Capabilities
> >         Device Caps   : 0x04204000
> >                 Video Memory-to-Memory Multiplanar
> >                 Streaming
> >                 Extended Pix Format
> > 
> > Compliance test for device /dev/video1 (not using libv4l2):
> > 
> > Required ioctls:
> >         test VIDIOC_QUERYCAP: OK
> > 
> > Allow for multiple opens:
> >         test second video open: OK
> >         test VIDIOC_QUERYCAP: OK
> >         test VIDIOC_G/S_PRIORITY: OK
> > 
> > Debug ioctls:
> >         test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> >         test VIDIOC_LOG_STATUS: OK (Not Supported)
> > 
> > Input ioctls:
> >         test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> >         test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >         test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> >         test VIDIOC_ENUMAUDIO: OK (Not Supported)
> >         test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> >         test VIDIOC_G/S_AUDIO: OK (Not Supported)
> >         Inputs: 0 Audio Inputs: 0 Tuners: 0
> > 
> > Output ioctls:
> >         test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> >         test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >         test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> >         test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> >         test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> >         Outputs: 0 Audio Outputs: 0 Modulators: 0
> > 
> > Input/Output configuration ioctls:
> >         test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> >         test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> >         test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> >         test VIDIOC_G/S_EDID: OK (Not Supported)
> > 
> >         Control ioctls:
> >                 test VIDIOC_QUERYCTRL/MENU: OK
> >                 test VIDIOC_G/S_CTRL: OK
> >                 test VIDIOC_G/S/TRY_EXT_CTRLS: OK
> >                 test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
> >                 test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> >                 Standard Controls: 12 Private Controls: 0
> > 
> >         Format ioctls:
> >                 test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> >                 test VIDIOC_G/S_PARM: OK (Not Supported)
> >                 test VIDIOC_G_FBUF: OK (Not Supported)
> >                 test VIDIOC_G_FMT: OK
> >                 test VIDIOC_TRY_FMT: OK
> >                 test VIDIOC_S_FMT: OK
> >                 test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> > 
> >         Codec ioctls:
> >                 test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> >                 test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> >                 test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> > 
> >         Buffer ioctls:
> >                 warn: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-buffers.cpp(475): VIDIOC_CREATE_BUFS not supported
> >                 warn: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-buffers.cpp(475): VIDIOC_CREATE_BUFS not supported
> >                 test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
> >                 test VIDIOC_EXPBUF: OK
> > 
> > Total: 38, Succeeded: 38, Failed: 0, Warnings: 2
> 
> Looks great!
> 
> Having reviewed the v4 patch series I'd say it is close to being ready (at least
> w.r.t. the V4L2 parts). Most of my comments are minor, but fixing the open()
> limitation is probably a bigger change on your side.
> 
> It should be well worth the effort, though. In my experience limiting the number
> of time you can call open() is always a sign of a design flaw and the code improves
> if this is handled in the right place (usually start_streaming, which is when you
> actually start to use the hardware).
> 
> And besides, limiting the number of open()s is out-of-spec.
> 
You are right. We did not think about this before.
We will remove open() limitation and put limitation control in right
place.
Appreciated your time and suggestions.



> Regards,
> 
> 	Hans
> 
> > 
> > 
> > Change in v3:
> > 1.Refine code to pass v4l2-compliance test, now it still has 2 issues 2.Refine code according to latest MTK IOMMU patches[1] 3.Remove MFC51 specific CIDs and add MTK specific CIDs for for keyframe and
> >   skip I-frame
> > 4.Refine code according to review comments
> > 
> > Below is the v1.6 version v4l2-compliance report for the mt8173 encoder driver.
> > Now there are still 2 test fail in v1.6.
> > For VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF, we directly use v4l2_m2m_ioctl_* functions, but it still fail. It pass in kernel 3.18 but fail in kernel 4.4.
> > We will try v1.8 in next version.
> > VIDIOC_EXPBUF is becuase we support all three memory types VB2_DMABUF, VB2_MMAP and VB2_USERPTR. VIDIOC_EXPBUF only allowed when only VB2_MMAP supported.
> > localhost ~ # /usr/bin/v4l2-compliance -d /dev/video1 Driver Info:
> >         Driver name   : mtk-vcodec-en
> >         Card type     : platform:mt817
> >         Bus info      : platform:mt817
> >         Driver version: 4.4.0
> >         Capabilities  : 0x84204000
> >                 Video Memory-to-Memory Multiplanar
> >                 Streaming
> >                 Extended Pix Format
> >                 Device Capabilities
> >         Device Caps   : 0x04204000
> >                 Video Memory-to-Memory Multiplanar
> >                 Streaming
> >                 Extended Pix Format
> > 
> > Compliance test for device /dev/video1 (not using libv4l2):
> > 
> > Required ioctls:
> >         test VIDIOC_QUERYCAP: OK
> > 
> > Allow for multiple opens:
> >         test second video open: OK
> >         test VIDIOC_QUERYCAP: OK
> >         test VIDIOC_G/S_PRIORITY: OK
> > 
> > Debug ioctls:
> >         test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> >         test VIDIOC_LOG_STATUS: OK (Not Supported)
> > 
> > Input ioctls:
> >         test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> >         test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >         test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> >         test VIDIOC_ENUMAUDIO: OK (Not Supported)
> >         test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> >         test VIDIOC_G/S_AUDIO: OK (Not Supported)
> >         Inputs: 0 Audio Inputs: 0 Tuners: 0
> > 
> > Output ioctls:
> >         test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> >         test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >         test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> >         test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> >         test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> >         Outputs: 0 Audio Outputs: 0 Modulators: 0
> > 
> > Input/Output configuration ioctls:
> >         test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> >         test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> >         test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> >         test VIDIOC_G/S_EDID: OK (Not Supported)
> > 
> >         Control ioctls:
> >                 test VIDIOC_QUERYCTRL/MENU: OK
> >                 test VIDIOC_G/S_CTRL: OK
> >                 test VIDIOC_G/S/TRY_EXT_CTRLS: OK
> >                 test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
> >                 test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> >                 Standard Controls: 11 Private Controls: 2
> > 
> >         Format ioctls:
> >                 test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> >                 test VIDIOC_G/S_PARM: OK (Not Supported)
> >                 test VIDIOC_G_FBUF: OK (Not Supported)
> >                 test VIDIOC_G_FMT: OK
> >                 test VIDIOC_TRY_FMT: OK
> >                 test VIDIOC_S_FMT: OK
> >                 test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> > 
> >         Codec ioctls:
> >                 test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> >                 test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> >                 test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> > 
> >         Buffer ioctls:
> >                 fail: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-buffers.cpp(266): vp->length == 0
> >                 fail: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-buffers.cpp(335): buf.check(Unqueued, i)
> >                 fail: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-buffers.cpp(420): testQueryBuf(node, i, q.g_buffers())
> >                 test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
> >                 fail: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-buffers.cpp(500): q.has_expbuf(node)
> >                 test VIDIOC_EXPBUF: FAIL
> > Total: 38, Succeeded: 36, Failed: 2, Warnings: 0
> > 
> > 
> > Change in v2:
> > Vcodec Part
> > 1.Remove common and include directory in mtk-vcodec 2.Refine vb2ops_venc_start_streaming and vb2ops_venc_stop_streaming and state machine 3.Remove venc_if_init and venc_if_deinit 4.Refine debug message 5.Refine lab and vpu decription in mediatek-vcodec.txt
> > 
> > VPU Part
> > 1. Modify VPU Kconfig
> > 2. Move encoder header files to other patch sets 3. Remove marcos for extended virtual/iova address 4. Change register and variable names 5. Add a reference counter for VPU watchdog 6. Remove one busy waiting in function vpu_ipi_send 7. Operate VPU clock in VPU driver (not called by encoder drivers) 8. Refine memory mapping, firmware download and extended memory allocation/free functions 9. Release more allocated resources in driver remove function
> > 
> > 
> > 
> > Andrew-CT Chen (2):
> >   dt-bindings: Add a binding for Mediatek Video Processor
> >   [Media] vcodec: mediatek: Add Mediatek V4L2 Video Encoder Driver
> > 
> > Tiffany Lin (6):
> >   [media] VPU: mediatek: support Mediatek VPU
> >   arm64: dts: mediatek: Add node for Mediatek Video Processor Unit
> >   dt-bindings: Add a binding for Mediatek Video Encoder
> >   [media] vcodec: mediatek: Add Mediatek VP8 Video Encoder Driver
> >   [media] vcodec: mediatek: Add Mediatek H264 Video Encoder Driver
> >   arm64: dts: mediatek: Add Video Encoder for MT8173
> > 
> >  .../devicetree/bindings/media/mediatek-vcodec.txt  |   59 +
> >  .../devicetree/bindings/media/mediatek-vpu.txt     |   31 +
> >  arch/arm64/boot/dts/mediatek/mt8173.dtsi           |   62 +
> >  drivers/media/platform/Kconfig                     |   20 +
> >  drivers/media/platform/Makefile                    |    4 +
> >  drivers/media/platform/mtk-vcodec/Makefile         |   11 +
> >  .../media/platform/mtk-vcodec/h264_enc/Makefile    |    6 +
> >  .../platform/mtk-vcodec/h264_enc/venc_h264_if.c    |  540 ++++++++
> >  .../platform/mtk-vcodec/h264_enc/venc_h264_if.h    |  165 +++
> >  .../platform/mtk-vcodec/h264_enc/venc_h264_vpu.c   |  309 +++++
> >  .../platform/mtk-vcodec/h264_enc/venc_h264_vpu.h   |   30 +
> >  drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h |  388 ++++++
> >  drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c | 1380 ++++++++++++++++++++
> >  drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h |   46 +
> >  .../media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c |  476 +++++++
> >  .../media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c  |  132 ++
> >  .../media/platform/mtk-vcodec/mtk_vcodec_intr.c    |  102 ++
> >  .../media/platform/mtk-vcodec/mtk_vcodec_intr.h    |   29 +
> >  drivers/media/platform/mtk-vcodec/mtk_vcodec_pm.h  |   26 +
> >  .../media/platform/mtk-vcodec/mtk_vcodec_util.c    |  106 ++
> >  .../media/platform/mtk-vcodec/mtk_vcodec_util.h    |   85 ++
> >  drivers/media/platform/mtk-vcodec/venc_drv_base.h  |   62 +
> >  drivers/media/platform/mtk-vcodec/venc_drv_if.c    |  106 ++
> >  drivers/media/platform/mtk-vcodec/venc_drv_if.h    |  175 +++
> >  drivers/media/platform/mtk-vcodec/venc_ipi_msg.h   |  212 +++
> >  drivers/media/platform/mtk-vcodec/vp8_enc/Makefile |    6 +
> >  .../platform/mtk-vcodec/vp8_enc/venc_vp8_if.c      |  422 ++++++
> >  .../platform/mtk-vcodec/vp8_enc/venc_vp8_if.h      |  149 +++
> >  .../platform/mtk-vcodec/vp8_enc/venc_vp8_vpu.c     |  240 ++++
> >  .../platform/mtk-vcodec/vp8_enc/venc_vp8_vpu.h     |   28 +
> >  drivers/media/platform/mtk-vpu/Makefile            |    1 +
> >  drivers/media/platform/mtk-vpu/mtk_vpu.c           |  994 ++++++++++++++
> >  drivers/media/platform/mtk-vpu/mtk_vpu.h           |  167 +++
> >  include/uapi/linux/v4l2-controls.h                 |    4 +
> >  34 files changed, 6573 insertions(+)
> >  create mode 100644 Documentation/devicetree/bindings/media/mediatek-vcodec.txt
> >  create mode 100644 Documentation/devicetree/bindings/media/mediatek-vpu.txt
> >  create mode 100644 drivers/media/platform/mtk-vcodec/Makefile
> >  create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/Makefile
> >  create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.c
> >  create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.h
> >  create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.c
> >  create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.h
> >  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
> >  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
> >  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h
> >  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
> >  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
> >  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.c
> >  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.h
> >  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_pm.h
> >  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c
> >  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
> >  create mode 100644 drivers/media/platform/mtk-vcodec/venc_drv_base.h
> >  create mode 100644 drivers/media/platform/mtk-vcodec/venc_drv_if.c
> >  create mode 100644 drivers/media/platform/mtk-vcodec/venc_drv_if.h
> >  create mode 100644 drivers/media/platform/mtk-vcodec/venc_ipi_msg.h
> >  create mode 100644 drivers/media/platform/mtk-vcodec/vp8_enc/Makefile
> >  create mode 100644 drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_if.c
> >  create mode 100644 drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_if.h
> >  create mode 100644 drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_vpu.c
> >  create mode 100644 drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_vpu.h
> >  create mode 100644 drivers/media/platform/mtk-vpu/Makefile
> >  create mode 100644 drivers/media/platform/mtk-vpu/mtk_vpu.c
> >  create mode 100644 drivers/media/platform/mtk-vpu/mtk_vpu.h
> >  mode change 100644 => 100755 include/uapi/linux/v4l2-controls.h
> > 
> 


--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 102+ messages in thread

* [PATCH v4 0/8] Add MT8173 Video Encoder Driver and VPU Driver
@ 2016-02-16  6:46     ` tiffany lin
  0 siblings, 0 replies; 102+ messages in thread
From: tiffany lin @ 2016-02-16  6:46 UTC (permalink / raw)
  To: linux-arm-kernel

Hi Hans,

On Mon, 2016-02-15 at 13:04 +0100, Hans Verkuil wrote:
> Hi Tiffany,
> 
> Once again, thanks for your patch series!
> 
> I have some more comments below.
> 
> On 02/04/2016 12:34 PM, Tiffany Lin wrote:
> > 
> > ==============
> >  Introduction
> > ==============
> > 
> > The purpose of this series is to add the driver for video codec hw embedded in the Mediatek's MT8173 SoCs.
> > Mediatek Video Codec is able to handle video encoding of in a range of formats.
> > 
> > This patch series also include VPU driver. Mediatek Video Codec driver rely on VPU driver to load, communicate with VPU.
> > 
> > Internally the driver uses videobuf2 framework and MTK IOMMU and MTK SMI.
> > MTK IOMMU[1] and MTK SMI[2] have not yet been merged, but we wanted to start discussion about the driver earlier so it could be merged sooner.
> > 
> > [1]https://patchwork.kernel.org/patch/8120921/
> > [2]https://patchwork.kernel.org/patch/8120941/
> > 
> > 
> > ==================
> >  Device interface
> > ==================
> > 
> > In principle the driver bases on v4l2 memory-to-memory framework:
> > it provides a single video node and each opened file handle gets its own private context with separate buffer queues. Each context consist of 2 buffer queues: OUTPUT (for source buffers, i.e. raw video frames) and CAPTURE (for destination buffers, i.e. encoded video frames).
> > 
> > ==============================
> >  VPU (Video Processor Unit)
> > ==============================
> > The VPU driver for hw video codec embedded in Mediatek's MT8173 SOCs.
> > It is able to handle video decoding/encoding in a range of formats.
> > The driver provides with VPU firmware download, memory management and the communication interface between CPU and VPU.
> > For VPU initialization, it will create virtual memory for CPU access and physical address for VPU hw device access. When a decode/encode instance opens a device node, vpu driver will download vpu firmware to the device.
> > A decode/encode instant will decode/encode a frame using VPU interface to interrupt vpu to handle decoding/encoding jobs.
> > 
> > Please have a look at the code and comments will be very much appreciated.
> > 
> > 
> > Change in v4:
> > Vcodec Part
> > 1. Remove MTK_ENCODE_PARAM_SKIP_FRAME support
> > 2. Remove MTK_ENCODE_PARAM_FRAME_TYPE and change to use V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME[3]
> > 3. Refine Encoder HW clock source
> > 4. Refine debug log
> > 5. Add watchdog support
> > 6. With patch "media: v4l2-compat-ioctl32: fix missing length copy in put_v4l2_buffer32"[4], v4l2-compliance test passed[5] in v4.4-rc5 
> > 
> > VPU Part
> > 1. These two patches were Acked-by: Rob Herring <robh@kernel.org> in v3
> >    [PATCH v3 1/8] dt-bindings: Add a binding for Mediatek Video Processor
> >    [PATCH v3 3/8] arm64: dts: mediatek: Add node for Mediatek Video Processor Unit
> >    Because we were wrong about how the hardware works, there is no connection between VPU and IOMMU HW
> >    We remove VPU attaching to IOMMU
> > 2. Support VPU running on 4GB DRAM system
> > 3. Support VPU watchdog reset
> > 4. Refine for coding style 
> > 
> > [3]https://patchwork.linuxtv.org/patch/32670/
> > [4] https://patchwork.linuxtv.org/patch/32631/
> > [5]localhost ~ # /usr/bin/v4l2-compliance -d /dev/video1
> > Driver Info:
> >         Driver name   : mtk-vcodec-en
> >         Card type     : platform:mt817
> >         Bus info      : platform:mt817
> 
> Perhaps this should be mt817x?
> 
> >         Driver version: 4.4.0
> >         Capabilities  : 0x84204000
> >                 Video Memory-to-Memory Multiplanar
> >                 Streaming
> >                 Extended Pix Format
> >                 Device Capabilities
> >         Device Caps   : 0x04204000
> >                 Video Memory-to-Memory Multiplanar
> >                 Streaming
> >                 Extended Pix Format
> > 
> > Compliance test for device /dev/video1 (not using libv4l2):
> > 
> > Required ioctls:
> >         test VIDIOC_QUERYCAP: OK
> > 
> > Allow for multiple opens:
> >         test second video open: OK
> >         test VIDIOC_QUERYCAP: OK
> >         test VIDIOC_G/S_PRIORITY: OK
> > 
> > Debug ioctls:
> >         test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> >         test VIDIOC_LOG_STATUS: OK (Not Supported)
> > 
> > Input ioctls:
> >         test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> >         test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >         test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> >         test VIDIOC_ENUMAUDIO: OK (Not Supported)
> >         test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> >         test VIDIOC_G/S_AUDIO: OK (Not Supported)
> >         Inputs: 0 Audio Inputs: 0 Tuners: 0
> > 
> > Output ioctls:
> >         test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> >         test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >         test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> >         test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> >         test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> >         Outputs: 0 Audio Outputs: 0 Modulators: 0
> > 
> > Input/Output configuration ioctls:
> >         test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> >         test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> >         test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> >         test VIDIOC_G/S_EDID: OK (Not Supported)
> > 
> >         Control ioctls:
> >                 test VIDIOC_QUERYCTRL/MENU: OK
> >                 test VIDIOC_G/S_CTRL: OK
> >                 test VIDIOC_G/S/TRY_EXT_CTRLS: OK
> >                 test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
> >                 test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> >                 Standard Controls: 12 Private Controls: 0
> > 
> >         Format ioctls:
> >                 test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> >                 test VIDIOC_G/S_PARM: OK (Not Supported)
> >                 test VIDIOC_G_FBUF: OK (Not Supported)
> >                 test VIDIOC_G_FMT: OK
> >                 test VIDIOC_TRY_FMT: OK
> >                 test VIDIOC_S_FMT: OK
> >                 test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> > 
> >         Codec ioctls:
> >                 test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> >                 test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> >                 test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> > 
> >         Buffer ioctls:
> >                 warn: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-buffers.cpp(475): VIDIOC_CREATE_BUFS not supported
> >                 warn: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-buffers.cpp(475): VIDIOC_CREATE_BUFS not supported
> >                 test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
> >                 test VIDIOC_EXPBUF: OK
> > 
> > Total: 38, Succeeded: 38, Failed: 0, Warnings: 2
> 
> Looks great!
> 
> Having reviewed the v4 patch series I'd say it is close to being ready (at least
> w.r.t. the V4L2 parts). Most of my comments are minor, but fixing the open()
> limitation is probably a bigger change on your side.
> 
> It should be well worth the effort, though. In my experience limiting the number
> of time you can call open() is always a sign of a design flaw and the code improves
> if this is handled in the right place (usually start_streaming, which is when you
> actually start to use the hardware).
> 
> And besides, limiting the number of open()s is out-of-spec.
> 
You are right. We did not think about this before.
We will remove open() limitation and put limitation control in right
place.
Appreciated your time and suggestions.



> Regards,
> 
> 	Hans
> 
> > 
> > 
> > Change in v3:
> > 1.Refine code to pass v4l2-compliance test, now it still has 2 issues 2.Refine code according to latest MTK IOMMU patches[1] 3.Remove MFC51 specific CIDs and add MTK specific CIDs for for keyframe and
> >   skip I-frame
> > 4.Refine code according to review comments
> > 
> > Below is the v1.6 version v4l2-compliance report for the mt8173 encoder driver.
> > Now there are still 2 test fail in v1.6.
> > For VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF, we directly use v4l2_m2m_ioctl_* functions, but it still fail. It pass in kernel 3.18 but fail in kernel 4.4.
> > We will try v1.8 in next version.
> > VIDIOC_EXPBUF is becuase we support all three memory types VB2_DMABUF, VB2_MMAP and VB2_USERPTR. VIDIOC_EXPBUF only allowed when only VB2_MMAP supported.
> > localhost ~ # /usr/bin/v4l2-compliance -d /dev/video1 Driver Info:
> >         Driver name   : mtk-vcodec-en
> >         Card type     : platform:mt817
> >         Bus info      : platform:mt817
> >         Driver version: 4.4.0
> >         Capabilities  : 0x84204000
> >                 Video Memory-to-Memory Multiplanar
> >                 Streaming
> >                 Extended Pix Format
> >                 Device Capabilities
> >         Device Caps   : 0x04204000
> >                 Video Memory-to-Memory Multiplanar
> >                 Streaming
> >                 Extended Pix Format
> > 
> > Compliance test for device /dev/video1 (not using libv4l2):
> > 
> > Required ioctls:
> >         test VIDIOC_QUERYCAP: OK
> > 
> > Allow for multiple opens:
> >         test second video open: OK
> >         test VIDIOC_QUERYCAP: OK
> >         test VIDIOC_G/S_PRIORITY: OK
> > 
> > Debug ioctls:
> >         test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> >         test VIDIOC_LOG_STATUS: OK (Not Supported)
> > 
> > Input ioctls:
> >         test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> >         test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >         test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> >         test VIDIOC_ENUMAUDIO: OK (Not Supported)
> >         test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> >         test VIDIOC_G/S_AUDIO: OK (Not Supported)
> >         Inputs: 0 Audio Inputs: 0 Tuners: 0
> > 
> > Output ioctls:
> >         test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> >         test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >         test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> >         test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> >         test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> >         Outputs: 0 Audio Outputs: 0 Modulators: 0
> > 
> > Input/Output configuration ioctls:
> >         test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> >         test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> >         test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> >         test VIDIOC_G/S_EDID: OK (Not Supported)
> > 
> >         Control ioctls:
> >                 test VIDIOC_QUERYCTRL/MENU: OK
> >                 test VIDIOC_G/S_CTRL: OK
> >                 test VIDIOC_G/S/TRY_EXT_CTRLS: OK
> >                 test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
> >                 test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> >                 Standard Controls: 11 Private Controls: 2
> > 
> >         Format ioctls:
> >                 test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> >                 test VIDIOC_G/S_PARM: OK (Not Supported)
> >                 test VIDIOC_G_FBUF: OK (Not Supported)
> >                 test VIDIOC_G_FMT: OK
> >                 test VIDIOC_TRY_FMT: OK
> >                 test VIDIOC_S_FMT: OK
> >                 test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> > 
> >         Codec ioctls:
> >                 test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> >                 test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> >                 test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> > 
> >         Buffer ioctls:
> >                 fail: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-buffers.cpp(266): vp->length == 0
> >                 fail: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-buffers.cpp(335): buf.check(Unqueued, i)
> >                 fail: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-buffers.cpp(420): testQueryBuf(node, i, q.g_buffers())
> >                 test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
> >                 fail: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-buffers.cpp(500): q.has_expbuf(node)
> >                 test VIDIOC_EXPBUF: FAIL
> > Total: 38, Succeeded: 36, Failed: 2, Warnings: 0
> > 
> > 
> > Change in v2:
> > Vcodec Part
> > 1.Remove common and include directory in mtk-vcodec 2.Refine vb2ops_venc_start_streaming and vb2ops_venc_stop_streaming and state machine 3.Remove venc_if_init and venc_if_deinit 4.Refine debug message 5.Refine lab and vpu decription in mediatek-vcodec.txt
> > 
> > VPU Part
> > 1. Modify VPU Kconfig
> > 2. Move encoder header files to other patch sets 3. Remove marcos for extended virtual/iova address 4. Change register and variable names 5. Add a reference counter for VPU watchdog 6. Remove one busy waiting in function vpu_ipi_send 7. Operate VPU clock in VPU driver (not called by encoder drivers) 8. Refine memory mapping, firmware download and extended memory allocation/free functions 9. Release more allocated resources in driver remove function
> > 
> > 
> > 
> > Andrew-CT Chen (2):
> >   dt-bindings: Add a binding for Mediatek Video Processor
> >   [Media] vcodec: mediatek: Add Mediatek V4L2 Video Encoder Driver
> > 
> > Tiffany Lin (6):
> >   [media] VPU: mediatek: support Mediatek VPU
> >   arm64: dts: mediatek: Add node for Mediatek Video Processor Unit
> >   dt-bindings: Add a binding for Mediatek Video Encoder
> >   [media] vcodec: mediatek: Add Mediatek VP8 Video Encoder Driver
> >   [media] vcodec: mediatek: Add Mediatek H264 Video Encoder Driver
> >   arm64: dts: mediatek: Add Video Encoder for MT8173
> > 
> >  .../devicetree/bindings/media/mediatek-vcodec.txt  |   59 +
> >  .../devicetree/bindings/media/mediatek-vpu.txt     |   31 +
> >  arch/arm64/boot/dts/mediatek/mt8173.dtsi           |   62 +
> >  drivers/media/platform/Kconfig                     |   20 +
> >  drivers/media/platform/Makefile                    |    4 +
> >  drivers/media/platform/mtk-vcodec/Makefile         |   11 +
> >  .../media/platform/mtk-vcodec/h264_enc/Makefile    |    6 +
> >  .../platform/mtk-vcodec/h264_enc/venc_h264_if.c    |  540 ++++++++
> >  .../platform/mtk-vcodec/h264_enc/venc_h264_if.h    |  165 +++
> >  .../platform/mtk-vcodec/h264_enc/venc_h264_vpu.c   |  309 +++++
> >  .../platform/mtk-vcodec/h264_enc/venc_h264_vpu.h   |   30 +
> >  drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h |  388 ++++++
> >  drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c | 1380 ++++++++++++++++++++
> >  drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h |   46 +
> >  .../media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c |  476 +++++++
> >  .../media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c  |  132 ++
> >  .../media/platform/mtk-vcodec/mtk_vcodec_intr.c    |  102 ++
> >  .../media/platform/mtk-vcodec/mtk_vcodec_intr.h    |   29 +
> >  drivers/media/platform/mtk-vcodec/mtk_vcodec_pm.h  |   26 +
> >  .../media/platform/mtk-vcodec/mtk_vcodec_util.c    |  106 ++
> >  .../media/platform/mtk-vcodec/mtk_vcodec_util.h    |   85 ++
> >  drivers/media/platform/mtk-vcodec/venc_drv_base.h  |   62 +
> >  drivers/media/platform/mtk-vcodec/venc_drv_if.c    |  106 ++
> >  drivers/media/platform/mtk-vcodec/venc_drv_if.h    |  175 +++
> >  drivers/media/platform/mtk-vcodec/venc_ipi_msg.h   |  212 +++
> >  drivers/media/platform/mtk-vcodec/vp8_enc/Makefile |    6 +
> >  .../platform/mtk-vcodec/vp8_enc/venc_vp8_if.c      |  422 ++++++
> >  .../platform/mtk-vcodec/vp8_enc/venc_vp8_if.h      |  149 +++
> >  .../platform/mtk-vcodec/vp8_enc/venc_vp8_vpu.c     |  240 ++++
> >  .../platform/mtk-vcodec/vp8_enc/venc_vp8_vpu.h     |   28 +
> >  drivers/media/platform/mtk-vpu/Makefile            |    1 +
> >  drivers/media/platform/mtk-vpu/mtk_vpu.c           |  994 ++++++++++++++
> >  drivers/media/platform/mtk-vpu/mtk_vpu.h           |  167 +++
> >  include/uapi/linux/v4l2-controls.h                 |    4 +
> >  34 files changed, 6573 insertions(+)
> >  create mode 100644 Documentation/devicetree/bindings/media/mediatek-vcodec.txt
> >  create mode 100644 Documentation/devicetree/bindings/media/mediatek-vpu.txt
> >  create mode 100644 drivers/media/platform/mtk-vcodec/Makefile
> >  create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/Makefile
> >  create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.c
> >  create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.h
> >  create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.c
> >  create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.h
> >  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
> >  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
> >  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h
> >  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
> >  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
> >  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.c
> >  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.h
> >  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_pm.h
> >  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c
> >  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
> >  create mode 100644 drivers/media/platform/mtk-vcodec/venc_drv_base.h
> >  create mode 100644 drivers/media/platform/mtk-vcodec/venc_drv_if.c
> >  create mode 100644 drivers/media/platform/mtk-vcodec/venc_drv_if.h
> >  create mode 100644 drivers/media/platform/mtk-vcodec/venc_ipi_msg.h
> >  create mode 100644 drivers/media/platform/mtk-vcodec/vp8_enc/Makefile
> >  create mode 100644 drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_if.c
> >  create mode 100644 drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_if.h
> >  create mode 100644 drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_vpu.c
> >  create mode 100644 drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_vpu.h
> >  create mode 100644 drivers/media/platform/mtk-vpu/Makefile
> >  create mode 100644 drivers/media/platform/mtk-vpu/mtk_vpu.c
> >  create mode 100644 drivers/media/platform/mtk-vpu/mtk_vpu.h
> >  mode change 100644 => 100755 include/uapi/linux/v4l2-controls.h
> > 
> 

^ permalink raw reply	[flat|nested] 102+ messages in thread

* Re: [PATCH v4 5/8] [Media] vcodec: mediatek: Add Mediatek V4L2 Video Encoder Driver
  2016-02-16  6:37               ` tiffany lin
  (?)
@ 2016-02-16  7:44                 ` Hans Verkuil
  -1 siblings, 0 replies; 102+ messages in thread
From: Hans Verkuil @ 2016-02-16  7:44 UTC (permalink / raw)
  To: tiffany lin
  Cc: Hans Verkuil, daniel.thompson, Rob Herring,
	Mauro Carvalho Chehab, Matthias Brugger, Daniel Kurtz,
	Pawel Osciak, Eddie Huang, Yingjoe Chen, devicetree,
	linux-kernel, linux-arm-kernel, linux-media, linux-mediatek,
	PoChun.Lin, Andrew-CT Chen

On 02/16/2016 07:37 AM, tiffany lin wrote:

<snip>

>>> +static int vidioc_venc_s_parm(struct file *file, void *priv,
>>> +			      struct v4l2_streamparm *a)
>>> +{
>>> +	struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
>>> +
>>> +	if (a->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
>>> +		ctx->enc_params.framerate_num =
>>> +			a->parm.output.timeperframe.denominator;
>>> +		ctx->enc_params.framerate_denom =
>>> +			a->parm.output.timeperframe.numerator;
>>> +		ctx->param_change |= MTK_ENCODE_PARAM_FRAMERATE;
>>> +	} else {
>>> +		return -EINVAL;
>>> +	}
>>
>> I'd invert the test:
>>
>> 	if (a->type != V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE)
>> 		return -EINVAL;
>>
>> and now you can just set ctx->enc_params.
>>
> We will fix this in next version.
> 
> 
>>> +	return 0;
>>> +}
>>
>> And if there is an s_parm, then there should be a g_parm as well!
>>
> Now our driver does not support g_parm, our use cases do not use g_parm
> too. 
> Do we need to add g_parm at this moment? Or we could add it when we need
> g_parm?

No, you need it. You can see why if you look at the v4l2-compliance output:

                 test VIDIOC_G/S_PARM: OK (Not Supported)

Why does it think it is unsupported? Because (just like most applications) it
tries to call G_PARM first, and if that succeeds it tries to call S_PARM with
the value it got from G_PARM. Thus ensuring the application doesn't change the
driver state. So you can have a 'get' ioctl without the 'set' ioctl, but if
there is a 'set' ioctl there must always be a 'get' ioctl.

<snip>

>>> +static int vidioc_venc_g_s_selection(struct file *file, void *priv,
>>> +                                struct v4l2_selection *s)
>>
>> Why support s_selection if you can only return the current width and height?
>> And why support g_selection if you can't change the selection?
>>
>> In other words, why implement this at all?
>>
>> Unless I am missing something here, I would just drop this.
>>
> Now our driver do not support these capabilities, but userspace app will
> check whether g/s_crop are implemented when using encoder.
> Because g/s_crop are deprecated as you mentioned in previous v2 review
> comments. We change to use g_s_selection.
> We will check if we could add this capability.

It's true that you should use g/s_selection instead of g/s_crop, but only if
there is actually something to select. As long as you don't offer this capability,
just drop this for now.

When you add the capability later you can just add the g/s_selection functions.

Getting selection right can be tricky. I wouldn't mind if this is done later in a
separate patch.

> 
>>> +{
>>> +	struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
>>> +	struct mtk_q_data *q_data;
>>> +
>>> +	if (V4L2_TYPE_IS_OUTPUT(s->type)) {
>>> +		if (s->target !=  V4L2_SEL_TGT_COMPOSE)
>>> +			return -EINVAL;
>>> +	} else {
>>> +		if (s->target != V4L2_SEL_TGT_CROP)
>>> +			return -EINVAL;
>>> +	}
>>> +
>>> +	if (s->r.left || s->r.top)
>>> +		return -EINVAL;
>>> +
>>> +	q_data = mtk_venc_get_q_data(ctx, s->type);
>>> +	if (!q_data)
>>> +		return -EINVAL;
>>> +
>>> +	s->r.width = q_data->width;
>>> +	s->r.height = q_data->height;
>>> +
>>> +	return 0;
>>> +}
>>> +
>>> +
>>> +static int vidioc_venc_qbuf(struct file *file, void *priv,
>>> +			    struct v4l2_buffer *buf)
>>> +{
>>> +
>>> +	struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
>>> +
>>> +	if (ctx->state == MTK_STATE_ABORT) {
>>> +		mtk_v4l2_err("[%d] Call on QBUF after unrecoverable error\n", ctx->idx);
>>> +		return -EIO;
>>> +	}
>>> +
>>> +	return v4l2_m2m_qbuf(file, ctx->m2m_ctx, buf);
>>> +}
>>> +
>>> +static int vidioc_venc_dqbuf(struct file *file, void *priv,
>>> +			     struct v4l2_buffer *buf)
>>> +{
>>> +	struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
>>> +	if (ctx->state == MTK_STATE_ABORT) {
>>> +		mtk_v4l2_err("[%d] Call on QBUF after unrecoverable error\n", ctx->idx);
>>> +		return -EIO;
>>> +	}
>>> +
>>> +	return v4l2_m2m_dqbuf(file, ctx->m2m_ctx, buf);
>>> +}
>>> +
>>> +
>>> +const struct v4l2_ioctl_ops mtk_venc_ioctl_ops = {
>>> +	.vidioc_streamon		= v4l2_m2m_ioctl_streamon,
>>> +	.vidioc_streamoff		= v4l2_m2m_ioctl_streamoff,
>>> +
>>> +	.vidioc_reqbufs			= v4l2_m2m_ioctl_reqbufs,
>>> +	.vidioc_querybuf		= v4l2_m2m_ioctl_querybuf,
>>> +	.vidioc_qbuf			= vidioc_venc_qbuf,
>>> +	.vidioc_dqbuf			= vidioc_venc_dqbuf,
>>> +
>>> +	.vidioc_querycap		= vidioc_venc_querycap,
>>> +	.vidioc_enum_fmt_vid_cap_mplane = vidioc_enum_fmt_vid_cap_mplane,
>>> +	.vidioc_enum_fmt_vid_out_mplane = vidioc_enum_fmt_vid_out_mplane,
>>> +	.vidioc_enum_framesizes		= vidioc_enum_framesizes,
>>> +
>>> +	.vidioc_try_fmt_vid_cap_mplane	= vidioc_try_fmt_vid_cap_mplane,
>>> +	.vidioc_try_fmt_vid_out_mplane	= vidioc_try_fmt_vid_out_mplane,
>>> +	.vidioc_expbuf			= v4l2_m2m_ioctl_expbuf,
>>
>> Please add vidioc_create_bufs and vidioc_prepare_buf as well.
>>
> 
> Currently we do not support these use cases, do we need to add
> vidioc_create_bufs and vidioc_prepare_buf now?

I would suggest you do. The vb2 framework gives it (almost) for free.
prepare_buf is completely free (just add the helper) and create_bufs
needs a few small changes in the queue_setup function, that's all.

> 
> 
>>> +	.vidioc_subscribe_event		= v4l2_ctrl_subscribe_event,
>>> +	.vidioc_unsubscribe_event	= v4l2_event_unsubscribe,
>>> +
>>> +	.vidioc_s_parm			= vidioc_venc_s_parm,
>>> +
>>> +	.vidioc_s_fmt_vid_cap_mplane	= vidioc_venc_s_fmt,
>>> +	.vidioc_s_fmt_vid_out_mplane	= vidioc_venc_s_fmt,
>>> +
>>> +	.vidioc_g_fmt_vid_cap_mplane	= vidioc_venc_g_fmt,
>>> +	.vidioc_g_fmt_vid_out_mplane	= vidioc_venc_g_fmt,
>>> +
>>> +	.vidioc_g_selection		= vidioc_venc_g_s_selection,
>>> +	.vidioc_s_selection		= vidioc_venc_g_s_selection,
>>> +};

<snip>

>>> +int mtk_vcodec_enc_queue_init(void *priv, struct vb2_queue *src_vq,
>>> +			   struct vb2_queue *dst_vq)
>>> +{
>>> +	struct mtk_vcodec_ctx *ctx = priv;
>>> +	int ret;
>>> +
>>> +	src_vq->type		= V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE;
>>> +	src_vq->io_modes	= VB2_DMABUF | VB2_MMAP | VB2_USERPTR;
>>
>> I recomment dropping VB2_USERPTR. That only makes sense for scatter-gather dma,
>> and you use physically contiguous DMA.
>>
> Now our userspace app use VB2_USERPTR. I need to check if we could drop
> VB2_USERPTR.
> We use src_vq->mem_ops = &vb2_dma_contig_memops;
> And there are
> 	.get_userptr	= vb2_dc_get_userptr,
> 	.put_userptr	= vb2_dc_put_userptr,
> I was confused why it only make sense for scatter-gather.
> Could you kindly explain more?

VB2_USERPTR indicates that the application can use malloc to allocate buffers
and pass those to the driver. Since malloc uses virtual memory the physical
memory is scattered all over. And the first page typically does not start at
the beginning of the page but at a random offset.

To support that the DMA generally has to be able to do scatter-gather.

Now, where things get ugly is that a hack was added to the USERPTR support where
apps could pass a pointer to physically contiguous memory as a user pointer. This
was a hack for embedded systems that preallocated a pool of buffers and needed to
pass those pointers around somehow. So the dma-contig USERPTR support is for that
'feature'. If you try to pass a malloc()ed buffer to a dma-contig driver it will
reject it. One big problem is that this specific hack isn't signaled anywhere, so
applications have no way of knowing if the USERPTR support is the proper version
or the hack where physically contiguous memory is expected.

This hack has been replaced with DMABUF which is the proper way of passing buffers
around.

New dma-contig drivers should not use that old hack anymore. Use dmabuf to pass
external buffers around.

How do you use it in your app? With malloc()ed buffers? Or with 'special' pointers
to physically contiguous buffers?

> 
>>> +	src_vq->drv_priv	= ctx;
>>> +	src_vq->buf_struct_size = sizeof(struct mtk_video_enc_buf);
>>> +	src_vq->ops		= &mtk_venc_vb2_ops;
>>> +	src_vq->mem_ops		= &vb2_dma_contig_memops;
>>> +	src_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
>>> +	src_vq->lock = &ctx->dev->dev_mutex;
>>> +
>>> +	ret = vb2_queue_init(src_vq);
>>> +	if (ret)
>>> +		return ret;
>>> +
>>> +	dst_vq->type		= V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
>>> +	dst_vq->io_modes	= VB2_DMABUF | VB2_MMAP | VB2_USERPTR;
>>> +	dst_vq->drv_priv	= ctx;
>>> +	dst_vq->buf_struct_size = sizeof(struct v4l2_m2m_buffer);
>>> +	dst_vq->ops		= &mtk_venc_vb2_ops;
>>> +	dst_vq->mem_ops		= &vb2_dma_contig_memops;
>>> +	dst_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
>>> +	dst_vq->lock = &ctx->dev->dev_mutex;
>>> +
>>> +	return vb2_queue_init(dst_vq);
>>> +}

Regards,

	Hans

^ permalink raw reply	[flat|nested] 102+ messages in thread

* Re: [PATCH v4 5/8] [Media] vcodec: mediatek: Add Mediatek V4L2 Video Encoder Driver
@ 2016-02-16  7:44                 ` Hans Verkuil
  0 siblings, 0 replies; 102+ messages in thread
From: Hans Verkuil @ 2016-02-16  7:44 UTC (permalink / raw)
  To: tiffany lin
  Cc: Hans Verkuil, daniel.thompson-QSEj5FYQhm4dnm+yROfE0A,
	Rob Herring, Mauro Carvalho Chehab, Matthias Brugger,
	Daniel Kurtz, Pawel Osciak, Eddie Huang, Yingjoe Chen,
	devicetree-u79uwXL29TY76Z2rM5mHXA,
	linux-kernel-u79uwXL29TY76Z2rM5mHXA,
	linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
	linux-media-u79uwXL29TY76Z2rM5mHXA,
	linux-mediatek-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
	PoChun.Lin-NuS5LvNUpcJWk0Htik3J/w, Andrew-CT Chen

On 02/16/2016 07:37 AM, tiffany lin wrote:

<snip>

>>> +static int vidioc_venc_s_parm(struct file *file, void *priv,
>>> +			      struct v4l2_streamparm *a)
>>> +{
>>> +	struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
>>> +
>>> +	if (a->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
>>> +		ctx->enc_params.framerate_num =
>>> +			a->parm.output.timeperframe.denominator;
>>> +		ctx->enc_params.framerate_denom =
>>> +			a->parm.output.timeperframe.numerator;
>>> +		ctx->param_change |= MTK_ENCODE_PARAM_FRAMERATE;
>>> +	} else {
>>> +		return -EINVAL;
>>> +	}
>>
>> I'd invert the test:
>>
>> 	if (a->type != V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE)
>> 		return -EINVAL;
>>
>> and now you can just set ctx->enc_params.
>>
> We will fix this in next version.
> 
> 
>>> +	return 0;
>>> +}
>>
>> And if there is an s_parm, then there should be a g_parm as well!
>>
> Now our driver does not support g_parm, our use cases do not use g_parm
> too. 
> Do we need to add g_parm at this moment? Or we could add it when we need
> g_parm?

No, you need it. You can see why if you look at the v4l2-compliance output:

                 test VIDIOC_G/S_PARM: OK (Not Supported)

Why does it think it is unsupported? Because (just like most applications) it
tries to call G_PARM first, and if that succeeds it tries to call S_PARM with
the value it got from G_PARM. Thus ensuring the application doesn't change the
driver state. So you can have a 'get' ioctl without the 'set' ioctl, but if
there is a 'set' ioctl there must always be a 'get' ioctl.

<snip>

>>> +static int vidioc_venc_g_s_selection(struct file *file, void *priv,
>>> +                                struct v4l2_selection *s)
>>
>> Why support s_selection if you can only return the current width and height?
>> And why support g_selection if you can't change the selection?
>>
>> In other words, why implement this at all?
>>
>> Unless I am missing something here, I would just drop this.
>>
> Now our driver do not support these capabilities, but userspace app will
> check whether g/s_crop are implemented when using encoder.
> Because g/s_crop are deprecated as you mentioned in previous v2 review
> comments. We change to use g_s_selection.
> We will check if we could add this capability.

It's true that you should use g/s_selection instead of g/s_crop, but only if
there is actually something to select. As long as you don't offer this capability,
just drop this for now.

When you add the capability later you can just add the g/s_selection functions.

Getting selection right can be tricky. I wouldn't mind if this is done later in a
separate patch.

> 
>>> +{
>>> +	struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
>>> +	struct mtk_q_data *q_data;
>>> +
>>> +	if (V4L2_TYPE_IS_OUTPUT(s->type)) {
>>> +		if (s->target !=  V4L2_SEL_TGT_COMPOSE)
>>> +			return -EINVAL;
>>> +	} else {
>>> +		if (s->target != V4L2_SEL_TGT_CROP)
>>> +			return -EINVAL;
>>> +	}
>>> +
>>> +	if (s->r.left || s->r.top)
>>> +		return -EINVAL;
>>> +
>>> +	q_data = mtk_venc_get_q_data(ctx, s->type);
>>> +	if (!q_data)
>>> +		return -EINVAL;
>>> +
>>> +	s->r.width = q_data->width;
>>> +	s->r.height = q_data->height;
>>> +
>>> +	return 0;
>>> +}
>>> +
>>> +
>>> +static int vidioc_venc_qbuf(struct file *file, void *priv,
>>> +			    struct v4l2_buffer *buf)
>>> +{
>>> +
>>> +	struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
>>> +
>>> +	if (ctx->state == MTK_STATE_ABORT) {
>>> +		mtk_v4l2_err("[%d] Call on QBUF after unrecoverable error\n", ctx->idx);
>>> +		return -EIO;
>>> +	}
>>> +
>>> +	return v4l2_m2m_qbuf(file, ctx->m2m_ctx, buf);
>>> +}
>>> +
>>> +static int vidioc_venc_dqbuf(struct file *file, void *priv,
>>> +			     struct v4l2_buffer *buf)
>>> +{
>>> +	struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
>>> +	if (ctx->state == MTK_STATE_ABORT) {
>>> +		mtk_v4l2_err("[%d] Call on QBUF after unrecoverable error\n", ctx->idx);
>>> +		return -EIO;
>>> +	}
>>> +
>>> +	return v4l2_m2m_dqbuf(file, ctx->m2m_ctx, buf);
>>> +}
>>> +
>>> +
>>> +const struct v4l2_ioctl_ops mtk_venc_ioctl_ops = {
>>> +	.vidioc_streamon		= v4l2_m2m_ioctl_streamon,
>>> +	.vidioc_streamoff		= v4l2_m2m_ioctl_streamoff,
>>> +
>>> +	.vidioc_reqbufs			= v4l2_m2m_ioctl_reqbufs,
>>> +	.vidioc_querybuf		= v4l2_m2m_ioctl_querybuf,
>>> +	.vidioc_qbuf			= vidioc_venc_qbuf,
>>> +	.vidioc_dqbuf			= vidioc_venc_dqbuf,
>>> +
>>> +	.vidioc_querycap		= vidioc_venc_querycap,
>>> +	.vidioc_enum_fmt_vid_cap_mplane = vidioc_enum_fmt_vid_cap_mplane,
>>> +	.vidioc_enum_fmt_vid_out_mplane = vidioc_enum_fmt_vid_out_mplane,
>>> +	.vidioc_enum_framesizes		= vidioc_enum_framesizes,
>>> +
>>> +	.vidioc_try_fmt_vid_cap_mplane	= vidioc_try_fmt_vid_cap_mplane,
>>> +	.vidioc_try_fmt_vid_out_mplane	= vidioc_try_fmt_vid_out_mplane,
>>> +	.vidioc_expbuf			= v4l2_m2m_ioctl_expbuf,
>>
>> Please add vidioc_create_bufs and vidioc_prepare_buf as well.
>>
> 
> Currently we do not support these use cases, do we need to add
> vidioc_create_bufs and vidioc_prepare_buf now?

I would suggest you do. The vb2 framework gives it (almost) for free.
prepare_buf is completely free (just add the helper) and create_bufs
needs a few small changes in the queue_setup function, that's all.

> 
> 
>>> +	.vidioc_subscribe_event		= v4l2_ctrl_subscribe_event,
>>> +	.vidioc_unsubscribe_event	= v4l2_event_unsubscribe,
>>> +
>>> +	.vidioc_s_parm			= vidioc_venc_s_parm,
>>> +
>>> +	.vidioc_s_fmt_vid_cap_mplane	= vidioc_venc_s_fmt,
>>> +	.vidioc_s_fmt_vid_out_mplane	= vidioc_venc_s_fmt,
>>> +
>>> +	.vidioc_g_fmt_vid_cap_mplane	= vidioc_venc_g_fmt,
>>> +	.vidioc_g_fmt_vid_out_mplane	= vidioc_venc_g_fmt,
>>> +
>>> +	.vidioc_g_selection		= vidioc_venc_g_s_selection,
>>> +	.vidioc_s_selection		= vidioc_venc_g_s_selection,
>>> +};

<snip>

>>> +int mtk_vcodec_enc_queue_init(void *priv, struct vb2_queue *src_vq,
>>> +			   struct vb2_queue *dst_vq)
>>> +{
>>> +	struct mtk_vcodec_ctx *ctx = priv;
>>> +	int ret;
>>> +
>>> +	src_vq->type		= V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE;
>>> +	src_vq->io_modes	= VB2_DMABUF | VB2_MMAP | VB2_USERPTR;
>>
>> I recomment dropping VB2_USERPTR. That only makes sense for scatter-gather dma,
>> and you use physically contiguous DMA.
>>
> Now our userspace app use VB2_USERPTR. I need to check if we could drop
> VB2_USERPTR.
> We use src_vq->mem_ops = &vb2_dma_contig_memops;
> And there are
> 	.get_userptr	= vb2_dc_get_userptr,
> 	.put_userptr	= vb2_dc_put_userptr,
> I was confused why it only make sense for scatter-gather.
> Could you kindly explain more?

VB2_USERPTR indicates that the application can use malloc to allocate buffers
and pass those to the driver. Since malloc uses virtual memory the physical
memory is scattered all over. And the first page typically does not start at
the beginning of the page but at a random offset.

To support that the DMA generally has to be able to do scatter-gather.

Now, where things get ugly is that a hack was added to the USERPTR support where
apps could pass a pointer to physically contiguous memory as a user pointer. This
was a hack for embedded systems that preallocated a pool of buffers and needed to
pass those pointers around somehow. So the dma-contig USERPTR support is for that
'feature'. If you try to pass a malloc()ed buffer to a dma-contig driver it will
reject it. One big problem is that this specific hack isn't signaled anywhere, so
applications have no way of knowing if the USERPTR support is the proper version
or the hack where physically contiguous memory is expected.

This hack has been replaced with DMABUF which is the proper way of passing buffers
around.

New dma-contig drivers should not use that old hack anymore. Use dmabuf to pass
external buffers around.

How do you use it in your app? With malloc()ed buffers? Or with 'special' pointers
to physically contiguous buffers?

> 
>>> +	src_vq->drv_priv	= ctx;
>>> +	src_vq->buf_struct_size = sizeof(struct mtk_video_enc_buf);
>>> +	src_vq->ops		= &mtk_venc_vb2_ops;
>>> +	src_vq->mem_ops		= &vb2_dma_contig_memops;
>>> +	src_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
>>> +	src_vq->lock = &ctx->dev->dev_mutex;
>>> +
>>> +	ret = vb2_queue_init(src_vq);
>>> +	if (ret)
>>> +		return ret;
>>> +
>>> +	dst_vq->type		= V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
>>> +	dst_vq->io_modes	= VB2_DMABUF | VB2_MMAP | VB2_USERPTR;
>>> +	dst_vq->drv_priv	= ctx;
>>> +	dst_vq->buf_struct_size = sizeof(struct v4l2_m2m_buffer);
>>> +	dst_vq->ops		= &mtk_venc_vb2_ops;
>>> +	dst_vq->mem_ops		= &vb2_dma_contig_memops;
>>> +	dst_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
>>> +	dst_vq->lock = &ctx->dev->dev_mutex;
>>> +
>>> +	return vb2_queue_init(dst_vq);
>>> +}

Regards,

	Hans

--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 102+ messages in thread

* [PATCH v4 5/8] [Media] vcodec: mediatek: Add Mediatek V4L2 Video Encoder Driver
@ 2016-02-16  7:44                 ` Hans Verkuil
  0 siblings, 0 replies; 102+ messages in thread
From: Hans Verkuil @ 2016-02-16  7:44 UTC (permalink / raw)
  To: linux-arm-kernel

On 02/16/2016 07:37 AM, tiffany lin wrote:

<snip>

>>> +static int vidioc_venc_s_parm(struct file *file, void *priv,
>>> +			      struct v4l2_streamparm *a)
>>> +{
>>> +	struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
>>> +
>>> +	if (a->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
>>> +		ctx->enc_params.framerate_num =
>>> +			a->parm.output.timeperframe.denominator;
>>> +		ctx->enc_params.framerate_denom =
>>> +			a->parm.output.timeperframe.numerator;
>>> +		ctx->param_change |= MTK_ENCODE_PARAM_FRAMERATE;
>>> +	} else {
>>> +		return -EINVAL;
>>> +	}
>>
>> I'd invert the test:
>>
>> 	if (a->type != V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE)
>> 		return -EINVAL;
>>
>> and now you can just set ctx->enc_params.
>>
> We will fix this in next version.
> 
> 
>>> +	return 0;
>>> +}
>>
>> And if there is an s_parm, then there should be a g_parm as well!
>>
> Now our driver does not support g_parm, our use cases do not use g_parm
> too. 
> Do we need to add g_parm at this moment? Or we could add it when we need
> g_parm?

No, you need it. You can see why if you look at the v4l2-compliance output:

                 test VIDIOC_G/S_PARM: OK (Not Supported)

Why does it think it is unsupported? Because (just like most applications) it
tries to call G_PARM first, and if that succeeds it tries to call S_PARM with
the value it got from G_PARM. Thus ensuring the application doesn't change the
driver state. So you can have a 'get' ioctl without the 'set' ioctl, but if
there is a 'set' ioctl there must always be a 'get' ioctl.

<snip>

>>> +static int vidioc_venc_g_s_selection(struct file *file, void *priv,
>>> +                                struct v4l2_selection *s)
>>
>> Why support s_selection if you can only return the current width and height?
>> And why support g_selection if you can't change the selection?
>>
>> In other words, why implement this at all?
>>
>> Unless I am missing something here, I would just drop this.
>>
> Now our driver do not support these capabilities, but userspace app will
> check whether g/s_crop are implemented when using encoder.
> Because g/s_crop are deprecated as you mentioned in previous v2 review
> comments. We change to use g_s_selection.
> We will check if we could add this capability.

It's true that you should use g/s_selection instead of g/s_crop, but only if
there is actually something to select. As long as you don't offer this capability,
just drop this for now.

When you add the capability later you can just add the g/s_selection functions.

Getting selection right can be tricky. I wouldn't mind if this is done later in a
separate patch.

> 
>>> +{
>>> +	struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
>>> +	struct mtk_q_data *q_data;
>>> +
>>> +	if (V4L2_TYPE_IS_OUTPUT(s->type)) {
>>> +		if (s->target !=  V4L2_SEL_TGT_COMPOSE)
>>> +			return -EINVAL;
>>> +	} else {
>>> +		if (s->target != V4L2_SEL_TGT_CROP)
>>> +			return -EINVAL;
>>> +	}
>>> +
>>> +	if (s->r.left || s->r.top)
>>> +		return -EINVAL;
>>> +
>>> +	q_data = mtk_venc_get_q_data(ctx, s->type);
>>> +	if (!q_data)
>>> +		return -EINVAL;
>>> +
>>> +	s->r.width = q_data->width;
>>> +	s->r.height = q_data->height;
>>> +
>>> +	return 0;
>>> +}
>>> +
>>> +
>>> +static int vidioc_venc_qbuf(struct file *file, void *priv,
>>> +			    struct v4l2_buffer *buf)
>>> +{
>>> +
>>> +	struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
>>> +
>>> +	if (ctx->state == MTK_STATE_ABORT) {
>>> +		mtk_v4l2_err("[%d] Call on QBUF after unrecoverable error\n", ctx->idx);
>>> +		return -EIO;
>>> +	}
>>> +
>>> +	return v4l2_m2m_qbuf(file, ctx->m2m_ctx, buf);
>>> +}
>>> +
>>> +static int vidioc_venc_dqbuf(struct file *file, void *priv,
>>> +			     struct v4l2_buffer *buf)
>>> +{
>>> +	struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
>>> +	if (ctx->state == MTK_STATE_ABORT) {
>>> +		mtk_v4l2_err("[%d] Call on QBUF after unrecoverable error\n", ctx->idx);
>>> +		return -EIO;
>>> +	}
>>> +
>>> +	return v4l2_m2m_dqbuf(file, ctx->m2m_ctx, buf);
>>> +}
>>> +
>>> +
>>> +const struct v4l2_ioctl_ops mtk_venc_ioctl_ops = {
>>> +	.vidioc_streamon		= v4l2_m2m_ioctl_streamon,
>>> +	.vidioc_streamoff		= v4l2_m2m_ioctl_streamoff,
>>> +
>>> +	.vidioc_reqbufs			= v4l2_m2m_ioctl_reqbufs,
>>> +	.vidioc_querybuf		= v4l2_m2m_ioctl_querybuf,
>>> +	.vidioc_qbuf			= vidioc_venc_qbuf,
>>> +	.vidioc_dqbuf			= vidioc_venc_dqbuf,
>>> +
>>> +	.vidioc_querycap		= vidioc_venc_querycap,
>>> +	.vidioc_enum_fmt_vid_cap_mplane = vidioc_enum_fmt_vid_cap_mplane,
>>> +	.vidioc_enum_fmt_vid_out_mplane = vidioc_enum_fmt_vid_out_mplane,
>>> +	.vidioc_enum_framesizes		= vidioc_enum_framesizes,
>>> +
>>> +	.vidioc_try_fmt_vid_cap_mplane	= vidioc_try_fmt_vid_cap_mplane,
>>> +	.vidioc_try_fmt_vid_out_mplane	= vidioc_try_fmt_vid_out_mplane,
>>> +	.vidioc_expbuf			= v4l2_m2m_ioctl_expbuf,
>>
>> Please add vidioc_create_bufs and vidioc_prepare_buf as well.
>>
> 
> Currently we do not support these use cases, do we need to add
> vidioc_create_bufs and vidioc_prepare_buf now?

I would suggest you do. The vb2 framework gives it (almost) for free.
prepare_buf is completely free (just add the helper) and create_bufs
needs a few small changes in the queue_setup function, that's all.

> 
> 
>>> +	.vidioc_subscribe_event		= v4l2_ctrl_subscribe_event,
>>> +	.vidioc_unsubscribe_event	= v4l2_event_unsubscribe,
>>> +
>>> +	.vidioc_s_parm			= vidioc_venc_s_parm,
>>> +
>>> +	.vidioc_s_fmt_vid_cap_mplane	= vidioc_venc_s_fmt,
>>> +	.vidioc_s_fmt_vid_out_mplane	= vidioc_venc_s_fmt,
>>> +
>>> +	.vidioc_g_fmt_vid_cap_mplane	= vidioc_venc_g_fmt,
>>> +	.vidioc_g_fmt_vid_out_mplane	= vidioc_venc_g_fmt,
>>> +
>>> +	.vidioc_g_selection		= vidioc_venc_g_s_selection,
>>> +	.vidioc_s_selection		= vidioc_venc_g_s_selection,
>>> +};

<snip>

>>> +int mtk_vcodec_enc_queue_init(void *priv, struct vb2_queue *src_vq,
>>> +			   struct vb2_queue *dst_vq)
>>> +{
>>> +	struct mtk_vcodec_ctx *ctx = priv;
>>> +	int ret;
>>> +
>>> +	src_vq->type		= V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE;
>>> +	src_vq->io_modes	= VB2_DMABUF | VB2_MMAP | VB2_USERPTR;
>>
>> I recomment dropping VB2_USERPTR. That only makes sense for scatter-gather dma,
>> and you use physically contiguous DMA.
>>
> Now our userspace app use VB2_USERPTR. I need to check if we could drop
> VB2_USERPTR.
> We use src_vq->mem_ops = &vb2_dma_contig_memops;
> And there are
> 	.get_userptr	= vb2_dc_get_userptr,
> 	.put_userptr	= vb2_dc_put_userptr,
> I was confused why it only make sense for scatter-gather.
> Could you kindly explain more?

VB2_USERPTR indicates that the application can use malloc to allocate buffers
and pass those to the driver. Since malloc uses virtual memory the physical
memory is scattered all over. And the first page typically does not start at
the beginning of the page but at a random offset.

To support that the DMA generally has to be able to do scatter-gather.

Now, where things get ugly is that a hack was added to the USERPTR support where
apps could pass a pointer to physically contiguous memory as a user pointer. This
was a hack for embedded systems that preallocated a pool of buffers and needed to
pass those pointers around somehow. So the dma-contig USERPTR support is for that
'feature'. If you try to pass a malloc()ed buffer to a dma-contig driver it will
reject it. One big problem is that this specific hack isn't signaled anywhere, so
applications have no way of knowing if the USERPTR support is the proper version
or the hack where physically contiguous memory is expected.

This hack has been replaced with DMABUF which is the proper way of passing buffers
around.

New dma-contig drivers should not use that old hack anymore. Use dmabuf to pass
external buffers around.

How do you use it in your app? With malloc()ed buffers? Or with 'special' pointers
to physically contiguous buffers?

> 
>>> +	src_vq->drv_priv	= ctx;
>>> +	src_vq->buf_struct_size = sizeof(struct mtk_video_enc_buf);
>>> +	src_vq->ops		= &mtk_venc_vb2_ops;
>>> +	src_vq->mem_ops		= &vb2_dma_contig_memops;
>>> +	src_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
>>> +	src_vq->lock = &ctx->dev->dev_mutex;
>>> +
>>> +	ret = vb2_queue_init(src_vq);
>>> +	if (ret)
>>> +		return ret;
>>> +
>>> +	dst_vq->type		= V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
>>> +	dst_vq->io_modes	= VB2_DMABUF | VB2_MMAP | VB2_USERPTR;
>>> +	dst_vq->drv_priv	= ctx;
>>> +	dst_vq->buf_struct_size = sizeof(struct v4l2_m2m_buffer);
>>> +	dst_vq->ops		= &mtk_venc_vb2_ops;
>>> +	dst_vq->mem_ops		= &vb2_dma_contig_memops;
>>> +	dst_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
>>> +	dst_vq->lock = &ctx->dev->dev_mutex;
>>> +
>>> +	return vb2_queue_init(dst_vq);
>>> +}

Regards,

	Hans

^ permalink raw reply	[flat|nested] 102+ messages in thread

* Re: [PATCH v4 2/8] [media] VPU: mediatek: support Mediatek VPU
  2016-02-15 13:59       ` Wu-Cheng Li (李務誠)
  (?)
@ 2016-02-16  9:36         ` andrew-ct chen
  -1 siblings, 0 replies; 102+ messages in thread
From: andrew-ct chen @ 2016-02-16  9:36 UTC (permalink / raw)
  To: Wu-Cheng Li (李務誠)
  Cc: Tiffany Lin, Hans Verkuil, daniel.thompson, Rob Herring,
	Mauro Carvalho Chehab, Matthias Brugger, Daniel Kurtz,
	Pawel Osciak, Eddie Huang, Yingjoe Chen, devicetree,
	linux-kernel, linux-arm-kernel, linux-media, linux-mediatek,
	Lin PoChun

On Mon, 2016-02-15 at 21:59 +0800, Wu-Cheng Li (李務誠) wrote:
> On Thu, Feb 4, 2016 at 7:34 PM, Tiffany Lin <tiffany.lin@mediatek.com> wrote:
> > The VPU driver for hw video codec embedded in Mediatek's MT8173 SOCs.
> > It is able to handle video decoding/encoding of in a range of formats.
> > The driver provides with VPU firmware download, memory management and
> > the communication interface between CPU and VPU.
> > For VPU initialization, it will create virtual memory for CPU access and
> > IOMMU address for vcodec hw device access. When a decode/encode instance
> > opens a device node, vpu driver will download vpu firmware to the device.
> > A decode/encode instant will decode/encode a frame using VPU
> > interface to interrupt vpu to handle decoding/encoding jobs.
> >
> > Signed-off-by: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
> > Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
> > ---
> >  drivers/media/platform/Kconfig           |    9 +
> >  drivers/media/platform/Makefile          |    2 +
> >  drivers/media/platform/mtk-vpu/Makefile  |    1 +
> >  drivers/media/platform/mtk-vpu/mtk_vpu.c |  994 ++++++++++++++++++++++++++++++
> >  drivers/media/platform/mtk-vpu/mtk_vpu.h |  167 +++++
> >  5 files changed, 1173 insertions(+)
> >  create mode 100644 drivers/media/platform/mtk-vpu/Makefile
> >  create mode 100644 drivers/media/platform/mtk-vpu/mtk_vpu.c
> >  create mode 100644 drivers/media/platform/mtk-vpu/mtk_vpu.h
> >
> > diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
> > index ccbc974..ba812d6 100644
> > --- a/drivers/media/platform/Kconfig
> > +++ b/drivers/media/platform/Kconfig
> > @@ -148,6 +148,15 @@ config VIDEO_CODA
> >            Coda is a range of video codec IPs that supports
> >            H.264, MPEG-4, and other video formats.
> >
> > +config VIDEO_MEDIATEK_VPU
> > +       tristate "Mediatek Video Processor Unit"
> > +       depends on VIDEO_DEV && VIDEO_V4L2 && ARCH_MEDIATEK
> > +       ---help---
> > +           This driver provides downloading VPU firmware and
> > +           communicating with VPU. This driver for hw video
> > +           codec embedded in new Mediatek's SOCs. It is able
> > +           to handle video decoding/encoding in a range of formats.
> > +
> >  config VIDEO_MEM2MEM_DEINTERLACE
> >         tristate "Deinterlace support"
> >         depends on VIDEO_DEV && VIDEO_V4L2 && DMA_ENGINE
> > diff --git a/drivers/media/platform/Makefile b/drivers/media/platform/Makefile
> > index efa0295..e5b19c6 100644
> > --- a/drivers/media/platform/Makefile
> > +++ b/drivers/media/platform/Makefile
> > @@ -55,3 +55,5 @@ obj-$(CONFIG_VIDEO_AM437X_VPFE)               += am437x/
> >  obj-$(CONFIG_VIDEO_XILINX)             += xilinx/
> >
> >  ccflags-y += -I$(srctree)/drivers/media/i2c
> > +
> > +obj-$(CONFIG_VIDEO_MEDIATEK_VPU)       += mtk-vpu/
> > diff --git a/drivers/media/platform/mtk-vpu/Makefile b/drivers/media/platform/mtk-vpu/Makefile
> > new file mode 100644
> > index 0000000..d890a66
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vpu/Makefile
> > @@ -0,0 +1 @@
> > +obj-y += mtk_vpu.o
> > diff --git a/drivers/media/platform/mtk-vpu/mtk_vpu.c b/drivers/media/platform/mtk-vpu/mtk_vpu.c
> > new file mode 100644
> > index 0000000..f54fd89
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vpu/mtk_vpu.c
> > @@ -0,0 +1,994 @@
> > +/*
> > +* Copyright (c) 2015 MediaTek Inc.
> > +* Author: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
> > +*
> > +* This program is free software; you can redistribute it and/or modify
> > +* it under the terms of the GNU General Public License version 2 as
> > +* published by the Free Software Foundation.
> > +*
> > +* This program is distributed in the hope that it will be useful,
> > +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> > +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > +* GNU General Public License for more details.
> > +*/
> > +#include <linux/bootmem.h>
> > +#include <linux/clk.h>
> > +#include <linux/debugfs.h>
> > +#include <linux/firmware.h>
> > +#include <linux/interrupt.h>
> > +#include <linux/iommu.h>
> > +#include <linux/module.h>
> > +#include <linux/of_address.h>
> > +#include <linux/of_irq.h>
> > +#include <linux/of_platform.h>
> > +#include <linux/of_reserved_mem.h>
> > +#include <linux/sched.h>
> > +#include <linux/sizes.h>
> > +
> > +#include "mtk_vpu.h"
> > +
> > +/**
> > + * VPU (video processor unit) is a tiny processor controlling video hardware
> > + * related to video codec, scaling and color format converting.
> > + * VPU interfaces with other blocks by share memory and interrupt.
> > + **/
> > +
> > +#define INIT_TIMEOUT_MS                2000U
> > +#define IPI_TIMEOUT_MS         2000U
> > +#define VPU_FW_VER_LEN         16
> > +
> > +/* maximum program/data TCM (Tightly-Coupled Memory) size */
> > +#define VPU_PTCM_SIZE          (96 * SZ_1K)
> > +#define VPU_DTCM_SIZE          (32 * SZ_1K)
> > +/* the offset to get data tcm address */
> > +#define VPU_DTCM_OFFSET                0x18000UL
> > +/* daynamic allocated maximum extended memory size */
> > +#define VPU_EXT_P_SIZE         SZ_1M
> > +#define VPU_EXT_D_SIZE         SZ_4M
> > +/* maximum binary firmware size */
> > +#define VPU_P_FW_SIZE          (VPU_PTCM_SIZE + VPU_EXT_P_SIZE)
> > +#define VPU_D_FW_SIZE          (VPU_DTCM_SIZE + VPU_EXT_D_SIZE)
> > +/* the size of share buffer between Host and  VPU */
> > +#define SHARE_BUF_SIZE         48
> > +
> > +/* binary firmware name */
> > +#define VPU_P_FW               "vpu_p.bin"
> > +#define VPU_D_FW               "vpu_d.bin"
> > +
> > +#define VPU_RESET              0x0
> > +#define VPU_TCM_CFG            0x0008
> > +#define VPU_PMEM_EXT0_ADDR     0x000C
> > +#define VPU_PMEM_EXT1_ADDR     0x0010
> > +#define VPU_TO_HOST            0x001C
> > +#define VPU_DMEM_EXT0_ADDR     0x0014
> > +#define VPU_DMEM_EXT1_ADDR     0x0018
> > +#define HOST_TO_VPU            0x0024
> > +#define VPU_PC_REG             0x0060
> > +#define VPU_WDT_REG            0x0084
> > +
> > +/* vpu inter-processor communication interrupt */
> > +#define VPU_IPC_INT            BIT(8)
> > +
> > +/**
> > + * enum vpu_fw_type - VPU firmware type
> > + *
> > + * @P_FW: program firmware
> > + * @D_FW: data firmware
> > + *
> > + */
> > +enum vpu_fw_type {
> > +       P_FW,
> > +       D_FW,
> > +};
> > +
> > +/**
> > + * struct vpu_mem - VPU extended program/data memory information
> > + *
> > + * @va:                the kernel virtual memory address of VPU extended memory
> > + * @pa:                the physical memory address of VPU extended memory
> This should be device memory address? If yes, s/physical/device/ and
> s/pa/dma_addr/.

It's really a physical memory since vpu is not attached to IOMMU.It
should be s/dma_addr_t/phys_addr_t for the data type.Thanks.

> > + *
> > + */
> > +struct vpu_mem {
> > +       void *va;
> > +       dma_addr_t pa;
> > +};
> > +
> > +/**
> > + * struct vpu_regs - VPU TCM and configuration registers
> > + *
> > + * @tcm:       the register for VPU Tightly-Coupled Memory
> > + * @cfg:       the register for VPU configuration
> > + * @irq:       the irq number for VPU interrupt
> > + */
> > +struct vpu_regs {
> > +       void __iomem *tcm;
> > +       void __iomem *cfg;
> > +       int irq;
> > +};
> > +
> > +/**
> > + * struct vpu_wdt_handler - VPU watchdog reset handler
> > + *
> > + * @reset_func:        reset handler
> > + * @priv:      private data
> > + */
> > +struct vpu_wdt_handler {
> > +       void (*reset_func)(void *);
> > +       void *priv;
> > +};
> > +
> > +/**
> > + * struct vpu_wdt - VPU watchdog workqueue
> > + *
> > + * @handler:   VPU watchdog reset handler
> > + * @ws:                workstruct for VPU watchdog
> > + * @wq:                workqueue for VPU watchdog
> > + */
> > +struct vpu_wdt {
> > +       struct vpu_wdt_handler handler[VPU_RST_MAX];
> > +       struct work_struct ws;
> > +       struct workqueue_struct *wq;
> > +};
> > +
> > +/**
> > + * struct vpu_run - VPU initialization status
> > + *
> > + * @signaled:          the signal of vpu initialization completed
> > + * @fw_ver:            VPU firmware version
> > + * @enc_capability:    encoder capability
> We should document the meaning of the bit used. Since no bit is used
> by the codec driver, document this is not used for now and the value
> is reserved for future use.

This is not used for now. I will document this in next version. Thanks.

> > + * @wq:                        wait queue for VPU initialization status
> > + */
> > +struct vpu_run {
> > +       u32 signaled;
> > +       char fw_ver[VPU_FW_VER_LEN];
> > +       unsigned int    enc_capability;
> > +       wait_queue_head_t wq;
> > +};
> > +
> > +/**
> > + * struct vpu_ipi_desc - VPU IPI descriptor
> > + *
> > + * @handler:   IPI handler
> > + * @name:      the name of IPI handler
> > + * @priv:      the private data of IPI handler
> > + */
> > +struct vpu_ipi_desc {
> > +       ipi_handler_t handler;
> > +       const char *name;
> > +       void *priv;
> > +};
> > +
> > +/**
> > + * struct share_obj - DTCM (Data Tightly-Coupled Memory) buffer shared with
> > + *                   AP and VPU
> > + *
> > + * @id:                IPI id
> > + * @len:       share buffer length
> > + * @share_buf: share buffer data
> > + */
> > +struct share_obj {
> > +       s32 id;
> > +       u32 len;
> > +       unsigned char share_buf[SHARE_BUF_SIZE];
> > +};
> > +
> > +/**
> > + * struct mtk_vpu - vpu driver data
> > + * @extmem:            VPU extended memory information
> > + * @reg:               VPU TCM and configuration registers
> > + * @run:               VPU initialization status
> > + * @ipi_desc:          VPU IPI descriptor
> > + * @recv_buf:          VPU DTCM share buffer for receiving. The
> > + *                     receive buffer is only accessed in interrupt context.
> > + * @send_buf:          VPU DTCM share buffer for sending
> > + * @dev:               VPU struct device
> > + * @clk:               VPU clock on/off
> > + * @enable_4GB:                VPU 4GB mode on/off
> > + * @vpu_mutex:         protect mtk_vpu (except recv_buf) and ensure only
> > + *                     one client to use VPU service at a time. For example,
> > + *                     suppose a client is using VPU to decode VP8.
> > + *                     If the other client wants to encode VP8,
> > + *                     it has to wait until VP8 decode completes.
> > + * @wdt_refcnt         WDT reference count to make sure the watchdog can be
> > + *                     disabled if no other client is using VPU service
> > + * @ipi_ack_signaled:  The ACKs for registered IPI function sending
> > + *                     interrupt to VPU
> s/ipi_ack_signaled/ipi_id_ack/. Move to after |ack_wq| to be
> consistent with the order of variable declaration.

I will change this in next version. Thanks.

> > + * @ack_wq:            The wait queue for each codec and mdp. When sleeping
> > + *                     processes wake up, they will check the condition
> > + *                     "ipi_ack_signaled" to run the corresponding action or
> > + *                     go back to sleep.
> > + *
> > + */
> > +struct mtk_vpu {
> > +       struct vpu_mem extmem[2];
> > +       struct vpu_regs reg;
> > +       struct vpu_run run;
> > +       struct vpu_wdt wdt;
> > +       struct vpu_ipi_desc ipi_desc[IPI_MAX];
> > +       struct share_obj *recv_buf;
> > +       struct share_obj *send_buf;
> > +       struct device *dev;
> > +       struct clk *clk;
> > +       bool enable_4GB;
> > +       struct mutex vpu_mutex; /* for protecting vpu data data structure */
> Remove the comment here. It should be documented in function comment above

It will get message "CHECK" from checkpatch script if removing this
comment.

> .
> > +       atomic_t wdt_refcnt;
> > +       wait_queue_head_t ack_wq;
> > +       bool ipi_id_ack[IPI_MAX];
> > +};
> > +
> > +static inline void vpu_cfg_writel(struct mtk_vpu *vpu, u32 val, u32 offset)
> > +{
> > +       writel(val, vpu->reg.cfg + offset);
> > +}
> > +
> > +static inline u32 vpu_cfg_readl(struct mtk_vpu *vpu, u32 offset)
> > +{
> > +       return readl(vpu->reg.cfg + offset);
> > +}
> > +
> > +static inline bool vpu_running(struct mtk_vpu *vpu)
> > +{
> > +       return vpu_cfg_readl(vpu, VPU_RESET) & BIT(0);
> > +}
> > +
> > +void vpu_clock_disable(struct mtk_vpu *vpu)
> > +{
> > +       /* Disable VPU watchdog */
> > +       if (atomic_dec_and_test(&vpu->wdt_refcnt))
> Checking wdt_refcnt and doing vpu_cfg_writel should be done
> atomically. For example, if vpu_clock_enable is called after
> atomic_dec_and_test(&vpu->wdt_refcnt) and before vpu_cfg_writel(...),
> the state of watchdog could be wrong. Adding a spinlock to protect
> wdt_refcnt and vpu_cfg_writel together. Then wdt_refcnt doesn't need
> to be atomic_t.

I would use mutex to protect this since this only be called in process
context. I will change this in next version. Thanks.

> > +               vpu_cfg_writel(vpu,
> > +                              vpu_cfg_readl(vpu, VPU_WDT_REG) & ~(1L << 31),
> > +                              VPU_WDT_REG);
> > +
> > +       clk_disable(vpu->clk);
> > +}
> > +
> > +int vpu_clock_enable(struct mtk_vpu *vpu)
> > +{
> > +       int ret;
> > +
> > +       ret = clk_enable(vpu->clk);
> > +       if (ret)
> > +               return ret;
> > +       /* Enable VPU watchdog */
> > +       if (!atomic_read(&vpu->wdt_refcnt))
> Do we need to enable the watchdog other than vpu_ipi_send? The only
> place that AP waits for VPU is in vpu_ipi_send. Right? If yes, can we
> just enable and disable the watchdog in vpu_ipi_send?

Watchdog also should be enabled during driver initialization and
firmware download.

> > +               vpu_cfg_writel(vpu,
> > +                              vpu_cfg_readl(vpu, VPU_WDT_REG) | (1L << 31),
> > +                              VPU_WDT_REG);
> > +
> > +       atomic_inc(&vpu->wdt_refcnt);
> Same above. atomic_read, vpu_cfg_writel, and atomic_inc should be done
> atomically.

I will change this in next version. Thanks.

> > +
> > +       return ret;
> > +}
> > +
> > +int vpu_ipi_register(struct platform_device *pdev,
> > +                    enum ipi_id id, ipi_handler_t handler,
> > +                    const char *name, void *priv)
> > +{
> > +       struct mtk_vpu *vpu = platform_get_drvdata(pdev);
> > +       struct vpu_ipi_desc *ipi_desc;
> > +
> > +       if (!vpu) {
> > +               dev_err(&pdev->dev, "vpu device in not ready\n");
> > +               return -EPROBE_DEFER;
> > +       }
> > +
> > +       if (id < IPI_MAX && handler) {
> Check if id >= 0 because type of enum is implementation dependent.

I will change this in next version. Thanks.

> > +               ipi_desc = vpu->ipi_desc;
> > +               ipi_desc[id].name = name;
> > +               ipi_desc[id].handler = handler;
> > +               ipi_desc[id].priv = priv;
> > +               return 0;
> > +       }
> > +
> > +       dev_err(&pdev->dev, "register vpu ipi with invalid arguments\n");
> print id

I will change this in next version. Thanks.

> > +       return -EINVAL;
> > +}
> > +
> > +int vpu_ipi_send(struct platform_device *pdev,
> > +                enum ipi_id id, void *buf,
> > +                unsigned int len)
> > +{
> > +       struct mtk_vpu *vpu = platform_get_drvdata(pdev);
> > +       struct share_obj *send_obj = vpu->send_buf;
> > +       unsigned long timeout;
> > +       int ret = 0;
> > +
> > +       if (id <= IPI_VPU_INIT || id >= IPI_MAX ||
> > +           len > sizeof(send_obj->share_buf) || !buf) {
> > +               dev_err(vpu->dev, "failed to send ipi message\n");
> > +               return -EINVAL;
> > +       }
> > +
> > +       ret = vpu_clock_enable(vpu);
> > +       if (ret) {
> > +               dev_err(vpu->dev, "failed to enable vpu clock\n");
> > +               return ret;
> > +       }
> > +       if (!vpu_running(vpu)) {
> > +               dev_err(vpu->dev, "vpu_ipi_send: VPU is not running\n");
> > +               ret = -EINVAL;
> > +               goto clock_disable;
> > +       }
> > +
> > +       mutex_lock(&vpu->vpu_mutex);
> > +
> > +        /* Wait until VPU receives the last command */
> > +       timeout = jiffies + msecs_to_jiffies(IPI_TIMEOUT_MS);
> > +       do {
> > +               if (time_after(jiffies, timeout)) {
> > +                       dev_err(vpu->dev, "vpu_ipi_send: IPI timeout!\n");
> > +                       ret = -EIO;
> > +                       goto mut_unlock;
> > +               }
> > +       } while (vpu_cfg_readl(vpu, HOST_TO_VPU));
> > +
> > +       memcpy((void *)send_obj->share_buf, buf, len);
> > +       send_obj->len = len;
> > +       send_obj->id = id;
> > +
> > +       vpu->ipi_id_ack[id] = false;
> > +       /* send the command to VPU */
> > +       vpu_cfg_writel(vpu, 0x1, HOST_TO_VPU);
> > +
> > +       mutex_unlock(&vpu->vpu_mutex);
> > +
> > +       /* wait for VPU's ACK */
> > +       timeout = msecs_to_jiffies(IPI_TIMEOUT_MS);
> > +       ret = wait_event_interruptible_timeout(vpu->ack_wq,
> > +                                              vpu->ipi_id_ack[id], timeout);
> > +       vpu->ipi_id_ack[id] = false;
> > +       if (ret == 0) {
> > +               dev_err(vpu->dev, "vpu ipi %d ack time out !", id);
> > +               ret = -EIO;
> > +               goto clock_disable;
> > +       } else if (-ERESTARTSYS == ret) {
> > +               dev_err(vpu->dev, "vpu ipi %d ack wait interrupted by a signal",
> > +                       id);
> > +               ret = -ERESTARTSYS;
> > +               goto clock_disable;
> > +       }
> > +       vpu_clock_disable(vpu);
> > +
> > +       return 0;
> > +
> > +mut_unlock:
> > +       vpu->ipi_id_ack[id] = false;
> I don't see why we need to set it to false here. Remove if not needed.
I will remove this in next version. Thanks.
> > +       mutex_unlock(&vpu->vpu_mutex);
> > +clock_disable:
> > +       vpu_clock_disable(vpu);
> > +
> > +       return ret;
> > +}
> > +
> > +static void vpu_wdt_reset_func(struct work_struct *ws)
> > +{
> > +       struct vpu_wdt *wdt = container_of(ws, struct vpu_wdt, ws);
> > +       struct mtk_vpu *vpu = container_of(wdt, struct mtk_vpu, wdt);
> > +       struct vpu_wdt_handler *handler = wdt->handler;
> > +       int index, ret;
> > +
> > +       dev_info(vpu->dev, "vpu reset\n");
> > +       mutex_lock(&vpu->vpu_mutex);
> > +       ret = vpu_clock_enable(vpu);
> > +       if (ret) {
> This is missing mutex_unlock. Move mutex_lock right before
> vpu_cfg_writel(vpu, 0x0, VPU_RESET);. So we don't need to unlock when
> vpu_clock_enable fails.

I will change this in next version. Thanks.

> > +               dev_err(vpu->dev, "[VPU] wdt enables clock failed %d\n", ret);
> > +               return;
> > +       }
> > +       vpu_cfg_writel(vpu, 0x0, VPU_RESET);
> > +       vpu_clock_disable(vpu);
> > +       mutex_unlock(&vpu->vpu_mutex);
> > +
> > +       for (index = 0; index < VPU_RST_MAX; index++) {
> > +               if (handler[index].reset_func) {
> > +                       handler[index].reset_func(handler[index].priv);
> > +                       dev_dbg(vpu->dev, "wdt handler func %d\n", index);
> > +               }
> > +       }
> > +}
> > +
> > +int vpu_wdt_reg_handler(struct platform_device *pdev,
> > +                       void wdt_reset(void *),
> > +                       void *priv, enum rst_id id)
> > +{
> > +       struct mtk_vpu *vpu = platform_get_drvdata(pdev);
> > +       struct vpu_wdt_handler *handler = vpu->wdt.handler;
> > +
> > +       if (!vpu) {
> > +               dev_err(vpu->dev, "vpu device in not ready\n");
> > +               return -EPROBE_DEFER;
> > +       }
> > +
> > +       if (id < VPU_RST_MAX && wdt_reset != NULL) {
> Check id >= 0

I will change this in next version. Thanks.

> > +               dev_dbg(vpu->dev, "wdt register id %d\n", id);
> > +               mutex_lock(&vpu->vpu_mutex);
> > +               handler[id].reset_func = wdt_reset;
> > +               handler[id].priv = priv;
> > +               mutex_unlock(&vpu->vpu_mutex);
> > +               return 0;
> > +       }
> > +
> > +       dev_err(vpu->dev, "register vpu wdt handler failed\n");
> > +       return -EINVAL;
> > +}
> > +
> > +unsigned int vpu_get_venc_hw_capa(struct platform_device *pdev)
> > +{
> > +       struct mtk_vpu *vpu = platform_get_drvdata(pdev);
> > +
> > +       return vpu->run.enc_capability;
> > +}
> > +
> > +void *vpu_mapping_dm_addr(struct platform_device *pdev,
> > +                         u32 dtcm_dmem_addr)
> > +{
> > +       struct mtk_vpu *vpu = platform_get_drvdata(pdev);
> > +
> > +       if (!dtcm_dmem_addr ||
> > +           (dtcm_dmem_addr > (VPU_DTCM_SIZE + VPU_EXT_D_SIZE))) {
> > +               dev_err(vpu->dev, "invalid virtual data memory address\n");
> > +               return ERR_PTR(-EINVAL);
> > +       }
> > +
> > +       if (dtcm_dmem_addr < VPU_DTCM_SIZE)
> > +               return dtcm_dmem_addr + vpu->reg.tcm + VPU_DTCM_OFFSET;
> > +
> > +       return vpu->extmem[D_FW].va + (dtcm_dmem_addr - VPU_DTCM_SIZE);
> > +}
> > +
> > +struct platform_device *vpu_get_plat_device(struct platform_device *pdev)
> > +{
> > +       struct device *dev = &pdev->dev;
> > +       struct device_node *vpu_node;
> > +       struct platform_device *vpu_pdev;
> > +
> > +       vpu_node = of_parse_phandle(dev->of_node, "mediatek,vpu", 0);
> > +       if (!vpu_node) {
> > +               dev_err(dev, "can't get vpu node\n");
> > +               return NULL;
> > +       }
> > +
> > +       vpu_pdev = of_find_device_by_node(vpu_node);
> > +       if (WARN_ON(!vpu_pdev)) {
> > +               dev_err(dev, "vpu pdev failed\n");
> > +               of_node_put(vpu_node);
> > +               return NULL;
> > +       }
> > +
> > +       return vpu_pdev;
> > +}
> > +
> > +/* load vpu program/data memory */
> > +static int load_requested_vpu(struct mtk_vpu *vpu,
> > +                             const struct firmware *vpu_fw,
> > +                             u8 fw_type)
> > +{
> > +       size_t tcm_size = fw_type ? VPU_DTCM_SIZE : VPU_PTCM_SIZE;
> > +       size_t fw_size = fw_type ? VPU_D_FW_SIZE : VPU_P_FW_SIZE;
> > +       char *fw_name = fw_type ? VPU_D_FW : VPU_P_FW;
> > +       size_t dl_size = 0;
> > +       size_t extra_fw_size = 0;
> > +       void *dest;
> > +       int ret;
> > +
> > +       ret = request_firmware(&vpu_fw, fw_name, vpu->dev);
> > +       if (ret < 0) {
> > +               dev_err(vpu->dev, "Failed to load %s, %d\n", fw_name, ret);
> > +               return ret;
> > +       }
> > +       dl_size = vpu_fw->size;
> > +       if (dl_size > fw_size) {
> > +               dev_err(vpu->dev, "fw %s size %zu is abnormal\n", fw_name,
> > +                       dl_size);
> > +               release_firmware(vpu_fw);
> > +               return  -EFBIG;
> > +       }
> > +       dev_dbg(vpu->dev, "Downloaded fw %s size: %zu.\n",
> > +               fw_name,
> > +               dl_size);
> > +       /* reset VPU */
> > +       vpu_cfg_writel(vpu, 0x0, VPU_RESET);
> > +
> > +       /* handle extended firmware size */
> > +       if (dl_size > tcm_size) {
> > +               dev_dbg(vpu->dev, "fw size %lx > limited fw size %lx\n",
> > +                       dl_size, tcm_size);
> > +               extra_fw_size = dl_size - tcm_size;
> > +               dev_dbg(vpu->dev, "extra_fw_size %lx\n", extra_fw_size);
> > +               dl_size = tcm_size;
> > +       }
> > +       dest = vpu->reg.tcm;
> > +       if (fw_type == D_FW)
> > +               dest += VPU_DTCM_OFFSET;
> > +       memcpy(dest, vpu_fw->data, dl_size);
> > +       /* download to extended memory if need */
> > +       if (extra_fw_size > 0) {
> > +               dest = vpu->extmem[fw_type].va;
> > +               dev_dbg(vpu->dev, "download extended memory type %x\n",
> > +                       fw_type);
> > +               memcpy(dest, vpu_fw->data + tcm_size, extra_fw_size);
> > +       }
> > +
> > +       release_firmware(vpu_fw);
> > +
> > +       return 0;
> > +}
> > +
> > +int vpu_load_firmware(struct platform_device *pdev)
> > +{
> > +       struct mtk_vpu *vpu = platform_get_drvdata(pdev);
> > +       struct device *dev = &pdev->dev;
> > +       struct vpu_run *run = &vpu->run;
> > +       const struct firmware *vpu_fw;
> > +       int ret;
> > +
> > +       if (!pdev) {
> > +               dev_err(dev, "VPU platform device is invalid\n");
> > +               return -EINVAL;
> > +       }
> > +
> > +       mutex_lock(&vpu->vpu_mutex);
> > +
> > +       ret = vpu_clock_enable(vpu);
> > +       if (ret) {
> > +               dev_err(dev, "enable clock failed %d\n", ret);
> > +               goto OUT_LOAD_FW;
> > +       }
> > +
> > +       if (vpu_running(vpu)) {
> > +               vpu_clock_disable(vpu);
> > +               mutex_unlock(&vpu->vpu_mutex);
> > +               dev_warn(dev, "vpu is running already\n");
> > +               return 0;
> > +       }
> > +
> > +       run->signaled = false;
> > +       dev_dbg(vpu->dev, "firmware request\n");
> > +       /* Downloading program firmware to device*/
> > +       ret = load_requested_vpu(vpu, vpu_fw, P_FW);
> > +       if (ret < 0) {
> > +               dev_err(dev, "Failed to request %s, %d\n", VPU_P_FW, ret);
> > +               goto OUT_LOAD_FW;
> > +       }
> > +
> > +       /* Downloading data firmware to device */
> > +       ret = load_requested_vpu(vpu, vpu_fw, D_FW);
> > +       if (ret < 0) {
> > +               dev_err(dev, "Failed to request %s, %d\n", VPU_D_FW, ret);
> > +               goto OUT_LOAD_FW;
> > +       }
> > +
> > +       /* boot up vpu */
> > +       vpu_cfg_writel(vpu, 0x1, VPU_RESET);
> > +
> > +       ret = wait_event_interruptible_timeout(run->wq,
> > +                                              run->signaled,
> > +                                              msecs_to_jiffies(INIT_TIMEOUT_MS)
> > +                                              );
> > +       if (ret == 0) {
> > +               ret = -ETIME;
> > +               dev_err(dev, "wait vpu initialization timout!\n");
> > +               goto OUT_LOAD_FW;
> > +       } else if (-ERESTARTSYS == ret) {
> > +               dev_err(dev, "wait vpu interrupted by a signal!\n");
> > +               goto OUT_LOAD_FW;
> > +       }
> > +
> > +       ret = 0;
> > +       dev_info(dev, "vpu is ready. Fw version %s\n", run->fw_ver);
> > +
> > +OUT_LOAD_FW:
> > +       vpu_clock_disable(vpu);
> > +       mutex_unlock(&vpu->vpu_mutex);
> > +
> > +       return ret;
> > +}
> > +
> > +int vpu_compare_version(struct platform_device *pdev,
> > +                       const char *expected_version)
> > +{
> > +       struct mtk_vpu *vpu = platform_get_drvdata(pdev);
> > +       int cur_major, cur_minor, cur_build, cur_rel, cur_ver_num;
> > +       int major, minor, build, rel, ver_num;
> > +       char *cur_version = vpu->run.fw_ver;
> > +
> > +       cur_ver_num = sscanf(cur_version, "%d.%d.%d-rc%d",
> > +                            &cur_major, &cur_minor, &cur_build, &cur_rel);
> > +       if (cur_ver_num < 3)
> > +               return -1;
> > +       ver_num = sscanf(expected_version, "%d.%d.%d-rc%d",
> > +                        &major, &minor, &build, &rel);
> > +       if (ver_num < 3)
> > +               return -1;
> > +
> > +       if (cur_major < major)
> > +               return -1;
> > +       if (cur_major > major)
> > +               return 1;
> > +
> > +       if (cur_minor < minor)
> > +               return -1;
> > +       if (cur_minor > minor)
> > +               return 1;
> > +
> > +       if (cur_build < build)
> > +               return -1;
> > +       if (cur_build > build)
> > +               return 1;
> > +
> > +       if (cur_ver_num < ver_num)
> > +               return -1;
> > +       if (cur_ver_num > ver_num)
> > +               return 1;
> > +
> > +       if (ver_num > 3) {
> > +               if (cur_rel < rel)
> > +                       return -1;
> > +               if (cur_rel > rel)
> > +                       return 1;
> > +       }
> > +
> > +       return 0;
> > +}
> > +
> > +static void vpu_init_ipi_handler(void *data, unsigned int len, void *priv)
> > +{
> > +       struct mtk_vpu *vpu = (struct mtk_vpu *)priv;
> > +       struct vpu_run *run = (struct vpu_run *)data;
> > +
> > +       vpu->run.signaled = run->signaled;
> > +       strncpy(vpu->run.fw_ver, run->fw_ver, VPU_FW_VER_LEN);
> > +       vpu->run.enc_capability = run->enc_capability;
> > +       wake_up_interruptible(&vpu->run.wq);
> > +}
> > +
> > +#ifdef CONFIG_DEBUG_FS
> > +static int vpu_debug_open(struct inode *inode, struct file *file)
> > +{
> > +       file->private_data = inode->i_private;
> > +       return 0;
> > +}
> > +
> > +static ssize_t vpu_debug_read(struct file *file, char __user *user_buf,
> > +                             size_t count, loff_t *ppos)
> > +{
> > +       char buf[256];
> > +       unsigned int len;
> > +       unsigned int running, pc, vpu_to_host, host_to_vpu, wdt;
> > +       int ret;
> > +       struct device *dev = file->private_data;
> > +       struct mtk_vpu *vpu = dev_get_drvdata(dev);
> > +
> > +       ret = vpu_clock_enable(vpu);
> > +       if (ret) {
> > +               dev_err(vpu->dev, "[VPU] enable clock failed %d\n", ret);
> > +               return 0;
> > +       }
> > +
> > +       /* vpu register status */
> > +       running = vpu_running(vpu);
> > +       pc = vpu_cfg_readl(vpu, VPU_PC_REG);
> > +       wdt = vpu_cfg_readl(vpu, VPU_WDT_REG);
> > +       host_to_vpu = vpu_cfg_readl(vpu, HOST_TO_VPU);
> > +       vpu_to_host = vpu_cfg_readl(vpu, VPU_TO_HOST);
> > +       vpu_clock_disable(vpu);
> > +
> > +       if (running) {
> > +               len = sprintf(buf, "VPU is running\n\n"
> > +               "FW Version: %s\n"
> > +               "PC: 0x%x\n"
> > +               "WDT: 0x%x\n"
> > +               "Host to VPU: 0x%x\n"
> > +               "VPU to Host: 0x%x\n",
> > +               vpu->run.fw_ver, pc, wdt,
> > +               host_to_vpu, vpu_to_host);
> > +       } else {
> > +               len = sprintf(buf, "VPU not running\n");
> > +       }
> > +
> > +       return simple_read_from_buffer(user_buf, count, ppos, buf, len);
> > +}
> > +
> > +static const struct file_operations vpu_debug_fops = {
> > +       .open = vpu_debug_open,
> > +       .read = vpu_debug_read,
> > +};
> > +#endif /* CONFIG_DEBUG_FS */
> > +
> > +static void vpu_free_ext_mem(struct mtk_vpu *vpu, u8 fw_type)
> > +{
> > +       struct device *dev = vpu->dev;
> > +       size_t fw_ext_size = fw_type ? VPU_EXT_D_SIZE : VPU_EXT_P_SIZE;
> > +
> > +       dma_free_coherent(dev, fw_ext_size, vpu->extmem[fw_type].va,
> > +                         vpu->extmem[fw_type].pa);
> > +}
> > +
> > +static int vpu_alloc_ext_mem(struct mtk_vpu *vpu, u32 fw_type)
> > +{
> > +       struct device *dev = vpu->dev;
> > +       size_t fw_ext_size = fw_type ? VPU_EXT_D_SIZE : VPU_EXT_P_SIZE;
> > +       u32 vpu_ext_mem0 = fw_type ? VPU_DMEM_EXT0_ADDR : VPU_PMEM_EXT0_ADDR;
> > +       u32 vpu_ext_mem1 = fw_type ? VPU_DMEM_EXT1_ADDR : VPU_PMEM_EXT1_ADDR;
> > +       u32 offset_4gb = vpu->enable_4GB ? 0x40000000 : 0;
> > +
> > +       vpu->extmem[fw_type].va = dma_alloc_coherent(dev,
> > +                                              fw_ext_size,
> > +                                              &vpu->extmem[fw_type].pa,
> > +                                              GFP_KERNEL);
> > +       if (!vpu->extmem[fw_type].va) {
> > +               dev_err(dev, "Failed to allocate the extended program memory\n");
> > +               return PTR_ERR(vpu->extmem[fw_type].va);
> > +       }
> > +
> > +       /* Disable extend0. Enable extend1 */
> > +       vpu_cfg_writel(vpu, 0x1, vpu_ext_mem0);
> > +       vpu_cfg_writel(vpu, (vpu->extmem[fw_type].pa & 0xFFFFF000) + offset_4gb,
> > +                      vpu_ext_mem1);
> > +
> > +       dev_info(dev, "%s extend memory phy=0x%llx virt=0x%p\n",
> > +                fw_type ? "Data" : "Program",
> > +                (unsigned long long)vpu->extmem[fw_type].pa,
> > +                vpu->extmem[fw_type].va);
> > +
> > +       return 0;
> > +}
> > +
> > +static void vpu_ipi_handler(struct mtk_vpu *vpu)
> > +{
> > +       struct share_obj *rcv_obj = vpu->recv_buf;
> > +       struct vpu_ipi_desc *ipi_desc = vpu->ipi_desc;
> > +
> > +       if (rcv_obj->id < IPI_MAX && ipi_desc[rcv_obj->id].handler) {
> check rcv_obj->id >= 0

The variable type of "rcv_obj->id" is u32. I think we don't need to
check this range.Thanks.

> > +               ipi_desc[rcv_obj->id].handler(rcv_obj->share_buf,
> > +                                             rcv_obj->len,
> > +                                             ipi_desc[rcv_obj->id].priv);
> > +               if (rcv_obj->id > IPI_VPU_INIT) {
> > +                       vpu->ipi_id_ack[rcv_obj->id] = true;
> > +                       wake_up_interruptible(&vpu->ack_wq);
> > +               }
> > +       } else {
> > +               dev_err(vpu->dev, "No such ipi id = %d\n", rcv_obj->id);
> > +       }
> > +}
> > +
> > +static int vpu_ipi_init(struct mtk_vpu *vpu)
> > +{
> > +       /* Disable VPU to host interrupt */
> > +       vpu_cfg_writel(vpu, 0x0, VPU_TO_HOST);
> > +
> > +       /* shared buffer initialization */
> > +       vpu->recv_buf = (struct share_obj *)(vpu->reg.tcm + VPU_DTCM_OFFSET);
> > +       vpu->send_buf = vpu->recv_buf + 1;
> > +       memset(vpu->recv_buf, 0, sizeof(struct share_obj));
> > +       memset(vpu->send_buf, 0, sizeof(struct share_obj));
> > +       mutex_init(&vpu->vpu_mutex);
> > +
> > +       return 0;
> > +}
> > +
> > +static irqreturn_t vpu_irq_handler(int irq, void *priv)
> > +{
> > +       struct mtk_vpu *vpu = priv;
> > +       u32 vpu_to_host;
> > +       int ret;
> > +
> > +       /*
> > +        * Clock should have been enabled already.
> > +        * Enable again in case vpu_ipi_send times out
> > +        * and has disabled the clock.
> > +        */
> > +       ret = clk_enable(vpu->clk);
> > +       if (ret) {
> > +               dev_err(vpu->dev, "[VPU] enable clock failed %d\n", ret);
> > +               return IRQ_NONE;
> > +       }
> > +       vpu_to_host = vpu_cfg_readl(vpu, VPU_TO_HOST);
> > +       if (vpu_to_host & VPU_IPC_INT) {
> > +               vpu_ipi_handler(vpu);
> > +       } else {
> > +               dev_err(vpu->dev, "vpu watchdog timeout! 0x%x", vpu_to_host);
> > +               if (vpu->wdt.wq)
> Remove. mtk_vpu_probe makes sure wdt.wq is not NULL.
I will change this in next version. Thanks.

> > +                       queue_work(vpu->wdt.wq, &vpu->wdt.ws);
> > +       }
> > +
> > +       /* VPU won't send another interrupt until we set VPU_TO_HOST to 0. */
> > +       vpu_cfg_writel(vpu, 0x0, VPU_TO_HOST);
> > +       clk_disable(vpu->clk);
> > +
> > +       return IRQ_HANDLED;
> > +}
> > +
> > +#ifdef CONFIG_DEBUG_FS
> > +static struct dentry *vpu_debugfs;
> > +#endif
> > +static int mtk_vpu_probe(struct platform_device *pdev)
> > +{
> > +       struct mtk_vpu *vpu;
> > +       struct device *dev;
> > +       struct resource *res;
> > +       int ret = 0;
> > +
> > +       dev_dbg(&pdev->dev, "initialization\n");
> > +
> > +       dev = &pdev->dev;
> > +       vpu = devm_kzalloc(dev, sizeof(*vpu), GFP_KERNEL);
> > +       if (!vpu)
> > +               return -ENOMEM;
> > +
> > +       vpu->dev = &pdev->dev;
> > +       res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "tcm");
> > +       vpu->reg.tcm = devm_ioremap_resource(dev, res);
> > +       if (IS_ERR(vpu->reg.tcm)) {
> > +               dev_err(dev, "devm_ioremap_resource vpu tcm failed.\n");
> > +               return PTR_ERR(vpu->reg.tcm);
> > +       }
> > +
> > +       res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "cfg_reg");
> > +       vpu->reg.cfg = devm_ioremap_resource(dev, res);
> > +       if (IS_ERR(vpu->reg.cfg)) {
> > +               dev_err(dev, "devm_ioremap_resource vpu cfg failed.\n");
> > +               return PTR_ERR(vpu->reg.cfg);
> > +       }
> > +
> > +       /* Get VPU clock */
> > +       vpu->clk = devm_clk_get(dev, "main");
> > +       if (!vpu->clk) {
> > +               dev_err(dev, "get vpu clock failed\n");
> > +               return -EINVAL;
> > +       }
> > +
> > +       platform_set_drvdata(pdev, vpu);
> > +
> > +       ret = clk_prepare(vpu->clk);
> > +       if (ret) {
> > +               dev_err(dev, "prepare vpu clock failed\n");
> > +               return ret;
> > +       }
> > +
> > +       /* VPU watchdog */
> > +       vpu->wdt.wq = create_singlethread_workqueue("vpu_wdt");
> > +       if (!vpu->wdt.wq) {
> > +               dev_err(dev, "initialize wdt workqueue failed\n");
> > +               return -ENOMEM;
> > +       }
> > +       INIT_WORK(&vpu->wdt.ws, vpu_wdt_reset_func);
> > +
> > +       ret = vpu_clock_enable(vpu);
> > +       if (ret) {
> > +               dev_err(dev, "enable vpu clock failed\n");
> > +               goto workqueue_destroy;
> > +       }
> > +
> > +       dev_dbg(dev, "vpu ipi init\n");
> > +       ret = vpu_ipi_init(vpu);
> > +       if (ret) {
> > +               dev_err(dev, "Failed to init ipi\n");
> > +               goto disable_vpu_clk;
> > +       }
> > +
> > +       /* register vpu initialization IPI */
> > +       ret = vpu_ipi_register(pdev, IPI_VPU_INIT, vpu_init_ipi_handler,
> > +                              "vpu_init", vpu);
> > +       if (ret) {
> > +               dev_err(dev, "Failed to register IPI_VPU_INIT\n");
> > +               goto vpu_mutex_destroy;
> > +       }
> > +
> > +#ifdef CONFIG_DEBUG_FS
> > +       vpu_debugfs = debugfs_create_file("mtk_vpu", S_IRUGO, NULL, (void *)dev,
> > +                                         &vpu_debug_fops);
> > +       if (!vpu_debugfs) {
> > +               ret = -ENOMEM;
> > +               goto cleanup_ipi;
> > +       }
> > +#endif
> > +
> > +       /* Set PTCM to 96K and DTCM to 32K */
> > +       vpu_cfg_writel(vpu, 0x2, VPU_TCM_CFG);
> > +
> > +       vpu->enable_4GB = !!(max_pfn > (0xffffffffUL >> PAGE_SHIFT));
> > +       dev_dbg(dev, "4GB mode %u\n", vpu->enable_4GB);
> > +
> > +       if (vpu->enable_4GB) {
> > +               ret = of_reserved_mem_device_init(dev);
> > +               if (ret)
> > +                       dev_info(dev, "init reserved memory failed\n");
> > +                       /* continue to use dynamic allocation if failed */
> > +       }
> > +
> > +       ret = vpu_alloc_ext_mem(vpu, D_FW);
> > +       if (ret) {
> > +               dev_err(dev, "Allocate DM failed\n");
> > +               goto remove_debugfs;
> > +       }
> > +
> > +       ret = vpu_alloc_ext_mem(vpu, P_FW);
> > +       if (ret) {
> > +               dev_err(dev, "Allocate PM failed\n");
> > +               goto free_d_mem;
> > +       }
> > +
> > +       init_waitqueue_head(&vpu->run.wq);
> > +       init_waitqueue_head(&vpu->ack_wq);
> > +
> > +       res = platform_get_resource(pdev, IORESOURCE_IRQ, 0);
> > +       if (!res) {
> > +               dev_err(dev, "get IRQ resource failed.\n");
> > +               ret = -ENXIO;
> > +               goto free_p_mem;
> > +       }
> > +       vpu->reg.irq = platform_get_irq(pdev, 0);
> > +       ret = devm_request_irq(dev, vpu->reg.irq, vpu_irq_handler, 0,
> > +                              pdev->name, vpu);
> > +       if (ret) {
> > +               dev_err(dev, "failed to request irq\n");
> > +               goto free_p_mem;
> > +       }
> > +
> > +       vpu_clock_disable(vpu);
> > +       dev_dbg(dev, "initialization completed\n");
> > +
> > +       return 0;
> > +
> > +free_p_mem:
> > +       vpu_free_ext_mem(vpu, P_FW);
> > +free_d_mem:
> > +       vpu_free_ext_mem(vpu, D_FW);
> > +remove_debugfs:
> > +       of_reserved_mem_device_release(dev);
> > +#ifdef CONFIG_DEBUG_FS
> > +       debugfs_remove(vpu_debugfs);
> > +cleanup_ipi:
> > +#endif
> > +       memset(vpu->ipi_desc, 0, sizeof(struct vpu_ipi_desc) * IPI_MAX);
> > +vpu_mutex_destroy:
> > +       mutex_destroy(&vpu->vpu_mutex);
> > +disable_vpu_clk:
> > +       vpu_clock_disable(vpu);
> > +workqueue_destroy:
> > +       destroy_workqueue(vpu->wdt.wq);
> > +
> > +       return ret;
> > +}
> > +
> > +static const struct of_device_id mtk_vpu_match[] = {
> > +       {
> > +               .compatible = "mediatek,mt8173-vpu",
> > +       },
> > +       {},
> > +};
> > +MODULE_DEVICE_TABLE(of, mtk_vpu_match);
> > +
> > +static int mtk_vpu_remove(struct platform_device *pdev)
> > +{
> > +       struct mtk_vpu *vpu = platform_get_drvdata(pdev);
> > +
> > +#ifdef CONFIG_DEBUG_FS
> > +       debugfs_remove(vpu_debugfs);
> > +#endif
> > +       if (vpu->wdt.wq) {
> > +               flush_workqueue(vpu->wdt.wq);
> > +               destroy_workqueue(vpu->wdt.wq);
> > +       }
> > +       vpu_free_ext_mem(vpu, P_FW);
> > +       vpu_free_ext_mem(vpu, D_FW);
> > +       mutex_destroy(&vpu->vpu_mutex);
> > +       clk_unprepare(vpu->clk);
> > +
> > +       return 0;
> > +}
> > +
> > +static struct platform_driver mtk_vpu_driver = {
> > +       .probe  = mtk_vpu_probe,
> > +       .remove = mtk_vpu_remove,
> > +       .driver = {
> > +               .name   = "mtk_vpu",
> > +               .owner  = THIS_MODULE,
> > +               .of_match_table = mtk_vpu_match,
> > +       },
> > +};
> > +
> > +module_platform_driver(mtk_vpu_driver);
> > +
> > +MODULE_LICENSE("GPL v2");
> > +MODULE_DESCRIPTION("Mediatek Video Prosessor Unit driver");
> > diff --git a/drivers/media/platform/mtk-vpu/mtk_vpu.h b/drivers/media/platform/mtk-vpu/mtk_vpu.h
> > new file mode 100644
> > index 0000000..d9c3cde
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vpu/mtk_vpu.h
> > @@ -0,0 +1,167 @@
> > +/*
> > +* Copyright (c) 2015 MediaTek Inc.
> > +* Author: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
> > +*
> > +* This program is free software; you can redistribute it and/or modify
> > +* it under the terms of the GNU General Public License version 2 as
> > +* published by the Free Software Foundation.
> > +*
> > +* This program is distributed in the hope that it will be useful,
> > +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> > +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > +* GNU General Public License for more details.
> > +*/
> > +
> > +#ifndef _MTK_VPU_H
> > +#define _MTK_VPU_H
> > +
> > +#include <linux/platform_device.h>
> > +
> > +/**
> > + * VPU (video processor unit) is a tiny processor controlling video hardware
> > + * related to video codec, scaling and color format converting.
> > + * VPU interfaces with other blocks by share memory and interrupt.
> > + **/
> > +
> > +typedef void (*ipi_handler_t) (void *data,
> > +                              unsigned int len,
> > +                              void *priv);
> > +
> > +/**
> > + * enum ipi_id - the id of inter-processor interrupt
> > + *
> > + * @IPI_VPU_INIT:       The interrupt from vpu is to notfiy kernel
> > +                        VPU initialization completed.
> > +                        IPI_VPU_INIT is sent from VPU when firmware is
> > +                        loaded. AP doesn't need to send IPI_VPU_INIT
> > +                        command to VPU.
> > +                        For other IPI below, AP should send the request
> > +                        to VPU to trigger the interrupt.
> > + * @IPI_VENC_H264:      The interrupt from vpu is to notify kernel to
> > +                        handle H264 video encoder job, and vice versa.
> > + * @IPI_VENC_VP8:       The interrupt fro vpu is to notify kernel to
> > +                        handle VP8 video encoder job,, and vice versa.
> > + * @IPI_MAX:            The maximum IPI number
> > + */
> > +
> > +enum ipi_id {
> > +       IPI_VPU_INIT = 0,
> > +       IPI_VENC_H264,
> > +       IPI_VENC_VP8,
> > +       IPI_MAX,
> > +};
> > +
> > +/**
> > + * enum rst_id - reset id to register reset function for VPU watchdog timeout
> > + *
> > + * @VPU_RST_DEC: decoder reset id
> > + * @VPU_RST_ENC: encoder reset id
> > + * @VPU_RST_MDP: MDP (Media Data Path) reset id
> > + * @VPU_RST_MAX: maximum reset id
> > + */
> > +enum rst_id {
> > +       VPU_RST_ENC,
> > +       VPU_RST_DEC,
> > +       VPU_RST_MDP,
> > +       VPU_RST_MAX,
> > +};
> > +
> > +/**
> > + * vpu_ipi_register - register an ipi function
> > + *
> > + * @pdev:      VPU platform device
> > + * @id:                IPI ID
> > + * @handler:   IPI handler
> > + * @name:      IPI name
> > + * @priv:      private data for IPI handler
> > + *
> > + * Register an ipi function to receive ipi interrupt from VPU.
> > + *
> > + * Return: Return 0 if ipi registers successfully, otherwise it is failed.
> > + */
> > +int vpu_ipi_register(struct platform_device *pdev, enum ipi_id id,
> > +                    ipi_handler_t handler, const char *name, void *priv);
> > +
> > +/**
> > + * vpu_ipi_send - send data from AP to vpu.
> > + *
> > + * @pdev:      VPU platform device
> > + * @id:                IPI ID
> > + * @buf:       the data buffer
> > + * @len:       the data buffer length
> > + *
> > + * This function is thread-safe. When this function returns,
> > + * VPU has received the data and starts the processing.
> > + * When the processing completes, IPI handler registered
> > + * by vpu_ipi_register will be called in interrupt context.
> > + *
> > + * Return: Return 0 if sending data successfully, otherwise it is failed.
> > + **/
> > +int vpu_ipi_send(struct platform_device *pdev,
> > +                enum ipi_id id, void *buf,
> > +                unsigned int len);
> > +
> > +/**
> > + * vpu_get_plat_device - get VPU's platform device
> > + *
> > + * @pdev:      the platform device of the module requesting VPU platform
> > + *             device for using VPU API.
> > + *
> > + * Return: Return NULL if it is failed.
> > + * otherwise it is VPU's platform device
> > + **/
> > +struct platform_device *vpu_get_plat_device(struct platform_device *pdev);
> > +
> > +/**
> > + * vpu_wdt_reg_handler - register a VPU watchdog handler
> > + *
> > + * @pdev:               VPU platform device
> > + * @vpu_wdt_reset_func:        the callback reset function
> > + * @private_data:       the private data for reset function
> > + * @rst_id:            reset id
> > + *
> > + * Register a handler performing own tasks when vpu reset by watchdog
> > + *
> > + * Return: Return 0 if the handler is added successfully,
> > + * otherwise it is failed.
> > + *
> > + **/
> > +int vpu_wdt_reg_handler(struct platform_device *pdev,
> > +                       void vpu_wdt_reset_func(void *),
> > +                       void *priv, enum rst_id id);
> > +
> > +/**
> > + * vpu_get_venc_hw_capa - get video encoder hardware capability
> > + *
> > + * @pdev:      VPU platform device
> > + *
> > + * Return: video encoder hardware capability
> > + **/
> > +unsigned int vpu_get_venc_hw_capa(struct platform_device *pdev);
> > +
> > +/**
> > + * vpu_load_firmware - download VPU firmware and boot it
> > + *
> > + * @pdev:      VPU platform device
> > + *
> > + * Return: Return 0 if downloading firmware successfully,
> > + * otherwise it is failed
> > + **/
> > +int vpu_load_firmware(struct platform_device *pdev);
> > +
> > +/**
> > + * vpu_mapping_dm_addr - Mapping DTCM/DMEM to kernel virtual address
> > + *
> > + * @pdev:      VPU platform device
> > + * @dmem_addr: VPU's data memory address
> > + *
> > + * Mapping the VPU's DTCM (Data Tightly-Coupled Memory) /
> > + * DMEM (Data Extended Memory) memory address to
> > + * kernel virtual address.
> > + *
> > + * Return: Return ERR_PTR(-EINVAL) if mapping failed,
> > + * otherwise the mapped kernel virtual address
> > + **/
> > +void *vpu_mapping_dm_addr(struct platform_device *pdev,
> > +                         u32 dtcm_dmem_addr);
> > +#endif /* _MTK_VPU_H */
> > --
> > 1.7.9.5
> >
> > --
> > To unsubscribe from this list: send the line "unsubscribe linux-media" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 102+ messages in thread

* Re: [PATCH v4 2/8] [media] VPU: mediatek: support Mediatek VPU
@ 2016-02-16  9:36         ` andrew-ct chen
  0 siblings, 0 replies; 102+ messages in thread
From: andrew-ct chen @ 2016-02-16  9:36 UTC (permalink / raw)
  To: Wu-Cheng Li (李務誠)
  Cc: Tiffany Lin, Hans Verkuil, daniel.thompson, Rob Herring,
	Mauro Carvalho Chehab, Matthias Brugger, Daniel Kurtz,
	Pawel Osciak, Eddie Huang, Yingjoe Chen, devicetree,
	linux-kernel, linux-arm-kernel, linux-media, linux-mediatek,
	Lin PoChun

On Mon, 2016-02-15 at 21:59 +0800, Wu-Cheng Li (李務誠) wrote:
> On Thu, Feb 4, 2016 at 7:34 PM, Tiffany Lin <tiffany.lin@mediatek.com> wrote:
> > The VPU driver for hw video codec embedded in Mediatek's MT8173 SOCs.
> > It is able to handle video decoding/encoding of in a range of formats.
> > The driver provides with VPU firmware download, memory management and
> > the communication interface between CPU and VPU.
> > For VPU initialization, it will create virtual memory for CPU access and
> > IOMMU address for vcodec hw device access. When a decode/encode instance
> > opens a device node, vpu driver will download vpu firmware to the device.
> > A decode/encode instant will decode/encode a frame using VPU
> > interface to interrupt vpu to handle decoding/encoding jobs.
> >
> > Signed-off-by: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
> > Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
> > ---
> >  drivers/media/platform/Kconfig           |    9 +
> >  drivers/media/platform/Makefile          |    2 +
> >  drivers/media/platform/mtk-vpu/Makefile  |    1 +
> >  drivers/media/platform/mtk-vpu/mtk_vpu.c |  994 ++++++++++++++++++++++++++++++
> >  drivers/media/platform/mtk-vpu/mtk_vpu.h |  167 +++++
> >  5 files changed, 1173 insertions(+)
> >  create mode 100644 drivers/media/platform/mtk-vpu/Makefile
> >  create mode 100644 drivers/media/platform/mtk-vpu/mtk_vpu.c
> >  create mode 100644 drivers/media/platform/mtk-vpu/mtk_vpu.h
> >
> > diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
> > index ccbc974..ba812d6 100644
> > --- a/drivers/media/platform/Kconfig
> > +++ b/drivers/media/platform/Kconfig
> > @@ -148,6 +148,15 @@ config VIDEO_CODA
> >            Coda is a range of video codec IPs that supports
> >            H.264, MPEG-4, and other video formats.
> >
> > +config VIDEO_MEDIATEK_VPU
> > +       tristate "Mediatek Video Processor Unit"
> > +       depends on VIDEO_DEV && VIDEO_V4L2 && ARCH_MEDIATEK
> > +       ---help---
> > +           This driver provides downloading VPU firmware and
> > +           communicating with VPU. This driver for hw video
> > +           codec embedded in new Mediatek's SOCs. It is able
> > +           to handle video decoding/encoding in a range of formats.
> > +
> >  config VIDEO_MEM2MEM_DEINTERLACE
> >         tristate "Deinterlace support"
> >         depends on VIDEO_DEV && VIDEO_V4L2 && DMA_ENGINE
> > diff --git a/drivers/media/platform/Makefile b/drivers/media/platform/Makefile
> > index efa0295..e5b19c6 100644
> > --- a/drivers/media/platform/Makefile
> > +++ b/drivers/media/platform/Makefile
> > @@ -55,3 +55,5 @@ obj-$(CONFIG_VIDEO_AM437X_VPFE)               += am437x/
> >  obj-$(CONFIG_VIDEO_XILINX)             += xilinx/
> >
> >  ccflags-y += -I$(srctree)/drivers/media/i2c
> > +
> > +obj-$(CONFIG_VIDEO_MEDIATEK_VPU)       += mtk-vpu/
> > diff --git a/drivers/media/platform/mtk-vpu/Makefile b/drivers/media/platform/mtk-vpu/Makefile
> > new file mode 100644
> > index 0000000..d890a66
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vpu/Makefile
> > @@ -0,0 +1 @@
> > +obj-y += mtk_vpu.o
> > diff --git a/drivers/media/platform/mtk-vpu/mtk_vpu.c b/drivers/media/platform/mtk-vpu/mtk_vpu.c
> > new file mode 100644
> > index 0000000..f54fd89
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vpu/mtk_vpu.c
> > @@ -0,0 +1,994 @@
> > +/*
> > +* Copyright (c) 2015 MediaTek Inc.
> > +* Author: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
> > +*
> > +* This program is free software; you can redistribute it and/or modify
> > +* it under the terms of the GNU General Public License version 2 as
> > +* published by the Free Software Foundation.
> > +*
> > +* This program is distributed in the hope that it will be useful,
> > +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> > +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > +* GNU General Public License for more details.
> > +*/
> > +#include <linux/bootmem.h>
> > +#include <linux/clk.h>
> > +#include <linux/debugfs.h>
> > +#include <linux/firmware.h>
> > +#include <linux/interrupt.h>
> > +#include <linux/iommu.h>
> > +#include <linux/module.h>
> > +#include <linux/of_address.h>
> > +#include <linux/of_irq.h>
> > +#include <linux/of_platform.h>
> > +#include <linux/of_reserved_mem.h>
> > +#include <linux/sched.h>
> > +#include <linux/sizes.h>
> > +
> > +#include "mtk_vpu.h"
> > +
> > +/**
> > + * VPU (video processor unit) is a tiny processor controlling video hardware
> > + * related to video codec, scaling and color format converting.
> > + * VPU interfaces with other blocks by share memory and interrupt.
> > + **/
> > +
> > +#define INIT_TIMEOUT_MS                2000U
> > +#define IPI_TIMEOUT_MS         2000U
> > +#define VPU_FW_VER_LEN         16
> > +
> > +/* maximum program/data TCM (Tightly-Coupled Memory) size */
> > +#define VPU_PTCM_SIZE          (96 * SZ_1K)
> > +#define VPU_DTCM_SIZE          (32 * SZ_1K)
> > +/* the offset to get data tcm address */
> > +#define VPU_DTCM_OFFSET                0x18000UL
> > +/* daynamic allocated maximum extended memory size */
> > +#define VPU_EXT_P_SIZE         SZ_1M
> > +#define VPU_EXT_D_SIZE         SZ_4M
> > +/* maximum binary firmware size */
> > +#define VPU_P_FW_SIZE          (VPU_PTCM_SIZE + VPU_EXT_P_SIZE)
> > +#define VPU_D_FW_SIZE          (VPU_DTCM_SIZE + VPU_EXT_D_SIZE)
> > +/* the size of share buffer between Host and  VPU */
> > +#define SHARE_BUF_SIZE         48
> > +
> > +/* binary firmware name */
> > +#define VPU_P_FW               "vpu_p.bin"
> > +#define VPU_D_FW               "vpu_d.bin"
> > +
> > +#define VPU_RESET              0x0
> > +#define VPU_TCM_CFG            0x0008
> > +#define VPU_PMEM_EXT0_ADDR     0x000C
> > +#define VPU_PMEM_EXT1_ADDR     0x0010
> > +#define VPU_TO_HOST            0x001C
> > +#define VPU_DMEM_EXT0_ADDR     0x0014
> > +#define VPU_DMEM_EXT1_ADDR     0x0018
> > +#define HOST_TO_VPU            0x0024
> > +#define VPU_PC_REG             0x0060
> > +#define VPU_WDT_REG            0x0084
> > +
> > +/* vpu inter-processor communication interrupt */
> > +#define VPU_IPC_INT            BIT(8)
> > +
> > +/**
> > + * enum vpu_fw_type - VPU firmware type
> > + *
> > + * @P_FW: program firmware
> > + * @D_FW: data firmware
> > + *
> > + */
> > +enum vpu_fw_type {
> > +       P_FW,
> > +       D_FW,
> > +};
> > +
> > +/**
> > + * struct vpu_mem - VPU extended program/data memory information
> > + *
> > + * @va:                the kernel virtual memory address of VPU extended memory
> > + * @pa:                the physical memory address of VPU extended memory
> This should be device memory address? If yes, s/physical/device/ and
> s/pa/dma_addr/.

It's really a physical memory since vpu is not attached to IOMMU.It
should be s/dma_addr_t/phys_addr_t for the data type.Thanks.

> > + *
> > + */
> > +struct vpu_mem {
> > +       void *va;
> > +       dma_addr_t pa;
> > +};
> > +
> > +/**
> > + * struct vpu_regs - VPU TCM and configuration registers
> > + *
> > + * @tcm:       the register for VPU Tightly-Coupled Memory
> > + * @cfg:       the register for VPU configuration
> > + * @irq:       the irq number for VPU interrupt
> > + */
> > +struct vpu_regs {
> > +       void __iomem *tcm;
> > +       void __iomem *cfg;
> > +       int irq;
> > +};
> > +
> > +/**
> > + * struct vpu_wdt_handler - VPU watchdog reset handler
> > + *
> > + * @reset_func:        reset handler
> > + * @priv:      private data
> > + */
> > +struct vpu_wdt_handler {
> > +       void (*reset_func)(void *);
> > +       void *priv;
> > +};
> > +
> > +/**
> > + * struct vpu_wdt - VPU watchdog workqueue
> > + *
> > + * @handler:   VPU watchdog reset handler
> > + * @ws:                workstruct for VPU watchdog
> > + * @wq:                workqueue for VPU watchdog
> > + */
> > +struct vpu_wdt {
> > +       struct vpu_wdt_handler handler[VPU_RST_MAX];
> > +       struct work_struct ws;
> > +       struct workqueue_struct *wq;
> > +};
> > +
> > +/**
> > + * struct vpu_run - VPU initialization status
> > + *
> > + * @signaled:          the signal of vpu initialization completed
> > + * @fw_ver:            VPU firmware version
> > + * @enc_capability:    encoder capability
> We should document the meaning of the bit used. Since no bit is used
> by the codec driver, document this is not used for now and the value
> is reserved for future use.

This is not used for now. I will document this in next version. Thanks.

> > + * @wq:                        wait queue for VPU initialization status
> > + */
> > +struct vpu_run {
> > +       u32 signaled;
> > +       char fw_ver[VPU_FW_VER_LEN];
> > +       unsigned int    enc_capability;
> > +       wait_queue_head_t wq;
> > +};
> > +
> > +/**
> > + * struct vpu_ipi_desc - VPU IPI descriptor
> > + *
> > + * @handler:   IPI handler
> > + * @name:      the name of IPI handler
> > + * @priv:      the private data of IPI handler
> > + */
> > +struct vpu_ipi_desc {
> > +       ipi_handler_t handler;
> > +       const char *name;
> > +       void *priv;
> > +};
> > +
> > +/**
> > + * struct share_obj - DTCM (Data Tightly-Coupled Memory) buffer shared with
> > + *                   AP and VPU
> > + *
> > + * @id:                IPI id
> > + * @len:       share buffer length
> > + * @share_buf: share buffer data
> > + */
> > +struct share_obj {
> > +       s32 id;
> > +       u32 len;
> > +       unsigned char share_buf[SHARE_BUF_SIZE];
> > +};
> > +
> > +/**
> > + * struct mtk_vpu - vpu driver data
> > + * @extmem:            VPU extended memory information
> > + * @reg:               VPU TCM and configuration registers
> > + * @run:               VPU initialization status
> > + * @ipi_desc:          VPU IPI descriptor
> > + * @recv_buf:          VPU DTCM share buffer for receiving. The
> > + *                     receive buffer is only accessed in interrupt context.
> > + * @send_buf:          VPU DTCM share buffer for sending
> > + * @dev:               VPU struct device
> > + * @clk:               VPU clock on/off
> > + * @enable_4GB:                VPU 4GB mode on/off
> > + * @vpu_mutex:         protect mtk_vpu (except recv_buf) and ensure only
> > + *                     one client to use VPU service at a time. For example,
> > + *                     suppose a client is using VPU to decode VP8.
> > + *                     If the other client wants to encode VP8,
> > + *                     it has to wait until VP8 decode completes.
> > + * @wdt_refcnt         WDT reference count to make sure the watchdog can be
> > + *                     disabled if no other client is using VPU service
> > + * @ipi_ack_signaled:  The ACKs for registered IPI function sending
> > + *                     interrupt to VPU
> s/ipi_ack_signaled/ipi_id_ack/. Move to after |ack_wq| to be
> consistent with the order of variable declaration.

I will change this in next version. Thanks.

> > + * @ack_wq:            The wait queue for each codec and mdp. When sleeping
> > + *                     processes wake up, they will check the condition
> > + *                     "ipi_ack_signaled" to run the corresponding action or
> > + *                     go back to sleep.
> > + *
> > + */
> > +struct mtk_vpu {
> > +       struct vpu_mem extmem[2];
> > +       struct vpu_regs reg;
> > +       struct vpu_run run;
> > +       struct vpu_wdt wdt;
> > +       struct vpu_ipi_desc ipi_desc[IPI_MAX];
> > +       struct share_obj *recv_buf;
> > +       struct share_obj *send_buf;
> > +       struct device *dev;
> > +       struct clk *clk;
> > +       bool enable_4GB;
> > +       struct mutex vpu_mutex; /* for protecting vpu data data structure */
> Remove the comment here. It should be documented in function comment above

It will get message "CHECK" from checkpatch script if removing this
comment.

> .
> > +       atomic_t wdt_refcnt;
> > +       wait_queue_head_t ack_wq;
> > +       bool ipi_id_ack[IPI_MAX];
> > +};
> > +
> > +static inline void vpu_cfg_writel(struct mtk_vpu *vpu, u32 val, u32 offset)
> > +{
> > +       writel(val, vpu->reg.cfg + offset);
> > +}
> > +
> > +static inline u32 vpu_cfg_readl(struct mtk_vpu *vpu, u32 offset)
> > +{
> > +       return readl(vpu->reg.cfg + offset);
> > +}
> > +
> > +static inline bool vpu_running(struct mtk_vpu *vpu)
> > +{
> > +       return vpu_cfg_readl(vpu, VPU_RESET) & BIT(0);
> > +}
> > +
> > +void vpu_clock_disable(struct mtk_vpu *vpu)
> > +{
> > +       /* Disable VPU watchdog */
> > +       if (atomic_dec_and_test(&vpu->wdt_refcnt))
> Checking wdt_refcnt and doing vpu_cfg_writel should be done
> atomically. For example, if vpu_clock_enable is called after
> atomic_dec_and_test(&vpu->wdt_refcnt) and before vpu_cfg_writel(...),
> the state of watchdog could be wrong. Adding a spinlock to protect
> wdt_refcnt and vpu_cfg_writel together. Then wdt_refcnt doesn't need
> to be atomic_t.

I would use mutex to protect this since this only be called in process
context. I will change this in next version. Thanks.

> > +               vpu_cfg_writel(vpu,
> > +                              vpu_cfg_readl(vpu, VPU_WDT_REG) & ~(1L << 31),
> > +                              VPU_WDT_REG);
> > +
> > +       clk_disable(vpu->clk);
> > +}
> > +
> > +int vpu_clock_enable(struct mtk_vpu *vpu)
> > +{
> > +       int ret;
> > +
> > +       ret = clk_enable(vpu->clk);
> > +       if (ret)
> > +               return ret;
> > +       /* Enable VPU watchdog */
> > +       if (!atomic_read(&vpu->wdt_refcnt))
> Do we need to enable the watchdog other than vpu_ipi_send? The only
> place that AP waits for VPU is in vpu_ipi_send. Right? If yes, can we
> just enable and disable the watchdog in vpu_ipi_send?

Watchdog also should be enabled during driver initialization and
firmware download.

> > +               vpu_cfg_writel(vpu,
> > +                              vpu_cfg_readl(vpu, VPU_WDT_REG) | (1L << 31),
> > +                              VPU_WDT_REG);
> > +
> > +       atomic_inc(&vpu->wdt_refcnt);
> Same above. atomic_read, vpu_cfg_writel, and atomic_inc should be done
> atomically.

I will change this in next version. Thanks.

> > +
> > +       return ret;
> > +}
> > +
> > +int vpu_ipi_register(struct platform_device *pdev,
> > +                    enum ipi_id id, ipi_handler_t handler,
> > +                    const char *name, void *priv)
> > +{
> > +       struct mtk_vpu *vpu = platform_get_drvdata(pdev);
> > +       struct vpu_ipi_desc *ipi_desc;
> > +
> > +       if (!vpu) {
> > +               dev_err(&pdev->dev, "vpu device in not ready\n");
> > +               return -EPROBE_DEFER;
> > +       }
> > +
> > +       if (id < IPI_MAX && handler) {
> Check if id >= 0 because type of enum is implementation dependent.

I will change this in next version. Thanks.

> > +               ipi_desc = vpu->ipi_desc;
> > +               ipi_desc[id].name = name;
> > +               ipi_desc[id].handler = handler;
> > +               ipi_desc[id].priv = priv;
> > +               return 0;
> > +       }
> > +
> > +       dev_err(&pdev->dev, "register vpu ipi with invalid arguments\n");
> print id

I will change this in next version. Thanks.

> > +       return -EINVAL;
> > +}
> > +
> > +int vpu_ipi_send(struct platform_device *pdev,
> > +                enum ipi_id id, void *buf,
> > +                unsigned int len)
> > +{
> > +       struct mtk_vpu *vpu = platform_get_drvdata(pdev);
> > +       struct share_obj *send_obj = vpu->send_buf;
> > +       unsigned long timeout;
> > +       int ret = 0;
> > +
> > +       if (id <= IPI_VPU_INIT || id >= IPI_MAX ||
> > +           len > sizeof(send_obj->share_buf) || !buf) {
> > +               dev_err(vpu->dev, "failed to send ipi message\n");
> > +               return -EINVAL;
> > +       }
> > +
> > +       ret = vpu_clock_enable(vpu);
> > +       if (ret) {
> > +               dev_err(vpu->dev, "failed to enable vpu clock\n");
> > +               return ret;
> > +       }
> > +       if (!vpu_running(vpu)) {
> > +               dev_err(vpu->dev, "vpu_ipi_send: VPU is not running\n");
> > +               ret = -EINVAL;
> > +               goto clock_disable;
> > +       }
> > +
> > +       mutex_lock(&vpu->vpu_mutex);
> > +
> > +        /* Wait until VPU receives the last command */
> > +       timeout = jiffies + msecs_to_jiffies(IPI_TIMEOUT_MS);
> > +       do {
> > +               if (time_after(jiffies, timeout)) {
> > +                       dev_err(vpu->dev, "vpu_ipi_send: IPI timeout!\n");
> > +                       ret = -EIO;
> > +                       goto mut_unlock;
> > +               }
> > +       } while (vpu_cfg_readl(vpu, HOST_TO_VPU));
> > +
> > +       memcpy((void *)send_obj->share_buf, buf, len);
> > +       send_obj->len = len;
> > +       send_obj->id = id;
> > +
> > +       vpu->ipi_id_ack[id] = false;
> > +       /* send the command to VPU */
> > +       vpu_cfg_writel(vpu, 0x1, HOST_TO_VPU);
> > +
> > +       mutex_unlock(&vpu->vpu_mutex);
> > +
> > +       /* wait for VPU's ACK */
> > +       timeout = msecs_to_jiffies(IPI_TIMEOUT_MS);
> > +       ret = wait_event_interruptible_timeout(vpu->ack_wq,
> > +                                              vpu->ipi_id_ack[id], timeout);
> > +       vpu->ipi_id_ack[id] = false;
> > +       if (ret == 0) {
> > +               dev_err(vpu->dev, "vpu ipi %d ack time out !", id);
> > +               ret = -EIO;
> > +               goto clock_disable;
> > +       } else if (-ERESTARTSYS == ret) {
> > +               dev_err(vpu->dev, "vpu ipi %d ack wait interrupted by a signal",
> > +                       id);
> > +               ret = -ERESTARTSYS;
> > +               goto clock_disable;
> > +       }
> > +       vpu_clock_disable(vpu);
> > +
> > +       return 0;
> > +
> > +mut_unlock:
> > +       vpu->ipi_id_ack[id] = false;
> I don't see why we need to set it to false here. Remove if not needed.
I will remove this in next version. Thanks.
> > +       mutex_unlock(&vpu->vpu_mutex);
> > +clock_disable:
> > +       vpu_clock_disable(vpu);
> > +
> > +       return ret;
> > +}
> > +
> > +static void vpu_wdt_reset_func(struct work_struct *ws)
> > +{
> > +       struct vpu_wdt *wdt = container_of(ws, struct vpu_wdt, ws);
> > +       struct mtk_vpu *vpu = container_of(wdt, struct mtk_vpu, wdt);
> > +       struct vpu_wdt_handler *handler = wdt->handler;
> > +       int index, ret;
> > +
> > +       dev_info(vpu->dev, "vpu reset\n");
> > +       mutex_lock(&vpu->vpu_mutex);
> > +       ret = vpu_clock_enable(vpu);
> > +       if (ret) {
> This is missing mutex_unlock. Move mutex_lock right before
> vpu_cfg_writel(vpu, 0x0, VPU_RESET);. So we don't need to unlock when
> vpu_clock_enable fails.

I will change this in next version. Thanks.

> > +               dev_err(vpu->dev, "[VPU] wdt enables clock failed %d\n", ret);
> > +               return;
> > +       }
> > +       vpu_cfg_writel(vpu, 0x0, VPU_RESET);
> > +       vpu_clock_disable(vpu);
> > +       mutex_unlock(&vpu->vpu_mutex);
> > +
> > +       for (index = 0; index < VPU_RST_MAX; index++) {
> > +               if (handler[index].reset_func) {
> > +                       handler[index].reset_func(handler[index].priv);
> > +                       dev_dbg(vpu->dev, "wdt handler func %d\n", index);
> > +               }
> > +       }
> > +}
> > +
> > +int vpu_wdt_reg_handler(struct platform_device *pdev,
> > +                       void wdt_reset(void *),
> > +                       void *priv, enum rst_id id)
> > +{
> > +       struct mtk_vpu *vpu = platform_get_drvdata(pdev);
> > +       struct vpu_wdt_handler *handler = vpu->wdt.handler;
> > +
> > +       if (!vpu) {
> > +               dev_err(vpu->dev, "vpu device in not ready\n");
> > +               return -EPROBE_DEFER;
> > +       }
> > +
> > +       if (id < VPU_RST_MAX && wdt_reset != NULL) {
> Check id >= 0

I will change this in next version. Thanks.

> > +               dev_dbg(vpu->dev, "wdt register id %d\n", id);
> > +               mutex_lock(&vpu->vpu_mutex);
> > +               handler[id].reset_func = wdt_reset;
> > +               handler[id].priv = priv;
> > +               mutex_unlock(&vpu->vpu_mutex);
> > +               return 0;
> > +       }
> > +
> > +       dev_err(vpu->dev, "register vpu wdt handler failed\n");
> > +       return -EINVAL;
> > +}
> > +
> > +unsigned int vpu_get_venc_hw_capa(struct platform_device *pdev)
> > +{
> > +       struct mtk_vpu *vpu = platform_get_drvdata(pdev);
> > +
> > +       return vpu->run.enc_capability;
> > +}
> > +
> > +void *vpu_mapping_dm_addr(struct platform_device *pdev,
> > +                         u32 dtcm_dmem_addr)
> > +{
> > +       struct mtk_vpu *vpu = platform_get_drvdata(pdev);
> > +
> > +       if (!dtcm_dmem_addr ||
> > +           (dtcm_dmem_addr > (VPU_DTCM_SIZE + VPU_EXT_D_SIZE))) {
> > +               dev_err(vpu->dev, "invalid virtual data memory address\n");
> > +               return ERR_PTR(-EINVAL);
> > +       }
> > +
> > +       if (dtcm_dmem_addr < VPU_DTCM_SIZE)
> > +               return dtcm_dmem_addr + vpu->reg.tcm + VPU_DTCM_OFFSET;
> > +
> > +       return vpu->extmem[D_FW].va + (dtcm_dmem_addr - VPU_DTCM_SIZE);
> > +}
> > +
> > +struct platform_device *vpu_get_plat_device(struct platform_device *pdev)
> > +{
> > +       struct device *dev = &pdev->dev;
> > +       struct device_node *vpu_node;
> > +       struct platform_device *vpu_pdev;
> > +
> > +       vpu_node = of_parse_phandle(dev->of_node, "mediatek,vpu", 0);
> > +       if (!vpu_node) {
> > +               dev_err(dev, "can't get vpu node\n");
> > +               return NULL;
> > +       }
> > +
> > +       vpu_pdev = of_find_device_by_node(vpu_node);
> > +       if (WARN_ON(!vpu_pdev)) {
> > +               dev_err(dev, "vpu pdev failed\n");
> > +               of_node_put(vpu_node);
> > +               return NULL;
> > +       }
> > +
> > +       return vpu_pdev;
> > +}
> > +
> > +/* load vpu program/data memory */
> > +static int load_requested_vpu(struct mtk_vpu *vpu,
> > +                             const struct firmware *vpu_fw,
> > +                             u8 fw_type)
> > +{
> > +       size_t tcm_size = fw_type ? VPU_DTCM_SIZE : VPU_PTCM_SIZE;
> > +       size_t fw_size = fw_type ? VPU_D_FW_SIZE : VPU_P_FW_SIZE;
> > +       char *fw_name = fw_type ? VPU_D_FW : VPU_P_FW;
> > +       size_t dl_size = 0;
> > +       size_t extra_fw_size = 0;
> > +       void *dest;
> > +       int ret;
> > +
> > +       ret = request_firmware(&vpu_fw, fw_name, vpu->dev);
> > +       if (ret < 0) {
> > +               dev_err(vpu->dev, "Failed to load %s, %d\n", fw_name, ret);
> > +               return ret;
> > +       }
> > +       dl_size = vpu_fw->size;
> > +       if (dl_size > fw_size) {
> > +               dev_err(vpu->dev, "fw %s size %zu is abnormal\n", fw_name,
> > +                       dl_size);
> > +               release_firmware(vpu_fw);
> > +               return  -EFBIG;
> > +       }
> > +       dev_dbg(vpu->dev, "Downloaded fw %s size: %zu.\n",
> > +               fw_name,
> > +               dl_size);
> > +       /* reset VPU */
> > +       vpu_cfg_writel(vpu, 0x0, VPU_RESET);
> > +
> > +       /* handle extended firmware size */
> > +       if (dl_size > tcm_size) {
> > +               dev_dbg(vpu->dev, "fw size %lx > limited fw size %lx\n",
> > +                       dl_size, tcm_size);
> > +               extra_fw_size = dl_size - tcm_size;
> > +               dev_dbg(vpu->dev, "extra_fw_size %lx\n", extra_fw_size);
> > +               dl_size = tcm_size;
> > +       }
> > +       dest = vpu->reg.tcm;
> > +       if (fw_type == D_FW)
> > +               dest += VPU_DTCM_OFFSET;
> > +       memcpy(dest, vpu_fw->data, dl_size);
> > +       /* download to extended memory if need */
> > +       if (extra_fw_size > 0) {
> > +               dest = vpu->extmem[fw_type].va;
> > +               dev_dbg(vpu->dev, "download extended memory type %x\n",
> > +                       fw_type);
> > +               memcpy(dest, vpu_fw->data + tcm_size, extra_fw_size);
> > +       }
> > +
> > +       release_firmware(vpu_fw);
> > +
> > +       return 0;
> > +}
> > +
> > +int vpu_load_firmware(struct platform_device *pdev)
> > +{
> > +       struct mtk_vpu *vpu = platform_get_drvdata(pdev);
> > +       struct device *dev = &pdev->dev;
> > +       struct vpu_run *run = &vpu->run;
> > +       const struct firmware *vpu_fw;
> > +       int ret;
> > +
> > +       if (!pdev) {
> > +               dev_err(dev, "VPU platform device is invalid\n");
> > +               return -EINVAL;
> > +       }
> > +
> > +       mutex_lock(&vpu->vpu_mutex);
> > +
> > +       ret = vpu_clock_enable(vpu);
> > +       if (ret) {
> > +               dev_err(dev, "enable clock failed %d\n", ret);
> > +               goto OUT_LOAD_FW;
> > +       }
> > +
> > +       if (vpu_running(vpu)) {
> > +               vpu_clock_disable(vpu);
> > +               mutex_unlock(&vpu->vpu_mutex);
> > +               dev_warn(dev, "vpu is running already\n");
> > +               return 0;
> > +       }
> > +
> > +       run->signaled = false;
> > +       dev_dbg(vpu->dev, "firmware request\n");
> > +       /* Downloading program firmware to device*/
> > +       ret = load_requested_vpu(vpu, vpu_fw, P_FW);
> > +       if (ret < 0) {
> > +               dev_err(dev, "Failed to request %s, %d\n", VPU_P_FW, ret);
> > +               goto OUT_LOAD_FW;
> > +       }
> > +
> > +       /* Downloading data firmware to device */
> > +       ret = load_requested_vpu(vpu, vpu_fw, D_FW);
> > +       if (ret < 0) {
> > +               dev_err(dev, "Failed to request %s, %d\n", VPU_D_FW, ret);
> > +               goto OUT_LOAD_FW;
> > +       }
> > +
> > +       /* boot up vpu */
> > +       vpu_cfg_writel(vpu, 0x1, VPU_RESET);
> > +
> > +       ret = wait_event_interruptible_timeout(run->wq,
> > +                                              run->signaled,
> > +                                              msecs_to_jiffies(INIT_TIMEOUT_MS)
> > +                                              );
> > +       if (ret == 0) {
> > +               ret = -ETIME;
> > +               dev_err(dev, "wait vpu initialization timout!\n");
> > +               goto OUT_LOAD_FW;
> > +       } else if (-ERESTARTSYS == ret) {
> > +               dev_err(dev, "wait vpu interrupted by a signal!\n");
> > +               goto OUT_LOAD_FW;
> > +       }
> > +
> > +       ret = 0;
> > +       dev_info(dev, "vpu is ready. Fw version %s\n", run->fw_ver);
> > +
> > +OUT_LOAD_FW:
> > +       vpu_clock_disable(vpu);
> > +       mutex_unlock(&vpu->vpu_mutex);
> > +
> > +       return ret;
> > +}
> > +
> > +int vpu_compare_version(struct platform_device *pdev,
> > +                       const char *expected_version)
> > +{
> > +       struct mtk_vpu *vpu = platform_get_drvdata(pdev);
> > +       int cur_major, cur_minor, cur_build, cur_rel, cur_ver_num;
> > +       int major, minor, build, rel, ver_num;
> > +       char *cur_version = vpu->run.fw_ver;
> > +
> > +       cur_ver_num = sscanf(cur_version, "%d.%d.%d-rc%d",
> > +                            &cur_major, &cur_minor, &cur_build, &cur_rel);
> > +       if (cur_ver_num < 3)
> > +               return -1;
> > +       ver_num = sscanf(expected_version, "%d.%d.%d-rc%d",
> > +                        &major, &minor, &build, &rel);
> > +       if (ver_num < 3)
> > +               return -1;
> > +
> > +       if (cur_major < major)
> > +               return -1;
> > +       if (cur_major > major)
> > +               return 1;
> > +
> > +       if (cur_minor < minor)
> > +               return -1;
> > +       if (cur_minor > minor)
> > +               return 1;
> > +
> > +       if (cur_build < build)
> > +               return -1;
> > +       if (cur_build > build)
> > +               return 1;
> > +
> > +       if (cur_ver_num < ver_num)
> > +               return -1;
> > +       if (cur_ver_num > ver_num)
> > +               return 1;
> > +
> > +       if (ver_num > 3) {
> > +               if (cur_rel < rel)
> > +                       return -1;
> > +               if (cur_rel > rel)
> > +                       return 1;
> > +       }
> > +
> > +       return 0;
> > +}
> > +
> > +static void vpu_init_ipi_handler(void *data, unsigned int len, void *priv)
> > +{
> > +       struct mtk_vpu *vpu = (struct mtk_vpu *)priv;
> > +       struct vpu_run *run = (struct vpu_run *)data;
> > +
> > +       vpu->run.signaled = run->signaled;
> > +       strncpy(vpu->run.fw_ver, run->fw_ver, VPU_FW_VER_LEN);
> > +       vpu->run.enc_capability = run->enc_capability;
> > +       wake_up_interruptible(&vpu->run.wq);
> > +}
> > +
> > +#ifdef CONFIG_DEBUG_FS
> > +static int vpu_debug_open(struct inode *inode, struct file *file)
> > +{
> > +       file->private_data = inode->i_private;
> > +       return 0;
> > +}
> > +
> > +static ssize_t vpu_debug_read(struct file *file, char __user *user_buf,
> > +                             size_t count, loff_t *ppos)
> > +{
> > +       char buf[256];
> > +       unsigned int len;
> > +       unsigned int running, pc, vpu_to_host, host_to_vpu, wdt;
> > +       int ret;
> > +       struct device *dev = file->private_data;
> > +       struct mtk_vpu *vpu = dev_get_drvdata(dev);
> > +
> > +       ret = vpu_clock_enable(vpu);
> > +       if (ret) {
> > +               dev_err(vpu->dev, "[VPU] enable clock failed %d\n", ret);
> > +               return 0;
> > +       }
> > +
> > +       /* vpu register status */
> > +       running = vpu_running(vpu);
> > +       pc = vpu_cfg_readl(vpu, VPU_PC_REG);
> > +       wdt = vpu_cfg_readl(vpu, VPU_WDT_REG);
> > +       host_to_vpu = vpu_cfg_readl(vpu, HOST_TO_VPU);
> > +       vpu_to_host = vpu_cfg_readl(vpu, VPU_TO_HOST);
> > +       vpu_clock_disable(vpu);
> > +
> > +       if (running) {
> > +               len = sprintf(buf, "VPU is running\n\n"
> > +               "FW Version: %s\n"
> > +               "PC: 0x%x\n"
> > +               "WDT: 0x%x\n"
> > +               "Host to VPU: 0x%x\n"
> > +               "VPU to Host: 0x%x\n",
> > +               vpu->run.fw_ver, pc, wdt,
> > +               host_to_vpu, vpu_to_host);
> > +       } else {
> > +               len = sprintf(buf, "VPU not running\n");
> > +       }
> > +
> > +       return simple_read_from_buffer(user_buf, count, ppos, buf, len);
> > +}
> > +
> > +static const struct file_operations vpu_debug_fops = {
> > +       .open = vpu_debug_open,
> > +       .read = vpu_debug_read,
> > +};
> > +#endif /* CONFIG_DEBUG_FS */
> > +
> > +static void vpu_free_ext_mem(struct mtk_vpu *vpu, u8 fw_type)
> > +{
> > +       struct device *dev = vpu->dev;
> > +       size_t fw_ext_size = fw_type ? VPU_EXT_D_SIZE : VPU_EXT_P_SIZE;
> > +
> > +       dma_free_coherent(dev, fw_ext_size, vpu->extmem[fw_type].va,
> > +                         vpu->extmem[fw_type].pa);
> > +}
> > +
> > +static int vpu_alloc_ext_mem(struct mtk_vpu *vpu, u32 fw_type)
> > +{
> > +       struct device *dev = vpu->dev;
> > +       size_t fw_ext_size = fw_type ? VPU_EXT_D_SIZE : VPU_EXT_P_SIZE;
> > +       u32 vpu_ext_mem0 = fw_type ? VPU_DMEM_EXT0_ADDR : VPU_PMEM_EXT0_ADDR;
> > +       u32 vpu_ext_mem1 = fw_type ? VPU_DMEM_EXT1_ADDR : VPU_PMEM_EXT1_ADDR;
> > +       u32 offset_4gb = vpu->enable_4GB ? 0x40000000 : 0;
> > +
> > +       vpu->extmem[fw_type].va = dma_alloc_coherent(dev,
> > +                                              fw_ext_size,
> > +                                              &vpu->extmem[fw_type].pa,
> > +                                              GFP_KERNEL);
> > +       if (!vpu->extmem[fw_type].va) {
> > +               dev_err(dev, "Failed to allocate the extended program memory\n");
> > +               return PTR_ERR(vpu->extmem[fw_type].va);
> > +       }
> > +
> > +       /* Disable extend0. Enable extend1 */
> > +       vpu_cfg_writel(vpu, 0x1, vpu_ext_mem0);
> > +       vpu_cfg_writel(vpu, (vpu->extmem[fw_type].pa & 0xFFFFF000) + offset_4gb,
> > +                      vpu_ext_mem1);
> > +
> > +       dev_info(dev, "%s extend memory phy=0x%llx virt=0x%p\n",
> > +                fw_type ? "Data" : "Program",
> > +                (unsigned long long)vpu->extmem[fw_type].pa,
> > +                vpu->extmem[fw_type].va);
> > +
> > +       return 0;
> > +}
> > +
> > +static void vpu_ipi_handler(struct mtk_vpu *vpu)
> > +{
> > +       struct share_obj *rcv_obj = vpu->recv_buf;
> > +       struct vpu_ipi_desc *ipi_desc = vpu->ipi_desc;
> > +
> > +       if (rcv_obj->id < IPI_MAX && ipi_desc[rcv_obj->id].handler) {
> check rcv_obj->id >= 0

The variable type of "rcv_obj->id" is u32. I think we don't need to
check this range.Thanks.

> > +               ipi_desc[rcv_obj->id].handler(rcv_obj->share_buf,
> > +                                             rcv_obj->len,
> > +                                             ipi_desc[rcv_obj->id].priv);
> > +               if (rcv_obj->id > IPI_VPU_INIT) {
> > +                       vpu->ipi_id_ack[rcv_obj->id] = true;
> > +                       wake_up_interruptible(&vpu->ack_wq);
> > +               }
> > +       } else {
> > +               dev_err(vpu->dev, "No such ipi id = %d\n", rcv_obj->id);
> > +       }
> > +}
> > +
> > +static int vpu_ipi_init(struct mtk_vpu *vpu)
> > +{
> > +       /* Disable VPU to host interrupt */
> > +       vpu_cfg_writel(vpu, 0x0, VPU_TO_HOST);
> > +
> > +       /* shared buffer initialization */
> > +       vpu->recv_buf = (struct share_obj *)(vpu->reg.tcm + VPU_DTCM_OFFSET);
> > +       vpu->send_buf = vpu->recv_buf + 1;
> > +       memset(vpu->recv_buf, 0, sizeof(struct share_obj));
> > +       memset(vpu->send_buf, 0, sizeof(struct share_obj));
> > +       mutex_init(&vpu->vpu_mutex);
> > +
> > +       return 0;
> > +}
> > +
> > +static irqreturn_t vpu_irq_handler(int irq, void *priv)
> > +{
> > +       struct mtk_vpu *vpu = priv;
> > +       u32 vpu_to_host;
> > +       int ret;
> > +
> > +       /*
> > +        * Clock should have been enabled already.
> > +        * Enable again in case vpu_ipi_send times out
> > +        * and has disabled the clock.
> > +        */
> > +       ret = clk_enable(vpu->clk);
> > +       if (ret) {
> > +               dev_err(vpu->dev, "[VPU] enable clock failed %d\n", ret);
> > +               return IRQ_NONE;
> > +       }
> > +       vpu_to_host = vpu_cfg_readl(vpu, VPU_TO_HOST);
> > +       if (vpu_to_host & VPU_IPC_INT) {
> > +               vpu_ipi_handler(vpu);
> > +       } else {
> > +               dev_err(vpu->dev, "vpu watchdog timeout! 0x%x", vpu_to_host);
> > +               if (vpu->wdt.wq)
> Remove. mtk_vpu_probe makes sure wdt.wq is not NULL.
I will change this in next version. Thanks.

> > +                       queue_work(vpu->wdt.wq, &vpu->wdt.ws);
> > +       }
> > +
> > +       /* VPU won't send another interrupt until we set VPU_TO_HOST to 0. */
> > +       vpu_cfg_writel(vpu, 0x0, VPU_TO_HOST);
> > +       clk_disable(vpu->clk);
> > +
> > +       return IRQ_HANDLED;
> > +}
> > +
> > +#ifdef CONFIG_DEBUG_FS
> > +static struct dentry *vpu_debugfs;
> > +#endif
> > +static int mtk_vpu_probe(struct platform_device *pdev)
> > +{
> > +       struct mtk_vpu *vpu;
> > +       struct device *dev;
> > +       struct resource *res;
> > +       int ret = 0;
> > +
> > +       dev_dbg(&pdev->dev, "initialization\n");
> > +
> > +       dev = &pdev->dev;
> > +       vpu = devm_kzalloc(dev, sizeof(*vpu), GFP_KERNEL);
> > +       if (!vpu)
> > +               return -ENOMEM;
> > +
> > +       vpu->dev = &pdev->dev;
> > +       res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "tcm");
> > +       vpu->reg.tcm = devm_ioremap_resource(dev, res);
> > +       if (IS_ERR(vpu->reg.tcm)) {
> > +               dev_err(dev, "devm_ioremap_resource vpu tcm failed.\n");
> > +               return PTR_ERR(vpu->reg.tcm);
> > +       }
> > +
> > +       res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "cfg_reg");
> > +       vpu->reg.cfg = devm_ioremap_resource(dev, res);
> > +       if (IS_ERR(vpu->reg.cfg)) {
> > +               dev_err(dev, "devm_ioremap_resource vpu cfg failed.\n");
> > +               return PTR_ERR(vpu->reg.cfg);
> > +       }
> > +
> > +       /* Get VPU clock */
> > +       vpu->clk = devm_clk_get(dev, "main");
> > +       if (!vpu->clk) {
> > +               dev_err(dev, "get vpu clock failed\n");
> > +               return -EINVAL;
> > +       }
> > +
> > +       platform_set_drvdata(pdev, vpu);
> > +
> > +       ret = clk_prepare(vpu->clk);
> > +       if (ret) {
> > +               dev_err(dev, "prepare vpu clock failed\n");
> > +               return ret;
> > +       }
> > +
> > +       /* VPU watchdog */
> > +       vpu->wdt.wq = create_singlethread_workqueue("vpu_wdt");
> > +       if (!vpu->wdt.wq) {
> > +               dev_err(dev, "initialize wdt workqueue failed\n");
> > +               return -ENOMEM;
> > +       }
> > +       INIT_WORK(&vpu->wdt.ws, vpu_wdt_reset_func);
> > +
> > +       ret = vpu_clock_enable(vpu);
> > +       if (ret) {
> > +               dev_err(dev, "enable vpu clock failed\n");
> > +               goto workqueue_destroy;
> > +       }
> > +
> > +       dev_dbg(dev, "vpu ipi init\n");
> > +       ret = vpu_ipi_init(vpu);
> > +       if (ret) {
> > +               dev_err(dev, "Failed to init ipi\n");
> > +               goto disable_vpu_clk;
> > +       }
> > +
> > +       /* register vpu initialization IPI */
> > +       ret = vpu_ipi_register(pdev, IPI_VPU_INIT, vpu_init_ipi_handler,
> > +                              "vpu_init", vpu);
> > +       if (ret) {
> > +               dev_err(dev, "Failed to register IPI_VPU_INIT\n");
> > +               goto vpu_mutex_destroy;
> > +       }
> > +
> > +#ifdef CONFIG_DEBUG_FS
> > +       vpu_debugfs = debugfs_create_file("mtk_vpu", S_IRUGO, NULL, (void *)dev,
> > +                                         &vpu_debug_fops);
> > +       if (!vpu_debugfs) {
> > +               ret = -ENOMEM;
> > +               goto cleanup_ipi;
> > +       }
> > +#endif
> > +
> > +       /* Set PTCM to 96K and DTCM to 32K */
> > +       vpu_cfg_writel(vpu, 0x2, VPU_TCM_CFG);
> > +
> > +       vpu->enable_4GB = !!(max_pfn > (0xffffffffUL >> PAGE_SHIFT));
> > +       dev_dbg(dev, "4GB mode %u\n", vpu->enable_4GB);
> > +
> > +       if (vpu->enable_4GB) {
> > +               ret = of_reserved_mem_device_init(dev);
> > +               if (ret)
> > +                       dev_info(dev, "init reserved memory failed\n");
> > +                       /* continue to use dynamic allocation if failed */
> > +       }
> > +
> > +       ret = vpu_alloc_ext_mem(vpu, D_FW);
> > +       if (ret) {
> > +               dev_err(dev, "Allocate DM failed\n");
> > +               goto remove_debugfs;
> > +       }
> > +
> > +       ret = vpu_alloc_ext_mem(vpu, P_FW);
> > +       if (ret) {
> > +               dev_err(dev, "Allocate PM failed\n");
> > +               goto free_d_mem;
> > +       }
> > +
> > +       init_waitqueue_head(&vpu->run.wq);
> > +       init_waitqueue_head(&vpu->ack_wq);
> > +
> > +       res = platform_get_resource(pdev, IORESOURCE_IRQ, 0);
> > +       if (!res) {
> > +               dev_err(dev, "get IRQ resource failed.\n");
> > +               ret = -ENXIO;
> > +               goto free_p_mem;
> > +       }
> > +       vpu->reg.irq = platform_get_irq(pdev, 0);
> > +       ret = devm_request_irq(dev, vpu->reg.irq, vpu_irq_handler, 0,
> > +                              pdev->name, vpu);
> > +       if (ret) {
> > +               dev_err(dev, "failed to request irq\n");
> > +               goto free_p_mem;
> > +       }
> > +
> > +       vpu_clock_disable(vpu);
> > +       dev_dbg(dev, "initialization completed\n");
> > +
> > +       return 0;
> > +
> > +free_p_mem:
> > +       vpu_free_ext_mem(vpu, P_FW);
> > +free_d_mem:
> > +       vpu_free_ext_mem(vpu, D_FW);
> > +remove_debugfs:
> > +       of_reserved_mem_device_release(dev);
> > +#ifdef CONFIG_DEBUG_FS
> > +       debugfs_remove(vpu_debugfs);
> > +cleanup_ipi:
> > +#endif
> > +       memset(vpu->ipi_desc, 0, sizeof(struct vpu_ipi_desc) * IPI_MAX);
> > +vpu_mutex_destroy:
> > +       mutex_destroy(&vpu->vpu_mutex);
> > +disable_vpu_clk:
> > +       vpu_clock_disable(vpu);
> > +workqueue_destroy:
> > +       destroy_workqueue(vpu->wdt.wq);
> > +
> > +       return ret;
> > +}
> > +
> > +static const struct of_device_id mtk_vpu_match[] = {
> > +       {
> > +               .compatible = "mediatek,mt8173-vpu",
> > +       },
> > +       {},
> > +};
> > +MODULE_DEVICE_TABLE(of, mtk_vpu_match);
> > +
> > +static int mtk_vpu_remove(struct platform_device *pdev)
> > +{
> > +       struct mtk_vpu *vpu = platform_get_drvdata(pdev);
> > +
> > +#ifdef CONFIG_DEBUG_FS
> > +       debugfs_remove(vpu_debugfs);
> > +#endif
> > +       if (vpu->wdt.wq) {
> > +               flush_workqueue(vpu->wdt.wq);
> > +               destroy_workqueue(vpu->wdt.wq);
> > +       }
> > +       vpu_free_ext_mem(vpu, P_FW);
> > +       vpu_free_ext_mem(vpu, D_FW);
> > +       mutex_destroy(&vpu->vpu_mutex);
> > +       clk_unprepare(vpu->clk);
> > +
> > +       return 0;
> > +}
> > +
> > +static struct platform_driver mtk_vpu_driver = {
> > +       .probe  = mtk_vpu_probe,
> > +       .remove = mtk_vpu_remove,
> > +       .driver = {
> > +               .name   = "mtk_vpu",
> > +               .owner  = THIS_MODULE,
> > +               .of_match_table = mtk_vpu_match,
> > +       },
> > +};
> > +
> > +module_platform_driver(mtk_vpu_driver);
> > +
> > +MODULE_LICENSE("GPL v2");
> > +MODULE_DESCRIPTION("Mediatek Video Prosessor Unit driver");
> > diff --git a/drivers/media/platform/mtk-vpu/mtk_vpu.h b/drivers/media/platform/mtk-vpu/mtk_vpu.h
> > new file mode 100644
> > index 0000000..d9c3cde
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vpu/mtk_vpu.h
> > @@ -0,0 +1,167 @@
> > +/*
> > +* Copyright (c) 2015 MediaTek Inc.
> > +* Author: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
> > +*
> > +* This program is free software; you can redistribute it and/or modify
> > +* it under the terms of the GNU General Public License version 2 as
> > +* published by the Free Software Foundation.
> > +*
> > +* This program is distributed in the hope that it will be useful,
> > +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> > +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > +* GNU General Public License for more details.
> > +*/
> > +
> > +#ifndef _MTK_VPU_H
> > +#define _MTK_VPU_H
> > +
> > +#include <linux/platform_device.h>
> > +
> > +/**
> > + * VPU (video processor unit) is a tiny processor controlling video hardware
> > + * related to video codec, scaling and color format converting.
> > + * VPU interfaces with other blocks by share memory and interrupt.
> > + **/
> > +
> > +typedef void (*ipi_handler_t) (void *data,
> > +                              unsigned int len,
> > +                              void *priv);
> > +
> > +/**
> > + * enum ipi_id - the id of inter-processor interrupt
> > + *
> > + * @IPI_VPU_INIT:       The interrupt from vpu is to notfiy kernel
> > +                        VPU initialization completed.
> > +                        IPI_VPU_INIT is sent from VPU when firmware is
> > +                        loaded. AP doesn't need to send IPI_VPU_INIT
> > +                        command to VPU.
> > +                        For other IPI below, AP should send the request
> > +                        to VPU to trigger the interrupt.
> > + * @IPI_VENC_H264:      The interrupt from vpu is to notify kernel to
> > +                        handle H264 video encoder job, and vice versa.
> > + * @IPI_VENC_VP8:       The interrupt fro vpu is to notify kernel to
> > +                        handle VP8 video encoder job,, and vice versa.
> > + * @IPI_MAX:            The maximum IPI number
> > + */
> > +
> > +enum ipi_id {
> > +       IPI_VPU_INIT = 0,
> > +       IPI_VENC_H264,
> > +       IPI_VENC_VP8,
> > +       IPI_MAX,
> > +};
> > +
> > +/**
> > + * enum rst_id - reset id to register reset function for VPU watchdog timeout
> > + *
> > + * @VPU_RST_DEC: decoder reset id
> > + * @VPU_RST_ENC: encoder reset id
> > + * @VPU_RST_MDP: MDP (Media Data Path) reset id
> > + * @VPU_RST_MAX: maximum reset id
> > + */
> > +enum rst_id {
> > +       VPU_RST_ENC,
> > +       VPU_RST_DEC,
> > +       VPU_RST_MDP,
> > +       VPU_RST_MAX,
> > +};
> > +
> > +/**
> > + * vpu_ipi_register - register an ipi function
> > + *
> > + * @pdev:      VPU platform device
> > + * @id:                IPI ID
> > + * @handler:   IPI handler
> > + * @name:      IPI name
> > + * @priv:      private data for IPI handler
> > + *
> > + * Register an ipi function to receive ipi interrupt from VPU.
> > + *
> > + * Return: Return 0 if ipi registers successfully, otherwise it is failed.
> > + */
> > +int vpu_ipi_register(struct platform_device *pdev, enum ipi_id id,
> > +                    ipi_handler_t handler, const char *name, void *priv);
> > +
> > +/**
> > + * vpu_ipi_send - send data from AP to vpu.
> > + *
> > + * @pdev:      VPU platform device
> > + * @id:                IPI ID
> > + * @buf:       the data buffer
> > + * @len:       the data buffer length
> > + *
> > + * This function is thread-safe. When this function returns,
> > + * VPU has received the data and starts the processing.
> > + * When the processing completes, IPI handler registered
> > + * by vpu_ipi_register will be called in interrupt context.
> > + *
> > + * Return: Return 0 if sending data successfully, otherwise it is failed.
> > + **/
> > +int vpu_ipi_send(struct platform_device *pdev,
> > +                enum ipi_id id, void *buf,
> > +                unsigned int len);
> > +
> > +/**
> > + * vpu_get_plat_device - get VPU's platform device
> > + *
> > + * @pdev:      the platform device of the module requesting VPU platform
> > + *             device for using VPU API.
> > + *
> > + * Return: Return NULL if it is failed.
> > + * otherwise it is VPU's platform device
> > + **/
> > +struct platform_device *vpu_get_plat_device(struct platform_device *pdev);
> > +
> > +/**
> > + * vpu_wdt_reg_handler - register a VPU watchdog handler
> > + *
> > + * @pdev:               VPU platform device
> > + * @vpu_wdt_reset_func:        the callback reset function
> > + * @private_data:       the private data for reset function
> > + * @rst_id:            reset id
> > + *
> > + * Register a handler performing own tasks when vpu reset by watchdog
> > + *
> > + * Return: Return 0 if the handler is added successfully,
> > + * otherwise it is failed.
> > + *
> > + **/
> > +int vpu_wdt_reg_handler(struct platform_device *pdev,
> > +                       void vpu_wdt_reset_func(void *),
> > +                       void *priv, enum rst_id id);
> > +
> > +/**
> > + * vpu_get_venc_hw_capa - get video encoder hardware capability
> > + *
> > + * @pdev:      VPU platform device
> > + *
> > + * Return: video encoder hardware capability
> > + **/
> > +unsigned int vpu_get_venc_hw_capa(struct platform_device *pdev);
> > +
> > +/**
> > + * vpu_load_firmware - download VPU firmware and boot it
> > + *
> > + * @pdev:      VPU platform device
> > + *
> > + * Return: Return 0 if downloading firmware successfully,
> > + * otherwise it is failed
> > + **/
> > +int vpu_load_firmware(struct platform_device *pdev);
> > +
> > +/**
> > + * vpu_mapping_dm_addr - Mapping DTCM/DMEM to kernel virtual address
> > + *
> > + * @pdev:      VPU platform device
> > + * @dmem_addr: VPU's data memory address
> > + *
> > + * Mapping the VPU's DTCM (Data Tightly-Coupled Memory) /
> > + * DMEM (Data Extended Memory) memory address to
> > + * kernel virtual address.
> > + *
> > + * Return: Return ERR_PTR(-EINVAL) if mapping failed,
> > + * otherwise the mapped kernel virtual address
> > + **/
> > +void *vpu_mapping_dm_addr(struct platform_device *pdev,
> > +                         u32 dtcm_dmem_addr);
> > +#endif /* _MTK_VPU_H */
> > --
> > 1.7.9.5
> >
> > --
> > To unsubscribe from this list: send the line "unsubscribe linux-media" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 102+ messages in thread

* [PATCH v4 2/8] [media] VPU: mediatek: support Mediatek VPU
@ 2016-02-16  9:36         ` andrew-ct chen
  0 siblings, 0 replies; 102+ messages in thread
From: andrew-ct chen @ 2016-02-16  9:36 UTC (permalink / raw)
  To: linux-arm-kernel

On Mon, 2016-02-15 at 21:59 +0800, Wu-Cheng Li (???) wrote:
> On Thu, Feb 4, 2016 at 7:34 PM, Tiffany Lin <tiffany.lin@mediatek.com> wrote:
> > The VPU driver for hw video codec embedded in Mediatek's MT8173 SOCs.
> > It is able to handle video decoding/encoding of in a range of formats.
> > The driver provides with VPU firmware download, memory management and
> > the communication interface between CPU and VPU.
> > For VPU initialization, it will create virtual memory for CPU access and
> > IOMMU address for vcodec hw device access. When a decode/encode instance
> > opens a device node, vpu driver will download vpu firmware to the device.
> > A decode/encode instant will decode/encode a frame using VPU
> > interface to interrupt vpu to handle decoding/encoding jobs.
> >
> > Signed-off-by: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
> > Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
> > ---
> >  drivers/media/platform/Kconfig           |    9 +
> >  drivers/media/platform/Makefile          |    2 +
> >  drivers/media/platform/mtk-vpu/Makefile  |    1 +
> >  drivers/media/platform/mtk-vpu/mtk_vpu.c |  994 ++++++++++++++++++++++++++++++
> >  drivers/media/platform/mtk-vpu/mtk_vpu.h |  167 +++++
> >  5 files changed, 1173 insertions(+)
> >  create mode 100644 drivers/media/platform/mtk-vpu/Makefile
> >  create mode 100644 drivers/media/platform/mtk-vpu/mtk_vpu.c
> >  create mode 100644 drivers/media/platform/mtk-vpu/mtk_vpu.h
> >
> > diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
> > index ccbc974..ba812d6 100644
> > --- a/drivers/media/platform/Kconfig
> > +++ b/drivers/media/platform/Kconfig
> > @@ -148,6 +148,15 @@ config VIDEO_CODA
> >            Coda is a range of video codec IPs that supports
> >            H.264, MPEG-4, and other video formats.
> >
> > +config VIDEO_MEDIATEK_VPU
> > +       tristate "Mediatek Video Processor Unit"
> > +       depends on VIDEO_DEV && VIDEO_V4L2 && ARCH_MEDIATEK
> > +       ---help---
> > +           This driver provides downloading VPU firmware and
> > +           communicating with VPU. This driver for hw video
> > +           codec embedded in new Mediatek's SOCs. It is able
> > +           to handle video decoding/encoding in a range of formats.
> > +
> >  config VIDEO_MEM2MEM_DEINTERLACE
> >         tristate "Deinterlace support"
> >         depends on VIDEO_DEV && VIDEO_V4L2 && DMA_ENGINE
> > diff --git a/drivers/media/platform/Makefile b/drivers/media/platform/Makefile
> > index efa0295..e5b19c6 100644
> > --- a/drivers/media/platform/Makefile
> > +++ b/drivers/media/platform/Makefile
> > @@ -55,3 +55,5 @@ obj-$(CONFIG_VIDEO_AM437X_VPFE)               += am437x/
> >  obj-$(CONFIG_VIDEO_XILINX)             += xilinx/
> >
> >  ccflags-y += -I$(srctree)/drivers/media/i2c
> > +
> > +obj-$(CONFIG_VIDEO_MEDIATEK_VPU)       += mtk-vpu/
> > diff --git a/drivers/media/platform/mtk-vpu/Makefile b/drivers/media/platform/mtk-vpu/Makefile
> > new file mode 100644
> > index 0000000..d890a66
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vpu/Makefile
> > @@ -0,0 +1 @@
> > +obj-y += mtk_vpu.o
> > diff --git a/drivers/media/platform/mtk-vpu/mtk_vpu.c b/drivers/media/platform/mtk-vpu/mtk_vpu.c
> > new file mode 100644
> > index 0000000..f54fd89
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vpu/mtk_vpu.c
> > @@ -0,0 +1,994 @@
> > +/*
> > +* Copyright (c) 2015 MediaTek Inc.
> > +* Author: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
> > +*
> > +* This program is free software; you can redistribute it and/or modify
> > +* it under the terms of the GNU General Public License version 2 as
> > +* published by the Free Software Foundation.
> > +*
> > +* This program is distributed in the hope that it will be useful,
> > +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> > +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > +* GNU General Public License for more details.
> > +*/
> > +#include <linux/bootmem.h>
> > +#include <linux/clk.h>
> > +#include <linux/debugfs.h>
> > +#include <linux/firmware.h>
> > +#include <linux/interrupt.h>
> > +#include <linux/iommu.h>
> > +#include <linux/module.h>
> > +#include <linux/of_address.h>
> > +#include <linux/of_irq.h>
> > +#include <linux/of_platform.h>
> > +#include <linux/of_reserved_mem.h>
> > +#include <linux/sched.h>
> > +#include <linux/sizes.h>
> > +
> > +#include "mtk_vpu.h"
> > +
> > +/**
> > + * VPU (video processor unit) is a tiny processor controlling video hardware
> > + * related to video codec, scaling and color format converting.
> > + * VPU interfaces with other blocks by share memory and interrupt.
> > + **/
> > +
> > +#define INIT_TIMEOUT_MS                2000U
> > +#define IPI_TIMEOUT_MS         2000U
> > +#define VPU_FW_VER_LEN         16
> > +
> > +/* maximum program/data TCM (Tightly-Coupled Memory) size */
> > +#define VPU_PTCM_SIZE          (96 * SZ_1K)
> > +#define VPU_DTCM_SIZE          (32 * SZ_1K)
> > +/* the offset to get data tcm address */
> > +#define VPU_DTCM_OFFSET                0x18000UL
> > +/* daynamic allocated maximum extended memory size */
> > +#define VPU_EXT_P_SIZE         SZ_1M
> > +#define VPU_EXT_D_SIZE         SZ_4M
> > +/* maximum binary firmware size */
> > +#define VPU_P_FW_SIZE          (VPU_PTCM_SIZE + VPU_EXT_P_SIZE)
> > +#define VPU_D_FW_SIZE          (VPU_DTCM_SIZE + VPU_EXT_D_SIZE)
> > +/* the size of share buffer between Host and  VPU */
> > +#define SHARE_BUF_SIZE         48
> > +
> > +/* binary firmware name */
> > +#define VPU_P_FW               "vpu_p.bin"
> > +#define VPU_D_FW               "vpu_d.bin"
> > +
> > +#define VPU_RESET              0x0
> > +#define VPU_TCM_CFG            0x0008
> > +#define VPU_PMEM_EXT0_ADDR     0x000C
> > +#define VPU_PMEM_EXT1_ADDR     0x0010
> > +#define VPU_TO_HOST            0x001C
> > +#define VPU_DMEM_EXT0_ADDR     0x0014
> > +#define VPU_DMEM_EXT1_ADDR     0x0018
> > +#define HOST_TO_VPU            0x0024
> > +#define VPU_PC_REG             0x0060
> > +#define VPU_WDT_REG            0x0084
> > +
> > +/* vpu inter-processor communication interrupt */
> > +#define VPU_IPC_INT            BIT(8)
> > +
> > +/**
> > + * enum vpu_fw_type - VPU firmware type
> > + *
> > + * @P_FW: program firmware
> > + * @D_FW: data firmware
> > + *
> > + */
> > +enum vpu_fw_type {
> > +       P_FW,
> > +       D_FW,
> > +};
> > +
> > +/**
> > + * struct vpu_mem - VPU extended program/data memory information
> > + *
> > + * @va:                the kernel virtual memory address of VPU extended memory
> > + * @pa:                the physical memory address of VPU extended memory
> This should be device memory address? If yes, s/physical/device/ and
> s/pa/dma_addr/.

It's really a physical memory since vpu is not attached to IOMMU.It
should be s/dma_addr_t/phys_addr_t for the data type.Thanks.

> > + *
> > + */
> > +struct vpu_mem {
> > +       void *va;
> > +       dma_addr_t pa;
> > +};
> > +
> > +/**
> > + * struct vpu_regs - VPU TCM and configuration registers
> > + *
> > + * @tcm:       the register for VPU Tightly-Coupled Memory
> > + * @cfg:       the register for VPU configuration
> > + * @irq:       the irq number for VPU interrupt
> > + */
> > +struct vpu_regs {
> > +       void __iomem *tcm;
> > +       void __iomem *cfg;
> > +       int irq;
> > +};
> > +
> > +/**
> > + * struct vpu_wdt_handler - VPU watchdog reset handler
> > + *
> > + * @reset_func:        reset handler
> > + * @priv:      private data
> > + */
> > +struct vpu_wdt_handler {
> > +       void (*reset_func)(void *);
> > +       void *priv;
> > +};
> > +
> > +/**
> > + * struct vpu_wdt - VPU watchdog workqueue
> > + *
> > + * @handler:   VPU watchdog reset handler
> > + * @ws:                workstruct for VPU watchdog
> > + * @wq:                workqueue for VPU watchdog
> > + */
> > +struct vpu_wdt {
> > +       struct vpu_wdt_handler handler[VPU_RST_MAX];
> > +       struct work_struct ws;
> > +       struct workqueue_struct *wq;
> > +};
> > +
> > +/**
> > + * struct vpu_run - VPU initialization status
> > + *
> > + * @signaled:          the signal of vpu initialization completed
> > + * @fw_ver:            VPU firmware version
> > + * @enc_capability:    encoder capability
> We should document the meaning of the bit used. Since no bit is used
> by the codec driver, document this is not used for now and the value
> is reserved for future use.

This is not used for now. I will document this in next version. Thanks.

> > + * @wq:                        wait queue for VPU initialization status
> > + */
> > +struct vpu_run {
> > +       u32 signaled;
> > +       char fw_ver[VPU_FW_VER_LEN];
> > +       unsigned int    enc_capability;
> > +       wait_queue_head_t wq;
> > +};
> > +
> > +/**
> > + * struct vpu_ipi_desc - VPU IPI descriptor
> > + *
> > + * @handler:   IPI handler
> > + * @name:      the name of IPI handler
> > + * @priv:      the private data of IPI handler
> > + */
> > +struct vpu_ipi_desc {
> > +       ipi_handler_t handler;
> > +       const char *name;
> > +       void *priv;
> > +};
> > +
> > +/**
> > + * struct share_obj - DTCM (Data Tightly-Coupled Memory) buffer shared with
> > + *                   AP and VPU
> > + *
> > + * @id:                IPI id
> > + * @len:       share buffer length
> > + * @share_buf: share buffer data
> > + */
> > +struct share_obj {
> > +       s32 id;
> > +       u32 len;
> > +       unsigned char share_buf[SHARE_BUF_SIZE];
> > +};
> > +
> > +/**
> > + * struct mtk_vpu - vpu driver data
> > + * @extmem:            VPU extended memory information
> > + * @reg:               VPU TCM and configuration registers
> > + * @run:               VPU initialization status
> > + * @ipi_desc:          VPU IPI descriptor
> > + * @recv_buf:          VPU DTCM share buffer for receiving. The
> > + *                     receive buffer is only accessed in interrupt context.
> > + * @send_buf:          VPU DTCM share buffer for sending
> > + * @dev:               VPU struct device
> > + * @clk:               VPU clock on/off
> > + * @enable_4GB:                VPU 4GB mode on/off
> > + * @vpu_mutex:         protect mtk_vpu (except recv_buf) and ensure only
> > + *                     one client to use VPU service at a time. For example,
> > + *                     suppose a client is using VPU to decode VP8.
> > + *                     If the other client wants to encode VP8,
> > + *                     it has to wait until VP8 decode completes.
> > + * @wdt_refcnt         WDT reference count to make sure the watchdog can be
> > + *                     disabled if no other client is using VPU service
> > + * @ipi_ack_signaled:  The ACKs for registered IPI function sending
> > + *                     interrupt to VPU
> s/ipi_ack_signaled/ipi_id_ack/. Move to after |ack_wq| to be
> consistent with the order of variable declaration.

I will change this in next version. Thanks.

> > + * @ack_wq:            The wait queue for each codec and mdp. When sleeping
> > + *                     processes wake up, they will check the condition
> > + *                     "ipi_ack_signaled" to run the corresponding action or
> > + *                     go back to sleep.
> > + *
> > + */
> > +struct mtk_vpu {
> > +       struct vpu_mem extmem[2];
> > +       struct vpu_regs reg;
> > +       struct vpu_run run;
> > +       struct vpu_wdt wdt;
> > +       struct vpu_ipi_desc ipi_desc[IPI_MAX];
> > +       struct share_obj *recv_buf;
> > +       struct share_obj *send_buf;
> > +       struct device *dev;
> > +       struct clk *clk;
> > +       bool enable_4GB;
> > +       struct mutex vpu_mutex; /* for protecting vpu data data structure */
> Remove the comment here. It should be documented in function comment above

It will get message "CHECK" from checkpatch script if removing this
comment.

> .
> > +       atomic_t wdt_refcnt;
> > +       wait_queue_head_t ack_wq;
> > +       bool ipi_id_ack[IPI_MAX];
> > +};
> > +
> > +static inline void vpu_cfg_writel(struct mtk_vpu *vpu, u32 val, u32 offset)
> > +{
> > +       writel(val, vpu->reg.cfg + offset);
> > +}
> > +
> > +static inline u32 vpu_cfg_readl(struct mtk_vpu *vpu, u32 offset)
> > +{
> > +       return readl(vpu->reg.cfg + offset);
> > +}
> > +
> > +static inline bool vpu_running(struct mtk_vpu *vpu)
> > +{
> > +       return vpu_cfg_readl(vpu, VPU_RESET) & BIT(0);
> > +}
> > +
> > +void vpu_clock_disable(struct mtk_vpu *vpu)
> > +{
> > +       /* Disable VPU watchdog */
> > +       if (atomic_dec_and_test(&vpu->wdt_refcnt))
> Checking wdt_refcnt and doing vpu_cfg_writel should be done
> atomically. For example, if vpu_clock_enable is called after
> atomic_dec_and_test(&vpu->wdt_refcnt) and before vpu_cfg_writel(...),
> the state of watchdog could be wrong. Adding a spinlock to protect
> wdt_refcnt and vpu_cfg_writel together. Then wdt_refcnt doesn't need
> to be atomic_t.

I would use mutex to protect this since this only be called in process
context. I will change this in next version. Thanks.

> > +               vpu_cfg_writel(vpu,
> > +                              vpu_cfg_readl(vpu, VPU_WDT_REG) & ~(1L << 31),
> > +                              VPU_WDT_REG);
> > +
> > +       clk_disable(vpu->clk);
> > +}
> > +
> > +int vpu_clock_enable(struct mtk_vpu *vpu)
> > +{
> > +       int ret;
> > +
> > +       ret = clk_enable(vpu->clk);
> > +       if (ret)
> > +               return ret;
> > +       /* Enable VPU watchdog */
> > +       if (!atomic_read(&vpu->wdt_refcnt))
> Do we need to enable the watchdog other than vpu_ipi_send? The only
> place that AP waits for VPU is in vpu_ipi_send. Right? If yes, can we
> just enable and disable the watchdog in vpu_ipi_send?

Watchdog also should be enabled during driver initialization and
firmware download.

> > +               vpu_cfg_writel(vpu,
> > +                              vpu_cfg_readl(vpu, VPU_WDT_REG) | (1L << 31),
> > +                              VPU_WDT_REG);
> > +
> > +       atomic_inc(&vpu->wdt_refcnt);
> Same above. atomic_read, vpu_cfg_writel, and atomic_inc should be done
> atomically.

I will change this in next version. Thanks.

> > +
> > +       return ret;
> > +}
> > +
> > +int vpu_ipi_register(struct platform_device *pdev,
> > +                    enum ipi_id id, ipi_handler_t handler,
> > +                    const char *name, void *priv)
> > +{
> > +       struct mtk_vpu *vpu = platform_get_drvdata(pdev);
> > +       struct vpu_ipi_desc *ipi_desc;
> > +
> > +       if (!vpu) {
> > +               dev_err(&pdev->dev, "vpu device in not ready\n");
> > +               return -EPROBE_DEFER;
> > +       }
> > +
> > +       if (id < IPI_MAX && handler) {
> Check if id >= 0 because type of enum is implementation dependent.

I will change this in next version. Thanks.

> > +               ipi_desc = vpu->ipi_desc;
> > +               ipi_desc[id].name = name;
> > +               ipi_desc[id].handler = handler;
> > +               ipi_desc[id].priv = priv;
> > +               return 0;
> > +       }
> > +
> > +       dev_err(&pdev->dev, "register vpu ipi with invalid arguments\n");
> print id

I will change this in next version. Thanks.

> > +       return -EINVAL;
> > +}
> > +
> > +int vpu_ipi_send(struct platform_device *pdev,
> > +                enum ipi_id id, void *buf,
> > +                unsigned int len)
> > +{
> > +       struct mtk_vpu *vpu = platform_get_drvdata(pdev);
> > +       struct share_obj *send_obj = vpu->send_buf;
> > +       unsigned long timeout;
> > +       int ret = 0;
> > +
> > +       if (id <= IPI_VPU_INIT || id >= IPI_MAX ||
> > +           len > sizeof(send_obj->share_buf) || !buf) {
> > +               dev_err(vpu->dev, "failed to send ipi message\n");
> > +               return -EINVAL;
> > +       }
> > +
> > +       ret = vpu_clock_enable(vpu);
> > +       if (ret) {
> > +               dev_err(vpu->dev, "failed to enable vpu clock\n");
> > +               return ret;
> > +       }
> > +       if (!vpu_running(vpu)) {
> > +               dev_err(vpu->dev, "vpu_ipi_send: VPU is not running\n");
> > +               ret = -EINVAL;
> > +               goto clock_disable;
> > +       }
> > +
> > +       mutex_lock(&vpu->vpu_mutex);
> > +
> > +        /* Wait until VPU receives the last command */
> > +       timeout = jiffies + msecs_to_jiffies(IPI_TIMEOUT_MS);
> > +       do {
> > +               if (time_after(jiffies, timeout)) {
> > +                       dev_err(vpu->dev, "vpu_ipi_send: IPI timeout!\n");
> > +                       ret = -EIO;
> > +                       goto mut_unlock;
> > +               }
> > +       } while (vpu_cfg_readl(vpu, HOST_TO_VPU));
> > +
> > +       memcpy((void *)send_obj->share_buf, buf, len);
> > +       send_obj->len = len;
> > +       send_obj->id = id;
> > +
> > +       vpu->ipi_id_ack[id] = false;
> > +       /* send the command to VPU */
> > +       vpu_cfg_writel(vpu, 0x1, HOST_TO_VPU);
> > +
> > +       mutex_unlock(&vpu->vpu_mutex);
> > +
> > +       /* wait for VPU's ACK */
> > +       timeout = msecs_to_jiffies(IPI_TIMEOUT_MS);
> > +       ret = wait_event_interruptible_timeout(vpu->ack_wq,
> > +                                              vpu->ipi_id_ack[id], timeout);
> > +       vpu->ipi_id_ack[id] = false;
> > +       if (ret == 0) {
> > +               dev_err(vpu->dev, "vpu ipi %d ack time out !", id);
> > +               ret = -EIO;
> > +               goto clock_disable;
> > +       } else if (-ERESTARTSYS == ret) {
> > +               dev_err(vpu->dev, "vpu ipi %d ack wait interrupted by a signal",
> > +                       id);
> > +               ret = -ERESTARTSYS;
> > +               goto clock_disable;
> > +       }
> > +       vpu_clock_disable(vpu);
> > +
> > +       return 0;
> > +
> > +mut_unlock:
> > +       vpu->ipi_id_ack[id] = false;
> I don't see why we need to set it to false here. Remove if not needed.
I will remove this in next version. Thanks.
> > +       mutex_unlock(&vpu->vpu_mutex);
> > +clock_disable:
> > +       vpu_clock_disable(vpu);
> > +
> > +       return ret;
> > +}
> > +
> > +static void vpu_wdt_reset_func(struct work_struct *ws)
> > +{
> > +       struct vpu_wdt *wdt = container_of(ws, struct vpu_wdt, ws);
> > +       struct mtk_vpu *vpu = container_of(wdt, struct mtk_vpu, wdt);
> > +       struct vpu_wdt_handler *handler = wdt->handler;
> > +       int index, ret;
> > +
> > +       dev_info(vpu->dev, "vpu reset\n");
> > +       mutex_lock(&vpu->vpu_mutex);
> > +       ret = vpu_clock_enable(vpu);
> > +       if (ret) {
> This is missing mutex_unlock. Move mutex_lock right before
> vpu_cfg_writel(vpu, 0x0, VPU_RESET);. So we don't need to unlock when
> vpu_clock_enable fails.

I will change this in next version. Thanks.

> > +               dev_err(vpu->dev, "[VPU] wdt enables clock failed %d\n", ret);
> > +               return;
> > +       }
> > +       vpu_cfg_writel(vpu, 0x0, VPU_RESET);
> > +       vpu_clock_disable(vpu);
> > +       mutex_unlock(&vpu->vpu_mutex);
> > +
> > +       for (index = 0; index < VPU_RST_MAX; index++) {
> > +               if (handler[index].reset_func) {
> > +                       handler[index].reset_func(handler[index].priv);
> > +                       dev_dbg(vpu->dev, "wdt handler func %d\n", index);
> > +               }
> > +       }
> > +}
> > +
> > +int vpu_wdt_reg_handler(struct platform_device *pdev,
> > +                       void wdt_reset(void *),
> > +                       void *priv, enum rst_id id)
> > +{
> > +       struct mtk_vpu *vpu = platform_get_drvdata(pdev);
> > +       struct vpu_wdt_handler *handler = vpu->wdt.handler;
> > +
> > +       if (!vpu) {
> > +               dev_err(vpu->dev, "vpu device in not ready\n");
> > +               return -EPROBE_DEFER;
> > +       }
> > +
> > +       if (id < VPU_RST_MAX && wdt_reset != NULL) {
> Check id >= 0

I will change this in next version. Thanks.

> > +               dev_dbg(vpu->dev, "wdt register id %d\n", id);
> > +               mutex_lock(&vpu->vpu_mutex);
> > +               handler[id].reset_func = wdt_reset;
> > +               handler[id].priv = priv;
> > +               mutex_unlock(&vpu->vpu_mutex);
> > +               return 0;
> > +       }
> > +
> > +       dev_err(vpu->dev, "register vpu wdt handler failed\n");
> > +       return -EINVAL;
> > +}
> > +
> > +unsigned int vpu_get_venc_hw_capa(struct platform_device *pdev)
> > +{
> > +       struct mtk_vpu *vpu = platform_get_drvdata(pdev);
> > +
> > +       return vpu->run.enc_capability;
> > +}
> > +
> > +void *vpu_mapping_dm_addr(struct platform_device *pdev,
> > +                         u32 dtcm_dmem_addr)
> > +{
> > +       struct mtk_vpu *vpu = platform_get_drvdata(pdev);
> > +
> > +       if (!dtcm_dmem_addr ||
> > +           (dtcm_dmem_addr > (VPU_DTCM_SIZE + VPU_EXT_D_SIZE))) {
> > +               dev_err(vpu->dev, "invalid virtual data memory address\n");
> > +               return ERR_PTR(-EINVAL);
> > +       }
> > +
> > +       if (dtcm_dmem_addr < VPU_DTCM_SIZE)
> > +               return dtcm_dmem_addr + vpu->reg.tcm + VPU_DTCM_OFFSET;
> > +
> > +       return vpu->extmem[D_FW].va + (dtcm_dmem_addr - VPU_DTCM_SIZE);
> > +}
> > +
> > +struct platform_device *vpu_get_plat_device(struct platform_device *pdev)
> > +{
> > +       struct device *dev = &pdev->dev;
> > +       struct device_node *vpu_node;
> > +       struct platform_device *vpu_pdev;
> > +
> > +       vpu_node = of_parse_phandle(dev->of_node, "mediatek,vpu", 0);
> > +       if (!vpu_node) {
> > +               dev_err(dev, "can't get vpu node\n");
> > +               return NULL;
> > +       }
> > +
> > +       vpu_pdev = of_find_device_by_node(vpu_node);
> > +       if (WARN_ON(!vpu_pdev)) {
> > +               dev_err(dev, "vpu pdev failed\n");
> > +               of_node_put(vpu_node);
> > +               return NULL;
> > +       }
> > +
> > +       return vpu_pdev;
> > +}
> > +
> > +/* load vpu program/data memory */
> > +static int load_requested_vpu(struct mtk_vpu *vpu,
> > +                             const struct firmware *vpu_fw,
> > +                             u8 fw_type)
> > +{
> > +       size_t tcm_size = fw_type ? VPU_DTCM_SIZE : VPU_PTCM_SIZE;
> > +       size_t fw_size = fw_type ? VPU_D_FW_SIZE : VPU_P_FW_SIZE;
> > +       char *fw_name = fw_type ? VPU_D_FW : VPU_P_FW;
> > +       size_t dl_size = 0;
> > +       size_t extra_fw_size = 0;
> > +       void *dest;
> > +       int ret;
> > +
> > +       ret = request_firmware(&vpu_fw, fw_name, vpu->dev);
> > +       if (ret < 0) {
> > +               dev_err(vpu->dev, "Failed to load %s, %d\n", fw_name, ret);
> > +               return ret;
> > +       }
> > +       dl_size = vpu_fw->size;
> > +       if (dl_size > fw_size) {
> > +               dev_err(vpu->dev, "fw %s size %zu is abnormal\n", fw_name,
> > +                       dl_size);
> > +               release_firmware(vpu_fw);
> > +               return  -EFBIG;
> > +       }
> > +       dev_dbg(vpu->dev, "Downloaded fw %s size: %zu.\n",
> > +               fw_name,
> > +               dl_size);
> > +       /* reset VPU */
> > +       vpu_cfg_writel(vpu, 0x0, VPU_RESET);
> > +
> > +       /* handle extended firmware size */
> > +       if (dl_size > tcm_size) {
> > +               dev_dbg(vpu->dev, "fw size %lx > limited fw size %lx\n",
> > +                       dl_size, tcm_size);
> > +               extra_fw_size = dl_size - tcm_size;
> > +               dev_dbg(vpu->dev, "extra_fw_size %lx\n", extra_fw_size);
> > +               dl_size = tcm_size;
> > +       }
> > +       dest = vpu->reg.tcm;
> > +       if (fw_type == D_FW)
> > +               dest += VPU_DTCM_OFFSET;
> > +       memcpy(dest, vpu_fw->data, dl_size);
> > +       /* download to extended memory if need */
> > +       if (extra_fw_size > 0) {
> > +               dest = vpu->extmem[fw_type].va;
> > +               dev_dbg(vpu->dev, "download extended memory type %x\n",
> > +                       fw_type);
> > +               memcpy(dest, vpu_fw->data + tcm_size, extra_fw_size);
> > +       }
> > +
> > +       release_firmware(vpu_fw);
> > +
> > +       return 0;
> > +}
> > +
> > +int vpu_load_firmware(struct platform_device *pdev)
> > +{
> > +       struct mtk_vpu *vpu = platform_get_drvdata(pdev);
> > +       struct device *dev = &pdev->dev;
> > +       struct vpu_run *run = &vpu->run;
> > +       const struct firmware *vpu_fw;
> > +       int ret;
> > +
> > +       if (!pdev) {
> > +               dev_err(dev, "VPU platform device is invalid\n");
> > +               return -EINVAL;
> > +       }
> > +
> > +       mutex_lock(&vpu->vpu_mutex);
> > +
> > +       ret = vpu_clock_enable(vpu);
> > +       if (ret) {
> > +               dev_err(dev, "enable clock failed %d\n", ret);
> > +               goto OUT_LOAD_FW;
> > +       }
> > +
> > +       if (vpu_running(vpu)) {
> > +               vpu_clock_disable(vpu);
> > +               mutex_unlock(&vpu->vpu_mutex);
> > +               dev_warn(dev, "vpu is running already\n");
> > +               return 0;
> > +       }
> > +
> > +       run->signaled = false;
> > +       dev_dbg(vpu->dev, "firmware request\n");
> > +       /* Downloading program firmware to device*/
> > +       ret = load_requested_vpu(vpu, vpu_fw, P_FW);
> > +       if (ret < 0) {
> > +               dev_err(dev, "Failed to request %s, %d\n", VPU_P_FW, ret);
> > +               goto OUT_LOAD_FW;
> > +       }
> > +
> > +       /* Downloading data firmware to device */
> > +       ret = load_requested_vpu(vpu, vpu_fw, D_FW);
> > +       if (ret < 0) {
> > +               dev_err(dev, "Failed to request %s, %d\n", VPU_D_FW, ret);
> > +               goto OUT_LOAD_FW;
> > +       }
> > +
> > +       /* boot up vpu */
> > +       vpu_cfg_writel(vpu, 0x1, VPU_RESET);
> > +
> > +       ret = wait_event_interruptible_timeout(run->wq,
> > +                                              run->signaled,
> > +                                              msecs_to_jiffies(INIT_TIMEOUT_MS)
> > +                                              );
> > +       if (ret == 0) {
> > +               ret = -ETIME;
> > +               dev_err(dev, "wait vpu initialization timout!\n");
> > +               goto OUT_LOAD_FW;
> > +       } else if (-ERESTARTSYS == ret) {
> > +               dev_err(dev, "wait vpu interrupted by a signal!\n");
> > +               goto OUT_LOAD_FW;
> > +       }
> > +
> > +       ret = 0;
> > +       dev_info(dev, "vpu is ready. Fw version %s\n", run->fw_ver);
> > +
> > +OUT_LOAD_FW:
> > +       vpu_clock_disable(vpu);
> > +       mutex_unlock(&vpu->vpu_mutex);
> > +
> > +       return ret;
> > +}
> > +
> > +int vpu_compare_version(struct platform_device *pdev,
> > +                       const char *expected_version)
> > +{
> > +       struct mtk_vpu *vpu = platform_get_drvdata(pdev);
> > +       int cur_major, cur_minor, cur_build, cur_rel, cur_ver_num;
> > +       int major, minor, build, rel, ver_num;
> > +       char *cur_version = vpu->run.fw_ver;
> > +
> > +       cur_ver_num = sscanf(cur_version, "%d.%d.%d-rc%d",
> > +                            &cur_major, &cur_minor, &cur_build, &cur_rel);
> > +       if (cur_ver_num < 3)
> > +               return -1;
> > +       ver_num = sscanf(expected_version, "%d.%d.%d-rc%d",
> > +                        &major, &minor, &build, &rel);
> > +       if (ver_num < 3)
> > +               return -1;
> > +
> > +       if (cur_major < major)
> > +               return -1;
> > +       if (cur_major > major)
> > +               return 1;
> > +
> > +       if (cur_minor < minor)
> > +               return -1;
> > +       if (cur_minor > minor)
> > +               return 1;
> > +
> > +       if (cur_build < build)
> > +               return -1;
> > +       if (cur_build > build)
> > +               return 1;
> > +
> > +       if (cur_ver_num < ver_num)
> > +               return -1;
> > +       if (cur_ver_num > ver_num)
> > +               return 1;
> > +
> > +       if (ver_num > 3) {
> > +               if (cur_rel < rel)
> > +                       return -1;
> > +               if (cur_rel > rel)
> > +                       return 1;
> > +       }
> > +
> > +       return 0;
> > +}
> > +
> > +static void vpu_init_ipi_handler(void *data, unsigned int len, void *priv)
> > +{
> > +       struct mtk_vpu *vpu = (struct mtk_vpu *)priv;
> > +       struct vpu_run *run = (struct vpu_run *)data;
> > +
> > +       vpu->run.signaled = run->signaled;
> > +       strncpy(vpu->run.fw_ver, run->fw_ver, VPU_FW_VER_LEN);
> > +       vpu->run.enc_capability = run->enc_capability;
> > +       wake_up_interruptible(&vpu->run.wq);
> > +}
> > +
> > +#ifdef CONFIG_DEBUG_FS
> > +static int vpu_debug_open(struct inode *inode, struct file *file)
> > +{
> > +       file->private_data = inode->i_private;
> > +       return 0;
> > +}
> > +
> > +static ssize_t vpu_debug_read(struct file *file, char __user *user_buf,
> > +                             size_t count, loff_t *ppos)
> > +{
> > +       char buf[256];
> > +       unsigned int len;
> > +       unsigned int running, pc, vpu_to_host, host_to_vpu, wdt;
> > +       int ret;
> > +       struct device *dev = file->private_data;
> > +       struct mtk_vpu *vpu = dev_get_drvdata(dev);
> > +
> > +       ret = vpu_clock_enable(vpu);
> > +       if (ret) {
> > +               dev_err(vpu->dev, "[VPU] enable clock failed %d\n", ret);
> > +               return 0;
> > +       }
> > +
> > +       /* vpu register status */
> > +       running = vpu_running(vpu);
> > +       pc = vpu_cfg_readl(vpu, VPU_PC_REG);
> > +       wdt = vpu_cfg_readl(vpu, VPU_WDT_REG);
> > +       host_to_vpu = vpu_cfg_readl(vpu, HOST_TO_VPU);
> > +       vpu_to_host = vpu_cfg_readl(vpu, VPU_TO_HOST);
> > +       vpu_clock_disable(vpu);
> > +
> > +       if (running) {
> > +               len = sprintf(buf, "VPU is running\n\n"
> > +               "FW Version: %s\n"
> > +               "PC: 0x%x\n"
> > +               "WDT: 0x%x\n"
> > +               "Host to VPU: 0x%x\n"
> > +               "VPU to Host: 0x%x\n",
> > +               vpu->run.fw_ver, pc, wdt,
> > +               host_to_vpu, vpu_to_host);
> > +       } else {
> > +               len = sprintf(buf, "VPU not running\n");
> > +       }
> > +
> > +       return simple_read_from_buffer(user_buf, count, ppos, buf, len);
> > +}
> > +
> > +static const struct file_operations vpu_debug_fops = {
> > +       .open = vpu_debug_open,
> > +       .read = vpu_debug_read,
> > +};
> > +#endif /* CONFIG_DEBUG_FS */
> > +
> > +static void vpu_free_ext_mem(struct mtk_vpu *vpu, u8 fw_type)
> > +{
> > +       struct device *dev = vpu->dev;
> > +       size_t fw_ext_size = fw_type ? VPU_EXT_D_SIZE : VPU_EXT_P_SIZE;
> > +
> > +       dma_free_coherent(dev, fw_ext_size, vpu->extmem[fw_type].va,
> > +                         vpu->extmem[fw_type].pa);
> > +}
> > +
> > +static int vpu_alloc_ext_mem(struct mtk_vpu *vpu, u32 fw_type)
> > +{
> > +       struct device *dev = vpu->dev;
> > +       size_t fw_ext_size = fw_type ? VPU_EXT_D_SIZE : VPU_EXT_P_SIZE;
> > +       u32 vpu_ext_mem0 = fw_type ? VPU_DMEM_EXT0_ADDR : VPU_PMEM_EXT0_ADDR;
> > +       u32 vpu_ext_mem1 = fw_type ? VPU_DMEM_EXT1_ADDR : VPU_PMEM_EXT1_ADDR;
> > +       u32 offset_4gb = vpu->enable_4GB ? 0x40000000 : 0;
> > +
> > +       vpu->extmem[fw_type].va = dma_alloc_coherent(dev,
> > +                                              fw_ext_size,
> > +                                              &vpu->extmem[fw_type].pa,
> > +                                              GFP_KERNEL);
> > +       if (!vpu->extmem[fw_type].va) {
> > +               dev_err(dev, "Failed to allocate the extended program memory\n");
> > +               return PTR_ERR(vpu->extmem[fw_type].va);
> > +       }
> > +
> > +       /* Disable extend0. Enable extend1 */
> > +       vpu_cfg_writel(vpu, 0x1, vpu_ext_mem0);
> > +       vpu_cfg_writel(vpu, (vpu->extmem[fw_type].pa & 0xFFFFF000) + offset_4gb,
> > +                      vpu_ext_mem1);
> > +
> > +       dev_info(dev, "%s extend memory phy=0x%llx virt=0x%p\n",
> > +                fw_type ? "Data" : "Program",
> > +                (unsigned long long)vpu->extmem[fw_type].pa,
> > +                vpu->extmem[fw_type].va);
> > +
> > +       return 0;
> > +}
> > +
> > +static void vpu_ipi_handler(struct mtk_vpu *vpu)
> > +{
> > +       struct share_obj *rcv_obj = vpu->recv_buf;
> > +       struct vpu_ipi_desc *ipi_desc = vpu->ipi_desc;
> > +
> > +       if (rcv_obj->id < IPI_MAX && ipi_desc[rcv_obj->id].handler) {
> check rcv_obj->id >= 0

The variable type of "rcv_obj->id" is u32. I think we don't need to
check this range.Thanks.

> > +               ipi_desc[rcv_obj->id].handler(rcv_obj->share_buf,
> > +                                             rcv_obj->len,
> > +                                             ipi_desc[rcv_obj->id].priv);
> > +               if (rcv_obj->id > IPI_VPU_INIT) {
> > +                       vpu->ipi_id_ack[rcv_obj->id] = true;
> > +                       wake_up_interruptible(&vpu->ack_wq);
> > +               }
> > +       } else {
> > +               dev_err(vpu->dev, "No such ipi id = %d\n", rcv_obj->id);
> > +       }
> > +}
> > +
> > +static int vpu_ipi_init(struct mtk_vpu *vpu)
> > +{
> > +       /* Disable VPU to host interrupt */
> > +       vpu_cfg_writel(vpu, 0x0, VPU_TO_HOST);
> > +
> > +       /* shared buffer initialization */
> > +       vpu->recv_buf = (struct share_obj *)(vpu->reg.tcm + VPU_DTCM_OFFSET);
> > +       vpu->send_buf = vpu->recv_buf + 1;
> > +       memset(vpu->recv_buf, 0, sizeof(struct share_obj));
> > +       memset(vpu->send_buf, 0, sizeof(struct share_obj));
> > +       mutex_init(&vpu->vpu_mutex);
> > +
> > +       return 0;
> > +}
> > +
> > +static irqreturn_t vpu_irq_handler(int irq, void *priv)
> > +{
> > +       struct mtk_vpu *vpu = priv;
> > +       u32 vpu_to_host;
> > +       int ret;
> > +
> > +       /*
> > +        * Clock should have been enabled already.
> > +        * Enable again in case vpu_ipi_send times out
> > +        * and has disabled the clock.
> > +        */
> > +       ret = clk_enable(vpu->clk);
> > +       if (ret) {
> > +               dev_err(vpu->dev, "[VPU] enable clock failed %d\n", ret);
> > +               return IRQ_NONE;
> > +       }
> > +       vpu_to_host = vpu_cfg_readl(vpu, VPU_TO_HOST);
> > +       if (vpu_to_host & VPU_IPC_INT) {
> > +               vpu_ipi_handler(vpu);
> > +       } else {
> > +               dev_err(vpu->dev, "vpu watchdog timeout! 0x%x", vpu_to_host);
> > +               if (vpu->wdt.wq)
> Remove. mtk_vpu_probe makes sure wdt.wq is not NULL.
I will change this in next version. Thanks.

> > +                       queue_work(vpu->wdt.wq, &vpu->wdt.ws);
> > +       }
> > +
> > +       /* VPU won't send another interrupt until we set VPU_TO_HOST to 0. */
> > +       vpu_cfg_writel(vpu, 0x0, VPU_TO_HOST);
> > +       clk_disable(vpu->clk);
> > +
> > +       return IRQ_HANDLED;
> > +}
> > +
> > +#ifdef CONFIG_DEBUG_FS
> > +static struct dentry *vpu_debugfs;
> > +#endif
> > +static int mtk_vpu_probe(struct platform_device *pdev)
> > +{
> > +       struct mtk_vpu *vpu;
> > +       struct device *dev;
> > +       struct resource *res;
> > +       int ret = 0;
> > +
> > +       dev_dbg(&pdev->dev, "initialization\n");
> > +
> > +       dev = &pdev->dev;
> > +       vpu = devm_kzalloc(dev, sizeof(*vpu), GFP_KERNEL);
> > +       if (!vpu)
> > +               return -ENOMEM;
> > +
> > +       vpu->dev = &pdev->dev;
> > +       res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "tcm");
> > +       vpu->reg.tcm = devm_ioremap_resource(dev, res);
> > +       if (IS_ERR(vpu->reg.tcm)) {
> > +               dev_err(dev, "devm_ioremap_resource vpu tcm failed.\n");
> > +               return PTR_ERR(vpu->reg.tcm);
> > +       }
> > +
> > +       res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "cfg_reg");
> > +       vpu->reg.cfg = devm_ioremap_resource(dev, res);
> > +       if (IS_ERR(vpu->reg.cfg)) {
> > +               dev_err(dev, "devm_ioremap_resource vpu cfg failed.\n");
> > +               return PTR_ERR(vpu->reg.cfg);
> > +       }
> > +
> > +       /* Get VPU clock */
> > +       vpu->clk = devm_clk_get(dev, "main");
> > +       if (!vpu->clk) {
> > +               dev_err(dev, "get vpu clock failed\n");
> > +               return -EINVAL;
> > +       }
> > +
> > +       platform_set_drvdata(pdev, vpu);
> > +
> > +       ret = clk_prepare(vpu->clk);
> > +       if (ret) {
> > +               dev_err(dev, "prepare vpu clock failed\n");
> > +               return ret;
> > +       }
> > +
> > +       /* VPU watchdog */
> > +       vpu->wdt.wq = create_singlethread_workqueue("vpu_wdt");
> > +       if (!vpu->wdt.wq) {
> > +               dev_err(dev, "initialize wdt workqueue failed\n");
> > +               return -ENOMEM;
> > +       }
> > +       INIT_WORK(&vpu->wdt.ws, vpu_wdt_reset_func);
> > +
> > +       ret = vpu_clock_enable(vpu);
> > +       if (ret) {
> > +               dev_err(dev, "enable vpu clock failed\n");
> > +               goto workqueue_destroy;
> > +       }
> > +
> > +       dev_dbg(dev, "vpu ipi init\n");
> > +       ret = vpu_ipi_init(vpu);
> > +       if (ret) {
> > +               dev_err(dev, "Failed to init ipi\n");
> > +               goto disable_vpu_clk;
> > +       }
> > +
> > +       /* register vpu initialization IPI */
> > +       ret = vpu_ipi_register(pdev, IPI_VPU_INIT, vpu_init_ipi_handler,
> > +                              "vpu_init", vpu);
> > +       if (ret) {
> > +               dev_err(dev, "Failed to register IPI_VPU_INIT\n");
> > +               goto vpu_mutex_destroy;
> > +       }
> > +
> > +#ifdef CONFIG_DEBUG_FS
> > +       vpu_debugfs = debugfs_create_file("mtk_vpu", S_IRUGO, NULL, (void *)dev,
> > +                                         &vpu_debug_fops);
> > +       if (!vpu_debugfs) {
> > +               ret = -ENOMEM;
> > +               goto cleanup_ipi;
> > +       }
> > +#endif
> > +
> > +       /* Set PTCM to 96K and DTCM to 32K */
> > +       vpu_cfg_writel(vpu, 0x2, VPU_TCM_CFG);
> > +
> > +       vpu->enable_4GB = !!(max_pfn > (0xffffffffUL >> PAGE_SHIFT));
> > +       dev_dbg(dev, "4GB mode %u\n", vpu->enable_4GB);
> > +
> > +       if (vpu->enable_4GB) {
> > +               ret = of_reserved_mem_device_init(dev);
> > +               if (ret)
> > +                       dev_info(dev, "init reserved memory failed\n");
> > +                       /* continue to use dynamic allocation if failed */
> > +       }
> > +
> > +       ret = vpu_alloc_ext_mem(vpu, D_FW);
> > +       if (ret) {
> > +               dev_err(dev, "Allocate DM failed\n");
> > +               goto remove_debugfs;
> > +       }
> > +
> > +       ret = vpu_alloc_ext_mem(vpu, P_FW);
> > +       if (ret) {
> > +               dev_err(dev, "Allocate PM failed\n");
> > +               goto free_d_mem;
> > +       }
> > +
> > +       init_waitqueue_head(&vpu->run.wq);
> > +       init_waitqueue_head(&vpu->ack_wq);
> > +
> > +       res = platform_get_resource(pdev, IORESOURCE_IRQ, 0);
> > +       if (!res) {
> > +               dev_err(dev, "get IRQ resource failed.\n");
> > +               ret = -ENXIO;
> > +               goto free_p_mem;
> > +       }
> > +       vpu->reg.irq = platform_get_irq(pdev, 0);
> > +       ret = devm_request_irq(dev, vpu->reg.irq, vpu_irq_handler, 0,
> > +                              pdev->name, vpu);
> > +       if (ret) {
> > +               dev_err(dev, "failed to request irq\n");
> > +               goto free_p_mem;
> > +       }
> > +
> > +       vpu_clock_disable(vpu);
> > +       dev_dbg(dev, "initialization completed\n");
> > +
> > +       return 0;
> > +
> > +free_p_mem:
> > +       vpu_free_ext_mem(vpu, P_FW);
> > +free_d_mem:
> > +       vpu_free_ext_mem(vpu, D_FW);
> > +remove_debugfs:
> > +       of_reserved_mem_device_release(dev);
> > +#ifdef CONFIG_DEBUG_FS
> > +       debugfs_remove(vpu_debugfs);
> > +cleanup_ipi:
> > +#endif
> > +       memset(vpu->ipi_desc, 0, sizeof(struct vpu_ipi_desc) * IPI_MAX);
> > +vpu_mutex_destroy:
> > +       mutex_destroy(&vpu->vpu_mutex);
> > +disable_vpu_clk:
> > +       vpu_clock_disable(vpu);
> > +workqueue_destroy:
> > +       destroy_workqueue(vpu->wdt.wq);
> > +
> > +       return ret;
> > +}
> > +
> > +static const struct of_device_id mtk_vpu_match[] = {
> > +       {
> > +               .compatible = "mediatek,mt8173-vpu",
> > +       },
> > +       {},
> > +};
> > +MODULE_DEVICE_TABLE(of, mtk_vpu_match);
> > +
> > +static int mtk_vpu_remove(struct platform_device *pdev)
> > +{
> > +       struct mtk_vpu *vpu = platform_get_drvdata(pdev);
> > +
> > +#ifdef CONFIG_DEBUG_FS
> > +       debugfs_remove(vpu_debugfs);
> > +#endif
> > +       if (vpu->wdt.wq) {
> > +               flush_workqueue(vpu->wdt.wq);
> > +               destroy_workqueue(vpu->wdt.wq);
> > +       }
> > +       vpu_free_ext_mem(vpu, P_FW);
> > +       vpu_free_ext_mem(vpu, D_FW);
> > +       mutex_destroy(&vpu->vpu_mutex);
> > +       clk_unprepare(vpu->clk);
> > +
> > +       return 0;
> > +}
> > +
> > +static struct platform_driver mtk_vpu_driver = {
> > +       .probe  = mtk_vpu_probe,
> > +       .remove = mtk_vpu_remove,
> > +       .driver = {
> > +               .name   = "mtk_vpu",
> > +               .owner  = THIS_MODULE,
> > +               .of_match_table = mtk_vpu_match,
> > +       },
> > +};
> > +
> > +module_platform_driver(mtk_vpu_driver);
> > +
> > +MODULE_LICENSE("GPL v2");
> > +MODULE_DESCRIPTION("Mediatek Video Prosessor Unit driver");
> > diff --git a/drivers/media/platform/mtk-vpu/mtk_vpu.h b/drivers/media/platform/mtk-vpu/mtk_vpu.h
> > new file mode 100644
> > index 0000000..d9c3cde
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vpu/mtk_vpu.h
> > @@ -0,0 +1,167 @@
> > +/*
> > +* Copyright (c) 2015 MediaTek Inc.
> > +* Author: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
> > +*
> > +* This program is free software; you can redistribute it and/or modify
> > +* it under the terms of the GNU General Public License version 2 as
> > +* published by the Free Software Foundation.
> > +*
> > +* This program is distributed in the hope that it will be useful,
> > +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> > +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > +* GNU General Public License for more details.
> > +*/
> > +
> > +#ifndef _MTK_VPU_H
> > +#define _MTK_VPU_H
> > +
> > +#include <linux/platform_device.h>
> > +
> > +/**
> > + * VPU (video processor unit) is a tiny processor controlling video hardware
> > + * related to video codec, scaling and color format converting.
> > + * VPU interfaces with other blocks by share memory and interrupt.
> > + **/
> > +
> > +typedef void (*ipi_handler_t) (void *data,
> > +                              unsigned int len,
> > +                              void *priv);
> > +
> > +/**
> > + * enum ipi_id - the id of inter-processor interrupt
> > + *
> > + * @IPI_VPU_INIT:       The interrupt from vpu is to notfiy kernel
> > +                        VPU initialization completed.
> > +                        IPI_VPU_INIT is sent from VPU when firmware is
> > +                        loaded. AP doesn't need to send IPI_VPU_INIT
> > +                        command to VPU.
> > +                        For other IPI below, AP should send the request
> > +                        to VPU to trigger the interrupt.
> > + * @IPI_VENC_H264:      The interrupt from vpu is to notify kernel to
> > +                        handle H264 video encoder job, and vice versa.
> > + * @IPI_VENC_VP8:       The interrupt fro vpu is to notify kernel to
> > +                        handle VP8 video encoder job,, and vice versa.
> > + * @IPI_MAX:            The maximum IPI number
> > + */
> > +
> > +enum ipi_id {
> > +       IPI_VPU_INIT = 0,
> > +       IPI_VENC_H264,
> > +       IPI_VENC_VP8,
> > +       IPI_MAX,
> > +};
> > +
> > +/**
> > + * enum rst_id - reset id to register reset function for VPU watchdog timeout
> > + *
> > + * @VPU_RST_DEC: decoder reset id
> > + * @VPU_RST_ENC: encoder reset id
> > + * @VPU_RST_MDP: MDP (Media Data Path) reset id
> > + * @VPU_RST_MAX: maximum reset id
> > + */
> > +enum rst_id {
> > +       VPU_RST_ENC,
> > +       VPU_RST_DEC,
> > +       VPU_RST_MDP,
> > +       VPU_RST_MAX,
> > +};
> > +
> > +/**
> > + * vpu_ipi_register - register an ipi function
> > + *
> > + * @pdev:      VPU platform device
> > + * @id:                IPI ID
> > + * @handler:   IPI handler
> > + * @name:      IPI name
> > + * @priv:      private data for IPI handler
> > + *
> > + * Register an ipi function to receive ipi interrupt from VPU.
> > + *
> > + * Return: Return 0 if ipi registers successfully, otherwise it is failed.
> > + */
> > +int vpu_ipi_register(struct platform_device *pdev, enum ipi_id id,
> > +                    ipi_handler_t handler, const char *name, void *priv);
> > +
> > +/**
> > + * vpu_ipi_send - send data from AP to vpu.
> > + *
> > + * @pdev:      VPU platform device
> > + * @id:                IPI ID
> > + * @buf:       the data buffer
> > + * @len:       the data buffer length
> > + *
> > + * This function is thread-safe. When this function returns,
> > + * VPU has received the data and starts the processing.
> > + * When the processing completes, IPI handler registered
> > + * by vpu_ipi_register will be called in interrupt context.
> > + *
> > + * Return: Return 0 if sending data successfully, otherwise it is failed.
> > + **/
> > +int vpu_ipi_send(struct platform_device *pdev,
> > +                enum ipi_id id, void *buf,
> > +                unsigned int len);
> > +
> > +/**
> > + * vpu_get_plat_device - get VPU's platform device
> > + *
> > + * @pdev:      the platform device of the module requesting VPU platform
> > + *             device for using VPU API.
> > + *
> > + * Return: Return NULL if it is failed.
> > + * otherwise it is VPU's platform device
> > + **/
> > +struct platform_device *vpu_get_plat_device(struct platform_device *pdev);
> > +
> > +/**
> > + * vpu_wdt_reg_handler - register a VPU watchdog handler
> > + *
> > + * @pdev:               VPU platform device
> > + * @vpu_wdt_reset_func:        the callback reset function
> > + * @private_data:       the private data for reset function
> > + * @rst_id:            reset id
> > + *
> > + * Register a handler performing own tasks when vpu reset by watchdog
> > + *
> > + * Return: Return 0 if the handler is added successfully,
> > + * otherwise it is failed.
> > + *
> > + **/
> > +int vpu_wdt_reg_handler(struct platform_device *pdev,
> > +                       void vpu_wdt_reset_func(void *),
> > +                       void *priv, enum rst_id id);
> > +
> > +/**
> > + * vpu_get_venc_hw_capa - get video encoder hardware capability
> > + *
> > + * @pdev:      VPU platform device
> > + *
> > + * Return: video encoder hardware capability
> > + **/
> > +unsigned int vpu_get_venc_hw_capa(struct platform_device *pdev);
> > +
> > +/**
> > + * vpu_load_firmware - download VPU firmware and boot it
> > + *
> > + * @pdev:      VPU platform device
> > + *
> > + * Return: Return 0 if downloading firmware successfully,
> > + * otherwise it is failed
> > + **/
> > +int vpu_load_firmware(struct platform_device *pdev);
> > +
> > +/**
> > + * vpu_mapping_dm_addr - Mapping DTCM/DMEM to kernel virtual address
> > + *
> > + * @pdev:      VPU platform device
> > + * @dmem_addr: VPU's data memory address
> > + *
> > + * Mapping the VPU's DTCM (Data Tightly-Coupled Memory) /
> > + * DMEM (Data Extended Memory) memory address to
> > + * kernel virtual address.
> > + *
> > + * Return: Return ERR_PTR(-EINVAL) if mapping failed,
> > + * otherwise the mapped kernel virtual address
> > + **/
> > +void *vpu_mapping_dm_addr(struct platform_device *pdev,
> > +                         u32 dtcm_dmem_addr);
> > +#endif /* _MTK_VPU_H */
> > --
> > 1.7.9.5
> >
> > --
> > To unsubscribe from this list: send the line "unsubscribe linux-media" in
> > the body of a message to majordomo at vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 102+ messages in thread

* Re: [PATCH v4 7/8] [media] vcodec: mediatek: Add Mediatek H264 Video Encoder Driver
  2016-02-15 11:33                 ` Hans Verkuil
  (?)
@ 2016-02-16 11:57                   ` pochun lin
  -1 siblings, 0 replies; 102+ messages in thread
From: pochun lin @ 2016-02-16 11:57 UTC (permalink / raw)
  To: Hans Verkuil
  Cc: Tiffany Lin (林慧珊),
	Hans Verkuil, daniel.thompson, Rob Herring,
	Mauro Carvalho Chehab, Matthias Brugger, Daniel Kurtz,
	Pawel Osciak, Eddie Huang (黃智傑),
	Yingjoe Chen (陳英洲),
	devicetree, linux-kernel, linux-arm-kernel, linux-media,
	linux-mediatek, Daniel Hsiao (蕭伯剛)

Hi Hans,

On Mon, 2016-02-15 at 19:33 +0800, Hans Verkuil wrote:
> On 02/04/2016 12:35 PM, Tiffany Lin wrote:
> > Add h264 encoder driver for MT8173
> > 
> > Signed-off-by: Daniel Hsiao <daniel.hsiao@mediatek.com>
> > Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
> > ---
> >  drivers/media/platform/mtk-vcodec/Makefile         |    3 +-
> >  .../media/platform/mtk-vcodec/h264_enc/Makefile    |    6 +
> >  .../platform/mtk-vcodec/h264_enc/venc_h264_if.c    |  540 ++++++++++++++++++++
> >  .../platform/mtk-vcodec/h264_enc/venc_h264_if.h    |  165 ++++++
> >  .../platform/mtk-vcodec/h264_enc/venc_h264_vpu.c   |  309 +++++++++++
> >  .../platform/mtk-vcodec/h264_enc/venc_h264_vpu.h   |   30 ++
> >  drivers/media/platform/mtk-vcodec/venc_drv_if.c    |    3 +
> >  7 files changed, 1055 insertions(+), 1 deletion(-)
> >  create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/Makefile
> >  create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.c
> >  create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.h
> >  create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.c
> >  create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.h
> > 
> > diff --git a/drivers/media/platform/mtk-vcodec/Makefile b/drivers/media/platform/mtk-vcodec/Makefile
> > index f4ef502..f47dfc7 100644
> > --- a/drivers/media/platform/mtk-vcodec/Makefile
> > +++ b/drivers/media/platform/mtk-vcodec/Makefile
> > @@ -5,6 +5,7 @@ obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += mtk_vcodec_intr.o \
> >  				       mtk_vcodec_enc_pm.o \
> >  				       venc_drv_if.o
> >  
> > -obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += vp8_enc/
> > +
> > +obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += vp8_enc/ h264_enc/
> >  
> >  ccflags-y += -I$(srctree)/drivers/media/platform/mtk-vpu
> > diff --git a/drivers/media/platform/mtk-vcodec/h264_enc/Makefile b/drivers/media/platform/mtk-vcodec/h264_enc/Makefile
> > new file mode 100644
> > index 0000000..765b45f
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/h264_enc/Makefile
> > @@ -0,0 +1,6 @@
> > +obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += venc_h264_if.o venc_h264_vpu.o
> > +
> > +ccflags-y += \
> > +    -I$(srctree)/drivers/media/platform/mtk-vcodec/ \
> > +    -I$(srctree)/drivers/media/platform/mtk-vcodec/h264_enc \
> > +    -I$(srctree)/drivers/media/platform/mtk-vpu
> > diff --git a/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.c b/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.c
> > new file mode 100644
> > index 0000000..c35fb26
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.c
> > @@ -0,0 +1,540 @@
> > +/*
> > + * Copyright (c) 2015 MediaTek Inc.
> > + * Author: Jungchang Tsao <jungchang.tsao@mediatek.com>
> > + *         Daniel Hsiao <daniel.hsiao@mediatek.com>
> > + *         PoChun Lin <pochun.lin@mediatek.com>
> > + *
> > + * This program is free software; you can redistribute it and/or
> > + * modify
> > + * it under the terms of the GNU General Public License version 2 as
> > + * published by the Free Software Foundation.
> > + *
> > + * This program is distributed in the hope that it will be useful,
> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > + * GNU General Public License for more details.
> > + */
> > +
> > +#include <linux/interrupt.h>
> > +#include <linux/kernel.h>
> > +#include <linux/slab.h>
> > +
> > +#include "mtk_vcodec_drv.h"
> > +#include "mtk_vcodec_util.h"
> > +#include "mtk_vcodec_intr.h"
> > +#include "mtk_vcodec_enc.h"
> > +#include "mtk_vcodec_pm.h"
> > +#include "mtk_vpu.h"
> > +
> > +#include "venc_h264_if.h"
> > +#include "venc_h264_vpu.h"
> > +
> > +#define VENC_PIC_BITSTREAM_BYTE_CNT 0x0098
> > +
> > +enum venc_h264_irq_status {
> > +	H264_IRQ_STATUS_ENC_SPS_INT = (1 << 0),
> > +	H264_IRQ_STATUS_ENC_PPS_INT = (1 << 1),
> > +	H264_IRQ_STATUS_ENC_FRM_INT = (1 << 2),
> > +};
> > +
> > +static inline void h264_write_reg(struct venc_h264_inst *inst, u32 addr,
> > +				  u32 val)
> > +{
> > +	writel(val, inst->hw_base + addr);
> > +}
> > +
> > +static inline u32 h264_read_reg(struct venc_h264_inst *inst, u32 addr)
> > +{
> > +	return readl(inst->hw_base + addr);
> > +}
> > +
> > +static void h264_enc_free_work_buf(struct venc_h264_inst *inst)
> > +{
> > +	int i;
> > +
> > +	mtk_vcodec_debug_enter(inst);
> > +
> > +	/* Except the SKIP_FRAME buffers,
> > +	 * other buffers need to be freed by AP.
> > +	 */
> > +	for (i = 0; i < VENC_H264_VPU_WORK_BUF_MAX; i++) {
> > +		if (i != VENC_H264_VPU_WORK_BUF_SKIP_FRAME)
> > +			if (inst->work_bufs[i].va != NULL)
> > +				mtk_vcodec_mem_free(inst->ctx,
> > +						    &inst->work_bufs[i]);
> > +	}
> > +
> > +	if (inst->pps_buf.va != NULL)
> > +		mtk_vcodec_mem_free(inst->ctx, &inst->pps_buf);
> > +
> > +	mtk_vcodec_debug_leave(inst);
> > +}
> > +
> > +static int h264_enc_alloc_work_buf(struct venc_h264_inst *inst, void *param)
> > +{
> > +	int i;
> > +	int ret = 0;
> > +	struct venc_h264_vpu_buf *wb = inst->vpu_inst.drv->work_bufs;
> > +	struct venc_enc_prm *enc_param = param;
> > +
> > +	mtk_vcodec_debug_enter(inst);
> > +
> > +	for (i = 0; i < VENC_H264_VPU_WORK_BUF_MAX; i++) {
> > +		/*
> > +		 * This 'wb' structure is set by VPU side and shared to AP for
> > +		 * buffer allocation and IO virtual addr mapping. For most of
> > +		 * the buffers, AP will allocate the buffer according to 'size'
> > +		 * field and store the IO virtual addr in 'iova' field. There
> > +		 * are two exceptions:
> > +		 * (1) RC_CODE buffer, it's pre-allocated in the VPU side, and
> > +		 * save the VPU addr in the 'vpua' field. The AP will translate
> > +		 * the VPU addr to the corresponding IO virtual addr and store
> > +		 * in 'iova' field for reg setting in VPU side.
> > +		 * (2) SKIP_FRAME buffer, it's pre-allocated in the VPU side,
> > +		 * and save the VPU addr in the 'vpua' field. The AP will
> > +		 * translate the VPU addr to the corresponding AP side virtual
> > +		 * address and do some memcpy access to move to bitstream buffer
> > +		 * assigned by v4l2 layer.
> > +		 */
> > +		if (i == VENC_H264_VPU_WORK_BUF_RC_CODE) {
> > +			void *tmp_va;
> > +
> > +			tmp_va = vpu_mapping_dm_addr(inst->dev, wb[i].vpua);
> > +			inst->work_bufs[i].size = wb[i].size;
> > +			if (mtk_vcodec_mem_alloc(inst->ctx,
> > +						 &inst->work_bufs[i])) {
> > +				mtk_vcodec_err(inst,
> > +					       "cannot allocate buf %d", i);
> > +				ret = -ENOMEM;
> > +				goto err_alloc;
> > +			}
> > +			memcpy(inst->work_bufs[i].va, tmp_va, wb[i].size);
> > +			wb[i].iova = inst->work_bufs[i].dma_addr;
> > +		} else if (i == VENC_H264_VPU_WORK_BUF_SKIP_FRAME) {
> > +			inst->work_bufs[i].size = wb[i].size;
> > +			inst->work_bufs[i].va = vpu_mapping_dm_addr(
> > +				inst->dev, wb[i].vpua);
> > +			inst->work_bufs[i].dma_addr = 0;
> > +			wb[i].iova = inst->work_bufs[i].dma_addr;
> > +		} else if (i == VENC_H264_VPU_WORK_BUF_SRC_LUMA ||
> > +			   i == VENC_H264_VPU_WORK_BUF_SRC_CHROMA ||
> > +			   i == VENC_H264_VPU_WORK_BUF_SRC_CHROMA_CB ||
> > +			   i == VENC_H264_VPU_WORK_BUF_SRC_CHROMA_CR) {
> > +			inst->work_bufs[i].size = wb[i].size;
> > +			inst->work_bufs[i].dma_addr = 0;
> > +			inst->work_bufs[i].va = NULL;
> > +			wb[i].iova = inst->work_bufs[i].dma_addr;
> > +		} else {
> > +			inst->work_bufs[i].size = wb[i].size;
> > +			if (mtk_vcodec_mem_alloc(inst->ctx,
> > +						 &inst->work_bufs[i])) {
> > +				mtk_vcodec_err(inst,
> > +					       "cannot allocate buf %d", i);
> > +				ret = -ENOMEM;
> > +				goto err_alloc;
> > +			}
> > +			wb[i].iova = inst->work_bufs[i].dma_addr;
> > +		}
> > +		mtk_vcodec_debug(inst, "buf[%d] va=0x%p iova=0x%p size=0x%lx",
> > +				 i, inst->work_bufs[i].va,
> > +				 (void *)inst->work_bufs[i].dma_addr,
> > +				 inst->work_bufs[i].size);
> > +	}
> > +
> > +	if (enc_param->input_fourcc == VENC_YUV_FORMAT_NV12 ||
> > +	    enc_param->input_fourcc == VENC_YUV_FORMAT_NV21) {
> > +		enc_param->sizeimage[0] =
> > +			inst->work_bufs[VENC_H264_VPU_WORK_BUF_SRC_LUMA].size;
> > +		enc_param->sizeimage[1] =
> > +			inst->work_bufs[VENC_H264_VPU_WORK_BUF_SRC_CHROMA].size;
> > +		enc_param->sizeimage[2] = 0;
> > +	} else {
> > +		enc_param->sizeimage[0] =
> > +			inst->work_bufs[VENC_H264_VPU_WORK_BUF_SRC_LUMA].size;
> > +		enc_param->sizeimage[1] =
> > +			inst->work_bufs[VENC_H264_VPU_WORK_BUF_SRC_CHROMA_CB].size;
> > +		enc_param->sizeimage[2] =
> > +			inst->work_bufs[VENC_H264_VPU_WORK_BUF_SRC_CHROMA_CR].size;
> > +	}
> > +
> > +	/* the pps_buf is used by AP side only */
> > +	inst->pps_buf.size = 128;
> > +	if (mtk_vcodec_mem_alloc(inst->ctx,
> > +				 &inst->pps_buf)) {
> > +		mtk_vcodec_err(inst, "cannot allocate pps_buf");
> > +		ret = -ENOMEM;
> > +		goto err_alloc;
> > +	}
> > +	mtk_vcodec_debug_leave(inst);
> > +
> > +	return ret;
> > +
> > +err_alloc:
> > +	h264_enc_free_work_buf(inst);
> > +
> > +	return ret;
> > +}
> > +
> > +static unsigned int h264_enc_wait_venc_done(struct venc_h264_inst *inst)
> > +{
> > +	unsigned int irq_status = 0;
> > +	struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)inst->ctx;
> > +
> > +	mtk_vcodec_wait_for_done_ctx(ctx, MTK_INST_IRQ_RECEIVED,
> > +				     WAIT_INTR_TIMEOUT, true);
> > +	irq_status = ctx->irq_status;
> > +	mtk_vcodec_debug(inst, "irq_status %x <-", irq_status);
> > +
> > +	return irq_status;
> > +}
> > +
> > +static int h264_encode_sps(struct venc_h264_inst *inst,
> > +			   struct mtk_vcodec_mem *bs_buf,
> > +			   unsigned int *bs_size)
> > +{
> > +	int ret = 0;
> > +	unsigned int irq_status;
> > +
> > +	mtk_vcodec_debug_enter(inst);
> > +
> > +	ret = h264_enc_vpu_encode(inst, H264_BS_MODE_SPS, NULL,
> > +				  bs_buf, bs_size);
> > +	if (ret)
> > +		return ret;
> > +
> > +	irq_status = h264_enc_wait_venc_done(inst);
> > +	if (irq_status != H264_IRQ_STATUS_ENC_SPS_INT) {
> > +		mtk_vcodec_err(inst, "expect irq status %d",
> > +			       H264_IRQ_STATUS_ENC_SPS_INT);
> > +		return -EINVAL;
> > +	}
> > +
> > +	*bs_size = h264_read_reg(inst, VENC_PIC_BITSTREAM_BYTE_CNT);
> > +	mtk_vcodec_debug(inst, "bs size %d <-", *bs_size);
> > +
> > +	return ret;
> > +}
> > +
> > +static int h264_encode_pps(struct venc_h264_inst *inst,
> > +			   struct mtk_vcodec_mem *bs_buf,
> > +			   unsigned int *bs_size)
> > +{
> > +	int ret = 0;
> > +	unsigned int irq_status;
> > +
> > +	mtk_vcodec_debug_enter(inst);
> > +
> > +	ret = h264_enc_vpu_encode(inst, H264_BS_MODE_PPS, NULL,
> > +				  bs_buf, bs_size);
> > +	if (ret)
> > +		return ret;
> > +
> > +	irq_status = h264_enc_wait_venc_done(inst);
> > +	if (irq_status != H264_IRQ_STATUS_ENC_PPS_INT) {
> > +		mtk_vcodec_err(inst, "expect irq status %d",
> > +			       H264_IRQ_STATUS_ENC_PPS_INT);
> > +		return -EINVAL;
> > +	}
> > +
> > +	*bs_size = h264_read_reg(inst, VENC_PIC_BITSTREAM_BYTE_CNT);
> > +	mtk_vcodec_debug(inst, "bs size %d <-", *bs_size);
> > +
> > +	return ret;
> > +}
> > +
> > +static int h264_encode_frame(struct venc_h264_inst *inst,
> > +			     struct venc_frm_buf *frm_buf,
> > +			     struct mtk_vcodec_mem *bs_buf,
> > +			     unsigned int *bs_size)
> > +{
> > +	int ret = 0;
> > +	unsigned int irq_status;
> > +
> > +	mtk_vcodec_debug_enter(inst);
> > +
> > +	ret = h264_enc_vpu_encode(inst, H264_BS_MODE_FRAME, frm_buf,
> > +				  bs_buf, bs_size);
> > +	if (ret)
> > +		return ret;
> > +
> > +	/*
> > +	 * skip frame case: The skip frame buffer is composed by vpu side only,
> > +	 * it does not trigger the hw, so skip the wait interrupt operation.
> > +	 */
> > +	if (!inst->vpu_inst.wait_int) {
> > +		++inst->frm_cnt;
> > +		return ret;
> > +	}
> > +
> > +	irq_status = h264_enc_wait_venc_done(inst);
> > +	if (irq_status != H264_IRQ_STATUS_ENC_FRM_INT) {
> > +		mtk_vcodec_err(inst, "irq_status=%d failed", irq_status);
> > +		return -EINVAL;
> > +	}
> > +
> > +	*bs_size = h264_read_reg(inst,
> > +				 VENC_PIC_BITSTREAM_BYTE_CNT);
> > +	++inst->frm_cnt;
> > +	mtk_vcodec_debug(inst, "frm %d bs size %d key_frm %d <-",
> > +			 inst->frm_cnt,
> > +			 *bs_size, inst->is_key_frm);
> > +
> > +	return ret;
> > +}
> > +
> > +static void h264_encode_filler(struct venc_h264_inst *inst, void *buf,
> > +			       int size)
> > +{
> > +	unsigned char *p = buf;
> > +
> > +	*p++ = 0x0;
> > +	*p++ = 0x0;
> > +	*p++ = 0x0;
> > +	*p++ = 0x1;
> > +	*p++ = 0xc;
> > +	size -= 5;
> > +	while (size) {
> > +		*p++ = 0xff;
> > +		size -= 1;
> > +	}
> > +}
> > +
> > +static int h264_enc_init(struct mtk_vcodec_ctx *ctx, unsigned long *handle)
> > +{
> > +	int ret = 0;
> > +	struct venc_h264_inst *inst;
> > +
> > +	inst = kzalloc(sizeof(*inst), GFP_KERNEL);
> > +	if (!inst)
> > +		return -ENOMEM;
> > +
> > +	inst->ctx = ctx;
> > +	inst->dev = mtk_vcodec_get_plat_dev(ctx);
> > +	inst->hw_base = mtk_vcodec_get_reg_addr(inst->ctx, VENC_SYS);
> > +
> > +	mtk_vcodec_debug_enter(inst);
> > +	ret = h264_enc_vpu_init(inst);
> > +	if (ret)
> > +		kfree(inst);
> > +	else
> > +		(*handle) = (unsigned long)inst;
> > +
> > +	mtk_vcodec_debug_leave(inst);
> > +
> > +	return ret;
> > +}
> > +
> > +static int h264_enc_encode(unsigned long handle,
> > +			   enum venc_start_opt opt,
> > +			   struct venc_frm_buf *frm_buf,
> > +			   struct mtk_vcodec_mem *bs_buf,
> > +			   struct venc_done_result *result)
> > +{
> > +	int ret = 0;
> > +	struct venc_h264_inst *inst = (struct venc_h264_inst *)handle;
> > +	struct mtk_vcodec_ctx *ctx = inst->ctx;
> > +
> > +	mtk_vcodec_debug(inst, "opt %d ->", opt);
> > +
> > +	enable_irq(ctx->dev->enc_irq);
> > +
> > +	switch (opt) {
> > +	case VENC_START_OPT_ENCODE_SEQUENCE_HEADER: {
> > +		unsigned int bs_size_sps;
> > +		unsigned int bs_size_pps;
> > +
> > +		memset(bs_buf->va, 0x38, 20);
> > +		ret = h264_encode_sps(inst, bs_buf, &bs_size_sps);
> > +		if (ret)
> > +			goto encode_err;
> > +
> > +		memset(inst->pps_buf.va, 0x49, 20);
> > +		ret = h264_encode_pps(inst, &inst->pps_buf, &bs_size_pps);
> > +		if (ret)
> > +			goto encode_err;
> > +
> > +		memcpy(bs_buf->va + bs_size_sps,
> > +		       inst->pps_buf.va,
> > +		       bs_size_pps);
> > +		result->bs_size = bs_size_sps + bs_size_pps;
> > +		result->is_key_frm = false;
> > +	}
> > +	break;
> 
> Wrong indentation for break, put it before the '}'.
> 

I will change this in next version. Thanks.

> > +
> > +	case VENC_START_OPT_ENCODE_FRAME:
> > +		if (inst->prepend_hdr) {
> 
> Invert the condition to make it more readable:
> 
> 		if (!inst->prepend_hdr) {
> 			ret = h264_encode_frame(inst, frm_buf, bs_buf,
> 						&result->bs_size);
> 			if (ret)
> 				goto encode_err;
> 			result->is_key_frm = inst->is_key_frm;
> 			break;
> 		}
> 
> Now you can shift the remainder one tab to the left and join lines that
> are now broken up (probably because they exceed 80 chars).
> 

I will change this in next version. Thanks.

> > +			int hdr_sz;
> > +			int hdr_sz_ext;
> > +			int bs_alignment = 128;
> > +			int filler_sz = 0;
> > +			struct mtk_vcodec_mem tmp_bs_buf;
> > +			unsigned int bs_size_sps;
> > +			unsigned int bs_size_pps;
> > +			unsigned int bs_size_frm;
> > +
> > +			mtk_vcodec_debug(inst,
> > +					 "h264_encode_frame prepend SPS/PPS");
> > +			ret = h264_encode_sps(inst, bs_buf, &bs_size_sps);
> > +			if (ret)
> > +				goto encode_err;
> > +
> > +			ret = h264_encode_pps(inst, &inst->pps_buf,
> > +					      &bs_size_pps);
> > +			if (ret)
> > +				goto encode_err;
> > +
> > +			memcpy(bs_buf->va + bs_size_sps,
> > +			       inst->pps_buf.va,
> > +			       bs_size_pps);
> > +
> > +			hdr_sz = bs_size_sps + bs_size_pps;
> > +			hdr_sz_ext = (hdr_sz & (bs_alignment - 1));
> > +			if (hdr_sz_ext) {
> > +				filler_sz = bs_alignment - hdr_sz_ext;
> > +				if (hdr_sz_ext + 5 > bs_alignment)
> > +					filler_sz += bs_alignment;
> > +				h264_encode_filler(
> > +					inst, bs_buf->va + hdr_sz,
> > +					filler_sz);
> > +			}
> > +
> > +			tmp_bs_buf.va = bs_buf->va + hdr_sz +
> > +				filler_sz;
> > +			tmp_bs_buf.dma_addr = bs_buf->dma_addr + hdr_sz +
> > +				filler_sz;
> > +			tmp_bs_buf.size = bs_buf->size -
> > +				(hdr_sz + filler_sz);
> > +
> > +			ret = h264_encode_frame(inst, frm_buf, &tmp_bs_buf,
> > +						&bs_size_frm);
> > +			if (ret)
> > +				goto encode_err;
> > +
> > +			result->bs_size = hdr_sz + filler_sz + bs_size_frm;
> > +			mtk_vcodec_debug(inst,
> > +					 "hdr %d filler %d frame %d bs %d",
> > +					 hdr_sz, filler_sz, bs_size_frm,
> > +					 result->bs_size);
> > +
> > +			inst->prepend_hdr = 0;
> > +		} else {
> > +			ret = h264_encode_frame(inst, frm_buf, bs_buf,
> > +						&result->bs_size);
> > +			if (ret)
> > +				goto encode_err;
> > +		}
> > +		result->is_key_frm = inst->is_key_frm;
> > +		break;
> > +
> > +	default:
> > +		mtk_vcodec_err(inst, "venc_start_opt %d not supported", opt);
> > +		ret = -EINVAL;
> > +		break;
> > +	}
> > +
> > +encode_err:
> > +	if (ret)
> > +		result->msg = VENC_MESSAGE_ERR;
> > +	else
> > +		result->msg = VENC_MESSAGE_OK;
> > +
> > +	disable_irq(ctx->dev->enc_irq);
> > +	mtk_vcodec_debug(inst, "opt %d <-", opt);
> > +
> > +	return ret;
> > +}
> > +
> > +static int h264_enc_set_param(unsigned long handle,
> > +			      enum venc_set_param_type type, void *in)
> > +{
> > +	int ret = 0;
> > +	struct venc_h264_inst *inst = (struct venc_h264_inst *)handle;
> > +	struct venc_enc_prm *enc_prm;
> > +
> > +	mtk_vcodec_debug(inst, "->type=%d", type);
> > +
> > +	switch (type) {
> > +	case VENC_SET_PARAM_ENC:
> > +		enc_prm = in;
> > +		ret = h264_enc_vpu_set_param(inst, type, enc_prm);
> > +		if (ret)
> > +			break;
> > +		if (inst->work_buf_allocated == 1) {
> > +			h264_enc_free_work_buf(inst);
> > +			inst->work_buf_allocated = 0;
> > +		}
> > +		if (inst->work_buf_allocated == 0) {
> > +			ret = h264_enc_alloc_work_buf(inst, enc_prm);
> > +			if (ret)
> > +				break;
> > +			inst->work_buf_allocated = 1;
> > +		}
> > +		break;
> > +
> > +	case VENC_SET_PARAM_FORCE_INTRA:
> > +		ret = h264_enc_vpu_set_param(inst, type, 0);
> > +		break;
> > +
> > +	case VENC_SET_PARAM_ADJUST_BITRATE:
> > +		enc_prm = in;
> > +		ret = h264_enc_vpu_set_param(inst, type, &enc_prm->bitrate);
> > +		break;
> > +
> > +	case VENC_SET_PARAM_ADJUST_FRAMERATE:
> > +		enc_prm = in;
> > +		ret = h264_enc_vpu_set_param(inst, type, &enc_prm->frm_rate);
> > +		break;
> > +
> > +	case VENC_SET_PARAM_I_FRAME_INTERVAL:
> > +		ret = h264_enc_vpu_set_param(inst, type, in);
> > +		break;
> > +
> > +	case VENC_SET_PARAM_SKIP_FRAME:
> > +		ret = h264_enc_vpu_set_param(inst, type, 0);
> > +		break;
> > +
> > +	case VENC_SET_PARAM_PREPEND_HEADER:
> > +		inst->prepend_hdr = 1;
> > +		mtk_vcodec_debug(inst, "set prepend header mode");
> > +		break;
> > +
> > +	default:
> > +		mtk_vcodec_err(inst, "type %d not supported", type);
> > +		ret = -EINVAL;
> > +		break;
> > +	}
> > +
> > +	mtk_vcodec_debug_leave(inst);
> > +
> > +	return ret;
> > +}
> > +
> > +static int h264_enc_deinit(unsigned long handle)
> > +{
> > +	int ret = 0;
> > +	struct venc_h264_inst *inst = (struct venc_h264_inst *)handle;
> > +
> > +	mtk_vcodec_debug_enter(inst);
> > +
> > +	ret = h264_enc_vpu_deinit(inst);
> > +
> > +	if (inst->work_buf_allocated)
> > +		h264_enc_free_work_buf(inst);
> > +
> > +	mtk_vcodec_debug_leave(inst);
> > +	kfree(inst);
> > +
> > +	return ret;
> > +}
> > +
> > +static struct venc_common_if venc_h264_if = {
> > +	h264_enc_init,
> > +	h264_enc_encode,
> > +	h264_enc_set_param,
> > +	h264_enc_deinit,
> > +};
> > +
> > +struct venc_common_if *get_h264_enc_comm_if(void)
> > +{
> > +	return &venc_h264_if;
> > +}
> > diff --git a/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.h b/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.h
> > new file mode 100644
> > index 0000000..9ac317a
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.h
> > @@ -0,0 +1,165 @@
> > +/*
> > + * Copyright (c) 2015 MediaTek Inc.
> > + * Author: Jungchang Tsao <jungchang.tsao@mediatek.com>
> > + *         Daniel Hsiao <daniel.hsiao@mediatek.com>
> > + *         PoChun Lin <pochun.lin@mediatek.com>
> > + *
> > + * This program is free software; you can redistribute it and/or
> > + * modify
> > + * it under the terms of the GNU General Public License version 2 as
> > + * published by the Free Software Foundation.
> > + *
> > + * This program is distributed in the hope that it will be useful,
> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > + * GNU General Public License for more details.
> > + */
> > +
> > +#ifndef _VENC_H264_IF_H_
> > +#define _VENC_H264_IF_H_
> > +
> > +#include "venc_drv_base.h"
> > +
> > +/**
> > + * enum venc_h264_vpu_work_buf - h264 encoder buffer index
> > + */
> > +enum venc_h264_vpu_work_buf {
> > +	VENC_H264_VPU_WORK_BUF_RC_INFO,
> > +	VENC_H264_VPU_WORK_BUF_RC_CODE,
> > +	VENC_H264_VPU_WORK_BUF_REC_LUMA,
> > +	VENC_H264_VPU_WORK_BUF_REC_CHROMA,
> > +	VENC_H264_VPU_WORK_BUF_REF_LUMA,
> > +	VENC_H264_VPU_WORK_BUF_REF_CHROMA,
> > +	VENC_H264_VPU_WORK_BUF_MV_INFO_1,
> > +	VENC_H264_VPU_WORK_BUF_MV_INFO_2,
> > +	VENC_H264_VPU_WORK_BUF_SKIP_FRAME,
> > +	VENC_H264_VPU_WORK_BUF_SRC_LUMA,
> > +	VENC_H264_VPU_WORK_BUF_SRC_CHROMA,
> > +	VENC_H264_VPU_WORK_BUF_SRC_CHROMA_CB,
> > +	VENC_H264_VPU_WORK_BUF_SRC_CHROMA_CR,
> > +	VENC_H264_VPU_WORK_BUF_MAX,
> > +};
> > +
> > +/**
> > + * enum venc_h264_bs_mode - for bs_mode argument in h264_enc_vpu_encode
> > + */
> > +enum venc_h264_bs_mode {
> > +	H264_BS_MODE_SPS,
> > +	H264_BS_MODE_PPS,
> > +	H264_BS_MODE_FRAME,
> > +};
> > +
> > +/*
> > + * struct venc_h264_vpu_config - Structure for h264 encoder configuration
> > + * @input_fourcc: input fourcc
> > + * @bitrate: target bitrate (in bps)
> > + * @pic_w: picture width
> > + * @pic_h: picture height
> > + * @buf_w: buffer width
> > + * @buf_h: buffer height
> > + * @intra_period: intra frame period
> > + * @framerate: frame rate
> > + * @profile: as specified in standard
> > + * @level: as specified in standard
> > + * @wfd: WFD mode 1:on, 0:off
> > + */
> > +struct venc_h264_vpu_config {
> > +	u32 input_fourcc;
> > +	u32 bitrate;
> > +	u32 pic_w;
> > +	u32 pic_h;
> > +	u32 buf_w;
> > +	u32 buf_h;
> > +	u32 intra_period;
> > +	u32 framerate;
> > +	u32 profile;
> > +	u32 level;
> > +	u32 wfd;
> > +};
> > +
> > +/*
> > + * struct venc_h264_vpu_buf - Structure for buffer information
> > + * @align: buffer alignment (in bytes)
> > + * @iova: IO virtual address
> > + * @vpua: VPU side memory addr which is used by RC_CODE
> > + * @size: buffer size (in bytes)
> > + */
> > +struct venc_h264_vpu_buf {
> > +	u32 align;
> > +	u32 iova;
> > +	u32 vpua;
> > +	u32 size;
> > +};
> > +
> > +/*
> > + * struct venc_h264_vpu_drv - Structure for VPU driver control and info share
> > + * This structure is allocated in VPU side and shared to AP side.
> > + * @config: h264 encoder configuration
> > + * @work_bufs: working buffer information in VPU side
> > + * The work_bufs here is for storing the 'size' info shared to AP side.
> > + * The similar item in struct venc_h264_inst is for memory allocation
> > + * in AP side. The AP driver will copy the 'size' from here to the one in
> > + * struct mtk_vcodec_mem, then invoke mtk_vcodec_mem_alloc to allocate
> > + * the buffer. After that, bypass the 'dma_addr' to the 'iova' field here for
> > + * register setting in VPU side.
> > + */
> > +struct venc_h264_vpu_drv {
> > +	struct venc_h264_vpu_config config;
> > +	struct venc_h264_vpu_buf work_bufs[VENC_H264_VPU_WORK_BUF_MAX];
> > +};
> > +
> > +/*
> > + * struct venc_h264_vpu_inst - h264 encoder VPU driver instance
> > + * @wq_hd: wait queue used for vpu cmd trigger then wait vpu interrupt done
> > + * @signaled: flag used for checking vpu interrupt done
> > + * @failure: flag to show vpu cmd succeeds or not
> > + * @state: enum venc_ipi_msg_enc_state
> > + * @bs_size: bitstream size for skip frame case usage
> > + * @wait_int: flag to wait interrupt done (0: for skip frame case, 1: normal
> > + *	      case)
> > + * @id: VPU instance id
> > + * @drv: driver structure allocated by VPU side and shared to AP side for
> > + *	 control and info share
> > + */
> > +struct venc_h264_vpu_inst {
> > +	wait_queue_head_t wq_hd;
> > +	int signaled;
> > +	int failure;
> > +	int state;
> > +	int bs_size;
> > +	int wait_int;
> > +	unsigned int id;
> > +	struct venc_h264_vpu_drv *drv;
> > +};
> > +
> > +/*
> > + * struct venc_h264_inst - h264 encoder AP driver instance
> > + * @hw_base: h264 encoder hardware register base
> > + * @work_bufs: working buffer
> > + * @pps_buf: buffer to store the pps bitstream
> > + * @work_buf_allocated: working buffer allocated flag
> > + * @frm_cnt: encoded frame count
> > + * @prepend_hdr: when the v4l2 layer send VENC_SET_PARAM_PREPEND_HEADER cmd
> > + *  through h264_enc_set_param interface, it will set this flag and prepend the
> > + *  sps/pps in h264_enc_encode function.
> > + * @is_key_frm: key frame flag
> > + * @vpu_inst: VPU instance to exchange information between AP and VPU
> > + * @ctx: context for v4l2 layer integration
> > + * @dev: device for v4l2 layer integration
> > + */
> > +struct venc_h264_inst {
> > +	void __iomem *hw_base;
> > +	struct mtk_vcodec_mem work_bufs[VENC_H264_VPU_WORK_BUF_MAX];
> > +	struct mtk_vcodec_mem pps_buf;
> > +	bool work_buf_allocated;
> > +	unsigned int frm_cnt;
> > +	unsigned int prepend_hdr;
> > +	unsigned int is_key_frm;
> > +	struct venc_h264_vpu_inst vpu_inst;
> > +	void *ctx;
> > +	struct platform_device *dev;
> > +};
> > +
> > +struct venc_common_if *get_h264_enc_comm_if(void);
> > +
> > +#endif
> > diff --git a/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.c b/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.c
> > new file mode 100644
> > index 0000000..450be45
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.c
> > @@ -0,0 +1,309 @@
> > +/*
> > + * Copyright (c) 2015 MediaTek Inc.
> > + * Author: Jungchang Tsao <jungchang.tsao@mediatek.com>
> > + *         Daniel Hsiao <daniel.hsiao@mediatek.com>
> > + *         PoChun Lin <pochun.lin@mediatek.com>
> > + *
> > + * This program is free software; you can redistribute it and/or
> > + * modify
> > + * it under the terms of the GNU General Public License version 2 as
> > + * published by the Free Software Foundation.
> > + *
> > + * This program is distributed in the hope that it will be useful,
> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > + * GNU General Public License for more details.
> > + */
> > +
> > +#include "mtk_vpu.h"
> > +
> > +#include "venc_h264_if.h"
> > +#include "venc_h264_vpu.h"
> > +#include "venc_ipi_msg.h"
> > +
> > +static unsigned int h264_get_profile(unsigned int profile)
> > +{
> > +	/* (Baseline=66, Main=77, High=100) */
> > +	switch (profile) {
> > +	case V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE:
> > +		return 66;
> > +	case V4L2_MPEG_VIDEO_H264_PROFILE_MAIN:
> > +		return 77;
> > +	case V4L2_MPEG_VIDEO_H264_PROFILE_HIGH:
> > +		return 100;
> > +	default:
> > +		return 100;
> > +	}
> > +}
> > +
> > +static unsigned int h264_get_level(unsigned int level)
> > +{
> > +	/* (UpTo4.1(HighProfile)) */
> > +	switch (level) {
> > +	case V4L2_MPEG_VIDEO_H264_LEVEL_1_0:
> > +		return 10;
> > +	case V4L2_MPEG_VIDEO_H264_LEVEL_1_1:
> > +		return 11;
> > +	case V4L2_MPEG_VIDEO_H264_LEVEL_1_2:
> > +		return 12;
> > +	case V4L2_MPEG_VIDEO_H264_LEVEL_1_3:
> > +		return 13;
> > +	case V4L2_MPEG_VIDEO_H264_LEVEL_2_0:
> > +		return 20;
> > +	case V4L2_MPEG_VIDEO_H264_LEVEL_2_1:
> > +		return 21;
> > +	case V4L2_MPEG_VIDEO_H264_LEVEL_2_2:
> > +		return 22;
> > +	case V4L2_MPEG_VIDEO_H264_LEVEL_3_0:
> > +		return 30;
> > +	case V4L2_MPEG_VIDEO_H264_LEVEL_3_1:
> > +		return 31;
> > +	case V4L2_MPEG_VIDEO_H264_LEVEL_3_2:
> > +		return 32;
> > +	case V4L2_MPEG_VIDEO_H264_LEVEL_4_0:
> > +		return 40;
> > +	case V4L2_MPEG_VIDEO_H264_LEVEL_4_1:
> > +		return 41;
> > +	default:
> > +		return 31;
> > +	}
> > +}
> > +
> > +static void handle_h264_enc_init_msg(struct venc_h264_inst *inst, void *data)
> > +{
> > +	struct venc_vpu_ipi_msg_init *msg = data;
> > +
> > +	inst->vpu_inst.id = msg->inst_id;
> > +	inst->vpu_inst.drv = (struct venc_h264_vpu_drv *)vpu_mapping_dm_addr(
> > +		inst->dev, msg->inst_id);
> > +}
> > +
> > +static void handle_h264_enc_encode_msg(struct venc_h264_inst *inst, void *data)
> > +{
> > +	struct venc_vpu_ipi_msg_enc *msg = data;
> > +
> > +	inst->vpu_inst.state = msg->state;
> > +	inst->vpu_inst.bs_size = msg->bs_size;
> > +	inst->is_key_frm = msg->key_frame;
> > +}
> > +
> > +static void h264_enc_vpu_ipi_handler(void *data, unsigned int len, void *priv)
> > +{
> > +	struct venc_vpu_ipi_msg_common *msg = data;
> > +	struct venc_h264_inst *inst = (struct venc_h264_inst *)msg->venc_inst;
> > +
> > +	mtk_vcodec_debug(inst, "msg_id %x inst %p status %d",
> > +			 msg->msg_id, inst, msg->status);
> > +
> > +	switch (msg->msg_id) {
> > +	case VPU_IPIMSG_H264_ENC_INIT_DONE:
> > +		handle_h264_enc_init_msg(inst, data);
> > +		break;
> > +	case VPU_IPIMSG_H264_ENC_SET_PARAM_DONE:
> > +		break;
> > +	case VPU_IPIMSG_H264_ENC_ENCODE_DONE:
> > +		handle_h264_enc_encode_msg(inst, data);
> > +		break;
> > +	case VPU_IPIMSG_H264_ENC_DEINIT_DONE:
> > +		break;
> > +	default:
> > +		mtk_vcodec_err(inst, "unknown msg id %x", msg->msg_id);
> > +		break;
> > +	}
> > +
> > +	inst->vpu_inst.signaled = 1;
> > +	inst->vpu_inst.failure = (msg->status != VENC_IPI_MSG_STATUS_OK);
> > +
> > +	mtk_vcodec_debug_leave(inst);
> > +}
> > +
> > +static int h264_enc_vpu_send_msg(struct venc_h264_inst *inst, void *msg,
> > +				 int len)
> > +{
> > +	int status;
> > +
> > +	mtk_vcodec_debug_enter(inst);
> > +	status = vpu_ipi_send(inst->dev, IPI_VENC_H264, msg, len);
> > +	if (status) {
> > +		mtk_vcodec_err(inst, "vpu_ipi_send msg %x len %d fail %d",
> > +			       *(unsigned int *)msg, len, status);
> > +		return -EINVAL;
> > +	}
> > +	mtk_vcodec_debug_leave(inst);
> > +
> > +	return 0;
> > +}
> > +
> > +int h264_enc_vpu_init(struct venc_h264_inst *inst)
> > +{
> > +	int status;
> > +	struct venc_ap_ipi_msg_init out;
> > +
> > +	mtk_vcodec_debug_enter(inst);
> > +
> > +	init_waitqueue_head(&inst->vpu_inst.wq_hd);
> > +	inst->vpu_inst.signaled = 0;
> > +	inst->vpu_inst.failure = 0;
> > +
> > +	status = vpu_ipi_register(inst->dev, IPI_VENC_H264,
> > +				  h264_enc_vpu_ipi_handler,
> > +				  "h264_enc", NULL);
> > +	if (status) {
> > +		mtk_vcodec_err(inst, "vpu_ipi_register fail %d", status);
> > +		return -EINVAL;
> > +	}
> > +
> > +	out.msg_id = AP_IPIMSG_H264_ENC_INIT;
> > +	out.venc_inst = (unsigned long)inst;
> > +	if (h264_enc_vpu_send_msg(inst, &out, sizeof(out)) ||
> > +	    inst->vpu_inst.failure) {
> > +		mtk_vcodec_err(inst, "AP_IPIMSG_H264_ENC_INIT failed");
> > +		return -EINVAL;
> > +	}
> > +
> > +	mtk_vcodec_debug_leave(inst);
> > +
> > +	return 0;
> > +}
> > +
> > +int h264_enc_vpu_set_param(struct venc_h264_inst *inst, unsigned int id,
> > +			   void *param)
> > +{
> > +	struct venc_ap_ipi_msg_set_param out;
> > +
> > +	mtk_vcodec_debug(inst, "id %d ->", id);
> > +
> > +	out.msg_id = AP_IPIMSG_H264_ENC_SET_PARAM;
> > +	out.inst_id = inst->vpu_inst.id;
> > +	out.param_id = id;
> > +	switch (id) {
> > +	case VENC_SET_PARAM_ENC: {
> > +		struct venc_enc_prm *enc_param = (struct venc_enc_prm *)param;
> > +
> > +		inst->vpu_inst.drv->config.input_fourcc =
> > +			enc_param->input_fourcc;
> > +		inst->vpu_inst.drv->config.bitrate = enc_param->bitrate;
> > +		inst->vpu_inst.drv->config.pic_w = enc_param->width;
> > +		inst->vpu_inst.drv->config.pic_h = enc_param->height;
> > +		inst->vpu_inst.drv->config.buf_w = enc_param->buf_width;
> > +		inst->vpu_inst.drv->config.buf_h = enc_param->buf_height;
> > +		inst->vpu_inst.drv->config.intra_period =
> > +			enc_param->intra_period;
> > +		inst->vpu_inst.drv->config.framerate = enc_param->frm_rate;
> > +		inst->vpu_inst.drv->config.profile =
> > +			h264_get_profile(enc_param->h264_profile);
> > +		inst->vpu_inst.drv->config.level =
> > +			h264_get_level(enc_param->h264_level);
> > +		inst->vpu_inst.drv->config.wfd = 0;
> > +		out.data_item = 0;
> > +		break;
> > +	}
> > +	case VENC_SET_PARAM_FORCE_INTRA:
> > +		out.data_item = 0;
> > +		break;
> > +	case VENC_SET_PARAM_ADJUST_BITRATE:
> > +		out.data_item = 1;
> > +		out.data[0] = *(unsigned int *)param;
> > +		break;
> > +	case VENC_SET_PARAM_ADJUST_FRAMERATE:
> > +		out.data_item = 1;
> > +		out.data[0] = *(unsigned int *)param;
> > +		break;
> > +	case VENC_SET_PARAM_I_FRAME_INTERVAL:
> > +		out.data_item = 1;
> > +		out.data[0] = *(unsigned int *)param;
> > +		break;
> > +	case VENC_SET_PARAM_SKIP_FRAME:
> > +		out.data_item = 0;
> > +		break;
> > +	}
> > +	if (h264_enc_vpu_send_msg(inst, &out, sizeof(out)) ||
> > +	    inst->vpu_inst.failure) {
> > +		mtk_vcodec_err(inst,
> > +			       "AP_IPIMSG_H264_ENC_SET_PARAM %d fail", id);
> > +		return -EINVAL;
> > +	}
> > +
> > +	mtk_vcodec_debug(inst, "id %d <-", id);
> > +
> > +	return 0;
> > +}
> > +
> > +int h264_enc_vpu_encode(struct venc_h264_inst *inst, unsigned int bs_mode,
> > +			struct venc_frm_buf *frm_buf,
> > +			struct mtk_vcodec_mem *bs_buf,
> > +			unsigned int *bs_size)
> > +{
> > +	struct venc_ap_ipi_msg_enc out;
> > +
> > +	mtk_vcodec_debug(inst, "bs_mode %d ->", bs_mode);
> > +
> > +	out.msg_id = AP_IPIMSG_H264_ENC_ENCODE;
> > +	out.inst_id = inst->vpu_inst.id;
> > +	out.bs_mode = bs_mode;
> > +	if (frm_buf) {
> > +		if ((frm_buf->fb_addr.dma_addr % 16 == 0) &&
> > +		    (frm_buf->fb_addr1.dma_addr % 16 == 0) &&
> > +		    (frm_buf->fb_addr2.dma_addr % 16 == 0)) {
> > +			out.input_addr[0] = frm_buf->fb_addr.dma_addr;
> > +			out.input_addr[1] = frm_buf->fb_addr1.dma_addr;
> > +			out.input_addr[2] = frm_buf->fb_addr2.dma_addr;
> > +		} else {
> > +			mtk_vcodec_err(inst, "dma_addr not align to 16");
> > +			return -EINVAL;
> > +		}
> > +	} else {
> > +		out.input_addr[0] = 0;
> > +		out.input_addr[1] = 0;
> > +		out.input_addr[2] = 0;
> > +	}
> > +	if (bs_buf) {
> > +		out.bs_addr = bs_buf->dma_addr;
> > +		out.bs_size = bs_buf->size;
> > +	} else {
> > +		out.bs_addr = 0;
> > +		out.bs_size = 0;
> > +	}
> > +	if (h264_enc_vpu_send_msg(inst, &out, sizeof(out)) ||
> > +	    inst->vpu_inst.failure) {
> > +		mtk_vcodec_err(inst, "AP_IPIMSG_H264_ENC_ENCODE %d fail",
> > +			       bs_mode);
> > +		return -EINVAL;
> > +	}
> > +
> > +	mtk_vcodec_debug(inst, "state %d size %d key_frm %d",
> > +			 inst->vpu_inst.state, inst->vpu_inst.bs_size,
> > +			 inst->is_key_frm);
> > +
> > +	inst->vpu_inst.wait_int = 1;
> > +	if (inst->vpu_inst.state == VEN_IPI_MSG_ENC_STATE_SKIP) {
> > +		*bs_size = inst->vpu_inst.bs_size;
> > +		memcpy(bs_buf->va,
> > +		       inst->work_bufs[VENC_H264_VPU_WORK_BUF_SKIP_FRAME].va,
> > +		       *bs_size);
> > +		inst->vpu_inst.wait_int = 0;
> > +	}
> > +
> > +	mtk_vcodec_debug(inst, "bs_mode %d <-", bs_mode);
> > +
> > +	return 0;
> > +}
> > +
> > +int h264_enc_vpu_deinit(struct venc_h264_inst *inst)
> > +{
> > +	struct venc_ap_ipi_msg_deinit out;
> > +
> > +	mtk_vcodec_debug_enter(inst);
> > +
> > +	out.msg_id = AP_IPIMSG_H264_ENC_DEINIT;
> > +	out.inst_id = inst->vpu_inst.id;
> > +	if (h264_enc_vpu_send_msg(inst, &out, sizeof(out)) ||
> > +	    inst->vpu_inst.failure) {
> > +		mtk_vcodec_err(inst, "AP_IPIMSG_H264_ENC_DEINIT fail");
> > +		return -EINVAL;
> > +	}
> > +
> > +	mtk_vcodec_debug_leave(inst);
> > +
> > +	return 0;
> > +}
> > diff --git a/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.h b/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.h
> > new file mode 100644
> > index 0000000..deccc6f
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.h
> > @@ -0,0 +1,30 @@
> > +/*
> > + * Copyright (c) 2015 MediaTek Inc.
> > + * Author: Jungchang Tsao <jungchang.tsao@mediatek.com>
> > + *         Daniel Hsiao <daniel.hsiao@mediatek.com>
> > + *         PoChun Lin <pochun.lin@mediatek.com>
> > + *
> > + * This program is free software; you can redistribute it and/or
> > + * modify
> > + * it under the terms of the GNU General Public License version 2 as
> > + * published by the Free Software Foundation.
> > + *
> > + * This program is distributed in the hope that it will be useful,
> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > + * GNU General Public License for more details.
> > + */
> > +
> > +#ifndef _VENC_H264_VPU_H_
> > +#define _VENC_H264_VPU_H_
> > +
> > +int h264_enc_vpu_init(struct venc_h264_inst *inst);
> > +int h264_enc_vpu_set_param(struct venc_h264_inst *inst, unsigned int id,
> > +			   void *param);
> > +int h264_enc_vpu_encode(struct venc_h264_inst *inst, unsigned int bs_mode,
> > +			struct venc_frm_buf *frm_buf,
> > +			struct mtk_vcodec_mem *bs_buf,
> > +			unsigned int *bs_size);
> > +int h264_enc_vpu_deinit(struct venc_h264_inst *inst);
> > +
> > +#endif
> > diff --git a/drivers/media/platform/mtk-vcodec/venc_drv_if.c b/drivers/media/platform/mtk-vcodec/venc_drv_if.c
> > index d293f2c..28ef4a7 100644
> > --- a/drivers/media/platform/mtk-vcodec/venc_drv_if.c
> > +++ b/drivers/media/platform/mtk-vcodec/venc_drv_if.c
> > @@ -25,6 +25,7 @@
> >  
> >  #include "venc_drv_base.h"
> >  #include "vp8_enc/venc_vp8_if.h"
> > +#include "h264_enc/venc_h264_if.h"
> >  
> >  int venc_if_create(struct mtk_vcodec_ctx *ctx, unsigned int fourcc)
> >  {
> > @@ -38,6 +39,8 @@ int venc_if_create(struct mtk_vcodec_ctx *ctx, unsigned int fourcc)
> >                  ctx->enc_if = get_vp8_enc_comm_if();
> >                  break;
> >  	case V4L2_PIX_FMT_H264:
> > +	        ctx->enc_if = get_h264_enc_comm_if();
> > +	        break;
> >  	default:
> >  		return -EINVAL;
> >  	}
> > 
> 
> Regards,
> 
> 	Hans

Best Regards
PoChun

^ permalink raw reply	[flat|nested] 102+ messages in thread

* Re: [PATCH v4 7/8] [media] vcodec: mediatek: Add Mediatek H264 Video Encoder Driver
@ 2016-02-16 11:57                   ` pochun lin
  0 siblings, 0 replies; 102+ messages in thread
From: pochun lin @ 2016-02-16 11:57 UTC (permalink / raw)
  To: Hans Verkuil
  Cc: Tiffany Lin (林慧珊),
	Hans Verkuil, daniel.thompson, Rob Herring,
	Mauro Carvalho Chehab, Matthias Brugger, Daniel Kurtz,
	Pawel Osciak, Eddie Huang (黃智傑),
	Yingjoe Chen (陳英洲),
	devicetree, linux-kernel, linux-arm-kernel, linux-media

Hi Hans,

On Mon, 2016-02-15 at 19:33 +0800, Hans Verkuil wrote:
> On 02/04/2016 12:35 PM, Tiffany Lin wrote:
> > Add h264 encoder driver for MT8173
> > 
> > Signed-off-by: Daniel Hsiao <daniel.hsiao@mediatek.com>
> > Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
> > ---
> >  drivers/media/platform/mtk-vcodec/Makefile         |    3 +-
> >  .../media/platform/mtk-vcodec/h264_enc/Makefile    |    6 +
> >  .../platform/mtk-vcodec/h264_enc/venc_h264_if.c    |  540 ++++++++++++++++++++
> >  .../platform/mtk-vcodec/h264_enc/venc_h264_if.h    |  165 ++++++
> >  .../platform/mtk-vcodec/h264_enc/venc_h264_vpu.c   |  309 +++++++++++
> >  .../platform/mtk-vcodec/h264_enc/venc_h264_vpu.h   |   30 ++
> >  drivers/media/platform/mtk-vcodec/venc_drv_if.c    |    3 +
> >  7 files changed, 1055 insertions(+), 1 deletion(-)
> >  create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/Makefile
> >  create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.c
> >  create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.h
> >  create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.c
> >  create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.h
> > 
> > diff --git a/drivers/media/platform/mtk-vcodec/Makefile b/drivers/media/platform/mtk-vcodec/Makefile
> > index f4ef502..f47dfc7 100644
> > --- a/drivers/media/platform/mtk-vcodec/Makefile
> > +++ b/drivers/media/platform/mtk-vcodec/Makefile
> > @@ -5,6 +5,7 @@ obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += mtk_vcodec_intr.o \
> >  				       mtk_vcodec_enc_pm.o \
> >  				       venc_drv_if.o
> >  
> > -obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += vp8_enc/
> > +
> > +obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += vp8_enc/ h264_enc/
> >  
> >  ccflags-y += -I$(srctree)/drivers/media/platform/mtk-vpu
> > diff --git a/drivers/media/platform/mtk-vcodec/h264_enc/Makefile b/drivers/media/platform/mtk-vcodec/h264_enc/Makefile
> > new file mode 100644
> > index 0000000..765b45f
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/h264_enc/Makefile
> > @@ -0,0 +1,6 @@
> > +obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += venc_h264_if.o venc_h264_vpu.o
> > +
> > +ccflags-y += \
> > +    -I$(srctree)/drivers/media/platform/mtk-vcodec/ \
> > +    -I$(srctree)/drivers/media/platform/mtk-vcodec/h264_enc \
> > +    -I$(srctree)/drivers/media/platform/mtk-vpu
> > diff --git a/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.c b/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.c
> > new file mode 100644
> > index 0000000..c35fb26
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.c
> > @@ -0,0 +1,540 @@
> > +/*
> > + * Copyright (c) 2015 MediaTek Inc.
> > + * Author: Jungchang Tsao <jungchang.tsao@mediatek.com>
> > + *         Daniel Hsiao <daniel.hsiao@mediatek.com>
> > + *         PoChun Lin <pochun.lin@mediatek.com>
> > + *
> > + * This program is free software; you can redistribute it and/or
> > + * modify
> > + * it under the terms of the GNU General Public License version 2 as
> > + * published by the Free Software Foundation.
> > + *
> > + * This program is distributed in the hope that it will be useful,
> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > + * GNU General Public License for more details.
> > + */
> > +
> > +#include <linux/interrupt.h>
> > +#include <linux/kernel.h>
> > +#include <linux/slab.h>
> > +
> > +#include "mtk_vcodec_drv.h"
> > +#include "mtk_vcodec_util.h"
> > +#include "mtk_vcodec_intr.h"
> > +#include "mtk_vcodec_enc.h"
> > +#include "mtk_vcodec_pm.h"
> > +#include "mtk_vpu.h"
> > +
> > +#include "venc_h264_if.h"
> > +#include "venc_h264_vpu.h"
> > +
> > +#define VENC_PIC_BITSTREAM_BYTE_CNT 0x0098
> > +
> > +enum venc_h264_irq_status {
> > +	H264_IRQ_STATUS_ENC_SPS_INT = (1 << 0),
> > +	H264_IRQ_STATUS_ENC_PPS_INT = (1 << 1),
> > +	H264_IRQ_STATUS_ENC_FRM_INT = (1 << 2),
> > +};
> > +
> > +static inline void h264_write_reg(struct venc_h264_inst *inst, u32 addr,
> > +				  u32 val)
> > +{
> > +	writel(val, inst->hw_base + addr);
> > +}
> > +
> > +static inline u32 h264_read_reg(struct venc_h264_inst *inst, u32 addr)
> > +{
> > +	return readl(inst->hw_base + addr);
> > +}
> > +
> > +static void h264_enc_free_work_buf(struct venc_h264_inst *inst)
> > +{
> > +	int i;
> > +
> > +	mtk_vcodec_debug_enter(inst);
> > +
> > +	/* Except the SKIP_FRAME buffers,
> > +	 * other buffers need to be freed by AP.
> > +	 */
> > +	for (i = 0; i < VENC_H264_VPU_WORK_BUF_MAX; i++) {
> > +		if (i != VENC_H264_VPU_WORK_BUF_SKIP_FRAME)
> > +			if (inst->work_bufs[i].va != NULL)
> > +				mtk_vcodec_mem_free(inst->ctx,
> > +						    &inst->work_bufs[i]);
> > +	}
> > +
> > +	if (inst->pps_buf.va != NULL)
> > +		mtk_vcodec_mem_free(inst->ctx, &inst->pps_buf);
> > +
> > +	mtk_vcodec_debug_leave(inst);
> > +}
> > +
> > +static int h264_enc_alloc_work_buf(struct venc_h264_inst *inst, void *param)
> > +{
> > +	int i;
> > +	int ret = 0;
> > +	struct venc_h264_vpu_buf *wb = inst->vpu_inst.drv->work_bufs;
> > +	struct venc_enc_prm *enc_param = param;
> > +
> > +	mtk_vcodec_debug_enter(inst);
> > +
> > +	for (i = 0; i < VENC_H264_VPU_WORK_BUF_MAX; i++) {
> > +		/*
> > +		 * This 'wb' structure is set by VPU side and shared to AP for
> > +		 * buffer allocation and IO virtual addr mapping. For most of
> > +		 * the buffers, AP will allocate the buffer according to 'size'
> > +		 * field and store the IO virtual addr in 'iova' field. There
> > +		 * are two exceptions:
> > +		 * (1) RC_CODE buffer, it's pre-allocated in the VPU side, and
> > +		 * save the VPU addr in the 'vpua' field. The AP will translate
> > +		 * the VPU addr to the corresponding IO virtual addr and store
> > +		 * in 'iova' field for reg setting in VPU side.
> > +		 * (2) SKIP_FRAME buffer, it's pre-allocated in the VPU side,
> > +		 * and save the VPU addr in the 'vpua' field. The AP will
> > +		 * translate the VPU addr to the corresponding AP side virtual
> > +		 * address and do some memcpy access to move to bitstream buffer
> > +		 * assigned by v4l2 layer.
> > +		 */
> > +		if (i == VENC_H264_VPU_WORK_BUF_RC_CODE) {
> > +			void *tmp_va;
> > +
> > +			tmp_va = vpu_mapping_dm_addr(inst->dev, wb[i].vpua);
> > +			inst->work_bufs[i].size = wb[i].size;
> > +			if (mtk_vcodec_mem_alloc(inst->ctx,
> > +						 &inst->work_bufs[i])) {
> > +				mtk_vcodec_err(inst,
> > +					       "cannot allocate buf %d", i);
> > +				ret = -ENOMEM;
> > +				goto err_alloc;
> > +			}
> > +			memcpy(inst->work_bufs[i].va, tmp_va, wb[i].size);
> > +			wb[i].iova = inst->work_bufs[i].dma_addr;
> > +		} else if (i == VENC_H264_VPU_WORK_BUF_SKIP_FRAME) {
> > +			inst->work_bufs[i].size = wb[i].size;
> > +			inst->work_bufs[i].va = vpu_mapping_dm_addr(
> > +				inst->dev, wb[i].vpua);
> > +			inst->work_bufs[i].dma_addr = 0;
> > +			wb[i].iova = inst->work_bufs[i].dma_addr;
> > +		} else if (i == VENC_H264_VPU_WORK_BUF_SRC_LUMA ||
> > +			   i == VENC_H264_VPU_WORK_BUF_SRC_CHROMA ||
> > +			   i == VENC_H264_VPU_WORK_BUF_SRC_CHROMA_CB ||
> > +			   i == VENC_H264_VPU_WORK_BUF_SRC_CHROMA_CR) {
> > +			inst->work_bufs[i].size = wb[i].size;
> > +			inst->work_bufs[i].dma_addr = 0;
> > +			inst->work_bufs[i].va = NULL;
> > +			wb[i].iova = inst->work_bufs[i].dma_addr;
> > +		} else {
> > +			inst->work_bufs[i].size = wb[i].size;
> > +			if (mtk_vcodec_mem_alloc(inst->ctx,
> > +						 &inst->work_bufs[i])) {
> > +				mtk_vcodec_err(inst,
> > +					       "cannot allocate buf %d", i);
> > +				ret = -ENOMEM;
> > +				goto err_alloc;
> > +			}
> > +			wb[i].iova = inst->work_bufs[i].dma_addr;
> > +		}
> > +		mtk_vcodec_debug(inst, "buf[%d] va=0x%p iova=0x%p size=0x%lx",
> > +				 i, inst->work_bufs[i].va,
> > +				 (void *)inst->work_bufs[i].dma_addr,
> > +				 inst->work_bufs[i].size);
> > +	}
> > +
> > +	if (enc_param->input_fourcc == VENC_YUV_FORMAT_NV12 ||
> > +	    enc_param->input_fourcc == VENC_YUV_FORMAT_NV21) {
> > +		enc_param->sizeimage[0] =
> > +			inst->work_bufs[VENC_H264_VPU_WORK_BUF_SRC_LUMA].size;
> > +		enc_param->sizeimage[1] =
> > +			inst->work_bufs[VENC_H264_VPU_WORK_BUF_SRC_CHROMA].size;
> > +		enc_param->sizeimage[2] = 0;
> > +	} else {
> > +		enc_param->sizeimage[0] =
> > +			inst->work_bufs[VENC_H264_VPU_WORK_BUF_SRC_LUMA].size;
> > +		enc_param->sizeimage[1] =
> > +			inst->work_bufs[VENC_H264_VPU_WORK_BUF_SRC_CHROMA_CB].size;
> > +		enc_param->sizeimage[2] =
> > +			inst->work_bufs[VENC_H264_VPU_WORK_BUF_SRC_CHROMA_CR].size;
> > +	}
> > +
> > +	/* the pps_buf is used by AP side only */
> > +	inst->pps_buf.size = 128;
> > +	if (mtk_vcodec_mem_alloc(inst->ctx,
> > +				 &inst->pps_buf)) {
> > +		mtk_vcodec_err(inst, "cannot allocate pps_buf");
> > +		ret = -ENOMEM;
> > +		goto err_alloc;
> > +	}
> > +	mtk_vcodec_debug_leave(inst);
> > +
> > +	return ret;
> > +
> > +err_alloc:
> > +	h264_enc_free_work_buf(inst);
> > +
> > +	return ret;
> > +}
> > +
> > +static unsigned int h264_enc_wait_venc_done(struct venc_h264_inst *inst)
> > +{
> > +	unsigned int irq_status = 0;
> > +	struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)inst->ctx;
> > +
> > +	mtk_vcodec_wait_for_done_ctx(ctx, MTK_INST_IRQ_RECEIVED,
> > +				     WAIT_INTR_TIMEOUT, true);
> > +	irq_status = ctx->irq_status;
> > +	mtk_vcodec_debug(inst, "irq_status %x <-", irq_status);
> > +
> > +	return irq_status;
> > +}
> > +
> > +static int h264_encode_sps(struct venc_h264_inst *inst,
> > +			   struct mtk_vcodec_mem *bs_buf,
> > +			   unsigned int *bs_size)
> > +{
> > +	int ret = 0;
> > +	unsigned int irq_status;
> > +
> > +	mtk_vcodec_debug_enter(inst);
> > +
> > +	ret = h264_enc_vpu_encode(inst, H264_BS_MODE_SPS, NULL,
> > +				  bs_buf, bs_size);
> > +	if (ret)
> > +		return ret;
> > +
> > +	irq_status = h264_enc_wait_venc_done(inst);
> > +	if (irq_status != H264_IRQ_STATUS_ENC_SPS_INT) {
> > +		mtk_vcodec_err(inst, "expect irq status %d",
> > +			       H264_IRQ_STATUS_ENC_SPS_INT);
> > +		return -EINVAL;
> > +	}
> > +
> > +	*bs_size = h264_read_reg(inst, VENC_PIC_BITSTREAM_BYTE_CNT);
> > +	mtk_vcodec_debug(inst, "bs size %d <-", *bs_size);
> > +
> > +	return ret;
> > +}
> > +
> > +static int h264_encode_pps(struct venc_h264_inst *inst,
> > +			   struct mtk_vcodec_mem *bs_buf,
> > +			   unsigned int *bs_size)
> > +{
> > +	int ret = 0;
> > +	unsigned int irq_status;
> > +
> > +	mtk_vcodec_debug_enter(inst);
> > +
> > +	ret = h264_enc_vpu_encode(inst, H264_BS_MODE_PPS, NULL,
> > +				  bs_buf, bs_size);
> > +	if (ret)
> > +		return ret;
> > +
> > +	irq_status = h264_enc_wait_venc_done(inst);
> > +	if (irq_status != H264_IRQ_STATUS_ENC_PPS_INT) {
> > +		mtk_vcodec_err(inst, "expect irq status %d",
> > +			       H264_IRQ_STATUS_ENC_PPS_INT);
> > +		return -EINVAL;
> > +	}
> > +
> > +	*bs_size = h264_read_reg(inst, VENC_PIC_BITSTREAM_BYTE_CNT);
> > +	mtk_vcodec_debug(inst, "bs size %d <-", *bs_size);
> > +
> > +	return ret;
> > +}
> > +
> > +static int h264_encode_frame(struct venc_h264_inst *inst,
> > +			     struct venc_frm_buf *frm_buf,
> > +			     struct mtk_vcodec_mem *bs_buf,
> > +			     unsigned int *bs_size)
> > +{
> > +	int ret = 0;
> > +	unsigned int irq_status;
> > +
> > +	mtk_vcodec_debug_enter(inst);
> > +
> > +	ret = h264_enc_vpu_encode(inst, H264_BS_MODE_FRAME, frm_buf,
> > +				  bs_buf, bs_size);
> > +	if (ret)
> > +		return ret;
> > +
> > +	/*
> > +	 * skip frame case: The skip frame buffer is composed by vpu side only,
> > +	 * it does not trigger the hw, so skip the wait interrupt operation.
> > +	 */
> > +	if (!inst->vpu_inst.wait_int) {
> > +		++inst->frm_cnt;
> > +		return ret;
> > +	}
> > +
> > +	irq_status = h264_enc_wait_venc_done(inst);
> > +	if (irq_status != H264_IRQ_STATUS_ENC_FRM_INT) {
> > +		mtk_vcodec_err(inst, "irq_status=%d failed", irq_status);
> > +		return -EINVAL;
> > +	}
> > +
> > +	*bs_size = h264_read_reg(inst,
> > +				 VENC_PIC_BITSTREAM_BYTE_CNT);
> > +	++inst->frm_cnt;
> > +	mtk_vcodec_debug(inst, "frm %d bs size %d key_frm %d <-",
> > +			 inst->frm_cnt,
> > +			 *bs_size, inst->is_key_frm);
> > +
> > +	return ret;
> > +}
> > +
> > +static void h264_encode_filler(struct venc_h264_inst *inst, void *buf,
> > +			       int size)
> > +{
> > +	unsigned char *p = buf;
> > +
> > +	*p++ = 0x0;
> > +	*p++ = 0x0;
> > +	*p++ = 0x0;
> > +	*p++ = 0x1;
> > +	*p++ = 0xc;
> > +	size -= 5;
> > +	while (size) {
> > +		*p++ = 0xff;
> > +		size -= 1;
> > +	}
> > +}
> > +
> > +static int h264_enc_init(struct mtk_vcodec_ctx *ctx, unsigned long *handle)
> > +{
> > +	int ret = 0;
> > +	struct venc_h264_inst *inst;
> > +
> > +	inst = kzalloc(sizeof(*inst), GFP_KERNEL);
> > +	if (!inst)
> > +		return -ENOMEM;
> > +
> > +	inst->ctx = ctx;
> > +	inst->dev = mtk_vcodec_get_plat_dev(ctx);
> > +	inst->hw_base = mtk_vcodec_get_reg_addr(inst->ctx, VENC_SYS);
> > +
> > +	mtk_vcodec_debug_enter(inst);
> > +	ret = h264_enc_vpu_init(inst);
> > +	if (ret)
> > +		kfree(inst);
> > +	else
> > +		(*handle) = (unsigned long)inst;
> > +
> > +	mtk_vcodec_debug_leave(inst);
> > +
> > +	return ret;
> > +}
> > +
> > +static int h264_enc_encode(unsigned long handle,
> > +			   enum venc_start_opt opt,
> > +			   struct venc_frm_buf *frm_buf,
> > +			   struct mtk_vcodec_mem *bs_buf,
> > +			   struct venc_done_result *result)
> > +{
> > +	int ret = 0;
> > +	struct venc_h264_inst *inst = (struct venc_h264_inst *)handle;
> > +	struct mtk_vcodec_ctx *ctx = inst->ctx;
> > +
> > +	mtk_vcodec_debug(inst, "opt %d ->", opt);
> > +
> > +	enable_irq(ctx->dev->enc_irq);
> > +
> > +	switch (opt) {
> > +	case VENC_START_OPT_ENCODE_SEQUENCE_HEADER: {
> > +		unsigned int bs_size_sps;
> > +		unsigned int bs_size_pps;
> > +
> > +		memset(bs_buf->va, 0x38, 20);
> > +		ret = h264_encode_sps(inst, bs_buf, &bs_size_sps);
> > +		if (ret)
> > +			goto encode_err;
> > +
> > +		memset(inst->pps_buf.va, 0x49, 20);
> > +		ret = h264_encode_pps(inst, &inst->pps_buf, &bs_size_pps);
> > +		if (ret)
> > +			goto encode_err;
> > +
> > +		memcpy(bs_buf->va + bs_size_sps,
> > +		       inst->pps_buf.va,
> > +		       bs_size_pps);
> > +		result->bs_size = bs_size_sps + bs_size_pps;
> > +		result->is_key_frm = false;
> > +	}
> > +	break;
> 
> Wrong indentation for break, put it before the '}'.
> 

I will change this in next version. Thanks.

> > +
> > +	case VENC_START_OPT_ENCODE_FRAME:
> > +		if (inst->prepend_hdr) {
> 
> Invert the condition to make it more readable:
> 
> 		if (!inst->prepend_hdr) {
> 			ret = h264_encode_frame(inst, frm_buf, bs_buf,
> 						&result->bs_size);
> 			if (ret)
> 				goto encode_err;
> 			result->is_key_frm = inst->is_key_frm;
> 			break;
> 		}
> 
> Now you can shift the remainder one tab to the left and join lines that
> are now broken up (probably because they exceed 80 chars).
> 

I will change this in next version. Thanks.

> > +			int hdr_sz;
> > +			int hdr_sz_ext;
> > +			int bs_alignment = 128;
> > +			int filler_sz = 0;
> > +			struct mtk_vcodec_mem tmp_bs_buf;
> > +			unsigned int bs_size_sps;
> > +			unsigned int bs_size_pps;
> > +			unsigned int bs_size_frm;
> > +
> > +			mtk_vcodec_debug(inst,
> > +					 "h264_encode_frame prepend SPS/PPS");
> > +			ret = h264_encode_sps(inst, bs_buf, &bs_size_sps);
> > +			if (ret)
> > +				goto encode_err;
> > +
> > +			ret = h264_encode_pps(inst, &inst->pps_buf,
> > +					      &bs_size_pps);
> > +			if (ret)
> > +				goto encode_err;
> > +
> > +			memcpy(bs_buf->va + bs_size_sps,
> > +			       inst->pps_buf.va,
> > +			       bs_size_pps);
> > +
> > +			hdr_sz = bs_size_sps + bs_size_pps;
> > +			hdr_sz_ext = (hdr_sz & (bs_alignment - 1));
> > +			if (hdr_sz_ext) {
> > +				filler_sz = bs_alignment - hdr_sz_ext;
> > +				if (hdr_sz_ext + 5 > bs_alignment)
> > +					filler_sz += bs_alignment;
> > +				h264_encode_filler(
> > +					inst, bs_buf->va + hdr_sz,
> > +					filler_sz);
> > +			}
> > +
> > +			tmp_bs_buf.va = bs_buf->va + hdr_sz +
> > +				filler_sz;
> > +			tmp_bs_buf.dma_addr = bs_buf->dma_addr + hdr_sz +
> > +				filler_sz;
> > +			tmp_bs_buf.size = bs_buf->size -
> > +				(hdr_sz + filler_sz);
> > +
> > +			ret = h264_encode_frame(inst, frm_buf, &tmp_bs_buf,
> > +						&bs_size_frm);
> > +			if (ret)
> > +				goto encode_err;
> > +
> > +			result->bs_size = hdr_sz + filler_sz + bs_size_frm;
> > +			mtk_vcodec_debug(inst,
> > +					 "hdr %d filler %d frame %d bs %d",
> > +					 hdr_sz, filler_sz, bs_size_frm,
> > +					 result->bs_size);
> > +
> > +			inst->prepend_hdr = 0;
> > +		} else {
> > +			ret = h264_encode_frame(inst, frm_buf, bs_buf,
> > +						&result->bs_size);
> > +			if (ret)
> > +				goto encode_err;
> > +		}
> > +		result->is_key_frm = inst->is_key_frm;
> > +		break;
> > +
> > +	default:
> > +		mtk_vcodec_err(inst, "venc_start_opt %d not supported", opt);
> > +		ret = -EINVAL;
> > +		break;
> > +	}
> > +
> > +encode_err:
> > +	if (ret)
> > +		result->msg = VENC_MESSAGE_ERR;
> > +	else
> > +		result->msg = VENC_MESSAGE_OK;
> > +
> > +	disable_irq(ctx->dev->enc_irq);
> > +	mtk_vcodec_debug(inst, "opt %d <-", opt);
> > +
> > +	return ret;
> > +}
> > +
> > +static int h264_enc_set_param(unsigned long handle,
> > +			      enum venc_set_param_type type, void *in)
> > +{
> > +	int ret = 0;
> > +	struct venc_h264_inst *inst = (struct venc_h264_inst *)handle;
> > +	struct venc_enc_prm *enc_prm;
> > +
> > +	mtk_vcodec_debug(inst, "->type=%d", type);
> > +
> > +	switch (type) {
> > +	case VENC_SET_PARAM_ENC:
> > +		enc_prm = in;
> > +		ret = h264_enc_vpu_set_param(inst, type, enc_prm);
> > +		if (ret)
> > +			break;
> > +		if (inst->work_buf_allocated == 1) {
> > +			h264_enc_free_work_buf(inst);
> > +			inst->work_buf_allocated = 0;
> > +		}
> > +		if (inst->work_buf_allocated == 0) {
> > +			ret = h264_enc_alloc_work_buf(inst, enc_prm);
> > +			if (ret)
> > +				break;
> > +			inst->work_buf_allocated = 1;
> > +		}
> > +		break;
> > +
> > +	case VENC_SET_PARAM_FORCE_INTRA:
> > +		ret = h264_enc_vpu_set_param(inst, type, 0);
> > +		break;
> > +
> > +	case VENC_SET_PARAM_ADJUST_BITRATE:
> > +		enc_prm = in;
> > +		ret = h264_enc_vpu_set_param(inst, type, &enc_prm->bitrate);
> > +		break;
> > +
> > +	case VENC_SET_PARAM_ADJUST_FRAMERATE:
> > +		enc_prm = in;
> > +		ret = h264_enc_vpu_set_param(inst, type, &enc_prm->frm_rate);
> > +		break;
> > +
> > +	case VENC_SET_PARAM_I_FRAME_INTERVAL:
> > +		ret = h264_enc_vpu_set_param(inst, type, in);
> > +		break;
> > +
> > +	case VENC_SET_PARAM_SKIP_FRAME:
> > +		ret = h264_enc_vpu_set_param(inst, type, 0);
> > +		break;
> > +
> > +	case VENC_SET_PARAM_PREPEND_HEADER:
> > +		inst->prepend_hdr = 1;
> > +		mtk_vcodec_debug(inst, "set prepend header mode");
> > +		break;
> > +
> > +	default:
> > +		mtk_vcodec_err(inst, "type %d not supported", type);
> > +		ret = -EINVAL;
> > +		break;
> > +	}
> > +
> > +	mtk_vcodec_debug_leave(inst);
> > +
> > +	return ret;
> > +}
> > +
> > +static int h264_enc_deinit(unsigned long handle)
> > +{
> > +	int ret = 0;
> > +	struct venc_h264_inst *inst = (struct venc_h264_inst *)handle;
> > +
> > +	mtk_vcodec_debug_enter(inst);
> > +
> > +	ret = h264_enc_vpu_deinit(inst);
> > +
> > +	if (inst->work_buf_allocated)
> > +		h264_enc_free_work_buf(inst);
> > +
> > +	mtk_vcodec_debug_leave(inst);
> > +	kfree(inst);
> > +
> > +	return ret;
> > +}
> > +
> > +static struct venc_common_if venc_h264_if = {
> > +	h264_enc_init,
> > +	h264_enc_encode,
> > +	h264_enc_set_param,
> > +	h264_enc_deinit,
> > +};
> > +
> > +struct venc_common_if *get_h264_enc_comm_if(void)
> > +{
> > +	return &venc_h264_if;
> > +}
> > diff --git a/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.h b/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.h
> > new file mode 100644
> > index 0000000..9ac317a
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.h
> > @@ -0,0 +1,165 @@
> > +/*
> > + * Copyright (c) 2015 MediaTek Inc.
> > + * Author: Jungchang Tsao <jungchang.tsao@mediatek.com>
> > + *         Daniel Hsiao <daniel.hsiao@mediatek.com>
> > + *         PoChun Lin <pochun.lin@mediatek.com>
> > + *
> > + * This program is free software; you can redistribute it and/or
> > + * modify
> > + * it under the terms of the GNU General Public License version 2 as
> > + * published by the Free Software Foundation.
> > + *
> > + * This program is distributed in the hope that it will be useful,
> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > + * GNU General Public License for more details.
> > + */
> > +
> > +#ifndef _VENC_H264_IF_H_
> > +#define _VENC_H264_IF_H_
> > +
> > +#include "venc_drv_base.h"
> > +
> > +/**
> > + * enum venc_h264_vpu_work_buf - h264 encoder buffer index
> > + */
> > +enum venc_h264_vpu_work_buf {
> > +	VENC_H264_VPU_WORK_BUF_RC_INFO,
> > +	VENC_H264_VPU_WORK_BUF_RC_CODE,
> > +	VENC_H264_VPU_WORK_BUF_REC_LUMA,
> > +	VENC_H264_VPU_WORK_BUF_REC_CHROMA,
> > +	VENC_H264_VPU_WORK_BUF_REF_LUMA,
> > +	VENC_H264_VPU_WORK_BUF_REF_CHROMA,
> > +	VENC_H264_VPU_WORK_BUF_MV_INFO_1,
> > +	VENC_H264_VPU_WORK_BUF_MV_INFO_2,
> > +	VENC_H264_VPU_WORK_BUF_SKIP_FRAME,
> > +	VENC_H264_VPU_WORK_BUF_SRC_LUMA,
> > +	VENC_H264_VPU_WORK_BUF_SRC_CHROMA,
> > +	VENC_H264_VPU_WORK_BUF_SRC_CHROMA_CB,
> > +	VENC_H264_VPU_WORK_BUF_SRC_CHROMA_CR,
> > +	VENC_H264_VPU_WORK_BUF_MAX,
> > +};
> > +
> > +/**
> > + * enum venc_h264_bs_mode - for bs_mode argument in h264_enc_vpu_encode
> > + */
> > +enum venc_h264_bs_mode {
> > +	H264_BS_MODE_SPS,
> > +	H264_BS_MODE_PPS,
> > +	H264_BS_MODE_FRAME,
> > +};
> > +
> > +/*
> > + * struct venc_h264_vpu_config - Structure for h264 encoder configuration
> > + * @input_fourcc: input fourcc
> > + * @bitrate: target bitrate (in bps)
> > + * @pic_w: picture width
> > + * @pic_h: picture height
> > + * @buf_w: buffer width
> > + * @buf_h: buffer height
> > + * @intra_period: intra frame period
> > + * @framerate: frame rate
> > + * @profile: as specified in standard
> > + * @level: as specified in standard
> > + * @wfd: WFD mode 1:on, 0:off
> > + */
> > +struct venc_h264_vpu_config {
> > +	u32 input_fourcc;
> > +	u32 bitrate;
> > +	u32 pic_w;
> > +	u32 pic_h;
> > +	u32 buf_w;
> > +	u32 buf_h;
> > +	u32 intra_period;
> > +	u32 framerate;
> > +	u32 profile;
> > +	u32 level;
> > +	u32 wfd;
> > +};
> > +
> > +/*
> > + * struct venc_h264_vpu_buf - Structure for buffer information
> > + * @align: buffer alignment (in bytes)
> > + * @iova: IO virtual address
> > + * @vpua: VPU side memory addr which is used by RC_CODE
> > + * @size: buffer size (in bytes)
> > + */
> > +struct venc_h264_vpu_buf {
> > +	u32 align;
> > +	u32 iova;
> > +	u32 vpua;
> > +	u32 size;
> > +};
> > +
> > +/*
> > + * struct venc_h264_vpu_drv - Structure for VPU driver control and info share
> > + * This structure is allocated in VPU side and shared to AP side.
> > + * @config: h264 encoder configuration
> > + * @work_bufs: working buffer information in VPU side
> > + * The work_bufs here is for storing the 'size' info shared to AP side.
> > + * The similar item in struct venc_h264_inst is for memory allocation
> > + * in AP side. The AP driver will copy the 'size' from here to the one in
> > + * struct mtk_vcodec_mem, then invoke mtk_vcodec_mem_alloc to allocate
> > + * the buffer. After that, bypass the 'dma_addr' to the 'iova' field here for
> > + * register setting in VPU side.
> > + */
> > +struct venc_h264_vpu_drv {
> > +	struct venc_h264_vpu_config config;
> > +	struct venc_h264_vpu_buf work_bufs[VENC_H264_VPU_WORK_BUF_MAX];
> > +};
> > +
> > +/*
> > + * struct venc_h264_vpu_inst - h264 encoder VPU driver instance
> > + * @wq_hd: wait queue used for vpu cmd trigger then wait vpu interrupt done
> > + * @signaled: flag used for checking vpu interrupt done
> > + * @failure: flag to show vpu cmd succeeds or not
> > + * @state: enum venc_ipi_msg_enc_state
> > + * @bs_size: bitstream size for skip frame case usage
> > + * @wait_int: flag to wait interrupt done (0: for skip frame case, 1: normal
> > + *	      case)
> > + * @id: VPU instance id
> > + * @drv: driver structure allocated by VPU side and shared to AP side for
> > + *	 control and info share
> > + */
> > +struct venc_h264_vpu_inst {
> > +	wait_queue_head_t wq_hd;
> > +	int signaled;
> > +	int failure;
> > +	int state;
> > +	int bs_size;
> > +	int wait_int;
> > +	unsigned int id;
> > +	struct venc_h264_vpu_drv *drv;
> > +};
> > +
> > +/*
> > + * struct venc_h264_inst - h264 encoder AP driver instance
> > + * @hw_base: h264 encoder hardware register base
> > + * @work_bufs: working buffer
> > + * @pps_buf: buffer to store the pps bitstream
> > + * @work_buf_allocated: working buffer allocated flag
> > + * @frm_cnt: encoded frame count
> > + * @prepend_hdr: when the v4l2 layer send VENC_SET_PARAM_PREPEND_HEADER cmd
> > + *  through h264_enc_set_param interface, it will set this flag and prepend the
> > + *  sps/pps in h264_enc_encode function.
> > + * @is_key_frm: key frame flag
> > + * @vpu_inst: VPU instance to exchange information between AP and VPU
> > + * @ctx: context for v4l2 layer integration
> > + * @dev: device for v4l2 layer integration
> > + */
> > +struct venc_h264_inst {
> > +	void __iomem *hw_base;
> > +	struct mtk_vcodec_mem work_bufs[VENC_H264_VPU_WORK_BUF_MAX];
> > +	struct mtk_vcodec_mem pps_buf;
> > +	bool work_buf_allocated;
> > +	unsigned int frm_cnt;
> > +	unsigned int prepend_hdr;
> > +	unsigned int is_key_frm;
> > +	struct venc_h264_vpu_inst vpu_inst;
> > +	void *ctx;
> > +	struct platform_device *dev;
> > +};
> > +
> > +struct venc_common_if *get_h264_enc_comm_if(void);
> > +
> > +#endif
> > diff --git a/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.c b/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.c
> > new file mode 100644
> > index 0000000..450be45
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.c
> > @@ -0,0 +1,309 @@
> > +/*
> > + * Copyright (c) 2015 MediaTek Inc.
> > + * Author: Jungchang Tsao <jungchang.tsao@mediatek.com>
> > + *         Daniel Hsiao <daniel.hsiao@mediatek.com>
> > + *         PoChun Lin <pochun.lin@mediatek.com>
> > + *
> > + * This program is free software; you can redistribute it and/or
> > + * modify
> > + * it under the terms of the GNU General Public License version 2 as
> > + * published by the Free Software Foundation.
> > + *
> > + * This program is distributed in the hope that it will be useful,
> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > + * GNU General Public License for more details.
> > + */
> > +
> > +#include "mtk_vpu.h"
> > +
> > +#include "venc_h264_if.h"
> > +#include "venc_h264_vpu.h"
> > +#include "venc_ipi_msg.h"
> > +
> > +static unsigned int h264_get_profile(unsigned int profile)
> > +{
> > +	/* (Baseline=66, Main=77, High=100) */
> > +	switch (profile) {
> > +	case V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE:
> > +		return 66;
> > +	case V4L2_MPEG_VIDEO_H264_PROFILE_MAIN:
> > +		return 77;
> > +	case V4L2_MPEG_VIDEO_H264_PROFILE_HIGH:
> > +		return 100;
> > +	default:
> > +		return 100;
> > +	}
> > +}
> > +
> > +static unsigned int h264_get_level(unsigned int level)
> > +{
> > +	/* (UpTo4.1(HighProfile)) */
> > +	switch (level) {
> > +	case V4L2_MPEG_VIDEO_H264_LEVEL_1_0:
> > +		return 10;
> > +	case V4L2_MPEG_VIDEO_H264_LEVEL_1_1:
> > +		return 11;
> > +	case V4L2_MPEG_VIDEO_H264_LEVEL_1_2:
> > +		return 12;
> > +	case V4L2_MPEG_VIDEO_H264_LEVEL_1_3:
> > +		return 13;
> > +	case V4L2_MPEG_VIDEO_H264_LEVEL_2_0:
> > +		return 20;
> > +	case V4L2_MPEG_VIDEO_H264_LEVEL_2_1:
> > +		return 21;
> > +	case V4L2_MPEG_VIDEO_H264_LEVEL_2_2:
> > +		return 22;
> > +	case V4L2_MPEG_VIDEO_H264_LEVEL_3_0:
> > +		return 30;
> > +	case V4L2_MPEG_VIDEO_H264_LEVEL_3_1:
> > +		return 31;
> > +	case V4L2_MPEG_VIDEO_H264_LEVEL_3_2:
> > +		return 32;
> > +	case V4L2_MPEG_VIDEO_H264_LEVEL_4_0:
> > +		return 40;
> > +	case V4L2_MPEG_VIDEO_H264_LEVEL_4_1:
> > +		return 41;
> > +	default:
> > +		return 31;
> > +	}
> > +}
> > +
> > +static void handle_h264_enc_init_msg(struct venc_h264_inst *inst, void *data)
> > +{
> > +	struct venc_vpu_ipi_msg_init *msg = data;
> > +
> > +	inst->vpu_inst.id = msg->inst_id;
> > +	inst->vpu_inst.drv = (struct venc_h264_vpu_drv *)vpu_mapping_dm_addr(
> > +		inst->dev, msg->inst_id);
> > +}
> > +
> > +static void handle_h264_enc_encode_msg(struct venc_h264_inst *inst, void *data)
> > +{
> > +	struct venc_vpu_ipi_msg_enc *msg = data;
> > +
> > +	inst->vpu_inst.state = msg->state;
> > +	inst->vpu_inst.bs_size = msg->bs_size;
> > +	inst->is_key_frm = msg->key_frame;
> > +}
> > +
> > +static void h264_enc_vpu_ipi_handler(void *data, unsigned int len, void *priv)
> > +{
> > +	struct venc_vpu_ipi_msg_common *msg = data;
> > +	struct venc_h264_inst *inst = (struct venc_h264_inst *)msg->venc_inst;
> > +
> > +	mtk_vcodec_debug(inst, "msg_id %x inst %p status %d",
> > +			 msg->msg_id, inst, msg->status);
> > +
> > +	switch (msg->msg_id) {
> > +	case VPU_IPIMSG_H264_ENC_INIT_DONE:
> > +		handle_h264_enc_init_msg(inst, data);
> > +		break;
> > +	case VPU_IPIMSG_H264_ENC_SET_PARAM_DONE:
> > +		break;
> > +	case VPU_IPIMSG_H264_ENC_ENCODE_DONE:
> > +		handle_h264_enc_encode_msg(inst, data);
> > +		break;
> > +	case VPU_IPIMSG_H264_ENC_DEINIT_DONE:
> > +		break;
> > +	default:
> > +		mtk_vcodec_err(inst, "unknown msg id %x", msg->msg_id);
> > +		break;
> > +	}
> > +
> > +	inst->vpu_inst.signaled = 1;
> > +	inst->vpu_inst.failure = (msg->status != VENC_IPI_MSG_STATUS_OK);
> > +
> > +	mtk_vcodec_debug_leave(inst);
> > +}
> > +
> > +static int h264_enc_vpu_send_msg(struct venc_h264_inst *inst, void *msg,
> > +				 int len)
> > +{
> > +	int status;
> > +
> > +	mtk_vcodec_debug_enter(inst);
> > +	status = vpu_ipi_send(inst->dev, IPI_VENC_H264, msg, len);
> > +	if (status) {
> > +		mtk_vcodec_err(inst, "vpu_ipi_send msg %x len %d fail %d",
> > +			       *(unsigned int *)msg, len, status);
> > +		return -EINVAL;
> > +	}
> > +	mtk_vcodec_debug_leave(inst);
> > +
> > +	return 0;
> > +}
> > +
> > +int h264_enc_vpu_init(struct venc_h264_inst *inst)
> > +{
> > +	int status;
> > +	struct venc_ap_ipi_msg_init out;
> > +
> > +	mtk_vcodec_debug_enter(inst);
> > +
> > +	init_waitqueue_head(&inst->vpu_inst.wq_hd);
> > +	inst->vpu_inst.signaled = 0;
> > +	inst->vpu_inst.failure = 0;
> > +
> > +	status = vpu_ipi_register(inst->dev, IPI_VENC_H264,
> > +				  h264_enc_vpu_ipi_handler,
> > +				  "h264_enc", NULL);
> > +	if (status) {
> > +		mtk_vcodec_err(inst, "vpu_ipi_register fail %d", status);
> > +		return -EINVAL;
> > +	}
> > +
> > +	out.msg_id = AP_IPIMSG_H264_ENC_INIT;
> > +	out.venc_inst = (unsigned long)inst;
> > +	if (h264_enc_vpu_send_msg(inst, &out, sizeof(out)) ||
> > +	    inst->vpu_inst.failure) {
> > +		mtk_vcodec_err(inst, "AP_IPIMSG_H264_ENC_INIT failed");
> > +		return -EINVAL;
> > +	}
> > +
> > +	mtk_vcodec_debug_leave(inst);
> > +
> > +	return 0;
> > +}
> > +
> > +int h264_enc_vpu_set_param(struct venc_h264_inst *inst, unsigned int id,
> > +			   void *param)
> > +{
> > +	struct venc_ap_ipi_msg_set_param out;
> > +
> > +	mtk_vcodec_debug(inst, "id %d ->", id);
> > +
> > +	out.msg_id = AP_IPIMSG_H264_ENC_SET_PARAM;
> > +	out.inst_id = inst->vpu_inst.id;
> > +	out.param_id = id;
> > +	switch (id) {
> > +	case VENC_SET_PARAM_ENC: {
> > +		struct venc_enc_prm *enc_param = (struct venc_enc_prm *)param;
> > +
> > +		inst->vpu_inst.drv->config.input_fourcc =
> > +			enc_param->input_fourcc;
> > +		inst->vpu_inst.drv->config.bitrate = enc_param->bitrate;
> > +		inst->vpu_inst.drv->config.pic_w = enc_param->width;
> > +		inst->vpu_inst.drv->config.pic_h = enc_param->height;
> > +		inst->vpu_inst.drv->config.buf_w = enc_param->buf_width;
> > +		inst->vpu_inst.drv->config.buf_h = enc_param->buf_height;
> > +		inst->vpu_inst.drv->config.intra_period =
> > +			enc_param->intra_period;
> > +		inst->vpu_inst.drv->config.framerate = enc_param->frm_rate;
> > +		inst->vpu_inst.drv->config.profile =
> > +			h264_get_profile(enc_param->h264_profile);
> > +		inst->vpu_inst.drv->config.level =
> > +			h264_get_level(enc_param->h264_level);
> > +		inst->vpu_inst.drv->config.wfd = 0;
> > +		out.data_item = 0;
> > +		break;
> > +	}
> > +	case VENC_SET_PARAM_FORCE_INTRA:
> > +		out.data_item = 0;
> > +		break;
> > +	case VENC_SET_PARAM_ADJUST_BITRATE:
> > +		out.data_item = 1;
> > +		out.data[0] = *(unsigned int *)param;
> > +		break;
> > +	case VENC_SET_PARAM_ADJUST_FRAMERATE:
> > +		out.data_item = 1;
> > +		out.data[0] = *(unsigned int *)param;
> > +		break;
> > +	case VENC_SET_PARAM_I_FRAME_INTERVAL:
> > +		out.data_item = 1;
> > +		out.data[0] = *(unsigned int *)param;
> > +		break;
> > +	case VENC_SET_PARAM_SKIP_FRAME:
> > +		out.data_item = 0;
> > +		break;
> > +	}
> > +	if (h264_enc_vpu_send_msg(inst, &out, sizeof(out)) ||
> > +	    inst->vpu_inst.failure) {
> > +		mtk_vcodec_err(inst,
> > +			       "AP_IPIMSG_H264_ENC_SET_PARAM %d fail", id);
> > +		return -EINVAL;
> > +	}
> > +
> > +	mtk_vcodec_debug(inst, "id %d <-", id);
> > +
> > +	return 0;
> > +}
> > +
> > +int h264_enc_vpu_encode(struct venc_h264_inst *inst, unsigned int bs_mode,
> > +			struct venc_frm_buf *frm_buf,
> > +			struct mtk_vcodec_mem *bs_buf,
> > +			unsigned int *bs_size)
> > +{
> > +	struct venc_ap_ipi_msg_enc out;
> > +
> > +	mtk_vcodec_debug(inst, "bs_mode %d ->", bs_mode);
> > +
> > +	out.msg_id = AP_IPIMSG_H264_ENC_ENCODE;
> > +	out.inst_id = inst->vpu_inst.id;
> > +	out.bs_mode = bs_mode;
> > +	if (frm_buf) {
> > +		if ((frm_buf->fb_addr.dma_addr % 16 == 0) &&
> > +		    (frm_buf->fb_addr1.dma_addr % 16 == 0) &&
> > +		    (frm_buf->fb_addr2.dma_addr % 16 == 0)) {
> > +			out.input_addr[0] = frm_buf->fb_addr.dma_addr;
> > +			out.input_addr[1] = frm_buf->fb_addr1.dma_addr;
> > +			out.input_addr[2] = frm_buf->fb_addr2.dma_addr;
> > +		} else {
> > +			mtk_vcodec_err(inst, "dma_addr not align to 16");
> > +			return -EINVAL;
> > +		}
> > +	} else {
> > +		out.input_addr[0] = 0;
> > +		out.input_addr[1] = 0;
> > +		out.input_addr[2] = 0;
> > +	}
> > +	if (bs_buf) {
> > +		out.bs_addr = bs_buf->dma_addr;
> > +		out.bs_size = bs_buf->size;
> > +	} else {
> > +		out.bs_addr = 0;
> > +		out.bs_size = 0;
> > +	}
> > +	if (h264_enc_vpu_send_msg(inst, &out, sizeof(out)) ||
> > +	    inst->vpu_inst.failure) {
> > +		mtk_vcodec_err(inst, "AP_IPIMSG_H264_ENC_ENCODE %d fail",
> > +			       bs_mode);
> > +		return -EINVAL;
> > +	}
> > +
> > +	mtk_vcodec_debug(inst, "state %d size %d key_frm %d",
> > +			 inst->vpu_inst.state, inst->vpu_inst.bs_size,
> > +			 inst->is_key_frm);
> > +
> > +	inst->vpu_inst.wait_int = 1;
> > +	if (inst->vpu_inst.state == VEN_IPI_MSG_ENC_STATE_SKIP) {
> > +		*bs_size = inst->vpu_inst.bs_size;
> > +		memcpy(bs_buf->va,
> > +		       inst->work_bufs[VENC_H264_VPU_WORK_BUF_SKIP_FRAME].va,
> > +		       *bs_size);
> > +		inst->vpu_inst.wait_int = 0;
> > +	}
> > +
> > +	mtk_vcodec_debug(inst, "bs_mode %d <-", bs_mode);
> > +
> > +	return 0;
> > +}
> > +
> > +int h264_enc_vpu_deinit(struct venc_h264_inst *inst)
> > +{
> > +	struct venc_ap_ipi_msg_deinit out;
> > +
> > +	mtk_vcodec_debug_enter(inst);
> > +
> > +	out.msg_id = AP_IPIMSG_H264_ENC_DEINIT;
> > +	out.inst_id = inst->vpu_inst.id;
> > +	if (h264_enc_vpu_send_msg(inst, &out, sizeof(out)) ||
> > +	    inst->vpu_inst.failure) {
> > +		mtk_vcodec_err(inst, "AP_IPIMSG_H264_ENC_DEINIT fail");
> > +		return -EINVAL;
> > +	}
> > +
> > +	mtk_vcodec_debug_leave(inst);
> > +
> > +	return 0;
> > +}
> > diff --git a/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.h b/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.h
> > new file mode 100644
> > index 0000000..deccc6f
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.h
> > @@ -0,0 +1,30 @@
> > +/*
> > + * Copyright (c) 2015 MediaTek Inc.
> > + * Author: Jungchang Tsao <jungchang.tsao@mediatek.com>
> > + *         Daniel Hsiao <daniel.hsiao@mediatek.com>
> > + *         PoChun Lin <pochun.lin@mediatek.com>
> > + *
> > + * This program is free software; you can redistribute it and/or
> > + * modify
> > + * it under the terms of the GNU General Public License version 2 as
> > + * published by the Free Software Foundation.
> > + *
> > + * This program is distributed in the hope that it will be useful,
> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > + * GNU General Public License for more details.
> > + */
> > +
> > +#ifndef _VENC_H264_VPU_H_
> > +#define _VENC_H264_VPU_H_
> > +
> > +int h264_enc_vpu_init(struct venc_h264_inst *inst);
> > +int h264_enc_vpu_set_param(struct venc_h264_inst *inst, unsigned int id,
> > +			   void *param);
> > +int h264_enc_vpu_encode(struct venc_h264_inst *inst, unsigned int bs_mode,
> > +			struct venc_frm_buf *frm_buf,
> > +			struct mtk_vcodec_mem *bs_buf,
> > +			unsigned int *bs_size);
> > +int h264_enc_vpu_deinit(struct venc_h264_inst *inst);
> > +
> > +#endif
> > diff --git a/drivers/media/platform/mtk-vcodec/venc_drv_if.c b/drivers/media/platform/mtk-vcodec/venc_drv_if.c
> > index d293f2c..28ef4a7 100644
> > --- a/drivers/media/platform/mtk-vcodec/venc_drv_if.c
> > +++ b/drivers/media/platform/mtk-vcodec/venc_drv_if.c
> > @@ -25,6 +25,7 @@
> >  
> >  #include "venc_drv_base.h"
> >  #include "vp8_enc/venc_vp8_if.h"
> > +#include "h264_enc/venc_h264_if.h"
> >  
> >  int venc_if_create(struct mtk_vcodec_ctx *ctx, unsigned int fourcc)
> >  {
> > @@ -38,6 +39,8 @@ int venc_if_create(struct mtk_vcodec_ctx *ctx, unsigned int fourcc)
> >                  ctx->enc_if = get_vp8_enc_comm_if();
> >                  break;
> >  	case V4L2_PIX_FMT_H264:
> > +	        ctx->enc_if = get_h264_enc_comm_if();
> > +	        break;
> >  	default:
> >  		return -EINVAL;
> >  	}
> > 
> 
> Regards,
> 
> 	Hans

Best Regards
PoChun

^ permalink raw reply	[flat|nested] 102+ messages in thread

* [PATCH v4 7/8] [media] vcodec: mediatek: Add Mediatek H264 Video Encoder Driver
@ 2016-02-16 11:57                   ` pochun lin
  0 siblings, 0 replies; 102+ messages in thread
From: pochun lin @ 2016-02-16 11:57 UTC (permalink / raw)
  To: linux-arm-kernel

Hi Hans,

On Mon, 2016-02-15 at 19:33 +0800, Hans Verkuil wrote:
> On 02/04/2016 12:35 PM, Tiffany Lin wrote:
> > Add h264 encoder driver for MT8173
> > 
> > Signed-off-by: Daniel Hsiao <daniel.hsiao@mediatek.com>
> > Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
> > ---
> >  drivers/media/platform/mtk-vcodec/Makefile         |    3 +-
> >  .../media/platform/mtk-vcodec/h264_enc/Makefile    |    6 +
> >  .../platform/mtk-vcodec/h264_enc/venc_h264_if.c    |  540 ++++++++++++++++++++
> >  .../platform/mtk-vcodec/h264_enc/venc_h264_if.h    |  165 ++++++
> >  .../platform/mtk-vcodec/h264_enc/venc_h264_vpu.c   |  309 +++++++++++
> >  .../platform/mtk-vcodec/h264_enc/venc_h264_vpu.h   |   30 ++
> >  drivers/media/platform/mtk-vcodec/venc_drv_if.c    |    3 +
> >  7 files changed, 1055 insertions(+), 1 deletion(-)
> >  create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/Makefile
> >  create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.c
> >  create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.h
> >  create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.c
> >  create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.h
> > 
> > diff --git a/drivers/media/platform/mtk-vcodec/Makefile b/drivers/media/platform/mtk-vcodec/Makefile
> > index f4ef502..f47dfc7 100644
> > --- a/drivers/media/platform/mtk-vcodec/Makefile
> > +++ b/drivers/media/platform/mtk-vcodec/Makefile
> > @@ -5,6 +5,7 @@ obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += mtk_vcodec_intr.o \
> >  				       mtk_vcodec_enc_pm.o \
> >  				       venc_drv_if.o
> >  
> > -obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += vp8_enc/
> > +
> > +obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += vp8_enc/ h264_enc/
> >  
> >  ccflags-y += -I$(srctree)/drivers/media/platform/mtk-vpu
> > diff --git a/drivers/media/platform/mtk-vcodec/h264_enc/Makefile b/drivers/media/platform/mtk-vcodec/h264_enc/Makefile
> > new file mode 100644
> > index 0000000..765b45f
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/h264_enc/Makefile
> > @@ -0,0 +1,6 @@
> > +obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += venc_h264_if.o venc_h264_vpu.o
> > +
> > +ccflags-y += \
> > +    -I$(srctree)/drivers/media/platform/mtk-vcodec/ \
> > +    -I$(srctree)/drivers/media/platform/mtk-vcodec/h264_enc \
> > +    -I$(srctree)/drivers/media/platform/mtk-vpu
> > diff --git a/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.c b/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.c
> > new file mode 100644
> > index 0000000..c35fb26
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.c
> > @@ -0,0 +1,540 @@
> > +/*
> > + * Copyright (c) 2015 MediaTek Inc.
> > + * Author: Jungchang Tsao <jungchang.tsao@mediatek.com>
> > + *         Daniel Hsiao <daniel.hsiao@mediatek.com>
> > + *         PoChun Lin <pochun.lin@mediatek.com>
> > + *
> > + * This program is free software; you can redistribute it and/or
> > + * modify
> > + * it under the terms of the GNU General Public License version 2 as
> > + * published by the Free Software Foundation.
> > + *
> > + * This program is distributed in the hope that it will be useful,
> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > + * GNU General Public License for more details.
> > + */
> > +
> > +#include <linux/interrupt.h>
> > +#include <linux/kernel.h>
> > +#include <linux/slab.h>
> > +
> > +#include "mtk_vcodec_drv.h"
> > +#include "mtk_vcodec_util.h"
> > +#include "mtk_vcodec_intr.h"
> > +#include "mtk_vcodec_enc.h"
> > +#include "mtk_vcodec_pm.h"
> > +#include "mtk_vpu.h"
> > +
> > +#include "venc_h264_if.h"
> > +#include "venc_h264_vpu.h"
> > +
> > +#define VENC_PIC_BITSTREAM_BYTE_CNT 0x0098
> > +
> > +enum venc_h264_irq_status {
> > +	H264_IRQ_STATUS_ENC_SPS_INT = (1 << 0),
> > +	H264_IRQ_STATUS_ENC_PPS_INT = (1 << 1),
> > +	H264_IRQ_STATUS_ENC_FRM_INT = (1 << 2),
> > +};
> > +
> > +static inline void h264_write_reg(struct venc_h264_inst *inst, u32 addr,
> > +				  u32 val)
> > +{
> > +	writel(val, inst->hw_base + addr);
> > +}
> > +
> > +static inline u32 h264_read_reg(struct venc_h264_inst *inst, u32 addr)
> > +{
> > +	return readl(inst->hw_base + addr);
> > +}
> > +
> > +static void h264_enc_free_work_buf(struct venc_h264_inst *inst)
> > +{
> > +	int i;
> > +
> > +	mtk_vcodec_debug_enter(inst);
> > +
> > +	/* Except the SKIP_FRAME buffers,
> > +	 * other buffers need to be freed by AP.
> > +	 */
> > +	for (i = 0; i < VENC_H264_VPU_WORK_BUF_MAX; i++) {
> > +		if (i != VENC_H264_VPU_WORK_BUF_SKIP_FRAME)
> > +			if (inst->work_bufs[i].va != NULL)
> > +				mtk_vcodec_mem_free(inst->ctx,
> > +						    &inst->work_bufs[i]);
> > +	}
> > +
> > +	if (inst->pps_buf.va != NULL)
> > +		mtk_vcodec_mem_free(inst->ctx, &inst->pps_buf);
> > +
> > +	mtk_vcodec_debug_leave(inst);
> > +}
> > +
> > +static int h264_enc_alloc_work_buf(struct venc_h264_inst *inst, void *param)
> > +{
> > +	int i;
> > +	int ret = 0;
> > +	struct venc_h264_vpu_buf *wb = inst->vpu_inst.drv->work_bufs;
> > +	struct venc_enc_prm *enc_param = param;
> > +
> > +	mtk_vcodec_debug_enter(inst);
> > +
> > +	for (i = 0; i < VENC_H264_VPU_WORK_BUF_MAX; i++) {
> > +		/*
> > +		 * This 'wb' structure is set by VPU side and shared to AP for
> > +		 * buffer allocation and IO virtual addr mapping. For most of
> > +		 * the buffers, AP will allocate the buffer according to 'size'
> > +		 * field and store the IO virtual addr in 'iova' field. There
> > +		 * are two exceptions:
> > +		 * (1) RC_CODE buffer, it's pre-allocated in the VPU side, and
> > +		 * save the VPU addr in the 'vpua' field. The AP will translate
> > +		 * the VPU addr to the corresponding IO virtual addr and store
> > +		 * in 'iova' field for reg setting in VPU side.
> > +		 * (2) SKIP_FRAME buffer, it's pre-allocated in the VPU side,
> > +		 * and save the VPU addr in the 'vpua' field. The AP will
> > +		 * translate the VPU addr to the corresponding AP side virtual
> > +		 * address and do some memcpy access to move to bitstream buffer
> > +		 * assigned by v4l2 layer.
> > +		 */
> > +		if (i == VENC_H264_VPU_WORK_BUF_RC_CODE) {
> > +			void *tmp_va;
> > +
> > +			tmp_va = vpu_mapping_dm_addr(inst->dev, wb[i].vpua);
> > +			inst->work_bufs[i].size = wb[i].size;
> > +			if (mtk_vcodec_mem_alloc(inst->ctx,
> > +						 &inst->work_bufs[i])) {
> > +				mtk_vcodec_err(inst,
> > +					       "cannot allocate buf %d", i);
> > +				ret = -ENOMEM;
> > +				goto err_alloc;
> > +			}
> > +			memcpy(inst->work_bufs[i].va, tmp_va, wb[i].size);
> > +			wb[i].iova = inst->work_bufs[i].dma_addr;
> > +		} else if (i == VENC_H264_VPU_WORK_BUF_SKIP_FRAME) {
> > +			inst->work_bufs[i].size = wb[i].size;
> > +			inst->work_bufs[i].va = vpu_mapping_dm_addr(
> > +				inst->dev, wb[i].vpua);
> > +			inst->work_bufs[i].dma_addr = 0;
> > +			wb[i].iova = inst->work_bufs[i].dma_addr;
> > +		} else if (i == VENC_H264_VPU_WORK_BUF_SRC_LUMA ||
> > +			   i == VENC_H264_VPU_WORK_BUF_SRC_CHROMA ||
> > +			   i == VENC_H264_VPU_WORK_BUF_SRC_CHROMA_CB ||
> > +			   i == VENC_H264_VPU_WORK_BUF_SRC_CHROMA_CR) {
> > +			inst->work_bufs[i].size = wb[i].size;
> > +			inst->work_bufs[i].dma_addr = 0;
> > +			inst->work_bufs[i].va = NULL;
> > +			wb[i].iova = inst->work_bufs[i].dma_addr;
> > +		} else {
> > +			inst->work_bufs[i].size = wb[i].size;
> > +			if (mtk_vcodec_mem_alloc(inst->ctx,
> > +						 &inst->work_bufs[i])) {
> > +				mtk_vcodec_err(inst,
> > +					       "cannot allocate buf %d", i);
> > +				ret = -ENOMEM;
> > +				goto err_alloc;
> > +			}
> > +			wb[i].iova = inst->work_bufs[i].dma_addr;
> > +		}
> > +		mtk_vcodec_debug(inst, "buf[%d] va=0x%p iova=0x%p size=0x%lx",
> > +				 i, inst->work_bufs[i].va,
> > +				 (void *)inst->work_bufs[i].dma_addr,
> > +				 inst->work_bufs[i].size);
> > +	}
> > +
> > +	if (enc_param->input_fourcc == VENC_YUV_FORMAT_NV12 ||
> > +	    enc_param->input_fourcc == VENC_YUV_FORMAT_NV21) {
> > +		enc_param->sizeimage[0] =
> > +			inst->work_bufs[VENC_H264_VPU_WORK_BUF_SRC_LUMA].size;
> > +		enc_param->sizeimage[1] =
> > +			inst->work_bufs[VENC_H264_VPU_WORK_BUF_SRC_CHROMA].size;
> > +		enc_param->sizeimage[2] = 0;
> > +	} else {
> > +		enc_param->sizeimage[0] =
> > +			inst->work_bufs[VENC_H264_VPU_WORK_BUF_SRC_LUMA].size;
> > +		enc_param->sizeimage[1] =
> > +			inst->work_bufs[VENC_H264_VPU_WORK_BUF_SRC_CHROMA_CB].size;
> > +		enc_param->sizeimage[2] =
> > +			inst->work_bufs[VENC_H264_VPU_WORK_BUF_SRC_CHROMA_CR].size;
> > +	}
> > +
> > +	/* the pps_buf is used by AP side only */
> > +	inst->pps_buf.size = 128;
> > +	if (mtk_vcodec_mem_alloc(inst->ctx,
> > +				 &inst->pps_buf)) {
> > +		mtk_vcodec_err(inst, "cannot allocate pps_buf");
> > +		ret = -ENOMEM;
> > +		goto err_alloc;
> > +	}
> > +	mtk_vcodec_debug_leave(inst);
> > +
> > +	return ret;
> > +
> > +err_alloc:
> > +	h264_enc_free_work_buf(inst);
> > +
> > +	return ret;
> > +}
> > +
> > +static unsigned int h264_enc_wait_venc_done(struct venc_h264_inst *inst)
> > +{
> > +	unsigned int irq_status = 0;
> > +	struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)inst->ctx;
> > +
> > +	mtk_vcodec_wait_for_done_ctx(ctx, MTK_INST_IRQ_RECEIVED,
> > +				     WAIT_INTR_TIMEOUT, true);
> > +	irq_status = ctx->irq_status;
> > +	mtk_vcodec_debug(inst, "irq_status %x <-", irq_status);
> > +
> > +	return irq_status;
> > +}
> > +
> > +static int h264_encode_sps(struct venc_h264_inst *inst,
> > +			   struct mtk_vcodec_mem *bs_buf,
> > +			   unsigned int *bs_size)
> > +{
> > +	int ret = 0;
> > +	unsigned int irq_status;
> > +
> > +	mtk_vcodec_debug_enter(inst);
> > +
> > +	ret = h264_enc_vpu_encode(inst, H264_BS_MODE_SPS, NULL,
> > +				  bs_buf, bs_size);
> > +	if (ret)
> > +		return ret;
> > +
> > +	irq_status = h264_enc_wait_venc_done(inst);
> > +	if (irq_status != H264_IRQ_STATUS_ENC_SPS_INT) {
> > +		mtk_vcodec_err(inst, "expect irq status %d",
> > +			       H264_IRQ_STATUS_ENC_SPS_INT);
> > +		return -EINVAL;
> > +	}
> > +
> > +	*bs_size = h264_read_reg(inst, VENC_PIC_BITSTREAM_BYTE_CNT);
> > +	mtk_vcodec_debug(inst, "bs size %d <-", *bs_size);
> > +
> > +	return ret;
> > +}
> > +
> > +static int h264_encode_pps(struct venc_h264_inst *inst,
> > +			   struct mtk_vcodec_mem *bs_buf,
> > +			   unsigned int *bs_size)
> > +{
> > +	int ret = 0;
> > +	unsigned int irq_status;
> > +
> > +	mtk_vcodec_debug_enter(inst);
> > +
> > +	ret = h264_enc_vpu_encode(inst, H264_BS_MODE_PPS, NULL,
> > +				  bs_buf, bs_size);
> > +	if (ret)
> > +		return ret;
> > +
> > +	irq_status = h264_enc_wait_venc_done(inst);
> > +	if (irq_status != H264_IRQ_STATUS_ENC_PPS_INT) {
> > +		mtk_vcodec_err(inst, "expect irq status %d",
> > +			       H264_IRQ_STATUS_ENC_PPS_INT);
> > +		return -EINVAL;
> > +	}
> > +
> > +	*bs_size = h264_read_reg(inst, VENC_PIC_BITSTREAM_BYTE_CNT);
> > +	mtk_vcodec_debug(inst, "bs size %d <-", *bs_size);
> > +
> > +	return ret;
> > +}
> > +
> > +static int h264_encode_frame(struct venc_h264_inst *inst,
> > +			     struct venc_frm_buf *frm_buf,
> > +			     struct mtk_vcodec_mem *bs_buf,
> > +			     unsigned int *bs_size)
> > +{
> > +	int ret = 0;
> > +	unsigned int irq_status;
> > +
> > +	mtk_vcodec_debug_enter(inst);
> > +
> > +	ret = h264_enc_vpu_encode(inst, H264_BS_MODE_FRAME, frm_buf,
> > +				  bs_buf, bs_size);
> > +	if (ret)
> > +		return ret;
> > +
> > +	/*
> > +	 * skip frame case: The skip frame buffer is composed by vpu side only,
> > +	 * it does not trigger the hw, so skip the wait interrupt operation.
> > +	 */
> > +	if (!inst->vpu_inst.wait_int) {
> > +		++inst->frm_cnt;
> > +		return ret;
> > +	}
> > +
> > +	irq_status = h264_enc_wait_venc_done(inst);
> > +	if (irq_status != H264_IRQ_STATUS_ENC_FRM_INT) {
> > +		mtk_vcodec_err(inst, "irq_status=%d failed", irq_status);
> > +		return -EINVAL;
> > +	}
> > +
> > +	*bs_size = h264_read_reg(inst,
> > +				 VENC_PIC_BITSTREAM_BYTE_CNT);
> > +	++inst->frm_cnt;
> > +	mtk_vcodec_debug(inst, "frm %d bs size %d key_frm %d <-",
> > +			 inst->frm_cnt,
> > +			 *bs_size, inst->is_key_frm);
> > +
> > +	return ret;
> > +}
> > +
> > +static void h264_encode_filler(struct venc_h264_inst *inst, void *buf,
> > +			       int size)
> > +{
> > +	unsigned char *p = buf;
> > +
> > +	*p++ = 0x0;
> > +	*p++ = 0x0;
> > +	*p++ = 0x0;
> > +	*p++ = 0x1;
> > +	*p++ = 0xc;
> > +	size -= 5;
> > +	while (size) {
> > +		*p++ = 0xff;
> > +		size -= 1;
> > +	}
> > +}
> > +
> > +static int h264_enc_init(struct mtk_vcodec_ctx *ctx, unsigned long *handle)
> > +{
> > +	int ret = 0;
> > +	struct venc_h264_inst *inst;
> > +
> > +	inst = kzalloc(sizeof(*inst), GFP_KERNEL);
> > +	if (!inst)
> > +		return -ENOMEM;
> > +
> > +	inst->ctx = ctx;
> > +	inst->dev = mtk_vcodec_get_plat_dev(ctx);
> > +	inst->hw_base = mtk_vcodec_get_reg_addr(inst->ctx, VENC_SYS);
> > +
> > +	mtk_vcodec_debug_enter(inst);
> > +	ret = h264_enc_vpu_init(inst);
> > +	if (ret)
> > +		kfree(inst);
> > +	else
> > +		(*handle) = (unsigned long)inst;
> > +
> > +	mtk_vcodec_debug_leave(inst);
> > +
> > +	return ret;
> > +}
> > +
> > +static int h264_enc_encode(unsigned long handle,
> > +			   enum venc_start_opt opt,
> > +			   struct venc_frm_buf *frm_buf,
> > +			   struct mtk_vcodec_mem *bs_buf,
> > +			   struct venc_done_result *result)
> > +{
> > +	int ret = 0;
> > +	struct venc_h264_inst *inst = (struct venc_h264_inst *)handle;
> > +	struct mtk_vcodec_ctx *ctx = inst->ctx;
> > +
> > +	mtk_vcodec_debug(inst, "opt %d ->", opt);
> > +
> > +	enable_irq(ctx->dev->enc_irq);
> > +
> > +	switch (opt) {
> > +	case VENC_START_OPT_ENCODE_SEQUENCE_HEADER: {
> > +		unsigned int bs_size_sps;
> > +		unsigned int bs_size_pps;
> > +
> > +		memset(bs_buf->va, 0x38, 20);
> > +		ret = h264_encode_sps(inst, bs_buf, &bs_size_sps);
> > +		if (ret)
> > +			goto encode_err;
> > +
> > +		memset(inst->pps_buf.va, 0x49, 20);
> > +		ret = h264_encode_pps(inst, &inst->pps_buf, &bs_size_pps);
> > +		if (ret)
> > +			goto encode_err;
> > +
> > +		memcpy(bs_buf->va + bs_size_sps,
> > +		       inst->pps_buf.va,
> > +		       bs_size_pps);
> > +		result->bs_size = bs_size_sps + bs_size_pps;
> > +		result->is_key_frm = false;
> > +	}
> > +	break;
> 
> Wrong indentation for break, put it before the '}'.
> 

I will change this in next version. Thanks.

> > +
> > +	case VENC_START_OPT_ENCODE_FRAME:
> > +		if (inst->prepend_hdr) {
> 
> Invert the condition to make it more readable:
> 
> 		if (!inst->prepend_hdr) {
> 			ret = h264_encode_frame(inst, frm_buf, bs_buf,
> 						&result->bs_size);
> 			if (ret)
> 				goto encode_err;
> 			result->is_key_frm = inst->is_key_frm;
> 			break;
> 		}
> 
> Now you can shift the remainder one tab to the left and join lines that
> are now broken up (probably because they exceed 80 chars).
> 

I will change this in next version. Thanks.

> > +			int hdr_sz;
> > +			int hdr_sz_ext;
> > +			int bs_alignment = 128;
> > +			int filler_sz = 0;
> > +			struct mtk_vcodec_mem tmp_bs_buf;
> > +			unsigned int bs_size_sps;
> > +			unsigned int bs_size_pps;
> > +			unsigned int bs_size_frm;
> > +
> > +			mtk_vcodec_debug(inst,
> > +					 "h264_encode_frame prepend SPS/PPS");
> > +			ret = h264_encode_sps(inst, bs_buf, &bs_size_sps);
> > +			if (ret)
> > +				goto encode_err;
> > +
> > +			ret = h264_encode_pps(inst, &inst->pps_buf,
> > +					      &bs_size_pps);
> > +			if (ret)
> > +				goto encode_err;
> > +
> > +			memcpy(bs_buf->va + bs_size_sps,
> > +			       inst->pps_buf.va,
> > +			       bs_size_pps);
> > +
> > +			hdr_sz = bs_size_sps + bs_size_pps;
> > +			hdr_sz_ext = (hdr_sz & (bs_alignment - 1));
> > +			if (hdr_sz_ext) {
> > +				filler_sz = bs_alignment - hdr_sz_ext;
> > +				if (hdr_sz_ext + 5 > bs_alignment)
> > +					filler_sz += bs_alignment;
> > +				h264_encode_filler(
> > +					inst, bs_buf->va + hdr_sz,
> > +					filler_sz);
> > +			}
> > +
> > +			tmp_bs_buf.va = bs_buf->va + hdr_sz +
> > +				filler_sz;
> > +			tmp_bs_buf.dma_addr = bs_buf->dma_addr + hdr_sz +
> > +				filler_sz;
> > +			tmp_bs_buf.size = bs_buf->size -
> > +				(hdr_sz + filler_sz);
> > +
> > +			ret = h264_encode_frame(inst, frm_buf, &tmp_bs_buf,
> > +						&bs_size_frm);
> > +			if (ret)
> > +				goto encode_err;
> > +
> > +			result->bs_size = hdr_sz + filler_sz + bs_size_frm;
> > +			mtk_vcodec_debug(inst,
> > +					 "hdr %d filler %d frame %d bs %d",
> > +					 hdr_sz, filler_sz, bs_size_frm,
> > +					 result->bs_size);
> > +
> > +			inst->prepend_hdr = 0;
> > +		} else {
> > +			ret = h264_encode_frame(inst, frm_buf, bs_buf,
> > +						&result->bs_size);
> > +			if (ret)
> > +				goto encode_err;
> > +		}
> > +		result->is_key_frm = inst->is_key_frm;
> > +		break;
> > +
> > +	default:
> > +		mtk_vcodec_err(inst, "venc_start_opt %d not supported", opt);
> > +		ret = -EINVAL;
> > +		break;
> > +	}
> > +
> > +encode_err:
> > +	if (ret)
> > +		result->msg = VENC_MESSAGE_ERR;
> > +	else
> > +		result->msg = VENC_MESSAGE_OK;
> > +
> > +	disable_irq(ctx->dev->enc_irq);
> > +	mtk_vcodec_debug(inst, "opt %d <-", opt);
> > +
> > +	return ret;
> > +}
> > +
> > +static int h264_enc_set_param(unsigned long handle,
> > +			      enum venc_set_param_type type, void *in)
> > +{
> > +	int ret = 0;
> > +	struct venc_h264_inst *inst = (struct venc_h264_inst *)handle;
> > +	struct venc_enc_prm *enc_prm;
> > +
> > +	mtk_vcodec_debug(inst, "->type=%d", type);
> > +
> > +	switch (type) {
> > +	case VENC_SET_PARAM_ENC:
> > +		enc_prm = in;
> > +		ret = h264_enc_vpu_set_param(inst, type, enc_prm);
> > +		if (ret)
> > +			break;
> > +		if (inst->work_buf_allocated == 1) {
> > +			h264_enc_free_work_buf(inst);
> > +			inst->work_buf_allocated = 0;
> > +		}
> > +		if (inst->work_buf_allocated == 0) {
> > +			ret = h264_enc_alloc_work_buf(inst, enc_prm);
> > +			if (ret)
> > +				break;
> > +			inst->work_buf_allocated = 1;
> > +		}
> > +		break;
> > +
> > +	case VENC_SET_PARAM_FORCE_INTRA:
> > +		ret = h264_enc_vpu_set_param(inst, type, 0);
> > +		break;
> > +
> > +	case VENC_SET_PARAM_ADJUST_BITRATE:
> > +		enc_prm = in;
> > +		ret = h264_enc_vpu_set_param(inst, type, &enc_prm->bitrate);
> > +		break;
> > +
> > +	case VENC_SET_PARAM_ADJUST_FRAMERATE:
> > +		enc_prm = in;
> > +		ret = h264_enc_vpu_set_param(inst, type, &enc_prm->frm_rate);
> > +		break;
> > +
> > +	case VENC_SET_PARAM_I_FRAME_INTERVAL:
> > +		ret = h264_enc_vpu_set_param(inst, type, in);
> > +		break;
> > +
> > +	case VENC_SET_PARAM_SKIP_FRAME:
> > +		ret = h264_enc_vpu_set_param(inst, type, 0);
> > +		break;
> > +
> > +	case VENC_SET_PARAM_PREPEND_HEADER:
> > +		inst->prepend_hdr = 1;
> > +		mtk_vcodec_debug(inst, "set prepend header mode");
> > +		break;
> > +
> > +	default:
> > +		mtk_vcodec_err(inst, "type %d not supported", type);
> > +		ret = -EINVAL;
> > +		break;
> > +	}
> > +
> > +	mtk_vcodec_debug_leave(inst);
> > +
> > +	return ret;
> > +}
> > +
> > +static int h264_enc_deinit(unsigned long handle)
> > +{
> > +	int ret = 0;
> > +	struct venc_h264_inst *inst = (struct venc_h264_inst *)handle;
> > +
> > +	mtk_vcodec_debug_enter(inst);
> > +
> > +	ret = h264_enc_vpu_deinit(inst);
> > +
> > +	if (inst->work_buf_allocated)
> > +		h264_enc_free_work_buf(inst);
> > +
> > +	mtk_vcodec_debug_leave(inst);
> > +	kfree(inst);
> > +
> > +	return ret;
> > +}
> > +
> > +static struct venc_common_if venc_h264_if = {
> > +	h264_enc_init,
> > +	h264_enc_encode,
> > +	h264_enc_set_param,
> > +	h264_enc_deinit,
> > +};
> > +
> > +struct venc_common_if *get_h264_enc_comm_if(void)
> > +{
> > +	return &venc_h264_if;
> > +}
> > diff --git a/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.h b/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.h
> > new file mode 100644
> > index 0000000..9ac317a
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.h
> > @@ -0,0 +1,165 @@
> > +/*
> > + * Copyright (c) 2015 MediaTek Inc.
> > + * Author: Jungchang Tsao <jungchang.tsao@mediatek.com>
> > + *         Daniel Hsiao <daniel.hsiao@mediatek.com>
> > + *         PoChun Lin <pochun.lin@mediatek.com>
> > + *
> > + * This program is free software; you can redistribute it and/or
> > + * modify
> > + * it under the terms of the GNU General Public License version 2 as
> > + * published by the Free Software Foundation.
> > + *
> > + * This program is distributed in the hope that it will be useful,
> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > + * GNU General Public License for more details.
> > + */
> > +
> > +#ifndef _VENC_H264_IF_H_
> > +#define _VENC_H264_IF_H_
> > +
> > +#include "venc_drv_base.h"
> > +
> > +/**
> > + * enum venc_h264_vpu_work_buf - h264 encoder buffer index
> > + */
> > +enum venc_h264_vpu_work_buf {
> > +	VENC_H264_VPU_WORK_BUF_RC_INFO,
> > +	VENC_H264_VPU_WORK_BUF_RC_CODE,
> > +	VENC_H264_VPU_WORK_BUF_REC_LUMA,
> > +	VENC_H264_VPU_WORK_BUF_REC_CHROMA,
> > +	VENC_H264_VPU_WORK_BUF_REF_LUMA,
> > +	VENC_H264_VPU_WORK_BUF_REF_CHROMA,
> > +	VENC_H264_VPU_WORK_BUF_MV_INFO_1,
> > +	VENC_H264_VPU_WORK_BUF_MV_INFO_2,
> > +	VENC_H264_VPU_WORK_BUF_SKIP_FRAME,
> > +	VENC_H264_VPU_WORK_BUF_SRC_LUMA,
> > +	VENC_H264_VPU_WORK_BUF_SRC_CHROMA,
> > +	VENC_H264_VPU_WORK_BUF_SRC_CHROMA_CB,
> > +	VENC_H264_VPU_WORK_BUF_SRC_CHROMA_CR,
> > +	VENC_H264_VPU_WORK_BUF_MAX,
> > +};
> > +
> > +/**
> > + * enum venc_h264_bs_mode - for bs_mode argument in h264_enc_vpu_encode
> > + */
> > +enum venc_h264_bs_mode {
> > +	H264_BS_MODE_SPS,
> > +	H264_BS_MODE_PPS,
> > +	H264_BS_MODE_FRAME,
> > +};
> > +
> > +/*
> > + * struct venc_h264_vpu_config - Structure for h264 encoder configuration
> > + * @input_fourcc: input fourcc
> > + * @bitrate: target bitrate (in bps)
> > + * @pic_w: picture width
> > + * @pic_h: picture height
> > + * @buf_w: buffer width
> > + * @buf_h: buffer height
> > + * @intra_period: intra frame period
> > + * @framerate: frame rate
> > + * @profile: as specified in standard
> > + * @level: as specified in standard
> > + * @wfd: WFD mode 1:on, 0:off
> > + */
> > +struct venc_h264_vpu_config {
> > +	u32 input_fourcc;
> > +	u32 bitrate;
> > +	u32 pic_w;
> > +	u32 pic_h;
> > +	u32 buf_w;
> > +	u32 buf_h;
> > +	u32 intra_period;
> > +	u32 framerate;
> > +	u32 profile;
> > +	u32 level;
> > +	u32 wfd;
> > +};
> > +
> > +/*
> > + * struct venc_h264_vpu_buf - Structure for buffer information
> > + * @align: buffer alignment (in bytes)
> > + * @iova: IO virtual address
> > + * @vpua: VPU side memory addr which is used by RC_CODE
> > + * @size: buffer size (in bytes)
> > + */
> > +struct venc_h264_vpu_buf {
> > +	u32 align;
> > +	u32 iova;
> > +	u32 vpua;
> > +	u32 size;
> > +};
> > +
> > +/*
> > + * struct venc_h264_vpu_drv - Structure for VPU driver control and info share
> > + * This structure is allocated in VPU side and shared to AP side.
> > + * @config: h264 encoder configuration
> > + * @work_bufs: working buffer information in VPU side
> > + * The work_bufs here is for storing the 'size' info shared to AP side.
> > + * The similar item in struct venc_h264_inst is for memory allocation
> > + * in AP side. The AP driver will copy the 'size' from here to the one in
> > + * struct mtk_vcodec_mem, then invoke mtk_vcodec_mem_alloc to allocate
> > + * the buffer. After that, bypass the 'dma_addr' to the 'iova' field here for
> > + * register setting in VPU side.
> > + */
> > +struct venc_h264_vpu_drv {
> > +	struct venc_h264_vpu_config config;
> > +	struct venc_h264_vpu_buf work_bufs[VENC_H264_VPU_WORK_BUF_MAX];
> > +};
> > +
> > +/*
> > + * struct venc_h264_vpu_inst - h264 encoder VPU driver instance
> > + * @wq_hd: wait queue used for vpu cmd trigger then wait vpu interrupt done
> > + * @signaled: flag used for checking vpu interrupt done
> > + * @failure: flag to show vpu cmd succeeds or not
> > + * @state: enum venc_ipi_msg_enc_state
> > + * @bs_size: bitstream size for skip frame case usage
> > + * @wait_int: flag to wait interrupt done (0: for skip frame case, 1: normal
> > + *	      case)
> > + * @id: VPU instance id
> > + * @drv: driver structure allocated by VPU side and shared to AP side for
> > + *	 control and info share
> > + */
> > +struct venc_h264_vpu_inst {
> > +	wait_queue_head_t wq_hd;
> > +	int signaled;
> > +	int failure;
> > +	int state;
> > +	int bs_size;
> > +	int wait_int;
> > +	unsigned int id;
> > +	struct venc_h264_vpu_drv *drv;
> > +};
> > +
> > +/*
> > + * struct venc_h264_inst - h264 encoder AP driver instance
> > + * @hw_base: h264 encoder hardware register base
> > + * @work_bufs: working buffer
> > + * @pps_buf: buffer to store the pps bitstream
> > + * @work_buf_allocated: working buffer allocated flag
> > + * @frm_cnt: encoded frame count
> > + * @prepend_hdr: when the v4l2 layer send VENC_SET_PARAM_PREPEND_HEADER cmd
> > + *  through h264_enc_set_param interface, it will set this flag and prepend the
> > + *  sps/pps in h264_enc_encode function.
> > + * @is_key_frm: key frame flag
> > + * @vpu_inst: VPU instance to exchange information between AP and VPU
> > + * @ctx: context for v4l2 layer integration
> > + * @dev: device for v4l2 layer integration
> > + */
> > +struct venc_h264_inst {
> > +	void __iomem *hw_base;
> > +	struct mtk_vcodec_mem work_bufs[VENC_H264_VPU_WORK_BUF_MAX];
> > +	struct mtk_vcodec_mem pps_buf;
> > +	bool work_buf_allocated;
> > +	unsigned int frm_cnt;
> > +	unsigned int prepend_hdr;
> > +	unsigned int is_key_frm;
> > +	struct venc_h264_vpu_inst vpu_inst;
> > +	void *ctx;
> > +	struct platform_device *dev;
> > +};
> > +
> > +struct venc_common_if *get_h264_enc_comm_if(void);
> > +
> > +#endif
> > diff --git a/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.c b/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.c
> > new file mode 100644
> > index 0000000..450be45
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.c
> > @@ -0,0 +1,309 @@
> > +/*
> > + * Copyright (c) 2015 MediaTek Inc.
> > + * Author: Jungchang Tsao <jungchang.tsao@mediatek.com>
> > + *         Daniel Hsiao <daniel.hsiao@mediatek.com>
> > + *         PoChun Lin <pochun.lin@mediatek.com>
> > + *
> > + * This program is free software; you can redistribute it and/or
> > + * modify
> > + * it under the terms of the GNU General Public License version 2 as
> > + * published by the Free Software Foundation.
> > + *
> > + * This program is distributed in the hope that it will be useful,
> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > + * GNU General Public License for more details.
> > + */
> > +
> > +#include "mtk_vpu.h"
> > +
> > +#include "venc_h264_if.h"
> > +#include "venc_h264_vpu.h"
> > +#include "venc_ipi_msg.h"
> > +
> > +static unsigned int h264_get_profile(unsigned int profile)
> > +{
> > +	/* (Baseline=66, Main=77, High=100) */
> > +	switch (profile) {
> > +	case V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE:
> > +		return 66;
> > +	case V4L2_MPEG_VIDEO_H264_PROFILE_MAIN:
> > +		return 77;
> > +	case V4L2_MPEG_VIDEO_H264_PROFILE_HIGH:
> > +		return 100;
> > +	default:
> > +		return 100;
> > +	}
> > +}
> > +
> > +static unsigned int h264_get_level(unsigned int level)
> > +{
> > +	/* (UpTo4.1(HighProfile)) */
> > +	switch (level) {
> > +	case V4L2_MPEG_VIDEO_H264_LEVEL_1_0:
> > +		return 10;
> > +	case V4L2_MPEG_VIDEO_H264_LEVEL_1_1:
> > +		return 11;
> > +	case V4L2_MPEG_VIDEO_H264_LEVEL_1_2:
> > +		return 12;
> > +	case V4L2_MPEG_VIDEO_H264_LEVEL_1_3:
> > +		return 13;
> > +	case V4L2_MPEG_VIDEO_H264_LEVEL_2_0:
> > +		return 20;
> > +	case V4L2_MPEG_VIDEO_H264_LEVEL_2_1:
> > +		return 21;
> > +	case V4L2_MPEG_VIDEO_H264_LEVEL_2_2:
> > +		return 22;
> > +	case V4L2_MPEG_VIDEO_H264_LEVEL_3_0:
> > +		return 30;
> > +	case V4L2_MPEG_VIDEO_H264_LEVEL_3_1:
> > +		return 31;
> > +	case V4L2_MPEG_VIDEO_H264_LEVEL_3_2:
> > +		return 32;
> > +	case V4L2_MPEG_VIDEO_H264_LEVEL_4_0:
> > +		return 40;
> > +	case V4L2_MPEG_VIDEO_H264_LEVEL_4_1:
> > +		return 41;
> > +	default:
> > +		return 31;
> > +	}
> > +}
> > +
> > +static void handle_h264_enc_init_msg(struct venc_h264_inst *inst, void *data)
> > +{
> > +	struct venc_vpu_ipi_msg_init *msg = data;
> > +
> > +	inst->vpu_inst.id = msg->inst_id;
> > +	inst->vpu_inst.drv = (struct venc_h264_vpu_drv *)vpu_mapping_dm_addr(
> > +		inst->dev, msg->inst_id);
> > +}
> > +
> > +static void handle_h264_enc_encode_msg(struct venc_h264_inst *inst, void *data)
> > +{
> > +	struct venc_vpu_ipi_msg_enc *msg = data;
> > +
> > +	inst->vpu_inst.state = msg->state;
> > +	inst->vpu_inst.bs_size = msg->bs_size;
> > +	inst->is_key_frm = msg->key_frame;
> > +}
> > +
> > +static void h264_enc_vpu_ipi_handler(void *data, unsigned int len, void *priv)
> > +{
> > +	struct venc_vpu_ipi_msg_common *msg = data;
> > +	struct venc_h264_inst *inst = (struct venc_h264_inst *)msg->venc_inst;
> > +
> > +	mtk_vcodec_debug(inst, "msg_id %x inst %p status %d",
> > +			 msg->msg_id, inst, msg->status);
> > +
> > +	switch (msg->msg_id) {
> > +	case VPU_IPIMSG_H264_ENC_INIT_DONE:
> > +		handle_h264_enc_init_msg(inst, data);
> > +		break;
> > +	case VPU_IPIMSG_H264_ENC_SET_PARAM_DONE:
> > +		break;
> > +	case VPU_IPIMSG_H264_ENC_ENCODE_DONE:
> > +		handle_h264_enc_encode_msg(inst, data);
> > +		break;
> > +	case VPU_IPIMSG_H264_ENC_DEINIT_DONE:
> > +		break;
> > +	default:
> > +		mtk_vcodec_err(inst, "unknown msg id %x", msg->msg_id);
> > +		break;
> > +	}
> > +
> > +	inst->vpu_inst.signaled = 1;
> > +	inst->vpu_inst.failure = (msg->status != VENC_IPI_MSG_STATUS_OK);
> > +
> > +	mtk_vcodec_debug_leave(inst);
> > +}
> > +
> > +static int h264_enc_vpu_send_msg(struct venc_h264_inst *inst, void *msg,
> > +				 int len)
> > +{
> > +	int status;
> > +
> > +	mtk_vcodec_debug_enter(inst);
> > +	status = vpu_ipi_send(inst->dev, IPI_VENC_H264, msg, len);
> > +	if (status) {
> > +		mtk_vcodec_err(inst, "vpu_ipi_send msg %x len %d fail %d",
> > +			       *(unsigned int *)msg, len, status);
> > +		return -EINVAL;
> > +	}
> > +	mtk_vcodec_debug_leave(inst);
> > +
> > +	return 0;
> > +}
> > +
> > +int h264_enc_vpu_init(struct venc_h264_inst *inst)
> > +{
> > +	int status;
> > +	struct venc_ap_ipi_msg_init out;
> > +
> > +	mtk_vcodec_debug_enter(inst);
> > +
> > +	init_waitqueue_head(&inst->vpu_inst.wq_hd);
> > +	inst->vpu_inst.signaled = 0;
> > +	inst->vpu_inst.failure = 0;
> > +
> > +	status = vpu_ipi_register(inst->dev, IPI_VENC_H264,
> > +				  h264_enc_vpu_ipi_handler,
> > +				  "h264_enc", NULL);
> > +	if (status) {
> > +		mtk_vcodec_err(inst, "vpu_ipi_register fail %d", status);
> > +		return -EINVAL;
> > +	}
> > +
> > +	out.msg_id = AP_IPIMSG_H264_ENC_INIT;
> > +	out.venc_inst = (unsigned long)inst;
> > +	if (h264_enc_vpu_send_msg(inst, &out, sizeof(out)) ||
> > +	    inst->vpu_inst.failure) {
> > +		mtk_vcodec_err(inst, "AP_IPIMSG_H264_ENC_INIT failed");
> > +		return -EINVAL;
> > +	}
> > +
> > +	mtk_vcodec_debug_leave(inst);
> > +
> > +	return 0;
> > +}
> > +
> > +int h264_enc_vpu_set_param(struct venc_h264_inst *inst, unsigned int id,
> > +			   void *param)
> > +{
> > +	struct venc_ap_ipi_msg_set_param out;
> > +
> > +	mtk_vcodec_debug(inst, "id %d ->", id);
> > +
> > +	out.msg_id = AP_IPIMSG_H264_ENC_SET_PARAM;
> > +	out.inst_id = inst->vpu_inst.id;
> > +	out.param_id = id;
> > +	switch (id) {
> > +	case VENC_SET_PARAM_ENC: {
> > +		struct venc_enc_prm *enc_param = (struct venc_enc_prm *)param;
> > +
> > +		inst->vpu_inst.drv->config.input_fourcc =
> > +			enc_param->input_fourcc;
> > +		inst->vpu_inst.drv->config.bitrate = enc_param->bitrate;
> > +		inst->vpu_inst.drv->config.pic_w = enc_param->width;
> > +		inst->vpu_inst.drv->config.pic_h = enc_param->height;
> > +		inst->vpu_inst.drv->config.buf_w = enc_param->buf_width;
> > +		inst->vpu_inst.drv->config.buf_h = enc_param->buf_height;
> > +		inst->vpu_inst.drv->config.intra_period =
> > +			enc_param->intra_period;
> > +		inst->vpu_inst.drv->config.framerate = enc_param->frm_rate;
> > +		inst->vpu_inst.drv->config.profile =
> > +			h264_get_profile(enc_param->h264_profile);
> > +		inst->vpu_inst.drv->config.level =
> > +			h264_get_level(enc_param->h264_level);
> > +		inst->vpu_inst.drv->config.wfd = 0;
> > +		out.data_item = 0;
> > +		break;
> > +	}
> > +	case VENC_SET_PARAM_FORCE_INTRA:
> > +		out.data_item = 0;
> > +		break;
> > +	case VENC_SET_PARAM_ADJUST_BITRATE:
> > +		out.data_item = 1;
> > +		out.data[0] = *(unsigned int *)param;
> > +		break;
> > +	case VENC_SET_PARAM_ADJUST_FRAMERATE:
> > +		out.data_item = 1;
> > +		out.data[0] = *(unsigned int *)param;
> > +		break;
> > +	case VENC_SET_PARAM_I_FRAME_INTERVAL:
> > +		out.data_item = 1;
> > +		out.data[0] = *(unsigned int *)param;
> > +		break;
> > +	case VENC_SET_PARAM_SKIP_FRAME:
> > +		out.data_item = 0;
> > +		break;
> > +	}
> > +	if (h264_enc_vpu_send_msg(inst, &out, sizeof(out)) ||
> > +	    inst->vpu_inst.failure) {
> > +		mtk_vcodec_err(inst,
> > +			       "AP_IPIMSG_H264_ENC_SET_PARAM %d fail", id);
> > +		return -EINVAL;
> > +	}
> > +
> > +	mtk_vcodec_debug(inst, "id %d <-", id);
> > +
> > +	return 0;
> > +}
> > +
> > +int h264_enc_vpu_encode(struct venc_h264_inst *inst, unsigned int bs_mode,
> > +			struct venc_frm_buf *frm_buf,
> > +			struct mtk_vcodec_mem *bs_buf,
> > +			unsigned int *bs_size)
> > +{
> > +	struct venc_ap_ipi_msg_enc out;
> > +
> > +	mtk_vcodec_debug(inst, "bs_mode %d ->", bs_mode);
> > +
> > +	out.msg_id = AP_IPIMSG_H264_ENC_ENCODE;
> > +	out.inst_id = inst->vpu_inst.id;
> > +	out.bs_mode = bs_mode;
> > +	if (frm_buf) {
> > +		if ((frm_buf->fb_addr.dma_addr % 16 == 0) &&
> > +		    (frm_buf->fb_addr1.dma_addr % 16 == 0) &&
> > +		    (frm_buf->fb_addr2.dma_addr % 16 == 0)) {
> > +			out.input_addr[0] = frm_buf->fb_addr.dma_addr;
> > +			out.input_addr[1] = frm_buf->fb_addr1.dma_addr;
> > +			out.input_addr[2] = frm_buf->fb_addr2.dma_addr;
> > +		} else {
> > +			mtk_vcodec_err(inst, "dma_addr not align to 16");
> > +			return -EINVAL;
> > +		}
> > +	} else {
> > +		out.input_addr[0] = 0;
> > +		out.input_addr[1] = 0;
> > +		out.input_addr[2] = 0;
> > +	}
> > +	if (bs_buf) {
> > +		out.bs_addr = bs_buf->dma_addr;
> > +		out.bs_size = bs_buf->size;
> > +	} else {
> > +		out.bs_addr = 0;
> > +		out.bs_size = 0;
> > +	}
> > +	if (h264_enc_vpu_send_msg(inst, &out, sizeof(out)) ||
> > +	    inst->vpu_inst.failure) {
> > +		mtk_vcodec_err(inst, "AP_IPIMSG_H264_ENC_ENCODE %d fail",
> > +			       bs_mode);
> > +		return -EINVAL;
> > +	}
> > +
> > +	mtk_vcodec_debug(inst, "state %d size %d key_frm %d",
> > +			 inst->vpu_inst.state, inst->vpu_inst.bs_size,
> > +			 inst->is_key_frm);
> > +
> > +	inst->vpu_inst.wait_int = 1;
> > +	if (inst->vpu_inst.state == VEN_IPI_MSG_ENC_STATE_SKIP) {
> > +		*bs_size = inst->vpu_inst.bs_size;
> > +		memcpy(bs_buf->va,
> > +		       inst->work_bufs[VENC_H264_VPU_WORK_BUF_SKIP_FRAME].va,
> > +		       *bs_size);
> > +		inst->vpu_inst.wait_int = 0;
> > +	}
> > +
> > +	mtk_vcodec_debug(inst, "bs_mode %d <-", bs_mode);
> > +
> > +	return 0;
> > +}
> > +
> > +int h264_enc_vpu_deinit(struct venc_h264_inst *inst)
> > +{
> > +	struct venc_ap_ipi_msg_deinit out;
> > +
> > +	mtk_vcodec_debug_enter(inst);
> > +
> > +	out.msg_id = AP_IPIMSG_H264_ENC_DEINIT;
> > +	out.inst_id = inst->vpu_inst.id;
> > +	if (h264_enc_vpu_send_msg(inst, &out, sizeof(out)) ||
> > +	    inst->vpu_inst.failure) {
> > +		mtk_vcodec_err(inst, "AP_IPIMSG_H264_ENC_DEINIT fail");
> > +		return -EINVAL;
> > +	}
> > +
> > +	mtk_vcodec_debug_leave(inst);
> > +
> > +	return 0;
> > +}
> > diff --git a/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.h b/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.h
> > new file mode 100644
> > index 0000000..deccc6f
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.h
> > @@ -0,0 +1,30 @@
> > +/*
> > + * Copyright (c) 2015 MediaTek Inc.
> > + * Author: Jungchang Tsao <jungchang.tsao@mediatek.com>
> > + *         Daniel Hsiao <daniel.hsiao@mediatek.com>
> > + *         PoChun Lin <pochun.lin@mediatek.com>
> > + *
> > + * This program is free software; you can redistribute it and/or
> > + * modify
> > + * it under the terms of the GNU General Public License version 2 as
> > + * published by the Free Software Foundation.
> > + *
> > + * This program is distributed in the hope that it will be useful,
> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > + * GNU General Public License for more details.
> > + */
> > +
> > +#ifndef _VENC_H264_VPU_H_
> > +#define _VENC_H264_VPU_H_
> > +
> > +int h264_enc_vpu_init(struct venc_h264_inst *inst);
> > +int h264_enc_vpu_set_param(struct venc_h264_inst *inst, unsigned int id,
> > +			   void *param);
> > +int h264_enc_vpu_encode(struct venc_h264_inst *inst, unsigned int bs_mode,
> > +			struct venc_frm_buf *frm_buf,
> > +			struct mtk_vcodec_mem *bs_buf,
> > +			unsigned int *bs_size);
> > +int h264_enc_vpu_deinit(struct venc_h264_inst *inst);
> > +
> > +#endif
> > diff --git a/drivers/media/platform/mtk-vcodec/venc_drv_if.c b/drivers/media/platform/mtk-vcodec/venc_drv_if.c
> > index d293f2c..28ef4a7 100644
> > --- a/drivers/media/platform/mtk-vcodec/venc_drv_if.c
> > +++ b/drivers/media/platform/mtk-vcodec/venc_drv_if.c
> > @@ -25,6 +25,7 @@
> >  
> >  #include "venc_drv_base.h"
> >  #include "vp8_enc/venc_vp8_if.h"
> > +#include "h264_enc/venc_h264_if.h"
> >  
> >  int venc_if_create(struct mtk_vcodec_ctx *ctx, unsigned int fourcc)
> >  {
> > @@ -38,6 +39,8 @@ int venc_if_create(struct mtk_vcodec_ctx *ctx, unsigned int fourcc)
> >                  ctx->enc_if = get_vp8_enc_comm_if();
> >                  break;
> >  	case V4L2_PIX_FMT_H264:
> > +	        ctx->enc_if = get_h264_enc_comm_if();
> > +	        break;
> >  	default:
> >  		return -EINVAL;
> >  	}
> > 
> 
> Regards,
> 
> 	Hans

Best Regards
PoChun

^ permalink raw reply	[flat|nested] 102+ messages in thread

* Re: [PATCH v4 5/8] [Media] vcodec: mediatek: Add Mediatek V4L2 Video Encoder Driver
@ 2016-02-16 13:20                   ` tiffany lin
  0 siblings, 0 replies; 102+ messages in thread
From: tiffany lin @ 2016-02-16 13:20 UTC (permalink / raw)
  To: Hans Verkuil
  Cc: Hans Verkuil, daniel.thompson, Rob Herring,
	Mauro Carvalho Chehab, Matthias Brugger, Daniel Kurtz,
	Pawel Osciak, Eddie Huang, Yingjoe Chen, devicetree,
	linux-kernel, linux-arm-kernel, linux-media, linux-mediatek,
	PoChun.Lin, Andrew-CT Chen

On Tue, 2016-02-16 at 08:44 +0100, Hans Verkuil wrote:
> On 02/16/2016 07:37 AM, tiffany lin wrote:
> 
> <snip>
> 
> >>> +static int vidioc_venc_s_parm(struct file *file, void *priv,
> >>> +			      struct v4l2_streamparm *a)
> >>> +{
> >>> +	struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
> >>> +
> >>> +	if (a->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
> >>> +		ctx->enc_params.framerate_num =
> >>> +			a->parm.output.timeperframe.denominator;
> >>> +		ctx->enc_params.framerate_denom =
> >>> +			a->parm.output.timeperframe.numerator;
> >>> +		ctx->param_change |= MTK_ENCODE_PARAM_FRAMERATE;
> >>> +	} else {
> >>> +		return -EINVAL;
> >>> +	}
> >>
> >> I'd invert the test:
> >>
> >> 	if (a->type != V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE)
> >> 		return -EINVAL;
> >>
> >> and now you can just set ctx->enc_params.
> >>
> > We will fix this in next version.
> > 
> > 
> >>> +	return 0;
> >>> +}
> >>
> >> And if there is an s_parm, then there should be a g_parm as well!
> >>
> > Now our driver does not support g_parm, our use cases do not use g_parm
> > too. 
> > Do we need to add g_parm at this moment? Or we could add it when we need
> > g_parm?
> 
> No, you need it. You can see why if you look at the v4l2-compliance output:
> 
>                  test VIDIOC_G/S_PARM: OK (Not Supported)
> 
> Why does it think it is unsupported? Because (just like most applications) it
> tries to call G_PARM first, and if that succeeds it tries to call S_PARM with
> the value it got from G_PARM. Thus ensuring the application doesn't change the
> driver state. So you can have a 'get' ioctl without the 'set' ioctl, but if
> there is a 'set' ioctl there must always be a 'get' ioctl.
> 
Got it. We will add g_parm in next version.

> <snip>
> 
> >>> +static int vidioc_venc_g_s_selection(struct file *file, void *priv,
> >>> +                                struct v4l2_selection *s)
> >>
> >> Why support s_selection if you can only return the current width and height?
> >> And why support g_selection if you can't change the selection?
> >>
> >> In other words, why implement this at all?
> >>
> >> Unless I am missing something here, I would just drop this.
> >>
> > Now our driver do not support these capabilities, but userspace app will
> > check whether g/s_crop are implemented when using encoder.
> > Because g/s_crop are deprecated as you mentioned in previous v2 review
> > comments. We change to use g_s_selection.
> > We will check if we could add this capability.
> 
> It's true that you should use g/s_selection instead of g/s_crop, but only if
> there is actually something to select. As long as you don't offer this capability,
> just drop this for now.
> 
> When you add the capability later you can just add the g/s_selection functions.
> 
> Getting selection right can be tricky. I wouldn't mind if this is done later in a
> separate patch.
> 
Got it. We will add the capability later.


> > 
> >>> +{
> >>> +	struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
> >>> +	struct mtk_q_data *q_data;
> >>> +
> >>> +	if (V4L2_TYPE_IS_OUTPUT(s->type)) {
> >>> +		if (s->target !=  V4L2_SEL_TGT_COMPOSE)
> >>> +			return -EINVAL;
> >>> +	} else {
> >>> +		if (s->target != V4L2_SEL_TGT_CROP)
> >>> +			return -EINVAL;
> >>> +	}
> >>> +
> >>> +	if (s->r.left || s->r.top)
> >>> +		return -EINVAL;
> >>> +
> >>> +	q_data = mtk_venc_get_q_data(ctx, s->type);
> >>> +	if (!q_data)
> >>> +		return -EINVAL;
> >>> +
> >>> +	s->r.width = q_data->width;
> >>> +	s->r.height = q_data->height;
> >>> +
> >>> +	return 0;
> >>> +}
> >>> +
> >>> +
> >>> +static int vidioc_venc_qbuf(struct file *file, void *priv,
> >>> +			    struct v4l2_buffer *buf)
> >>> +{
> >>> +
> >>> +	struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
> >>> +
> >>> +	if (ctx->state == MTK_STATE_ABORT) {
> >>> +		mtk_v4l2_err("[%d] Call on QBUF after unrecoverable error\n", ctx->idx);
> >>> +		return -EIO;
> >>> +	}
> >>> +
> >>> +	return v4l2_m2m_qbuf(file, ctx->m2m_ctx, buf);
> >>> +}
> >>> +
> >>> +static int vidioc_venc_dqbuf(struct file *file, void *priv,
> >>> +			     struct v4l2_buffer *buf)
> >>> +{
> >>> +	struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
> >>> +	if (ctx->state == MTK_STATE_ABORT) {
> >>> +		mtk_v4l2_err("[%d] Call on QBUF after unrecoverable error\n", ctx->idx);
> >>> +		return -EIO;
> >>> +	}
> >>> +
> >>> +	return v4l2_m2m_dqbuf(file, ctx->m2m_ctx, buf);
> >>> +}
> >>> +
> >>> +
> >>> +const struct v4l2_ioctl_ops mtk_venc_ioctl_ops = {
> >>> +	.vidioc_streamon		= v4l2_m2m_ioctl_streamon,
> >>> +	.vidioc_streamoff		= v4l2_m2m_ioctl_streamoff,
> >>> +
> >>> +	.vidioc_reqbufs			= v4l2_m2m_ioctl_reqbufs,
> >>> +	.vidioc_querybuf		= v4l2_m2m_ioctl_querybuf,
> >>> +	.vidioc_qbuf			= vidioc_venc_qbuf,
> >>> +	.vidioc_dqbuf			= vidioc_venc_dqbuf,
> >>> +
> >>> +	.vidioc_querycap		= vidioc_venc_querycap,
> >>> +	.vidioc_enum_fmt_vid_cap_mplane = vidioc_enum_fmt_vid_cap_mplane,
> >>> +	.vidioc_enum_fmt_vid_out_mplane = vidioc_enum_fmt_vid_out_mplane,
> >>> +	.vidioc_enum_framesizes		= vidioc_enum_framesizes,
> >>> +
> >>> +	.vidioc_try_fmt_vid_cap_mplane	= vidioc_try_fmt_vid_cap_mplane,
> >>> +	.vidioc_try_fmt_vid_out_mplane	= vidioc_try_fmt_vid_out_mplane,
> >>> +	.vidioc_expbuf			= v4l2_m2m_ioctl_expbuf,
> >>
> >> Please add vidioc_create_bufs and vidioc_prepare_buf as well.
> >>
> > 
> > Currently we do not support these use cases, do we need to add
> > vidioc_create_bufs and vidioc_prepare_buf now?
> 
> I would suggest you do. The vb2 framework gives it (almost) for free.
> prepare_buf is completely free (just add the helper) and create_bufs
> needs a few small changes in the queue_setup function, that's all.
> 
Got it. We will add these in next version.

> > 
> > 
> >>> +	.vidioc_subscribe_event		= v4l2_ctrl_subscribe_event,
> >>> +	.vidioc_unsubscribe_event	= v4l2_event_unsubscribe,
> >>> +
> >>> +	.vidioc_s_parm			= vidioc_venc_s_parm,
> >>> +
> >>> +	.vidioc_s_fmt_vid_cap_mplane	= vidioc_venc_s_fmt,
> >>> +	.vidioc_s_fmt_vid_out_mplane	= vidioc_venc_s_fmt,
> >>> +
> >>> +	.vidioc_g_fmt_vid_cap_mplane	= vidioc_venc_g_fmt,
> >>> +	.vidioc_g_fmt_vid_out_mplane	= vidioc_venc_g_fmt,
> >>> +
> >>> +	.vidioc_g_selection		= vidioc_venc_g_s_selection,
> >>> +	.vidioc_s_selection		= vidioc_venc_g_s_selection,
> >>> +};
> 
> <snip>
> 
> >>> +int mtk_vcodec_enc_queue_init(void *priv, struct vb2_queue *src_vq,
> >>> +			   struct vb2_queue *dst_vq)
> >>> +{
> >>> +	struct mtk_vcodec_ctx *ctx = priv;
> >>> +	int ret;
> >>> +
> >>> +	src_vq->type		= V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE;
> >>> +	src_vq->io_modes	= VB2_DMABUF | VB2_MMAP | VB2_USERPTR;
> >>
> >> I recomment dropping VB2_USERPTR. That only makes sense for scatter-gather dma,
> >> and you use physically contiguous DMA.
> >>
> > Now our userspace app use VB2_USERPTR. I need to check if we could drop
> > VB2_USERPTR.
> > We use src_vq->mem_ops = &vb2_dma_contig_memops;
> > And there are
> > 	.get_userptr	= vb2_dc_get_userptr,
> > 	.put_userptr	= vb2_dc_put_userptr,
> > I was confused why it only make sense for scatter-gather.
> > Could you kindly explain more?
> 
> VB2_USERPTR indicates that the application can use malloc to allocate buffers
> and pass those to the driver. Since malloc uses virtual memory the physical
> memory is scattered all over. And the first page typically does not start at
> the beginning of the page but at a random offset.
> 
> To support that the DMA generally has to be able to do scatter-gather.
> 
> Now, where things get ugly is that a hack was added to the USERPTR support where
> apps could pass a pointer to physically contiguous memory as a user pointer. This
> was a hack for embedded systems that preallocated a pool of buffers and needed to
> pass those pointers around somehow. So the dma-contig USERPTR support is for that
> 'feature'. If you try to pass a malloc()ed buffer to a dma-contig driver it will
> reject it. One big problem is that this specific hack isn't signaled anywhere, so
> applications have no way of knowing if the USERPTR support is the proper version
> or the hack where physically contiguous memory is expected.
> 
> This hack has been replaced with DMABUF which is the proper way of passing buffers
> around.
> 
> New dma-contig drivers should not use that old hack anymore. Use dmabuf to pass
> external buffers around.
> 
> How do you use it in your app? With malloc()ed buffers? Or with 'special' pointers
> to physically contiguous buffers?
> 
Understood now. Thanks for your explanation.
Now our app use malloc()ed buffers and we hook vb2_dma_contig_memops. 
I don't know why that dma-contig driver do not reject it.
I will try to figure it out.

> > 
> >>> +	src_vq->drv_priv	= ctx;
> >>> +	src_vq->buf_struct_size = sizeof(struct mtk_video_enc_buf);
> >>> +	src_vq->ops		= &mtk_venc_vb2_ops;
> >>> +	src_vq->mem_ops		= &vb2_dma_contig_memops;
> >>> +	src_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
> >>> +	src_vq->lock = &ctx->dev->dev_mutex;
> >>> +
> >>> +	ret = vb2_queue_init(src_vq);
> >>> +	if (ret)
> >>> +		return ret;
> >>> +
> >>> +	dst_vq->type		= V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
> >>> +	dst_vq->io_modes	= VB2_DMABUF | VB2_MMAP | VB2_USERPTR;
> >>> +	dst_vq->drv_priv	= ctx;
> >>> +	dst_vq->buf_struct_size = sizeof(struct v4l2_m2m_buffer);
> >>> +	dst_vq->ops		= &mtk_venc_vb2_ops;
> >>> +	dst_vq->mem_ops		= &vb2_dma_contig_memops;
> >>> +	dst_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
> >>> +	dst_vq->lock = &ctx->dev->dev_mutex;
> >>> +
> >>> +	return vb2_queue_init(dst_vq);
> >>> +}
> 
> Regards,
> 
> 	Hans
> 

^ permalink raw reply	[flat|nested] 102+ messages in thread

* Re: [PATCH v4 5/8] [Media] vcodec: mediatek: Add Mediatek V4L2 Video Encoder Driver
@ 2016-02-16 13:20                   ` tiffany lin
  0 siblings, 0 replies; 102+ messages in thread
From: tiffany lin @ 2016-02-16 13:20 UTC (permalink / raw)
  To: Hans Verkuil
  Cc: Hans Verkuil, daniel.thompson-QSEj5FYQhm4dnm+yROfE0A,
	Rob Herring, Mauro Carvalho Chehab, Matthias Brugger,
	Daniel Kurtz, Pawel Osciak, Eddie Huang, Yingjoe Chen,
	devicetree-u79uwXL29TY76Z2rM5mHXA,
	linux-kernel-u79uwXL29TY76Z2rM5mHXA,
	linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
	linux-media-u79uwXL29TY76Z2rM5mHXA,
	linux-mediatek-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
	PoChun.Lin-NuS5LvNUpcJWk0Htik3J/w, Andrew-CT Chen

On Tue, 2016-02-16 at 08:44 +0100, Hans Verkuil wrote:
> On 02/16/2016 07:37 AM, tiffany lin wrote:
> 
> <snip>
> 
> >>> +static int vidioc_venc_s_parm(struct file *file, void *priv,
> >>> +			      struct v4l2_streamparm *a)
> >>> +{
> >>> +	struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
> >>> +
> >>> +	if (a->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
> >>> +		ctx->enc_params.framerate_num =
> >>> +			a->parm.output.timeperframe.denominator;
> >>> +		ctx->enc_params.framerate_denom =
> >>> +			a->parm.output.timeperframe.numerator;
> >>> +		ctx->param_change |= MTK_ENCODE_PARAM_FRAMERATE;
> >>> +	} else {
> >>> +		return -EINVAL;
> >>> +	}
> >>
> >> I'd invert the test:
> >>
> >> 	if (a->type != V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE)
> >> 		return -EINVAL;
> >>
> >> and now you can just set ctx->enc_params.
> >>
> > We will fix this in next version.
> > 
> > 
> >>> +	return 0;
> >>> +}
> >>
> >> And if there is an s_parm, then there should be a g_parm as well!
> >>
> > Now our driver does not support g_parm, our use cases do not use g_parm
> > too. 
> > Do we need to add g_parm at this moment? Or we could add it when we need
> > g_parm?
> 
> No, you need it. You can see why if you look at the v4l2-compliance output:
> 
>                  test VIDIOC_G/S_PARM: OK (Not Supported)
> 
> Why does it think it is unsupported? Because (just like most applications) it
> tries to call G_PARM first, and if that succeeds it tries to call S_PARM with
> the value it got from G_PARM. Thus ensuring the application doesn't change the
> driver state. So you can have a 'get' ioctl without the 'set' ioctl, but if
> there is a 'set' ioctl there must always be a 'get' ioctl.
> 
Got it. We will add g_parm in next version.

> <snip>
> 
> >>> +static int vidioc_venc_g_s_selection(struct file *file, void *priv,
> >>> +                                struct v4l2_selection *s)
> >>
> >> Why support s_selection if you can only return the current width and height?
> >> And why support g_selection if you can't change the selection?
> >>
> >> In other words, why implement this at all?
> >>
> >> Unless I am missing something here, I would just drop this.
> >>
> > Now our driver do not support these capabilities, but userspace app will
> > check whether g/s_crop are implemented when using encoder.
> > Because g/s_crop are deprecated as you mentioned in previous v2 review
> > comments. We change to use g_s_selection.
> > We will check if we could add this capability.
> 
> It's true that you should use g/s_selection instead of g/s_crop, but only if
> there is actually something to select. As long as you don't offer this capability,
> just drop this for now.
> 
> When you add the capability later you can just add the g/s_selection functions.
> 
> Getting selection right can be tricky. I wouldn't mind if this is done later in a
> separate patch.
> 
Got it. We will add the capability later.


> > 
> >>> +{
> >>> +	struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
> >>> +	struct mtk_q_data *q_data;
> >>> +
> >>> +	if (V4L2_TYPE_IS_OUTPUT(s->type)) {
> >>> +		if (s->target !=  V4L2_SEL_TGT_COMPOSE)
> >>> +			return -EINVAL;
> >>> +	} else {
> >>> +		if (s->target != V4L2_SEL_TGT_CROP)
> >>> +			return -EINVAL;
> >>> +	}
> >>> +
> >>> +	if (s->r.left || s->r.top)
> >>> +		return -EINVAL;
> >>> +
> >>> +	q_data = mtk_venc_get_q_data(ctx, s->type);
> >>> +	if (!q_data)
> >>> +		return -EINVAL;
> >>> +
> >>> +	s->r.width = q_data->width;
> >>> +	s->r.height = q_data->height;
> >>> +
> >>> +	return 0;
> >>> +}
> >>> +
> >>> +
> >>> +static int vidioc_venc_qbuf(struct file *file, void *priv,
> >>> +			    struct v4l2_buffer *buf)
> >>> +{
> >>> +
> >>> +	struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
> >>> +
> >>> +	if (ctx->state == MTK_STATE_ABORT) {
> >>> +		mtk_v4l2_err("[%d] Call on QBUF after unrecoverable error\n", ctx->idx);
> >>> +		return -EIO;
> >>> +	}
> >>> +
> >>> +	return v4l2_m2m_qbuf(file, ctx->m2m_ctx, buf);
> >>> +}
> >>> +
> >>> +static int vidioc_venc_dqbuf(struct file *file, void *priv,
> >>> +			     struct v4l2_buffer *buf)
> >>> +{
> >>> +	struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
> >>> +	if (ctx->state == MTK_STATE_ABORT) {
> >>> +		mtk_v4l2_err("[%d] Call on QBUF after unrecoverable error\n", ctx->idx);
> >>> +		return -EIO;
> >>> +	}
> >>> +
> >>> +	return v4l2_m2m_dqbuf(file, ctx->m2m_ctx, buf);
> >>> +}
> >>> +
> >>> +
> >>> +const struct v4l2_ioctl_ops mtk_venc_ioctl_ops = {
> >>> +	.vidioc_streamon		= v4l2_m2m_ioctl_streamon,
> >>> +	.vidioc_streamoff		= v4l2_m2m_ioctl_streamoff,
> >>> +
> >>> +	.vidioc_reqbufs			= v4l2_m2m_ioctl_reqbufs,
> >>> +	.vidioc_querybuf		= v4l2_m2m_ioctl_querybuf,
> >>> +	.vidioc_qbuf			= vidioc_venc_qbuf,
> >>> +	.vidioc_dqbuf			= vidioc_venc_dqbuf,
> >>> +
> >>> +	.vidioc_querycap		= vidioc_venc_querycap,
> >>> +	.vidioc_enum_fmt_vid_cap_mplane = vidioc_enum_fmt_vid_cap_mplane,
> >>> +	.vidioc_enum_fmt_vid_out_mplane = vidioc_enum_fmt_vid_out_mplane,
> >>> +	.vidioc_enum_framesizes		= vidioc_enum_framesizes,
> >>> +
> >>> +	.vidioc_try_fmt_vid_cap_mplane	= vidioc_try_fmt_vid_cap_mplane,
> >>> +	.vidioc_try_fmt_vid_out_mplane	= vidioc_try_fmt_vid_out_mplane,
> >>> +	.vidioc_expbuf			= v4l2_m2m_ioctl_expbuf,
> >>
> >> Please add vidioc_create_bufs and vidioc_prepare_buf as well.
> >>
> > 
> > Currently we do not support these use cases, do we need to add
> > vidioc_create_bufs and vidioc_prepare_buf now?
> 
> I would suggest you do. The vb2 framework gives it (almost) for free.
> prepare_buf is completely free (just add the helper) and create_bufs
> needs a few small changes in the queue_setup function, that's all.
> 
Got it. We will add these in next version.

> > 
> > 
> >>> +	.vidioc_subscribe_event		= v4l2_ctrl_subscribe_event,
> >>> +	.vidioc_unsubscribe_event	= v4l2_event_unsubscribe,
> >>> +
> >>> +	.vidioc_s_parm			= vidioc_venc_s_parm,
> >>> +
> >>> +	.vidioc_s_fmt_vid_cap_mplane	= vidioc_venc_s_fmt,
> >>> +	.vidioc_s_fmt_vid_out_mplane	= vidioc_venc_s_fmt,
> >>> +
> >>> +	.vidioc_g_fmt_vid_cap_mplane	= vidioc_venc_g_fmt,
> >>> +	.vidioc_g_fmt_vid_out_mplane	= vidioc_venc_g_fmt,
> >>> +
> >>> +	.vidioc_g_selection		= vidioc_venc_g_s_selection,
> >>> +	.vidioc_s_selection		= vidioc_venc_g_s_selection,
> >>> +};
> 
> <snip>
> 
> >>> +int mtk_vcodec_enc_queue_init(void *priv, struct vb2_queue *src_vq,
> >>> +			   struct vb2_queue *dst_vq)
> >>> +{
> >>> +	struct mtk_vcodec_ctx *ctx = priv;
> >>> +	int ret;
> >>> +
> >>> +	src_vq->type		= V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE;
> >>> +	src_vq->io_modes	= VB2_DMABUF | VB2_MMAP | VB2_USERPTR;
> >>
> >> I recomment dropping VB2_USERPTR. That only makes sense for scatter-gather dma,
> >> and you use physically contiguous DMA.
> >>
> > Now our userspace app use VB2_USERPTR. I need to check if we could drop
> > VB2_USERPTR.
> > We use src_vq->mem_ops = &vb2_dma_contig_memops;
> > And there are
> > 	.get_userptr	= vb2_dc_get_userptr,
> > 	.put_userptr	= vb2_dc_put_userptr,
> > I was confused why it only make sense for scatter-gather.
> > Could you kindly explain more?
> 
> VB2_USERPTR indicates that the application can use malloc to allocate buffers
> and pass those to the driver. Since malloc uses virtual memory the physical
> memory is scattered all over. And the first page typically does not start at
> the beginning of the page but at a random offset.
> 
> To support that the DMA generally has to be able to do scatter-gather.
> 
> Now, where things get ugly is that a hack was added to the USERPTR support where
> apps could pass a pointer to physically contiguous memory as a user pointer. This
> was a hack for embedded systems that preallocated a pool of buffers and needed to
> pass those pointers around somehow. So the dma-contig USERPTR support is for that
> 'feature'. If you try to pass a malloc()ed buffer to a dma-contig driver it will
> reject it. One big problem is that this specific hack isn't signaled anywhere, so
> applications have no way of knowing if the USERPTR support is the proper version
> or the hack where physically contiguous memory is expected.
> 
> This hack has been replaced with DMABUF which is the proper way of passing buffers
> around.
> 
> New dma-contig drivers should not use that old hack anymore. Use dmabuf to pass
> external buffers around.
> 
> How do you use it in your app? With malloc()ed buffers? Or with 'special' pointers
> to physically contiguous buffers?
> 
Understood now. Thanks for your explanation.
Now our app use malloc()ed buffers and we hook vb2_dma_contig_memops. 
I don't know why that dma-contig driver do not reject it.
I will try to figure it out.

> > 
> >>> +	src_vq->drv_priv	= ctx;
> >>> +	src_vq->buf_struct_size = sizeof(struct mtk_video_enc_buf);
> >>> +	src_vq->ops		= &mtk_venc_vb2_ops;
> >>> +	src_vq->mem_ops		= &vb2_dma_contig_memops;
> >>> +	src_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
> >>> +	src_vq->lock = &ctx->dev->dev_mutex;
> >>> +
> >>> +	ret = vb2_queue_init(src_vq);
> >>> +	if (ret)
> >>> +		return ret;
> >>> +
> >>> +	dst_vq->type		= V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
> >>> +	dst_vq->io_modes	= VB2_DMABUF | VB2_MMAP | VB2_USERPTR;
> >>> +	dst_vq->drv_priv	= ctx;
> >>> +	dst_vq->buf_struct_size = sizeof(struct v4l2_m2m_buffer);
> >>> +	dst_vq->ops		= &mtk_venc_vb2_ops;
> >>> +	dst_vq->mem_ops		= &vb2_dma_contig_memops;
> >>> +	dst_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
> >>> +	dst_vq->lock = &ctx->dev->dev_mutex;
> >>> +
> >>> +	return vb2_queue_init(dst_vq);
> >>> +}
> 
> Regards,
> 
> 	Hans
> 


--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 102+ messages in thread

* [PATCH v4 5/8] [Media] vcodec: mediatek: Add Mediatek V4L2 Video Encoder Driver
@ 2016-02-16 13:20                   ` tiffany lin
  0 siblings, 0 replies; 102+ messages in thread
From: tiffany lin @ 2016-02-16 13:20 UTC (permalink / raw)
  To: linux-arm-kernel

On Tue, 2016-02-16 at 08:44 +0100, Hans Verkuil wrote:
> On 02/16/2016 07:37 AM, tiffany lin wrote:
> 
> <snip>
> 
> >>> +static int vidioc_venc_s_parm(struct file *file, void *priv,
> >>> +			      struct v4l2_streamparm *a)
> >>> +{
> >>> +	struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
> >>> +
> >>> +	if (a->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
> >>> +		ctx->enc_params.framerate_num =
> >>> +			a->parm.output.timeperframe.denominator;
> >>> +		ctx->enc_params.framerate_denom =
> >>> +			a->parm.output.timeperframe.numerator;
> >>> +		ctx->param_change |= MTK_ENCODE_PARAM_FRAMERATE;
> >>> +	} else {
> >>> +		return -EINVAL;
> >>> +	}
> >>
> >> I'd invert the test:
> >>
> >> 	if (a->type != V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE)
> >> 		return -EINVAL;
> >>
> >> and now you can just set ctx->enc_params.
> >>
> > We will fix this in next version.
> > 
> > 
> >>> +	return 0;
> >>> +}
> >>
> >> And if there is an s_parm, then there should be a g_parm as well!
> >>
> > Now our driver does not support g_parm, our use cases do not use g_parm
> > too. 
> > Do we need to add g_parm at this moment? Or we could add it when we need
> > g_parm?
> 
> No, you need it. You can see why if you look at the v4l2-compliance output:
> 
>                  test VIDIOC_G/S_PARM: OK (Not Supported)
> 
> Why does it think it is unsupported? Because (just like most applications) it
> tries to call G_PARM first, and if that succeeds it tries to call S_PARM with
> the value it got from G_PARM. Thus ensuring the application doesn't change the
> driver state. So you can have a 'get' ioctl without the 'set' ioctl, but if
> there is a 'set' ioctl there must always be a 'get' ioctl.
> 
Got it. We will add g_parm in next version.

> <snip>
> 
> >>> +static int vidioc_venc_g_s_selection(struct file *file, void *priv,
> >>> +                                struct v4l2_selection *s)
> >>
> >> Why support s_selection if you can only return the current width and height?
> >> And why support g_selection if you can't change the selection?
> >>
> >> In other words, why implement this at all?
> >>
> >> Unless I am missing something here, I would just drop this.
> >>
> > Now our driver do not support these capabilities, but userspace app will
> > check whether g/s_crop are implemented when using encoder.
> > Because g/s_crop are deprecated as you mentioned in previous v2 review
> > comments. We change to use g_s_selection.
> > We will check if we could add this capability.
> 
> It's true that you should use g/s_selection instead of g/s_crop, but only if
> there is actually something to select. As long as you don't offer this capability,
> just drop this for now.
> 
> When you add the capability later you can just add the g/s_selection functions.
> 
> Getting selection right can be tricky. I wouldn't mind if this is done later in a
> separate patch.
> 
Got it. We will add the capability later.


> > 
> >>> +{
> >>> +	struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
> >>> +	struct mtk_q_data *q_data;
> >>> +
> >>> +	if (V4L2_TYPE_IS_OUTPUT(s->type)) {
> >>> +		if (s->target !=  V4L2_SEL_TGT_COMPOSE)
> >>> +			return -EINVAL;
> >>> +	} else {
> >>> +		if (s->target != V4L2_SEL_TGT_CROP)
> >>> +			return -EINVAL;
> >>> +	}
> >>> +
> >>> +	if (s->r.left || s->r.top)
> >>> +		return -EINVAL;
> >>> +
> >>> +	q_data = mtk_venc_get_q_data(ctx, s->type);
> >>> +	if (!q_data)
> >>> +		return -EINVAL;
> >>> +
> >>> +	s->r.width = q_data->width;
> >>> +	s->r.height = q_data->height;
> >>> +
> >>> +	return 0;
> >>> +}
> >>> +
> >>> +
> >>> +static int vidioc_venc_qbuf(struct file *file, void *priv,
> >>> +			    struct v4l2_buffer *buf)
> >>> +{
> >>> +
> >>> +	struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
> >>> +
> >>> +	if (ctx->state == MTK_STATE_ABORT) {
> >>> +		mtk_v4l2_err("[%d] Call on QBUF after unrecoverable error\n", ctx->idx);
> >>> +		return -EIO;
> >>> +	}
> >>> +
> >>> +	return v4l2_m2m_qbuf(file, ctx->m2m_ctx, buf);
> >>> +}
> >>> +
> >>> +static int vidioc_venc_dqbuf(struct file *file, void *priv,
> >>> +			     struct v4l2_buffer *buf)
> >>> +{
> >>> +	struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
> >>> +	if (ctx->state == MTK_STATE_ABORT) {
> >>> +		mtk_v4l2_err("[%d] Call on QBUF after unrecoverable error\n", ctx->idx);
> >>> +		return -EIO;
> >>> +	}
> >>> +
> >>> +	return v4l2_m2m_dqbuf(file, ctx->m2m_ctx, buf);
> >>> +}
> >>> +
> >>> +
> >>> +const struct v4l2_ioctl_ops mtk_venc_ioctl_ops = {
> >>> +	.vidioc_streamon		= v4l2_m2m_ioctl_streamon,
> >>> +	.vidioc_streamoff		= v4l2_m2m_ioctl_streamoff,
> >>> +
> >>> +	.vidioc_reqbufs			= v4l2_m2m_ioctl_reqbufs,
> >>> +	.vidioc_querybuf		= v4l2_m2m_ioctl_querybuf,
> >>> +	.vidioc_qbuf			= vidioc_venc_qbuf,
> >>> +	.vidioc_dqbuf			= vidioc_venc_dqbuf,
> >>> +
> >>> +	.vidioc_querycap		= vidioc_venc_querycap,
> >>> +	.vidioc_enum_fmt_vid_cap_mplane = vidioc_enum_fmt_vid_cap_mplane,
> >>> +	.vidioc_enum_fmt_vid_out_mplane = vidioc_enum_fmt_vid_out_mplane,
> >>> +	.vidioc_enum_framesizes		= vidioc_enum_framesizes,
> >>> +
> >>> +	.vidioc_try_fmt_vid_cap_mplane	= vidioc_try_fmt_vid_cap_mplane,
> >>> +	.vidioc_try_fmt_vid_out_mplane	= vidioc_try_fmt_vid_out_mplane,
> >>> +	.vidioc_expbuf			= v4l2_m2m_ioctl_expbuf,
> >>
> >> Please add vidioc_create_bufs and vidioc_prepare_buf as well.
> >>
> > 
> > Currently we do not support these use cases, do we need to add
> > vidioc_create_bufs and vidioc_prepare_buf now?
> 
> I would suggest you do. The vb2 framework gives it (almost) for free.
> prepare_buf is completely free (just add the helper) and create_bufs
> needs a few small changes in the queue_setup function, that's all.
> 
Got it. We will add these in next version.

> > 
> > 
> >>> +	.vidioc_subscribe_event		= v4l2_ctrl_subscribe_event,
> >>> +	.vidioc_unsubscribe_event	= v4l2_event_unsubscribe,
> >>> +
> >>> +	.vidioc_s_parm			= vidioc_venc_s_parm,
> >>> +
> >>> +	.vidioc_s_fmt_vid_cap_mplane	= vidioc_venc_s_fmt,
> >>> +	.vidioc_s_fmt_vid_out_mplane	= vidioc_venc_s_fmt,
> >>> +
> >>> +	.vidioc_g_fmt_vid_cap_mplane	= vidioc_venc_g_fmt,
> >>> +	.vidioc_g_fmt_vid_out_mplane	= vidioc_venc_g_fmt,
> >>> +
> >>> +	.vidioc_g_selection		= vidioc_venc_g_s_selection,
> >>> +	.vidioc_s_selection		= vidioc_venc_g_s_selection,
> >>> +};
> 
> <snip>
> 
> >>> +int mtk_vcodec_enc_queue_init(void *priv, struct vb2_queue *src_vq,
> >>> +			   struct vb2_queue *dst_vq)
> >>> +{
> >>> +	struct mtk_vcodec_ctx *ctx = priv;
> >>> +	int ret;
> >>> +
> >>> +	src_vq->type		= V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE;
> >>> +	src_vq->io_modes	= VB2_DMABUF | VB2_MMAP | VB2_USERPTR;
> >>
> >> I recomment dropping VB2_USERPTR. That only makes sense for scatter-gather dma,
> >> and you use physically contiguous DMA.
> >>
> > Now our userspace app use VB2_USERPTR. I need to check if we could drop
> > VB2_USERPTR.
> > We use src_vq->mem_ops = &vb2_dma_contig_memops;
> > And there are
> > 	.get_userptr	= vb2_dc_get_userptr,
> > 	.put_userptr	= vb2_dc_put_userptr,
> > I was confused why it only make sense for scatter-gather.
> > Could you kindly explain more?
> 
> VB2_USERPTR indicates that the application can use malloc to allocate buffers
> and pass those to the driver. Since malloc uses virtual memory the physical
> memory is scattered all over. And the first page typically does not start at
> the beginning of the page but at a random offset.
> 
> To support that the DMA generally has to be able to do scatter-gather.
> 
> Now, where things get ugly is that a hack was added to the USERPTR support where
> apps could pass a pointer to physically contiguous memory as a user pointer. This
> was a hack for embedded systems that preallocated a pool of buffers and needed to
> pass those pointers around somehow. So the dma-contig USERPTR support is for that
> 'feature'. If you try to pass a malloc()ed buffer to a dma-contig driver it will
> reject it. One big problem is that this specific hack isn't signaled anywhere, so
> applications have no way of knowing if the USERPTR support is the proper version
> or the hack where physically contiguous memory is expected.
> 
> This hack has been replaced with DMABUF which is the proper way of passing buffers
> around.
> 
> New dma-contig drivers should not use that old hack anymore. Use dmabuf to pass
> external buffers around.
> 
> How do you use it in your app? With malloc()ed buffers? Or with 'special' pointers
> to physically contiguous buffers?
> 
Understood now. Thanks for your explanation.
Now our app use malloc()ed buffers and we hook vb2_dma_contig_memops. 
I don't know why that dma-contig driver do not reject it.
I will try to figure it out.

> > 
> >>> +	src_vq->drv_priv	= ctx;
> >>> +	src_vq->buf_struct_size = sizeof(struct mtk_video_enc_buf);
> >>> +	src_vq->ops		= &mtk_venc_vb2_ops;
> >>> +	src_vq->mem_ops		= &vb2_dma_contig_memops;
> >>> +	src_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
> >>> +	src_vq->lock = &ctx->dev->dev_mutex;
> >>> +
> >>> +	ret = vb2_queue_init(src_vq);
> >>> +	if (ret)
> >>> +		return ret;
> >>> +
> >>> +	dst_vq->type		= V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
> >>> +	dst_vq->io_modes	= VB2_DMABUF | VB2_MMAP | VB2_USERPTR;
> >>> +	dst_vq->drv_priv	= ctx;
> >>> +	dst_vq->buf_struct_size = sizeof(struct v4l2_m2m_buffer);
> >>> +	dst_vq->ops		= &mtk_venc_vb2_ops;
> >>> +	dst_vq->mem_ops		= &vb2_dma_contig_memops;
> >>> +	dst_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
> >>> +	dst_vq->lock = &ctx->dev->dev_mutex;
> >>> +
> >>> +	return vb2_queue_init(dst_vq);
> >>> +}
> 
> Regards,
> 
> 	Hans
> 

^ permalink raw reply	[flat|nested] 102+ messages in thread

* Re: [PATCH v4 5/8] [Media] vcodec: mediatek: Add Mediatek V4L2 Video Encoder Driver
  2016-02-16 13:20                   ` tiffany lin
@ 2016-02-16 13:48                     ` Hans Verkuil
  -1 siblings, 0 replies; 102+ messages in thread
From: Hans Verkuil @ 2016-02-16 13:48 UTC (permalink / raw)
  To: tiffany lin, Hans Verkuil
  Cc: Hans Verkuil, daniel.thompson, Rob Herring,
	Mauro Carvalho Chehab, Matthias Brugger, Daniel Kurtz,
	Pawel Osciak, Eddie Huang, Yingjoe Chen, devicetree,
	linux-kernel, linux-arm-kernel, linux-media, linux-mediatek,
	PoChun.Lin, Andrew-CT Chen

Hi Tiffany,

>>>>> +int mtk_vcodec_enc_queue_init(void *priv, struct vb2_queue *src_vq,
>>>>> +			   struct vb2_queue *dst_vq)
>>>>> +{
>>>>> +	struct mtk_vcodec_ctx *ctx = priv;
>>>>> +	int ret;
>>>>> +
>>>>> +	src_vq->type		= V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE;
>>>>> +	src_vq->io_modes	= VB2_DMABUF | VB2_MMAP | VB2_USERPTR;
>>>>
>>>> I recomment dropping VB2_USERPTR. That only makes sense for scatter-gather dma,
>>>> and you use physically contiguous DMA.
>>>>
>>> Now our userspace app use VB2_USERPTR. I need to check if we could drop
>>> VB2_USERPTR.
>>> We use src_vq->mem_ops = &vb2_dma_contig_memops;
>>> And there are
>>> 	.get_userptr	= vb2_dc_get_userptr,
>>> 	.put_userptr	= vb2_dc_put_userptr,
>>> I was confused why it only make sense for scatter-gather.
>>> Could you kindly explain more?
>>
>> VB2_USERPTR indicates that the application can use malloc to allocate buffers
>> and pass those to the driver. Since malloc uses virtual memory the physical
>> memory is scattered all over. And the first page typically does not start at
>> the beginning of the page but at a random offset.
>>
>> To support that the DMA generally has to be able to do scatter-gather.
>>
>> Now, where things get ugly is that a hack was added to the USERPTR support where
>> apps could pass a pointer to physically contiguous memory as a user pointer. This
>> was a hack for embedded systems that preallocated a pool of buffers and needed to
>> pass those pointers around somehow. So the dma-contig USERPTR support is for that
>> 'feature'. If you try to pass a malloc()ed buffer to a dma-contig driver it will
>> reject it. One big problem is that this specific hack isn't signaled anywhere, so
>> applications have no way of knowing if the USERPTR support is the proper version
>> or the hack where physically contiguous memory is expected.
>>
>> This hack has been replaced with DMABUF which is the proper way of passing buffers
>> around.
>>
>> New dma-contig drivers should not use that old hack anymore. Use dmabuf to pass
>> external buffers around.
>>
>> How do you use it in your app? With malloc()ed buffers? Or with 'special' pointers
>> to physically contiguous buffers?
>>
> Understood now. Thanks for your explanation.
> Now our app use malloc()ed buffers and we hook vb2_dma_contig_memops. 
> I don't know why that dma-contig driver do not reject it.
> I will try to figure it out.

Is there an iommu involved that turns the scatter-gather list into what looks like
contiguous memory for the DMA?

At the end of vb2_dc_get_userptr() in videobuf2-dma-contig.c there is a check
'if (contig_size < size)' that verifies that the sg DMA is contiguous. This would
work if there is an iommu involved (if I understand it correctly).

If that's the case, then it's OK to keep VB2_USERPTR because you have real sg
support (although not via the DMA engine, but via iommu mappings).

Regards,

	Hans

^ permalink raw reply	[flat|nested] 102+ messages in thread

* [PATCH v4 5/8] [Media] vcodec: mediatek: Add Mediatek V4L2 Video Encoder Driver
@ 2016-02-16 13:48                     ` Hans Verkuil
  0 siblings, 0 replies; 102+ messages in thread
From: Hans Verkuil @ 2016-02-16 13:48 UTC (permalink / raw)
  To: linux-arm-kernel

Hi Tiffany,

>>>>> +int mtk_vcodec_enc_queue_init(void *priv, struct vb2_queue *src_vq,
>>>>> +			   struct vb2_queue *dst_vq)
>>>>> +{
>>>>> +	struct mtk_vcodec_ctx *ctx = priv;
>>>>> +	int ret;
>>>>> +
>>>>> +	src_vq->type		= V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE;
>>>>> +	src_vq->io_modes	= VB2_DMABUF | VB2_MMAP | VB2_USERPTR;
>>>>
>>>> I recomment dropping VB2_USERPTR. That only makes sense for scatter-gather dma,
>>>> and you use physically contiguous DMA.
>>>>
>>> Now our userspace app use VB2_USERPTR. I need to check if we could drop
>>> VB2_USERPTR.
>>> We use src_vq->mem_ops = &vb2_dma_contig_memops;
>>> And there are
>>> 	.get_userptr	= vb2_dc_get_userptr,
>>> 	.put_userptr	= vb2_dc_put_userptr,
>>> I was confused why it only make sense for scatter-gather.
>>> Could you kindly explain more?
>>
>> VB2_USERPTR indicates that the application can use malloc to allocate buffers
>> and pass those to the driver. Since malloc uses virtual memory the physical
>> memory is scattered all over. And the first page typically does not start at
>> the beginning of the page but at a random offset.
>>
>> To support that the DMA generally has to be able to do scatter-gather.
>>
>> Now, where things get ugly is that a hack was added to the USERPTR support where
>> apps could pass a pointer to physically contiguous memory as a user pointer. This
>> was a hack for embedded systems that preallocated a pool of buffers and needed to
>> pass those pointers around somehow. So the dma-contig USERPTR support is for that
>> 'feature'. If you try to pass a malloc()ed buffer to a dma-contig driver it will
>> reject it. One big problem is that this specific hack isn't signaled anywhere, so
>> applications have no way of knowing if the USERPTR support is the proper version
>> or the hack where physically contiguous memory is expected.
>>
>> This hack has been replaced with DMABUF which is the proper way of passing buffers
>> around.
>>
>> New dma-contig drivers should not use that old hack anymore. Use dmabuf to pass
>> external buffers around.
>>
>> How do you use it in your app? With malloc()ed buffers? Or with 'special' pointers
>> to physically contiguous buffers?
>>
> Understood now. Thanks for your explanation.
> Now our app use malloc()ed buffers and we hook vb2_dma_contig_memops. 
> I don't know why that dma-contig driver do not reject it.
> I will try to figure it out.

Is there an iommu involved that turns the scatter-gather list into what looks like
contiguous memory for the DMA?

At the end of vb2_dc_get_userptr() in videobuf2-dma-contig.c there is a check
'if (contig_size < size)' that verifies that the sg DMA is contiguous. This would
work if there is an iommu involved (if I understand it correctly).

If that's the case, then it's OK to keep VB2_USERPTR because you have real sg
support (although not via the DMA engine, but via iommu mappings).

Regards,

	Hans

^ permalink raw reply	[flat|nested] 102+ messages in thread

* Re: [PATCH v4 5/8] [Media] vcodec: mediatek: Add Mediatek V4L2 Video Encoder Driver
  2016-02-16  6:37               ` tiffany lin
@ 2016-02-17  7:47                 ` Hans Verkuil
  -1 siblings, 0 replies; 102+ messages in thread
From: Hans Verkuil @ 2016-02-17  7:47 UTC (permalink / raw)
  To: tiffany lin
  Cc: Hans Verkuil, daniel.thompson, Rob Herring,
	Mauro Carvalho Chehab, Matthias Brugger, Daniel Kurtz,
	Pawel Osciak, Eddie Huang, Yingjoe Chen, devicetree,
	linux-kernel, linux-arm-kernel, linux-media, linux-mediatek,
	PoChun.Lin, Andrew-CT Chen

On 02/16/2016 07:37 AM, tiffany lin wrote:
>>> +static int fops_vcodec_open(struct file *file)
>>> +{
>>> +	struct video_device *vfd = video_devdata(file);
>>> +	struct mtk_vcodec_dev *dev = video_drvdata(file);
>>> +	struct mtk_vcodec_ctx *ctx = NULL;
>>> +	int ret = 0;
>>> +
>>> +	mutex_lock(&dev->dev_mutex);
>>> +
>>> +	ctx = devm_kzalloc(&dev->plat_dev->dev, sizeof(*ctx), GFP_KERNEL);
>>> +	if (!ctx) {
>>> +		ret = -ENOMEM;
>>> +		goto err_alloc;
>>> +	}
>>> +
>>> +	if (dev->num_instances >= MTK_VCODEC_MAX_ENCODER_INSTANCES) {
>>> +		mtk_v4l2_err("Too many open contexts\n");
>>> +		ret = -EBUSY;
>>> +		goto err_no_ctx;
>>
>> Hmm. I never like it if you can't open a video node because of a reason like this.
>>
>> I.e. a simple 'v4l2-ctl -D' (i.e. calling QUERYCAP) should never fail.
>>
>> If there are hardware limitation that prevent more than X instances from running at
>> the same time, then those limitations typically kick in when you start to stream
>> (or possibly when calling REQBUFS). But before that it should always be possible to
>> open the device.
>>
>> Having this check at open() is an indication of a poor design.
>>
>> Is this is a hardware limitation at all?
>>
> This is to make sure performance meet requirements, such as bitrate and
> framerate.

Is it the driver's job to make enforce this? What if the application only
deals with low-res video, but wants to encode a lot of those? Or is encoding
a video off-line?

The driver generally doesn't know the use-case, so if this is an artificial
limitation as opposed to a hardware limitation, then I would just drop this.

Regards,

	Hans

> We got your point. We will remove this and move limitation control to
> start_streaming or REQBUFS.
> Appreciated for your suggestion.:)
> 
> 
>>> +	}

^ permalink raw reply	[flat|nested] 102+ messages in thread

* [PATCH v4 5/8] [Media] vcodec: mediatek: Add Mediatek V4L2 Video Encoder Driver
@ 2016-02-17  7:47                 ` Hans Verkuil
  0 siblings, 0 replies; 102+ messages in thread
From: Hans Verkuil @ 2016-02-17  7:47 UTC (permalink / raw)
  To: linux-arm-kernel

On 02/16/2016 07:37 AM, tiffany lin wrote:
>>> +static int fops_vcodec_open(struct file *file)
>>> +{
>>> +	struct video_device *vfd = video_devdata(file);
>>> +	struct mtk_vcodec_dev *dev = video_drvdata(file);
>>> +	struct mtk_vcodec_ctx *ctx = NULL;
>>> +	int ret = 0;
>>> +
>>> +	mutex_lock(&dev->dev_mutex);
>>> +
>>> +	ctx = devm_kzalloc(&dev->plat_dev->dev, sizeof(*ctx), GFP_KERNEL);
>>> +	if (!ctx) {
>>> +		ret = -ENOMEM;
>>> +		goto err_alloc;
>>> +	}
>>> +
>>> +	if (dev->num_instances >= MTK_VCODEC_MAX_ENCODER_INSTANCES) {
>>> +		mtk_v4l2_err("Too many open contexts\n");
>>> +		ret = -EBUSY;
>>> +		goto err_no_ctx;
>>
>> Hmm. I never like it if you can't open a video node because of a reason like this.
>>
>> I.e. a simple 'v4l2-ctl -D' (i.e. calling QUERYCAP) should never fail.
>>
>> If there are hardware limitation that prevent more than X instances from running at
>> the same time, then those limitations typically kick in when you start to stream
>> (or possibly when calling REQBUFS). But before that it should always be possible to
>> open the device.
>>
>> Having this check at open() is an indication of a poor design.
>>
>> Is this is a hardware limitation at all?
>>
> This is to make sure performance meet requirements, such as bitrate and
> framerate.

Is it the driver's job to make enforce this? What if the application only
deals with low-res video, but wants to encode a lot of those? Or is encoding
a video off-line?

The driver generally doesn't know the use-case, so if this is an artificial
limitation as opposed to a hardware limitation, then I would just drop this.

Regards,

	Hans

> We got your point. We will remove this and move limitation control to
> start_streaming or REQBUFS.
> Appreciated for your suggestion.:)
> 
> 
>>> +	}

^ permalink raw reply	[flat|nested] 102+ messages in thread

* Re: [PATCH v4 5/8] [Media] vcodec: mediatek: Add Mediatek V4L2 Video Encoder Driver
  2016-02-16 13:48                     ` Hans Verkuil
  (?)
@ 2016-02-17  8:01                       ` tiffany lin
  -1 siblings, 0 replies; 102+ messages in thread
From: tiffany lin @ 2016-02-17  8:01 UTC (permalink / raw)
  To: Hans Verkuil
  Cc: Hans Verkuil, Hans Verkuil, daniel.thompson, Rob Herring,
	Mauro Carvalho Chehab, Matthias Brugger, Daniel Kurtz,
	Pawel Osciak, Eddie Huang, Yingjoe Chen, devicetree,
	linux-kernel, linux-arm-kernel, linux-media, linux-mediatek,
	PoChun.Lin, Andrew-CT Chen

On Tue, 2016-02-16 at 14:48 +0100, Hans Verkuil wrote:
> Hi Tiffany,
> 
> >>>>> +int mtk_vcodec_enc_queue_init(void *priv, struct vb2_queue *src_vq,
> >>>>> +			   struct vb2_queue *dst_vq)
> >>>>> +{
> >>>>> +	struct mtk_vcodec_ctx *ctx = priv;
> >>>>> +	int ret;
> >>>>> +
> >>>>> +	src_vq->type		= V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE;
> >>>>> +	src_vq->io_modes	= VB2_DMABUF | VB2_MMAP | VB2_USERPTR;
> >>>>
> >>>> I recomment dropping VB2_USERPTR. That only makes sense for scatter-gather dma,
> >>>> and you use physically contiguous DMA.
> >>>>
> >>> Now our userspace app use VB2_USERPTR. I need to check if we could drop
> >>> VB2_USERPTR.
> >>> We use src_vq->mem_ops = &vb2_dma_contig_memops;
> >>> And there are
> >>> 	.get_userptr	= vb2_dc_get_userptr,
> >>> 	.put_userptr	= vb2_dc_put_userptr,
> >>> I was confused why it only make sense for scatter-gather.
> >>> Could you kindly explain more?
> >>
> >> VB2_USERPTR indicates that the application can use malloc to allocate buffers
> >> and pass those to the driver. Since malloc uses virtual memory the physical
> >> memory is scattered all over. And the first page typically does not start at
> >> the beginning of the page but at a random offset.
> >>
> >> To support that the DMA generally has to be able to do scatter-gather.
> >>
> >> Now, where things get ugly is that a hack was added to the USERPTR support where
> >> apps could pass a pointer to physically contiguous memory as a user pointer. This
> >> was a hack for embedded systems that preallocated a pool of buffers and needed to
> >> pass those pointers around somehow. So the dma-contig USERPTR support is for that
> >> 'feature'. If you try to pass a malloc()ed buffer to a dma-contig driver it will
> >> reject it. One big problem is that this specific hack isn't signaled anywhere, so
> >> applications have no way of knowing if the USERPTR support is the proper version
> >> or the hack where physically contiguous memory is expected.
> >>
> >> This hack has been replaced with DMABUF which is the proper way of passing buffers
> >> around.
> >>
> >> New dma-contig drivers should not use that old hack anymore. Use dmabuf to pass
> >> external buffers around.
> >>
> >> How do you use it in your app? With malloc()ed buffers? Or with 'special' pointers
> >> to physically contiguous buffers?
> >>
> > Understood now. Thanks for your explanation.
> > Now our app use malloc()ed buffers and we hook vb2_dma_contig_memops. 
> > I don't know why that dma-contig driver do not reject it.
> > I will try to figure it out.
> 
> Is there an iommu involved that turns the scatter-gather list into what looks like
> contiguous memory for the DMA?
> 
Yes, We have iommu that could make scatter-gather list looks like
contiguous memory.

> At the end of vb2_dc_get_userptr() in videobuf2-dma-contig.c there is a check
> 'if (contig_size < size)' that verifies that the sg DMA is contiguous. This would
> work if there is an iommu involved (if I understand it correctly).
> 
I see. We saw this error before we add iommu support.

> If that's the case, then it's OK to keep VB2_USERPTR because you have real sg
> support (although not via the DMA engine, but via iommu mappings).
> 
Got it. We will keep VB2_USERPTR.

> Regards,
> 
> 	Hans

^ permalink raw reply	[flat|nested] 102+ messages in thread

* Re: [PATCH v4 5/8] [Media] vcodec: mediatek: Add Mediatek V4L2 Video Encoder Driver
@ 2016-02-17  8:01                       ` tiffany lin
  0 siblings, 0 replies; 102+ messages in thread
From: tiffany lin @ 2016-02-17  8:01 UTC (permalink / raw)
  To: Hans Verkuil
  Cc: Hans Verkuil, Hans Verkuil, daniel.thompson, Rob Herring,
	Mauro Carvalho Chehab, Matthias Brugger, Daniel Kurtz,
	Pawel Osciak, Eddie Huang, Yingjoe Chen, devicetree,
	linux-kernel, linux-arm-kernel, linux-media, linux-mediatek,
	PoChun.Lin, Andrew-CT Chen

On Tue, 2016-02-16 at 14:48 +0100, Hans Verkuil wrote:
> Hi Tiffany,
> 
> >>>>> +int mtk_vcodec_enc_queue_init(void *priv, struct vb2_queue *src_vq,
> >>>>> +			   struct vb2_queue *dst_vq)
> >>>>> +{
> >>>>> +	struct mtk_vcodec_ctx *ctx = priv;
> >>>>> +	int ret;
> >>>>> +
> >>>>> +	src_vq->type		= V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE;
> >>>>> +	src_vq->io_modes	= VB2_DMABUF | VB2_MMAP | VB2_USERPTR;
> >>>>
> >>>> I recomment dropping VB2_USERPTR. That only makes sense for scatter-gather dma,
> >>>> and you use physically contiguous DMA.
> >>>>
> >>> Now our userspace app use VB2_USERPTR. I need to check if we could drop
> >>> VB2_USERPTR.
> >>> We use src_vq->mem_ops = &vb2_dma_contig_memops;
> >>> And there are
> >>> 	.get_userptr	= vb2_dc_get_userptr,
> >>> 	.put_userptr	= vb2_dc_put_userptr,
> >>> I was confused why it only make sense for scatter-gather.
> >>> Could you kindly explain more?
> >>
> >> VB2_USERPTR indicates that the application can use malloc to allocate buffers
> >> and pass those to the driver. Since malloc uses virtual memory the physical
> >> memory is scattered all over. And the first page typically does not start at
> >> the beginning of the page but at a random offset.
> >>
> >> To support that the DMA generally has to be able to do scatter-gather.
> >>
> >> Now, where things get ugly is that a hack was added to the USERPTR support where
> >> apps could pass a pointer to physically contiguous memory as a user pointer. This
> >> was a hack for embedded systems that preallocated a pool of buffers and needed to
> >> pass those pointers around somehow. So the dma-contig USERPTR support is for that
> >> 'feature'. If you try to pass a malloc()ed buffer to a dma-contig driver it will
> >> reject it. One big problem is that this specific hack isn't signaled anywhere, so
> >> applications have no way of knowing if the USERPTR support is the proper version
> >> or the hack where physically contiguous memory is expected.
> >>
> >> This hack has been replaced with DMABUF which is the proper way of passing buffers
> >> around.
> >>
> >> New dma-contig drivers should not use that old hack anymore. Use dmabuf to pass
> >> external buffers around.
> >>
> >> How do you use it in your app? With malloc()ed buffers? Or with 'special' pointers
> >> to physically contiguous buffers?
> >>
> > Understood now. Thanks for your explanation.
> > Now our app use malloc()ed buffers and we hook vb2_dma_contig_memops. 
> > I don't know why that dma-contig driver do not reject it.
> > I will try to figure it out.
> 
> Is there an iommu involved that turns the scatter-gather list into what looks like
> contiguous memory for the DMA?
> 
Yes, We have iommu that could make scatter-gather list looks like
contiguous memory.

> At the end of vb2_dc_get_userptr() in videobuf2-dma-contig.c there is a check
> 'if (contig_size < size)' that verifies that the sg DMA is contiguous. This would
> work if there is an iommu involved (if I understand it correctly).
> 
I see. We saw this error before we add iommu support.

> If that's the case, then it's OK to keep VB2_USERPTR because you have real sg
> support (although not via the DMA engine, but via iommu mappings).
> 
Got it. We will keep VB2_USERPTR.

> Regards,
> 
> 	Hans

^ permalink raw reply	[flat|nested] 102+ messages in thread

* [PATCH v4 5/8] [Media] vcodec: mediatek: Add Mediatek V4L2 Video Encoder Driver
@ 2016-02-17  8:01                       ` tiffany lin
  0 siblings, 0 replies; 102+ messages in thread
From: tiffany lin @ 2016-02-17  8:01 UTC (permalink / raw)
  To: linux-arm-kernel

On Tue, 2016-02-16 at 14:48 +0100, Hans Verkuil wrote:
> Hi Tiffany,
> 
> >>>>> +int mtk_vcodec_enc_queue_init(void *priv, struct vb2_queue *src_vq,
> >>>>> +			   struct vb2_queue *dst_vq)
> >>>>> +{
> >>>>> +	struct mtk_vcodec_ctx *ctx = priv;
> >>>>> +	int ret;
> >>>>> +
> >>>>> +	src_vq->type		= V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE;
> >>>>> +	src_vq->io_modes	= VB2_DMABUF | VB2_MMAP | VB2_USERPTR;
> >>>>
> >>>> I recomment dropping VB2_USERPTR. That only makes sense for scatter-gather dma,
> >>>> and you use physically contiguous DMA.
> >>>>
> >>> Now our userspace app use VB2_USERPTR. I need to check if we could drop
> >>> VB2_USERPTR.
> >>> We use src_vq->mem_ops = &vb2_dma_contig_memops;
> >>> And there are
> >>> 	.get_userptr	= vb2_dc_get_userptr,
> >>> 	.put_userptr	= vb2_dc_put_userptr,
> >>> I was confused why it only make sense for scatter-gather.
> >>> Could you kindly explain more?
> >>
> >> VB2_USERPTR indicates that the application can use malloc to allocate buffers
> >> and pass those to the driver. Since malloc uses virtual memory the physical
> >> memory is scattered all over. And the first page typically does not start at
> >> the beginning of the page but at a random offset.
> >>
> >> To support that the DMA generally has to be able to do scatter-gather.
> >>
> >> Now, where things get ugly is that a hack was added to the USERPTR support where
> >> apps could pass a pointer to physically contiguous memory as a user pointer. This
> >> was a hack for embedded systems that preallocated a pool of buffers and needed to
> >> pass those pointers around somehow. So the dma-contig USERPTR support is for that
> >> 'feature'. If you try to pass a malloc()ed buffer to a dma-contig driver it will
> >> reject it. One big problem is that this specific hack isn't signaled anywhere, so
> >> applications have no way of knowing if the USERPTR support is the proper version
> >> or the hack where physically contiguous memory is expected.
> >>
> >> This hack has been replaced with DMABUF which is the proper way of passing buffers
> >> around.
> >>
> >> New dma-contig drivers should not use that old hack anymore. Use dmabuf to pass
> >> external buffers around.
> >>
> >> How do you use it in your app? With malloc()ed buffers? Or with 'special' pointers
> >> to physically contiguous buffers?
> >>
> > Understood now. Thanks for your explanation.
> > Now our app use malloc()ed buffers and we hook vb2_dma_contig_memops. 
> > I don't know why that dma-contig driver do not reject it.
> > I will try to figure it out.
> 
> Is there an iommu involved that turns the scatter-gather list into what looks like
> contiguous memory for the DMA?
> 
Yes, We have iommu that could make scatter-gather list looks like
contiguous memory.

> At the end of vb2_dc_get_userptr() in videobuf2-dma-contig.c there is a check
> 'if (contig_size < size)' that verifies that the sg DMA is contiguous. This would
> work if there is an iommu involved (if I understand it correctly).
> 
I see. We saw this error before we add iommu support.

> If that's the case, then it's OK to keep VB2_USERPTR because you have real sg
> support (although not via the DMA engine, but via iommu mappings).
> 
Got it. We will keep VB2_USERPTR.

> Regards,
> 
> 	Hans

^ permalink raw reply	[flat|nested] 102+ messages in thread

* Re: [PATCH v4 5/8] [Media] vcodec: mediatek: Add Mediatek V4L2 Video Encoder Driver
  2016-02-17  8:01                       ` tiffany lin
@ 2016-02-17  8:31                         ` Hans Verkuil
  -1 siblings, 0 replies; 102+ messages in thread
From: Hans Verkuil @ 2016-02-17  8:31 UTC (permalink / raw)
  To: tiffany lin, Hans Verkuil
  Cc: Hans Verkuil, daniel.thompson, Rob Herring,
	Mauro Carvalho Chehab, Matthias Brugger, Daniel Kurtz,
	Pawel Osciak, Eddie Huang, Yingjoe Chen, devicetree,
	linux-kernel, linux-arm-kernel, linux-media, linux-mediatek,
	PoChun.Lin, Andrew-CT Chen

On 02/17/16 09:01, tiffany lin wrote:
> On Tue, 2016-02-16 at 14:48 +0100, Hans Verkuil wrote:
>> Hi Tiffany,
>>
>>>>>>> +int mtk_vcodec_enc_queue_init(void *priv, struct vb2_queue *src_vq,
>>>>>>> +			   struct vb2_queue *dst_vq)
>>>>>>> +{
>>>>>>> +	struct mtk_vcodec_ctx *ctx = priv;
>>>>>>> +	int ret;
>>>>>>> +
>>>>>>> +	src_vq->type		= V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE;
>>>>>>> +	src_vq->io_modes	= VB2_DMABUF | VB2_MMAP | VB2_USERPTR;
>>>>>>
>>>>>> I recomment dropping VB2_USERPTR. That only makes sense for scatter-gather dma,
>>>>>> and you use physically contiguous DMA.
>>>>>>
>>>>> Now our userspace app use VB2_USERPTR. I need to check if we could drop
>>>>> VB2_USERPTR.
>>>>> We use src_vq->mem_ops = &vb2_dma_contig_memops;
>>>>> And there are
>>>>> 	.get_userptr	= vb2_dc_get_userptr,
>>>>> 	.put_userptr	= vb2_dc_put_userptr,
>>>>> I was confused why it only make sense for scatter-gather.
>>>>> Could you kindly explain more?
>>>>
>>>> VB2_USERPTR indicates that the application can use malloc to allocate buffers
>>>> and pass those to the driver. Since malloc uses virtual memory the physical
>>>> memory is scattered all over. And the first page typically does not start at
>>>> the beginning of the page but at a random offset.
>>>>
>>>> To support that the DMA generally has to be able to do scatter-gather.
>>>>
>>>> Now, where things get ugly is that a hack was added to the USERPTR support where
>>>> apps could pass a pointer to physically contiguous memory as a user pointer. This
>>>> was a hack for embedded systems that preallocated a pool of buffers and needed to
>>>> pass those pointers around somehow. So the dma-contig USERPTR support is for that
>>>> 'feature'. If you try to pass a malloc()ed buffer to a dma-contig driver it will
>>>> reject it. One big problem is that this specific hack isn't signaled anywhere, so
>>>> applications have no way of knowing if the USERPTR support is the proper version
>>>> or the hack where physically contiguous memory is expected.
>>>>
>>>> This hack has been replaced with DMABUF which is the proper way of passing buffers
>>>> around.
>>>>
>>>> New dma-contig drivers should not use that old hack anymore. Use dmabuf to pass
>>>> external buffers around.
>>>>
>>>> How do you use it in your app? With malloc()ed buffers? Or with 'special' pointers
>>>> to physically contiguous buffers?
>>>>
>>> Understood now. Thanks for your explanation.
>>> Now our app use malloc()ed buffers and we hook vb2_dma_contig_memops. 
>>> I don't know why that dma-contig driver do not reject it.
>>> I will try to figure it out.
>>
>> Is there an iommu involved that turns the scatter-gather list into what looks like
>> contiguous memory for the DMA?
>>
> Yes, We have iommu that could make scatter-gather list looks like
> contiguous memory.
> 
>> At the end of vb2_dc_get_userptr() in videobuf2-dma-contig.c there is a check
>> 'if (contig_size < size)' that verifies that the sg DMA is contiguous. This would
>> work if there is an iommu involved (if I understand it correctly).
>>
> I see. We saw this error before we add iommu support.
> 
>> If that's the case, then it's OK to keep VB2_USERPTR because you have real sg
>> support (although not via the DMA engine, but via iommu mappings).
>>
> Got it. We will keep VB2_USERPTR.

Can you add a comment here mentioning that VB2_USERPTR works with dma-contig because
there is an iommu? That should clarify this.

Regards,

	Hans

^ permalink raw reply	[flat|nested] 102+ messages in thread

* [PATCH v4 5/8] [Media] vcodec: mediatek: Add Mediatek V4L2 Video Encoder Driver
@ 2016-02-17  8:31                         ` Hans Verkuil
  0 siblings, 0 replies; 102+ messages in thread
From: Hans Verkuil @ 2016-02-17  8:31 UTC (permalink / raw)
  To: linux-arm-kernel

On 02/17/16 09:01, tiffany lin wrote:
> On Tue, 2016-02-16 at 14:48 +0100, Hans Verkuil wrote:
>> Hi Tiffany,
>>
>>>>>>> +int mtk_vcodec_enc_queue_init(void *priv, struct vb2_queue *src_vq,
>>>>>>> +			   struct vb2_queue *dst_vq)
>>>>>>> +{
>>>>>>> +	struct mtk_vcodec_ctx *ctx = priv;
>>>>>>> +	int ret;
>>>>>>> +
>>>>>>> +	src_vq->type		= V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE;
>>>>>>> +	src_vq->io_modes	= VB2_DMABUF | VB2_MMAP | VB2_USERPTR;
>>>>>>
>>>>>> I recomment dropping VB2_USERPTR. That only makes sense for scatter-gather dma,
>>>>>> and you use physically contiguous DMA.
>>>>>>
>>>>> Now our userspace app use VB2_USERPTR. I need to check if we could drop
>>>>> VB2_USERPTR.
>>>>> We use src_vq->mem_ops = &vb2_dma_contig_memops;
>>>>> And there are
>>>>> 	.get_userptr	= vb2_dc_get_userptr,
>>>>> 	.put_userptr	= vb2_dc_put_userptr,
>>>>> I was confused why it only make sense for scatter-gather.
>>>>> Could you kindly explain more?
>>>>
>>>> VB2_USERPTR indicates that the application can use malloc to allocate buffers
>>>> and pass those to the driver. Since malloc uses virtual memory the physical
>>>> memory is scattered all over. And the first page typically does not start at
>>>> the beginning of the page but at a random offset.
>>>>
>>>> To support that the DMA generally has to be able to do scatter-gather.
>>>>
>>>> Now, where things get ugly is that a hack was added to the USERPTR support where
>>>> apps could pass a pointer to physically contiguous memory as a user pointer. This
>>>> was a hack for embedded systems that preallocated a pool of buffers and needed to
>>>> pass those pointers around somehow. So the dma-contig USERPTR support is for that
>>>> 'feature'. If you try to pass a malloc()ed buffer to a dma-contig driver it will
>>>> reject it. One big problem is that this specific hack isn't signaled anywhere, so
>>>> applications have no way of knowing if the USERPTR support is the proper version
>>>> or the hack where physically contiguous memory is expected.
>>>>
>>>> This hack has been replaced with DMABUF which is the proper way of passing buffers
>>>> around.
>>>>
>>>> New dma-contig drivers should not use that old hack anymore. Use dmabuf to pass
>>>> external buffers around.
>>>>
>>>> How do you use it in your app? With malloc()ed buffers? Or with 'special' pointers
>>>> to physically contiguous buffers?
>>>>
>>> Understood now. Thanks for your explanation.
>>> Now our app use malloc()ed buffers and we hook vb2_dma_contig_memops. 
>>> I don't know why that dma-contig driver do not reject it.
>>> I will try to figure it out.
>>
>> Is there an iommu involved that turns the scatter-gather list into what looks like
>> contiguous memory for the DMA?
>>
> Yes, We have iommu that could make scatter-gather list looks like
> contiguous memory.
> 
>> At the end of vb2_dc_get_userptr() in videobuf2-dma-contig.c there is a check
>> 'if (contig_size < size)' that verifies that the sg DMA is contiguous. This would
>> work if there is an iommu involved (if I understand it correctly).
>>
> I see. We saw this error before we add iommu support.
> 
>> If that's the case, then it's OK to keep VB2_USERPTR because you have real sg
>> support (although not via the DMA engine, but via iommu mappings).
>>
> Got it. We will keep VB2_USERPTR.

Can you add a comment here mentioning that VB2_USERPTR works with dma-contig because
there is an iommu? That should clarify this.

Regards,

	Hans

^ permalink raw reply	[flat|nested] 102+ messages in thread

* Re: [PATCH v4 5/8] [Media] vcodec: mediatek: Add Mediatek V4L2 Video Encoder Driver
  2016-02-17  7:47                 ` Hans Verkuil
  (?)
@ 2016-02-17  8:33                   ` tiffany lin
  -1 siblings, 0 replies; 102+ messages in thread
From: tiffany lin @ 2016-02-17  8:33 UTC (permalink / raw)
  To: Hans Verkuil
  Cc: Hans Verkuil, daniel.thompson, Rob Herring,
	Mauro Carvalho Chehab, Matthias Brugger, Daniel Kurtz,
	Pawel Osciak, Eddie Huang, Yingjoe Chen, devicetree,
	linux-kernel, linux-arm-kernel, linux-media, linux-mediatek,
	PoChun.Lin, Andrew-CT Chen

Hi Hans,

On Wed, 2016-02-17 at 08:47 +0100, Hans Verkuil wrote:
> On 02/16/2016 07:37 AM, tiffany lin wrote:
> >>> +static int fops_vcodec_open(struct file *file)
> >>> +{
> >>> +	struct video_device *vfd = video_devdata(file);
> >>> +	struct mtk_vcodec_dev *dev = video_drvdata(file);
> >>> +	struct mtk_vcodec_ctx *ctx = NULL;
> >>> +	int ret = 0;
> >>> +
> >>> +	mutex_lock(&dev->dev_mutex);
> >>> +
> >>> +	ctx = devm_kzalloc(&dev->plat_dev->dev, sizeof(*ctx), GFP_KERNEL);
> >>> +	if (!ctx) {
> >>> +		ret = -ENOMEM;
> >>> +		goto err_alloc;
> >>> +	}
> >>> +
> >>> +	if (dev->num_instances >= MTK_VCODEC_MAX_ENCODER_INSTANCES) {
> >>> +		mtk_v4l2_err("Too many open contexts\n");
> >>> +		ret = -EBUSY;
> >>> +		goto err_no_ctx;
> >>
> >> Hmm. I never like it if you can't open a video node because of a reason like this.
> >>
> >> I.e. a simple 'v4l2-ctl -D' (i.e. calling QUERYCAP) should never fail.
> >>
> >> If there are hardware limitation that prevent more than X instances from running at
> >> the same time, then those limitations typically kick in when you start to stream
> >> (or possibly when calling REQBUFS). But before that it should always be possible to
> >> open the device.
> >>
> >> Having this check at open() is an indication of a poor design.
> >>
> >> Is this is a hardware limitation at all?
> >>
> > This is to make sure performance meet requirements, such as bitrate and
> > framerate.
> 
> Is it the driver's job to make enforce this? What if the application only
> deals with low-res video, but wants to encode a lot of those? Or is encoding
> a video off-line?
> 
> The driver generally doesn't know the use-case, so if this is an artificial
> limitation as opposed to a hardware limitation, then I would just drop this.
> 
We got your point, we will remove this artificial limitation in next
version.

best regards,
Tiffany


> Regards,
> 
> 	Hans
> 
> > We got your point. We will remove this and move limitation control to
> > start_streaming or REQBUFS.
> > Appreciated for your suggestion.:)
> > 
> > 
> >>> +	}
> 

^ permalink raw reply	[flat|nested] 102+ messages in thread

* Re: [PATCH v4 5/8] [Media] vcodec: mediatek: Add Mediatek V4L2 Video Encoder Driver
@ 2016-02-17  8:33                   ` tiffany lin
  0 siblings, 0 replies; 102+ messages in thread
From: tiffany lin @ 2016-02-17  8:33 UTC (permalink / raw)
  To: Hans Verkuil
  Cc: Hans Verkuil, daniel.thompson, Rob Herring,
	Mauro Carvalho Chehab, Matthias Brugger, Daniel Kurtz,
	Pawel Osciak, Eddie Huang, Yingjoe Chen, devicetree,
	linux-kernel, linux-arm-kernel, linux-media, linux-mediatek,
	PoChun.Lin, Andrew-CT Chen

Hi Hans,

On Wed, 2016-02-17 at 08:47 +0100, Hans Verkuil wrote:
> On 02/16/2016 07:37 AM, tiffany lin wrote:
> >>> +static int fops_vcodec_open(struct file *file)
> >>> +{
> >>> +	struct video_device *vfd = video_devdata(file);
> >>> +	struct mtk_vcodec_dev *dev = video_drvdata(file);
> >>> +	struct mtk_vcodec_ctx *ctx = NULL;
> >>> +	int ret = 0;
> >>> +
> >>> +	mutex_lock(&dev->dev_mutex);
> >>> +
> >>> +	ctx = devm_kzalloc(&dev->plat_dev->dev, sizeof(*ctx), GFP_KERNEL);
> >>> +	if (!ctx) {
> >>> +		ret = -ENOMEM;
> >>> +		goto err_alloc;
> >>> +	}
> >>> +
> >>> +	if (dev->num_instances >= MTK_VCODEC_MAX_ENCODER_INSTANCES) {
> >>> +		mtk_v4l2_err("Too many open contexts\n");
> >>> +		ret = -EBUSY;
> >>> +		goto err_no_ctx;
> >>
> >> Hmm. I never like it if you can't open a video node because of a reason like this.
> >>
> >> I.e. a simple 'v4l2-ctl -D' (i.e. calling QUERYCAP) should never fail.
> >>
> >> If there are hardware limitation that prevent more than X instances from running at
> >> the same time, then those limitations typically kick in when you start to stream
> >> (or possibly when calling REQBUFS). But before that it should always be possible to
> >> open the device.
> >>
> >> Having this check at open() is an indication of a poor design.
> >>
> >> Is this is a hardware limitation at all?
> >>
> > This is to make sure performance meet requirements, such as bitrate and
> > framerate.
> 
> Is it the driver's job to make enforce this? What if the application only
> deals with low-res video, but wants to encode a lot of those? Or is encoding
> a video off-line?
> 
> The driver generally doesn't know the use-case, so if this is an artificial
> limitation as opposed to a hardware limitation, then I would just drop this.
> 
We got your point, we will remove this artificial limitation in next
version.

best regards,
Tiffany


> Regards,
> 
> 	Hans
> 
> > We got your point. We will remove this and move limitation control to
> > start_streaming or REQBUFS.
> > Appreciated for your suggestion.:)
> > 
> > 
> >>> +	}
> 

^ permalink raw reply	[flat|nested] 102+ messages in thread

* [PATCH v4 5/8] [Media] vcodec: mediatek: Add Mediatek V4L2 Video Encoder Driver
@ 2016-02-17  8:33                   ` tiffany lin
  0 siblings, 0 replies; 102+ messages in thread
From: tiffany lin @ 2016-02-17  8:33 UTC (permalink / raw)
  To: linux-arm-kernel

Hi Hans,

On Wed, 2016-02-17 at 08:47 +0100, Hans Verkuil wrote:
> On 02/16/2016 07:37 AM, tiffany lin wrote:
> >>> +static int fops_vcodec_open(struct file *file)
> >>> +{
> >>> +	struct video_device *vfd = video_devdata(file);
> >>> +	struct mtk_vcodec_dev *dev = video_drvdata(file);
> >>> +	struct mtk_vcodec_ctx *ctx = NULL;
> >>> +	int ret = 0;
> >>> +
> >>> +	mutex_lock(&dev->dev_mutex);
> >>> +
> >>> +	ctx = devm_kzalloc(&dev->plat_dev->dev, sizeof(*ctx), GFP_KERNEL);
> >>> +	if (!ctx) {
> >>> +		ret = -ENOMEM;
> >>> +		goto err_alloc;
> >>> +	}
> >>> +
> >>> +	if (dev->num_instances >= MTK_VCODEC_MAX_ENCODER_INSTANCES) {
> >>> +		mtk_v4l2_err("Too many open contexts\n");
> >>> +		ret = -EBUSY;
> >>> +		goto err_no_ctx;
> >>
> >> Hmm. I never like it if you can't open a video node because of a reason like this.
> >>
> >> I.e. a simple 'v4l2-ctl -D' (i.e. calling QUERYCAP) should never fail.
> >>
> >> If there are hardware limitation that prevent more than X instances from running at
> >> the same time, then those limitations typically kick in when you start to stream
> >> (or possibly when calling REQBUFS). But before that it should always be possible to
> >> open the device.
> >>
> >> Having this check at open() is an indication of a poor design.
> >>
> >> Is this is a hardware limitation at all?
> >>
> > This is to make sure performance meet requirements, such as bitrate and
> > framerate.
> 
> Is it the driver's job to make enforce this? What if the application only
> deals with low-res video, but wants to encode a lot of those? Or is encoding
> a video off-line?
> 
> The driver generally doesn't know the use-case, so if this is an artificial
> limitation as opposed to a hardware limitation, then I would just drop this.
> 
We got your point, we will remove this artificial limitation in next
version.

best regards,
Tiffany


> Regards,
> 
> 	Hans
> 
> > We got your point. We will remove this and move limitation control to
> > start_streaming or REQBUFS.
> > Appreciated for your suggestion.:)
> > 
> > 
> >>> +	}
> 

^ permalink raw reply	[flat|nested] 102+ messages in thread

* Re: [PATCH v4 5/8] [Media] vcodec: mediatek: Add Mediatek V4L2 Video Encoder Driver
@ 2016-02-17  9:23                           ` tiffany lin
  0 siblings, 0 replies; 102+ messages in thread
From: tiffany lin @ 2016-02-17  9:23 UTC (permalink / raw)
  To: Hans Verkuil
  Cc: Hans Verkuil, Hans Verkuil, daniel.thompson, Rob Herring,
	Mauro Carvalho Chehab, Matthias Brugger, Daniel Kurtz,
	Pawel Osciak, Eddie Huang, Yingjoe Chen, devicetree,
	linux-kernel, linux-arm-kernel, linux-media, linux-mediatek,
	PoChun.Lin, Andrew-CT Chen

On Wed, 2016-02-17 at 09:31 +0100, Hans Verkuil wrote:
> On 02/17/16 09:01, tiffany lin wrote:
> > On Tue, 2016-02-16 at 14:48 +0100, Hans Verkuil wrote:
> >> Hi Tiffany,
> >>
> >>>>>>> +int mtk_vcodec_enc_queue_init(void *priv, struct vb2_queue *src_vq,
> >>>>>>> +			   struct vb2_queue *dst_vq)
> >>>>>>> +{
> >>>>>>> +	struct mtk_vcodec_ctx *ctx = priv;
> >>>>>>> +	int ret;
> >>>>>>> +
> >>>>>>> +	src_vq->type		= V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE;
> >>>>>>> +	src_vq->io_modes	= VB2_DMABUF | VB2_MMAP | VB2_USERPTR;
> >>>>>>
> >>>>>> I recomment dropping VB2_USERPTR. That only makes sense for scatter-gather dma,
> >>>>>> and you use physically contiguous DMA.
> >>>>>>
> >>>>> Now our userspace app use VB2_USERPTR. I need to check if we could drop
> >>>>> VB2_USERPTR.
> >>>>> We use src_vq->mem_ops = &vb2_dma_contig_memops;
> >>>>> And there are
> >>>>> 	.get_userptr	= vb2_dc_get_userptr,
> >>>>> 	.put_userptr	= vb2_dc_put_userptr,
> >>>>> I was confused why it only make sense for scatter-gather.
> >>>>> Could you kindly explain more?
> >>>>
> >>>> VB2_USERPTR indicates that the application can use malloc to allocate buffers
> >>>> and pass those to the driver. Since malloc uses virtual memory the physical
> >>>> memory is scattered all over. And the first page typically does not start at
> >>>> the beginning of the page but at a random offset.
> >>>>
> >>>> To support that the DMA generally has to be able to do scatter-gather.
> >>>>
> >>>> Now, where things get ugly is that a hack was added to the USERPTR support where
> >>>> apps could pass a pointer to physically contiguous memory as a user pointer. This
> >>>> was a hack for embedded systems that preallocated a pool of buffers and needed to
> >>>> pass those pointers around somehow. So the dma-contig USERPTR support is for that
> >>>> 'feature'. If you try to pass a malloc()ed buffer to a dma-contig driver it will
> >>>> reject it. One big problem is that this specific hack isn't signaled anywhere, so
> >>>> applications have no way of knowing if the USERPTR support is the proper version
> >>>> or the hack where physically contiguous memory is expected.
> >>>>
> >>>> This hack has been replaced with DMABUF which is the proper way of passing buffers
> >>>> around.
> >>>>
> >>>> New dma-contig drivers should not use that old hack anymore. Use dmabuf to pass
> >>>> external buffers around.
> >>>>
> >>>> How do you use it in your app? With malloc()ed buffers? Or with 'special' pointers
> >>>> to physically contiguous buffers?
> >>>>
> >>> Understood now. Thanks for your explanation.
> >>> Now our app use malloc()ed buffers and we hook vb2_dma_contig_memops. 
> >>> I don't know why that dma-contig driver do not reject it.
> >>> I will try to figure it out.
> >>
> >> Is there an iommu involved that turns the scatter-gather list into what looks like
> >> contiguous memory for the DMA?
> >>
> > Yes, We have iommu that could make scatter-gather list looks like
> > contiguous memory.
> > 
> >> At the end of vb2_dc_get_userptr() in videobuf2-dma-contig.c there is a check
> >> 'if (contig_size < size)' that verifies that the sg DMA is contiguous. This would
> >> work if there is an iommu involved (if I understand it correctly).
> >>
> > I see. We saw this error before we add iommu support.
> > 
> >> If that's the case, then it's OK to keep VB2_USERPTR because you have real sg
> >> support (although not via the DMA engine, but via iommu mappings).
> >>
> > Got it. We will keep VB2_USERPTR.
> 
> Can you add a comment here mentioning that VB2_USERPTR works with dma-contig because
> there is an iommu? That should clarify this.
> 
Got it. I will add comment before this line.
src_vq->io_modes = VB2_DMABUF | VB2_MMAP | VB2_USERPTR;.

I will also add out IOMMU patches information in cover letter, like:
The following patches are needed to support VB2_USERPTR works with
dma-contig.
https://patchwork.kernel.org/patch/8335461/
https://patchwork.kernel.org/patch/8335471/
https://patchwork.kernel.org/patch/8335481/
https://patchwork.kernel.org/patch/8335491/
https://patchwork.kernel.org/patch/8335501/
https://patchwork.kernel.org/patch/7596181/


best regards,
Tiffany



> Regards,
> 
> 	Hans

^ permalink raw reply	[flat|nested] 102+ messages in thread

* Re: [PATCH v4 5/8] [Media] vcodec: mediatek: Add Mediatek V4L2 Video Encoder Driver
@ 2016-02-17  9:23                           ` tiffany lin
  0 siblings, 0 replies; 102+ messages in thread
From: tiffany lin @ 2016-02-17  9:23 UTC (permalink / raw)
  To: Hans Verkuil
  Cc: Hans Verkuil, Hans Verkuil,
	daniel.thompson-QSEj5FYQhm4dnm+yROfE0A, Rob Herring,
	Mauro Carvalho Chehab, Matthias Brugger, Daniel Kurtz,
	Pawel Osciak, Eddie Huang, Yingjoe Chen,
	devicetree-u79uwXL29TY76Z2rM5mHXA,
	linux-kernel-u79uwXL29TY76Z2rM5mHXA,
	linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
	linux-media-u79uwXL29TY76Z2rM5mHXA,
	linux-mediatek-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
	PoChun.Lin-NuS5LvNUpcJWk0Htik3J/w, Andrew-CT Chen

On Wed, 2016-02-17 at 09:31 +0100, Hans Verkuil wrote:
> On 02/17/16 09:01, tiffany lin wrote:
> > On Tue, 2016-02-16 at 14:48 +0100, Hans Verkuil wrote:
> >> Hi Tiffany,
> >>
> >>>>>>> +int mtk_vcodec_enc_queue_init(void *priv, struct vb2_queue *src_vq,
> >>>>>>> +			   struct vb2_queue *dst_vq)
> >>>>>>> +{
> >>>>>>> +	struct mtk_vcodec_ctx *ctx = priv;
> >>>>>>> +	int ret;
> >>>>>>> +
> >>>>>>> +	src_vq->type		= V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE;
> >>>>>>> +	src_vq->io_modes	= VB2_DMABUF | VB2_MMAP | VB2_USERPTR;
> >>>>>>
> >>>>>> I recomment dropping VB2_USERPTR. That only makes sense for scatter-gather dma,
> >>>>>> and you use physically contiguous DMA.
> >>>>>>
> >>>>> Now our userspace app use VB2_USERPTR. I need to check if we could drop
> >>>>> VB2_USERPTR.
> >>>>> We use src_vq->mem_ops = &vb2_dma_contig_memops;
> >>>>> And there are
> >>>>> 	.get_userptr	= vb2_dc_get_userptr,
> >>>>> 	.put_userptr	= vb2_dc_put_userptr,
> >>>>> I was confused why it only make sense for scatter-gather.
> >>>>> Could you kindly explain more?
> >>>>
> >>>> VB2_USERPTR indicates that the application can use malloc to allocate buffers
> >>>> and pass those to the driver. Since malloc uses virtual memory the physical
> >>>> memory is scattered all over. And the first page typically does not start at
> >>>> the beginning of the page but at a random offset.
> >>>>
> >>>> To support that the DMA generally has to be able to do scatter-gather.
> >>>>
> >>>> Now, where things get ugly is that a hack was added to the USERPTR support where
> >>>> apps could pass a pointer to physically contiguous memory as a user pointer. This
> >>>> was a hack for embedded systems that preallocated a pool of buffers and needed to
> >>>> pass those pointers around somehow. So the dma-contig USERPTR support is for that
> >>>> 'feature'. If you try to pass a malloc()ed buffer to a dma-contig driver it will
> >>>> reject it. One big problem is that this specific hack isn't signaled anywhere, so
> >>>> applications have no way of knowing if the USERPTR support is the proper version
> >>>> or the hack where physically contiguous memory is expected.
> >>>>
> >>>> This hack has been replaced with DMABUF which is the proper way of passing buffers
> >>>> around.
> >>>>
> >>>> New dma-contig drivers should not use that old hack anymore. Use dmabuf to pass
> >>>> external buffers around.
> >>>>
> >>>> How do you use it in your app? With malloc()ed buffers? Or with 'special' pointers
> >>>> to physically contiguous buffers?
> >>>>
> >>> Understood now. Thanks for your explanation.
> >>> Now our app use malloc()ed buffers and we hook vb2_dma_contig_memops. 
> >>> I don't know why that dma-contig driver do not reject it.
> >>> I will try to figure it out.
> >>
> >> Is there an iommu involved that turns the scatter-gather list into what looks like
> >> contiguous memory for the DMA?
> >>
> > Yes, We have iommu that could make scatter-gather list looks like
> > contiguous memory.
> > 
> >> At the end of vb2_dc_get_userptr() in videobuf2-dma-contig.c there is a check
> >> 'if (contig_size < size)' that verifies that the sg DMA is contiguous. This would
> >> work if there is an iommu involved (if I understand it correctly).
> >>
> > I see. We saw this error before we add iommu support.
> > 
> >> If that's the case, then it's OK to keep VB2_USERPTR because you have real sg
> >> support (although not via the DMA engine, but via iommu mappings).
> >>
> > Got it. We will keep VB2_USERPTR.
> 
> Can you add a comment here mentioning that VB2_USERPTR works with dma-contig because
> there is an iommu? That should clarify this.
> 
Got it. I will add comment before this line.
src_vq->io_modes = VB2_DMABUF | VB2_MMAP | VB2_USERPTR;.

I will also add out IOMMU patches information in cover letter, like:
The following patches are needed to support VB2_USERPTR works with
dma-contig.
https://patchwork.kernel.org/patch/8335461/
https://patchwork.kernel.org/patch/8335471/
https://patchwork.kernel.org/patch/8335481/
https://patchwork.kernel.org/patch/8335491/
https://patchwork.kernel.org/patch/8335501/
https://patchwork.kernel.org/patch/7596181/


best regards,
Tiffany



> Regards,
> 
> 	Hans


--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 102+ messages in thread

* [PATCH v4 5/8] [Media] vcodec: mediatek: Add Mediatek V4L2 Video Encoder Driver
@ 2016-02-17  9:23                           ` tiffany lin
  0 siblings, 0 replies; 102+ messages in thread
From: tiffany lin @ 2016-02-17  9:23 UTC (permalink / raw)
  To: linux-arm-kernel

On Wed, 2016-02-17 at 09:31 +0100, Hans Verkuil wrote:
> On 02/17/16 09:01, tiffany lin wrote:
> > On Tue, 2016-02-16 at 14:48 +0100, Hans Verkuil wrote:
> >> Hi Tiffany,
> >>
> >>>>>>> +int mtk_vcodec_enc_queue_init(void *priv, struct vb2_queue *src_vq,
> >>>>>>> +			   struct vb2_queue *dst_vq)
> >>>>>>> +{
> >>>>>>> +	struct mtk_vcodec_ctx *ctx = priv;
> >>>>>>> +	int ret;
> >>>>>>> +
> >>>>>>> +	src_vq->type		= V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE;
> >>>>>>> +	src_vq->io_modes	= VB2_DMABUF | VB2_MMAP | VB2_USERPTR;
> >>>>>>
> >>>>>> I recomment dropping VB2_USERPTR. That only makes sense for scatter-gather dma,
> >>>>>> and you use physically contiguous DMA.
> >>>>>>
> >>>>> Now our userspace app use VB2_USERPTR. I need to check if we could drop
> >>>>> VB2_USERPTR.
> >>>>> We use src_vq->mem_ops = &vb2_dma_contig_memops;
> >>>>> And there are
> >>>>> 	.get_userptr	= vb2_dc_get_userptr,
> >>>>> 	.put_userptr	= vb2_dc_put_userptr,
> >>>>> I was confused why it only make sense for scatter-gather.
> >>>>> Could you kindly explain more?
> >>>>
> >>>> VB2_USERPTR indicates that the application can use malloc to allocate buffers
> >>>> and pass those to the driver. Since malloc uses virtual memory the physical
> >>>> memory is scattered all over. And the first page typically does not start at
> >>>> the beginning of the page but at a random offset.
> >>>>
> >>>> To support that the DMA generally has to be able to do scatter-gather.
> >>>>
> >>>> Now, where things get ugly is that a hack was added to the USERPTR support where
> >>>> apps could pass a pointer to physically contiguous memory as a user pointer. This
> >>>> was a hack for embedded systems that preallocated a pool of buffers and needed to
> >>>> pass those pointers around somehow. So the dma-contig USERPTR support is for that
> >>>> 'feature'. If you try to pass a malloc()ed buffer to a dma-contig driver it will
> >>>> reject it. One big problem is that this specific hack isn't signaled anywhere, so
> >>>> applications have no way of knowing if the USERPTR support is the proper version
> >>>> or the hack where physically contiguous memory is expected.
> >>>>
> >>>> This hack has been replaced with DMABUF which is the proper way of passing buffers
> >>>> around.
> >>>>
> >>>> New dma-contig drivers should not use that old hack anymore. Use dmabuf to pass
> >>>> external buffers around.
> >>>>
> >>>> How do you use it in your app? With malloc()ed buffers? Or with 'special' pointers
> >>>> to physically contiguous buffers?
> >>>>
> >>> Understood now. Thanks for your explanation.
> >>> Now our app use malloc()ed buffers and we hook vb2_dma_contig_memops. 
> >>> I don't know why that dma-contig driver do not reject it.
> >>> I will try to figure it out.
> >>
> >> Is there an iommu involved that turns the scatter-gather list into what looks like
> >> contiguous memory for the DMA?
> >>
> > Yes, We have iommu that could make scatter-gather list looks like
> > contiguous memory.
> > 
> >> At the end of vb2_dc_get_userptr() in videobuf2-dma-contig.c there is a check
> >> 'if (contig_size < size)' that verifies that the sg DMA is contiguous. This would
> >> work if there is an iommu involved (if I understand it correctly).
> >>
> > I see. We saw this error before we add iommu support.
> > 
> >> If that's the case, then it's OK to keep VB2_USERPTR because you have real sg
> >> support (although not via the DMA engine, but via iommu mappings).
> >>
> > Got it. We will keep VB2_USERPTR.
> 
> Can you add a comment here mentioning that VB2_USERPTR works with dma-contig because
> there is an iommu? That should clarify this.
> 
Got it. I will add comment before this line.
src_vq->io_modes = VB2_DMABUF | VB2_MMAP | VB2_USERPTR;.

I will also add out IOMMU patches information in cover letter, like:
The following patches are needed to support VB2_USERPTR works with
dma-contig.
https://patchwork.kernel.org/patch/8335461/
https://patchwork.kernel.org/patch/8335471/
https://patchwork.kernel.org/patch/8335481/
https://patchwork.kernel.org/patch/8335491/
https://patchwork.kernel.org/patch/8335501/
https://patchwork.kernel.org/patch/7596181/


best regards,
Tiffany



> Regards,
> 
> 	Hans

^ permalink raw reply	[flat|nested] 102+ messages in thread

* Re: [PATCH v4 5/8] [Media] vcodec: mediatek: Add Mediatek V4L2 Video Encoder Driver
@ 2016-02-20  9:11                   ` tiffany lin
  0 siblings, 0 replies; 102+ messages in thread
From: tiffany lin @ 2016-02-20  9:11 UTC (permalink / raw)
  To: Hans Verkuil
  Cc: Hans Verkuil, daniel.thompson, Rob Herring,
	Mauro Carvalho Chehab, Matthias Brugger, Daniel Kurtz,
	Pawel Osciak, Eddie Huang, Yingjoe Chen, devicetree,
	linux-kernel, linux-arm-kernel, linux-media, linux-mediatek,
	PoChun.Lin, Andrew-CT Chen

Hi Hans,

On Tue, 2016-02-16 at 08:44 +0100, Hans Verkuil wrote:
> On 02/16/2016 07:37 AM, tiffany lin wrote:
> >>> +
> >>> +const struct v4l2_ioctl_ops mtk_venc_ioctl_ops = {
> >>> +	.vidioc_streamon		= v4l2_m2m_ioctl_streamon,
> >>> +	.vidioc_streamoff		= v4l2_m2m_ioctl_streamoff,
> >>> +
> >>> +	.vidioc_reqbufs			= v4l2_m2m_ioctl_reqbufs,
> >>> +	.vidioc_querybuf		= v4l2_m2m_ioctl_querybuf,
> >>> +	.vidioc_qbuf			= vidioc_venc_qbuf,
> >>> +	.vidioc_dqbuf			= vidioc_venc_dqbuf,
> >>> +
> >>> +	.vidioc_querycap		= vidioc_venc_querycap,
> >>> +	.vidioc_enum_fmt_vid_cap_mplane = vidioc_enum_fmt_vid_cap_mplane,
> >>> +	.vidioc_enum_fmt_vid_out_mplane = vidioc_enum_fmt_vid_out_mplane,
> >>> +	.vidioc_enum_framesizes		= vidioc_enum_framesizes,
> >>> +
> >>> +	.vidioc_try_fmt_vid_cap_mplane	= vidioc_try_fmt_vid_cap_mplane,
> >>> +	.vidioc_try_fmt_vid_out_mplane	= vidioc_try_fmt_vid_out_mplane,
> >>> +	.vidioc_expbuf			= v4l2_m2m_ioctl_expbuf,
> >>
> >> Please add vidioc_create_bufs and vidioc_prepare_buf as well.
> >>
> > 
> > Currently we do not support these use cases, do we need to add
> > vidioc_create_bufs and vidioc_prepare_buf now?
> 
> I would suggest you do. The vb2 framework gives it (almost) for free.
> prepare_buf is completely free (just add the helper) and create_bufs
> needs a few small changes in the queue_setup function, that's all.
> 
After try to add vidioc_create_bufs directly using
vb2_ioctl_create_bufs, it will have problem in 
	int res = vb2_verify_memory_type(vdev->queue, p->memory,
			p->format.type);
We do not init our video_device queue in device probe function.

Our vb2_queues for OUTPUT and CAPTURE are initialized in
v4l2_m2m_ctx_init when ctx instance open.
What is queue in video_device for?
If we should init vdev->queue in probe function, this queue format
should be CAPTURE queue or OUTPUT queue?

best regards,
Tiffany

> > 
> > 
> >>> +	.vidioc_subscribe_event		= v4l2_ctrl_subscribe_event,
> >>> +	.vidioc_unsubscribe_event	= v4l2_event_unsubscribe,
> >>> +
> >>> +	.vidioc_s_parm			= vidioc_venc_s_parm,
> >>> +
> >>> +	.vidioc_s_fmt_vid_cap_mplane	= vidioc_venc_s_fmt,
> >>> +	.vidioc_s_fmt_vid_out_mplane	= vidioc_venc_s_fmt,
> >>> +
> >>> +	.vidioc_g_fmt_vid_cap_mplane	= vidioc_venc_g_fmt,
> >>> +	.vidioc_g_fmt_vid_out_mplane	= vidioc_venc_g_fmt,
> >>> +
> >>> +	.vidioc_g_selection		= vidioc_venc_g_s_selection,
> >>> +	.vidioc_s_selection		= vidioc_venc_g_s_selection,
> >>> +};
> 
> <snip>
> 
> >>> +int mtk_vcodec_enc_queue_init(void *priv, struct vb2_queue *src_vq,
> >>> +			   struct vb2_queue *dst_vq)
> >>> +{
> >>> +	struct mtk_vcodec_ctx *ctx = priv;
> >>> +	int ret;
> >>> +
> >>> +	src_vq->type		= V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE;
> >>> +	src_vq->io_modes	= VB2_DMABUF | VB2_MMAP | VB2_USERPTR;
> >>
> >> I recomment dropping VB2_USERPTR. That only makes sense for scatter-gather dma,
> >> and you use physically contiguous DMA.
> >>
> > Now our userspace app use VB2_USERPTR. I need to check if we could drop
> > VB2_USERPTR.
> > We use src_vq->mem_ops = &vb2_dma_contig_memops;
> > And there are
> > 	.get_userptr	= vb2_dc_get_userptr,
> > 	.put_userptr	= vb2_dc_put_userptr,
> > I was confused why it only make sense for scatter-gather.
> > Could you kindly explain more?
> 
> VB2_USERPTR indicates that the application can use malloc to allocate buffers
> and pass those to the driver. Since malloc uses virtual memory the physical
> memory is scattered all over. And the first page typically does not start at
> the beginning of the page but at a random offset.
> 
> To support that the DMA generally has to be able to do scatter-gather.
> 
> Now, where things get ugly is that a hack was added to the USERPTR support where
> apps could pass a pointer to physically contiguous memory as a user pointer. This
> was a hack for embedded systems that preallocated a pool of buffers and needed to
> pass those pointers around somehow. So the dma-contig USERPTR support is for that
> 'feature'. If you try to pass a malloc()ed buffer to a dma-contig driver it will
> reject it. One big problem is that this specific hack isn't signaled anywhere, so
> applications have no way of knowing if the USERPTR support is the proper version
> or the hack where physically contiguous memory is expected.
> 
> This hack has been replaced with DMABUF which is the proper way of passing buffers
> around.
> 
> New dma-contig drivers should not use that old hack anymore. Use dmabuf to pass
> external buffers around.
> 
> How do you use it in your app? With malloc()ed buffers? Or with 'special' pointers
> to physically contiguous buffers?
> 
> > 
> >>> +	src_vq->drv_priv	= ctx;
> >>> +	src_vq->buf_struct_size = sizeof(struct mtk_video_enc_buf);
> >>> +	src_vq->ops		= &mtk_venc_vb2_ops;
> >>> +	src_vq->mem_ops		= &vb2_dma_contig_memops;
> >>> +	src_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
> >>> +	src_vq->lock = &ctx->dev->dev_mutex;
> >>> +
> >>> +	ret = vb2_queue_init(src_vq);
> >>> +	if (ret)
> >>> +		return ret;
> >>> +
> >>> +	dst_vq->type		= V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
> >>> +	dst_vq->io_modes	= VB2_DMABUF | VB2_MMAP | VB2_USERPTR;
> >>> +	dst_vq->drv_priv	= ctx;
> >>> +	dst_vq->buf_struct_size = sizeof(struct v4l2_m2m_buffer);
> >>> +	dst_vq->ops		= &mtk_venc_vb2_ops;
> >>> +	dst_vq->mem_ops		= &vb2_dma_contig_memops;
> >>> +	dst_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
> >>> +	dst_vq->lock = &ctx->dev->dev_mutex;
> >>> +
> >>> +	return vb2_queue_init(dst_vq);
> >>> +}
> 
> Regards,
> 
> 	Hans
> 

^ permalink raw reply	[flat|nested] 102+ messages in thread

* Re: [PATCH v4 5/8] [Media] vcodec: mediatek: Add Mediatek V4L2 Video Encoder Driver
@ 2016-02-20  9:11                   ` tiffany lin
  0 siblings, 0 replies; 102+ messages in thread
From: tiffany lin @ 2016-02-20  9:11 UTC (permalink / raw)
  To: Hans Verkuil
  Cc: devicetree-u79uwXL29TY76Z2rM5mHXA,
	daniel.thompson-QSEj5FYQhm4dnm+yROfE0A, Mauro Carvalho Chehab,
	linux-mediatek-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
	linux-kernel-u79uwXL29TY76Z2rM5mHXA,
	PoChun.Lin-NuS5LvNUpcJWk0Htik3J/w, Rob Herring, Hans Verkuil,
	linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
	Andrew-CT Chen, Matthias Brugger, Yingjoe Chen, Eddie Huang,
	Pawel Osciak, linux-media-u79uwXL29TY76Z2rM5mHXA

Hi Hans,

On Tue, 2016-02-16 at 08:44 +0100, Hans Verkuil wrote:
> On 02/16/2016 07:37 AM, tiffany lin wrote:
> >>> +
> >>> +const struct v4l2_ioctl_ops mtk_venc_ioctl_ops = {
> >>> +	.vidioc_streamon		= v4l2_m2m_ioctl_streamon,
> >>> +	.vidioc_streamoff		= v4l2_m2m_ioctl_streamoff,
> >>> +
> >>> +	.vidioc_reqbufs			= v4l2_m2m_ioctl_reqbufs,
> >>> +	.vidioc_querybuf		= v4l2_m2m_ioctl_querybuf,
> >>> +	.vidioc_qbuf			= vidioc_venc_qbuf,
> >>> +	.vidioc_dqbuf			= vidioc_venc_dqbuf,
> >>> +
> >>> +	.vidioc_querycap		= vidioc_venc_querycap,
> >>> +	.vidioc_enum_fmt_vid_cap_mplane = vidioc_enum_fmt_vid_cap_mplane,
> >>> +	.vidioc_enum_fmt_vid_out_mplane = vidioc_enum_fmt_vid_out_mplane,
> >>> +	.vidioc_enum_framesizes		= vidioc_enum_framesizes,
> >>> +
> >>> +	.vidioc_try_fmt_vid_cap_mplane	= vidioc_try_fmt_vid_cap_mplane,
> >>> +	.vidioc_try_fmt_vid_out_mplane	= vidioc_try_fmt_vid_out_mplane,
> >>> +	.vidioc_expbuf			= v4l2_m2m_ioctl_expbuf,
> >>
> >> Please add vidioc_create_bufs and vidioc_prepare_buf as well.
> >>
> > 
> > Currently we do not support these use cases, do we need to add
> > vidioc_create_bufs and vidioc_prepare_buf now?
> 
> I would suggest you do. The vb2 framework gives it (almost) for free.
> prepare_buf is completely free (just add the helper) and create_bufs
> needs a few small changes in the queue_setup function, that's all.
> 
After try to add vidioc_create_bufs directly using
vb2_ioctl_create_bufs, it will have problem in 
	int res = vb2_verify_memory_type(vdev->queue, p->memory,
			p->format.type);
We do not init our video_device queue in device probe function.

Our vb2_queues for OUTPUT and CAPTURE are initialized in
v4l2_m2m_ctx_init when ctx instance open.
What is queue in video_device for?
If we should init vdev->queue in probe function, this queue format
should be CAPTURE queue or OUTPUT queue?

best regards,
Tiffany

> > 
> > 
> >>> +	.vidioc_subscribe_event		= v4l2_ctrl_subscribe_event,
> >>> +	.vidioc_unsubscribe_event	= v4l2_event_unsubscribe,
> >>> +
> >>> +	.vidioc_s_parm			= vidioc_venc_s_parm,
> >>> +
> >>> +	.vidioc_s_fmt_vid_cap_mplane	= vidioc_venc_s_fmt,
> >>> +	.vidioc_s_fmt_vid_out_mplane	= vidioc_venc_s_fmt,
> >>> +
> >>> +	.vidioc_g_fmt_vid_cap_mplane	= vidioc_venc_g_fmt,
> >>> +	.vidioc_g_fmt_vid_out_mplane	= vidioc_venc_g_fmt,
> >>> +
> >>> +	.vidioc_g_selection		= vidioc_venc_g_s_selection,
> >>> +	.vidioc_s_selection		= vidioc_venc_g_s_selection,
> >>> +};
> 
> <snip>
> 
> >>> +int mtk_vcodec_enc_queue_init(void *priv, struct vb2_queue *src_vq,
> >>> +			   struct vb2_queue *dst_vq)
> >>> +{
> >>> +	struct mtk_vcodec_ctx *ctx = priv;
> >>> +	int ret;
> >>> +
> >>> +	src_vq->type		= V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE;
> >>> +	src_vq->io_modes	= VB2_DMABUF | VB2_MMAP | VB2_USERPTR;
> >>
> >> I recomment dropping VB2_USERPTR. That only makes sense for scatter-gather dma,
> >> and you use physically contiguous DMA.
> >>
> > Now our userspace app use VB2_USERPTR. I need to check if we could drop
> > VB2_USERPTR.
> > We use src_vq->mem_ops = &vb2_dma_contig_memops;
> > And there are
> > 	.get_userptr	= vb2_dc_get_userptr,
> > 	.put_userptr	= vb2_dc_put_userptr,
> > I was confused why it only make sense for scatter-gather.
> > Could you kindly explain more?
> 
> VB2_USERPTR indicates that the application can use malloc to allocate buffers
> and pass those to the driver. Since malloc uses virtual memory the physical
> memory is scattered all over. And the first page typically does not start at
> the beginning of the page but at a random offset.
> 
> To support that the DMA generally has to be able to do scatter-gather.
> 
> Now, where things get ugly is that a hack was added to the USERPTR support where
> apps could pass a pointer to physically contiguous memory as a user pointer. This
> was a hack for embedded systems that preallocated a pool of buffers and needed to
> pass those pointers around somehow. So the dma-contig USERPTR support is for that
> 'feature'. If you try to pass a malloc()ed buffer to a dma-contig driver it will
> reject it. One big problem is that this specific hack isn't signaled anywhere, so
> applications have no way of knowing if the USERPTR support is the proper version
> or the hack where physically contiguous memory is expected.
> 
> This hack has been replaced with DMABUF which is the proper way of passing buffers
> around.
> 
> New dma-contig drivers should not use that old hack anymore. Use dmabuf to pass
> external buffers around.
> 
> How do you use it in your app? With malloc()ed buffers? Or with 'special' pointers
> to physically contiguous buffers?
> 
> > 
> >>> +	src_vq->drv_priv	= ctx;
> >>> +	src_vq->buf_struct_size = sizeof(struct mtk_video_enc_buf);
> >>> +	src_vq->ops		= &mtk_venc_vb2_ops;
> >>> +	src_vq->mem_ops		= &vb2_dma_contig_memops;
> >>> +	src_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
> >>> +	src_vq->lock = &ctx->dev->dev_mutex;
> >>> +
> >>> +	ret = vb2_queue_init(src_vq);
> >>> +	if (ret)
> >>> +		return ret;
> >>> +
> >>> +	dst_vq->type		= V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
> >>> +	dst_vq->io_modes	= VB2_DMABUF | VB2_MMAP | VB2_USERPTR;
> >>> +	dst_vq->drv_priv	= ctx;
> >>> +	dst_vq->buf_struct_size = sizeof(struct v4l2_m2m_buffer);
> >>> +	dst_vq->ops		= &mtk_venc_vb2_ops;
> >>> +	dst_vq->mem_ops		= &vb2_dma_contig_memops;
> >>> +	dst_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
> >>> +	dst_vq->lock = &ctx->dev->dev_mutex;
> >>> +
> >>> +	return vb2_queue_init(dst_vq);
> >>> +}
> 
> Regards,
> 
> 	Hans
> 

^ permalink raw reply	[flat|nested] 102+ messages in thread

* [PATCH v4 5/8] [Media] vcodec: mediatek: Add Mediatek V4L2 Video Encoder Driver
@ 2016-02-20  9:11                   ` tiffany lin
  0 siblings, 0 replies; 102+ messages in thread
From: tiffany lin @ 2016-02-20  9:11 UTC (permalink / raw)
  To: linux-arm-kernel

Hi Hans,

On Tue, 2016-02-16 at 08:44 +0100, Hans Verkuil wrote:
> On 02/16/2016 07:37 AM, tiffany lin wrote:
> >>> +
> >>> +const struct v4l2_ioctl_ops mtk_venc_ioctl_ops = {
> >>> +	.vidioc_streamon		= v4l2_m2m_ioctl_streamon,
> >>> +	.vidioc_streamoff		= v4l2_m2m_ioctl_streamoff,
> >>> +
> >>> +	.vidioc_reqbufs			= v4l2_m2m_ioctl_reqbufs,
> >>> +	.vidioc_querybuf		= v4l2_m2m_ioctl_querybuf,
> >>> +	.vidioc_qbuf			= vidioc_venc_qbuf,
> >>> +	.vidioc_dqbuf			= vidioc_venc_dqbuf,
> >>> +
> >>> +	.vidioc_querycap		= vidioc_venc_querycap,
> >>> +	.vidioc_enum_fmt_vid_cap_mplane = vidioc_enum_fmt_vid_cap_mplane,
> >>> +	.vidioc_enum_fmt_vid_out_mplane = vidioc_enum_fmt_vid_out_mplane,
> >>> +	.vidioc_enum_framesizes		= vidioc_enum_framesizes,
> >>> +
> >>> +	.vidioc_try_fmt_vid_cap_mplane	= vidioc_try_fmt_vid_cap_mplane,
> >>> +	.vidioc_try_fmt_vid_out_mplane	= vidioc_try_fmt_vid_out_mplane,
> >>> +	.vidioc_expbuf			= v4l2_m2m_ioctl_expbuf,
> >>
> >> Please add vidioc_create_bufs and vidioc_prepare_buf as well.
> >>
> > 
> > Currently we do not support these use cases, do we need to add
> > vidioc_create_bufs and vidioc_prepare_buf now?
> 
> I would suggest you do. The vb2 framework gives it (almost) for free.
> prepare_buf is completely free (just add the helper) and create_bufs
> needs a few small changes in the queue_setup function, that's all.
> 
After try to add vidioc_create_bufs directly using
vb2_ioctl_create_bufs, it will have problem in 
	int res = vb2_verify_memory_type(vdev->queue, p->memory,
			p->format.type);
We do not init our video_device queue in device probe function.

Our vb2_queues for OUTPUT and CAPTURE are initialized in
v4l2_m2m_ctx_init when ctx instance open.
What is queue in video_device for?
If we should init vdev->queue in probe function, this queue format
should be CAPTURE queue or OUTPUT queue?

best regards,
Tiffany

> > 
> > 
> >>> +	.vidioc_subscribe_event		= v4l2_ctrl_subscribe_event,
> >>> +	.vidioc_unsubscribe_event	= v4l2_event_unsubscribe,
> >>> +
> >>> +	.vidioc_s_parm			= vidioc_venc_s_parm,
> >>> +
> >>> +	.vidioc_s_fmt_vid_cap_mplane	= vidioc_venc_s_fmt,
> >>> +	.vidioc_s_fmt_vid_out_mplane	= vidioc_venc_s_fmt,
> >>> +
> >>> +	.vidioc_g_fmt_vid_cap_mplane	= vidioc_venc_g_fmt,
> >>> +	.vidioc_g_fmt_vid_out_mplane	= vidioc_venc_g_fmt,
> >>> +
> >>> +	.vidioc_g_selection		= vidioc_venc_g_s_selection,
> >>> +	.vidioc_s_selection		= vidioc_venc_g_s_selection,
> >>> +};
> 
> <snip>
> 
> >>> +int mtk_vcodec_enc_queue_init(void *priv, struct vb2_queue *src_vq,
> >>> +			   struct vb2_queue *dst_vq)
> >>> +{
> >>> +	struct mtk_vcodec_ctx *ctx = priv;
> >>> +	int ret;
> >>> +
> >>> +	src_vq->type		= V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE;
> >>> +	src_vq->io_modes	= VB2_DMABUF | VB2_MMAP | VB2_USERPTR;
> >>
> >> I recomment dropping VB2_USERPTR. That only makes sense for scatter-gather dma,
> >> and you use physically contiguous DMA.
> >>
> > Now our userspace app use VB2_USERPTR. I need to check if we could drop
> > VB2_USERPTR.
> > We use src_vq->mem_ops = &vb2_dma_contig_memops;
> > And there are
> > 	.get_userptr	= vb2_dc_get_userptr,
> > 	.put_userptr	= vb2_dc_put_userptr,
> > I was confused why it only make sense for scatter-gather.
> > Could you kindly explain more?
> 
> VB2_USERPTR indicates that the application can use malloc to allocate buffers
> and pass those to the driver. Since malloc uses virtual memory the physical
> memory is scattered all over. And the first page typically does not start at
> the beginning of the page but at a random offset.
> 
> To support that the DMA generally has to be able to do scatter-gather.
> 
> Now, where things get ugly is that a hack was added to the USERPTR support where
> apps could pass a pointer to physically contiguous memory as a user pointer. This
> was a hack for embedded systems that preallocated a pool of buffers and needed to
> pass those pointers around somehow. So the dma-contig USERPTR support is for that
> 'feature'. If you try to pass a malloc()ed buffer to a dma-contig driver it will
> reject it. One big problem is that this specific hack isn't signaled anywhere, so
> applications have no way of knowing if the USERPTR support is the proper version
> or the hack where physically contiguous memory is expected.
> 
> This hack has been replaced with DMABUF which is the proper way of passing buffers
> around.
> 
> New dma-contig drivers should not use that old hack anymore. Use dmabuf to pass
> external buffers around.
> 
> How do you use it in your app? With malloc()ed buffers? Or with 'special' pointers
> to physically contiguous buffers?
> 
> > 
> >>> +	src_vq->drv_priv	= ctx;
> >>> +	src_vq->buf_struct_size = sizeof(struct mtk_video_enc_buf);
> >>> +	src_vq->ops		= &mtk_venc_vb2_ops;
> >>> +	src_vq->mem_ops		= &vb2_dma_contig_memops;
> >>> +	src_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
> >>> +	src_vq->lock = &ctx->dev->dev_mutex;
> >>> +
> >>> +	ret = vb2_queue_init(src_vq);
> >>> +	if (ret)
> >>> +		return ret;
> >>> +
> >>> +	dst_vq->type		= V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
> >>> +	dst_vq->io_modes	= VB2_DMABUF | VB2_MMAP | VB2_USERPTR;
> >>> +	dst_vq->drv_priv	= ctx;
> >>> +	dst_vq->buf_struct_size = sizeof(struct v4l2_m2m_buffer);
> >>> +	dst_vq->ops		= &mtk_venc_vb2_ops;
> >>> +	dst_vq->mem_ops		= &vb2_dma_contig_memops;
> >>> +	dst_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
> >>> +	dst_vq->lock = &ctx->dev->dev_mutex;
> >>> +
> >>> +	return vb2_queue_init(dst_vq);
> >>> +}
> 
> Regards,
> 
> 	Hans
> 

^ permalink raw reply	[flat|nested] 102+ messages in thread

* Re: [PATCH v4 5/8] [Media] vcodec: mediatek: Add Mediatek V4L2 Video Encoder Driver
  2016-02-20  9:11                   ` tiffany lin
  (?)
@ 2016-02-20  9:18                     ` Hans Verkuil
  -1 siblings, 0 replies; 102+ messages in thread
From: Hans Verkuil @ 2016-02-20  9:18 UTC (permalink / raw)
  To: tiffany lin
  Cc: Hans Verkuil, daniel.thompson, Rob Herring,
	Mauro Carvalho Chehab, Matthias Brugger, Daniel Kurtz,
	Pawel Osciak, Eddie Huang, Yingjoe Chen, devicetree,
	linux-kernel, linux-arm-kernel, linux-media, linux-mediatek,
	PoChun.Lin, Andrew-CT Chen

On 02/20/2016 10:11 AM, tiffany lin wrote:
> Hi Hans,
> 
> On Tue, 2016-02-16 at 08:44 +0100, Hans Verkuil wrote:
>> On 02/16/2016 07:37 AM, tiffany lin wrote:
>>>>> +
>>>>> +const struct v4l2_ioctl_ops mtk_venc_ioctl_ops = {
>>>>> +	.vidioc_streamon		= v4l2_m2m_ioctl_streamon,
>>>>> +	.vidioc_streamoff		= v4l2_m2m_ioctl_streamoff,
>>>>> +
>>>>> +	.vidioc_reqbufs			= v4l2_m2m_ioctl_reqbufs,
>>>>> +	.vidioc_querybuf		= v4l2_m2m_ioctl_querybuf,
>>>>> +	.vidioc_qbuf			= vidioc_venc_qbuf,
>>>>> +	.vidioc_dqbuf			= vidioc_venc_dqbuf,
>>>>> +
>>>>> +	.vidioc_querycap		= vidioc_venc_querycap,
>>>>> +	.vidioc_enum_fmt_vid_cap_mplane = vidioc_enum_fmt_vid_cap_mplane,
>>>>> +	.vidioc_enum_fmt_vid_out_mplane = vidioc_enum_fmt_vid_out_mplane,
>>>>> +	.vidioc_enum_framesizes		= vidioc_enum_framesizes,
>>>>> +
>>>>> +	.vidioc_try_fmt_vid_cap_mplane	= vidioc_try_fmt_vid_cap_mplane,
>>>>> +	.vidioc_try_fmt_vid_out_mplane	= vidioc_try_fmt_vid_out_mplane,
>>>>> +	.vidioc_expbuf			= v4l2_m2m_ioctl_expbuf,
>>>>
>>>> Please add vidioc_create_bufs and vidioc_prepare_buf as well.
>>>>
>>>
>>> Currently we do not support these use cases, do we need to add
>>> vidioc_create_bufs and vidioc_prepare_buf now?
>>
>> I would suggest you do. The vb2 framework gives it (almost) for free.
>> prepare_buf is completely free (just add the helper) and create_bufs
>> needs a few small changes in the queue_setup function, that's all.
>>
> After try to add vidioc_create_bufs directly using
> vb2_ioctl_create_bufs, it will have problem in 

This is a m2m device, so you should use the m2m variant of this:
v4l2_m2m_ioctl_create_bufs

That should solve this problem.

Ditto for prepare_buf: you need to use v4l2_m2m_ioctl_prepare_buf.

Regards,

	Hans

^ permalink raw reply	[flat|nested] 102+ messages in thread

* Re: [PATCH v4 5/8] [Media] vcodec: mediatek: Add Mediatek V4L2 Video Encoder Driver
@ 2016-02-20  9:18                     ` Hans Verkuil
  0 siblings, 0 replies; 102+ messages in thread
From: Hans Verkuil @ 2016-02-20  9:18 UTC (permalink / raw)
  To: tiffany lin
  Cc: Hans Verkuil, daniel.thompson-QSEj5FYQhm4dnm+yROfE0A,
	Rob Herring, Mauro Carvalho Chehab, Matthias Brugger,
	Daniel Kurtz, Pawel Osciak, Eddie Huang, Yingjoe Chen,
	devicetree-u79uwXL29TY76Z2rM5mHXA,
	linux-kernel-u79uwXL29TY76Z2rM5mHXA,
	linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
	linux-media-u79uwXL29TY76Z2rM5mHXA,
	linux-mediatek-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
	PoChun.Lin-NuS5LvNUpcJWk0Htik3J/w, Andrew-CT Chen

On 02/20/2016 10:11 AM, tiffany lin wrote:
> Hi Hans,
> 
> On Tue, 2016-02-16 at 08:44 +0100, Hans Verkuil wrote:
>> On 02/16/2016 07:37 AM, tiffany lin wrote:
>>>>> +
>>>>> +const struct v4l2_ioctl_ops mtk_venc_ioctl_ops = {
>>>>> +	.vidioc_streamon		= v4l2_m2m_ioctl_streamon,
>>>>> +	.vidioc_streamoff		= v4l2_m2m_ioctl_streamoff,
>>>>> +
>>>>> +	.vidioc_reqbufs			= v4l2_m2m_ioctl_reqbufs,
>>>>> +	.vidioc_querybuf		= v4l2_m2m_ioctl_querybuf,
>>>>> +	.vidioc_qbuf			= vidioc_venc_qbuf,
>>>>> +	.vidioc_dqbuf			= vidioc_venc_dqbuf,
>>>>> +
>>>>> +	.vidioc_querycap		= vidioc_venc_querycap,
>>>>> +	.vidioc_enum_fmt_vid_cap_mplane = vidioc_enum_fmt_vid_cap_mplane,
>>>>> +	.vidioc_enum_fmt_vid_out_mplane = vidioc_enum_fmt_vid_out_mplane,
>>>>> +	.vidioc_enum_framesizes		= vidioc_enum_framesizes,
>>>>> +
>>>>> +	.vidioc_try_fmt_vid_cap_mplane	= vidioc_try_fmt_vid_cap_mplane,
>>>>> +	.vidioc_try_fmt_vid_out_mplane	= vidioc_try_fmt_vid_out_mplane,
>>>>> +	.vidioc_expbuf			= v4l2_m2m_ioctl_expbuf,
>>>>
>>>> Please add vidioc_create_bufs and vidioc_prepare_buf as well.
>>>>
>>>
>>> Currently we do not support these use cases, do we need to add
>>> vidioc_create_bufs and vidioc_prepare_buf now?
>>
>> I would suggest you do. The vb2 framework gives it (almost) for free.
>> prepare_buf is completely free (just add the helper) and create_bufs
>> needs a few small changes in the queue_setup function, that's all.
>>
> After try to add vidioc_create_bufs directly using
> vb2_ioctl_create_bufs, it will have problem in 

This is a m2m device, so you should use the m2m variant of this:
v4l2_m2m_ioctl_create_bufs

That should solve this problem.

Ditto for prepare_buf: you need to use v4l2_m2m_ioctl_prepare_buf.

Regards,

	Hans
--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 102+ messages in thread

* [PATCH v4 5/8] [Media] vcodec: mediatek: Add Mediatek V4L2 Video Encoder Driver
@ 2016-02-20  9:18                     ` Hans Verkuil
  0 siblings, 0 replies; 102+ messages in thread
From: Hans Verkuil @ 2016-02-20  9:18 UTC (permalink / raw)
  To: linux-arm-kernel

On 02/20/2016 10:11 AM, tiffany lin wrote:
> Hi Hans,
> 
> On Tue, 2016-02-16 at 08:44 +0100, Hans Verkuil wrote:
>> On 02/16/2016 07:37 AM, tiffany lin wrote:
>>>>> +
>>>>> +const struct v4l2_ioctl_ops mtk_venc_ioctl_ops = {
>>>>> +	.vidioc_streamon		= v4l2_m2m_ioctl_streamon,
>>>>> +	.vidioc_streamoff		= v4l2_m2m_ioctl_streamoff,
>>>>> +
>>>>> +	.vidioc_reqbufs			= v4l2_m2m_ioctl_reqbufs,
>>>>> +	.vidioc_querybuf		= v4l2_m2m_ioctl_querybuf,
>>>>> +	.vidioc_qbuf			= vidioc_venc_qbuf,
>>>>> +	.vidioc_dqbuf			= vidioc_venc_dqbuf,
>>>>> +
>>>>> +	.vidioc_querycap		= vidioc_venc_querycap,
>>>>> +	.vidioc_enum_fmt_vid_cap_mplane = vidioc_enum_fmt_vid_cap_mplane,
>>>>> +	.vidioc_enum_fmt_vid_out_mplane = vidioc_enum_fmt_vid_out_mplane,
>>>>> +	.vidioc_enum_framesizes		= vidioc_enum_framesizes,
>>>>> +
>>>>> +	.vidioc_try_fmt_vid_cap_mplane	= vidioc_try_fmt_vid_cap_mplane,
>>>>> +	.vidioc_try_fmt_vid_out_mplane	= vidioc_try_fmt_vid_out_mplane,
>>>>> +	.vidioc_expbuf			= v4l2_m2m_ioctl_expbuf,
>>>>
>>>> Please add vidioc_create_bufs and vidioc_prepare_buf as well.
>>>>
>>>
>>> Currently we do not support these use cases, do we need to add
>>> vidioc_create_bufs and vidioc_prepare_buf now?
>>
>> I would suggest you do. The vb2 framework gives it (almost) for free.
>> prepare_buf is completely free (just add the helper) and create_bufs
>> needs a few small changes in the queue_setup function, that's all.
>>
> After try to add vidioc_create_bufs directly using
> vb2_ioctl_create_bufs, it will have problem in 

This is a m2m device, so you should use the m2m variant of this:
v4l2_m2m_ioctl_create_bufs

That should solve this problem.

Ditto for prepare_buf: you need to use v4l2_m2m_ioctl_prepare_buf.

Regards,

	Hans

^ permalink raw reply	[flat|nested] 102+ messages in thread

* Re: [PATCH v4 5/8] [Media] vcodec: mediatek: Add Mediatek V4L2 Video Encoder Driver
@ 2016-02-22 15:19                       ` tiffany lin
  0 siblings, 0 replies; 102+ messages in thread
From: tiffany lin @ 2016-02-22 15:19 UTC (permalink / raw)
  To: Hans Verkuil
  Cc: Hans Verkuil, daniel.thompson, Rob Herring,
	Mauro Carvalho Chehab, Matthias Brugger, Daniel Kurtz,
	Pawel Osciak, Eddie Huang, Yingjoe Chen, devicetree,
	linux-kernel, linux-arm-kernel, linux-media, linux-mediatek,
	PoChun.Lin, Andrew-CT Chen

Hi Hans,

On Sat, 2016-02-20 at 10:18 +0100, Hans Verkuil wrote:
> On 02/20/2016 10:11 AM, tiffany lin wrote:
> > Hi Hans,
> > 
> > On Tue, 2016-02-16 at 08:44 +0100, Hans Verkuil wrote:
> >> On 02/16/2016 07:37 AM, tiffany lin wrote:
> >>>>> +
> >>>>> +const struct v4l2_ioctl_ops mtk_venc_ioctl_ops = {
> >>>>> +	.vidioc_streamon		= v4l2_m2m_ioctl_streamon,
> >>>>> +	.vidioc_streamoff		= v4l2_m2m_ioctl_streamoff,
> >>>>> +
> >>>>> +	.vidioc_reqbufs			= v4l2_m2m_ioctl_reqbufs,
> >>>>> +	.vidioc_querybuf		= v4l2_m2m_ioctl_querybuf,
> >>>>> +	.vidioc_qbuf			= vidioc_venc_qbuf,
> >>>>> +	.vidioc_dqbuf			= vidioc_venc_dqbuf,
> >>>>> +
> >>>>> +	.vidioc_querycap		= vidioc_venc_querycap,
> >>>>> +	.vidioc_enum_fmt_vid_cap_mplane = vidioc_enum_fmt_vid_cap_mplane,
> >>>>> +	.vidioc_enum_fmt_vid_out_mplane = vidioc_enum_fmt_vid_out_mplane,
> >>>>> +	.vidioc_enum_framesizes		= vidioc_enum_framesizes,
> >>>>> +
> >>>>> +	.vidioc_try_fmt_vid_cap_mplane	= vidioc_try_fmt_vid_cap_mplane,
> >>>>> +	.vidioc_try_fmt_vid_out_mplane	= vidioc_try_fmt_vid_out_mplane,
> >>>>> +	.vidioc_expbuf			= v4l2_m2m_ioctl_expbuf,
> >>>>
> >>>> Please add vidioc_create_bufs and vidioc_prepare_buf as well.
> >>>>
> >>>
> >>> Currently we do not support these use cases, do we need to add
> >>> vidioc_create_bufs and vidioc_prepare_buf now?
> >>
> >> I would suggest you do. The vb2 framework gives it (almost) for free.
> >> prepare_buf is completely free (just add the helper) and create_bufs
> >> needs a few small changes in the queue_setup function, that's all.
> >>
> > After try to add vidioc_create_bufs directly using
> > vb2_ioctl_create_bufs, it will have problem in 
> 
> This is a m2m device, so you should use the m2m variant of this:
> v4l2_m2m_ioctl_create_bufs
> 
> That should solve this problem.
> 
> Ditto for prepare_buf: you need to use v4l2_m2m_ioctl_prepare_buf.
> 
Got it. After using v4l2_m2m_ioctl_create_bufs, the problem was solved.
Thanks for your help.

> Regards,
> 
> 	Hans

^ permalink raw reply	[flat|nested] 102+ messages in thread

* Re: [PATCH v4 5/8] [Media] vcodec: mediatek: Add Mediatek V4L2 Video Encoder Driver
@ 2016-02-22 15:19                       ` tiffany lin
  0 siblings, 0 replies; 102+ messages in thread
From: tiffany lin @ 2016-02-22 15:19 UTC (permalink / raw)
  To: Hans Verkuil
  Cc: Hans Verkuil, daniel.thompson-QSEj5FYQhm4dnm+yROfE0A,
	Rob Herring, Mauro Carvalho Chehab, Matthias Brugger,
	Daniel Kurtz, Pawel Osciak, Eddie Huang, Yingjoe Chen,
	devicetree-u79uwXL29TY76Z2rM5mHXA,
	linux-kernel-u79uwXL29TY76Z2rM5mHXA,
	linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
	linux-media-u79uwXL29TY76Z2rM5mHXA,
	linux-mediatek-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
	PoChun.Lin-NuS5LvNUpcJWk0Htik3J/w, Andrew-CT Chen

Hi Hans,

On Sat, 2016-02-20 at 10:18 +0100, Hans Verkuil wrote:
> On 02/20/2016 10:11 AM, tiffany lin wrote:
> > Hi Hans,
> > 
> > On Tue, 2016-02-16 at 08:44 +0100, Hans Verkuil wrote:
> >> On 02/16/2016 07:37 AM, tiffany lin wrote:
> >>>>> +
> >>>>> +const struct v4l2_ioctl_ops mtk_venc_ioctl_ops = {
> >>>>> +	.vidioc_streamon		= v4l2_m2m_ioctl_streamon,
> >>>>> +	.vidioc_streamoff		= v4l2_m2m_ioctl_streamoff,
> >>>>> +
> >>>>> +	.vidioc_reqbufs			= v4l2_m2m_ioctl_reqbufs,
> >>>>> +	.vidioc_querybuf		= v4l2_m2m_ioctl_querybuf,
> >>>>> +	.vidioc_qbuf			= vidioc_venc_qbuf,
> >>>>> +	.vidioc_dqbuf			= vidioc_venc_dqbuf,
> >>>>> +
> >>>>> +	.vidioc_querycap		= vidioc_venc_querycap,
> >>>>> +	.vidioc_enum_fmt_vid_cap_mplane = vidioc_enum_fmt_vid_cap_mplane,
> >>>>> +	.vidioc_enum_fmt_vid_out_mplane = vidioc_enum_fmt_vid_out_mplane,
> >>>>> +	.vidioc_enum_framesizes		= vidioc_enum_framesizes,
> >>>>> +
> >>>>> +	.vidioc_try_fmt_vid_cap_mplane	= vidioc_try_fmt_vid_cap_mplane,
> >>>>> +	.vidioc_try_fmt_vid_out_mplane	= vidioc_try_fmt_vid_out_mplane,
> >>>>> +	.vidioc_expbuf			= v4l2_m2m_ioctl_expbuf,
> >>>>
> >>>> Please add vidioc_create_bufs and vidioc_prepare_buf as well.
> >>>>
> >>>
> >>> Currently we do not support these use cases, do we need to add
> >>> vidioc_create_bufs and vidioc_prepare_buf now?
> >>
> >> I would suggest you do. The vb2 framework gives it (almost) for free.
> >> prepare_buf is completely free (just add the helper) and create_bufs
> >> needs a few small changes in the queue_setup function, that's all.
> >>
> > After try to add vidioc_create_bufs directly using
> > vb2_ioctl_create_bufs, it will have problem in 
> 
> This is a m2m device, so you should use the m2m variant of this:
> v4l2_m2m_ioctl_create_bufs
> 
> That should solve this problem.
> 
> Ditto for prepare_buf: you need to use v4l2_m2m_ioctl_prepare_buf.
> 
Got it. After using v4l2_m2m_ioctl_create_bufs, the problem was solved.
Thanks for your help.

> Regards,
> 
> 	Hans





--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 102+ messages in thread

* [PATCH v4 5/8] [Media] vcodec: mediatek: Add Mediatek V4L2 Video Encoder Driver
@ 2016-02-22 15:19                       ` tiffany lin
  0 siblings, 0 replies; 102+ messages in thread
From: tiffany lin @ 2016-02-22 15:19 UTC (permalink / raw)
  To: linux-arm-kernel

Hi Hans,

On Sat, 2016-02-20 at 10:18 +0100, Hans Verkuil wrote:
> On 02/20/2016 10:11 AM, tiffany lin wrote:
> > Hi Hans,
> > 
> > On Tue, 2016-02-16 at 08:44 +0100, Hans Verkuil wrote:
> >> On 02/16/2016 07:37 AM, tiffany lin wrote:
> >>>>> +
> >>>>> +const struct v4l2_ioctl_ops mtk_venc_ioctl_ops = {
> >>>>> +	.vidioc_streamon		= v4l2_m2m_ioctl_streamon,
> >>>>> +	.vidioc_streamoff		= v4l2_m2m_ioctl_streamoff,
> >>>>> +
> >>>>> +	.vidioc_reqbufs			= v4l2_m2m_ioctl_reqbufs,
> >>>>> +	.vidioc_querybuf		= v4l2_m2m_ioctl_querybuf,
> >>>>> +	.vidioc_qbuf			= vidioc_venc_qbuf,
> >>>>> +	.vidioc_dqbuf			= vidioc_venc_dqbuf,
> >>>>> +
> >>>>> +	.vidioc_querycap		= vidioc_venc_querycap,
> >>>>> +	.vidioc_enum_fmt_vid_cap_mplane = vidioc_enum_fmt_vid_cap_mplane,
> >>>>> +	.vidioc_enum_fmt_vid_out_mplane = vidioc_enum_fmt_vid_out_mplane,
> >>>>> +	.vidioc_enum_framesizes		= vidioc_enum_framesizes,
> >>>>> +
> >>>>> +	.vidioc_try_fmt_vid_cap_mplane	= vidioc_try_fmt_vid_cap_mplane,
> >>>>> +	.vidioc_try_fmt_vid_out_mplane	= vidioc_try_fmt_vid_out_mplane,
> >>>>> +	.vidioc_expbuf			= v4l2_m2m_ioctl_expbuf,
> >>>>
> >>>> Please add vidioc_create_bufs and vidioc_prepare_buf as well.
> >>>>
> >>>
> >>> Currently we do not support these use cases, do we need to add
> >>> vidioc_create_bufs and vidioc_prepare_buf now?
> >>
> >> I would suggest you do. The vb2 framework gives it (almost) for free.
> >> prepare_buf is completely free (just add the helper) and create_bufs
> >> needs a few small changes in the queue_setup function, that's all.
> >>
> > After try to add vidioc_create_bufs directly using
> > vb2_ioctl_create_bufs, it will have problem in 
> 
> This is a m2m device, so you should use the m2m variant of this:
> v4l2_m2m_ioctl_create_bufs
> 
> That should solve this problem.
> 
> Ditto for prepare_buf: you need to use v4l2_m2m_ioctl_prepare_buf.
> 
Got it. After using v4l2_m2m_ioctl_create_bufs, the problem was solved.
Thanks for your help.

> Regards,
> 
> 	Hans

^ permalink raw reply	[flat|nested] 102+ messages in thread

* Re: [PATCH v4 5/8] [Media] vcodec: mediatek: Add Mediatek V4L2 Video Encoder Driver
  2016-02-04 11:35           ` Tiffany Lin
                             ` (2 preceding siblings ...)
  (?)
@ 2016-02-23  5:46           ` Wu-Cheng Li (李務誠)
  2016-02-24  8:26               ` tiffany lin
  -1 siblings, 1 reply; 102+ messages in thread
From: Wu-Cheng Li (李務誠) @ 2016-02-23  5:46 UTC (permalink / raw)
  To: Tiffany Lin
  Cc: Hans Verkuil, Daniel Thompson, Rob Herring,
	Mauro Carvalho Chehab, Matthias Brugger, Daniel Kurtz,
	Pawel Osciak, Eddie Huang, Yingjoe Chen, devicetree,
	linux-kernel, linux-arm-kernel, linux-media, linux-mediatek,
	Lin PoChun, Andrew-CT Chen

On Thu, Feb 4, 2016 at 7:35 PM, Tiffany Lin <tiffany.lin@mediatek.com> wrote:
> From: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
>
> Add v4l2 layer encoder driver for MT8173
>
> Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
> ---
>  drivers/media/platform/Kconfig                     |   11 +
>  drivers/media/platform/Makefile                    |    2 +
>  drivers/media/platform/mtk-vcodec/Makefile         |    8 +
>  drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h |  388 ++++++
>  drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c | 1380 ++++++++++++++++++++
>  drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h |   46 +
>  .../media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c |  476 +++++++
>  .../media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c  |  132 ++
>  .../media/platform/mtk-vcodec/mtk_vcodec_intr.c    |  102 ++
>  .../media/platform/mtk-vcodec/mtk_vcodec_intr.h    |   29 +
>  drivers/media/platform/mtk-vcodec/mtk_vcodec_pm.h  |   26 +
>  .../media/platform/mtk-vcodec/mtk_vcodec_util.c    |  106 ++
>  .../media/platform/mtk-vcodec/mtk_vcodec_util.h    |   85 ++
>  drivers/media/platform/mtk-vcodec/venc_drv_base.h  |   62 +
>  drivers/media/platform/mtk-vcodec/venc_drv_if.c    |  100 ++
>  drivers/media/platform/mtk-vcodec/venc_drv_if.h    |  175 +++
>  drivers/media/platform/mtk-vcodec/venc_ipi_msg.h   |  212 +++
>  include/uapi/linux/v4l2-controls.h                 |    4 +
>  18 files changed, 3344 insertions(+)
>  create mode 100644 drivers/media/platform/mtk-vcodec/Makefile
>  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
>  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
>  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h
>  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
>  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
>  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.c
>  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.h
>  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_pm.h
>  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c
>  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
>  create mode 100644 drivers/media/platform/mtk-vcodec/venc_drv_base.h
>  create mode 100644 drivers/media/platform/mtk-vcodec/venc_drv_if.c
>  create mode 100644 drivers/media/platform/mtk-vcodec/venc_drv_if.h
>  create mode 100644 drivers/media/platform/mtk-vcodec/venc_ipi_msg.h
>  mode change 100644 => 100755 include/uapi/linux/v4l2-controls.h
>
> diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
> index ba812d6..3e831c5 100644
> --- a/drivers/media/platform/Kconfig
> +++ b/drivers/media/platform/Kconfig
> @@ -157,6 +157,17 @@ config VIDEO_MEDIATEK_VPU
>             codec embedded in new Mediatek's SOCs. It is able
>             to handle video decoding/encoding in a range of formats.
>
> +config VIDEO_MEDIATEK_VCODEC
> +        tristate "Mediatek Video Codec driver"
> +        depends on VIDEO_DEV && VIDEO_V4L2
> +        depends on ARCH_MEDIATEK || COMPILE_TEST
> +        select VIDEOBUF2_DMA_CONTIG
> +        select V4L2_MEM2MEM_DEV
> +        select MEDIATEK_VPU
> +        default n
> +        ---help---
> +            Mediatek video codec driver for V4L2
> +
>  config VIDEO_MEM2MEM_DEINTERLACE
>         tristate "Deinterlace support"
>         depends on VIDEO_DEV && VIDEO_V4L2 && DMA_ENGINE
> diff --git a/drivers/media/platform/Makefile b/drivers/media/platform/Makefile
> index e5b19c6..510e06b 100644
> --- a/drivers/media/platform/Makefile
> +++ b/drivers/media/platform/Makefile
> @@ -57,3 +57,5 @@ obj-$(CONFIG_VIDEO_XILINX)            += xilinx/
>  ccflags-y += -I$(srctree)/drivers/media/i2c
>
>  obj-$(CONFIG_VIDEO_MEDIATEK_VPU)       += mtk-vpu/
> +
> +obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC)    += mtk-vcodec/
> diff --git a/drivers/media/platform/mtk-vcodec/Makefile b/drivers/media/platform/mtk-vcodec/Makefile
> new file mode 100644
> index 0000000..ce38689
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vcodec/Makefile
> @@ -0,0 +1,8 @@
> +obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += mtk_vcodec_intr.o \
> +                                      mtk_vcodec_util.o \
> +                                      mtk_vcodec_enc_drv.o \
> +                                      mtk_vcodec_enc.o \
> +                                      mtk_vcodec_enc_pm.o \
> +                                      venc_drv_if.o
> +
> +ccflags-y += -I$(srctree)/drivers/media/platform/mtk-vpu
> diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
> new file mode 100644
> index 0000000..9da2818
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
> @@ -0,0 +1,388 @@
> +/*
> +* Copyright (c) 2015 MediaTek Inc.
> +* Author: PC Chen <pc.chen@mediatek.com>
> +*         Tiffany Lin <tiffany.lin@mediatek.com>
> +*
> +* This program is free software; you can redistribute it and/or modify
> +* it under the terms of the GNU General Public License version 2 as
> +* published by the Free Software Foundation.
> +*
> +* This program is distributed in the hope that it will be useful,
> +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +* GNU General Public License for more details.
> +*/
> +
> +#ifndef _MTK_VCODEC_DRV_H_
> +#define _MTK_VCODEC_DRV_H_
> +
> +#include <linux/platform_device.h>
> +#include <linux/videodev2.h>
> +#include <media/v4l2-ctrls.h>
> +#include <media/v4l2-device.h>
> +#include <media/v4l2-ioctl.h>
> +#include <media/videobuf2-core.h>
> +
> +#include "mtk_vcodec_util.h"
> +
> +#define MTK_VCODEC_MAX_INSTANCES               10
> +#define MTK_VCODEC_MAX_ENCODER_INSTANCES       3
> +
> +#define MTK_VCODEC_MAX_FRAME_SIZE      0x800000
> +#define MTK_VIDEO_MAX_FRAME            32
> +#define MTK_MAX_CTRLS                  20
> +
> +#define MTK_VCODEC_DRV_NAME            "mtk_vcodec_drv"
> +#define MTK_VCODEC_ENC_NAME            "mtk-vcodec-enc"
> +#define MTK_PLATFORM_STR               "platform:mt8173"
> +
> +#define MTK_VENC_IRQ_STATUS_SPS          0x1
> +#define MTK_VENC_IRQ_STATUS_PPS          0x2
> +#define MTK_VENC_IRQ_STATUS_FRM          0x4
> +#define MTK_VENC_IRQ_STATUS_DRAM         0x8
> +#define MTK_VENC_IRQ_STATUS_PAUSE        0x10
> +#define MTK_VENC_IRQ_STATUS_SWITCH       0x20
> +
> +#define MTK_VENC_IRQ_STATUS_OFFSET       0x05C
> +#define MTK_VENC_IRQ_ACK_OFFSET          0x060
> +
> +#define MTK_VCODEC_MAX_PLANES          3
> +
> +#define VDEC_HW_ACTIVE 0x10
> +#define VDEC_IRQ_CFG    0x11
> +#define VDEC_IRQ_CLR    0x10
> +
> +#define VDEC_IRQ_CFG_REG       0xa4
> +#define WAIT_INTR_TIMEOUT      1000
> +
> +/**
> + * enum mtk_hw_reg_idx - MTK hw register base index
> + */
> +enum mtk_hw_reg_idx {
> +       VDEC_SYS,
> +       VDEC_MISC,
> +       VDEC_LD,
> +       VDEC_TOP,
> +       VDEC_CM,
> +       VDEC_AD,
> +       VDEC_AV,
> +       VDEC_PP,
> +       VDEC_HWD,
> +       VDEC_HWQ,
> +       VDEC_HWB,
> +       VDEC_HWG,
> +       NUM_MAX_VDEC_REG_BASE,
> +       VENC_SYS = NUM_MAX_VDEC_REG_BASE,
> +       VENC_LT_SYS,
> +       NUM_MAX_VCODEC_REG_BASE
> +};
> +
> +/**
> + * enum mtk_instance_type - The type of an MTK Vcodec instance.
> + */
> +enum mtk_instance_type {
> +       MTK_INST_DECODER                = 0,
> +       MTK_INST_ENCODER                = 1,
> +};
> +
> +/**
> + * enum mtk_instance_state - The state of an MTK Vcodec instance.
> + * @MTK_STATE_FREE - default state when instance is created
> + * @MTK_STATE_INIT - vcodec instance is initialized
> + * @MTK_STATE_HEADER - vdec had sps/pps header parsed or venc
> + *                     had sps/pps header encoded
> + * @MTK_STATE_FLUSH - vdec is flushing. Only used by decoder
> + * @MTK_STATE_RES_CHANGE - vdec detect resolution change.
> + *                     Only used by decoder
> + * @MTK_STATE_ABORT - vcodec should be aborted
> + */
> +enum mtk_instance_state {
> +       MTK_STATE_FREE = 0,
> +       MTK_STATE_INIT = 1,
> +       MTK_STATE_HEADER = 2,
> +       MTK_STATE_FLUSH = 3,
> +       MTK_STATE_RES_CHANGE = 4,
> +       MTK_STATE_ABORT = 5,
> +};
> +
> +/**
> + * struct mtk_param_change - General encoding parameters type
> + */
> +enum mtk_encode_param {
> +       MTK_ENCODE_PARAM_NONE = 0,
> +       MTK_ENCODE_PARAM_BITRATE = (1 << 0),
> +       MTK_ENCODE_PARAM_FRAMERATE = (1 << 1),
> +       MTK_ENCODE_PARAM_INTRA_PERIOD = (1 << 2),
> +       MTK_ENCODE_PARAM_FRAME_TYPE = (1 << 3),
> +};
> +
> +/**
> + * enum mtk_fmt_type - Type of the pixelformat
> + * @MTK_FMT_FRAME - mtk vcodec raw frame
> + */
> +enum mtk_fmt_type {
> +       MTK_FMT_DEC             = 0,
> +       MTK_FMT_ENC             = 1,
> +       MTK_FMT_FRAME           = 2,
> +};
> +
> +/**
> + * struct mtk_video_fmt - Structure used to store information about pixelformats
> + */
> +struct mtk_video_fmt {
> +       u32 fourcc;
> +       enum mtk_fmt_type type;
> +       u32 num_planes;
> +};
> +
> +/**
> + * struct mtk_codec_framesizes - Structure used to store information about framesizes
> + */
> +struct mtk_codec_framesizes {
> +       u32 fourcc;
> +       struct  v4l2_frmsize_stepwise   stepwise;
> +};
> +
> +/**
> + * struct mtk_q_type - Type of queue
> + */
> +enum mtk_q_type {
> +       MTK_Q_DATA_SRC          = 0,
> +       MTK_Q_DATA_DST          = 1,
> +};
> +
> +/**
> + * struct mtk_q_data - Structure used to store information about queue
> + * @colorspace reserved for encoder
> + * @field              reserved for encoder
> + */
> +struct mtk_q_data {
> +       unsigned int            width;
> +       unsigned int            height;
> +       enum v4l2_field         field;
> +       enum v4l2_colorspace    colorspace;
> +       unsigned int            bytesperline[MTK_VCODEC_MAX_PLANES];
> +       unsigned int            sizeimage[MTK_VCODEC_MAX_PLANES];
> +       struct mtk_video_fmt    *fmt;
> +};
> +
> +/**
> + * struct mtk_enc_params - General encoding parameters
> + * @bitrate - target bitrate
> + * @num_b_frame - number of b frames between p-frame
> + * @rc_frame - frame based rate control
> + * @rc_mb - macroblock based rate control
> + * @seq_hdr_mode - H.264 sequence header is encoded separately or joined with the first frame
> + * @gop_size - group of picture size, it's used as the intra frame period
> + * @framerate_num - frame rate numerator
> + * @framerate_denom - frame rate denominator
> + * @h264_max_qp - Max value for H.264 quantization parameter
> + * @h264_profile - V4L2 defined H.264 profile
> + * @h264_level - V4L2 defined H.264 level
> + * @force_intra - force/insert intra frame
> + * @skip_frame - encode in skip frame mode that use minimum number of bits
> + */
> +struct mtk_enc_params {
> +       unsigned int    bitrate;
> +       unsigned int    num_b_frame;
> +       unsigned int    rc_frame;
> +       unsigned int    rc_mb;
> +       unsigned int    seq_hdr_mode;
> +       unsigned int    gop_size;
> +       unsigned int    framerate_num;
> +       unsigned int    framerate_denom;
> +       unsigned int    h264_max_qp;
> +       unsigned int    h264_profile;
> +       unsigned int    h264_level;
> +       unsigned int    force_intra;
> +};
> +
> +/**
> + * struct mtk_vcodec_pm - Power management data structure
> + */
> +struct mtk_vcodec_pm {
> +       struct clk      *vcodecpll;
> +       struct clk      *univpll_d2;
> +       struct clk      *clk_cci400_sel;
> +       struct clk      *vdecpll;
> +       struct clk      *vdec_sel;
> +       struct clk      *vencpll_d2;
> +       struct clk      *venc_sel;
> +       struct clk      *univpll1_d2;
> +       struct clk      *venc_lt_sel;
> +       struct device   *larbvdec;
> +       struct device   *larbvenc;
> +       struct device   *larbvenclt;
> +       struct device   *dev;
> +       struct mtk_vcodec_dev *mtkdev;
> +};
> +
> +
> +/**
> + * struct mtk_vcodec_ctx - Context (instance) private data.
> + *
> + * @type:              type of the instance - decoder or encoder
> + * @dev:               pointer to the mtk_vcodec_dev of the device
> + * @fh:                        struct v4l2_fh
> + * @m2m_ctx:           pointer to the v4l2_m2m_ctx of the context
> + * @q_data:            store information of input and output queue
> + *                     of the context
> + * @idx:               index of the context that this structure describes
> + * @state:             state of the context
> + * @param_change:      encode parameters
> + * @enc_params:                encoding parameters
> + * @colorspace:
> + * @enc_if:            hoooked encoder driver interface
> + * @drv_handle:                driver handle for specific decode/encode instance
> + *
> + * @picinfo:           store width/height of image and buffer and planes' size for decoder
> + *                     and encoder
> + * @dpb_count:         count of the DPB buffers required by MTK Vcodec hw
> + *
> + * @int_cond:          variable used by the waitqueue
> + * @int_type:          type of the last interrupt
> + * @queue:             waitqueue that can be used to wait for this context to
> + *                     finish
> + * @irq_status:                irq status
> + *
> + * @ctrl_hdl:          handler for v4l2 framework
> + * @ctrls:             array of controls, used when adding controls to the
> + *                     v4l2 control framework
> + *
> + * @encode_work:       worker for the encoding
> + */
> +struct mtk_vcodec_ctx {
> +       enum mtk_instance_type type;
> +       struct mtk_vcodec_dev *dev;
> +       struct v4l2_fh fh;
> +       struct v4l2_m2m_ctx *m2m_ctx;
> +       struct mtk_q_data q_data[2];
> +       int idx;
> +       enum mtk_instance_state state;
> +       enum mtk_encode_param param_change;
> +       struct mtk_enc_params enc_params;
> +
> +       struct venc_common_if *enc_if;
> +       unsigned long drv_handle;
> +
> +
> +       int int_cond;
> +       int int_type;
> +       wait_queue_head_t queue;
> +       unsigned int irq_status;
> +
> +       struct v4l2_ctrl_handler ctrl_hdl;
> +       struct v4l2_ctrl *ctrls[MTK_MAX_CTRLS];
> +
> +       struct work_struct encode_work;
> +
> +};
> +
> +/**
> + * struct mtk_vcodec_dev - driver data
> + * @v4l2_dev:          V4L2 device to register video devices for.
> + * @vfd_enc:           Video device for encoder.
> + *
> + * @m2m_dev_enc:       m2m device for encoder.
> + * @plat_dev:          platform device
> + * @alloc_ctx:         VB2 allocator context
> + *                     (for allocations without kernel mapping).
> + * @ctx:               array of driver contexts
> + *
> + * @curr_ctx:          The context that is waiting for codec hardware
> + *
> + * @reg_base:          Mapped address of MTK Vcodec registers.
> + *
> + * @instance_mask:     used to mark which contexts are opened
> + * @num_instances:     counter of active MTK Vcodec instances
> + *
> + * @encode_workqueue:  encode work queue
> + *
> + * @int_cond:          used to identify interrupt condition happen
> + * @int_type:          used to identify what kind of interrupt condition happen
> + * @dev_mutex:         video_device lock
> + * @queue:             waitqueue for waiting for completion of device commands
> + *
> + * @enc_irq:           h264 encoder irq resource
> + * @enc_lt_irq:                vp8 encoder irq resource
> + *
> + * @enc_mutex:         encoder hardware lock.
> + *
> + * @pm:                        power management control
> + * @dec_capability:    used to identify decode capability, ex: 4k
> + * @enc_capability:     used to identify encode capability
> + */
> +struct mtk_vcodec_dev {
> +       struct v4l2_device      v4l2_dev;
> +       struct video_device     *vfd_enc;
> +
> +       struct v4l2_m2m_dev     *m2m_dev_enc;
> +       struct platform_device  *plat_dev;
> +       struct platform_device  *vpu_plat_dev;
> +       struct vb2_alloc_ctx    *alloc_ctx;
> +       struct mtk_vcodec_ctx   *ctx[MTK_VCODEC_MAX_INSTANCES];
> +       int curr_ctx;
> +       void __iomem            *reg_base[NUM_MAX_VCODEC_REG_BASE];
> +
> +       unsigned long   instance_mask[BITS_TO_LONGS(MTK_VCODEC_MAX_INSTANCES)];
> +       int                     num_instances;
> +
> +       struct workqueue_struct *encode_workqueue;
> +
> +       int                     int_cond;
> +       int                     int_type;
> +       struct mutex            dev_mutex;
> +       wait_queue_head_t       queue;
> +
> +       int                     enc_irq;
> +       int                     enc_lt_irq;
> +
> +       struct mutex            enc_mutex;
> +
> +       struct mtk_vcodec_pm    pm;
> +       unsigned int            dec_capability;
> +       unsigned int            enc_capability;
> +};
> +
> +/**
> + * struct mtk_vcodec_ctrl - information about controls to be registered.
> + * @id:                        Control ID.
> + * @type:              Type of the control.
> + * @name:              Human readable name of the control.
> + * @minimum:           Minimum value of the control.
> + * @maximum:           Maximum value of the control.
> + * @step:              Control value increase step.
> + * @menu_skip_mask:    Mask of invalid menu positions.
> + * @default_value:     Initial value of the control.
> + * @is_volatile:       Control is volatile.
> + *
> + * See also struct v4l2_ctrl_config.
> + */
> +struct mtk_vcodec_ctrl {
> +       u32                     id;
> +       enum v4l2_ctrl_type     type;
> +       u8                      name[32];
> +       s32                     minimum;
> +       s32                     maximum;
> +       s32                     step;
> +       u32                     menu_skip_mask;
> +       s32                     default_value;
> +       u8                      is_volatile;
> +};
> +
> +static inline struct mtk_vcodec_ctx *fh_to_ctx(struct v4l2_fh *fh)
> +{
> +       return container_of(fh, struct mtk_vcodec_ctx, fh);
> +}
> +
> +static inline struct mtk_vcodec_ctx *ctrl_to_ctx(struct v4l2_ctrl *ctrl)
> +{
> +       return container_of(ctrl->handler, struct mtk_vcodec_ctx, ctrl_hdl);
> +}
> +
> +extern const struct v4l2_ioctl_ops mtk_vdec_ioctl_ops;
> +extern const struct v4l2_m2m_ops mtk_vdec_m2m_ops;
> +extern const struct v4l2_ioctl_ops mtk_venc_ioctl_ops;
> +extern const struct v4l2_m2m_ops mtk_venc_m2m_ops;
> +
> +#endif /* _MTK_VCODEC_DRV_H_ */
> diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
> new file mode 100644
> index 0000000..ee602fe
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
> @@ -0,0 +1,1380 @@
> +/*
> +* Copyright (c) 2015 MediaTek Inc.
> +* Author: PC Chen <pc.chen@mediatek.com>
> +*         Tiffany Lin <tiffany.lin@mediatek.com>
> +*
> +* This program is free software; you can redistribute it and/or modify
> +* it under the terms of the GNU General Public License version 2 as
> +* published by the Free Software Foundation.
> +*
> +* This program is distributed in the hope that it will be useful,
> +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +* GNU General Public License for more details.
> +*/
> +
> +#include <media/v4l2-event.h>
> +#include <media/v4l2-mem2mem.h>
> +#include <media/videobuf2-dma-contig.h>
> +
> +#include "mtk_vcodec_drv.h"
> +#include "mtk_vcodec_enc.h"
> +#include "mtk_vcodec_intr.h"
> +#include "mtk_vcodec_util.h"
> +#include "venc_drv_if.h"
> +
> +#define MTK_VENC_MIN_W 32
> +#define MTK_VENC_MIN_H 32
> +#define MTK_VENC_MAX_W 1920
> +#define MTK_VENC_MAX_H 1080
> +#define DFT_CFG_WIDTH  MTK_VENC_MIN_W
> +#define DFT_CFG_HEIGHT MTK_VENC_MIN_H
> +
> +static void mtk_venc_worker(struct work_struct *work);
> +
> +static struct mtk_video_fmt mtk_video_formats[] = {
> +       {
> +               .fourcc         = V4L2_PIX_FMT_YUV420,
> +               .type           = MTK_FMT_FRAME,
> +               .num_planes     = 3,
This is not right. V4L2_PIX_FMT_YUV420 only has one physical plane.
Same for V4L2_PIX_FMT_YVU420, V4L2_PIX_FMT_NV12, and
V4L2_PIX_FMT_NV21.
> +       },
> +       {
> +               .fourcc         = V4L2_PIX_FMT_YVU420,
> +               .type           = MTK_FMT_FRAME,
> +               .num_planes     = 3,
> +       },
> +       {
> +               .fourcc         = V4L2_PIX_FMT_NV12,
> +               .type           = MTK_FMT_FRAME,
> +               .num_planes     = 2,
> +       },
> +       {
> +               .fourcc         = V4L2_PIX_FMT_NV21,
> +               .type           = MTK_FMT_FRAME,
> +               .num_planes     = 2,
> +       },
> +       {
> +               .fourcc         = V4L2_PIX_FMT_YUV420M,
> +               .type           = MTK_FMT_FRAME,
> +               .num_planes     = 3,
> +       },
> +       {
> +               .fourcc         = V4L2_PIX_FMT_YVU420M,
> +               .type           = MTK_FMT_FRAME,
> +               .num_planes     = 3,
> +       },
> +       {
> +               .fourcc         = V4L2_PIX_FMT_NV12M,
> +               .type           = MTK_FMT_FRAME,
> +               .num_planes     = 2,
> +       },
> +       {
> +               .fourcc         = V4L2_PIX_FMT_NV21M,
> +               .type           = MTK_FMT_FRAME,
> +               .num_planes     = 2,
> +       },
> +       {
> +               .fourcc         = V4L2_PIX_FMT_H264,
> +               .type           = MTK_FMT_ENC,
> +               .num_planes     = 1,
> +       },
> +       {
> +               .fourcc         = V4L2_PIX_FMT_VP8,
> +               .type           = MTK_FMT_ENC,
> +               .num_planes     = 1,
> +       },
> +};
> +
> +#define NUM_FORMATS ARRAY_SIZE(mtk_video_formats)
> +
> +static const struct mtk_codec_framesizes mtk_venc_framesizes[] = {
> +       {
> +               .fourcc = V4L2_PIX_FMT_H264,
> +               .stepwise = {  160, 1920, 16, 128, 1088, 16 },
> +       },
> +       {
> +               .fourcc = V4L2_PIX_FMT_VP8,
> +               .stepwise = {  160, 1920, 16, 128, 1088, 16 },
> +       },
> +};
> +
> +#define NUM_SUPPORTED_FRAMESIZE ARRAY_SIZE(mtk_venc_framesizes)
> +
> +static int vidioc_venc_s_ctrl(struct v4l2_ctrl *ctrl)
> +{
> +       struct mtk_vcodec_ctx *ctx = ctrl_to_ctx(ctrl);
> +       struct mtk_enc_params *p = &ctx->enc_params;
> +       int ret = 0;
> +
> +       switch (ctrl->id) {
> +       case V4L2_CID_MPEG_VIDEO_BITRATE:
> +               mtk_v4l2_debug(2, "V4L2_CID_MPEG_VIDEO_BITRATE val = %d",
> +                       ctrl->val);
> +               p->bitrate = ctrl->val;
> +               ctx->param_change |= MTK_ENCODE_PARAM_BITRATE;
> +               break;
> +       case V4L2_CID_MPEG_VIDEO_B_FRAMES:
> +               mtk_v4l2_debug(2, "V4L2_CID_MPEG_VIDEO_B_FRAMES val = %d",
> +                       ctrl->val);
> +               p->num_b_frame = ctrl->val;
> +               break;
> +       case V4L2_CID_MPEG_VIDEO_FRAME_RC_ENABLE:
> +               mtk_v4l2_debug(2, "V4L2_CID_MPEG_VIDEO_FRAME_RC_ENABLE val = %d",
> +                       ctrl->val);
> +               p->rc_frame = ctrl->val;
> +               break;
> +       case V4L2_CID_MPEG_VIDEO_H264_MAX_QP:
> +               mtk_v4l2_debug(2, "V4L2_CID_MPEG_VIDEO_H264_MAX_QP val = %d",
> +                       ctrl->val);
> +               p->h264_max_qp = ctrl->val;
> +               break;
> +       case V4L2_CID_MPEG_VIDEO_HEADER_MODE:
> +               mtk_v4l2_debug(2, "V4L2_CID_MPEG_VIDEO_HEADER_MODE val = %d",
> +                       ctrl->val);
> +               p->seq_hdr_mode = ctrl->val;
> +               break;
> +       case V4L2_CID_MPEG_VIDEO_MB_RC_ENABLE:
> +               mtk_v4l2_debug(2, "V4L2_CID_MPEG_VIDEO_MB_RC_ENABLE val = %d",
> +                       ctrl->val);
> +               p->rc_mb = ctrl->val;
> +               break;
> +       case V4L2_CID_MPEG_VIDEO_H264_PROFILE:
> +               mtk_v4l2_debug(2, "V4L2_CID_MPEG_VIDEO_H264_PROFILE val = %d",
> +                       ctrl->val);
> +               p->h264_profile = ctrl->val;
> +               break;
> +       case V4L2_CID_MPEG_VIDEO_H264_LEVEL:
> +               mtk_v4l2_debug(2, "V4L2_CID_MPEG_VIDEO_H264_LEVEL val = %d",
> +                       ctrl->val);
> +               p->h264_level = ctrl->val;
> +               break;
> +       case V4L2_CID_MPEG_VIDEO_H264_I_PERIOD:
> +               mtk_v4l2_debug(2, "V4L2_CID_MPEG_VIDEO_H264_I_PERIOD val = %d",
> +                       ctrl->val);
> +       case V4L2_CID_MPEG_VIDEO_GOP_SIZE:
> +               mtk_v4l2_debug(2, "V4L2_CID_MPEG_VIDEO_GOP_SIZE val = %d",
> +                       ctrl->val);
> +               p->gop_size = ctrl->val;
> +               ctx->param_change |= MTK_ENCODE_PARAM_INTRA_PERIOD;
> +               break;
> +       case V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME:
> +               mtk_v4l2_debug(2, "V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME");
> +               p->force_intra = 1;
> +               ctx->param_change |= MTK_ENCODE_PARAM_FRAME_TYPE;
> +               break;
> +       default:
> +               ret = -EINVAL;
> +               break;
> +       }
> +
> +       return ret;
> +}
> +
> +static const struct v4l2_ctrl_ops mtk_vcodec_enc_ctrl_ops = {
> +       .s_ctrl = vidioc_venc_s_ctrl,
> +};
> +
> +static int vidioc_enum_fmt(struct file *file, struct v4l2_fmtdesc *f,
> +                          bool out)
> +{
> +       struct mtk_video_fmt *fmt;
> +       int i, j = 0;
> +
> +       for (i = 0; i < NUM_FORMATS; ++i) {
> +               if (out && mtk_video_formats[i].type != MTK_FMT_FRAME)
> +                       continue;
> +               else if (!out && mtk_video_formats[i].type != MTK_FMT_ENC)
> +                       continue;
> +
> +               if (j == f->index) {
> +                       fmt = &mtk_video_formats[i];
> +                       f->pixelformat = fmt->fourcc;
> +                       return 0;
> +               }
> +               ++j;
> +       }
> +
> +       return -EINVAL;
> +}
> +
> +static int vidioc_enum_framesizes(struct file *file, void *fh,
> +                                 struct v4l2_frmsizeenum *fsize)
> +{
> +       int i = 0;
> +
> +       for (i = 0; i < NUM_SUPPORTED_FRAMESIZE; ++i) {
> +               if (fsize->pixel_format != mtk_venc_framesizes[i].fourcc)
> +                       continue;
> +
> +               if (!fsize->index) {
> +                       fsize->type = V4L2_FRMSIZE_TYPE_STEPWISE;
> +                       fsize->stepwise = mtk_venc_framesizes[i].stepwise;
> +                       return 0;
> +               }
> +       }
> +
> +       return -EINVAL;
> +}
> +
> +static int vidioc_enum_fmt_vid_cap_mplane(struct file *file, void *pirv,
> +                                         struct v4l2_fmtdesc *f)
> +{
> +       return vidioc_enum_fmt(file, f, false);
> +}
> +
> +static int vidioc_enum_fmt_vid_out_mplane(struct file *file, void *prov,
> +                                         struct v4l2_fmtdesc *f)
> +{
> +       return vidioc_enum_fmt(file, f, true);
> +}
> +
> +static int vidioc_venc_querycap(struct file *file, void *priv,
> +                               struct v4l2_capability *cap)
> +{
> +        strlcpy(cap->driver, MTK_VCODEC_ENC_NAME, strlen(MTK_VCODEC_ENC_NAME));
> +        cap->driver[strlen(MTK_VCODEC_ENC_NAME)]=0;
> +        strlcpy(cap->bus_info, MTK_PLATFORM_STR, strlen(MTK_PLATFORM_STR));
> +        cap->bus_info[strlen(MTK_PLATFORM_STR)]=0;
> +        strlcpy(cap->card, MTK_PLATFORM_STR, strlen(MTK_PLATFORM_STR));
> +        cap->card[strlen(MTK_PLATFORM_STR)]=0;
> +
> +       /*
> +        * This is only a mem-to-mem video device. The capture and output
> +        * device capability flags are left only for backward compatibility
> +        * and are scheduled for removal.
> +        */
> +       cap->device_caps  = V4L2_CAP_VIDEO_M2M_MPLANE | V4L2_CAP_STREAMING;
> +       cap->capabilities = cap->device_caps | V4L2_CAP_DEVICE_CAPS;
> +
> +       return 0;
> +}
> +static int vidioc_venc_s_parm(struct file *file, void *priv,
> +                             struct v4l2_streamparm *a)
> +{
> +       struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
> +
> +       if (a->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
> +               ctx->enc_params.framerate_num =
> +                       a->parm.output.timeperframe.denominator;
> +               ctx->enc_params.framerate_denom =
> +                       a->parm.output.timeperframe.numerator;
> +               ctx->param_change |= MTK_ENCODE_PARAM_FRAMERATE;
> +       } else {
> +               return -EINVAL;
> +       }
> +       return 0;
> +}
> +
> +static struct mtk_q_data *mtk_venc_get_q_data(struct mtk_vcodec_ctx *ctx,
> +                                             enum v4l2_buf_type type)
> +{
> +       if (V4L2_TYPE_IS_OUTPUT(type))
> +               return &ctx->q_data[MTK_Q_DATA_SRC];
> +
> +       return &ctx->q_data[MTK_Q_DATA_DST];
> +}
> +
> +static struct mtk_video_fmt *mtk_venc_find_format(struct v4l2_format *f)
> +{
> +       struct mtk_video_fmt *fmt;
> +       unsigned int k;
> +
> +       for (k = 0; k < NUM_FORMATS; k++) {
> +               fmt = &mtk_video_formats[k];
> +               if (fmt->fourcc == f->fmt.pix.pixelformat)
> +                       return fmt;
> +       }
> +
> +       return NULL;
> +}
> +
> +static void mtk_vcodec_enc_calc_src_size(
> +       unsigned int num_planes, unsigned int pic_width,
> +       unsigned int pic_height, unsigned int sizeimage[],
> +       unsigned int bytesperline[])
> +{
> +       unsigned int y_pitch_w_div16;
> +       unsigned int c_pitch_w_div16;
> +
> +       y_pitch_w_div16 = ALIGN(pic_width, 16) >> 4;
> +       c_pitch_w_div16 = ALIGN(pic_width, 16) >> 4;
> +
> +       if (num_planes == 2) {
> +               sizeimage[0] =
> +                       (y_pitch_w_div16) * (((pic_height + 31) / 32) * 2) * 256 +
> +                       ((y_pitch_w_div16 % 8 == 0) ? 0 : ((ALIGN(pic_width, 16) * 2) * 16));
> +
> +               sizeimage[1] =
> +                       (c_pitch_w_div16) * (((pic_height + 31) / 32) * 2) * 128 +
> +                       ((c_pitch_w_div16 % 8 == 0) ? 0 : (ALIGN(pic_width, 16) * 16));
> +
> +               sizeimage[2] = 0;
> +
> +               bytesperline[0] = ALIGN(pic_width, 16);
> +               bytesperline[1] = ALIGN(pic_width, 16);
> +               bytesperline[2] = 0;
> +
> +       } else {
> +               sizeimage[0] =
> +                       (y_pitch_w_div16) * (((pic_height + 31) / 32) * 2) * 256 +
> +                       ((y_pitch_w_div16 % 8 == 0) ? 0 : ((ALIGN(pic_width, 16) * 2) * 16));
> +
> +               sizeimage[1] =
> +                       (c_pitch_w_div16) * (((pic_height + 31) / 32) * 2) * 64 +
> +                       ((c_pitch_w_div16 % 8 == 0) ? 0 : ((ALIGN(pic_width, 16) / 2) * 16));
> +
> +               sizeimage[2] =
> +                       (c_pitch_w_div16) * (((pic_height + 31) / 32) * 2) * 64 +
> +                       ((c_pitch_w_div16 % 8 == 0) ? 0 : ((ALIGN(pic_width, 16) / 2) * 16));
> +
> +               bytesperline[0] = ALIGN(pic_width, 16);
> +               bytesperline[1] = ALIGN(pic_width, 16) / 2;
> +               bytesperline[2] = ALIGN(pic_width, 16) / 2;
> +       }
> +}
> +
> +static int vidioc_try_fmt(struct v4l2_format *f, struct mtk_video_fmt *fmt)
> +{
> +       struct v4l2_pix_format_mplane *pix_fmt_mp = &f->fmt.pix_mp;
> +
> +       /* V4L2 specification suggests the driver corrects the format struct
> +         * if any of the dimensions is unsupported */
> +        if (pix_fmt_mp->height < MTK_VENC_MIN_H)
> +                pix_fmt_mp->height = MTK_VENC_MIN_H;
> +        else if (pix_fmt_mp->height > MTK_VENC_MAX_H)
> +                pix_fmt_mp->height = MTK_VENC_MAX_H;
> +
> +        if (pix_fmt_mp->width < MTK_VENC_MIN_W)
> +                pix_fmt_mp->width = MTK_VENC_MIN_W;
> +        else if (pix_fmt_mp->width > MTK_VENC_MAX_W)
> +                pix_fmt_mp->width = MTK_VENC_MAX_W;
> +
> +        pix_fmt_mp->field = V4L2_FIELD_NONE;
> +
> +       if (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
> +               int size = pix_fmt_mp->height * pix_fmt_mp->width;
> +               if (fmt->num_planes != pix_fmt_mp->num_planes)
> +                       pix_fmt_mp->num_planes = fmt->num_planes;
> +
> +               if(pix_fmt_mp->plane_fmt[0].sizeimage != size)
> +                       pix_fmt_mp->plane_fmt[0].sizeimage = size;
> +               pix_fmt_mp->plane_fmt[0].bytesperline = 0;
> +               memset(&(pix_fmt_mp->plane_fmt[0].reserved[0]), 0x0,
> +                       sizeof(pix_fmt_mp->plane_fmt[0].reserved));
> +       } else if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
> +               int i;
> +               unsigned int sizeimage[VIDEO_MAX_PLANES];
> +               unsigned int bytesperline[VIDEO_MAX_PLANES];
> +
> +               v4l_bound_align_image(&pix_fmt_mp->width, 8, 1920, 1,
> +                                     &pix_fmt_mp->height, 4, 1080, 1, 0);
> +
> +               if (fmt->num_planes != pix_fmt_mp->num_planes)
> +                       pix_fmt_mp->num_planes = fmt->num_planes;
> +
> +               mtk_vcodec_enc_calc_src_size(pix_fmt_mp->num_planes,
> +                                       pix_fmt_mp->width,
> +                                       pix_fmt_mp->height,
> +                                       sizeimage, bytesperline);
> +
> +               for (i=0; i < pix_fmt_mp->num_planes; i++) {
> +                       pix_fmt_mp->plane_fmt[i].sizeimage = sizeimage[i];
> +                       pix_fmt_mp->plane_fmt[i].bytesperline = bytesperline[i];
> +                       memset(&(pix_fmt_mp->plane_fmt[i].reserved[0]), 0x0,
> +                               sizeof(pix_fmt_mp->plane_fmt[0].reserved));
> +               }
> +       } else {
> +               return -EINVAL;
> +       }
> +
> +       pix_fmt_mp->flags = 0;
> +       pix_fmt_mp->ycbcr_enc = 0;
> +       pix_fmt_mp->quantization = 0;
> +       pix_fmt_mp->xfer_func = 0;
> +       memset(&pix_fmt_mp->reserved[0], 0x0, sizeof(pix_fmt_mp->reserved));
> +
> +       return 0;
> +}
> +
> +static void mtk_venc_set_param(struct mtk_vcodec_ctx *ctx, void *param)
> +{
> +       struct venc_enc_prm *p = (struct venc_enc_prm *)param;
> +       struct mtk_q_data *q_data_src = &ctx->q_data[MTK_Q_DATA_SRC];
> +       struct mtk_enc_params *enc_params = &ctx->enc_params;
> +       unsigned int frame_rate;
> +
> +       frame_rate = enc_params->framerate_num / enc_params->framerate_denom;
> +
> +       switch (q_data_src->fmt->fourcc) {
> +       case V4L2_PIX_FMT_YUV420:
> +       case V4L2_PIX_FMT_YUV420M:
> +               p->input_fourcc = VENC_YUV_FORMAT_420;
> +               break;
> +       case V4L2_PIX_FMT_YVU420:
> +       case V4L2_PIX_FMT_YVU420M:
> +               p->input_fourcc = VENC_YUV_FORMAT_YV12;
> +               break;
> +       case V4L2_PIX_FMT_NV12:
> +       case V4L2_PIX_FMT_NV12M:
> +               p->input_fourcc = VENC_YUV_FORMAT_NV12;
> +               break;
> +       case V4L2_PIX_FMT_NV21:
> +       case V4L2_PIX_FMT_NV21M:
> +               p->input_fourcc = VENC_YUV_FORMAT_NV21;
> +               break;
> +       }
> +       p->h264_profile = enc_params->h264_profile;
> +       p->h264_level = enc_params->h264_level;
> +       p->width = q_data_src->width;
> +       p->height = q_data_src->height;
> +       p->buf_width = q_data_src->bytesperline[0];
> +       p->buf_height = ((q_data_src->height + 0xf) & (~0xf));
> +       p->frm_rate = frame_rate;
> +       p->intra_period = enc_params->gop_size;
> +       p->bitrate = enc_params->bitrate;
> +
> +       ctx->param_change = MTK_ENCODE_PARAM_NONE;
> +
> +       mtk_v4l2_debug(1, "fmt 0x%x, P/L %d/%d, w/h %d/%d, buf %d/%d, fps/bps %d/%d, gop %d",
> +                      p->input_fourcc, p->h264_profile, p->h264_level,
> +                      p->width, p->height, p->buf_width, p->buf_height,
> +                      p->frm_rate, p->bitrate, p->intra_period);
> +}
> +
> +static int vidioc_venc_s_fmt(struct file *file, void *priv,
> +                            struct v4l2_format *f)
> +{
> +       struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
> +       struct v4l2_pix_format_mplane *pix_fmt_mp = &f->fmt.pix_mp;
> +       struct vb2_queue *vq;
> +       struct mtk_q_data *q_data;
> +       struct venc_enc_prm param;
> +       int i, ret;
> +       struct mtk_video_fmt *fmt;
> +
> +       vq = v4l2_m2m_get_vq(ctx->m2m_ctx, f->type);
> +       if (!vq) {
> +               mtk_v4l2_err("fail to get vq\n");
> +               return -EINVAL;
> +       }
> +
> +       if (vb2_is_busy(vq)) {
> +               mtk_v4l2_err("queue busy\n");
> +               return -EBUSY;
> +       }
> +
> +       q_data = mtk_venc_get_q_data(ctx, f->type);
> +       if (!q_data) {
> +               mtk_v4l2_err("fail to get q data\n");
> +               return -EINVAL;
> +       }
> +
> +       fmt = mtk_venc_find_format(f);
> +       if (!fmt) {
> +               if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
> +                       f->fmt.pix.pixelformat = mtk_video_formats[0].fourcc;
> +                       fmt = mtk_venc_find_format(f);
> +               } else {
> +                       f->fmt.pix.pixelformat = mtk_video_formats[8].fourcc;
> +                       fmt = mtk_venc_find_format(f);
> +               }
> +       }
> +
> +       q_data->fmt = fmt;
> +       ret = vidioc_try_fmt(f, q_data->fmt);
> +       if (ret)
> +               return ret;
> +
> +       q_data->width           = f->fmt.pix_mp.width;
> +       q_data->height          = f->fmt.pix_mp.height;
> +       q_data->field           = f->fmt.pix_mp.field;
> +
> +       if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
> +               q_data->colorspace = f->fmt.pix_mp.colorspace;
> +               ctx->q_data[MTK_Q_DATA_SRC].bytesperline[0] =
> +                       ALIGN(q_data->width, 16);
> +
> +               if (q_data->fmt->num_planes == 2) {
> +                       ctx->q_data[MTK_Q_DATA_SRC].bytesperline[1] =
> +                               ALIGN(q_data->width, 16);
> +                       ctx->q_data[MTK_Q_DATA_SRC].bytesperline[2] = 0;
> +               } else {
> +                       ctx->q_data[MTK_Q_DATA_SRC].bytesperline[1] =
> +                               ALIGN(q_data->width, 16) / 2;
> +                       ctx->q_data[MTK_Q_DATA_SRC].bytesperline[2] =
> +                               ALIGN(q_data->width, 16) / 2;
> +               }
> +
> +               memset(&param, 0, sizeof(param));
> +               mtk_venc_set_param(ctx, &param);
> +               if (ctx->state == MTK_STATE_INIT) {
> +                       ret = venc_if_set_param(ctx,
> +                                               VENC_SET_PARAM_ENC,
> +                                               &param);
> +                       if (ret)
> +                               mtk_v4l2_err("venc_if_set_param failed=%d\n",
> +                                               ret);
> +
> +                       /* Get codec driver advice sizeimage from vpu */
> +                       for (i = 0; i < MTK_VCODEC_MAX_PLANES; i++) {
> +                               q_data->sizeimage[i] = param.sizeimage[i];
> +                               pix_fmt_mp->plane_fmt[i].sizeimage =
> +                                       param.sizeimage[i];
> +                       }
> +                       q_data->bytesperline[0] =
> +                               pix_fmt_mp->plane_fmt[0].bytesperline;
> +                       q_data->bytesperline[1] =
> +                               pix_fmt_mp->plane_fmt[1].bytesperline;
> +                       q_data->bytesperline[2] =
> +                               pix_fmt_mp->plane_fmt[2].bytesperline;
> +               } else {
> +                       for (i = 0; i < MTK_VCODEC_MAX_PLANES; i++) {
> +                               q_data->sizeimage[i] =
> +                                       pix_fmt_mp->plane_fmt[i].sizeimage;
> +                               q_data->bytesperline[i] =
> +                                       pix_fmt_mp->plane_fmt[i].bytesperline;
> +                       }
> +               }
> +       } else if (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE){
> +               for (i = 0; i < f->fmt.pix_mp.num_planes; i++) {
> +                       struct v4l2_plane_pix_format    *plane_fmt;
> +
> +                       plane_fmt = &f->fmt.pix_mp.plane_fmt[i];
> +                       q_data->bytesperline[i] = plane_fmt->bytesperline;
> +                       q_data->sizeimage[i]    = plane_fmt->sizeimage;
> +               }
> +
> +               if (ctx->state == MTK_STATE_FREE) {
> +                       ret = venc_if_create(ctx, q_data->fmt->fourcc);
> +                       if (ret) {
> +                               mtk_v4l2_err("venc_if_create failed=%d, codec type=%x\n",
> +                                       ret, q_data->fmt->fourcc);
> +                               return 0;
> +                       }
> +
> +                       ctx->state = MTK_STATE_INIT;
> +               }
> +       }
> +
> +       return 0;
> +}
> +
> +static int vidioc_venc_g_fmt(struct file *file, void *priv,
> +                            struct v4l2_format *f)
> +{
> +       struct v4l2_pix_format_mplane *pix = &f->fmt.pix_mp;
> +       struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
> +       struct vb2_queue *vq;
> +       struct mtk_q_data *q_data;
> +       int i;
> +
> +       vq = v4l2_m2m_get_vq(ctx->m2m_ctx, f->type);
> +       if (!vq)
> +               return -EINVAL;
> +
> +       q_data = mtk_venc_get_q_data(ctx, f->type);
> +
> +       pix->width = q_data->width;
> +       pix->height = q_data->height;
> +       pix->pixelformat = q_data->fmt->fourcc;
> +       pix->field = q_data->field;
> +       pix->colorspace = q_data->colorspace;
> +       pix->num_planes = q_data->fmt->num_planes;
> +       for (i = 0; i < pix->num_planes; i++) {
> +               pix->plane_fmt[i].bytesperline = q_data->bytesperline[i];
> +               pix->plane_fmt[i].sizeimage = q_data->sizeimage[i];
> +               memset(&(pix->plane_fmt[i].reserved[0]), 0x0,
> +                       sizeof(pix->plane_fmt[i].reserved));
> +       }
> +       pix->flags = 0;
> +       pix->ycbcr_enc = 0;
> +       pix->quantization = 0;
> +       pix->xfer_func = 0;
> +       memset(&pix->reserved[0], 0x0, sizeof(pix->reserved));
> +
> +       return 0;
> +}
> +
> +static int vidioc_try_fmt_vid_cap_mplane(struct file *file, void *priv,
> +                                  struct v4l2_format *f)
> +{
> +        struct mtk_video_fmt *fmt;
> +       struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
> +
> +        fmt = mtk_venc_find_format(f);
> +        if (!fmt) {
> +                f->fmt.pix.pixelformat = mtk_video_formats[8].fourcc;
> +                fmt = mtk_venc_find_format(f);
> +        }
> +        if (fmt->type != MTK_FMT_ENC) {
> +               mtk_v4l2_err("Fourcc format (0x%08x) invalid.\n",
> +                            f->fmt.pix.pixelformat);
> +               return -EINVAL;
> +        }
> +        f->fmt.pix_mp.colorspace = ctx->q_data[MTK_Q_DATA_SRC].colorspace;
> +
> +        return vidioc_try_fmt(f, fmt);
> +}
> +
> +static int vidioc_try_fmt_vid_out_mplane(struct file *file, void *priv,
> +                                  struct v4l2_format *f)
> +{
> +        struct mtk_video_fmt *fmt;
> +
> +        fmt = mtk_venc_find_format(f);
> +        if (!fmt) {
> +                f->fmt.pix.pixelformat = mtk_video_formats[0].fourcc;
> +                fmt = mtk_venc_find_format(f);
> +        }
> +        if (!(fmt->type & MTK_FMT_FRAME)) {
> +               mtk_v4l2_err("Fourcc format (0x%08x) invalid.\n",
> +                            f->fmt.pix.pixelformat);
> +               return -EINVAL;
> +        }
> +        if (!f->fmt.pix_mp.colorspace)
> +                f->fmt.pix_mp.colorspace = V4L2_COLORSPACE_REC709;
> +
> +        return vidioc_try_fmt(f, fmt);
> +}
> +
> +static int vidioc_venc_g_s_selection(struct file *file, void *priv,
> +                                struct v4l2_selection *s)
> +{
> +       struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
> +       struct mtk_q_data *q_data;
> +
> +       if (V4L2_TYPE_IS_OUTPUT(s->type)) {
> +               if (s->target !=  V4L2_SEL_TGT_COMPOSE)
> +                       return -EINVAL;
> +       } else {
> +               if (s->target != V4L2_SEL_TGT_CROP)
> +                       return -EINVAL;
> +       }
> +
> +       if (s->r.left || s->r.top)
> +               return -EINVAL;
> +
> +       q_data = mtk_venc_get_q_data(ctx, s->type);
> +       if (!q_data)
> +               return -EINVAL;
> +
> +       s->r.width = q_data->width;
> +       s->r.height = q_data->height;
> +
> +       return 0;
> +}
> +
> +
> +static int vidioc_venc_qbuf(struct file *file, void *priv,
> +                           struct v4l2_buffer *buf)
> +{
> +
> +       struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
> +
> +       if (ctx->state == MTK_STATE_ABORT) {
> +               mtk_v4l2_err("[%d] Call on QBUF after unrecoverable error\n", ctx->idx);
> +               return -EIO;
> +       }
> +
> +       return v4l2_m2m_qbuf(file, ctx->m2m_ctx, buf);
> +}
> +
> +static int vidioc_venc_dqbuf(struct file *file, void *priv,
> +                            struct v4l2_buffer *buf)
> +{
> +       struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
> +       if (ctx->state == MTK_STATE_ABORT) {
> +               mtk_v4l2_err("[%d] Call on QBUF after unrecoverable error\n", ctx->idx);
> +               return -EIO;
> +       }
> +
> +       return v4l2_m2m_dqbuf(file, ctx->m2m_ctx, buf);
> +}
> +
> +
> +const struct v4l2_ioctl_ops mtk_venc_ioctl_ops = {
> +       .vidioc_streamon                = v4l2_m2m_ioctl_streamon,
> +       .vidioc_streamoff               = v4l2_m2m_ioctl_streamoff,
> +
> +       .vidioc_reqbufs                 = v4l2_m2m_ioctl_reqbufs,
> +       .vidioc_querybuf                = v4l2_m2m_ioctl_querybuf,
> +       .vidioc_qbuf                    = vidioc_venc_qbuf,
> +       .vidioc_dqbuf                   = vidioc_venc_dqbuf,
> +
> +       .vidioc_querycap                = vidioc_venc_querycap,
> +       .vidioc_enum_fmt_vid_cap_mplane = vidioc_enum_fmt_vid_cap_mplane,
> +       .vidioc_enum_fmt_vid_out_mplane = vidioc_enum_fmt_vid_out_mplane,
> +       .vidioc_enum_framesizes         = vidioc_enum_framesizes,
> +
> +       .vidioc_try_fmt_vid_cap_mplane  = vidioc_try_fmt_vid_cap_mplane,
> +       .vidioc_try_fmt_vid_out_mplane  = vidioc_try_fmt_vid_out_mplane,
> +       .vidioc_expbuf                  = v4l2_m2m_ioctl_expbuf,
> +       .vidioc_subscribe_event         = v4l2_ctrl_subscribe_event,
> +       .vidioc_unsubscribe_event       = v4l2_event_unsubscribe,
> +
> +       .vidioc_s_parm                  = vidioc_venc_s_parm,
> +
> +       .vidioc_s_fmt_vid_cap_mplane    = vidioc_venc_s_fmt,
> +       .vidioc_s_fmt_vid_out_mplane    = vidioc_venc_s_fmt,
> +
> +       .vidioc_g_fmt_vid_cap_mplane    = vidioc_venc_g_fmt,
> +       .vidioc_g_fmt_vid_out_mplane    = vidioc_venc_g_fmt,
> +
> +       .vidioc_g_selection             = vidioc_venc_g_s_selection,
> +       .vidioc_s_selection             = vidioc_venc_g_s_selection,
> +};
> +
> +static int vb2ops_venc_queue_setup(struct vb2_queue *vq,
> +                                  const void *parg,
> +                                  unsigned int *nbuffers,
> +                                  unsigned int *nplanes,
> +                                  unsigned int sizes[], void *alloc_ctxs[])
> +{
> +       struct mtk_vcodec_ctx *ctx = vb2_get_drv_priv(vq);
> +       struct mtk_q_data *q_data;
> +
> +       q_data = mtk_venc_get_q_data(ctx, vq->type);
> +
> +       if (*nbuffers < 1)
> +               *nbuffers = 1;
> +       if (*nbuffers > MTK_VIDEO_MAX_FRAME)
> +               *nbuffers = MTK_VIDEO_MAX_FRAME;
> +
> +       *nplanes = q_data->fmt->num_planes;
> +
> +       if (vq->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
> +               unsigned int i;
> +
> +               for (i = 0; i < *nplanes; i++) {
> +                       sizes[i] = q_data->sizeimage[i];
> +                       alloc_ctxs[i] = ctx->dev->alloc_ctx;
> +               }
> +       } else if (vq->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
> +               sizes[0] = q_data->sizeimage[0];
> +               alloc_ctxs[0] = ctx->dev->alloc_ctx;
> +       } else {
> +               return -EINVAL;
> +       }
> +
> +       mtk_v4l2_debug(2, "[%d]get %d buffer(s) of size 0x%x each, vq->memory=%d",
> +                      ctx->idx, *nbuffers, sizes[0], vq->memory);
> +
> +       return 0;
> +}
> +
> +static int vb2ops_venc_buf_prepare(struct vb2_buffer *vb)
> +{
> +       struct mtk_vcodec_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue);
> +       struct mtk_q_data *q_data;
> +       int i;
> +
> +       q_data = mtk_venc_get_q_data(ctx, vb->vb2_queue->type);
> +
> +       for (i = 0; i < q_data->fmt->num_planes; i++) {
> +               if (vb2_plane_size(vb, i) < q_data->sizeimage[i]) {
> +                       mtk_v4l2_debug(2, "data will not fit into plane %d (%lu < %d)",
> +                                      i, vb2_plane_size(vb, i),
> +                                      q_data->sizeimage[i]);
> +                       return -EINVAL;
> +               }
> +       }
> +
> +       return 0;
> +}
> +
> +static void vb2ops_venc_buf_queue(struct vb2_buffer *vb)
> +{
> +       struct mtk_vcodec_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue);
> +       struct vb2_v4l2_buffer *vb2_v4l2 =
> +                       container_of(vb, struct vb2_v4l2_buffer, vb2_buf);
> +       struct mtk_video_enc_buf *mtk_buf =
> +                       container_of(vb2_v4l2, struct mtk_video_enc_buf, vb);
> +
> +       if ((vb->vb2_queue->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) &&
> +               (ctx->param_change != MTK_ENCODE_PARAM_NONE)) {
> +               mtk_v4l2_debug(1,
> +                               "[%d] Before id=%d encode parameter change %x",
> +                               ctx->idx, vb->index,
> +                               ctx->param_change);
> +               mtk_buf->param_change = ctx->param_change;
> +               if (mtk_buf->param_change & MTK_ENCODE_PARAM_BITRATE) {
> +                       mtk_buf->enc_params.bitrate = ctx->enc_params.bitrate;
> +                       mtk_v4l2_debug(1, "[%d] idx=%d change param br=%d",
> +                               ctx->idx,
> +                               mtk_buf->vb.vb2_buf.index,
> +                               mtk_buf->enc_params.bitrate);
> +               }
> +               if (ctx->param_change & MTK_ENCODE_PARAM_FRAMERATE) {
> +                       mtk_buf->enc_params.framerate_num =
> +                               ctx->enc_params.framerate_num;
> +                       mtk_buf->enc_params.framerate_denom =
> +                               ctx->enc_params.framerate_denom;
> +                       mtk_v4l2_debug(1, "[%d] idx=%d, change param fr=%d/%d",
> +                                       ctx->idx,
> +                                       mtk_buf->vb.vb2_buf.index,
> +                                       mtk_buf->enc_params.framerate_num,
> +                                       mtk_buf->enc_params.framerate_denom);
> +               }
> +               if (ctx->param_change & MTK_ENCODE_PARAM_INTRA_PERIOD) {
> +                       mtk_buf->enc_params.gop_size = ctx->enc_params.gop_size;
> +                       mtk_v4l2_debug(1, "[%d] idx=%d, change param intra period=%d",
> +                                       ctx->idx,
> +                                       mtk_buf->vb.vb2_buf.index,
> +                                       mtk_buf->enc_params.gop_size);
> +               }
> +               if (ctx->param_change & MTK_ENCODE_PARAM_FRAME_TYPE) {
> +                       mtk_buf->enc_params.force_intra =
> +                               ctx->enc_params.force_intra;
> +                       mtk_v4l2_debug(1, "[%d] idx=%d, change param force I=%d",
> +                                       ctx->idx,
> +                                       mtk_buf->vb.vb2_buf.index,
> +                                       mtk_buf->enc_params.force_intra);
> +               }
> +               ctx->param_change = MTK_ENCODE_PARAM_NONE;
> +       }
> +
> +       v4l2_m2m_buf_queue(ctx->m2m_ctx, to_vb2_v4l2_buffer(vb));
> +}
> +
> +static int vb2ops_venc_start_streaming(struct vb2_queue *q, unsigned int count)
> +{
> +       struct mtk_vcodec_ctx *ctx = vb2_get_drv_priv(q);
> +       struct venc_enc_prm param;
> +       int ret;
> +       int i;
> +
> +       /* Once state turn into MTK_STATE_ABORT, we need stop_streaming to clear it */
> +       if ((ctx->state == MTK_STATE_ABORT) || (ctx->state == MTK_STATE_FREE))
> +               goto err_set_param;
> +
> +       if (!(vb2_start_streaming_called(&ctx->m2m_ctx->out_q_ctx.q) &
> +             vb2_start_streaming_called(&ctx->m2m_ctx->cap_q_ctx.q))) {
> +               mtk_v4l2_debug(1, "[%d]-> out=%d cap=%d",
> +                ctx->idx,
> +                vb2_start_streaming_called(&ctx->m2m_ctx->out_q_ctx.q),
> +                vb2_start_streaming_called(&ctx->m2m_ctx->cap_q_ctx.q));
> +               return 0;
> +       }
> +
> +       mtk_venc_set_param(ctx, &param);
> +       ret = venc_if_set_param(ctx,
> +                               VENC_SET_PARAM_ENC,
> +                               &param);
> +       if (ret) {
> +               mtk_v4l2_err("venc_if_set_param failed=%d\n", ret);
> +               ctx->state = MTK_STATE_ABORT;
> +               goto err_set_param;
> +       }
> +
> +       if ((ctx->q_data[MTK_Q_DATA_DST].fmt->fourcc == V4L2_PIX_FMT_H264) &&
> +           (ctx->enc_params.seq_hdr_mode != V4L2_MPEG_VIDEO_HEADER_MODE_SEPARATE)) {
> +               ret = venc_if_set_param(ctx,
> +                                       VENC_SET_PARAM_PREPEND_HEADER,
> +                                       0);
> +               if (ret) {
> +                       mtk_v4l2_err("venc_if_set_param failed=%d\n", ret);
> +                       ctx->state = MTK_STATE_ABORT;
> +                       goto err_set_param;
> +               }
> +               ctx->state = MTK_STATE_HEADER;
> +       }
> +
> +       return 0;
> +
> +err_set_param:
> +       for (i = 0; i < q->num_buffers; ++i) {
> +               if (q->bufs[i]->state == VB2_BUF_STATE_ACTIVE) {
> +                       mtk_v4l2_debug(0, "[%d] idx=%d, type=%d, %d -> VB2_BUF_STATE_QUEUED",
> +                                       ctx->idx, i, q->type,
> +                                       (int)q->bufs[i]->state );
> +                       v4l2_m2m_buf_done(to_vb2_v4l2_buffer(q->bufs[i]), VB2_BUF_STATE_QUEUED);
> +               }
> +       }
> +
> +       return -EINVAL;
> +}
> +
> +static void vb2ops_venc_stop_streaming(struct vb2_queue *q)
> +{
> +       struct mtk_vcodec_ctx *ctx = vb2_get_drv_priv(q);
> +       struct vb2_buffer *src_buf, *dst_buf;
> +       int ret;
> +
> +       mtk_v4l2_debug(2, "[%d]-> type=%d", ctx->idx, q->type);
> +
> +       if (q->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
> +               while ((dst_buf = v4l2_m2m_dst_buf_remove(ctx->m2m_ctx))) {
> +                       dst_buf->planes[0].bytesused = 0;
> +                       v4l2_m2m_buf_done(to_vb2_v4l2_buffer(dst_buf),
> +                                               VB2_BUF_STATE_ERROR);
> +               }
> +       } else {
> +               while ((src_buf = v4l2_m2m_src_buf_remove(ctx->m2m_ctx)))
> +                       v4l2_m2m_buf_done(to_vb2_v4l2_buffer(src_buf),
> +                                               VB2_BUF_STATE_ERROR);
> +       }
> +
> +       if ((q->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE &&
> +            vb2_is_streaming(&ctx->m2m_ctx->out_q_ctx.q)) ||
> +           (q->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE &&
> +            vb2_is_streaming(&ctx->m2m_ctx->cap_q_ctx.q))) {
> +               mtk_v4l2_debug(1, "[%d]-> q type %d out=%d cap=%d",
> +                              ctx->idx, q->type,
> +                              vb2_is_streaming(&ctx->m2m_ctx->out_q_ctx.q),
> +                              vb2_is_streaming(&ctx->m2m_ctx->cap_q_ctx.q));
> +               return;
> +       }
> +
> +       ret = venc_if_release(ctx);
> +       if (ret)
> +               mtk_v4l2_err("venc_if_release failed=%d\n", ret);
> +
> +       ctx->state = MTK_STATE_FREE;
> +}
> +
> +static struct vb2_ops mtk_venc_vb2_ops = {
> +       .queue_setup                    = vb2ops_venc_queue_setup,
> +       .buf_prepare                    = vb2ops_venc_buf_prepare,
> +       .buf_queue                      = vb2ops_venc_buf_queue,
> +       .wait_prepare                   = vb2_ops_wait_prepare,
> +       .wait_finish                    = vb2_ops_wait_finish,
> +       .start_streaming                = vb2ops_venc_start_streaming,
> +       .stop_streaming                 = vb2ops_venc_stop_streaming,
> +};
> +
> +static int mtk_venc_encode_header(void *priv)
> +{
> +       struct mtk_vcodec_ctx *ctx = priv;
> +       int ret;
> +       struct vb2_buffer *dst_buf;
> +       struct mtk_vcodec_mem bs_buf;
> +       struct venc_done_result enc_result;
> +
> +       dst_buf = v4l2_m2m_dst_buf_remove(ctx->m2m_ctx);
> +       if (!dst_buf) {
> +               mtk_v4l2_debug(1, "No dst buffer");
> +               return -EINVAL;
> +       }
> +
> +       bs_buf.va = vb2_plane_vaddr(dst_buf, 0);
> +       bs_buf.dma_addr = vb2_dma_contig_plane_dma_addr(dst_buf, 0);
> +       bs_buf.size = (unsigned int)dst_buf->planes[0].length;
> +
> +       mtk_v4l2_debug(1,
> +                       "[%d] buf idx=%d va=0x%p dma_addr=0x%llx size=0x%lx",
> +                       ctx->idx,
> +                       dst_buf->index, bs_buf.va,
> +                       (u64)bs_buf.dma_addr,
> +                       bs_buf.size);
> +
> +       ret = venc_if_encode(ctx,
> +                       VENC_START_OPT_ENCODE_SEQUENCE_HEADER,
> +                       0, &bs_buf, &enc_result);
> +
> +       if (ret) {
> +               dst_buf->planes[0].bytesused = 0;
> +               ctx->state = MTK_STATE_ABORT;
> +               v4l2_m2m_buf_done(to_vb2_v4l2_buffer(dst_buf), VB2_BUF_STATE_ERROR);
> +               mtk_v4l2_err("venc_if_encode failed=%d", ret);
> +               return -EINVAL;
> +       }
> +
> +       ctx->state = MTK_STATE_HEADER;
> +       dst_buf->planes[0].bytesused = enc_result.bs_size;
> +
> +#if defined(DEBUG)
> +{
> +       int i;
> +       mtk_v4l2_debug(1, "[%d] venc_if_encode header len=%d",
> +                       ctx->idx,
> +                       enc_result.bs_size);
> +       for (i = 0; i < enc_result.bs_size; i++) {
> +               unsigned char *p = (unsigned char *)bs_buf.va;
> +
> +               mtk_v4l2_debug(1, "[%d] buf[%d]=0x%2x", ctx->idx, i, p[i]);
> +       }
> +}
> +#endif
> +       v4l2_m2m_buf_done(to_vb2_v4l2_buffer(dst_buf), VB2_BUF_STATE_DONE);
> +
> +       return 0;
> +}
> +
> +static int mtk_venc_param_change(struct mtk_vcodec_ctx *ctx, void *priv)
> +{
> +       struct vb2_buffer *vb = priv;
> +       struct vb2_v4l2_buffer *vb2_v4l2 =
> +                       container_of(vb, struct vb2_v4l2_buffer, vb2_buf);
> +       struct mtk_video_enc_buf *mtk_buf =
> +                       container_of(vb2_v4l2, struct mtk_video_enc_buf, vb);
> +       int ret = 0;
> +
> +       if (mtk_buf->param_change == MTK_ENCODE_PARAM_NONE)
> +               return 0;
> +
> +       mtk_v4l2_debug(1, "encode parameters change id=%d", vb->index);
> +       if (mtk_buf->param_change & MTK_ENCODE_PARAM_BITRATE) {
> +               struct venc_enc_prm enc_prm;
> +
> +               enc_prm.bitrate = mtk_buf->enc_params.bitrate;
> +               mtk_v4l2_debug(1, "[%d] idx=%d, change param br=%d",
> +                               ctx->idx,
> +                               mtk_buf->vb.vb2_buf.index,
> +                               enc_prm.bitrate);
> +               ret |= venc_if_set_param(ctx,
> +                                        VENC_SET_PARAM_ADJUST_BITRATE,
> +                                        &enc_prm);
> +       }
> +       if (mtk_buf->param_change & MTK_ENCODE_PARAM_FRAMERATE) {
> +               struct venc_enc_prm enc_prm;
> +
> +               enc_prm.frm_rate = mtk_buf->enc_params.framerate_num /
> +                                  mtk_buf->enc_params.framerate_denom;
> +               mtk_v4l2_debug(1, "[%d] idx=%d, change param fr=%d",
> +                              ctx->idx,
> +                              mtk_buf->vb.vb2_buf.index,
> +                              enc_prm.frm_rate);
> +               ret |= venc_if_set_param(ctx,
> +                                        VENC_SET_PARAM_ADJUST_FRAMERATE,
> +                                        &enc_prm);
> +       }
> +       if (mtk_buf->param_change & MTK_ENCODE_PARAM_INTRA_PERIOD) {
> +               mtk_v4l2_debug(1, "change param intra period=%d",
> +                                mtk_buf->enc_params.gop_size);
> +               ret |= venc_if_set_param(ctx,
> +                                        VENC_SET_PARAM_I_FRAME_INTERVAL,
> +                                        &mtk_buf->enc_params.gop_size);
> +       }
> +       if (mtk_buf->param_change & MTK_ENCODE_PARAM_FRAME_TYPE) {
> +               mtk_v4l2_debug(1, "[%d] idx=%d, change param force I=%d",
> +                               ctx->idx,
> +                               mtk_buf->vb.vb2_buf.index,
> +                               mtk_buf->enc_params.force_intra);
> +               if (mtk_buf->enc_params.force_intra)
> +                       ret |= venc_if_set_param(ctx,
> +                                                VENC_SET_PARAM_FORCE_INTRA,
> +                                                0);
> +       }
> +
> +       mtk_buf->param_change = MTK_ENCODE_PARAM_NONE;
> +
> +       if (ret) {
> +               ctx->state = MTK_STATE_ABORT;
> +               mtk_v4l2_err("venc_if_set_param %d failed=%d\n",
> +                       MTK_ENCODE_PARAM_FRAME_TYPE, ret);
> +               return -1;
> +       }
> +
> +       return 0;
> +}
> +
> +static void mtk_venc_worker(struct work_struct *work)
> +{
> +       struct mtk_vcodec_ctx *ctx = container_of(work, struct mtk_vcodec_ctx,
> +                                   encode_work);
> +       struct vb2_buffer *src_buf, *dst_buf;
> +       struct venc_frm_buf frm_buf;
> +       struct mtk_vcodec_mem bs_buf;
> +       struct venc_done_result enc_result;
> +       int ret;
> +       struct vb2_v4l2_buffer *v4l2_vb;
> +
> +       if ((ctx->q_data[MTK_Q_DATA_DST].fmt->fourcc == V4L2_PIX_FMT_H264) &&
> +           (ctx->state != MTK_STATE_HEADER)) {
> +               /* encode h264 sps/pps header */
> +               mtk_venc_encode_header(ctx);
> +               v4l2_m2m_job_finish(ctx->dev->m2m_dev_enc, ctx->m2m_ctx);
> +               return;
> +       }
> +
> +       src_buf = v4l2_m2m_next_src_buf(ctx->m2m_ctx);
> +       if (!src_buf) {
> +               v4l2_m2m_job_finish(ctx->dev->m2m_dev_enc, ctx->m2m_ctx);
> +               return;
> +       }
> +
> +       dst_buf = v4l2_m2m_dst_buf_remove(ctx->m2m_ctx);
> +       if (!dst_buf) {
> +               v4l2_m2m_job_finish(ctx->dev->m2m_dev_enc, ctx->m2m_ctx);
> +               return;
> +       }
> +
> +       mtk_venc_param_change(ctx, src_buf);
> +
> +       frm_buf.fb_addr.va = vb2_plane_vaddr(src_buf, 0);
> +       frm_buf.fb_addr.dma_addr = vb2_dma_contig_plane_dma_addr(src_buf, 0);
> +       frm_buf.fb_addr.size = (unsigned int)src_buf->planes[0].length;
> +       frm_buf.fb_addr1.va = vb2_plane_vaddr(src_buf, 1);
> +       frm_buf.fb_addr1.dma_addr = vb2_dma_contig_plane_dma_addr(src_buf, 1);
> +       frm_buf.fb_addr1.size = (unsigned int)src_buf->planes[1].length;
> +       if (src_buf->num_planes == 3) {
> +               frm_buf.fb_addr2.va = vb2_plane_vaddr(src_buf, 2);
> +               frm_buf.fb_addr2.dma_addr =
> +                       vb2_dma_contig_plane_dma_addr(src_buf, 2);
> +               frm_buf.fb_addr2.size =
> +                       (unsigned int)src_buf->planes[2].length;
> +       } else {
> +               frm_buf.fb_addr2.va = NULL;
> +               frm_buf.fb_addr2.dma_addr = 0;
> +               frm_buf.fb_addr2.size = 0;
> +       }
> +       bs_buf.va = vb2_plane_vaddr(dst_buf, 0);
> +       bs_buf.dma_addr = vb2_dma_contig_plane_dma_addr(dst_buf, 0);
> +       bs_buf.size = (unsigned int)dst_buf->planes[0].length;
> +
> +       mtk_v4l2_debug(2,
> +                       "Framebuf VA=%p PA=%llx Size=0x%lx;VA=%p PA=0x%llx Size=0x%lx;VA=%p PA=0x%llx Size=0x%lx",
> +                       frm_buf.fb_addr.va,
> +                       (u64)frm_buf.fb_addr.dma_addr,
> +                       frm_buf.fb_addr.size,
> +                       frm_buf.fb_addr1.va,
> +                       (u64)frm_buf.fb_addr1.dma_addr,
> +                       frm_buf.fb_addr1.size,
> +                       frm_buf.fb_addr2.va,
> +                       (u64)frm_buf.fb_addr2.dma_addr,
> +                       frm_buf.fb_addr2.size);
> +
> +       ret = venc_if_encode(ctx, VENC_START_OPT_ENCODE_FRAME,
> +                            &frm_buf, &bs_buf, &enc_result);
> +
> +       src_buf = v4l2_m2m_src_buf_remove(ctx->m2m_ctx);
> +       if (enc_result.msg == VENC_MESSAGE_OK)
> +               v4l2_m2m_buf_done(to_vb2_v4l2_buffer(src_buf), VB2_BUF_STATE_DONE);
> +       else
> +               v4l2_m2m_buf_done(to_vb2_v4l2_buffer(src_buf), VB2_BUF_STATE_ERROR);
> +
> +       if (enc_result.is_key_frm) {
> +               v4l2_vb = to_vb2_v4l2_buffer(dst_buf);
> +               v4l2_vb->flags |= V4L2_BUF_FLAG_KEYFRAME;
> +       }
> +
> +       if (ret) {
> +               dst_buf->planes[0].bytesused = 0;
> +               v4l2_m2m_buf_done(to_vb2_v4l2_buffer(dst_buf), VB2_BUF_STATE_ERROR);
> +               mtk_v4l2_err("venc_if_encode failed=%d", ret);
> +       } else {
> +               dst_buf->planes[0].bytesused = enc_result.bs_size;
> +               v4l2_m2m_buf_done(to_vb2_v4l2_buffer(dst_buf), VB2_BUF_STATE_DONE);
> +               mtk_v4l2_debug(2, "venc_if_encode bs size=%d",
> +                                enc_result.bs_size);
> +       }
> +
> +       v4l2_m2m_job_finish(ctx->dev->m2m_dev_enc, ctx->m2m_ctx);
> +
> +       mtk_v4l2_debug(1, "<=== src_buf[%d] dst_buf[%d] venc_if_encode ret=%d Size=%u===>",
> +                       src_buf->index, dst_buf->index, ret,
> +                       enc_result.bs_size);
> +}
> +
> +static void m2mops_venc_device_run(void *priv)
> +{
> +       struct mtk_vcodec_ctx *ctx = priv;
> +       queue_work(ctx->dev->encode_workqueue, &ctx->encode_work);
> +}
> +
> +static int m2mops_venc_job_ready(void *m2m_priv)
> +{
> +       struct mtk_vcodec_ctx *ctx = m2m_priv;
> +
> +       if (!v4l2_m2m_num_dst_bufs_ready(ctx->m2m_ctx)) {
> +               mtk_v4l2_debug(3, "[%d]Not ready: not enough video dst buffers.",
> +                              ctx->idx);
> +               return 0;
> +       }
> +
> +       if (!v4l2_m2m_num_src_bufs_ready(ctx->m2m_ctx)) {
> +               mtk_v4l2_debug(3, "[%d]Not ready: not enough video src buffers.",
> +                              ctx->idx);
> +                       return 0;
> +               }
> +
> +       if (ctx->state == MTK_STATE_ABORT) {
> +               mtk_v4l2_debug(3, "[%d]Not ready: state=0x%x.",
> +                              ctx->idx, ctx->state);
> +               return 0;
> +       }
> +
> +       if (ctx->state == MTK_STATE_FREE) {
> +               mtk_v4l2_debug(3, "[%d]Not ready: state=0x%x.",
> +                              ctx->idx, ctx->state);
> +               return 0;
> +       }
> +
> +       return 1;
> +}
> +
> +static void m2mops_venc_job_abort(void *priv)
> +{
> +       struct mtk_vcodec_ctx *ctx = priv;
> +
> +       ctx->state = MTK_STATE_ABORT;
> +}
> +
> +static void m2mops_venc_lock(void *m2m_priv)
> +{
> +       struct mtk_vcodec_ctx *ctx = m2m_priv;
> +
> +       mutex_lock(&ctx->dev->dev_mutex);
> +}
> +
> +static void m2mops_venc_unlock(void *m2m_priv)
> +{
> +       struct mtk_vcodec_ctx *ctx = m2m_priv;
> +
> +       mutex_unlock(&ctx->dev->dev_mutex);
> +}
> +
> +const struct v4l2_m2m_ops mtk_venc_m2m_ops = {
> +       .device_run                     = m2mops_venc_device_run,
> +       .job_ready                      = m2mops_venc_job_ready,
> +       .job_abort                      = m2mops_venc_job_abort,
> +       .lock                           = m2mops_venc_lock,
> +       .unlock                         = m2mops_venc_unlock,
> +};
> +
> +#define IS_MTK_VENC_PRIV(x) ((V4L2_CTRL_ID2CLASS(x) == V4L2_CTRL_CLASS_MPEG) &&\
> +                            V4L2_CTRL_DRIVER_PRIV(x))
> +
> +void mtk_vcodec_enc_ctx_params_setup(struct mtk_vcodec_ctx *ctx)
> +{
> +       struct mtk_q_data *q_data;
> +       struct mtk_video_fmt *fmt;
> +
> +       ctx->m2m_ctx->q_lock = &ctx->dev->dev_mutex;
> +       ctx->fh.m2m_ctx = ctx->m2m_ctx;
> +       ctx->fh.ctrl_handler = &ctx->ctrl_hdl;
> +       INIT_WORK(&ctx->encode_work, mtk_venc_worker);
> +
> +       ctx->q_data[MTK_Q_DATA_SRC].width = DFT_CFG_WIDTH;
> +       ctx->q_data[MTK_Q_DATA_SRC].height = DFT_CFG_HEIGHT;
> +       ctx->q_data[MTK_Q_DATA_SRC].fmt = &mtk_video_formats[0];
> +       ctx->q_data[MTK_Q_DATA_SRC].colorspace = V4L2_COLORSPACE_REC709;
> +       ctx->q_data[MTK_Q_DATA_SRC].field = V4L2_FIELD_NONE;
> +
> +       q_data = &ctx->q_data[MTK_Q_DATA_SRC];
> +       fmt = ctx->q_data[MTK_Q_DATA_SRC].fmt;
> +       mtk_vcodec_enc_calc_src_size(fmt->num_planes, q_data->width,
> +                       q_data->height,
> +                       ctx->q_data[MTK_Q_DATA_SRC].sizeimage,
> +                       ctx->q_data[MTK_Q_DATA_SRC].bytesperline);
> +
> +       ctx->q_data[MTK_Q_DATA_DST].width = DFT_CFG_WIDTH;
> +       ctx->q_data[MTK_Q_DATA_DST].height = DFT_CFG_HEIGHT;
> +       ctx->q_data[MTK_Q_DATA_DST].fmt = &mtk_video_formats[9];
> +       ctx->q_data[MTK_Q_DATA_DST].colorspace = V4L2_COLORSPACE_REC709;
> +       ctx->q_data[MTK_Q_DATA_DST].field = V4L2_FIELD_NONE;
> +
> +       q_data = &ctx->q_data[MTK_Q_DATA_DST];
> +       fmt = ctx->q_data[MTK_Q_DATA_DST].fmt;
> +       ctx->q_data[MTK_Q_DATA_DST].sizeimage[0] = q_data->width * q_data->height;
> +       ctx->q_data[MTK_Q_DATA_DST].bytesperline[0] = 0;
> +
> +}
> +
> +int mtk_vcodec_enc_ctrls_setup(struct mtk_vcodec_ctx *ctx)
> +{
> +       const struct v4l2_ctrl_ops *ops = &mtk_vcodec_enc_ctrl_ops;
> +       struct v4l2_ctrl_handler *handler = &ctx->ctrl_hdl;
> +       struct v4l2_ctrl_config cfg;
> +
> +       v4l2_ctrl_handler_init(handler, MTK_MAX_CTRLS);
> +       if (handler->error) {
> +               mtk_v4l2_err("Init control handler fail %d\n",
> +                               handler->error);
> +               return handler->error;
> +       }
> +
> +       ctx->ctrls[0] = v4l2_ctrl_new_std(handler, ops,
> +                                       V4L2_CID_MPEG_VIDEO_BITRATE,
> +                                       1, 4000000, 1, 4000000);
> +       ctx->ctrls[1] = v4l2_ctrl_new_std(handler, ops,
> +                                       V4L2_CID_MPEG_VIDEO_B_FRAMES,
> +                                       0, 2, 1, 0);
> +       ctx->ctrls[2] = v4l2_ctrl_new_std(handler, ops,
> +                                       V4L2_CID_MPEG_VIDEO_FRAME_RC_ENABLE,
> +                                       0, 1, 1, 1);
> +       ctx->ctrls[3] = v4l2_ctrl_new_std(handler, ops,
> +                                       V4L2_CID_MPEG_VIDEO_H264_MAX_QP,
> +                                       0, 51, 1, 51);
> +       ctx->ctrls[4] = v4l2_ctrl_new_std(handler, ops,
> +                                       V4L2_CID_MPEG_VIDEO_H264_I_PERIOD,
> +                                       0, 65535, 1, 30);
> +       ctx->ctrls[5] = v4l2_ctrl_new_std(handler, ops,
> +                                       V4L2_CID_MPEG_VIDEO_GOP_SIZE,
> +                                       0, 65535, 1, 30);
> +       ctx->ctrls[6] = v4l2_ctrl_new_std(handler, ops,
> +                                       V4L2_CID_MPEG_VIDEO_MB_RC_ENABLE,
> +                                       0, 1, 1, 0);
> +       ctx->ctrls[7] = v4l2_ctrl_new_std_menu(handler, ops,
> +                                       V4L2_CID_MPEG_VIDEO_HEADER_MODE,
> +                                       V4L2_MPEG_VIDEO_HEADER_MODE_JOINED_WITH_1ST_FRAME,
> +                                       0, V4L2_MPEG_VIDEO_HEADER_MODE_SEPARATE);
> +       ctx->ctrls[8] = v4l2_ctrl_new_std_menu(handler, ops,
> +                                       V4L2_CID_MPEG_VIDEO_H264_PROFILE,
> +                                       V4L2_MPEG_VIDEO_H264_PROFILE_HIGH,
> +                                       0, V4L2_MPEG_VIDEO_H264_PROFILE_MAIN);
> +       ctx->ctrls[9] = v4l2_ctrl_new_std_menu(handler, ops,
> +                                       V4L2_CID_MPEG_VIDEO_H264_LEVEL,
> +                                       V4L2_MPEG_VIDEO_H264_LEVEL_4_2,
> +                                       0, V4L2_MPEG_VIDEO_H264_LEVEL_4_0);
> +       ctx->ctrls[6] = v4l2_ctrl_new_std(handler, ops,
> +                                       V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME,
> +                                       0, 0, 0, 0);
> +
> +       v4l2_ctrl_handler_setup(&ctx->ctrl_hdl);
> +
> +       return 0;
> +}
> +
> +int mtk_vcodec_enc_queue_init(void *priv, struct vb2_queue *src_vq,
> +                          struct vb2_queue *dst_vq)
> +{
> +       struct mtk_vcodec_ctx *ctx = priv;
> +       int ret;
> +
> +       src_vq->type            = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE;
> +       src_vq->io_modes        = VB2_DMABUF | VB2_MMAP | VB2_USERPTR;
> +       src_vq->drv_priv        = ctx;
> +       src_vq->buf_struct_size = sizeof(struct mtk_video_enc_buf);
> +       src_vq->ops             = &mtk_venc_vb2_ops;
> +       src_vq->mem_ops         = &vb2_dma_contig_memops;
> +       src_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
> +       src_vq->lock = &ctx->dev->dev_mutex;
> +
> +       ret = vb2_queue_init(src_vq);
> +       if (ret)
> +               return ret;
> +
> +       dst_vq->type            = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
> +       dst_vq->io_modes        = VB2_DMABUF | VB2_MMAP | VB2_USERPTR;
> +       dst_vq->drv_priv        = ctx;
> +       dst_vq->buf_struct_size = sizeof(struct v4l2_m2m_buffer);
> +       dst_vq->ops             = &mtk_venc_vb2_ops;
> +       dst_vq->mem_ops         = &vb2_dma_contig_memops;
> +       dst_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
> +       dst_vq->lock = &ctx->dev->dev_mutex;
> +
> +       return vb2_queue_init(dst_vq);
> +}
> +
> +int mtk_venc_unlock(struct mtk_vcodec_ctx *ctx)
> +{
> +       struct mtk_vcodec_dev *dev = ctx->dev;
> +       dev->curr_ctx = -1;
> +       mutex_unlock(&dev->enc_mutex);
> +       return 0;
> +}
> +
> +int mtk_venc_lock(struct mtk_vcodec_ctx *ctx)
> +{
> +       struct mtk_vcodec_dev *dev = ctx->dev;
> +
> +       mutex_lock(&dev->enc_mutex);
> +       dev->curr_ctx = ctx->idx;
> +       return 0;
> +}
> +
> +void mtk_vcodec_enc_release(struct mtk_vcodec_ctx *ctx)
> +{
> +       venc_if_release(ctx);
> +}
> diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h
> new file mode 100644
> index 0000000..e09524b
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h
> @@ -0,0 +1,46 @@
> +/*
> +* Copyright (c) 2015 MediaTek Inc.
> +* Author: PC Chen <pc.chen@mediatek.com>
> +*         Tiffany Lin <tiffany.lin@mediatek.com>
> +*
> +* This program is free software; you can redistribute it and/or modify
> +* it under the terms of the GNU General Public License version 2 as
> +* published by the Free Software Foundation.
> +*
> +* This program is distributed in the hope that it will be useful,
> +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +* GNU General Public License for more details.
> +*/
> +
> +
> +#ifndef _MTK_VCODEC_ENC_H_
> +#define _MTK_VCODEC_ENC_H_
> +
> +#include <media/videobuf2-core.h>
> +#include <media/videobuf2-v4l2.h>
> +
> +/**
> + * struct mtk_video_enc_buf - Private data related to each VB2 buffer.
> + * @b:                 Pointer to related VB2 buffer.
> + * @param_change:      Types of encode parameter change before encode this
> + *                     buffer
> + * @enc_params         Encode parameters changed before encode this buffer
> + */
> +struct mtk_video_enc_buf {
> +       struct vb2_v4l2_buffer  vb;
> +       struct list_head        list;
> +
> +       enum mtk_encode_param param_change;
> +       struct mtk_enc_params enc_params;
> +};
> +
> +int mtk_venc_unlock(struct mtk_vcodec_ctx *ctx);
> +int mtk_venc_lock(struct mtk_vcodec_ctx *ctx);
> +int mtk_vcodec_enc_queue_init(void *priv, struct vb2_queue *src_vq,
> +                                       struct vb2_queue *dst_vq);
> +void mtk_vcodec_enc_release(struct mtk_vcodec_ctx *ctx);
> +int mtk_vcodec_enc_ctrls_setup(struct mtk_vcodec_ctx *ctx);
> +void mtk_vcodec_enc_ctx_params_setup(struct mtk_vcodec_ctx *ctx);
> +
> +#endif /* _MTK_VCODEC_ENC_H_ */
> diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
> new file mode 100644
> index 0000000..e7ab14a
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
> @@ -0,0 +1,476 @@
> +/*
> +* Copyright (c) 2015 MediaTek Inc.
> +* Author: PC Chen <pc.chen@mediatek.com>
> +*         Tiffany Lin <tiffany.lin@mediatek.com>
> +*
> +* This program is free software; you can redistribute it and/or modify
> +* it under the terms of the GNU General Public License version 2 as
> +* published by the Free Software Foundation.
> +*
> +* This program is distributed in the hope that it will be useful,
> +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +* GNU General Public License for more details.
> +*/
> +
> +#include <linux/interrupt.h>
> +#include <linux/irq.h>
> +#include <linux/module.h>
> +#include <linux/of_device.h>
> +#include <linux/of.h>
> +#include <media/v4l2-event.h>
> +#include <media/v4l2-mem2mem.h>
> +#include <media/videobuf2-dma-contig.h>
> +#include <linux/pm_runtime.h>
> +
> +#include "mtk_vcodec_drv.h"
> +#include "mtk_vcodec_enc.h"
> +#include "mtk_vcodec_pm.h"
> +#include "mtk_vcodec_intr.h"
> +#include "mtk_vcodec_util.h"
> +#include "mtk_vpu.h"
> +
> +
> +/* Wake up context wait_queue */
> +static void wake_up_ctx(struct mtk_vcodec_ctx *ctx, unsigned int reason)
> +{
> +       ctx->int_cond = 1;
> +       ctx->int_type = reason;
> +       wake_up_interruptible(&ctx->queue);
> +}
> +
> +static irqreturn_t mtk_vcodec_enc_irq_handler(int irq, void *priv)
> +{
> +       struct mtk_vcodec_dev *dev = priv;
> +       struct mtk_vcodec_ctx *ctx;
> +       unsigned int irq_status;
> +
> +       if (dev->curr_ctx == -1) {
> +               mtk_v4l2_err("curr_ctx = -1");
> +               return IRQ_HANDLED;
> +       }
> +
> +       ctx = dev->ctx[dev->curr_ctx];
> +       if (ctx == NULL) {
> +               mtk_v4l2_err("curr_ctx==NULL");
> +               return IRQ_HANDLED;
> +       }
> +       mtk_v4l2_debug(1, "idx=%d", ctx->idx);
> +       irq_status = readl(dev->reg_base[VENC_SYS] +
> +                               (MTK_VENC_IRQ_STATUS_OFFSET));
> +       if (irq_status & MTK_VENC_IRQ_STATUS_PAUSE)
> +               writel((MTK_VENC_IRQ_STATUS_PAUSE),
> +                      dev->reg_base[VENC_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
> +
> +       if (irq_status & MTK_VENC_IRQ_STATUS_SWITCH)
> +               writel((MTK_VENC_IRQ_STATUS_SWITCH),
> +                      dev->reg_base[VENC_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
> +
> +       if (irq_status & MTK_VENC_IRQ_STATUS_DRAM)
> +               writel((MTK_VENC_IRQ_STATUS_DRAM),
> +                      dev->reg_base[VENC_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
> +
> +       if (irq_status & MTK_VENC_IRQ_STATUS_SPS)
> +               writel((MTK_VENC_IRQ_STATUS_SPS),
> +                      dev->reg_base[VENC_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
> +
> +       if (irq_status & MTK_VENC_IRQ_STATUS_PPS)
> +               writel((MTK_VENC_IRQ_STATUS_PPS),
> +                      dev->reg_base[VENC_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
> +
> +       if (irq_status & MTK_VENC_IRQ_STATUS_FRM)
> +               writel((MTK_VENC_IRQ_STATUS_FRM),
> +                      dev->reg_base[VENC_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
> +
> +       ctx->irq_status = irq_status;
> +       wake_up_ctx(ctx, MTK_INST_IRQ_RECEIVED);
> +       return IRQ_HANDLED;
> +}
> +
> +#if 1 /* VENC_LT */
> +static irqreturn_t mtk_vcodec_enc_irq_handler2(int irq, void *priv)
> +{
> +       struct mtk_vcodec_dev *dev = priv;
> +       struct mtk_vcodec_ctx *ctx;
> +       unsigned int irq_status;
> +
> +       ctx = dev->ctx[dev->curr_ctx];
> +       if (ctx == NULL) {
> +               mtk_v4l2_err("ctx==NULL");
> +               return IRQ_HANDLED;
> +       }
> +       mtk_v4l2_debug(1, "idx=%d", ctx->idx);
> +       irq_status = readl(dev->reg_base[VENC_LT_SYS] +
> +                               (MTK_VENC_IRQ_STATUS_OFFSET));
> +       if (irq_status & MTK_VENC_IRQ_STATUS_PAUSE)
> +               writel((MTK_VENC_IRQ_STATUS_PAUSE),
> +                      dev->reg_base[VENC_LT_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
> +
> +       if (irq_status & MTK_VENC_IRQ_STATUS_SWITCH)
> +               writel((MTK_VENC_IRQ_STATUS_SWITCH),
> +                      dev->reg_base[VENC_LT_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
> +
> +       if (irq_status & MTK_VENC_IRQ_STATUS_DRAM)
> +               writel((MTK_VENC_IRQ_STATUS_DRAM),
> +                      dev->reg_base[VENC_LT_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
> +
> +       if (irq_status & MTK_VENC_IRQ_STATUS_SPS)
> +               writel((MTK_VENC_IRQ_STATUS_SPS),
> +                      dev->reg_base[VENC_LT_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
> +
> +       if (irq_status & MTK_VENC_IRQ_STATUS_PPS)
> +               writel((MTK_VENC_IRQ_STATUS_PPS),
> +                      dev->reg_base[VENC_LT_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
> +
> +       if (irq_status & MTK_VENC_IRQ_STATUS_FRM)
> +               writel((MTK_VENC_IRQ_STATUS_FRM),
> +                      dev->reg_base[VENC_LT_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
> +
> +       ctx->irq_status = irq_status;
> +       wake_up_ctx(ctx, MTK_INST_IRQ_RECEIVED);
> +       return IRQ_HANDLED;
> +}
> +#endif
> +
> +static void mtk_vcodec_enc_reset_handler(void *priv)
> +{
> +       int i;
> +       struct mtk_vcodec_dev *dev = priv;
> +       struct mtk_vcodec_ctx *ctx;
> +
> +       mtk_v4l2_debug(0, "Watchdog timeout!!");
> +
> +       mutex_lock(&dev->dev_mutex);
> +       for(i = 0; i < MTK_VCODEC_MAX_ENCODER_INSTANCES; i++) {
> +               ctx = dev->ctx[i];
> +               if (ctx) {
> +                       ctx->state = MTK_STATE_ABORT;
> +                       mtk_v4l2_debug(0, "[%d] Change to state MTK_STATE_ERROR", ctx->idx);
> +               }
> +
> +       }
> +       mutex_unlock(&dev->dev_mutex);
> +}
> +
> +static int fops_vcodec_open(struct file *file)
> +{
> +       struct video_device *vfd = video_devdata(file);
> +       struct mtk_vcodec_dev *dev = video_drvdata(file);
> +       struct mtk_vcodec_ctx *ctx = NULL;
> +       int ret = 0;
> +
> +       mutex_lock(&dev->dev_mutex);
> +
> +       ctx = devm_kzalloc(&dev->plat_dev->dev, sizeof(*ctx), GFP_KERNEL);
> +       if (!ctx) {
> +               ret = -ENOMEM;
> +               goto err_alloc;
> +       }
> +
> +       if (dev->num_instances >= MTK_VCODEC_MAX_ENCODER_INSTANCES) {
> +               mtk_v4l2_err("Too many open contexts\n");
> +               ret = -EBUSY;
> +               goto err_no_ctx;
> +       }
> +
> +       ctx->idx = ffz(dev->instance_mask[0]);
> +       v4l2_fh_init(&ctx->fh, video_devdata(file));
> +       file->private_data = &ctx->fh;
> +       v4l2_fh_add(&ctx->fh);
> +       ctx->dev = dev;
> +
> +       if (vfd == dev->vfd_enc) {
> +               ctx->type = MTK_INST_ENCODER;
> +
> +               ret = mtk_vcodec_enc_ctrls_setup(ctx);
> +               if (ret) {
> +                       mtk_v4l2_err("Failed to setup controls() (%d)\n",
> +                                      ret);
> +                       goto err_ctrls_setup;
> +               }
> +               ctx->m2m_ctx = v4l2_m2m_ctx_init(dev->m2m_dev_enc, ctx,
> +                                                &mtk_vcodec_enc_queue_init);
> +               if (IS_ERR(ctx->m2m_ctx)) {
> +                       ret = PTR_ERR(ctx->m2m_ctx);
> +                       mtk_v4l2_err("Failed to v4l2_m2m_ctx_init() (%d)\n",
> +                                      ret);
> +                       goto err_ctx_init;
> +               }
> +               mtk_vcodec_enc_ctx_params_setup(ctx);
> +       } else {
> +               mtk_v4l2_err("Invalid vfd !\n");
> +               ret = -ENOENT;
> +               goto err_ctx_init;
> +       }
> +
> +       init_waitqueue_head(&ctx->queue);
> +       dev->num_instances++;
> +
> +       if (dev->num_instances == 1) {
> +               ret = vpu_load_firmware(dev->vpu_plat_dev);
> +               if (ret < 0) {
> +                               mtk_v4l2_err("vpu_load_firmware failed!\n");
> +                       goto err_load_fw;
> +               }
> +
> +               dev->enc_capability =
> +                       vpu_get_venc_hw_capa(dev->vpu_plat_dev);
> +               mtk_v4l2_debug(0, "encoder capability %x", dev->enc_capability);
> +       }
> +
> +       mtk_v4l2_debug(2, "Create instance [%d]@%p m2m_ctx=%p type=%d\n",
> +                        ctx->idx, ctx, ctx->m2m_ctx, ctx->type);
> +       set_bit(ctx->idx, &dev->instance_mask[0]);
> +       dev->ctx[ctx->idx] = ctx;
> +
> +       mutex_unlock(&dev->dev_mutex);
> +       mtk_v4l2_debug(0, "%s encoder [%d]", dev_name(&dev->plat_dev->dev), ctx->idx);
> +       return ret;
> +
> +       /* Deinit when failure occurred */
> +err_load_fw:
> +       v4l2_m2m_ctx_release(ctx->m2m_ctx);
> +       v4l2_fh_del(&ctx->fh);
> +       v4l2_fh_exit(&ctx->fh);
> +       dev->num_instances--;
> +err_ctx_init:
> +err_ctrls_setup:
> +       v4l2_ctrl_handler_free(&ctx->ctrl_hdl);
> +err_no_ctx:
> +       devm_kfree(&dev->plat_dev->dev, ctx);
> +err_alloc:
> +       mutex_unlock(&dev->dev_mutex);
> +       return ret;
> +}
> +
> +static int fops_vcodec_release(struct file *file)
> +{
> +       struct mtk_vcodec_dev *dev = video_drvdata(file);
> +       struct mtk_vcodec_ctx *ctx = fh_to_ctx(file->private_data);
> +
> +       mtk_v4l2_debug(0, "[%d] encoder\n", ctx->idx);
> +       mutex_lock(&dev->dev_mutex);
> +
> +       mtk_vcodec_enc_release(ctx);
> +       v4l2_fh_del(&ctx->fh);
> +       v4l2_fh_exit(&ctx->fh);
> +       v4l2_ctrl_handler_free(&ctx->ctrl_hdl);
> +       v4l2_m2m_ctx_release(ctx->m2m_ctx);
> +
> +       dev->ctx[ctx->idx] = NULL;
> +       dev->num_instances--;
> +       clear_bit(ctx->idx, &dev->instance_mask[0]);
> +       devm_kfree(&dev->plat_dev->dev, ctx);
> +       mutex_unlock(&dev->dev_mutex);
> +       return 0;
> +}
> +
> +static const struct v4l2_file_operations mtk_vcodec_fops = {
> +       .owner                          = THIS_MODULE,
> +       .open                           = fops_vcodec_open,
> +       .release                        = fops_vcodec_release,
> +       .poll                           = v4l2_m2m_fop_poll,
> +       .unlocked_ioctl                 = video_ioctl2,
> +       .mmap                           = v4l2_m2m_fop_mmap,
> +};
> +
> +static int mtk_vcodec_probe(struct platform_device *pdev)
> +{
> +       struct mtk_vcodec_dev *dev;
> +       struct video_device *vfd_enc;
> +       struct resource *res;
> +       int i, j, ret;
> +
> +       dev = devm_kzalloc(&pdev->dev, sizeof(*dev), GFP_KERNEL);
> +       if (!dev)
> +               return -ENOMEM;
> +
> +       dev->plat_dev = pdev;
> +
> +       dev->vpu_plat_dev = vpu_get_plat_device(dev->plat_dev);
> +       if (dev->vpu_plat_dev == NULL) {
> +               mtk_v4l2_err("[VPU] vpu device in not ready\n");
> +               return -EPROBE_DEFER;
> +       }
> +
> +       vpu_wdt_reg_handler(dev->vpu_plat_dev, mtk_vcodec_enc_reset_handler, dev,
> +                           VPU_RST_ENC);
> +
> +       ret = mtk_vcodec_init_enc_pm(dev);
> +       if (ret < 0) {
> +               dev_err(&pdev->dev, "Failed to get mt vcodec clock source!\n");
> +               return ret;
> +       }
> +
> +       for (i = VENC_SYS, j = 0; i < NUM_MAX_VCODEC_REG_BASE; i++, j++) {
> +               res = platform_get_resource(pdev, IORESOURCE_MEM, j);
> +               if (res == NULL) {
> +                       dev_err(&pdev->dev, "get memory resource failed.\n");
> +                       ret = -ENXIO;
> +                       goto err_res;
> +               }
> +               dev->reg_base[i] = devm_ioremap_resource(&pdev->dev, res);
> +               if (IS_ERR(dev->reg_base[i])) {
> +                       dev_err(&pdev->dev,
> +                               "devm_ioremap_resource %d failed.\n", i);
> +                       ret = PTR_ERR(dev->reg_base);
> +                       goto err_res;
> +               }
> +               mtk_v4l2_debug(2, "reg[%d] base=0x%p\n", i, dev->reg_base[i]);
> +       }
> +
> +       res = platform_get_resource(pdev, IORESOURCE_IRQ, 0);
> +       if (res == NULL) {
> +               dev_err(&pdev->dev, "failed to get irq resource\n");
> +               ret = -ENOENT;
> +               goto err_res;
> +       }
> +
> +       dev->enc_irq = platform_get_irq(pdev, 0);
> +       ret = devm_request_irq(&pdev->dev, dev->enc_irq,
> +                              mtk_vcodec_enc_irq_handler,
> +                              0, pdev->name, dev);
> +       if (ret) {
> +               dev_err(&pdev->dev, "Failed to install dev->enc_irq %d (%d)\n",
> +                       dev->enc_irq,
> +                       ret);
> +               ret = -EINVAL;
> +               goto err_res;
> +       }
> +
> +       dev->enc_lt_irq = platform_get_irq(pdev, 1);
> +       ret = devm_request_irq(&pdev->dev,
> +                              dev->enc_lt_irq, mtk_vcodec_enc_irq_handler2,
> +                              0, pdev->name, dev);
> +       if (ret) {
> +               dev_err(&pdev->dev,
> +                       "Failed to install dev->enc_lt_irq %d (%d)\n",
> +                       dev->enc_lt_irq, ret);
> +               ret = -EINVAL;
> +               goto err_res;
> +       }
> +
> +       disable_irq(dev->enc_irq);
> +       disable_irq(dev->enc_lt_irq); /* VENC_LT */
> +       mutex_init(&dev->enc_mutex);
> +       mutex_init(&dev->dev_mutex);
> +
> +       snprintf(dev->v4l2_dev.name, sizeof(dev->v4l2_dev.name), "%s",
> +                "[MTK_V4L2_VENC]");
> +
> +       ret = v4l2_device_register(&pdev->dev, &dev->v4l2_dev);
> +       if (ret) {
> +               mtk_v4l2_err("v4l2_device_register err=%d\n", ret);
> +               return ret;
> +       }
> +
> +       init_waitqueue_head(&dev->queue);
> +
> +       /* allocate video device for encoder and register it */
> +       vfd_enc = video_device_alloc();
> +       if (!vfd_enc) {
> +               mtk_v4l2_err("Failed to allocate video device\n");
> +               ret = -ENOMEM;
> +               goto err_enc_alloc;
> +       }
> +       vfd_enc->fops           = &mtk_vcodec_fops;
> +       vfd_enc->ioctl_ops      = &mtk_venc_ioctl_ops;
> +       vfd_enc->release        = video_device_release;
> +       vfd_enc->lock           = &dev->dev_mutex;
> +       vfd_enc->v4l2_dev       = &dev->v4l2_dev;
> +       vfd_enc->vfl_dir        = VFL_DIR_M2M;
> +
> +       snprintf(vfd_enc->name, sizeof(vfd_enc->name), "%s",
> +                MTK_VCODEC_ENC_NAME);
> +       video_set_drvdata(vfd_enc, dev);
> +       dev->vfd_enc = vfd_enc;
> +       platform_set_drvdata(pdev, dev);
> +       ret = video_register_device(vfd_enc, VFL_TYPE_GRABBER, 1);
> +       if (ret) {
> +               mtk_v4l2_err("Failed to register video device\n");
> +               goto err_enc_reg;
> +       }
> +       mtk_v4l2_debug(0, "encoder registered as /dev/video%d\n",
> +                        vfd_enc->num);
> +
> +       dev->alloc_ctx = vb2_dma_contig_init_ctx(&pdev->dev);
> +       if (IS_ERR(dev->alloc_ctx)) {
> +               mtk_v4l2_err("Failed to alloc vb2 dma context 0\n");
> +               ret = PTR_ERR(dev->alloc_ctx);
> +               goto err_vb2_ctx_init;
> +       }
> +
> +       dev->m2m_dev_enc = v4l2_m2m_init(&mtk_venc_m2m_ops);
> +       if (IS_ERR(dev->m2m_dev_enc)) {
> +               mtk_v4l2_err("Failed to init mem2mem enc device\n");
> +               ret = PTR_ERR(dev->m2m_dev_enc);
> +               goto err_enc_mem_init;
> +       }
> +
> +       dev->encode_workqueue =
> +                       alloc_ordered_workqueue(MTK_VCODEC_ENC_NAME, WQ_MEM_RECLAIM | WQ_FREEZABLE);
> +       if (!dev->encode_workqueue) {
> +               mtk_v4l2_err("Failed to create encode workqueue\n");
> +               ret = -EINVAL;
> +               goto err_event_workq;
> +       }
> +
> +       return 0;
> +
> +err_event_workq:
> +       v4l2_m2m_release(dev->m2m_dev_enc);
> +err_enc_mem_init:
> +       vb2_dma_contig_cleanup_ctx(dev->alloc_ctx);
> +err_vb2_ctx_init:
> +       video_unregister_device(vfd_enc);
> +err_enc_reg:
> +       video_device_release(vfd_enc);
> +err_enc_alloc:
> +       v4l2_device_unregister(&dev->v4l2_dev);
> +err_res:
> +       mtk_vcodec_release_enc_pm(dev);
> +       return ret;
> +}
> +
> +static const struct of_device_id mtk_vcodec_match[] = {
> +       {.compatible = "mediatek,mt8173-vcodec-enc",},
> +       {},
> +};
> +MODULE_DEVICE_TABLE(of, mtk_vcodec_match);
> +
> +static int mtk_vcodec_remove(struct platform_device *pdev)
> +{
> +       struct mtk_vcodec_dev *dev = platform_get_drvdata(pdev);
> +
> +       mtk_v4l2_debug_enter();
> +       flush_workqueue(dev->encode_workqueue);
> +       destroy_workqueue(dev->encode_workqueue);
> +       if (dev->m2m_dev_enc)
> +               v4l2_m2m_release(dev->m2m_dev_enc);
> +       if (dev->alloc_ctx)
> +               vb2_dma_contig_cleanup_ctx(dev->alloc_ctx);
> +
> +       if (dev->vfd_enc) {
> +               video_unregister_device(dev->vfd_enc);
> +               video_device_release(dev->vfd_enc);
> +       }
> +       v4l2_device_unregister(&dev->v4l2_dev);
> +       mtk_vcodec_release_enc_pm(dev);
> +       return 0;
> +}
> +
> +static struct platform_driver mtk_vcodec_driver = {
> +       .probe  = mtk_vcodec_probe,
> +       .remove = mtk_vcodec_remove,
> +       .driver = {
> +               .name   = MTK_VCODEC_ENC_NAME,
> +               .owner  = THIS_MODULE,
> +               .of_match_table = mtk_vcodec_match,
> +       },
> +};
> +
> +module_platform_driver(mtk_vcodec_driver);
> +
> +
> +MODULE_LICENSE("GPL v2");
> +MODULE_DESCRIPTION("Mediatek video codec V4L2 driver");
> diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
> new file mode 100644
> index 0000000..518fba7
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
> @@ -0,0 +1,132 @@
> +/*
> +* Copyright (c) 2015 MediaTek Inc.
> +* Author: Tiffany Lin <tiffany.lin@mediatek.com>
> +*
> +* This program is free software; you can redistribute it and/or modify
> +* it under the terms of the GNU General Public License version 2 as
> +* published by the Free Software Foundation.
> +*
> +* This program is distributed in the hope that it will be useful,
> +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +* GNU General Public License for more details.
> +*/
> +
> +#include <linux/clk.h>
> +#include <linux/of_address.h>
> +#include <linux/of_platform.h>
> +#include <linux/pm_runtime.h>
> +#include <soc/mediatek/smi.h>
> +
> +#include "mtk_vcodec_pm.h"
> +#include "mtk_vcodec_util.h"
> +#include "mtk_vpu.h"
> +
> +
> +int mtk_vcodec_init_enc_pm(struct mtk_vcodec_dev *mtkdev)
> +{
> +       struct device_node *node;
> +       struct platform_device *pdev;
> +       struct device *dev;
> +       struct mtk_vcodec_pm *pm;
> +       int ret = 0;
> +
> +       pdev = mtkdev->plat_dev;
> +       pm = &mtkdev->pm;
> +       memset(pm, 0, sizeof(struct mtk_vcodec_pm));
> +       pm->mtkdev = mtkdev;
> +       dev = &pdev->dev;
> +
> +       node = of_parse_phandle(dev->of_node, "mediatek,larb", 0);
> +       if (!node)
> +               return -1;
> +       pdev = of_find_device_by_node(node);
> +       if (WARN_ON(!pdev)) {
> +               of_node_put(node);
> +               return -1;
> +       }
> +       pm->larbvenc = &pdev->dev;
> +
> +       node = of_parse_phandle(dev->of_node, "mediatek,larb", 1);
> +       if (!node)
> +               return -1;
> +
> +       pdev = of_find_device_by_node(node);
> +       if (WARN_ON(!pdev)) {
> +               of_node_put(node);
> +               return -EINVAL;
> +       }
> +       pm->larbvenclt = &pdev->dev;
> +
> +       pdev = mtkdev->plat_dev;
> +       pm->dev = &pdev->dev;
> +
> +       pm->vencpll_d2 = devm_clk_get(&pdev->dev, "vencpll_d2");
> +       if (pm->vencpll_d2 == NULL) {
> +               mtk_v4l2_err("devm_clk_get vencpll_d2 fail");
> +               ret = -1;
> +       }
> +
> +       pm->venc_sel = devm_clk_get(&pdev->dev, "venc_sel");
> +       if (pm->venc_sel == NULL) {
> +               mtk_v4l2_err("devm_clk_get venc_sel fail");
> +               ret = -1;
> +       }
> +
> +       pm->univpll1_d2 = devm_clk_get(&pdev->dev, "univpll1_d2");
> +       if (pm->univpll1_d2 == NULL) {
> +               mtk_v4l2_err("devm_clk_get univpll1_d2 fail");
> +               ret = -1;
> +       }
> +
> +       pm->venc_lt_sel = devm_clk_get(&pdev->dev, "venc_lt_sel");
> +       if (pm->venc_lt_sel == NULL) {
> +               mtk_v4l2_err("devm_clk_get venc_lt_sel fail");
> +               ret = -1;
> +       }
> +
> +       return ret;
> +}
> +
> +void mtk_vcodec_release_enc_pm(struct mtk_vcodec_dev *mtkdev)
> +{
> +}
> +
> +
> +void mtk_vcodec_enc_clock_on(struct mtk_vcodec_pm *pm)
> +{
> +       int ret;
> +
> +       ret = clk_prepare_enable(pm->venc_sel);
> +       if (ret)
> +               mtk_v4l2_err("venc_sel fail %d", ret);
> +
> +       ret = clk_set_parent(pm->venc_sel, pm->vencpll_d2);
> +       if (ret)
> +               mtk_v4l2_err("clk_set_parent fail %d", ret);
> +
> +       ret = clk_prepare_enable(pm->venc_lt_sel);
> +       if (ret)
> +               mtk_v4l2_err("venc_lt_sel fail %d", ret);
> +
> +       ret = clk_set_parent(pm->venc_lt_sel, pm->univpll1_d2);
> +       if (ret)
> +               mtk_v4l2_err("clk_set_parent fail %d", ret);
> +
> +       ret = mtk_smi_larb_get(pm->larbvenc);
> +       if (ret)
> +               mtk_v4l2_err("mtk_smi_larb_get larb3 fail %d\n", ret);
> +
> +       ret = mtk_smi_larb_get(pm->larbvenclt);
> +       if (ret)
> +               mtk_v4l2_err("mtk_smi_larb_get larb4 fail %d\n", ret);
> +
> +}
> +
> +void mtk_vcodec_enc_clock_off(struct mtk_vcodec_pm *pm)
> +{
> +       mtk_smi_larb_put(pm->larbvenc);
> +       mtk_smi_larb_put(pm->larbvenclt);
> +       clk_disable_unprepare(pm->venc_lt_sel);
> +       clk_disable_unprepare(pm->venc_sel);
> +}
> diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.c
> new file mode 100644
> index 0000000..919b949
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.c
> @@ -0,0 +1,102 @@
> +/*
> +* Copyright (c) 2015 MediaTek Inc.
> +* Author: Tiffany Lin <tiffany.lin@mediatek.com>
> +*
> +* This program is free software; you can redistribute it and/or modify
> +* it under the terms of the GNU General Public License version 2 as
> +* published by the Free Software Foundation.
> +*
> +* This program is distributed in the hope that it will be useful,
> +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +* GNU General Public License for more details.
> +*/
> +
> +#include <linux/errno.h>
> +#include <linux/wait.h>
> +
> +#include "mtk_vcodec_drv.h"
> +#include "mtk_vcodec_intr.h"
> +#include "mtk_vcodec_util.h"
> +
> +void mtk_vcodec_clean_dev_int_flags(void *data)
> +{
> +       struct mtk_vcodec_dev *dev = (struct mtk_vcodec_dev *)data;
> +
> +       dev->int_cond = 0;
> +       dev->int_type = 0;
> +}
> +
> +int mtk_vcodec_wait_for_done_ctx(void *data, int command,
> +                                unsigned int timeout_ms, int interrupt)
> +{
> +       wait_queue_head_t *waitqueue;
> +       long timeout_jiff, ret;
> +       int status = 0;
> +       struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)data;
> +
> +       waitqueue = (wait_queue_head_t *)&ctx->queue;
> +       timeout_jiff = msecs_to_jiffies(timeout_ms);
> +       if (interrupt) {
> +               ret = wait_event_interruptible_timeout(*waitqueue,
> +                               (ctx->int_cond &&
> +                               (ctx->int_type == command)),
> +                               timeout_jiff);
> +       } else {
> +               ret = wait_event_timeout(*waitqueue,
> +                               (ctx->int_cond &&
> +                               (ctx->int_type == command)),
> +                                timeout_jiff);
> +       }
> +       if (0 == ret) {
> +               status = -1;    /* timeout */
> +               mtk_v4l2_err("[%d] cmd=%d, ctx->type=%d, wait_event_interruptible_timeout time=%ums out %d %d!",
> +                               ctx->idx, ctx->type, command, timeout_ms,
> +                               ctx->int_cond, ctx->int_type);
> +       } else if (-ERESTARTSYS == ret) {
> +               mtk_v4l2_err("[%d] cmd=%d, ctx->type=%d, wait_event_interruptible_timeout interrupted by a signal %d %d",
> +                               ctx->idx, ctx->type, command, ctx->int_cond,
> +                               ctx->int_type);
> +               status = -1;
> +       }
> +
> +       ctx->int_cond = 0;
> +       ctx->int_type = 0;
> +
> +       return status;
> +}
> +
> +int mtk_vcodec_wait_for_done_dev(void *data, int command,
> +                                unsigned int timeout, int interrupt)
> +{
> +       wait_queue_head_t *waitqueue;
> +       long timeout_jiff, ret;
> +       int status = 0;
> +       struct mtk_vcodec_dev *dev = (struct mtk_vcodec_dev *)data;
> +
> +       waitqueue = (wait_queue_head_t *)&dev->queue;
> +       timeout_jiff = msecs_to_jiffies(timeout);
> +       if (interrupt) {
> +               ret = wait_event_interruptible_timeout(*waitqueue,
> +                               (dev->int_cond &&
> +                               (dev->int_type == command)),
> +                               timeout_jiff);
> +       } else {
> +               ret = wait_event_timeout(*waitqueue,
> +                               (dev->int_cond &&
> +                               (dev->int_type == command)),
> +                               timeout_jiff);
> +       }
> +       if (0 == ret) {
> +               status = -1;    /* timeout */
> +               mtk_v4l2_err("wait_event_interruptible_timeout time=%lu out %d %d!",
> +                               timeout_jiff, dev->int_cond, dev->int_type);
> +       } else if (-ERESTARTSYS == ret) {
> +               mtk_v4l2_err("wait_event_interruptible_timeout interrupted by a signal %d %d",
> +                               dev->int_cond, dev->int_type);
> +               status = -1;
> +       }
> +       dev->int_cond = 0;
> +       dev->int_type = 0;
> +       return status;
> +}
> diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.h
> new file mode 100644
> index 0000000..e9b7f94
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.h
> @@ -0,0 +1,29 @@
> +/*
> +* Copyright (c) 2015 MediaTek Inc.
> +* Author: Tiffany Lin <tiffany.lin@mediatek.com>
> +*
> +* This program is free software; you can redistribute it and/or modify
> +* it under the terms of the GNU General Public License version 2 as
> +* published by the Free Software Foundation.
> +*
> +* This program is distributed in the hope that it will be useful,
> +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +* GNU General Public License for more details.
> +*/
> +
> +#ifndef _MTK_VCODEC_INTR_H_
> +#define _MTK_VCODEC_INTR_H_
> +
> +#define MTK_INST_IRQ_RECEIVED          0x1
> +#define MTK_INST_WORK_THREAD_ABORT_DONE        0x2
> +
> +/* timeout is ms */
> +int mtk_vcodec_wait_for_done_ctx(void *data, int command, unsigned int timeout,
> +                                int interrupt);
> +int mtk_vcodec_wait_for_done_dev(void *data, int command, unsigned int timeout,
> +                                int interrupt);
> +
> +void mtk_vcodec_clean_dev_int_flags(void *data);
> +
> +#endif /* _MTK_VCODEC_INTR_H_ */
> diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_pm.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_pm.h
> new file mode 100644
> index 0000000..fdadec9
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_pm.h
> @@ -0,0 +1,26 @@
> +/*
> +* Copyright (c) 2015 MediaTek Inc.
> +* Author: Tiffany Lin <tiffany.lin@mediatek.com>
> +*
> +* This program is free software; you can redistribute it and/or modify
> +* it under the terms of the GNU General Public License version 2 as
> +* published by the Free Software Foundation.
> +*
> +* This program is distributed in the hope that it will be useful,
> +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +* GNU General Public License for more details.
> +*/
> +
> +#ifndef _MTK_VCODEC_PM_H_
> +#define _MTK_VCODEC_PM_H_
> +
> +#include "mtk_vcodec_drv.h"
> +
> +int mtk_vcodec_init_enc_pm(struct mtk_vcodec_dev *dev);
> +void mtk_vcodec_release_enc_pm(struct mtk_vcodec_dev *dev);
> +
> +void mtk_vcodec_enc_clock_on(struct mtk_vcodec_pm *pm);
> +void mtk_vcodec_enc_clock_off(struct mtk_vcodec_pm *pm);
> +
> +#endif /* _MTK_VCODEC_PM_H_ */
> diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c
> new file mode 100644
> index 0000000..3fede8d
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c
> @@ -0,0 +1,106 @@
> +/*
> +* Copyright (c) 2015 MediaTek Inc.
> +* Author: PC Chen <pc.chen@mediatek.com>
> +*         Tiffany Lin <tiffany.lin@mediatek.com>
> +*
> +* This program is free software; you can redistribute it and/or modify
> +* it under the terms of the GNU General Public License version 2 as
> +* published by the Free Software Foundation.
> +*
> +* This program is distributed in the hope that it will be useful,
> +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +* GNU General Public License for more details.
> +*/
> +
> +#include <linux/module.h>
> +
> +#include "mtk_vcodec_drv.h"
> +#include "mtk_vcodec_util.h"
> +#include "mtk_vpu.h"
> +
> +bool mtk_vcodec_dbg = false;
> +int mtk_v4l2_dbg_level = 0;
> +
> +module_param(mtk_v4l2_dbg_level, int, S_IRUGO | S_IWUSR);
> +module_param(mtk_vcodec_dbg, bool, S_IRUGO | S_IWUSR);
> +
> +void __iomem *mtk_vcodec_get_reg_addr(void *data, unsigned int reg_idx)
> +{
> +       struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)data;
> +
> +       if (!data || reg_idx >= NUM_MAX_VCODEC_REG_BASE) {
> +               mtk_v4l2_err("Invalid arguments");
> +               return NULL;
> +       }
> +       return ctx->dev->reg_base[reg_idx];
> +}
> +
> +int mtk_vcodec_mem_alloc(void *data, struct mtk_vcodec_mem *mem)
> +{
> +       unsigned long size = mem->size;
> +       struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)data;
> +       struct device *dev = &ctx->dev->plat_dev->dev;
> +
> +       mem->va = dma_alloc_coherent(dev, size, &mem->dma_addr, GFP_KERNEL);
> +
> +       if (!mem->va) {
> +               mtk_v4l2_err("%s dma_alloc size=%ld failed!", dev_name(dev),
> +                              size);
> +               return -ENOMEM;
> +       }
> +
> +       memset(mem->va, 0, size);
> +
> +       mtk_v4l2_debug(3, "[%d]  - va      = %p", ctx->idx, mem->va);
> +       mtk_v4l2_debug(3, "[%d]  - dma     = 0x%lx", ctx->idx,
> +                        (unsigned long)mem->dma_addr);
> +       mtk_v4l2_debug(3, "[%d]    size = 0x%lx", ctx->idx, size);
> +
> +       return 0;
> +}
> +
> +void mtk_vcodec_mem_free(void *data, struct mtk_vcodec_mem *mem)
> +{
> +       unsigned long size = mem->size;
> +       struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)data;
> +       struct device *dev = &ctx->dev->plat_dev->dev;
> +
> +       dma_free_coherent(dev, size, mem->va, mem->dma_addr);
> +       mem->va = NULL;
> +
> +       mtk_v4l2_debug(3, "[%d]  - va      = %p", ctx->idx, mem->va);
> +       mtk_v4l2_debug(3, "[%d]  - dma     = 0x%lx", ctx->idx,
> +                        (unsigned long)mem->dma_addr);
> +       mtk_v4l2_debug(3, "[%d]    size = 0x%lx", ctx->idx, size);
> +}
> +
> +int mtk_vcodec_get_ctx_id(void *data)
> +{
> +       struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)data;
> +
> +       if (!ctx)
> +               return -1;
> +
> +       return ctx->idx;
> +}
> +
> +struct platform_device *mtk_vcodec_get_plat_dev(void *data)
> +{
> +       struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)data;
> +
> +       if (!ctx)
> +               return NULL;
> +
> +       return vpu_get_plat_device(ctx->dev->plat_dev);
> +}
> +
> +void mtk_vcodec_fmt2str(u32 fmt, char *str)
> +{
> +       char a = fmt & 0xFF;
> +       char b = (fmt >> 8) & 0xFF;
> +       char c = (fmt >> 16) & 0xFF;
> +       char d = (fmt >> 24) & 0xFF;
> +
> +       sprintf(str, "%c%c%c%c", a, b, c, d);
> +}
> diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
> new file mode 100644
> index 0000000..47016ae
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
> @@ -0,0 +1,85 @@
> +/*
> +* Copyright (c) 2015 MediaTek Inc.
> +* Author: PC Chen <pc.chen@mediatek.com>
> +*         Tiffany Lin <tiffany.lin@mediatek.com>
> +*
> +* This program is free software; you can redistribute it and/or modify
> +* it under the terms of the GNU General Public License version 2 as
> +* published by the Free Software Foundation.
> +*
> +* This program is distributed in the hope that it will be useful,
> +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +* GNU General Public License for more details.
> +*/
> +
> +#ifndef _MTK_VCODEC_UTIL_H_
> +#define _MTK_VCODEC_UTIL_H_
> +
> +#include <linux/types.h>
> +#include <linux/dma-direction.h>
> +
> +struct mtk_vcodec_mem {
> +       size_t size;
> +       void *va;
> +       dma_addr_t dma_addr;
> +};
> +
> +extern int mtk_v4l2_dbg_level;
> +extern bool mtk_vcodec_dbg;
> +
> +#define DEBUG  1
> +
> +#if defined(DEBUG)
> +
> +#define mtk_v4l2_debug(level, fmt, args...)                             \
> +       do {                                                             \
> +               if (mtk_v4l2_dbg_level >= level)                         \
> +                       pr_info("[MTK_V4L2] level=%d %s(),%d: " fmt "\n",\
> +                               level, __func__, __LINE__, ##args);      \
> +       } while (0)
> +
> +#define mtk_v4l2_err(fmt, args...)                \
> +       pr_err("[MTK_V4L2][ERROR] %s:%d: " fmt "\n", __func__, __LINE__, \
> +              ##args)
> +
> +
> +#define mtk_v4l2_debug_enter()  mtk_v4l2_debug(3, "+\n")
> +#define mtk_v4l2_debug_leave()  mtk_v4l2_debug(3, "-\n")
> +
> +#define mtk_vcodec_debug(h, fmt, args...)                              \
> +       do {                                                            \
> +               if (mtk_vcodec_dbg)                                     \
> +                       pr_info("[MTK_VCODEC][%d]: %s() " fmt "\n",     \
> +                               ((struct mtk_vcodec_ctx *)h->ctx)->idx, \
> +                               __func__, ##args);                      \
> +       } while (0)
> +
> +#define mtk_vcodec_err(h, fmt, args...)                                        \
> +       pr_err("[MTK_VCODEC][ERROR][%d]: %s() " fmt "\n",               \
> +              ((struct mtk_vcodec_ctx *)h->ctx)->idx, __func__, ##args)
> +
> +#define mtk_vcodec_debug_enter(h)  mtk_vcodec_debug(h, "+\n")
> +#define mtk_vcodec_debug_leave(h)  mtk_vcodec_debug(h, "-\n")
> +
> +#else
> +
> +#define mtk_v4l2_debug(level, fmt, args...)
> +#define mtk_v4l2_err(fmt, args...)
> +#define mtk_v4l2_debug_enter()
> +#define mtk_v4l2_debug_leave()
> +
> +#define mtk_vcodec_debug(h, fmt, args...)
> +#define mtk_vcodec_err(h, fmt, args...)
> +#define mtk_vcodec_debug_enter(h)
> +#define mtk_vcodec_debug_leave(h)
> +
> +#endif
> +
> +void __iomem *mtk_vcodec_get_reg_addr(void *data, unsigned int reg_idx);
> +int mtk_vcodec_mem_alloc(void *data, struct mtk_vcodec_mem *mem);
> +void mtk_vcodec_mem_free(void *data, struct mtk_vcodec_mem *mem);
> +int mtk_vcodec_get_ctx_id(void *data);
> +struct platform_device *mtk_vcodec_get_plat_dev(void *data);
> +void mtk_vcodec_fmt2str(u32 fmt, char *str);
> +#endif /* _MTK_VCODEC_UTIL_H_ */
> diff --git a/drivers/media/platform/mtk-vcodec/venc_drv_base.h b/drivers/media/platform/mtk-vcodec/venc_drv_base.h
> new file mode 100644
> index 0000000..ed9cbf0
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vcodec/venc_drv_base.h
> @@ -0,0 +1,62 @@
> +/*
> + * Copyright (c) 2015 MediaTek Inc.
> + * Author: Daniel Hsiao <daniel.hsiao@mediatek.com>
> + *         Jungchang Tsao <jungchang.tsao@mediatek.com>
> + *
> + * This program is free software; you can redistribute it and/or
> + * modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + */
> +
> +#ifndef _VENC_DRV_BASE_
> +#define _VENC_DRV_BASE_
> +
> +#include "mtk_vcodec_drv.h"
> +
> +#include "venc_drv_if.h"
> +
> +struct venc_common_if {
> +       /**
> +        * (*init)() - initialize driver
> +        * @ctx:        [in] mtk v4l2 context
> +        * @handle: [out] driver handle
> +        */
> +       int (*init)(struct mtk_vcodec_ctx *ctx, unsigned long *handle);
> +
> +       /**
> +        * (*encode)() - trigger encode
> +        * @handle: [in] driver handle
> +        * @opt: [in] encode option
> +        * @frm_buf: [in] frame buffer to store input frame
> +        * @bs_buf: [in] bitstream buffer to store output bitstream
> +        * @result: [out] encode result
> +        */
> +       int (*encode)(unsigned long handle, enum venc_start_opt opt,
> +                     struct venc_frm_buf *frm_buf,
> +                     struct mtk_vcodec_mem *bs_buf,
> +                     struct venc_done_result *result);
> +
> +       /**
> +        * (*set_param)() - set driver's parameter
> +        * @handle: [in] driver handle
> +        * @type: [in] parameter type
> +        * @in: [in] buffer to store the parameter
> +        */
> +       int (*set_param)(unsigned long handle, enum venc_set_param_type type,
> +                        void *in);
> +
> +       /**
> +        * (*deinit)() - deinitialize driver.
> +        * @handle: [in] driver handle
> +        */
> +       int (*deinit)(unsigned long handle);
> +};
> +
> +
> +#endif
> diff --git a/drivers/media/platform/mtk-vcodec/venc_drv_if.c b/drivers/media/platform/mtk-vcodec/venc_drv_if.c
> new file mode 100644
> index 0000000..daa8e93
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vcodec/venc_drv_if.c
> @@ -0,0 +1,100 @@
> +/*
> + * Copyright (c) 2015 MediaTek Inc.
> + * Author: Daniel Hsiao <daniel.hsiao@mediatek.com>
> + *         Jungchang Tsao <jungchang.tsao@mediatek.com>
> + *
> + * This program is free software; you can redistribute it and/or
> + * modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + */
> +
> +#include <linux/interrupt.h>
> +#include <linux/kernel.h>
> +#include <linux/slab.h>
> +
> +#include "venc_drv_if.h"
> +#include "mtk_vcodec_enc.h"
> +#include "mtk_vcodec_pm.h"
> +#include "mtk_vpu.h"
> +
> +#include "venc_drv_base.h"
> +
> +int venc_if_create(struct mtk_vcodec_ctx *ctx, unsigned int fourcc)
> +{
> +       char str[10];
> +       int ret = 0;
> +
> +       mtk_vcodec_fmt2str(fourcc, str);
> +
> +       switch (fourcc) {
> +       case V4L2_PIX_FMT_VP8:
> +       case V4L2_PIX_FMT_H264:
> +       default:
> +               return -EINVAL;
> +       }
> +
> +       mtk_venc_lock(ctx);
> +       mtk_vcodec_enc_clock_on(&ctx->dev->pm);
> +       ret = ctx->enc_if->init(ctx, (unsigned long *)&ctx->drv_handle);
> +       mtk_vcodec_enc_clock_off(&ctx->dev->pm);
> +       mtk_venc_unlock(ctx);
> +
> +       return ret;
> +
> +}
> +
> +
> +int venc_if_set_param(struct mtk_vcodec_ctx *ctx,
> +                     enum venc_set_param_type type, void *in)
> +{
> +       int ret = 0;
> +
> +       mtk_venc_lock(ctx);
> +       mtk_vcodec_enc_clock_on(&ctx->dev->pm);
> +       ret = ctx->enc_if->set_param(ctx->drv_handle, type, in);
> +       mtk_vcodec_enc_clock_off(&ctx->dev->pm);
> +       mtk_venc_unlock(ctx);
> +
> +       return ret;
> +}
> +
> +int venc_if_encode(struct mtk_vcodec_ctx *ctx,
> +                  enum venc_start_opt opt, struct venc_frm_buf *frm_buf,
> +                  struct mtk_vcodec_mem *bs_buf,
> +                  struct venc_done_result *result)
> +{
> +       int ret = 0;
> +
> +       mtk_venc_lock(ctx);
> +       mtk_vcodec_enc_clock_on(&ctx->dev->pm);
> +       ret = ctx->enc_if->encode(ctx->drv_handle, opt, frm_buf, bs_buf, result);
> +       mtk_vcodec_enc_clock_off(&ctx->dev->pm);
> +       mtk_venc_unlock(ctx);
> +
> +       return ret;
> +}
> +
> +int venc_if_release(struct mtk_vcodec_ctx *ctx)
> +{
> +       int ret = 0;
> +
> +       if(ctx->drv_handle == 0)
> +               return 0;
> +
> +       mtk_venc_lock(ctx);
> +       mtk_vcodec_enc_clock_on(&ctx->dev->pm);
> +       ret = ctx->enc_if->deinit(ctx->drv_handle);
> +       mtk_vcodec_enc_clock_off(&ctx->dev->pm);
> +       mtk_venc_unlock(ctx);
> +
> +       ctx->drv_handle = 0;
> +
> +       return ret;
> +}
> +
> diff --git a/drivers/media/platform/mtk-vcodec/venc_drv_if.h b/drivers/media/platform/mtk-vcodec/venc_drv_if.h
> new file mode 100644
> index 0000000..a387011
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vcodec/venc_drv_if.h
> @@ -0,0 +1,175 @@
> +/*
> + * Copyright (c) 2015 MediaTek Inc.
> + * Author: Daniel Hsiao <daniel.hsiao@mediatek.com>
> + *         Jungchang Tsao <jungchang.tsao@mediatek.com>
> + *
> + * This program is free software; you can redistribute it and/or
> + * modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + */
> +
> +#ifndef _VENC_DRV_IF_H_
> +#define _VENC_DRV_IF_H_
> +
> +#include "mtk_vcodec_drv.h"
> +#include "mtk_vcodec_util.h"
> +
> +/*
> + * enum venc_yuv_fmt - The type of input yuv format
> + * (VPU related: If you change the order, you must also update the VPU codes.)
> + * @VENC_YUV_FORMAT_420: 420 YUV format
> + * @VENC_YUV_FORMAT_YV12: YV12 YUV format
> + * @VENC_YUV_FORMAT_NV12: NV12 YUV format
> + * @VENC_YUV_FORMAT_NV21: NV21 YUV format
> + */
> +enum venc_yuv_fmt {
> +       VENC_YUV_FORMAT_420 = 3,
> +       VENC_YUV_FORMAT_YV12 = 5,
> +       VENC_YUV_FORMAT_NV12 = 6,
> +       VENC_YUV_FORMAT_NV21 = 7,
> +};
> +
> +/*
> + * enum venc_start_opt - encode frame option used in venc_if_encode()
> + * @VENC_START_OPT_ENCODE_SEQUENCE_HEADER: encode SPS/PPS for H264
> + * @VENC_START_OPT_ENCODE_FRAME: encode normal frame
> + */
> +enum venc_start_opt {
> +       VENC_START_OPT_ENCODE_SEQUENCE_HEADER,
> +       VENC_START_OPT_ENCODE_FRAME,
> +};
> +
> +/*
> + * enum venc_drv_msg - The type of encode frame status used in venc_if_encode()
> + * @VENC_MESSAGE_OK: encode ok
> + * @VENC_MESSAGE_ERR: encode error
> + */
> +enum venc_drv_msg {
> +       VENC_MESSAGE_OK,
> +       VENC_MESSAGE_ERR,
> +};
> +
> +/*
> + * enum venc_set_param_type - The type of set parameter used in venc_if_set_param()
> + * (VPU related: If you change the order, you must also update the VPU codes.)
> + * @VENC_SET_PARAM_ENC: set encoder parameters
> + * @VENC_SET_PARAM_FORCE_INTRA: set force intra frame
> + * @VENC_SET_PARAM_ADJUST_BITRATE: set to adjust bitrate (in bps)
> + * @VENC_SET_PARAM_ADJUST_FRAMERATE: set frame rate
> + * @VENC_SET_PARAM_I_FRAME_INTERVAL: set I frame interval
> + * @VENC_SET_PARAM_SKIP_FRAME: set H264 skip one frame
> + * @VENC_SET_PARAM_PREPEND_HEADER: set H264 prepend SPS/PPS before IDR
> + * @VENC_SET_PARAM_TS_MODE: set VP8 temporal scalability mode
> + */
> +enum venc_set_param_type {
> +       VENC_SET_PARAM_ENC,
> +       VENC_SET_PARAM_FORCE_INTRA,
> +       VENC_SET_PARAM_ADJUST_BITRATE,
> +       VENC_SET_PARAM_ADJUST_FRAMERATE,
> +       VENC_SET_PARAM_I_FRAME_INTERVAL,
> +       VENC_SET_PARAM_SKIP_FRAME,
> +       VENC_SET_PARAM_PREPEND_HEADER,
> +       VENC_SET_PARAM_TS_MODE,
> +};
> +
> +/*
> + * struct venc_enc_prm - encoder settings for VENC_SET_PARAM_ENC used in venc_if_set_param()
> + * @input_fourcc: input fourcc
> + * @h264_profile: V4L2 defined H.264 profile
> + * @h264_level: V4L2 defined H.264 level
> + * @width: image width
> + * @height: image height
> + * @buf_width: buffer width
> + * @buf_height: buffer height
> + * @frm_rate: frame rate
> + * @intra_period: intra frame period
> + * @bitrate: target bitrate in kbps
> + */
> +struct venc_enc_prm {
> +       enum venc_yuv_fmt input_fourcc;
> +       unsigned int h264_profile;
> +       unsigned int h264_level;
> +       unsigned int width;
> +       unsigned int height;
> +       unsigned int buf_width;
> +       unsigned int buf_height;
> +       unsigned int frm_rate;
> +       unsigned int intra_period;
> +       unsigned int bitrate;
> +       unsigned int sizeimage[MTK_VCODEC_MAX_PLANES];
> +};
> +
> +/*
> + * struct venc_frm_buf - frame buffer information used in venc_if_encode()
> + * @fb_addr: plane 0 frame buffer address
> + * @fb_addr1: plane 1 frame buffer address
> + * @fb_addr2: plane 2 frame buffer address
> + */
> +struct venc_frm_buf {
> +       struct mtk_vcodec_mem fb_addr;
> +       struct mtk_vcodec_mem fb_addr1;
> +       struct mtk_vcodec_mem fb_addr2;
> +};
> +
> +/*
> + * struct venc_done_result - This is return information used in venc_if_encode()
> + * @msg: message, such as success or error code
> + * @bs_size: output bitstream size
> + * @is_key_frm: output is key frame or not
> + */
> +struct venc_done_result {
> +       enum venc_drv_msg msg;
> +       unsigned int bs_size;
> +       bool is_key_frm;
> +};
> +
> +/*
> + * venc_if_create - Create the driver handle
> + * @ctx: device context
> + * @fourcc: encoder output format
> + * @handle: driver handle
> + * Return: 0 if creating handle successfully, otherwise it is failed.
> + */
> +int venc_if_create(struct mtk_vcodec_ctx *ctx, unsigned int fourcc);
> +
> +/*
> + * venc_if_release - Release the driver handle
> + * @handle: driver handle
> + * Return: 0 if releasing handle successfully, otherwise it is failed.
> + */
> +int venc_if_release(struct mtk_vcodec_ctx *ctx);
> +
> +/*
> + * venc_if_set_param - Set parameter to driver
> + * @handle: driver handle
> + * @type: set type
> + * @in: input parameter
> + * @out: output parameter
> + * Return: 0 if setting param successfully, otherwise it is failed.
> + */
> +int venc_if_set_param(struct mtk_vcodec_ctx *ctx,
> +                     enum venc_set_param_type type,
> +                     void *in);
> +
> +/*
> + * venc_if_encode - Encode frame
> + * @handle: driver handle
> + * @opt: encode frame option
> + * @frm_buf: input frame buffer information
> + * @bs_buf: output bitstream buffer infomraiton
> + * @result: encode result
> + * Return: 0 if encoding frame successfully, otherwise it is failed.
> + */
> +int venc_if_encode(struct mtk_vcodec_ctx *ctx,
> +                  enum venc_start_opt opt,
> +                  struct venc_frm_buf *frm_buf,
> +                  struct mtk_vcodec_mem *bs_buf,
> +                  struct venc_done_result *result);
> +
> +#endif /* _VENC_DRV_IF_H_ */
> diff --git a/drivers/media/platform/mtk-vcodec/venc_ipi_msg.h b/drivers/media/platform/mtk-vcodec/venc_ipi_msg.h
> new file mode 100644
> index 0000000..a345b98
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vcodec/venc_ipi_msg.h
> @@ -0,0 +1,212 @@
> +/*
> + * Copyright (c) 2015 MediaTek Inc.
> + * Author: Jungchang Tsao <jungchang.tsao@mediatek.com>
> + *         Daniel Hsiao <daniel.hsiao@mediatek.com>
> + *
> + * This program is free software; you can redistribute it and/or
> + * modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + */
> +
> +#ifndef _VENC_IPI_MSG_H_
> +#define _VENC_IPI_MSG_H_
> +
> +#define IPIMSG_H264_ENC_ID 0x100
> +#define IPIMSG_VP8_ENC_ID 0x200
> +
> +#define AP_IPIMSG_VENC_BASE 0x20000
> +#define VPU_IPIMSG_VENC_BASE 0x30000
> +
> +/**
> + * enum venc_ipi_msg_id - message id between AP and VPU
> + * (ipi stands for inter-processor interrupt)
> + * @AP_IPIMSG_XXX:             AP to VPU cmd message id
> + * @VPU_IPIMSG_XXX_DONE:       VPU ack AP cmd message id
> + */
> +enum venc_ipi_msg_id {
> +       AP_IPIMSG_H264_ENC_INIT = AP_IPIMSG_VENC_BASE +
> +                                 IPIMSG_H264_ENC_ID,
> +       AP_IPIMSG_H264_ENC_SET_PARAM,
> +       AP_IPIMSG_H264_ENC_ENCODE,
> +       AP_IPIMSG_H264_ENC_DEINIT,
> +
> +       AP_IPIMSG_VP8_ENC_INIT = AP_IPIMSG_VENC_BASE +
> +                                IPIMSG_VP8_ENC_ID,
> +       AP_IPIMSG_VP8_ENC_SET_PARAM,
> +       AP_IPIMSG_VP8_ENC_ENCODE,
> +       AP_IPIMSG_VP8_ENC_DEINIT,
> +
> +       VPU_IPIMSG_H264_ENC_INIT_DONE = VPU_IPIMSG_VENC_BASE +
> +                                       IPIMSG_H264_ENC_ID,
> +       VPU_IPIMSG_H264_ENC_SET_PARAM_DONE,
> +       VPU_IPIMSG_H264_ENC_ENCODE_DONE,
> +       VPU_IPIMSG_H264_ENC_DEINIT_DONE,
> +
> +       VPU_IPIMSG_VP8_ENC_INIT_DONE = VPU_IPIMSG_VENC_BASE +
> +                                      IPIMSG_VP8_ENC_ID,
> +       VPU_IPIMSG_VP8_ENC_SET_PARAM_DONE,
> +       VPU_IPIMSG_VP8_ENC_ENCODE_DONE,
> +       VPU_IPIMSG_VP8_ENC_DEINIT_DONE,
> +};
> +
> +/**
> + * struct venc_ap_ipi_msg_init - AP to VPU init cmd structure
> + * @msg_id:    message id (AP_IPIMSG_XXX_ENC_INIT)
> + * @venc_inst: AP encoder instance (struct venc_vp8_handle/venc_h264_handle *)
> + */
> +struct venc_ap_ipi_msg_init {
> +       uint32_t msg_id;
> +       uint32_t reserved;
> +       uint64_t venc_inst;
> +};
> +
> +/**
> + * struct venc_ap_ipi_msg_set_param - AP to VPU set_param cmd structure
> + * @msg_id:    message id (AP_IPIMSG_XXX_ENC_SET_PARAM)
> + * @inst_id:   VPU encoder instance id (struct venc_vp8_vpu_drv/venc_h264_vpu_drv *)
> + * @param_id:  parameter id (venc_set_param_type)
> + * @data_item: number of items in the data array
> + * @data[8]:   data array to store the set parameters
> + */
> +struct venc_ap_ipi_msg_set_param {
> +       uint32_t msg_id;
> +       uint32_t inst_id;
> +       uint32_t param_id;
> +       uint32_t data_item;
> +       uint32_t data[8];
> +};
> +
> +/**
> + * struct venc_ap_ipi_msg_enc - AP to VPU enc cmd structure
> + * @msg_id:    message id (AP_IPIMSG_XXX_ENC_ENCODE)
> + * @inst_id:   VPU encoder instance id (struct venc_vp8_vpu_drv/venc_h264_vpu_drv *)
> + * @bs_mode:   bitstream mode for h264
> + *             (H264_BS_MODE_SPS/H264_BS_MODE_PPS/H264_BS_MODE_FRAME)
> + * @input_addr:        pointer to input image buffer plane
> + * @bs_addr:   pointer to output bit stream buffer
> + * @bs_size:   bit stream buffer size
> + */
> +struct venc_ap_ipi_msg_enc {
> +       uint32_t msg_id;
> +       uint32_t inst_id;
> +       uint32_t bs_mode;
> +       uint32_t input_addr[3];
> +       uint32_t bs_addr;
> +       uint32_t bs_size;
> +};
> +
> +/**
> + * struct venc_ap_ipi_msg_deinit - AP to VPU deinit cmd structure
> + * @msg_id:    message id (AP_IPIMSG_XXX_ENC_DEINIT)
> + * @inst_id:   VPU encoder instance id (struct venc_vp8_vpu_drv/venc_h264_vpu_drv *)
> + */
> +struct venc_ap_ipi_msg_deinit {
> +       uint32_t msg_id;
> +       uint32_t inst_id;
> +};
> +
> +/**
> + * enum venc_ipi_msg_status - VPU ack AP cmd status
> + */
> +enum venc_ipi_msg_status {
> +       VENC_IPI_MSG_STATUS_OK,
> +       VENC_IPI_MSG_STATUS_FAIL,
> +};
> +
> +/**
> + * struct venc_vpu_ipi_msg_common - VPU ack AP cmd common structure
> + * @msg_id:    message id (VPU_IPIMSG_XXX_DONE)
> + * @status:    cmd status (venc_ipi_msg_status)
> + * @venc_inst: AP encoder instance (struct venc_vp8_handle/venc_h264_handle *)
> + */
> +struct venc_vpu_ipi_msg_common {
> +       uint32_t msg_id;
> +       uint32_t status;
> +       uint64_t venc_inst;
> +};
> +
> +/**
> + * struct venc_vpu_ipi_msg_init - VPU ack AP init cmd structure
> + * @msg_id:    message id (VPU_IPIMSG_XXX_ENC_SET_PARAM_DONE)
> + * @status:    cmd status (venc_ipi_msg_status)
> + * @venc_inst: AP encoder instance (struct venc_vp8_handle/venc_h264_handle *)
> + * @inst_id:   VPU encoder instance id (struct venc_vp8_vpu_drv/venc_h264_vpu_drv *)
> + */
> +struct venc_vpu_ipi_msg_init {
> +       uint32_t msg_id;
> +       uint32_t status;
> +       uint64_t venc_inst;
> +       uint32_t inst_id;
> +       uint32_t reserved;
> +};
> +
> +/**
> + * struct venc_vpu_ipi_msg_set_param - VPU ack AP set_param cmd structure
> + * @msg_id:    message id (VPU_IPIMSG_XXX_ENC_SET_PARAM_DONE)
> + * @status:    cmd status (venc_ipi_msg_status)
> + * @venc_inst: AP encoder instance (struct venc_vp8_handle/venc_h264_handle *)
> + * @param_id:  parameter id (venc_set_param_type)
> + * @data_item: number of items in the data array
> + * @data[6]:   data array to store the return result
> + */
> +struct venc_vpu_ipi_msg_set_param {
> +       uint32_t msg_id;
> +       uint32_t status;
> +       uint64_t venc_inst;
> +       uint32_t param_id;
> +       uint32_t data_item;
> +       uint32_t data[6];
> +};
> +
> +/**
> + * enum venc_ipi_msg_enc_state - Type of encode state
> + * VEN_IPI_MSG_ENC_STATE_FRAME:        one frame being encoded
> + * VEN_IPI_MSG_ENC_STATE_PART: bit stream buffer full
> + * VEN_IPI_MSG_ENC_STATE_SKIP: encoded skip frame
> + * VEN_IPI_MSG_ENC_STATE_ERROR:        encounter error
> + */
> +enum venc_ipi_msg_enc_state {
> +       VEN_IPI_MSG_ENC_STATE_FRAME,
> +       VEN_IPI_MSG_ENC_STATE_PART,
> +       VEN_IPI_MSG_ENC_STATE_SKIP,
> +       VEN_IPI_MSG_ENC_STATE_ERROR,
> +};
> +
> +/**
> + * struct venc_vpu_ipi_msg_enc - VPU ack AP enc cmd structure
> + * @msg_id:    message id (VPU_IPIMSG_XXX_ENC_ENCODE_DONE)
> + * @status:    cmd status (venc_ipi_msg_status)
> + * @venc_inst: AP encoder instance (struct venc_vp8_handle/venc_h264_handle *)
> + * @state:     encode state (venc_ipi_msg_enc_state)
> + * @key_frame: whether the encoded frame is key frame
> + * @bs_size:   encoded bitstream size
> + */
> +struct venc_vpu_ipi_msg_enc {
> +       uint32_t msg_id;
> +       uint32_t status;
> +       uint64_t venc_inst;
> +       uint32_t state;
> +       uint32_t key_frame;
> +       uint32_t bs_size;
> +       uint32_t reserved;
> +};
> +
> +/**
> + * struct venc_vpu_ipi_msg_deinit - VPU ack AP deinit cmd structure
> + * @msg_id:   message id (VPU_IPIMSG_XXX_ENC_DEINIT_DONE)
> + * @status:   cmd status (venc_ipi_msg_status)
> + * @venc_inst: AP encoder instance (struct venc_vp8_handle/venc_h264_handle *)
> + */
> +struct venc_vpu_ipi_msg_deinit {
> +       uint32_t msg_id;
> +       uint32_t status;
> +       uint64_t venc_inst;
> +};
> +
> +#endif /* _VENC_IPI_MSG_H_ */
> diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
> old mode 100644
> new mode 100755
> index ee9d530..3ac35c4
> --- a/include/uapi/linux/v4l2-controls.h
> +++ b/include/uapi/linux/v4l2-controls.h
> @@ -646,6 +646,10 @@ enum v4l2_mpeg_mfc51_video_force_frame_type {
>  #define V4L2_CID_MPEG_MFC51_VIDEO_H264_NUM_REF_PIC_FOR_P               (V4L2_CID_MPEG_MFC51_BASE+54)
>
>
> +#define V4L2_CID_MPEG_MTK_BASE                                 (V4L2_CTRL_CLASS_MPEG | 0x5500)
> +#define V4L2_CID_MPEG_MTK_VIDEO_FRAME_SKIP_ENABLE              (V4L2_CID_MPEG_MTK_BASE+0)
> +#define V4L2_CID_MPEG_MTK_VIDEO_FORCE_FRAME_TYPE_I_FRAME       (V4L2_CID_MPEG_MTK_BASE+1)
> +
>  /*  Camera class control IDs */
>
>  #define V4L2_CID_CAMERA_CLASS_BASE     (V4L2_CTRL_CLASS_CAMERA | 0x900)
> --
> 1.7.9.5
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-media" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 102+ messages in thread

* Re: [PATCH v4 5/8] [Media] vcodec: mediatek: Add Mediatek V4L2 Video Encoder Driver
@ 2016-02-24  8:26               ` tiffany lin
  0 siblings, 0 replies; 102+ messages in thread
From: tiffany lin @ 2016-02-24  8:26 UTC (permalink / raw)
  To: Wu-Cheng Li (李務誠)
  Cc: Hans Verkuil, Daniel Thompson, Rob Herring,
	Mauro Carvalho Chehab, Matthias Brugger, Daniel Kurtz,
	Pawel Osciak, Eddie Huang, Yingjoe Chen, devicetree,
	linux-kernel, linux-arm-kernel, linux-media, linux-mediatek,
	Lin PoChun, Andrew-CT Chen

Hi Wucheng,

On Tue, 2016-02-23 at 13:46 +0800, Wu-Cheng Li (李務誠) wrote:
> On Thu, Feb 4, 2016 at 7:35 PM, Tiffany Lin <tiffany.lin@mediatek.com> wrote:
> > From: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
> >
> > Add v4l2 layer encoder driver for MT8173
> >
> > Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
> > ---
> >  drivers/media/platform/Kconfig                     |   11 +
> >  drivers/media/platform/Makefile                    |    2 +
> >  drivers/media/platform/mtk-vcodec/Makefile         |    8 +
> >  drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h |  388 ++++++
> >  drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c | 1380 ++++++++++++++++++++
> >  drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h |   46 +
> >  .../media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c |  476 +++++++
> >  .../media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c  |  132 ++
> >  .../media/platform/mtk-vcodec/mtk_vcodec_intr.c    |  102 ++
> >  .../media/platform/mtk-vcodec/mtk_vcodec_intr.h    |   29 +
> >  drivers/media/platform/mtk-vcodec/mtk_vcodec_pm.h  |   26 +
> >  .../media/platform/mtk-vcodec/mtk_vcodec_util.c    |  106 ++
> >  .../media/platform/mtk-vcodec/mtk_vcodec_util.h    |   85 ++
> >  drivers/media/platform/mtk-vcodec/venc_drv_base.h  |   62 +
> >  drivers/media/platform/mtk-vcodec/venc_drv_if.c    |  100 ++
> >  drivers/media/platform/mtk-vcodec/venc_drv_if.h    |  175 +++
> >  drivers/media/platform/mtk-vcodec/venc_ipi_msg.h   |  212 +++
> >  include/uapi/linux/v4l2-controls.h                 |    4 +
> >  18 files changed, 3344 insertions(+)
> >  create mode 100644 drivers/media/platform/mtk-vcodec/Makefile
> >  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
> >  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
> >  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h
> >  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
> >  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
> >  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.c
> >  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.h
> >  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_pm.h
> >  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c
> >  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
> >  create mode 100644 drivers/media/platform/mtk-vcodec/venc_drv_base.h
> >  create mode 100644 drivers/media/platform/mtk-vcodec/venc_drv_if.c
> >  create mode 100644 drivers/media/platform/mtk-vcodec/venc_drv_if.h
> >  create mode 100644 drivers/media/platform/mtk-vcodec/venc_ipi_msg.h
> >  mode change 100644 => 100755 include/uapi/linux/v4l2-controls.h
> >
> > diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
> > index ba812d6..3e831c5 100644
> > --- a/drivers/media/platform/Kconfig
> > +++ b/drivers/media/platform/Kconfig
> > @@ -157,6 +157,17 @@ config VIDEO_MEDIATEK_VPU
> >             codec embedded in new Mediatek's SOCs. It is able
> >             to handle video decoding/encoding in a range of formats.
> >
> > +config VIDEO_MEDIATEK_VCODEC
> > +        tristate "Mediatek Video Codec driver"
> > +        depends on VIDEO_DEV && VIDEO_V4L2
> > +        depends on ARCH_MEDIATEK || COMPILE_TEST
> > +        select VIDEOBUF2_DMA_CONTIG
> > +        select V4L2_MEM2MEM_DEV
> > +        select MEDIATEK_VPU
> > +        default n
> > +        ---help---
> > +            Mediatek video codec driver for V4L2
> > +
> >  config VIDEO_MEM2MEM_DEINTERLACE
> >         tristate "Deinterlace support"
> >         depends on VIDEO_DEV && VIDEO_V4L2 && DMA_ENGINE
> > diff --git a/drivers/media/platform/Makefile b/drivers/media/platform/Makefile
> > index e5b19c6..510e06b 100644
> > --- a/drivers/media/platform/Makefile
> > +++ b/drivers/media/platform/Makefile
> > @@ -57,3 +57,5 @@ obj-$(CONFIG_VIDEO_XILINX)            += xilinx/
> >  ccflags-y += -I$(srctree)/drivers/media/i2c
> >
> >  obj-$(CONFIG_VIDEO_MEDIATEK_VPU)       += mtk-vpu/
> > +
> > +obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC)    += mtk-vcodec/
> > diff --git a/drivers/media/platform/mtk-vcodec/Makefile b/drivers/media/platform/mtk-vcodec/Makefile
> > new file mode 100644
> > index 0000000..ce38689
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/Makefile
> > @@ -0,0 +1,8 @@
> > +obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += mtk_vcodec_intr.o \
> > +                                      mtk_vcodec_util.o \
> > +                                      mtk_vcodec_enc_drv.o \
> > +                                      mtk_vcodec_enc.o \
> > +                                      mtk_vcodec_enc_pm.o \
> > +                                      venc_drv_if.o
> > +
> > +ccflags-y += -I$(srctree)/drivers/media/platform/mtk-vpu
> > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
> > new file mode 100644
> > index 0000000..9da2818
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
> > @@ -0,0 +1,388 @@
> > +/*
> > +* Copyright (c) 2015 MediaTek Inc.
> > +* Author: PC Chen <pc.chen@mediatek.com>
> > +*         Tiffany Lin <tiffany.lin@mediatek.com>
> > +*
> > +* This program is free software; you can redistribute it and/or modify
> > +* it under the terms of the GNU General Public License version 2 as
> > +* published by the Free Software Foundation.
> > +*
> > +* This program is distributed in the hope that it will be useful,
> > +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> > +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > +* GNU General Public License for more details.
> > +*/
> > +
> > +#ifndef _MTK_VCODEC_DRV_H_
> > +#define _MTK_VCODEC_DRV_H_
> > +
> > +#include <linux/platform_device.h>
> > +#include <linux/videodev2.h>
> > +#include <media/v4l2-ctrls.h>
> > +#include <media/v4l2-device.h>
> > +#include <media/v4l2-ioctl.h>
> > +#include <media/videobuf2-core.h>
> > +
> > +#include "mtk_vcodec_util.h"
> > +
> > +#define MTK_VCODEC_MAX_INSTANCES               10
> > +#define MTK_VCODEC_MAX_ENCODER_INSTANCES       3
> > +
> > +#define MTK_VCODEC_MAX_FRAME_SIZE      0x800000
> > +#define MTK_VIDEO_MAX_FRAME            32
> > +#define MTK_MAX_CTRLS                  20
> > +
> > +#define MTK_VCODEC_DRV_NAME            "mtk_vcodec_drv"
> > +#define MTK_VCODEC_ENC_NAME            "mtk-vcodec-enc"
> > +#define MTK_PLATFORM_STR               "platform:mt8173"
> > +
> > +#define MTK_VENC_IRQ_STATUS_SPS          0x1
> > +#define MTK_VENC_IRQ_STATUS_PPS          0x2
> > +#define MTK_VENC_IRQ_STATUS_FRM          0x4
> > +#define MTK_VENC_IRQ_STATUS_DRAM         0x8
> > +#define MTK_VENC_IRQ_STATUS_PAUSE        0x10
> > +#define MTK_VENC_IRQ_STATUS_SWITCH       0x20
> > +
> > +#define MTK_VENC_IRQ_STATUS_OFFSET       0x05C
> > +#define MTK_VENC_IRQ_ACK_OFFSET          0x060
> > +
> > +#define MTK_VCODEC_MAX_PLANES          3
> > +
> > +#define VDEC_HW_ACTIVE 0x10
> > +#define VDEC_IRQ_CFG    0x11
> > +#define VDEC_IRQ_CLR    0x10
> > +
> > +#define VDEC_IRQ_CFG_REG       0xa4
> > +#define WAIT_INTR_TIMEOUT      1000
> > +
> > +/**
> > + * enum mtk_hw_reg_idx - MTK hw register base index
> > + */
> > +enum mtk_hw_reg_idx {
> > +       VDEC_SYS,
> > +       VDEC_MISC,
> > +       VDEC_LD,
> > +       VDEC_TOP,
> > +       VDEC_CM,
> > +       VDEC_AD,
> > +       VDEC_AV,
> > +       VDEC_PP,
> > +       VDEC_HWD,
> > +       VDEC_HWQ,
> > +       VDEC_HWB,
> > +       VDEC_HWG,
> > +       NUM_MAX_VDEC_REG_BASE,
> > +       VENC_SYS = NUM_MAX_VDEC_REG_BASE,
> > +       VENC_LT_SYS,
> > +       NUM_MAX_VCODEC_REG_BASE
> > +};
> > +
> > +/**
> > + * enum mtk_instance_type - The type of an MTK Vcodec instance.
> > + */
> > +enum mtk_instance_type {
> > +       MTK_INST_DECODER                = 0,
> > +       MTK_INST_ENCODER                = 1,
> > +};
> > +
> > +/**
> > + * enum mtk_instance_state - The state of an MTK Vcodec instance.
> > + * @MTK_STATE_FREE - default state when instance is created
> > + * @MTK_STATE_INIT - vcodec instance is initialized
> > + * @MTK_STATE_HEADER - vdec had sps/pps header parsed or venc
> > + *                     had sps/pps header encoded
> > + * @MTK_STATE_FLUSH - vdec is flushing. Only used by decoder
> > + * @MTK_STATE_RES_CHANGE - vdec detect resolution change.
> > + *                     Only used by decoder
> > + * @MTK_STATE_ABORT - vcodec should be aborted
> > + */
> > +enum mtk_instance_state {
> > +       MTK_STATE_FREE = 0,
> > +       MTK_STATE_INIT = 1,
> > +       MTK_STATE_HEADER = 2,
> > +       MTK_STATE_FLUSH = 3,
> > +       MTK_STATE_RES_CHANGE = 4,
> > +       MTK_STATE_ABORT = 5,
> > +};
> > +
> > +/**
> > + * struct mtk_param_change - General encoding parameters type
> > + */
> > +enum mtk_encode_param {
> > +       MTK_ENCODE_PARAM_NONE = 0,
> > +       MTK_ENCODE_PARAM_BITRATE = (1 << 0),
> > +       MTK_ENCODE_PARAM_FRAMERATE = (1 << 1),
> > +       MTK_ENCODE_PARAM_INTRA_PERIOD = (1 << 2),
> > +       MTK_ENCODE_PARAM_FRAME_TYPE = (1 << 3),
> > +};
> > +
> > +/**
> > + * enum mtk_fmt_type - Type of the pixelformat
> > + * @MTK_FMT_FRAME - mtk vcodec raw frame
> > + */
> > +enum mtk_fmt_type {
> > +       MTK_FMT_DEC             = 0,
> > +       MTK_FMT_ENC             = 1,
> > +       MTK_FMT_FRAME           = 2,
> > +};
> > +
> > +/**
> > + * struct mtk_video_fmt - Structure used to store information about pixelformats
> > + */
> > +struct mtk_video_fmt {
> > +       u32 fourcc;
> > +       enum mtk_fmt_type type;
> > +       u32 num_planes;
> > +};
> > +
> > +/**
> > + * struct mtk_codec_framesizes - Structure used to store information about framesizes
> > + */
> > +struct mtk_codec_framesizes {
> > +       u32 fourcc;
> > +       struct  v4l2_frmsize_stepwise   stepwise;
> > +};
> > +
> > +/**
> > + * struct mtk_q_type - Type of queue
> > + */
> > +enum mtk_q_type {
> > +       MTK_Q_DATA_SRC          = 0,
> > +       MTK_Q_DATA_DST          = 1,
> > +};
> > +
> > +/**
> > + * struct mtk_q_data - Structure used to store information about queue
> > + * @colorspace reserved for encoder
> > + * @field              reserved for encoder
> > + */
> > +struct mtk_q_data {
> > +       unsigned int            width;
> > +       unsigned int            height;
> > +       enum v4l2_field         field;
> > +       enum v4l2_colorspace    colorspace;
> > +       unsigned int            bytesperline[MTK_VCODEC_MAX_PLANES];
> > +       unsigned int            sizeimage[MTK_VCODEC_MAX_PLANES];
> > +       struct mtk_video_fmt    *fmt;
> > +};
> > +
> > +/**
> > + * struct mtk_enc_params - General encoding parameters
> > + * @bitrate - target bitrate
> > + * @num_b_frame - number of b frames between p-frame
> > + * @rc_frame - frame based rate control
> > + * @rc_mb - macroblock based rate control
> > + * @seq_hdr_mode - H.264 sequence header is encoded separately or joined with the first frame
> > + * @gop_size - group of picture size, it's used as the intra frame period
> > + * @framerate_num - frame rate numerator
> > + * @framerate_denom - frame rate denominator
> > + * @h264_max_qp - Max value for H.264 quantization parameter
> > + * @h264_profile - V4L2 defined H.264 profile
> > + * @h264_level - V4L2 defined H.264 level
> > + * @force_intra - force/insert intra frame
> > + * @skip_frame - encode in skip frame mode that use minimum number of bits
> > + */
> > +struct mtk_enc_params {
> > +       unsigned int    bitrate;
> > +       unsigned int    num_b_frame;
> > +       unsigned int    rc_frame;
> > +       unsigned int    rc_mb;
> > +       unsigned int    seq_hdr_mode;
> > +       unsigned int    gop_size;
> > +       unsigned int    framerate_num;
> > +       unsigned int    framerate_denom;
> > +       unsigned int    h264_max_qp;
> > +       unsigned int    h264_profile;
> > +       unsigned int    h264_level;
> > +       unsigned int    force_intra;
> > +};
> > +
> > +/**
> > + * struct mtk_vcodec_pm - Power management data structure
> > + */
> > +struct mtk_vcodec_pm {
> > +       struct clk      *vcodecpll;
> > +       struct clk      *univpll_d2;
> > +       struct clk      *clk_cci400_sel;
> > +       struct clk      *vdecpll;
> > +       struct clk      *vdec_sel;
> > +       struct clk      *vencpll_d2;
> > +       struct clk      *venc_sel;
> > +       struct clk      *univpll1_d2;
> > +       struct clk      *venc_lt_sel;
> > +       struct device   *larbvdec;
> > +       struct device   *larbvenc;
> > +       struct device   *larbvenclt;
> > +       struct device   *dev;
> > +       struct mtk_vcodec_dev *mtkdev;
> > +};
> > +
> > +
> > +/**
> > + * struct mtk_vcodec_ctx - Context (instance) private data.
> > + *
> > + * @type:              type of the instance - decoder or encoder
> > + * @dev:               pointer to the mtk_vcodec_dev of the device
> > + * @fh:                        struct v4l2_fh
> > + * @m2m_ctx:           pointer to the v4l2_m2m_ctx of the context
> > + * @q_data:            store information of input and output queue
> > + *                     of the context
> > + * @idx:               index of the context that this structure describes
> > + * @state:             state of the context
> > + * @param_change:      encode parameters
> > + * @enc_params:                encoding parameters
> > + * @colorspace:
> > + * @enc_if:            hoooked encoder driver interface
> > + * @drv_handle:                driver handle for specific decode/encode instance
> > + *
> > + * @picinfo:           store width/height of image and buffer and planes' size for decoder
> > + *                     and encoder
> > + * @dpb_count:         count of the DPB buffers required by MTK Vcodec hw
> > + *
> > + * @int_cond:          variable used by the waitqueue
> > + * @int_type:          type of the last interrupt
> > + * @queue:             waitqueue that can be used to wait for this context to
> > + *                     finish
> > + * @irq_status:                irq status
> > + *
> > + * @ctrl_hdl:          handler for v4l2 framework
> > + * @ctrls:             array of controls, used when adding controls to the
> > + *                     v4l2 control framework
> > + *
> > + * @encode_work:       worker for the encoding
> > + */
> > +struct mtk_vcodec_ctx {
> > +       enum mtk_instance_type type;
> > +       struct mtk_vcodec_dev *dev;
> > +       struct v4l2_fh fh;
> > +       struct v4l2_m2m_ctx *m2m_ctx;
> > +       struct mtk_q_data q_data[2];
> > +       int idx;
> > +       enum mtk_instance_state state;
> > +       enum mtk_encode_param param_change;
> > +       struct mtk_enc_params enc_params;
> > +
> > +       struct venc_common_if *enc_if;
> > +       unsigned long drv_handle;
> > +
> > +
> > +       int int_cond;
> > +       int int_type;
> > +       wait_queue_head_t queue;
> > +       unsigned int irq_status;
> > +
> > +       struct v4l2_ctrl_handler ctrl_hdl;
> > +       struct v4l2_ctrl *ctrls[MTK_MAX_CTRLS];
> > +
> > +       struct work_struct encode_work;
> > +
> > +};
> > +
> > +/**
> > + * struct mtk_vcodec_dev - driver data
> > + * @v4l2_dev:          V4L2 device to register video devices for.
> > + * @vfd_enc:           Video device for encoder.
> > + *
> > + * @m2m_dev_enc:       m2m device for encoder.
> > + * @plat_dev:          platform device
> > + * @alloc_ctx:         VB2 allocator context
> > + *                     (for allocations without kernel mapping).
> > + * @ctx:               array of driver contexts
> > + *
> > + * @curr_ctx:          The context that is waiting for codec hardware
> > + *
> > + * @reg_base:          Mapped address of MTK Vcodec registers.
> > + *
> > + * @instance_mask:     used to mark which contexts are opened
> > + * @num_instances:     counter of active MTK Vcodec instances
> > + *
> > + * @encode_workqueue:  encode work queue
> > + *
> > + * @int_cond:          used to identify interrupt condition happen
> > + * @int_type:          used to identify what kind of interrupt condition happen
> > + * @dev_mutex:         video_device lock
> > + * @queue:             waitqueue for waiting for completion of device commands
> > + *
> > + * @enc_irq:           h264 encoder irq resource
> > + * @enc_lt_irq:                vp8 encoder irq resource
> > + *
> > + * @enc_mutex:         encoder hardware lock.
> > + *
> > + * @pm:                        power management control
> > + * @dec_capability:    used to identify decode capability, ex: 4k
> > + * @enc_capability:     used to identify encode capability
> > + */
> > +struct mtk_vcodec_dev {
> > +       struct v4l2_device      v4l2_dev;
> > +       struct video_device     *vfd_enc;
> > +
> > +       struct v4l2_m2m_dev     *m2m_dev_enc;
> > +       struct platform_device  *plat_dev;
> > +       struct platform_device  *vpu_plat_dev;
> > +       struct vb2_alloc_ctx    *alloc_ctx;
> > +       struct mtk_vcodec_ctx   *ctx[MTK_VCODEC_MAX_INSTANCES];
> > +       int curr_ctx;
> > +       void __iomem            *reg_base[NUM_MAX_VCODEC_REG_BASE];
> > +
> > +       unsigned long   instance_mask[BITS_TO_LONGS(MTK_VCODEC_MAX_INSTANCES)];
> > +       int                     num_instances;
> > +
> > +       struct workqueue_struct *encode_workqueue;
> > +
> > +       int                     int_cond;
> > +       int                     int_type;
> > +       struct mutex            dev_mutex;
> > +       wait_queue_head_t       queue;
> > +
> > +       int                     enc_irq;
> > +       int                     enc_lt_irq;
> > +
> > +       struct mutex            enc_mutex;
> > +
> > +       struct mtk_vcodec_pm    pm;
> > +       unsigned int            dec_capability;
> > +       unsigned int            enc_capability;
> > +};
> > +
> > +/**
> > + * struct mtk_vcodec_ctrl - information about controls to be registered.
> > + * @id:                        Control ID.
> > + * @type:              Type of the control.
> > + * @name:              Human readable name of the control.
> > + * @minimum:           Minimum value of the control.
> > + * @maximum:           Maximum value of the control.
> > + * @step:              Control value increase step.
> > + * @menu_skip_mask:    Mask of invalid menu positions.
> > + * @default_value:     Initial value of the control.
> > + * @is_volatile:       Control is volatile.
> > + *
> > + * See also struct v4l2_ctrl_config.
> > + */
> > +struct mtk_vcodec_ctrl {
> > +       u32                     id;
> > +       enum v4l2_ctrl_type     type;
> > +       u8                      name[32];
> > +       s32                     minimum;
> > +       s32                     maximum;
> > +       s32                     step;
> > +       u32                     menu_skip_mask;
> > +       s32                     default_value;
> > +       u8                      is_volatile;
> > +};
> > +
> > +static inline struct mtk_vcodec_ctx *fh_to_ctx(struct v4l2_fh *fh)
> > +{
> > +       return container_of(fh, struct mtk_vcodec_ctx, fh);
> > +}
> > +
> > +static inline struct mtk_vcodec_ctx *ctrl_to_ctx(struct v4l2_ctrl *ctrl)
> > +{
> > +       return container_of(ctrl->handler, struct mtk_vcodec_ctx, ctrl_hdl);
> > +}
> > +
> > +extern const struct v4l2_ioctl_ops mtk_vdec_ioctl_ops;
> > +extern const struct v4l2_m2m_ops mtk_vdec_m2m_ops;
> > +extern const struct v4l2_ioctl_ops mtk_venc_ioctl_ops;
> > +extern const struct v4l2_m2m_ops mtk_venc_m2m_ops;
> > +
> > +#endif /* _MTK_VCODEC_DRV_H_ */
> > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
> > new file mode 100644
> > index 0000000..ee602fe
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
> > @@ -0,0 +1,1380 @@
> > +/*
> > +* Copyright (c) 2015 MediaTek Inc.
> > +* Author: PC Chen <pc.chen@mediatek.com>
> > +*         Tiffany Lin <tiffany.lin@mediatek.com>
> > +*
> > +* This program is free software; you can redistribute it and/or modify
> > +* it under the terms of the GNU General Public License version 2 as
> > +* published by the Free Software Foundation.
> > +*
> > +* This program is distributed in the hope that it will be useful,
> > +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> > +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > +* GNU General Public License for more details.
> > +*/
> > +
> > +#include <media/v4l2-event.h>
> > +#include <media/v4l2-mem2mem.h>
> > +#include <media/videobuf2-dma-contig.h>
> > +
> > +#include "mtk_vcodec_drv.h"
> > +#include "mtk_vcodec_enc.h"
> > +#include "mtk_vcodec_intr.h"
> > +#include "mtk_vcodec_util.h"
> > +#include "venc_drv_if.h"
> > +
> > +#define MTK_VENC_MIN_W 32
> > +#define MTK_VENC_MIN_H 32
> > +#define MTK_VENC_MAX_W 1920
> > +#define MTK_VENC_MAX_H 1080
> > +#define DFT_CFG_WIDTH  MTK_VENC_MIN_W
> > +#define DFT_CFG_HEIGHT MTK_VENC_MIN_H
> > +
> > +static void mtk_venc_worker(struct work_struct *work);
> > +
> > +static struct mtk_video_fmt mtk_video_formats[] = {
> > +       {
> > +               .fourcc         = V4L2_PIX_FMT_YUV420,
> > +               .type           = MTK_FMT_FRAME,
> > +               .num_planes     = 3,
> This is not right. V4L2_PIX_FMT_YUV420 only has one physical plane.
> Same for V4L2_PIX_FMT_YVU420, V4L2_PIX_FMT_NV12, and
> V4L2_PIX_FMT_NV21.
We will fix this in next version.

best regards,
Tiffany

> > +       },
> > +       {
> > +               .fourcc         = V4L2_PIX_FMT_YVU420,
> > +               .type           = MTK_FMT_FRAME,
> > +               .num_planes     = 3,
> > +       },
> > +       {
> > +               .fourcc         = V4L2_PIX_FMT_NV12,
> > +               .type           = MTK_FMT_FRAME,
> > +               .num_planes     = 2,
> > +       },
> > +       {
> > +               .fourcc         = V4L2_PIX_FMT_NV21,
> > +               .type           = MTK_FMT_FRAME,
> > +               .num_planes     = 2,
> > +       },
> > +       {
> > +               .fourcc         = V4L2_PIX_FMT_YUV420M,
> > +               .type           = MTK_FMT_FRAME,
> > +               .num_planes     = 3,
> > +       },
> > +       {
> > +               .fourcc         = V4L2_PIX_FMT_YVU420M,
> > +               .type           = MTK_FMT_FRAME,
> > +               .num_planes     = 3,
> > +       },
> > +       {
> > +               .fourcc         = V4L2_PIX_FMT_NV12M,
> > +               .type           = MTK_FMT_FRAME,
> > +               .num_planes     = 2,
> > +       },
> > +       {
> > +               .fourcc         = V4L2_PIX_FMT_NV21M,
> > +               .type           = MTK_FMT_FRAME,
> > +               .num_planes     = 2,
> > +       },
> > +       {
> > +               .fourcc         = V4L2_PIX_FMT_H264,
> > +               .type           = MTK_FMT_ENC,
> > +               .num_planes     = 1,
> > +       },
> > +       {
> > +               .fourcc         = V4L2_PIX_FMT_VP8,
> > +               .type           = MTK_FMT_ENC,
> > +               .num_planes     = 1,
> > +       },
> > +};
> > +
> > +#define NUM_FORMATS ARRAY_SIZE(mtk_video_formats)
> > +
> > +static const struct mtk_codec_framesizes mtk_venc_framesizes[] = {
> > +       {
> > +               .fourcc = V4L2_PIX_FMT_H264,
> > +               .stepwise = {  160, 1920, 16, 128, 1088, 16 },
> > +       },
> > +       {
> > +               .fourcc = V4L2_PIX_FMT_VP8,
> > +               .stepwise = {  160, 1920, 16, 128, 1088, 16 },
> > +       },
> > +};
> > +
> > +#define NUM_SUPPORTED_FRAMESIZE ARRAY_SIZE(mtk_venc_framesizes)
> > +
> > +static int vidioc_venc_s_ctrl(struct v4l2_ctrl *ctrl)
> > +{
> > +       struct mtk_vcodec_ctx *ctx = ctrl_to_ctx(ctrl);
> > +       struct mtk_enc_params *p = &ctx->enc_params;
> > +       int ret = 0;
> > +
> > +       switch (ctrl->id) {
> > +       case V4L2_CID_MPEG_VIDEO_BITRATE:
> > +               mtk_v4l2_debug(2, "V4L2_CID_MPEG_VIDEO_BITRATE val = %d",
> > +                       ctrl->val);
> > +               p->bitrate = ctrl->val;
> > +               ctx->param_change |= MTK_ENCODE_PARAM_BITRATE;
> > +               break;
> > +       case V4L2_CID_MPEG_VIDEO_B_FRAMES:
> > +               mtk_v4l2_debug(2, "V4L2_CID_MPEG_VIDEO_B_FRAMES val = %d",
> > +                       ctrl->val);
> > +               p->num_b_frame = ctrl->val;
> > +               break;
> > +       case V4L2_CID_MPEG_VIDEO_FRAME_RC_ENABLE:
> > +               mtk_v4l2_debug(2, "V4L2_CID_MPEG_VIDEO_FRAME_RC_ENABLE val = %d",
> > +                       ctrl->val);
> > +               p->rc_frame = ctrl->val;
> > +               break;
> > +       case V4L2_CID_MPEG_VIDEO_H264_MAX_QP:
> > +               mtk_v4l2_debug(2, "V4L2_CID_MPEG_VIDEO_H264_MAX_QP val = %d",
> > +                       ctrl->val);
> > +               p->h264_max_qp = ctrl->val;
> > +               break;
> > +       case V4L2_CID_MPEG_VIDEO_HEADER_MODE:
> > +               mtk_v4l2_debug(2, "V4L2_CID_MPEG_VIDEO_HEADER_MODE val = %d",
> > +                       ctrl->val);
> > +               p->seq_hdr_mode = ctrl->val;
> > +               break;
> > +       case V4L2_CID_MPEG_VIDEO_MB_RC_ENABLE:
> > +               mtk_v4l2_debug(2, "V4L2_CID_MPEG_VIDEO_MB_RC_ENABLE val = %d",
> > +                       ctrl->val);
> > +               p->rc_mb = ctrl->val;
> > +               break;
> > +       case V4L2_CID_MPEG_VIDEO_H264_PROFILE:
> > +               mtk_v4l2_debug(2, "V4L2_CID_MPEG_VIDEO_H264_PROFILE val = %d",
> > +                       ctrl->val);
> > +               p->h264_profile = ctrl->val;
> > +               break;
> > +       case V4L2_CID_MPEG_VIDEO_H264_LEVEL:
> > +               mtk_v4l2_debug(2, "V4L2_CID_MPEG_VIDEO_H264_LEVEL val = %d",
> > +                       ctrl->val);
> > +               p->h264_level = ctrl->val;
> > +               break;
> > +       case V4L2_CID_MPEG_VIDEO_H264_I_PERIOD:
> > +               mtk_v4l2_debug(2, "V4L2_CID_MPEG_VIDEO_H264_I_PERIOD val = %d",
> > +                       ctrl->val);
> > +       case V4L2_CID_MPEG_VIDEO_GOP_SIZE:
> > +               mtk_v4l2_debug(2, "V4L2_CID_MPEG_VIDEO_GOP_SIZE val = %d",
> > +                       ctrl->val);
> > +               p->gop_size = ctrl->val;
> > +               ctx->param_change |= MTK_ENCODE_PARAM_INTRA_PERIOD;
> > +               break;
> > +       case V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME:
> > +               mtk_v4l2_debug(2, "V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME");
> > +               p->force_intra = 1;
> > +               ctx->param_change |= MTK_ENCODE_PARAM_FRAME_TYPE;
> > +               break;
> > +       default:
> > +               ret = -EINVAL;
> > +               break;
> > +       }
> > +
> > +       return ret;
> > +}
> > +
> > +static const struct v4l2_ctrl_ops mtk_vcodec_enc_ctrl_ops = {
> > +       .s_ctrl = vidioc_venc_s_ctrl,
> > +};
> > +
> > +static int vidioc_enum_fmt(struct file *file, struct v4l2_fmtdesc *f,
> > +                          bool out)
> > +{
> > +       struct mtk_video_fmt *fmt;
> > +       int i, j = 0;
> > +
> > +       for (i = 0; i < NUM_FORMATS; ++i) {
> > +               if (out && mtk_video_formats[i].type != MTK_FMT_FRAME)
> > +                       continue;
> > +               else if (!out && mtk_video_formats[i].type != MTK_FMT_ENC)
> > +                       continue;
> > +
> > +               if (j == f->index) {
> > +                       fmt = &mtk_video_formats[i];
> > +                       f->pixelformat = fmt->fourcc;
> > +                       return 0;
> > +               }
> > +               ++j;
> > +       }
> > +
> > +       return -EINVAL;
> > +}
> > +
> > +static int vidioc_enum_framesizes(struct file *file, void *fh,
> > +                                 struct v4l2_frmsizeenum *fsize)
> > +{
> > +       int i = 0;
> > +
> > +       for (i = 0; i < NUM_SUPPORTED_FRAMESIZE; ++i) {
> > +               if (fsize->pixel_format != mtk_venc_framesizes[i].fourcc)
> > +                       continue;
> > +
> > +               if (!fsize->index) {
> > +                       fsize->type = V4L2_FRMSIZE_TYPE_STEPWISE;
> > +                       fsize->stepwise = mtk_venc_framesizes[i].stepwise;
> > +                       return 0;
> > +               }
> > +       }
> > +
> > +       return -EINVAL;
> > +}
> > +
> > +static int vidioc_enum_fmt_vid_cap_mplane(struct file *file, void *pirv,
> > +                                         struct v4l2_fmtdesc *f)
> > +{
> > +       return vidioc_enum_fmt(file, f, false);
> > +}
> > +
> > +static int vidioc_enum_fmt_vid_out_mplane(struct file *file, void *prov,
> > +                                         struct v4l2_fmtdesc *f)
> > +{
> > +       return vidioc_enum_fmt(file, f, true);
> > +}
> > +
> > +static int vidioc_venc_querycap(struct file *file, void *priv,
> > +                               struct v4l2_capability *cap)
> > +{
> > +        strlcpy(cap->driver, MTK_VCODEC_ENC_NAME, strlen(MTK_VCODEC_ENC_NAME));
> > +        cap->driver[strlen(MTK_VCODEC_ENC_NAME)]=0;
> > +        strlcpy(cap->bus_info, MTK_PLATFORM_STR, strlen(MTK_PLATFORM_STR));
> > +        cap->bus_info[strlen(MTK_PLATFORM_STR)]=0;
> > +        strlcpy(cap->card, MTK_PLATFORM_STR, strlen(MTK_PLATFORM_STR));
> > +        cap->card[strlen(MTK_PLATFORM_STR)]=0;
> > +
> > +       /*
> > +        * This is only a mem-to-mem video device. The capture and output
> > +        * device capability flags are left only for backward compatibility
> > +        * and are scheduled for removal.
> > +        */
> > +       cap->device_caps  = V4L2_CAP_VIDEO_M2M_MPLANE | V4L2_CAP_STREAMING;
> > +       cap->capabilities = cap->device_caps | V4L2_CAP_DEVICE_CAPS;
> > +
> > +       return 0;
> > +}
> > +static int vidioc_venc_s_parm(struct file *file, void *priv,
> > +                             struct v4l2_streamparm *a)
> > +{
> > +       struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
> > +
> > +       if (a->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
> > +               ctx->enc_params.framerate_num =
> > +                       a->parm.output.timeperframe.denominator;
> > +               ctx->enc_params.framerate_denom =
> > +                       a->parm.output.timeperframe.numerator;
> > +               ctx->param_change |= MTK_ENCODE_PARAM_FRAMERATE;
> > +       } else {
> > +               return -EINVAL;
> > +       }
> > +       return 0;
> > +}
> > +
> > +static struct mtk_q_data *mtk_venc_get_q_data(struct mtk_vcodec_ctx *ctx,
> > +                                             enum v4l2_buf_type type)
> > +{
> > +       if (V4L2_TYPE_IS_OUTPUT(type))
> > +               return &ctx->q_data[MTK_Q_DATA_SRC];
> > +
> > +       return &ctx->q_data[MTK_Q_DATA_DST];
> > +}
> > +
> > +static struct mtk_video_fmt *mtk_venc_find_format(struct v4l2_format *f)
> > +{
> > +       struct mtk_video_fmt *fmt;
> > +       unsigned int k;
> > +
> > +       for (k = 0; k < NUM_FORMATS; k++) {
> > +               fmt = &mtk_video_formats[k];
> > +               if (fmt->fourcc == f->fmt.pix.pixelformat)
> > +                       return fmt;
> > +       }
> > +
> > +       return NULL;
> > +}
> > +
> > +static void mtk_vcodec_enc_calc_src_size(
> > +       unsigned int num_planes, unsigned int pic_width,
> > +       unsigned int pic_height, unsigned int sizeimage[],
> > +       unsigned int bytesperline[])
> > +{
> > +       unsigned int y_pitch_w_div16;
> > +       unsigned int c_pitch_w_div16;
> > +
> > +       y_pitch_w_div16 = ALIGN(pic_width, 16) >> 4;
> > +       c_pitch_w_div16 = ALIGN(pic_width, 16) >> 4;
> > +
> > +       if (num_planes == 2) {
> > +               sizeimage[0] =
> > +                       (y_pitch_w_div16) * (((pic_height + 31) / 32) * 2) * 256 +
> > +                       ((y_pitch_w_div16 % 8 == 0) ? 0 : ((ALIGN(pic_width, 16) * 2) * 16));
> > +
> > +               sizeimage[1] =
> > +                       (c_pitch_w_div16) * (((pic_height + 31) / 32) * 2) * 128 +
> > +                       ((c_pitch_w_div16 % 8 == 0) ? 0 : (ALIGN(pic_width, 16) * 16));
> > +
> > +               sizeimage[2] = 0;
> > +
> > +               bytesperline[0] = ALIGN(pic_width, 16);
> > +               bytesperline[1] = ALIGN(pic_width, 16);
> > +               bytesperline[2] = 0;
> > +
> > +       } else {
> > +               sizeimage[0] =
> > +                       (y_pitch_w_div16) * (((pic_height + 31) / 32) * 2) * 256 +
> > +                       ((y_pitch_w_div16 % 8 == 0) ? 0 : ((ALIGN(pic_width, 16) * 2) * 16));
> > +
> > +               sizeimage[1] =
> > +                       (c_pitch_w_div16) * (((pic_height + 31) / 32) * 2) * 64 +
> > +                       ((c_pitch_w_div16 % 8 == 0) ? 0 : ((ALIGN(pic_width, 16) / 2) * 16));
> > +
> > +               sizeimage[2] =
> > +                       (c_pitch_w_div16) * (((pic_height + 31) / 32) * 2) * 64 +
> > +                       ((c_pitch_w_div16 % 8 == 0) ? 0 : ((ALIGN(pic_width, 16) / 2) * 16));
> > +
> > +               bytesperline[0] = ALIGN(pic_width, 16);
> > +               bytesperline[1] = ALIGN(pic_width, 16) / 2;
> > +               bytesperline[2] = ALIGN(pic_width, 16) / 2;
> > +       }
> > +}
> > +
> > +static int vidioc_try_fmt(struct v4l2_format *f, struct mtk_video_fmt *fmt)
> > +{
> > +       struct v4l2_pix_format_mplane *pix_fmt_mp = &f->fmt.pix_mp;
> > +
> > +       /* V4L2 specification suggests the driver corrects the format struct
> > +         * if any of the dimensions is unsupported */
> > +        if (pix_fmt_mp->height < MTK_VENC_MIN_H)
> > +                pix_fmt_mp->height = MTK_VENC_MIN_H;
> > +        else if (pix_fmt_mp->height > MTK_VENC_MAX_H)
> > +                pix_fmt_mp->height = MTK_VENC_MAX_H;
> > +
> > +        if (pix_fmt_mp->width < MTK_VENC_MIN_W)
> > +                pix_fmt_mp->width = MTK_VENC_MIN_W;
> > +        else if (pix_fmt_mp->width > MTK_VENC_MAX_W)
> > +                pix_fmt_mp->width = MTK_VENC_MAX_W;
> > +
> > +        pix_fmt_mp->field = V4L2_FIELD_NONE;
> > +
> > +       if (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
> > +               int size = pix_fmt_mp->height * pix_fmt_mp->width;
> > +               if (fmt->num_planes != pix_fmt_mp->num_planes)
> > +                       pix_fmt_mp->num_planes = fmt->num_planes;
> > +
> > +               if(pix_fmt_mp->plane_fmt[0].sizeimage != size)
> > +                       pix_fmt_mp->plane_fmt[0].sizeimage = size;
> > +               pix_fmt_mp->plane_fmt[0].bytesperline = 0;
> > +               memset(&(pix_fmt_mp->plane_fmt[0].reserved[0]), 0x0,
> > +                       sizeof(pix_fmt_mp->plane_fmt[0].reserved));
> > +       } else if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
> > +               int i;
> > +               unsigned int sizeimage[VIDEO_MAX_PLANES];
> > +               unsigned int bytesperline[VIDEO_MAX_PLANES];
> > +
> > +               v4l_bound_align_image(&pix_fmt_mp->width, 8, 1920, 1,
> > +                                     &pix_fmt_mp->height, 4, 1080, 1, 0);
> > +
> > +               if (fmt->num_planes != pix_fmt_mp->num_planes)
> > +                       pix_fmt_mp->num_planes = fmt->num_planes;
> > +
> > +               mtk_vcodec_enc_calc_src_size(pix_fmt_mp->num_planes,
> > +                                       pix_fmt_mp->width,
> > +                                       pix_fmt_mp->height,
> > +                                       sizeimage, bytesperline);
> > +
> > +               for (i=0; i < pix_fmt_mp->num_planes; i++) {
> > +                       pix_fmt_mp->plane_fmt[i].sizeimage = sizeimage[i];
> > +                       pix_fmt_mp->plane_fmt[i].bytesperline = bytesperline[i];
> > +                       memset(&(pix_fmt_mp->plane_fmt[i].reserved[0]), 0x0,
> > +                               sizeof(pix_fmt_mp->plane_fmt[0].reserved));
> > +               }
> > +       } else {
> > +               return -EINVAL;
> > +       }
> > +
> > +       pix_fmt_mp->flags = 0;
> > +       pix_fmt_mp->ycbcr_enc = 0;
> > +       pix_fmt_mp->quantization = 0;
> > +       pix_fmt_mp->xfer_func = 0;
> > +       memset(&pix_fmt_mp->reserved[0], 0x0, sizeof(pix_fmt_mp->reserved));
> > +
> > +       return 0;
> > +}
> > +
> > +static void mtk_venc_set_param(struct mtk_vcodec_ctx *ctx, void *param)
> > +{
> > +       struct venc_enc_prm *p = (struct venc_enc_prm *)param;
> > +       struct mtk_q_data *q_data_src = &ctx->q_data[MTK_Q_DATA_SRC];
> > +       struct mtk_enc_params *enc_params = &ctx->enc_params;
> > +       unsigned int frame_rate;
> > +
> > +       frame_rate = enc_params->framerate_num / enc_params->framerate_denom;
> > +
> > +       switch (q_data_src->fmt->fourcc) {
> > +       case V4L2_PIX_FMT_YUV420:
> > +       case V4L2_PIX_FMT_YUV420M:
> > +               p->input_fourcc = VENC_YUV_FORMAT_420;
> > +               break;
> > +       case V4L2_PIX_FMT_YVU420:
> > +       case V4L2_PIX_FMT_YVU420M:
> > +               p->input_fourcc = VENC_YUV_FORMAT_YV12;
> > +               break;
> > +       case V4L2_PIX_FMT_NV12:
> > +       case V4L2_PIX_FMT_NV12M:
> > +               p->input_fourcc = VENC_YUV_FORMAT_NV12;
> > +               break;
> > +       case V4L2_PIX_FMT_NV21:
> > +       case V4L2_PIX_FMT_NV21M:
> > +               p->input_fourcc = VENC_YUV_FORMAT_NV21;
> > +               break;
> > +       }
> > +       p->h264_profile = enc_params->h264_profile;
> > +       p->h264_level = enc_params->h264_level;
> > +       p->width = q_data_src->width;
> > +       p->height = q_data_src->height;
> > +       p->buf_width = q_data_src->bytesperline[0];
> > +       p->buf_height = ((q_data_src->height + 0xf) & (~0xf));
> > +       p->frm_rate = frame_rate;
> > +       p->intra_period = enc_params->gop_size;
> > +       p->bitrate = enc_params->bitrate;
> > +
> > +       ctx->param_change = MTK_ENCODE_PARAM_NONE;
> > +
> > +       mtk_v4l2_debug(1, "fmt 0x%x, P/L %d/%d, w/h %d/%d, buf %d/%d, fps/bps %d/%d, gop %d",
> > +                      p->input_fourcc, p->h264_profile, p->h264_level,
> > +                      p->width, p->height, p->buf_width, p->buf_height,
> > +                      p->frm_rate, p->bitrate, p->intra_period);
> > +}
> > +
> > +static int vidioc_venc_s_fmt(struct file *file, void *priv,
> > +                            struct v4l2_format *f)
> > +{
> > +       struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
> > +       struct v4l2_pix_format_mplane *pix_fmt_mp = &f->fmt.pix_mp;
> > +       struct vb2_queue *vq;
> > +       struct mtk_q_data *q_data;
> > +       struct venc_enc_prm param;
> > +       int i, ret;
> > +       struct mtk_video_fmt *fmt;
> > +
> > +       vq = v4l2_m2m_get_vq(ctx->m2m_ctx, f->type);
> > +       if (!vq) {
> > +               mtk_v4l2_err("fail to get vq\n");
> > +               return -EINVAL;
> > +       }
> > +
> > +       if (vb2_is_busy(vq)) {
> > +               mtk_v4l2_err("queue busy\n");
> > +               return -EBUSY;
> > +       }
> > +
> > +       q_data = mtk_venc_get_q_data(ctx, f->type);
> > +       if (!q_data) {
> > +               mtk_v4l2_err("fail to get q data\n");
> > +               return -EINVAL;
> > +       }
> > +
> > +       fmt = mtk_venc_find_format(f);
> > +       if (!fmt) {
> > +               if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
> > +                       f->fmt.pix.pixelformat = mtk_video_formats[0].fourcc;
> > +                       fmt = mtk_venc_find_format(f);
> > +               } else {
> > +                       f->fmt.pix.pixelformat = mtk_video_formats[8].fourcc;
> > +                       fmt = mtk_venc_find_format(f);
> > +               }
> > +       }
> > +
> > +       q_data->fmt = fmt;
> > +       ret = vidioc_try_fmt(f, q_data->fmt);
> > +       if (ret)
> > +               return ret;
> > +
> > +       q_data->width           = f->fmt.pix_mp.width;
> > +       q_data->height          = f->fmt.pix_mp.height;
> > +       q_data->field           = f->fmt.pix_mp.field;
> > +
> > +       if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
> > +               q_data->colorspace = f->fmt.pix_mp.colorspace;
> > +               ctx->q_data[MTK_Q_DATA_SRC].bytesperline[0] =
> > +                       ALIGN(q_data->width, 16);
> > +
> > +               if (q_data->fmt->num_planes == 2) {
> > +                       ctx->q_data[MTK_Q_DATA_SRC].bytesperline[1] =
> > +                               ALIGN(q_data->width, 16);
> > +                       ctx->q_data[MTK_Q_DATA_SRC].bytesperline[2] = 0;
> > +               } else {
> > +                       ctx->q_data[MTK_Q_DATA_SRC].bytesperline[1] =
> > +                               ALIGN(q_data->width, 16) / 2;
> > +                       ctx->q_data[MTK_Q_DATA_SRC].bytesperline[2] =
> > +                               ALIGN(q_data->width, 16) / 2;
> > +               }
> > +
> > +               memset(&param, 0, sizeof(param));
> > +               mtk_venc_set_param(ctx, &param);
> > +               if (ctx->state == MTK_STATE_INIT) {
> > +                       ret = venc_if_set_param(ctx,
> > +                                               VENC_SET_PARAM_ENC,
> > +                                               &param);
> > +                       if (ret)
> > +                               mtk_v4l2_err("venc_if_set_param failed=%d\n",
> > +                                               ret);
> > +
> > +                       /* Get codec driver advice sizeimage from vpu */
> > +                       for (i = 0; i < MTK_VCODEC_MAX_PLANES; i++) {
> > +                               q_data->sizeimage[i] = param.sizeimage[i];
> > +                               pix_fmt_mp->plane_fmt[i].sizeimage =
> > +                                       param.sizeimage[i];
> > +                       }
> > +                       q_data->bytesperline[0] =
> > +                               pix_fmt_mp->plane_fmt[0].bytesperline;
> > +                       q_data->bytesperline[1] =
> > +                               pix_fmt_mp->plane_fmt[1].bytesperline;
> > +                       q_data->bytesperline[2] =
> > +                               pix_fmt_mp->plane_fmt[2].bytesperline;
> > +               } else {
> > +                       for (i = 0; i < MTK_VCODEC_MAX_PLANES; i++) {
> > +                               q_data->sizeimage[i] =
> > +                                       pix_fmt_mp->plane_fmt[i].sizeimage;
> > +                               q_data->bytesperline[i] =
> > +                                       pix_fmt_mp->plane_fmt[i].bytesperline;
> > +                       }
> > +               }
> > +       } else if (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE){
> > +               for (i = 0; i < f->fmt.pix_mp.num_planes; i++) {
> > +                       struct v4l2_plane_pix_format    *plane_fmt;
> > +
> > +                       plane_fmt = &f->fmt.pix_mp.plane_fmt[i];
> > +                       q_data->bytesperline[i] = plane_fmt->bytesperline;
> > +                       q_data->sizeimage[i]    = plane_fmt->sizeimage;
> > +               }
> > +
> > +               if (ctx->state == MTK_STATE_FREE) {
> > +                       ret = venc_if_create(ctx, q_data->fmt->fourcc);
> > +                       if (ret) {
> > +                               mtk_v4l2_err("venc_if_create failed=%d, codec type=%x\n",
> > +                                       ret, q_data->fmt->fourcc);
> > +                               return 0;
> > +                       }
> > +
> > +                       ctx->state = MTK_STATE_INIT;
> > +               }
> > +       }
> > +
> > +       return 0;
> > +}
> > +
> > +static int vidioc_venc_g_fmt(struct file *file, void *priv,
> > +                            struct v4l2_format *f)
> > +{
> > +       struct v4l2_pix_format_mplane *pix = &f->fmt.pix_mp;
> > +       struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
> > +       struct vb2_queue *vq;
> > +       struct mtk_q_data *q_data;
> > +       int i;
> > +
> > +       vq = v4l2_m2m_get_vq(ctx->m2m_ctx, f->type);
> > +       if (!vq)
> > +               return -EINVAL;
> > +
> > +       q_data = mtk_venc_get_q_data(ctx, f->type);
> > +
> > +       pix->width = q_data->width;
> > +       pix->height = q_data->height;
> > +       pix->pixelformat = q_data->fmt->fourcc;
> > +       pix->field = q_data->field;
> > +       pix->colorspace = q_data->colorspace;
> > +       pix->num_planes = q_data->fmt->num_planes;
> > +       for (i = 0; i < pix->num_planes; i++) {
> > +               pix->plane_fmt[i].bytesperline = q_data->bytesperline[i];
> > +               pix->plane_fmt[i].sizeimage = q_data->sizeimage[i];
> > +               memset(&(pix->plane_fmt[i].reserved[0]), 0x0,
> > +                       sizeof(pix->plane_fmt[i].reserved));
> > +       }
> > +       pix->flags = 0;
> > +       pix->ycbcr_enc = 0;
> > +       pix->quantization = 0;
> > +       pix->xfer_func = 0;
> > +       memset(&pix->reserved[0], 0x0, sizeof(pix->reserved));
> > +
> > +       return 0;
> > +}
> > +
> > +static int vidioc_try_fmt_vid_cap_mplane(struct file *file, void *priv,
> > +                                  struct v4l2_format *f)
> > +{
> > +        struct mtk_video_fmt *fmt;
> > +       struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
> > +
> > +        fmt = mtk_venc_find_format(f);
> > +        if (!fmt) {
> > +                f->fmt.pix.pixelformat = mtk_video_formats[8].fourcc;
> > +                fmt = mtk_venc_find_format(f);
> > +        }
> > +        if (fmt->type != MTK_FMT_ENC) {
> > +               mtk_v4l2_err("Fourcc format (0x%08x) invalid.\n",
> > +                            f->fmt.pix.pixelformat);
> > +               return -EINVAL;
> > +        }
> > +        f->fmt.pix_mp.colorspace = ctx->q_data[MTK_Q_DATA_SRC].colorspace;
> > +
> > +        return vidioc_try_fmt(f, fmt);
> > +}
> > +
> > +static int vidioc_try_fmt_vid_out_mplane(struct file *file, void *priv,
> > +                                  struct v4l2_format *f)
> > +{
> > +        struct mtk_video_fmt *fmt;
> > +
> > +        fmt = mtk_venc_find_format(f);
> > +        if (!fmt) {
> > +                f->fmt.pix.pixelformat = mtk_video_formats[0].fourcc;
> > +                fmt = mtk_venc_find_format(f);
> > +        }
> > +        if (!(fmt->type & MTK_FMT_FRAME)) {
> > +               mtk_v4l2_err("Fourcc format (0x%08x) invalid.\n",
> > +                            f->fmt.pix.pixelformat);
> > +               return -EINVAL;
> > +        }
> > +        if (!f->fmt.pix_mp.colorspace)
> > +                f->fmt.pix_mp.colorspace = V4L2_COLORSPACE_REC709;
> > +
> > +        return vidioc_try_fmt(f, fmt);
> > +}
> > +
> > +static int vidioc_venc_g_s_selection(struct file *file, void *priv,
> > +                                struct v4l2_selection *s)
> > +{
> > +       struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
> > +       struct mtk_q_data *q_data;
> > +
> > +       if (V4L2_TYPE_IS_OUTPUT(s->type)) {
> > +               if (s->target !=  V4L2_SEL_TGT_COMPOSE)
> > +                       return -EINVAL;
> > +       } else {
> > +               if (s->target != V4L2_SEL_TGT_CROP)
> > +                       return -EINVAL;
> > +       }
> > +
> > +       if (s->r.left || s->r.top)
> > +               return -EINVAL;
> > +
> > +       q_data = mtk_venc_get_q_data(ctx, s->type);
> > +       if (!q_data)
> > +               return -EINVAL;
> > +
> > +       s->r.width = q_data->width;
> > +       s->r.height = q_data->height;
> > +
> > +       return 0;
> > +}
> > +
> > +
> > +static int vidioc_venc_qbuf(struct file *file, void *priv,
> > +                           struct v4l2_buffer *buf)
> > +{
> > +
> > +       struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
> > +
> > +       if (ctx->state == MTK_STATE_ABORT) {
> > +               mtk_v4l2_err("[%d] Call on QBUF after unrecoverable error\n", ctx->idx);
> > +               return -EIO;
> > +       }
> > +
> > +       return v4l2_m2m_qbuf(file, ctx->m2m_ctx, buf);
> > +}
> > +
> > +static int vidioc_venc_dqbuf(struct file *file, void *priv,
> > +                            struct v4l2_buffer *buf)
> > +{
> > +       struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
> > +       if (ctx->state == MTK_STATE_ABORT) {
> > +               mtk_v4l2_err("[%d] Call on QBUF after unrecoverable error\n", ctx->idx);
> > +               return -EIO;
> > +       }
> > +
> > +       return v4l2_m2m_dqbuf(file, ctx->m2m_ctx, buf);
> > +}
> > +
> > +
> > +const struct v4l2_ioctl_ops mtk_venc_ioctl_ops = {
> > +       .vidioc_streamon                = v4l2_m2m_ioctl_streamon,
> > +       .vidioc_streamoff               = v4l2_m2m_ioctl_streamoff,
> > +
> > +       .vidioc_reqbufs                 = v4l2_m2m_ioctl_reqbufs,
> > +       .vidioc_querybuf                = v4l2_m2m_ioctl_querybuf,
> > +       .vidioc_qbuf                    = vidioc_venc_qbuf,
> > +       .vidioc_dqbuf                   = vidioc_venc_dqbuf,
> > +
> > +       .vidioc_querycap                = vidioc_venc_querycap,
> > +       .vidioc_enum_fmt_vid_cap_mplane = vidioc_enum_fmt_vid_cap_mplane,
> > +       .vidioc_enum_fmt_vid_out_mplane = vidioc_enum_fmt_vid_out_mplane,
> > +       .vidioc_enum_framesizes         = vidioc_enum_framesizes,
> > +
> > +       .vidioc_try_fmt_vid_cap_mplane  = vidioc_try_fmt_vid_cap_mplane,
> > +       .vidioc_try_fmt_vid_out_mplane  = vidioc_try_fmt_vid_out_mplane,
> > +       .vidioc_expbuf                  = v4l2_m2m_ioctl_expbuf,
> > +       .vidioc_subscribe_event         = v4l2_ctrl_subscribe_event,
> > +       .vidioc_unsubscribe_event       = v4l2_event_unsubscribe,
> > +
> > +       .vidioc_s_parm                  = vidioc_venc_s_parm,
> > +
> > +       .vidioc_s_fmt_vid_cap_mplane    = vidioc_venc_s_fmt,
> > +       .vidioc_s_fmt_vid_out_mplane    = vidioc_venc_s_fmt,
> > +
> > +       .vidioc_g_fmt_vid_cap_mplane    = vidioc_venc_g_fmt,
> > +       .vidioc_g_fmt_vid_out_mplane    = vidioc_venc_g_fmt,
> > +
> > +       .vidioc_g_selection             = vidioc_venc_g_s_selection,
> > +       .vidioc_s_selection             = vidioc_venc_g_s_selection,
> > +};
> > +
> > +static int vb2ops_venc_queue_setup(struct vb2_queue *vq,
> > +                                  const void *parg,
> > +                                  unsigned int *nbuffers,
> > +                                  unsigned int *nplanes,
> > +                                  unsigned int sizes[], void *alloc_ctxs[])
> > +{
> > +       struct mtk_vcodec_ctx *ctx = vb2_get_drv_priv(vq);
> > +       struct mtk_q_data *q_data;
> > +
> > +       q_data = mtk_venc_get_q_data(ctx, vq->type);
> > +
> > +       if (*nbuffers < 1)
> > +               *nbuffers = 1;
> > +       if (*nbuffers > MTK_VIDEO_MAX_FRAME)
> > +               *nbuffers = MTK_VIDEO_MAX_FRAME;
> > +
> > +       *nplanes = q_data->fmt->num_planes;
> > +
> > +       if (vq->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
> > +               unsigned int i;
> > +
> > +               for (i = 0; i < *nplanes; i++) {
> > +                       sizes[i] = q_data->sizeimage[i];
> > +                       alloc_ctxs[i] = ctx->dev->alloc_ctx;
> > +               }
> > +       } else if (vq->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
> > +               sizes[0] = q_data->sizeimage[0];
> > +               alloc_ctxs[0] = ctx->dev->alloc_ctx;
> > +       } else {
> > +               return -EINVAL;
> > +       }
> > +
> > +       mtk_v4l2_debug(2, "[%d]get %d buffer(s) of size 0x%x each, vq->memory=%d",
> > +                      ctx->idx, *nbuffers, sizes[0], vq->memory);
> > +
> > +       return 0;
> > +}
> > +
> > +static int vb2ops_venc_buf_prepare(struct vb2_buffer *vb)
> > +{
> > +       struct mtk_vcodec_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue);
> > +       struct mtk_q_data *q_data;
> > +       int i;
> > +
> > +       q_data = mtk_venc_get_q_data(ctx, vb->vb2_queue->type);
> > +
> > +       for (i = 0; i < q_data->fmt->num_planes; i++) {
> > +               if (vb2_plane_size(vb, i) < q_data->sizeimage[i]) {
> > +                       mtk_v4l2_debug(2, "data will not fit into plane %d (%lu < %d)",
> > +                                      i, vb2_plane_size(vb, i),
> > +                                      q_data->sizeimage[i]);
> > +                       return -EINVAL;
> > +               }
> > +       }
> > +
> > +       return 0;
> > +}
> > +
> > +static void vb2ops_venc_buf_queue(struct vb2_buffer *vb)
> > +{
> > +       struct mtk_vcodec_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue);
> > +       struct vb2_v4l2_buffer *vb2_v4l2 =
> > +                       container_of(vb, struct vb2_v4l2_buffer, vb2_buf);
> > +       struct mtk_video_enc_buf *mtk_buf =
> > +                       container_of(vb2_v4l2, struct mtk_video_enc_buf, vb);
> > +
> > +       if ((vb->vb2_queue->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) &&
> > +               (ctx->param_change != MTK_ENCODE_PARAM_NONE)) {
> > +               mtk_v4l2_debug(1,
> > +                               "[%d] Before id=%d encode parameter change %x",
> > +                               ctx->idx, vb->index,
> > +                               ctx->param_change);
> > +               mtk_buf->param_change = ctx->param_change;
> > +               if (mtk_buf->param_change & MTK_ENCODE_PARAM_BITRATE) {
> > +                       mtk_buf->enc_params.bitrate = ctx->enc_params.bitrate;
> > +                       mtk_v4l2_debug(1, "[%d] idx=%d change param br=%d",
> > +                               ctx->idx,
> > +                               mtk_buf->vb.vb2_buf.index,
> > +                               mtk_buf->enc_params.bitrate);
> > +               }
> > +               if (ctx->param_change & MTK_ENCODE_PARAM_FRAMERATE) {
> > +                       mtk_buf->enc_params.framerate_num =
> > +                               ctx->enc_params.framerate_num;
> > +                       mtk_buf->enc_params.framerate_denom =
> > +                               ctx->enc_params.framerate_denom;
> > +                       mtk_v4l2_debug(1, "[%d] idx=%d, change param fr=%d/%d",
> > +                                       ctx->idx,
> > +                                       mtk_buf->vb.vb2_buf.index,
> > +                                       mtk_buf->enc_params.framerate_num,
> > +                                       mtk_buf->enc_params.framerate_denom);
> > +               }
> > +               if (ctx->param_change & MTK_ENCODE_PARAM_INTRA_PERIOD) {
> > +                       mtk_buf->enc_params.gop_size = ctx->enc_params.gop_size;
> > +                       mtk_v4l2_debug(1, "[%d] idx=%d, change param intra period=%d",
> > +                                       ctx->idx,
> > +                                       mtk_buf->vb.vb2_buf.index,
> > +                                       mtk_buf->enc_params.gop_size);
> > +               }
> > +               if (ctx->param_change & MTK_ENCODE_PARAM_FRAME_TYPE) {
> > +                       mtk_buf->enc_params.force_intra =
> > +                               ctx->enc_params.force_intra;
> > +                       mtk_v4l2_debug(1, "[%d] idx=%d, change param force I=%d",
> > +                                       ctx->idx,
> > +                                       mtk_buf->vb.vb2_buf.index,
> > +                                       mtk_buf->enc_params.force_intra);
> > +               }
> > +               ctx->param_change = MTK_ENCODE_PARAM_NONE;
> > +       }
> > +
> > +       v4l2_m2m_buf_queue(ctx->m2m_ctx, to_vb2_v4l2_buffer(vb));
> > +}
> > +
> > +static int vb2ops_venc_start_streaming(struct vb2_queue *q, unsigned int count)
> > +{
> > +       struct mtk_vcodec_ctx *ctx = vb2_get_drv_priv(q);
> > +       struct venc_enc_prm param;
> > +       int ret;
> > +       int i;
> > +
> > +       /* Once state turn into MTK_STATE_ABORT, we need stop_streaming to clear it */
> > +       if ((ctx->state == MTK_STATE_ABORT) || (ctx->state == MTK_STATE_FREE))
> > +               goto err_set_param;
> > +
> > +       if (!(vb2_start_streaming_called(&ctx->m2m_ctx->out_q_ctx.q) &
> > +             vb2_start_streaming_called(&ctx->m2m_ctx->cap_q_ctx.q))) {
> > +               mtk_v4l2_debug(1, "[%d]-> out=%d cap=%d",
> > +                ctx->idx,
> > +                vb2_start_streaming_called(&ctx->m2m_ctx->out_q_ctx.q),
> > +                vb2_start_streaming_called(&ctx->m2m_ctx->cap_q_ctx.q));
> > +               return 0;
> > +       }
> > +
> > +       mtk_venc_set_param(ctx, &param);
> > +       ret = venc_if_set_param(ctx,
> > +                               VENC_SET_PARAM_ENC,
> > +                               &param);
> > +       if (ret) {
> > +               mtk_v4l2_err("venc_if_set_param failed=%d\n", ret);
> > +               ctx->state = MTK_STATE_ABORT;
> > +               goto err_set_param;
> > +       }
> > +
> > +       if ((ctx->q_data[MTK_Q_DATA_DST].fmt->fourcc == V4L2_PIX_FMT_H264) &&
> > +           (ctx->enc_params.seq_hdr_mode != V4L2_MPEG_VIDEO_HEADER_MODE_SEPARATE)) {
> > +               ret = venc_if_set_param(ctx,
> > +                                       VENC_SET_PARAM_PREPEND_HEADER,
> > +                                       0);
> > +               if (ret) {
> > +                       mtk_v4l2_err("venc_if_set_param failed=%d\n", ret);
> > +                       ctx->state = MTK_STATE_ABORT;
> > +                       goto err_set_param;
> > +               }
> > +               ctx->state = MTK_STATE_HEADER;
> > +       }
> > +
> > +       return 0;
> > +
> > +err_set_param:
> > +       for (i = 0; i < q->num_buffers; ++i) {
> > +               if (q->bufs[i]->state == VB2_BUF_STATE_ACTIVE) {
> > +                       mtk_v4l2_debug(0, "[%d] idx=%d, type=%d, %d -> VB2_BUF_STATE_QUEUED",
> > +                                       ctx->idx, i, q->type,
> > +                                       (int)q->bufs[i]->state );
> > +                       v4l2_m2m_buf_done(to_vb2_v4l2_buffer(q->bufs[i]), VB2_BUF_STATE_QUEUED);
> > +               }
> > +       }
> > +
> > +       return -EINVAL;
> > +}
> > +
> > +static void vb2ops_venc_stop_streaming(struct vb2_queue *q)
> > +{
> > +       struct mtk_vcodec_ctx *ctx = vb2_get_drv_priv(q);
> > +       struct vb2_buffer *src_buf, *dst_buf;
> > +       int ret;
> > +
> > +       mtk_v4l2_debug(2, "[%d]-> type=%d", ctx->idx, q->type);
> > +
> > +       if (q->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
> > +               while ((dst_buf = v4l2_m2m_dst_buf_remove(ctx->m2m_ctx))) {
> > +                       dst_buf->planes[0].bytesused = 0;
> > +                       v4l2_m2m_buf_done(to_vb2_v4l2_buffer(dst_buf),
> > +                                               VB2_BUF_STATE_ERROR);
> > +               }
> > +       } else {
> > +               while ((src_buf = v4l2_m2m_src_buf_remove(ctx->m2m_ctx)))
> > +                       v4l2_m2m_buf_done(to_vb2_v4l2_buffer(src_buf),
> > +                                               VB2_BUF_STATE_ERROR);
> > +       }
> > +
> > +       if ((q->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE &&
> > +            vb2_is_streaming(&ctx->m2m_ctx->out_q_ctx.q)) ||
> > +           (q->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE &&
> > +            vb2_is_streaming(&ctx->m2m_ctx->cap_q_ctx.q))) {
> > +               mtk_v4l2_debug(1, "[%d]-> q type %d out=%d cap=%d",
> > +                              ctx->idx, q->type,
> > +                              vb2_is_streaming(&ctx->m2m_ctx->out_q_ctx.q),
> > +                              vb2_is_streaming(&ctx->m2m_ctx->cap_q_ctx.q));
> > +               return;
> > +       }
> > +
> > +       ret = venc_if_release(ctx);
> > +       if (ret)
> > +               mtk_v4l2_err("venc_if_release failed=%d\n", ret);
> > +
> > +       ctx->state = MTK_STATE_FREE;
> > +}
> > +
> > +static struct vb2_ops mtk_venc_vb2_ops = {
> > +       .queue_setup                    = vb2ops_venc_queue_setup,
> > +       .buf_prepare                    = vb2ops_venc_buf_prepare,
> > +       .buf_queue                      = vb2ops_venc_buf_queue,
> > +       .wait_prepare                   = vb2_ops_wait_prepare,
> > +       .wait_finish                    = vb2_ops_wait_finish,
> > +       .start_streaming                = vb2ops_venc_start_streaming,
> > +       .stop_streaming                 = vb2ops_venc_stop_streaming,
> > +};
> > +
> > +static int mtk_venc_encode_header(void *priv)
> > +{
> > +       struct mtk_vcodec_ctx *ctx = priv;
> > +       int ret;
> > +       struct vb2_buffer *dst_buf;
> > +       struct mtk_vcodec_mem bs_buf;
> > +       struct venc_done_result enc_result;
> > +
> > +       dst_buf = v4l2_m2m_dst_buf_remove(ctx->m2m_ctx);
> > +       if (!dst_buf) {
> > +               mtk_v4l2_debug(1, "No dst buffer");
> > +               return -EINVAL;
> > +       }
> > +
> > +       bs_buf.va = vb2_plane_vaddr(dst_buf, 0);
> > +       bs_buf.dma_addr = vb2_dma_contig_plane_dma_addr(dst_buf, 0);
> > +       bs_buf.size = (unsigned int)dst_buf->planes[0].length;
> > +
> > +       mtk_v4l2_debug(1,
> > +                       "[%d] buf idx=%d va=0x%p dma_addr=0x%llx size=0x%lx",
> > +                       ctx->idx,
> > +                       dst_buf->index, bs_buf.va,
> > +                       (u64)bs_buf.dma_addr,
> > +                       bs_buf.size);
> > +
> > +       ret = venc_if_encode(ctx,
> > +                       VENC_START_OPT_ENCODE_SEQUENCE_HEADER,
> > +                       0, &bs_buf, &enc_result);
> > +
> > +       if (ret) {
> > +               dst_buf->planes[0].bytesused = 0;
> > +               ctx->state = MTK_STATE_ABORT;
> > +               v4l2_m2m_buf_done(to_vb2_v4l2_buffer(dst_buf), VB2_BUF_STATE_ERROR);
> > +               mtk_v4l2_err("venc_if_encode failed=%d", ret);
> > +               return -EINVAL;
> > +       }
> > +
> > +       ctx->state = MTK_STATE_HEADER;
> > +       dst_buf->planes[0].bytesused = enc_result.bs_size;
> > +
> > +#if defined(DEBUG)
> > +{
> > +       int i;
> > +       mtk_v4l2_debug(1, "[%d] venc_if_encode header len=%d",
> > +                       ctx->idx,
> > +                       enc_result.bs_size);
> > +       for (i = 0; i < enc_result.bs_size; i++) {
> > +               unsigned char *p = (unsigned char *)bs_buf.va;
> > +
> > +               mtk_v4l2_debug(1, "[%d] buf[%d]=0x%2x", ctx->idx, i, p[i]);
> > +       }
> > +}
> > +#endif
> > +       v4l2_m2m_buf_done(to_vb2_v4l2_buffer(dst_buf), VB2_BUF_STATE_DONE);
> > +
> > +       return 0;
> > +}
> > +
> > +static int mtk_venc_param_change(struct mtk_vcodec_ctx *ctx, void *priv)
> > +{
> > +       struct vb2_buffer *vb = priv;
> > +       struct vb2_v4l2_buffer *vb2_v4l2 =
> > +                       container_of(vb, struct vb2_v4l2_buffer, vb2_buf);
> > +       struct mtk_video_enc_buf *mtk_buf =
> > +                       container_of(vb2_v4l2, struct mtk_video_enc_buf, vb);
> > +       int ret = 0;
> > +
> > +       if (mtk_buf->param_change == MTK_ENCODE_PARAM_NONE)
> > +               return 0;
> > +
> > +       mtk_v4l2_debug(1, "encode parameters change id=%d", vb->index);
> > +       if (mtk_buf->param_change & MTK_ENCODE_PARAM_BITRATE) {
> > +               struct venc_enc_prm enc_prm;
> > +
> > +               enc_prm.bitrate = mtk_buf->enc_params.bitrate;
> > +               mtk_v4l2_debug(1, "[%d] idx=%d, change param br=%d",
> > +                               ctx->idx,
> > +                               mtk_buf->vb.vb2_buf.index,
> > +                               enc_prm.bitrate);
> > +               ret |= venc_if_set_param(ctx,
> > +                                        VENC_SET_PARAM_ADJUST_BITRATE,
> > +                                        &enc_prm);
> > +       }
> > +       if (mtk_buf->param_change & MTK_ENCODE_PARAM_FRAMERATE) {
> > +               struct venc_enc_prm enc_prm;
> > +
> > +               enc_prm.frm_rate = mtk_buf->enc_params.framerate_num /
> > +                                  mtk_buf->enc_params.framerate_denom;
> > +               mtk_v4l2_debug(1, "[%d] idx=%d, change param fr=%d",
> > +                              ctx->idx,
> > +                              mtk_buf->vb.vb2_buf.index,
> > +                              enc_prm.frm_rate);
> > +               ret |= venc_if_set_param(ctx,
> > +                                        VENC_SET_PARAM_ADJUST_FRAMERATE,
> > +                                        &enc_prm);
> > +       }
> > +       if (mtk_buf->param_change & MTK_ENCODE_PARAM_INTRA_PERIOD) {
> > +               mtk_v4l2_debug(1, "change param intra period=%d",
> > +                                mtk_buf->enc_params.gop_size);
> > +               ret |= venc_if_set_param(ctx,
> > +                                        VENC_SET_PARAM_I_FRAME_INTERVAL,
> > +                                        &mtk_buf->enc_params.gop_size);
> > +       }
> > +       if (mtk_buf->param_change & MTK_ENCODE_PARAM_FRAME_TYPE) {
> > +               mtk_v4l2_debug(1, "[%d] idx=%d, change param force I=%d",
> > +                               ctx->idx,
> > +                               mtk_buf->vb.vb2_buf.index,
> > +                               mtk_buf->enc_params.force_intra);
> > +               if (mtk_buf->enc_params.force_intra)
> > +                       ret |= venc_if_set_param(ctx,
> > +                                                VENC_SET_PARAM_FORCE_INTRA,
> > +                                                0);
> > +       }
> > +
> > +       mtk_buf->param_change = MTK_ENCODE_PARAM_NONE;
> > +
> > +       if (ret) {
> > +               ctx->state = MTK_STATE_ABORT;
> > +               mtk_v4l2_err("venc_if_set_param %d failed=%d\n",
> > +                       MTK_ENCODE_PARAM_FRAME_TYPE, ret);
> > +               return -1;
> > +       }
> > +
> > +       return 0;
> > +}
> > +
> > +static void mtk_venc_worker(struct work_struct *work)
> > +{
> > +       struct mtk_vcodec_ctx *ctx = container_of(work, struct mtk_vcodec_ctx,
> > +                                   encode_work);
> > +       struct vb2_buffer *src_buf, *dst_buf;
> > +       struct venc_frm_buf frm_buf;
> > +       struct mtk_vcodec_mem bs_buf;
> > +       struct venc_done_result enc_result;
> > +       int ret;
> > +       struct vb2_v4l2_buffer *v4l2_vb;
> > +
> > +       if ((ctx->q_data[MTK_Q_DATA_DST].fmt->fourcc == V4L2_PIX_FMT_H264) &&
> > +           (ctx->state != MTK_STATE_HEADER)) {
> > +               /* encode h264 sps/pps header */
> > +               mtk_venc_encode_header(ctx);
> > +               v4l2_m2m_job_finish(ctx->dev->m2m_dev_enc, ctx->m2m_ctx);
> > +               return;
> > +       }
> > +
> > +       src_buf = v4l2_m2m_next_src_buf(ctx->m2m_ctx);
> > +       if (!src_buf) {
> > +               v4l2_m2m_job_finish(ctx->dev->m2m_dev_enc, ctx->m2m_ctx);
> > +               return;
> > +       }
> > +
> > +       dst_buf = v4l2_m2m_dst_buf_remove(ctx->m2m_ctx);
> > +       if (!dst_buf) {
> > +               v4l2_m2m_job_finish(ctx->dev->m2m_dev_enc, ctx->m2m_ctx);
> > +               return;
> > +       }
> > +
> > +       mtk_venc_param_change(ctx, src_buf);
> > +
> > +       frm_buf.fb_addr.va = vb2_plane_vaddr(src_buf, 0);
> > +       frm_buf.fb_addr.dma_addr = vb2_dma_contig_plane_dma_addr(src_buf, 0);
> > +       frm_buf.fb_addr.size = (unsigned int)src_buf->planes[0].length;
> > +       frm_buf.fb_addr1.va = vb2_plane_vaddr(src_buf, 1);
> > +       frm_buf.fb_addr1.dma_addr = vb2_dma_contig_plane_dma_addr(src_buf, 1);
> > +       frm_buf.fb_addr1.size = (unsigned int)src_buf->planes[1].length;
> > +       if (src_buf->num_planes == 3) {
> > +               frm_buf.fb_addr2.va = vb2_plane_vaddr(src_buf, 2);
> > +               frm_buf.fb_addr2.dma_addr =
> > +                       vb2_dma_contig_plane_dma_addr(src_buf, 2);
> > +               frm_buf.fb_addr2.size =
> > +                       (unsigned int)src_buf->planes[2].length;
> > +       } else {
> > +               frm_buf.fb_addr2.va = NULL;
> > +               frm_buf.fb_addr2.dma_addr = 0;
> > +               frm_buf.fb_addr2.size = 0;
> > +       }
> > +       bs_buf.va = vb2_plane_vaddr(dst_buf, 0);
> > +       bs_buf.dma_addr = vb2_dma_contig_plane_dma_addr(dst_buf, 0);
> > +       bs_buf.size = (unsigned int)dst_buf->planes[0].length;
> > +
> > +       mtk_v4l2_debug(2,
> > +                       "Framebuf VA=%p PA=%llx Size=0x%lx;VA=%p PA=0x%llx Size=0x%lx;VA=%p PA=0x%llx Size=0x%lx",
> > +                       frm_buf.fb_addr.va,
> > +                       (u64)frm_buf.fb_addr.dma_addr,
> > +                       frm_buf.fb_addr.size,
> > +                       frm_buf.fb_addr1.va,
> > +                       (u64)frm_buf.fb_addr1.dma_addr,
> > +                       frm_buf.fb_addr1.size,
> > +                       frm_buf.fb_addr2.va,
> > +                       (u64)frm_buf.fb_addr2.dma_addr,
> > +                       frm_buf.fb_addr2.size);
> > +
> > +       ret = venc_if_encode(ctx, VENC_START_OPT_ENCODE_FRAME,
> > +                            &frm_buf, &bs_buf, &enc_result);
> > +
> > +       src_buf = v4l2_m2m_src_buf_remove(ctx->m2m_ctx);
> > +       if (enc_result.msg == VENC_MESSAGE_OK)
> > +               v4l2_m2m_buf_done(to_vb2_v4l2_buffer(src_buf), VB2_BUF_STATE_DONE);
> > +       else
> > +               v4l2_m2m_buf_done(to_vb2_v4l2_buffer(src_buf), VB2_BUF_STATE_ERROR);
> > +
> > +       if (enc_result.is_key_frm) {
> > +               v4l2_vb = to_vb2_v4l2_buffer(dst_buf);
> > +               v4l2_vb->flags |= V4L2_BUF_FLAG_KEYFRAME;
> > +       }
> > +
> > +       if (ret) {
> > +               dst_buf->planes[0].bytesused = 0;
> > +               v4l2_m2m_buf_done(to_vb2_v4l2_buffer(dst_buf), VB2_BUF_STATE_ERROR);
> > +               mtk_v4l2_err("venc_if_encode failed=%d", ret);
> > +       } else {
> > +               dst_buf->planes[0].bytesused = enc_result.bs_size;
> > +               v4l2_m2m_buf_done(to_vb2_v4l2_buffer(dst_buf), VB2_BUF_STATE_DONE);
> > +               mtk_v4l2_debug(2, "venc_if_encode bs size=%d",
> > +                                enc_result.bs_size);
> > +       }
> > +
> > +       v4l2_m2m_job_finish(ctx->dev->m2m_dev_enc, ctx->m2m_ctx);
> > +
> > +       mtk_v4l2_debug(1, "<=== src_buf[%d] dst_buf[%d] venc_if_encode ret=%d Size=%u===>",
> > +                       src_buf->index, dst_buf->index, ret,
> > +                       enc_result.bs_size);
> > +}
> > +
> > +static void m2mops_venc_device_run(void *priv)
> > +{
> > +       struct mtk_vcodec_ctx *ctx = priv;
> > +       queue_work(ctx->dev->encode_workqueue, &ctx->encode_work);
> > +}
> > +
> > +static int m2mops_venc_job_ready(void *m2m_priv)
> > +{
> > +       struct mtk_vcodec_ctx *ctx = m2m_priv;
> > +
> > +       if (!v4l2_m2m_num_dst_bufs_ready(ctx->m2m_ctx)) {
> > +               mtk_v4l2_debug(3, "[%d]Not ready: not enough video dst buffers.",
> > +                              ctx->idx);
> > +               return 0;
> > +       }
> > +
> > +       if (!v4l2_m2m_num_src_bufs_ready(ctx->m2m_ctx)) {
> > +               mtk_v4l2_debug(3, "[%d]Not ready: not enough video src buffers.",
> > +                              ctx->idx);
> > +                       return 0;
> > +               }
> > +
> > +       if (ctx->state == MTK_STATE_ABORT) {
> > +               mtk_v4l2_debug(3, "[%d]Not ready: state=0x%x.",
> > +                              ctx->idx, ctx->state);
> > +               return 0;
> > +       }
> > +
> > +       if (ctx->state == MTK_STATE_FREE) {
> > +               mtk_v4l2_debug(3, "[%d]Not ready: state=0x%x.",
> > +                              ctx->idx, ctx->state);
> > +               return 0;
> > +       }
> > +
> > +       return 1;
> > +}
> > +
> > +static void m2mops_venc_job_abort(void *priv)
> > +{
> > +       struct mtk_vcodec_ctx *ctx = priv;
> > +
> > +       ctx->state = MTK_STATE_ABORT;
> > +}
> > +
> > +static void m2mops_venc_lock(void *m2m_priv)
> > +{
> > +       struct mtk_vcodec_ctx *ctx = m2m_priv;
> > +
> > +       mutex_lock(&ctx->dev->dev_mutex);
> > +}
> > +
> > +static void m2mops_venc_unlock(void *m2m_priv)
> > +{
> > +       struct mtk_vcodec_ctx *ctx = m2m_priv;
> > +
> > +       mutex_unlock(&ctx->dev->dev_mutex);
> > +}
> > +
> > +const struct v4l2_m2m_ops mtk_venc_m2m_ops = {
> > +       .device_run                     = m2mops_venc_device_run,
> > +       .job_ready                      = m2mops_venc_job_ready,
> > +       .job_abort                      = m2mops_venc_job_abort,
> > +       .lock                           = m2mops_venc_lock,
> > +       .unlock                         = m2mops_venc_unlock,
> > +};
> > +
> > +#define IS_MTK_VENC_PRIV(x) ((V4L2_CTRL_ID2CLASS(x) == V4L2_CTRL_CLASS_MPEG) &&\
> > +                            V4L2_CTRL_DRIVER_PRIV(x))
> > +
> > +void mtk_vcodec_enc_ctx_params_setup(struct mtk_vcodec_ctx *ctx)
> > +{
> > +       struct mtk_q_data *q_data;
> > +       struct mtk_video_fmt *fmt;
> > +
> > +       ctx->m2m_ctx->q_lock = &ctx->dev->dev_mutex;
> > +       ctx->fh.m2m_ctx = ctx->m2m_ctx;
> > +       ctx->fh.ctrl_handler = &ctx->ctrl_hdl;
> > +       INIT_WORK(&ctx->encode_work, mtk_venc_worker);
> > +
> > +       ctx->q_data[MTK_Q_DATA_SRC].width = DFT_CFG_WIDTH;
> > +       ctx->q_data[MTK_Q_DATA_SRC].height = DFT_CFG_HEIGHT;
> > +       ctx->q_data[MTK_Q_DATA_SRC].fmt = &mtk_video_formats[0];
> > +       ctx->q_data[MTK_Q_DATA_SRC].colorspace = V4L2_COLORSPACE_REC709;
> > +       ctx->q_data[MTK_Q_DATA_SRC].field = V4L2_FIELD_NONE;
> > +
> > +       q_data = &ctx->q_data[MTK_Q_DATA_SRC];
> > +       fmt = ctx->q_data[MTK_Q_DATA_SRC].fmt;
> > +       mtk_vcodec_enc_calc_src_size(fmt->num_planes, q_data->width,
> > +                       q_data->height,
> > +                       ctx->q_data[MTK_Q_DATA_SRC].sizeimage,
> > +                       ctx->q_data[MTK_Q_DATA_SRC].bytesperline);
> > +
> > +       ctx->q_data[MTK_Q_DATA_DST].width = DFT_CFG_WIDTH;
> > +       ctx->q_data[MTK_Q_DATA_DST].height = DFT_CFG_HEIGHT;
> > +       ctx->q_data[MTK_Q_DATA_DST].fmt = &mtk_video_formats[9];
> > +       ctx->q_data[MTK_Q_DATA_DST].colorspace = V4L2_COLORSPACE_REC709;
> > +       ctx->q_data[MTK_Q_DATA_DST].field = V4L2_FIELD_NONE;
> > +
> > +       q_data = &ctx->q_data[MTK_Q_DATA_DST];
> > +       fmt = ctx->q_data[MTK_Q_DATA_DST].fmt;
> > +       ctx->q_data[MTK_Q_DATA_DST].sizeimage[0] = q_data->width * q_data->height;
> > +       ctx->q_data[MTK_Q_DATA_DST].bytesperline[0] = 0;
> > +
> > +}
> > +
> > +int mtk_vcodec_enc_ctrls_setup(struct mtk_vcodec_ctx *ctx)
> > +{
> > +       const struct v4l2_ctrl_ops *ops = &mtk_vcodec_enc_ctrl_ops;
> > +       struct v4l2_ctrl_handler *handler = &ctx->ctrl_hdl;
> > +       struct v4l2_ctrl_config cfg;
> > +
> > +       v4l2_ctrl_handler_init(handler, MTK_MAX_CTRLS);
> > +       if (handler->error) {
> > +               mtk_v4l2_err("Init control handler fail %d\n",
> > +                               handler->error);
> > +               return handler->error;
> > +       }
> > +
> > +       ctx->ctrls[0] = v4l2_ctrl_new_std(handler, ops,
> > +                                       V4L2_CID_MPEG_VIDEO_BITRATE,
> > +                                       1, 4000000, 1, 4000000);
> > +       ctx->ctrls[1] = v4l2_ctrl_new_std(handler, ops,
> > +                                       V4L2_CID_MPEG_VIDEO_B_FRAMES,
> > +                                       0, 2, 1, 0);
> > +       ctx->ctrls[2] = v4l2_ctrl_new_std(handler, ops,
> > +                                       V4L2_CID_MPEG_VIDEO_FRAME_RC_ENABLE,
> > +                                       0, 1, 1, 1);
> > +       ctx->ctrls[3] = v4l2_ctrl_new_std(handler, ops,
> > +                                       V4L2_CID_MPEG_VIDEO_H264_MAX_QP,
> > +                                       0, 51, 1, 51);
> > +       ctx->ctrls[4] = v4l2_ctrl_new_std(handler, ops,
> > +                                       V4L2_CID_MPEG_VIDEO_H264_I_PERIOD,
> > +                                       0, 65535, 1, 30);
> > +       ctx->ctrls[5] = v4l2_ctrl_new_std(handler, ops,
> > +                                       V4L2_CID_MPEG_VIDEO_GOP_SIZE,
> > +                                       0, 65535, 1, 30);
> > +       ctx->ctrls[6] = v4l2_ctrl_new_std(handler, ops,
> > +                                       V4L2_CID_MPEG_VIDEO_MB_RC_ENABLE,
> > +                                       0, 1, 1, 0);
> > +       ctx->ctrls[7] = v4l2_ctrl_new_std_menu(handler, ops,
> > +                                       V4L2_CID_MPEG_VIDEO_HEADER_MODE,
> > +                                       V4L2_MPEG_VIDEO_HEADER_MODE_JOINED_WITH_1ST_FRAME,
> > +                                       0, V4L2_MPEG_VIDEO_HEADER_MODE_SEPARATE);
> > +       ctx->ctrls[8] = v4l2_ctrl_new_std_menu(handler, ops,
> > +                                       V4L2_CID_MPEG_VIDEO_H264_PROFILE,
> > +                                       V4L2_MPEG_VIDEO_H264_PROFILE_HIGH,
> > +                                       0, V4L2_MPEG_VIDEO_H264_PROFILE_MAIN);
> > +       ctx->ctrls[9] = v4l2_ctrl_new_std_menu(handler, ops,
> > +                                       V4L2_CID_MPEG_VIDEO_H264_LEVEL,
> > +                                       V4L2_MPEG_VIDEO_H264_LEVEL_4_2,
> > +                                       0, V4L2_MPEG_VIDEO_H264_LEVEL_4_0);
> > +       ctx->ctrls[6] = v4l2_ctrl_new_std(handler, ops,
> > +                                       V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME,
> > +                                       0, 0, 0, 0);
> > +
> > +       v4l2_ctrl_handler_setup(&ctx->ctrl_hdl);
> > +
> > +       return 0;
> > +}
> > +
> > +int mtk_vcodec_enc_queue_init(void *priv, struct vb2_queue *src_vq,
> > +                          struct vb2_queue *dst_vq)
> > +{
> > +       struct mtk_vcodec_ctx *ctx = priv;
> > +       int ret;
> > +
> > +       src_vq->type            = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE;
> > +       src_vq->io_modes        = VB2_DMABUF | VB2_MMAP | VB2_USERPTR;
> > +       src_vq->drv_priv        = ctx;
> > +       src_vq->buf_struct_size = sizeof(struct mtk_video_enc_buf);
> > +       src_vq->ops             = &mtk_venc_vb2_ops;
> > +       src_vq->mem_ops         = &vb2_dma_contig_memops;
> > +       src_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
> > +       src_vq->lock = &ctx->dev->dev_mutex;
> > +
> > +       ret = vb2_queue_init(src_vq);
> > +       if (ret)
> > +               return ret;
> > +
> > +       dst_vq->type            = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
> > +       dst_vq->io_modes        = VB2_DMABUF | VB2_MMAP | VB2_USERPTR;
> > +       dst_vq->drv_priv        = ctx;
> > +       dst_vq->buf_struct_size = sizeof(struct v4l2_m2m_buffer);
> > +       dst_vq->ops             = &mtk_venc_vb2_ops;
> > +       dst_vq->mem_ops         = &vb2_dma_contig_memops;
> > +       dst_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
> > +       dst_vq->lock = &ctx->dev->dev_mutex;
> > +
> > +       return vb2_queue_init(dst_vq);
> > +}
> > +
> > +int mtk_venc_unlock(struct mtk_vcodec_ctx *ctx)
> > +{
> > +       struct mtk_vcodec_dev *dev = ctx->dev;
> > +       dev->curr_ctx = -1;
> > +       mutex_unlock(&dev->enc_mutex);
> > +       return 0;
> > +}
> > +
> > +int mtk_venc_lock(struct mtk_vcodec_ctx *ctx)
> > +{
> > +       struct mtk_vcodec_dev *dev = ctx->dev;
> > +
> > +       mutex_lock(&dev->enc_mutex);
> > +       dev->curr_ctx = ctx->idx;
> > +       return 0;
> > +}
> > +
> > +void mtk_vcodec_enc_release(struct mtk_vcodec_ctx *ctx)
> > +{
> > +       venc_if_release(ctx);
> > +}
> > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h
> > new file mode 100644
> > index 0000000..e09524b
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h
> > @@ -0,0 +1,46 @@
> > +/*
> > +* Copyright (c) 2015 MediaTek Inc.
> > +* Author: PC Chen <pc.chen@mediatek.com>
> > +*         Tiffany Lin <tiffany.lin@mediatek.com>
> > +*
> > +* This program is free software; you can redistribute it and/or modify
> > +* it under the terms of the GNU General Public License version 2 as
> > +* published by the Free Software Foundation.
> > +*
> > +* This program is distributed in the hope that it will be useful,
> > +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> > +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > +* GNU General Public License for more details.
> > +*/
> > +
> > +
> > +#ifndef _MTK_VCODEC_ENC_H_
> > +#define _MTK_VCODEC_ENC_H_
> > +
> > +#include <media/videobuf2-core.h>
> > +#include <media/videobuf2-v4l2.h>
> > +
> > +/**
> > + * struct mtk_video_enc_buf - Private data related to each VB2 buffer.
> > + * @b:                 Pointer to related VB2 buffer.
> > + * @param_change:      Types of encode parameter change before encode this
> > + *                     buffer
> > + * @enc_params         Encode parameters changed before encode this buffer
> > + */
> > +struct mtk_video_enc_buf {
> > +       struct vb2_v4l2_buffer  vb;
> > +       struct list_head        list;
> > +
> > +       enum mtk_encode_param param_change;
> > +       struct mtk_enc_params enc_params;
> > +};
> > +
> > +int mtk_venc_unlock(struct mtk_vcodec_ctx *ctx);
> > +int mtk_venc_lock(struct mtk_vcodec_ctx *ctx);
> > +int mtk_vcodec_enc_queue_init(void *priv, struct vb2_queue *src_vq,
> > +                                       struct vb2_queue *dst_vq);
> > +void mtk_vcodec_enc_release(struct mtk_vcodec_ctx *ctx);
> > +int mtk_vcodec_enc_ctrls_setup(struct mtk_vcodec_ctx *ctx);
> > +void mtk_vcodec_enc_ctx_params_setup(struct mtk_vcodec_ctx *ctx);
> > +
> > +#endif /* _MTK_VCODEC_ENC_H_ */
> > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
> > new file mode 100644
> > index 0000000..e7ab14a
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
> > @@ -0,0 +1,476 @@
> > +/*
> > +* Copyright (c) 2015 MediaTek Inc.
> > +* Author: PC Chen <pc.chen@mediatek.com>
> > +*         Tiffany Lin <tiffany.lin@mediatek.com>
> > +*
> > +* This program is free software; you can redistribute it and/or modify
> > +* it under the terms of the GNU General Public License version 2 as
> > +* published by the Free Software Foundation.
> > +*
> > +* This program is distributed in the hope that it will be useful,
> > +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> > +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > +* GNU General Public License for more details.
> > +*/
> > +
> > +#include <linux/interrupt.h>
> > +#include <linux/irq.h>
> > +#include <linux/module.h>
> > +#include <linux/of_device.h>
> > +#include <linux/of.h>
> > +#include <media/v4l2-event.h>
> > +#include <media/v4l2-mem2mem.h>
> > +#include <media/videobuf2-dma-contig.h>
> > +#include <linux/pm_runtime.h>
> > +
> > +#include "mtk_vcodec_drv.h"
> > +#include "mtk_vcodec_enc.h"
> > +#include "mtk_vcodec_pm.h"
> > +#include "mtk_vcodec_intr.h"
> > +#include "mtk_vcodec_util.h"
> > +#include "mtk_vpu.h"
> > +
> > +
> > +/* Wake up context wait_queue */
> > +static void wake_up_ctx(struct mtk_vcodec_ctx *ctx, unsigned int reason)
> > +{
> > +       ctx->int_cond = 1;
> > +       ctx->int_type = reason;
> > +       wake_up_interruptible(&ctx->queue);
> > +}
> > +
> > +static irqreturn_t mtk_vcodec_enc_irq_handler(int irq, void *priv)
> > +{
> > +       struct mtk_vcodec_dev *dev = priv;
> > +       struct mtk_vcodec_ctx *ctx;
> > +       unsigned int irq_status;
> > +
> > +       if (dev->curr_ctx == -1) {
> > +               mtk_v4l2_err("curr_ctx = -1");
> > +               return IRQ_HANDLED;
> > +       }
> > +
> > +       ctx = dev->ctx[dev->curr_ctx];
> > +       if (ctx == NULL) {
> > +               mtk_v4l2_err("curr_ctx==NULL");
> > +               return IRQ_HANDLED;
> > +       }
> > +       mtk_v4l2_debug(1, "idx=%d", ctx->idx);
> > +       irq_status = readl(dev->reg_base[VENC_SYS] +
> > +                               (MTK_VENC_IRQ_STATUS_OFFSET));
> > +       if (irq_status & MTK_VENC_IRQ_STATUS_PAUSE)
> > +               writel((MTK_VENC_IRQ_STATUS_PAUSE),
> > +                      dev->reg_base[VENC_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
> > +
> > +       if (irq_status & MTK_VENC_IRQ_STATUS_SWITCH)
> > +               writel((MTK_VENC_IRQ_STATUS_SWITCH),
> > +                      dev->reg_base[VENC_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
> > +
> > +       if (irq_status & MTK_VENC_IRQ_STATUS_DRAM)
> > +               writel((MTK_VENC_IRQ_STATUS_DRAM),
> > +                      dev->reg_base[VENC_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
> > +
> > +       if (irq_status & MTK_VENC_IRQ_STATUS_SPS)
> > +               writel((MTK_VENC_IRQ_STATUS_SPS),
> > +                      dev->reg_base[VENC_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
> > +
> > +       if (irq_status & MTK_VENC_IRQ_STATUS_PPS)
> > +               writel((MTK_VENC_IRQ_STATUS_PPS),
> > +                      dev->reg_base[VENC_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
> > +
> > +       if (irq_status & MTK_VENC_IRQ_STATUS_FRM)
> > +               writel((MTK_VENC_IRQ_STATUS_FRM),
> > +                      dev->reg_base[VENC_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
> > +
> > +       ctx->irq_status = irq_status;
> > +       wake_up_ctx(ctx, MTK_INST_IRQ_RECEIVED);
> > +       return IRQ_HANDLED;
> > +}
> > +
> > +#if 1 /* VENC_LT */
> > +static irqreturn_t mtk_vcodec_enc_irq_handler2(int irq, void *priv)
> > +{
> > +       struct mtk_vcodec_dev *dev = priv;
> > +       struct mtk_vcodec_ctx *ctx;
> > +       unsigned int irq_status;
> > +
> > +       ctx = dev->ctx[dev->curr_ctx];
> > +       if (ctx == NULL) {
> > +               mtk_v4l2_err("ctx==NULL");
> > +               return IRQ_HANDLED;
> > +       }
> > +       mtk_v4l2_debug(1, "idx=%d", ctx->idx);
> > +       irq_status = readl(dev->reg_base[VENC_LT_SYS] +
> > +                               (MTK_VENC_IRQ_STATUS_OFFSET));
> > +       if (irq_status & MTK_VENC_IRQ_STATUS_PAUSE)
> > +               writel((MTK_VENC_IRQ_STATUS_PAUSE),
> > +                      dev->reg_base[VENC_LT_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
> > +
> > +       if (irq_status & MTK_VENC_IRQ_STATUS_SWITCH)
> > +               writel((MTK_VENC_IRQ_STATUS_SWITCH),
> > +                      dev->reg_base[VENC_LT_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
> > +
> > +       if (irq_status & MTK_VENC_IRQ_STATUS_DRAM)
> > +               writel((MTK_VENC_IRQ_STATUS_DRAM),
> > +                      dev->reg_base[VENC_LT_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
> > +
> > +       if (irq_status & MTK_VENC_IRQ_STATUS_SPS)
> > +               writel((MTK_VENC_IRQ_STATUS_SPS),
> > +                      dev->reg_base[VENC_LT_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
> > +
> > +       if (irq_status & MTK_VENC_IRQ_STATUS_PPS)
> > +               writel((MTK_VENC_IRQ_STATUS_PPS),
> > +                      dev->reg_base[VENC_LT_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
> > +
> > +       if (irq_status & MTK_VENC_IRQ_STATUS_FRM)
> > +               writel((MTK_VENC_IRQ_STATUS_FRM),
> > +                      dev->reg_base[VENC_LT_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
> > +
> > +       ctx->irq_status = irq_status;
> > +       wake_up_ctx(ctx, MTK_INST_IRQ_RECEIVED);
> > +       return IRQ_HANDLED;
> > +}
> > +#endif
> > +
> > +static void mtk_vcodec_enc_reset_handler(void *priv)
> > +{
> > +       int i;
> > +       struct mtk_vcodec_dev *dev = priv;
> > +       struct mtk_vcodec_ctx *ctx;
> > +
> > +       mtk_v4l2_debug(0, "Watchdog timeout!!");
> > +
> > +       mutex_lock(&dev->dev_mutex);
> > +       for(i = 0; i < MTK_VCODEC_MAX_ENCODER_INSTANCES; i++) {
> > +               ctx = dev->ctx[i];
> > +               if (ctx) {
> > +                       ctx->state = MTK_STATE_ABORT;
> > +                       mtk_v4l2_debug(0, "[%d] Change to state MTK_STATE_ERROR", ctx->idx);
> > +               }
> > +
> > +       }
> > +       mutex_unlock(&dev->dev_mutex);
> > +}
> > +
> > +static int fops_vcodec_open(struct file *file)
> > +{
> > +       struct video_device *vfd = video_devdata(file);
> > +       struct mtk_vcodec_dev *dev = video_drvdata(file);
> > +       struct mtk_vcodec_ctx *ctx = NULL;
> > +       int ret = 0;
> > +
> > +       mutex_lock(&dev->dev_mutex);
> > +
> > +       ctx = devm_kzalloc(&dev->plat_dev->dev, sizeof(*ctx), GFP_KERNEL);
> > +       if (!ctx) {
> > +               ret = -ENOMEM;
> > +               goto err_alloc;
> > +       }
> > +
> > +       if (dev->num_instances >= MTK_VCODEC_MAX_ENCODER_INSTANCES) {
> > +               mtk_v4l2_err("Too many open contexts\n");
> > +               ret = -EBUSY;
> > +               goto err_no_ctx;
> > +       }
> > +
> > +       ctx->idx = ffz(dev->instance_mask[0]);
> > +       v4l2_fh_init(&ctx->fh, video_devdata(file));
> > +       file->private_data = &ctx->fh;
> > +       v4l2_fh_add(&ctx->fh);
> > +       ctx->dev = dev;
> > +
> > +       if (vfd == dev->vfd_enc) {
> > +               ctx->type = MTK_INST_ENCODER;
> > +
> > +               ret = mtk_vcodec_enc_ctrls_setup(ctx);
> > +               if (ret) {
> > +                       mtk_v4l2_err("Failed to setup controls() (%d)\n",
> > +                                      ret);
> > +                       goto err_ctrls_setup;
> > +               }
> > +               ctx->m2m_ctx = v4l2_m2m_ctx_init(dev->m2m_dev_enc, ctx,
> > +                                                &mtk_vcodec_enc_queue_init);
> > +               if (IS_ERR(ctx->m2m_ctx)) {
> > +                       ret = PTR_ERR(ctx->m2m_ctx);
> > +                       mtk_v4l2_err("Failed to v4l2_m2m_ctx_init() (%d)\n",
> > +                                      ret);
> > +                       goto err_ctx_init;
> > +               }
> > +               mtk_vcodec_enc_ctx_params_setup(ctx);
> > +       } else {
> > +               mtk_v4l2_err("Invalid vfd !\n");
> > +               ret = -ENOENT;
> > +               goto err_ctx_init;
> > +       }
> > +
> > +       init_waitqueue_head(&ctx->queue);
> > +       dev->num_instances++;
> > +
> > +       if (dev->num_instances == 1) {
> > +               ret = vpu_load_firmware(dev->vpu_plat_dev);
> > +               if (ret < 0) {
> > +                               mtk_v4l2_err("vpu_load_firmware failed!\n");
> > +                       goto err_load_fw;
> > +               }
> > +
> > +               dev->enc_capability =
> > +                       vpu_get_venc_hw_capa(dev->vpu_plat_dev);
> > +               mtk_v4l2_debug(0, "encoder capability %x", dev->enc_capability);
> > +       }
> > +
> > +       mtk_v4l2_debug(2, "Create instance [%d]@%p m2m_ctx=%p type=%d\n",
> > +                        ctx->idx, ctx, ctx->m2m_ctx, ctx->type);
> > +       set_bit(ctx->idx, &dev->instance_mask[0]);
> > +       dev->ctx[ctx->idx] = ctx;
> > +
> > +       mutex_unlock(&dev->dev_mutex);
> > +       mtk_v4l2_debug(0, "%s encoder [%d]", dev_name(&dev->plat_dev->dev), ctx->idx);
> > +       return ret;
> > +
> > +       /* Deinit when failure occurred */
> > +err_load_fw:
> > +       v4l2_m2m_ctx_release(ctx->m2m_ctx);
> > +       v4l2_fh_del(&ctx->fh);
> > +       v4l2_fh_exit(&ctx->fh);
> > +       dev->num_instances--;
> > +err_ctx_init:
> > +err_ctrls_setup:
> > +       v4l2_ctrl_handler_free(&ctx->ctrl_hdl);
> > +err_no_ctx:
> > +       devm_kfree(&dev->plat_dev->dev, ctx);
> > +err_alloc:
> > +       mutex_unlock(&dev->dev_mutex);
> > +       return ret;
> > +}
> > +
> > +static int fops_vcodec_release(struct file *file)
> > +{
> > +       struct mtk_vcodec_dev *dev = video_drvdata(file);
> > +       struct mtk_vcodec_ctx *ctx = fh_to_ctx(file->private_data);
> > +
> > +       mtk_v4l2_debug(0, "[%d] encoder\n", ctx->idx);
> > +       mutex_lock(&dev->dev_mutex);
> > +
> > +       mtk_vcodec_enc_release(ctx);
> > +       v4l2_fh_del(&ctx->fh);
> > +       v4l2_fh_exit(&ctx->fh);
> > +       v4l2_ctrl_handler_free(&ctx->ctrl_hdl);
> > +       v4l2_m2m_ctx_release(ctx->m2m_ctx);
> > +
> > +       dev->ctx[ctx->idx] = NULL;
> > +       dev->num_instances--;
> > +       clear_bit(ctx->idx, &dev->instance_mask[0]);
> > +       devm_kfree(&dev->plat_dev->dev, ctx);
> > +       mutex_unlock(&dev->dev_mutex);
> > +       return 0;
> > +}
> > +
> > +static const struct v4l2_file_operations mtk_vcodec_fops = {
> > +       .owner                          = THIS_MODULE,
> > +       .open                           = fops_vcodec_open,
> > +       .release                        = fops_vcodec_release,
> > +       .poll                           = v4l2_m2m_fop_poll,
> > +       .unlocked_ioctl                 = video_ioctl2,
> > +       .mmap                           = v4l2_m2m_fop_mmap,
> > +};
> > +
> > +static int mtk_vcodec_probe(struct platform_device *pdev)
> > +{
> > +       struct mtk_vcodec_dev *dev;
> > +       struct video_device *vfd_enc;
> > +       struct resource *res;
> > +       int i, j, ret;
> > +
> > +       dev = devm_kzalloc(&pdev->dev, sizeof(*dev), GFP_KERNEL);
> > +       if (!dev)
> > +               return -ENOMEM;
> > +
> > +       dev->plat_dev = pdev;
> > +
> > +       dev->vpu_plat_dev = vpu_get_plat_device(dev->plat_dev);
> > +       if (dev->vpu_plat_dev == NULL) {
> > +               mtk_v4l2_err("[VPU] vpu device in not ready\n");
> > +               return -EPROBE_DEFER;
> > +       }
> > +
> > +       vpu_wdt_reg_handler(dev->vpu_plat_dev, mtk_vcodec_enc_reset_handler, dev,
> > +                           VPU_RST_ENC);
> > +
> > +       ret = mtk_vcodec_init_enc_pm(dev);
> > +       if (ret < 0) {
> > +               dev_err(&pdev->dev, "Failed to get mt vcodec clock source!\n");
> > +               return ret;
> > +       }
> > +
> > +       for (i = VENC_SYS, j = 0; i < NUM_MAX_VCODEC_REG_BASE; i++, j++) {
> > +               res = platform_get_resource(pdev, IORESOURCE_MEM, j);
> > +               if (res == NULL) {
> > +                       dev_err(&pdev->dev, "get memory resource failed.\n");
> > +                       ret = -ENXIO;
> > +                       goto err_res;
> > +               }
> > +               dev->reg_base[i] = devm_ioremap_resource(&pdev->dev, res);
> > +               if (IS_ERR(dev->reg_base[i])) {
> > +                       dev_err(&pdev->dev,
> > +                               "devm_ioremap_resource %d failed.\n", i);
> > +                       ret = PTR_ERR(dev->reg_base);
> > +                       goto err_res;
> > +               }
> > +               mtk_v4l2_debug(2, "reg[%d] base=0x%p\n", i, dev->reg_base[i]);
> > +       }
> > +
> > +       res = platform_get_resource(pdev, IORESOURCE_IRQ, 0);
> > +       if (res == NULL) {
> > +               dev_err(&pdev->dev, "failed to get irq resource\n");
> > +               ret = -ENOENT;
> > +               goto err_res;
> > +       }
> > +
> > +       dev->enc_irq = platform_get_irq(pdev, 0);
> > +       ret = devm_request_irq(&pdev->dev, dev->enc_irq,
> > +                              mtk_vcodec_enc_irq_handler,
> > +                              0, pdev->name, dev);
> > +       if (ret) {
> > +               dev_err(&pdev->dev, "Failed to install dev->enc_irq %d (%d)\n",
> > +                       dev->enc_irq,
> > +                       ret);
> > +               ret = -EINVAL;
> > +               goto err_res;
> > +       }
> > +
> > +       dev->enc_lt_irq = platform_get_irq(pdev, 1);
> > +       ret = devm_request_irq(&pdev->dev,
> > +                              dev->enc_lt_irq, mtk_vcodec_enc_irq_handler2,
> > +                              0, pdev->name, dev);
> > +       if (ret) {
> > +               dev_err(&pdev->dev,
> > +                       "Failed to install dev->enc_lt_irq %d (%d)\n",
> > +                       dev->enc_lt_irq, ret);
> > +               ret = -EINVAL;
> > +               goto err_res;
> > +       }
> > +
> > +       disable_irq(dev->enc_irq);
> > +       disable_irq(dev->enc_lt_irq); /* VENC_LT */
> > +       mutex_init(&dev->enc_mutex);
> > +       mutex_init(&dev->dev_mutex);
> > +
> > +       snprintf(dev->v4l2_dev.name, sizeof(dev->v4l2_dev.name), "%s",
> > +                "[MTK_V4L2_VENC]");
> > +
> > +       ret = v4l2_device_register(&pdev->dev, &dev->v4l2_dev);
> > +       if (ret) {
> > +               mtk_v4l2_err("v4l2_device_register err=%d\n", ret);
> > +               return ret;
> > +       }
> > +
> > +       init_waitqueue_head(&dev->queue);
> > +
> > +       /* allocate video device for encoder and register it */
> > +       vfd_enc = video_device_alloc();
> > +       if (!vfd_enc) {
> > +               mtk_v4l2_err("Failed to allocate video device\n");
> > +               ret = -ENOMEM;
> > +               goto err_enc_alloc;
> > +       }
> > +       vfd_enc->fops           = &mtk_vcodec_fops;
> > +       vfd_enc->ioctl_ops      = &mtk_venc_ioctl_ops;
> > +       vfd_enc->release        = video_device_release;
> > +       vfd_enc->lock           = &dev->dev_mutex;
> > +       vfd_enc->v4l2_dev       = &dev->v4l2_dev;
> > +       vfd_enc->vfl_dir        = VFL_DIR_M2M;
> > +
> > +       snprintf(vfd_enc->name, sizeof(vfd_enc->name), "%s",
> > +                MTK_VCODEC_ENC_NAME);
> > +       video_set_drvdata(vfd_enc, dev);
> > +       dev->vfd_enc = vfd_enc;
> > +       platform_set_drvdata(pdev, dev);
> > +       ret = video_register_device(vfd_enc, VFL_TYPE_GRABBER, 1);
> > +       if (ret) {
> > +               mtk_v4l2_err("Failed to register video device\n");
> > +               goto err_enc_reg;
> > +       }
> > +       mtk_v4l2_debug(0, "encoder registered as /dev/video%d\n",
> > +                        vfd_enc->num);
> > +
> > +       dev->alloc_ctx = vb2_dma_contig_init_ctx(&pdev->dev);
> > +       if (IS_ERR(dev->alloc_ctx)) {
> > +               mtk_v4l2_err("Failed to alloc vb2 dma context 0\n");
> > +               ret = PTR_ERR(dev->alloc_ctx);
> > +               goto err_vb2_ctx_init;
> > +       }
> > +
> > +       dev->m2m_dev_enc = v4l2_m2m_init(&mtk_venc_m2m_ops);
> > +       if (IS_ERR(dev->m2m_dev_enc)) {
> > +               mtk_v4l2_err("Failed to init mem2mem enc device\n");
> > +               ret = PTR_ERR(dev->m2m_dev_enc);
> > +               goto err_enc_mem_init;
> > +       }
> > +
> > +       dev->encode_workqueue =
> > +                       alloc_ordered_workqueue(MTK_VCODEC_ENC_NAME, WQ_MEM_RECLAIM | WQ_FREEZABLE);
> > +       if (!dev->encode_workqueue) {
> > +               mtk_v4l2_err("Failed to create encode workqueue\n");
> > +               ret = -EINVAL;
> > +               goto err_event_workq;
> > +       }
> > +
> > +       return 0;
> > +
> > +err_event_workq:
> > +       v4l2_m2m_release(dev->m2m_dev_enc);
> > +err_enc_mem_init:
> > +       vb2_dma_contig_cleanup_ctx(dev->alloc_ctx);
> > +err_vb2_ctx_init:
> > +       video_unregister_device(vfd_enc);
> > +err_enc_reg:
> > +       video_device_release(vfd_enc);
> > +err_enc_alloc:
> > +       v4l2_device_unregister(&dev->v4l2_dev);
> > +err_res:
> > +       mtk_vcodec_release_enc_pm(dev);
> > +       return ret;
> > +}
> > +
> > +static const struct of_device_id mtk_vcodec_match[] = {
> > +       {.compatible = "mediatek,mt8173-vcodec-enc",},
> > +       {},
> > +};
> > +MODULE_DEVICE_TABLE(of, mtk_vcodec_match);
> > +
> > +static int mtk_vcodec_remove(struct platform_device *pdev)
> > +{
> > +       struct mtk_vcodec_dev *dev = platform_get_drvdata(pdev);
> > +
> > +       mtk_v4l2_debug_enter();
> > +       flush_workqueue(dev->encode_workqueue);
> > +       destroy_workqueue(dev->encode_workqueue);
> > +       if (dev->m2m_dev_enc)
> > +               v4l2_m2m_release(dev->m2m_dev_enc);
> > +       if (dev->alloc_ctx)
> > +               vb2_dma_contig_cleanup_ctx(dev->alloc_ctx);
> > +
> > +       if (dev->vfd_enc) {
> > +               video_unregister_device(dev->vfd_enc);
> > +               video_device_release(dev->vfd_enc);
> > +       }
> > +       v4l2_device_unregister(&dev->v4l2_dev);
> > +       mtk_vcodec_release_enc_pm(dev);
> > +       return 0;
> > +}
> > +
> > +static struct platform_driver mtk_vcodec_driver = {
> > +       .probe  = mtk_vcodec_probe,
> > +       .remove = mtk_vcodec_remove,
> > +       .driver = {
> > +               .name   = MTK_VCODEC_ENC_NAME,
> > +               .owner  = THIS_MODULE,
> > +               .of_match_table = mtk_vcodec_match,
> > +       },
> > +};
> > +
> > +module_platform_driver(mtk_vcodec_driver);
> > +
> > +
> > +MODULE_LICENSE("GPL v2");
> > +MODULE_DESCRIPTION("Mediatek video codec V4L2 driver");
> > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
> > new file mode 100644
> > index 0000000..518fba7
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
> > @@ -0,0 +1,132 @@
> > +/*
> > +* Copyright (c) 2015 MediaTek Inc.
> > +* Author: Tiffany Lin <tiffany.lin@mediatek.com>
> > +*
> > +* This program is free software; you can redistribute it and/or modify
> > +* it under the terms of the GNU General Public License version 2 as
> > +* published by the Free Software Foundation.
> > +*
> > +* This program is distributed in the hope that it will be useful,
> > +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> > +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > +* GNU General Public License for more details.
> > +*/
> > +
> > +#include <linux/clk.h>
> > +#include <linux/of_address.h>
> > +#include <linux/of_platform.h>
> > +#include <linux/pm_runtime.h>
> > +#include <soc/mediatek/smi.h>
> > +
> > +#include "mtk_vcodec_pm.h"
> > +#include "mtk_vcodec_util.h"
> > +#include "mtk_vpu.h"
> > +
> > +
> > +int mtk_vcodec_init_enc_pm(struct mtk_vcodec_dev *mtkdev)
> > +{
> > +       struct device_node *node;
> > +       struct platform_device *pdev;
> > +       struct device *dev;
> > +       struct mtk_vcodec_pm *pm;
> > +       int ret = 0;
> > +
> > +       pdev = mtkdev->plat_dev;
> > +       pm = &mtkdev->pm;
> > +       memset(pm, 0, sizeof(struct mtk_vcodec_pm));
> > +       pm->mtkdev = mtkdev;
> > +       dev = &pdev->dev;
> > +
> > +       node = of_parse_phandle(dev->of_node, "mediatek,larb", 0);
> > +       if (!node)
> > +               return -1;
> > +       pdev = of_find_device_by_node(node);
> > +       if (WARN_ON(!pdev)) {
> > +               of_node_put(node);
> > +               return -1;
> > +       }
> > +       pm->larbvenc = &pdev->dev;
> > +
> > +       node = of_parse_phandle(dev->of_node, "mediatek,larb", 1);
> > +       if (!node)
> > +               return -1;
> > +
> > +       pdev = of_find_device_by_node(node);
> > +       if (WARN_ON(!pdev)) {
> > +               of_node_put(node);
> > +               return -EINVAL;
> > +       }
> > +       pm->larbvenclt = &pdev->dev;
> > +
> > +       pdev = mtkdev->plat_dev;
> > +       pm->dev = &pdev->dev;
> > +
> > +       pm->vencpll_d2 = devm_clk_get(&pdev->dev, "vencpll_d2");
> > +       if (pm->vencpll_d2 == NULL) {
> > +               mtk_v4l2_err("devm_clk_get vencpll_d2 fail");
> > +               ret = -1;
> > +       }
> > +
> > +       pm->venc_sel = devm_clk_get(&pdev->dev, "venc_sel");
> > +       if (pm->venc_sel == NULL) {
> > +               mtk_v4l2_err("devm_clk_get venc_sel fail");
> > +               ret = -1;
> > +       }
> > +
> > +       pm->univpll1_d2 = devm_clk_get(&pdev->dev, "univpll1_d2");
> > +       if (pm->univpll1_d2 == NULL) {
> > +               mtk_v4l2_err("devm_clk_get univpll1_d2 fail");
> > +               ret = -1;
> > +       }
> > +
> > +       pm->venc_lt_sel = devm_clk_get(&pdev->dev, "venc_lt_sel");
> > +       if (pm->venc_lt_sel == NULL) {
> > +               mtk_v4l2_err("devm_clk_get venc_lt_sel fail");
> > +               ret = -1;
> > +       }
> > +
> > +       return ret;
> > +}
> > +
> > +void mtk_vcodec_release_enc_pm(struct mtk_vcodec_dev *mtkdev)
> > +{
> > +}
> > +
> > +
> > +void mtk_vcodec_enc_clock_on(struct mtk_vcodec_pm *pm)
> > +{
> > +       int ret;
> > +
> > +       ret = clk_prepare_enable(pm->venc_sel);
> > +       if (ret)
> > +               mtk_v4l2_err("venc_sel fail %d", ret);
> > +
> > +       ret = clk_set_parent(pm->venc_sel, pm->vencpll_d2);
> > +       if (ret)
> > +               mtk_v4l2_err("clk_set_parent fail %d", ret);
> > +
> > +       ret = clk_prepare_enable(pm->venc_lt_sel);
> > +       if (ret)
> > +               mtk_v4l2_err("venc_lt_sel fail %d", ret);
> > +
> > +       ret = clk_set_parent(pm->venc_lt_sel, pm->univpll1_d2);
> > +       if (ret)
> > +               mtk_v4l2_err("clk_set_parent fail %d", ret);
> > +
> > +       ret = mtk_smi_larb_get(pm->larbvenc);
> > +       if (ret)
> > +               mtk_v4l2_err("mtk_smi_larb_get larb3 fail %d\n", ret);
> > +
> > +       ret = mtk_smi_larb_get(pm->larbvenclt);
> > +       if (ret)
> > +               mtk_v4l2_err("mtk_smi_larb_get larb4 fail %d\n", ret);
> > +
> > +}
> > +
> > +void mtk_vcodec_enc_clock_off(struct mtk_vcodec_pm *pm)
> > +{
> > +       mtk_smi_larb_put(pm->larbvenc);
> > +       mtk_smi_larb_put(pm->larbvenclt);
> > +       clk_disable_unprepare(pm->venc_lt_sel);
> > +       clk_disable_unprepare(pm->venc_sel);
> > +}
> > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.c
> > new file mode 100644
> > index 0000000..919b949
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.c
> > @@ -0,0 +1,102 @@
> > +/*
> > +* Copyright (c) 2015 MediaTek Inc.
> > +* Author: Tiffany Lin <tiffany.lin@mediatek.com>
> > +*
> > +* This program is free software; you can redistribute it and/or modify
> > +* it under the terms of the GNU General Public License version 2 as
> > +* published by the Free Software Foundation.
> > +*
> > +* This program is distributed in the hope that it will be useful,
> > +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> > +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > +* GNU General Public License for more details.
> > +*/
> > +
> > +#include <linux/errno.h>
> > +#include <linux/wait.h>
> > +
> > +#include "mtk_vcodec_drv.h"
> > +#include "mtk_vcodec_intr.h"
> > +#include "mtk_vcodec_util.h"
> > +
> > +void mtk_vcodec_clean_dev_int_flags(void *data)
> > +{
> > +       struct mtk_vcodec_dev *dev = (struct mtk_vcodec_dev *)data;
> > +
> > +       dev->int_cond = 0;
> > +       dev->int_type = 0;
> > +}
> > +
> > +int mtk_vcodec_wait_for_done_ctx(void *data, int command,
> > +                                unsigned int timeout_ms, int interrupt)
> > +{
> > +       wait_queue_head_t *waitqueue;
> > +       long timeout_jiff, ret;
> > +       int status = 0;
> > +       struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)data;
> > +
> > +       waitqueue = (wait_queue_head_t *)&ctx->queue;
> > +       timeout_jiff = msecs_to_jiffies(timeout_ms);
> > +       if (interrupt) {
> > +               ret = wait_event_interruptible_timeout(*waitqueue,
> > +                               (ctx->int_cond &&
> > +                               (ctx->int_type == command)),
> > +                               timeout_jiff);
> > +       } else {
> > +               ret = wait_event_timeout(*waitqueue,
> > +                               (ctx->int_cond &&
> > +                               (ctx->int_type == command)),
> > +                                timeout_jiff);
> > +       }
> > +       if (0 == ret) {
> > +               status = -1;    /* timeout */
> > +               mtk_v4l2_err("[%d] cmd=%d, ctx->type=%d, wait_event_interruptible_timeout time=%ums out %d %d!",
> > +                               ctx->idx, ctx->type, command, timeout_ms,
> > +                               ctx->int_cond, ctx->int_type);
> > +       } else if (-ERESTARTSYS == ret) {
> > +               mtk_v4l2_err("[%d] cmd=%d, ctx->type=%d, wait_event_interruptible_timeout interrupted by a signal %d %d",
> > +                               ctx->idx, ctx->type, command, ctx->int_cond,
> > +                               ctx->int_type);
> > +               status = -1;
> > +       }
> > +
> > +       ctx->int_cond = 0;
> > +       ctx->int_type = 0;
> > +
> > +       return status;
> > +}
> > +
> > +int mtk_vcodec_wait_for_done_dev(void *data, int command,
> > +                                unsigned int timeout, int interrupt)
> > +{
> > +       wait_queue_head_t *waitqueue;
> > +       long timeout_jiff, ret;
> > +       int status = 0;
> > +       struct mtk_vcodec_dev *dev = (struct mtk_vcodec_dev *)data;
> > +
> > +       waitqueue = (wait_queue_head_t *)&dev->queue;
> > +       timeout_jiff = msecs_to_jiffies(timeout);
> > +       if (interrupt) {
> > +               ret = wait_event_interruptible_timeout(*waitqueue,
> > +                               (dev->int_cond &&
> > +                               (dev->int_type == command)),
> > +                               timeout_jiff);
> > +       } else {
> > +               ret = wait_event_timeout(*waitqueue,
> > +                               (dev->int_cond &&
> > +                               (dev->int_type == command)),
> > +                               timeout_jiff);
> > +       }
> > +       if (0 == ret) {
> > +               status = -1;    /* timeout */
> > +               mtk_v4l2_err("wait_event_interruptible_timeout time=%lu out %d %d!",
> > +                               timeout_jiff, dev->int_cond, dev->int_type);
> > +       } else if (-ERESTARTSYS == ret) {
> > +               mtk_v4l2_err("wait_event_interruptible_timeout interrupted by a signal %d %d",
> > +                               dev->int_cond, dev->int_type);
> > +               status = -1;
> > +       }
> > +       dev->int_cond = 0;
> > +       dev->int_type = 0;
> > +       return status;
> > +}
> > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.h
> > new file mode 100644
> > index 0000000..e9b7f94
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.h
> > @@ -0,0 +1,29 @@
> > +/*
> > +* Copyright (c) 2015 MediaTek Inc.
> > +* Author: Tiffany Lin <tiffany.lin@mediatek.com>
> > +*
> > +* This program is free software; you can redistribute it and/or modify
> > +* it under the terms of the GNU General Public License version 2 as
> > +* published by the Free Software Foundation.
> > +*
> > +* This program is distributed in the hope that it will be useful,
> > +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> > +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > +* GNU General Public License for more details.
> > +*/
> > +
> > +#ifndef _MTK_VCODEC_INTR_H_
> > +#define _MTK_VCODEC_INTR_H_
> > +
> > +#define MTK_INST_IRQ_RECEIVED          0x1
> > +#define MTK_INST_WORK_THREAD_ABORT_DONE        0x2
> > +
> > +/* timeout is ms */
> > +int mtk_vcodec_wait_for_done_ctx(void *data, int command, unsigned int timeout,
> > +                                int interrupt);
> > +int mtk_vcodec_wait_for_done_dev(void *data, int command, unsigned int timeout,
> > +                                int interrupt);
> > +
> > +void mtk_vcodec_clean_dev_int_flags(void *data);
> > +
> > +#endif /* _MTK_VCODEC_INTR_H_ */
> > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_pm.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_pm.h
> > new file mode 100644
> > index 0000000..fdadec9
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_pm.h
> > @@ -0,0 +1,26 @@
> > +/*
> > +* Copyright (c) 2015 MediaTek Inc.
> > +* Author: Tiffany Lin <tiffany.lin@mediatek.com>
> > +*
> > +* This program is free software; you can redistribute it and/or modify
> > +* it under the terms of the GNU General Public License version 2 as
> > +* published by the Free Software Foundation.
> > +*
> > +* This program is distributed in the hope that it will be useful,
> > +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> > +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > +* GNU General Public License for more details.
> > +*/
> > +
> > +#ifndef _MTK_VCODEC_PM_H_
> > +#define _MTK_VCODEC_PM_H_
> > +
> > +#include "mtk_vcodec_drv.h"
> > +
> > +int mtk_vcodec_init_enc_pm(struct mtk_vcodec_dev *dev);
> > +void mtk_vcodec_release_enc_pm(struct mtk_vcodec_dev *dev);
> > +
> > +void mtk_vcodec_enc_clock_on(struct mtk_vcodec_pm *pm);
> > +void mtk_vcodec_enc_clock_off(struct mtk_vcodec_pm *pm);
> > +
> > +#endif /* _MTK_VCODEC_PM_H_ */
> > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c
> > new file mode 100644
> > index 0000000..3fede8d
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c
> > @@ -0,0 +1,106 @@
> > +/*
> > +* Copyright (c) 2015 MediaTek Inc.
> > +* Author: PC Chen <pc.chen@mediatek.com>
> > +*         Tiffany Lin <tiffany.lin@mediatek.com>
> > +*
> > +* This program is free software; you can redistribute it and/or modify
> > +* it under the terms of the GNU General Public License version 2 as
> > +* published by the Free Software Foundation.
> > +*
> > +* This program is distributed in the hope that it will be useful,
> > +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> > +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > +* GNU General Public License for more details.
> > +*/
> > +
> > +#include <linux/module.h>
> > +
> > +#include "mtk_vcodec_drv.h"
> > +#include "mtk_vcodec_util.h"
> > +#include "mtk_vpu.h"
> > +
> > +bool mtk_vcodec_dbg = false;
> > +int mtk_v4l2_dbg_level = 0;
> > +
> > +module_param(mtk_v4l2_dbg_level, int, S_IRUGO | S_IWUSR);
> > +module_param(mtk_vcodec_dbg, bool, S_IRUGO | S_IWUSR);
> > +
> > +void __iomem *mtk_vcodec_get_reg_addr(void *data, unsigned int reg_idx)
> > +{
> > +       struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)data;
> > +
> > +       if (!data || reg_idx >= NUM_MAX_VCODEC_REG_BASE) {
> > +               mtk_v4l2_err("Invalid arguments");
> > +               return NULL;
> > +       }
> > +       return ctx->dev->reg_base[reg_idx];
> > +}
> > +
> > +int mtk_vcodec_mem_alloc(void *data, struct mtk_vcodec_mem *mem)
> > +{
> > +       unsigned long size = mem->size;
> > +       struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)data;
> > +       struct device *dev = &ctx->dev->plat_dev->dev;
> > +
> > +       mem->va = dma_alloc_coherent(dev, size, &mem->dma_addr, GFP_KERNEL);
> > +
> > +       if (!mem->va) {
> > +               mtk_v4l2_err("%s dma_alloc size=%ld failed!", dev_name(dev),
> > +                              size);
> > +               return -ENOMEM;
> > +       }
> > +
> > +       memset(mem->va, 0, size);
> > +
> > +       mtk_v4l2_debug(3, "[%d]  - va      = %p", ctx->idx, mem->va);
> > +       mtk_v4l2_debug(3, "[%d]  - dma     = 0x%lx", ctx->idx,
> > +                        (unsigned long)mem->dma_addr);
> > +       mtk_v4l2_debug(3, "[%d]    size = 0x%lx", ctx->idx, size);
> > +
> > +       return 0;
> > +}
> > +
> > +void mtk_vcodec_mem_free(void *data, struct mtk_vcodec_mem *mem)
> > +{
> > +       unsigned long size = mem->size;
> > +       struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)data;
> > +       struct device *dev = &ctx->dev->plat_dev->dev;
> > +
> > +       dma_free_coherent(dev, size, mem->va, mem->dma_addr);
> > +       mem->va = NULL;
> > +
> > +       mtk_v4l2_debug(3, "[%d]  - va      = %p", ctx->idx, mem->va);
> > +       mtk_v4l2_debug(3, "[%d]  - dma     = 0x%lx", ctx->idx,
> > +                        (unsigned long)mem->dma_addr);
> > +       mtk_v4l2_debug(3, "[%d]    size = 0x%lx", ctx->idx, size);
> > +}
> > +
> > +int mtk_vcodec_get_ctx_id(void *data)
> > +{
> > +       struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)data;
> > +
> > +       if (!ctx)
> > +               return -1;
> > +
> > +       return ctx->idx;
> > +}
> > +
> > +struct platform_device *mtk_vcodec_get_plat_dev(void *data)
> > +{
> > +       struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)data;
> > +
> > +       if (!ctx)
> > +               return NULL;
> > +
> > +       return vpu_get_plat_device(ctx->dev->plat_dev);
> > +}
> > +
> > +void mtk_vcodec_fmt2str(u32 fmt, char *str)
> > +{
> > +       char a = fmt & 0xFF;
> > +       char b = (fmt >> 8) & 0xFF;
> > +       char c = (fmt >> 16) & 0xFF;
> > +       char d = (fmt >> 24) & 0xFF;
> > +
> > +       sprintf(str, "%c%c%c%c", a, b, c, d);
> > +}
> > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
> > new file mode 100644
> > index 0000000..47016ae
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
> > @@ -0,0 +1,85 @@
> > +/*
> > +* Copyright (c) 2015 MediaTek Inc.
> > +* Author: PC Chen <pc.chen@mediatek.com>
> > +*         Tiffany Lin <tiffany.lin@mediatek.com>
> > +*
> > +* This program is free software; you can redistribute it and/or modify
> > +* it under the terms of the GNU General Public License version 2 as
> > +* published by the Free Software Foundation.
> > +*
> > +* This program is distributed in the hope that it will be useful,
> > +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> > +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > +* GNU General Public License for more details.
> > +*/
> > +
> > +#ifndef _MTK_VCODEC_UTIL_H_
> > +#define _MTK_VCODEC_UTIL_H_
> > +
> > +#include <linux/types.h>
> > +#include <linux/dma-direction.h>
> > +
> > +struct mtk_vcodec_mem {
> > +       size_t size;
> > +       void *va;
> > +       dma_addr_t dma_addr;
> > +};
> > +
> > +extern int mtk_v4l2_dbg_level;
> > +extern bool mtk_vcodec_dbg;
> > +
> > +#define DEBUG  1
> > +
> > +#if defined(DEBUG)
> > +
> > +#define mtk_v4l2_debug(level, fmt, args...)                             \
> > +       do {                                                             \
> > +               if (mtk_v4l2_dbg_level >= level)                         \
> > +                       pr_info("[MTK_V4L2] level=%d %s(),%d: " fmt "\n",\
> > +                               level, __func__, __LINE__, ##args);      \
> > +       } while (0)
> > +
> > +#define mtk_v4l2_err(fmt, args...)                \
> > +       pr_err("[MTK_V4L2][ERROR] %s:%d: " fmt "\n", __func__, __LINE__, \
> > +              ##args)
> > +
> > +
> > +#define mtk_v4l2_debug_enter()  mtk_v4l2_debug(3, "+\n")
> > +#define mtk_v4l2_debug_leave()  mtk_v4l2_debug(3, "-\n")
> > +
> > +#define mtk_vcodec_debug(h, fmt, args...)                              \
> > +       do {                                                            \
> > +               if (mtk_vcodec_dbg)                                     \
> > +                       pr_info("[MTK_VCODEC][%d]: %s() " fmt "\n",     \
> > +                               ((struct mtk_vcodec_ctx *)h->ctx)->idx, \
> > +                               __func__, ##args);                      \
> > +       } while (0)
> > +
> > +#define mtk_vcodec_err(h, fmt, args...)                                        \
> > +       pr_err("[MTK_VCODEC][ERROR][%d]: %s() " fmt "\n",               \
> > +              ((struct mtk_vcodec_ctx *)h->ctx)->idx, __func__, ##args)
> > +
> > +#define mtk_vcodec_debug_enter(h)  mtk_vcodec_debug(h, "+\n")
> > +#define mtk_vcodec_debug_leave(h)  mtk_vcodec_debug(h, "-\n")
> > +
> > +#else
> > +
> > +#define mtk_v4l2_debug(level, fmt, args...)
> > +#define mtk_v4l2_err(fmt, args...)
> > +#define mtk_v4l2_debug_enter()
> > +#define mtk_v4l2_debug_leave()
> > +
> > +#define mtk_vcodec_debug(h, fmt, args...)
> > +#define mtk_vcodec_err(h, fmt, args...)
> > +#define mtk_vcodec_debug_enter(h)
> > +#define mtk_vcodec_debug_leave(h)
> > +
> > +#endif
> > +
> > +void __iomem *mtk_vcodec_get_reg_addr(void *data, unsigned int reg_idx);
> > +int mtk_vcodec_mem_alloc(void *data, struct mtk_vcodec_mem *mem);
> > +void mtk_vcodec_mem_free(void *data, struct mtk_vcodec_mem *mem);
> > +int mtk_vcodec_get_ctx_id(void *data);
> > +struct platform_device *mtk_vcodec_get_plat_dev(void *data);
> > +void mtk_vcodec_fmt2str(u32 fmt, char *str);
> > +#endif /* _MTK_VCODEC_UTIL_H_ */
> > diff --git a/drivers/media/platform/mtk-vcodec/venc_drv_base.h b/drivers/media/platform/mtk-vcodec/venc_drv_base.h
> > new file mode 100644
> > index 0000000..ed9cbf0
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/venc_drv_base.h
> > @@ -0,0 +1,62 @@
> > +/*
> > + * Copyright (c) 2015 MediaTek Inc.
> > + * Author: Daniel Hsiao <daniel.hsiao@mediatek.com>
> > + *         Jungchang Tsao <jungchang.tsao@mediatek.com>
> > + *
> > + * This program is free software; you can redistribute it and/or
> > + * modify
> > + * it under the terms of the GNU General Public License version 2 as
> > + * published by the Free Software Foundation.
> > + *
> > + * This program is distributed in the hope that it will be useful,
> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > + * GNU General Public License for more details.
> > + */
> > +
> > +#ifndef _VENC_DRV_BASE_
> > +#define _VENC_DRV_BASE_
> > +
> > +#include "mtk_vcodec_drv.h"
> > +
> > +#include "venc_drv_if.h"
> > +
> > +struct venc_common_if {
> > +       /**
> > +        * (*init)() - initialize driver
> > +        * @ctx:        [in] mtk v4l2 context
> > +        * @handle: [out] driver handle
> > +        */
> > +       int (*init)(struct mtk_vcodec_ctx *ctx, unsigned long *handle);
> > +
> > +       /**
> > +        * (*encode)() - trigger encode
> > +        * @handle: [in] driver handle
> > +        * @opt: [in] encode option
> > +        * @frm_buf: [in] frame buffer to store input frame
> > +        * @bs_buf: [in] bitstream buffer to store output bitstream
> > +        * @result: [out] encode result
> > +        */
> > +       int (*encode)(unsigned long handle, enum venc_start_opt opt,
> > +                     struct venc_frm_buf *frm_buf,
> > +                     struct mtk_vcodec_mem *bs_buf,
> > +                     struct venc_done_result *result);
> > +
> > +       /**
> > +        * (*set_param)() - set driver's parameter
> > +        * @handle: [in] driver handle
> > +        * @type: [in] parameter type
> > +        * @in: [in] buffer to store the parameter
> > +        */
> > +       int (*set_param)(unsigned long handle, enum venc_set_param_type type,
> > +                        void *in);
> > +
> > +       /**
> > +        * (*deinit)() - deinitialize driver.
> > +        * @handle: [in] driver handle
> > +        */
> > +       int (*deinit)(unsigned long handle);
> > +};
> > +
> > +
> > +#endif
> > diff --git a/drivers/media/platform/mtk-vcodec/venc_drv_if.c b/drivers/media/platform/mtk-vcodec/venc_drv_if.c
> > new file mode 100644
> > index 0000000..daa8e93
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/venc_drv_if.c
> > @@ -0,0 +1,100 @@
> > +/*
> > + * Copyright (c) 2015 MediaTek Inc.
> > + * Author: Daniel Hsiao <daniel.hsiao@mediatek.com>
> > + *         Jungchang Tsao <jungchang.tsao@mediatek.com>
> > + *
> > + * This program is free software; you can redistribute it and/or
> > + * modify
> > + * it under the terms of the GNU General Public License version 2 as
> > + * published by the Free Software Foundation.
> > + *
> > + * This program is distributed in the hope that it will be useful,
> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > + * GNU General Public License for more details.
> > + */
> > +
> > +#include <linux/interrupt.h>
> > +#include <linux/kernel.h>
> > +#include <linux/slab.h>
> > +
> > +#include "venc_drv_if.h"
> > +#include "mtk_vcodec_enc.h"
> > +#include "mtk_vcodec_pm.h"
> > +#include "mtk_vpu.h"
> > +
> > +#include "venc_drv_base.h"
> > +
> > +int venc_if_create(struct mtk_vcodec_ctx *ctx, unsigned int fourcc)
> > +{
> > +       char str[10];
> > +       int ret = 0;
> > +
> > +       mtk_vcodec_fmt2str(fourcc, str);
> > +
> > +       switch (fourcc) {
> > +       case V4L2_PIX_FMT_VP8:
> > +       case V4L2_PIX_FMT_H264:
> > +       default:
> > +               return -EINVAL;
> > +       }
> > +
> > +       mtk_venc_lock(ctx);
> > +       mtk_vcodec_enc_clock_on(&ctx->dev->pm);
> > +       ret = ctx->enc_if->init(ctx, (unsigned long *)&ctx->drv_handle);
> > +       mtk_vcodec_enc_clock_off(&ctx->dev->pm);
> > +       mtk_venc_unlock(ctx);
> > +
> > +       return ret;
> > +
> > +}
> > +
> > +
> > +int venc_if_set_param(struct mtk_vcodec_ctx *ctx,
> > +                     enum venc_set_param_type type, void *in)
> > +{
> > +       int ret = 0;
> > +
> > +       mtk_venc_lock(ctx);
> > +       mtk_vcodec_enc_clock_on(&ctx->dev->pm);
> > +       ret = ctx->enc_if->set_param(ctx->drv_handle, type, in);
> > +       mtk_vcodec_enc_clock_off(&ctx->dev->pm);
> > +       mtk_venc_unlock(ctx);
> > +
> > +       return ret;
> > +}
> > +
> > +int venc_if_encode(struct mtk_vcodec_ctx *ctx,
> > +                  enum venc_start_opt opt, struct venc_frm_buf *frm_buf,
> > +                  struct mtk_vcodec_mem *bs_buf,
> > +                  struct venc_done_result *result)
> > +{
> > +       int ret = 0;
> > +
> > +       mtk_venc_lock(ctx);
> > +       mtk_vcodec_enc_clock_on(&ctx->dev->pm);
> > +       ret = ctx->enc_if->encode(ctx->drv_handle, opt, frm_buf, bs_buf, result);
> > +       mtk_vcodec_enc_clock_off(&ctx->dev->pm);
> > +       mtk_venc_unlock(ctx);
> > +
> > +       return ret;
> > +}
> > +
> > +int venc_if_release(struct mtk_vcodec_ctx *ctx)
> > +{
> > +       int ret = 0;
> > +
> > +       if(ctx->drv_handle == 0)
> > +               return 0;
> > +
> > +       mtk_venc_lock(ctx);
> > +       mtk_vcodec_enc_clock_on(&ctx->dev->pm);
> > +       ret = ctx->enc_if->deinit(ctx->drv_handle);
> > +       mtk_vcodec_enc_clock_off(&ctx->dev->pm);
> > +       mtk_venc_unlock(ctx);
> > +
> > +       ctx->drv_handle = 0;
> > +
> > +       return ret;
> > +}
> > +
> > diff --git a/drivers/media/platform/mtk-vcodec/venc_drv_if.h b/drivers/media/platform/mtk-vcodec/venc_drv_if.h
> > new file mode 100644
> > index 0000000..a387011
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/venc_drv_if.h
> > @@ -0,0 +1,175 @@
> > +/*
> > + * Copyright (c) 2015 MediaTek Inc.
> > + * Author: Daniel Hsiao <daniel.hsiao@mediatek.com>
> > + *         Jungchang Tsao <jungchang.tsao@mediatek.com>
> > + *
> > + * This program is free software; you can redistribute it and/or
> > + * modify
> > + * it under the terms of the GNU General Public License version 2 as
> > + * published by the Free Software Foundation.
> > + *
> > + * This program is distributed in the hope that it will be useful,
> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > + * GNU General Public License for more details.
> > + */
> > +
> > +#ifndef _VENC_DRV_IF_H_
> > +#define _VENC_DRV_IF_H_
> > +
> > +#include "mtk_vcodec_drv.h"
> > +#include "mtk_vcodec_util.h"
> > +
> > +/*
> > + * enum venc_yuv_fmt - The type of input yuv format
> > + * (VPU related: If you change the order, you must also update the VPU codes.)
> > + * @VENC_YUV_FORMAT_420: 420 YUV format
> > + * @VENC_YUV_FORMAT_YV12: YV12 YUV format
> > + * @VENC_YUV_FORMAT_NV12: NV12 YUV format
> > + * @VENC_YUV_FORMAT_NV21: NV21 YUV format
> > + */
> > +enum venc_yuv_fmt {
> > +       VENC_YUV_FORMAT_420 = 3,
> > +       VENC_YUV_FORMAT_YV12 = 5,
> > +       VENC_YUV_FORMAT_NV12 = 6,
> > +       VENC_YUV_FORMAT_NV21 = 7,
> > +};
> > +
> > +/*
> > + * enum venc_start_opt - encode frame option used in venc_if_encode()
> > + * @VENC_START_OPT_ENCODE_SEQUENCE_HEADER: encode SPS/PPS for H264
> > + * @VENC_START_OPT_ENCODE_FRAME: encode normal frame
> > + */
> > +enum venc_start_opt {
> > +       VENC_START_OPT_ENCODE_SEQUENCE_HEADER,
> > +       VENC_START_OPT_ENCODE_FRAME,
> > +};
> > +
> > +/*
> > + * enum venc_drv_msg - The type of encode frame status used in venc_if_encode()
> > + * @VENC_MESSAGE_OK: encode ok
> > + * @VENC_MESSAGE_ERR: encode error
> > + */
> > +enum venc_drv_msg {
> > +       VENC_MESSAGE_OK,
> > +       VENC_MESSAGE_ERR,
> > +};
> > +
> > +/*
> > + * enum venc_set_param_type - The type of set parameter used in venc_if_set_param()
> > + * (VPU related: If you change the order, you must also update the VPU codes.)
> > + * @VENC_SET_PARAM_ENC: set encoder parameters
> > + * @VENC_SET_PARAM_FORCE_INTRA: set force intra frame
> > + * @VENC_SET_PARAM_ADJUST_BITRATE: set to adjust bitrate (in bps)
> > + * @VENC_SET_PARAM_ADJUST_FRAMERATE: set frame rate
> > + * @VENC_SET_PARAM_I_FRAME_INTERVAL: set I frame interval
> > + * @VENC_SET_PARAM_SKIP_FRAME: set H264 skip one frame
> > + * @VENC_SET_PARAM_PREPEND_HEADER: set H264 prepend SPS/PPS before IDR
> > + * @VENC_SET_PARAM_TS_MODE: set VP8 temporal scalability mode
> > + */
> > +enum venc_set_param_type {
> > +       VENC_SET_PARAM_ENC,
> > +       VENC_SET_PARAM_FORCE_INTRA,
> > +       VENC_SET_PARAM_ADJUST_BITRATE,
> > +       VENC_SET_PARAM_ADJUST_FRAMERATE,
> > +       VENC_SET_PARAM_I_FRAME_INTERVAL,
> > +       VENC_SET_PARAM_SKIP_FRAME,
> > +       VENC_SET_PARAM_PREPEND_HEADER,
> > +       VENC_SET_PARAM_TS_MODE,
> > +};
> > +
> > +/*
> > + * struct venc_enc_prm - encoder settings for VENC_SET_PARAM_ENC used in venc_if_set_param()
> > + * @input_fourcc: input fourcc
> > + * @h264_profile: V4L2 defined H.264 profile
> > + * @h264_level: V4L2 defined H.264 level
> > + * @width: image width
> > + * @height: image height
> > + * @buf_width: buffer width
> > + * @buf_height: buffer height
> > + * @frm_rate: frame rate
> > + * @intra_period: intra frame period
> > + * @bitrate: target bitrate in kbps
> > + */
> > +struct venc_enc_prm {
> > +       enum venc_yuv_fmt input_fourcc;
> > +       unsigned int h264_profile;
> > +       unsigned int h264_level;
> > +       unsigned int width;
> > +       unsigned int height;
> > +       unsigned int buf_width;
> > +       unsigned int buf_height;
> > +       unsigned int frm_rate;
> > +       unsigned int intra_period;
> > +       unsigned int bitrate;
> > +       unsigned int sizeimage[MTK_VCODEC_MAX_PLANES];
> > +};
> > +
> > +/*
> > + * struct venc_frm_buf - frame buffer information used in venc_if_encode()
> > + * @fb_addr: plane 0 frame buffer address
> > + * @fb_addr1: plane 1 frame buffer address
> > + * @fb_addr2: plane 2 frame buffer address
> > + */
> > +struct venc_frm_buf {
> > +       struct mtk_vcodec_mem fb_addr;
> > +       struct mtk_vcodec_mem fb_addr1;
> > +       struct mtk_vcodec_mem fb_addr2;
> > +};
> > +
> > +/*
> > + * struct venc_done_result - This is return information used in venc_if_encode()
> > + * @msg: message, such as success or error code
> > + * @bs_size: output bitstream size
> > + * @is_key_frm: output is key frame or not
> > + */
> > +struct venc_done_result {
> > +       enum venc_drv_msg msg;
> > +       unsigned int bs_size;
> > +       bool is_key_frm;
> > +};
> > +
> > +/*
> > + * venc_if_create - Create the driver handle
> > + * @ctx: device context
> > + * @fourcc: encoder output format
> > + * @handle: driver handle
> > + * Return: 0 if creating handle successfully, otherwise it is failed.
> > + */
> > +int venc_if_create(struct mtk_vcodec_ctx *ctx, unsigned int fourcc);
> > +
> > +/*
> > + * venc_if_release - Release the driver handle
> > + * @handle: driver handle
> > + * Return: 0 if releasing handle successfully, otherwise it is failed.
> > + */
> > +int venc_if_release(struct mtk_vcodec_ctx *ctx);
> > +
> > +/*
> > + * venc_if_set_param - Set parameter to driver
> > + * @handle: driver handle
> > + * @type: set type
> > + * @in: input parameter
> > + * @out: output parameter
> > + * Return: 0 if setting param successfully, otherwise it is failed.
> > + */
> > +int venc_if_set_param(struct mtk_vcodec_ctx *ctx,
> > +                     enum venc_set_param_type type,
> > +                     void *in);
> > +
> > +/*
> > + * venc_if_encode - Encode frame
> > + * @handle: driver handle
> > + * @opt: encode frame option
> > + * @frm_buf: input frame buffer information
> > + * @bs_buf: output bitstream buffer infomraiton
> > + * @result: encode result
> > + * Return: 0 if encoding frame successfully, otherwise it is failed.
> > + */
> > +int venc_if_encode(struct mtk_vcodec_ctx *ctx,
> > +                  enum venc_start_opt opt,
> > +                  struct venc_frm_buf *frm_buf,
> > +                  struct mtk_vcodec_mem *bs_buf,
> > +                  struct venc_done_result *result);
> > +
> > +#endif /* _VENC_DRV_IF_H_ */
> > diff --git a/drivers/media/platform/mtk-vcodec/venc_ipi_msg.h b/drivers/media/platform/mtk-vcodec/venc_ipi_msg.h
> > new file mode 100644
> > index 0000000..a345b98
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/venc_ipi_msg.h
> > @@ -0,0 +1,212 @@
> > +/*
> > + * Copyright (c) 2015 MediaTek Inc.
> > + * Author: Jungchang Tsao <jungchang.tsao@mediatek.com>
> > + *         Daniel Hsiao <daniel.hsiao@mediatek.com>
> > + *
> > + * This program is free software; you can redistribute it and/or
> > + * modify
> > + * it under the terms of the GNU General Public License version 2 as
> > + * published by the Free Software Foundation.
> > + *
> > + * This program is distributed in the hope that it will be useful,
> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > + * GNU General Public License for more details.
> > + */
> > +
> > +#ifndef _VENC_IPI_MSG_H_
> > +#define _VENC_IPI_MSG_H_
> > +
> > +#define IPIMSG_H264_ENC_ID 0x100
> > +#define IPIMSG_VP8_ENC_ID 0x200
> > +
> > +#define AP_IPIMSG_VENC_BASE 0x20000
> > +#define VPU_IPIMSG_VENC_BASE 0x30000
> > +
> > +/**
> > + * enum venc_ipi_msg_id - message id between AP and VPU
> > + * (ipi stands for inter-processor interrupt)
> > + * @AP_IPIMSG_XXX:             AP to VPU cmd message id
> > + * @VPU_IPIMSG_XXX_DONE:       VPU ack AP cmd message id
> > + */
> > +enum venc_ipi_msg_id {
> > +       AP_IPIMSG_H264_ENC_INIT = AP_IPIMSG_VENC_BASE +
> > +                                 IPIMSG_H264_ENC_ID,
> > +       AP_IPIMSG_H264_ENC_SET_PARAM,
> > +       AP_IPIMSG_H264_ENC_ENCODE,
> > +       AP_IPIMSG_H264_ENC_DEINIT,
> > +
> > +       AP_IPIMSG_VP8_ENC_INIT = AP_IPIMSG_VENC_BASE +
> > +                                IPIMSG_VP8_ENC_ID,
> > +       AP_IPIMSG_VP8_ENC_SET_PARAM,
> > +       AP_IPIMSG_VP8_ENC_ENCODE,
> > +       AP_IPIMSG_VP8_ENC_DEINIT,
> > +
> > +       VPU_IPIMSG_H264_ENC_INIT_DONE = VPU_IPIMSG_VENC_BASE +
> > +                                       IPIMSG_H264_ENC_ID,
> > +       VPU_IPIMSG_H264_ENC_SET_PARAM_DONE,
> > +       VPU_IPIMSG_H264_ENC_ENCODE_DONE,
> > +       VPU_IPIMSG_H264_ENC_DEINIT_DONE,
> > +
> > +       VPU_IPIMSG_VP8_ENC_INIT_DONE = VPU_IPIMSG_VENC_BASE +
> > +                                      IPIMSG_VP8_ENC_ID,
> > +       VPU_IPIMSG_VP8_ENC_SET_PARAM_DONE,
> > +       VPU_IPIMSG_VP8_ENC_ENCODE_DONE,
> > +       VPU_IPIMSG_VP8_ENC_DEINIT_DONE,
> > +};
> > +
> > +/**
> > + * struct venc_ap_ipi_msg_init - AP to VPU init cmd structure
> > + * @msg_id:    message id (AP_IPIMSG_XXX_ENC_INIT)
> > + * @venc_inst: AP encoder instance (struct venc_vp8_handle/venc_h264_handle *)
> > + */
> > +struct venc_ap_ipi_msg_init {
> > +       uint32_t msg_id;
> > +       uint32_t reserved;
> > +       uint64_t venc_inst;
> > +};
> > +
> > +/**
> > + * struct venc_ap_ipi_msg_set_param - AP to VPU set_param cmd structure
> > + * @msg_id:    message id (AP_IPIMSG_XXX_ENC_SET_PARAM)
> > + * @inst_id:   VPU encoder instance id (struct venc_vp8_vpu_drv/venc_h264_vpu_drv *)
> > + * @param_id:  parameter id (venc_set_param_type)
> > + * @data_item: number of items in the data array
> > + * @data[8]:   data array to store the set parameters
> > + */
> > +struct venc_ap_ipi_msg_set_param {
> > +       uint32_t msg_id;
> > +       uint32_t inst_id;
> > +       uint32_t param_id;
> > +       uint32_t data_item;
> > +       uint32_t data[8];
> > +};
> > +
> > +/**
> > + * struct venc_ap_ipi_msg_enc - AP to VPU enc cmd structure
> > + * @msg_id:    message id (AP_IPIMSG_XXX_ENC_ENCODE)
> > + * @inst_id:   VPU encoder instance id (struct venc_vp8_vpu_drv/venc_h264_vpu_drv *)
> > + * @bs_mode:   bitstream mode for h264
> > + *             (H264_BS_MODE_SPS/H264_BS_MODE_PPS/H264_BS_MODE_FRAME)
> > + * @input_addr:        pointer to input image buffer plane
> > + * @bs_addr:   pointer to output bit stream buffer
> > + * @bs_size:   bit stream buffer size
> > + */
> > +struct venc_ap_ipi_msg_enc {
> > +       uint32_t msg_id;
> > +       uint32_t inst_id;
> > +       uint32_t bs_mode;
> > +       uint32_t input_addr[3];
> > +       uint32_t bs_addr;
> > +       uint32_t bs_size;
> > +};
> > +
> > +/**
> > + * struct venc_ap_ipi_msg_deinit - AP to VPU deinit cmd structure
> > + * @msg_id:    message id (AP_IPIMSG_XXX_ENC_DEINIT)
> > + * @inst_id:   VPU encoder instance id (struct venc_vp8_vpu_drv/venc_h264_vpu_drv *)
> > + */
> > +struct venc_ap_ipi_msg_deinit {
> > +       uint32_t msg_id;
> > +       uint32_t inst_id;
> > +};
> > +
> > +/**
> > + * enum venc_ipi_msg_status - VPU ack AP cmd status
> > + */
> > +enum venc_ipi_msg_status {
> > +       VENC_IPI_MSG_STATUS_OK,
> > +       VENC_IPI_MSG_STATUS_FAIL,
> > +};
> > +
> > +/**
> > + * struct venc_vpu_ipi_msg_common - VPU ack AP cmd common structure
> > + * @msg_id:    message id (VPU_IPIMSG_XXX_DONE)
> > + * @status:    cmd status (venc_ipi_msg_status)
> > + * @venc_inst: AP encoder instance (struct venc_vp8_handle/venc_h264_handle *)
> > + */
> > +struct venc_vpu_ipi_msg_common {
> > +       uint32_t msg_id;
> > +       uint32_t status;
> > +       uint64_t venc_inst;
> > +};
> > +
> > +/**
> > + * struct venc_vpu_ipi_msg_init - VPU ack AP init cmd structure
> > + * @msg_id:    message id (VPU_IPIMSG_XXX_ENC_SET_PARAM_DONE)
> > + * @status:    cmd status (venc_ipi_msg_status)
> > + * @venc_inst: AP encoder instance (struct venc_vp8_handle/venc_h264_handle *)
> > + * @inst_id:   VPU encoder instance id (struct venc_vp8_vpu_drv/venc_h264_vpu_drv *)
> > + */
> > +struct venc_vpu_ipi_msg_init {
> > +       uint32_t msg_id;
> > +       uint32_t status;
> > +       uint64_t venc_inst;
> > +       uint32_t inst_id;
> > +       uint32_t reserved;
> > +};
> > +
> > +/**
> > + * struct venc_vpu_ipi_msg_set_param - VPU ack AP set_param cmd structure
> > + * @msg_id:    message id (VPU_IPIMSG_XXX_ENC_SET_PARAM_DONE)
> > + * @status:    cmd status (venc_ipi_msg_status)
> > + * @venc_inst: AP encoder instance (struct venc_vp8_handle/venc_h264_handle *)
> > + * @param_id:  parameter id (venc_set_param_type)
> > + * @data_item: number of items in the data array
> > + * @data[6]:   data array to store the return result
> > + */
> > +struct venc_vpu_ipi_msg_set_param {
> > +       uint32_t msg_id;
> > +       uint32_t status;
> > +       uint64_t venc_inst;
> > +       uint32_t param_id;
> > +       uint32_t data_item;
> > +       uint32_t data[6];
> > +};
> > +
> > +/**
> > + * enum venc_ipi_msg_enc_state - Type of encode state
> > + * VEN_IPI_MSG_ENC_STATE_FRAME:        one frame being encoded
> > + * VEN_IPI_MSG_ENC_STATE_PART: bit stream buffer full
> > + * VEN_IPI_MSG_ENC_STATE_SKIP: encoded skip frame
> > + * VEN_IPI_MSG_ENC_STATE_ERROR:        encounter error
> > + */
> > +enum venc_ipi_msg_enc_state {
> > +       VEN_IPI_MSG_ENC_STATE_FRAME,
> > +       VEN_IPI_MSG_ENC_STATE_PART,
> > +       VEN_IPI_MSG_ENC_STATE_SKIP,
> > +       VEN_IPI_MSG_ENC_STATE_ERROR,
> > +};
> > +
> > +/**
> > + * struct venc_vpu_ipi_msg_enc - VPU ack AP enc cmd structure
> > + * @msg_id:    message id (VPU_IPIMSG_XXX_ENC_ENCODE_DONE)
> > + * @status:    cmd status (venc_ipi_msg_status)
> > + * @venc_inst: AP encoder instance (struct venc_vp8_handle/venc_h264_handle *)
> > + * @state:     encode state (venc_ipi_msg_enc_state)
> > + * @key_frame: whether the encoded frame is key frame
> > + * @bs_size:   encoded bitstream size
> > + */
> > +struct venc_vpu_ipi_msg_enc {
> > +       uint32_t msg_id;
> > +       uint32_t status;
> > +       uint64_t venc_inst;
> > +       uint32_t state;
> > +       uint32_t key_frame;
> > +       uint32_t bs_size;
> > +       uint32_t reserved;
> > +};
> > +
> > +/**
> > + * struct venc_vpu_ipi_msg_deinit - VPU ack AP deinit cmd structure
> > + * @msg_id:   message id (VPU_IPIMSG_XXX_ENC_DEINIT_DONE)
> > + * @status:   cmd status (venc_ipi_msg_status)
> > + * @venc_inst: AP encoder instance (struct venc_vp8_handle/venc_h264_handle *)
> > + */
> > +struct venc_vpu_ipi_msg_deinit {
> > +       uint32_t msg_id;
> > +       uint32_t status;
> > +       uint64_t venc_inst;
> > +};
> > +
> > +#endif /* _VENC_IPI_MSG_H_ */
> > diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
> > old mode 100644
> > new mode 100755
> > index ee9d530..3ac35c4
> > --- a/include/uapi/linux/v4l2-controls.h
> > +++ b/include/uapi/linux/v4l2-controls.h
> > @@ -646,6 +646,10 @@ enum v4l2_mpeg_mfc51_video_force_frame_type {
> >  #define V4L2_CID_MPEG_MFC51_VIDEO_H264_NUM_REF_PIC_FOR_P               (V4L2_CID_MPEG_MFC51_BASE+54)
> >
> >
> > +#define V4L2_CID_MPEG_MTK_BASE                                 (V4L2_CTRL_CLASS_MPEG | 0x5500)
> > +#define V4L2_CID_MPEG_MTK_VIDEO_FRAME_SKIP_ENABLE              (V4L2_CID_MPEG_MTK_BASE+0)
> > +#define V4L2_CID_MPEG_MTK_VIDEO_FORCE_FRAME_TYPE_I_FRAME       (V4L2_CID_MPEG_MTK_BASE+1)
> > +
> >  /*  Camera class control IDs */
> >
> >  #define V4L2_CID_CAMERA_CLASS_BASE     (V4L2_CTRL_CLASS_CAMERA | 0x900)
> > --
> > 1.7.9.5
> >
> > --
> > To unsubscribe from this list: send the line "unsubscribe linux-media" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 102+ messages in thread

* Re: [PATCH v4 5/8] [Media] vcodec: mediatek: Add Mediatek V4L2 Video Encoder Driver
@ 2016-02-24  8:26               ` tiffany lin
  0 siblings, 0 replies; 102+ messages in thread
From: tiffany lin @ 2016-02-24  8:26 UTC (permalink / raw)
  To: Wu-Cheng Li (李務誠)
  Cc: Hans Verkuil, Daniel Thompson, Rob Herring,
	Mauro Carvalho Chehab, Matthias Brugger, Daniel Kurtz,
	Pawel Osciak, Eddie Huang, Yingjoe Chen,
	devicetree-u79uwXL29TY76Z2rM5mHXA,
	linux-kernel-u79uwXL29TY76Z2rM5mHXA,
	linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
	linux-media-u79uwXL29TY76Z2rM5mHXA,
	linux-mediatek-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r, Lin PoChun,
	Andrew-CT Chen

Hi Wucheng,

On Tue, 2016-02-23 at 13:46 +0800, Wu-Cheng Li (李務誠) wrote:
> On Thu, Feb 4, 2016 at 7:35 PM, Tiffany Lin <tiffany.lin-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org> wrote:
> > From: Andrew-CT Chen <andrew-ct.chen-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
> >
> > Add v4l2 layer encoder driver for MT8173
> >
> > Signed-off-by: Tiffany Lin <tiffany.lin-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
> > ---
> >  drivers/media/platform/Kconfig                     |   11 +
> >  drivers/media/platform/Makefile                    |    2 +
> >  drivers/media/platform/mtk-vcodec/Makefile         |    8 +
> >  drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h |  388 ++++++
> >  drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c | 1380 ++++++++++++++++++++
> >  drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h |   46 +
> >  .../media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c |  476 +++++++
> >  .../media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c  |  132 ++
> >  .../media/platform/mtk-vcodec/mtk_vcodec_intr.c    |  102 ++
> >  .../media/platform/mtk-vcodec/mtk_vcodec_intr.h    |   29 +
> >  drivers/media/platform/mtk-vcodec/mtk_vcodec_pm.h  |   26 +
> >  .../media/platform/mtk-vcodec/mtk_vcodec_util.c    |  106 ++
> >  .../media/platform/mtk-vcodec/mtk_vcodec_util.h    |   85 ++
> >  drivers/media/platform/mtk-vcodec/venc_drv_base.h  |   62 +
> >  drivers/media/platform/mtk-vcodec/venc_drv_if.c    |  100 ++
> >  drivers/media/platform/mtk-vcodec/venc_drv_if.h    |  175 +++
> >  drivers/media/platform/mtk-vcodec/venc_ipi_msg.h   |  212 +++
> >  include/uapi/linux/v4l2-controls.h                 |    4 +
> >  18 files changed, 3344 insertions(+)
> >  create mode 100644 drivers/media/platform/mtk-vcodec/Makefile
> >  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
> >  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
> >  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h
> >  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
> >  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
> >  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.c
> >  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.h
> >  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_pm.h
> >  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c
> >  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
> >  create mode 100644 drivers/media/platform/mtk-vcodec/venc_drv_base.h
> >  create mode 100644 drivers/media/platform/mtk-vcodec/venc_drv_if.c
> >  create mode 100644 drivers/media/platform/mtk-vcodec/venc_drv_if.h
> >  create mode 100644 drivers/media/platform/mtk-vcodec/venc_ipi_msg.h
> >  mode change 100644 => 100755 include/uapi/linux/v4l2-controls.h
> >
> > diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
> > index ba812d6..3e831c5 100644
> > --- a/drivers/media/platform/Kconfig
> > +++ b/drivers/media/platform/Kconfig
> > @@ -157,6 +157,17 @@ config VIDEO_MEDIATEK_VPU
> >             codec embedded in new Mediatek's SOCs. It is able
> >             to handle video decoding/encoding in a range of formats.
> >
> > +config VIDEO_MEDIATEK_VCODEC
> > +        tristate "Mediatek Video Codec driver"
> > +        depends on VIDEO_DEV && VIDEO_V4L2
> > +        depends on ARCH_MEDIATEK || COMPILE_TEST
> > +        select VIDEOBUF2_DMA_CONTIG
> > +        select V4L2_MEM2MEM_DEV
> > +        select MEDIATEK_VPU
> > +        default n
> > +        ---help---
> > +            Mediatek video codec driver for V4L2
> > +
> >  config VIDEO_MEM2MEM_DEINTERLACE
> >         tristate "Deinterlace support"
> >         depends on VIDEO_DEV && VIDEO_V4L2 && DMA_ENGINE
> > diff --git a/drivers/media/platform/Makefile b/drivers/media/platform/Makefile
> > index e5b19c6..510e06b 100644
> > --- a/drivers/media/platform/Makefile
> > +++ b/drivers/media/platform/Makefile
> > @@ -57,3 +57,5 @@ obj-$(CONFIG_VIDEO_XILINX)            += xilinx/
> >  ccflags-y += -I$(srctree)/drivers/media/i2c
> >
> >  obj-$(CONFIG_VIDEO_MEDIATEK_VPU)       += mtk-vpu/
> > +
> > +obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC)    += mtk-vcodec/
> > diff --git a/drivers/media/platform/mtk-vcodec/Makefile b/drivers/media/platform/mtk-vcodec/Makefile
> > new file mode 100644
> > index 0000000..ce38689
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/Makefile
> > @@ -0,0 +1,8 @@
> > +obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += mtk_vcodec_intr.o \
> > +                                      mtk_vcodec_util.o \
> > +                                      mtk_vcodec_enc_drv.o \
> > +                                      mtk_vcodec_enc.o \
> > +                                      mtk_vcodec_enc_pm.o \
> > +                                      venc_drv_if.o
> > +
> > +ccflags-y += -I$(srctree)/drivers/media/platform/mtk-vpu
> > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
> > new file mode 100644
> > index 0000000..9da2818
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
> > @@ -0,0 +1,388 @@
> > +/*
> > +* Copyright (c) 2015 MediaTek Inc.
> > +* Author: PC Chen <pc.chen-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
> > +*         Tiffany Lin <tiffany.lin-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
> > +*
> > +* This program is free software; you can redistribute it and/or modify
> > +* it under the terms of the GNU General Public License version 2 as
> > +* published by the Free Software Foundation.
> > +*
> > +* This program is distributed in the hope that it will be useful,
> > +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> > +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > +* GNU General Public License for more details.
> > +*/
> > +
> > +#ifndef _MTK_VCODEC_DRV_H_
> > +#define _MTK_VCODEC_DRV_H_
> > +
> > +#include <linux/platform_device.h>
> > +#include <linux/videodev2.h>
> > +#include <media/v4l2-ctrls.h>
> > +#include <media/v4l2-device.h>
> > +#include <media/v4l2-ioctl.h>
> > +#include <media/videobuf2-core.h>
> > +
> > +#include "mtk_vcodec_util.h"
> > +
> > +#define MTK_VCODEC_MAX_INSTANCES               10
> > +#define MTK_VCODEC_MAX_ENCODER_INSTANCES       3
> > +
> > +#define MTK_VCODEC_MAX_FRAME_SIZE      0x800000
> > +#define MTK_VIDEO_MAX_FRAME            32
> > +#define MTK_MAX_CTRLS                  20
> > +
> > +#define MTK_VCODEC_DRV_NAME            "mtk_vcodec_drv"
> > +#define MTK_VCODEC_ENC_NAME            "mtk-vcodec-enc"
> > +#define MTK_PLATFORM_STR               "platform:mt8173"
> > +
> > +#define MTK_VENC_IRQ_STATUS_SPS          0x1
> > +#define MTK_VENC_IRQ_STATUS_PPS          0x2
> > +#define MTK_VENC_IRQ_STATUS_FRM          0x4
> > +#define MTK_VENC_IRQ_STATUS_DRAM         0x8
> > +#define MTK_VENC_IRQ_STATUS_PAUSE        0x10
> > +#define MTK_VENC_IRQ_STATUS_SWITCH       0x20
> > +
> > +#define MTK_VENC_IRQ_STATUS_OFFSET       0x05C
> > +#define MTK_VENC_IRQ_ACK_OFFSET          0x060
> > +
> > +#define MTK_VCODEC_MAX_PLANES          3
> > +
> > +#define VDEC_HW_ACTIVE 0x10
> > +#define VDEC_IRQ_CFG    0x11
> > +#define VDEC_IRQ_CLR    0x10
> > +
> > +#define VDEC_IRQ_CFG_REG       0xa4
> > +#define WAIT_INTR_TIMEOUT      1000
> > +
> > +/**
> > + * enum mtk_hw_reg_idx - MTK hw register base index
> > + */
> > +enum mtk_hw_reg_idx {
> > +       VDEC_SYS,
> > +       VDEC_MISC,
> > +       VDEC_LD,
> > +       VDEC_TOP,
> > +       VDEC_CM,
> > +       VDEC_AD,
> > +       VDEC_AV,
> > +       VDEC_PP,
> > +       VDEC_HWD,
> > +       VDEC_HWQ,
> > +       VDEC_HWB,
> > +       VDEC_HWG,
> > +       NUM_MAX_VDEC_REG_BASE,
> > +       VENC_SYS = NUM_MAX_VDEC_REG_BASE,
> > +       VENC_LT_SYS,
> > +       NUM_MAX_VCODEC_REG_BASE
> > +};
> > +
> > +/**
> > + * enum mtk_instance_type - The type of an MTK Vcodec instance.
> > + */
> > +enum mtk_instance_type {
> > +       MTK_INST_DECODER                = 0,
> > +       MTK_INST_ENCODER                = 1,
> > +};
> > +
> > +/**
> > + * enum mtk_instance_state - The state of an MTK Vcodec instance.
> > + * @MTK_STATE_FREE - default state when instance is created
> > + * @MTK_STATE_INIT - vcodec instance is initialized
> > + * @MTK_STATE_HEADER - vdec had sps/pps header parsed or venc
> > + *                     had sps/pps header encoded
> > + * @MTK_STATE_FLUSH - vdec is flushing. Only used by decoder
> > + * @MTK_STATE_RES_CHANGE - vdec detect resolution change.
> > + *                     Only used by decoder
> > + * @MTK_STATE_ABORT - vcodec should be aborted
> > + */
> > +enum mtk_instance_state {
> > +       MTK_STATE_FREE = 0,
> > +       MTK_STATE_INIT = 1,
> > +       MTK_STATE_HEADER = 2,
> > +       MTK_STATE_FLUSH = 3,
> > +       MTK_STATE_RES_CHANGE = 4,
> > +       MTK_STATE_ABORT = 5,
> > +};
> > +
> > +/**
> > + * struct mtk_param_change - General encoding parameters type
> > + */
> > +enum mtk_encode_param {
> > +       MTK_ENCODE_PARAM_NONE = 0,
> > +       MTK_ENCODE_PARAM_BITRATE = (1 << 0),
> > +       MTK_ENCODE_PARAM_FRAMERATE = (1 << 1),
> > +       MTK_ENCODE_PARAM_INTRA_PERIOD = (1 << 2),
> > +       MTK_ENCODE_PARAM_FRAME_TYPE = (1 << 3),
> > +};
> > +
> > +/**
> > + * enum mtk_fmt_type - Type of the pixelformat
> > + * @MTK_FMT_FRAME - mtk vcodec raw frame
> > + */
> > +enum mtk_fmt_type {
> > +       MTK_FMT_DEC             = 0,
> > +       MTK_FMT_ENC             = 1,
> > +       MTK_FMT_FRAME           = 2,
> > +};
> > +
> > +/**
> > + * struct mtk_video_fmt - Structure used to store information about pixelformats
> > + */
> > +struct mtk_video_fmt {
> > +       u32 fourcc;
> > +       enum mtk_fmt_type type;
> > +       u32 num_planes;
> > +};
> > +
> > +/**
> > + * struct mtk_codec_framesizes - Structure used to store information about framesizes
> > + */
> > +struct mtk_codec_framesizes {
> > +       u32 fourcc;
> > +       struct  v4l2_frmsize_stepwise   stepwise;
> > +};
> > +
> > +/**
> > + * struct mtk_q_type - Type of queue
> > + */
> > +enum mtk_q_type {
> > +       MTK_Q_DATA_SRC          = 0,
> > +       MTK_Q_DATA_DST          = 1,
> > +};
> > +
> > +/**
> > + * struct mtk_q_data - Structure used to store information about queue
> > + * @colorspace reserved for encoder
> > + * @field              reserved for encoder
> > + */
> > +struct mtk_q_data {
> > +       unsigned int            width;
> > +       unsigned int            height;
> > +       enum v4l2_field         field;
> > +       enum v4l2_colorspace    colorspace;
> > +       unsigned int            bytesperline[MTK_VCODEC_MAX_PLANES];
> > +       unsigned int            sizeimage[MTK_VCODEC_MAX_PLANES];
> > +       struct mtk_video_fmt    *fmt;
> > +};
> > +
> > +/**
> > + * struct mtk_enc_params - General encoding parameters
> > + * @bitrate - target bitrate
> > + * @num_b_frame - number of b frames between p-frame
> > + * @rc_frame - frame based rate control
> > + * @rc_mb - macroblock based rate control
> > + * @seq_hdr_mode - H.264 sequence header is encoded separately or joined with the first frame
> > + * @gop_size - group of picture size, it's used as the intra frame period
> > + * @framerate_num - frame rate numerator
> > + * @framerate_denom - frame rate denominator
> > + * @h264_max_qp - Max value for H.264 quantization parameter
> > + * @h264_profile - V4L2 defined H.264 profile
> > + * @h264_level - V4L2 defined H.264 level
> > + * @force_intra - force/insert intra frame
> > + * @skip_frame - encode in skip frame mode that use minimum number of bits
> > + */
> > +struct mtk_enc_params {
> > +       unsigned int    bitrate;
> > +       unsigned int    num_b_frame;
> > +       unsigned int    rc_frame;
> > +       unsigned int    rc_mb;
> > +       unsigned int    seq_hdr_mode;
> > +       unsigned int    gop_size;
> > +       unsigned int    framerate_num;
> > +       unsigned int    framerate_denom;
> > +       unsigned int    h264_max_qp;
> > +       unsigned int    h264_profile;
> > +       unsigned int    h264_level;
> > +       unsigned int    force_intra;
> > +};
> > +
> > +/**
> > + * struct mtk_vcodec_pm - Power management data structure
> > + */
> > +struct mtk_vcodec_pm {
> > +       struct clk      *vcodecpll;
> > +       struct clk      *univpll_d2;
> > +       struct clk      *clk_cci400_sel;
> > +       struct clk      *vdecpll;
> > +       struct clk      *vdec_sel;
> > +       struct clk      *vencpll_d2;
> > +       struct clk      *venc_sel;
> > +       struct clk      *univpll1_d2;
> > +       struct clk      *venc_lt_sel;
> > +       struct device   *larbvdec;
> > +       struct device   *larbvenc;
> > +       struct device   *larbvenclt;
> > +       struct device   *dev;
> > +       struct mtk_vcodec_dev *mtkdev;
> > +};
> > +
> > +
> > +/**
> > + * struct mtk_vcodec_ctx - Context (instance) private data.
> > + *
> > + * @type:              type of the instance - decoder or encoder
> > + * @dev:               pointer to the mtk_vcodec_dev of the device
> > + * @fh:                        struct v4l2_fh
> > + * @m2m_ctx:           pointer to the v4l2_m2m_ctx of the context
> > + * @q_data:            store information of input and output queue
> > + *                     of the context
> > + * @idx:               index of the context that this structure describes
> > + * @state:             state of the context
> > + * @param_change:      encode parameters
> > + * @enc_params:                encoding parameters
> > + * @colorspace:
> > + * @enc_if:            hoooked encoder driver interface
> > + * @drv_handle:                driver handle for specific decode/encode instance
> > + *
> > + * @picinfo:           store width/height of image and buffer and planes' size for decoder
> > + *                     and encoder
> > + * @dpb_count:         count of the DPB buffers required by MTK Vcodec hw
> > + *
> > + * @int_cond:          variable used by the waitqueue
> > + * @int_type:          type of the last interrupt
> > + * @queue:             waitqueue that can be used to wait for this context to
> > + *                     finish
> > + * @irq_status:                irq status
> > + *
> > + * @ctrl_hdl:          handler for v4l2 framework
> > + * @ctrls:             array of controls, used when adding controls to the
> > + *                     v4l2 control framework
> > + *
> > + * @encode_work:       worker for the encoding
> > + */
> > +struct mtk_vcodec_ctx {
> > +       enum mtk_instance_type type;
> > +       struct mtk_vcodec_dev *dev;
> > +       struct v4l2_fh fh;
> > +       struct v4l2_m2m_ctx *m2m_ctx;
> > +       struct mtk_q_data q_data[2];
> > +       int idx;
> > +       enum mtk_instance_state state;
> > +       enum mtk_encode_param param_change;
> > +       struct mtk_enc_params enc_params;
> > +
> > +       struct venc_common_if *enc_if;
> > +       unsigned long drv_handle;
> > +
> > +
> > +       int int_cond;
> > +       int int_type;
> > +       wait_queue_head_t queue;
> > +       unsigned int irq_status;
> > +
> > +       struct v4l2_ctrl_handler ctrl_hdl;
> > +       struct v4l2_ctrl *ctrls[MTK_MAX_CTRLS];
> > +
> > +       struct work_struct encode_work;
> > +
> > +};
> > +
> > +/**
> > + * struct mtk_vcodec_dev - driver data
> > + * @v4l2_dev:          V4L2 device to register video devices for.
> > + * @vfd_enc:           Video device for encoder.
> > + *
> > + * @m2m_dev_enc:       m2m device for encoder.
> > + * @plat_dev:          platform device
> > + * @alloc_ctx:         VB2 allocator context
> > + *                     (for allocations without kernel mapping).
> > + * @ctx:               array of driver contexts
> > + *
> > + * @curr_ctx:          The context that is waiting for codec hardware
> > + *
> > + * @reg_base:          Mapped address of MTK Vcodec registers.
> > + *
> > + * @instance_mask:     used to mark which contexts are opened
> > + * @num_instances:     counter of active MTK Vcodec instances
> > + *
> > + * @encode_workqueue:  encode work queue
> > + *
> > + * @int_cond:          used to identify interrupt condition happen
> > + * @int_type:          used to identify what kind of interrupt condition happen
> > + * @dev_mutex:         video_device lock
> > + * @queue:             waitqueue for waiting for completion of device commands
> > + *
> > + * @enc_irq:           h264 encoder irq resource
> > + * @enc_lt_irq:                vp8 encoder irq resource
> > + *
> > + * @enc_mutex:         encoder hardware lock.
> > + *
> > + * @pm:                        power management control
> > + * @dec_capability:    used to identify decode capability, ex: 4k
> > + * @enc_capability:     used to identify encode capability
> > + */
> > +struct mtk_vcodec_dev {
> > +       struct v4l2_device      v4l2_dev;
> > +       struct video_device     *vfd_enc;
> > +
> > +       struct v4l2_m2m_dev     *m2m_dev_enc;
> > +       struct platform_device  *plat_dev;
> > +       struct platform_device  *vpu_plat_dev;
> > +       struct vb2_alloc_ctx    *alloc_ctx;
> > +       struct mtk_vcodec_ctx   *ctx[MTK_VCODEC_MAX_INSTANCES];
> > +       int curr_ctx;
> > +       void __iomem            *reg_base[NUM_MAX_VCODEC_REG_BASE];
> > +
> > +       unsigned long   instance_mask[BITS_TO_LONGS(MTK_VCODEC_MAX_INSTANCES)];
> > +       int                     num_instances;
> > +
> > +       struct workqueue_struct *encode_workqueue;
> > +
> > +       int                     int_cond;
> > +       int                     int_type;
> > +       struct mutex            dev_mutex;
> > +       wait_queue_head_t       queue;
> > +
> > +       int                     enc_irq;
> > +       int                     enc_lt_irq;
> > +
> > +       struct mutex            enc_mutex;
> > +
> > +       struct mtk_vcodec_pm    pm;
> > +       unsigned int            dec_capability;
> > +       unsigned int            enc_capability;
> > +};
> > +
> > +/**
> > + * struct mtk_vcodec_ctrl - information about controls to be registered.
> > + * @id:                        Control ID.
> > + * @type:              Type of the control.
> > + * @name:              Human readable name of the control.
> > + * @minimum:           Minimum value of the control.
> > + * @maximum:           Maximum value of the control.
> > + * @step:              Control value increase step.
> > + * @menu_skip_mask:    Mask of invalid menu positions.
> > + * @default_value:     Initial value of the control.
> > + * @is_volatile:       Control is volatile.
> > + *
> > + * See also struct v4l2_ctrl_config.
> > + */
> > +struct mtk_vcodec_ctrl {
> > +       u32                     id;
> > +       enum v4l2_ctrl_type     type;
> > +       u8                      name[32];
> > +       s32                     minimum;
> > +       s32                     maximum;
> > +       s32                     step;
> > +       u32                     menu_skip_mask;
> > +       s32                     default_value;
> > +       u8                      is_volatile;
> > +};
> > +
> > +static inline struct mtk_vcodec_ctx *fh_to_ctx(struct v4l2_fh *fh)
> > +{
> > +       return container_of(fh, struct mtk_vcodec_ctx, fh);
> > +}
> > +
> > +static inline struct mtk_vcodec_ctx *ctrl_to_ctx(struct v4l2_ctrl *ctrl)
> > +{
> > +       return container_of(ctrl->handler, struct mtk_vcodec_ctx, ctrl_hdl);
> > +}
> > +
> > +extern const struct v4l2_ioctl_ops mtk_vdec_ioctl_ops;
> > +extern const struct v4l2_m2m_ops mtk_vdec_m2m_ops;
> > +extern const struct v4l2_ioctl_ops mtk_venc_ioctl_ops;
> > +extern const struct v4l2_m2m_ops mtk_venc_m2m_ops;
> > +
> > +#endif /* _MTK_VCODEC_DRV_H_ */
> > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
> > new file mode 100644
> > index 0000000..ee602fe
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
> > @@ -0,0 +1,1380 @@
> > +/*
> > +* Copyright (c) 2015 MediaTek Inc.
> > +* Author: PC Chen <pc.chen-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
> > +*         Tiffany Lin <tiffany.lin-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
> > +*
> > +* This program is free software; you can redistribute it and/or modify
> > +* it under the terms of the GNU General Public License version 2 as
> > +* published by the Free Software Foundation.
> > +*
> > +* This program is distributed in the hope that it will be useful,
> > +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> > +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > +* GNU General Public License for more details.
> > +*/
> > +
> > +#include <media/v4l2-event.h>
> > +#include <media/v4l2-mem2mem.h>
> > +#include <media/videobuf2-dma-contig.h>
> > +
> > +#include "mtk_vcodec_drv.h"
> > +#include "mtk_vcodec_enc.h"
> > +#include "mtk_vcodec_intr.h"
> > +#include "mtk_vcodec_util.h"
> > +#include "venc_drv_if.h"
> > +
> > +#define MTK_VENC_MIN_W 32
> > +#define MTK_VENC_MIN_H 32
> > +#define MTK_VENC_MAX_W 1920
> > +#define MTK_VENC_MAX_H 1080
> > +#define DFT_CFG_WIDTH  MTK_VENC_MIN_W
> > +#define DFT_CFG_HEIGHT MTK_VENC_MIN_H
> > +
> > +static void mtk_venc_worker(struct work_struct *work);
> > +
> > +static struct mtk_video_fmt mtk_video_formats[] = {
> > +       {
> > +               .fourcc         = V4L2_PIX_FMT_YUV420,
> > +               .type           = MTK_FMT_FRAME,
> > +               .num_planes     = 3,
> This is not right. V4L2_PIX_FMT_YUV420 only has one physical plane.
> Same for V4L2_PIX_FMT_YVU420, V4L2_PIX_FMT_NV12, and
> V4L2_PIX_FMT_NV21.
We will fix this in next version.

best regards,
Tiffany

> > +       },
> > +       {
> > +               .fourcc         = V4L2_PIX_FMT_YVU420,
> > +               .type           = MTK_FMT_FRAME,
> > +               .num_planes     = 3,
> > +       },
> > +       {
> > +               .fourcc         = V4L2_PIX_FMT_NV12,
> > +               .type           = MTK_FMT_FRAME,
> > +               .num_planes     = 2,
> > +       },
> > +       {
> > +               .fourcc         = V4L2_PIX_FMT_NV21,
> > +               .type           = MTK_FMT_FRAME,
> > +               .num_planes     = 2,
> > +       },
> > +       {
> > +               .fourcc         = V4L2_PIX_FMT_YUV420M,
> > +               .type           = MTK_FMT_FRAME,
> > +               .num_planes     = 3,
> > +       },
> > +       {
> > +               .fourcc         = V4L2_PIX_FMT_YVU420M,
> > +               .type           = MTK_FMT_FRAME,
> > +               .num_planes     = 3,
> > +       },
> > +       {
> > +               .fourcc         = V4L2_PIX_FMT_NV12M,
> > +               .type           = MTK_FMT_FRAME,
> > +               .num_planes     = 2,
> > +       },
> > +       {
> > +               .fourcc         = V4L2_PIX_FMT_NV21M,
> > +               .type           = MTK_FMT_FRAME,
> > +               .num_planes     = 2,
> > +       },
> > +       {
> > +               .fourcc         = V4L2_PIX_FMT_H264,
> > +               .type           = MTK_FMT_ENC,
> > +               .num_planes     = 1,
> > +       },
> > +       {
> > +               .fourcc         = V4L2_PIX_FMT_VP8,
> > +               .type           = MTK_FMT_ENC,
> > +               .num_planes     = 1,
> > +       },
> > +};
> > +
> > +#define NUM_FORMATS ARRAY_SIZE(mtk_video_formats)
> > +
> > +static const struct mtk_codec_framesizes mtk_venc_framesizes[] = {
> > +       {
> > +               .fourcc = V4L2_PIX_FMT_H264,
> > +               .stepwise = {  160, 1920, 16, 128, 1088, 16 },
> > +       },
> > +       {
> > +               .fourcc = V4L2_PIX_FMT_VP8,
> > +               .stepwise = {  160, 1920, 16, 128, 1088, 16 },
> > +       },
> > +};
> > +
> > +#define NUM_SUPPORTED_FRAMESIZE ARRAY_SIZE(mtk_venc_framesizes)
> > +
> > +static int vidioc_venc_s_ctrl(struct v4l2_ctrl *ctrl)
> > +{
> > +       struct mtk_vcodec_ctx *ctx = ctrl_to_ctx(ctrl);
> > +       struct mtk_enc_params *p = &ctx->enc_params;
> > +       int ret = 0;
> > +
> > +       switch (ctrl->id) {
> > +       case V4L2_CID_MPEG_VIDEO_BITRATE:
> > +               mtk_v4l2_debug(2, "V4L2_CID_MPEG_VIDEO_BITRATE val = %d",
> > +                       ctrl->val);
> > +               p->bitrate = ctrl->val;
> > +               ctx->param_change |= MTK_ENCODE_PARAM_BITRATE;
> > +               break;
> > +       case V4L2_CID_MPEG_VIDEO_B_FRAMES:
> > +               mtk_v4l2_debug(2, "V4L2_CID_MPEG_VIDEO_B_FRAMES val = %d",
> > +                       ctrl->val);
> > +               p->num_b_frame = ctrl->val;
> > +               break;
> > +       case V4L2_CID_MPEG_VIDEO_FRAME_RC_ENABLE:
> > +               mtk_v4l2_debug(2, "V4L2_CID_MPEG_VIDEO_FRAME_RC_ENABLE val = %d",
> > +                       ctrl->val);
> > +               p->rc_frame = ctrl->val;
> > +               break;
> > +       case V4L2_CID_MPEG_VIDEO_H264_MAX_QP:
> > +               mtk_v4l2_debug(2, "V4L2_CID_MPEG_VIDEO_H264_MAX_QP val = %d",
> > +                       ctrl->val);
> > +               p->h264_max_qp = ctrl->val;
> > +               break;
> > +       case V4L2_CID_MPEG_VIDEO_HEADER_MODE:
> > +               mtk_v4l2_debug(2, "V4L2_CID_MPEG_VIDEO_HEADER_MODE val = %d",
> > +                       ctrl->val);
> > +               p->seq_hdr_mode = ctrl->val;
> > +               break;
> > +       case V4L2_CID_MPEG_VIDEO_MB_RC_ENABLE:
> > +               mtk_v4l2_debug(2, "V4L2_CID_MPEG_VIDEO_MB_RC_ENABLE val = %d",
> > +                       ctrl->val);
> > +               p->rc_mb = ctrl->val;
> > +               break;
> > +       case V4L2_CID_MPEG_VIDEO_H264_PROFILE:
> > +               mtk_v4l2_debug(2, "V4L2_CID_MPEG_VIDEO_H264_PROFILE val = %d",
> > +                       ctrl->val);
> > +               p->h264_profile = ctrl->val;
> > +               break;
> > +       case V4L2_CID_MPEG_VIDEO_H264_LEVEL:
> > +               mtk_v4l2_debug(2, "V4L2_CID_MPEG_VIDEO_H264_LEVEL val = %d",
> > +                       ctrl->val);
> > +               p->h264_level = ctrl->val;
> > +               break;
> > +       case V4L2_CID_MPEG_VIDEO_H264_I_PERIOD:
> > +               mtk_v4l2_debug(2, "V4L2_CID_MPEG_VIDEO_H264_I_PERIOD val = %d",
> > +                       ctrl->val);
> > +       case V4L2_CID_MPEG_VIDEO_GOP_SIZE:
> > +               mtk_v4l2_debug(2, "V4L2_CID_MPEG_VIDEO_GOP_SIZE val = %d",
> > +                       ctrl->val);
> > +               p->gop_size = ctrl->val;
> > +               ctx->param_change |= MTK_ENCODE_PARAM_INTRA_PERIOD;
> > +               break;
> > +       case V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME:
> > +               mtk_v4l2_debug(2, "V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME");
> > +               p->force_intra = 1;
> > +               ctx->param_change |= MTK_ENCODE_PARAM_FRAME_TYPE;
> > +               break;
> > +       default:
> > +               ret = -EINVAL;
> > +               break;
> > +       }
> > +
> > +       return ret;
> > +}
> > +
> > +static const struct v4l2_ctrl_ops mtk_vcodec_enc_ctrl_ops = {
> > +       .s_ctrl = vidioc_venc_s_ctrl,
> > +};
> > +
> > +static int vidioc_enum_fmt(struct file *file, struct v4l2_fmtdesc *f,
> > +                          bool out)
> > +{
> > +       struct mtk_video_fmt *fmt;
> > +       int i, j = 0;
> > +
> > +       for (i = 0; i < NUM_FORMATS; ++i) {
> > +               if (out && mtk_video_formats[i].type != MTK_FMT_FRAME)
> > +                       continue;
> > +               else if (!out && mtk_video_formats[i].type != MTK_FMT_ENC)
> > +                       continue;
> > +
> > +               if (j == f->index) {
> > +                       fmt = &mtk_video_formats[i];
> > +                       f->pixelformat = fmt->fourcc;
> > +                       return 0;
> > +               }
> > +               ++j;
> > +       }
> > +
> > +       return -EINVAL;
> > +}
> > +
> > +static int vidioc_enum_framesizes(struct file *file, void *fh,
> > +                                 struct v4l2_frmsizeenum *fsize)
> > +{
> > +       int i = 0;
> > +
> > +       for (i = 0; i < NUM_SUPPORTED_FRAMESIZE; ++i) {
> > +               if (fsize->pixel_format != mtk_venc_framesizes[i].fourcc)
> > +                       continue;
> > +
> > +               if (!fsize->index) {
> > +                       fsize->type = V4L2_FRMSIZE_TYPE_STEPWISE;
> > +                       fsize->stepwise = mtk_venc_framesizes[i].stepwise;
> > +                       return 0;
> > +               }
> > +       }
> > +
> > +       return -EINVAL;
> > +}
> > +
> > +static int vidioc_enum_fmt_vid_cap_mplane(struct file *file, void *pirv,
> > +                                         struct v4l2_fmtdesc *f)
> > +{
> > +       return vidioc_enum_fmt(file, f, false);
> > +}
> > +
> > +static int vidioc_enum_fmt_vid_out_mplane(struct file *file, void *prov,
> > +                                         struct v4l2_fmtdesc *f)
> > +{
> > +       return vidioc_enum_fmt(file, f, true);
> > +}
> > +
> > +static int vidioc_venc_querycap(struct file *file, void *priv,
> > +                               struct v4l2_capability *cap)
> > +{
> > +        strlcpy(cap->driver, MTK_VCODEC_ENC_NAME, strlen(MTK_VCODEC_ENC_NAME));
> > +        cap->driver[strlen(MTK_VCODEC_ENC_NAME)]=0;
> > +        strlcpy(cap->bus_info, MTK_PLATFORM_STR, strlen(MTK_PLATFORM_STR));
> > +        cap->bus_info[strlen(MTK_PLATFORM_STR)]=0;
> > +        strlcpy(cap->card, MTK_PLATFORM_STR, strlen(MTK_PLATFORM_STR));
> > +        cap->card[strlen(MTK_PLATFORM_STR)]=0;
> > +
> > +       /*
> > +        * This is only a mem-to-mem video device. The capture and output
> > +        * device capability flags are left only for backward compatibility
> > +        * and are scheduled for removal.
> > +        */
> > +       cap->device_caps  = V4L2_CAP_VIDEO_M2M_MPLANE | V4L2_CAP_STREAMING;
> > +       cap->capabilities = cap->device_caps | V4L2_CAP_DEVICE_CAPS;
> > +
> > +       return 0;
> > +}
> > +static int vidioc_venc_s_parm(struct file *file, void *priv,
> > +                             struct v4l2_streamparm *a)
> > +{
> > +       struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
> > +
> > +       if (a->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
> > +               ctx->enc_params.framerate_num =
> > +                       a->parm.output.timeperframe.denominator;
> > +               ctx->enc_params.framerate_denom =
> > +                       a->parm.output.timeperframe.numerator;
> > +               ctx->param_change |= MTK_ENCODE_PARAM_FRAMERATE;
> > +       } else {
> > +               return -EINVAL;
> > +       }
> > +       return 0;
> > +}
> > +
> > +static struct mtk_q_data *mtk_venc_get_q_data(struct mtk_vcodec_ctx *ctx,
> > +                                             enum v4l2_buf_type type)
> > +{
> > +       if (V4L2_TYPE_IS_OUTPUT(type))
> > +               return &ctx->q_data[MTK_Q_DATA_SRC];
> > +
> > +       return &ctx->q_data[MTK_Q_DATA_DST];
> > +}
> > +
> > +static struct mtk_video_fmt *mtk_venc_find_format(struct v4l2_format *f)
> > +{
> > +       struct mtk_video_fmt *fmt;
> > +       unsigned int k;
> > +
> > +       for (k = 0; k < NUM_FORMATS; k++) {
> > +               fmt = &mtk_video_formats[k];
> > +               if (fmt->fourcc == f->fmt.pix.pixelformat)
> > +                       return fmt;
> > +       }
> > +
> > +       return NULL;
> > +}
> > +
> > +static void mtk_vcodec_enc_calc_src_size(
> > +       unsigned int num_planes, unsigned int pic_width,
> > +       unsigned int pic_height, unsigned int sizeimage[],
> > +       unsigned int bytesperline[])
> > +{
> > +       unsigned int y_pitch_w_div16;
> > +       unsigned int c_pitch_w_div16;
> > +
> > +       y_pitch_w_div16 = ALIGN(pic_width, 16) >> 4;
> > +       c_pitch_w_div16 = ALIGN(pic_width, 16) >> 4;
> > +
> > +       if (num_planes == 2) {
> > +               sizeimage[0] =
> > +                       (y_pitch_w_div16) * (((pic_height + 31) / 32) * 2) * 256 +
> > +                       ((y_pitch_w_div16 % 8 == 0) ? 0 : ((ALIGN(pic_width, 16) * 2) * 16));
> > +
> > +               sizeimage[1] =
> > +                       (c_pitch_w_div16) * (((pic_height + 31) / 32) * 2) * 128 +
> > +                       ((c_pitch_w_div16 % 8 == 0) ? 0 : (ALIGN(pic_width, 16) * 16));
> > +
> > +               sizeimage[2] = 0;
> > +
> > +               bytesperline[0] = ALIGN(pic_width, 16);
> > +               bytesperline[1] = ALIGN(pic_width, 16);
> > +               bytesperline[2] = 0;
> > +
> > +       } else {
> > +               sizeimage[0] =
> > +                       (y_pitch_w_div16) * (((pic_height + 31) / 32) * 2) * 256 +
> > +                       ((y_pitch_w_div16 % 8 == 0) ? 0 : ((ALIGN(pic_width, 16) * 2) * 16));
> > +
> > +               sizeimage[1] =
> > +                       (c_pitch_w_div16) * (((pic_height + 31) / 32) * 2) * 64 +
> > +                       ((c_pitch_w_div16 % 8 == 0) ? 0 : ((ALIGN(pic_width, 16) / 2) * 16));
> > +
> > +               sizeimage[2] =
> > +                       (c_pitch_w_div16) * (((pic_height + 31) / 32) * 2) * 64 +
> > +                       ((c_pitch_w_div16 % 8 == 0) ? 0 : ((ALIGN(pic_width, 16) / 2) * 16));
> > +
> > +               bytesperline[0] = ALIGN(pic_width, 16);
> > +               bytesperline[1] = ALIGN(pic_width, 16) / 2;
> > +               bytesperline[2] = ALIGN(pic_width, 16) / 2;
> > +       }
> > +}
> > +
> > +static int vidioc_try_fmt(struct v4l2_format *f, struct mtk_video_fmt *fmt)
> > +{
> > +       struct v4l2_pix_format_mplane *pix_fmt_mp = &f->fmt.pix_mp;
> > +
> > +       /* V4L2 specification suggests the driver corrects the format struct
> > +         * if any of the dimensions is unsupported */
> > +        if (pix_fmt_mp->height < MTK_VENC_MIN_H)
> > +                pix_fmt_mp->height = MTK_VENC_MIN_H;
> > +        else if (pix_fmt_mp->height > MTK_VENC_MAX_H)
> > +                pix_fmt_mp->height = MTK_VENC_MAX_H;
> > +
> > +        if (pix_fmt_mp->width < MTK_VENC_MIN_W)
> > +                pix_fmt_mp->width = MTK_VENC_MIN_W;
> > +        else if (pix_fmt_mp->width > MTK_VENC_MAX_W)
> > +                pix_fmt_mp->width = MTK_VENC_MAX_W;
> > +
> > +        pix_fmt_mp->field = V4L2_FIELD_NONE;
> > +
> > +       if (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
> > +               int size = pix_fmt_mp->height * pix_fmt_mp->width;
> > +               if (fmt->num_planes != pix_fmt_mp->num_planes)
> > +                       pix_fmt_mp->num_planes = fmt->num_planes;
> > +
> > +               if(pix_fmt_mp->plane_fmt[0].sizeimage != size)
> > +                       pix_fmt_mp->plane_fmt[0].sizeimage = size;
> > +               pix_fmt_mp->plane_fmt[0].bytesperline = 0;
> > +               memset(&(pix_fmt_mp->plane_fmt[0].reserved[0]), 0x0,
> > +                       sizeof(pix_fmt_mp->plane_fmt[0].reserved));
> > +       } else if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
> > +               int i;
> > +               unsigned int sizeimage[VIDEO_MAX_PLANES];
> > +               unsigned int bytesperline[VIDEO_MAX_PLANES];
> > +
> > +               v4l_bound_align_image(&pix_fmt_mp->width, 8, 1920, 1,
> > +                                     &pix_fmt_mp->height, 4, 1080, 1, 0);
> > +
> > +               if (fmt->num_planes != pix_fmt_mp->num_planes)
> > +                       pix_fmt_mp->num_planes = fmt->num_planes;
> > +
> > +               mtk_vcodec_enc_calc_src_size(pix_fmt_mp->num_planes,
> > +                                       pix_fmt_mp->width,
> > +                                       pix_fmt_mp->height,
> > +                                       sizeimage, bytesperline);
> > +
> > +               for (i=0; i < pix_fmt_mp->num_planes; i++) {
> > +                       pix_fmt_mp->plane_fmt[i].sizeimage = sizeimage[i];
> > +                       pix_fmt_mp->plane_fmt[i].bytesperline = bytesperline[i];
> > +                       memset(&(pix_fmt_mp->plane_fmt[i].reserved[0]), 0x0,
> > +                               sizeof(pix_fmt_mp->plane_fmt[0].reserved));
> > +               }
> > +       } else {
> > +               return -EINVAL;
> > +       }
> > +
> > +       pix_fmt_mp->flags = 0;
> > +       pix_fmt_mp->ycbcr_enc = 0;
> > +       pix_fmt_mp->quantization = 0;
> > +       pix_fmt_mp->xfer_func = 0;
> > +       memset(&pix_fmt_mp->reserved[0], 0x0, sizeof(pix_fmt_mp->reserved));
> > +
> > +       return 0;
> > +}
> > +
> > +static void mtk_venc_set_param(struct mtk_vcodec_ctx *ctx, void *param)
> > +{
> > +       struct venc_enc_prm *p = (struct venc_enc_prm *)param;
> > +       struct mtk_q_data *q_data_src = &ctx->q_data[MTK_Q_DATA_SRC];
> > +       struct mtk_enc_params *enc_params = &ctx->enc_params;
> > +       unsigned int frame_rate;
> > +
> > +       frame_rate = enc_params->framerate_num / enc_params->framerate_denom;
> > +
> > +       switch (q_data_src->fmt->fourcc) {
> > +       case V4L2_PIX_FMT_YUV420:
> > +       case V4L2_PIX_FMT_YUV420M:
> > +               p->input_fourcc = VENC_YUV_FORMAT_420;
> > +               break;
> > +       case V4L2_PIX_FMT_YVU420:
> > +       case V4L2_PIX_FMT_YVU420M:
> > +               p->input_fourcc = VENC_YUV_FORMAT_YV12;
> > +               break;
> > +       case V4L2_PIX_FMT_NV12:
> > +       case V4L2_PIX_FMT_NV12M:
> > +               p->input_fourcc = VENC_YUV_FORMAT_NV12;
> > +               break;
> > +       case V4L2_PIX_FMT_NV21:
> > +       case V4L2_PIX_FMT_NV21M:
> > +               p->input_fourcc = VENC_YUV_FORMAT_NV21;
> > +               break;
> > +       }
> > +       p->h264_profile = enc_params->h264_profile;
> > +       p->h264_level = enc_params->h264_level;
> > +       p->width = q_data_src->width;
> > +       p->height = q_data_src->height;
> > +       p->buf_width = q_data_src->bytesperline[0];
> > +       p->buf_height = ((q_data_src->height + 0xf) & (~0xf));
> > +       p->frm_rate = frame_rate;
> > +       p->intra_period = enc_params->gop_size;
> > +       p->bitrate = enc_params->bitrate;
> > +
> > +       ctx->param_change = MTK_ENCODE_PARAM_NONE;
> > +
> > +       mtk_v4l2_debug(1, "fmt 0x%x, P/L %d/%d, w/h %d/%d, buf %d/%d, fps/bps %d/%d, gop %d",
> > +                      p->input_fourcc, p->h264_profile, p->h264_level,
> > +                      p->width, p->height, p->buf_width, p->buf_height,
> > +                      p->frm_rate, p->bitrate, p->intra_period);
> > +}
> > +
> > +static int vidioc_venc_s_fmt(struct file *file, void *priv,
> > +                            struct v4l2_format *f)
> > +{
> > +       struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
> > +       struct v4l2_pix_format_mplane *pix_fmt_mp = &f->fmt.pix_mp;
> > +       struct vb2_queue *vq;
> > +       struct mtk_q_data *q_data;
> > +       struct venc_enc_prm param;
> > +       int i, ret;
> > +       struct mtk_video_fmt *fmt;
> > +
> > +       vq = v4l2_m2m_get_vq(ctx->m2m_ctx, f->type);
> > +       if (!vq) {
> > +               mtk_v4l2_err("fail to get vq\n");
> > +               return -EINVAL;
> > +       }
> > +
> > +       if (vb2_is_busy(vq)) {
> > +               mtk_v4l2_err("queue busy\n");
> > +               return -EBUSY;
> > +       }
> > +
> > +       q_data = mtk_venc_get_q_data(ctx, f->type);
> > +       if (!q_data) {
> > +               mtk_v4l2_err("fail to get q data\n");
> > +               return -EINVAL;
> > +       }
> > +
> > +       fmt = mtk_venc_find_format(f);
> > +       if (!fmt) {
> > +               if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
> > +                       f->fmt.pix.pixelformat = mtk_video_formats[0].fourcc;
> > +                       fmt = mtk_venc_find_format(f);
> > +               } else {
> > +                       f->fmt.pix.pixelformat = mtk_video_formats[8].fourcc;
> > +                       fmt = mtk_venc_find_format(f);
> > +               }
> > +       }
> > +
> > +       q_data->fmt = fmt;
> > +       ret = vidioc_try_fmt(f, q_data->fmt);
> > +       if (ret)
> > +               return ret;
> > +
> > +       q_data->width           = f->fmt.pix_mp.width;
> > +       q_data->height          = f->fmt.pix_mp.height;
> > +       q_data->field           = f->fmt.pix_mp.field;
> > +
> > +       if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
> > +               q_data->colorspace = f->fmt.pix_mp.colorspace;
> > +               ctx->q_data[MTK_Q_DATA_SRC].bytesperline[0] =
> > +                       ALIGN(q_data->width, 16);
> > +
> > +               if (q_data->fmt->num_planes == 2) {
> > +                       ctx->q_data[MTK_Q_DATA_SRC].bytesperline[1] =
> > +                               ALIGN(q_data->width, 16);
> > +                       ctx->q_data[MTK_Q_DATA_SRC].bytesperline[2] = 0;
> > +               } else {
> > +                       ctx->q_data[MTK_Q_DATA_SRC].bytesperline[1] =
> > +                               ALIGN(q_data->width, 16) / 2;
> > +                       ctx->q_data[MTK_Q_DATA_SRC].bytesperline[2] =
> > +                               ALIGN(q_data->width, 16) / 2;
> > +               }
> > +
> > +               memset(&param, 0, sizeof(param));
> > +               mtk_venc_set_param(ctx, &param);
> > +               if (ctx->state == MTK_STATE_INIT) {
> > +                       ret = venc_if_set_param(ctx,
> > +                                               VENC_SET_PARAM_ENC,
> > +                                               &param);
> > +                       if (ret)
> > +                               mtk_v4l2_err("venc_if_set_param failed=%d\n",
> > +                                               ret);
> > +
> > +                       /* Get codec driver advice sizeimage from vpu */
> > +                       for (i = 0; i < MTK_VCODEC_MAX_PLANES; i++) {
> > +                               q_data->sizeimage[i] = param.sizeimage[i];
> > +                               pix_fmt_mp->plane_fmt[i].sizeimage =
> > +                                       param.sizeimage[i];
> > +                       }
> > +                       q_data->bytesperline[0] =
> > +                               pix_fmt_mp->plane_fmt[0].bytesperline;
> > +                       q_data->bytesperline[1] =
> > +                               pix_fmt_mp->plane_fmt[1].bytesperline;
> > +                       q_data->bytesperline[2] =
> > +                               pix_fmt_mp->plane_fmt[2].bytesperline;
> > +               } else {
> > +                       for (i = 0; i < MTK_VCODEC_MAX_PLANES; i++) {
> > +                               q_data->sizeimage[i] =
> > +                                       pix_fmt_mp->plane_fmt[i].sizeimage;
> > +                               q_data->bytesperline[i] =
> > +                                       pix_fmt_mp->plane_fmt[i].bytesperline;
> > +                       }
> > +               }
> > +       } else if (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE){
> > +               for (i = 0; i < f->fmt.pix_mp.num_planes; i++) {
> > +                       struct v4l2_plane_pix_format    *plane_fmt;
> > +
> > +                       plane_fmt = &f->fmt.pix_mp.plane_fmt[i];
> > +                       q_data->bytesperline[i] = plane_fmt->bytesperline;
> > +                       q_data->sizeimage[i]    = plane_fmt->sizeimage;
> > +               }
> > +
> > +               if (ctx->state == MTK_STATE_FREE) {
> > +                       ret = venc_if_create(ctx, q_data->fmt->fourcc);
> > +                       if (ret) {
> > +                               mtk_v4l2_err("venc_if_create failed=%d, codec type=%x\n",
> > +                                       ret, q_data->fmt->fourcc);
> > +                               return 0;
> > +                       }
> > +
> > +                       ctx->state = MTK_STATE_INIT;
> > +               }
> > +       }
> > +
> > +       return 0;
> > +}
> > +
> > +static int vidioc_venc_g_fmt(struct file *file, void *priv,
> > +                            struct v4l2_format *f)
> > +{
> > +       struct v4l2_pix_format_mplane *pix = &f->fmt.pix_mp;
> > +       struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
> > +       struct vb2_queue *vq;
> > +       struct mtk_q_data *q_data;
> > +       int i;
> > +
> > +       vq = v4l2_m2m_get_vq(ctx->m2m_ctx, f->type);
> > +       if (!vq)
> > +               return -EINVAL;
> > +
> > +       q_data = mtk_venc_get_q_data(ctx, f->type);
> > +
> > +       pix->width = q_data->width;
> > +       pix->height = q_data->height;
> > +       pix->pixelformat = q_data->fmt->fourcc;
> > +       pix->field = q_data->field;
> > +       pix->colorspace = q_data->colorspace;
> > +       pix->num_planes = q_data->fmt->num_planes;
> > +       for (i = 0; i < pix->num_planes; i++) {
> > +               pix->plane_fmt[i].bytesperline = q_data->bytesperline[i];
> > +               pix->plane_fmt[i].sizeimage = q_data->sizeimage[i];
> > +               memset(&(pix->plane_fmt[i].reserved[0]), 0x0,
> > +                       sizeof(pix->plane_fmt[i].reserved));
> > +       }
> > +       pix->flags = 0;
> > +       pix->ycbcr_enc = 0;
> > +       pix->quantization = 0;
> > +       pix->xfer_func = 0;
> > +       memset(&pix->reserved[0], 0x0, sizeof(pix->reserved));
> > +
> > +       return 0;
> > +}
> > +
> > +static int vidioc_try_fmt_vid_cap_mplane(struct file *file, void *priv,
> > +                                  struct v4l2_format *f)
> > +{
> > +        struct mtk_video_fmt *fmt;
> > +       struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
> > +
> > +        fmt = mtk_venc_find_format(f);
> > +        if (!fmt) {
> > +                f->fmt.pix.pixelformat = mtk_video_formats[8].fourcc;
> > +                fmt = mtk_venc_find_format(f);
> > +        }
> > +        if (fmt->type != MTK_FMT_ENC) {
> > +               mtk_v4l2_err("Fourcc format (0x%08x) invalid.\n",
> > +                            f->fmt.pix.pixelformat);
> > +               return -EINVAL;
> > +        }
> > +        f->fmt.pix_mp.colorspace = ctx->q_data[MTK_Q_DATA_SRC].colorspace;
> > +
> > +        return vidioc_try_fmt(f, fmt);
> > +}
> > +
> > +static int vidioc_try_fmt_vid_out_mplane(struct file *file, void *priv,
> > +                                  struct v4l2_format *f)
> > +{
> > +        struct mtk_video_fmt *fmt;
> > +
> > +        fmt = mtk_venc_find_format(f);
> > +        if (!fmt) {
> > +                f->fmt.pix.pixelformat = mtk_video_formats[0].fourcc;
> > +                fmt = mtk_venc_find_format(f);
> > +        }
> > +        if (!(fmt->type & MTK_FMT_FRAME)) {
> > +               mtk_v4l2_err("Fourcc format (0x%08x) invalid.\n",
> > +                            f->fmt.pix.pixelformat);
> > +               return -EINVAL;
> > +        }
> > +        if (!f->fmt.pix_mp.colorspace)
> > +                f->fmt.pix_mp.colorspace = V4L2_COLORSPACE_REC709;
> > +
> > +        return vidioc_try_fmt(f, fmt);
> > +}
> > +
> > +static int vidioc_venc_g_s_selection(struct file *file, void *priv,
> > +                                struct v4l2_selection *s)
> > +{
> > +       struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
> > +       struct mtk_q_data *q_data;
> > +
> > +       if (V4L2_TYPE_IS_OUTPUT(s->type)) {
> > +               if (s->target !=  V4L2_SEL_TGT_COMPOSE)
> > +                       return -EINVAL;
> > +       } else {
> > +               if (s->target != V4L2_SEL_TGT_CROP)
> > +                       return -EINVAL;
> > +       }
> > +
> > +       if (s->r.left || s->r.top)
> > +               return -EINVAL;
> > +
> > +       q_data = mtk_venc_get_q_data(ctx, s->type);
> > +       if (!q_data)
> > +               return -EINVAL;
> > +
> > +       s->r.width = q_data->width;
> > +       s->r.height = q_data->height;
> > +
> > +       return 0;
> > +}
> > +
> > +
> > +static int vidioc_venc_qbuf(struct file *file, void *priv,
> > +                           struct v4l2_buffer *buf)
> > +{
> > +
> > +       struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
> > +
> > +       if (ctx->state == MTK_STATE_ABORT) {
> > +               mtk_v4l2_err("[%d] Call on QBUF after unrecoverable error\n", ctx->idx);
> > +               return -EIO;
> > +       }
> > +
> > +       return v4l2_m2m_qbuf(file, ctx->m2m_ctx, buf);
> > +}
> > +
> > +static int vidioc_venc_dqbuf(struct file *file, void *priv,
> > +                            struct v4l2_buffer *buf)
> > +{
> > +       struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
> > +       if (ctx->state == MTK_STATE_ABORT) {
> > +               mtk_v4l2_err("[%d] Call on QBUF after unrecoverable error\n", ctx->idx);
> > +               return -EIO;
> > +       }
> > +
> > +       return v4l2_m2m_dqbuf(file, ctx->m2m_ctx, buf);
> > +}
> > +
> > +
> > +const struct v4l2_ioctl_ops mtk_venc_ioctl_ops = {
> > +       .vidioc_streamon                = v4l2_m2m_ioctl_streamon,
> > +       .vidioc_streamoff               = v4l2_m2m_ioctl_streamoff,
> > +
> > +       .vidioc_reqbufs                 = v4l2_m2m_ioctl_reqbufs,
> > +       .vidioc_querybuf                = v4l2_m2m_ioctl_querybuf,
> > +       .vidioc_qbuf                    = vidioc_venc_qbuf,
> > +       .vidioc_dqbuf                   = vidioc_venc_dqbuf,
> > +
> > +       .vidioc_querycap                = vidioc_venc_querycap,
> > +       .vidioc_enum_fmt_vid_cap_mplane = vidioc_enum_fmt_vid_cap_mplane,
> > +       .vidioc_enum_fmt_vid_out_mplane = vidioc_enum_fmt_vid_out_mplane,
> > +       .vidioc_enum_framesizes         = vidioc_enum_framesizes,
> > +
> > +       .vidioc_try_fmt_vid_cap_mplane  = vidioc_try_fmt_vid_cap_mplane,
> > +       .vidioc_try_fmt_vid_out_mplane  = vidioc_try_fmt_vid_out_mplane,
> > +       .vidioc_expbuf                  = v4l2_m2m_ioctl_expbuf,
> > +       .vidioc_subscribe_event         = v4l2_ctrl_subscribe_event,
> > +       .vidioc_unsubscribe_event       = v4l2_event_unsubscribe,
> > +
> > +       .vidioc_s_parm                  = vidioc_venc_s_parm,
> > +
> > +       .vidioc_s_fmt_vid_cap_mplane    = vidioc_venc_s_fmt,
> > +       .vidioc_s_fmt_vid_out_mplane    = vidioc_venc_s_fmt,
> > +
> > +       .vidioc_g_fmt_vid_cap_mplane    = vidioc_venc_g_fmt,
> > +       .vidioc_g_fmt_vid_out_mplane    = vidioc_venc_g_fmt,
> > +
> > +       .vidioc_g_selection             = vidioc_venc_g_s_selection,
> > +       .vidioc_s_selection             = vidioc_venc_g_s_selection,
> > +};
> > +
> > +static int vb2ops_venc_queue_setup(struct vb2_queue *vq,
> > +                                  const void *parg,
> > +                                  unsigned int *nbuffers,
> > +                                  unsigned int *nplanes,
> > +                                  unsigned int sizes[], void *alloc_ctxs[])
> > +{
> > +       struct mtk_vcodec_ctx *ctx = vb2_get_drv_priv(vq);
> > +       struct mtk_q_data *q_data;
> > +
> > +       q_data = mtk_venc_get_q_data(ctx, vq->type);
> > +
> > +       if (*nbuffers < 1)
> > +               *nbuffers = 1;
> > +       if (*nbuffers > MTK_VIDEO_MAX_FRAME)
> > +               *nbuffers = MTK_VIDEO_MAX_FRAME;
> > +
> > +       *nplanes = q_data->fmt->num_planes;
> > +
> > +       if (vq->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
> > +               unsigned int i;
> > +
> > +               for (i = 0; i < *nplanes; i++) {
> > +                       sizes[i] = q_data->sizeimage[i];
> > +                       alloc_ctxs[i] = ctx->dev->alloc_ctx;
> > +               }
> > +       } else if (vq->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
> > +               sizes[0] = q_data->sizeimage[0];
> > +               alloc_ctxs[0] = ctx->dev->alloc_ctx;
> > +       } else {
> > +               return -EINVAL;
> > +       }
> > +
> > +       mtk_v4l2_debug(2, "[%d]get %d buffer(s) of size 0x%x each, vq->memory=%d",
> > +                      ctx->idx, *nbuffers, sizes[0], vq->memory);
> > +
> > +       return 0;
> > +}
> > +
> > +static int vb2ops_venc_buf_prepare(struct vb2_buffer *vb)
> > +{
> > +       struct mtk_vcodec_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue);
> > +       struct mtk_q_data *q_data;
> > +       int i;
> > +
> > +       q_data = mtk_venc_get_q_data(ctx, vb->vb2_queue->type);
> > +
> > +       for (i = 0; i < q_data->fmt->num_planes; i++) {
> > +               if (vb2_plane_size(vb, i) < q_data->sizeimage[i]) {
> > +                       mtk_v4l2_debug(2, "data will not fit into plane %d (%lu < %d)",
> > +                                      i, vb2_plane_size(vb, i),
> > +                                      q_data->sizeimage[i]);
> > +                       return -EINVAL;
> > +               }
> > +       }
> > +
> > +       return 0;
> > +}
> > +
> > +static void vb2ops_venc_buf_queue(struct vb2_buffer *vb)
> > +{
> > +       struct mtk_vcodec_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue);
> > +       struct vb2_v4l2_buffer *vb2_v4l2 =
> > +                       container_of(vb, struct vb2_v4l2_buffer, vb2_buf);
> > +       struct mtk_video_enc_buf *mtk_buf =
> > +                       container_of(vb2_v4l2, struct mtk_video_enc_buf, vb);
> > +
> > +       if ((vb->vb2_queue->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) &&
> > +               (ctx->param_change != MTK_ENCODE_PARAM_NONE)) {
> > +               mtk_v4l2_debug(1,
> > +                               "[%d] Before id=%d encode parameter change %x",
> > +                               ctx->idx, vb->index,
> > +                               ctx->param_change);
> > +               mtk_buf->param_change = ctx->param_change;
> > +               if (mtk_buf->param_change & MTK_ENCODE_PARAM_BITRATE) {
> > +                       mtk_buf->enc_params.bitrate = ctx->enc_params.bitrate;
> > +                       mtk_v4l2_debug(1, "[%d] idx=%d change param br=%d",
> > +                               ctx->idx,
> > +                               mtk_buf->vb.vb2_buf.index,
> > +                               mtk_buf->enc_params.bitrate);
> > +               }
> > +               if (ctx->param_change & MTK_ENCODE_PARAM_FRAMERATE) {
> > +                       mtk_buf->enc_params.framerate_num =
> > +                               ctx->enc_params.framerate_num;
> > +                       mtk_buf->enc_params.framerate_denom =
> > +                               ctx->enc_params.framerate_denom;
> > +                       mtk_v4l2_debug(1, "[%d] idx=%d, change param fr=%d/%d",
> > +                                       ctx->idx,
> > +                                       mtk_buf->vb.vb2_buf.index,
> > +                                       mtk_buf->enc_params.framerate_num,
> > +                                       mtk_buf->enc_params.framerate_denom);
> > +               }
> > +               if (ctx->param_change & MTK_ENCODE_PARAM_INTRA_PERIOD) {
> > +                       mtk_buf->enc_params.gop_size = ctx->enc_params.gop_size;
> > +                       mtk_v4l2_debug(1, "[%d] idx=%d, change param intra period=%d",
> > +                                       ctx->idx,
> > +                                       mtk_buf->vb.vb2_buf.index,
> > +                                       mtk_buf->enc_params.gop_size);
> > +               }
> > +               if (ctx->param_change & MTK_ENCODE_PARAM_FRAME_TYPE) {
> > +                       mtk_buf->enc_params.force_intra =
> > +                               ctx->enc_params.force_intra;
> > +                       mtk_v4l2_debug(1, "[%d] idx=%d, change param force I=%d",
> > +                                       ctx->idx,
> > +                                       mtk_buf->vb.vb2_buf.index,
> > +                                       mtk_buf->enc_params.force_intra);
> > +               }
> > +               ctx->param_change = MTK_ENCODE_PARAM_NONE;
> > +       }
> > +
> > +       v4l2_m2m_buf_queue(ctx->m2m_ctx, to_vb2_v4l2_buffer(vb));
> > +}
> > +
> > +static int vb2ops_venc_start_streaming(struct vb2_queue *q, unsigned int count)
> > +{
> > +       struct mtk_vcodec_ctx *ctx = vb2_get_drv_priv(q);
> > +       struct venc_enc_prm param;
> > +       int ret;
> > +       int i;
> > +
> > +       /* Once state turn into MTK_STATE_ABORT, we need stop_streaming to clear it */
> > +       if ((ctx->state == MTK_STATE_ABORT) || (ctx->state == MTK_STATE_FREE))
> > +               goto err_set_param;
> > +
> > +       if (!(vb2_start_streaming_called(&ctx->m2m_ctx->out_q_ctx.q) &
> > +             vb2_start_streaming_called(&ctx->m2m_ctx->cap_q_ctx.q))) {
> > +               mtk_v4l2_debug(1, "[%d]-> out=%d cap=%d",
> > +                ctx->idx,
> > +                vb2_start_streaming_called(&ctx->m2m_ctx->out_q_ctx.q),
> > +                vb2_start_streaming_called(&ctx->m2m_ctx->cap_q_ctx.q));
> > +               return 0;
> > +       }
> > +
> > +       mtk_venc_set_param(ctx, &param);
> > +       ret = venc_if_set_param(ctx,
> > +                               VENC_SET_PARAM_ENC,
> > +                               &param);
> > +       if (ret) {
> > +               mtk_v4l2_err("venc_if_set_param failed=%d\n", ret);
> > +               ctx->state = MTK_STATE_ABORT;
> > +               goto err_set_param;
> > +       }
> > +
> > +       if ((ctx->q_data[MTK_Q_DATA_DST].fmt->fourcc == V4L2_PIX_FMT_H264) &&
> > +           (ctx->enc_params.seq_hdr_mode != V4L2_MPEG_VIDEO_HEADER_MODE_SEPARATE)) {
> > +               ret = venc_if_set_param(ctx,
> > +                                       VENC_SET_PARAM_PREPEND_HEADER,
> > +                                       0);
> > +               if (ret) {
> > +                       mtk_v4l2_err("venc_if_set_param failed=%d\n", ret);
> > +                       ctx->state = MTK_STATE_ABORT;
> > +                       goto err_set_param;
> > +               }
> > +               ctx->state = MTK_STATE_HEADER;
> > +       }
> > +
> > +       return 0;
> > +
> > +err_set_param:
> > +       for (i = 0; i < q->num_buffers; ++i) {
> > +               if (q->bufs[i]->state == VB2_BUF_STATE_ACTIVE) {
> > +                       mtk_v4l2_debug(0, "[%d] idx=%d, type=%d, %d -> VB2_BUF_STATE_QUEUED",
> > +                                       ctx->idx, i, q->type,
> > +                                       (int)q->bufs[i]->state );
> > +                       v4l2_m2m_buf_done(to_vb2_v4l2_buffer(q->bufs[i]), VB2_BUF_STATE_QUEUED);
> > +               }
> > +       }
> > +
> > +       return -EINVAL;
> > +}
> > +
> > +static void vb2ops_venc_stop_streaming(struct vb2_queue *q)
> > +{
> > +       struct mtk_vcodec_ctx *ctx = vb2_get_drv_priv(q);
> > +       struct vb2_buffer *src_buf, *dst_buf;
> > +       int ret;
> > +
> > +       mtk_v4l2_debug(2, "[%d]-> type=%d", ctx->idx, q->type);
> > +
> > +       if (q->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
> > +               while ((dst_buf = v4l2_m2m_dst_buf_remove(ctx->m2m_ctx))) {
> > +                       dst_buf->planes[0].bytesused = 0;
> > +                       v4l2_m2m_buf_done(to_vb2_v4l2_buffer(dst_buf),
> > +                                               VB2_BUF_STATE_ERROR);
> > +               }
> > +       } else {
> > +               while ((src_buf = v4l2_m2m_src_buf_remove(ctx->m2m_ctx)))
> > +                       v4l2_m2m_buf_done(to_vb2_v4l2_buffer(src_buf),
> > +                                               VB2_BUF_STATE_ERROR);
> > +       }
> > +
> > +       if ((q->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE &&
> > +            vb2_is_streaming(&ctx->m2m_ctx->out_q_ctx.q)) ||
> > +           (q->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE &&
> > +            vb2_is_streaming(&ctx->m2m_ctx->cap_q_ctx.q))) {
> > +               mtk_v4l2_debug(1, "[%d]-> q type %d out=%d cap=%d",
> > +                              ctx->idx, q->type,
> > +                              vb2_is_streaming(&ctx->m2m_ctx->out_q_ctx.q),
> > +                              vb2_is_streaming(&ctx->m2m_ctx->cap_q_ctx.q));
> > +               return;
> > +       }
> > +
> > +       ret = venc_if_release(ctx);
> > +       if (ret)
> > +               mtk_v4l2_err("venc_if_release failed=%d\n", ret);
> > +
> > +       ctx->state = MTK_STATE_FREE;
> > +}
> > +
> > +static struct vb2_ops mtk_venc_vb2_ops = {
> > +       .queue_setup                    = vb2ops_venc_queue_setup,
> > +       .buf_prepare                    = vb2ops_venc_buf_prepare,
> > +       .buf_queue                      = vb2ops_venc_buf_queue,
> > +       .wait_prepare                   = vb2_ops_wait_prepare,
> > +       .wait_finish                    = vb2_ops_wait_finish,
> > +       .start_streaming                = vb2ops_venc_start_streaming,
> > +       .stop_streaming                 = vb2ops_venc_stop_streaming,
> > +};
> > +
> > +static int mtk_venc_encode_header(void *priv)
> > +{
> > +       struct mtk_vcodec_ctx *ctx = priv;
> > +       int ret;
> > +       struct vb2_buffer *dst_buf;
> > +       struct mtk_vcodec_mem bs_buf;
> > +       struct venc_done_result enc_result;
> > +
> > +       dst_buf = v4l2_m2m_dst_buf_remove(ctx->m2m_ctx);
> > +       if (!dst_buf) {
> > +               mtk_v4l2_debug(1, "No dst buffer");
> > +               return -EINVAL;
> > +       }
> > +
> > +       bs_buf.va = vb2_plane_vaddr(dst_buf, 0);
> > +       bs_buf.dma_addr = vb2_dma_contig_plane_dma_addr(dst_buf, 0);
> > +       bs_buf.size = (unsigned int)dst_buf->planes[0].length;
> > +
> > +       mtk_v4l2_debug(1,
> > +                       "[%d] buf idx=%d va=0x%p dma_addr=0x%llx size=0x%lx",
> > +                       ctx->idx,
> > +                       dst_buf->index, bs_buf.va,
> > +                       (u64)bs_buf.dma_addr,
> > +                       bs_buf.size);
> > +
> > +       ret = venc_if_encode(ctx,
> > +                       VENC_START_OPT_ENCODE_SEQUENCE_HEADER,
> > +                       0, &bs_buf, &enc_result);
> > +
> > +       if (ret) {
> > +               dst_buf->planes[0].bytesused = 0;
> > +               ctx->state = MTK_STATE_ABORT;
> > +               v4l2_m2m_buf_done(to_vb2_v4l2_buffer(dst_buf), VB2_BUF_STATE_ERROR);
> > +               mtk_v4l2_err("venc_if_encode failed=%d", ret);
> > +               return -EINVAL;
> > +       }
> > +
> > +       ctx->state = MTK_STATE_HEADER;
> > +       dst_buf->planes[0].bytesused = enc_result.bs_size;
> > +
> > +#if defined(DEBUG)
> > +{
> > +       int i;
> > +       mtk_v4l2_debug(1, "[%d] venc_if_encode header len=%d",
> > +                       ctx->idx,
> > +                       enc_result.bs_size);
> > +       for (i = 0; i < enc_result.bs_size; i++) {
> > +               unsigned char *p = (unsigned char *)bs_buf.va;
> > +
> > +               mtk_v4l2_debug(1, "[%d] buf[%d]=0x%2x", ctx->idx, i, p[i]);
> > +       }
> > +}
> > +#endif
> > +       v4l2_m2m_buf_done(to_vb2_v4l2_buffer(dst_buf), VB2_BUF_STATE_DONE);
> > +
> > +       return 0;
> > +}
> > +
> > +static int mtk_venc_param_change(struct mtk_vcodec_ctx *ctx, void *priv)
> > +{
> > +       struct vb2_buffer *vb = priv;
> > +       struct vb2_v4l2_buffer *vb2_v4l2 =
> > +                       container_of(vb, struct vb2_v4l2_buffer, vb2_buf);
> > +       struct mtk_video_enc_buf *mtk_buf =
> > +                       container_of(vb2_v4l2, struct mtk_video_enc_buf, vb);
> > +       int ret = 0;
> > +
> > +       if (mtk_buf->param_change == MTK_ENCODE_PARAM_NONE)
> > +               return 0;
> > +
> > +       mtk_v4l2_debug(1, "encode parameters change id=%d", vb->index);
> > +       if (mtk_buf->param_change & MTK_ENCODE_PARAM_BITRATE) {
> > +               struct venc_enc_prm enc_prm;
> > +
> > +               enc_prm.bitrate = mtk_buf->enc_params.bitrate;
> > +               mtk_v4l2_debug(1, "[%d] idx=%d, change param br=%d",
> > +                               ctx->idx,
> > +                               mtk_buf->vb.vb2_buf.index,
> > +                               enc_prm.bitrate);
> > +               ret |= venc_if_set_param(ctx,
> > +                                        VENC_SET_PARAM_ADJUST_BITRATE,
> > +                                        &enc_prm);
> > +       }
> > +       if (mtk_buf->param_change & MTK_ENCODE_PARAM_FRAMERATE) {
> > +               struct venc_enc_prm enc_prm;
> > +
> > +               enc_prm.frm_rate = mtk_buf->enc_params.framerate_num /
> > +                                  mtk_buf->enc_params.framerate_denom;
> > +               mtk_v4l2_debug(1, "[%d] idx=%d, change param fr=%d",
> > +                              ctx->idx,
> > +                              mtk_buf->vb.vb2_buf.index,
> > +                              enc_prm.frm_rate);
> > +               ret |= venc_if_set_param(ctx,
> > +                                        VENC_SET_PARAM_ADJUST_FRAMERATE,
> > +                                        &enc_prm);
> > +       }
> > +       if (mtk_buf->param_change & MTK_ENCODE_PARAM_INTRA_PERIOD) {
> > +               mtk_v4l2_debug(1, "change param intra period=%d",
> > +                                mtk_buf->enc_params.gop_size);
> > +               ret |= venc_if_set_param(ctx,
> > +                                        VENC_SET_PARAM_I_FRAME_INTERVAL,
> > +                                        &mtk_buf->enc_params.gop_size);
> > +       }
> > +       if (mtk_buf->param_change & MTK_ENCODE_PARAM_FRAME_TYPE) {
> > +               mtk_v4l2_debug(1, "[%d] idx=%d, change param force I=%d",
> > +                               ctx->idx,
> > +                               mtk_buf->vb.vb2_buf.index,
> > +                               mtk_buf->enc_params.force_intra);
> > +               if (mtk_buf->enc_params.force_intra)
> > +                       ret |= venc_if_set_param(ctx,
> > +                                                VENC_SET_PARAM_FORCE_INTRA,
> > +                                                0);
> > +       }
> > +
> > +       mtk_buf->param_change = MTK_ENCODE_PARAM_NONE;
> > +
> > +       if (ret) {
> > +               ctx->state = MTK_STATE_ABORT;
> > +               mtk_v4l2_err("venc_if_set_param %d failed=%d\n",
> > +                       MTK_ENCODE_PARAM_FRAME_TYPE, ret);
> > +               return -1;
> > +       }
> > +
> > +       return 0;
> > +}
> > +
> > +static void mtk_venc_worker(struct work_struct *work)
> > +{
> > +       struct mtk_vcodec_ctx *ctx = container_of(work, struct mtk_vcodec_ctx,
> > +                                   encode_work);
> > +       struct vb2_buffer *src_buf, *dst_buf;
> > +       struct venc_frm_buf frm_buf;
> > +       struct mtk_vcodec_mem bs_buf;
> > +       struct venc_done_result enc_result;
> > +       int ret;
> > +       struct vb2_v4l2_buffer *v4l2_vb;
> > +
> > +       if ((ctx->q_data[MTK_Q_DATA_DST].fmt->fourcc == V4L2_PIX_FMT_H264) &&
> > +           (ctx->state != MTK_STATE_HEADER)) {
> > +               /* encode h264 sps/pps header */
> > +               mtk_venc_encode_header(ctx);
> > +               v4l2_m2m_job_finish(ctx->dev->m2m_dev_enc, ctx->m2m_ctx);
> > +               return;
> > +       }
> > +
> > +       src_buf = v4l2_m2m_next_src_buf(ctx->m2m_ctx);
> > +       if (!src_buf) {
> > +               v4l2_m2m_job_finish(ctx->dev->m2m_dev_enc, ctx->m2m_ctx);
> > +               return;
> > +       }
> > +
> > +       dst_buf = v4l2_m2m_dst_buf_remove(ctx->m2m_ctx);
> > +       if (!dst_buf) {
> > +               v4l2_m2m_job_finish(ctx->dev->m2m_dev_enc, ctx->m2m_ctx);
> > +               return;
> > +       }
> > +
> > +       mtk_venc_param_change(ctx, src_buf);
> > +
> > +       frm_buf.fb_addr.va = vb2_plane_vaddr(src_buf, 0);
> > +       frm_buf.fb_addr.dma_addr = vb2_dma_contig_plane_dma_addr(src_buf, 0);
> > +       frm_buf.fb_addr.size = (unsigned int)src_buf->planes[0].length;
> > +       frm_buf.fb_addr1.va = vb2_plane_vaddr(src_buf, 1);
> > +       frm_buf.fb_addr1.dma_addr = vb2_dma_contig_plane_dma_addr(src_buf, 1);
> > +       frm_buf.fb_addr1.size = (unsigned int)src_buf->planes[1].length;
> > +       if (src_buf->num_planes == 3) {
> > +               frm_buf.fb_addr2.va = vb2_plane_vaddr(src_buf, 2);
> > +               frm_buf.fb_addr2.dma_addr =
> > +                       vb2_dma_contig_plane_dma_addr(src_buf, 2);
> > +               frm_buf.fb_addr2.size =
> > +                       (unsigned int)src_buf->planes[2].length;
> > +       } else {
> > +               frm_buf.fb_addr2.va = NULL;
> > +               frm_buf.fb_addr2.dma_addr = 0;
> > +               frm_buf.fb_addr2.size = 0;
> > +       }
> > +       bs_buf.va = vb2_plane_vaddr(dst_buf, 0);
> > +       bs_buf.dma_addr = vb2_dma_contig_plane_dma_addr(dst_buf, 0);
> > +       bs_buf.size = (unsigned int)dst_buf->planes[0].length;
> > +
> > +       mtk_v4l2_debug(2,
> > +                       "Framebuf VA=%p PA=%llx Size=0x%lx;VA=%p PA=0x%llx Size=0x%lx;VA=%p PA=0x%llx Size=0x%lx",
> > +                       frm_buf.fb_addr.va,
> > +                       (u64)frm_buf.fb_addr.dma_addr,
> > +                       frm_buf.fb_addr.size,
> > +                       frm_buf.fb_addr1.va,
> > +                       (u64)frm_buf.fb_addr1.dma_addr,
> > +                       frm_buf.fb_addr1.size,
> > +                       frm_buf.fb_addr2.va,
> > +                       (u64)frm_buf.fb_addr2.dma_addr,
> > +                       frm_buf.fb_addr2.size);
> > +
> > +       ret = venc_if_encode(ctx, VENC_START_OPT_ENCODE_FRAME,
> > +                            &frm_buf, &bs_buf, &enc_result);
> > +
> > +       src_buf = v4l2_m2m_src_buf_remove(ctx->m2m_ctx);
> > +       if (enc_result.msg == VENC_MESSAGE_OK)
> > +               v4l2_m2m_buf_done(to_vb2_v4l2_buffer(src_buf), VB2_BUF_STATE_DONE);
> > +       else
> > +               v4l2_m2m_buf_done(to_vb2_v4l2_buffer(src_buf), VB2_BUF_STATE_ERROR);
> > +
> > +       if (enc_result.is_key_frm) {
> > +               v4l2_vb = to_vb2_v4l2_buffer(dst_buf);
> > +               v4l2_vb->flags |= V4L2_BUF_FLAG_KEYFRAME;
> > +       }
> > +
> > +       if (ret) {
> > +               dst_buf->planes[0].bytesused = 0;
> > +               v4l2_m2m_buf_done(to_vb2_v4l2_buffer(dst_buf), VB2_BUF_STATE_ERROR);
> > +               mtk_v4l2_err("venc_if_encode failed=%d", ret);
> > +       } else {
> > +               dst_buf->planes[0].bytesused = enc_result.bs_size;
> > +               v4l2_m2m_buf_done(to_vb2_v4l2_buffer(dst_buf), VB2_BUF_STATE_DONE);
> > +               mtk_v4l2_debug(2, "venc_if_encode bs size=%d",
> > +                                enc_result.bs_size);
> > +       }
> > +
> > +       v4l2_m2m_job_finish(ctx->dev->m2m_dev_enc, ctx->m2m_ctx);
> > +
> > +       mtk_v4l2_debug(1, "<=== src_buf[%d] dst_buf[%d] venc_if_encode ret=%d Size=%u===>",
> > +                       src_buf->index, dst_buf->index, ret,
> > +                       enc_result.bs_size);
> > +}
> > +
> > +static void m2mops_venc_device_run(void *priv)
> > +{
> > +       struct mtk_vcodec_ctx *ctx = priv;
> > +       queue_work(ctx->dev->encode_workqueue, &ctx->encode_work);
> > +}
> > +
> > +static int m2mops_venc_job_ready(void *m2m_priv)
> > +{
> > +       struct mtk_vcodec_ctx *ctx = m2m_priv;
> > +
> > +       if (!v4l2_m2m_num_dst_bufs_ready(ctx->m2m_ctx)) {
> > +               mtk_v4l2_debug(3, "[%d]Not ready: not enough video dst buffers.",
> > +                              ctx->idx);
> > +               return 0;
> > +       }
> > +
> > +       if (!v4l2_m2m_num_src_bufs_ready(ctx->m2m_ctx)) {
> > +               mtk_v4l2_debug(3, "[%d]Not ready: not enough video src buffers.",
> > +                              ctx->idx);
> > +                       return 0;
> > +               }
> > +
> > +       if (ctx->state == MTK_STATE_ABORT) {
> > +               mtk_v4l2_debug(3, "[%d]Not ready: state=0x%x.",
> > +                              ctx->idx, ctx->state);
> > +               return 0;
> > +       }
> > +
> > +       if (ctx->state == MTK_STATE_FREE) {
> > +               mtk_v4l2_debug(3, "[%d]Not ready: state=0x%x.",
> > +                              ctx->idx, ctx->state);
> > +               return 0;
> > +       }
> > +
> > +       return 1;
> > +}
> > +
> > +static void m2mops_venc_job_abort(void *priv)
> > +{
> > +       struct mtk_vcodec_ctx *ctx = priv;
> > +
> > +       ctx->state = MTK_STATE_ABORT;
> > +}
> > +
> > +static void m2mops_venc_lock(void *m2m_priv)
> > +{
> > +       struct mtk_vcodec_ctx *ctx = m2m_priv;
> > +
> > +       mutex_lock(&ctx->dev->dev_mutex);
> > +}
> > +
> > +static void m2mops_venc_unlock(void *m2m_priv)
> > +{
> > +       struct mtk_vcodec_ctx *ctx = m2m_priv;
> > +
> > +       mutex_unlock(&ctx->dev->dev_mutex);
> > +}
> > +
> > +const struct v4l2_m2m_ops mtk_venc_m2m_ops = {
> > +       .device_run                     = m2mops_venc_device_run,
> > +       .job_ready                      = m2mops_venc_job_ready,
> > +       .job_abort                      = m2mops_venc_job_abort,
> > +       .lock                           = m2mops_venc_lock,
> > +       .unlock                         = m2mops_venc_unlock,
> > +};
> > +
> > +#define IS_MTK_VENC_PRIV(x) ((V4L2_CTRL_ID2CLASS(x) == V4L2_CTRL_CLASS_MPEG) &&\
> > +                            V4L2_CTRL_DRIVER_PRIV(x))
> > +
> > +void mtk_vcodec_enc_ctx_params_setup(struct mtk_vcodec_ctx *ctx)
> > +{
> > +       struct mtk_q_data *q_data;
> > +       struct mtk_video_fmt *fmt;
> > +
> > +       ctx->m2m_ctx->q_lock = &ctx->dev->dev_mutex;
> > +       ctx->fh.m2m_ctx = ctx->m2m_ctx;
> > +       ctx->fh.ctrl_handler = &ctx->ctrl_hdl;
> > +       INIT_WORK(&ctx->encode_work, mtk_venc_worker);
> > +
> > +       ctx->q_data[MTK_Q_DATA_SRC].width = DFT_CFG_WIDTH;
> > +       ctx->q_data[MTK_Q_DATA_SRC].height = DFT_CFG_HEIGHT;
> > +       ctx->q_data[MTK_Q_DATA_SRC].fmt = &mtk_video_formats[0];
> > +       ctx->q_data[MTK_Q_DATA_SRC].colorspace = V4L2_COLORSPACE_REC709;
> > +       ctx->q_data[MTK_Q_DATA_SRC].field = V4L2_FIELD_NONE;
> > +
> > +       q_data = &ctx->q_data[MTK_Q_DATA_SRC];
> > +       fmt = ctx->q_data[MTK_Q_DATA_SRC].fmt;
> > +       mtk_vcodec_enc_calc_src_size(fmt->num_planes, q_data->width,
> > +                       q_data->height,
> > +                       ctx->q_data[MTK_Q_DATA_SRC].sizeimage,
> > +                       ctx->q_data[MTK_Q_DATA_SRC].bytesperline);
> > +
> > +       ctx->q_data[MTK_Q_DATA_DST].width = DFT_CFG_WIDTH;
> > +       ctx->q_data[MTK_Q_DATA_DST].height = DFT_CFG_HEIGHT;
> > +       ctx->q_data[MTK_Q_DATA_DST].fmt = &mtk_video_formats[9];
> > +       ctx->q_data[MTK_Q_DATA_DST].colorspace = V4L2_COLORSPACE_REC709;
> > +       ctx->q_data[MTK_Q_DATA_DST].field = V4L2_FIELD_NONE;
> > +
> > +       q_data = &ctx->q_data[MTK_Q_DATA_DST];
> > +       fmt = ctx->q_data[MTK_Q_DATA_DST].fmt;
> > +       ctx->q_data[MTK_Q_DATA_DST].sizeimage[0] = q_data->width * q_data->height;
> > +       ctx->q_data[MTK_Q_DATA_DST].bytesperline[0] = 0;
> > +
> > +}
> > +
> > +int mtk_vcodec_enc_ctrls_setup(struct mtk_vcodec_ctx *ctx)
> > +{
> > +       const struct v4l2_ctrl_ops *ops = &mtk_vcodec_enc_ctrl_ops;
> > +       struct v4l2_ctrl_handler *handler = &ctx->ctrl_hdl;
> > +       struct v4l2_ctrl_config cfg;
> > +
> > +       v4l2_ctrl_handler_init(handler, MTK_MAX_CTRLS);
> > +       if (handler->error) {
> > +               mtk_v4l2_err("Init control handler fail %d\n",
> > +                               handler->error);
> > +               return handler->error;
> > +       }
> > +
> > +       ctx->ctrls[0] = v4l2_ctrl_new_std(handler, ops,
> > +                                       V4L2_CID_MPEG_VIDEO_BITRATE,
> > +                                       1, 4000000, 1, 4000000);
> > +       ctx->ctrls[1] = v4l2_ctrl_new_std(handler, ops,
> > +                                       V4L2_CID_MPEG_VIDEO_B_FRAMES,
> > +                                       0, 2, 1, 0);
> > +       ctx->ctrls[2] = v4l2_ctrl_new_std(handler, ops,
> > +                                       V4L2_CID_MPEG_VIDEO_FRAME_RC_ENABLE,
> > +                                       0, 1, 1, 1);
> > +       ctx->ctrls[3] = v4l2_ctrl_new_std(handler, ops,
> > +                                       V4L2_CID_MPEG_VIDEO_H264_MAX_QP,
> > +                                       0, 51, 1, 51);
> > +       ctx->ctrls[4] = v4l2_ctrl_new_std(handler, ops,
> > +                                       V4L2_CID_MPEG_VIDEO_H264_I_PERIOD,
> > +                                       0, 65535, 1, 30);
> > +       ctx->ctrls[5] = v4l2_ctrl_new_std(handler, ops,
> > +                                       V4L2_CID_MPEG_VIDEO_GOP_SIZE,
> > +                                       0, 65535, 1, 30);
> > +       ctx->ctrls[6] = v4l2_ctrl_new_std(handler, ops,
> > +                                       V4L2_CID_MPEG_VIDEO_MB_RC_ENABLE,
> > +                                       0, 1, 1, 0);
> > +       ctx->ctrls[7] = v4l2_ctrl_new_std_menu(handler, ops,
> > +                                       V4L2_CID_MPEG_VIDEO_HEADER_MODE,
> > +                                       V4L2_MPEG_VIDEO_HEADER_MODE_JOINED_WITH_1ST_FRAME,
> > +                                       0, V4L2_MPEG_VIDEO_HEADER_MODE_SEPARATE);
> > +       ctx->ctrls[8] = v4l2_ctrl_new_std_menu(handler, ops,
> > +                                       V4L2_CID_MPEG_VIDEO_H264_PROFILE,
> > +                                       V4L2_MPEG_VIDEO_H264_PROFILE_HIGH,
> > +                                       0, V4L2_MPEG_VIDEO_H264_PROFILE_MAIN);
> > +       ctx->ctrls[9] = v4l2_ctrl_new_std_menu(handler, ops,
> > +                                       V4L2_CID_MPEG_VIDEO_H264_LEVEL,
> > +                                       V4L2_MPEG_VIDEO_H264_LEVEL_4_2,
> > +                                       0, V4L2_MPEG_VIDEO_H264_LEVEL_4_0);
> > +       ctx->ctrls[6] = v4l2_ctrl_new_std(handler, ops,
> > +                                       V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME,
> > +                                       0, 0, 0, 0);
> > +
> > +       v4l2_ctrl_handler_setup(&ctx->ctrl_hdl);
> > +
> > +       return 0;
> > +}
> > +
> > +int mtk_vcodec_enc_queue_init(void *priv, struct vb2_queue *src_vq,
> > +                          struct vb2_queue *dst_vq)
> > +{
> > +       struct mtk_vcodec_ctx *ctx = priv;
> > +       int ret;
> > +
> > +       src_vq->type            = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE;
> > +       src_vq->io_modes        = VB2_DMABUF | VB2_MMAP | VB2_USERPTR;
> > +       src_vq->drv_priv        = ctx;
> > +       src_vq->buf_struct_size = sizeof(struct mtk_video_enc_buf);
> > +       src_vq->ops             = &mtk_venc_vb2_ops;
> > +       src_vq->mem_ops         = &vb2_dma_contig_memops;
> > +       src_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
> > +       src_vq->lock = &ctx->dev->dev_mutex;
> > +
> > +       ret = vb2_queue_init(src_vq);
> > +       if (ret)
> > +               return ret;
> > +
> > +       dst_vq->type            = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
> > +       dst_vq->io_modes        = VB2_DMABUF | VB2_MMAP | VB2_USERPTR;
> > +       dst_vq->drv_priv        = ctx;
> > +       dst_vq->buf_struct_size = sizeof(struct v4l2_m2m_buffer);
> > +       dst_vq->ops             = &mtk_venc_vb2_ops;
> > +       dst_vq->mem_ops         = &vb2_dma_contig_memops;
> > +       dst_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
> > +       dst_vq->lock = &ctx->dev->dev_mutex;
> > +
> > +       return vb2_queue_init(dst_vq);
> > +}
> > +
> > +int mtk_venc_unlock(struct mtk_vcodec_ctx *ctx)
> > +{
> > +       struct mtk_vcodec_dev *dev = ctx->dev;
> > +       dev->curr_ctx = -1;
> > +       mutex_unlock(&dev->enc_mutex);
> > +       return 0;
> > +}
> > +
> > +int mtk_venc_lock(struct mtk_vcodec_ctx *ctx)
> > +{
> > +       struct mtk_vcodec_dev *dev = ctx->dev;
> > +
> > +       mutex_lock(&dev->enc_mutex);
> > +       dev->curr_ctx = ctx->idx;
> > +       return 0;
> > +}
> > +
> > +void mtk_vcodec_enc_release(struct mtk_vcodec_ctx *ctx)
> > +{
> > +       venc_if_release(ctx);
> > +}
> > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h
> > new file mode 100644
> > index 0000000..e09524b
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h
> > @@ -0,0 +1,46 @@
> > +/*
> > +* Copyright (c) 2015 MediaTek Inc.
> > +* Author: PC Chen <pc.chen-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
> > +*         Tiffany Lin <tiffany.lin-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
> > +*
> > +* This program is free software; you can redistribute it and/or modify
> > +* it under the terms of the GNU General Public License version 2 as
> > +* published by the Free Software Foundation.
> > +*
> > +* This program is distributed in the hope that it will be useful,
> > +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> > +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > +* GNU General Public License for more details.
> > +*/
> > +
> > +
> > +#ifndef _MTK_VCODEC_ENC_H_
> > +#define _MTK_VCODEC_ENC_H_
> > +
> > +#include <media/videobuf2-core.h>
> > +#include <media/videobuf2-v4l2.h>
> > +
> > +/**
> > + * struct mtk_video_enc_buf - Private data related to each VB2 buffer.
> > + * @b:                 Pointer to related VB2 buffer.
> > + * @param_change:      Types of encode parameter change before encode this
> > + *                     buffer
> > + * @enc_params         Encode parameters changed before encode this buffer
> > + */
> > +struct mtk_video_enc_buf {
> > +       struct vb2_v4l2_buffer  vb;
> > +       struct list_head        list;
> > +
> > +       enum mtk_encode_param param_change;
> > +       struct mtk_enc_params enc_params;
> > +};
> > +
> > +int mtk_venc_unlock(struct mtk_vcodec_ctx *ctx);
> > +int mtk_venc_lock(struct mtk_vcodec_ctx *ctx);
> > +int mtk_vcodec_enc_queue_init(void *priv, struct vb2_queue *src_vq,
> > +                                       struct vb2_queue *dst_vq);
> > +void mtk_vcodec_enc_release(struct mtk_vcodec_ctx *ctx);
> > +int mtk_vcodec_enc_ctrls_setup(struct mtk_vcodec_ctx *ctx);
> > +void mtk_vcodec_enc_ctx_params_setup(struct mtk_vcodec_ctx *ctx);
> > +
> > +#endif /* _MTK_VCODEC_ENC_H_ */
> > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
> > new file mode 100644
> > index 0000000..e7ab14a
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
> > @@ -0,0 +1,476 @@
> > +/*
> > +* Copyright (c) 2015 MediaTek Inc.
> > +* Author: PC Chen <pc.chen-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
> > +*         Tiffany Lin <tiffany.lin-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
> > +*
> > +* This program is free software; you can redistribute it and/or modify
> > +* it under the terms of the GNU General Public License version 2 as
> > +* published by the Free Software Foundation.
> > +*
> > +* This program is distributed in the hope that it will be useful,
> > +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> > +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > +* GNU General Public License for more details.
> > +*/
> > +
> > +#include <linux/interrupt.h>
> > +#include <linux/irq.h>
> > +#include <linux/module.h>
> > +#include <linux/of_device.h>
> > +#include <linux/of.h>
> > +#include <media/v4l2-event.h>
> > +#include <media/v4l2-mem2mem.h>
> > +#include <media/videobuf2-dma-contig.h>
> > +#include <linux/pm_runtime.h>
> > +
> > +#include "mtk_vcodec_drv.h"
> > +#include "mtk_vcodec_enc.h"
> > +#include "mtk_vcodec_pm.h"
> > +#include "mtk_vcodec_intr.h"
> > +#include "mtk_vcodec_util.h"
> > +#include "mtk_vpu.h"
> > +
> > +
> > +/* Wake up context wait_queue */
> > +static void wake_up_ctx(struct mtk_vcodec_ctx *ctx, unsigned int reason)
> > +{
> > +       ctx->int_cond = 1;
> > +       ctx->int_type = reason;
> > +       wake_up_interruptible(&ctx->queue);
> > +}
> > +
> > +static irqreturn_t mtk_vcodec_enc_irq_handler(int irq, void *priv)
> > +{
> > +       struct mtk_vcodec_dev *dev = priv;
> > +       struct mtk_vcodec_ctx *ctx;
> > +       unsigned int irq_status;
> > +
> > +       if (dev->curr_ctx == -1) {
> > +               mtk_v4l2_err("curr_ctx = -1");
> > +               return IRQ_HANDLED;
> > +       }
> > +
> > +       ctx = dev->ctx[dev->curr_ctx];
> > +       if (ctx == NULL) {
> > +               mtk_v4l2_err("curr_ctx==NULL");
> > +               return IRQ_HANDLED;
> > +       }
> > +       mtk_v4l2_debug(1, "idx=%d", ctx->idx);
> > +       irq_status = readl(dev->reg_base[VENC_SYS] +
> > +                               (MTK_VENC_IRQ_STATUS_OFFSET));
> > +       if (irq_status & MTK_VENC_IRQ_STATUS_PAUSE)
> > +               writel((MTK_VENC_IRQ_STATUS_PAUSE),
> > +                      dev->reg_base[VENC_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
> > +
> > +       if (irq_status & MTK_VENC_IRQ_STATUS_SWITCH)
> > +               writel((MTK_VENC_IRQ_STATUS_SWITCH),
> > +                      dev->reg_base[VENC_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
> > +
> > +       if (irq_status & MTK_VENC_IRQ_STATUS_DRAM)
> > +               writel((MTK_VENC_IRQ_STATUS_DRAM),
> > +                      dev->reg_base[VENC_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
> > +
> > +       if (irq_status & MTK_VENC_IRQ_STATUS_SPS)
> > +               writel((MTK_VENC_IRQ_STATUS_SPS),
> > +                      dev->reg_base[VENC_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
> > +
> > +       if (irq_status & MTK_VENC_IRQ_STATUS_PPS)
> > +               writel((MTK_VENC_IRQ_STATUS_PPS),
> > +                      dev->reg_base[VENC_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
> > +
> > +       if (irq_status & MTK_VENC_IRQ_STATUS_FRM)
> > +               writel((MTK_VENC_IRQ_STATUS_FRM),
> > +                      dev->reg_base[VENC_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
> > +
> > +       ctx->irq_status = irq_status;
> > +       wake_up_ctx(ctx, MTK_INST_IRQ_RECEIVED);
> > +       return IRQ_HANDLED;
> > +}
> > +
> > +#if 1 /* VENC_LT */
> > +static irqreturn_t mtk_vcodec_enc_irq_handler2(int irq, void *priv)
> > +{
> > +       struct mtk_vcodec_dev *dev = priv;
> > +       struct mtk_vcodec_ctx *ctx;
> > +       unsigned int irq_status;
> > +
> > +       ctx = dev->ctx[dev->curr_ctx];
> > +       if (ctx == NULL) {
> > +               mtk_v4l2_err("ctx==NULL");
> > +               return IRQ_HANDLED;
> > +       }
> > +       mtk_v4l2_debug(1, "idx=%d", ctx->idx);
> > +       irq_status = readl(dev->reg_base[VENC_LT_SYS] +
> > +                               (MTK_VENC_IRQ_STATUS_OFFSET));
> > +       if (irq_status & MTK_VENC_IRQ_STATUS_PAUSE)
> > +               writel((MTK_VENC_IRQ_STATUS_PAUSE),
> > +                      dev->reg_base[VENC_LT_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
> > +
> > +       if (irq_status & MTK_VENC_IRQ_STATUS_SWITCH)
> > +               writel((MTK_VENC_IRQ_STATUS_SWITCH),
> > +                      dev->reg_base[VENC_LT_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
> > +
> > +       if (irq_status & MTK_VENC_IRQ_STATUS_DRAM)
> > +               writel((MTK_VENC_IRQ_STATUS_DRAM),
> > +                      dev->reg_base[VENC_LT_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
> > +
> > +       if (irq_status & MTK_VENC_IRQ_STATUS_SPS)
> > +               writel((MTK_VENC_IRQ_STATUS_SPS),
> > +                      dev->reg_base[VENC_LT_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
> > +
> > +       if (irq_status & MTK_VENC_IRQ_STATUS_PPS)
> > +               writel((MTK_VENC_IRQ_STATUS_PPS),
> > +                      dev->reg_base[VENC_LT_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
> > +
> > +       if (irq_status & MTK_VENC_IRQ_STATUS_FRM)
> > +               writel((MTK_VENC_IRQ_STATUS_FRM),
> > +                      dev->reg_base[VENC_LT_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
> > +
> > +       ctx->irq_status = irq_status;
> > +       wake_up_ctx(ctx, MTK_INST_IRQ_RECEIVED);
> > +       return IRQ_HANDLED;
> > +}
> > +#endif
> > +
> > +static void mtk_vcodec_enc_reset_handler(void *priv)
> > +{
> > +       int i;
> > +       struct mtk_vcodec_dev *dev = priv;
> > +       struct mtk_vcodec_ctx *ctx;
> > +
> > +       mtk_v4l2_debug(0, "Watchdog timeout!!");
> > +
> > +       mutex_lock(&dev->dev_mutex);
> > +       for(i = 0; i < MTK_VCODEC_MAX_ENCODER_INSTANCES; i++) {
> > +               ctx = dev->ctx[i];
> > +               if (ctx) {
> > +                       ctx->state = MTK_STATE_ABORT;
> > +                       mtk_v4l2_debug(0, "[%d] Change to state MTK_STATE_ERROR", ctx->idx);
> > +               }
> > +
> > +       }
> > +       mutex_unlock(&dev->dev_mutex);
> > +}
> > +
> > +static int fops_vcodec_open(struct file *file)
> > +{
> > +       struct video_device *vfd = video_devdata(file);
> > +       struct mtk_vcodec_dev *dev = video_drvdata(file);
> > +       struct mtk_vcodec_ctx *ctx = NULL;
> > +       int ret = 0;
> > +
> > +       mutex_lock(&dev->dev_mutex);
> > +
> > +       ctx = devm_kzalloc(&dev->plat_dev->dev, sizeof(*ctx), GFP_KERNEL);
> > +       if (!ctx) {
> > +               ret = -ENOMEM;
> > +               goto err_alloc;
> > +       }
> > +
> > +       if (dev->num_instances >= MTK_VCODEC_MAX_ENCODER_INSTANCES) {
> > +               mtk_v4l2_err("Too many open contexts\n");
> > +               ret = -EBUSY;
> > +               goto err_no_ctx;
> > +       }
> > +
> > +       ctx->idx = ffz(dev->instance_mask[0]);
> > +       v4l2_fh_init(&ctx->fh, video_devdata(file));
> > +       file->private_data = &ctx->fh;
> > +       v4l2_fh_add(&ctx->fh);
> > +       ctx->dev = dev;
> > +
> > +       if (vfd == dev->vfd_enc) {
> > +               ctx->type = MTK_INST_ENCODER;
> > +
> > +               ret = mtk_vcodec_enc_ctrls_setup(ctx);
> > +               if (ret) {
> > +                       mtk_v4l2_err("Failed to setup controls() (%d)\n",
> > +                                      ret);
> > +                       goto err_ctrls_setup;
> > +               }
> > +               ctx->m2m_ctx = v4l2_m2m_ctx_init(dev->m2m_dev_enc, ctx,
> > +                                                &mtk_vcodec_enc_queue_init);
> > +               if (IS_ERR(ctx->m2m_ctx)) {
> > +                       ret = PTR_ERR(ctx->m2m_ctx);
> > +                       mtk_v4l2_err("Failed to v4l2_m2m_ctx_init() (%d)\n",
> > +                                      ret);
> > +                       goto err_ctx_init;
> > +               }
> > +               mtk_vcodec_enc_ctx_params_setup(ctx);
> > +       } else {
> > +               mtk_v4l2_err("Invalid vfd !\n");
> > +               ret = -ENOENT;
> > +               goto err_ctx_init;
> > +       }
> > +
> > +       init_waitqueue_head(&ctx->queue);
> > +       dev->num_instances++;
> > +
> > +       if (dev->num_instances == 1) {
> > +               ret = vpu_load_firmware(dev->vpu_plat_dev);
> > +               if (ret < 0) {
> > +                               mtk_v4l2_err("vpu_load_firmware failed!\n");
> > +                       goto err_load_fw;
> > +               }
> > +
> > +               dev->enc_capability =
> > +                       vpu_get_venc_hw_capa(dev->vpu_plat_dev);
> > +               mtk_v4l2_debug(0, "encoder capability %x", dev->enc_capability);
> > +       }
> > +
> > +       mtk_v4l2_debug(2, "Create instance [%d]@%p m2m_ctx=%p type=%d\n",
> > +                        ctx->idx, ctx, ctx->m2m_ctx, ctx->type);
> > +       set_bit(ctx->idx, &dev->instance_mask[0]);
> > +       dev->ctx[ctx->idx] = ctx;
> > +
> > +       mutex_unlock(&dev->dev_mutex);
> > +       mtk_v4l2_debug(0, "%s encoder [%d]", dev_name(&dev->plat_dev->dev), ctx->idx);
> > +       return ret;
> > +
> > +       /* Deinit when failure occurred */
> > +err_load_fw:
> > +       v4l2_m2m_ctx_release(ctx->m2m_ctx);
> > +       v4l2_fh_del(&ctx->fh);
> > +       v4l2_fh_exit(&ctx->fh);
> > +       dev->num_instances--;
> > +err_ctx_init:
> > +err_ctrls_setup:
> > +       v4l2_ctrl_handler_free(&ctx->ctrl_hdl);
> > +err_no_ctx:
> > +       devm_kfree(&dev->plat_dev->dev, ctx);
> > +err_alloc:
> > +       mutex_unlock(&dev->dev_mutex);
> > +       return ret;
> > +}
> > +
> > +static int fops_vcodec_release(struct file *file)
> > +{
> > +       struct mtk_vcodec_dev *dev = video_drvdata(file);
> > +       struct mtk_vcodec_ctx *ctx = fh_to_ctx(file->private_data);
> > +
> > +       mtk_v4l2_debug(0, "[%d] encoder\n", ctx->idx);
> > +       mutex_lock(&dev->dev_mutex);
> > +
> > +       mtk_vcodec_enc_release(ctx);
> > +       v4l2_fh_del(&ctx->fh);
> > +       v4l2_fh_exit(&ctx->fh);
> > +       v4l2_ctrl_handler_free(&ctx->ctrl_hdl);
> > +       v4l2_m2m_ctx_release(ctx->m2m_ctx);
> > +
> > +       dev->ctx[ctx->idx] = NULL;
> > +       dev->num_instances--;
> > +       clear_bit(ctx->idx, &dev->instance_mask[0]);
> > +       devm_kfree(&dev->plat_dev->dev, ctx);
> > +       mutex_unlock(&dev->dev_mutex);
> > +       return 0;
> > +}
> > +
> > +static const struct v4l2_file_operations mtk_vcodec_fops = {
> > +       .owner                          = THIS_MODULE,
> > +       .open                           = fops_vcodec_open,
> > +       .release                        = fops_vcodec_release,
> > +       .poll                           = v4l2_m2m_fop_poll,
> > +       .unlocked_ioctl                 = video_ioctl2,
> > +       .mmap                           = v4l2_m2m_fop_mmap,
> > +};
> > +
> > +static int mtk_vcodec_probe(struct platform_device *pdev)
> > +{
> > +       struct mtk_vcodec_dev *dev;
> > +       struct video_device *vfd_enc;
> > +       struct resource *res;
> > +       int i, j, ret;
> > +
> > +       dev = devm_kzalloc(&pdev->dev, sizeof(*dev), GFP_KERNEL);
> > +       if (!dev)
> > +               return -ENOMEM;
> > +
> > +       dev->plat_dev = pdev;
> > +
> > +       dev->vpu_plat_dev = vpu_get_plat_device(dev->plat_dev);
> > +       if (dev->vpu_plat_dev == NULL) {
> > +               mtk_v4l2_err("[VPU] vpu device in not ready\n");
> > +               return -EPROBE_DEFER;
> > +       }
> > +
> > +       vpu_wdt_reg_handler(dev->vpu_plat_dev, mtk_vcodec_enc_reset_handler, dev,
> > +                           VPU_RST_ENC);
> > +
> > +       ret = mtk_vcodec_init_enc_pm(dev);
> > +       if (ret < 0) {
> > +               dev_err(&pdev->dev, "Failed to get mt vcodec clock source!\n");
> > +               return ret;
> > +       }
> > +
> > +       for (i = VENC_SYS, j = 0; i < NUM_MAX_VCODEC_REG_BASE; i++, j++) {
> > +               res = platform_get_resource(pdev, IORESOURCE_MEM, j);
> > +               if (res == NULL) {
> > +                       dev_err(&pdev->dev, "get memory resource failed.\n");
> > +                       ret = -ENXIO;
> > +                       goto err_res;
> > +               }
> > +               dev->reg_base[i] = devm_ioremap_resource(&pdev->dev, res);
> > +               if (IS_ERR(dev->reg_base[i])) {
> > +                       dev_err(&pdev->dev,
> > +                               "devm_ioremap_resource %d failed.\n", i);
> > +                       ret = PTR_ERR(dev->reg_base);
> > +                       goto err_res;
> > +               }
> > +               mtk_v4l2_debug(2, "reg[%d] base=0x%p\n", i, dev->reg_base[i]);
> > +       }
> > +
> > +       res = platform_get_resource(pdev, IORESOURCE_IRQ, 0);
> > +       if (res == NULL) {
> > +               dev_err(&pdev->dev, "failed to get irq resource\n");
> > +               ret = -ENOENT;
> > +               goto err_res;
> > +       }
> > +
> > +       dev->enc_irq = platform_get_irq(pdev, 0);
> > +       ret = devm_request_irq(&pdev->dev, dev->enc_irq,
> > +                              mtk_vcodec_enc_irq_handler,
> > +                              0, pdev->name, dev);
> > +       if (ret) {
> > +               dev_err(&pdev->dev, "Failed to install dev->enc_irq %d (%d)\n",
> > +                       dev->enc_irq,
> > +                       ret);
> > +               ret = -EINVAL;
> > +               goto err_res;
> > +       }
> > +
> > +       dev->enc_lt_irq = platform_get_irq(pdev, 1);
> > +       ret = devm_request_irq(&pdev->dev,
> > +                              dev->enc_lt_irq, mtk_vcodec_enc_irq_handler2,
> > +                              0, pdev->name, dev);
> > +       if (ret) {
> > +               dev_err(&pdev->dev,
> > +                       "Failed to install dev->enc_lt_irq %d (%d)\n",
> > +                       dev->enc_lt_irq, ret);
> > +               ret = -EINVAL;
> > +               goto err_res;
> > +       }
> > +
> > +       disable_irq(dev->enc_irq);
> > +       disable_irq(dev->enc_lt_irq); /* VENC_LT */
> > +       mutex_init(&dev->enc_mutex);
> > +       mutex_init(&dev->dev_mutex);
> > +
> > +       snprintf(dev->v4l2_dev.name, sizeof(dev->v4l2_dev.name), "%s",
> > +                "[MTK_V4L2_VENC]");
> > +
> > +       ret = v4l2_device_register(&pdev->dev, &dev->v4l2_dev);
> > +       if (ret) {
> > +               mtk_v4l2_err("v4l2_device_register err=%d\n", ret);
> > +               return ret;
> > +       }
> > +
> > +       init_waitqueue_head(&dev->queue);
> > +
> > +       /* allocate video device for encoder and register it */
> > +       vfd_enc = video_device_alloc();
> > +       if (!vfd_enc) {
> > +               mtk_v4l2_err("Failed to allocate video device\n");
> > +               ret = -ENOMEM;
> > +               goto err_enc_alloc;
> > +       }
> > +       vfd_enc->fops           = &mtk_vcodec_fops;
> > +       vfd_enc->ioctl_ops      = &mtk_venc_ioctl_ops;
> > +       vfd_enc->release        = video_device_release;
> > +       vfd_enc->lock           = &dev->dev_mutex;
> > +       vfd_enc->v4l2_dev       = &dev->v4l2_dev;
> > +       vfd_enc->vfl_dir        = VFL_DIR_M2M;
> > +
> > +       snprintf(vfd_enc->name, sizeof(vfd_enc->name), "%s",
> > +                MTK_VCODEC_ENC_NAME);
> > +       video_set_drvdata(vfd_enc, dev);
> > +       dev->vfd_enc = vfd_enc;
> > +       platform_set_drvdata(pdev, dev);
> > +       ret = video_register_device(vfd_enc, VFL_TYPE_GRABBER, 1);
> > +       if (ret) {
> > +               mtk_v4l2_err("Failed to register video device\n");
> > +               goto err_enc_reg;
> > +       }
> > +       mtk_v4l2_debug(0, "encoder registered as /dev/video%d\n",
> > +                        vfd_enc->num);
> > +
> > +       dev->alloc_ctx = vb2_dma_contig_init_ctx(&pdev->dev);
> > +       if (IS_ERR(dev->alloc_ctx)) {
> > +               mtk_v4l2_err("Failed to alloc vb2 dma context 0\n");
> > +               ret = PTR_ERR(dev->alloc_ctx);
> > +               goto err_vb2_ctx_init;
> > +       }
> > +
> > +       dev->m2m_dev_enc = v4l2_m2m_init(&mtk_venc_m2m_ops);
> > +       if (IS_ERR(dev->m2m_dev_enc)) {
> > +               mtk_v4l2_err("Failed to init mem2mem enc device\n");
> > +               ret = PTR_ERR(dev->m2m_dev_enc);
> > +               goto err_enc_mem_init;
> > +       }
> > +
> > +       dev->encode_workqueue =
> > +                       alloc_ordered_workqueue(MTK_VCODEC_ENC_NAME, WQ_MEM_RECLAIM | WQ_FREEZABLE);
> > +       if (!dev->encode_workqueue) {
> > +               mtk_v4l2_err("Failed to create encode workqueue\n");
> > +               ret = -EINVAL;
> > +               goto err_event_workq;
> > +       }
> > +
> > +       return 0;
> > +
> > +err_event_workq:
> > +       v4l2_m2m_release(dev->m2m_dev_enc);
> > +err_enc_mem_init:
> > +       vb2_dma_contig_cleanup_ctx(dev->alloc_ctx);
> > +err_vb2_ctx_init:
> > +       video_unregister_device(vfd_enc);
> > +err_enc_reg:
> > +       video_device_release(vfd_enc);
> > +err_enc_alloc:
> > +       v4l2_device_unregister(&dev->v4l2_dev);
> > +err_res:
> > +       mtk_vcodec_release_enc_pm(dev);
> > +       return ret;
> > +}
> > +
> > +static const struct of_device_id mtk_vcodec_match[] = {
> > +       {.compatible = "mediatek,mt8173-vcodec-enc",},
> > +       {},
> > +};
> > +MODULE_DEVICE_TABLE(of, mtk_vcodec_match);
> > +
> > +static int mtk_vcodec_remove(struct platform_device *pdev)
> > +{
> > +       struct mtk_vcodec_dev *dev = platform_get_drvdata(pdev);
> > +
> > +       mtk_v4l2_debug_enter();
> > +       flush_workqueue(dev->encode_workqueue);
> > +       destroy_workqueue(dev->encode_workqueue);
> > +       if (dev->m2m_dev_enc)
> > +               v4l2_m2m_release(dev->m2m_dev_enc);
> > +       if (dev->alloc_ctx)
> > +               vb2_dma_contig_cleanup_ctx(dev->alloc_ctx);
> > +
> > +       if (dev->vfd_enc) {
> > +               video_unregister_device(dev->vfd_enc);
> > +               video_device_release(dev->vfd_enc);
> > +       }
> > +       v4l2_device_unregister(&dev->v4l2_dev);
> > +       mtk_vcodec_release_enc_pm(dev);
> > +       return 0;
> > +}
> > +
> > +static struct platform_driver mtk_vcodec_driver = {
> > +       .probe  = mtk_vcodec_probe,
> > +       .remove = mtk_vcodec_remove,
> > +       .driver = {
> > +               .name   = MTK_VCODEC_ENC_NAME,
> > +               .owner  = THIS_MODULE,
> > +               .of_match_table = mtk_vcodec_match,
> > +       },
> > +};
> > +
> > +module_platform_driver(mtk_vcodec_driver);
> > +
> > +
> > +MODULE_LICENSE("GPL v2");
> > +MODULE_DESCRIPTION("Mediatek video codec V4L2 driver");
> > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
> > new file mode 100644
> > index 0000000..518fba7
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
> > @@ -0,0 +1,132 @@
> > +/*
> > +* Copyright (c) 2015 MediaTek Inc.
> > +* Author: Tiffany Lin <tiffany.lin-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
> > +*
> > +* This program is free software; you can redistribute it and/or modify
> > +* it under the terms of the GNU General Public License version 2 as
> > +* published by the Free Software Foundation.
> > +*
> > +* This program is distributed in the hope that it will be useful,
> > +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> > +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > +* GNU General Public License for more details.
> > +*/
> > +
> > +#include <linux/clk.h>
> > +#include <linux/of_address.h>
> > +#include <linux/of_platform.h>
> > +#include <linux/pm_runtime.h>
> > +#include <soc/mediatek/smi.h>
> > +
> > +#include "mtk_vcodec_pm.h"
> > +#include "mtk_vcodec_util.h"
> > +#include "mtk_vpu.h"
> > +
> > +
> > +int mtk_vcodec_init_enc_pm(struct mtk_vcodec_dev *mtkdev)
> > +{
> > +       struct device_node *node;
> > +       struct platform_device *pdev;
> > +       struct device *dev;
> > +       struct mtk_vcodec_pm *pm;
> > +       int ret = 0;
> > +
> > +       pdev = mtkdev->plat_dev;
> > +       pm = &mtkdev->pm;
> > +       memset(pm, 0, sizeof(struct mtk_vcodec_pm));
> > +       pm->mtkdev = mtkdev;
> > +       dev = &pdev->dev;
> > +
> > +       node = of_parse_phandle(dev->of_node, "mediatek,larb", 0);
> > +       if (!node)
> > +               return -1;
> > +       pdev = of_find_device_by_node(node);
> > +       if (WARN_ON(!pdev)) {
> > +               of_node_put(node);
> > +               return -1;
> > +       }
> > +       pm->larbvenc = &pdev->dev;
> > +
> > +       node = of_parse_phandle(dev->of_node, "mediatek,larb", 1);
> > +       if (!node)
> > +               return -1;
> > +
> > +       pdev = of_find_device_by_node(node);
> > +       if (WARN_ON(!pdev)) {
> > +               of_node_put(node);
> > +               return -EINVAL;
> > +       }
> > +       pm->larbvenclt = &pdev->dev;
> > +
> > +       pdev = mtkdev->plat_dev;
> > +       pm->dev = &pdev->dev;
> > +
> > +       pm->vencpll_d2 = devm_clk_get(&pdev->dev, "vencpll_d2");
> > +       if (pm->vencpll_d2 == NULL) {
> > +               mtk_v4l2_err("devm_clk_get vencpll_d2 fail");
> > +               ret = -1;
> > +       }
> > +
> > +       pm->venc_sel = devm_clk_get(&pdev->dev, "venc_sel");
> > +       if (pm->venc_sel == NULL) {
> > +               mtk_v4l2_err("devm_clk_get venc_sel fail");
> > +               ret = -1;
> > +       }
> > +
> > +       pm->univpll1_d2 = devm_clk_get(&pdev->dev, "univpll1_d2");
> > +       if (pm->univpll1_d2 == NULL) {
> > +               mtk_v4l2_err("devm_clk_get univpll1_d2 fail");
> > +               ret = -1;
> > +       }
> > +
> > +       pm->venc_lt_sel = devm_clk_get(&pdev->dev, "venc_lt_sel");
> > +       if (pm->venc_lt_sel == NULL) {
> > +               mtk_v4l2_err("devm_clk_get venc_lt_sel fail");
> > +               ret = -1;
> > +       }
> > +
> > +       return ret;
> > +}
> > +
> > +void mtk_vcodec_release_enc_pm(struct mtk_vcodec_dev *mtkdev)
> > +{
> > +}
> > +
> > +
> > +void mtk_vcodec_enc_clock_on(struct mtk_vcodec_pm *pm)
> > +{
> > +       int ret;
> > +
> > +       ret = clk_prepare_enable(pm->venc_sel);
> > +       if (ret)
> > +               mtk_v4l2_err("venc_sel fail %d", ret);
> > +
> > +       ret = clk_set_parent(pm->venc_sel, pm->vencpll_d2);
> > +       if (ret)
> > +               mtk_v4l2_err("clk_set_parent fail %d", ret);
> > +
> > +       ret = clk_prepare_enable(pm->venc_lt_sel);
> > +       if (ret)
> > +               mtk_v4l2_err("venc_lt_sel fail %d", ret);
> > +
> > +       ret = clk_set_parent(pm->venc_lt_sel, pm->univpll1_d2);
> > +       if (ret)
> > +               mtk_v4l2_err("clk_set_parent fail %d", ret);
> > +
> > +       ret = mtk_smi_larb_get(pm->larbvenc);
> > +       if (ret)
> > +               mtk_v4l2_err("mtk_smi_larb_get larb3 fail %d\n", ret);
> > +
> > +       ret = mtk_smi_larb_get(pm->larbvenclt);
> > +       if (ret)
> > +               mtk_v4l2_err("mtk_smi_larb_get larb4 fail %d\n", ret);
> > +
> > +}
> > +
> > +void mtk_vcodec_enc_clock_off(struct mtk_vcodec_pm *pm)
> > +{
> > +       mtk_smi_larb_put(pm->larbvenc);
> > +       mtk_smi_larb_put(pm->larbvenclt);
> > +       clk_disable_unprepare(pm->venc_lt_sel);
> > +       clk_disable_unprepare(pm->venc_sel);
> > +}
> > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.c
> > new file mode 100644
> > index 0000000..919b949
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.c
> > @@ -0,0 +1,102 @@
> > +/*
> > +* Copyright (c) 2015 MediaTek Inc.
> > +* Author: Tiffany Lin <tiffany.lin-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
> > +*
> > +* This program is free software; you can redistribute it and/or modify
> > +* it under the terms of the GNU General Public License version 2 as
> > +* published by the Free Software Foundation.
> > +*
> > +* This program is distributed in the hope that it will be useful,
> > +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> > +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > +* GNU General Public License for more details.
> > +*/
> > +
> > +#include <linux/errno.h>
> > +#include <linux/wait.h>
> > +
> > +#include "mtk_vcodec_drv.h"
> > +#include "mtk_vcodec_intr.h"
> > +#include "mtk_vcodec_util.h"
> > +
> > +void mtk_vcodec_clean_dev_int_flags(void *data)
> > +{
> > +       struct mtk_vcodec_dev *dev = (struct mtk_vcodec_dev *)data;
> > +
> > +       dev->int_cond = 0;
> > +       dev->int_type = 0;
> > +}
> > +
> > +int mtk_vcodec_wait_for_done_ctx(void *data, int command,
> > +                                unsigned int timeout_ms, int interrupt)
> > +{
> > +       wait_queue_head_t *waitqueue;
> > +       long timeout_jiff, ret;
> > +       int status = 0;
> > +       struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)data;
> > +
> > +       waitqueue = (wait_queue_head_t *)&ctx->queue;
> > +       timeout_jiff = msecs_to_jiffies(timeout_ms);
> > +       if (interrupt) {
> > +               ret = wait_event_interruptible_timeout(*waitqueue,
> > +                               (ctx->int_cond &&
> > +                               (ctx->int_type == command)),
> > +                               timeout_jiff);
> > +       } else {
> > +               ret = wait_event_timeout(*waitqueue,
> > +                               (ctx->int_cond &&
> > +                               (ctx->int_type == command)),
> > +                                timeout_jiff);
> > +       }
> > +       if (0 == ret) {
> > +               status = -1;    /* timeout */
> > +               mtk_v4l2_err("[%d] cmd=%d, ctx->type=%d, wait_event_interruptible_timeout time=%ums out %d %d!",
> > +                               ctx->idx, ctx->type, command, timeout_ms,
> > +                               ctx->int_cond, ctx->int_type);
> > +       } else if (-ERESTARTSYS == ret) {
> > +               mtk_v4l2_err("[%d] cmd=%d, ctx->type=%d, wait_event_interruptible_timeout interrupted by a signal %d %d",
> > +                               ctx->idx, ctx->type, command, ctx->int_cond,
> > +                               ctx->int_type);
> > +               status = -1;
> > +       }
> > +
> > +       ctx->int_cond = 0;
> > +       ctx->int_type = 0;
> > +
> > +       return status;
> > +}
> > +
> > +int mtk_vcodec_wait_for_done_dev(void *data, int command,
> > +                                unsigned int timeout, int interrupt)
> > +{
> > +       wait_queue_head_t *waitqueue;
> > +       long timeout_jiff, ret;
> > +       int status = 0;
> > +       struct mtk_vcodec_dev *dev = (struct mtk_vcodec_dev *)data;
> > +
> > +       waitqueue = (wait_queue_head_t *)&dev->queue;
> > +       timeout_jiff = msecs_to_jiffies(timeout);
> > +       if (interrupt) {
> > +               ret = wait_event_interruptible_timeout(*waitqueue,
> > +                               (dev->int_cond &&
> > +                               (dev->int_type == command)),
> > +                               timeout_jiff);
> > +       } else {
> > +               ret = wait_event_timeout(*waitqueue,
> > +                               (dev->int_cond &&
> > +                               (dev->int_type == command)),
> > +                               timeout_jiff);
> > +       }
> > +       if (0 == ret) {
> > +               status = -1;    /* timeout */
> > +               mtk_v4l2_err("wait_event_interruptible_timeout time=%lu out %d %d!",
> > +                               timeout_jiff, dev->int_cond, dev->int_type);
> > +       } else if (-ERESTARTSYS == ret) {
> > +               mtk_v4l2_err("wait_event_interruptible_timeout interrupted by a signal %d %d",
> > +                               dev->int_cond, dev->int_type);
> > +               status = -1;
> > +       }
> > +       dev->int_cond = 0;
> > +       dev->int_type = 0;
> > +       return status;
> > +}
> > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.h
> > new file mode 100644
> > index 0000000..e9b7f94
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.h
> > @@ -0,0 +1,29 @@
> > +/*
> > +* Copyright (c) 2015 MediaTek Inc.
> > +* Author: Tiffany Lin <tiffany.lin-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
> > +*
> > +* This program is free software; you can redistribute it and/or modify
> > +* it under the terms of the GNU General Public License version 2 as
> > +* published by the Free Software Foundation.
> > +*
> > +* This program is distributed in the hope that it will be useful,
> > +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> > +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > +* GNU General Public License for more details.
> > +*/
> > +
> > +#ifndef _MTK_VCODEC_INTR_H_
> > +#define _MTK_VCODEC_INTR_H_
> > +
> > +#define MTK_INST_IRQ_RECEIVED          0x1
> > +#define MTK_INST_WORK_THREAD_ABORT_DONE        0x2
> > +
> > +/* timeout is ms */
> > +int mtk_vcodec_wait_for_done_ctx(void *data, int command, unsigned int timeout,
> > +                                int interrupt);
> > +int mtk_vcodec_wait_for_done_dev(void *data, int command, unsigned int timeout,
> > +                                int interrupt);
> > +
> > +void mtk_vcodec_clean_dev_int_flags(void *data);
> > +
> > +#endif /* _MTK_VCODEC_INTR_H_ */
> > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_pm.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_pm.h
> > new file mode 100644
> > index 0000000..fdadec9
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_pm.h
> > @@ -0,0 +1,26 @@
> > +/*
> > +* Copyright (c) 2015 MediaTek Inc.
> > +* Author: Tiffany Lin <tiffany.lin-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
> > +*
> > +* This program is free software; you can redistribute it and/or modify
> > +* it under the terms of the GNU General Public License version 2 as
> > +* published by the Free Software Foundation.
> > +*
> > +* This program is distributed in the hope that it will be useful,
> > +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> > +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > +* GNU General Public License for more details.
> > +*/
> > +
> > +#ifndef _MTK_VCODEC_PM_H_
> > +#define _MTK_VCODEC_PM_H_
> > +
> > +#include "mtk_vcodec_drv.h"
> > +
> > +int mtk_vcodec_init_enc_pm(struct mtk_vcodec_dev *dev);
> > +void mtk_vcodec_release_enc_pm(struct mtk_vcodec_dev *dev);
> > +
> > +void mtk_vcodec_enc_clock_on(struct mtk_vcodec_pm *pm);
> > +void mtk_vcodec_enc_clock_off(struct mtk_vcodec_pm *pm);
> > +
> > +#endif /* _MTK_VCODEC_PM_H_ */
> > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c
> > new file mode 100644
> > index 0000000..3fede8d
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c
> > @@ -0,0 +1,106 @@
> > +/*
> > +* Copyright (c) 2015 MediaTek Inc.
> > +* Author: PC Chen <pc.chen-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
> > +*         Tiffany Lin <tiffany.lin-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
> > +*
> > +* This program is free software; you can redistribute it and/or modify
> > +* it under the terms of the GNU General Public License version 2 as
> > +* published by the Free Software Foundation.
> > +*
> > +* This program is distributed in the hope that it will be useful,
> > +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> > +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > +* GNU General Public License for more details.
> > +*/
> > +
> > +#include <linux/module.h>
> > +
> > +#include "mtk_vcodec_drv.h"
> > +#include "mtk_vcodec_util.h"
> > +#include "mtk_vpu.h"
> > +
> > +bool mtk_vcodec_dbg = false;
> > +int mtk_v4l2_dbg_level = 0;
> > +
> > +module_param(mtk_v4l2_dbg_level, int, S_IRUGO | S_IWUSR);
> > +module_param(mtk_vcodec_dbg, bool, S_IRUGO | S_IWUSR);
> > +
> > +void __iomem *mtk_vcodec_get_reg_addr(void *data, unsigned int reg_idx)
> > +{
> > +       struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)data;
> > +
> > +       if (!data || reg_idx >= NUM_MAX_VCODEC_REG_BASE) {
> > +               mtk_v4l2_err("Invalid arguments");
> > +               return NULL;
> > +       }
> > +       return ctx->dev->reg_base[reg_idx];
> > +}
> > +
> > +int mtk_vcodec_mem_alloc(void *data, struct mtk_vcodec_mem *mem)
> > +{
> > +       unsigned long size = mem->size;
> > +       struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)data;
> > +       struct device *dev = &ctx->dev->plat_dev->dev;
> > +
> > +       mem->va = dma_alloc_coherent(dev, size, &mem->dma_addr, GFP_KERNEL);
> > +
> > +       if (!mem->va) {
> > +               mtk_v4l2_err("%s dma_alloc size=%ld failed!", dev_name(dev),
> > +                              size);
> > +               return -ENOMEM;
> > +       }
> > +
> > +       memset(mem->va, 0, size);
> > +
> > +       mtk_v4l2_debug(3, "[%d]  - va      = %p", ctx->idx, mem->va);
> > +       mtk_v4l2_debug(3, "[%d]  - dma     = 0x%lx", ctx->idx,
> > +                        (unsigned long)mem->dma_addr);
> > +       mtk_v4l2_debug(3, "[%d]    size = 0x%lx", ctx->idx, size);
> > +
> > +       return 0;
> > +}
> > +
> > +void mtk_vcodec_mem_free(void *data, struct mtk_vcodec_mem *mem)
> > +{
> > +       unsigned long size = mem->size;
> > +       struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)data;
> > +       struct device *dev = &ctx->dev->plat_dev->dev;
> > +
> > +       dma_free_coherent(dev, size, mem->va, mem->dma_addr);
> > +       mem->va = NULL;
> > +
> > +       mtk_v4l2_debug(3, "[%d]  - va      = %p", ctx->idx, mem->va);
> > +       mtk_v4l2_debug(3, "[%d]  - dma     = 0x%lx", ctx->idx,
> > +                        (unsigned long)mem->dma_addr);
> > +       mtk_v4l2_debug(3, "[%d]    size = 0x%lx", ctx->idx, size);
> > +}
> > +
> > +int mtk_vcodec_get_ctx_id(void *data)
> > +{
> > +       struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)data;
> > +
> > +       if (!ctx)
> > +               return -1;
> > +
> > +       return ctx->idx;
> > +}
> > +
> > +struct platform_device *mtk_vcodec_get_plat_dev(void *data)
> > +{
> > +       struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)data;
> > +
> > +       if (!ctx)
> > +               return NULL;
> > +
> > +       return vpu_get_plat_device(ctx->dev->plat_dev);
> > +}
> > +
> > +void mtk_vcodec_fmt2str(u32 fmt, char *str)
> > +{
> > +       char a = fmt & 0xFF;
> > +       char b = (fmt >> 8) & 0xFF;
> > +       char c = (fmt >> 16) & 0xFF;
> > +       char d = (fmt >> 24) & 0xFF;
> > +
> > +       sprintf(str, "%c%c%c%c", a, b, c, d);
> > +}
> > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
> > new file mode 100644
> > index 0000000..47016ae
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
> > @@ -0,0 +1,85 @@
> > +/*
> > +* Copyright (c) 2015 MediaTek Inc.
> > +* Author: PC Chen <pc.chen-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
> > +*         Tiffany Lin <tiffany.lin-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
> > +*
> > +* This program is free software; you can redistribute it and/or modify
> > +* it under the terms of the GNU General Public License version 2 as
> > +* published by the Free Software Foundation.
> > +*
> > +* This program is distributed in the hope that it will be useful,
> > +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> > +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > +* GNU General Public License for more details.
> > +*/
> > +
> > +#ifndef _MTK_VCODEC_UTIL_H_
> > +#define _MTK_VCODEC_UTIL_H_
> > +
> > +#include <linux/types.h>
> > +#include <linux/dma-direction.h>
> > +
> > +struct mtk_vcodec_mem {
> > +       size_t size;
> > +       void *va;
> > +       dma_addr_t dma_addr;
> > +};
> > +
> > +extern int mtk_v4l2_dbg_level;
> > +extern bool mtk_vcodec_dbg;
> > +
> > +#define DEBUG  1
> > +
> > +#if defined(DEBUG)
> > +
> > +#define mtk_v4l2_debug(level, fmt, args...)                             \
> > +       do {                                                             \
> > +               if (mtk_v4l2_dbg_level >= level)                         \
> > +                       pr_info("[MTK_V4L2] level=%d %s(),%d: " fmt "\n",\
> > +                               level, __func__, __LINE__, ##args);      \
> > +       } while (0)
> > +
> > +#define mtk_v4l2_err(fmt, args...)                \
> > +       pr_err("[MTK_V4L2][ERROR] %s:%d: " fmt "\n", __func__, __LINE__, \
> > +              ##args)
> > +
> > +
> > +#define mtk_v4l2_debug_enter()  mtk_v4l2_debug(3, "+\n")
> > +#define mtk_v4l2_debug_leave()  mtk_v4l2_debug(3, "-\n")
> > +
> > +#define mtk_vcodec_debug(h, fmt, args...)                              \
> > +       do {                                                            \
> > +               if (mtk_vcodec_dbg)                                     \
> > +                       pr_info("[MTK_VCODEC][%d]: %s() " fmt "\n",     \
> > +                               ((struct mtk_vcodec_ctx *)h->ctx)->idx, \
> > +                               __func__, ##args);                      \
> > +       } while (0)
> > +
> > +#define mtk_vcodec_err(h, fmt, args...)                                        \
> > +       pr_err("[MTK_VCODEC][ERROR][%d]: %s() " fmt "\n",               \
> > +              ((struct mtk_vcodec_ctx *)h->ctx)->idx, __func__, ##args)
> > +
> > +#define mtk_vcodec_debug_enter(h)  mtk_vcodec_debug(h, "+\n")
> > +#define mtk_vcodec_debug_leave(h)  mtk_vcodec_debug(h, "-\n")
> > +
> > +#else
> > +
> > +#define mtk_v4l2_debug(level, fmt, args...)
> > +#define mtk_v4l2_err(fmt, args...)
> > +#define mtk_v4l2_debug_enter()
> > +#define mtk_v4l2_debug_leave()
> > +
> > +#define mtk_vcodec_debug(h, fmt, args...)
> > +#define mtk_vcodec_err(h, fmt, args...)
> > +#define mtk_vcodec_debug_enter(h)
> > +#define mtk_vcodec_debug_leave(h)
> > +
> > +#endif
> > +
> > +void __iomem *mtk_vcodec_get_reg_addr(void *data, unsigned int reg_idx);
> > +int mtk_vcodec_mem_alloc(void *data, struct mtk_vcodec_mem *mem);
> > +void mtk_vcodec_mem_free(void *data, struct mtk_vcodec_mem *mem);
> > +int mtk_vcodec_get_ctx_id(void *data);
> > +struct platform_device *mtk_vcodec_get_plat_dev(void *data);
> > +void mtk_vcodec_fmt2str(u32 fmt, char *str);
> > +#endif /* _MTK_VCODEC_UTIL_H_ */
> > diff --git a/drivers/media/platform/mtk-vcodec/venc_drv_base.h b/drivers/media/platform/mtk-vcodec/venc_drv_base.h
> > new file mode 100644
> > index 0000000..ed9cbf0
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/venc_drv_base.h
> > @@ -0,0 +1,62 @@
> > +/*
> > + * Copyright (c) 2015 MediaTek Inc.
> > + * Author: Daniel Hsiao <daniel.hsiao-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
> > + *         Jungchang Tsao <jungchang.tsao-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
> > + *
> > + * This program is free software; you can redistribute it and/or
> > + * modify
> > + * it under the terms of the GNU General Public License version 2 as
> > + * published by the Free Software Foundation.
> > + *
> > + * This program is distributed in the hope that it will be useful,
> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > + * GNU General Public License for more details.
> > + */
> > +
> > +#ifndef _VENC_DRV_BASE_
> > +#define _VENC_DRV_BASE_
> > +
> > +#include "mtk_vcodec_drv.h"
> > +
> > +#include "venc_drv_if.h"
> > +
> > +struct venc_common_if {
> > +       /**
> > +        * (*init)() - initialize driver
> > +        * @ctx:        [in] mtk v4l2 context
> > +        * @handle: [out] driver handle
> > +        */
> > +       int (*init)(struct mtk_vcodec_ctx *ctx, unsigned long *handle);
> > +
> > +       /**
> > +        * (*encode)() - trigger encode
> > +        * @handle: [in] driver handle
> > +        * @opt: [in] encode option
> > +        * @frm_buf: [in] frame buffer to store input frame
> > +        * @bs_buf: [in] bitstream buffer to store output bitstream
> > +        * @result: [out] encode result
> > +        */
> > +       int (*encode)(unsigned long handle, enum venc_start_opt opt,
> > +                     struct venc_frm_buf *frm_buf,
> > +                     struct mtk_vcodec_mem *bs_buf,
> > +                     struct venc_done_result *result);
> > +
> > +       /**
> > +        * (*set_param)() - set driver's parameter
> > +        * @handle: [in] driver handle
> > +        * @type: [in] parameter type
> > +        * @in: [in] buffer to store the parameter
> > +        */
> > +       int (*set_param)(unsigned long handle, enum venc_set_param_type type,
> > +                        void *in);
> > +
> > +       /**
> > +        * (*deinit)() - deinitialize driver.
> > +        * @handle: [in] driver handle
> > +        */
> > +       int (*deinit)(unsigned long handle);
> > +};
> > +
> > +
> > +#endif
> > diff --git a/drivers/media/platform/mtk-vcodec/venc_drv_if.c b/drivers/media/platform/mtk-vcodec/venc_drv_if.c
> > new file mode 100644
> > index 0000000..daa8e93
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/venc_drv_if.c
> > @@ -0,0 +1,100 @@
> > +/*
> > + * Copyright (c) 2015 MediaTek Inc.
> > + * Author: Daniel Hsiao <daniel.hsiao-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
> > + *         Jungchang Tsao <jungchang.tsao-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
> > + *
> > + * This program is free software; you can redistribute it and/or
> > + * modify
> > + * it under the terms of the GNU General Public License version 2 as
> > + * published by the Free Software Foundation.
> > + *
> > + * This program is distributed in the hope that it will be useful,
> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > + * GNU General Public License for more details.
> > + */
> > +
> > +#include <linux/interrupt.h>
> > +#include <linux/kernel.h>
> > +#include <linux/slab.h>
> > +
> > +#include "venc_drv_if.h"
> > +#include "mtk_vcodec_enc.h"
> > +#include "mtk_vcodec_pm.h"
> > +#include "mtk_vpu.h"
> > +
> > +#include "venc_drv_base.h"
> > +
> > +int venc_if_create(struct mtk_vcodec_ctx *ctx, unsigned int fourcc)
> > +{
> > +       char str[10];
> > +       int ret = 0;
> > +
> > +       mtk_vcodec_fmt2str(fourcc, str);
> > +
> > +       switch (fourcc) {
> > +       case V4L2_PIX_FMT_VP8:
> > +       case V4L2_PIX_FMT_H264:
> > +       default:
> > +               return -EINVAL;
> > +       }
> > +
> > +       mtk_venc_lock(ctx);
> > +       mtk_vcodec_enc_clock_on(&ctx->dev->pm);
> > +       ret = ctx->enc_if->init(ctx, (unsigned long *)&ctx->drv_handle);
> > +       mtk_vcodec_enc_clock_off(&ctx->dev->pm);
> > +       mtk_venc_unlock(ctx);
> > +
> > +       return ret;
> > +
> > +}
> > +
> > +
> > +int venc_if_set_param(struct mtk_vcodec_ctx *ctx,
> > +                     enum venc_set_param_type type, void *in)
> > +{
> > +       int ret = 0;
> > +
> > +       mtk_venc_lock(ctx);
> > +       mtk_vcodec_enc_clock_on(&ctx->dev->pm);
> > +       ret = ctx->enc_if->set_param(ctx->drv_handle, type, in);
> > +       mtk_vcodec_enc_clock_off(&ctx->dev->pm);
> > +       mtk_venc_unlock(ctx);
> > +
> > +       return ret;
> > +}
> > +
> > +int venc_if_encode(struct mtk_vcodec_ctx *ctx,
> > +                  enum venc_start_opt opt, struct venc_frm_buf *frm_buf,
> > +                  struct mtk_vcodec_mem *bs_buf,
> > +                  struct venc_done_result *result)
> > +{
> > +       int ret = 0;
> > +
> > +       mtk_venc_lock(ctx);
> > +       mtk_vcodec_enc_clock_on(&ctx->dev->pm);
> > +       ret = ctx->enc_if->encode(ctx->drv_handle, opt, frm_buf, bs_buf, result);
> > +       mtk_vcodec_enc_clock_off(&ctx->dev->pm);
> > +       mtk_venc_unlock(ctx);
> > +
> > +       return ret;
> > +}
> > +
> > +int venc_if_release(struct mtk_vcodec_ctx *ctx)
> > +{
> > +       int ret = 0;
> > +
> > +       if(ctx->drv_handle == 0)
> > +               return 0;
> > +
> > +       mtk_venc_lock(ctx);
> > +       mtk_vcodec_enc_clock_on(&ctx->dev->pm);
> > +       ret = ctx->enc_if->deinit(ctx->drv_handle);
> > +       mtk_vcodec_enc_clock_off(&ctx->dev->pm);
> > +       mtk_venc_unlock(ctx);
> > +
> > +       ctx->drv_handle = 0;
> > +
> > +       return ret;
> > +}
> > +
> > diff --git a/drivers/media/platform/mtk-vcodec/venc_drv_if.h b/drivers/media/platform/mtk-vcodec/venc_drv_if.h
> > new file mode 100644
> > index 0000000..a387011
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/venc_drv_if.h
> > @@ -0,0 +1,175 @@
> > +/*
> > + * Copyright (c) 2015 MediaTek Inc.
> > + * Author: Daniel Hsiao <daniel.hsiao-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
> > + *         Jungchang Tsao <jungchang.tsao-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
> > + *
> > + * This program is free software; you can redistribute it and/or
> > + * modify
> > + * it under the terms of the GNU General Public License version 2 as
> > + * published by the Free Software Foundation.
> > + *
> > + * This program is distributed in the hope that it will be useful,
> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > + * GNU General Public License for more details.
> > + */
> > +
> > +#ifndef _VENC_DRV_IF_H_
> > +#define _VENC_DRV_IF_H_
> > +
> > +#include "mtk_vcodec_drv.h"
> > +#include "mtk_vcodec_util.h"
> > +
> > +/*
> > + * enum venc_yuv_fmt - The type of input yuv format
> > + * (VPU related: If you change the order, you must also update the VPU codes.)
> > + * @VENC_YUV_FORMAT_420: 420 YUV format
> > + * @VENC_YUV_FORMAT_YV12: YV12 YUV format
> > + * @VENC_YUV_FORMAT_NV12: NV12 YUV format
> > + * @VENC_YUV_FORMAT_NV21: NV21 YUV format
> > + */
> > +enum venc_yuv_fmt {
> > +       VENC_YUV_FORMAT_420 = 3,
> > +       VENC_YUV_FORMAT_YV12 = 5,
> > +       VENC_YUV_FORMAT_NV12 = 6,
> > +       VENC_YUV_FORMAT_NV21 = 7,
> > +};
> > +
> > +/*
> > + * enum venc_start_opt - encode frame option used in venc_if_encode()
> > + * @VENC_START_OPT_ENCODE_SEQUENCE_HEADER: encode SPS/PPS for H264
> > + * @VENC_START_OPT_ENCODE_FRAME: encode normal frame
> > + */
> > +enum venc_start_opt {
> > +       VENC_START_OPT_ENCODE_SEQUENCE_HEADER,
> > +       VENC_START_OPT_ENCODE_FRAME,
> > +};
> > +
> > +/*
> > + * enum venc_drv_msg - The type of encode frame status used in venc_if_encode()
> > + * @VENC_MESSAGE_OK: encode ok
> > + * @VENC_MESSAGE_ERR: encode error
> > + */
> > +enum venc_drv_msg {
> > +       VENC_MESSAGE_OK,
> > +       VENC_MESSAGE_ERR,
> > +};
> > +
> > +/*
> > + * enum venc_set_param_type - The type of set parameter used in venc_if_set_param()
> > + * (VPU related: If you change the order, you must also update the VPU codes.)
> > + * @VENC_SET_PARAM_ENC: set encoder parameters
> > + * @VENC_SET_PARAM_FORCE_INTRA: set force intra frame
> > + * @VENC_SET_PARAM_ADJUST_BITRATE: set to adjust bitrate (in bps)
> > + * @VENC_SET_PARAM_ADJUST_FRAMERATE: set frame rate
> > + * @VENC_SET_PARAM_I_FRAME_INTERVAL: set I frame interval
> > + * @VENC_SET_PARAM_SKIP_FRAME: set H264 skip one frame
> > + * @VENC_SET_PARAM_PREPEND_HEADER: set H264 prepend SPS/PPS before IDR
> > + * @VENC_SET_PARAM_TS_MODE: set VP8 temporal scalability mode
> > + */
> > +enum venc_set_param_type {
> > +       VENC_SET_PARAM_ENC,
> > +       VENC_SET_PARAM_FORCE_INTRA,
> > +       VENC_SET_PARAM_ADJUST_BITRATE,
> > +       VENC_SET_PARAM_ADJUST_FRAMERATE,
> > +       VENC_SET_PARAM_I_FRAME_INTERVAL,
> > +       VENC_SET_PARAM_SKIP_FRAME,
> > +       VENC_SET_PARAM_PREPEND_HEADER,
> > +       VENC_SET_PARAM_TS_MODE,
> > +};
> > +
> > +/*
> > + * struct venc_enc_prm - encoder settings for VENC_SET_PARAM_ENC used in venc_if_set_param()
> > + * @input_fourcc: input fourcc
> > + * @h264_profile: V4L2 defined H.264 profile
> > + * @h264_level: V4L2 defined H.264 level
> > + * @width: image width
> > + * @height: image height
> > + * @buf_width: buffer width
> > + * @buf_height: buffer height
> > + * @frm_rate: frame rate
> > + * @intra_period: intra frame period
> > + * @bitrate: target bitrate in kbps
> > + */
> > +struct venc_enc_prm {
> > +       enum venc_yuv_fmt input_fourcc;
> > +       unsigned int h264_profile;
> > +       unsigned int h264_level;
> > +       unsigned int width;
> > +       unsigned int height;
> > +       unsigned int buf_width;
> > +       unsigned int buf_height;
> > +       unsigned int frm_rate;
> > +       unsigned int intra_period;
> > +       unsigned int bitrate;
> > +       unsigned int sizeimage[MTK_VCODEC_MAX_PLANES];
> > +};
> > +
> > +/*
> > + * struct venc_frm_buf - frame buffer information used in venc_if_encode()
> > + * @fb_addr: plane 0 frame buffer address
> > + * @fb_addr1: plane 1 frame buffer address
> > + * @fb_addr2: plane 2 frame buffer address
> > + */
> > +struct venc_frm_buf {
> > +       struct mtk_vcodec_mem fb_addr;
> > +       struct mtk_vcodec_mem fb_addr1;
> > +       struct mtk_vcodec_mem fb_addr2;
> > +};
> > +
> > +/*
> > + * struct venc_done_result - This is return information used in venc_if_encode()
> > + * @msg: message, such as success or error code
> > + * @bs_size: output bitstream size
> > + * @is_key_frm: output is key frame or not
> > + */
> > +struct venc_done_result {
> > +       enum venc_drv_msg msg;
> > +       unsigned int bs_size;
> > +       bool is_key_frm;
> > +};
> > +
> > +/*
> > + * venc_if_create - Create the driver handle
> > + * @ctx: device context
> > + * @fourcc: encoder output format
> > + * @handle: driver handle
> > + * Return: 0 if creating handle successfully, otherwise it is failed.
> > + */
> > +int venc_if_create(struct mtk_vcodec_ctx *ctx, unsigned int fourcc);
> > +
> > +/*
> > + * venc_if_release - Release the driver handle
> > + * @handle: driver handle
> > + * Return: 0 if releasing handle successfully, otherwise it is failed.
> > + */
> > +int venc_if_release(struct mtk_vcodec_ctx *ctx);
> > +
> > +/*
> > + * venc_if_set_param - Set parameter to driver
> > + * @handle: driver handle
> > + * @type: set type
> > + * @in: input parameter
> > + * @out: output parameter
> > + * Return: 0 if setting param successfully, otherwise it is failed.
> > + */
> > +int venc_if_set_param(struct mtk_vcodec_ctx *ctx,
> > +                     enum venc_set_param_type type,
> > +                     void *in);
> > +
> > +/*
> > + * venc_if_encode - Encode frame
> > + * @handle: driver handle
> > + * @opt: encode frame option
> > + * @frm_buf: input frame buffer information
> > + * @bs_buf: output bitstream buffer infomraiton
> > + * @result: encode result
> > + * Return: 0 if encoding frame successfully, otherwise it is failed.
> > + */
> > +int venc_if_encode(struct mtk_vcodec_ctx *ctx,
> > +                  enum venc_start_opt opt,
> > +                  struct venc_frm_buf *frm_buf,
> > +                  struct mtk_vcodec_mem *bs_buf,
> > +                  struct venc_done_result *result);
> > +
> > +#endif /* _VENC_DRV_IF_H_ */
> > diff --git a/drivers/media/platform/mtk-vcodec/venc_ipi_msg.h b/drivers/media/platform/mtk-vcodec/venc_ipi_msg.h
> > new file mode 100644
> > index 0000000..a345b98
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/venc_ipi_msg.h
> > @@ -0,0 +1,212 @@
> > +/*
> > + * Copyright (c) 2015 MediaTek Inc.
> > + * Author: Jungchang Tsao <jungchang.tsao-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
> > + *         Daniel Hsiao <daniel.hsiao-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
> > + *
> > + * This program is free software; you can redistribute it and/or
> > + * modify
> > + * it under the terms of the GNU General Public License version 2 as
> > + * published by the Free Software Foundation.
> > + *
> > + * This program is distributed in the hope that it will be useful,
> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > + * GNU General Public License for more details.
> > + */
> > +
> > +#ifndef _VENC_IPI_MSG_H_
> > +#define _VENC_IPI_MSG_H_
> > +
> > +#define IPIMSG_H264_ENC_ID 0x100
> > +#define IPIMSG_VP8_ENC_ID 0x200
> > +
> > +#define AP_IPIMSG_VENC_BASE 0x20000
> > +#define VPU_IPIMSG_VENC_BASE 0x30000
> > +
> > +/**
> > + * enum venc_ipi_msg_id - message id between AP and VPU
> > + * (ipi stands for inter-processor interrupt)
> > + * @AP_IPIMSG_XXX:             AP to VPU cmd message id
> > + * @VPU_IPIMSG_XXX_DONE:       VPU ack AP cmd message id
> > + */
> > +enum venc_ipi_msg_id {
> > +       AP_IPIMSG_H264_ENC_INIT = AP_IPIMSG_VENC_BASE +
> > +                                 IPIMSG_H264_ENC_ID,
> > +       AP_IPIMSG_H264_ENC_SET_PARAM,
> > +       AP_IPIMSG_H264_ENC_ENCODE,
> > +       AP_IPIMSG_H264_ENC_DEINIT,
> > +
> > +       AP_IPIMSG_VP8_ENC_INIT = AP_IPIMSG_VENC_BASE +
> > +                                IPIMSG_VP8_ENC_ID,
> > +       AP_IPIMSG_VP8_ENC_SET_PARAM,
> > +       AP_IPIMSG_VP8_ENC_ENCODE,
> > +       AP_IPIMSG_VP8_ENC_DEINIT,
> > +
> > +       VPU_IPIMSG_H264_ENC_INIT_DONE = VPU_IPIMSG_VENC_BASE +
> > +                                       IPIMSG_H264_ENC_ID,
> > +       VPU_IPIMSG_H264_ENC_SET_PARAM_DONE,
> > +       VPU_IPIMSG_H264_ENC_ENCODE_DONE,
> > +       VPU_IPIMSG_H264_ENC_DEINIT_DONE,
> > +
> > +       VPU_IPIMSG_VP8_ENC_INIT_DONE = VPU_IPIMSG_VENC_BASE +
> > +                                      IPIMSG_VP8_ENC_ID,
> > +       VPU_IPIMSG_VP8_ENC_SET_PARAM_DONE,
> > +       VPU_IPIMSG_VP8_ENC_ENCODE_DONE,
> > +       VPU_IPIMSG_VP8_ENC_DEINIT_DONE,
> > +};
> > +
> > +/**
> > + * struct venc_ap_ipi_msg_init - AP to VPU init cmd structure
> > + * @msg_id:    message id (AP_IPIMSG_XXX_ENC_INIT)
> > + * @venc_inst: AP encoder instance (struct venc_vp8_handle/venc_h264_handle *)
> > + */
> > +struct venc_ap_ipi_msg_init {
> > +       uint32_t msg_id;
> > +       uint32_t reserved;
> > +       uint64_t venc_inst;
> > +};
> > +
> > +/**
> > + * struct venc_ap_ipi_msg_set_param - AP to VPU set_param cmd structure
> > + * @msg_id:    message id (AP_IPIMSG_XXX_ENC_SET_PARAM)
> > + * @inst_id:   VPU encoder instance id (struct venc_vp8_vpu_drv/venc_h264_vpu_drv *)
> > + * @param_id:  parameter id (venc_set_param_type)
> > + * @data_item: number of items in the data array
> > + * @data[8]:   data array to store the set parameters
> > + */
> > +struct venc_ap_ipi_msg_set_param {
> > +       uint32_t msg_id;
> > +       uint32_t inst_id;
> > +       uint32_t param_id;
> > +       uint32_t data_item;
> > +       uint32_t data[8];
> > +};
> > +
> > +/**
> > + * struct venc_ap_ipi_msg_enc - AP to VPU enc cmd structure
> > + * @msg_id:    message id (AP_IPIMSG_XXX_ENC_ENCODE)
> > + * @inst_id:   VPU encoder instance id (struct venc_vp8_vpu_drv/venc_h264_vpu_drv *)
> > + * @bs_mode:   bitstream mode for h264
> > + *             (H264_BS_MODE_SPS/H264_BS_MODE_PPS/H264_BS_MODE_FRAME)
> > + * @input_addr:        pointer to input image buffer plane
> > + * @bs_addr:   pointer to output bit stream buffer
> > + * @bs_size:   bit stream buffer size
> > + */
> > +struct venc_ap_ipi_msg_enc {
> > +       uint32_t msg_id;
> > +       uint32_t inst_id;
> > +       uint32_t bs_mode;
> > +       uint32_t input_addr[3];
> > +       uint32_t bs_addr;
> > +       uint32_t bs_size;
> > +};
> > +
> > +/**
> > + * struct venc_ap_ipi_msg_deinit - AP to VPU deinit cmd structure
> > + * @msg_id:    message id (AP_IPIMSG_XXX_ENC_DEINIT)
> > + * @inst_id:   VPU encoder instance id (struct venc_vp8_vpu_drv/venc_h264_vpu_drv *)
> > + */
> > +struct venc_ap_ipi_msg_deinit {
> > +       uint32_t msg_id;
> > +       uint32_t inst_id;
> > +};
> > +
> > +/**
> > + * enum venc_ipi_msg_status - VPU ack AP cmd status
> > + */
> > +enum venc_ipi_msg_status {
> > +       VENC_IPI_MSG_STATUS_OK,
> > +       VENC_IPI_MSG_STATUS_FAIL,
> > +};
> > +
> > +/**
> > + * struct venc_vpu_ipi_msg_common - VPU ack AP cmd common structure
> > + * @msg_id:    message id (VPU_IPIMSG_XXX_DONE)
> > + * @status:    cmd status (venc_ipi_msg_status)
> > + * @venc_inst: AP encoder instance (struct venc_vp8_handle/venc_h264_handle *)
> > + */
> > +struct venc_vpu_ipi_msg_common {
> > +       uint32_t msg_id;
> > +       uint32_t status;
> > +       uint64_t venc_inst;
> > +};
> > +
> > +/**
> > + * struct venc_vpu_ipi_msg_init - VPU ack AP init cmd structure
> > + * @msg_id:    message id (VPU_IPIMSG_XXX_ENC_SET_PARAM_DONE)
> > + * @status:    cmd status (venc_ipi_msg_status)
> > + * @venc_inst: AP encoder instance (struct venc_vp8_handle/venc_h264_handle *)
> > + * @inst_id:   VPU encoder instance id (struct venc_vp8_vpu_drv/venc_h264_vpu_drv *)
> > + */
> > +struct venc_vpu_ipi_msg_init {
> > +       uint32_t msg_id;
> > +       uint32_t status;
> > +       uint64_t venc_inst;
> > +       uint32_t inst_id;
> > +       uint32_t reserved;
> > +};
> > +
> > +/**
> > + * struct venc_vpu_ipi_msg_set_param - VPU ack AP set_param cmd structure
> > + * @msg_id:    message id (VPU_IPIMSG_XXX_ENC_SET_PARAM_DONE)
> > + * @status:    cmd status (venc_ipi_msg_status)
> > + * @venc_inst: AP encoder instance (struct venc_vp8_handle/venc_h264_handle *)
> > + * @param_id:  parameter id (venc_set_param_type)
> > + * @data_item: number of items in the data array
> > + * @data[6]:   data array to store the return result
> > + */
> > +struct venc_vpu_ipi_msg_set_param {
> > +       uint32_t msg_id;
> > +       uint32_t status;
> > +       uint64_t venc_inst;
> > +       uint32_t param_id;
> > +       uint32_t data_item;
> > +       uint32_t data[6];
> > +};
> > +
> > +/**
> > + * enum venc_ipi_msg_enc_state - Type of encode state
> > + * VEN_IPI_MSG_ENC_STATE_FRAME:        one frame being encoded
> > + * VEN_IPI_MSG_ENC_STATE_PART: bit stream buffer full
> > + * VEN_IPI_MSG_ENC_STATE_SKIP: encoded skip frame
> > + * VEN_IPI_MSG_ENC_STATE_ERROR:        encounter error
> > + */
> > +enum venc_ipi_msg_enc_state {
> > +       VEN_IPI_MSG_ENC_STATE_FRAME,
> > +       VEN_IPI_MSG_ENC_STATE_PART,
> > +       VEN_IPI_MSG_ENC_STATE_SKIP,
> > +       VEN_IPI_MSG_ENC_STATE_ERROR,
> > +};
> > +
> > +/**
> > + * struct venc_vpu_ipi_msg_enc - VPU ack AP enc cmd structure
> > + * @msg_id:    message id (VPU_IPIMSG_XXX_ENC_ENCODE_DONE)
> > + * @status:    cmd status (venc_ipi_msg_status)
> > + * @venc_inst: AP encoder instance (struct venc_vp8_handle/venc_h264_handle *)
> > + * @state:     encode state (venc_ipi_msg_enc_state)
> > + * @key_frame: whether the encoded frame is key frame
> > + * @bs_size:   encoded bitstream size
> > + */
> > +struct venc_vpu_ipi_msg_enc {
> > +       uint32_t msg_id;
> > +       uint32_t status;
> > +       uint64_t venc_inst;
> > +       uint32_t state;
> > +       uint32_t key_frame;
> > +       uint32_t bs_size;
> > +       uint32_t reserved;
> > +};
> > +
> > +/**
> > + * struct venc_vpu_ipi_msg_deinit - VPU ack AP deinit cmd structure
> > + * @msg_id:   message id (VPU_IPIMSG_XXX_ENC_DEINIT_DONE)
> > + * @status:   cmd status (venc_ipi_msg_status)
> > + * @venc_inst: AP encoder instance (struct venc_vp8_handle/venc_h264_handle *)
> > + */
> > +struct venc_vpu_ipi_msg_deinit {
> > +       uint32_t msg_id;
> > +       uint32_t status;
> > +       uint64_t venc_inst;
> > +};
> > +
> > +#endif /* _VENC_IPI_MSG_H_ */
> > diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
> > old mode 100644
> > new mode 100755
> > index ee9d530..3ac35c4
> > --- a/include/uapi/linux/v4l2-controls.h
> > +++ b/include/uapi/linux/v4l2-controls.h
> > @@ -646,6 +646,10 @@ enum v4l2_mpeg_mfc51_video_force_frame_type {
> >  #define V4L2_CID_MPEG_MFC51_VIDEO_H264_NUM_REF_PIC_FOR_P               (V4L2_CID_MPEG_MFC51_BASE+54)
> >
> >
> > +#define V4L2_CID_MPEG_MTK_BASE                                 (V4L2_CTRL_CLASS_MPEG | 0x5500)
> > +#define V4L2_CID_MPEG_MTK_VIDEO_FRAME_SKIP_ENABLE              (V4L2_CID_MPEG_MTK_BASE+0)
> > +#define V4L2_CID_MPEG_MTK_VIDEO_FORCE_FRAME_TYPE_I_FRAME       (V4L2_CID_MPEG_MTK_BASE+1)
> > +
> >  /*  Camera class control IDs */
> >
> >  #define V4L2_CID_CAMERA_CLASS_BASE     (V4L2_CTRL_CLASS_CAMERA | 0x900)
> > --
> > 1.7.9.5
> >
> > --
> > To unsubscribe from this list: send the line "unsubscribe linux-media" in
> > the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html


--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 102+ messages in thread

end of thread, other threads:[~2016-02-24  8:26 UTC | newest]

Thread overview: 102+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-02-04 11:34 [PATCH v4 0/8] Add MT8173 Video Encoder Driver and VPU Driver Tiffany Lin
2016-02-04 11:34 ` Tiffany Lin
2016-02-04 11:34 ` Tiffany Lin
2016-02-04 11:34 ` [PATCH v4 1/8] dt-bindings: Add a binding for Mediatek Video Processor Tiffany Lin
2016-02-04 11:34   ` Tiffany Lin
2016-02-04 11:34   ` Tiffany Lin
2016-02-04 11:34   ` [PATCH v4 2/8] [media] VPU: mediatek: support Mediatek VPU Tiffany Lin
2016-02-04 11:34     ` Tiffany Lin
2016-02-04 11:34     ` Tiffany Lin
2016-02-04 11:34     ` [PATCH v4 3/8] arm64: dts: mediatek: Add node for Mediatek Video Processor Unit Tiffany Lin
2016-02-04 11:34       ` Tiffany Lin
2016-02-04 11:34       ` Tiffany Lin
2016-02-04 11:34       ` [PATCH v4 4/8] dt-bindings: Add a binding for Mediatek Video Encoder Tiffany Lin
2016-02-04 11:34         ` Tiffany Lin
2016-02-04 11:34         ` Tiffany Lin
2016-02-04 11:35         ` [PATCH v4 5/8] [Media] vcodec: mediatek: Add Mediatek V4L2 Video Encoder Driver Tiffany Lin
2016-02-04 11:35           ` Tiffany Lin
2016-02-04 11:35           ` [PATCH v4 6/8] [media] vcodec: mediatek: Add Mediatek VP8 " Tiffany Lin
2016-02-04 11:35             ` Tiffany Lin
2016-02-04 11:35             ` Tiffany Lin
2016-02-04 11:35             ` [PATCH v4 7/8] [media] vcodec: mediatek: Add Mediatek H264 " Tiffany Lin
2016-02-04 11:35               ` Tiffany Lin
2016-02-04 11:35               ` Tiffany Lin
2016-02-04 11:35               ` [PATCH v4 8/8] arm64: dts: mediatek: Add Video Encoder for MT8173 Tiffany Lin
2016-02-04 11:35                 ` Tiffany Lin
2016-02-04 11:35                 ` Tiffany Lin
2016-02-15 11:33               ` [PATCH v4 7/8] [media] vcodec: mediatek: Add Mediatek H264 Video Encoder Driver Hans Verkuil
2016-02-15 11:33                 ` Hans Verkuil
2016-02-16 11:57                 ` pochun lin
2016-02-16 11:57                   ` pochun lin
2016-02-16 11:57                   ` pochun lin
2016-02-15 11:21           ` [PATCH v4 5/8] [Media] vcodec: mediatek: Add Mediatek V4L2 " Hans Verkuil
2016-02-16  6:37             ` tiffany lin
2016-02-16  6:37               ` tiffany lin
2016-02-16  7:44               ` Hans Verkuil
2016-02-16  7:44                 ` Hans Verkuil
2016-02-16  7:44                 ` Hans Verkuil
2016-02-16 13:20                 ` tiffany lin
2016-02-16 13:20                   ` tiffany lin
2016-02-16 13:20                   ` tiffany lin
2016-02-16 13:48                   ` Hans Verkuil
2016-02-16 13:48                     ` Hans Verkuil
2016-02-17  8:01                     ` tiffany lin
2016-02-17  8:01                       ` tiffany lin
2016-02-17  8:01                       ` tiffany lin
2016-02-17  8:31                       ` Hans Verkuil
2016-02-17  8:31                         ` Hans Verkuil
2016-02-17  9:23                         ` tiffany lin
2016-02-17  9:23                           ` tiffany lin
2016-02-17  9:23                           ` tiffany lin
2016-02-20  9:11                 ` tiffany lin
2016-02-20  9:11                   ` tiffany lin
2016-02-20  9:11                   ` tiffany lin
2016-02-20  9:18                   ` Hans Verkuil
2016-02-20  9:18                     ` Hans Verkuil
2016-02-20  9:18                     ` Hans Verkuil
2016-02-22 15:19                     ` tiffany lin
2016-02-22 15:19                       ` tiffany lin
2016-02-22 15:19                       ` tiffany lin
2016-02-17  7:47               ` Hans Verkuil
2016-02-17  7:47                 ` Hans Verkuil
2016-02-17  8:33                 ` tiffany lin
2016-02-17  8:33                   ` tiffany lin
2016-02-17  8:33                   ` tiffany lin
2016-02-23  5:46           ` Wu-Cheng Li (李務誠)
2016-02-24  8:26             ` tiffany lin
2016-02-24  8:26               ` tiffany lin
2016-02-08 18:56         ` [PATCH v4 4/8] dt-bindings: Add a binding for Mediatek Video Encoder Rob Herring
2016-02-08 18:56           ` Rob Herring
2016-02-08 18:56           ` Rob Herring
2016-02-09 11:29         ` Daniel Kurtz
2016-02-09 11:29           ` Daniel Kurtz
2016-02-09 11:29           ` Daniel Kurtz
2016-02-15 10:42           ` Daniel Kurtz
2016-02-15 10:42             ` Daniel Kurtz
2016-02-15 10:42             ` Daniel Kurtz
2016-02-16  2:09             ` tiffany lin
2016-02-16  2:09               ` tiffany lin
2016-02-16  2:09               ` tiffany lin
2016-02-16  2:09               ` tiffany lin
2016-02-15 10:07     ` [PATCH v4 2/8] [media] VPU: mediatek: support Mediatek VPU Hans Verkuil
2016-02-15 10:07       ` Hans Verkuil
2016-02-15 11:20       ` tiffany lin
2016-02-15 11:20         ` tiffany lin
2016-02-15 11:20         ` tiffany lin
2016-02-15 10:13     ` Hans Verkuil
2016-02-15 10:13       ` Hans Verkuil
2016-02-15 11:27       ` tiffany lin
2016-02-15 11:27         ` tiffany lin
2016-02-15 11:27         ` tiffany lin
2016-02-15 13:59     ` Wu-Cheng Li (李務誠)
2016-02-15 13:59       ` Wu-Cheng Li (李務誠)
2016-02-16  9:36       ` andrew-ct chen
2016-02-16  9:36         ` andrew-ct chen
2016-02-16  9:36         ` andrew-ct chen
2016-02-08 18:54   ` [PATCH v4 1/8] dt-bindings: Add a binding for Mediatek Video Processor Rob Herring
2016-02-08 18:54     ` Rob Herring
2016-02-15 12:04 ` [PATCH v4 0/8] Add MT8173 Video Encoder Driver and VPU Driver Hans Verkuil
2016-02-15 12:04   ` Hans Verkuil
2016-02-16  6:46   ` tiffany lin
2016-02-16  6:46     ` tiffany lin
2016-02-16  6:46     ` tiffany lin

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.