* [RESEND RFC/PATCH 0/8] Add MT8173 Video Encoder Driver and VPU Driver
@ 2015-11-17 12:54 ` Tiffany Lin
0 siblings, 0 replies; 84+ messages in thread
From: Tiffany Lin @ 2015-11-17 12:54 UTC (permalink / raw)
To: Rob Herring, Pawel Moll, Mark Rutland, Ian Campbell, Kumar Gala,
Catalin Marinas, Will Deacon, Mauro Carvalho Chehab,
Matthias Brugger, Daniel Kurtz, Sascha Hauer, Hongzhou Yang,
Hans Verkuil, Laurent Pinchart, Sakari Ailus, Geert Uytterhoeven,
Mikhail Ulyanov, Fabien Dessenne, Arnd Bergmann,
Darren Etheridge, Peter Griffin, Benoit Parrot
Cc: Tiffany Lin, Andrew-CT Chen, Eddie Huang, Yingjoe Chen,
James Liao, Daniel Hsiao, devicetree, linux-kernel,
linux-arm-kernel, linux-media, linux-mediatek
==============
Introduction
==============
The purpose of this RFC is to discuss the driver for a hw video codec
embedded in the Mediatek's MT8173 SoCs. Mediatek Video Codec is able to
handle video encoding of in a range of formats.
This RFC also include VPU driver. Mediatek Video Codec driver rely on
VPU driver to load, communicate with VPU.
Internally the driver uses videobuf2 framework and MTK IOMMU and MTK SMI.
MTK IOMMU and MTK SMI have not yet been merged, but we wanted to start
discussion about the driver earlier so it could be merged sooner. The
driver posted here is the initial version, so I suppose it will require
more work.
[1]http://lists.infradead.org/pipermail/linux-mediatek/2015-October/002525.html
==================
Device interface
==================
In principle the driver bases on memory-to-memory framework:
it provides a single video node and each opened file handle gets its own
private context with separate buffer queues. Each context consist of 2
buffer queues: OUTPUT (for source buffers, i.e. raw video frames)
and CAPTURE (for destination buffers, i.e. encoded video frames).
The process of encoding video data from stream is a bit more complicated
than typical memory-to-memory processing. We base on memory-to-memory
framework and add the complicated part in our vb2 and v4l2 callback
functionss. So we can base on well done m2m memory-to-memory framework,
reduce duplicate code and make our driver code simple.
==============================
VPU (Video Processor Unit)
==============================
The VPU driver for hw video codec embedded in Mediatek's MT8173 SOCs.
It is able to handle video decoding/encoding of in a range of formats.
The driver provides with VPU firmware download, memory management and
the communication interface between CPU and VPU.
For VPU initialization, it will create virtual memory for CPU access and
IOMMU address for vcodec hw device access. When a decode/encode instance
opens a device node, vpu driver will download vpu firmware to the device.
A decode/encode instant will decode/encode a frame using VPU
interface to interrupt vpu to handle decoding/encoding jobs.
Please have a look at the code and comments will be very much appreciated.
Andrew-CT Chen (3):
dt-bindings: Add a binding for Mediatek Video Processor Unit
arm64: dts: mediatek: Add node for Mediatek Video Processor Unit
media: platform: mtk-vpu: Support Mediatek VPU
Daniel Hsiao (1):
media: platform: mtk-vcodec: Add Mediatek VP8 Video Encoder Driver
Tiffany Lin (4):
dt-bindings: Add a binding for Mediatek Video Encoder
arm64: dts: mediatek: Add Video Encoder for MT8173
media: platform: mtk-vcodec: Add Mediatek V4L2 Video Encoder Driver
media: platform: mtk-vcodec: Add Mediatek H264 Video Encoder Driver
.../devicetree/bindings/media/mediatek-vcodec.txt | 58 +
.../devicetree/bindings/media/mediatek-vpu.txt | 27 +
arch/arm64/boot/dts/mediatek/mt8173.dtsi | 58 +
drivers/media/platform/Kconfig | 19 +
drivers/media/platform/Makefile | 5 +
drivers/media/platform/mtk-vcodec/Kconfig | 5 +
drivers/media/platform/mtk-vcodec/Makefile | 12 +
drivers/media/platform/mtk-vcodec/common/Makefile | 12 +
.../media/platform/mtk-vcodec/common/venc_drv_if.c | 159 ++
.../media/platform/mtk-vcodec/h264_enc/Makefile | 9 +
.../platform/mtk-vcodec/h264_enc/venc_h264_if.c | 529 ++++++
.../platform/mtk-vcodec/h264_enc/venc_h264_if.h | 53 +
.../platform/mtk-vcodec/h264_enc/venc_h264_vpu.c | 341 ++++
.../platform/mtk-vcodec/include/venc_drv_base.h | 68 +
.../platform/mtk-vcodec/include/venc_drv_if.h | 187 +++
.../platform/mtk-vcodec/include/venc_ipi_msg.h | 212 +++
drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h | 441 +++++
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c | 1773 ++++++++++++++++++++
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h | 28 +
.../media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c | 535 ++++++
.../media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c | 122 ++
.../media/platform/mtk-vcodec/mtk_vcodec_intr.c | 110 ++
.../media/platform/mtk-vcodec/mtk_vcodec_intr.h | 30 +
drivers/media/platform/mtk-vcodec/mtk_vcodec_pm.h | 26 +
.../media/platform/mtk-vcodec/mtk_vcodec_util.c | 106 ++
.../media/platform/mtk-vcodec/mtk_vcodec_util.h | 66 +
drivers/media/platform/mtk-vcodec/vp8_enc/Makefile | 9 +
.../platform/mtk-vcodec/vp8_enc/venc_vp8_if.c | 371 ++++
.../platform/mtk-vcodec/vp8_enc/venc_vp8_if.h | 48 +
.../platform/mtk-vcodec/vp8_enc/venc_vp8_vpu.c | 245 +++
drivers/media/platform/mtk-vpu/Makefile | 1 +
.../platform/mtk-vpu/h264_enc/venc_h264_vpu.h | 127 ++
.../media/platform/mtk-vpu/include/venc_ipi_msg.h | 212 +++
drivers/media/platform/mtk-vpu/mtk_vpu_core.c | 823 +++++++++
drivers/media/platform/mtk-vpu/mtk_vpu_core.h | 161 ++
.../media/platform/mtk-vpu/vp8_enc/venc_vp8_vpu.h | 119 ++
36 files changed, 7107 insertions(+)
create mode 100644 Documentation/devicetree/bindings/media/mediatek-vcodec.txt
create mode 100644 Documentation/devicetree/bindings/media/mediatek-vpu.txt
create mode 100644 drivers/media/platform/mtk-vcodec/Kconfig
create mode 100644 drivers/media/platform/mtk-vcodec/Makefile
create mode 100644 drivers/media/platform/mtk-vcodec/common/Makefile
create mode 100644 drivers/media/platform/mtk-vcodec/common/venc_drv_if.c
create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/Makefile
create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.c
create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.h
create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.c
create mode 100644 drivers/media/platform/mtk-vcodec/include/venc_drv_base.h
create mode 100644 drivers/media/platform/mtk-vcodec/include/venc_drv_if.h
create mode 100644 drivers/media/platform/mtk-vcodec/include/venc_ipi_msg.h
create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h
create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.c
create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.h
create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_pm.h
create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c
create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
create mode 100644 drivers/media/platform/mtk-vcodec/vp8_enc/Makefile
create mode 100644 drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_if.c
create mode 100644 drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_if.h
create mode 100644 drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_vpu.c
create mode 100644 drivers/media/platform/mtk-vpu/Makefile
create mode 100644 drivers/media/platform/mtk-vpu/h264_enc/venc_h264_vpu.h
create mode 100644 drivers/media/platform/mtk-vpu/include/venc_ipi_msg.h
create mode 100644 drivers/media/platform/mtk-vpu/mtk_vpu_core.c
create mode 100644 drivers/media/platform/mtk-vpu/mtk_vpu_core.h
create mode 100644 drivers/media/platform/mtk-vpu/vp8_enc/venc_vp8_vpu.h
--
1.7.9.5
^ permalink raw reply [flat|nested] 84+ messages in thread
* [RESEND RFC/PATCH 0/8] Add MT8173 Video Encoder Driver and VPU Driver
@ 2015-11-17 12:54 ` Tiffany Lin
0 siblings, 0 replies; 84+ messages in thread
From: Tiffany Lin @ 2015-11-17 12:54 UTC (permalink / raw)
To: Rob Herring, Pawel Moll, Mark Rutland, Ian Campbell, Kumar Gala,
Catalin Marinas, Will Deacon, Mauro Carvalho Chehab,
Matthias Brugger, Daniel Kurtz, Sascha Hauer, Hongzhou Yang,
Hans Verkuil, Laurent Pinchart, Sakari Ailus, Geert Uytterhoeven,
Mikhail Ulyanov, Fabien Dessenne, Arnd Bergmann,
Darren Etheridge, Peter Griffin, Benoit Parrot
Cc: Tiffany Lin, Andrew-CT Chen, Eddie Huang, Yingjoe Chen,
James Liao, Daniel Hsiao, devicetree, linux-kernel,
linux-arm-kernel, linux-media, linux-mediatek
==============
Introduction
==============
The purpose of this RFC is to discuss the driver for a hw video codec
embedded in the Mediatek's MT8173 SoCs. Mediatek Video Codec is able to
handle video encoding of in a range of formats.
This RFC also include VPU driver. Mediatek Video Codec driver rely on
VPU driver to load, communicate with VPU.
Internally the driver uses videobuf2 framework and MTK IOMMU and MTK SMI.
MTK IOMMU and MTK SMI have not yet been merged, but we wanted to start
discussion about the driver earlier so it could be merged sooner. The
driver posted here is the initial version, so I suppose it will require
more work.
[1]http://lists.infradead.org/pipermail/linux-mediatek/2015-October/002525.html
==================
Device interface
==================
In principle the driver bases on memory-to-memory framework:
it provides a single video node and each opened file handle gets its own
private context with separate buffer queues. Each context consist of 2
buffer queues: OUTPUT (for source buffers, i.e. raw video frames)
and CAPTURE (for destination buffers, i.e. encoded video frames).
The process of encoding video data from stream is a bit more complicated
than typical memory-to-memory processing. We base on memory-to-memory
framework and add the complicated part in our vb2 and v4l2 callback
functionss. So we can base on well done m2m memory-to-memory framework,
reduce duplicate code and make our driver code simple.
==============================
VPU (Video Processor Unit)
==============================
The VPU driver for hw video codec embedded in Mediatek's MT8173 SOCs.
It is able to handle video decoding/encoding of in a range of formats.
The driver provides with VPU firmware download, memory management and
the communication interface between CPU and VPU.
For VPU initialization, it will create virtual memory for CPU access and
IOMMU address for vcodec hw device access. When a decode/encode instance
opens a device node, vpu driver will download vpu firmware to the device.
A decode/encode instant will decode/encode a frame using VPU
interface to interrupt vpu to handle decoding/encoding jobs.
Please have a look at the code and comments will be very much appreciated.
Andrew-CT Chen (3):
dt-bindings: Add a binding for Mediatek Video Processor Unit
arm64: dts: mediatek: Add node for Mediatek Video Processor Unit
media: platform: mtk-vpu: Support Mediatek VPU
Daniel Hsiao (1):
media: platform: mtk-vcodec: Add Mediatek VP8 Video Encoder Driver
Tiffany Lin (4):
dt-bindings: Add a binding for Mediatek Video Encoder
arm64: dts: mediatek: Add Video Encoder for MT8173
media: platform: mtk-vcodec: Add Mediatek V4L2 Video Encoder Driver
media: platform: mtk-vcodec: Add Mediatek H264 Video Encoder Driver
.../devicetree/bindings/media/mediatek-vcodec.txt | 58 +
.../devicetree/bindings/media/mediatek-vpu.txt | 27 +
arch/arm64/boot/dts/mediatek/mt8173.dtsi | 58 +
drivers/media/platform/Kconfig | 19 +
drivers/media/platform/Makefile | 5 +
drivers/media/platform/mtk-vcodec/Kconfig | 5 +
drivers/media/platform/mtk-vcodec/Makefile | 12 +
drivers/media/platform/mtk-vcodec/common/Makefile | 12 +
.../media/platform/mtk-vcodec/common/venc_drv_if.c | 159 ++
.../media/platform/mtk-vcodec/h264_enc/Makefile | 9 +
.../platform/mtk-vcodec/h264_enc/venc_h264_if.c | 529 ++++++
.../platform/mtk-vcodec/h264_enc/venc_h264_if.h | 53 +
.../platform/mtk-vcodec/h264_enc/venc_h264_vpu.c | 341 ++++
.../platform/mtk-vcodec/include/venc_drv_base.h | 68 +
.../platform/mtk-vcodec/include/venc_drv_if.h | 187 +++
.../platform/mtk-vcodec/include/venc_ipi_msg.h | 212 +++
drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h | 441 +++++
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c | 1773 ++++++++++++++++++++
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h | 28 +
.../media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c | 535 ++++++
.../media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c | 122 ++
.../media/platform/mtk-vcodec/mtk_vcodec_intr.c | 110 ++
.../media/platform/mtk-vcodec/mtk_vcodec_intr.h | 30 +
drivers/media/platform/mtk-vcodec/mtk_vcodec_pm.h | 26 +
.../media/platform/mtk-vcodec/mtk_vcodec_util.c | 106 ++
.../media/platform/mtk-vcodec/mtk_vcodec_util.h | 66 +
drivers/media/platform/mtk-vcodec/vp8_enc/Makefile | 9 +
.../platform/mtk-vcodec/vp8_enc/venc_vp8_if.c | 371 ++++
.../platform/mtk-vcodec/vp8_enc/venc_vp8_if.h | 48 +
.../platform/mtk-vcodec/vp8_enc/venc_vp8_vpu.c | 245 +++
drivers/media/platform/mtk-vpu/Makefile | 1 +
.../platform/mtk-vpu/h264_enc/venc_h264_vpu.h | 127 ++
.../media/platform/mtk-vpu/include/venc_ipi_msg.h | 212 +++
drivers/media/platform/mtk-vpu/mtk_vpu_core.c | 823 +++++++++
drivers/media/platform/mtk-vpu/mtk_vpu_core.h | 161 ++
.../media/platform/mtk-vpu/vp8_enc/venc_vp8_vpu.h | 119 ++
36 files changed, 7107 insertions(+)
create mode 100644 Documentation/devicetree/bindings/media/mediatek-vcodec.txt
create mode 100644 Documentation/devicetree/bindings/media/mediatek-vpu.txt
create mode 100644 drivers/media/platform/mtk-vcodec/Kconfig
create mode 100644 drivers/media/platform/mtk-vcodec/Makefile
create mode 100644 drivers/media/platform/mtk-vcodec/common/Makefile
create mode 100644 drivers/media/platform/mtk-vcodec/common/venc_drv_if.c
create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/Makefile
create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.c
create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.h
create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.c
create mode 100644 drivers/media/platform/mtk-vcodec/include/venc_drv_base.h
create mode 100644 drivers/media/platform/mtk-vcodec/include/venc_drv_if.h
create mode 100644 drivers/media/platform/mtk-vcodec/include/venc_ipi_msg.h
create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h
create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.c
create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.h
create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_pm.h
create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c
create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
create mode 100644 drivers/media/platform/mtk-vcodec/vp8_enc/Makefile
create mode 100644 drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_if.c
create mode 100644 drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_if.h
create mode 100644 drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_vpu.c
create mode 100644 drivers/media/platform/mtk-vpu/Makefile
create mode 100644 drivers/media/platform/mtk-vpu/h264_enc/venc_h264_vpu.h
create mode 100644 drivers/media/platform/mtk-vpu/include/venc_ipi_msg.h
create mode 100644 drivers/media/platform/mtk-vpu/mtk_vpu_core.c
create mode 100644 drivers/media/platform/mtk-vpu/mtk_vpu_core.h
create mode 100644 drivers/media/platform/mtk-vpu/vp8_enc/venc_vp8_vpu.h
--
1.7.9.5
^ permalink raw reply [flat|nested] 84+ messages in thread
* [RESEND RFC/PATCH 0/8] Add MT8173 Video Encoder Driver and VPU Driver
@ 2015-11-17 12:54 ` Tiffany Lin
0 siblings, 0 replies; 84+ messages in thread
From: Tiffany Lin @ 2015-11-17 12:54 UTC (permalink / raw)
To: linux-arm-kernel
==============
Introduction
==============
The purpose of this RFC is to discuss the driver for a hw video codec
embedded in the Mediatek's MT8173 SoCs. Mediatek Video Codec is able to
handle video encoding of in a range of formats.
This RFC also include VPU driver. Mediatek Video Codec driver rely on
VPU driver to load, communicate with VPU.
Internally the driver uses videobuf2 framework and MTK IOMMU and MTK SMI.
MTK IOMMU and MTK SMI have not yet been merged, but we wanted to start
discussion about the driver earlier so it could be merged sooner. The
driver posted here is the initial version, so I suppose it will require
more work.
[1]http://lists.infradead.org/pipermail/linux-mediatek/2015-October/002525.html
==================
Device interface
==================
In principle the driver bases on memory-to-memory framework:
it provides a single video node and each opened file handle gets its own
private context with separate buffer queues. Each context consist of 2
buffer queues: OUTPUT (for source buffers, i.e. raw video frames)
and CAPTURE (for destination buffers, i.e. encoded video frames).
The process of encoding video data from stream is a bit more complicated
than typical memory-to-memory processing. We base on memory-to-memory
framework and add the complicated part in our vb2 and v4l2 callback
functionss. So we can base on well done m2m memory-to-memory framework,
reduce duplicate code and make our driver code simple.
==============================
VPU (Video Processor Unit)
==============================
The VPU driver for hw video codec embedded in Mediatek's MT8173 SOCs.
It is able to handle video decoding/encoding of in a range of formats.
The driver provides with VPU firmware download, memory management and
the communication interface between CPU and VPU.
For VPU initialization, it will create virtual memory for CPU access and
IOMMU address for vcodec hw device access. When a decode/encode instance
opens a device node, vpu driver will download vpu firmware to the device.
A decode/encode instant will decode/encode a frame using VPU
interface to interrupt vpu to handle decoding/encoding jobs.
Please have a look at the code and comments will be very much appreciated.
Andrew-CT Chen (3):
dt-bindings: Add a binding for Mediatek Video Processor Unit
arm64: dts: mediatek: Add node for Mediatek Video Processor Unit
media: platform: mtk-vpu: Support Mediatek VPU
Daniel Hsiao (1):
media: platform: mtk-vcodec: Add Mediatek VP8 Video Encoder Driver
Tiffany Lin (4):
dt-bindings: Add a binding for Mediatek Video Encoder
arm64: dts: mediatek: Add Video Encoder for MT8173
media: platform: mtk-vcodec: Add Mediatek V4L2 Video Encoder Driver
media: platform: mtk-vcodec: Add Mediatek H264 Video Encoder Driver
.../devicetree/bindings/media/mediatek-vcodec.txt | 58 +
.../devicetree/bindings/media/mediatek-vpu.txt | 27 +
arch/arm64/boot/dts/mediatek/mt8173.dtsi | 58 +
drivers/media/platform/Kconfig | 19 +
drivers/media/platform/Makefile | 5 +
drivers/media/platform/mtk-vcodec/Kconfig | 5 +
drivers/media/platform/mtk-vcodec/Makefile | 12 +
drivers/media/platform/mtk-vcodec/common/Makefile | 12 +
.../media/platform/mtk-vcodec/common/venc_drv_if.c | 159 ++
.../media/platform/mtk-vcodec/h264_enc/Makefile | 9 +
.../platform/mtk-vcodec/h264_enc/venc_h264_if.c | 529 ++++++
.../platform/mtk-vcodec/h264_enc/venc_h264_if.h | 53 +
.../platform/mtk-vcodec/h264_enc/venc_h264_vpu.c | 341 ++++
.../platform/mtk-vcodec/include/venc_drv_base.h | 68 +
.../platform/mtk-vcodec/include/venc_drv_if.h | 187 +++
.../platform/mtk-vcodec/include/venc_ipi_msg.h | 212 +++
drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h | 441 +++++
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c | 1773 ++++++++++++++++++++
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h | 28 +
.../media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c | 535 ++++++
.../media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c | 122 ++
.../media/platform/mtk-vcodec/mtk_vcodec_intr.c | 110 ++
.../media/platform/mtk-vcodec/mtk_vcodec_intr.h | 30 +
drivers/media/platform/mtk-vcodec/mtk_vcodec_pm.h | 26 +
.../media/platform/mtk-vcodec/mtk_vcodec_util.c | 106 ++
.../media/platform/mtk-vcodec/mtk_vcodec_util.h | 66 +
drivers/media/platform/mtk-vcodec/vp8_enc/Makefile | 9 +
.../platform/mtk-vcodec/vp8_enc/venc_vp8_if.c | 371 ++++
.../platform/mtk-vcodec/vp8_enc/venc_vp8_if.h | 48 +
.../platform/mtk-vcodec/vp8_enc/venc_vp8_vpu.c | 245 +++
drivers/media/platform/mtk-vpu/Makefile | 1 +
.../platform/mtk-vpu/h264_enc/venc_h264_vpu.h | 127 ++
.../media/platform/mtk-vpu/include/venc_ipi_msg.h | 212 +++
drivers/media/platform/mtk-vpu/mtk_vpu_core.c | 823 +++++++++
drivers/media/platform/mtk-vpu/mtk_vpu_core.h | 161 ++
.../media/platform/mtk-vpu/vp8_enc/venc_vp8_vpu.h | 119 ++
36 files changed, 7107 insertions(+)
create mode 100644 Documentation/devicetree/bindings/media/mediatek-vcodec.txt
create mode 100644 Documentation/devicetree/bindings/media/mediatek-vpu.txt
create mode 100644 drivers/media/platform/mtk-vcodec/Kconfig
create mode 100644 drivers/media/platform/mtk-vcodec/Makefile
create mode 100644 drivers/media/platform/mtk-vcodec/common/Makefile
create mode 100644 drivers/media/platform/mtk-vcodec/common/venc_drv_if.c
create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/Makefile
create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.c
create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.h
create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.c
create mode 100644 drivers/media/platform/mtk-vcodec/include/venc_drv_base.h
create mode 100644 drivers/media/platform/mtk-vcodec/include/venc_drv_if.h
create mode 100644 drivers/media/platform/mtk-vcodec/include/venc_ipi_msg.h
create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h
create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.c
create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.h
create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_pm.h
create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c
create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
create mode 100644 drivers/media/platform/mtk-vcodec/vp8_enc/Makefile
create mode 100644 drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_if.c
create mode 100644 drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_if.h
create mode 100644 drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_vpu.c
create mode 100644 drivers/media/platform/mtk-vpu/Makefile
create mode 100644 drivers/media/platform/mtk-vpu/h264_enc/venc_h264_vpu.h
create mode 100644 drivers/media/platform/mtk-vpu/include/venc_ipi_msg.h
create mode 100644 drivers/media/platform/mtk-vpu/mtk_vpu_core.c
create mode 100644 drivers/media/platform/mtk-vpu/mtk_vpu_core.h
create mode 100644 drivers/media/platform/mtk-vpu/vp8_enc/venc_vp8_vpu.h
--
1.7.9.5
^ permalink raw reply [flat|nested] 84+ messages in thread
* [RESEND RFC/PATCH 1/8] dt-bindings: Add a binding for Mediatek Video Processor Unit
@ 2015-11-17 12:54 ` Tiffany Lin
0 siblings, 0 replies; 84+ messages in thread
From: Tiffany Lin @ 2015-11-17 12:54 UTC (permalink / raw)
To: Rob Herring, Pawel Moll, Mark Rutland, Ian Campbell, Kumar Gala,
Catalin Marinas, Will Deacon, Mauro Carvalho Chehab,
Matthias Brugger, Daniel Kurtz, Sascha Hauer, Hongzhou Yang,
Hans Verkuil, Laurent Pinchart, Sakari Ailus, Geert Uytterhoeven,
Mikhail Ulyanov, Fabien Dessenne, Arnd Bergmann,
Darren Etheridge, Peter Griffin, Benoit Parrot
Cc: Tiffany Lin, Andrew-CT Chen, Eddie Huang, Yingjoe Chen,
James Liao, Daniel Hsiao, devicetree, linux-kernel,
linux-arm-kernel, linux-media, linux-mediatek
From: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
Add a DT binding documentation of Video Processor Unit for the
MT8173 SoC from Mediatek.
Signed-off-by: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
---
.../devicetree/bindings/media/mediatek-vpu.txt | 27 ++++++++++++++++++++
1 file changed, 27 insertions(+)
create mode 100644 Documentation/devicetree/bindings/media/mediatek-vpu.txt
diff --git a/Documentation/devicetree/bindings/media/mediatek-vpu.txt b/Documentation/devicetree/bindings/media/mediatek-vpu.txt
new file mode 100644
index 0000000..99a4e5e
--- /dev/null
+++ b/Documentation/devicetree/bindings/media/mediatek-vpu.txt
@@ -0,0 +1,27 @@
+* Mediatek Video Processor Unit
+
+Video Processor Unit is a HW video controller. It controls HW Codec including
+H.264/VP8/VP9 Decode, H.264/VP8 Encode and Image Processor (scale/rotate/color convert).
+
+Required properties:
+ - compatible: "mediatek,mt8173-vpu"
+ - reg: Must contain an entry for each entry in reg-names.
+ - reg-names: Must include the following entries:
+ "sram": SRAM base
+ "cfg_reg": Main configuration registers base
+ - interrupts: interrupt number to the cpu.
+ - clocks : clock name from clock manager
+ - clock-names: the clocks of the VPU H/W
+ - iommus : phandle and IOMMU spcifier for the IOMMU that serves the VPU.
+
+Example:
+ vpu: vpu@10020000 {
+ compatible = "mediatek,mt8173-vpu";
+ reg = <0 0x10020000 0 0x30000>,
+ <0 0x10050000 0 0x100>;
+ reg-names = "sram", "cfg_reg";
+ interrupts = <GIC_SPI 166 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&topckgen TOP_SCP_SEL>;
+ clock-names = "main";
+ iommus = <&iommu M4U_PORT_VENC_RCPU>;
+ };
--
1.7.9.5
^ permalink raw reply related [flat|nested] 84+ messages in thread
* [RESEND RFC/PATCH 1/8] dt-bindings: Add a binding for Mediatek Video Processor Unit
@ 2015-11-17 12:54 ` Tiffany Lin
0 siblings, 0 replies; 84+ messages in thread
From: Tiffany Lin @ 2015-11-17 12:54 UTC (permalink / raw)
To: Rob Herring, Pawel Moll, Mark Rutland, Ian Campbell, Kumar Gala,
Catalin Marinas, Will Deacon, Mauro Carvalho Chehab,
Matthias Brugger, Daniel Kurtz, Sascha Hauer, Hongzhou Yang,
Hans Verkuil, Laurent Pinchart, Sakari Ailus, Geert Uytterhoeven,
Mikhail Ulyanov, Fabien Dessenne, Arnd Bergmann,
Darren Etheridge, Peter Griffin, Benoit Parrot
Cc: Tiffany Lin, Andrew-CT Chen, Eddie Huang, Yingjoe Chen,
James Liao, Daniel Hsiao, devicetree-u79uwXL29TY76Z2rM5mHXA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA,
linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
linux-media-u79uwXL29TY76Z2rM5mHXA,
linux-mediatek-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r
From: Andrew-CT Chen <andrew-ct.chen-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
Add a DT binding documentation of Video Processor Unit for the
MT8173 SoC from Mediatek.
Signed-off-by: Andrew-CT Chen <andrew-ct.chen-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
---
.../devicetree/bindings/media/mediatek-vpu.txt | 27 ++++++++++++++++++++
1 file changed, 27 insertions(+)
create mode 100644 Documentation/devicetree/bindings/media/mediatek-vpu.txt
diff --git a/Documentation/devicetree/bindings/media/mediatek-vpu.txt b/Documentation/devicetree/bindings/media/mediatek-vpu.txt
new file mode 100644
index 0000000..99a4e5e
--- /dev/null
+++ b/Documentation/devicetree/bindings/media/mediatek-vpu.txt
@@ -0,0 +1,27 @@
+* Mediatek Video Processor Unit
+
+Video Processor Unit is a HW video controller. It controls HW Codec including
+H.264/VP8/VP9 Decode, H.264/VP8 Encode and Image Processor (scale/rotate/color convert).
+
+Required properties:
+ - compatible: "mediatek,mt8173-vpu"
+ - reg: Must contain an entry for each entry in reg-names.
+ - reg-names: Must include the following entries:
+ "sram": SRAM base
+ "cfg_reg": Main configuration registers base
+ - interrupts: interrupt number to the cpu.
+ - clocks : clock name from clock manager
+ - clock-names: the clocks of the VPU H/W
+ - iommus : phandle and IOMMU spcifier for the IOMMU that serves the VPU.
+
+Example:
+ vpu: vpu@10020000 {
+ compatible = "mediatek,mt8173-vpu";
+ reg = <0 0x10020000 0 0x30000>,
+ <0 0x10050000 0 0x100>;
+ reg-names = "sram", "cfg_reg";
+ interrupts = <GIC_SPI 166 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&topckgen TOP_SCP_SEL>;
+ clock-names = "main";
+ iommus = <&iommu M4U_PORT_VENC_RCPU>;
+ };
--
1.7.9.5
--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related [flat|nested] 84+ messages in thread
* [RESEND RFC/PATCH 1/8] dt-bindings: Add a binding for Mediatek Video Processor Unit
@ 2015-11-17 12:54 ` Tiffany Lin
0 siblings, 0 replies; 84+ messages in thread
From: Tiffany Lin @ 2015-11-17 12:54 UTC (permalink / raw)
To: linux-arm-kernel
From: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
Add a DT binding documentation of Video Processor Unit for the
MT8173 SoC from Mediatek.
Signed-off-by: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
---
.../devicetree/bindings/media/mediatek-vpu.txt | 27 ++++++++++++++++++++
1 file changed, 27 insertions(+)
create mode 100644 Documentation/devicetree/bindings/media/mediatek-vpu.txt
diff --git a/Documentation/devicetree/bindings/media/mediatek-vpu.txt b/Documentation/devicetree/bindings/media/mediatek-vpu.txt
new file mode 100644
index 0000000..99a4e5e
--- /dev/null
+++ b/Documentation/devicetree/bindings/media/mediatek-vpu.txt
@@ -0,0 +1,27 @@
+* Mediatek Video Processor Unit
+
+Video Processor Unit is a HW video controller. It controls HW Codec including
+H.264/VP8/VP9 Decode, H.264/VP8 Encode and Image Processor (scale/rotate/color convert).
+
+Required properties:
+ - compatible: "mediatek,mt8173-vpu"
+ - reg: Must contain an entry for each entry in reg-names.
+ - reg-names: Must include the following entries:
+ "sram": SRAM base
+ "cfg_reg": Main configuration registers base
+ - interrupts: interrupt number to the cpu.
+ - clocks : clock name from clock manager
+ - clock-names: the clocks of the VPU H/W
+ - iommus : phandle and IOMMU spcifier for the IOMMU that serves the VPU.
+
+Example:
+ vpu: vpu at 10020000 {
+ compatible = "mediatek,mt8173-vpu";
+ reg = <0 0x10020000 0 0x30000>,
+ <0 0x10050000 0 0x100>;
+ reg-names = "sram", "cfg_reg";
+ interrupts = <GIC_SPI 166 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&topckgen TOP_SCP_SEL>;
+ clock-names = "main";
+ iommus = <&iommu M4U_PORT_VENC_RCPU>;
+ };
--
1.7.9.5
^ permalink raw reply related [flat|nested] 84+ messages in thread
* [RESEND RFC/PATCH 2/8] arm64: dts: mediatek: Add node for Mediatek Video Processor Unit
2015-11-17 12:54 ` Tiffany Lin
(?)
@ 2015-11-17 12:54 ` Tiffany Lin
-1 siblings, 0 replies; 84+ messages in thread
From: Tiffany Lin @ 2015-11-17 12:54 UTC (permalink / raw)
To: Rob Herring, Pawel Moll, Mark Rutland, Ian Campbell, Kumar Gala,
Catalin Marinas, Will Deacon, Mauro Carvalho Chehab,
Matthias Brugger, Daniel Kurtz, Sascha Hauer, Hongzhou Yang,
Hans Verkuil, Laurent Pinchart, Sakari Ailus, Geert Uytterhoeven,
Mikhail Ulyanov, Fabien Dessenne, Arnd Bergmann,
Darren Etheridge, Peter Griffin, Benoit Parrot
Cc: Tiffany Lin, Andrew-CT Chen, Eddie Huang, Yingjoe Chen,
James Liao, Daniel Hsiao, devicetree, linux-kernel,
linux-arm-kernel, linux-media, linux-mediatek
From: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
add VPU drivers for MT8173
Signed-off-by: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
---
arch/arm64/boot/dts/mediatek/mt8173.dtsi | 11 +++++++++++
1 file changed, 11 insertions(+)
diff --git a/arch/arm64/boot/dts/mediatek/mt8173.dtsi b/arch/arm64/boot/dts/mediatek/mt8173.dtsi
index 4dd5f93..098c15e 100644
--- a/arch/arm64/boot/dts/mediatek/mt8173.dtsi
+++ b/arch/arm64/boot/dts/mediatek/mt8173.dtsi
@@ -258,6 +258,17 @@
clock-names = "spi", "wrap";
};
+ vpu: vpu@10020000 {
+ compatible = "mediatek,mt8173-vpu";
+ reg = <0 0x10020000 0 0x30000>,
+ <0 0x10050000 0 0x100>;
+ reg-names = "sram", "cfg_reg";
+ interrupts = <GIC_SPI 166 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&topckgen CLK_TOP_SCP_SEL>;
+ clock-names = "main";
+ iommus = <&iommu M4U_LARB3_ID M4U_PORT_VENC_RCPU>;
+ };
+
sysirq: intpol-controller@10200620 {
compatible = "mediatek,mt8173-sysirq",
"mediatek,mt6577-sysirq";
--
1.7.9.5
^ permalink raw reply related [flat|nested] 84+ messages in thread
* [RESEND RFC/PATCH 2/8] arm64: dts: mediatek: Add node for Mediatek Video Processor Unit
@ 2015-11-17 12:54 ` Tiffany Lin
0 siblings, 0 replies; 84+ messages in thread
From: Tiffany Lin @ 2015-11-17 12:54 UTC (permalink / raw)
To: Rob Herring, Pawel Moll, Mark Rutland, Ian Campbell, Kumar Gala,
Catalin Marinas, Will Deacon, Mauro Carvalho Chehab,
Matthias Brugger, Daniel Kurtz, Sascha Hauer, Hongzhou Yang,
Hans Verkuil, Laurent Pinchart, Sakari Ailus, Geert Uytterhoeven,
Mikhail Ulyanov, Fabien Dessenne, Arnd Bergmann,
Darren Etheridge, Peter Griffin, Benoit Parrot
Cc: Tiffany Lin, Andrew-CT Chen, Eddie Huang, Yingjoe Chen,
James Liao, Daniel Hsiao, devicetree, linux-kernel,
linux-arm-kernel, linux-media, linux-mediatek
From: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
add VPU drivers for MT8173
Signed-off-by: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
---
arch/arm64/boot/dts/mediatek/mt8173.dtsi | 11 +++++++++++
1 file changed, 11 insertions(+)
diff --git a/arch/arm64/boot/dts/mediatek/mt8173.dtsi b/arch/arm64/boot/dts/mediatek/mt8173.dtsi
index 4dd5f93..098c15e 100644
--- a/arch/arm64/boot/dts/mediatek/mt8173.dtsi
+++ b/arch/arm64/boot/dts/mediatek/mt8173.dtsi
@@ -258,6 +258,17 @@
clock-names = "spi", "wrap";
};
+ vpu: vpu@10020000 {
+ compatible = "mediatek,mt8173-vpu";
+ reg = <0 0x10020000 0 0x30000>,
+ <0 0x10050000 0 0x100>;
+ reg-names = "sram", "cfg_reg";
+ interrupts = <GIC_SPI 166 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&topckgen CLK_TOP_SCP_SEL>;
+ clock-names = "main";
+ iommus = <&iommu M4U_LARB3_ID M4U_PORT_VENC_RCPU>;
+ };
+
sysirq: intpol-controller@10200620 {
compatible = "mediatek,mt8173-sysirq",
"mediatek,mt6577-sysirq";
--
1.7.9.5
^ permalink raw reply related [flat|nested] 84+ messages in thread
* [RESEND RFC/PATCH 2/8] arm64: dts: mediatek: Add node for Mediatek Video Processor Unit
@ 2015-11-17 12:54 ` Tiffany Lin
0 siblings, 0 replies; 84+ messages in thread
From: Tiffany Lin @ 2015-11-17 12:54 UTC (permalink / raw)
To: linux-arm-kernel
From: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
add VPU drivers for MT8173
Signed-off-by: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
---
arch/arm64/boot/dts/mediatek/mt8173.dtsi | 11 +++++++++++
1 file changed, 11 insertions(+)
diff --git a/arch/arm64/boot/dts/mediatek/mt8173.dtsi b/arch/arm64/boot/dts/mediatek/mt8173.dtsi
index 4dd5f93..098c15e 100644
--- a/arch/arm64/boot/dts/mediatek/mt8173.dtsi
+++ b/arch/arm64/boot/dts/mediatek/mt8173.dtsi
@@ -258,6 +258,17 @@
clock-names = "spi", "wrap";
};
+ vpu: vpu at 10020000 {
+ compatible = "mediatek,mt8173-vpu";
+ reg = <0 0x10020000 0 0x30000>,
+ <0 0x10050000 0 0x100>;
+ reg-names = "sram", "cfg_reg";
+ interrupts = <GIC_SPI 166 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&topckgen CLK_TOP_SCP_SEL>;
+ clock-names = "main";
+ iommus = <&iommu M4U_LARB3_ID M4U_PORT_VENC_RCPU>;
+ };
+
sysirq: intpol-controller at 10200620 {
compatible = "mediatek,mt8173-sysirq",
"mediatek,mt6577-sysirq";
--
1.7.9.5
^ permalink raw reply related [flat|nested] 84+ messages in thread
* [RESEND RFC/PATCH 3/8] media: platform: mtk-vpu: Support Mediatek VPU
@ 2015-11-17 12:54 ` Tiffany Lin
0 siblings, 0 replies; 84+ messages in thread
From: Tiffany Lin @ 2015-11-17 12:54 UTC (permalink / raw)
To: Rob Herring, Pawel Moll, Mark Rutland, Ian Campbell, Kumar Gala,
Catalin Marinas, Will Deacon, Mauro Carvalho Chehab,
Matthias Brugger, Daniel Kurtz, Sascha Hauer, Hongzhou Yang,
Hans Verkuil, Laurent Pinchart, Sakari Ailus, Geert Uytterhoeven,
Mikhail Ulyanov, Fabien Dessenne, Arnd Bergmann,
Darren Etheridge, Peter Griffin, Benoit Parrot
Cc: Tiffany Lin, Andrew-CT Chen, Eddie Huang, Yingjoe Chen,
James Liao, Daniel Hsiao, devicetree, linux-kernel,
linux-arm-kernel, linux-media, linux-mediatek
From: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
The VPU driver for hw video codec embedded in Mediatek's MT8173 SOCs.
It is able to handle video decoding/encoding of in a range of formats.
The driver provides with VPU firmware download, memory management and
the communication interface between CPU and VPU.
For VPU initialization, it will create virtual memory for CPU access and
IOMMU address for vcodec hw device access. When a decode/encode instance
opens a device node, vpu driver will download vpu firmware to the device.
A decode/encode instant will decode/encode a frame using VPU
interface to interrupt vpu to handle decoding/encoding jobs.
Signed-off-by: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
---
drivers/media/platform/Kconfig | 6 +
drivers/media/platform/Makefile | 2 +
drivers/media/platform/mtk-vpu/Makefile | 1 +
.../platform/mtk-vpu/h264_enc/venc_h264_vpu.h | 127 +++
.../media/platform/mtk-vpu/include/venc_ipi_msg.h | 212 +++++
drivers/media/platform/mtk-vpu/mtk_vpu_core.c | 823 ++++++++++++++++++++
drivers/media/platform/mtk-vpu/mtk_vpu_core.h | 161 ++++
.../media/platform/mtk-vpu/vp8_enc/venc_vp8_vpu.h | 119 +++
8 files changed, 1451 insertions(+)
create mode 100644 drivers/media/platform/mtk-vpu/Makefile
create mode 100644 drivers/media/platform/mtk-vpu/h264_enc/venc_h264_vpu.h
create mode 100644 drivers/media/platform/mtk-vpu/include/venc_ipi_msg.h
create mode 100644 drivers/media/platform/mtk-vpu/mtk_vpu_core.c
create mode 100644 drivers/media/platform/mtk-vpu/mtk_vpu_core.h
create mode 100644 drivers/media/platform/mtk-vpu/vp8_enc/venc_vp8_vpu.h
diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
index ccbc974..f98eb47 100644
--- a/drivers/media/platform/Kconfig
+++ b/drivers/media/platform/Kconfig
@@ -148,6 +148,12 @@ config VIDEO_CODA
Coda is a range of video codec IPs that supports
H.264, MPEG-4, and other video formats.
+config MEDIATEK_VPU
+ bool "Mediatek Video Processor Unit"
+ ---help---
+ This driver provides downloading firmware vpu and
+ communicating with vpu.
+
config VIDEO_MEM2MEM_DEINTERLACE
tristate "Deinterlace support"
depends on VIDEO_DEV && VIDEO_V4L2 && DMA_ENGINE
diff --git a/drivers/media/platform/Makefile b/drivers/media/platform/Makefile
index efa0295..1b4c539 100644
--- a/drivers/media/platform/Makefile
+++ b/drivers/media/platform/Makefile
@@ -55,3 +55,5 @@ obj-$(CONFIG_VIDEO_AM437X_VPFE) += am437x/
obj-$(CONFIG_VIDEO_XILINX) += xilinx/
ccflags-y += -I$(srctree)/drivers/media/i2c
+
+obj-$(CONFIG_MEDIATEK_VPU) += mtk-vpu/
diff --git a/drivers/media/platform/mtk-vpu/Makefile b/drivers/media/platform/mtk-vpu/Makefile
new file mode 100644
index 0000000..5de84d1
--- /dev/null
+++ b/drivers/media/platform/mtk-vpu/Makefile
@@ -0,0 +1 @@
+obj-$(CONFIG_MEDIATEK_VPU) += mtk_vpu_core.o
diff --git a/drivers/media/platform/mtk-vpu/h264_enc/venc_h264_vpu.h b/drivers/media/platform/mtk-vpu/h264_enc/venc_h264_vpu.h
new file mode 100644
index 0000000..9c8ebdd
--- /dev/null
+++ b/drivers/media/platform/mtk-vpu/h264_enc/venc_h264_vpu.h
@@ -0,0 +1,127 @@
+/*
+ * Copyright (c) 2015 MediaTek Inc.
+ * Author: Jungchang Tsao <jungchang.tsao@mediatek.com>
+ * Daniel Hsiao <daniel.hsiao@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef VENC_H264_VPU_H_
+#define VENC_H264_VPU_H_
+
+/**
+ * enum venc_h264_vpu_work_buf - h264 encoder buffer index
+ */
+enum venc_h264_vpu_work_buf {
+ VENC_H264_VPU_WORK_BUF_RC_INFO,
+ VENC_H264_VPU_WORK_BUF_RC_CODE,
+ VENC_H264_VPU_WORK_BUF_REC_LUMA,
+ VENC_H264_VPU_WORK_BUF_REC_CHROMA,
+ VENC_H264_VPU_WORK_BUF_REF_LUMA,
+ VENC_H264_VPU_WORK_BUF_REF_CHROMA,
+ VENC_H264_VPU_WORK_BUF_MV_INFO_1,
+ VENC_H264_VPU_WORK_BUF_MV_INFO_2,
+ VENC_H264_VPU_WORK_BUF_SKIP_FRAME,
+ VENC_H264_VPU_WORK_BUF_MAX,
+};
+
+/**
+ * enum venc_h264_bs_mode - for bs_mode argument in h264_enc_vpu_encode
+ */
+enum venc_h264_bs_mode {
+ H264_BS_MODE_SPS,
+ H264_BS_MODE_PPS,
+ H264_BS_MODE_FRAME,
+};
+
+/*
+ * struct venc_h264_vpu_config - Structure for h264 encoder configuration
+ * @input_fourcc: input fourcc
+ * @bitrate: target bitrate (in bps)
+ * @pic_w: picture width
+ * @pic_h: picture height
+ * @buf_w: buffer width
+ * @buf_h: buffer height
+ * @intra_period: intra frame period
+ * @framerate: frame rate
+ * @profile: as specified in standard
+ * @level: as specified in standard
+ * @wfd: WFD mode 1:on, 0:off
+ */
+struct venc_h264_vpu_config {
+ u32 input_fourcc;
+ u32 bitrate;
+ u32 pic_w;
+ u32 pic_h;
+ u32 buf_w;
+ u32 buf_h;
+ u32 intra_period;
+ u32 framerate;
+ u32 profile;
+ u32 level;
+ u32 wfd;
+};
+
+/*
+ * struct venc_h264_vpu_buf - Structure for buffer information
+ * @align: buffer alignment (in bytes)
+ * @pa: physical address
+ * @vpua: VPU side memory addr which is used by RC_CODE
+ * @size: buffer size (in bytes)
+ */
+struct venc_h264_vpu_buf {
+ u32 align;
+ u32 pa;
+ u32 vpua;
+ u32 size;
+};
+
+/*
+ * struct venc_h264_vpu_drv - Structure for VPU driver control and info share
+ * @config: h264 encoder configuration
+ * @work_bufs: working buffer information
+ */
+struct venc_h264_vpu_drv {
+ struct venc_h264_vpu_config config;
+ struct venc_h264_vpu_buf work_bufs[VENC_H264_VPU_WORK_BUF_MAX];
+};
+
+/*
+ * struct venc_h264_vpu_inst - h264 encoder VPU driver instance
+ * @wq_hd: wait queue used for vpu cmd trigger then wait vpu interrupt done
+ * @signaled: flag used for checking vpu interrupt done
+ * @failure: flag to show vpu cmd succeeds or not
+ * @state: enum venc_ipi_msg_enc_state
+ * @bs_size: bitstream size for skip frame case usage
+ * @wait_int: flag to wait interrupt done (0: for skip frame case, 1: normal case)
+ * @id: VPU instance id
+ * @drv: driver structure allocated by VPU side for control and info share
+ */
+struct venc_h264_vpu_inst {
+ wait_queue_head_t wq_hd;
+ int signaled;
+ int failure;
+ int state;
+ int bs_size;
+ int wait_int;
+ unsigned int id;
+ struct venc_h264_vpu_drv *drv;
+};
+
+int h264_enc_vpu_init(void *handle);
+int h264_enc_vpu_set_param(void *handle, unsigned int id, void *param);
+int h264_enc_vpu_encode(void *handle, unsigned int bs_mode,
+ struct venc_frm_buf *frm_buf,
+ struct mtk_vcodec_mem *bs_buf,
+ unsigned int *bs_size);
+int h264_enc_vpu_deinit(void *handle);
+
+#endif
diff --git a/drivers/media/platform/mtk-vpu/include/venc_ipi_msg.h b/drivers/media/platform/mtk-vpu/include/venc_ipi_msg.h
new file mode 100644
index 0000000..a345b98
--- /dev/null
+++ b/drivers/media/platform/mtk-vpu/include/venc_ipi_msg.h
@@ -0,0 +1,212 @@
+/*
+ * Copyright (c) 2015 MediaTek Inc.
+ * Author: Jungchang Tsao <jungchang.tsao@mediatek.com>
+ * Daniel Hsiao <daniel.hsiao@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef _VENC_IPI_MSG_H_
+#define _VENC_IPI_MSG_H_
+
+#define IPIMSG_H264_ENC_ID 0x100
+#define IPIMSG_VP8_ENC_ID 0x200
+
+#define AP_IPIMSG_VENC_BASE 0x20000
+#define VPU_IPIMSG_VENC_BASE 0x30000
+
+/**
+ * enum venc_ipi_msg_id - message id between AP and VPU
+ * (ipi stands for inter-processor interrupt)
+ * @AP_IPIMSG_XXX: AP to VPU cmd message id
+ * @VPU_IPIMSG_XXX_DONE: VPU ack AP cmd message id
+ */
+enum venc_ipi_msg_id {
+ AP_IPIMSG_H264_ENC_INIT = AP_IPIMSG_VENC_BASE +
+ IPIMSG_H264_ENC_ID,
+ AP_IPIMSG_H264_ENC_SET_PARAM,
+ AP_IPIMSG_H264_ENC_ENCODE,
+ AP_IPIMSG_H264_ENC_DEINIT,
+
+ AP_IPIMSG_VP8_ENC_INIT = AP_IPIMSG_VENC_BASE +
+ IPIMSG_VP8_ENC_ID,
+ AP_IPIMSG_VP8_ENC_SET_PARAM,
+ AP_IPIMSG_VP8_ENC_ENCODE,
+ AP_IPIMSG_VP8_ENC_DEINIT,
+
+ VPU_IPIMSG_H264_ENC_INIT_DONE = VPU_IPIMSG_VENC_BASE +
+ IPIMSG_H264_ENC_ID,
+ VPU_IPIMSG_H264_ENC_SET_PARAM_DONE,
+ VPU_IPIMSG_H264_ENC_ENCODE_DONE,
+ VPU_IPIMSG_H264_ENC_DEINIT_DONE,
+
+ VPU_IPIMSG_VP8_ENC_INIT_DONE = VPU_IPIMSG_VENC_BASE +
+ IPIMSG_VP8_ENC_ID,
+ VPU_IPIMSG_VP8_ENC_SET_PARAM_DONE,
+ VPU_IPIMSG_VP8_ENC_ENCODE_DONE,
+ VPU_IPIMSG_VP8_ENC_DEINIT_DONE,
+};
+
+/**
+ * struct venc_ap_ipi_msg_init - AP to VPU init cmd structure
+ * @msg_id: message id (AP_IPIMSG_XXX_ENC_INIT)
+ * @venc_inst: AP encoder instance (struct venc_vp8_handle/venc_h264_handle *)
+ */
+struct venc_ap_ipi_msg_init {
+ uint32_t msg_id;
+ uint32_t reserved;
+ uint64_t venc_inst;
+};
+
+/**
+ * struct venc_ap_ipi_msg_set_param - AP to VPU set_param cmd structure
+ * @msg_id: message id (AP_IPIMSG_XXX_ENC_SET_PARAM)
+ * @inst_id: VPU encoder instance id (struct venc_vp8_vpu_drv/venc_h264_vpu_drv *)
+ * @param_id: parameter id (venc_set_param_type)
+ * @data_item: number of items in the data array
+ * @data[8]: data array to store the set parameters
+ */
+struct venc_ap_ipi_msg_set_param {
+ uint32_t msg_id;
+ uint32_t inst_id;
+ uint32_t param_id;
+ uint32_t data_item;
+ uint32_t data[8];
+};
+
+/**
+ * struct venc_ap_ipi_msg_enc - AP to VPU enc cmd structure
+ * @msg_id: message id (AP_IPIMSG_XXX_ENC_ENCODE)
+ * @inst_id: VPU encoder instance id (struct venc_vp8_vpu_drv/venc_h264_vpu_drv *)
+ * @bs_mode: bitstream mode for h264
+ * (H264_BS_MODE_SPS/H264_BS_MODE_PPS/H264_BS_MODE_FRAME)
+ * @input_addr: pointer to input image buffer plane
+ * @bs_addr: pointer to output bit stream buffer
+ * @bs_size: bit stream buffer size
+ */
+struct venc_ap_ipi_msg_enc {
+ uint32_t msg_id;
+ uint32_t inst_id;
+ uint32_t bs_mode;
+ uint32_t input_addr[3];
+ uint32_t bs_addr;
+ uint32_t bs_size;
+};
+
+/**
+ * struct venc_ap_ipi_msg_deinit - AP to VPU deinit cmd structure
+ * @msg_id: message id (AP_IPIMSG_XXX_ENC_DEINIT)
+ * @inst_id: VPU encoder instance id (struct venc_vp8_vpu_drv/venc_h264_vpu_drv *)
+ */
+struct venc_ap_ipi_msg_deinit {
+ uint32_t msg_id;
+ uint32_t inst_id;
+};
+
+/**
+ * enum venc_ipi_msg_status - VPU ack AP cmd status
+ */
+enum venc_ipi_msg_status {
+ VENC_IPI_MSG_STATUS_OK,
+ VENC_IPI_MSG_STATUS_FAIL,
+};
+
+/**
+ * struct venc_vpu_ipi_msg_common - VPU ack AP cmd common structure
+ * @msg_id: message id (VPU_IPIMSG_XXX_DONE)
+ * @status: cmd status (venc_ipi_msg_status)
+ * @venc_inst: AP encoder instance (struct venc_vp8_handle/venc_h264_handle *)
+ */
+struct venc_vpu_ipi_msg_common {
+ uint32_t msg_id;
+ uint32_t status;
+ uint64_t venc_inst;
+};
+
+/**
+ * struct venc_vpu_ipi_msg_init - VPU ack AP init cmd structure
+ * @msg_id: message id (VPU_IPIMSG_XXX_ENC_SET_PARAM_DONE)
+ * @status: cmd status (venc_ipi_msg_status)
+ * @venc_inst: AP encoder instance (struct venc_vp8_handle/venc_h264_handle *)
+ * @inst_id: VPU encoder instance id (struct venc_vp8_vpu_drv/venc_h264_vpu_drv *)
+ */
+struct venc_vpu_ipi_msg_init {
+ uint32_t msg_id;
+ uint32_t status;
+ uint64_t venc_inst;
+ uint32_t inst_id;
+ uint32_t reserved;
+};
+
+/**
+ * struct venc_vpu_ipi_msg_set_param - VPU ack AP set_param cmd structure
+ * @msg_id: message id (VPU_IPIMSG_XXX_ENC_SET_PARAM_DONE)
+ * @status: cmd status (venc_ipi_msg_status)
+ * @venc_inst: AP encoder instance (struct venc_vp8_handle/venc_h264_handle *)
+ * @param_id: parameter id (venc_set_param_type)
+ * @data_item: number of items in the data array
+ * @data[6]: data array to store the return result
+ */
+struct venc_vpu_ipi_msg_set_param {
+ uint32_t msg_id;
+ uint32_t status;
+ uint64_t venc_inst;
+ uint32_t param_id;
+ uint32_t data_item;
+ uint32_t data[6];
+};
+
+/**
+ * enum venc_ipi_msg_enc_state - Type of encode state
+ * VEN_IPI_MSG_ENC_STATE_FRAME: one frame being encoded
+ * VEN_IPI_MSG_ENC_STATE_PART: bit stream buffer full
+ * VEN_IPI_MSG_ENC_STATE_SKIP: encoded skip frame
+ * VEN_IPI_MSG_ENC_STATE_ERROR: encounter error
+ */
+enum venc_ipi_msg_enc_state {
+ VEN_IPI_MSG_ENC_STATE_FRAME,
+ VEN_IPI_MSG_ENC_STATE_PART,
+ VEN_IPI_MSG_ENC_STATE_SKIP,
+ VEN_IPI_MSG_ENC_STATE_ERROR,
+};
+
+/**
+ * struct venc_vpu_ipi_msg_enc - VPU ack AP enc cmd structure
+ * @msg_id: message id (VPU_IPIMSG_XXX_ENC_ENCODE_DONE)
+ * @status: cmd status (venc_ipi_msg_status)
+ * @venc_inst: AP encoder instance (struct venc_vp8_handle/venc_h264_handle *)
+ * @state: encode state (venc_ipi_msg_enc_state)
+ * @key_frame: whether the encoded frame is key frame
+ * @bs_size: encoded bitstream size
+ */
+struct venc_vpu_ipi_msg_enc {
+ uint32_t msg_id;
+ uint32_t status;
+ uint64_t venc_inst;
+ uint32_t state;
+ uint32_t key_frame;
+ uint32_t bs_size;
+ uint32_t reserved;
+};
+
+/**
+ * struct venc_vpu_ipi_msg_deinit - VPU ack AP deinit cmd structure
+ * @msg_id: message id (VPU_IPIMSG_XXX_ENC_DEINIT_DONE)
+ * @status: cmd status (venc_ipi_msg_status)
+ * @venc_inst: AP encoder instance (struct venc_vp8_handle/venc_h264_handle *)
+ */
+struct venc_vpu_ipi_msg_deinit {
+ uint32_t msg_id;
+ uint32_t status;
+ uint64_t venc_inst;
+};
+
+#endif /* _VENC_IPI_MSG_H_ */
diff --git a/drivers/media/platform/mtk-vpu/mtk_vpu_core.c b/drivers/media/platform/mtk-vpu/mtk_vpu_core.c
new file mode 100644
index 0000000..b524dbc
--- /dev/null
+++ b/drivers/media/platform/mtk-vpu/mtk_vpu_core.c
@@ -0,0 +1,823 @@
+/*
+* Copyright (c) 2015 MediaTek Inc.
+* Author: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
+*
+* This program is free software; you can redistribute it and/or modify
+* it under the terms of the GNU General Public License version 2 as
+* published by the Free Software Foundation.
+*
+* This program is distributed in the hope that it will be useful,
+* but WITHOUT ANY WARRANTY; without even the implied warranty of
+* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+* GNU General Public License for more details.
+*/
+#include <linux/clk.h>
+#include <linux/debugfs.h>
+#include <linux/firmware.h>
+#include <linux/interrupt.h>
+#include <linux/iommu.h>
+#include <linux/module.h>
+#include <linux/of_address.h>
+#include <linux/of_irq.h>
+#include <linux/of_platform.h>
+#include <linux/sched.h>
+#include <linux/sizes.h>
+
+#include "mtk_vpu_core.h"
+
+/**
+ * VPU (video processor unit) is a tiny processor controlling video hardware
+ * related to video codec, scaling and color format converting.
+ * VPU interfaces with other blocks by share memory and interrupt.
+ **/
+#define MTK_VPU_DRV_NAME "mtk_vpu"
+
+#define INIT_TIMEOUT_MS 2000U
+#define IPI_TIMEOUT_MS 2000U
+#define VPU_FW_VER_LEN 16
+
+/* vpu extended virtural address */
+#define VPU_PMEM0_VIRT(vpu) ((vpu)->mem.p_va)
+#define VPU_DMEM0_VIRT(vpu) ((vpu)->mem.d_va)
+/* vpu extended iova address*/
+#define VPU_PMEM0_IOVA(vpu) ((vpu)->mem.p_iova)
+#define VPU_DMEM0_IOVA(vpu) ((vpu)->mem.d_iova)
+
+#define VPU_PTCM(dev) ((dev)->reg.sram)
+#define VPU_DTCM(dev) ((dev)->reg.sram + VPU_DTCM_OFFSET)
+
+#define VPU_PTCM_SIZE (96 * SZ_1K)
+#define VPU_DTCM_SIZE (32 * SZ_1K)
+#define VPU_DTCM_OFFSET 0x18000UL
+#define VPU_EXT_P_SIZE SZ_1M
+#define VPU_EXT_D_SIZE SZ_4M
+#define VPU_P_FW_SIZE (VPU_PTCM_SIZE + VPU_EXT_P_SIZE)
+#define VPU_D_FW_SIZE (VPU_DTCM_SIZE + VPU_EXT_D_SIZE)
+#define SHARE_BUF_SIZE 48
+
+#define VPU_P_FW "vpu_p.bin"
+#define VPU_D_FW "vpu_d.bin"
+
+#define VPU_BASE 0x0
+#define VPU_TCM_CFG 0x0008
+#define VPU_PMEM_EXT0_ADDR 0x000C
+#define VPU_PMEM_EXT1_ADDR 0x0010
+#define VPU_TO_HOST 0x001C
+#define VPU_DMEM_EXT0_ADDR 0x0014
+#define VPU_DMEM_EXT1_ADDR 0x0018
+#define HOST_TO_VPU 0x0024
+#define VPU_PC_REG 0x0060
+#define VPU_WDT_REG 0x0084
+
+/* vpu inter-processor communication interrupt */
+#define VPU_IPC_INT BIT(8)
+
+/**
+ * enum vpu_fw_type - VPU firmware type
+ *
+ * @P_FW: program firmware
+ * @D_FW: data firmware
+ *
+ */
+enum vpu_fw_type {
+ P_FW,
+ D_FW,
+};
+
+/**
+ * struct vpu_mem - VPU memory information
+ *
+ * @p_va: the kernel virtual memory address of
+ * VPU extended program memory
+ * @d_va: the kernel virtual memory address of VPU extended data memory
+ * @p_iova: the iova memory address of VPU extended program memory
+ * @d_iova: the iova memory address of VPU extended data memory
+ */
+struct vpu_mem {
+ void *p_va;
+ void *d_va;
+ dma_addr_t p_iova;
+ dma_addr_t d_iova;
+};
+
+/**
+ * struct vpu_regs - VPU SRAM and configuration registers
+ *
+ * @sram: the register for VPU sram
+ * @cfg: the register for VPU configuration
+ * @irq: the irq number for VPU interrupt
+ */
+struct vpu_regs {
+ void __iomem *sram;
+ void __iomem *cfg;
+ int irq;
+};
+
+/**
+ * struct vpu_run - VPU initialization status
+ *
+ * @signaled: the signal of vpu initialization completed
+ * @fw_ver: VPU firmware version
+ * @wq: wait queue for VPU initialization status
+ */
+struct vpu_run {
+ u32 signaled;
+ char fw_ver[VPU_FW_VER_LEN];
+ wait_queue_head_t wq;
+};
+
+/**
+ * struct vpu_ipi_desc - VPU IPI descriptor
+ *
+ * @handler: IPI handler
+ * @name: the name of IPI handler
+ * @priv: the private data of IPI handler
+ */
+struct vpu_ipi_desc {
+ ipi_handler_t handler;
+ const char *name;
+ void *priv;
+};
+
+/**
+ * struct share_obj - The DTCM (Data Tightly-Coupled Memory) buffer shared with
+ * AP and VPU
+ *
+ * @id: IPI id
+ * @len: share buffer length
+ * @share_buf: share buffer data
+ */
+struct share_obj {
+ int32_t id;
+ uint32_t len;
+ unsigned char share_buf[SHARE_BUF_SIZE];
+};
+
+/**
+ * struct mtk_vpu - vpu driver data
+ * @mem: VPU extended memory information
+ * @reg: VPU SRAM and configuration registers
+ * @run: VPU initialization status
+ * @ipi_desc: VPU IPI descriptor
+ * @recv_buf: VPU DTCM share buffer for receiving. The
+ * receive buffer is only accessed in interrupt context.
+ * @send_buf: VPU DTCM share buffer for sending
+ * @dev: VPU struct device
+ * @clk: VPU clock on/off
+ * @vpu_mutex: protect mtk_vpu (except recv_buf) and ensure only
+ * one client to use VPU service at a time. For example,
+ * suppose a client is using VPU to decode VP8.
+ * If the other client wants to encode VP8,
+ * it has to wait until VP8 decode completes.
+ *
+ */
+struct mtk_vpu {
+ struct vpu_mem mem;
+ struct vpu_regs reg;
+ struct vpu_run run;
+ struct vpu_ipi_desc ipi_desc[IPI_MAX];
+ struct share_obj *recv_buf;
+ struct share_obj *send_buf;
+ struct device *dev;
+ struct clk *clk;
+ struct mutex vpu_mutex; /* for protecting vpu data data structure */
+};
+
+/* the thread calls the function should hold the |vpu_mutex| */
+static inline void vpu_cfg_writel(struct mtk_vpu *vpu, u32 val, u32 offset)
+{
+ writel(val, vpu->reg.cfg + offset);
+}
+
+static inline u32 vpu_cfg_readl(struct mtk_vpu *vpu, u32 offset)
+{
+ return readl(vpu->reg.cfg + offset);
+}
+
+static inline bool vpu_running(struct mtk_vpu *vpu)
+{
+ return vpu_cfg_readl(vpu, VPU_BASE) & BIT(0);
+}
+
+void vpu_disable_clock(struct platform_device *pdev)
+{
+ struct mtk_vpu *vpu = platform_get_drvdata(pdev);
+
+ /* Disable VPU watchdog */
+ vpu_cfg_writel(vpu,
+ vpu_cfg_readl(vpu, VPU_WDT_REG) & ~(1L<<31),
+ VPU_WDT_REG);
+
+ clk_disable_unprepare(vpu->clk);
+}
+
+int vpu_enable_clock(struct platform_device *pdev)
+{
+ struct mtk_vpu *vpu = platform_get_drvdata(pdev);
+ int ret;
+
+ ret = clk_prepare_enable(vpu->clk);
+ if (ret)
+ return ret;
+ /* Enable VPU watchdog */
+ vpu_cfg_writel(vpu, vpu_cfg_readl(vpu, VPU_WDT_REG) | (1L << 31),
+ VPU_WDT_REG);
+
+ return ret;
+}
+
+int vpu_ipi_register(struct platform_device *pdev,
+ enum ipi_id id, ipi_handler_t handler,
+ const char *name, void *priv)
+{
+ struct mtk_vpu *vpu = platform_get_drvdata(pdev);
+ struct vpu_ipi_desc *ipi_desc;
+
+ if (!vpu) {
+ dev_err(&pdev->dev, "vpu device in not ready\n");
+ return -EPROBE_DEFER;
+ }
+
+ if (id < IPI_MAX && handler != NULL) {
+ ipi_desc = vpu->ipi_desc;
+ ipi_desc[id].name = name;
+ ipi_desc[id].handler = handler;
+ ipi_desc[id].priv = priv;
+ return 0;
+ }
+
+ dev_err(&pdev->dev, "register vpu ipi with invalid arguments\n");
+ return -EINVAL;
+}
+
+int vpu_ipi_send(struct platform_device *pdev,
+ enum ipi_id id, void *buf,
+ unsigned int len, unsigned int wait)
+{
+ struct mtk_vpu *vpu = platform_get_drvdata(pdev);
+ struct share_obj *send_obj = vpu->send_buf;
+ unsigned long timeout;
+
+ if (id >= IPI_MAX || len > sizeof(send_obj->share_buf) || buf == NULL) {
+ dev_err(vpu->dev, "failed to send ipi message\n");
+ return -EINVAL;
+ }
+
+ if (!vpu_running(vpu)) {
+ dev_err(vpu->dev, "vpu_ipi_send: VPU is not running\n");
+ return -ENXIO;
+ }
+
+ mutex_lock(&vpu->vpu_mutex);
+ if (vpu_cfg_readl(vpu, HOST_TO_VPU) && !wait) {
+ mutex_unlock(&vpu->vpu_mutex);
+ return -EBUSY;
+ }
+
+ if (wait)
+ while (vpu_cfg_readl(vpu, HOST_TO_VPU))
+ ;
+
+ memcpy((void *)send_obj->share_buf, buf, len);
+ send_obj->len = len;
+ send_obj->id = id;
+ vpu_cfg_writel(vpu, 0x1, HOST_TO_VPU);
+
+ /* Wait until VPU receives the command */
+ timeout = jiffies + msecs_to_jiffies(IPI_TIMEOUT_MS);
+ do {
+ if (time_after(jiffies, timeout)) {
+ dev_err(vpu->dev, "vpu_ipi_send: IPI timeout!\n");
+ return -EIO;
+ }
+ } while (vpu_cfg_readl(vpu, HOST_TO_VPU));
+
+ mutex_unlock(&vpu->vpu_mutex);
+
+ return 0;
+}
+
+void *vpu_mapping_dm_addr(struct platform_device *pdev,
+ void *dtcm_dmem_addr)
+{
+ struct mtk_vpu *vpu = platform_get_drvdata(pdev);
+ unsigned long p_vpu_dtcm = (unsigned long)VPU_DTCM(vpu);
+ unsigned long ul_dtcm_dmem_addr = (unsigned long)(dtcm_dmem_addr);
+
+ if (dtcm_dmem_addr == NULL ||
+ (ul_dtcm_dmem_addr > (VPU_DTCM_SIZE + VPU_EXT_D_SIZE))) {
+ dev_err(vpu->dev, "invalid virtual data memory address\n");
+ return ERR_PTR(-EINVAL);
+ }
+
+ if (ul_dtcm_dmem_addr < VPU_DTCM_SIZE)
+ return (void *)(ul_dtcm_dmem_addr + p_vpu_dtcm);
+
+ return (void *)((ul_dtcm_dmem_addr - VPU_DTCM_SIZE) +
+ VPU_DMEM0_VIRT(vpu));
+}
+
+dma_addr_t *vpu_mapping_iommu_dm_addr(struct platform_device *pdev,
+ void *dmem_addr)
+{
+ unsigned long ul_dmem_addr = (unsigned long)(dmem_addr);
+ struct mtk_vpu *vpu = platform_get_drvdata(pdev);
+
+ if (dmem_addr == NULL ||
+ (ul_dmem_addr < VPU_DTCM_SIZE) ||
+ (ul_dmem_addr > (VPU_DTCM_SIZE + VPU_EXT_D_SIZE))) {
+ dev_err(vpu->dev, "invalid IOMMU data memory address\n");
+ return ERR_PTR(-EINVAL);
+ }
+
+ return (dma_addr_t *)((ul_dmem_addr - VPU_DTCM_SIZE) +
+ VPU_DMEM0_IOVA(vpu));
+}
+
+struct platform_device *vpu_get_plat_device(struct platform_device *pdev)
+{
+ struct device *dev = &pdev->dev;
+ struct device_node *vpu_node;
+ struct platform_device *vpu_pdev;
+
+ vpu_node = of_parse_phandle(dev->of_node, "vpu", 0);
+ if (!vpu_node) {
+ dev_err(dev, "can't get vpu node\n");
+ return NULL;
+ }
+
+ vpu_pdev = of_find_device_by_node(vpu_node);
+ if (WARN_ON(!vpu_pdev)) {
+ dev_err(dev, "vpu pdev failed\n");
+ of_node_put(vpu_node);
+ return NULL;
+ }
+
+ return vpu_pdev;
+}
+
+/* load vpu program/data memory */
+static void load_requested_vpu(struct mtk_vpu *vpu,
+ size_t fw_size,
+ const u8 *fw_data,
+ u8 fw_type)
+{
+ size_t target_size = fw_type ? VPU_DTCM_SIZE : VPU_PTCM_SIZE;
+ size_t extra_fw_size = 0;
+ void *dest;
+
+ /* reset VPU */
+ vpu_cfg_writel(vpu, 0x0, VPU_BASE);
+
+ /* handle extended firmware size */
+ if (fw_size > target_size) {
+ dev_dbg(vpu->dev, "fw size %lx > limited fw size %lx\n",
+ fw_size, target_size);
+ extra_fw_size = fw_size - target_size;
+ dev_dbg(vpu->dev, "extra_fw_size %lx\n", extra_fw_size);
+ fw_size = target_size;
+ }
+ dest = fw_type ? VPU_DTCM(vpu) : VPU_PTCM(vpu);
+ memcpy(dest, fw_data, fw_size);
+ /* download to extended memory if need */
+ if (extra_fw_size > 0) {
+ dest = fw_type ?
+ VPU_DMEM0_VIRT(vpu) : VPU_PMEM0_VIRT(vpu);
+
+ dev_dbg(vpu->dev, "download extended memory type %x\n",
+ fw_type);
+ memcpy(dest, fw_data + target_size, extra_fw_size);
+ }
+}
+
+int vpu_load_firmware(struct platform_device *pdev)
+{
+ struct mtk_vpu *vpu = platform_get_drvdata(pdev);
+ struct device *dev = &pdev->dev;
+ struct vpu_run *run = &vpu->run;
+ const struct firmware *vpu_fw;
+ int ret;
+
+ if (!pdev) {
+ dev_err(dev, "VPU platform device is invalid\n");
+ return -EINVAL;
+ }
+
+ mutex_lock(&vpu->vpu_mutex);
+
+ ret = vpu_enable_clock(pdev);
+ if (ret) {
+ dev_err(dev, "enable clock failed %d\n", ret);
+ goto OUT_LOAD_FW;
+ }
+
+ if (vpu_running(vpu)) {
+ vpu_disable_clock(pdev);
+ mutex_unlock(&vpu->vpu_mutex);
+ dev_warn(dev, "vpu is running already\n");
+ return 0;
+ }
+
+ run->signaled = false;
+ dev_dbg(vpu->dev, "firmware request\n");
+ ret = request_firmware(&vpu_fw, VPU_P_FW, dev);
+ if (ret < 0) {
+ dev_err(dev, "Failed to load %s, %d\n", VPU_P_FW, ret);
+ goto OUT_LOAD_FW;
+ }
+ if (vpu_fw->size > VPU_P_FW_SIZE) {
+ ret = -EFBIG;
+ dev_err(dev, "program fw size %zu is abnormal\n", vpu_fw->size);
+ goto OUT_LOAD_FW;
+ }
+ dev_dbg(vpu->dev, "Downloaded program fw size: %zu.\n",
+ vpu_fw->size);
+ /* Downloading program firmware to device*/
+ load_requested_vpu(vpu, vpu_fw->size, vpu_fw->data,
+ P_FW);
+ release_firmware(vpu_fw);
+
+ ret = request_firmware(&vpu_fw, VPU_D_FW, dev);
+ if (ret < 0) {
+ dev_err(dev, "Failed to load %s, %d\n", VPU_D_FW, ret);
+ goto OUT_LOAD_FW;
+ }
+ if (vpu_fw->size > VPU_D_FW_SIZE) {
+ ret = -EFBIG;
+ dev_err(dev, "data fw size %zu is abnormal\n", vpu_fw->size);
+ goto OUT_LOAD_FW;
+ }
+ dev_dbg(vpu->dev, "Downloaded data fw size: %zu.\n",
+ vpu_fw->size);
+ /* Downloading data firmware to device */
+ load_requested_vpu(vpu, vpu_fw->size, vpu_fw->data,
+ D_FW);
+ release_firmware(vpu_fw);
+ /* boot up vpu */
+ vpu_cfg_writel(vpu, 0x1, VPU_BASE);
+
+ ret = wait_event_interruptible_timeout(run->wq,
+ run->signaled,
+ msecs_to_jiffies(INIT_TIMEOUT_MS)
+ );
+ if (0 == ret) {
+ ret = -ETIME;
+ dev_err(dev, "wait vpu initialization timout!\n");
+ goto OUT_LOAD_FW;
+ } else if (-ERESTARTSYS == ret) {
+ dev_err(dev, "wait vpu interrupted by a signal!\n");
+ goto OUT_LOAD_FW;
+ }
+
+ ret = 0;
+ dev_info(dev, "vpu is ready. Fw version %s\n", run->fw_ver);
+
+OUT_LOAD_FW:
+ vpu_disable_clock(pdev);
+ mutex_unlock(&vpu->vpu_mutex);
+
+ return ret;
+}
+
+static void vpu_init_ipi_handler(void *data, unsigned int len, void *priv)
+{
+ struct mtk_vpu *vpu = (struct mtk_vpu *)priv;
+ struct vpu_run *run = (struct vpu_run *)data;
+
+ vpu->run.signaled = run->signaled;
+ strncpy(vpu->run.fw_ver, run->fw_ver, VPU_FW_VER_LEN);
+ wake_up_interruptible(&vpu->run.wq);
+}
+
+static int vpu_debug_open(struct inode *inode, struct file *file)
+{
+ file->private_data = inode->i_private;
+ return 0;
+}
+
+static ssize_t vpu_debug_read(struct file *file, char __user *user_buf,
+ size_t count, loff_t *ppos)
+{
+ char buf[256];
+ unsigned int len;
+ unsigned int running, pc, vpu_to_host, host_to_vpu, wdt;
+ int ret;
+ struct device *dev = file->private_data;
+ struct platform_device *pdev = to_platform_device(dev);
+ struct mtk_vpu *vpu = dev_get_drvdata(dev);
+
+ ret = vpu_enable_clock(pdev);
+ if (ret) {
+ dev_err(vpu->dev, "[VPU] enable clock failed %d\n", ret);
+ return 0;
+ }
+
+ /* vpu register status */
+ running = vpu_running(vpu);
+ pc = vpu_cfg_readl(vpu, VPU_PC_REG);
+ wdt = vpu_cfg_readl(vpu, VPU_WDT_REG);
+ host_to_vpu = vpu_cfg_readl(vpu, HOST_TO_VPU);
+ vpu_to_host = vpu_cfg_readl(vpu, VPU_TO_HOST);
+ vpu_disable_clock(pdev);
+
+ if (running) {
+ len = sprintf(buf, "VPU is running\n\n"
+ "FW Version: %s\n"
+ "PC: 0x%x\n"
+ "WDT: 0x%x\n"
+ "Host to VPU: 0x%x\n"
+ "VPU to Host: 0x%x\n",
+ vpu->run.fw_ver, pc, wdt,
+ host_to_vpu, vpu_to_host);
+ } else {
+ len = sprintf(buf, "VPU not running\n");
+ }
+
+ return simple_read_from_buffer(user_buf, count, ppos, buf, len);
+}
+
+static const struct file_operations vpu_debug_fops = {
+ .open = vpu_debug_open,
+ .read = vpu_debug_read,
+};
+
+static void vpu_free_p_ext_mem(struct mtk_vpu *vpu)
+{
+ struct device *dev = vpu->dev;
+ struct iommu_domain *domain = iommu_get_domain_for_dev(dev);
+
+ dma_free_coherent(dev, VPU_EXT_P_SIZE, VPU_PMEM0_VIRT(vpu),
+ VPU_PMEM0_IOVA(vpu));
+
+ if (domain)
+ iommu_detach_device(domain, vpu->dev);
+}
+
+static void vpu_free_d_ext_mem(struct mtk_vpu *vpu)
+{
+ struct device *dev = vpu->dev;
+ struct iommu_domain *domain = iommu_get_domain_for_dev(dev);
+
+ dma_free_coherent(dev, VPU_EXT_D_SIZE, VPU_DMEM0_VIRT(vpu),
+ VPU_DMEM0_IOVA(vpu));
+
+ if (domain)
+ iommu_detach_device(domain, dev);
+}
+
+static int vpu_alloc_p_ext_mem(struct mtk_vpu *vpu)
+{
+ struct device *dev = vpu->dev;
+ struct iommu_domain *domain = iommu_get_domain_for_dev(dev);
+ phys_addr_t p_pa;
+
+ VPU_PMEM0_VIRT(vpu) = dma_alloc_coherent(dev,
+ VPU_EXT_P_SIZE,
+ &(VPU_PMEM0_IOVA(vpu)),
+ GFP_KERNEL);
+ if (VPU_PMEM0_VIRT(vpu) == NULL) {
+ dev_err(dev, "Failed to allocate the extended program memory\n");
+ return PTR_ERR(VPU_PMEM0_VIRT(vpu));
+ }
+
+ p_pa = iommu_iova_to_phys(domain, vpu->mem.p_iova);
+ /* Disable extend0. Enable extend1 */
+ vpu_cfg_writel(vpu, 0x1, VPU_PMEM_EXT0_ADDR);
+ vpu_cfg_writel(vpu, (p_pa & 0xFFFFF000), VPU_PMEM_EXT1_ADDR);
+
+ dev_info(dev, "Program extend memory phy=0x%llx virt=0x%p iova=0x%llx\n",
+ (unsigned long long)p_pa,
+ VPU_PMEM0_VIRT(vpu),
+ (unsigned long long)VPU_PMEM0_IOVA(vpu));
+
+ return 0;
+}
+
+static int vpu_alloc_d_ext_mem(struct mtk_vpu *vpu)
+{
+ struct device *dev = vpu->dev;
+ struct iommu_domain *domain = iommu_get_domain_for_dev(dev);
+ phys_addr_t d_pa;
+
+ VPU_DMEM0_VIRT(vpu) = dma_alloc_coherent(dev,
+ VPU_EXT_D_SIZE,
+ &(VPU_DMEM0_IOVA(vpu)),
+ GFP_KERNEL);
+ if (VPU_DMEM0_VIRT(vpu) == NULL) {
+ dev_err(dev, "Failed to allocate the extended data memory\n");
+ return PTR_ERR(VPU_DMEM0_VIRT(vpu));
+ }
+
+ d_pa = iommu_iova_to_phys(domain, vpu->mem.d_iova);
+
+ /* Disable extend0. Enable extend1 */
+ vpu_cfg_writel(vpu, 0x1, VPU_DMEM_EXT0_ADDR);
+ vpu_cfg_writel(vpu, (d_pa & 0xFFFFF000),
+ VPU_DMEM_EXT1_ADDR);
+
+ dev_info(dev, "Data extend memory phy=0x%llx virt=0x%p iova=0x%llx\n",
+ (unsigned long long)d_pa,
+ VPU_DMEM0_VIRT(vpu),
+ (unsigned long long)VPU_DMEM0_IOVA(vpu));
+
+ return 0;
+}
+
+static void vpu_ipi_handler(struct mtk_vpu *vpu)
+{
+ struct share_obj *rcv_obj = vpu->recv_buf;
+ struct vpu_ipi_desc *ipi_desc = vpu->ipi_desc;
+
+ if (rcv_obj->id < IPI_MAX && ipi_desc[rcv_obj->id].handler) {
+ ipi_desc[rcv_obj->id].handler(rcv_obj->share_buf,
+ rcv_obj->len,
+ ipi_desc[rcv_obj->id].priv);
+ } else {
+ dev_err(vpu->dev, "No such ipi id = %d\n", rcv_obj->id);
+ }
+}
+
+static int vpu_ipi_init(struct mtk_vpu *vpu)
+{
+ /* Disable VPU to host interrupt */
+ vpu_cfg_writel(vpu, 0x0, VPU_TO_HOST);
+
+ /* shared buffer initialization */
+ vpu->recv_buf = (struct share_obj *)VPU_DTCM(vpu);
+ vpu->send_buf = vpu->recv_buf + 1;
+ memset(vpu->recv_buf, 0, sizeof(struct share_obj));
+ memset(vpu->send_buf, 0, sizeof(struct share_obj));
+ mutex_init(&vpu->vpu_mutex);
+
+ return 0;
+}
+
+static irqreturn_t vpu_irq_handler(int irq, void *priv)
+{
+ struct mtk_vpu *vpu = priv;
+ uint32_t vpu_to_host = vpu_cfg_readl(vpu, VPU_TO_HOST);
+
+ if (vpu_to_host & VPU_IPC_INT)
+ vpu_ipi_handler(vpu);
+ else
+ dev_err(vpu->dev, "vpu watchdog timeout!\n");
+
+ /* VPU won't send another interrupt until we set VPU_TO_HOST to 0. */
+ vpu_cfg_writel(vpu, 0x0, VPU_TO_HOST);
+
+ return IRQ_HANDLED;
+}
+
+static struct dentry *vpu_debugfs;
+static int mtk_vpu_probe(struct platform_device *pdev)
+{
+ struct mtk_vpu *vpu;
+ struct device *dev;
+ struct resource *res;
+ int ret = 0;
+
+ dev_dbg(&pdev->dev, "initialization\n");
+
+ dev = &pdev->dev;
+ vpu = devm_kzalloc(dev, sizeof(*vpu), GFP_KERNEL);
+ if (!vpu)
+ return -ENOMEM;
+
+ vpu->dev = &pdev->dev;
+ res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "sram");
+ vpu->reg.sram = devm_ioremap_resource(dev, res);
+ if (IS_ERR(vpu->reg.sram)) {
+ dev_err(dev, "devm_ioremap_resource vpu sram failed.\n");
+ return PTR_ERR(vpu->reg.sram);
+ }
+
+ res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "cfg_reg");
+ vpu->reg.cfg = devm_ioremap_resource(dev, res);
+ if (IS_ERR(vpu->reg.cfg)) {
+ dev_err(dev, "devm_ioremap_resource vpu cfg failed.\n");
+ return PTR_ERR(vpu->reg.cfg);
+ }
+
+ /* Get VPU clock */
+ vpu->clk = devm_clk_get(dev, "main");
+ if (vpu->clk == NULL) {
+ dev_err(dev, "get vpu clock fail\n");
+ return -EINVAL;
+ }
+
+ platform_set_drvdata(pdev, vpu);
+
+ ret = vpu_enable_clock(pdev);
+ if (ret) {
+ ret = -EINVAL;
+ return ret;
+ }
+
+ dev_dbg(dev, "vpu ipi init\n");
+ ret = vpu_ipi_init(vpu);
+ if (ret) {
+ dev_err(dev, "Failed to init ipi\n");
+ goto disable_vpu_clk;
+ }
+
+ platform_set_drvdata(pdev, vpu);
+
+ /* register vpu initialization IPI */
+ ret = vpu_ipi_register(pdev, IPI_VPU_INIT, vpu_init_ipi_handler,
+ "vpu_init", vpu);
+ if (ret) {
+ dev_err(dev, "Failed to register IPI_VPU_INIT\n");
+ goto vpu_mutex_destroy;
+ }
+
+ vpu_debugfs = debugfs_create_file("mtk_vpu", S_IRUGO, NULL, (void *)dev,
+ &vpu_debug_fops);
+ if (!vpu_debugfs) {
+ ret = -ENOMEM;
+ goto cleanup_ipi;
+ }
+
+ /* Set PTCM to 96K and DTCM to 32K */
+ vpu_cfg_writel(vpu, 0x2, VPU_TCM_CFG);
+
+ ret = vpu_alloc_p_ext_mem(vpu);
+ if (ret) {
+ dev_err(dev, "Allocate PM failed\n");
+ goto remove_debugfs;
+ }
+
+ ret = vpu_alloc_d_ext_mem(vpu);
+ if (ret) {
+ dev_err(dev, "Allocate DM failed\n");
+ goto free_p_mem;
+ }
+
+ init_waitqueue_head(&vpu->run.wq);
+
+ res = platform_get_resource(pdev, IORESOURCE_IRQ, 0);
+ if (res == NULL) {
+ dev_err(dev, "get IRQ resource failed.\n");
+ ret = -ENXIO;
+ goto free_d_mem;
+ }
+ vpu->reg.irq = platform_get_irq(pdev, 0);
+ ret = devm_request_irq(dev, vpu->reg.irq, vpu_irq_handler, 0,
+ pdev->name, vpu);
+ if (ret) {
+ dev_err(dev, "failed to request irq\n");
+ goto free_d_mem;
+ }
+
+ vpu_disable_clock(pdev);
+ dev_dbg(dev, "initialization completed\n");
+
+ return 0;
+
+free_d_mem:
+ vpu_free_d_ext_mem(vpu);
+free_p_mem:
+ vpu_free_p_ext_mem(vpu);
+remove_debugfs:
+ debugfs_remove(vpu_debugfs);
+cleanup_ipi:
+ memset(vpu->ipi_desc, 0, sizeof(struct vpu_ipi_desc)*IPI_MAX);
+vpu_mutex_destroy:
+ mutex_destroy(&vpu->vpu_mutex);
+disable_vpu_clk:
+ vpu_disable_clock(pdev);
+
+ return ret;
+}
+
+static const struct of_device_id mtk_vpu_match[] = {
+ {
+ .compatible = "mediatek,mt8173-vpu",
+ },
+ {},
+};
+MODULE_DEVICE_TABLE(of, mtk_vpu_match);
+
+static int mtk_vpu_remove(struct platform_device *pdev)
+{
+ struct mtk_vpu *vpu = platform_get_drvdata(pdev);
+
+ vpu_free_p_ext_mem(vpu);
+ vpu_free_d_ext_mem(vpu);
+
+ return 0;
+}
+
+static struct platform_driver mtk_vpu_driver = {
+ .probe = mtk_vpu_probe,
+ .remove = mtk_vpu_remove,
+ .driver = {
+ .name = MTK_VPU_DRV_NAME,
+ .owner = THIS_MODULE,
+ .of_match_table = mtk_vpu_match,
+ },
+};
+
+module_platform_driver(mtk_vpu_driver);
+
+MODULE_LICENSE("GPL v2");
+MODULE_DESCRIPTION("Mediatek Video Prosessor Unit driver");
diff --git a/drivers/media/platform/mtk-vpu/mtk_vpu_core.h b/drivers/media/platform/mtk-vpu/mtk_vpu_core.h
new file mode 100644
index 0000000..20cf2a0
--- /dev/null
+++ b/drivers/media/platform/mtk-vpu/mtk_vpu_core.h
@@ -0,0 +1,161 @@
+/*
+* Copyright (c) 2015 MediaTek Inc.
+* Author: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
+*
+* This program is free software; you can redistribute it and/or modify
+* it under the terms of the GNU General Public License version 2 as
+* published by the Free Software Foundation.
+*
+* This program is distributed in the hope that it will be useful,
+* but WITHOUT ANY WARRANTY; without even the implied warranty of
+* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+* GNU General Public License for more details.
+*/
+
+#ifndef _MTK_VPU_CORE_H
+#define _MTK_VPU_CORE_H
+
+#include <linux/platform_device.h>
+
+/**
+ * VPU (video processor unit) is a tiny processor controlling video hardware
+ * related to video codec, scaling and color format converting.
+ * VPU interfaces with other blocks by share memory and interrupt.
+ **/
+
+typedef void (*ipi_handler_t) (void *data,
+ unsigned int len,
+ void *priv);
+
+/**
+ * enum ipi_id - the id of inter-processor interrupt
+ *
+ * @IPI_VPU_INIT: The interrupt from vpu is to notfiy kernel
+ VPU initialization completed.
+ * @IPI_VENC_H264: The interrupt from vpu is to notify kernel to
+ handle H264 video encoder job, and vice versa.
+ * @IPI_VENC_VP8: The interrupt fro vpu is to notify kernel to
+ handle VP8 video encoder job,, and vice versa.
+ * @IPI_VENC_CAPABILITY: The interrupt from vpu is to
+ get venc hardware capability.
+ * @IPI_MAX: The maximum IPI number
+ */
+enum ipi_id {
+ IPI_VPU_INIT = 0,
+ IPI_VENC_H264,
+ IPI_VENC_VP8,
+ IPI_VENC_CAPABILITY,
+ IPI_MAX,
+};
+
+/**
+ * vpu_disable_clock - Disable VPU clock
+ *
+ * @pdev: VPU platform device
+ *
+ *
+ * Return: Return 0 if the clock is disabled successfully,
+ * otherwise it is failed.
+ *
+ **/
+void vpu_disable_clock(struct platform_device *pdev);
+
+/**
+ * vpu_enable_clock - Enable VPU clock
+ *
+ * @pdev: VPU platform device
+ *
+ * Return: Return 0 if the clock is enabled successfully,
+ * otherwise it is failed.
+ *
+ **/
+int vpu_enable_clock(struct platform_device *pdev);
+
+/**
+ * vpu_ipi_register - register an ipi function
+ *
+ * @pdev: VPU platform device
+ * @id: IPI ID
+ * @handler: IPI handler
+ * @name: IPI name
+ * @priv: private data for IPI handler
+ *
+ * Register an ipi function to receive ipi interrupt from VPU.
+ *
+ * Return: Return 0 if ipi registers successfully, otherwise it is failed.
+ */
+int vpu_ipi_register(struct platform_device *pdev, enum ipi_id id,
+ ipi_handler_t handler, const char *name, void *priv);
+
+/**
+ * vpu_ipi_send - send data from AP to vpu.
+ *
+ * @pdev: VPU platform device
+ * @id: IPI ID
+ * @buf: the data buffer
+ * @len: the data buffer length
+ * @wait: wait for the last ipi completed.
+ *
+ * This function is thread-safe. When this function returns,
+ * VPU has received the data and starts the processing.
+ * When the processing completes, IPI handler registered
+ * by vpu_ipi_register will be called in interrupt context.
+ *
+ * Return: Return 0 if sending data successfully, otherwise it is failed.
+ **/
+int vpu_ipi_send(struct platform_device *pdev,
+ enum ipi_id id, void *buf,
+ unsigned int len,
+ unsigned int wait);
+
+/**
+ * vpu_get_plat_device - get VPU's platform device
+ *
+ * @pdev: the platform device of the module requesting VPU platform
+ * device for using VPU API.
+ *
+ * Return: Return NULL if it is failed.
+ * otherwise it is VPU's platform device
+ **/
+struct platform_device *vpu_get_plat_device(struct platform_device *pdev);
+
+/**
+ * vpu_load_firmware - download VPU firmware and boot it
+ *
+ * @pdev: VPU platform device
+ *
+ * Return: Return 0 if downloading firmware successfully,
+ * otherwise it is failed
+ **/
+int vpu_load_firmware(struct platform_device *pdev);
+
+/**
+ * vpu_mapping_dm_addr - Mapping DTCM/DMEM to kernel virtual address
+ *
+ * @pdev: VPU platform device
+ * @dmem_addr: VPU's data memory address
+ *
+ * Mapping the VPU's DTCM (Data Tightly-Coupled Memory) /
+ * DMEM (Data Extended Memory) memory address to
+ * kernel virtual address.
+ *
+ * Return: Return ERR_PTR(-EINVAL) if mapping failed,
+ * otherwise the mapped kernel virtual address
+ **/
+void *vpu_mapping_dm_addr(struct platform_device *pdev,
+ void *dtcm_dmem_addr);
+
+/**
+ * vpu_mapping_iommu_dm_addr - Mapping to iommu address
+ *
+ * @pdev: VPU platform device
+ * @dmem_addr: VPU's extended data memory address
+ *
+ * Mapping the VPU's extended data address to iommu address
+ *
+ * Return: Return ERR_PTR(-EINVAL) if mapping failed,
+ * otherwise the mapped iommu address
+ **/
+dma_addr_t *vpu_mapping_iommu_dm_addr(struct platform_device *pdev,
+ void *dmem_addr);
+#endif /* _MTK_VPU_CORE_H */
diff --git a/drivers/media/platform/mtk-vpu/vp8_enc/venc_vp8_vpu.h b/drivers/media/platform/mtk-vpu/vp8_enc/venc_vp8_vpu.h
new file mode 100644
index 0000000..4e09eec
--- /dev/null
+++ b/drivers/media/platform/mtk-vpu/vp8_enc/venc_vp8_vpu.h
@@ -0,0 +1,119 @@
+/*
+ * Copyright (c) 2015 MediaTek Inc.
+ * Author: Daniel Hsiao <daniel.hsiao@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef VENC_VP8_VPU_H_
+#define VENC_VP8_VPU_H_
+
+/**
+ * enum venc_vp8_vpu_work_buf - vp8 encoder buffer index
+ */
+enum venc_vp8_vpu_work_buf {
+ VENC_VP8_VPU_WORK_BUF_LUMA,
+ VENC_VP8_VPU_WORK_BUF_LUMA2,
+ VENC_VP8_VPU_WORK_BUF_LUMA3,
+ VENC_VP8_VPU_WORK_BUF_CHROMA,
+ VENC_VP8_VPU_WORK_BUF_CHROMA2,
+ VENC_VP8_VPU_WORK_BUF_CHROMA3,
+ VENC_VP8_VPU_WORK_BUF_MV_INFO,
+ VENC_VP8_VPU_WORK_BUF_BS_HD,
+ VENC_VP8_VPU_WORK_BUF_PROB_BUF,
+ VENC_VP8_VPU_WORK_BUF_RC_INFO,
+ VENC_VP8_VPU_WORK_BUF_RC_CODE,
+ VENC_VP8_VPU_WORK_BUF_RC_CODE2,
+ VENC_VP8_VPU_WORK_BUF_RC_CODE3,
+ VENC_VP8_VPU_WORK_BUF_MAX,
+};
+
+/*
+ * struct venc_vp8_vpu_config - Structure for vp8 encoder configuration
+ * @input_fourcc: input fourcc
+ * @bitrate: target bitrate (in bps)
+ * @pic_w: picture width
+ * @pic_h: picture height
+ * @buf_w: buffer width (with 16 alignment)
+ * @buf_h: buffer height (with 16 alignment)
+ * @intra_period: intra frame period
+ * @framerate: frame rate
+ * @ts_mode: temporal scalability mode (0: disable, 1: enable)
+ * support three temporal layers - 0: 7.5fps 1: 7.5fps 2: 15fps.
+ */
+struct venc_vp8_vpu_config {
+ u32 input_fourcc;
+ u32 bitrate;
+ u32 pic_w;
+ u32 pic_h;
+ u32 buf_w;
+ u32 buf_h;
+ u32 intra_period;
+ u32 framerate;
+ u32 ts_mode;
+};
+
+/*
+ * struct venc_vp8_vpu_buf -Structure for buffer information
+ * @align: buffer alignment (in bytes)
+ * @pa: physical address
+ * @vpua: VPU side memory addr which is used by RC_CODE
+ * @size: buffer size (in bytes)
+ */
+struct venc_vp8_vpu_buf {
+ u32 align;
+ u32 pa;
+ u32 vpua;
+ u32 size;
+};
+
+/*
+ * struct venc_vp8_vpu_drv - Structure for VPU driver control and info share
+ * This structure is allocated in VPU side and shared to AP side.
+ * @config: vp8 encoder configuration
+ * @work_bufs: working buffer information in VPU side
+ * The work_bufs here is for storing the 'size' info shared to AP side.
+ * The similar item in struct venc_vp8_handle is for memory allocation
+ * in AP side. The AP driver will copy the 'size' from here to the one in
+ * struct mtk_vcodec_mem, then invoke mtk_vcodec_mem_alloc to allocate
+ * the buffer. After that, bypass the 'dma_addr' to the 'pa' field here for
+ * register setting in VPU side.
+ */
+struct venc_vp8_vpu_drv {
+ struct venc_vp8_vpu_config config;
+ struct venc_vp8_vpu_buf work_bufs[VENC_VP8_VPU_WORK_BUF_MAX];
+};
+
+/*
+ * struct venc_vp8_vpu_inst - vp8 encoder VPU driver instance
+ * @wq_hd: wait queue used for vpu cmd trigger then wait vpu interrupt done
+ * @signaled: flag used for checking vpu interrupt done
+ * @failure: flag to show vpu cmd succeeds or not
+ * @id: VPU instance id
+ * @drv: driver structure allocated by VPU side and shared to AP side for
+ * control and info share
+ */
+struct venc_vp8_vpu_inst {
+ wait_queue_head_t wq_hd;
+ int signaled;
+ int failure;
+ unsigned int id;
+ struct venc_vp8_vpu_drv *drv;
+};
+
+int vp8_enc_vpu_init(void *handle);
+int vp8_enc_vpu_set_param(void *handle, unsigned int id, void *param);
+int vp8_enc_vpu_encode(void *handle,
+ struct venc_frm_buf *frm_buf,
+ struct mtk_vcodec_mem *bs_buf);
+int vp8_enc_vpu_deinit(void *handle);
+
+#endif
--
1.7.9.5
^ permalink raw reply related [flat|nested] 84+ messages in thread
* [RESEND RFC/PATCH 3/8] media: platform: mtk-vpu: Support Mediatek VPU
@ 2015-11-17 12:54 ` Tiffany Lin
0 siblings, 0 replies; 84+ messages in thread
From: Tiffany Lin @ 2015-11-17 12:54 UTC (permalink / raw)
To: Rob Herring, Pawel Moll, Mark Rutland, Ian Campbell, Kumar Gala,
Catalin Marinas, Will Deacon, Mauro Carvalho Chehab,
Matthias Brugger, Daniel Kurtz, Sascha Hauer, Hongzhou Yang,
Hans Verkuil, Laurent Pinchart, Sakari Ailus, Geert Uytterhoeven,
Mikhail Ulyanov, Fabien Dessenne, Arnd Bergmann,
Darren Etheridge, Peter Griffin, Benoit Parrot
Cc: James Liao, Eddie Huang, Daniel Hsiao, Andrew-CT Chen,
linux-kernel-u79uwXL29TY76Z2rM5mHXA,
devicetree-u79uwXL29TY76Z2rM5mHXA,
linux-mediatek-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r, Yingjoe Chen,
Tiffany Lin, linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
linux-media-u79uwXL29TY76Z2rM5mHXA
From: Andrew-CT Chen <andrew-ct.chen-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
The VPU driver for hw video codec embedded in Mediatek's MT8173 SOCs.
It is able to handle video decoding/encoding of in a range of formats.
The driver provides with VPU firmware download, memory management and
the communication interface between CPU and VPU.
For VPU initialization, it will create virtual memory for CPU access and
IOMMU address for vcodec hw device access. When a decode/encode instance
opens a device node, vpu driver will download vpu firmware to the device.
A decode/encode instant will decode/encode a frame using VPU
interface to interrupt vpu to handle decoding/encoding jobs.
Signed-off-by: Andrew-CT Chen <andrew-ct.chen-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
---
drivers/media/platform/Kconfig | 6 +
drivers/media/platform/Makefile | 2 +
drivers/media/platform/mtk-vpu/Makefile | 1 +
.../platform/mtk-vpu/h264_enc/venc_h264_vpu.h | 127 +++
.../media/platform/mtk-vpu/include/venc_ipi_msg.h | 212 +++++
drivers/media/platform/mtk-vpu/mtk_vpu_core.c | 823 ++++++++++++++++++++
drivers/media/platform/mtk-vpu/mtk_vpu_core.h | 161 ++++
.../media/platform/mtk-vpu/vp8_enc/venc_vp8_vpu.h | 119 +++
8 files changed, 1451 insertions(+)
create mode 100644 drivers/media/platform/mtk-vpu/Makefile
create mode 100644 drivers/media/platform/mtk-vpu/h264_enc/venc_h264_vpu.h
create mode 100644 drivers/media/platform/mtk-vpu/include/venc_ipi_msg.h
create mode 100644 drivers/media/platform/mtk-vpu/mtk_vpu_core.c
create mode 100644 drivers/media/platform/mtk-vpu/mtk_vpu_core.h
create mode 100644 drivers/media/platform/mtk-vpu/vp8_enc/venc_vp8_vpu.h
diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
index ccbc974..f98eb47 100644
--- a/drivers/media/platform/Kconfig
+++ b/drivers/media/platform/Kconfig
@@ -148,6 +148,12 @@ config VIDEO_CODA
Coda is a range of video codec IPs that supports
H.264, MPEG-4, and other video formats.
+config MEDIATEK_VPU
+ bool "Mediatek Video Processor Unit"
+ ---help---
+ This driver provides downloading firmware vpu and
+ communicating with vpu.
+
config VIDEO_MEM2MEM_DEINTERLACE
tristate "Deinterlace support"
depends on VIDEO_DEV && VIDEO_V4L2 && DMA_ENGINE
diff --git a/drivers/media/platform/Makefile b/drivers/media/platform/Makefile
index efa0295..1b4c539 100644
--- a/drivers/media/platform/Makefile
+++ b/drivers/media/platform/Makefile
@@ -55,3 +55,5 @@ obj-$(CONFIG_VIDEO_AM437X_VPFE) += am437x/
obj-$(CONFIG_VIDEO_XILINX) += xilinx/
ccflags-y += -I$(srctree)/drivers/media/i2c
+
+obj-$(CONFIG_MEDIATEK_VPU) += mtk-vpu/
diff --git a/drivers/media/platform/mtk-vpu/Makefile b/drivers/media/platform/mtk-vpu/Makefile
new file mode 100644
index 0000000..5de84d1
--- /dev/null
+++ b/drivers/media/platform/mtk-vpu/Makefile
@@ -0,0 +1 @@
+obj-$(CONFIG_MEDIATEK_VPU) += mtk_vpu_core.o
diff --git a/drivers/media/platform/mtk-vpu/h264_enc/venc_h264_vpu.h b/drivers/media/platform/mtk-vpu/h264_enc/venc_h264_vpu.h
new file mode 100644
index 0000000..9c8ebdd
--- /dev/null
+++ b/drivers/media/platform/mtk-vpu/h264_enc/venc_h264_vpu.h
@@ -0,0 +1,127 @@
+/*
+ * Copyright (c) 2015 MediaTek Inc.
+ * Author: Jungchang Tsao <jungchang.tsao-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
+ * Daniel Hsiao <daniel.hsiao-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef VENC_H264_VPU_H_
+#define VENC_H264_VPU_H_
+
+/**
+ * enum venc_h264_vpu_work_buf - h264 encoder buffer index
+ */
+enum venc_h264_vpu_work_buf {
+ VENC_H264_VPU_WORK_BUF_RC_INFO,
+ VENC_H264_VPU_WORK_BUF_RC_CODE,
+ VENC_H264_VPU_WORK_BUF_REC_LUMA,
+ VENC_H264_VPU_WORK_BUF_REC_CHROMA,
+ VENC_H264_VPU_WORK_BUF_REF_LUMA,
+ VENC_H264_VPU_WORK_BUF_REF_CHROMA,
+ VENC_H264_VPU_WORK_BUF_MV_INFO_1,
+ VENC_H264_VPU_WORK_BUF_MV_INFO_2,
+ VENC_H264_VPU_WORK_BUF_SKIP_FRAME,
+ VENC_H264_VPU_WORK_BUF_MAX,
+};
+
+/**
+ * enum venc_h264_bs_mode - for bs_mode argument in h264_enc_vpu_encode
+ */
+enum venc_h264_bs_mode {
+ H264_BS_MODE_SPS,
+ H264_BS_MODE_PPS,
+ H264_BS_MODE_FRAME,
+};
+
+/*
+ * struct venc_h264_vpu_config - Structure for h264 encoder configuration
+ * @input_fourcc: input fourcc
+ * @bitrate: target bitrate (in bps)
+ * @pic_w: picture width
+ * @pic_h: picture height
+ * @buf_w: buffer width
+ * @buf_h: buffer height
+ * @intra_period: intra frame period
+ * @framerate: frame rate
+ * @profile: as specified in standard
+ * @level: as specified in standard
+ * @wfd: WFD mode 1:on, 0:off
+ */
+struct venc_h264_vpu_config {
+ u32 input_fourcc;
+ u32 bitrate;
+ u32 pic_w;
+ u32 pic_h;
+ u32 buf_w;
+ u32 buf_h;
+ u32 intra_period;
+ u32 framerate;
+ u32 profile;
+ u32 level;
+ u32 wfd;
+};
+
+/*
+ * struct venc_h264_vpu_buf - Structure for buffer information
+ * @align: buffer alignment (in bytes)
+ * @pa: physical address
+ * @vpua: VPU side memory addr which is used by RC_CODE
+ * @size: buffer size (in bytes)
+ */
+struct venc_h264_vpu_buf {
+ u32 align;
+ u32 pa;
+ u32 vpua;
+ u32 size;
+};
+
+/*
+ * struct venc_h264_vpu_drv - Structure for VPU driver control and info share
+ * @config: h264 encoder configuration
+ * @work_bufs: working buffer information
+ */
+struct venc_h264_vpu_drv {
+ struct venc_h264_vpu_config config;
+ struct venc_h264_vpu_buf work_bufs[VENC_H264_VPU_WORK_BUF_MAX];
+};
+
+/*
+ * struct venc_h264_vpu_inst - h264 encoder VPU driver instance
+ * @wq_hd: wait queue used for vpu cmd trigger then wait vpu interrupt done
+ * @signaled: flag used for checking vpu interrupt done
+ * @failure: flag to show vpu cmd succeeds or not
+ * @state: enum venc_ipi_msg_enc_state
+ * @bs_size: bitstream size for skip frame case usage
+ * @wait_int: flag to wait interrupt done (0: for skip frame case, 1: normal case)
+ * @id: VPU instance id
+ * @drv: driver structure allocated by VPU side for control and info share
+ */
+struct venc_h264_vpu_inst {
+ wait_queue_head_t wq_hd;
+ int signaled;
+ int failure;
+ int state;
+ int bs_size;
+ int wait_int;
+ unsigned int id;
+ struct venc_h264_vpu_drv *drv;
+};
+
+int h264_enc_vpu_init(void *handle);
+int h264_enc_vpu_set_param(void *handle, unsigned int id, void *param);
+int h264_enc_vpu_encode(void *handle, unsigned int bs_mode,
+ struct venc_frm_buf *frm_buf,
+ struct mtk_vcodec_mem *bs_buf,
+ unsigned int *bs_size);
+int h264_enc_vpu_deinit(void *handle);
+
+#endif
diff --git a/drivers/media/platform/mtk-vpu/include/venc_ipi_msg.h b/drivers/media/platform/mtk-vpu/include/venc_ipi_msg.h
new file mode 100644
index 0000000..a345b98
--- /dev/null
+++ b/drivers/media/platform/mtk-vpu/include/venc_ipi_msg.h
@@ -0,0 +1,212 @@
+/*
+ * Copyright (c) 2015 MediaTek Inc.
+ * Author: Jungchang Tsao <jungchang.tsao-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
+ * Daniel Hsiao <daniel.hsiao-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef _VENC_IPI_MSG_H_
+#define _VENC_IPI_MSG_H_
+
+#define IPIMSG_H264_ENC_ID 0x100
+#define IPIMSG_VP8_ENC_ID 0x200
+
+#define AP_IPIMSG_VENC_BASE 0x20000
+#define VPU_IPIMSG_VENC_BASE 0x30000
+
+/**
+ * enum venc_ipi_msg_id - message id between AP and VPU
+ * (ipi stands for inter-processor interrupt)
+ * @AP_IPIMSG_XXX: AP to VPU cmd message id
+ * @VPU_IPIMSG_XXX_DONE: VPU ack AP cmd message id
+ */
+enum venc_ipi_msg_id {
+ AP_IPIMSG_H264_ENC_INIT = AP_IPIMSG_VENC_BASE +
+ IPIMSG_H264_ENC_ID,
+ AP_IPIMSG_H264_ENC_SET_PARAM,
+ AP_IPIMSG_H264_ENC_ENCODE,
+ AP_IPIMSG_H264_ENC_DEINIT,
+
+ AP_IPIMSG_VP8_ENC_INIT = AP_IPIMSG_VENC_BASE +
+ IPIMSG_VP8_ENC_ID,
+ AP_IPIMSG_VP8_ENC_SET_PARAM,
+ AP_IPIMSG_VP8_ENC_ENCODE,
+ AP_IPIMSG_VP8_ENC_DEINIT,
+
+ VPU_IPIMSG_H264_ENC_INIT_DONE = VPU_IPIMSG_VENC_BASE +
+ IPIMSG_H264_ENC_ID,
+ VPU_IPIMSG_H264_ENC_SET_PARAM_DONE,
+ VPU_IPIMSG_H264_ENC_ENCODE_DONE,
+ VPU_IPIMSG_H264_ENC_DEINIT_DONE,
+
+ VPU_IPIMSG_VP8_ENC_INIT_DONE = VPU_IPIMSG_VENC_BASE +
+ IPIMSG_VP8_ENC_ID,
+ VPU_IPIMSG_VP8_ENC_SET_PARAM_DONE,
+ VPU_IPIMSG_VP8_ENC_ENCODE_DONE,
+ VPU_IPIMSG_VP8_ENC_DEINIT_DONE,
+};
+
+/**
+ * struct venc_ap_ipi_msg_init - AP to VPU init cmd structure
+ * @msg_id: message id (AP_IPIMSG_XXX_ENC_INIT)
+ * @venc_inst: AP encoder instance (struct venc_vp8_handle/venc_h264_handle *)
+ */
+struct venc_ap_ipi_msg_init {
+ uint32_t msg_id;
+ uint32_t reserved;
+ uint64_t venc_inst;
+};
+
+/**
+ * struct venc_ap_ipi_msg_set_param - AP to VPU set_param cmd structure
+ * @msg_id: message id (AP_IPIMSG_XXX_ENC_SET_PARAM)
+ * @inst_id: VPU encoder instance id (struct venc_vp8_vpu_drv/venc_h264_vpu_drv *)
+ * @param_id: parameter id (venc_set_param_type)
+ * @data_item: number of items in the data array
+ * @data[8]: data array to store the set parameters
+ */
+struct venc_ap_ipi_msg_set_param {
+ uint32_t msg_id;
+ uint32_t inst_id;
+ uint32_t param_id;
+ uint32_t data_item;
+ uint32_t data[8];
+};
+
+/**
+ * struct venc_ap_ipi_msg_enc - AP to VPU enc cmd structure
+ * @msg_id: message id (AP_IPIMSG_XXX_ENC_ENCODE)
+ * @inst_id: VPU encoder instance id (struct venc_vp8_vpu_drv/venc_h264_vpu_drv *)
+ * @bs_mode: bitstream mode for h264
+ * (H264_BS_MODE_SPS/H264_BS_MODE_PPS/H264_BS_MODE_FRAME)
+ * @input_addr: pointer to input image buffer plane
+ * @bs_addr: pointer to output bit stream buffer
+ * @bs_size: bit stream buffer size
+ */
+struct venc_ap_ipi_msg_enc {
+ uint32_t msg_id;
+ uint32_t inst_id;
+ uint32_t bs_mode;
+ uint32_t input_addr[3];
+ uint32_t bs_addr;
+ uint32_t bs_size;
+};
+
+/**
+ * struct venc_ap_ipi_msg_deinit - AP to VPU deinit cmd structure
+ * @msg_id: message id (AP_IPIMSG_XXX_ENC_DEINIT)
+ * @inst_id: VPU encoder instance id (struct venc_vp8_vpu_drv/venc_h264_vpu_drv *)
+ */
+struct venc_ap_ipi_msg_deinit {
+ uint32_t msg_id;
+ uint32_t inst_id;
+};
+
+/**
+ * enum venc_ipi_msg_status - VPU ack AP cmd status
+ */
+enum venc_ipi_msg_status {
+ VENC_IPI_MSG_STATUS_OK,
+ VENC_IPI_MSG_STATUS_FAIL,
+};
+
+/**
+ * struct venc_vpu_ipi_msg_common - VPU ack AP cmd common structure
+ * @msg_id: message id (VPU_IPIMSG_XXX_DONE)
+ * @status: cmd status (venc_ipi_msg_status)
+ * @venc_inst: AP encoder instance (struct venc_vp8_handle/venc_h264_handle *)
+ */
+struct venc_vpu_ipi_msg_common {
+ uint32_t msg_id;
+ uint32_t status;
+ uint64_t venc_inst;
+};
+
+/**
+ * struct venc_vpu_ipi_msg_init - VPU ack AP init cmd structure
+ * @msg_id: message id (VPU_IPIMSG_XXX_ENC_SET_PARAM_DONE)
+ * @status: cmd status (venc_ipi_msg_status)
+ * @venc_inst: AP encoder instance (struct venc_vp8_handle/venc_h264_handle *)
+ * @inst_id: VPU encoder instance id (struct venc_vp8_vpu_drv/venc_h264_vpu_drv *)
+ */
+struct venc_vpu_ipi_msg_init {
+ uint32_t msg_id;
+ uint32_t status;
+ uint64_t venc_inst;
+ uint32_t inst_id;
+ uint32_t reserved;
+};
+
+/**
+ * struct venc_vpu_ipi_msg_set_param - VPU ack AP set_param cmd structure
+ * @msg_id: message id (VPU_IPIMSG_XXX_ENC_SET_PARAM_DONE)
+ * @status: cmd status (venc_ipi_msg_status)
+ * @venc_inst: AP encoder instance (struct venc_vp8_handle/venc_h264_handle *)
+ * @param_id: parameter id (venc_set_param_type)
+ * @data_item: number of items in the data array
+ * @data[6]: data array to store the return result
+ */
+struct venc_vpu_ipi_msg_set_param {
+ uint32_t msg_id;
+ uint32_t status;
+ uint64_t venc_inst;
+ uint32_t param_id;
+ uint32_t data_item;
+ uint32_t data[6];
+};
+
+/**
+ * enum venc_ipi_msg_enc_state - Type of encode state
+ * VEN_IPI_MSG_ENC_STATE_FRAME: one frame being encoded
+ * VEN_IPI_MSG_ENC_STATE_PART: bit stream buffer full
+ * VEN_IPI_MSG_ENC_STATE_SKIP: encoded skip frame
+ * VEN_IPI_MSG_ENC_STATE_ERROR: encounter error
+ */
+enum venc_ipi_msg_enc_state {
+ VEN_IPI_MSG_ENC_STATE_FRAME,
+ VEN_IPI_MSG_ENC_STATE_PART,
+ VEN_IPI_MSG_ENC_STATE_SKIP,
+ VEN_IPI_MSG_ENC_STATE_ERROR,
+};
+
+/**
+ * struct venc_vpu_ipi_msg_enc - VPU ack AP enc cmd structure
+ * @msg_id: message id (VPU_IPIMSG_XXX_ENC_ENCODE_DONE)
+ * @status: cmd status (venc_ipi_msg_status)
+ * @venc_inst: AP encoder instance (struct venc_vp8_handle/venc_h264_handle *)
+ * @state: encode state (venc_ipi_msg_enc_state)
+ * @key_frame: whether the encoded frame is key frame
+ * @bs_size: encoded bitstream size
+ */
+struct venc_vpu_ipi_msg_enc {
+ uint32_t msg_id;
+ uint32_t status;
+ uint64_t venc_inst;
+ uint32_t state;
+ uint32_t key_frame;
+ uint32_t bs_size;
+ uint32_t reserved;
+};
+
+/**
+ * struct venc_vpu_ipi_msg_deinit - VPU ack AP deinit cmd structure
+ * @msg_id: message id (VPU_IPIMSG_XXX_ENC_DEINIT_DONE)
+ * @status: cmd status (venc_ipi_msg_status)
+ * @venc_inst: AP encoder instance (struct venc_vp8_handle/venc_h264_handle *)
+ */
+struct venc_vpu_ipi_msg_deinit {
+ uint32_t msg_id;
+ uint32_t status;
+ uint64_t venc_inst;
+};
+
+#endif /* _VENC_IPI_MSG_H_ */
diff --git a/drivers/media/platform/mtk-vpu/mtk_vpu_core.c b/drivers/media/platform/mtk-vpu/mtk_vpu_core.c
new file mode 100644
index 0000000..b524dbc
--- /dev/null
+++ b/drivers/media/platform/mtk-vpu/mtk_vpu_core.c
@@ -0,0 +1,823 @@
+/*
+* Copyright (c) 2015 MediaTek Inc.
+* Author: Andrew-CT Chen <andrew-ct.chen-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
+*
+* This program is free software; you can redistribute it and/or modify
+* it under the terms of the GNU General Public License version 2 as
+* published by the Free Software Foundation.
+*
+* This program is distributed in the hope that it will be useful,
+* but WITHOUT ANY WARRANTY; without even the implied warranty of
+* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+* GNU General Public License for more details.
+*/
+#include <linux/clk.h>
+#include <linux/debugfs.h>
+#include <linux/firmware.h>
+#include <linux/interrupt.h>
+#include <linux/iommu.h>
+#include <linux/module.h>
+#include <linux/of_address.h>
+#include <linux/of_irq.h>
+#include <linux/of_platform.h>
+#include <linux/sched.h>
+#include <linux/sizes.h>
+
+#include "mtk_vpu_core.h"
+
+/**
+ * VPU (video processor unit) is a tiny processor controlling video hardware
+ * related to video codec, scaling and color format converting.
+ * VPU interfaces with other blocks by share memory and interrupt.
+ **/
+#define MTK_VPU_DRV_NAME "mtk_vpu"
+
+#define INIT_TIMEOUT_MS 2000U
+#define IPI_TIMEOUT_MS 2000U
+#define VPU_FW_VER_LEN 16
+
+/* vpu extended virtural address */
+#define VPU_PMEM0_VIRT(vpu) ((vpu)->mem.p_va)
+#define VPU_DMEM0_VIRT(vpu) ((vpu)->mem.d_va)
+/* vpu extended iova address*/
+#define VPU_PMEM0_IOVA(vpu) ((vpu)->mem.p_iova)
+#define VPU_DMEM0_IOVA(vpu) ((vpu)->mem.d_iova)
+
+#define VPU_PTCM(dev) ((dev)->reg.sram)
+#define VPU_DTCM(dev) ((dev)->reg.sram + VPU_DTCM_OFFSET)
+
+#define VPU_PTCM_SIZE (96 * SZ_1K)
+#define VPU_DTCM_SIZE (32 * SZ_1K)
+#define VPU_DTCM_OFFSET 0x18000UL
+#define VPU_EXT_P_SIZE SZ_1M
+#define VPU_EXT_D_SIZE SZ_4M
+#define VPU_P_FW_SIZE (VPU_PTCM_SIZE + VPU_EXT_P_SIZE)
+#define VPU_D_FW_SIZE (VPU_DTCM_SIZE + VPU_EXT_D_SIZE)
+#define SHARE_BUF_SIZE 48
+
+#define VPU_P_FW "vpu_p.bin"
+#define VPU_D_FW "vpu_d.bin"
+
+#define VPU_BASE 0x0
+#define VPU_TCM_CFG 0x0008
+#define VPU_PMEM_EXT0_ADDR 0x000C
+#define VPU_PMEM_EXT1_ADDR 0x0010
+#define VPU_TO_HOST 0x001C
+#define VPU_DMEM_EXT0_ADDR 0x0014
+#define VPU_DMEM_EXT1_ADDR 0x0018
+#define HOST_TO_VPU 0x0024
+#define VPU_PC_REG 0x0060
+#define VPU_WDT_REG 0x0084
+
+/* vpu inter-processor communication interrupt */
+#define VPU_IPC_INT BIT(8)
+
+/**
+ * enum vpu_fw_type - VPU firmware type
+ *
+ * @P_FW: program firmware
+ * @D_FW: data firmware
+ *
+ */
+enum vpu_fw_type {
+ P_FW,
+ D_FW,
+};
+
+/**
+ * struct vpu_mem - VPU memory information
+ *
+ * @p_va: the kernel virtual memory address of
+ * VPU extended program memory
+ * @d_va: the kernel virtual memory address of VPU extended data memory
+ * @p_iova: the iova memory address of VPU extended program memory
+ * @d_iova: the iova memory address of VPU extended data memory
+ */
+struct vpu_mem {
+ void *p_va;
+ void *d_va;
+ dma_addr_t p_iova;
+ dma_addr_t d_iova;
+};
+
+/**
+ * struct vpu_regs - VPU SRAM and configuration registers
+ *
+ * @sram: the register for VPU sram
+ * @cfg: the register for VPU configuration
+ * @irq: the irq number for VPU interrupt
+ */
+struct vpu_regs {
+ void __iomem *sram;
+ void __iomem *cfg;
+ int irq;
+};
+
+/**
+ * struct vpu_run - VPU initialization status
+ *
+ * @signaled: the signal of vpu initialization completed
+ * @fw_ver: VPU firmware version
+ * @wq: wait queue for VPU initialization status
+ */
+struct vpu_run {
+ u32 signaled;
+ char fw_ver[VPU_FW_VER_LEN];
+ wait_queue_head_t wq;
+};
+
+/**
+ * struct vpu_ipi_desc - VPU IPI descriptor
+ *
+ * @handler: IPI handler
+ * @name: the name of IPI handler
+ * @priv: the private data of IPI handler
+ */
+struct vpu_ipi_desc {
+ ipi_handler_t handler;
+ const char *name;
+ void *priv;
+};
+
+/**
+ * struct share_obj - The DTCM (Data Tightly-Coupled Memory) buffer shared with
+ * AP and VPU
+ *
+ * @id: IPI id
+ * @len: share buffer length
+ * @share_buf: share buffer data
+ */
+struct share_obj {
+ int32_t id;
+ uint32_t len;
+ unsigned char share_buf[SHARE_BUF_SIZE];
+};
+
+/**
+ * struct mtk_vpu - vpu driver data
+ * @mem: VPU extended memory information
+ * @reg: VPU SRAM and configuration registers
+ * @run: VPU initialization status
+ * @ipi_desc: VPU IPI descriptor
+ * @recv_buf: VPU DTCM share buffer for receiving. The
+ * receive buffer is only accessed in interrupt context.
+ * @send_buf: VPU DTCM share buffer for sending
+ * @dev: VPU struct device
+ * @clk: VPU clock on/off
+ * @vpu_mutex: protect mtk_vpu (except recv_buf) and ensure only
+ * one client to use VPU service at a time. For example,
+ * suppose a client is using VPU to decode VP8.
+ * If the other client wants to encode VP8,
+ * it has to wait until VP8 decode completes.
+ *
+ */
+struct mtk_vpu {
+ struct vpu_mem mem;
+ struct vpu_regs reg;
+ struct vpu_run run;
+ struct vpu_ipi_desc ipi_desc[IPI_MAX];
+ struct share_obj *recv_buf;
+ struct share_obj *send_buf;
+ struct device *dev;
+ struct clk *clk;
+ struct mutex vpu_mutex; /* for protecting vpu data data structure */
+};
+
+/* the thread calls the function should hold the |vpu_mutex| */
+static inline void vpu_cfg_writel(struct mtk_vpu *vpu, u32 val, u32 offset)
+{
+ writel(val, vpu->reg.cfg + offset);
+}
+
+static inline u32 vpu_cfg_readl(struct mtk_vpu *vpu, u32 offset)
+{
+ return readl(vpu->reg.cfg + offset);
+}
+
+static inline bool vpu_running(struct mtk_vpu *vpu)
+{
+ return vpu_cfg_readl(vpu, VPU_BASE) & BIT(0);
+}
+
+void vpu_disable_clock(struct platform_device *pdev)
+{
+ struct mtk_vpu *vpu = platform_get_drvdata(pdev);
+
+ /* Disable VPU watchdog */
+ vpu_cfg_writel(vpu,
+ vpu_cfg_readl(vpu, VPU_WDT_REG) & ~(1L<<31),
+ VPU_WDT_REG);
+
+ clk_disable_unprepare(vpu->clk);
+}
+
+int vpu_enable_clock(struct platform_device *pdev)
+{
+ struct mtk_vpu *vpu = platform_get_drvdata(pdev);
+ int ret;
+
+ ret = clk_prepare_enable(vpu->clk);
+ if (ret)
+ return ret;
+ /* Enable VPU watchdog */
+ vpu_cfg_writel(vpu, vpu_cfg_readl(vpu, VPU_WDT_REG) | (1L << 31),
+ VPU_WDT_REG);
+
+ return ret;
+}
+
+int vpu_ipi_register(struct platform_device *pdev,
+ enum ipi_id id, ipi_handler_t handler,
+ const char *name, void *priv)
+{
+ struct mtk_vpu *vpu = platform_get_drvdata(pdev);
+ struct vpu_ipi_desc *ipi_desc;
+
+ if (!vpu) {
+ dev_err(&pdev->dev, "vpu device in not ready\n");
+ return -EPROBE_DEFER;
+ }
+
+ if (id < IPI_MAX && handler != NULL) {
+ ipi_desc = vpu->ipi_desc;
+ ipi_desc[id].name = name;
+ ipi_desc[id].handler = handler;
+ ipi_desc[id].priv = priv;
+ return 0;
+ }
+
+ dev_err(&pdev->dev, "register vpu ipi with invalid arguments\n");
+ return -EINVAL;
+}
+
+int vpu_ipi_send(struct platform_device *pdev,
+ enum ipi_id id, void *buf,
+ unsigned int len, unsigned int wait)
+{
+ struct mtk_vpu *vpu = platform_get_drvdata(pdev);
+ struct share_obj *send_obj = vpu->send_buf;
+ unsigned long timeout;
+
+ if (id >= IPI_MAX || len > sizeof(send_obj->share_buf) || buf == NULL) {
+ dev_err(vpu->dev, "failed to send ipi message\n");
+ return -EINVAL;
+ }
+
+ if (!vpu_running(vpu)) {
+ dev_err(vpu->dev, "vpu_ipi_send: VPU is not running\n");
+ return -ENXIO;
+ }
+
+ mutex_lock(&vpu->vpu_mutex);
+ if (vpu_cfg_readl(vpu, HOST_TO_VPU) && !wait) {
+ mutex_unlock(&vpu->vpu_mutex);
+ return -EBUSY;
+ }
+
+ if (wait)
+ while (vpu_cfg_readl(vpu, HOST_TO_VPU))
+ ;
+
+ memcpy((void *)send_obj->share_buf, buf, len);
+ send_obj->len = len;
+ send_obj->id = id;
+ vpu_cfg_writel(vpu, 0x1, HOST_TO_VPU);
+
+ /* Wait until VPU receives the command */
+ timeout = jiffies + msecs_to_jiffies(IPI_TIMEOUT_MS);
+ do {
+ if (time_after(jiffies, timeout)) {
+ dev_err(vpu->dev, "vpu_ipi_send: IPI timeout!\n");
+ return -EIO;
+ }
+ } while (vpu_cfg_readl(vpu, HOST_TO_VPU));
+
+ mutex_unlock(&vpu->vpu_mutex);
+
+ return 0;
+}
+
+void *vpu_mapping_dm_addr(struct platform_device *pdev,
+ void *dtcm_dmem_addr)
+{
+ struct mtk_vpu *vpu = platform_get_drvdata(pdev);
+ unsigned long p_vpu_dtcm = (unsigned long)VPU_DTCM(vpu);
+ unsigned long ul_dtcm_dmem_addr = (unsigned long)(dtcm_dmem_addr);
+
+ if (dtcm_dmem_addr == NULL ||
+ (ul_dtcm_dmem_addr > (VPU_DTCM_SIZE + VPU_EXT_D_SIZE))) {
+ dev_err(vpu->dev, "invalid virtual data memory address\n");
+ return ERR_PTR(-EINVAL);
+ }
+
+ if (ul_dtcm_dmem_addr < VPU_DTCM_SIZE)
+ return (void *)(ul_dtcm_dmem_addr + p_vpu_dtcm);
+
+ return (void *)((ul_dtcm_dmem_addr - VPU_DTCM_SIZE) +
+ VPU_DMEM0_VIRT(vpu));
+}
+
+dma_addr_t *vpu_mapping_iommu_dm_addr(struct platform_device *pdev,
+ void *dmem_addr)
+{
+ unsigned long ul_dmem_addr = (unsigned long)(dmem_addr);
+ struct mtk_vpu *vpu = platform_get_drvdata(pdev);
+
+ if (dmem_addr == NULL ||
+ (ul_dmem_addr < VPU_DTCM_SIZE) ||
+ (ul_dmem_addr > (VPU_DTCM_SIZE + VPU_EXT_D_SIZE))) {
+ dev_err(vpu->dev, "invalid IOMMU data memory address\n");
+ return ERR_PTR(-EINVAL);
+ }
+
+ return (dma_addr_t *)((ul_dmem_addr - VPU_DTCM_SIZE) +
+ VPU_DMEM0_IOVA(vpu));
+}
+
+struct platform_device *vpu_get_plat_device(struct platform_device *pdev)
+{
+ struct device *dev = &pdev->dev;
+ struct device_node *vpu_node;
+ struct platform_device *vpu_pdev;
+
+ vpu_node = of_parse_phandle(dev->of_node, "vpu", 0);
+ if (!vpu_node) {
+ dev_err(dev, "can't get vpu node\n");
+ return NULL;
+ }
+
+ vpu_pdev = of_find_device_by_node(vpu_node);
+ if (WARN_ON(!vpu_pdev)) {
+ dev_err(dev, "vpu pdev failed\n");
+ of_node_put(vpu_node);
+ return NULL;
+ }
+
+ return vpu_pdev;
+}
+
+/* load vpu program/data memory */
+static void load_requested_vpu(struct mtk_vpu *vpu,
+ size_t fw_size,
+ const u8 *fw_data,
+ u8 fw_type)
+{
+ size_t target_size = fw_type ? VPU_DTCM_SIZE : VPU_PTCM_SIZE;
+ size_t extra_fw_size = 0;
+ void *dest;
+
+ /* reset VPU */
+ vpu_cfg_writel(vpu, 0x0, VPU_BASE);
+
+ /* handle extended firmware size */
+ if (fw_size > target_size) {
+ dev_dbg(vpu->dev, "fw size %lx > limited fw size %lx\n",
+ fw_size, target_size);
+ extra_fw_size = fw_size - target_size;
+ dev_dbg(vpu->dev, "extra_fw_size %lx\n", extra_fw_size);
+ fw_size = target_size;
+ }
+ dest = fw_type ? VPU_DTCM(vpu) : VPU_PTCM(vpu);
+ memcpy(dest, fw_data, fw_size);
+ /* download to extended memory if need */
+ if (extra_fw_size > 0) {
+ dest = fw_type ?
+ VPU_DMEM0_VIRT(vpu) : VPU_PMEM0_VIRT(vpu);
+
+ dev_dbg(vpu->dev, "download extended memory type %x\n",
+ fw_type);
+ memcpy(dest, fw_data + target_size, extra_fw_size);
+ }
+}
+
+int vpu_load_firmware(struct platform_device *pdev)
+{
+ struct mtk_vpu *vpu = platform_get_drvdata(pdev);
+ struct device *dev = &pdev->dev;
+ struct vpu_run *run = &vpu->run;
+ const struct firmware *vpu_fw;
+ int ret;
+
+ if (!pdev) {
+ dev_err(dev, "VPU platform device is invalid\n");
+ return -EINVAL;
+ }
+
+ mutex_lock(&vpu->vpu_mutex);
+
+ ret = vpu_enable_clock(pdev);
+ if (ret) {
+ dev_err(dev, "enable clock failed %d\n", ret);
+ goto OUT_LOAD_FW;
+ }
+
+ if (vpu_running(vpu)) {
+ vpu_disable_clock(pdev);
+ mutex_unlock(&vpu->vpu_mutex);
+ dev_warn(dev, "vpu is running already\n");
+ return 0;
+ }
+
+ run->signaled = false;
+ dev_dbg(vpu->dev, "firmware request\n");
+ ret = request_firmware(&vpu_fw, VPU_P_FW, dev);
+ if (ret < 0) {
+ dev_err(dev, "Failed to load %s, %d\n", VPU_P_FW, ret);
+ goto OUT_LOAD_FW;
+ }
+ if (vpu_fw->size > VPU_P_FW_SIZE) {
+ ret = -EFBIG;
+ dev_err(dev, "program fw size %zu is abnormal\n", vpu_fw->size);
+ goto OUT_LOAD_FW;
+ }
+ dev_dbg(vpu->dev, "Downloaded program fw size: %zu.\n",
+ vpu_fw->size);
+ /* Downloading program firmware to device*/
+ load_requested_vpu(vpu, vpu_fw->size, vpu_fw->data,
+ P_FW);
+ release_firmware(vpu_fw);
+
+ ret = request_firmware(&vpu_fw, VPU_D_FW, dev);
+ if (ret < 0) {
+ dev_err(dev, "Failed to load %s, %d\n", VPU_D_FW, ret);
+ goto OUT_LOAD_FW;
+ }
+ if (vpu_fw->size > VPU_D_FW_SIZE) {
+ ret = -EFBIG;
+ dev_err(dev, "data fw size %zu is abnormal\n", vpu_fw->size);
+ goto OUT_LOAD_FW;
+ }
+ dev_dbg(vpu->dev, "Downloaded data fw size: %zu.\n",
+ vpu_fw->size);
+ /* Downloading data firmware to device */
+ load_requested_vpu(vpu, vpu_fw->size, vpu_fw->data,
+ D_FW);
+ release_firmware(vpu_fw);
+ /* boot up vpu */
+ vpu_cfg_writel(vpu, 0x1, VPU_BASE);
+
+ ret = wait_event_interruptible_timeout(run->wq,
+ run->signaled,
+ msecs_to_jiffies(INIT_TIMEOUT_MS)
+ );
+ if (0 == ret) {
+ ret = -ETIME;
+ dev_err(dev, "wait vpu initialization timout!\n");
+ goto OUT_LOAD_FW;
+ } else if (-ERESTARTSYS == ret) {
+ dev_err(dev, "wait vpu interrupted by a signal!\n");
+ goto OUT_LOAD_FW;
+ }
+
+ ret = 0;
+ dev_info(dev, "vpu is ready. Fw version %s\n", run->fw_ver);
+
+OUT_LOAD_FW:
+ vpu_disable_clock(pdev);
+ mutex_unlock(&vpu->vpu_mutex);
+
+ return ret;
+}
+
+static void vpu_init_ipi_handler(void *data, unsigned int len, void *priv)
+{
+ struct mtk_vpu *vpu = (struct mtk_vpu *)priv;
+ struct vpu_run *run = (struct vpu_run *)data;
+
+ vpu->run.signaled = run->signaled;
+ strncpy(vpu->run.fw_ver, run->fw_ver, VPU_FW_VER_LEN);
+ wake_up_interruptible(&vpu->run.wq);
+}
+
+static int vpu_debug_open(struct inode *inode, struct file *file)
+{
+ file->private_data = inode->i_private;
+ return 0;
+}
+
+static ssize_t vpu_debug_read(struct file *file, char __user *user_buf,
+ size_t count, loff_t *ppos)
+{
+ char buf[256];
+ unsigned int len;
+ unsigned int running, pc, vpu_to_host, host_to_vpu, wdt;
+ int ret;
+ struct device *dev = file->private_data;
+ struct platform_device *pdev = to_platform_device(dev);
+ struct mtk_vpu *vpu = dev_get_drvdata(dev);
+
+ ret = vpu_enable_clock(pdev);
+ if (ret) {
+ dev_err(vpu->dev, "[VPU] enable clock failed %d\n", ret);
+ return 0;
+ }
+
+ /* vpu register status */
+ running = vpu_running(vpu);
+ pc = vpu_cfg_readl(vpu, VPU_PC_REG);
+ wdt = vpu_cfg_readl(vpu, VPU_WDT_REG);
+ host_to_vpu = vpu_cfg_readl(vpu, HOST_TO_VPU);
+ vpu_to_host = vpu_cfg_readl(vpu, VPU_TO_HOST);
+ vpu_disable_clock(pdev);
+
+ if (running) {
+ len = sprintf(buf, "VPU is running\n\n"
+ "FW Version: %s\n"
+ "PC: 0x%x\n"
+ "WDT: 0x%x\n"
+ "Host to VPU: 0x%x\n"
+ "VPU to Host: 0x%x\n",
+ vpu->run.fw_ver, pc, wdt,
+ host_to_vpu, vpu_to_host);
+ } else {
+ len = sprintf(buf, "VPU not running\n");
+ }
+
+ return simple_read_from_buffer(user_buf, count, ppos, buf, len);
+}
+
+static const struct file_operations vpu_debug_fops = {
+ .open = vpu_debug_open,
+ .read = vpu_debug_read,
+};
+
+static void vpu_free_p_ext_mem(struct mtk_vpu *vpu)
+{
+ struct device *dev = vpu->dev;
+ struct iommu_domain *domain = iommu_get_domain_for_dev(dev);
+
+ dma_free_coherent(dev, VPU_EXT_P_SIZE, VPU_PMEM0_VIRT(vpu),
+ VPU_PMEM0_IOVA(vpu));
+
+ if (domain)
+ iommu_detach_device(domain, vpu->dev);
+}
+
+static void vpu_free_d_ext_mem(struct mtk_vpu *vpu)
+{
+ struct device *dev = vpu->dev;
+ struct iommu_domain *domain = iommu_get_domain_for_dev(dev);
+
+ dma_free_coherent(dev, VPU_EXT_D_SIZE, VPU_DMEM0_VIRT(vpu),
+ VPU_DMEM0_IOVA(vpu));
+
+ if (domain)
+ iommu_detach_device(domain, dev);
+}
+
+static int vpu_alloc_p_ext_mem(struct mtk_vpu *vpu)
+{
+ struct device *dev = vpu->dev;
+ struct iommu_domain *domain = iommu_get_domain_for_dev(dev);
+ phys_addr_t p_pa;
+
+ VPU_PMEM0_VIRT(vpu) = dma_alloc_coherent(dev,
+ VPU_EXT_P_SIZE,
+ &(VPU_PMEM0_IOVA(vpu)),
+ GFP_KERNEL);
+ if (VPU_PMEM0_VIRT(vpu) == NULL) {
+ dev_err(dev, "Failed to allocate the extended program memory\n");
+ return PTR_ERR(VPU_PMEM0_VIRT(vpu));
+ }
+
+ p_pa = iommu_iova_to_phys(domain, vpu->mem.p_iova);
+ /* Disable extend0. Enable extend1 */
+ vpu_cfg_writel(vpu, 0x1, VPU_PMEM_EXT0_ADDR);
+ vpu_cfg_writel(vpu, (p_pa & 0xFFFFF000), VPU_PMEM_EXT1_ADDR);
+
+ dev_info(dev, "Program extend memory phy=0x%llx virt=0x%p iova=0x%llx\n",
+ (unsigned long long)p_pa,
+ VPU_PMEM0_VIRT(vpu),
+ (unsigned long long)VPU_PMEM0_IOVA(vpu));
+
+ return 0;
+}
+
+static int vpu_alloc_d_ext_mem(struct mtk_vpu *vpu)
+{
+ struct device *dev = vpu->dev;
+ struct iommu_domain *domain = iommu_get_domain_for_dev(dev);
+ phys_addr_t d_pa;
+
+ VPU_DMEM0_VIRT(vpu) = dma_alloc_coherent(dev,
+ VPU_EXT_D_SIZE,
+ &(VPU_DMEM0_IOVA(vpu)),
+ GFP_KERNEL);
+ if (VPU_DMEM0_VIRT(vpu) == NULL) {
+ dev_err(dev, "Failed to allocate the extended data memory\n");
+ return PTR_ERR(VPU_DMEM0_VIRT(vpu));
+ }
+
+ d_pa = iommu_iova_to_phys(domain, vpu->mem.d_iova);
+
+ /* Disable extend0. Enable extend1 */
+ vpu_cfg_writel(vpu, 0x1, VPU_DMEM_EXT0_ADDR);
+ vpu_cfg_writel(vpu, (d_pa & 0xFFFFF000),
+ VPU_DMEM_EXT1_ADDR);
+
+ dev_info(dev, "Data extend memory phy=0x%llx virt=0x%p iova=0x%llx\n",
+ (unsigned long long)d_pa,
+ VPU_DMEM0_VIRT(vpu),
+ (unsigned long long)VPU_DMEM0_IOVA(vpu));
+
+ return 0;
+}
+
+static void vpu_ipi_handler(struct mtk_vpu *vpu)
+{
+ struct share_obj *rcv_obj = vpu->recv_buf;
+ struct vpu_ipi_desc *ipi_desc = vpu->ipi_desc;
+
+ if (rcv_obj->id < IPI_MAX && ipi_desc[rcv_obj->id].handler) {
+ ipi_desc[rcv_obj->id].handler(rcv_obj->share_buf,
+ rcv_obj->len,
+ ipi_desc[rcv_obj->id].priv);
+ } else {
+ dev_err(vpu->dev, "No such ipi id = %d\n", rcv_obj->id);
+ }
+}
+
+static int vpu_ipi_init(struct mtk_vpu *vpu)
+{
+ /* Disable VPU to host interrupt */
+ vpu_cfg_writel(vpu, 0x0, VPU_TO_HOST);
+
+ /* shared buffer initialization */
+ vpu->recv_buf = (struct share_obj *)VPU_DTCM(vpu);
+ vpu->send_buf = vpu->recv_buf + 1;
+ memset(vpu->recv_buf, 0, sizeof(struct share_obj));
+ memset(vpu->send_buf, 0, sizeof(struct share_obj));
+ mutex_init(&vpu->vpu_mutex);
+
+ return 0;
+}
+
+static irqreturn_t vpu_irq_handler(int irq, void *priv)
+{
+ struct mtk_vpu *vpu = priv;
+ uint32_t vpu_to_host = vpu_cfg_readl(vpu, VPU_TO_HOST);
+
+ if (vpu_to_host & VPU_IPC_INT)
+ vpu_ipi_handler(vpu);
+ else
+ dev_err(vpu->dev, "vpu watchdog timeout!\n");
+
+ /* VPU won't send another interrupt until we set VPU_TO_HOST to 0. */
+ vpu_cfg_writel(vpu, 0x0, VPU_TO_HOST);
+
+ return IRQ_HANDLED;
+}
+
+static struct dentry *vpu_debugfs;
+static int mtk_vpu_probe(struct platform_device *pdev)
+{
+ struct mtk_vpu *vpu;
+ struct device *dev;
+ struct resource *res;
+ int ret = 0;
+
+ dev_dbg(&pdev->dev, "initialization\n");
+
+ dev = &pdev->dev;
+ vpu = devm_kzalloc(dev, sizeof(*vpu), GFP_KERNEL);
+ if (!vpu)
+ return -ENOMEM;
+
+ vpu->dev = &pdev->dev;
+ res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "sram");
+ vpu->reg.sram = devm_ioremap_resource(dev, res);
+ if (IS_ERR(vpu->reg.sram)) {
+ dev_err(dev, "devm_ioremap_resource vpu sram failed.\n");
+ return PTR_ERR(vpu->reg.sram);
+ }
+
+ res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "cfg_reg");
+ vpu->reg.cfg = devm_ioremap_resource(dev, res);
+ if (IS_ERR(vpu->reg.cfg)) {
+ dev_err(dev, "devm_ioremap_resource vpu cfg failed.\n");
+ return PTR_ERR(vpu->reg.cfg);
+ }
+
+ /* Get VPU clock */
+ vpu->clk = devm_clk_get(dev, "main");
+ if (vpu->clk == NULL) {
+ dev_err(dev, "get vpu clock fail\n");
+ return -EINVAL;
+ }
+
+ platform_set_drvdata(pdev, vpu);
+
+ ret = vpu_enable_clock(pdev);
+ if (ret) {
+ ret = -EINVAL;
+ return ret;
+ }
+
+ dev_dbg(dev, "vpu ipi init\n");
+ ret = vpu_ipi_init(vpu);
+ if (ret) {
+ dev_err(dev, "Failed to init ipi\n");
+ goto disable_vpu_clk;
+ }
+
+ platform_set_drvdata(pdev, vpu);
+
+ /* register vpu initialization IPI */
+ ret = vpu_ipi_register(pdev, IPI_VPU_INIT, vpu_init_ipi_handler,
+ "vpu_init", vpu);
+ if (ret) {
+ dev_err(dev, "Failed to register IPI_VPU_INIT\n");
+ goto vpu_mutex_destroy;
+ }
+
+ vpu_debugfs = debugfs_create_file("mtk_vpu", S_IRUGO, NULL, (void *)dev,
+ &vpu_debug_fops);
+ if (!vpu_debugfs) {
+ ret = -ENOMEM;
+ goto cleanup_ipi;
+ }
+
+ /* Set PTCM to 96K and DTCM to 32K */
+ vpu_cfg_writel(vpu, 0x2, VPU_TCM_CFG);
+
+ ret = vpu_alloc_p_ext_mem(vpu);
+ if (ret) {
+ dev_err(dev, "Allocate PM failed\n");
+ goto remove_debugfs;
+ }
+
+ ret = vpu_alloc_d_ext_mem(vpu);
+ if (ret) {
+ dev_err(dev, "Allocate DM failed\n");
+ goto free_p_mem;
+ }
+
+ init_waitqueue_head(&vpu->run.wq);
+
+ res = platform_get_resource(pdev, IORESOURCE_IRQ, 0);
+ if (res == NULL) {
+ dev_err(dev, "get IRQ resource failed.\n");
+ ret = -ENXIO;
+ goto free_d_mem;
+ }
+ vpu->reg.irq = platform_get_irq(pdev, 0);
+ ret = devm_request_irq(dev, vpu->reg.irq, vpu_irq_handler, 0,
+ pdev->name, vpu);
+ if (ret) {
+ dev_err(dev, "failed to request irq\n");
+ goto free_d_mem;
+ }
+
+ vpu_disable_clock(pdev);
+ dev_dbg(dev, "initialization completed\n");
+
+ return 0;
+
+free_d_mem:
+ vpu_free_d_ext_mem(vpu);
+free_p_mem:
+ vpu_free_p_ext_mem(vpu);
+remove_debugfs:
+ debugfs_remove(vpu_debugfs);
+cleanup_ipi:
+ memset(vpu->ipi_desc, 0, sizeof(struct vpu_ipi_desc)*IPI_MAX);
+vpu_mutex_destroy:
+ mutex_destroy(&vpu->vpu_mutex);
+disable_vpu_clk:
+ vpu_disable_clock(pdev);
+
+ return ret;
+}
+
+static const struct of_device_id mtk_vpu_match[] = {
+ {
+ .compatible = "mediatek,mt8173-vpu",
+ },
+ {},
+};
+MODULE_DEVICE_TABLE(of, mtk_vpu_match);
+
+static int mtk_vpu_remove(struct platform_device *pdev)
+{
+ struct mtk_vpu *vpu = platform_get_drvdata(pdev);
+
+ vpu_free_p_ext_mem(vpu);
+ vpu_free_d_ext_mem(vpu);
+
+ return 0;
+}
+
+static struct platform_driver mtk_vpu_driver = {
+ .probe = mtk_vpu_probe,
+ .remove = mtk_vpu_remove,
+ .driver = {
+ .name = MTK_VPU_DRV_NAME,
+ .owner = THIS_MODULE,
+ .of_match_table = mtk_vpu_match,
+ },
+};
+
+module_platform_driver(mtk_vpu_driver);
+
+MODULE_LICENSE("GPL v2");
+MODULE_DESCRIPTION("Mediatek Video Prosessor Unit driver");
diff --git a/drivers/media/platform/mtk-vpu/mtk_vpu_core.h b/drivers/media/platform/mtk-vpu/mtk_vpu_core.h
new file mode 100644
index 0000000..20cf2a0
--- /dev/null
+++ b/drivers/media/platform/mtk-vpu/mtk_vpu_core.h
@@ -0,0 +1,161 @@
+/*
+* Copyright (c) 2015 MediaTek Inc.
+* Author: Andrew-CT Chen <andrew-ct.chen-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
+*
+* This program is free software; you can redistribute it and/or modify
+* it under the terms of the GNU General Public License version 2 as
+* published by the Free Software Foundation.
+*
+* This program is distributed in the hope that it will be useful,
+* but WITHOUT ANY WARRANTY; without even the implied warranty of
+* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+* GNU General Public License for more details.
+*/
+
+#ifndef _MTK_VPU_CORE_H
+#define _MTK_VPU_CORE_H
+
+#include <linux/platform_device.h>
+
+/**
+ * VPU (video processor unit) is a tiny processor controlling video hardware
+ * related to video codec, scaling and color format converting.
+ * VPU interfaces with other blocks by share memory and interrupt.
+ **/
+
+typedef void (*ipi_handler_t) (void *data,
+ unsigned int len,
+ void *priv);
+
+/**
+ * enum ipi_id - the id of inter-processor interrupt
+ *
+ * @IPI_VPU_INIT: The interrupt from vpu is to notfiy kernel
+ VPU initialization completed.
+ * @IPI_VENC_H264: The interrupt from vpu is to notify kernel to
+ handle H264 video encoder job, and vice versa.
+ * @IPI_VENC_VP8: The interrupt fro vpu is to notify kernel to
+ handle VP8 video encoder job,, and vice versa.
+ * @IPI_VENC_CAPABILITY: The interrupt from vpu is to
+ get venc hardware capability.
+ * @IPI_MAX: The maximum IPI number
+ */
+enum ipi_id {
+ IPI_VPU_INIT = 0,
+ IPI_VENC_H264,
+ IPI_VENC_VP8,
+ IPI_VENC_CAPABILITY,
+ IPI_MAX,
+};
+
+/**
+ * vpu_disable_clock - Disable VPU clock
+ *
+ * @pdev: VPU platform device
+ *
+ *
+ * Return: Return 0 if the clock is disabled successfully,
+ * otherwise it is failed.
+ *
+ **/
+void vpu_disable_clock(struct platform_device *pdev);
+
+/**
+ * vpu_enable_clock - Enable VPU clock
+ *
+ * @pdev: VPU platform device
+ *
+ * Return: Return 0 if the clock is enabled successfully,
+ * otherwise it is failed.
+ *
+ **/
+int vpu_enable_clock(struct platform_device *pdev);
+
+/**
+ * vpu_ipi_register - register an ipi function
+ *
+ * @pdev: VPU platform device
+ * @id: IPI ID
+ * @handler: IPI handler
+ * @name: IPI name
+ * @priv: private data for IPI handler
+ *
+ * Register an ipi function to receive ipi interrupt from VPU.
+ *
+ * Return: Return 0 if ipi registers successfully, otherwise it is failed.
+ */
+int vpu_ipi_register(struct platform_device *pdev, enum ipi_id id,
+ ipi_handler_t handler, const char *name, void *priv);
+
+/**
+ * vpu_ipi_send - send data from AP to vpu.
+ *
+ * @pdev: VPU platform device
+ * @id: IPI ID
+ * @buf: the data buffer
+ * @len: the data buffer length
+ * @wait: wait for the last ipi completed.
+ *
+ * This function is thread-safe. When this function returns,
+ * VPU has received the data and starts the processing.
+ * When the processing completes, IPI handler registered
+ * by vpu_ipi_register will be called in interrupt context.
+ *
+ * Return: Return 0 if sending data successfully, otherwise it is failed.
+ **/
+int vpu_ipi_send(struct platform_device *pdev,
+ enum ipi_id id, void *buf,
+ unsigned int len,
+ unsigned int wait);
+
+/**
+ * vpu_get_plat_device - get VPU's platform device
+ *
+ * @pdev: the platform device of the module requesting VPU platform
+ * device for using VPU API.
+ *
+ * Return: Return NULL if it is failed.
+ * otherwise it is VPU's platform device
+ **/
+struct platform_device *vpu_get_plat_device(struct platform_device *pdev);
+
+/**
+ * vpu_load_firmware - download VPU firmware and boot it
+ *
+ * @pdev: VPU platform device
+ *
+ * Return: Return 0 if downloading firmware successfully,
+ * otherwise it is failed
+ **/
+int vpu_load_firmware(struct platform_device *pdev);
+
+/**
+ * vpu_mapping_dm_addr - Mapping DTCM/DMEM to kernel virtual address
+ *
+ * @pdev: VPU platform device
+ * @dmem_addr: VPU's data memory address
+ *
+ * Mapping the VPU's DTCM (Data Tightly-Coupled Memory) /
+ * DMEM (Data Extended Memory) memory address to
+ * kernel virtual address.
+ *
+ * Return: Return ERR_PTR(-EINVAL) if mapping failed,
+ * otherwise the mapped kernel virtual address
+ **/
+void *vpu_mapping_dm_addr(struct platform_device *pdev,
+ void *dtcm_dmem_addr);
+
+/**
+ * vpu_mapping_iommu_dm_addr - Mapping to iommu address
+ *
+ * @pdev: VPU platform device
+ * @dmem_addr: VPU's extended data memory address
+ *
+ * Mapping the VPU's extended data address to iommu address
+ *
+ * Return: Return ERR_PTR(-EINVAL) if mapping failed,
+ * otherwise the mapped iommu address
+ **/
+dma_addr_t *vpu_mapping_iommu_dm_addr(struct platform_device *pdev,
+ void *dmem_addr);
+#endif /* _MTK_VPU_CORE_H */
diff --git a/drivers/media/platform/mtk-vpu/vp8_enc/venc_vp8_vpu.h b/drivers/media/platform/mtk-vpu/vp8_enc/venc_vp8_vpu.h
new file mode 100644
index 0000000..4e09eec
--- /dev/null
+++ b/drivers/media/platform/mtk-vpu/vp8_enc/venc_vp8_vpu.h
@@ -0,0 +1,119 @@
+/*
+ * Copyright (c) 2015 MediaTek Inc.
+ * Author: Daniel Hsiao <daniel.hsiao-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef VENC_VP8_VPU_H_
+#define VENC_VP8_VPU_H_
+
+/**
+ * enum venc_vp8_vpu_work_buf - vp8 encoder buffer index
+ */
+enum venc_vp8_vpu_work_buf {
+ VENC_VP8_VPU_WORK_BUF_LUMA,
+ VENC_VP8_VPU_WORK_BUF_LUMA2,
+ VENC_VP8_VPU_WORK_BUF_LUMA3,
+ VENC_VP8_VPU_WORK_BUF_CHROMA,
+ VENC_VP8_VPU_WORK_BUF_CHROMA2,
+ VENC_VP8_VPU_WORK_BUF_CHROMA3,
+ VENC_VP8_VPU_WORK_BUF_MV_INFO,
+ VENC_VP8_VPU_WORK_BUF_BS_HD,
+ VENC_VP8_VPU_WORK_BUF_PROB_BUF,
+ VENC_VP8_VPU_WORK_BUF_RC_INFO,
+ VENC_VP8_VPU_WORK_BUF_RC_CODE,
+ VENC_VP8_VPU_WORK_BUF_RC_CODE2,
+ VENC_VP8_VPU_WORK_BUF_RC_CODE3,
+ VENC_VP8_VPU_WORK_BUF_MAX,
+};
+
+/*
+ * struct venc_vp8_vpu_config - Structure for vp8 encoder configuration
+ * @input_fourcc: input fourcc
+ * @bitrate: target bitrate (in bps)
+ * @pic_w: picture width
+ * @pic_h: picture height
+ * @buf_w: buffer width (with 16 alignment)
+ * @buf_h: buffer height (with 16 alignment)
+ * @intra_period: intra frame period
+ * @framerate: frame rate
+ * @ts_mode: temporal scalability mode (0: disable, 1: enable)
+ * support three temporal layers - 0: 7.5fps 1: 7.5fps 2: 15fps.
+ */
+struct venc_vp8_vpu_config {
+ u32 input_fourcc;
+ u32 bitrate;
+ u32 pic_w;
+ u32 pic_h;
+ u32 buf_w;
+ u32 buf_h;
+ u32 intra_period;
+ u32 framerate;
+ u32 ts_mode;
+};
+
+/*
+ * struct venc_vp8_vpu_buf -Structure for buffer information
+ * @align: buffer alignment (in bytes)
+ * @pa: physical address
+ * @vpua: VPU side memory addr which is used by RC_CODE
+ * @size: buffer size (in bytes)
+ */
+struct venc_vp8_vpu_buf {
+ u32 align;
+ u32 pa;
+ u32 vpua;
+ u32 size;
+};
+
+/*
+ * struct venc_vp8_vpu_drv - Structure for VPU driver control and info share
+ * This structure is allocated in VPU side and shared to AP side.
+ * @config: vp8 encoder configuration
+ * @work_bufs: working buffer information in VPU side
+ * The work_bufs here is for storing the 'size' info shared to AP side.
+ * The similar item in struct venc_vp8_handle is for memory allocation
+ * in AP side. The AP driver will copy the 'size' from here to the one in
+ * struct mtk_vcodec_mem, then invoke mtk_vcodec_mem_alloc to allocate
+ * the buffer. After that, bypass the 'dma_addr' to the 'pa' field here for
+ * register setting in VPU side.
+ */
+struct venc_vp8_vpu_drv {
+ struct venc_vp8_vpu_config config;
+ struct venc_vp8_vpu_buf work_bufs[VENC_VP8_VPU_WORK_BUF_MAX];
+};
+
+/*
+ * struct venc_vp8_vpu_inst - vp8 encoder VPU driver instance
+ * @wq_hd: wait queue used for vpu cmd trigger then wait vpu interrupt done
+ * @signaled: flag used for checking vpu interrupt done
+ * @failure: flag to show vpu cmd succeeds or not
+ * @id: VPU instance id
+ * @drv: driver structure allocated by VPU side and shared to AP side for
+ * control and info share
+ */
+struct venc_vp8_vpu_inst {
+ wait_queue_head_t wq_hd;
+ int signaled;
+ int failure;
+ unsigned int id;
+ struct venc_vp8_vpu_drv *drv;
+};
+
+int vp8_enc_vpu_init(void *handle);
+int vp8_enc_vpu_set_param(void *handle, unsigned int id, void *param);
+int vp8_enc_vpu_encode(void *handle,
+ struct venc_frm_buf *frm_buf,
+ struct mtk_vcodec_mem *bs_buf);
+int vp8_enc_vpu_deinit(void *handle);
+
+#endif
--
1.7.9.5
^ permalink raw reply related [flat|nested] 84+ messages in thread
* [RESEND RFC/PATCH 3/8] media: platform: mtk-vpu: Support Mediatek VPU
@ 2015-11-17 12:54 ` Tiffany Lin
0 siblings, 0 replies; 84+ messages in thread
From: Tiffany Lin @ 2015-11-17 12:54 UTC (permalink / raw)
To: linux-arm-kernel
From: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
The VPU driver for hw video codec embedded in Mediatek's MT8173 SOCs.
It is able to handle video decoding/encoding of in a range of formats.
The driver provides with VPU firmware download, memory management and
the communication interface between CPU and VPU.
For VPU initialization, it will create virtual memory for CPU access and
IOMMU address for vcodec hw device access. When a decode/encode instance
opens a device node, vpu driver will download vpu firmware to the device.
A decode/encode instant will decode/encode a frame using VPU
interface to interrupt vpu to handle decoding/encoding jobs.
Signed-off-by: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
---
drivers/media/platform/Kconfig | 6 +
drivers/media/platform/Makefile | 2 +
drivers/media/platform/mtk-vpu/Makefile | 1 +
.../platform/mtk-vpu/h264_enc/venc_h264_vpu.h | 127 +++
.../media/platform/mtk-vpu/include/venc_ipi_msg.h | 212 +++++
drivers/media/platform/mtk-vpu/mtk_vpu_core.c | 823 ++++++++++++++++++++
drivers/media/platform/mtk-vpu/mtk_vpu_core.h | 161 ++++
.../media/platform/mtk-vpu/vp8_enc/venc_vp8_vpu.h | 119 +++
8 files changed, 1451 insertions(+)
create mode 100644 drivers/media/platform/mtk-vpu/Makefile
create mode 100644 drivers/media/platform/mtk-vpu/h264_enc/venc_h264_vpu.h
create mode 100644 drivers/media/platform/mtk-vpu/include/venc_ipi_msg.h
create mode 100644 drivers/media/platform/mtk-vpu/mtk_vpu_core.c
create mode 100644 drivers/media/platform/mtk-vpu/mtk_vpu_core.h
create mode 100644 drivers/media/platform/mtk-vpu/vp8_enc/venc_vp8_vpu.h
diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
index ccbc974..f98eb47 100644
--- a/drivers/media/platform/Kconfig
+++ b/drivers/media/platform/Kconfig
@@ -148,6 +148,12 @@ config VIDEO_CODA
Coda is a range of video codec IPs that supports
H.264, MPEG-4, and other video formats.
+config MEDIATEK_VPU
+ bool "Mediatek Video Processor Unit"
+ ---help---
+ This driver provides downloading firmware vpu and
+ communicating with vpu.
+
config VIDEO_MEM2MEM_DEINTERLACE
tristate "Deinterlace support"
depends on VIDEO_DEV && VIDEO_V4L2 && DMA_ENGINE
diff --git a/drivers/media/platform/Makefile b/drivers/media/platform/Makefile
index efa0295..1b4c539 100644
--- a/drivers/media/platform/Makefile
+++ b/drivers/media/platform/Makefile
@@ -55,3 +55,5 @@ obj-$(CONFIG_VIDEO_AM437X_VPFE) += am437x/
obj-$(CONFIG_VIDEO_XILINX) += xilinx/
ccflags-y += -I$(srctree)/drivers/media/i2c
+
+obj-$(CONFIG_MEDIATEK_VPU) += mtk-vpu/
diff --git a/drivers/media/platform/mtk-vpu/Makefile b/drivers/media/platform/mtk-vpu/Makefile
new file mode 100644
index 0000000..5de84d1
--- /dev/null
+++ b/drivers/media/platform/mtk-vpu/Makefile
@@ -0,0 +1 @@
+obj-$(CONFIG_MEDIATEK_VPU) += mtk_vpu_core.o
diff --git a/drivers/media/platform/mtk-vpu/h264_enc/venc_h264_vpu.h b/drivers/media/platform/mtk-vpu/h264_enc/venc_h264_vpu.h
new file mode 100644
index 0000000..9c8ebdd
--- /dev/null
+++ b/drivers/media/platform/mtk-vpu/h264_enc/venc_h264_vpu.h
@@ -0,0 +1,127 @@
+/*
+ * Copyright (c) 2015 MediaTek Inc.
+ * Author: Jungchang Tsao <jungchang.tsao@mediatek.com>
+ * Daniel Hsiao <daniel.hsiao@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef VENC_H264_VPU_H_
+#define VENC_H264_VPU_H_
+
+/**
+ * enum venc_h264_vpu_work_buf - h264 encoder buffer index
+ */
+enum venc_h264_vpu_work_buf {
+ VENC_H264_VPU_WORK_BUF_RC_INFO,
+ VENC_H264_VPU_WORK_BUF_RC_CODE,
+ VENC_H264_VPU_WORK_BUF_REC_LUMA,
+ VENC_H264_VPU_WORK_BUF_REC_CHROMA,
+ VENC_H264_VPU_WORK_BUF_REF_LUMA,
+ VENC_H264_VPU_WORK_BUF_REF_CHROMA,
+ VENC_H264_VPU_WORK_BUF_MV_INFO_1,
+ VENC_H264_VPU_WORK_BUF_MV_INFO_2,
+ VENC_H264_VPU_WORK_BUF_SKIP_FRAME,
+ VENC_H264_VPU_WORK_BUF_MAX,
+};
+
+/**
+ * enum venc_h264_bs_mode - for bs_mode argument in h264_enc_vpu_encode
+ */
+enum venc_h264_bs_mode {
+ H264_BS_MODE_SPS,
+ H264_BS_MODE_PPS,
+ H264_BS_MODE_FRAME,
+};
+
+/*
+ * struct venc_h264_vpu_config - Structure for h264 encoder configuration
+ * @input_fourcc: input fourcc
+ * @bitrate: target bitrate (in bps)
+ * @pic_w: picture width
+ * @pic_h: picture height
+ * @buf_w: buffer width
+ * @buf_h: buffer height
+ * @intra_period: intra frame period
+ * @framerate: frame rate
+ * @profile: as specified in standard
+ * @level: as specified in standard
+ * @wfd: WFD mode 1:on, 0:off
+ */
+struct venc_h264_vpu_config {
+ u32 input_fourcc;
+ u32 bitrate;
+ u32 pic_w;
+ u32 pic_h;
+ u32 buf_w;
+ u32 buf_h;
+ u32 intra_period;
+ u32 framerate;
+ u32 profile;
+ u32 level;
+ u32 wfd;
+};
+
+/*
+ * struct venc_h264_vpu_buf - Structure for buffer information
+ * @align: buffer alignment (in bytes)
+ * @pa: physical address
+ * @vpua: VPU side memory addr which is used by RC_CODE
+ * @size: buffer size (in bytes)
+ */
+struct venc_h264_vpu_buf {
+ u32 align;
+ u32 pa;
+ u32 vpua;
+ u32 size;
+};
+
+/*
+ * struct venc_h264_vpu_drv - Structure for VPU driver control and info share
+ * @config: h264 encoder configuration
+ * @work_bufs: working buffer information
+ */
+struct venc_h264_vpu_drv {
+ struct venc_h264_vpu_config config;
+ struct venc_h264_vpu_buf work_bufs[VENC_H264_VPU_WORK_BUF_MAX];
+};
+
+/*
+ * struct venc_h264_vpu_inst - h264 encoder VPU driver instance
+ * @wq_hd: wait queue used for vpu cmd trigger then wait vpu interrupt done
+ * @signaled: flag used for checking vpu interrupt done
+ * @failure: flag to show vpu cmd succeeds or not
+ * @state: enum venc_ipi_msg_enc_state
+ * @bs_size: bitstream size for skip frame case usage
+ * @wait_int: flag to wait interrupt done (0: for skip frame case, 1: normal case)
+ * @id: VPU instance id
+ * @drv: driver structure allocated by VPU side for control and info share
+ */
+struct venc_h264_vpu_inst {
+ wait_queue_head_t wq_hd;
+ int signaled;
+ int failure;
+ int state;
+ int bs_size;
+ int wait_int;
+ unsigned int id;
+ struct venc_h264_vpu_drv *drv;
+};
+
+int h264_enc_vpu_init(void *handle);
+int h264_enc_vpu_set_param(void *handle, unsigned int id, void *param);
+int h264_enc_vpu_encode(void *handle, unsigned int bs_mode,
+ struct venc_frm_buf *frm_buf,
+ struct mtk_vcodec_mem *bs_buf,
+ unsigned int *bs_size);
+int h264_enc_vpu_deinit(void *handle);
+
+#endif
diff --git a/drivers/media/platform/mtk-vpu/include/venc_ipi_msg.h b/drivers/media/platform/mtk-vpu/include/venc_ipi_msg.h
new file mode 100644
index 0000000..a345b98
--- /dev/null
+++ b/drivers/media/platform/mtk-vpu/include/venc_ipi_msg.h
@@ -0,0 +1,212 @@
+/*
+ * Copyright (c) 2015 MediaTek Inc.
+ * Author: Jungchang Tsao <jungchang.tsao@mediatek.com>
+ * Daniel Hsiao <daniel.hsiao@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef _VENC_IPI_MSG_H_
+#define _VENC_IPI_MSG_H_
+
+#define IPIMSG_H264_ENC_ID 0x100
+#define IPIMSG_VP8_ENC_ID 0x200
+
+#define AP_IPIMSG_VENC_BASE 0x20000
+#define VPU_IPIMSG_VENC_BASE 0x30000
+
+/**
+ * enum venc_ipi_msg_id - message id between AP and VPU
+ * (ipi stands for inter-processor interrupt)
+ * @AP_IPIMSG_XXX: AP to VPU cmd message id
+ * @VPU_IPIMSG_XXX_DONE: VPU ack AP cmd message id
+ */
+enum venc_ipi_msg_id {
+ AP_IPIMSG_H264_ENC_INIT = AP_IPIMSG_VENC_BASE +
+ IPIMSG_H264_ENC_ID,
+ AP_IPIMSG_H264_ENC_SET_PARAM,
+ AP_IPIMSG_H264_ENC_ENCODE,
+ AP_IPIMSG_H264_ENC_DEINIT,
+
+ AP_IPIMSG_VP8_ENC_INIT = AP_IPIMSG_VENC_BASE +
+ IPIMSG_VP8_ENC_ID,
+ AP_IPIMSG_VP8_ENC_SET_PARAM,
+ AP_IPIMSG_VP8_ENC_ENCODE,
+ AP_IPIMSG_VP8_ENC_DEINIT,
+
+ VPU_IPIMSG_H264_ENC_INIT_DONE = VPU_IPIMSG_VENC_BASE +
+ IPIMSG_H264_ENC_ID,
+ VPU_IPIMSG_H264_ENC_SET_PARAM_DONE,
+ VPU_IPIMSG_H264_ENC_ENCODE_DONE,
+ VPU_IPIMSG_H264_ENC_DEINIT_DONE,
+
+ VPU_IPIMSG_VP8_ENC_INIT_DONE = VPU_IPIMSG_VENC_BASE +
+ IPIMSG_VP8_ENC_ID,
+ VPU_IPIMSG_VP8_ENC_SET_PARAM_DONE,
+ VPU_IPIMSG_VP8_ENC_ENCODE_DONE,
+ VPU_IPIMSG_VP8_ENC_DEINIT_DONE,
+};
+
+/**
+ * struct venc_ap_ipi_msg_init - AP to VPU init cmd structure
+ * @msg_id: message id (AP_IPIMSG_XXX_ENC_INIT)
+ * @venc_inst: AP encoder instance (struct venc_vp8_handle/venc_h264_handle *)
+ */
+struct venc_ap_ipi_msg_init {
+ uint32_t msg_id;
+ uint32_t reserved;
+ uint64_t venc_inst;
+};
+
+/**
+ * struct venc_ap_ipi_msg_set_param - AP to VPU set_param cmd structure
+ * @msg_id: message id (AP_IPIMSG_XXX_ENC_SET_PARAM)
+ * @inst_id: VPU encoder instance id (struct venc_vp8_vpu_drv/venc_h264_vpu_drv *)
+ * @param_id: parameter id (venc_set_param_type)
+ * @data_item: number of items in the data array
+ * @data[8]: data array to store the set parameters
+ */
+struct venc_ap_ipi_msg_set_param {
+ uint32_t msg_id;
+ uint32_t inst_id;
+ uint32_t param_id;
+ uint32_t data_item;
+ uint32_t data[8];
+};
+
+/**
+ * struct venc_ap_ipi_msg_enc - AP to VPU enc cmd structure
+ * @msg_id: message id (AP_IPIMSG_XXX_ENC_ENCODE)
+ * @inst_id: VPU encoder instance id (struct venc_vp8_vpu_drv/venc_h264_vpu_drv *)
+ * @bs_mode: bitstream mode for h264
+ * (H264_BS_MODE_SPS/H264_BS_MODE_PPS/H264_BS_MODE_FRAME)
+ * @input_addr: pointer to input image buffer plane
+ * @bs_addr: pointer to output bit stream buffer
+ * @bs_size: bit stream buffer size
+ */
+struct venc_ap_ipi_msg_enc {
+ uint32_t msg_id;
+ uint32_t inst_id;
+ uint32_t bs_mode;
+ uint32_t input_addr[3];
+ uint32_t bs_addr;
+ uint32_t bs_size;
+};
+
+/**
+ * struct venc_ap_ipi_msg_deinit - AP to VPU deinit cmd structure
+ * @msg_id: message id (AP_IPIMSG_XXX_ENC_DEINIT)
+ * @inst_id: VPU encoder instance id (struct venc_vp8_vpu_drv/venc_h264_vpu_drv *)
+ */
+struct venc_ap_ipi_msg_deinit {
+ uint32_t msg_id;
+ uint32_t inst_id;
+};
+
+/**
+ * enum venc_ipi_msg_status - VPU ack AP cmd status
+ */
+enum venc_ipi_msg_status {
+ VENC_IPI_MSG_STATUS_OK,
+ VENC_IPI_MSG_STATUS_FAIL,
+};
+
+/**
+ * struct venc_vpu_ipi_msg_common - VPU ack AP cmd common structure
+ * @msg_id: message id (VPU_IPIMSG_XXX_DONE)
+ * @status: cmd status (venc_ipi_msg_status)
+ * @venc_inst: AP encoder instance (struct venc_vp8_handle/venc_h264_handle *)
+ */
+struct venc_vpu_ipi_msg_common {
+ uint32_t msg_id;
+ uint32_t status;
+ uint64_t venc_inst;
+};
+
+/**
+ * struct venc_vpu_ipi_msg_init - VPU ack AP init cmd structure
+ * @msg_id: message id (VPU_IPIMSG_XXX_ENC_SET_PARAM_DONE)
+ * @status: cmd status (venc_ipi_msg_status)
+ * @venc_inst: AP encoder instance (struct venc_vp8_handle/venc_h264_handle *)
+ * @inst_id: VPU encoder instance id (struct venc_vp8_vpu_drv/venc_h264_vpu_drv *)
+ */
+struct venc_vpu_ipi_msg_init {
+ uint32_t msg_id;
+ uint32_t status;
+ uint64_t venc_inst;
+ uint32_t inst_id;
+ uint32_t reserved;
+};
+
+/**
+ * struct venc_vpu_ipi_msg_set_param - VPU ack AP set_param cmd structure
+ * @msg_id: message id (VPU_IPIMSG_XXX_ENC_SET_PARAM_DONE)
+ * @status: cmd status (venc_ipi_msg_status)
+ * @venc_inst: AP encoder instance (struct venc_vp8_handle/venc_h264_handle *)
+ * @param_id: parameter id (venc_set_param_type)
+ * @data_item: number of items in the data array
+ * @data[6]: data array to store the return result
+ */
+struct venc_vpu_ipi_msg_set_param {
+ uint32_t msg_id;
+ uint32_t status;
+ uint64_t venc_inst;
+ uint32_t param_id;
+ uint32_t data_item;
+ uint32_t data[6];
+};
+
+/**
+ * enum venc_ipi_msg_enc_state - Type of encode state
+ * VEN_IPI_MSG_ENC_STATE_FRAME: one frame being encoded
+ * VEN_IPI_MSG_ENC_STATE_PART: bit stream buffer full
+ * VEN_IPI_MSG_ENC_STATE_SKIP: encoded skip frame
+ * VEN_IPI_MSG_ENC_STATE_ERROR: encounter error
+ */
+enum venc_ipi_msg_enc_state {
+ VEN_IPI_MSG_ENC_STATE_FRAME,
+ VEN_IPI_MSG_ENC_STATE_PART,
+ VEN_IPI_MSG_ENC_STATE_SKIP,
+ VEN_IPI_MSG_ENC_STATE_ERROR,
+};
+
+/**
+ * struct venc_vpu_ipi_msg_enc - VPU ack AP enc cmd structure
+ * @msg_id: message id (VPU_IPIMSG_XXX_ENC_ENCODE_DONE)
+ * @status: cmd status (venc_ipi_msg_status)
+ * @venc_inst: AP encoder instance (struct venc_vp8_handle/venc_h264_handle *)
+ * @state: encode state (venc_ipi_msg_enc_state)
+ * @key_frame: whether the encoded frame is key frame
+ * @bs_size: encoded bitstream size
+ */
+struct venc_vpu_ipi_msg_enc {
+ uint32_t msg_id;
+ uint32_t status;
+ uint64_t venc_inst;
+ uint32_t state;
+ uint32_t key_frame;
+ uint32_t bs_size;
+ uint32_t reserved;
+};
+
+/**
+ * struct venc_vpu_ipi_msg_deinit - VPU ack AP deinit cmd structure
+ * @msg_id: message id (VPU_IPIMSG_XXX_ENC_DEINIT_DONE)
+ * @status: cmd status (venc_ipi_msg_status)
+ * @venc_inst: AP encoder instance (struct venc_vp8_handle/venc_h264_handle *)
+ */
+struct venc_vpu_ipi_msg_deinit {
+ uint32_t msg_id;
+ uint32_t status;
+ uint64_t venc_inst;
+};
+
+#endif /* _VENC_IPI_MSG_H_ */
diff --git a/drivers/media/platform/mtk-vpu/mtk_vpu_core.c b/drivers/media/platform/mtk-vpu/mtk_vpu_core.c
new file mode 100644
index 0000000..b524dbc
--- /dev/null
+++ b/drivers/media/platform/mtk-vpu/mtk_vpu_core.c
@@ -0,0 +1,823 @@
+/*
+* Copyright (c) 2015 MediaTek Inc.
+* Author: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
+*
+* This program is free software; you can redistribute it and/or modify
+* it under the terms of the GNU General Public License version 2 as
+* published by the Free Software Foundation.
+*
+* This program is distributed in the hope that it will be useful,
+* but WITHOUT ANY WARRANTY; without even the implied warranty of
+* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+* GNU General Public License for more details.
+*/
+#include <linux/clk.h>
+#include <linux/debugfs.h>
+#include <linux/firmware.h>
+#include <linux/interrupt.h>
+#include <linux/iommu.h>
+#include <linux/module.h>
+#include <linux/of_address.h>
+#include <linux/of_irq.h>
+#include <linux/of_platform.h>
+#include <linux/sched.h>
+#include <linux/sizes.h>
+
+#include "mtk_vpu_core.h"
+
+/**
+ * VPU (video processor unit) is a tiny processor controlling video hardware
+ * related to video codec, scaling and color format converting.
+ * VPU interfaces with other blocks by share memory and interrupt.
+ **/
+#define MTK_VPU_DRV_NAME "mtk_vpu"
+
+#define INIT_TIMEOUT_MS 2000U
+#define IPI_TIMEOUT_MS 2000U
+#define VPU_FW_VER_LEN 16
+
+/* vpu extended virtural address */
+#define VPU_PMEM0_VIRT(vpu) ((vpu)->mem.p_va)
+#define VPU_DMEM0_VIRT(vpu) ((vpu)->mem.d_va)
+/* vpu extended iova address*/
+#define VPU_PMEM0_IOVA(vpu) ((vpu)->mem.p_iova)
+#define VPU_DMEM0_IOVA(vpu) ((vpu)->mem.d_iova)
+
+#define VPU_PTCM(dev) ((dev)->reg.sram)
+#define VPU_DTCM(dev) ((dev)->reg.sram + VPU_DTCM_OFFSET)
+
+#define VPU_PTCM_SIZE (96 * SZ_1K)
+#define VPU_DTCM_SIZE (32 * SZ_1K)
+#define VPU_DTCM_OFFSET 0x18000UL
+#define VPU_EXT_P_SIZE SZ_1M
+#define VPU_EXT_D_SIZE SZ_4M
+#define VPU_P_FW_SIZE (VPU_PTCM_SIZE + VPU_EXT_P_SIZE)
+#define VPU_D_FW_SIZE (VPU_DTCM_SIZE + VPU_EXT_D_SIZE)
+#define SHARE_BUF_SIZE 48
+
+#define VPU_P_FW "vpu_p.bin"
+#define VPU_D_FW "vpu_d.bin"
+
+#define VPU_BASE 0x0
+#define VPU_TCM_CFG 0x0008
+#define VPU_PMEM_EXT0_ADDR 0x000C
+#define VPU_PMEM_EXT1_ADDR 0x0010
+#define VPU_TO_HOST 0x001C
+#define VPU_DMEM_EXT0_ADDR 0x0014
+#define VPU_DMEM_EXT1_ADDR 0x0018
+#define HOST_TO_VPU 0x0024
+#define VPU_PC_REG 0x0060
+#define VPU_WDT_REG 0x0084
+
+/* vpu inter-processor communication interrupt */
+#define VPU_IPC_INT BIT(8)
+
+/**
+ * enum vpu_fw_type - VPU firmware type
+ *
+ * @P_FW: program firmware
+ * @D_FW: data firmware
+ *
+ */
+enum vpu_fw_type {
+ P_FW,
+ D_FW,
+};
+
+/**
+ * struct vpu_mem - VPU memory information
+ *
+ * @p_va: the kernel virtual memory address of
+ * VPU extended program memory
+ * @d_va: the kernel virtual memory address of VPU extended data memory
+ * @p_iova: the iova memory address of VPU extended program memory
+ * @d_iova: the iova memory address of VPU extended data memory
+ */
+struct vpu_mem {
+ void *p_va;
+ void *d_va;
+ dma_addr_t p_iova;
+ dma_addr_t d_iova;
+};
+
+/**
+ * struct vpu_regs - VPU SRAM and configuration registers
+ *
+ * @sram: the register for VPU sram
+ * @cfg: the register for VPU configuration
+ * @irq: the irq number for VPU interrupt
+ */
+struct vpu_regs {
+ void __iomem *sram;
+ void __iomem *cfg;
+ int irq;
+};
+
+/**
+ * struct vpu_run - VPU initialization status
+ *
+ * @signaled: the signal of vpu initialization completed
+ * @fw_ver: VPU firmware version
+ * @wq: wait queue for VPU initialization status
+ */
+struct vpu_run {
+ u32 signaled;
+ char fw_ver[VPU_FW_VER_LEN];
+ wait_queue_head_t wq;
+};
+
+/**
+ * struct vpu_ipi_desc - VPU IPI descriptor
+ *
+ * @handler: IPI handler
+ * @name: the name of IPI handler
+ * @priv: the private data of IPI handler
+ */
+struct vpu_ipi_desc {
+ ipi_handler_t handler;
+ const char *name;
+ void *priv;
+};
+
+/**
+ * struct share_obj - The DTCM (Data Tightly-Coupled Memory) buffer shared with
+ * AP and VPU
+ *
+ * @id: IPI id
+ * @len: share buffer length
+ * @share_buf: share buffer data
+ */
+struct share_obj {
+ int32_t id;
+ uint32_t len;
+ unsigned char share_buf[SHARE_BUF_SIZE];
+};
+
+/**
+ * struct mtk_vpu - vpu driver data
+ * @mem: VPU extended memory information
+ * @reg: VPU SRAM and configuration registers
+ * @run: VPU initialization status
+ * @ipi_desc: VPU IPI descriptor
+ * @recv_buf: VPU DTCM share buffer for receiving. The
+ * receive buffer is only accessed in interrupt context.
+ * @send_buf: VPU DTCM share buffer for sending
+ * @dev: VPU struct device
+ * @clk: VPU clock on/off
+ * @vpu_mutex: protect mtk_vpu (except recv_buf) and ensure only
+ * one client to use VPU service at a time. For example,
+ * suppose a client is using VPU to decode VP8.
+ * If the other client wants to encode VP8,
+ * it has to wait until VP8 decode completes.
+ *
+ */
+struct mtk_vpu {
+ struct vpu_mem mem;
+ struct vpu_regs reg;
+ struct vpu_run run;
+ struct vpu_ipi_desc ipi_desc[IPI_MAX];
+ struct share_obj *recv_buf;
+ struct share_obj *send_buf;
+ struct device *dev;
+ struct clk *clk;
+ struct mutex vpu_mutex; /* for protecting vpu data data structure */
+};
+
+/* the thread calls the function should hold the |vpu_mutex| */
+static inline void vpu_cfg_writel(struct mtk_vpu *vpu, u32 val, u32 offset)
+{
+ writel(val, vpu->reg.cfg + offset);
+}
+
+static inline u32 vpu_cfg_readl(struct mtk_vpu *vpu, u32 offset)
+{
+ return readl(vpu->reg.cfg + offset);
+}
+
+static inline bool vpu_running(struct mtk_vpu *vpu)
+{
+ return vpu_cfg_readl(vpu, VPU_BASE) & BIT(0);
+}
+
+void vpu_disable_clock(struct platform_device *pdev)
+{
+ struct mtk_vpu *vpu = platform_get_drvdata(pdev);
+
+ /* Disable VPU watchdog */
+ vpu_cfg_writel(vpu,
+ vpu_cfg_readl(vpu, VPU_WDT_REG) & ~(1L<<31),
+ VPU_WDT_REG);
+
+ clk_disable_unprepare(vpu->clk);
+}
+
+int vpu_enable_clock(struct platform_device *pdev)
+{
+ struct mtk_vpu *vpu = platform_get_drvdata(pdev);
+ int ret;
+
+ ret = clk_prepare_enable(vpu->clk);
+ if (ret)
+ return ret;
+ /* Enable VPU watchdog */
+ vpu_cfg_writel(vpu, vpu_cfg_readl(vpu, VPU_WDT_REG) | (1L << 31),
+ VPU_WDT_REG);
+
+ return ret;
+}
+
+int vpu_ipi_register(struct platform_device *pdev,
+ enum ipi_id id, ipi_handler_t handler,
+ const char *name, void *priv)
+{
+ struct mtk_vpu *vpu = platform_get_drvdata(pdev);
+ struct vpu_ipi_desc *ipi_desc;
+
+ if (!vpu) {
+ dev_err(&pdev->dev, "vpu device in not ready\n");
+ return -EPROBE_DEFER;
+ }
+
+ if (id < IPI_MAX && handler != NULL) {
+ ipi_desc = vpu->ipi_desc;
+ ipi_desc[id].name = name;
+ ipi_desc[id].handler = handler;
+ ipi_desc[id].priv = priv;
+ return 0;
+ }
+
+ dev_err(&pdev->dev, "register vpu ipi with invalid arguments\n");
+ return -EINVAL;
+}
+
+int vpu_ipi_send(struct platform_device *pdev,
+ enum ipi_id id, void *buf,
+ unsigned int len, unsigned int wait)
+{
+ struct mtk_vpu *vpu = platform_get_drvdata(pdev);
+ struct share_obj *send_obj = vpu->send_buf;
+ unsigned long timeout;
+
+ if (id >= IPI_MAX || len > sizeof(send_obj->share_buf) || buf == NULL) {
+ dev_err(vpu->dev, "failed to send ipi message\n");
+ return -EINVAL;
+ }
+
+ if (!vpu_running(vpu)) {
+ dev_err(vpu->dev, "vpu_ipi_send: VPU is not running\n");
+ return -ENXIO;
+ }
+
+ mutex_lock(&vpu->vpu_mutex);
+ if (vpu_cfg_readl(vpu, HOST_TO_VPU) && !wait) {
+ mutex_unlock(&vpu->vpu_mutex);
+ return -EBUSY;
+ }
+
+ if (wait)
+ while (vpu_cfg_readl(vpu, HOST_TO_VPU))
+ ;
+
+ memcpy((void *)send_obj->share_buf, buf, len);
+ send_obj->len = len;
+ send_obj->id = id;
+ vpu_cfg_writel(vpu, 0x1, HOST_TO_VPU);
+
+ /* Wait until VPU receives the command */
+ timeout = jiffies + msecs_to_jiffies(IPI_TIMEOUT_MS);
+ do {
+ if (time_after(jiffies, timeout)) {
+ dev_err(vpu->dev, "vpu_ipi_send: IPI timeout!\n");
+ return -EIO;
+ }
+ } while (vpu_cfg_readl(vpu, HOST_TO_VPU));
+
+ mutex_unlock(&vpu->vpu_mutex);
+
+ return 0;
+}
+
+void *vpu_mapping_dm_addr(struct platform_device *pdev,
+ void *dtcm_dmem_addr)
+{
+ struct mtk_vpu *vpu = platform_get_drvdata(pdev);
+ unsigned long p_vpu_dtcm = (unsigned long)VPU_DTCM(vpu);
+ unsigned long ul_dtcm_dmem_addr = (unsigned long)(dtcm_dmem_addr);
+
+ if (dtcm_dmem_addr == NULL ||
+ (ul_dtcm_dmem_addr > (VPU_DTCM_SIZE + VPU_EXT_D_SIZE))) {
+ dev_err(vpu->dev, "invalid virtual data memory address\n");
+ return ERR_PTR(-EINVAL);
+ }
+
+ if (ul_dtcm_dmem_addr < VPU_DTCM_SIZE)
+ return (void *)(ul_dtcm_dmem_addr + p_vpu_dtcm);
+
+ return (void *)((ul_dtcm_dmem_addr - VPU_DTCM_SIZE) +
+ VPU_DMEM0_VIRT(vpu));
+}
+
+dma_addr_t *vpu_mapping_iommu_dm_addr(struct platform_device *pdev,
+ void *dmem_addr)
+{
+ unsigned long ul_dmem_addr = (unsigned long)(dmem_addr);
+ struct mtk_vpu *vpu = platform_get_drvdata(pdev);
+
+ if (dmem_addr == NULL ||
+ (ul_dmem_addr < VPU_DTCM_SIZE) ||
+ (ul_dmem_addr > (VPU_DTCM_SIZE + VPU_EXT_D_SIZE))) {
+ dev_err(vpu->dev, "invalid IOMMU data memory address\n");
+ return ERR_PTR(-EINVAL);
+ }
+
+ return (dma_addr_t *)((ul_dmem_addr - VPU_DTCM_SIZE) +
+ VPU_DMEM0_IOVA(vpu));
+}
+
+struct platform_device *vpu_get_plat_device(struct platform_device *pdev)
+{
+ struct device *dev = &pdev->dev;
+ struct device_node *vpu_node;
+ struct platform_device *vpu_pdev;
+
+ vpu_node = of_parse_phandle(dev->of_node, "vpu", 0);
+ if (!vpu_node) {
+ dev_err(dev, "can't get vpu node\n");
+ return NULL;
+ }
+
+ vpu_pdev = of_find_device_by_node(vpu_node);
+ if (WARN_ON(!vpu_pdev)) {
+ dev_err(dev, "vpu pdev failed\n");
+ of_node_put(vpu_node);
+ return NULL;
+ }
+
+ return vpu_pdev;
+}
+
+/* load vpu program/data memory */
+static void load_requested_vpu(struct mtk_vpu *vpu,
+ size_t fw_size,
+ const u8 *fw_data,
+ u8 fw_type)
+{
+ size_t target_size = fw_type ? VPU_DTCM_SIZE : VPU_PTCM_SIZE;
+ size_t extra_fw_size = 0;
+ void *dest;
+
+ /* reset VPU */
+ vpu_cfg_writel(vpu, 0x0, VPU_BASE);
+
+ /* handle extended firmware size */
+ if (fw_size > target_size) {
+ dev_dbg(vpu->dev, "fw size %lx > limited fw size %lx\n",
+ fw_size, target_size);
+ extra_fw_size = fw_size - target_size;
+ dev_dbg(vpu->dev, "extra_fw_size %lx\n", extra_fw_size);
+ fw_size = target_size;
+ }
+ dest = fw_type ? VPU_DTCM(vpu) : VPU_PTCM(vpu);
+ memcpy(dest, fw_data, fw_size);
+ /* download to extended memory if need */
+ if (extra_fw_size > 0) {
+ dest = fw_type ?
+ VPU_DMEM0_VIRT(vpu) : VPU_PMEM0_VIRT(vpu);
+
+ dev_dbg(vpu->dev, "download extended memory type %x\n",
+ fw_type);
+ memcpy(dest, fw_data + target_size, extra_fw_size);
+ }
+}
+
+int vpu_load_firmware(struct platform_device *pdev)
+{
+ struct mtk_vpu *vpu = platform_get_drvdata(pdev);
+ struct device *dev = &pdev->dev;
+ struct vpu_run *run = &vpu->run;
+ const struct firmware *vpu_fw;
+ int ret;
+
+ if (!pdev) {
+ dev_err(dev, "VPU platform device is invalid\n");
+ return -EINVAL;
+ }
+
+ mutex_lock(&vpu->vpu_mutex);
+
+ ret = vpu_enable_clock(pdev);
+ if (ret) {
+ dev_err(dev, "enable clock failed %d\n", ret);
+ goto OUT_LOAD_FW;
+ }
+
+ if (vpu_running(vpu)) {
+ vpu_disable_clock(pdev);
+ mutex_unlock(&vpu->vpu_mutex);
+ dev_warn(dev, "vpu is running already\n");
+ return 0;
+ }
+
+ run->signaled = false;
+ dev_dbg(vpu->dev, "firmware request\n");
+ ret = request_firmware(&vpu_fw, VPU_P_FW, dev);
+ if (ret < 0) {
+ dev_err(dev, "Failed to load %s, %d\n", VPU_P_FW, ret);
+ goto OUT_LOAD_FW;
+ }
+ if (vpu_fw->size > VPU_P_FW_SIZE) {
+ ret = -EFBIG;
+ dev_err(dev, "program fw size %zu is abnormal\n", vpu_fw->size);
+ goto OUT_LOAD_FW;
+ }
+ dev_dbg(vpu->dev, "Downloaded program fw size: %zu.\n",
+ vpu_fw->size);
+ /* Downloading program firmware to device*/
+ load_requested_vpu(vpu, vpu_fw->size, vpu_fw->data,
+ P_FW);
+ release_firmware(vpu_fw);
+
+ ret = request_firmware(&vpu_fw, VPU_D_FW, dev);
+ if (ret < 0) {
+ dev_err(dev, "Failed to load %s, %d\n", VPU_D_FW, ret);
+ goto OUT_LOAD_FW;
+ }
+ if (vpu_fw->size > VPU_D_FW_SIZE) {
+ ret = -EFBIG;
+ dev_err(dev, "data fw size %zu is abnormal\n", vpu_fw->size);
+ goto OUT_LOAD_FW;
+ }
+ dev_dbg(vpu->dev, "Downloaded data fw size: %zu.\n",
+ vpu_fw->size);
+ /* Downloading data firmware to device */
+ load_requested_vpu(vpu, vpu_fw->size, vpu_fw->data,
+ D_FW);
+ release_firmware(vpu_fw);
+ /* boot up vpu */
+ vpu_cfg_writel(vpu, 0x1, VPU_BASE);
+
+ ret = wait_event_interruptible_timeout(run->wq,
+ run->signaled,
+ msecs_to_jiffies(INIT_TIMEOUT_MS)
+ );
+ if (0 == ret) {
+ ret = -ETIME;
+ dev_err(dev, "wait vpu initialization timout!\n");
+ goto OUT_LOAD_FW;
+ } else if (-ERESTARTSYS == ret) {
+ dev_err(dev, "wait vpu interrupted by a signal!\n");
+ goto OUT_LOAD_FW;
+ }
+
+ ret = 0;
+ dev_info(dev, "vpu is ready. Fw version %s\n", run->fw_ver);
+
+OUT_LOAD_FW:
+ vpu_disable_clock(pdev);
+ mutex_unlock(&vpu->vpu_mutex);
+
+ return ret;
+}
+
+static void vpu_init_ipi_handler(void *data, unsigned int len, void *priv)
+{
+ struct mtk_vpu *vpu = (struct mtk_vpu *)priv;
+ struct vpu_run *run = (struct vpu_run *)data;
+
+ vpu->run.signaled = run->signaled;
+ strncpy(vpu->run.fw_ver, run->fw_ver, VPU_FW_VER_LEN);
+ wake_up_interruptible(&vpu->run.wq);
+}
+
+static int vpu_debug_open(struct inode *inode, struct file *file)
+{
+ file->private_data = inode->i_private;
+ return 0;
+}
+
+static ssize_t vpu_debug_read(struct file *file, char __user *user_buf,
+ size_t count, loff_t *ppos)
+{
+ char buf[256];
+ unsigned int len;
+ unsigned int running, pc, vpu_to_host, host_to_vpu, wdt;
+ int ret;
+ struct device *dev = file->private_data;
+ struct platform_device *pdev = to_platform_device(dev);
+ struct mtk_vpu *vpu = dev_get_drvdata(dev);
+
+ ret = vpu_enable_clock(pdev);
+ if (ret) {
+ dev_err(vpu->dev, "[VPU] enable clock failed %d\n", ret);
+ return 0;
+ }
+
+ /* vpu register status */
+ running = vpu_running(vpu);
+ pc = vpu_cfg_readl(vpu, VPU_PC_REG);
+ wdt = vpu_cfg_readl(vpu, VPU_WDT_REG);
+ host_to_vpu = vpu_cfg_readl(vpu, HOST_TO_VPU);
+ vpu_to_host = vpu_cfg_readl(vpu, VPU_TO_HOST);
+ vpu_disable_clock(pdev);
+
+ if (running) {
+ len = sprintf(buf, "VPU is running\n\n"
+ "FW Version: %s\n"
+ "PC: 0x%x\n"
+ "WDT: 0x%x\n"
+ "Host to VPU: 0x%x\n"
+ "VPU to Host: 0x%x\n",
+ vpu->run.fw_ver, pc, wdt,
+ host_to_vpu, vpu_to_host);
+ } else {
+ len = sprintf(buf, "VPU not running\n");
+ }
+
+ return simple_read_from_buffer(user_buf, count, ppos, buf, len);
+}
+
+static const struct file_operations vpu_debug_fops = {
+ .open = vpu_debug_open,
+ .read = vpu_debug_read,
+};
+
+static void vpu_free_p_ext_mem(struct mtk_vpu *vpu)
+{
+ struct device *dev = vpu->dev;
+ struct iommu_domain *domain = iommu_get_domain_for_dev(dev);
+
+ dma_free_coherent(dev, VPU_EXT_P_SIZE, VPU_PMEM0_VIRT(vpu),
+ VPU_PMEM0_IOVA(vpu));
+
+ if (domain)
+ iommu_detach_device(domain, vpu->dev);
+}
+
+static void vpu_free_d_ext_mem(struct mtk_vpu *vpu)
+{
+ struct device *dev = vpu->dev;
+ struct iommu_domain *domain = iommu_get_domain_for_dev(dev);
+
+ dma_free_coherent(dev, VPU_EXT_D_SIZE, VPU_DMEM0_VIRT(vpu),
+ VPU_DMEM0_IOVA(vpu));
+
+ if (domain)
+ iommu_detach_device(domain, dev);
+}
+
+static int vpu_alloc_p_ext_mem(struct mtk_vpu *vpu)
+{
+ struct device *dev = vpu->dev;
+ struct iommu_domain *domain = iommu_get_domain_for_dev(dev);
+ phys_addr_t p_pa;
+
+ VPU_PMEM0_VIRT(vpu) = dma_alloc_coherent(dev,
+ VPU_EXT_P_SIZE,
+ &(VPU_PMEM0_IOVA(vpu)),
+ GFP_KERNEL);
+ if (VPU_PMEM0_VIRT(vpu) == NULL) {
+ dev_err(dev, "Failed to allocate the extended program memory\n");
+ return PTR_ERR(VPU_PMEM0_VIRT(vpu));
+ }
+
+ p_pa = iommu_iova_to_phys(domain, vpu->mem.p_iova);
+ /* Disable extend0. Enable extend1 */
+ vpu_cfg_writel(vpu, 0x1, VPU_PMEM_EXT0_ADDR);
+ vpu_cfg_writel(vpu, (p_pa & 0xFFFFF000), VPU_PMEM_EXT1_ADDR);
+
+ dev_info(dev, "Program extend memory phy=0x%llx virt=0x%p iova=0x%llx\n",
+ (unsigned long long)p_pa,
+ VPU_PMEM0_VIRT(vpu),
+ (unsigned long long)VPU_PMEM0_IOVA(vpu));
+
+ return 0;
+}
+
+static int vpu_alloc_d_ext_mem(struct mtk_vpu *vpu)
+{
+ struct device *dev = vpu->dev;
+ struct iommu_domain *domain = iommu_get_domain_for_dev(dev);
+ phys_addr_t d_pa;
+
+ VPU_DMEM0_VIRT(vpu) = dma_alloc_coherent(dev,
+ VPU_EXT_D_SIZE,
+ &(VPU_DMEM0_IOVA(vpu)),
+ GFP_KERNEL);
+ if (VPU_DMEM0_VIRT(vpu) == NULL) {
+ dev_err(dev, "Failed to allocate the extended data memory\n");
+ return PTR_ERR(VPU_DMEM0_VIRT(vpu));
+ }
+
+ d_pa = iommu_iova_to_phys(domain, vpu->mem.d_iova);
+
+ /* Disable extend0. Enable extend1 */
+ vpu_cfg_writel(vpu, 0x1, VPU_DMEM_EXT0_ADDR);
+ vpu_cfg_writel(vpu, (d_pa & 0xFFFFF000),
+ VPU_DMEM_EXT1_ADDR);
+
+ dev_info(dev, "Data extend memory phy=0x%llx virt=0x%p iova=0x%llx\n",
+ (unsigned long long)d_pa,
+ VPU_DMEM0_VIRT(vpu),
+ (unsigned long long)VPU_DMEM0_IOVA(vpu));
+
+ return 0;
+}
+
+static void vpu_ipi_handler(struct mtk_vpu *vpu)
+{
+ struct share_obj *rcv_obj = vpu->recv_buf;
+ struct vpu_ipi_desc *ipi_desc = vpu->ipi_desc;
+
+ if (rcv_obj->id < IPI_MAX && ipi_desc[rcv_obj->id].handler) {
+ ipi_desc[rcv_obj->id].handler(rcv_obj->share_buf,
+ rcv_obj->len,
+ ipi_desc[rcv_obj->id].priv);
+ } else {
+ dev_err(vpu->dev, "No such ipi id = %d\n", rcv_obj->id);
+ }
+}
+
+static int vpu_ipi_init(struct mtk_vpu *vpu)
+{
+ /* Disable VPU to host interrupt */
+ vpu_cfg_writel(vpu, 0x0, VPU_TO_HOST);
+
+ /* shared buffer initialization */
+ vpu->recv_buf = (struct share_obj *)VPU_DTCM(vpu);
+ vpu->send_buf = vpu->recv_buf + 1;
+ memset(vpu->recv_buf, 0, sizeof(struct share_obj));
+ memset(vpu->send_buf, 0, sizeof(struct share_obj));
+ mutex_init(&vpu->vpu_mutex);
+
+ return 0;
+}
+
+static irqreturn_t vpu_irq_handler(int irq, void *priv)
+{
+ struct mtk_vpu *vpu = priv;
+ uint32_t vpu_to_host = vpu_cfg_readl(vpu, VPU_TO_HOST);
+
+ if (vpu_to_host & VPU_IPC_INT)
+ vpu_ipi_handler(vpu);
+ else
+ dev_err(vpu->dev, "vpu watchdog timeout!\n");
+
+ /* VPU won't send another interrupt until we set VPU_TO_HOST to 0. */
+ vpu_cfg_writel(vpu, 0x0, VPU_TO_HOST);
+
+ return IRQ_HANDLED;
+}
+
+static struct dentry *vpu_debugfs;
+static int mtk_vpu_probe(struct platform_device *pdev)
+{
+ struct mtk_vpu *vpu;
+ struct device *dev;
+ struct resource *res;
+ int ret = 0;
+
+ dev_dbg(&pdev->dev, "initialization\n");
+
+ dev = &pdev->dev;
+ vpu = devm_kzalloc(dev, sizeof(*vpu), GFP_KERNEL);
+ if (!vpu)
+ return -ENOMEM;
+
+ vpu->dev = &pdev->dev;
+ res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "sram");
+ vpu->reg.sram = devm_ioremap_resource(dev, res);
+ if (IS_ERR(vpu->reg.sram)) {
+ dev_err(dev, "devm_ioremap_resource vpu sram failed.\n");
+ return PTR_ERR(vpu->reg.sram);
+ }
+
+ res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "cfg_reg");
+ vpu->reg.cfg = devm_ioremap_resource(dev, res);
+ if (IS_ERR(vpu->reg.cfg)) {
+ dev_err(dev, "devm_ioremap_resource vpu cfg failed.\n");
+ return PTR_ERR(vpu->reg.cfg);
+ }
+
+ /* Get VPU clock */
+ vpu->clk = devm_clk_get(dev, "main");
+ if (vpu->clk == NULL) {
+ dev_err(dev, "get vpu clock fail\n");
+ return -EINVAL;
+ }
+
+ platform_set_drvdata(pdev, vpu);
+
+ ret = vpu_enable_clock(pdev);
+ if (ret) {
+ ret = -EINVAL;
+ return ret;
+ }
+
+ dev_dbg(dev, "vpu ipi init\n");
+ ret = vpu_ipi_init(vpu);
+ if (ret) {
+ dev_err(dev, "Failed to init ipi\n");
+ goto disable_vpu_clk;
+ }
+
+ platform_set_drvdata(pdev, vpu);
+
+ /* register vpu initialization IPI */
+ ret = vpu_ipi_register(pdev, IPI_VPU_INIT, vpu_init_ipi_handler,
+ "vpu_init", vpu);
+ if (ret) {
+ dev_err(dev, "Failed to register IPI_VPU_INIT\n");
+ goto vpu_mutex_destroy;
+ }
+
+ vpu_debugfs = debugfs_create_file("mtk_vpu", S_IRUGO, NULL, (void *)dev,
+ &vpu_debug_fops);
+ if (!vpu_debugfs) {
+ ret = -ENOMEM;
+ goto cleanup_ipi;
+ }
+
+ /* Set PTCM to 96K and DTCM to 32K */
+ vpu_cfg_writel(vpu, 0x2, VPU_TCM_CFG);
+
+ ret = vpu_alloc_p_ext_mem(vpu);
+ if (ret) {
+ dev_err(dev, "Allocate PM failed\n");
+ goto remove_debugfs;
+ }
+
+ ret = vpu_alloc_d_ext_mem(vpu);
+ if (ret) {
+ dev_err(dev, "Allocate DM failed\n");
+ goto free_p_mem;
+ }
+
+ init_waitqueue_head(&vpu->run.wq);
+
+ res = platform_get_resource(pdev, IORESOURCE_IRQ, 0);
+ if (res == NULL) {
+ dev_err(dev, "get IRQ resource failed.\n");
+ ret = -ENXIO;
+ goto free_d_mem;
+ }
+ vpu->reg.irq = platform_get_irq(pdev, 0);
+ ret = devm_request_irq(dev, vpu->reg.irq, vpu_irq_handler, 0,
+ pdev->name, vpu);
+ if (ret) {
+ dev_err(dev, "failed to request irq\n");
+ goto free_d_mem;
+ }
+
+ vpu_disable_clock(pdev);
+ dev_dbg(dev, "initialization completed\n");
+
+ return 0;
+
+free_d_mem:
+ vpu_free_d_ext_mem(vpu);
+free_p_mem:
+ vpu_free_p_ext_mem(vpu);
+remove_debugfs:
+ debugfs_remove(vpu_debugfs);
+cleanup_ipi:
+ memset(vpu->ipi_desc, 0, sizeof(struct vpu_ipi_desc)*IPI_MAX);
+vpu_mutex_destroy:
+ mutex_destroy(&vpu->vpu_mutex);
+disable_vpu_clk:
+ vpu_disable_clock(pdev);
+
+ return ret;
+}
+
+static const struct of_device_id mtk_vpu_match[] = {
+ {
+ .compatible = "mediatek,mt8173-vpu",
+ },
+ {},
+};
+MODULE_DEVICE_TABLE(of, mtk_vpu_match);
+
+static int mtk_vpu_remove(struct platform_device *pdev)
+{
+ struct mtk_vpu *vpu = platform_get_drvdata(pdev);
+
+ vpu_free_p_ext_mem(vpu);
+ vpu_free_d_ext_mem(vpu);
+
+ return 0;
+}
+
+static struct platform_driver mtk_vpu_driver = {
+ .probe = mtk_vpu_probe,
+ .remove = mtk_vpu_remove,
+ .driver = {
+ .name = MTK_VPU_DRV_NAME,
+ .owner = THIS_MODULE,
+ .of_match_table = mtk_vpu_match,
+ },
+};
+
+module_platform_driver(mtk_vpu_driver);
+
+MODULE_LICENSE("GPL v2");
+MODULE_DESCRIPTION("Mediatek Video Prosessor Unit driver");
diff --git a/drivers/media/platform/mtk-vpu/mtk_vpu_core.h b/drivers/media/platform/mtk-vpu/mtk_vpu_core.h
new file mode 100644
index 0000000..20cf2a0
--- /dev/null
+++ b/drivers/media/platform/mtk-vpu/mtk_vpu_core.h
@@ -0,0 +1,161 @@
+/*
+* Copyright (c) 2015 MediaTek Inc.
+* Author: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
+*
+* This program is free software; you can redistribute it and/or modify
+* it under the terms of the GNU General Public License version 2 as
+* published by the Free Software Foundation.
+*
+* This program is distributed in the hope that it will be useful,
+* but WITHOUT ANY WARRANTY; without even the implied warranty of
+* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+* GNU General Public License for more details.
+*/
+
+#ifndef _MTK_VPU_CORE_H
+#define _MTK_VPU_CORE_H
+
+#include <linux/platform_device.h>
+
+/**
+ * VPU (video processor unit) is a tiny processor controlling video hardware
+ * related to video codec, scaling and color format converting.
+ * VPU interfaces with other blocks by share memory and interrupt.
+ **/
+
+typedef void (*ipi_handler_t) (void *data,
+ unsigned int len,
+ void *priv);
+
+/**
+ * enum ipi_id - the id of inter-processor interrupt
+ *
+ * @IPI_VPU_INIT: The interrupt from vpu is to notfiy kernel
+ VPU initialization completed.
+ * @IPI_VENC_H264: The interrupt from vpu is to notify kernel to
+ handle H264 video encoder job, and vice versa.
+ * @IPI_VENC_VP8: The interrupt fro vpu is to notify kernel to
+ handle VP8 video encoder job,, and vice versa.
+ * @IPI_VENC_CAPABILITY: The interrupt from vpu is to
+ get venc hardware capability.
+ * @IPI_MAX: The maximum IPI number
+ */
+enum ipi_id {
+ IPI_VPU_INIT = 0,
+ IPI_VENC_H264,
+ IPI_VENC_VP8,
+ IPI_VENC_CAPABILITY,
+ IPI_MAX,
+};
+
+/**
+ * vpu_disable_clock - Disable VPU clock
+ *
+ * @pdev: VPU platform device
+ *
+ *
+ * Return: Return 0 if the clock is disabled successfully,
+ * otherwise it is failed.
+ *
+ **/
+void vpu_disable_clock(struct platform_device *pdev);
+
+/**
+ * vpu_enable_clock - Enable VPU clock
+ *
+ * @pdev: VPU platform device
+ *
+ * Return: Return 0 if the clock is enabled successfully,
+ * otherwise it is failed.
+ *
+ **/
+int vpu_enable_clock(struct platform_device *pdev);
+
+/**
+ * vpu_ipi_register - register an ipi function
+ *
+ * @pdev: VPU platform device
+ * @id: IPI ID
+ * @handler: IPI handler
+ * @name: IPI name
+ * @priv: private data for IPI handler
+ *
+ * Register an ipi function to receive ipi interrupt from VPU.
+ *
+ * Return: Return 0 if ipi registers successfully, otherwise it is failed.
+ */
+int vpu_ipi_register(struct platform_device *pdev, enum ipi_id id,
+ ipi_handler_t handler, const char *name, void *priv);
+
+/**
+ * vpu_ipi_send - send data from AP to vpu.
+ *
+ * @pdev: VPU platform device
+ * @id: IPI ID
+ * @buf: the data buffer
+ * @len: the data buffer length
+ * @wait: wait for the last ipi completed.
+ *
+ * This function is thread-safe. When this function returns,
+ * VPU has received the data and starts the processing.
+ * When the processing completes, IPI handler registered
+ * by vpu_ipi_register will be called in interrupt context.
+ *
+ * Return: Return 0 if sending data successfully, otherwise it is failed.
+ **/
+int vpu_ipi_send(struct platform_device *pdev,
+ enum ipi_id id, void *buf,
+ unsigned int len,
+ unsigned int wait);
+
+/**
+ * vpu_get_plat_device - get VPU's platform device
+ *
+ * @pdev: the platform device of the module requesting VPU platform
+ * device for using VPU API.
+ *
+ * Return: Return NULL if it is failed.
+ * otherwise it is VPU's platform device
+ **/
+struct platform_device *vpu_get_plat_device(struct platform_device *pdev);
+
+/**
+ * vpu_load_firmware - download VPU firmware and boot it
+ *
+ * @pdev: VPU platform device
+ *
+ * Return: Return 0 if downloading firmware successfully,
+ * otherwise it is failed
+ **/
+int vpu_load_firmware(struct platform_device *pdev);
+
+/**
+ * vpu_mapping_dm_addr - Mapping DTCM/DMEM to kernel virtual address
+ *
+ * @pdev: VPU platform device
+ * @dmem_addr: VPU's data memory address
+ *
+ * Mapping the VPU's DTCM (Data Tightly-Coupled Memory) /
+ * DMEM (Data Extended Memory) memory address to
+ * kernel virtual address.
+ *
+ * Return: Return ERR_PTR(-EINVAL) if mapping failed,
+ * otherwise the mapped kernel virtual address
+ **/
+void *vpu_mapping_dm_addr(struct platform_device *pdev,
+ void *dtcm_dmem_addr);
+
+/**
+ * vpu_mapping_iommu_dm_addr - Mapping to iommu address
+ *
+ * @pdev: VPU platform device
+ * @dmem_addr: VPU's extended data memory address
+ *
+ * Mapping the VPU's extended data address to iommu address
+ *
+ * Return: Return ERR_PTR(-EINVAL) if mapping failed,
+ * otherwise the mapped iommu address
+ **/
+dma_addr_t *vpu_mapping_iommu_dm_addr(struct platform_device *pdev,
+ void *dmem_addr);
+#endif /* _MTK_VPU_CORE_H */
diff --git a/drivers/media/platform/mtk-vpu/vp8_enc/venc_vp8_vpu.h b/drivers/media/platform/mtk-vpu/vp8_enc/venc_vp8_vpu.h
new file mode 100644
index 0000000..4e09eec
--- /dev/null
+++ b/drivers/media/platform/mtk-vpu/vp8_enc/venc_vp8_vpu.h
@@ -0,0 +1,119 @@
+/*
+ * Copyright (c) 2015 MediaTek Inc.
+ * Author: Daniel Hsiao <daniel.hsiao@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef VENC_VP8_VPU_H_
+#define VENC_VP8_VPU_H_
+
+/**
+ * enum venc_vp8_vpu_work_buf - vp8 encoder buffer index
+ */
+enum venc_vp8_vpu_work_buf {
+ VENC_VP8_VPU_WORK_BUF_LUMA,
+ VENC_VP8_VPU_WORK_BUF_LUMA2,
+ VENC_VP8_VPU_WORK_BUF_LUMA3,
+ VENC_VP8_VPU_WORK_BUF_CHROMA,
+ VENC_VP8_VPU_WORK_BUF_CHROMA2,
+ VENC_VP8_VPU_WORK_BUF_CHROMA3,
+ VENC_VP8_VPU_WORK_BUF_MV_INFO,
+ VENC_VP8_VPU_WORK_BUF_BS_HD,
+ VENC_VP8_VPU_WORK_BUF_PROB_BUF,
+ VENC_VP8_VPU_WORK_BUF_RC_INFO,
+ VENC_VP8_VPU_WORK_BUF_RC_CODE,
+ VENC_VP8_VPU_WORK_BUF_RC_CODE2,
+ VENC_VP8_VPU_WORK_BUF_RC_CODE3,
+ VENC_VP8_VPU_WORK_BUF_MAX,
+};
+
+/*
+ * struct venc_vp8_vpu_config - Structure for vp8 encoder configuration
+ * @input_fourcc: input fourcc
+ * @bitrate: target bitrate (in bps)
+ * @pic_w: picture width
+ * @pic_h: picture height
+ * @buf_w: buffer width (with 16 alignment)
+ * @buf_h: buffer height (with 16 alignment)
+ * @intra_period: intra frame period
+ * @framerate: frame rate
+ * @ts_mode: temporal scalability mode (0: disable, 1: enable)
+ * support three temporal layers - 0: 7.5fps 1: 7.5fps 2: 15fps.
+ */
+struct venc_vp8_vpu_config {
+ u32 input_fourcc;
+ u32 bitrate;
+ u32 pic_w;
+ u32 pic_h;
+ u32 buf_w;
+ u32 buf_h;
+ u32 intra_period;
+ u32 framerate;
+ u32 ts_mode;
+};
+
+/*
+ * struct venc_vp8_vpu_buf -Structure for buffer information
+ * @align: buffer alignment (in bytes)
+ * @pa: physical address
+ * @vpua: VPU side memory addr which is used by RC_CODE
+ * @size: buffer size (in bytes)
+ */
+struct venc_vp8_vpu_buf {
+ u32 align;
+ u32 pa;
+ u32 vpua;
+ u32 size;
+};
+
+/*
+ * struct venc_vp8_vpu_drv - Structure for VPU driver control and info share
+ * This structure is allocated in VPU side and shared to AP side.
+ * @config: vp8 encoder configuration
+ * @work_bufs: working buffer information in VPU side
+ * The work_bufs here is for storing the 'size' info shared to AP side.
+ * The similar item in struct venc_vp8_handle is for memory allocation
+ * in AP side. The AP driver will copy the 'size' from here to the one in
+ * struct mtk_vcodec_mem, then invoke mtk_vcodec_mem_alloc to allocate
+ * the buffer. After that, bypass the 'dma_addr' to the 'pa' field here for
+ * register setting in VPU side.
+ */
+struct venc_vp8_vpu_drv {
+ struct venc_vp8_vpu_config config;
+ struct venc_vp8_vpu_buf work_bufs[VENC_VP8_VPU_WORK_BUF_MAX];
+};
+
+/*
+ * struct venc_vp8_vpu_inst - vp8 encoder VPU driver instance
+ * @wq_hd: wait queue used for vpu cmd trigger then wait vpu interrupt done
+ * @signaled: flag used for checking vpu interrupt done
+ * @failure: flag to show vpu cmd succeeds or not
+ * @id: VPU instance id
+ * @drv: driver structure allocated by VPU side and shared to AP side for
+ * control and info share
+ */
+struct venc_vp8_vpu_inst {
+ wait_queue_head_t wq_hd;
+ int signaled;
+ int failure;
+ unsigned int id;
+ struct venc_vp8_vpu_drv *drv;
+};
+
+int vp8_enc_vpu_init(void *handle);
+int vp8_enc_vpu_set_param(void *handle, unsigned int id, void *param);
+int vp8_enc_vpu_encode(void *handle,
+ struct venc_frm_buf *frm_buf,
+ struct mtk_vcodec_mem *bs_buf);
+int vp8_enc_vpu_deinit(void *handle);
+
+#endif
--
1.7.9.5
^ permalink raw reply related [flat|nested] 84+ messages in thread
* [RESEND RFC/PATCH 4/8] dt-bindings: Add a binding for Mediatek Video Encoder
@ 2015-11-17 12:54 ` Tiffany Lin
0 siblings, 0 replies; 84+ messages in thread
From: Tiffany Lin @ 2015-11-17 12:54 UTC (permalink / raw)
To: Rob Herring, Pawel Moll, Mark Rutland, Ian Campbell, Kumar Gala,
Catalin Marinas, Will Deacon, Mauro Carvalho Chehab,
Matthias Brugger, Daniel Kurtz, Sascha Hauer, Hongzhou Yang,
Hans Verkuil, Laurent Pinchart, Sakari Ailus, Geert Uytterhoeven,
Mikhail Ulyanov, Fabien Dessenne, Arnd Bergmann,
Darren Etheridge, Peter Griffin, Benoit Parrot
Cc: Tiffany Lin, Andrew-CT Chen, Eddie Huang, Yingjoe Chen,
James Liao, Daniel Hsiao, devicetree, linux-kernel,
linux-arm-kernel, linux-media, linux-mediatek
add a DT binding documentation of Video Encoder for the
MT8173 SoC from Mediatek.
Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
---
.../devicetree/bindings/media/mediatek-vcodec.txt | 58 ++++++++++++++++++++
1 file changed, 58 insertions(+)
create mode 100644 Documentation/devicetree/bindings/media/mediatek-vcodec.txt
diff --git a/Documentation/devicetree/bindings/media/mediatek-vcodec.txt b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
new file mode 100644
index 0000000..fea4d7c
--- /dev/null
+++ b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
@@ -0,0 +1,58 @@
+Mediatek Video Codec
+
+Mediatek Video Codec is the video codec hw present in Mediatek SoCs which
+supports high resolution encoding functionalities.
+
+Required properties:
+- compatible : "mediatek,mt8173-vcodec-enc" for encoder
+- reg : Physical base address of the video codec registers and length of
+ memory mapped region.
+- interrupts : interrupt number to the cpu.
+- larb : must contain the larbes of current platform
+- clocks : list of clock specifiers, corresponding to entries in
+ the clock-names property;
+- clock-names: must contain "vencpll", "venc_lt_sel", "vcodecpll_370p5_ck"
+- iommus : list of iommus specifiers should be enabled for hw encode.
+ There are 2 cells needed to enable/disable iommu.
+ The first one is local arbiter index(larbid), and the other is port
+ index(portid) within local arbiter. Specifies the larbid and portid
+ as defined in dt-binding/memory/mt8173-larb-port.h.
+- vpu : the node of video processor unit
+
+Example:
+vcodec_enc: vcodec@0x18002000 {
+ compatible = "mediatek,mt8173-vcodec-enc";
+ reg = <0 0x18002000 0 0x1000>, /*VENC_SYS*/
+ <0 0x19002000 0 0x1000>; /*VENC_LT_SYS*/
+ interrupts = <GIC_SPI 198 IRQ_TYPE_LEVEL_LOW>,
+ <GIC_SPI 202 IRQ_TYPE_LEVEL_LOW>;
+ larb = <&larb3>,
+ <&larb5>;
+ iommus = <&iommu M4U_LARB3_ID M4U_PORT_VENC_RCPU>,
+ <&iommu M4U_LARB3_ID M4U_PORT_VENC_REC>,
+ <&iommu M4U_LARB3_ID M4U_PORT_VENC_BSDMA>,
+ <&iommu M4U_LARB3_ID M4U_PORT_VENC_SV_COMV>,
+ <&iommu M4U_LARB3_ID M4U_PORT_VENC_RD_COMV>,
+ <&iommu M4U_LARB3_ID M4U_PORT_VENC_CUR_LUMA>,
+ <&iommu M4U_LARB3_ID M4U_PORT_VENC_CUR_CHROMA>,
+ <&iommu M4U_LARB3_ID M4U_PORT_VENC_REF_LUMA>,
+ <&iommu M4U_LARB3_ID M4U_PORT_VENC_REF_CHROMA>,
+ <&iommu M4U_LARB3_ID M4U_PORT_VENC_NBM_RDMA>,
+ <&iommu M4U_LARB3_ID M4U_PORT_VENC_NBM_WDMA>,
+ <&iommu M4U_LARB5_ID M4U_PORT_VENC_RCPU_SET2>,
+ <&iommu M4U_LARB5_ID M4U_PORT_VENC_REC_FRM_SET2>,
+ <&iommu M4U_LARB5_ID M4U_PORT_VENC_BSDMA_SET2>,
+ <&iommu M4U_LARB5_ID M4U_PORT_VENC_SV_COMA_SET2>,
+ <&iommu M4U_LARB5_ID M4U_PORT_VENC_RD_COMA_SET2>,
+ <&iommu M4U_LARB5_ID M4U_PORT_VENC_CUR_LUMA_SET2>,
+ <&iommu M4U_LARB5_ID M4U_PORT_VENC_CUR_CHROMA_SET2>,
+ <&iommu M4U_LARB5_ID M4U_PORT_VENC_REF_LUMA_SET2>,
+ <&iommu M4U_LARB5_ID M4U_PORT_VENC_REC_CHROMA_SET2>;
+ vpu = <&vpu>;
+ clocks = <&apmixedsys CLK_APMIXED_VENCPLL>,
+ <&topckgen CLK_TOP_VENC_LT_SEL>,
+ <&topckgen CLK_TOP_VCODECPLL_370P5>;
+ clock-names = "vencpll",
+ "venc_lt_sel",
+ "vcodecpll_370p5_ck";
+ };
--
1.7.9.5
^ permalink raw reply related [flat|nested] 84+ messages in thread
* [RESEND RFC/PATCH 4/8] dt-bindings: Add a binding for Mediatek Video Encoder
@ 2015-11-17 12:54 ` Tiffany Lin
0 siblings, 0 replies; 84+ messages in thread
From: Tiffany Lin @ 2015-11-17 12:54 UTC (permalink / raw)
To: Rob Herring, Pawel Moll, Mark Rutland, Ian Campbell, Kumar Gala,
Catalin Marinas, Will Deacon, Mauro Carvalho Chehab,
Matthias Brugger, Daniel Kurtz, Sascha Hauer, Hongzhou Yang,
Hans Verkuil, Laurent Pinchart, Sakari Ailus, Geert Uytterhoeven,
Mikhail Ulyanov, Fabien Dessenne, Arnd Bergmann,
Darren Etheridge, Peter Griffin, Benoit Parrot
Cc: James Liao, Eddie Huang, Daniel Hsiao, Andrew-CT Chen,
linux-kernel-u79uwXL29TY76Z2rM5mHXA,
devicetree-u79uwXL29TY76Z2rM5mHXA,
linux-mediatek-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r, Yingjoe Chen,
Tiffany Lin, linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
linux-media-u79uwXL29TY76Z2rM5mHXA
add a DT binding documentation of Video Encoder for the
MT8173 SoC from Mediatek.
Signed-off-by: Tiffany Lin <tiffany.lin-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
---
.../devicetree/bindings/media/mediatek-vcodec.txt | 58 ++++++++++++++++++++
1 file changed, 58 insertions(+)
create mode 100644 Documentation/devicetree/bindings/media/mediatek-vcodec.txt
diff --git a/Documentation/devicetree/bindings/media/mediatek-vcodec.txt b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
new file mode 100644
index 0000000..fea4d7c
--- /dev/null
+++ b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
@@ -0,0 +1,58 @@
+Mediatek Video Codec
+
+Mediatek Video Codec is the video codec hw present in Mediatek SoCs which
+supports high resolution encoding functionalities.
+
+Required properties:
+- compatible : "mediatek,mt8173-vcodec-enc" for encoder
+- reg : Physical base address of the video codec registers and length of
+ memory mapped region.
+- interrupts : interrupt number to the cpu.
+- larb : must contain the larbes of current platform
+- clocks : list of clock specifiers, corresponding to entries in
+ the clock-names property;
+- clock-names: must contain "vencpll", "venc_lt_sel", "vcodecpll_370p5_ck"
+- iommus : list of iommus specifiers should be enabled for hw encode.
+ There are 2 cells needed to enable/disable iommu.
+ The first one is local arbiter index(larbid), and the other is port
+ index(portid) within local arbiter. Specifies the larbid and portid
+ as defined in dt-binding/memory/mt8173-larb-port.h.
+- vpu : the node of video processor unit
+
+Example:
+vcodec_enc: vcodec@0x18002000 {
+ compatible = "mediatek,mt8173-vcodec-enc";
+ reg = <0 0x18002000 0 0x1000>, /*VENC_SYS*/
+ <0 0x19002000 0 0x1000>; /*VENC_LT_SYS*/
+ interrupts = <GIC_SPI 198 IRQ_TYPE_LEVEL_LOW>,
+ <GIC_SPI 202 IRQ_TYPE_LEVEL_LOW>;
+ larb = <&larb3>,
+ <&larb5>;
+ iommus = <&iommu M4U_LARB3_ID M4U_PORT_VENC_RCPU>,
+ <&iommu M4U_LARB3_ID M4U_PORT_VENC_REC>,
+ <&iommu M4U_LARB3_ID M4U_PORT_VENC_BSDMA>,
+ <&iommu M4U_LARB3_ID M4U_PORT_VENC_SV_COMV>,
+ <&iommu M4U_LARB3_ID M4U_PORT_VENC_RD_COMV>,
+ <&iommu M4U_LARB3_ID M4U_PORT_VENC_CUR_LUMA>,
+ <&iommu M4U_LARB3_ID M4U_PORT_VENC_CUR_CHROMA>,
+ <&iommu M4U_LARB3_ID M4U_PORT_VENC_REF_LUMA>,
+ <&iommu M4U_LARB3_ID M4U_PORT_VENC_REF_CHROMA>,
+ <&iommu M4U_LARB3_ID M4U_PORT_VENC_NBM_RDMA>,
+ <&iommu M4U_LARB3_ID M4U_PORT_VENC_NBM_WDMA>,
+ <&iommu M4U_LARB5_ID M4U_PORT_VENC_RCPU_SET2>,
+ <&iommu M4U_LARB5_ID M4U_PORT_VENC_REC_FRM_SET2>,
+ <&iommu M4U_LARB5_ID M4U_PORT_VENC_BSDMA_SET2>,
+ <&iommu M4U_LARB5_ID M4U_PORT_VENC_SV_COMA_SET2>,
+ <&iommu M4U_LARB5_ID M4U_PORT_VENC_RD_COMA_SET2>,
+ <&iommu M4U_LARB5_ID M4U_PORT_VENC_CUR_LUMA_SET2>,
+ <&iommu M4U_LARB5_ID M4U_PORT_VENC_CUR_CHROMA_SET2>,
+ <&iommu M4U_LARB5_ID M4U_PORT_VENC_REF_LUMA_SET2>,
+ <&iommu M4U_LARB5_ID M4U_PORT_VENC_REC_CHROMA_SET2>;
+ vpu = <&vpu>;
+ clocks = <&apmixedsys CLK_APMIXED_VENCPLL>,
+ <&topckgen CLK_TOP_VENC_LT_SEL>,
+ <&topckgen CLK_TOP_VCODECPLL_370P5>;
+ clock-names = "vencpll",
+ "venc_lt_sel",
+ "vcodecpll_370p5_ck";
+ };
--
1.7.9.5
^ permalink raw reply related [flat|nested] 84+ messages in thread
* [RESEND RFC/PATCH 4/8] dt-bindings: Add a binding for Mediatek Video Encoder
@ 2015-11-17 12:54 ` Tiffany Lin
0 siblings, 0 replies; 84+ messages in thread
From: Tiffany Lin @ 2015-11-17 12:54 UTC (permalink / raw)
To: linux-arm-kernel
add a DT binding documentation of Video Encoder for the
MT8173 SoC from Mediatek.
Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
---
.../devicetree/bindings/media/mediatek-vcodec.txt | 58 ++++++++++++++++++++
1 file changed, 58 insertions(+)
create mode 100644 Documentation/devicetree/bindings/media/mediatek-vcodec.txt
diff --git a/Documentation/devicetree/bindings/media/mediatek-vcodec.txt b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
new file mode 100644
index 0000000..fea4d7c
--- /dev/null
+++ b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
@@ -0,0 +1,58 @@
+Mediatek Video Codec
+
+Mediatek Video Codec is the video codec hw present in Mediatek SoCs which
+supports high resolution encoding functionalities.
+
+Required properties:
+- compatible : "mediatek,mt8173-vcodec-enc" for encoder
+- reg : Physical base address of the video codec registers and length of
+ memory mapped region.
+- interrupts : interrupt number to the cpu.
+- larb : must contain the larbes of current platform
+- clocks : list of clock specifiers, corresponding to entries in
+ the clock-names property;
+- clock-names: must contain "vencpll", "venc_lt_sel", "vcodecpll_370p5_ck"
+- iommus : list of iommus specifiers should be enabled for hw encode.
+ There are 2 cells needed to enable/disable iommu.
+ The first one is local arbiter index(larbid), and the other is port
+ index(portid) within local arbiter. Specifies the larbid and portid
+ as defined in dt-binding/memory/mt8173-larb-port.h.
+- vpu : the node of video processor unit
+
+Example:
+vcodec_enc: vcodec at 0x18002000 {
+ compatible = "mediatek,mt8173-vcodec-enc";
+ reg = <0 0x18002000 0 0x1000>, /*VENC_SYS*/
+ <0 0x19002000 0 0x1000>; /*VENC_LT_SYS*/
+ interrupts = <GIC_SPI 198 IRQ_TYPE_LEVEL_LOW>,
+ <GIC_SPI 202 IRQ_TYPE_LEVEL_LOW>;
+ larb = <&larb3>,
+ <&larb5>;
+ iommus = <&iommu M4U_LARB3_ID M4U_PORT_VENC_RCPU>,
+ <&iommu M4U_LARB3_ID M4U_PORT_VENC_REC>,
+ <&iommu M4U_LARB3_ID M4U_PORT_VENC_BSDMA>,
+ <&iommu M4U_LARB3_ID M4U_PORT_VENC_SV_COMV>,
+ <&iommu M4U_LARB3_ID M4U_PORT_VENC_RD_COMV>,
+ <&iommu M4U_LARB3_ID M4U_PORT_VENC_CUR_LUMA>,
+ <&iommu M4U_LARB3_ID M4U_PORT_VENC_CUR_CHROMA>,
+ <&iommu M4U_LARB3_ID M4U_PORT_VENC_REF_LUMA>,
+ <&iommu M4U_LARB3_ID M4U_PORT_VENC_REF_CHROMA>,
+ <&iommu M4U_LARB3_ID M4U_PORT_VENC_NBM_RDMA>,
+ <&iommu M4U_LARB3_ID M4U_PORT_VENC_NBM_WDMA>,
+ <&iommu M4U_LARB5_ID M4U_PORT_VENC_RCPU_SET2>,
+ <&iommu M4U_LARB5_ID M4U_PORT_VENC_REC_FRM_SET2>,
+ <&iommu M4U_LARB5_ID M4U_PORT_VENC_BSDMA_SET2>,
+ <&iommu M4U_LARB5_ID M4U_PORT_VENC_SV_COMA_SET2>,
+ <&iommu M4U_LARB5_ID M4U_PORT_VENC_RD_COMA_SET2>,
+ <&iommu M4U_LARB5_ID M4U_PORT_VENC_CUR_LUMA_SET2>,
+ <&iommu M4U_LARB5_ID M4U_PORT_VENC_CUR_CHROMA_SET2>,
+ <&iommu M4U_LARB5_ID M4U_PORT_VENC_REF_LUMA_SET2>,
+ <&iommu M4U_LARB5_ID M4U_PORT_VENC_REC_CHROMA_SET2>;
+ vpu = <&vpu>;
+ clocks = <&apmixedsys CLK_APMIXED_VENCPLL>,
+ <&topckgen CLK_TOP_VENC_LT_SEL>,
+ <&topckgen CLK_TOP_VCODECPLL_370P5>;
+ clock-names = "vencpll",
+ "venc_lt_sel",
+ "vcodecpll_370p5_ck";
+ };
--
1.7.9.5
^ permalink raw reply related [flat|nested] 84+ messages in thread
* [RESEND RFC/PATCH 5/8] arm64: dts: mediatek: Add Video Encoder for MT8173
@ 2015-11-17 12:54 ` Tiffany Lin
0 siblings, 0 replies; 84+ messages in thread
From: Tiffany Lin @ 2015-11-17 12:54 UTC (permalink / raw)
To: Rob Herring, Pawel Moll, Mark Rutland, Ian Campbell, Kumar Gala,
Catalin Marinas, Will Deacon, Mauro Carvalho Chehab,
Matthias Brugger, Daniel Kurtz, Sascha Hauer, Hongzhou Yang,
Hans Verkuil, Laurent Pinchart, Sakari Ailus, Geert Uytterhoeven,
Mikhail Ulyanov, Fabien Dessenne, Arnd Bergmann,
Darren Etheridge, Peter Griffin, Benoit Parrot
Cc: Tiffany Lin, Andrew-CT Chen, Eddie Huang, Yingjoe Chen,
James Liao, Daniel Hsiao, devicetree, linux-kernel,
linux-arm-kernel, linux-media, linux-mediatek
add video encoder driver for MT8173
Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
---
arch/arm64/boot/dts/mediatek/mt8173.dtsi | 47 ++++++++++++++++++++++++++++++
1 file changed, 47 insertions(+)
diff --git a/arch/arm64/boot/dts/mediatek/mt8173.dtsi b/arch/arm64/boot/dts/mediatek/mt8173.dtsi
index 098c15e..85ba167 100644
--- a/arch/arm64/boot/dts/mediatek/mt8173.dtsi
+++ b/arch/arm64/boot/dts/mediatek/mt8173.dtsi
@@ -545,6 +545,53 @@
#clock-cells = <1>;
};
+ larb3: larb@18001000 {
+ compatible = "mediatek,mt8173-smi-larb";
+ reg = <0 0x18001000 0 0x1000>;
+ mediatek,smi = <&smi_common>;
+ power-domains = <&scpsys MT8173_POWER_DOMAIN_VENC>;
+ clocks = <&vencsys CLK_VENC_CKE1>,
+ <&vencsys CLK_VENC_CKE0>;
+ clock-names = "apb", "smi";
+ };
+
+ vcodec_enc: vcodec@18002000 {
+ compatible = "mediatek,mt8173-vcodec-enc";
+ reg = <0 0x18002000 0 0x1000>, /* VENC_SYS */
+ <0 0x19002000 0 0x1000>; /* VENC_LT_SYS */
+ interrupts = <GIC_SPI 198 IRQ_TYPE_LEVEL_LOW>,
+ <GIC_SPI 202 IRQ_TYPE_LEVEL_LOW>;
+ larb = <&larb3>,
+ <&larb5>;
+ iommus = <&iommu M4U_LARB3_ID M4U_PORT_VENC_RCPU>,
+ <&iommu M4U_LARB3_ID M4U_PORT_VENC_REC>,
+ <&iommu M4U_LARB3_ID M4U_PORT_VENC_BSDMA>,
+ <&iommu M4U_LARB3_ID M4U_PORT_VENC_SV_COMV>,
+ <&iommu M4U_LARB3_ID M4U_PORT_VENC_RD_COMV>,
+ <&iommu M4U_LARB3_ID M4U_PORT_VENC_CUR_LUMA>,
+ <&iommu M4U_LARB3_ID M4U_PORT_VENC_CUR_CHROMA>,
+ <&iommu M4U_LARB3_ID M4U_PORT_VENC_REF_LUMA>,
+ <&iommu M4U_LARB3_ID M4U_PORT_VENC_REF_CHROMA>,
+ <&iommu M4U_LARB3_ID M4U_PORT_VENC_NBM_RDMA>,
+ <&iommu M4U_LARB3_ID M4U_PORT_VENC_NBM_WDMA>,
+ <&iommu M4U_LARB5_ID M4U_PORT_VENC_RCPU_SET2>,
+ <&iommu M4U_LARB5_ID M4U_PORT_VENC_REC_FRM_SET2>,
+ <&iommu M4U_LARB5_ID M4U_PORT_VENC_BSDMA_SET2>,
+ <&iommu M4U_LARB5_ID M4U_PORT_VENC_SV_COMA_SET2>,
+ <&iommu M4U_LARB5_ID M4U_PORT_VENC_RD_COMA_SET2>,
+ <&iommu M4U_LARB5_ID M4U_PORT_VENC_CUR_LUMA_SET2>,
+ <&iommu M4U_LARB5_ID M4U_PORT_VENC_CUR_CHROMA_SET2>,
+ <&iommu M4U_LARB5_ID M4U_PORT_VENC_REF_LUMA_SET2>,
+ <&iommu M4U_LARB5_ID M4U_PORT_VENC_REC_CHROMA_SET2>;
+ vpu = <&vpu>;
+ clocks = <&apmixedsys CLK_APMIXED_VENCPLL>,
+ <&topckgen CLK_TOP_VENC_LT_SEL>,
+ <&topckgen CLK_TOP_VCODECPLL_370P5>;
+ clock-names = "vencpll",
+ "venc_lt_sel",
+ "vcodecpll_370p5_ck";
+ };
+
vencltsys: clock-controller@19000000 {
compatible = "mediatek,mt8173-vencltsys", "syscon";
reg = <0 0x19000000 0 0x1000>;
--
1.7.9.5
^ permalink raw reply related [flat|nested] 84+ messages in thread
* [RESEND RFC/PATCH 5/8] arm64: dts: mediatek: Add Video Encoder for MT8173
@ 2015-11-17 12:54 ` Tiffany Lin
0 siblings, 0 replies; 84+ messages in thread
From: Tiffany Lin @ 2015-11-17 12:54 UTC (permalink / raw)
To: Rob Herring, Pawel Moll, Mark Rutland, Ian Campbell, Kumar Gala,
Catalin Marinas, Will Deacon, Mauro Carvalho Chehab,
Matthias Brugger, Daniel Kurtz, Sascha Hauer, Hongzhou Yang,
Hans Verkuil, Laurent Pinchart, Sakari Ailus, Geert Uytterhoeven,
Mikhail Ulyanov, Fabien Dessenne, Arnd Bergmann,
Darren Etheridge, Peter Griffin, Benoit Parrot
Cc: James Liao, Eddie Huang, Daniel Hsiao, Andrew-CT Chen,
linux-kernel-u79uwXL29TY76Z2rM5mHXA,
devicetree-u79uwXL29TY76Z2rM5mHXA,
linux-mediatek-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r, Yingjoe Chen,
Tiffany Lin, linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
linux-media-u79uwXL29TY76Z2rM5mHXA
add video encoder driver for MT8173
Signed-off-by: Tiffany Lin <tiffany.lin-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
---
arch/arm64/boot/dts/mediatek/mt8173.dtsi | 47 ++++++++++++++++++++++++++++++
1 file changed, 47 insertions(+)
diff --git a/arch/arm64/boot/dts/mediatek/mt8173.dtsi b/arch/arm64/boot/dts/mediatek/mt8173.dtsi
index 098c15e..85ba167 100644
--- a/arch/arm64/boot/dts/mediatek/mt8173.dtsi
+++ b/arch/arm64/boot/dts/mediatek/mt8173.dtsi
@@ -545,6 +545,53 @@
#clock-cells = <1>;
};
+ larb3: larb@18001000 {
+ compatible = "mediatek,mt8173-smi-larb";
+ reg = <0 0x18001000 0 0x1000>;
+ mediatek,smi = <&smi_common>;
+ power-domains = <&scpsys MT8173_POWER_DOMAIN_VENC>;
+ clocks = <&vencsys CLK_VENC_CKE1>,
+ <&vencsys CLK_VENC_CKE0>;
+ clock-names = "apb", "smi";
+ };
+
+ vcodec_enc: vcodec@18002000 {
+ compatible = "mediatek,mt8173-vcodec-enc";
+ reg = <0 0x18002000 0 0x1000>, /* VENC_SYS */
+ <0 0x19002000 0 0x1000>; /* VENC_LT_SYS */
+ interrupts = <GIC_SPI 198 IRQ_TYPE_LEVEL_LOW>,
+ <GIC_SPI 202 IRQ_TYPE_LEVEL_LOW>;
+ larb = <&larb3>,
+ <&larb5>;
+ iommus = <&iommu M4U_LARB3_ID M4U_PORT_VENC_RCPU>,
+ <&iommu M4U_LARB3_ID M4U_PORT_VENC_REC>,
+ <&iommu M4U_LARB3_ID M4U_PORT_VENC_BSDMA>,
+ <&iommu M4U_LARB3_ID M4U_PORT_VENC_SV_COMV>,
+ <&iommu M4U_LARB3_ID M4U_PORT_VENC_RD_COMV>,
+ <&iommu M4U_LARB3_ID M4U_PORT_VENC_CUR_LUMA>,
+ <&iommu M4U_LARB3_ID M4U_PORT_VENC_CUR_CHROMA>,
+ <&iommu M4U_LARB3_ID M4U_PORT_VENC_REF_LUMA>,
+ <&iommu M4U_LARB3_ID M4U_PORT_VENC_REF_CHROMA>,
+ <&iommu M4U_LARB3_ID M4U_PORT_VENC_NBM_RDMA>,
+ <&iommu M4U_LARB3_ID M4U_PORT_VENC_NBM_WDMA>,
+ <&iommu M4U_LARB5_ID M4U_PORT_VENC_RCPU_SET2>,
+ <&iommu M4U_LARB5_ID M4U_PORT_VENC_REC_FRM_SET2>,
+ <&iommu M4U_LARB5_ID M4U_PORT_VENC_BSDMA_SET2>,
+ <&iommu M4U_LARB5_ID M4U_PORT_VENC_SV_COMA_SET2>,
+ <&iommu M4U_LARB5_ID M4U_PORT_VENC_RD_COMA_SET2>,
+ <&iommu M4U_LARB5_ID M4U_PORT_VENC_CUR_LUMA_SET2>,
+ <&iommu M4U_LARB5_ID M4U_PORT_VENC_CUR_CHROMA_SET2>,
+ <&iommu M4U_LARB5_ID M4U_PORT_VENC_REF_LUMA_SET2>,
+ <&iommu M4U_LARB5_ID M4U_PORT_VENC_REC_CHROMA_SET2>;
+ vpu = <&vpu>;
+ clocks = <&apmixedsys CLK_APMIXED_VENCPLL>,
+ <&topckgen CLK_TOP_VENC_LT_SEL>,
+ <&topckgen CLK_TOP_VCODECPLL_370P5>;
+ clock-names = "vencpll",
+ "venc_lt_sel",
+ "vcodecpll_370p5_ck";
+ };
+
vencltsys: clock-controller@19000000 {
compatible = "mediatek,mt8173-vencltsys", "syscon";
reg = <0 0x19000000 0 0x1000>;
--
1.7.9.5
^ permalink raw reply related [flat|nested] 84+ messages in thread
* [RESEND RFC/PATCH 5/8] arm64: dts: mediatek: Add Video Encoder for MT8173
@ 2015-11-17 12:54 ` Tiffany Lin
0 siblings, 0 replies; 84+ messages in thread
From: Tiffany Lin @ 2015-11-17 12:54 UTC (permalink / raw)
To: linux-arm-kernel
add video encoder driver for MT8173
Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
---
arch/arm64/boot/dts/mediatek/mt8173.dtsi | 47 ++++++++++++++++++++++++++++++
1 file changed, 47 insertions(+)
diff --git a/arch/arm64/boot/dts/mediatek/mt8173.dtsi b/arch/arm64/boot/dts/mediatek/mt8173.dtsi
index 098c15e..85ba167 100644
--- a/arch/arm64/boot/dts/mediatek/mt8173.dtsi
+++ b/arch/arm64/boot/dts/mediatek/mt8173.dtsi
@@ -545,6 +545,53 @@
#clock-cells = <1>;
};
+ larb3: larb at 18001000 {
+ compatible = "mediatek,mt8173-smi-larb";
+ reg = <0 0x18001000 0 0x1000>;
+ mediatek,smi = <&smi_common>;
+ power-domains = <&scpsys MT8173_POWER_DOMAIN_VENC>;
+ clocks = <&vencsys CLK_VENC_CKE1>,
+ <&vencsys CLK_VENC_CKE0>;
+ clock-names = "apb", "smi";
+ };
+
+ vcodec_enc: vcodec at 18002000 {
+ compatible = "mediatek,mt8173-vcodec-enc";
+ reg = <0 0x18002000 0 0x1000>, /* VENC_SYS */
+ <0 0x19002000 0 0x1000>; /* VENC_LT_SYS */
+ interrupts = <GIC_SPI 198 IRQ_TYPE_LEVEL_LOW>,
+ <GIC_SPI 202 IRQ_TYPE_LEVEL_LOW>;
+ larb = <&larb3>,
+ <&larb5>;
+ iommus = <&iommu M4U_LARB3_ID M4U_PORT_VENC_RCPU>,
+ <&iommu M4U_LARB3_ID M4U_PORT_VENC_REC>,
+ <&iommu M4U_LARB3_ID M4U_PORT_VENC_BSDMA>,
+ <&iommu M4U_LARB3_ID M4U_PORT_VENC_SV_COMV>,
+ <&iommu M4U_LARB3_ID M4U_PORT_VENC_RD_COMV>,
+ <&iommu M4U_LARB3_ID M4U_PORT_VENC_CUR_LUMA>,
+ <&iommu M4U_LARB3_ID M4U_PORT_VENC_CUR_CHROMA>,
+ <&iommu M4U_LARB3_ID M4U_PORT_VENC_REF_LUMA>,
+ <&iommu M4U_LARB3_ID M4U_PORT_VENC_REF_CHROMA>,
+ <&iommu M4U_LARB3_ID M4U_PORT_VENC_NBM_RDMA>,
+ <&iommu M4U_LARB3_ID M4U_PORT_VENC_NBM_WDMA>,
+ <&iommu M4U_LARB5_ID M4U_PORT_VENC_RCPU_SET2>,
+ <&iommu M4U_LARB5_ID M4U_PORT_VENC_REC_FRM_SET2>,
+ <&iommu M4U_LARB5_ID M4U_PORT_VENC_BSDMA_SET2>,
+ <&iommu M4U_LARB5_ID M4U_PORT_VENC_SV_COMA_SET2>,
+ <&iommu M4U_LARB5_ID M4U_PORT_VENC_RD_COMA_SET2>,
+ <&iommu M4U_LARB5_ID M4U_PORT_VENC_CUR_LUMA_SET2>,
+ <&iommu M4U_LARB5_ID M4U_PORT_VENC_CUR_CHROMA_SET2>,
+ <&iommu M4U_LARB5_ID M4U_PORT_VENC_REF_LUMA_SET2>,
+ <&iommu M4U_LARB5_ID M4U_PORT_VENC_REC_CHROMA_SET2>;
+ vpu = <&vpu>;
+ clocks = <&apmixedsys CLK_APMIXED_VENCPLL>,
+ <&topckgen CLK_TOP_VENC_LT_SEL>,
+ <&topckgen CLK_TOP_VCODECPLL_370P5>;
+ clock-names = "vencpll",
+ "venc_lt_sel",
+ "vcodecpll_370p5_ck";
+ };
+
vencltsys: clock-controller at 19000000 {
compatible = "mediatek,mt8173-vencltsys", "syscon";
reg = <0 0x19000000 0 0x1000>;
--
1.7.9.5
^ permalink raw reply related [flat|nested] 84+ messages in thread
* [RESEND RFC/PATCH 6/8] media: platform: mtk-vcodec: Add Mediatek V4L2 Video Encoder Driver
2015-11-17 12:54 ` Tiffany Lin
@ 2015-11-17 12:54 ` Tiffany Lin
-1 siblings, 0 replies; 84+ messages in thread
From: Tiffany Lin @ 2015-11-17 12:54 UTC (permalink / raw)
To: Rob Herring, Pawel Moll, Mark Rutland, Ian Campbell, Kumar Gala,
Catalin Marinas, Will Deacon, Mauro Carvalho Chehab,
Matthias Brugger, Daniel Kurtz, Sascha Hauer, Hongzhou Yang,
Hans Verkuil, Laurent Pinchart, Sakari Ailus, Geert Uytterhoeven,
Mikhail Ulyanov, Fabien Dessenne, Arnd Bergmann,
Darren Etheridge, Peter Griffin, Benoit Parrot
Cc: Tiffany Lin, Andrew-CT Chen, Eddie Huang, Yingjoe Chen,
James Liao, Daniel Hsiao, devicetree, linux-kernel,
linux-arm-kernel, linux-media, linux-mediatek
Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
Signed-off-by: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
---
drivers/media/platform/Kconfig | 13 +
drivers/media/platform/Makefile | 3 +
drivers/media/platform/mtk-vcodec/Kconfig | 5 +
drivers/media/platform/mtk-vcodec/Makefile | 12 +
drivers/media/platform/mtk-vcodec/common/Makefile | 8 +
.../media/platform/mtk-vcodec/common/venc_drv_if.c | 152 ++
.../platform/mtk-vcodec/include/venc_drv_base.h | 68 +
.../platform/mtk-vcodec/include/venc_drv_if.h | 187 +++
.../platform/mtk-vcodec/include/venc_ipi_msg.h | 212 +++
drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h | 441 +++++
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c | 1773 ++++++++++++++++++++
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h | 28 +
.../media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c | 535 ++++++
.../media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c | 122 ++
.../media/platform/mtk-vcodec/mtk_vcodec_intr.c | 110 ++
.../media/platform/mtk-vcodec/mtk_vcodec_intr.h | 30 +
drivers/media/platform/mtk-vcodec/mtk_vcodec_pm.h | 26 +
.../media/platform/mtk-vcodec/mtk_vcodec_util.c | 106 ++
.../media/platform/mtk-vcodec/mtk_vcodec_util.h | 66 +
19 files changed, 3897 insertions(+)
create mode 100644 drivers/media/platform/mtk-vcodec/Kconfig
create mode 100644 drivers/media/platform/mtk-vcodec/Makefile
create mode 100644 drivers/media/platform/mtk-vcodec/common/Makefile
create mode 100644 drivers/media/platform/mtk-vcodec/common/venc_drv_if.c
create mode 100644 drivers/media/platform/mtk-vcodec/include/venc_drv_base.h
create mode 100644 drivers/media/platform/mtk-vcodec/include/venc_drv_if.h
create mode 100644 drivers/media/platform/mtk-vcodec/include/venc_ipi_msg.h
create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h
create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.c
create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.h
create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_pm.h
create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c
create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
index f98eb47..b66cf1f 100644
--- a/drivers/media/platform/Kconfig
+++ b/drivers/media/platform/Kconfig
@@ -154,6 +154,19 @@ config MEDIATEK_VPU
This driver provides downloading firmware vpu and
communicating with vpu.
+config VIDEO_MEDIATEK_VCODEC
+ tristate "Mediatek Video Codec driver"
+ depends on VIDEO_DEV && VIDEO_V4L2
+ depends on ARCH_MEDIATEK || COMPILE_TEST
+ select VIDEOBUF2_DMA_CONTIG
+ select V4L2_MEM2MEM_DEV
+ select MEDIATEK_VPU
+ default n
+ ---help---
+ Mediatek video codec driver for V4L2
+
+source "drivers/media/platform/mtk-vcodec/Kconfig"
+
config VIDEO_MEM2MEM_DEINTERLACE
tristate "Deinterlace support"
depends on VIDEO_DEV && VIDEO_V4L2 && DMA_ENGINE
diff --git a/drivers/media/platform/Makefile b/drivers/media/platform/Makefile
index 1b4c539..423b9f6 100644
--- a/drivers/media/platform/Makefile
+++ b/drivers/media/platform/Makefile
@@ -57,3 +57,6 @@ obj-$(CONFIG_VIDEO_XILINX) += xilinx/
ccflags-y += -I$(srctree)/drivers/media/i2c
obj-$(CONFIG_MEDIATEK_VPU) += mtk-vpu/
+
+obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += mtk-vcodec/
+
diff --git a/drivers/media/platform/mtk-vcodec/Kconfig b/drivers/media/platform/mtk-vcodec/Kconfig
new file mode 100644
index 0000000..1c0b935
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/Kconfig
@@ -0,0 +1,5 @@
+config MEDIATEK_VPU
+ bool
+ ---help---
+ This driver provides downloading firmware vpu (video processor unit)
+ and communicating with vpu.
diff --git a/drivers/media/platform/mtk-vcodec/Makefile b/drivers/media/platform/mtk-vcodec/Makefile
new file mode 100644
index 0000000..c7f7174
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/Makefile
@@ -0,0 +1,12 @@
+obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += mtk_vcodec_intr.o \
+ mtk_vcodec_util.o \
+ mtk_vcodec_enc_drv.o \
+ mtk_vcodec_enc.o \
+ mtk_vcodec_enc_pm.o
+
+obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += common/
+
+ccflags-y += -I$(srctree)/drivers/media/platform/mtk-vcodec/include \
+ -I$(srctree)/drivers/media/platform/mtk-vcodec \
+ -I$(srctree)/drivers/media/platform/mtk-vpu
+
diff --git a/drivers/media/platform/mtk-vcodec/common/Makefile b/drivers/media/platform/mtk-vcodec/common/Makefile
new file mode 100644
index 0000000..477ab80
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/common/Makefile
@@ -0,0 +1,8 @@
+obj-y += \
+ venc_drv_if.o
+
+ccflags-y += \
+ -I$(srctree)/include/ \
+ -I$(srctree)/drivers/media/platform/mtk-vcodec \
+ -I$(srctree)/drivers/media/platform/mtk-vcodec/include \
+ -I$(srctree)/drivers/media/platform/mtk-vpu
\ No newline at end of file
diff --git a/drivers/media/platform/mtk-vcodec/common/venc_drv_if.c b/drivers/media/platform/mtk-vcodec/common/venc_drv_if.c
new file mode 100644
index 0000000..9b3f025
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/common/venc_drv_if.c
@@ -0,0 +1,152 @@
+/*
+ * Copyright (c) 2015 MediaTek Inc.
+ * Author: Daniel Hsiao <daniel.hsiao@mediatek.com>
+ * Jungchang Tsao <jungchang.tsao@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/interrupt.h>
+#include <linux/kernel.h>
+#include <linux/slab.h>
+
+#include "mtk_vcodec_drv.h"
+#include "mtk_vcodec_enc.h"
+#include "mtk_vcodec_pm.h"
+#include "mtk_vcodec_util.h"
+#include "mtk_vpu_core.h"
+
+#include "venc_drv_if.h"
+#include "venc_drv_base.h"
+
+
+int venc_if_create(void *ctx, unsigned int fourcc, unsigned long *handle)
+{
+ struct venc_handle *h;
+ char str[10];
+
+ mtk_vcodec_fmt2str(fourcc, str);
+
+ h = kzalloc(sizeof(*h), GFP_KERNEL);
+ if (!h)
+ return -ENOMEM;
+
+ h->fourcc = fourcc;
+ h->ctx = ctx;
+ mtk_vcodec_debug(h, "fmt = %s handle = %p", str, h);
+
+ switch (fourcc) {
+ default:
+ mtk_vcodec_err(h, "invalid format %s", str);
+ goto err_out;
+ }
+
+ *handle = (unsigned long)h;
+ return 0;
+
+err_out:
+ kfree(h);
+ return -EINVAL;
+}
+
+int venc_if_init(unsigned long handle)
+{
+ int ret = 0;
+ struct venc_handle *h = (struct venc_handle *)handle;
+
+ mtk_vcodec_debug_enter(h);
+
+ mtk_venc_lock(h->ctx);
+ mtk_vcodec_enc_clock_on();
+ vpu_enable_clock(vpu_get_plat_device(h->ctx->dev->plat_dev));
+ ret = h->enc_if->init(h->ctx, (unsigned long *)&h->drv_handle);
+ vpu_disable_clock(vpu_get_plat_device(h->ctx->dev->plat_dev));
+ mtk_vcodec_enc_clock_off();
+ mtk_venc_unlock(h->ctx);
+
+ return ret;
+}
+
+int venc_if_set_param(unsigned long handle,
+ enum venc_set_param_type type, void *in)
+{
+ int ret = 0;
+ struct venc_handle *h = (struct venc_handle *)handle;
+
+ mtk_vcodec_debug(h, "type=%d", type);
+
+ mtk_venc_lock(h->ctx);
+ mtk_vcodec_enc_clock_on();
+ vpu_enable_clock(vpu_get_plat_device(h->ctx->dev->plat_dev));
+ ret = h->enc_if->set_param(h->drv_handle, type, in);
+ vpu_disable_clock(vpu_get_plat_device(h->ctx->dev->plat_dev));
+ mtk_vcodec_enc_clock_off();
+ mtk_venc_unlock(h->ctx);
+
+ return ret;
+}
+
+int venc_if_encode(unsigned long handle,
+ enum venc_start_opt opt, struct venc_frm_buf *frm_buf,
+ struct mtk_vcodec_mem *bs_buf,
+ struct venc_done_result *result)
+{
+ int ret = 0;
+ struct venc_handle *h = (struct venc_handle *)handle;
+ char str[10];
+
+ mtk_vcodec_fmt2str(h->fourcc, str);
+ mtk_vcodec_debug(h, "fmt=%s", str);
+
+ mtk_venc_lock(h->ctx);
+ mtk_vcodec_enc_clock_on();
+ vpu_enable_clock(vpu_get_plat_device(h->ctx->dev->plat_dev));
+ enable_irq((h->fourcc == V4L2_PIX_FMT_H264) ?
+ h->ctx->dev->enc_irq : h->ctx->dev->enc_lt_irq);
+ ret = h->enc_if->encode(h->drv_handle, opt, frm_buf, bs_buf, result);
+ disable_irq((h->fourcc == V4L2_PIX_FMT_H264) ?
+ h->ctx->dev->enc_irq : h->ctx->dev->enc_lt_irq);
+ vpu_disable_clock(vpu_get_plat_device(h->ctx->dev->plat_dev));
+ mtk_vcodec_enc_clock_off();
+ mtk_venc_unlock(h->ctx);
+
+ mtk_vcodec_debug(h, "ret=%d", ret);
+
+ return ret;
+}
+
+int venc_if_deinit(unsigned long handle)
+{
+ int ret = 0;
+ struct venc_handle *h = (struct venc_handle *)handle;
+
+ mtk_vcodec_debug_enter(h);
+
+ mtk_venc_lock(h->ctx);
+ mtk_vcodec_enc_clock_on();
+ vpu_enable_clock(vpu_get_plat_device(h->ctx->dev->plat_dev));
+ ret = h->enc_if->deinit(h->drv_handle);
+ vpu_disable_clock(vpu_get_plat_device(h->ctx->dev->plat_dev));
+ mtk_vcodec_enc_clock_off();
+ mtk_venc_unlock(h->ctx);
+
+ return ret;
+}
+
+int venc_if_release(unsigned long handle)
+{
+ struct venc_handle *h = (struct venc_handle *)handle;
+
+ mtk_vcodec_debug_enter(h);
+ kfree(h);
+
+ return 0;
+}
diff --git a/drivers/media/platform/mtk-vcodec/include/venc_drv_base.h b/drivers/media/platform/mtk-vcodec/include/venc_drv_base.h
new file mode 100644
index 0000000..8828294
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/include/venc_drv_base.h
@@ -0,0 +1,68 @@
+/*
+ * Copyright (c) 2015 MediaTek Inc.
+ * Author: Daniel Hsiao <daniel.hsiao@mediatek.com>
+ * Jungchang Tsao <jungchang.tsao@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef _VENC_DRV_BASE_
+#define _VENC_DRV_BASE_
+
+#include "mtk_vcodec_drv.h"
+
+#include "venc_drv_if.h"
+
+struct venc_common_if {
+ /**
+ * (*init)() - initialize driver
+ * @ctx: [in] mtk v4l2 context
+ * @handle: [out] driver handle
+ */
+ int (*init)(struct mtk_vcodec_ctx *ctx, unsigned long *handle);
+
+ /**
+ * (*encode)() - trigger encode
+ * @handle: [in] driver handle
+ * @opt: [in] encode option
+ * @frm_buf: [in] frame buffer to store input frame
+ * @bs_buf: [in] bitstream buffer to store output bitstream
+ * @result: [out] encode result
+ */
+ int (*encode)(unsigned long handle, enum venc_start_opt opt,
+ struct venc_frm_buf *frm_buf,
+ struct mtk_vcodec_mem *bs_buf,
+ struct venc_done_result *result);
+
+ /**
+ * (*set_param)() - set driver's parameter
+ * @handle: [in] driver handle
+ * @type: [in] parameter type
+ * @in: [in] buffer to store the parameter
+ */
+ int (*set_param)(unsigned long handle, enum venc_set_param_type type,
+ void *in);
+
+ /**
+ * (*deinit)() - deinitialize driver.
+ * @handle: [in] driver handle
+ */
+ int (*deinit)(unsigned long handle);
+};
+
+struct venc_handle {
+ unsigned int fourcc;
+ struct venc_common_if *enc_if;
+ unsigned long drv_handle;
+ struct mtk_vcodec_ctx *ctx;
+};
+
+#endif
diff --git a/drivers/media/platform/mtk-vcodec/include/venc_drv_if.h b/drivers/media/platform/mtk-vcodec/include/venc_drv_if.h
new file mode 100644
index 0000000..ef0b0b9
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/include/venc_drv_if.h
@@ -0,0 +1,187 @@
+/*
+ * Copyright (c) 2015 MediaTek Inc.
+ * Author: Daniel Hsiao <daniel.hsiao@mediatek.com>
+ * Jungchang Tsao <jungchang.tsao@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef _VENC_DRV_IF_H_
+#define _VENC_DRV_IF_H_
+
+#include "mtk_vcodec_util.h"
+
+/*
+ * enum venc_yuv_fmt - The type of input yuv format
+ * (VPU related: If you change the order, you must also update the VPU codes.)
+ * @VENC_YUV_FORMAT_420: 420 YUV format
+ * @VENC_YUV_FORMAT_YV12: YV12 YUV format
+ * @VENC_YUV_FORMAT_NV12: NV12 YUV format
+ * @VENC_YUV_FORMAT_NV21: NV21 YUV format
+ */
+enum venc_yuv_fmt {
+ VENC_YUV_FORMAT_420 = 3,
+ VENC_YUV_FORMAT_YV12 = 5,
+ VENC_YUV_FORMAT_NV12 = 6,
+ VENC_YUV_FORMAT_NV21 = 7,
+};
+
+/*
+ * enum venc_start_opt - encode frame option used in venc_if_encode()
+ * @VENC_START_OPT_ENCODE_SEQUENCE_HEADER: encode SPS/PPS for H264
+ * @VENC_START_OPT_ENCODE_FRAME: encode normal frame
+ */
+enum venc_start_opt {
+ VENC_START_OPT_ENCODE_SEQUENCE_HEADER,
+ VENC_START_OPT_ENCODE_FRAME,
+};
+
+/*
+ * enum venc_drv_msg - The type of encode frame status used in venc_if_encode()
+ * @VENC_MESSAGE_OK: encode ok
+ * @VENC_MESSAGE_ERR: encode error
+ */
+enum venc_drv_msg {
+ VENC_MESSAGE_OK,
+ VENC_MESSAGE_ERR,
+};
+
+/*
+ * enum venc_set_param_type - The type of set parameter used in venc_if_set_param()
+ * (VPU related: If you change the order, you must also update the VPU codes.)
+ * @VENC_SET_PARAM_ENC: set encoder parameters
+ * @VENC_SET_PARAM_FORCE_INTRA: set force intra frame
+ * @VENC_SET_PARAM_ADJUST_BITRATE: set to adjust bitrate (in bps)
+ * @VENC_SET_PARAM_ADJUST_FRAMERATE: set frame rate
+ * @VENC_SET_PARAM_I_FRAME_INTERVAL: set I frame interval
+ * @VENC_SET_PARAM_SKIP_FRAME: set H264 skip one frame
+ * @VENC_SET_PARAM_PREPEND_HEADER: set H264 prepend SPS/PPS before IDR
+ * @VENC_SET_PARAM_TS_MODE: set VP8 temporal scalability mode
+ */
+enum venc_set_param_type {
+ VENC_SET_PARAM_ENC,
+ VENC_SET_PARAM_FORCE_INTRA,
+ VENC_SET_PARAM_ADJUST_BITRATE,
+ VENC_SET_PARAM_ADJUST_FRAMERATE,
+ VENC_SET_PARAM_I_FRAME_INTERVAL,
+ VENC_SET_PARAM_SKIP_FRAME,
+ VENC_SET_PARAM_PREPEND_HEADER,
+ VENC_SET_PARAM_TS_MODE,
+};
+
+/*
+ * struct venc_enc_prm - encoder settings for VENC_SET_PARAM_ENC used in venc_if_set_param()
+ * @input_fourcc: input fourcc
+ * @h264_profile: V4L2 defined H.264 profile
+ * @h264_level: V4L2 defined H.264 level
+ * @width: image width
+ * @height: image height
+ * @buf_width: buffer width
+ * @buf_height: buffer height
+ * @frm_rate: frame rate
+ * @intra_period: intra frame period
+ * @bitrate: target bitrate in kbps
+ */
+struct venc_enc_prm {
+ enum venc_yuv_fmt input_fourcc;
+ unsigned int h264_profile;
+ unsigned int h264_level;
+ unsigned int width;
+ unsigned int height;
+ unsigned int buf_width;
+ unsigned int buf_height;
+ unsigned int frm_rate;
+ unsigned int intra_period;
+ unsigned int bitrate;
+};
+
+/*
+ * struct venc_frm_buf - frame buffer information used in venc_if_encode()
+ * @fb_addr: plane 0 frame buffer address
+ * @fb_addr1: plane 1 frame buffer address
+ * @fb_addr2: plane 2 frame buffer address
+ */
+struct venc_frm_buf {
+ struct mtk_vcodec_mem fb_addr;
+ struct mtk_vcodec_mem fb_addr1;
+ struct mtk_vcodec_mem fb_addr2;
+};
+
+/*
+ * struct venc_done_result - This is return information used in venc_if_encode()
+ * @msg: message, such as success or error code
+ * @bs_size: output bitstream size
+ * @is_key_frm: output is key frame or not
+ */
+struct venc_done_result {
+ enum venc_drv_msg msg;
+ unsigned int bs_size;
+ bool is_key_frm;
+};
+
+/*
+ * venc_if_create - Create the driver handle
+ * @ctx: device context
+ * @fourcc: encoder output format
+ * @handle: driver handle
+ * Return: 0 if creating handle successfully, otherwise it is failed.
+ */
+int venc_if_create(void *ctx, unsigned int fourcc, unsigned long *handle);
+
+/*
+ * venc_if_release - Release the driver handle
+ * @handle: driver handle
+ * Return: 0 if releasing handle successfully, otherwise it is failed.
+ */
+int venc_if_release(unsigned long handle);
+
+/*
+ * venc_if_init - Init the driver setting, alloc working memory ... etc.
+ * @handle: driver handle
+ * Return: 0 if init handle successfully, otherwise it is failed.
+ */
+int venc_if_init(unsigned long handle);
+
+/*
+ * venc_if_deinit - DeInit the driver setting, free working memory ... etc.
+ * @handle: driver handle
+ * Return: 0 if deinit handle successfully, otherwise it is failed.
+ */
+int venc_if_deinit(unsigned long handle);
+
+/*
+ * venc_if_set_param - Set parameter to driver
+ * @handle: driver handle
+ * @type: set type
+ * @in: input parameter
+ * @out: output parameter
+ * Return: 0 if setting param successfully, otherwise it is failed.
+ */
+int venc_if_set_param(unsigned long handle,
+ enum venc_set_param_type type,
+ void *in);
+
+/*
+ * venc_if_encode - Encode frame
+ * @handle: driver handle
+ * @opt: encode frame option
+ * @frm_buf: input frame buffer information
+ * @bs_buf: output bitstream buffer infomraiton
+ * @result: encode result
+ * Return: 0 if encoding frame successfully, otherwise it is failed.
+ */
+int venc_if_encode(unsigned long handle,
+ enum venc_start_opt opt,
+ struct venc_frm_buf *frm_buf,
+ struct mtk_vcodec_mem *bs_buf,
+ struct venc_done_result *result);
+
+#endif /* _VENC_DRV_IF_H_ */
diff --git a/drivers/media/platform/mtk-vcodec/include/venc_ipi_msg.h b/drivers/media/platform/mtk-vcodec/include/venc_ipi_msg.h
new file mode 100644
index 0000000..a345b98
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/include/venc_ipi_msg.h
@@ -0,0 +1,212 @@
+/*
+ * Copyright (c) 2015 MediaTek Inc.
+ * Author: Jungchang Tsao <jungchang.tsao@mediatek.com>
+ * Daniel Hsiao <daniel.hsiao@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef _VENC_IPI_MSG_H_
+#define _VENC_IPI_MSG_H_
+
+#define IPIMSG_H264_ENC_ID 0x100
+#define IPIMSG_VP8_ENC_ID 0x200
+
+#define AP_IPIMSG_VENC_BASE 0x20000
+#define VPU_IPIMSG_VENC_BASE 0x30000
+
+/**
+ * enum venc_ipi_msg_id - message id between AP and VPU
+ * (ipi stands for inter-processor interrupt)
+ * @AP_IPIMSG_XXX: AP to VPU cmd message id
+ * @VPU_IPIMSG_XXX_DONE: VPU ack AP cmd message id
+ */
+enum venc_ipi_msg_id {
+ AP_IPIMSG_H264_ENC_INIT = AP_IPIMSG_VENC_BASE +
+ IPIMSG_H264_ENC_ID,
+ AP_IPIMSG_H264_ENC_SET_PARAM,
+ AP_IPIMSG_H264_ENC_ENCODE,
+ AP_IPIMSG_H264_ENC_DEINIT,
+
+ AP_IPIMSG_VP8_ENC_INIT = AP_IPIMSG_VENC_BASE +
+ IPIMSG_VP8_ENC_ID,
+ AP_IPIMSG_VP8_ENC_SET_PARAM,
+ AP_IPIMSG_VP8_ENC_ENCODE,
+ AP_IPIMSG_VP8_ENC_DEINIT,
+
+ VPU_IPIMSG_H264_ENC_INIT_DONE = VPU_IPIMSG_VENC_BASE +
+ IPIMSG_H264_ENC_ID,
+ VPU_IPIMSG_H264_ENC_SET_PARAM_DONE,
+ VPU_IPIMSG_H264_ENC_ENCODE_DONE,
+ VPU_IPIMSG_H264_ENC_DEINIT_DONE,
+
+ VPU_IPIMSG_VP8_ENC_INIT_DONE = VPU_IPIMSG_VENC_BASE +
+ IPIMSG_VP8_ENC_ID,
+ VPU_IPIMSG_VP8_ENC_SET_PARAM_DONE,
+ VPU_IPIMSG_VP8_ENC_ENCODE_DONE,
+ VPU_IPIMSG_VP8_ENC_DEINIT_DONE,
+};
+
+/**
+ * struct venc_ap_ipi_msg_init - AP to VPU init cmd structure
+ * @msg_id: message id (AP_IPIMSG_XXX_ENC_INIT)
+ * @venc_inst: AP encoder instance (struct venc_vp8_handle/venc_h264_handle *)
+ */
+struct venc_ap_ipi_msg_init {
+ uint32_t msg_id;
+ uint32_t reserved;
+ uint64_t venc_inst;
+};
+
+/**
+ * struct venc_ap_ipi_msg_set_param - AP to VPU set_param cmd structure
+ * @msg_id: message id (AP_IPIMSG_XXX_ENC_SET_PARAM)
+ * @inst_id: VPU encoder instance id (struct venc_vp8_vpu_drv/venc_h264_vpu_drv *)
+ * @param_id: parameter id (venc_set_param_type)
+ * @data_item: number of items in the data array
+ * @data[8]: data array to store the set parameters
+ */
+struct venc_ap_ipi_msg_set_param {
+ uint32_t msg_id;
+ uint32_t inst_id;
+ uint32_t param_id;
+ uint32_t data_item;
+ uint32_t data[8];
+};
+
+/**
+ * struct venc_ap_ipi_msg_enc - AP to VPU enc cmd structure
+ * @msg_id: message id (AP_IPIMSG_XXX_ENC_ENCODE)
+ * @inst_id: VPU encoder instance id (struct venc_vp8_vpu_drv/venc_h264_vpu_drv *)
+ * @bs_mode: bitstream mode for h264
+ * (H264_BS_MODE_SPS/H264_BS_MODE_PPS/H264_BS_MODE_FRAME)
+ * @input_addr: pointer to input image buffer plane
+ * @bs_addr: pointer to output bit stream buffer
+ * @bs_size: bit stream buffer size
+ */
+struct venc_ap_ipi_msg_enc {
+ uint32_t msg_id;
+ uint32_t inst_id;
+ uint32_t bs_mode;
+ uint32_t input_addr[3];
+ uint32_t bs_addr;
+ uint32_t bs_size;
+};
+
+/**
+ * struct venc_ap_ipi_msg_deinit - AP to VPU deinit cmd structure
+ * @msg_id: message id (AP_IPIMSG_XXX_ENC_DEINIT)
+ * @inst_id: VPU encoder instance id (struct venc_vp8_vpu_drv/venc_h264_vpu_drv *)
+ */
+struct venc_ap_ipi_msg_deinit {
+ uint32_t msg_id;
+ uint32_t inst_id;
+};
+
+/**
+ * enum venc_ipi_msg_status - VPU ack AP cmd status
+ */
+enum venc_ipi_msg_status {
+ VENC_IPI_MSG_STATUS_OK,
+ VENC_IPI_MSG_STATUS_FAIL,
+};
+
+/**
+ * struct venc_vpu_ipi_msg_common - VPU ack AP cmd common structure
+ * @msg_id: message id (VPU_IPIMSG_XXX_DONE)
+ * @status: cmd status (venc_ipi_msg_status)
+ * @venc_inst: AP encoder instance (struct venc_vp8_handle/venc_h264_handle *)
+ */
+struct venc_vpu_ipi_msg_common {
+ uint32_t msg_id;
+ uint32_t status;
+ uint64_t venc_inst;
+};
+
+/**
+ * struct venc_vpu_ipi_msg_init - VPU ack AP init cmd structure
+ * @msg_id: message id (VPU_IPIMSG_XXX_ENC_SET_PARAM_DONE)
+ * @status: cmd status (venc_ipi_msg_status)
+ * @venc_inst: AP encoder instance (struct venc_vp8_handle/venc_h264_handle *)
+ * @inst_id: VPU encoder instance id (struct venc_vp8_vpu_drv/venc_h264_vpu_drv *)
+ */
+struct venc_vpu_ipi_msg_init {
+ uint32_t msg_id;
+ uint32_t status;
+ uint64_t venc_inst;
+ uint32_t inst_id;
+ uint32_t reserved;
+};
+
+/**
+ * struct venc_vpu_ipi_msg_set_param - VPU ack AP set_param cmd structure
+ * @msg_id: message id (VPU_IPIMSG_XXX_ENC_SET_PARAM_DONE)
+ * @status: cmd status (venc_ipi_msg_status)
+ * @venc_inst: AP encoder instance (struct venc_vp8_handle/venc_h264_handle *)
+ * @param_id: parameter id (venc_set_param_type)
+ * @data_item: number of items in the data array
+ * @data[6]: data array to store the return result
+ */
+struct venc_vpu_ipi_msg_set_param {
+ uint32_t msg_id;
+ uint32_t status;
+ uint64_t venc_inst;
+ uint32_t param_id;
+ uint32_t data_item;
+ uint32_t data[6];
+};
+
+/**
+ * enum venc_ipi_msg_enc_state - Type of encode state
+ * VEN_IPI_MSG_ENC_STATE_FRAME: one frame being encoded
+ * VEN_IPI_MSG_ENC_STATE_PART: bit stream buffer full
+ * VEN_IPI_MSG_ENC_STATE_SKIP: encoded skip frame
+ * VEN_IPI_MSG_ENC_STATE_ERROR: encounter error
+ */
+enum venc_ipi_msg_enc_state {
+ VEN_IPI_MSG_ENC_STATE_FRAME,
+ VEN_IPI_MSG_ENC_STATE_PART,
+ VEN_IPI_MSG_ENC_STATE_SKIP,
+ VEN_IPI_MSG_ENC_STATE_ERROR,
+};
+
+/**
+ * struct venc_vpu_ipi_msg_enc - VPU ack AP enc cmd structure
+ * @msg_id: message id (VPU_IPIMSG_XXX_ENC_ENCODE_DONE)
+ * @status: cmd status (venc_ipi_msg_status)
+ * @venc_inst: AP encoder instance (struct venc_vp8_handle/venc_h264_handle *)
+ * @state: encode state (venc_ipi_msg_enc_state)
+ * @key_frame: whether the encoded frame is key frame
+ * @bs_size: encoded bitstream size
+ */
+struct venc_vpu_ipi_msg_enc {
+ uint32_t msg_id;
+ uint32_t status;
+ uint64_t venc_inst;
+ uint32_t state;
+ uint32_t key_frame;
+ uint32_t bs_size;
+ uint32_t reserved;
+};
+
+/**
+ * struct venc_vpu_ipi_msg_deinit - VPU ack AP deinit cmd structure
+ * @msg_id: message id (VPU_IPIMSG_XXX_ENC_DEINIT_DONE)
+ * @status: cmd status (venc_ipi_msg_status)
+ * @venc_inst: AP encoder instance (struct venc_vp8_handle/venc_h264_handle *)
+ */
+struct venc_vpu_ipi_msg_deinit {
+ uint32_t msg_id;
+ uint32_t status;
+ uint64_t venc_inst;
+};
+
+#endif /* _VENC_IPI_MSG_H_ */
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
new file mode 100644
index 0000000..22239f8
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
@@ -0,0 +1,441 @@
+/*
+* Copyright (c) 2015 MediaTek Inc.
+* Author: PC Chen <pc.chen@mediatek.com>
+* Tiffany Lin <tiffany.lin@mediatek.com>
+*
+* This program is free software; you can redistribute it and/or modify
+* it under the terms of the GNU General Public License version 2 as
+* published by the Free Software Foundation.
+*
+* This program is distributed in the hope that it will be useful,
+* but WITHOUT ANY WARRANTY; without even the implied warranty of
+* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+* GNU General Public License for more details.
+*/
+
+#ifndef _MTK_VCODEC_DRV_H_
+#define _MTK_VCODEC_DRV_H_
+
+#include <linux/platform_device.h>
+#include <linux/videodev2.h>
+#include <media/v4l2-ctrls.h>
+#include <media/v4l2-device.h>
+#include <media/v4l2-ioctl.h>
+#include <media/videobuf2-core.h>
+#include <media/videobuf2-v4l2.h>
+
+#include "venc_drv_if.h"
+
+#define MTK_VCODEC_MAX_INSTANCES 32
+#define MTK_VCODEC_MAX_FRAME_SIZE 0x800000
+#define MTK_VIDEO_MAX_FRAME 32
+#define MTK_MAX_CTRLS 10
+
+#define MTK_VCODEC_DRV_NAME "mtk_vcodec_drv"
+#define MTK_VCODEC_ENC_NAME "mtk-vcodec-enc"
+
+#define MTK_VENC_IRQ_STATUS_SPS 0x1
+#define MTK_VENC_IRQ_STATUS_PPS 0x2
+#define MTK_VENC_IRQ_STATUS_FRM 0x4
+#define MTK_VENC_IRQ_STATUS_DRAM 0x8
+#define MTK_VENC_IRQ_STATUS_PAUSE 0x10
+#define MTK_VENC_IRQ_STATUS_SWITCH 0x20
+
+#define MTK_VENC_IRQ_STATUS_OFFSET 0x05C
+#define MTK_VENC_IRQ_ACK_OFFSET 0x060
+
+#define MTK_VCODEC_MAX_PLANES 3
+
+#define VDEC_HW_ACTIVE 0x10
+#define VDEC_IRQ_CFG 0x11
+#define VDEC_IRQ_CLR 0x10
+
+#define VDEC_IRQ_CFG_REG 0xa4
+#define NUM_MAX_ALLOC_CTX 4
+#define MTK_V4L2_BENCHMARK 0
+#define USE_ENCODE_THREAD 1
+
+/**
+ * enum mtk_hw_reg_idx - MTK hw register base index
+ */
+enum mtk_hw_reg_idx {
+ VDEC_SYS,
+ VDEC_MISC,
+ VDEC_LD,
+ VDEC_TOP,
+ VDEC_CM,
+ VDEC_AD,
+ VDEC_AV,
+ VDEC_PP,
+ VDEC_HWD,
+ VDEC_HWQ,
+ VDEC_HWB,
+ VDEC_HWG,
+ NUM_MAX_VDEC_REG_BASE,
+ VENC_SYS = NUM_MAX_VDEC_REG_BASE,
+ VENC_LT_SYS,
+ NUM_MAX_VCODEC_REG_BASE
+};
+
+/**
+ * enum mtk_instance_type - The type of an MTK Vcodec instance.
+ */
+enum mtk_instance_type {
+ MTK_INST_DECODER = 0,
+ MTK_INST_ENCODER = 1,
+};
+
+/**
+ * enum mtk_instance_state - The state of an MTK Vcodec instance.
+ * @MTK_STATE_FREE - default state when instance create
+ * @MTK_STATE_CREATE - vdec instance is create
+ * @MTK_STATE_INIT - vdec instance is init
+ * @MTK_STATE_CONFIG - reserved for encoder
+ * @MTK_STATE_HEADER - vdec had sps/pps header parsed
+ * @MTK_STATE_RUNNING - vdec is decoding
+ * @MTK_STATE_FLUSH - vdec is flushing
+ * @MTK_STATE_RES_CHANGE - vdec detect resolution change
+ * @MTK_STATE_FINISH - ctx instance is stopped streaming
+ * @MTK_STATE_DEINIT - before release ctx instance
+ * @MTK_STATE_ERROR - vdec has something wrong
+ * @MTK_STATE_ABORT - abort work in working thread
+ */
+enum mtk_instance_state {
+ MTK_STATE_FREE = 0,
+ MTK_STATE_CREATE = (1 << 0),
+ MTK_STATE_INIT = (1 << 1),
+ MTK_STATE_CONFIG = (1 << 2),
+ MTK_STATE_HEADER = (1 << 3),
+ MTK_STATE_RUNNING = (1 << 4),
+ MTK_STATE_FLUSH = (1 << 5),
+ MTK_STATE_RES_CHANGE = (1 << 6),
+ MTK_STATE_FINISH = (1 << 7),
+ MTK_STATE_DEINIT = (1 << 8),
+ MTK_STATE_ERROR = (1 << 9),
+ MTK_STATE_ABORT = (1 << 10),
+};
+
+/**
+ * struct mtk_param_change - General encoding parameters type
+ */
+enum mtk_encode_param {
+ MTK_ENCODE_PARAM_NONE = 0,
+ MTK_ENCODE_PARAM_BITRATE = (1 << 0),
+ MTK_ENCODE_PARAM_FRAMERATE = (1 << 1),
+ MTK_ENCODE_PARAM_INTRA_PERIOD = (1 << 2),
+ MTK_ENCODE_PARAM_FRAME_TYPE = (1 << 3),
+ MTK_ENCODE_PARAM_SKIP_FRAME = (1 << 4),
+};
+
+/**
+ * enum mtk_fmt_type - Type of the pixelformat
+ * @MTK_FMT_FRAME - mtk vcodec raw frame
+ */
+enum mtk_fmt_type {
+ MTK_FMT_DEC = 0,
+ MTK_FMT_ENC = 1,
+ MTK_FMT_FRAME = 2,
+};
+
+/**
+ * struct mtk_video_fmt - Structure used to store information about pixelformats
+ */
+struct mtk_video_fmt {
+ char *name;
+ u32 fourcc;
+ enum mtk_fmt_type type;
+ u32 num_planes;
+};
+
+/**
+ * struct mtk_codec_framesizes - Structure used to store information about framesizes
+ */
+struct mtk_codec_framesizes {
+ u32 fourcc;
+ struct v4l2_frmsize_stepwise stepwise;
+};
+
+/**
+ * struct mtk_q_type - Type of queue
+ */
+enum mtk_q_type {
+ MTK_Q_DATA_SRC = 0,
+ MTK_Q_DATA_DST = 1,
+};
+
+/**
+ * struct mtk_q_data - Structure used to store information about queue
+ * @colorspace reserved for encoder
+ * @field reserved for encoder
+ */
+struct mtk_q_data {
+ unsigned int width;
+ unsigned int height;
+ enum v4l2_field field;
+ enum v4l2_colorspace colorspace;
+ unsigned int bytesperline[MTK_VCODEC_MAX_PLANES];
+ unsigned int sizeimage[MTK_VCODEC_MAX_PLANES];
+ struct mtk_video_fmt *fmt;
+};
+
+/**
+ * struct mtk_enc_params - General encoding parameters
+ * @bitrate - target bitrate
+ * @num_b_frame - number of b frames between p-frame
+ * @rc_frame - frame based rate control
+ * @rc_mb - macroblock based rate control
+ * @seq_hdr_mode - H.264 sequence header is encoded separately or joined with the first frame
+ * @gop_size - group of picture size, it's used as the intra frame period
+ * @framerate_num - frame rate numerator
+ * @framerate_denom - frame rate denominator
+ * @h264_max_qp - Max value for H.264 quantization parameter
+ * @h264_profile - V4L2 defined H.264 profile
+ * @h264_level - V4L2 defined H.264 level
+ * @force_intra - force/insert intra frame
+ * @skip_frame - encode in skip frame mode that use minimum number of bits
+ */
+struct mtk_enc_params {
+ unsigned int bitrate;
+ unsigned int num_b_frame;
+ unsigned int rc_frame;
+ unsigned int rc_mb;
+ unsigned int seq_hdr_mode;
+ unsigned int gop_size;
+ unsigned int framerate_num;
+ unsigned int framerate_denom;
+ unsigned int h264_max_qp;
+ unsigned int h264_profile;
+ unsigned int h264_level;
+ unsigned int force_intra;
+ unsigned int skip_frame;
+};
+
+/**
+ * struct mtk_vcodec_pm - Power management data structure
+ */
+struct mtk_vcodec_pm {
+ struct clk *mmpll;
+ struct clk *vcodecpll;
+ struct clk *univpll_d2;
+ struct clk *clk_cci400_sel;
+ struct clk *vdecpll;
+ struct clk *vdec_sel;
+ struct clk *vencpll;
+ struct clk *venc_lt_sel;
+ struct clk *vcodecpll_370p5_ck;
+ struct device *larbvdec;
+ struct device *larbvenc;
+ struct device *larbvenclt;
+ struct device *dev;
+ struct mtk_vcodec_dev *mtkdev;
+};
+
+/**
+ * struct mtk_video_enc_buf - Private data related to each VB2 buffer.
+ * @b: Pointer to related VB2 buffer.
+ * @param_change: Types of encode parameter change before encode this
+ * buffer
+ * @enc_params Encode parameters changed before encode this buffer
+ */
+struct mtk_video_enc_buf {
+ struct vb2_v4l2_buffer b;
+ struct list_head list;
+
+ enum mtk_encode_param param_change;
+ struct mtk_enc_params enc_params;
+};
+
+/**
+ * struct mtk_vcodec_ctx - Context (instance) private data.
+ *
+ * @type: type of the instance - decoder or encoder
+ * @dev: pointer to the mtk_vcodec_dev of the device
+ * @fh: struct v4l2_fh
+ * @m2m_ctx: pointer to the v4l2_m2m_ctx of the context
+ * @q_data: store information of input and output queue
+ * of the context
+ * @idx: index of the context that this structure describes
+ * @state: state of the context
+ * @aborting:
+ * @param_change:
+ * @param_change_mutex:
+ * @enc_params: encoding parameters
+ * @colorspace:
+ *
+ * @h_enc: encoder handler
+ * @picinfo: store width/height of image and buffer and planes' size for decoder
+ * and encoder
+ * @pb_count: count of the DPB buffers required by MTK Vcodec hw
+ * @hdr:
+ *
+ * @int_cond: variable used by the waitqueue
+ * @int_type: type of the last interrupt
+ * @queue: waitqueue that can be used to wait for this context to
+ * finish
+ * @irq_status:
+ *
+ * @display_time_info: display time for destination buffers
+ * @in_time_idx: latest insert display time index
+ * @out_time_idx: first remove display time index
+ *
+ * @ctrl_hdl: handler for v4l2 framework
+ * @ctrls: array of controls, used when adding controls to the
+ * v4l2 control framework
+ *
+ * @encode_work: worker for the encoding
+ */
+struct mtk_vcodec_ctx {
+ enum mtk_instance_type type;
+ struct mtk_vcodec_dev *dev;
+ struct v4l2_fh fh;
+ struct v4l2_m2m_ctx *m2m_ctx;
+ struct mtk_q_data q_data[2];
+ int idx;
+ enum mtk_instance_state state;
+ int aborting;
+ enum mtk_encode_param param_change;
+ struct mutex encode_param_mutex;
+ struct mutex vb2_mutex;
+ struct mtk_enc_params enc_params;
+
+ unsigned long h_enc;
+ int hdr;
+
+ int int_cond;
+ int int_type;
+ wait_queue_head_t queue;
+ unsigned int irq_status;
+
+ struct v4l2_ctrl_handler ctrl_hdl;
+ struct v4l2_ctrl *ctrls[MTK_MAX_CTRLS];
+
+ struct work_struct encode_work;
+
+#if MTK_V4L2_BENCHMARK
+ unsigned int total_enc_dec_cnt;
+ unsigned int total_enc_dec_time;
+ unsigned int total_enc_hdr_time;
+ unsigned int total_enc_dec_init_time;
+
+ unsigned int total_qbuf_out_time;
+ unsigned int total_qbuf_cap_time;
+ unsigned int total_qbuf_out_cnt;
+ unsigned int total_qbuf_cap_cnt;
+ unsigned int total_dqbuf_out_time;
+ unsigned int total_dqbuf_cap_time;
+ unsigned int total_dqbuf_out_cnt;
+ unsigned int total_dqbuf_cap_cnt;
+ unsigned int total_dqbuf_cnt;
+ unsigned int total_expbuf_time;
+#endif
+
+};
+
+/**
+ * struct mtk_vcodec_dev - driver data
+ * @v4l2_dev: V4L2 device to register video devices for.
+ * @vfd_enc: Video device for encoder.
+ *
+ * @m2m_dev_enc: m2m device for encoder.
+ * @plat_dev: platform device
+ * @alloc_ctx: VB2 allocator context
+ * (for allocations without kernel mapping).
+ * @ctx: array of driver contexts
+ *
+ * @curr_ctx: The context that is waiting for codec hardware
+ *
+ * @reg_base: Mapped address of MTK Vcodec registers.
+ *
+ * @instance_mask: used to mark which contexts are opened
+ * @num_instances: counter of active MTK Vcodec instances
+ *
+ * @encode_workqueue: encode work queue
+ *
+ * @int_cond: used to identify interrupt condition happen
+ * @int_type: used to identify what kind of interrupt condition happen
+ * @dev_mutex: video_device lock
+ * @queue: waitqueue for waiting for completion of device commands
+ *
+ * @enc_irq: encoder irq resource.
+ * @enc_lt_irq: encoder lt irq resource.
+ *
+ * @enc_mutex: encoder hardware lock.
+ *
+ * @pm: power management control
+ * @dec_capability: used to identify decode capability, ex: 4k
+ * @enc_capability: used to identify encode capability
+ */
+struct mtk_vcodec_dev {
+ struct v4l2_device v4l2_dev;
+ struct video_device *vfd_enc;
+
+ struct v4l2_m2m_dev *m2m_dev_enc;
+ struct platform_device *plat_dev;
+ struct vb2_alloc_ctx *alloc_ctx;
+ struct mtk_vcodec_ctx *ctx[MTK_VCODEC_MAX_INSTANCES];
+ int curr_ctx;
+ void __iomem *reg_base[NUM_MAX_VCODEC_REG_BASE];
+
+ unsigned long instance_mask[BITS_TO_LONGS(MTK_VCODEC_MAX_INSTANCES)];
+ int num_instances;
+
+ struct workqueue_struct *encode_workqueue;
+
+ int int_cond;
+ int int_type;
+ struct mutex dev_mutex;
+ wait_queue_head_t queue;
+
+ int enc_irq;
+ int enc_lt_irq;
+
+ struct mutex enc_mutex;
+ unsigned long enter_suspend;
+
+ struct mtk_vcodec_pm pm;
+ unsigned int dec_capability;
+ unsigned int enc_capability;
+};
+
+/**
+ * struct mtk_vcodec_ctrl - information about controls to be registered.
+ * @id: Control ID.
+ * @type: Type of the control.
+ * @name: Human readable name of the control.
+ * @minimum: Minimum value of the control.
+ * @maximum: Maximum value of the control.
+ * @step: Control value increase step.
+ * @menu_skip_mask: Mask of invalid menu positions.
+ * @default_value: Initial value of the control.
+ * @is_volatile: Control is volatile.
+ *
+ * See also struct v4l2_ctrl_config.
+ */
+struct mtk_vcodec_ctrl {
+ u32 id;
+ enum v4l2_ctrl_type type;
+ u8 name[32];
+ s32 minimum;
+ s32 maximum;
+ s32 step;
+ u32 menu_skip_mask;
+ s32 default_value;
+ u8 is_volatile;
+};
+
+static inline struct mtk_vcodec_ctx *fh_to_ctx(struct v4l2_fh *fh)
+{
+ return container_of(fh, struct mtk_vcodec_ctx, fh);
+}
+
+static inline struct mtk_vcodec_ctx *ctrl_to_ctx(struct v4l2_ctrl *ctrl)
+{
+ return container_of(ctrl->handler, struct mtk_vcodec_ctx, ctrl_hdl);
+}
+
+extern const struct v4l2_ioctl_ops mtk_vdec_ioctl_ops;
+extern const struct v4l2_m2m_ops mtk_vdec_m2m_ops;
+extern const struct v4l2_ioctl_ops mtk_venc_ioctl_ops;
+extern const struct v4l2_m2m_ops mtk_venc_m2m_ops;
+
+#endif /* _MTK_VCODEC_DRV_H_ */
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
new file mode 100644
index 0000000..8e1b6f0
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
@@ -0,0 +1,1773 @@
+/*
+* Copyright (c) 2015 MediaTek Inc.
+* Author: PC Chen <pc.chen@mediatek.com>
+* Tiffany Lin <tiffany.lin@mediatek.com>
+*
+* This program is free software; you can redistribute it and/or modify
+* it under the terms of the GNU General Public License version 2 as
+* published by the Free Software Foundation.
+*
+* This program is distributed in the hope that it will be useful,
+* but WITHOUT ANY WARRANTY; without even the implied warranty of
+* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+* GNU General Public License for more details.
+*/
+
+#include <media/videobuf2-dma-contig.h>
+#include <media/videobuf2-core.h>
+#include <media/v4l2-event.h>
+#include <media/v4l2-mem2mem.h>
+
+#include "mtk_vcodec_drv.h"
+#include "mtk_vcodec_enc.h"
+#include "mtk_vcodec_intr.h"
+#include "mtk_vcodec_util.h"
+#include "venc_drv_if.h"
+
+static void mtk_venc_worker(struct work_struct *work);
+
+static struct mtk_video_fmt mtk_video_formats[] = {
+ {
+ .name = "4:2:0 3 Planes Y/Cb/Cr",
+ .fourcc = V4L2_PIX_FMT_YUV420,
+ .type = MTK_FMT_FRAME,
+ .num_planes = 3,
+ },
+ {
+ .name = "4:2:0 3 Planes Y/Cr/Cb",
+ .fourcc = V4L2_PIX_FMT_YVU420,
+ .type = MTK_FMT_FRAME,
+ .num_planes = 3,
+ },
+ {
+ .name = "4:2:0 2 Planes Y/CbCr",
+ .fourcc = V4L2_PIX_FMT_NV12,
+ .type = MTK_FMT_FRAME,
+ .num_planes = 2,
+ },
+ {
+ .name = "4:2:0 2 Planes Y/CrCb",
+ .fourcc = V4L2_PIX_FMT_NV21,
+ .type = MTK_FMT_FRAME,
+ .num_planes = 2,
+ },
+ {
+ .name = "4:2:0 3 none contiguous Planes Y/Cb/Cr",
+ .fourcc = V4L2_PIX_FMT_YUV420M,
+ .type = MTK_FMT_FRAME,
+ .num_planes = 3,
+ },
+ {
+ .name = "4:2:0 3 none contiguous Planes Y/Cr/Cb",
+ .fourcc = V4L2_PIX_FMT_YVU420M,
+ .type = MTK_FMT_FRAME,
+ .num_planes = 3,
+ },
+ {
+ .name = "4:2:0 2 none contiguous Planes Y/CbCr",
+ .fourcc = V4L2_PIX_FMT_NV12M,
+ .type = MTK_FMT_FRAME,
+ .num_planes = 2,
+ },
+ {
+ .name = "4:2:0 2 none contiguous Planes Y/CrCb",
+ .fourcc = V4L2_PIX_FMT_NV21M,
+ .type = MTK_FMT_FRAME,
+ .num_planes = 2,
+ },
+ {
+ .name = "H264 Encoded Stream",
+ .fourcc = V4L2_PIX_FMT_H264,
+ .type = MTK_FMT_ENC,
+ .num_planes = 1,
+ },
+ {
+ .name = "VP8 Encoded Stream",
+ .fourcc = V4L2_PIX_FMT_VP8,
+ .type = MTK_FMT_ENC,
+ .num_planes = 1,
+ },
+};
+
+#define NUM_FORMATS ARRAY_SIZE(mtk_video_formats)
+
+static struct mtk_vcodec_ctrl controls[] = {
+ {
+ .id = V4L2_CID_MPEG_VIDEO_BITRATE,
+ .type = V4L2_CTRL_TYPE_INTEGER,
+ .minimum = 1,
+ .maximum = (1 << 30) - 1,
+ .step = 1,
+ .default_value = 4000000,
+ },
+ {
+ .id = V4L2_CID_MPEG_VIDEO_B_FRAMES,
+ .type = V4L2_CTRL_TYPE_INTEGER,
+ .minimum = 0,
+ .maximum = 2,
+ .step = 1,
+ .default_value = 0,
+ },
+ {
+ .id = V4L2_CID_MPEG_VIDEO_FRAME_RC_ENABLE,
+ .type = V4L2_CTRL_TYPE_BOOLEAN,
+ .minimum = 0,
+ .maximum = 1,
+ .step = 1,
+ .default_value = 1,
+ },
+ {
+ .id = V4L2_CID_MPEG_VIDEO_H264_MAX_QP,
+ .type = V4L2_CTRL_TYPE_INTEGER,
+ .minimum = 0,
+ .maximum = 51,
+ .step = 1,
+ .default_value = 51,
+ },
+ {
+ .id = V4L2_CID_MPEG_VIDEO_HEADER_MODE,
+ .type = V4L2_CTRL_TYPE_MENU,
+ .minimum = V4L2_MPEG_VIDEO_HEADER_MODE_SEPARATE,
+ .maximum = V4L2_MPEG_VIDEO_HEADER_MODE_JOINED_WITH_1ST_FRAME,
+ .default_value = V4L2_MPEG_VIDEO_HEADER_MODE_SEPARATE,
+ .menu_skip_mask = 0,
+ },
+ {
+ .id = V4L2_CID_MPEG_VIDEO_MB_RC_ENABLE,
+ .type = V4L2_CTRL_TYPE_BOOLEAN,
+ .minimum = 0,
+ .maximum = 1,
+ .step = 1,
+ .default_value = 0,
+ },
+ {
+ .id = V4L2_CID_MPEG_VIDEO_H264_PROFILE,
+ .type = V4L2_CTRL_TYPE_MENU,
+ .minimum = V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE,
+ .maximum = V4L2_MPEG_VIDEO_H264_PROFILE_HIGH,
+ .default_value = V4L2_MPEG_VIDEO_H264_PROFILE_MAIN,
+ },
+ {
+ .id = V4L2_CID_MPEG_VIDEO_H264_LEVEL,
+ .type = V4L2_CTRL_TYPE_MENU,
+ .minimum = V4L2_MPEG_VIDEO_H264_LEVEL_1_0,
+ .maximum = V4L2_MPEG_VIDEO_H264_LEVEL_4_2,
+ .default_value = V4L2_MPEG_VIDEO_H264_LEVEL_4_0,
+ },
+ {
+ .id = V4L2_CID_MPEG_VIDEO_H264_I_PERIOD,
+ .type = V4L2_CTRL_TYPE_INTEGER,
+ .minimum = 0,
+ .maximum = (1 << 16) - 1,
+ .step = 1,
+ .default_value = 30,
+ },
+ {
+ .id = V4L2_CID_MPEG_VIDEO_GOP_SIZE,
+ .type = V4L2_CTRL_TYPE_INTEGER,
+ .minimum = 0,
+ .maximum = (1 << 16) - 1,
+ .step = 1,
+ .default_value = 30,
+ },
+ {
+ .id = V4L2_CID_MPEG_MFC51_VIDEO_FORCE_FRAME_TYPE,
+ .type = V4L2_CTRL_TYPE_MENU,
+ .minimum = V4L2_MPEG_MFC51_VIDEO_FORCE_FRAME_TYPE_DISABLED,
+ .maximum = V4L2_MPEG_MFC51_VIDEO_FORCE_FRAME_TYPE_NOT_CODED,
+ .default_value =
+ V4L2_MPEG_MFC51_VIDEO_FORCE_FRAME_TYPE_DISABLED,
+ .menu_skip_mask = 0,
+ },
+ {
+ .id = V4L2_CID_MPEG_MFC51_VIDEO_FRAME_SKIP_MODE,
+ .type = V4L2_CTRL_TYPE_MENU,
+ .minimum = V4L2_MPEG_MFC51_VIDEO_FRAME_SKIP_MODE_DISABLED,
+ .maximum = V4L2_MPEG_MFC51_VIDEO_FRAME_SKIP_MODE_BUF_LIMIT,
+ .default_value = V4L2_MPEG_MFC51_VIDEO_FRAME_SKIP_MODE_DISABLED,
+ .menu_skip_mask = 0,
+ },
+};
+
+#define NUM_CTRLS ARRAY_SIZE(controls)
+
+static const struct mtk_codec_framesizes mtk_venc_framesizes[] = {
+ {
+ .fourcc = V4L2_PIX_FMT_H264,
+ .stepwise = { 160, 1920, 16, 128, 1088, 16 },
+ },
+ {
+ .fourcc = V4L2_PIX_FMT_VP8,
+ .stepwise = { 160, 1920, 16, 128, 1088, 16 },
+ },
+};
+
+#define NUM_SUPPORTED_FRAMESIZE ARRAY_SIZE(mtk_venc_framesizes)
+
+static int vidioc_venc_s_ctrl(struct v4l2_ctrl *ctrl)
+{
+ struct mtk_vcodec_ctx *ctx = ctrl_to_ctx(ctrl);
+ struct mtk_vcodec_dev *dev = ctx->dev;
+ struct mtk_enc_params *p = &ctx->enc_params;
+ int ret = 0;
+
+ mtk_v4l2_debug(1, "[%d] id = %d/%d, val = %d", ctrl->id,
+ ctx->idx, ctrl->id - V4L2_CID_MPEG_BASE, ctrl->val);
+
+ switch (ctrl->id) {
+ case V4L2_CID_MPEG_VIDEO_BITRATE:
+
+ mtk_v4l2_debug(1, "V4L2_CID_MPEG_VIDEO_BITRATE val = %d",
+ ctrl->val);
+
+ mutex_lock(&ctx->encode_param_mutex);
+ p->bitrate = ctrl->val;
+ ctx->param_change |= MTK_ENCODE_PARAM_BITRATE;
+ mutex_unlock(&ctx->encode_param_mutex);
+ break;
+
+ case V4L2_CID_MPEG_VIDEO_B_FRAMES:
+ mtk_v4l2_debug(1, "V4L2_CID_MPEG_VIDEO_B_FRAMES val = %d",
+ ctrl->val);
+ p->num_b_frame = ctrl->val;
+ break;
+
+ case V4L2_CID_MPEG_VIDEO_FRAME_RC_ENABLE:
+ mtk_v4l2_debug(1, "V4L2_CID_MPEG_VIDEO_FRAME_RC_ENABLE val = %d",
+ ctrl->val);
+ p->rc_frame = ctrl->val;
+ break;
+
+ case V4L2_CID_MPEG_VIDEO_H264_MAX_QP:
+ mtk_v4l2_debug(1, "V4L2_CID_MPEG_VIDEO_H264_MAX_QP val = %d",
+ ctrl->val);
+ p->h264_max_qp = ctrl->val;
+ break;
+
+ case V4L2_CID_MPEG_VIDEO_HEADER_MODE:
+ mtk_v4l2_debug(1, "V4L2_CID_MPEG_VIDEO_HEADER_MODE val = %d",
+ ctrl->val);
+ p->seq_hdr_mode = ctrl->val;
+ break;
+
+ case V4L2_CID_MPEG_VIDEO_MB_RC_ENABLE:
+ mtk_v4l2_debug(1, "V4L2_CID_MPEG_VIDEO_MB_RC_ENABLE val = %d",
+ ctrl->val);
+ p->rc_mb = ctrl->val;
+ break;
+
+ case V4L2_CID_MPEG_VIDEO_H264_PROFILE:
+ mtk_v4l2_debug(1, "V4L2_CID_MPEG_VIDEO_H264_PROFILE val = %d",
+ ctrl->val);
+ p->h264_profile = ctrl->val;
+ break;
+
+ case V4L2_CID_MPEG_VIDEO_H264_LEVEL:
+ mtk_v4l2_debug(1, "V4L2_CID_MPEG_VIDEO_H264_LEVEL val = %d",
+ ctrl->val);
+ p->h264_level = ctrl->val;
+ break;
+ case V4L2_CID_MPEG_VIDEO_H264_I_PERIOD:
+ mtk_v4l2_debug(1, "V4L2_CID_MPEG_VIDEO_H264_I_PERIOD val = %d",
+ ctrl->val);
+ case V4L2_CID_MPEG_VIDEO_GOP_SIZE:
+ mtk_v4l2_debug(1, "V4L2_CID_MPEG_VIDEO_GOP_SIZE val = %d",
+ ctrl->val);
+
+ mutex_lock(&ctx->encode_param_mutex);
+ p->gop_size = ctrl->val;
+ ctx->param_change |= MTK_ENCODE_PARAM_INTRA_PERIOD;
+ mutex_unlock(&ctx->encode_param_mutex);
+ break;
+ case V4L2_CID_MPEG_MFC51_VIDEO_FORCE_FRAME_TYPE:
+ mtk_v4l2_debug(1, "V4L2_CID_MPEG_MFC51_VIDEO_FORCE_FRAME_TYPE val = %d",
+ ctrl->val);
+ if (ctrl->val ==
+ V4L2_MPEG_MFC51_VIDEO_FORCE_FRAME_TYPE_NOT_CODED) {
+ v4l2_err(&dev->v4l2_dev, "unsupported frame type %x\n",
+ ctrl->val);
+ ret = -EINVAL;
+ break;
+ }
+ mutex_lock(&ctx->encode_param_mutex);
+ if (ctrl->val == V4L2_MPEG_MFC51_VIDEO_FORCE_FRAME_TYPE_I_FRAME)
+ p->force_intra = 1;
+ else if (ctrl->val ==
+ V4L2_MPEG_MFC51_VIDEO_FORCE_FRAME_TYPE_DISABLED)
+ p->force_intra = 0;
+ /* always allow user to insert I frame */
+ ctrl->val = V4L2_MPEG_MFC51_VIDEO_FORCE_FRAME_TYPE_DISABLED;
+ ctx->param_change |= MTK_ENCODE_PARAM_FRAME_TYPE;
+ mutex_unlock(&ctx->encode_param_mutex);
+ break;
+
+ case V4L2_CID_MPEG_MFC51_VIDEO_FRAME_SKIP_MODE:
+ mtk_v4l2_debug(1, "V4L2_CID_MPEG_MFC51_VIDEO_FRAME_SKIP_MODE val = %d",
+ ctrl->val);
+
+ mutex_lock(&ctx->encode_param_mutex);
+ if (ctrl->val == V4L2_MPEG_MFC51_VIDEO_FRAME_SKIP_MODE_DISABLED)
+ p->skip_frame = 0;
+ else
+ p->skip_frame = 1;
+ /* always allow user to skip frame */
+ ctrl->val = V4L2_MPEG_MFC51_VIDEO_FRAME_SKIP_MODE_DISABLED;
+ ctx->param_change |= MTK_ENCODE_PARAM_SKIP_FRAME;
+ mutex_unlock(&ctx->encode_param_mutex);
+ break;
+
+ default:
+ mtk_v4l2_err("Invalid control, id=%d, val=%d\n",
+ ctrl->id, ctrl->val);
+ ret = -EINVAL;
+ break;
+ }
+
+ return ret;
+}
+
+static const struct v4l2_ctrl_ops mtk_vcodec_enc_ctrl_ops = {
+ .s_ctrl = vidioc_venc_s_ctrl,
+};
+
+static int vidioc_enum_fmt(struct file *file, struct v4l2_fmtdesc *f,
+ bool out)
+{
+ struct mtk_video_fmt *fmt;
+ int i, j = 0;
+
+ for (i = 0; i < NUM_FORMATS; ++i) {
+ if (out && mtk_video_formats[i].type != MTK_FMT_FRAME)
+ continue;
+ else if (!out && mtk_video_formats[i].type != MTK_FMT_ENC)
+ continue;
+
+ if (j == f->index) {
+ fmt = &mtk_video_formats[i];
+ strlcpy(f->description, fmt->name,
+ sizeof(f->description));
+ f->pixelformat = fmt->fourcc;
+ mtk_v4l2_debug(1, "f->index=%d i=%d fmt->name=%s",
+ f->index, i, fmt->name);
+ return 0;
+ }
+ ++j;
+ }
+
+ return -EINVAL;
+}
+
+static int vidioc_enum_framesizes(struct file *file, void *fh,
+ struct v4l2_frmsizeenum *fsize)
+{
+ int i = 0;
+
+ for (i = 0; i < NUM_SUPPORTED_FRAMESIZE; ++i) {
+ if (fsize->pixel_format != mtk_venc_framesizes[i].fourcc)
+ continue;
+
+ if (!fsize->index) {
+ fsize->type = V4L2_FRMSIZE_TYPE_STEPWISE;
+ fsize->stepwise = mtk_venc_framesizes[i].stepwise;
+ mtk_v4l2_debug(1, "%d %d %d %d %d %d",
+ fsize->stepwise.min_width,
+ fsize->stepwise.max_width,
+ fsize->stepwise.step_width,
+ fsize->stepwise.min_height,
+ fsize->stepwise.max_height,
+ fsize->stepwise.step_height);
+ return 0;
+ }
+ }
+
+ return -EINVAL;
+}
+
+static int vidioc_enum_fmt_vid_cap_mplane(struct file *file, void *pirv,
+ struct v4l2_fmtdesc *f)
+{
+ return vidioc_enum_fmt(file, f, false);
+}
+
+static int vidioc_enum_fmt_vid_out_mplane(struct file *file, void *prov,
+ struct v4l2_fmtdesc *f)
+{
+ return vidioc_enum_fmt(file, f, true);
+}
+
+static int vidioc_venc_streamon(struct file *file, void *priv,
+ enum v4l2_buf_type type)
+{
+ struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
+
+ return v4l2_m2m_streamon(file, ctx->m2m_ctx, type);
+}
+
+static int vidioc_venc_streamoff(struct file *file, void *priv,
+ enum v4l2_buf_type type)
+{
+ struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
+
+ return v4l2_m2m_streamoff(file, ctx->m2m_ctx, type);
+}
+
+static int vidioc_venc_reqbufs(struct file *file, void *priv,
+ struct v4l2_requestbuffers *reqbufs)
+{
+ struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
+
+ mtk_v4l2_debug(1, "[%d]-> type=%d count=%d",
+ ctx->idx, reqbufs->type, reqbufs->count);
+
+ return v4l2_m2m_reqbufs(file, ctx->m2m_ctx, reqbufs);
+}
+
+static int vidioc_venc_querybuf(struct file *file, void *priv,
+ struct v4l2_buffer *buf)
+{
+ struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
+
+ return v4l2_m2m_querybuf(file, ctx->m2m_ctx, buf);
+}
+
+static int vidioc_venc_qbuf(struct file *file, void *priv,
+ struct v4l2_buffer *buf)
+{
+ struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
+ int ret;
+#if MTK_V4L2_BENCHMARK
+ struct timeval begin, end;
+
+ do_gettimeofday(&begin);
+#endif
+
+ ret = v4l2_m2m_qbuf(file, ctx->m2m_ctx, buf);
+
+#if MTK_V4L2_BENCHMARK
+ do_gettimeofday(&end);
+
+ if (buf->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
+ ctx->total_qbuf_cap_cnt++;
+ ctx->total_qbuf_cap_time +=
+ ((end.tv_sec - begin.tv_sec) * 1000000 +
+ end.tv_usec - begin.tv_usec);
+ }
+
+ if (buf->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
+ ctx->total_qbuf_out_cnt++;
+ ctx->total_qbuf_out_time +=
+ ((end.tv_sec - begin.tv_sec) * 1000000 +
+ end.tv_usec - begin.tv_usec);
+ }
+
+#endif
+
+ return ret;
+}
+
+static int vidioc_venc_dqbuf(struct file *file, void *priv,
+ struct v4l2_buffer *buf)
+{
+ struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
+ int ret;
+#if MTK_V4L2_BENCHMARK
+ struct timeval begin, end;
+
+ do_gettimeofday(&begin);
+#endif
+
+ ret = v4l2_m2m_dqbuf(file, ctx->m2m_ctx, buf);
+#if MTK_V4L2_BENCHMARK
+
+ do_gettimeofday(&end);
+ if (buf->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
+ ctx->total_dqbuf_cap_cnt++;
+ ctx->total_dqbuf_cap_time +=
+ ((end.tv_sec - begin.tv_sec) * 1000000 +
+ end.tv_usec - begin.tv_usec);
+ }
+
+ if (buf->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
+ ctx->total_dqbuf_out_cnt++;
+ ctx->total_dqbuf_out_time +=
+ ((end.tv_sec - begin.tv_sec) * 1000000 +
+ end.tv_usec - begin.tv_usec);
+ }
+
+#endif
+ return ret;
+}
+static int vidioc_venc_expbuf(struct file *file, void *priv,
+ struct v4l2_exportbuffer *eb)
+{
+ struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
+ int ret;
+#if MTK_V4L2_BENCHMARK
+ struct timeval begin, end;
+
+ do_gettimeofday(&begin);
+#endif
+
+ ret = v4l2_m2m_expbuf(file, ctx->m2m_ctx, eb);
+
+#if MTK_V4L2_BENCHMARK
+ do_gettimeofday(&end);
+ ctx->total_expbuf_time +=
+ ((end.tv_sec - begin.tv_sec) * 1000000 +
+ end.tv_usec - begin.tv_usec);
+#endif
+ return ret;
+}
+
+static int vidioc_venc_querycap(struct file *file, void *priv,
+ struct v4l2_capability *cap)
+{
+ strncpy(cap->driver, MTK_VCODEC_ENC_NAME, sizeof(cap->driver) - 1);
+ cap->bus_info[0] = 0;
+ cap->version = KERNEL_VERSION(1, 0, 0);
+ /*
+ * This is only a mem-to-mem video device. The capture and output
+ * device capability flags are left only for backward compatibility
+ * and are scheduled for removal.
+ */
+ cap->capabilities = V4L2_CAP_VIDEO_M2M_MPLANE | V4L2_CAP_STREAMING |
+ V4L2_CAP_VIDEO_CAPTURE_MPLANE |
+ V4L2_CAP_VIDEO_OUTPUT_MPLANE;
+
+ return 0;
+}
+
+static int vidioc_venc_subscribe_event(struct v4l2_fh *fh,
+ const struct v4l2_event_subscription *sub)
+{
+ return v4l2_event_subscribe(fh, sub, 0, NULL);
+}
+
+static int vidioc_venc_s_parm(struct file *file, void *priv,
+ struct v4l2_streamparm *a)
+{
+ struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
+
+ if (a->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
+ mutex_lock(&ctx->encode_param_mutex);
+ ctx->enc_params.framerate_num =
+ a->parm.output.timeperframe.denominator;
+ ctx->enc_params.framerate_denom =
+ a->parm.output.timeperframe.numerator;
+ ctx->param_change |= MTK_ENCODE_PARAM_FRAMERATE;
+ mutex_unlock(&ctx->encode_param_mutex);
+ mtk_v4l2_debug(1, "framerate = %d/%d",
+ ctx->enc_params.framerate_num,
+ ctx->enc_params.framerate_denom);
+ } else {
+ mtk_v4l2_err("Non support param type %d",
+ a->type);
+ return -EINVAL;
+ }
+ return 0;
+}
+
+static struct mtk_q_data *mtk_venc_get_q_data(struct mtk_vcodec_ctx *ctx,
+ enum v4l2_buf_type type)
+{
+ switch (type) {
+ case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE:
+ case V4L2_BUF_TYPE_VIDEO_OUTPUT:
+ return &ctx->q_data[MTK_Q_DATA_SRC];
+ case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE:
+ case V4L2_BUF_TYPE_VIDEO_CAPTURE:
+ return &ctx->q_data[MTK_Q_DATA_DST];
+ default:
+ BUG();
+ }
+ return NULL;
+}
+
+static struct mtk_video_fmt *mtk_venc_find_format(struct v4l2_format *f)
+{
+ struct mtk_video_fmt *fmt;
+ unsigned int k;
+
+ for (k = 0; k < NUM_FORMATS; k++) {
+ fmt = &mtk_video_formats[k];
+ if (fmt->fourcc == f->fmt.pix.pixelformat)
+ return fmt;
+ }
+
+ return NULL;
+}
+
+static int vidioc_try_fmt(struct file *file, void *priv, struct v4l2_format *f)
+{
+ struct mtk_video_fmt *fmt;
+ char str[10];
+ struct v4l2_pix_format_mplane *pix_fmt_mp = &f->fmt.pix_mp;
+
+ mtk_vcodec_fmt2str(f->fmt.pix_mp.pixelformat, str);
+
+ if (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
+ fmt = mtk_venc_find_format(f);
+ if (!fmt) {
+ mtk_v4l2_err("failed to try output format %s\n", str);
+ return -EINVAL;
+ }
+ if (pix_fmt_mp->plane_fmt[0].sizeimage == 0) {
+ mtk_v4l2_err("must be set encoding output size %s\n",
+ str);
+ return -EINVAL;
+ }
+
+ pix_fmt_mp->plane_fmt[0].bytesperline =
+ pix_fmt_mp->plane_fmt[0].sizeimage;
+ } else if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
+ fmt = mtk_venc_find_format(f);
+ if (!fmt) {
+ mtk_v4l2_err("failed to try output format %s\n", str);
+ return -EINVAL;
+ }
+
+ if (fmt->num_planes != pix_fmt_mp->num_planes) {
+ mtk_v4l2_err("failed to try output format %d %d %s\n",
+ fmt->num_planes, pix_fmt_mp->num_planes,
+ str);
+ return -EINVAL;
+ }
+
+ v4l_bound_align_image(&pix_fmt_mp->width, 8, 1920, 1,
+ &pix_fmt_mp->height, 4, 1080, 1, 0);
+ } else {
+ pr_err("invalid buf type %d\n", f->type);
+ return -EINVAL;
+ }
+ return 0;
+}
+
+static void mtk_vcodec_enc_calc_src_size(struct mtk_vcodec_ctx *ctx)
+{
+ struct mtk_video_fmt *fmt = ctx->q_data[MTK_Q_DATA_SRC].fmt;
+ struct mtk_q_data *q_data = &ctx->q_data[MTK_Q_DATA_SRC];
+
+ ctx->q_data[MTK_Q_DATA_SRC].sizeimage[0] =
+ ((q_data->width + 15) / 16) *
+ ((q_data->height + 15) / 16) * 256;
+ ctx->q_data[MTK_Q_DATA_SRC].bytesperline[0] = ALIGN(q_data->width, 16);
+
+ if (fmt->num_planes == 2) {
+ ctx->q_data[MTK_Q_DATA_SRC].sizeimage[1] =
+ ((q_data->width + 15) / 16) *
+ ((q_data->height + 15) / 16) * 128;
+ ctx->q_data[MTK_Q_DATA_SRC].sizeimage[2] = 0;
+ ctx->q_data[MTK_Q_DATA_SRC].bytesperline[1] =
+ ALIGN(q_data->width, 16);
+ ctx->q_data[MTK_Q_DATA_SRC].bytesperline[2] = 0;
+ } else {
+ ctx->q_data[MTK_Q_DATA_SRC].sizeimage[1] =
+ ((q_data->width + 15) / 16) *
+ ((q_data->height + 15) / 16) * 64;
+ ctx->q_data[MTK_Q_DATA_SRC].sizeimage[2] =
+ ((q_data->width + 15) / 16) *
+ ((q_data->height + 15) / 16) * 64;
+ ctx->q_data[MTK_Q_DATA_SRC].bytesperline[1] =
+ ALIGN(q_data->width, 16) / 2;
+ ctx->q_data[MTK_Q_DATA_SRC].bytesperline[2] =
+ ALIGN(q_data->width, 16) / 2;
+ }
+}
+
+static int vidioc_venc_s_fmt(struct file *file, void *priv,
+ struct v4l2_format *f)
+{
+ struct mtk_vcodec_dev *dev = video_drvdata(file);
+ struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
+ struct v4l2_pix_format_mplane *pix_fmt_mp = &f->fmt.pix_mp;
+ struct vb2_queue *vq;
+ struct mtk_q_data *q_data;
+ int i, ret;
+
+ ret = vidioc_try_fmt(file, priv, f);
+ if (ret)
+ return ret;
+
+ vq = v4l2_m2m_get_vq(ctx->m2m_ctx, f->type);
+ if (!vq) {
+ v4l2_err(&dev->v4l2_dev, "fail to get vq\n");
+ return -EINVAL;
+ }
+
+ if (vb2_is_busy(vq)) {
+ v4l2_err(&dev->v4l2_dev, "queue busy\n");
+ return -EBUSY;
+ }
+
+ q_data = mtk_venc_get_q_data(ctx, f->type);
+ if (!q_data) {
+ v4l2_err(&dev->v4l2_dev, "fail to get q data\n");
+ return -EINVAL;
+ }
+
+ q_data->fmt = mtk_venc_find_format(f);
+ if (!q_data->fmt) {
+ v4l2_err(&dev->v4l2_dev, "q data null format\n");
+ return -EINVAL;
+ }
+
+ q_data->width = f->fmt.pix_mp.width;
+ q_data->height = f->fmt.pix_mp.height;
+ q_data->colorspace = f->fmt.pix_mp.colorspace;
+ q_data->field = f->fmt.pix_mp.field;
+
+ for (i = 0; i < f->fmt.pix_mp.num_planes; i++) {
+ struct v4l2_plane_pix_format *plane_fmt;
+
+ plane_fmt = &f->fmt.pix_mp.plane_fmt[i];
+ q_data->bytesperline[i] = plane_fmt->bytesperline;
+ q_data->sizeimage[i] = plane_fmt->sizeimage;
+ }
+
+ if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
+ q_data->width = f->fmt.pix_mp.width;
+ q_data->height = f->fmt.pix_mp.height;
+
+ mtk_vcodec_enc_calc_src_size(ctx);
+ pix_fmt_mp->plane_fmt[0].sizeimage = q_data->sizeimage[0];
+ pix_fmt_mp->plane_fmt[0].bytesperline =
+ ctx->q_data[MTK_Q_DATA_SRC].bytesperline[0];
+ pix_fmt_mp->plane_fmt[1].sizeimage = q_data->sizeimage[1];
+ pix_fmt_mp->plane_fmt[1].bytesperline =
+ ctx->q_data[MTK_Q_DATA_SRC].bytesperline[1];
+ pix_fmt_mp->plane_fmt[2].sizeimage = q_data->sizeimage[2];
+ pix_fmt_mp->plane_fmt[2].bytesperline =
+ ctx->q_data[MTK_Q_DATA_SRC].bytesperline[2];
+ }
+
+ mtk_v4l2_debug(1,
+ "[%d]: t=%d wxh=%dx%d fmt=%c%c%c%c sz=0x%x-%x-%x",
+ ctx->idx,
+ f->type,
+ q_data->width, q_data->height,
+ (f->fmt.pix_mp.pixelformat & 0xff),
+ (f->fmt.pix_mp.pixelformat >> 8) & 0xff,
+ (f->fmt.pix_mp.pixelformat >> 16) & 0xff,
+ (f->fmt.pix_mp.pixelformat >> 24) & 0xff,
+ q_data->sizeimage[0],
+ q_data->sizeimage[1],
+ q_data->sizeimage[2]);
+
+ return 0;
+}
+
+static int vidioc_venc_g_fmt(struct file *file, void *priv,
+ struct v4l2_format *f)
+{
+ struct v4l2_pix_format_mplane *pix = &f->fmt.pix_mp;
+ struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
+ struct vb2_queue *vq;
+ struct mtk_q_data *q_data;
+ int i;
+
+ vq = v4l2_m2m_get_vq(ctx->m2m_ctx, f->type);
+ if (!vq)
+ return -EINVAL;
+
+ q_data = mtk_venc_get_q_data(ctx, f->type);
+
+ pix->width = q_data->width;
+ pix->height = q_data->height;
+ pix->pixelformat = q_data->fmt->fourcc;
+ pix->field = q_data->field;
+ pix->colorspace = q_data->colorspace;
+ pix->num_planes = q_data->fmt->num_planes;
+
+ for (i = 0; i < pix->num_planes; i++) {
+ pix->plane_fmt[i].bytesperline = q_data->bytesperline[i];
+ pix->plane_fmt[i].sizeimage = q_data->sizeimage[i];
+ }
+
+ mtk_v4l2_debug(1,
+ "[%d]<- type=%d wxh=%dx%d fmt=%c%c%c%c sz[0]=0x%x sz[1]=0x%x",
+ ctx->idx, f->type,
+ pix->width, pix->height,
+ (pix->pixelformat & 0xff),
+ (pix->pixelformat >> 8) & 0xff,
+ (pix->pixelformat >> 16) & 0xff,
+ (pix->pixelformat >> 24) & 0xff,
+ pix->plane_fmt[0].sizeimage,
+ pix->plane_fmt[1].sizeimage);
+
+ return 0;
+}
+
+static int vidioc_venc_g_ctrl(struct file *file, void *fh,
+ struct v4l2_control *ctrl)
+{
+ int ret = 0;
+
+ switch (ctrl->id) {
+ case V4L2_CID_MIN_BUFFERS_FOR_CAPTURE:
+ case V4L2_CID_MIN_BUFFERS_FOR_OUTPUT:
+ ctrl->value = 1;
+ break;
+ default:
+ ret = -EINVAL;
+ break;
+ }
+ return ret;
+}
+
+static int vidioc_venc_s_crop(struct file *file, void *fh,
+ const struct v4l2_crop *a)
+{
+ struct mtk_vcodec_ctx *ctx = fh_to_ctx(fh);
+ struct mtk_q_data *q_data;
+
+ if (a->c.left || a->c.top)
+ return -EINVAL;
+
+ q_data = mtk_venc_get_q_data(ctx, a->type);
+ if (!q_data)
+ return -EINVAL;
+
+ if (a->c.width != q_data->width ||
+ a->c.height != q_data->height) {
+ mtk_v4l2_err("crop right %d bottom %d w %d h %d\n",
+ a->c.width, a->c.height, q_data->width,
+ q_data->height);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static int vidioc_venc_g_crop(struct file *file, void *fh,
+ struct v4l2_crop *a)
+{
+ struct mtk_vcodec_ctx *ctx = fh_to_ctx(fh);
+ struct mtk_q_data *q_data;
+
+ if (a->c.left || a->c.top)
+ return -EINVAL;
+
+ q_data = mtk_venc_get_q_data(ctx, a->type);
+ if (!q_data)
+ return -EINVAL;
+
+ a->c.width = q_data->width;
+ a->c.height = q_data->height;
+
+ return 0;
+}
+
+
+const struct v4l2_ioctl_ops mtk_venc_ioctl_ops = {
+ .vidioc_streamon = vidioc_venc_streamon,
+ .vidioc_streamoff = vidioc_venc_streamoff,
+
+ .vidioc_reqbufs = vidioc_venc_reqbufs,
+ .vidioc_querybuf = vidioc_venc_querybuf,
+ .vidioc_qbuf = vidioc_venc_qbuf,
+ .vidioc_expbuf = vidioc_venc_expbuf,
+ .vidioc_dqbuf = vidioc_venc_dqbuf,
+
+ .vidioc_querycap = vidioc_venc_querycap,
+ .vidioc_enum_fmt_vid_cap_mplane = vidioc_enum_fmt_vid_cap_mplane,
+ .vidioc_enum_fmt_vid_out_mplane = vidioc_enum_fmt_vid_out_mplane,
+ .vidioc_enum_framesizes = vidioc_enum_framesizes,
+
+ .vidioc_subscribe_event = vidioc_venc_subscribe_event,
+
+ .vidioc_s_parm = vidioc_venc_s_parm,
+
+ .vidioc_s_fmt_vid_cap_mplane = vidioc_venc_s_fmt,
+ .vidioc_s_fmt_vid_out_mplane = vidioc_venc_s_fmt,
+
+ .vidioc_g_fmt_vid_cap_mplane = vidioc_venc_g_fmt,
+ .vidioc_g_fmt_vid_out_mplane = vidioc_venc_g_fmt,
+
+ .vidioc_g_ctrl = vidioc_venc_g_ctrl,
+
+ .vidioc_s_crop = vidioc_venc_s_crop,
+ .vidioc_g_crop = vidioc_venc_g_crop,
+
+};
+
+static int vb2ops_venc_queue_setup(struct vb2_queue *vq,
+ const void *parg,
+ unsigned int *nbuffers,
+ unsigned int *nplanes,
+ unsigned int sizes[], void *alloc_ctxs[])
+{
+ struct mtk_vcodec_ctx *ctx = vb2_get_drv_priv(vq);
+ struct mtk_q_data *q_data;
+
+ q_data = mtk_venc_get_q_data(ctx, vq->type);
+
+ if (*nbuffers < 1)
+ *nbuffers = 1;
+ if (*nbuffers > MTK_VIDEO_MAX_FRAME)
+ *nbuffers = MTK_VIDEO_MAX_FRAME;
+
+ *nplanes = q_data->fmt->num_planes;
+
+ if (vq->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
+ unsigned int i;
+
+ for (i = 0; i < *nplanes; i++) {
+ sizes[i] = q_data->sizeimage[i];
+ alloc_ctxs[i] = ctx->dev->alloc_ctx;
+ }
+ } else if (vq->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
+ sizes[0] = q_data->sizeimage[0];
+ alloc_ctxs[0] = ctx->dev->alloc_ctx;
+ } else {
+ BUG();
+ }
+
+ mtk_v4l2_debug(2,
+ "[%d]get %d buffer(s) of size 0x%x each",
+ ctx->idx, *nbuffers, sizes[0]);
+
+ return 0;
+}
+
+static int vb2ops_venc_buf_prepare(struct vb2_buffer *vb)
+{
+ struct mtk_vcodec_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue);
+ struct mtk_q_data *q_data;
+ int i;
+
+ q_data = mtk_venc_get_q_data(ctx, vb->vb2_queue->type);
+
+ for (i = 0; i < q_data->fmt->num_planes; i++) {
+ if (vb2_plane_size(vb, i) < q_data->sizeimage[i]) {
+ mtk_v4l2_debug(2,
+ "data will not fit into plane %d (%lu < %d)",
+ i, vb2_plane_size(vb, i),
+ q_data->sizeimage[i]);
+ return -EINVAL;
+ }
+ }
+
+ return 0;
+}
+
+static void vb2ops_venc_buf_queue(struct vb2_buffer *vb)
+{
+ struct mtk_vcodec_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue);
+ struct vb2_v4l2_buffer *v4l2_vb = to_vb2_v4l2_buffer(vb);
+ struct mtk_video_enc_buf *buf =
+ container_of(v4l2_vb, struct mtk_video_enc_buf, b);
+
+ mutex_lock(&ctx->encode_param_mutex);
+ if ((vb->vb2_queue->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) &&
+ (ctx->param_change != MTK_ENCODE_PARAM_NONE)) {
+ mtk_v4l2_debug(1,
+ "[%d] Before id=%d encode parameter change %x",
+ ctx->idx, vb->index,
+ ctx->param_change);
+ buf->param_change = ctx->param_change;
+ if (buf->param_change & MTK_ENCODE_PARAM_BITRATE) {
+ buf->enc_params.bitrate = ctx->enc_params.bitrate;
+ mtk_v4l2_debug(1, "change param br=%d",
+ buf->enc_params.bitrate);
+ }
+ if (ctx->param_change & MTK_ENCODE_PARAM_FRAMERATE) {
+ buf->enc_params.framerate_num =
+ ctx->enc_params.framerate_num;
+ buf->enc_params.framerate_denom =
+ ctx->enc_params.framerate_denom;
+ mtk_v4l2_debug(1, "change param fr=%d",
+ buf->enc_params.framerate_num /
+ buf->enc_params.framerate_denom);
+ }
+ if (ctx->param_change & MTK_ENCODE_PARAM_INTRA_PERIOD) {
+ buf->enc_params.gop_size = ctx->enc_params.gop_size;
+ mtk_v4l2_debug(1, "change param intra period=%d",
+ buf->enc_params.gop_size);
+ }
+ if (ctx->param_change & MTK_ENCODE_PARAM_FRAME_TYPE) {
+ buf->enc_params.force_intra =
+ ctx->enc_params.force_intra;
+ mtk_v4l2_debug(1, "change param force I=%d",
+ buf->enc_params.force_intra);
+ }
+ if (ctx->param_change & MTK_ENCODE_PARAM_SKIP_FRAME) {
+ buf->enc_params.skip_frame =
+ ctx->enc_params.skip_frame;
+ mtk_v4l2_debug(1, "change param skip frame=%d",
+ buf->enc_params.skip_frame);
+ }
+ ctx->param_change = MTK_ENCODE_PARAM_NONE;
+ }
+ mutex_unlock(&ctx->encode_param_mutex);
+ v4l2_m2m_buf_queue(ctx->m2m_ctx, to_vb2_v4l2_buffer(vb));
+}
+
+static void mtk_venc_set_param(struct mtk_vcodec_ctx *ctx, void *param)
+{
+ struct venc_enc_prm *p = (struct venc_enc_prm *)param;
+ struct mtk_q_data *q_data_src = &ctx->q_data[MTK_Q_DATA_SRC];
+ struct mtk_enc_params *enc_params = &ctx->enc_params;
+ unsigned int frame_rate;
+
+ mutex_lock(&ctx->encode_param_mutex);
+
+ frame_rate = enc_params->framerate_num / enc_params->framerate_denom;
+
+ switch (q_data_src->fmt->fourcc) {
+ case V4L2_PIX_FMT_YUV420:
+ case V4L2_PIX_FMT_YUV420M:
+ p->input_fourcc = VENC_YUV_FORMAT_420;
+ break;
+ case V4L2_PIX_FMT_YVU420:
+ case V4L2_PIX_FMT_YVU420M:
+ p->input_fourcc = VENC_YUV_FORMAT_YV12;
+ break;
+ case V4L2_PIX_FMT_NV12:
+ case V4L2_PIX_FMT_NV12M:
+ p->input_fourcc = VENC_YUV_FORMAT_NV12;
+ break;
+ case V4L2_PIX_FMT_NV21:
+ case V4L2_PIX_FMT_NV21M:
+ p->input_fourcc = VENC_YUV_FORMAT_NV21;
+ break;
+ }
+ p->h264_profile = enc_params->h264_profile;
+ p->h264_level = enc_params->h264_level;
+ p->width = q_data_src->width;
+ p->height = q_data_src->height;
+ p->buf_width = q_data_src->bytesperline[0];
+ p->buf_height = ((q_data_src->height + 0xf) & (~0xf));
+ p->frm_rate = frame_rate;
+ p->intra_period = enc_params->gop_size;
+ p->bitrate = enc_params->bitrate;
+
+ ctx->param_change = MTK_ENCODE_PARAM_NONE;
+ mutex_unlock(&ctx->encode_param_mutex);
+
+ mtk_v4l2_debug(1,
+ "fmt 0x%x P/L %d/%d w/h %d/%d buf %d/%d fps/bps %d/%d gop %d",
+ p->input_fourcc, p->h264_profile, p->h264_level, p->width,
+ p->height, p->buf_width, p->buf_height, p->frm_rate,
+ p->bitrate, p->intra_period);
+}
+
+static int vb2ops_venc_start_streaming(struct vb2_queue *q, unsigned int count)
+{
+ struct mtk_vcodec_ctx *ctx = vb2_get_drv_priv(q);
+ struct v4l2_device *v4l2_dev = &ctx->dev->v4l2_dev;
+ int ret;
+#if MTK_V4L2_BENCHMARK
+ struct timeval begin, end;
+
+ do_gettimeofday(&begin);
+#endif
+
+ if (!(vb2_start_streaming_called(&ctx->m2m_ctx->out_q_ctx.q) &
+ vb2_start_streaming_called(&ctx->m2m_ctx->cap_q_ctx.q))) {
+ mtk_v4l2_debug(1, "[%d]-> out=%d cap=%d",
+ ctx->idx,
+ vb2_start_streaming_called(&ctx->m2m_ctx->out_q_ctx.q),
+ vb2_start_streaming_called(&ctx->m2m_ctx->cap_q_ctx.q));
+ return 0;
+ }
+
+ if ((ctx->state & (MTK_STATE_ERROR | MTK_STATE_ABORT)))
+ return -EINVAL;
+
+ if (ctx->state == MTK_STATE_FREE) {
+ ret = venc_if_create(ctx,
+ ctx->q_data[MTK_Q_DATA_DST].fmt->fourcc,
+ &ctx->h_enc);
+
+ if (ret != 0) {
+ ctx->state |= MTK_STATE_ERROR;
+ v4l2_err(v4l2_dev, "invalid codec type=%x\n",
+ ctx->q_data[MTK_Q_DATA_DST].fmt->fourcc);
+ v4l2_err(v4l2_dev, "venc_if_create failed=%d\n", ret);
+ return -EINVAL;
+ }
+
+ if (ctx->q_data[MTK_Q_DATA_DST].fmt->fourcc ==
+ V4L2_PIX_FMT_H264)
+ ctx->hdr = 1;
+
+ ctx->state |= MTK_STATE_CREATE;
+ }
+
+ if ((ctx->state & MTK_STATE_CREATE) && !(ctx->state & MTK_STATE_INIT)) {
+ ret = venc_if_init(ctx->h_enc);
+ if (ret != 0) {
+ ctx->state |= MTK_STATE_ERROR;
+ v4l2_err(v4l2_dev, "venc_if_init failed=%d\n", ret);
+ return -EINVAL;
+ }
+ INIT_WORK(&ctx->encode_work, mtk_venc_worker);
+ ctx->state |= MTK_STATE_INIT;
+ }
+
+ if ((ctx->state & MTK_STATE_INIT) && !(ctx->state & MTK_STATE_CONFIG)) {
+ struct venc_enc_prm param;
+
+ mtk_venc_set_param(ctx, ¶m);
+ ret = venc_if_set_param(ctx->h_enc,
+ VENC_SET_PARAM_ENC, ¶m);
+ if (ret != 0) {
+ ctx->state |= MTK_STATE_ERROR;
+ v4l2_err(v4l2_dev, "venc_if_set_param failed=%d\n",
+ ret);
+ return -EINVAL;
+ }
+ if (ctx->hdr) {
+ if (ctx->enc_params.seq_hdr_mode ==
+ V4L2_MPEG_VIDEO_HEADER_MODE_SEPARATE) {
+ ctx->state |= MTK_STATE_CONFIG;
+ } else {
+ ret = venc_if_set_param(ctx->h_enc,
+ VENC_SET_PARAM_PREPEND_HEADER,
+ 0);
+ if (ret != 0) {
+ ctx->state |= MTK_STATE_ERROR;
+ v4l2_err(v4l2_dev,
+ "venc_if_set_param failed=%d\n",
+ ret);
+ return -EINVAL;
+ }
+ ctx->state |= (MTK_STATE_CONFIG |
+ MTK_STATE_HEADER);
+ }
+ } else {
+ ctx->state |= (MTK_STATE_CONFIG | MTK_STATE_HEADER);
+ }
+ }
+
+#if MTK_V4L2_BENCHMARK
+ do_gettimeofday(&end);
+ ctx->total_enc_dec_init_time =
+ ((end.tv_sec - begin.tv_sec) * 1000000 +
+ end.tv_usec - begin.tv_usec);
+
+#endif
+ if ((ctx->state & MTK_STATE_HEADER))
+ return 0;
+
+ if (!(ctx->state & MTK_STATE_CONFIG)) {
+ ctx->state |= MTK_STATE_ERROR;
+ mtk_v4l2_err("not configured, state=0x%x\n", ctx->state);
+ return -EINVAL;
+ }
+ return 0;
+}
+
+static void vb2ops_venc_stop_streaming(struct vb2_queue *q)
+{
+ struct mtk_vcodec_ctx *ctx = vb2_get_drv_priv(q);
+ struct v4l2_device *v4l2_dev = &ctx->dev->v4l2_dev;
+ struct vb2_buffer *src_buf, *dst_buf;
+ int retry;
+ int ret;
+
+ mtk_v4l2_debug(2, "[%d]-> type=%d", ctx->idx, q->type);
+
+ retry = 0;
+ while ((ctx->state & MTK_STATE_RUNNING) && (retry < 10)) {
+ mtk_vcodec_clean_ctx_int_flags(ctx);
+ ctx->state |= MTK_STATE_ABORT;
+ ret = mtk_vcodec_wait_for_done_ctx(ctx,
+ MTK_INST_WORK_THREAD_ABORT_DONE,
+ 200, true);
+ mtk_v4l2_debug(1, "[%d] (%d) state=%d, retry=%d",
+ ctx->idx, q->type, ctx->state, retry);
+ retry++;
+ }
+
+ if (q->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
+ while ((dst_buf = v4l2_m2m_dst_buf_remove(ctx->m2m_ctx))) {
+ dst_buf->planes[0].bytesused = 0;
+ v4l2_m2m_buf_done(to_vb2_v4l2_buffer(dst_buf), VB2_BUF_STATE_DONE);
+ }
+ } else {
+ while ((src_buf = v4l2_m2m_src_buf_remove(ctx->m2m_ctx)))
+ v4l2_m2m_buf_done(to_vb2_v4l2_buffer(src_buf), VB2_BUF_STATE_DONE);
+ }
+
+ /* todo : stop HW encoder and flush all buffers*/
+ if ((q->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE &&
+ vb2_is_streaming(&ctx->m2m_ctx->out_q_ctx.q)) ||
+ (q->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE &&
+ vb2_is_streaming(&ctx->m2m_ctx->cap_q_ctx.q))) {
+ mtk_v4l2_debug(1, "[%d]-> q type %d out=%d cap=%d",
+ ctx->idx, q->type,
+ vb2_is_streaming(&ctx->m2m_ctx->out_q_ctx.q),
+ vb2_is_streaming(&ctx->m2m_ctx->cap_q_ctx.q));
+ return;
+ }
+
+ ctx->state |= MTK_STATE_FINISH;
+
+ if ((ctx->state & MTK_STATE_INIT) && !(ctx->state & MTK_STATE_DEINIT)) {
+ ret = venc_if_deinit(ctx->h_enc);
+ if (ret != 0) {
+ ctx->state |= MTK_STATE_ERROR;
+ v4l2_err(v4l2_dev, "venc_if_deinit failed=%d\n", ret);
+ return;
+ }
+ ctx->state |= MTK_STATE_DEINIT;
+ }
+
+ if ((ctx->state & MTK_STATE_CREATE)) {
+ ret = 0;
+ switch (ctx->q_data[MTK_Q_DATA_DST].fmt->fourcc) {
+ case V4L2_PIX_FMT_H264:
+ case V4L2_PIX_FMT_VP8:
+ ret = venc_if_release(ctx->h_enc);
+ ctx->h_enc = 0;
+ break;
+ default:
+ ctx->state |= MTK_STATE_ERROR;
+ v4l2_err(v4l2_dev, "invalid codec type=%x\n",
+ ctx->q_data[MTK_Q_DATA_DST].fmt->fourcc);
+ break;
+ }
+ if (ret != 0) {
+ ctx->state |= MTK_STATE_ERROR;
+ v4l2_err(v4l2_dev, "venc_if_release failed=%d\n", ret);
+ return;
+ }
+ }
+
+ mtk_v4l2_debug(2, "[%d]-> state=0x%x", ctx->idx, ctx->state);
+
+ ctx->state = MTK_STATE_FREE;
+}
+
+static struct vb2_ops mtk_venc_vb2_ops = {
+ .queue_setup = vb2ops_venc_queue_setup,
+ .buf_prepare = vb2ops_venc_buf_prepare,
+ .buf_queue = vb2ops_venc_buf_queue,
+ .wait_prepare = vb2_ops_wait_prepare,
+ .wait_finish = vb2_ops_wait_finish,
+ .start_streaming = vb2ops_venc_start_streaming,
+ .stop_streaming = vb2ops_venc_stop_streaming,
+};
+
+static int mtk_venc_encode_header(void *priv)
+{
+ struct mtk_vcodec_ctx *ctx = priv;
+ struct v4l2_device *v4l2_dev = &ctx->dev->v4l2_dev;
+ int ret;
+ struct vb2_buffer *dst_buf;
+ struct mtk_vcodec_mem bs_buf;
+ struct venc_done_result enc_result;
+ int i;
+
+ dst_buf = v4l2_m2m_dst_buf_remove(ctx->m2m_ctx);
+ if (!dst_buf) {
+ ctx->state |= MTK_STATE_ERROR;
+ mtk_v4l2_debug(1, "No dst buffer");
+ return -EINVAL;
+ }
+
+ bs_buf.va = vb2_plane_vaddr(dst_buf, 0);
+ bs_buf.dma_addr = vb2_dma_contig_plane_dma_addr(dst_buf, 0);
+ bs_buf.size = (unsigned int)dst_buf->planes[0].length;
+
+ mtk_v4l2_debug(1,
+ "buf idx=%d va=0x%p dma_addr=0x%llx size=0x%lx",
+ dst_buf->index, bs_buf.va,
+ (u64)bs_buf.dma_addr, bs_buf.size);
+
+ ret = venc_if_encode(ctx->h_enc,
+ VENC_START_OPT_ENCODE_SEQUENCE_HEADER,
+ 0, &bs_buf, &enc_result);
+
+ if (ret != 0) {
+ dst_buf->planes[0].bytesused = 0;
+ ctx->state |= MTK_STATE_ERROR;
+ v4l2_m2m_buf_done(to_vb2_v4l2_buffer(dst_buf), VB2_BUF_STATE_ERROR);
+ v4l2_err(v4l2_dev, "venc_if_encode failed=%d", ret);
+ return -EINVAL;
+ }
+
+ ctx->state |= MTK_STATE_HEADER;
+ dst_buf->planes[0].bytesused = enc_result.bs_size;
+
+ mtk_v4l2_debug(1, "venc_if_encode header len=%d",
+ enc_result.bs_size);
+ for (i = 0; i < enc_result.bs_size; i++) {
+ unsigned char *p = (unsigned char *)bs_buf.va;
+
+ mtk_v4l2_debug(1, "buf[%d]=0x%2x", i, p[i]);
+ }
+
+ v4l2_m2m_buf_done(to_vb2_v4l2_buffer(dst_buf), VB2_BUF_STATE_DONE);
+
+ return 0;
+}
+
+static int mtk_venc_param_change(struct mtk_vcodec_ctx *ctx, void *priv)
+{
+ struct vb2_buffer *vb = priv;
+ struct vb2_v4l2_buffer *v4l2_vb = to_vb2_v4l2_buffer(vb);
+ struct mtk_video_enc_buf *buf =
+ container_of(v4l2_vb, struct mtk_video_enc_buf, b);
+ int ret = 0;
+
+ if (buf->param_change == MTK_ENCODE_PARAM_NONE)
+ return 0;
+
+ mtk_v4l2_debug(1, "encode parameters change id=%d", vb->index);
+ if (buf->param_change & MTK_ENCODE_PARAM_BITRATE) {
+ struct venc_enc_prm enc_prm;
+
+ enc_prm.bitrate = buf->enc_params.bitrate;
+ mtk_v4l2_debug(1, "change param br=%d",
+ enc_prm.bitrate);
+ ret |= venc_if_set_param(ctx->h_enc,
+ VENC_SET_PARAM_ADJUST_BITRATE,
+ &enc_prm);
+ }
+ if (buf->param_change & MTK_ENCODE_PARAM_FRAMERATE) {
+ struct venc_enc_prm enc_prm;
+
+ enc_prm.frm_rate = buf->enc_params.framerate_num /
+ buf->enc_params.framerate_denom;
+ mtk_v4l2_debug(1, "change param fr=%d",
+ enc_prm.frm_rate);
+ ret |= venc_if_set_param(ctx->h_enc,
+ VENC_SET_PARAM_ADJUST_FRAMERATE,
+ &enc_prm);
+ }
+ if (buf->param_change & MTK_ENCODE_PARAM_INTRA_PERIOD) {
+ mtk_v4l2_debug(1, "change param intra period=%d",
+ buf->enc_params.gop_size);
+ ret |= venc_if_set_param(ctx->h_enc,
+ VENC_SET_PARAM_I_FRAME_INTERVAL,
+ &buf->enc_params.gop_size);
+ }
+ if (buf->param_change & MTK_ENCODE_PARAM_FRAME_TYPE) {
+ mtk_v4l2_debug(1, "change param force I=%d",
+ buf->enc_params.force_intra);
+ if (buf->enc_params.force_intra)
+ ret |= venc_if_set_param(ctx->h_enc,
+ VENC_SET_PARAM_FORCE_INTRA,
+ 0);
+ }
+ if (buf->param_change & MTK_ENCODE_PARAM_SKIP_FRAME) {
+ mtk_v4l2_debug(1, "change param skip frame=%d",
+ buf->enc_params.skip_frame);
+ if (buf->enc_params.skip_frame)
+ ret |= venc_if_set_param(ctx->h_enc,
+ VENC_SET_PARAM_SKIP_FRAME,
+ 0);
+ }
+ buf->param_change = MTK_ENCODE_PARAM_NONE;
+
+ if (ret != 0) {
+ ctx->state |= MTK_STATE_ERROR;
+ mtk_v4l2_err("venc_if_set_param failed=%d\n", ret);
+ return -1;
+ }
+
+ return 0;
+}
+
+static void mtk_venc_worker(struct work_struct *work)
+{
+ struct mtk_vcodec_ctx *ctx = container_of(work, struct mtk_vcodec_ctx,
+ encode_work);
+ struct vb2_buffer *src_buf, *dst_buf;
+ struct vb2_v4l2_buffer *v4l2_vb;
+ struct venc_frm_buf frm_buf;
+ struct mtk_vcodec_mem bs_buf;
+ struct venc_done_result enc_result;
+ int ret;
+
+#if MTK_V4L2_BENCHMARK
+ struct timeval begin, end;
+ struct timeval begin1, end1;
+
+ do_gettimeofday(&begin);
+#endif
+
+ ctx->state |= MTK_STATE_RUNNING;
+
+ if (!(ctx->state & MTK_STATE_HEADER)) {
+#if MTK_V4L2_BENCHMARK
+ do_gettimeofday(&begin1);
+#endif
+ mtk_venc_encode_header(ctx);
+#if MTK_V4L2_BENCHMARK
+ do_gettimeofday(&end1);
+ ctx->total_enc_hdr_time +=
+ ((end1.tv_sec - begin1.tv_sec) * 1000000 +
+ end1.tv_usec - begin1.tv_usec);
+#endif
+
+ ctx->state &= ~MTK_STATE_RUNNING;
+ v4l2_m2m_job_finish(ctx->dev->m2m_dev_enc, ctx->m2m_ctx);
+ return;
+ }
+
+ if (ctx->state & MTK_STATE_ABORT) {
+ ctx->state &= ~MTK_STATE_RUNNING;
+ v4l2_m2m_job_finish(ctx->dev->m2m_dev_enc, ctx->m2m_ctx);
+ mtk_v4l2_debug(1, "[%d] [MTK_INST_ABORT]", ctx->idx);
+ ctx->int_cond = 1;
+ ctx->int_type = MTK_INST_WORK_THREAD_ABORT_DONE;
+ wake_up_interruptible(&ctx->queue);
+ return;
+ }
+
+ src_buf = v4l2_m2m_next_src_buf(ctx->m2m_ctx);
+ if (!src_buf)
+ return;
+
+ mtk_venc_param_change(ctx, src_buf);
+
+ dst_buf = v4l2_m2m_dst_buf_remove(ctx->m2m_ctx);
+ if (!dst_buf)
+ return;
+
+ frm_buf.fb_addr.va = vb2_plane_vaddr(src_buf, 0);
+ frm_buf.fb_addr.dma_addr = vb2_dma_contig_plane_dma_addr(src_buf, 0);
+ frm_buf.fb_addr.size = (unsigned int)src_buf->planes[0].length;
+ frm_buf.fb_addr1.va = vb2_plane_vaddr(src_buf, 1);
+ frm_buf.fb_addr1.dma_addr = vb2_dma_contig_plane_dma_addr(src_buf, 1);
+ frm_buf.fb_addr1.size = (unsigned int)src_buf->planes[1].length;
+ if (src_buf->num_planes == 3) {
+ frm_buf.fb_addr2.va = vb2_plane_vaddr(src_buf, 2);
+ frm_buf.fb_addr2.dma_addr =
+ vb2_dma_contig_plane_dma_addr(src_buf, 2);
+ frm_buf.fb_addr2.size =
+ (unsigned int)src_buf->planes[2].length;
+ }
+ bs_buf.va = vb2_plane_vaddr(dst_buf, 0);
+ bs_buf.dma_addr = vb2_dma_contig_plane_dma_addr(dst_buf, 0);
+ bs_buf.size = (unsigned int)dst_buf->planes[0].length;
+
+ mtk_v4l2_debug(1,
+ "Framebuf VA=%p PA=%llx Size=0x%lx;VA=%p PA=0x%llx Size=0x%lx;VA=%p PA=0x%llx Size=0x%lx",
+ frm_buf.fb_addr.va,
+ (u64)frm_buf.fb_addr.dma_addr,
+ frm_buf.fb_addr.size,
+ frm_buf.fb_addr1.va,
+ (u64)frm_buf.fb_addr1.dma_addr,
+ frm_buf.fb_addr1.size,
+ frm_buf.fb_addr2.va,
+ (u64)frm_buf.fb_addr2.dma_addr,
+ frm_buf.fb_addr2.size);
+
+ ret = venc_if_encode(ctx->h_enc, VENC_START_OPT_ENCODE_FRAME,
+ &frm_buf, &bs_buf, &enc_result);
+
+ switch (enc_result.msg) {
+ case VENC_MESSAGE_OK:
+ src_buf = v4l2_m2m_src_buf_remove(ctx->m2m_ctx);
+ v4l2_m2m_buf_done(to_vb2_v4l2_buffer(src_buf), VB2_BUF_STATE_DONE);
+ dst_buf->planes[0].bytesused = enc_result.bs_size;
+ break;
+ default:
+ src_buf = v4l2_m2m_src_buf_remove(ctx->m2m_ctx);
+ v4l2_m2m_buf_done(to_vb2_v4l2_buffer(src_buf), VB2_BUF_STATE_ERROR);
+ dst_buf->planes[0].bytesused = 0;
+ break;
+ }
+ if (enc_result.is_key_frm) {
+ v4l2_vb = to_vb2_v4l2_buffer(dst_buf);
+ v4l2_vb->flags |= V4L2_BUF_FLAG_KEYFRAME;
+ }
+
+ if (ret != 0) {
+ ctx->state |= MTK_STATE_ERROR;
+ v4l2_m2m_buf_done(to_vb2_v4l2_buffer(dst_buf), VB2_BUF_STATE_ERROR);
+ mtk_v4l2_err("venc_if_encode failed=%d", ret);
+ } else {
+ v4l2_m2m_buf_done(to_vb2_v4l2_buffer(dst_buf), VB2_BUF_STATE_DONE);
+ mtk_v4l2_debug(1, "venc_if_encode bs size=%d",
+ enc_result.bs_size);
+ }
+
+#if MTK_V4L2_BENCHMARK
+ do_gettimeofday(&end);
+ ctx->total_enc_dec_cnt++;
+ ctx->total_enc_dec_time +=
+ ((end.tv_sec - begin.tv_sec) * 1000000 +
+ end.tv_usec - begin.tv_usec);
+#endif
+
+ ctx->state &= ~MTK_STATE_RUNNING;
+ v4l2_m2m_job_finish(ctx->dev->m2m_dev_enc, ctx->m2m_ctx);
+
+ mtk_v4l2_debug(1, "<=== src_buf[%d] dst_buf[%d] venc_if_encode ret=%d Size=%u===>",
+ src_buf->index, dst_buf->index, ret,
+ enc_result.bs_size);
+
+}
+
+static void m2mops_venc_device_run(void *priv)
+{
+ struct mtk_vcodec_ctx *ctx = priv;
+#ifdef USE_ENCODE_THREAD
+ queue_work(ctx->dev->encode_workqueue, &ctx->encode_work);
+#else
+ mtk_venc_worker(&ctx->encode_work);
+#endif
+}
+
+static int m2mops_venc_job_ready(void *m2m_priv)
+{
+ struct mtk_vcodec_ctx *ctx = m2m_priv;
+
+ if (!v4l2_m2m_num_dst_bufs_ready(ctx->m2m_ctx)) {
+ mtk_v4l2_debug(3,
+ "[%d]Not ready: not enough video dst buffers.",
+ ctx->idx);
+ return 0;
+ }
+
+ if (!v4l2_m2m_num_src_bufs_ready(ctx->m2m_ctx)) {
+ if ((ctx->state & MTK_STATE_HEADER)) {
+ mtk_v4l2_debug(3,
+ "[%d]Not ready: not enough video src buffers.",
+ ctx->idx);
+ return 0;
+ }
+ }
+
+ if ((ctx->state & (MTK_STATE_ERROR | MTK_STATE_ABORT))) {
+ mtk_v4l2_debug(3,
+ "[%d]Not ready: state=0x%x.",
+ ctx->idx, ctx->state);
+ return 0;
+ }
+
+ if (!(ctx->state & MTK_STATE_CONFIG)) {
+ mtk_v4l2_debug(3,
+ "[%d]Not ready: state=0x%x.",
+ ctx->idx, ctx->state);
+ return 0;
+ }
+
+ mtk_v4l2_debug(3, "[%d]ready!", ctx->idx);
+
+ return 1;
+}
+
+static void m2mops_venc_job_abort(void *priv)
+{
+ struct mtk_vcodec_ctx *ctx = priv;
+
+ mtk_v4l2_debug(3, "[%d]type=%d", ctx->idx, ctx->type);
+
+ ctx->aborting = 1;
+ ctx->state |= MTK_STATE_ABORT;
+
+ v4l2_m2m_job_finish(ctx->dev->m2m_dev_enc, ctx->m2m_ctx);
+}
+
+static void m2mops_venc_lock(void *m2m_priv)
+{
+ struct mtk_vcodec_ctx *ctx = m2m_priv;
+
+ mutex_lock(&ctx->dev->dev_mutex);
+}
+
+static void m2mops_venc_unlock(void *m2m_priv)
+{
+ struct mtk_vcodec_ctx *ctx = m2m_priv;
+
+ mutex_unlock(&ctx->dev->dev_mutex);
+}
+
+const struct v4l2_m2m_ops mtk_venc_m2m_ops = {
+ .device_run = m2mops_venc_device_run,
+ .job_ready = m2mops_venc_job_ready,
+ .job_abort = m2mops_venc_job_abort,
+ .lock = m2mops_venc_lock,
+ .unlock = m2mops_venc_unlock,
+};
+
+#define IS_MTK_VENC_PRIV(x) ((V4L2_CTRL_ID2CLASS(x) == V4L2_CTRL_CLASS_MPEG) &&\
+ V4L2_CTRL_DRIVER_PRIV(x))
+
+static const char *const *mtk_vcodec_enc_get_menu(u32 id)
+{
+ static const char *const mtk_vcodec_enc_video_frame_skip[] = {
+ "Disabled",
+ "Level Limit",
+ "VBV/CPB Limit",
+ NULL,
+ };
+ static const char *const mtk_vcodec_enc_video_force_frame[] = {
+ "Disabled",
+ "I Frame",
+ "Not Coded",
+ NULL,
+ };
+ switch (id) {
+ case V4L2_CID_MPEG_MFC51_VIDEO_FRAME_SKIP_MODE:
+ return mtk_vcodec_enc_video_frame_skip;
+ case V4L2_CID_MPEG_MFC51_VIDEO_FORCE_FRAME_TYPE:
+ return mtk_vcodec_enc_video_force_frame;
+ }
+ return NULL;
+}
+
+int mtk_venc_ctrls_setup(struct mtk_vcodec_ctx *ctx)
+{
+ struct v4l2_ctrl_config cfg;
+ int i;
+
+ v4l2_ctrl_handler_init(&ctx->ctrl_hdl, NUM_CTRLS);
+ if (ctx->ctrl_hdl.error) {
+ v4l2_err(&ctx->dev->v4l2_dev, "Init control handler fail %d\n",
+ ctx->ctrl_hdl.error);
+ return ctx->ctrl_hdl.error;
+ }
+ for (i = 0; i < NUM_CTRLS; i++) {
+ if (IS_MTK_VENC_PRIV(controls[i].id)) {
+ memset(&cfg, 0, sizeof(struct v4l2_ctrl_config));
+ cfg.ops = &mtk_vcodec_enc_ctrl_ops;
+ cfg.id = controls[i].id;
+ cfg.min = controls[i].minimum;
+ cfg.max = controls[i].maximum;
+ cfg.def = controls[i].default_value;
+ cfg.name = controls[i].name;
+ cfg.type = controls[i].type;
+ cfg.flags = 0;
+ if (cfg.type == V4L2_CTRL_TYPE_MENU) {
+ cfg.step = 0;
+ cfg.menu_skip_mask = cfg.menu_skip_mask;
+ cfg.qmenu = mtk_vcodec_enc_get_menu(cfg.id);
+ } else {
+ cfg.step = controls[i].step;
+ cfg.menu_skip_mask = 0;
+ }
+ v4l2_ctrl_new_custom(&ctx->ctrl_hdl, &cfg, NULL);
+ } else {
+ if ((controls[i].type == V4L2_CTRL_TYPE_MENU) ||
+ (controls[i].type == V4L2_CTRL_TYPE_INTEGER_MENU)) {
+ v4l2_ctrl_new_std_menu(
+ &ctx->ctrl_hdl,
+ &mtk_vcodec_enc_ctrl_ops,
+ controls[i].id,
+ controls[i].maximum, 0,
+ controls[i].default_value);
+ } else {
+ v4l2_ctrl_new_std(
+ &ctx->ctrl_hdl,
+ &mtk_vcodec_enc_ctrl_ops,
+ controls[i].id,
+ controls[i].minimum,
+ controls[i].maximum,
+ controls[i].step,
+ controls[i].default_value);
+ }
+ }
+
+ if (ctx->ctrl_hdl.error) {
+ v4l2_err(&ctx->dev->v4l2_dev,
+ "Adding control (%d) failed %d\n",
+ i, ctx->ctrl_hdl.error);
+ return ctx->ctrl_hdl.error;
+ }
+ }
+
+ v4l2_ctrl_handler_setup(&ctx->ctrl_hdl);
+ return 0;
+}
+
+void mtk_venc_ctrls_free(struct mtk_vcodec_ctx *ctx)
+{
+ v4l2_ctrl_handler_free(&ctx->ctrl_hdl);
+}
+
+int m2mctx_venc_queue_init(void *priv, struct vb2_queue *src_vq,
+ struct vb2_queue *dst_vq)
+{
+ struct mtk_vcodec_ctx *ctx = priv;
+ int ret;
+
+ src_vq->type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE;
+ src_vq->io_modes = VB2_DMABUF | VB2_MMAP | VB2_USERPTR;
+ src_vq->drv_priv = ctx;
+ src_vq->buf_struct_size = sizeof(struct mtk_video_enc_buf);
+ src_vq->ops = &mtk_venc_vb2_ops;
+ src_vq->mem_ops = &vb2_dma_contig_memops;
+ src_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
+ src_vq->lock = &ctx->vb2_mutex;
+
+ ret = vb2_queue_init(src_vq);
+ if (ret)
+ return ret;
+
+ dst_vq->type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
+ dst_vq->io_modes = VB2_DMABUF | VB2_MMAP | VB2_USERPTR;
+ dst_vq->drv_priv = ctx;
+ dst_vq->buf_struct_size = sizeof(struct v4l2_m2m_buffer);
+ dst_vq->ops = &mtk_venc_vb2_ops;
+ dst_vq->mem_ops = &vb2_dma_contig_memops;
+ dst_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
+ dst_vq->lock = &ctx->vb2_mutex;
+
+ return vb2_queue_init(dst_vq);
+}
+
+int mtk_venc_unlock(struct mtk_vcodec_ctx *ctx)
+{
+ struct mtk_vcodec_dev *dev = ctx->dev;
+
+ mutex_unlock(&dev->enc_mutex);
+ return 0;
+}
+
+int mtk_venc_lock(struct mtk_vcodec_ctx *ctx)
+{
+ struct mtk_vcodec_dev *dev = ctx->dev;
+
+ mutex_lock(&dev->enc_mutex);
+ dev->curr_ctx = ctx->idx;
+ return 0;
+}
+
+void mtk_vcodec_venc_release(struct mtk_vcodec_ctx *ctx)
+{
+ if (ctx->h_enc) {
+ venc_if_deinit(ctx->h_enc);
+ venc_if_release(ctx->h_enc);
+ }
+
+#if MTK_V4L2_BENCHMARK
+ mtk_v4l2_debug(0, "\n\nMTK_V4L2_BENCHMARK");
+
+ mtk_v4l2_debug(0, " total_enc_dec_cnt: %d ", ctx->total_enc_dec_cnt);
+ mtk_v4l2_debug(0, " total_enc_dec_time: %d us",
+ ctx->total_enc_dec_time);
+ mtk_v4l2_debug(0, " total_enc_dec_init_time: %d us",
+ ctx->total_enc_dec_init_time);
+ mtk_v4l2_debug(0, " total_enc_hdr_time: %d us",
+ ctx->total_enc_hdr_time);
+ mtk_v4l2_debug(0, " total_qbuf_out_time: %d us",
+ ctx->total_qbuf_out_time);
+ mtk_v4l2_debug(0, " total_qbuf_out_cnt: %d ",
+ ctx->total_qbuf_out_cnt);
+ mtk_v4l2_debug(0, " total_qbuf_cap_time: %d us",
+ ctx->total_qbuf_cap_time);
+ mtk_v4l2_debug(0, " total_qbuf_cap_cnt: %d ",
+ ctx->total_qbuf_cap_cnt);
+
+ mtk_v4l2_debug(0, " total_dqbuf_out_time: %d us",
+ ctx->total_dqbuf_out_time);
+ mtk_v4l2_debug(0, " total_dqbuf_out_cnt: %d ",
+ ctx->total_dqbuf_out_cnt);
+ mtk_v4l2_debug(0, " total_dqbuf_cap_time: %d us",
+ ctx->total_dqbuf_cap_time);
+ mtk_v4l2_debug(0, " total_dqbuf_cap_cnt: %d ",
+ ctx->total_dqbuf_cap_cnt);
+
+ mtk_v4l2_debug(0, " total_expbuf_time: %d us",
+ ctx->total_expbuf_time);
+
+#endif
+
+}
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h
new file mode 100644
index 0000000..6802c93
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h
@@ -0,0 +1,28 @@
+/*
+* Copyright (c) 2015 MediaTek Inc.
+* Author: PC Chen <pc.chen@mediatek.com>
+* Tiffany Lin <tiffany.lin@mediatek.com>
+*
+* This program is free software; you can redistribute it and/or modify
+* it under the terms of the GNU General Public License version 2 as
+* published by the Free Software Foundation.
+*
+* This program is distributed in the hope that it will be useful,
+* but WITHOUT ANY WARRANTY; without even the implied warranty of
+* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+* GNU General Public License for more details.
+*/
+
+
+#ifndef _MTK_VCODEC_ENC_H_
+#define _MTK_VCODEC_ENC_H_
+
+int mtk_venc_unlock(struct mtk_vcodec_ctx *ctx);
+int mtk_venc_lock(struct mtk_vcodec_ctx *ctx);
+int m2mctx_venc_queue_init(void *priv, struct vb2_queue *src_vq,
+ struct vb2_queue *dst_vq);
+void mtk_vcodec_venc_release(struct mtk_vcodec_ctx *ctx);
+int mtk_venc_ctrls_setup(struct mtk_vcodec_ctx *ctx);
+void mtk_venc_ctrls_free(struct mtk_vcodec_ctx *ctx);
+
+#endif /* _MTK_VCODEC_ENC_H_ */
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
new file mode 100644
index 0000000..0f89880
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
@@ -0,0 +1,535 @@
+/*
+* Copyright (c) 2015 MediaTek Inc.
+* Author: PC Chen <pc.chen@mediatek.com>
+* Tiffany Lin <tiffany.lin@mediatek.com>
+*
+* This program is free software; you can redistribute it and/or modify
+* it under the terms of the GNU General Public License version 2 as
+* published by the Free Software Foundation.
+*
+* This program is distributed in the hope that it will be useful,
+* but WITHOUT ANY WARRANTY; without even the implied warranty of
+* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+* GNU General Public License for more details.
+*/
+
+#include <linux/interrupt.h>
+#include <linux/irq.h>
+#include <linux/module.h>
+#include <linux/of_device.h>
+#include <linux/of.h>
+#include <media/v4l2-event.h>
+#include <media/v4l2-mem2mem.h>
+#include <media/videobuf2-dma-contig.h>
+#include <linux/pm_runtime.h>
+
+#include "mtk_vcodec_drv.h"
+#include "mtk_vcodec_enc.h"
+#include "mtk_vcodec_pm.h"
+#include "mtk_vcodec_intr.h"
+#include "mtk_vcodec_util.h"
+#include "mtk_vpu_core.h"
+
+static irqreturn_t mtk_vcodec_enc_irq_handler(int irq, void *priv);
+
+static void vpu_enc_capability_ipi_handler(void *data, unsigned int len, void *priv)
+{
+ struct mtk_vcodec_dev *dev = (struct mtk_vcodec_dev *)priv;
+
+ dev->enc_capability = *((u32 *)data);
+ mtk_v4l2_debug(0, "IPI=%d encoder capability %x",
+ IPI_VENC_CAPABILITY, dev->enc_capability);
+}
+
+
+/* Wake up context wait_queue */
+static void wake_up_ctx(struct mtk_vcodec_ctx *ctx, unsigned int reason)
+{
+ ctx->int_cond = 1;
+ ctx->int_type = reason;
+ wake_up_interruptible(&ctx->queue);
+}
+
+static irqreturn_t mtk_vcodec_enc_irq_handler(int irq, void *priv)
+{
+ struct mtk_vcodec_dev *dev = priv;
+ struct mtk_vcodec_ctx *ctx;
+ unsigned int irq_status;
+
+ ctx = dev->ctx[dev->curr_ctx];
+ if (ctx == NULL) {
+ mtk_v4l2_err("ctx==NULL");
+ return IRQ_HANDLED;
+ }
+ mtk_v4l2_debug(1, "idx=%d", ctx->idx);
+ irq_status = readl(dev->reg_base[VENC_SYS] +
+ (MTK_VENC_IRQ_STATUS_OFFSET));
+ if (irq_status & MTK_VENC_IRQ_STATUS_PAUSE)
+ writel((MTK_VENC_IRQ_STATUS_PAUSE),
+ dev->reg_base[VENC_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
+
+ if (irq_status & MTK_VENC_IRQ_STATUS_SWITCH)
+ writel((MTK_VENC_IRQ_STATUS_SWITCH),
+ dev->reg_base[VENC_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
+
+ if (irq_status & MTK_VENC_IRQ_STATUS_DRAM)
+ writel((MTK_VENC_IRQ_STATUS_DRAM),
+ dev->reg_base[VENC_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
+
+ if (irq_status & MTK_VENC_IRQ_STATUS_SPS)
+ writel((MTK_VENC_IRQ_STATUS_SPS),
+ dev->reg_base[VENC_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
+
+ if (irq_status & MTK_VENC_IRQ_STATUS_PPS)
+ writel((MTK_VENC_IRQ_STATUS_PPS),
+ dev->reg_base[VENC_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
+
+ if (irq_status & MTK_VENC_IRQ_STATUS_FRM)
+ writel((MTK_VENC_IRQ_STATUS_FRM),
+ dev->reg_base[VENC_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
+
+ ctx->irq_status = irq_status;
+ wake_up_ctx(ctx, MTK_INST_IRQ_RECEIVED);
+ return IRQ_HANDLED;
+}
+
+#if 1 /* VENC_LT */
+static irqreturn_t mtk_vcodec_enc_irq_handler2(int irq, void *priv)
+{
+ struct mtk_vcodec_dev *dev = priv;
+ struct mtk_vcodec_ctx *ctx;
+ unsigned int irq_status;
+
+ ctx = dev->ctx[dev->curr_ctx];
+ if (ctx == NULL) {
+ mtk_v4l2_err("ctx==NULL");
+ return IRQ_HANDLED;
+ }
+ mtk_v4l2_debug(1, "idx=%d", ctx->idx);
+ irq_status = readl(dev->reg_base[VENC_LT_SYS] +
+ (MTK_VENC_IRQ_STATUS_OFFSET));
+ if (irq_status & MTK_VENC_IRQ_STATUS_PAUSE)
+ writel((MTK_VENC_IRQ_STATUS_PAUSE),
+ dev->reg_base[VENC_LT_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
+
+ if (irq_status & MTK_VENC_IRQ_STATUS_SWITCH)
+ writel((MTK_VENC_IRQ_STATUS_SWITCH),
+ dev->reg_base[VENC_LT_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
+
+ if (irq_status & MTK_VENC_IRQ_STATUS_DRAM)
+ writel((MTK_VENC_IRQ_STATUS_DRAM),
+ dev->reg_base[VENC_LT_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
+
+ if (irq_status & MTK_VENC_IRQ_STATUS_SPS)
+ writel((MTK_VENC_IRQ_STATUS_SPS),
+ dev->reg_base[VENC_LT_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
+
+ if (irq_status & MTK_VENC_IRQ_STATUS_PPS)
+ writel((MTK_VENC_IRQ_STATUS_PPS),
+ dev->reg_base[VENC_LT_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
+
+ if (irq_status & MTK_VENC_IRQ_STATUS_FRM)
+ writel((MTK_VENC_IRQ_STATUS_FRM),
+ dev->reg_base[VENC_LT_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
+
+ ctx->irq_status = irq_status;
+ wake_up_ctx(ctx, MTK_INST_IRQ_RECEIVED);
+ return IRQ_HANDLED;
+}
+#endif
+
+static int fops_vcodec_open(struct file *file)
+{
+ struct video_device *vfd = video_devdata(file);
+ struct mtk_vcodec_dev *dev = video_drvdata(file);
+ struct mtk_vcodec_ctx *ctx = NULL;
+ int ret = 0;
+
+ mtk_v4l2_debug(0, "%s encoder", dev_name(&dev->plat_dev->dev));
+ mutex_lock(&dev->dev_mutex);
+
+ ctx = devm_kzalloc(&dev->plat_dev->dev, sizeof(*ctx), GFP_KERNEL);
+ if (!ctx) {
+ ret = -ENOMEM;
+ goto err_alloc;
+ }
+
+ if (dev->num_instances >= MTK_VCODEC_MAX_INSTANCES) {
+ mtk_v4l2_err("Too many open contexts\n");
+ ret = -EBUSY;
+ goto err_no_ctx;
+ }
+
+ ctx->idx = ffz(dev->instance_mask[0]);
+ v4l2_fh_init(&ctx->fh, video_devdata(file));
+ file->private_data = &ctx->fh;
+ v4l2_fh_add(&ctx->fh);
+ ctx->dev = dev;
+ mutex_init(&ctx->encode_param_mutex);
+ mutex_init(&ctx->vb2_mutex);
+ if (vfd == dev->vfd_enc) {
+ ctx->type = MTK_INST_ENCODER;
+ ret = mtk_venc_ctrls_setup(ctx);
+ if (ret) {
+ mtk_v4l2_err("Failed to setup controls() (%d)\n",
+ ret);
+ goto err_ctrls_setup;
+ }
+ ctx->m2m_ctx = v4l2_m2m_ctx_init(dev->m2m_dev_enc, ctx,
+ &m2mctx_venc_queue_init);
+ if (IS_ERR(ctx->m2m_ctx)) {
+ ret = PTR_ERR(ctx->m2m_ctx);
+ mtk_v4l2_err("Failed to v4l2_m2m_ctx_init() (%d)\n",
+ ret);
+ goto err_ctx_init;
+ }
+ ctx->fh.ctrl_handler = &ctx->ctrl_hdl;
+ } else {
+ mtk_v4l2_err("Invalid vfd !\n");
+ ret = -ENOENT;
+ goto err_ctx_init;
+ }
+
+ init_waitqueue_head(&ctx->queue);
+ dev->num_instances++;
+
+ if (dev->num_instances == 1) {
+ ret = vpu_load_firmware(vpu_get_plat_device(dev->plat_dev));
+ if (ret < 0) {
+ mtk_v4l2_err("vpu_load_firmware failed!\n");
+ goto err_load_fw;
+ }
+ mtk_v4l2_debug(0, "encoder capability %x", dev->enc_capability);
+ }
+
+ mtk_v4l2_debug(2, "Create instance [%d]@%p m2m_ctx=%p type=%d\n",
+ ctx->idx, ctx, ctx->m2m_ctx, ctx->type);
+ ctx->state = MTK_STATE_FREE;
+ mtk_vcodec_clean_ctx_int_flags(ctx);
+ set_bit(ctx->idx, &dev->instance_mask[0]);
+ dev->ctx[ctx->idx] = ctx;
+ vpu_enable_clock(vpu_get_plat_device(dev->plat_dev));
+
+ mutex_unlock(&dev->dev_mutex);
+ return ret;
+
+ /* Deinit when failure occurred */
+err_load_fw:
+ v4l2_m2m_ctx_release(ctx->m2m_ctx);
+ v4l2_fh_del(&ctx->fh);
+ v4l2_fh_exit(&ctx->fh);
+ dev->num_instances--;
+err_ctx_init:
+err_ctrls_setup:
+ mtk_venc_ctrls_free(ctx);
+err_no_ctx:
+ devm_kfree(&dev->plat_dev->dev, ctx);
+err_alloc:
+ mutex_unlock(&dev->dev_mutex);
+ return ret;
+}
+
+static int fops_vcodec_release(struct file *file)
+{
+ struct mtk_vcodec_dev *dev = video_drvdata(file);
+ struct mtk_vcodec_ctx *ctx = fh_to_ctx(file->private_data);
+
+ mtk_v4l2_debug(2, "[%d]\n", ctx->idx);
+ mutex_lock(&dev->dev_mutex);
+ v4l2_m2m_ctx_release(ctx->m2m_ctx);
+ mtk_vcodec_venc_release(ctx);
+ ctx->state = MTK_STATE_DEINIT;
+ mtk_venc_ctrls_free(ctx);
+ v4l2_fh_del(&ctx->fh);
+ v4l2_fh_exit(&ctx->fh);
+ dev->ctx[ctx->idx] = NULL;
+ dev->num_instances--;
+ if (dev->num_instances == 0) {
+ vpu_disable_clock(vpu_get_plat_device(dev->plat_dev));
+ }
+ clear_bit(ctx->idx, &dev->instance_mask[0]);
+ devm_kfree(&dev->plat_dev->dev, ctx);
+ mutex_unlock(&dev->dev_mutex);
+ return 0;
+}
+
+static unsigned int fops_vcodec_poll(struct file *file,
+ struct poll_table_struct *wait)
+{
+ struct mtk_vcodec_ctx *ctx = fh_to_ctx(file->private_data);
+ struct mtk_vcodec_dev *dev = ctx->dev;
+ int ret;
+
+ mutex_lock(&dev->dev_mutex);
+ ret = v4l2_m2m_poll(file, ctx->m2m_ctx, wait);
+ mutex_unlock(&dev->dev_mutex);
+
+ return ret;
+}
+
+static int fops_vcodec_mmap(struct file *file, struct vm_area_struct *vma)
+{
+ struct mtk_vcodec_ctx *ctx = fh_to_ctx(file->private_data);
+
+ return v4l2_m2m_mmap(file, ctx->m2m_ctx, vma);
+}
+
+static const struct v4l2_file_operations mtk_vcodec_fops = {
+ .owner = THIS_MODULE,
+ .open = fops_vcodec_open,
+ .release = fops_vcodec_release,
+ .poll = fops_vcodec_poll,
+ .unlocked_ioctl = video_ioctl2,
+ .mmap = fops_vcodec_mmap,
+};
+
+static int mtk_vcodec_probe(struct platform_device *pdev)
+{
+ struct mtk_vcodec_dev *dev;
+ struct video_device *vfd_enc;
+ struct resource *res;
+ int i, j, ret;
+ struct platform_device *vpu_pdev;
+
+ dev = devm_kzalloc(&pdev->dev, sizeof(*dev), GFP_KERNEL);
+ if (!dev)
+ return -ENOMEM;
+
+ dev->plat_dev = pdev;
+
+ vpu_pdev = vpu_get_plat_device(dev->plat_dev);
+ if (vpu_pdev == NULL) {
+ mtk_v4l2_err("[VPU] vpu device in not ready\n");
+ return -EPROBE_DEFER;
+ }
+
+ ret = vpu_ipi_register(vpu_pdev, IPI_VENC_CAPABILITY,
+ vpu_enc_capability_ipi_handler,
+ "vpu_init", dev);
+ if (ret != 0) {
+ mtk_v4l2_err("[VPU] failed to register IPI_VENC_CAPBILITY\n");
+ return ret;
+ }
+
+ ret = mtk_vcodec_init_enc_pm(dev);
+ if (ret < 0) {
+ dev_err(&pdev->dev, "Failed to get mt vcodec clock source!\n");
+ return ret;
+ }
+
+ for (i = VENC_SYS, j = 0; i < NUM_MAX_VCODEC_REG_BASE; i++, j++) {
+ res = platform_get_resource(pdev, IORESOURCE_MEM, j);
+ if (res == NULL) {
+ dev_err(&pdev->dev, "get memory resource failed.\n");
+ ret = -ENXIO;
+ goto err_res;
+ }
+ dev->reg_base[i] = devm_ioremap_resource(&pdev->dev, res);
+ if (IS_ERR(dev->reg_base[i])) {
+ dev_err(&pdev->dev,
+ "devm_ioremap_resource %d failed.\n", i);
+ ret = PTR_ERR(dev->reg_base);
+ goto err_res;
+ }
+ mtk_v4l2_debug(2, "reg[%d] base=0x%p\n", i, dev->reg_base[i]);
+ }
+
+ res = platform_get_resource(pdev, IORESOURCE_IRQ, 0);
+ if (res == NULL) {
+ dev_err(&pdev->dev, "failed to get irq resource\n");
+ ret = -ENOENT;
+ goto err_res;
+ }
+
+ dev->enc_irq = platform_get_irq(pdev, 0);
+ ret = devm_request_irq(&pdev->dev, dev->enc_irq,
+ mtk_vcodec_enc_irq_handler,
+ 0, pdev->name, dev);
+ if (ret) {
+ dev_err(&pdev->dev, "Failed to install dev->enc_irq %d (%d)\n",
+ dev->enc_irq,
+ ret);
+ ret = -EINVAL;
+ goto err_res;
+ }
+
+ dev->enc_lt_irq = platform_get_irq(pdev, 1);
+ ret = devm_request_irq(&pdev->dev,
+ dev->enc_lt_irq, mtk_vcodec_enc_irq_handler2,
+ 0, pdev->name, dev);
+ if (ret) {
+ dev_err(&pdev->dev,
+ "Failed to install dev->enc_lt_irq %d (%d)\n",
+ dev->enc_lt_irq, ret);
+ ret = -EINVAL;
+ goto err_res;
+ }
+
+ disable_irq(dev->enc_irq);
+ disable_irq(dev->enc_lt_irq); /* VENC_LT */
+ mutex_init(&dev->enc_mutex);
+ mutex_init(&dev->dev_mutex);
+
+ snprintf(dev->v4l2_dev.name, sizeof(dev->v4l2_dev.name), "%s",
+ "[MTK_V4L2_VENC]");
+
+ ret = v4l2_device_register(&pdev->dev, &dev->v4l2_dev);
+ if (ret) {
+ mtk_v4l2_err("v4l2_device_register err=%d\n", ret);
+ return ret;
+ }
+
+ init_waitqueue_head(&dev->queue);
+
+ /* allocate video device for encoder and register it */
+ vfd_enc = video_device_alloc();
+ if (!vfd_enc) {
+ mtk_v4l2_err("Failed to allocate video device\n");
+ ret = -ENOMEM;
+ goto err_enc_alloc;
+ }
+ vfd_enc->fops = &mtk_vcodec_fops;
+ vfd_enc->ioctl_ops = &mtk_venc_ioctl_ops;
+ vfd_enc->release = video_device_release;
+ vfd_enc->lock = &dev->dev_mutex;
+ vfd_enc->v4l2_dev = &dev->v4l2_dev;
+ vfd_enc->vfl_dir = VFL_DIR_M2M;
+ snprintf(vfd_enc->name, sizeof(vfd_enc->name), "%s",
+ MTK_VCODEC_ENC_NAME);
+ video_set_drvdata(vfd_enc, dev);
+ dev->vfd_enc = vfd_enc;
+ platform_set_drvdata(pdev, dev);
+ ret = video_register_device(vfd_enc, VFL_TYPE_GRABBER, 1);
+ if (ret) {
+ mtk_v4l2_err("Failed to register video device\n");
+ goto err_enc_reg;
+ }
+ mtk_v4l2_debug(0, "encoder registered as /dev/video%d\n",
+ vfd_enc->num);
+
+ dev->alloc_ctx = vb2_dma_contig_init_ctx(&pdev->dev);
+ if (IS_ERR(dev->alloc_ctx)) {
+ mtk_v4l2_err("Failed to alloc vb2 dma context 0\n");
+ ret = PTR_ERR(dev->alloc_ctx);
+ goto err_vb2_ctx_init;
+ }
+
+ dev->m2m_dev_enc = v4l2_m2m_init(&mtk_venc_m2m_ops);
+ if (IS_ERR(dev->m2m_dev_enc)) {
+ mtk_v4l2_err("Failed to init mem2mem enc device\n");
+ ret = PTR_ERR(dev->m2m_dev_enc);
+ goto err_enc_mem_init;
+ }
+
+ dev->encode_workqueue =
+ create_singlethread_workqueue(MTK_VCODEC_ENC_NAME);
+ if (!dev->encode_workqueue) {
+ mtk_v4l2_err("Failed to create encode workqueue\n");
+ ret = -EINVAL;
+ goto err_event_workq;
+ }
+
+ return 0;
+
+err_event_workq:
+ v4l2_m2m_release(dev->m2m_dev_enc);
+err_enc_mem_init:
+ vb2_dma_contig_cleanup_ctx(dev->alloc_ctx);
+err_vb2_ctx_init:
+ video_unregister_device(vfd_enc);
+err_enc_reg:
+ video_device_release(vfd_enc);
+err_enc_alloc:
+ v4l2_device_unregister(&dev->v4l2_dev);
+err_res:
+ mtk_vcodec_release_enc_pm(dev);
+ return ret;
+}
+
+static const struct of_device_id mtk_vcodec_match[] = {
+ {.compatible = "mediatek,mt8173-vcodec-enc",},
+ {},
+};
+MODULE_DEVICE_TABLE(of, mtk_vcodec_match);
+
+static int mtk_vcodec_remove(struct platform_device *pdev)
+{
+ struct mtk_vcodec_dev *dev = platform_get_drvdata(pdev);
+
+ mtk_v4l2_debug_enter();
+ flush_workqueue(dev->encode_workqueue);
+ destroy_workqueue(dev->encode_workqueue);
+ if (dev->m2m_dev_enc)
+ v4l2_m2m_release(dev->m2m_dev_enc);
+ if (dev->alloc_ctx)
+ vb2_dma_contig_cleanup_ctx(dev->alloc_ctx);
+
+ if (dev->vfd_enc) {
+ video_unregister_device(dev->vfd_enc);
+ video_device_release(dev->vfd_enc);
+ }
+ v4l2_device_unregister(&dev->v4l2_dev);
+ mtk_vcodec_release_enc_pm(dev);
+ return 0;
+}
+
+static int mtk_vcodec_venc_suspend(struct device *dev)
+{
+ struct platform_device *pdev = to_platform_device(dev);
+ struct mtk_vcodec_dev *m_dev = platform_get_drvdata(pdev);
+
+ mtk_v4l2_debug(0, "m_dev->num_instances=%d", m_dev->num_instances);
+ if (m_dev->num_instances == 0)
+ return 0;
+ mtk_vcodec_clean_dev_int_flags(m_dev);
+ if (test_and_set_bit(0, &m_dev->enter_suspend) != 0) {
+ mtk_v4l2_err("Error: going to suspend for a second time\n");
+ return -EIO;
+ }
+ mtk_vcodec_wait_for_done_dev(m_dev,
+ MTK_INST_WORK_THREAD_SUSPEND_DONE,
+ 3000, true);
+
+ mutex_lock(&m_dev->enc_mutex);
+ mtk_vcodec_enc_clock_off();
+ mtk_v4l2_debug(0, "m_dev->num_instances=%d", m_dev->num_instances);
+ return 0;
+}
+
+static int mtk_vcodec_venc_resume(struct device *dev)
+{
+ struct platform_device *pdev = to_platform_device(dev);
+ struct mtk_vcodec_dev *m_dev = platform_get_drvdata(pdev);
+
+ mtk_v4l2_debug(0, "m_dev->num_instances=%d", m_dev->num_instances);
+ if (m_dev->num_instances == 0)
+ return 0;
+
+ test_and_clear_bit(0, &m_dev->enter_suspend);
+ mtk_vcodec_enc_clock_on();
+ mutex_unlock(&m_dev->enc_mutex);
+ return 0;
+}
+
+
+static const struct dev_pm_ops mtk_vcodec_venc_pm_ops = {
+ SET_SYSTEM_SLEEP_PM_OPS(mtk_vcodec_venc_suspend,
+ mtk_vcodec_venc_resume)
+};
+
+static struct platform_driver mtk_vcodec_driver = {
+ .probe = mtk_vcodec_probe,
+ .remove = mtk_vcodec_remove,
+ .driver = {
+ .name = MTK_VCODEC_ENC_NAME,
+ .owner = THIS_MODULE,
+ .of_match_table = mtk_vcodec_match,
+ .pm = &mtk_vcodec_venc_pm_ops,
+ },
+};
+
+module_platform_driver(mtk_vcodec_driver);
+
+
+MODULE_LICENSE("GPL v2");
+MODULE_DESCRIPTION("Mediatek video codec V4L2 driver");
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
new file mode 100644
index 0000000..ba97344
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
@@ -0,0 +1,122 @@
+/*
+* Copyright (c) 2015 MediaTek Inc.
+* Author: Tiffany Lin <tiffany.lin@mediatek.com>
+*
+* This program is free software; you can redistribute it and/or modify
+* it under the terms of the GNU General Public License version 2 as
+* published by the Free Software Foundation.
+*
+* This program is distributed in the hope that it will be useful,
+* but WITHOUT ANY WARRANTY; without even the implied warranty of
+* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+* GNU General Public License for more details.
+*/
+
+#include <linux/clk.h>
+#include <linux/of_address.h>
+#include <linux/of_platform.h>
+#include <linux/pm_runtime.h>
+#include <soc/mediatek/smi.h>
+
+#include "mtk_vcodec_pm.h"
+#include "mtk_vcodec_util.h"
+#include "mtk_vpu_core.h"
+
+static struct mtk_vcodec_pm *pm;
+
+int mtk_vcodec_init_enc_pm(struct mtk_vcodec_dev *mtkdev)
+{
+ struct device_node *node;
+ struct platform_device *pdev;
+ struct device *dev;
+ int ret = 0;
+
+ pdev = mtkdev->plat_dev;
+ pm = &mtkdev->pm;
+ memset(pm, 0, sizeof(struct mtk_vcodec_pm));
+ pm->mtkdev = mtkdev;
+ dev = &pdev->dev;
+
+ node = of_parse_phandle(dev->of_node, "larb", 0);
+ if (!node)
+ return -1;
+ pdev = of_find_device_by_node(node);
+ if (WARN_ON(!pdev)) {
+ of_node_put(node);
+ return -1;
+ }
+ pm->larbvenc = &pdev->dev;
+
+ node = of_parse_phandle(dev->of_node, "larb", 1);
+ if (!node)
+ return -1;
+
+ pdev = of_find_device_by_node(node);
+ if (WARN_ON(!pdev)) {
+ of_node_put(node);
+ return -EINVAL;
+ }
+ pm->larbvenclt = &pdev->dev;
+
+ pdev = mtkdev->plat_dev;
+ pm->dev = &pdev->dev;
+
+ pm->vencpll = devm_clk_get(&pdev->dev, "vencpll");
+ if (pm->vencpll == NULL) {
+ mtk_v4l2_err("devm_clk_get vencpll fail");
+ ret = -1;
+ }
+
+ pm->venc_lt_sel = devm_clk_get(&pdev->dev, "venc_lt_sel");
+ if (pm->venc_lt_sel == NULL) {
+ mtk_v4l2_err("devm_clk_get venc_lt_sel fail");
+ ret = -1;
+ }
+
+ pm->vcodecpll_370p5_ck = devm_clk_get(&pdev->dev, "vcodecpll_370p5_ck");
+ if (pm->vcodecpll_370p5_ck == NULL) {
+ mtk_v4l2_err("devm_clk_get vcodecpll_370p5_ck fail");
+ ret = -1;
+ }
+
+ return ret;
+}
+
+void mtk_vcodec_release_enc_pm(struct mtk_vcodec_dev *dev)
+{
+}
+
+
+void mtk_vcodec_enc_clock_on(void)
+{
+ int ret;
+
+ clk_set_rate(pm->vencpll, 800 * 1000000);
+
+ ret = clk_prepare_enable(pm->vcodecpll_370p5_ck);
+ if (ret)
+ mtk_v4l2_err("vcodecpll_370p5_ck fail %d", ret);
+
+ ret = clk_prepare_enable(pm->venc_lt_sel);
+ if (ret)
+ mtk_v4l2_err("venc_lt_sel fail %d", ret);
+
+ ret = clk_set_parent(pm->venc_lt_sel, pm->vcodecpll_370p5_ck);
+ if (ret)
+ mtk_v4l2_err("clk_set_parent fail %d", ret);
+
+ ret = mtk_smi_larb_get(pm->larbvenc);
+ if (ret)
+ mtk_v4l2_err("mtk_smi_larb_get larb3 fail %d\n", ret);
+
+ ret = mtk_smi_larb_get(pm->larbvenclt);
+ if (ret)
+ mtk_v4l2_err("mtk_smi_larb_get larb4 fail %d\n", ret);
+
+}
+
+void mtk_vcodec_enc_clock_off(void)
+{
+ mtk_smi_larb_put(pm->larbvenc);
+ mtk_smi_larb_put(pm->larbvenclt);
+}
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.c
new file mode 100644
index 0000000..7a2cacf
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.c
@@ -0,0 +1,110 @@
+/*
+* Copyright (c) 2015 MediaTek Inc.
+* Author: Tiffany Lin <tiffany.lin@mediatek.com>
+*
+* This program is free software; you can redistribute it and/or modify
+* it under the terms of the GNU General Public License version 2 as
+* published by the Free Software Foundation.
+*
+* This program is distributed in the hope that it will be useful,
+* but WITHOUT ANY WARRANTY; without even the implied warranty of
+* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+* GNU General Public License for more details.
+*/
+
+#include <linux/errno.h>
+#include <linux/wait.h>
+
+#include "mtk_vcodec_intr.h"
+#include "mtk_vcodec_drv.h"
+#include "mtk_vcodec_util.h"
+
+void mtk_vcodec_clean_ctx_int_flags(void *data)
+{
+ struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)data;
+
+ ctx->int_cond = 0;
+ ctx->int_type = 0;
+}
+
+void mtk_vcodec_clean_dev_int_flags(void *data)
+{
+ struct mtk_vcodec_dev *dev = (struct mtk_vcodec_dev *)data;
+
+ dev->int_cond = 0;
+ dev->int_type = 0;
+}
+
+int mtk_vcodec_wait_for_done_ctx(void *data, int command,
+ unsigned int timeout, int interrupt)
+{
+ wait_queue_head_t *waitqueue;
+ long timeout_jiff, ret;
+ int status = 0;
+ struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)data;
+
+ waitqueue = (wait_queue_head_t *)&ctx->queue;
+ timeout_jiff = msecs_to_jiffies(timeout);
+ if (interrupt) {
+ ret = wait_event_interruptible_timeout(*waitqueue,
+ (ctx->int_cond &&
+ (ctx->int_type == command)),
+ timeout_jiff);
+ } else {
+ ret = wait_event_timeout(*waitqueue,
+ (ctx->int_cond &&
+ (ctx->int_type == command)),
+ timeout_jiff);
+ }
+ if (0 == ret) {
+ status = -1; /* timeout */
+ mtk_v4l2_err("[%d] cmd=%d, wait_event_interruptible_timeout time=%lu out %d %d!",
+ ctx->idx, command, timeout_jiff, ctx->int_cond,
+ ctx->int_type);
+ } else if (-ERESTARTSYS == ret) {
+ mtk_v4l2_err("[%d] cmd=%d, wait_event_interruptible_timeout interrupted by a signal %d %d",
+ ctx->idx, command, ctx->int_cond,
+ ctx->int_type);
+ status = -1;
+ }
+
+ ctx->int_cond = 0;
+ ctx->int_type = 0;
+
+ return status;
+}
+
+int mtk_vcodec_wait_for_done_dev(void *data, int command,
+ unsigned int timeout, int interrupt)
+{
+ wait_queue_head_t *waitqueue;
+ long timeout_jiff, ret;
+ int status = 0;
+ struct mtk_vcodec_dev *dev = (struct mtk_vcodec_dev *)data;
+
+ waitqueue = (wait_queue_head_t *)&dev->queue;
+ timeout_jiff = msecs_to_jiffies(timeout);
+ if (interrupt) {
+ ret = wait_event_interruptible_timeout(*waitqueue,
+ (dev->int_cond &&
+ (dev->int_type == command)),
+ timeout_jiff);
+ } else {
+ ret = wait_event_timeout(*waitqueue,
+ (dev->int_cond &&
+ (dev->int_type == command)),
+ timeout_jiff);
+ }
+ if (0 == ret) {
+ status = -1; /* timeout */
+ mtk_v4l2_err("wait_event_interruptible_timeout time=%lu out %d %d!",
+ timeout_jiff, dev->int_cond, dev->int_type);
+ } else if (-ERESTARTSYS == ret) {
+ mtk_v4l2_err("wait_event_interruptible_timeout interrupted by a signal %d %d",
+ dev->int_cond, dev->int_type);
+ status = -1;
+ }
+ dev->int_cond = 0;
+ dev->int_type = 0;
+ return status;
+}
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.h
new file mode 100644
index 0000000..e2d2eaa
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.h
@@ -0,0 +1,30 @@
+/*
+* Copyright (c) 2015 MediaTek Inc.
+* Author: Tiffany Lin <tiffany.lin@mediatek.com>
+*
+* This program is free software; you can redistribute it and/or modify
+* it under the terms of the GNU General Public License version 2 as
+* published by the Free Software Foundation.
+*
+* This program is distributed in the hope that it will be useful,
+* but WITHOUT ANY WARRANTY; without even the implied warranty of
+* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+* GNU General Public License for more details.
+*/
+
+#ifndef _MTK_VCODEC_INTR_H_
+#define _MTK_VCODEC_INTR_H_
+
+#define MTK_INST_IRQ_RECEIVED 0x1
+#define MTK_INST_WORK_THREAD_ABORT_DONE 0x2
+#define MTK_INST_WORK_THREAD_SUSPEND_DONE 0x3
+
+/* timeout is ms */
+int mtk_vcodec_wait_for_done_ctx(void *data, int command, unsigned int timeout,
+ int interrupt);
+int mtk_vcodec_wait_for_done_dev(void *data, int command, unsigned int timeout,
+ int interrupt);
+void mtk_vcodec_clean_ctx_int_flags(void *data);
+void mtk_vcodec_clean_dev_int_flags(void *data);
+
+#endif /* _MTK_VCODEC_INTR_H_ */
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_pm.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_pm.h
new file mode 100644
index 0000000..a28bb7c
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_pm.h
@@ -0,0 +1,26 @@
+/*
+* Copyright (c) 2015 MediaTek Inc.
+* Author: Tiffany Lin <tiffany.lin@mediatek.com>
+*
+* This program is free software; you can redistribute it and/or modify
+* it under the terms of the GNU General Public License version 2 as
+* published by the Free Software Foundation.
+*
+* This program is distributed in the hope that it will be useful,
+* but WITHOUT ANY WARRANTY; without even the implied warranty of
+* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+* GNU General Public License for more details.
+*/
+
+#ifndef _MTK_VCODEC_PM_H_
+#define _MTK_VCODEC_PM_H_
+
+#include "mtk_vcodec_drv.h"
+
+int mtk_vcodec_init_enc_pm(struct mtk_vcodec_dev *dev);
+void mtk_vcodec_release_enc_pm(struct mtk_vcodec_dev *dev);
+
+void mtk_vcodec_enc_clock_on(void);
+void mtk_vcodec_enc_clock_off(void);
+
+#endif /* _MTK_VCODEC_PM_H_ */
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c
new file mode 100644
index 0000000..5b6943c
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c
@@ -0,0 +1,106 @@
+/*
+* Copyright (c) 2015 MediaTek Inc.
+* Author: PC Chen <pc.chen@mediatek.com>
+* Tiffany Lin <tiffany.lin@mediatek.com>
+*
+* This program is free software; you can redistribute it and/or modify
+* it under the terms of the GNU General Public License version 2 as
+* published by the Free Software Foundation.
+*
+* This program is distributed in the hope that it will be useful,
+* but WITHOUT ANY WARRANTY; without even the implied warranty of
+* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+* GNU General Public License for more details.
+*/
+
+#include <linux/module.h>
+
+#include "mtk_vcodec_drv.h"
+#include "mtk_vcodec_util.h"
+#include "mtk_vpu_core.h"
+
+bool mtk_vcodec_dbg = false;
+int mtk_v4l2_dbg_level = 0;
+
+module_param(mtk_v4l2_dbg_level, int, S_IRUGO | S_IWUSR);
+module_param(mtk_vcodec_dbg, bool, S_IRUGO | S_IWUSR);
+
+void __iomem *mtk_vcodec_get_reg_addr(void *data, unsigned int reg_idx)
+{
+ struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)data;
+
+ if (!data || reg_idx >= NUM_MAX_VCODEC_REG_BASE) {
+ mtk_v4l2_err("Invalid arguments");
+ return NULL;
+ }
+ return ctx->dev->reg_base[reg_idx];
+}
+
+int mtk_vcodec_mem_alloc(void *data, struct mtk_vcodec_mem *mem)
+{
+ unsigned long size = mem->size;
+ struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)data;
+ struct device *dev = &ctx->dev->plat_dev->dev;
+
+ mem->va = dma_alloc_coherent(dev, size, &mem->dma_addr, GFP_KERNEL);
+
+ if (!mem->va) {
+ mtk_v4l2_err("%s dma_alloc size=%ld failed!", dev_name(dev),
+ size);
+ return -ENOMEM;
+ }
+
+ memset(mem->va, 0, size);
+
+ mtk_v4l2_debug(4, "[%d] - va = %p", ctx->idx, mem->va);
+ mtk_v4l2_debug(4, "[%d] - dma = 0x%lx", ctx->idx,
+ (unsigned long)mem->dma_addr);
+ mtk_v4l2_debug(4, "[%d] size = 0x%lx", ctx->idx, size);
+
+ return 0;
+}
+
+void mtk_vcodec_mem_free(void *data, struct mtk_vcodec_mem *mem)
+{
+ unsigned long size = mem->size;
+ struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)data;
+ struct device *dev = &ctx->dev->plat_dev->dev;
+
+ dma_free_coherent(dev, size, mem->va, mem->dma_addr);
+ mem->va = NULL;
+
+ mtk_v4l2_debug(4, "[%d] - va = %p", ctx->idx, mem->va);
+ mtk_v4l2_debug(4, "[%d] - dma = 0x%lx", ctx->idx,
+ (unsigned long)mem->dma_addr);
+ mtk_v4l2_debug(4, "[%d] size = 0x%lx", ctx->idx, size);
+}
+
+int mtk_vcodec_get_ctx_id(void *data)
+{
+ struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)data;
+
+ if (!ctx)
+ return -1;
+
+ return ctx->idx;
+}
+
+struct platform_device *mtk_vcodec_get_plat_dev(void *data)
+{
+ struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)data;
+
+ if (!ctx)
+ return NULL;
+
+ return vpu_get_plat_device(ctx->dev->plat_dev);
+}
+
+void mtk_vcodec_fmt2str(u32 fmt, char *str)
+{
+ char a = fmt & 0xFF;
+ char b = (fmt >> 8) & 0xFF;
+ char c = (fmt >> 16) & 0xFF;
+ char d = (fmt >> 24) & 0xFF;
+
+ sprintf(str, "%c%c%c%c", a, b, c, d);
+}
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
new file mode 100644
index 0000000..a8e683a
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
@@ -0,0 +1,66 @@
+/*
+* Copyright (c) 2015 MediaTek Inc.
+* Author: PC Chen <pc.chen@mediatek.com>
+* Tiffany Lin <tiffany.lin@mediatek.com>
+*
+* This program is free software; you can redistribute it and/or modify
+* it under the terms of the GNU General Public License version 2 as
+* published by the Free Software Foundation.
+*
+* This program is distributed in the hope that it will be useful,
+* but WITHOUT ANY WARRANTY; without even the implied warranty of
+* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+* GNU General Public License for more details.
+*/
+
+#ifndef _MTK_VCODEC_UTIL_H_
+#define _MTK_VCODEC_UTIL_H_
+
+#include <linux/types.h>
+#include <linux/dma-direction.h>
+
+struct mtk_vcodec_mem {
+ size_t size;
+ void *va;
+ dma_addr_t dma_addr;
+};
+
+extern int mtk_v4l2_dbg_level;
+extern bool mtk_vcodec_dbg;
+
+#define mtk_v4l2_debug(level, fmt, args...) \
+ do { \
+ if (mtk_v4l2_dbg_level >= level) \
+ pr_info("[MTK_V4L2] level=%d %s(),%d: " fmt "\n",\
+ level, __func__, __LINE__, ##args); \
+ } while (0)
+
+#define mtk_v4l2_err(fmt, args...) \
+ pr_err("[MTK_V4L2][ERROR] %s:%d: " fmt "\n", __func__, __LINE__, \
+ ##args)
+
+#define mtk_v4l2_debug_enter() mtk_v4l2_debug(5, "+\n")
+#define mtk_v4l2_debug_leave() mtk_v4l2_debug(5, "-\n")
+
+#define mtk_vcodec_debug(h, fmt, args...) \
+ do { \
+ if (mtk_vcodec_dbg) \
+ pr_info("[MTK_VCODEC][%d]: %s() " fmt "\n", \
+ ((struct mtk_vcodec_ctx *)h->ctx)->idx, \
+ __func__, ##args); \
+ } while (0)
+
+#define mtk_vcodec_err(h, fmt, args...) \
+ pr_err("[MTK_VCODEC][ERROR][%d]: %s() " fmt "\n", \
+ ((struct mtk_vcodec_ctx *)h->ctx)->idx, __func__, ##args)
+
+#define mtk_vcodec_debug_enter(h) mtk_vcodec_debug(h, "+\n")
+#define mtk_vcodec_debug_leave(h) mtk_vcodec_debug(h, "-\n")
+
+void __iomem *mtk_vcodec_get_reg_addr(void *data, unsigned int reg_idx);
+int mtk_vcodec_mem_alloc(void *data, struct mtk_vcodec_mem *mem);
+void mtk_vcodec_mem_free(void *data, struct mtk_vcodec_mem *mem);
+int mtk_vcodec_get_ctx_id(void *data);
+struct platform_device *mtk_vcodec_get_plat_dev(void *data);
+void mtk_vcodec_fmt2str(u32 fmt, char *str);
+#endif /* _MTK_VCODEC_UTIL_H_ */
--
1.7.9.5
^ permalink raw reply related [flat|nested] 84+ messages in thread
* [RESEND RFC/PATCH 6/8] media: platform: mtk-vcodec: Add Mediatek V4L2 Video Encoder Driver
@ 2015-11-17 12:54 ` Tiffany Lin
0 siblings, 0 replies; 84+ messages in thread
From: Tiffany Lin @ 2015-11-17 12:54 UTC (permalink / raw)
To: Rob Herring, Pawel Moll, Mark Rutland, Ian Campbell, Kumar Gala,
Catalin Marinas, Will Deacon, Mauro Carvalho Chehab,
Matthias Brugger, Daniel Kurtz, Sascha Hauer, Hongzhou Yang,
Hans Verkuil, Laurent Pinchart, Sakari Ailus, Geert Uytterhoeven,
Mikhail Ulyanov, Fabien Dessenne, Arnd Bergmann,
Darren Etheridge, Peter Griffin, Benoit Parrot
Cc: Tiffany Lin, Andrew-CT Chen, Eddie Huang, Yingjoe Chen,
James Liao, Daniel Hsiao, devicetree, linux-kernel,
linux-arm-kernel, linux-media, linux-mediatek
Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
Signed-off-by: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
---
drivers/media/platform/Kconfig | 13 +
drivers/media/platform/Makefile | 3 +
drivers/media/platform/mtk-vcodec/Kconfig | 5 +
drivers/media/platform/mtk-vcodec/Makefile | 12 +
drivers/media/platform/mtk-vcodec/common/Makefile | 8 +
.../media/platform/mtk-vcodec/common/venc_drv_if.c | 152 ++
.../platform/mtk-vcodec/include/venc_drv_base.h | 68 +
.../platform/mtk-vcodec/include/venc_drv_if.h | 187 +++
.../platform/mtk-vcodec/include/venc_ipi_msg.h | 212 +++
drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h | 441 +++++
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c | 1773 ++++++++++++++++++++
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h | 28 +
.../media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c | 535 ++++++
.../media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c | 122 ++
.../media/platform/mtk-vcodec/mtk_vcodec_intr.c | 110 ++
.../media/platform/mtk-vcodec/mtk_vcodec_intr.h | 30 +
drivers/media/platform/mtk-vcodec/mtk_vcodec_pm.h | 26 +
.../media/platform/mtk-vcodec/mtk_vcodec_util.c | 106 ++
.../media/platform/mtk-vcodec/mtk_vcodec_util.h | 66 +
19 files changed, 3897 insertions(+)
create mode 100644 drivers/media/platform/mtk-vcodec/Kconfig
create mode 100644 drivers/media/platform/mtk-vcodec/Makefile
create mode 100644 drivers/media/platform/mtk-vcodec/common/Makefile
create mode 100644 drivers/media/platform/mtk-vcodec/common/venc_drv_if.c
create mode 100644 drivers/media/platform/mtk-vcodec/include/venc_drv_base.h
create mode 100644 drivers/media/platform/mtk-vcodec/include/venc_drv_if.h
create mode 100644 drivers/media/platform/mtk-vcodec/include/venc_ipi_msg.h
create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h
create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.c
create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.h
create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_pm.h
create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c
create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
index f98eb47..b66cf1f 100644
--- a/drivers/media/platform/Kconfig
+++ b/drivers/media/platform/Kconfig
@@ -154,6 +154,19 @@ config MEDIATEK_VPU
This driver provides downloading firmware vpu and
communicating with vpu.
+config VIDEO_MEDIATEK_VCODEC
+ tristate "Mediatek Video Codec driver"
+ depends on VIDEO_DEV && VIDEO_V4L2
+ depends on ARCH_MEDIATEK || COMPILE_TEST
+ select VIDEOBUF2_DMA_CONTIG
+ select V4L2_MEM2MEM_DEV
+ select MEDIATEK_VPU
+ default n
+ ---help---
+ Mediatek video codec driver for V4L2
+
+source "drivers/media/platform/mtk-vcodec/Kconfig"
+
config VIDEO_MEM2MEM_DEINTERLACE
tristate "Deinterlace support"
depends on VIDEO_DEV && VIDEO_V4L2 && DMA_ENGINE
diff --git a/drivers/media/platform/Makefile b/drivers/media/platform/Makefile
index 1b4c539..423b9f6 100644
--- a/drivers/media/platform/Makefile
+++ b/drivers/media/platform/Makefile
@@ -57,3 +57,6 @@ obj-$(CONFIG_VIDEO_XILINX) += xilinx/
ccflags-y += -I$(srctree)/drivers/media/i2c
obj-$(CONFIG_MEDIATEK_VPU) += mtk-vpu/
+
+obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += mtk-vcodec/
+
diff --git a/drivers/media/platform/mtk-vcodec/Kconfig b/drivers/media/platform/mtk-vcodec/Kconfig
new file mode 100644
index 0000000..1c0b935
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/Kconfig
@@ -0,0 +1,5 @@
+config MEDIATEK_VPU
+ bool
+ ---help---
+ This driver provides downloading firmware vpu (video processor unit)
+ and communicating with vpu.
diff --git a/drivers/media/platform/mtk-vcodec/Makefile b/drivers/media/platform/mtk-vcodec/Makefile
new file mode 100644
index 0000000..c7f7174
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/Makefile
@@ -0,0 +1,12 @@
+obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += mtk_vcodec_intr.o \
+ mtk_vcodec_util.o \
+ mtk_vcodec_enc_drv.o \
+ mtk_vcodec_enc.o \
+ mtk_vcodec_enc_pm.o
+
+obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += common/
+
+ccflags-y += -I$(srctree)/drivers/media/platform/mtk-vcodec/include \
+ -I$(srctree)/drivers/media/platform/mtk-vcodec \
+ -I$(srctree)/drivers/media/platform/mtk-vpu
+
diff --git a/drivers/media/platform/mtk-vcodec/common/Makefile b/drivers/media/platform/mtk-vcodec/common/Makefile
new file mode 100644
index 0000000..477ab80
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/common/Makefile
@@ -0,0 +1,8 @@
+obj-y += \
+ venc_drv_if.o
+
+ccflags-y += \
+ -I$(srctree)/include/ \
+ -I$(srctree)/drivers/media/platform/mtk-vcodec \
+ -I$(srctree)/drivers/media/platform/mtk-vcodec/include \
+ -I$(srctree)/drivers/media/platform/mtk-vpu
\ No newline at end of file
diff --git a/drivers/media/platform/mtk-vcodec/common/venc_drv_if.c b/drivers/media/platform/mtk-vcodec/common/venc_drv_if.c
new file mode 100644
index 0000000..9b3f025
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/common/venc_drv_if.c
@@ -0,0 +1,152 @@
+/*
+ * Copyright (c) 2015 MediaTek Inc.
+ * Author: Daniel Hsiao <daniel.hsiao@mediatek.com>
+ * Jungchang Tsao <jungchang.tsao@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/interrupt.h>
+#include <linux/kernel.h>
+#include <linux/slab.h>
+
+#include "mtk_vcodec_drv.h"
+#include "mtk_vcodec_enc.h"
+#include "mtk_vcodec_pm.h"
+#include "mtk_vcodec_util.h"
+#include "mtk_vpu_core.h"
+
+#include "venc_drv_if.h"
+#include "venc_drv_base.h"
+
+
+int venc_if_create(void *ctx, unsigned int fourcc, unsigned long *handle)
+{
+ struct venc_handle *h;
+ char str[10];
+
+ mtk_vcodec_fmt2str(fourcc, str);
+
+ h = kzalloc(sizeof(*h), GFP_KERNEL);
+ if (!h)
+ return -ENOMEM;
+
+ h->fourcc = fourcc;
+ h->ctx = ctx;
+ mtk_vcodec_debug(h, "fmt = %s handle = %p", str, h);
+
+ switch (fourcc) {
+ default:
+ mtk_vcodec_err(h, "invalid format %s", str);
+ goto err_out;
+ }
+
+ *handle = (unsigned long)h;
+ return 0;
+
+err_out:
+ kfree(h);
+ return -EINVAL;
+}
+
+int venc_if_init(unsigned long handle)
+{
+ int ret = 0;
+ struct venc_handle *h = (struct venc_handle *)handle;
+
+ mtk_vcodec_debug_enter(h);
+
+ mtk_venc_lock(h->ctx);
+ mtk_vcodec_enc_clock_on();
+ vpu_enable_clock(vpu_get_plat_device(h->ctx->dev->plat_dev));
+ ret = h->enc_if->init(h->ctx, (unsigned long *)&h->drv_handle);
+ vpu_disable_clock(vpu_get_plat_device(h->ctx->dev->plat_dev));
+ mtk_vcodec_enc_clock_off();
+ mtk_venc_unlock(h->ctx);
+
+ return ret;
+}
+
+int venc_if_set_param(unsigned long handle,
+ enum venc_set_param_type type, void *in)
+{
+ int ret = 0;
+ struct venc_handle *h = (struct venc_handle *)handle;
+
+ mtk_vcodec_debug(h, "type=%d", type);
+
+ mtk_venc_lock(h->ctx);
+ mtk_vcodec_enc_clock_on();
+ vpu_enable_clock(vpu_get_plat_device(h->ctx->dev->plat_dev));
+ ret = h->enc_if->set_param(h->drv_handle, type, in);
+ vpu_disable_clock(vpu_get_plat_device(h->ctx->dev->plat_dev));
+ mtk_vcodec_enc_clock_off();
+ mtk_venc_unlock(h->ctx);
+
+ return ret;
+}
+
+int venc_if_encode(unsigned long handle,
+ enum venc_start_opt opt, struct venc_frm_buf *frm_buf,
+ struct mtk_vcodec_mem *bs_buf,
+ struct venc_done_result *result)
+{
+ int ret = 0;
+ struct venc_handle *h = (struct venc_handle *)handle;
+ char str[10];
+
+ mtk_vcodec_fmt2str(h->fourcc, str);
+ mtk_vcodec_debug(h, "fmt=%s", str);
+
+ mtk_venc_lock(h->ctx);
+ mtk_vcodec_enc_clock_on();
+ vpu_enable_clock(vpu_get_plat_device(h->ctx->dev->plat_dev));
+ enable_irq((h->fourcc == V4L2_PIX_FMT_H264) ?
+ h->ctx->dev->enc_irq : h->ctx->dev->enc_lt_irq);
+ ret = h->enc_if->encode(h->drv_handle, opt, frm_buf, bs_buf, result);
+ disable_irq((h->fourcc == V4L2_PIX_FMT_H264) ?
+ h->ctx->dev->enc_irq : h->ctx->dev->enc_lt_irq);
+ vpu_disable_clock(vpu_get_plat_device(h->ctx->dev->plat_dev));
+ mtk_vcodec_enc_clock_off();
+ mtk_venc_unlock(h->ctx);
+
+ mtk_vcodec_debug(h, "ret=%d", ret);
+
+ return ret;
+}
+
+int venc_if_deinit(unsigned long handle)
+{
+ int ret = 0;
+ struct venc_handle *h = (struct venc_handle *)handle;
+
+ mtk_vcodec_debug_enter(h);
+
+ mtk_venc_lock(h->ctx);
+ mtk_vcodec_enc_clock_on();
+ vpu_enable_clock(vpu_get_plat_device(h->ctx->dev->plat_dev));
+ ret = h->enc_if->deinit(h->drv_handle);
+ vpu_disable_clock(vpu_get_plat_device(h->ctx->dev->plat_dev));
+ mtk_vcodec_enc_clock_off();
+ mtk_venc_unlock(h->ctx);
+
+ return ret;
+}
+
+int venc_if_release(unsigned long handle)
+{
+ struct venc_handle *h = (struct venc_handle *)handle;
+
+ mtk_vcodec_debug_enter(h);
+ kfree(h);
+
+ return 0;
+}
diff --git a/drivers/media/platform/mtk-vcodec/include/venc_drv_base.h b/drivers/media/platform/mtk-vcodec/include/venc_drv_base.h
new file mode 100644
index 0000000..8828294
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/include/venc_drv_base.h
@@ -0,0 +1,68 @@
+/*
+ * Copyright (c) 2015 MediaTek Inc.
+ * Author: Daniel Hsiao <daniel.hsiao@mediatek.com>
+ * Jungchang Tsao <jungchang.tsao@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef _VENC_DRV_BASE_
+#define _VENC_DRV_BASE_
+
+#include "mtk_vcodec_drv.h"
+
+#include "venc_drv_if.h"
+
+struct venc_common_if {
+ /**
+ * (*init)() - initialize driver
+ * @ctx: [in] mtk v4l2 context
+ * @handle: [out] driver handle
+ */
+ int (*init)(struct mtk_vcodec_ctx *ctx, unsigned long *handle);
+
+ /**
+ * (*encode)() - trigger encode
+ * @handle: [in] driver handle
+ * @opt: [in] encode option
+ * @frm_buf: [in] frame buffer to store input frame
+ * @bs_buf: [in] bitstream buffer to store output bitstream
+ * @result: [out] encode result
+ */
+ int (*encode)(unsigned long handle, enum venc_start_opt opt,
+ struct venc_frm_buf *frm_buf,
+ struct mtk_vcodec_mem *bs_buf,
+ struct venc_done_result *result);
+
+ /**
+ * (*set_param)() - set driver's parameter
+ * @handle: [in] driver handle
+ * @type: [in] parameter type
+ * @in: [in] buffer to store the parameter
+ */
+ int (*set_param)(unsigned long handle, enum venc_set_param_type type,
+ void *in);
+
+ /**
+ * (*deinit)() - deinitialize driver.
+ * @handle: [in] driver handle
+ */
+ int (*deinit)(unsigned long handle);
+};
+
+struct venc_handle {
+ unsigned int fourcc;
+ struct venc_common_if *enc_if;
+ unsigned long drv_handle;
+ struct mtk_vcodec_ctx *ctx;
+};
+
+#endif
diff --git a/drivers/media/platform/mtk-vcodec/include/venc_drv_if.h b/drivers/media/platform/mtk-vcodec/include/venc_drv_if.h
new file mode 100644
index 0000000..ef0b0b9
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/include/venc_drv_if.h
@@ -0,0 +1,187 @@
+/*
+ * Copyright (c) 2015 MediaTek Inc.
+ * Author: Daniel Hsiao <daniel.hsiao@mediatek.com>
+ * Jungchang Tsao <jungchang.tsao@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef _VENC_DRV_IF_H_
+#define _VENC_DRV_IF_H_
+
+#include "mtk_vcodec_util.h"
+
+/*
+ * enum venc_yuv_fmt - The type of input yuv format
+ * (VPU related: If you change the order, you must also update the VPU codes.)
+ * @VENC_YUV_FORMAT_420: 420 YUV format
+ * @VENC_YUV_FORMAT_YV12: YV12 YUV format
+ * @VENC_YUV_FORMAT_NV12: NV12 YUV format
+ * @VENC_YUV_FORMAT_NV21: NV21 YUV format
+ */
+enum venc_yuv_fmt {
+ VENC_YUV_FORMAT_420 = 3,
+ VENC_YUV_FORMAT_YV12 = 5,
+ VENC_YUV_FORMAT_NV12 = 6,
+ VENC_YUV_FORMAT_NV21 = 7,
+};
+
+/*
+ * enum venc_start_opt - encode frame option used in venc_if_encode()
+ * @VENC_START_OPT_ENCODE_SEQUENCE_HEADER: encode SPS/PPS for H264
+ * @VENC_START_OPT_ENCODE_FRAME: encode normal frame
+ */
+enum venc_start_opt {
+ VENC_START_OPT_ENCODE_SEQUENCE_HEADER,
+ VENC_START_OPT_ENCODE_FRAME,
+};
+
+/*
+ * enum venc_drv_msg - The type of encode frame status used in venc_if_encode()
+ * @VENC_MESSAGE_OK: encode ok
+ * @VENC_MESSAGE_ERR: encode error
+ */
+enum venc_drv_msg {
+ VENC_MESSAGE_OK,
+ VENC_MESSAGE_ERR,
+};
+
+/*
+ * enum venc_set_param_type - The type of set parameter used in venc_if_set_param()
+ * (VPU related: If you change the order, you must also update the VPU codes.)
+ * @VENC_SET_PARAM_ENC: set encoder parameters
+ * @VENC_SET_PARAM_FORCE_INTRA: set force intra frame
+ * @VENC_SET_PARAM_ADJUST_BITRATE: set to adjust bitrate (in bps)
+ * @VENC_SET_PARAM_ADJUST_FRAMERATE: set frame rate
+ * @VENC_SET_PARAM_I_FRAME_INTERVAL: set I frame interval
+ * @VENC_SET_PARAM_SKIP_FRAME: set H264 skip one frame
+ * @VENC_SET_PARAM_PREPEND_HEADER: set H264 prepend SPS/PPS before IDR
+ * @VENC_SET_PARAM_TS_MODE: set VP8 temporal scalability mode
+ */
+enum venc_set_param_type {
+ VENC_SET_PARAM_ENC,
+ VENC_SET_PARAM_FORCE_INTRA,
+ VENC_SET_PARAM_ADJUST_BITRATE,
+ VENC_SET_PARAM_ADJUST_FRAMERATE,
+ VENC_SET_PARAM_I_FRAME_INTERVAL,
+ VENC_SET_PARAM_SKIP_FRAME,
+ VENC_SET_PARAM_PREPEND_HEADER,
+ VENC_SET_PARAM_TS_MODE,
+};
+
+/*
+ * struct venc_enc_prm - encoder settings for VENC_SET_PARAM_ENC used in venc_if_set_param()
+ * @input_fourcc: input fourcc
+ * @h264_profile: V4L2 defined H.264 profile
+ * @h264_level: V4L2 defined H.264 level
+ * @width: image width
+ * @height: image height
+ * @buf_width: buffer width
+ * @buf_height: buffer height
+ * @frm_rate: frame rate
+ * @intra_period: intra frame period
+ * @bitrate: target bitrate in kbps
+ */
+struct venc_enc_prm {
+ enum venc_yuv_fmt input_fourcc;
+ unsigned int h264_profile;
+ unsigned int h264_level;
+ unsigned int width;
+ unsigned int height;
+ unsigned int buf_width;
+ unsigned int buf_height;
+ unsigned int frm_rate;
+ unsigned int intra_period;
+ unsigned int bitrate;
+};
+
+/*
+ * struct venc_frm_buf - frame buffer information used in venc_if_encode()
+ * @fb_addr: plane 0 frame buffer address
+ * @fb_addr1: plane 1 frame buffer address
+ * @fb_addr2: plane 2 frame buffer address
+ */
+struct venc_frm_buf {
+ struct mtk_vcodec_mem fb_addr;
+ struct mtk_vcodec_mem fb_addr1;
+ struct mtk_vcodec_mem fb_addr2;
+};
+
+/*
+ * struct venc_done_result - This is return information used in venc_if_encode()
+ * @msg: message, such as success or error code
+ * @bs_size: output bitstream size
+ * @is_key_frm: output is key frame or not
+ */
+struct venc_done_result {
+ enum venc_drv_msg msg;
+ unsigned int bs_size;
+ bool is_key_frm;
+};
+
+/*
+ * venc_if_create - Create the driver handle
+ * @ctx: device context
+ * @fourcc: encoder output format
+ * @handle: driver handle
+ * Return: 0 if creating handle successfully, otherwise it is failed.
+ */
+int venc_if_create(void *ctx, unsigned int fourcc, unsigned long *handle);
+
+/*
+ * venc_if_release - Release the driver handle
+ * @handle: driver handle
+ * Return: 0 if releasing handle successfully, otherwise it is failed.
+ */
+int venc_if_release(unsigned long handle);
+
+/*
+ * venc_if_init - Init the driver setting, alloc working memory ... etc.
+ * @handle: driver handle
+ * Return: 0 if init handle successfully, otherwise it is failed.
+ */
+int venc_if_init(unsigned long handle);
+
+/*
+ * venc_if_deinit - DeInit the driver setting, free working memory ... etc.
+ * @handle: driver handle
+ * Return: 0 if deinit handle successfully, otherwise it is failed.
+ */
+int venc_if_deinit(unsigned long handle);
+
+/*
+ * venc_if_set_param - Set parameter to driver
+ * @handle: driver handle
+ * @type: set type
+ * @in: input parameter
+ * @out: output parameter
+ * Return: 0 if setting param successfully, otherwise it is failed.
+ */
+int venc_if_set_param(unsigned long handle,
+ enum venc_set_param_type type,
+ void *in);
+
+/*
+ * venc_if_encode - Encode frame
+ * @handle: driver handle
+ * @opt: encode frame option
+ * @frm_buf: input frame buffer information
+ * @bs_buf: output bitstream buffer infomraiton
+ * @result: encode result
+ * Return: 0 if encoding frame successfully, otherwise it is failed.
+ */
+int venc_if_encode(unsigned long handle,
+ enum venc_start_opt opt,
+ struct venc_frm_buf *frm_buf,
+ struct mtk_vcodec_mem *bs_buf,
+ struct venc_done_result *result);
+
+#endif /* _VENC_DRV_IF_H_ */
diff --git a/drivers/media/platform/mtk-vcodec/include/venc_ipi_msg.h b/drivers/media/platform/mtk-vcodec/include/venc_ipi_msg.h
new file mode 100644
index 0000000..a345b98
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/include/venc_ipi_msg.h
@@ -0,0 +1,212 @@
+/*
+ * Copyright (c) 2015 MediaTek Inc.
+ * Author: Jungchang Tsao <jungchang.tsao@mediatek.com>
+ * Daniel Hsiao <daniel.hsiao@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef _VENC_IPI_MSG_H_
+#define _VENC_IPI_MSG_H_
+
+#define IPIMSG_H264_ENC_ID 0x100
+#define IPIMSG_VP8_ENC_ID 0x200
+
+#define AP_IPIMSG_VENC_BASE 0x20000
+#define VPU_IPIMSG_VENC_BASE 0x30000
+
+/**
+ * enum venc_ipi_msg_id - message id between AP and VPU
+ * (ipi stands for inter-processor interrupt)
+ * @AP_IPIMSG_XXX: AP to VPU cmd message id
+ * @VPU_IPIMSG_XXX_DONE: VPU ack AP cmd message id
+ */
+enum venc_ipi_msg_id {
+ AP_IPIMSG_H264_ENC_INIT = AP_IPIMSG_VENC_BASE +
+ IPIMSG_H264_ENC_ID,
+ AP_IPIMSG_H264_ENC_SET_PARAM,
+ AP_IPIMSG_H264_ENC_ENCODE,
+ AP_IPIMSG_H264_ENC_DEINIT,
+
+ AP_IPIMSG_VP8_ENC_INIT = AP_IPIMSG_VENC_BASE +
+ IPIMSG_VP8_ENC_ID,
+ AP_IPIMSG_VP8_ENC_SET_PARAM,
+ AP_IPIMSG_VP8_ENC_ENCODE,
+ AP_IPIMSG_VP8_ENC_DEINIT,
+
+ VPU_IPIMSG_H264_ENC_INIT_DONE = VPU_IPIMSG_VENC_BASE +
+ IPIMSG_H264_ENC_ID,
+ VPU_IPIMSG_H264_ENC_SET_PARAM_DONE,
+ VPU_IPIMSG_H264_ENC_ENCODE_DONE,
+ VPU_IPIMSG_H264_ENC_DEINIT_DONE,
+
+ VPU_IPIMSG_VP8_ENC_INIT_DONE = VPU_IPIMSG_VENC_BASE +
+ IPIMSG_VP8_ENC_ID,
+ VPU_IPIMSG_VP8_ENC_SET_PARAM_DONE,
+ VPU_IPIMSG_VP8_ENC_ENCODE_DONE,
+ VPU_IPIMSG_VP8_ENC_DEINIT_DONE,
+};
+
+/**
+ * struct venc_ap_ipi_msg_init - AP to VPU init cmd structure
+ * @msg_id: message id (AP_IPIMSG_XXX_ENC_INIT)
+ * @venc_inst: AP encoder instance (struct venc_vp8_handle/venc_h264_handle *)
+ */
+struct venc_ap_ipi_msg_init {
+ uint32_t msg_id;
+ uint32_t reserved;
+ uint64_t venc_inst;
+};
+
+/**
+ * struct venc_ap_ipi_msg_set_param - AP to VPU set_param cmd structure
+ * @msg_id: message id (AP_IPIMSG_XXX_ENC_SET_PARAM)
+ * @inst_id: VPU encoder instance id (struct venc_vp8_vpu_drv/venc_h264_vpu_drv *)
+ * @param_id: parameter id (venc_set_param_type)
+ * @data_item: number of items in the data array
+ * @data[8]: data array to store the set parameters
+ */
+struct venc_ap_ipi_msg_set_param {
+ uint32_t msg_id;
+ uint32_t inst_id;
+ uint32_t param_id;
+ uint32_t data_item;
+ uint32_t data[8];
+};
+
+/**
+ * struct venc_ap_ipi_msg_enc - AP to VPU enc cmd structure
+ * @msg_id: message id (AP_IPIMSG_XXX_ENC_ENCODE)
+ * @inst_id: VPU encoder instance id (struct venc_vp8_vpu_drv/venc_h264_vpu_drv *)
+ * @bs_mode: bitstream mode for h264
+ * (H264_BS_MODE_SPS/H264_BS_MODE_PPS/H264_BS_MODE_FRAME)
+ * @input_addr: pointer to input image buffer plane
+ * @bs_addr: pointer to output bit stream buffer
+ * @bs_size: bit stream buffer size
+ */
+struct venc_ap_ipi_msg_enc {
+ uint32_t msg_id;
+ uint32_t inst_id;
+ uint32_t bs_mode;
+ uint32_t input_addr[3];
+ uint32_t bs_addr;
+ uint32_t bs_size;
+};
+
+/**
+ * struct venc_ap_ipi_msg_deinit - AP to VPU deinit cmd structure
+ * @msg_id: message id (AP_IPIMSG_XXX_ENC_DEINIT)
+ * @inst_id: VPU encoder instance id (struct venc_vp8_vpu_drv/venc_h264_vpu_drv *)
+ */
+struct venc_ap_ipi_msg_deinit {
+ uint32_t msg_id;
+ uint32_t inst_id;
+};
+
+/**
+ * enum venc_ipi_msg_status - VPU ack AP cmd status
+ */
+enum venc_ipi_msg_status {
+ VENC_IPI_MSG_STATUS_OK,
+ VENC_IPI_MSG_STATUS_FAIL,
+};
+
+/**
+ * struct venc_vpu_ipi_msg_common - VPU ack AP cmd common structure
+ * @msg_id: message id (VPU_IPIMSG_XXX_DONE)
+ * @status: cmd status (venc_ipi_msg_status)
+ * @venc_inst: AP encoder instance (struct venc_vp8_handle/venc_h264_handle *)
+ */
+struct venc_vpu_ipi_msg_common {
+ uint32_t msg_id;
+ uint32_t status;
+ uint64_t venc_inst;
+};
+
+/**
+ * struct venc_vpu_ipi_msg_init - VPU ack AP init cmd structure
+ * @msg_id: message id (VPU_IPIMSG_XXX_ENC_SET_PARAM_DONE)
+ * @status: cmd status (venc_ipi_msg_status)
+ * @venc_inst: AP encoder instance (struct venc_vp8_handle/venc_h264_handle *)
+ * @inst_id: VPU encoder instance id (struct venc_vp8_vpu_drv/venc_h264_vpu_drv *)
+ */
+struct venc_vpu_ipi_msg_init {
+ uint32_t msg_id;
+ uint32_t status;
+ uint64_t venc_inst;
+ uint32_t inst_id;
+ uint32_t reserved;
+};
+
+/**
+ * struct venc_vpu_ipi_msg_set_param - VPU ack AP set_param cmd structure
+ * @msg_id: message id (VPU_IPIMSG_XXX_ENC_SET_PARAM_DONE)
+ * @status: cmd status (venc_ipi_msg_status)
+ * @venc_inst: AP encoder instance (struct venc_vp8_handle/venc_h264_handle *)
+ * @param_id: parameter id (venc_set_param_type)
+ * @data_item: number of items in the data array
+ * @data[6]: data array to store the return result
+ */
+struct venc_vpu_ipi_msg_set_param {
+ uint32_t msg_id;
+ uint32_t status;
+ uint64_t venc_inst;
+ uint32_t param_id;
+ uint32_t data_item;
+ uint32_t data[6];
+};
+
+/**
+ * enum venc_ipi_msg_enc_state - Type of encode state
+ * VEN_IPI_MSG_ENC_STATE_FRAME: one frame being encoded
+ * VEN_IPI_MSG_ENC_STATE_PART: bit stream buffer full
+ * VEN_IPI_MSG_ENC_STATE_SKIP: encoded skip frame
+ * VEN_IPI_MSG_ENC_STATE_ERROR: encounter error
+ */
+enum venc_ipi_msg_enc_state {
+ VEN_IPI_MSG_ENC_STATE_FRAME,
+ VEN_IPI_MSG_ENC_STATE_PART,
+ VEN_IPI_MSG_ENC_STATE_SKIP,
+ VEN_IPI_MSG_ENC_STATE_ERROR,
+};
+
+/**
+ * struct venc_vpu_ipi_msg_enc - VPU ack AP enc cmd structure
+ * @msg_id: message id (VPU_IPIMSG_XXX_ENC_ENCODE_DONE)
+ * @status: cmd status (venc_ipi_msg_status)
+ * @venc_inst: AP encoder instance (struct venc_vp8_handle/venc_h264_handle *)
+ * @state: encode state (venc_ipi_msg_enc_state)
+ * @key_frame: whether the encoded frame is key frame
+ * @bs_size: encoded bitstream size
+ */
+struct venc_vpu_ipi_msg_enc {
+ uint32_t msg_id;
+ uint32_t status;
+ uint64_t venc_inst;
+ uint32_t state;
+ uint32_t key_frame;
+ uint32_t bs_size;
+ uint32_t reserved;
+};
+
+/**
+ * struct venc_vpu_ipi_msg_deinit - VPU ack AP deinit cmd structure
+ * @msg_id: message id (VPU_IPIMSG_XXX_ENC_DEINIT_DONE)
+ * @status: cmd status (venc_ipi_msg_status)
+ * @venc_inst: AP encoder instance (struct venc_vp8_handle/venc_h264_handle *)
+ */
+struct venc_vpu_ipi_msg_deinit {
+ uint32_t msg_id;
+ uint32_t status;
+ uint64_t venc_inst;
+};
+
+#endif /* _VENC_IPI_MSG_H_ */
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
new file mode 100644
index 0000000..22239f8
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
@@ -0,0 +1,441 @@
+/*
+* Copyright (c) 2015 MediaTek Inc.
+* Author: PC Chen <pc.chen@mediatek.com>
+* Tiffany Lin <tiffany.lin@mediatek.com>
+*
+* This program is free software; you can redistribute it and/or modify
+* it under the terms of the GNU General Public License version 2 as
+* published by the Free Software Foundation.
+*
+* This program is distributed in the hope that it will be useful,
+* but WITHOUT ANY WARRANTY; without even the implied warranty of
+* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+* GNU General Public License for more details.
+*/
+
+#ifndef _MTK_VCODEC_DRV_H_
+#define _MTK_VCODEC_DRV_H_
+
+#include <linux/platform_device.h>
+#include <linux/videodev2.h>
+#include <media/v4l2-ctrls.h>
+#include <media/v4l2-device.h>
+#include <media/v4l2-ioctl.h>
+#include <media/videobuf2-core.h>
+#include <media/videobuf2-v4l2.h>
+
+#include "venc_drv_if.h"
+
+#define MTK_VCODEC_MAX_INSTANCES 32
+#define MTK_VCODEC_MAX_FRAME_SIZE 0x800000
+#define MTK_VIDEO_MAX_FRAME 32
+#define MTK_MAX_CTRLS 10
+
+#define MTK_VCODEC_DRV_NAME "mtk_vcodec_drv"
+#define MTK_VCODEC_ENC_NAME "mtk-vcodec-enc"
+
+#define MTK_VENC_IRQ_STATUS_SPS 0x1
+#define MTK_VENC_IRQ_STATUS_PPS 0x2
+#define MTK_VENC_IRQ_STATUS_FRM 0x4
+#define MTK_VENC_IRQ_STATUS_DRAM 0x8
+#define MTK_VENC_IRQ_STATUS_PAUSE 0x10
+#define MTK_VENC_IRQ_STATUS_SWITCH 0x20
+
+#define MTK_VENC_IRQ_STATUS_OFFSET 0x05C
+#define MTK_VENC_IRQ_ACK_OFFSET 0x060
+
+#define MTK_VCODEC_MAX_PLANES 3
+
+#define VDEC_HW_ACTIVE 0x10
+#define VDEC_IRQ_CFG 0x11
+#define VDEC_IRQ_CLR 0x10
+
+#define VDEC_IRQ_CFG_REG 0xa4
+#define NUM_MAX_ALLOC_CTX 4
+#define MTK_V4L2_BENCHMARK 0
+#define USE_ENCODE_THREAD 1
+
+/**
+ * enum mtk_hw_reg_idx - MTK hw register base index
+ */
+enum mtk_hw_reg_idx {
+ VDEC_SYS,
+ VDEC_MISC,
+ VDEC_LD,
+ VDEC_TOP,
+ VDEC_CM,
+ VDEC_AD,
+ VDEC_AV,
+ VDEC_PP,
+ VDEC_HWD,
+ VDEC_HWQ,
+ VDEC_HWB,
+ VDEC_HWG,
+ NUM_MAX_VDEC_REG_BASE,
+ VENC_SYS = NUM_MAX_VDEC_REG_BASE,
+ VENC_LT_SYS,
+ NUM_MAX_VCODEC_REG_BASE
+};
+
+/**
+ * enum mtk_instance_type - The type of an MTK Vcodec instance.
+ */
+enum mtk_instance_type {
+ MTK_INST_DECODER = 0,
+ MTK_INST_ENCODER = 1,
+};
+
+/**
+ * enum mtk_instance_state - The state of an MTK Vcodec instance.
+ * @MTK_STATE_FREE - default state when instance create
+ * @MTK_STATE_CREATE - vdec instance is create
+ * @MTK_STATE_INIT - vdec instance is init
+ * @MTK_STATE_CONFIG - reserved for encoder
+ * @MTK_STATE_HEADER - vdec had sps/pps header parsed
+ * @MTK_STATE_RUNNING - vdec is decoding
+ * @MTK_STATE_FLUSH - vdec is flushing
+ * @MTK_STATE_RES_CHANGE - vdec detect resolution change
+ * @MTK_STATE_FINISH - ctx instance is stopped streaming
+ * @MTK_STATE_DEINIT - before release ctx instance
+ * @MTK_STATE_ERROR - vdec has something wrong
+ * @MTK_STATE_ABORT - abort work in working thread
+ */
+enum mtk_instance_state {
+ MTK_STATE_FREE = 0,
+ MTK_STATE_CREATE = (1 << 0),
+ MTK_STATE_INIT = (1 << 1),
+ MTK_STATE_CONFIG = (1 << 2),
+ MTK_STATE_HEADER = (1 << 3),
+ MTK_STATE_RUNNING = (1 << 4),
+ MTK_STATE_FLUSH = (1 << 5),
+ MTK_STATE_RES_CHANGE = (1 << 6),
+ MTK_STATE_FINISH = (1 << 7),
+ MTK_STATE_DEINIT = (1 << 8),
+ MTK_STATE_ERROR = (1 << 9),
+ MTK_STATE_ABORT = (1 << 10),
+};
+
+/**
+ * struct mtk_param_change - General encoding parameters type
+ */
+enum mtk_encode_param {
+ MTK_ENCODE_PARAM_NONE = 0,
+ MTK_ENCODE_PARAM_BITRATE = (1 << 0),
+ MTK_ENCODE_PARAM_FRAMERATE = (1 << 1),
+ MTK_ENCODE_PARAM_INTRA_PERIOD = (1 << 2),
+ MTK_ENCODE_PARAM_FRAME_TYPE = (1 << 3),
+ MTK_ENCODE_PARAM_SKIP_FRAME = (1 << 4),
+};
+
+/**
+ * enum mtk_fmt_type - Type of the pixelformat
+ * @MTK_FMT_FRAME - mtk vcodec raw frame
+ */
+enum mtk_fmt_type {
+ MTK_FMT_DEC = 0,
+ MTK_FMT_ENC = 1,
+ MTK_FMT_FRAME = 2,
+};
+
+/**
+ * struct mtk_video_fmt - Structure used to store information about pixelformats
+ */
+struct mtk_video_fmt {
+ char *name;
+ u32 fourcc;
+ enum mtk_fmt_type type;
+ u32 num_planes;
+};
+
+/**
+ * struct mtk_codec_framesizes - Structure used to store information about framesizes
+ */
+struct mtk_codec_framesizes {
+ u32 fourcc;
+ struct v4l2_frmsize_stepwise stepwise;
+};
+
+/**
+ * struct mtk_q_type - Type of queue
+ */
+enum mtk_q_type {
+ MTK_Q_DATA_SRC = 0,
+ MTK_Q_DATA_DST = 1,
+};
+
+/**
+ * struct mtk_q_data - Structure used to store information about queue
+ * @colorspace reserved for encoder
+ * @field reserved for encoder
+ */
+struct mtk_q_data {
+ unsigned int width;
+ unsigned int height;
+ enum v4l2_field field;
+ enum v4l2_colorspace colorspace;
+ unsigned int bytesperline[MTK_VCODEC_MAX_PLANES];
+ unsigned int sizeimage[MTK_VCODEC_MAX_PLANES];
+ struct mtk_video_fmt *fmt;
+};
+
+/**
+ * struct mtk_enc_params - General encoding parameters
+ * @bitrate - target bitrate
+ * @num_b_frame - number of b frames between p-frame
+ * @rc_frame - frame based rate control
+ * @rc_mb - macroblock based rate control
+ * @seq_hdr_mode - H.264 sequence header is encoded separately or joined with the first frame
+ * @gop_size - group of picture size, it's used as the intra frame period
+ * @framerate_num - frame rate numerator
+ * @framerate_denom - frame rate denominator
+ * @h264_max_qp - Max value for H.264 quantization parameter
+ * @h264_profile - V4L2 defined H.264 profile
+ * @h264_level - V4L2 defined H.264 level
+ * @force_intra - force/insert intra frame
+ * @skip_frame - encode in skip frame mode that use minimum number of bits
+ */
+struct mtk_enc_params {
+ unsigned int bitrate;
+ unsigned int num_b_frame;
+ unsigned int rc_frame;
+ unsigned int rc_mb;
+ unsigned int seq_hdr_mode;
+ unsigned int gop_size;
+ unsigned int framerate_num;
+ unsigned int framerate_denom;
+ unsigned int h264_max_qp;
+ unsigned int h264_profile;
+ unsigned int h264_level;
+ unsigned int force_intra;
+ unsigned int skip_frame;
+};
+
+/**
+ * struct mtk_vcodec_pm - Power management data structure
+ */
+struct mtk_vcodec_pm {
+ struct clk *mmpll;
+ struct clk *vcodecpll;
+ struct clk *univpll_d2;
+ struct clk *clk_cci400_sel;
+ struct clk *vdecpll;
+ struct clk *vdec_sel;
+ struct clk *vencpll;
+ struct clk *venc_lt_sel;
+ struct clk *vcodecpll_370p5_ck;
+ struct device *larbvdec;
+ struct device *larbvenc;
+ struct device *larbvenclt;
+ struct device *dev;
+ struct mtk_vcodec_dev *mtkdev;
+};
+
+/**
+ * struct mtk_video_enc_buf - Private data related to each VB2 buffer.
+ * @b: Pointer to related VB2 buffer.
+ * @param_change: Types of encode parameter change before encode this
+ * buffer
+ * @enc_params Encode parameters changed before encode this buffer
+ */
+struct mtk_video_enc_buf {
+ struct vb2_v4l2_buffer b;
+ struct list_head list;
+
+ enum mtk_encode_param param_change;
+ struct mtk_enc_params enc_params;
+};
+
+/**
+ * struct mtk_vcodec_ctx - Context (instance) private data.
+ *
+ * @type: type of the instance - decoder or encoder
+ * @dev: pointer to the mtk_vcodec_dev of the device
+ * @fh: struct v4l2_fh
+ * @m2m_ctx: pointer to the v4l2_m2m_ctx of the context
+ * @q_data: store information of input and output queue
+ * of the context
+ * @idx: index of the context that this structure describes
+ * @state: state of the context
+ * @aborting:
+ * @param_change:
+ * @param_change_mutex:
+ * @enc_params: encoding parameters
+ * @colorspace:
+ *
+ * @h_enc: encoder handler
+ * @picinfo: store width/height of image and buffer and planes' size for decoder
+ * and encoder
+ * @pb_count: count of the DPB buffers required by MTK Vcodec hw
+ * @hdr:
+ *
+ * @int_cond: variable used by the waitqueue
+ * @int_type: type of the last interrupt
+ * @queue: waitqueue that can be used to wait for this context to
+ * finish
+ * @irq_status:
+ *
+ * @display_time_info: display time for destination buffers
+ * @in_time_idx: latest insert display time index
+ * @out_time_idx: first remove display time index
+ *
+ * @ctrl_hdl: handler for v4l2 framework
+ * @ctrls: array of controls, used when adding controls to the
+ * v4l2 control framework
+ *
+ * @encode_work: worker for the encoding
+ */
+struct mtk_vcodec_ctx {
+ enum mtk_instance_type type;
+ struct mtk_vcodec_dev *dev;
+ struct v4l2_fh fh;
+ struct v4l2_m2m_ctx *m2m_ctx;
+ struct mtk_q_data q_data[2];
+ int idx;
+ enum mtk_instance_state state;
+ int aborting;
+ enum mtk_encode_param param_change;
+ struct mutex encode_param_mutex;
+ struct mutex vb2_mutex;
+ struct mtk_enc_params enc_params;
+
+ unsigned long h_enc;
+ int hdr;
+
+ int int_cond;
+ int int_type;
+ wait_queue_head_t queue;
+ unsigned int irq_status;
+
+ struct v4l2_ctrl_handler ctrl_hdl;
+ struct v4l2_ctrl *ctrls[MTK_MAX_CTRLS];
+
+ struct work_struct encode_work;
+
+#if MTK_V4L2_BENCHMARK
+ unsigned int total_enc_dec_cnt;
+ unsigned int total_enc_dec_time;
+ unsigned int total_enc_hdr_time;
+ unsigned int total_enc_dec_init_time;
+
+ unsigned int total_qbuf_out_time;
+ unsigned int total_qbuf_cap_time;
+ unsigned int total_qbuf_out_cnt;
+ unsigned int total_qbuf_cap_cnt;
+ unsigned int total_dqbuf_out_time;
+ unsigned int total_dqbuf_cap_time;
+ unsigned int total_dqbuf_out_cnt;
+ unsigned int total_dqbuf_cap_cnt;
+ unsigned int total_dqbuf_cnt;
+ unsigned int total_expbuf_time;
+#endif
+
+};
+
+/**
+ * struct mtk_vcodec_dev - driver data
+ * @v4l2_dev: V4L2 device to register video devices for.
+ * @vfd_enc: Video device for encoder.
+ *
+ * @m2m_dev_enc: m2m device for encoder.
+ * @plat_dev: platform device
+ * @alloc_ctx: VB2 allocator context
+ * (for allocations without kernel mapping).
+ * @ctx: array of driver contexts
+ *
+ * @curr_ctx: The context that is waiting for codec hardware
+ *
+ * @reg_base: Mapped address of MTK Vcodec registers.
+ *
+ * @instance_mask: used to mark which contexts are opened
+ * @num_instances: counter of active MTK Vcodec instances
+ *
+ * @encode_workqueue: encode work queue
+ *
+ * @int_cond: used to identify interrupt condition happen
+ * @int_type: used to identify what kind of interrupt condition happen
+ * @dev_mutex: video_device lock
+ * @queue: waitqueue for waiting for completion of device commands
+ *
+ * @enc_irq: encoder irq resource.
+ * @enc_lt_irq: encoder lt irq resource.
+ *
+ * @enc_mutex: encoder hardware lock.
+ *
+ * @pm: power management control
+ * @dec_capability: used to identify decode capability, ex: 4k
+ * @enc_capability: used to identify encode capability
+ */
+struct mtk_vcodec_dev {
+ struct v4l2_device v4l2_dev;
+ struct video_device *vfd_enc;
+
+ struct v4l2_m2m_dev *m2m_dev_enc;
+ struct platform_device *plat_dev;
+ struct vb2_alloc_ctx *alloc_ctx;
+ struct mtk_vcodec_ctx *ctx[MTK_VCODEC_MAX_INSTANCES];
+ int curr_ctx;
+ void __iomem *reg_base[NUM_MAX_VCODEC_REG_BASE];
+
+ unsigned long instance_mask[BITS_TO_LONGS(MTK_VCODEC_MAX_INSTANCES)];
+ int num_instances;
+
+ struct workqueue_struct *encode_workqueue;
+
+ int int_cond;
+ int int_type;
+ struct mutex dev_mutex;
+ wait_queue_head_t queue;
+
+ int enc_irq;
+ int enc_lt_irq;
+
+ struct mutex enc_mutex;
+ unsigned long enter_suspend;
+
+ struct mtk_vcodec_pm pm;
+ unsigned int dec_capability;
+ unsigned int enc_capability;
+};
+
+/**
+ * struct mtk_vcodec_ctrl - information about controls to be registered.
+ * @id: Control ID.
+ * @type: Type of the control.
+ * @name: Human readable name of the control.
+ * @minimum: Minimum value of the control.
+ * @maximum: Maximum value of the control.
+ * @step: Control value increase step.
+ * @menu_skip_mask: Mask of invalid menu positions.
+ * @default_value: Initial value of the control.
+ * @is_volatile: Control is volatile.
+ *
+ * See also struct v4l2_ctrl_config.
+ */
+struct mtk_vcodec_ctrl {
+ u32 id;
+ enum v4l2_ctrl_type type;
+ u8 name[32];
+ s32 minimum;
+ s32 maximum;
+ s32 step;
+ u32 menu_skip_mask;
+ s32 default_value;
+ u8 is_volatile;
+};
+
+static inline struct mtk_vcodec_ctx *fh_to_ctx(struct v4l2_fh *fh)
+{
+ return container_of(fh, struct mtk_vcodec_ctx, fh);
+}
+
+static inline struct mtk_vcodec_ctx *ctrl_to_ctx(struct v4l2_ctrl *ctrl)
+{
+ return container_of(ctrl->handler, struct mtk_vcodec_ctx, ctrl_hdl);
+}
+
+extern const struct v4l2_ioctl_ops mtk_vdec_ioctl_ops;
+extern const struct v4l2_m2m_ops mtk_vdec_m2m_ops;
+extern const struct v4l2_ioctl_ops mtk_venc_ioctl_ops;
+extern const struct v4l2_m2m_ops mtk_venc_m2m_ops;
+
+#endif /* _MTK_VCODEC_DRV_H_ */
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
new file mode 100644
index 0000000..8e1b6f0
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
@@ -0,0 +1,1773 @@
+/*
+* Copyright (c) 2015 MediaTek Inc.
+* Author: PC Chen <pc.chen@mediatek.com>
+* Tiffany Lin <tiffany.lin@mediatek.com>
+*
+* This program is free software; you can redistribute it and/or modify
+* it under the terms of the GNU General Public License version 2 as
+* published by the Free Software Foundation.
+*
+* This program is distributed in the hope that it will be useful,
+* but WITHOUT ANY WARRANTY; without even the implied warranty of
+* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+* GNU General Public License for more details.
+*/
+
+#include <media/videobuf2-dma-contig.h>
+#include <media/videobuf2-core.h>
+#include <media/v4l2-event.h>
+#include <media/v4l2-mem2mem.h>
+
+#include "mtk_vcodec_drv.h"
+#include "mtk_vcodec_enc.h"
+#include "mtk_vcodec_intr.h"
+#include "mtk_vcodec_util.h"
+#include "venc_drv_if.h"
+
+static void mtk_venc_worker(struct work_struct *work);
+
+static struct mtk_video_fmt mtk_video_formats[] = {
+ {
+ .name = "4:2:0 3 Planes Y/Cb/Cr",
+ .fourcc = V4L2_PIX_FMT_YUV420,
+ .type = MTK_FMT_FRAME,
+ .num_planes = 3,
+ },
+ {
+ .name = "4:2:0 3 Planes Y/Cr/Cb",
+ .fourcc = V4L2_PIX_FMT_YVU420,
+ .type = MTK_FMT_FRAME,
+ .num_planes = 3,
+ },
+ {
+ .name = "4:2:0 2 Planes Y/CbCr",
+ .fourcc = V4L2_PIX_FMT_NV12,
+ .type = MTK_FMT_FRAME,
+ .num_planes = 2,
+ },
+ {
+ .name = "4:2:0 2 Planes Y/CrCb",
+ .fourcc = V4L2_PIX_FMT_NV21,
+ .type = MTK_FMT_FRAME,
+ .num_planes = 2,
+ },
+ {
+ .name = "4:2:0 3 none contiguous Planes Y/Cb/Cr",
+ .fourcc = V4L2_PIX_FMT_YUV420M,
+ .type = MTK_FMT_FRAME,
+ .num_planes = 3,
+ },
+ {
+ .name = "4:2:0 3 none contiguous Planes Y/Cr/Cb",
+ .fourcc = V4L2_PIX_FMT_YVU420M,
+ .type = MTK_FMT_FRAME,
+ .num_planes = 3,
+ },
+ {
+ .name = "4:2:0 2 none contiguous Planes Y/CbCr",
+ .fourcc = V4L2_PIX_FMT_NV12M,
+ .type = MTK_FMT_FRAME,
+ .num_planes = 2,
+ },
+ {
+ .name = "4:2:0 2 none contiguous Planes Y/CrCb",
+ .fourcc = V4L2_PIX_FMT_NV21M,
+ .type = MTK_FMT_FRAME,
+ .num_planes = 2,
+ },
+ {
+ .name = "H264 Encoded Stream",
+ .fourcc = V4L2_PIX_FMT_H264,
+ .type = MTK_FMT_ENC,
+ .num_planes = 1,
+ },
+ {
+ .name = "VP8 Encoded Stream",
+ .fourcc = V4L2_PIX_FMT_VP8,
+ .type = MTK_FMT_ENC,
+ .num_planes = 1,
+ },
+};
+
+#define NUM_FORMATS ARRAY_SIZE(mtk_video_formats)
+
+static struct mtk_vcodec_ctrl controls[] = {
+ {
+ .id = V4L2_CID_MPEG_VIDEO_BITRATE,
+ .type = V4L2_CTRL_TYPE_INTEGER,
+ .minimum = 1,
+ .maximum = (1 << 30) - 1,
+ .step = 1,
+ .default_value = 4000000,
+ },
+ {
+ .id = V4L2_CID_MPEG_VIDEO_B_FRAMES,
+ .type = V4L2_CTRL_TYPE_INTEGER,
+ .minimum = 0,
+ .maximum = 2,
+ .step = 1,
+ .default_value = 0,
+ },
+ {
+ .id = V4L2_CID_MPEG_VIDEO_FRAME_RC_ENABLE,
+ .type = V4L2_CTRL_TYPE_BOOLEAN,
+ .minimum = 0,
+ .maximum = 1,
+ .step = 1,
+ .default_value = 1,
+ },
+ {
+ .id = V4L2_CID_MPEG_VIDEO_H264_MAX_QP,
+ .type = V4L2_CTRL_TYPE_INTEGER,
+ .minimum = 0,
+ .maximum = 51,
+ .step = 1,
+ .default_value = 51,
+ },
+ {
+ .id = V4L2_CID_MPEG_VIDEO_HEADER_MODE,
+ .type = V4L2_CTRL_TYPE_MENU,
+ .minimum = V4L2_MPEG_VIDEO_HEADER_MODE_SEPARATE,
+ .maximum = V4L2_MPEG_VIDEO_HEADER_MODE_JOINED_WITH_1ST_FRAME,
+ .default_value = V4L2_MPEG_VIDEO_HEADER_MODE_SEPARATE,
+ .menu_skip_mask = 0,
+ },
+ {
+ .id = V4L2_CID_MPEG_VIDEO_MB_RC_ENABLE,
+ .type = V4L2_CTRL_TYPE_BOOLEAN,
+ .minimum = 0,
+ .maximum = 1,
+ .step = 1,
+ .default_value = 0,
+ },
+ {
+ .id = V4L2_CID_MPEG_VIDEO_H264_PROFILE,
+ .type = V4L2_CTRL_TYPE_MENU,
+ .minimum = V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE,
+ .maximum = V4L2_MPEG_VIDEO_H264_PROFILE_HIGH,
+ .default_value = V4L2_MPEG_VIDEO_H264_PROFILE_MAIN,
+ },
+ {
+ .id = V4L2_CID_MPEG_VIDEO_H264_LEVEL,
+ .type = V4L2_CTRL_TYPE_MENU,
+ .minimum = V4L2_MPEG_VIDEO_H264_LEVEL_1_0,
+ .maximum = V4L2_MPEG_VIDEO_H264_LEVEL_4_2,
+ .default_value = V4L2_MPEG_VIDEO_H264_LEVEL_4_0,
+ },
+ {
+ .id = V4L2_CID_MPEG_VIDEO_H264_I_PERIOD,
+ .type = V4L2_CTRL_TYPE_INTEGER,
+ .minimum = 0,
+ .maximum = (1 << 16) - 1,
+ .step = 1,
+ .default_value = 30,
+ },
+ {
+ .id = V4L2_CID_MPEG_VIDEO_GOP_SIZE,
+ .type = V4L2_CTRL_TYPE_INTEGER,
+ .minimum = 0,
+ .maximum = (1 << 16) - 1,
+ .step = 1,
+ .default_value = 30,
+ },
+ {
+ .id = V4L2_CID_MPEG_MFC51_VIDEO_FORCE_FRAME_TYPE,
+ .type = V4L2_CTRL_TYPE_MENU,
+ .minimum = V4L2_MPEG_MFC51_VIDEO_FORCE_FRAME_TYPE_DISABLED,
+ .maximum = V4L2_MPEG_MFC51_VIDEO_FORCE_FRAME_TYPE_NOT_CODED,
+ .default_value =
+ V4L2_MPEG_MFC51_VIDEO_FORCE_FRAME_TYPE_DISABLED,
+ .menu_skip_mask = 0,
+ },
+ {
+ .id = V4L2_CID_MPEG_MFC51_VIDEO_FRAME_SKIP_MODE,
+ .type = V4L2_CTRL_TYPE_MENU,
+ .minimum = V4L2_MPEG_MFC51_VIDEO_FRAME_SKIP_MODE_DISABLED,
+ .maximum = V4L2_MPEG_MFC51_VIDEO_FRAME_SKIP_MODE_BUF_LIMIT,
+ .default_value = V4L2_MPEG_MFC51_VIDEO_FRAME_SKIP_MODE_DISABLED,
+ .menu_skip_mask = 0,
+ },
+};
+
+#define NUM_CTRLS ARRAY_SIZE(controls)
+
+static const struct mtk_codec_framesizes mtk_venc_framesizes[] = {
+ {
+ .fourcc = V4L2_PIX_FMT_H264,
+ .stepwise = { 160, 1920, 16, 128, 1088, 16 },
+ },
+ {
+ .fourcc = V4L2_PIX_FMT_VP8,
+ .stepwise = { 160, 1920, 16, 128, 1088, 16 },
+ },
+};
+
+#define NUM_SUPPORTED_FRAMESIZE ARRAY_SIZE(mtk_venc_framesizes)
+
+static int vidioc_venc_s_ctrl(struct v4l2_ctrl *ctrl)
+{
+ struct mtk_vcodec_ctx *ctx = ctrl_to_ctx(ctrl);
+ struct mtk_vcodec_dev *dev = ctx->dev;
+ struct mtk_enc_params *p = &ctx->enc_params;
+ int ret = 0;
+
+ mtk_v4l2_debug(1, "[%d] id = %d/%d, val = %d", ctrl->id,
+ ctx->idx, ctrl->id - V4L2_CID_MPEG_BASE, ctrl->val);
+
+ switch (ctrl->id) {
+ case V4L2_CID_MPEG_VIDEO_BITRATE:
+
+ mtk_v4l2_debug(1, "V4L2_CID_MPEG_VIDEO_BITRATE val = %d",
+ ctrl->val);
+
+ mutex_lock(&ctx->encode_param_mutex);
+ p->bitrate = ctrl->val;
+ ctx->param_change |= MTK_ENCODE_PARAM_BITRATE;
+ mutex_unlock(&ctx->encode_param_mutex);
+ break;
+
+ case V4L2_CID_MPEG_VIDEO_B_FRAMES:
+ mtk_v4l2_debug(1, "V4L2_CID_MPEG_VIDEO_B_FRAMES val = %d",
+ ctrl->val);
+ p->num_b_frame = ctrl->val;
+ break;
+
+ case V4L2_CID_MPEG_VIDEO_FRAME_RC_ENABLE:
+ mtk_v4l2_debug(1, "V4L2_CID_MPEG_VIDEO_FRAME_RC_ENABLE val = %d",
+ ctrl->val);
+ p->rc_frame = ctrl->val;
+ break;
+
+ case V4L2_CID_MPEG_VIDEO_H264_MAX_QP:
+ mtk_v4l2_debug(1, "V4L2_CID_MPEG_VIDEO_H264_MAX_QP val = %d",
+ ctrl->val);
+ p->h264_max_qp = ctrl->val;
+ break;
+
+ case V4L2_CID_MPEG_VIDEO_HEADER_MODE:
+ mtk_v4l2_debug(1, "V4L2_CID_MPEG_VIDEO_HEADER_MODE val = %d",
+ ctrl->val);
+ p->seq_hdr_mode = ctrl->val;
+ break;
+
+ case V4L2_CID_MPEG_VIDEO_MB_RC_ENABLE:
+ mtk_v4l2_debug(1, "V4L2_CID_MPEG_VIDEO_MB_RC_ENABLE val = %d",
+ ctrl->val);
+ p->rc_mb = ctrl->val;
+ break;
+
+ case V4L2_CID_MPEG_VIDEO_H264_PROFILE:
+ mtk_v4l2_debug(1, "V4L2_CID_MPEG_VIDEO_H264_PROFILE val = %d",
+ ctrl->val);
+ p->h264_profile = ctrl->val;
+ break;
+
+ case V4L2_CID_MPEG_VIDEO_H264_LEVEL:
+ mtk_v4l2_debug(1, "V4L2_CID_MPEG_VIDEO_H264_LEVEL val = %d",
+ ctrl->val);
+ p->h264_level = ctrl->val;
+ break;
+ case V4L2_CID_MPEG_VIDEO_H264_I_PERIOD:
+ mtk_v4l2_debug(1, "V4L2_CID_MPEG_VIDEO_H264_I_PERIOD val = %d",
+ ctrl->val);
+ case V4L2_CID_MPEG_VIDEO_GOP_SIZE:
+ mtk_v4l2_debug(1, "V4L2_CID_MPEG_VIDEO_GOP_SIZE val = %d",
+ ctrl->val);
+
+ mutex_lock(&ctx->encode_param_mutex);
+ p->gop_size = ctrl->val;
+ ctx->param_change |= MTK_ENCODE_PARAM_INTRA_PERIOD;
+ mutex_unlock(&ctx->encode_param_mutex);
+ break;
+ case V4L2_CID_MPEG_MFC51_VIDEO_FORCE_FRAME_TYPE:
+ mtk_v4l2_debug(1, "V4L2_CID_MPEG_MFC51_VIDEO_FORCE_FRAME_TYPE val = %d",
+ ctrl->val);
+ if (ctrl->val ==
+ V4L2_MPEG_MFC51_VIDEO_FORCE_FRAME_TYPE_NOT_CODED) {
+ v4l2_err(&dev->v4l2_dev, "unsupported frame type %x\n",
+ ctrl->val);
+ ret = -EINVAL;
+ break;
+ }
+ mutex_lock(&ctx->encode_param_mutex);
+ if (ctrl->val == V4L2_MPEG_MFC51_VIDEO_FORCE_FRAME_TYPE_I_FRAME)
+ p->force_intra = 1;
+ else if (ctrl->val ==
+ V4L2_MPEG_MFC51_VIDEO_FORCE_FRAME_TYPE_DISABLED)
+ p->force_intra = 0;
+ /* always allow user to insert I frame */
+ ctrl->val = V4L2_MPEG_MFC51_VIDEO_FORCE_FRAME_TYPE_DISABLED;
+ ctx->param_change |= MTK_ENCODE_PARAM_FRAME_TYPE;
+ mutex_unlock(&ctx->encode_param_mutex);
+ break;
+
+ case V4L2_CID_MPEG_MFC51_VIDEO_FRAME_SKIP_MODE:
+ mtk_v4l2_debug(1, "V4L2_CID_MPEG_MFC51_VIDEO_FRAME_SKIP_MODE val = %d",
+ ctrl->val);
+
+ mutex_lock(&ctx->encode_param_mutex);
+ if (ctrl->val == V4L2_MPEG_MFC51_VIDEO_FRAME_SKIP_MODE_DISABLED)
+ p->skip_frame = 0;
+ else
+ p->skip_frame = 1;
+ /* always allow user to skip frame */
+ ctrl->val = V4L2_MPEG_MFC51_VIDEO_FRAME_SKIP_MODE_DISABLED;
+ ctx->param_change |= MTK_ENCODE_PARAM_SKIP_FRAME;
+ mutex_unlock(&ctx->encode_param_mutex);
+ break;
+
+ default:
+ mtk_v4l2_err("Invalid control, id=%d, val=%d\n",
+ ctrl->id, ctrl->val);
+ ret = -EINVAL;
+ break;
+ }
+
+ return ret;
+}
+
+static const struct v4l2_ctrl_ops mtk_vcodec_enc_ctrl_ops = {
+ .s_ctrl = vidioc_venc_s_ctrl,
+};
+
+static int vidioc_enum_fmt(struct file *file, struct v4l2_fmtdesc *f,
+ bool out)
+{
+ struct mtk_video_fmt *fmt;
+ int i, j = 0;
+
+ for (i = 0; i < NUM_FORMATS; ++i) {
+ if (out && mtk_video_formats[i].type != MTK_FMT_FRAME)
+ continue;
+ else if (!out && mtk_video_formats[i].type != MTK_FMT_ENC)
+ continue;
+
+ if (j == f->index) {
+ fmt = &mtk_video_formats[i];
+ strlcpy(f->description, fmt->name,
+ sizeof(f->description));
+ f->pixelformat = fmt->fourcc;
+ mtk_v4l2_debug(1, "f->index=%d i=%d fmt->name=%s",
+ f->index, i, fmt->name);
+ return 0;
+ }
+ ++j;
+ }
+
+ return -EINVAL;
+}
+
+static int vidioc_enum_framesizes(struct file *file, void *fh,
+ struct v4l2_frmsizeenum *fsize)
+{
+ int i = 0;
+
+ for (i = 0; i < NUM_SUPPORTED_FRAMESIZE; ++i) {
+ if (fsize->pixel_format != mtk_venc_framesizes[i].fourcc)
+ continue;
+
+ if (!fsize->index) {
+ fsize->type = V4L2_FRMSIZE_TYPE_STEPWISE;
+ fsize->stepwise = mtk_venc_framesizes[i].stepwise;
+ mtk_v4l2_debug(1, "%d %d %d %d %d %d",
+ fsize->stepwise.min_width,
+ fsize->stepwise.max_width,
+ fsize->stepwise.step_width,
+ fsize->stepwise.min_height,
+ fsize->stepwise.max_height,
+ fsize->stepwise.step_height);
+ return 0;
+ }
+ }
+
+ return -EINVAL;
+}
+
+static int vidioc_enum_fmt_vid_cap_mplane(struct file *file, void *pirv,
+ struct v4l2_fmtdesc *f)
+{
+ return vidioc_enum_fmt(file, f, false);
+}
+
+static int vidioc_enum_fmt_vid_out_mplane(struct file *file, void *prov,
+ struct v4l2_fmtdesc *f)
+{
+ return vidioc_enum_fmt(file, f, true);
+}
+
+static int vidioc_venc_streamon(struct file *file, void *priv,
+ enum v4l2_buf_type type)
+{
+ struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
+
+ return v4l2_m2m_streamon(file, ctx->m2m_ctx, type);
+}
+
+static int vidioc_venc_streamoff(struct file *file, void *priv,
+ enum v4l2_buf_type type)
+{
+ struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
+
+ return v4l2_m2m_streamoff(file, ctx->m2m_ctx, type);
+}
+
+static int vidioc_venc_reqbufs(struct file *file, void *priv,
+ struct v4l2_requestbuffers *reqbufs)
+{
+ struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
+
+ mtk_v4l2_debug(1, "[%d]-> type=%d count=%d",
+ ctx->idx, reqbufs->type, reqbufs->count);
+
+ return v4l2_m2m_reqbufs(file, ctx->m2m_ctx, reqbufs);
+}
+
+static int vidioc_venc_querybuf(struct file *file, void *priv,
+ struct v4l2_buffer *buf)
+{
+ struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
+
+ return v4l2_m2m_querybuf(file, ctx->m2m_ctx, buf);
+}
+
+static int vidioc_venc_qbuf(struct file *file, void *priv,
+ struct v4l2_buffer *buf)
+{
+ struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
+ int ret;
+#if MTK_V4L2_BENCHMARK
+ struct timeval begin, end;
+
+ do_gettimeofday(&begin);
+#endif
+
+ ret = v4l2_m2m_qbuf(file, ctx->m2m_ctx, buf);
+
+#if MTK_V4L2_BENCHMARK
+ do_gettimeofday(&end);
+
+ if (buf->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
+ ctx->total_qbuf_cap_cnt++;
+ ctx->total_qbuf_cap_time +=
+ ((end.tv_sec - begin.tv_sec) * 1000000 +
+ end.tv_usec - begin.tv_usec);
+ }
+
+ if (buf->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
+ ctx->total_qbuf_out_cnt++;
+ ctx->total_qbuf_out_time +=
+ ((end.tv_sec - begin.tv_sec) * 1000000 +
+ end.tv_usec - begin.tv_usec);
+ }
+
+#endif
+
+ return ret;
+}
+
+static int vidioc_venc_dqbuf(struct file *file, void *priv,
+ struct v4l2_buffer *buf)
+{
+ struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
+ int ret;
+#if MTK_V4L2_BENCHMARK
+ struct timeval begin, end;
+
+ do_gettimeofday(&begin);
+#endif
+
+ ret = v4l2_m2m_dqbuf(file, ctx->m2m_ctx, buf);
+#if MTK_V4L2_BENCHMARK
+
+ do_gettimeofday(&end);
+ if (buf->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
+ ctx->total_dqbuf_cap_cnt++;
+ ctx->total_dqbuf_cap_time +=
+ ((end.tv_sec - begin.tv_sec) * 1000000 +
+ end.tv_usec - begin.tv_usec);
+ }
+
+ if (buf->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
+ ctx->total_dqbuf_out_cnt++;
+ ctx->total_dqbuf_out_time +=
+ ((end.tv_sec - begin.tv_sec) * 1000000 +
+ end.tv_usec - begin.tv_usec);
+ }
+
+#endif
+ return ret;
+}
+static int vidioc_venc_expbuf(struct file *file, void *priv,
+ struct v4l2_exportbuffer *eb)
+{
+ struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
+ int ret;
+#if MTK_V4L2_BENCHMARK
+ struct timeval begin, end;
+
+ do_gettimeofday(&begin);
+#endif
+
+ ret = v4l2_m2m_expbuf(file, ctx->m2m_ctx, eb);
+
+#if MTK_V4L2_BENCHMARK
+ do_gettimeofday(&end);
+ ctx->total_expbuf_time +=
+ ((end.tv_sec - begin.tv_sec) * 1000000 +
+ end.tv_usec - begin.tv_usec);
+#endif
+ return ret;
+}
+
+static int vidioc_venc_querycap(struct file *file, void *priv,
+ struct v4l2_capability *cap)
+{
+ strncpy(cap->driver, MTK_VCODEC_ENC_NAME, sizeof(cap->driver) - 1);
+ cap->bus_info[0] = 0;
+ cap->version = KERNEL_VERSION(1, 0, 0);
+ /*
+ * This is only a mem-to-mem video device. The capture and output
+ * device capability flags are left only for backward compatibility
+ * and are scheduled for removal.
+ */
+ cap->capabilities = V4L2_CAP_VIDEO_M2M_MPLANE | V4L2_CAP_STREAMING |
+ V4L2_CAP_VIDEO_CAPTURE_MPLANE |
+ V4L2_CAP_VIDEO_OUTPUT_MPLANE;
+
+ return 0;
+}
+
+static int vidioc_venc_subscribe_event(struct v4l2_fh *fh,
+ const struct v4l2_event_subscription *sub)
+{
+ return v4l2_event_subscribe(fh, sub, 0, NULL);
+}
+
+static int vidioc_venc_s_parm(struct file *file, void *priv,
+ struct v4l2_streamparm *a)
+{
+ struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
+
+ if (a->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
+ mutex_lock(&ctx->encode_param_mutex);
+ ctx->enc_params.framerate_num =
+ a->parm.output.timeperframe.denominator;
+ ctx->enc_params.framerate_denom =
+ a->parm.output.timeperframe.numerator;
+ ctx->param_change |= MTK_ENCODE_PARAM_FRAMERATE;
+ mutex_unlock(&ctx->encode_param_mutex);
+ mtk_v4l2_debug(1, "framerate = %d/%d",
+ ctx->enc_params.framerate_num,
+ ctx->enc_params.framerate_denom);
+ } else {
+ mtk_v4l2_err("Non support param type %d",
+ a->type);
+ return -EINVAL;
+ }
+ return 0;
+}
+
+static struct mtk_q_data *mtk_venc_get_q_data(struct mtk_vcodec_ctx *ctx,
+ enum v4l2_buf_type type)
+{
+ switch (type) {
+ case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE:
+ case V4L2_BUF_TYPE_VIDEO_OUTPUT:
+ return &ctx->q_data[MTK_Q_DATA_SRC];
+ case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE:
+ case V4L2_BUF_TYPE_VIDEO_CAPTURE:
+ return &ctx->q_data[MTK_Q_DATA_DST];
+ default:
+ BUG();
+ }
+ return NULL;
+}
+
+static struct mtk_video_fmt *mtk_venc_find_format(struct v4l2_format *f)
+{
+ struct mtk_video_fmt *fmt;
+ unsigned int k;
+
+ for (k = 0; k < NUM_FORMATS; k++) {
+ fmt = &mtk_video_formats[k];
+ if (fmt->fourcc == f->fmt.pix.pixelformat)
+ return fmt;
+ }
+
+ return NULL;
+}
+
+static int vidioc_try_fmt(struct file *file, void *priv, struct v4l2_format *f)
+{
+ struct mtk_video_fmt *fmt;
+ char str[10];
+ struct v4l2_pix_format_mplane *pix_fmt_mp = &f->fmt.pix_mp;
+
+ mtk_vcodec_fmt2str(f->fmt.pix_mp.pixelformat, str);
+
+ if (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
+ fmt = mtk_venc_find_format(f);
+ if (!fmt) {
+ mtk_v4l2_err("failed to try output format %s\n", str);
+ return -EINVAL;
+ }
+ if (pix_fmt_mp->plane_fmt[0].sizeimage == 0) {
+ mtk_v4l2_err("must be set encoding output size %s\n",
+ str);
+ return -EINVAL;
+ }
+
+ pix_fmt_mp->plane_fmt[0].bytesperline =
+ pix_fmt_mp->plane_fmt[0].sizeimage;
+ } else if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
+ fmt = mtk_venc_find_format(f);
+ if (!fmt) {
+ mtk_v4l2_err("failed to try output format %s\n", str);
+ return -EINVAL;
+ }
+
+ if (fmt->num_planes != pix_fmt_mp->num_planes) {
+ mtk_v4l2_err("failed to try output format %d %d %s\n",
+ fmt->num_planes, pix_fmt_mp->num_planes,
+ str);
+ return -EINVAL;
+ }
+
+ v4l_bound_align_image(&pix_fmt_mp->width, 8, 1920, 1,
+ &pix_fmt_mp->height, 4, 1080, 1, 0);
+ } else {
+ pr_err("invalid buf type %d\n", f->type);
+ return -EINVAL;
+ }
+ return 0;
+}
+
+static void mtk_vcodec_enc_calc_src_size(struct mtk_vcodec_ctx *ctx)
+{
+ struct mtk_video_fmt *fmt = ctx->q_data[MTK_Q_DATA_SRC].fmt;
+ struct mtk_q_data *q_data = &ctx->q_data[MTK_Q_DATA_SRC];
+
+ ctx->q_data[MTK_Q_DATA_SRC].sizeimage[0] =
+ ((q_data->width + 15) / 16) *
+ ((q_data->height + 15) / 16) * 256;
+ ctx->q_data[MTK_Q_DATA_SRC].bytesperline[0] = ALIGN(q_data->width, 16);
+
+ if (fmt->num_planes == 2) {
+ ctx->q_data[MTK_Q_DATA_SRC].sizeimage[1] =
+ ((q_data->width + 15) / 16) *
+ ((q_data->height + 15) / 16) * 128;
+ ctx->q_data[MTK_Q_DATA_SRC].sizeimage[2] = 0;
+ ctx->q_data[MTK_Q_DATA_SRC].bytesperline[1] =
+ ALIGN(q_data->width, 16);
+ ctx->q_data[MTK_Q_DATA_SRC].bytesperline[2] = 0;
+ } else {
+ ctx->q_data[MTK_Q_DATA_SRC].sizeimage[1] =
+ ((q_data->width + 15) / 16) *
+ ((q_data->height + 15) / 16) * 64;
+ ctx->q_data[MTK_Q_DATA_SRC].sizeimage[2] =
+ ((q_data->width + 15) / 16) *
+ ((q_data->height + 15) / 16) * 64;
+ ctx->q_data[MTK_Q_DATA_SRC].bytesperline[1] =
+ ALIGN(q_data->width, 16) / 2;
+ ctx->q_data[MTK_Q_DATA_SRC].bytesperline[2] =
+ ALIGN(q_data->width, 16) / 2;
+ }
+}
+
+static int vidioc_venc_s_fmt(struct file *file, void *priv,
+ struct v4l2_format *f)
+{
+ struct mtk_vcodec_dev *dev = video_drvdata(file);
+ struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
+ struct v4l2_pix_format_mplane *pix_fmt_mp = &f->fmt.pix_mp;
+ struct vb2_queue *vq;
+ struct mtk_q_data *q_data;
+ int i, ret;
+
+ ret = vidioc_try_fmt(file, priv, f);
+ if (ret)
+ return ret;
+
+ vq = v4l2_m2m_get_vq(ctx->m2m_ctx, f->type);
+ if (!vq) {
+ v4l2_err(&dev->v4l2_dev, "fail to get vq\n");
+ return -EINVAL;
+ }
+
+ if (vb2_is_busy(vq)) {
+ v4l2_err(&dev->v4l2_dev, "queue busy\n");
+ return -EBUSY;
+ }
+
+ q_data = mtk_venc_get_q_data(ctx, f->type);
+ if (!q_data) {
+ v4l2_err(&dev->v4l2_dev, "fail to get q data\n");
+ return -EINVAL;
+ }
+
+ q_data->fmt = mtk_venc_find_format(f);
+ if (!q_data->fmt) {
+ v4l2_err(&dev->v4l2_dev, "q data null format\n");
+ return -EINVAL;
+ }
+
+ q_data->width = f->fmt.pix_mp.width;
+ q_data->height = f->fmt.pix_mp.height;
+ q_data->colorspace = f->fmt.pix_mp.colorspace;
+ q_data->field = f->fmt.pix_mp.field;
+
+ for (i = 0; i < f->fmt.pix_mp.num_planes; i++) {
+ struct v4l2_plane_pix_format *plane_fmt;
+
+ plane_fmt = &f->fmt.pix_mp.plane_fmt[i];
+ q_data->bytesperline[i] = plane_fmt->bytesperline;
+ q_data->sizeimage[i] = plane_fmt->sizeimage;
+ }
+
+ if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
+ q_data->width = f->fmt.pix_mp.width;
+ q_data->height = f->fmt.pix_mp.height;
+
+ mtk_vcodec_enc_calc_src_size(ctx);
+ pix_fmt_mp->plane_fmt[0].sizeimage = q_data->sizeimage[0];
+ pix_fmt_mp->plane_fmt[0].bytesperline =
+ ctx->q_data[MTK_Q_DATA_SRC].bytesperline[0];
+ pix_fmt_mp->plane_fmt[1].sizeimage = q_data->sizeimage[1];
+ pix_fmt_mp->plane_fmt[1].bytesperline =
+ ctx->q_data[MTK_Q_DATA_SRC].bytesperline[1];
+ pix_fmt_mp->plane_fmt[2].sizeimage = q_data->sizeimage[2];
+ pix_fmt_mp->plane_fmt[2].bytesperline =
+ ctx->q_data[MTK_Q_DATA_SRC].bytesperline[2];
+ }
+
+ mtk_v4l2_debug(1,
+ "[%d]: t=%d wxh=%dx%d fmt=%c%c%c%c sz=0x%x-%x-%x",
+ ctx->idx,
+ f->type,
+ q_data->width, q_data->height,
+ (f->fmt.pix_mp.pixelformat & 0xff),
+ (f->fmt.pix_mp.pixelformat >> 8) & 0xff,
+ (f->fmt.pix_mp.pixelformat >> 16) & 0xff,
+ (f->fmt.pix_mp.pixelformat >> 24) & 0xff,
+ q_data->sizeimage[0],
+ q_data->sizeimage[1],
+ q_data->sizeimage[2]);
+
+ return 0;
+}
+
+static int vidioc_venc_g_fmt(struct file *file, void *priv,
+ struct v4l2_format *f)
+{
+ struct v4l2_pix_format_mplane *pix = &f->fmt.pix_mp;
+ struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
+ struct vb2_queue *vq;
+ struct mtk_q_data *q_data;
+ int i;
+
+ vq = v4l2_m2m_get_vq(ctx->m2m_ctx, f->type);
+ if (!vq)
+ return -EINVAL;
+
+ q_data = mtk_venc_get_q_data(ctx, f->type);
+
+ pix->width = q_data->width;
+ pix->height = q_data->height;
+ pix->pixelformat = q_data->fmt->fourcc;
+ pix->field = q_data->field;
+ pix->colorspace = q_data->colorspace;
+ pix->num_planes = q_data->fmt->num_planes;
+
+ for (i = 0; i < pix->num_planes; i++) {
+ pix->plane_fmt[i].bytesperline = q_data->bytesperline[i];
+ pix->plane_fmt[i].sizeimage = q_data->sizeimage[i];
+ }
+
+ mtk_v4l2_debug(1,
+ "[%d]<- type=%d wxh=%dx%d fmt=%c%c%c%c sz[0]=0x%x sz[1]=0x%x",
+ ctx->idx, f->type,
+ pix->width, pix->height,
+ (pix->pixelformat & 0xff),
+ (pix->pixelformat >> 8) & 0xff,
+ (pix->pixelformat >> 16) & 0xff,
+ (pix->pixelformat >> 24) & 0xff,
+ pix->plane_fmt[0].sizeimage,
+ pix->plane_fmt[1].sizeimage);
+
+ return 0;
+}
+
+static int vidioc_venc_g_ctrl(struct file *file, void *fh,
+ struct v4l2_control *ctrl)
+{
+ int ret = 0;
+
+ switch (ctrl->id) {
+ case V4L2_CID_MIN_BUFFERS_FOR_CAPTURE:
+ case V4L2_CID_MIN_BUFFERS_FOR_OUTPUT:
+ ctrl->value = 1;
+ break;
+ default:
+ ret = -EINVAL;
+ break;
+ }
+ return ret;
+}
+
+static int vidioc_venc_s_crop(struct file *file, void *fh,
+ const struct v4l2_crop *a)
+{
+ struct mtk_vcodec_ctx *ctx = fh_to_ctx(fh);
+ struct mtk_q_data *q_data;
+
+ if (a->c.left || a->c.top)
+ return -EINVAL;
+
+ q_data = mtk_venc_get_q_data(ctx, a->type);
+ if (!q_data)
+ return -EINVAL;
+
+ if (a->c.width != q_data->width ||
+ a->c.height != q_data->height) {
+ mtk_v4l2_err("crop right %d bottom %d w %d h %d\n",
+ a->c.width, a->c.height, q_data->width,
+ q_data->height);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static int vidioc_venc_g_crop(struct file *file, void *fh,
+ struct v4l2_crop *a)
+{
+ struct mtk_vcodec_ctx *ctx = fh_to_ctx(fh);
+ struct mtk_q_data *q_data;
+
+ if (a->c.left || a->c.top)
+ return -EINVAL;
+
+ q_data = mtk_venc_get_q_data(ctx, a->type);
+ if (!q_data)
+ return -EINVAL;
+
+ a->c.width = q_data->width;
+ a->c.height = q_data->height;
+
+ return 0;
+}
+
+
+const struct v4l2_ioctl_ops mtk_venc_ioctl_ops = {
+ .vidioc_streamon = vidioc_venc_streamon,
+ .vidioc_streamoff = vidioc_venc_streamoff,
+
+ .vidioc_reqbufs = vidioc_venc_reqbufs,
+ .vidioc_querybuf = vidioc_venc_querybuf,
+ .vidioc_qbuf = vidioc_venc_qbuf,
+ .vidioc_expbuf = vidioc_venc_expbuf,
+ .vidioc_dqbuf = vidioc_venc_dqbuf,
+
+ .vidioc_querycap = vidioc_venc_querycap,
+ .vidioc_enum_fmt_vid_cap_mplane = vidioc_enum_fmt_vid_cap_mplane,
+ .vidioc_enum_fmt_vid_out_mplane = vidioc_enum_fmt_vid_out_mplane,
+ .vidioc_enum_framesizes = vidioc_enum_framesizes,
+
+ .vidioc_subscribe_event = vidioc_venc_subscribe_event,
+
+ .vidioc_s_parm = vidioc_venc_s_parm,
+
+ .vidioc_s_fmt_vid_cap_mplane = vidioc_venc_s_fmt,
+ .vidioc_s_fmt_vid_out_mplane = vidioc_venc_s_fmt,
+
+ .vidioc_g_fmt_vid_cap_mplane = vidioc_venc_g_fmt,
+ .vidioc_g_fmt_vid_out_mplane = vidioc_venc_g_fmt,
+
+ .vidioc_g_ctrl = vidioc_venc_g_ctrl,
+
+ .vidioc_s_crop = vidioc_venc_s_crop,
+ .vidioc_g_crop = vidioc_venc_g_crop,
+
+};
+
+static int vb2ops_venc_queue_setup(struct vb2_queue *vq,
+ const void *parg,
+ unsigned int *nbuffers,
+ unsigned int *nplanes,
+ unsigned int sizes[], void *alloc_ctxs[])
+{
+ struct mtk_vcodec_ctx *ctx = vb2_get_drv_priv(vq);
+ struct mtk_q_data *q_data;
+
+ q_data = mtk_venc_get_q_data(ctx, vq->type);
+
+ if (*nbuffers < 1)
+ *nbuffers = 1;
+ if (*nbuffers > MTK_VIDEO_MAX_FRAME)
+ *nbuffers = MTK_VIDEO_MAX_FRAME;
+
+ *nplanes = q_data->fmt->num_planes;
+
+ if (vq->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
+ unsigned int i;
+
+ for (i = 0; i < *nplanes; i++) {
+ sizes[i] = q_data->sizeimage[i];
+ alloc_ctxs[i] = ctx->dev->alloc_ctx;
+ }
+ } else if (vq->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
+ sizes[0] = q_data->sizeimage[0];
+ alloc_ctxs[0] = ctx->dev->alloc_ctx;
+ } else {
+ BUG();
+ }
+
+ mtk_v4l2_debug(2,
+ "[%d]get %d buffer(s) of size 0x%x each",
+ ctx->idx, *nbuffers, sizes[0]);
+
+ return 0;
+}
+
+static int vb2ops_venc_buf_prepare(struct vb2_buffer *vb)
+{
+ struct mtk_vcodec_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue);
+ struct mtk_q_data *q_data;
+ int i;
+
+ q_data = mtk_venc_get_q_data(ctx, vb->vb2_queue->type);
+
+ for (i = 0; i < q_data->fmt->num_planes; i++) {
+ if (vb2_plane_size(vb, i) < q_data->sizeimage[i]) {
+ mtk_v4l2_debug(2,
+ "data will not fit into plane %d (%lu < %d)",
+ i, vb2_plane_size(vb, i),
+ q_data->sizeimage[i]);
+ return -EINVAL;
+ }
+ }
+
+ return 0;
+}
+
+static void vb2ops_venc_buf_queue(struct vb2_buffer *vb)
+{
+ struct mtk_vcodec_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue);
+ struct vb2_v4l2_buffer *v4l2_vb = to_vb2_v4l2_buffer(vb);
+ struct mtk_video_enc_buf *buf =
+ container_of(v4l2_vb, struct mtk_video_enc_buf, b);
+
+ mutex_lock(&ctx->encode_param_mutex);
+ if ((vb->vb2_queue->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) &&
+ (ctx->param_change != MTK_ENCODE_PARAM_NONE)) {
+ mtk_v4l2_debug(1,
+ "[%d] Before id=%d encode parameter change %x",
+ ctx->idx, vb->index,
+ ctx->param_change);
+ buf->param_change = ctx->param_change;
+ if (buf->param_change & MTK_ENCODE_PARAM_BITRATE) {
+ buf->enc_params.bitrate = ctx->enc_params.bitrate;
+ mtk_v4l2_debug(1, "change param br=%d",
+ buf->enc_params.bitrate);
+ }
+ if (ctx->param_change & MTK_ENCODE_PARAM_FRAMERATE) {
+ buf->enc_params.framerate_num =
+ ctx->enc_params.framerate_num;
+ buf->enc_params.framerate_denom =
+ ctx->enc_params.framerate_denom;
+ mtk_v4l2_debug(1, "change param fr=%d",
+ buf->enc_params.framerate_num /
+ buf->enc_params.framerate_denom);
+ }
+ if (ctx->param_change & MTK_ENCODE_PARAM_INTRA_PERIOD) {
+ buf->enc_params.gop_size = ctx->enc_params.gop_size;
+ mtk_v4l2_debug(1, "change param intra period=%d",
+ buf->enc_params.gop_size);
+ }
+ if (ctx->param_change & MTK_ENCODE_PARAM_FRAME_TYPE) {
+ buf->enc_params.force_intra =
+ ctx->enc_params.force_intra;
+ mtk_v4l2_debug(1, "change param force I=%d",
+ buf->enc_params.force_intra);
+ }
+ if (ctx->param_change & MTK_ENCODE_PARAM_SKIP_FRAME) {
+ buf->enc_params.skip_frame =
+ ctx->enc_params.skip_frame;
+ mtk_v4l2_debug(1, "change param skip frame=%d",
+ buf->enc_params.skip_frame);
+ }
+ ctx->param_change = MTK_ENCODE_PARAM_NONE;
+ }
+ mutex_unlock(&ctx->encode_param_mutex);
+ v4l2_m2m_buf_queue(ctx->m2m_ctx, to_vb2_v4l2_buffer(vb));
+}
+
+static void mtk_venc_set_param(struct mtk_vcodec_ctx *ctx, void *param)
+{
+ struct venc_enc_prm *p = (struct venc_enc_prm *)param;
+ struct mtk_q_data *q_data_src = &ctx->q_data[MTK_Q_DATA_SRC];
+ struct mtk_enc_params *enc_params = &ctx->enc_params;
+ unsigned int frame_rate;
+
+ mutex_lock(&ctx->encode_param_mutex);
+
+ frame_rate = enc_params->framerate_num / enc_params->framerate_denom;
+
+ switch (q_data_src->fmt->fourcc) {
+ case V4L2_PIX_FMT_YUV420:
+ case V4L2_PIX_FMT_YUV420M:
+ p->input_fourcc = VENC_YUV_FORMAT_420;
+ break;
+ case V4L2_PIX_FMT_YVU420:
+ case V4L2_PIX_FMT_YVU420M:
+ p->input_fourcc = VENC_YUV_FORMAT_YV12;
+ break;
+ case V4L2_PIX_FMT_NV12:
+ case V4L2_PIX_FMT_NV12M:
+ p->input_fourcc = VENC_YUV_FORMAT_NV12;
+ break;
+ case V4L2_PIX_FMT_NV21:
+ case V4L2_PIX_FMT_NV21M:
+ p->input_fourcc = VENC_YUV_FORMAT_NV21;
+ break;
+ }
+ p->h264_profile = enc_params->h264_profile;
+ p->h264_level = enc_params->h264_level;
+ p->width = q_data_src->width;
+ p->height = q_data_src->height;
+ p->buf_width = q_data_src->bytesperline[0];
+ p->buf_height = ((q_data_src->height + 0xf) & (~0xf));
+ p->frm_rate = frame_rate;
+ p->intra_period = enc_params->gop_size;
+ p->bitrate = enc_params->bitrate;
+
+ ctx->param_change = MTK_ENCODE_PARAM_NONE;
+ mutex_unlock(&ctx->encode_param_mutex);
+
+ mtk_v4l2_debug(1,
+ "fmt 0x%x P/L %d/%d w/h %d/%d buf %d/%d fps/bps %d/%d gop %d",
+ p->input_fourcc, p->h264_profile, p->h264_level, p->width,
+ p->height, p->buf_width, p->buf_height, p->frm_rate,
+ p->bitrate, p->intra_period);
+}
+
+static int vb2ops_venc_start_streaming(struct vb2_queue *q, unsigned int count)
+{
+ struct mtk_vcodec_ctx *ctx = vb2_get_drv_priv(q);
+ struct v4l2_device *v4l2_dev = &ctx->dev->v4l2_dev;
+ int ret;
+#if MTK_V4L2_BENCHMARK
+ struct timeval begin, end;
+
+ do_gettimeofday(&begin);
+#endif
+
+ if (!(vb2_start_streaming_called(&ctx->m2m_ctx->out_q_ctx.q) &
+ vb2_start_streaming_called(&ctx->m2m_ctx->cap_q_ctx.q))) {
+ mtk_v4l2_debug(1, "[%d]-> out=%d cap=%d",
+ ctx->idx,
+ vb2_start_streaming_called(&ctx->m2m_ctx->out_q_ctx.q),
+ vb2_start_streaming_called(&ctx->m2m_ctx->cap_q_ctx.q));
+ return 0;
+ }
+
+ if ((ctx->state & (MTK_STATE_ERROR | MTK_STATE_ABORT)))
+ return -EINVAL;
+
+ if (ctx->state == MTK_STATE_FREE) {
+ ret = venc_if_create(ctx,
+ ctx->q_data[MTK_Q_DATA_DST].fmt->fourcc,
+ &ctx->h_enc);
+
+ if (ret != 0) {
+ ctx->state |= MTK_STATE_ERROR;
+ v4l2_err(v4l2_dev, "invalid codec type=%x\n",
+ ctx->q_data[MTK_Q_DATA_DST].fmt->fourcc);
+ v4l2_err(v4l2_dev, "venc_if_create failed=%d\n", ret);
+ return -EINVAL;
+ }
+
+ if (ctx->q_data[MTK_Q_DATA_DST].fmt->fourcc ==
+ V4L2_PIX_FMT_H264)
+ ctx->hdr = 1;
+
+ ctx->state |= MTK_STATE_CREATE;
+ }
+
+ if ((ctx->state & MTK_STATE_CREATE) && !(ctx->state & MTK_STATE_INIT)) {
+ ret = venc_if_init(ctx->h_enc);
+ if (ret != 0) {
+ ctx->state |= MTK_STATE_ERROR;
+ v4l2_err(v4l2_dev, "venc_if_init failed=%d\n", ret);
+ return -EINVAL;
+ }
+ INIT_WORK(&ctx->encode_work, mtk_venc_worker);
+ ctx->state |= MTK_STATE_INIT;
+ }
+
+ if ((ctx->state & MTK_STATE_INIT) && !(ctx->state & MTK_STATE_CONFIG)) {
+ struct venc_enc_prm param;
+
+ mtk_venc_set_param(ctx, ¶m);
+ ret = venc_if_set_param(ctx->h_enc,
+ VENC_SET_PARAM_ENC, ¶m);
+ if (ret != 0) {
+ ctx->state |= MTK_STATE_ERROR;
+ v4l2_err(v4l2_dev, "venc_if_set_param failed=%d\n",
+ ret);
+ return -EINVAL;
+ }
+ if (ctx->hdr) {
+ if (ctx->enc_params.seq_hdr_mode ==
+ V4L2_MPEG_VIDEO_HEADER_MODE_SEPARATE) {
+ ctx->state |= MTK_STATE_CONFIG;
+ } else {
+ ret = venc_if_set_param(ctx->h_enc,
+ VENC_SET_PARAM_PREPEND_HEADER,
+ 0);
+ if (ret != 0) {
+ ctx->state |= MTK_STATE_ERROR;
+ v4l2_err(v4l2_dev,
+ "venc_if_set_param failed=%d\n",
+ ret);
+ return -EINVAL;
+ }
+ ctx->state |= (MTK_STATE_CONFIG |
+ MTK_STATE_HEADER);
+ }
+ } else {
+ ctx->state |= (MTK_STATE_CONFIG | MTK_STATE_HEADER);
+ }
+ }
+
+#if MTK_V4L2_BENCHMARK
+ do_gettimeofday(&end);
+ ctx->total_enc_dec_init_time =
+ ((end.tv_sec - begin.tv_sec) * 1000000 +
+ end.tv_usec - begin.tv_usec);
+
+#endif
+ if ((ctx->state & MTK_STATE_HEADER))
+ return 0;
+
+ if (!(ctx->state & MTK_STATE_CONFIG)) {
+ ctx->state |= MTK_STATE_ERROR;
+ mtk_v4l2_err("not configured, state=0x%x\n", ctx->state);
+ return -EINVAL;
+ }
+ return 0;
+}
+
+static void vb2ops_venc_stop_streaming(struct vb2_queue *q)
+{
+ struct mtk_vcodec_ctx *ctx = vb2_get_drv_priv(q);
+ struct v4l2_device *v4l2_dev = &ctx->dev->v4l2_dev;
+ struct vb2_buffer *src_buf, *dst_buf;
+ int retry;
+ int ret;
+
+ mtk_v4l2_debug(2, "[%d]-> type=%d", ctx->idx, q->type);
+
+ retry = 0;
+ while ((ctx->state & MTK_STATE_RUNNING) && (retry < 10)) {
+ mtk_vcodec_clean_ctx_int_flags(ctx);
+ ctx->state |= MTK_STATE_ABORT;
+ ret = mtk_vcodec_wait_for_done_ctx(ctx,
+ MTK_INST_WORK_THREAD_ABORT_DONE,
+ 200, true);
+ mtk_v4l2_debug(1, "[%d] (%d) state=%d, retry=%d",
+ ctx->idx, q->type, ctx->state, retry);
+ retry++;
+ }
+
+ if (q->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
+ while ((dst_buf = v4l2_m2m_dst_buf_remove(ctx->m2m_ctx))) {
+ dst_buf->planes[0].bytesused = 0;
+ v4l2_m2m_buf_done(to_vb2_v4l2_buffer(dst_buf), VB2_BUF_STATE_DONE);
+ }
+ } else {
+ while ((src_buf = v4l2_m2m_src_buf_remove(ctx->m2m_ctx)))
+ v4l2_m2m_buf_done(to_vb2_v4l2_buffer(src_buf), VB2_BUF_STATE_DONE);
+ }
+
+ /* todo : stop HW encoder and flush all buffers*/
+ if ((q->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE &&
+ vb2_is_streaming(&ctx->m2m_ctx->out_q_ctx.q)) ||
+ (q->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE &&
+ vb2_is_streaming(&ctx->m2m_ctx->cap_q_ctx.q))) {
+ mtk_v4l2_debug(1, "[%d]-> q type %d out=%d cap=%d",
+ ctx->idx, q->type,
+ vb2_is_streaming(&ctx->m2m_ctx->out_q_ctx.q),
+ vb2_is_streaming(&ctx->m2m_ctx->cap_q_ctx.q));
+ return;
+ }
+
+ ctx->state |= MTK_STATE_FINISH;
+
+ if ((ctx->state & MTK_STATE_INIT) && !(ctx->state & MTK_STATE_DEINIT)) {
+ ret = venc_if_deinit(ctx->h_enc);
+ if (ret != 0) {
+ ctx->state |= MTK_STATE_ERROR;
+ v4l2_err(v4l2_dev, "venc_if_deinit failed=%d\n", ret);
+ return;
+ }
+ ctx->state |= MTK_STATE_DEINIT;
+ }
+
+ if ((ctx->state & MTK_STATE_CREATE)) {
+ ret = 0;
+ switch (ctx->q_data[MTK_Q_DATA_DST].fmt->fourcc) {
+ case V4L2_PIX_FMT_H264:
+ case V4L2_PIX_FMT_VP8:
+ ret = venc_if_release(ctx->h_enc);
+ ctx->h_enc = 0;
+ break;
+ default:
+ ctx->state |= MTK_STATE_ERROR;
+ v4l2_err(v4l2_dev, "invalid codec type=%x\n",
+ ctx->q_data[MTK_Q_DATA_DST].fmt->fourcc);
+ break;
+ }
+ if (ret != 0) {
+ ctx->state |= MTK_STATE_ERROR;
+ v4l2_err(v4l2_dev, "venc_if_release failed=%d\n", ret);
+ return;
+ }
+ }
+
+ mtk_v4l2_debug(2, "[%d]-> state=0x%x", ctx->idx, ctx->state);
+
+ ctx->state = MTK_STATE_FREE;
+}
+
+static struct vb2_ops mtk_venc_vb2_ops = {
+ .queue_setup = vb2ops_venc_queue_setup,
+ .buf_prepare = vb2ops_venc_buf_prepare,
+ .buf_queue = vb2ops_venc_buf_queue,
+ .wait_prepare = vb2_ops_wait_prepare,
+ .wait_finish = vb2_ops_wait_finish,
+ .start_streaming = vb2ops_venc_start_streaming,
+ .stop_streaming = vb2ops_venc_stop_streaming,
+};
+
+static int mtk_venc_encode_header(void *priv)
+{
+ struct mtk_vcodec_ctx *ctx = priv;
+ struct v4l2_device *v4l2_dev = &ctx->dev->v4l2_dev;
+ int ret;
+ struct vb2_buffer *dst_buf;
+ struct mtk_vcodec_mem bs_buf;
+ struct venc_done_result enc_result;
+ int i;
+
+ dst_buf = v4l2_m2m_dst_buf_remove(ctx->m2m_ctx);
+ if (!dst_buf) {
+ ctx->state |= MTK_STATE_ERROR;
+ mtk_v4l2_debug(1, "No dst buffer");
+ return -EINVAL;
+ }
+
+ bs_buf.va = vb2_plane_vaddr(dst_buf, 0);
+ bs_buf.dma_addr = vb2_dma_contig_plane_dma_addr(dst_buf, 0);
+ bs_buf.size = (unsigned int)dst_buf->planes[0].length;
+
+ mtk_v4l2_debug(1,
+ "buf idx=%d va=0x%p dma_addr=0x%llx size=0x%lx",
+ dst_buf->index, bs_buf.va,
+ (u64)bs_buf.dma_addr, bs_buf.size);
+
+ ret = venc_if_encode(ctx->h_enc,
+ VENC_START_OPT_ENCODE_SEQUENCE_HEADER,
+ 0, &bs_buf, &enc_result);
+
+ if (ret != 0) {
+ dst_buf->planes[0].bytesused = 0;
+ ctx->state |= MTK_STATE_ERROR;
+ v4l2_m2m_buf_done(to_vb2_v4l2_buffer(dst_buf), VB2_BUF_STATE_ERROR);
+ v4l2_err(v4l2_dev, "venc_if_encode failed=%d", ret);
+ return -EINVAL;
+ }
+
+ ctx->state |= MTK_STATE_HEADER;
+ dst_buf->planes[0].bytesused = enc_result.bs_size;
+
+ mtk_v4l2_debug(1, "venc_if_encode header len=%d",
+ enc_result.bs_size);
+ for (i = 0; i < enc_result.bs_size; i++) {
+ unsigned char *p = (unsigned char *)bs_buf.va;
+
+ mtk_v4l2_debug(1, "buf[%d]=0x%2x", i, p[i]);
+ }
+
+ v4l2_m2m_buf_done(to_vb2_v4l2_buffer(dst_buf), VB2_BUF_STATE_DONE);
+
+ return 0;
+}
+
+static int mtk_venc_param_change(struct mtk_vcodec_ctx *ctx, void *priv)
+{
+ struct vb2_buffer *vb = priv;
+ struct vb2_v4l2_buffer *v4l2_vb = to_vb2_v4l2_buffer(vb);
+ struct mtk_video_enc_buf *buf =
+ container_of(v4l2_vb, struct mtk_video_enc_buf, b);
+ int ret = 0;
+
+ if (buf->param_change == MTK_ENCODE_PARAM_NONE)
+ return 0;
+
+ mtk_v4l2_debug(1, "encode parameters change id=%d", vb->index);
+ if (buf->param_change & MTK_ENCODE_PARAM_BITRATE) {
+ struct venc_enc_prm enc_prm;
+
+ enc_prm.bitrate = buf->enc_params.bitrate;
+ mtk_v4l2_debug(1, "change param br=%d",
+ enc_prm.bitrate);
+ ret |= venc_if_set_param(ctx->h_enc,
+ VENC_SET_PARAM_ADJUST_BITRATE,
+ &enc_prm);
+ }
+ if (buf->param_change & MTK_ENCODE_PARAM_FRAMERATE) {
+ struct venc_enc_prm enc_prm;
+
+ enc_prm.frm_rate = buf->enc_params.framerate_num /
+ buf->enc_params.framerate_denom;
+ mtk_v4l2_debug(1, "change param fr=%d",
+ enc_prm.frm_rate);
+ ret |= venc_if_set_param(ctx->h_enc,
+ VENC_SET_PARAM_ADJUST_FRAMERATE,
+ &enc_prm);
+ }
+ if (buf->param_change & MTK_ENCODE_PARAM_INTRA_PERIOD) {
+ mtk_v4l2_debug(1, "change param intra period=%d",
+ buf->enc_params.gop_size);
+ ret |= venc_if_set_param(ctx->h_enc,
+ VENC_SET_PARAM_I_FRAME_INTERVAL,
+ &buf->enc_params.gop_size);
+ }
+ if (buf->param_change & MTK_ENCODE_PARAM_FRAME_TYPE) {
+ mtk_v4l2_debug(1, "change param force I=%d",
+ buf->enc_params.force_intra);
+ if (buf->enc_params.force_intra)
+ ret |= venc_if_set_param(ctx->h_enc,
+ VENC_SET_PARAM_FORCE_INTRA,
+ 0);
+ }
+ if (buf->param_change & MTK_ENCODE_PARAM_SKIP_FRAME) {
+ mtk_v4l2_debug(1, "change param skip frame=%d",
+ buf->enc_params.skip_frame);
+ if (buf->enc_params.skip_frame)
+ ret |= venc_if_set_param(ctx->h_enc,
+ VENC_SET_PARAM_SKIP_FRAME,
+ 0);
+ }
+ buf->param_change = MTK_ENCODE_PARAM_NONE;
+
+ if (ret != 0) {
+ ctx->state |= MTK_STATE_ERROR;
+ mtk_v4l2_err("venc_if_set_param failed=%d\n", ret);
+ return -1;
+ }
+
+ return 0;
+}
+
+static void mtk_venc_worker(struct work_struct *work)
+{
+ struct mtk_vcodec_ctx *ctx = container_of(work, struct mtk_vcodec_ctx,
+ encode_work);
+ struct vb2_buffer *src_buf, *dst_buf;
+ struct vb2_v4l2_buffer *v4l2_vb;
+ struct venc_frm_buf frm_buf;
+ struct mtk_vcodec_mem bs_buf;
+ struct venc_done_result enc_result;
+ int ret;
+
+#if MTK_V4L2_BENCHMARK
+ struct timeval begin, end;
+ struct timeval begin1, end1;
+
+ do_gettimeofday(&begin);
+#endif
+
+ ctx->state |= MTK_STATE_RUNNING;
+
+ if (!(ctx->state & MTK_STATE_HEADER)) {
+#if MTK_V4L2_BENCHMARK
+ do_gettimeofday(&begin1);
+#endif
+ mtk_venc_encode_header(ctx);
+#if MTK_V4L2_BENCHMARK
+ do_gettimeofday(&end1);
+ ctx->total_enc_hdr_time +=
+ ((end1.tv_sec - begin1.tv_sec) * 1000000 +
+ end1.tv_usec - begin1.tv_usec);
+#endif
+
+ ctx->state &= ~MTK_STATE_RUNNING;
+ v4l2_m2m_job_finish(ctx->dev->m2m_dev_enc, ctx->m2m_ctx);
+ return;
+ }
+
+ if (ctx->state & MTK_STATE_ABORT) {
+ ctx->state &= ~MTK_STATE_RUNNING;
+ v4l2_m2m_job_finish(ctx->dev->m2m_dev_enc, ctx->m2m_ctx);
+ mtk_v4l2_debug(1, "[%d] [MTK_INST_ABORT]", ctx->idx);
+ ctx->int_cond = 1;
+ ctx->int_type = MTK_INST_WORK_THREAD_ABORT_DONE;
+ wake_up_interruptible(&ctx->queue);
+ return;
+ }
+
+ src_buf = v4l2_m2m_next_src_buf(ctx->m2m_ctx);
+ if (!src_buf)
+ return;
+
+ mtk_venc_param_change(ctx, src_buf);
+
+ dst_buf = v4l2_m2m_dst_buf_remove(ctx->m2m_ctx);
+ if (!dst_buf)
+ return;
+
+ frm_buf.fb_addr.va = vb2_plane_vaddr(src_buf, 0);
+ frm_buf.fb_addr.dma_addr = vb2_dma_contig_plane_dma_addr(src_buf, 0);
+ frm_buf.fb_addr.size = (unsigned int)src_buf->planes[0].length;
+ frm_buf.fb_addr1.va = vb2_plane_vaddr(src_buf, 1);
+ frm_buf.fb_addr1.dma_addr = vb2_dma_contig_plane_dma_addr(src_buf, 1);
+ frm_buf.fb_addr1.size = (unsigned int)src_buf->planes[1].length;
+ if (src_buf->num_planes == 3) {
+ frm_buf.fb_addr2.va = vb2_plane_vaddr(src_buf, 2);
+ frm_buf.fb_addr2.dma_addr =
+ vb2_dma_contig_plane_dma_addr(src_buf, 2);
+ frm_buf.fb_addr2.size =
+ (unsigned int)src_buf->planes[2].length;
+ }
+ bs_buf.va = vb2_plane_vaddr(dst_buf, 0);
+ bs_buf.dma_addr = vb2_dma_contig_plane_dma_addr(dst_buf, 0);
+ bs_buf.size = (unsigned int)dst_buf->planes[0].length;
+
+ mtk_v4l2_debug(1,
+ "Framebuf VA=%p PA=%llx Size=0x%lx;VA=%p PA=0x%llx Size=0x%lx;VA=%p PA=0x%llx Size=0x%lx",
+ frm_buf.fb_addr.va,
+ (u64)frm_buf.fb_addr.dma_addr,
+ frm_buf.fb_addr.size,
+ frm_buf.fb_addr1.va,
+ (u64)frm_buf.fb_addr1.dma_addr,
+ frm_buf.fb_addr1.size,
+ frm_buf.fb_addr2.va,
+ (u64)frm_buf.fb_addr2.dma_addr,
+ frm_buf.fb_addr2.size);
+
+ ret = venc_if_encode(ctx->h_enc, VENC_START_OPT_ENCODE_FRAME,
+ &frm_buf, &bs_buf, &enc_result);
+
+ switch (enc_result.msg) {
+ case VENC_MESSAGE_OK:
+ src_buf = v4l2_m2m_src_buf_remove(ctx->m2m_ctx);
+ v4l2_m2m_buf_done(to_vb2_v4l2_buffer(src_buf), VB2_BUF_STATE_DONE);
+ dst_buf->planes[0].bytesused = enc_result.bs_size;
+ break;
+ default:
+ src_buf = v4l2_m2m_src_buf_remove(ctx->m2m_ctx);
+ v4l2_m2m_buf_done(to_vb2_v4l2_buffer(src_buf), VB2_BUF_STATE_ERROR);
+ dst_buf->planes[0].bytesused = 0;
+ break;
+ }
+ if (enc_result.is_key_frm) {
+ v4l2_vb = to_vb2_v4l2_buffer(dst_buf);
+ v4l2_vb->flags |= V4L2_BUF_FLAG_KEYFRAME;
+ }
+
+ if (ret != 0) {
+ ctx->state |= MTK_STATE_ERROR;
+ v4l2_m2m_buf_done(to_vb2_v4l2_buffer(dst_buf), VB2_BUF_STATE_ERROR);
+ mtk_v4l2_err("venc_if_encode failed=%d", ret);
+ } else {
+ v4l2_m2m_buf_done(to_vb2_v4l2_buffer(dst_buf), VB2_BUF_STATE_DONE);
+ mtk_v4l2_debug(1, "venc_if_encode bs size=%d",
+ enc_result.bs_size);
+ }
+
+#if MTK_V4L2_BENCHMARK
+ do_gettimeofday(&end);
+ ctx->total_enc_dec_cnt++;
+ ctx->total_enc_dec_time +=
+ ((end.tv_sec - begin.tv_sec) * 1000000 +
+ end.tv_usec - begin.tv_usec);
+#endif
+
+ ctx->state &= ~MTK_STATE_RUNNING;
+ v4l2_m2m_job_finish(ctx->dev->m2m_dev_enc, ctx->m2m_ctx);
+
+ mtk_v4l2_debug(1, "<=== src_buf[%d] dst_buf[%d] venc_if_encode ret=%d Size=%u===>",
+ src_buf->index, dst_buf->index, ret,
+ enc_result.bs_size);
+
+}
+
+static void m2mops_venc_device_run(void *priv)
+{
+ struct mtk_vcodec_ctx *ctx = priv;
+#ifdef USE_ENCODE_THREAD
+ queue_work(ctx->dev->encode_workqueue, &ctx->encode_work);
+#else
+ mtk_venc_worker(&ctx->encode_work);
+#endif
+}
+
+static int m2mops_venc_job_ready(void *m2m_priv)
+{
+ struct mtk_vcodec_ctx *ctx = m2m_priv;
+
+ if (!v4l2_m2m_num_dst_bufs_ready(ctx->m2m_ctx)) {
+ mtk_v4l2_debug(3,
+ "[%d]Not ready: not enough video dst buffers.",
+ ctx->idx);
+ return 0;
+ }
+
+ if (!v4l2_m2m_num_src_bufs_ready(ctx->m2m_ctx)) {
+ if ((ctx->state & MTK_STATE_HEADER)) {
+ mtk_v4l2_debug(3,
+ "[%d]Not ready: not enough video src buffers.",
+ ctx->idx);
+ return 0;
+ }
+ }
+
+ if ((ctx->state & (MTK_STATE_ERROR | MTK_STATE_ABORT))) {
+ mtk_v4l2_debug(3,
+ "[%d]Not ready: state=0x%x.",
+ ctx->idx, ctx->state);
+ return 0;
+ }
+
+ if (!(ctx->state & MTK_STATE_CONFIG)) {
+ mtk_v4l2_debug(3,
+ "[%d]Not ready: state=0x%x.",
+ ctx->idx, ctx->state);
+ return 0;
+ }
+
+ mtk_v4l2_debug(3, "[%d]ready!", ctx->idx);
+
+ return 1;
+}
+
+static void m2mops_venc_job_abort(void *priv)
+{
+ struct mtk_vcodec_ctx *ctx = priv;
+
+ mtk_v4l2_debug(3, "[%d]type=%d", ctx->idx, ctx->type);
+
+ ctx->aborting = 1;
+ ctx->state |= MTK_STATE_ABORT;
+
+ v4l2_m2m_job_finish(ctx->dev->m2m_dev_enc, ctx->m2m_ctx);
+}
+
+static void m2mops_venc_lock(void *m2m_priv)
+{
+ struct mtk_vcodec_ctx *ctx = m2m_priv;
+
+ mutex_lock(&ctx->dev->dev_mutex);
+}
+
+static void m2mops_venc_unlock(void *m2m_priv)
+{
+ struct mtk_vcodec_ctx *ctx = m2m_priv;
+
+ mutex_unlock(&ctx->dev->dev_mutex);
+}
+
+const struct v4l2_m2m_ops mtk_venc_m2m_ops = {
+ .device_run = m2mops_venc_device_run,
+ .job_ready = m2mops_venc_job_ready,
+ .job_abort = m2mops_venc_job_abort,
+ .lock = m2mops_venc_lock,
+ .unlock = m2mops_venc_unlock,
+};
+
+#define IS_MTK_VENC_PRIV(x) ((V4L2_CTRL_ID2CLASS(x) == V4L2_CTRL_CLASS_MPEG) &&\
+ V4L2_CTRL_DRIVER_PRIV(x))
+
+static const char *const *mtk_vcodec_enc_get_menu(u32 id)
+{
+ static const char *const mtk_vcodec_enc_video_frame_skip[] = {
+ "Disabled",
+ "Level Limit",
+ "VBV/CPB Limit",
+ NULL,
+ };
+ static const char *const mtk_vcodec_enc_video_force_frame[] = {
+ "Disabled",
+ "I Frame",
+ "Not Coded",
+ NULL,
+ };
+ switch (id) {
+ case V4L2_CID_MPEG_MFC51_VIDEO_FRAME_SKIP_MODE:
+ return mtk_vcodec_enc_video_frame_skip;
+ case V4L2_CID_MPEG_MFC51_VIDEO_FORCE_FRAME_TYPE:
+ return mtk_vcodec_enc_video_force_frame;
+ }
+ return NULL;
+}
+
+int mtk_venc_ctrls_setup(struct mtk_vcodec_ctx *ctx)
+{
+ struct v4l2_ctrl_config cfg;
+ int i;
+
+ v4l2_ctrl_handler_init(&ctx->ctrl_hdl, NUM_CTRLS);
+ if (ctx->ctrl_hdl.error) {
+ v4l2_err(&ctx->dev->v4l2_dev, "Init control handler fail %d\n",
+ ctx->ctrl_hdl.error);
+ return ctx->ctrl_hdl.error;
+ }
+ for (i = 0; i < NUM_CTRLS; i++) {
+ if (IS_MTK_VENC_PRIV(controls[i].id)) {
+ memset(&cfg, 0, sizeof(struct v4l2_ctrl_config));
+ cfg.ops = &mtk_vcodec_enc_ctrl_ops;
+ cfg.id = controls[i].id;
+ cfg.min = controls[i].minimum;
+ cfg.max = controls[i].maximum;
+ cfg.def = controls[i].default_value;
+ cfg.name = controls[i].name;
+ cfg.type = controls[i].type;
+ cfg.flags = 0;
+ if (cfg.type == V4L2_CTRL_TYPE_MENU) {
+ cfg.step = 0;
+ cfg.menu_skip_mask = cfg.menu_skip_mask;
+ cfg.qmenu = mtk_vcodec_enc_get_menu(cfg.id);
+ } else {
+ cfg.step = controls[i].step;
+ cfg.menu_skip_mask = 0;
+ }
+ v4l2_ctrl_new_custom(&ctx->ctrl_hdl, &cfg, NULL);
+ } else {
+ if ((controls[i].type == V4L2_CTRL_TYPE_MENU) ||
+ (controls[i].type == V4L2_CTRL_TYPE_INTEGER_MENU)) {
+ v4l2_ctrl_new_std_menu(
+ &ctx->ctrl_hdl,
+ &mtk_vcodec_enc_ctrl_ops,
+ controls[i].id,
+ controls[i].maximum, 0,
+ controls[i].default_value);
+ } else {
+ v4l2_ctrl_new_std(
+ &ctx->ctrl_hdl,
+ &mtk_vcodec_enc_ctrl_ops,
+ controls[i].id,
+ controls[i].minimum,
+ controls[i].maximum,
+ controls[i].step,
+ controls[i].default_value);
+ }
+ }
+
+ if (ctx->ctrl_hdl.error) {
+ v4l2_err(&ctx->dev->v4l2_dev,
+ "Adding control (%d) failed %d\n",
+ i, ctx->ctrl_hdl.error);
+ return ctx->ctrl_hdl.error;
+ }
+ }
+
+ v4l2_ctrl_handler_setup(&ctx->ctrl_hdl);
+ return 0;
+}
+
+void mtk_venc_ctrls_free(struct mtk_vcodec_ctx *ctx)
+{
+ v4l2_ctrl_handler_free(&ctx->ctrl_hdl);
+}
+
+int m2mctx_venc_queue_init(void *priv, struct vb2_queue *src_vq,
+ struct vb2_queue *dst_vq)
+{
+ struct mtk_vcodec_ctx *ctx = priv;
+ int ret;
+
+ src_vq->type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE;
+ src_vq->io_modes = VB2_DMABUF | VB2_MMAP | VB2_USERPTR;
+ src_vq->drv_priv = ctx;
+ src_vq->buf_struct_size = sizeof(struct mtk_video_enc_buf);
+ src_vq->ops = &mtk_venc_vb2_ops;
+ src_vq->mem_ops = &vb2_dma_contig_memops;
+ src_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
+ src_vq->lock = &ctx->vb2_mutex;
+
+ ret = vb2_queue_init(src_vq);
+ if (ret)
+ return ret;
+
+ dst_vq->type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
+ dst_vq->io_modes = VB2_DMABUF | VB2_MMAP | VB2_USERPTR;
+ dst_vq->drv_priv = ctx;
+ dst_vq->buf_struct_size = sizeof(struct v4l2_m2m_buffer);
+ dst_vq->ops = &mtk_venc_vb2_ops;
+ dst_vq->mem_ops = &vb2_dma_contig_memops;
+ dst_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
+ dst_vq->lock = &ctx->vb2_mutex;
+
+ return vb2_queue_init(dst_vq);
+}
+
+int mtk_venc_unlock(struct mtk_vcodec_ctx *ctx)
+{
+ struct mtk_vcodec_dev *dev = ctx->dev;
+
+ mutex_unlock(&dev->enc_mutex);
+ return 0;
+}
+
+int mtk_venc_lock(struct mtk_vcodec_ctx *ctx)
+{
+ struct mtk_vcodec_dev *dev = ctx->dev;
+
+ mutex_lock(&dev->enc_mutex);
+ dev->curr_ctx = ctx->idx;
+ return 0;
+}
+
+void mtk_vcodec_venc_release(struct mtk_vcodec_ctx *ctx)
+{
+ if (ctx->h_enc) {
+ venc_if_deinit(ctx->h_enc);
+ venc_if_release(ctx->h_enc);
+ }
+
+#if MTK_V4L2_BENCHMARK
+ mtk_v4l2_debug(0, "\n\nMTK_V4L2_BENCHMARK");
+
+ mtk_v4l2_debug(0, " total_enc_dec_cnt: %d ", ctx->total_enc_dec_cnt);
+ mtk_v4l2_debug(0, " total_enc_dec_time: %d us",
+ ctx->total_enc_dec_time);
+ mtk_v4l2_debug(0, " total_enc_dec_init_time: %d us",
+ ctx->total_enc_dec_init_time);
+ mtk_v4l2_debug(0, " total_enc_hdr_time: %d us",
+ ctx->total_enc_hdr_time);
+ mtk_v4l2_debug(0, " total_qbuf_out_time: %d us",
+ ctx->total_qbuf_out_time);
+ mtk_v4l2_debug(0, " total_qbuf_out_cnt: %d ",
+ ctx->total_qbuf_out_cnt);
+ mtk_v4l2_debug(0, " total_qbuf_cap_time: %d us",
+ ctx->total_qbuf_cap_time);
+ mtk_v4l2_debug(0, " total_qbuf_cap_cnt: %d ",
+ ctx->total_qbuf_cap_cnt);
+
+ mtk_v4l2_debug(0, " total_dqbuf_out_time: %d us",
+ ctx->total_dqbuf_out_time);
+ mtk_v4l2_debug(0, " total_dqbuf_out_cnt: %d ",
+ ctx->total_dqbuf_out_cnt);
+ mtk_v4l2_debug(0, " total_dqbuf_cap_time: %d us",
+ ctx->total_dqbuf_cap_time);
+ mtk_v4l2_debug(0, " total_dqbuf_cap_cnt: %d ",
+ ctx->total_dqbuf_cap_cnt);
+
+ mtk_v4l2_debug(0, " total_expbuf_time: %d us",
+ ctx->total_expbuf_time);
+
+#endif
+
+}
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h
new file mode 100644
index 0000000..6802c93
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h
@@ -0,0 +1,28 @@
+/*
+* Copyright (c) 2015 MediaTek Inc.
+* Author: PC Chen <pc.chen@mediatek.com>
+* Tiffany Lin <tiffany.lin@mediatek.com>
+*
+* This program is free software; you can redistribute it and/or modify
+* it under the terms of the GNU General Public License version 2 as
+* published by the Free Software Foundation.
+*
+* This program is distributed in the hope that it will be useful,
+* but WITHOUT ANY WARRANTY; without even the implied warranty of
+* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+* GNU General Public License for more details.
+*/
+
+
+#ifndef _MTK_VCODEC_ENC_H_
+#define _MTK_VCODEC_ENC_H_
+
+int mtk_venc_unlock(struct mtk_vcodec_ctx *ctx);
+int mtk_venc_lock(struct mtk_vcodec_ctx *ctx);
+int m2mctx_venc_queue_init(void *priv, struct vb2_queue *src_vq,
+ struct vb2_queue *dst_vq);
+void mtk_vcodec_venc_release(struct mtk_vcodec_ctx *ctx);
+int mtk_venc_ctrls_setup(struct mtk_vcodec_ctx *ctx);
+void mtk_venc_ctrls_free(struct mtk_vcodec_ctx *ctx);
+
+#endif /* _MTK_VCODEC_ENC_H_ */
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
new file mode 100644
index 0000000..0f89880
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
@@ -0,0 +1,535 @@
+/*
+* Copyright (c) 2015 MediaTek Inc.
+* Author: PC Chen <pc.chen@mediatek.com>
+* Tiffany Lin <tiffany.lin@mediatek.com>
+*
+* This program is free software; you can redistribute it and/or modify
+* it under the terms of the GNU General Public License version 2 as
+* published by the Free Software Foundation.
+*
+* This program is distributed in the hope that it will be useful,
+* but WITHOUT ANY WARRANTY; without even the implied warranty of
+* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+* GNU General Public License for more details.
+*/
+
+#include <linux/interrupt.h>
+#include <linux/irq.h>
+#include <linux/module.h>
+#include <linux/of_device.h>
+#include <linux/of.h>
+#include <media/v4l2-event.h>
+#include <media/v4l2-mem2mem.h>
+#include <media/videobuf2-dma-contig.h>
+#include <linux/pm_runtime.h>
+
+#include "mtk_vcodec_drv.h"
+#include "mtk_vcodec_enc.h"
+#include "mtk_vcodec_pm.h"
+#include "mtk_vcodec_intr.h"
+#include "mtk_vcodec_util.h"
+#include "mtk_vpu_core.h"
+
+static irqreturn_t mtk_vcodec_enc_irq_handler(int irq, void *priv);
+
+static void vpu_enc_capability_ipi_handler(void *data, unsigned int len, void *priv)
+{
+ struct mtk_vcodec_dev *dev = (struct mtk_vcodec_dev *)priv;
+
+ dev->enc_capability = *((u32 *)data);
+ mtk_v4l2_debug(0, "IPI=%d encoder capability %x",
+ IPI_VENC_CAPABILITY, dev->enc_capability);
+}
+
+
+/* Wake up context wait_queue */
+static void wake_up_ctx(struct mtk_vcodec_ctx *ctx, unsigned int reason)
+{
+ ctx->int_cond = 1;
+ ctx->int_type = reason;
+ wake_up_interruptible(&ctx->queue);
+}
+
+static irqreturn_t mtk_vcodec_enc_irq_handler(int irq, void *priv)
+{
+ struct mtk_vcodec_dev *dev = priv;
+ struct mtk_vcodec_ctx *ctx;
+ unsigned int irq_status;
+
+ ctx = dev->ctx[dev->curr_ctx];
+ if (ctx == NULL) {
+ mtk_v4l2_err("ctx==NULL");
+ return IRQ_HANDLED;
+ }
+ mtk_v4l2_debug(1, "idx=%d", ctx->idx);
+ irq_status = readl(dev->reg_base[VENC_SYS] +
+ (MTK_VENC_IRQ_STATUS_OFFSET));
+ if (irq_status & MTK_VENC_IRQ_STATUS_PAUSE)
+ writel((MTK_VENC_IRQ_STATUS_PAUSE),
+ dev->reg_base[VENC_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
+
+ if (irq_status & MTK_VENC_IRQ_STATUS_SWITCH)
+ writel((MTK_VENC_IRQ_STATUS_SWITCH),
+ dev->reg_base[VENC_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
+
+ if (irq_status & MTK_VENC_IRQ_STATUS_DRAM)
+ writel((MTK_VENC_IRQ_STATUS_DRAM),
+ dev->reg_base[VENC_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
+
+ if (irq_status & MTK_VENC_IRQ_STATUS_SPS)
+ writel((MTK_VENC_IRQ_STATUS_SPS),
+ dev->reg_base[VENC_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
+
+ if (irq_status & MTK_VENC_IRQ_STATUS_PPS)
+ writel((MTK_VENC_IRQ_STATUS_PPS),
+ dev->reg_base[VENC_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
+
+ if (irq_status & MTK_VENC_IRQ_STATUS_FRM)
+ writel((MTK_VENC_IRQ_STATUS_FRM),
+ dev->reg_base[VENC_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
+
+ ctx->irq_status = irq_status;
+ wake_up_ctx(ctx, MTK_INST_IRQ_RECEIVED);
+ return IRQ_HANDLED;
+}
+
+#if 1 /* VENC_LT */
+static irqreturn_t mtk_vcodec_enc_irq_handler2(int irq, void *priv)
+{
+ struct mtk_vcodec_dev *dev = priv;
+ struct mtk_vcodec_ctx *ctx;
+ unsigned int irq_status;
+
+ ctx = dev->ctx[dev->curr_ctx];
+ if (ctx == NULL) {
+ mtk_v4l2_err("ctx==NULL");
+ return IRQ_HANDLED;
+ }
+ mtk_v4l2_debug(1, "idx=%d", ctx->idx);
+ irq_status = readl(dev->reg_base[VENC_LT_SYS] +
+ (MTK_VENC_IRQ_STATUS_OFFSET));
+ if (irq_status & MTK_VENC_IRQ_STATUS_PAUSE)
+ writel((MTK_VENC_IRQ_STATUS_PAUSE),
+ dev->reg_base[VENC_LT_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
+
+ if (irq_status & MTK_VENC_IRQ_STATUS_SWITCH)
+ writel((MTK_VENC_IRQ_STATUS_SWITCH),
+ dev->reg_base[VENC_LT_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
+
+ if (irq_status & MTK_VENC_IRQ_STATUS_DRAM)
+ writel((MTK_VENC_IRQ_STATUS_DRAM),
+ dev->reg_base[VENC_LT_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
+
+ if (irq_status & MTK_VENC_IRQ_STATUS_SPS)
+ writel((MTK_VENC_IRQ_STATUS_SPS),
+ dev->reg_base[VENC_LT_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
+
+ if (irq_status & MTK_VENC_IRQ_STATUS_PPS)
+ writel((MTK_VENC_IRQ_STATUS_PPS),
+ dev->reg_base[VENC_LT_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
+
+ if (irq_status & MTK_VENC_IRQ_STATUS_FRM)
+ writel((MTK_VENC_IRQ_STATUS_FRM),
+ dev->reg_base[VENC_LT_SYS] + (MTK_VENC_IRQ_ACK_OFFSET));
+
+ ctx->irq_status = irq_status;
+ wake_up_ctx(ctx, MTK_INST_IRQ_RECEIVED);
+ return IRQ_HANDLED;
+}
+#endif
+
+static int fops_vcodec_open(struct file *file)
+{
+ struct video_device *vfd = video_devdata(file);
+ struct mtk_vcodec_dev *dev = video_drvdata(file);
+ struct mtk_vcodec_ctx *ctx = NULL;
+ int ret = 0;
+
+ mtk_v4l2_debug(0, "%s encoder", dev_name(&dev->plat_dev->dev));
+ mutex_lock(&dev->dev_mutex);
+
+ ctx = devm_kzalloc(&dev->plat_dev->dev, sizeof(*ctx), GFP_KERNEL);
+ if (!ctx) {
+ ret = -ENOMEM;
+ goto err_alloc;
+ }
+
+ if (dev->num_instances >= MTK_VCODEC_MAX_INSTANCES) {
+ mtk_v4l2_err("Too many open contexts\n");
+ ret = -EBUSY;
+ goto err_no_ctx;
+ }
+
+ ctx->idx = ffz(dev->instance_mask[0]);
+ v4l2_fh_init(&ctx->fh, video_devdata(file));
+ file->private_data = &ctx->fh;
+ v4l2_fh_add(&ctx->fh);
+ ctx->dev = dev;
+ mutex_init(&ctx->encode_param_mutex);
+ mutex_init(&ctx->vb2_mutex);
+ if (vfd == dev->vfd_enc) {
+ ctx->type = MTK_INST_ENCODER;
+ ret = mtk_venc_ctrls_setup(ctx);
+ if (ret) {
+ mtk_v4l2_err("Failed to setup controls() (%d)\n",
+ ret);
+ goto err_ctrls_setup;
+ }
+ ctx->m2m_ctx = v4l2_m2m_ctx_init(dev->m2m_dev_enc, ctx,
+ &m2mctx_venc_queue_init);
+ if (IS_ERR(ctx->m2m_ctx)) {
+ ret = PTR_ERR(ctx->m2m_ctx);
+ mtk_v4l2_err("Failed to v4l2_m2m_ctx_init() (%d)\n",
+ ret);
+ goto err_ctx_init;
+ }
+ ctx->fh.ctrl_handler = &ctx->ctrl_hdl;
+ } else {
+ mtk_v4l2_err("Invalid vfd !\n");
+ ret = -ENOENT;
+ goto err_ctx_init;
+ }
+
+ init_waitqueue_head(&ctx->queue);
+ dev->num_instances++;
+
+ if (dev->num_instances == 1) {
+ ret = vpu_load_firmware(vpu_get_plat_device(dev->plat_dev));
+ if (ret < 0) {
+ mtk_v4l2_err("vpu_load_firmware failed!\n");
+ goto err_load_fw;
+ }
+ mtk_v4l2_debug(0, "encoder capability %x", dev->enc_capability);
+ }
+
+ mtk_v4l2_debug(2, "Create instance [%d]@%p m2m_ctx=%p type=%d\n",
+ ctx->idx, ctx, ctx->m2m_ctx, ctx->type);
+ ctx->state = MTK_STATE_FREE;
+ mtk_vcodec_clean_ctx_int_flags(ctx);
+ set_bit(ctx->idx, &dev->instance_mask[0]);
+ dev->ctx[ctx->idx] = ctx;
+ vpu_enable_clock(vpu_get_plat_device(dev->plat_dev));
+
+ mutex_unlock(&dev->dev_mutex);
+ return ret;
+
+ /* Deinit when failure occurred */
+err_load_fw:
+ v4l2_m2m_ctx_release(ctx->m2m_ctx);
+ v4l2_fh_del(&ctx->fh);
+ v4l2_fh_exit(&ctx->fh);
+ dev->num_instances--;
+err_ctx_init:
+err_ctrls_setup:
+ mtk_venc_ctrls_free(ctx);
+err_no_ctx:
+ devm_kfree(&dev->plat_dev->dev, ctx);
+err_alloc:
+ mutex_unlock(&dev->dev_mutex);
+ return ret;
+}
+
+static int fops_vcodec_release(struct file *file)
+{
+ struct mtk_vcodec_dev *dev = video_drvdata(file);
+ struct mtk_vcodec_ctx *ctx = fh_to_ctx(file->private_data);
+
+ mtk_v4l2_debug(2, "[%d]\n", ctx->idx);
+ mutex_lock(&dev->dev_mutex);
+ v4l2_m2m_ctx_release(ctx->m2m_ctx);
+ mtk_vcodec_venc_release(ctx);
+ ctx->state = MTK_STATE_DEINIT;
+ mtk_venc_ctrls_free(ctx);
+ v4l2_fh_del(&ctx->fh);
+ v4l2_fh_exit(&ctx->fh);
+ dev->ctx[ctx->idx] = NULL;
+ dev->num_instances--;
+ if (dev->num_instances == 0) {
+ vpu_disable_clock(vpu_get_plat_device(dev->plat_dev));
+ }
+ clear_bit(ctx->idx, &dev->instance_mask[0]);
+ devm_kfree(&dev->plat_dev->dev, ctx);
+ mutex_unlock(&dev->dev_mutex);
+ return 0;
+}
+
+static unsigned int fops_vcodec_poll(struct file *file,
+ struct poll_table_struct *wait)
+{
+ struct mtk_vcodec_ctx *ctx = fh_to_ctx(file->private_data);
+ struct mtk_vcodec_dev *dev = ctx->dev;
+ int ret;
+
+ mutex_lock(&dev->dev_mutex);
+ ret = v4l2_m2m_poll(file, ctx->m2m_ctx, wait);
+ mutex_unlock(&dev->dev_mutex);
+
+ return ret;
+}
+
+static int fops_vcodec_mmap(struct file *file, struct vm_area_struct *vma)
+{
+ struct mtk_vcodec_ctx *ctx = fh_to_ctx(file->private_data);
+
+ return v4l2_m2m_mmap(file, ctx->m2m_ctx, vma);
+}
+
+static const struct v4l2_file_operations mtk_vcodec_fops = {
+ .owner = THIS_MODULE,
+ .open = fops_vcodec_open,
+ .release = fops_vcodec_release,
+ .poll = fops_vcodec_poll,
+ .unlocked_ioctl = video_ioctl2,
+ .mmap = fops_vcodec_mmap,
+};
+
+static int mtk_vcodec_probe(struct platform_device *pdev)
+{
+ struct mtk_vcodec_dev *dev;
+ struct video_device *vfd_enc;
+ struct resource *res;
+ int i, j, ret;
+ struct platform_device *vpu_pdev;
+
+ dev = devm_kzalloc(&pdev->dev, sizeof(*dev), GFP_KERNEL);
+ if (!dev)
+ return -ENOMEM;
+
+ dev->plat_dev = pdev;
+
+ vpu_pdev = vpu_get_plat_device(dev->plat_dev);
+ if (vpu_pdev == NULL) {
+ mtk_v4l2_err("[VPU] vpu device in not ready\n");
+ return -EPROBE_DEFER;
+ }
+
+ ret = vpu_ipi_register(vpu_pdev, IPI_VENC_CAPABILITY,
+ vpu_enc_capability_ipi_handler,
+ "vpu_init", dev);
+ if (ret != 0) {
+ mtk_v4l2_err("[VPU] failed to register IPI_VENC_CAPBILITY\n");
+ return ret;
+ }
+
+ ret = mtk_vcodec_init_enc_pm(dev);
+ if (ret < 0) {
+ dev_err(&pdev->dev, "Failed to get mt vcodec clock source!\n");
+ return ret;
+ }
+
+ for (i = VENC_SYS, j = 0; i < NUM_MAX_VCODEC_REG_BASE; i++, j++) {
+ res = platform_get_resource(pdev, IORESOURCE_MEM, j);
+ if (res == NULL) {
+ dev_err(&pdev->dev, "get memory resource failed.\n");
+ ret = -ENXIO;
+ goto err_res;
+ }
+ dev->reg_base[i] = devm_ioremap_resource(&pdev->dev, res);
+ if (IS_ERR(dev->reg_base[i])) {
+ dev_err(&pdev->dev,
+ "devm_ioremap_resource %d failed.\n", i);
+ ret = PTR_ERR(dev->reg_base);
+ goto err_res;
+ }
+ mtk_v4l2_debug(2, "reg[%d] base=0x%p\n", i, dev->reg_base[i]);
+ }
+
+ res = platform_get_resource(pdev, IORESOURCE_IRQ, 0);
+ if (res == NULL) {
+ dev_err(&pdev->dev, "failed to get irq resource\n");
+ ret = -ENOENT;
+ goto err_res;
+ }
+
+ dev->enc_irq = platform_get_irq(pdev, 0);
+ ret = devm_request_irq(&pdev->dev, dev->enc_irq,
+ mtk_vcodec_enc_irq_handler,
+ 0, pdev->name, dev);
+ if (ret) {
+ dev_err(&pdev->dev, "Failed to install dev->enc_irq %d (%d)\n",
+ dev->enc_irq,
+ ret);
+ ret = -EINVAL;
+ goto err_res;
+ }
+
+ dev->enc_lt_irq = platform_get_irq(pdev, 1);
+ ret = devm_request_irq(&pdev->dev,
+ dev->enc_lt_irq, mtk_vcodec_enc_irq_handler2,
+ 0, pdev->name, dev);
+ if (ret) {
+ dev_err(&pdev->dev,
+ "Failed to install dev->enc_lt_irq %d (%d)\n",
+ dev->enc_lt_irq, ret);
+ ret = -EINVAL;
+ goto err_res;
+ }
+
+ disable_irq(dev->enc_irq);
+ disable_irq(dev->enc_lt_irq); /* VENC_LT */
+ mutex_init(&dev->enc_mutex);
+ mutex_init(&dev->dev_mutex);
+
+ snprintf(dev->v4l2_dev.name, sizeof(dev->v4l2_dev.name), "%s",
+ "[MTK_V4L2_VENC]");
+
+ ret = v4l2_device_register(&pdev->dev, &dev->v4l2_dev);
+ if (ret) {
+ mtk_v4l2_err("v4l2_device_register err=%d\n", ret);
+ return ret;
+ }
+
+ init_waitqueue_head(&dev->queue);
+
+ /* allocate video device for encoder and register it */
+ vfd_enc = video_device_alloc();
+ if (!vfd_enc) {
+ mtk_v4l2_err("Failed to allocate video device\n");
+ ret = -ENOMEM;
+ goto err_enc_alloc;
+ }
+ vfd_enc->fops = &mtk_vcodec_fops;
+ vfd_enc->ioctl_ops = &mtk_venc_ioctl_ops;
+ vfd_enc->release = video_device_release;
+ vfd_enc->lock = &dev->dev_mutex;
+ vfd_enc->v4l2_dev = &dev->v4l2_dev;
+ vfd_enc->vfl_dir = VFL_DIR_M2M;
+ snprintf(vfd_enc->name, sizeof(vfd_enc->name), "%s",
+ MTK_VCODEC_ENC_NAME);
+ video_set_drvdata(vfd_enc, dev);
+ dev->vfd_enc = vfd_enc;
+ platform_set_drvdata(pdev, dev);
+ ret = video_register_device(vfd_enc, VFL_TYPE_GRABBER, 1);
+ if (ret) {
+ mtk_v4l2_err("Failed to register video device\n");
+ goto err_enc_reg;
+ }
+ mtk_v4l2_debug(0, "encoder registered as /dev/video%d\n",
+ vfd_enc->num);
+
+ dev->alloc_ctx = vb2_dma_contig_init_ctx(&pdev->dev);
+ if (IS_ERR(dev->alloc_ctx)) {
+ mtk_v4l2_err("Failed to alloc vb2 dma context 0\n");
+ ret = PTR_ERR(dev->alloc_ctx);
+ goto err_vb2_ctx_init;
+ }
+
+ dev->m2m_dev_enc = v4l2_m2m_init(&mtk_venc_m2m_ops);
+ if (IS_ERR(dev->m2m_dev_enc)) {
+ mtk_v4l2_err("Failed to init mem2mem enc device\n");
+ ret = PTR_ERR(dev->m2m_dev_enc);
+ goto err_enc_mem_init;
+ }
+
+ dev->encode_workqueue =
+ create_singlethread_workqueue(MTK_VCODEC_ENC_NAME);
+ if (!dev->encode_workqueue) {
+ mtk_v4l2_err("Failed to create encode workqueue\n");
+ ret = -EINVAL;
+ goto err_event_workq;
+ }
+
+ return 0;
+
+err_event_workq:
+ v4l2_m2m_release(dev->m2m_dev_enc);
+err_enc_mem_init:
+ vb2_dma_contig_cleanup_ctx(dev->alloc_ctx);
+err_vb2_ctx_init:
+ video_unregister_device(vfd_enc);
+err_enc_reg:
+ video_device_release(vfd_enc);
+err_enc_alloc:
+ v4l2_device_unregister(&dev->v4l2_dev);
+err_res:
+ mtk_vcodec_release_enc_pm(dev);
+ return ret;
+}
+
+static const struct of_device_id mtk_vcodec_match[] = {
+ {.compatible = "mediatek,mt8173-vcodec-enc",},
+ {},
+};
+MODULE_DEVICE_TABLE(of, mtk_vcodec_match);
+
+static int mtk_vcodec_remove(struct platform_device *pdev)
+{
+ struct mtk_vcodec_dev *dev = platform_get_drvdata(pdev);
+
+ mtk_v4l2_debug_enter();
+ flush_workqueue(dev->encode_workqueue);
+ destroy_workqueue(dev->encode_workqueue);
+ if (dev->m2m_dev_enc)
+ v4l2_m2m_release(dev->m2m_dev_enc);
+ if (dev->alloc_ctx)
+ vb2_dma_contig_cleanup_ctx(dev->alloc_ctx);
+
+ if (dev->vfd_enc) {
+ video_unregister_device(dev->vfd_enc);
+ video_device_release(dev->vfd_enc);
+ }
+ v4l2_device_unregister(&dev->v4l2_dev);
+ mtk_vcodec_release_enc_pm(dev);
+ return 0;
+}
+
+static int mtk_vcodec_venc_suspend(struct device *dev)
+{
+ struct platform_device *pdev = to_platform_device(dev);
+ struct mtk_vcodec_dev *m_dev = platform_get_drvdata(pdev);
+
+ mtk_v4l2_debug(0, "m_dev->num_instances=%d", m_dev->num_instances);
+ if (m_dev->num_instances == 0)
+ return 0;
+ mtk_vcodec_clean_dev_int_flags(m_dev);
+ if (test_and_set_bit(0, &m_dev->enter_suspend) != 0) {
+ mtk_v4l2_err("Error: going to suspend for a second time\n");
+ return -EIO;
+ }
+ mtk_vcodec_wait_for_done_dev(m_dev,
+ MTK_INST_WORK_THREAD_SUSPEND_DONE,
+ 3000, true);
+
+ mutex_lock(&m_dev->enc_mutex);
+ mtk_vcodec_enc_clock_off();
+ mtk_v4l2_debug(0, "m_dev->num_instances=%d", m_dev->num_instances);
+ return 0;
+}
+
+static int mtk_vcodec_venc_resume(struct device *dev)
+{
+ struct platform_device *pdev = to_platform_device(dev);
+ struct mtk_vcodec_dev *m_dev = platform_get_drvdata(pdev);
+
+ mtk_v4l2_debug(0, "m_dev->num_instances=%d", m_dev->num_instances);
+ if (m_dev->num_instances == 0)
+ return 0;
+
+ test_and_clear_bit(0, &m_dev->enter_suspend);
+ mtk_vcodec_enc_clock_on();
+ mutex_unlock(&m_dev->enc_mutex);
+ return 0;
+}
+
+
+static const struct dev_pm_ops mtk_vcodec_venc_pm_ops = {
+ SET_SYSTEM_SLEEP_PM_OPS(mtk_vcodec_venc_suspend,
+ mtk_vcodec_venc_resume)
+};
+
+static struct platform_driver mtk_vcodec_driver = {
+ .probe = mtk_vcodec_probe,
+ .remove = mtk_vcodec_remove,
+ .driver = {
+ .name = MTK_VCODEC_ENC_NAME,
+ .owner = THIS_MODULE,
+ .of_match_table = mtk_vcodec_match,
+ .pm = &mtk_vcodec_venc_pm_ops,
+ },
+};
+
+module_platform_driver(mtk_vcodec_driver);
+
+
+MODULE_LICENSE("GPL v2");
+MODULE_DESCRIPTION("Mediatek video codec V4L2 driver");
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
new file mode 100644
index 0000000..ba97344
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
@@ -0,0 +1,122 @@
+/*
+* Copyright (c) 2015 MediaTek Inc.
+* Author: Tiffany Lin <tiffany.lin@mediatek.com>
+*
+* This program is free software; you can redistribute it and/or modify
+* it under the terms of the GNU General Public License version 2 as
+* published by the Free Software Foundation.
+*
+* This program is distributed in the hope that it will be useful,
+* but WITHOUT ANY WARRANTY; without even the implied warranty of
+* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+* GNU General Public License for more details.
+*/
+
+#include <linux/clk.h>
+#include <linux/of_address.h>
+#include <linux/of_platform.h>
+#include <linux/pm_runtime.h>
+#include <soc/mediatek/smi.h>
+
+#include "mtk_vcodec_pm.h"
+#include "mtk_vcodec_util.h"
+#include "mtk_vpu_core.h"
+
+static struct mtk_vcodec_pm *pm;
+
+int mtk_vcodec_init_enc_pm(struct mtk_vcodec_dev *mtkdev)
+{
+ struct device_node *node;
+ struct platform_device *pdev;
+ struct device *dev;
+ int ret = 0;
+
+ pdev = mtkdev->plat_dev;
+ pm = &mtkdev->pm;
+ memset(pm, 0, sizeof(struct mtk_vcodec_pm));
+ pm->mtkdev = mtkdev;
+ dev = &pdev->dev;
+
+ node = of_parse_phandle(dev->of_node, "larb", 0);
+ if (!node)
+ return -1;
+ pdev = of_find_device_by_node(node);
+ if (WARN_ON(!pdev)) {
+ of_node_put(node);
+ return -1;
+ }
+ pm->larbvenc = &pdev->dev;
+
+ node = of_parse_phandle(dev->of_node, "larb", 1);
+ if (!node)
+ return -1;
+
+ pdev = of_find_device_by_node(node);
+ if (WARN_ON(!pdev)) {
+ of_node_put(node);
+ return -EINVAL;
+ }
+ pm->larbvenclt = &pdev->dev;
+
+ pdev = mtkdev->plat_dev;
+ pm->dev = &pdev->dev;
+
+ pm->vencpll = devm_clk_get(&pdev->dev, "vencpll");
+ if (pm->vencpll == NULL) {
+ mtk_v4l2_err("devm_clk_get vencpll fail");
+ ret = -1;
+ }
+
+ pm->venc_lt_sel = devm_clk_get(&pdev->dev, "venc_lt_sel");
+ if (pm->venc_lt_sel == NULL) {
+ mtk_v4l2_err("devm_clk_get venc_lt_sel fail");
+ ret = -1;
+ }
+
+ pm->vcodecpll_370p5_ck = devm_clk_get(&pdev->dev, "vcodecpll_370p5_ck");
+ if (pm->vcodecpll_370p5_ck == NULL) {
+ mtk_v4l2_err("devm_clk_get vcodecpll_370p5_ck fail");
+ ret = -1;
+ }
+
+ return ret;
+}
+
+void mtk_vcodec_release_enc_pm(struct mtk_vcodec_dev *dev)
+{
+}
+
+
+void mtk_vcodec_enc_clock_on(void)
+{
+ int ret;
+
+ clk_set_rate(pm->vencpll, 800 * 1000000);
+
+ ret = clk_prepare_enable(pm->vcodecpll_370p5_ck);
+ if (ret)
+ mtk_v4l2_err("vcodecpll_370p5_ck fail %d", ret);
+
+ ret = clk_prepare_enable(pm->venc_lt_sel);
+ if (ret)
+ mtk_v4l2_err("venc_lt_sel fail %d", ret);
+
+ ret = clk_set_parent(pm->venc_lt_sel, pm->vcodecpll_370p5_ck);
+ if (ret)
+ mtk_v4l2_err("clk_set_parent fail %d", ret);
+
+ ret = mtk_smi_larb_get(pm->larbvenc);
+ if (ret)
+ mtk_v4l2_err("mtk_smi_larb_get larb3 fail %d\n", ret);
+
+ ret = mtk_smi_larb_get(pm->larbvenclt);
+ if (ret)
+ mtk_v4l2_err("mtk_smi_larb_get larb4 fail %d\n", ret);
+
+}
+
+void mtk_vcodec_enc_clock_off(void)
+{
+ mtk_smi_larb_put(pm->larbvenc);
+ mtk_smi_larb_put(pm->larbvenclt);
+}
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.c
new file mode 100644
index 0000000..7a2cacf
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.c
@@ -0,0 +1,110 @@
+/*
+* Copyright (c) 2015 MediaTek Inc.
+* Author: Tiffany Lin <tiffany.lin@mediatek.com>
+*
+* This program is free software; you can redistribute it and/or modify
+* it under the terms of the GNU General Public License version 2 as
+* published by the Free Software Foundation.
+*
+* This program is distributed in the hope that it will be useful,
+* but WITHOUT ANY WARRANTY; without even the implied warranty of
+* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+* GNU General Public License for more details.
+*/
+
+#include <linux/errno.h>
+#include <linux/wait.h>
+
+#include "mtk_vcodec_intr.h"
+#include "mtk_vcodec_drv.h"
+#include "mtk_vcodec_util.h"
+
+void mtk_vcodec_clean_ctx_int_flags(void *data)
+{
+ struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)data;
+
+ ctx->int_cond = 0;
+ ctx->int_type = 0;
+}
+
+void mtk_vcodec_clean_dev_int_flags(void *data)
+{
+ struct mtk_vcodec_dev *dev = (struct mtk_vcodec_dev *)data;
+
+ dev->int_cond = 0;
+ dev->int_type = 0;
+}
+
+int mtk_vcodec_wait_for_done_ctx(void *data, int command,
+ unsigned int timeout, int interrupt)
+{
+ wait_queue_head_t *waitqueue;
+ long timeout_jiff, ret;
+ int status = 0;
+ struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)data;
+
+ waitqueue = (wait_queue_head_t *)&ctx->queue;
+ timeout_jiff = msecs_to_jiffies(timeout);
+ if (interrupt) {
+ ret = wait_event_interruptible_timeout(*waitqueue,
+ (ctx->int_cond &&
+ (ctx->int_type == command)),
+ timeout_jiff);
+ } else {
+ ret = wait_event_timeout(*waitqueue,
+ (ctx->int_cond &&
+ (ctx->int_type == command)),
+ timeout_jiff);
+ }
+ if (0 == ret) {
+ status = -1; /* timeout */
+ mtk_v4l2_err("[%d] cmd=%d, wait_event_interruptible_timeout time=%lu out %d %d!",
+ ctx->idx, command, timeout_jiff, ctx->int_cond,
+ ctx->int_type);
+ } else if (-ERESTARTSYS == ret) {
+ mtk_v4l2_err("[%d] cmd=%d, wait_event_interruptible_timeout interrupted by a signal %d %d",
+ ctx->idx, command, ctx->int_cond,
+ ctx->int_type);
+ status = -1;
+ }
+
+ ctx->int_cond = 0;
+ ctx->int_type = 0;
+
+ return status;
+}
+
+int mtk_vcodec_wait_for_done_dev(void *data, int command,
+ unsigned int timeout, int interrupt)
+{
+ wait_queue_head_t *waitqueue;
+ long timeout_jiff, ret;
+ int status = 0;
+ struct mtk_vcodec_dev *dev = (struct mtk_vcodec_dev *)data;
+
+ waitqueue = (wait_queue_head_t *)&dev->queue;
+ timeout_jiff = msecs_to_jiffies(timeout);
+ if (interrupt) {
+ ret = wait_event_interruptible_timeout(*waitqueue,
+ (dev->int_cond &&
+ (dev->int_type == command)),
+ timeout_jiff);
+ } else {
+ ret = wait_event_timeout(*waitqueue,
+ (dev->int_cond &&
+ (dev->int_type == command)),
+ timeout_jiff);
+ }
+ if (0 == ret) {
+ status = -1; /* timeout */
+ mtk_v4l2_err("wait_event_interruptible_timeout time=%lu out %d %d!",
+ timeout_jiff, dev->int_cond, dev->int_type);
+ } else if (-ERESTARTSYS == ret) {
+ mtk_v4l2_err("wait_event_interruptible_timeout interrupted by a signal %d %d",
+ dev->int_cond, dev->int_type);
+ status = -1;
+ }
+ dev->int_cond = 0;
+ dev->int_type = 0;
+ return status;
+}
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.h
new file mode 100644
index 0000000..e2d2eaa
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.h
@@ -0,0 +1,30 @@
+/*
+* Copyright (c) 2015 MediaTek Inc.
+* Author: Tiffany Lin <tiffany.lin@mediatek.com>
+*
+* This program is free software; you can redistribute it and/or modify
+* it under the terms of the GNU General Public License version 2 as
+* published by the Free Software Foundation.
+*
+* This program is distributed in the hope that it will be useful,
+* but WITHOUT ANY WARRANTY; without even the implied warranty of
+* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+* GNU General Public License for more details.
+*/
+
+#ifndef _MTK_VCODEC_INTR_H_
+#define _MTK_VCODEC_INTR_H_
+
+#define MTK_INST_IRQ_RECEIVED 0x1
+#define MTK_INST_WORK_THREAD_ABORT_DONE 0x2
+#define MTK_INST_WORK_THREAD_SUSPEND_DONE 0x3
+
+/* timeout is ms */
+int mtk_vcodec_wait_for_done_ctx(void *data, int command, unsigned int timeout,
+ int interrupt);
+int mtk_vcodec_wait_for_done_dev(void *data, int command, unsigned int timeout,
+ int interrupt);
+void mtk_vcodec_clean_ctx_int_flags(void *data);
+void mtk_vcodec_clean_dev_int_flags(void *data);
+
+#endif /* _MTK_VCODEC_INTR_H_ */
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_pm.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_pm.h
new file mode 100644
index 0000000..a28bb7c
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_pm.h
@@ -0,0 +1,26 @@
+/*
+* Copyright (c) 2015 MediaTek Inc.
+* Author: Tiffany Lin <tiffany.lin@mediatek.com>
+*
+* This program is free software; you can redistribute it and/or modify
+* it under the terms of the GNU General Public License version 2 as
+* published by the Free Software Foundation.
+*
+* This program is distributed in the hope that it will be useful,
+* but WITHOUT ANY WARRANTY; without even the implied warranty of
+* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+* GNU General Public License for more details.
+*/
+
+#ifndef _MTK_VCODEC_PM_H_
+#define _MTK_VCODEC_PM_H_
+
+#include "mtk_vcodec_drv.h"
+
+int mtk_vcodec_init_enc_pm(struct mtk_vcodec_dev *dev);
+void mtk_vcodec_release_enc_pm(struct mtk_vcodec_dev *dev);
+
+void mtk_vcodec_enc_clock_on(void);
+void mtk_vcodec_enc_clock_off(void);
+
+#endif /* _MTK_VCODEC_PM_H_ */
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c
new file mode 100644
index 0000000..5b6943c
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c
@@ -0,0 +1,106 @@
+/*
+* Copyright (c) 2015 MediaTek Inc.
+* Author: PC Chen <pc.chen@mediatek.com>
+* Tiffany Lin <tiffany.lin@mediatek.com>
+*
+* This program is free software; you can redistribute it and/or modify
+* it under the terms of the GNU General Public License version 2 as
+* published by the Free Software Foundation.
+*
+* This program is distributed in the hope that it will be useful,
+* but WITHOUT ANY WARRANTY; without even the implied warranty of
+* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+* GNU General Public License for more details.
+*/
+
+#include <linux/module.h>
+
+#include "mtk_vcodec_drv.h"
+#include "mtk_vcodec_util.h"
+#include "mtk_vpu_core.h"
+
+bool mtk_vcodec_dbg = false;
+int mtk_v4l2_dbg_level = 0;
+
+module_param(mtk_v4l2_dbg_level, int, S_IRUGO | S_IWUSR);
+module_param(mtk_vcodec_dbg, bool, S_IRUGO | S_IWUSR);
+
+void __iomem *mtk_vcodec_get_reg_addr(void *data, unsigned int reg_idx)
+{
+ struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)data;
+
+ if (!data || reg_idx >= NUM_MAX_VCODEC_REG_BASE) {
+ mtk_v4l2_err("Invalid arguments");
+ return NULL;
+ }
+ return ctx->dev->reg_base[reg_idx];
+}
+
+int mtk_vcodec_mem_alloc(void *data, struct mtk_vcodec_mem *mem)
+{
+ unsigned long size = mem->size;
+ struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)data;
+ struct device *dev = &ctx->dev->plat_dev->dev;
+
+ mem->va = dma_alloc_coherent(dev, size, &mem->dma_addr, GFP_KERNEL);
+
+ if (!mem->va) {
+ mtk_v4l2_err("%s dma_alloc size=%ld failed!", dev_name(dev),
+ size);
+ return -ENOMEM;
+ }
+
+ memset(mem->va, 0, size);
+
+ mtk_v4l2_debug(4, "[%d] - va = %p", ctx->idx, mem->va);
+ mtk_v4l2_debug(4, "[%d] - dma = 0x%lx", ctx->idx,
+ (unsigned long)mem->dma_addr);
+ mtk_v4l2_debug(4, "[%d] size = 0x%lx", ctx->idx, size);
+
+ return 0;
+}
+
+void mtk_vcodec_mem_free(void *data, struct mtk_vcodec_mem *mem)
+{
+ unsigned long size = mem->size;
+ struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)data;
+ struct device *dev = &ctx->dev->plat_dev->dev;
+
+ dma_free_coherent(dev, size, mem->va, mem->dma_addr);
+ mem->va = NULL;
+
+ mtk_v4l2_debug(4, "[%d] - va = %p", ctx->idx, mem->va);
+ mtk_v4l2_debug(4, "[%d] - dma = 0x%lx", ctx->idx,
+ (unsigned long)mem->dma_addr);
+ mtk_v4l2_debug(4, "[%d] size = 0x%lx", ctx->idx, size);
+}
+
+int mtk_vcodec_get_ctx_id(void *data)
+{
+ struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)data;
+
+ if (!ctx)
+ return -1;
+
+ return ctx->idx;
+}
+
+struct platform_device *mtk_vcodec_get_plat_dev(void *data)
+{
+ struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)data;
+
+ if (!ctx)
+ return NULL;
+
+ return vpu_get_plat_device(ctx->dev->plat_dev);
+}
+
+void mtk_vcodec_fmt2str(u32 fmt, char *str)
+{
+ char a = fmt & 0xFF;
+ char b = (fmt >> 8) & 0xFF;
+ char c = (fmt >> 16) & 0xFF;
+ char d = (fmt >> 24) & 0xFF;
+
+ sprintf(str, "%c%c%c%c", a, b, c, d);
+}
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
new file mode 100644
index 0000000..a8e683a
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
@@ -0,0 +1,66 @@
+/*
+* Copyright (c) 2015 MediaTek Inc.
+* Author: PC Chen <pc.chen@mediatek.com>
+* Tiffany Lin <tiffany.lin@mediatek.com>
+*
+* This program is free software; you can redistribute it and/or modify
+* it under the terms of the GNU General Public License version 2 as
+* published by the Free Software Foundation.
+*
+* This program is distributed in the hope that it will be useful,
+* but WITHOUT ANY WARRANTY; without even the implied warranty of
+* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+* GNU General Public License for more details.
+*/
+
+#ifndef _MTK_VCODEC_UTIL_H_
+#define _MTK_VCODEC_UTIL_H_
+
+#include <linux/types.h>
+#include <linux/dma-direction.h>
+
+struct mtk_vcodec_mem {
+ size_t size;
+ void *va;
+ dma_addr_t dma_addr;
+};
+
+extern int mtk_v4l2_dbg_level;
+extern bool mtk_vcodec_dbg;
+
+#define mtk_v4l2_debug(level, fmt, args...) \
+ do { \
+ if (mtk_v4l2_dbg_level >= level) \
+ pr_info("[MTK_V4L2] level=%d %s(),%d: " fmt "\n",\
+ level, __func__, __LINE__, ##args); \
+ } while (0)
+
+#define mtk_v4l2_err(fmt, args...) \
+ pr_err("[MTK_V4L2][ERROR] %s:%d: " fmt "\n", __func__, __LINE__, \
+ ##args)
+
+#define mtk_v4l2_debug_enter() mtk_v4l2_debug(5, "+\n")
+#define mtk_v4l2_debug_leave() mtk_v4l2_debug(5, "-\n")
+
+#define mtk_vcodec_debug(h, fmt, args...) \
+ do { \
+ if (mtk_vcodec_dbg) \
+ pr_info("[MTK_VCODEC][%d]: %s() " fmt "\n", \
+ ((struct mtk_vcodec_ctx *)h->ctx)->idx, \
+ __func__, ##args); \
+ } while (0)
+
+#define mtk_vcodec_err(h, fmt, args...) \
+ pr_err("[MTK_VCODEC][ERROR][%d]: %s() " fmt "\n", \
+ ((struct mtk_vcodec_ctx *)h->ctx)->idx, __func__, ##args)
+
+#define mtk_vcodec_debug_enter(h) mtk_vcodec_debug(h, "+\n")
+#define mtk_vcodec_debug_leave(h) mtk_vcodec_debug(h, "-\n")
+
+void __iomem *mtk_vcodec_get_reg_addr(void *data, unsigned int reg_idx);
+int mtk_vcodec_mem_alloc(void *data, struct mtk_vcodec_mem *mem);
+void mtk_vcodec_mem_free(void *data, struct mtk_vcodec_mem *mem);
+int mtk_vcodec_get_ctx_id(void *data);
+struct platform_device *mtk_vcodec_get_plat_dev(void *data);
+void mtk_vcodec_fmt2str(u32 fmt, char *str);
+#endif /* _MTK_VCODEC_UTIL_H_ */
--
1.7.9.5
^ permalink raw reply related [flat|nested] 84+ messages in thread
* [RESEND RFC/PATCH 7/8] media: platform: mtk-vcodec: Add Mediatek VP8 Video Encoder Driver
2015-11-17 12:54 ` Tiffany Lin
(?)
@ 2015-11-17 12:54 ` Tiffany Lin
-1 siblings, 0 replies; 84+ messages in thread
From: Tiffany Lin @ 2015-11-17 12:54 UTC (permalink / raw)
To: Rob Herring, Pawel Moll, Mark Rutland, Ian Campbell, Kumar Gala,
Catalin Marinas, Will Deacon, Mauro Carvalho Chehab,
Matthias Brugger, Daniel Kurtz, Sascha Hauer, Hongzhou Yang,
Hans Verkuil, Laurent Pinchart, Sakari Ailus, Geert Uytterhoeven,
Mikhail Ulyanov, Fabien Dessenne, Arnd Bergmann,
Darren Etheridge, Peter Griffin, Benoit Parrot
Cc: Tiffany Lin, Andrew-CT Chen, Eddie Huang, Yingjoe Chen,
James Liao, Daniel Hsiao, devicetree, linux-kernel,
linux-arm-kernel, linux-media, linux-mediatek
From: Daniel Hsiao <daniel.hsiao@mediatek.com>
Signed-off-by: Daniel Hsiao <daniel.hsiao@mediatek.com>
---
drivers/media/platform/mtk-vcodec/Makefile | 2 +-
drivers/media/platform/mtk-vcodec/common/Makefile | 4 +-
.../media/platform/mtk-vcodec/common/venc_drv_if.c | 6 +-
drivers/media/platform/mtk-vcodec/vp8_enc/Makefile | 9 +
.../platform/mtk-vcodec/vp8_enc/venc_vp8_if.c | 371 ++++++++++++++++++++
.../platform/mtk-vcodec/vp8_enc/venc_vp8_if.h | 48 +++
.../platform/mtk-vcodec/vp8_enc/venc_vp8_vpu.c | 245 +++++++++++++
7 files changed, 682 insertions(+), 3 deletions(-)
create mode 100644 drivers/media/platform/mtk-vcodec/vp8_enc/Makefile
create mode 100644 drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_if.c
create mode 100644 drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_if.h
create mode 100644 drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_vpu.c
diff --git a/drivers/media/platform/mtk-vcodec/Makefile b/drivers/media/platform/mtk-vcodec/Makefile
index c7f7174..b881a8b 100644
--- a/drivers/media/platform/mtk-vcodec/Makefile
+++ b/drivers/media/platform/mtk-vcodec/Makefile
@@ -4,7 +4,7 @@ obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += mtk_vcodec_intr.o \
mtk_vcodec_enc.o \
mtk_vcodec_enc_pm.o
-obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += common/
+obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += common/ vp8_enc/
ccflags-y += -I$(srctree)/drivers/media/platform/mtk-vcodec/include \
-I$(srctree)/drivers/media/platform/mtk-vcodec \
diff --git a/drivers/media/platform/mtk-vcodec/common/Makefile b/drivers/media/platform/mtk-vcodec/common/Makefile
index 477ab80..71ae856 100644
--- a/drivers/media/platform/mtk-vcodec/common/Makefile
+++ b/drivers/media/platform/mtk-vcodec/common/Makefile
@@ -5,4 +5,6 @@ ccflags-y += \
-I$(srctree)/include/ \
-I$(srctree)/drivers/media/platform/mtk-vcodec \
-I$(srctree)/drivers/media/platform/mtk-vcodec/include \
- -I$(srctree)/drivers/media/platform/mtk-vpu
\ No newline at end of file
+ -I$(srctree)/drivers/media/platform/mtk-vcodec/vp8_enc \
+ -I$(srctree)/drivers/media/platform/mtk-vpu \
+ -I$(srctree)/drivers/media/platform/mtk-vpu/vp8_enc
diff --git a/drivers/media/platform/mtk-vcodec/common/venc_drv_if.c b/drivers/media/platform/mtk-vcodec/common/venc_drv_if.c
index 9b3f025..e9be186 100644
--- a/drivers/media/platform/mtk-vcodec/common/venc_drv_if.c
+++ b/drivers/media/platform/mtk-vcodec/common/venc_drv_if.c
@@ -26,7 +26,7 @@
#include "venc_drv_if.h"
#include "venc_drv_base.h"
-
+#include "venc_vp8_if.h"
int venc_if_create(void *ctx, unsigned int fourcc, unsigned long *handle)
{
@@ -44,6 +44,10 @@ int venc_if_create(void *ctx, unsigned int fourcc, unsigned long *handle)
mtk_vcodec_debug(h, "fmt = %s handle = %p", str, h);
switch (fourcc) {
+ case V4L2_PIX_FMT_VP8:
+ h->enc_if = get_vp8_enc_comm_if();
+ break;
+ case V4L2_PIX_FMT_H264:
default:
mtk_vcodec_err(h, "invalid format %s", str);
goto err_out;
diff --git a/drivers/media/platform/mtk-vcodec/vp8_enc/Makefile b/drivers/media/platform/mtk-vcodec/vp8_enc/Makefile
new file mode 100644
index 0000000..ac78c33
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/vp8_enc/Makefile
@@ -0,0 +1,9 @@
+obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += venc_vp8_if.o venc_vp8_vpu.o
+
+ccflags-y += \
+ -I$(srctree)/include/ \
+ -I$(srctree)/drivers/media/platform/mtk-vcodec/ \
+ -I$(srctree)/drivers/media/platform/mtk-vcodec/include \
+ -I$(srctree)/drivers/media/platform/mtk-vcodec/vp8_enc \
+ -I$(srctree)/drivers/media/platform/mtk-vpu/ \
+ -I$(srctree)/drivers/media/platform/mtk-vpu/vp8_enc
diff --git a/drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_if.c b/drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_if.c
new file mode 100644
index 0000000..cc6aaf4
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_if.c
@@ -0,0 +1,371 @@
+/*
+ * Copyright (c) 2015 MediaTek Inc.
+ * Author: Daniel Hsiao <daniel.hsiao@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/interrupt.h>
+#include <linux/kernel.h>
+#include <linux/slab.h>
+#include <linux/time.h>
+
+#include "mtk_vcodec_drv.h"
+#include "mtk_vcodec_util.h"
+#include "mtk_vcodec_intr.h"
+#include "mtk_vcodec_enc.h"
+#include "mtk_vcodec_pm.h"
+#include "mtk_vpu_core.h"
+
+#include "venc_vp8_if.h"
+#include "venc_vp8_vpu.h"
+
+#define vp8_enc_write_reg(h, addr, val) writel(val, h->hw_base + addr)
+#define vp8_enc_read_reg(h, addr) readl(h->hw_base + addr)
+
+#define VENC_PIC_BITSTREAM_BYTE_CNT 0x0098
+#define VENC_PIC_BITSTREAM_BYTE_CNT1 0x00e8
+#define VENC_IRQ_STATUS_ENC_FRM_INT 0x04
+
+#define MAX_AC_TAG_SZ 10
+
+static void vp8_enc_free_work_buf(struct venc_vp8_handle *hndl)
+{
+ int i;
+
+ mtk_vcodec_debug_enter(hndl);
+
+ /* Except the RC_CODEx buffers, other buffers need to be freed by AP. */
+ for (i = 0; i < VENC_VP8_VPU_WORK_BUF_RC_CODE; i++)
+ if (hndl->work_bufs[i].va != NULL)
+ mtk_vcodec_mem_free(hndl->ctx, &hndl->work_bufs[i]);
+
+ mtk_vcodec_debug_leave(hndl);
+}
+
+static int vp8_enc_alloc_work_buf(struct venc_vp8_handle *hndl)
+{
+ int i;
+ int ret = 0;
+ struct venc_vp8_vpu_buf *wb = hndl->vpu_inst.drv->work_bufs;
+
+ mtk_vcodec_debug_enter(hndl);
+ for (i = 0; i < VENC_VP8_VPU_WORK_BUF_MAX; i++) {
+ /*
+ * Only temporal scalability mode will use RC_CODE2 & RC_CODE3
+ * Each three temporal layer has its own rate control code.
+ */
+ if ((i == VENC_VP8_VPU_WORK_BUF_RC_CODE2 ||
+ i == VENC_VP8_VPU_WORK_BUF_RC_CODE3) && !hndl->ts_mode)
+ continue;
+
+ /*
+ * This 'wb' structure is set by VPU side and shared to AP for
+ * buffer allocation and physical addr mapping. For most of
+ * the buffers, AP will allocate the buffer according to 'size'
+ * field and store the physical addr in 'pa' field. For the
+ * RC_CODEx buffers, they are pre-allocated in the VPU side
+ * because they are inside VPU SRAM, and save the VPU addr in
+ * the 'vpua' field. The AP will translate the VPU addr to the
+ * corresponding physical addr and store in 'pa' field.
+ */
+ if (i < VENC_VP8_VPU_WORK_BUF_RC_CODE) {
+ hndl->work_bufs[i].size = wb[i].size;
+ ret = mtk_vcodec_mem_alloc(hndl->ctx,
+ &hndl->work_bufs[i]);
+ if (ret) {
+ mtk_vcodec_err(hndl,
+ "cannot alloc work_bufs[%d]", i);
+ goto err_alloc;
+ }
+ wb[i].pa = hndl->work_bufs[i].dma_addr;
+
+ mtk_vcodec_debug(hndl,
+ "work_bufs[%d] va=0x%p,pa=0x%p,size=0x%lx",
+ i, hndl->work_bufs[i].va,
+ (void *)hndl->work_bufs[i].dma_addr,
+ hndl->work_bufs[i].size);
+ } else {
+ hndl->work_bufs[i].size = wb[i].size;
+ hndl->work_bufs[i].va =
+ vpu_mapping_dm_addr(hndl->dev,
+ (uintptr_t *)
+ (unsigned long)wb[i].vpua);
+ hndl->work_bufs[i].dma_addr =
+ (dma_addr_t)vpu_mapping_iommu_dm_addr(hndl->dev,
+ (uintptr_t *)(unsigned long)wb[i].vpua);
+ wb[i].pa = hndl->work_bufs[i].dma_addr;
+ }
+ }
+ mtk_vcodec_debug_leave(hndl);
+
+ return ret;
+
+err_alloc:
+ vp8_enc_free_work_buf(hndl);
+ return ret;
+}
+
+static unsigned int vp8_enc_wait_venc_done(struct venc_vp8_handle *hndl)
+{
+ struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)hndl->ctx;
+ unsigned int irq_status;
+
+ mtk_vcodec_wait_for_done_ctx(ctx, MTK_INST_IRQ_RECEIVED, 1000, true);
+ irq_status = ctx->irq_status;
+ mtk_vcodec_debug(hndl, "isr return %x", irq_status);
+
+ return irq_status;
+}
+
+/*
+ * Compose ac_tag, bitstream header and bitstream payload into
+ * one bitstream buffer.
+ */
+static int vp8_enc_compose_one_frame(struct venc_vp8_handle *hndl,
+ struct mtk_vcodec_mem *bs_buf,
+ unsigned int *bs_size)
+{
+ unsigned int is_key;
+ u32 bs_size_frm;
+ u32 bs_hdr_len;
+ unsigned int ac_tag_sz;
+ u8 ac_tag[MAX_AC_TAG_SZ];
+
+ bs_size_frm = vp8_enc_read_reg(hndl,
+ VENC_PIC_BITSTREAM_BYTE_CNT);
+ bs_hdr_len = vp8_enc_read_reg(hndl,
+ VENC_PIC_BITSTREAM_BYTE_CNT1);
+
+ /* if a frame is key frame, is_key is 0 */
+ is_key = (hndl->frm_cnt %
+ hndl->vpu_inst.drv->config.intra_period) ? 1 : 0;
+ *(u32 *)ac_tag = __cpu_to_le32((bs_hdr_len << 5) | 0x10 | is_key);
+ /* key frame */
+ if (is_key == 0) {
+ ac_tag[3] = 0x9d;
+ ac_tag[4] = 0x01;
+ ac_tag[5] = 0x2a;
+ ac_tag[6] = hndl->vpu_inst.drv->config.pic_w;
+ ac_tag[7] = hndl->vpu_inst.drv->config.pic_w >> 8;
+ ac_tag[8] = hndl->vpu_inst.drv->config.pic_h;
+ ac_tag[9] = hndl->vpu_inst.drv->config.pic_h >> 8;
+ }
+
+ if (is_key == 0)
+ ac_tag_sz = MAX_AC_TAG_SZ;
+ else
+ ac_tag_sz = 3;
+
+ if (bs_buf->size <= bs_hdr_len + bs_size_frm + ac_tag_sz) {
+ mtk_vcodec_err(hndl, "bitstream buf size is too small(%ld)",
+ bs_buf->size);
+ return -EINVAL;
+ }
+
+ /*
+ * (1) The vp8 bitstream header and body are generated by the HW vp8
+ * encoder separately at the same time. We cannot know the bitstream
+ * header length in advance.
+ * (2) From the vp8 spec, there is no stuffing byte allowed between the
+ * ac tag, bitstream header and bitstream body.
+ */
+ memmove(bs_buf->va + bs_hdr_len + ac_tag_sz,
+ bs_buf->va, bs_size_frm);
+ memcpy(bs_buf->va + ac_tag_sz,
+ hndl->work_bufs[VENC_VP8_VPU_WORK_BUF_BS_HD].va,
+ bs_hdr_len);
+ memcpy(bs_buf->va, ac_tag, ac_tag_sz);
+ *bs_size = bs_size_frm + bs_hdr_len + ac_tag_sz;
+
+ return 0;
+}
+
+static int vp8_enc_encode_frame(struct venc_vp8_handle *hndl,
+ struct venc_frm_buf *frm_buf,
+ struct mtk_vcodec_mem *bs_buf,
+ unsigned int *bs_size)
+{
+ int ret = 0;
+ unsigned int irq_status;
+
+ mtk_vcodec_debug(hndl, "->frm_cnt=%d", hndl->frm_cnt);
+
+ ret = vp8_enc_vpu_encode(hndl, frm_buf, bs_buf);
+
+ irq_status = vp8_enc_wait_venc_done(hndl);
+ if (irq_status != VENC_IRQ_STATUS_ENC_FRM_INT) {
+ mtk_vcodec_err(hndl, "irq_status=%d failed", irq_status);
+ return -EINVAL;
+ }
+
+ if (vp8_enc_compose_one_frame(hndl, bs_buf, bs_size)) {
+ mtk_vcodec_err(hndl, "vp8_enc_compose_one_frame failed");
+ return -EINVAL;
+ }
+
+ hndl->frm_cnt++;
+ mtk_vcodec_debug(hndl, "<-size=%d", *bs_size);
+
+ return ret;
+}
+
+int vp8_enc_init(struct mtk_vcodec_ctx *ctx, unsigned long *handle)
+{
+ int ret = 0;
+ struct venc_vp8_handle *h;
+
+ h = kzalloc(sizeof(*h), GFP_KERNEL);
+ if (!h)
+ return -ENOMEM;
+
+ h->ctx = ctx;
+ h->dev = mtk_vcodec_get_plat_dev(ctx);
+ h->hw_base = mtk_vcodec_get_reg_addr(h->ctx, VENC_LT_SYS);
+
+ ret = vp8_enc_vpu_init(h);
+ if (ret)
+ kfree(h);
+ else
+ (*handle) = (unsigned long)h;
+
+ return ret;
+}
+
+int vp8_enc_encode(unsigned long handle,
+ enum venc_start_opt opt,
+ struct venc_frm_buf *frm_buf,
+ struct mtk_vcodec_mem *bs_buf,
+ struct venc_done_result *result)
+{
+ int ret = 0;
+ struct venc_vp8_handle *h = (struct venc_vp8_handle *)handle;
+
+ mtk_vcodec_debug_enter(h);
+
+ switch (opt) {
+ case VENC_START_OPT_ENCODE_FRAME:
+ ret = vp8_enc_encode_frame(h, frm_buf, bs_buf,
+ &result->bs_size);
+ if (ret) {
+ result->msg = VENC_MESSAGE_ERR;
+ } else {
+ result->msg = VENC_MESSAGE_OK;
+ result->is_key_frm = ((*((unsigned char *)bs_buf->va) &
+ 0x01) == 0);
+ }
+ break;
+
+ default:
+ mtk_vcodec_err(h, "opt not support:%d", opt);
+ ret = -EINVAL;
+ break;
+ }
+
+ mtk_vcodec_debug_leave(h);
+ return ret;
+}
+
+int vp8_enc_set_param(unsigned long handle,
+ enum venc_set_param_type type, void *in)
+{
+ int ret = 0;
+ struct venc_vp8_handle *h = (struct venc_vp8_handle *)handle;
+ struct venc_enc_prm *enc_prm;
+
+ mtk_vcodec_debug(h, "->type=%d", type);
+
+ switch (type) {
+ case VENC_SET_PARAM_ENC:
+ enc_prm = in;
+ ret = vp8_enc_vpu_set_param(h, type, enc_prm);
+ if (ret)
+ break;
+ if (h->work_buf_allocated == 0) {
+ ret = vp8_enc_alloc_work_buf(h);
+ if (ret)
+ break;
+ h->work_buf_allocated = 1;
+ }
+ break;
+
+ case VENC_SET_PARAM_FORCE_INTRA:
+ ret = vp8_enc_vpu_set_param(h, type, 0);
+ if (ret)
+ break;
+ h->frm_cnt = 0;
+ break;
+
+ case VENC_SET_PARAM_ADJUST_BITRATE:
+ enc_prm = in;
+ ret = vp8_enc_vpu_set_param(h, type, &enc_prm->bitrate);
+ break;
+
+ case VENC_SET_PARAM_ADJUST_FRAMERATE:
+ enc_prm = in;
+ ret = vp8_enc_vpu_set_param(h, type, &enc_prm->frm_rate);
+ break;
+
+ case VENC_SET_PARAM_I_FRAME_INTERVAL:
+ ret = vp8_enc_vpu_set_param(h, type, in);
+ if (ret)
+ break;
+ h->frm_cnt = 0; /* reset counter */
+ break;
+
+ /*
+ * VENC_SET_PARAM_TS_MODE must be called before
+ * VENC_SET_PARAM_ENC
+ */
+ case VENC_SET_PARAM_TS_MODE:
+ h->ts_mode = 1;
+ mtk_vcodec_debug(h, "set ts_mode");
+ break;
+
+ default:
+ mtk_vcodec_err(h, "type not support:%d", type);
+ ret = -EINVAL;
+ break;
+ }
+
+ mtk_vcodec_debug_leave(h);
+ return ret;
+}
+
+int vp8_enc_deinit(unsigned long handle)
+{
+ int ret = 0;
+ struct venc_vp8_handle *h = (struct venc_vp8_handle *)handle;
+
+ mtk_vcodec_debug_enter(h);
+
+ ret = vp8_enc_vpu_deinit(h);
+
+ if (h->work_buf_allocated)
+ vp8_enc_free_work_buf(h);
+
+ mtk_vcodec_debug_leave(h);
+ kfree(h);
+
+ return ret;
+}
+
+struct venc_common_if venc_vp8_if = {
+ vp8_enc_init,
+ vp8_enc_encode,
+ vp8_enc_set_param,
+ vp8_enc_deinit,
+};
+
+struct venc_common_if *get_vp8_enc_comm_if(void)
+{
+ return &venc_vp8_if;
+}
diff --git a/drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_if.h b/drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_if.h
new file mode 100644
index 0000000..57caf88
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_if.h
@@ -0,0 +1,48 @@
+/*
+ * Copyright (c) 2015 MediaTek Inc.
+ * Author: Daniel Hsiao <daniel.hsiao@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef _VENC_VP8_IF_H_
+#define _VENC_VP8_IF_H_
+
+#include "venc_drv_base.h"
+#include "venc_vp8_vpu.h"
+
+/*
+ * struct venc_vp8_handle - vp8 encoder AP driver handle
+ * @hw_base: vp8 encoder hardware register base
+ * @work_bufs: working buffer
+ * @work_buf_allocated: working buffer allocated flag
+ * @frm_cnt: encoded frame count, it's used for I-frame judgement and
+ * reset when force intra cmd received.
+ * @ts_mode: temporal scalability mode (0: disable, 1: enable)
+ * support three temporal layers - 0: 7.5fps 1: 7.5fps 2: 15fps.
+ * @vpu_inst: VPU instance to exchange information between AP and VPU
+ * @ctx: context for v4l2 layer integration
+ * @dev: device for v4l2 layer integration
+ */
+struct venc_vp8_handle {
+ void __iomem *hw_base;
+ struct mtk_vcodec_mem work_bufs[VENC_VP8_VPU_WORK_BUF_MAX];
+ bool work_buf_allocated;
+ unsigned int frm_cnt;
+ unsigned int ts_mode;
+ struct venc_vp8_vpu_inst vpu_inst;
+ void *ctx;
+ struct platform_device *dev;
+};
+
+struct venc_common_if *get_vp8_enc_comm_if(void);
+
+#endif
diff --git a/drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_vpu.c b/drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_vpu.c
new file mode 100644
index 0000000..06a1ad3
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_vpu.c
@@ -0,0 +1,245 @@
+/*
+ * Copyright (c) 2015 MediaTek Inc.
+ * Author: Daniel Hsiao <daniel.hsiao@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include "mtk_vpu_core.h"
+
+#include "venc_vp8_if.h"
+#include "venc_vp8_vpu.h"
+#include "venc_ipi_msg.h"
+
+static void handle_vp8_enc_init_msg(struct venc_vp8_handle *hndl, void *data)
+{
+ struct venc_vpu_ipi_msg_init *msg = data;
+
+ hndl->vpu_inst.id = msg->inst_id;
+ hndl->vpu_inst.drv = (struct venc_vp8_vpu_drv *)
+ vpu_mapping_dm_addr(hndl->dev,
+ (uintptr_t *)((unsigned long)msg->inst_id));
+}
+
+static void vp8_enc_vpu_ipi_handler(void *data, unsigned int len, void *priv)
+{
+ struct venc_vpu_ipi_msg_common *msg = data;
+ struct venc_vp8_handle *hndl = (struct venc_vp8_handle *)msg->venc_inst;
+
+ mtk_vcodec_debug(hndl, "->msg_id=%x status=%d",
+ msg->msg_id, msg->status);
+
+ switch (msg->msg_id) {
+ case VPU_IPIMSG_VP8_ENC_INIT_DONE:
+ handle_vp8_enc_init_msg(hndl, data);
+ break;
+ case VPU_IPIMSG_VP8_ENC_SET_PARAM_DONE:
+ case VPU_IPIMSG_VP8_ENC_ENCODE_DONE:
+ case VPU_IPIMSG_VP8_ENC_DEINIT_DONE:
+ break;
+ default:
+ mtk_vcodec_err(hndl, "unknown msg id=%x", msg->msg_id);
+ break;
+ }
+
+ hndl->vpu_inst.signaled = 1;
+ hndl->vpu_inst.failure = (msg->status != VENC_IPI_MSG_STATUS_OK);
+ wake_up_interruptible(&hndl->vpu_inst.wq_hd);
+
+ mtk_vcodec_debug_leave(hndl);
+}
+
+static int vp8_enc_vpu_wait_ack(struct venc_vp8_handle *hndl,
+ unsigned int timeout_ms)
+{
+ int ret;
+
+ mtk_vcodec_debug_enter(hndl);
+ ret = wait_event_interruptible_timeout(hndl->vpu_inst.wq_hd,
+ (hndl->vpu_inst.signaled == 1),
+ msecs_to_jiffies(timeout_ms));
+ if (0 == ret) {
+ mtk_vcodec_err(hndl, "wait vpu ack time out");
+ return -EINVAL;
+ }
+ if (-ERESTARTSYS == ret) {
+ mtk_vcodec_err(hndl, "wait vpu ack interrupted by a signal");
+ return -EINVAL;
+ }
+
+ hndl->vpu_inst.signaled = 0;
+
+ mtk_vcodec_debug_leave(hndl);
+ return 0;
+}
+
+static int vp8_enc_vpu_send_msg(struct venc_vp8_handle *hndl, void *msg,
+ int len, int wait_ack)
+{
+ int status;
+
+ mtk_vcodec_debug_enter(hndl);
+ status = vpu_ipi_send(hndl->dev, IPI_VENC_VP8, (void *)msg, len, 1);
+ if (status) {
+ mtk_vcodec_err(hndl,
+ "vpu_ipi_send msg_id=%x len=%d failed status=%d",
+ *(unsigned int *)msg, len, status);
+ return -EINVAL;
+ }
+
+ if (wait_ack && vp8_enc_vpu_wait_ack(hndl, 2000)) {
+ mtk_vcodec_err(hndl, "vp8_enc_vpu_wait_ack failed");
+ return -EINVAL;
+ }
+
+ mtk_vcodec_debug_leave(hndl);
+ return 0;
+}
+
+int vp8_enc_vpu_init(void *handle)
+{
+ int status;
+ struct venc_vp8_handle *hndl = handle;
+ struct venc_ap_ipi_msg_init out;
+
+ mtk_vcodec_debug_enter(hndl);
+ init_waitqueue_head(&hndl->vpu_inst.wq_hd);
+ hndl->vpu_inst.signaled = 0;
+ hndl->vpu_inst.failure = 0;
+
+ status = vpu_ipi_register(hndl->dev, IPI_VENC_VP8,
+ vp8_enc_vpu_ipi_handler,
+ "vp8_enc", NULL);
+ if (status) {
+ mtk_vcodec_err(hndl,
+ "vpu_ipi_register failed status=%d", status);
+ return -EINVAL;
+ }
+
+ out.msg_id = AP_IPIMSG_VP8_ENC_INIT;
+ out.venc_inst = (unsigned long)hndl;
+ if (vp8_enc_vpu_send_msg(hndl, &out, sizeof(out), 1) ||
+ hndl->vpu_inst.failure) {
+ mtk_vcodec_err(hndl, "failed");
+ return -EINVAL;
+ }
+
+ mtk_vcodec_debug_leave(hndl);
+ return 0;
+}
+
+int vp8_enc_vpu_set_param(void *handle, unsigned int id,
+ void *param)
+{
+ struct venc_vp8_handle *hndl = handle;
+ struct venc_ap_ipi_msg_set_param out;
+
+ mtk_vcodec_debug_enter(hndl);
+ out.msg_id = AP_IPIMSG_VP8_ENC_SET_PARAM;
+ out.inst_id = hndl->vpu_inst.id;
+ out.param_id = id;
+ switch (id) {
+ case VENC_SET_PARAM_ENC: {
+ struct venc_enc_prm *enc_param = (struct venc_enc_prm *)param;
+
+ hndl->vpu_inst.drv->config.input_fourcc = enc_param->input_fourcc;
+ hndl->vpu_inst.drv->config.bitrate = enc_param->bitrate;
+ hndl->vpu_inst.drv->config.pic_w = enc_param->width;
+ hndl->vpu_inst.drv->config.pic_h = enc_param->height;
+ hndl->vpu_inst.drv->config.buf_w = enc_param->buf_width;
+ hndl->vpu_inst.drv->config.buf_h = enc_param->buf_height;
+ hndl->vpu_inst.drv->config.intra_period =
+ enc_param->intra_period;
+ hndl->vpu_inst.drv->config.framerate = enc_param->frm_rate;
+ hndl->vpu_inst.drv->config.ts_mode = hndl->ts_mode;
+ out.data_item = 0;
+ break;
+ }
+ case VENC_SET_PARAM_FORCE_INTRA:
+ out.data_item = 0;
+ break;
+ case VENC_SET_PARAM_ADJUST_BITRATE:
+ out.data_item = 1;
+ out.data[0] = *(unsigned int *)param;
+ break;
+ case VENC_SET_PARAM_ADJUST_FRAMERATE:
+ out.data_item = 1;
+ out.data[0] = *(unsigned int *)param;
+ break;
+ case VENC_SET_PARAM_I_FRAME_INTERVAL:
+ out.data_item = 1;
+ out.data[0] = *(unsigned int *)param;
+ break;
+ }
+ if (vp8_enc_vpu_send_msg(hndl, &out, sizeof(out), 1) ||
+ hndl->vpu_inst.failure) {
+ mtk_vcodec_err(hndl, "failed");
+ return -EINVAL;
+ }
+
+ mtk_vcodec_debug_leave(hndl);
+ return 0;
+}
+
+int vp8_enc_vpu_encode(void *handle,
+ struct venc_frm_buf *frm_buf,
+ struct mtk_vcodec_mem *bs_buf)
+{
+ struct venc_vp8_handle *hndl = handle;
+ struct venc_ap_ipi_msg_enc out;
+
+ mtk_vcodec_debug_enter(hndl);
+ out.msg_id = AP_IPIMSG_VP8_ENC_ENCODE;
+ out.inst_id = hndl->vpu_inst.id;
+ if (frm_buf) {
+ out.input_addr[0] = frm_buf->fb_addr.dma_addr;
+ out.input_addr[1] = frm_buf->fb_addr1.dma_addr;
+ out.input_addr[2] = frm_buf->fb_addr2.dma_addr;
+ } else {
+ out.input_addr[0] = 0;
+ out.input_addr[1] = 0;
+ out.input_addr[2] = 0;
+ }
+ if (bs_buf) {
+ out.bs_addr = bs_buf->dma_addr;
+ out.bs_size = bs_buf->size;
+ } else {
+ out.bs_addr = 0;
+ out.bs_size = 0;
+ }
+
+ if (vp8_enc_vpu_send_msg(hndl, &out, sizeof(out), 1) ||
+ hndl->vpu_inst.failure) {
+ mtk_vcodec_err(hndl, "failed");
+ return -EINVAL;
+ }
+
+ mtk_vcodec_debug_leave(hndl);
+ return 0;
+}
+
+int vp8_enc_vpu_deinit(void *handle)
+{
+ struct venc_vp8_handle *hndl = handle;
+ struct venc_ap_ipi_msg_deinit out;
+
+ mtk_vcodec_debug_enter(hndl);
+ out.msg_id = AP_IPIMSG_VP8_ENC_DEINIT;
+ out.inst_id = hndl->vpu_inst.id;
+ if (vp8_enc_vpu_send_msg(hndl, &out, sizeof(out), 1) ||
+ hndl->vpu_inst.failure) {
+ mtk_vcodec_err(hndl, "failed");
+ return -EINVAL;
+ }
+
+ mtk_vcodec_debug_leave(hndl);
+ return 0;
+}
--
1.7.9.5
^ permalink raw reply related [flat|nested] 84+ messages in thread
* [RESEND RFC/PATCH 7/8] media: platform: mtk-vcodec: Add Mediatek VP8 Video Encoder Driver
@ 2015-11-17 12:54 ` Tiffany Lin
0 siblings, 0 replies; 84+ messages in thread
From: Tiffany Lin @ 2015-11-17 12:54 UTC (permalink / raw)
To: Rob Herring, Pawel Moll, Mark Rutland, Ian Campbell, Kumar Gala,
Catalin Marinas, Will Deacon, Mauro Carvalho Chehab,
Matthias Brugger, Daniel Kurtz, Sascha Hauer, Hongzhou Yang,
Hans Verkuil, Laurent Pinchart, Sakari Ailus, Geert Uytterhoeven,
Mikhail Ulyanov, Fabien Dessenne, Arnd Bergmann,
Darren Etheridge, Peter Griffin, Benoit Parrot
Cc: Tiffany Lin, Andrew-CT Chen, Eddie Huang, Yingjoe Chen,
James Liao, Daniel Hsiao, devicetree, linux-kernel,
linux-arm-kernel, linux-media, linux-mediatek
From: Daniel Hsiao <daniel.hsiao@mediatek.com>
Signed-off-by: Daniel Hsiao <daniel.hsiao@mediatek.com>
---
drivers/media/platform/mtk-vcodec/Makefile | 2 +-
drivers/media/platform/mtk-vcodec/common/Makefile | 4 +-
.../media/platform/mtk-vcodec/common/venc_drv_if.c | 6 +-
drivers/media/platform/mtk-vcodec/vp8_enc/Makefile | 9 +
.../platform/mtk-vcodec/vp8_enc/venc_vp8_if.c | 371 ++++++++++++++++++++
.../platform/mtk-vcodec/vp8_enc/venc_vp8_if.h | 48 +++
.../platform/mtk-vcodec/vp8_enc/venc_vp8_vpu.c | 245 +++++++++++++
7 files changed, 682 insertions(+), 3 deletions(-)
create mode 100644 drivers/media/platform/mtk-vcodec/vp8_enc/Makefile
create mode 100644 drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_if.c
create mode 100644 drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_if.h
create mode 100644 drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_vpu.c
diff --git a/drivers/media/platform/mtk-vcodec/Makefile b/drivers/media/platform/mtk-vcodec/Makefile
index c7f7174..b881a8b 100644
--- a/drivers/media/platform/mtk-vcodec/Makefile
+++ b/drivers/media/platform/mtk-vcodec/Makefile
@@ -4,7 +4,7 @@ obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += mtk_vcodec_intr.o \
mtk_vcodec_enc.o \
mtk_vcodec_enc_pm.o
-obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += common/
+obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += common/ vp8_enc/
ccflags-y += -I$(srctree)/drivers/media/platform/mtk-vcodec/include \
-I$(srctree)/drivers/media/platform/mtk-vcodec \
diff --git a/drivers/media/platform/mtk-vcodec/common/Makefile b/drivers/media/platform/mtk-vcodec/common/Makefile
index 477ab80..71ae856 100644
--- a/drivers/media/platform/mtk-vcodec/common/Makefile
+++ b/drivers/media/platform/mtk-vcodec/common/Makefile
@@ -5,4 +5,6 @@ ccflags-y += \
-I$(srctree)/include/ \
-I$(srctree)/drivers/media/platform/mtk-vcodec \
-I$(srctree)/drivers/media/platform/mtk-vcodec/include \
- -I$(srctree)/drivers/media/platform/mtk-vpu
\ No newline at end of file
+ -I$(srctree)/drivers/media/platform/mtk-vcodec/vp8_enc \
+ -I$(srctree)/drivers/media/platform/mtk-vpu \
+ -I$(srctree)/drivers/media/platform/mtk-vpu/vp8_enc
diff --git a/drivers/media/platform/mtk-vcodec/common/venc_drv_if.c b/drivers/media/platform/mtk-vcodec/common/venc_drv_if.c
index 9b3f025..e9be186 100644
--- a/drivers/media/platform/mtk-vcodec/common/venc_drv_if.c
+++ b/drivers/media/platform/mtk-vcodec/common/venc_drv_if.c
@@ -26,7 +26,7 @@
#include "venc_drv_if.h"
#include "venc_drv_base.h"
-
+#include "venc_vp8_if.h"
int venc_if_create(void *ctx, unsigned int fourcc, unsigned long *handle)
{
@@ -44,6 +44,10 @@ int venc_if_create(void *ctx, unsigned int fourcc, unsigned long *handle)
mtk_vcodec_debug(h, "fmt = %s handle = %p", str, h);
switch (fourcc) {
+ case V4L2_PIX_FMT_VP8:
+ h->enc_if = get_vp8_enc_comm_if();
+ break;
+ case V4L2_PIX_FMT_H264:
default:
mtk_vcodec_err(h, "invalid format %s", str);
goto err_out;
diff --git a/drivers/media/platform/mtk-vcodec/vp8_enc/Makefile b/drivers/media/platform/mtk-vcodec/vp8_enc/Makefile
new file mode 100644
index 0000000..ac78c33
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/vp8_enc/Makefile
@@ -0,0 +1,9 @@
+obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += venc_vp8_if.o venc_vp8_vpu.o
+
+ccflags-y += \
+ -I$(srctree)/include/ \
+ -I$(srctree)/drivers/media/platform/mtk-vcodec/ \
+ -I$(srctree)/drivers/media/platform/mtk-vcodec/include \
+ -I$(srctree)/drivers/media/platform/mtk-vcodec/vp8_enc \
+ -I$(srctree)/drivers/media/platform/mtk-vpu/ \
+ -I$(srctree)/drivers/media/platform/mtk-vpu/vp8_enc
diff --git a/drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_if.c b/drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_if.c
new file mode 100644
index 0000000..cc6aaf4
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_if.c
@@ -0,0 +1,371 @@
+/*
+ * Copyright (c) 2015 MediaTek Inc.
+ * Author: Daniel Hsiao <daniel.hsiao@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/interrupt.h>
+#include <linux/kernel.h>
+#include <linux/slab.h>
+#include <linux/time.h>
+
+#include "mtk_vcodec_drv.h"
+#include "mtk_vcodec_util.h"
+#include "mtk_vcodec_intr.h"
+#include "mtk_vcodec_enc.h"
+#include "mtk_vcodec_pm.h"
+#include "mtk_vpu_core.h"
+
+#include "venc_vp8_if.h"
+#include "venc_vp8_vpu.h"
+
+#define vp8_enc_write_reg(h, addr, val) writel(val, h->hw_base + addr)
+#define vp8_enc_read_reg(h, addr) readl(h->hw_base + addr)
+
+#define VENC_PIC_BITSTREAM_BYTE_CNT 0x0098
+#define VENC_PIC_BITSTREAM_BYTE_CNT1 0x00e8
+#define VENC_IRQ_STATUS_ENC_FRM_INT 0x04
+
+#define MAX_AC_TAG_SZ 10
+
+static void vp8_enc_free_work_buf(struct venc_vp8_handle *hndl)
+{
+ int i;
+
+ mtk_vcodec_debug_enter(hndl);
+
+ /* Except the RC_CODEx buffers, other buffers need to be freed by AP. */
+ for (i = 0; i < VENC_VP8_VPU_WORK_BUF_RC_CODE; i++)
+ if (hndl->work_bufs[i].va != NULL)
+ mtk_vcodec_mem_free(hndl->ctx, &hndl->work_bufs[i]);
+
+ mtk_vcodec_debug_leave(hndl);
+}
+
+static int vp8_enc_alloc_work_buf(struct venc_vp8_handle *hndl)
+{
+ int i;
+ int ret = 0;
+ struct venc_vp8_vpu_buf *wb = hndl->vpu_inst.drv->work_bufs;
+
+ mtk_vcodec_debug_enter(hndl);
+ for (i = 0; i < VENC_VP8_VPU_WORK_BUF_MAX; i++) {
+ /*
+ * Only temporal scalability mode will use RC_CODE2 & RC_CODE3
+ * Each three temporal layer has its own rate control code.
+ */
+ if ((i == VENC_VP8_VPU_WORK_BUF_RC_CODE2 ||
+ i == VENC_VP8_VPU_WORK_BUF_RC_CODE3) && !hndl->ts_mode)
+ continue;
+
+ /*
+ * This 'wb' structure is set by VPU side and shared to AP for
+ * buffer allocation and physical addr mapping. For most of
+ * the buffers, AP will allocate the buffer according to 'size'
+ * field and store the physical addr in 'pa' field. For the
+ * RC_CODEx buffers, they are pre-allocated in the VPU side
+ * because they are inside VPU SRAM, and save the VPU addr in
+ * the 'vpua' field. The AP will translate the VPU addr to the
+ * corresponding physical addr and store in 'pa' field.
+ */
+ if (i < VENC_VP8_VPU_WORK_BUF_RC_CODE) {
+ hndl->work_bufs[i].size = wb[i].size;
+ ret = mtk_vcodec_mem_alloc(hndl->ctx,
+ &hndl->work_bufs[i]);
+ if (ret) {
+ mtk_vcodec_err(hndl,
+ "cannot alloc work_bufs[%d]", i);
+ goto err_alloc;
+ }
+ wb[i].pa = hndl->work_bufs[i].dma_addr;
+
+ mtk_vcodec_debug(hndl,
+ "work_bufs[%d] va=0x%p,pa=0x%p,size=0x%lx",
+ i, hndl->work_bufs[i].va,
+ (void *)hndl->work_bufs[i].dma_addr,
+ hndl->work_bufs[i].size);
+ } else {
+ hndl->work_bufs[i].size = wb[i].size;
+ hndl->work_bufs[i].va =
+ vpu_mapping_dm_addr(hndl->dev,
+ (uintptr_t *)
+ (unsigned long)wb[i].vpua);
+ hndl->work_bufs[i].dma_addr =
+ (dma_addr_t)vpu_mapping_iommu_dm_addr(hndl->dev,
+ (uintptr_t *)(unsigned long)wb[i].vpua);
+ wb[i].pa = hndl->work_bufs[i].dma_addr;
+ }
+ }
+ mtk_vcodec_debug_leave(hndl);
+
+ return ret;
+
+err_alloc:
+ vp8_enc_free_work_buf(hndl);
+ return ret;
+}
+
+static unsigned int vp8_enc_wait_venc_done(struct venc_vp8_handle *hndl)
+{
+ struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)hndl->ctx;
+ unsigned int irq_status;
+
+ mtk_vcodec_wait_for_done_ctx(ctx, MTK_INST_IRQ_RECEIVED, 1000, true);
+ irq_status = ctx->irq_status;
+ mtk_vcodec_debug(hndl, "isr return %x", irq_status);
+
+ return irq_status;
+}
+
+/*
+ * Compose ac_tag, bitstream header and bitstream payload into
+ * one bitstream buffer.
+ */
+static int vp8_enc_compose_one_frame(struct venc_vp8_handle *hndl,
+ struct mtk_vcodec_mem *bs_buf,
+ unsigned int *bs_size)
+{
+ unsigned int is_key;
+ u32 bs_size_frm;
+ u32 bs_hdr_len;
+ unsigned int ac_tag_sz;
+ u8 ac_tag[MAX_AC_TAG_SZ];
+
+ bs_size_frm = vp8_enc_read_reg(hndl,
+ VENC_PIC_BITSTREAM_BYTE_CNT);
+ bs_hdr_len = vp8_enc_read_reg(hndl,
+ VENC_PIC_BITSTREAM_BYTE_CNT1);
+
+ /* if a frame is key frame, is_key is 0 */
+ is_key = (hndl->frm_cnt %
+ hndl->vpu_inst.drv->config.intra_period) ? 1 : 0;
+ *(u32 *)ac_tag = __cpu_to_le32((bs_hdr_len << 5) | 0x10 | is_key);
+ /* key frame */
+ if (is_key == 0) {
+ ac_tag[3] = 0x9d;
+ ac_tag[4] = 0x01;
+ ac_tag[5] = 0x2a;
+ ac_tag[6] = hndl->vpu_inst.drv->config.pic_w;
+ ac_tag[7] = hndl->vpu_inst.drv->config.pic_w >> 8;
+ ac_tag[8] = hndl->vpu_inst.drv->config.pic_h;
+ ac_tag[9] = hndl->vpu_inst.drv->config.pic_h >> 8;
+ }
+
+ if (is_key == 0)
+ ac_tag_sz = MAX_AC_TAG_SZ;
+ else
+ ac_tag_sz = 3;
+
+ if (bs_buf->size <= bs_hdr_len + bs_size_frm + ac_tag_sz) {
+ mtk_vcodec_err(hndl, "bitstream buf size is too small(%ld)",
+ bs_buf->size);
+ return -EINVAL;
+ }
+
+ /*
+ * (1) The vp8 bitstream header and body are generated by the HW vp8
+ * encoder separately at the same time. We cannot know the bitstream
+ * header length in advance.
+ * (2) From the vp8 spec, there is no stuffing byte allowed between the
+ * ac tag, bitstream header and bitstream body.
+ */
+ memmove(bs_buf->va + bs_hdr_len + ac_tag_sz,
+ bs_buf->va, bs_size_frm);
+ memcpy(bs_buf->va + ac_tag_sz,
+ hndl->work_bufs[VENC_VP8_VPU_WORK_BUF_BS_HD].va,
+ bs_hdr_len);
+ memcpy(bs_buf->va, ac_tag, ac_tag_sz);
+ *bs_size = bs_size_frm + bs_hdr_len + ac_tag_sz;
+
+ return 0;
+}
+
+static int vp8_enc_encode_frame(struct venc_vp8_handle *hndl,
+ struct venc_frm_buf *frm_buf,
+ struct mtk_vcodec_mem *bs_buf,
+ unsigned int *bs_size)
+{
+ int ret = 0;
+ unsigned int irq_status;
+
+ mtk_vcodec_debug(hndl, "->frm_cnt=%d", hndl->frm_cnt);
+
+ ret = vp8_enc_vpu_encode(hndl, frm_buf, bs_buf);
+
+ irq_status = vp8_enc_wait_venc_done(hndl);
+ if (irq_status != VENC_IRQ_STATUS_ENC_FRM_INT) {
+ mtk_vcodec_err(hndl, "irq_status=%d failed", irq_status);
+ return -EINVAL;
+ }
+
+ if (vp8_enc_compose_one_frame(hndl, bs_buf, bs_size)) {
+ mtk_vcodec_err(hndl, "vp8_enc_compose_one_frame failed");
+ return -EINVAL;
+ }
+
+ hndl->frm_cnt++;
+ mtk_vcodec_debug(hndl, "<-size=%d", *bs_size);
+
+ return ret;
+}
+
+int vp8_enc_init(struct mtk_vcodec_ctx *ctx, unsigned long *handle)
+{
+ int ret = 0;
+ struct venc_vp8_handle *h;
+
+ h = kzalloc(sizeof(*h), GFP_KERNEL);
+ if (!h)
+ return -ENOMEM;
+
+ h->ctx = ctx;
+ h->dev = mtk_vcodec_get_plat_dev(ctx);
+ h->hw_base = mtk_vcodec_get_reg_addr(h->ctx, VENC_LT_SYS);
+
+ ret = vp8_enc_vpu_init(h);
+ if (ret)
+ kfree(h);
+ else
+ (*handle) = (unsigned long)h;
+
+ return ret;
+}
+
+int vp8_enc_encode(unsigned long handle,
+ enum venc_start_opt opt,
+ struct venc_frm_buf *frm_buf,
+ struct mtk_vcodec_mem *bs_buf,
+ struct venc_done_result *result)
+{
+ int ret = 0;
+ struct venc_vp8_handle *h = (struct venc_vp8_handle *)handle;
+
+ mtk_vcodec_debug_enter(h);
+
+ switch (opt) {
+ case VENC_START_OPT_ENCODE_FRAME:
+ ret = vp8_enc_encode_frame(h, frm_buf, bs_buf,
+ &result->bs_size);
+ if (ret) {
+ result->msg = VENC_MESSAGE_ERR;
+ } else {
+ result->msg = VENC_MESSAGE_OK;
+ result->is_key_frm = ((*((unsigned char *)bs_buf->va) &
+ 0x01) == 0);
+ }
+ break;
+
+ default:
+ mtk_vcodec_err(h, "opt not support:%d", opt);
+ ret = -EINVAL;
+ break;
+ }
+
+ mtk_vcodec_debug_leave(h);
+ return ret;
+}
+
+int vp8_enc_set_param(unsigned long handle,
+ enum venc_set_param_type type, void *in)
+{
+ int ret = 0;
+ struct venc_vp8_handle *h = (struct venc_vp8_handle *)handle;
+ struct venc_enc_prm *enc_prm;
+
+ mtk_vcodec_debug(h, "->type=%d", type);
+
+ switch (type) {
+ case VENC_SET_PARAM_ENC:
+ enc_prm = in;
+ ret = vp8_enc_vpu_set_param(h, type, enc_prm);
+ if (ret)
+ break;
+ if (h->work_buf_allocated == 0) {
+ ret = vp8_enc_alloc_work_buf(h);
+ if (ret)
+ break;
+ h->work_buf_allocated = 1;
+ }
+ break;
+
+ case VENC_SET_PARAM_FORCE_INTRA:
+ ret = vp8_enc_vpu_set_param(h, type, 0);
+ if (ret)
+ break;
+ h->frm_cnt = 0;
+ break;
+
+ case VENC_SET_PARAM_ADJUST_BITRATE:
+ enc_prm = in;
+ ret = vp8_enc_vpu_set_param(h, type, &enc_prm->bitrate);
+ break;
+
+ case VENC_SET_PARAM_ADJUST_FRAMERATE:
+ enc_prm = in;
+ ret = vp8_enc_vpu_set_param(h, type, &enc_prm->frm_rate);
+ break;
+
+ case VENC_SET_PARAM_I_FRAME_INTERVAL:
+ ret = vp8_enc_vpu_set_param(h, type, in);
+ if (ret)
+ break;
+ h->frm_cnt = 0; /* reset counter */
+ break;
+
+ /*
+ * VENC_SET_PARAM_TS_MODE must be called before
+ * VENC_SET_PARAM_ENC
+ */
+ case VENC_SET_PARAM_TS_MODE:
+ h->ts_mode = 1;
+ mtk_vcodec_debug(h, "set ts_mode");
+ break;
+
+ default:
+ mtk_vcodec_err(h, "type not support:%d", type);
+ ret = -EINVAL;
+ break;
+ }
+
+ mtk_vcodec_debug_leave(h);
+ return ret;
+}
+
+int vp8_enc_deinit(unsigned long handle)
+{
+ int ret = 0;
+ struct venc_vp8_handle *h = (struct venc_vp8_handle *)handle;
+
+ mtk_vcodec_debug_enter(h);
+
+ ret = vp8_enc_vpu_deinit(h);
+
+ if (h->work_buf_allocated)
+ vp8_enc_free_work_buf(h);
+
+ mtk_vcodec_debug_leave(h);
+ kfree(h);
+
+ return ret;
+}
+
+struct venc_common_if venc_vp8_if = {
+ vp8_enc_init,
+ vp8_enc_encode,
+ vp8_enc_set_param,
+ vp8_enc_deinit,
+};
+
+struct venc_common_if *get_vp8_enc_comm_if(void)
+{
+ return &venc_vp8_if;
+}
diff --git a/drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_if.h b/drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_if.h
new file mode 100644
index 0000000..57caf88
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_if.h
@@ -0,0 +1,48 @@
+/*
+ * Copyright (c) 2015 MediaTek Inc.
+ * Author: Daniel Hsiao <daniel.hsiao@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef _VENC_VP8_IF_H_
+#define _VENC_VP8_IF_H_
+
+#include "venc_drv_base.h"
+#include "venc_vp8_vpu.h"
+
+/*
+ * struct venc_vp8_handle - vp8 encoder AP driver handle
+ * @hw_base: vp8 encoder hardware register base
+ * @work_bufs: working buffer
+ * @work_buf_allocated: working buffer allocated flag
+ * @frm_cnt: encoded frame count, it's used for I-frame judgement and
+ * reset when force intra cmd received.
+ * @ts_mode: temporal scalability mode (0: disable, 1: enable)
+ * support three temporal layers - 0: 7.5fps 1: 7.5fps 2: 15fps.
+ * @vpu_inst: VPU instance to exchange information between AP and VPU
+ * @ctx: context for v4l2 layer integration
+ * @dev: device for v4l2 layer integration
+ */
+struct venc_vp8_handle {
+ void __iomem *hw_base;
+ struct mtk_vcodec_mem work_bufs[VENC_VP8_VPU_WORK_BUF_MAX];
+ bool work_buf_allocated;
+ unsigned int frm_cnt;
+ unsigned int ts_mode;
+ struct venc_vp8_vpu_inst vpu_inst;
+ void *ctx;
+ struct platform_device *dev;
+};
+
+struct venc_common_if *get_vp8_enc_comm_if(void);
+
+#endif
diff --git a/drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_vpu.c b/drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_vpu.c
new file mode 100644
index 0000000..06a1ad3
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_vpu.c
@@ -0,0 +1,245 @@
+/*
+ * Copyright (c) 2015 MediaTek Inc.
+ * Author: Daniel Hsiao <daniel.hsiao@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include "mtk_vpu_core.h"
+
+#include "venc_vp8_if.h"
+#include "venc_vp8_vpu.h"
+#include "venc_ipi_msg.h"
+
+static void handle_vp8_enc_init_msg(struct venc_vp8_handle *hndl, void *data)
+{
+ struct venc_vpu_ipi_msg_init *msg = data;
+
+ hndl->vpu_inst.id = msg->inst_id;
+ hndl->vpu_inst.drv = (struct venc_vp8_vpu_drv *)
+ vpu_mapping_dm_addr(hndl->dev,
+ (uintptr_t *)((unsigned long)msg->inst_id));
+}
+
+static void vp8_enc_vpu_ipi_handler(void *data, unsigned int len, void *priv)
+{
+ struct venc_vpu_ipi_msg_common *msg = data;
+ struct venc_vp8_handle *hndl = (struct venc_vp8_handle *)msg->venc_inst;
+
+ mtk_vcodec_debug(hndl, "->msg_id=%x status=%d",
+ msg->msg_id, msg->status);
+
+ switch (msg->msg_id) {
+ case VPU_IPIMSG_VP8_ENC_INIT_DONE:
+ handle_vp8_enc_init_msg(hndl, data);
+ break;
+ case VPU_IPIMSG_VP8_ENC_SET_PARAM_DONE:
+ case VPU_IPIMSG_VP8_ENC_ENCODE_DONE:
+ case VPU_IPIMSG_VP8_ENC_DEINIT_DONE:
+ break;
+ default:
+ mtk_vcodec_err(hndl, "unknown msg id=%x", msg->msg_id);
+ break;
+ }
+
+ hndl->vpu_inst.signaled = 1;
+ hndl->vpu_inst.failure = (msg->status != VENC_IPI_MSG_STATUS_OK);
+ wake_up_interruptible(&hndl->vpu_inst.wq_hd);
+
+ mtk_vcodec_debug_leave(hndl);
+}
+
+static int vp8_enc_vpu_wait_ack(struct venc_vp8_handle *hndl,
+ unsigned int timeout_ms)
+{
+ int ret;
+
+ mtk_vcodec_debug_enter(hndl);
+ ret = wait_event_interruptible_timeout(hndl->vpu_inst.wq_hd,
+ (hndl->vpu_inst.signaled == 1),
+ msecs_to_jiffies(timeout_ms));
+ if (0 == ret) {
+ mtk_vcodec_err(hndl, "wait vpu ack time out");
+ return -EINVAL;
+ }
+ if (-ERESTARTSYS == ret) {
+ mtk_vcodec_err(hndl, "wait vpu ack interrupted by a signal");
+ return -EINVAL;
+ }
+
+ hndl->vpu_inst.signaled = 0;
+
+ mtk_vcodec_debug_leave(hndl);
+ return 0;
+}
+
+static int vp8_enc_vpu_send_msg(struct venc_vp8_handle *hndl, void *msg,
+ int len, int wait_ack)
+{
+ int status;
+
+ mtk_vcodec_debug_enter(hndl);
+ status = vpu_ipi_send(hndl->dev, IPI_VENC_VP8, (void *)msg, len, 1);
+ if (status) {
+ mtk_vcodec_err(hndl,
+ "vpu_ipi_send msg_id=%x len=%d failed status=%d",
+ *(unsigned int *)msg, len, status);
+ return -EINVAL;
+ }
+
+ if (wait_ack && vp8_enc_vpu_wait_ack(hndl, 2000)) {
+ mtk_vcodec_err(hndl, "vp8_enc_vpu_wait_ack failed");
+ return -EINVAL;
+ }
+
+ mtk_vcodec_debug_leave(hndl);
+ return 0;
+}
+
+int vp8_enc_vpu_init(void *handle)
+{
+ int status;
+ struct venc_vp8_handle *hndl = handle;
+ struct venc_ap_ipi_msg_init out;
+
+ mtk_vcodec_debug_enter(hndl);
+ init_waitqueue_head(&hndl->vpu_inst.wq_hd);
+ hndl->vpu_inst.signaled = 0;
+ hndl->vpu_inst.failure = 0;
+
+ status = vpu_ipi_register(hndl->dev, IPI_VENC_VP8,
+ vp8_enc_vpu_ipi_handler,
+ "vp8_enc", NULL);
+ if (status) {
+ mtk_vcodec_err(hndl,
+ "vpu_ipi_register failed status=%d", status);
+ return -EINVAL;
+ }
+
+ out.msg_id = AP_IPIMSG_VP8_ENC_INIT;
+ out.venc_inst = (unsigned long)hndl;
+ if (vp8_enc_vpu_send_msg(hndl, &out, sizeof(out), 1) ||
+ hndl->vpu_inst.failure) {
+ mtk_vcodec_err(hndl, "failed");
+ return -EINVAL;
+ }
+
+ mtk_vcodec_debug_leave(hndl);
+ return 0;
+}
+
+int vp8_enc_vpu_set_param(void *handle, unsigned int id,
+ void *param)
+{
+ struct venc_vp8_handle *hndl = handle;
+ struct venc_ap_ipi_msg_set_param out;
+
+ mtk_vcodec_debug_enter(hndl);
+ out.msg_id = AP_IPIMSG_VP8_ENC_SET_PARAM;
+ out.inst_id = hndl->vpu_inst.id;
+ out.param_id = id;
+ switch (id) {
+ case VENC_SET_PARAM_ENC: {
+ struct venc_enc_prm *enc_param = (struct venc_enc_prm *)param;
+
+ hndl->vpu_inst.drv->config.input_fourcc = enc_param->input_fourcc;
+ hndl->vpu_inst.drv->config.bitrate = enc_param->bitrate;
+ hndl->vpu_inst.drv->config.pic_w = enc_param->width;
+ hndl->vpu_inst.drv->config.pic_h = enc_param->height;
+ hndl->vpu_inst.drv->config.buf_w = enc_param->buf_width;
+ hndl->vpu_inst.drv->config.buf_h = enc_param->buf_height;
+ hndl->vpu_inst.drv->config.intra_period =
+ enc_param->intra_period;
+ hndl->vpu_inst.drv->config.framerate = enc_param->frm_rate;
+ hndl->vpu_inst.drv->config.ts_mode = hndl->ts_mode;
+ out.data_item = 0;
+ break;
+ }
+ case VENC_SET_PARAM_FORCE_INTRA:
+ out.data_item = 0;
+ break;
+ case VENC_SET_PARAM_ADJUST_BITRATE:
+ out.data_item = 1;
+ out.data[0] = *(unsigned int *)param;
+ break;
+ case VENC_SET_PARAM_ADJUST_FRAMERATE:
+ out.data_item = 1;
+ out.data[0] = *(unsigned int *)param;
+ break;
+ case VENC_SET_PARAM_I_FRAME_INTERVAL:
+ out.data_item = 1;
+ out.data[0] = *(unsigned int *)param;
+ break;
+ }
+ if (vp8_enc_vpu_send_msg(hndl, &out, sizeof(out), 1) ||
+ hndl->vpu_inst.failure) {
+ mtk_vcodec_err(hndl, "failed");
+ return -EINVAL;
+ }
+
+ mtk_vcodec_debug_leave(hndl);
+ return 0;
+}
+
+int vp8_enc_vpu_encode(void *handle,
+ struct venc_frm_buf *frm_buf,
+ struct mtk_vcodec_mem *bs_buf)
+{
+ struct venc_vp8_handle *hndl = handle;
+ struct venc_ap_ipi_msg_enc out;
+
+ mtk_vcodec_debug_enter(hndl);
+ out.msg_id = AP_IPIMSG_VP8_ENC_ENCODE;
+ out.inst_id = hndl->vpu_inst.id;
+ if (frm_buf) {
+ out.input_addr[0] = frm_buf->fb_addr.dma_addr;
+ out.input_addr[1] = frm_buf->fb_addr1.dma_addr;
+ out.input_addr[2] = frm_buf->fb_addr2.dma_addr;
+ } else {
+ out.input_addr[0] = 0;
+ out.input_addr[1] = 0;
+ out.input_addr[2] = 0;
+ }
+ if (bs_buf) {
+ out.bs_addr = bs_buf->dma_addr;
+ out.bs_size = bs_buf->size;
+ } else {
+ out.bs_addr = 0;
+ out.bs_size = 0;
+ }
+
+ if (vp8_enc_vpu_send_msg(hndl, &out, sizeof(out), 1) ||
+ hndl->vpu_inst.failure) {
+ mtk_vcodec_err(hndl, "failed");
+ return -EINVAL;
+ }
+
+ mtk_vcodec_debug_leave(hndl);
+ return 0;
+}
+
+int vp8_enc_vpu_deinit(void *handle)
+{
+ struct venc_vp8_handle *hndl = handle;
+ struct venc_ap_ipi_msg_deinit out;
+
+ mtk_vcodec_debug_enter(hndl);
+ out.msg_id = AP_IPIMSG_VP8_ENC_DEINIT;
+ out.inst_id = hndl->vpu_inst.id;
+ if (vp8_enc_vpu_send_msg(hndl, &out, sizeof(out), 1) ||
+ hndl->vpu_inst.failure) {
+ mtk_vcodec_err(hndl, "failed");
+ return -EINVAL;
+ }
+
+ mtk_vcodec_debug_leave(hndl);
+ return 0;
+}
--
1.7.9.5
^ permalink raw reply related [flat|nested] 84+ messages in thread
* [RESEND RFC/PATCH 7/8] media: platform: mtk-vcodec: Add Mediatek VP8 Video Encoder Driver
@ 2015-11-17 12:54 ` Tiffany Lin
0 siblings, 0 replies; 84+ messages in thread
From: Tiffany Lin @ 2015-11-17 12:54 UTC (permalink / raw)
To: linux-arm-kernel
From: Daniel Hsiao <daniel.hsiao@mediatek.com>
Signed-off-by: Daniel Hsiao <daniel.hsiao@mediatek.com>
---
drivers/media/platform/mtk-vcodec/Makefile | 2 +-
drivers/media/platform/mtk-vcodec/common/Makefile | 4 +-
.../media/platform/mtk-vcodec/common/venc_drv_if.c | 6 +-
drivers/media/platform/mtk-vcodec/vp8_enc/Makefile | 9 +
.../platform/mtk-vcodec/vp8_enc/venc_vp8_if.c | 371 ++++++++++++++++++++
.../platform/mtk-vcodec/vp8_enc/venc_vp8_if.h | 48 +++
.../platform/mtk-vcodec/vp8_enc/venc_vp8_vpu.c | 245 +++++++++++++
7 files changed, 682 insertions(+), 3 deletions(-)
create mode 100644 drivers/media/platform/mtk-vcodec/vp8_enc/Makefile
create mode 100644 drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_if.c
create mode 100644 drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_if.h
create mode 100644 drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_vpu.c
diff --git a/drivers/media/platform/mtk-vcodec/Makefile b/drivers/media/platform/mtk-vcodec/Makefile
index c7f7174..b881a8b 100644
--- a/drivers/media/platform/mtk-vcodec/Makefile
+++ b/drivers/media/platform/mtk-vcodec/Makefile
@@ -4,7 +4,7 @@ obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += mtk_vcodec_intr.o \
mtk_vcodec_enc.o \
mtk_vcodec_enc_pm.o
-obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += common/
+obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += common/ vp8_enc/
ccflags-y += -I$(srctree)/drivers/media/platform/mtk-vcodec/include \
-I$(srctree)/drivers/media/platform/mtk-vcodec \
diff --git a/drivers/media/platform/mtk-vcodec/common/Makefile b/drivers/media/platform/mtk-vcodec/common/Makefile
index 477ab80..71ae856 100644
--- a/drivers/media/platform/mtk-vcodec/common/Makefile
+++ b/drivers/media/platform/mtk-vcodec/common/Makefile
@@ -5,4 +5,6 @@ ccflags-y += \
-I$(srctree)/include/ \
-I$(srctree)/drivers/media/platform/mtk-vcodec \
-I$(srctree)/drivers/media/platform/mtk-vcodec/include \
- -I$(srctree)/drivers/media/platform/mtk-vpu
\ No newline at end of file
+ -I$(srctree)/drivers/media/platform/mtk-vcodec/vp8_enc \
+ -I$(srctree)/drivers/media/platform/mtk-vpu \
+ -I$(srctree)/drivers/media/platform/mtk-vpu/vp8_enc
diff --git a/drivers/media/platform/mtk-vcodec/common/venc_drv_if.c b/drivers/media/platform/mtk-vcodec/common/venc_drv_if.c
index 9b3f025..e9be186 100644
--- a/drivers/media/platform/mtk-vcodec/common/venc_drv_if.c
+++ b/drivers/media/platform/mtk-vcodec/common/venc_drv_if.c
@@ -26,7 +26,7 @@
#include "venc_drv_if.h"
#include "venc_drv_base.h"
-
+#include "venc_vp8_if.h"
int venc_if_create(void *ctx, unsigned int fourcc, unsigned long *handle)
{
@@ -44,6 +44,10 @@ int venc_if_create(void *ctx, unsigned int fourcc, unsigned long *handle)
mtk_vcodec_debug(h, "fmt = %s handle = %p", str, h);
switch (fourcc) {
+ case V4L2_PIX_FMT_VP8:
+ h->enc_if = get_vp8_enc_comm_if();
+ break;
+ case V4L2_PIX_FMT_H264:
default:
mtk_vcodec_err(h, "invalid format %s", str);
goto err_out;
diff --git a/drivers/media/platform/mtk-vcodec/vp8_enc/Makefile b/drivers/media/platform/mtk-vcodec/vp8_enc/Makefile
new file mode 100644
index 0000000..ac78c33
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/vp8_enc/Makefile
@@ -0,0 +1,9 @@
+obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += venc_vp8_if.o venc_vp8_vpu.o
+
+ccflags-y += \
+ -I$(srctree)/include/ \
+ -I$(srctree)/drivers/media/platform/mtk-vcodec/ \
+ -I$(srctree)/drivers/media/platform/mtk-vcodec/include \
+ -I$(srctree)/drivers/media/platform/mtk-vcodec/vp8_enc \
+ -I$(srctree)/drivers/media/platform/mtk-vpu/ \
+ -I$(srctree)/drivers/media/platform/mtk-vpu/vp8_enc
diff --git a/drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_if.c b/drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_if.c
new file mode 100644
index 0000000..cc6aaf4
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_if.c
@@ -0,0 +1,371 @@
+/*
+ * Copyright (c) 2015 MediaTek Inc.
+ * Author: Daniel Hsiao <daniel.hsiao@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/interrupt.h>
+#include <linux/kernel.h>
+#include <linux/slab.h>
+#include <linux/time.h>
+
+#include "mtk_vcodec_drv.h"
+#include "mtk_vcodec_util.h"
+#include "mtk_vcodec_intr.h"
+#include "mtk_vcodec_enc.h"
+#include "mtk_vcodec_pm.h"
+#include "mtk_vpu_core.h"
+
+#include "venc_vp8_if.h"
+#include "venc_vp8_vpu.h"
+
+#define vp8_enc_write_reg(h, addr, val) writel(val, h->hw_base + addr)
+#define vp8_enc_read_reg(h, addr) readl(h->hw_base + addr)
+
+#define VENC_PIC_BITSTREAM_BYTE_CNT 0x0098
+#define VENC_PIC_BITSTREAM_BYTE_CNT1 0x00e8
+#define VENC_IRQ_STATUS_ENC_FRM_INT 0x04
+
+#define MAX_AC_TAG_SZ 10
+
+static void vp8_enc_free_work_buf(struct venc_vp8_handle *hndl)
+{
+ int i;
+
+ mtk_vcodec_debug_enter(hndl);
+
+ /* Except the RC_CODEx buffers, other buffers need to be freed by AP. */
+ for (i = 0; i < VENC_VP8_VPU_WORK_BUF_RC_CODE; i++)
+ if (hndl->work_bufs[i].va != NULL)
+ mtk_vcodec_mem_free(hndl->ctx, &hndl->work_bufs[i]);
+
+ mtk_vcodec_debug_leave(hndl);
+}
+
+static int vp8_enc_alloc_work_buf(struct venc_vp8_handle *hndl)
+{
+ int i;
+ int ret = 0;
+ struct venc_vp8_vpu_buf *wb = hndl->vpu_inst.drv->work_bufs;
+
+ mtk_vcodec_debug_enter(hndl);
+ for (i = 0; i < VENC_VP8_VPU_WORK_BUF_MAX; i++) {
+ /*
+ * Only temporal scalability mode will use RC_CODE2 & RC_CODE3
+ * Each three temporal layer has its own rate control code.
+ */
+ if ((i == VENC_VP8_VPU_WORK_BUF_RC_CODE2 ||
+ i == VENC_VP8_VPU_WORK_BUF_RC_CODE3) && !hndl->ts_mode)
+ continue;
+
+ /*
+ * This 'wb' structure is set by VPU side and shared to AP for
+ * buffer allocation and physical addr mapping. For most of
+ * the buffers, AP will allocate the buffer according to 'size'
+ * field and store the physical addr in 'pa' field. For the
+ * RC_CODEx buffers, they are pre-allocated in the VPU side
+ * because they are inside VPU SRAM, and save the VPU addr in
+ * the 'vpua' field. The AP will translate the VPU addr to the
+ * corresponding physical addr and store in 'pa' field.
+ */
+ if (i < VENC_VP8_VPU_WORK_BUF_RC_CODE) {
+ hndl->work_bufs[i].size = wb[i].size;
+ ret = mtk_vcodec_mem_alloc(hndl->ctx,
+ &hndl->work_bufs[i]);
+ if (ret) {
+ mtk_vcodec_err(hndl,
+ "cannot alloc work_bufs[%d]", i);
+ goto err_alloc;
+ }
+ wb[i].pa = hndl->work_bufs[i].dma_addr;
+
+ mtk_vcodec_debug(hndl,
+ "work_bufs[%d] va=0x%p,pa=0x%p,size=0x%lx",
+ i, hndl->work_bufs[i].va,
+ (void *)hndl->work_bufs[i].dma_addr,
+ hndl->work_bufs[i].size);
+ } else {
+ hndl->work_bufs[i].size = wb[i].size;
+ hndl->work_bufs[i].va =
+ vpu_mapping_dm_addr(hndl->dev,
+ (uintptr_t *)
+ (unsigned long)wb[i].vpua);
+ hndl->work_bufs[i].dma_addr =
+ (dma_addr_t)vpu_mapping_iommu_dm_addr(hndl->dev,
+ (uintptr_t *)(unsigned long)wb[i].vpua);
+ wb[i].pa = hndl->work_bufs[i].dma_addr;
+ }
+ }
+ mtk_vcodec_debug_leave(hndl);
+
+ return ret;
+
+err_alloc:
+ vp8_enc_free_work_buf(hndl);
+ return ret;
+}
+
+static unsigned int vp8_enc_wait_venc_done(struct venc_vp8_handle *hndl)
+{
+ struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)hndl->ctx;
+ unsigned int irq_status;
+
+ mtk_vcodec_wait_for_done_ctx(ctx, MTK_INST_IRQ_RECEIVED, 1000, true);
+ irq_status = ctx->irq_status;
+ mtk_vcodec_debug(hndl, "isr return %x", irq_status);
+
+ return irq_status;
+}
+
+/*
+ * Compose ac_tag, bitstream header and bitstream payload into
+ * one bitstream buffer.
+ */
+static int vp8_enc_compose_one_frame(struct venc_vp8_handle *hndl,
+ struct mtk_vcodec_mem *bs_buf,
+ unsigned int *bs_size)
+{
+ unsigned int is_key;
+ u32 bs_size_frm;
+ u32 bs_hdr_len;
+ unsigned int ac_tag_sz;
+ u8 ac_tag[MAX_AC_TAG_SZ];
+
+ bs_size_frm = vp8_enc_read_reg(hndl,
+ VENC_PIC_BITSTREAM_BYTE_CNT);
+ bs_hdr_len = vp8_enc_read_reg(hndl,
+ VENC_PIC_BITSTREAM_BYTE_CNT1);
+
+ /* if a frame is key frame, is_key is 0 */
+ is_key = (hndl->frm_cnt %
+ hndl->vpu_inst.drv->config.intra_period) ? 1 : 0;
+ *(u32 *)ac_tag = __cpu_to_le32((bs_hdr_len << 5) | 0x10 | is_key);
+ /* key frame */
+ if (is_key == 0) {
+ ac_tag[3] = 0x9d;
+ ac_tag[4] = 0x01;
+ ac_tag[5] = 0x2a;
+ ac_tag[6] = hndl->vpu_inst.drv->config.pic_w;
+ ac_tag[7] = hndl->vpu_inst.drv->config.pic_w >> 8;
+ ac_tag[8] = hndl->vpu_inst.drv->config.pic_h;
+ ac_tag[9] = hndl->vpu_inst.drv->config.pic_h >> 8;
+ }
+
+ if (is_key == 0)
+ ac_tag_sz = MAX_AC_TAG_SZ;
+ else
+ ac_tag_sz = 3;
+
+ if (bs_buf->size <= bs_hdr_len + bs_size_frm + ac_tag_sz) {
+ mtk_vcodec_err(hndl, "bitstream buf size is too small(%ld)",
+ bs_buf->size);
+ return -EINVAL;
+ }
+
+ /*
+ * (1) The vp8 bitstream header and body are generated by the HW vp8
+ * encoder separately at the same time. We cannot know the bitstream
+ * header length in advance.
+ * (2) From the vp8 spec, there is no stuffing byte allowed between the
+ * ac tag, bitstream header and bitstream body.
+ */
+ memmove(bs_buf->va + bs_hdr_len + ac_tag_sz,
+ bs_buf->va, bs_size_frm);
+ memcpy(bs_buf->va + ac_tag_sz,
+ hndl->work_bufs[VENC_VP8_VPU_WORK_BUF_BS_HD].va,
+ bs_hdr_len);
+ memcpy(bs_buf->va, ac_tag, ac_tag_sz);
+ *bs_size = bs_size_frm + bs_hdr_len + ac_tag_sz;
+
+ return 0;
+}
+
+static int vp8_enc_encode_frame(struct venc_vp8_handle *hndl,
+ struct venc_frm_buf *frm_buf,
+ struct mtk_vcodec_mem *bs_buf,
+ unsigned int *bs_size)
+{
+ int ret = 0;
+ unsigned int irq_status;
+
+ mtk_vcodec_debug(hndl, "->frm_cnt=%d", hndl->frm_cnt);
+
+ ret = vp8_enc_vpu_encode(hndl, frm_buf, bs_buf);
+
+ irq_status = vp8_enc_wait_venc_done(hndl);
+ if (irq_status != VENC_IRQ_STATUS_ENC_FRM_INT) {
+ mtk_vcodec_err(hndl, "irq_status=%d failed", irq_status);
+ return -EINVAL;
+ }
+
+ if (vp8_enc_compose_one_frame(hndl, bs_buf, bs_size)) {
+ mtk_vcodec_err(hndl, "vp8_enc_compose_one_frame failed");
+ return -EINVAL;
+ }
+
+ hndl->frm_cnt++;
+ mtk_vcodec_debug(hndl, "<-size=%d", *bs_size);
+
+ return ret;
+}
+
+int vp8_enc_init(struct mtk_vcodec_ctx *ctx, unsigned long *handle)
+{
+ int ret = 0;
+ struct venc_vp8_handle *h;
+
+ h = kzalloc(sizeof(*h), GFP_KERNEL);
+ if (!h)
+ return -ENOMEM;
+
+ h->ctx = ctx;
+ h->dev = mtk_vcodec_get_plat_dev(ctx);
+ h->hw_base = mtk_vcodec_get_reg_addr(h->ctx, VENC_LT_SYS);
+
+ ret = vp8_enc_vpu_init(h);
+ if (ret)
+ kfree(h);
+ else
+ (*handle) = (unsigned long)h;
+
+ return ret;
+}
+
+int vp8_enc_encode(unsigned long handle,
+ enum venc_start_opt opt,
+ struct venc_frm_buf *frm_buf,
+ struct mtk_vcodec_mem *bs_buf,
+ struct venc_done_result *result)
+{
+ int ret = 0;
+ struct venc_vp8_handle *h = (struct venc_vp8_handle *)handle;
+
+ mtk_vcodec_debug_enter(h);
+
+ switch (opt) {
+ case VENC_START_OPT_ENCODE_FRAME:
+ ret = vp8_enc_encode_frame(h, frm_buf, bs_buf,
+ &result->bs_size);
+ if (ret) {
+ result->msg = VENC_MESSAGE_ERR;
+ } else {
+ result->msg = VENC_MESSAGE_OK;
+ result->is_key_frm = ((*((unsigned char *)bs_buf->va) &
+ 0x01) == 0);
+ }
+ break;
+
+ default:
+ mtk_vcodec_err(h, "opt not support:%d", opt);
+ ret = -EINVAL;
+ break;
+ }
+
+ mtk_vcodec_debug_leave(h);
+ return ret;
+}
+
+int vp8_enc_set_param(unsigned long handle,
+ enum venc_set_param_type type, void *in)
+{
+ int ret = 0;
+ struct venc_vp8_handle *h = (struct venc_vp8_handle *)handle;
+ struct venc_enc_prm *enc_prm;
+
+ mtk_vcodec_debug(h, "->type=%d", type);
+
+ switch (type) {
+ case VENC_SET_PARAM_ENC:
+ enc_prm = in;
+ ret = vp8_enc_vpu_set_param(h, type, enc_prm);
+ if (ret)
+ break;
+ if (h->work_buf_allocated == 0) {
+ ret = vp8_enc_alloc_work_buf(h);
+ if (ret)
+ break;
+ h->work_buf_allocated = 1;
+ }
+ break;
+
+ case VENC_SET_PARAM_FORCE_INTRA:
+ ret = vp8_enc_vpu_set_param(h, type, 0);
+ if (ret)
+ break;
+ h->frm_cnt = 0;
+ break;
+
+ case VENC_SET_PARAM_ADJUST_BITRATE:
+ enc_prm = in;
+ ret = vp8_enc_vpu_set_param(h, type, &enc_prm->bitrate);
+ break;
+
+ case VENC_SET_PARAM_ADJUST_FRAMERATE:
+ enc_prm = in;
+ ret = vp8_enc_vpu_set_param(h, type, &enc_prm->frm_rate);
+ break;
+
+ case VENC_SET_PARAM_I_FRAME_INTERVAL:
+ ret = vp8_enc_vpu_set_param(h, type, in);
+ if (ret)
+ break;
+ h->frm_cnt = 0; /* reset counter */
+ break;
+
+ /*
+ * VENC_SET_PARAM_TS_MODE must be called before
+ * VENC_SET_PARAM_ENC
+ */
+ case VENC_SET_PARAM_TS_MODE:
+ h->ts_mode = 1;
+ mtk_vcodec_debug(h, "set ts_mode");
+ break;
+
+ default:
+ mtk_vcodec_err(h, "type not support:%d", type);
+ ret = -EINVAL;
+ break;
+ }
+
+ mtk_vcodec_debug_leave(h);
+ return ret;
+}
+
+int vp8_enc_deinit(unsigned long handle)
+{
+ int ret = 0;
+ struct venc_vp8_handle *h = (struct venc_vp8_handle *)handle;
+
+ mtk_vcodec_debug_enter(h);
+
+ ret = vp8_enc_vpu_deinit(h);
+
+ if (h->work_buf_allocated)
+ vp8_enc_free_work_buf(h);
+
+ mtk_vcodec_debug_leave(h);
+ kfree(h);
+
+ return ret;
+}
+
+struct venc_common_if venc_vp8_if = {
+ vp8_enc_init,
+ vp8_enc_encode,
+ vp8_enc_set_param,
+ vp8_enc_deinit,
+};
+
+struct venc_common_if *get_vp8_enc_comm_if(void)
+{
+ return &venc_vp8_if;
+}
diff --git a/drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_if.h b/drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_if.h
new file mode 100644
index 0000000..57caf88
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_if.h
@@ -0,0 +1,48 @@
+/*
+ * Copyright (c) 2015 MediaTek Inc.
+ * Author: Daniel Hsiao <daniel.hsiao@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef _VENC_VP8_IF_H_
+#define _VENC_VP8_IF_H_
+
+#include "venc_drv_base.h"
+#include "venc_vp8_vpu.h"
+
+/*
+ * struct venc_vp8_handle - vp8 encoder AP driver handle
+ * @hw_base: vp8 encoder hardware register base
+ * @work_bufs: working buffer
+ * @work_buf_allocated: working buffer allocated flag
+ * @frm_cnt: encoded frame count, it's used for I-frame judgement and
+ * reset when force intra cmd received.
+ * @ts_mode: temporal scalability mode (0: disable, 1: enable)
+ * support three temporal layers - 0: 7.5fps 1: 7.5fps 2: 15fps.
+ * @vpu_inst: VPU instance to exchange information between AP and VPU
+ * @ctx: context for v4l2 layer integration
+ * @dev: device for v4l2 layer integration
+ */
+struct venc_vp8_handle {
+ void __iomem *hw_base;
+ struct mtk_vcodec_mem work_bufs[VENC_VP8_VPU_WORK_BUF_MAX];
+ bool work_buf_allocated;
+ unsigned int frm_cnt;
+ unsigned int ts_mode;
+ struct venc_vp8_vpu_inst vpu_inst;
+ void *ctx;
+ struct platform_device *dev;
+};
+
+struct venc_common_if *get_vp8_enc_comm_if(void);
+
+#endif
diff --git a/drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_vpu.c b/drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_vpu.c
new file mode 100644
index 0000000..06a1ad3
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_vpu.c
@@ -0,0 +1,245 @@
+/*
+ * Copyright (c) 2015 MediaTek Inc.
+ * Author: Daniel Hsiao <daniel.hsiao@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include "mtk_vpu_core.h"
+
+#include "venc_vp8_if.h"
+#include "venc_vp8_vpu.h"
+#include "venc_ipi_msg.h"
+
+static void handle_vp8_enc_init_msg(struct venc_vp8_handle *hndl, void *data)
+{
+ struct venc_vpu_ipi_msg_init *msg = data;
+
+ hndl->vpu_inst.id = msg->inst_id;
+ hndl->vpu_inst.drv = (struct venc_vp8_vpu_drv *)
+ vpu_mapping_dm_addr(hndl->dev,
+ (uintptr_t *)((unsigned long)msg->inst_id));
+}
+
+static void vp8_enc_vpu_ipi_handler(void *data, unsigned int len, void *priv)
+{
+ struct venc_vpu_ipi_msg_common *msg = data;
+ struct venc_vp8_handle *hndl = (struct venc_vp8_handle *)msg->venc_inst;
+
+ mtk_vcodec_debug(hndl, "->msg_id=%x status=%d",
+ msg->msg_id, msg->status);
+
+ switch (msg->msg_id) {
+ case VPU_IPIMSG_VP8_ENC_INIT_DONE:
+ handle_vp8_enc_init_msg(hndl, data);
+ break;
+ case VPU_IPIMSG_VP8_ENC_SET_PARAM_DONE:
+ case VPU_IPIMSG_VP8_ENC_ENCODE_DONE:
+ case VPU_IPIMSG_VP8_ENC_DEINIT_DONE:
+ break;
+ default:
+ mtk_vcodec_err(hndl, "unknown msg id=%x", msg->msg_id);
+ break;
+ }
+
+ hndl->vpu_inst.signaled = 1;
+ hndl->vpu_inst.failure = (msg->status != VENC_IPI_MSG_STATUS_OK);
+ wake_up_interruptible(&hndl->vpu_inst.wq_hd);
+
+ mtk_vcodec_debug_leave(hndl);
+}
+
+static int vp8_enc_vpu_wait_ack(struct venc_vp8_handle *hndl,
+ unsigned int timeout_ms)
+{
+ int ret;
+
+ mtk_vcodec_debug_enter(hndl);
+ ret = wait_event_interruptible_timeout(hndl->vpu_inst.wq_hd,
+ (hndl->vpu_inst.signaled == 1),
+ msecs_to_jiffies(timeout_ms));
+ if (0 == ret) {
+ mtk_vcodec_err(hndl, "wait vpu ack time out");
+ return -EINVAL;
+ }
+ if (-ERESTARTSYS == ret) {
+ mtk_vcodec_err(hndl, "wait vpu ack interrupted by a signal");
+ return -EINVAL;
+ }
+
+ hndl->vpu_inst.signaled = 0;
+
+ mtk_vcodec_debug_leave(hndl);
+ return 0;
+}
+
+static int vp8_enc_vpu_send_msg(struct venc_vp8_handle *hndl, void *msg,
+ int len, int wait_ack)
+{
+ int status;
+
+ mtk_vcodec_debug_enter(hndl);
+ status = vpu_ipi_send(hndl->dev, IPI_VENC_VP8, (void *)msg, len, 1);
+ if (status) {
+ mtk_vcodec_err(hndl,
+ "vpu_ipi_send msg_id=%x len=%d failed status=%d",
+ *(unsigned int *)msg, len, status);
+ return -EINVAL;
+ }
+
+ if (wait_ack && vp8_enc_vpu_wait_ack(hndl, 2000)) {
+ mtk_vcodec_err(hndl, "vp8_enc_vpu_wait_ack failed");
+ return -EINVAL;
+ }
+
+ mtk_vcodec_debug_leave(hndl);
+ return 0;
+}
+
+int vp8_enc_vpu_init(void *handle)
+{
+ int status;
+ struct venc_vp8_handle *hndl = handle;
+ struct venc_ap_ipi_msg_init out;
+
+ mtk_vcodec_debug_enter(hndl);
+ init_waitqueue_head(&hndl->vpu_inst.wq_hd);
+ hndl->vpu_inst.signaled = 0;
+ hndl->vpu_inst.failure = 0;
+
+ status = vpu_ipi_register(hndl->dev, IPI_VENC_VP8,
+ vp8_enc_vpu_ipi_handler,
+ "vp8_enc", NULL);
+ if (status) {
+ mtk_vcodec_err(hndl,
+ "vpu_ipi_register failed status=%d", status);
+ return -EINVAL;
+ }
+
+ out.msg_id = AP_IPIMSG_VP8_ENC_INIT;
+ out.venc_inst = (unsigned long)hndl;
+ if (vp8_enc_vpu_send_msg(hndl, &out, sizeof(out), 1) ||
+ hndl->vpu_inst.failure) {
+ mtk_vcodec_err(hndl, "failed");
+ return -EINVAL;
+ }
+
+ mtk_vcodec_debug_leave(hndl);
+ return 0;
+}
+
+int vp8_enc_vpu_set_param(void *handle, unsigned int id,
+ void *param)
+{
+ struct venc_vp8_handle *hndl = handle;
+ struct venc_ap_ipi_msg_set_param out;
+
+ mtk_vcodec_debug_enter(hndl);
+ out.msg_id = AP_IPIMSG_VP8_ENC_SET_PARAM;
+ out.inst_id = hndl->vpu_inst.id;
+ out.param_id = id;
+ switch (id) {
+ case VENC_SET_PARAM_ENC: {
+ struct venc_enc_prm *enc_param = (struct venc_enc_prm *)param;
+
+ hndl->vpu_inst.drv->config.input_fourcc = enc_param->input_fourcc;
+ hndl->vpu_inst.drv->config.bitrate = enc_param->bitrate;
+ hndl->vpu_inst.drv->config.pic_w = enc_param->width;
+ hndl->vpu_inst.drv->config.pic_h = enc_param->height;
+ hndl->vpu_inst.drv->config.buf_w = enc_param->buf_width;
+ hndl->vpu_inst.drv->config.buf_h = enc_param->buf_height;
+ hndl->vpu_inst.drv->config.intra_period =
+ enc_param->intra_period;
+ hndl->vpu_inst.drv->config.framerate = enc_param->frm_rate;
+ hndl->vpu_inst.drv->config.ts_mode = hndl->ts_mode;
+ out.data_item = 0;
+ break;
+ }
+ case VENC_SET_PARAM_FORCE_INTRA:
+ out.data_item = 0;
+ break;
+ case VENC_SET_PARAM_ADJUST_BITRATE:
+ out.data_item = 1;
+ out.data[0] = *(unsigned int *)param;
+ break;
+ case VENC_SET_PARAM_ADJUST_FRAMERATE:
+ out.data_item = 1;
+ out.data[0] = *(unsigned int *)param;
+ break;
+ case VENC_SET_PARAM_I_FRAME_INTERVAL:
+ out.data_item = 1;
+ out.data[0] = *(unsigned int *)param;
+ break;
+ }
+ if (vp8_enc_vpu_send_msg(hndl, &out, sizeof(out), 1) ||
+ hndl->vpu_inst.failure) {
+ mtk_vcodec_err(hndl, "failed");
+ return -EINVAL;
+ }
+
+ mtk_vcodec_debug_leave(hndl);
+ return 0;
+}
+
+int vp8_enc_vpu_encode(void *handle,
+ struct venc_frm_buf *frm_buf,
+ struct mtk_vcodec_mem *bs_buf)
+{
+ struct venc_vp8_handle *hndl = handle;
+ struct venc_ap_ipi_msg_enc out;
+
+ mtk_vcodec_debug_enter(hndl);
+ out.msg_id = AP_IPIMSG_VP8_ENC_ENCODE;
+ out.inst_id = hndl->vpu_inst.id;
+ if (frm_buf) {
+ out.input_addr[0] = frm_buf->fb_addr.dma_addr;
+ out.input_addr[1] = frm_buf->fb_addr1.dma_addr;
+ out.input_addr[2] = frm_buf->fb_addr2.dma_addr;
+ } else {
+ out.input_addr[0] = 0;
+ out.input_addr[1] = 0;
+ out.input_addr[2] = 0;
+ }
+ if (bs_buf) {
+ out.bs_addr = bs_buf->dma_addr;
+ out.bs_size = bs_buf->size;
+ } else {
+ out.bs_addr = 0;
+ out.bs_size = 0;
+ }
+
+ if (vp8_enc_vpu_send_msg(hndl, &out, sizeof(out), 1) ||
+ hndl->vpu_inst.failure) {
+ mtk_vcodec_err(hndl, "failed");
+ return -EINVAL;
+ }
+
+ mtk_vcodec_debug_leave(hndl);
+ return 0;
+}
+
+int vp8_enc_vpu_deinit(void *handle)
+{
+ struct venc_vp8_handle *hndl = handle;
+ struct venc_ap_ipi_msg_deinit out;
+
+ mtk_vcodec_debug_enter(hndl);
+ out.msg_id = AP_IPIMSG_VP8_ENC_DEINIT;
+ out.inst_id = hndl->vpu_inst.id;
+ if (vp8_enc_vpu_send_msg(hndl, &out, sizeof(out), 1) ||
+ hndl->vpu_inst.failure) {
+ mtk_vcodec_err(hndl, "failed");
+ return -EINVAL;
+ }
+
+ mtk_vcodec_debug_leave(hndl);
+ return 0;
+}
--
1.7.9.5
^ permalink raw reply related [flat|nested] 84+ messages in thread
* [RESEND RFC/PATCH 8/8] media: platform: mtk-vcodec: Add Mediatek H264 Video Encoder Driver
2015-11-17 12:54 ` Tiffany Lin
(?)
@ 2015-11-17 12:54 ` Tiffany Lin
-1 siblings, 0 replies; 84+ messages in thread
From: Tiffany Lin @ 2015-11-17 12:54 UTC (permalink / raw)
To: Rob Herring, Pawel Moll, Mark Rutland, Ian Campbell, Kumar Gala,
Catalin Marinas, Will Deacon, Mauro Carvalho Chehab,
Matthias Brugger, Daniel Kurtz, Sascha Hauer, Hongzhou Yang,
Hans Verkuil, Laurent Pinchart, Sakari Ailus, Geert Uytterhoeven,
Mikhail Ulyanov, Fabien Dessenne, Arnd Bergmann,
Darren Etheridge, Peter Griffin, Benoit Parrot
Cc: Tiffany Lin, Andrew-CT Chen, Eddie Huang, Yingjoe Chen,
James Liao, Daniel Hsiao, devicetree, linux-kernel,
linux-arm-kernel, linux-media, linux-mediatek
Signed-off-by: Daniel Hsiao <daniel.hsiao@mediatek.com>
---
drivers/media/platform/mtk-vcodec/Makefile | 2 +-
drivers/media/platform/mtk-vcodec/common/Makefile | 4 +-
.../media/platform/mtk-vcodec/common/venc_drv_if.c | 3 +
.../media/platform/mtk-vcodec/h264_enc/Makefile | 9 +
.../platform/mtk-vcodec/h264_enc/venc_h264_if.c | 529 ++++++++++++++++++++
.../platform/mtk-vcodec/h264_enc/venc_h264_if.h | 53 ++
.../platform/mtk-vcodec/h264_enc/venc_h264_vpu.c | 341 +++++++++++++
7 files changed, 939 insertions(+), 2 deletions(-)
create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/Makefile
create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.c
create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.h
create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.c
diff --git a/drivers/media/platform/mtk-vcodec/Makefile b/drivers/media/platform/mtk-vcodec/Makefile
index b881a8b..d2189f7 100644
--- a/drivers/media/platform/mtk-vcodec/Makefile
+++ b/drivers/media/platform/mtk-vcodec/Makefile
@@ -4,7 +4,7 @@ obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += mtk_vcodec_intr.o \
mtk_vcodec_enc.o \
mtk_vcodec_enc_pm.o
-obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += common/ vp8_enc/
+obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += common/ vp8_enc/ h264_enc/
ccflags-y += -I$(srctree)/drivers/media/platform/mtk-vcodec/include \
-I$(srctree)/drivers/media/platform/mtk-vcodec \
diff --git a/drivers/media/platform/mtk-vcodec/common/Makefile b/drivers/media/platform/mtk-vcodec/common/Makefile
index 71ae856..b33d48d 100644
--- a/drivers/media/platform/mtk-vcodec/common/Makefile
+++ b/drivers/media/platform/mtk-vcodec/common/Makefile
@@ -6,5 +6,7 @@ ccflags-y += \
-I$(srctree)/drivers/media/platform/mtk-vcodec \
-I$(srctree)/drivers/media/platform/mtk-vcodec/include \
-I$(srctree)/drivers/media/platform/mtk-vcodec/vp8_enc \
+ -I$(srctree)/drivers/media/platform/mtk-vcodec/h264_enc \
-I$(srctree)/drivers/media/platform/mtk-vpu \
- -I$(srctree)/drivers/media/platform/mtk-vpu/vp8_enc
+ -I$(srctree)/drivers/media/platform/mtk-vpu/vp8_enc \
+ -I$(srctree)/drivers/media/platform/mtk-vpu/h264_enc
diff --git a/drivers/media/platform/mtk-vcodec/common/venc_drv_if.c b/drivers/media/platform/mtk-vcodec/common/venc_drv_if.c
index e9be186..930254b 100644
--- a/drivers/media/platform/mtk-vcodec/common/venc_drv_if.c
+++ b/drivers/media/platform/mtk-vcodec/common/venc_drv_if.c
@@ -27,6 +27,7 @@
#include "venc_drv_if.h"
#include "venc_drv_base.h"
#include "venc_vp8_if.h"
+#include "venc_h264_if.h"
int venc_if_create(void *ctx, unsigned int fourcc, unsigned long *handle)
{
@@ -48,6 +49,8 @@ int venc_if_create(void *ctx, unsigned int fourcc, unsigned long *handle)
h->enc_if = get_vp8_enc_comm_if();
break;
case V4L2_PIX_FMT_H264:
+ h->enc_if = get_h264_enc_comm_if();
+ break;
default:
mtk_vcodec_err(h, "invalid format %s", str);
goto err_out;
diff --git a/drivers/media/platform/mtk-vcodec/h264_enc/Makefile b/drivers/media/platform/mtk-vcodec/h264_enc/Makefile
new file mode 100644
index 0000000..6559908
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/h264_enc/Makefile
@@ -0,0 +1,9 @@
+obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += venc_h264_if.o venc_h264_vpu.o
+
+ccflags-y += \
+ -I$(srctree)/include/ \
+ -I$(srctree)/drivers/media/platform/mtk-vcodec/ \
+ -I$(srctree)/drivers/media/platform/mtk-vcodec/include \
+ -I$(srctree)/drivers/media/platform/mtk-vcodec/h264_enc \
+ -I$(srctree)/drivers/media/platform/mtk-vpu \
+ -I$(srctree)/drivers/media/platform/mtk-vpu/h264_enc
diff --git a/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.c b/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.c
new file mode 100644
index 0000000..c880865
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.c
@@ -0,0 +1,529 @@
+/*
+ * Copyright (c) 2015 MediaTek Inc.
+ * Author: Jungchang Tsao <jungchang.tsao@mediatek.com>
+ * Daniel Hsiao <daniel.hsiao@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/kernel.h>
+#include <linux/slab.h>
+#include <linux/interrupt.h>
+
+#include "mtk_vcodec_drv.h"
+#include "mtk_vcodec_util.h"
+#include "mtk_vcodec_intr.h"
+#include "mtk_vcodec_enc.h"
+#include "mtk_vcodec_pm.h"
+#include "mtk_vpu_core.h"
+
+#include "venc_h264_if.h"
+#include "venc_h264_vpu.h"
+
+#define h264_write_reg(h, addr, val) writel(val, h->hw_base + addr)
+#define h264_read_reg(h, addr) readl(h->hw_base + addr)
+
+#define VENC_PIC_BITSTREAM_BYTE_CNT 0x0098
+
+enum venc_h264_irq_status {
+ H264_IRQ_STATUS_ENC_SPS_INT = (1 << 0),
+ H264_IRQ_STATUS_ENC_PPS_INT = (1 << 1),
+ H264_IRQ_STATUS_ENC_FRM_INT = (1 << 2),
+};
+
+static int h264_enc_alloc_work_buf(struct venc_h264_handle *handle)
+{
+ int i, j;
+ int ret = 0;
+ struct venc_h264_vpu_buf *wb = handle->vpu_inst.drv->work_bufs;
+
+ mtk_vcodec_debug_enter(handle);
+
+ for (i = 0; i < VENC_H264_VPU_WORK_BUF_MAX; i++) {
+ /*
+ * This 'wb' structure is set by VPU side and shared to AP for
+ * buffer allocation and physical addr mapping. For most of
+ * the buffers, AP will allocate the buffer according to 'size'
+ * field and store the physical addr in 'pa' field. There are two
+ * exceptions:
+ * (1) RC_CODE buffer, it's pre-allocated in the VPU side, and
+ * save the VPU addr in the 'vpua' field. The AP will translate
+ * the VPU addr to the corresponding physical addr and store
+ * in 'pa' field for reg setting in VPU side.
+ * (2) SKIP_FRAME buffer, it's pre-allocated in the VPU side, and
+ * save the VPU addr in the 'vpua' field. The AP will translate
+ * the VPU addr to the corresponding AP side virtual address and
+ * do some memcpy access to move to bitstream buffer assigned
+ * by v4l2 layer.
+ */
+ if (i == VENC_H264_VPU_WORK_BUF_RC_CODE) {
+ handle->work_bufs[i].size = wb[i].size;
+ handle->work_bufs[i].va = vpu_mapping_dm_addr(
+ handle->dev, (uintptr_t *)(unsigned long)
+ wb[i].vpua);
+ handle->work_bufs[i].dma_addr =
+ (dma_addr_t)vpu_mapping_iommu_dm_addr(
+ handle->dev, (uintptr_t *)(unsigned long)
+ wb[i].vpua);
+ wb[i].pa = handle->work_bufs[i].dma_addr;
+ } else if (i == VENC_H264_VPU_WORK_BUF_SKIP_FRAME) {
+ handle->work_bufs[i].size = wb[i].size;
+ handle->work_bufs[i].va = vpu_mapping_dm_addr(
+ handle->dev, (uintptr_t *)(unsigned long)
+ wb[i].vpua);
+ handle->work_bufs[i].dma_addr = 0;
+ wb[i].pa = handle->work_bufs[i].dma_addr;
+ } else {
+ handle->work_bufs[i].size = wb[i].size;
+ if (mtk_vcodec_mem_alloc(handle->ctx,
+ &handle->work_bufs[i])) {
+ mtk_vcodec_err(handle, "cannot allocate buf %d", i);
+ ret = -ENOMEM;
+ goto err_alloc;
+ }
+ wb[i].pa = handle->work_bufs[i].dma_addr;
+ }
+ mtk_vcodec_debug(handle, "buf[%d] va=0x%p pa=0x%p size=0x%lx", i,
+ handle->work_bufs[i].va,
+ (void *)handle->work_bufs[i].dma_addr,
+ handle->work_bufs[i].size);
+ }
+
+ /* the pps_buf is used by AP side only */
+ handle->pps_buf.size = 128;
+ if (mtk_vcodec_mem_alloc(handle->ctx,
+ &handle->pps_buf)) {
+ mtk_vcodec_err(handle, "cannot allocate pps_buf");
+ ret = -ENOMEM;
+ goto err_alloc;
+ }
+ mtk_vcodec_debug_leave(handle);
+
+ return ret;
+
+err_alloc:
+ for (j = 0; j < i; j++) {
+ if ((j != VENC_H264_VPU_WORK_BUF_RC_CODE) &&
+ (j != VENC_H264_VPU_WORK_BUF_SKIP_FRAME))
+ mtk_vcodec_mem_free(handle->ctx, &handle->work_bufs[j]);
+ }
+
+ return ret;
+}
+
+static void h264_enc_free_work_buf(struct venc_h264_handle *handle)
+{
+ int i;
+
+ mtk_vcodec_debug_enter(handle);
+ for (i = 0; i < VENC_H264_VPU_WORK_BUF_MAX; i++) {
+ if ((i != VENC_H264_VPU_WORK_BUF_RC_CODE) &&
+ (i != VENC_H264_VPU_WORK_BUF_SKIP_FRAME))
+ mtk_vcodec_mem_free(handle->ctx, &handle->work_bufs[i]);
+ }
+ mtk_vcodec_mem_free(handle->ctx, &handle->pps_buf);
+ mtk_vcodec_debug_leave(handle);
+}
+
+static unsigned int h264_enc_wait_venc_done(struct venc_h264_handle *handle)
+{
+ unsigned int irq_status = 0;
+ struct mtk_vcodec_ctx *pctx = handle->ctx;
+
+ mtk_vcodec_debug_enter(handle);
+ mtk_vcodec_wait_for_done_ctx(pctx, MTK_INST_IRQ_RECEIVED, 1000, true);
+ irq_status = pctx->irq_status;
+ mtk_vcodec_debug(handle, "irq_status %x <-", irq_status);
+
+ return irq_status;
+}
+
+static int h264_encode_sps(struct venc_h264_handle *handle,
+ struct mtk_vcodec_mem *bs_buf,
+ unsigned int *bs_size)
+{
+ unsigned int irq_status;
+
+ mtk_vcodec_debug_enter(handle);
+
+ if (h264_enc_vpu_encode(handle, H264_BS_MODE_SPS, NULL,
+ bs_buf, bs_size)) {
+ mtk_vcodec_err(handle, "h264_enc_vpu_encode sps failed");
+ return -EINVAL;
+ }
+
+ irq_status = h264_enc_wait_venc_done(handle);
+ if (irq_status != H264_IRQ_STATUS_ENC_SPS_INT) {
+ mtk_vcodec_err(handle, "expect irq status %d",
+ H264_IRQ_STATUS_ENC_SPS_INT);
+ return -EINVAL;
+ }
+
+ *bs_size = h264_read_reg(handle, VENC_PIC_BITSTREAM_BYTE_CNT);
+ mtk_vcodec_debug(handle, "bs size %d <-", *bs_size);
+
+ return 0;
+}
+
+static int h264_encode_pps(struct venc_h264_handle *handle,
+ struct mtk_vcodec_mem *bs_buf,
+ unsigned int *bs_size)
+{
+ unsigned int irq_status;
+
+ mtk_vcodec_debug_enter(handle);
+
+ if (h264_enc_vpu_encode(handle, H264_BS_MODE_PPS, NULL,
+ bs_buf, bs_size)) {
+ mtk_vcodec_err(handle, "h264_enc_vpu_encode pps failed");
+ return -EINVAL;
+ }
+
+ irq_status = h264_enc_wait_venc_done(handle);
+ if (irq_status != H264_IRQ_STATUS_ENC_PPS_INT) {
+ mtk_vcodec_err(handle, "expect irq status %d",
+ H264_IRQ_STATUS_ENC_PPS_INT);
+ return -EINVAL;
+ }
+
+ *bs_size = h264_read_reg(handle, VENC_PIC_BITSTREAM_BYTE_CNT);
+ mtk_vcodec_debug(handle, "bs size %d <-", *bs_size);
+
+ return 0;
+}
+
+static int h264_encode_frame(struct venc_h264_handle *handle,
+ struct venc_frm_buf *frm_buf,
+ struct mtk_vcodec_mem *bs_buf,
+ unsigned int *bs_size)
+{
+ unsigned int irq_status;
+
+ mtk_vcodec_debug_enter(handle);
+
+ if (h264_enc_vpu_encode(handle, H264_BS_MODE_FRAME, frm_buf,
+ bs_buf, bs_size)) {
+ mtk_vcodec_err(handle, "h264_enc_vpu_encode frame failed");
+ return -EINVAL;
+ }
+
+ /*
+ * skip frame case: The skip frame buffer is composed by vpu side only,
+ * it does not trigger the hw, so skip the wait interrupt operation.
+ */
+ if (!handle->vpu_inst.wait_int) {
+ ++handle->frm_cnt;
+ return 0;
+ }
+
+ irq_status = h264_enc_wait_venc_done(handle);
+ if (irq_status != H264_IRQ_STATUS_ENC_FRM_INT) {
+ mtk_vcodec_err(handle, "irq_status=%d failed", irq_status);
+ return -EINVAL;
+ }
+
+ *bs_size = h264_read_reg(handle,
+ VENC_PIC_BITSTREAM_BYTE_CNT);
+ ++handle->frm_cnt;
+ mtk_vcodec_debug(handle, "frm %d bs size %d key_frm %d <-",
+ handle->frm_cnt,
+ *bs_size, handle->is_key_frm);
+
+ return 0;
+}
+
+static void h264_encode_filler(struct venc_h264_handle *handle, void *buf, int size)
+{
+ unsigned char *p = buf;
+
+ *p++ = 0x0;
+ *p++ = 0x0;
+ *p++ = 0x0;
+ *p++ = 0x1;
+ *p++ = 0xc;
+ size -= 5;
+ while (size) {
+ *p++ = 0xff;
+ size -= 1;
+ }
+}
+
+int h264_enc_init(struct mtk_vcodec_ctx *ctx, unsigned long *handle)
+{
+ struct venc_h264_handle *h;
+
+ h = kzalloc(sizeof(*h), GFP_KERNEL);
+ if (!h)
+ return -ENOMEM;
+
+ h->ctx = ctx;
+ h->dev = mtk_vcodec_get_plat_dev(ctx);
+ h->hw_base = mtk_vcodec_get_reg_addr(h->ctx, VENC_SYS);
+
+ if (h264_enc_vpu_init(h)) {
+ mtk_vcodec_err(h, "h264_enc_init failed");
+ return -EINVAL;
+ }
+
+ (*handle) = (unsigned long)h;
+
+ return 0;
+}
+
+int h264_enc_encode(unsigned long handle,
+ enum venc_start_opt opt,
+ struct venc_frm_buf *frm_buf,
+ struct mtk_vcodec_mem *bs_buf,
+ struct venc_done_result *result)
+{
+ int ret = 0;
+ struct venc_h264_handle *h = (struct venc_h264_handle *)handle;
+
+ mtk_vcodec_debug(h, "opt %d ->", opt);
+
+ switch (opt) {
+ case VENC_START_OPT_ENCODE_SEQUENCE_HEADER: {
+ unsigned int bs_size_sps;
+ unsigned int bs_size_pps;
+
+ memset(bs_buf->va, 0x38, 20);
+ if (h264_encode_sps(h, bs_buf, &bs_size_sps)) {
+ mtk_vcodec_err(h, "h264_encode_sps failed");
+ ret = -EINVAL;
+ goto encode_err;
+ }
+ memset(h->pps_buf.va, 0x49, 20);
+
+ if (h264_encode_pps(h, &h->pps_buf, &bs_size_pps)) {
+ mtk_vcodec_err(h, "h264_encode_pps failed");
+ ret = -EINVAL;
+ goto encode_err;
+ }
+
+ memcpy(bs_buf->va + bs_size_sps,
+ h->pps_buf.va,
+ bs_size_pps);
+ result->bs_size = bs_size_sps + bs_size_pps;
+ result->is_key_frm = false;
+ }
+ break;
+
+ case VENC_START_OPT_ENCODE_FRAME:
+ if (h->prepend_hdr) {
+ int hdr_sz;
+ int hdr_sz_ext;
+ int bs_alignment = 128;
+ int filler_sz = 0;
+ struct mtk_vcodec_mem tmp_bs_buf;
+ unsigned int bs_size_sps;
+ unsigned int bs_size_pps;
+ unsigned int bs_size_frm;
+
+ mtk_vcodec_debug(h,
+ "h264_encode_frame prepend SPS/PPS");
+ if (h264_encode_sps(h, bs_buf, &bs_size_sps)) {
+ mtk_vcodec_err(h,
+ "h264_encode_sps failed");
+ ret = -EINVAL;
+ goto encode_err;
+ }
+
+ if (h264_encode_pps(h, &h->pps_buf, &bs_size_pps)) {
+ mtk_vcodec_err(h,
+ "h264_encode_pps failed");
+ ret = -EINVAL;
+ goto encode_err;
+ }
+ memcpy(bs_buf->va + bs_size_sps,
+ h->pps_buf.va,
+ bs_size_pps);
+
+ hdr_sz = bs_size_sps + bs_size_pps;
+ hdr_sz_ext = (hdr_sz & (bs_alignment - 1));
+ if (hdr_sz_ext) {
+ filler_sz = bs_alignment - hdr_sz_ext;
+ if (hdr_sz_ext + 5 > bs_alignment)
+ filler_sz += bs_alignment;
+ h264_encode_filler(
+ h, bs_buf->va + hdr_sz,
+ filler_sz);
+ }
+
+ tmp_bs_buf.va = bs_buf->va + hdr_sz +
+ filler_sz;
+ tmp_bs_buf.dma_addr = bs_buf->dma_addr + hdr_sz +
+ filler_sz;
+ tmp_bs_buf.size = bs_buf->size -
+ (hdr_sz + filler_sz);
+
+ if (h264_encode_frame(h, frm_buf,
+ &tmp_bs_buf,
+ &bs_size_frm)) {
+ mtk_vcodec_err(h,
+ "h264_encode_frame failed");
+ ret = -EINVAL;
+ goto encode_err;
+ }
+
+ result->bs_size = hdr_sz + filler_sz + bs_size_frm;
+ mtk_vcodec_debug(h,
+ "hdr %d filler %d frame %d bs %d",
+ hdr_sz, filler_sz, bs_size_frm,
+ result->bs_size);
+
+ h->prepend_hdr = 0;
+ } else {
+ if (h264_encode_frame(h, frm_buf, bs_buf,
+ &result->bs_size)) {
+ mtk_vcodec_err(h,
+ "h264_encode_frame failed");
+ ret = -EINVAL;
+ goto encode_err;
+ }
+ }
+ result->is_key_frm = h->is_key_frm;
+ break;
+
+ default:
+ mtk_vcodec_err(h, "venc_start_opt %d not supported",
+ opt);
+ ret = -EINVAL;
+ break;
+ }
+
+encode_err:
+ if (ret)
+ result->msg = VENC_MESSAGE_ERR;
+ else
+ result->msg = VENC_MESSAGE_OK;
+
+ mtk_vcodec_debug(h, "opt %d <-", opt);
+ return ret;
+}
+
+int h264_enc_set_param(unsigned long handle,
+ enum venc_set_param_type type, void *in)
+{
+ int ret = 0;
+ struct venc_h264_handle *h = (struct venc_h264_handle *)handle;
+ struct venc_enc_prm *enc_prm;
+
+ mtk_vcodec_debug(h, "->type=%d", type);
+
+ switch (type) {
+ case VENC_SET_PARAM_ENC:
+ enc_prm = in;
+ if (h264_enc_vpu_set_param(h,
+ VENC_SET_PARAM_ENC,
+ enc_prm)) {
+ ret = -EINVAL;
+ break;
+ }
+ if (h->work_buf_alloc == 0) {
+ if (h264_enc_alloc_work_buf(h)) {
+ mtk_vcodec_err(h,
+ "h264_enc_alloc_work_buf failed");
+ ret = -ENOMEM;
+ } else {
+ h->work_buf_alloc = 1;
+ }
+ }
+ break;
+
+ case VENC_SET_PARAM_FORCE_INTRA:
+ if (h264_enc_vpu_set_param(h,
+ VENC_SET_PARAM_FORCE_INTRA, 0)) {
+ mtk_vcodec_err(h, "force intra failed");
+ ret = -EINVAL;
+ }
+ break;
+
+ case VENC_SET_PARAM_ADJUST_BITRATE:
+ enc_prm = in;
+ if (h264_enc_vpu_set_param(h, VENC_SET_PARAM_ADJUST_BITRATE,
+ &enc_prm->bitrate)) {
+ mtk_vcodec_err(h, "adjust bitrate failed");
+ ret = -EINVAL;
+ }
+ break;
+
+ case VENC_SET_PARAM_ADJUST_FRAMERATE:
+ enc_prm = in;
+ if (h264_enc_vpu_set_param(h, VENC_SET_PARAM_ADJUST_FRAMERATE,
+ &enc_prm->frm_rate)) {
+ mtk_vcodec_err(h, "adjust frame rate failed");
+ ret = -EINVAL;
+ }
+ break;
+
+ case VENC_SET_PARAM_I_FRAME_INTERVAL:
+ if (h264_enc_vpu_set_param(h,
+ VENC_SET_PARAM_I_FRAME_INTERVAL,
+ in)) {
+ mtk_vcodec_err(h, "set I frame interval failed");
+ ret = -EINVAL;
+ }
+ break;
+
+ case VENC_SET_PARAM_SKIP_FRAME:
+ if (h264_enc_vpu_set_param(h, VENC_SET_PARAM_SKIP_FRAME, 0)) {
+ mtk_vcodec_err(h, "skip frame failed");
+ ret = -EINVAL;
+ }
+ break;
+
+ case VENC_SET_PARAM_PREPEND_HEADER:
+ h->prepend_hdr = 1;
+ mtk_vcodec_debug(h, "set prepend header mode");
+ break;
+
+ default:
+ mtk_vcodec_err(h, "type %d not supported", type);
+ ret = -EINVAL;
+ break;
+ }
+
+ mtk_vcodec_debug_leave(h);
+ return ret;
+}
+
+int h264_enc_deinit(unsigned long handle)
+{
+ int ret = 0;
+ struct venc_h264_handle *h = (struct venc_h264_handle *)handle;
+
+ mtk_vcodec_debug_enter(h);
+
+ if (h264_enc_vpu_deinit(h)) {
+ mtk_vcodec_err(h, "h264_enc_vpu_deinit failed");
+ ret = -EINVAL;
+ }
+
+ if (h->work_buf_alloc)
+ h264_enc_free_work_buf(h);
+
+ mtk_vcodec_debug_leave(h);
+ kfree(h);
+
+ return ret;
+}
+
+struct venc_common_if venc_h264_if = {
+ h264_enc_init,
+ h264_enc_encode,
+ h264_enc_set_param,
+ h264_enc_deinit,
+};
+
+struct venc_common_if *get_h264_enc_comm_if(void)
+{
+ return &venc_h264_if;
+}
diff --git a/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.h b/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.h
new file mode 100644
index 0000000..4deb7d4
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.h
@@ -0,0 +1,53 @@
+/*
+ * Copyright (c) 2015 MediaTek Inc.
+ * Author: Jungchang Tsao <jungchang.tsao@mediatek.com>
+ * Daniel Hsiao <daniel.hsiao@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef _VENC_H264_IF_H_
+#define _VENC_H264_IF_H_
+
+#include "venc_drv_base.h"
+#include "venc_h264_vpu.h"
+
+/*
+ * struct venc_h264_handle - h264 encoder AP driver handle
+ * @hw_base: h264 encoder hardware register base
+ * @work_bufs: working buffer
+ * @pps_buf: buffer to store the pps bitstream
+ * @work_buf_alloc: working buffer allocated flag
+ * @frm_cnt: encoded frame count
+ * @prepend_hdr: when the v4l2 layer send VENC_SET_PARAM_PREPEND_HEADER cmd
+ * through h264_enc_set_param interface, it will set this flag and prepend the
+ * sps/pps in h264_enc_encode function.
+ * @is_key_frm: key frame flag
+ * @vpu_inst: VPU instance to exchange information between AP and VPU
+ * @ctx: context for v4l2 layer integration
+ * @dev: device for v4l2 layer integration
+ */
+struct venc_h264_handle {
+ void __iomem *hw_base;
+ struct mtk_vcodec_mem work_bufs[VENC_H264_VPU_WORK_BUF_MAX];
+ struct mtk_vcodec_mem pps_buf;
+ unsigned int work_buf_alloc;
+ unsigned int frm_cnt;
+ unsigned int prepend_hdr;
+ unsigned int is_key_frm;
+ struct venc_h264_vpu_inst vpu_inst;
+ void *ctx;
+ struct platform_device *dev;
+};
+
+struct venc_common_if *get_h264_enc_comm_if(void);
+
+#endif
diff --git a/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.c b/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.c
new file mode 100644
index 0000000..5715971
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.c
@@ -0,0 +1,341 @@
+/*
+ * Copyright (c) 2015 MediaTek Inc.
+ * Author: Jungchang Tsao <jungchang.tsao@mediatek.com>
+ * Daniel Hsiao <daniel.hsiao@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include "mtk_vpu_core.h"
+
+#include "venc_h264_if.h"
+#include "venc_h264_vpu.h"
+#include "venc_ipi_msg.h"
+
+static unsigned int h264_get_profile(unsigned int profile)
+{
+ /* (Baseline=66, Main=77, High=100) */
+ switch (profile) {
+ case V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE:
+ return 66;
+ case V4L2_MPEG_VIDEO_H264_PROFILE_MAIN:
+ return 77;
+ case V4L2_MPEG_VIDEO_H264_PROFILE_HIGH:
+ return 100;
+ default:
+ return 100;
+ }
+}
+
+static unsigned int h264_get_level(unsigned int level)
+{
+ /* (UpTo4.1(HighProfile)) */
+ switch (level) {
+ case V4L2_MPEG_VIDEO_H264_LEVEL_1_0:
+ return 10;
+ case V4L2_MPEG_VIDEO_H264_LEVEL_1_1:
+ return 11;
+ case V4L2_MPEG_VIDEO_H264_LEVEL_1_2:
+ return 12;
+ case V4L2_MPEG_VIDEO_H264_LEVEL_1_3:
+ return 13;
+ case V4L2_MPEG_VIDEO_H264_LEVEL_2_0:
+ return 20;
+ case V4L2_MPEG_VIDEO_H264_LEVEL_2_1:
+ return 21;
+ case V4L2_MPEG_VIDEO_H264_LEVEL_2_2:
+ return 22;
+ case V4L2_MPEG_VIDEO_H264_LEVEL_3_0:
+ return 30;
+ case V4L2_MPEG_VIDEO_H264_LEVEL_3_1:
+ return 31;
+ case V4L2_MPEG_VIDEO_H264_LEVEL_3_2:
+ return 32;
+ case V4L2_MPEG_VIDEO_H264_LEVEL_4_0:
+ return 40;
+ case V4L2_MPEG_VIDEO_H264_LEVEL_4_1:
+ return 41;
+ default:
+ return 31;
+ }
+}
+
+static void handle_h264_enc_init_msg(struct venc_h264_handle *hndl, void *data)
+{
+ struct venc_vpu_ipi_msg_init *msg = data;
+
+ hndl->vpu_inst.id = msg->inst_id;
+ hndl->vpu_inst.drv = (struct venc_h264_vpu_drv *)vpu_mapping_dm_addr(
+ hndl->dev, (uintptr_t *)(unsigned long)msg->inst_id);
+}
+
+static void handle_h264_enc_encode_msg(struct venc_h264_handle *hndl,
+ void *data)
+{
+ struct venc_vpu_ipi_msg_enc *msg = data;
+
+ hndl->vpu_inst.state = msg->state;
+ hndl->vpu_inst.bs_size = msg->bs_size;
+ hndl->is_key_frm = msg->key_frame;
+}
+
+static void h264_enc_vpu_ipi_handler(void *data, unsigned int len, void *priv)
+{
+ struct venc_vpu_ipi_msg_common *msg = data;
+ struct venc_h264_handle *hndl = (struct venc_h264_handle *)msg->venc_inst;
+
+ mtk_vcodec_debug_enter(hndl);
+
+ mtk_vcodec_debug(hndl, "msg_id %x hndl %p status %d",
+ msg->msg_id, hndl, msg->status);
+
+ switch (msg->msg_id) {
+ case VPU_IPIMSG_H264_ENC_INIT_DONE:
+ handle_h264_enc_init_msg(hndl, data);
+ break;
+ case VPU_IPIMSG_H264_ENC_SET_PARAM_DONE:
+ break;
+ case VPU_IPIMSG_H264_ENC_ENCODE_DONE:
+ handle_h264_enc_encode_msg(hndl, data);
+ break;
+ case VPU_IPIMSG_H264_ENC_DEINIT_DONE:
+ break;
+ default:
+ mtk_vcodec_err(hndl, "unknown msg id %x", msg->msg_id);
+ break;
+ }
+
+ hndl->vpu_inst.signaled = 1;
+ hndl->vpu_inst.failure = (msg->status != VENC_IPI_MSG_STATUS_OK);
+ wake_up_interruptible(&hndl->vpu_inst.wq_hd);
+
+ mtk_vcodec_debug_leave(hndl);
+}
+
+static int h264_enc_vpu_wait_ack(struct venc_h264_handle *hndl,
+ unsigned int timeout_ms)
+{
+ int ret;
+
+ mtk_vcodec_debug_enter(hndl);
+
+ ret = wait_event_interruptible_timeout(hndl->vpu_inst.wq_hd,
+ hndl->vpu_inst.signaled == 1,
+ msecs_to_jiffies(timeout_ms));
+ if (0 == ret) {
+ mtk_vcodec_err(hndl, "wait vpu ack time out !");
+ return -EINVAL;
+ }
+ if (-ERESTARTSYS == ret) {
+ mtk_vcodec_err(hndl,
+ "wait vpu ack interrupted by a signal");
+ return -EINVAL;
+ }
+
+ hndl->vpu_inst.signaled = 0;
+
+ mtk_vcodec_debug_leave(hndl);
+ return 0;
+}
+
+static int h264_enc_vpu_send_msg(struct venc_h264_handle *hndl, void *msg,
+ int len, int wait_ack)
+{
+ int status;
+
+ mtk_vcodec_debug_enter(hndl);
+
+ status = vpu_ipi_send(hndl->dev, IPI_VENC_H264, msg, len, 1);
+ if (status) {
+ mtk_vcodec_err(hndl, "vpu_ipi_send msg %x len %d fail %d",
+ *(unsigned int *)msg, len, status);
+ return -EINVAL;
+ }
+ mtk_vcodec_debug(hndl, "vpu_ipi_send msg %x success",
+ *(unsigned int *)msg);
+
+ if (wait_ack && h264_enc_vpu_wait_ack(hndl, 2000)) {
+ mtk_vcodec_err(hndl, "h264_enc_vpu_wait_ack failed");
+ return -EINVAL;
+ }
+
+ mtk_vcodec_debug_leave(hndl);
+ return 0;
+}
+
+int h264_enc_vpu_init(void *handle)
+{
+ int status;
+ struct venc_h264_handle *hndl = handle;
+ struct venc_ap_ipi_msg_init out;
+
+ mtk_vcodec_debug_enter(hndl);
+
+ init_waitqueue_head(&hndl->vpu_inst.wq_hd);
+ hndl->vpu_inst.signaled = 0;
+ hndl->vpu_inst.failure = 0;
+
+ status = vpu_ipi_register(hndl->dev, IPI_VENC_H264,
+ h264_enc_vpu_ipi_handler,
+ "h264_enc", NULL);
+ if (status) {
+ mtk_vcodec_err(hndl, "vpu_ipi_register fail %d", status);
+ return -EINVAL;
+ }
+ mtk_vcodec_debug(hndl, "vpu_ipi_register success");
+
+ out.msg_id = AP_IPIMSG_H264_ENC_INIT;
+ out.venc_inst = (unsigned long)hndl;
+ if (h264_enc_vpu_send_msg(hndl, &out, sizeof(out), 1) ||
+ hndl->vpu_inst.failure) {
+ mtk_vcodec_err(hndl, "AP_IPIMSG_H264_ENC_INIT failed");
+ return -EINVAL;
+ }
+
+ mtk_vcodec_debug_leave(hndl);
+
+ return 0;
+}
+
+int h264_enc_vpu_set_param(void *handle, unsigned int id, void *param)
+{
+ struct venc_h264_handle *hndl = handle;
+ struct venc_ap_ipi_msg_set_param out;
+
+ mtk_vcodec_debug(hndl, "id %d ->", id);
+
+ out.msg_id = AP_IPIMSG_H264_ENC_SET_PARAM;
+ out.inst_id = hndl->vpu_inst.id;
+ out.param_id = id;
+ switch (id) {
+ case VENC_SET_PARAM_ENC: {
+ struct venc_enc_prm *enc_param = param;
+
+ hndl->vpu_inst.drv->config.input_fourcc = enc_param->input_fourcc;
+ hndl->vpu_inst.drv->config.bitrate = enc_param->bitrate;
+ hndl->vpu_inst.drv->config.pic_w = enc_param->width;
+ hndl->vpu_inst.drv->config.pic_h = enc_param->height;
+ hndl->vpu_inst.drv->config.buf_w = enc_param->buf_width;
+ hndl->vpu_inst.drv->config.buf_h = enc_param->buf_height;
+ hndl->vpu_inst.drv->config.intra_period =
+ enc_param->intra_period;
+ hndl->vpu_inst.drv->config.framerate = enc_param->frm_rate;
+ hndl->vpu_inst.drv->config.profile =
+ h264_get_profile(enc_param->h264_profile);
+ hndl->vpu_inst.drv->config.level =
+ h264_get_level(enc_param->h264_level);
+ hndl->vpu_inst.drv->config.wfd = 0;
+ out.data_item = 0;
+ break;
+ }
+ case VENC_SET_PARAM_FORCE_INTRA:
+ out.data_item = 0;
+ break;
+ case VENC_SET_PARAM_ADJUST_BITRATE:
+ out.data_item = 1;
+ out.data[0] = *(unsigned int *)param;
+ break;
+ case VENC_SET_PARAM_ADJUST_FRAMERATE:
+ out.data_item = 1;
+ out.data[0] = *(unsigned int *)param;
+ break;
+ case VENC_SET_PARAM_I_FRAME_INTERVAL:
+ out.data_item = 1;
+ out.data[0] = *(unsigned int *)param;
+ break;
+ case VENC_SET_PARAM_SKIP_FRAME:
+ out.data_item = 0;
+ break;
+ }
+ if (h264_enc_vpu_send_msg(hndl, &out, sizeof(out), 1) ||
+ hndl->vpu_inst.failure) {
+ mtk_vcodec_err(hndl,
+ "AP_IPIMSG_H264_ENC_SET_PARAM %d fail", id);
+ return -EINVAL;
+ }
+
+ mtk_vcodec_debug(hndl, "id %d <-", id);
+
+ return 0;
+}
+
+int h264_enc_vpu_encode(void *handle, unsigned int bs_mode,
+ struct venc_frm_buf *frm_buf,
+ struct mtk_vcodec_mem *bs_buf,
+ unsigned int *bs_size)
+{
+ struct venc_h264_handle *hndl = handle;
+ struct venc_ap_ipi_msg_enc out;
+
+ mtk_vcodec_debug(hndl, "bs_mode %d ->", bs_mode);
+
+ out.msg_id = AP_IPIMSG_H264_ENC_ENCODE;
+ out.inst_id = hndl->vpu_inst.id;
+ out.bs_mode = bs_mode;
+ if (frm_buf) {
+ out.input_addr[0] = frm_buf->fb_addr.dma_addr;
+ out.input_addr[1] = frm_buf->fb_addr1.dma_addr;
+ out.input_addr[2] = frm_buf->fb_addr2.dma_addr;
+ } else {
+ out.input_addr[0] = 0;
+ out.input_addr[1] = 0;
+ out.input_addr[2] = 0;
+ }
+ if (bs_buf) {
+ out.bs_addr = bs_buf->dma_addr;
+ out.bs_size = bs_buf->size;
+ } else {
+ out.bs_addr = 0;
+ out.bs_size = 0;
+ }
+ if (h264_enc_vpu_send_msg(hndl, &out, sizeof(out), 1) ||
+ hndl->vpu_inst.failure) {
+ mtk_vcodec_err(hndl, "AP_IPIMSG_H264_ENC_ENCODE %d fail",
+ bs_mode);
+ return -EINVAL;
+ }
+
+ mtk_vcodec_debug(hndl, "state %d size %d key_frm %d",
+ hndl->vpu_inst.state, hndl->vpu_inst.bs_size,
+ hndl->is_key_frm);
+ hndl->vpu_inst.wait_int = 1;
+ if (hndl->vpu_inst.state == VEN_IPI_MSG_ENC_STATE_SKIP) {
+ *bs_size = hndl->vpu_inst.bs_size;
+ memcpy(bs_buf->va,
+ hndl->work_bufs[VENC_H264_VPU_WORK_BUF_SKIP_FRAME].va,
+ *bs_size);
+ hndl->vpu_inst.wait_int = 0;
+ }
+
+ mtk_vcodec_debug(hndl, "bs_mode %d ->", bs_mode);
+
+ return 0;
+}
+
+int h264_enc_vpu_deinit(void *handle)
+{
+ struct venc_h264_handle *hndl = handle;
+ struct venc_ap_ipi_msg_deinit out;
+
+ mtk_vcodec_debug_enter(hndl);
+
+ out.msg_id = AP_IPIMSG_H264_ENC_DEINIT;
+ out.inst_id = hndl->vpu_inst.id;
+ if (h264_enc_vpu_send_msg(hndl, &out, sizeof(out), 1) ||
+ hndl->vpu_inst.failure) {
+ mtk_vcodec_err(hndl, "AP_IPIMSG_H264_ENC_DEINIT fail");
+ return -EINVAL;
+ }
+
+ mtk_vcodec_debug_leave(hndl);
+
+ return 0;
+}
--
1.7.9.5
^ permalink raw reply related [flat|nested] 84+ messages in thread
* [RESEND RFC/PATCH 8/8] media: platform: mtk-vcodec: Add Mediatek H264 Video Encoder Driver
@ 2015-11-17 12:54 ` Tiffany Lin
0 siblings, 0 replies; 84+ messages in thread
From: Tiffany Lin @ 2015-11-17 12:54 UTC (permalink / raw)
To: Rob Herring, Pawel Moll, Mark Rutland, Ian Campbell, Kumar Gala,
Catalin Marinas, Will Deacon, Mauro Carvalho Chehab,
Matthias Brugger, Daniel Kurtz, Sascha Hauer, Hongzhou Yang,
Hans Verkuil, Laurent Pinchart, Sakari Ailus, Geert Uytterhoeven,
Mikhail Ulyanov, Fabien Dessenne, Arnd Bergmann,
Darren Etheridge, Peter Griffin, Benoit Parrot
Cc: Tiffany Lin, Andrew-CT Chen, Eddie Huang, Yingjoe Chen,
James Liao, Daniel Hsiao, devicetree, linux-kernel,
linux-arm-kernel, linux-media, linux-mediatek
Signed-off-by: Daniel Hsiao <daniel.hsiao@mediatek.com>
---
drivers/media/platform/mtk-vcodec/Makefile | 2 +-
drivers/media/platform/mtk-vcodec/common/Makefile | 4 +-
.../media/platform/mtk-vcodec/common/venc_drv_if.c | 3 +
.../media/platform/mtk-vcodec/h264_enc/Makefile | 9 +
.../platform/mtk-vcodec/h264_enc/venc_h264_if.c | 529 ++++++++++++++++++++
.../platform/mtk-vcodec/h264_enc/venc_h264_if.h | 53 ++
.../platform/mtk-vcodec/h264_enc/venc_h264_vpu.c | 341 +++++++++++++
7 files changed, 939 insertions(+), 2 deletions(-)
create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/Makefile
create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.c
create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.h
create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.c
diff --git a/drivers/media/platform/mtk-vcodec/Makefile b/drivers/media/platform/mtk-vcodec/Makefile
index b881a8b..d2189f7 100644
--- a/drivers/media/platform/mtk-vcodec/Makefile
+++ b/drivers/media/platform/mtk-vcodec/Makefile
@@ -4,7 +4,7 @@ obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += mtk_vcodec_intr.o \
mtk_vcodec_enc.o \
mtk_vcodec_enc_pm.o
-obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += common/ vp8_enc/
+obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += common/ vp8_enc/ h264_enc/
ccflags-y += -I$(srctree)/drivers/media/platform/mtk-vcodec/include \
-I$(srctree)/drivers/media/platform/mtk-vcodec \
diff --git a/drivers/media/platform/mtk-vcodec/common/Makefile b/drivers/media/platform/mtk-vcodec/common/Makefile
index 71ae856..b33d48d 100644
--- a/drivers/media/platform/mtk-vcodec/common/Makefile
+++ b/drivers/media/platform/mtk-vcodec/common/Makefile
@@ -6,5 +6,7 @@ ccflags-y += \
-I$(srctree)/drivers/media/platform/mtk-vcodec \
-I$(srctree)/drivers/media/platform/mtk-vcodec/include \
-I$(srctree)/drivers/media/platform/mtk-vcodec/vp8_enc \
+ -I$(srctree)/drivers/media/platform/mtk-vcodec/h264_enc \
-I$(srctree)/drivers/media/platform/mtk-vpu \
- -I$(srctree)/drivers/media/platform/mtk-vpu/vp8_enc
+ -I$(srctree)/drivers/media/platform/mtk-vpu/vp8_enc \
+ -I$(srctree)/drivers/media/platform/mtk-vpu/h264_enc
diff --git a/drivers/media/platform/mtk-vcodec/common/venc_drv_if.c b/drivers/media/platform/mtk-vcodec/common/venc_drv_if.c
index e9be186..930254b 100644
--- a/drivers/media/platform/mtk-vcodec/common/venc_drv_if.c
+++ b/drivers/media/platform/mtk-vcodec/common/venc_drv_if.c
@@ -27,6 +27,7 @@
#include "venc_drv_if.h"
#include "venc_drv_base.h"
#include "venc_vp8_if.h"
+#include "venc_h264_if.h"
int venc_if_create(void *ctx, unsigned int fourcc, unsigned long *handle)
{
@@ -48,6 +49,8 @@ int venc_if_create(void *ctx, unsigned int fourcc, unsigned long *handle)
h->enc_if = get_vp8_enc_comm_if();
break;
case V4L2_PIX_FMT_H264:
+ h->enc_if = get_h264_enc_comm_if();
+ break;
default:
mtk_vcodec_err(h, "invalid format %s", str);
goto err_out;
diff --git a/drivers/media/platform/mtk-vcodec/h264_enc/Makefile b/drivers/media/platform/mtk-vcodec/h264_enc/Makefile
new file mode 100644
index 0000000..6559908
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/h264_enc/Makefile
@@ -0,0 +1,9 @@
+obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += venc_h264_if.o venc_h264_vpu.o
+
+ccflags-y += \
+ -I$(srctree)/include/ \
+ -I$(srctree)/drivers/media/platform/mtk-vcodec/ \
+ -I$(srctree)/drivers/media/platform/mtk-vcodec/include \
+ -I$(srctree)/drivers/media/platform/mtk-vcodec/h264_enc \
+ -I$(srctree)/drivers/media/platform/mtk-vpu \
+ -I$(srctree)/drivers/media/platform/mtk-vpu/h264_enc
diff --git a/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.c b/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.c
new file mode 100644
index 0000000..c880865
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.c
@@ -0,0 +1,529 @@
+/*
+ * Copyright (c) 2015 MediaTek Inc.
+ * Author: Jungchang Tsao <jungchang.tsao@mediatek.com>
+ * Daniel Hsiao <daniel.hsiao@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/kernel.h>
+#include <linux/slab.h>
+#include <linux/interrupt.h>
+
+#include "mtk_vcodec_drv.h"
+#include "mtk_vcodec_util.h"
+#include "mtk_vcodec_intr.h"
+#include "mtk_vcodec_enc.h"
+#include "mtk_vcodec_pm.h"
+#include "mtk_vpu_core.h"
+
+#include "venc_h264_if.h"
+#include "venc_h264_vpu.h"
+
+#define h264_write_reg(h, addr, val) writel(val, h->hw_base + addr)
+#define h264_read_reg(h, addr) readl(h->hw_base + addr)
+
+#define VENC_PIC_BITSTREAM_BYTE_CNT 0x0098
+
+enum venc_h264_irq_status {
+ H264_IRQ_STATUS_ENC_SPS_INT = (1 << 0),
+ H264_IRQ_STATUS_ENC_PPS_INT = (1 << 1),
+ H264_IRQ_STATUS_ENC_FRM_INT = (1 << 2),
+};
+
+static int h264_enc_alloc_work_buf(struct venc_h264_handle *handle)
+{
+ int i, j;
+ int ret = 0;
+ struct venc_h264_vpu_buf *wb = handle->vpu_inst.drv->work_bufs;
+
+ mtk_vcodec_debug_enter(handle);
+
+ for (i = 0; i < VENC_H264_VPU_WORK_BUF_MAX; i++) {
+ /*
+ * This 'wb' structure is set by VPU side and shared to AP for
+ * buffer allocation and physical addr mapping. For most of
+ * the buffers, AP will allocate the buffer according to 'size'
+ * field and store the physical addr in 'pa' field. There are two
+ * exceptions:
+ * (1) RC_CODE buffer, it's pre-allocated in the VPU side, and
+ * save the VPU addr in the 'vpua' field. The AP will translate
+ * the VPU addr to the corresponding physical addr and store
+ * in 'pa' field for reg setting in VPU side.
+ * (2) SKIP_FRAME buffer, it's pre-allocated in the VPU side, and
+ * save the VPU addr in the 'vpua' field. The AP will translate
+ * the VPU addr to the corresponding AP side virtual address and
+ * do some memcpy access to move to bitstream buffer assigned
+ * by v4l2 layer.
+ */
+ if (i == VENC_H264_VPU_WORK_BUF_RC_CODE) {
+ handle->work_bufs[i].size = wb[i].size;
+ handle->work_bufs[i].va = vpu_mapping_dm_addr(
+ handle->dev, (uintptr_t *)(unsigned long)
+ wb[i].vpua);
+ handle->work_bufs[i].dma_addr =
+ (dma_addr_t)vpu_mapping_iommu_dm_addr(
+ handle->dev, (uintptr_t *)(unsigned long)
+ wb[i].vpua);
+ wb[i].pa = handle->work_bufs[i].dma_addr;
+ } else if (i == VENC_H264_VPU_WORK_BUF_SKIP_FRAME) {
+ handle->work_bufs[i].size = wb[i].size;
+ handle->work_bufs[i].va = vpu_mapping_dm_addr(
+ handle->dev, (uintptr_t *)(unsigned long)
+ wb[i].vpua);
+ handle->work_bufs[i].dma_addr = 0;
+ wb[i].pa = handle->work_bufs[i].dma_addr;
+ } else {
+ handle->work_bufs[i].size = wb[i].size;
+ if (mtk_vcodec_mem_alloc(handle->ctx,
+ &handle->work_bufs[i])) {
+ mtk_vcodec_err(handle, "cannot allocate buf %d", i);
+ ret = -ENOMEM;
+ goto err_alloc;
+ }
+ wb[i].pa = handle->work_bufs[i].dma_addr;
+ }
+ mtk_vcodec_debug(handle, "buf[%d] va=0x%p pa=0x%p size=0x%lx", i,
+ handle->work_bufs[i].va,
+ (void *)handle->work_bufs[i].dma_addr,
+ handle->work_bufs[i].size);
+ }
+
+ /* the pps_buf is used by AP side only */
+ handle->pps_buf.size = 128;
+ if (mtk_vcodec_mem_alloc(handle->ctx,
+ &handle->pps_buf)) {
+ mtk_vcodec_err(handle, "cannot allocate pps_buf");
+ ret = -ENOMEM;
+ goto err_alloc;
+ }
+ mtk_vcodec_debug_leave(handle);
+
+ return ret;
+
+err_alloc:
+ for (j = 0; j < i; j++) {
+ if ((j != VENC_H264_VPU_WORK_BUF_RC_CODE) &&
+ (j != VENC_H264_VPU_WORK_BUF_SKIP_FRAME))
+ mtk_vcodec_mem_free(handle->ctx, &handle->work_bufs[j]);
+ }
+
+ return ret;
+}
+
+static void h264_enc_free_work_buf(struct venc_h264_handle *handle)
+{
+ int i;
+
+ mtk_vcodec_debug_enter(handle);
+ for (i = 0; i < VENC_H264_VPU_WORK_BUF_MAX; i++) {
+ if ((i != VENC_H264_VPU_WORK_BUF_RC_CODE) &&
+ (i != VENC_H264_VPU_WORK_BUF_SKIP_FRAME))
+ mtk_vcodec_mem_free(handle->ctx, &handle->work_bufs[i]);
+ }
+ mtk_vcodec_mem_free(handle->ctx, &handle->pps_buf);
+ mtk_vcodec_debug_leave(handle);
+}
+
+static unsigned int h264_enc_wait_venc_done(struct venc_h264_handle *handle)
+{
+ unsigned int irq_status = 0;
+ struct mtk_vcodec_ctx *pctx = handle->ctx;
+
+ mtk_vcodec_debug_enter(handle);
+ mtk_vcodec_wait_for_done_ctx(pctx, MTK_INST_IRQ_RECEIVED, 1000, true);
+ irq_status = pctx->irq_status;
+ mtk_vcodec_debug(handle, "irq_status %x <-", irq_status);
+
+ return irq_status;
+}
+
+static int h264_encode_sps(struct venc_h264_handle *handle,
+ struct mtk_vcodec_mem *bs_buf,
+ unsigned int *bs_size)
+{
+ unsigned int irq_status;
+
+ mtk_vcodec_debug_enter(handle);
+
+ if (h264_enc_vpu_encode(handle, H264_BS_MODE_SPS, NULL,
+ bs_buf, bs_size)) {
+ mtk_vcodec_err(handle, "h264_enc_vpu_encode sps failed");
+ return -EINVAL;
+ }
+
+ irq_status = h264_enc_wait_venc_done(handle);
+ if (irq_status != H264_IRQ_STATUS_ENC_SPS_INT) {
+ mtk_vcodec_err(handle, "expect irq status %d",
+ H264_IRQ_STATUS_ENC_SPS_INT);
+ return -EINVAL;
+ }
+
+ *bs_size = h264_read_reg(handle, VENC_PIC_BITSTREAM_BYTE_CNT);
+ mtk_vcodec_debug(handle, "bs size %d <-", *bs_size);
+
+ return 0;
+}
+
+static int h264_encode_pps(struct venc_h264_handle *handle,
+ struct mtk_vcodec_mem *bs_buf,
+ unsigned int *bs_size)
+{
+ unsigned int irq_status;
+
+ mtk_vcodec_debug_enter(handle);
+
+ if (h264_enc_vpu_encode(handle, H264_BS_MODE_PPS, NULL,
+ bs_buf, bs_size)) {
+ mtk_vcodec_err(handle, "h264_enc_vpu_encode pps failed");
+ return -EINVAL;
+ }
+
+ irq_status = h264_enc_wait_venc_done(handle);
+ if (irq_status != H264_IRQ_STATUS_ENC_PPS_INT) {
+ mtk_vcodec_err(handle, "expect irq status %d",
+ H264_IRQ_STATUS_ENC_PPS_INT);
+ return -EINVAL;
+ }
+
+ *bs_size = h264_read_reg(handle, VENC_PIC_BITSTREAM_BYTE_CNT);
+ mtk_vcodec_debug(handle, "bs size %d <-", *bs_size);
+
+ return 0;
+}
+
+static int h264_encode_frame(struct venc_h264_handle *handle,
+ struct venc_frm_buf *frm_buf,
+ struct mtk_vcodec_mem *bs_buf,
+ unsigned int *bs_size)
+{
+ unsigned int irq_status;
+
+ mtk_vcodec_debug_enter(handle);
+
+ if (h264_enc_vpu_encode(handle, H264_BS_MODE_FRAME, frm_buf,
+ bs_buf, bs_size)) {
+ mtk_vcodec_err(handle, "h264_enc_vpu_encode frame failed");
+ return -EINVAL;
+ }
+
+ /*
+ * skip frame case: The skip frame buffer is composed by vpu side only,
+ * it does not trigger the hw, so skip the wait interrupt operation.
+ */
+ if (!handle->vpu_inst.wait_int) {
+ ++handle->frm_cnt;
+ return 0;
+ }
+
+ irq_status = h264_enc_wait_venc_done(handle);
+ if (irq_status != H264_IRQ_STATUS_ENC_FRM_INT) {
+ mtk_vcodec_err(handle, "irq_status=%d failed", irq_status);
+ return -EINVAL;
+ }
+
+ *bs_size = h264_read_reg(handle,
+ VENC_PIC_BITSTREAM_BYTE_CNT);
+ ++handle->frm_cnt;
+ mtk_vcodec_debug(handle, "frm %d bs size %d key_frm %d <-",
+ handle->frm_cnt,
+ *bs_size, handle->is_key_frm);
+
+ return 0;
+}
+
+static void h264_encode_filler(struct venc_h264_handle *handle, void *buf, int size)
+{
+ unsigned char *p = buf;
+
+ *p++ = 0x0;
+ *p++ = 0x0;
+ *p++ = 0x0;
+ *p++ = 0x1;
+ *p++ = 0xc;
+ size -= 5;
+ while (size) {
+ *p++ = 0xff;
+ size -= 1;
+ }
+}
+
+int h264_enc_init(struct mtk_vcodec_ctx *ctx, unsigned long *handle)
+{
+ struct venc_h264_handle *h;
+
+ h = kzalloc(sizeof(*h), GFP_KERNEL);
+ if (!h)
+ return -ENOMEM;
+
+ h->ctx = ctx;
+ h->dev = mtk_vcodec_get_plat_dev(ctx);
+ h->hw_base = mtk_vcodec_get_reg_addr(h->ctx, VENC_SYS);
+
+ if (h264_enc_vpu_init(h)) {
+ mtk_vcodec_err(h, "h264_enc_init failed");
+ return -EINVAL;
+ }
+
+ (*handle) = (unsigned long)h;
+
+ return 0;
+}
+
+int h264_enc_encode(unsigned long handle,
+ enum venc_start_opt opt,
+ struct venc_frm_buf *frm_buf,
+ struct mtk_vcodec_mem *bs_buf,
+ struct venc_done_result *result)
+{
+ int ret = 0;
+ struct venc_h264_handle *h = (struct venc_h264_handle *)handle;
+
+ mtk_vcodec_debug(h, "opt %d ->", opt);
+
+ switch (opt) {
+ case VENC_START_OPT_ENCODE_SEQUENCE_HEADER: {
+ unsigned int bs_size_sps;
+ unsigned int bs_size_pps;
+
+ memset(bs_buf->va, 0x38, 20);
+ if (h264_encode_sps(h, bs_buf, &bs_size_sps)) {
+ mtk_vcodec_err(h, "h264_encode_sps failed");
+ ret = -EINVAL;
+ goto encode_err;
+ }
+ memset(h->pps_buf.va, 0x49, 20);
+
+ if (h264_encode_pps(h, &h->pps_buf, &bs_size_pps)) {
+ mtk_vcodec_err(h, "h264_encode_pps failed");
+ ret = -EINVAL;
+ goto encode_err;
+ }
+
+ memcpy(bs_buf->va + bs_size_sps,
+ h->pps_buf.va,
+ bs_size_pps);
+ result->bs_size = bs_size_sps + bs_size_pps;
+ result->is_key_frm = false;
+ }
+ break;
+
+ case VENC_START_OPT_ENCODE_FRAME:
+ if (h->prepend_hdr) {
+ int hdr_sz;
+ int hdr_sz_ext;
+ int bs_alignment = 128;
+ int filler_sz = 0;
+ struct mtk_vcodec_mem tmp_bs_buf;
+ unsigned int bs_size_sps;
+ unsigned int bs_size_pps;
+ unsigned int bs_size_frm;
+
+ mtk_vcodec_debug(h,
+ "h264_encode_frame prepend SPS/PPS");
+ if (h264_encode_sps(h, bs_buf, &bs_size_sps)) {
+ mtk_vcodec_err(h,
+ "h264_encode_sps failed");
+ ret = -EINVAL;
+ goto encode_err;
+ }
+
+ if (h264_encode_pps(h, &h->pps_buf, &bs_size_pps)) {
+ mtk_vcodec_err(h,
+ "h264_encode_pps failed");
+ ret = -EINVAL;
+ goto encode_err;
+ }
+ memcpy(bs_buf->va + bs_size_sps,
+ h->pps_buf.va,
+ bs_size_pps);
+
+ hdr_sz = bs_size_sps + bs_size_pps;
+ hdr_sz_ext = (hdr_sz & (bs_alignment - 1));
+ if (hdr_sz_ext) {
+ filler_sz = bs_alignment - hdr_sz_ext;
+ if (hdr_sz_ext + 5 > bs_alignment)
+ filler_sz += bs_alignment;
+ h264_encode_filler(
+ h, bs_buf->va + hdr_sz,
+ filler_sz);
+ }
+
+ tmp_bs_buf.va = bs_buf->va + hdr_sz +
+ filler_sz;
+ tmp_bs_buf.dma_addr = bs_buf->dma_addr + hdr_sz +
+ filler_sz;
+ tmp_bs_buf.size = bs_buf->size -
+ (hdr_sz + filler_sz);
+
+ if (h264_encode_frame(h, frm_buf,
+ &tmp_bs_buf,
+ &bs_size_frm)) {
+ mtk_vcodec_err(h,
+ "h264_encode_frame failed");
+ ret = -EINVAL;
+ goto encode_err;
+ }
+
+ result->bs_size = hdr_sz + filler_sz + bs_size_frm;
+ mtk_vcodec_debug(h,
+ "hdr %d filler %d frame %d bs %d",
+ hdr_sz, filler_sz, bs_size_frm,
+ result->bs_size);
+
+ h->prepend_hdr = 0;
+ } else {
+ if (h264_encode_frame(h, frm_buf, bs_buf,
+ &result->bs_size)) {
+ mtk_vcodec_err(h,
+ "h264_encode_frame failed");
+ ret = -EINVAL;
+ goto encode_err;
+ }
+ }
+ result->is_key_frm = h->is_key_frm;
+ break;
+
+ default:
+ mtk_vcodec_err(h, "venc_start_opt %d not supported",
+ opt);
+ ret = -EINVAL;
+ break;
+ }
+
+encode_err:
+ if (ret)
+ result->msg = VENC_MESSAGE_ERR;
+ else
+ result->msg = VENC_MESSAGE_OK;
+
+ mtk_vcodec_debug(h, "opt %d <-", opt);
+ return ret;
+}
+
+int h264_enc_set_param(unsigned long handle,
+ enum venc_set_param_type type, void *in)
+{
+ int ret = 0;
+ struct venc_h264_handle *h = (struct venc_h264_handle *)handle;
+ struct venc_enc_prm *enc_prm;
+
+ mtk_vcodec_debug(h, "->type=%d", type);
+
+ switch (type) {
+ case VENC_SET_PARAM_ENC:
+ enc_prm = in;
+ if (h264_enc_vpu_set_param(h,
+ VENC_SET_PARAM_ENC,
+ enc_prm)) {
+ ret = -EINVAL;
+ break;
+ }
+ if (h->work_buf_alloc == 0) {
+ if (h264_enc_alloc_work_buf(h)) {
+ mtk_vcodec_err(h,
+ "h264_enc_alloc_work_buf failed");
+ ret = -ENOMEM;
+ } else {
+ h->work_buf_alloc = 1;
+ }
+ }
+ break;
+
+ case VENC_SET_PARAM_FORCE_INTRA:
+ if (h264_enc_vpu_set_param(h,
+ VENC_SET_PARAM_FORCE_INTRA, 0)) {
+ mtk_vcodec_err(h, "force intra failed");
+ ret = -EINVAL;
+ }
+ break;
+
+ case VENC_SET_PARAM_ADJUST_BITRATE:
+ enc_prm = in;
+ if (h264_enc_vpu_set_param(h, VENC_SET_PARAM_ADJUST_BITRATE,
+ &enc_prm->bitrate)) {
+ mtk_vcodec_err(h, "adjust bitrate failed");
+ ret = -EINVAL;
+ }
+ break;
+
+ case VENC_SET_PARAM_ADJUST_FRAMERATE:
+ enc_prm = in;
+ if (h264_enc_vpu_set_param(h, VENC_SET_PARAM_ADJUST_FRAMERATE,
+ &enc_prm->frm_rate)) {
+ mtk_vcodec_err(h, "adjust frame rate failed");
+ ret = -EINVAL;
+ }
+ break;
+
+ case VENC_SET_PARAM_I_FRAME_INTERVAL:
+ if (h264_enc_vpu_set_param(h,
+ VENC_SET_PARAM_I_FRAME_INTERVAL,
+ in)) {
+ mtk_vcodec_err(h, "set I frame interval failed");
+ ret = -EINVAL;
+ }
+ break;
+
+ case VENC_SET_PARAM_SKIP_FRAME:
+ if (h264_enc_vpu_set_param(h, VENC_SET_PARAM_SKIP_FRAME, 0)) {
+ mtk_vcodec_err(h, "skip frame failed");
+ ret = -EINVAL;
+ }
+ break;
+
+ case VENC_SET_PARAM_PREPEND_HEADER:
+ h->prepend_hdr = 1;
+ mtk_vcodec_debug(h, "set prepend header mode");
+ break;
+
+ default:
+ mtk_vcodec_err(h, "type %d not supported", type);
+ ret = -EINVAL;
+ break;
+ }
+
+ mtk_vcodec_debug_leave(h);
+ return ret;
+}
+
+int h264_enc_deinit(unsigned long handle)
+{
+ int ret = 0;
+ struct venc_h264_handle *h = (struct venc_h264_handle *)handle;
+
+ mtk_vcodec_debug_enter(h);
+
+ if (h264_enc_vpu_deinit(h)) {
+ mtk_vcodec_err(h, "h264_enc_vpu_deinit failed");
+ ret = -EINVAL;
+ }
+
+ if (h->work_buf_alloc)
+ h264_enc_free_work_buf(h);
+
+ mtk_vcodec_debug_leave(h);
+ kfree(h);
+
+ return ret;
+}
+
+struct venc_common_if venc_h264_if = {
+ h264_enc_init,
+ h264_enc_encode,
+ h264_enc_set_param,
+ h264_enc_deinit,
+};
+
+struct venc_common_if *get_h264_enc_comm_if(void)
+{
+ return &venc_h264_if;
+}
diff --git a/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.h b/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.h
new file mode 100644
index 0000000..4deb7d4
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.h
@@ -0,0 +1,53 @@
+/*
+ * Copyright (c) 2015 MediaTek Inc.
+ * Author: Jungchang Tsao <jungchang.tsao@mediatek.com>
+ * Daniel Hsiao <daniel.hsiao@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef _VENC_H264_IF_H_
+#define _VENC_H264_IF_H_
+
+#include "venc_drv_base.h"
+#include "venc_h264_vpu.h"
+
+/*
+ * struct venc_h264_handle - h264 encoder AP driver handle
+ * @hw_base: h264 encoder hardware register base
+ * @work_bufs: working buffer
+ * @pps_buf: buffer to store the pps bitstream
+ * @work_buf_alloc: working buffer allocated flag
+ * @frm_cnt: encoded frame count
+ * @prepend_hdr: when the v4l2 layer send VENC_SET_PARAM_PREPEND_HEADER cmd
+ * through h264_enc_set_param interface, it will set this flag and prepend the
+ * sps/pps in h264_enc_encode function.
+ * @is_key_frm: key frame flag
+ * @vpu_inst: VPU instance to exchange information between AP and VPU
+ * @ctx: context for v4l2 layer integration
+ * @dev: device for v4l2 layer integration
+ */
+struct venc_h264_handle {
+ void __iomem *hw_base;
+ struct mtk_vcodec_mem work_bufs[VENC_H264_VPU_WORK_BUF_MAX];
+ struct mtk_vcodec_mem pps_buf;
+ unsigned int work_buf_alloc;
+ unsigned int frm_cnt;
+ unsigned int prepend_hdr;
+ unsigned int is_key_frm;
+ struct venc_h264_vpu_inst vpu_inst;
+ void *ctx;
+ struct platform_device *dev;
+};
+
+struct venc_common_if *get_h264_enc_comm_if(void);
+
+#endif
diff --git a/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.c b/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.c
new file mode 100644
index 0000000..5715971
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.c
@@ -0,0 +1,341 @@
+/*
+ * Copyright (c) 2015 MediaTek Inc.
+ * Author: Jungchang Tsao <jungchang.tsao@mediatek.com>
+ * Daniel Hsiao <daniel.hsiao@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include "mtk_vpu_core.h"
+
+#include "venc_h264_if.h"
+#include "venc_h264_vpu.h"
+#include "venc_ipi_msg.h"
+
+static unsigned int h264_get_profile(unsigned int profile)
+{
+ /* (Baseline=66, Main=77, High=100) */
+ switch (profile) {
+ case V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE:
+ return 66;
+ case V4L2_MPEG_VIDEO_H264_PROFILE_MAIN:
+ return 77;
+ case V4L2_MPEG_VIDEO_H264_PROFILE_HIGH:
+ return 100;
+ default:
+ return 100;
+ }
+}
+
+static unsigned int h264_get_level(unsigned int level)
+{
+ /* (UpTo4.1(HighProfile)) */
+ switch (level) {
+ case V4L2_MPEG_VIDEO_H264_LEVEL_1_0:
+ return 10;
+ case V4L2_MPEG_VIDEO_H264_LEVEL_1_1:
+ return 11;
+ case V4L2_MPEG_VIDEO_H264_LEVEL_1_2:
+ return 12;
+ case V4L2_MPEG_VIDEO_H264_LEVEL_1_3:
+ return 13;
+ case V4L2_MPEG_VIDEO_H264_LEVEL_2_0:
+ return 20;
+ case V4L2_MPEG_VIDEO_H264_LEVEL_2_1:
+ return 21;
+ case V4L2_MPEG_VIDEO_H264_LEVEL_2_2:
+ return 22;
+ case V4L2_MPEG_VIDEO_H264_LEVEL_3_0:
+ return 30;
+ case V4L2_MPEG_VIDEO_H264_LEVEL_3_1:
+ return 31;
+ case V4L2_MPEG_VIDEO_H264_LEVEL_3_2:
+ return 32;
+ case V4L2_MPEG_VIDEO_H264_LEVEL_4_0:
+ return 40;
+ case V4L2_MPEG_VIDEO_H264_LEVEL_4_1:
+ return 41;
+ default:
+ return 31;
+ }
+}
+
+static void handle_h264_enc_init_msg(struct venc_h264_handle *hndl, void *data)
+{
+ struct venc_vpu_ipi_msg_init *msg = data;
+
+ hndl->vpu_inst.id = msg->inst_id;
+ hndl->vpu_inst.drv = (struct venc_h264_vpu_drv *)vpu_mapping_dm_addr(
+ hndl->dev, (uintptr_t *)(unsigned long)msg->inst_id);
+}
+
+static void handle_h264_enc_encode_msg(struct venc_h264_handle *hndl,
+ void *data)
+{
+ struct venc_vpu_ipi_msg_enc *msg = data;
+
+ hndl->vpu_inst.state = msg->state;
+ hndl->vpu_inst.bs_size = msg->bs_size;
+ hndl->is_key_frm = msg->key_frame;
+}
+
+static void h264_enc_vpu_ipi_handler(void *data, unsigned int len, void *priv)
+{
+ struct venc_vpu_ipi_msg_common *msg = data;
+ struct venc_h264_handle *hndl = (struct venc_h264_handle *)msg->venc_inst;
+
+ mtk_vcodec_debug_enter(hndl);
+
+ mtk_vcodec_debug(hndl, "msg_id %x hndl %p status %d",
+ msg->msg_id, hndl, msg->status);
+
+ switch (msg->msg_id) {
+ case VPU_IPIMSG_H264_ENC_INIT_DONE:
+ handle_h264_enc_init_msg(hndl, data);
+ break;
+ case VPU_IPIMSG_H264_ENC_SET_PARAM_DONE:
+ break;
+ case VPU_IPIMSG_H264_ENC_ENCODE_DONE:
+ handle_h264_enc_encode_msg(hndl, data);
+ break;
+ case VPU_IPIMSG_H264_ENC_DEINIT_DONE:
+ break;
+ default:
+ mtk_vcodec_err(hndl, "unknown msg id %x", msg->msg_id);
+ break;
+ }
+
+ hndl->vpu_inst.signaled = 1;
+ hndl->vpu_inst.failure = (msg->status != VENC_IPI_MSG_STATUS_OK);
+ wake_up_interruptible(&hndl->vpu_inst.wq_hd);
+
+ mtk_vcodec_debug_leave(hndl);
+}
+
+static int h264_enc_vpu_wait_ack(struct venc_h264_handle *hndl,
+ unsigned int timeout_ms)
+{
+ int ret;
+
+ mtk_vcodec_debug_enter(hndl);
+
+ ret = wait_event_interruptible_timeout(hndl->vpu_inst.wq_hd,
+ hndl->vpu_inst.signaled == 1,
+ msecs_to_jiffies(timeout_ms));
+ if (0 == ret) {
+ mtk_vcodec_err(hndl, "wait vpu ack time out !");
+ return -EINVAL;
+ }
+ if (-ERESTARTSYS == ret) {
+ mtk_vcodec_err(hndl,
+ "wait vpu ack interrupted by a signal");
+ return -EINVAL;
+ }
+
+ hndl->vpu_inst.signaled = 0;
+
+ mtk_vcodec_debug_leave(hndl);
+ return 0;
+}
+
+static int h264_enc_vpu_send_msg(struct venc_h264_handle *hndl, void *msg,
+ int len, int wait_ack)
+{
+ int status;
+
+ mtk_vcodec_debug_enter(hndl);
+
+ status = vpu_ipi_send(hndl->dev, IPI_VENC_H264, msg, len, 1);
+ if (status) {
+ mtk_vcodec_err(hndl, "vpu_ipi_send msg %x len %d fail %d",
+ *(unsigned int *)msg, len, status);
+ return -EINVAL;
+ }
+ mtk_vcodec_debug(hndl, "vpu_ipi_send msg %x success",
+ *(unsigned int *)msg);
+
+ if (wait_ack && h264_enc_vpu_wait_ack(hndl, 2000)) {
+ mtk_vcodec_err(hndl, "h264_enc_vpu_wait_ack failed");
+ return -EINVAL;
+ }
+
+ mtk_vcodec_debug_leave(hndl);
+ return 0;
+}
+
+int h264_enc_vpu_init(void *handle)
+{
+ int status;
+ struct venc_h264_handle *hndl = handle;
+ struct venc_ap_ipi_msg_init out;
+
+ mtk_vcodec_debug_enter(hndl);
+
+ init_waitqueue_head(&hndl->vpu_inst.wq_hd);
+ hndl->vpu_inst.signaled = 0;
+ hndl->vpu_inst.failure = 0;
+
+ status = vpu_ipi_register(hndl->dev, IPI_VENC_H264,
+ h264_enc_vpu_ipi_handler,
+ "h264_enc", NULL);
+ if (status) {
+ mtk_vcodec_err(hndl, "vpu_ipi_register fail %d", status);
+ return -EINVAL;
+ }
+ mtk_vcodec_debug(hndl, "vpu_ipi_register success");
+
+ out.msg_id = AP_IPIMSG_H264_ENC_INIT;
+ out.venc_inst = (unsigned long)hndl;
+ if (h264_enc_vpu_send_msg(hndl, &out, sizeof(out), 1) ||
+ hndl->vpu_inst.failure) {
+ mtk_vcodec_err(hndl, "AP_IPIMSG_H264_ENC_INIT failed");
+ return -EINVAL;
+ }
+
+ mtk_vcodec_debug_leave(hndl);
+
+ return 0;
+}
+
+int h264_enc_vpu_set_param(void *handle, unsigned int id, void *param)
+{
+ struct venc_h264_handle *hndl = handle;
+ struct venc_ap_ipi_msg_set_param out;
+
+ mtk_vcodec_debug(hndl, "id %d ->", id);
+
+ out.msg_id = AP_IPIMSG_H264_ENC_SET_PARAM;
+ out.inst_id = hndl->vpu_inst.id;
+ out.param_id = id;
+ switch (id) {
+ case VENC_SET_PARAM_ENC: {
+ struct venc_enc_prm *enc_param = param;
+
+ hndl->vpu_inst.drv->config.input_fourcc = enc_param->input_fourcc;
+ hndl->vpu_inst.drv->config.bitrate = enc_param->bitrate;
+ hndl->vpu_inst.drv->config.pic_w = enc_param->width;
+ hndl->vpu_inst.drv->config.pic_h = enc_param->height;
+ hndl->vpu_inst.drv->config.buf_w = enc_param->buf_width;
+ hndl->vpu_inst.drv->config.buf_h = enc_param->buf_height;
+ hndl->vpu_inst.drv->config.intra_period =
+ enc_param->intra_period;
+ hndl->vpu_inst.drv->config.framerate = enc_param->frm_rate;
+ hndl->vpu_inst.drv->config.profile =
+ h264_get_profile(enc_param->h264_profile);
+ hndl->vpu_inst.drv->config.level =
+ h264_get_level(enc_param->h264_level);
+ hndl->vpu_inst.drv->config.wfd = 0;
+ out.data_item = 0;
+ break;
+ }
+ case VENC_SET_PARAM_FORCE_INTRA:
+ out.data_item = 0;
+ break;
+ case VENC_SET_PARAM_ADJUST_BITRATE:
+ out.data_item = 1;
+ out.data[0] = *(unsigned int *)param;
+ break;
+ case VENC_SET_PARAM_ADJUST_FRAMERATE:
+ out.data_item = 1;
+ out.data[0] = *(unsigned int *)param;
+ break;
+ case VENC_SET_PARAM_I_FRAME_INTERVAL:
+ out.data_item = 1;
+ out.data[0] = *(unsigned int *)param;
+ break;
+ case VENC_SET_PARAM_SKIP_FRAME:
+ out.data_item = 0;
+ break;
+ }
+ if (h264_enc_vpu_send_msg(hndl, &out, sizeof(out), 1) ||
+ hndl->vpu_inst.failure) {
+ mtk_vcodec_err(hndl,
+ "AP_IPIMSG_H264_ENC_SET_PARAM %d fail", id);
+ return -EINVAL;
+ }
+
+ mtk_vcodec_debug(hndl, "id %d <-", id);
+
+ return 0;
+}
+
+int h264_enc_vpu_encode(void *handle, unsigned int bs_mode,
+ struct venc_frm_buf *frm_buf,
+ struct mtk_vcodec_mem *bs_buf,
+ unsigned int *bs_size)
+{
+ struct venc_h264_handle *hndl = handle;
+ struct venc_ap_ipi_msg_enc out;
+
+ mtk_vcodec_debug(hndl, "bs_mode %d ->", bs_mode);
+
+ out.msg_id = AP_IPIMSG_H264_ENC_ENCODE;
+ out.inst_id = hndl->vpu_inst.id;
+ out.bs_mode = bs_mode;
+ if (frm_buf) {
+ out.input_addr[0] = frm_buf->fb_addr.dma_addr;
+ out.input_addr[1] = frm_buf->fb_addr1.dma_addr;
+ out.input_addr[2] = frm_buf->fb_addr2.dma_addr;
+ } else {
+ out.input_addr[0] = 0;
+ out.input_addr[1] = 0;
+ out.input_addr[2] = 0;
+ }
+ if (bs_buf) {
+ out.bs_addr = bs_buf->dma_addr;
+ out.bs_size = bs_buf->size;
+ } else {
+ out.bs_addr = 0;
+ out.bs_size = 0;
+ }
+ if (h264_enc_vpu_send_msg(hndl, &out, sizeof(out), 1) ||
+ hndl->vpu_inst.failure) {
+ mtk_vcodec_err(hndl, "AP_IPIMSG_H264_ENC_ENCODE %d fail",
+ bs_mode);
+ return -EINVAL;
+ }
+
+ mtk_vcodec_debug(hndl, "state %d size %d key_frm %d",
+ hndl->vpu_inst.state, hndl->vpu_inst.bs_size,
+ hndl->is_key_frm);
+ hndl->vpu_inst.wait_int = 1;
+ if (hndl->vpu_inst.state == VEN_IPI_MSG_ENC_STATE_SKIP) {
+ *bs_size = hndl->vpu_inst.bs_size;
+ memcpy(bs_buf->va,
+ hndl->work_bufs[VENC_H264_VPU_WORK_BUF_SKIP_FRAME].va,
+ *bs_size);
+ hndl->vpu_inst.wait_int = 0;
+ }
+
+ mtk_vcodec_debug(hndl, "bs_mode %d ->", bs_mode);
+
+ return 0;
+}
+
+int h264_enc_vpu_deinit(void *handle)
+{
+ struct venc_h264_handle *hndl = handle;
+ struct venc_ap_ipi_msg_deinit out;
+
+ mtk_vcodec_debug_enter(hndl);
+
+ out.msg_id = AP_IPIMSG_H264_ENC_DEINIT;
+ out.inst_id = hndl->vpu_inst.id;
+ if (h264_enc_vpu_send_msg(hndl, &out, sizeof(out), 1) ||
+ hndl->vpu_inst.failure) {
+ mtk_vcodec_err(hndl, "AP_IPIMSG_H264_ENC_DEINIT fail");
+ return -EINVAL;
+ }
+
+ mtk_vcodec_debug_leave(hndl);
+
+ return 0;
+}
--
1.7.9.5
^ permalink raw reply related [flat|nested] 84+ messages in thread
* [RESEND RFC/PATCH 8/8] media: platform: mtk-vcodec: Add Mediatek H264 Video Encoder Driver
@ 2015-11-17 12:54 ` Tiffany Lin
0 siblings, 0 replies; 84+ messages in thread
From: Tiffany Lin @ 2015-11-17 12:54 UTC (permalink / raw)
To: linux-arm-kernel
Signed-off-by: Daniel Hsiao <daniel.hsiao@mediatek.com>
---
drivers/media/platform/mtk-vcodec/Makefile | 2 +-
drivers/media/platform/mtk-vcodec/common/Makefile | 4 +-
.../media/platform/mtk-vcodec/common/venc_drv_if.c | 3 +
.../media/platform/mtk-vcodec/h264_enc/Makefile | 9 +
.../platform/mtk-vcodec/h264_enc/venc_h264_if.c | 529 ++++++++++++++++++++
.../platform/mtk-vcodec/h264_enc/venc_h264_if.h | 53 ++
.../platform/mtk-vcodec/h264_enc/venc_h264_vpu.c | 341 +++++++++++++
7 files changed, 939 insertions(+), 2 deletions(-)
create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/Makefile
create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.c
create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.h
create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.c
diff --git a/drivers/media/platform/mtk-vcodec/Makefile b/drivers/media/platform/mtk-vcodec/Makefile
index b881a8b..d2189f7 100644
--- a/drivers/media/platform/mtk-vcodec/Makefile
+++ b/drivers/media/platform/mtk-vcodec/Makefile
@@ -4,7 +4,7 @@ obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += mtk_vcodec_intr.o \
mtk_vcodec_enc.o \
mtk_vcodec_enc_pm.o
-obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += common/ vp8_enc/
+obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += common/ vp8_enc/ h264_enc/
ccflags-y += -I$(srctree)/drivers/media/platform/mtk-vcodec/include \
-I$(srctree)/drivers/media/platform/mtk-vcodec \
diff --git a/drivers/media/platform/mtk-vcodec/common/Makefile b/drivers/media/platform/mtk-vcodec/common/Makefile
index 71ae856..b33d48d 100644
--- a/drivers/media/platform/mtk-vcodec/common/Makefile
+++ b/drivers/media/platform/mtk-vcodec/common/Makefile
@@ -6,5 +6,7 @@ ccflags-y += \
-I$(srctree)/drivers/media/platform/mtk-vcodec \
-I$(srctree)/drivers/media/platform/mtk-vcodec/include \
-I$(srctree)/drivers/media/platform/mtk-vcodec/vp8_enc \
+ -I$(srctree)/drivers/media/platform/mtk-vcodec/h264_enc \
-I$(srctree)/drivers/media/platform/mtk-vpu \
- -I$(srctree)/drivers/media/platform/mtk-vpu/vp8_enc
+ -I$(srctree)/drivers/media/platform/mtk-vpu/vp8_enc \
+ -I$(srctree)/drivers/media/platform/mtk-vpu/h264_enc
diff --git a/drivers/media/platform/mtk-vcodec/common/venc_drv_if.c b/drivers/media/platform/mtk-vcodec/common/venc_drv_if.c
index e9be186..930254b 100644
--- a/drivers/media/platform/mtk-vcodec/common/venc_drv_if.c
+++ b/drivers/media/platform/mtk-vcodec/common/venc_drv_if.c
@@ -27,6 +27,7 @@
#include "venc_drv_if.h"
#include "venc_drv_base.h"
#include "venc_vp8_if.h"
+#include "venc_h264_if.h"
int venc_if_create(void *ctx, unsigned int fourcc, unsigned long *handle)
{
@@ -48,6 +49,8 @@ int venc_if_create(void *ctx, unsigned int fourcc, unsigned long *handle)
h->enc_if = get_vp8_enc_comm_if();
break;
case V4L2_PIX_FMT_H264:
+ h->enc_if = get_h264_enc_comm_if();
+ break;
default:
mtk_vcodec_err(h, "invalid format %s", str);
goto err_out;
diff --git a/drivers/media/platform/mtk-vcodec/h264_enc/Makefile b/drivers/media/platform/mtk-vcodec/h264_enc/Makefile
new file mode 100644
index 0000000..6559908
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/h264_enc/Makefile
@@ -0,0 +1,9 @@
+obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += venc_h264_if.o venc_h264_vpu.o
+
+ccflags-y += \
+ -I$(srctree)/include/ \
+ -I$(srctree)/drivers/media/platform/mtk-vcodec/ \
+ -I$(srctree)/drivers/media/platform/mtk-vcodec/include \
+ -I$(srctree)/drivers/media/platform/mtk-vcodec/h264_enc \
+ -I$(srctree)/drivers/media/platform/mtk-vpu \
+ -I$(srctree)/drivers/media/platform/mtk-vpu/h264_enc
diff --git a/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.c b/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.c
new file mode 100644
index 0000000..c880865
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.c
@@ -0,0 +1,529 @@
+/*
+ * Copyright (c) 2015 MediaTek Inc.
+ * Author: Jungchang Tsao <jungchang.tsao@mediatek.com>
+ * Daniel Hsiao <daniel.hsiao@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/kernel.h>
+#include <linux/slab.h>
+#include <linux/interrupt.h>
+
+#include "mtk_vcodec_drv.h"
+#include "mtk_vcodec_util.h"
+#include "mtk_vcodec_intr.h"
+#include "mtk_vcodec_enc.h"
+#include "mtk_vcodec_pm.h"
+#include "mtk_vpu_core.h"
+
+#include "venc_h264_if.h"
+#include "venc_h264_vpu.h"
+
+#define h264_write_reg(h, addr, val) writel(val, h->hw_base + addr)
+#define h264_read_reg(h, addr) readl(h->hw_base + addr)
+
+#define VENC_PIC_BITSTREAM_BYTE_CNT 0x0098
+
+enum venc_h264_irq_status {
+ H264_IRQ_STATUS_ENC_SPS_INT = (1 << 0),
+ H264_IRQ_STATUS_ENC_PPS_INT = (1 << 1),
+ H264_IRQ_STATUS_ENC_FRM_INT = (1 << 2),
+};
+
+static int h264_enc_alloc_work_buf(struct venc_h264_handle *handle)
+{
+ int i, j;
+ int ret = 0;
+ struct venc_h264_vpu_buf *wb = handle->vpu_inst.drv->work_bufs;
+
+ mtk_vcodec_debug_enter(handle);
+
+ for (i = 0; i < VENC_H264_VPU_WORK_BUF_MAX; i++) {
+ /*
+ * This 'wb' structure is set by VPU side and shared to AP for
+ * buffer allocation and physical addr mapping. For most of
+ * the buffers, AP will allocate the buffer according to 'size'
+ * field and store the physical addr in 'pa' field. There are two
+ * exceptions:
+ * (1) RC_CODE buffer, it's pre-allocated in the VPU side, and
+ * save the VPU addr in the 'vpua' field. The AP will translate
+ * the VPU addr to the corresponding physical addr and store
+ * in 'pa' field for reg setting in VPU side.
+ * (2) SKIP_FRAME buffer, it's pre-allocated in the VPU side, and
+ * save the VPU addr in the 'vpua' field. The AP will translate
+ * the VPU addr to the corresponding AP side virtual address and
+ * do some memcpy access to move to bitstream buffer assigned
+ * by v4l2 layer.
+ */
+ if (i == VENC_H264_VPU_WORK_BUF_RC_CODE) {
+ handle->work_bufs[i].size = wb[i].size;
+ handle->work_bufs[i].va = vpu_mapping_dm_addr(
+ handle->dev, (uintptr_t *)(unsigned long)
+ wb[i].vpua);
+ handle->work_bufs[i].dma_addr =
+ (dma_addr_t)vpu_mapping_iommu_dm_addr(
+ handle->dev, (uintptr_t *)(unsigned long)
+ wb[i].vpua);
+ wb[i].pa = handle->work_bufs[i].dma_addr;
+ } else if (i == VENC_H264_VPU_WORK_BUF_SKIP_FRAME) {
+ handle->work_bufs[i].size = wb[i].size;
+ handle->work_bufs[i].va = vpu_mapping_dm_addr(
+ handle->dev, (uintptr_t *)(unsigned long)
+ wb[i].vpua);
+ handle->work_bufs[i].dma_addr = 0;
+ wb[i].pa = handle->work_bufs[i].dma_addr;
+ } else {
+ handle->work_bufs[i].size = wb[i].size;
+ if (mtk_vcodec_mem_alloc(handle->ctx,
+ &handle->work_bufs[i])) {
+ mtk_vcodec_err(handle, "cannot allocate buf %d", i);
+ ret = -ENOMEM;
+ goto err_alloc;
+ }
+ wb[i].pa = handle->work_bufs[i].dma_addr;
+ }
+ mtk_vcodec_debug(handle, "buf[%d] va=0x%p pa=0x%p size=0x%lx", i,
+ handle->work_bufs[i].va,
+ (void *)handle->work_bufs[i].dma_addr,
+ handle->work_bufs[i].size);
+ }
+
+ /* the pps_buf is used by AP side only */
+ handle->pps_buf.size = 128;
+ if (mtk_vcodec_mem_alloc(handle->ctx,
+ &handle->pps_buf)) {
+ mtk_vcodec_err(handle, "cannot allocate pps_buf");
+ ret = -ENOMEM;
+ goto err_alloc;
+ }
+ mtk_vcodec_debug_leave(handle);
+
+ return ret;
+
+err_alloc:
+ for (j = 0; j < i; j++) {
+ if ((j != VENC_H264_VPU_WORK_BUF_RC_CODE) &&
+ (j != VENC_H264_VPU_WORK_BUF_SKIP_FRAME))
+ mtk_vcodec_mem_free(handle->ctx, &handle->work_bufs[j]);
+ }
+
+ return ret;
+}
+
+static void h264_enc_free_work_buf(struct venc_h264_handle *handle)
+{
+ int i;
+
+ mtk_vcodec_debug_enter(handle);
+ for (i = 0; i < VENC_H264_VPU_WORK_BUF_MAX; i++) {
+ if ((i != VENC_H264_VPU_WORK_BUF_RC_CODE) &&
+ (i != VENC_H264_VPU_WORK_BUF_SKIP_FRAME))
+ mtk_vcodec_mem_free(handle->ctx, &handle->work_bufs[i]);
+ }
+ mtk_vcodec_mem_free(handle->ctx, &handle->pps_buf);
+ mtk_vcodec_debug_leave(handle);
+}
+
+static unsigned int h264_enc_wait_venc_done(struct venc_h264_handle *handle)
+{
+ unsigned int irq_status = 0;
+ struct mtk_vcodec_ctx *pctx = handle->ctx;
+
+ mtk_vcodec_debug_enter(handle);
+ mtk_vcodec_wait_for_done_ctx(pctx, MTK_INST_IRQ_RECEIVED, 1000, true);
+ irq_status = pctx->irq_status;
+ mtk_vcodec_debug(handle, "irq_status %x <-", irq_status);
+
+ return irq_status;
+}
+
+static int h264_encode_sps(struct venc_h264_handle *handle,
+ struct mtk_vcodec_mem *bs_buf,
+ unsigned int *bs_size)
+{
+ unsigned int irq_status;
+
+ mtk_vcodec_debug_enter(handle);
+
+ if (h264_enc_vpu_encode(handle, H264_BS_MODE_SPS, NULL,
+ bs_buf, bs_size)) {
+ mtk_vcodec_err(handle, "h264_enc_vpu_encode sps failed");
+ return -EINVAL;
+ }
+
+ irq_status = h264_enc_wait_venc_done(handle);
+ if (irq_status != H264_IRQ_STATUS_ENC_SPS_INT) {
+ mtk_vcodec_err(handle, "expect irq status %d",
+ H264_IRQ_STATUS_ENC_SPS_INT);
+ return -EINVAL;
+ }
+
+ *bs_size = h264_read_reg(handle, VENC_PIC_BITSTREAM_BYTE_CNT);
+ mtk_vcodec_debug(handle, "bs size %d <-", *bs_size);
+
+ return 0;
+}
+
+static int h264_encode_pps(struct venc_h264_handle *handle,
+ struct mtk_vcodec_mem *bs_buf,
+ unsigned int *bs_size)
+{
+ unsigned int irq_status;
+
+ mtk_vcodec_debug_enter(handle);
+
+ if (h264_enc_vpu_encode(handle, H264_BS_MODE_PPS, NULL,
+ bs_buf, bs_size)) {
+ mtk_vcodec_err(handle, "h264_enc_vpu_encode pps failed");
+ return -EINVAL;
+ }
+
+ irq_status = h264_enc_wait_venc_done(handle);
+ if (irq_status != H264_IRQ_STATUS_ENC_PPS_INT) {
+ mtk_vcodec_err(handle, "expect irq status %d",
+ H264_IRQ_STATUS_ENC_PPS_INT);
+ return -EINVAL;
+ }
+
+ *bs_size = h264_read_reg(handle, VENC_PIC_BITSTREAM_BYTE_CNT);
+ mtk_vcodec_debug(handle, "bs size %d <-", *bs_size);
+
+ return 0;
+}
+
+static int h264_encode_frame(struct venc_h264_handle *handle,
+ struct venc_frm_buf *frm_buf,
+ struct mtk_vcodec_mem *bs_buf,
+ unsigned int *bs_size)
+{
+ unsigned int irq_status;
+
+ mtk_vcodec_debug_enter(handle);
+
+ if (h264_enc_vpu_encode(handle, H264_BS_MODE_FRAME, frm_buf,
+ bs_buf, bs_size)) {
+ mtk_vcodec_err(handle, "h264_enc_vpu_encode frame failed");
+ return -EINVAL;
+ }
+
+ /*
+ * skip frame case: The skip frame buffer is composed by vpu side only,
+ * it does not trigger the hw, so skip the wait interrupt operation.
+ */
+ if (!handle->vpu_inst.wait_int) {
+ ++handle->frm_cnt;
+ return 0;
+ }
+
+ irq_status = h264_enc_wait_venc_done(handle);
+ if (irq_status != H264_IRQ_STATUS_ENC_FRM_INT) {
+ mtk_vcodec_err(handle, "irq_status=%d failed", irq_status);
+ return -EINVAL;
+ }
+
+ *bs_size = h264_read_reg(handle,
+ VENC_PIC_BITSTREAM_BYTE_CNT);
+ ++handle->frm_cnt;
+ mtk_vcodec_debug(handle, "frm %d bs size %d key_frm %d <-",
+ handle->frm_cnt,
+ *bs_size, handle->is_key_frm);
+
+ return 0;
+}
+
+static void h264_encode_filler(struct venc_h264_handle *handle, void *buf, int size)
+{
+ unsigned char *p = buf;
+
+ *p++ = 0x0;
+ *p++ = 0x0;
+ *p++ = 0x0;
+ *p++ = 0x1;
+ *p++ = 0xc;
+ size -= 5;
+ while (size) {
+ *p++ = 0xff;
+ size -= 1;
+ }
+}
+
+int h264_enc_init(struct mtk_vcodec_ctx *ctx, unsigned long *handle)
+{
+ struct venc_h264_handle *h;
+
+ h = kzalloc(sizeof(*h), GFP_KERNEL);
+ if (!h)
+ return -ENOMEM;
+
+ h->ctx = ctx;
+ h->dev = mtk_vcodec_get_plat_dev(ctx);
+ h->hw_base = mtk_vcodec_get_reg_addr(h->ctx, VENC_SYS);
+
+ if (h264_enc_vpu_init(h)) {
+ mtk_vcodec_err(h, "h264_enc_init failed");
+ return -EINVAL;
+ }
+
+ (*handle) = (unsigned long)h;
+
+ return 0;
+}
+
+int h264_enc_encode(unsigned long handle,
+ enum venc_start_opt opt,
+ struct venc_frm_buf *frm_buf,
+ struct mtk_vcodec_mem *bs_buf,
+ struct venc_done_result *result)
+{
+ int ret = 0;
+ struct venc_h264_handle *h = (struct venc_h264_handle *)handle;
+
+ mtk_vcodec_debug(h, "opt %d ->", opt);
+
+ switch (opt) {
+ case VENC_START_OPT_ENCODE_SEQUENCE_HEADER: {
+ unsigned int bs_size_sps;
+ unsigned int bs_size_pps;
+
+ memset(bs_buf->va, 0x38, 20);
+ if (h264_encode_sps(h, bs_buf, &bs_size_sps)) {
+ mtk_vcodec_err(h, "h264_encode_sps failed");
+ ret = -EINVAL;
+ goto encode_err;
+ }
+ memset(h->pps_buf.va, 0x49, 20);
+
+ if (h264_encode_pps(h, &h->pps_buf, &bs_size_pps)) {
+ mtk_vcodec_err(h, "h264_encode_pps failed");
+ ret = -EINVAL;
+ goto encode_err;
+ }
+
+ memcpy(bs_buf->va + bs_size_sps,
+ h->pps_buf.va,
+ bs_size_pps);
+ result->bs_size = bs_size_sps + bs_size_pps;
+ result->is_key_frm = false;
+ }
+ break;
+
+ case VENC_START_OPT_ENCODE_FRAME:
+ if (h->prepend_hdr) {
+ int hdr_sz;
+ int hdr_sz_ext;
+ int bs_alignment = 128;
+ int filler_sz = 0;
+ struct mtk_vcodec_mem tmp_bs_buf;
+ unsigned int bs_size_sps;
+ unsigned int bs_size_pps;
+ unsigned int bs_size_frm;
+
+ mtk_vcodec_debug(h,
+ "h264_encode_frame prepend SPS/PPS");
+ if (h264_encode_sps(h, bs_buf, &bs_size_sps)) {
+ mtk_vcodec_err(h,
+ "h264_encode_sps failed");
+ ret = -EINVAL;
+ goto encode_err;
+ }
+
+ if (h264_encode_pps(h, &h->pps_buf, &bs_size_pps)) {
+ mtk_vcodec_err(h,
+ "h264_encode_pps failed");
+ ret = -EINVAL;
+ goto encode_err;
+ }
+ memcpy(bs_buf->va + bs_size_sps,
+ h->pps_buf.va,
+ bs_size_pps);
+
+ hdr_sz = bs_size_sps + bs_size_pps;
+ hdr_sz_ext = (hdr_sz & (bs_alignment - 1));
+ if (hdr_sz_ext) {
+ filler_sz = bs_alignment - hdr_sz_ext;
+ if (hdr_sz_ext + 5 > bs_alignment)
+ filler_sz += bs_alignment;
+ h264_encode_filler(
+ h, bs_buf->va + hdr_sz,
+ filler_sz);
+ }
+
+ tmp_bs_buf.va = bs_buf->va + hdr_sz +
+ filler_sz;
+ tmp_bs_buf.dma_addr = bs_buf->dma_addr + hdr_sz +
+ filler_sz;
+ tmp_bs_buf.size = bs_buf->size -
+ (hdr_sz + filler_sz);
+
+ if (h264_encode_frame(h, frm_buf,
+ &tmp_bs_buf,
+ &bs_size_frm)) {
+ mtk_vcodec_err(h,
+ "h264_encode_frame failed");
+ ret = -EINVAL;
+ goto encode_err;
+ }
+
+ result->bs_size = hdr_sz + filler_sz + bs_size_frm;
+ mtk_vcodec_debug(h,
+ "hdr %d filler %d frame %d bs %d",
+ hdr_sz, filler_sz, bs_size_frm,
+ result->bs_size);
+
+ h->prepend_hdr = 0;
+ } else {
+ if (h264_encode_frame(h, frm_buf, bs_buf,
+ &result->bs_size)) {
+ mtk_vcodec_err(h,
+ "h264_encode_frame failed");
+ ret = -EINVAL;
+ goto encode_err;
+ }
+ }
+ result->is_key_frm = h->is_key_frm;
+ break;
+
+ default:
+ mtk_vcodec_err(h, "venc_start_opt %d not supported",
+ opt);
+ ret = -EINVAL;
+ break;
+ }
+
+encode_err:
+ if (ret)
+ result->msg = VENC_MESSAGE_ERR;
+ else
+ result->msg = VENC_MESSAGE_OK;
+
+ mtk_vcodec_debug(h, "opt %d <-", opt);
+ return ret;
+}
+
+int h264_enc_set_param(unsigned long handle,
+ enum venc_set_param_type type, void *in)
+{
+ int ret = 0;
+ struct venc_h264_handle *h = (struct venc_h264_handle *)handle;
+ struct venc_enc_prm *enc_prm;
+
+ mtk_vcodec_debug(h, "->type=%d", type);
+
+ switch (type) {
+ case VENC_SET_PARAM_ENC:
+ enc_prm = in;
+ if (h264_enc_vpu_set_param(h,
+ VENC_SET_PARAM_ENC,
+ enc_prm)) {
+ ret = -EINVAL;
+ break;
+ }
+ if (h->work_buf_alloc == 0) {
+ if (h264_enc_alloc_work_buf(h)) {
+ mtk_vcodec_err(h,
+ "h264_enc_alloc_work_buf failed");
+ ret = -ENOMEM;
+ } else {
+ h->work_buf_alloc = 1;
+ }
+ }
+ break;
+
+ case VENC_SET_PARAM_FORCE_INTRA:
+ if (h264_enc_vpu_set_param(h,
+ VENC_SET_PARAM_FORCE_INTRA, 0)) {
+ mtk_vcodec_err(h, "force intra failed");
+ ret = -EINVAL;
+ }
+ break;
+
+ case VENC_SET_PARAM_ADJUST_BITRATE:
+ enc_prm = in;
+ if (h264_enc_vpu_set_param(h, VENC_SET_PARAM_ADJUST_BITRATE,
+ &enc_prm->bitrate)) {
+ mtk_vcodec_err(h, "adjust bitrate failed");
+ ret = -EINVAL;
+ }
+ break;
+
+ case VENC_SET_PARAM_ADJUST_FRAMERATE:
+ enc_prm = in;
+ if (h264_enc_vpu_set_param(h, VENC_SET_PARAM_ADJUST_FRAMERATE,
+ &enc_prm->frm_rate)) {
+ mtk_vcodec_err(h, "adjust frame rate failed");
+ ret = -EINVAL;
+ }
+ break;
+
+ case VENC_SET_PARAM_I_FRAME_INTERVAL:
+ if (h264_enc_vpu_set_param(h,
+ VENC_SET_PARAM_I_FRAME_INTERVAL,
+ in)) {
+ mtk_vcodec_err(h, "set I frame interval failed");
+ ret = -EINVAL;
+ }
+ break;
+
+ case VENC_SET_PARAM_SKIP_FRAME:
+ if (h264_enc_vpu_set_param(h, VENC_SET_PARAM_SKIP_FRAME, 0)) {
+ mtk_vcodec_err(h, "skip frame failed");
+ ret = -EINVAL;
+ }
+ break;
+
+ case VENC_SET_PARAM_PREPEND_HEADER:
+ h->prepend_hdr = 1;
+ mtk_vcodec_debug(h, "set prepend header mode");
+ break;
+
+ default:
+ mtk_vcodec_err(h, "type %d not supported", type);
+ ret = -EINVAL;
+ break;
+ }
+
+ mtk_vcodec_debug_leave(h);
+ return ret;
+}
+
+int h264_enc_deinit(unsigned long handle)
+{
+ int ret = 0;
+ struct venc_h264_handle *h = (struct venc_h264_handle *)handle;
+
+ mtk_vcodec_debug_enter(h);
+
+ if (h264_enc_vpu_deinit(h)) {
+ mtk_vcodec_err(h, "h264_enc_vpu_deinit failed");
+ ret = -EINVAL;
+ }
+
+ if (h->work_buf_alloc)
+ h264_enc_free_work_buf(h);
+
+ mtk_vcodec_debug_leave(h);
+ kfree(h);
+
+ return ret;
+}
+
+struct venc_common_if venc_h264_if = {
+ h264_enc_init,
+ h264_enc_encode,
+ h264_enc_set_param,
+ h264_enc_deinit,
+};
+
+struct venc_common_if *get_h264_enc_comm_if(void)
+{
+ return &venc_h264_if;
+}
diff --git a/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.h b/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.h
new file mode 100644
index 0000000..4deb7d4
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.h
@@ -0,0 +1,53 @@
+/*
+ * Copyright (c) 2015 MediaTek Inc.
+ * Author: Jungchang Tsao <jungchang.tsao@mediatek.com>
+ * Daniel Hsiao <daniel.hsiao@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef _VENC_H264_IF_H_
+#define _VENC_H264_IF_H_
+
+#include "venc_drv_base.h"
+#include "venc_h264_vpu.h"
+
+/*
+ * struct venc_h264_handle - h264 encoder AP driver handle
+ * @hw_base: h264 encoder hardware register base
+ * @work_bufs: working buffer
+ * @pps_buf: buffer to store the pps bitstream
+ * @work_buf_alloc: working buffer allocated flag
+ * @frm_cnt: encoded frame count
+ * @prepend_hdr: when the v4l2 layer send VENC_SET_PARAM_PREPEND_HEADER cmd
+ * through h264_enc_set_param interface, it will set this flag and prepend the
+ * sps/pps in h264_enc_encode function.
+ * @is_key_frm: key frame flag
+ * @vpu_inst: VPU instance to exchange information between AP and VPU
+ * @ctx: context for v4l2 layer integration
+ * @dev: device for v4l2 layer integration
+ */
+struct venc_h264_handle {
+ void __iomem *hw_base;
+ struct mtk_vcodec_mem work_bufs[VENC_H264_VPU_WORK_BUF_MAX];
+ struct mtk_vcodec_mem pps_buf;
+ unsigned int work_buf_alloc;
+ unsigned int frm_cnt;
+ unsigned int prepend_hdr;
+ unsigned int is_key_frm;
+ struct venc_h264_vpu_inst vpu_inst;
+ void *ctx;
+ struct platform_device *dev;
+};
+
+struct venc_common_if *get_h264_enc_comm_if(void);
+
+#endif
diff --git a/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.c b/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.c
new file mode 100644
index 0000000..5715971
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.c
@@ -0,0 +1,341 @@
+/*
+ * Copyright (c) 2015 MediaTek Inc.
+ * Author: Jungchang Tsao <jungchang.tsao@mediatek.com>
+ * Daniel Hsiao <daniel.hsiao@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include "mtk_vpu_core.h"
+
+#include "venc_h264_if.h"
+#include "venc_h264_vpu.h"
+#include "venc_ipi_msg.h"
+
+static unsigned int h264_get_profile(unsigned int profile)
+{
+ /* (Baseline=66, Main=77, High=100) */
+ switch (profile) {
+ case V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE:
+ return 66;
+ case V4L2_MPEG_VIDEO_H264_PROFILE_MAIN:
+ return 77;
+ case V4L2_MPEG_VIDEO_H264_PROFILE_HIGH:
+ return 100;
+ default:
+ return 100;
+ }
+}
+
+static unsigned int h264_get_level(unsigned int level)
+{
+ /* (UpTo4.1(HighProfile)) */
+ switch (level) {
+ case V4L2_MPEG_VIDEO_H264_LEVEL_1_0:
+ return 10;
+ case V4L2_MPEG_VIDEO_H264_LEVEL_1_1:
+ return 11;
+ case V4L2_MPEG_VIDEO_H264_LEVEL_1_2:
+ return 12;
+ case V4L2_MPEG_VIDEO_H264_LEVEL_1_3:
+ return 13;
+ case V4L2_MPEG_VIDEO_H264_LEVEL_2_0:
+ return 20;
+ case V4L2_MPEG_VIDEO_H264_LEVEL_2_1:
+ return 21;
+ case V4L2_MPEG_VIDEO_H264_LEVEL_2_2:
+ return 22;
+ case V4L2_MPEG_VIDEO_H264_LEVEL_3_0:
+ return 30;
+ case V4L2_MPEG_VIDEO_H264_LEVEL_3_1:
+ return 31;
+ case V4L2_MPEG_VIDEO_H264_LEVEL_3_2:
+ return 32;
+ case V4L2_MPEG_VIDEO_H264_LEVEL_4_0:
+ return 40;
+ case V4L2_MPEG_VIDEO_H264_LEVEL_4_1:
+ return 41;
+ default:
+ return 31;
+ }
+}
+
+static void handle_h264_enc_init_msg(struct venc_h264_handle *hndl, void *data)
+{
+ struct venc_vpu_ipi_msg_init *msg = data;
+
+ hndl->vpu_inst.id = msg->inst_id;
+ hndl->vpu_inst.drv = (struct venc_h264_vpu_drv *)vpu_mapping_dm_addr(
+ hndl->dev, (uintptr_t *)(unsigned long)msg->inst_id);
+}
+
+static void handle_h264_enc_encode_msg(struct venc_h264_handle *hndl,
+ void *data)
+{
+ struct venc_vpu_ipi_msg_enc *msg = data;
+
+ hndl->vpu_inst.state = msg->state;
+ hndl->vpu_inst.bs_size = msg->bs_size;
+ hndl->is_key_frm = msg->key_frame;
+}
+
+static void h264_enc_vpu_ipi_handler(void *data, unsigned int len, void *priv)
+{
+ struct venc_vpu_ipi_msg_common *msg = data;
+ struct venc_h264_handle *hndl = (struct venc_h264_handle *)msg->venc_inst;
+
+ mtk_vcodec_debug_enter(hndl);
+
+ mtk_vcodec_debug(hndl, "msg_id %x hndl %p status %d",
+ msg->msg_id, hndl, msg->status);
+
+ switch (msg->msg_id) {
+ case VPU_IPIMSG_H264_ENC_INIT_DONE:
+ handle_h264_enc_init_msg(hndl, data);
+ break;
+ case VPU_IPIMSG_H264_ENC_SET_PARAM_DONE:
+ break;
+ case VPU_IPIMSG_H264_ENC_ENCODE_DONE:
+ handle_h264_enc_encode_msg(hndl, data);
+ break;
+ case VPU_IPIMSG_H264_ENC_DEINIT_DONE:
+ break;
+ default:
+ mtk_vcodec_err(hndl, "unknown msg id %x", msg->msg_id);
+ break;
+ }
+
+ hndl->vpu_inst.signaled = 1;
+ hndl->vpu_inst.failure = (msg->status != VENC_IPI_MSG_STATUS_OK);
+ wake_up_interruptible(&hndl->vpu_inst.wq_hd);
+
+ mtk_vcodec_debug_leave(hndl);
+}
+
+static int h264_enc_vpu_wait_ack(struct venc_h264_handle *hndl,
+ unsigned int timeout_ms)
+{
+ int ret;
+
+ mtk_vcodec_debug_enter(hndl);
+
+ ret = wait_event_interruptible_timeout(hndl->vpu_inst.wq_hd,
+ hndl->vpu_inst.signaled == 1,
+ msecs_to_jiffies(timeout_ms));
+ if (0 == ret) {
+ mtk_vcodec_err(hndl, "wait vpu ack time out !");
+ return -EINVAL;
+ }
+ if (-ERESTARTSYS == ret) {
+ mtk_vcodec_err(hndl,
+ "wait vpu ack interrupted by a signal");
+ return -EINVAL;
+ }
+
+ hndl->vpu_inst.signaled = 0;
+
+ mtk_vcodec_debug_leave(hndl);
+ return 0;
+}
+
+static int h264_enc_vpu_send_msg(struct venc_h264_handle *hndl, void *msg,
+ int len, int wait_ack)
+{
+ int status;
+
+ mtk_vcodec_debug_enter(hndl);
+
+ status = vpu_ipi_send(hndl->dev, IPI_VENC_H264, msg, len, 1);
+ if (status) {
+ mtk_vcodec_err(hndl, "vpu_ipi_send msg %x len %d fail %d",
+ *(unsigned int *)msg, len, status);
+ return -EINVAL;
+ }
+ mtk_vcodec_debug(hndl, "vpu_ipi_send msg %x success",
+ *(unsigned int *)msg);
+
+ if (wait_ack && h264_enc_vpu_wait_ack(hndl, 2000)) {
+ mtk_vcodec_err(hndl, "h264_enc_vpu_wait_ack failed");
+ return -EINVAL;
+ }
+
+ mtk_vcodec_debug_leave(hndl);
+ return 0;
+}
+
+int h264_enc_vpu_init(void *handle)
+{
+ int status;
+ struct venc_h264_handle *hndl = handle;
+ struct venc_ap_ipi_msg_init out;
+
+ mtk_vcodec_debug_enter(hndl);
+
+ init_waitqueue_head(&hndl->vpu_inst.wq_hd);
+ hndl->vpu_inst.signaled = 0;
+ hndl->vpu_inst.failure = 0;
+
+ status = vpu_ipi_register(hndl->dev, IPI_VENC_H264,
+ h264_enc_vpu_ipi_handler,
+ "h264_enc", NULL);
+ if (status) {
+ mtk_vcodec_err(hndl, "vpu_ipi_register fail %d", status);
+ return -EINVAL;
+ }
+ mtk_vcodec_debug(hndl, "vpu_ipi_register success");
+
+ out.msg_id = AP_IPIMSG_H264_ENC_INIT;
+ out.venc_inst = (unsigned long)hndl;
+ if (h264_enc_vpu_send_msg(hndl, &out, sizeof(out), 1) ||
+ hndl->vpu_inst.failure) {
+ mtk_vcodec_err(hndl, "AP_IPIMSG_H264_ENC_INIT failed");
+ return -EINVAL;
+ }
+
+ mtk_vcodec_debug_leave(hndl);
+
+ return 0;
+}
+
+int h264_enc_vpu_set_param(void *handle, unsigned int id, void *param)
+{
+ struct venc_h264_handle *hndl = handle;
+ struct venc_ap_ipi_msg_set_param out;
+
+ mtk_vcodec_debug(hndl, "id %d ->", id);
+
+ out.msg_id = AP_IPIMSG_H264_ENC_SET_PARAM;
+ out.inst_id = hndl->vpu_inst.id;
+ out.param_id = id;
+ switch (id) {
+ case VENC_SET_PARAM_ENC: {
+ struct venc_enc_prm *enc_param = param;
+
+ hndl->vpu_inst.drv->config.input_fourcc = enc_param->input_fourcc;
+ hndl->vpu_inst.drv->config.bitrate = enc_param->bitrate;
+ hndl->vpu_inst.drv->config.pic_w = enc_param->width;
+ hndl->vpu_inst.drv->config.pic_h = enc_param->height;
+ hndl->vpu_inst.drv->config.buf_w = enc_param->buf_width;
+ hndl->vpu_inst.drv->config.buf_h = enc_param->buf_height;
+ hndl->vpu_inst.drv->config.intra_period =
+ enc_param->intra_period;
+ hndl->vpu_inst.drv->config.framerate = enc_param->frm_rate;
+ hndl->vpu_inst.drv->config.profile =
+ h264_get_profile(enc_param->h264_profile);
+ hndl->vpu_inst.drv->config.level =
+ h264_get_level(enc_param->h264_level);
+ hndl->vpu_inst.drv->config.wfd = 0;
+ out.data_item = 0;
+ break;
+ }
+ case VENC_SET_PARAM_FORCE_INTRA:
+ out.data_item = 0;
+ break;
+ case VENC_SET_PARAM_ADJUST_BITRATE:
+ out.data_item = 1;
+ out.data[0] = *(unsigned int *)param;
+ break;
+ case VENC_SET_PARAM_ADJUST_FRAMERATE:
+ out.data_item = 1;
+ out.data[0] = *(unsigned int *)param;
+ break;
+ case VENC_SET_PARAM_I_FRAME_INTERVAL:
+ out.data_item = 1;
+ out.data[0] = *(unsigned int *)param;
+ break;
+ case VENC_SET_PARAM_SKIP_FRAME:
+ out.data_item = 0;
+ break;
+ }
+ if (h264_enc_vpu_send_msg(hndl, &out, sizeof(out), 1) ||
+ hndl->vpu_inst.failure) {
+ mtk_vcodec_err(hndl,
+ "AP_IPIMSG_H264_ENC_SET_PARAM %d fail", id);
+ return -EINVAL;
+ }
+
+ mtk_vcodec_debug(hndl, "id %d <-", id);
+
+ return 0;
+}
+
+int h264_enc_vpu_encode(void *handle, unsigned int bs_mode,
+ struct venc_frm_buf *frm_buf,
+ struct mtk_vcodec_mem *bs_buf,
+ unsigned int *bs_size)
+{
+ struct venc_h264_handle *hndl = handle;
+ struct venc_ap_ipi_msg_enc out;
+
+ mtk_vcodec_debug(hndl, "bs_mode %d ->", bs_mode);
+
+ out.msg_id = AP_IPIMSG_H264_ENC_ENCODE;
+ out.inst_id = hndl->vpu_inst.id;
+ out.bs_mode = bs_mode;
+ if (frm_buf) {
+ out.input_addr[0] = frm_buf->fb_addr.dma_addr;
+ out.input_addr[1] = frm_buf->fb_addr1.dma_addr;
+ out.input_addr[2] = frm_buf->fb_addr2.dma_addr;
+ } else {
+ out.input_addr[0] = 0;
+ out.input_addr[1] = 0;
+ out.input_addr[2] = 0;
+ }
+ if (bs_buf) {
+ out.bs_addr = bs_buf->dma_addr;
+ out.bs_size = bs_buf->size;
+ } else {
+ out.bs_addr = 0;
+ out.bs_size = 0;
+ }
+ if (h264_enc_vpu_send_msg(hndl, &out, sizeof(out), 1) ||
+ hndl->vpu_inst.failure) {
+ mtk_vcodec_err(hndl, "AP_IPIMSG_H264_ENC_ENCODE %d fail",
+ bs_mode);
+ return -EINVAL;
+ }
+
+ mtk_vcodec_debug(hndl, "state %d size %d key_frm %d",
+ hndl->vpu_inst.state, hndl->vpu_inst.bs_size,
+ hndl->is_key_frm);
+ hndl->vpu_inst.wait_int = 1;
+ if (hndl->vpu_inst.state == VEN_IPI_MSG_ENC_STATE_SKIP) {
+ *bs_size = hndl->vpu_inst.bs_size;
+ memcpy(bs_buf->va,
+ hndl->work_bufs[VENC_H264_VPU_WORK_BUF_SKIP_FRAME].va,
+ *bs_size);
+ hndl->vpu_inst.wait_int = 0;
+ }
+
+ mtk_vcodec_debug(hndl, "bs_mode %d ->", bs_mode);
+
+ return 0;
+}
+
+int h264_enc_vpu_deinit(void *handle)
+{
+ struct venc_h264_handle *hndl = handle;
+ struct venc_ap_ipi_msg_deinit out;
+
+ mtk_vcodec_debug_enter(hndl);
+
+ out.msg_id = AP_IPIMSG_H264_ENC_DEINIT;
+ out.inst_id = hndl->vpu_inst.id;
+ if (h264_enc_vpu_send_msg(hndl, &out, sizeof(out), 1) ||
+ hndl->vpu_inst.failure) {
+ mtk_vcodec_err(hndl, "AP_IPIMSG_H264_ENC_DEINIT fail");
+ return -EINVAL;
+ }
+
+ mtk_vcodec_debug_leave(hndl);
+
+ return 0;
+}
--
1.7.9.5
^ permalink raw reply related [flat|nested] 84+ messages in thread
* Re: [RESEND RFC/PATCH 1/8] dt-bindings: Add a binding for Mediatek Video Processor Unit
2015-11-17 12:54 ` Tiffany Lin
(?)
@ 2015-11-17 14:13 ` Mark Rutland
-1 siblings, 0 replies; 84+ messages in thread
From: Mark Rutland @ 2015-11-17 14:13 UTC (permalink / raw)
To: Tiffany Lin
Cc: Rob Herring, Pawel Moll, Ian Campbell, Kumar Gala,
Catalin Marinas, Will Deacon, Mauro Carvalho Chehab,
Matthias Brugger, Daniel Kurtz, Sascha Hauer, Hongzhou Yang,
Hans Verkuil, Laurent Pinchart, Sakari Ailus, Geert Uytterhoeven,
Mikhail Ulyanov, Fabien Dessenne, Arnd Bergmann,
Darren Etheridge, Peter Griffin, Benoit Parrot, Andrew-CT Chen,
Eddie Huang, Yingjoe Chen, James Liao, Daniel Hsiao, devicetree,
linux-kernel, linux-arm-kernel, linux-media, linux-mediatek
On Tue, Nov 17, 2015 at 08:54:38PM +0800, Tiffany Lin wrote:
> From: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
>
> Add a DT binding documentation of Video Processor Unit for the
> MT8173 SoC from Mediatek.
>
> Signed-off-by: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
> ---
> .../devicetree/bindings/media/mediatek-vpu.txt | 27 ++++++++++++++++++++
> 1 file changed, 27 insertions(+)
> create mode 100644 Documentation/devicetree/bindings/media/mediatek-vpu.txt
>
> diff --git a/Documentation/devicetree/bindings/media/mediatek-vpu.txt b/Documentation/devicetree/bindings/media/mediatek-vpu.txt
> new file mode 100644
> index 0000000..99a4e5e
> --- /dev/null
> +++ b/Documentation/devicetree/bindings/media/mediatek-vpu.txt
> @@ -0,0 +1,27 @@
> +* Mediatek Video Processor Unit
> +
> +Video Processor Unit is a HW video controller. It controls HW Codec including
> +H.264/VP8/VP9 Decode, H.264/VP8 Encode and Image Processor (scale/rotate/color convert).
> +
> +Required properties:
> + - compatible: "mediatek,mt8173-vpu"
> + - reg: Must contain an entry for each entry in reg-names.
> + - reg-names: Must include the following entries:
> + "sram": SRAM base
> + "cfg_reg": Main configuration registers base
> + - interrupts: interrupt number to the cpu.
> + - clocks : clock name from clock manager
> + - clock-names: the clocks of the VPU H/W
You need to explicitly define the set of clock-names you expect here.
Mark.
> + - iommus : phandle and IOMMU spcifier for the IOMMU that serves the VPU.
> +
> +Example:
> + vpu: vpu@10020000 {
> + compatible = "mediatek,mt8173-vpu";
> + reg = <0 0x10020000 0 0x30000>,
> + <0 0x10050000 0 0x100>;
> + reg-names = "sram", "cfg_reg";
> + interrupts = <GIC_SPI 166 IRQ_TYPE_LEVEL_HIGH>;
> + clocks = <&topckgen TOP_SCP_SEL>;
> + clock-names = "main";
> + iommus = <&iommu M4U_PORT_VENC_RCPU>;
> + };
> --
> 1.7.9.5
>
^ permalink raw reply [flat|nested] 84+ messages in thread
* Re: [RESEND RFC/PATCH 1/8] dt-bindings: Add a binding for Mediatek Video Processor Unit
@ 2015-11-17 14:13 ` Mark Rutland
0 siblings, 0 replies; 84+ messages in thread
From: Mark Rutland @ 2015-11-17 14:13 UTC (permalink / raw)
To: Tiffany Lin
Cc: Rob Herring, Pawel Moll, Ian Campbell, Kumar Gala,
Catalin Marinas, Will Deacon, Mauro Carvalho Chehab,
Matthias Brugger, Daniel Kurtz, Sascha Hauer, Hongzhou Yang,
Hans Verkuil, Laurent Pinchart, Sakari Ailus, Geert Uytterhoeven,
Mikhail Ulyanov, Fabien Dessenne, Arnd Bergmann,
Darren Etheridge, Peter Griffin, Benoit Parrot, Andrew-CT Chen
On Tue, Nov 17, 2015 at 08:54:38PM +0800, Tiffany Lin wrote:
> From: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
>
> Add a DT binding documentation of Video Processor Unit for the
> MT8173 SoC from Mediatek.
>
> Signed-off-by: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
> ---
> .../devicetree/bindings/media/mediatek-vpu.txt | 27 ++++++++++++++++++++
> 1 file changed, 27 insertions(+)
> create mode 100644 Documentation/devicetree/bindings/media/mediatek-vpu.txt
>
> diff --git a/Documentation/devicetree/bindings/media/mediatek-vpu.txt b/Documentation/devicetree/bindings/media/mediatek-vpu.txt
> new file mode 100644
> index 0000000..99a4e5e
> --- /dev/null
> +++ b/Documentation/devicetree/bindings/media/mediatek-vpu.txt
> @@ -0,0 +1,27 @@
> +* Mediatek Video Processor Unit
> +
> +Video Processor Unit is a HW video controller. It controls HW Codec including
> +H.264/VP8/VP9 Decode, H.264/VP8 Encode and Image Processor (scale/rotate/color convert).
> +
> +Required properties:
> + - compatible: "mediatek,mt8173-vpu"
> + - reg: Must contain an entry for each entry in reg-names.
> + - reg-names: Must include the following entries:
> + "sram": SRAM base
> + "cfg_reg": Main configuration registers base
> + - interrupts: interrupt number to the cpu.
> + - clocks : clock name from clock manager
> + - clock-names: the clocks of the VPU H/W
You need to explicitly define the set of clock-names you expect here.
Mark.
> + - iommus : phandle and IOMMU spcifier for the IOMMU that serves the VPU.
> +
> +Example:
> + vpu: vpu@10020000 {
> + compatible = "mediatek,mt8173-vpu";
> + reg = <0 0x10020000 0 0x30000>,
> + <0 0x10050000 0 0x100>;
> + reg-names = "sram", "cfg_reg";
> + interrupts = <GIC_SPI 166 IRQ_TYPE_LEVEL_HIGH>;
> + clocks = <&topckgen TOP_SCP_SEL>;
> + clock-names = "main";
> + iommus = <&iommu M4U_PORT_VENC_RCPU>;
> + };
> --
> 1.7.9.5
>
^ permalink raw reply [flat|nested] 84+ messages in thread
* [RESEND RFC/PATCH 1/8] dt-bindings: Add a binding for Mediatek Video Processor Unit
@ 2015-11-17 14:13 ` Mark Rutland
0 siblings, 0 replies; 84+ messages in thread
From: Mark Rutland @ 2015-11-17 14:13 UTC (permalink / raw)
To: linux-arm-kernel
On Tue, Nov 17, 2015 at 08:54:38PM +0800, Tiffany Lin wrote:
> From: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
>
> Add a DT binding documentation of Video Processor Unit for the
> MT8173 SoC from Mediatek.
>
> Signed-off-by: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
> ---
> .../devicetree/bindings/media/mediatek-vpu.txt | 27 ++++++++++++++++++++
> 1 file changed, 27 insertions(+)
> create mode 100644 Documentation/devicetree/bindings/media/mediatek-vpu.txt
>
> diff --git a/Documentation/devicetree/bindings/media/mediatek-vpu.txt b/Documentation/devicetree/bindings/media/mediatek-vpu.txt
> new file mode 100644
> index 0000000..99a4e5e
> --- /dev/null
> +++ b/Documentation/devicetree/bindings/media/mediatek-vpu.txt
> @@ -0,0 +1,27 @@
> +* Mediatek Video Processor Unit
> +
> +Video Processor Unit is a HW video controller. It controls HW Codec including
> +H.264/VP8/VP9 Decode, H.264/VP8 Encode and Image Processor (scale/rotate/color convert).
> +
> +Required properties:
> + - compatible: "mediatek,mt8173-vpu"
> + - reg: Must contain an entry for each entry in reg-names.
> + - reg-names: Must include the following entries:
> + "sram": SRAM base
> + "cfg_reg": Main configuration registers base
> + - interrupts: interrupt number to the cpu.
> + - clocks : clock name from clock manager
> + - clock-names: the clocks of the VPU H/W
You need to explicitly define the set of clock-names you expect here.
Mark.
> + - iommus : phandle and IOMMU spcifier for the IOMMU that serves the VPU.
> +
> +Example:
> + vpu: vpu at 10020000 {
> + compatible = "mediatek,mt8173-vpu";
> + reg = <0 0x10020000 0 0x30000>,
> + <0 0x10050000 0 0x100>;
> + reg-names = "sram", "cfg_reg";
> + interrupts = <GIC_SPI 166 IRQ_TYPE_LEVEL_HIGH>;
> + clocks = <&topckgen TOP_SCP_SEL>;
> + clock-names = "main";
> + iommus = <&iommu M4U_PORT_VENC_RCPU>;
> + };
> --
> 1.7.9.5
>
^ permalink raw reply [flat|nested] 84+ messages in thread
* Re: [RESEND RFC/PATCH 4/8] dt-bindings: Add a binding for Mediatek Video Encoder
2015-11-17 12:54 ` Tiffany Lin
(?)
@ 2015-11-17 19:41 ` Rob Herring
-1 siblings, 0 replies; 84+ messages in thread
From: Rob Herring @ 2015-11-17 19:41 UTC (permalink / raw)
To: Tiffany Lin
Cc: Pawel Moll, Mark Rutland, Ian Campbell, Kumar Gala,
Catalin Marinas, Will Deacon, Mauro Carvalho Chehab,
Matthias Brugger, Daniel Kurtz, Sascha Hauer, Hongzhou Yang,
Hans Verkuil, Laurent Pinchart, Sakari Ailus, Geert Uytterhoeven,
Mikhail Ulyanov, Fabien Dessenne, Arnd Bergmann,
Darren Etheridge, Peter Griffin, Benoit Parrot, Andrew-CT Chen,
Eddie Huang, Yingjoe Chen, James Liao, Daniel Hsiao, devicetree,
linux-kernel, linux-arm-kernel, linux-media, linux-mediatek
On Tue, Nov 17, 2015 at 08:54:41PM +0800, Tiffany Lin wrote:
> add a DT binding documentation of Video Encoder for the
> MT8173 SoC from Mediatek.
>
> Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
> ---
> .../devicetree/bindings/media/mediatek-vcodec.txt | 58 ++++++++++++++++++++
> 1 file changed, 58 insertions(+)
> create mode 100644 Documentation/devicetree/bindings/media/mediatek-vcodec.txt
>
> diff --git a/Documentation/devicetree/bindings/media/mediatek-vcodec.txt b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
> new file mode 100644
> index 0000000..fea4d7c
> --- /dev/null
> +++ b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
> @@ -0,0 +1,58 @@
> +Mediatek Video Codec
> +
> +Mediatek Video Codec is the video codec hw present in Mediatek SoCs which
> +supports high resolution encoding functionalities.
> +
> +Required properties:
> +- compatible : "mediatek,mt8173-vcodec-enc" for encoder
> +- reg : Physical base address of the video codec registers and length of
> + memory mapped region.
> +- interrupts : interrupt number to the cpu.
> +- larb : must contain the larbes of current platform
What is this?
> +- clocks : list of clock specifiers, corresponding to entries in
> + the clock-names property;
> +- clock-names: must contain "vencpll", "venc_lt_sel", "vcodecpll_370p5_ck"
> +- iommus : list of iommus specifiers should be enabled for hw encode.
> + There are 2 cells needed to enable/disable iommu.
> + The first one is local arbiter index(larbid), and the other is port
> + index(portid) within local arbiter. Specifies the larbid and portid
> + as defined in dt-binding/memory/mt8173-larb-port.h.
> +- vpu : the node of video processor unit
This should be prefixed with mediatek.
> +
> +Example:
> +vcodec_enc: vcodec@0x18002000 {
> + compatible = "mediatek,mt8173-vcodec-enc";
> + reg = <0 0x18002000 0 0x1000>, /*VENC_SYS*/
> + <0 0x19002000 0 0x1000>; /*VENC_LT_SYS*/
> + interrupts = <GIC_SPI 198 IRQ_TYPE_LEVEL_LOW>,
> + <GIC_SPI 202 IRQ_TYPE_LEVEL_LOW>;
> + larb = <&larb3>,
> + <&larb5>;
> + iommus = <&iommu M4U_LARB3_ID M4U_PORT_VENC_RCPU>,
> + <&iommu M4U_LARB3_ID M4U_PORT_VENC_REC>,
> + <&iommu M4U_LARB3_ID M4U_PORT_VENC_BSDMA>,
> + <&iommu M4U_LARB3_ID M4U_PORT_VENC_SV_COMV>,
> + <&iommu M4U_LARB3_ID M4U_PORT_VENC_RD_COMV>,
> + <&iommu M4U_LARB3_ID M4U_PORT_VENC_CUR_LUMA>,
> + <&iommu M4U_LARB3_ID M4U_PORT_VENC_CUR_CHROMA>,
> + <&iommu M4U_LARB3_ID M4U_PORT_VENC_REF_LUMA>,
> + <&iommu M4U_LARB3_ID M4U_PORT_VENC_REF_CHROMA>,
> + <&iommu M4U_LARB3_ID M4U_PORT_VENC_NBM_RDMA>,
> + <&iommu M4U_LARB3_ID M4U_PORT_VENC_NBM_WDMA>,
> + <&iommu M4U_LARB5_ID M4U_PORT_VENC_RCPU_SET2>,
> + <&iommu M4U_LARB5_ID M4U_PORT_VENC_REC_FRM_SET2>,
> + <&iommu M4U_LARB5_ID M4U_PORT_VENC_BSDMA_SET2>,
> + <&iommu M4U_LARB5_ID M4U_PORT_VENC_SV_COMA_SET2>,
> + <&iommu M4U_LARB5_ID M4U_PORT_VENC_RD_COMA_SET2>,
> + <&iommu M4U_LARB5_ID M4U_PORT_VENC_CUR_LUMA_SET2>,
> + <&iommu M4U_LARB5_ID M4U_PORT_VENC_CUR_CHROMA_SET2>,
> + <&iommu M4U_LARB5_ID M4U_PORT_VENC_REF_LUMA_SET2>,
> + <&iommu M4U_LARB5_ID M4U_PORT_VENC_REC_CHROMA_SET2>;
> + vpu = <&vpu>;
> + clocks = <&apmixedsys CLK_APMIXED_VENCPLL>,
> + <&topckgen CLK_TOP_VENC_LT_SEL>,
> + <&topckgen CLK_TOP_VCODECPLL_370P5>;
> + clock-names = "vencpll",
> + "venc_lt_sel",
> + "vcodecpll_370p5_ck";
> + };
> --
> 1.7.9.5
>
^ permalink raw reply [flat|nested] 84+ messages in thread
* Re: [RESEND RFC/PATCH 4/8] dt-bindings: Add a binding for Mediatek Video Encoder
@ 2015-11-17 19:41 ` Rob Herring
0 siblings, 0 replies; 84+ messages in thread
From: Rob Herring @ 2015-11-17 19:41 UTC (permalink / raw)
To: Tiffany Lin
Cc: Pawel Moll, Mark Rutland, Ian Campbell, Kumar Gala,
Catalin Marinas, Will Deacon, Mauro Carvalho Chehab,
Matthias Brugger, Daniel Kurtz, Sascha Hauer, Hongzhou Yang,
Hans Verkuil, Laurent Pinchart, Sakari Ailus, Geert Uytterhoeven,
Mikhail Ulyanov, Fabien Dessenne, Arnd Bergmann,
Darren Etheridge, Peter Griffin, Benoit Parrot, Andrew-CT Chen
On Tue, Nov 17, 2015 at 08:54:41PM +0800, Tiffany Lin wrote:
> add a DT binding documentation of Video Encoder for the
> MT8173 SoC from Mediatek.
>
> Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
> ---
> .../devicetree/bindings/media/mediatek-vcodec.txt | 58 ++++++++++++++++++++
> 1 file changed, 58 insertions(+)
> create mode 100644 Documentation/devicetree/bindings/media/mediatek-vcodec.txt
>
> diff --git a/Documentation/devicetree/bindings/media/mediatek-vcodec.txt b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
> new file mode 100644
> index 0000000..fea4d7c
> --- /dev/null
> +++ b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
> @@ -0,0 +1,58 @@
> +Mediatek Video Codec
> +
> +Mediatek Video Codec is the video codec hw present in Mediatek SoCs which
> +supports high resolution encoding functionalities.
> +
> +Required properties:
> +- compatible : "mediatek,mt8173-vcodec-enc" for encoder
> +- reg : Physical base address of the video codec registers and length of
> + memory mapped region.
> +- interrupts : interrupt number to the cpu.
> +- larb : must contain the larbes of current platform
What is this?
> +- clocks : list of clock specifiers, corresponding to entries in
> + the clock-names property;
> +- clock-names: must contain "vencpll", "venc_lt_sel", "vcodecpll_370p5_ck"
> +- iommus : list of iommus specifiers should be enabled for hw encode.
> + There are 2 cells needed to enable/disable iommu.
> + The first one is local arbiter index(larbid), and the other is port
> + index(portid) within local arbiter. Specifies the larbid and portid
> + as defined in dt-binding/memory/mt8173-larb-port.h.
> +- vpu : the node of video processor unit
This should be prefixed with mediatek.
> +
> +Example:
> +vcodec_enc: vcodec@0x18002000 {
> + compatible = "mediatek,mt8173-vcodec-enc";
> + reg = <0 0x18002000 0 0x1000>, /*VENC_SYS*/
> + <0 0x19002000 0 0x1000>; /*VENC_LT_SYS*/
> + interrupts = <GIC_SPI 198 IRQ_TYPE_LEVEL_LOW>,
> + <GIC_SPI 202 IRQ_TYPE_LEVEL_LOW>;
> + larb = <&larb3>,
> + <&larb5>;
> + iommus = <&iommu M4U_LARB3_ID M4U_PORT_VENC_RCPU>,
> + <&iommu M4U_LARB3_ID M4U_PORT_VENC_REC>,
> + <&iommu M4U_LARB3_ID M4U_PORT_VENC_BSDMA>,
> + <&iommu M4U_LARB3_ID M4U_PORT_VENC_SV_COMV>,
> + <&iommu M4U_LARB3_ID M4U_PORT_VENC_RD_COMV>,
> + <&iommu M4U_LARB3_ID M4U_PORT_VENC_CUR_LUMA>,
> + <&iommu M4U_LARB3_ID M4U_PORT_VENC_CUR_CHROMA>,
> + <&iommu M4U_LARB3_ID M4U_PORT_VENC_REF_LUMA>,
> + <&iommu M4U_LARB3_ID M4U_PORT_VENC_REF_CHROMA>,
> + <&iommu M4U_LARB3_ID M4U_PORT_VENC_NBM_RDMA>,
> + <&iommu M4U_LARB3_ID M4U_PORT_VENC_NBM_WDMA>,
> + <&iommu M4U_LARB5_ID M4U_PORT_VENC_RCPU_SET2>,
> + <&iommu M4U_LARB5_ID M4U_PORT_VENC_REC_FRM_SET2>,
> + <&iommu M4U_LARB5_ID M4U_PORT_VENC_BSDMA_SET2>,
> + <&iommu M4U_LARB5_ID M4U_PORT_VENC_SV_COMA_SET2>,
> + <&iommu M4U_LARB5_ID M4U_PORT_VENC_RD_COMA_SET2>,
> + <&iommu M4U_LARB5_ID M4U_PORT_VENC_CUR_LUMA_SET2>,
> + <&iommu M4U_LARB5_ID M4U_PORT_VENC_CUR_CHROMA_SET2>,
> + <&iommu M4U_LARB5_ID M4U_PORT_VENC_REF_LUMA_SET2>,
> + <&iommu M4U_LARB5_ID M4U_PORT_VENC_REC_CHROMA_SET2>;
> + vpu = <&vpu>;
> + clocks = <&apmixedsys CLK_APMIXED_VENCPLL>,
> + <&topckgen CLK_TOP_VENC_LT_SEL>,
> + <&topckgen CLK_TOP_VCODECPLL_370P5>;
> + clock-names = "vencpll",
> + "venc_lt_sel",
> + "vcodecpll_370p5_ck";
> + };
> --
> 1.7.9.5
>
^ permalink raw reply [flat|nested] 84+ messages in thread
* [RESEND RFC/PATCH 4/8] dt-bindings: Add a binding for Mediatek Video Encoder
@ 2015-11-17 19:41 ` Rob Herring
0 siblings, 0 replies; 84+ messages in thread
From: Rob Herring @ 2015-11-17 19:41 UTC (permalink / raw)
To: linux-arm-kernel
On Tue, Nov 17, 2015 at 08:54:41PM +0800, Tiffany Lin wrote:
> add a DT binding documentation of Video Encoder for the
> MT8173 SoC from Mediatek.
>
> Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
> ---
> .../devicetree/bindings/media/mediatek-vcodec.txt | 58 ++++++++++++++++++++
> 1 file changed, 58 insertions(+)
> create mode 100644 Documentation/devicetree/bindings/media/mediatek-vcodec.txt
>
> diff --git a/Documentation/devicetree/bindings/media/mediatek-vcodec.txt b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
> new file mode 100644
> index 0000000..fea4d7c
> --- /dev/null
> +++ b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
> @@ -0,0 +1,58 @@
> +Mediatek Video Codec
> +
> +Mediatek Video Codec is the video codec hw present in Mediatek SoCs which
> +supports high resolution encoding functionalities.
> +
> +Required properties:
> +- compatible : "mediatek,mt8173-vcodec-enc" for encoder
> +- reg : Physical base address of the video codec registers and length of
> + memory mapped region.
> +- interrupts : interrupt number to the cpu.
> +- larb : must contain the larbes of current platform
What is this?
> +- clocks : list of clock specifiers, corresponding to entries in
> + the clock-names property;
> +- clock-names: must contain "vencpll", "venc_lt_sel", "vcodecpll_370p5_ck"
> +- iommus : list of iommus specifiers should be enabled for hw encode.
> + There are 2 cells needed to enable/disable iommu.
> + The first one is local arbiter index(larbid), and the other is port
> + index(portid) within local arbiter. Specifies the larbid and portid
> + as defined in dt-binding/memory/mt8173-larb-port.h.
> +- vpu : the node of video processor unit
This should be prefixed with mediatek.
> +
> +Example:
> +vcodec_enc: vcodec at 0x18002000 {
> + compatible = "mediatek,mt8173-vcodec-enc";
> + reg = <0 0x18002000 0 0x1000>, /*VENC_SYS*/
> + <0 0x19002000 0 0x1000>; /*VENC_LT_SYS*/
> + interrupts = <GIC_SPI 198 IRQ_TYPE_LEVEL_LOW>,
> + <GIC_SPI 202 IRQ_TYPE_LEVEL_LOW>;
> + larb = <&larb3>,
> + <&larb5>;
> + iommus = <&iommu M4U_LARB3_ID M4U_PORT_VENC_RCPU>,
> + <&iommu M4U_LARB3_ID M4U_PORT_VENC_REC>,
> + <&iommu M4U_LARB3_ID M4U_PORT_VENC_BSDMA>,
> + <&iommu M4U_LARB3_ID M4U_PORT_VENC_SV_COMV>,
> + <&iommu M4U_LARB3_ID M4U_PORT_VENC_RD_COMV>,
> + <&iommu M4U_LARB3_ID M4U_PORT_VENC_CUR_LUMA>,
> + <&iommu M4U_LARB3_ID M4U_PORT_VENC_CUR_CHROMA>,
> + <&iommu M4U_LARB3_ID M4U_PORT_VENC_REF_LUMA>,
> + <&iommu M4U_LARB3_ID M4U_PORT_VENC_REF_CHROMA>,
> + <&iommu M4U_LARB3_ID M4U_PORT_VENC_NBM_RDMA>,
> + <&iommu M4U_LARB3_ID M4U_PORT_VENC_NBM_WDMA>,
> + <&iommu M4U_LARB5_ID M4U_PORT_VENC_RCPU_SET2>,
> + <&iommu M4U_LARB5_ID M4U_PORT_VENC_REC_FRM_SET2>,
> + <&iommu M4U_LARB5_ID M4U_PORT_VENC_BSDMA_SET2>,
> + <&iommu M4U_LARB5_ID M4U_PORT_VENC_SV_COMA_SET2>,
> + <&iommu M4U_LARB5_ID M4U_PORT_VENC_RD_COMA_SET2>,
> + <&iommu M4U_LARB5_ID M4U_PORT_VENC_CUR_LUMA_SET2>,
> + <&iommu M4U_LARB5_ID M4U_PORT_VENC_CUR_CHROMA_SET2>,
> + <&iommu M4U_LARB5_ID M4U_PORT_VENC_REF_LUMA_SET2>,
> + <&iommu M4U_LARB5_ID M4U_PORT_VENC_REC_CHROMA_SET2>;
> + vpu = <&vpu>;
> + clocks = <&apmixedsys CLK_APMIXED_VENCPLL>,
> + <&topckgen CLK_TOP_VENC_LT_SEL>,
> + <&topckgen CLK_TOP_VCODECPLL_370P5>;
> + clock-names = "vencpll",
> + "venc_lt_sel",
> + "vcodecpll_370p5_ck";
> + };
> --
> 1.7.9.5
>
^ permalink raw reply [flat|nested] 84+ messages in thread
* Re: [RESEND RFC/PATCH 4/8] dt-bindings: Add a binding for Mediatek Video Encoder
2015-11-17 19:41 ` Rob Herring
(?)
(?)
@ 2015-11-18 6:21 ` tiffany lin
-1 siblings, 0 replies; 84+ messages in thread
From: tiffany lin @ 2015-11-18 6:21 UTC (permalink / raw)
To: Rob Herring
Cc: Pawel Moll, Mark Rutland, Ian Campbell, Kumar Gala,
Catalin Marinas, Will Deacon, Mauro Carvalho Chehab,
Matthias Brugger, Daniel Kurtz, Sascha Hauer, Hongzhou Yang,
Hans Verkuil, Laurent Pinchart, Sakari Ailus, Mikhail Ulyanov,
Fabien Dessenne, Arnd Bergmann, Darren Etheridge, Peter Griffin,
Benoit Parrot, Andrew-CT Chen, Eddie Huang, Yingjoe Chen,
James Liao, Daniel Hsiao, devicetree, linux-kernel,
linux-arm-kernel, linux-media, linux-mediatek
[-- Attachment #1: Type: text/plain, Size: 4397 bytes --]
Hi Rob,
On Tue, 2015-11-17 at 13:41 -0600, Rob Herring wrote:
> On Tue, Nov 17, 2015 at 08:54:41PM +0800, Tiffany Lin wrote:
> > add a DT binding documentation of Video Encoder for the
> > MT8173 SoC from Mediatek.
> >
> > Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
> > ---
> > .../devicetree/bindings/media/mediatek-vcodec.txt | 58 ++++++++++++++++++++
> > 1 file changed, 58 insertions(+)
> > create mode 100644 Documentation/devicetree/bindings/media/mediatek-vcodec.txt
> >
> > diff --git a/Documentation/devicetree/bindings/media/mediatek-vcodec.txt b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
> > new file mode 100644
> > index 0000000..fea4d7c
> > --- /dev/null
> > +++ b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
> > @@ -0,0 +1,58 @@
> > +Mediatek Video Codec
> > +
> > +Mediatek Video Codec is the video codec hw present in Mediatek SoCs which
> > +supports high resolution encoding functionalities.
> > +
> > +Required properties:
> > +- compatible : "mediatek,mt8173-vcodec-enc" for encoder
> > +- reg : Physical base address of the video codec registers and length of
> > + memory mapped region.
> > +- interrupts : interrupt number to the cpu.
> > +- larb : must contain the larbes of current platform
>
> What is this?
This is SMI (Smart Multimedia Interface) Local Arbiter.
MT8173 has different local arbiters(larb).
Please see
http://lists.linuxfoundation.org/pipermail/iommu/2015-October/014587.html
Video Encoder HW has it's local arbiter (larb), by configuring ports in
this larb, we can have encoder HW go through the m4u to talk with EMI.
We will change to mediatek,larb in next version.
>
> > +- clocks : list of clock specifiers, corresponding to entries in
> > + the clock-names property;
> > +- clock-names: must contain "vencpll", "venc_lt_sel", "vcodecpll_370p5_ck"
> > +- iommus : list of iommus specifiers should be enabled for hw encode.
> > + There are 2 cells needed to enable/disable iommu.
> > + The first one is local arbiter index(larbid), and the other is port
> > + index(portid) within local arbiter. Specifies the larbid and portid
> > + as defined in dt-binding/memory/mt8173-larb-port.h.
> > +- vpu : the node of video processor unit
>
> This should be prefixed with mediatek.
Got it, we will prefix it with mediatek in next version.
>
> > +
> > +Example:
> > +vcodec_enc: vcodec@0x18002000 {
> > + compatible = "mediatek,mt8173-vcodec-enc";
> > + reg = <0 0x18002000 0 0x1000>, /*VENC_SYS*/
> > + <0 0x19002000 0 0x1000>; /*VENC_LT_SYS*/
> > + interrupts = <GIC_SPI 198 IRQ_TYPE_LEVEL_LOW>,
> > + <GIC_SPI 202 IRQ_TYPE_LEVEL_LOW>;
> > + larb = <&larb3>,
> > + <&larb5>;
> > + iommus = <&iommu M4U_LARB3_ID M4U_PORT_VENC_RCPU>,
> > + <&iommu M4U_LARB3_ID M4U_PORT_VENC_REC>,
> > + <&iommu M4U_LARB3_ID M4U_PORT_VENC_BSDMA>,
> > + <&iommu M4U_LARB3_ID M4U_PORT_VENC_SV_COMV>,
> > + <&iommu M4U_LARB3_ID M4U_PORT_VENC_RD_COMV>,
> > + <&iommu M4U_LARB3_ID M4U_PORT_VENC_CUR_LUMA>,
> > + <&iommu M4U_LARB3_ID M4U_PORT_VENC_CUR_CHROMA>,
> > + <&iommu M4U_LARB3_ID M4U_PORT_VENC_REF_LUMA>,
> > + <&iommu M4U_LARB3_ID M4U_PORT_VENC_REF_CHROMA>,
> > + <&iommu M4U_LARB3_ID M4U_PORT_VENC_NBM_RDMA>,
> > + <&iommu M4U_LARB3_ID M4U_PORT_VENC_NBM_WDMA>,
> > + <&iommu M4U_LARB5_ID M4U_PORT_VENC_RCPU_SET2>,
> > + <&iommu M4U_LARB5_ID M4U_PORT_VENC_REC_FRM_SET2>,
> > + <&iommu M4U_LARB5_ID M4U_PORT_VENC_BSDMA_SET2>,
> > + <&iommu M4U_LARB5_ID M4U_PORT_VENC_SV_COMA_SET2>,
> > + <&iommu M4U_LARB5_ID M4U_PORT_VENC_RD_COMA_SET2>,
> > + <&iommu M4U_LARB5_ID M4U_PORT_VENC_CUR_LUMA_SET2>,
> > + <&iommu M4U_LARB5_ID M4U_PORT_VENC_CUR_CHROMA_SET2>,
> > + <&iommu M4U_LARB5_ID M4U_PORT_VENC_REF_LUMA_SET2>,
> > + <&iommu M4U_LARB5_ID M4U_PORT_VENC_REC_CHROMA_SET2>;
> > + vpu = <&vpu>;
> > + clocks = <&apmixedsys CLK_APMIXED_VENCPLL>,
> > + <&topckgen CLK_TOP_VENC_LT_SEL>,
> > + <&topckgen CLK_TOP_VCODECPLL_370P5>;
> > + clock-names = "vencpll",
> > + "venc_lt_sel",
> > + "vcodecpll_370p5_ck";
> > + };
> > --
> > 1.7.9.5
> >
best regards,
Tiffany
[-- Attachment #2: Type: text/html, Size: 5622 bytes --]
^ permalink raw reply [flat|nested] 84+ messages in thread
* Re: [RESEND RFC/PATCH 4/8] dt-bindings: Add a binding for Mediatek Video Encoder
2015-11-17 19:41 ` Rob Herring
(?)
@ 2015-11-18 7:09 ` tiffany lin
-1 siblings, 0 replies; 84+ messages in thread
From: tiffany lin @ 2015-11-18 7:09 UTC (permalink / raw)
To: Rob Herring
Cc: Pawel Moll, Mark Rutland, Ian Campbell, Kumar Gala,
Catalin Marinas, Will Deacon, Mauro Carvalho Chehab,
Matthias Brugger, Daniel Kurtz, Sascha Hauer, Hongzhou Yang,
Hans Verkuil, Laurent Pinchart, Sakari Ailus, Mikhail Ulyanov,
Fabien Dessenne, Arnd Bergmann, Darren Etheridge, Peter Griffin,
Benoit Parrot, Andrew-CT Chen, Eddie Huang, Yingjoe Chen,
James Liao, Daniel Hsiao, devicetree, linux-kernel,
linux-arm-kernel, linux-media, linux-mediatek
Hi Rob,
On Tue, 2015-11-17 at 13:41 -0600, Rob Herring wrote:
> On Tue, Nov 17, 2015 at 08:54:41PM +0800, Tiffany Lin wrote:
> > add a DT binding documentation of Video Encoder for the
> > MT8173 SoC from Mediatek.
> >
> > Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
> > ---
> > .../devicetree/bindings/media/mediatek-vcodec.txt | 58 ++++++++++++++++++++
> > 1 file changed, 58 insertions(+)
> > create mode 100644 Documentation/devicetree/bindings/media/mediatek-vcodec.txt
> >
> > diff --git a/Documentation/devicetree/bindings/media/mediatek-vcodec.txt b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
> > new file mode 100644
> > index 0000000..fea4d7c
> > --- /dev/null
> > +++ b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
> > @@ -0,0 +1,58 @@
> > +Mediatek Video Codec
> > +
> > +Mediatek Video Codec is the video codec hw present in Mediatek SoCs which
> > +supports high resolution encoding functionalities.
> > +
> > +Required properties:
> > +- compatible : "mediatek,mt8173-vcodec-enc" for encoder
> > +- reg : Physical base address of the video codec registers and length of
> > + memory mapped region.
> > +- interrupts : interrupt number to the cpu.
> > +- larb : must contain the larbes of current platform
>
> What is this?
Resend due to previous mail has html format and reject by some mail
servers.
This is SMI (Smart Multimedia Interface) Local Arbiter.
MT8173 has different local arbiters (larb).
Please see
http://lists.linuxfoundation.org/pipermail/iommu/2015-October/014587.html
Video Encoder HW has it's local arbiter (larb), by configure ports in
this larb, we can have encoder HW go through the m4u to talk with EMI.
We will change it to mediatek,larb in next version.
>
> > +- clocks : list of clock specifiers, corresponding to entries in
> > + the clock-names property;
> > +- clock-names: must contain "vencpll", "venc_lt_sel", "vcodecpll_370p5_ck"
> > +- iommus : list of iommus specifiers should be enabled for hw encode.
> > + There are 2 cells needed to enable/disable iommu.
> > + The first one is local arbiter index(larbid), and the other is port
> > + index(portid) within local arbiter. Specifies the larbid and portid
> > + as defined in dt-binding/memory/mt8173-larb-port.h.
> > +- vpu : the node of video processor unit
>
> This should be prefixed with mediatek.
We will prefix it with mediatek in next version.
>
> > +
> > +Example:
> > +vcodec_enc: vcodec@0x18002000 {
> > + compatible = "mediatek,mt8173-vcodec-enc";
> > + reg = <0 0x18002000 0 0x1000>, /*VENC_SYS*/
> > + <0 0x19002000 0 0x1000>; /*VENC_LT_SYS*/
> > + interrupts = <GIC_SPI 198 IRQ_TYPE_LEVEL_LOW>,
> > + <GIC_SPI 202 IRQ_TYPE_LEVEL_LOW>;
> > + larb = <&larb3>,
> > + <&larb5>;
> > + iommus = <&iommu M4U_LARB3_ID M4U_PORT_VENC_RCPU>,
> > + <&iommu M4U_LARB3_ID M4U_PORT_VENC_REC>,
> > + <&iommu M4U_LARB3_ID M4U_PORT_VENC_BSDMA>,
> > + <&iommu M4U_LARB3_ID M4U_PORT_VENC_SV_COMV>,
> > + <&iommu M4U_LARB3_ID M4U_PORT_VENC_RD_COMV>,
> > + <&iommu M4U_LARB3_ID M4U_PORT_VENC_CUR_LUMA>,
> > + <&iommu M4U_LARB3_ID M4U_PORT_VENC_CUR_CHROMA>,
> > + <&iommu M4U_LARB3_ID M4U_PORT_VENC_REF_LUMA>,
> > + <&iommu M4U_LARB3_ID M4U_PORT_VENC_REF_CHROMA>,
> > + <&iommu M4U_LARB3_ID M4U_PORT_VENC_NBM_RDMA>,
> > + <&iommu M4U_LARB3_ID M4U_PORT_VENC_NBM_WDMA>,
> > + <&iommu M4U_LARB5_ID M4U_PORT_VENC_RCPU_SET2>,
> > + <&iommu M4U_LARB5_ID M4U_PORT_VENC_REC_FRM_SET2>,
> > + <&iommu M4U_LARB5_ID M4U_PORT_VENC_BSDMA_SET2>,
> > + <&iommu M4U_LARB5_ID M4U_PORT_VENC_SV_COMA_SET2>,
> > + <&iommu M4U_LARB5_ID M4U_PORT_VENC_RD_COMA_SET2>,
> > + <&iommu M4U_LARB5_ID M4U_PORT_VENC_CUR_LUMA_SET2>,
> > + <&iommu M4U_LARB5_ID M4U_PORT_VENC_CUR_CHROMA_SET2>,
> > + <&iommu M4U_LARB5_ID M4U_PORT_VENC_REF_LUMA_SET2>,
> > + <&iommu M4U_LARB5_ID M4U_PORT_VENC_REC_CHROMA_SET2>;
> > + vpu = <&vpu>;
> > + clocks = <&apmixedsys CLK_APMIXED_VENCPLL>,
> > + <&topckgen CLK_TOP_VENC_LT_SEL>,
> > + <&topckgen CLK_TOP_VCODECPLL_370P5>;
> > + clock-names = "vencpll",
> > + "venc_lt_sel",
> > + "vcodecpll_370p5_ck";
> > + };
> > --
> > 1.7.9.5
> >
best regards,
Tiffany
^ permalink raw reply [flat|nested] 84+ messages in thread
* Re: [RESEND RFC/PATCH 4/8] dt-bindings: Add a binding for Mediatek Video Encoder
@ 2015-11-18 7:09 ` tiffany lin
0 siblings, 0 replies; 84+ messages in thread
From: tiffany lin @ 2015-11-18 7:09 UTC (permalink / raw)
To: Rob Herring
Cc: Pawel Moll, Mark Rutland, Ian Campbell, Kumar Gala,
Catalin Marinas, Will Deacon, Mauro Carvalho Chehab,
Matthias Brugger, Daniel Kurtz, Sascha Hauer, Hongzhou Yang,
Hans Verkuil, Laurent Pinchart, Sakari Ailus, Mikhail Ulyanov,
Fabien Dessenne, Arnd Bergmann, Darren Etheridge, Peter Griffin,
Benoit Parrot, Andrew-CT Chen, Eddie Huang
Hi Rob,
On Tue, 2015-11-17 at 13:41 -0600, Rob Herring wrote:
> On Tue, Nov 17, 2015 at 08:54:41PM +0800, Tiffany Lin wrote:
> > add a DT binding documentation of Video Encoder for the
> > MT8173 SoC from Mediatek.
> >
> > Signed-off-by: Tiffany Lin <tiffany.lin-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
> > ---
> > .../devicetree/bindings/media/mediatek-vcodec.txt | 58 ++++++++++++++++++++
> > 1 file changed, 58 insertions(+)
> > create mode 100644 Documentation/devicetree/bindings/media/mediatek-vcodec.txt
> >
> > diff --git a/Documentation/devicetree/bindings/media/mediatek-vcodec.txt b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
> > new file mode 100644
> > index 0000000..fea4d7c
> > --- /dev/null
> > +++ b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
> > @@ -0,0 +1,58 @@
> > +Mediatek Video Codec
> > +
> > +Mediatek Video Codec is the video codec hw present in Mediatek SoCs which
> > +supports high resolution encoding functionalities.
> > +
> > +Required properties:
> > +- compatible : "mediatek,mt8173-vcodec-enc" for encoder
> > +- reg : Physical base address of the video codec registers and length of
> > + memory mapped region.
> > +- interrupts : interrupt number to the cpu.
> > +- larb : must contain the larbes of current platform
>
> What is this?
Resend due to previous mail has html format and reject by some mail
servers.
This is SMI (Smart Multimedia Interface) Local Arbiter.
MT8173 has different local arbiters (larb).
Please see
http://lists.linuxfoundation.org/pipermail/iommu/2015-October/014587.html
Video Encoder HW has it's local arbiter (larb), by configure ports in
this larb, we can have encoder HW go through the m4u to talk with EMI.
We will change it to mediatek,larb in next version.
>
> > +- clocks : list of clock specifiers, corresponding to entries in
> > + the clock-names property;
> > +- clock-names: must contain "vencpll", "venc_lt_sel", "vcodecpll_370p5_ck"
> > +- iommus : list of iommus specifiers should be enabled for hw encode.
> > + There are 2 cells needed to enable/disable iommu.
> > + The first one is local arbiter index(larbid), and the other is port
> > + index(portid) within local arbiter. Specifies the larbid and portid
> > + as defined in dt-binding/memory/mt8173-larb-port.h.
> > +- vpu : the node of video processor unit
>
> This should be prefixed with mediatek.
We will prefix it with mediatek in next version.
>
> > +
> > +Example:
> > +vcodec_enc: vcodec@0x18002000 {
> > + compatible = "mediatek,mt8173-vcodec-enc";
> > + reg = <0 0x18002000 0 0x1000>, /*VENC_SYS*/
> > + <0 0x19002000 0 0x1000>; /*VENC_LT_SYS*/
> > + interrupts = <GIC_SPI 198 IRQ_TYPE_LEVEL_LOW>,
> > + <GIC_SPI 202 IRQ_TYPE_LEVEL_LOW>;
> > + larb = <&larb3>,
> > + <&larb5>;
> > + iommus = <&iommu M4U_LARB3_ID M4U_PORT_VENC_RCPU>,
> > + <&iommu M4U_LARB3_ID M4U_PORT_VENC_REC>,
> > + <&iommu M4U_LARB3_ID M4U_PORT_VENC_BSDMA>,
> > + <&iommu M4U_LARB3_ID M4U_PORT_VENC_SV_COMV>,
> > + <&iommu M4U_LARB3_ID M4U_PORT_VENC_RD_COMV>,
> > + <&iommu M4U_LARB3_ID M4U_PORT_VENC_CUR_LUMA>,
> > + <&iommu M4U_LARB3_ID M4U_PORT_VENC_CUR_CHROMA>,
> > + <&iommu M4U_LARB3_ID M4U_PORT_VENC_REF_LUMA>,
> > + <&iommu M4U_LARB3_ID M4U_PORT_VENC_REF_CHROMA>,
> > + <&iommu M4U_LARB3_ID M4U_PORT_VENC_NBM_RDMA>,
> > + <&iommu M4U_LARB3_ID M4U_PORT_VENC_NBM_WDMA>,
> > + <&iommu M4U_LARB5_ID M4U_PORT_VENC_RCPU_SET2>,
> > + <&iommu M4U_LARB5_ID M4U_PORT_VENC_REC_FRM_SET2>,
> > + <&iommu M4U_LARB5_ID M4U_PORT_VENC_BSDMA_SET2>,
> > + <&iommu M4U_LARB5_ID M4U_PORT_VENC_SV_COMA_SET2>,
> > + <&iommu M4U_LARB5_ID M4U_PORT_VENC_RD_COMA_SET2>,
> > + <&iommu M4U_LARB5_ID M4U_PORT_VENC_CUR_LUMA_SET2>,
> > + <&iommu M4U_LARB5_ID M4U_PORT_VENC_CUR_CHROMA_SET2>,
> > + <&iommu M4U_LARB5_ID M4U_PORT_VENC_REF_LUMA_SET2>,
> > + <&iommu M4U_LARB5_ID M4U_PORT_VENC_REC_CHROMA_SET2>;
> > + vpu = <&vpu>;
> > + clocks = <&apmixedsys CLK_APMIXED_VENCPLL>,
> > + <&topckgen CLK_TOP_VENC_LT_SEL>,
> > + <&topckgen CLK_TOP_VCODECPLL_370P5>;
> > + clock-names = "vencpll",
> > + "venc_lt_sel",
> > + "vcodecpll_370p5_ck";
> > + };
> > --
> > 1.7.9.5
> >
best regards,
Tiffany
--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 84+ messages in thread
* [RESEND RFC/PATCH 4/8] dt-bindings: Add a binding for Mediatek Video Encoder
@ 2015-11-18 7:09 ` tiffany lin
0 siblings, 0 replies; 84+ messages in thread
From: tiffany lin @ 2015-11-18 7:09 UTC (permalink / raw)
To: linux-arm-kernel
Hi Rob,
On Tue, 2015-11-17 at 13:41 -0600, Rob Herring wrote:
> On Tue, Nov 17, 2015 at 08:54:41PM +0800, Tiffany Lin wrote:
> > add a DT binding documentation of Video Encoder for the
> > MT8173 SoC from Mediatek.
> >
> > Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
> > ---
> > .../devicetree/bindings/media/mediatek-vcodec.txt | 58 ++++++++++++++++++++
> > 1 file changed, 58 insertions(+)
> > create mode 100644 Documentation/devicetree/bindings/media/mediatek-vcodec.txt
> >
> > diff --git a/Documentation/devicetree/bindings/media/mediatek-vcodec.txt b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
> > new file mode 100644
> > index 0000000..fea4d7c
> > --- /dev/null
> > +++ b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
> > @@ -0,0 +1,58 @@
> > +Mediatek Video Codec
> > +
> > +Mediatek Video Codec is the video codec hw present in Mediatek SoCs which
> > +supports high resolution encoding functionalities.
> > +
> > +Required properties:
> > +- compatible : "mediatek,mt8173-vcodec-enc" for encoder
> > +- reg : Physical base address of the video codec registers and length of
> > + memory mapped region.
> > +- interrupts : interrupt number to the cpu.
> > +- larb : must contain the larbes of current platform
>
> What is this?
Resend due to previous mail has html format and reject by some mail
servers.
This is SMI (Smart Multimedia Interface) Local Arbiter.
MT8173 has different local arbiters (larb).
Please see
http://lists.linuxfoundation.org/pipermail/iommu/2015-October/014587.html
Video Encoder HW has it's local arbiter (larb), by configure ports in
this larb, we can have encoder HW go through the m4u to talk with EMI.
We will change it to mediatek,larb in next version.
>
> > +- clocks : list of clock specifiers, corresponding to entries in
> > + the clock-names property;
> > +- clock-names: must contain "vencpll", "venc_lt_sel", "vcodecpll_370p5_ck"
> > +- iommus : list of iommus specifiers should be enabled for hw encode.
> > + There are 2 cells needed to enable/disable iommu.
> > + The first one is local arbiter index(larbid), and the other is port
> > + index(portid) within local arbiter. Specifies the larbid and portid
> > + as defined in dt-binding/memory/mt8173-larb-port.h.
> > +- vpu : the node of video processor unit
>
> This should be prefixed with mediatek.
We will prefix it with mediatek in next version.
>
> > +
> > +Example:
> > +vcodec_enc: vcodec at 0x18002000 {
> > + compatible = "mediatek,mt8173-vcodec-enc";
> > + reg = <0 0x18002000 0 0x1000>, /*VENC_SYS*/
> > + <0 0x19002000 0 0x1000>; /*VENC_LT_SYS*/
> > + interrupts = <GIC_SPI 198 IRQ_TYPE_LEVEL_LOW>,
> > + <GIC_SPI 202 IRQ_TYPE_LEVEL_LOW>;
> > + larb = <&larb3>,
> > + <&larb5>;
> > + iommus = <&iommu M4U_LARB3_ID M4U_PORT_VENC_RCPU>,
> > + <&iommu M4U_LARB3_ID M4U_PORT_VENC_REC>,
> > + <&iommu M4U_LARB3_ID M4U_PORT_VENC_BSDMA>,
> > + <&iommu M4U_LARB3_ID M4U_PORT_VENC_SV_COMV>,
> > + <&iommu M4U_LARB3_ID M4U_PORT_VENC_RD_COMV>,
> > + <&iommu M4U_LARB3_ID M4U_PORT_VENC_CUR_LUMA>,
> > + <&iommu M4U_LARB3_ID M4U_PORT_VENC_CUR_CHROMA>,
> > + <&iommu M4U_LARB3_ID M4U_PORT_VENC_REF_LUMA>,
> > + <&iommu M4U_LARB3_ID M4U_PORT_VENC_REF_CHROMA>,
> > + <&iommu M4U_LARB3_ID M4U_PORT_VENC_NBM_RDMA>,
> > + <&iommu M4U_LARB3_ID M4U_PORT_VENC_NBM_WDMA>,
> > + <&iommu M4U_LARB5_ID M4U_PORT_VENC_RCPU_SET2>,
> > + <&iommu M4U_LARB5_ID M4U_PORT_VENC_REC_FRM_SET2>,
> > + <&iommu M4U_LARB5_ID M4U_PORT_VENC_BSDMA_SET2>,
> > + <&iommu M4U_LARB5_ID M4U_PORT_VENC_SV_COMA_SET2>,
> > + <&iommu M4U_LARB5_ID M4U_PORT_VENC_RD_COMA_SET2>,
> > + <&iommu M4U_LARB5_ID M4U_PORT_VENC_CUR_LUMA_SET2>,
> > + <&iommu M4U_LARB5_ID M4U_PORT_VENC_CUR_CHROMA_SET2>,
> > + <&iommu M4U_LARB5_ID M4U_PORT_VENC_REF_LUMA_SET2>,
> > + <&iommu M4U_LARB5_ID M4U_PORT_VENC_REC_CHROMA_SET2>;
> > + vpu = <&vpu>;
> > + clocks = <&apmixedsys CLK_APMIXED_VENCPLL>,
> > + <&topckgen CLK_TOP_VENC_LT_SEL>,
> > + <&topckgen CLK_TOP_VCODECPLL_370P5>;
> > + clock-names = "vencpll",
> > + "venc_lt_sel",
> > + "vcodecpll_370p5_ck";
> > + };
> > --
> > 1.7.9.5
> >
best regards,
Tiffany
^ permalink raw reply [flat|nested] 84+ messages in thread
* Re: [RESEND RFC/PATCH 1/8] dt-bindings: Add a binding for Mediatek Video Processor Unit
2015-11-17 14:13 ` Mark Rutland
(?)
@ 2015-11-19 2:47 ` andrew-ct chen
-1 siblings, 0 replies; 84+ messages in thread
From: andrew-ct chen @ 2015-11-19 2:47 UTC (permalink / raw)
To: Mark Rutland
Cc: Tiffany Lin, Rob Herring, Pawel Moll, Ian Campbell, Kumar Gala,
Catalin Marinas, Will Deacon, Mauro Carvalho Chehab,
Matthias Brugger, Daniel Kurtz, Sascha Hauer, Hongzhou Yang,
Hans Verkuil, Laurent Pinchart, Sakari Ailus, Geert Uytterhoeven,
Mikhail Ulyanov, Fabien Dessenne, Arnd Bergmann,
Darren Etheridge, Peter Griffin, Benoit Parrot, Eddie Huang,
Yingjoe Chen, James Liao, Daniel Hsiao, devicetree, linux-kernel,
linux-arm-kernel, linux-media, linux-mediatek
On Tue, 2015-11-17 at 14:13 +0000, Mark Rutland wrote:
> On Tue, Nov 17, 2015 at 08:54:38PM +0800, Tiffany Lin wrote:
> > From: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
> >
> > Add a DT binding documentation of Video Processor Unit for the
> > MT8173 SoC from Mediatek.
> >
> > Signed-off-by: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
> > ---
> > .../devicetree/bindings/media/mediatek-vpu.txt | 27 ++++++++++++++++++++
> > 1 file changed, 27 insertions(+)
> > create mode 100644 Documentation/devicetree/bindings/media/mediatek-vpu.txt
> >
> > diff --git a/Documentation/devicetree/bindings/media/mediatek-vpu.txt b/Documentation/devicetree/bindings/media/mediatek-vpu.txt
> > new file mode 100644
> > index 0000000..99a4e5e
> > --- /dev/null
> > +++ b/Documentation/devicetree/bindings/media/mediatek-vpu.txt
> > @@ -0,0 +1,27 @@
> > +* Mediatek Video Processor Unit
> > +
> > +Video Processor Unit is a HW video controller. It controls HW Codec including
> > +H.264/VP8/VP9 Decode, H.264/VP8 Encode and Image Processor (scale/rotate/color convert).
> > +
> > +Required properties:
> > + - compatible: "mediatek,mt8173-vpu"
> > + - reg: Must contain an entry for each entry in reg-names.
> > + - reg-names: Must include the following entries:
> > + "sram": SRAM base
> > + "cfg_reg": Main configuration registers base
> > + - interrupts: interrupt number to the cpu.
> > + - clocks : clock name from clock manager
> > + - clock-names: the clocks of the VPU H/W
>
> You need to explicitly define the set of clock-names you expect here.
>
> Mark.
Sorry, only one clock to enable VPU hardware.
We will modify to
- clocks : must contain one entry for each clock-names.
- clock-names : must be "main", It is the main clock of VPU.
Thanks..
>
> > + - iommus : phandle and IOMMU spcifier for the IOMMU that serves the VPU.
> > +
> > +Example:
> > + vpu: vpu@10020000 {
> > + compatible = "mediatek,mt8173-vpu";
> > + reg = <0 0x10020000 0 0x30000>,
> > + <0 0x10050000 0 0x100>;
> > + reg-names = "sram", "cfg_reg";
> > + interrupts = <GIC_SPI 166 IRQ_TYPE_LEVEL_HIGH>;
> > + clocks = <&topckgen TOP_SCP_SEL>;
> > + clock-names = "main";
> > + iommus = <&iommu M4U_PORT_VENC_RCPU>;
> > + };
> > --
> > 1.7.9.5
> >
^ permalink raw reply [flat|nested] 84+ messages in thread
* Re: [RESEND RFC/PATCH 1/8] dt-bindings: Add a binding for Mediatek Video Processor Unit
@ 2015-11-19 2:47 ` andrew-ct chen
0 siblings, 0 replies; 84+ messages in thread
From: andrew-ct chen @ 2015-11-19 2:47 UTC (permalink / raw)
To: Mark Rutland
Cc: Tiffany Lin, Rob Herring, Pawel Moll, Ian Campbell, Kumar Gala,
Catalin Marinas, Will Deacon, Mauro Carvalho Chehab,
Matthias Brugger, Daniel Kurtz, Sascha Hauer, Hongzhou Yang,
Hans Verkuil, Laurent Pinchart, Sakari Ailus, Geert Uytterhoeven,
Mikhail Ulyanov, Fabien Dessenne, Arnd Bergmann,
Darren Etheridge, Peter Griffin, Benoit
On Tue, 2015-11-17 at 14:13 +0000, Mark Rutland wrote:
> On Tue, Nov 17, 2015 at 08:54:38PM +0800, Tiffany Lin wrote:
> > From: Andrew-CT Chen <andrew-ct.chen-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
> >
> > Add a DT binding documentation of Video Processor Unit for the
> > MT8173 SoC from Mediatek.
> >
> > Signed-off-by: Andrew-CT Chen <andrew-ct.chen-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
> > ---
> > .../devicetree/bindings/media/mediatek-vpu.txt | 27 ++++++++++++++++++++
> > 1 file changed, 27 insertions(+)
> > create mode 100644 Documentation/devicetree/bindings/media/mediatek-vpu.txt
> >
> > diff --git a/Documentation/devicetree/bindings/media/mediatek-vpu.txt b/Documentation/devicetree/bindings/media/mediatek-vpu.txt
> > new file mode 100644
> > index 0000000..99a4e5e
> > --- /dev/null
> > +++ b/Documentation/devicetree/bindings/media/mediatek-vpu.txt
> > @@ -0,0 +1,27 @@
> > +* Mediatek Video Processor Unit
> > +
> > +Video Processor Unit is a HW video controller. It controls HW Codec including
> > +H.264/VP8/VP9 Decode, H.264/VP8 Encode and Image Processor (scale/rotate/color convert).
> > +
> > +Required properties:
> > + - compatible: "mediatek,mt8173-vpu"
> > + - reg: Must contain an entry for each entry in reg-names.
> > + - reg-names: Must include the following entries:
> > + "sram": SRAM base
> > + "cfg_reg": Main configuration registers base
> > + - interrupts: interrupt number to the cpu.
> > + - clocks : clock name from clock manager
> > + - clock-names: the clocks of the VPU H/W
>
> You need to explicitly define the set of clock-names you expect here.
>
> Mark.
Sorry, only one clock to enable VPU hardware.
We will modify to
- clocks : must contain one entry for each clock-names.
- clock-names : must be "main", It is the main clock of VPU.
Thanks..
>
> > + - iommus : phandle and IOMMU spcifier for the IOMMU that serves the VPU.
> > +
> > +Example:
> > + vpu: vpu@10020000 {
> > + compatible = "mediatek,mt8173-vpu";
> > + reg = <0 0x10020000 0 0x30000>,
> > + <0 0x10050000 0 0x100>;
> > + reg-names = "sram", "cfg_reg";
> > + interrupts = <GIC_SPI 166 IRQ_TYPE_LEVEL_HIGH>;
> > + clocks = <&topckgen TOP_SCP_SEL>;
> > + clock-names = "main";
> > + iommus = <&iommu M4U_PORT_VENC_RCPU>;
> > + };
> > --
> > 1.7.9.5
> >
--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 84+ messages in thread
* [RESEND RFC/PATCH 1/8] dt-bindings: Add a binding for Mediatek Video Processor Unit
@ 2015-11-19 2:47 ` andrew-ct chen
0 siblings, 0 replies; 84+ messages in thread
From: andrew-ct chen @ 2015-11-19 2:47 UTC (permalink / raw)
To: linux-arm-kernel
On Tue, 2015-11-17 at 14:13 +0000, Mark Rutland wrote:
> On Tue, Nov 17, 2015 at 08:54:38PM +0800, Tiffany Lin wrote:
> > From: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
> >
> > Add a DT binding documentation of Video Processor Unit for the
> > MT8173 SoC from Mediatek.
> >
> > Signed-off-by: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
> > ---
> > .../devicetree/bindings/media/mediatek-vpu.txt | 27 ++++++++++++++++++++
> > 1 file changed, 27 insertions(+)
> > create mode 100644 Documentation/devicetree/bindings/media/mediatek-vpu.txt
> >
> > diff --git a/Documentation/devicetree/bindings/media/mediatek-vpu.txt b/Documentation/devicetree/bindings/media/mediatek-vpu.txt
> > new file mode 100644
> > index 0000000..99a4e5e
> > --- /dev/null
> > +++ b/Documentation/devicetree/bindings/media/mediatek-vpu.txt
> > @@ -0,0 +1,27 @@
> > +* Mediatek Video Processor Unit
> > +
> > +Video Processor Unit is a HW video controller. It controls HW Codec including
> > +H.264/VP8/VP9 Decode, H.264/VP8 Encode and Image Processor (scale/rotate/color convert).
> > +
> > +Required properties:
> > + - compatible: "mediatek,mt8173-vpu"
> > + - reg: Must contain an entry for each entry in reg-names.
> > + - reg-names: Must include the following entries:
> > + "sram": SRAM base
> > + "cfg_reg": Main configuration registers base
> > + - interrupts: interrupt number to the cpu.
> > + - clocks : clock name from clock manager
> > + - clock-names: the clocks of the VPU H/W
>
> You need to explicitly define the set of clock-names you expect here.
>
> Mark.
Sorry, only one clock to enable VPU hardware.
We will modify to
- clocks : must contain one entry for each clock-names.
- clock-names : must be "main", It is the main clock of VPU.
Thanks..
>
> > + - iommus : phandle and IOMMU spcifier for the IOMMU that serves the VPU.
> > +
> > +Example:
> > + vpu: vpu at 10020000 {
> > + compatible = "mediatek,mt8173-vpu";
> > + reg = <0 0x10020000 0 0x30000>,
> > + <0 0x10050000 0 0x100>;
> > + reg-names = "sram", "cfg_reg";
> > + interrupts = <GIC_SPI 166 IRQ_TYPE_LEVEL_HIGH>;
> > + clocks = <&topckgen TOP_SCP_SEL>;
> > + clock-names = "main";
> > + iommus = <&iommu M4U_PORT_VENC_RCPU>;
> > + };
> > --
> > 1.7.9.5
> >
^ permalink raw reply [flat|nested] 84+ messages in thread
* Re: [RESEND RFC/PATCH 0/8] Add MT8173 Video Encoder Driver and VPU Driver
@ 2015-11-19 7:40 ` Hans Verkuil
0 siblings, 0 replies; 84+ messages in thread
From: Hans Verkuil @ 2015-11-19 7:40 UTC (permalink / raw)
To: Tiffany Lin, Rob Herring, Pawel Moll, Mark Rutland, Ian Campbell,
Kumar Gala, Catalin Marinas, Will Deacon, Mauro Carvalho Chehab,
Matthias Brugger, Daniel Kurtz, Sascha Hauer, Hongzhou Yang,
Hans Verkuil, Laurent Pinchart, Sakari Ailus, Geert Uytterhoeven,
Mikhail Ulyanov, Fabien Dessenne, Arnd Bergmann,
Darren Etheridge, Peter Griffin, Benoit Parrot
Cc: Andrew-CT Chen, Eddie Huang, Yingjoe Chen, James Liao,
Daniel Hsiao, devicetree, linux-kernel, linux-arm-kernel,
linux-media, linux-mediatek
On 11/17/2015 01:54 PM, Tiffany Lin wrote:
> ==============
> Introduction
> ==============
>
> The purpose of this RFC is to discuss the driver for a hw video codec
> embedded in the Mediatek's MT8173 SoCs. Mediatek Video Codec is able to
> handle video encoding of in a range of formats.
>
> This RFC also include VPU driver. Mediatek Video Codec driver rely on
> VPU driver to load, communicate with VPU.
>
> Internally the driver uses videobuf2 framework and MTK IOMMU and MTK SMI.
> MTK IOMMU and MTK SMI have not yet been merged, but we wanted to start
> discussion about the driver earlier so it could be merged sooner. The
> driver posted here is the initial version, so I suppose it will require
> more work.
I plan on reviewing this patch series (at least the non-dt parts). It's busy,
though, and I don't know exactly when I get the chance. But just so you know
that someone will be reviewing it.
Regards,
Hans
>
> [1]http://lists.infradead.org/pipermail/linux-mediatek/2015-October/002525.html
>
> ==================
> Device interface
> ==================
>
> In principle the driver bases on memory-to-memory framework:
> it provides a single video node and each opened file handle gets its own
> private context with separate buffer queues. Each context consist of 2
> buffer queues: OUTPUT (for source buffers, i.e. raw video frames)
> and CAPTURE (for destination buffers, i.e. encoded video frames).
>
> The process of encoding video data from stream is a bit more complicated
> than typical memory-to-memory processing. We base on memory-to-memory
> framework and add the complicated part in our vb2 and v4l2 callback
> functionss. So we can base on well done m2m memory-to-memory framework,
> reduce duplicate code and make our driver code simple.
>
> ==============================
> VPU (Video Processor Unit)
> ==============================
> The VPU driver for hw video codec embedded in Mediatek's MT8173 SOCs.
> It is able to handle video decoding/encoding of in a range of formats.
> The driver provides with VPU firmware download, memory management and
> the communication interface between CPU and VPU.
> For VPU initialization, it will create virtual memory for CPU access and
> IOMMU address for vcodec hw device access. When a decode/encode instance
> opens a device node, vpu driver will download vpu firmware to the device.
> A decode/encode instant will decode/encode a frame using VPU
> interface to interrupt vpu to handle decoding/encoding jobs.
>
> Please have a look at the code and comments will be very much appreciated.
>
> Andrew-CT Chen (3):
> dt-bindings: Add a binding for Mediatek Video Processor Unit
> arm64: dts: mediatek: Add node for Mediatek Video Processor Unit
> media: platform: mtk-vpu: Support Mediatek VPU
>
> Daniel Hsiao (1):
> media: platform: mtk-vcodec: Add Mediatek VP8 Video Encoder Driver
>
> Tiffany Lin (4):
> dt-bindings: Add a binding for Mediatek Video Encoder
> arm64: dts: mediatek: Add Video Encoder for MT8173
> media: platform: mtk-vcodec: Add Mediatek V4L2 Video Encoder Driver
> media: platform: mtk-vcodec: Add Mediatek H264 Video Encoder Driver
>
> .../devicetree/bindings/media/mediatek-vcodec.txt | 58 +
> .../devicetree/bindings/media/mediatek-vpu.txt | 27 +
> arch/arm64/boot/dts/mediatek/mt8173.dtsi | 58 +
> drivers/media/platform/Kconfig | 19 +
> drivers/media/platform/Makefile | 5 +
> drivers/media/platform/mtk-vcodec/Kconfig | 5 +
> drivers/media/platform/mtk-vcodec/Makefile | 12 +
> drivers/media/platform/mtk-vcodec/common/Makefile | 12 +
> .../media/platform/mtk-vcodec/common/venc_drv_if.c | 159 ++
> .../media/platform/mtk-vcodec/h264_enc/Makefile | 9 +
> .../platform/mtk-vcodec/h264_enc/venc_h264_if.c | 529 ++++++
> .../platform/mtk-vcodec/h264_enc/venc_h264_if.h | 53 +
> .../platform/mtk-vcodec/h264_enc/venc_h264_vpu.c | 341 ++++
> .../platform/mtk-vcodec/include/venc_drv_base.h | 68 +
> .../platform/mtk-vcodec/include/venc_drv_if.h | 187 +++
> .../platform/mtk-vcodec/include/venc_ipi_msg.h | 212 +++
> drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h | 441 +++++
> drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c | 1773 ++++++++++++++++++++
> drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h | 28 +
> .../media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c | 535 ++++++
> .../media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c | 122 ++
> .../media/platform/mtk-vcodec/mtk_vcodec_intr.c | 110 ++
> .../media/platform/mtk-vcodec/mtk_vcodec_intr.h | 30 +
> drivers/media/platform/mtk-vcodec/mtk_vcodec_pm.h | 26 +
> .../media/platform/mtk-vcodec/mtk_vcodec_util.c | 106 ++
> .../media/platform/mtk-vcodec/mtk_vcodec_util.h | 66 +
> drivers/media/platform/mtk-vcodec/vp8_enc/Makefile | 9 +
> .../platform/mtk-vcodec/vp8_enc/venc_vp8_if.c | 371 ++++
> .../platform/mtk-vcodec/vp8_enc/venc_vp8_if.h | 48 +
> .../platform/mtk-vcodec/vp8_enc/venc_vp8_vpu.c | 245 +++
> drivers/media/platform/mtk-vpu/Makefile | 1 +
> .../platform/mtk-vpu/h264_enc/venc_h264_vpu.h | 127 ++
> .../media/platform/mtk-vpu/include/venc_ipi_msg.h | 212 +++
> drivers/media/platform/mtk-vpu/mtk_vpu_core.c | 823 +++++++++
> drivers/media/platform/mtk-vpu/mtk_vpu_core.h | 161 ++
> .../media/platform/mtk-vpu/vp8_enc/venc_vp8_vpu.h | 119 ++
> 36 files changed, 7107 insertions(+)
> create mode 100644 Documentation/devicetree/bindings/media/mediatek-vcodec.txt
> create mode 100644 Documentation/devicetree/bindings/media/mediatek-vpu.txt
> create mode 100644 drivers/media/platform/mtk-vcodec/Kconfig
> create mode 100644 drivers/media/platform/mtk-vcodec/Makefile
> create mode 100644 drivers/media/platform/mtk-vcodec/common/Makefile
> create mode 100644 drivers/media/platform/mtk-vcodec/common/venc_drv_if.c
> create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/Makefile
> create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.c
> create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.h
> create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.c
> create mode 100644 drivers/media/platform/mtk-vcodec/include/venc_drv_base.h
> create mode 100644 drivers/media/platform/mtk-vcodec/include/venc_drv_if.h
> create mode 100644 drivers/media/platform/mtk-vcodec/include/venc_ipi_msg.h
> create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
> create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
> create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h
> create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
> create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
> create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.c
> create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.h
> create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_pm.h
> create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c
> create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
> create mode 100644 drivers/media/platform/mtk-vcodec/vp8_enc/Makefile
> create mode 100644 drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_if.c
> create mode 100644 drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_if.h
> create mode 100644 drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_vpu.c
> create mode 100644 drivers/media/platform/mtk-vpu/Makefile
> create mode 100644 drivers/media/platform/mtk-vpu/h264_enc/venc_h264_vpu.h
> create mode 100644 drivers/media/platform/mtk-vpu/include/venc_ipi_msg.h
> create mode 100644 drivers/media/platform/mtk-vpu/mtk_vpu_core.c
> create mode 100644 drivers/media/platform/mtk-vpu/mtk_vpu_core.h
> create mode 100644 drivers/media/platform/mtk-vpu/vp8_enc/venc_vp8_vpu.h
>
^ permalink raw reply [flat|nested] 84+ messages in thread
* Re: [RESEND RFC/PATCH 0/8] Add MT8173 Video Encoder Driver and VPU Driver
@ 2015-11-19 7:40 ` Hans Verkuil
0 siblings, 0 replies; 84+ messages in thread
From: Hans Verkuil @ 2015-11-19 7:40 UTC (permalink / raw)
To: Tiffany Lin, Rob Herring, Pawel Moll, Mark Rutland, Ian Campbell,
Kumar Gala, Catalin Marinas, Will Deacon, Mauro Carvalho Chehab,
Matthias Brugger, Daniel Kurtz, Sascha Hauer, Hongzhou Yang,
Hans Verkuil, Laurent Pinchart, Sakari Ailus, Geert Uytterhoeven,
Mikhail Ulyanov, Fabien Dessenne, Arnd Bergmann,
Darren Etheridge, Peter Griffin
Cc: Andrew-CT Chen, Eddie Huang, Yingjoe Chen, James Liao,
Daniel Hsiao, devicetree-u79uwXL29TY76Z2rM5mHXA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA,
linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
linux-media-u79uwXL29TY76Z2rM5mHXA,
linux-mediatek-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r
On 11/17/2015 01:54 PM, Tiffany Lin wrote:
> ==============
> Introduction
> ==============
>
> The purpose of this RFC is to discuss the driver for a hw video codec
> embedded in the Mediatek's MT8173 SoCs. Mediatek Video Codec is able to
> handle video encoding of in a range of formats.
>
> This RFC also include VPU driver. Mediatek Video Codec driver rely on
> VPU driver to load, communicate with VPU.
>
> Internally the driver uses videobuf2 framework and MTK IOMMU and MTK SMI.
> MTK IOMMU and MTK SMI have not yet been merged, but we wanted to start
> discussion about the driver earlier so it could be merged sooner. The
> driver posted here is the initial version, so I suppose it will require
> more work.
I plan on reviewing this patch series (at least the non-dt parts). It's busy,
though, and I don't know exactly when I get the chance. But just so you know
that someone will be reviewing it.
Regards,
Hans
>
> [1]http://lists.infradead.org/pipermail/linux-mediatek/2015-October/002525.html
>
> ==================
> Device interface
> ==================
>
> In principle the driver bases on memory-to-memory framework:
> it provides a single video node and each opened file handle gets its own
> private context with separate buffer queues. Each context consist of 2
> buffer queues: OUTPUT (for source buffers, i.e. raw video frames)
> and CAPTURE (for destination buffers, i.e. encoded video frames).
>
> The process of encoding video data from stream is a bit more complicated
> than typical memory-to-memory processing. We base on memory-to-memory
> framework and add the complicated part in our vb2 and v4l2 callback
> functionss. So we can base on well done m2m memory-to-memory framework,
> reduce duplicate code and make our driver code simple.
>
> ==============================
> VPU (Video Processor Unit)
> ==============================
> The VPU driver for hw video codec embedded in Mediatek's MT8173 SOCs.
> It is able to handle video decoding/encoding of in a range of formats.
> The driver provides with VPU firmware download, memory management and
> the communication interface between CPU and VPU.
> For VPU initialization, it will create virtual memory for CPU access and
> IOMMU address for vcodec hw device access. When a decode/encode instance
> opens a device node, vpu driver will download vpu firmware to the device.
> A decode/encode instant will decode/encode a frame using VPU
> interface to interrupt vpu to handle decoding/encoding jobs.
>
> Please have a look at the code and comments will be very much appreciated.
>
> Andrew-CT Chen (3):
> dt-bindings: Add a binding for Mediatek Video Processor Unit
> arm64: dts: mediatek: Add node for Mediatek Video Processor Unit
> media: platform: mtk-vpu: Support Mediatek VPU
>
> Daniel Hsiao (1):
> media: platform: mtk-vcodec: Add Mediatek VP8 Video Encoder Driver
>
> Tiffany Lin (4):
> dt-bindings: Add a binding for Mediatek Video Encoder
> arm64: dts: mediatek: Add Video Encoder for MT8173
> media: platform: mtk-vcodec: Add Mediatek V4L2 Video Encoder Driver
> media: platform: mtk-vcodec: Add Mediatek H264 Video Encoder Driver
>
> .../devicetree/bindings/media/mediatek-vcodec.txt | 58 +
> .../devicetree/bindings/media/mediatek-vpu.txt | 27 +
> arch/arm64/boot/dts/mediatek/mt8173.dtsi | 58 +
> drivers/media/platform/Kconfig | 19 +
> drivers/media/platform/Makefile | 5 +
> drivers/media/platform/mtk-vcodec/Kconfig | 5 +
> drivers/media/platform/mtk-vcodec/Makefile | 12 +
> drivers/media/platform/mtk-vcodec/common/Makefile | 12 +
> .../media/platform/mtk-vcodec/common/venc_drv_if.c | 159 ++
> .../media/platform/mtk-vcodec/h264_enc/Makefile | 9 +
> .../platform/mtk-vcodec/h264_enc/venc_h264_if.c | 529 ++++++
> .../platform/mtk-vcodec/h264_enc/venc_h264_if.h | 53 +
> .../platform/mtk-vcodec/h264_enc/venc_h264_vpu.c | 341 ++++
> .../platform/mtk-vcodec/include/venc_drv_base.h | 68 +
> .../platform/mtk-vcodec/include/venc_drv_if.h | 187 +++
> .../platform/mtk-vcodec/include/venc_ipi_msg.h | 212 +++
> drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h | 441 +++++
> drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c | 1773 ++++++++++++++++++++
> drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h | 28 +
> .../media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c | 535 ++++++
> .../media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c | 122 ++
> .../media/platform/mtk-vcodec/mtk_vcodec_intr.c | 110 ++
> .../media/platform/mtk-vcodec/mtk_vcodec_intr.h | 30 +
> drivers/media/platform/mtk-vcodec/mtk_vcodec_pm.h | 26 +
> .../media/platform/mtk-vcodec/mtk_vcodec_util.c | 106 ++
> .../media/platform/mtk-vcodec/mtk_vcodec_util.h | 66 +
> drivers/media/platform/mtk-vcodec/vp8_enc/Makefile | 9 +
> .../platform/mtk-vcodec/vp8_enc/venc_vp8_if.c | 371 ++++
> .../platform/mtk-vcodec/vp8_enc/venc_vp8_if.h | 48 +
> .../platform/mtk-vcodec/vp8_enc/venc_vp8_vpu.c | 245 +++
> drivers/media/platform/mtk-vpu/Makefile | 1 +
> .../platform/mtk-vpu/h264_enc/venc_h264_vpu.h | 127 ++
> .../media/platform/mtk-vpu/include/venc_ipi_msg.h | 212 +++
> drivers/media/platform/mtk-vpu/mtk_vpu_core.c | 823 +++++++++
> drivers/media/platform/mtk-vpu/mtk_vpu_core.h | 161 ++
> .../media/platform/mtk-vpu/vp8_enc/venc_vp8_vpu.h | 119 ++
> 36 files changed, 7107 insertions(+)
> create mode 100644 Documentation/devicetree/bindings/media/mediatek-vcodec.txt
> create mode 100644 Documentation/devicetree/bindings/media/mediatek-vpu.txt
> create mode 100644 drivers/media/platform/mtk-vcodec/Kconfig
> create mode 100644 drivers/media/platform/mtk-vcodec/Makefile
> create mode 100644 drivers/media/platform/mtk-vcodec/common/Makefile
> create mode 100644 drivers/media/platform/mtk-vcodec/common/venc_drv_if.c
> create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/Makefile
> create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.c
> create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.h
> create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.c
> create mode 100644 drivers/media/platform/mtk-vcodec/include/venc_drv_base.h
> create mode 100644 drivers/media/platform/mtk-vcodec/include/venc_drv_if.h
> create mode 100644 drivers/media/platform/mtk-vcodec/include/venc_ipi_msg.h
> create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
> create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
> create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h
> create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
> create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
> create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.c
> create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.h
> create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_pm.h
> create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c
> create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
> create mode 100644 drivers/media/platform/mtk-vcodec/vp8_enc/Makefile
> create mode 100644 drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_if.c
> create mode 100644 drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_if.h
> create mode 100644 drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_vpu.c
> create mode 100644 drivers/media/platform/mtk-vpu/Makefile
> create mode 100644 drivers/media/platform/mtk-vpu/h264_enc/venc_h264_vpu.h
> create mode 100644 drivers/media/platform/mtk-vpu/include/venc_ipi_msg.h
> create mode 100644 drivers/media/platform/mtk-vpu/mtk_vpu_core.c
> create mode 100644 drivers/media/platform/mtk-vpu/mtk_vpu_core.h
> create mode 100644 drivers/media/platform/mtk-vpu/vp8_enc/venc_vp8_vpu.h
>
--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 84+ messages in thread
* [RESEND RFC/PATCH 0/8] Add MT8173 Video Encoder Driver and VPU Driver
@ 2015-11-19 7:40 ` Hans Verkuil
0 siblings, 0 replies; 84+ messages in thread
From: Hans Verkuil @ 2015-11-19 7:40 UTC (permalink / raw)
To: linux-arm-kernel
On 11/17/2015 01:54 PM, Tiffany Lin wrote:
> ==============
> Introduction
> ==============
>
> The purpose of this RFC is to discuss the driver for a hw video codec
> embedded in the Mediatek's MT8173 SoCs. Mediatek Video Codec is able to
> handle video encoding of in a range of formats.
>
> This RFC also include VPU driver. Mediatek Video Codec driver rely on
> VPU driver to load, communicate with VPU.
>
> Internally the driver uses videobuf2 framework and MTK IOMMU and MTK SMI.
> MTK IOMMU and MTK SMI have not yet been merged, but we wanted to start
> discussion about the driver earlier so it could be merged sooner. The
> driver posted here is the initial version, so I suppose it will require
> more work.
I plan on reviewing this patch series (at least the non-dt parts). It's busy,
though, and I don't know exactly when I get the chance. But just so you know
that someone will be reviewing it.
Regards,
Hans
>
> [1]http://lists.infradead.org/pipermail/linux-mediatek/2015-October/002525.html
>
> ==================
> Device interface
> ==================
>
> In principle the driver bases on memory-to-memory framework:
> it provides a single video node and each opened file handle gets its own
> private context with separate buffer queues. Each context consist of 2
> buffer queues: OUTPUT (for source buffers, i.e. raw video frames)
> and CAPTURE (for destination buffers, i.e. encoded video frames).
>
> The process of encoding video data from stream is a bit more complicated
> than typical memory-to-memory processing. We base on memory-to-memory
> framework and add the complicated part in our vb2 and v4l2 callback
> functionss. So we can base on well done m2m memory-to-memory framework,
> reduce duplicate code and make our driver code simple.
>
> ==============================
> VPU (Video Processor Unit)
> ==============================
> The VPU driver for hw video codec embedded in Mediatek's MT8173 SOCs.
> It is able to handle video decoding/encoding of in a range of formats.
> The driver provides with VPU firmware download, memory management and
> the communication interface between CPU and VPU.
> For VPU initialization, it will create virtual memory for CPU access and
> IOMMU address for vcodec hw device access. When a decode/encode instance
> opens a device node, vpu driver will download vpu firmware to the device.
> A decode/encode instant will decode/encode a frame using VPU
> interface to interrupt vpu to handle decoding/encoding jobs.
>
> Please have a look at the code and comments will be very much appreciated.
>
> Andrew-CT Chen (3):
> dt-bindings: Add a binding for Mediatek Video Processor Unit
> arm64: dts: mediatek: Add node for Mediatek Video Processor Unit
> media: platform: mtk-vpu: Support Mediatek VPU
>
> Daniel Hsiao (1):
> media: platform: mtk-vcodec: Add Mediatek VP8 Video Encoder Driver
>
> Tiffany Lin (4):
> dt-bindings: Add a binding for Mediatek Video Encoder
> arm64: dts: mediatek: Add Video Encoder for MT8173
> media: platform: mtk-vcodec: Add Mediatek V4L2 Video Encoder Driver
> media: platform: mtk-vcodec: Add Mediatek H264 Video Encoder Driver
>
> .../devicetree/bindings/media/mediatek-vcodec.txt | 58 +
> .../devicetree/bindings/media/mediatek-vpu.txt | 27 +
> arch/arm64/boot/dts/mediatek/mt8173.dtsi | 58 +
> drivers/media/platform/Kconfig | 19 +
> drivers/media/platform/Makefile | 5 +
> drivers/media/platform/mtk-vcodec/Kconfig | 5 +
> drivers/media/platform/mtk-vcodec/Makefile | 12 +
> drivers/media/platform/mtk-vcodec/common/Makefile | 12 +
> .../media/platform/mtk-vcodec/common/venc_drv_if.c | 159 ++
> .../media/platform/mtk-vcodec/h264_enc/Makefile | 9 +
> .../platform/mtk-vcodec/h264_enc/venc_h264_if.c | 529 ++++++
> .../platform/mtk-vcodec/h264_enc/venc_h264_if.h | 53 +
> .../platform/mtk-vcodec/h264_enc/venc_h264_vpu.c | 341 ++++
> .../platform/mtk-vcodec/include/venc_drv_base.h | 68 +
> .../platform/mtk-vcodec/include/venc_drv_if.h | 187 +++
> .../platform/mtk-vcodec/include/venc_ipi_msg.h | 212 +++
> drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h | 441 +++++
> drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c | 1773 ++++++++++++++++++++
> drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h | 28 +
> .../media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c | 535 ++++++
> .../media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c | 122 ++
> .../media/platform/mtk-vcodec/mtk_vcodec_intr.c | 110 ++
> .../media/platform/mtk-vcodec/mtk_vcodec_intr.h | 30 +
> drivers/media/platform/mtk-vcodec/mtk_vcodec_pm.h | 26 +
> .../media/platform/mtk-vcodec/mtk_vcodec_util.c | 106 ++
> .../media/platform/mtk-vcodec/mtk_vcodec_util.h | 66 +
> drivers/media/platform/mtk-vcodec/vp8_enc/Makefile | 9 +
> .../platform/mtk-vcodec/vp8_enc/venc_vp8_if.c | 371 ++++
> .../platform/mtk-vcodec/vp8_enc/venc_vp8_if.h | 48 +
> .../platform/mtk-vcodec/vp8_enc/venc_vp8_vpu.c | 245 +++
> drivers/media/platform/mtk-vpu/Makefile | 1 +
> .../platform/mtk-vpu/h264_enc/venc_h264_vpu.h | 127 ++
> .../media/platform/mtk-vpu/include/venc_ipi_msg.h | 212 +++
> drivers/media/platform/mtk-vpu/mtk_vpu_core.c | 823 +++++++++
> drivers/media/platform/mtk-vpu/mtk_vpu_core.h | 161 ++
> .../media/platform/mtk-vpu/vp8_enc/venc_vp8_vpu.h | 119 ++
> 36 files changed, 7107 insertions(+)
> create mode 100644 Documentation/devicetree/bindings/media/mediatek-vcodec.txt
> create mode 100644 Documentation/devicetree/bindings/media/mediatek-vpu.txt
> create mode 100644 drivers/media/platform/mtk-vcodec/Kconfig
> create mode 100644 drivers/media/platform/mtk-vcodec/Makefile
> create mode 100644 drivers/media/platform/mtk-vcodec/common/Makefile
> create mode 100644 drivers/media/platform/mtk-vcodec/common/venc_drv_if.c
> create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/Makefile
> create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.c
> create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_if.h
> create mode 100644 drivers/media/platform/mtk-vcodec/h264_enc/venc_h264_vpu.c
> create mode 100644 drivers/media/platform/mtk-vcodec/include/venc_drv_base.h
> create mode 100644 drivers/media/platform/mtk-vcodec/include/venc_drv_if.h
> create mode 100644 drivers/media/platform/mtk-vcodec/include/venc_ipi_msg.h
> create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
> create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
> create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h
> create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
> create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
> create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.c
> create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.h
> create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_pm.h
> create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c
> create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
> create mode 100644 drivers/media/platform/mtk-vcodec/vp8_enc/Makefile
> create mode 100644 drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_if.c
> create mode 100644 drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_if.h
> create mode 100644 drivers/media/platform/mtk-vcodec/vp8_enc/venc_vp8_vpu.c
> create mode 100644 drivers/media/platform/mtk-vpu/Makefile
> create mode 100644 drivers/media/platform/mtk-vpu/h264_enc/venc_h264_vpu.h
> create mode 100644 drivers/media/platform/mtk-vpu/include/venc_ipi_msg.h
> create mode 100644 drivers/media/platform/mtk-vpu/mtk_vpu_core.c
> create mode 100644 drivers/media/platform/mtk-vpu/mtk_vpu_core.h
> create mode 100644 drivers/media/platform/mtk-vpu/vp8_enc/venc_vp8_vpu.h
>
^ permalink raw reply [flat|nested] 84+ messages in thread
* Re: [RESEND RFC/PATCH 3/8] media: platform: mtk-vpu: Support Mediatek VPU
@ 2015-11-25 16:11 ` Daniel Thompson
0 siblings, 0 replies; 84+ messages in thread
From: Daniel Thompson @ 2015-11-25 16:11 UTC (permalink / raw)
To: Tiffany Lin, Rob Herring, Pawel Moll, Mark Rutland, Ian Campbell,
Kumar Gala, Catalin Marinas, Will Deacon, Mauro Carvalho Chehab,
Matthias Brugger, Daniel Kurtz, Sascha Hauer, Hongzhou Yang,
Hans Verkuil, Laurent Pinchart, Sakari Ailus, Geert Uytterhoeven,
Mikhail Ulyanov, Fabien Dessenne, Arnd Bergmann,
Darren Etheridge, Peter Griffin, Benoit Parrot
Cc: Andrew-CT Chen, Eddie Huang, Yingjoe Chen, James Liao,
Daniel Hsiao, devicetree, linux-kernel, linux-arm-kernel,
linux-media, linux-mediatek
On 17/11/15 12:54, Tiffany Lin wrote:
> From: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
>
> The VPU driver for hw video codec embedded in Mediatek's MT8173 SOCs.
> It is able to handle video decoding/encoding of in a range of formats.
> The driver provides with VPU firmware download, memory management and
> the communication interface between CPU and VPU.
> For VPU initialization, it will create virtual memory for CPU access and
> IOMMU address for vcodec hw device access. When a decode/encode instance
> opens a device node, vpu driver will download vpu firmware to the device.
> A decode/encode instant will decode/encode a frame using VPU
> interface to interrupt vpu to handle decoding/encoding jobs.
>
> Signed-off-by: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
> ---
> drivers/media/platform/Kconfig | 6 +
> drivers/media/platform/Makefile | 2 +
> drivers/media/platform/mtk-vpu/Makefile | 1 +
> .../platform/mtk-vpu/h264_enc/venc_h264_vpu.h | 127 +++
> .../media/platform/mtk-vpu/include/venc_ipi_msg.h | 212 +++++
> drivers/media/platform/mtk-vpu/mtk_vpu_core.c | 823 ++++++++++++++++++++
> drivers/media/platform/mtk-vpu/mtk_vpu_core.h | 161 ++++
> .../media/platform/mtk-vpu/vp8_enc/venc_vp8_vpu.h | 119 +++
> 8 files changed, 1451 insertions(+)
> create mode 100644 drivers/media/platform/mtk-vpu/Makefile
> create mode 100644 drivers/media/platform/mtk-vpu/h264_enc/venc_h264_vpu.h
> create mode 100644 drivers/media/platform/mtk-vpu/include/venc_ipi_msg.h
> create mode 100644 drivers/media/platform/mtk-vpu/mtk_vpu_core.c
> create mode 100644 drivers/media/platform/mtk-vpu/mtk_vpu_core.h
> create mode 100644 drivers/media/platform/mtk-vpu/vp8_enc/venc_vp8_vpu.h
>
> diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
> index ccbc974..f98eb47 100644
> --- a/drivers/media/platform/Kconfig
> +++ b/drivers/media/platform/Kconfig
> @@ -148,6 +148,12 @@ config VIDEO_CODA
> Coda is a range of video codec IPs that supports
> H.264, MPEG-4, and other video formats.
>
> +config MEDIATEK_VPU
> + bool "Mediatek Video Processor Unit"
> + ---help---
> + This driver provides downloading firmware vpu and
> + communicating with vpu.
> +
This looks pretty broken.
Shouldn't this option be tristate? Why are there no depends-on or selects?
Also I think the help text could be more descriptive here (and so does
checkpatch ;-) ).
> diff --git a/drivers/media/platform/mtk-vpu/h264_enc/venc_h264_vpu.h b/drivers/media/platform/mtk-vpu/h264_enc/venc_h264_vpu.h
> new file mode 100644
> index 0000000..9c8ebdd
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vpu/h264_enc/venc_h264_vpu.h
Why is this file included in *this* patch? It is not included by
anything in the patch and defines functions that do not exist yet.
> diff --git a/drivers/media/platform/mtk-vpu/include/venc_ipi_msg.h b/drivers/media/platform/mtk-vpu/include/venc_ipi_msg.h
> new file mode 100644
> index 0000000..a345b98
This file also is not included by anything and should, perhaps be
included in a different patch.
> diff --git a/drivers/media/platform/mtk-vpu/mtk_vpu_core.c b/drivers/media/platform/mtk-vpu/mtk_vpu_core.c
> new file mode 100644
> index 0000000..b524dbc
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vpu/mtk_vpu_core.c
> @@ -0,0 +1,823 @@
> +/*
> +* Copyright (c) 2015 MediaTek Inc.
> +* Author: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
> +*
> +* This program is free software; you can redistribute it and/or modify
> +* it under the terms of the GNU General Public License version 2 as
> +* published by the Free Software Foundation.
> +*
> +* This program is distributed in the hope that it will be useful,
> +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> +* GNU General Public License for more details.
> +*/
> +#include <linux/clk.h>
> +#include <linux/debugfs.h>
> +#include <linux/firmware.h>
> +#include <linux/interrupt.h>
> +#include <linux/iommu.h>
> +#include <linux/module.h>
> +#include <linux/of_address.h>
> +#include <linux/of_irq.h>
> +#include <linux/of_platform.h>
> +#include <linux/sched.h>
> +#include <linux/sizes.h>
> +
> +#include "mtk_vpu_core.h"
> +
> +/**
> + * VPU (video processor unit) is a tiny processor controlling video hardware
> + * related to video codec, scaling and color format converting.
> + * VPU interfaces with other blocks by share memory and interrupt.
> + **/
> +#define MTK_VPU_DRV_NAME "mtk_vpu"
This is only used once, why not just put this string directly into the
.name field?
> +
> +#define INIT_TIMEOUT_MS 2000U
> +#define IPI_TIMEOUT_MS 2000U
> +#define VPU_FW_VER_LEN 16
> +
> +/* vpu extended virtural address */
> +#define VPU_PMEM0_VIRT(vpu) ((vpu)->mem.p_va)
> +#define VPU_DMEM0_VIRT(vpu) ((vpu)->mem.d_va)
> +/* vpu extended iova address*/
> +#define VPU_PMEM0_IOVA(vpu) ((vpu)->mem.p_iova)
> +#define VPU_DMEM0_IOVA(vpu) ((vpu)->mem.d_iova)
These feel like obfuscation to me. The code looks clearer without the
macro For example:
vpu->mem.p_va = dma_alloc_coherent(dev, ...);
Is much clearer than:
VPU_PMEM0_VIRT(vpu) = dma_alloc_coherent(dev, ...);
> +
> +#define VPU_PTCM(dev) ((dev)->reg.sram)
> +#define VPU_DTCM(dev) ((dev)->reg.sram + VPU_DTCM_OFFSET)
These macros also seem to add very little value. They are consumed only
a couple of times and only serve to conceal how the sram is mapped and
consumed:
dest = vpu->reg.sram;
if (fw_type == D_FW)
dest += VPU_DTCM_OFFSET;
Compared to:
dest = fw_type ? VPU_DTCM(vpu) : VPU_PTCM(vpu);
> +
> +#define VPU_PTCM_SIZE (96 * SZ_1K)
> +#define VPU_DTCM_SIZE (32 * SZ_1K)
> +#define VPU_DTCM_OFFSET 0x18000UL
> +#define VPU_EXT_P_SIZE SZ_1M
> +#define VPU_EXT_D_SIZE SZ_4M
> +#define VPU_P_FW_SIZE (VPU_PTCM_SIZE + VPU_EXT_P_SIZE)
> +#define VPU_D_FW_SIZE (VPU_DTCM_SIZE + VPU_EXT_D_SIZE)
> +#define SHARE_BUF_SIZE 48
> +
> +#define VPU_P_FW "vpu_p.bin"
> +#define VPU_D_FW "vpu_d.bin"
These macros are of questionable value.
> +
> +#define VPU_BASE 0x0
Is this register really called "base" in the datasheet? From the use in
the code it looks like it performs a reset and/or PM function.
> +#define VPU_TCM_CFG 0x0008
> +#define VPU_PMEM_EXT0_ADDR 0x000C
> +#define VPU_PMEM_EXT1_ADDR 0x0010
> +#define VPU_TO_HOST 0x001C
> +#define VPU_DMEM_EXT0_ADDR 0x0014
> +#define VPU_DMEM_EXT1_ADDR 0x0018
> +#define HOST_TO_VPU 0x0024
> +#define VPU_PC_REG 0x0060
> +#define VPU_WDT_REG 0x0084
> +
> +/* vpu inter-processor communication interrupt */
> +#define VPU_IPC_INT BIT(8)
> +
> +/**
> + * enum vpu_fw_type - VPU firmware type
> + *
> + * @P_FW: program firmware
> + * @D_FW: data firmware
> + *
> + */
> +enum vpu_fw_type {
> + P_FW,
> + D_FW,
> +};
> +
> +/**
> + * struct vpu_mem - VPU memory information
Perhaps "VPU extended program/data memory information" instead.
> + *
> + * @p_va: the kernel virtual memory address of
> + * VPU extended program memory
> + * @d_va: the kernel virtual memory address of VPU extended data memory
> + * @p_iova: the iova memory address of VPU extended program memory
> + * @d_iova: the iova memory address of VPU extended data memory
> + */
> +struct vpu_mem {
> + void *p_va;
> + void *d_va;
> + dma_addr_t p_iova;
> + dma_addr_t d_iova;
> +};
Might be better as:
struct {
void *va;
dma_addr_t iova;
}
This can be then be used as: vpu->mem[P_FW].va . This doesn't matter
much yet but it would helps us common up some code later.
> +
> +/**
> + * struct vpu_regs - VPU SRAM and configuration registers
> + *
> + * @sram: the register for VPU sram
> + * @cfg: the register for VPU configuration
> + * @irq: the irq number for VPU interrupt
> + */
> +struct vpu_regs {
> + void __iomem *sram;
This is called TCM everywhere else. Why a different name for it here?
> + void __iomem *cfg;
> + int irq;
> +};
> +
> +/**
> + * struct vpu_run - VPU initialization status
> + *
> + * @signaled: the signal of vpu initialization completed
> + * @fw_ver: VPU firmware version
> + * @wq: wait queue for VPU initialization status
> + */
> +struct vpu_run {
> + u32 signaled;
> + char fw_ver[VPU_FW_VER_LEN];
> + wait_queue_head_t wq;
> +};
> +
> +/**
> + * struct vpu_ipi_desc - VPU IPI descriptor
> + *
> + * @handler: IPI handler
> + * @name: the name of IPI handler
> + * @priv: the private data of IPI handler
> + */
> +struct vpu_ipi_desc {
> + ipi_handler_t handler;
> + const char *name;
> + void *priv;
> +};
> +
> +/**
> + * struct share_obj - The DTCM (Data Tightly-Coupled Memory) buffer shared with
> + * AP and VPU
Remove "The" (there are more than one of these buffers).
> + *
> + * @id: IPI id
> + * @len: share buffer length
> + * @share_buf: share buffer data
> + */
> +struct share_obj {
> + int32_t id;
> + uint32_t len;
> + unsigned char share_buf[SHARE_BUF_SIZE];
> +};
> +
> +/**
> + * struct mtk_vpu - vpu driver data
> + * @mem: VPU extended memory information
> + * @reg: VPU SRAM and configuration registers
> + * @run: VPU initialization status
> + * @ipi_desc: VPU IPI descriptor
> + * @recv_buf: VPU DTCM share buffer for receiving. The
> + * receive buffer is only accessed in interrupt context.
> + * @send_buf: VPU DTCM share buffer for sending
> + * @dev: VPU struct device
> + * @clk: VPU clock on/off
> + * @vpu_mutex: protect mtk_vpu (except recv_buf) and ensure only
> + * one client to use VPU service at a time. For example,
> + * suppose a client is using VPU to decode VP8.
> + * If the other client wants to encode VP8,
> + * it has to wait until VP8 decode completes.
> + *
> + */
> +struct mtk_vpu {
> + struct vpu_mem mem;
Rename to extmem?
> + struct vpu_regs reg;
> + struct vpu_run run;
> + struct vpu_ipi_desc ipi_desc[IPI_MAX];
> + struct share_obj *recv_buf;
> + struct share_obj *send_buf;
> + struct device *dev;
> + struct clk *clk;
> + struct mutex vpu_mutex; /* for protecting vpu data data structure */
> +};
> +
> +/* the thread calls the function should hold the |vpu_mutex| */
Remove this comment. Its unhelpful: the code does not meet this
requirement because it overstates the scope of vpu_mutex .
> +static inline void vpu_cfg_writel(struct mtk_vpu *vpu, u32 val, u32 offset)
> +{
> + writel(val, vpu->reg.cfg + offset);
> +}
> +
> +static inline u32 vpu_cfg_readl(struct mtk_vpu *vpu, u32 offset)
> +{
> + return readl(vpu->reg.cfg + offset);
> +}
> +
> +static inline bool vpu_running(struct mtk_vpu *vpu)
> +{
> + return vpu_cfg_readl(vpu, VPU_BASE) & BIT(0);
> +}
> +
> +void vpu_disable_clock(struct platform_device *pdev)
> +{
> + struct mtk_vpu *vpu = platform_get_drvdata(pdev);
> +
> + /* Disable VPU watchdog */
> + vpu_cfg_writel(vpu,
> + vpu_cfg_readl(vpu, VPU_WDT_REG) & ~(1L<<31),
> + VPU_WDT_REG);
This code combines a reference counted clock disable with a
not-reference counted register write and will result in the watchdot
being spuriously disabled.
This will definitely happen if vpu_debug_read() is called at the wrong
time, possibly may also be an issue for concurrent H.264 and VP8 operations.
> +
> + clk_disable_unprepare(vpu->clk);
> +}
> +
> +int vpu_enable_clock(struct platform_device *pdev)
> +{
> + struct mtk_vpu *vpu = platform_get_drvdata(pdev);
> + int ret;
> +
> + ret = clk_prepare_enable(vpu->clk);
> + if (ret)
> + return ret;
> + /* Enable VPU watchdog */
> + vpu_cfg_writel(vpu, vpu_cfg_readl(vpu, VPU_WDT_REG) | (1L << 31),
> + VPU_WDT_REG);
As above.
> +
> + return ret;
> +}
> +
> +int vpu_ipi_register(struct platform_device *pdev,
> + enum ipi_id id, ipi_handler_t handler,
> + const char *name, void *priv)
> +{
> + struct mtk_vpu *vpu = platform_get_drvdata(pdev);
> + struct vpu_ipi_desc *ipi_desc;
> +
> + if (!vpu) {
> + dev_err(&pdev->dev, "vpu device in not ready\n");
> + return -EPROBE_DEFER;
> + }
> +
> + if (id < IPI_MAX && handler != NULL) {
> + ipi_desc = vpu->ipi_desc;
> + ipi_desc[id].name = name;
> + ipi_desc[id].handler = handler;
> + ipi_desc[id].priv = priv;
> + return 0;
> + }
> +
> + dev_err(&pdev->dev, "register vpu ipi with invalid arguments\n");
> + return -EINVAL;
> +}
This API, and manu of its friends lower down the file, appear to be a
way to send 32 byte messages to different endpoints on another processor
on the SoC.
Just interested to know if you evaluated the mailbox driver
infrastructure for this? Its a good fit for the messaging but doesn't
offer any support for the direct access to TCM or extended memory.
> +int vpu_ipi_send(struct platform_device *pdev,
> + enum ipi_id id, void *buf,
> + unsigned int len, unsigned int wait)
> +{
> + struct mtk_vpu *vpu = platform_get_drvdata(pdev);
> + struct share_obj *send_obj = vpu->send_buf;
> + unsigned long timeout;
> +
> + if (id >= IPI_MAX || len > sizeof(send_obj->share_buf) || buf == NULL) {
> + dev_err(vpu->dev, "failed to send ipi message\n");
> + return -EINVAL;
> + }
> +
> + if (!vpu_running(vpu)) {
> + dev_err(vpu->dev, "vpu_ipi_send: VPU is not running\n");
> + return -ENXIO;
> + }
> +
> + mutex_lock(&vpu->vpu_mutex);
> + if (vpu_cfg_readl(vpu, HOST_TO_VPU) && !wait) {
> + mutex_unlock(&vpu->vpu_mutex);
> + return -EBUSY;
> + }
This branch is unreachable (no caller ever sets wait is never set to false).
> +
> + if (wait)
> + while (vpu_cfg_readl(vpu, HOST_TO_VPU))
> + ;
What is this loop for? This code should only be reachable if we timed
out in the code below so the likely effect of this loop is to
permantently wedge a thread in a manner that frustrates signal delivery.
> +
> + memcpy((void *)send_obj->share_buf, buf, len);
> + send_obj->len = len;
> + send_obj->id = id;
> + vpu_cfg_writel(vpu, 0x1, HOST_TO_VPU);
> +
> + /* Wait until VPU receives the command */
> + timeout = jiffies + msecs_to_jiffies(IPI_TIMEOUT_MS);
> + do {
> + if (time_after(jiffies, timeout)) {
> + dev_err(vpu->dev, "vpu_ipi_send: IPI timeout!\n");
> + return -EIO;
> + }
> + } while (vpu_cfg_readl(vpu, HOST_TO_VPU));
Do we need to busy wait every time we communicate with the co-processor?
Couldn't we put this wait *before* we write to HOST_TO_VPU above.
That way we only spin when there is a need to.
> +
> + mutex_unlock(&vpu->vpu_mutex);
> +
> + return 0;
> +}
> +
> +void *vpu_mapping_dm_addr(struct platform_device *pdev,
> + void *dtcm_dmem_addr)
Use a different type: dtcm_dmem_addr is not a pointer on this CPU.
> +{
> + struct mtk_vpu *vpu = platform_get_drvdata(pdev);
> + unsigned long p_vpu_dtcm = (unsigned long)VPU_DTCM(vpu);
p_vpu_dtcm *is* a pointer on this CPU. Why cast it so that it isn't.
> + unsigned long ul_dtcm_dmem_addr = (unsigned long)(dtcm_dmem_addr);
> +
> + if (dtcm_dmem_addr == NULL ||
> + (ul_dtcm_dmem_addr > (VPU_DTCM_SIZE + VPU_EXT_D_SIZE))) {
> + dev_err(vpu->dev, "invalid virtual data memory address\n");
> + return ERR_PTR(-EINVAL);
> + }
> +
> + if (ul_dtcm_dmem_addr < VPU_DTCM_SIZE)
> + return (void *)(ul_dtcm_dmem_addr + p_vpu_dtcm);
Starting with the pointer will preserve type better:
return vpu->reg.sram + VPU_DTCM_OFFSET + ul_dtcm_dmem_addr;
> +
> + return (void *)((ul_dtcm_dmem_addr - VPU_DTCM_SIZE) +
> + VPU_DMEM0_VIRT(vpu));
Likewise this code is clearer with the pointer first.
return vpu->mem.d_va + (ul_dtcm_dmem_addr - VPU_DTCM_SIZE);
> +}
> +
> +dma_addr_t *vpu_mapping_iommu_dm_addr(struct platform_device *pdev,
> + void *dmem_addr)
This function does not return a pointer to a dma_addr_t. It just returns
a regular dma_addr_t (that's why all the callers of this function cast
it back ;-) ).
dmem_addr is also not a pointer and this function does not return a
pointer to a dma_addr_t. It just returns a regular dma_addr_t.
> +{
> + unsigned long ul_dmem_addr = (unsigned long)(dmem_addr);
> + struct mtk_vpu *vpu = platform_get_drvdata(pdev);
> +
> + if (dmem_addr == NULL ||
> + (ul_dmem_addr < VPU_DTCM_SIZE) ||
> + (ul_dmem_addr > (VPU_DTCM_SIZE + VPU_EXT_D_SIZE))) {
> + dev_err(vpu->dev, "invalid IOMMU data memory address\n");
> + return ERR_PTR(-EINVAL);
> + }
> +
> + return (dma_addr_t *)((ul_dmem_addr - VPU_DTCM_SIZE) +
> + VPU_DMEM0_IOVA(vpu));
Better written as (this would also have made explicit the type problem
with the function:
return vpu->mem.d_iova + (ul_dmem_addr - VPU_DTCM_SIZE);
> +}
> +
> +struct platform_device *vpu_get_plat_device(struct platform_device *pdev)
> +{
> + struct device *dev = &pdev->dev;
> + struct device_node *vpu_node;
> + struct platform_device *vpu_pdev;
> +
> + vpu_node = of_parse_phandle(dev->of_node, "vpu", 0);
> + if (!vpu_node) {
> + dev_err(dev, "can't get vpu node\n");
> + return NULL;
> + }
> +
> + vpu_pdev = of_find_device_by_node(vpu_node);
> + if (WARN_ON(!vpu_pdev)) {
> + dev_err(dev, "vpu pdev failed\n");
> + of_node_put(vpu_node);
> + return NULL;
> + }
> +
> + return vpu_pdev;
> +}
This function looks a bit weird to me. Why do we need to keep consulting
the devicetree every time we want to start/stop the clock?
I would have expected this code to be part of the init routine of
anything that needs this and the platform_device would then be cached.
> +
> +/* load vpu program/data memory */
> +static void load_requested_vpu(struct mtk_vpu *vpu,
> + size_t fw_size,
> + const u8 *fw_data,
> + u8 fw_type)
> +{
> + size_t target_size = fw_type ? VPU_DTCM_SIZE : VPU_PTCM_SIZE;
> + size_t extra_fw_size = 0;
> + void *dest;
> +
> + /* reset VPU */
> + vpu_cfg_writel(vpu, 0x0, VPU_BASE);
> +
> + /* handle extended firmware size */
> + if (fw_size > target_size) {
> + dev_dbg(vpu->dev, "fw size %lx > limited fw size %lx\n",
> + fw_size, target_size);
> + extra_fw_size = fw_size - target_size;
> + dev_dbg(vpu->dev, "extra_fw_size %lx\n", extra_fw_size);
> + fw_size = target_size;
> + }
> + dest = fw_type ? VPU_DTCM(vpu) : VPU_PTCM(vpu);
> + memcpy(dest, fw_data, fw_size);
> + /* download to extended memory if need */
> + if (extra_fw_size > 0) {
> + dest = fw_type ?
> + VPU_DMEM0_VIRT(vpu) : VPU_PMEM0_VIRT(vpu);
> +
> + dev_dbg(vpu->dev, "download extended memory type %x\n",
> + fw_type);
> + memcpy(dest, fw_data + target_size, extra_fw_size);
> + }
> +}
> +
> +int vpu_load_firmware(struct platform_device *pdev)
> +{
> + struct mtk_vpu *vpu = platform_get_drvdata(pdev);
> + struct device *dev = &pdev->dev;
> + struct vpu_run *run = &vpu->run;
> + const struct firmware *vpu_fw;
> + int ret;
> +
> + if (!pdev) {
> + dev_err(dev, "VPU platform device is invalid\n");
> + return -EINVAL;
> + }
> +
> + mutex_lock(&vpu->vpu_mutex);
> +
> + ret = vpu_enable_clock(pdev);
> + if (ret) {
> + dev_err(dev, "enable clock failed %d\n", ret);
> + goto OUT_LOAD_FW;
> + }
> +
> + if (vpu_running(vpu)) {
> + vpu_disable_clock(pdev);
> + mutex_unlock(&vpu->vpu_mutex);
> + dev_warn(dev, "vpu is running already\n");
> + return 0;
> + }
> +
> + run->signaled = false;
> + dev_dbg(vpu->dev, "firmware request\n");
> + ret = request_firmware(&vpu_fw, VPU_P_FW, dev);
> + if (ret < 0) {
> + dev_err(dev, "Failed to load %s, %d\n", VPU_P_FW, ret);
> + goto OUT_LOAD_FW;
> + }
> + if (vpu_fw->size > VPU_P_FW_SIZE) {
> + ret = -EFBIG;
> + dev_err(dev, "program fw size %zu is abnormal\n", vpu_fw->size);
> + goto OUT_LOAD_FW;
> + }
Possibly move request_firmware(), release_firmware() and the associated
error handling into load_requested_vpu(). It can all be parameterized
and the filename of the firmware means error reports will still be clear
about whether the p or d firmware is faulty.
> + dev_dbg(vpu->dev, "Downloaded program fw size: %zu.\n",
> + vpu_fw->size);
> + /* Downloading program firmware to device*/
> + load_requested_vpu(vpu, vpu_fw->size, vpu_fw->data,
> + P_FW);
> + release_firmware(vpu_fw);
> +
> + ret = request_firmware(&vpu_fw, VPU_D_FW, dev);
> + if (ret < 0) {
> + dev_err(dev, "Failed to load %s, %d\n", VPU_D_FW, ret);
> + goto OUT_LOAD_FW;
> + }
> + if (vpu_fw->size > VPU_D_FW_SIZE) {
> + ret = -EFBIG;
> + dev_err(dev, "data fw size %zu is abnormal\n", vpu_fw->size);
> + goto OUT_LOAD_FW;
> + }
> + dev_dbg(vpu->dev, "Downloaded data fw size: %zu.\n",
> + vpu_fw->size);
> + /* Downloading data firmware to device */
> + load_requested_vpu(vpu, vpu_fw->size, vpu_fw->data,
> + D_FW);
> + release_firmware(vpu_fw);
> + /* boot up vpu */
> + vpu_cfg_writel(vpu, 0x1, VPU_BASE);
> +
> + ret = wait_event_interruptible_timeout(run->wq,
> + run->signaled,
> + msecs_to_jiffies(INIT_TIMEOUT_MS)
> + );
> + if (0 == ret) {
> + ret = -ETIME;
> + dev_err(dev, "wait vpu initialization timout!\n");
> + goto OUT_LOAD_FW;
> + } else if (-ERESTARTSYS == ret) {
> + dev_err(dev, "wait vpu interrupted by a signal!\n");
> + goto OUT_LOAD_FW;
> + }
> +
> + ret = 0;
> + dev_info(dev, "vpu is ready. Fw version %s\n", run->fw_ver);
> +
> +OUT_LOAD_FW:
> + vpu_disable_clock(pdev);
> + mutex_unlock(&vpu->vpu_mutex);
> +
> + return ret;
> +}
> +
> +static void vpu_init_ipi_handler(void *data, unsigned int len, void *priv)
> +{
> + struct mtk_vpu *vpu = (struct mtk_vpu *)priv;
> + struct vpu_run *run = (struct vpu_run *)data;
> +
> + vpu->run.signaled = run->signaled;
> + strncpy(vpu->run.fw_ver, run->fw_ver, VPU_FW_VER_LEN);
> + wake_up_interruptible(&vpu->run.wq);
> +}
> +
> +static int vpu_debug_open(struct inode *inode, struct file *file)
> +{
> + file->private_data = inode->i_private;
> + return 0;
> +}
> +
> +static ssize_t vpu_debug_read(struct file *file, char __user *user_buf,
> + size_t count, loff_t *ppos)
> +{
> + char buf[256];
> + unsigned int len;
> + unsigned int running, pc, vpu_to_host, host_to_vpu, wdt;
> + int ret;
> + struct device *dev = file->private_data;
> + struct platform_device *pdev = to_platform_device(dev);
> + struct mtk_vpu *vpu = dev_get_drvdata(dev);
> +
> + ret = vpu_enable_clock(pdev);
> + if (ret) {
> + dev_err(vpu->dev, "[VPU] enable clock failed %d\n", ret);
> + return 0;
> + }
> +
> + /* vpu register status */
> + running = vpu_running(vpu);
> + pc = vpu_cfg_readl(vpu, VPU_PC_REG);
> + wdt = vpu_cfg_readl(vpu, VPU_WDT_REG);
> + host_to_vpu = vpu_cfg_readl(vpu, HOST_TO_VPU);
> + vpu_to_host = vpu_cfg_readl(vpu, VPU_TO_HOST);
> + vpu_disable_clock(pdev);
> +
> + if (running) {
> + len = sprintf(buf, "VPU is running\n\n"
> + "FW Version: %s\n"
> + "PC: 0x%x\n"
> + "WDT: 0x%x\n"
> + "Host to VPU: 0x%x\n"
> + "VPU to Host: 0x%x\n",
> + vpu->run.fw_ver, pc, wdt,
> + host_to_vpu, vpu_to_host);
> + } else {
> + len = sprintf(buf, "VPU not running\n");
> + }
> +
> + return simple_read_from_buffer(user_buf, count, ppos, buf, len);
> +}
> +
> +static const struct file_operations vpu_debug_fops = {
> + .open = vpu_debug_open,
> + .read = vpu_debug_read,
> +};
These operations should be conditional on CONFIG_DEBUG_FS.
> +
> +static void vpu_free_p_ext_mem(struct mtk_vpu *vpu)
> +{
> + struct device *dev = vpu->dev;
> + struct iommu_domain *domain = iommu_get_domain_for_dev(dev);
> +
> + dma_free_coherent(dev, VPU_EXT_P_SIZE, VPU_PMEM0_VIRT(vpu),
> + VPU_PMEM0_IOVA(vpu));
> +
> + if (domain)
> + iommu_detach_device(domain, vpu->dev);
> +}
> +
> +static void vpu_free_d_ext_mem(struct mtk_vpu *vpu)
> +{
> + struct device *dev = vpu->dev;
> + struct iommu_domain *domain = iommu_get_domain_for_dev(dev);
> +
> + dma_free_coherent(dev, VPU_EXT_D_SIZE, VPU_DMEM0_VIRT(vpu),
> + VPU_DMEM0_IOVA(vpu));
> +
> + if (domain)
> + iommu_detach_device(domain, dev);
> +}
Look like this could be parameterized and combined with vpu_free_p_ext_mem.
> +
> +static int vpu_alloc_p_ext_mem(struct mtk_vpu *vpu)
> +{
> + struct device *dev = vpu->dev;
> + struct iommu_domain *domain = iommu_get_domain_for_dev(dev);
> + phys_addr_t p_pa;
> +
> + VPU_PMEM0_VIRT(vpu) = dma_alloc_coherent(dev,
> + VPU_EXT_P_SIZE,
> + &(VPU_PMEM0_IOVA(vpu)),
> + GFP_KERNEL);
> + if (VPU_PMEM0_VIRT(vpu) == NULL) {
> + dev_err(dev, "Failed to allocate the extended program memory\n");
> + return PTR_ERR(VPU_PMEM0_VIRT(vpu));
> + }
> +
> + p_pa = iommu_iova_to_phys(domain, vpu->mem.p_iova);
> + /* Disable extend0. Enable extend1 */
> + vpu_cfg_writel(vpu, 0x1, VPU_PMEM_EXT0_ADDR);
> + vpu_cfg_writel(vpu, (p_pa & 0xFFFFF000), VPU_PMEM_EXT1_ADDR);
> +
> + dev_info(dev, "Program extend memory phy=0x%llx virt=0x%p iova=0x%llx\n",
> + (unsigned long long)p_pa,
> + VPU_PMEM0_VIRT(vpu),
> + (unsigned long long)VPU_PMEM0_IOVA(vpu));
> +
> + return 0;
> +}
> +
> +static int vpu_alloc_d_ext_mem(struct mtk_vpu *vpu)
> +{
> + struct device *dev = vpu->dev;
> + struct iommu_domain *domain = iommu_get_domain_for_dev(dev);
> + phys_addr_t d_pa;
> +
> + VPU_DMEM0_VIRT(vpu) = dma_alloc_coherent(dev,
> + VPU_EXT_D_SIZE,
> + &(VPU_DMEM0_IOVA(vpu)),
> + GFP_KERNEL);
> + if (VPU_DMEM0_VIRT(vpu) == NULL) {
> + dev_err(dev, "Failed to allocate the extended data memory\n");
> + return PTR_ERR(VPU_DMEM0_VIRT(vpu));
> + }
> +
> + d_pa = iommu_iova_to_phys(domain, vpu->mem.d_iova);
> +
> + /* Disable extend0. Enable extend1 */
> + vpu_cfg_writel(vpu, 0x1, VPU_DMEM_EXT0_ADDR);
> + vpu_cfg_writel(vpu, (d_pa & 0xFFFFF000),
> + VPU_DMEM_EXT1_ADDR);
> +
> + dev_info(dev, "Data extend memory phy=0x%llx virt=0x%p iova=0x%llx\n",
> + (unsigned long long)d_pa,
> + VPU_DMEM0_VIRT(vpu),
> + (unsigned long long)VPU_DMEM0_IOVA(vpu));
> +
> + return 0;
> +}
Also looks suitable for parameterizing and combining with
vpu_alloc_p_ext_mem .
> +
> +static void vpu_ipi_handler(struct mtk_vpu *vpu)
> +{
> + struct share_obj *rcv_obj = vpu->recv_buf;
> + struct vpu_ipi_desc *ipi_desc = vpu->ipi_desc;
> +
> + if (rcv_obj->id < IPI_MAX && ipi_desc[rcv_obj->id].handler) {
> + ipi_desc[rcv_obj->id].handler(rcv_obj->share_buf,
> + rcv_obj->len,
> + ipi_desc[rcv_obj->id].priv);
> + } else {
> + dev_err(vpu->dev, "No such ipi id = %d\n", rcv_obj->id);
> + }
> +}
> +
> +static int vpu_ipi_init(struct mtk_vpu *vpu)
> +{
> + /* Disable VPU to host interrupt */
> + vpu_cfg_writel(vpu, 0x0, VPU_TO_HOST);
> +
> + /* shared buffer initialization */
> + vpu->recv_buf = (struct share_obj *)VPU_DTCM(vpu);
> + vpu->send_buf = vpu->recv_buf + 1;
> + memset(vpu->recv_buf, 0, sizeof(struct share_obj));
> + memset(vpu->send_buf, 0, sizeof(struct share_obj));
> + mutex_init(&vpu->vpu_mutex);
> +
> + return 0;
> +}
> +
> +static irqreturn_t vpu_irq_handler(int irq, void *priv)
> +{
> + struct mtk_vpu *vpu = priv;
> + uint32_t vpu_to_host = vpu_cfg_readl(vpu, VPU_TO_HOST);
> +
> + if (vpu_to_host & VPU_IPC_INT)
> + vpu_ipi_handler(vpu);
> + else
> + dev_err(vpu->dev, "vpu watchdog timeout!\n");
> +
> + /* VPU won't send another interrupt until we set VPU_TO_HOST to 0. */
> + vpu_cfg_writel(vpu, 0x0, VPU_TO_HOST);
If we were triggered by a watchdog then how long will it be before the
next watchdog interrupt? Will we end up spamming the logs?
> +
> + return IRQ_HANDLED;
> +}
> +
> +static struct dentry *vpu_debugfs;
> +static int mtk_vpu_probe(struct platform_device *pdev)
> +{
> + struct mtk_vpu *vpu;
> + struct device *dev;
> + struct resource *res;
> + int ret = 0;
> +
> + dev_dbg(&pdev->dev, "initialization\n");
> +
> + dev = &pdev->dev;
> + vpu = devm_kzalloc(dev, sizeof(*vpu), GFP_KERNEL);
> + if (!vpu)
> + return -ENOMEM;
> +
> + vpu->dev = &pdev->dev;
> + res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "sram");
> + vpu->reg.sram = devm_ioremap_resource(dev, res);
> + if (IS_ERR(vpu->reg.sram)) {
> + dev_err(dev, "devm_ioremap_resource vpu sram failed.\n");
> + return PTR_ERR(vpu->reg.sram);
> + }
> +
> + res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "cfg_reg");
> + vpu->reg.cfg = devm_ioremap_resource(dev, res);
> + if (IS_ERR(vpu->reg.cfg)) {
> + dev_err(dev, "devm_ioremap_resource vpu cfg failed.\n");
> + return PTR_ERR(vpu->reg.cfg);
> + }
> +
> + /* Get VPU clock */
> + vpu->clk = devm_clk_get(dev, "main");
> + if (vpu->clk == NULL) {
> + dev_err(dev, "get vpu clock fail\n");
> + return -EINVAL;
> + }
> +
> + platform_set_drvdata(pdev, vpu);
> +
> + ret = vpu_enable_clock(pdev);
> + if (ret) {
> + ret = -EINVAL;
> + return ret;
Why not just "return ret" (or "return -EINVAL")?
> + }
> +
> + dev_dbg(dev, "vpu ipi init\n");
> + ret = vpu_ipi_init(vpu);
> + if (ret) {
> + dev_err(dev, "Failed to init ipi\n");
> + goto disable_vpu_clk;
> + }
> +
> + platform_set_drvdata(pdev, vpu);
> +
> + /* register vpu initialization IPI */
> + ret = vpu_ipi_register(pdev, IPI_VPU_INIT, vpu_init_ipi_handler,
> + "vpu_init", vpu);
> + if (ret) {
> + dev_err(dev, "Failed to register IPI_VPU_INIT\n");
> + goto vpu_mutex_destroy;
> + }
> +
> + vpu_debugfs = debugfs_create_file("mtk_vpu", S_IRUGO, NULL, (void *)dev,
> + &vpu_debug_fops);
> + if (!vpu_debugfs) {
> + ret = -ENOMEM;
> + goto cleanup_ipi;
> + }
> +
> + /* Set PTCM to 96K and DTCM to 32K */
> + vpu_cfg_writel(vpu, 0x2, VPU_TCM_CFG);
> +
> + ret = vpu_alloc_p_ext_mem(vpu);
> + if (ret) {
> + dev_err(dev, "Allocate PM failed\n");
> + goto remove_debugfs;
> + }
> +
> + ret = vpu_alloc_d_ext_mem(vpu);
> + if (ret) {
> + dev_err(dev, "Allocate DM failed\n");
> + goto free_p_mem;
> + }
> +
> + init_waitqueue_head(&vpu->run.wq);
> +
> + res = platform_get_resource(pdev, IORESOURCE_IRQ, 0);
> + if (res == NULL) {
> + dev_err(dev, "get IRQ resource failed.\n");
> + ret = -ENXIO;
> + goto free_d_mem;
> + }
> + vpu->reg.irq = platform_get_irq(pdev, 0);
> + ret = devm_request_irq(dev, vpu->reg.irq, vpu_irq_handler, 0,
> + pdev->name, vpu);
> + if (ret) {
> + dev_err(dev, "failed to request irq\n");
> + goto free_d_mem;
> + }
> +
> + vpu_disable_clock(pdev);
> + dev_dbg(dev, "initialization completed\n");
> +
> + return 0;
> +
> +free_d_mem:
> + vpu_free_d_ext_mem(vpu);
> +free_p_mem:
> + vpu_free_p_ext_mem(vpu);
> +remove_debugfs:
> + debugfs_remove(vpu_debugfs);
> +cleanup_ipi:
> + memset(vpu->ipi_desc, 0, sizeof(struct vpu_ipi_desc)*IPI_MAX);
> +vpu_mutex_destroy:
> + mutex_destroy(&vpu->vpu_mutex);
> +disable_vpu_clk:
> + vpu_disable_clock(pdev);
> +
> + return ret;
> +}
> +
> +static const struct of_device_id mtk_vpu_match[] = {
> + {
> + .compatible = "mediatek,mt8173-vpu",
> + },
> + {},
> +};
> +MODULE_DEVICE_TABLE(of, mtk_vpu_match);
> +
> +static int mtk_vpu_remove(struct platform_device *pdev)
> +{
> + struct mtk_vpu *vpu = platform_get_drvdata(pdev);
> +
> + vpu_free_p_ext_mem(vpu);
> + vpu_free_d_ext_mem(vpu);
This looks like it leaks cpu_debugfs and the vpu_mutex.
> +
> + return 0;
> +}
> +
> +static struct platform_driver mtk_vpu_driver = {
> + .probe = mtk_vpu_probe,
> + .remove = mtk_vpu_remove,
> + .driver = {
> + .name = MTK_VPU_DRV_NAME,
> + .owner = THIS_MODULE,
> + .of_match_table = mtk_vpu_match,
> + },
> +};
> +
> +module_platform_driver(mtk_vpu_driver);
> +
> +MODULE_LICENSE("GPL v2");
> +MODULE_DESCRIPTION("Mediatek Video Prosessor Unit driver");
> diff --git a/drivers/media/platform/mtk-vpu/mtk_vpu_core.h b/drivers/media/platform/mtk-vpu/mtk_vpu_core.h
> new file mode 100644
> index 0000000..20cf2a0
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vpu/mtk_vpu_core.h
> @@ -0,0 +1,161 @@
> +/*
> +* Copyright (c) 2015 MediaTek Inc.
> +* Author: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
> +*
> +* This program is free software; you can redistribute it and/or modify
> +* it under the terms of the GNU General Public License version 2 as
> +* published by the Free Software Foundation.
> +*
> +* This program is distributed in the hope that it will be useful,
> +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> +* GNU General Public License for more details.
> +*/
> +
> +#ifndef _MTK_VPU_CORE_H
> +#define _MTK_VPU_CORE_H
> +
> +#include <linux/platform_device.h>
> +
> +/**
> + * VPU (video processor unit) is a tiny processor controlling video hardware
> + * related to video codec, scaling and color format converting.
> + * VPU interfaces with other blocks by share memory and interrupt.
> + **/
> +
> +typedef void (*ipi_handler_t) (void *data,
> + unsigned int len,
> + void *priv);
> +
> +/**
> + * enum ipi_id - the id of inter-processor interrupt
> + *
> + * @IPI_VPU_INIT: The interrupt from vpu is to notfiy kernel
> + VPU initialization completed.
> + * @IPI_VENC_H264: The interrupt from vpu is to notify kernel to
> + handle H264 video encoder job, and vice versa.
> + * @IPI_VENC_VP8: The interrupt fro vpu is to notify kernel to
> + handle VP8 video encoder job,, and vice versa.
> + * @IPI_VENC_CAPABILITY: The interrupt from vpu is to
> + get venc hardware capability.
> + * @IPI_MAX: The maximum IPI number
> + */
> +enum ipi_id {
> + IPI_VPU_INIT = 0,
> + IPI_VENC_H264,
> + IPI_VENC_VP8,
> + IPI_VENC_CAPABILITY,
> + IPI_MAX,
> +};
> +
> +/**
> + * vpu_disable_clock - Disable VPU clock
> + *
> + * @pdev: VPU platform device
> + *
> + *
> + * Return: Return 0 if the clock is disabled successfully,
> + * otherwise it is failed.
> + *
> + **/
> +void vpu_disable_clock(struct platform_device *pdev);
> +
> +/**
> + * vpu_enable_clock - Enable VPU clock
> + *
> + * @pdev: VPU platform device
> + *
> + * Return: Return 0 if the clock is enabled successfully,
> + * otherwise it is failed.
> + *
> + **/
> +int vpu_enable_clock(struct platform_device *pdev);
> +
> +/**
> + * vpu_ipi_register - register an ipi function
> + *
> + * @pdev: VPU platform device
> + * @id: IPI ID
> + * @handler: IPI handler
> + * @name: IPI name
> + * @priv: private data for IPI handler
> + *
> + * Register an ipi function to receive ipi interrupt from VPU.
> + *
> + * Return: Return 0 if ipi registers successfully, otherwise it is failed.
> + */
> +int vpu_ipi_register(struct platform_device *pdev, enum ipi_id id,
> + ipi_handler_t handler, const char *name, void *priv);
> +
> +/**
> + * vpu_ipi_send - send data from AP to vpu.
> + *
> + * @pdev: VPU platform device
> + * @id: IPI ID
> + * @buf: the data buffer
> + * @len: the data buffer length
> + * @wait: wait for the last ipi completed.
> + *
> + * This function is thread-safe. When this function returns,
> + * VPU has received the data and starts the processing.
> + * When the processing completes, IPI handler registered
> + * by vpu_ipi_register will be called in interrupt context.
> + *
> + * Return: Return 0 if sending data successfully, otherwise it is failed.
> + **/
> +int vpu_ipi_send(struct platform_device *pdev,
> + enum ipi_id id, void *buf,
> + unsigned int len,
> + unsigned int wait);
> +
> +/**
> + * vpu_get_plat_device - get VPU's platform device
> + *
> + * @pdev: the platform device of the module requesting VPU platform
> + * device for using VPU API.
> + *
> + * Return: Return NULL if it is failed.
> + * otherwise it is VPU's platform device
> + **/
> +struct platform_device *vpu_get_plat_device(struct platform_device *pdev);
> +
> +/**
> + * vpu_load_firmware - download VPU firmware and boot it
> + *
> + * @pdev: VPU platform device
> + *
> + * Return: Return 0 if downloading firmware successfully,
> + * otherwise it is failed
> + **/
> +int vpu_load_firmware(struct platform_device *pdev);
> +
> +/**
> + * vpu_mapping_dm_addr - Mapping DTCM/DMEM to kernel virtual address
> + *
> + * @pdev: VPU platform device
> + * @dmem_addr: VPU's data memory address
> + *
> + * Mapping the VPU's DTCM (Data Tightly-Coupled Memory) /
> + * DMEM (Data Extended Memory) memory address to
> + * kernel virtual address.
> + *
> + * Return: Return ERR_PTR(-EINVAL) if mapping failed,
> + * otherwise the mapped kernel virtual address
> + **/
> +void *vpu_mapping_dm_addr(struct platform_device *pdev,
> + void *dtcm_dmem_addr);
> +
> +/**
> + * vpu_mapping_iommu_dm_addr - Mapping to iommu address
> + *
> + * @pdev: VPU platform device
> + * @dmem_addr: VPU's extended data memory address
> + *
> + * Mapping the VPU's extended data address to iommu address
> + *
> + * Return: Return ERR_PTR(-EINVAL) if mapping failed,
> + * otherwise the mapped iommu address
> + **/
> +dma_addr_t *vpu_mapping_iommu_dm_addr(struct platform_device *pdev,
> + void *dmem_addr);
> +#endif /* _MTK_VPU_CORE_H */
> diff --git a/drivers/media/platform/mtk-vpu/vp8_enc/venc_vp8_vpu.h b/drivers/media/platform/mtk-vpu/vp8_enc/venc_vp8_vpu.h
> new file mode 100644
> index 0000000..4e09eec
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vpu/vp8_enc/venc_vp8_vpu.h
Like the H.264 header. Why is this file included in this patch? It is
not included by anything in the patch and defines symbols that do not
exist yet.
Daniel.
^ permalink raw reply [flat|nested] 84+ messages in thread
* Re: [RESEND RFC/PATCH 3/8] media: platform: mtk-vpu: Support Mediatek VPU
@ 2015-11-25 16:11 ` Daniel Thompson
0 siblings, 0 replies; 84+ messages in thread
From: Daniel Thompson @ 2015-11-25 16:11 UTC (permalink / raw)
To: Tiffany Lin, Rob Herring, Pawel Moll, Mark Rutland, Ian Campbell,
Kumar Gala, Catalin Marinas, Will Deacon, Mauro Carvalho Chehab,
Matthias Brugger, Daniel Kurtz, Sascha Hauer, Hongzhou Yang,
Hans Verkuil, Laurent Pinchart, Sakari Ailus, Geert Uytterhoeven,
Mikhail Ulyanov, Fabien Dessenne, Arnd Bergmann,
Darren Etheridge, Peter Griffin
Cc: Andrew-CT Chen, Eddie Huang, Yingjoe Chen, James Liao,
Daniel Hsiao, devicetree-u79uwXL29TY76Z2rM5mHXA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA,
linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
linux-media-u79uwXL29TY76Z2rM5mHXA,
linux-mediatek-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r
On 17/11/15 12:54, Tiffany Lin wrote:
> From: Andrew-CT Chen <andrew-ct.chen-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
>
> The VPU driver for hw video codec embedded in Mediatek's MT8173 SOCs.
> It is able to handle video decoding/encoding of in a range of formats.
> The driver provides with VPU firmware download, memory management and
> the communication interface between CPU and VPU.
> For VPU initialization, it will create virtual memory for CPU access and
> IOMMU address for vcodec hw device access. When a decode/encode instance
> opens a device node, vpu driver will download vpu firmware to the device.
> A decode/encode instant will decode/encode a frame using VPU
> interface to interrupt vpu to handle decoding/encoding jobs.
>
> Signed-off-by: Andrew-CT Chen <andrew-ct.chen-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
> ---
> drivers/media/platform/Kconfig | 6 +
> drivers/media/platform/Makefile | 2 +
> drivers/media/platform/mtk-vpu/Makefile | 1 +
> .../platform/mtk-vpu/h264_enc/venc_h264_vpu.h | 127 +++
> .../media/platform/mtk-vpu/include/venc_ipi_msg.h | 212 +++++
> drivers/media/platform/mtk-vpu/mtk_vpu_core.c | 823 ++++++++++++++++++++
> drivers/media/platform/mtk-vpu/mtk_vpu_core.h | 161 ++++
> .../media/platform/mtk-vpu/vp8_enc/venc_vp8_vpu.h | 119 +++
> 8 files changed, 1451 insertions(+)
> create mode 100644 drivers/media/platform/mtk-vpu/Makefile
> create mode 100644 drivers/media/platform/mtk-vpu/h264_enc/venc_h264_vpu.h
> create mode 100644 drivers/media/platform/mtk-vpu/include/venc_ipi_msg.h
> create mode 100644 drivers/media/platform/mtk-vpu/mtk_vpu_core.c
> create mode 100644 drivers/media/platform/mtk-vpu/mtk_vpu_core.h
> create mode 100644 drivers/media/platform/mtk-vpu/vp8_enc/venc_vp8_vpu.h
>
> diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
> index ccbc974..f98eb47 100644
> --- a/drivers/media/platform/Kconfig
> +++ b/drivers/media/platform/Kconfig
> @@ -148,6 +148,12 @@ config VIDEO_CODA
> Coda is a range of video codec IPs that supports
> H.264, MPEG-4, and other video formats.
>
> +config MEDIATEK_VPU
> + bool "Mediatek Video Processor Unit"
> + ---help---
> + This driver provides downloading firmware vpu and
> + communicating with vpu.
> +
This looks pretty broken.
Shouldn't this option be tristate? Why are there no depends-on or selects?
Also I think the help text could be more descriptive here (and so does
checkpatch ;-) ).
> diff --git a/drivers/media/platform/mtk-vpu/h264_enc/venc_h264_vpu.h b/drivers/media/platform/mtk-vpu/h264_enc/venc_h264_vpu.h
> new file mode 100644
> index 0000000..9c8ebdd
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vpu/h264_enc/venc_h264_vpu.h
Why is this file included in *this* patch? It is not included by
anything in the patch and defines functions that do not exist yet.
> diff --git a/drivers/media/platform/mtk-vpu/include/venc_ipi_msg.h b/drivers/media/platform/mtk-vpu/include/venc_ipi_msg.h
> new file mode 100644
> index 0000000..a345b98
This file also is not included by anything and should, perhaps be
included in a different patch.
> diff --git a/drivers/media/platform/mtk-vpu/mtk_vpu_core.c b/drivers/media/platform/mtk-vpu/mtk_vpu_core.c
> new file mode 100644
> index 0000000..b524dbc
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vpu/mtk_vpu_core.c
> @@ -0,0 +1,823 @@
> +/*
> +* Copyright (c) 2015 MediaTek Inc.
> +* Author: Andrew-CT Chen <andrew-ct.chen-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
> +*
> +* This program is free software; you can redistribute it and/or modify
> +* it under the terms of the GNU General Public License version 2 as
> +* published by the Free Software Foundation.
> +*
> +* This program is distributed in the hope that it will be useful,
> +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> +* GNU General Public License for more details.
> +*/
> +#include <linux/clk.h>
> +#include <linux/debugfs.h>
> +#include <linux/firmware.h>
> +#include <linux/interrupt.h>
> +#include <linux/iommu.h>
> +#include <linux/module.h>
> +#include <linux/of_address.h>
> +#include <linux/of_irq.h>
> +#include <linux/of_platform.h>
> +#include <linux/sched.h>
> +#include <linux/sizes.h>
> +
> +#include "mtk_vpu_core.h"
> +
> +/**
> + * VPU (video processor unit) is a tiny processor controlling video hardware
> + * related to video codec, scaling and color format converting.
> + * VPU interfaces with other blocks by share memory and interrupt.
> + **/
> +#define MTK_VPU_DRV_NAME "mtk_vpu"
This is only used once, why not just put this string directly into the
.name field?
> +
> +#define INIT_TIMEOUT_MS 2000U
> +#define IPI_TIMEOUT_MS 2000U
> +#define VPU_FW_VER_LEN 16
> +
> +/* vpu extended virtural address */
> +#define VPU_PMEM0_VIRT(vpu) ((vpu)->mem.p_va)
> +#define VPU_DMEM0_VIRT(vpu) ((vpu)->mem.d_va)
> +/* vpu extended iova address*/
> +#define VPU_PMEM0_IOVA(vpu) ((vpu)->mem.p_iova)
> +#define VPU_DMEM0_IOVA(vpu) ((vpu)->mem.d_iova)
These feel like obfuscation to me. The code looks clearer without the
macro For example:
vpu->mem.p_va = dma_alloc_coherent(dev, ...);
Is much clearer than:
VPU_PMEM0_VIRT(vpu) = dma_alloc_coherent(dev, ...);
> +
> +#define VPU_PTCM(dev) ((dev)->reg.sram)
> +#define VPU_DTCM(dev) ((dev)->reg.sram + VPU_DTCM_OFFSET)
These macros also seem to add very little value. They are consumed only
a couple of times and only serve to conceal how the sram is mapped and
consumed:
dest = vpu->reg.sram;
if (fw_type == D_FW)
dest += VPU_DTCM_OFFSET;
Compared to:
dest = fw_type ? VPU_DTCM(vpu) : VPU_PTCM(vpu);
> +
> +#define VPU_PTCM_SIZE (96 * SZ_1K)
> +#define VPU_DTCM_SIZE (32 * SZ_1K)
> +#define VPU_DTCM_OFFSET 0x18000UL
> +#define VPU_EXT_P_SIZE SZ_1M
> +#define VPU_EXT_D_SIZE SZ_4M
> +#define VPU_P_FW_SIZE (VPU_PTCM_SIZE + VPU_EXT_P_SIZE)
> +#define VPU_D_FW_SIZE (VPU_DTCM_SIZE + VPU_EXT_D_SIZE)
> +#define SHARE_BUF_SIZE 48
> +
> +#define VPU_P_FW "vpu_p.bin"
> +#define VPU_D_FW "vpu_d.bin"
These macros are of questionable value.
> +
> +#define VPU_BASE 0x0
Is this register really called "base" in the datasheet? From the use in
the code it looks like it performs a reset and/or PM function.
> +#define VPU_TCM_CFG 0x0008
> +#define VPU_PMEM_EXT0_ADDR 0x000C
> +#define VPU_PMEM_EXT1_ADDR 0x0010
> +#define VPU_TO_HOST 0x001C
> +#define VPU_DMEM_EXT0_ADDR 0x0014
> +#define VPU_DMEM_EXT1_ADDR 0x0018
> +#define HOST_TO_VPU 0x0024
> +#define VPU_PC_REG 0x0060
> +#define VPU_WDT_REG 0x0084
> +
> +/* vpu inter-processor communication interrupt */
> +#define VPU_IPC_INT BIT(8)
> +
> +/**
> + * enum vpu_fw_type - VPU firmware type
> + *
> + * @P_FW: program firmware
> + * @D_FW: data firmware
> + *
> + */
> +enum vpu_fw_type {
> + P_FW,
> + D_FW,
> +};
> +
> +/**
> + * struct vpu_mem - VPU memory information
Perhaps "VPU extended program/data memory information" instead.
> + *
> + * @p_va: the kernel virtual memory address of
> + * VPU extended program memory
> + * @d_va: the kernel virtual memory address of VPU extended data memory
> + * @p_iova: the iova memory address of VPU extended program memory
> + * @d_iova: the iova memory address of VPU extended data memory
> + */
> +struct vpu_mem {
> + void *p_va;
> + void *d_va;
> + dma_addr_t p_iova;
> + dma_addr_t d_iova;
> +};
Might be better as:
struct {
void *va;
dma_addr_t iova;
}
This can be then be used as: vpu->mem[P_FW].va . This doesn't matter
much yet but it would helps us common up some code later.
> +
> +/**
> + * struct vpu_regs - VPU SRAM and configuration registers
> + *
> + * @sram: the register for VPU sram
> + * @cfg: the register for VPU configuration
> + * @irq: the irq number for VPU interrupt
> + */
> +struct vpu_regs {
> + void __iomem *sram;
This is called TCM everywhere else. Why a different name for it here?
> + void __iomem *cfg;
> + int irq;
> +};
> +
> +/**
> + * struct vpu_run - VPU initialization status
> + *
> + * @signaled: the signal of vpu initialization completed
> + * @fw_ver: VPU firmware version
> + * @wq: wait queue for VPU initialization status
> + */
> +struct vpu_run {
> + u32 signaled;
> + char fw_ver[VPU_FW_VER_LEN];
> + wait_queue_head_t wq;
> +};
> +
> +/**
> + * struct vpu_ipi_desc - VPU IPI descriptor
> + *
> + * @handler: IPI handler
> + * @name: the name of IPI handler
> + * @priv: the private data of IPI handler
> + */
> +struct vpu_ipi_desc {
> + ipi_handler_t handler;
> + const char *name;
> + void *priv;
> +};
> +
> +/**
> + * struct share_obj - The DTCM (Data Tightly-Coupled Memory) buffer shared with
> + * AP and VPU
Remove "The" (there are more than one of these buffers).
> + *
> + * @id: IPI id
> + * @len: share buffer length
> + * @share_buf: share buffer data
> + */
> +struct share_obj {
> + int32_t id;
> + uint32_t len;
> + unsigned char share_buf[SHARE_BUF_SIZE];
> +};
> +
> +/**
> + * struct mtk_vpu - vpu driver data
> + * @mem: VPU extended memory information
> + * @reg: VPU SRAM and configuration registers
> + * @run: VPU initialization status
> + * @ipi_desc: VPU IPI descriptor
> + * @recv_buf: VPU DTCM share buffer for receiving. The
> + * receive buffer is only accessed in interrupt context.
> + * @send_buf: VPU DTCM share buffer for sending
> + * @dev: VPU struct device
> + * @clk: VPU clock on/off
> + * @vpu_mutex: protect mtk_vpu (except recv_buf) and ensure only
> + * one client to use VPU service at a time. For example,
> + * suppose a client is using VPU to decode VP8.
> + * If the other client wants to encode VP8,
> + * it has to wait until VP8 decode completes.
> + *
> + */
> +struct mtk_vpu {
> + struct vpu_mem mem;
Rename to extmem?
> + struct vpu_regs reg;
> + struct vpu_run run;
> + struct vpu_ipi_desc ipi_desc[IPI_MAX];
> + struct share_obj *recv_buf;
> + struct share_obj *send_buf;
> + struct device *dev;
> + struct clk *clk;
> + struct mutex vpu_mutex; /* for protecting vpu data data structure */
> +};
> +
> +/* the thread calls the function should hold the |vpu_mutex| */
Remove this comment. Its unhelpful: the code does not meet this
requirement because it overstates the scope of vpu_mutex .
> +static inline void vpu_cfg_writel(struct mtk_vpu *vpu, u32 val, u32 offset)
> +{
> + writel(val, vpu->reg.cfg + offset);
> +}
> +
> +static inline u32 vpu_cfg_readl(struct mtk_vpu *vpu, u32 offset)
> +{
> + return readl(vpu->reg.cfg + offset);
> +}
> +
> +static inline bool vpu_running(struct mtk_vpu *vpu)
> +{
> + return vpu_cfg_readl(vpu, VPU_BASE) & BIT(0);
> +}
> +
> +void vpu_disable_clock(struct platform_device *pdev)
> +{
> + struct mtk_vpu *vpu = platform_get_drvdata(pdev);
> +
> + /* Disable VPU watchdog */
> + vpu_cfg_writel(vpu,
> + vpu_cfg_readl(vpu, VPU_WDT_REG) & ~(1L<<31),
> + VPU_WDT_REG);
This code combines a reference counted clock disable with a
not-reference counted register write and will result in the watchdot
being spuriously disabled.
This will definitely happen if vpu_debug_read() is called at the wrong
time, possibly may also be an issue for concurrent H.264 and VP8 operations.
> +
> + clk_disable_unprepare(vpu->clk);
> +}
> +
> +int vpu_enable_clock(struct platform_device *pdev)
> +{
> + struct mtk_vpu *vpu = platform_get_drvdata(pdev);
> + int ret;
> +
> + ret = clk_prepare_enable(vpu->clk);
> + if (ret)
> + return ret;
> + /* Enable VPU watchdog */
> + vpu_cfg_writel(vpu, vpu_cfg_readl(vpu, VPU_WDT_REG) | (1L << 31),
> + VPU_WDT_REG);
As above.
> +
> + return ret;
> +}
> +
> +int vpu_ipi_register(struct platform_device *pdev,
> + enum ipi_id id, ipi_handler_t handler,
> + const char *name, void *priv)
> +{
> + struct mtk_vpu *vpu = platform_get_drvdata(pdev);
> + struct vpu_ipi_desc *ipi_desc;
> +
> + if (!vpu) {
> + dev_err(&pdev->dev, "vpu device in not ready\n");
> + return -EPROBE_DEFER;
> + }
> +
> + if (id < IPI_MAX && handler != NULL) {
> + ipi_desc = vpu->ipi_desc;
> + ipi_desc[id].name = name;
> + ipi_desc[id].handler = handler;
> + ipi_desc[id].priv = priv;
> + return 0;
> + }
> +
> + dev_err(&pdev->dev, "register vpu ipi with invalid arguments\n");
> + return -EINVAL;
> +}
This API, and manu of its friends lower down the file, appear to be a
way to send 32 byte messages to different endpoints on another processor
on the SoC.
Just interested to know if you evaluated the mailbox driver
infrastructure for this? Its a good fit for the messaging but doesn't
offer any support for the direct access to TCM or extended memory.
> +int vpu_ipi_send(struct platform_device *pdev,
> + enum ipi_id id, void *buf,
> + unsigned int len, unsigned int wait)
> +{
> + struct mtk_vpu *vpu = platform_get_drvdata(pdev);
> + struct share_obj *send_obj = vpu->send_buf;
> + unsigned long timeout;
> +
> + if (id >= IPI_MAX || len > sizeof(send_obj->share_buf) || buf == NULL) {
> + dev_err(vpu->dev, "failed to send ipi message\n");
> + return -EINVAL;
> + }
> +
> + if (!vpu_running(vpu)) {
> + dev_err(vpu->dev, "vpu_ipi_send: VPU is not running\n");
> + return -ENXIO;
> + }
> +
> + mutex_lock(&vpu->vpu_mutex);
> + if (vpu_cfg_readl(vpu, HOST_TO_VPU) && !wait) {
> + mutex_unlock(&vpu->vpu_mutex);
> + return -EBUSY;
> + }
This branch is unreachable (no caller ever sets wait is never set to false).
> +
> + if (wait)
> + while (vpu_cfg_readl(vpu, HOST_TO_VPU))
> + ;
What is this loop for? This code should only be reachable if we timed
out in the code below so the likely effect of this loop is to
permantently wedge a thread in a manner that frustrates signal delivery.
> +
> + memcpy((void *)send_obj->share_buf, buf, len);
> + send_obj->len = len;
> + send_obj->id = id;
> + vpu_cfg_writel(vpu, 0x1, HOST_TO_VPU);
> +
> + /* Wait until VPU receives the command */
> + timeout = jiffies + msecs_to_jiffies(IPI_TIMEOUT_MS);
> + do {
> + if (time_after(jiffies, timeout)) {
> + dev_err(vpu->dev, "vpu_ipi_send: IPI timeout!\n");
> + return -EIO;
> + }
> + } while (vpu_cfg_readl(vpu, HOST_TO_VPU));
Do we need to busy wait every time we communicate with the co-processor?
Couldn't we put this wait *before* we write to HOST_TO_VPU above.
That way we only spin when there is a need to.
> +
> + mutex_unlock(&vpu->vpu_mutex);
> +
> + return 0;
> +}
> +
> +void *vpu_mapping_dm_addr(struct platform_device *pdev,
> + void *dtcm_dmem_addr)
Use a different type: dtcm_dmem_addr is not a pointer on this CPU.
> +{
> + struct mtk_vpu *vpu = platform_get_drvdata(pdev);
> + unsigned long p_vpu_dtcm = (unsigned long)VPU_DTCM(vpu);
p_vpu_dtcm *is* a pointer on this CPU. Why cast it so that it isn't.
> + unsigned long ul_dtcm_dmem_addr = (unsigned long)(dtcm_dmem_addr);
> +
> + if (dtcm_dmem_addr == NULL ||
> + (ul_dtcm_dmem_addr > (VPU_DTCM_SIZE + VPU_EXT_D_SIZE))) {
> + dev_err(vpu->dev, "invalid virtual data memory address\n");
> + return ERR_PTR(-EINVAL);
> + }
> +
> + if (ul_dtcm_dmem_addr < VPU_DTCM_SIZE)
> + return (void *)(ul_dtcm_dmem_addr + p_vpu_dtcm);
Starting with the pointer will preserve type better:
return vpu->reg.sram + VPU_DTCM_OFFSET + ul_dtcm_dmem_addr;
> +
> + return (void *)((ul_dtcm_dmem_addr - VPU_DTCM_SIZE) +
> + VPU_DMEM0_VIRT(vpu));
Likewise this code is clearer with the pointer first.
return vpu->mem.d_va + (ul_dtcm_dmem_addr - VPU_DTCM_SIZE);
> +}
> +
> +dma_addr_t *vpu_mapping_iommu_dm_addr(struct platform_device *pdev,
> + void *dmem_addr)
This function does not return a pointer to a dma_addr_t. It just returns
a regular dma_addr_t (that's why all the callers of this function cast
it back ;-) ).
dmem_addr is also not a pointer and this function does not return a
pointer to a dma_addr_t. It just returns a regular dma_addr_t.
> +{
> + unsigned long ul_dmem_addr = (unsigned long)(dmem_addr);
> + struct mtk_vpu *vpu = platform_get_drvdata(pdev);
> +
> + if (dmem_addr == NULL ||
> + (ul_dmem_addr < VPU_DTCM_SIZE) ||
> + (ul_dmem_addr > (VPU_DTCM_SIZE + VPU_EXT_D_SIZE))) {
> + dev_err(vpu->dev, "invalid IOMMU data memory address\n");
> + return ERR_PTR(-EINVAL);
> + }
> +
> + return (dma_addr_t *)((ul_dmem_addr - VPU_DTCM_SIZE) +
> + VPU_DMEM0_IOVA(vpu));
Better written as (this would also have made explicit the type problem
with the function:
return vpu->mem.d_iova + (ul_dmem_addr - VPU_DTCM_SIZE);
> +}
> +
> +struct platform_device *vpu_get_plat_device(struct platform_device *pdev)
> +{
> + struct device *dev = &pdev->dev;
> + struct device_node *vpu_node;
> + struct platform_device *vpu_pdev;
> +
> + vpu_node = of_parse_phandle(dev->of_node, "vpu", 0);
> + if (!vpu_node) {
> + dev_err(dev, "can't get vpu node\n");
> + return NULL;
> + }
> +
> + vpu_pdev = of_find_device_by_node(vpu_node);
> + if (WARN_ON(!vpu_pdev)) {
> + dev_err(dev, "vpu pdev failed\n");
> + of_node_put(vpu_node);
> + return NULL;
> + }
> +
> + return vpu_pdev;
> +}
This function looks a bit weird to me. Why do we need to keep consulting
the devicetree every time we want to start/stop the clock?
I would have expected this code to be part of the init routine of
anything that needs this and the platform_device would then be cached.
> +
> +/* load vpu program/data memory */
> +static void load_requested_vpu(struct mtk_vpu *vpu,
> + size_t fw_size,
> + const u8 *fw_data,
> + u8 fw_type)
> +{
> + size_t target_size = fw_type ? VPU_DTCM_SIZE : VPU_PTCM_SIZE;
> + size_t extra_fw_size = 0;
> + void *dest;
> +
> + /* reset VPU */
> + vpu_cfg_writel(vpu, 0x0, VPU_BASE);
> +
> + /* handle extended firmware size */
> + if (fw_size > target_size) {
> + dev_dbg(vpu->dev, "fw size %lx > limited fw size %lx\n",
> + fw_size, target_size);
> + extra_fw_size = fw_size - target_size;
> + dev_dbg(vpu->dev, "extra_fw_size %lx\n", extra_fw_size);
> + fw_size = target_size;
> + }
> + dest = fw_type ? VPU_DTCM(vpu) : VPU_PTCM(vpu);
> + memcpy(dest, fw_data, fw_size);
> + /* download to extended memory if need */
> + if (extra_fw_size > 0) {
> + dest = fw_type ?
> + VPU_DMEM0_VIRT(vpu) : VPU_PMEM0_VIRT(vpu);
> +
> + dev_dbg(vpu->dev, "download extended memory type %x\n",
> + fw_type);
> + memcpy(dest, fw_data + target_size, extra_fw_size);
> + }
> +}
> +
> +int vpu_load_firmware(struct platform_device *pdev)
> +{
> + struct mtk_vpu *vpu = platform_get_drvdata(pdev);
> + struct device *dev = &pdev->dev;
> + struct vpu_run *run = &vpu->run;
> + const struct firmware *vpu_fw;
> + int ret;
> +
> + if (!pdev) {
> + dev_err(dev, "VPU platform device is invalid\n");
> + return -EINVAL;
> + }
> +
> + mutex_lock(&vpu->vpu_mutex);
> +
> + ret = vpu_enable_clock(pdev);
> + if (ret) {
> + dev_err(dev, "enable clock failed %d\n", ret);
> + goto OUT_LOAD_FW;
> + }
> +
> + if (vpu_running(vpu)) {
> + vpu_disable_clock(pdev);
> + mutex_unlock(&vpu->vpu_mutex);
> + dev_warn(dev, "vpu is running already\n");
> + return 0;
> + }
> +
> + run->signaled = false;
> + dev_dbg(vpu->dev, "firmware request\n");
> + ret = request_firmware(&vpu_fw, VPU_P_FW, dev);
> + if (ret < 0) {
> + dev_err(dev, "Failed to load %s, %d\n", VPU_P_FW, ret);
> + goto OUT_LOAD_FW;
> + }
> + if (vpu_fw->size > VPU_P_FW_SIZE) {
> + ret = -EFBIG;
> + dev_err(dev, "program fw size %zu is abnormal\n", vpu_fw->size);
> + goto OUT_LOAD_FW;
> + }
Possibly move request_firmware(), release_firmware() and the associated
error handling into load_requested_vpu(). It can all be parameterized
and the filename of the firmware means error reports will still be clear
about whether the p or d firmware is faulty.
> + dev_dbg(vpu->dev, "Downloaded program fw size: %zu.\n",
> + vpu_fw->size);
> + /* Downloading program firmware to device*/
> + load_requested_vpu(vpu, vpu_fw->size, vpu_fw->data,
> + P_FW);
> + release_firmware(vpu_fw);
> +
> + ret = request_firmware(&vpu_fw, VPU_D_FW, dev);
> + if (ret < 0) {
> + dev_err(dev, "Failed to load %s, %d\n", VPU_D_FW, ret);
> + goto OUT_LOAD_FW;
> + }
> + if (vpu_fw->size > VPU_D_FW_SIZE) {
> + ret = -EFBIG;
> + dev_err(dev, "data fw size %zu is abnormal\n", vpu_fw->size);
> + goto OUT_LOAD_FW;
> + }
> + dev_dbg(vpu->dev, "Downloaded data fw size: %zu.\n",
> + vpu_fw->size);
> + /* Downloading data firmware to device */
> + load_requested_vpu(vpu, vpu_fw->size, vpu_fw->data,
> + D_FW);
> + release_firmware(vpu_fw);
> + /* boot up vpu */
> + vpu_cfg_writel(vpu, 0x1, VPU_BASE);
> +
> + ret = wait_event_interruptible_timeout(run->wq,
> + run->signaled,
> + msecs_to_jiffies(INIT_TIMEOUT_MS)
> + );
> + if (0 == ret) {
> + ret = -ETIME;
> + dev_err(dev, "wait vpu initialization timout!\n");
> + goto OUT_LOAD_FW;
> + } else if (-ERESTARTSYS == ret) {
> + dev_err(dev, "wait vpu interrupted by a signal!\n");
> + goto OUT_LOAD_FW;
> + }
> +
> + ret = 0;
> + dev_info(dev, "vpu is ready. Fw version %s\n", run->fw_ver);
> +
> +OUT_LOAD_FW:
> + vpu_disable_clock(pdev);
> + mutex_unlock(&vpu->vpu_mutex);
> +
> + return ret;
> +}
> +
> +static void vpu_init_ipi_handler(void *data, unsigned int len, void *priv)
> +{
> + struct mtk_vpu *vpu = (struct mtk_vpu *)priv;
> + struct vpu_run *run = (struct vpu_run *)data;
> +
> + vpu->run.signaled = run->signaled;
> + strncpy(vpu->run.fw_ver, run->fw_ver, VPU_FW_VER_LEN);
> + wake_up_interruptible(&vpu->run.wq);
> +}
> +
> +static int vpu_debug_open(struct inode *inode, struct file *file)
> +{
> + file->private_data = inode->i_private;
> + return 0;
> +}
> +
> +static ssize_t vpu_debug_read(struct file *file, char __user *user_buf,
> + size_t count, loff_t *ppos)
> +{
> + char buf[256];
> + unsigned int len;
> + unsigned int running, pc, vpu_to_host, host_to_vpu, wdt;
> + int ret;
> + struct device *dev = file->private_data;
> + struct platform_device *pdev = to_platform_device(dev);
> + struct mtk_vpu *vpu = dev_get_drvdata(dev);
> +
> + ret = vpu_enable_clock(pdev);
> + if (ret) {
> + dev_err(vpu->dev, "[VPU] enable clock failed %d\n", ret);
> + return 0;
> + }
> +
> + /* vpu register status */
> + running = vpu_running(vpu);
> + pc = vpu_cfg_readl(vpu, VPU_PC_REG);
> + wdt = vpu_cfg_readl(vpu, VPU_WDT_REG);
> + host_to_vpu = vpu_cfg_readl(vpu, HOST_TO_VPU);
> + vpu_to_host = vpu_cfg_readl(vpu, VPU_TO_HOST);
> + vpu_disable_clock(pdev);
> +
> + if (running) {
> + len = sprintf(buf, "VPU is running\n\n"
> + "FW Version: %s\n"
> + "PC: 0x%x\n"
> + "WDT: 0x%x\n"
> + "Host to VPU: 0x%x\n"
> + "VPU to Host: 0x%x\n",
> + vpu->run.fw_ver, pc, wdt,
> + host_to_vpu, vpu_to_host);
> + } else {
> + len = sprintf(buf, "VPU not running\n");
> + }
> +
> + return simple_read_from_buffer(user_buf, count, ppos, buf, len);
> +}
> +
> +static const struct file_operations vpu_debug_fops = {
> + .open = vpu_debug_open,
> + .read = vpu_debug_read,
> +};
These operations should be conditional on CONFIG_DEBUG_FS.
> +
> +static void vpu_free_p_ext_mem(struct mtk_vpu *vpu)
> +{
> + struct device *dev = vpu->dev;
> + struct iommu_domain *domain = iommu_get_domain_for_dev(dev);
> +
> + dma_free_coherent(dev, VPU_EXT_P_SIZE, VPU_PMEM0_VIRT(vpu),
> + VPU_PMEM0_IOVA(vpu));
> +
> + if (domain)
> + iommu_detach_device(domain, vpu->dev);
> +}
> +
> +static void vpu_free_d_ext_mem(struct mtk_vpu *vpu)
> +{
> + struct device *dev = vpu->dev;
> + struct iommu_domain *domain = iommu_get_domain_for_dev(dev);
> +
> + dma_free_coherent(dev, VPU_EXT_D_SIZE, VPU_DMEM0_VIRT(vpu),
> + VPU_DMEM0_IOVA(vpu));
> +
> + if (domain)
> + iommu_detach_device(domain, dev);
> +}
Look like this could be parameterized and combined with vpu_free_p_ext_mem.
> +
> +static int vpu_alloc_p_ext_mem(struct mtk_vpu *vpu)
> +{
> + struct device *dev = vpu->dev;
> + struct iommu_domain *domain = iommu_get_domain_for_dev(dev);
> + phys_addr_t p_pa;
> +
> + VPU_PMEM0_VIRT(vpu) = dma_alloc_coherent(dev,
> + VPU_EXT_P_SIZE,
> + &(VPU_PMEM0_IOVA(vpu)),
> + GFP_KERNEL);
> + if (VPU_PMEM0_VIRT(vpu) == NULL) {
> + dev_err(dev, "Failed to allocate the extended program memory\n");
> + return PTR_ERR(VPU_PMEM0_VIRT(vpu));
> + }
> +
> + p_pa = iommu_iova_to_phys(domain, vpu->mem.p_iova);
> + /* Disable extend0. Enable extend1 */
> + vpu_cfg_writel(vpu, 0x1, VPU_PMEM_EXT0_ADDR);
> + vpu_cfg_writel(vpu, (p_pa & 0xFFFFF000), VPU_PMEM_EXT1_ADDR);
> +
> + dev_info(dev, "Program extend memory phy=0x%llx virt=0x%p iova=0x%llx\n",
> + (unsigned long long)p_pa,
> + VPU_PMEM0_VIRT(vpu),
> + (unsigned long long)VPU_PMEM0_IOVA(vpu));
> +
> + return 0;
> +}
> +
> +static int vpu_alloc_d_ext_mem(struct mtk_vpu *vpu)
> +{
> + struct device *dev = vpu->dev;
> + struct iommu_domain *domain = iommu_get_domain_for_dev(dev);
> + phys_addr_t d_pa;
> +
> + VPU_DMEM0_VIRT(vpu) = dma_alloc_coherent(dev,
> + VPU_EXT_D_SIZE,
> + &(VPU_DMEM0_IOVA(vpu)),
> + GFP_KERNEL);
> + if (VPU_DMEM0_VIRT(vpu) == NULL) {
> + dev_err(dev, "Failed to allocate the extended data memory\n");
> + return PTR_ERR(VPU_DMEM0_VIRT(vpu));
> + }
> +
> + d_pa = iommu_iova_to_phys(domain, vpu->mem.d_iova);
> +
> + /* Disable extend0. Enable extend1 */
> + vpu_cfg_writel(vpu, 0x1, VPU_DMEM_EXT0_ADDR);
> + vpu_cfg_writel(vpu, (d_pa & 0xFFFFF000),
> + VPU_DMEM_EXT1_ADDR);
> +
> + dev_info(dev, "Data extend memory phy=0x%llx virt=0x%p iova=0x%llx\n",
> + (unsigned long long)d_pa,
> + VPU_DMEM0_VIRT(vpu),
> + (unsigned long long)VPU_DMEM0_IOVA(vpu));
> +
> + return 0;
> +}
Also looks suitable for parameterizing and combining with
vpu_alloc_p_ext_mem .
> +
> +static void vpu_ipi_handler(struct mtk_vpu *vpu)
> +{
> + struct share_obj *rcv_obj = vpu->recv_buf;
> + struct vpu_ipi_desc *ipi_desc = vpu->ipi_desc;
> +
> + if (rcv_obj->id < IPI_MAX && ipi_desc[rcv_obj->id].handler) {
> + ipi_desc[rcv_obj->id].handler(rcv_obj->share_buf,
> + rcv_obj->len,
> + ipi_desc[rcv_obj->id].priv);
> + } else {
> + dev_err(vpu->dev, "No such ipi id = %d\n", rcv_obj->id);
> + }
> +}
> +
> +static int vpu_ipi_init(struct mtk_vpu *vpu)
> +{
> + /* Disable VPU to host interrupt */
> + vpu_cfg_writel(vpu, 0x0, VPU_TO_HOST);
> +
> + /* shared buffer initialization */
> + vpu->recv_buf = (struct share_obj *)VPU_DTCM(vpu);
> + vpu->send_buf = vpu->recv_buf + 1;
> + memset(vpu->recv_buf, 0, sizeof(struct share_obj));
> + memset(vpu->send_buf, 0, sizeof(struct share_obj));
> + mutex_init(&vpu->vpu_mutex);
> +
> + return 0;
> +}
> +
> +static irqreturn_t vpu_irq_handler(int irq, void *priv)
> +{
> + struct mtk_vpu *vpu = priv;
> + uint32_t vpu_to_host = vpu_cfg_readl(vpu, VPU_TO_HOST);
> +
> + if (vpu_to_host & VPU_IPC_INT)
> + vpu_ipi_handler(vpu);
> + else
> + dev_err(vpu->dev, "vpu watchdog timeout!\n");
> +
> + /* VPU won't send another interrupt until we set VPU_TO_HOST to 0. */
> + vpu_cfg_writel(vpu, 0x0, VPU_TO_HOST);
If we were triggered by a watchdog then how long will it be before the
next watchdog interrupt? Will we end up spamming the logs?
> +
> + return IRQ_HANDLED;
> +}
> +
> +static struct dentry *vpu_debugfs;
> +static int mtk_vpu_probe(struct platform_device *pdev)
> +{
> + struct mtk_vpu *vpu;
> + struct device *dev;
> + struct resource *res;
> + int ret = 0;
> +
> + dev_dbg(&pdev->dev, "initialization\n");
> +
> + dev = &pdev->dev;
> + vpu = devm_kzalloc(dev, sizeof(*vpu), GFP_KERNEL);
> + if (!vpu)
> + return -ENOMEM;
> +
> + vpu->dev = &pdev->dev;
> + res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "sram");
> + vpu->reg.sram = devm_ioremap_resource(dev, res);
> + if (IS_ERR(vpu->reg.sram)) {
> + dev_err(dev, "devm_ioremap_resource vpu sram failed.\n");
> + return PTR_ERR(vpu->reg.sram);
> + }
> +
> + res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "cfg_reg");
> + vpu->reg.cfg = devm_ioremap_resource(dev, res);
> + if (IS_ERR(vpu->reg.cfg)) {
> + dev_err(dev, "devm_ioremap_resource vpu cfg failed.\n");
> + return PTR_ERR(vpu->reg.cfg);
> + }
> +
> + /* Get VPU clock */
> + vpu->clk = devm_clk_get(dev, "main");
> + if (vpu->clk == NULL) {
> + dev_err(dev, "get vpu clock fail\n");
> + return -EINVAL;
> + }
> +
> + platform_set_drvdata(pdev, vpu);
> +
> + ret = vpu_enable_clock(pdev);
> + if (ret) {
> + ret = -EINVAL;
> + return ret;
Why not just "return ret" (or "return -EINVAL")?
> + }
> +
> + dev_dbg(dev, "vpu ipi init\n");
> + ret = vpu_ipi_init(vpu);
> + if (ret) {
> + dev_err(dev, "Failed to init ipi\n");
> + goto disable_vpu_clk;
> + }
> +
> + platform_set_drvdata(pdev, vpu);
> +
> + /* register vpu initialization IPI */
> + ret = vpu_ipi_register(pdev, IPI_VPU_INIT, vpu_init_ipi_handler,
> + "vpu_init", vpu);
> + if (ret) {
> + dev_err(dev, "Failed to register IPI_VPU_INIT\n");
> + goto vpu_mutex_destroy;
> + }
> +
> + vpu_debugfs = debugfs_create_file("mtk_vpu", S_IRUGO, NULL, (void *)dev,
> + &vpu_debug_fops);
> + if (!vpu_debugfs) {
> + ret = -ENOMEM;
> + goto cleanup_ipi;
> + }
> +
> + /* Set PTCM to 96K and DTCM to 32K */
> + vpu_cfg_writel(vpu, 0x2, VPU_TCM_CFG);
> +
> + ret = vpu_alloc_p_ext_mem(vpu);
> + if (ret) {
> + dev_err(dev, "Allocate PM failed\n");
> + goto remove_debugfs;
> + }
> +
> + ret = vpu_alloc_d_ext_mem(vpu);
> + if (ret) {
> + dev_err(dev, "Allocate DM failed\n");
> + goto free_p_mem;
> + }
> +
> + init_waitqueue_head(&vpu->run.wq);
> +
> + res = platform_get_resource(pdev, IORESOURCE_IRQ, 0);
> + if (res == NULL) {
> + dev_err(dev, "get IRQ resource failed.\n");
> + ret = -ENXIO;
> + goto free_d_mem;
> + }
> + vpu->reg.irq = platform_get_irq(pdev, 0);
> + ret = devm_request_irq(dev, vpu->reg.irq, vpu_irq_handler, 0,
> + pdev->name, vpu);
> + if (ret) {
> + dev_err(dev, "failed to request irq\n");
> + goto free_d_mem;
> + }
> +
> + vpu_disable_clock(pdev);
> + dev_dbg(dev, "initialization completed\n");
> +
> + return 0;
> +
> +free_d_mem:
> + vpu_free_d_ext_mem(vpu);
> +free_p_mem:
> + vpu_free_p_ext_mem(vpu);
> +remove_debugfs:
> + debugfs_remove(vpu_debugfs);
> +cleanup_ipi:
> + memset(vpu->ipi_desc, 0, sizeof(struct vpu_ipi_desc)*IPI_MAX);
> +vpu_mutex_destroy:
> + mutex_destroy(&vpu->vpu_mutex);
> +disable_vpu_clk:
> + vpu_disable_clock(pdev);
> +
> + return ret;
> +}
> +
> +static const struct of_device_id mtk_vpu_match[] = {
> + {
> + .compatible = "mediatek,mt8173-vpu",
> + },
> + {},
> +};
> +MODULE_DEVICE_TABLE(of, mtk_vpu_match);
> +
> +static int mtk_vpu_remove(struct platform_device *pdev)
> +{
> + struct mtk_vpu *vpu = platform_get_drvdata(pdev);
> +
> + vpu_free_p_ext_mem(vpu);
> + vpu_free_d_ext_mem(vpu);
This looks like it leaks cpu_debugfs and the vpu_mutex.
> +
> + return 0;
> +}
> +
> +static struct platform_driver mtk_vpu_driver = {
> + .probe = mtk_vpu_probe,
> + .remove = mtk_vpu_remove,
> + .driver = {
> + .name = MTK_VPU_DRV_NAME,
> + .owner = THIS_MODULE,
> + .of_match_table = mtk_vpu_match,
> + },
> +};
> +
> +module_platform_driver(mtk_vpu_driver);
> +
> +MODULE_LICENSE("GPL v2");
> +MODULE_DESCRIPTION("Mediatek Video Prosessor Unit driver");
> diff --git a/drivers/media/platform/mtk-vpu/mtk_vpu_core.h b/drivers/media/platform/mtk-vpu/mtk_vpu_core.h
> new file mode 100644
> index 0000000..20cf2a0
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vpu/mtk_vpu_core.h
> @@ -0,0 +1,161 @@
> +/*
> +* Copyright (c) 2015 MediaTek Inc.
> +* Author: Andrew-CT Chen <andrew-ct.chen-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
> +*
> +* This program is free software; you can redistribute it and/or modify
> +* it under the terms of the GNU General Public License version 2 as
> +* published by the Free Software Foundation.
> +*
> +* This program is distributed in the hope that it will be useful,
> +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> +* GNU General Public License for more details.
> +*/
> +
> +#ifndef _MTK_VPU_CORE_H
> +#define _MTK_VPU_CORE_H
> +
> +#include <linux/platform_device.h>
> +
> +/**
> + * VPU (video processor unit) is a tiny processor controlling video hardware
> + * related to video codec, scaling and color format converting.
> + * VPU interfaces with other blocks by share memory and interrupt.
> + **/
> +
> +typedef void (*ipi_handler_t) (void *data,
> + unsigned int len,
> + void *priv);
> +
> +/**
> + * enum ipi_id - the id of inter-processor interrupt
> + *
> + * @IPI_VPU_INIT: The interrupt from vpu is to notfiy kernel
> + VPU initialization completed.
> + * @IPI_VENC_H264: The interrupt from vpu is to notify kernel to
> + handle H264 video encoder job, and vice versa.
> + * @IPI_VENC_VP8: The interrupt fro vpu is to notify kernel to
> + handle VP8 video encoder job,, and vice versa.
> + * @IPI_VENC_CAPABILITY: The interrupt from vpu is to
> + get venc hardware capability.
> + * @IPI_MAX: The maximum IPI number
> + */
> +enum ipi_id {
> + IPI_VPU_INIT = 0,
> + IPI_VENC_H264,
> + IPI_VENC_VP8,
> + IPI_VENC_CAPABILITY,
> + IPI_MAX,
> +};
> +
> +/**
> + * vpu_disable_clock - Disable VPU clock
> + *
> + * @pdev: VPU platform device
> + *
> + *
> + * Return: Return 0 if the clock is disabled successfully,
> + * otherwise it is failed.
> + *
> + **/
> +void vpu_disable_clock(struct platform_device *pdev);
> +
> +/**
> + * vpu_enable_clock - Enable VPU clock
> + *
> + * @pdev: VPU platform device
> + *
> + * Return: Return 0 if the clock is enabled successfully,
> + * otherwise it is failed.
> + *
> + **/
> +int vpu_enable_clock(struct platform_device *pdev);
> +
> +/**
> + * vpu_ipi_register - register an ipi function
> + *
> + * @pdev: VPU platform device
> + * @id: IPI ID
> + * @handler: IPI handler
> + * @name: IPI name
> + * @priv: private data for IPI handler
> + *
> + * Register an ipi function to receive ipi interrupt from VPU.
> + *
> + * Return: Return 0 if ipi registers successfully, otherwise it is failed.
> + */
> +int vpu_ipi_register(struct platform_device *pdev, enum ipi_id id,
> + ipi_handler_t handler, const char *name, void *priv);
> +
> +/**
> + * vpu_ipi_send - send data from AP to vpu.
> + *
> + * @pdev: VPU platform device
> + * @id: IPI ID
> + * @buf: the data buffer
> + * @len: the data buffer length
> + * @wait: wait for the last ipi completed.
> + *
> + * This function is thread-safe. When this function returns,
> + * VPU has received the data and starts the processing.
> + * When the processing completes, IPI handler registered
> + * by vpu_ipi_register will be called in interrupt context.
> + *
> + * Return: Return 0 if sending data successfully, otherwise it is failed.
> + **/
> +int vpu_ipi_send(struct platform_device *pdev,
> + enum ipi_id id, void *buf,
> + unsigned int len,
> + unsigned int wait);
> +
> +/**
> + * vpu_get_plat_device - get VPU's platform device
> + *
> + * @pdev: the platform device of the module requesting VPU platform
> + * device for using VPU API.
> + *
> + * Return: Return NULL if it is failed.
> + * otherwise it is VPU's platform device
> + **/
> +struct platform_device *vpu_get_plat_device(struct platform_device *pdev);
> +
> +/**
> + * vpu_load_firmware - download VPU firmware and boot it
> + *
> + * @pdev: VPU platform device
> + *
> + * Return: Return 0 if downloading firmware successfully,
> + * otherwise it is failed
> + **/
> +int vpu_load_firmware(struct platform_device *pdev);
> +
> +/**
> + * vpu_mapping_dm_addr - Mapping DTCM/DMEM to kernel virtual address
> + *
> + * @pdev: VPU platform device
> + * @dmem_addr: VPU's data memory address
> + *
> + * Mapping the VPU's DTCM (Data Tightly-Coupled Memory) /
> + * DMEM (Data Extended Memory) memory address to
> + * kernel virtual address.
> + *
> + * Return: Return ERR_PTR(-EINVAL) if mapping failed,
> + * otherwise the mapped kernel virtual address
> + **/
> +void *vpu_mapping_dm_addr(struct platform_device *pdev,
> + void *dtcm_dmem_addr);
> +
> +/**
> + * vpu_mapping_iommu_dm_addr - Mapping to iommu address
> + *
> + * @pdev: VPU platform device
> + * @dmem_addr: VPU's extended data memory address
> + *
> + * Mapping the VPU's extended data address to iommu address
> + *
> + * Return: Return ERR_PTR(-EINVAL) if mapping failed,
> + * otherwise the mapped iommu address
> + **/
> +dma_addr_t *vpu_mapping_iommu_dm_addr(struct platform_device *pdev,
> + void *dmem_addr);
> +#endif /* _MTK_VPU_CORE_H */
> diff --git a/drivers/media/platform/mtk-vpu/vp8_enc/venc_vp8_vpu.h b/drivers/media/platform/mtk-vpu/vp8_enc/venc_vp8_vpu.h
> new file mode 100644
> index 0000000..4e09eec
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vpu/vp8_enc/venc_vp8_vpu.h
Like the H.264 header. Why is this file included in this patch? It is
not included by anything in the patch and defines symbols that do not
exist yet.
Daniel.
--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 84+ messages in thread
* [RESEND RFC/PATCH 3/8] media: platform: mtk-vpu: Support Mediatek VPU
@ 2015-11-25 16:11 ` Daniel Thompson
0 siblings, 0 replies; 84+ messages in thread
From: Daniel Thompson @ 2015-11-25 16:11 UTC (permalink / raw)
To: linux-arm-kernel
On 17/11/15 12:54, Tiffany Lin wrote:
> From: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
>
> The VPU driver for hw video codec embedded in Mediatek's MT8173 SOCs.
> It is able to handle video decoding/encoding of in a range of formats.
> The driver provides with VPU firmware download, memory management and
> the communication interface between CPU and VPU.
> For VPU initialization, it will create virtual memory for CPU access and
> IOMMU address for vcodec hw device access. When a decode/encode instance
> opens a device node, vpu driver will download vpu firmware to the device.
> A decode/encode instant will decode/encode a frame using VPU
> interface to interrupt vpu to handle decoding/encoding jobs.
>
> Signed-off-by: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
> ---
> drivers/media/platform/Kconfig | 6 +
> drivers/media/platform/Makefile | 2 +
> drivers/media/platform/mtk-vpu/Makefile | 1 +
> .../platform/mtk-vpu/h264_enc/venc_h264_vpu.h | 127 +++
> .../media/platform/mtk-vpu/include/venc_ipi_msg.h | 212 +++++
> drivers/media/platform/mtk-vpu/mtk_vpu_core.c | 823 ++++++++++++++++++++
> drivers/media/platform/mtk-vpu/mtk_vpu_core.h | 161 ++++
> .../media/platform/mtk-vpu/vp8_enc/venc_vp8_vpu.h | 119 +++
> 8 files changed, 1451 insertions(+)
> create mode 100644 drivers/media/platform/mtk-vpu/Makefile
> create mode 100644 drivers/media/platform/mtk-vpu/h264_enc/venc_h264_vpu.h
> create mode 100644 drivers/media/platform/mtk-vpu/include/venc_ipi_msg.h
> create mode 100644 drivers/media/platform/mtk-vpu/mtk_vpu_core.c
> create mode 100644 drivers/media/platform/mtk-vpu/mtk_vpu_core.h
> create mode 100644 drivers/media/platform/mtk-vpu/vp8_enc/venc_vp8_vpu.h
>
> diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
> index ccbc974..f98eb47 100644
> --- a/drivers/media/platform/Kconfig
> +++ b/drivers/media/platform/Kconfig
> @@ -148,6 +148,12 @@ config VIDEO_CODA
> Coda is a range of video codec IPs that supports
> H.264, MPEG-4, and other video formats.
>
> +config MEDIATEK_VPU
> + bool "Mediatek Video Processor Unit"
> + ---help---
> + This driver provides downloading firmware vpu and
> + communicating with vpu.
> +
This looks pretty broken.
Shouldn't this option be tristate? Why are there no depends-on or selects?
Also I think the help text could be more descriptive here (and so does
checkpatch ;-) ).
> diff --git a/drivers/media/platform/mtk-vpu/h264_enc/venc_h264_vpu.h b/drivers/media/platform/mtk-vpu/h264_enc/venc_h264_vpu.h
> new file mode 100644
> index 0000000..9c8ebdd
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vpu/h264_enc/venc_h264_vpu.h
Why is this file included in *this* patch? It is not included by
anything in the patch and defines functions that do not exist yet.
> diff --git a/drivers/media/platform/mtk-vpu/include/venc_ipi_msg.h b/drivers/media/platform/mtk-vpu/include/venc_ipi_msg.h
> new file mode 100644
> index 0000000..a345b98
This file also is not included by anything and should, perhaps be
included in a different patch.
> diff --git a/drivers/media/platform/mtk-vpu/mtk_vpu_core.c b/drivers/media/platform/mtk-vpu/mtk_vpu_core.c
> new file mode 100644
> index 0000000..b524dbc
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vpu/mtk_vpu_core.c
> @@ -0,0 +1,823 @@
> +/*
> +* Copyright (c) 2015 MediaTek Inc.
> +* Author: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
> +*
> +* This program is free software; you can redistribute it and/or modify
> +* it under the terms of the GNU General Public License version 2 as
> +* published by the Free Software Foundation.
> +*
> +* This program is distributed in the hope that it will be useful,
> +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> +* GNU General Public License for more details.
> +*/
> +#include <linux/clk.h>
> +#include <linux/debugfs.h>
> +#include <linux/firmware.h>
> +#include <linux/interrupt.h>
> +#include <linux/iommu.h>
> +#include <linux/module.h>
> +#include <linux/of_address.h>
> +#include <linux/of_irq.h>
> +#include <linux/of_platform.h>
> +#include <linux/sched.h>
> +#include <linux/sizes.h>
> +
> +#include "mtk_vpu_core.h"
> +
> +/**
> + * VPU (video processor unit) is a tiny processor controlling video hardware
> + * related to video codec, scaling and color format converting.
> + * VPU interfaces with other blocks by share memory and interrupt.
> + **/
> +#define MTK_VPU_DRV_NAME "mtk_vpu"
This is only used once, why not just put this string directly into the
.name field?
> +
> +#define INIT_TIMEOUT_MS 2000U
> +#define IPI_TIMEOUT_MS 2000U
> +#define VPU_FW_VER_LEN 16
> +
> +/* vpu extended virtural address */
> +#define VPU_PMEM0_VIRT(vpu) ((vpu)->mem.p_va)
> +#define VPU_DMEM0_VIRT(vpu) ((vpu)->mem.d_va)
> +/* vpu extended iova address*/
> +#define VPU_PMEM0_IOVA(vpu) ((vpu)->mem.p_iova)
> +#define VPU_DMEM0_IOVA(vpu) ((vpu)->mem.d_iova)
These feel like obfuscation to me. The code looks clearer without the
macro For example:
vpu->mem.p_va = dma_alloc_coherent(dev, ...);
Is much clearer than:
VPU_PMEM0_VIRT(vpu) = dma_alloc_coherent(dev, ...);
> +
> +#define VPU_PTCM(dev) ((dev)->reg.sram)
> +#define VPU_DTCM(dev) ((dev)->reg.sram + VPU_DTCM_OFFSET)
These macros also seem to add very little value. They are consumed only
a couple of times and only serve to conceal how the sram is mapped and
consumed:
dest = vpu->reg.sram;
if (fw_type == D_FW)
dest += VPU_DTCM_OFFSET;
Compared to:
dest = fw_type ? VPU_DTCM(vpu) : VPU_PTCM(vpu);
> +
> +#define VPU_PTCM_SIZE (96 * SZ_1K)
> +#define VPU_DTCM_SIZE (32 * SZ_1K)
> +#define VPU_DTCM_OFFSET 0x18000UL
> +#define VPU_EXT_P_SIZE SZ_1M
> +#define VPU_EXT_D_SIZE SZ_4M
> +#define VPU_P_FW_SIZE (VPU_PTCM_SIZE + VPU_EXT_P_SIZE)
> +#define VPU_D_FW_SIZE (VPU_DTCM_SIZE + VPU_EXT_D_SIZE)
> +#define SHARE_BUF_SIZE 48
> +
> +#define VPU_P_FW "vpu_p.bin"
> +#define VPU_D_FW "vpu_d.bin"
These macros are of questionable value.
> +
> +#define VPU_BASE 0x0
Is this register really called "base" in the datasheet? From the use in
the code it looks like it performs a reset and/or PM function.
> +#define VPU_TCM_CFG 0x0008
> +#define VPU_PMEM_EXT0_ADDR 0x000C
> +#define VPU_PMEM_EXT1_ADDR 0x0010
> +#define VPU_TO_HOST 0x001C
> +#define VPU_DMEM_EXT0_ADDR 0x0014
> +#define VPU_DMEM_EXT1_ADDR 0x0018
> +#define HOST_TO_VPU 0x0024
> +#define VPU_PC_REG 0x0060
> +#define VPU_WDT_REG 0x0084
> +
> +/* vpu inter-processor communication interrupt */
> +#define VPU_IPC_INT BIT(8)
> +
> +/**
> + * enum vpu_fw_type - VPU firmware type
> + *
> + * @P_FW: program firmware
> + * @D_FW: data firmware
> + *
> + */
> +enum vpu_fw_type {
> + P_FW,
> + D_FW,
> +};
> +
> +/**
> + * struct vpu_mem - VPU memory information
Perhaps "VPU extended program/data memory information" instead.
> + *
> + * @p_va: the kernel virtual memory address of
> + * VPU extended program memory
> + * @d_va: the kernel virtual memory address of VPU extended data memory
> + * @p_iova: the iova memory address of VPU extended program memory
> + * @d_iova: the iova memory address of VPU extended data memory
> + */
> +struct vpu_mem {
> + void *p_va;
> + void *d_va;
> + dma_addr_t p_iova;
> + dma_addr_t d_iova;
> +};
Might be better as:
struct {
void *va;
dma_addr_t iova;
}
This can be then be used as: vpu->mem[P_FW].va . This doesn't matter
much yet but it would helps us common up some code later.
> +
> +/**
> + * struct vpu_regs - VPU SRAM and configuration registers
> + *
> + * @sram: the register for VPU sram
> + * @cfg: the register for VPU configuration
> + * @irq: the irq number for VPU interrupt
> + */
> +struct vpu_regs {
> + void __iomem *sram;
This is called TCM everywhere else. Why a different name for it here?
> + void __iomem *cfg;
> + int irq;
> +};
> +
> +/**
> + * struct vpu_run - VPU initialization status
> + *
> + * @signaled: the signal of vpu initialization completed
> + * @fw_ver: VPU firmware version
> + * @wq: wait queue for VPU initialization status
> + */
> +struct vpu_run {
> + u32 signaled;
> + char fw_ver[VPU_FW_VER_LEN];
> + wait_queue_head_t wq;
> +};
> +
> +/**
> + * struct vpu_ipi_desc - VPU IPI descriptor
> + *
> + * @handler: IPI handler
> + * @name: the name of IPI handler
> + * @priv: the private data of IPI handler
> + */
> +struct vpu_ipi_desc {
> + ipi_handler_t handler;
> + const char *name;
> + void *priv;
> +};
> +
> +/**
> + * struct share_obj - The DTCM (Data Tightly-Coupled Memory) buffer shared with
> + * AP and VPU
Remove "The" (there are more than one of these buffers).
> + *
> + * @id: IPI id
> + * @len: share buffer length
> + * @share_buf: share buffer data
> + */
> +struct share_obj {
> + int32_t id;
> + uint32_t len;
> + unsigned char share_buf[SHARE_BUF_SIZE];
> +};
> +
> +/**
> + * struct mtk_vpu - vpu driver data
> + * @mem: VPU extended memory information
> + * @reg: VPU SRAM and configuration registers
> + * @run: VPU initialization status
> + * @ipi_desc: VPU IPI descriptor
> + * @recv_buf: VPU DTCM share buffer for receiving. The
> + * receive buffer is only accessed in interrupt context.
> + * @send_buf: VPU DTCM share buffer for sending
> + * @dev: VPU struct device
> + * @clk: VPU clock on/off
> + * @vpu_mutex: protect mtk_vpu (except recv_buf) and ensure only
> + * one client to use VPU service at a time. For example,
> + * suppose a client is using VPU to decode VP8.
> + * If the other client wants to encode VP8,
> + * it has to wait until VP8 decode completes.
> + *
> + */
> +struct mtk_vpu {
> + struct vpu_mem mem;
Rename to extmem?
> + struct vpu_regs reg;
> + struct vpu_run run;
> + struct vpu_ipi_desc ipi_desc[IPI_MAX];
> + struct share_obj *recv_buf;
> + struct share_obj *send_buf;
> + struct device *dev;
> + struct clk *clk;
> + struct mutex vpu_mutex; /* for protecting vpu data data structure */
> +};
> +
> +/* the thread calls the function should hold the |vpu_mutex| */
Remove this comment. Its unhelpful: the code does not meet this
requirement because it overstates the scope of vpu_mutex .
> +static inline void vpu_cfg_writel(struct mtk_vpu *vpu, u32 val, u32 offset)
> +{
> + writel(val, vpu->reg.cfg + offset);
> +}
> +
> +static inline u32 vpu_cfg_readl(struct mtk_vpu *vpu, u32 offset)
> +{
> + return readl(vpu->reg.cfg + offset);
> +}
> +
> +static inline bool vpu_running(struct mtk_vpu *vpu)
> +{
> + return vpu_cfg_readl(vpu, VPU_BASE) & BIT(0);
> +}
> +
> +void vpu_disable_clock(struct platform_device *pdev)
> +{
> + struct mtk_vpu *vpu = platform_get_drvdata(pdev);
> +
> + /* Disable VPU watchdog */
> + vpu_cfg_writel(vpu,
> + vpu_cfg_readl(vpu, VPU_WDT_REG) & ~(1L<<31),
> + VPU_WDT_REG);
This code combines a reference counted clock disable with a
not-reference counted register write and will result in the watchdot
being spuriously disabled.
This will definitely happen if vpu_debug_read() is called at the wrong
time, possibly may also be an issue for concurrent H.264 and VP8 operations.
> +
> + clk_disable_unprepare(vpu->clk);
> +}
> +
> +int vpu_enable_clock(struct platform_device *pdev)
> +{
> + struct mtk_vpu *vpu = platform_get_drvdata(pdev);
> + int ret;
> +
> + ret = clk_prepare_enable(vpu->clk);
> + if (ret)
> + return ret;
> + /* Enable VPU watchdog */
> + vpu_cfg_writel(vpu, vpu_cfg_readl(vpu, VPU_WDT_REG) | (1L << 31),
> + VPU_WDT_REG);
As above.
> +
> + return ret;
> +}
> +
> +int vpu_ipi_register(struct platform_device *pdev,
> + enum ipi_id id, ipi_handler_t handler,
> + const char *name, void *priv)
> +{
> + struct mtk_vpu *vpu = platform_get_drvdata(pdev);
> + struct vpu_ipi_desc *ipi_desc;
> +
> + if (!vpu) {
> + dev_err(&pdev->dev, "vpu device in not ready\n");
> + return -EPROBE_DEFER;
> + }
> +
> + if (id < IPI_MAX && handler != NULL) {
> + ipi_desc = vpu->ipi_desc;
> + ipi_desc[id].name = name;
> + ipi_desc[id].handler = handler;
> + ipi_desc[id].priv = priv;
> + return 0;
> + }
> +
> + dev_err(&pdev->dev, "register vpu ipi with invalid arguments\n");
> + return -EINVAL;
> +}
This API, and manu of its friends lower down the file, appear to be a
way to send 32 byte messages to different endpoints on another processor
on the SoC.
Just interested to know if you evaluated the mailbox driver
infrastructure for this? Its a good fit for the messaging but doesn't
offer any support for the direct access to TCM or extended memory.
> +int vpu_ipi_send(struct platform_device *pdev,
> + enum ipi_id id, void *buf,
> + unsigned int len, unsigned int wait)
> +{
> + struct mtk_vpu *vpu = platform_get_drvdata(pdev);
> + struct share_obj *send_obj = vpu->send_buf;
> + unsigned long timeout;
> +
> + if (id >= IPI_MAX || len > sizeof(send_obj->share_buf) || buf == NULL) {
> + dev_err(vpu->dev, "failed to send ipi message\n");
> + return -EINVAL;
> + }
> +
> + if (!vpu_running(vpu)) {
> + dev_err(vpu->dev, "vpu_ipi_send: VPU is not running\n");
> + return -ENXIO;
> + }
> +
> + mutex_lock(&vpu->vpu_mutex);
> + if (vpu_cfg_readl(vpu, HOST_TO_VPU) && !wait) {
> + mutex_unlock(&vpu->vpu_mutex);
> + return -EBUSY;
> + }
This branch is unreachable (no caller ever sets wait is never set to false).
> +
> + if (wait)
> + while (vpu_cfg_readl(vpu, HOST_TO_VPU))
> + ;
What is this loop for? This code should only be reachable if we timed
out in the code below so the likely effect of this loop is to
permantently wedge a thread in a manner that frustrates signal delivery.
> +
> + memcpy((void *)send_obj->share_buf, buf, len);
> + send_obj->len = len;
> + send_obj->id = id;
> + vpu_cfg_writel(vpu, 0x1, HOST_TO_VPU);
> +
> + /* Wait until VPU receives the command */
> + timeout = jiffies + msecs_to_jiffies(IPI_TIMEOUT_MS);
> + do {
> + if (time_after(jiffies, timeout)) {
> + dev_err(vpu->dev, "vpu_ipi_send: IPI timeout!\n");
> + return -EIO;
> + }
> + } while (vpu_cfg_readl(vpu, HOST_TO_VPU));
Do we need to busy wait every time we communicate with the co-processor?
Couldn't we put this wait *before* we write to HOST_TO_VPU above.
That way we only spin when there is a need to.
> +
> + mutex_unlock(&vpu->vpu_mutex);
> +
> + return 0;
> +}
> +
> +void *vpu_mapping_dm_addr(struct platform_device *pdev,
> + void *dtcm_dmem_addr)
Use a different type: dtcm_dmem_addr is not a pointer on this CPU.
> +{
> + struct mtk_vpu *vpu = platform_get_drvdata(pdev);
> + unsigned long p_vpu_dtcm = (unsigned long)VPU_DTCM(vpu);
p_vpu_dtcm *is* a pointer on this CPU. Why cast it so that it isn't.
> + unsigned long ul_dtcm_dmem_addr = (unsigned long)(dtcm_dmem_addr);
> +
> + if (dtcm_dmem_addr == NULL ||
> + (ul_dtcm_dmem_addr > (VPU_DTCM_SIZE + VPU_EXT_D_SIZE))) {
> + dev_err(vpu->dev, "invalid virtual data memory address\n");
> + return ERR_PTR(-EINVAL);
> + }
> +
> + if (ul_dtcm_dmem_addr < VPU_DTCM_SIZE)
> + return (void *)(ul_dtcm_dmem_addr + p_vpu_dtcm);
Starting with the pointer will preserve type better:
return vpu->reg.sram + VPU_DTCM_OFFSET + ul_dtcm_dmem_addr;
> +
> + return (void *)((ul_dtcm_dmem_addr - VPU_DTCM_SIZE) +
> + VPU_DMEM0_VIRT(vpu));
Likewise this code is clearer with the pointer first.
return vpu->mem.d_va + (ul_dtcm_dmem_addr - VPU_DTCM_SIZE);
> +}
> +
> +dma_addr_t *vpu_mapping_iommu_dm_addr(struct platform_device *pdev,
> + void *dmem_addr)
This function does not return a pointer to a dma_addr_t. It just returns
a regular dma_addr_t (that's why all the callers of this function cast
it back ;-) ).
dmem_addr is also not a pointer and this function does not return a
pointer to a dma_addr_t. It just returns a regular dma_addr_t.
> +{
> + unsigned long ul_dmem_addr = (unsigned long)(dmem_addr);
> + struct mtk_vpu *vpu = platform_get_drvdata(pdev);
> +
> + if (dmem_addr == NULL ||
> + (ul_dmem_addr < VPU_DTCM_SIZE) ||
> + (ul_dmem_addr > (VPU_DTCM_SIZE + VPU_EXT_D_SIZE))) {
> + dev_err(vpu->dev, "invalid IOMMU data memory address\n");
> + return ERR_PTR(-EINVAL);
> + }
> +
> + return (dma_addr_t *)((ul_dmem_addr - VPU_DTCM_SIZE) +
> + VPU_DMEM0_IOVA(vpu));
Better written as (this would also have made explicit the type problem
with the function:
return vpu->mem.d_iova + (ul_dmem_addr - VPU_DTCM_SIZE);
> +}
> +
> +struct platform_device *vpu_get_plat_device(struct platform_device *pdev)
> +{
> + struct device *dev = &pdev->dev;
> + struct device_node *vpu_node;
> + struct platform_device *vpu_pdev;
> +
> + vpu_node = of_parse_phandle(dev->of_node, "vpu", 0);
> + if (!vpu_node) {
> + dev_err(dev, "can't get vpu node\n");
> + return NULL;
> + }
> +
> + vpu_pdev = of_find_device_by_node(vpu_node);
> + if (WARN_ON(!vpu_pdev)) {
> + dev_err(dev, "vpu pdev failed\n");
> + of_node_put(vpu_node);
> + return NULL;
> + }
> +
> + return vpu_pdev;
> +}
This function looks a bit weird to me. Why do we need to keep consulting
the devicetree every time we want to start/stop the clock?
I would have expected this code to be part of the init routine of
anything that needs this and the platform_device would then be cached.
> +
> +/* load vpu program/data memory */
> +static void load_requested_vpu(struct mtk_vpu *vpu,
> + size_t fw_size,
> + const u8 *fw_data,
> + u8 fw_type)
> +{
> + size_t target_size = fw_type ? VPU_DTCM_SIZE : VPU_PTCM_SIZE;
> + size_t extra_fw_size = 0;
> + void *dest;
> +
> + /* reset VPU */
> + vpu_cfg_writel(vpu, 0x0, VPU_BASE);
> +
> + /* handle extended firmware size */
> + if (fw_size > target_size) {
> + dev_dbg(vpu->dev, "fw size %lx > limited fw size %lx\n",
> + fw_size, target_size);
> + extra_fw_size = fw_size - target_size;
> + dev_dbg(vpu->dev, "extra_fw_size %lx\n", extra_fw_size);
> + fw_size = target_size;
> + }
> + dest = fw_type ? VPU_DTCM(vpu) : VPU_PTCM(vpu);
> + memcpy(dest, fw_data, fw_size);
> + /* download to extended memory if need */
> + if (extra_fw_size > 0) {
> + dest = fw_type ?
> + VPU_DMEM0_VIRT(vpu) : VPU_PMEM0_VIRT(vpu);
> +
> + dev_dbg(vpu->dev, "download extended memory type %x\n",
> + fw_type);
> + memcpy(dest, fw_data + target_size, extra_fw_size);
> + }
> +}
> +
> +int vpu_load_firmware(struct platform_device *pdev)
> +{
> + struct mtk_vpu *vpu = platform_get_drvdata(pdev);
> + struct device *dev = &pdev->dev;
> + struct vpu_run *run = &vpu->run;
> + const struct firmware *vpu_fw;
> + int ret;
> +
> + if (!pdev) {
> + dev_err(dev, "VPU platform device is invalid\n");
> + return -EINVAL;
> + }
> +
> + mutex_lock(&vpu->vpu_mutex);
> +
> + ret = vpu_enable_clock(pdev);
> + if (ret) {
> + dev_err(dev, "enable clock failed %d\n", ret);
> + goto OUT_LOAD_FW;
> + }
> +
> + if (vpu_running(vpu)) {
> + vpu_disable_clock(pdev);
> + mutex_unlock(&vpu->vpu_mutex);
> + dev_warn(dev, "vpu is running already\n");
> + return 0;
> + }
> +
> + run->signaled = false;
> + dev_dbg(vpu->dev, "firmware request\n");
> + ret = request_firmware(&vpu_fw, VPU_P_FW, dev);
> + if (ret < 0) {
> + dev_err(dev, "Failed to load %s, %d\n", VPU_P_FW, ret);
> + goto OUT_LOAD_FW;
> + }
> + if (vpu_fw->size > VPU_P_FW_SIZE) {
> + ret = -EFBIG;
> + dev_err(dev, "program fw size %zu is abnormal\n", vpu_fw->size);
> + goto OUT_LOAD_FW;
> + }
Possibly move request_firmware(), release_firmware() and the associated
error handling into load_requested_vpu(). It can all be parameterized
and the filename of the firmware means error reports will still be clear
about whether the p or d firmware is faulty.
> + dev_dbg(vpu->dev, "Downloaded program fw size: %zu.\n",
> + vpu_fw->size);
> + /* Downloading program firmware to device*/
> + load_requested_vpu(vpu, vpu_fw->size, vpu_fw->data,
> + P_FW);
> + release_firmware(vpu_fw);
> +
> + ret = request_firmware(&vpu_fw, VPU_D_FW, dev);
> + if (ret < 0) {
> + dev_err(dev, "Failed to load %s, %d\n", VPU_D_FW, ret);
> + goto OUT_LOAD_FW;
> + }
> + if (vpu_fw->size > VPU_D_FW_SIZE) {
> + ret = -EFBIG;
> + dev_err(dev, "data fw size %zu is abnormal\n", vpu_fw->size);
> + goto OUT_LOAD_FW;
> + }
> + dev_dbg(vpu->dev, "Downloaded data fw size: %zu.\n",
> + vpu_fw->size);
> + /* Downloading data firmware to device */
> + load_requested_vpu(vpu, vpu_fw->size, vpu_fw->data,
> + D_FW);
> + release_firmware(vpu_fw);
> + /* boot up vpu */
> + vpu_cfg_writel(vpu, 0x1, VPU_BASE);
> +
> + ret = wait_event_interruptible_timeout(run->wq,
> + run->signaled,
> + msecs_to_jiffies(INIT_TIMEOUT_MS)
> + );
> + if (0 == ret) {
> + ret = -ETIME;
> + dev_err(dev, "wait vpu initialization timout!\n");
> + goto OUT_LOAD_FW;
> + } else if (-ERESTARTSYS == ret) {
> + dev_err(dev, "wait vpu interrupted by a signal!\n");
> + goto OUT_LOAD_FW;
> + }
> +
> + ret = 0;
> + dev_info(dev, "vpu is ready. Fw version %s\n", run->fw_ver);
> +
> +OUT_LOAD_FW:
> + vpu_disable_clock(pdev);
> + mutex_unlock(&vpu->vpu_mutex);
> +
> + return ret;
> +}
> +
> +static void vpu_init_ipi_handler(void *data, unsigned int len, void *priv)
> +{
> + struct mtk_vpu *vpu = (struct mtk_vpu *)priv;
> + struct vpu_run *run = (struct vpu_run *)data;
> +
> + vpu->run.signaled = run->signaled;
> + strncpy(vpu->run.fw_ver, run->fw_ver, VPU_FW_VER_LEN);
> + wake_up_interruptible(&vpu->run.wq);
> +}
> +
> +static int vpu_debug_open(struct inode *inode, struct file *file)
> +{
> + file->private_data = inode->i_private;
> + return 0;
> +}
> +
> +static ssize_t vpu_debug_read(struct file *file, char __user *user_buf,
> + size_t count, loff_t *ppos)
> +{
> + char buf[256];
> + unsigned int len;
> + unsigned int running, pc, vpu_to_host, host_to_vpu, wdt;
> + int ret;
> + struct device *dev = file->private_data;
> + struct platform_device *pdev = to_platform_device(dev);
> + struct mtk_vpu *vpu = dev_get_drvdata(dev);
> +
> + ret = vpu_enable_clock(pdev);
> + if (ret) {
> + dev_err(vpu->dev, "[VPU] enable clock failed %d\n", ret);
> + return 0;
> + }
> +
> + /* vpu register status */
> + running = vpu_running(vpu);
> + pc = vpu_cfg_readl(vpu, VPU_PC_REG);
> + wdt = vpu_cfg_readl(vpu, VPU_WDT_REG);
> + host_to_vpu = vpu_cfg_readl(vpu, HOST_TO_VPU);
> + vpu_to_host = vpu_cfg_readl(vpu, VPU_TO_HOST);
> + vpu_disable_clock(pdev);
> +
> + if (running) {
> + len = sprintf(buf, "VPU is running\n\n"
> + "FW Version: %s\n"
> + "PC: 0x%x\n"
> + "WDT: 0x%x\n"
> + "Host to VPU: 0x%x\n"
> + "VPU to Host: 0x%x\n",
> + vpu->run.fw_ver, pc, wdt,
> + host_to_vpu, vpu_to_host);
> + } else {
> + len = sprintf(buf, "VPU not running\n");
> + }
> +
> + return simple_read_from_buffer(user_buf, count, ppos, buf, len);
> +}
> +
> +static const struct file_operations vpu_debug_fops = {
> + .open = vpu_debug_open,
> + .read = vpu_debug_read,
> +};
These operations should be conditional on CONFIG_DEBUG_FS.
> +
> +static void vpu_free_p_ext_mem(struct mtk_vpu *vpu)
> +{
> + struct device *dev = vpu->dev;
> + struct iommu_domain *domain = iommu_get_domain_for_dev(dev);
> +
> + dma_free_coherent(dev, VPU_EXT_P_SIZE, VPU_PMEM0_VIRT(vpu),
> + VPU_PMEM0_IOVA(vpu));
> +
> + if (domain)
> + iommu_detach_device(domain, vpu->dev);
> +}
> +
> +static void vpu_free_d_ext_mem(struct mtk_vpu *vpu)
> +{
> + struct device *dev = vpu->dev;
> + struct iommu_domain *domain = iommu_get_domain_for_dev(dev);
> +
> + dma_free_coherent(dev, VPU_EXT_D_SIZE, VPU_DMEM0_VIRT(vpu),
> + VPU_DMEM0_IOVA(vpu));
> +
> + if (domain)
> + iommu_detach_device(domain, dev);
> +}
Look like this could be parameterized and combined with vpu_free_p_ext_mem.
> +
> +static int vpu_alloc_p_ext_mem(struct mtk_vpu *vpu)
> +{
> + struct device *dev = vpu->dev;
> + struct iommu_domain *domain = iommu_get_domain_for_dev(dev);
> + phys_addr_t p_pa;
> +
> + VPU_PMEM0_VIRT(vpu) = dma_alloc_coherent(dev,
> + VPU_EXT_P_SIZE,
> + &(VPU_PMEM0_IOVA(vpu)),
> + GFP_KERNEL);
> + if (VPU_PMEM0_VIRT(vpu) == NULL) {
> + dev_err(dev, "Failed to allocate the extended program memory\n");
> + return PTR_ERR(VPU_PMEM0_VIRT(vpu));
> + }
> +
> + p_pa = iommu_iova_to_phys(domain, vpu->mem.p_iova);
> + /* Disable extend0. Enable extend1 */
> + vpu_cfg_writel(vpu, 0x1, VPU_PMEM_EXT0_ADDR);
> + vpu_cfg_writel(vpu, (p_pa & 0xFFFFF000), VPU_PMEM_EXT1_ADDR);
> +
> + dev_info(dev, "Program extend memory phy=0x%llx virt=0x%p iova=0x%llx\n",
> + (unsigned long long)p_pa,
> + VPU_PMEM0_VIRT(vpu),
> + (unsigned long long)VPU_PMEM0_IOVA(vpu));
> +
> + return 0;
> +}
> +
> +static int vpu_alloc_d_ext_mem(struct mtk_vpu *vpu)
> +{
> + struct device *dev = vpu->dev;
> + struct iommu_domain *domain = iommu_get_domain_for_dev(dev);
> + phys_addr_t d_pa;
> +
> + VPU_DMEM0_VIRT(vpu) = dma_alloc_coherent(dev,
> + VPU_EXT_D_SIZE,
> + &(VPU_DMEM0_IOVA(vpu)),
> + GFP_KERNEL);
> + if (VPU_DMEM0_VIRT(vpu) == NULL) {
> + dev_err(dev, "Failed to allocate the extended data memory\n");
> + return PTR_ERR(VPU_DMEM0_VIRT(vpu));
> + }
> +
> + d_pa = iommu_iova_to_phys(domain, vpu->mem.d_iova);
> +
> + /* Disable extend0. Enable extend1 */
> + vpu_cfg_writel(vpu, 0x1, VPU_DMEM_EXT0_ADDR);
> + vpu_cfg_writel(vpu, (d_pa & 0xFFFFF000),
> + VPU_DMEM_EXT1_ADDR);
> +
> + dev_info(dev, "Data extend memory phy=0x%llx virt=0x%p iova=0x%llx\n",
> + (unsigned long long)d_pa,
> + VPU_DMEM0_VIRT(vpu),
> + (unsigned long long)VPU_DMEM0_IOVA(vpu));
> +
> + return 0;
> +}
Also looks suitable for parameterizing and combining with
vpu_alloc_p_ext_mem .
> +
> +static void vpu_ipi_handler(struct mtk_vpu *vpu)
> +{
> + struct share_obj *rcv_obj = vpu->recv_buf;
> + struct vpu_ipi_desc *ipi_desc = vpu->ipi_desc;
> +
> + if (rcv_obj->id < IPI_MAX && ipi_desc[rcv_obj->id].handler) {
> + ipi_desc[rcv_obj->id].handler(rcv_obj->share_buf,
> + rcv_obj->len,
> + ipi_desc[rcv_obj->id].priv);
> + } else {
> + dev_err(vpu->dev, "No such ipi id = %d\n", rcv_obj->id);
> + }
> +}
> +
> +static int vpu_ipi_init(struct mtk_vpu *vpu)
> +{
> + /* Disable VPU to host interrupt */
> + vpu_cfg_writel(vpu, 0x0, VPU_TO_HOST);
> +
> + /* shared buffer initialization */
> + vpu->recv_buf = (struct share_obj *)VPU_DTCM(vpu);
> + vpu->send_buf = vpu->recv_buf + 1;
> + memset(vpu->recv_buf, 0, sizeof(struct share_obj));
> + memset(vpu->send_buf, 0, sizeof(struct share_obj));
> + mutex_init(&vpu->vpu_mutex);
> +
> + return 0;
> +}
> +
> +static irqreturn_t vpu_irq_handler(int irq, void *priv)
> +{
> + struct mtk_vpu *vpu = priv;
> + uint32_t vpu_to_host = vpu_cfg_readl(vpu, VPU_TO_HOST);
> +
> + if (vpu_to_host & VPU_IPC_INT)
> + vpu_ipi_handler(vpu);
> + else
> + dev_err(vpu->dev, "vpu watchdog timeout!\n");
> +
> + /* VPU won't send another interrupt until we set VPU_TO_HOST to 0. */
> + vpu_cfg_writel(vpu, 0x0, VPU_TO_HOST);
If we were triggered by a watchdog then how long will it be before the
next watchdog interrupt? Will we end up spamming the logs?
> +
> + return IRQ_HANDLED;
> +}
> +
> +static struct dentry *vpu_debugfs;
> +static int mtk_vpu_probe(struct platform_device *pdev)
> +{
> + struct mtk_vpu *vpu;
> + struct device *dev;
> + struct resource *res;
> + int ret = 0;
> +
> + dev_dbg(&pdev->dev, "initialization\n");
> +
> + dev = &pdev->dev;
> + vpu = devm_kzalloc(dev, sizeof(*vpu), GFP_KERNEL);
> + if (!vpu)
> + return -ENOMEM;
> +
> + vpu->dev = &pdev->dev;
> + res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "sram");
> + vpu->reg.sram = devm_ioremap_resource(dev, res);
> + if (IS_ERR(vpu->reg.sram)) {
> + dev_err(dev, "devm_ioremap_resource vpu sram failed.\n");
> + return PTR_ERR(vpu->reg.sram);
> + }
> +
> + res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "cfg_reg");
> + vpu->reg.cfg = devm_ioremap_resource(dev, res);
> + if (IS_ERR(vpu->reg.cfg)) {
> + dev_err(dev, "devm_ioremap_resource vpu cfg failed.\n");
> + return PTR_ERR(vpu->reg.cfg);
> + }
> +
> + /* Get VPU clock */
> + vpu->clk = devm_clk_get(dev, "main");
> + if (vpu->clk == NULL) {
> + dev_err(dev, "get vpu clock fail\n");
> + return -EINVAL;
> + }
> +
> + platform_set_drvdata(pdev, vpu);
> +
> + ret = vpu_enable_clock(pdev);
> + if (ret) {
> + ret = -EINVAL;
> + return ret;
Why not just "return ret" (or "return -EINVAL")?
> + }
> +
> + dev_dbg(dev, "vpu ipi init\n");
> + ret = vpu_ipi_init(vpu);
> + if (ret) {
> + dev_err(dev, "Failed to init ipi\n");
> + goto disable_vpu_clk;
> + }
> +
> + platform_set_drvdata(pdev, vpu);
> +
> + /* register vpu initialization IPI */
> + ret = vpu_ipi_register(pdev, IPI_VPU_INIT, vpu_init_ipi_handler,
> + "vpu_init", vpu);
> + if (ret) {
> + dev_err(dev, "Failed to register IPI_VPU_INIT\n");
> + goto vpu_mutex_destroy;
> + }
> +
> + vpu_debugfs = debugfs_create_file("mtk_vpu", S_IRUGO, NULL, (void *)dev,
> + &vpu_debug_fops);
> + if (!vpu_debugfs) {
> + ret = -ENOMEM;
> + goto cleanup_ipi;
> + }
> +
> + /* Set PTCM to 96K and DTCM to 32K */
> + vpu_cfg_writel(vpu, 0x2, VPU_TCM_CFG);
> +
> + ret = vpu_alloc_p_ext_mem(vpu);
> + if (ret) {
> + dev_err(dev, "Allocate PM failed\n");
> + goto remove_debugfs;
> + }
> +
> + ret = vpu_alloc_d_ext_mem(vpu);
> + if (ret) {
> + dev_err(dev, "Allocate DM failed\n");
> + goto free_p_mem;
> + }
> +
> + init_waitqueue_head(&vpu->run.wq);
> +
> + res = platform_get_resource(pdev, IORESOURCE_IRQ, 0);
> + if (res == NULL) {
> + dev_err(dev, "get IRQ resource failed.\n");
> + ret = -ENXIO;
> + goto free_d_mem;
> + }
> + vpu->reg.irq = platform_get_irq(pdev, 0);
> + ret = devm_request_irq(dev, vpu->reg.irq, vpu_irq_handler, 0,
> + pdev->name, vpu);
> + if (ret) {
> + dev_err(dev, "failed to request irq\n");
> + goto free_d_mem;
> + }
> +
> + vpu_disable_clock(pdev);
> + dev_dbg(dev, "initialization completed\n");
> +
> + return 0;
> +
> +free_d_mem:
> + vpu_free_d_ext_mem(vpu);
> +free_p_mem:
> + vpu_free_p_ext_mem(vpu);
> +remove_debugfs:
> + debugfs_remove(vpu_debugfs);
> +cleanup_ipi:
> + memset(vpu->ipi_desc, 0, sizeof(struct vpu_ipi_desc)*IPI_MAX);
> +vpu_mutex_destroy:
> + mutex_destroy(&vpu->vpu_mutex);
> +disable_vpu_clk:
> + vpu_disable_clock(pdev);
> +
> + return ret;
> +}
> +
> +static const struct of_device_id mtk_vpu_match[] = {
> + {
> + .compatible = "mediatek,mt8173-vpu",
> + },
> + {},
> +};
> +MODULE_DEVICE_TABLE(of, mtk_vpu_match);
> +
> +static int mtk_vpu_remove(struct platform_device *pdev)
> +{
> + struct mtk_vpu *vpu = platform_get_drvdata(pdev);
> +
> + vpu_free_p_ext_mem(vpu);
> + vpu_free_d_ext_mem(vpu);
This looks like it leaks cpu_debugfs and the vpu_mutex.
> +
> + return 0;
> +}
> +
> +static struct platform_driver mtk_vpu_driver = {
> + .probe = mtk_vpu_probe,
> + .remove = mtk_vpu_remove,
> + .driver = {
> + .name = MTK_VPU_DRV_NAME,
> + .owner = THIS_MODULE,
> + .of_match_table = mtk_vpu_match,
> + },
> +};
> +
> +module_platform_driver(mtk_vpu_driver);
> +
> +MODULE_LICENSE("GPL v2");
> +MODULE_DESCRIPTION("Mediatek Video Prosessor Unit driver");
> diff --git a/drivers/media/platform/mtk-vpu/mtk_vpu_core.h b/drivers/media/platform/mtk-vpu/mtk_vpu_core.h
> new file mode 100644
> index 0000000..20cf2a0
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vpu/mtk_vpu_core.h
> @@ -0,0 +1,161 @@
> +/*
> +* Copyright (c) 2015 MediaTek Inc.
> +* Author: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
> +*
> +* This program is free software; you can redistribute it and/or modify
> +* it under the terms of the GNU General Public License version 2 as
> +* published by the Free Software Foundation.
> +*
> +* This program is distributed in the hope that it will be useful,
> +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> +* GNU General Public License for more details.
> +*/
> +
> +#ifndef _MTK_VPU_CORE_H
> +#define _MTK_VPU_CORE_H
> +
> +#include <linux/platform_device.h>
> +
> +/**
> + * VPU (video processor unit) is a tiny processor controlling video hardware
> + * related to video codec, scaling and color format converting.
> + * VPU interfaces with other blocks by share memory and interrupt.
> + **/
> +
> +typedef void (*ipi_handler_t) (void *data,
> + unsigned int len,
> + void *priv);
> +
> +/**
> + * enum ipi_id - the id of inter-processor interrupt
> + *
> + * @IPI_VPU_INIT: The interrupt from vpu is to notfiy kernel
> + VPU initialization completed.
> + * @IPI_VENC_H264: The interrupt from vpu is to notify kernel to
> + handle H264 video encoder job, and vice versa.
> + * @IPI_VENC_VP8: The interrupt fro vpu is to notify kernel to
> + handle VP8 video encoder job,, and vice versa.
> + * @IPI_VENC_CAPABILITY: The interrupt from vpu is to
> + get venc hardware capability.
> + * @IPI_MAX: The maximum IPI number
> + */
> +enum ipi_id {
> + IPI_VPU_INIT = 0,
> + IPI_VENC_H264,
> + IPI_VENC_VP8,
> + IPI_VENC_CAPABILITY,
> + IPI_MAX,
> +};
> +
> +/**
> + * vpu_disable_clock - Disable VPU clock
> + *
> + * @pdev: VPU platform device
> + *
> + *
> + * Return: Return 0 if the clock is disabled successfully,
> + * otherwise it is failed.
> + *
> + **/
> +void vpu_disable_clock(struct platform_device *pdev);
> +
> +/**
> + * vpu_enable_clock - Enable VPU clock
> + *
> + * @pdev: VPU platform device
> + *
> + * Return: Return 0 if the clock is enabled successfully,
> + * otherwise it is failed.
> + *
> + **/
> +int vpu_enable_clock(struct platform_device *pdev);
> +
> +/**
> + * vpu_ipi_register - register an ipi function
> + *
> + * @pdev: VPU platform device
> + * @id: IPI ID
> + * @handler: IPI handler
> + * @name: IPI name
> + * @priv: private data for IPI handler
> + *
> + * Register an ipi function to receive ipi interrupt from VPU.
> + *
> + * Return: Return 0 if ipi registers successfully, otherwise it is failed.
> + */
> +int vpu_ipi_register(struct platform_device *pdev, enum ipi_id id,
> + ipi_handler_t handler, const char *name, void *priv);
> +
> +/**
> + * vpu_ipi_send - send data from AP to vpu.
> + *
> + * @pdev: VPU platform device
> + * @id: IPI ID
> + * @buf: the data buffer
> + * @len: the data buffer length
> + * @wait: wait for the last ipi completed.
> + *
> + * This function is thread-safe. When this function returns,
> + * VPU has received the data and starts the processing.
> + * When the processing completes, IPI handler registered
> + * by vpu_ipi_register will be called in interrupt context.
> + *
> + * Return: Return 0 if sending data successfully, otherwise it is failed.
> + **/
> +int vpu_ipi_send(struct platform_device *pdev,
> + enum ipi_id id, void *buf,
> + unsigned int len,
> + unsigned int wait);
> +
> +/**
> + * vpu_get_plat_device - get VPU's platform device
> + *
> + * @pdev: the platform device of the module requesting VPU platform
> + * device for using VPU API.
> + *
> + * Return: Return NULL if it is failed.
> + * otherwise it is VPU's platform device
> + **/
> +struct platform_device *vpu_get_plat_device(struct platform_device *pdev);
> +
> +/**
> + * vpu_load_firmware - download VPU firmware and boot it
> + *
> + * @pdev: VPU platform device
> + *
> + * Return: Return 0 if downloading firmware successfully,
> + * otherwise it is failed
> + **/
> +int vpu_load_firmware(struct platform_device *pdev);
> +
> +/**
> + * vpu_mapping_dm_addr - Mapping DTCM/DMEM to kernel virtual address
> + *
> + * @pdev: VPU platform device
> + * @dmem_addr: VPU's data memory address
> + *
> + * Mapping the VPU's DTCM (Data Tightly-Coupled Memory) /
> + * DMEM (Data Extended Memory) memory address to
> + * kernel virtual address.
> + *
> + * Return: Return ERR_PTR(-EINVAL) if mapping failed,
> + * otherwise the mapped kernel virtual address
> + **/
> +void *vpu_mapping_dm_addr(struct platform_device *pdev,
> + void *dtcm_dmem_addr);
> +
> +/**
> + * vpu_mapping_iommu_dm_addr - Mapping to iommu address
> + *
> + * @pdev: VPU platform device
> + * @dmem_addr: VPU's extended data memory address
> + *
> + * Mapping the VPU's extended data address to iommu address
> + *
> + * Return: Return ERR_PTR(-EINVAL) if mapping failed,
> + * otherwise the mapped iommu address
> + **/
> +dma_addr_t *vpu_mapping_iommu_dm_addr(struct platform_device *pdev,
> + void *dmem_addr);
> +#endif /* _MTK_VPU_CORE_H */
> diff --git a/drivers/media/platform/mtk-vpu/vp8_enc/venc_vp8_vpu.h b/drivers/media/platform/mtk-vpu/vp8_enc/venc_vp8_vpu.h
> new file mode 100644
> index 0000000..4e09eec
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vpu/vp8_enc/venc_vp8_vpu.h
Like the H.264 header. Why is this file included in this patch? It is
not included by anything in the patch and defines symbols that do not
exist yet.
Daniel.
^ permalink raw reply [flat|nested] 84+ messages in thread
* Re: [RESEND RFC/PATCH 3/8] media: platform: mtk-vpu: Support Mediatek VPU
@ 2015-11-27 12:10 ` andrew-ct chen
0 siblings, 0 replies; 84+ messages in thread
From: andrew-ct chen @ 2015-11-27 12:10 UTC (permalink / raw)
To: Daniel Thompson
Cc: Tiffany Lin, Rob Herring, Pawel Moll, Mark Rutland, Ian Campbell,
Kumar Gala, Catalin Marinas, Will Deacon, Mauro Carvalho Chehab,
Matthias Brugger, Daniel Kurtz, Sascha Hauer, Hongzhou Yang,
Hans Verkuil, Laurent Pinchart, Sakari Ailus, Geert Uytterhoeven,
Mikhail Ulyanov, Fabien Dessenne, Arnd Bergmann,
Darren Etheridge, Peter Griffin, Benoit Parrot, Eddie Huang,
Yingjoe Chen, James Liao, Daniel Hsiao, devicetree, linux-kernel,
linux-arm-kernel, linux-media, linux-mediatek
Thanks a lot for your review comments.
On Wed, 2015-11-25 at 16:11 +0000, Daniel Thompson wrote:
> On 17/11/15 12:54, Tiffany Lin wrote:
> > From: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
> >
> > The VPU driver for hw video codec embedded in Mediatek's MT8173 SOCs.
> > It is able to handle video decoding/encoding of in a range of formats.
> > The driver provides with VPU firmware download, memory management and
> > the communication interface between CPU and VPU.
> > For VPU initialization, it will create virtual memory for CPU access and
> > IOMMU address for vcodec hw device access. When a decode/encode instance
> > opens a device node, vpu driver will download vpu firmware to the device.
> > A decode/encode instant will decode/encode a frame using VPU
> > interface to interrupt vpu to handle decoding/encoding jobs.
> >
> > Signed-off-by: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
> > ---
> > drivers/media/platform/Kconfig | 6 +
> > drivers/media/platform/Makefile | 2 +
> > drivers/media/platform/mtk-vpu/Makefile | 1 +
> > .../platform/mtk-vpu/h264_enc/venc_h264_vpu.h | 127 +++
> > .../media/platform/mtk-vpu/include/venc_ipi_msg.h | 212 +++++
> > drivers/media/platform/mtk-vpu/mtk_vpu_core.c | 823 ++++++++++++++++++++
> > drivers/media/platform/mtk-vpu/mtk_vpu_core.h | 161 ++++
> > .../media/platform/mtk-vpu/vp8_enc/venc_vp8_vpu.h | 119 +++
> > 8 files changed, 1451 insertions(+)
> > create mode 100644 drivers/media/platform/mtk-vpu/Makefile
> > create mode 100644 drivers/media/platform/mtk-vpu/h264_enc/venc_h264_vpu.h
> > create mode 100644 drivers/media/platform/mtk-vpu/include/venc_ipi_msg.h
> > create mode 100644 drivers/media/platform/mtk-vpu/mtk_vpu_core.c
> > create mode 100644 drivers/media/platform/mtk-vpu/mtk_vpu_core.h
> > create mode 100644 drivers/media/platform/mtk-vpu/vp8_enc/venc_vp8_vpu.h
> >
> > diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
> > index ccbc974..f98eb47 100644
> > --- a/drivers/media/platform/Kconfig
> > +++ b/drivers/media/platform/Kconfig
> > @@ -148,6 +148,12 @@ config VIDEO_CODA
> > Coda is a range of video codec IPs that supports
> > H.264, MPEG-4, and other video formats.
> >
> > +config MEDIATEK_VPU
> > + bool "Mediatek Video Processor Unit"
> > + ---help---
> > + This driver provides downloading firmware vpu and
> > + communicating with vpu.
> > +
>
> This looks pretty broken.
>
> Shouldn't this option be tristate? Why are there no depends-on or selects?
>
Yes, this should be tristate and depends on VIDEO_DEV && VIDEO_V4L2 &&
ARCH_MEDIATEK
> Also I think the help text could be more descriptive here (and so does
> checkpatch ;-) ).
>
I'll put more descriptive. Thanks.
>
> > diff --git a/drivers/media/platform/mtk-vpu/h264_enc/venc_h264_vpu.h b/drivers/media/platform/mtk-vpu/h264_enc/venc_h264_vpu.h
> > new file mode 100644
> > index 0000000..9c8ebdd
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vpu/h264_enc/venc_h264_vpu.h
>
> Why is this file included in *this* patch? It is not included by
> anything in the patch and defines functions that do not exist yet.
I'll move this to the h264 patch.Thanks
>
>
> > diff --git a/drivers/media/platform/mtk-vpu/include/venc_ipi_msg.h b/drivers/media/platform/mtk-vpu/include/venc_ipi_msg.h
> > new file mode 100644
> > index 0000000..a345b98
>
> This file also is not included by anything and should, perhaps be
> included in a different patch.
>
I'll move this to the v4l2 encoder patch.
>
> > diff --git a/drivers/media/platform/mtk-vpu/mtk_vpu_core.c b/drivers/media/platform/mtk-vpu/mtk_vpu_core.c
> > new file mode 100644
> > index 0000000..b524dbc
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vpu/mtk_vpu_core.c
> > @@ -0,0 +1,823 @@
> > +/*
> > +* Copyright (c) 2015 MediaTek Inc.
> > +* Author: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
> > +*
> > +* This program is free software; you can redistribute it and/or modify
> > +* it under the terms of the GNU General Public License version 2 as
> > +* published by the Free Software Foundation.
> > +*
> > +* This program is distributed in the hope that it will be useful,
> > +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> > +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> > +* GNU General Public License for more details.
> > +*/
> > +#include <linux/clk.h>
> > +#include <linux/debugfs.h>
> > +#include <linux/firmware.h>
> > +#include <linux/interrupt.h>
> > +#include <linux/iommu.h>
> > +#include <linux/module.h>
> > +#include <linux/of_address.h>
> > +#include <linux/of_irq.h>
> > +#include <linux/of_platform.h>
> > +#include <linux/sched.h>
> > +#include <linux/sizes.h>
> > +
> > +#include "mtk_vpu_core.h"
> > +
> > +/**
> > + * VPU (video processor unit) is a tiny processor controlling video hardware
> > + * related to video codec, scaling and color format converting.
> > + * VPU interfaces with other blocks by share memory and interrupt.
> > + **/
> > +#define MTK_VPU_DRV_NAME "mtk_vpu"
>
> This is only used once, why not just put this string directly into the
> .name field?
>
I'll change this.
> > +
> > +#define INIT_TIMEOUT_MS 2000U
> > +#define IPI_TIMEOUT_MS 2000U
> > +#define VPU_FW_VER_LEN 16
> > +
> > +/* vpu extended virtural address */
> > +#define VPU_PMEM0_VIRT(vpu) ((vpu)->mem.p_va)
> > +#define VPU_DMEM0_VIRT(vpu) ((vpu)->mem.d_va)
> > +/* vpu extended iova address*/
> > +#define VPU_PMEM0_IOVA(vpu) ((vpu)->mem.p_iova)
> > +#define VPU_DMEM0_IOVA(vpu) ((vpu)->mem.d_iova)
>
> These feel like obfuscation to me. The code looks clearer without the
> macro For example:
>
> vpu->mem.p_va = dma_alloc_coherent(dev, ...);
>
> Is much clearer than:
>
> VPU_PMEM0_VIRT(vpu) = dma_alloc_coherent(dev, ...);
>
Thanks. I'll change this.
> > +
> > +#define VPU_PTCM(dev) ((dev)->reg.sram)
> > +#define VPU_DTCM(dev) ((dev)->reg.sram + VPU_DTCM_OFFSET)
>
> These macros also seem to add very little value. They are consumed only
> a couple of times and only serve to conceal how the sram is mapped and
> consumed:
>
> dest = vpu->reg.sram;
> if (fw_type == D_FW)
> dest += VPU_DTCM_OFFSET;
>
> Compared to:
>
> dest = fw_type ? VPU_DTCM(vpu) : VPU_PTCM(vpu);
>
Thanks. I'll change this.
> > +
> > +#define VPU_PTCM_SIZE (96 * SZ_1K)
> > +#define VPU_DTCM_SIZE (32 * SZ_1K)
> > +#define VPU_DTCM_OFFSET 0x18000UL
> > +#define VPU_EXT_P_SIZE SZ_1M
> > +#define VPU_EXT_D_SIZE SZ_4M
> > +#define VPU_P_FW_SIZE (VPU_PTCM_SIZE + VPU_EXT_P_SIZE)
> > +#define VPU_D_FW_SIZE (VPU_DTCM_SIZE + VPU_EXT_D_SIZE)
> > +#define SHARE_BUF_SIZE 48
> > +
> > +#define VPU_P_FW "vpu_p.bin"
> > +#define VPU_D_FW "vpu_d.bin"
>
> These macros are of questionable value.
I'll put more comments. Thanks.
>
> > +
> > +#define VPU_BASE 0x0
>
> Is this register really called "base" in the datasheet? From the use in
> the code it looks like it performs a reset and/or PM function.
>
This should be a vpu reset register. We will change this. Thanks.
>
> > +#define VPU_TCM_CFG 0x0008
> > +#define VPU_PMEM_EXT0_ADDR 0x000C
> > +#define VPU_PMEM_EXT1_ADDR 0x0010
> > +#define VPU_TO_HOST 0x001C
> > +#define VPU_DMEM_EXT0_ADDR 0x0014
> > +#define VPU_DMEM_EXT1_ADDR 0x0018
> > +#define HOST_TO_VPU 0x0024
> > +#define VPU_PC_REG 0x0060
> > +#define VPU_WDT_REG 0x0084
> > +
> > +/* vpu inter-processor communication interrupt */
> > +#define VPU_IPC_INT BIT(8)
> > +
> > +/**
> > + * enum vpu_fw_type - VPU firmware type
> > + *
> > + * @P_FW: program firmware
> > + * @D_FW: data firmware
> > + *
> > + */
> > +enum vpu_fw_type {
> > + P_FW,
> > + D_FW,
> > +};
> > +
> > +/**
> > + * struct vpu_mem - VPU memory information
>
> Perhaps "VPU extended program/data memory information" instead.
>
I'll change this.
>
> > + *
> > + * @p_va: the kernel virtual memory address of
> > + * VPU extended program memory
> > + * @d_va: the kernel virtual memory address of VPU extended data memory
> > + * @p_iova: the iova memory address of VPU extended program memory
> > + * @d_iova: the iova memory address of VPU extended data memory
> > + */
> > +struct vpu_mem {
> > + void *p_va;
> > + void *d_va;
> > + dma_addr_t p_iova;
> > + dma_addr_t d_iova;
> > +};
>
> Might be better as:
>
> struct {
> void *va;
> dma_addr_t iova;
> }
>
> This can be then be used as: vpu->mem[P_FW].va . This doesn't matter
> much yet but it would helps us common up some code later.
I'll change this. Thanks
>
>
> > +
> > +/**
> > + * struct vpu_regs - VPU SRAM and configuration registers
> > + *
> > + * @sram: the register for VPU sram
> > + * @cfg: the register for VPU configuration
> > + * @irq: the irq number for VPU interrupt
> > + */
> > +struct vpu_regs {
> > + void __iomem *sram;
>
> This is called TCM everywhere else. Why a different name for it here?
Yes, for the consistency, we should call it TCM.Thanks.
>
>
> > + void __iomem *cfg;
> > + int irq;
> > +};
> > +
> > +/**
> > + * struct vpu_run - VPU initialization status
> > + *
> > + * @signaled: the signal of vpu initialization completed
> > + * @fw_ver: VPU firmware version
> > + * @wq: wait queue for VPU initialization status
> > + */
> > +struct vpu_run {
> > + u32 signaled;
> > + char fw_ver[VPU_FW_VER_LEN];
> > + wait_queue_head_t wq;
> > +};
> > +
> > +/**
> > + * struct vpu_ipi_desc - VPU IPI descriptor
> > + *
> > + * @handler: IPI handler
> > + * @name: the name of IPI handler
> > + * @priv: the private data of IPI handler
> > + */
> > +struct vpu_ipi_desc {
> > + ipi_handler_t handler;
> > + const char *name;
> > + void *priv;
> > +};
> > +
> > +/**
> > + * struct share_obj - The DTCM (Data Tightly-Coupled Memory) buffer shared with
> > + * AP and VPU
>
> Remove "The" (there are more than one of these buffers).
I'll change this.
>
> > + *
> > + * @id: IPI id
> > + * @len: share buffer length
> > + * @share_buf: share buffer data
> > + */
> > +struct share_obj {
> > + int32_t id;
> > + uint32_t len;
> > + unsigned char share_buf[SHARE_BUF_SIZE];
> > +};
> > +
> > +/**
> > + * struct mtk_vpu - vpu driver data
> > + * @mem: VPU extended memory information
> > + * @reg: VPU SRAM and configuration registers
> > + * @run: VPU initialization status
> > + * @ipi_desc: VPU IPI descriptor
> > + * @recv_buf: VPU DTCM share buffer for receiving. The
> > + * receive buffer is only accessed in interrupt context.
> > + * @send_buf: VPU DTCM share buffer for sending
> > + * @dev: VPU struct device
> > + * @clk: VPU clock on/off
> > + * @vpu_mutex: protect mtk_vpu (except recv_buf) and ensure only
> > + * one client to use VPU service at a time. For example,
> > + * suppose a client is using VPU to decode VP8.
> > + * If the other client wants to encode VP8,
> > + * it has to wait until VP8 decode completes.
> > + *
> > + */
> > +struct mtk_vpu {
> > + struct vpu_mem mem;
>
> Rename to extmem?
Yes, I'll rename this.
>
> > + struct vpu_regs reg;
> > + struct vpu_run run;
> > + struct vpu_ipi_desc ipi_desc[IPI_MAX];
> > + struct share_obj *recv_buf;
> > + struct share_obj *send_buf;
> > + struct device *dev;
> > + struct clk *clk;
> > + struct mutex vpu_mutex; /* for protecting vpu data data structure */
> > +};
> > +
> > +/* the thread calls the function should hold the |vpu_mutex| */
>
> Remove this comment. Its unhelpful: the code does not meet this
> requirement because it overstates the scope of vpu_mutex .
I'll change this.
>
> > +static inline void vpu_cfg_writel(struct mtk_vpu *vpu, u32 val, u32 offset)
> > +{
> > + writel(val, vpu->reg.cfg + offset);
> > +}
> > +
> > +static inline u32 vpu_cfg_readl(struct mtk_vpu *vpu, u32 offset)
> > +{
> > + return readl(vpu->reg.cfg + offset);
> > +}
> > +
> > +static inline bool vpu_running(struct mtk_vpu *vpu)
> > +{
> > + return vpu_cfg_readl(vpu, VPU_BASE) & BIT(0);
> > +}
> > +
> > +void vpu_disable_clock(struct platform_device *pdev)
> > +{
> > + struct mtk_vpu *vpu = platform_get_drvdata(pdev);
> > +
> > + /* Disable VPU watchdog */
> > + vpu_cfg_writel(vpu,
> > + vpu_cfg_readl(vpu, VPU_WDT_REG) & ~(1L<<31),
> > + VPU_WDT_REG);
>
> This code combines a reference counted clock disable with a
> not-reference counted register write and will result in the watchdot
> being spuriously disabled.
>
> This will definitely happen if vpu_debug_read() is called at the wrong
> time, possibly may also be an issue for concurrent H.264 and VP8 operations.
>
Yes, you are right. I'll add a reference count to avoid this problem.
Thanks.
> > +
> > + clk_disable_unprepare(vpu->clk);
> > +}
> > +
> > +int vpu_enable_clock(struct platform_device *pdev)
> > +{
> > + struct mtk_vpu *vpu = platform_get_drvdata(pdev);
> > + int ret;
> > +
> > + ret = clk_prepare_enable(vpu->clk);
> > + if (ret)
> > + return ret;
> > + /* Enable VPU watchdog */
> > + vpu_cfg_writel(vpu, vpu_cfg_readl(vpu, VPU_WDT_REG) | (1L << 31),
> > + VPU_WDT_REG);
>
> As above.
>
Ok.
>
> > +
> > + return ret;
> > +}
> > +
> > +int vpu_ipi_register(struct platform_device *pdev,
> > + enum ipi_id id, ipi_handler_t handler,
> > + const char *name, void *priv)
> > +{
> > + struct mtk_vpu *vpu = platform_get_drvdata(pdev);
> > + struct vpu_ipi_desc *ipi_desc;
> > +
> > + if (!vpu) {
> > + dev_err(&pdev->dev, "vpu device in not ready\n");
> > + return -EPROBE_DEFER;
> > + }
> > +
> > + if (id < IPI_MAX && handler != NULL) {
> > + ipi_desc = vpu->ipi_desc;
> > + ipi_desc[id].name = name;
> > + ipi_desc[id].handler = handler;
> > + ipi_desc[id].priv = priv;
> > + return 0;
> > + }
> > +
> > + dev_err(&pdev->dev, "register vpu ipi with invalid arguments\n");
> > + return -EINVAL;
> > +}
>
> This API, and manu of its friends lower down the file, appear to be a
> way to send 32 byte messages to different endpoints on another processor
> on the SoC.
>
> Just interested to know if you evaluated the mailbox driver
> infrastructure for this? Its a good fit for the messaging but doesn't
> offer any support for the direct access to TCM or extended memory.
>
Each codec (vp8/h264) has it own data structure on VPU. CPU can get the
their kernel virtual address with this driver APIs.
When CPU accesses the data structure in VPU, VPU won't write the data
structure at the same time and vice versa.
>
> > +int vpu_ipi_send(struct platform_device *pdev,
> > + enum ipi_id id, void *buf,
> > + unsigned int len, unsigned int wait)
> > +{
> > + struct mtk_vpu *vpu = platform_get_drvdata(pdev);
> > + struct share_obj *send_obj = vpu->send_buf;
> > + unsigned long timeout;
> > +
> > + if (id >= IPI_MAX || len > sizeof(send_obj->share_buf) || buf == NULL) {
> > + dev_err(vpu->dev, "failed to send ipi message\n");
> > + return -EINVAL;
> > + }
> > +
> > + if (!vpu_running(vpu)) {
> > + dev_err(vpu->dev, "vpu_ipi_send: VPU is not running\n");
> > + return -ENXIO;
> > + }
> > +
> > + mutex_lock(&vpu->vpu_mutex);
> > + if (vpu_cfg_readl(vpu, HOST_TO_VPU) && !wait) {
> > + mutex_unlock(&vpu->vpu_mutex);
> > + return -EBUSY;
> > + }
>
> This branch is unreachable (no caller ever sets wait is never set to false).
The parameter "wait" will be removed. Thanks.
>
>
> > +
> > + if (wait)
> > + while (vpu_cfg_readl(vpu, HOST_TO_VPU))
> > + ;
>
> What is this loop for? This code should only be reachable if we timed
> out in the code below so the likely effect of this loop is to
> permantently wedge a thread in a manner that frustrates signal delivery.
This busy waiting will be removed since the mutex_lock and the blow busy
waiting make sure a client sending commanded can be accepted by VPU
hardware.
>
>
> > +
> > + memcpy((void *)send_obj->share_buf, buf, len);
> > + send_obj->len = len;
> > + send_obj->id = id;
> > + vpu_cfg_writel(vpu, 0x1, HOST_TO_VPU);
> > +
> > + /* Wait until VPU receives the command */
> > + timeout = jiffies + msecs_to_jiffies(IPI_TIMEOUT_MS);
> > + do {
> > + if (time_after(jiffies, timeout)) {
> > + dev_err(vpu->dev, "vpu_ipi_send: IPI timeout!\n");
> > + return -EIO;
> > + }
> > + } while (vpu_cfg_readl(vpu, HOST_TO_VPU));
>
> Do we need to busy wait every time we communicate with the co-processor?
> Couldn't we put this wait *before* we write to HOST_TO_VPU above.
>
> That way we only spin when there is a need to.
>
Since the hardware VPU only allows that one client sends the command to
it each time.
We need the wait to make sure VPU accepted the command and cleared the
interrupt and then the next command would be served.
>
> > +
> > + mutex_unlock(&vpu->vpu_mutex);
> > +
> > + return 0;
> > +}
> > +
> > +void *vpu_mapping_dm_addr(struct platform_device *pdev,
> > + void *dtcm_dmem_addr)
>
>
> Use a different type: dtcm_dmem_addr is not a pointer on this CPU.
I'll change this.
>
> > +{
> > + struct mtk_vpu *vpu = platform_get_drvdata(pdev);
> > + unsigned long p_vpu_dtcm = (unsigned long)VPU_DTCM(vpu);
>
> p_vpu_dtcm *is* a pointer on this CPU. Why cast it so that it isn't.
>
> > + unsigned long ul_dtcm_dmem_addr = (unsigned long)(dtcm_dmem_addr);
> > +
> > + if (dtcm_dmem_addr == NULL ||
> > + (ul_dtcm_dmem_addr > (VPU_DTCM_SIZE + VPU_EXT_D_SIZE))) {
> > + dev_err(vpu->dev, "invalid virtual data memory address\n");
> > + return ERR_PTR(-EINVAL);
> > + }
> > +
> > + if (ul_dtcm_dmem_addr < VPU_DTCM_SIZE)
> > + return (void *)(ul_dtcm_dmem_addr + p_vpu_dtcm);
>
> Starting with the pointer will preserve type better:
>
> return vpu->reg.sram + VPU_DTCM_OFFSET + ul_dtcm_dmem_addr;
>
> > +
> > + return (void *)((ul_dtcm_dmem_addr - VPU_DTCM_SIZE) +
> > + VPU_DMEM0_VIRT(vpu));
>
> Likewise this code is clearer with the pointer first.
>
> return vpu->mem.d_va + (ul_dtcm_dmem_addr - VPU_DTCM_SIZE);
>
>
I'll change these above. Thanks.
> > +}
> > +
> > +dma_addr_t *vpu_mapping_iommu_dm_addr(struct platform_device *pdev,
> > + void *dmem_addr)
>
> This function does not return a pointer to a dma_addr_t. It just returns
> a regular dma_addr_t (that's why all the callers of this function cast
> it back ;-) ).
>
I'll change this. Thanks.
>
> dmem_addr is also not a pointer and this function does not return a
> pointer to a dma_addr_t. It just returns a regular dma_addr_t.
>
> > +{
> > + unsigned long ul_dmem_addr = (unsigned long)(dmem_addr);
> > + struct mtk_vpu *vpu = platform_get_drvdata(pdev);
> > +
> > + if (dmem_addr == NULL ||
> > + (ul_dmem_addr < VPU_DTCM_SIZE) ||
> > + (ul_dmem_addr > (VPU_DTCM_SIZE + VPU_EXT_D_SIZE))) {
> > + dev_err(vpu->dev, "invalid IOMMU data memory address\n");
> > + return ERR_PTR(-EINVAL);
> > + }
> > +
> > + return (dma_addr_t *)((ul_dmem_addr - VPU_DTCM_SIZE) +
> > + VPU_DMEM0_IOVA(vpu));
>
> Better written as (this would also have made explicit the type problem
> with the function:
>
> return vpu->mem.d_iova + (ul_dmem_addr - VPU_DTCM_SIZE);
>
I'll change this. Thanks.
> > +}
> > +
> > +struct platform_device *vpu_get_plat_device(struct platform_device *pdev)
> > +{
> > + struct device *dev = &pdev->dev;
> > + struct device_node *vpu_node;
> > + struct platform_device *vpu_pdev;
> > +
> > + vpu_node = of_parse_phandle(dev->of_node, "vpu", 0);
> > + if (!vpu_node) {
> > + dev_err(dev, "can't get vpu node\n");
> > + return NULL;
> > + }
> > +
> > + vpu_pdev = of_find_device_by_node(vpu_node);
> > + if (WARN_ON(!vpu_pdev)) {
> > + dev_err(dev, "vpu pdev failed\n");
> > + of_node_put(vpu_node);
> > + return NULL;
> > + }
> > +
> > + return vpu_pdev;
> > +}
>
> This function looks a bit weird to me. Why do we need to keep consulting
> the devicetree every time we want to start/stop the clock?
>
> I would have expected this code to be part of the init routine of
> anything that needs this and the platform_device would then be cached.
>
Yes, I think every callers should call the function to get vpu platform
device and cache it in their probe function. Thanks.
>
> > +
> > +/* load vpu program/data memory */
> > +static void load_requested_vpu(struct mtk_vpu *vpu,
> > + size_t fw_size,
> > + const u8 *fw_data,
> > + u8 fw_type)
> > +{
> > + size_t target_size = fw_type ? VPU_DTCM_SIZE : VPU_PTCM_SIZE;
> > + size_t extra_fw_size = 0;
> > + void *dest;
> > +
> > + /* reset VPU */
> > + vpu_cfg_writel(vpu, 0x0, VPU_BASE);
> > +
> > + /* handle extended firmware size */
> > + if (fw_size > target_size) {
> > + dev_dbg(vpu->dev, "fw size %lx > limited fw size %lx\n",
> > + fw_size, target_size);
> > + extra_fw_size = fw_size - target_size;
> > + dev_dbg(vpu->dev, "extra_fw_size %lx\n", extra_fw_size);
> > + fw_size = target_size;
> > + }
> > + dest = fw_type ? VPU_DTCM(vpu) : VPU_PTCM(vpu);
> > + memcpy(dest, fw_data, fw_size);
> > + /* download to extended memory if need */
> > + if (extra_fw_size > 0) {
> > + dest = fw_type ?
> > + VPU_DMEM0_VIRT(vpu) : VPU_PMEM0_VIRT(vpu);
> > +
> > + dev_dbg(vpu->dev, "download extended memory type %x\n",
> > + fw_type);
> > + memcpy(dest, fw_data + target_size, extra_fw_size);
> > + }
> > +}
> > +
> > +int vpu_load_firmware(struct platform_device *pdev)
> > +{
> > + struct mtk_vpu *vpu = platform_get_drvdata(pdev);
> > + struct device *dev = &pdev->dev;
> > + struct vpu_run *run = &vpu->run;
> > + const struct firmware *vpu_fw;
> > + int ret;
> > +
> > + if (!pdev) {
> > + dev_err(dev, "VPU platform device is invalid\n");
> > + return -EINVAL;
> > + }
> > +
> > + mutex_lock(&vpu->vpu_mutex);
> > +
> > + ret = vpu_enable_clock(pdev);
> > + if (ret) {
> > + dev_err(dev, "enable clock failed %d\n", ret);
> > + goto OUT_LOAD_FW;
> > + }
> > +
> > + if (vpu_running(vpu)) {
> > + vpu_disable_clock(pdev);
> > + mutex_unlock(&vpu->vpu_mutex);
> > + dev_warn(dev, "vpu is running already\n");
> > + return 0;
> > + }
> > +
> > + run->signaled = false;
> > + dev_dbg(vpu->dev, "firmware request\n");
> > + ret = request_firmware(&vpu_fw, VPU_P_FW, dev);
> > + if (ret < 0) {
> > + dev_err(dev, "Failed to load %s, %d\n", VPU_P_FW, ret);
> > + goto OUT_LOAD_FW;
> > + }
> > + if (vpu_fw->size > VPU_P_FW_SIZE) {
> > + ret = -EFBIG;
> > + dev_err(dev, "program fw size %zu is abnormal\n", vpu_fw->size);
> > + goto OUT_LOAD_FW;
> > + }
>
> Possibly move request_firmware(), release_firmware() and the associated
> error handling into load_requested_vpu(). It can all be parameterized
> and the filename of the firmware means error reports will still be clear
> about whether the p or d firmware is faulty.
>
I'll change this.
>
> > + dev_dbg(vpu->dev, "Downloaded program fw size: %zu.\n",
> > + vpu_fw->size);
> > + /* Downloading program firmware to device*/
> > + load_requested_vpu(vpu, vpu_fw->size, vpu_fw->data,
> > + P_FW);
> > + release_firmware(vpu_fw);
> > +
> > + ret = request_firmware(&vpu_fw, VPU_D_FW, dev);
> > + if (ret < 0) {
> > + dev_err(dev, "Failed to load %s, %d\n", VPU_D_FW, ret);
> > + goto OUT_LOAD_FW;
> > + }
> > + if (vpu_fw->size > VPU_D_FW_SIZE) {
> > + ret = -EFBIG;
> > + dev_err(dev, "data fw size %zu is abnormal\n", vpu_fw->size);
> > + goto OUT_LOAD_FW;
> > + }
> > + dev_dbg(vpu->dev, "Downloaded data fw size: %zu.\n",
> > + vpu_fw->size);
> > + /* Downloading data firmware to device */
> > + load_requested_vpu(vpu, vpu_fw->size, vpu_fw->data,
> > + D_FW);
> > + release_firmware(vpu_fw);
> > + /* boot up vpu */
> > + vpu_cfg_writel(vpu, 0x1, VPU_BASE);
> > +
> > + ret = wait_event_interruptible_timeout(run->wq,
> > + run->signaled,
> > + msecs_to_jiffies(INIT_TIMEOUT_MS)
> > + );
> > + if (0 == ret) {
> > + ret = -ETIME;
> > + dev_err(dev, "wait vpu initialization timout!\n");
> > + goto OUT_LOAD_FW;
> > + } else if (-ERESTARTSYS == ret) {
> > + dev_err(dev, "wait vpu interrupted by a signal!\n");
> > + goto OUT_LOAD_FW;
> > + }
> > +
> > + ret = 0;
> > + dev_info(dev, "vpu is ready. Fw version %s\n", run->fw_ver);
> > +
> > +OUT_LOAD_FW:
> > + vpu_disable_clock(pdev);
> > + mutex_unlock(&vpu->vpu_mutex);
> > +
> > + return ret;
> > +}
> > +
> > +static void vpu_init_ipi_handler(void *data, unsigned int len, void *priv)
> > +{
> > + struct mtk_vpu *vpu = (struct mtk_vpu *)priv;
> > + struct vpu_run *run = (struct vpu_run *)data;
> > +
> > + vpu->run.signaled = run->signaled;
> > + strncpy(vpu->run.fw_ver, run->fw_ver, VPU_FW_VER_LEN);
> > + wake_up_interruptible(&vpu->run.wq);
> > +}
> > +
> > +static int vpu_debug_open(struct inode *inode, struct file *file)
> > +{
> > + file->private_data = inode->i_private;
> > + return 0;
> > +}
> > +
> > +static ssize_t vpu_debug_read(struct file *file, char __user *user_buf,
> > + size_t count, loff_t *ppos)
> > +{
> > + char buf[256];
> > + unsigned int len;
> > + unsigned int running, pc, vpu_to_host, host_to_vpu, wdt;
> > + int ret;
> > + struct device *dev = file->private_data;
> > + struct platform_device *pdev = to_platform_device(dev);
> > + struct mtk_vpu *vpu = dev_get_drvdata(dev);
> > +
> > + ret = vpu_enable_clock(pdev);
> > + if (ret) {
> > + dev_err(vpu->dev, "[VPU] enable clock failed %d\n", ret);
> > + return 0;
> > + }
> > +
> > + /* vpu register status */
> > + running = vpu_running(vpu);
> > + pc = vpu_cfg_readl(vpu, VPU_PC_REG);
> > + wdt = vpu_cfg_readl(vpu, VPU_WDT_REG);
> > + host_to_vpu = vpu_cfg_readl(vpu, HOST_TO_VPU);
> > + vpu_to_host = vpu_cfg_readl(vpu, VPU_TO_HOST);
> > + vpu_disable_clock(pdev);
> > +
> > + if (running) {
> > + len = sprintf(buf, "VPU is running\n\n"
> > + "FW Version: %s\n"
> > + "PC: 0x%x\n"
> > + "WDT: 0x%x\n"
> > + "Host to VPU: 0x%x\n"
> > + "VPU to Host: 0x%x\n",
> > + vpu->run.fw_ver, pc, wdt,
> > + host_to_vpu, vpu_to_host);
> > + } else {
> > + len = sprintf(buf, "VPU not running\n");
> > + }
> > +
> > + return simple_read_from_buffer(user_buf, count, ppos, buf, len);
> > +}
> > +
> > +static const struct file_operations vpu_debug_fops = {
> > + .open = vpu_debug_open,
> > + .read = vpu_debug_read,
> > +};
>
> These operations should be conditional on CONFIG_DEBUG_FS.
I'll change this.
>
> > +
> > +static void vpu_free_p_ext_mem(struct mtk_vpu *vpu)
> > +{
> > + struct device *dev = vpu->dev;
> > + struct iommu_domain *domain = iommu_get_domain_for_dev(dev);
> > +
> > + dma_free_coherent(dev, VPU_EXT_P_SIZE, VPU_PMEM0_VIRT(vpu),
> > + VPU_PMEM0_IOVA(vpu));
> > +
> > + if (domain)
> > + iommu_detach_device(domain, vpu->dev);
> > +}
> > +
> > +static void vpu_free_d_ext_mem(struct mtk_vpu *vpu)
> > +{
> > + struct device *dev = vpu->dev;
> > + struct iommu_domain *domain = iommu_get_domain_for_dev(dev);
> > +
> > + dma_free_coherent(dev, VPU_EXT_D_SIZE, VPU_DMEM0_VIRT(vpu),
> > + VPU_DMEM0_IOVA(vpu));
> > +
> > + if (domain)
> > + iommu_detach_device(domain, dev);
> > +}
>
> Look like this could be parameterized and combined with vpu_free_p_ext_mem.
I'll change this.
>
>
> > +
> > +static int vpu_alloc_p_ext_mem(struct mtk_vpu *vpu)
> > +{
> > + struct device *dev = vpu->dev;
> > + struct iommu_domain *domain = iommu_get_domain_for_dev(dev);
> > + phys_addr_t p_pa;
> > +
> > + VPU_PMEM0_VIRT(vpu) = dma_alloc_coherent(dev,
> > + VPU_EXT_P_SIZE,
> > + &(VPU_PMEM0_IOVA(vpu)),
> > + GFP_KERNEL);
> > + if (VPU_PMEM0_VIRT(vpu) == NULL) {
> > + dev_err(dev, "Failed to allocate the extended program memory\n");
> > + return PTR_ERR(VPU_PMEM0_VIRT(vpu));
> > + }
> > +
> > + p_pa = iommu_iova_to_phys(domain, vpu->mem.p_iova);
> > + /* Disable extend0. Enable extend1 */
> > + vpu_cfg_writel(vpu, 0x1, VPU_PMEM_EXT0_ADDR);
> > + vpu_cfg_writel(vpu, (p_pa & 0xFFFFF000), VPU_PMEM_EXT1_ADDR);
> > +
> > + dev_info(dev, "Program extend memory phy=0x%llx virt=0x%p iova=0x%llx\n",
> > + (unsigned long long)p_pa,
> > + VPU_PMEM0_VIRT(vpu),
> > + (unsigned long long)VPU_PMEM0_IOVA(vpu));
> > +
> > + return 0;
> > +}
> > +
> > +static int vpu_alloc_d_ext_mem(struct mtk_vpu *vpu)
> > +{
> > + struct device *dev = vpu->dev;
> > + struct iommu_domain *domain = iommu_get_domain_for_dev(dev);
> > + phys_addr_t d_pa;
> > +
> > + VPU_DMEM0_VIRT(vpu) = dma_alloc_coherent(dev,
> > + VPU_EXT_D_SIZE,
> > + &(VPU_DMEM0_IOVA(vpu)),
> > + GFP_KERNEL);
> > + if (VPU_DMEM0_VIRT(vpu) == NULL) {
> > + dev_err(dev, "Failed to allocate the extended data memory\n");
> > + return PTR_ERR(VPU_DMEM0_VIRT(vpu));
> > + }
> > +
> > + d_pa = iommu_iova_to_phys(domain, vpu->mem.d_iova);
> > +
> > + /* Disable extend0. Enable extend1 */
> > + vpu_cfg_writel(vpu, 0x1, VPU_DMEM_EXT0_ADDR);
> > + vpu_cfg_writel(vpu, (d_pa & 0xFFFFF000),
> > + VPU_DMEM_EXT1_ADDR);
> > +
> > + dev_info(dev, "Data extend memory phy=0x%llx virt=0x%p iova=0x%llx\n",
> > + (unsigned long long)d_pa,
> > + VPU_DMEM0_VIRT(vpu),
> > + (unsigned long long)VPU_DMEM0_IOVA(vpu));
> > +
> > + return 0;
> > +}
>
> Also looks suitable for parameterizing and combining with
> vpu_alloc_p_ext_mem .
>
I'll change this.
>
> > +
> > +static void vpu_ipi_handler(struct mtk_vpu *vpu)
> > +{
> > + struct share_obj *rcv_obj = vpu->recv_buf;
> > + struct vpu_ipi_desc *ipi_desc = vpu->ipi_desc;
> > +
> > + if (rcv_obj->id < IPI_MAX && ipi_desc[rcv_obj->id].handler) {
> > + ipi_desc[rcv_obj->id].handler(rcv_obj->share_buf,
> > + rcv_obj->len,
> > + ipi_desc[rcv_obj->id].priv);
> > + } else {
> > + dev_err(vpu->dev, "No such ipi id = %d\n", rcv_obj->id);
> > + }
> > +}
> > +
> > +static int vpu_ipi_init(struct mtk_vpu *vpu)
> > +{
> > + /* Disable VPU to host interrupt */
> > + vpu_cfg_writel(vpu, 0x0, VPU_TO_HOST);
> > +
> > + /* shared buffer initialization */
> > + vpu->recv_buf = (struct share_obj *)VPU_DTCM(vpu);
> > + vpu->send_buf = vpu->recv_buf + 1;
> > + memset(vpu->recv_buf, 0, sizeof(struct share_obj));
> > + memset(vpu->send_buf, 0, sizeof(struct share_obj));
> > + mutex_init(&vpu->vpu_mutex);
> > +
> > + return 0;
> > +}
> > +
> > +static irqreturn_t vpu_irq_handler(int irq, void *priv)
> > +{
> > + struct mtk_vpu *vpu = priv;
> > + uint32_t vpu_to_host = vpu_cfg_readl(vpu, VPU_TO_HOST);
> > +
> > + if (vpu_to_host & VPU_IPC_INT)
> > + vpu_ipi_handler(vpu);
> > + else
> > + dev_err(vpu->dev, "vpu watchdog timeout!\n");
> > +
> > + /* VPU won't send another interrupt until we set VPU_TO_HOST to 0. */
> > + vpu_cfg_writel(vpu, 0x0, VPU_TO_HOST);
>
> If we were triggered by a watchdog then how long will it be before the
> next watchdog interrupt? Will we end up spamming the logs?
Thanks for reminding me.
When watchdog was timeout, I think VPU driver should notify registered
drivers and reset vpu hardware for next run. I'll change this.
> > +
> > + return IRQ_HANDLED;
> > +}
> > +
> > +static struct dentry *vpu_debugfs;
> > +static int mtk_vpu_probe(struct platform_device *pdev)
> > +{
> > + struct mtk_vpu *vpu;
> > + struct device *dev;
> > + struct resource *res;
> > + int ret = 0;
> > +
> > + dev_dbg(&pdev->dev, "initialization\n");
> > +
> > + dev = &pdev->dev;
> > + vpu = devm_kzalloc(dev, sizeof(*vpu), GFP_KERNEL);
> > + if (!vpu)
> > + return -ENOMEM;
> > +
> > + vpu->dev = &pdev->dev;
> > + res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "sram");
> > + vpu->reg.sram = devm_ioremap_resource(dev, res);
> > + if (IS_ERR(vpu->reg.sram)) {
> > + dev_err(dev, "devm_ioremap_resource vpu sram failed.\n");
> > + return PTR_ERR(vpu->reg.sram);
> > + }
> > +
> > + res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "cfg_reg");
> > + vpu->reg.cfg = devm_ioremap_resource(dev, res);
> > + if (IS_ERR(vpu->reg.cfg)) {
> > + dev_err(dev, "devm_ioremap_resource vpu cfg failed.\n");
> > + return PTR_ERR(vpu->reg.cfg);
> > + }
> > +
> > + /* Get VPU clock */
> > + vpu->clk = devm_clk_get(dev, "main");
> > + if (vpu->clk == NULL) {
> > + dev_err(dev, "get vpu clock fail\n");
> > + return -EINVAL;
> > + }
> > +
> > + platform_set_drvdata(pdev, vpu);
> > +
> > + ret = vpu_enable_clock(pdev);
> > + if (ret) {
> > + ret = -EINVAL;
> > + return ret;
>
> Why not just "return ret" (or "return -EINVAL")?
I'll change this.
>
> > + }
> > +
> > + dev_dbg(dev, "vpu ipi init\n");
> > + ret = vpu_ipi_init(vpu);
> > + if (ret) {
> > + dev_err(dev, "Failed to init ipi\n");
> > + goto disable_vpu_clk;
> > + }
> > +
> > + platform_set_drvdata(pdev, vpu);
> > +
> > + /* register vpu initialization IPI */
> > + ret = vpu_ipi_register(pdev, IPI_VPU_INIT, vpu_init_ipi_handler,
> > + "vpu_init", vpu);
> > + if (ret) {
> > + dev_err(dev, "Failed to register IPI_VPU_INIT\n");
> > + goto vpu_mutex_destroy;
> > + }
> > +
> > + vpu_debugfs = debugfs_create_file("mtk_vpu", S_IRUGO, NULL, (void *)dev,
> > + &vpu_debug_fops);
> > + if (!vpu_debugfs) {
> > + ret = -ENOMEM;
> > + goto cleanup_ipi;
> > + }
> > +
> > + /* Set PTCM to 96K and DTCM to 32K */
> > + vpu_cfg_writel(vpu, 0x2, VPU_TCM_CFG);
> > +
> > + ret = vpu_alloc_p_ext_mem(vpu);
> > + if (ret) {
> > + dev_err(dev, "Allocate PM failed\n");
> > + goto remove_debugfs;
> > + }
> > +
> > + ret = vpu_alloc_d_ext_mem(vpu);
> > + if (ret) {
> > + dev_err(dev, "Allocate DM failed\n");
> > + goto free_p_mem;
> > + }
> > +
> > + init_waitqueue_head(&vpu->run.wq);
> > +
> > + res = platform_get_resource(pdev, IORESOURCE_IRQ, 0);
> > + if (res == NULL) {
> > + dev_err(dev, "get IRQ resource failed.\n");
> > + ret = -ENXIO;
> > + goto free_d_mem;
> > + }
> > + vpu->reg.irq = platform_get_irq(pdev, 0);
> > + ret = devm_request_irq(dev, vpu->reg.irq, vpu_irq_handler, 0,
> > + pdev->name, vpu);
> > + if (ret) {
> > + dev_err(dev, "failed to request irq\n");
> > + goto free_d_mem;
> > + }
> > +
> > + vpu_disable_clock(pdev);
> > + dev_dbg(dev, "initialization completed\n");
> > +
> > + return 0;
> > +
> > +free_d_mem:
> > + vpu_free_d_ext_mem(vpu);
> > +free_p_mem:
> > + vpu_free_p_ext_mem(vpu);
> > +remove_debugfs:
> > + debugfs_remove(vpu_debugfs);
> > +cleanup_ipi:
> > + memset(vpu->ipi_desc, 0, sizeof(struct vpu_ipi_desc)*IPI_MAX);
> > +vpu_mutex_destroy:
> > + mutex_destroy(&vpu->vpu_mutex);
> > +disable_vpu_clk:
> > + vpu_disable_clock(pdev);
> > +
> > + return ret;
> > +}
> > +
> > +static const struct of_device_id mtk_vpu_match[] = {
> > + {
> > + .compatible = "mediatek,mt8173-vpu",
> > + },
> > + {},
> > +};
> > +MODULE_DEVICE_TABLE(of, mtk_vpu_match);
> > +
> > +static int mtk_vpu_remove(struct platform_device *pdev)
> > +{
> > + struct mtk_vpu *vpu = platform_get_drvdata(pdev);
> > +
> > + vpu_free_p_ext_mem(vpu);
> > + vpu_free_d_ext_mem(vpu);
>
> This looks like it leaks cpu_debugfs and the vpu_mutex.
Yes, I'll change this. Thanks.
>
> > +
> > + return 0;
> > +}
> > +
> > +static struct platform_driver mtk_vpu_driver = {
> > + .probe = mtk_vpu_probe,
> > + .remove = mtk_vpu_remove,
> > + .driver = {
> > + .name = MTK_VPU_DRV_NAME,
> > + .owner = THIS_MODULE,
> > + .of_match_table = mtk_vpu_match,
> > + },
> > +};
> > +
> > +module_platform_driver(mtk_vpu_driver);
> > +
> > +MODULE_LICENSE("GPL v2");
> > +MODULE_DESCRIPTION("Mediatek Video Prosessor Unit driver");
> > diff --git a/drivers/media/platform/mtk-vpu/mtk_vpu_core.h b/drivers/media/platform/mtk-vpu/mtk_vpu_core.h
> > new file mode 100644
> > index 0000000..20cf2a0
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vpu/mtk_vpu_core.h
> > @@ -0,0 +1,161 @@
> > +/*
> > +* Copyright (c) 2015 MediaTek Inc.
> > +* Author: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
> > +*
> > +* This program is free software; you can redistribute it and/or modify
> > +* it under the terms of the GNU General Public License version 2 as
> > +* published by the Free Software Foundation.
> > +*
> > +* This program is distributed in the hope that it will be useful,
> > +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> > +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> > +* GNU General Public License for more details.
> > +*/
> > +
> > +#ifndef _MTK_VPU_CORE_H
> > +#define _MTK_VPU_CORE_H
> > +
> > +#include <linux/platform_device.h>
> > +
> > +/**
> > + * VPU (video processor unit) is a tiny processor controlling video hardware
> > + * related to video codec, scaling and color format converting.
> > + * VPU interfaces with other blocks by share memory and interrupt.
> > + **/
> > +
> > +typedef void (*ipi_handler_t) (void *data,
> > + unsigned int len,
> > + void *priv);
> > +
> > +/**
> > + * enum ipi_id - the id of inter-processor interrupt
> > + *
> > + * @IPI_VPU_INIT: The interrupt from vpu is to notfiy kernel
> > + VPU initialization completed.
> > + * @IPI_VENC_H264: The interrupt from vpu is to notify kernel to
> > + handle H264 video encoder job, and vice versa.
> > + * @IPI_VENC_VP8: The interrupt fro vpu is to notify kernel to
> > + handle VP8 video encoder job,, and vice versa.
> > + * @IPI_VENC_CAPABILITY: The interrupt from vpu is to
> > + get venc hardware capability.
> > + * @IPI_MAX: The maximum IPI number
> > + */
> > +enum ipi_id {
> > + IPI_VPU_INIT = 0,
> > + IPI_VENC_H264,
> > + IPI_VENC_VP8,
> > + IPI_VENC_CAPABILITY,
> > + IPI_MAX,
> > +};
> > +
> > +/**
> > + * vpu_disable_clock - Disable VPU clock
> > + *
> > + * @pdev: VPU platform device
> > + *
> > + *
> > + * Return: Return 0 if the clock is disabled successfully,
> > + * otherwise it is failed.
> > + *
> > + **/
> > +void vpu_disable_clock(struct platform_device *pdev);
> > +
> > +/**
> > + * vpu_enable_clock - Enable VPU clock
> > + *
> > + * @pdev: VPU platform device
> > + *
> > + * Return: Return 0 if the clock is enabled successfully,
> > + * otherwise it is failed.
> > + *
> > + **/
> > +int vpu_enable_clock(struct platform_device *pdev);
> > +
> > +/**
> > + * vpu_ipi_register - register an ipi function
> > + *
> > + * @pdev: VPU platform device
> > + * @id: IPI ID
> > + * @handler: IPI handler
> > + * @name: IPI name
> > + * @priv: private data for IPI handler
> > + *
> > + * Register an ipi function to receive ipi interrupt from VPU.
> > + *
> > + * Return: Return 0 if ipi registers successfully, otherwise it is failed.
> > + */
> > +int vpu_ipi_register(struct platform_device *pdev, enum ipi_id id,
> > + ipi_handler_t handler, const char *name, void *priv);
> > +
> > +/**
> > + * vpu_ipi_send - send data from AP to vpu.
> > + *
> > + * @pdev: VPU platform device
> > + * @id: IPI ID
> > + * @buf: the data buffer
> > + * @len: the data buffer length
> > + * @wait: wait for the last ipi completed.
> > + *
> > + * This function is thread-safe. When this function returns,
> > + * VPU has received the data and starts the processing.
> > + * When the processing completes, IPI handler registered
> > + * by vpu_ipi_register will be called in interrupt context.
> > + *
> > + * Return: Return 0 if sending data successfully, otherwise it is failed.
> > + **/
> > +int vpu_ipi_send(struct platform_device *pdev,
> > + enum ipi_id id, void *buf,
> > + unsigned int len,
> > + unsigned int wait);
> > +
> > +/**
> > + * vpu_get_plat_device - get VPU's platform device
> > + *
> > + * @pdev: the platform device of the module requesting VPU platform
> > + * device for using VPU API.
> > + *
> > + * Return: Return NULL if it is failed.
> > + * otherwise it is VPU's platform device
> > + **/
> > +struct platform_device *vpu_get_plat_device(struct platform_device *pdev);
> > +
> > +/**
> > + * vpu_load_firmware - download VPU firmware and boot it
> > + *
> > + * @pdev: VPU platform device
> > + *
> > + * Return: Return 0 if downloading firmware successfully,
> > + * otherwise it is failed
> > + **/
> > +int vpu_load_firmware(struct platform_device *pdev);
> > +
> > +/**
> > + * vpu_mapping_dm_addr - Mapping DTCM/DMEM to kernel virtual address
> > + *
> > + * @pdev: VPU platform device
> > + * @dmem_addr: VPU's data memory address
> > + *
> > + * Mapping the VPU's DTCM (Data Tightly-Coupled Memory) /
> > + * DMEM (Data Extended Memory) memory address to
> > + * kernel virtual address.
> > + *
> > + * Return: Return ERR_PTR(-EINVAL) if mapping failed,
> > + * otherwise the mapped kernel virtual address
> > + **/
> > +void *vpu_mapping_dm_addr(struct platform_device *pdev,
> > + void *dtcm_dmem_addr);
> > +
> > +/**
> > + * vpu_mapping_iommu_dm_addr - Mapping to iommu address
> > + *
> > + * @pdev: VPU platform device
> > + * @dmem_addr: VPU's extended data memory address
> > + *
> > + * Mapping the VPU's extended data address to iommu address
> > + *
> > + * Return: Return ERR_PTR(-EINVAL) if mapping failed,
> > + * otherwise the mapped iommu address
> > + **/
> > +dma_addr_t *vpu_mapping_iommu_dm_addr(struct platform_device *pdev,
> > + void *dmem_addr);
> > +#endif /* _MTK_VPU_CORE_H */
> > diff --git a/drivers/media/platform/mtk-vpu/vp8_enc/venc_vp8_vpu.h b/drivers/media/platform/mtk-vpu/vp8_enc/venc_vp8_vpu.h
> > new file mode 100644
> > index 0000000..4e09eec
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vpu/vp8_enc/venc_vp8_vpu.h
>
> Like the H.264 header. Why is this file included in this patch? It is
> not included by anything in the patch and defines symbols that do not
> exist yet.
This should be moved to vp8 patch. Thanks.
>
>
> Daniel.
^ permalink raw reply [flat|nested] 84+ messages in thread
* Re: [RESEND RFC/PATCH 3/8] media: platform: mtk-vpu: Support Mediatek VPU
@ 2015-11-27 12:10 ` andrew-ct chen
0 siblings, 0 replies; 84+ messages in thread
From: andrew-ct chen @ 2015-11-27 12:10 UTC (permalink / raw)
To: Daniel Thompson
Cc: Tiffany Lin, Rob Herring, Pawel Moll, Mark Rutland, Ian Campbell,
Kumar Gala, Catalin Marinas, Will Deacon, Mauro Carvalho Chehab,
Matthias Brugger, Daniel Kurtz, Sascha Hauer, Hongzhou Yang,
Hans Verkuil, Laurent Pinchart, Sakari Ailus, Geert Uytterhoeven,
Mikhail Ulyanov, Fabien Dessenne, Arnd Bergmann,
Darren Etheridge, Peter Griffin
Thanks a lot for your review comments.
On Wed, 2015-11-25 at 16:11 +0000, Daniel Thompson wrote:
> On 17/11/15 12:54, Tiffany Lin wrote:
> > From: Andrew-CT Chen <andrew-ct.chen-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
> >
> > The VPU driver for hw video codec embedded in Mediatek's MT8173 SOCs.
> > It is able to handle video decoding/encoding of in a range of formats.
> > The driver provides with VPU firmware download, memory management and
> > the communication interface between CPU and VPU.
> > For VPU initialization, it will create virtual memory for CPU access and
> > IOMMU address for vcodec hw device access. When a decode/encode instance
> > opens a device node, vpu driver will download vpu firmware to the device.
> > A decode/encode instant will decode/encode a frame using VPU
> > interface to interrupt vpu to handle decoding/encoding jobs.
> >
> > Signed-off-by: Andrew-CT Chen <andrew-ct.chen-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
> > ---
> > drivers/media/platform/Kconfig | 6 +
> > drivers/media/platform/Makefile | 2 +
> > drivers/media/platform/mtk-vpu/Makefile | 1 +
> > .../platform/mtk-vpu/h264_enc/venc_h264_vpu.h | 127 +++
> > .../media/platform/mtk-vpu/include/venc_ipi_msg.h | 212 +++++
> > drivers/media/platform/mtk-vpu/mtk_vpu_core.c | 823 ++++++++++++++++++++
> > drivers/media/platform/mtk-vpu/mtk_vpu_core.h | 161 ++++
> > .../media/platform/mtk-vpu/vp8_enc/venc_vp8_vpu.h | 119 +++
> > 8 files changed, 1451 insertions(+)
> > create mode 100644 drivers/media/platform/mtk-vpu/Makefile
> > create mode 100644 drivers/media/platform/mtk-vpu/h264_enc/venc_h264_vpu.h
> > create mode 100644 drivers/media/platform/mtk-vpu/include/venc_ipi_msg.h
> > create mode 100644 drivers/media/platform/mtk-vpu/mtk_vpu_core.c
> > create mode 100644 drivers/media/platform/mtk-vpu/mtk_vpu_core.h
> > create mode 100644 drivers/media/platform/mtk-vpu/vp8_enc/venc_vp8_vpu.h
> >
> > diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
> > index ccbc974..f98eb47 100644
> > --- a/drivers/media/platform/Kconfig
> > +++ b/drivers/media/platform/Kconfig
> > @@ -148,6 +148,12 @@ config VIDEO_CODA
> > Coda is a range of video codec IPs that supports
> > H.264, MPEG-4, and other video formats.
> >
> > +config MEDIATEK_VPU
> > + bool "Mediatek Video Processor Unit"
> > + ---help---
> > + This driver provides downloading firmware vpu and
> > + communicating with vpu.
> > +
>
> This looks pretty broken.
>
> Shouldn't this option be tristate? Why are there no depends-on or selects?
>
Yes, this should be tristate and depends on VIDEO_DEV && VIDEO_V4L2 &&
ARCH_MEDIATEK
> Also I think the help text could be more descriptive here (and so does
> checkpatch ;-) ).
>
I'll put more descriptive. Thanks.
>
> > diff --git a/drivers/media/platform/mtk-vpu/h264_enc/venc_h264_vpu.h b/drivers/media/platform/mtk-vpu/h264_enc/venc_h264_vpu.h
> > new file mode 100644
> > index 0000000..9c8ebdd
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vpu/h264_enc/venc_h264_vpu.h
>
> Why is this file included in *this* patch? It is not included by
> anything in the patch and defines functions that do not exist yet.
I'll move this to the h264 patch.Thanks
>
>
> > diff --git a/drivers/media/platform/mtk-vpu/include/venc_ipi_msg.h b/drivers/media/platform/mtk-vpu/include/venc_ipi_msg.h
> > new file mode 100644
> > index 0000000..a345b98
>
> This file also is not included by anything and should, perhaps be
> included in a different patch.
>
I'll move this to the v4l2 encoder patch.
>
> > diff --git a/drivers/media/platform/mtk-vpu/mtk_vpu_core.c b/drivers/media/platform/mtk-vpu/mtk_vpu_core.c
> > new file mode 100644
> > index 0000000..b524dbc
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vpu/mtk_vpu_core.c
> > @@ -0,0 +1,823 @@
> > +/*
> > +* Copyright (c) 2015 MediaTek Inc.
> > +* Author: Andrew-CT Chen <andrew-ct.chen-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
> > +*
> > +* This program is free software; you can redistribute it and/or modify
> > +* it under the terms of the GNU General Public License version 2 as
> > +* published by the Free Software Foundation.
> > +*
> > +* This program is distributed in the hope that it will be useful,
> > +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> > +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> > +* GNU General Public License for more details.
> > +*/
> > +#include <linux/clk.h>
> > +#include <linux/debugfs.h>
> > +#include <linux/firmware.h>
> > +#include <linux/interrupt.h>
> > +#include <linux/iommu.h>
> > +#include <linux/module.h>
> > +#include <linux/of_address.h>
> > +#include <linux/of_irq.h>
> > +#include <linux/of_platform.h>
> > +#include <linux/sched.h>
> > +#include <linux/sizes.h>
> > +
> > +#include "mtk_vpu_core.h"
> > +
> > +/**
> > + * VPU (video processor unit) is a tiny processor controlling video hardware
> > + * related to video codec, scaling and color format converting.
> > + * VPU interfaces with other blocks by share memory and interrupt.
> > + **/
> > +#define MTK_VPU_DRV_NAME "mtk_vpu"
>
> This is only used once, why not just put this string directly into the
> .name field?
>
I'll change this.
> > +
> > +#define INIT_TIMEOUT_MS 2000U
> > +#define IPI_TIMEOUT_MS 2000U
> > +#define VPU_FW_VER_LEN 16
> > +
> > +/* vpu extended virtural address */
> > +#define VPU_PMEM0_VIRT(vpu) ((vpu)->mem.p_va)
> > +#define VPU_DMEM0_VIRT(vpu) ((vpu)->mem.d_va)
> > +/* vpu extended iova address*/
> > +#define VPU_PMEM0_IOVA(vpu) ((vpu)->mem.p_iova)
> > +#define VPU_DMEM0_IOVA(vpu) ((vpu)->mem.d_iova)
>
> These feel like obfuscation to me. The code looks clearer without the
> macro For example:
>
> vpu->mem.p_va = dma_alloc_coherent(dev, ...);
>
> Is much clearer than:
>
> VPU_PMEM0_VIRT(vpu) = dma_alloc_coherent(dev, ...);
>
Thanks. I'll change this.
> > +
> > +#define VPU_PTCM(dev) ((dev)->reg.sram)
> > +#define VPU_DTCM(dev) ((dev)->reg.sram + VPU_DTCM_OFFSET)
>
> These macros also seem to add very little value. They are consumed only
> a couple of times and only serve to conceal how the sram is mapped and
> consumed:
>
> dest = vpu->reg.sram;
> if (fw_type == D_FW)
> dest += VPU_DTCM_OFFSET;
>
> Compared to:
>
> dest = fw_type ? VPU_DTCM(vpu) : VPU_PTCM(vpu);
>
Thanks. I'll change this.
> > +
> > +#define VPU_PTCM_SIZE (96 * SZ_1K)
> > +#define VPU_DTCM_SIZE (32 * SZ_1K)
> > +#define VPU_DTCM_OFFSET 0x18000UL
> > +#define VPU_EXT_P_SIZE SZ_1M
> > +#define VPU_EXT_D_SIZE SZ_4M
> > +#define VPU_P_FW_SIZE (VPU_PTCM_SIZE + VPU_EXT_P_SIZE)
> > +#define VPU_D_FW_SIZE (VPU_DTCM_SIZE + VPU_EXT_D_SIZE)
> > +#define SHARE_BUF_SIZE 48
> > +
> > +#define VPU_P_FW "vpu_p.bin"
> > +#define VPU_D_FW "vpu_d.bin"
>
> These macros are of questionable value.
I'll put more comments. Thanks.
>
> > +
> > +#define VPU_BASE 0x0
>
> Is this register really called "base" in the datasheet? From the use in
> the code it looks like it performs a reset and/or PM function.
>
This should be a vpu reset register. We will change this. Thanks.
>
> > +#define VPU_TCM_CFG 0x0008
> > +#define VPU_PMEM_EXT0_ADDR 0x000C
> > +#define VPU_PMEM_EXT1_ADDR 0x0010
> > +#define VPU_TO_HOST 0x001C
> > +#define VPU_DMEM_EXT0_ADDR 0x0014
> > +#define VPU_DMEM_EXT1_ADDR 0x0018
> > +#define HOST_TO_VPU 0x0024
> > +#define VPU_PC_REG 0x0060
> > +#define VPU_WDT_REG 0x0084
> > +
> > +/* vpu inter-processor communication interrupt */
> > +#define VPU_IPC_INT BIT(8)
> > +
> > +/**
> > + * enum vpu_fw_type - VPU firmware type
> > + *
> > + * @P_FW: program firmware
> > + * @D_FW: data firmware
> > + *
> > + */
> > +enum vpu_fw_type {
> > + P_FW,
> > + D_FW,
> > +};
> > +
> > +/**
> > + * struct vpu_mem - VPU memory information
>
> Perhaps "VPU extended program/data memory information" instead.
>
I'll change this.
>
> > + *
> > + * @p_va: the kernel virtual memory address of
> > + * VPU extended program memory
> > + * @d_va: the kernel virtual memory address of VPU extended data memory
> > + * @p_iova: the iova memory address of VPU extended program memory
> > + * @d_iova: the iova memory address of VPU extended data memory
> > + */
> > +struct vpu_mem {
> > + void *p_va;
> > + void *d_va;
> > + dma_addr_t p_iova;
> > + dma_addr_t d_iova;
> > +};
>
> Might be better as:
>
> struct {
> void *va;
> dma_addr_t iova;
> }
>
> This can be then be used as: vpu->mem[P_FW].va . This doesn't matter
> much yet but it would helps us common up some code later.
I'll change this. Thanks
>
>
> > +
> > +/**
> > + * struct vpu_regs - VPU SRAM and configuration registers
> > + *
> > + * @sram: the register for VPU sram
> > + * @cfg: the register for VPU configuration
> > + * @irq: the irq number for VPU interrupt
> > + */
> > +struct vpu_regs {
> > + void __iomem *sram;
>
> This is called TCM everywhere else. Why a different name for it here?
Yes, for the consistency, we should call it TCM.Thanks.
>
>
> > + void __iomem *cfg;
> > + int irq;
> > +};
> > +
> > +/**
> > + * struct vpu_run - VPU initialization status
> > + *
> > + * @signaled: the signal of vpu initialization completed
> > + * @fw_ver: VPU firmware version
> > + * @wq: wait queue for VPU initialization status
> > + */
> > +struct vpu_run {
> > + u32 signaled;
> > + char fw_ver[VPU_FW_VER_LEN];
> > + wait_queue_head_t wq;
> > +};
> > +
> > +/**
> > + * struct vpu_ipi_desc - VPU IPI descriptor
> > + *
> > + * @handler: IPI handler
> > + * @name: the name of IPI handler
> > + * @priv: the private data of IPI handler
> > + */
> > +struct vpu_ipi_desc {
> > + ipi_handler_t handler;
> > + const char *name;
> > + void *priv;
> > +};
> > +
> > +/**
> > + * struct share_obj - The DTCM (Data Tightly-Coupled Memory) buffer shared with
> > + * AP and VPU
>
> Remove "The" (there are more than one of these buffers).
I'll change this.
>
> > + *
> > + * @id: IPI id
> > + * @len: share buffer length
> > + * @share_buf: share buffer data
> > + */
> > +struct share_obj {
> > + int32_t id;
> > + uint32_t len;
> > + unsigned char share_buf[SHARE_BUF_SIZE];
> > +};
> > +
> > +/**
> > + * struct mtk_vpu - vpu driver data
> > + * @mem: VPU extended memory information
> > + * @reg: VPU SRAM and configuration registers
> > + * @run: VPU initialization status
> > + * @ipi_desc: VPU IPI descriptor
> > + * @recv_buf: VPU DTCM share buffer for receiving. The
> > + * receive buffer is only accessed in interrupt context.
> > + * @send_buf: VPU DTCM share buffer for sending
> > + * @dev: VPU struct device
> > + * @clk: VPU clock on/off
> > + * @vpu_mutex: protect mtk_vpu (except recv_buf) and ensure only
> > + * one client to use VPU service at a time. For example,
> > + * suppose a client is using VPU to decode VP8.
> > + * If the other client wants to encode VP8,
> > + * it has to wait until VP8 decode completes.
> > + *
> > + */
> > +struct mtk_vpu {
> > + struct vpu_mem mem;
>
> Rename to extmem?
Yes, I'll rename this.
>
> > + struct vpu_regs reg;
> > + struct vpu_run run;
> > + struct vpu_ipi_desc ipi_desc[IPI_MAX];
> > + struct share_obj *recv_buf;
> > + struct share_obj *send_buf;
> > + struct device *dev;
> > + struct clk *clk;
> > + struct mutex vpu_mutex; /* for protecting vpu data data structure */
> > +};
> > +
> > +/* the thread calls the function should hold the |vpu_mutex| */
>
> Remove this comment. Its unhelpful: the code does not meet this
> requirement because it overstates the scope of vpu_mutex .
I'll change this.
>
> > +static inline void vpu_cfg_writel(struct mtk_vpu *vpu, u32 val, u32 offset)
> > +{
> > + writel(val, vpu->reg.cfg + offset);
> > +}
> > +
> > +static inline u32 vpu_cfg_readl(struct mtk_vpu *vpu, u32 offset)
> > +{
> > + return readl(vpu->reg.cfg + offset);
> > +}
> > +
> > +static inline bool vpu_running(struct mtk_vpu *vpu)
> > +{
> > + return vpu_cfg_readl(vpu, VPU_BASE) & BIT(0);
> > +}
> > +
> > +void vpu_disable_clock(struct platform_device *pdev)
> > +{
> > + struct mtk_vpu *vpu = platform_get_drvdata(pdev);
> > +
> > + /* Disable VPU watchdog */
> > + vpu_cfg_writel(vpu,
> > + vpu_cfg_readl(vpu, VPU_WDT_REG) & ~(1L<<31),
> > + VPU_WDT_REG);
>
> This code combines a reference counted clock disable with a
> not-reference counted register write and will result in the watchdot
> being spuriously disabled.
>
> This will definitely happen if vpu_debug_read() is called at the wrong
> time, possibly may also be an issue for concurrent H.264 and VP8 operations.
>
Yes, you are right. I'll add a reference count to avoid this problem.
Thanks.
> > +
> > + clk_disable_unprepare(vpu->clk);
> > +}
> > +
> > +int vpu_enable_clock(struct platform_device *pdev)
> > +{
> > + struct mtk_vpu *vpu = platform_get_drvdata(pdev);
> > + int ret;
> > +
> > + ret = clk_prepare_enable(vpu->clk);
> > + if (ret)
> > + return ret;
> > + /* Enable VPU watchdog */
> > + vpu_cfg_writel(vpu, vpu_cfg_readl(vpu, VPU_WDT_REG) | (1L << 31),
> > + VPU_WDT_REG);
>
> As above.
>
Ok.
>
> > +
> > + return ret;
> > +}
> > +
> > +int vpu_ipi_register(struct platform_device *pdev,
> > + enum ipi_id id, ipi_handler_t handler,
> > + const char *name, void *priv)
> > +{
> > + struct mtk_vpu *vpu = platform_get_drvdata(pdev);
> > + struct vpu_ipi_desc *ipi_desc;
> > +
> > + if (!vpu) {
> > + dev_err(&pdev->dev, "vpu device in not ready\n");
> > + return -EPROBE_DEFER;
> > + }
> > +
> > + if (id < IPI_MAX && handler != NULL) {
> > + ipi_desc = vpu->ipi_desc;
> > + ipi_desc[id].name = name;
> > + ipi_desc[id].handler = handler;
> > + ipi_desc[id].priv = priv;
> > + return 0;
> > + }
> > +
> > + dev_err(&pdev->dev, "register vpu ipi with invalid arguments\n");
> > + return -EINVAL;
> > +}
>
> This API, and manu of its friends lower down the file, appear to be a
> way to send 32 byte messages to different endpoints on another processor
> on the SoC.
>
> Just interested to know if you evaluated the mailbox driver
> infrastructure for this? Its a good fit for the messaging but doesn't
> offer any support for the direct access to TCM or extended memory.
>
Each codec (vp8/h264) has it own data structure on VPU. CPU can get the
their kernel virtual address with this driver APIs.
When CPU accesses the data structure in VPU, VPU won't write the data
structure at the same time and vice versa.
>
> > +int vpu_ipi_send(struct platform_device *pdev,
> > + enum ipi_id id, void *buf,
> > + unsigned int len, unsigned int wait)
> > +{
> > + struct mtk_vpu *vpu = platform_get_drvdata(pdev);
> > + struct share_obj *send_obj = vpu->send_buf;
> > + unsigned long timeout;
> > +
> > + if (id >= IPI_MAX || len > sizeof(send_obj->share_buf) || buf == NULL) {
> > + dev_err(vpu->dev, "failed to send ipi message\n");
> > + return -EINVAL;
> > + }
> > +
> > + if (!vpu_running(vpu)) {
> > + dev_err(vpu->dev, "vpu_ipi_send: VPU is not running\n");
> > + return -ENXIO;
> > + }
> > +
> > + mutex_lock(&vpu->vpu_mutex);
> > + if (vpu_cfg_readl(vpu, HOST_TO_VPU) && !wait) {
> > + mutex_unlock(&vpu->vpu_mutex);
> > + return -EBUSY;
> > + }
>
> This branch is unreachable (no caller ever sets wait is never set to false).
The parameter "wait" will be removed. Thanks.
>
>
> > +
> > + if (wait)
> > + while (vpu_cfg_readl(vpu, HOST_TO_VPU))
> > + ;
>
> What is this loop for? This code should only be reachable if we timed
> out in the code below so the likely effect of this loop is to
> permantently wedge a thread in a manner that frustrates signal delivery.
This busy waiting will be removed since the mutex_lock and the blow busy
waiting make sure a client sending commanded can be accepted by VPU
hardware.
>
>
> > +
> > + memcpy((void *)send_obj->share_buf, buf, len);
> > + send_obj->len = len;
> > + send_obj->id = id;
> > + vpu_cfg_writel(vpu, 0x1, HOST_TO_VPU);
> > +
> > + /* Wait until VPU receives the command */
> > + timeout = jiffies + msecs_to_jiffies(IPI_TIMEOUT_MS);
> > + do {
> > + if (time_after(jiffies, timeout)) {
> > + dev_err(vpu->dev, "vpu_ipi_send: IPI timeout!\n");
> > + return -EIO;
> > + }
> > + } while (vpu_cfg_readl(vpu, HOST_TO_VPU));
>
> Do we need to busy wait every time we communicate with the co-processor?
> Couldn't we put this wait *before* we write to HOST_TO_VPU above.
>
> That way we only spin when there is a need to.
>
Since the hardware VPU only allows that one client sends the command to
it each time.
We need the wait to make sure VPU accepted the command and cleared the
interrupt and then the next command would be served.
>
> > +
> > + mutex_unlock(&vpu->vpu_mutex);
> > +
> > + return 0;
> > +}
> > +
> > +void *vpu_mapping_dm_addr(struct platform_device *pdev,
> > + void *dtcm_dmem_addr)
>
>
> Use a different type: dtcm_dmem_addr is not a pointer on this CPU.
I'll change this.
>
> > +{
> > + struct mtk_vpu *vpu = platform_get_drvdata(pdev);
> > + unsigned long p_vpu_dtcm = (unsigned long)VPU_DTCM(vpu);
>
> p_vpu_dtcm *is* a pointer on this CPU. Why cast it so that it isn't.
>
> > + unsigned long ul_dtcm_dmem_addr = (unsigned long)(dtcm_dmem_addr);
> > +
> > + if (dtcm_dmem_addr == NULL ||
> > + (ul_dtcm_dmem_addr > (VPU_DTCM_SIZE + VPU_EXT_D_SIZE))) {
> > + dev_err(vpu->dev, "invalid virtual data memory address\n");
> > + return ERR_PTR(-EINVAL);
> > + }
> > +
> > + if (ul_dtcm_dmem_addr < VPU_DTCM_SIZE)
> > + return (void *)(ul_dtcm_dmem_addr + p_vpu_dtcm);
>
> Starting with the pointer will preserve type better:
>
> return vpu->reg.sram + VPU_DTCM_OFFSET + ul_dtcm_dmem_addr;
>
> > +
> > + return (void *)((ul_dtcm_dmem_addr - VPU_DTCM_SIZE) +
> > + VPU_DMEM0_VIRT(vpu));
>
> Likewise this code is clearer with the pointer first.
>
> return vpu->mem.d_va + (ul_dtcm_dmem_addr - VPU_DTCM_SIZE);
>
>
I'll change these above. Thanks.
> > +}
> > +
> > +dma_addr_t *vpu_mapping_iommu_dm_addr(struct platform_device *pdev,
> > + void *dmem_addr)
>
> This function does not return a pointer to a dma_addr_t. It just returns
> a regular dma_addr_t (that's why all the callers of this function cast
> it back ;-) ).
>
I'll change this. Thanks.
>
> dmem_addr is also not a pointer and this function does not return a
> pointer to a dma_addr_t. It just returns a regular dma_addr_t.
>
> > +{
> > + unsigned long ul_dmem_addr = (unsigned long)(dmem_addr);
> > + struct mtk_vpu *vpu = platform_get_drvdata(pdev);
> > +
> > + if (dmem_addr == NULL ||
> > + (ul_dmem_addr < VPU_DTCM_SIZE) ||
> > + (ul_dmem_addr > (VPU_DTCM_SIZE + VPU_EXT_D_SIZE))) {
> > + dev_err(vpu->dev, "invalid IOMMU data memory address\n");
> > + return ERR_PTR(-EINVAL);
> > + }
> > +
> > + return (dma_addr_t *)((ul_dmem_addr - VPU_DTCM_SIZE) +
> > + VPU_DMEM0_IOVA(vpu));
>
> Better written as (this would also have made explicit the type problem
> with the function:
>
> return vpu->mem.d_iova + (ul_dmem_addr - VPU_DTCM_SIZE);
>
I'll change this. Thanks.
> > +}
> > +
> > +struct platform_device *vpu_get_plat_device(struct platform_device *pdev)
> > +{
> > + struct device *dev = &pdev->dev;
> > + struct device_node *vpu_node;
> > + struct platform_device *vpu_pdev;
> > +
> > + vpu_node = of_parse_phandle(dev->of_node, "vpu", 0);
> > + if (!vpu_node) {
> > + dev_err(dev, "can't get vpu node\n");
> > + return NULL;
> > + }
> > +
> > + vpu_pdev = of_find_device_by_node(vpu_node);
> > + if (WARN_ON(!vpu_pdev)) {
> > + dev_err(dev, "vpu pdev failed\n");
> > + of_node_put(vpu_node);
> > + return NULL;
> > + }
> > +
> > + return vpu_pdev;
> > +}
>
> This function looks a bit weird to me. Why do we need to keep consulting
> the devicetree every time we want to start/stop the clock?
>
> I would have expected this code to be part of the init routine of
> anything that needs this and the platform_device would then be cached.
>
Yes, I think every callers should call the function to get vpu platform
device and cache it in their probe function. Thanks.
>
> > +
> > +/* load vpu program/data memory */
> > +static void load_requested_vpu(struct mtk_vpu *vpu,
> > + size_t fw_size,
> > + const u8 *fw_data,
> > + u8 fw_type)
> > +{
> > + size_t target_size = fw_type ? VPU_DTCM_SIZE : VPU_PTCM_SIZE;
> > + size_t extra_fw_size = 0;
> > + void *dest;
> > +
> > + /* reset VPU */
> > + vpu_cfg_writel(vpu, 0x0, VPU_BASE);
> > +
> > + /* handle extended firmware size */
> > + if (fw_size > target_size) {
> > + dev_dbg(vpu->dev, "fw size %lx > limited fw size %lx\n",
> > + fw_size, target_size);
> > + extra_fw_size = fw_size - target_size;
> > + dev_dbg(vpu->dev, "extra_fw_size %lx\n", extra_fw_size);
> > + fw_size = target_size;
> > + }
> > + dest = fw_type ? VPU_DTCM(vpu) : VPU_PTCM(vpu);
> > + memcpy(dest, fw_data, fw_size);
> > + /* download to extended memory if need */
> > + if (extra_fw_size > 0) {
> > + dest = fw_type ?
> > + VPU_DMEM0_VIRT(vpu) : VPU_PMEM0_VIRT(vpu);
> > +
> > + dev_dbg(vpu->dev, "download extended memory type %x\n",
> > + fw_type);
> > + memcpy(dest, fw_data + target_size, extra_fw_size);
> > + }
> > +}
> > +
> > +int vpu_load_firmware(struct platform_device *pdev)
> > +{
> > + struct mtk_vpu *vpu = platform_get_drvdata(pdev);
> > + struct device *dev = &pdev->dev;
> > + struct vpu_run *run = &vpu->run;
> > + const struct firmware *vpu_fw;
> > + int ret;
> > +
> > + if (!pdev) {
> > + dev_err(dev, "VPU platform device is invalid\n");
> > + return -EINVAL;
> > + }
> > +
> > + mutex_lock(&vpu->vpu_mutex);
> > +
> > + ret = vpu_enable_clock(pdev);
> > + if (ret) {
> > + dev_err(dev, "enable clock failed %d\n", ret);
> > + goto OUT_LOAD_FW;
> > + }
> > +
> > + if (vpu_running(vpu)) {
> > + vpu_disable_clock(pdev);
> > + mutex_unlock(&vpu->vpu_mutex);
> > + dev_warn(dev, "vpu is running already\n");
> > + return 0;
> > + }
> > +
> > + run->signaled = false;
> > + dev_dbg(vpu->dev, "firmware request\n");
> > + ret = request_firmware(&vpu_fw, VPU_P_FW, dev);
> > + if (ret < 0) {
> > + dev_err(dev, "Failed to load %s, %d\n", VPU_P_FW, ret);
> > + goto OUT_LOAD_FW;
> > + }
> > + if (vpu_fw->size > VPU_P_FW_SIZE) {
> > + ret = -EFBIG;
> > + dev_err(dev, "program fw size %zu is abnormal\n", vpu_fw->size);
> > + goto OUT_LOAD_FW;
> > + }
>
> Possibly move request_firmware(), release_firmware() and the associated
> error handling into load_requested_vpu(). It can all be parameterized
> and the filename of the firmware means error reports will still be clear
> about whether the p or d firmware is faulty.
>
I'll change this.
>
> > + dev_dbg(vpu->dev, "Downloaded program fw size: %zu.\n",
> > + vpu_fw->size);
> > + /* Downloading program firmware to device*/
> > + load_requested_vpu(vpu, vpu_fw->size, vpu_fw->data,
> > + P_FW);
> > + release_firmware(vpu_fw);
> > +
> > + ret = request_firmware(&vpu_fw, VPU_D_FW, dev);
> > + if (ret < 0) {
> > + dev_err(dev, "Failed to load %s, %d\n", VPU_D_FW, ret);
> > + goto OUT_LOAD_FW;
> > + }
> > + if (vpu_fw->size > VPU_D_FW_SIZE) {
> > + ret = -EFBIG;
> > + dev_err(dev, "data fw size %zu is abnormal\n", vpu_fw->size);
> > + goto OUT_LOAD_FW;
> > + }
> > + dev_dbg(vpu->dev, "Downloaded data fw size: %zu.\n",
> > + vpu_fw->size);
> > + /* Downloading data firmware to device */
> > + load_requested_vpu(vpu, vpu_fw->size, vpu_fw->data,
> > + D_FW);
> > + release_firmware(vpu_fw);
> > + /* boot up vpu */
> > + vpu_cfg_writel(vpu, 0x1, VPU_BASE);
> > +
> > + ret = wait_event_interruptible_timeout(run->wq,
> > + run->signaled,
> > + msecs_to_jiffies(INIT_TIMEOUT_MS)
> > + );
> > + if (0 == ret) {
> > + ret = -ETIME;
> > + dev_err(dev, "wait vpu initialization timout!\n");
> > + goto OUT_LOAD_FW;
> > + } else if (-ERESTARTSYS == ret) {
> > + dev_err(dev, "wait vpu interrupted by a signal!\n");
> > + goto OUT_LOAD_FW;
> > + }
> > +
> > + ret = 0;
> > + dev_info(dev, "vpu is ready. Fw version %s\n", run->fw_ver);
> > +
> > +OUT_LOAD_FW:
> > + vpu_disable_clock(pdev);
> > + mutex_unlock(&vpu->vpu_mutex);
> > +
> > + return ret;
> > +}
> > +
> > +static void vpu_init_ipi_handler(void *data, unsigned int len, void *priv)
> > +{
> > + struct mtk_vpu *vpu = (struct mtk_vpu *)priv;
> > + struct vpu_run *run = (struct vpu_run *)data;
> > +
> > + vpu->run.signaled = run->signaled;
> > + strncpy(vpu->run.fw_ver, run->fw_ver, VPU_FW_VER_LEN);
> > + wake_up_interruptible(&vpu->run.wq);
> > +}
> > +
> > +static int vpu_debug_open(struct inode *inode, struct file *file)
> > +{
> > + file->private_data = inode->i_private;
> > + return 0;
> > +}
> > +
> > +static ssize_t vpu_debug_read(struct file *file, char __user *user_buf,
> > + size_t count, loff_t *ppos)
> > +{
> > + char buf[256];
> > + unsigned int len;
> > + unsigned int running, pc, vpu_to_host, host_to_vpu, wdt;
> > + int ret;
> > + struct device *dev = file->private_data;
> > + struct platform_device *pdev = to_platform_device(dev);
> > + struct mtk_vpu *vpu = dev_get_drvdata(dev);
> > +
> > + ret = vpu_enable_clock(pdev);
> > + if (ret) {
> > + dev_err(vpu->dev, "[VPU] enable clock failed %d\n", ret);
> > + return 0;
> > + }
> > +
> > + /* vpu register status */
> > + running = vpu_running(vpu);
> > + pc = vpu_cfg_readl(vpu, VPU_PC_REG);
> > + wdt = vpu_cfg_readl(vpu, VPU_WDT_REG);
> > + host_to_vpu = vpu_cfg_readl(vpu, HOST_TO_VPU);
> > + vpu_to_host = vpu_cfg_readl(vpu, VPU_TO_HOST);
> > + vpu_disable_clock(pdev);
> > +
> > + if (running) {
> > + len = sprintf(buf, "VPU is running\n\n"
> > + "FW Version: %s\n"
> > + "PC: 0x%x\n"
> > + "WDT: 0x%x\n"
> > + "Host to VPU: 0x%x\n"
> > + "VPU to Host: 0x%x\n",
> > + vpu->run.fw_ver, pc, wdt,
> > + host_to_vpu, vpu_to_host);
> > + } else {
> > + len = sprintf(buf, "VPU not running\n");
> > + }
> > +
> > + return simple_read_from_buffer(user_buf, count, ppos, buf, len);
> > +}
> > +
> > +static const struct file_operations vpu_debug_fops = {
> > + .open = vpu_debug_open,
> > + .read = vpu_debug_read,
> > +};
>
> These operations should be conditional on CONFIG_DEBUG_FS.
I'll change this.
>
> > +
> > +static void vpu_free_p_ext_mem(struct mtk_vpu *vpu)
> > +{
> > + struct device *dev = vpu->dev;
> > + struct iommu_domain *domain = iommu_get_domain_for_dev(dev);
> > +
> > + dma_free_coherent(dev, VPU_EXT_P_SIZE, VPU_PMEM0_VIRT(vpu),
> > + VPU_PMEM0_IOVA(vpu));
> > +
> > + if (domain)
> > + iommu_detach_device(domain, vpu->dev);
> > +}
> > +
> > +static void vpu_free_d_ext_mem(struct mtk_vpu *vpu)
> > +{
> > + struct device *dev = vpu->dev;
> > + struct iommu_domain *domain = iommu_get_domain_for_dev(dev);
> > +
> > + dma_free_coherent(dev, VPU_EXT_D_SIZE, VPU_DMEM0_VIRT(vpu),
> > + VPU_DMEM0_IOVA(vpu));
> > +
> > + if (domain)
> > + iommu_detach_device(domain, dev);
> > +}
>
> Look like this could be parameterized and combined with vpu_free_p_ext_mem.
I'll change this.
>
>
> > +
> > +static int vpu_alloc_p_ext_mem(struct mtk_vpu *vpu)
> > +{
> > + struct device *dev = vpu->dev;
> > + struct iommu_domain *domain = iommu_get_domain_for_dev(dev);
> > + phys_addr_t p_pa;
> > +
> > + VPU_PMEM0_VIRT(vpu) = dma_alloc_coherent(dev,
> > + VPU_EXT_P_SIZE,
> > + &(VPU_PMEM0_IOVA(vpu)),
> > + GFP_KERNEL);
> > + if (VPU_PMEM0_VIRT(vpu) == NULL) {
> > + dev_err(dev, "Failed to allocate the extended program memory\n");
> > + return PTR_ERR(VPU_PMEM0_VIRT(vpu));
> > + }
> > +
> > + p_pa = iommu_iova_to_phys(domain, vpu->mem.p_iova);
> > + /* Disable extend0. Enable extend1 */
> > + vpu_cfg_writel(vpu, 0x1, VPU_PMEM_EXT0_ADDR);
> > + vpu_cfg_writel(vpu, (p_pa & 0xFFFFF000), VPU_PMEM_EXT1_ADDR);
> > +
> > + dev_info(dev, "Program extend memory phy=0x%llx virt=0x%p iova=0x%llx\n",
> > + (unsigned long long)p_pa,
> > + VPU_PMEM0_VIRT(vpu),
> > + (unsigned long long)VPU_PMEM0_IOVA(vpu));
> > +
> > + return 0;
> > +}
> > +
> > +static int vpu_alloc_d_ext_mem(struct mtk_vpu *vpu)
> > +{
> > + struct device *dev = vpu->dev;
> > + struct iommu_domain *domain = iommu_get_domain_for_dev(dev);
> > + phys_addr_t d_pa;
> > +
> > + VPU_DMEM0_VIRT(vpu) = dma_alloc_coherent(dev,
> > + VPU_EXT_D_SIZE,
> > + &(VPU_DMEM0_IOVA(vpu)),
> > + GFP_KERNEL);
> > + if (VPU_DMEM0_VIRT(vpu) == NULL) {
> > + dev_err(dev, "Failed to allocate the extended data memory\n");
> > + return PTR_ERR(VPU_DMEM0_VIRT(vpu));
> > + }
> > +
> > + d_pa = iommu_iova_to_phys(domain, vpu->mem.d_iova);
> > +
> > + /* Disable extend0. Enable extend1 */
> > + vpu_cfg_writel(vpu, 0x1, VPU_DMEM_EXT0_ADDR);
> > + vpu_cfg_writel(vpu, (d_pa & 0xFFFFF000),
> > + VPU_DMEM_EXT1_ADDR);
> > +
> > + dev_info(dev, "Data extend memory phy=0x%llx virt=0x%p iova=0x%llx\n",
> > + (unsigned long long)d_pa,
> > + VPU_DMEM0_VIRT(vpu),
> > + (unsigned long long)VPU_DMEM0_IOVA(vpu));
> > +
> > + return 0;
> > +}
>
> Also looks suitable for parameterizing and combining with
> vpu_alloc_p_ext_mem .
>
I'll change this.
>
> > +
> > +static void vpu_ipi_handler(struct mtk_vpu *vpu)
> > +{
> > + struct share_obj *rcv_obj = vpu->recv_buf;
> > + struct vpu_ipi_desc *ipi_desc = vpu->ipi_desc;
> > +
> > + if (rcv_obj->id < IPI_MAX && ipi_desc[rcv_obj->id].handler) {
> > + ipi_desc[rcv_obj->id].handler(rcv_obj->share_buf,
> > + rcv_obj->len,
> > + ipi_desc[rcv_obj->id].priv);
> > + } else {
> > + dev_err(vpu->dev, "No such ipi id = %d\n", rcv_obj->id);
> > + }
> > +}
> > +
> > +static int vpu_ipi_init(struct mtk_vpu *vpu)
> > +{
> > + /* Disable VPU to host interrupt */
> > + vpu_cfg_writel(vpu, 0x0, VPU_TO_HOST);
> > +
> > + /* shared buffer initialization */
> > + vpu->recv_buf = (struct share_obj *)VPU_DTCM(vpu);
> > + vpu->send_buf = vpu->recv_buf + 1;
> > + memset(vpu->recv_buf, 0, sizeof(struct share_obj));
> > + memset(vpu->send_buf, 0, sizeof(struct share_obj));
> > + mutex_init(&vpu->vpu_mutex);
> > +
> > + return 0;
> > +}
> > +
> > +static irqreturn_t vpu_irq_handler(int irq, void *priv)
> > +{
> > + struct mtk_vpu *vpu = priv;
> > + uint32_t vpu_to_host = vpu_cfg_readl(vpu, VPU_TO_HOST);
> > +
> > + if (vpu_to_host & VPU_IPC_INT)
> > + vpu_ipi_handler(vpu);
> > + else
> > + dev_err(vpu->dev, "vpu watchdog timeout!\n");
> > +
> > + /* VPU won't send another interrupt until we set VPU_TO_HOST to 0. */
> > + vpu_cfg_writel(vpu, 0x0, VPU_TO_HOST);
>
> If we were triggered by a watchdog then how long will it be before the
> next watchdog interrupt? Will we end up spamming the logs?
Thanks for reminding me.
When watchdog was timeout, I think VPU driver should notify registered
drivers and reset vpu hardware for next run. I'll change this.
> > +
> > + return IRQ_HANDLED;
> > +}
> > +
> > +static struct dentry *vpu_debugfs;
> > +static int mtk_vpu_probe(struct platform_device *pdev)
> > +{
> > + struct mtk_vpu *vpu;
> > + struct device *dev;
> > + struct resource *res;
> > + int ret = 0;
> > +
> > + dev_dbg(&pdev->dev, "initialization\n");
> > +
> > + dev = &pdev->dev;
> > + vpu = devm_kzalloc(dev, sizeof(*vpu), GFP_KERNEL);
> > + if (!vpu)
> > + return -ENOMEM;
> > +
> > + vpu->dev = &pdev->dev;
> > + res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "sram");
> > + vpu->reg.sram = devm_ioremap_resource(dev, res);
> > + if (IS_ERR(vpu->reg.sram)) {
> > + dev_err(dev, "devm_ioremap_resource vpu sram failed.\n");
> > + return PTR_ERR(vpu->reg.sram);
> > + }
> > +
> > + res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "cfg_reg");
> > + vpu->reg.cfg = devm_ioremap_resource(dev, res);
> > + if (IS_ERR(vpu->reg.cfg)) {
> > + dev_err(dev, "devm_ioremap_resource vpu cfg failed.\n");
> > + return PTR_ERR(vpu->reg.cfg);
> > + }
> > +
> > + /* Get VPU clock */
> > + vpu->clk = devm_clk_get(dev, "main");
> > + if (vpu->clk == NULL) {
> > + dev_err(dev, "get vpu clock fail\n");
> > + return -EINVAL;
> > + }
> > +
> > + platform_set_drvdata(pdev, vpu);
> > +
> > + ret = vpu_enable_clock(pdev);
> > + if (ret) {
> > + ret = -EINVAL;
> > + return ret;
>
> Why not just "return ret" (or "return -EINVAL")?
I'll change this.
>
> > + }
> > +
> > + dev_dbg(dev, "vpu ipi init\n");
> > + ret = vpu_ipi_init(vpu);
> > + if (ret) {
> > + dev_err(dev, "Failed to init ipi\n");
> > + goto disable_vpu_clk;
> > + }
> > +
> > + platform_set_drvdata(pdev, vpu);
> > +
> > + /* register vpu initialization IPI */
> > + ret = vpu_ipi_register(pdev, IPI_VPU_INIT, vpu_init_ipi_handler,
> > + "vpu_init", vpu);
> > + if (ret) {
> > + dev_err(dev, "Failed to register IPI_VPU_INIT\n");
> > + goto vpu_mutex_destroy;
> > + }
> > +
> > + vpu_debugfs = debugfs_create_file("mtk_vpu", S_IRUGO, NULL, (void *)dev,
> > + &vpu_debug_fops);
> > + if (!vpu_debugfs) {
> > + ret = -ENOMEM;
> > + goto cleanup_ipi;
> > + }
> > +
> > + /* Set PTCM to 96K and DTCM to 32K */
> > + vpu_cfg_writel(vpu, 0x2, VPU_TCM_CFG);
> > +
> > + ret = vpu_alloc_p_ext_mem(vpu);
> > + if (ret) {
> > + dev_err(dev, "Allocate PM failed\n");
> > + goto remove_debugfs;
> > + }
> > +
> > + ret = vpu_alloc_d_ext_mem(vpu);
> > + if (ret) {
> > + dev_err(dev, "Allocate DM failed\n");
> > + goto free_p_mem;
> > + }
> > +
> > + init_waitqueue_head(&vpu->run.wq);
> > +
> > + res = platform_get_resource(pdev, IORESOURCE_IRQ, 0);
> > + if (res == NULL) {
> > + dev_err(dev, "get IRQ resource failed.\n");
> > + ret = -ENXIO;
> > + goto free_d_mem;
> > + }
> > + vpu->reg.irq = platform_get_irq(pdev, 0);
> > + ret = devm_request_irq(dev, vpu->reg.irq, vpu_irq_handler, 0,
> > + pdev->name, vpu);
> > + if (ret) {
> > + dev_err(dev, "failed to request irq\n");
> > + goto free_d_mem;
> > + }
> > +
> > + vpu_disable_clock(pdev);
> > + dev_dbg(dev, "initialization completed\n");
> > +
> > + return 0;
> > +
> > +free_d_mem:
> > + vpu_free_d_ext_mem(vpu);
> > +free_p_mem:
> > + vpu_free_p_ext_mem(vpu);
> > +remove_debugfs:
> > + debugfs_remove(vpu_debugfs);
> > +cleanup_ipi:
> > + memset(vpu->ipi_desc, 0, sizeof(struct vpu_ipi_desc)*IPI_MAX);
> > +vpu_mutex_destroy:
> > + mutex_destroy(&vpu->vpu_mutex);
> > +disable_vpu_clk:
> > + vpu_disable_clock(pdev);
> > +
> > + return ret;
> > +}
> > +
> > +static const struct of_device_id mtk_vpu_match[] = {
> > + {
> > + .compatible = "mediatek,mt8173-vpu",
> > + },
> > + {},
> > +};
> > +MODULE_DEVICE_TABLE(of, mtk_vpu_match);
> > +
> > +static int mtk_vpu_remove(struct platform_device *pdev)
> > +{
> > + struct mtk_vpu *vpu = platform_get_drvdata(pdev);
> > +
> > + vpu_free_p_ext_mem(vpu);
> > + vpu_free_d_ext_mem(vpu);
>
> This looks like it leaks cpu_debugfs and the vpu_mutex.
Yes, I'll change this. Thanks.
>
> > +
> > + return 0;
> > +}
> > +
> > +static struct platform_driver mtk_vpu_driver = {
> > + .probe = mtk_vpu_probe,
> > + .remove = mtk_vpu_remove,
> > + .driver = {
> > + .name = MTK_VPU_DRV_NAME,
> > + .owner = THIS_MODULE,
> > + .of_match_table = mtk_vpu_match,
> > + },
> > +};
> > +
> > +module_platform_driver(mtk_vpu_driver);
> > +
> > +MODULE_LICENSE("GPL v2");
> > +MODULE_DESCRIPTION("Mediatek Video Prosessor Unit driver");
> > diff --git a/drivers/media/platform/mtk-vpu/mtk_vpu_core.h b/drivers/media/platform/mtk-vpu/mtk_vpu_core.h
> > new file mode 100644
> > index 0000000..20cf2a0
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vpu/mtk_vpu_core.h
> > @@ -0,0 +1,161 @@
> > +/*
> > +* Copyright (c) 2015 MediaTek Inc.
> > +* Author: Andrew-CT Chen <andrew-ct.chen-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
> > +*
> > +* This program is free software; you can redistribute it and/or modify
> > +* it under the terms of the GNU General Public License version 2 as
> > +* published by the Free Software Foundation.
> > +*
> > +* This program is distributed in the hope that it will be useful,
> > +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> > +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> > +* GNU General Public License for more details.
> > +*/
> > +
> > +#ifndef _MTK_VPU_CORE_H
> > +#define _MTK_VPU_CORE_H
> > +
> > +#include <linux/platform_device.h>
> > +
> > +/**
> > + * VPU (video processor unit) is a tiny processor controlling video hardware
> > + * related to video codec, scaling and color format converting.
> > + * VPU interfaces with other blocks by share memory and interrupt.
> > + **/
> > +
> > +typedef void (*ipi_handler_t) (void *data,
> > + unsigned int len,
> > + void *priv);
> > +
> > +/**
> > + * enum ipi_id - the id of inter-processor interrupt
> > + *
> > + * @IPI_VPU_INIT: The interrupt from vpu is to notfiy kernel
> > + VPU initialization completed.
> > + * @IPI_VENC_H264: The interrupt from vpu is to notify kernel to
> > + handle H264 video encoder job, and vice versa.
> > + * @IPI_VENC_VP8: The interrupt fro vpu is to notify kernel to
> > + handle VP8 video encoder job,, and vice versa.
> > + * @IPI_VENC_CAPABILITY: The interrupt from vpu is to
> > + get venc hardware capability.
> > + * @IPI_MAX: The maximum IPI number
> > + */
> > +enum ipi_id {
> > + IPI_VPU_INIT = 0,
> > + IPI_VENC_H264,
> > + IPI_VENC_VP8,
> > + IPI_VENC_CAPABILITY,
> > + IPI_MAX,
> > +};
> > +
> > +/**
> > + * vpu_disable_clock - Disable VPU clock
> > + *
> > + * @pdev: VPU platform device
> > + *
> > + *
> > + * Return: Return 0 if the clock is disabled successfully,
> > + * otherwise it is failed.
> > + *
> > + **/
> > +void vpu_disable_clock(struct platform_device *pdev);
> > +
> > +/**
> > + * vpu_enable_clock - Enable VPU clock
> > + *
> > + * @pdev: VPU platform device
> > + *
> > + * Return: Return 0 if the clock is enabled successfully,
> > + * otherwise it is failed.
> > + *
> > + **/
> > +int vpu_enable_clock(struct platform_device *pdev);
> > +
> > +/**
> > + * vpu_ipi_register - register an ipi function
> > + *
> > + * @pdev: VPU platform device
> > + * @id: IPI ID
> > + * @handler: IPI handler
> > + * @name: IPI name
> > + * @priv: private data for IPI handler
> > + *
> > + * Register an ipi function to receive ipi interrupt from VPU.
> > + *
> > + * Return: Return 0 if ipi registers successfully, otherwise it is failed.
> > + */
> > +int vpu_ipi_register(struct platform_device *pdev, enum ipi_id id,
> > + ipi_handler_t handler, const char *name, void *priv);
> > +
> > +/**
> > + * vpu_ipi_send - send data from AP to vpu.
> > + *
> > + * @pdev: VPU platform device
> > + * @id: IPI ID
> > + * @buf: the data buffer
> > + * @len: the data buffer length
> > + * @wait: wait for the last ipi completed.
> > + *
> > + * This function is thread-safe. When this function returns,
> > + * VPU has received the data and starts the processing.
> > + * When the processing completes, IPI handler registered
> > + * by vpu_ipi_register will be called in interrupt context.
> > + *
> > + * Return: Return 0 if sending data successfully, otherwise it is failed.
> > + **/
> > +int vpu_ipi_send(struct platform_device *pdev,
> > + enum ipi_id id, void *buf,
> > + unsigned int len,
> > + unsigned int wait);
> > +
> > +/**
> > + * vpu_get_plat_device - get VPU's platform device
> > + *
> > + * @pdev: the platform device of the module requesting VPU platform
> > + * device for using VPU API.
> > + *
> > + * Return: Return NULL if it is failed.
> > + * otherwise it is VPU's platform device
> > + **/
> > +struct platform_device *vpu_get_plat_device(struct platform_device *pdev);
> > +
> > +/**
> > + * vpu_load_firmware - download VPU firmware and boot it
> > + *
> > + * @pdev: VPU platform device
> > + *
> > + * Return: Return 0 if downloading firmware successfully,
> > + * otherwise it is failed
> > + **/
> > +int vpu_load_firmware(struct platform_device *pdev);
> > +
> > +/**
> > + * vpu_mapping_dm_addr - Mapping DTCM/DMEM to kernel virtual address
> > + *
> > + * @pdev: VPU platform device
> > + * @dmem_addr: VPU's data memory address
> > + *
> > + * Mapping the VPU's DTCM (Data Tightly-Coupled Memory) /
> > + * DMEM (Data Extended Memory) memory address to
> > + * kernel virtual address.
> > + *
> > + * Return: Return ERR_PTR(-EINVAL) if mapping failed,
> > + * otherwise the mapped kernel virtual address
> > + **/
> > +void *vpu_mapping_dm_addr(struct platform_device *pdev,
> > + void *dtcm_dmem_addr);
> > +
> > +/**
> > + * vpu_mapping_iommu_dm_addr - Mapping to iommu address
> > + *
> > + * @pdev: VPU platform device
> > + * @dmem_addr: VPU's extended data memory address
> > + *
> > + * Mapping the VPU's extended data address to iommu address
> > + *
> > + * Return: Return ERR_PTR(-EINVAL) if mapping failed,
> > + * otherwise the mapped iommu address
> > + **/
> > +dma_addr_t *vpu_mapping_iommu_dm_addr(struct platform_device *pdev,
> > + void *dmem_addr);
> > +#endif /* _MTK_VPU_CORE_H */
> > diff --git a/drivers/media/platform/mtk-vpu/vp8_enc/venc_vp8_vpu.h b/drivers/media/platform/mtk-vpu/vp8_enc/venc_vp8_vpu.h
> > new file mode 100644
> > index 0000000..4e09eec
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vpu/vp8_enc/venc_vp8_vpu.h
>
> Like the H.264 header. Why is this file included in this patch? It is
> not included by anything in the patch and defines symbols that do not
> exist yet.
This should be moved to vp8 patch. Thanks.
>
>
> Daniel.
--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 84+ messages in thread
* [RESEND RFC/PATCH 3/8] media: platform: mtk-vpu: Support Mediatek VPU
@ 2015-11-27 12:10 ` andrew-ct chen
0 siblings, 0 replies; 84+ messages in thread
From: andrew-ct chen @ 2015-11-27 12:10 UTC (permalink / raw)
To: linux-arm-kernel
Thanks a lot for your review comments.
On Wed, 2015-11-25 at 16:11 +0000, Daniel Thompson wrote:
> On 17/11/15 12:54, Tiffany Lin wrote:
> > From: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
> >
> > The VPU driver for hw video codec embedded in Mediatek's MT8173 SOCs.
> > It is able to handle video decoding/encoding of in a range of formats.
> > The driver provides with VPU firmware download, memory management and
> > the communication interface between CPU and VPU.
> > For VPU initialization, it will create virtual memory for CPU access and
> > IOMMU address for vcodec hw device access. When a decode/encode instance
> > opens a device node, vpu driver will download vpu firmware to the device.
> > A decode/encode instant will decode/encode a frame using VPU
> > interface to interrupt vpu to handle decoding/encoding jobs.
> >
> > Signed-off-by: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
> > ---
> > drivers/media/platform/Kconfig | 6 +
> > drivers/media/platform/Makefile | 2 +
> > drivers/media/platform/mtk-vpu/Makefile | 1 +
> > .../platform/mtk-vpu/h264_enc/venc_h264_vpu.h | 127 +++
> > .../media/platform/mtk-vpu/include/venc_ipi_msg.h | 212 +++++
> > drivers/media/platform/mtk-vpu/mtk_vpu_core.c | 823 ++++++++++++++++++++
> > drivers/media/platform/mtk-vpu/mtk_vpu_core.h | 161 ++++
> > .../media/platform/mtk-vpu/vp8_enc/venc_vp8_vpu.h | 119 +++
> > 8 files changed, 1451 insertions(+)
> > create mode 100644 drivers/media/platform/mtk-vpu/Makefile
> > create mode 100644 drivers/media/platform/mtk-vpu/h264_enc/venc_h264_vpu.h
> > create mode 100644 drivers/media/platform/mtk-vpu/include/venc_ipi_msg.h
> > create mode 100644 drivers/media/platform/mtk-vpu/mtk_vpu_core.c
> > create mode 100644 drivers/media/platform/mtk-vpu/mtk_vpu_core.h
> > create mode 100644 drivers/media/platform/mtk-vpu/vp8_enc/venc_vp8_vpu.h
> >
> > diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
> > index ccbc974..f98eb47 100644
> > --- a/drivers/media/platform/Kconfig
> > +++ b/drivers/media/platform/Kconfig
> > @@ -148,6 +148,12 @@ config VIDEO_CODA
> > Coda is a range of video codec IPs that supports
> > H.264, MPEG-4, and other video formats.
> >
> > +config MEDIATEK_VPU
> > + bool "Mediatek Video Processor Unit"
> > + ---help---
> > + This driver provides downloading firmware vpu and
> > + communicating with vpu.
> > +
>
> This looks pretty broken.
>
> Shouldn't this option be tristate? Why are there no depends-on or selects?
>
Yes, this should be tristate and depends on VIDEO_DEV && VIDEO_V4L2 &&
ARCH_MEDIATEK
> Also I think the help text could be more descriptive here (and so does
> checkpatch ;-) ).
>
I'll put more descriptive. Thanks.
>
> > diff --git a/drivers/media/platform/mtk-vpu/h264_enc/venc_h264_vpu.h b/drivers/media/platform/mtk-vpu/h264_enc/venc_h264_vpu.h
> > new file mode 100644
> > index 0000000..9c8ebdd
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vpu/h264_enc/venc_h264_vpu.h
>
> Why is this file included in *this* patch? It is not included by
> anything in the patch and defines functions that do not exist yet.
I'll move this to the h264 patch.Thanks
>
>
> > diff --git a/drivers/media/platform/mtk-vpu/include/venc_ipi_msg.h b/drivers/media/platform/mtk-vpu/include/venc_ipi_msg.h
> > new file mode 100644
> > index 0000000..a345b98
>
> This file also is not included by anything and should, perhaps be
> included in a different patch.
>
I'll move this to the v4l2 encoder patch.
>
> > diff --git a/drivers/media/platform/mtk-vpu/mtk_vpu_core.c b/drivers/media/platform/mtk-vpu/mtk_vpu_core.c
> > new file mode 100644
> > index 0000000..b524dbc
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vpu/mtk_vpu_core.c
> > @@ -0,0 +1,823 @@
> > +/*
> > +* Copyright (c) 2015 MediaTek Inc.
> > +* Author: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
> > +*
> > +* This program is free software; you can redistribute it and/or modify
> > +* it under the terms of the GNU General Public License version 2 as
> > +* published by the Free Software Foundation.
> > +*
> > +* This program is distributed in the hope that it will be useful,
> > +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> > +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> > +* GNU General Public License for more details.
> > +*/
> > +#include <linux/clk.h>
> > +#include <linux/debugfs.h>
> > +#include <linux/firmware.h>
> > +#include <linux/interrupt.h>
> > +#include <linux/iommu.h>
> > +#include <linux/module.h>
> > +#include <linux/of_address.h>
> > +#include <linux/of_irq.h>
> > +#include <linux/of_platform.h>
> > +#include <linux/sched.h>
> > +#include <linux/sizes.h>
> > +
> > +#include "mtk_vpu_core.h"
> > +
> > +/**
> > + * VPU (video processor unit) is a tiny processor controlling video hardware
> > + * related to video codec, scaling and color format converting.
> > + * VPU interfaces with other blocks by share memory and interrupt.
> > + **/
> > +#define MTK_VPU_DRV_NAME "mtk_vpu"
>
> This is only used once, why not just put this string directly into the
> .name field?
>
I'll change this.
> > +
> > +#define INIT_TIMEOUT_MS 2000U
> > +#define IPI_TIMEOUT_MS 2000U
> > +#define VPU_FW_VER_LEN 16
> > +
> > +/* vpu extended virtural address */
> > +#define VPU_PMEM0_VIRT(vpu) ((vpu)->mem.p_va)
> > +#define VPU_DMEM0_VIRT(vpu) ((vpu)->mem.d_va)
> > +/* vpu extended iova address*/
> > +#define VPU_PMEM0_IOVA(vpu) ((vpu)->mem.p_iova)
> > +#define VPU_DMEM0_IOVA(vpu) ((vpu)->mem.d_iova)
>
> These feel like obfuscation to me. The code looks clearer without the
> macro For example:
>
> vpu->mem.p_va = dma_alloc_coherent(dev, ...);
>
> Is much clearer than:
>
> VPU_PMEM0_VIRT(vpu) = dma_alloc_coherent(dev, ...);
>
Thanks. I'll change this.
> > +
> > +#define VPU_PTCM(dev) ((dev)->reg.sram)
> > +#define VPU_DTCM(dev) ((dev)->reg.sram + VPU_DTCM_OFFSET)
>
> These macros also seem to add very little value. They are consumed only
> a couple of times and only serve to conceal how the sram is mapped and
> consumed:
>
> dest = vpu->reg.sram;
> if (fw_type == D_FW)
> dest += VPU_DTCM_OFFSET;
>
> Compared to:
>
> dest = fw_type ? VPU_DTCM(vpu) : VPU_PTCM(vpu);
>
Thanks. I'll change this.
> > +
> > +#define VPU_PTCM_SIZE (96 * SZ_1K)
> > +#define VPU_DTCM_SIZE (32 * SZ_1K)
> > +#define VPU_DTCM_OFFSET 0x18000UL
> > +#define VPU_EXT_P_SIZE SZ_1M
> > +#define VPU_EXT_D_SIZE SZ_4M
> > +#define VPU_P_FW_SIZE (VPU_PTCM_SIZE + VPU_EXT_P_SIZE)
> > +#define VPU_D_FW_SIZE (VPU_DTCM_SIZE + VPU_EXT_D_SIZE)
> > +#define SHARE_BUF_SIZE 48
> > +
> > +#define VPU_P_FW "vpu_p.bin"
> > +#define VPU_D_FW "vpu_d.bin"
>
> These macros are of questionable value.
I'll put more comments. Thanks.
>
> > +
> > +#define VPU_BASE 0x0
>
> Is this register really called "base" in the datasheet? From the use in
> the code it looks like it performs a reset and/or PM function.
>
This should be a vpu reset register. We will change this. Thanks.
>
> > +#define VPU_TCM_CFG 0x0008
> > +#define VPU_PMEM_EXT0_ADDR 0x000C
> > +#define VPU_PMEM_EXT1_ADDR 0x0010
> > +#define VPU_TO_HOST 0x001C
> > +#define VPU_DMEM_EXT0_ADDR 0x0014
> > +#define VPU_DMEM_EXT1_ADDR 0x0018
> > +#define HOST_TO_VPU 0x0024
> > +#define VPU_PC_REG 0x0060
> > +#define VPU_WDT_REG 0x0084
> > +
> > +/* vpu inter-processor communication interrupt */
> > +#define VPU_IPC_INT BIT(8)
> > +
> > +/**
> > + * enum vpu_fw_type - VPU firmware type
> > + *
> > + * @P_FW: program firmware
> > + * @D_FW: data firmware
> > + *
> > + */
> > +enum vpu_fw_type {
> > + P_FW,
> > + D_FW,
> > +};
> > +
> > +/**
> > + * struct vpu_mem - VPU memory information
>
> Perhaps "VPU extended program/data memory information" instead.
>
I'll change this.
>
> > + *
> > + * @p_va: the kernel virtual memory address of
> > + * VPU extended program memory
> > + * @d_va: the kernel virtual memory address of VPU extended data memory
> > + * @p_iova: the iova memory address of VPU extended program memory
> > + * @d_iova: the iova memory address of VPU extended data memory
> > + */
> > +struct vpu_mem {
> > + void *p_va;
> > + void *d_va;
> > + dma_addr_t p_iova;
> > + dma_addr_t d_iova;
> > +};
>
> Might be better as:
>
> struct {
> void *va;
> dma_addr_t iova;
> }
>
> This can be then be used as: vpu->mem[P_FW].va . This doesn't matter
> much yet but it would helps us common up some code later.
I'll change this. Thanks
>
>
> > +
> > +/**
> > + * struct vpu_regs - VPU SRAM and configuration registers
> > + *
> > + * @sram: the register for VPU sram
> > + * @cfg: the register for VPU configuration
> > + * @irq: the irq number for VPU interrupt
> > + */
> > +struct vpu_regs {
> > + void __iomem *sram;
>
> This is called TCM everywhere else. Why a different name for it here?
Yes, for the consistency, we should call it TCM.Thanks.
>
>
> > + void __iomem *cfg;
> > + int irq;
> > +};
> > +
> > +/**
> > + * struct vpu_run - VPU initialization status
> > + *
> > + * @signaled: the signal of vpu initialization completed
> > + * @fw_ver: VPU firmware version
> > + * @wq: wait queue for VPU initialization status
> > + */
> > +struct vpu_run {
> > + u32 signaled;
> > + char fw_ver[VPU_FW_VER_LEN];
> > + wait_queue_head_t wq;
> > +};
> > +
> > +/**
> > + * struct vpu_ipi_desc - VPU IPI descriptor
> > + *
> > + * @handler: IPI handler
> > + * @name: the name of IPI handler
> > + * @priv: the private data of IPI handler
> > + */
> > +struct vpu_ipi_desc {
> > + ipi_handler_t handler;
> > + const char *name;
> > + void *priv;
> > +};
> > +
> > +/**
> > + * struct share_obj - The DTCM (Data Tightly-Coupled Memory) buffer shared with
> > + * AP and VPU
>
> Remove "The" (there are more than one of these buffers).
I'll change this.
>
> > + *
> > + * @id: IPI id
> > + * @len: share buffer length
> > + * @share_buf: share buffer data
> > + */
> > +struct share_obj {
> > + int32_t id;
> > + uint32_t len;
> > + unsigned char share_buf[SHARE_BUF_SIZE];
> > +};
> > +
> > +/**
> > + * struct mtk_vpu - vpu driver data
> > + * @mem: VPU extended memory information
> > + * @reg: VPU SRAM and configuration registers
> > + * @run: VPU initialization status
> > + * @ipi_desc: VPU IPI descriptor
> > + * @recv_buf: VPU DTCM share buffer for receiving. The
> > + * receive buffer is only accessed in interrupt context.
> > + * @send_buf: VPU DTCM share buffer for sending
> > + * @dev: VPU struct device
> > + * @clk: VPU clock on/off
> > + * @vpu_mutex: protect mtk_vpu (except recv_buf) and ensure only
> > + * one client to use VPU service at a time. For example,
> > + * suppose a client is using VPU to decode VP8.
> > + * If the other client wants to encode VP8,
> > + * it has to wait until VP8 decode completes.
> > + *
> > + */
> > +struct mtk_vpu {
> > + struct vpu_mem mem;
>
> Rename to extmem?
Yes, I'll rename this.
>
> > + struct vpu_regs reg;
> > + struct vpu_run run;
> > + struct vpu_ipi_desc ipi_desc[IPI_MAX];
> > + struct share_obj *recv_buf;
> > + struct share_obj *send_buf;
> > + struct device *dev;
> > + struct clk *clk;
> > + struct mutex vpu_mutex; /* for protecting vpu data data structure */
> > +};
> > +
> > +/* the thread calls the function should hold the |vpu_mutex| */
>
> Remove this comment. Its unhelpful: the code does not meet this
> requirement because it overstates the scope of vpu_mutex .
I'll change this.
>
> > +static inline void vpu_cfg_writel(struct mtk_vpu *vpu, u32 val, u32 offset)
> > +{
> > + writel(val, vpu->reg.cfg + offset);
> > +}
> > +
> > +static inline u32 vpu_cfg_readl(struct mtk_vpu *vpu, u32 offset)
> > +{
> > + return readl(vpu->reg.cfg + offset);
> > +}
> > +
> > +static inline bool vpu_running(struct mtk_vpu *vpu)
> > +{
> > + return vpu_cfg_readl(vpu, VPU_BASE) & BIT(0);
> > +}
> > +
> > +void vpu_disable_clock(struct platform_device *pdev)
> > +{
> > + struct mtk_vpu *vpu = platform_get_drvdata(pdev);
> > +
> > + /* Disable VPU watchdog */
> > + vpu_cfg_writel(vpu,
> > + vpu_cfg_readl(vpu, VPU_WDT_REG) & ~(1L<<31),
> > + VPU_WDT_REG);
>
> This code combines a reference counted clock disable with a
> not-reference counted register write and will result in the watchdot
> being spuriously disabled.
>
> This will definitely happen if vpu_debug_read() is called at the wrong
> time, possibly may also be an issue for concurrent H.264 and VP8 operations.
>
Yes, you are right. I'll add a reference count to avoid this problem.
Thanks.
> > +
> > + clk_disable_unprepare(vpu->clk);
> > +}
> > +
> > +int vpu_enable_clock(struct platform_device *pdev)
> > +{
> > + struct mtk_vpu *vpu = platform_get_drvdata(pdev);
> > + int ret;
> > +
> > + ret = clk_prepare_enable(vpu->clk);
> > + if (ret)
> > + return ret;
> > + /* Enable VPU watchdog */
> > + vpu_cfg_writel(vpu, vpu_cfg_readl(vpu, VPU_WDT_REG) | (1L << 31),
> > + VPU_WDT_REG);
>
> As above.
>
Ok.
>
> > +
> > + return ret;
> > +}
> > +
> > +int vpu_ipi_register(struct platform_device *pdev,
> > + enum ipi_id id, ipi_handler_t handler,
> > + const char *name, void *priv)
> > +{
> > + struct mtk_vpu *vpu = platform_get_drvdata(pdev);
> > + struct vpu_ipi_desc *ipi_desc;
> > +
> > + if (!vpu) {
> > + dev_err(&pdev->dev, "vpu device in not ready\n");
> > + return -EPROBE_DEFER;
> > + }
> > +
> > + if (id < IPI_MAX && handler != NULL) {
> > + ipi_desc = vpu->ipi_desc;
> > + ipi_desc[id].name = name;
> > + ipi_desc[id].handler = handler;
> > + ipi_desc[id].priv = priv;
> > + return 0;
> > + }
> > +
> > + dev_err(&pdev->dev, "register vpu ipi with invalid arguments\n");
> > + return -EINVAL;
> > +}
>
> This API, and manu of its friends lower down the file, appear to be a
> way to send 32 byte messages to different endpoints on another processor
> on the SoC.
>
> Just interested to know if you evaluated the mailbox driver
> infrastructure for this? Its a good fit for the messaging but doesn't
> offer any support for the direct access to TCM or extended memory.
>
Each codec (vp8/h264) has it own data structure on VPU. CPU can get the
their kernel virtual address with this driver APIs.
When CPU accesses the data structure in VPU, VPU won't write the data
structure at the same time and vice versa.
>
> > +int vpu_ipi_send(struct platform_device *pdev,
> > + enum ipi_id id, void *buf,
> > + unsigned int len, unsigned int wait)
> > +{
> > + struct mtk_vpu *vpu = platform_get_drvdata(pdev);
> > + struct share_obj *send_obj = vpu->send_buf;
> > + unsigned long timeout;
> > +
> > + if (id >= IPI_MAX || len > sizeof(send_obj->share_buf) || buf == NULL) {
> > + dev_err(vpu->dev, "failed to send ipi message\n");
> > + return -EINVAL;
> > + }
> > +
> > + if (!vpu_running(vpu)) {
> > + dev_err(vpu->dev, "vpu_ipi_send: VPU is not running\n");
> > + return -ENXIO;
> > + }
> > +
> > + mutex_lock(&vpu->vpu_mutex);
> > + if (vpu_cfg_readl(vpu, HOST_TO_VPU) && !wait) {
> > + mutex_unlock(&vpu->vpu_mutex);
> > + return -EBUSY;
> > + }
>
> This branch is unreachable (no caller ever sets wait is never set to false).
The parameter "wait" will be removed. Thanks.
>
>
> > +
> > + if (wait)
> > + while (vpu_cfg_readl(vpu, HOST_TO_VPU))
> > + ;
>
> What is this loop for? This code should only be reachable if we timed
> out in the code below so the likely effect of this loop is to
> permantently wedge a thread in a manner that frustrates signal delivery.
This busy waiting will be removed since the mutex_lock and the blow busy
waiting make sure a client sending commanded can be accepted by VPU
hardware.
>
>
> > +
> > + memcpy((void *)send_obj->share_buf, buf, len);
> > + send_obj->len = len;
> > + send_obj->id = id;
> > + vpu_cfg_writel(vpu, 0x1, HOST_TO_VPU);
> > +
> > + /* Wait until VPU receives the command */
> > + timeout = jiffies + msecs_to_jiffies(IPI_TIMEOUT_MS);
> > + do {
> > + if (time_after(jiffies, timeout)) {
> > + dev_err(vpu->dev, "vpu_ipi_send: IPI timeout!\n");
> > + return -EIO;
> > + }
> > + } while (vpu_cfg_readl(vpu, HOST_TO_VPU));
>
> Do we need to busy wait every time we communicate with the co-processor?
> Couldn't we put this wait *before* we write to HOST_TO_VPU above.
>
> That way we only spin when there is a need to.
>
Since the hardware VPU only allows that one client sends the command to
it each time.
We need the wait to make sure VPU accepted the command and cleared the
interrupt and then the next command would be served.
>
> > +
> > + mutex_unlock(&vpu->vpu_mutex);
> > +
> > + return 0;
> > +}
> > +
> > +void *vpu_mapping_dm_addr(struct platform_device *pdev,
> > + void *dtcm_dmem_addr)
>
>
> Use a different type: dtcm_dmem_addr is not a pointer on this CPU.
I'll change this.
>
> > +{
> > + struct mtk_vpu *vpu = platform_get_drvdata(pdev);
> > + unsigned long p_vpu_dtcm = (unsigned long)VPU_DTCM(vpu);
>
> p_vpu_dtcm *is* a pointer on this CPU. Why cast it so that it isn't.
>
> > + unsigned long ul_dtcm_dmem_addr = (unsigned long)(dtcm_dmem_addr);
> > +
> > + if (dtcm_dmem_addr == NULL ||
> > + (ul_dtcm_dmem_addr > (VPU_DTCM_SIZE + VPU_EXT_D_SIZE))) {
> > + dev_err(vpu->dev, "invalid virtual data memory address\n");
> > + return ERR_PTR(-EINVAL);
> > + }
> > +
> > + if (ul_dtcm_dmem_addr < VPU_DTCM_SIZE)
> > + return (void *)(ul_dtcm_dmem_addr + p_vpu_dtcm);
>
> Starting with the pointer will preserve type better:
>
> return vpu->reg.sram + VPU_DTCM_OFFSET + ul_dtcm_dmem_addr;
>
> > +
> > + return (void *)((ul_dtcm_dmem_addr - VPU_DTCM_SIZE) +
> > + VPU_DMEM0_VIRT(vpu));
>
> Likewise this code is clearer with the pointer first.
>
> return vpu->mem.d_va + (ul_dtcm_dmem_addr - VPU_DTCM_SIZE);
>
>
I'll change these above. Thanks.
> > +}
> > +
> > +dma_addr_t *vpu_mapping_iommu_dm_addr(struct platform_device *pdev,
> > + void *dmem_addr)
>
> This function does not return a pointer to a dma_addr_t. It just returns
> a regular dma_addr_t (that's why all the callers of this function cast
> it back ;-) ).
>
I'll change this. Thanks.
>
> dmem_addr is also not a pointer and this function does not return a
> pointer to a dma_addr_t. It just returns a regular dma_addr_t.
>
> > +{
> > + unsigned long ul_dmem_addr = (unsigned long)(dmem_addr);
> > + struct mtk_vpu *vpu = platform_get_drvdata(pdev);
> > +
> > + if (dmem_addr == NULL ||
> > + (ul_dmem_addr < VPU_DTCM_SIZE) ||
> > + (ul_dmem_addr > (VPU_DTCM_SIZE + VPU_EXT_D_SIZE))) {
> > + dev_err(vpu->dev, "invalid IOMMU data memory address\n");
> > + return ERR_PTR(-EINVAL);
> > + }
> > +
> > + return (dma_addr_t *)((ul_dmem_addr - VPU_DTCM_SIZE) +
> > + VPU_DMEM0_IOVA(vpu));
>
> Better written as (this would also have made explicit the type problem
> with the function:
>
> return vpu->mem.d_iova + (ul_dmem_addr - VPU_DTCM_SIZE);
>
I'll change this. Thanks.
> > +}
> > +
> > +struct platform_device *vpu_get_plat_device(struct platform_device *pdev)
> > +{
> > + struct device *dev = &pdev->dev;
> > + struct device_node *vpu_node;
> > + struct platform_device *vpu_pdev;
> > +
> > + vpu_node = of_parse_phandle(dev->of_node, "vpu", 0);
> > + if (!vpu_node) {
> > + dev_err(dev, "can't get vpu node\n");
> > + return NULL;
> > + }
> > +
> > + vpu_pdev = of_find_device_by_node(vpu_node);
> > + if (WARN_ON(!vpu_pdev)) {
> > + dev_err(dev, "vpu pdev failed\n");
> > + of_node_put(vpu_node);
> > + return NULL;
> > + }
> > +
> > + return vpu_pdev;
> > +}
>
> This function looks a bit weird to me. Why do we need to keep consulting
> the devicetree every time we want to start/stop the clock?
>
> I would have expected this code to be part of the init routine of
> anything that needs this and the platform_device would then be cached.
>
Yes, I think every callers should call the function to get vpu platform
device and cache it in their probe function. Thanks.
>
> > +
> > +/* load vpu program/data memory */
> > +static void load_requested_vpu(struct mtk_vpu *vpu,
> > + size_t fw_size,
> > + const u8 *fw_data,
> > + u8 fw_type)
> > +{
> > + size_t target_size = fw_type ? VPU_DTCM_SIZE : VPU_PTCM_SIZE;
> > + size_t extra_fw_size = 0;
> > + void *dest;
> > +
> > + /* reset VPU */
> > + vpu_cfg_writel(vpu, 0x0, VPU_BASE);
> > +
> > + /* handle extended firmware size */
> > + if (fw_size > target_size) {
> > + dev_dbg(vpu->dev, "fw size %lx > limited fw size %lx\n",
> > + fw_size, target_size);
> > + extra_fw_size = fw_size - target_size;
> > + dev_dbg(vpu->dev, "extra_fw_size %lx\n", extra_fw_size);
> > + fw_size = target_size;
> > + }
> > + dest = fw_type ? VPU_DTCM(vpu) : VPU_PTCM(vpu);
> > + memcpy(dest, fw_data, fw_size);
> > + /* download to extended memory if need */
> > + if (extra_fw_size > 0) {
> > + dest = fw_type ?
> > + VPU_DMEM0_VIRT(vpu) : VPU_PMEM0_VIRT(vpu);
> > +
> > + dev_dbg(vpu->dev, "download extended memory type %x\n",
> > + fw_type);
> > + memcpy(dest, fw_data + target_size, extra_fw_size);
> > + }
> > +}
> > +
> > +int vpu_load_firmware(struct platform_device *pdev)
> > +{
> > + struct mtk_vpu *vpu = platform_get_drvdata(pdev);
> > + struct device *dev = &pdev->dev;
> > + struct vpu_run *run = &vpu->run;
> > + const struct firmware *vpu_fw;
> > + int ret;
> > +
> > + if (!pdev) {
> > + dev_err(dev, "VPU platform device is invalid\n");
> > + return -EINVAL;
> > + }
> > +
> > + mutex_lock(&vpu->vpu_mutex);
> > +
> > + ret = vpu_enable_clock(pdev);
> > + if (ret) {
> > + dev_err(dev, "enable clock failed %d\n", ret);
> > + goto OUT_LOAD_FW;
> > + }
> > +
> > + if (vpu_running(vpu)) {
> > + vpu_disable_clock(pdev);
> > + mutex_unlock(&vpu->vpu_mutex);
> > + dev_warn(dev, "vpu is running already\n");
> > + return 0;
> > + }
> > +
> > + run->signaled = false;
> > + dev_dbg(vpu->dev, "firmware request\n");
> > + ret = request_firmware(&vpu_fw, VPU_P_FW, dev);
> > + if (ret < 0) {
> > + dev_err(dev, "Failed to load %s, %d\n", VPU_P_FW, ret);
> > + goto OUT_LOAD_FW;
> > + }
> > + if (vpu_fw->size > VPU_P_FW_SIZE) {
> > + ret = -EFBIG;
> > + dev_err(dev, "program fw size %zu is abnormal\n", vpu_fw->size);
> > + goto OUT_LOAD_FW;
> > + }
>
> Possibly move request_firmware(), release_firmware() and the associated
> error handling into load_requested_vpu(). It can all be parameterized
> and the filename of the firmware means error reports will still be clear
> about whether the p or d firmware is faulty.
>
I'll change this.
>
> > + dev_dbg(vpu->dev, "Downloaded program fw size: %zu.\n",
> > + vpu_fw->size);
> > + /* Downloading program firmware to device*/
> > + load_requested_vpu(vpu, vpu_fw->size, vpu_fw->data,
> > + P_FW);
> > + release_firmware(vpu_fw);
> > +
> > + ret = request_firmware(&vpu_fw, VPU_D_FW, dev);
> > + if (ret < 0) {
> > + dev_err(dev, "Failed to load %s, %d\n", VPU_D_FW, ret);
> > + goto OUT_LOAD_FW;
> > + }
> > + if (vpu_fw->size > VPU_D_FW_SIZE) {
> > + ret = -EFBIG;
> > + dev_err(dev, "data fw size %zu is abnormal\n", vpu_fw->size);
> > + goto OUT_LOAD_FW;
> > + }
> > + dev_dbg(vpu->dev, "Downloaded data fw size: %zu.\n",
> > + vpu_fw->size);
> > + /* Downloading data firmware to device */
> > + load_requested_vpu(vpu, vpu_fw->size, vpu_fw->data,
> > + D_FW);
> > + release_firmware(vpu_fw);
> > + /* boot up vpu */
> > + vpu_cfg_writel(vpu, 0x1, VPU_BASE);
> > +
> > + ret = wait_event_interruptible_timeout(run->wq,
> > + run->signaled,
> > + msecs_to_jiffies(INIT_TIMEOUT_MS)
> > + );
> > + if (0 == ret) {
> > + ret = -ETIME;
> > + dev_err(dev, "wait vpu initialization timout!\n");
> > + goto OUT_LOAD_FW;
> > + } else if (-ERESTARTSYS == ret) {
> > + dev_err(dev, "wait vpu interrupted by a signal!\n");
> > + goto OUT_LOAD_FW;
> > + }
> > +
> > + ret = 0;
> > + dev_info(dev, "vpu is ready. Fw version %s\n", run->fw_ver);
> > +
> > +OUT_LOAD_FW:
> > + vpu_disable_clock(pdev);
> > + mutex_unlock(&vpu->vpu_mutex);
> > +
> > + return ret;
> > +}
> > +
> > +static void vpu_init_ipi_handler(void *data, unsigned int len, void *priv)
> > +{
> > + struct mtk_vpu *vpu = (struct mtk_vpu *)priv;
> > + struct vpu_run *run = (struct vpu_run *)data;
> > +
> > + vpu->run.signaled = run->signaled;
> > + strncpy(vpu->run.fw_ver, run->fw_ver, VPU_FW_VER_LEN);
> > + wake_up_interruptible(&vpu->run.wq);
> > +}
> > +
> > +static int vpu_debug_open(struct inode *inode, struct file *file)
> > +{
> > + file->private_data = inode->i_private;
> > + return 0;
> > +}
> > +
> > +static ssize_t vpu_debug_read(struct file *file, char __user *user_buf,
> > + size_t count, loff_t *ppos)
> > +{
> > + char buf[256];
> > + unsigned int len;
> > + unsigned int running, pc, vpu_to_host, host_to_vpu, wdt;
> > + int ret;
> > + struct device *dev = file->private_data;
> > + struct platform_device *pdev = to_platform_device(dev);
> > + struct mtk_vpu *vpu = dev_get_drvdata(dev);
> > +
> > + ret = vpu_enable_clock(pdev);
> > + if (ret) {
> > + dev_err(vpu->dev, "[VPU] enable clock failed %d\n", ret);
> > + return 0;
> > + }
> > +
> > + /* vpu register status */
> > + running = vpu_running(vpu);
> > + pc = vpu_cfg_readl(vpu, VPU_PC_REG);
> > + wdt = vpu_cfg_readl(vpu, VPU_WDT_REG);
> > + host_to_vpu = vpu_cfg_readl(vpu, HOST_TO_VPU);
> > + vpu_to_host = vpu_cfg_readl(vpu, VPU_TO_HOST);
> > + vpu_disable_clock(pdev);
> > +
> > + if (running) {
> > + len = sprintf(buf, "VPU is running\n\n"
> > + "FW Version: %s\n"
> > + "PC: 0x%x\n"
> > + "WDT: 0x%x\n"
> > + "Host to VPU: 0x%x\n"
> > + "VPU to Host: 0x%x\n",
> > + vpu->run.fw_ver, pc, wdt,
> > + host_to_vpu, vpu_to_host);
> > + } else {
> > + len = sprintf(buf, "VPU not running\n");
> > + }
> > +
> > + return simple_read_from_buffer(user_buf, count, ppos, buf, len);
> > +}
> > +
> > +static const struct file_operations vpu_debug_fops = {
> > + .open = vpu_debug_open,
> > + .read = vpu_debug_read,
> > +};
>
> These operations should be conditional on CONFIG_DEBUG_FS.
I'll change this.
>
> > +
> > +static void vpu_free_p_ext_mem(struct mtk_vpu *vpu)
> > +{
> > + struct device *dev = vpu->dev;
> > + struct iommu_domain *domain = iommu_get_domain_for_dev(dev);
> > +
> > + dma_free_coherent(dev, VPU_EXT_P_SIZE, VPU_PMEM0_VIRT(vpu),
> > + VPU_PMEM0_IOVA(vpu));
> > +
> > + if (domain)
> > + iommu_detach_device(domain, vpu->dev);
> > +}
> > +
> > +static void vpu_free_d_ext_mem(struct mtk_vpu *vpu)
> > +{
> > + struct device *dev = vpu->dev;
> > + struct iommu_domain *domain = iommu_get_domain_for_dev(dev);
> > +
> > + dma_free_coherent(dev, VPU_EXT_D_SIZE, VPU_DMEM0_VIRT(vpu),
> > + VPU_DMEM0_IOVA(vpu));
> > +
> > + if (domain)
> > + iommu_detach_device(domain, dev);
> > +}
>
> Look like this could be parameterized and combined with vpu_free_p_ext_mem.
I'll change this.
>
>
> > +
> > +static int vpu_alloc_p_ext_mem(struct mtk_vpu *vpu)
> > +{
> > + struct device *dev = vpu->dev;
> > + struct iommu_domain *domain = iommu_get_domain_for_dev(dev);
> > + phys_addr_t p_pa;
> > +
> > + VPU_PMEM0_VIRT(vpu) = dma_alloc_coherent(dev,
> > + VPU_EXT_P_SIZE,
> > + &(VPU_PMEM0_IOVA(vpu)),
> > + GFP_KERNEL);
> > + if (VPU_PMEM0_VIRT(vpu) == NULL) {
> > + dev_err(dev, "Failed to allocate the extended program memory\n");
> > + return PTR_ERR(VPU_PMEM0_VIRT(vpu));
> > + }
> > +
> > + p_pa = iommu_iova_to_phys(domain, vpu->mem.p_iova);
> > + /* Disable extend0. Enable extend1 */
> > + vpu_cfg_writel(vpu, 0x1, VPU_PMEM_EXT0_ADDR);
> > + vpu_cfg_writel(vpu, (p_pa & 0xFFFFF000), VPU_PMEM_EXT1_ADDR);
> > +
> > + dev_info(dev, "Program extend memory phy=0x%llx virt=0x%p iova=0x%llx\n",
> > + (unsigned long long)p_pa,
> > + VPU_PMEM0_VIRT(vpu),
> > + (unsigned long long)VPU_PMEM0_IOVA(vpu));
> > +
> > + return 0;
> > +}
> > +
> > +static int vpu_alloc_d_ext_mem(struct mtk_vpu *vpu)
> > +{
> > + struct device *dev = vpu->dev;
> > + struct iommu_domain *domain = iommu_get_domain_for_dev(dev);
> > + phys_addr_t d_pa;
> > +
> > + VPU_DMEM0_VIRT(vpu) = dma_alloc_coherent(dev,
> > + VPU_EXT_D_SIZE,
> > + &(VPU_DMEM0_IOVA(vpu)),
> > + GFP_KERNEL);
> > + if (VPU_DMEM0_VIRT(vpu) == NULL) {
> > + dev_err(dev, "Failed to allocate the extended data memory\n");
> > + return PTR_ERR(VPU_DMEM0_VIRT(vpu));
> > + }
> > +
> > + d_pa = iommu_iova_to_phys(domain, vpu->mem.d_iova);
> > +
> > + /* Disable extend0. Enable extend1 */
> > + vpu_cfg_writel(vpu, 0x1, VPU_DMEM_EXT0_ADDR);
> > + vpu_cfg_writel(vpu, (d_pa & 0xFFFFF000),
> > + VPU_DMEM_EXT1_ADDR);
> > +
> > + dev_info(dev, "Data extend memory phy=0x%llx virt=0x%p iova=0x%llx\n",
> > + (unsigned long long)d_pa,
> > + VPU_DMEM0_VIRT(vpu),
> > + (unsigned long long)VPU_DMEM0_IOVA(vpu));
> > +
> > + return 0;
> > +}
>
> Also looks suitable for parameterizing and combining with
> vpu_alloc_p_ext_mem .
>
I'll change this.
>
> > +
> > +static void vpu_ipi_handler(struct mtk_vpu *vpu)
> > +{
> > + struct share_obj *rcv_obj = vpu->recv_buf;
> > + struct vpu_ipi_desc *ipi_desc = vpu->ipi_desc;
> > +
> > + if (rcv_obj->id < IPI_MAX && ipi_desc[rcv_obj->id].handler) {
> > + ipi_desc[rcv_obj->id].handler(rcv_obj->share_buf,
> > + rcv_obj->len,
> > + ipi_desc[rcv_obj->id].priv);
> > + } else {
> > + dev_err(vpu->dev, "No such ipi id = %d\n", rcv_obj->id);
> > + }
> > +}
> > +
> > +static int vpu_ipi_init(struct mtk_vpu *vpu)
> > +{
> > + /* Disable VPU to host interrupt */
> > + vpu_cfg_writel(vpu, 0x0, VPU_TO_HOST);
> > +
> > + /* shared buffer initialization */
> > + vpu->recv_buf = (struct share_obj *)VPU_DTCM(vpu);
> > + vpu->send_buf = vpu->recv_buf + 1;
> > + memset(vpu->recv_buf, 0, sizeof(struct share_obj));
> > + memset(vpu->send_buf, 0, sizeof(struct share_obj));
> > + mutex_init(&vpu->vpu_mutex);
> > +
> > + return 0;
> > +}
> > +
> > +static irqreturn_t vpu_irq_handler(int irq, void *priv)
> > +{
> > + struct mtk_vpu *vpu = priv;
> > + uint32_t vpu_to_host = vpu_cfg_readl(vpu, VPU_TO_HOST);
> > +
> > + if (vpu_to_host & VPU_IPC_INT)
> > + vpu_ipi_handler(vpu);
> > + else
> > + dev_err(vpu->dev, "vpu watchdog timeout!\n");
> > +
> > + /* VPU won't send another interrupt until we set VPU_TO_HOST to 0. */
> > + vpu_cfg_writel(vpu, 0x0, VPU_TO_HOST);
>
> If we were triggered by a watchdog then how long will it be before the
> next watchdog interrupt? Will we end up spamming the logs?
Thanks for reminding me.
When watchdog was timeout, I think VPU driver should notify registered
drivers and reset vpu hardware for next run. I'll change this.
> > +
> > + return IRQ_HANDLED;
> > +}
> > +
> > +static struct dentry *vpu_debugfs;
> > +static int mtk_vpu_probe(struct platform_device *pdev)
> > +{
> > + struct mtk_vpu *vpu;
> > + struct device *dev;
> > + struct resource *res;
> > + int ret = 0;
> > +
> > + dev_dbg(&pdev->dev, "initialization\n");
> > +
> > + dev = &pdev->dev;
> > + vpu = devm_kzalloc(dev, sizeof(*vpu), GFP_KERNEL);
> > + if (!vpu)
> > + return -ENOMEM;
> > +
> > + vpu->dev = &pdev->dev;
> > + res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "sram");
> > + vpu->reg.sram = devm_ioremap_resource(dev, res);
> > + if (IS_ERR(vpu->reg.sram)) {
> > + dev_err(dev, "devm_ioremap_resource vpu sram failed.\n");
> > + return PTR_ERR(vpu->reg.sram);
> > + }
> > +
> > + res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "cfg_reg");
> > + vpu->reg.cfg = devm_ioremap_resource(dev, res);
> > + if (IS_ERR(vpu->reg.cfg)) {
> > + dev_err(dev, "devm_ioremap_resource vpu cfg failed.\n");
> > + return PTR_ERR(vpu->reg.cfg);
> > + }
> > +
> > + /* Get VPU clock */
> > + vpu->clk = devm_clk_get(dev, "main");
> > + if (vpu->clk == NULL) {
> > + dev_err(dev, "get vpu clock fail\n");
> > + return -EINVAL;
> > + }
> > +
> > + platform_set_drvdata(pdev, vpu);
> > +
> > + ret = vpu_enable_clock(pdev);
> > + if (ret) {
> > + ret = -EINVAL;
> > + return ret;
>
> Why not just "return ret" (or "return -EINVAL")?
I'll change this.
>
> > + }
> > +
> > + dev_dbg(dev, "vpu ipi init\n");
> > + ret = vpu_ipi_init(vpu);
> > + if (ret) {
> > + dev_err(dev, "Failed to init ipi\n");
> > + goto disable_vpu_clk;
> > + }
> > +
> > + platform_set_drvdata(pdev, vpu);
> > +
> > + /* register vpu initialization IPI */
> > + ret = vpu_ipi_register(pdev, IPI_VPU_INIT, vpu_init_ipi_handler,
> > + "vpu_init", vpu);
> > + if (ret) {
> > + dev_err(dev, "Failed to register IPI_VPU_INIT\n");
> > + goto vpu_mutex_destroy;
> > + }
> > +
> > + vpu_debugfs = debugfs_create_file("mtk_vpu", S_IRUGO, NULL, (void *)dev,
> > + &vpu_debug_fops);
> > + if (!vpu_debugfs) {
> > + ret = -ENOMEM;
> > + goto cleanup_ipi;
> > + }
> > +
> > + /* Set PTCM to 96K and DTCM to 32K */
> > + vpu_cfg_writel(vpu, 0x2, VPU_TCM_CFG);
> > +
> > + ret = vpu_alloc_p_ext_mem(vpu);
> > + if (ret) {
> > + dev_err(dev, "Allocate PM failed\n");
> > + goto remove_debugfs;
> > + }
> > +
> > + ret = vpu_alloc_d_ext_mem(vpu);
> > + if (ret) {
> > + dev_err(dev, "Allocate DM failed\n");
> > + goto free_p_mem;
> > + }
> > +
> > + init_waitqueue_head(&vpu->run.wq);
> > +
> > + res = platform_get_resource(pdev, IORESOURCE_IRQ, 0);
> > + if (res == NULL) {
> > + dev_err(dev, "get IRQ resource failed.\n");
> > + ret = -ENXIO;
> > + goto free_d_mem;
> > + }
> > + vpu->reg.irq = platform_get_irq(pdev, 0);
> > + ret = devm_request_irq(dev, vpu->reg.irq, vpu_irq_handler, 0,
> > + pdev->name, vpu);
> > + if (ret) {
> > + dev_err(dev, "failed to request irq\n");
> > + goto free_d_mem;
> > + }
> > +
> > + vpu_disable_clock(pdev);
> > + dev_dbg(dev, "initialization completed\n");
> > +
> > + return 0;
> > +
> > +free_d_mem:
> > + vpu_free_d_ext_mem(vpu);
> > +free_p_mem:
> > + vpu_free_p_ext_mem(vpu);
> > +remove_debugfs:
> > + debugfs_remove(vpu_debugfs);
> > +cleanup_ipi:
> > + memset(vpu->ipi_desc, 0, sizeof(struct vpu_ipi_desc)*IPI_MAX);
> > +vpu_mutex_destroy:
> > + mutex_destroy(&vpu->vpu_mutex);
> > +disable_vpu_clk:
> > + vpu_disable_clock(pdev);
> > +
> > + return ret;
> > +}
> > +
> > +static const struct of_device_id mtk_vpu_match[] = {
> > + {
> > + .compatible = "mediatek,mt8173-vpu",
> > + },
> > + {},
> > +};
> > +MODULE_DEVICE_TABLE(of, mtk_vpu_match);
> > +
> > +static int mtk_vpu_remove(struct platform_device *pdev)
> > +{
> > + struct mtk_vpu *vpu = platform_get_drvdata(pdev);
> > +
> > + vpu_free_p_ext_mem(vpu);
> > + vpu_free_d_ext_mem(vpu);
>
> This looks like it leaks cpu_debugfs and the vpu_mutex.
Yes, I'll change this. Thanks.
>
> > +
> > + return 0;
> > +}
> > +
> > +static struct platform_driver mtk_vpu_driver = {
> > + .probe = mtk_vpu_probe,
> > + .remove = mtk_vpu_remove,
> > + .driver = {
> > + .name = MTK_VPU_DRV_NAME,
> > + .owner = THIS_MODULE,
> > + .of_match_table = mtk_vpu_match,
> > + },
> > +};
> > +
> > +module_platform_driver(mtk_vpu_driver);
> > +
> > +MODULE_LICENSE("GPL v2");
> > +MODULE_DESCRIPTION("Mediatek Video Prosessor Unit driver");
> > diff --git a/drivers/media/platform/mtk-vpu/mtk_vpu_core.h b/drivers/media/platform/mtk-vpu/mtk_vpu_core.h
> > new file mode 100644
> > index 0000000..20cf2a0
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vpu/mtk_vpu_core.h
> > @@ -0,0 +1,161 @@
> > +/*
> > +* Copyright (c) 2015 MediaTek Inc.
> > +* Author: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
> > +*
> > +* This program is free software; you can redistribute it and/or modify
> > +* it under the terms of the GNU General Public License version 2 as
> > +* published by the Free Software Foundation.
> > +*
> > +* This program is distributed in the hope that it will be useful,
> > +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> > +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> > +* GNU General Public License for more details.
> > +*/
> > +
> > +#ifndef _MTK_VPU_CORE_H
> > +#define _MTK_VPU_CORE_H
> > +
> > +#include <linux/platform_device.h>
> > +
> > +/**
> > + * VPU (video processor unit) is a tiny processor controlling video hardware
> > + * related to video codec, scaling and color format converting.
> > + * VPU interfaces with other blocks by share memory and interrupt.
> > + **/
> > +
> > +typedef void (*ipi_handler_t) (void *data,
> > + unsigned int len,
> > + void *priv);
> > +
> > +/**
> > + * enum ipi_id - the id of inter-processor interrupt
> > + *
> > + * @IPI_VPU_INIT: The interrupt from vpu is to notfiy kernel
> > + VPU initialization completed.
> > + * @IPI_VENC_H264: The interrupt from vpu is to notify kernel to
> > + handle H264 video encoder job, and vice versa.
> > + * @IPI_VENC_VP8: The interrupt fro vpu is to notify kernel to
> > + handle VP8 video encoder job,, and vice versa.
> > + * @IPI_VENC_CAPABILITY: The interrupt from vpu is to
> > + get venc hardware capability.
> > + * @IPI_MAX: The maximum IPI number
> > + */
> > +enum ipi_id {
> > + IPI_VPU_INIT = 0,
> > + IPI_VENC_H264,
> > + IPI_VENC_VP8,
> > + IPI_VENC_CAPABILITY,
> > + IPI_MAX,
> > +};
> > +
> > +/**
> > + * vpu_disable_clock - Disable VPU clock
> > + *
> > + * @pdev: VPU platform device
> > + *
> > + *
> > + * Return: Return 0 if the clock is disabled successfully,
> > + * otherwise it is failed.
> > + *
> > + **/
> > +void vpu_disable_clock(struct platform_device *pdev);
> > +
> > +/**
> > + * vpu_enable_clock - Enable VPU clock
> > + *
> > + * @pdev: VPU platform device
> > + *
> > + * Return: Return 0 if the clock is enabled successfully,
> > + * otherwise it is failed.
> > + *
> > + **/
> > +int vpu_enable_clock(struct platform_device *pdev);
> > +
> > +/**
> > + * vpu_ipi_register - register an ipi function
> > + *
> > + * @pdev: VPU platform device
> > + * @id: IPI ID
> > + * @handler: IPI handler
> > + * @name: IPI name
> > + * @priv: private data for IPI handler
> > + *
> > + * Register an ipi function to receive ipi interrupt from VPU.
> > + *
> > + * Return: Return 0 if ipi registers successfully, otherwise it is failed.
> > + */
> > +int vpu_ipi_register(struct platform_device *pdev, enum ipi_id id,
> > + ipi_handler_t handler, const char *name, void *priv);
> > +
> > +/**
> > + * vpu_ipi_send - send data from AP to vpu.
> > + *
> > + * @pdev: VPU platform device
> > + * @id: IPI ID
> > + * @buf: the data buffer
> > + * @len: the data buffer length
> > + * @wait: wait for the last ipi completed.
> > + *
> > + * This function is thread-safe. When this function returns,
> > + * VPU has received the data and starts the processing.
> > + * When the processing completes, IPI handler registered
> > + * by vpu_ipi_register will be called in interrupt context.
> > + *
> > + * Return: Return 0 if sending data successfully, otherwise it is failed.
> > + **/
> > +int vpu_ipi_send(struct platform_device *pdev,
> > + enum ipi_id id, void *buf,
> > + unsigned int len,
> > + unsigned int wait);
> > +
> > +/**
> > + * vpu_get_plat_device - get VPU's platform device
> > + *
> > + * @pdev: the platform device of the module requesting VPU platform
> > + * device for using VPU API.
> > + *
> > + * Return: Return NULL if it is failed.
> > + * otherwise it is VPU's platform device
> > + **/
> > +struct platform_device *vpu_get_plat_device(struct platform_device *pdev);
> > +
> > +/**
> > + * vpu_load_firmware - download VPU firmware and boot it
> > + *
> > + * @pdev: VPU platform device
> > + *
> > + * Return: Return 0 if downloading firmware successfully,
> > + * otherwise it is failed
> > + **/
> > +int vpu_load_firmware(struct platform_device *pdev);
> > +
> > +/**
> > + * vpu_mapping_dm_addr - Mapping DTCM/DMEM to kernel virtual address
> > + *
> > + * @pdev: VPU platform device
> > + * @dmem_addr: VPU's data memory address
> > + *
> > + * Mapping the VPU's DTCM (Data Tightly-Coupled Memory) /
> > + * DMEM (Data Extended Memory) memory address to
> > + * kernel virtual address.
> > + *
> > + * Return: Return ERR_PTR(-EINVAL) if mapping failed,
> > + * otherwise the mapped kernel virtual address
> > + **/
> > +void *vpu_mapping_dm_addr(struct platform_device *pdev,
> > + void *dtcm_dmem_addr);
> > +
> > +/**
> > + * vpu_mapping_iommu_dm_addr - Mapping to iommu address
> > + *
> > + * @pdev: VPU platform device
> > + * @dmem_addr: VPU's extended data memory address
> > + *
> > + * Mapping the VPU's extended data address to iommu address
> > + *
> > + * Return: Return ERR_PTR(-EINVAL) if mapping failed,
> > + * otherwise the mapped iommu address
> > + **/
> > +dma_addr_t *vpu_mapping_iommu_dm_addr(struct platform_device *pdev,
> > + void *dmem_addr);
> > +#endif /* _MTK_VPU_CORE_H */
> > diff --git a/drivers/media/platform/mtk-vpu/vp8_enc/venc_vp8_vpu.h b/drivers/media/platform/mtk-vpu/vp8_enc/venc_vp8_vpu.h
> > new file mode 100644
> > index 0000000..4e09eec
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vpu/vp8_enc/venc_vp8_vpu.h
>
> Like the H.264 header. Why is this file included in this patch? It is
> not included by anything in the patch and defines symbols that do not
> exist yet.
This should be moved to vp8 patch. Thanks.
>
>
> Daniel.
^ permalink raw reply [flat|nested] 84+ messages in thread
* Re: [RESEND RFC/PATCH 3/8] media: platform: mtk-vpu: Support Mediatek VPU
2015-11-27 12:10 ` andrew-ct chen
(?)
@ 2015-11-27 12:21 ` Daniel Thompson
-1 siblings, 0 replies; 84+ messages in thread
From: Daniel Thompson @ 2015-11-27 12:21 UTC (permalink / raw)
To: andrew-ct chen
Cc: Tiffany Lin, Rob Herring, Pawel Moll, Mark Rutland, Ian Campbell,
Kumar Gala, Catalin Marinas, Will Deacon, Mauro Carvalho Chehab,
Matthias Brugger, Daniel Kurtz, Sascha Hauer, Hongzhou Yang,
Hans Verkuil, Laurent Pinchart, Sakari Ailus, Geert Uytterhoeven,
Mikhail Ulyanov, Fabien Dessenne, Arnd Bergmann,
Darren Etheridge, Peter Griffin, Benoit Parrot, Eddie Huang,
Yingjoe Chen, James Liao, Daniel Hsiao, devicetree, linux-kernel,
linux-arm-kernel, linux-media, linux-mediatek
On 27/11/15 12:10, andrew-ct chen wrote:
>>> +
>>> > >+ memcpy((void *)send_obj->share_buf, buf, len);
>>> > >+ send_obj->len = len;
>>> > >+ send_obj->id = id;
>>> > >+ vpu_cfg_writel(vpu, 0x1, HOST_TO_VPU);
>>> > >+
>>> > >+ /* Wait until VPU receives the command */
>>> > >+ timeout = jiffies + msecs_to_jiffies(IPI_TIMEOUT_MS);
>>> > >+ do {
>>> > >+ if (time_after(jiffies, timeout)) {
>>> > >+ dev_err(vpu->dev, "vpu_ipi_send: IPI timeout!\n");
>>> > >+ return -EIO;
>>> > >+ }
>>> > >+ } while (vpu_cfg_readl(vpu, HOST_TO_VPU));
>> >
>> >Do we need to busy wait every time we communicate with the co-processor?
>> >Couldn't we put this wait*before* we write to HOST_TO_VPU above.
>> >
>> >That way we only spin when there is a need to.
>> >
> Since the hardware VPU only allows that one client sends the command to
> it each time.
> We need the wait to make sure VPU accepted the command and cleared the
> interrupt and then the next command would be served.
I understand that the VPU can only have on message outstanding at once.
I just wonder why we busy wait *after* sending the first command rather
than *before* sending the second one.
Streamed decode/encode typically ends up being rate controlled by
capture or display meaning that in these cases we don't need to busy
wait at all (because by the time we send the next frame the VPU has
already accepted the previous message).
Daniel.
^ permalink raw reply [flat|nested] 84+ messages in thread
* Re: [RESEND RFC/PATCH 3/8] media: platform: mtk-vpu: Support Mediatek VPU
@ 2015-11-27 12:21 ` Daniel Thompson
0 siblings, 0 replies; 84+ messages in thread
From: Daniel Thompson @ 2015-11-27 12:21 UTC (permalink / raw)
To: andrew-ct chen
Cc: Mark Rutland, James Liao, Catalin Marinas, Will Deacon,
Daniel Kurtz, Darren Etheridge, Laurent Pinchart, Eddie Huang,
Pawel Moll, Hongzhou Yang, Mauro Carvalho Chehab,
Fabien Dessenne, Peter Griffin, Geert Uytterhoeven,
Mikhail Ulyanov, Hans Verkuil, linux-media, devicetree,
Arnd Bergmann, Ian Campbell, Sascha Hauer, Benoit Parrot, Rob
On 27/11/15 12:10, andrew-ct chen wrote:
>>> +
>>> > >+ memcpy((void *)send_obj->share_buf, buf, len);
>>> > >+ send_obj->len = len;
>>> > >+ send_obj->id = id;
>>> > >+ vpu_cfg_writel(vpu, 0x1, HOST_TO_VPU);
>>> > >+
>>> > >+ /* Wait until VPU receives the command */
>>> > >+ timeout = jiffies + msecs_to_jiffies(IPI_TIMEOUT_MS);
>>> > >+ do {
>>> > >+ if (time_after(jiffies, timeout)) {
>>> > >+ dev_err(vpu->dev, "vpu_ipi_send: IPI timeout!\n");
>>> > >+ return -EIO;
>>> > >+ }
>>> > >+ } while (vpu_cfg_readl(vpu, HOST_TO_VPU));
>> >
>> >Do we need to busy wait every time we communicate with the co-processor?
>> >Couldn't we put this wait*before* we write to HOST_TO_VPU above.
>> >
>> >That way we only spin when there is a need to.
>> >
> Since the hardware VPU only allows that one client sends the command to
> it each time.
> We need the wait to make sure VPU accepted the command and cleared the
> interrupt and then the next command would be served.
I understand that the VPU can only have on message outstanding at once.
I just wonder why we busy wait *after* sending the first command rather
than *before* sending the second one.
Streamed decode/encode typically ends up being rate controlled by
capture or display meaning that in these cases we don't need to busy
wait at all (because by the time we send the next frame the VPU has
already accepted the previous message).
Daniel.
^ permalink raw reply [flat|nested] 84+ messages in thread
* [RESEND RFC/PATCH 3/8] media: platform: mtk-vpu: Support Mediatek VPU
@ 2015-11-27 12:21 ` Daniel Thompson
0 siblings, 0 replies; 84+ messages in thread
From: Daniel Thompson @ 2015-11-27 12:21 UTC (permalink / raw)
To: linux-arm-kernel
On 27/11/15 12:10, andrew-ct chen wrote:
>>> +
>>> > >+ memcpy((void *)send_obj->share_buf, buf, len);
>>> > >+ send_obj->len = len;
>>> > >+ send_obj->id = id;
>>> > >+ vpu_cfg_writel(vpu, 0x1, HOST_TO_VPU);
>>> > >+
>>> > >+ /* Wait until VPU receives the command */
>>> > >+ timeout = jiffies + msecs_to_jiffies(IPI_TIMEOUT_MS);
>>> > >+ do {
>>> > >+ if (time_after(jiffies, timeout)) {
>>> > >+ dev_err(vpu->dev, "vpu_ipi_send: IPI timeout!\n");
>>> > >+ return -EIO;
>>> > >+ }
>>> > >+ } while (vpu_cfg_readl(vpu, HOST_TO_VPU));
>> >
>> >Do we need to busy wait every time we communicate with the co-processor?
>> >Couldn't we put this wait*before* we write to HOST_TO_VPU above.
>> >
>> >That way we only spin when there is a need to.
>> >
> Since the hardware VPU only allows that one client sends the command to
> it each time.
> We need the wait to make sure VPU accepted the command and cleared the
> interrupt and then the next command would be served.
I understand that the VPU can only have on message outstanding at once.
I just wonder why we busy wait *after* sending the first command rather
than *before* sending the second one.
Streamed decode/encode typically ends up being rate controlled by
capture or display meaning that in these cases we don't need to busy
wait at all (because by the time we send the next frame the VPU has
already accepted the previous message).
Daniel.
^ permalink raw reply [flat|nested] 84+ messages in thread
* Re: [RESEND RFC/PATCH 6/8] media: platform: mtk-vcodec: Add Mediatek V4L2 Video Encoder Driver
@ 2015-11-27 16:34 ` Daniel Thompson
0 siblings, 0 replies; 84+ messages in thread
From: Daniel Thompson @ 2015-11-27 16:34 UTC (permalink / raw)
To: Tiffany Lin, Rob Herring, Pawel Moll, Mark Rutland, Ian Campbell,
Kumar Gala, Catalin Marinas, Will Deacon, Mauro Carvalho Chehab,
Matthias Brugger, Daniel Kurtz, Sascha Hauer, Hongzhou Yang,
Hans Verkuil, Laurent Pinchart, Sakari Ailus, Geert Uytterhoeven,
Mikhail Ulyanov, Fabien Dessenne, Arnd Bergmann,
Darren Etheridge, Peter Griffin, Benoit Parrot
Cc: Andrew-CT Chen, Eddie Huang, Yingjoe Chen, James Liao,
Daniel Hsiao, devicetree, linux-kernel, linux-arm-kernel,
linux-media, linux-mediatek
Hi Tiffany/Andrew
This review is a rather more superficial than my previous one. Mostly
I'm just commenting on some of the bits I spotted whilst trying to find
my way around the patchset.
I hope to another more detailed review for v2 (and feel free to add me
to Cc:).
On 17/11/15 12:54, Tiffany Lin wrote:
> Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
> Signed-off-by: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
There is no description of what this patch does. Its not enough to have
it on the cover letter (because that won't end up in version control).
You need something here.
> diff --git a/drivers/media/platform/mtk-vcodec/Kconfig b/drivers/media/platform/mtk-vcodec/Kconfig
> new file mode 100644
> index 0000000..1c0b935
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vcodec/Kconfig
> @@ -0,0 +1,5 @@
> +config MEDIATEK_VPU
> + bool
> + ---help---
> + This driver provides downloading firmware vpu (video processor unit)
> + and communicating with vpu.
Haven't I seen this before (in patch 3)? Why is it being added to
another Kconfig file?
> diff --git a/drivers/media/platform/mtk-vcodec/Makefile b/drivers/media/platform/mtk-vcodec/Makefile
> new file mode 100644
> index 0000000..c7f7174
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vcodec/Makefile
> @@ -0,0 +1,12 @@
> +obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += mtk_vcodec_intr.o \
> + mtk_vcodec_util.o \
> + mtk_vcodec_enc_drv.o \
> + mtk_vcodec_enc.o \
> + mtk_vcodec_enc_pm.o
> +
> +obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += common/
> +
> +ccflags-y += -I$(srctree)/drivers/media/platform/mtk-vcodec/include \
> + -I$(srctree)/drivers/media/platform/mtk-vcodec \
> + -I$(srctree)/drivers/media/platform/mtk-vpu
Seems like there's a lot of directories here. Are these files
(framework, common, vcodec, etc) so unrelated they really need to live
in separate directories?
Why not just drivers/media/platform/mediatek?
> diff --git a/drivers/media/platform/mtk-vcodec/common/Makefile b/drivers/media/platform/mtk-vcodec/common/Makefile
> new file mode 100644
> index 0000000..477ab80
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vcodec/common/Makefile
> @@ -0,0 +1,8 @@
> +obj-y += \
> + venc_drv_if.o
> +
> +ccflags-y += \
> + -I$(srctree)/include/ \
> + -I$(srctree)/drivers/media/platform/mtk-vcodec \
> + -I$(srctree)/drivers/media/platform/mtk-vcodec/include \
> + -I$(srctree)/drivers/media/platform/mtk-vpu
As above, this appears to be a directory to hold just one file.
> diff --git a/drivers/media/platform/mtk-vcodec/common/venc_drv_if.c
b/drivers/media/platform/mtk-vcodec/common/venc_drv_if.c
> new file mode 100644
> index 0000000..9b3f025
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vcodec/common/venc_drv_if.c
> @@ -0,0 +1,152 @@
> +/*
> + * Copyright (c) 2015 MediaTek Inc.
> + * Author: Daniel Hsiao <daniel.hsiao@mediatek.com>
> + * Jungchang Tsao <jungchang.tsao@mediatek.com>
> + *
> + * This program is free software; you can redistribute it and/or
> + * modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> + * GNU General Public License for more details.
> + */
> +
> +#include <linux/interrupt.h>
> +#include <linux/kernel.h>
> +#include <linux/slab.h>
> +
> +#include "mtk_vcodec_drv.h"
> +#include "mtk_vcodec_enc.h"
> +#include "mtk_vcodec_pm.h"
> +#include "mtk_vcodec_util.h"
> +#include "mtk_vpu_core.h"
> +
> +#include "venc_drv_if.h"
> +#include "venc_drv_base.h"
> +
> +
> +int venc_if_create(void *ctx, unsigned int fourcc, unsigned long
*handle)
> +{
> + struct venc_handle *h;
> + char str[10];
> +
> + mtk_vcodec_fmt2str(fourcc, str);
> +
> + h = kzalloc(sizeof(*h), GFP_KERNEL);
> + if (!h)
> + return -ENOMEM;
> +
> + h->fourcc = fourcc;
> + h->ctx = ctx;
> + mtk_vcodec_debug(h, "fmt = %s handle = %p", str, h);
> +
> + switch (fourcc) {
> + default:
> + mtk_vcodec_err(h, "invalid format %s", str);
> + goto err_out;
> + }
> +
> + *handle = (unsigned long)h;
> + return 0;
> +
> +err_out:
> + kfree(h);
> + return -EINVAL;
> +}
> +
> +int venc_if_init(unsigned long handle)
> +{
> + int ret = 0;
> + struct venc_handle *h = (struct venc_handle *)handle;
> +
> + mtk_vcodec_debug_enter(h);
> +
> + mtk_venc_lock(h->ctx);
> + mtk_vcodec_enc_clock_on();
> + vpu_enable_clock(vpu_get_plat_device(h->ctx->dev->plat_dev));
> + ret = h->enc_if->init(h->ctx, (unsigned long *)&h->drv_handle);
> + vpu_disable_clock(vpu_get_plat_device(h->ctx->dev->plat_dev));
> + mtk_vcodec_enc_clock_off();
> + mtk_venc_unlock(h->ctx);
> +
> + return ret;
> +}
To me this looks more like an obfuscation layer rather than a
abstraction layer. I don't understand why we need to hide things from
the V4L2 implementation that this code forms part of.
More importantly, if this code was included somewhere where it could be
properly integrated with the device model you might be able to use the
pm_runtime system to avoid this sort of "heroics" to manage the clocks
anyway.
> diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
> new file mode 100644
> index 0000000..22239f8
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
> @@ -0,0 +1,441 @@
> +/*
> +* Copyright (c) 2015 MediaTek Inc.
> +* Author: PC Chen <pc.chen@mediatek.com>
> +* Tiffany Lin <tiffany.lin@mediatek.com>
> +*
> +* This program is free software; you can redistribute it and/or modify
> +* it under the terms of the GNU General Public License version 2 as
> +* published by the Free Software Foundation.
> +*
> +* This program is distributed in the hope that it will be useful,
> +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> +* GNU General Public License for more details.
> +*/
> +
> +#ifndef _MTK_VCODEC_DRV_H_
> +#define _MTK_VCODEC_DRV_H_
> +
> +#include <linux/platform_device.h>
> +#include <linux/videodev2.h>
> +#include <media/v4l2-ctrls.h>
> +#include <media/v4l2-device.h>
> +#include <media/v4l2-ioctl.h>
> +#include <media/videobuf2-core.h>
> +#include <media/videobuf2-v4l2.h>
> +
> +#include "venc_drv_if.h"
> +
> +#define MTK_VCODEC_MAX_INSTANCES 32
> +#define MTK_VCODEC_MAX_FRAME_SIZE 0x800000
> +#define MTK_VIDEO_MAX_FRAME 32
> +#define MTK_MAX_CTRLS 10
> +
> +#define MTK_VCODEC_DRV_NAME "mtk_vcodec_drv"
> +#define MTK_VCODEC_ENC_NAME "mtk-vcodec-enc"
> +
> +#define MTK_VENC_IRQ_STATUS_SPS 0x1
> +#define MTK_VENC_IRQ_STATUS_PPS 0x2
> +#define MTK_VENC_IRQ_STATUS_FRM 0x4
> +#define MTK_VENC_IRQ_STATUS_DRAM 0x8
> +#define MTK_VENC_IRQ_STATUS_PAUSE 0x10
> +#define MTK_VENC_IRQ_STATUS_SWITCH 0x20
Probably better to use BIT(0) .. BIT(5).
> +#define MTK_VENC_IRQ_STATUS_OFFSET 0x05C
> +#define MTK_VENC_IRQ_ACK_OFFSET 0x060
> +
> +#define MTK_VCODEC_MAX_PLANES 3
> +
> +#define VDEC_HW_ACTIVE 0x10
> +#define VDEC_IRQ_CFG 0x11
> +#define VDEC_IRQ_CLR 0x10
> +
> +#define VDEC_IRQ_CFG_REG 0xa4
> +#define NUM_MAX_ALLOC_CTX 4
> +#define MTK_V4L2_BENCHMARK 0
> +#define USE_ENCODE_THREAD 1
> +
> +/**
> + * enum mtk_hw_reg_idx - MTK hw register base index
> + */
> +enum mtk_hw_reg_idx {
> + VDEC_SYS,
> + VDEC_MISC,
> + VDEC_LD,
> + VDEC_TOP,
> + VDEC_CM,
> + VDEC_AD,
> + VDEC_AV,
> + VDEC_PP,
> + VDEC_HWD,
> + VDEC_HWQ,
> + VDEC_HWB,
> + VDEC_HWG,
> + NUM_MAX_VDEC_REG_BASE,
> + VENC_SYS = NUM_MAX_VDEC_REG_BASE,
> + VENC_LT_SYS,
> + NUM_MAX_VCODEC_REG_BASE
> +};
> +
> +/**
> + * enum mtk_instance_type - The type of an MTK Vcodec instance.
> + */
> +enum mtk_instance_type {
> + MTK_INST_DECODER = 0,
> + MTK_INST_ENCODER = 1,
> +};
> +
> +/**
> + * enum mtk_instance_state - The state of an MTK Vcodec instance.
> + * @MTK_STATE_FREE - default state when instance create
> + * @MTK_STATE_CREATE - vdec instance is create
> + * @MTK_STATE_INIT - vdec instance is init
> + * @MTK_STATE_CONFIG - reserved for encoder
> + * @MTK_STATE_HEADER - vdec had sps/pps header parsed
> + * @MTK_STATE_RUNNING - vdec is decoding
> + * @MTK_STATE_FLUSH - vdec is flushing
> + * @MTK_STATE_RES_CHANGE - vdec detect resolution change
> + * @MTK_STATE_FINISH - ctx instance is stopped streaming
> + * @MTK_STATE_DEINIT - before release ctx instance
> + * @MTK_STATE_ERROR - vdec has something wrong
> + * @MTK_STATE_ABORT - abort work in working thread
> + */
> +enum mtk_instance_state {
> + MTK_STATE_FREE = 0,
> + MTK_STATE_CREATE = (1 << 0),
> + MTK_STATE_INIT = (1 << 1),
> + MTK_STATE_CONFIG = (1 << 2),
> + MTK_STATE_HEADER = (1 << 3),
> + MTK_STATE_RUNNING = (1 << 4),
> + MTK_STATE_FLUSH = (1 << 5),
> + MTK_STATE_RES_CHANGE = (1 << 6),
> + MTK_STATE_FINISH = (1 << 7),
> + MTK_STATE_DEINIT = (1 << 8),
> + MTK_STATE_ERROR = (1 << 9),
> + MTK_STATE_ABORT = (1 << 10),
This looks like it started as a state machine and somehow turned into
flags, resulting in a state machine with 2048 states or, to give it a
different name, a debugging nightmare.
If the start streaming operation implemented cleanup-on-error properly
then there would only be two useful states: Started and stopped. Even
the "sticky" error behavior looks unnecessary to me (meaning we don't
need to track its state).
> diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
> new file mode 100644
> index 0000000..8e1b6f0
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
> @@ -0,0 +1,1773 @@
> [...]
> +static int vb2ops_venc_start_streaming(struct vb2_queue *q, unsigned int count)
> +{
> + struct mtk_vcodec_ctx *ctx = vb2_get_drv_priv(q);
> + struct v4l2_device *v4l2_dev = &ctx->dev->v4l2_dev;
> + int ret;
> +#if MTK_V4L2_BENCHMARK
> + struct timeval begin, end;
> +
> + do_gettimeofday(&begin);
> +#endif
> +
> + if (!(vb2_start_streaming_called(&ctx->m2m_ctx->out_q_ctx.q) &
> + vb2_start_streaming_called(&ctx->m2m_ctx->cap_q_ctx.q))) {
> + mtk_v4l2_debug(1, "[%d]-> out=%d cap=%d",
> + ctx->idx,
> + vb2_start_streaming_called(&ctx->m2m_ctx->out_q_ctx.q),
> + vb2_start_streaming_called(&ctx->m2m_ctx->cap_q_ctx.q));
> + return 0;
> + }
> +
> + if ((ctx->state & (MTK_STATE_ERROR | MTK_STATE_ABORT)))
> + return -EINVAL;
This is the sort of thing I mean.
This sticky error behaviour means that every subsequent call to
vb2ops_venc_start_streaming() will fail. Note also that the user will
never try to stop streaming (which can clear the error state) because
according to the return code it got when it tried to start streaming we
never actually started.
This is what I mean about having two many states. From the user's
perspective there are only two states. There needs to be a good reason
for the driver to manage so many extra secret states internally.
> +
> + if (ctx->state == MTK_STATE_FREE) {
> + ret = venc_if_create(ctx,
> + ctx->q_data[MTK_Q_DATA_DST].fmt->fourcc,
> + &ctx->h_enc);
> +
> + if (ret != 0) {
> + ctx->state |= MTK_STATE_ERROR;
> + v4l2_err(v4l2_dev, "invalid codec type=%x\n",
> + ctx->q_data[MTK_Q_DATA_DST].fmt->fourcc);
> + v4l2_err(v4l2_dev, "venc_if_create failed=%d\n", ret);
> + return -EINVAL;
> + }
> +
> + if (ctx->q_data[MTK_Q_DATA_DST].fmt->fourcc ==
> + V4L2_PIX_FMT_H264)
> + ctx->hdr = 1;
> +
> + ctx->state |= MTK_STATE_CREATE;
> + }
> +
> + if ((ctx->state & MTK_STATE_CREATE) && !(ctx->state & MTK_STATE_INIT)) {
> + ret = venc_if_init(ctx->h_enc);
> + if (ret != 0) {
> + ctx->state |= MTK_STATE_ERROR;
> + v4l2_err(v4l2_dev, "venc_if_init failed=%d\n", ret);
> + return -EINVAL;
This error path leaves the encoder partially constructed and relies on
something else to tidy things up. It would be much better to tidy things
up from this function and
Also I don't think both venc_if_create and venc_if_init are needed. They
are only ever called one after the other and thus they only serve to
complicate the error handling code.
> diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
> new file mode 100644
> index 0000000..a8e683a
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
> @@ -0,0 +1,66 @@
> +/*
> +* Copyright (c) 2015 MediaTek Inc.
> +* Author: PC Chen <pc.chen@mediatek.com>
> +* Tiffany Lin <tiffany.lin@mediatek.com>
> +*
> +* This program is free software; you can redistribute it and/or modify
> +* it under the terms of the GNU General Public License version 2 as
> +* published by the Free Software Foundation.
> +*
> +* This program is distributed in the hope that it will be useful,
> +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> +* GNU General Public License for more details.
> +*/
> +
> +#ifndef _MTK_VCODEC_UTIL_H_
> +#define _MTK_VCODEC_UTIL_H_
> +
> +#include <linux/types.h>
> +#include <linux/dma-direction.h>
> +
> +struct mtk_vcodec_mem {
> + size_t size;
> + void *va;
> + dma_addr_t dma_addr;
> +};
> +
> +extern int mtk_v4l2_dbg_level;
> +extern bool mtk_vcodec_dbg;
> +
> +#define mtk_v4l2_debug(level, fmt, args...) \
> + do { \
> + if (mtk_v4l2_dbg_level >= level) \
> + pr_info("[MTK_V4L2] level=%d %s(),%d: " fmt "\n",\
> + level, __func__, __LINE__, ##args); \
> + } while (0)
> +
> +#define mtk_v4l2_err(fmt, args...) \
> + pr_err("[MTK_V4L2][ERROR] %s:%d: " fmt "\n", __func__, __LINE__, \
> + ##args)
Obviously the code should be structured to make use of dev_dbg/dev_err
possible.
However where this won't work do you really need special macros for
this. Assuming your error messages are well written 'git grep' and the
following should be enough:
#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
> +#define mtk_v4l2_debug_enter() mtk_v4l2_debug(5, "+\n")
> +#define mtk_v4l2_debug_leave() mtk_v4l2_debug(5, "-\n")
Remove these. If you care about function entry and exit for debugging
you should be able to use ftrace.
> +#define mtk_vcodec_debug(h, fmt, args...) \
> + do { \
> + if (mtk_vcodec_dbg) \
> + pr_info("[MTK_VCODEC][%d]: %s() " fmt "\n", \
> + ((struct mtk_vcodec_ctx *)h->ctx)->idx, \
> + __func__, ##args); \
> + } while (0)
> +
> +#define mtk_vcodec_err(h, fmt, args...) \
> + pr_err("[MTK_VCODEC][ERROR][%d]: %s() " fmt "\n", \
> + ((struct mtk_vcodec_ctx *)h->ctx)->idx, __func__, ##args)
> +
> +#define mtk_vcodec_debug_enter(h) mtk_vcodec_debug(h, "+\n")
> +#define mtk_vcodec_debug_leave(h) mtk_vcodec_debug(h, "-\n")
All above comments apply to these too.
^ permalink raw reply [flat|nested] 84+ messages in thread
* Re: [RESEND RFC/PATCH 6/8] media: platform: mtk-vcodec: Add Mediatek V4L2 Video Encoder Driver
@ 2015-11-27 16:34 ` Daniel Thompson
0 siblings, 0 replies; 84+ messages in thread
From: Daniel Thompson @ 2015-11-27 16:34 UTC (permalink / raw)
To: Tiffany Lin, Rob Herring, Pawel Moll, Mark Rutland, Ian Campbell,
Kumar Gala, Catalin Marinas, Will Deacon, Mauro Carvalho Chehab,
Matthias Brugger, Daniel Kurtz, Sascha Hauer, Hongzhou Yang,
Hans Verkuil, Laurent Pinchart, Sakari Ailus, Geert Uytterhoeven,
Mikhail Ulyanov, Fabien Dessenne, Arnd Bergmann,
Darren Etheridge, Peter Griffin
Cc: Andrew-CT Chen, Eddie Huang, Yingjoe Chen, James Liao,
Daniel Hsiao, devicetree-u79uwXL29TY76Z2rM5mHXA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA,
linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
linux-media-u79uwXL29TY76Z2rM5mHXA,
linux-mediatek-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r
Hi Tiffany/Andrew
This review is a rather more superficial than my previous one. Mostly
I'm just commenting on some of the bits I spotted whilst trying to find
my way around the patchset.
I hope to another more detailed review for v2 (and feel free to add me
to Cc:).
On 17/11/15 12:54, Tiffany Lin wrote:
> Signed-off-by: Tiffany Lin <tiffany.lin-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
> Signed-off-by: Andrew-CT Chen <andrew-ct.chen-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
There is no description of what this patch does. Its not enough to have
it on the cover letter (because that won't end up in version control).
You need something here.
> diff --git a/drivers/media/platform/mtk-vcodec/Kconfig b/drivers/media/platform/mtk-vcodec/Kconfig
> new file mode 100644
> index 0000000..1c0b935
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vcodec/Kconfig
> @@ -0,0 +1,5 @@
> +config MEDIATEK_VPU
> + bool
> + ---help---
> + This driver provides downloading firmware vpu (video processor unit)
> + and communicating with vpu.
Haven't I seen this before (in patch 3)? Why is it being added to
another Kconfig file?
> diff --git a/drivers/media/platform/mtk-vcodec/Makefile b/drivers/media/platform/mtk-vcodec/Makefile
> new file mode 100644
> index 0000000..c7f7174
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vcodec/Makefile
> @@ -0,0 +1,12 @@
> +obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += mtk_vcodec_intr.o \
> + mtk_vcodec_util.o \
> + mtk_vcodec_enc_drv.o \
> + mtk_vcodec_enc.o \
> + mtk_vcodec_enc_pm.o
> +
> +obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += common/
> +
> +ccflags-y += -I$(srctree)/drivers/media/platform/mtk-vcodec/include \
> + -I$(srctree)/drivers/media/platform/mtk-vcodec \
> + -I$(srctree)/drivers/media/platform/mtk-vpu
Seems like there's a lot of directories here. Are these files
(framework, common, vcodec, etc) so unrelated they really need to live
in separate directories?
Why not just drivers/media/platform/mediatek?
> diff --git a/drivers/media/platform/mtk-vcodec/common/Makefile b/drivers/media/platform/mtk-vcodec/common/Makefile
> new file mode 100644
> index 0000000..477ab80
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vcodec/common/Makefile
> @@ -0,0 +1,8 @@
> +obj-y += \
> + venc_drv_if.o
> +
> +ccflags-y += \
> + -I$(srctree)/include/ \
> + -I$(srctree)/drivers/media/platform/mtk-vcodec \
> + -I$(srctree)/drivers/media/platform/mtk-vcodec/include \
> + -I$(srctree)/drivers/media/platform/mtk-vpu
As above, this appears to be a directory to hold just one file.
> diff --git a/drivers/media/platform/mtk-vcodec/common/venc_drv_if.c
b/drivers/media/platform/mtk-vcodec/common/venc_drv_if.c
> new file mode 100644
> index 0000000..9b3f025
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vcodec/common/venc_drv_if.c
> @@ -0,0 +1,152 @@
> +/*
> + * Copyright (c) 2015 MediaTek Inc.
> + * Author: Daniel Hsiao <daniel.hsiao-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
> + * Jungchang Tsao <jungchang.tsao-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
> + *
> + * This program is free software; you can redistribute it and/or
> + * modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> + * GNU General Public License for more details.
> + */
> +
> +#include <linux/interrupt.h>
> +#include <linux/kernel.h>
> +#include <linux/slab.h>
> +
> +#include "mtk_vcodec_drv.h"
> +#include "mtk_vcodec_enc.h"
> +#include "mtk_vcodec_pm.h"
> +#include "mtk_vcodec_util.h"
> +#include "mtk_vpu_core.h"
> +
> +#include "venc_drv_if.h"
> +#include "venc_drv_base.h"
> +
> +
> +int venc_if_create(void *ctx, unsigned int fourcc, unsigned long
*handle)
> +{
> + struct venc_handle *h;
> + char str[10];
> +
> + mtk_vcodec_fmt2str(fourcc, str);
> +
> + h = kzalloc(sizeof(*h), GFP_KERNEL);
> + if (!h)
> + return -ENOMEM;
> +
> + h->fourcc = fourcc;
> + h->ctx = ctx;
> + mtk_vcodec_debug(h, "fmt = %s handle = %p", str, h);
> +
> + switch (fourcc) {
> + default:
> + mtk_vcodec_err(h, "invalid format %s", str);
> + goto err_out;
> + }
> +
> + *handle = (unsigned long)h;
> + return 0;
> +
> +err_out:
> + kfree(h);
> + return -EINVAL;
> +}
> +
> +int venc_if_init(unsigned long handle)
> +{
> + int ret = 0;
> + struct venc_handle *h = (struct venc_handle *)handle;
> +
> + mtk_vcodec_debug_enter(h);
> +
> + mtk_venc_lock(h->ctx);
> + mtk_vcodec_enc_clock_on();
> + vpu_enable_clock(vpu_get_plat_device(h->ctx->dev->plat_dev));
> + ret = h->enc_if->init(h->ctx, (unsigned long *)&h->drv_handle);
> + vpu_disable_clock(vpu_get_plat_device(h->ctx->dev->plat_dev));
> + mtk_vcodec_enc_clock_off();
> + mtk_venc_unlock(h->ctx);
> +
> + return ret;
> +}
To me this looks more like an obfuscation layer rather than a
abstraction layer. I don't understand why we need to hide things from
the V4L2 implementation that this code forms part of.
More importantly, if this code was included somewhere where it could be
properly integrated with the device model you might be able to use the
pm_runtime system to avoid this sort of "heroics" to manage the clocks
anyway.
> diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
> new file mode 100644
> index 0000000..22239f8
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
> @@ -0,0 +1,441 @@
> +/*
> +* Copyright (c) 2015 MediaTek Inc.
> +* Author: PC Chen <pc.chen-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
> +* Tiffany Lin <tiffany.lin-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
> +*
> +* This program is free software; you can redistribute it and/or modify
> +* it under the terms of the GNU General Public License version 2 as
> +* published by the Free Software Foundation.
> +*
> +* This program is distributed in the hope that it will be useful,
> +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> +* GNU General Public License for more details.
> +*/
> +
> +#ifndef _MTK_VCODEC_DRV_H_
> +#define _MTK_VCODEC_DRV_H_
> +
> +#include <linux/platform_device.h>
> +#include <linux/videodev2.h>
> +#include <media/v4l2-ctrls.h>
> +#include <media/v4l2-device.h>
> +#include <media/v4l2-ioctl.h>
> +#include <media/videobuf2-core.h>
> +#include <media/videobuf2-v4l2.h>
> +
> +#include "venc_drv_if.h"
> +
> +#define MTK_VCODEC_MAX_INSTANCES 32
> +#define MTK_VCODEC_MAX_FRAME_SIZE 0x800000
> +#define MTK_VIDEO_MAX_FRAME 32
> +#define MTK_MAX_CTRLS 10
> +
> +#define MTK_VCODEC_DRV_NAME "mtk_vcodec_drv"
> +#define MTK_VCODEC_ENC_NAME "mtk-vcodec-enc"
> +
> +#define MTK_VENC_IRQ_STATUS_SPS 0x1
> +#define MTK_VENC_IRQ_STATUS_PPS 0x2
> +#define MTK_VENC_IRQ_STATUS_FRM 0x4
> +#define MTK_VENC_IRQ_STATUS_DRAM 0x8
> +#define MTK_VENC_IRQ_STATUS_PAUSE 0x10
> +#define MTK_VENC_IRQ_STATUS_SWITCH 0x20
Probably better to use BIT(0) .. BIT(5).
> +#define MTK_VENC_IRQ_STATUS_OFFSET 0x05C
> +#define MTK_VENC_IRQ_ACK_OFFSET 0x060
> +
> +#define MTK_VCODEC_MAX_PLANES 3
> +
> +#define VDEC_HW_ACTIVE 0x10
> +#define VDEC_IRQ_CFG 0x11
> +#define VDEC_IRQ_CLR 0x10
> +
> +#define VDEC_IRQ_CFG_REG 0xa4
> +#define NUM_MAX_ALLOC_CTX 4
> +#define MTK_V4L2_BENCHMARK 0
> +#define USE_ENCODE_THREAD 1
> +
> +/**
> + * enum mtk_hw_reg_idx - MTK hw register base index
> + */
> +enum mtk_hw_reg_idx {
> + VDEC_SYS,
> + VDEC_MISC,
> + VDEC_LD,
> + VDEC_TOP,
> + VDEC_CM,
> + VDEC_AD,
> + VDEC_AV,
> + VDEC_PP,
> + VDEC_HWD,
> + VDEC_HWQ,
> + VDEC_HWB,
> + VDEC_HWG,
> + NUM_MAX_VDEC_REG_BASE,
> + VENC_SYS = NUM_MAX_VDEC_REG_BASE,
> + VENC_LT_SYS,
> + NUM_MAX_VCODEC_REG_BASE
> +};
> +
> +/**
> + * enum mtk_instance_type - The type of an MTK Vcodec instance.
> + */
> +enum mtk_instance_type {
> + MTK_INST_DECODER = 0,
> + MTK_INST_ENCODER = 1,
> +};
> +
> +/**
> + * enum mtk_instance_state - The state of an MTK Vcodec instance.
> + * @MTK_STATE_FREE - default state when instance create
> + * @MTK_STATE_CREATE - vdec instance is create
> + * @MTK_STATE_INIT - vdec instance is init
> + * @MTK_STATE_CONFIG - reserved for encoder
> + * @MTK_STATE_HEADER - vdec had sps/pps header parsed
> + * @MTK_STATE_RUNNING - vdec is decoding
> + * @MTK_STATE_FLUSH - vdec is flushing
> + * @MTK_STATE_RES_CHANGE - vdec detect resolution change
> + * @MTK_STATE_FINISH - ctx instance is stopped streaming
> + * @MTK_STATE_DEINIT - before release ctx instance
> + * @MTK_STATE_ERROR - vdec has something wrong
> + * @MTK_STATE_ABORT - abort work in working thread
> + */
> +enum mtk_instance_state {
> + MTK_STATE_FREE = 0,
> + MTK_STATE_CREATE = (1 << 0),
> + MTK_STATE_INIT = (1 << 1),
> + MTK_STATE_CONFIG = (1 << 2),
> + MTK_STATE_HEADER = (1 << 3),
> + MTK_STATE_RUNNING = (1 << 4),
> + MTK_STATE_FLUSH = (1 << 5),
> + MTK_STATE_RES_CHANGE = (1 << 6),
> + MTK_STATE_FINISH = (1 << 7),
> + MTK_STATE_DEINIT = (1 << 8),
> + MTK_STATE_ERROR = (1 << 9),
> + MTK_STATE_ABORT = (1 << 10),
This looks like it started as a state machine and somehow turned into
flags, resulting in a state machine with 2048 states or, to give it a
different name, a debugging nightmare.
If the start streaming operation implemented cleanup-on-error properly
then there would only be two useful states: Started and stopped. Even
the "sticky" error behavior looks unnecessary to me (meaning we don't
need to track its state).
> diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
> new file mode 100644
> index 0000000..8e1b6f0
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
> @@ -0,0 +1,1773 @@
> [...]
> +static int vb2ops_venc_start_streaming(struct vb2_queue *q, unsigned int count)
> +{
> + struct mtk_vcodec_ctx *ctx = vb2_get_drv_priv(q);
> + struct v4l2_device *v4l2_dev = &ctx->dev->v4l2_dev;
> + int ret;
> +#if MTK_V4L2_BENCHMARK
> + struct timeval begin, end;
> +
> + do_gettimeofday(&begin);
> +#endif
> +
> + if (!(vb2_start_streaming_called(&ctx->m2m_ctx->out_q_ctx.q) &
> + vb2_start_streaming_called(&ctx->m2m_ctx->cap_q_ctx.q))) {
> + mtk_v4l2_debug(1, "[%d]-> out=%d cap=%d",
> + ctx->idx,
> + vb2_start_streaming_called(&ctx->m2m_ctx->out_q_ctx.q),
> + vb2_start_streaming_called(&ctx->m2m_ctx->cap_q_ctx.q));
> + return 0;
> + }
> +
> + if ((ctx->state & (MTK_STATE_ERROR | MTK_STATE_ABORT)))
> + return -EINVAL;
This is the sort of thing I mean.
This sticky error behaviour means that every subsequent call to
vb2ops_venc_start_streaming() will fail. Note also that the user will
never try to stop streaming (which can clear the error state) because
according to the return code it got when it tried to start streaming we
never actually started.
This is what I mean about having two many states. From the user's
perspective there are only two states. There needs to be a good reason
for the driver to manage so many extra secret states internally.
> +
> + if (ctx->state == MTK_STATE_FREE) {
> + ret = venc_if_create(ctx,
> + ctx->q_data[MTK_Q_DATA_DST].fmt->fourcc,
> + &ctx->h_enc);
> +
> + if (ret != 0) {
> + ctx->state |= MTK_STATE_ERROR;
> + v4l2_err(v4l2_dev, "invalid codec type=%x\n",
> + ctx->q_data[MTK_Q_DATA_DST].fmt->fourcc);
> + v4l2_err(v4l2_dev, "venc_if_create failed=%d\n", ret);
> + return -EINVAL;
> + }
> +
> + if (ctx->q_data[MTK_Q_DATA_DST].fmt->fourcc ==
> + V4L2_PIX_FMT_H264)
> + ctx->hdr = 1;
> +
> + ctx->state |= MTK_STATE_CREATE;
> + }
> +
> + if ((ctx->state & MTK_STATE_CREATE) && !(ctx->state & MTK_STATE_INIT)) {
> + ret = venc_if_init(ctx->h_enc);
> + if (ret != 0) {
> + ctx->state |= MTK_STATE_ERROR;
> + v4l2_err(v4l2_dev, "venc_if_init failed=%d\n", ret);
> + return -EINVAL;
This error path leaves the encoder partially constructed and relies on
something else to tidy things up. It would be much better to tidy things
up from this function and
Also I don't think both venc_if_create and venc_if_init are needed. They
are only ever called one after the other and thus they only serve to
complicate the error handling code.
> diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
> new file mode 100644
> index 0000000..a8e683a
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
> @@ -0,0 +1,66 @@
> +/*
> +* Copyright (c) 2015 MediaTek Inc.
> +* Author: PC Chen <pc.chen-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
> +* Tiffany Lin <tiffany.lin-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
> +*
> +* This program is free software; you can redistribute it and/or modify
> +* it under the terms of the GNU General Public License version 2 as
> +* published by the Free Software Foundation.
> +*
> +* This program is distributed in the hope that it will be useful,
> +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> +* GNU General Public License for more details.
> +*/
> +
> +#ifndef _MTK_VCODEC_UTIL_H_
> +#define _MTK_VCODEC_UTIL_H_
> +
> +#include <linux/types.h>
> +#include <linux/dma-direction.h>
> +
> +struct mtk_vcodec_mem {
> + size_t size;
> + void *va;
> + dma_addr_t dma_addr;
> +};
> +
> +extern int mtk_v4l2_dbg_level;
> +extern bool mtk_vcodec_dbg;
> +
> +#define mtk_v4l2_debug(level, fmt, args...) \
> + do { \
> + if (mtk_v4l2_dbg_level >= level) \
> + pr_info("[MTK_V4L2] level=%d %s(),%d: " fmt "\n",\
> + level, __func__, __LINE__, ##args); \
> + } while (0)
> +
> +#define mtk_v4l2_err(fmt, args...) \
> + pr_err("[MTK_V4L2][ERROR] %s:%d: " fmt "\n", __func__, __LINE__, \
> + ##args)
Obviously the code should be structured to make use of dev_dbg/dev_err
possible.
However where this won't work do you really need special macros for
this. Assuming your error messages are well written 'git grep' and the
following should be enough:
#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
> +#define mtk_v4l2_debug_enter() mtk_v4l2_debug(5, "+\n")
> +#define mtk_v4l2_debug_leave() mtk_v4l2_debug(5, "-\n")
Remove these. If you care about function entry and exit for debugging
you should be able to use ftrace.
> +#define mtk_vcodec_debug(h, fmt, args...) \
> + do { \
> + if (mtk_vcodec_dbg) \
> + pr_info("[MTK_VCODEC][%d]: %s() " fmt "\n", \
> + ((struct mtk_vcodec_ctx *)h->ctx)->idx, \
> + __func__, ##args); \
> + } while (0)
> +
> +#define mtk_vcodec_err(h, fmt, args...) \
> + pr_err("[MTK_VCODEC][ERROR][%d]: %s() " fmt "\n", \
> + ((struct mtk_vcodec_ctx *)h->ctx)->idx, __func__, ##args)
> +
> +#define mtk_vcodec_debug_enter(h) mtk_vcodec_debug(h, "+\n")
> +#define mtk_vcodec_debug_leave(h) mtk_vcodec_debug(h, "-\n")
All above comments apply to these too.
--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 84+ messages in thread
* [RESEND RFC/PATCH 6/8] media: platform: mtk-vcodec: Add Mediatek V4L2 Video Encoder Driver
@ 2015-11-27 16:34 ` Daniel Thompson
0 siblings, 0 replies; 84+ messages in thread
From: Daniel Thompson @ 2015-11-27 16:34 UTC (permalink / raw)
To: linux-arm-kernel
Hi Tiffany/Andrew
This review is a rather more superficial than my previous one. Mostly
I'm just commenting on some of the bits I spotted whilst trying to find
my way around the patchset.
I hope to another more detailed review for v2 (and feel free to add me
to Cc:).
On 17/11/15 12:54, Tiffany Lin wrote:
> Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
> Signed-off-by: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
There is no description of what this patch does. Its not enough to have
it on the cover letter (because that won't end up in version control).
You need something here.
> diff --git a/drivers/media/platform/mtk-vcodec/Kconfig b/drivers/media/platform/mtk-vcodec/Kconfig
> new file mode 100644
> index 0000000..1c0b935
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vcodec/Kconfig
> @@ -0,0 +1,5 @@
> +config MEDIATEK_VPU
> + bool
> + ---help---
> + This driver provides downloading firmware vpu (video processor unit)
> + and communicating with vpu.
Haven't I seen this before (in patch 3)? Why is it being added to
another Kconfig file?
> diff --git a/drivers/media/platform/mtk-vcodec/Makefile b/drivers/media/platform/mtk-vcodec/Makefile
> new file mode 100644
> index 0000000..c7f7174
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vcodec/Makefile
> @@ -0,0 +1,12 @@
> +obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += mtk_vcodec_intr.o \
> + mtk_vcodec_util.o \
> + mtk_vcodec_enc_drv.o \
> + mtk_vcodec_enc.o \
> + mtk_vcodec_enc_pm.o
> +
> +obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += common/
> +
> +ccflags-y += -I$(srctree)/drivers/media/platform/mtk-vcodec/include \
> + -I$(srctree)/drivers/media/platform/mtk-vcodec \
> + -I$(srctree)/drivers/media/platform/mtk-vpu
Seems like there's a lot of directories here. Are these files
(framework, common, vcodec, etc) so unrelated they really need to live
in separate directories?
Why not just drivers/media/platform/mediatek?
> diff --git a/drivers/media/platform/mtk-vcodec/common/Makefile b/drivers/media/platform/mtk-vcodec/common/Makefile
> new file mode 100644
> index 0000000..477ab80
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vcodec/common/Makefile
> @@ -0,0 +1,8 @@
> +obj-y += \
> + venc_drv_if.o
> +
> +ccflags-y += \
> + -I$(srctree)/include/ \
> + -I$(srctree)/drivers/media/platform/mtk-vcodec \
> + -I$(srctree)/drivers/media/platform/mtk-vcodec/include \
> + -I$(srctree)/drivers/media/platform/mtk-vpu
As above, this appears to be a directory to hold just one file.
> diff --git a/drivers/media/platform/mtk-vcodec/common/venc_drv_if.c
b/drivers/media/platform/mtk-vcodec/common/venc_drv_if.c
> new file mode 100644
> index 0000000..9b3f025
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vcodec/common/venc_drv_if.c
> @@ -0,0 +1,152 @@
> +/*
> + * Copyright (c) 2015 MediaTek Inc.
> + * Author: Daniel Hsiao <daniel.hsiao@mediatek.com>
> + * Jungchang Tsao <jungchang.tsao@mediatek.com>
> + *
> + * This program is free software; you can redistribute it and/or
> + * modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> + * GNU General Public License for more details.
> + */
> +
> +#include <linux/interrupt.h>
> +#include <linux/kernel.h>
> +#include <linux/slab.h>
> +
> +#include "mtk_vcodec_drv.h"
> +#include "mtk_vcodec_enc.h"
> +#include "mtk_vcodec_pm.h"
> +#include "mtk_vcodec_util.h"
> +#include "mtk_vpu_core.h"
> +
> +#include "venc_drv_if.h"
> +#include "venc_drv_base.h"
> +
> +
> +int venc_if_create(void *ctx, unsigned int fourcc, unsigned long
*handle)
> +{
> + struct venc_handle *h;
> + char str[10];
> +
> + mtk_vcodec_fmt2str(fourcc, str);
> +
> + h = kzalloc(sizeof(*h), GFP_KERNEL);
> + if (!h)
> + return -ENOMEM;
> +
> + h->fourcc = fourcc;
> + h->ctx = ctx;
> + mtk_vcodec_debug(h, "fmt = %s handle = %p", str, h);
> +
> + switch (fourcc) {
> + default:
> + mtk_vcodec_err(h, "invalid format %s", str);
> + goto err_out;
> + }
> +
> + *handle = (unsigned long)h;
> + return 0;
> +
> +err_out:
> + kfree(h);
> + return -EINVAL;
> +}
> +
> +int venc_if_init(unsigned long handle)
> +{
> + int ret = 0;
> + struct venc_handle *h = (struct venc_handle *)handle;
> +
> + mtk_vcodec_debug_enter(h);
> +
> + mtk_venc_lock(h->ctx);
> + mtk_vcodec_enc_clock_on();
> + vpu_enable_clock(vpu_get_plat_device(h->ctx->dev->plat_dev));
> + ret = h->enc_if->init(h->ctx, (unsigned long *)&h->drv_handle);
> + vpu_disable_clock(vpu_get_plat_device(h->ctx->dev->plat_dev));
> + mtk_vcodec_enc_clock_off();
> + mtk_venc_unlock(h->ctx);
> +
> + return ret;
> +}
To me this looks more like an obfuscation layer rather than a
abstraction layer. I don't understand why we need to hide things from
the V4L2 implementation that this code forms part of.
More importantly, if this code was included somewhere where it could be
properly integrated with the device model you might be able to use the
pm_runtime system to avoid this sort of "heroics" to manage the clocks
anyway.
> diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
> new file mode 100644
> index 0000000..22239f8
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
> @@ -0,0 +1,441 @@
> +/*
> +* Copyright (c) 2015 MediaTek Inc.
> +* Author: PC Chen <pc.chen@mediatek.com>
> +* Tiffany Lin <tiffany.lin@mediatek.com>
> +*
> +* This program is free software; you can redistribute it and/or modify
> +* it under the terms of the GNU General Public License version 2 as
> +* published by the Free Software Foundation.
> +*
> +* This program is distributed in the hope that it will be useful,
> +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> +* GNU General Public License for more details.
> +*/
> +
> +#ifndef _MTK_VCODEC_DRV_H_
> +#define _MTK_VCODEC_DRV_H_
> +
> +#include <linux/platform_device.h>
> +#include <linux/videodev2.h>
> +#include <media/v4l2-ctrls.h>
> +#include <media/v4l2-device.h>
> +#include <media/v4l2-ioctl.h>
> +#include <media/videobuf2-core.h>
> +#include <media/videobuf2-v4l2.h>
> +
> +#include "venc_drv_if.h"
> +
> +#define MTK_VCODEC_MAX_INSTANCES 32
> +#define MTK_VCODEC_MAX_FRAME_SIZE 0x800000
> +#define MTK_VIDEO_MAX_FRAME 32
> +#define MTK_MAX_CTRLS 10
> +
> +#define MTK_VCODEC_DRV_NAME "mtk_vcodec_drv"
> +#define MTK_VCODEC_ENC_NAME "mtk-vcodec-enc"
> +
> +#define MTK_VENC_IRQ_STATUS_SPS 0x1
> +#define MTK_VENC_IRQ_STATUS_PPS 0x2
> +#define MTK_VENC_IRQ_STATUS_FRM 0x4
> +#define MTK_VENC_IRQ_STATUS_DRAM 0x8
> +#define MTK_VENC_IRQ_STATUS_PAUSE 0x10
> +#define MTK_VENC_IRQ_STATUS_SWITCH 0x20
Probably better to use BIT(0) .. BIT(5).
> +#define MTK_VENC_IRQ_STATUS_OFFSET 0x05C
> +#define MTK_VENC_IRQ_ACK_OFFSET 0x060
> +
> +#define MTK_VCODEC_MAX_PLANES 3
> +
> +#define VDEC_HW_ACTIVE 0x10
> +#define VDEC_IRQ_CFG 0x11
> +#define VDEC_IRQ_CLR 0x10
> +
> +#define VDEC_IRQ_CFG_REG 0xa4
> +#define NUM_MAX_ALLOC_CTX 4
> +#define MTK_V4L2_BENCHMARK 0
> +#define USE_ENCODE_THREAD 1
> +
> +/**
> + * enum mtk_hw_reg_idx - MTK hw register base index
> + */
> +enum mtk_hw_reg_idx {
> + VDEC_SYS,
> + VDEC_MISC,
> + VDEC_LD,
> + VDEC_TOP,
> + VDEC_CM,
> + VDEC_AD,
> + VDEC_AV,
> + VDEC_PP,
> + VDEC_HWD,
> + VDEC_HWQ,
> + VDEC_HWB,
> + VDEC_HWG,
> + NUM_MAX_VDEC_REG_BASE,
> + VENC_SYS = NUM_MAX_VDEC_REG_BASE,
> + VENC_LT_SYS,
> + NUM_MAX_VCODEC_REG_BASE
> +};
> +
> +/**
> + * enum mtk_instance_type - The type of an MTK Vcodec instance.
> + */
> +enum mtk_instance_type {
> + MTK_INST_DECODER = 0,
> + MTK_INST_ENCODER = 1,
> +};
> +
> +/**
> + * enum mtk_instance_state - The state of an MTK Vcodec instance.
> + * @MTK_STATE_FREE - default state when instance create
> + * @MTK_STATE_CREATE - vdec instance is create
> + * @MTK_STATE_INIT - vdec instance is init
> + * @MTK_STATE_CONFIG - reserved for encoder
> + * @MTK_STATE_HEADER - vdec had sps/pps header parsed
> + * @MTK_STATE_RUNNING - vdec is decoding
> + * @MTK_STATE_FLUSH - vdec is flushing
> + * @MTK_STATE_RES_CHANGE - vdec detect resolution change
> + * @MTK_STATE_FINISH - ctx instance is stopped streaming
> + * @MTK_STATE_DEINIT - before release ctx instance
> + * @MTK_STATE_ERROR - vdec has something wrong
> + * @MTK_STATE_ABORT - abort work in working thread
> + */
> +enum mtk_instance_state {
> + MTK_STATE_FREE = 0,
> + MTK_STATE_CREATE = (1 << 0),
> + MTK_STATE_INIT = (1 << 1),
> + MTK_STATE_CONFIG = (1 << 2),
> + MTK_STATE_HEADER = (1 << 3),
> + MTK_STATE_RUNNING = (1 << 4),
> + MTK_STATE_FLUSH = (1 << 5),
> + MTK_STATE_RES_CHANGE = (1 << 6),
> + MTK_STATE_FINISH = (1 << 7),
> + MTK_STATE_DEINIT = (1 << 8),
> + MTK_STATE_ERROR = (1 << 9),
> + MTK_STATE_ABORT = (1 << 10),
This looks like it started as a state machine and somehow turned into
flags, resulting in a state machine with 2048 states or, to give it a
different name, a debugging nightmare.
If the start streaming operation implemented cleanup-on-error properly
then there would only be two useful states: Started and stopped. Even
the "sticky" error behavior looks unnecessary to me (meaning we don't
need to track its state).
> diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
> new file mode 100644
> index 0000000..8e1b6f0
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
> @@ -0,0 +1,1773 @@
> [...]
> +static int vb2ops_venc_start_streaming(struct vb2_queue *q, unsigned int count)
> +{
> + struct mtk_vcodec_ctx *ctx = vb2_get_drv_priv(q);
> + struct v4l2_device *v4l2_dev = &ctx->dev->v4l2_dev;
> + int ret;
> +#if MTK_V4L2_BENCHMARK
> + struct timeval begin, end;
> +
> + do_gettimeofday(&begin);
> +#endif
> +
> + if (!(vb2_start_streaming_called(&ctx->m2m_ctx->out_q_ctx.q) &
> + vb2_start_streaming_called(&ctx->m2m_ctx->cap_q_ctx.q))) {
> + mtk_v4l2_debug(1, "[%d]-> out=%d cap=%d",
> + ctx->idx,
> + vb2_start_streaming_called(&ctx->m2m_ctx->out_q_ctx.q),
> + vb2_start_streaming_called(&ctx->m2m_ctx->cap_q_ctx.q));
> + return 0;
> + }
> +
> + if ((ctx->state & (MTK_STATE_ERROR | MTK_STATE_ABORT)))
> + return -EINVAL;
This is the sort of thing I mean.
This sticky error behaviour means that every subsequent call to
vb2ops_venc_start_streaming() will fail. Note also that the user will
never try to stop streaming (which can clear the error state) because
according to the return code it got when it tried to start streaming we
never actually started.
This is what I mean about having two many states. From the user's
perspective there are only two states. There needs to be a good reason
for the driver to manage so many extra secret states internally.
> +
> + if (ctx->state == MTK_STATE_FREE) {
> + ret = venc_if_create(ctx,
> + ctx->q_data[MTK_Q_DATA_DST].fmt->fourcc,
> + &ctx->h_enc);
> +
> + if (ret != 0) {
> + ctx->state |= MTK_STATE_ERROR;
> + v4l2_err(v4l2_dev, "invalid codec type=%x\n",
> + ctx->q_data[MTK_Q_DATA_DST].fmt->fourcc);
> + v4l2_err(v4l2_dev, "venc_if_create failed=%d\n", ret);
> + return -EINVAL;
> + }
> +
> + if (ctx->q_data[MTK_Q_DATA_DST].fmt->fourcc ==
> + V4L2_PIX_FMT_H264)
> + ctx->hdr = 1;
> +
> + ctx->state |= MTK_STATE_CREATE;
> + }
> +
> + if ((ctx->state & MTK_STATE_CREATE) && !(ctx->state & MTK_STATE_INIT)) {
> + ret = venc_if_init(ctx->h_enc);
> + if (ret != 0) {
> + ctx->state |= MTK_STATE_ERROR;
> + v4l2_err(v4l2_dev, "venc_if_init failed=%d\n", ret);
> + return -EINVAL;
This error path leaves the encoder partially constructed and relies on
something else to tidy things up. It would be much better to tidy things
up from this function and
Also I don't think both venc_if_create and venc_if_init are needed. They
are only ever called one after the other and thus they only serve to
complicate the error handling code.
> diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
> new file mode 100644
> index 0000000..a8e683a
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
> @@ -0,0 +1,66 @@
> +/*
> +* Copyright (c) 2015 MediaTek Inc.
> +* Author: PC Chen <pc.chen@mediatek.com>
> +* Tiffany Lin <tiffany.lin@mediatek.com>
> +*
> +* This program is free software; you can redistribute it and/or modify
> +* it under the terms of the GNU General Public License version 2 as
> +* published by the Free Software Foundation.
> +*
> +* This program is distributed in the hope that it will be useful,
> +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> +* GNU General Public License for more details.
> +*/
> +
> +#ifndef _MTK_VCODEC_UTIL_H_
> +#define _MTK_VCODEC_UTIL_H_
> +
> +#include <linux/types.h>
> +#include <linux/dma-direction.h>
> +
> +struct mtk_vcodec_mem {
> + size_t size;
> + void *va;
> + dma_addr_t dma_addr;
> +};
> +
> +extern int mtk_v4l2_dbg_level;
> +extern bool mtk_vcodec_dbg;
> +
> +#define mtk_v4l2_debug(level, fmt, args...) \
> + do { \
> + if (mtk_v4l2_dbg_level >= level) \
> + pr_info("[MTK_V4L2] level=%d %s(),%d: " fmt "\n",\
> + level, __func__, __LINE__, ##args); \
> + } while (0)
> +
> +#define mtk_v4l2_err(fmt, args...) \
> + pr_err("[MTK_V4L2][ERROR] %s:%d: " fmt "\n", __func__, __LINE__, \
> + ##args)
Obviously the code should be structured to make use of dev_dbg/dev_err
possible.
However where this won't work do you really need special macros for
this. Assuming your error messages are well written 'git grep' and the
following should be enough:
#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
> +#define mtk_v4l2_debug_enter() mtk_v4l2_debug(5, "+\n")
> +#define mtk_v4l2_debug_leave() mtk_v4l2_debug(5, "-\n")
Remove these. If you care about function entry and exit for debugging
you should be able to use ftrace.
> +#define mtk_vcodec_debug(h, fmt, args...) \
> + do { \
> + if (mtk_vcodec_dbg) \
> + pr_info("[MTK_VCODEC][%d]: %s() " fmt "\n", \
> + ((struct mtk_vcodec_ctx *)h->ctx)->idx, \
> + __func__, ##args); \
> + } while (0)
> +
> +#define mtk_vcodec_err(h, fmt, args...) \
> + pr_err("[MTK_VCODEC][ERROR][%d]: %s() " fmt "\n", \
> + ((struct mtk_vcodec_ctx *)h->ctx)->idx, __func__, ##args)
> +
> +#define mtk_vcodec_debug_enter(h) mtk_vcodec_debug(h, "+\n")
> +#define mtk_vcodec_debug_leave(h) mtk_vcodec_debug(h, "-\n")
All above comments apply to these too.
^ permalink raw reply [flat|nested] 84+ messages in thread
* Re: [RESEND RFC/PATCH 6/8] media: platform: mtk-vcodec: Add Mediatek V4L2 Video Encoder Driver
@ 2015-11-30 11:39 ` tiffany lin
0 siblings, 0 replies; 84+ messages in thread
From: tiffany lin @ 2015-11-30 11:39 UTC (permalink / raw)
To: Daniel Thompson
Cc: Rob Herring, Pawel Moll, Mark Rutland, Ian Campbell, Kumar Gala,
Catalin Marinas, Will Deacon, Mauro Carvalho Chehab,
Matthias Brugger, Daniel Kurtz, Sascha Hauer, Hongzhou Yang,
Hans Verkuil, Laurent Pinchart, Sakari Ailus, Geert Uytterhoeven,
Mikhail Ulyanov, Fabien Dessenne, Arnd Bergmann,
Darren Etheridge, Peter Griffin, Benoit Parrot, Andrew-CT Chen,
Eddie Huang, Yingjoe Chen, James Liao, Daniel Hsiao, devicetree,
linux-kernel, linux-arm-kernel, linux-media, linux-mediatek,
PoChun.Lin
On Fri, 2015-11-27 at 16:34 +0000, Daniel Thompson wrote:
> Hi Tiffany/Andrew
>
> This review is a rather more superficial than my previous one. Mostly
> I'm just commenting on some of the bits I spotted whilst trying to find
> my way around the patchset.
>
> I hope to another more detailed review for v2 (and feel free to add me
> to Cc:).
>
Thanks for your comments.
>
> On 17/11/15 12:54, Tiffany Lin wrote:
> > Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
> > Signed-off-by: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
>
> There is no description of what this patch does. Its not enough to have
> it on the cover letter (because that won't end up in version control).
> You need something here.
>
Got it, We will add description for each patch in next version.
>
> > diff --git a/drivers/media/platform/mtk-vcodec/Kconfig b/drivers/media/platform/mtk-vcodec/Kconfig
> > new file mode 100644
> > index 0000000..1c0b935
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/Kconfig
> > @@ -0,0 +1,5 @@
> > +config MEDIATEK_VPU
> > + bool
> > + ---help---
> > + This driver provides downloading firmware vpu (video processor unit)
> > + and communicating with vpu.
>
> Haven't I seen this before (in patch 3)? Why is it being added to
> another Kconfig file?
>
We will remove this in next version.
>
> > diff --git a/drivers/media/platform/mtk-vcodec/Makefile b/drivers/media/platform/mtk-vcodec/Makefile
> > new file mode 100644
> > index 0000000..c7f7174
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/Makefile
> > @@ -0,0 +1,12 @@
> > +obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += mtk_vcodec_intr.o \
> > + mtk_vcodec_util.o \
> > + mtk_vcodec_enc_drv.o \
> > + mtk_vcodec_enc.o \
> > + mtk_vcodec_enc_pm.o
> > +
> > +obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += common/
> > +
> > +ccflags-y += -I$(srctree)/drivers/media/platform/mtk-vcodec/include \
> > + -I$(srctree)/drivers/media/platform/mtk-vcodec \
> > + -I$(srctree)/drivers/media/platform/mtk-vpu
>
> Seems like there's a lot of directories here. Are these files
> (framework, common, vcodec, etc) so unrelated they really need to live
> in separate directories?
>
> Why not just drivers/media/platform/mediatek?
This is because VPU and Vcodec are two different drivers.
Driver in mtk-vpu is for controlling VPU device and provide
communication API to VPU.
Driver in mtk-vcodec is for control different encoder (vp8, h264), it
include v4l2 driver layer, glue layer between encoders and vp8 and h264
encoder.
>
>
> > diff --git a/drivers/media/platform/mtk-vcodec/common/Makefile b/drivers/media/platform/mtk-vcodec/common/Makefile
> > new file mode 100644
> > index 0000000..477ab80
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/common/Makefile
> > @@ -0,0 +1,8 @@
> > +obj-y += \
> > + venc_drv_if.o
> > +
> > +ccflags-y += \
> > + -I$(srctree)/include/ \
> > + -I$(srctree)/drivers/media/platform/mtk-vcodec \
> > + -I$(srctree)/drivers/media/platform/mtk-vcodec/include \
> > + -I$(srctree)/drivers/media/platform/mtk-vpu
>
> As above, this appears to be a directory to hold just one file.
>
Sorry, I didn't get it. Could you explain more?
>
> > diff --git a/drivers/media/platform/mtk-vcodec/common/venc_drv_if.c
> b/drivers/media/platform/mtk-vcodec/common/venc_drv_if.c
> > new file mode 100644
> > index 0000000..9b3f025
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/common/venc_drv_if.c
> > @@ -0,0 +1,152 @@
> > +/*
> > + * Copyright (c) 2015 MediaTek Inc.
> > + * Author: Daniel Hsiao <daniel.hsiao@mediatek.com>
> > + * Jungchang Tsao <jungchang.tsao@mediatek.com>
> > + *
> > + * This program is free software; you can redistribute it and/or
> > + * modify
> > + * it under the terms of the GNU General Public License version 2 as
> > + * published by the Free Software Foundation.
> > + *
> > + * This program is distributed in the hope that it will be useful,
> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> > + * GNU General Public License for more details.
> > + */
> > +
> > +#include <linux/interrupt.h>
> > +#include <linux/kernel.h>
> > +#include <linux/slab.h>
> > +
> > +#include "mtk_vcodec_drv.h"
> > +#include "mtk_vcodec_enc.h"
> > +#include "mtk_vcodec_pm.h"
> > +#include "mtk_vcodec_util.h"
> > +#include "mtk_vpu_core.h"
> > +
> > +#include "venc_drv_if.h"
> > +#include "venc_drv_base.h"
> > +
> > +
> > +int venc_if_create(void *ctx, unsigned int fourcc, unsigned long
> *handle)
> > +{
> > + struct venc_handle *h;
> > + char str[10];
> > +
> > + mtk_vcodec_fmt2str(fourcc, str);
> > +
> > + h = kzalloc(sizeof(*h), GFP_KERNEL);
> > + if (!h)
> > + return -ENOMEM;
> > +
> > + h->fourcc = fourcc;
> > + h->ctx = ctx;
> > + mtk_vcodec_debug(h, "fmt = %s handle = %p", str, h);
> > +
> > + switch (fourcc) {
> > + default:
> > + mtk_vcodec_err(h, "invalid format %s", str);
> > + goto err_out;
> > + }
> > +
> > + *handle = (unsigned long)h;
> > + return 0;
> > +
> > +err_out:
> > + kfree(h);
> > + return -EINVAL;
> > +}
> > +
> > +int venc_if_init(unsigned long handle)
> > +{
> > + int ret = 0;
> > + struct venc_handle *h = (struct venc_handle *)handle;
> > +
> > + mtk_vcodec_debug_enter(h);
> > +
> > + mtk_venc_lock(h->ctx);
> > + mtk_vcodec_enc_clock_on();
> > + vpu_enable_clock(vpu_get_plat_device(h->ctx->dev->plat_dev));
> > + ret = h->enc_if->init(h->ctx, (unsigned long *)&h->drv_handle);
> > + vpu_disable_clock(vpu_get_plat_device(h->ctx->dev->plat_dev));
> > + mtk_vcodec_enc_clock_off();
> > + mtk_venc_unlock(h->ctx);
> > +
> > + return ret;
> > +}
>
> To me this looks more like an obfuscation layer rather than a
> abstraction layer. I don't understand why we need to hide things from
> the V4L2 implementation that this code forms part of.
>
> More importantly, if this code was included somewhere where it could be
> properly integrated with the device model you might be able to use the
> pm_runtime system to avoid this sort of "heroics" to manage the clocks
> anyway.
>
We want to abstract common part from encoder driver.
Every encoder driver follow same calling flow and only need to take care
about how to communicate with vpu to encode specific format.
Encoder driver do not need to take care clock and multiple instance
issue.
>
> > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
> > new file mode 100644
> > index 0000000..22239f8
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
> > @@ -0,0 +1,441 @@
> > +/*
> > +* Copyright (c) 2015 MediaTek Inc.
> > +* Author: PC Chen <pc.chen@mediatek.com>
> > +* Tiffany Lin <tiffany.lin@mediatek.com>
> > +*
> > +* This program is free software; you can redistribute it and/or modify
> > +* it under the terms of the GNU General Public License version 2 as
> > +* published by the Free Software Foundation.
> > +*
> > +* This program is distributed in the hope that it will be useful,
> > +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> > +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> > +* GNU General Public License for more details.
> > +*/
> > +
> > +#ifndef _MTK_VCODEC_DRV_H_
> > +#define _MTK_VCODEC_DRV_H_
> > +
> > +#include <linux/platform_device.h>
> > +#include <linux/videodev2.h>
> > +#include <media/v4l2-ctrls.h>
> > +#include <media/v4l2-device.h>
> > +#include <media/v4l2-ioctl.h>
> > +#include <media/videobuf2-core.h>
> > +#include <media/videobuf2-v4l2.h>
> > +
> > +#include "venc_drv_if.h"
> > +
> > +#define MTK_VCODEC_MAX_INSTANCES 32
> > +#define MTK_VCODEC_MAX_FRAME_SIZE 0x800000
> > +#define MTK_VIDEO_MAX_FRAME 32
> > +#define MTK_MAX_CTRLS 10
> > +
> > +#define MTK_VCODEC_DRV_NAME "mtk_vcodec_drv"
> > +#define MTK_VCODEC_ENC_NAME "mtk-vcodec-enc"
> > +
> > +#define MTK_VENC_IRQ_STATUS_SPS 0x1
> > +#define MTK_VENC_IRQ_STATUS_PPS 0x2
> > +#define MTK_VENC_IRQ_STATUS_FRM 0x4
> > +#define MTK_VENC_IRQ_STATUS_DRAM 0x8
> > +#define MTK_VENC_IRQ_STATUS_PAUSE 0x10
> > +#define MTK_VENC_IRQ_STATUS_SWITCH 0x20
>
> Probably better to use BIT(0) .. BIT(5).
Will change to use BIT(0) in next version.
>
>
> > +#define MTK_VENC_IRQ_STATUS_OFFSET 0x05C
> > +#define MTK_VENC_IRQ_ACK_OFFSET 0x060
> > +
> > +#define MTK_VCODEC_MAX_PLANES 3
> > +
> > +#define VDEC_HW_ACTIVE 0x10
> > +#define VDEC_IRQ_CFG 0x11
> > +#define VDEC_IRQ_CLR 0x10
> > +
> > +#define VDEC_IRQ_CFG_REG 0xa4
> > +#define NUM_MAX_ALLOC_CTX 4
> > +#define MTK_V4L2_BENCHMARK 0
> > +#define USE_ENCODE_THREAD 1
> > +
> > +/**
> > + * enum mtk_hw_reg_idx - MTK hw register base index
> > + */
> > +enum mtk_hw_reg_idx {
> > + VDEC_SYS,
> > + VDEC_MISC,
> > + VDEC_LD,
> > + VDEC_TOP,
> > + VDEC_CM,
> > + VDEC_AD,
> > + VDEC_AV,
> > + VDEC_PP,
> > + VDEC_HWD,
> > + VDEC_HWQ,
> > + VDEC_HWB,
> > + VDEC_HWG,
> > + NUM_MAX_VDEC_REG_BASE,
> > + VENC_SYS = NUM_MAX_VDEC_REG_BASE,
> > + VENC_LT_SYS,
> > + NUM_MAX_VCODEC_REG_BASE
> > +};
> > +
> > +/**
> > + * enum mtk_instance_type - The type of an MTK Vcodec instance.
> > + */
> > +enum mtk_instance_type {
> > + MTK_INST_DECODER = 0,
> > + MTK_INST_ENCODER = 1,
> > +};
> > +
> > +/**
> > + * enum mtk_instance_state - The state of an MTK Vcodec instance.
> > + * @MTK_STATE_FREE - default state when instance create
> > + * @MTK_STATE_CREATE - vdec instance is create
> > + * @MTK_STATE_INIT - vdec instance is init
> > + * @MTK_STATE_CONFIG - reserved for encoder
> > + * @MTK_STATE_HEADER - vdec had sps/pps header parsed
> > + * @MTK_STATE_RUNNING - vdec is decoding
> > + * @MTK_STATE_FLUSH - vdec is flushing
> > + * @MTK_STATE_RES_CHANGE - vdec detect resolution change
> > + * @MTK_STATE_FINISH - ctx instance is stopped streaming
> > + * @MTK_STATE_DEINIT - before release ctx instance
> > + * @MTK_STATE_ERROR - vdec has something wrong
> > + * @MTK_STATE_ABORT - abort work in working thread
> > + */
> > +enum mtk_instance_state {
> > + MTK_STATE_FREE = 0,
> > + MTK_STATE_CREATE = (1 << 0),
> > + MTK_STATE_INIT = (1 << 1),
> > + MTK_STATE_CONFIG = (1 << 2),
> > + MTK_STATE_HEADER = (1 << 3),
> > + MTK_STATE_RUNNING = (1 << 4),
> > + MTK_STATE_FLUSH = (1 << 5),
> > + MTK_STATE_RES_CHANGE = (1 << 6),
> > + MTK_STATE_FINISH = (1 << 7),
> > + MTK_STATE_DEINIT = (1 << 8),
> > + MTK_STATE_ERROR = (1 << 9),
> > + MTK_STATE_ABORT = (1 << 10),
>
> This looks like it started as a state machine and somehow turned into
> flags, resulting in a state machine with 2048 states or, to give it a
> different name, a debugging nightmare.
>
It's define some state happened rather than state machine.
Though some states are for v4l2 decoder driver and not used in encoder
driver.
> If the start streaming operation implemented cleanup-on-error properly
> then there would only be two useful states: Started and stopped. Even
> the "sticky" error behavior looks unnecessary to me (meaning we don't
> need to track its state).
>
We cannot guaranteed that IOCTLs called from the user space follow
required sequence.
We need states to know if our driver could accept IOCTL command.
>
> > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
> > new file mode 100644
> > index 0000000..8e1b6f0
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
> > @@ -0,0 +1,1773 @@
> > [...]
> > +static int vb2ops_venc_start_streaming(struct vb2_queue *q, unsigned int count)
> > +{
> > + struct mtk_vcodec_ctx *ctx = vb2_get_drv_priv(q);
> > + struct v4l2_device *v4l2_dev = &ctx->dev->v4l2_dev;
> > + int ret;
> > +#if MTK_V4L2_BENCHMARK
> > + struct timeval begin, end;
> > +
> > + do_gettimeofday(&begin);
> > +#endif
> > +
> > + if (!(vb2_start_streaming_called(&ctx->m2m_ctx->out_q_ctx.q) &
> > + vb2_start_streaming_called(&ctx->m2m_ctx->cap_q_ctx.q))) {
> > + mtk_v4l2_debug(1, "[%d]-> out=%d cap=%d",
> > + ctx->idx,
> > + vb2_start_streaming_called(&ctx->m2m_ctx->out_q_ctx.q),
> > + vb2_start_streaming_called(&ctx->m2m_ctx->cap_q_ctx.q));
> > + return 0;
> > + }
> > +
> > + if ((ctx->state & (MTK_STATE_ERROR | MTK_STATE_ABORT)))
> > + return -EINVAL;
>
> This is the sort of thing I mean.
>
> This sticky error behaviour means that every subsequent call to
> vb2ops_venc_start_streaming() will fail. Note also that the user will
> never try to stop streaming (which can clear the error state) because
> according to the return code it got when it tried to start streaming we
> never actually started.
>
> This is what I mean about having two many states. From the user's
> perspective there are only two states. There needs to be a good reason
> for the driver to manage so many extra secret states internally.
>
For my understanding, that vb2ops_venc_start_streaming cannot fail.
If it fail, user space will close and release this encoder instance
(fd).
We really need to state driver to see what it should do when receive
current IOCTL.
>
> > +
> > + if (ctx->state == MTK_STATE_FREE) {
> > + ret = venc_if_create(ctx,
> > + ctx->q_data[MTK_Q_DATA_DST].fmt->fourcc,
> > + &ctx->h_enc);
> > +
> > + if (ret != 0) {
> > + ctx->state |= MTK_STATE_ERROR;
> > + v4l2_err(v4l2_dev, "invalid codec type=%x\n",
> > + ctx->q_data[MTK_Q_DATA_DST].fmt->fourcc);
> > + v4l2_err(v4l2_dev, "venc_if_create failed=%d\n", ret);
> > + return -EINVAL;
> > + }
> > +
> > + if (ctx->q_data[MTK_Q_DATA_DST].fmt->fourcc ==
> > + V4L2_PIX_FMT_H264)
> > + ctx->hdr = 1;
> > +
> > + ctx->state |= MTK_STATE_CREATE;
> > + }
> > +
> > + if ((ctx->state & MTK_STATE_CREATE) && !(ctx->state & MTK_STATE_INIT)) {
> > + ret = venc_if_init(ctx->h_enc);
> > + if (ret != 0) {
> > + ctx->state |= MTK_STATE_ERROR;
> > + v4l2_err(v4l2_dev, "venc_if_init failed=%d\n", ret);
> > + return -EINVAL;
>
> This error path leaves the encoder partially constructed and relies on
> something else to tidy things up. It would be much better to tidy things
> up from this function and
>
> Also I don't think both venc_if_create and venc_if_init are needed. They
> are only ever called one after the other and thus they only serve to
> complicate the error handling code.
>
venc_if_create is for creating instance in arm side and base on encode
format hook corresponding encoder driver interface.
venc_if_init is trying to init encoder instance in VPU side.
Failures from two functions should have different error handling.
We will enhance this part in next version.
>
> > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
> > new file mode 100644
> > index 0000000..a8e683a
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
> > @@ -0,0 +1,66 @@
> > +/*
> > +* Copyright (c) 2015 MediaTek Inc.
> > +* Author: PC Chen <pc.chen@mediatek.com>
> > +* Tiffany Lin <tiffany.lin@mediatek.com>
> > +*
> > +* This program is free software; you can redistribute it and/or modify
> > +* it under the terms of the GNU General Public License version 2 as
> > +* published by the Free Software Foundation.
> > +*
> > +* This program is distributed in the hope that it will be useful,
> > +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> > +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> > +* GNU General Public License for more details.
> > +*/
> > +
> > +#ifndef _MTK_VCODEC_UTIL_H_
> > +#define _MTK_VCODEC_UTIL_H_
> > +
> > +#include <linux/types.h>
> > +#include <linux/dma-direction.h>
> > +
> > +struct mtk_vcodec_mem {
> > + size_t size;
> > + void *va;
> > + dma_addr_t dma_addr;
> > +};
> > +
> > +extern int mtk_v4l2_dbg_level;
> > +extern bool mtk_vcodec_dbg;
> > +
> > +#define mtk_v4l2_debug(level, fmt, args...) \
> > + do { \
> > + if (mtk_v4l2_dbg_level >= level) \
> > + pr_info("[MTK_V4L2] level=%d %s(),%d: " fmt "\n",\
> > + level, __func__, __LINE__, ##args); \
> > + } while (0)
> > +
> > +#define mtk_v4l2_err(fmt, args...) \
> > + pr_err("[MTK_V4L2][ERROR] %s:%d: " fmt "\n", __func__, __LINE__, \
> > + ##args)
>
> Obviously the code should be structured to make use of dev_dbg/dev_err
> possible.
>
> However where this won't work do you really need special macros for
> this. Assuming your error messages are well written 'git grep' and the
> following should be enough:
>
> #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
>
Thanks.
For pr_err case, we will try to use "#define pr_fmt(fmt) KBUILD_MODNAME
": " fmt" in next version.
For pr_info case, we still need debug level to control output messages.
>
> > +#define mtk_v4l2_debug_enter() mtk_v4l2_debug(5, "+\n")
> > +#define mtk_v4l2_debug_leave() mtk_v4l2_debug(5, "-\n")
>
> Remove these. If you care about function entry and exit for debugging
> you should be able to use ftrace.
>
I am not familiar with ftrace.
What if we only want to trace v4l2 video encoder driver called flow not
called stack? And only for functions we are interested not all
functions.
I will check if it is convenience for us using ftrace.
>
> > +#define mtk_vcodec_debug(h, fmt, args...) \
> > + do { \
> > + if (mtk_vcodec_dbg) \
> > + pr_info("[MTK_VCODEC][%d]: %s() " fmt "\n", \
> > + ((struct mtk_vcodec_ctx *)h->ctx)->idx, \
> > + __func__, ##args); \
> > + } while (0)
> > +
> > +#define mtk_vcodec_err(h, fmt, args...) \
> > + pr_err("[MTK_VCODEC][ERROR][%d]: %s() " fmt "\n", \
> > + ((struct mtk_vcodec_ctx *)h->ctx)->idx, __func__, ##args)
> > +
> > +#define mtk_vcodec_debug_enter(h) mtk_vcodec_debug(h, "+\n")
> > +#define mtk_vcodec_debug_leave(h) mtk_vcodec_debug(h, "-\n")
>
> All above comments apply to these too.
>
best regards,
Tiffany
^ permalink raw reply [flat|nested] 84+ messages in thread
* Re: [RESEND RFC/PATCH 6/8] media: platform: mtk-vcodec: Add Mediatek V4L2 Video Encoder Driver
@ 2015-11-30 11:39 ` tiffany lin
0 siblings, 0 replies; 84+ messages in thread
From: tiffany lin @ 2015-11-30 11:39 UTC (permalink / raw)
To: Daniel Thompson
Cc: Rob Herring, Pawel Moll, Mark Rutland, Ian Campbell, Kumar Gala,
Catalin Marinas, Will Deacon, Mauro Carvalho Chehab,
Matthias Brugger, Daniel Kurtz, Sascha Hauer, Hongzhou Yang,
Hans Verkuil, Laurent Pinchart, Sakari Ailus, Geert Uytterhoeven,
Mikhail Ulyanov, Fabien Dessenne, Arnd Bergmann,
Darren Etheridge, Peter Griffin, Benoit
On Fri, 2015-11-27 at 16:34 +0000, Daniel Thompson wrote:
> Hi Tiffany/Andrew
>
> This review is a rather more superficial than my previous one. Mostly
> I'm just commenting on some of the bits I spotted whilst trying to find
> my way around the patchset.
>
> I hope to another more detailed review for v2 (and feel free to add me
> to Cc:).
>
Thanks for your comments.
>
> On 17/11/15 12:54, Tiffany Lin wrote:
> > Signed-off-by: Tiffany Lin <tiffany.lin-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
> > Signed-off-by: Andrew-CT Chen <andrew-ct.chen-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
>
> There is no description of what this patch does. Its not enough to have
> it on the cover letter (because that won't end up in version control).
> You need something here.
>
Got it, We will add description for each patch in next version.
>
> > diff --git a/drivers/media/platform/mtk-vcodec/Kconfig b/drivers/media/platform/mtk-vcodec/Kconfig
> > new file mode 100644
> > index 0000000..1c0b935
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/Kconfig
> > @@ -0,0 +1,5 @@
> > +config MEDIATEK_VPU
> > + bool
> > + ---help---
> > + This driver provides downloading firmware vpu (video processor unit)
> > + and communicating with vpu.
>
> Haven't I seen this before (in patch 3)? Why is it being added to
> another Kconfig file?
>
We will remove this in next version.
>
> > diff --git a/drivers/media/platform/mtk-vcodec/Makefile b/drivers/media/platform/mtk-vcodec/Makefile
> > new file mode 100644
> > index 0000000..c7f7174
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/Makefile
> > @@ -0,0 +1,12 @@
> > +obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += mtk_vcodec_intr.o \
> > + mtk_vcodec_util.o \
> > + mtk_vcodec_enc_drv.o \
> > + mtk_vcodec_enc.o \
> > + mtk_vcodec_enc_pm.o
> > +
> > +obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += common/
> > +
> > +ccflags-y += -I$(srctree)/drivers/media/platform/mtk-vcodec/include \
> > + -I$(srctree)/drivers/media/platform/mtk-vcodec \
> > + -I$(srctree)/drivers/media/platform/mtk-vpu
>
> Seems like there's a lot of directories here. Are these files
> (framework, common, vcodec, etc) so unrelated they really need to live
> in separate directories?
>
> Why not just drivers/media/platform/mediatek?
This is because VPU and Vcodec are two different drivers.
Driver in mtk-vpu is for controlling VPU device and provide
communication API to VPU.
Driver in mtk-vcodec is for control different encoder (vp8, h264), it
include v4l2 driver layer, glue layer between encoders and vp8 and h264
encoder.
>
>
> > diff --git a/drivers/media/platform/mtk-vcodec/common/Makefile b/drivers/media/platform/mtk-vcodec/common/Makefile
> > new file mode 100644
> > index 0000000..477ab80
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/common/Makefile
> > @@ -0,0 +1,8 @@
> > +obj-y += \
> > + venc_drv_if.o
> > +
> > +ccflags-y += \
> > + -I$(srctree)/include/ \
> > + -I$(srctree)/drivers/media/platform/mtk-vcodec \
> > + -I$(srctree)/drivers/media/platform/mtk-vcodec/include \
> > + -I$(srctree)/drivers/media/platform/mtk-vpu
>
> As above, this appears to be a directory to hold just one file.
>
Sorry, I didn't get it. Could you explain more?
>
> > diff --git a/drivers/media/platform/mtk-vcodec/common/venc_drv_if.c
> b/drivers/media/platform/mtk-vcodec/common/venc_drv_if.c
> > new file mode 100644
> > index 0000000..9b3f025
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/common/venc_drv_if.c
> > @@ -0,0 +1,152 @@
> > +/*
> > + * Copyright (c) 2015 MediaTek Inc.
> > + * Author: Daniel Hsiao <daniel.hsiao-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
> > + * Jungchang Tsao <jungchang.tsao-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
> > + *
> > + * This program is free software; you can redistribute it and/or
> > + * modify
> > + * it under the terms of the GNU General Public License version 2 as
> > + * published by the Free Software Foundation.
> > + *
> > + * This program is distributed in the hope that it will be useful,
> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> > + * GNU General Public License for more details.
> > + */
> > +
> > +#include <linux/interrupt.h>
> > +#include <linux/kernel.h>
> > +#include <linux/slab.h>
> > +
> > +#include "mtk_vcodec_drv.h"
> > +#include "mtk_vcodec_enc.h"
> > +#include "mtk_vcodec_pm.h"
> > +#include "mtk_vcodec_util.h"
> > +#include "mtk_vpu_core.h"
> > +
> > +#include "venc_drv_if.h"
> > +#include "venc_drv_base.h"
> > +
> > +
> > +int venc_if_create(void *ctx, unsigned int fourcc, unsigned long
> *handle)
> > +{
> > + struct venc_handle *h;
> > + char str[10];
> > +
> > + mtk_vcodec_fmt2str(fourcc, str);
> > +
> > + h = kzalloc(sizeof(*h), GFP_KERNEL);
> > + if (!h)
> > + return -ENOMEM;
> > +
> > + h->fourcc = fourcc;
> > + h->ctx = ctx;
> > + mtk_vcodec_debug(h, "fmt = %s handle = %p", str, h);
> > +
> > + switch (fourcc) {
> > + default:
> > + mtk_vcodec_err(h, "invalid format %s", str);
> > + goto err_out;
> > + }
> > +
> > + *handle = (unsigned long)h;
> > + return 0;
> > +
> > +err_out:
> > + kfree(h);
> > + return -EINVAL;
> > +}
> > +
> > +int venc_if_init(unsigned long handle)
> > +{
> > + int ret = 0;
> > + struct venc_handle *h = (struct venc_handle *)handle;
> > +
> > + mtk_vcodec_debug_enter(h);
> > +
> > + mtk_venc_lock(h->ctx);
> > + mtk_vcodec_enc_clock_on();
> > + vpu_enable_clock(vpu_get_plat_device(h->ctx->dev->plat_dev));
> > + ret = h->enc_if->init(h->ctx, (unsigned long *)&h->drv_handle);
> > + vpu_disable_clock(vpu_get_plat_device(h->ctx->dev->plat_dev));
> > + mtk_vcodec_enc_clock_off();
> > + mtk_venc_unlock(h->ctx);
> > +
> > + return ret;
> > +}
>
> To me this looks more like an obfuscation layer rather than a
> abstraction layer. I don't understand why we need to hide things from
> the V4L2 implementation that this code forms part of.
>
> More importantly, if this code was included somewhere where it could be
> properly integrated with the device model you might be able to use the
> pm_runtime system to avoid this sort of "heroics" to manage the clocks
> anyway.
>
We want to abstract common part from encoder driver.
Every encoder driver follow same calling flow and only need to take care
about how to communicate with vpu to encode specific format.
Encoder driver do not need to take care clock and multiple instance
issue.
>
> > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
> > new file mode 100644
> > index 0000000..22239f8
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
> > @@ -0,0 +1,441 @@
> > +/*
> > +* Copyright (c) 2015 MediaTek Inc.
> > +* Author: PC Chen <pc.chen-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
> > +* Tiffany Lin <tiffany.lin-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
> > +*
> > +* This program is free software; you can redistribute it and/or modify
> > +* it under the terms of the GNU General Public License version 2 as
> > +* published by the Free Software Foundation.
> > +*
> > +* This program is distributed in the hope that it will be useful,
> > +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> > +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> > +* GNU General Public License for more details.
> > +*/
> > +
> > +#ifndef _MTK_VCODEC_DRV_H_
> > +#define _MTK_VCODEC_DRV_H_
> > +
> > +#include <linux/platform_device.h>
> > +#include <linux/videodev2.h>
> > +#include <media/v4l2-ctrls.h>
> > +#include <media/v4l2-device.h>
> > +#include <media/v4l2-ioctl.h>
> > +#include <media/videobuf2-core.h>
> > +#include <media/videobuf2-v4l2.h>
> > +
> > +#include "venc_drv_if.h"
> > +
> > +#define MTK_VCODEC_MAX_INSTANCES 32
> > +#define MTK_VCODEC_MAX_FRAME_SIZE 0x800000
> > +#define MTK_VIDEO_MAX_FRAME 32
> > +#define MTK_MAX_CTRLS 10
> > +
> > +#define MTK_VCODEC_DRV_NAME "mtk_vcodec_drv"
> > +#define MTK_VCODEC_ENC_NAME "mtk-vcodec-enc"
> > +
> > +#define MTK_VENC_IRQ_STATUS_SPS 0x1
> > +#define MTK_VENC_IRQ_STATUS_PPS 0x2
> > +#define MTK_VENC_IRQ_STATUS_FRM 0x4
> > +#define MTK_VENC_IRQ_STATUS_DRAM 0x8
> > +#define MTK_VENC_IRQ_STATUS_PAUSE 0x10
> > +#define MTK_VENC_IRQ_STATUS_SWITCH 0x20
>
> Probably better to use BIT(0) .. BIT(5).
Will change to use BIT(0) in next version.
>
>
> > +#define MTK_VENC_IRQ_STATUS_OFFSET 0x05C
> > +#define MTK_VENC_IRQ_ACK_OFFSET 0x060
> > +
> > +#define MTK_VCODEC_MAX_PLANES 3
> > +
> > +#define VDEC_HW_ACTIVE 0x10
> > +#define VDEC_IRQ_CFG 0x11
> > +#define VDEC_IRQ_CLR 0x10
> > +
> > +#define VDEC_IRQ_CFG_REG 0xa4
> > +#define NUM_MAX_ALLOC_CTX 4
> > +#define MTK_V4L2_BENCHMARK 0
> > +#define USE_ENCODE_THREAD 1
> > +
> > +/**
> > + * enum mtk_hw_reg_idx - MTK hw register base index
> > + */
> > +enum mtk_hw_reg_idx {
> > + VDEC_SYS,
> > + VDEC_MISC,
> > + VDEC_LD,
> > + VDEC_TOP,
> > + VDEC_CM,
> > + VDEC_AD,
> > + VDEC_AV,
> > + VDEC_PP,
> > + VDEC_HWD,
> > + VDEC_HWQ,
> > + VDEC_HWB,
> > + VDEC_HWG,
> > + NUM_MAX_VDEC_REG_BASE,
> > + VENC_SYS = NUM_MAX_VDEC_REG_BASE,
> > + VENC_LT_SYS,
> > + NUM_MAX_VCODEC_REG_BASE
> > +};
> > +
> > +/**
> > + * enum mtk_instance_type - The type of an MTK Vcodec instance.
> > + */
> > +enum mtk_instance_type {
> > + MTK_INST_DECODER = 0,
> > + MTK_INST_ENCODER = 1,
> > +};
> > +
> > +/**
> > + * enum mtk_instance_state - The state of an MTK Vcodec instance.
> > + * @MTK_STATE_FREE - default state when instance create
> > + * @MTK_STATE_CREATE - vdec instance is create
> > + * @MTK_STATE_INIT - vdec instance is init
> > + * @MTK_STATE_CONFIG - reserved for encoder
> > + * @MTK_STATE_HEADER - vdec had sps/pps header parsed
> > + * @MTK_STATE_RUNNING - vdec is decoding
> > + * @MTK_STATE_FLUSH - vdec is flushing
> > + * @MTK_STATE_RES_CHANGE - vdec detect resolution change
> > + * @MTK_STATE_FINISH - ctx instance is stopped streaming
> > + * @MTK_STATE_DEINIT - before release ctx instance
> > + * @MTK_STATE_ERROR - vdec has something wrong
> > + * @MTK_STATE_ABORT - abort work in working thread
> > + */
> > +enum mtk_instance_state {
> > + MTK_STATE_FREE = 0,
> > + MTK_STATE_CREATE = (1 << 0),
> > + MTK_STATE_INIT = (1 << 1),
> > + MTK_STATE_CONFIG = (1 << 2),
> > + MTK_STATE_HEADER = (1 << 3),
> > + MTK_STATE_RUNNING = (1 << 4),
> > + MTK_STATE_FLUSH = (1 << 5),
> > + MTK_STATE_RES_CHANGE = (1 << 6),
> > + MTK_STATE_FINISH = (1 << 7),
> > + MTK_STATE_DEINIT = (1 << 8),
> > + MTK_STATE_ERROR = (1 << 9),
> > + MTK_STATE_ABORT = (1 << 10),
>
> This looks like it started as a state machine and somehow turned into
> flags, resulting in a state machine with 2048 states or, to give it a
> different name, a debugging nightmare.
>
It's define some state happened rather than state machine.
Though some states are for v4l2 decoder driver and not used in encoder
driver.
> If the start streaming operation implemented cleanup-on-error properly
> then there would only be two useful states: Started and stopped. Even
> the "sticky" error behavior looks unnecessary to me (meaning we don't
> need to track its state).
>
We cannot guaranteed that IOCTLs called from the user space follow
required sequence.
We need states to know if our driver could accept IOCTL command.
>
> > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
> > new file mode 100644
> > index 0000000..8e1b6f0
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
> > @@ -0,0 +1,1773 @@
> > [...]
> > +static int vb2ops_venc_start_streaming(struct vb2_queue *q, unsigned int count)
> > +{
> > + struct mtk_vcodec_ctx *ctx = vb2_get_drv_priv(q);
> > + struct v4l2_device *v4l2_dev = &ctx->dev->v4l2_dev;
> > + int ret;
> > +#if MTK_V4L2_BENCHMARK
> > + struct timeval begin, end;
> > +
> > + do_gettimeofday(&begin);
> > +#endif
> > +
> > + if (!(vb2_start_streaming_called(&ctx->m2m_ctx->out_q_ctx.q) &
> > + vb2_start_streaming_called(&ctx->m2m_ctx->cap_q_ctx.q))) {
> > + mtk_v4l2_debug(1, "[%d]-> out=%d cap=%d",
> > + ctx->idx,
> > + vb2_start_streaming_called(&ctx->m2m_ctx->out_q_ctx.q),
> > + vb2_start_streaming_called(&ctx->m2m_ctx->cap_q_ctx.q));
> > + return 0;
> > + }
> > +
> > + if ((ctx->state & (MTK_STATE_ERROR | MTK_STATE_ABORT)))
> > + return -EINVAL;
>
> This is the sort of thing I mean.
>
> This sticky error behaviour means that every subsequent call to
> vb2ops_venc_start_streaming() will fail. Note also that the user will
> never try to stop streaming (which can clear the error state) because
> according to the return code it got when it tried to start streaming we
> never actually started.
>
> This is what I mean about having two many states. From the user's
> perspective there are only two states. There needs to be a good reason
> for the driver to manage so many extra secret states internally.
>
For my understanding, that vb2ops_venc_start_streaming cannot fail.
If it fail, user space will close and release this encoder instance
(fd).
We really need to state driver to see what it should do when receive
current IOCTL.
>
> > +
> > + if (ctx->state == MTK_STATE_FREE) {
> > + ret = venc_if_create(ctx,
> > + ctx->q_data[MTK_Q_DATA_DST].fmt->fourcc,
> > + &ctx->h_enc);
> > +
> > + if (ret != 0) {
> > + ctx->state |= MTK_STATE_ERROR;
> > + v4l2_err(v4l2_dev, "invalid codec type=%x\n",
> > + ctx->q_data[MTK_Q_DATA_DST].fmt->fourcc);
> > + v4l2_err(v4l2_dev, "venc_if_create failed=%d\n", ret);
> > + return -EINVAL;
> > + }
> > +
> > + if (ctx->q_data[MTK_Q_DATA_DST].fmt->fourcc ==
> > + V4L2_PIX_FMT_H264)
> > + ctx->hdr = 1;
> > +
> > + ctx->state |= MTK_STATE_CREATE;
> > + }
> > +
> > + if ((ctx->state & MTK_STATE_CREATE) && !(ctx->state & MTK_STATE_INIT)) {
> > + ret = venc_if_init(ctx->h_enc);
> > + if (ret != 0) {
> > + ctx->state |= MTK_STATE_ERROR;
> > + v4l2_err(v4l2_dev, "venc_if_init failed=%d\n", ret);
> > + return -EINVAL;
>
> This error path leaves the encoder partially constructed and relies on
> something else to tidy things up. It would be much better to tidy things
> up from this function and
>
> Also I don't think both venc_if_create and venc_if_init are needed. They
> are only ever called one after the other and thus they only serve to
> complicate the error handling code.
>
venc_if_create is for creating instance in arm side and base on encode
format hook corresponding encoder driver interface.
venc_if_init is trying to init encoder instance in VPU side.
Failures from two functions should have different error handling.
We will enhance this part in next version.
>
> > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
> > new file mode 100644
> > index 0000000..a8e683a
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
> > @@ -0,0 +1,66 @@
> > +/*
> > +* Copyright (c) 2015 MediaTek Inc.
> > +* Author: PC Chen <pc.chen-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
> > +* Tiffany Lin <tiffany.lin-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
> > +*
> > +* This program is free software; you can redistribute it and/or modify
> > +* it under the terms of the GNU General Public License version 2 as
> > +* published by the Free Software Foundation.
> > +*
> > +* This program is distributed in the hope that it will be useful,
> > +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> > +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> > +* GNU General Public License for more details.
> > +*/
> > +
> > +#ifndef _MTK_VCODEC_UTIL_H_
> > +#define _MTK_VCODEC_UTIL_H_
> > +
> > +#include <linux/types.h>
> > +#include <linux/dma-direction.h>
> > +
> > +struct mtk_vcodec_mem {
> > + size_t size;
> > + void *va;
> > + dma_addr_t dma_addr;
> > +};
> > +
> > +extern int mtk_v4l2_dbg_level;
> > +extern bool mtk_vcodec_dbg;
> > +
> > +#define mtk_v4l2_debug(level, fmt, args...) \
> > + do { \
> > + if (mtk_v4l2_dbg_level >= level) \
> > + pr_info("[MTK_V4L2] level=%d %s(),%d: " fmt "\n",\
> > + level, __func__, __LINE__, ##args); \
> > + } while (0)
> > +
> > +#define mtk_v4l2_err(fmt, args...) \
> > + pr_err("[MTK_V4L2][ERROR] %s:%d: " fmt "\n", __func__, __LINE__, \
> > + ##args)
>
> Obviously the code should be structured to make use of dev_dbg/dev_err
> possible.
>
> However where this won't work do you really need special macros for
> this. Assuming your error messages are well written 'git grep' and the
> following should be enough:
>
> #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
>
Thanks.
For pr_err case, we will try to use "#define pr_fmt(fmt) KBUILD_MODNAME
": " fmt" in next version.
For pr_info case, we still need debug level to control output messages.
>
> > +#define mtk_v4l2_debug_enter() mtk_v4l2_debug(5, "+\n")
> > +#define mtk_v4l2_debug_leave() mtk_v4l2_debug(5, "-\n")
>
> Remove these. If you care about function entry and exit for debugging
> you should be able to use ftrace.
>
I am not familiar with ftrace.
What if we only want to trace v4l2 video encoder driver called flow not
called stack? And only for functions we are interested not all
functions.
I will check if it is convenience for us using ftrace.
>
> > +#define mtk_vcodec_debug(h, fmt, args...) \
> > + do { \
> > + if (mtk_vcodec_dbg) \
> > + pr_info("[MTK_VCODEC][%d]: %s() " fmt "\n", \
> > + ((struct mtk_vcodec_ctx *)h->ctx)->idx, \
> > + __func__, ##args); \
> > + } while (0)
> > +
> > +#define mtk_vcodec_err(h, fmt, args...) \
> > + pr_err("[MTK_VCODEC][ERROR][%d]: %s() " fmt "\n", \
> > + ((struct mtk_vcodec_ctx *)h->ctx)->idx, __func__, ##args)
> > +
> > +#define mtk_vcodec_debug_enter(h) mtk_vcodec_debug(h, "+\n")
> > +#define mtk_vcodec_debug_leave(h) mtk_vcodec_debug(h, "-\n")
>
> All above comments apply to these too.
>
best regards,
Tiffany
--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 84+ messages in thread
* [RESEND RFC/PATCH 6/8] media: platform: mtk-vcodec: Add Mediatek V4L2 Video Encoder Driver
@ 2015-11-30 11:39 ` tiffany lin
0 siblings, 0 replies; 84+ messages in thread
From: tiffany lin @ 2015-11-30 11:39 UTC (permalink / raw)
To: linux-arm-kernel
On Fri, 2015-11-27 at 16:34 +0000, Daniel Thompson wrote:
> Hi Tiffany/Andrew
>
> This review is a rather more superficial than my previous one. Mostly
> I'm just commenting on some of the bits I spotted whilst trying to find
> my way around the patchset.
>
> I hope to another more detailed review for v2 (and feel free to add me
> to Cc:).
>
Thanks for your comments.
>
> On 17/11/15 12:54, Tiffany Lin wrote:
> > Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
> > Signed-off-by: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
>
> There is no description of what this patch does. Its not enough to have
> it on the cover letter (because that won't end up in version control).
> You need something here.
>
Got it, We will add description for each patch in next version.
>
> > diff --git a/drivers/media/platform/mtk-vcodec/Kconfig b/drivers/media/platform/mtk-vcodec/Kconfig
> > new file mode 100644
> > index 0000000..1c0b935
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/Kconfig
> > @@ -0,0 +1,5 @@
> > +config MEDIATEK_VPU
> > + bool
> > + ---help---
> > + This driver provides downloading firmware vpu (video processor unit)
> > + and communicating with vpu.
>
> Haven't I seen this before (in patch 3)? Why is it being added to
> another Kconfig file?
>
We will remove this in next version.
>
> > diff --git a/drivers/media/platform/mtk-vcodec/Makefile b/drivers/media/platform/mtk-vcodec/Makefile
> > new file mode 100644
> > index 0000000..c7f7174
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/Makefile
> > @@ -0,0 +1,12 @@
> > +obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += mtk_vcodec_intr.o \
> > + mtk_vcodec_util.o \
> > + mtk_vcodec_enc_drv.o \
> > + mtk_vcodec_enc.o \
> > + mtk_vcodec_enc_pm.o
> > +
> > +obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += common/
> > +
> > +ccflags-y += -I$(srctree)/drivers/media/platform/mtk-vcodec/include \
> > + -I$(srctree)/drivers/media/platform/mtk-vcodec \
> > + -I$(srctree)/drivers/media/platform/mtk-vpu
>
> Seems like there's a lot of directories here. Are these files
> (framework, common, vcodec, etc) so unrelated they really need to live
> in separate directories?
>
> Why not just drivers/media/platform/mediatek?
This is because VPU and Vcodec are two different drivers.
Driver in mtk-vpu is for controlling VPU device and provide
communication API to VPU.
Driver in mtk-vcodec is for control different encoder (vp8, h264), it
include v4l2 driver layer, glue layer between encoders and vp8 and h264
encoder.
>
>
> > diff --git a/drivers/media/platform/mtk-vcodec/common/Makefile b/drivers/media/platform/mtk-vcodec/common/Makefile
> > new file mode 100644
> > index 0000000..477ab80
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/common/Makefile
> > @@ -0,0 +1,8 @@
> > +obj-y += \
> > + venc_drv_if.o
> > +
> > +ccflags-y += \
> > + -I$(srctree)/include/ \
> > + -I$(srctree)/drivers/media/platform/mtk-vcodec \
> > + -I$(srctree)/drivers/media/platform/mtk-vcodec/include \
> > + -I$(srctree)/drivers/media/platform/mtk-vpu
>
> As above, this appears to be a directory to hold just one file.
>
Sorry, I didn't get it. Could you explain more?
>
> > diff --git a/drivers/media/platform/mtk-vcodec/common/venc_drv_if.c
> b/drivers/media/platform/mtk-vcodec/common/venc_drv_if.c
> > new file mode 100644
> > index 0000000..9b3f025
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/common/venc_drv_if.c
> > @@ -0,0 +1,152 @@
> > +/*
> > + * Copyright (c) 2015 MediaTek Inc.
> > + * Author: Daniel Hsiao <daniel.hsiao@mediatek.com>
> > + * Jungchang Tsao <jungchang.tsao@mediatek.com>
> > + *
> > + * This program is free software; you can redistribute it and/or
> > + * modify
> > + * it under the terms of the GNU General Public License version 2 as
> > + * published by the Free Software Foundation.
> > + *
> > + * This program is distributed in the hope that it will be useful,
> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> > + * GNU General Public License for more details.
> > + */
> > +
> > +#include <linux/interrupt.h>
> > +#include <linux/kernel.h>
> > +#include <linux/slab.h>
> > +
> > +#include "mtk_vcodec_drv.h"
> > +#include "mtk_vcodec_enc.h"
> > +#include "mtk_vcodec_pm.h"
> > +#include "mtk_vcodec_util.h"
> > +#include "mtk_vpu_core.h"
> > +
> > +#include "venc_drv_if.h"
> > +#include "venc_drv_base.h"
> > +
> > +
> > +int venc_if_create(void *ctx, unsigned int fourcc, unsigned long
> *handle)
> > +{
> > + struct venc_handle *h;
> > + char str[10];
> > +
> > + mtk_vcodec_fmt2str(fourcc, str);
> > +
> > + h = kzalloc(sizeof(*h), GFP_KERNEL);
> > + if (!h)
> > + return -ENOMEM;
> > +
> > + h->fourcc = fourcc;
> > + h->ctx = ctx;
> > + mtk_vcodec_debug(h, "fmt = %s handle = %p", str, h);
> > +
> > + switch (fourcc) {
> > + default:
> > + mtk_vcodec_err(h, "invalid format %s", str);
> > + goto err_out;
> > + }
> > +
> > + *handle = (unsigned long)h;
> > + return 0;
> > +
> > +err_out:
> > + kfree(h);
> > + return -EINVAL;
> > +}
> > +
> > +int venc_if_init(unsigned long handle)
> > +{
> > + int ret = 0;
> > + struct venc_handle *h = (struct venc_handle *)handle;
> > +
> > + mtk_vcodec_debug_enter(h);
> > +
> > + mtk_venc_lock(h->ctx);
> > + mtk_vcodec_enc_clock_on();
> > + vpu_enable_clock(vpu_get_plat_device(h->ctx->dev->plat_dev));
> > + ret = h->enc_if->init(h->ctx, (unsigned long *)&h->drv_handle);
> > + vpu_disable_clock(vpu_get_plat_device(h->ctx->dev->plat_dev));
> > + mtk_vcodec_enc_clock_off();
> > + mtk_venc_unlock(h->ctx);
> > +
> > + return ret;
> > +}
>
> To me this looks more like an obfuscation layer rather than a
> abstraction layer. I don't understand why we need to hide things from
> the V4L2 implementation that this code forms part of.
>
> More importantly, if this code was included somewhere where it could be
> properly integrated with the device model you might be able to use the
> pm_runtime system to avoid this sort of "heroics" to manage the clocks
> anyway.
>
We want to abstract common part from encoder driver.
Every encoder driver follow same calling flow and only need to take care
about how to communicate with vpu to encode specific format.
Encoder driver do not need to take care clock and multiple instance
issue.
>
> > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
> > new file mode 100644
> > index 0000000..22239f8
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
> > @@ -0,0 +1,441 @@
> > +/*
> > +* Copyright (c) 2015 MediaTek Inc.
> > +* Author: PC Chen <pc.chen@mediatek.com>
> > +* Tiffany Lin <tiffany.lin@mediatek.com>
> > +*
> > +* This program is free software; you can redistribute it and/or modify
> > +* it under the terms of the GNU General Public License version 2 as
> > +* published by the Free Software Foundation.
> > +*
> > +* This program is distributed in the hope that it will be useful,
> > +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> > +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> > +* GNU General Public License for more details.
> > +*/
> > +
> > +#ifndef _MTK_VCODEC_DRV_H_
> > +#define _MTK_VCODEC_DRV_H_
> > +
> > +#include <linux/platform_device.h>
> > +#include <linux/videodev2.h>
> > +#include <media/v4l2-ctrls.h>
> > +#include <media/v4l2-device.h>
> > +#include <media/v4l2-ioctl.h>
> > +#include <media/videobuf2-core.h>
> > +#include <media/videobuf2-v4l2.h>
> > +
> > +#include "venc_drv_if.h"
> > +
> > +#define MTK_VCODEC_MAX_INSTANCES 32
> > +#define MTK_VCODEC_MAX_FRAME_SIZE 0x800000
> > +#define MTK_VIDEO_MAX_FRAME 32
> > +#define MTK_MAX_CTRLS 10
> > +
> > +#define MTK_VCODEC_DRV_NAME "mtk_vcodec_drv"
> > +#define MTK_VCODEC_ENC_NAME "mtk-vcodec-enc"
> > +
> > +#define MTK_VENC_IRQ_STATUS_SPS 0x1
> > +#define MTK_VENC_IRQ_STATUS_PPS 0x2
> > +#define MTK_VENC_IRQ_STATUS_FRM 0x4
> > +#define MTK_VENC_IRQ_STATUS_DRAM 0x8
> > +#define MTK_VENC_IRQ_STATUS_PAUSE 0x10
> > +#define MTK_VENC_IRQ_STATUS_SWITCH 0x20
>
> Probably better to use BIT(0) .. BIT(5).
Will change to use BIT(0) in next version.
>
>
> > +#define MTK_VENC_IRQ_STATUS_OFFSET 0x05C
> > +#define MTK_VENC_IRQ_ACK_OFFSET 0x060
> > +
> > +#define MTK_VCODEC_MAX_PLANES 3
> > +
> > +#define VDEC_HW_ACTIVE 0x10
> > +#define VDEC_IRQ_CFG 0x11
> > +#define VDEC_IRQ_CLR 0x10
> > +
> > +#define VDEC_IRQ_CFG_REG 0xa4
> > +#define NUM_MAX_ALLOC_CTX 4
> > +#define MTK_V4L2_BENCHMARK 0
> > +#define USE_ENCODE_THREAD 1
> > +
> > +/**
> > + * enum mtk_hw_reg_idx - MTK hw register base index
> > + */
> > +enum mtk_hw_reg_idx {
> > + VDEC_SYS,
> > + VDEC_MISC,
> > + VDEC_LD,
> > + VDEC_TOP,
> > + VDEC_CM,
> > + VDEC_AD,
> > + VDEC_AV,
> > + VDEC_PP,
> > + VDEC_HWD,
> > + VDEC_HWQ,
> > + VDEC_HWB,
> > + VDEC_HWG,
> > + NUM_MAX_VDEC_REG_BASE,
> > + VENC_SYS = NUM_MAX_VDEC_REG_BASE,
> > + VENC_LT_SYS,
> > + NUM_MAX_VCODEC_REG_BASE
> > +};
> > +
> > +/**
> > + * enum mtk_instance_type - The type of an MTK Vcodec instance.
> > + */
> > +enum mtk_instance_type {
> > + MTK_INST_DECODER = 0,
> > + MTK_INST_ENCODER = 1,
> > +};
> > +
> > +/**
> > + * enum mtk_instance_state - The state of an MTK Vcodec instance.
> > + * @MTK_STATE_FREE - default state when instance create
> > + * @MTK_STATE_CREATE - vdec instance is create
> > + * @MTK_STATE_INIT - vdec instance is init
> > + * @MTK_STATE_CONFIG - reserved for encoder
> > + * @MTK_STATE_HEADER - vdec had sps/pps header parsed
> > + * @MTK_STATE_RUNNING - vdec is decoding
> > + * @MTK_STATE_FLUSH - vdec is flushing
> > + * @MTK_STATE_RES_CHANGE - vdec detect resolution change
> > + * @MTK_STATE_FINISH - ctx instance is stopped streaming
> > + * @MTK_STATE_DEINIT - before release ctx instance
> > + * @MTK_STATE_ERROR - vdec has something wrong
> > + * @MTK_STATE_ABORT - abort work in working thread
> > + */
> > +enum mtk_instance_state {
> > + MTK_STATE_FREE = 0,
> > + MTK_STATE_CREATE = (1 << 0),
> > + MTK_STATE_INIT = (1 << 1),
> > + MTK_STATE_CONFIG = (1 << 2),
> > + MTK_STATE_HEADER = (1 << 3),
> > + MTK_STATE_RUNNING = (1 << 4),
> > + MTK_STATE_FLUSH = (1 << 5),
> > + MTK_STATE_RES_CHANGE = (1 << 6),
> > + MTK_STATE_FINISH = (1 << 7),
> > + MTK_STATE_DEINIT = (1 << 8),
> > + MTK_STATE_ERROR = (1 << 9),
> > + MTK_STATE_ABORT = (1 << 10),
>
> This looks like it started as a state machine and somehow turned into
> flags, resulting in a state machine with 2048 states or, to give it a
> different name, a debugging nightmare.
>
It's define some state happened rather than state machine.
Though some states are for v4l2 decoder driver and not used in encoder
driver.
> If the start streaming operation implemented cleanup-on-error properly
> then there would only be two useful states: Started and stopped. Even
> the "sticky" error behavior looks unnecessary to me (meaning we don't
> need to track its state).
>
We cannot guaranteed that IOCTLs called from the user space follow
required sequence.
We need states to know if our driver could accept IOCTL command.
>
> > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
> > new file mode 100644
> > index 0000000..8e1b6f0
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
> > @@ -0,0 +1,1773 @@
> > [...]
> > +static int vb2ops_venc_start_streaming(struct vb2_queue *q, unsigned int count)
> > +{
> > + struct mtk_vcodec_ctx *ctx = vb2_get_drv_priv(q);
> > + struct v4l2_device *v4l2_dev = &ctx->dev->v4l2_dev;
> > + int ret;
> > +#if MTK_V4L2_BENCHMARK
> > + struct timeval begin, end;
> > +
> > + do_gettimeofday(&begin);
> > +#endif
> > +
> > + if (!(vb2_start_streaming_called(&ctx->m2m_ctx->out_q_ctx.q) &
> > + vb2_start_streaming_called(&ctx->m2m_ctx->cap_q_ctx.q))) {
> > + mtk_v4l2_debug(1, "[%d]-> out=%d cap=%d",
> > + ctx->idx,
> > + vb2_start_streaming_called(&ctx->m2m_ctx->out_q_ctx.q),
> > + vb2_start_streaming_called(&ctx->m2m_ctx->cap_q_ctx.q));
> > + return 0;
> > + }
> > +
> > + if ((ctx->state & (MTK_STATE_ERROR | MTK_STATE_ABORT)))
> > + return -EINVAL;
>
> This is the sort of thing I mean.
>
> This sticky error behaviour means that every subsequent call to
> vb2ops_venc_start_streaming() will fail. Note also that the user will
> never try to stop streaming (which can clear the error state) because
> according to the return code it got when it tried to start streaming we
> never actually started.
>
> This is what I mean about having two many states. From the user's
> perspective there are only two states. There needs to be a good reason
> for the driver to manage so many extra secret states internally.
>
For my understanding, that vb2ops_venc_start_streaming cannot fail.
If it fail, user space will close and release this encoder instance
(fd).
We really need to state driver to see what it should do when receive
current IOCTL.
>
> > +
> > + if (ctx->state == MTK_STATE_FREE) {
> > + ret = venc_if_create(ctx,
> > + ctx->q_data[MTK_Q_DATA_DST].fmt->fourcc,
> > + &ctx->h_enc);
> > +
> > + if (ret != 0) {
> > + ctx->state |= MTK_STATE_ERROR;
> > + v4l2_err(v4l2_dev, "invalid codec type=%x\n",
> > + ctx->q_data[MTK_Q_DATA_DST].fmt->fourcc);
> > + v4l2_err(v4l2_dev, "venc_if_create failed=%d\n", ret);
> > + return -EINVAL;
> > + }
> > +
> > + if (ctx->q_data[MTK_Q_DATA_DST].fmt->fourcc ==
> > + V4L2_PIX_FMT_H264)
> > + ctx->hdr = 1;
> > +
> > + ctx->state |= MTK_STATE_CREATE;
> > + }
> > +
> > + if ((ctx->state & MTK_STATE_CREATE) && !(ctx->state & MTK_STATE_INIT)) {
> > + ret = venc_if_init(ctx->h_enc);
> > + if (ret != 0) {
> > + ctx->state |= MTK_STATE_ERROR;
> > + v4l2_err(v4l2_dev, "venc_if_init failed=%d\n", ret);
> > + return -EINVAL;
>
> This error path leaves the encoder partially constructed and relies on
> something else to tidy things up. It would be much better to tidy things
> up from this function and
>
> Also I don't think both venc_if_create and venc_if_init are needed. They
> are only ever called one after the other and thus they only serve to
> complicate the error handling code.
>
venc_if_create is for creating instance in arm side and base on encode
format hook corresponding encoder driver interface.
venc_if_init is trying to init encoder instance in VPU side.
Failures from two functions should have different error handling.
We will enhance this part in next version.
>
> > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
> > new file mode 100644
> > index 0000000..a8e683a
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
> > @@ -0,0 +1,66 @@
> > +/*
> > +* Copyright (c) 2015 MediaTek Inc.
> > +* Author: PC Chen <pc.chen@mediatek.com>
> > +* Tiffany Lin <tiffany.lin@mediatek.com>
> > +*
> > +* This program is free software; you can redistribute it and/or modify
> > +* it under the terms of the GNU General Public License version 2 as
> > +* published by the Free Software Foundation.
> > +*
> > +* This program is distributed in the hope that it will be useful,
> > +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> > +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> > +* GNU General Public License for more details.
> > +*/
> > +
> > +#ifndef _MTK_VCODEC_UTIL_H_
> > +#define _MTK_VCODEC_UTIL_H_
> > +
> > +#include <linux/types.h>
> > +#include <linux/dma-direction.h>
> > +
> > +struct mtk_vcodec_mem {
> > + size_t size;
> > + void *va;
> > + dma_addr_t dma_addr;
> > +};
> > +
> > +extern int mtk_v4l2_dbg_level;
> > +extern bool mtk_vcodec_dbg;
> > +
> > +#define mtk_v4l2_debug(level, fmt, args...) \
> > + do { \
> > + if (mtk_v4l2_dbg_level >= level) \
> > + pr_info("[MTK_V4L2] level=%d %s(),%d: " fmt "\n",\
> > + level, __func__, __LINE__, ##args); \
> > + } while (0)
> > +
> > +#define mtk_v4l2_err(fmt, args...) \
> > + pr_err("[MTK_V4L2][ERROR] %s:%d: " fmt "\n", __func__, __LINE__, \
> > + ##args)
>
> Obviously the code should be structured to make use of dev_dbg/dev_err
> possible.
>
> However where this won't work do you really need special macros for
> this. Assuming your error messages are well written 'git grep' and the
> following should be enough:
>
> #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
>
Thanks.
For pr_err case, we will try to use "#define pr_fmt(fmt) KBUILD_MODNAME
": " fmt" in next version.
For pr_info case, we still need debug level to control output messages.
>
> > +#define mtk_v4l2_debug_enter() mtk_v4l2_debug(5, "+\n")
> > +#define mtk_v4l2_debug_leave() mtk_v4l2_debug(5, "-\n")
>
> Remove these. If you care about function entry and exit for debugging
> you should be able to use ftrace.
>
I am not familiar with ftrace.
What if we only want to trace v4l2 video encoder driver called flow not
called stack? And only for functions we are interested not all
functions.
I will check if it is convenience for us using ftrace.
>
> > +#define mtk_vcodec_debug(h, fmt, args...) \
> > + do { \
> > + if (mtk_vcodec_dbg) \
> > + pr_info("[MTK_VCODEC][%d]: %s() " fmt "\n", \
> > + ((struct mtk_vcodec_ctx *)h->ctx)->idx, \
> > + __func__, ##args); \
> > + } while (0)
> > +
> > +#define mtk_vcodec_err(h, fmt, args...) \
> > + pr_err("[MTK_VCODEC][ERROR][%d]: %s() " fmt "\n", \
> > + ((struct mtk_vcodec_ctx *)h->ctx)->idx, __func__, ##args)
> > +
> > +#define mtk_vcodec_debug_enter(h) mtk_vcodec_debug(h, "+\n")
> > +#define mtk_vcodec_debug_leave(h) mtk_vcodec_debug(h, "-\n")
>
> All above comments apply to these too.
>
best regards,
Tiffany
^ permalink raw reply [flat|nested] 84+ messages in thread
* Re: [RESEND RFC/PATCH 3/8] media: platform: mtk-vpu: Support Mediatek VPU
@ 2015-11-30 11:43 ` andrew-ct chen
0 siblings, 0 replies; 84+ messages in thread
From: andrew-ct chen @ 2015-11-30 11:43 UTC (permalink / raw)
To: Daniel Thompson
Cc: Mark Rutland, James Liao, Catalin Marinas, Will Deacon,
Darren Etheridge, Laurent Pinchart, Eddie Huang, Pawel Moll,
Hongzhou Yang, Mauro Carvalho Chehab, Fabien Dessenne,
Peter Griffin, Geert Uytterhoeven, Mikhail Ulyanov, Hans Verkuil,
linux-media, devicetree, Arnd Bergmann, Ian Campbell,
Sascha Hauer, Benoit Parrot, Rob Herring, linux-mediatek,
Yingjoe Chen, Matthias Brugger, Tiffany Lin, linux-arm-kernel,
Daniel Hsiao, linux-kernel, Sakari Ailus, Kumar Gala
On Fri, 2015-11-27 at 12:21 +0000, Daniel Thompson wrote:
> On 27/11/15 12:10, andrew-ct chen wrote:
> >>> +
> >>> > >+ memcpy((void *)send_obj->share_buf, buf, len);
> >>> > >+ send_obj->len = len;
> >>> > >+ send_obj->id = id;
> >>> > >+ vpu_cfg_writel(vpu, 0x1, HOST_TO_VPU);
> >>> > >+
> >>> > >+ /* Wait until VPU receives the command */
> >>> > >+ timeout = jiffies + msecs_to_jiffies(IPI_TIMEOUT_MS);
> >>> > >+ do {
> >>> > >+ if (time_after(jiffies, timeout)) {
> >>> > >+ dev_err(vpu->dev, "vpu_ipi_send: IPI timeout!\n");
> >>> > >+ return -EIO;
> >>> > >+ }
> >>> > >+ } while (vpu_cfg_readl(vpu, HOST_TO_VPU));
> >> >
> >> >Do we need to busy wait every time we communicate with the co-processor?
> >> >Couldn't we put this wait*before* we write to HOST_TO_VPU above.
> >> >
> >> >That way we only spin when there is a need to.
> >> >
> > Since the hardware VPU only allows that one client sends the command to
> > it each time.
> > We need the wait to make sure VPU accepted the command and cleared the
> > interrupt and then the next command would be served.
>
> I understand that the VPU can only have on message outstanding at once.
>
> I just wonder why we busy wait *after* sending the first command rather
> than *before* sending the second one.
No other special reasons. Just send one command and wait until VPU gets
the command. Then, I think this wait also can be put before we write to
HOST_TO_VPU.Is this better than former? May I know the reason?
>
> Streamed decode/encode typically ends up being rate controlled by
> capture or display meaning that in these cases we don't need to busy
> wait at all (because by the time we send the next frame the VPU has
> already accepted the previous message).
For now, only one device "encoder" exists, it is true.
But, we'll have encoder and decoder devices, the decode and encode
requested to VPU are simultaneous.
Is this supposed to be removed for this patches and we can add it back
if the another device(decoder) is ready for review?
Andrew
>
>
> Daniel.
>
>
> _______________________________________________
> Linux-mediatek mailing list
> Linux-mediatek@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-mediatek
^ permalink raw reply [flat|nested] 84+ messages in thread
* Re: [RESEND RFC/PATCH 3/8] media: platform: mtk-vpu: Support Mediatek VPU
@ 2015-11-30 11:43 ` andrew-ct chen
0 siblings, 0 replies; 84+ messages in thread
From: andrew-ct chen @ 2015-11-30 11:43 UTC (permalink / raw)
To: Daniel Thompson
Cc: Mark Rutland, James Liao, Catalin Marinas, Will Deacon,
Darren Etheridge, Laurent Pinchart, Eddie Huang, Pawel Moll,
Hongzhou Yang, Mauro Carvalho Chehab, Fabien Dessenne,
Peter Griffin, Geert Uytterhoeven, Mikhail Ulyanov, Hans Verkuil,
linux-media-u79uwXL29TY76Z2rM5mHXA,
devicetree-u79uwXL29TY76Z2rM5mHXA, Arnd Bergmann, Ian Campbell,
Sascha Hauer, Benoit Parrot, Rob Herring
On Fri, 2015-11-27 at 12:21 +0000, Daniel Thompson wrote:
> On 27/11/15 12:10, andrew-ct chen wrote:
> >>> +
> >>> > >+ memcpy((void *)send_obj->share_buf, buf, len);
> >>> > >+ send_obj->len = len;
> >>> > >+ send_obj->id = id;
> >>> > >+ vpu_cfg_writel(vpu, 0x1, HOST_TO_VPU);
> >>> > >+
> >>> > >+ /* Wait until VPU receives the command */
> >>> > >+ timeout = jiffies + msecs_to_jiffies(IPI_TIMEOUT_MS);
> >>> > >+ do {
> >>> > >+ if (time_after(jiffies, timeout)) {
> >>> > >+ dev_err(vpu->dev, "vpu_ipi_send: IPI timeout!\n");
> >>> > >+ return -EIO;
> >>> > >+ }
> >>> > >+ } while (vpu_cfg_readl(vpu, HOST_TO_VPU));
> >> >
> >> >Do we need to busy wait every time we communicate with the co-processor?
> >> >Couldn't we put this wait*before* we write to HOST_TO_VPU above.
> >> >
> >> >That way we only spin when there is a need to.
> >> >
> > Since the hardware VPU only allows that one client sends the command to
> > it each time.
> > We need the wait to make sure VPU accepted the command and cleared the
> > interrupt and then the next command would be served.
>
> I understand that the VPU can only have on message outstanding at once.
>
> I just wonder why we busy wait *after* sending the first command rather
> than *before* sending the second one.
No other special reasons. Just send one command and wait until VPU gets
the command. Then, I think this wait also can be put before we write to
HOST_TO_VPU.Is this better than former? May I know the reason?
>
> Streamed decode/encode typically ends up being rate controlled by
> capture or display meaning that in these cases we don't need to busy
> wait at all (because by the time we send the next frame the VPU has
> already accepted the previous message).
For now, only one device "encoder" exists, it is true.
But, we'll have encoder and decoder devices, the decode and encode
requested to VPU are simultaneous.
Is this supposed to be removed for this patches and we can add it back
if the another device(decoder) is ready for review?
Andrew
>
>
> Daniel.
>
>
> _______________________________________________
> Linux-mediatek mailing list
> Linux-mediatek-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org
> http://lists.infradead.org/mailman/listinfo/linux-mediatek
--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 84+ messages in thread
* [RESEND RFC/PATCH 3/8] media: platform: mtk-vpu: Support Mediatek VPU
@ 2015-11-30 11:43 ` andrew-ct chen
0 siblings, 0 replies; 84+ messages in thread
From: andrew-ct chen @ 2015-11-30 11:43 UTC (permalink / raw)
To: linux-arm-kernel
On Fri, 2015-11-27 at 12:21 +0000, Daniel Thompson wrote:
> On 27/11/15 12:10, andrew-ct chen wrote:
> >>> +
> >>> > >+ memcpy((void *)send_obj->share_buf, buf, len);
> >>> > >+ send_obj->len = len;
> >>> > >+ send_obj->id = id;
> >>> > >+ vpu_cfg_writel(vpu, 0x1, HOST_TO_VPU);
> >>> > >+
> >>> > >+ /* Wait until VPU receives the command */
> >>> > >+ timeout = jiffies + msecs_to_jiffies(IPI_TIMEOUT_MS);
> >>> > >+ do {
> >>> > >+ if (time_after(jiffies, timeout)) {
> >>> > >+ dev_err(vpu->dev, "vpu_ipi_send: IPI timeout!\n");
> >>> > >+ return -EIO;
> >>> > >+ }
> >>> > >+ } while (vpu_cfg_readl(vpu, HOST_TO_VPU));
> >> >
> >> >Do we need to busy wait every time we communicate with the co-processor?
> >> >Couldn't we put this wait*before* we write to HOST_TO_VPU above.
> >> >
> >> >That way we only spin when there is a need to.
> >> >
> > Since the hardware VPU only allows that one client sends the command to
> > it each time.
> > We need the wait to make sure VPU accepted the command and cleared the
> > interrupt and then the next command would be served.
>
> I understand that the VPU can only have on message outstanding at once.
>
> I just wonder why we busy wait *after* sending the first command rather
> than *before* sending the second one.
No other special reasons. Just send one command and wait until VPU gets
the command. Then, I think this wait also can be put before we write to
HOST_TO_VPU.Is this better than former? May I know the reason?
>
> Streamed decode/encode typically ends up being rate controlled by
> capture or display meaning that in these cases we don't need to busy
> wait at all (because by the time we send the next frame the VPU has
> already accepted the previous message).
For now, only one device "encoder" exists, it is true.
But, we'll have encoder and decoder devices, the decode and encode
requested to VPU are simultaneous.
Is this supposed to be removed for this patches and we can add it back
if the another device(decoder) is ready for review?
Andrew
>
>
> Daniel.
>
>
> _______________________________________________
> Linux-mediatek mailing list
> Linux-mediatek at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-mediatek
^ permalink raw reply [flat|nested] 84+ messages in thread
* Re: [RESEND RFC/PATCH 6/8] media: platform: mtk-vcodec: Add Mediatek V4L2 Video Encoder Driver
2015-11-30 11:39 ` tiffany lin
(?)
(?)
@ 2015-11-30 14:58 ` Daniel Thompson
-1 siblings, 0 replies; 84+ messages in thread
From: Daniel Thompson @ 2015-11-30 14:58 UTC (permalink / raw)
To: tiffany lin
Cc: Rob Herring, Pawel Moll, Mark Rutland, Ian Campbell, Kumar Gala,
Catalin Marinas, Will Deacon, Mauro Carvalho Chehab,
Matthias Brugger, Daniel Kurtz, Sascha Hauer, Hongzhou Yang,
Hans Verkuil, Laurent Pinchart, Sakari Ailus, Geert Uytterhoeven,
Mikhail Ulyanov, Fabien Dessenne, Arnd Bergmann,
Darren Etheridge, Peter Griffin, Benoit Parrot, Andrew-CT Chen,
Eddie Huang, Yingjoe Chen, James Liao, Daniel Hsiao, devicetree,
lkml, linux-arm-kernel, linux-media, linux-mediatek, PoChun.Lin
On 30 November 2015 at 11:39, tiffany lin <tiffany.lin@mediatek.com> wrote:
>> > diff --git a/drivers/media/platform/mtk-vcodec/Makefile b/drivers/media/platform/mtk-vcodec/Makefile
>> > new file mode 100644
>> > index 0000000..c7f7174
>> > --- /dev/null
>> > +++ b/drivers/media/platform/mtk-vcodec/Makefile
>> > @@ -0,0 +1,12 @@
>> > +obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += mtk_vcodec_intr.o \
>> > + mtk_vcodec_util.o \
>> > + mtk_vcodec_enc_drv.o \
>> > + mtk_vcodec_enc.o \
>> > + mtk_vcodec_enc_pm.o
>> > +
>> > +obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += common/
>> > +
>> > +ccflags-y += -I$(srctree)/drivers/media/platform/mtk-vcodec/include \
>> > + -I$(srctree)/drivers/media/platform/mtk-vcodec \
>> > + -I$(srctree)/drivers/media/platform/mtk-vpu
>>
>> Seems like there's a lot of directories here. Are these files
>> (framework, common, vcodec, etc) so unrelated they really need to live
>> in separate directories?
>>
>> Why not just drivers/media/platform/mediatek?
> This is because VPU and Vcodec are two different drivers.
> Driver in mtk-vpu is for controlling VPU device and provide
> communication API to VPU.
> Driver in mtk-vcodec is for control different encoder (vp8, h264), it
> include v4l2 driver layer, glue layer between encoders and vp8 and h264
> encoder.
They may be separate pieces of hardware the drivers for them are very
clearly interlinked. This is obvious because the Makefiles are having
to set ccflags to pick up the headers of the other drivers.
No other V4L2 driver uses ccflags-y in this manner.
>> > diff --git a/drivers/media/platform/mtk-vcodec/common/Makefile b/drivers/media/platform/mtk-vcodec/common/Makefile
>> > new file mode 100644
>> > index 0000000..477ab80
>> > --- /dev/null
>> > +++ b/drivers/media/platform/mtk-vcodec/common/Makefile
>> > @@ -0,0 +1,8 @@
>> > +obj-y += \
>> > + venc_drv_if.o
>> > +
>> > +ccflags-y += \
>> > + -I$(srctree)/include/ \
>> > + -I$(srctree)/drivers/media/platform/mtk-vcodec \
>> > + -I$(srctree)/drivers/media/platform/mtk-vcodec/include \
>> > + -I$(srctree)/drivers/media/platform/mtk-vpu
>>
>> As above, this appears to be a directory to hold just one file.
>>
> Sorry, I didn't get it. Could you explain more?
Just that this is another example of the excessive directory structure.
A directory that contains only one source file is a strong indication
that the splitting of the V4L2 implementation into directories is
excessive.
>> > diff --git a/drivers/media/platform/mtk-vcodec/common/venc_drv_if.c
>> b/drivers/media/platform/mtk-vcodec/common/venc_drv_if.c
>> > new file mode 100644
>> > index 0000000..9b3f025
>> > --- /dev/null
>> > +++ b/drivers/media/platform/mtk-vcodec/common/venc_drv_if.c
>> > @@ -0,0 +1,152 @@
>> > +/*
>> > + * Copyright (c) 2015 MediaTek Inc.
>> > + * Author: Daniel Hsiao <daniel.hsiao@mediatek.com>
>> > + * Jungchang Tsao <jungchang.tsao@mediatek.com>
>> > + *
>> > + * This program is free software; you can redistribute it and/or
>> > + * modify
>> > + * it under the terms of the GNU General Public License version 2 as
>> > + * published by the Free Software Foundation.
>> > + *
>> > + * This program is distributed in the hope that it will be useful,
>> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
>> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
>> > + * GNU General Public License for more details.
>> > + */
>> > +
>> > +#include <linux/interrupt.h>
>> > +#include <linux/kernel.h>
>> > +#include <linux/slab.h>
>> > +
>> > +#include "mtk_vcodec_drv.h"
>> > +#include "mtk_vcodec_enc.h"
>> > +#include "mtk_vcodec_pm.h"
>> > +#include "mtk_vcodec_util.h"
>> > +#include "mtk_vpu_core.h"
>> > +
>> > +#include "venc_drv_if.h"
>> > +#include "venc_drv_base.h"
>> > +
>> > +
>> > +int venc_if_create(void *ctx, unsigned int fourcc, unsigned long
>> *handle)
>> > +{
>> > + struct venc_handle *h;
>> > + char str[10];
>> > +
>> > + mtk_vcodec_fmt2str(fourcc, str);
>> > +
>> > + h = kzalloc(sizeof(*h), GFP_KERNEL);
>> > + if (!h)
>> > + return -ENOMEM;
>> > +
>> > + h->fourcc = fourcc;
>> > + h->ctx = ctx;
>> > + mtk_vcodec_debug(h, "fmt = %s handle = %p", str, h);
>> > +
>> > + switch (fourcc) {
>> > + default:
>> > + mtk_vcodec_err(h, "invalid format %s", str);
>> > + goto err_out;
>> > + }
>> > +
>> > + *handle = (unsigned long)h;
>> > + return 0;
>> > +
>> > +err_out:
>> > + kfree(h);
>> > + return -EINVAL;
>> > +}
>> > +
>> > +int venc_if_init(unsigned long handle)
>> > +{
>> > + int ret = 0;
>> > + struct venc_handle *h = (struct venc_handle *)handle;
>> > +
>> > + mtk_vcodec_debug_enter(h);
>> > +
>> > + mtk_venc_lock(h->ctx);
>> > + mtk_vcodec_enc_clock_on();
>> > + vpu_enable_clock(vpu_get_plat_device(h->ctx->dev->plat_dev));
>> > + ret = h->enc_if->init(h->ctx, (unsigned long *)&h->drv_handle);
>> > + vpu_disable_clock(vpu_get_plat_device(h->ctx->dev->plat_dev));
>> > + mtk_vcodec_enc_clock_off();
>> > + mtk_venc_unlock(h->ctx);
>> > +
>> > + return ret;
>> > +}
>>
>> To me this looks more like an obfuscation layer rather than a
>> abstraction layer. I don't understand why we need to hide things from
>> the V4L2 implementation that this code forms part of.
>>
>> More importantly, if this code was included somewhere where it could be
>> properly integrated with the device model you might be able to use the
>> pm_runtime system to avoid this sort of "heroics" to manage the clocks
>> anyway.
>>
> We want to abstract common part from encoder driver.
> Every encoder driver follow same calling flow and only need to take care
> about how to communicate with vpu to encode specific format.
> Encoder driver do not need to take care clock and multiple instance
> issue.
Looking at each of those stages:
mtk_venc_lock():
Why isn't one of the existing V4L2 locking strategies ok for you?
mtk_vcodec_enc_clock_on():
This does seem like something a sub-driver *should* be doing for itself
vpu_enable_clock():
Why can't the VPU driver manage this internally using pm_runtime?
That is why I described this as an obfuscation layer. It is collecting
a bunch of stuff that can be handled using the kernel driver model and
clumping them together in a special middle layer.
>> > +/**
>> > + * enum mtk_instance_type - The type of an MTK Vcodec instance.
>> > + */
>> > +enum mtk_instance_type {
>> > + MTK_INST_DECODER = 0,
>> > + MTK_INST_ENCODER = 1,
>> > +};
>> > +
>> > +/**
>> > + * enum mtk_instance_state - The state of an MTK Vcodec instance.
>> > + * @MTK_STATE_FREE - default state when instance create
>> > + * @MTK_STATE_CREATE - vdec instance is create
>> > + * @MTK_STATE_INIT - vdec instance is init
>> > + * @MTK_STATE_CONFIG - reserved for encoder
>> > + * @MTK_STATE_HEADER - vdec had sps/pps header parsed
>> > + * @MTK_STATE_RUNNING - vdec is decoding
>> > + * @MTK_STATE_FLUSH - vdec is flushing
>> > + * @MTK_STATE_RES_CHANGE - vdec detect resolution change
>> > + * @MTK_STATE_FINISH - ctx instance is stopped streaming
>> > + * @MTK_STATE_DEINIT - before release ctx instance
>> > + * @MTK_STATE_ERROR - vdec has something wrong
>> > + * @MTK_STATE_ABORT - abort work in working thread
>> > + */
>> > +enum mtk_instance_state {
>> > + MTK_STATE_FREE = 0,
>> > + MTK_STATE_CREATE = (1 << 0),
>> > + MTK_STATE_INIT = (1 << 1),
>> > + MTK_STATE_CONFIG = (1 << 2),
>> > + MTK_STATE_HEADER = (1 << 3),
>> > + MTK_STATE_RUNNING = (1 << 4),
>> > + MTK_STATE_FLUSH = (1 << 5),
>> > + MTK_STATE_RES_CHANGE = (1 << 6),
>> > + MTK_STATE_FINISH = (1 << 7),
>> > + MTK_STATE_DEINIT = (1 << 8),
>> > + MTK_STATE_ERROR = (1 << 9),
>> > + MTK_STATE_ABORT = (1 << 10),
>>
>> This looks like it started as a state machine and somehow turned into
>> flags, resulting in a state machine with 2048 states or, to give it a
>> different name, a debugging nightmare.
>>
> It's define some state happened rather than state machine.
> Though some states are for v4l2 decoder driver and not used in encoder
> driver.
Saying the flags track when "something happened" doesn't stop this
from being an extremely complex (and poorly documented) state machine.
There are way too many states compared to what is needed to implement
V4L2 correctly. To make clear why I am raising this point: with the
current driver state management code it is close to impossible to
properly review the error paths in this driver. The cause of error and
the recovery after error are too decoupled.
>> If the start streaming operation implemented cleanup-on-error properly
>> then there would only be two useful states: Started and stopped. Even
>> the "sticky" error behavior looks unnecessary to me (meaning we don't
>> need to track its state).
>>
> We cannot guaranteed that IOCTLs called from the user space follow
> required sequence.
> We need states to know if our driver could accept IOCTL command.
I believe that knowing whether the streaming is started or stopped
(e.g. two states) is sufficient for a driver to correctly handle
abitrary ioctls from userspace and even then, the core code tracks
this state for you so there's no need for you do it.
The queue/dequeue ioctls succeed or fail based on the length of the
queue (i.e. is the buffer queue overflowing or not) and have no need
to check the streaming state.
If you are absolutely sure that the other states are needed then
please provide an example of an ioctl() sequence where the additional
state is needed.
>> > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
>> > new file mode 100644
>> > index 0000000..8e1b6f0
>> > --- /dev/null
>> > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
>> > @@ -0,0 +1,1773 @@
>> > [...]
>> > +static int vb2ops_venc_start_streaming(struct vb2_queue *q, unsigned int count)
>> > +{
>> > + struct mtk_vcodec_ctx *ctx = vb2_get_drv_priv(q);
>> > + struct v4l2_device *v4l2_dev = &ctx->dev->v4l2_dev;
>> > + int ret;
>> > +#if MTK_V4L2_BENCHMARK
>> > + struct timeval begin, end;
>> > +
>> > + do_gettimeofday(&begin);
>> > +#endif
>> > +
>> > + if (!(vb2_start_streaming_called(&ctx->m2m_ctx->out_q_ctx.q) &
>> > + vb2_start_streaming_called(&ctx->m2m_ctx->cap_q_ctx.q))) {
>> > + mtk_v4l2_debug(1, "[%d]-> out=%d cap=%d",
>> > + ctx->idx,
>> > + vb2_start_streaming_called(&ctx->m2m_ctx->out_q_ctx.q),
>> > + vb2_start_streaming_called(&ctx->m2m_ctx->cap_q_ctx.q));
>> > + return 0;
>> > + }
>> > +
>> > + if ((ctx->state & (MTK_STATE_ERROR | MTK_STATE_ABORT)))
>> > + return -EINVAL;
>>
>> This is the sort of thing I mean.
>>
>> This sticky error behaviour means that every subsequent call to
>> vb2ops_venc_start_streaming() will fail. Note also that the user will
>> never try to stop streaming (which can clear the error state) because
>> according to the return code it got when it tried to start streaming we
>> never actually started.
>>
>> This is what I mean about having two many states. From the user's
>> perspective there are only two states. There needs to be a good reason
>> for the driver to manage so many extra secret states internally.
>>
> For my understanding, that vb2ops_venc_start_streaming cannot fail.
I disagree: See
http://lxr.free-electrons.com/source/include/media/videobuf2-core.h#L288
How did you confirm your understanding before replying?
When this function returns an error the simplest (and easiest to
review) error recovery strategy is simply to undo any actions which
have already been performed (like resource allocation) and return an
error code.#
There is no need for the driver to remember that it has already
reported an error. If the userspace tries again then its OK for us to
fail again.
> If it fail, user space will close and release this encoder instance
> (fd).
The userspace is not required to do this and the driver must not
assume that it will. It could attempt some kind of reconfiguration and
retry.
> We really need to state driver to see what it should do when receive
> current IOCTL.
I think you'll find that the v4l2-core does this for you.
>> > +
>> > + if (ctx->state == MTK_STATE_FREE) {
>> > + ret = venc_if_create(ctx,
>> > + ctx->q_data[MTK_Q_DATA_DST].fmt->fourcc,
>> > + &ctx->h_enc);
>> > +
>> > + if (ret != 0) {
>> > + ctx->state |= MTK_STATE_ERROR;
>> > + v4l2_err(v4l2_dev, "invalid codec type=%x\n",
>> > + ctx->q_data[MTK_Q_DATA_DST].fmt->fourcc);
>> > + v4l2_err(v4l2_dev, "venc_if_create failed=%d\n", ret);
>> > + return -EINVAL;
>> > + }
>> > +
>> > + if (ctx->q_data[MTK_Q_DATA_DST].fmt->fourcc ==
>> > + V4L2_PIX_FMT_H264)
>> > + ctx->hdr = 1;
>> > +
>> > + ctx->state |= MTK_STATE_CREATE;
>> > + }
>> > +
>> > + if ((ctx->state & MTK_STATE_CREATE) && !(ctx->state & MTK_STATE_INIT)) {
>> > + ret = venc_if_init(ctx->h_enc);
>> > + if (ret != 0) {
>> > + ctx->state |= MTK_STATE_ERROR;
>> > + v4l2_err(v4l2_dev, "venc_if_init failed=%d\n", ret);
>> > + return -EINVAL;
>>
>> This error path leaves the encoder partially constructed and relies on
>> something else to tidy things up. It would be much better to tidy things
>> up from this function and
>>
>> Also I don't think both venc_if_create and venc_if_init are needed. They
>> are only ever called one after the other and thus they only serve to
>> complicate the error handling code.
>>
> venc_if_create is for creating instance in arm side and base on encode
> format hook corresponding encoder driver interface.
> venc_if_init is trying to init encoder instance in VPU side.
> Failures from two functions should have different error handling.
> We will enhance this part in next version.
As mentioned above, I'm very uncomfortable about this API in its
entirety and think it should be reconsidered.
So whilst I disagree here (the caller does not have any significant
difference in error handling so using -ENOMEM/-EINVAL/-EIO should be
quite sufficient to distringuish between errors) I rather you spent
some time trying to eliminate this API.
>> > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
>> > new file mode 100644
>> > index 0000000..a8e683a
>> > --- /dev/null
>> > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
>> > @@ -0,0 +1,66 @@
>> > +/*
>> > +* Copyright (c) 2015 MediaTek Inc.
>> > +* Author: PC Chen <pc.chen@mediatek.com>
>> > +* Tiffany Lin <tiffany.lin@mediatek.com>
>> > +*
>> > +* This program is free software; you can redistribute it and/or modify
>> > +* it under the terms of the GNU General Public License version 2 as
>> > +* published by the Free Software Foundation.
>> > +*
>> > +* This program is distributed in the hope that it will be useful,
>> > +* but WITHOUT ANY WARRANTY; without even the implied warranty of
>> > +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
>> > +* GNU General Public License for more details.
>> > +*/
>> > +
>> > +#ifndef _MTK_VCODEC_UTIL_H_
>> > +#define _MTK_VCODEC_UTIL_H_
>> > +
>> > +#include <linux/types.h>
>> > +#include <linux/dma-direction.h>
>> > +
>> > +struct mtk_vcodec_mem {
>> > + size_t size;
>> > + void *va;
>> > + dma_addr_t dma_addr;
>> > +};
>> > +
>> > +extern int mtk_v4l2_dbg_level;
>> > +extern bool mtk_vcodec_dbg;
>> > +
>> > +#define mtk_v4l2_debug(level, fmt, args...) \
>> > + do { \
>> > + if (mtk_v4l2_dbg_level >= level) \
>> > + pr_info("[MTK_V4L2] level=%d %s(),%d: " fmt "\n",\
>> > + level, __func__, __LINE__, ##args); \
>> > + } while (0)
>> > +
>> > +#define mtk_v4l2_err(fmt, args...) \
>> > + pr_err("[MTK_V4L2][ERROR] %s:%d: " fmt "\n", __func__, __LINE__, \
>> > + ##args)
>>
>> Obviously the code should be structured to make use of dev_dbg/dev_err
>> possible.
>>
>> However where this won't work do you really need special macros for
>> this. Assuming your error messages are well written 'git grep' and the
>> following should be enough:
>>
>> #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
>>
> Thanks.
> For pr_err case, we will try to use "#define pr_fmt(fmt) KBUILD_MODNAME
> ": " fmt" in next version.
> For pr_info case, we still need debug level to control output messages.
To be honest I expect new code to be able to rely on -DDEBUG and/or
CONFIG_DYNAMIC_DEBUG.
I really can't see why a single V4L2 driver needs to hand roll a six
level debug message framework. If it really, really, really needs it
then it should at least have the good manners to copy the prior art in
the existing V4L2 drivers.
>
>>
>> > +#define mtk_v4l2_debug_enter() mtk_v4l2_debug(5, "+\n")
>> > +#define mtk_v4l2_debug_leave() mtk_v4l2_debug(5, "-\n")
>>
>> Remove these. If you care about function entry and exit for debugging
>> you should be able to use ftrace.
>>
> I am not familiar with ftrace.
> What if we only want to trace v4l2 video encoder driver called flow not
> called stack? And only for functions we are interested not all
> functions.
> I will check if it is convenience for us using ftrace.
It is find for ftrace to only track a subset of functions.
Daniel.
^ permalink raw reply [flat|nested] 84+ messages in thread
* Re: [RESEND RFC/PATCH 6/8] media: platform: mtk-vcodec: Add Mediatek V4L2 Video Encoder Driver
@ 2015-11-30 14:58 ` Daniel Thompson
0 siblings, 0 replies; 84+ messages in thread
From: Daniel Thompson @ 2015-11-30 14:58 UTC (permalink / raw)
To: tiffany lin
Cc: Rob Herring, Pawel Moll, Mark Rutland, Ian Campbell, Kumar Gala,
Catalin Marinas, Will Deacon, Mauro Carvalho Chehab,
Matthias Brugger, Daniel Kurtz, Sascha Hauer, Hongzhou Yang,
Hans Verkuil, Laurent Pinchart, Sakari Ailus, Geert Uytterhoeven,
Mikhail Ulyanov, Fabien Dessenne, Arnd Bergmann,
Darren Etheridge, Peter Griffin, Benoit Parrot
On 30 November 2015 at 11:39, tiffany lin <tiffany.lin@mediatek.com> wrote:
>> > diff --git a/drivers/media/platform/mtk-vcodec/Makefile b/drivers/media/platform/mtk-vcodec/Makefile
>> > new file mode 100644
>> > index 0000000..c7f7174
>> > --- /dev/null
>> > +++ b/drivers/media/platform/mtk-vcodec/Makefile
>> > @@ -0,0 +1,12 @@
>> > +obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += mtk_vcodec_intr.o \
>> > + mtk_vcodec_util.o \
>> > + mtk_vcodec_enc_drv.o \
>> > + mtk_vcodec_enc.o \
>> > + mtk_vcodec_enc_pm.o
>> > +
>> > +obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += common/
>> > +
>> > +ccflags-y += -I$(srctree)/drivers/media/platform/mtk-vcodec/include \
>> > + -I$(srctree)/drivers/media/platform/mtk-vcodec \
>> > + -I$(srctree)/drivers/media/platform/mtk-vpu
>>
>> Seems like there's a lot of directories here. Are these files
>> (framework, common, vcodec, etc) so unrelated they really need to live
>> in separate directories?
>>
>> Why not just drivers/media/platform/mediatek?
> This is because VPU and Vcodec are two different drivers.
> Driver in mtk-vpu is for controlling VPU device and provide
> communication API to VPU.
> Driver in mtk-vcodec is for control different encoder (vp8, h264), it
> include v4l2 driver layer, glue layer between encoders and vp8 and h264
> encoder.
They may be separate pieces of hardware the drivers for them are very
clearly interlinked. This is obvious because the Makefiles are having
to set ccflags to pick up the headers of the other drivers.
No other V4L2 driver uses ccflags-y in this manner.
>> > diff --git a/drivers/media/platform/mtk-vcodec/common/Makefile b/drivers/media/platform/mtk-vcodec/common/Makefile
>> > new file mode 100644
>> > index 0000000..477ab80
>> > --- /dev/null
>> > +++ b/drivers/media/platform/mtk-vcodec/common/Makefile
>> > @@ -0,0 +1,8 @@
>> > +obj-y += \
>> > + venc_drv_if.o
>> > +
>> > +ccflags-y += \
>> > + -I$(srctree)/include/ \
>> > + -I$(srctree)/drivers/media/platform/mtk-vcodec \
>> > + -I$(srctree)/drivers/media/platform/mtk-vcodec/include \
>> > + -I$(srctree)/drivers/media/platform/mtk-vpu
>>
>> As above, this appears to be a directory to hold just one file.
>>
> Sorry, I didn't get it. Could you explain more?
Just that this is another example of the excessive directory structure.
A directory that contains only one source file is a strong indication
that the splitting of the V4L2 implementation into directories is
excessive.
>> > diff --git a/drivers/media/platform/mtk-vcodec/common/venc_drv_if.c
>> b/drivers/media/platform/mtk-vcodec/common/venc_drv_if.c
>> > new file mode 100644
>> > index 0000000..9b3f025
>> > --- /dev/null
>> > +++ b/drivers/media/platform/mtk-vcodec/common/venc_drv_if.c
>> > @@ -0,0 +1,152 @@
>> > +/*
>> > + * Copyright (c) 2015 MediaTek Inc.
>> > + * Author: Daniel Hsiao <daniel.hsiao@mediatek.com>
>> > + * Jungchang Tsao <jungchang.tsao@mediatek.com>
>> > + *
>> > + * This program is free software; you can redistribute it and/or
>> > + * modify
>> > + * it under the terms of the GNU General Public License version 2 as
>> > + * published by the Free Software Foundation.
>> > + *
>> > + * This program is distributed in the hope that it will be useful,
>> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
>> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
>> > + * GNU General Public License for more details.
>> > + */
>> > +
>> > +#include <linux/interrupt.h>
>> > +#include <linux/kernel.h>
>> > +#include <linux/slab.h>
>> > +
>> > +#include "mtk_vcodec_drv.h"
>> > +#include "mtk_vcodec_enc.h"
>> > +#include "mtk_vcodec_pm.h"
>> > +#include "mtk_vcodec_util.h"
>> > +#include "mtk_vpu_core.h"
>> > +
>> > +#include "venc_drv_if.h"
>> > +#include "venc_drv_base.h"
>> > +
>> > +
>> > +int venc_if_create(void *ctx, unsigned int fourcc, unsigned long
>> *handle)
>> > +{
>> > + struct venc_handle *h;
>> > + char str[10];
>> > +
>> > + mtk_vcodec_fmt2str(fourcc, str);
>> > +
>> > + h = kzalloc(sizeof(*h), GFP_KERNEL);
>> > + if (!h)
>> > + return -ENOMEM;
>> > +
>> > + h->fourcc = fourcc;
>> > + h->ctx = ctx;
>> > + mtk_vcodec_debug(h, "fmt = %s handle = %p", str, h);
>> > +
>> > + switch (fourcc) {
>> > + default:
>> > + mtk_vcodec_err(h, "invalid format %s", str);
>> > + goto err_out;
>> > + }
>> > +
>> > + *handle = (unsigned long)h;
>> > + return 0;
>> > +
>> > +err_out:
>> > + kfree(h);
>> > + return -EINVAL;
>> > +}
>> > +
>> > +int venc_if_init(unsigned long handle)
>> > +{
>> > + int ret = 0;
>> > + struct venc_handle *h = (struct venc_handle *)handle;
>> > +
>> > + mtk_vcodec_debug_enter(h);
>> > +
>> > + mtk_venc_lock(h->ctx);
>> > + mtk_vcodec_enc_clock_on();
>> > + vpu_enable_clock(vpu_get_plat_device(h->ctx->dev->plat_dev));
>> > + ret = h->enc_if->init(h->ctx, (unsigned long *)&h->drv_handle);
>> > + vpu_disable_clock(vpu_get_plat_device(h->ctx->dev->plat_dev));
>> > + mtk_vcodec_enc_clock_off();
>> > + mtk_venc_unlock(h->ctx);
>> > +
>> > + return ret;
>> > +}
>>
>> To me this looks more like an obfuscation layer rather than a
>> abstraction layer. I don't understand why we need to hide things from
>> the V4L2 implementation that this code forms part of.
>>
>> More importantly, if this code was included somewhere where it could be
>> properly integrated with the device model you might be able to use the
>> pm_runtime system to avoid this sort of "heroics" to manage the clocks
>> anyway.
>>
> We want to abstract common part from encoder driver.
> Every encoder driver follow same calling flow and only need to take care
> about how to communicate with vpu to encode specific format.
> Encoder driver do not need to take care clock and multiple instance
> issue.
Looking at each of those stages:
mtk_venc_lock():
Why isn't one of the existing V4L2 locking strategies ok for you?
mtk_vcodec_enc_clock_on():
This does seem like something a sub-driver *should* be doing for itself
vpu_enable_clock():
Why can't the VPU driver manage this internally using pm_runtime?
That is why I described this as an obfuscation layer. It is collecting
a bunch of stuff that can be handled using the kernel driver model and
clumping them together in a special middle layer.
>> > +/**
>> > + * enum mtk_instance_type - The type of an MTK Vcodec instance.
>> > + */
>> > +enum mtk_instance_type {
>> > + MTK_INST_DECODER = 0,
>> > + MTK_INST_ENCODER = 1,
>> > +};
>> > +
>> > +/**
>> > + * enum mtk_instance_state - The state of an MTK Vcodec instance.
>> > + * @MTK_STATE_FREE - default state when instance create
>> > + * @MTK_STATE_CREATE - vdec instance is create
>> > + * @MTK_STATE_INIT - vdec instance is init
>> > + * @MTK_STATE_CONFIG - reserved for encoder
>> > + * @MTK_STATE_HEADER - vdec had sps/pps header parsed
>> > + * @MTK_STATE_RUNNING - vdec is decoding
>> > + * @MTK_STATE_FLUSH - vdec is flushing
>> > + * @MTK_STATE_RES_CHANGE - vdec detect resolution change
>> > + * @MTK_STATE_FINISH - ctx instance is stopped streaming
>> > + * @MTK_STATE_DEINIT - before release ctx instance
>> > + * @MTK_STATE_ERROR - vdec has something wrong
>> > + * @MTK_STATE_ABORT - abort work in working thread
>> > + */
>> > +enum mtk_instance_state {
>> > + MTK_STATE_FREE = 0,
>> > + MTK_STATE_CREATE = (1 << 0),
>> > + MTK_STATE_INIT = (1 << 1),
>> > + MTK_STATE_CONFIG = (1 << 2),
>> > + MTK_STATE_HEADER = (1 << 3),
>> > + MTK_STATE_RUNNING = (1 << 4),
>> > + MTK_STATE_FLUSH = (1 << 5),
>> > + MTK_STATE_RES_CHANGE = (1 << 6),
>> > + MTK_STATE_FINISH = (1 << 7),
>> > + MTK_STATE_DEINIT = (1 << 8),
>> > + MTK_STATE_ERROR = (1 << 9),
>> > + MTK_STATE_ABORT = (1 << 10),
>>
>> This looks like it started as a state machine and somehow turned into
>> flags, resulting in a state machine with 2048 states or, to give it a
>> different name, a debugging nightmare.
>>
> It's define some state happened rather than state machine.
> Though some states are for v4l2 decoder driver and not used in encoder
> driver.
Saying the flags track when "something happened" doesn't stop this
from being an extremely complex (and poorly documented) state machine.
There are way too many states compared to what is needed to implement
V4L2 correctly. To make clear why I am raising this point: with the
current driver state management code it is close to impossible to
properly review the error paths in this driver. The cause of error and
the recovery after error are too decoupled.
>> If the start streaming operation implemented cleanup-on-error properly
>> then there would only be two useful states: Started and stopped. Even
>> the "sticky" error behavior looks unnecessary to me (meaning we don't
>> need to track its state).
>>
> We cannot guaranteed that IOCTLs called from the user space follow
> required sequence.
> We need states to know if our driver could accept IOCTL command.
I believe that knowing whether the streaming is started or stopped
(e.g. two states) is sufficient for a driver to correctly handle
abitrary ioctls from userspace and even then, the core code tracks
this state for you so there's no need for you do it.
The queue/dequeue ioctls succeed or fail based on the length of the
queue (i.e. is the buffer queue overflowing or not) and have no need
to check the streaming state.
If you are absolutely sure that the other states are needed then
please provide an example of an ioctl() sequence where the additional
state is needed.
>> > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
>> > new file mode 100644
>> > index 0000000..8e1b6f0
>> > --- /dev/null
>> > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
>> > @@ -0,0 +1,1773 @@
>> > [...]
>> > +static int vb2ops_venc_start_streaming(struct vb2_queue *q, unsigned int count)
>> > +{
>> > + struct mtk_vcodec_ctx *ctx = vb2_get_drv_priv(q);
>> > + struct v4l2_device *v4l2_dev = &ctx->dev->v4l2_dev;
>> > + int ret;
>> > +#if MTK_V4L2_BENCHMARK
>> > + struct timeval begin, end;
>> > +
>> > + do_gettimeofday(&begin);
>> > +#endif
>> > +
>> > + if (!(vb2_start_streaming_called(&ctx->m2m_ctx->out_q_ctx.q) &
>> > + vb2_start_streaming_called(&ctx->m2m_ctx->cap_q_ctx.q))) {
>> > + mtk_v4l2_debug(1, "[%d]-> out=%d cap=%d",
>> > + ctx->idx,
>> > + vb2_start_streaming_called(&ctx->m2m_ctx->out_q_ctx.q),
>> > + vb2_start_streaming_called(&ctx->m2m_ctx->cap_q_ctx.q));
>> > + return 0;
>> > + }
>> > +
>> > + if ((ctx->state & (MTK_STATE_ERROR | MTK_STATE_ABORT)))
>> > + return -EINVAL;
>>
>> This is the sort of thing I mean.
>>
>> This sticky error behaviour means that every subsequent call to
>> vb2ops_venc_start_streaming() will fail. Note also that the user will
>> never try to stop streaming (which can clear the error state) because
>> according to the return code it got when it tried to start streaming we
>> never actually started.
>>
>> This is what I mean about having two many states. From the user's
>> perspective there are only two states. There needs to be a good reason
>> for the driver to manage so many extra secret states internally.
>>
> For my understanding, that vb2ops_venc_start_streaming cannot fail.
I disagree: See
http://lxr.free-electrons.com/source/include/media/videobuf2-core.h#L288
How did you confirm your understanding before replying?
When this function returns an error the simplest (and easiest to
review) error recovery strategy is simply to undo any actions which
have already been performed (like resource allocation) and return an
error code.#
There is no need for the driver to remember that it has already
reported an error. If the userspace tries again then its OK for us to
fail again.
> If it fail, user space will close and release this encoder instance
> (fd).
The userspace is not required to do this and the driver must not
assume that it will. It could attempt some kind of reconfiguration and
retry.
> We really need to state driver to see what it should do when receive
> current IOCTL.
I think you'll find that the v4l2-core does this for you.
>> > +
>> > + if (ctx->state == MTK_STATE_FREE) {
>> > + ret = venc_if_create(ctx,
>> > + ctx->q_data[MTK_Q_DATA_DST].fmt->fourcc,
>> > + &ctx->h_enc);
>> > +
>> > + if (ret != 0) {
>> > + ctx->state |= MTK_STATE_ERROR;
>> > + v4l2_err(v4l2_dev, "invalid codec type=%x\n",
>> > + ctx->q_data[MTK_Q_DATA_DST].fmt->fourcc);
>> > + v4l2_err(v4l2_dev, "venc_if_create failed=%d\n", ret);
>> > + return -EINVAL;
>> > + }
>> > +
>> > + if (ctx->q_data[MTK_Q_DATA_DST].fmt->fourcc ==
>> > + V4L2_PIX_FMT_H264)
>> > + ctx->hdr = 1;
>> > +
>> > + ctx->state |= MTK_STATE_CREATE;
>> > + }
>> > +
>> > + if ((ctx->state & MTK_STATE_CREATE) && !(ctx->state & MTK_STATE_INIT)) {
>> > + ret = venc_if_init(ctx->h_enc);
>> > + if (ret != 0) {
>> > + ctx->state |= MTK_STATE_ERROR;
>> > + v4l2_err(v4l2_dev, "venc_if_init failed=%d\n", ret);
>> > + return -EINVAL;
>>
>> This error path leaves the encoder partially constructed and relies on
>> something else to tidy things up. It would be much better to tidy things
>> up from this function and
>>
>> Also I don't think both venc_if_create and venc_if_init are needed. They
>> are only ever called one after the other and thus they only serve to
>> complicate the error handling code.
>>
> venc_if_create is for creating instance in arm side and base on encode
> format hook corresponding encoder driver interface.
> venc_if_init is trying to init encoder instance in VPU side.
> Failures from two functions should have different error handling.
> We will enhance this part in next version.
As mentioned above, I'm very uncomfortable about this API in its
entirety and think it should be reconsidered.
So whilst I disagree here (the caller does not have any significant
difference in error handling so using -ENOMEM/-EINVAL/-EIO should be
quite sufficient to distringuish between errors) I rather you spent
some time trying to eliminate this API.
>> > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
>> > new file mode 100644
>> > index 0000000..a8e683a
>> > --- /dev/null
>> > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
>> > @@ -0,0 +1,66 @@
>> > +/*
>> > +* Copyright (c) 2015 MediaTek Inc.
>> > +* Author: PC Chen <pc.chen@mediatek.com>
>> > +* Tiffany Lin <tiffany.lin@mediatek.com>
>> > +*
>> > +* This program is free software; you can redistribute it and/or modify
>> > +* it under the terms of the GNU General Public License version 2 as
>> > +* published by the Free Software Foundation.
>> > +*
>> > +* This program is distributed in the hope that it will be useful,
>> > +* but WITHOUT ANY WARRANTY; without even the implied warranty of
>> > +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
>> > +* GNU General Public License for more details.
>> > +*/
>> > +
>> > +#ifndef _MTK_VCODEC_UTIL_H_
>> > +#define _MTK_VCODEC_UTIL_H_
>> > +
>> > +#include <linux/types.h>
>> > +#include <linux/dma-direction.h>
>> > +
>> > +struct mtk_vcodec_mem {
>> > + size_t size;
>> > + void *va;
>> > + dma_addr_t dma_addr;
>> > +};
>> > +
>> > +extern int mtk_v4l2_dbg_level;
>> > +extern bool mtk_vcodec_dbg;
>> > +
>> > +#define mtk_v4l2_debug(level, fmt, args...) \
>> > + do { \
>> > + if (mtk_v4l2_dbg_level >= level) \
>> > + pr_info("[MTK_V4L2] level=%d %s(),%d: " fmt "\n",\
>> > + level, __func__, __LINE__, ##args); \
>> > + } while (0)
>> > +
>> > +#define mtk_v4l2_err(fmt, args...) \
>> > + pr_err("[MTK_V4L2][ERROR] %s:%d: " fmt "\n", __func__, __LINE__, \
>> > + ##args)
>>
>> Obviously the code should be structured to make use of dev_dbg/dev_err
>> possible.
>>
>> However where this won't work do you really need special macros for
>> this. Assuming your error messages are well written 'git grep' and the
>> following should be enough:
>>
>> #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
>>
> Thanks.
> For pr_err case, we will try to use "#define pr_fmt(fmt) KBUILD_MODNAME
> ": " fmt" in next version.
> For pr_info case, we still need debug level to control output messages.
To be honest I expect new code to be able to rely on -DDEBUG and/or
CONFIG_DYNAMIC_DEBUG.
I really can't see why a single V4L2 driver needs to hand roll a six
level debug message framework. If it really, really, really needs it
then it should at least have the good manners to copy the prior art in
the existing V4L2 drivers.
>
>>
>> > +#define mtk_v4l2_debug_enter() mtk_v4l2_debug(5, "+\n")
>> > +#define mtk_v4l2_debug_leave() mtk_v4l2_debug(5, "-\n")
>>
>> Remove these. If you care about function entry and exit for debugging
>> you should be able to use ftrace.
>>
> I am not familiar with ftrace.
> What if we only want to trace v4l2 video encoder driver called flow not
> called stack? And only for functions we are interested not all
> functions.
> I will check if it is convenience for us using ftrace.
It is find for ftrace to only track a subset of functions.
Daniel.
^ permalink raw reply [flat|nested] 84+ messages in thread
* Re: [RESEND RFC/PATCH 6/8] media: platform: mtk-vcodec: Add Mediatek V4L2 Video Encoder Driver
@ 2015-11-30 14:58 ` Daniel Thompson
0 siblings, 0 replies; 84+ messages in thread
From: Daniel Thompson @ 2015-11-30 14:58 UTC (permalink / raw)
To: tiffany lin
Cc: Rob Herring, Pawel Moll, Mark Rutland, Ian Campbell, Kumar Gala,
Catalin Marinas, Will Deacon, Mauro Carvalho Chehab,
Matthias Brugger, Daniel Kurtz, Sascha Hauer, Hongzhou Yang,
Hans Verkuil, Laurent Pinchart, Sakari Ailus, Geert Uytterhoeven,
Mikhail Ulyanov, Fabien Dessenne, Arnd Bergmann,
Darren Etheridge, Peter Griffin, Benoit Parrot, Andrew-CT Chen,
Eddie Huang, Yingjoe Chen, James Liao, Daniel Hsiao, devicetree,
lkml, linux-arm-kernel, linux-media, linux-mediatek, PoChun.Lin
On 30 November 2015 at 11:39, tiffany lin <tiffany.lin@mediatek.com> wrote:
>> > diff --git a/drivers/media/platform/mtk-vcodec/Makefile b/drivers/media/platform/mtk-vcodec/Makefile
>> > new file mode 100644
>> > index 0000000..c7f7174
>> > --- /dev/null
>> > +++ b/drivers/media/platform/mtk-vcodec/Makefile
>> > @@ -0,0 +1,12 @@
>> > +obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += mtk_vcodec_intr.o \
>> > + mtk_vcodec_util.o \
>> > + mtk_vcodec_enc_drv.o \
>> > + mtk_vcodec_enc.o \
>> > + mtk_vcodec_enc_pm.o
>> > +
>> > +obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += common/
>> > +
>> > +ccflags-y += -I$(srctree)/drivers/media/platform/mtk-vcodec/include \
>> > + -I$(srctree)/drivers/media/platform/mtk-vcodec \
>> > + -I$(srctree)/drivers/media/platform/mtk-vpu
>>
>> Seems like there's a lot of directories here. Are these files
>> (framework, common, vcodec, etc) so unrelated they really need to live
>> in separate directories?
>>
>> Why not just drivers/media/platform/mediatek?
> This is because VPU and Vcodec are two different drivers.
> Driver in mtk-vpu is for controlling VPU device and provide
> communication API to VPU.
> Driver in mtk-vcodec is for control different encoder (vp8, h264), it
> include v4l2 driver layer, glue layer between encoders and vp8 and h264
> encoder.
They may be separate pieces of hardware the drivers for them are very
clearly interlinked. This is obvious because the Makefiles are having
to set ccflags to pick up the headers of the other drivers.
No other V4L2 driver uses ccflags-y in this manner.
>> > diff --git a/drivers/media/platform/mtk-vcodec/common/Makefile b/drivers/media/platform/mtk-vcodec/common/Makefile
>> > new file mode 100644
>> > index 0000000..477ab80
>> > --- /dev/null
>> > +++ b/drivers/media/platform/mtk-vcodec/common/Makefile
>> > @@ -0,0 +1,8 @@
>> > +obj-y += \
>> > + venc_drv_if.o
>> > +
>> > +ccflags-y += \
>> > + -I$(srctree)/include/ \
>> > + -I$(srctree)/drivers/media/platform/mtk-vcodec \
>> > + -I$(srctree)/drivers/media/platform/mtk-vcodec/include \
>> > + -I$(srctree)/drivers/media/platform/mtk-vpu
>>
>> As above, this appears to be a directory to hold just one file.
>>
> Sorry, I didn't get it. Could you explain more?
Just that this is another example of the excessive directory structure.
A directory that contains only one source file is a strong indication
that the splitting of the V4L2 implementation into directories is
excessive.
>> > diff --git a/drivers/media/platform/mtk-vcodec/common/venc_drv_if.c
>> b/drivers/media/platform/mtk-vcodec/common/venc_drv_if.c
>> > new file mode 100644
>> > index 0000000..9b3f025
>> > --- /dev/null
>> > +++ b/drivers/media/platform/mtk-vcodec/common/venc_drv_if.c
>> > @@ -0,0 +1,152 @@
>> > +/*
>> > + * Copyright (c) 2015 MediaTek Inc.
>> > + * Author: Daniel Hsiao <daniel.hsiao@mediatek.com>
>> > + * Jungchang Tsao <jungchang.tsao@mediatek.com>
>> > + *
>> > + * This program is free software; you can redistribute it and/or
>> > + * modify
>> > + * it under the terms of the GNU General Public License version 2 as
>> > + * published by the Free Software Foundation.
>> > + *
>> > + * This program is distributed in the hope that it will be useful,
>> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
>> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
>> > + * GNU General Public License for more details.
>> > + */
>> > +
>> > +#include <linux/interrupt.h>
>> > +#include <linux/kernel.h>
>> > +#include <linux/slab.h>
>> > +
>> > +#include "mtk_vcodec_drv.h"
>> > +#include "mtk_vcodec_enc.h"
>> > +#include "mtk_vcodec_pm.h"
>> > +#include "mtk_vcodec_util.h"
>> > +#include "mtk_vpu_core.h"
>> > +
>> > +#include "venc_drv_if.h"
>> > +#include "venc_drv_base.h"
>> > +
>> > +
>> > +int venc_if_create(void *ctx, unsigned int fourcc, unsigned long
>> *handle)
>> > +{
>> > + struct venc_handle *h;
>> > + char str[10];
>> > +
>> > + mtk_vcodec_fmt2str(fourcc, str);
>> > +
>> > + h = kzalloc(sizeof(*h), GFP_KERNEL);
>> > + if (!h)
>> > + return -ENOMEM;
>> > +
>> > + h->fourcc = fourcc;
>> > + h->ctx = ctx;
>> > + mtk_vcodec_debug(h, "fmt = %s handle = %p", str, h);
>> > +
>> > + switch (fourcc) {
>> > + default:
>> > + mtk_vcodec_err(h, "invalid format %s", str);
>> > + goto err_out;
>> > + }
>> > +
>> > + *handle = (unsigned long)h;
>> > + return 0;
>> > +
>> > +err_out:
>> > + kfree(h);
>> > + return -EINVAL;
>> > +}
>> > +
>> > +int venc_if_init(unsigned long handle)
>> > +{
>> > + int ret = 0;
>> > + struct venc_handle *h = (struct venc_handle *)handle;
>> > +
>> > + mtk_vcodec_debug_enter(h);
>> > +
>> > + mtk_venc_lock(h->ctx);
>> > + mtk_vcodec_enc_clock_on();
>> > + vpu_enable_clock(vpu_get_plat_device(h->ctx->dev->plat_dev));
>> > + ret = h->enc_if->init(h->ctx, (unsigned long *)&h->drv_handle);
>> > + vpu_disable_clock(vpu_get_plat_device(h->ctx->dev->plat_dev));
>> > + mtk_vcodec_enc_clock_off();
>> > + mtk_venc_unlock(h->ctx);
>> > +
>> > + return ret;
>> > +}
>>
>> To me this looks more like an obfuscation layer rather than a
>> abstraction layer. I don't understand why we need to hide things from
>> the V4L2 implementation that this code forms part of.
>>
>> More importantly, if this code was included somewhere where it could be
>> properly integrated with the device model you might be able to use the
>> pm_runtime system to avoid this sort of "heroics" to manage the clocks
>> anyway.
>>
> We want to abstract common part from encoder driver.
> Every encoder driver follow same calling flow and only need to take care
> about how to communicate with vpu to encode specific format.
> Encoder driver do not need to take care clock and multiple instance
> issue.
Looking at each of those stages:
mtk_venc_lock():
Why isn't one of the existing V4L2 locking strategies ok for you?
mtk_vcodec_enc_clock_on():
This does seem like something a sub-driver *should* be doing for itself
vpu_enable_clock():
Why can't the VPU driver manage this internally using pm_runtime?
That is why I described this as an obfuscation layer. It is collecting
a bunch of stuff that can be handled using the kernel driver model and
clumping them together in a special middle layer.
>> > +/**
>> > + * enum mtk_instance_type - The type of an MTK Vcodec instance.
>> > + */
>> > +enum mtk_instance_type {
>> > + MTK_INST_DECODER = 0,
>> > + MTK_INST_ENCODER = 1,
>> > +};
>> > +
>> > +/**
>> > + * enum mtk_instance_state - The state of an MTK Vcodec instance.
>> > + * @MTK_STATE_FREE - default state when instance create
>> > + * @MTK_STATE_CREATE - vdec instance is create
>> > + * @MTK_STATE_INIT - vdec instance is init
>> > + * @MTK_STATE_CONFIG - reserved for encoder
>> > + * @MTK_STATE_HEADER - vdec had sps/pps header parsed
>> > + * @MTK_STATE_RUNNING - vdec is decoding
>> > + * @MTK_STATE_FLUSH - vdec is flushing
>> > + * @MTK_STATE_RES_CHANGE - vdec detect resolution change
>> > + * @MTK_STATE_FINISH - ctx instance is stopped streaming
>> > + * @MTK_STATE_DEINIT - before release ctx instance
>> > + * @MTK_STATE_ERROR - vdec has something wrong
>> > + * @MTK_STATE_ABORT - abort work in working thread
>> > + */
>> > +enum mtk_instance_state {
>> > + MTK_STATE_FREE = 0,
>> > + MTK_STATE_CREATE = (1 << 0),
>> > + MTK_STATE_INIT = (1 << 1),
>> > + MTK_STATE_CONFIG = (1 << 2),
>> > + MTK_STATE_HEADER = (1 << 3),
>> > + MTK_STATE_RUNNING = (1 << 4),
>> > + MTK_STATE_FLUSH = (1 << 5),
>> > + MTK_STATE_RES_CHANGE = (1 << 6),
>> > + MTK_STATE_FINISH = (1 << 7),
>> > + MTK_STATE_DEINIT = (1 << 8),
>> > + MTK_STATE_ERROR = (1 << 9),
>> > + MTK_STATE_ABORT = (1 << 10),
>>
>> This looks like it started as a state machine and somehow turned into
>> flags, resulting in a state machine with 2048 states or, to give it a
>> different name, a debugging nightmare.
>>
> It's define some state happened rather than state machine.
> Though some states are for v4l2 decoder driver and not used in encoder
> driver.
Saying the flags track when "something happened" doesn't stop this
from being an extremely complex (and poorly documented) state machine.
There are way too many states compared to what is needed to implement
V4L2 correctly. To make clear why I am raising this point: with the
current driver state management code it is close to impossible to
properly review the error paths in this driver. The cause of error and
the recovery after error are too decoupled.
>> If the start streaming operation implemented cleanup-on-error properly
>> then there would only be two useful states: Started and stopped. Even
>> the "sticky" error behavior looks unnecessary to me (meaning we don't
>> need to track its state).
>>
> We cannot guaranteed that IOCTLs called from the user space follow
> required sequence.
> We need states to know if our driver could accept IOCTL command.
I believe that knowing whether the streaming is started or stopped
(e.g. two states) is sufficient for a driver to correctly handle
abitrary ioctls from userspace and even then, the core code tracks
this state for you so there's no need for you do it.
The queue/dequeue ioctls succeed or fail based on the length of the
queue (i.e. is the buffer queue overflowing or not) and have no need
to check the streaming state.
If you are absolutely sure that the other states are needed then
please provide an example of an ioctl() sequence where the additional
state is needed.
>> > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
>> > new file mode 100644
>> > index 0000000..8e1b6f0
>> > --- /dev/null
>> > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
>> > @@ -0,0 +1,1773 @@
>> > [...]
>> > +static int vb2ops_venc_start_streaming(struct vb2_queue *q, unsigned int count)
>> > +{
>> > + struct mtk_vcodec_ctx *ctx = vb2_get_drv_priv(q);
>> > + struct v4l2_device *v4l2_dev = &ctx->dev->v4l2_dev;
>> > + int ret;
>> > +#if MTK_V4L2_BENCHMARK
>> > + struct timeval begin, end;
>> > +
>> > + do_gettimeofday(&begin);
>> > +#endif
>> > +
>> > + if (!(vb2_start_streaming_called(&ctx->m2m_ctx->out_q_ctx.q) &
>> > + vb2_start_streaming_called(&ctx->m2m_ctx->cap_q_ctx.q))) {
>> > + mtk_v4l2_debug(1, "[%d]-> out=%d cap=%d",
>> > + ctx->idx,
>> > + vb2_start_streaming_called(&ctx->m2m_ctx->out_q_ctx.q),
>> > + vb2_start_streaming_called(&ctx->m2m_ctx->cap_q_ctx.q));
>> > + return 0;
>> > + }
>> > +
>> > + if ((ctx->state & (MTK_STATE_ERROR | MTK_STATE_ABORT)))
>> > + return -EINVAL;
>>
>> This is the sort of thing I mean.
>>
>> This sticky error behaviour means that every subsequent call to
>> vb2ops_venc_start_streaming() will fail. Note also that the user will
>> never try to stop streaming (which can clear the error state) because
>> according to the return code it got when it tried to start streaming we
>> never actually started.
>>
>> This is what I mean about having two many states. From the user's
>> perspective there are only two states. There needs to be a good reason
>> for the driver to manage so many extra secret states internally.
>>
> For my understanding, that vb2ops_venc_start_streaming cannot fail.
I disagree: See
http://lxr.free-electrons.com/source/include/media/videobuf2-core.h#L288
How did you confirm your understanding before replying?
When this function returns an error the simplest (and easiest to
review) error recovery strategy is simply to undo any actions which
have already been performed (like resource allocation) and return an
error code.#
There is no need for the driver to remember that it has already
reported an error. If the userspace tries again then its OK for us to
fail again.
> If it fail, user space will close and release this encoder instance
> (fd).
The userspace is not required to do this and the driver must not
assume that it will. It could attempt some kind of reconfiguration and
retry.
> We really need to state driver to see what it should do when receive
> current IOCTL.
I think you'll find that the v4l2-core does this for you.
>> > +
>> > + if (ctx->state == MTK_STATE_FREE) {
>> > + ret = venc_if_create(ctx,
>> > + ctx->q_data[MTK_Q_DATA_DST].fmt->fourcc,
>> > + &ctx->h_enc);
>> > +
>> > + if (ret != 0) {
>> > + ctx->state |= MTK_STATE_ERROR;
>> > + v4l2_err(v4l2_dev, "invalid codec type=%x\n",
>> > + ctx->q_data[MTK_Q_DATA_DST].fmt->fourcc);
>> > + v4l2_err(v4l2_dev, "venc_if_create failed=%d\n", ret);
>> > + return -EINVAL;
>> > + }
>> > +
>> > + if (ctx->q_data[MTK_Q_DATA_DST].fmt->fourcc ==
>> > + V4L2_PIX_FMT_H264)
>> > + ctx->hdr = 1;
>> > +
>> > + ctx->state |= MTK_STATE_CREATE;
>> > + }
>> > +
>> > + if ((ctx->state & MTK_STATE_CREATE) && !(ctx->state & MTK_STATE_INIT)) {
>> > + ret = venc_if_init(ctx->h_enc);
>> > + if (ret != 0) {
>> > + ctx->state |= MTK_STATE_ERROR;
>> > + v4l2_err(v4l2_dev, "venc_if_init failed=%d\n", ret);
>> > + return -EINVAL;
>>
>> This error path leaves the encoder partially constructed and relies on
>> something else to tidy things up. It would be much better to tidy things
>> up from this function and
>>
>> Also I don't think both venc_if_create and venc_if_init are needed. They
>> are only ever called one after the other and thus they only serve to
>> complicate the error handling code.
>>
> venc_if_create is for creating instance in arm side and base on encode
> format hook corresponding encoder driver interface.
> venc_if_init is trying to init encoder instance in VPU side.
> Failures from two functions should have different error handling.
> We will enhance this part in next version.
As mentioned above, I'm very uncomfortable about this API in its
entirety and think it should be reconsidered.
So whilst I disagree here (the caller does not have any significant
difference in error handling so using -ENOMEM/-EINVAL/-EIO should be
quite sufficient to distringuish between errors) I rather you spent
some time trying to eliminate this API.
>> > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
>> > new file mode 100644
>> > index 0000000..a8e683a
>> > --- /dev/null
>> > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
>> > @@ -0,0 +1,66 @@
>> > +/*
>> > +* Copyright (c) 2015 MediaTek Inc.
>> > +* Author: PC Chen <pc.chen@mediatek.com>
>> > +* Tiffany Lin <tiffany.lin@mediatek.com>
>> > +*
>> > +* This program is free software; you can redistribute it and/or modify
>> > +* it under the terms of the GNU General Public License version 2 as
>> > +* published by the Free Software Foundation.
>> > +*
>> > +* This program is distributed in the hope that it will be useful,
>> > +* but WITHOUT ANY WARRANTY; without even the implied warranty of
>> > +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
>> > +* GNU General Public License for more details.
>> > +*/
>> > +
>> > +#ifndef _MTK_VCODEC_UTIL_H_
>> > +#define _MTK_VCODEC_UTIL_H_
>> > +
>> > +#include <linux/types.h>
>> > +#include <linux/dma-direction.h>
>> > +
>> > +struct mtk_vcodec_mem {
>> > + size_t size;
>> > + void *va;
>> > + dma_addr_t dma_addr;
>> > +};
>> > +
>> > +extern int mtk_v4l2_dbg_level;
>> > +extern bool mtk_vcodec_dbg;
>> > +
>> > +#define mtk_v4l2_debug(level, fmt, args...) \
>> > + do { \
>> > + if (mtk_v4l2_dbg_level >= level) \
>> > + pr_info("[MTK_V4L2] level=%d %s(),%d: " fmt "\n",\
>> > + level, __func__, __LINE__, ##args); \
>> > + } while (0)
>> > +
>> > +#define mtk_v4l2_err(fmt, args...) \
>> > + pr_err("[MTK_V4L2][ERROR] %s:%d: " fmt "\n", __func__, __LINE__, \
>> > + ##args)
>>
>> Obviously the code should be structured to make use of dev_dbg/dev_err
>> possible.
>>
>> However where this won't work do you really need special macros for
>> this. Assuming your error messages are well written 'git grep' and the
>> following should be enough:
>>
>> #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
>>
> Thanks.
> For pr_err case, we will try to use "#define pr_fmt(fmt) KBUILD_MODNAME
> ": " fmt" in next version.
> For pr_info case, we still need debug level to control output messages.
To be honest I expect new code to be able to rely on -DDEBUG and/or
CONFIG_DYNAMIC_DEBUG.
I really can't see why a single V4L2 driver needs to hand roll a six
level debug message framework. If it really, really, really needs it
then it should at least have the good manners to copy the prior art in
the existing V4L2 drivers.
>
>>
>> > +#define mtk_v4l2_debug_enter() mtk_v4l2_debug(5, "+\n")
>> > +#define mtk_v4l2_debug_leave() mtk_v4l2_debug(5, "-\n")
>>
>> Remove these. If you care about function entry and exit for debugging
>> you should be able to use ftrace.
>>
> I am not familiar with ftrace.
> What if we only want to trace v4l2 video encoder driver called flow not
> called stack? And only for functions we are interested not all
> functions.
> I will check if it is convenience for us using ftrace.
It is find for ftrace to only track a subset of functions.
Daniel.
^ permalink raw reply [flat|nested] 84+ messages in thread
* [RESEND RFC/PATCH 6/8] media: platform: mtk-vcodec: Add Mediatek V4L2 Video Encoder Driver
@ 2015-11-30 14:58 ` Daniel Thompson
0 siblings, 0 replies; 84+ messages in thread
From: Daniel Thompson @ 2015-11-30 14:58 UTC (permalink / raw)
To: linux-arm-kernel
On 30 November 2015 at 11:39, tiffany lin <tiffany.lin@mediatek.com> wrote:
>> > diff --git a/drivers/media/platform/mtk-vcodec/Makefile b/drivers/media/platform/mtk-vcodec/Makefile
>> > new file mode 100644
>> > index 0000000..c7f7174
>> > --- /dev/null
>> > +++ b/drivers/media/platform/mtk-vcodec/Makefile
>> > @@ -0,0 +1,12 @@
>> > +obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += mtk_vcodec_intr.o \
>> > + mtk_vcodec_util.o \
>> > + mtk_vcodec_enc_drv.o \
>> > + mtk_vcodec_enc.o \
>> > + mtk_vcodec_enc_pm.o
>> > +
>> > +obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += common/
>> > +
>> > +ccflags-y += -I$(srctree)/drivers/media/platform/mtk-vcodec/include \
>> > + -I$(srctree)/drivers/media/platform/mtk-vcodec \
>> > + -I$(srctree)/drivers/media/platform/mtk-vpu
>>
>> Seems like there's a lot of directories here. Are these files
>> (framework, common, vcodec, etc) so unrelated they really need to live
>> in separate directories?
>>
>> Why not just drivers/media/platform/mediatek?
> This is because VPU and Vcodec are two different drivers.
> Driver in mtk-vpu is for controlling VPU device and provide
> communication API to VPU.
> Driver in mtk-vcodec is for control different encoder (vp8, h264), it
> include v4l2 driver layer, glue layer between encoders and vp8 and h264
> encoder.
They may be separate pieces of hardware the drivers for them are very
clearly interlinked. This is obvious because the Makefiles are having
to set ccflags to pick up the headers of the other drivers.
No other V4L2 driver uses ccflags-y in this manner.
>> > diff --git a/drivers/media/platform/mtk-vcodec/common/Makefile b/drivers/media/platform/mtk-vcodec/common/Makefile
>> > new file mode 100644
>> > index 0000000..477ab80
>> > --- /dev/null
>> > +++ b/drivers/media/platform/mtk-vcodec/common/Makefile
>> > @@ -0,0 +1,8 @@
>> > +obj-y += \
>> > + venc_drv_if.o
>> > +
>> > +ccflags-y += \
>> > + -I$(srctree)/include/ \
>> > + -I$(srctree)/drivers/media/platform/mtk-vcodec \
>> > + -I$(srctree)/drivers/media/platform/mtk-vcodec/include \
>> > + -I$(srctree)/drivers/media/platform/mtk-vpu
>>
>> As above, this appears to be a directory to hold just one file.
>>
> Sorry, I didn't get it. Could you explain more?
Just that this is another example of the excessive directory structure.
A directory that contains only one source file is a strong indication
that the splitting of the V4L2 implementation into directories is
excessive.
>> > diff --git a/drivers/media/platform/mtk-vcodec/common/venc_drv_if.c
>> b/drivers/media/platform/mtk-vcodec/common/venc_drv_if.c
>> > new file mode 100644
>> > index 0000000..9b3f025
>> > --- /dev/null
>> > +++ b/drivers/media/platform/mtk-vcodec/common/venc_drv_if.c
>> > @@ -0,0 +1,152 @@
>> > +/*
>> > + * Copyright (c) 2015 MediaTek Inc.
>> > + * Author: Daniel Hsiao <daniel.hsiao@mediatek.com>
>> > + * Jungchang Tsao <jungchang.tsao@mediatek.com>
>> > + *
>> > + * This program is free software; you can redistribute it and/or
>> > + * modify
>> > + * it under the terms of the GNU General Public License version 2 as
>> > + * published by the Free Software Foundation.
>> > + *
>> > + * This program is distributed in the hope that it will be useful,
>> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
>> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
>> > + * GNU General Public License for more details.
>> > + */
>> > +
>> > +#include <linux/interrupt.h>
>> > +#include <linux/kernel.h>
>> > +#include <linux/slab.h>
>> > +
>> > +#include "mtk_vcodec_drv.h"
>> > +#include "mtk_vcodec_enc.h"
>> > +#include "mtk_vcodec_pm.h"
>> > +#include "mtk_vcodec_util.h"
>> > +#include "mtk_vpu_core.h"
>> > +
>> > +#include "venc_drv_if.h"
>> > +#include "venc_drv_base.h"
>> > +
>> > +
>> > +int venc_if_create(void *ctx, unsigned int fourcc, unsigned long
>> *handle)
>> > +{
>> > + struct venc_handle *h;
>> > + char str[10];
>> > +
>> > + mtk_vcodec_fmt2str(fourcc, str);
>> > +
>> > + h = kzalloc(sizeof(*h), GFP_KERNEL);
>> > + if (!h)
>> > + return -ENOMEM;
>> > +
>> > + h->fourcc = fourcc;
>> > + h->ctx = ctx;
>> > + mtk_vcodec_debug(h, "fmt = %s handle = %p", str, h);
>> > +
>> > + switch (fourcc) {
>> > + default:
>> > + mtk_vcodec_err(h, "invalid format %s", str);
>> > + goto err_out;
>> > + }
>> > +
>> > + *handle = (unsigned long)h;
>> > + return 0;
>> > +
>> > +err_out:
>> > + kfree(h);
>> > + return -EINVAL;
>> > +}
>> > +
>> > +int venc_if_init(unsigned long handle)
>> > +{
>> > + int ret = 0;
>> > + struct venc_handle *h = (struct venc_handle *)handle;
>> > +
>> > + mtk_vcodec_debug_enter(h);
>> > +
>> > + mtk_venc_lock(h->ctx);
>> > + mtk_vcodec_enc_clock_on();
>> > + vpu_enable_clock(vpu_get_plat_device(h->ctx->dev->plat_dev));
>> > + ret = h->enc_if->init(h->ctx, (unsigned long *)&h->drv_handle);
>> > + vpu_disable_clock(vpu_get_plat_device(h->ctx->dev->plat_dev));
>> > + mtk_vcodec_enc_clock_off();
>> > + mtk_venc_unlock(h->ctx);
>> > +
>> > + return ret;
>> > +}
>>
>> To me this looks more like an obfuscation layer rather than a
>> abstraction layer. I don't understand why we need to hide things from
>> the V4L2 implementation that this code forms part of.
>>
>> More importantly, if this code was included somewhere where it could be
>> properly integrated with the device model you might be able to use the
>> pm_runtime system to avoid this sort of "heroics" to manage the clocks
>> anyway.
>>
> We want to abstract common part from encoder driver.
> Every encoder driver follow same calling flow and only need to take care
> about how to communicate with vpu to encode specific format.
> Encoder driver do not need to take care clock and multiple instance
> issue.
Looking at each of those stages:
mtk_venc_lock():
Why isn't one of the existing V4L2 locking strategies ok for you?
mtk_vcodec_enc_clock_on():
This does seem like something a sub-driver *should* be doing for itself
vpu_enable_clock():
Why can't the VPU driver manage this internally using pm_runtime?
That is why I described this as an obfuscation layer. It is collecting
a bunch of stuff that can be handled using the kernel driver model and
clumping them together in a special middle layer.
>> > +/**
>> > + * enum mtk_instance_type - The type of an MTK Vcodec instance.
>> > + */
>> > +enum mtk_instance_type {
>> > + MTK_INST_DECODER = 0,
>> > + MTK_INST_ENCODER = 1,
>> > +};
>> > +
>> > +/**
>> > + * enum mtk_instance_state - The state of an MTK Vcodec instance.
>> > + * @MTK_STATE_FREE - default state when instance create
>> > + * @MTK_STATE_CREATE - vdec instance is create
>> > + * @MTK_STATE_INIT - vdec instance is init
>> > + * @MTK_STATE_CONFIG - reserved for encoder
>> > + * @MTK_STATE_HEADER - vdec had sps/pps header parsed
>> > + * @MTK_STATE_RUNNING - vdec is decoding
>> > + * @MTK_STATE_FLUSH - vdec is flushing
>> > + * @MTK_STATE_RES_CHANGE - vdec detect resolution change
>> > + * @MTK_STATE_FINISH - ctx instance is stopped streaming
>> > + * @MTK_STATE_DEINIT - before release ctx instance
>> > + * @MTK_STATE_ERROR - vdec has something wrong
>> > + * @MTK_STATE_ABORT - abort work in working thread
>> > + */
>> > +enum mtk_instance_state {
>> > + MTK_STATE_FREE = 0,
>> > + MTK_STATE_CREATE = (1 << 0),
>> > + MTK_STATE_INIT = (1 << 1),
>> > + MTK_STATE_CONFIG = (1 << 2),
>> > + MTK_STATE_HEADER = (1 << 3),
>> > + MTK_STATE_RUNNING = (1 << 4),
>> > + MTK_STATE_FLUSH = (1 << 5),
>> > + MTK_STATE_RES_CHANGE = (1 << 6),
>> > + MTK_STATE_FINISH = (1 << 7),
>> > + MTK_STATE_DEINIT = (1 << 8),
>> > + MTK_STATE_ERROR = (1 << 9),
>> > + MTK_STATE_ABORT = (1 << 10),
>>
>> This looks like it started as a state machine and somehow turned into
>> flags, resulting in a state machine with 2048 states or, to give it a
>> different name, a debugging nightmare.
>>
> It's define some state happened rather than state machine.
> Though some states are for v4l2 decoder driver and not used in encoder
> driver.
Saying the flags track when "something happened" doesn't stop this
from being an extremely complex (and poorly documented) state machine.
There are way too many states compared to what is needed to implement
V4L2 correctly. To make clear why I am raising this point: with the
current driver state management code it is close to impossible to
properly review the error paths in this driver. The cause of error and
the recovery after error are too decoupled.
>> If the start streaming operation implemented cleanup-on-error properly
>> then there would only be two useful states: Started and stopped. Even
>> the "sticky" error behavior looks unnecessary to me (meaning we don't
>> need to track its state).
>>
> We cannot guaranteed that IOCTLs called from the user space follow
> required sequence.
> We need states to know if our driver could accept IOCTL command.
I believe that knowing whether the streaming is started or stopped
(e.g. two states) is sufficient for a driver to correctly handle
abitrary ioctls from userspace and even then, the core code tracks
this state for you so there's no need for you do it.
The queue/dequeue ioctls succeed or fail based on the length of the
queue (i.e. is the buffer queue overflowing or not) and have no need
to check the streaming state.
If you are absolutely sure that the other states are needed then
please provide an example of an ioctl() sequence where the additional
state is needed.
>> > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
>> > new file mode 100644
>> > index 0000000..8e1b6f0
>> > --- /dev/null
>> > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
>> > @@ -0,0 +1,1773 @@
>> > [...]
>> > +static int vb2ops_venc_start_streaming(struct vb2_queue *q, unsigned int count)
>> > +{
>> > + struct mtk_vcodec_ctx *ctx = vb2_get_drv_priv(q);
>> > + struct v4l2_device *v4l2_dev = &ctx->dev->v4l2_dev;
>> > + int ret;
>> > +#if MTK_V4L2_BENCHMARK
>> > + struct timeval begin, end;
>> > +
>> > + do_gettimeofday(&begin);
>> > +#endif
>> > +
>> > + if (!(vb2_start_streaming_called(&ctx->m2m_ctx->out_q_ctx.q) &
>> > + vb2_start_streaming_called(&ctx->m2m_ctx->cap_q_ctx.q))) {
>> > + mtk_v4l2_debug(1, "[%d]-> out=%d cap=%d",
>> > + ctx->idx,
>> > + vb2_start_streaming_called(&ctx->m2m_ctx->out_q_ctx.q),
>> > + vb2_start_streaming_called(&ctx->m2m_ctx->cap_q_ctx.q));
>> > + return 0;
>> > + }
>> > +
>> > + if ((ctx->state & (MTK_STATE_ERROR | MTK_STATE_ABORT)))
>> > + return -EINVAL;
>>
>> This is the sort of thing I mean.
>>
>> This sticky error behaviour means that every subsequent call to
>> vb2ops_venc_start_streaming() will fail. Note also that the user will
>> never try to stop streaming (which can clear the error state) because
>> according to the return code it got when it tried to start streaming we
>> never actually started.
>>
>> This is what I mean about having two many states. From the user's
>> perspective there are only two states. There needs to be a good reason
>> for the driver to manage so many extra secret states internally.
>>
> For my understanding, that vb2ops_venc_start_streaming cannot fail.
I disagree: See
http://lxr.free-electrons.com/source/include/media/videobuf2-core.h#L288
How did you confirm your understanding before replying?
When this function returns an error the simplest (and easiest to
review) error recovery strategy is simply to undo any actions which
have already been performed (like resource allocation) and return an
error code.#
There is no need for the driver to remember that it has already
reported an error. If the userspace tries again then its OK for us to
fail again.
> If it fail, user space will close and release this encoder instance
> (fd).
The userspace is not required to do this and the driver must not
assume that it will. It could attempt some kind of reconfiguration and
retry.
> We really need to state driver to see what it should do when receive
> current IOCTL.
I think you'll find that the v4l2-core does this for you.
>> > +
>> > + if (ctx->state == MTK_STATE_FREE) {
>> > + ret = venc_if_create(ctx,
>> > + ctx->q_data[MTK_Q_DATA_DST].fmt->fourcc,
>> > + &ctx->h_enc);
>> > +
>> > + if (ret != 0) {
>> > + ctx->state |= MTK_STATE_ERROR;
>> > + v4l2_err(v4l2_dev, "invalid codec type=%x\n",
>> > + ctx->q_data[MTK_Q_DATA_DST].fmt->fourcc);
>> > + v4l2_err(v4l2_dev, "venc_if_create failed=%d\n", ret);
>> > + return -EINVAL;
>> > + }
>> > +
>> > + if (ctx->q_data[MTK_Q_DATA_DST].fmt->fourcc ==
>> > + V4L2_PIX_FMT_H264)
>> > + ctx->hdr = 1;
>> > +
>> > + ctx->state |= MTK_STATE_CREATE;
>> > + }
>> > +
>> > + if ((ctx->state & MTK_STATE_CREATE) && !(ctx->state & MTK_STATE_INIT)) {
>> > + ret = venc_if_init(ctx->h_enc);
>> > + if (ret != 0) {
>> > + ctx->state |= MTK_STATE_ERROR;
>> > + v4l2_err(v4l2_dev, "venc_if_init failed=%d\n", ret);
>> > + return -EINVAL;
>>
>> This error path leaves the encoder partially constructed and relies on
>> something else to tidy things up. It would be much better to tidy things
>> up from this function and
>>
>> Also I don't think both venc_if_create and venc_if_init are needed. They
>> are only ever called one after the other and thus they only serve to
>> complicate the error handling code.
>>
> venc_if_create is for creating instance in arm side and base on encode
> format hook corresponding encoder driver interface.
> venc_if_init is trying to init encoder instance in VPU side.
> Failures from two functions should have different error handling.
> We will enhance this part in next version.
As mentioned above, I'm very uncomfortable about this API in its
entirety and think it should be reconsidered.
So whilst I disagree here (the caller does not have any significant
difference in error handling so using -ENOMEM/-EINVAL/-EIO should be
quite sufficient to distringuish between errors) I rather you spent
some time trying to eliminate this API.
>> > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
>> > new file mode 100644
>> > index 0000000..a8e683a
>> > --- /dev/null
>> > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
>> > @@ -0,0 +1,66 @@
>> > +/*
>> > +* Copyright (c) 2015 MediaTek Inc.
>> > +* Author: PC Chen <pc.chen@mediatek.com>
>> > +* Tiffany Lin <tiffany.lin@mediatek.com>
>> > +*
>> > +* This program is free software; you can redistribute it and/or modify
>> > +* it under the terms of the GNU General Public License version 2 as
>> > +* published by the Free Software Foundation.
>> > +*
>> > +* This program is distributed in the hope that it will be useful,
>> > +* but WITHOUT ANY WARRANTY; without even the implied warranty of
>> > +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
>> > +* GNU General Public License for more details.
>> > +*/
>> > +
>> > +#ifndef _MTK_VCODEC_UTIL_H_
>> > +#define _MTK_VCODEC_UTIL_H_
>> > +
>> > +#include <linux/types.h>
>> > +#include <linux/dma-direction.h>
>> > +
>> > +struct mtk_vcodec_mem {
>> > + size_t size;
>> > + void *va;
>> > + dma_addr_t dma_addr;
>> > +};
>> > +
>> > +extern int mtk_v4l2_dbg_level;
>> > +extern bool mtk_vcodec_dbg;
>> > +
>> > +#define mtk_v4l2_debug(level, fmt, args...) \
>> > + do { \
>> > + if (mtk_v4l2_dbg_level >= level) \
>> > + pr_info("[MTK_V4L2] level=%d %s(),%d: " fmt "\n",\
>> > + level, __func__, __LINE__, ##args); \
>> > + } while (0)
>> > +
>> > +#define mtk_v4l2_err(fmt, args...) \
>> > + pr_err("[MTK_V4L2][ERROR] %s:%d: " fmt "\n", __func__, __LINE__, \
>> > + ##args)
>>
>> Obviously the code should be structured to make use of dev_dbg/dev_err
>> possible.
>>
>> However where this won't work do you really need special macros for
>> this. Assuming your error messages are well written 'git grep' and the
>> following should be enough:
>>
>> #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
>>
> Thanks.
> For pr_err case, we will try to use "#define pr_fmt(fmt) KBUILD_MODNAME
> ": " fmt" in next version.
> For pr_info case, we still need debug level to control output messages.
To be honest I expect new code to be able to rely on -DDEBUG and/or
CONFIG_DYNAMIC_DEBUG.
I really can't see why a single V4L2 driver needs to hand roll a six
level debug message framework. If it really, really, really needs it
then it should at least have the good manners to copy the prior art in
the existing V4L2 drivers.
>
>>
>> > +#define mtk_v4l2_debug_enter() mtk_v4l2_debug(5, "+\n")
>> > +#define mtk_v4l2_debug_leave() mtk_v4l2_debug(5, "-\n")
>>
>> Remove these. If you care about function entry and exit for debugging
>> you should be able to use ftrace.
>>
> I am not familiar with ftrace.
> What if we only want to trace v4l2 video encoder driver called flow not
> called stack? And only for functions we are interested not all
> functions.
> I will check if it is convenience for us using ftrace.
It is find for ftrace to only track a subset of functions.
Daniel.
^ permalink raw reply [flat|nested] 84+ messages in thread
* Re: [RESEND RFC/PATCH 3/8] media: platform: mtk-vpu: Support Mediatek VPU
2015-11-30 11:43 ` andrew-ct chen
(?)
(?)
@ 2015-11-30 15:36 ` Daniel Thompson
-1 siblings, 0 replies; 84+ messages in thread
From: Daniel Thompson @ 2015-11-30 15:36 UTC (permalink / raw)
To: andrew-ct chen
Cc: Mark Rutland, James Liao, Catalin Marinas, Will Deacon,
Darren Etheridge, Laurent Pinchart, Eddie Huang, Pawel Moll,
Hongzhou Yang, Mauro Carvalho Chehab, Fabien Dessenne,
Peter Griffin, Geert Uytterhoeven, Mikhail Ulyanov, Hans Verkuil,
linux-media, devicetree, Arnd Bergmann, Ian Campbell,
Sascha Hauer, Benoit Parrot, Rob Herring, linux-mediatek,
Yingjoe Chen, Matthias Brugger, Tiffany Lin, linux-arm-kernel,
Daniel Hsiao, lkml, Sakari Ailus, Kumar Gala
On 30 November 2015 at 11:43, andrew-ct chen
<andrew-ct.chen@mediatek.com> wrote:
> On Fri, 2015-11-27 at 12:21 +0000, Daniel Thompson wrote:
>> On 27/11/15 12:10, andrew-ct chen wrote:
>> >>> +
>> >>> > >+ memcpy((void *)send_obj->share_buf, buf, len);
>> >>> > >+ send_obj->len = len;
>> >>> > >+ send_obj->id = id;
>> >>> > >+ vpu_cfg_writel(vpu, 0x1, HOST_TO_VPU);
>> >>> > >+
>> >>> > >+ /* Wait until VPU receives the command */
>> >>> > >+ timeout = jiffies + msecs_to_jiffies(IPI_TIMEOUT_MS);
>> >>> > >+ do {
>> >>> > >+ if (time_after(jiffies, timeout)) {
>> >>> > >+ dev_err(vpu->dev, "vpu_ipi_send: IPI timeout!\n");
>> >>> > >+ return -EIO;
>> >>> > >+ }
>> >>> > >+ } while (vpu_cfg_readl(vpu, HOST_TO_VPU));
>> >> >
>> >> >Do we need to busy wait every time we communicate with the co-processor?
>> >> >Couldn't we put this wait*before* we write to HOST_TO_VPU above.
>> >> >
>> >> >That way we only spin when there is a need to.
>> >> >
>> > Since the hardware VPU only allows that one client sends the command to
>> > it each time.
>> > We need the wait to make sure VPU accepted the command and cleared the
>> > interrupt and then the next command would be served.
>>
>> I understand that the VPU can only have on message outstanding at once.
>>
>> I just wonder why we busy wait *after* sending the first command rather
>> than *before* sending the second one.
>
> No other special reasons. Just send one command and wait until VPU gets
> the command. Then, I think this wait also can be put before we write to
> HOST_TO_VPU.Is this better than former? May I know the reason?
Busy waiting is bad; it is a waste of host CPU processor time and/or power.
When the busy wait occurs after queuing then we will busy wait for
every command we send.
If busy wait occurs before next queuing then we will wait for a
shorter time in total because we have the chance to do something
useful on the host before we have to wait.
>> Streamed decode/encode typically ends up being rate controlled by
>> capture or display meaning that in these cases we don't need to busy
>> wait at all (because by the time we send the next frame the VPU has
>> already accepted the previous message).
>
> For now, only one device "encoder" exists, it is true.
> But, we'll have encoder and decoder devices, the decode and encode
> requested to VPU are simultaneous.
Sure, I accept that lock and busy-wait is an appropriate way to ensure
safety (assuming the VPU is fairly quick clearing the HOST_TO_VPU
flag).
> Is this supposed to be removed for this patches and we can add it back
> if the another device(decoder) is ready for review?
No I'm not suggesting that.
I only recommend moving the busy wait *before* end sending of command
(for efficiency reasons mentioned above).
Daniel.
^ permalink raw reply [flat|nested] 84+ messages in thread
* Re: [RESEND RFC/PATCH 3/8] media: platform: mtk-vpu: Support Mediatek VPU
@ 2015-11-30 15:36 ` Daniel Thompson
0 siblings, 0 replies; 84+ messages in thread
From: Daniel Thompson @ 2015-11-30 15:36 UTC (permalink / raw)
To: andrew-ct chen
Cc: Mark Rutland, James Liao, Catalin Marinas, Will Deacon,
Darren Etheridge, Laurent Pinchart, Eddie Huang, Pawel Moll,
Hongzhou Yang, Mauro Carvalho Chehab, Fabien Dessenne,
Peter Griffin, Geert Uytterhoeven, Mikhail Ulyanov, Hans Verkuil,
linux-media, devicetree, Arnd Bergmann, Ian Campbell,
Sascha Hauer, Benoit Parrot, Rob
On 30 November 2015 at 11:43, andrew-ct chen
<andrew-ct.chen@mediatek.com> wrote:
> On Fri, 2015-11-27 at 12:21 +0000, Daniel Thompson wrote:
>> On 27/11/15 12:10, andrew-ct chen wrote:
>> >>> +
>> >>> > >+ memcpy((void *)send_obj->share_buf, buf, len);
>> >>> > >+ send_obj->len = len;
>> >>> > >+ send_obj->id = id;
>> >>> > >+ vpu_cfg_writel(vpu, 0x1, HOST_TO_VPU);
>> >>> > >+
>> >>> > >+ /* Wait until VPU receives the command */
>> >>> > >+ timeout = jiffies + msecs_to_jiffies(IPI_TIMEOUT_MS);
>> >>> > >+ do {
>> >>> > >+ if (time_after(jiffies, timeout)) {
>> >>> > >+ dev_err(vpu->dev, "vpu_ipi_send: IPI timeout!\n");
>> >>> > >+ return -EIO;
>> >>> > >+ }
>> >>> > >+ } while (vpu_cfg_readl(vpu, HOST_TO_VPU));
>> >> >
>> >> >Do we need to busy wait every time we communicate with the co-processor?
>> >> >Couldn't we put this wait*before* we write to HOST_TO_VPU above.
>> >> >
>> >> >That way we only spin when there is a need to.
>> >> >
>> > Since the hardware VPU only allows that one client sends the command to
>> > it each time.
>> > We need the wait to make sure VPU accepted the command and cleared the
>> > interrupt and then the next command would be served.
>>
>> I understand that the VPU can only have on message outstanding at once.
>>
>> I just wonder why we busy wait *after* sending the first command rather
>> than *before* sending the second one.
>
> No other special reasons. Just send one command and wait until VPU gets
> the command. Then, I think this wait also can be put before we write to
> HOST_TO_VPU.Is this better than former? May I know the reason?
Busy waiting is bad; it is a waste of host CPU processor time and/or power.
When the busy wait occurs after queuing then we will busy wait for
every command we send.
If busy wait occurs before next queuing then we will wait for a
shorter time in total because we have the chance to do something
useful on the host before we have to wait.
>> Streamed decode/encode typically ends up being rate controlled by
>> capture or display meaning that in these cases we don't need to busy
>> wait at all (because by the time we send the next frame the VPU has
>> already accepted the previous message).
>
> For now, only one device "encoder" exists, it is true.
> But, we'll have encoder and decoder devices, the decode and encode
> requested to VPU are simultaneous.
Sure, I accept that lock and busy-wait is an appropriate way to ensure
safety (assuming the VPU is fairly quick clearing the HOST_TO_VPU
flag).
> Is this supposed to be removed for this patches and we can add it back
> if the another device(decoder) is ready for review?
No I'm not suggesting that.
I only recommend moving the busy wait *before* end sending of command
(for efficiency reasons mentioned above).
Daniel.
^ permalink raw reply [flat|nested] 84+ messages in thread
* Re: [RESEND RFC/PATCH 3/8] media: platform: mtk-vpu: Support Mediatek VPU
@ 2015-11-30 15:36 ` Daniel Thompson
0 siblings, 0 replies; 84+ messages in thread
From: Daniel Thompson @ 2015-11-30 15:36 UTC (permalink / raw)
To: andrew-ct chen
Cc: Mark Rutland, James Liao, Catalin Marinas, Will Deacon,
Darren Etheridge, Laurent Pinchart, Eddie Huang, Pawel Moll,
Hongzhou Yang, Mauro Carvalho Chehab, Fabien Dessenne,
Peter Griffin, Geert Uytterhoeven, Mikhail Ulyanov, Hans Verkuil,
linux-media, devicetree, Arnd Bergmann, Ian Campbell,
Sascha Hauer, Benoit Parrot, Rob Herring, linux-mediatek,
Yingjoe Chen, Matthias Brugger, Tiffany Lin, linux-arm-kernel,
Daniel Hsiao, lkml, Sakari Ailus, Kumar Gala
On 30 November 2015 at 11:43, andrew-ct chen
<andrew-ct.chen@mediatek.com> wrote:
> On Fri, 2015-11-27 at 12:21 +0000, Daniel Thompson wrote:
>> On 27/11/15 12:10, andrew-ct chen wrote:
>> >>> +
>> >>> > >+ memcpy((void *)send_obj->share_buf, buf, len);
>> >>> > >+ send_obj->len = len;
>> >>> > >+ send_obj->id = id;
>> >>> > >+ vpu_cfg_writel(vpu, 0x1, HOST_TO_VPU);
>> >>> > >+
>> >>> > >+ /* Wait until VPU receives the command */
>> >>> > >+ timeout = jiffies + msecs_to_jiffies(IPI_TIMEOUT_MS);
>> >>> > >+ do {
>> >>> > >+ if (time_after(jiffies, timeout)) {
>> >>> > >+ dev_err(vpu->dev, "vpu_ipi_send: IPI timeout!\n");
>> >>> > >+ return -EIO;
>> >>> > >+ }
>> >>> > >+ } while (vpu_cfg_readl(vpu, HOST_TO_VPU));
>> >> >
>> >> >Do we need to busy wait every time we communicate with the co-processor?
>> >> >Couldn't we put this wait*before* we write to HOST_TO_VPU above.
>> >> >
>> >> >That way we only spin when there is a need to.
>> >> >
>> > Since the hardware VPU only allows that one client sends the command to
>> > it each time.
>> > We need the wait to make sure VPU accepted the command and cleared the
>> > interrupt and then the next command would be served.
>>
>> I understand that the VPU can only have on message outstanding at once.
>>
>> I just wonder why we busy wait *after* sending the first command rather
>> than *before* sending the second one.
>
> No other special reasons. Just send one command and wait until VPU gets
> the command. Then, I think this wait also can be put before we write to
> HOST_TO_VPU.Is this better than former? May I know the reason?
Busy waiting is bad; it is a waste of host CPU processor time and/or power.
When the busy wait occurs after queuing then we will busy wait for
every command we send.
If busy wait occurs before next queuing then we will wait for a
shorter time in total because we have the chance to do something
useful on the host before we have to wait.
>> Streamed decode/encode typically ends up being rate controlled by
>> capture or display meaning that in these cases we don't need to busy
>> wait at all (because by the time we send the next frame the VPU has
>> already accepted the previous message).
>
> For now, only one device "encoder" exists, it is true.
> But, we'll have encoder and decoder devices, the decode and encode
> requested to VPU are simultaneous.
Sure, I accept that lock and busy-wait is an appropriate way to ensure
safety (assuming the VPU is fairly quick clearing the HOST_TO_VPU
flag).
> Is this supposed to be removed for this patches and we can add it back
> if the another device(decoder) is ready for review?
No I'm not suggesting that.
I only recommend moving the busy wait *before* end sending of command
(for efficiency reasons mentioned above).
Daniel.
^ permalink raw reply [flat|nested] 84+ messages in thread
* [RESEND RFC/PATCH 3/8] media: platform: mtk-vpu: Support Mediatek VPU
@ 2015-11-30 15:36 ` Daniel Thompson
0 siblings, 0 replies; 84+ messages in thread
From: Daniel Thompson @ 2015-11-30 15:36 UTC (permalink / raw)
To: linux-arm-kernel
On 30 November 2015 at 11:43, andrew-ct chen
<andrew-ct.chen@mediatek.com> wrote:
> On Fri, 2015-11-27 at 12:21 +0000, Daniel Thompson wrote:
>> On 27/11/15 12:10, andrew-ct chen wrote:
>> >>> +
>> >>> > >+ memcpy((void *)send_obj->share_buf, buf, len);
>> >>> > >+ send_obj->len = len;
>> >>> > >+ send_obj->id = id;
>> >>> > >+ vpu_cfg_writel(vpu, 0x1, HOST_TO_VPU);
>> >>> > >+
>> >>> > >+ /* Wait until VPU receives the command */
>> >>> > >+ timeout = jiffies + msecs_to_jiffies(IPI_TIMEOUT_MS);
>> >>> > >+ do {
>> >>> > >+ if (time_after(jiffies, timeout)) {
>> >>> > >+ dev_err(vpu->dev, "vpu_ipi_send: IPI timeout!\n");
>> >>> > >+ return -EIO;
>> >>> > >+ }
>> >>> > >+ } while (vpu_cfg_readl(vpu, HOST_TO_VPU));
>> >> >
>> >> >Do we need to busy wait every time we communicate with the co-processor?
>> >> >Couldn't we put this wait*before* we write to HOST_TO_VPU above.
>> >> >
>> >> >That way we only spin when there is a need to.
>> >> >
>> > Since the hardware VPU only allows that one client sends the command to
>> > it each time.
>> > We need the wait to make sure VPU accepted the command and cleared the
>> > interrupt and then the next command would be served.
>>
>> I understand that the VPU can only have on message outstanding at once.
>>
>> I just wonder why we busy wait *after* sending the first command rather
>> than *before* sending the second one.
>
> No other special reasons. Just send one command and wait until VPU gets
> the command. Then, I think this wait also can be put before we write to
> HOST_TO_VPU.Is this better than former? May I know the reason?
Busy waiting is bad; it is a waste of host CPU processor time and/or power.
When the busy wait occurs after queuing then we will busy wait for
every command we send.
If busy wait occurs before next queuing then we will wait for a
shorter time in total because we have the chance to do something
useful on the host before we have to wait.
>> Streamed decode/encode typically ends up being rate controlled by
>> capture or display meaning that in these cases we don't need to busy
>> wait at all (because by the time we send the next frame the VPU has
>> already accepted the previous message).
>
> For now, only one device "encoder" exists, it is true.
> But, we'll have encoder and decoder devices, the decode and encode
> requested to VPU are simultaneous.
Sure, I accept that lock and busy-wait is an appropriate way to ensure
safety (assuming the VPU is fairly quick clearing the HOST_TO_VPU
flag).
> Is this supposed to be removed for this patches and we can add it back
> if the another device(decoder) is ready for review?
No I'm not suggesting that.
I only recommend moving the busy wait *before* end sending of command
(for efficiency reasons mentioned above).
Daniel.
^ permalink raw reply [flat|nested] 84+ messages in thread
* Re: [RESEND RFC/PATCH 6/8] media: platform: mtk-vcodec: Add Mediatek V4L2 Video Encoder Driver
@ 2015-12-01 10:42 ` tiffany lin
0 siblings, 0 replies; 84+ messages in thread
From: tiffany lin @ 2015-12-01 10:42 UTC (permalink / raw)
To: Daniel Thompson
Cc: Rob Herring, Pawel Moll, Mark Rutland, Ian Campbell, Kumar Gala,
Catalin Marinas, Will Deacon, Mauro Carvalho Chehab,
Matthias Brugger, Daniel Kurtz, Sascha Hauer, Hongzhou Yang,
Hans Verkuil, Laurent Pinchart, Sakari Ailus, Mikhail Ulyanov,
Fabien Dessenne, Arnd Bergmann, Darren Etheridge, Peter Griffin,
Benoit Parrot, Andrew-CT Chen (陳智迪),
Eddie Huang (黃智傑),
Yingjoe Chen (陳英洲),
JamesJJ Liao (廖建智),
Daniel Hsiao (蕭伯剛),
devicetree, lkml, linux-arm-kernel, linux-media, linux-mediatek,
PoChun Lin (林柏君)
On Mon, 2015-11-30 at 22:58 +0800, Daniel Thompson wrote:
> On 30 November 2015 at 11:39, tiffany lin <tiffany.lin@mediatek.com> wrote:
> >> > diff --git a/drivers/media/platform/mtk-vcodec/Makefile b/drivers/media/platform/mtk-vcodec/Makefile
> >> > new file mode 100644
> >> > index 0000000..c7f7174
> >> > --- /dev/null
> >> > +++ b/drivers/media/platform/mtk-vcodec/Makefile
> >> > @@ -0,0 +1,12 @@
> >> > +obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += mtk_vcodec_intr.o \
> >> > + mtk_vcodec_util.o \
> >> > + mtk_vcodec_enc_drv.o \
> >> > + mtk_vcodec_enc.o \
> >> > + mtk_vcodec_enc_pm.o
> >> > +
> >> > +obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += common/
> >> > +
> >> > +ccflags-y += -I$(srctree)/drivers/media/platform/mtk-vcodec/include \
> >> > + -I$(srctree)/drivers/media/platform/mtk-vcodec \
> >> > + -I$(srctree)/drivers/media/platform/mtk-vpu
> >>
> >> Seems like there's a lot of directories here. Are these files
> >> (framework, common, vcodec, etc) so unrelated they really need to live
> >> in separate directories?
> >>
> >> Why not just drivers/media/platform/mediatek?
> > This is because VPU and Vcodec are two different drivers.
> > Driver in mtk-vpu is for controlling VPU device and provide
> > communication API to VPU.
> > Driver in mtk-vcodec is for control different encoder (vp8, h264), it
> > include v4l2 driver layer, glue layer between encoders and vp8 and h264
> > encoder.
>
> They may be separate pieces of hardware the drivers for them are very
> clearly interlinked. This is obvious because the Makefiles are having
> to set ccflags to pick up the headers of the other drivers.
>
> No other V4L2 driver uses ccflags-y in this manner.
>
Got it.
We will remove -I from Makefile and put VPU header file in
include/soc/mediatek
>
> >> > diff --git a/drivers/media/platform/mtk-vcodec/common/Makefile b/drivers/media/platform/mtk-vcodec/common/Makefile
> >> > new file mode 100644
> >> > index 0000000..477ab80
> >> > --- /dev/null
> >> > +++ b/drivers/media/platform/mtk-vcodec/common/Makefile
> >> > @@ -0,0 +1,8 @@
> >> > +obj-y += \
> >> > + venc_drv_if.o
> >> > +
> >> > +ccflags-y += \
> >> > + -I$(srctree)/include/ \
> >> > + -I$(srctree)/drivers/media/platform/mtk-vcodec \
> >> > + -I$(srctree)/drivers/media/platform/mtk-vcodec/include \
> >> > + -I$(srctree)/drivers/media/platform/mtk-vpu
> >>
> >> As above, this appears to be a directory to hold just one file.
> >>
> > Sorry, I didn't get it. Could you explain more?
>
> Just that this is another example of the excessive directory structure.
>
> A directory that contains only one source file is a strong indication
> that the splitting of the V4L2 implementation into directories is
> excessive.
>
The directory that contains only one source file is because now we only
upstream encoder patches. We have decoder patches in future.
We will remove "common", "include" two directories and put files in
mtk-vcodec in next version.
>
> >> > diff --git a/drivers/media/platform/mtk-vcodec/common/venc_drv_if.c
> >> b/drivers/media/platform/mtk-vcodec/common/venc_drv_if.c
> >> > new file mode 100644
> >> > index 0000000..9b3f025
> >> > --- /dev/null
> >> > +++ b/drivers/media/platform/mtk-vcodec/common/venc_drv_if.c
> >> > @@ -0,0 +1,152 @@
> >> > +/*
> >> > + * Copyright (c) 2015 MediaTek Inc.
> >> > + * Author: Daniel Hsiao <daniel.hsiao@mediatek.com>
> >> > + * Jungchang Tsao <jungchang.tsao@mediatek.com>
> >> > + *
> >> > + * This program is free software; you can redistribute it and/or
> >> > + * modify
> >> > + * it under the terms of the GNU General Public License version 2 as
> >> > + * published by the Free Software Foundation.
> >> > + *
> >> > + * This program is distributed in the hope that it will be useful,
> >> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> >> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> >> > + * GNU General Public License for more details.
> >> > + */
> >> > +
> >> > +#include <linux/interrupt.h>
> >> > +#include <linux/kernel.h>
> >> > +#include <linux/slab.h>
> >> > +
> >> > +#include "mtk_vcodec_drv.h"
> >> > +#include "mtk_vcodec_enc.h"
> >> > +#include "mtk_vcodec_pm.h"
> >> > +#include "mtk_vcodec_util.h"
> >> > +#include "mtk_vpu_core.h"
> >> > +
> >> > +#include "venc_drv_if.h"
> >> > +#include "venc_drv_base.h"
> >> > +
> >> > +
> >> > +int venc_if_create(void *ctx, unsigned int fourcc, unsigned long
> >> *handle)
> >> > +{
> >> > + struct venc_handle *h;
> >> > + char str[10];
> >> > +
> >> > + mtk_vcodec_fmt2str(fourcc, str);
> >> > +
> >> > + h = kzalloc(sizeof(*h), GFP_KERNEL);
> >> > + if (!h)
> >> > + return -ENOMEM;
> >> > +
> >> > + h->fourcc = fourcc;
> >> > + h->ctx = ctx;
> >> > + mtk_vcodec_debug(h, "fmt = %s handle = %p", str, h);
> >> > +
> >> > + switch (fourcc) {
> >> > + default:
> >> > + mtk_vcodec_err(h, "invalid format %s", str);
> >> > + goto err_out;
> >> > + }
> >> > +
> >> > + *handle = (unsigned long)h;
> >> > + return 0;
> >> > +
> >> > +err_out:
> >> > + kfree(h);
> >> > + return -EINVAL;
> >> > +}
> >> > +
> >> > +int venc_if_init(unsigned long handle)
> >> > +{
> >> > + int ret = 0;
> >> > + struct venc_handle *h = (struct venc_handle *)handle;
> >> > +
> >> > + mtk_vcodec_debug_enter(h);
> >> > +
> >> > + mtk_venc_lock(h->ctx);
> >> > + mtk_vcodec_enc_clock_on();
> >> > + vpu_enable_clock(vpu_get_plat_device(h->ctx->dev->plat_dev));
> >> > + ret = h->enc_if->init(h->ctx, (unsigned long *)&h->drv_handle);
> >> > + vpu_disable_clock(vpu_get_plat_device(h->ctx->dev->plat_dev));
> >> > + mtk_vcodec_enc_clock_off();
> >> > + mtk_venc_unlock(h->ctx);
> >> > +
> >> > + return ret;
> >> > +}
> >>
> >> To me this looks more like an obfuscation layer rather than a
> >> abstraction layer. I don't understand why we need to hide things from
> >> the V4L2 implementation that this code forms part of.
> >>
> >> More importantly, if this code was included somewhere where it could be
> >> properly integrated with the device model you might be able to use the
> >> pm_runtime system to avoid this sort of "heroics" to manage the clocks
> >> anyway.
> >>
> > We want to abstract common part from encoder driver.
> > Every encoder driver follow same calling flow and only need to take care
> > about how to communicate with vpu to encode specific format.
> > Encoder driver do not need to take care clock and multiple instance
> > issue.
>
> Looking at each of those stages:
>
> mtk_venc_lock():
> Why isn't one of the existing V4L2 locking strategies ok for you?
>
We only has one encoder hw.
To support multiple encode instances.
When one encoder ctx access encoder hw, it need to get lock first.
> mtk_vcodec_enc_clock_on():
> This does seem like something a sub-driver *should* be doing for itself
This is for enabling encoder hw related clock.
To support multiple instances, one encode ctx must get hw lock first
then clock on/off hw relate clock.
> vpu_enable_clock():
> Why can't the VPU driver manage this internally using pm_runtime?
>
Our VPU do not have power domain.
We will remove VPU clock on/off and let vpu control it in next version.
>
> That is why I described this as an obfuscation layer. It is collecting
> a bunch of stuff that can be handled using the kernel driver model and
> clumping them together in a special middle layer.
>
We do use kernel driver model, but we put it in
mtk_vcodec_enc_clock_on/mtk_vcodec_enc_clock_off.
Every sub-driver has no need to write the same code.
And once clock configuration change or porting to other chips, we don't
need to change sub-driver one-by-one, just change abstract layer.
>
> >> > +/**
> >> > + * enum mtk_instance_type - The type of an MTK Vcodec instance.
> >> > + */
> >> > +enum mtk_instance_type {
> >> > + MTK_INST_DECODER = 0,
> >> > + MTK_INST_ENCODER = 1,
> >> > +};
> >> > +
> >> > +/**
> >> > + * enum mtk_instance_state - The state of an MTK Vcodec instance.
> >> > + * @MTK_STATE_FREE - default state when instance create
> >> > + * @MTK_STATE_CREATE - vdec instance is create
> >> > + * @MTK_STATE_INIT - vdec instance is init
> >> > + * @MTK_STATE_CONFIG - reserved for encoder
> >> > + * @MTK_STATE_HEADER - vdec had sps/pps header parsed
> >> > + * @MTK_STATE_RUNNING - vdec is decoding
> >> > + * @MTK_STATE_FLUSH - vdec is flushing
> >> > + * @MTK_STATE_RES_CHANGE - vdec detect resolution change
> >> > + * @MTK_STATE_FINISH - ctx instance is stopped streaming
> >> > + * @MTK_STATE_DEINIT - before release ctx instance
> >> > + * @MTK_STATE_ERROR - vdec has something wrong
> >> > + * @MTK_STATE_ABORT - abort work in working thread
> >> > + */
> >> > +enum mtk_instance_state {
> >> > + MTK_STATE_FREE = 0,
> >> > + MTK_STATE_CREATE = (1 << 0),
> >> > + MTK_STATE_INIT = (1 << 1),
> >> > + MTK_STATE_CONFIG = (1 << 2),
> >> > + MTK_STATE_HEADER = (1 << 3),
> >> > + MTK_STATE_RUNNING = (1 << 4),
> >> > + MTK_STATE_FLUSH = (1 << 5),
> >> > + MTK_STATE_RES_CHANGE = (1 << 6),
> >> > + MTK_STATE_FINISH = (1 << 7),
> >> > + MTK_STATE_DEINIT = (1 << 8),
> >> > + MTK_STATE_ERROR = (1 << 9),
> >> > + MTK_STATE_ABORT = (1 << 10),
> >>
> >> This looks like it started as a state machine and somehow turned into
> >> flags, resulting in a state machine with 2048 states or, to give it a
> >> different name, a debugging nightmare.
> >>
> > It's define some state happened rather than state machine.
> > Though some states are for v4l2 decoder driver and not used in encoder
> > driver.
>
> Saying the flags track when "something happened" doesn't stop this
> from being an extremely complex (and poorly documented) state machine.
>
> There are way too many states compared to what is needed to implement
> V4L2 correctly. To make clear why I am raising this point: with the
> current driver state management code it is close to impossible to
> properly review the error paths in this driver. The cause of error and
> the recovery after error are too decoupled.
Some state defines are shared with decoder that we will upstream in
future.
We will review if each state is really needed and reduce state that
encoder need in next version.
>
> >> If the start streaming operation implemented cleanup-on-error properly
> >> then there would only be two useful states: Started and stopped. Even
> >> the "sticky" error behavior looks unnecessary to me (meaning we don't
> >> need to track its state).
> >>
> > We cannot guaranteed that IOCTLs called from the user space follow
> > required sequence.
> > We need states to know if our driver could accept IOCTL command.
>
> I believe that knowing whether the streaming is started or stopped
> (e.g. two states) is sufficient for a driver to correctly handle
> abitrary ioctls from userspace and even then, the core code tracks
> this state for you so there's no need for you do it.
>
> The queue/dequeue ioctls succeed or fail based on the length of the
> queue (i.e. is the buffer queue overflowing or not) and have no need
> to check the streaming state.
> If you are absolutely sure that the other states are needed then
> please provide an example of an ioctl() sequence where the additional
> state is needed.
>
I know your point that we have too many state changes in start_streaming
and stop_streaming function.
We will refine these two functions in next version.
For the example, we need MTK_STATE_HEADER state, to make sure before
encode start, driver already get information to set encode parameters.
We need MTK_STATE_ABORT to inform encoder thread (mtk_venc_worker) that
stop encodeing job from stopped ctx instance.
When user space qbuf, we need to make sure everything is ready to sent
buf to encode.
>
> >> > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
> >> > new file mode 100644
> >> > index 0000000..8e1b6f0
> >> > --- /dev/null
> >> > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
> >> > @@ -0,0 +1,1773 @@
> >> > [...]
> >> > +static int vb2ops_venc_start_streaming(struct vb2_queue *q, unsigned int count)
> >> > +{
> >> > + struct mtk_vcodec_ctx *ctx = vb2_get_drv_priv(q);
> >> > + struct v4l2_device *v4l2_dev = &ctx->dev->v4l2_dev;
> >> > + int ret;
> >> > +#if MTK_V4L2_BENCHMARK
> >> > + struct timeval begin, end;
> >> > +
> >> > + do_gettimeofday(&begin);
> >> > +#endif
> >> > +
> >> > + if (!(vb2_start_streaming_called(&ctx->m2m_ctx->out_q_ctx.q) &
> >> > + vb2_start_streaming_called(&ctx->m2m_ctx->cap_q_ctx.q))) {
> >> > + mtk_v4l2_debug(1, "[%d]-> out=%d cap=%d",
> >> > + ctx->idx,
> >> > + vb2_start_streaming_called(&ctx->m2m_ctx->out_q_ctx.q),
> >> > + vb2_start_streaming_called(&ctx->m2m_ctx->cap_q_ctx.q));
> >> > + return 0;
> >> > + }
> >> > +
> >> > + if ((ctx->state & (MTK_STATE_ERROR | MTK_STATE_ABORT)))
> >> > + return -EINVAL;
> >>
> >> This is the sort of thing I mean.
> >>
> >> This sticky error behaviour means that every subsequent call to
> >> vb2ops_venc_start_streaming() will fail. Note also that the user will
> >> never try to stop streaming (which can clear the error state) because
> >> according to the return code it got when it tried to start streaming we
> >> never actually started.
> >>
> >> This is what I mean about having two many states. From the user's
> >> perspective there are only two states. There needs to be a good reason
> >> for the driver to manage so many extra secret states internally.
> >>
> > For my understanding, that vb2ops_venc_start_streaming cannot fail.
>
> I disagree: See
> http://lxr.free-electrons.com/source/include/media/videobuf2-core.h#L288
>
> How did you confirm your understanding before replying?
>
Sorry that I did not explain well.
What I want to said about "cannot fail" is that once start streaming
fail, all subsequent calls will fail.
The only recover step is close this instance and open instance again.
> When this function returns an error the simplest (and easiest to
> review) error recovery strategy is simply to undo any actions which
> have already been performed (like resource allocation) and return an
> error code.#
>
> There is no need for the driver to remember that it has already
> reported an error. If the userspace tries again then its OK for us to
> fail again.
>
Our original though is that when start_streaming fail, user space will
close this instance. And if user space called start_streaming again, it
just return fail.
I got what you means now, we will try to reduce state changes and remove
state check in start_streaming/stop_streaming in next version.
>
> > If it fail, user space will close and release this encoder instance
> > (fd).
>
> The userspace is not required to do this and the driver must not
> assume that it will. It could attempt some kind of reconfiguration and
> retry.
>
>
> > We really need to state driver to see what it should do when receive
> > current IOCTL.
>
> I think you'll find that the v4l2-core does this for you.
>
We will review each state and make sure if it really need for encoder in
next version.
>
> >> > +
> >> > + if (ctx->state == MTK_STATE_FREE) {
> >> > + ret = venc_if_create(ctx,
> >> > + ctx->q_data[MTK_Q_DATA_DST].fmt->fourcc,
> >> > + &ctx->h_enc);
> >> > +
> >> > + if (ret != 0) {
> >> > + ctx->state |= MTK_STATE_ERROR;
> >> > + v4l2_err(v4l2_dev, "invalid codec type=%x\n",
> >> > + ctx->q_data[MTK_Q_DATA_DST].fmt->fourcc);
> >> > + v4l2_err(v4l2_dev, "venc_if_create failed=%d\n", ret);
> >> > + return -EINVAL;
> >> > + }
> >> > +
> >> > + if (ctx->q_data[MTK_Q_DATA_DST].fmt->fourcc ==
> >> > + V4L2_PIX_FMT_H264)
> >> > + ctx->hdr = 1;
> >> > +
> >> > + ctx->state |= MTK_STATE_CREATE;
> >> > + }
> >> > +
> >> > + if ((ctx->state & MTK_STATE_CREATE) && !(ctx->state & MTK_STATE_INIT)) {
> >> > + ret = venc_if_init(ctx->h_enc);
> >> > + if (ret != 0) {
> >> > + ctx->state |= MTK_STATE_ERROR;
> >> > + v4l2_err(v4l2_dev, "venc_if_init failed=%d\n", ret);
> >> > + return -EINVAL;
> >>
> >> This error path leaves the encoder partially constructed and relies on
> >> something else to tidy things up. It would be much better to tidy things
> >> up from this function and
> >>
> >> Also I don't think both venc_if_create and venc_if_init are needed. They
> >> are only ever called one after the other and thus they only serve to
> >> complicate the error handling code.
> >>
> > venc_if_create is for creating instance in arm side and base on encode
> > format hook corresponding encoder driver interface.
> > venc_if_init is trying to init encoder instance in VPU side.
> > Failures from two functions should have different error handling.
> > We will enhance this part in next version.
>
> As mentioned above, I'm very uncomfortable about this API in its
> entirety and think it should be reconsidered.
>
> So whilst I disagree here (the caller does not have any significant
> difference in error handling so using -ENOMEM/-EINVAL/-EIO should be
> quite sufficient to distringuish between errors) I rather you spent
> some time trying to eliminate this API.
I got it. We will merge venc_if_create and venc_if_init in next
version.
>
> >> > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
> >> > new file mode 100644
> >> > index 0000000..a8e683a
> >> > --- /dev/null
> >> > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
> >> > @@ -0,0 +1,66 @@
> >> > +/*
> >> > +* Copyright (c) 2015 MediaTek Inc.
> >> > +* Author: PC Chen <pc.chen@mediatek.com>
> >> > +* Tiffany Lin <tiffany.lin@mediatek.com>
> >> > +*
> >> > +* This program is free software; you can redistribute it and/or modify
> >> > +* it under the terms of the GNU General Public License version 2 as
> >> > +* published by the Free Software Foundation.
> >> > +*
> >> > +* This program is distributed in the hope that it will be useful,
> >> > +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> >> > +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> >> > +* GNU General Public License for more details.
> >> > +*/
> >> > +
> >> > +#ifndef _MTK_VCODEC_UTIL_H_
> >> > +#define _MTK_VCODEC_UTIL_H_
> >> > +
> >> > +#include <linux/types.h>
> >> > +#include <linux/dma-direction.h>
> >> > +
> >> > +struct mtk_vcodec_mem {
> >> > + size_t size;
> >> > + void *va;
> >> > + dma_addr_t dma_addr;
> >> > +};
> >> > +
> >> > +extern int mtk_v4l2_dbg_level;
> >> > +extern bool mtk_vcodec_dbg;
> >> > +
> >> > +#define mtk_v4l2_debug(level, fmt, args...) \
> >> > + do { \
> >> > + if (mtk_v4l2_dbg_level >= level) \
> >> > + pr_info("[MTK_V4L2] level=%d %s(),%d: " fmt "\n",\
> >> > + level, __func__, __LINE__, ##args); \
> >> > + } while (0)
> >> > +
> >> > +#define mtk_v4l2_err(fmt, args...) \
> >> > + pr_err("[MTK_V4L2][ERROR] %s:%d: " fmt "\n", __func__, __LINE__, \
> >> > + ##args)
> >>
> >> Obviously the code should be structured to make use of dev_dbg/dev_err
> >> possible.
> >>
> >> However where this won't work do you really need special macros for
> >> this. Assuming your error messages are well written 'git grep' and the
> >> following should be enough:
> >>
> >> #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
> >>
> > Thanks.
> > For pr_err case, we will try to use "#define pr_fmt(fmt) KBUILD_MODNAME
> > ": " fmt" in next version.
> > For pr_info case, we still need debug level to control output messages.
>
> To be honest I expect new code to be able to rely on -DDEBUG and/or
> CONFIG_DYNAMIC_DEBUG.
>
> I really can't see why a single V4L2 driver needs to hand roll a six
> level debug message framework. If it really, really, really needs it
> then it should at least have the good manners to copy the prior art in
> the existing V4L2 drivers.
>
We will add -DDEBUG in next version.
In development stage, we need it when debugging.
>
>
> >
> >>
> >> > +#define mtk_v4l2_debug_enter() mtk_v4l2_debug(5, "+\n")
> >> > +#define mtk_v4l2_debug_leave() mtk_v4l2_debug(5, "-\n")
> >>
> >> Remove these. If you care about function entry and exit for debugging
> >> you should be able to use ftrace.
> >>
> > I am not familiar with ftrace.
> > What if we only want to trace v4l2 video encoder driver called flow not
> > called stack? And only for functions we are interested not all
> > functions.
> > I will check if it is convenience for us using ftrace.
>
> It is find for ftrace to only track a subset of functions.
>
>
> Daniel.
best regards,
Tiffany
^ permalink raw reply [flat|nested] 84+ messages in thread
* Re: [RESEND RFC/PATCH 6/8] media: platform: mtk-vcodec: Add Mediatek V4L2 Video Encoder Driver
@ 2015-12-01 10:42 ` tiffany lin
0 siblings, 0 replies; 84+ messages in thread
From: tiffany lin @ 2015-12-01 10:42 UTC (permalink / raw)
To: Daniel Thompson
Cc: Rob Herring, Pawel Moll, Mark Rutland, Ian Campbell, Kumar Gala,
Catalin Marinas, Will Deacon, Mauro Carvalho Chehab,
Matthias Brugger, Daniel Kurtz, Sascha Hauer, Hongzhou Yang,
Hans Verkuil, Laurent Pinchart, Sakari Ailus, Mikhail Ulyanov,
Fabien Dessenne, Arnd Bergmann, Darren Etheridge, Peter Griffin,
Benoit Parrot, Andrew-CT
On Mon, 2015-11-30 at 22:58 +0800, Daniel Thompson wrote:
> On 30 November 2015 at 11:39, tiffany lin <tiffany.lin-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org> wrote:
> >> > diff --git a/drivers/media/platform/mtk-vcodec/Makefile b/drivers/media/platform/mtk-vcodec/Makefile
> >> > new file mode 100644
> >> > index 0000000..c7f7174
> >> > --- /dev/null
> >> > +++ b/drivers/media/platform/mtk-vcodec/Makefile
> >> > @@ -0,0 +1,12 @@
> >> > +obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += mtk_vcodec_intr.o \
> >> > + mtk_vcodec_util.o \
> >> > + mtk_vcodec_enc_drv.o \
> >> > + mtk_vcodec_enc.o \
> >> > + mtk_vcodec_enc_pm.o
> >> > +
> >> > +obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += common/
> >> > +
> >> > +ccflags-y += -I$(srctree)/drivers/media/platform/mtk-vcodec/include \
> >> > + -I$(srctree)/drivers/media/platform/mtk-vcodec \
> >> > + -I$(srctree)/drivers/media/platform/mtk-vpu
> >>
> >> Seems like there's a lot of directories here. Are these files
> >> (framework, common, vcodec, etc) so unrelated they really need to live
> >> in separate directories?
> >>
> >> Why not just drivers/media/platform/mediatek?
> > This is because VPU and Vcodec are two different drivers.
> > Driver in mtk-vpu is for controlling VPU device and provide
> > communication API to VPU.
> > Driver in mtk-vcodec is for control different encoder (vp8, h264), it
> > include v4l2 driver layer, glue layer between encoders and vp8 and h264
> > encoder.
>
> They may be separate pieces of hardware the drivers for them are very
> clearly interlinked. This is obvious because the Makefiles are having
> to set ccflags to pick up the headers of the other drivers.
>
> No other V4L2 driver uses ccflags-y in this manner.
>
Got it.
We will remove -I from Makefile and put VPU header file in
include/soc/mediatek
>
> >> > diff --git a/drivers/media/platform/mtk-vcodec/common/Makefile b/drivers/media/platform/mtk-vcodec/common/Makefile
> >> > new file mode 100644
> >> > index 0000000..477ab80
> >> > --- /dev/null
> >> > +++ b/drivers/media/platform/mtk-vcodec/common/Makefile
> >> > @@ -0,0 +1,8 @@
> >> > +obj-y += \
> >> > + venc_drv_if.o
> >> > +
> >> > +ccflags-y += \
> >> > + -I$(srctree)/include/ \
> >> > + -I$(srctree)/drivers/media/platform/mtk-vcodec \
> >> > + -I$(srctree)/drivers/media/platform/mtk-vcodec/include \
> >> > + -I$(srctree)/drivers/media/platform/mtk-vpu
> >>
> >> As above, this appears to be a directory to hold just one file.
> >>
> > Sorry, I didn't get it. Could you explain more?
>
> Just that this is another example of the excessive directory structure.
>
> A directory that contains only one source file is a strong indication
> that the splitting of the V4L2 implementation into directories is
> excessive.
>
The directory that contains only one source file is because now we only
upstream encoder patches. We have decoder patches in future.
We will remove "common", "include" two directories and put files in
mtk-vcodec in next version.
>
> >> > diff --git a/drivers/media/platform/mtk-vcodec/common/venc_drv_if.c
> >> b/drivers/media/platform/mtk-vcodec/common/venc_drv_if.c
> >> > new file mode 100644
> >> > index 0000000..9b3f025
> >> > --- /dev/null
> >> > +++ b/drivers/media/platform/mtk-vcodec/common/venc_drv_if.c
> >> > @@ -0,0 +1,152 @@
> >> > +/*
> >> > + * Copyright (c) 2015 MediaTek Inc.
> >> > + * Author: Daniel Hsiao <daniel.hsiao-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
> >> > + * Jungchang Tsao <jungchang.tsao-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
> >> > + *
> >> > + * This program is free software; you can redistribute it and/or
> >> > + * modify
> >> > + * it under the terms of the GNU General Public License version 2 as
> >> > + * published by the Free Software Foundation.
> >> > + *
> >> > + * This program is distributed in the hope that it will be useful,
> >> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> >> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> >> > + * GNU General Public License for more details.
> >> > + */
> >> > +
> >> > +#include <linux/interrupt.h>
> >> > +#include <linux/kernel.h>
> >> > +#include <linux/slab.h>
> >> > +
> >> > +#include "mtk_vcodec_drv.h"
> >> > +#include "mtk_vcodec_enc.h"
> >> > +#include "mtk_vcodec_pm.h"
> >> > +#include "mtk_vcodec_util.h"
> >> > +#include "mtk_vpu_core.h"
> >> > +
> >> > +#include "venc_drv_if.h"
> >> > +#include "venc_drv_base.h"
> >> > +
> >> > +
> >> > +int venc_if_create(void *ctx, unsigned int fourcc, unsigned long
> >> *handle)
> >> > +{
> >> > + struct venc_handle *h;
> >> > + char str[10];
> >> > +
> >> > + mtk_vcodec_fmt2str(fourcc, str);
> >> > +
> >> > + h = kzalloc(sizeof(*h), GFP_KERNEL);
> >> > + if (!h)
> >> > + return -ENOMEM;
> >> > +
> >> > + h->fourcc = fourcc;
> >> > + h->ctx = ctx;
> >> > + mtk_vcodec_debug(h, "fmt = %s handle = %p", str, h);
> >> > +
> >> > + switch (fourcc) {
> >> > + default:
> >> > + mtk_vcodec_err(h, "invalid format %s", str);
> >> > + goto err_out;
> >> > + }
> >> > +
> >> > + *handle = (unsigned long)h;
> >> > + return 0;
> >> > +
> >> > +err_out:
> >> > + kfree(h);
> >> > + return -EINVAL;
> >> > +}
> >> > +
> >> > +int venc_if_init(unsigned long handle)
> >> > +{
> >> > + int ret = 0;
> >> > + struct venc_handle *h = (struct venc_handle *)handle;
> >> > +
> >> > + mtk_vcodec_debug_enter(h);
> >> > +
> >> > + mtk_venc_lock(h->ctx);
> >> > + mtk_vcodec_enc_clock_on();
> >> > + vpu_enable_clock(vpu_get_plat_device(h->ctx->dev->plat_dev));
> >> > + ret = h->enc_if->init(h->ctx, (unsigned long *)&h->drv_handle);
> >> > + vpu_disable_clock(vpu_get_plat_device(h->ctx->dev->plat_dev));
> >> > + mtk_vcodec_enc_clock_off();
> >> > + mtk_venc_unlock(h->ctx);
> >> > +
> >> > + return ret;
> >> > +}
> >>
> >> To me this looks more like an obfuscation layer rather than a
> >> abstraction layer. I don't understand why we need to hide things from
> >> the V4L2 implementation that this code forms part of.
> >>
> >> More importantly, if this code was included somewhere where it could be
> >> properly integrated with the device model you might be able to use the
> >> pm_runtime system to avoid this sort of "heroics" to manage the clocks
> >> anyway.
> >>
> > We want to abstract common part from encoder driver.
> > Every encoder driver follow same calling flow and only need to take care
> > about how to communicate with vpu to encode specific format.
> > Encoder driver do not need to take care clock and multiple instance
> > issue.
>
> Looking at each of those stages:
>
> mtk_venc_lock():
> Why isn't one of the existing V4L2 locking strategies ok for you?
>
We only has one encoder hw.
To support multiple encode instances.
When one encoder ctx access encoder hw, it need to get lock first.
> mtk_vcodec_enc_clock_on():
> This does seem like something a sub-driver *should* be doing for itself
This is for enabling encoder hw related clock.
To support multiple instances, one encode ctx must get hw lock first
then clock on/off hw relate clock.
> vpu_enable_clock():
> Why can't the VPU driver manage this internally using pm_runtime?
>
Our VPU do not have power domain.
We will remove VPU clock on/off and let vpu control it in next version.
>
> That is why I described this as an obfuscation layer. It is collecting
> a bunch of stuff that can be handled using the kernel driver model and
> clumping them together in a special middle layer.
>
We do use kernel driver model, but we put it in
mtk_vcodec_enc_clock_on/mtk_vcodec_enc_clock_off.
Every sub-driver has no need to write the same code.
And once clock configuration change or porting to other chips, we don't
need to change sub-driver one-by-one, just change abstract layer.
>
> >> > +/**
> >> > + * enum mtk_instance_type - The type of an MTK Vcodec instance.
> >> > + */
> >> > +enum mtk_instance_type {
> >> > + MTK_INST_DECODER = 0,
> >> > + MTK_INST_ENCODER = 1,
> >> > +};
> >> > +
> >> > +/**
> >> > + * enum mtk_instance_state - The state of an MTK Vcodec instance.
> >> > + * @MTK_STATE_FREE - default state when instance create
> >> > + * @MTK_STATE_CREATE - vdec instance is create
> >> > + * @MTK_STATE_INIT - vdec instance is init
> >> > + * @MTK_STATE_CONFIG - reserved for encoder
> >> > + * @MTK_STATE_HEADER - vdec had sps/pps header parsed
> >> > + * @MTK_STATE_RUNNING - vdec is decoding
> >> > + * @MTK_STATE_FLUSH - vdec is flushing
> >> > + * @MTK_STATE_RES_CHANGE - vdec detect resolution change
> >> > + * @MTK_STATE_FINISH - ctx instance is stopped streaming
> >> > + * @MTK_STATE_DEINIT - before release ctx instance
> >> > + * @MTK_STATE_ERROR - vdec has something wrong
> >> > + * @MTK_STATE_ABORT - abort work in working thread
> >> > + */
> >> > +enum mtk_instance_state {
> >> > + MTK_STATE_FREE = 0,
> >> > + MTK_STATE_CREATE = (1 << 0),
> >> > + MTK_STATE_INIT = (1 << 1),
> >> > + MTK_STATE_CONFIG = (1 << 2),
> >> > + MTK_STATE_HEADER = (1 << 3),
> >> > + MTK_STATE_RUNNING = (1 << 4),
> >> > + MTK_STATE_FLUSH = (1 << 5),
> >> > + MTK_STATE_RES_CHANGE = (1 << 6),
> >> > + MTK_STATE_FINISH = (1 << 7),
> >> > + MTK_STATE_DEINIT = (1 << 8),
> >> > + MTK_STATE_ERROR = (1 << 9),
> >> > + MTK_STATE_ABORT = (1 << 10),
> >>
> >> This looks like it started as a state machine and somehow turned into
> >> flags, resulting in a state machine with 2048 states or, to give it a
> >> different name, a debugging nightmare.
> >>
> > It's define some state happened rather than state machine.
> > Though some states are for v4l2 decoder driver and not used in encoder
> > driver.
>
> Saying the flags track when "something happened" doesn't stop this
> from being an extremely complex (and poorly documented) state machine.
>
> There are way too many states compared to what is needed to implement
> V4L2 correctly. To make clear why I am raising this point: with the
> current driver state management code it is close to impossible to
> properly review the error paths in this driver. The cause of error and
> the recovery after error are too decoupled.
Some state defines are shared with decoder that we will upstream in
future.
We will review if each state is really needed and reduce state that
encoder need in next version.
>
> >> If the start streaming operation implemented cleanup-on-error properly
> >> then there would only be two useful states: Started and stopped. Even
> >> the "sticky" error behavior looks unnecessary to me (meaning we don't
> >> need to track its state).
> >>
> > We cannot guaranteed that IOCTLs called from the user space follow
> > required sequence.
> > We need states to know if our driver could accept IOCTL command.
>
> I believe that knowing whether the streaming is started or stopped
> (e.g. two states) is sufficient for a driver to correctly handle
> abitrary ioctls from userspace and even then, the core code tracks
> this state for you so there's no need for you do it.
>
> The queue/dequeue ioctls succeed or fail based on the length of the
> queue (i.e. is the buffer queue overflowing or not) and have no need
> to check the streaming state.
> If you are absolutely sure that the other states are needed then
> please provide an example of an ioctl() sequence where the additional
> state is needed.
>
I know your point that we have too many state changes in start_streaming
and stop_streaming function.
We will refine these two functions in next version.
For the example, we need MTK_STATE_HEADER state, to make sure before
encode start, driver already get information to set encode parameters.
We need MTK_STATE_ABORT to inform encoder thread (mtk_venc_worker) that
stop encodeing job from stopped ctx instance.
When user space qbuf, we need to make sure everything is ready to sent
buf to encode.
>
> >> > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
> >> > new file mode 100644
> >> > index 0000000..8e1b6f0
> >> > --- /dev/null
> >> > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
> >> > @@ -0,0 +1,1773 @@
> >> > [...]
> >> > +static int vb2ops_venc_start_streaming(struct vb2_queue *q, unsigned int count)
> >> > +{
> >> > + struct mtk_vcodec_ctx *ctx = vb2_get_drv_priv(q);
> >> > + struct v4l2_device *v4l2_dev = &ctx->dev->v4l2_dev;
> >> > + int ret;
> >> > +#if MTK_V4L2_BENCHMARK
> >> > + struct timeval begin, end;
> >> > +
> >> > + do_gettimeofday(&begin);
> >> > +#endif
> >> > +
> >> > + if (!(vb2_start_streaming_called(&ctx->m2m_ctx->out_q_ctx.q) &
> >> > + vb2_start_streaming_called(&ctx->m2m_ctx->cap_q_ctx.q))) {
> >> > + mtk_v4l2_debug(1, "[%d]-> out=%d cap=%d",
> >> > + ctx->idx,
> >> > + vb2_start_streaming_called(&ctx->m2m_ctx->out_q_ctx.q),
> >> > + vb2_start_streaming_called(&ctx->m2m_ctx->cap_q_ctx.q));
> >> > + return 0;
> >> > + }
> >> > +
> >> > + if ((ctx->state & (MTK_STATE_ERROR | MTK_STATE_ABORT)))
> >> > + return -EINVAL;
> >>
> >> This is the sort of thing I mean.
> >>
> >> This sticky error behaviour means that every subsequent call to
> >> vb2ops_venc_start_streaming() will fail. Note also that the user will
> >> never try to stop streaming (which can clear the error state) because
> >> according to the return code it got when it tried to start streaming we
> >> never actually started.
> >>
> >> This is what I mean about having two many states. From the user's
> >> perspective there are only two states. There needs to be a good reason
> >> for the driver to manage so many extra secret states internally.
> >>
> > For my understanding, that vb2ops_venc_start_streaming cannot fail.
>
> I disagree: See
> http://lxr.free-electrons.com/source/include/media/videobuf2-core.h#L288
>
> How did you confirm your understanding before replying?
>
Sorry that I did not explain well.
What I want to said about "cannot fail" is that once start streaming
fail, all subsequent calls will fail.
The only recover step is close this instance and open instance again.
> When this function returns an error the simplest (and easiest to
> review) error recovery strategy is simply to undo any actions which
> have already been performed (like resource allocation) and return an
> error code.#
>
> There is no need for the driver to remember that it has already
> reported an error. If the userspace tries again then its OK for us to
> fail again.
>
Our original though is that when start_streaming fail, user space will
close this instance. And if user space called start_streaming again, it
just return fail.
I got what you means now, we will try to reduce state changes and remove
state check in start_streaming/stop_streaming in next version.
>
> > If it fail, user space will close and release this encoder instance
> > (fd).
>
> The userspace is not required to do this and the driver must not
> assume that it will. It could attempt some kind of reconfiguration and
> retry.
>
>
> > We really need to state driver to see what it should do when receive
> > current IOCTL.
>
> I think you'll find that the v4l2-core does this for you.
>
We will review each state and make sure if it really need for encoder in
next version.
>
> >> > +
> >> > + if (ctx->state == MTK_STATE_FREE) {
> >> > + ret = venc_if_create(ctx,
> >> > + ctx->q_data[MTK_Q_DATA_DST].fmt->fourcc,
> >> > + &ctx->h_enc);
> >> > +
> >> > + if (ret != 0) {
> >> > + ctx->state |= MTK_STATE_ERROR;
> >> > + v4l2_err(v4l2_dev, "invalid codec type=%x\n",
> >> > + ctx->q_data[MTK_Q_DATA_DST].fmt->fourcc);
> >> > + v4l2_err(v4l2_dev, "venc_if_create failed=%d\n", ret);
> >> > + return -EINVAL;
> >> > + }
> >> > +
> >> > + if (ctx->q_data[MTK_Q_DATA_DST].fmt->fourcc ==
> >> > + V4L2_PIX_FMT_H264)
> >> > + ctx->hdr = 1;
> >> > +
> >> > + ctx->state |= MTK_STATE_CREATE;
> >> > + }
> >> > +
> >> > + if ((ctx->state & MTK_STATE_CREATE) && !(ctx->state & MTK_STATE_INIT)) {
> >> > + ret = venc_if_init(ctx->h_enc);
> >> > + if (ret != 0) {
> >> > + ctx->state |= MTK_STATE_ERROR;
> >> > + v4l2_err(v4l2_dev, "venc_if_init failed=%d\n", ret);
> >> > + return -EINVAL;
> >>
> >> This error path leaves the encoder partially constructed and relies on
> >> something else to tidy things up. It would be much better to tidy things
> >> up from this function and
> >>
> >> Also I don't think both venc_if_create and venc_if_init are needed. They
> >> are only ever called one after the other and thus they only serve to
> >> complicate the error handling code.
> >>
> > venc_if_create is for creating instance in arm side and base on encode
> > format hook corresponding encoder driver interface.
> > venc_if_init is trying to init encoder instance in VPU side.
> > Failures from two functions should have different error handling.
> > We will enhance this part in next version.
>
> As mentioned above, I'm very uncomfortable about this API in its
> entirety and think it should be reconsidered.
>
> So whilst I disagree here (the caller does not have any significant
> difference in error handling so using -ENOMEM/-EINVAL/-EIO should be
> quite sufficient to distringuish between errors) I rather you spent
> some time trying to eliminate this API.
I got it. We will merge venc_if_create and venc_if_init in next
version.
>
> >> > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
> >> > new file mode 100644
> >> > index 0000000..a8e683a
> >> > --- /dev/null
> >> > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
> >> > @@ -0,0 +1,66 @@
> >> > +/*
> >> > +* Copyright (c) 2015 MediaTek Inc.
> >> > +* Author: PC Chen <pc.chen-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
> >> > +* Tiffany Lin <tiffany.lin-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
> >> > +*
> >> > +* This program is free software; you can redistribute it and/or modify
> >> > +* it under the terms of the GNU General Public License version 2 as
> >> > +* published by the Free Software Foundation.
> >> > +*
> >> > +* This program is distributed in the hope that it will be useful,
> >> > +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> >> > +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> >> > +* GNU General Public License for more details.
> >> > +*/
> >> > +
> >> > +#ifndef _MTK_VCODEC_UTIL_H_
> >> > +#define _MTK_VCODEC_UTIL_H_
> >> > +
> >> > +#include <linux/types.h>
> >> > +#include <linux/dma-direction.h>
> >> > +
> >> > +struct mtk_vcodec_mem {
> >> > + size_t size;
> >> > + void *va;
> >> > + dma_addr_t dma_addr;
> >> > +};
> >> > +
> >> > +extern int mtk_v4l2_dbg_level;
> >> > +extern bool mtk_vcodec_dbg;
> >> > +
> >> > +#define mtk_v4l2_debug(level, fmt, args...) \
> >> > + do { \
> >> > + if (mtk_v4l2_dbg_level >= level) \
> >> > + pr_info("[MTK_V4L2] level=%d %s(),%d: " fmt "\n",\
> >> > + level, __func__, __LINE__, ##args); \
> >> > + } while (0)
> >> > +
> >> > +#define mtk_v4l2_err(fmt, args...) \
> >> > + pr_err("[MTK_V4L2][ERROR] %s:%d: " fmt "\n", __func__, __LINE__, \
> >> > + ##args)
> >>
> >> Obviously the code should be structured to make use of dev_dbg/dev_err
> >> possible.
> >>
> >> However where this won't work do you really need special macros for
> >> this. Assuming your error messages are well written 'git grep' and the
> >> following should be enough:
> >>
> >> #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
> >>
> > Thanks.
> > For pr_err case, we will try to use "#define pr_fmt(fmt) KBUILD_MODNAME
> > ": " fmt" in next version.
> > For pr_info case, we still need debug level to control output messages.
>
> To be honest I expect new code to be able to rely on -DDEBUG and/or
> CONFIG_DYNAMIC_DEBUG.
>
> I really can't see why a single V4L2 driver needs to hand roll a six
> level debug message framework. If it really, really, really needs it
> then it should at least have the good manners to copy the prior art in
> the existing V4L2 drivers.
>
We will add -DDEBUG in next version.
In development stage, we need it when debugging.
>
>
> >
> >>
> >> > +#define mtk_v4l2_debug_enter() mtk_v4l2_debug(5, "+\n")
> >> > +#define mtk_v4l2_debug_leave() mtk_v4l2_debug(5, "-\n")
> >>
> >> Remove these. If you care about function entry and exit for debugging
> >> you should be able to use ftrace.
> >>
> > I am not familiar with ftrace.
> > What if we only want to trace v4l2 video encoder driver called flow not
> > called stack? And only for functions we are interested not all
> > functions.
> > I will check if it is convenience for us using ftrace.
>
> It is find for ftrace to only track a subset of functions.
>
>
> Daniel.
best regards,
Tiffany
--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 84+ messages in thread
* [RESEND RFC/PATCH 6/8] media: platform: mtk-vcodec: Add Mediatek V4L2 Video Encoder Driver
@ 2015-12-01 10:42 ` tiffany lin
0 siblings, 0 replies; 84+ messages in thread
From: tiffany lin @ 2015-12-01 10:42 UTC (permalink / raw)
To: linux-arm-kernel
On Mon, 2015-11-30 at 22:58 +0800, Daniel Thompson wrote:
> On 30 November 2015 at 11:39, tiffany lin <tiffany.lin@mediatek.com> wrote:
> >> > diff --git a/drivers/media/platform/mtk-vcodec/Makefile b/drivers/media/platform/mtk-vcodec/Makefile
> >> > new file mode 100644
> >> > index 0000000..c7f7174
> >> > --- /dev/null
> >> > +++ b/drivers/media/platform/mtk-vcodec/Makefile
> >> > @@ -0,0 +1,12 @@
> >> > +obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += mtk_vcodec_intr.o \
> >> > + mtk_vcodec_util.o \
> >> > + mtk_vcodec_enc_drv.o \
> >> > + mtk_vcodec_enc.o \
> >> > + mtk_vcodec_enc_pm.o
> >> > +
> >> > +obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += common/
> >> > +
> >> > +ccflags-y += -I$(srctree)/drivers/media/platform/mtk-vcodec/include \
> >> > + -I$(srctree)/drivers/media/platform/mtk-vcodec \
> >> > + -I$(srctree)/drivers/media/platform/mtk-vpu
> >>
> >> Seems like there's a lot of directories here. Are these files
> >> (framework, common, vcodec, etc) so unrelated they really need to live
> >> in separate directories?
> >>
> >> Why not just drivers/media/platform/mediatek?
> > This is because VPU and Vcodec are two different drivers.
> > Driver in mtk-vpu is for controlling VPU device and provide
> > communication API to VPU.
> > Driver in mtk-vcodec is for control different encoder (vp8, h264), it
> > include v4l2 driver layer, glue layer between encoders and vp8 and h264
> > encoder.
>
> They may be separate pieces of hardware the drivers for them are very
> clearly interlinked. This is obvious because the Makefiles are having
> to set ccflags to pick up the headers of the other drivers.
>
> No other V4L2 driver uses ccflags-y in this manner.
>
Got it.
We will remove -I from Makefile and put VPU header file in
include/soc/mediatek
>
> >> > diff --git a/drivers/media/platform/mtk-vcodec/common/Makefile b/drivers/media/platform/mtk-vcodec/common/Makefile
> >> > new file mode 100644
> >> > index 0000000..477ab80
> >> > --- /dev/null
> >> > +++ b/drivers/media/platform/mtk-vcodec/common/Makefile
> >> > @@ -0,0 +1,8 @@
> >> > +obj-y += \
> >> > + venc_drv_if.o
> >> > +
> >> > +ccflags-y += \
> >> > + -I$(srctree)/include/ \
> >> > + -I$(srctree)/drivers/media/platform/mtk-vcodec \
> >> > + -I$(srctree)/drivers/media/platform/mtk-vcodec/include \
> >> > + -I$(srctree)/drivers/media/platform/mtk-vpu
> >>
> >> As above, this appears to be a directory to hold just one file.
> >>
> > Sorry, I didn't get it. Could you explain more?
>
> Just that this is another example of the excessive directory structure.
>
> A directory that contains only one source file is a strong indication
> that the splitting of the V4L2 implementation into directories is
> excessive.
>
The directory that contains only one source file is because now we only
upstream encoder patches. We have decoder patches in future.
We will remove "common", "include" two directories and put files in
mtk-vcodec in next version.
>
> >> > diff --git a/drivers/media/platform/mtk-vcodec/common/venc_drv_if.c
> >> b/drivers/media/platform/mtk-vcodec/common/venc_drv_if.c
> >> > new file mode 100644
> >> > index 0000000..9b3f025
> >> > --- /dev/null
> >> > +++ b/drivers/media/platform/mtk-vcodec/common/venc_drv_if.c
> >> > @@ -0,0 +1,152 @@
> >> > +/*
> >> > + * Copyright (c) 2015 MediaTek Inc.
> >> > + * Author: Daniel Hsiao <daniel.hsiao@mediatek.com>
> >> > + * Jungchang Tsao <jungchang.tsao@mediatek.com>
> >> > + *
> >> > + * This program is free software; you can redistribute it and/or
> >> > + * modify
> >> > + * it under the terms of the GNU General Public License version 2 as
> >> > + * published by the Free Software Foundation.
> >> > + *
> >> > + * This program is distributed in the hope that it will be useful,
> >> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> >> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> >> > + * GNU General Public License for more details.
> >> > + */
> >> > +
> >> > +#include <linux/interrupt.h>
> >> > +#include <linux/kernel.h>
> >> > +#include <linux/slab.h>
> >> > +
> >> > +#include "mtk_vcodec_drv.h"
> >> > +#include "mtk_vcodec_enc.h"
> >> > +#include "mtk_vcodec_pm.h"
> >> > +#include "mtk_vcodec_util.h"
> >> > +#include "mtk_vpu_core.h"
> >> > +
> >> > +#include "venc_drv_if.h"
> >> > +#include "venc_drv_base.h"
> >> > +
> >> > +
> >> > +int venc_if_create(void *ctx, unsigned int fourcc, unsigned long
> >> *handle)
> >> > +{
> >> > + struct venc_handle *h;
> >> > + char str[10];
> >> > +
> >> > + mtk_vcodec_fmt2str(fourcc, str);
> >> > +
> >> > + h = kzalloc(sizeof(*h), GFP_KERNEL);
> >> > + if (!h)
> >> > + return -ENOMEM;
> >> > +
> >> > + h->fourcc = fourcc;
> >> > + h->ctx = ctx;
> >> > + mtk_vcodec_debug(h, "fmt = %s handle = %p", str, h);
> >> > +
> >> > + switch (fourcc) {
> >> > + default:
> >> > + mtk_vcodec_err(h, "invalid format %s", str);
> >> > + goto err_out;
> >> > + }
> >> > +
> >> > + *handle = (unsigned long)h;
> >> > + return 0;
> >> > +
> >> > +err_out:
> >> > + kfree(h);
> >> > + return -EINVAL;
> >> > +}
> >> > +
> >> > +int venc_if_init(unsigned long handle)
> >> > +{
> >> > + int ret = 0;
> >> > + struct venc_handle *h = (struct venc_handle *)handle;
> >> > +
> >> > + mtk_vcodec_debug_enter(h);
> >> > +
> >> > + mtk_venc_lock(h->ctx);
> >> > + mtk_vcodec_enc_clock_on();
> >> > + vpu_enable_clock(vpu_get_plat_device(h->ctx->dev->plat_dev));
> >> > + ret = h->enc_if->init(h->ctx, (unsigned long *)&h->drv_handle);
> >> > + vpu_disable_clock(vpu_get_plat_device(h->ctx->dev->plat_dev));
> >> > + mtk_vcodec_enc_clock_off();
> >> > + mtk_venc_unlock(h->ctx);
> >> > +
> >> > + return ret;
> >> > +}
> >>
> >> To me this looks more like an obfuscation layer rather than a
> >> abstraction layer. I don't understand why we need to hide things from
> >> the V4L2 implementation that this code forms part of.
> >>
> >> More importantly, if this code was included somewhere where it could be
> >> properly integrated with the device model you might be able to use the
> >> pm_runtime system to avoid this sort of "heroics" to manage the clocks
> >> anyway.
> >>
> > We want to abstract common part from encoder driver.
> > Every encoder driver follow same calling flow and only need to take care
> > about how to communicate with vpu to encode specific format.
> > Encoder driver do not need to take care clock and multiple instance
> > issue.
>
> Looking at each of those stages:
>
> mtk_venc_lock():
> Why isn't one of the existing V4L2 locking strategies ok for you?
>
We only has one encoder hw.
To support multiple encode instances.
When one encoder ctx access encoder hw, it need to get lock first.
> mtk_vcodec_enc_clock_on():
> This does seem like something a sub-driver *should* be doing for itself
This is for enabling encoder hw related clock.
To support multiple instances, one encode ctx must get hw lock first
then clock on/off hw relate clock.
> vpu_enable_clock():
> Why can't the VPU driver manage this internally using pm_runtime?
>
Our VPU do not have power domain.
We will remove VPU clock on/off and let vpu control it in next version.
>
> That is why I described this as an obfuscation layer. It is collecting
> a bunch of stuff that can be handled using the kernel driver model and
> clumping them together in a special middle layer.
>
We do use kernel driver model, but we put it in
mtk_vcodec_enc_clock_on/mtk_vcodec_enc_clock_off.
Every sub-driver has no need to write the same code.
And once clock configuration change or porting to other chips, we don't
need to change sub-driver one-by-one, just change abstract layer.
>
> >> > +/**
> >> > + * enum mtk_instance_type - The type of an MTK Vcodec instance.
> >> > + */
> >> > +enum mtk_instance_type {
> >> > + MTK_INST_DECODER = 0,
> >> > + MTK_INST_ENCODER = 1,
> >> > +};
> >> > +
> >> > +/**
> >> > + * enum mtk_instance_state - The state of an MTK Vcodec instance.
> >> > + * @MTK_STATE_FREE - default state when instance create
> >> > + * @MTK_STATE_CREATE - vdec instance is create
> >> > + * @MTK_STATE_INIT - vdec instance is init
> >> > + * @MTK_STATE_CONFIG - reserved for encoder
> >> > + * @MTK_STATE_HEADER - vdec had sps/pps header parsed
> >> > + * @MTK_STATE_RUNNING - vdec is decoding
> >> > + * @MTK_STATE_FLUSH - vdec is flushing
> >> > + * @MTK_STATE_RES_CHANGE - vdec detect resolution change
> >> > + * @MTK_STATE_FINISH - ctx instance is stopped streaming
> >> > + * @MTK_STATE_DEINIT - before release ctx instance
> >> > + * @MTK_STATE_ERROR - vdec has something wrong
> >> > + * @MTK_STATE_ABORT - abort work in working thread
> >> > + */
> >> > +enum mtk_instance_state {
> >> > + MTK_STATE_FREE = 0,
> >> > + MTK_STATE_CREATE = (1 << 0),
> >> > + MTK_STATE_INIT = (1 << 1),
> >> > + MTK_STATE_CONFIG = (1 << 2),
> >> > + MTK_STATE_HEADER = (1 << 3),
> >> > + MTK_STATE_RUNNING = (1 << 4),
> >> > + MTK_STATE_FLUSH = (1 << 5),
> >> > + MTK_STATE_RES_CHANGE = (1 << 6),
> >> > + MTK_STATE_FINISH = (1 << 7),
> >> > + MTK_STATE_DEINIT = (1 << 8),
> >> > + MTK_STATE_ERROR = (1 << 9),
> >> > + MTK_STATE_ABORT = (1 << 10),
> >>
> >> This looks like it started as a state machine and somehow turned into
> >> flags, resulting in a state machine with 2048 states or, to give it a
> >> different name, a debugging nightmare.
> >>
> > It's define some state happened rather than state machine.
> > Though some states are for v4l2 decoder driver and not used in encoder
> > driver.
>
> Saying the flags track when "something happened" doesn't stop this
> from being an extremely complex (and poorly documented) state machine.
>
> There are way too many states compared to what is needed to implement
> V4L2 correctly. To make clear why I am raising this point: with the
> current driver state management code it is close to impossible to
> properly review the error paths in this driver. The cause of error and
> the recovery after error are too decoupled.
Some state defines are shared with decoder that we will upstream in
future.
We will review if each state is really needed and reduce state that
encoder need in next version.
>
> >> If the start streaming operation implemented cleanup-on-error properly
> >> then there would only be two useful states: Started and stopped. Even
> >> the "sticky" error behavior looks unnecessary to me (meaning we don't
> >> need to track its state).
> >>
> > We cannot guaranteed that IOCTLs called from the user space follow
> > required sequence.
> > We need states to know if our driver could accept IOCTL command.
>
> I believe that knowing whether the streaming is started or stopped
> (e.g. two states) is sufficient for a driver to correctly handle
> abitrary ioctls from userspace and even then, the core code tracks
> this state for you so there's no need for you do it.
>
> The queue/dequeue ioctls succeed or fail based on the length of the
> queue (i.e. is the buffer queue overflowing or not) and have no need
> to check the streaming state.
> If you are absolutely sure that the other states are needed then
> please provide an example of an ioctl() sequence where the additional
> state is needed.
>
I know your point that we have too many state changes in start_streaming
and stop_streaming function.
We will refine these two functions in next version.
For the example, we need MTK_STATE_HEADER state, to make sure before
encode start, driver already get information to set encode parameters.
We need MTK_STATE_ABORT to inform encoder thread (mtk_venc_worker) that
stop encodeing job from stopped ctx instance.
When user space qbuf, we need to make sure everything is ready to sent
buf to encode.
>
> >> > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
> >> > new file mode 100644
> >> > index 0000000..8e1b6f0
> >> > --- /dev/null
> >> > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
> >> > @@ -0,0 +1,1773 @@
> >> > [...]
> >> > +static int vb2ops_venc_start_streaming(struct vb2_queue *q, unsigned int count)
> >> > +{
> >> > + struct mtk_vcodec_ctx *ctx = vb2_get_drv_priv(q);
> >> > + struct v4l2_device *v4l2_dev = &ctx->dev->v4l2_dev;
> >> > + int ret;
> >> > +#if MTK_V4L2_BENCHMARK
> >> > + struct timeval begin, end;
> >> > +
> >> > + do_gettimeofday(&begin);
> >> > +#endif
> >> > +
> >> > + if (!(vb2_start_streaming_called(&ctx->m2m_ctx->out_q_ctx.q) &
> >> > + vb2_start_streaming_called(&ctx->m2m_ctx->cap_q_ctx.q))) {
> >> > + mtk_v4l2_debug(1, "[%d]-> out=%d cap=%d",
> >> > + ctx->idx,
> >> > + vb2_start_streaming_called(&ctx->m2m_ctx->out_q_ctx.q),
> >> > + vb2_start_streaming_called(&ctx->m2m_ctx->cap_q_ctx.q));
> >> > + return 0;
> >> > + }
> >> > +
> >> > + if ((ctx->state & (MTK_STATE_ERROR | MTK_STATE_ABORT)))
> >> > + return -EINVAL;
> >>
> >> This is the sort of thing I mean.
> >>
> >> This sticky error behaviour means that every subsequent call to
> >> vb2ops_venc_start_streaming() will fail. Note also that the user will
> >> never try to stop streaming (which can clear the error state) because
> >> according to the return code it got when it tried to start streaming we
> >> never actually started.
> >>
> >> This is what I mean about having two many states. From the user's
> >> perspective there are only two states. There needs to be a good reason
> >> for the driver to manage so many extra secret states internally.
> >>
> > For my understanding, that vb2ops_venc_start_streaming cannot fail.
>
> I disagree: See
> http://lxr.free-electrons.com/source/include/media/videobuf2-core.h#L288
>
> How did you confirm your understanding before replying?
>
Sorry that I did not explain well.
What I want to said about "cannot fail" is that once start streaming
fail, all subsequent calls will fail.
The only recover step is close this instance and open instance again.
> When this function returns an error the simplest (and easiest to
> review) error recovery strategy is simply to undo any actions which
> have already been performed (like resource allocation) and return an
> error code.#
>
> There is no need for the driver to remember that it has already
> reported an error. If the userspace tries again then its OK for us to
> fail again.
>
Our original though is that when start_streaming fail, user space will
close this instance. And if user space called start_streaming again, it
just return fail.
I got what you means now, we will try to reduce state changes and remove
state check in start_streaming/stop_streaming in next version.
>
> > If it fail, user space will close and release this encoder instance
> > (fd).
>
> The userspace is not required to do this and the driver must not
> assume that it will. It could attempt some kind of reconfiguration and
> retry.
>
>
> > We really need to state driver to see what it should do when receive
> > current IOCTL.
>
> I think you'll find that the v4l2-core does this for you.
>
We will review each state and make sure if it really need for encoder in
next version.
>
> >> > +
> >> > + if (ctx->state == MTK_STATE_FREE) {
> >> > + ret = venc_if_create(ctx,
> >> > + ctx->q_data[MTK_Q_DATA_DST].fmt->fourcc,
> >> > + &ctx->h_enc);
> >> > +
> >> > + if (ret != 0) {
> >> > + ctx->state |= MTK_STATE_ERROR;
> >> > + v4l2_err(v4l2_dev, "invalid codec type=%x\n",
> >> > + ctx->q_data[MTK_Q_DATA_DST].fmt->fourcc);
> >> > + v4l2_err(v4l2_dev, "venc_if_create failed=%d\n", ret);
> >> > + return -EINVAL;
> >> > + }
> >> > +
> >> > + if (ctx->q_data[MTK_Q_DATA_DST].fmt->fourcc ==
> >> > + V4L2_PIX_FMT_H264)
> >> > + ctx->hdr = 1;
> >> > +
> >> > + ctx->state |= MTK_STATE_CREATE;
> >> > + }
> >> > +
> >> > + if ((ctx->state & MTK_STATE_CREATE) && !(ctx->state & MTK_STATE_INIT)) {
> >> > + ret = venc_if_init(ctx->h_enc);
> >> > + if (ret != 0) {
> >> > + ctx->state |= MTK_STATE_ERROR;
> >> > + v4l2_err(v4l2_dev, "venc_if_init failed=%d\n", ret);
> >> > + return -EINVAL;
> >>
> >> This error path leaves the encoder partially constructed and relies on
> >> something else to tidy things up. It would be much better to tidy things
> >> up from this function and
> >>
> >> Also I don't think both venc_if_create and venc_if_init are needed. They
> >> are only ever called one after the other and thus they only serve to
> >> complicate the error handling code.
> >>
> > venc_if_create is for creating instance in arm side and base on encode
> > format hook corresponding encoder driver interface.
> > venc_if_init is trying to init encoder instance in VPU side.
> > Failures from two functions should have different error handling.
> > We will enhance this part in next version.
>
> As mentioned above, I'm very uncomfortable about this API in its
> entirety and think it should be reconsidered.
>
> So whilst I disagree here (the caller does not have any significant
> difference in error handling so using -ENOMEM/-EINVAL/-EIO should be
> quite sufficient to distringuish between errors) I rather you spent
> some time trying to eliminate this API.
I got it. We will merge venc_if_create and venc_if_init in next
version.
>
> >> > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
> >> > new file mode 100644
> >> > index 0000000..a8e683a
> >> > --- /dev/null
> >> > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
> >> > @@ -0,0 +1,66 @@
> >> > +/*
> >> > +* Copyright (c) 2015 MediaTek Inc.
> >> > +* Author: PC Chen <pc.chen@mediatek.com>
> >> > +* Tiffany Lin <tiffany.lin@mediatek.com>
> >> > +*
> >> > +* This program is free software; you can redistribute it and/or modify
> >> > +* it under the terms of the GNU General Public License version 2 as
> >> > +* published by the Free Software Foundation.
> >> > +*
> >> > +* This program is distributed in the hope that it will be useful,
> >> > +* but WITHOUT ANY WARRANTY; without even the implied warranty of
> >> > +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> >> > +* GNU General Public License for more details.
> >> > +*/
> >> > +
> >> > +#ifndef _MTK_VCODEC_UTIL_H_
> >> > +#define _MTK_VCODEC_UTIL_H_
> >> > +
> >> > +#include <linux/types.h>
> >> > +#include <linux/dma-direction.h>
> >> > +
> >> > +struct mtk_vcodec_mem {
> >> > + size_t size;
> >> > + void *va;
> >> > + dma_addr_t dma_addr;
> >> > +};
> >> > +
> >> > +extern int mtk_v4l2_dbg_level;
> >> > +extern bool mtk_vcodec_dbg;
> >> > +
> >> > +#define mtk_v4l2_debug(level, fmt, args...) \
> >> > + do { \
> >> > + if (mtk_v4l2_dbg_level >= level) \
> >> > + pr_info("[MTK_V4L2] level=%d %s(),%d: " fmt "\n",\
> >> > + level, __func__, __LINE__, ##args); \
> >> > + } while (0)
> >> > +
> >> > +#define mtk_v4l2_err(fmt, args...) \
> >> > + pr_err("[MTK_V4L2][ERROR] %s:%d: " fmt "\n", __func__, __LINE__, \
> >> > + ##args)
> >>
> >> Obviously the code should be structured to make use of dev_dbg/dev_err
> >> possible.
> >>
> >> However where this won't work do you really need special macros for
> >> this. Assuming your error messages are well written 'git grep' and the
> >> following should be enough:
> >>
> >> #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
> >>
> > Thanks.
> > For pr_err case, we will try to use "#define pr_fmt(fmt) KBUILD_MODNAME
> > ": " fmt" in next version.
> > For pr_info case, we still need debug level to control output messages.
>
> To be honest I expect new code to be able to rely on -DDEBUG and/or
> CONFIG_DYNAMIC_DEBUG.
>
> I really can't see why a single V4L2 driver needs to hand roll a six
> level debug message framework. If it really, really, really needs it
> then it should at least have the good manners to copy the prior art in
> the existing V4L2 drivers.
>
We will add -DDEBUG in next version.
In development stage, we need it when debugging.
>
>
> >
> >>
> >> > +#define mtk_v4l2_debug_enter() mtk_v4l2_debug(5, "+\n")
> >> > +#define mtk_v4l2_debug_leave() mtk_v4l2_debug(5, "-\n")
> >>
> >> Remove these. If you care about function entry and exit for debugging
> >> you should be able to use ftrace.
> >>
> > I am not familiar with ftrace.
> > What if we only want to trace v4l2 video encoder driver called flow not
> > called stack? And only for functions we are interested not all
> > functions.
> > I will check if it is convenience for us using ftrace.
>
> It is find for ftrace to only track a subset of functions.
>
>
> Daniel.
best regards,
Tiffany
^ permalink raw reply [flat|nested] 84+ messages in thread
* Re: [RESEND RFC/PATCH 3/8] media: platform: mtk-vpu: Support Mediatek VPU
@ 2015-12-01 14:31 ` andrew-ct chen
0 siblings, 0 replies; 84+ messages in thread
From: andrew-ct chen @ 2015-12-01 14:31 UTC (permalink / raw)
To: Daniel Thompson
Cc: Mark Rutland, James Liao, Catalin Marinas, Will Deacon,
Darren Etheridge, Laurent Pinchart, Eddie Huang, Pawel Moll,
Hongzhou Yang, Mauro Carvalho Chehab, Fabien Dessenne,
Peter Griffin, Geert Uytterhoeven, Mikhail Ulyanov, Hans Verkuil,
linux-media, devicetree, Arnd Bergmann, Ian Campbell,
Sascha Hauer, Benoit Parrot, Rob Herring, linux-mediatek,
Yingjoe Chen, Matthias Brugger, Tiffany Lin, linux-arm-kernel,
Daniel Hsiao, lkml, Sakari Ailus, Kumar Gala
On Mon, 2015-11-30 at 15:36 +0000, Daniel Thompson wrote:
> On 30 November 2015 at 11:43, andrew-ct chen
> <andrew-ct.chen@mediatek.com> wrote:
> > On Fri, 2015-11-27 at 12:21 +0000, Daniel Thompson wrote:
> >> On 27/11/15 12:10, andrew-ct chen wrote:
> >> >>> +
> >> >>> > >+ memcpy((void *)send_obj->share_buf, buf, len);
> >> >>> > >+ send_obj->len = len;
> >> >>> > >+ send_obj->id = id;
> >> >>> > >+ vpu_cfg_writel(vpu, 0x1, HOST_TO_VPU);
> >> >>> > >+
> >> >>> > >+ /* Wait until VPU receives the command */
> >> >>> > >+ timeout = jiffies + msecs_to_jiffies(IPI_TIMEOUT_MS);
> >> >>> > >+ do {
> >> >>> > >+ if (time_after(jiffies, timeout)) {
> >> >>> > >+ dev_err(vpu->dev, "vpu_ipi_send: IPI timeout!\n");
> >> >>> > >+ return -EIO;
> >> >>> > >+ }
> >> >>> > >+ } while (vpu_cfg_readl(vpu, HOST_TO_VPU));
> >> >> >
> >> >> >Do we need to busy wait every time we communicate with the co-processor?
> >> >> >Couldn't we put this wait*before* we write to HOST_TO_VPU above.
> >> >> >
> >> >> >That way we only spin when there is a need to.
> >> >> >
> >> > Since the hardware VPU only allows that one client sends the command to
> >> > it each time.
> >> > We need the wait to make sure VPU accepted the command and cleared the
> >> > interrupt and then the next command would be served.
> >>
> >> I understand that the VPU can only have on message outstanding at once.
> >>
> >> I just wonder why we busy wait *after* sending the first command rather
> >> than *before* sending the second one.
> >
> > No other special reasons. Just send one command and wait until VPU gets
> > the command. Then, I think this wait also can be put before we write to
> > HOST_TO_VPU.Is this better than former? May I know the reason?
>
> Busy waiting is bad; it is a waste of host CPU processor time and/or power.
>
> When the busy wait occurs after queuing then we will busy wait for
> every command we send.
>
> If busy wait occurs before next queuing then we will wait for a
> shorter time in total because we have the chance to do something
> useful on the host before we have to wait.
>
Got it. Thanks a lot for the explanation.
I'll put the busy wait before next queuing in next version.
>
> >> Streamed decode/encode typically ends up being rate controlled by
> >> capture or display meaning that in these cases we don't need to busy
> >> wait at all (because by the time we send the next frame the VPU has
> >> already accepted the previous message).
> >
> > For now, only one device "encoder" exists, it is true.
> > But, we'll have encoder and decoder devices, the decode and encode
> > requested to VPU are simultaneous.
>
> Sure, I accept that lock and busy-wait is an appropriate way to ensure
> safety (assuming the VPU is fairly quick clearing the HOST_TO_VPU
> flag).
>
The busy wait time is less than 1ms in average.
>
> > Is this supposed to be removed for this patches and we can add it back
> > if the another device(decoder) is ready for review?
>
> No I'm not suggesting that.
>
> I only recommend moving the busy wait *before* end sending of command
> (for efficiency reasons mentioned above).
Ok. Thanks.
>
>
> Daniel.
^ permalink raw reply [flat|nested] 84+ messages in thread
* Re: [RESEND RFC/PATCH 3/8] media: platform: mtk-vpu: Support Mediatek VPU
@ 2015-12-01 14:31 ` andrew-ct chen
0 siblings, 0 replies; 84+ messages in thread
From: andrew-ct chen @ 2015-12-01 14:31 UTC (permalink / raw)
To: Daniel Thompson
Cc: Mark Rutland, James Liao, Catalin Marinas, Will Deacon,
Darren Etheridge, Laurent Pinchart, Eddie Huang, Pawel Moll,
Hongzhou Yang, Mauro Carvalho Chehab, Fabien Dessenne,
Peter Griffin, Geert Uytterhoeven, Mikhail Ulyanov, Hans Verkuil,
linux-media-u79uwXL29TY76Z2rM5mHXA,
devicetree-u79uwXL29TY76Z2rM5mHXA, Arnd Bergmann, Ian Campbell,
Sascha Hauer, Benoit Parrot, Ro
On Mon, 2015-11-30 at 15:36 +0000, Daniel Thompson wrote:
> On 30 November 2015 at 11:43, andrew-ct chen
> <andrew-ct.chen-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org> wrote:
> > On Fri, 2015-11-27 at 12:21 +0000, Daniel Thompson wrote:
> >> On 27/11/15 12:10, andrew-ct chen wrote:
> >> >>> +
> >> >>> > >+ memcpy((void *)send_obj->share_buf, buf, len);
> >> >>> > >+ send_obj->len = len;
> >> >>> > >+ send_obj->id = id;
> >> >>> > >+ vpu_cfg_writel(vpu, 0x1, HOST_TO_VPU);
> >> >>> > >+
> >> >>> > >+ /* Wait until VPU receives the command */
> >> >>> > >+ timeout = jiffies + msecs_to_jiffies(IPI_TIMEOUT_MS);
> >> >>> > >+ do {
> >> >>> > >+ if (time_after(jiffies, timeout)) {
> >> >>> > >+ dev_err(vpu->dev, "vpu_ipi_send: IPI timeout!\n");
> >> >>> > >+ return -EIO;
> >> >>> > >+ }
> >> >>> > >+ } while (vpu_cfg_readl(vpu, HOST_TO_VPU));
> >> >> >
> >> >> >Do we need to busy wait every time we communicate with the co-processor?
> >> >> >Couldn't we put this wait*before* we write to HOST_TO_VPU above.
> >> >> >
> >> >> >That way we only spin when there is a need to.
> >> >> >
> >> > Since the hardware VPU only allows that one client sends the command to
> >> > it each time.
> >> > We need the wait to make sure VPU accepted the command and cleared the
> >> > interrupt and then the next command would be served.
> >>
> >> I understand that the VPU can only have on message outstanding at once.
> >>
> >> I just wonder why we busy wait *after* sending the first command rather
> >> than *before* sending the second one.
> >
> > No other special reasons. Just send one command and wait until VPU gets
> > the command. Then, I think this wait also can be put before we write to
> > HOST_TO_VPU.Is this better than former? May I know the reason?
>
> Busy waiting is bad; it is a waste of host CPU processor time and/or power.
>
> When the busy wait occurs after queuing then we will busy wait for
> every command we send.
>
> If busy wait occurs before next queuing then we will wait for a
> shorter time in total because we have the chance to do something
> useful on the host before we have to wait.
>
Got it. Thanks a lot for the explanation.
I'll put the busy wait before next queuing in next version.
>
> >> Streamed decode/encode typically ends up being rate controlled by
> >> capture or display meaning that in these cases we don't need to busy
> >> wait at all (because by the time we send the next frame the VPU has
> >> already accepted the previous message).
> >
> > For now, only one device "encoder" exists, it is true.
> > But, we'll have encoder and decoder devices, the decode and encode
> > requested to VPU are simultaneous.
>
> Sure, I accept that lock and busy-wait is an appropriate way to ensure
> safety (assuming the VPU is fairly quick clearing the HOST_TO_VPU
> flag).
>
The busy wait time is less than 1ms in average.
>
> > Is this supposed to be removed for this patches and we can add it back
> > if the another device(decoder) is ready for review?
>
> No I'm not suggesting that.
>
> I only recommend moving the busy wait *before* end sending of command
> (for efficiency reasons mentioned above).
Ok. Thanks.
>
>
> Daniel.
--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 84+ messages in thread
* Re: [RESEND RFC/PATCH 3/8] media: platform: mtk-vpu: Support Mediatek VPU
@ 2015-12-01 14:31 ` andrew-ct chen
0 siblings, 0 replies; 84+ messages in thread
From: andrew-ct chen @ 2015-12-01 14:31 UTC (permalink / raw)
To: Daniel Thompson
Cc: Mark Rutland, James Liao, Catalin Marinas, Will Deacon,
Darren Etheridge, Laurent Pinchart, Eddie Huang, Pawel Moll,
Hongzhou Yang, Mauro Carvalho Chehab, Fabien Dessenne,
Peter Griffin, Geert Uytterhoeven, Mikhail Ulyanov, Hans Verkuil,
linux-media, devicetree, Arnd Bergmann, Ian Campbell,
Sascha Hauer, Benoit Parrot, Rob Herring, linux-mediatek,
Yingjoe Chen, Matthias Brugger, Tiffany Lin, linux-arm-kernel,
Daniel Hsiao, lkml, Sakari Ailus, Kumar Gala
On Mon, 2015-11-30 at 15:36 +0000, Daniel Thompson wrote:
> On 30 November 2015 at 11:43, andrew-ct chen
> <andrew-ct.chen@mediatek.com> wrote:
> > On Fri, 2015-11-27 at 12:21 +0000, Daniel Thompson wrote:
> >> On 27/11/15 12:10, andrew-ct chen wrote:
> >> >>> +
> >> >>> > >+ memcpy((void *)send_obj->share_buf, buf, len);
> >> >>> > >+ send_obj->len = len;
> >> >>> > >+ send_obj->id = id;
> >> >>> > >+ vpu_cfg_writel(vpu, 0x1, HOST_TO_VPU);
> >> >>> > >+
> >> >>> > >+ /* Wait until VPU receives the command */
> >> >>> > >+ timeout = jiffies + msecs_to_jiffies(IPI_TIMEOUT_MS);
> >> >>> > >+ do {
> >> >>> > >+ if (time_after(jiffies, timeout)) {
> >> >>> > >+ dev_err(vpu->dev, "vpu_ipi_send: IPI timeout!\n");
> >> >>> > >+ return -EIO;
> >> >>> > >+ }
> >> >>> > >+ } while (vpu_cfg_readl(vpu, HOST_TO_VPU));
> >> >> >
> >> >> >Do we need to busy wait every time we communicate with the co-processor?
> >> >> >Couldn't we put this wait*before* we write to HOST_TO_VPU above.
> >> >> >
> >> >> >That way we only spin when there is a need to.
> >> >> >
> >> > Since the hardware VPU only allows that one client sends the command to
> >> > it each time.
> >> > We need the wait to make sure VPU accepted the command and cleared the
> >> > interrupt and then the next command would be served.
> >>
> >> I understand that the VPU can only have on message outstanding at once.
> >>
> >> I just wonder why we busy wait *after* sending the first command rather
> >> than *before* sending the second one.
> >
> > No other special reasons. Just send one command and wait until VPU gets
> > the command. Then, I think this wait also can be put before we write to
> > HOST_TO_VPU.Is this better than former? May I know the reason?
>
> Busy waiting is bad; it is a waste of host CPU processor time and/or power.
>
> When the busy wait occurs after queuing then we will busy wait for
> every command we send.
>
> If busy wait occurs before next queuing then we will wait for a
> shorter time in total because we have the chance to do something
> useful on the host before we have to wait.
>
Got it. Thanks a lot for the explanation.
I'll put the busy wait before next queuing in next version.
>
> >> Streamed decode/encode typically ends up being rate controlled by
> >> capture or display meaning that in these cases we don't need to busy
> >> wait at all (because by the time we send the next frame the VPU has
> >> already accepted the previous message).
> >
> > For now, only one device "encoder" exists, it is true.
> > But, we'll have encoder and decoder devices, the decode and encode
> > requested to VPU are simultaneous.
>
> Sure, I accept that lock and busy-wait is an appropriate way to ensure
> safety (assuming the VPU is fairly quick clearing the HOST_TO_VPU
> flag).
>
The busy wait time is less than 1ms in average.
>
> > Is this supposed to be removed for this patches and we can add it back
> > if the another device(decoder) is ready for review?
>
> No I'm not suggesting that.
>
> I only recommend moving the busy wait *before* end sending of command
> (for efficiency reasons mentioned above).
Ok. Thanks.
>
>
> Daniel.
^ permalink raw reply [flat|nested] 84+ messages in thread
* [RESEND RFC/PATCH 3/8] media: platform: mtk-vpu: Support Mediatek VPU
@ 2015-12-01 14:31 ` andrew-ct chen
0 siblings, 0 replies; 84+ messages in thread
From: andrew-ct chen @ 2015-12-01 14:31 UTC (permalink / raw)
To: linux-arm-kernel
On Mon, 2015-11-30 at 15:36 +0000, Daniel Thompson wrote:
> On 30 November 2015 at 11:43, andrew-ct chen
> <andrew-ct.chen@mediatek.com> wrote:
> > On Fri, 2015-11-27 at 12:21 +0000, Daniel Thompson wrote:
> >> On 27/11/15 12:10, andrew-ct chen wrote:
> >> >>> +
> >> >>> > >+ memcpy((void *)send_obj->share_buf, buf, len);
> >> >>> > >+ send_obj->len = len;
> >> >>> > >+ send_obj->id = id;
> >> >>> > >+ vpu_cfg_writel(vpu, 0x1, HOST_TO_VPU);
> >> >>> > >+
> >> >>> > >+ /* Wait until VPU receives the command */
> >> >>> > >+ timeout = jiffies + msecs_to_jiffies(IPI_TIMEOUT_MS);
> >> >>> > >+ do {
> >> >>> > >+ if (time_after(jiffies, timeout)) {
> >> >>> > >+ dev_err(vpu->dev, "vpu_ipi_send: IPI timeout!\n");
> >> >>> > >+ return -EIO;
> >> >>> > >+ }
> >> >>> > >+ } while (vpu_cfg_readl(vpu, HOST_TO_VPU));
> >> >> >
> >> >> >Do we need to busy wait every time we communicate with the co-processor?
> >> >> >Couldn't we put this wait*before* we write to HOST_TO_VPU above.
> >> >> >
> >> >> >That way we only spin when there is a need to.
> >> >> >
> >> > Since the hardware VPU only allows that one client sends the command to
> >> > it each time.
> >> > We need the wait to make sure VPU accepted the command and cleared the
> >> > interrupt and then the next command would be served.
> >>
> >> I understand that the VPU can only have on message outstanding at once.
> >>
> >> I just wonder why we busy wait *after* sending the first command rather
> >> than *before* sending the second one.
> >
> > No other special reasons. Just send one command and wait until VPU gets
> > the command. Then, I think this wait also can be put before we write to
> > HOST_TO_VPU.Is this better than former? May I know the reason?
>
> Busy waiting is bad; it is a waste of host CPU processor time and/or power.
>
> When the busy wait occurs after queuing then we will busy wait for
> every command we send.
>
> If busy wait occurs before next queuing then we will wait for a
> shorter time in total because we have the chance to do something
> useful on the host before we have to wait.
>
Got it. Thanks a lot for the explanation.
I'll put the busy wait before next queuing in next version.
>
> >> Streamed decode/encode typically ends up being rate controlled by
> >> capture or display meaning that in these cases we don't need to busy
> >> wait at all (because by the time we send the next frame the VPU has
> >> already accepted the previous message).
> >
> > For now, only one device "encoder" exists, it is true.
> > But, we'll have encoder and decoder devices, the decode and encode
> > requested to VPU are simultaneous.
>
> Sure, I accept that lock and busy-wait is an appropriate way to ensure
> safety (assuming the VPU is fairly quick clearing the HOST_TO_VPU
> flag).
>
The busy wait time is less than 1ms in average.
>
> > Is this supposed to be removed for this patches and we can add it back
> > if the another device(decoder) is ready for review?
>
> No I'm not suggesting that.
>
> I only recommend moving the busy wait *before* end sending of command
> (for efficiency reasons mentioned above).
Ok. Thanks.
>
>
> Daniel.
^ permalink raw reply [flat|nested] 84+ messages in thread
* Re: [RESEND RFC/PATCH 6/8] media: platform: mtk-vcodec: Add Mediatek V4L2 Video Encoder Driver
2015-12-01 10:42 ` tiffany lin
(?)
@ 2015-12-01 15:42 ` Daniel Thompson
-1 siblings, 0 replies; 84+ messages in thread
From: Daniel Thompson @ 2015-12-01 15:42 UTC (permalink / raw)
To: tiffany lin
Cc: Rob Herring, Pawel Moll, Mark Rutland, Ian Campbell, Kumar Gala,
Catalin Marinas, Will Deacon, Mauro Carvalho Chehab,
Matthias Brugger, Daniel Kurtz, Sascha Hauer, Hongzhou Yang,
Hans Verkuil, Laurent Pinchart, Sakari Ailus, Mikhail Ulyanov,
Fabien Dessenne, Arnd Bergmann, Darren Etheridge, Peter Griffin,
Benoit Parrot, Andrew-CT Chen (陳智迪),
Eddie Huang (黃智傑),
Yingjoe Chen (陳英洲),
JamesJJ Liao (廖建智),
Daniel Hsiao (蕭伯剛),
devicetree, lkml, linux-arm-kernel, linux-media, linux-mediatek,
PoChun Lin (林柏君)
On 01/12/15 10:42, tiffany lin wrote:
>>>> > diff --git a/drivers/media/platform/mtk-vcodec/common/venc_drv_if.c
>>>> b/drivers/media/platform/mtk-vcodec/common/venc_drv_if.c
>>>> > new file mode 100644
>>>> > index 0000000..9b3f025
>>>> > --- /dev/null
> [snip]
>>>> > +int venc_if_create(void *ctx, unsigned int fourcc, unsigned long
>>>> *handle)
>>>> > +{
>>>> > + struct venc_handle *h;
>>>> > + char str[10];
>>>> > +
>>>> > + mtk_vcodec_fmt2str(fourcc, str);
>>>> > +
>>>> > + h = kzalloc(sizeof(*h), GFP_KERNEL);
>>>> > + if (!h)
>>>> > + return -ENOMEM;
>>>> > +
>>>> > + h->fourcc = fourcc;
>>>> > + h->ctx = ctx;
>>>> > + mtk_vcodec_debug(h, "fmt = %s handle = %p", str, h);
>>>> > +
>>>> > + switch (fourcc) {
>>>> > + default:
>>>> > + mtk_vcodec_err(h, "invalid format %s", str);
>>>> > + goto err_out;
>>>> > + }
>>>> > +
>>>> > + *handle = (unsigned long)h;
>>>> > + return 0;
>>>> > +
>>>> > +err_out:
>>>> > + kfree(h);
>>>> > + return -EINVAL;
>>>> > +}
>>>> > +
>>>> > +int venc_if_init(unsigned long handle)
>>>> > +{
>>>> > + int ret = 0;
>>>> > + struct venc_handle *h = (struct venc_handle *)handle;
>>>> > +
>>>> > + mtk_vcodec_debug_enter(h);
>>>> > +
>>>> > + mtk_venc_lock(h->ctx);
>>>> > + mtk_vcodec_enc_clock_on();
>>>> > + vpu_enable_clock(vpu_get_plat_device(h->ctx->dev->plat_dev));
>>>> > + ret = h->enc_if->init(h->ctx, (unsigned long *)&h->drv_handle);
>>>> > + vpu_disable_clock(vpu_get_plat_device(h->ctx->dev->plat_dev));
>>>> > + mtk_vcodec_enc_clock_off();
>>>> > + mtk_venc_unlock(h->ctx);
>>>> > +
>>>> > + return ret;
>>>> > +}
>>>>
>>>> To me this looks more like an obfuscation layer rather than a
>>>> abstraction layer. I don't understand why we need to hide things from
>>>> the V4L2 implementation that this code forms part of.
>>>>
>>>> More importantly, if this code was included somewhere where it could be
>>>> properly integrated with the device model you might be able to use the
>>>> pm_runtime system to avoid this sort of "heroics" to manage the clocks
>>>> anyway.
>>>>
>>> We want to abstract common part from encoder driver.
>>> Every encoder driver follow same calling flow and only need to take care
>>> about how to communicate with vpu to encode specific format.
>>> Encoder driver do not need to take care clock and multiple instance
>>> issue.
>>
>> Looking at each of those stages:
>>
>> mtk_venc_lock():
>> Why isn't one of the existing V4L2 locking strategies ok for you?
>>
> We only has one encoder hw.
> To support multiple encode instances.
> When one encoder ctx access encoder hw, it need to get lock first.
>
>> mtk_vcodec_enc_clock_on():
>> This does seem like something a sub-driver *should* be doing for itself
> This is for enabling encoder hw related clock.
> To support multiple instances, one encode ctx must get hw lock first
> then clock on/off hw relate clock.
>
>> vpu_enable_clock():
>> Why can't the VPU driver manage this internally using pm_runtime?
>>
> Our VPU do not have power domain.
> We will remove VPU clock on/off and let vpu control it in next version.
>
>>
>> That is why I described this as an obfuscation layer. It is collecting
>> a bunch of stuff that can be handled using the kernel driver model and
>> clumping them together in a special middle layer.
>>
> We do use kernel driver model, but we put it in
> mtk_vcodec_enc_clock_on/mtk_vcodec_enc_clock_off.
> Every sub-driver has no need to write the same code.
> And once clock configuration change or porting to other chips, we don't
> need to change sub-driver one-by-one, just change abstract layer.
I'm afraid I remain extremely unconvinced by the value of this API. It
is possible that once the types are fixed and it is tidied up it won't
stick out so much but I will be very surprised.
Either way, I can wait until v2 before we discuss it further.
>>>> If the start streaming operation implemented cleanup-on-error properly
>>>> then there would only be two useful states: Started and stopped. Even
>>>> the "sticky" error behavior looks unnecessary to me (meaning we don't
>>>> need to track its state).
>>>>
>>> We cannot guaranteed that IOCTLs called from the user space follow
>>> required sequence.
>>> We need states to know if our driver could accept IOCTL command.
>>
>> I believe that knowing whether the streaming is started or stopped
>> (e.g. two states) is sufficient for a driver to correctly handle
>> abitrary ioctls from userspace and even then, the core code tracks
>> this state for you so there's no need for you do it.
>>
>> The queue/dequeue ioctls succeed or fail based on the length of the
>> queue (i.e. is the buffer queue overflowing or not) and have no need
>> to check the streaming state.
>
>> If you are absolutely sure that the other states are needed then
>> please provide an example of an ioctl() sequence where the additional
>> state is needed.
>>
> I know your point that we have too many state changes in start_streaming
> and stop_streaming function.
> We will refine these two functions in next version.
>
> For the example, we need MTK_STATE_HEADER state, to make sure before
> encode start, driver already get information to set encode parameters.
Interesting. Again, I'll wait to see how the state simplifcation goes
before commenting further.
> We need MTK_STATE_ABORT to inform encoder thread (mtk_venc_worker) that
> stop encodeing job from stopped ctx instance.
> When user space qbuf, we need to make sure everything is ready to sent
> buf to encode.
Agree that you need a flag here. In fact currently you have two,
MTK_STATE_ABORT and an unused one called aborting.
You need to be very careful with these flags though. They are a magnet
for data race bugs (especially combined with SMP).
For example at present I can't see any locking in the worker code. This
means there is nothing to make all those read-modify-write sequences
that manage the state atomic (thus risking state corruption).
Daniel.
^ permalink raw reply [flat|nested] 84+ messages in thread
* Re: [RESEND RFC/PATCH 6/8] media: platform: mtk-vcodec: Add Mediatek V4L2 Video Encoder Driver
@ 2015-12-01 15:42 ` Daniel Thompson
0 siblings, 0 replies; 84+ messages in thread
From: Daniel Thompson @ 2015-12-01 15:42 UTC (permalink / raw)
To: tiffany lin
Cc: Rob Herring, Pawel Moll, Mark Rutland, Ian Campbell, Kumar Gala,
Catalin Marinas, Will Deacon, Mauro Carvalho Chehab,
Matthias Brugger, Daniel Kurtz, Sascha Hauer, Hongzhou Yang,
Hans Verkuil, Laurent Pinchart, Sakari Ailus, Mikhail Ulyanov,
Fabien Dessenne, Arnd Bergmann, Darren Etheridge, Peter Griffin,
Benoit Parrot
On 01/12/15 10:42, tiffany lin wrote:
>>>> > diff --git a/drivers/media/platform/mtk-vcodec/common/venc_drv_if.c
>>>> b/drivers/media/platform/mtk-vcodec/common/venc_drv_if.c
>>>> > new file mode 100644
>>>> > index 0000000..9b3f025
>>>> > --- /dev/null
> [snip]
>>>> > +int venc_if_create(void *ctx, unsigned int fourcc, unsigned long
>>>> *handle)
>>>> > +{
>>>> > + struct venc_handle *h;
>>>> > + char str[10];
>>>> > +
>>>> > + mtk_vcodec_fmt2str(fourcc, str);
>>>> > +
>>>> > + h = kzalloc(sizeof(*h), GFP_KERNEL);
>>>> > + if (!h)
>>>> > + return -ENOMEM;
>>>> > +
>>>> > + h->fourcc = fourcc;
>>>> > + h->ctx = ctx;
>>>> > + mtk_vcodec_debug(h, "fmt = %s handle = %p", str, h);
>>>> > +
>>>> > + switch (fourcc) {
>>>> > + default:
>>>> > + mtk_vcodec_err(h, "invalid format %s", str);
>>>> > + goto err_out;
>>>> > + }
>>>> > +
>>>> > + *handle = (unsigned long)h;
>>>> > + return 0;
>>>> > +
>>>> > +err_out:
>>>> > + kfree(h);
>>>> > + return -EINVAL;
>>>> > +}
>>>> > +
>>>> > +int venc_if_init(unsigned long handle)
>>>> > +{
>>>> > + int ret = 0;
>>>> > + struct venc_handle *h = (struct venc_handle *)handle;
>>>> > +
>>>> > + mtk_vcodec_debug_enter(h);
>>>> > +
>>>> > + mtk_venc_lock(h->ctx);
>>>> > + mtk_vcodec_enc_clock_on();
>>>> > + vpu_enable_clock(vpu_get_plat_device(h->ctx->dev->plat_dev));
>>>> > + ret = h->enc_if->init(h->ctx, (unsigned long *)&h->drv_handle);
>>>> > + vpu_disable_clock(vpu_get_plat_device(h->ctx->dev->plat_dev));
>>>> > + mtk_vcodec_enc_clock_off();
>>>> > + mtk_venc_unlock(h->ctx);
>>>> > +
>>>> > + return ret;
>>>> > +}
>>>>
>>>> To me this looks more like an obfuscation layer rather than a
>>>> abstraction layer. I don't understand why we need to hide things from
>>>> the V4L2 implementation that this code forms part of.
>>>>
>>>> More importantly, if this code was included somewhere where it could be
>>>> properly integrated with the device model you might be able to use the
>>>> pm_runtime system to avoid this sort of "heroics" to manage the clocks
>>>> anyway.
>>>>
>>> We want to abstract common part from encoder driver.
>>> Every encoder driver follow same calling flow and only need to take care
>>> about how to communicate with vpu to encode specific format.
>>> Encoder driver do not need to take care clock and multiple instance
>>> issue.
>>
>> Looking at each of those stages:
>>
>> mtk_venc_lock():
>> Why isn't one of the existing V4L2 locking strategies ok for you?
>>
> We only has one encoder hw.
> To support multiple encode instances.
> When one encoder ctx access encoder hw, it need to get lock first.
>
>> mtk_vcodec_enc_clock_on():
>> This does seem like something a sub-driver *should* be doing for itself
> This is for enabling encoder hw related clock.
> To support multiple instances, one encode ctx must get hw lock first
> then clock on/off hw relate clock.
>
>> vpu_enable_clock():
>> Why can't the VPU driver manage this internally using pm_runtime?
>>
> Our VPU do not have power domain.
> We will remove VPU clock on/off and let vpu control it in next version.
>
>>
>> That is why I described this as an obfuscation layer. It is collecting
>> a bunch of stuff that can be handled using the kernel driver model and
>> clumping them together in a special middle layer.
>>
> We do use kernel driver model, but we put it in
> mtk_vcodec_enc_clock_on/mtk_vcodec_enc_clock_off.
> Every sub-driver has no need to write the same code.
> And once clock configuration change or porting to other chips, we don't
> need to change sub-driver one-by-one, just change abstract layer.
I'm afraid I remain extremely unconvinced by the value of this API. It
is possible that once the types are fixed and it is tidied up it won't
stick out so much but I will be very surprised.
Either way, I can wait until v2 before we discuss it further.
>>>> If the start streaming operation implemented cleanup-on-error properly
>>>> then there would only be two useful states: Started and stopped. Even
>>>> the "sticky" error behavior looks unnecessary to me (meaning we don't
>>>> need to track its state).
>>>>
>>> We cannot guaranteed that IOCTLs called from the user space follow
>>> required sequence.
>>> We need states to know if our driver could accept IOCTL command.
>>
>> I believe that knowing whether the streaming is started or stopped
>> (e.g. two states) is sufficient for a driver to correctly handle
>> abitrary ioctls from userspace and even then, the core code tracks
>> this state for you so there's no need for you do it.
>>
>> The queue/dequeue ioctls succeed or fail based on the length of the
>> queue (i.e. is the buffer queue overflowing or not) and have no need
>> to check the streaming state.
>
>> If you are absolutely sure that the other states are needed then
>> please provide an example of an ioctl() sequence where the additional
>> state is needed.
>>
> I know your point that we have too many state changes in start_streaming
> and stop_streaming function.
> We will refine these two functions in next version.
>
> For the example, we need MTK_STATE_HEADER state, to make sure before
> encode start, driver already get information to set encode parameters.
Interesting. Again, I'll wait to see how the state simplifcation goes
before commenting further.
> We need MTK_STATE_ABORT to inform encoder thread (mtk_venc_worker) that
> stop encodeing job from stopped ctx instance.
> When user space qbuf, we need to make sure everything is ready to sent
> buf to encode.
Agree that you need a flag here. In fact currently you have two,
MTK_STATE_ABORT and an unused one called aborting.
You need to be very careful with these flags though. They are a magnet
for data race bugs (especially combined with SMP).
For example at present I can't see any locking in the worker code. This
means there is nothing to make all those read-modify-write sequences
that manage the state atomic (thus risking state corruption).
Daniel.
--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 84+ messages in thread
* [RESEND RFC/PATCH 6/8] media: platform: mtk-vcodec: Add Mediatek V4L2 Video Encoder Driver
@ 2015-12-01 15:42 ` Daniel Thompson
0 siblings, 0 replies; 84+ messages in thread
From: Daniel Thompson @ 2015-12-01 15:42 UTC (permalink / raw)
To: linux-arm-kernel
On 01/12/15 10:42, tiffany lin wrote:
>>>> > diff --git a/drivers/media/platform/mtk-vcodec/common/venc_drv_if.c
>>>> b/drivers/media/platform/mtk-vcodec/common/venc_drv_if.c
>>>> > new file mode 100644
>>>> > index 0000000..9b3f025
>>>> > --- /dev/null
> [snip]
>>>> > +int venc_if_create(void *ctx, unsigned int fourcc, unsigned long
>>>> *handle)
>>>> > +{
>>>> > + struct venc_handle *h;
>>>> > + char str[10];
>>>> > +
>>>> > + mtk_vcodec_fmt2str(fourcc, str);
>>>> > +
>>>> > + h = kzalloc(sizeof(*h), GFP_KERNEL);
>>>> > + if (!h)
>>>> > + return -ENOMEM;
>>>> > +
>>>> > + h->fourcc = fourcc;
>>>> > + h->ctx = ctx;
>>>> > + mtk_vcodec_debug(h, "fmt = %s handle = %p", str, h);
>>>> > +
>>>> > + switch (fourcc) {
>>>> > + default:
>>>> > + mtk_vcodec_err(h, "invalid format %s", str);
>>>> > + goto err_out;
>>>> > + }
>>>> > +
>>>> > + *handle = (unsigned long)h;
>>>> > + return 0;
>>>> > +
>>>> > +err_out:
>>>> > + kfree(h);
>>>> > + return -EINVAL;
>>>> > +}
>>>> > +
>>>> > +int venc_if_init(unsigned long handle)
>>>> > +{
>>>> > + int ret = 0;
>>>> > + struct venc_handle *h = (struct venc_handle *)handle;
>>>> > +
>>>> > + mtk_vcodec_debug_enter(h);
>>>> > +
>>>> > + mtk_venc_lock(h->ctx);
>>>> > + mtk_vcodec_enc_clock_on();
>>>> > + vpu_enable_clock(vpu_get_plat_device(h->ctx->dev->plat_dev));
>>>> > + ret = h->enc_if->init(h->ctx, (unsigned long *)&h->drv_handle);
>>>> > + vpu_disable_clock(vpu_get_plat_device(h->ctx->dev->plat_dev));
>>>> > + mtk_vcodec_enc_clock_off();
>>>> > + mtk_venc_unlock(h->ctx);
>>>> > +
>>>> > + return ret;
>>>> > +}
>>>>
>>>> To me this looks more like an obfuscation layer rather than a
>>>> abstraction layer. I don't understand why we need to hide things from
>>>> the V4L2 implementation that this code forms part of.
>>>>
>>>> More importantly, if this code was included somewhere where it could be
>>>> properly integrated with the device model you might be able to use the
>>>> pm_runtime system to avoid this sort of "heroics" to manage the clocks
>>>> anyway.
>>>>
>>> We want to abstract common part from encoder driver.
>>> Every encoder driver follow same calling flow and only need to take care
>>> about how to communicate with vpu to encode specific format.
>>> Encoder driver do not need to take care clock and multiple instance
>>> issue.
>>
>> Looking at each of those stages:
>>
>> mtk_venc_lock():
>> Why isn't one of the existing V4L2 locking strategies ok for you?
>>
> We only has one encoder hw.
> To support multiple encode instances.
> When one encoder ctx access encoder hw, it need to get lock first.
>
>> mtk_vcodec_enc_clock_on():
>> This does seem like something a sub-driver *should* be doing for itself
> This is for enabling encoder hw related clock.
> To support multiple instances, one encode ctx must get hw lock first
> then clock on/off hw relate clock.
>
>> vpu_enable_clock():
>> Why can't the VPU driver manage this internally using pm_runtime?
>>
> Our VPU do not have power domain.
> We will remove VPU clock on/off and let vpu control it in next version.
>
>>
>> That is why I described this as an obfuscation layer. It is collecting
>> a bunch of stuff that can be handled using the kernel driver model and
>> clumping them together in a special middle layer.
>>
> We do use kernel driver model, but we put it in
> mtk_vcodec_enc_clock_on/mtk_vcodec_enc_clock_off.
> Every sub-driver has no need to write the same code.
> And once clock configuration change or porting to other chips, we don't
> need to change sub-driver one-by-one, just change abstract layer.
I'm afraid I remain extremely unconvinced by the value of this API. It
is possible that once the types are fixed and it is tidied up it won't
stick out so much but I will be very surprised.
Either way, I can wait until v2 before we discuss it further.
>>>> If the start streaming operation implemented cleanup-on-error properly
>>>> then there would only be two useful states: Started and stopped. Even
>>>> the "sticky" error behavior looks unnecessary to me (meaning we don't
>>>> need to track its state).
>>>>
>>> We cannot guaranteed that IOCTLs called from the user space follow
>>> required sequence.
>>> We need states to know if our driver could accept IOCTL command.
>>
>> I believe that knowing whether the streaming is started or stopped
>> (e.g. two states) is sufficient for a driver to correctly handle
>> abitrary ioctls from userspace and even then, the core code tracks
>> this state for you so there's no need for you do it.
>>
>> The queue/dequeue ioctls succeed or fail based on the length of the
>> queue (i.e. is the buffer queue overflowing or not) and have no need
>> to check the streaming state.
>
>> If you are absolutely sure that the other states are needed then
>> please provide an example of an ioctl() sequence where the additional
>> state is needed.
>>
> I know your point that we have too many state changes in start_streaming
> and stop_streaming function.
> We will refine these two functions in next version.
>
> For the example, we need MTK_STATE_HEADER state, to make sure before
> encode start, driver already get information to set encode parameters.
Interesting. Again, I'll wait to see how the state simplifcation goes
before commenting further.
> We need MTK_STATE_ABORT to inform encoder thread (mtk_venc_worker) that
> stop encodeing job from stopped ctx instance.
> When user space qbuf, we need to make sure everything is ready to sent
> buf to encode.
Agree that you need a flag here. In fact currently you have two,
MTK_STATE_ABORT and an unused one called aborting.
You need to be very careful with these flags though. They are a magnet
for data race bugs (especially combined with SMP).
For example at present I can't see any locking in the worker code. This
means there is nothing to make all those read-modify-write sequences
that manage the state atomic (thus risking state corruption).
Daniel.
^ permalink raw reply [flat|nested] 84+ messages in thread
* Re: [RESEND RFC/PATCH 6/8] media: platform: mtk-vcodec: Add Mediatek V4L2 Video Encoder Driver
2015-12-01 15:42 ` Daniel Thompson
(?)
@ 2015-12-02 13:08 ` tiffany lin
-1 siblings, 0 replies; 84+ messages in thread
From: tiffany lin @ 2015-12-02 13:08 UTC (permalink / raw)
To: Daniel Thompson
Cc: Rob Herring, Pawel Moll, Mark Rutland, Ian Campbell, Kumar Gala,
Catalin Marinas, Will Deacon, Mauro Carvalho Chehab,
Matthias Brugger, Daniel Kurtz, Sascha Hauer, Hongzhou Yang,
Hans Verkuil, Laurent Pinchart, Sakari Ailus, Mikhail Ulyanov,
Fabien Dessenne, Arnd Bergmann, Darren Etheridge, Peter Griffin,
Benoit Parrot, Andrew-CT Chen (陳智迪),
Eddie Huang (黃智傑),
Yingjoe Chen (陳英洲),
JamesJJ Liao (廖建智),
Daniel Hsiao (蕭伯剛),
devicetree, lkml, linux-arm-kernel, linux-media, linux-mediatek,
PoChun Lin (林柏君)
On Tue, 2015-12-01 at 15:42 +0000, Daniel Thompson wrote:
> On 01/12/15 10:42, tiffany lin wrote:
> >>>> > diff --git a/drivers/media/platform/mtk-vcodec/common/venc_drv_if.c
> >>>> b/drivers/media/platform/mtk-vcodec/common/venc_drv_if.c
> >>>> > new file mode 100644
> >>>> > index 0000000..9b3f025
> >>>> > --- /dev/null
> > [snip]
> >>>> > +int venc_if_create(void *ctx, unsigned int fourcc, unsigned long
> >>>> *handle)
> >>>> > +{
> >>>> > + struct venc_handle *h;
> >>>> > + char str[10];
> >>>> > +
> >>>> > + mtk_vcodec_fmt2str(fourcc, str);
> >>>> > +
> >>>> > + h = kzalloc(sizeof(*h), GFP_KERNEL);
> >>>> > + if (!h)
> >>>> > + return -ENOMEM;
> >>>> > +
> >>>> > + h->fourcc = fourcc;
> >>>> > + h->ctx = ctx;
> >>>> > + mtk_vcodec_debug(h, "fmt = %s handle = %p", str, h);
> >>>> > +
> >>>> > + switch (fourcc) {
> >>>> > + default:
> >>>> > + mtk_vcodec_err(h, "invalid format %s", str);
> >>>> > + goto err_out;
> >>>> > + }
> >>>> > +
> >>>> > + *handle = (unsigned long)h;
> >>>> > + return 0;
> >>>> > +
> >>>> > +err_out:
> >>>> > + kfree(h);
> >>>> > + return -EINVAL;
> >>>> > +}
> >>>> > +
> >>>> > +int venc_if_init(unsigned long handle)
> >>>> > +{
> >>>> > + int ret = 0;
> >>>> > + struct venc_handle *h = (struct venc_handle *)handle;
> >>>> > +
> >>>> > + mtk_vcodec_debug_enter(h);
> >>>> > +
> >>>> > + mtk_venc_lock(h->ctx);
> >>>> > + mtk_vcodec_enc_clock_on();
> >>>> > + vpu_enable_clock(vpu_get_plat_device(h->ctx->dev->plat_dev));
> >>>> > + ret = h->enc_if->init(h->ctx, (unsigned long *)&h->drv_handle);
> >>>> > + vpu_disable_clock(vpu_get_plat_device(h->ctx->dev->plat_dev));
> >>>> > + mtk_vcodec_enc_clock_off();
> >>>> > + mtk_venc_unlock(h->ctx);
> >>>> > +
> >>>> > + return ret;
> >>>> > +}
> >>>>
> >>>> To me this looks more like an obfuscation layer rather than a
> >>>> abstraction layer. I don't understand why we need to hide things from
> >>>> the V4L2 implementation that this code forms part of.
> >>>>
> >>>> More importantly, if this code was included somewhere where it could be
> >>>> properly integrated with the device model you might be able to use the
> >>>> pm_runtime system to avoid this sort of "heroics" to manage the clocks
> >>>> anyway.
> >>>>
> >>> We want to abstract common part from encoder driver.
> >>> Every encoder driver follow same calling flow and only need to take care
> >>> about how to communicate with vpu to encode specific format.
> >>> Encoder driver do not need to take care clock and multiple instance
> >>> issue.
> >>
> >> Looking at each of those stages:
> >>
> >> mtk_venc_lock():
> >> Why isn't one of the existing V4L2 locking strategies ok for you?
> >>
> > We only has one encoder hw.
> > To support multiple encode instances.
> > When one encoder ctx access encoder hw, it need to get lock first.
> >
> >> mtk_vcodec_enc_clock_on():
> >> This does seem like something a sub-driver *should* be doing for itself
> > This is for enabling encoder hw related clock.
> > To support multiple instances, one encode ctx must get hw lock first
> > then clock on/off hw relate clock.
> >
> >> vpu_enable_clock():
> >> Why can't the VPU driver manage this internally using pm_runtime?
> >>
> > Our VPU do not have power domain.
> > We will remove VPU clock on/off and let vpu control it in next version.
> >
> >>
> >> That is why I described this as an obfuscation layer. It is collecting
> >> a bunch of stuff that can be handled using the kernel driver model and
> >> clumping them together in a special middle layer.
> >>
> > We do use kernel driver model, but we put it in
> > mtk_vcodec_enc_clock_on/mtk_vcodec_enc_clock_off.
> > Every sub-driver has no need to write the same code.
> > And once clock configuration change or porting to other chips, we don't
> > need to change sub-driver one-by-one, just change abstract layer.
>
> I'm afraid I remain extremely unconvinced by the value of this API. It
> is possible that once the types are fixed and it is tidied up it won't
> stick out so much but I will be very surprised.
>
> Either way, I can wait until v2 before we discuss it further.
>
>
> >>>> If the start streaming operation implemented cleanup-on-error properly
> >>>> then there would only be two useful states: Started and stopped. Even
> >>>> the "sticky" error behavior looks unnecessary to me (meaning we don't
> >>>> need to track its state).
> >>>>
> >>> We cannot guaranteed that IOCTLs called from the user space follow
> >>> required sequence.
> >>> We need states to know if our driver could accept IOCTL command.
> >>
> >> I believe that knowing whether the streaming is started or stopped
> >> (e.g. two states) is sufficient for a driver to correctly handle
> >> abitrary ioctls from userspace and even then, the core code tracks
> >> this state for you so there's no need for you do it.
> >>
> >> The queue/dequeue ioctls succeed or fail based on the length of the
> >> queue (i.e. is the buffer queue overflowing or not) and have no need
> >> to check the streaming state.
> >
> >> If you are absolutely sure that the other states are needed then
> >> please provide an example of an ioctl() sequence where the additional
> >> state is needed.
> >>
> > I know your point that we have too many state changes in start_streaming
> > and stop_streaming function.
> > We will refine these two functions in next version.
> >
> > For the example, we need MTK_STATE_HEADER state, to make sure before
> > encode start, driver already get information to set encode parameters.
>
> Interesting. Again, I'll wait to see how the state simplifcation goes
> before commenting further.
>
>
> > We need MTK_STATE_ABORT to inform encoder thread (mtk_venc_worker) that
> > stop encodeing job from stopped ctx instance.
> > When user space qbuf, we need to make sure everything is ready to sent
> > buf to encode.
>
> Agree that you need a flag here. In fact currently you have two,
> MTK_STATE_ABORT and an unused one called aborting.
>
> You need to be very careful with these flags though. They are a magnet
> for data race bugs (especially combined with SMP).
>
> For example at present I can't see any locking in the worker code. This
> means there is nothing to make all those read-modify-write sequences
> that manage the state atomic (thus risking state corruption).
>
We prevent that one function set the flag and others clear the flag.
So there is no special lock to protect state.
>
> Daniel.
^ permalink raw reply [flat|nested] 84+ messages in thread
* Re: [RESEND RFC/PATCH 6/8] media: platform: mtk-vcodec: Add Mediatek V4L2 Video Encoder Driver
@ 2015-12-02 13:08 ` tiffany lin
0 siblings, 0 replies; 84+ messages in thread
From: tiffany lin @ 2015-12-02 13:08 UTC (permalink / raw)
To: Daniel Thompson
Cc: Rob Herring, Pawel Moll, Mark Rutland, Ian Campbell, Kumar Gala,
Catalin Marinas, Will Deacon, Mauro Carvalho Chehab,
Matthias Brugger, Daniel Kurtz, Sascha Hauer, Hongzhou Yang,
Hans Verkuil, Laurent Pinchart, Sakari Ailus, Mikhail Ulyanov,
Fabien Dessenne, Arnd Bergmann, Darren Etheridge, Peter Griffin,
Benoit Parrot, Andrew-CT
On Tue, 2015-12-01 at 15:42 +0000, Daniel Thompson wrote:
> On 01/12/15 10:42, tiffany lin wrote:
> >>>> > diff --git a/drivers/media/platform/mtk-vcodec/common/venc_drv_if.c
> >>>> b/drivers/media/platform/mtk-vcodec/common/venc_drv_if.c
> >>>> > new file mode 100644
> >>>> > index 0000000..9b3f025
> >>>> > --- /dev/null
> > [snip]
> >>>> > +int venc_if_create(void *ctx, unsigned int fourcc, unsigned long
> >>>> *handle)
> >>>> > +{
> >>>> > + struct venc_handle *h;
> >>>> > + char str[10];
> >>>> > +
> >>>> > + mtk_vcodec_fmt2str(fourcc, str);
> >>>> > +
> >>>> > + h = kzalloc(sizeof(*h), GFP_KERNEL);
> >>>> > + if (!h)
> >>>> > + return -ENOMEM;
> >>>> > +
> >>>> > + h->fourcc = fourcc;
> >>>> > + h->ctx = ctx;
> >>>> > + mtk_vcodec_debug(h, "fmt = %s handle = %p", str, h);
> >>>> > +
> >>>> > + switch (fourcc) {
> >>>> > + default:
> >>>> > + mtk_vcodec_err(h, "invalid format %s", str);
> >>>> > + goto err_out;
> >>>> > + }
> >>>> > +
> >>>> > + *handle = (unsigned long)h;
> >>>> > + return 0;
> >>>> > +
> >>>> > +err_out:
> >>>> > + kfree(h);
> >>>> > + return -EINVAL;
> >>>> > +}
> >>>> > +
> >>>> > +int venc_if_init(unsigned long handle)
> >>>> > +{
> >>>> > + int ret = 0;
> >>>> > + struct venc_handle *h = (struct venc_handle *)handle;
> >>>> > +
> >>>> > + mtk_vcodec_debug_enter(h);
> >>>> > +
> >>>> > + mtk_venc_lock(h->ctx);
> >>>> > + mtk_vcodec_enc_clock_on();
> >>>> > + vpu_enable_clock(vpu_get_plat_device(h->ctx->dev->plat_dev));
> >>>> > + ret = h->enc_if->init(h->ctx, (unsigned long *)&h->drv_handle);
> >>>> > + vpu_disable_clock(vpu_get_plat_device(h->ctx->dev->plat_dev));
> >>>> > + mtk_vcodec_enc_clock_off();
> >>>> > + mtk_venc_unlock(h->ctx);
> >>>> > +
> >>>> > + return ret;
> >>>> > +}
> >>>>
> >>>> To me this looks more like an obfuscation layer rather than a
> >>>> abstraction layer. I don't understand why we need to hide things from
> >>>> the V4L2 implementation that this code forms part of.
> >>>>
> >>>> More importantly, if this code was included somewhere where it could be
> >>>> properly integrated with the device model you might be able to use the
> >>>> pm_runtime system to avoid this sort of "heroics" to manage the clocks
> >>>> anyway.
> >>>>
> >>> We want to abstract common part from encoder driver.
> >>> Every encoder driver follow same calling flow and only need to take care
> >>> about how to communicate with vpu to encode specific format.
> >>> Encoder driver do not need to take care clock and multiple instance
> >>> issue.
> >>
> >> Looking at each of those stages:
> >>
> >> mtk_venc_lock():
> >> Why isn't one of the existing V4L2 locking strategies ok for you?
> >>
> > We only has one encoder hw.
> > To support multiple encode instances.
> > When one encoder ctx access encoder hw, it need to get lock first.
> >
> >> mtk_vcodec_enc_clock_on():
> >> This does seem like something a sub-driver *should* be doing for itself
> > This is for enabling encoder hw related clock.
> > To support multiple instances, one encode ctx must get hw lock first
> > then clock on/off hw relate clock.
> >
> >> vpu_enable_clock():
> >> Why can't the VPU driver manage this internally using pm_runtime?
> >>
> > Our VPU do not have power domain.
> > We will remove VPU clock on/off and let vpu control it in next version.
> >
> >>
> >> That is why I described this as an obfuscation layer. It is collecting
> >> a bunch of stuff that can be handled using the kernel driver model and
> >> clumping them together in a special middle layer.
> >>
> > We do use kernel driver model, but we put it in
> > mtk_vcodec_enc_clock_on/mtk_vcodec_enc_clock_off.
> > Every sub-driver has no need to write the same code.
> > And once clock configuration change or porting to other chips, we don't
> > need to change sub-driver one-by-one, just change abstract layer.
>
> I'm afraid I remain extremely unconvinced by the value of this API. It
> is possible that once the types are fixed and it is tidied up it won't
> stick out so much but I will be very surprised.
>
> Either way, I can wait until v2 before we discuss it further.
>
>
> >>>> If the start streaming operation implemented cleanup-on-error properly
> >>>> then there would only be two useful states: Started and stopped. Even
> >>>> the "sticky" error behavior looks unnecessary to me (meaning we don't
> >>>> need to track its state).
> >>>>
> >>> We cannot guaranteed that IOCTLs called from the user space follow
> >>> required sequence.
> >>> We need states to know if our driver could accept IOCTL command.
> >>
> >> I believe that knowing whether the streaming is started or stopped
> >> (e.g. two states) is sufficient for a driver to correctly handle
> >> abitrary ioctls from userspace and even then, the core code tracks
> >> this state for you so there's no need for you do it.
> >>
> >> The queue/dequeue ioctls succeed or fail based on the length of the
> >> queue (i.e. is the buffer queue overflowing or not) and have no need
> >> to check the streaming state.
> >
> >> If you are absolutely sure that the other states are needed then
> >> please provide an example of an ioctl() sequence where the additional
> >> state is needed.
> >>
> > I know your point that we have too many state changes in start_streaming
> > and stop_streaming function.
> > We will refine these two functions in next version.
> >
> > For the example, we need MTK_STATE_HEADER state, to make sure before
> > encode start, driver already get information to set encode parameters.
>
> Interesting. Again, I'll wait to see how the state simplifcation goes
> before commenting further.
>
>
> > We need MTK_STATE_ABORT to inform encoder thread (mtk_venc_worker) that
> > stop encodeing job from stopped ctx instance.
> > When user space qbuf, we need to make sure everything is ready to sent
> > buf to encode.
>
> Agree that you need a flag here. In fact currently you have two,
> MTK_STATE_ABORT and an unused one called aborting.
>
> You need to be very careful with these flags though. They are a magnet
> for data race bugs (especially combined with SMP).
>
> For example at present I can't see any locking in the worker code. This
> means there is nothing to make all those read-modify-write sequences
> that manage the state atomic (thus risking state corruption).
>
We prevent that one function set the flag and others clear the flag.
So there is no special lock to protect state.
>
> Daniel.
^ permalink raw reply [flat|nested] 84+ messages in thread
* [RESEND RFC/PATCH 6/8] media: platform: mtk-vcodec: Add Mediatek V4L2 Video Encoder Driver
@ 2015-12-02 13:08 ` tiffany lin
0 siblings, 0 replies; 84+ messages in thread
From: tiffany lin @ 2015-12-02 13:08 UTC (permalink / raw)
To: linux-arm-kernel
On Tue, 2015-12-01 at 15:42 +0000, Daniel Thompson wrote:
> On 01/12/15 10:42, tiffany lin wrote:
> >>>> > diff --git a/drivers/media/platform/mtk-vcodec/common/venc_drv_if.c
> >>>> b/drivers/media/platform/mtk-vcodec/common/venc_drv_if.c
> >>>> > new file mode 100644
> >>>> > index 0000000..9b3f025
> >>>> > --- /dev/null
> > [snip]
> >>>> > +int venc_if_create(void *ctx, unsigned int fourcc, unsigned long
> >>>> *handle)
> >>>> > +{
> >>>> > + struct venc_handle *h;
> >>>> > + char str[10];
> >>>> > +
> >>>> > + mtk_vcodec_fmt2str(fourcc, str);
> >>>> > +
> >>>> > + h = kzalloc(sizeof(*h), GFP_KERNEL);
> >>>> > + if (!h)
> >>>> > + return -ENOMEM;
> >>>> > +
> >>>> > + h->fourcc = fourcc;
> >>>> > + h->ctx = ctx;
> >>>> > + mtk_vcodec_debug(h, "fmt = %s handle = %p", str, h);
> >>>> > +
> >>>> > + switch (fourcc) {
> >>>> > + default:
> >>>> > + mtk_vcodec_err(h, "invalid format %s", str);
> >>>> > + goto err_out;
> >>>> > + }
> >>>> > +
> >>>> > + *handle = (unsigned long)h;
> >>>> > + return 0;
> >>>> > +
> >>>> > +err_out:
> >>>> > + kfree(h);
> >>>> > + return -EINVAL;
> >>>> > +}
> >>>> > +
> >>>> > +int venc_if_init(unsigned long handle)
> >>>> > +{
> >>>> > + int ret = 0;
> >>>> > + struct venc_handle *h = (struct venc_handle *)handle;
> >>>> > +
> >>>> > + mtk_vcodec_debug_enter(h);
> >>>> > +
> >>>> > + mtk_venc_lock(h->ctx);
> >>>> > + mtk_vcodec_enc_clock_on();
> >>>> > + vpu_enable_clock(vpu_get_plat_device(h->ctx->dev->plat_dev));
> >>>> > + ret = h->enc_if->init(h->ctx, (unsigned long *)&h->drv_handle);
> >>>> > + vpu_disable_clock(vpu_get_plat_device(h->ctx->dev->plat_dev));
> >>>> > + mtk_vcodec_enc_clock_off();
> >>>> > + mtk_venc_unlock(h->ctx);
> >>>> > +
> >>>> > + return ret;
> >>>> > +}
> >>>>
> >>>> To me this looks more like an obfuscation layer rather than a
> >>>> abstraction layer. I don't understand why we need to hide things from
> >>>> the V4L2 implementation that this code forms part of.
> >>>>
> >>>> More importantly, if this code was included somewhere where it could be
> >>>> properly integrated with the device model you might be able to use the
> >>>> pm_runtime system to avoid this sort of "heroics" to manage the clocks
> >>>> anyway.
> >>>>
> >>> We want to abstract common part from encoder driver.
> >>> Every encoder driver follow same calling flow and only need to take care
> >>> about how to communicate with vpu to encode specific format.
> >>> Encoder driver do not need to take care clock and multiple instance
> >>> issue.
> >>
> >> Looking at each of those stages:
> >>
> >> mtk_venc_lock():
> >> Why isn't one of the existing V4L2 locking strategies ok for you?
> >>
> > We only has one encoder hw.
> > To support multiple encode instances.
> > When one encoder ctx access encoder hw, it need to get lock first.
> >
> >> mtk_vcodec_enc_clock_on():
> >> This does seem like something a sub-driver *should* be doing for itself
> > This is for enabling encoder hw related clock.
> > To support multiple instances, one encode ctx must get hw lock first
> > then clock on/off hw relate clock.
> >
> >> vpu_enable_clock():
> >> Why can't the VPU driver manage this internally using pm_runtime?
> >>
> > Our VPU do not have power domain.
> > We will remove VPU clock on/off and let vpu control it in next version.
> >
> >>
> >> That is why I described this as an obfuscation layer. It is collecting
> >> a bunch of stuff that can be handled using the kernel driver model and
> >> clumping them together in a special middle layer.
> >>
> > We do use kernel driver model, but we put it in
> > mtk_vcodec_enc_clock_on/mtk_vcodec_enc_clock_off.
> > Every sub-driver has no need to write the same code.
> > And once clock configuration change or porting to other chips, we don't
> > need to change sub-driver one-by-one, just change abstract layer.
>
> I'm afraid I remain extremely unconvinced by the value of this API. It
> is possible that once the types are fixed and it is tidied up it won't
> stick out so much but I will be very surprised.
>
> Either way, I can wait until v2 before we discuss it further.
>
>
> >>>> If the start streaming operation implemented cleanup-on-error properly
> >>>> then there would only be two useful states: Started and stopped. Even
> >>>> the "sticky" error behavior looks unnecessary to me (meaning we don't
> >>>> need to track its state).
> >>>>
> >>> We cannot guaranteed that IOCTLs called from the user space follow
> >>> required sequence.
> >>> We need states to know if our driver could accept IOCTL command.
> >>
> >> I believe that knowing whether the streaming is started or stopped
> >> (e.g. two states) is sufficient for a driver to correctly handle
> >> abitrary ioctls from userspace and even then, the core code tracks
> >> this state for you so there's no need for you do it.
> >>
> >> The queue/dequeue ioctls succeed or fail based on the length of the
> >> queue (i.e. is the buffer queue overflowing or not) and have no need
> >> to check the streaming state.
> >
> >> If you are absolutely sure that the other states are needed then
> >> please provide an example of an ioctl() sequence where the additional
> >> state is needed.
> >>
> > I know your point that we have too many state changes in start_streaming
> > and stop_streaming function.
> > We will refine these two functions in next version.
> >
> > For the example, we need MTK_STATE_HEADER state, to make sure before
> > encode start, driver already get information to set encode parameters.
>
> Interesting. Again, I'll wait to see how the state simplifcation goes
> before commenting further.
>
>
> > We need MTK_STATE_ABORT to inform encoder thread (mtk_venc_worker) that
> > stop encodeing job from stopped ctx instance.
> > When user space qbuf, we need to make sure everything is ready to sent
> > buf to encode.
>
> Agree that you need a flag here. In fact currently you have two,
> MTK_STATE_ABORT and an unused one called aborting.
>
> You need to be very careful with these flags though. They are a magnet
> for data race bugs (especially combined with SMP).
>
> For example at present I can't see any locking in the worker code. This
> means there is nothing to make all those read-modify-write sequences
> that manage the state atomic (thus risking state corruption).
>
We prevent that one function set the flag and others clear the flag.
So there is no special lock to protect state.
>
> Daniel.
^ permalink raw reply [flat|nested] 84+ messages in thread
* Re: [RESEND RFC/PATCH 6/8] media: platform: mtk-vcodec: Add Mediatek V4L2 Video Encoder Driver
2015-12-02 13:08 ` tiffany lin
(?)
@ 2015-12-02 16:02 ` Daniel Thompson
-1 siblings, 0 replies; 84+ messages in thread
From: Daniel Thompson @ 2015-12-02 16:02 UTC (permalink / raw)
To: tiffany lin
Cc: Rob Herring, Pawel Moll, Mark Rutland, Ian Campbell, Kumar Gala,
Catalin Marinas, Will Deacon, Mauro Carvalho Chehab,
Matthias Brugger, Daniel Kurtz, Sascha Hauer, Hongzhou Yang,
Hans Verkuil, Laurent Pinchart, Sakari Ailus, Mikhail Ulyanov,
Fabien Dessenne, Arnd Bergmann, Darren Etheridge, Peter Griffin,
Benoit Parrot, Andrew-CT Chen (陳智迪),
Eddie Huang (黃智傑),
Yingjoe Chen (陳英洲),
JamesJJ Liao (廖建智),
Daniel Hsiao (蕭伯剛),
devicetree, lkml, linux-arm-kernel, linux-media, linux-mediatek,
PoChun Lin (林柏君)
On 02/12/15 13:08, tiffany lin wrote:
>>> We need MTK_STATE_ABORT to inform encoder thread (mtk_venc_worker) that
>>> stop encodeing job from stopped ctx instance.
>>> When user space qbuf, we need to make sure everything is ready to sent
>>> buf to encode.
>>
>> Agree that you need a flag here. In fact currently you have two,
>> MTK_STATE_ABORT and an unused one called aborting.
>>
>> You need to be very careful with these flags though. They are a magnet
>> for data race bugs (especially combined with SMP).
>>
>> For example at present I can't see any locking in the worker code. This
>> means there is nothing to make all those read-modify-write sequences
>> that manage the state atomic (thus risking state corruption).
>>
> We prevent that one function set the flag and others clear the flag.
> So there is no special lock to protect state.
What prevents concurrent access from different calling contexts? It
looks to me like the work on the work queue may run concurrently with
the ioctl calls.
> +static void vb2ops_venc_stop_streaming(struct vb2_queue *q)
> +{
> + struct mtk_vcodec_ctx *ctx = vb2_get_drv_priv(q);
> + struct v4l2_device *v4l2_dev = &ctx->dev->v4l2_dev;
> + struct vb2_buffer *src_buf, *dst_buf;
> + int retry;
> + int ret;
> +
> + mtk_v4l2_debug(2, "[%d]-> type=%d", ctx->idx, q->type);
> +
> + retry = 0;
> + while ((ctx->state & MTK_STATE_RUNNING) && (retry < 10)) {
> + mtk_vcodec_clean_ctx_int_flags(ctx);
> + ctx->state |= MTK_STATE_ABORT;
As a simple example I think the above line can run concurrently with the
following code near the end of the worker code.
> + ctx->state &= ~MTK_STATE_RUNNING;
> + v4l2_m2m_job_finish(ctx->dev->m2m_dev_enc, ctx->m2m_ctx);
If I'm right then the state of the flags can definitely get clobbered
due to the read-modify-write actions on the state.
Daniel.
^ permalink raw reply [flat|nested] 84+ messages in thread
* Re: [RESEND RFC/PATCH 6/8] media: platform: mtk-vcodec: Add Mediatek V4L2 Video Encoder Driver
@ 2015-12-02 16:02 ` Daniel Thompson
0 siblings, 0 replies; 84+ messages in thread
From: Daniel Thompson @ 2015-12-02 16:02 UTC (permalink / raw)
To: tiffany lin
Cc: Rob Herring, Pawel Moll, Mark Rutland, Ian Campbell, Kumar Gala,
Catalin Marinas, Will Deacon, Mauro Carvalho Chehab,
Matthias Brugger, Daniel Kurtz, Sascha Hauer, Hongzhou Yang,
Hans Verkuil, Laurent Pinchart, Sakari Ailus, Mikhail Ulyanov,
Fabien Dessenne, Arnd Bergmann, Darren Etheridge, Peter Griffin,
Benoit Parrot
On 02/12/15 13:08, tiffany lin wrote:
>>> We need MTK_STATE_ABORT to inform encoder thread (mtk_venc_worker) that
>>> stop encodeing job from stopped ctx instance.
>>> When user space qbuf, we need to make sure everything is ready to sent
>>> buf to encode.
>>
>> Agree that you need a flag here. In fact currently you have two,
>> MTK_STATE_ABORT and an unused one called aborting.
>>
>> You need to be very careful with these flags though. They are a magnet
>> for data race bugs (especially combined with SMP).
>>
>> For example at present I can't see any locking in the worker code. This
>> means there is nothing to make all those read-modify-write sequences
>> that manage the state atomic (thus risking state corruption).
>>
> We prevent that one function set the flag and others clear the flag.
> So there is no special lock to protect state.
What prevents concurrent access from different calling contexts? It
looks to me like the work on the work queue may run concurrently with
the ioctl calls.
> +static void vb2ops_venc_stop_streaming(struct vb2_queue *q)
> +{
> + struct mtk_vcodec_ctx *ctx = vb2_get_drv_priv(q);
> + struct v4l2_device *v4l2_dev = &ctx->dev->v4l2_dev;
> + struct vb2_buffer *src_buf, *dst_buf;
> + int retry;
> + int ret;
> +
> + mtk_v4l2_debug(2, "[%d]-> type=%d", ctx->idx, q->type);
> +
> + retry = 0;
> + while ((ctx->state & MTK_STATE_RUNNING) && (retry < 10)) {
> + mtk_vcodec_clean_ctx_int_flags(ctx);
> + ctx->state |= MTK_STATE_ABORT;
As a simple example I think the above line can run concurrently with the
following code near the end of the worker code.
> + ctx->state &= ~MTK_STATE_RUNNING;
> + v4l2_m2m_job_finish(ctx->dev->m2m_dev_enc, ctx->m2m_ctx);
If I'm right then the state of the flags can definitely get clobbered
due to the read-modify-write actions on the state.
Daniel.
--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 84+ messages in thread
* [RESEND RFC/PATCH 6/8] media: platform: mtk-vcodec: Add Mediatek V4L2 Video Encoder Driver
@ 2015-12-02 16:02 ` Daniel Thompson
0 siblings, 0 replies; 84+ messages in thread
From: Daniel Thompson @ 2015-12-02 16:02 UTC (permalink / raw)
To: linux-arm-kernel
On 02/12/15 13:08, tiffany lin wrote:
>>> We need MTK_STATE_ABORT to inform encoder thread (mtk_venc_worker) that
>>> stop encodeing job from stopped ctx instance.
>>> When user space qbuf, we need to make sure everything is ready to sent
>>> buf to encode.
>>
>> Agree that you need a flag here. In fact currently you have two,
>> MTK_STATE_ABORT and an unused one called aborting.
>>
>> You need to be very careful with these flags though. They are a magnet
>> for data race bugs (especially combined with SMP).
>>
>> For example at present I can't see any locking in the worker code. This
>> means there is nothing to make all those read-modify-write sequences
>> that manage the state atomic (thus risking state corruption).
>>
> We prevent that one function set the flag and others clear the flag.
> So there is no special lock to protect state.
What prevents concurrent access from different calling contexts? It
looks to me like the work on the work queue may run concurrently with
the ioctl calls.
> +static void vb2ops_venc_stop_streaming(struct vb2_queue *q)
> +{
> + struct mtk_vcodec_ctx *ctx = vb2_get_drv_priv(q);
> + struct v4l2_device *v4l2_dev = &ctx->dev->v4l2_dev;
> + struct vb2_buffer *src_buf, *dst_buf;
> + int retry;
> + int ret;
> +
> + mtk_v4l2_debug(2, "[%d]-> type=%d", ctx->idx, q->type);
> +
> + retry = 0;
> + while ((ctx->state & MTK_STATE_RUNNING) && (retry < 10)) {
> + mtk_vcodec_clean_ctx_int_flags(ctx);
> + ctx->state |= MTK_STATE_ABORT;
As a simple example I think the above line can run concurrently with the
following code near the end of the worker code.
> + ctx->state &= ~MTK_STATE_RUNNING;
> + v4l2_m2m_job_finish(ctx->dev->m2m_dev_enc, ctx->m2m_ctx);
If I'm right then the state of the flags can definitely get clobbered
due to the read-modify-write actions on the state.
Daniel.
^ permalink raw reply [flat|nested] 84+ messages in thread
end of thread, other threads:[~2015-12-02 16:02 UTC | newest]
Thread overview: 84+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2015-11-17 12:54 [RESEND RFC/PATCH 0/8] Add MT8173 Video Encoder Driver and VPU Driver Tiffany Lin
2015-11-17 12:54 ` Tiffany Lin
2015-11-17 12:54 ` Tiffany Lin
2015-11-17 12:54 ` [RESEND RFC/PATCH 1/8] dt-bindings: Add a binding for Mediatek Video Processor Unit Tiffany Lin
2015-11-17 12:54 ` Tiffany Lin
2015-11-17 12:54 ` Tiffany Lin
2015-11-17 14:13 ` Mark Rutland
2015-11-17 14:13 ` Mark Rutland
2015-11-17 14:13 ` Mark Rutland
2015-11-19 2:47 ` andrew-ct chen
2015-11-19 2:47 ` andrew-ct chen
2015-11-19 2:47 ` andrew-ct chen
2015-11-17 12:54 ` [RESEND RFC/PATCH 2/8] arm64: dts: mediatek: Add node " Tiffany Lin
2015-11-17 12:54 ` Tiffany Lin
2015-11-17 12:54 ` Tiffany Lin
2015-11-17 12:54 ` [RESEND RFC/PATCH 3/8] media: platform: mtk-vpu: Support Mediatek VPU Tiffany Lin
2015-11-17 12:54 ` Tiffany Lin
2015-11-17 12:54 ` Tiffany Lin
2015-11-25 16:11 ` Daniel Thompson
2015-11-25 16:11 ` Daniel Thompson
2015-11-25 16:11 ` Daniel Thompson
2015-11-27 12:10 ` andrew-ct chen
2015-11-27 12:10 ` andrew-ct chen
2015-11-27 12:10 ` andrew-ct chen
2015-11-27 12:21 ` Daniel Thompson
2015-11-27 12:21 ` Daniel Thompson
2015-11-27 12:21 ` Daniel Thompson
2015-11-30 11:43 ` andrew-ct chen
2015-11-30 11:43 ` andrew-ct chen
2015-11-30 11:43 ` andrew-ct chen
2015-11-30 15:36 ` Daniel Thompson
2015-11-30 15:36 ` Daniel Thompson
2015-11-30 15:36 ` Daniel Thompson
2015-11-30 15:36 ` Daniel Thompson
2015-12-01 14:31 ` andrew-ct chen
2015-12-01 14:31 ` andrew-ct chen
2015-12-01 14:31 ` andrew-ct chen
2015-12-01 14:31 ` andrew-ct chen
2015-11-17 12:54 ` [RESEND RFC/PATCH 4/8] dt-bindings: Add a binding for Mediatek Video Encoder Tiffany Lin
2015-11-17 12:54 ` Tiffany Lin
2015-11-17 12:54 ` Tiffany Lin
2015-11-17 19:41 ` Rob Herring
2015-11-17 19:41 ` Rob Herring
2015-11-17 19:41 ` Rob Herring
2015-11-18 6:21 ` tiffany lin
2015-11-18 7:09 ` tiffany lin
2015-11-18 7:09 ` tiffany lin
2015-11-18 7:09 ` tiffany lin
2015-11-17 12:54 ` [RESEND RFC/PATCH 5/8] arm64: dts: mediatek: Add Video Encoder for MT8173 Tiffany Lin
2015-11-17 12:54 ` Tiffany Lin
2015-11-17 12:54 ` Tiffany Lin
2015-11-17 12:54 ` [RESEND RFC/PATCH 6/8] media: platform: mtk-vcodec: Add Mediatek V4L2 Video Encoder Driver Tiffany Lin
2015-11-17 12:54 ` Tiffany Lin
2015-11-27 16:34 ` Daniel Thompson
2015-11-27 16:34 ` Daniel Thompson
2015-11-27 16:34 ` Daniel Thompson
2015-11-30 11:39 ` tiffany lin
2015-11-30 11:39 ` tiffany lin
2015-11-30 11:39 ` tiffany lin
2015-11-30 14:58 ` Daniel Thompson
2015-11-30 14:58 ` Daniel Thompson
2015-11-30 14:58 ` Daniel Thompson
2015-11-30 14:58 ` Daniel Thompson
2015-12-01 10:42 ` tiffany lin
2015-12-01 10:42 ` tiffany lin
2015-12-01 10:42 ` tiffany lin
2015-12-01 15:42 ` Daniel Thompson
2015-12-01 15:42 ` Daniel Thompson
2015-12-01 15:42 ` Daniel Thompson
2015-12-02 13:08 ` tiffany lin
2015-12-02 13:08 ` tiffany lin
2015-12-02 13:08 ` tiffany lin
2015-12-02 16:02 ` Daniel Thompson
2015-12-02 16:02 ` Daniel Thompson
2015-12-02 16:02 ` Daniel Thompson
2015-11-17 12:54 ` [RESEND RFC/PATCH 7/8] media: platform: mtk-vcodec: Add Mediatek VP8 " Tiffany Lin
2015-11-17 12:54 ` Tiffany Lin
2015-11-17 12:54 ` Tiffany Lin
2015-11-17 12:54 ` [RESEND RFC/PATCH 8/8] media: platform: mtk-vcodec: Add Mediatek H264 " Tiffany Lin
2015-11-17 12:54 ` Tiffany Lin
2015-11-17 12:54 ` Tiffany Lin
2015-11-19 7:40 ` [RESEND RFC/PATCH 0/8] Add MT8173 Video Encoder Driver and VPU Driver Hans Verkuil
2015-11-19 7:40 ` Hans Verkuil
2015-11-19 7:40 ` Hans Verkuil
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.