[PATCH 0/9] Enable two H264 encoder cores on MT8195

[PATCH 0/9] Enable two H264 encoder cores on MT8195

[Irui Wang]

MT8195 has two H264 encoder cores, they have their own power-domains,
clocks, interrupts, register base. The two H264 encoder cores can work
together to achieve higher performance.

This series of patches is to use enable two h264 encoder cores.
path[1..2]: use linux component framework to manage encoder hardware,
user call "mt8195-vcodec-enc" driver can get the encoder master device,
the encoding work is done by the two encoder core device. The hw_mode
variable is added to distinguish from old platform, two encoder cores
called "FRAME_RACING_MODE".

The hardware mode of two encoder cores work together(overlap, another
word called) on MT8195 called "frame_racing_mode", the two encoder
power-domains should be power on together while encoding, the encoding
process look like this:

    VENC Core0 frm#0....frm#2....frm#4
    VENC Core1  .frm#1....frm#3....frm#5

patch[3..5]: due to the component device, the master device has no
power-domains/clocks properties in dtsi, so the power/clock init function
can't use for "frame_racing_mode" device in master device probe process,
it should be called in component device probe process. Power on the
hardware power and clock on demand.

patch[6]: "frame_racing_mode" encoding need a new set of memory buffer
for two encoder cores. For compatibility, we should new a encoder driver
interface.

patch[7..9]: add "frame_racing_mode" encoding process:
As-Is: Synchronous
VIDIOC_QBUF#0 --> device_run(triger encoder) --> wait encoder IRQ -->
encode done with result --> job_finish

VIDIOC_QBUF#1 --> device_run(triger encoder) --> wait encoder IRQ -->
encode done with result --> job_finish
...

To-Be: Asynchronous
VIDIOC_QBUF#0 --> device_run(triger encoder core0) --> job_finish
..VIDIOC_QBUF#1 --> device_run(triger encoder core1) --> job_finish
(core0 may encode done here, return encode result to client)
VIDIOC_QBUF#2 --> device_run(triger encoder core0) --> job_finish

Thers is no "wait encoder IRQ" synchronous call during "frame_racing_mode"
encoding process, which can full use the two encoder cores to achieve
higher performance.

Irui Wang (9):
  dt-bindings: media: mtk-vcodec: Add binding for MT8195 two venc cores
  media: mtk-vcodec: Use component framework to manage encoder hardware
  media: mtk-vcodec: Rewrite venc power manage interface
  media: mtk-vcodec: Add venc power on/off interface
  media: mtk-vcodec: Rewrite venc clock interface
  media: mtk-vcodec: Add new venc drv interface for frame_racing mode
  media: mtk-vcodec: Add frame racing mode encode process
  media: mtk-vcodec: Return encode result to client
  media: mtk-vcodec: Add delayed worker for encode timeout

 .../bindings/media/mediatek-vcodec.txt        |   2 +
 drivers/media/platform/mtk-vcodec/Makefile    |   2 +
 .../platform/mtk-vcodec/mtk_vcodec_drv.h      |  34 +-
 .../platform/mtk-vcodec/mtk_vcodec_enc.c      | 120 +++-
 .../platform/mtk-vcodec/mtk_vcodec_enc.h      |  10 +-
 .../platform/mtk-vcodec/mtk_vcodec_enc_drv.c  | 204 +++++-
 .../platform/mtk-vcodec/mtk_vcodec_enc_hw.c   | 253 +++++++
 .../platform/mtk-vcodec/mtk_vcodec_enc_hw.h   |  38 +
 .../platform/mtk-vcodec/mtk_vcodec_enc_pm.c   | 213 ++++--
 .../platform/mtk-vcodec/mtk_vcodec_enc_pm.h   |  13 +-
 .../platform/mtk-vcodec/mtk_vcodec_util.c     |  19 +
 .../platform/mtk-vcodec/mtk_vcodec_util.h     |   5 +
 .../platform/mtk-vcodec/venc/venc_common_if.c | 675 ++++++++++++++++++
 .../platform/mtk-vcodec/venc/venc_h264_if.c   |   6 +-
 .../platform/mtk-vcodec/venc/venc_vp8_if.c    |   2 +-
 .../media/platform/mtk-vcodec/venc_drv_if.c   |  96 ++-
 .../media/platform/mtk-vcodec/venc_drv_if.h   |   7 +
 .../media/platform/mtk-vcodec/venc_vpu_if.c   |  11 +-
 .../media/platform/mtk-vcodec/venc_vpu_if.h   |   3 +-
 19 files changed, 1564 insertions(+), 149 deletions(-)
 create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.c
 create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.h
 create mode 100644 drivers/media/platform/mtk-vcodec/venc/venc_common_if.c

-- 
2.25.1

[PATCH 0/9] Enable two H264 encoder cores on MT8195

[Irui Wang]

MT8195 has two H264 encoder cores, they have their own power-domains,
clocks, interrupts, register base. The two H264 encoder cores can work
together to achieve higher performance.

This series of patches is to use enable two h264 encoder cores.
path[1..2]: use linux component framework to manage encoder hardware,
user call "mt8195-vcodec-enc" driver can get the encoder master device,
the encoding work is done by the two encoder core device. The hw_mode
variable is added to distinguish from old platform, two encoder cores
called "FRAME_RACING_MODE".

The hardware mode of two encoder cores work together(overlap, another
word called) on MT8195 called "frame_racing_mode", the two encoder
power-domains should be power on together while encoding, the encoding
process look like this:

    VENC Core0 frm#0....frm#2....frm#4
    VENC Core1  .frm#1....frm#3....frm#5

patch[3..5]: due to the component device, the master device has no
power-domains/clocks properties in dtsi, so the power/clock init function
can't use for "frame_racing_mode" device in master device probe process,
it should be called in component device probe process. Power on the
hardware power and clock on demand.

patch[6]: "frame_racing_mode" encoding need a new set of memory buffer
for two encoder cores. For compatibility, we should new a encoder driver
interface.

patch[7..9]: add "frame_racing_mode" encoding process:
As-Is: Synchronous
VIDIOC_QBUF#0 --> device_run(triger encoder) --> wait encoder IRQ -->
encode done with result --> job_finish

VIDIOC_QBUF#1 --> device_run(triger encoder) --> wait encoder IRQ -->
encode done with result --> job_finish
...

To-Be: Asynchronous
VIDIOC_QBUF#0 --> device_run(triger encoder core0) --> job_finish
..VIDIOC_QBUF#1 --> device_run(triger encoder core1) --> job_finish
(core0 may encode done here, return encode result to client)
VIDIOC_QBUF#2 --> device_run(triger encoder core0) --> job_finish

Thers is no "wait encoder IRQ" synchronous call during "frame_racing_mode"
encoding process, which can full use the two encoder cores to achieve
higher performance.

Irui Wang (9):
  dt-bindings: media: mtk-vcodec: Add binding for MT8195 two venc cores
  media: mtk-vcodec: Use component framework to manage encoder hardware
  media: mtk-vcodec: Rewrite venc power manage interface
  media: mtk-vcodec: Add venc power on/off interface
  media: mtk-vcodec: Rewrite venc clock interface
  media: mtk-vcodec: Add new venc drv interface for frame_racing mode
  media: mtk-vcodec: Add frame racing mode encode process
  media: mtk-vcodec: Return encode result to client
  media: mtk-vcodec: Add delayed worker for encode timeout

 .../bindings/media/mediatek-vcodec.txt        |   2 +
 drivers/media/platform/mtk-vcodec/Makefile    |   2 +
 .../platform/mtk-vcodec/mtk_vcodec_drv.h      |  34 +-
 .../platform/mtk-vcodec/mtk_vcodec_enc.c      | 120 +++-
 .../platform/mtk-vcodec/mtk_vcodec_enc.h      |  10 +-
 .../platform/mtk-vcodec/mtk_vcodec_enc_drv.c  | 204 +++++-
 .../platform/mtk-vcodec/mtk_vcodec_enc_hw.c   | 253 +++++++
 .../platform/mtk-vcodec/mtk_vcodec_enc_hw.h   |  38 +
 .../platform/mtk-vcodec/mtk_vcodec_enc_pm.c   | 213 ++++--
 .../platform/mtk-vcodec/mtk_vcodec_enc_pm.h   |  13 +-
 .../platform/mtk-vcodec/mtk_vcodec_util.c     |  19 +
 .../platform/mtk-vcodec/mtk_vcodec_util.h     |   5 +
 .../platform/mtk-vcodec/venc/venc_common_if.c | 675 ++++++++++++++++++
 .../platform/mtk-vcodec/venc/venc_h264_if.c   |   6 +-
 .../platform/mtk-vcodec/venc/venc_vp8_if.c    |   2 +-
 .../media/platform/mtk-vcodec/venc_drv_if.c   |  96 ++-
 .../media/platform/mtk-vcodec/venc_drv_if.h   |   7 +
 .../media/platform/mtk-vcodec/venc_vpu_if.c   |  11 +-
 .../media/platform/mtk-vcodec/venc_vpu_if.h   |   3 +-
 19 files changed, 1564 insertions(+), 149 deletions(-)
 create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.c
 create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.h
 create mode 100644 drivers/media/platform/mtk-vcodec/venc/venc_common_if.c

-- 
2.25.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH 0/9] Enable two H264 encoder cores on MT8195

[Irui Wang]

MT8195 has two H264 encoder cores, they have their own power-domains,
clocks, interrupts, register base. The two H264 encoder cores can work
together to achieve higher performance.

This series of patches is to use enable two h264 encoder cores.
path[1..2]: use linux component framework to manage encoder hardware,
user call "mt8195-vcodec-enc" driver can get the encoder master device,
the encoding work is done by the two encoder core device. The hw_mode
variable is added to distinguish from old platform, two encoder cores
called "FRAME_RACING_MODE".

The hardware mode of two encoder cores work together(overlap, another
word called) on MT8195 called "frame_racing_mode", the two encoder
power-domains should be power on together while encoding, the encoding
process look like this:

    VENC Core0 frm#0....frm#2....frm#4
    VENC Core1  .frm#1....frm#3....frm#5

patch[3..5]: due to the component device, the master device has no
power-domains/clocks properties in dtsi, so the power/clock init function
can't use for "frame_racing_mode" device in master device probe process,
it should be called in component device probe process. Power on the
hardware power and clock on demand.

patch[6]: "frame_racing_mode" encoding need a new set of memory buffer
for two encoder cores. For compatibility, we should new a encoder driver
interface.

patch[7..9]: add "frame_racing_mode" encoding process:
As-Is: Synchronous
VIDIOC_QBUF#0 --> device_run(triger encoder) --> wait encoder IRQ -->
encode done with result --> job_finish

VIDIOC_QBUF#1 --> device_run(triger encoder) --> wait encoder IRQ -->
encode done with result --> job_finish
...

To-Be: Asynchronous
VIDIOC_QBUF#0 --> device_run(triger encoder core0) --> job_finish
..VIDIOC_QBUF#1 --> device_run(triger encoder core1) --> job_finish
(core0 may encode done here, return encode result to client)
VIDIOC_QBUF#2 --> device_run(triger encoder core0) --> job_finish

Thers is no "wait encoder IRQ" synchronous call during "frame_racing_mode"
encoding process, which can full use the two encoder cores to achieve
higher performance.

Irui Wang (9):
  dt-bindings: media: mtk-vcodec: Add binding for MT8195 two venc cores
  media: mtk-vcodec: Use component framework to manage encoder hardware
  media: mtk-vcodec: Rewrite venc power manage interface
  media: mtk-vcodec: Add venc power on/off interface
  media: mtk-vcodec: Rewrite venc clock interface
  media: mtk-vcodec: Add new venc drv interface for frame_racing mode
  media: mtk-vcodec: Add frame racing mode encode process
  media: mtk-vcodec: Return encode result to client
  media: mtk-vcodec: Add delayed worker for encode timeout

 .../bindings/media/mediatek-vcodec.txt        |   2 +
 drivers/media/platform/mtk-vcodec/Makefile    |   2 +
 .../platform/mtk-vcodec/mtk_vcodec_drv.h      |  34 +-
 .../platform/mtk-vcodec/mtk_vcodec_enc.c      | 120 +++-
 .../platform/mtk-vcodec/mtk_vcodec_enc.h      |  10 +-
 .../platform/mtk-vcodec/mtk_vcodec_enc_drv.c  | 204 +++++-
 .../platform/mtk-vcodec/mtk_vcodec_enc_hw.c   | 253 +++++++
 .../platform/mtk-vcodec/mtk_vcodec_enc_hw.h   |  38 +
 .../platform/mtk-vcodec/mtk_vcodec_enc_pm.c   | 213 ++++--
 .../platform/mtk-vcodec/mtk_vcodec_enc_pm.h   |  13 +-
 .../platform/mtk-vcodec/mtk_vcodec_util.c     |  19 +
 .../platform/mtk-vcodec/mtk_vcodec_util.h     |   5 +
 .../platform/mtk-vcodec/venc/venc_common_if.c | 675 ++++++++++++++++++
 .../platform/mtk-vcodec/venc/venc_h264_if.c   |   6 +-
 .../platform/mtk-vcodec/venc/venc_vp8_if.c    |   2 +-
 .../media/platform/mtk-vcodec/venc_drv_if.c   |  96 ++-
 .../media/platform/mtk-vcodec/venc_drv_if.h   |   7 +
 .../media/platform/mtk-vcodec/venc_vpu_if.c   |  11 +-
 .../media/platform/mtk-vcodec/venc_vpu_if.h   |   3 +-
 19 files changed, 1564 insertions(+), 149 deletions(-)
 create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.c
 create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.h
 create mode 100644 drivers/media/platform/mtk-vcodec/venc/venc_common_if.c

-- 
2.25.1
_______________________________________________
Linux-mediatek mailing list
Linux-mediatek@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-mediatek

[PATCH 1/9] dt-bindings: media: mtk-vcodec: Add binding for MT8195 two venc cores

[Irui Wang]

Enable MT8195 two H.264 venc cores, updates vcodec binding document.

Signed-off-by: Irui Wang <irui.wang@mediatek.com>
---
 Documentation/devicetree/bindings/media/mediatek-vcodec.txt | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/Documentation/devicetree/bindings/media/mediatek-vcodec.txt b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
index de961699ba0a..eb2e24c32426 100644
--- a/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
+++ b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
@@ -11,6 +11,8 @@ Required properties:
   "mediatek,mt8173-vcodec-dec" for MT8173 decoder.
   "mediatek,mt8192-vcodec-enc" for MT8192 encoder.
   "mediatek,mt8195-vcodec-enc" for MT8195 encoder.
+  "mediatek,mtk-venc-core0" for MT8195 avc core0 device.
+  "mediatek,mtk-venc-core1" for MT8195 avc core1 device.
 - reg : Physical base address of the video codec registers and length of
   memory mapped region.
 - interrupts : interrupt number to the cpu.
-- 
2.25.1

[PATCH 1/9] dt-bindings: media: mtk-vcodec: Add binding for MT8195 two venc cores

[Irui Wang]

Enable MT8195 two H.264 venc cores, updates vcodec binding document.

Signed-off-by: Irui Wang <irui.wang@mediatek.com>
---
 Documentation/devicetree/bindings/media/mediatek-vcodec.txt | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/Documentation/devicetree/bindings/media/mediatek-vcodec.txt b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
index de961699ba0a..eb2e24c32426 100644
--- a/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
+++ b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
@@ -11,6 +11,8 @@ Required properties:
   "mediatek,mt8173-vcodec-dec" for MT8173 decoder.
   "mediatek,mt8192-vcodec-enc" for MT8192 encoder.
   "mediatek,mt8195-vcodec-enc" for MT8195 encoder.
+  "mediatek,mtk-venc-core0" for MT8195 avc core0 device.
+  "mediatek,mtk-venc-core1" for MT8195 avc core1 device.
 - reg : Physical base address of the video codec registers and length of
   memory mapped region.
 - interrupts : interrupt number to the cpu.
-- 
2.25.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH 1/9] dt-bindings: media: mtk-vcodec: Add binding for MT8195 two venc cores

[Irui Wang]

Enable MT8195 two H.264 venc cores, updates vcodec binding document.

Signed-off-by: Irui Wang <irui.wang@mediatek.com>
---
 Documentation/devicetree/bindings/media/mediatek-vcodec.txt | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/Documentation/devicetree/bindings/media/mediatek-vcodec.txt b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
index de961699ba0a..eb2e24c32426 100644
--- a/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
+++ b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
@@ -11,6 +11,8 @@ Required properties:
   "mediatek,mt8173-vcodec-dec" for MT8173 decoder.
   "mediatek,mt8192-vcodec-enc" for MT8192 encoder.
   "mediatek,mt8195-vcodec-enc" for MT8195 encoder.
+  "mediatek,mtk-venc-core0" for MT8195 avc core0 device.
+  "mediatek,mtk-venc-core1" for MT8195 avc core1 device.
 - reg : Physical base address of the video codec registers and length of
   memory mapped region.
 - interrupts : interrupt number to the cpu.
-- 
2.25.1
_______________________________________________
Linux-mediatek mailing list
Linux-mediatek@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-mediatek

Re: [PATCH 1/9] dt-bindings: media: mtk-vcodec: Add binding for MT8195 two venc cores

[Rob Herring]

On Mon, 16 Aug 2021 18:59:26 +0800, Irui Wang wrote:
> Enable MT8195 two H.264 venc cores, updates vcodec binding document.
> 
> Signed-off-by: Irui Wang <irui.wang@mediatek.com>
> ---
>  Documentation/devicetree/bindings/media/mediatek-vcodec.txt | 2 ++
>  1 file changed, 2 insertions(+)
> 

Acked-by: Rob Herring <robh@kernel.org>

Re: [PATCH 1/9] dt-bindings: media: mtk-vcodec: Add binding for MT8195 two venc cores

[Rob Herring]

On Mon, 16 Aug 2021 18:59:26 +0800, Irui Wang wrote:
> Enable MT8195 two H.264 venc cores, updates vcodec binding document.
> 
> Signed-off-by: Irui Wang <irui.wang@mediatek.com>
> ---
>  Documentation/devicetree/bindings/media/mediatek-vcodec.txt | 2 ++
>  1 file changed, 2 insertions(+)
> 

Acked-by: Rob Herring <robh@kernel.org>

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH 1/9] dt-bindings: media: mtk-vcodec: Add binding for MT8195 two venc cores

[Rob Herring]

On Mon, 16 Aug 2021 18:59:26 +0800, Irui Wang wrote:
> Enable MT8195 two H.264 venc cores, updates vcodec binding document.
> 
> Signed-off-by: Irui Wang <irui.wang@mediatek.com>
> ---
>  Documentation/devicetree/bindings/media/mediatek-vcodec.txt | 2 ++
>  1 file changed, 2 insertions(+)
> 

Acked-by: Rob Herring <robh@kernel.org>

_______________________________________________
Linux-mediatek mailing list
Linux-mediatek@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-mediatek

Re: [PATCH 1/9] dt-bindings: media: mtk-vcodec: Add binding for MT8195 two venc cores

[Ezequiel Garcia]

Hi Irui,

On Mon, 16 Aug 2021 at 08:00, Irui Wang <irui.wang@mediatek.com> wrote:
>
> Enable MT8195 two H.264 venc cores, updates vcodec binding document.
>
> Signed-off-by: Irui Wang <irui.wang@mediatek.com>
> ---
>  Documentation/devicetree/bindings/media/mediatek-vcodec.txt | 2 ++
>  1 file changed, 2 insertions(+)
>
> diff --git a/Documentation/devicetree/bindings/media/mediatek-vcodec.txt b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
> index de961699ba0a..eb2e24c32426 100644
> --- a/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
> +++ b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
> @@ -11,6 +11,8 @@ Required properties:
>    "mediatek,mt8173-vcodec-dec" for MT8173 decoder.
>    "mediatek,mt8192-vcodec-enc" for MT8192 encoder.
>    "mediatek,mt8195-vcodec-enc" for MT8195 encoder.
> +  "mediatek,mtk-venc-core0" for MT8195 avc core0 device.
> +  "mediatek,mtk-venc-core1" for MT8195 avc core1 device.

What is the difference between core0 and core1?

Thanks,
Ezequiel

Re: [PATCH 1/9] dt-bindings: media: mtk-vcodec: Add binding for MT8195 two venc cores

[Ezequiel Garcia]

Hi Irui,

On Mon, 16 Aug 2021 at 08:00, Irui Wang <irui.wang@mediatek.com> wrote:
>
> Enable MT8195 two H.264 venc cores, updates vcodec binding document.
>
> Signed-off-by: Irui Wang <irui.wang@mediatek.com>
> ---
>  Documentation/devicetree/bindings/media/mediatek-vcodec.txt | 2 ++
>  1 file changed, 2 insertions(+)
>
> diff --git a/Documentation/devicetree/bindings/media/mediatek-vcodec.txt b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
> index de961699ba0a..eb2e24c32426 100644
> --- a/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
> +++ b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
> @@ -11,6 +11,8 @@ Required properties:
>    "mediatek,mt8173-vcodec-dec" for MT8173 decoder.
>    "mediatek,mt8192-vcodec-enc" for MT8192 encoder.
>    "mediatek,mt8195-vcodec-enc" for MT8195 encoder.
> +  "mediatek,mtk-venc-core0" for MT8195 avc core0 device.
> +  "mediatek,mtk-venc-core1" for MT8195 avc core1 device.

What is the difference between core0 and core1?

Thanks,
Ezequiel

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH 1/9] dt-bindings: media: mtk-vcodec: Add binding for MT8195 two venc cores

[Ezequiel Garcia]

Hi Irui,

On Mon, 16 Aug 2021 at 08:00, Irui Wang <irui.wang@mediatek.com> wrote:
>
> Enable MT8195 two H.264 venc cores, updates vcodec binding document.
>
> Signed-off-by: Irui Wang <irui.wang@mediatek.com>
> ---
>  Documentation/devicetree/bindings/media/mediatek-vcodec.txt | 2 ++
>  1 file changed, 2 insertions(+)
>
> diff --git a/Documentation/devicetree/bindings/media/mediatek-vcodec.txt b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
> index de961699ba0a..eb2e24c32426 100644
> --- a/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
> +++ b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
> @@ -11,6 +11,8 @@ Required properties:
>    "mediatek,mt8173-vcodec-dec" for MT8173 decoder.
>    "mediatek,mt8192-vcodec-enc" for MT8192 encoder.
>    "mediatek,mt8195-vcodec-enc" for MT8195 encoder.
> +  "mediatek,mtk-venc-core0" for MT8195 avc core0 device.
> +  "mediatek,mtk-venc-core1" for MT8195 avc core1 device.

What is the difference between core0 and core1?

Thanks,
Ezequiel

_______________________________________________
Linux-mediatek mailing list
Linux-mediatek@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-mediatek

Re: [PATCH 1/9] dt-bindings: media: mtk-vcodec: Add binding for MT8195 two venc cores

[Irui Wang (王瑞)]

Hi,Ezequiel,

Thanks for your reviewing.

On Tue, 2021-08-24 at 08:02 -0300, Ezequiel Garcia wrote:
> Hi Irui,
> 
> On Mon, 16 Aug 2021 at 08:00, Irui Wang <irui.wang@mediatek.com>
> wrote:
> > 
> > Enable MT8195 two H.264 venc cores, updates vcodec binding
> > document.
> > 
> > Signed-off-by: Irui Wang <irui.wang@mediatek.com>
> > ---
> >  Documentation/devicetree/bindings/media/mediatek-vcodec.txt | 2 ++
> >  1 file changed, 2 insertions(+)
> > 
> > diff --git a/Documentation/devicetree/bindings/media/mediatek-
> > vcodec.txt b/Documentation/devicetree/bindings/media/mediatek-
> > vcodec.txt
> > index de961699ba0a..eb2e24c32426 100644
> > --- a/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
> > +++ b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
> > @@ -11,6 +11,8 @@ Required properties:
> >    "mediatek,mt8173-vcodec-dec" for MT8173 decoder.
> >    "mediatek,mt8192-vcodec-enc" for MT8192 encoder.
> >    "mediatek,mt8195-vcodec-enc" for MT8195 encoder.
> > +  "mediatek,mtk-venc-core0" for MT8195 avc core0 device.
> > +  "mediatek,mtk-venc-core1" for MT8195 avc core1 device.
> 
> What is the difference between core0 and core1?
> 
> Thanks,
> Ezequiel

Both core0 and core1 are H264 encoder hardware, they have their own
hardware register base, used power-domains/clocks/irqs. We can use any
of them for H.264 encoding, but the two cores can work together for
higher performance, it's called "frame racing", a hardware encoding
mode, control flow just like in the commit messages:

core0 frame#0.frame#2.frame#4...
core1    frame#1.frame#3.frame#5...

Thanks

Re: [PATCH 1/9] dt-bindings: media: mtk-vcodec: Add binding for MT8195 two venc cores

[Ezequiel Garcia]

On Tue, 24 Aug 2021 at 23:04, Irui Wang (王瑞) <Irui.Wang@mediatek.com> wrote:
>
> Hi,Ezequiel,
>
> Thanks for your reviewing.
>
> On Tue, 2021-08-24 at 08:02 -0300, Ezequiel Garcia wrote:
> > Hi Irui,
> >
> > On Mon, 16 Aug 2021 at 08:00, Irui Wang <irui.wang@mediatek.com>
> > wrote:
> > >
> > > Enable MT8195 two H.264 venc cores, updates vcodec binding
> > > document.
> > >
> > > Signed-off-by: Irui Wang <irui.wang@mediatek.com>
> > > ---
> > >  Documentation/devicetree/bindings/media/mediatek-vcodec.txt | 2 ++
> > >  1 file changed, 2 insertions(+)
> > >
> > > diff --git a/Documentation/devicetree/bindings/media/mediatek-
> > > vcodec.txt b/Documentation/devicetree/bindings/media/mediatek-
> > > vcodec.txt
> > > index de961699ba0a..eb2e24c32426 100644
> > > --- a/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
> > > +++ b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
> > > @@ -11,6 +11,8 @@ Required properties:
> > >    "mediatek,mt8173-vcodec-dec" for MT8173 decoder.
> > >    "mediatek,mt8192-vcodec-enc" for MT8192 encoder.
> > >    "mediatek,mt8195-vcodec-enc" for MT8195 encoder.
> > > +  "mediatek,mtk-venc-core0" for MT8195 avc core0 device.
> > > +  "mediatek,mtk-venc-core1" for MT8195 avc core1 device.
> >
> > What is the difference between core0 and core1?
> >
> > Thanks,
> > Ezequiel
>
> Both core0 and core1 are H264 encoder hardware, they have their own
> hardware register base, used power-domains/clocks/irqs. We can use any
> of them for H.264 encoding, but the two cores can work together for
> higher performance, it's called "frame racing", a hardware encoding
> mode, control flow just like in the commit messages:
>
> core0 frame#0.frame#2.frame#4...
> core1    frame#1.frame#3.frame#5...
>

If they are two encoder cores, why do you need different compatible strings?

It would be interesting to see a device tree which shows how this should
be used in the real world, but from the looks of it, it seems you don't
need a separate compatible.

It seems this series is somewhat related to Yunfei's "[PATCH v5,
00/15] Using component framework to support multi hardware decode",
but I don't see a device tree patch either in that series.

Given this is a complex architecture, I don't know if it
makes sense to discuss decoder and encoder independently.

If you guys unify the two series, and add the device tree patches for it,
or at least for the most complex cases, maybe that will surface the
architecture more clearly and come up with an easier solution that
doesn't involve
an async framework to pull in the parts together.

Thanks,
Ezequiel

Re: [PATCH 1/9] dt-bindings: media: mtk-vcodec: Add binding for MT8195 two venc cores

[Ezequiel Garcia]

On Tue, 24 Aug 2021 at 23:04, Irui Wang (王瑞) <Irui.Wang@mediatek.com> wrote:
>
> Hi,Ezequiel,
>
> Thanks for your reviewing.
>
> On Tue, 2021-08-24 at 08:02 -0300, Ezequiel Garcia wrote:
> > Hi Irui,
> >
> > On Mon, 16 Aug 2021 at 08:00, Irui Wang <irui.wang@mediatek.com>
> > wrote:
> > >
> > > Enable MT8195 two H.264 venc cores, updates vcodec binding
> > > document.
> > >
> > > Signed-off-by: Irui Wang <irui.wang@mediatek.com>
> > > ---
> > >  Documentation/devicetree/bindings/media/mediatek-vcodec.txt | 2 ++
> > >  1 file changed, 2 insertions(+)
> > >
> > > diff --git a/Documentation/devicetree/bindings/media/mediatek-
> > > vcodec.txt b/Documentation/devicetree/bindings/media/mediatek-
> > > vcodec.txt
> > > index de961699ba0a..eb2e24c32426 100644
> > > --- a/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
> > > +++ b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
> > > @@ -11,6 +11,8 @@ Required properties:
> > >    "mediatek,mt8173-vcodec-dec" for MT8173 decoder.
> > >    "mediatek,mt8192-vcodec-enc" for MT8192 encoder.
> > >    "mediatek,mt8195-vcodec-enc" for MT8195 encoder.
> > > +  "mediatek,mtk-venc-core0" for MT8195 avc core0 device.
> > > +  "mediatek,mtk-venc-core1" for MT8195 avc core1 device.
> >
> > What is the difference between core0 and core1?
> >
> > Thanks,
> > Ezequiel
>
> Both core0 and core1 are H264 encoder hardware, they have their own
> hardware register base, used power-domains/clocks/irqs. We can use any
> of them for H.264 encoding, but the two cores can work together for
> higher performance, it's called "frame racing", a hardware encoding
> mode, control flow just like in the commit messages:
>
> core0 frame#0.frame#2.frame#4...
> core1    frame#1.frame#3.frame#5...
>

If they are two encoder cores, why do you need different compatible strings?

It would be interesting to see a device tree which shows how this should
be used in the real world, but from the looks of it, it seems you don't
need a separate compatible.

It seems this series is somewhat related to Yunfei's "[PATCH v5,
00/15] Using component framework to support multi hardware decode",
but I don't see a device tree patch either in that series.

Given this is a complex architecture, I don't know if it
makes sense to discuss decoder and encoder independently.

If you guys unify the two series, and add the device tree patches for it,
or at least for the most complex cases, maybe that will surface the
architecture more clearly and come up with an easier solution that
doesn't involve
an async framework to pull in the parts together.

Thanks,
Ezequiel

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH 1/9] dt-bindings: media: mtk-vcodec: Add binding for MT8195 two venc cores

[Ezequiel Garcia]

On Tue, 24 Aug 2021 at 23:04, Irui Wang (王瑞) <Irui.Wang@mediatek.com> wrote:
>
> Hi,Ezequiel,
>
> Thanks for your reviewing.
>
> On Tue, 2021-08-24 at 08:02 -0300, Ezequiel Garcia wrote:
> > Hi Irui,
> >
> > On Mon, 16 Aug 2021 at 08:00, Irui Wang <irui.wang@mediatek.com>
> > wrote:
> > >
> > > Enable MT8195 two H.264 venc cores, updates vcodec binding
> > > document.
> > >
> > > Signed-off-by: Irui Wang <irui.wang@mediatek.com>
> > > ---
> > >  Documentation/devicetree/bindings/media/mediatek-vcodec.txt | 2 ++
> > >  1 file changed, 2 insertions(+)
> > >
> > > diff --git a/Documentation/devicetree/bindings/media/mediatek-
> > > vcodec.txt b/Documentation/devicetree/bindings/media/mediatek-
> > > vcodec.txt
> > > index de961699ba0a..eb2e24c32426 100644
> > > --- a/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
> > > +++ b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
> > > @@ -11,6 +11,8 @@ Required properties:
> > >    "mediatek,mt8173-vcodec-dec" for MT8173 decoder.
> > >    "mediatek,mt8192-vcodec-enc" for MT8192 encoder.
> > >    "mediatek,mt8195-vcodec-enc" for MT8195 encoder.
> > > +  "mediatek,mtk-venc-core0" for MT8195 avc core0 device.
> > > +  "mediatek,mtk-venc-core1" for MT8195 avc core1 device.
> >
> > What is the difference between core0 and core1?
> >
> > Thanks,
> > Ezequiel
>
> Both core0 and core1 are H264 encoder hardware, they have their own
> hardware register base, used power-domains/clocks/irqs. We can use any
> of them for H.264 encoding, but the two cores can work together for
> higher performance, it's called "frame racing", a hardware encoding
> mode, control flow just like in the commit messages:
>
> core0 frame#0.frame#2.frame#4...
> core1    frame#1.frame#3.frame#5...
>

If they are two encoder cores, why do you need different compatible strings?

It would be interesting to see a device tree which shows how this should
be used in the real world, but from the looks of it, it seems you don't
need a separate compatible.

It seems this series is somewhat related to Yunfei's "[PATCH v5,
00/15] Using component framework to support multi hardware decode",
but I don't see a device tree patch either in that series.

Given this is a complex architecture, I don't know if it
makes sense to discuss decoder and encoder independently.

If you guys unify the two series, and add the device tree patches for it,
or at least for the most complex cases, maybe that will surface the
architecture more clearly and come up with an easier solution that
doesn't involve
an async framework to pull in the parts together.

Thanks,
Ezequiel

_______________________________________________
Linux-mediatek mailing list
Linux-mediatek@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-mediatek

Re: [PATCH 1/9] dt-bindings: media: mtk-vcodec: Add binding for MT8195 two venc cores

[Irui Wang (王瑞)]

On Wed, 2021-08-25 at 00:49 -0300, Ezequiel Garcia wrote:
> On Tue, 24 Aug 2021 at 23:04, Irui Wang (王瑞) <Irui.Wang@mediatek.com>
> wrote:
> > 
> > Hi,Ezequiel,
> > 
> > Thanks for your reviewing.
> > 
> > On Tue, 2021-08-24 at 08:02 -0300, Ezequiel Garcia wrote:
> > > Hi Irui,
> > > 
> > > On Mon, 16 Aug 2021 at 08:00, Irui Wang <irui.wang@mediatek.com>
> > > wrote:
> > > > 
> > > > Enable MT8195 two H.264 venc cores, updates vcodec binding
> > > > document.
> > > > 
> > > > Signed-off-by: Irui Wang <irui.wang@mediatek.com>
> > > > ---
> > > >  Documentation/devicetree/bindings/media/mediatek-vcodec.txt |
> > > > 2 ++
> > > >  1 file changed, 2 insertions(+)
> > > > 
> > > > diff --git a/Documentation/devicetree/bindings/media/mediatek-
> > > > vcodec.txt b/Documentation/devicetree/bindings/media/mediatek-
> > > > vcodec.txt
> > > > index de961699ba0a..eb2e24c32426 100644
> > > > --- a/Documentation/devicetree/bindings/media/mediatek-
> > > > vcodec.txt
> > > > +++ b/Documentation/devicetree/bindings/media/mediatek-
> > > > vcodec.txt
> > > > @@ -11,6 +11,8 @@ Required properties:
> > > >    "mediatek,mt8173-vcodec-dec" for MT8173 decoder.
> > > >    "mediatek,mt8192-vcodec-enc" for MT8192 encoder.
> > > >    "mediatek,mt8195-vcodec-enc" for MT8195 encoder.
> > > > +  "mediatek,mtk-venc-core0" for MT8195 avc core0 device.
> > > > +  "mediatek,mtk-venc-core1" for MT8195 avc core1 device.
> > > 
> > > What is the difference between core0 and core1?
> > > 
> > > Thanks,
> > > Ezequiel
> > 
> > Both core0 and core1 are H264 encoder hardware, they have their own
> > hardware register base, used power-domains/clocks/irqs. We can use
> > any
> > of them for H.264 encoding, but the two cores can work together for
> > higher performance, it's called "frame racing", a hardware encoding
> > mode, control flow just like in the commit messages:
> > 
> > core0 frame#0.frame#2.frame#4...
> > core1    frame#1.frame#3.frame#5...
> > 
> 
> If they are two encoder cores, why do you need different compatible
> strings?
> 
> It would be interesting to see a device tree which shows how this
> should
> be used in the real world, but from the looks of it, it seems you
> don't
> need a separate compatible.
> 
We want to use the two cores through one device node, core0 and core1
bind to device "mediatek,mt8195-vcodec-enc", it's feasible to make two
cores work by just opening one device, the device tree looks like:

venc {
    compatible = "mediatek,mt8195-vcodec-enc"
}
venc_core0 {
    compatible = "mediatek,mtk-venc-core0"
    interrupts = xxxx
    clocks = xxxx
    power-domains = xxxx
    ...
}

venc_core1 {
    compatible = "mediatek,mtk-venc-core1"
    interrupts = xxxx
    clocks = xxxx
    power-domains = xxxx
    ...
}

as you can see, node "venc" just has compatible string, it's just used
for register "/dev/videoX" device node in probe process. The encoder
real hardware device are node "venc_core0" and "venc_core1", bind them
to "venc" in probe process.

> It seems this series is somewhat related to Yunfei's "[PATCH v5,
> 00/15] Using component framework to support multi hardware decode",
> but I don't see a device tree patch either in that series.
> 
> Given this is a complex architecture, I don't know if it
> makes sense to discuss decoder and encoder independently.
> 
> If you guys unify the two series, and add the device tree patches for
> it,
> or at least for the most complex cases, maybe that will surface the
> architecture more clearly and come up with an easier solution that
> doesn't involve
> an async framework to pull in the parts together.
> 
> Thanks,
> Ezequiel

Yes, this series have a similar purpose with Yunfei's patch, he has a
requirement for dual cores decoder, mine are dual cores encoder, about
the usage of framework component, it can be discussed together.

Thanks

[PATCH 2/9] media: mtk-vcodec: Use component framework to manage encoder hardware

[Irui Wang]

There are two venc cores for H264 inside MT8195, uses component
framework to manage each hardware so that the two cores can be
used through one venc driver.

Adds new encoder hardware mode to indicate different hardware
modes:
VENC_SINGLE_CORE_MODE means only one core, the device has its
own power/clk/irq, init_pm/request_irq helper can be used.

VENC_FRAME_RACING_MODE means more than one core inside, the core
device driver can use the init_pm/request_irq helper to
initialize their own power/clk/irq. And the master device doesn't
need use these helper anymore.

Signed-off-by: Irui Wang <irui.wang@mediatek.com>
---
 drivers/media/platform/mtk-vcodec/Makefile    |   1 +
 .../platform/mtk-vcodec/mtk_vcodec_drv.h      |  22 ++
 .../platform/mtk-vcodec/mtk_vcodec_enc.h      |   2 +
 .../platform/mtk-vcodec/mtk_vcodec_enc_drv.c  | 189 +++++++++++++++---
 .../platform/mtk-vcodec/mtk_vcodec_enc_hw.c   | 179 +++++++++++++++++
 .../platform/mtk-vcodec/mtk_vcodec_enc_hw.h   |  36 ++++
 6 files changed, 397 insertions(+), 32 deletions(-)
 create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.c
 create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.h

diff --git a/drivers/media/platform/mtk-vcodec/Makefile b/drivers/media/platform/mtk-vcodec/Makefile
index 4618d43dbbc8..661d4afeb628 100644
--- a/drivers/media/platform/mtk-vcodec/Makefile
+++ b/drivers/media/platform/mtk-vcodec/Makefile
@@ -20,6 +20,7 @@ mtk-vcodec-enc-y := venc/venc_vp8_if.o \
 		mtk_vcodec_enc_pm.o \
 		venc_drv_if.o \
 		venc_vpu_if.o \
+		mtk_vcodec_enc_hw.o \
 
 
 mtk-vcodec-common-y := mtk_vcodec_intr.o \
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
index 3f83710b4fa5..c160b9e505eb 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
@@ -8,6 +8,7 @@
 #ifndef _MTK_VCODEC_DRV_H_
 #define _MTK_VCODEC_DRV_H_
 
+#include <linux/component.h>
 #include <linux/platform_device.h>
 #include <linux/videodev2.h>
 #include <media/v4l2-ctrls.h>
@@ -92,6 +93,20 @@ enum mtk_fmt_type {
 	MTK_FMT_FRAME = 2,
 };
 
+enum mtk_venc_hw_id {
+	MTK_VENC_CORE0 = 0,
+	MTK_VENC_CORE1 = 1,
+	MTK_VENC_HW_MAX,
+};
+
+/**
+ * enmu mtk_venc_hw_mode - Used to indicate different encode mode
+ */
+enum mtk_venc_hw_mode {
+	VENC_SINGLE_CORE_MODE = 0,
+	VENC_FRAME_RACING_MODE = 1,
+};
+
 /*
  * struct mtk_video_fmt - Structure used to store information about pixelformats
  */
@@ -320,6 +335,7 @@ enum mtk_chip {
  * @output_formats: array of supported output formats
  * @num_output_formats: number of entries in output_formats
  * @core_id: stand for h264 or vp8 encode index
+ * @hw_mode: indicate encode mode
  */
 struct mtk_vcodec_enc_pdata {
 	enum mtk_chip chip;
@@ -332,6 +348,7 @@ struct mtk_vcodec_enc_pdata {
 	const struct mtk_video_fmt *output_formats;
 	size_t num_output_formats;
 	int core_id;
+	enum mtk_venc_hw_mode hw_mode;
 };
 
 #define MTK_ENC_CTX_IS_EXT(ctx) ((ctx)->dev->venc_pdata->uses_ext)
@@ -372,6 +389,8 @@ struct mtk_vcodec_enc_pdata {
  * @pm: power management control
  * @dec_capability: used to identify decode capability, ex: 4k
  * @enc_capability: used to identify encode capability
+ * @enc_comp_node: used to store venc component node
+ * @enc_comp_dev: used to store venc component device
  */
 struct mtk_vcodec_dev {
 	struct v4l2_device v4l2_dev;
@@ -407,6 +426,9 @@ struct mtk_vcodec_dev {
 	struct mtk_vcodec_pm pm;
 	unsigned int dec_capability;
 	unsigned int enc_capability;
+
+	struct device_node *enc_comp_node[MTK_VENC_HW_MAX];
+	void *enc_comp_dev[MTK_VENC_HW_MAX];
 };
 
 static inline struct mtk_vcodec_ctx *fh_to_ctx(struct v4l2_fh *fh)
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h
index 513ee7993e34..30b6d7f22ed7 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h
@@ -21,6 +21,8 @@
 #define MTK_VENC_IRQ_STATUS_OFFSET	0x05C
 #define MTK_VENC_IRQ_ACK_OFFSET	0x060
 
+extern struct platform_driver mtk_venc_comp_driver;
+
 /**
  * struct mtk_video_enc_buf - Private data related to each VB2 buffer.
  * @m2m_buf:	M2M buffer
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
index 7b3e0ea4c410..f7538d4b5cbb 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
@@ -26,6 +26,81 @@
 module_param(mtk_v4l2_dbg_level, int, S_IRUGO | S_IWUSR);
 module_param(mtk_vcodec_dbg, bool, S_IRUGO | S_IWUSR);
 
+static const struct of_device_id mtk_venc_comp_ids[] = {
+	{
+		.compatible = "mediatek,mtk-venc-core0",
+		.data = (void *)MTK_VENC_CORE0,
+	},
+	{
+		.compatible = "mediatek,mtk-venc-core1",
+		.data = (void *)MTK_VENC_CORE1,
+	},
+	{},
+};
+
+static inline int mtk_venc_compare_of(struct device *dev, void *data)
+{
+	return dev->of_node == data;
+}
+
+static inline void mtk_venc_release_of(struct device *dev, void *data)
+{
+	of_node_put(data);
+}
+
+static inline int mtk_venc_bind(struct device *dev)
+{
+	struct mtk_vcodec_dev *data = dev_get_drvdata(dev);
+
+	return component_bind_all(dev, data);
+}
+
+static inline void mtk_venc_unbind(struct device *dev)
+{
+	struct mtk_vcodec_dev *data = dev_get_drvdata(dev);
+
+	component_unbind_all(dev, data);
+}
+
+static const struct component_master_ops mtk_venc_ops = {
+	.bind = mtk_venc_bind,
+	.unbind = mtk_venc_unbind,
+};
+
+static struct component_match *mtk_venc_match_add(struct mtk_vcodec_dev *dev)
+{
+	struct platform_device *pdev = dev->plat_dev;
+	struct component_match *match = NULL;
+	int i;
+
+	for (i = 0; i < ARRAY_SIZE(mtk_venc_comp_ids); i++) {
+		enum mtk_venc_hw_id comp_idx;
+		struct device_node *comp_node;
+		const struct of_device_id *of_id;
+
+		comp_node = of_find_compatible_node(NULL, NULL,
+			mtk_venc_comp_ids[i].compatible);
+		if (!comp_node)
+			continue;
+
+		of_id = of_match_node(mtk_venc_comp_ids, comp_node);
+		if (!of_id) {
+			dev_err(&pdev->dev, "Failed to get match node\n");
+			return ERR_PTR(-EINVAL);
+		}
+
+		comp_idx = (enum mtk_venc_hw_id)of_id->data;
+		dev->enc_comp_node[comp_idx] = comp_node;
+
+		component_match_add_release(&pdev->dev, &match,
+					    mtk_venc_release_of,
+					    mtk_venc_compare_of,
+					    comp_node);
+	}
+
+	return match;
+}
+
 static const struct mtk_video_fmt mtk_video_formats_output[] = {
 	{
 		.fourcc = V4L2_PIX_FMT_NV12M,
@@ -239,6 +314,7 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
 	phandle rproc_phandle;
 	enum mtk_vcodec_fw_type fw_type;
 	int ret;
+	struct component_match *match = NULL;
 
 	dev = devm_kzalloc(&pdev->dev, sizeof(*dev), GFP_KERNEL);
 	if (!dev)
@@ -264,39 +340,41 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
 		return PTR_ERR(dev->fw_handler);
 
 	dev->venc_pdata = of_device_get_match_data(&pdev->dev);
-	ret = mtk_vcodec_init_enc_pm(dev);
-	if (ret < 0) {
-		dev_err(&pdev->dev, "Failed to get mtk vcodec clock source!");
-		goto err_enc_pm;
-	}
+	if (dev->venc_pdata->hw_mode == VENC_SINGLE_CORE_MODE) {
+		ret = mtk_vcodec_init_enc_pm(dev);
+		if (ret < 0) {
+			dev_err(&pdev->dev, "Failed to get mtk vcodec clock source!");
+			goto err_enc_pm;
+		}
 
-	pm_runtime_enable(&pdev->dev);
+		pm_runtime_enable(&pdev->dev);
 
-	dev->reg_base[dev->venc_pdata->core_id] =
-		devm_platform_ioremap_resource(pdev, 0);
-	if (IS_ERR(dev->reg_base[dev->venc_pdata->core_id])) {
-		ret = PTR_ERR(dev->reg_base[dev->venc_pdata->core_id]);
-		goto err_res;
-	}
+		dev->reg_base[dev->venc_pdata->core_id] =
+			devm_platform_ioremap_resource(pdev, 0);
+		if (IS_ERR(dev->reg_base[dev->venc_pdata->core_id])) {
+			ret = PTR_ERR(dev->reg_base[dev->venc_pdata->core_id]);
+			goto err_res;
+		}
 
-	res = platform_get_resource(pdev, IORESOURCE_IRQ, 0);
-	if (res == NULL) {
-		dev_err(&pdev->dev, "failed to get irq resource");
-		ret = -ENOENT;
-		goto err_res;
-	}
+		res = platform_get_resource(pdev, IORESOURCE_IRQ, 0);
+		if (!res) {
+			dev_err(&pdev->dev, "failed to get irq resource");
+			ret = -ENOENT;
+			goto err_res;
+		}
 
-	dev->enc_irq = platform_get_irq(pdev, 0);
-	irq_set_status_flags(dev->enc_irq, IRQ_NOAUTOEN);
-	ret = devm_request_irq(&pdev->dev, dev->enc_irq,
-			       mtk_vcodec_enc_irq_handler,
-			       0, pdev->name, dev);
-	if (ret) {
-		dev_err(&pdev->dev,
-			"Failed to install dev->enc_irq %d (%d) core_id (%d)",
-			dev->enc_irq, ret, dev->venc_pdata->core_id);
-		ret = -EINVAL;
-		goto err_res;
+		dev->enc_irq = platform_get_irq(pdev, 0);
+		irq_set_status_flags(dev->enc_irq, IRQ_NOAUTOEN);
+		ret = devm_request_irq(&pdev->dev, dev->enc_irq,
+				       mtk_vcodec_enc_irq_handler,
+				       0, pdev->name, dev);
+		if (ret) {
+			dev_err(&pdev->dev,
+				"Failed to install dev->enc_irq %d (%d) core_id (%d)",
+				dev->enc_irq, ret, dev->venc_pdata->core_id);
+			ret = -EINVAL;
+			goto err_res;
+		}
 	}
 
 	mutex_init(&dev->enc_mutex);
@@ -362,6 +440,22 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
 		goto err_enc_reg;
 	}
 
+	if (dev->venc_pdata->hw_mode == VENC_FRAME_RACING_MODE) {
+		match = mtk_venc_match_add(dev);
+		if (IS_ERR_OR_NULL(match)) {
+			mtk_v4l2_err("match venc component failed\n");
+			ret = -EINVAL;
+			goto err_enc_reg;
+		}
+		ret = component_master_add_with_match(&pdev->dev,
+						      &mtk_venc_ops, match);
+		if (ret) {
+			mtk_v4l2_err("Failed to bring up master");
+			ret = -EINVAL;
+			goto err_enc_reg;
+		}
+	}
+
 	mtk_v4l2_debug(0, "encoder %d registered as /dev/video%d",
 		       dev->venc_pdata->core_id, vfd_enc->num);
 
@@ -376,7 +470,8 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
 err_enc_alloc:
 	v4l2_device_unregister(&dev->v4l2_dev);
 err_res:
-	mtk_vcodec_release_enc_pm(dev);
+	if (dev->venc_pdata->hw_mode == VENC_SINGLE_CORE_MODE)
+		mtk_vcodec_release_enc_pm(dev);
 err_enc_pm:
 	mtk_vcodec_fw_release(dev->fw_handler);
 	return ret;
@@ -391,6 +486,7 @@ static const struct mtk_vcodec_enc_pdata mt8173_avc_pdata = {
 	.min_bitrate = 1,
 	.max_bitrate = 4000000,
 	.core_id = VENC_SYS,
+	.hw_mode = VENC_SINGLE_CORE_MODE,
 };
 
 static const struct mtk_vcodec_enc_pdata mt8173_vp8_pdata = {
@@ -402,6 +498,7 @@ static const struct mtk_vcodec_enc_pdata mt8173_vp8_pdata = {
 	.min_bitrate = 64,
 	.max_bitrate = 4000000,
 	.core_id = VENC_LT_SYS,
+	.hw_mode = VENC_SINGLE_CORE_MODE,
 };
 
 static const struct mtk_vcodec_enc_pdata mt8183_pdata = {
@@ -414,6 +511,7 @@ static const struct mtk_vcodec_enc_pdata mt8183_pdata = {
 	.min_bitrate = 64,
 	.max_bitrate = 40000000,
 	.core_id = VENC_SYS,
+	.hw_mode = VENC_SINGLE_CORE_MODE,
 };
 
 static const struct mtk_vcodec_enc_pdata mt8192_pdata = {
@@ -426,6 +524,7 @@ static const struct mtk_vcodec_enc_pdata mt8192_pdata = {
 	.min_bitrate = 64,
 	.max_bitrate = 100000000,
 	.core_id = VENC_SYS,
+	.hw_mode = VENC_SINGLE_CORE_MODE,
 };
 
 static const struct mtk_vcodec_enc_pdata mt8195_pdata = {
@@ -438,6 +537,7 @@ static const struct mtk_vcodec_enc_pdata mt8195_pdata = {
 	.min_bitrate = 64,
 	.max_bitrate = 100000000,
 	.core_id = VENC_SYS,
+	.hw_mode = VENC_FRAME_RACING_MODE,
 };
 
 static const struct of_device_id mtk_vcodec_enc_match[] = {
@@ -466,8 +566,15 @@ static int mtk_vcodec_enc_remove(struct platform_device *pdev)
 		video_unregister_device(dev->vfd_enc);
 
 	v4l2_device_unregister(&dev->v4l2_dev);
-	mtk_vcodec_release_enc_pm(dev);
+
+	if (dev->venc_pdata->hw_mode == VENC_SINGLE_CORE_MODE)
+		mtk_vcodec_release_enc_pm(dev);
+
 	mtk_vcodec_fw_release(dev->fw_handler);
+
+	if (dev->venc_pdata->hw_mode == VENC_FRAME_RACING_MODE)
+		component_master_del(&pdev->dev, &mtk_venc_ops);
+
 	return 0;
 }
 
@@ -480,7 +587,25 @@ static struct platform_driver mtk_vcodec_enc_driver = {
 	},
 };
 
-module_platform_driver(mtk_vcodec_enc_driver);
+static struct platform_driver * const mtk_venc_drivers[] = {
+	&mtk_venc_comp_driver,
+	&mtk_vcodec_enc_driver,
+};
+
+static int __init mtk_venc_init(void)
+{
+	return platform_register_drivers(mtk_venc_drivers,
+					 ARRAY_SIZE(mtk_venc_drivers));
+}
+
+static void __exit mtk_venc_exit(void)
+{
+	platform_unregister_drivers(mtk_venc_drivers,
+				    ARRAY_SIZE(mtk_venc_drivers));
+}
+
+module_init(mtk_venc_init);
+module_exit(mtk_venc_exit);
 
 
 MODULE_LICENSE("GPL v2");
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.c
new file mode 100644
index 000000000000..4e6a8a81ff67
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.c
@@ -0,0 +1,179 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2021 MediaTek Inc.
+ */
+
+#include <linux/pm_runtime.h>
+#include <linux/interrupt.h>
+#include <linux/irq.h>
+#include <linux/of_platform.h>
+#include <linux/module.h>
+
+#include "mtk_vcodec_enc_hw.h"
+#include "mtk_vcodec_enc.h"
+
+static void clean_irq_status(unsigned int irq_status, void __iomem *addr)
+{
+	if (irq_status & MTK_VENC_IRQ_STATUS_PAUSE)
+		writel(MTK_VENC_IRQ_STATUS_PAUSE, addr);
+
+	if (irq_status & MTK_VENC_IRQ_STATUS_SWITCH)
+		writel(MTK_VENC_IRQ_STATUS_SWITCH, addr);
+
+	if (irq_status & MTK_VENC_IRQ_STATUS_DRAM)
+		writel(MTK_VENC_IRQ_STATUS_DRAM, addr);
+
+	if (irq_status & MTK_VENC_IRQ_STATUS_SPS)
+		writel(MTK_VENC_IRQ_STATUS_SPS, addr);
+
+	if (irq_status & MTK_VENC_IRQ_STATUS_PPS)
+		writel(MTK_VENC_IRQ_STATUS_PPS, addr);
+
+	if (irq_status & MTK_VENC_IRQ_STATUS_FRM)
+		writel(MTK_VENC_IRQ_STATUS_FRM, addr);
+}
+
+static irqreturn_t mtk_enc_comp_irq_handler(int irq, void *priv)
+{
+	struct mtk_venc_comp_dev *dev = priv;
+	struct mtk_vcodec_ctx *ctx;
+	unsigned long flags;
+	void __iomem *addr;
+
+	spin_lock_irqsave(&dev->master_dev->irqlock, flags);
+	ctx = dev->curr_ctx;
+	spin_unlock_irqrestore(&dev->master_dev->irqlock, flags);
+	if (!ctx)
+		return IRQ_HANDLED;
+
+	addr = dev->reg_base + MTK_VENC_IRQ_ACK_OFFSET;
+	ctx->irq_status = readl(dev->reg_base + MTK_VENC_IRQ_STATUS_OFFSET);
+	clean_irq_status(ctx->irq_status, addr);
+
+	return IRQ_HANDLED;
+}
+
+static int mtk_venc_comp_bind(struct device *dev,
+			      struct device *master, void *data)
+{
+	struct mtk_venc_comp_dev *comp_dev = dev_get_drvdata(dev);
+	struct mtk_vcodec_dev *master_dev = data;
+	int i;
+
+	for (i = 0; i < MTK_VENC_HW_MAX; i++) {
+		if (dev->of_node != master_dev->enc_comp_node[i])
+			continue;
+
+		/*add component device by order*/
+		if (comp_dev->core_id == MTK_VENC_CORE0)
+			master_dev->enc_comp_dev[MTK_VENC_CORE0] = comp_dev;
+		else if (comp_dev->core_id == MTK_VENC_CORE1)
+			master_dev->enc_comp_dev[MTK_VENC_CORE1] = comp_dev;
+		else
+			return -EINVAL;
+
+		comp_dev->master_dev = master_dev;
+		break;
+	}
+
+	if (i == MTK_VENC_HW_MAX) {
+		dev_err(dev, "Failed to get component node\n");
+		return -EINVAL;
+	}
+
+	return 0;
+}
+
+static void mtk_venc_comp_unbind(struct device *dev,
+				 struct device *master, void *data)
+{
+}
+
+static const struct component_ops mtk_venc_component_ops = {
+	.bind = mtk_venc_comp_bind,
+	.unbind = mtk_venc_comp_unbind,
+};
+
+static int mtk_venc_comp_probe(struct platform_device *pdev)
+{
+	struct mtk_venc_comp_dev *comp_dev;
+	int ret;
+
+	comp_dev = devm_kzalloc(&pdev->dev, sizeof(*comp_dev), GFP_KERNEL);
+	if (!comp_dev)
+		return -ENOMEM;
+
+	comp_dev->plat_dev = pdev;
+
+	comp_dev->core_id =
+		(enum mtk_venc_hw_id)of_device_get_match_data(&pdev->dev);
+
+	comp_dev->reg_base = devm_platform_ioremap_resource(pdev, 0);
+	if (IS_ERR(comp_dev->reg_base)) {
+		dev_err(&pdev->dev, "Failed to get reg base");
+		ret = PTR_ERR(comp_dev->reg_base);
+		goto err;
+	}
+
+	comp_dev->enc_irq = platform_get_irq(pdev, 0);
+	if (comp_dev->enc_irq < 0) {
+		dev_err(&pdev->dev, "Failed to get irq resource");
+		ret = comp_dev->enc_irq;
+		goto err;
+	}
+
+	ret = devm_request_irq(&pdev->dev, comp_dev->enc_irq,
+			       mtk_enc_comp_irq_handler, 0,
+			       pdev->name, comp_dev);
+	if (ret) {
+		dev_err(&pdev->dev,
+			"Failed to install comp_dev->enc_irq %d (%d)",
+			comp_dev->enc_irq, ret);
+		ret = -EINVAL;
+		goto err;
+	}
+
+	if (of_get_property(pdev->dev.of_node, "dma-ranges", NULL))
+		dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(34));
+
+	platform_set_drvdata(pdev, comp_dev);
+
+	ret = component_add(&pdev->dev, &mtk_venc_component_ops);
+	if (ret) {
+		dev_err(&pdev->dev, "Failed to add component: %d\n", ret);
+		goto err;
+	}
+
+	return 0;
+err:
+	return ret;
+}
+
+static int mtk_venc_comp_remove(struct platform_device *pdev)
+{
+	component_del(&pdev->dev, &mtk_venc_component_ops);
+	return 0;
+}
+
+static const struct of_device_id mtk_venc_comp_ids[] = {
+	{
+		.compatible = "mediatek,mtk-venc-core0",
+		.data = (void *)MTK_VENC_CORE0,
+	},
+	{
+		.compatible = "mediatek,mtk-venc-core1",
+		.data = (void *)MTK_VENC_CORE1,
+	},
+	{},
+};
+
+MODULE_DEVICE_TABLE(of, mtk_venc_comp_ids);
+
+struct platform_driver mtk_venc_comp_driver = {
+	.probe  = mtk_venc_comp_probe,
+	.remove = mtk_venc_comp_remove,
+	.driver = {
+		.name	 = "mtk-venc-comp",
+		.of_match_table = mtk_venc_comp_ids,
+	},
+};
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.h
new file mode 100644
index 000000000000..f9ae97e252dc
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.h
@@ -0,0 +1,36 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2021 MediaTek Inc.
+ */
+
+#ifndef _MTK_VCODEC_ENC_HW_H_
+#define _MTK_VCODEC_ENC_HW_H_
+
+#include <linux/platform_device.h>
+#include "mtk_vcodec_drv.h"
+
+/*
+ * struct mtk_venc_comp_dev - driver data
+ * @plat_dev: platform_device
+ * @master_dev: master device
+ * @pm: power management data
+ * @curr_ctx: the context that is waiting for venc hardware
+ * @reg_base: mapped address of venc registers
+ * @irq_status: venc hw irq status
+ * @enc_irq: venc device irq
+ * @core id: for venc core id: core#0, core#1...
+ */
+struct mtk_venc_comp_dev {
+	struct platform_device *plat_dev;
+	struct mtk_vcodec_dev *master_dev;
+
+	struct mtk_vcodec_pm pm;
+	struct mtk_vcodec_ctx *curr_ctx;
+
+	void __iomem *reg_base;
+	unsigned int irq_status;
+	int enc_irq;
+	int core_id;
+};
+
+#endif /* _MTK_VCODEC_ENC_HW_H_ */
-- 
2.25.1

[PATCH 2/9] media: mtk-vcodec: Use component framework to manage encoder hardware

[Irui Wang]

There are two venc cores for H264 inside MT8195, uses component
framework to manage each hardware so that the two cores can be
used through one venc driver.

Adds new encoder hardware mode to indicate different hardware
modes:
VENC_SINGLE_CORE_MODE means only one core, the device has its
own power/clk/irq, init_pm/request_irq helper can be used.

VENC_FRAME_RACING_MODE means more than one core inside, the core
device driver can use the init_pm/request_irq helper to
initialize their own power/clk/irq. And the master device doesn't
need use these helper anymore.

Signed-off-by: Irui Wang <irui.wang@mediatek.com>
---
 drivers/media/platform/mtk-vcodec/Makefile    |   1 +
 .../platform/mtk-vcodec/mtk_vcodec_drv.h      |  22 ++
 .../platform/mtk-vcodec/mtk_vcodec_enc.h      |   2 +
 .../platform/mtk-vcodec/mtk_vcodec_enc_drv.c  | 189 +++++++++++++++---
 .../platform/mtk-vcodec/mtk_vcodec_enc_hw.c   | 179 +++++++++++++++++
 .../platform/mtk-vcodec/mtk_vcodec_enc_hw.h   |  36 ++++
 6 files changed, 397 insertions(+), 32 deletions(-)
 create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.c
 create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.h

diff --git a/drivers/media/platform/mtk-vcodec/Makefile b/drivers/media/platform/mtk-vcodec/Makefile
index 4618d43dbbc8..661d4afeb628 100644
--- a/drivers/media/platform/mtk-vcodec/Makefile
+++ b/drivers/media/platform/mtk-vcodec/Makefile
@@ -20,6 +20,7 @@ mtk-vcodec-enc-y := venc/venc_vp8_if.o \
 		mtk_vcodec_enc_pm.o \
 		venc_drv_if.o \
 		venc_vpu_if.o \
+		mtk_vcodec_enc_hw.o \
 
 
 mtk-vcodec-common-y := mtk_vcodec_intr.o \
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
index 3f83710b4fa5..c160b9e505eb 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
@@ -8,6 +8,7 @@
 #ifndef _MTK_VCODEC_DRV_H_
 #define _MTK_VCODEC_DRV_H_
 
+#include <linux/component.h>
 #include <linux/platform_device.h>
 #include <linux/videodev2.h>
 #include <media/v4l2-ctrls.h>
@@ -92,6 +93,20 @@ enum mtk_fmt_type {
 	MTK_FMT_FRAME = 2,
 };
 
+enum mtk_venc_hw_id {
+	MTK_VENC_CORE0 = 0,
+	MTK_VENC_CORE1 = 1,
+	MTK_VENC_HW_MAX,
+};
+
+/**
+ * enmu mtk_venc_hw_mode - Used to indicate different encode mode
+ */
+enum mtk_venc_hw_mode {
+	VENC_SINGLE_CORE_MODE = 0,
+	VENC_FRAME_RACING_MODE = 1,
+};
+
 /*
  * struct mtk_video_fmt - Structure used to store information about pixelformats
  */
@@ -320,6 +335,7 @@ enum mtk_chip {
  * @output_formats: array of supported output formats
  * @num_output_formats: number of entries in output_formats
  * @core_id: stand for h264 or vp8 encode index
+ * @hw_mode: indicate encode mode
  */
 struct mtk_vcodec_enc_pdata {
 	enum mtk_chip chip;
@@ -332,6 +348,7 @@ struct mtk_vcodec_enc_pdata {
 	const struct mtk_video_fmt *output_formats;
 	size_t num_output_formats;
 	int core_id;
+	enum mtk_venc_hw_mode hw_mode;
 };
 
 #define MTK_ENC_CTX_IS_EXT(ctx) ((ctx)->dev->venc_pdata->uses_ext)
@@ -372,6 +389,8 @@ struct mtk_vcodec_enc_pdata {
  * @pm: power management control
  * @dec_capability: used to identify decode capability, ex: 4k
  * @enc_capability: used to identify encode capability
+ * @enc_comp_node: used to store venc component node
+ * @enc_comp_dev: used to store venc component device
  */
 struct mtk_vcodec_dev {
 	struct v4l2_device v4l2_dev;
@@ -407,6 +426,9 @@ struct mtk_vcodec_dev {
 	struct mtk_vcodec_pm pm;
 	unsigned int dec_capability;
 	unsigned int enc_capability;
+
+	struct device_node *enc_comp_node[MTK_VENC_HW_MAX];
+	void *enc_comp_dev[MTK_VENC_HW_MAX];
 };
 
 static inline struct mtk_vcodec_ctx *fh_to_ctx(struct v4l2_fh *fh)
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h
index 513ee7993e34..30b6d7f22ed7 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h
@@ -21,6 +21,8 @@
 #define MTK_VENC_IRQ_STATUS_OFFSET	0x05C
 #define MTK_VENC_IRQ_ACK_OFFSET	0x060
 
+extern struct platform_driver mtk_venc_comp_driver;
+
 /**
  * struct mtk_video_enc_buf - Private data related to each VB2 buffer.
  * @m2m_buf:	M2M buffer
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
index 7b3e0ea4c410..f7538d4b5cbb 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
@@ -26,6 +26,81 @@
 module_param(mtk_v4l2_dbg_level, int, S_IRUGO | S_IWUSR);
 module_param(mtk_vcodec_dbg, bool, S_IRUGO | S_IWUSR);
 
+static const struct of_device_id mtk_venc_comp_ids[] = {
+	{
+		.compatible = "mediatek,mtk-venc-core0",
+		.data = (void *)MTK_VENC_CORE0,
+	},
+	{
+		.compatible = "mediatek,mtk-venc-core1",
+		.data = (void *)MTK_VENC_CORE1,
+	},
+	{},
+};
+
+static inline int mtk_venc_compare_of(struct device *dev, void *data)
+{
+	return dev->of_node == data;
+}
+
+static inline void mtk_venc_release_of(struct device *dev, void *data)
+{
+	of_node_put(data);
+}
+
+static inline int mtk_venc_bind(struct device *dev)
+{
+	struct mtk_vcodec_dev *data = dev_get_drvdata(dev);
+
+	return component_bind_all(dev, data);
+}
+
+static inline void mtk_venc_unbind(struct device *dev)
+{
+	struct mtk_vcodec_dev *data = dev_get_drvdata(dev);
+
+	component_unbind_all(dev, data);
+}
+
+static const struct component_master_ops mtk_venc_ops = {
+	.bind = mtk_venc_bind,
+	.unbind = mtk_venc_unbind,
+};
+
+static struct component_match *mtk_venc_match_add(struct mtk_vcodec_dev *dev)
+{
+	struct platform_device *pdev = dev->plat_dev;
+	struct component_match *match = NULL;
+	int i;
+
+	for (i = 0; i < ARRAY_SIZE(mtk_venc_comp_ids); i++) {
+		enum mtk_venc_hw_id comp_idx;
+		struct device_node *comp_node;
+		const struct of_device_id *of_id;
+
+		comp_node = of_find_compatible_node(NULL, NULL,
+			mtk_venc_comp_ids[i].compatible);
+		if (!comp_node)
+			continue;
+
+		of_id = of_match_node(mtk_venc_comp_ids, comp_node);
+		if (!of_id) {
+			dev_err(&pdev->dev, "Failed to get match node\n");
+			return ERR_PTR(-EINVAL);
+		}
+
+		comp_idx = (enum mtk_venc_hw_id)of_id->data;
+		dev->enc_comp_node[comp_idx] = comp_node;
+
+		component_match_add_release(&pdev->dev, &match,
+					    mtk_venc_release_of,
+					    mtk_venc_compare_of,
+					    comp_node);
+	}
+
+	return match;
+}
+
 static const struct mtk_video_fmt mtk_video_formats_output[] = {
 	{
 		.fourcc = V4L2_PIX_FMT_NV12M,
@@ -239,6 +314,7 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
 	phandle rproc_phandle;
 	enum mtk_vcodec_fw_type fw_type;
 	int ret;
+	struct component_match *match = NULL;
 
 	dev = devm_kzalloc(&pdev->dev, sizeof(*dev), GFP_KERNEL);
 	if (!dev)
@@ -264,39 +340,41 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
 		return PTR_ERR(dev->fw_handler);
 
 	dev->venc_pdata = of_device_get_match_data(&pdev->dev);
-	ret = mtk_vcodec_init_enc_pm(dev);
-	if (ret < 0) {
-		dev_err(&pdev->dev, "Failed to get mtk vcodec clock source!");
-		goto err_enc_pm;
-	}
+	if (dev->venc_pdata->hw_mode == VENC_SINGLE_CORE_MODE) {
+		ret = mtk_vcodec_init_enc_pm(dev);
+		if (ret < 0) {
+			dev_err(&pdev->dev, "Failed to get mtk vcodec clock source!");
+			goto err_enc_pm;
+		}
 
-	pm_runtime_enable(&pdev->dev);
+		pm_runtime_enable(&pdev->dev);
 
-	dev->reg_base[dev->venc_pdata->core_id] =
-		devm_platform_ioremap_resource(pdev, 0);
-	if (IS_ERR(dev->reg_base[dev->venc_pdata->core_id])) {
-		ret = PTR_ERR(dev->reg_base[dev->venc_pdata->core_id]);
-		goto err_res;
-	}
+		dev->reg_base[dev->venc_pdata->core_id] =
+			devm_platform_ioremap_resource(pdev, 0);
+		if (IS_ERR(dev->reg_base[dev->venc_pdata->core_id])) {
+			ret = PTR_ERR(dev->reg_base[dev->venc_pdata->core_id]);
+			goto err_res;
+		}
 
-	res = platform_get_resource(pdev, IORESOURCE_IRQ, 0);
-	if (res == NULL) {
-		dev_err(&pdev->dev, "failed to get irq resource");
-		ret = -ENOENT;
-		goto err_res;
-	}
+		res = platform_get_resource(pdev, IORESOURCE_IRQ, 0);
+		if (!res) {
+			dev_err(&pdev->dev, "failed to get irq resource");
+			ret = -ENOENT;
+			goto err_res;
+		}
 
-	dev->enc_irq = platform_get_irq(pdev, 0);
-	irq_set_status_flags(dev->enc_irq, IRQ_NOAUTOEN);
-	ret = devm_request_irq(&pdev->dev, dev->enc_irq,
-			       mtk_vcodec_enc_irq_handler,
-			       0, pdev->name, dev);
-	if (ret) {
-		dev_err(&pdev->dev,
-			"Failed to install dev->enc_irq %d (%d) core_id (%d)",
-			dev->enc_irq, ret, dev->venc_pdata->core_id);
-		ret = -EINVAL;
-		goto err_res;
+		dev->enc_irq = platform_get_irq(pdev, 0);
+		irq_set_status_flags(dev->enc_irq, IRQ_NOAUTOEN);
+		ret = devm_request_irq(&pdev->dev, dev->enc_irq,
+				       mtk_vcodec_enc_irq_handler,
+				       0, pdev->name, dev);
+		if (ret) {
+			dev_err(&pdev->dev,
+				"Failed to install dev->enc_irq %d (%d) core_id (%d)",
+				dev->enc_irq, ret, dev->venc_pdata->core_id);
+			ret = -EINVAL;
+			goto err_res;
+		}
 	}
 
 	mutex_init(&dev->enc_mutex);
@@ -362,6 +440,22 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
 		goto err_enc_reg;
 	}
 
+	if (dev->venc_pdata->hw_mode == VENC_FRAME_RACING_MODE) {
+		match = mtk_venc_match_add(dev);
+		if (IS_ERR_OR_NULL(match)) {
+			mtk_v4l2_err("match venc component failed\n");
+			ret = -EINVAL;
+			goto err_enc_reg;
+		}
+		ret = component_master_add_with_match(&pdev->dev,
+						      &mtk_venc_ops, match);
+		if (ret) {
+			mtk_v4l2_err("Failed to bring up master");
+			ret = -EINVAL;
+			goto err_enc_reg;
+		}
+	}
+
 	mtk_v4l2_debug(0, "encoder %d registered as /dev/video%d",
 		       dev->venc_pdata->core_id, vfd_enc->num);
 
@@ -376,7 +470,8 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
 err_enc_alloc:
 	v4l2_device_unregister(&dev->v4l2_dev);
 err_res:
-	mtk_vcodec_release_enc_pm(dev);
+	if (dev->venc_pdata->hw_mode == VENC_SINGLE_CORE_MODE)
+		mtk_vcodec_release_enc_pm(dev);
 err_enc_pm:
 	mtk_vcodec_fw_release(dev->fw_handler);
 	return ret;
@@ -391,6 +486,7 @@ static const struct mtk_vcodec_enc_pdata mt8173_avc_pdata = {
 	.min_bitrate = 1,
 	.max_bitrate = 4000000,
 	.core_id = VENC_SYS,
+	.hw_mode = VENC_SINGLE_CORE_MODE,
 };
 
 static const struct mtk_vcodec_enc_pdata mt8173_vp8_pdata = {
@@ -402,6 +498,7 @@ static const struct mtk_vcodec_enc_pdata mt8173_vp8_pdata = {
 	.min_bitrate = 64,
 	.max_bitrate = 4000000,
 	.core_id = VENC_LT_SYS,
+	.hw_mode = VENC_SINGLE_CORE_MODE,
 };
 
 static const struct mtk_vcodec_enc_pdata mt8183_pdata = {
@@ -414,6 +511,7 @@ static const struct mtk_vcodec_enc_pdata mt8183_pdata = {
 	.min_bitrate = 64,
 	.max_bitrate = 40000000,
 	.core_id = VENC_SYS,
+	.hw_mode = VENC_SINGLE_CORE_MODE,
 };
 
 static const struct mtk_vcodec_enc_pdata mt8192_pdata = {
@@ -426,6 +524,7 @@ static const struct mtk_vcodec_enc_pdata mt8192_pdata = {
 	.min_bitrate = 64,
 	.max_bitrate = 100000000,
 	.core_id = VENC_SYS,
+	.hw_mode = VENC_SINGLE_CORE_MODE,
 };
 
 static const struct mtk_vcodec_enc_pdata mt8195_pdata = {
@@ -438,6 +537,7 @@ static const struct mtk_vcodec_enc_pdata mt8195_pdata = {
 	.min_bitrate = 64,
 	.max_bitrate = 100000000,
 	.core_id = VENC_SYS,
+	.hw_mode = VENC_FRAME_RACING_MODE,
 };
 
 static const struct of_device_id mtk_vcodec_enc_match[] = {
@@ -466,8 +566,15 @@ static int mtk_vcodec_enc_remove(struct platform_device *pdev)
 		video_unregister_device(dev->vfd_enc);
 
 	v4l2_device_unregister(&dev->v4l2_dev);
-	mtk_vcodec_release_enc_pm(dev);
+
+	if (dev->venc_pdata->hw_mode == VENC_SINGLE_CORE_MODE)
+		mtk_vcodec_release_enc_pm(dev);
+
 	mtk_vcodec_fw_release(dev->fw_handler);
+
+	if (dev->venc_pdata->hw_mode == VENC_FRAME_RACING_MODE)
+		component_master_del(&pdev->dev, &mtk_venc_ops);
+
 	return 0;
 }
 
@@ -480,7 +587,25 @@ static struct platform_driver mtk_vcodec_enc_driver = {
 	},
 };
 
-module_platform_driver(mtk_vcodec_enc_driver);
+static struct platform_driver * const mtk_venc_drivers[] = {
+	&mtk_venc_comp_driver,
+	&mtk_vcodec_enc_driver,
+};
+
+static int __init mtk_venc_init(void)
+{
+	return platform_register_drivers(mtk_venc_drivers,
+					 ARRAY_SIZE(mtk_venc_drivers));
+}
+
+static void __exit mtk_venc_exit(void)
+{
+	platform_unregister_drivers(mtk_venc_drivers,
+				    ARRAY_SIZE(mtk_venc_drivers));
+}
+
+module_init(mtk_venc_init);
+module_exit(mtk_venc_exit);
 
 
 MODULE_LICENSE("GPL v2");
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.c
new file mode 100644
index 000000000000..4e6a8a81ff67
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.c
@@ -0,0 +1,179 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2021 MediaTek Inc.
+ */
+
+#include <linux/pm_runtime.h>
+#include <linux/interrupt.h>
+#include <linux/irq.h>
+#include <linux/of_platform.h>
+#include <linux/module.h>
+
+#include "mtk_vcodec_enc_hw.h"
+#include "mtk_vcodec_enc.h"
+
+static void clean_irq_status(unsigned int irq_status, void __iomem *addr)
+{
+	if (irq_status & MTK_VENC_IRQ_STATUS_PAUSE)
+		writel(MTK_VENC_IRQ_STATUS_PAUSE, addr);
+
+	if (irq_status & MTK_VENC_IRQ_STATUS_SWITCH)
+		writel(MTK_VENC_IRQ_STATUS_SWITCH, addr);
+
+	if (irq_status & MTK_VENC_IRQ_STATUS_DRAM)
+		writel(MTK_VENC_IRQ_STATUS_DRAM, addr);
+
+	if (irq_status & MTK_VENC_IRQ_STATUS_SPS)
+		writel(MTK_VENC_IRQ_STATUS_SPS, addr);
+
+	if (irq_status & MTK_VENC_IRQ_STATUS_PPS)
+		writel(MTK_VENC_IRQ_STATUS_PPS, addr);
+
+	if (irq_status & MTK_VENC_IRQ_STATUS_FRM)
+		writel(MTK_VENC_IRQ_STATUS_FRM, addr);
+}
+
+static irqreturn_t mtk_enc_comp_irq_handler(int irq, void *priv)
+{
+	struct mtk_venc_comp_dev *dev = priv;
+	struct mtk_vcodec_ctx *ctx;
+	unsigned long flags;
+	void __iomem *addr;
+
+	spin_lock_irqsave(&dev->master_dev->irqlock, flags);
+	ctx = dev->curr_ctx;
+	spin_unlock_irqrestore(&dev->master_dev->irqlock, flags);
+	if (!ctx)
+		return IRQ_HANDLED;
+
+	addr = dev->reg_base + MTK_VENC_IRQ_ACK_OFFSET;
+	ctx->irq_status = readl(dev->reg_base + MTK_VENC_IRQ_STATUS_OFFSET);
+	clean_irq_status(ctx->irq_status, addr);
+
+	return IRQ_HANDLED;
+}
+
+static int mtk_venc_comp_bind(struct device *dev,
+			      struct device *master, void *data)
+{
+	struct mtk_venc_comp_dev *comp_dev = dev_get_drvdata(dev);
+	struct mtk_vcodec_dev *master_dev = data;
+	int i;
+
+	for (i = 0; i < MTK_VENC_HW_MAX; i++) {
+		if (dev->of_node != master_dev->enc_comp_node[i])
+			continue;
+
+		/*add component device by order*/
+		if (comp_dev->core_id == MTK_VENC_CORE0)
+			master_dev->enc_comp_dev[MTK_VENC_CORE0] = comp_dev;
+		else if (comp_dev->core_id == MTK_VENC_CORE1)
+			master_dev->enc_comp_dev[MTK_VENC_CORE1] = comp_dev;
+		else
+			return -EINVAL;
+
+		comp_dev->master_dev = master_dev;
+		break;
+	}
+
+	if (i == MTK_VENC_HW_MAX) {
+		dev_err(dev, "Failed to get component node\n");
+		return -EINVAL;
+	}
+
+	return 0;
+}
+
+static void mtk_venc_comp_unbind(struct device *dev,
+				 struct device *master, void *data)
+{
+}
+
+static const struct component_ops mtk_venc_component_ops = {
+	.bind = mtk_venc_comp_bind,
+	.unbind = mtk_venc_comp_unbind,
+};
+
+static int mtk_venc_comp_probe(struct platform_device *pdev)
+{
+	struct mtk_venc_comp_dev *comp_dev;
+	int ret;
+
+	comp_dev = devm_kzalloc(&pdev->dev, sizeof(*comp_dev), GFP_KERNEL);
+	if (!comp_dev)
+		return -ENOMEM;
+
+	comp_dev->plat_dev = pdev;
+
+	comp_dev->core_id =
+		(enum mtk_venc_hw_id)of_device_get_match_data(&pdev->dev);
+
+	comp_dev->reg_base = devm_platform_ioremap_resource(pdev, 0);
+	if (IS_ERR(comp_dev->reg_base)) {
+		dev_err(&pdev->dev, "Failed to get reg base");
+		ret = PTR_ERR(comp_dev->reg_base);
+		goto err;
+	}
+
+	comp_dev->enc_irq = platform_get_irq(pdev, 0);
+	if (comp_dev->enc_irq < 0) {
+		dev_err(&pdev->dev, "Failed to get irq resource");
+		ret = comp_dev->enc_irq;
+		goto err;
+	}
+
+	ret = devm_request_irq(&pdev->dev, comp_dev->enc_irq,
+			       mtk_enc_comp_irq_handler, 0,
+			       pdev->name, comp_dev);
+	if (ret) {
+		dev_err(&pdev->dev,
+			"Failed to install comp_dev->enc_irq %d (%d)",
+			comp_dev->enc_irq, ret);
+		ret = -EINVAL;
+		goto err;
+	}
+
+	if (of_get_property(pdev->dev.of_node, "dma-ranges", NULL))
+		dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(34));
+
+	platform_set_drvdata(pdev, comp_dev);
+
+	ret = component_add(&pdev->dev, &mtk_venc_component_ops);
+	if (ret) {
+		dev_err(&pdev->dev, "Failed to add component: %d\n", ret);
+		goto err;
+	}
+
+	return 0;
+err:
+	return ret;
+}
+
+static int mtk_venc_comp_remove(struct platform_device *pdev)
+{
+	component_del(&pdev->dev, &mtk_venc_component_ops);
+	return 0;
+}
+
+static const struct of_device_id mtk_venc_comp_ids[] = {
+	{
+		.compatible = "mediatek,mtk-venc-core0",
+		.data = (void *)MTK_VENC_CORE0,
+	},
+	{
+		.compatible = "mediatek,mtk-venc-core1",
+		.data = (void *)MTK_VENC_CORE1,
+	},
+	{},
+};
+
+MODULE_DEVICE_TABLE(of, mtk_venc_comp_ids);
+
+struct platform_driver mtk_venc_comp_driver = {
+	.probe  = mtk_venc_comp_probe,
+	.remove = mtk_venc_comp_remove,
+	.driver = {
+		.name	 = "mtk-venc-comp",
+		.of_match_table = mtk_venc_comp_ids,
+	},
+};
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.h
new file mode 100644
index 000000000000..f9ae97e252dc
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.h
@@ -0,0 +1,36 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2021 MediaTek Inc.
+ */
+
+#ifndef _MTK_VCODEC_ENC_HW_H_
+#define _MTK_VCODEC_ENC_HW_H_
+
+#include <linux/platform_device.h>
+#include "mtk_vcodec_drv.h"
+
+/*
+ * struct mtk_venc_comp_dev - driver data
+ * @plat_dev: platform_device
+ * @master_dev: master device
+ * @pm: power management data
+ * @curr_ctx: the context that is waiting for venc hardware
+ * @reg_base: mapped address of venc registers
+ * @irq_status: venc hw irq status
+ * @enc_irq: venc device irq
+ * @core id: for venc core id: core#0, core#1...
+ */
+struct mtk_venc_comp_dev {
+	struct platform_device *plat_dev;
+	struct mtk_vcodec_dev *master_dev;
+
+	struct mtk_vcodec_pm pm;
+	struct mtk_vcodec_ctx *curr_ctx;
+
+	void __iomem *reg_base;
+	unsigned int irq_status;
+	int enc_irq;
+	int core_id;
+};
+
+#endif /* _MTK_VCODEC_ENC_HW_H_ */
-- 
2.25.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH 2/9] media: mtk-vcodec: Use component framework to manage encoder hardware

[Irui Wang]

There are two venc cores for H264 inside MT8195, uses component
framework to manage each hardware so that the two cores can be
used through one venc driver.

Adds new encoder hardware mode to indicate different hardware
modes:
VENC_SINGLE_CORE_MODE means only one core, the device has its
own power/clk/irq, init_pm/request_irq helper can be used.

VENC_FRAME_RACING_MODE means more than one core inside, the core
device driver can use the init_pm/request_irq helper to
initialize their own power/clk/irq. And the master device doesn't
need use these helper anymore.

Signed-off-by: Irui Wang <irui.wang@mediatek.com>
---
 drivers/media/platform/mtk-vcodec/Makefile    |   1 +
 .../platform/mtk-vcodec/mtk_vcodec_drv.h      |  22 ++
 .../platform/mtk-vcodec/mtk_vcodec_enc.h      |   2 +
 .../platform/mtk-vcodec/mtk_vcodec_enc_drv.c  | 189 +++++++++++++++---
 .../platform/mtk-vcodec/mtk_vcodec_enc_hw.c   | 179 +++++++++++++++++
 .../platform/mtk-vcodec/mtk_vcodec_enc_hw.h   |  36 ++++
 6 files changed, 397 insertions(+), 32 deletions(-)
 create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.c
 create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.h

diff --git a/drivers/media/platform/mtk-vcodec/Makefile b/drivers/media/platform/mtk-vcodec/Makefile
index 4618d43dbbc8..661d4afeb628 100644
--- a/drivers/media/platform/mtk-vcodec/Makefile
+++ b/drivers/media/platform/mtk-vcodec/Makefile
@@ -20,6 +20,7 @@ mtk-vcodec-enc-y := venc/venc_vp8_if.o \
 		mtk_vcodec_enc_pm.o \
 		venc_drv_if.o \
 		venc_vpu_if.o \
+		mtk_vcodec_enc_hw.o \
 
 
 mtk-vcodec-common-y := mtk_vcodec_intr.o \
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
index 3f83710b4fa5..c160b9e505eb 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
@@ -8,6 +8,7 @@
 #ifndef _MTK_VCODEC_DRV_H_
 #define _MTK_VCODEC_DRV_H_
 
+#include <linux/component.h>
 #include <linux/platform_device.h>
 #include <linux/videodev2.h>
 #include <media/v4l2-ctrls.h>
@@ -92,6 +93,20 @@ enum mtk_fmt_type {
 	MTK_FMT_FRAME = 2,
 };
 
+enum mtk_venc_hw_id {
+	MTK_VENC_CORE0 = 0,
+	MTK_VENC_CORE1 = 1,
+	MTK_VENC_HW_MAX,
+};
+
+/**
+ * enmu mtk_venc_hw_mode - Used to indicate different encode mode
+ */
+enum mtk_venc_hw_mode {
+	VENC_SINGLE_CORE_MODE = 0,
+	VENC_FRAME_RACING_MODE = 1,
+};
+
 /*
  * struct mtk_video_fmt - Structure used to store information about pixelformats
  */
@@ -320,6 +335,7 @@ enum mtk_chip {
  * @output_formats: array of supported output formats
  * @num_output_formats: number of entries in output_formats
  * @core_id: stand for h264 or vp8 encode index
+ * @hw_mode: indicate encode mode
  */
 struct mtk_vcodec_enc_pdata {
 	enum mtk_chip chip;
@@ -332,6 +348,7 @@ struct mtk_vcodec_enc_pdata {
 	const struct mtk_video_fmt *output_formats;
 	size_t num_output_formats;
 	int core_id;
+	enum mtk_venc_hw_mode hw_mode;
 };
 
 #define MTK_ENC_CTX_IS_EXT(ctx) ((ctx)->dev->venc_pdata->uses_ext)
@@ -372,6 +389,8 @@ struct mtk_vcodec_enc_pdata {
  * @pm: power management control
  * @dec_capability: used to identify decode capability, ex: 4k
  * @enc_capability: used to identify encode capability
+ * @enc_comp_node: used to store venc component node
+ * @enc_comp_dev: used to store venc component device
  */
 struct mtk_vcodec_dev {
 	struct v4l2_device v4l2_dev;
@@ -407,6 +426,9 @@ struct mtk_vcodec_dev {
 	struct mtk_vcodec_pm pm;
 	unsigned int dec_capability;
 	unsigned int enc_capability;
+
+	struct device_node *enc_comp_node[MTK_VENC_HW_MAX];
+	void *enc_comp_dev[MTK_VENC_HW_MAX];
 };
 
 static inline struct mtk_vcodec_ctx *fh_to_ctx(struct v4l2_fh *fh)
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h
index 513ee7993e34..30b6d7f22ed7 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h
@@ -21,6 +21,8 @@
 #define MTK_VENC_IRQ_STATUS_OFFSET	0x05C
 #define MTK_VENC_IRQ_ACK_OFFSET	0x060
 
+extern struct platform_driver mtk_venc_comp_driver;
+
 /**
  * struct mtk_video_enc_buf - Private data related to each VB2 buffer.
  * @m2m_buf:	M2M buffer
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
index 7b3e0ea4c410..f7538d4b5cbb 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
@@ -26,6 +26,81 @@
 module_param(mtk_v4l2_dbg_level, int, S_IRUGO | S_IWUSR);
 module_param(mtk_vcodec_dbg, bool, S_IRUGO | S_IWUSR);
 
+static const struct of_device_id mtk_venc_comp_ids[] = {
+	{
+		.compatible = "mediatek,mtk-venc-core0",
+		.data = (void *)MTK_VENC_CORE0,
+	},
+	{
+		.compatible = "mediatek,mtk-venc-core1",
+		.data = (void *)MTK_VENC_CORE1,
+	},
+	{},
+};
+
+static inline int mtk_venc_compare_of(struct device *dev, void *data)
+{
+	return dev->of_node == data;
+}
+
+static inline void mtk_venc_release_of(struct device *dev, void *data)
+{
+	of_node_put(data);
+}
+
+static inline int mtk_venc_bind(struct device *dev)
+{
+	struct mtk_vcodec_dev *data = dev_get_drvdata(dev);
+
+	return component_bind_all(dev, data);
+}
+
+static inline void mtk_venc_unbind(struct device *dev)
+{
+	struct mtk_vcodec_dev *data = dev_get_drvdata(dev);
+
+	component_unbind_all(dev, data);
+}
+
+static const struct component_master_ops mtk_venc_ops = {
+	.bind = mtk_venc_bind,
+	.unbind = mtk_venc_unbind,
+};
+
+static struct component_match *mtk_venc_match_add(struct mtk_vcodec_dev *dev)
+{
+	struct platform_device *pdev = dev->plat_dev;
+	struct component_match *match = NULL;
+	int i;
+
+	for (i = 0; i < ARRAY_SIZE(mtk_venc_comp_ids); i++) {
+		enum mtk_venc_hw_id comp_idx;
+		struct device_node *comp_node;
+		const struct of_device_id *of_id;
+
+		comp_node = of_find_compatible_node(NULL, NULL,
+			mtk_venc_comp_ids[i].compatible);
+		if (!comp_node)
+			continue;
+
+		of_id = of_match_node(mtk_venc_comp_ids, comp_node);
+		if (!of_id) {
+			dev_err(&pdev->dev, "Failed to get match node\n");
+			return ERR_PTR(-EINVAL);
+		}
+
+		comp_idx = (enum mtk_venc_hw_id)of_id->data;
+		dev->enc_comp_node[comp_idx] = comp_node;
+
+		component_match_add_release(&pdev->dev, &match,
+					    mtk_venc_release_of,
+					    mtk_venc_compare_of,
+					    comp_node);
+	}
+
+	return match;
+}
+
 static const struct mtk_video_fmt mtk_video_formats_output[] = {
 	{
 		.fourcc = V4L2_PIX_FMT_NV12M,
@@ -239,6 +314,7 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
 	phandle rproc_phandle;
 	enum mtk_vcodec_fw_type fw_type;
 	int ret;
+	struct component_match *match = NULL;
 
 	dev = devm_kzalloc(&pdev->dev, sizeof(*dev), GFP_KERNEL);
 	if (!dev)
@@ -264,39 +340,41 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
 		return PTR_ERR(dev->fw_handler);
 
 	dev->venc_pdata = of_device_get_match_data(&pdev->dev);
-	ret = mtk_vcodec_init_enc_pm(dev);
-	if (ret < 0) {
-		dev_err(&pdev->dev, "Failed to get mtk vcodec clock source!");
-		goto err_enc_pm;
-	}
+	if (dev->venc_pdata->hw_mode == VENC_SINGLE_CORE_MODE) {
+		ret = mtk_vcodec_init_enc_pm(dev);
+		if (ret < 0) {
+			dev_err(&pdev->dev, "Failed to get mtk vcodec clock source!");
+			goto err_enc_pm;
+		}
 
-	pm_runtime_enable(&pdev->dev);
+		pm_runtime_enable(&pdev->dev);
 
-	dev->reg_base[dev->venc_pdata->core_id] =
-		devm_platform_ioremap_resource(pdev, 0);
-	if (IS_ERR(dev->reg_base[dev->venc_pdata->core_id])) {
-		ret = PTR_ERR(dev->reg_base[dev->venc_pdata->core_id]);
-		goto err_res;
-	}
+		dev->reg_base[dev->venc_pdata->core_id] =
+			devm_platform_ioremap_resource(pdev, 0);
+		if (IS_ERR(dev->reg_base[dev->venc_pdata->core_id])) {
+			ret = PTR_ERR(dev->reg_base[dev->venc_pdata->core_id]);
+			goto err_res;
+		}
 
-	res = platform_get_resource(pdev, IORESOURCE_IRQ, 0);
-	if (res == NULL) {
-		dev_err(&pdev->dev, "failed to get irq resource");
-		ret = -ENOENT;
-		goto err_res;
-	}
+		res = platform_get_resource(pdev, IORESOURCE_IRQ, 0);
+		if (!res) {
+			dev_err(&pdev->dev, "failed to get irq resource");
+			ret = -ENOENT;
+			goto err_res;
+		}
 
-	dev->enc_irq = platform_get_irq(pdev, 0);
-	irq_set_status_flags(dev->enc_irq, IRQ_NOAUTOEN);
-	ret = devm_request_irq(&pdev->dev, dev->enc_irq,
-			       mtk_vcodec_enc_irq_handler,
-			       0, pdev->name, dev);
-	if (ret) {
-		dev_err(&pdev->dev,
-			"Failed to install dev->enc_irq %d (%d) core_id (%d)",
-			dev->enc_irq, ret, dev->venc_pdata->core_id);
-		ret = -EINVAL;
-		goto err_res;
+		dev->enc_irq = platform_get_irq(pdev, 0);
+		irq_set_status_flags(dev->enc_irq, IRQ_NOAUTOEN);
+		ret = devm_request_irq(&pdev->dev, dev->enc_irq,
+				       mtk_vcodec_enc_irq_handler,
+				       0, pdev->name, dev);
+		if (ret) {
+			dev_err(&pdev->dev,
+				"Failed to install dev->enc_irq %d (%d) core_id (%d)",
+				dev->enc_irq, ret, dev->venc_pdata->core_id);
+			ret = -EINVAL;
+			goto err_res;
+		}
 	}
 
 	mutex_init(&dev->enc_mutex);
@@ -362,6 +440,22 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
 		goto err_enc_reg;
 	}
 
+	if (dev->venc_pdata->hw_mode == VENC_FRAME_RACING_MODE) {
+		match = mtk_venc_match_add(dev);
+		if (IS_ERR_OR_NULL(match)) {
+			mtk_v4l2_err("match venc component failed\n");
+			ret = -EINVAL;
+			goto err_enc_reg;
+		}
+		ret = component_master_add_with_match(&pdev->dev,
+						      &mtk_venc_ops, match);
+		if (ret) {
+			mtk_v4l2_err("Failed to bring up master");
+			ret = -EINVAL;
+			goto err_enc_reg;
+		}
+	}
+
 	mtk_v4l2_debug(0, "encoder %d registered as /dev/video%d",
 		       dev->venc_pdata->core_id, vfd_enc->num);
 
@@ -376,7 +470,8 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
 err_enc_alloc:
 	v4l2_device_unregister(&dev->v4l2_dev);
 err_res:
-	mtk_vcodec_release_enc_pm(dev);
+	if (dev->venc_pdata->hw_mode == VENC_SINGLE_CORE_MODE)
+		mtk_vcodec_release_enc_pm(dev);
 err_enc_pm:
 	mtk_vcodec_fw_release(dev->fw_handler);
 	return ret;
@@ -391,6 +486,7 @@ static const struct mtk_vcodec_enc_pdata mt8173_avc_pdata = {
 	.min_bitrate = 1,
 	.max_bitrate = 4000000,
 	.core_id = VENC_SYS,
+	.hw_mode = VENC_SINGLE_CORE_MODE,
 };
 
 static const struct mtk_vcodec_enc_pdata mt8173_vp8_pdata = {
@@ -402,6 +498,7 @@ static const struct mtk_vcodec_enc_pdata mt8173_vp8_pdata = {
 	.min_bitrate = 64,
 	.max_bitrate = 4000000,
 	.core_id = VENC_LT_SYS,
+	.hw_mode = VENC_SINGLE_CORE_MODE,
 };
 
 static const struct mtk_vcodec_enc_pdata mt8183_pdata = {
@@ -414,6 +511,7 @@ static const struct mtk_vcodec_enc_pdata mt8183_pdata = {
 	.min_bitrate = 64,
 	.max_bitrate = 40000000,
 	.core_id = VENC_SYS,
+	.hw_mode = VENC_SINGLE_CORE_MODE,
 };
 
 static const struct mtk_vcodec_enc_pdata mt8192_pdata = {
@@ -426,6 +524,7 @@ static const struct mtk_vcodec_enc_pdata mt8192_pdata = {
 	.min_bitrate = 64,
 	.max_bitrate = 100000000,
 	.core_id = VENC_SYS,
+	.hw_mode = VENC_SINGLE_CORE_MODE,
 };
 
 static const struct mtk_vcodec_enc_pdata mt8195_pdata = {
@@ -438,6 +537,7 @@ static const struct mtk_vcodec_enc_pdata mt8195_pdata = {
 	.min_bitrate = 64,
 	.max_bitrate = 100000000,
 	.core_id = VENC_SYS,
+	.hw_mode = VENC_FRAME_RACING_MODE,
 };
 
 static const struct of_device_id mtk_vcodec_enc_match[] = {
@@ -466,8 +566,15 @@ static int mtk_vcodec_enc_remove(struct platform_device *pdev)
 		video_unregister_device(dev->vfd_enc);
 
 	v4l2_device_unregister(&dev->v4l2_dev);
-	mtk_vcodec_release_enc_pm(dev);
+
+	if (dev->venc_pdata->hw_mode == VENC_SINGLE_CORE_MODE)
+		mtk_vcodec_release_enc_pm(dev);
+
 	mtk_vcodec_fw_release(dev->fw_handler);
+
+	if (dev->venc_pdata->hw_mode == VENC_FRAME_RACING_MODE)
+		component_master_del(&pdev->dev, &mtk_venc_ops);
+
 	return 0;
 }
 
@@ -480,7 +587,25 @@ static struct platform_driver mtk_vcodec_enc_driver = {
 	},
 };
 
-module_platform_driver(mtk_vcodec_enc_driver);
+static struct platform_driver * const mtk_venc_drivers[] = {
+	&mtk_venc_comp_driver,
+	&mtk_vcodec_enc_driver,
+};
+
+static int __init mtk_venc_init(void)
+{
+	return platform_register_drivers(mtk_venc_drivers,
+					 ARRAY_SIZE(mtk_venc_drivers));
+}
+
+static void __exit mtk_venc_exit(void)
+{
+	platform_unregister_drivers(mtk_venc_drivers,
+				    ARRAY_SIZE(mtk_venc_drivers));
+}
+
+module_init(mtk_venc_init);
+module_exit(mtk_venc_exit);
 
 
 MODULE_LICENSE("GPL v2");
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.c
new file mode 100644
index 000000000000..4e6a8a81ff67
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.c
@@ -0,0 +1,179 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2021 MediaTek Inc.
+ */
+
+#include <linux/pm_runtime.h>
+#include <linux/interrupt.h>
+#include <linux/irq.h>
+#include <linux/of_platform.h>
+#include <linux/module.h>
+
+#include "mtk_vcodec_enc_hw.h"
+#include "mtk_vcodec_enc.h"
+
+static void clean_irq_status(unsigned int irq_status, void __iomem *addr)
+{
+	if (irq_status & MTK_VENC_IRQ_STATUS_PAUSE)
+		writel(MTK_VENC_IRQ_STATUS_PAUSE, addr);
+
+	if (irq_status & MTK_VENC_IRQ_STATUS_SWITCH)
+		writel(MTK_VENC_IRQ_STATUS_SWITCH, addr);
+
+	if (irq_status & MTK_VENC_IRQ_STATUS_DRAM)
+		writel(MTK_VENC_IRQ_STATUS_DRAM, addr);
+
+	if (irq_status & MTK_VENC_IRQ_STATUS_SPS)
+		writel(MTK_VENC_IRQ_STATUS_SPS, addr);
+
+	if (irq_status & MTK_VENC_IRQ_STATUS_PPS)
+		writel(MTK_VENC_IRQ_STATUS_PPS, addr);
+
+	if (irq_status & MTK_VENC_IRQ_STATUS_FRM)
+		writel(MTK_VENC_IRQ_STATUS_FRM, addr);
+}
+
+static irqreturn_t mtk_enc_comp_irq_handler(int irq, void *priv)
+{
+	struct mtk_venc_comp_dev *dev = priv;
+	struct mtk_vcodec_ctx *ctx;
+	unsigned long flags;
+	void __iomem *addr;
+
+	spin_lock_irqsave(&dev->master_dev->irqlock, flags);
+	ctx = dev->curr_ctx;
+	spin_unlock_irqrestore(&dev->master_dev->irqlock, flags);
+	if (!ctx)
+		return IRQ_HANDLED;
+
+	addr = dev->reg_base + MTK_VENC_IRQ_ACK_OFFSET;
+	ctx->irq_status = readl(dev->reg_base + MTK_VENC_IRQ_STATUS_OFFSET);
+	clean_irq_status(ctx->irq_status, addr);
+
+	return IRQ_HANDLED;
+}
+
+static int mtk_venc_comp_bind(struct device *dev,
+			      struct device *master, void *data)
+{
+	struct mtk_venc_comp_dev *comp_dev = dev_get_drvdata(dev);
+	struct mtk_vcodec_dev *master_dev = data;
+	int i;
+
+	for (i = 0; i < MTK_VENC_HW_MAX; i++) {
+		if (dev->of_node != master_dev->enc_comp_node[i])
+			continue;
+
+		/*add component device by order*/
+		if (comp_dev->core_id == MTK_VENC_CORE0)
+			master_dev->enc_comp_dev[MTK_VENC_CORE0] = comp_dev;
+		else if (comp_dev->core_id == MTK_VENC_CORE1)
+			master_dev->enc_comp_dev[MTK_VENC_CORE1] = comp_dev;
+		else
+			return -EINVAL;
+
+		comp_dev->master_dev = master_dev;
+		break;
+	}
+
+	if (i == MTK_VENC_HW_MAX) {
+		dev_err(dev, "Failed to get component node\n");
+		return -EINVAL;
+	}
+
+	return 0;
+}
+
+static void mtk_venc_comp_unbind(struct device *dev,
+				 struct device *master, void *data)
+{
+}
+
+static const struct component_ops mtk_venc_component_ops = {
+	.bind = mtk_venc_comp_bind,
+	.unbind = mtk_venc_comp_unbind,
+};
+
+static int mtk_venc_comp_probe(struct platform_device *pdev)
+{
+	struct mtk_venc_comp_dev *comp_dev;
+	int ret;
+
+	comp_dev = devm_kzalloc(&pdev->dev, sizeof(*comp_dev), GFP_KERNEL);
+	if (!comp_dev)
+		return -ENOMEM;
+
+	comp_dev->plat_dev = pdev;
+
+	comp_dev->core_id =
+		(enum mtk_venc_hw_id)of_device_get_match_data(&pdev->dev);
+
+	comp_dev->reg_base = devm_platform_ioremap_resource(pdev, 0);
+	if (IS_ERR(comp_dev->reg_base)) {
+		dev_err(&pdev->dev, "Failed to get reg base");
+		ret = PTR_ERR(comp_dev->reg_base);
+		goto err;
+	}
+
+	comp_dev->enc_irq = platform_get_irq(pdev, 0);
+	if (comp_dev->enc_irq < 0) {
+		dev_err(&pdev->dev, "Failed to get irq resource");
+		ret = comp_dev->enc_irq;
+		goto err;
+	}
+
+	ret = devm_request_irq(&pdev->dev, comp_dev->enc_irq,
+			       mtk_enc_comp_irq_handler, 0,
+			       pdev->name, comp_dev);
+	if (ret) {
+		dev_err(&pdev->dev,
+			"Failed to install comp_dev->enc_irq %d (%d)",
+			comp_dev->enc_irq, ret);
+		ret = -EINVAL;
+		goto err;
+	}
+
+	if (of_get_property(pdev->dev.of_node, "dma-ranges", NULL))
+		dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(34));
+
+	platform_set_drvdata(pdev, comp_dev);
+
+	ret = component_add(&pdev->dev, &mtk_venc_component_ops);
+	if (ret) {
+		dev_err(&pdev->dev, "Failed to add component: %d\n", ret);
+		goto err;
+	}
+
+	return 0;
+err:
+	return ret;
+}
+
+static int mtk_venc_comp_remove(struct platform_device *pdev)
+{
+	component_del(&pdev->dev, &mtk_venc_component_ops);
+	return 0;
+}
+
+static const struct of_device_id mtk_venc_comp_ids[] = {
+	{
+		.compatible = "mediatek,mtk-venc-core0",
+		.data = (void *)MTK_VENC_CORE0,
+	},
+	{
+		.compatible = "mediatek,mtk-venc-core1",
+		.data = (void *)MTK_VENC_CORE1,
+	},
+	{},
+};
+
+MODULE_DEVICE_TABLE(of, mtk_venc_comp_ids);
+
+struct platform_driver mtk_venc_comp_driver = {
+	.probe  = mtk_venc_comp_probe,
+	.remove = mtk_venc_comp_remove,
+	.driver = {
+		.name	 = "mtk-venc-comp",
+		.of_match_table = mtk_venc_comp_ids,
+	},
+};
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.h
new file mode 100644
index 000000000000..f9ae97e252dc
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.h
@@ -0,0 +1,36 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2021 MediaTek Inc.
+ */
+
+#ifndef _MTK_VCODEC_ENC_HW_H_
+#define _MTK_VCODEC_ENC_HW_H_
+
+#include <linux/platform_device.h>
+#include "mtk_vcodec_drv.h"
+
+/*
+ * struct mtk_venc_comp_dev - driver data
+ * @plat_dev: platform_device
+ * @master_dev: master device
+ * @pm: power management data
+ * @curr_ctx: the context that is waiting for venc hardware
+ * @reg_base: mapped address of venc registers
+ * @irq_status: venc hw irq status
+ * @enc_irq: venc device irq
+ * @core id: for venc core id: core#0, core#1...
+ */
+struct mtk_venc_comp_dev {
+	struct platform_device *plat_dev;
+	struct mtk_vcodec_dev *master_dev;
+
+	struct mtk_vcodec_pm pm;
+	struct mtk_vcodec_ctx *curr_ctx;
+
+	void __iomem *reg_base;
+	unsigned int irq_status;
+	int enc_irq;
+	int core_id;
+};
+
+#endif /* _MTK_VCODEC_ENC_HW_H_ */
-- 
2.25.1
_______________________________________________
Linux-mediatek mailing list
Linux-mediatek@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-mediatek

Re: [PATCH 2/9] media: mtk-vcodec: Use component framework to manage encoder hardware

[Tzung-Bi Shih]

On Mon, Aug 16, 2021 at 06:59:27PM +0800, Irui Wang wrote:
> +static struct component_match *mtk_venc_match_add(struct mtk_vcodec_dev *dev)
> +{
> +	struct platform_device *pdev = dev->plat_dev;
> +	struct component_match *match = NULL;
> +	int i;
> +
> +	for (i = 0; i < ARRAY_SIZE(mtk_venc_comp_ids); i++) {
> +		enum mtk_venc_hw_id comp_idx;
> +		struct device_node *comp_node;
> +		const struct of_device_id *of_id;
To be neat, prefer to define the variables outside of the loop (i.e. at the beginning of the function).

> +
> +		comp_node = of_find_compatible_node(NULL, NULL,
> +			mtk_venc_comp_ids[i].compatible);
> +		if (!comp_node)
> +			continue;
> +
> +		of_id = of_match_node(mtk_venc_comp_ids, comp_node);
> +		if (!of_id) {
> +			dev_err(&pdev->dev, "Failed to get match node\n");
Need to call of_node_put() actually, but see comment below.

> +			return ERR_PTR(-EINVAL);
> +		}
> +
> +		comp_idx = (enum mtk_venc_hw_id)of_id->data;
For getting the comp_idx, mtk_venc_comp_ids[i].data should be sufficient.  If so, of_match_node() can be removed so that the error handling path won't need to call of_node_put().

> @@ -239,6 +314,7 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
>  	phandle rproc_phandle;
>  	enum mtk_vcodec_fw_type fw_type;
>  	int ret;
> +	struct component_match *match = NULL;
It doesn't need to be initialized.

> -	res = platform_get_resource(pdev, IORESOURCE_IRQ, 0);
> -	if (res == NULL) {
> -		dev_err(&pdev->dev, "failed to get irq resource");
> -		ret = -ENOENT;
> -		goto err_res;
> -	}
> +		res = platform_get_resource(pdev, IORESOURCE_IRQ, 0);
> +		if (!res) {
> +			dev_err(&pdev->dev, "failed to get irq resource");
> +			ret = -ENOENT;
> +			goto err_res;
> +		}
res is not used.  Can be removed in next version or in another patch.

> diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.c
> new file mode 100644
> index 000000000000..4e6a8a81ff67
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.c
> @@ -0,0 +1,179 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Copyright (c) 2021 MediaTek Inc.
> + */
> +
> +#include <linux/pm_runtime.h>
> +#include <linux/interrupt.h>
> +#include <linux/irq.h>
> +#include <linux/of_platform.h>
> +#include <linux/module.h>
Would be better to maintain an order.

> +#include "mtk_vcodec_enc_hw.h"
> +#include "mtk_vcodec_enc.h"
Would be better to maintain an order.

> +static irqreturn_t mtk_enc_comp_irq_handler(int irq, void *priv)
> +{
> +	struct mtk_venc_comp_dev *dev = priv;
> +	struct mtk_vcodec_ctx *ctx;
> +	unsigned long flags;
> +	void __iomem *addr;
> +
> +	spin_lock_irqsave(&dev->master_dev->irqlock, flags);
> +	ctx = dev->curr_ctx;
> +	spin_unlock_irqrestore(&dev->master_dev->irqlock, flags);
> +	if (!ctx)
> +		return IRQ_HANDLED;
Here is a read lock for the curr_ctx.  The patch doesn't contain the write lock part.

I am not sure if the following situation would be happened:
1. curr_ctx is not NULL.
2. mtk_enc_comp_irq_handler() gets the curr_ctx.
3. The curr_ctx has been destroyed somewhere.
4. mtk_enc_comp_irq_handler() finds the ctx is not NULL so that it continues to execute.
5. Something wrong in latter mtk_enc_comp_irq_handler() because the ctx has been destroyed.

Does it make more sense to set curr_ctx to NULL to indicate the ownership has been transferred to mtk_enc_comp_irq_handler()?  For example:

spin_lock_irqsave(...);
ctx = dev->curr_ctx;
dev->curr_ctx = NULL;
spin_unlock_irqrestore(...);

> +static int mtk_venc_comp_bind(struct device *dev,
> +			      struct device *master, void *data)
> +{
> +	struct mtk_venc_comp_dev *comp_dev = dev_get_drvdata(dev);
> +	struct mtk_vcodec_dev *master_dev = data;
> +	int i;
> +
> +	for (i = 0; i < MTK_VENC_HW_MAX; i++) {
> +		if (dev->of_node != master_dev->enc_comp_node[i])
> +			continue;
> +
> +		/*add component device by order*/
> +		if (comp_dev->core_id == MTK_VENC_CORE0)
> +			master_dev->enc_comp_dev[MTK_VENC_CORE0] = comp_dev;
> +		else if (comp_dev->core_id == MTK_VENC_CORE1)
> +			master_dev->enc_comp_dev[MTK_VENC_CORE1] = comp_dev;
> +		else
> +			return -EINVAL;
if (comp_dev->core_id < 0 || comp_dev->core_id >= MTK_VENC_HW_MAX)
    return -EINVAL;

master_dev->enc_comp_dev[comp_dev->core_id] = comp_dev;

Re: [PATCH 2/9] media: mtk-vcodec: Use component framework to manage encoder hardware

[Tzung-Bi Shih]

On Mon, Aug 16, 2021 at 06:59:27PM +0800, Irui Wang wrote:
> +static struct component_match *mtk_venc_match_add(struct mtk_vcodec_dev *dev)
> +{
> +	struct platform_device *pdev = dev->plat_dev;
> +	struct component_match *match = NULL;
> +	int i;
> +
> +	for (i = 0; i < ARRAY_SIZE(mtk_venc_comp_ids); i++) {
> +		enum mtk_venc_hw_id comp_idx;
> +		struct device_node *comp_node;
> +		const struct of_device_id *of_id;
To be neat, prefer to define the variables outside of the loop (i.e. at the beginning of the function).

> +
> +		comp_node = of_find_compatible_node(NULL, NULL,
> +			mtk_venc_comp_ids[i].compatible);
> +		if (!comp_node)
> +			continue;
> +
> +		of_id = of_match_node(mtk_venc_comp_ids, comp_node);
> +		if (!of_id) {
> +			dev_err(&pdev->dev, "Failed to get match node\n");
Need to call of_node_put() actually, but see comment below.

> +			return ERR_PTR(-EINVAL);
> +		}
> +
> +		comp_idx = (enum mtk_venc_hw_id)of_id->data;
For getting the comp_idx, mtk_venc_comp_ids[i].data should be sufficient.  If so, of_match_node() can be removed so that the error handling path won't need to call of_node_put().

> @@ -239,6 +314,7 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
>  	phandle rproc_phandle;
>  	enum mtk_vcodec_fw_type fw_type;
>  	int ret;
> +	struct component_match *match = NULL;
It doesn't need to be initialized.

> -	res = platform_get_resource(pdev, IORESOURCE_IRQ, 0);
> -	if (res == NULL) {
> -		dev_err(&pdev->dev, "failed to get irq resource");
> -		ret = -ENOENT;
> -		goto err_res;
> -	}
> +		res = platform_get_resource(pdev, IORESOURCE_IRQ, 0);
> +		if (!res) {
> +			dev_err(&pdev->dev, "failed to get irq resource");
> +			ret = -ENOENT;
> +			goto err_res;
> +		}
res is not used.  Can be removed in next version or in another patch.

> diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.c
> new file mode 100644
> index 000000000000..4e6a8a81ff67
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.c
> @@ -0,0 +1,179 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Copyright (c) 2021 MediaTek Inc.
> + */
> +
> +#include <linux/pm_runtime.h>
> +#include <linux/interrupt.h>
> +#include <linux/irq.h>
> +#include <linux/of_platform.h>
> +#include <linux/module.h>
Would be better to maintain an order.

> +#include "mtk_vcodec_enc_hw.h"
> +#include "mtk_vcodec_enc.h"
Would be better to maintain an order.

> +static irqreturn_t mtk_enc_comp_irq_handler(int irq, void *priv)
> +{
> +	struct mtk_venc_comp_dev *dev = priv;
> +	struct mtk_vcodec_ctx *ctx;
> +	unsigned long flags;
> +	void __iomem *addr;
> +
> +	spin_lock_irqsave(&dev->master_dev->irqlock, flags);
> +	ctx = dev->curr_ctx;
> +	spin_unlock_irqrestore(&dev->master_dev->irqlock, flags);
> +	if (!ctx)
> +		return IRQ_HANDLED;
Here is a read lock for the curr_ctx.  The patch doesn't contain the write lock part.

I am not sure if the following situation would be happened:
1. curr_ctx is not NULL.
2. mtk_enc_comp_irq_handler() gets the curr_ctx.
3. The curr_ctx has been destroyed somewhere.
4. mtk_enc_comp_irq_handler() finds the ctx is not NULL so that it continues to execute.
5. Something wrong in latter mtk_enc_comp_irq_handler() because the ctx has been destroyed.

Does it make more sense to set curr_ctx to NULL to indicate the ownership has been transferred to mtk_enc_comp_irq_handler()?  For example:

spin_lock_irqsave(...);
ctx = dev->curr_ctx;
dev->curr_ctx = NULL;
spin_unlock_irqrestore(...);

> +static int mtk_venc_comp_bind(struct device *dev,
> +			      struct device *master, void *data)
> +{
> +	struct mtk_venc_comp_dev *comp_dev = dev_get_drvdata(dev);
> +	struct mtk_vcodec_dev *master_dev = data;
> +	int i;
> +
> +	for (i = 0; i < MTK_VENC_HW_MAX; i++) {
> +		if (dev->of_node != master_dev->enc_comp_node[i])
> +			continue;
> +
> +		/*add component device by order*/
> +		if (comp_dev->core_id == MTK_VENC_CORE0)
> +			master_dev->enc_comp_dev[MTK_VENC_CORE0] = comp_dev;
> +		else if (comp_dev->core_id == MTK_VENC_CORE1)
> +			master_dev->enc_comp_dev[MTK_VENC_CORE1] = comp_dev;
> +		else
> +			return -EINVAL;
if (comp_dev->core_id < 0 || comp_dev->core_id >= MTK_VENC_HW_MAX)
    return -EINVAL;

master_dev->enc_comp_dev[comp_dev->core_id] = comp_dev;

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH 2/9] media: mtk-vcodec: Use component framework to manage encoder hardware

[Tzung-Bi Shih]

On Mon, Aug 16, 2021 at 06:59:27PM +0800, Irui Wang wrote:
> +static struct component_match *mtk_venc_match_add(struct mtk_vcodec_dev *dev)
> +{
> +	struct platform_device *pdev = dev->plat_dev;
> +	struct component_match *match = NULL;
> +	int i;
> +
> +	for (i = 0; i < ARRAY_SIZE(mtk_venc_comp_ids); i++) {
> +		enum mtk_venc_hw_id comp_idx;
> +		struct device_node *comp_node;
> +		const struct of_device_id *of_id;
To be neat, prefer to define the variables outside of the loop (i.e. at the beginning of the function).

> +
> +		comp_node = of_find_compatible_node(NULL, NULL,
> +			mtk_venc_comp_ids[i].compatible);
> +		if (!comp_node)
> +			continue;
> +
> +		of_id = of_match_node(mtk_venc_comp_ids, comp_node);
> +		if (!of_id) {
> +			dev_err(&pdev->dev, "Failed to get match node\n");
Need to call of_node_put() actually, but see comment below.

> +			return ERR_PTR(-EINVAL);
> +		}
> +
> +		comp_idx = (enum mtk_venc_hw_id)of_id->data;
For getting the comp_idx, mtk_venc_comp_ids[i].data should be sufficient.  If so, of_match_node() can be removed so that the error handling path won't need to call of_node_put().

> @@ -239,6 +314,7 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
>  	phandle rproc_phandle;
>  	enum mtk_vcodec_fw_type fw_type;
>  	int ret;
> +	struct component_match *match = NULL;
It doesn't need to be initialized.

> -	res = platform_get_resource(pdev, IORESOURCE_IRQ, 0);
> -	if (res == NULL) {
> -		dev_err(&pdev->dev, "failed to get irq resource");
> -		ret = -ENOENT;
> -		goto err_res;
> -	}
> +		res = platform_get_resource(pdev, IORESOURCE_IRQ, 0);
> +		if (!res) {
> +			dev_err(&pdev->dev, "failed to get irq resource");
> +			ret = -ENOENT;
> +			goto err_res;
> +		}
res is not used.  Can be removed in next version or in another patch.

> diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.c
> new file mode 100644
> index 000000000000..4e6a8a81ff67
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.c
> @@ -0,0 +1,179 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Copyright (c) 2021 MediaTek Inc.
> + */
> +
> +#include <linux/pm_runtime.h>
> +#include <linux/interrupt.h>
> +#include <linux/irq.h>
> +#include <linux/of_platform.h>
> +#include <linux/module.h>
Would be better to maintain an order.

> +#include "mtk_vcodec_enc_hw.h"
> +#include "mtk_vcodec_enc.h"
Would be better to maintain an order.

> +static irqreturn_t mtk_enc_comp_irq_handler(int irq, void *priv)
> +{
> +	struct mtk_venc_comp_dev *dev = priv;
> +	struct mtk_vcodec_ctx *ctx;
> +	unsigned long flags;
> +	void __iomem *addr;
> +
> +	spin_lock_irqsave(&dev->master_dev->irqlock, flags);
> +	ctx = dev->curr_ctx;
> +	spin_unlock_irqrestore(&dev->master_dev->irqlock, flags);
> +	if (!ctx)
> +		return IRQ_HANDLED;
Here is a read lock for the curr_ctx.  The patch doesn't contain the write lock part.

I am not sure if the following situation would be happened:
1. curr_ctx is not NULL.
2. mtk_enc_comp_irq_handler() gets the curr_ctx.
3. The curr_ctx has been destroyed somewhere.
4. mtk_enc_comp_irq_handler() finds the ctx is not NULL so that it continues to execute.
5. Something wrong in latter mtk_enc_comp_irq_handler() because the ctx has been destroyed.

Does it make more sense to set curr_ctx to NULL to indicate the ownership has been transferred to mtk_enc_comp_irq_handler()?  For example:

spin_lock_irqsave(...);
ctx = dev->curr_ctx;
dev->curr_ctx = NULL;
spin_unlock_irqrestore(...);

> +static int mtk_venc_comp_bind(struct device *dev,
> +			      struct device *master, void *data)
> +{
> +	struct mtk_venc_comp_dev *comp_dev = dev_get_drvdata(dev);
> +	struct mtk_vcodec_dev *master_dev = data;
> +	int i;
> +
> +	for (i = 0; i < MTK_VENC_HW_MAX; i++) {
> +		if (dev->of_node != master_dev->enc_comp_node[i])
> +			continue;
> +
> +		/*add component device by order*/
> +		if (comp_dev->core_id == MTK_VENC_CORE0)
> +			master_dev->enc_comp_dev[MTK_VENC_CORE0] = comp_dev;
> +		else if (comp_dev->core_id == MTK_VENC_CORE1)
> +			master_dev->enc_comp_dev[MTK_VENC_CORE1] = comp_dev;
> +		else
> +			return -EINVAL;
if (comp_dev->core_id < 0 || comp_dev->core_id >= MTK_VENC_HW_MAX)
    return -EINVAL;

master_dev->enc_comp_dev[comp_dev->core_id] = comp_dev;

_______________________________________________
Linux-mediatek mailing list
Linux-mediatek@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-mediatek

[PATCH 3/9] media: mtk-vcodec: Rewrite venc power manage interface

[Irui Wang]

The args "struct mtk_vcodec_dev *" doesn't appropriate for
init/release_enc_pm functions because of component devices. For
master device, it has no "pm/clk" properties in dtsi, component
devices will init their own "pm/clk" instead. So rewrite the pm
interface with args "platform_device *" and "mtk_vcodec_pm*".

Signed-off-by: Irui Wang <irui.wang@mediatek.com>
---
 .../platform/mtk-vcodec/mtk_vcodec_enc_drv.c  |  6 ++--
 .../platform/mtk-vcodec/mtk_vcodec_enc_hw.c   | 13 ++++++++
 .../platform/mtk-vcodec/mtk_vcodec_enc_pm.c   | 31 +++++++------------
 .../platform/mtk-vcodec/mtk_vcodec_enc_pm.h   |  5 +--
 4 files changed, 30 insertions(+), 25 deletions(-)

diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
index f7538d4b5cbb..9c2224b199d0 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
@@ -341,7 +341,7 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
 
 	dev->venc_pdata = of_device_get_match_data(&pdev->dev);
 	if (dev->venc_pdata->hw_mode == VENC_SINGLE_CORE_MODE) {
-		ret = mtk_vcodec_init_enc_pm(dev);
+		ret = mtk_vcodec_init_enc_pm(dev->plat_dev, &dev->pm);
 		if (ret < 0) {
 			dev_err(&pdev->dev, "Failed to get mtk vcodec clock source!");
 			goto err_enc_pm;
@@ -471,7 +471,7 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
 	v4l2_device_unregister(&dev->v4l2_dev);
 err_res:
 	if (dev->venc_pdata->hw_mode == VENC_SINGLE_CORE_MODE)
-		mtk_vcodec_release_enc_pm(dev);
+		mtk_vcodec_release_enc_pm(&dev->pm);
 err_enc_pm:
 	mtk_vcodec_fw_release(dev->fw_handler);
 	return ret;
@@ -568,7 +568,7 @@ static int mtk_vcodec_enc_remove(struct platform_device *pdev)
 	v4l2_device_unregister(&dev->v4l2_dev);
 
 	if (dev->venc_pdata->hw_mode == VENC_SINGLE_CORE_MODE)
-		mtk_vcodec_release_enc_pm(dev);
+		mtk_vcodec_release_enc_pm(&dev->pm);
 
 	mtk_vcodec_fw_release(dev->fw_handler);
 
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.c
index 4e6a8a81ff67..2a9112b0561d 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.c
@@ -10,6 +10,7 @@
 #include <linux/module.h>
 
 #include "mtk_vcodec_enc_hw.h"
+#include "mtk_vcodec_enc_pm.h"
 #include "mtk_vcodec_enc.h"
 
 static void clean_irq_status(unsigned int irq_status, void __iomem *addr)
@@ -105,6 +106,14 @@ static int mtk_venc_comp_probe(struct platform_device *pdev)
 
 	comp_dev->plat_dev = pdev;
 
+	ret = mtk_vcodec_init_enc_pm(pdev, &comp_dev->pm);
+	if (ret < 0) {
+		dev_err(&pdev->dev, "Failed to get venc component clock source!");
+		return ret;
+	}
+
+	pm_runtime_enable(&pdev->dev);
+
 	comp_dev->core_id =
 		(enum mtk_venc_hw_id)of_device_get_match_data(&pdev->dev);
 
@@ -146,11 +155,15 @@ static int mtk_venc_comp_probe(struct platform_device *pdev)
 
 	return 0;
 err:
+	mtk_vcodec_release_enc_pm(&comp_dev->pm);
 	return ret;
 }
 
 static int mtk_venc_comp_remove(struct platform_device *pdev)
 {
+	struct mtk_venc_comp_dev *comp_dev = platform_get_drvdata(pdev);
+
+	mtk_vcodec_release_enc_pm(&comp_dev->pm);
 	component_del(&pdev->dev, &mtk_venc_component_ops);
 	return 0;
 }
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
index 1b2e4930ed27..5edeb93fca84 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
@@ -13,38 +13,29 @@
 #include "mtk_vcodec_enc_pm.h"
 #include "mtk_vcodec_util.h"
 
-int mtk_vcodec_init_enc_pm(struct mtk_vcodec_dev *mtkdev)
+int mtk_vcodec_init_enc_pm(struct platform_device *pdev,
+			   struct mtk_vcodec_pm *pm)
 {
 	struct device_node *node;
-	struct platform_device *pdev;
-	struct mtk_vcodec_pm *pm;
+	struct platform_device *larb_pdev;
 	struct mtk_vcodec_clk *enc_clk;
 	struct mtk_vcodec_clk_info *clk_info;
 	int ret = 0, i = 0;
-	struct device *dev;
 
-	pdev = mtkdev->plat_dev;
-	pm = &mtkdev->pm;
-	memset(pm, 0, sizeof(struct mtk_vcodec_pm));
-	pm->mtkdev = mtkdev;
-	pm->dev = &pdev->dev;
-	dev = &pdev->dev;
-	enc_clk = &pm->venc_clk;
-
-	node = of_parse_phandle(dev->of_node, "mediatek,larb", 0);
+	node = of_parse_phandle(pdev->dev.of_node, "mediatek,larb", 0);
 	if (!node) {
 		mtk_v4l2_err("no mediatek,larb found");
 		return -ENODEV;
 	}
-	pdev = of_find_device_by_node(node);
+	larb_pdev = of_find_device_by_node(node);
 	of_node_put(node);
-	if (!pdev) {
+	if (!larb_pdev) {
 		mtk_v4l2_err("no mediatek,larb device found");
 		return -ENODEV;
 	}
-	pm->larbvenc = &pdev->dev;
-	pdev = mtkdev->plat_dev;
+	pm->larbvenc = &larb_pdev->dev;
 	pm->dev = &pdev->dev;
+	enc_clk = &pm->venc_clk;
 
 	enc_clk->clk_num = of_property_count_strings(pdev->dev.of_node,
 		"clock-names");
@@ -87,10 +78,10 @@ int mtk_vcodec_init_enc_pm(struct mtk_vcodec_dev *mtkdev)
 	return ret;
 }
 
-void mtk_vcodec_release_enc_pm(struct mtk_vcodec_dev *mtkdev)
+void mtk_vcodec_release_enc_pm(struct mtk_vcodec_pm *pm)
 {
-	pm_runtime_disable(mtkdev->pm.dev);
-	put_device(mtkdev->pm.larbvenc);
+	pm_runtime_disable(pm->dev);
+	put_device(pm->larbvenc);
 }
 
 
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.h
index b7ecdfd74823..f4a4aa441185 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.h
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.h
@@ -9,8 +9,9 @@
 
 #include "mtk_vcodec_drv.h"
 
-int mtk_vcodec_init_enc_pm(struct mtk_vcodec_dev *dev);
-void mtk_vcodec_release_enc_pm(struct mtk_vcodec_dev *dev);
+int mtk_vcodec_init_enc_pm(struct platform_device *pdev,
+			   struct mtk_vcodec_pm *pm);
+void mtk_vcodec_release_enc_pm(struct mtk_vcodec_pm *pm);
 
 void mtk_vcodec_enc_clock_on(struct mtk_vcodec_pm *pm);
 void mtk_vcodec_enc_clock_off(struct mtk_vcodec_pm *pm);
-- 
2.25.1

[PATCH 3/9] media: mtk-vcodec: Rewrite venc power manage interface

[Irui Wang]

The args "struct mtk_vcodec_dev *" doesn't appropriate for
init/release_enc_pm functions because of component devices. For
master device, it has no "pm/clk" properties in dtsi, component
devices will init their own "pm/clk" instead. So rewrite the pm
interface with args "platform_device *" and "mtk_vcodec_pm*".

Signed-off-by: Irui Wang <irui.wang@mediatek.com>
---
 .../platform/mtk-vcodec/mtk_vcodec_enc_drv.c  |  6 ++--
 .../platform/mtk-vcodec/mtk_vcodec_enc_hw.c   | 13 ++++++++
 .../platform/mtk-vcodec/mtk_vcodec_enc_pm.c   | 31 +++++++------------
 .../platform/mtk-vcodec/mtk_vcodec_enc_pm.h   |  5 +--
 4 files changed, 30 insertions(+), 25 deletions(-)

diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
index f7538d4b5cbb..9c2224b199d0 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
@@ -341,7 +341,7 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
 
 	dev->venc_pdata = of_device_get_match_data(&pdev->dev);
 	if (dev->venc_pdata->hw_mode == VENC_SINGLE_CORE_MODE) {
-		ret = mtk_vcodec_init_enc_pm(dev);
+		ret = mtk_vcodec_init_enc_pm(dev->plat_dev, &dev->pm);
 		if (ret < 0) {
 			dev_err(&pdev->dev, "Failed to get mtk vcodec clock source!");
 			goto err_enc_pm;
@@ -471,7 +471,7 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
 	v4l2_device_unregister(&dev->v4l2_dev);
 err_res:
 	if (dev->venc_pdata->hw_mode == VENC_SINGLE_CORE_MODE)
-		mtk_vcodec_release_enc_pm(dev);
+		mtk_vcodec_release_enc_pm(&dev->pm);
 err_enc_pm:
 	mtk_vcodec_fw_release(dev->fw_handler);
 	return ret;
@@ -568,7 +568,7 @@ static int mtk_vcodec_enc_remove(struct platform_device *pdev)
 	v4l2_device_unregister(&dev->v4l2_dev);
 
 	if (dev->venc_pdata->hw_mode == VENC_SINGLE_CORE_MODE)
-		mtk_vcodec_release_enc_pm(dev);
+		mtk_vcodec_release_enc_pm(&dev->pm);
 
 	mtk_vcodec_fw_release(dev->fw_handler);
 
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.c
index 4e6a8a81ff67..2a9112b0561d 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.c
@@ -10,6 +10,7 @@
 #include <linux/module.h>
 
 #include "mtk_vcodec_enc_hw.h"
+#include "mtk_vcodec_enc_pm.h"
 #include "mtk_vcodec_enc.h"
 
 static void clean_irq_status(unsigned int irq_status, void __iomem *addr)
@@ -105,6 +106,14 @@ static int mtk_venc_comp_probe(struct platform_device *pdev)
 
 	comp_dev->plat_dev = pdev;
 
+	ret = mtk_vcodec_init_enc_pm(pdev, &comp_dev->pm);
+	if (ret < 0) {
+		dev_err(&pdev->dev, "Failed to get venc component clock source!");
+		return ret;
+	}
+
+	pm_runtime_enable(&pdev->dev);
+
 	comp_dev->core_id =
 		(enum mtk_venc_hw_id)of_device_get_match_data(&pdev->dev);
 
@@ -146,11 +155,15 @@ static int mtk_venc_comp_probe(struct platform_device *pdev)
 
 	return 0;
 err:
+	mtk_vcodec_release_enc_pm(&comp_dev->pm);
 	return ret;
 }
 
 static int mtk_venc_comp_remove(struct platform_device *pdev)
 {
+	struct mtk_venc_comp_dev *comp_dev = platform_get_drvdata(pdev);
+
+	mtk_vcodec_release_enc_pm(&comp_dev->pm);
 	component_del(&pdev->dev, &mtk_venc_component_ops);
 	return 0;
 }
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
index 1b2e4930ed27..5edeb93fca84 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
@@ -13,38 +13,29 @@
 #include "mtk_vcodec_enc_pm.h"
 #include "mtk_vcodec_util.h"
 
-int mtk_vcodec_init_enc_pm(struct mtk_vcodec_dev *mtkdev)
+int mtk_vcodec_init_enc_pm(struct platform_device *pdev,
+			   struct mtk_vcodec_pm *pm)
 {
 	struct device_node *node;
-	struct platform_device *pdev;
-	struct mtk_vcodec_pm *pm;
+	struct platform_device *larb_pdev;
 	struct mtk_vcodec_clk *enc_clk;
 	struct mtk_vcodec_clk_info *clk_info;
 	int ret = 0, i = 0;
-	struct device *dev;
 
-	pdev = mtkdev->plat_dev;
-	pm = &mtkdev->pm;
-	memset(pm, 0, sizeof(struct mtk_vcodec_pm));
-	pm->mtkdev = mtkdev;
-	pm->dev = &pdev->dev;
-	dev = &pdev->dev;
-	enc_clk = &pm->venc_clk;
-
-	node = of_parse_phandle(dev->of_node, "mediatek,larb", 0);
+	node = of_parse_phandle(pdev->dev.of_node, "mediatek,larb", 0);
 	if (!node) {
 		mtk_v4l2_err("no mediatek,larb found");
 		return -ENODEV;
 	}
-	pdev = of_find_device_by_node(node);
+	larb_pdev = of_find_device_by_node(node);
 	of_node_put(node);
-	if (!pdev) {
+	if (!larb_pdev) {
 		mtk_v4l2_err("no mediatek,larb device found");
 		return -ENODEV;
 	}
-	pm->larbvenc = &pdev->dev;
-	pdev = mtkdev->plat_dev;
+	pm->larbvenc = &larb_pdev->dev;
 	pm->dev = &pdev->dev;
+	enc_clk = &pm->venc_clk;
 
 	enc_clk->clk_num = of_property_count_strings(pdev->dev.of_node,
 		"clock-names");
@@ -87,10 +78,10 @@ int mtk_vcodec_init_enc_pm(struct mtk_vcodec_dev *mtkdev)
 	return ret;
 }
 
-void mtk_vcodec_release_enc_pm(struct mtk_vcodec_dev *mtkdev)
+void mtk_vcodec_release_enc_pm(struct mtk_vcodec_pm *pm)
 {
-	pm_runtime_disable(mtkdev->pm.dev);
-	put_device(mtkdev->pm.larbvenc);
+	pm_runtime_disable(pm->dev);
+	put_device(pm->larbvenc);
 }
 
 
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.h
index b7ecdfd74823..f4a4aa441185 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.h
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.h
@@ -9,8 +9,9 @@
 
 #include "mtk_vcodec_drv.h"
 
-int mtk_vcodec_init_enc_pm(struct mtk_vcodec_dev *dev);
-void mtk_vcodec_release_enc_pm(struct mtk_vcodec_dev *dev);
+int mtk_vcodec_init_enc_pm(struct platform_device *pdev,
+			   struct mtk_vcodec_pm *pm);
+void mtk_vcodec_release_enc_pm(struct mtk_vcodec_pm *pm);
 
 void mtk_vcodec_enc_clock_on(struct mtk_vcodec_pm *pm);
 void mtk_vcodec_enc_clock_off(struct mtk_vcodec_pm *pm);
-- 
2.25.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH 3/9] media: mtk-vcodec: Rewrite venc power manage interface

[Irui Wang]

The args "struct mtk_vcodec_dev *" doesn't appropriate for
init/release_enc_pm functions because of component devices. For
master device, it has no "pm/clk" properties in dtsi, component
devices will init their own "pm/clk" instead. So rewrite the pm
interface with args "platform_device *" and "mtk_vcodec_pm*".

Signed-off-by: Irui Wang <irui.wang@mediatek.com>
---
 .../platform/mtk-vcodec/mtk_vcodec_enc_drv.c  |  6 ++--
 .../platform/mtk-vcodec/mtk_vcodec_enc_hw.c   | 13 ++++++++
 .../platform/mtk-vcodec/mtk_vcodec_enc_pm.c   | 31 +++++++------------
 .../platform/mtk-vcodec/mtk_vcodec_enc_pm.h   |  5 +--
 4 files changed, 30 insertions(+), 25 deletions(-)

diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
index f7538d4b5cbb..9c2224b199d0 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
@@ -341,7 +341,7 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
 
 	dev->venc_pdata = of_device_get_match_data(&pdev->dev);
 	if (dev->venc_pdata->hw_mode == VENC_SINGLE_CORE_MODE) {
-		ret = mtk_vcodec_init_enc_pm(dev);
+		ret = mtk_vcodec_init_enc_pm(dev->plat_dev, &dev->pm);
 		if (ret < 0) {
 			dev_err(&pdev->dev, "Failed to get mtk vcodec clock source!");
 			goto err_enc_pm;
@@ -471,7 +471,7 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
 	v4l2_device_unregister(&dev->v4l2_dev);
 err_res:
 	if (dev->venc_pdata->hw_mode == VENC_SINGLE_CORE_MODE)
-		mtk_vcodec_release_enc_pm(dev);
+		mtk_vcodec_release_enc_pm(&dev->pm);
 err_enc_pm:
 	mtk_vcodec_fw_release(dev->fw_handler);
 	return ret;
@@ -568,7 +568,7 @@ static int mtk_vcodec_enc_remove(struct platform_device *pdev)
 	v4l2_device_unregister(&dev->v4l2_dev);
 
 	if (dev->venc_pdata->hw_mode == VENC_SINGLE_CORE_MODE)
-		mtk_vcodec_release_enc_pm(dev);
+		mtk_vcodec_release_enc_pm(&dev->pm);
 
 	mtk_vcodec_fw_release(dev->fw_handler);
 
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.c
index 4e6a8a81ff67..2a9112b0561d 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.c
@@ -10,6 +10,7 @@
 #include <linux/module.h>
 
 #include "mtk_vcodec_enc_hw.h"
+#include "mtk_vcodec_enc_pm.h"
 #include "mtk_vcodec_enc.h"
 
 static void clean_irq_status(unsigned int irq_status, void __iomem *addr)
@@ -105,6 +106,14 @@ static int mtk_venc_comp_probe(struct platform_device *pdev)
 
 	comp_dev->plat_dev = pdev;
 
+	ret = mtk_vcodec_init_enc_pm(pdev, &comp_dev->pm);
+	if (ret < 0) {
+		dev_err(&pdev->dev, "Failed to get venc component clock source!");
+		return ret;
+	}
+
+	pm_runtime_enable(&pdev->dev);
+
 	comp_dev->core_id =
 		(enum mtk_venc_hw_id)of_device_get_match_data(&pdev->dev);
 
@@ -146,11 +155,15 @@ static int mtk_venc_comp_probe(struct platform_device *pdev)
 
 	return 0;
 err:
+	mtk_vcodec_release_enc_pm(&comp_dev->pm);
 	return ret;
 }
 
 static int mtk_venc_comp_remove(struct platform_device *pdev)
 {
+	struct mtk_venc_comp_dev *comp_dev = platform_get_drvdata(pdev);
+
+	mtk_vcodec_release_enc_pm(&comp_dev->pm);
 	component_del(&pdev->dev, &mtk_venc_component_ops);
 	return 0;
 }
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
index 1b2e4930ed27..5edeb93fca84 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
@@ -13,38 +13,29 @@
 #include "mtk_vcodec_enc_pm.h"
 #include "mtk_vcodec_util.h"
 
-int mtk_vcodec_init_enc_pm(struct mtk_vcodec_dev *mtkdev)
+int mtk_vcodec_init_enc_pm(struct platform_device *pdev,
+			   struct mtk_vcodec_pm *pm)
 {
 	struct device_node *node;
-	struct platform_device *pdev;
-	struct mtk_vcodec_pm *pm;
+	struct platform_device *larb_pdev;
 	struct mtk_vcodec_clk *enc_clk;
 	struct mtk_vcodec_clk_info *clk_info;
 	int ret = 0, i = 0;
-	struct device *dev;
 
-	pdev = mtkdev->plat_dev;
-	pm = &mtkdev->pm;
-	memset(pm, 0, sizeof(struct mtk_vcodec_pm));
-	pm->mtkdev = mtkdev;
-	pm->dev = &pdev->dev;
-	dev = &pdev->dev;
-	enc_clk = &pm->venc_clk;
-
-	node = of_parse_phandle(dev->of_node, "mediatek,larb", 0);
+	node = of_parse_phandle(pdev->dev.of_node, "mediatek,larb", 0);
 	if (!node) {
 		mtk_v4l2_err("no mediatek,larb found");
 		return -ENODEV;
 	}
-	pdev = of_find_device_by_node(node);
+	larb_pdev = of_find_device_by_node(node);
 	of_node_put(node);
-	if (!pdev) {
+	if (!larb_pdev) {
 		mtk_v4l2_err("no mediatek,larb device found");
 		return -ENODEV;
 	}
-	pm->larbvenc = &pdev->dev;
-	pdev = mtkdev->plat_dev;
+	pm->larbvenc = &larb_pdev->dev;
 	pm->dev = &pdev->dev;
+	enc_clk = &pm->venc_clk;
 
 	enc_clk->clk_num = of_property_count_strings(pdev->dev.of_node,
 		"clock-names");
@@ -87,10 +78,10 @@ int mtk_vcodec_init_enc_pm(struct mtk_vcodec_dev *mtkdev)
 	return ret;
 }
 
-void mtk_vcodec_release_enc_pm(struct mtk_vcodec_dev *mtkdev)
+void mtk_vcodec_release_enc_pm(struct mtk_vcodec_pm *pm)
 {
-	pm_runtime_disable(mtkdev->pm.dev);
-	put_device(mtkdev->pm.larbvenc);
+	pm_runtime_disable(pm->dev);
+	put_device(pm->larbvenc);
 }
 
 
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.h
index b7ecdfd74823..f4a4aa441185 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.h
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.h
@@ -9,8 +9,9 @@
 
 #include "mtk_vcodec_drv.h"
 
-int mtk_vcodec_init_enc_pm(struct mtk_vcodec_dev *dev);
-void mtk_vcodec_release_enc_pm(struct mtk_vcodec_dev *dev);
+int mtk_vcodec_init_enc_pm(struct platform_device *pdev,
+			   struct mtk_vcodec_pm *pm);
+void mtk_vcodec_release_enc_pm(struct mtk_vcodec_pm *pm);
 
 void mtk_vcodec_enc_clock_on(struct mtk_vcodec_pm *pm);
 void mtk_vcodec_enc_clock_off(struct mtk_vcodec_pm *pm);
-- 
2.25.1
_______________________________________________
Linux-mediatek mailing list
Linux-mediatek@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-mediatek

Re: [PATCH 3/9] media: mtk-vcodec: Rewrite venc power manage interface

[Tzung-Bi Shih]

On Mon, Aug 16, 2021 at 06:59:28PM +0800, Irui Wang wrote:
> @@ -105,6 +106,14 @@ static int mtk_venc_comp_probe(struct platform_device *pdev)
>  
>  	comp_dev->plat_dev = pdev;
>  
> +	ret = mtk_vcodec_init_enc_pm(pdev, &comp_dev->pm);
> +	if (ret < 0) {
> +		dev_err(&pdev->dev, "Failed to get venc component clock source!");
> +		return ret;
> +	}
> +
> +	pm_runtime_enable(&pdev->dev);
mtk_vcodec_init_enc_pm() and mtk_vcodec_release_enc_pm() is more like a pair.  Does it make more sense to call pm_runtime_enable() in mtk_vcodec_init_enc_pm()?

Re: [PATCH 3/9] media: mtk-vcodec: Rewrite venc power manage interface

[Tzung-Bi Shih]

On Mon, Aug 16, 2021 at 06:59:28PM +0800, Irui Wang wrote:
> @@ -105,6 +106,14 @@ static int mtk_venc_comp_probe(struct platform_device *pdev)
>  
>  	comp_dev->plat_dev = pdev;
>  
> +	ret = mtk_vcodec_init_enc_pm(pdev, &comp_dev->pm);
> +	if (ret < 0) {
> +		dev_err(&pdev->dev, "Failed to get venc component clock source!");
> +		return ret;
> +	}
> +
> +	pm_runtime_enable(&pdev->dev);
mtk_vcodec_init_enc_pm() and mtk_vcodec_release_enc_pm() is more like a pair.  Does it make more sense to call pm_runtime_enable() in mtk_vcodec_init_enc_pm()?

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH 3/9] media: mtk-vcodec: Rewrite venc power manage interface

[Tzung-Bi Shih]

On Mon, Aug 16, 2021 at 06:59:28PM +0800, Irui Wang wrote:
> @@ -105,6 +106,14 @@ static int mtk_venc_comp_probe(struct platform_device *pdev)
>  
>  	comp_dev->plat_dev = pdev;
>  
> +	ret = mtk_vcodec_init_enc_pm(pdev, &comp_dev->pm);
> +	if (ret < 0) {
> +		dev_err(&pdev->dev, "Failed to get venc component clock source!");
> +		return ret;
> +	}
> +
> +	pm_runtime_enable(&pdev->dev);
mtk_vcodec_init_enc_pm() and mtk_vcodec_release_enc_pm() is more like a pair.  Does it make more sense to call pm_runtime_enable() in mtk_vcodec_init_enc_pm()?

_______________________________________________
Linux-mediatek mailing list
Linux-mediatek@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-mediatek

[PATCH 4/9] media: mtk-vcodec: Add venc power on/off interface

[Irui Wang]

Frame-racing mode need power on/off all venc available power,
pm_runtime_xx helper is not appropriate called directly, add new
power on/off interface for it.

Signed-off-by: Irui Wang <irui.wang@mediatek.com>
---
 .../platform/mtk-vcodec/mtk_vcodec_enc.c      | 18 +---
 .../platform/mtk-vcodec/mtk_vcodec_enc_drv.c  |  9 +-
 .../platform/mtk-vcodec/mtk_vcodec_enc_pm.c   | 82 +++++++++++++++++++
 .../platform/mtk-vcodec/mtk_vcodec_enc_pm.h   |  4 +
 4 files changed, 96 insertions(+), 17 deletions(-)

diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
index 416f356af363..d22f9c8fd063 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
@@ -16,6 +16,7 @@
 #include "mtk_vcodec_intr.h"
 #include "mtk_vcodec_util.h"
 #include "venc_drv_if.h"
+#include "mtk_vcodec_enc_pm.h"
 
 #define MTK_VENC_MIN_W	160U
 #define MTK_VENC_MIN_H	128U
@@ -801,7 +802,7 @@ static int vb2ops_venc_start_streaming(struct vb2_queue *q, unsigned int count)
 	  */
 	if ((ctx->state == MTK_STATE_ABORT) || (ctx->state == MTK_STATE_FREE)) {
 		ret = -EIO;
-		goto err_start_stream;
+		goto err_set_param;
 	}
 
 	/* Do the initialization when both start_streaming have been called */
@@ -813,12 +814,6 @@ static int vb2ops_venc_start_streaming(struct vb2_queue *q, unsigned int count)
 			return 0;
 	}
 
-	ret = pm_runtime_resume_and_get(&ctx->dev->plat_dev->dev);
-	if (ret < 0) {
-		mtk_v4l2_err("pm_runtime_resume_and_get fail %d", ret);
-		goto err_start_stream;
-	}
-
 	mtk_venc_set_param(ctx, &param);
 	ret = venc_if_set_param(ctx, VENC_SET_PARAM_ENC, &param);
 	if (ret) {
@@ -845,11 +840,6 @@ static int vb2ops_venc_start_streaming(struct vb2_queue *q, unsigned int count)
 	return 0;
 
 err_set_param:
-	ret = pm_runtime_put(&ctx->dev->plat_dev->dev);
-	if (ret < 0)
-		mtk_v4l2_err("pm_runtime_put fail %d", ret);
-
-err_start_stream:
 	for (i = 0; i < q->num_buffers; ++i) {
 		struct vb2_buffer *buf = vb2_get_buffer(q, i);
 
@@ -903,10 +893,6 @@ static void vb2ops_venc_stop_streaming(struct vb2_queue *q)
 	if (ret)
 		mtk_v4l2_err("venc_if_deinit failed=%d", ret);
 
-	ret = pm_runtime_put(&ctx->dev->plat_dev->dev);
-	if (ret < 0)
-		mtk_v4l2_err("pm_runtime_put fail %d", ret);
-
 	ctx->state = MTK_STATE_FREE;
 }
 
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
index 9c2224b199d0..b78435e8223a 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
@@ -253,6 +253,12 @@ static int fops_vcodec_open(struct file *file)
 		mtk_v4l2_debug(0, "encoder capability %x", dev->enc_capability);
 	}
 
+	ret = mtk_vcodec_enc_power_on(ctx);
+	if (ret < 0) {
+		mtk_v4l2_err("encoder power on failed %d", ret);
+		goto err_load_fw;
+	}
+
 	mtk_v4l2_debug(2, "Create instance [%d]@%p m2m_ctx=%p ",
 			ctx->id, ctx, ctx->m2m_ctx);
 
@@ -285,11 +291,12 @@ static int fops_vcodec_release(struct file *file)
 	mtk_v4l2_debug(1, "[%d] encoder", ctx->id);
 	mutex_lock(&dev->dev_mutex);
 
+	v4l2_m2m_ctx_release(ctx->m2m_ctx);
 	mtk_vcodec_enc_release(ctx);
 	v4l2_fh_del(&ctx->fh);
 	v4l2_fh_exit(&ctx->fh);
 	v4l2_ctrl_handler_free(&ctx->ctrl_hdl);
-	v4l2_m2m_ctx_release(ctx->m2m_ctx);
+	mtk_vcodec_enc_power_off(ctx);
 
 	list_del_init(&ctx->list);
 	kfree(ctx);
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
index 5edeb93fca84..d9df0e3701f1 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
@@ -12,6 +12,7 @@
 
 #include "mtk_vcodec_enc_pm.h"
 #include "mtk_vcodec_util.h"
+#include "mtk_vcodec_enc_hw.h"
 
 int mtk_vcodec_init_enc_pm(struct platform_device *pdev,
 			   struct mtk_vcodec_pm *pm)
@@ -120,3 +121,84 @@ void mtk_vcodec_enc_clock_off(struct mtk_vcodec_pm *pm)
 	for (i = enc_clk->clk_num - 1; i >= 0; i--)
 		clk_disable_unprepare(enc_clk->clk_info[i].vcodec_clk);
 }
+
+int mtk_venc_enable_comp_hw(struct mtk_vcodec_dev *dev)
+{
+	int i, ret;
+	struct mtk_venc_comp_dev *venc_comp;
+
+	/*
+	 * frame_racing mode needs power on all available component devices.
+	 */
+	for (i = 0; i < MTK_VENC_HW_MAX; i++) {
+		venc_comp = (struct mtk_venc_comp_dev *)dev->enc_comp_dev[i];
+		if (!venc_comp)
+			return 0;
+
+		ret = mtk_smi_larb_get(venc_comp->pm.larbvenc);
+		if (ret < 0) {
+			mtk_v4l2_err("power on core[%d] fail %d", i, ret);
+				goto pw_err;
+		}
+	}
+	return 0;
+
+pw_err:
+	for (i -= 1; i >= 0; i--) {
+		venc_comp = (struct mtk_venc_comp_dev *)dev->enc_comp_dev[i];
+
+		mtk_smi_larb_put(venc_comp->pm.larbvenc);
+	}
+
+	return ret;
+}
+
+int mtk_venc_disable_comp_hw(struct mtk_vcodec_dev *dev)
+{
+	int i;
+	struct mtk_venc_comp_dev *venc_comp;
+
+	/*power off all available component device*/
+	for (i = 0; i < MTK_VENC_HW_MAX; i++) {
+		venc_comp = (struct mtk_venc_comp_dev *)dev->enc_comp_dev[i];
+		if (!venc_comp)
+			return 0;
+
+		mtk_smi_larb_put(venc_comp->pm.larbvenc);
+	}
+
+	return 0;
+}
+
+int mtk_vcodec_enc_power_on(struct mtk_vcodec_ctx *ctx)
+{
+	int ret;
+	struct mtk_vcodec_dev *dev = ctx->dev;
+
+	if (dev->venc_pdata->hw_mode == VENC_FRAME_RACING_MODE) {
+		ret = mtk_venc_enable_comp_hw(dev);
+		if (ret < 0) {
+			mtk_v4l2_err("enable venc comp hw fail :%d", ret);
+			return ret;
+		}
+	} else {
+		ret = mtk_smi_larb_get(dev->pm.larbvenc);
+		if (ret < 0) {
+			mtk_v4l2_err("pm_runtime_resume_and_get fail %d", ret);
+			return ret;
+		}
+	}
+	return 0;
+}
+
+int mtk_vcodec_enc_power_off(struct mtk_vcodec_ctx *ctx)
+{
+	struct mtk_vcodec_dev *dev = ctx->dev;
+
+	if (dev->venc_pdata->hw_mode == VENC_FRAME_RACING_MODE)
+		mtk_venc_disable_comp_hw(dev);
+	else
+		mtk_smi_larb_put(dev->pm.larbvenc);
+
+	return 0;
+}
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.h
index f4a4aa441185..2e76b858fed1 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.h
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.h
@@ -16,4 +16,8 @@ void mtk_vcodec_release_enc_pm(struct mtk_vcodec_pm *pm);
 void mtk_vcodec_enc_clock_on(struct mtk_vcodec_pm *pm);
 void mtk_vcodec_enc_clock_off(struct mtk_vcodec_pm *pm);
 
+int mtk_venc_enable_comp_hw(struct mtk_vcodec_dev *dev);
+int mtk_venc_disable_comp_hw(struct mtk_vcodec_dev *dev);
+int mtk_vcodec_enc_power_on(struct mtk_vcodec_ctx *ctx);
+int mtk_vcodec_enc_power_off(struct mtk_vcodec_ctx *ctx);
 #endif /* _MTK_VCODEC_ENC_PM_H_ */
-- 
2.25.1

[PATCH 4/9] media: mtk-vcodec: Add venc power on/off interface

[Irui Wang]

Frame-racing mode need power on/off all venc available power,
pm_runtime_xx helper is not appropriate called directly, add new
power on/off interface for it.

Signed-off-by: Irui Wang <irui.wang@mediatek.com>
---
 .../platform/mtk-vcodec/mtk_vcodec_enc.c      | 18 +---
 .../platform/mtk-vcodec/mtk_vcodec_enc_drv.c  |  9 +-
 .../platform/mtk-vcodec/mtk_vcodec_enc_pm.c   | 82 +++++++++++++++++++
 .../platform/mtk-vcodec/mtk_vcodec_enc_pm.h   |  4 +
 4 files changed, 96 insertions(+), 17 deletions(-)

diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
index 416f356af363..d22f9c8fd063 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
@@ -16,6 +16,7 @@
 #include "mtk_vcodec_intr.h"
 #include "mtk_vcodec_util.h"
 #include "venc_drv_if.h"
+#include "mtk_vcodec_enc_pm.h"
 
 #define MTK_VENC_MIN_W	160U
 #define MTK_VENC_MIN_H	128U
@@ -801,7 +802,7 @@ static int vb2ops_venc_start_streaming(struct vb2_queue *q, unsigned int count)
 	  */
 	if ((ctx->state == MTK_STATE_ABORT) || (ctx->state == MTK_STATE_FREE)) {
 		ret = -EIO;
-		goto err_start_stream;
+		goto err_set_param;
 	}
 
 	/* Do the initialization when both start_streaming have been called */
@@ -813,12 +814,6 @@ static int vb2ops_venc_start_streaming(struct vb2_queue *q, unsigned int count)
 			return 0;
 	}
 
-	ret = pm_runtime_resume_and_get(&ctx->dev->plat_dev->dev);
-	if (ret < 0) {
-		mtk_v4l2_err("pm_runtime_resume_and_get fail %d", ret);
-		goto err_start_stream;
-	}
-
 	mtk_venc_set_param(ctx, &param);
 	ret = venc_if_set_param(ctx, VENC_SET_PARAM_ENC, &param);
 	if (ret) {
@@ -845,11 +840,6 @@ static int vb2ops_venc_start_streaming(struct vb2_queue *q, unsigned int count)
 	return 0;
 
 err_set_param:
-	ret = pm_runtime_put(&ctx->dev->plat_dev->dev);
-	if (ret < 0)
-		mtk_v4l2_err("pm_runtime_put fail %d", ret);
-
-err_start_stream:
 	for (i = 0; i < q->num_buffers; ++i) {
 		struct vb2_buffer *buf = vb2_get_buffer(q, i);
 
@@ -903,10 +893,6 @@ static void vb2ops_venc_stop_streaming(struct vb2_queue *q)
 	if (ret)
 		mtk_v4l2_err("venc_if_deinit failed=%d", ret);
 
-	ret = pm_runtime_put(&ctx->dev->plat_dev->dev);
-	if (ret < 0)
-		mtk_v4l2_err("pm_runtime_put fail %d", ret);
-
 	ctx->state = MTK_STATE_FREE;
 }
 
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
index 9c2224b199d0..b78435e8223a 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
@@ -253,6 +253,12 @@ static int fops_vcodec_open(struct file *file)
 		mtk_v4l2_debug(0, "encoder capability %x", dev->enc_capability);
 	}
 
+	ret = mtk_vcodec_enc_power_on(ctx);
+	if (ret < 0) {
+		mtk_v4l2_err("encoder power on failed %d", ret);
+		goto err_load_fw;
+	}
+
 	mtk_v4l2_debug(2, "Create instance [%d]@%p m2m_ctx=%p ",
 			ctx->id, ctx, ctx->m2m_ctx);
 
@@ -285,11 +291,12 @@ static int fops_vcodec_release(struct file *file)
 	mtk_v4l2_debug(1, "[%d] encoder", ctx->id);
 	mutex_lock(&dev->dev_mutex);
 
+	v4l2_m2m_ctx_release(ctx->m2m_ctx);
 	mtk_vcodec_enc_release(ctx);
 	v4l2_fh_del(&ctx->fh);
 	v4l2_fh_exit(&ctx->fh);
 	v4l2_ctrl_handler_free(&ctx->ctrl_hdl);
-	v4l2_m2m_ctx_release(ctx->m2m_ctx);
+	mtk_vcodec_enc_power_off(ctx);
 
 	list_del_init(&ctx->list);
 	kfree(ctx);
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
index 5edeb93fca84..d9df0e3701f1 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
@@ -12,6 +12,7 @@
 
 #include "mtk_vcodec_enc_pm.h"
 #include "mtk_vcodec_util.h"
+#include "mtk_vcodec_enc_hw.h"
 
 int mtk_vcodec_init_enc_pm(struct platform_device *pdev,
 			   struct mtk_vcodec_pm *pm)
@@ -120,3 +121,84 @@ void mtk_vcodec_enc_clock_off(struct mtk_vcodec_pm *pm)
 	for (i = enc_clk->clk_num - 1; i >= 0; i--)
 		clk_disable_unprepare(enc_clk->clk_info[i].vcodec_clk);
 }
+
+int mtk_venc_enable_comp_hw(struct mtk_vcodec_dev *dev)
+{
+	int i, ret;
+	struct mtk_venc_comp_dev *venc_comp;
+
+	/*
+	 * frame_racing mode needs power on all available component devices.
+	 */
+	for (i = 0; i < MTK_VENC_HW_MAX; i++) {
+		venc_comp = (struct mtk_venc_comp_dev *)dev->enc_comp_dev[i];
+		if (!venc_comp)
+			return 0;
+
+		ret = mtk_smi_larb_get(venc_comp->pm.larbvenc);
+		if (ret < 0) {
+			mtk_v4l2_err("power on core[%d] fail %d", i, ret);
+				goto pw_err;
+		}
+	}
+	return 0;
+
+pw_err:
+	for (i -= 1; i >= 0; i--) {
+		venc_comp = (struct mtk_venc_comp_dev *)dev->enc_comp_dev[i];
+
+		mtk_smi_larb_put(venc_comp->pm.larbvenc);
+	}
+
+	return ret;
+}
+
+int mtk_venc_disable_comp_hw(struct mtk_vcodec_dev *dev)
+{
+	int i;
+	struct mtk_venc_comp_dev *venc_comp;
+
+	/*power off all available component device*/
+	for (i = 0; i < MTK_VENC_HW_MAX; i++) {
+		venc_comp = (struct mtk_venc_comp_dev *)dev->enc_comp_dev[i];
+		if (!venc_comp)
+			return 0;
+
+		mtk_smi_larb_put(venc_comp->pm.larbvenc);
+	}
+
+	return 0;
+}
+
+int mtk_vcodec_enc_power_on(struct mtk_vcodec_ctx *ctx)
+{
+	int ret;
+	struct mtk_vcodec_dev *dev = ctx->dev;
+
+	if (dev->venc_pdata->hw_mode == VENC_FRAME_RACING_MODE) {
+		ret = mtk_venc_enable_comp_hw(dev);
+		if (ret < 0) {
+			mtk_v4l2_err("enable venc comp hw fail :%d", ret);
+			return ret;
+		}
+	} else {
+		ret = mtk_smi_larb_get(dev->pm.larbvenc);
+		if (ret < 0) {
+			mtk_v4l2_err("pm_runtime_resume_and_get fail %d", ret);
+			return ret;
+		}
+	}
+	return 0;
+}
+
+int mtk_vcodec_enc_power_off(struct mtk_vcodec_ctx *ctx)
+{
+	struct mtk_vcodec_dev *dev = ctx->dev;
+
+	if (dev->venc_pdata->hw_mode == VENC_FRAME_RACING_MODE)
+		mtk_venc_disable_comp_hw(dev);
+	else
+		mtk_smi_larb_put(dev->pm.larbvenc);
+
+	return 0;
+}
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.h
index f4a4aa441185..2e76b858fed1 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.h
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.h
@@ -16,4 +16,8 @@ void mtk_vcodec_release_enc_pm(struct mtk_vcodec_pm *pm);
 void mtk_vcodec_enc_clock_on(struct mtk_vcodec_pm *pm);
 void mtk_vcodec_enc_clock_off(struct mtk_vcodec_pm *pm);
 
+int mtk_venc_enable_comp_hw(struct mtk_vcodec_dev *dev);
+int mtk_venc_disable_comp_hw(struct mtk_vcodec_dev *dev);
+int mtk_vcodec_enc_power_on(struct mtk_vcodec_ctx *ctx);
+int mtk_vcodec_enc_power_off(struct mtk_vcodec_ctx *ctx);
 #endif /* _MTK_VCODEC_ENC_PM_H_ */
-- 
2.25.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH 4/9] media: mtk-vcodec: Add venc power on/off interface

[Irui Wang]

Frame-racing mode need power on/off all venc available power,
pm_runtime_xx helper is not appropriate called directly, add new
power on/off interface for it.

Signed-off-by: Irui Wang <irui.wang@mediatek.com>
---
 .../platform/mtk-vcodec/mtk_vcodec_enc.c      | 18 +---
 .../platform/mtk-vcodec/mtk_vcodec_enc_drv.c  |  9 +-
 .../platform/mtk-vcodec/mtk_vcodec_enc_pm.c   | 82 +++++++++++++++++++
 .../platform/mtk-vcodec/mtk_vcodec_enc_pm.h   |  4 +
 4 files changed, 96 insertions(+), 17 deletions(-)

diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
index 416f356af363..d22f9c8fd063 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
@@ -16,6 +16,7 @@
 #include "mtk_vcodec_intr.h"
 #include "mtk_vcodec_util.h"
 #include "venc_drv_if.h"
+#include "mtk_vcodec_enc_pm.h"
 
 #define MTK_VENC_MIN_W	160U
 #define MTK_VENC_MIN_H	128U
@@ -801,7 +802,7 @@ static int vb2ops_venc_start_streaming(struct vb2_queue *q, unsigned int count)
 	  */
 	if ((ctx->state == MTK_STATE_ABORT) || (ctx->state == MTK_STATE_FREE)) {
 		ret = -EIO;
-		goto err_start_stream;
+		goto err_set_param;
 	}
 
 	/* Do the initialization when both start_streaming have been called */
@@ -813,12 +814,6 @@ static int vb2ops_venc_start_streaming(struct vb2_queue *q, unsigned int count)
 			return 0;
 	}
 
-	ret = pm_runtime_resume_and_get(&ctx->dev->plat_dev->dev);
-	if (ret < 0) {
-		mtk_v4l2_err("pm_runtime_resume_and_get fail %d", ret);
-		goto err_start_stream;
-	}
-
 	mtk_venc_set_param(ctx, &param);
 	ret = venc_if_set_param(ctx, VENC_SET_PARAM_ENC, &param);
 	if (ret) {
@@ -845,11 +840,6 @@ static int vb2ops_venc_start_streaming(struct vb2_queue *q, unsigned int count)
 	return 0;
 
 err_set_param:
-	ret = pm_runtime_put(&ctx->dev->plat_dev->dev);
-	if (ret < 0)
-		mtk_v4l2_err("pm_runtime_put fail %d", ret);
-
-err_start_stream:
 	for (i = 0; i < q->num_buffers; ++i) {
 		struct vb2_buffer *buf = vb2_get_buffer(q, i);
 
@@ -903,10 +893,6 @@ static void vb2ops_venc_stop_streaming(struct vb2_queue *q)
 	if (ret)
 		mtk_v4l2_err("venc_if_deinit failed=%d", ret);
 
-	ret = pm_runtime_put(&ctx->dev->plat_dev->dev);
-	if (ret < 0)
-		mtk_v4l2_err("pm_runtime_put fail %d", ret);
-
 	ctx->state = MTK_STATE_FREE;
 }
 
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
index 9c2224b199d0..b78435e8223a 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
@@ -253,6 +253,12 @@ static int fops_vcodec_open(struct file *file)
 		mtk_v4l2_debug(0, "encoder capability %x", dev->enc_capability);
 	}
 
+	ret = mtk_vcodec_enc_power_on(ctx);
+	if (ret < 0) {
+		mtk_v4l2_err("encoder power on failed %d", ret);
+		goto err_load_fw;
+	}
+
 	mtk_v4l2_debug(2, "Create instance [%d]@%p m2m_ctx=%p ",
 			ctx->id, ctx, ctx->m2m_ctx);
 
@@ -285,11 +291,12 @@ static int fops_vcodec_release(struct file *file)
 	mtk_v4l2_debug(1, "[%d] encoder", ctx->id);
 	mutex_lock(&dev->dev_mutex);
 
+	v4l2_m2m_ctx_release(ctx->m2m_ctx);
 	mtk_vcodec_enc_release(ctx);
 	v4l2_fh_del(&ctx->fh);
 	v4l2_fh_exit(&ctx->fh);
 	v4l2_ctrl_handler_free(&ctx->ctrl_hdl);
-	v4l2_m2m_ctx_release(ctx->m2m_ctx);
+	mtk_vcodec_enc_power_off(ctx);
 
 	list_del_init(&ctx->list);
 	kfree(ctx);
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
index 5edeb93fca84..d9df0e3701f1 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
@@ -12,6 +12,7 @@
 
 #include "mtk_vcodec_enc_pm.h"
 #include "mtk_vcodec_util.h"
+#include "mtk_vcodec_enc_hw.h"
 
 int mtk_vcodec_init_enc_pm(struct platform_device *pdev,
 			   struct mtk_vcodec_pm *pm)
@@ -120,3 +121,84 @@ void mtk_vcodec_enc_clock_off(struct mtk_vcodec_pm *pm)
 	for (i = enc_clk->clk_num - 1; i >= 0; i--)
 		clk_disable_unprepare(enc_clk->clk_info[i].vcodec_clk);
 }
+
+int mtk_venc_enable_comp_hw(struct mtk_vcodec_dev *dev)
+{
+	int i, ret;
+	struct mtk_venc_comp_dev *venc_comp;
+
+	/*
+	 * frame_racing mode needs power on all available component devices.
+	 */
+	for (i = 0; i < MTK_VENC_HW_MAX; i++) {
+		venc_comp = (struct mtk_venc_comp_dev *)dev->enc_comp_dev[i];
+		if (!venc_comp)
+			return 0;
+
+		ret = mtk_smi_larb_get(venc_comp->pm.larbvenc);
+		if (ret < 0) {
+			mtk_v4l2_err("power on core[%d] fail %d", i, ret);
+				goto pw_err;
+		}
+	}
+	return 0;
+
+pw_err:
+	for (i -= 1; i >= 0; i--) {
+		venc_comp = (struct mtk_venc_comp_dev *)dev->enc_comp_dev[i];
+
+		mtk_smi_larb_put(venc_comp->pm.larbvenc);
+	}
+
+	return ret;
+}
+
+int mtk_venc_disable_comp_hw(struct mtk_vcodec_dev *dev)
+{
+	int i;
+	struct mtk_venc_comp_dev *venc_comp;
+
+	/*power off all available component device*/
+	for (i = 0; i < MTK_VENC_HW_MAX; i++) {
+		venc_comp = (struct mtk_venc_comp_dev *)dev->enc_comp_dev[i];
+		if (!venc_comp)
+			return 0;
+
+		mtk_smi_larb_put(venc_comp->pm.larbvenc);
+	}
+
+	return 0;
+}
+
+int mtk_vcodec_enc_power_on(struct mtk_vcodec_ctx *ctx)
+{
+	int ret;
+	struct mtk_vcodec_dev *dev = ctx->dev;
+
+	if (dev->venc_pdata->hw_mode == VENC_FRAME_RACING_MODE) {
+		ret = mtk_venc_enable_comp_hw(dev);
+		if (ret < 0) {
+			mtk_v4l2_err("enable venc comp hw fail :%d", ret);
+			return ret;
+		}
+	} else {
+		ret = mtk_smi_larb_get(dev->pm.larbvenc);
+		if (ret < 0) {
+			mtk_v4l2_err("pm_runtime_resume_and_get fail %d", ret);
+			return ret;
+		}
+	}
+	return 0;
+}
+
+int mtk_vcodec_enc_power_off(struct mtk_vcodec_ctx *ctx)
+{
+	struct mtk_vcodec_dev *dev = ctx->dev;
+
+	if (dev->venc_pdata->hw_mode == VENC_FRAME_RACING_MODE)
+		mtk_venc_disable_comp_hw(dev);
+	else
+		mtk_smi_larb_put(dev->pm.larbvenc);
+
+	return 0;
+}
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.h
index f4a4aa441185..2e76b858fed1 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.h
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.h
@@ -16,4 +16,8 @@ void mtk_vcodec_release_enc_pm(struct mtk_vcodec_pm *pm);
 void mtk_vcodec_enc_clock_on(struct mtk_vcodec_pm *pm);
 void mtk_vcodec_enc_clock_off(struct mtk_vcodec_pm *pm);
 
+int mtk_venc_enable_comp_hw(struct mtk_vcodec_dev *dev);
+int mtk_venc_disable_comp_hw(struct mtk_vcodec_dev *dev);
+int mtk_vcodec_enc_power_on(struct mtk_vcodec_ctx *ctx);
+int mtk_vcodec_enc_power_off(struct mtk_vcodec_ctx *ctx);
 #endif /* _MTK_VCODEC_ENC_PM_H_ */
-- 
2.25.1
_______________________________________________
Linux-mediatek mailing list
Linux-mediatek@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-mediatek

Re: [PATCH 4/9] media: mtk-vcodec: Add venc power on/off interface

[Tzung-Bi Shih]

On Mon, Aug 16, 2021 at 06:59:29PM +0800, Irui Wang wrote:
> --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
> +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
> @@ -16,6 +16,7 @@
>  #include "mtk_vcodec_intr.h"
>  #include "mtk_vcodec_util.h"
>  #include "venc_drv_if.h"
> +#include "mtk_vcodec_enc_pm.h"
Please try to maintain the order.

> @@ -285,11 +291,12 @@ static int fops_vcodec_release(struct file *file)
>  	mtk_v4l2_debug(1, "[%d] encoder", ctx->id);
>  	mutex_lock(&dev->dev_mutex);
>  
> +	v4l2_m2m_ctx_release(ctx->m2m_ctx);
>  	mtk_vcodec_enc_release(ctx);
>  	v4l2_fh_del(&ctx->fh);
>  	v4l2_fh_exit(&ctx->fh);
>  	v4l2_ctrl_handler_free(&ctx->ctrl_hdl);
> -	v4l2_m2m_ctx_release(ctx->m2m_ctx);
> +	mtk_vcodec_enc_power_off(ctx);
Any reason to move the v4l2_m2m_ctx_release()?

> --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
> +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
> @@ -12,6 +12,7 @@
>  
>  #include "mtk_vcodec_enc_pm.h"
>  #include "mtk_vcodec_util.h"
> +#include "mtk_vcodec_enc_hw.h"
Please try to maintain the order.

> +int mtk_venc_enable_comp_hw(struct mtk_vcodec_dev *dev)
> +{
> +	int i, ret;
> +	struct mtk_venc_comp_dev *venc_comp;
> +
> +	/*
> +	 * frame_racing mode needs power on all available component devices.
> +	 */
> +	for (i = 0; i < MTK_VENC_HW_MAX; i++) {
> +		venc_comp = (struct mtk_venc_comp_dev *)dev->enc_comp_dev[i];
> +		if (!venc_comp)
> +			return 0;
> +
> +		ret = mtk_smi_larb_get(venc_comp->pm.larbvenc);
> +		if (ret < 0) {
> +			mtk_v4l2_err("power on core[%d] fail %d", i, ret);
> +				goto pw_err;
The goto statement has extra indent layer.

> +int mtk_venc_disable_comp_hw(struct mtk_vcodec_dev *dev)
> +{
> +	int i;
> +	struct mtk_venc_comp_dev *venc_comp;
> +
> +	/*power off all available component device*/
Need extra space between /* */.

> +int mtk_vcodec_enc_power_on(struct mtk_vcodec_ctx *ctx)
> +{
> +	int ret;
> +	struct mtk_vcodec_dev *dev = ctx->dev;
> +
> +	if (dev->venc_pdata->hw_mode == VENC_FRAME_RACING_MODE) {
> +		ret = mtk_venc_enable_comp_hw(dev);
> +		if (ret < 0) {
> +			mtk_v4l2_err("enable venc comp hw fail :%d", ret);
> +			return ret;
> +		}
> +	} else {
> +		ret = mtk_smi_larb_get(dev->pm.larbvenc);
> +		if (ret < 0) {
> +			mtk_v4l2_err("pm_runtime_resume_and_get fail %d", ret);
> +			return ret;
> +		}
> +	}
> +	return 0;
> +}
It seems the function doesn't need struct mtk_vcodec_ctx.

Does mtk_vcodec_enc_power_on(struct mtk_vcodec_dev* ) make more sense?

> +int mtk_vcodec_enc_power_off(struct mtk_vcodec_ctx *ctx)
> +{
> +	struct mtk_vcodec_dev *dev = ctx->dev;
> +
> +	if (dev->venc_pdata->hw_mode == VENC_FRAME_RACING_MODE)
> +		mtk_venc_disable_comp_hw(dev);
> +	else
> +		mtk_smi_larb_put(dev->pm.larbvenc);
> +
> +	return 0;
> +}
It seems the function doesn't need struct mtk_vcodec_ctx.

Re: [PATCH 4/9] media: mtk-vcodec: Add venc power on/off interface

[Tzung-Bi Shih]

On Mon, Aug 16, 2021 at 06:59:29PM +0800, Irui Wang wrote:
> --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
> +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
> @@ -16,6 +16,7 @@
>  #include "mtk_vcodec_intr.h"
>  #include "mtk_vcodec_util.h"
>  #include "venc_drv_if.h"
> +#include "mtk_vcodec_enc_pm.h"
Please try to maintain the order.

> @@ -285,11 +291,12 @@ static int fops_vcodec_release(struct file *file)
>  	mtk_v4l2_debug(1, "[%d] encoder", ctx->id);
>  	mutex_lock(&dev->dev_mutex);
>  
> +	v4l2_m2m_ctx_release(ctx->m2m_ctx);
>  	mtk_vcodec_enc_release(ctx);
>  	v4l2_fh_del(&ctx->fh);
>  	v4l2_fh_exit(&ctx->fh);
>  	v4l2_ctrl_handler_free(&ctx->ctrl_hdl);
> -	v4l2_m2m_ctx_release(ctx->m2m_ctx);
> +	mtk_vcodec_enc_power_off(ctx);
Any reason to move the v4l2_m2m_ctx_release()?

> --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
> +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
> @@ -12,6 +12,7 @@
>  
>  #include "mtk_vcodec_enc_pm.h"
>  #include "mtk_vcodec_util.h"
> +#include "mtk_vcodec_enc_hw.h"
Please try to maintain the order.

> +int mtk_venc_enable_comp_hw(struct mtk_vcodec_dev *dev)
> +{
> +	int i, ret;
> +	struct mtk_venc_comp_dev *venc_comp;
> +
> +	/*
> +	 * frame_racing mode needs power on all available component devices.
> +	 */
> +	for (i = 0; i < MTK_VENC_HW_MAX; i++) {
> +		venc_comp = (struct mtk_venc_comp_dev *)dev->enc_comp_dev[i];
> +		if (!venc_comp)
> +			return 0;
> +
> +		ret = mtk_smi_larb_get(venc_comp->pm.larbvenc);
> +		if (ret < 0) {
> +			mtk_v4l2_err("power on core[%d] fail %d", i, ret);
> +				goto pw_err;
The goto statement has extra indent layer.

> +int mtk_venc_disable_comp_hw(struct mtk_vcodec_dev *dev)
> +{
> +	int i;
> +	struct mtk_venc_comp_dev *venc_comp;
> +
> +	/*power off all available component device*/
Need extra space between /* */.

> +int mtk_vcodec_enc_power_on(struct mtk_vcodec_ctx *ctx)
> +{
> +	int ret;
> +	struct mtk_vcodec_dev *dev = ctx->dev;
> +
> +	if (dev->venc_pdata->hw_mode == VENC_FRAME_RACING_MODE) {
> +		ret = mtk_venc_enable_comp_hw(dev);
> +		if (ret < 0) {
> +			mtk_v4l2_err("enable venc comp hw fail :%d", ret);
> +			return ret;
> +		}
> +	} else {
> +		ret = mtk_smi_larb_get(dev->pm.larbvenc);
> +		if (ret < 0) {
> +			mtk_v4l2_err("pm_runtime_resume_and_get fail %d", ret);
> +			return ret;
> +		}
> +	}
> +	return 0;
> +}
It seems the function doesn't need struct mtk_vcodec_ctx.

Does mtk_vcodec_enc_power_on(struct mtk_vcodec_dev* ) make more sense?

> +int mtk_vcodec_enc_power_off(struct mtk_vcodec_ctx *ctx)
> +{
> +	struct mtk_vcodec_dev *dev = ctx->dev;
> +
> +	if (dev->venc_pdata->hw_mode == VENC_FRAME_RACING_MODE)
> +		mtk_venc_disable_comp_hw(dev);
> +	else
> +		mtk_smi_larb_put(dev->pm.larbvenc);
> +
> +	return 0;
> +}
It seems the function doesn't need struct mtk_vcodec_ctx.

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH 4/9] media: mtk-vcodec: Add venc power on/off interface

[Tzung-Bi Shih]

On Mon, Aug 16, 2021 at 06:59:29PM +0800, Irui Wang wrote:
> --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
> +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
> @@ -16,6 +16,7 @@
>  #include "mtk_vcodec_intr.h"
>  #include "mtk_vcodec_util.h"
>  #include "venc_drv_if.h"
> +#include "mtk_vcodec_enc_pm.h"
Please try to maintain the order.

> @@ -285,11 +291,12 @@ static int fops_vcodec_release(struct file *file)
>  	mtk_v4l2_debug(1, "[%d] encoder", ctx->id);
>  	mutex_lock(&dev->dev_mutex);
>  
> +	v4l2_m2m_ctx_release(ctx->m2m_ctx);
>  	mtk_vcodec_enc_release(ctx);
>  	v4l2_fh_del(&ctx->fh);
>  	v4l2_fh_exit(&ctx->fh);
>  	v4l2_ctrl_handler_free(&ctx->ctrl_hdl);
> -	v4l2_m2m_ctx_release(ctx->m2m_ctx);
> +	mtk_vcodec_enc_power_off(ctx);
Any reason to move the v4l2_m2m_ctx_release()?

> --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
> +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
> @@ -12,6 +12,7 @@
>  
>  #include "mtk_vcodec_enc_pm.h"
>  #include "mtk_vcodec_util.h"
> +#include "mtk_vcodec_enc_hw.h"
Please try to maintain the order.

> +int mtk_venc_enable_comp_hw(struct mtk_vcodec_dev *dev)
> +{
> +	int i, ret;
> +	struct mtk_venc_comp_dev *venc_comp;
> +
> +	/*
> +	 * frame_racing mode needs power on all available component devices.
> +	 */
> +	for (i = 0; i < MTK_VENC_HW_MAX; i++) {
> +		venc_comp = (struct mtk_venc_comp_dev *)dev->enc_comp_dev[i];
> +		if (!venc_comp)
> +			return 0;
> +
> +		ret = mtk_smi_larb_get(venc_comp->pm.larbvenc);
> +		if (ret < 0) {
> +			mtk_v4l2_err("power on core[%d] fail %d", i, ret);
> +				goto pw_err;
The goto statement has extra indent layer.

> +int mtk_venc_disable_comp_hw(struct mtk_vcodec_dev *dev)
> +{
> +	int i;
> +	struct mtk_venc_comp_dev *venc_comp;
> +
> +	/*power off all available component device*/
Need extra space between /* */.

> +int mtk_vcodec_enc_power_on(struct mtk_vcodec_ctx *ctx)
> +{
> +	int ret;
> +	struct mtk_vcodec_dev *dev = ctx->dev;
> +
> +	if (dev->venc_pdata->hw_mode == VENC_FRAME_RACING_MODE) {
> +		ret = mtk_venc_enable_comp_hw(dev);
> +		if (ret < 0) {
> +			mtk_v4l2_err("enable venc comp hw fail :%d", ret);
> +			return ret;
> +		}
> +	} else {
> +		ret = mtk_smi_larb_get(dev->pm.larbvenc);
> +		if (ret < 0) {
> +			mtk_v4l2_err("pm_runtime_resume_and_get fail %d", ret);
> +			return ret;
> +		}
> +	}
> +	return 0;
> +}
It seems the function doesn't need struct mtk_vcodec_ctx.

Does mtk_vcodec_enc_power_on(struct mtk_vcodec_dev* ) make more sense?

> +int mtk_vcodec_enc_power_off(struct mtk_vcodec_ctx *ctx)
> +{
> +	struct mtk_vcodec_dev *dev = ctx->dev;
> +
> +	if (dev->venc_pdata->hw_mode == VENC_FRAME_RACING_MODE)
> +		mtk_venc_disable_comp_hw(dev);
> +	else
> +		mtk_smi_larb_put(dev->pm.larbvenc);
> +
> +	return 0;
> +}
It seems the function doesn't need struct mtk_vcodec_ctx.

_______________________________________________
Linux-mediatek mailing list
Linux-mediatek@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-mediatek

[PATCH 5/9] media: mtk-vcodec: Rewrite venc clock interface

[Irui Wang]

Frame-racing mode need enable the specific core's clk, add an
another param for clock_on/clock_off interface.

Signed-off-by: Irui Wang <irui.wang@mediatek.com>
---
 .../platform/mtk-vcodec/mtk_vcodec_enc_pm.c   | 104 ++++++++++++++----
 .../platform/mtk-vcodec/mtk_vcodec_enc_pm.h   |   4 +-
 .../media/platform/mtk-vcodec/venc_drv_if.c   |  10 +-
 3 files changed, 86 insertions(+), 32 deletions(-)

diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
index d9df0e3701f1..bfb9e21bfd26 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
@@ -86,25 +86,50 @@ void mtk_vcodec_release_enc_pm(struct mtk_vcodec_pm *pm)
 }
 
 
-void mtk_vcodec_enc_clock_on(struct mtk_vcodec_pm *pm)
+void mtk_vcodec_enc_clock_on(struct mtk_vcodec_dev *dev, int core_id)
 {
-	struct mtk_vcodec_clk *enc_clk = &pm->venc_clk;
-	int ret, i = 0;
+	struct mtk_venc_comp_dev *venc;
+	struct mtk_vcodec_pm *enc_pm;
+	struct mtk_vcodec_clk *enc_clk;
+	struct clk		*clk;
+	int ret, i;
 
-	for (i = 0; i < enc_clk->clk_num; i++) {
-		ret = clk_prepare_enable(enc_clk->clk_info[i].vcodec_clk);
-		if (ret) {
-			mtk_v4l2_err("venc clk_prepare_enable %d %s fail %d", i,
-				enc_clk->clk_info[i].clk_name, ret);
-			goto clkerr;
+	if (dev->venc_pdata->hw_mode == VENC_FRAME_RACING_MODE) {
+		venc = (struct mtk_venc_comp_dev *)dev->enc_comp_dev[core_id];
+		enc_pm = &venc->pm;
+		enc_clk = &enc_pm->venc_clk;
+
+		for (i = 0; i < enc_clk->clk_num; i++) {
+			clk = enc_clk->clk_info[i].vcodec_clk;
+			ret = clk_enable(clk);
+			if (ret) {
+				mtk_v4l2_err("clk_enable %d %s fail %d", i,
+					     enc_clk->clk_info[i].clk_name,
+					     ret);
+				goto comp_clk_err;
+			}
 		}
-	}
+	} else {
+		enc_pm = &dev->pm;
+		enc_clk = &enc_pm->venc_clk;
 
-	ret = mtk_smi_larb_get(pm->larbvenc);
-	if (ret) {
-		mtk_v4l2_err("mtk_smi_larb_get larb3 fail %d", ret);
-		goto clkerr;
+		for (i = 0; i < enc_clk->clk_num; i++) {
+			clk = enc_clk->clk_info[i].vcodec_clk;
+			ret = clk_prepare_enable(clk);
+			if (ret) {
+				mtk_v4l2_err("clk_prepare_enable %d %s fail %d",
+					     i, enc_clk->clk_info[i].clk_name,
+					     ret);
+				goto clkerr;
+			}
+		}
 	}
+
+	return;
+
+comp_clk_err:
+	for (i -= 1; i >= 0; i--)
+		clk_disable(enc_clk->clk_info[i].vcodec_clk);
 	return;
 
 clkerr:
@@ -112,20 +137,35 @@ void mtk_vcodec_enc_clock_on(struct mtk_vcodec_pm *pm)
 		clk_disable_unprepare(enc_clk->clk_info[i].vcodec_clk);
 }
 
-void mtk_vcodec_enc_clock_off(struct mtk_vcodec_pm *pm)
+void mtk_vcodec_enc_clock_off(struct mtk_vcodec_dev *dev, int core_id)
 {
-	struct mtk_vcodec_clk *enc_clk = &pm->venc_clk;
-	int i = 0;
+	struct mtk_venc_comp_dev *venc;
+	struct mtk_vcodec_pm *enc_pm;
+	struct mtk_vcodec_clk *enc_clk;
+	int i;
 
-	mtk_smi_larb_put(pm->larbvenc);
-	for (i = enc_clk->clk_num - 1; i >= 0; i--)
-		clk_disable_unprepare(enc_clk->clk_info[i].vcodec_clk);
+	if (dev->venc_pdata->hw_mode == VENC_FRAME_RACING_MODE) {
+		venc = (struct mtk_venc_comp_dev *)dev->enc_comp_dev[core_id];
+		enc_pm = &venc->pm;
+		enc_clk = &enc_pm->venc_clk;
+
+		for (i = enc_clk->clk_num - 1; i >= 0; i--)
+			clk_disable(enc_clk->clk_info[i].vcodec_clk);
+	} else {
+		enc_pm = &dev->pm;
+		enc_clk = &enc_pm->venc_clk;
+
+		for (i = enc_clk->clk_num - 1; i >= 0; i--)
+			clk_disable_unprepare(enc_clk->clk_info[i].vcodec_clk);
+	}
 }
 
 int mtk_venc_enable_comp_hw(struct mtk_vcodec_dev *dev)
 {
 	int i, ret;
 	struct mtk_venc_comp_dev *venc_comp;
+	struct mtk_vcodec_clk *enc_clk;
+	int j = 0;
 
 	/*
 	 * frame_racing mode needs power on all available component devices.
@@ -140,23 +180,38 @@ int mtk_venc_enable_comp_hw(struct mtk_vcodec_dev *dev)
 			mtk_v4l2_err("power on core[%d] fail %d", i, ret);
 				goto pw_err;
 		}
+
+		enc_clk = &venc_comp->pm.venc_clk;
+		for (j = 0; j < enc_clk->clk_num; j++) {
+			ret = clk_prepare(enc_clk->clk_info[j].vcodec_clk);
+			if (ret) {
+				mtk_v4l2_err("prepare clk [%s] fail %d",
+					     enc_clk->clk_info[j].clk_name,
+					     ret);
+				goto pw_err;
+			}
+		}
 	}
 	return 0;
 
 pw_err:
 	for (i -= 1; i >= 0; i--) {
 		venc_comp = (struct mtk_venc_comp_dev *)dev->enc_comp_dev[i];
+		enc_clk = &venc_comp->pm.venc_clk;
+
+		for (j -= 1; j >= 0; j--)
+			clk_unprepare(enc_clk->clk_info[j].vcodec_clk);
 
 		mtk_smi_larb_put(venc_comp->pm.larbvenc);
 	}
-
 	return ret;
 }
 
 int mtk_venc_disable_comp_hw(struct mtk_vcodec_dev *dev)
 {
-	int i;
+	int i, j;
 	struct mtk_venc_comp_dev *venc_comp;
+	struct mtk_vcodec_clk *enc_clk;
 
 	/*power off all available component device*/
 	for (i = 0; i < MTK_VENC_HW_MAX; i++) {
@@ -164,6 +219,11 @@ int mtk_venc_disable_comp_hw(struct mtk_vcodec_dev *dev)
 		if (!venc_comp)
 			return 0;
 
+		enc_clk = &venc_comp->pm.venc_clk;
+
+		for (j = enc_clk->clk_num - 1; j >= 0; j--)
+			clk_unprepare(enc_clk->clk_info[j].vcodec_clk);
+
 		mtk_smi_larb_put(venc_comp->pm.larbvenc);
 	}
 
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.h
index 2e76b858fed1..f3a819cc0e56 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.h
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.h
@@ -13,8 +13,8 @@ int mtk_vcodec_init_enc_pm(struct platform_device *pdev,
 			   struct mtk_vcodec_pm *pm);
 void mtk_vcodec_release_enc_pm(struct mtk_vcodec_pm *pm);
 
-void mtk_vcodec_enc_clock_on(struct mtk_vcodec_pm *pm);
-void mtk_vcodec_enc_clock_off(struct mtk_vcodec_pm *pm);
+void mtk_vcodec_enc_clock_on(struct mtk_vcodec_dev *dev, int core_id);
+void mtk_vcodec_enc_clock_off(struct mtk_vcodec_dev *dev, int core_id);
 
 int mtk_venc_enable_comp_hw(struct mtk_vcodec_dev *dev);
 int mtk_venc_disable_comp_hw(struct mtk_vcodec_dev *dev);
diff --git a/drivers/media/platform/mtk-vcodec/venc_drv_if.c b/drivers/media/platform/mtk-vcodec/venc_drv_if.c
index ce0bce811615..6cbdb7e30bb3 100644
--- a/drivers/media/platform/mtk-vcodec/venc_drv_if.c
+++ b/drivers/media/platform/mtk-vcodec/venc_drv_if.c
@@ -32,9 +32,7 @@ int venc_if_init(struct mtk_vcodec_ctx *ctx, unsigned int fourcc)
 	}
 
 	mtk_venc_lock(ctx);
-	mtk_vcodec_enc_clock_on(&ctx->dev->pm);
 	ret = ctx->enc_if->init(ctx);
-	mtk_vcodec_enc_clock_off(&ctx->dev->pm);
 	mtk_venc_unlock(ctx);
 
 	return ret;
@@ -46,9 +44,7 @@ int venc_if_set_param(struct mtk_vcodec_ctx *ctx,
 	int ret = 0;
 
 	mtk_venc_lock(ctx);
-	mtk_vcodec_enc_clock_on(&ctx->dev->pm);
 	ret = ctx->enc_if->set_param(ctx->drv_handle, type, in);
-	mtk_vcodec_enc_clock_off(&ctx->dev->pm);
 	mtk_venc_unlock(ctx);
 
 	return ret;
@@ -68,10 +64,10 @@ int venc_if_encode(struct mtk_vcodec_ctx *ctx,
 	ctx->dev->curr_ctx = ctx;
 	spin_unlock_irqrestore(&ctx->dev->irqlock, flags);
 
-	mtk_vcodec_enc_clock_on(&ctx->dev->pm);
+	mtk_vcodec_enc_clock_on(ctx->dev, 0);
 	ret = ctx->enc_if->encode(ctx->drv_handle, opt, frm_buf,
 				  bs_buf, result);
-	mtk_vcodec_enc_clock_off(&ctx->dev->pm);
+	mtk_vcodec_enc_clock_off(ctx->dev, 0);
 
 	spin_lock_irqsave(&ctx->dev->irqlock, flags);
 	ctx->dev->curr_ctx = NULL;
@@ -89,9 +85,7 @@ int venc_if_deinit(struct mtk_vcodec_ctx *ctx)
 		return 0;
 
 	mtk_venc_lock(ctx);
-	mtk_vcodec_enc_clock_on(&ctx->dev->pm);
 	ret = ctx->enc_if->deinit(ctx->drv_handle);
-	mtk_vcodec_enc_clock_off(&ctx->dev->pm);
 	mtk_venc_unlock(ctx);
 
 	ctx->drv_handle = NULL;
-- 
2.25.1

[PATCH 5/9] media: mtk-vcodec: Rewrite venc clock interface

[Irui Wang]

Frame-racing mode need enable the specific core's clk, add an
another param for clock_on/clock_off interface.

Signed-off-by: Irui Wang <irui.wang@mediatek.com>
---
 .../platform/mtk-vcodec/mtk_vcodec_enc_pm.c   | 104 ++++++++++++++----
 .../platform/mtk-vcodec/mtk_vcodec_enc_pm.h   |   4 +-
 .../media/platform/mtk-vcodec/venc_drv_if.c   |  10 +-
 3 files changed, 86 insertions(+), 32 deletions(-)

diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
index d9df0e3701f1..bfb9e21bfd26 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
@@ -86,25 +86,50 @@ void mtk_vcodec_release_enc_pm(struct mtk_vcodec_pm *pm)
 }
 
 
-void mtk_vcodec_enc_clock_on(struct mtk_vcodec_pm *pm)
+void mtk_vcodec_enc_clock_on(struct mtk_vcodec_dev *dev, int core_id)
 {
-	struct mtk_vcodec_clk *enc_clk = &pm->venc_clk;
-	int ret, i = 0;
+	struct mtk_venc_comp_dev *venc;
+	struct mtk_vcodec_pm *enc_pm;
+	struct mtk_vcodec_clk *enc_clk;
+	struct clk		*clk;
+	int ret, i;
 
-	for (i = 0; i < enc_clk->clk_num; i++) {
-		ret = clk_prepare_enable(enc_clk->clk_info[i].vcodec_clk);
-		if (ret) {
-			mtk_v4l2_err("venc clk_prepare_enable %d %s fail %d", i,
-				enc_clk->clk_info[i].clk_name, ret);
-			goto clkerr;
+	if (dev->venc_pdata->hw_mode == VENC_FRAME_RACING_MODE) {
+		venc = (struct mtk_venc_comp_dev *)dev->enc_comp_dev[core_id];
+		enc_pm = &venc->pm;
+		enc_clk = &enc_pm->venc_clk;
+
+		for (i = 0; i < enc_clk->clk_num; i++) {
+			clk = enc_clk->clk_info[i].vcodec_clk;
+			ret = clk_enable(clk);
+			if (ret) {
+				mtk_v4l2_err("clk_enable %d %s fail %d", i,
+					     enc_clk->clk_info[i].clk_name,
+					     ret);
+				goto comp_clk_err;
+			}
 		}
-	}
+	} else {
+		enc_pm = &dev->pm;
+		enc_clk = &enc_pm->venc_clk;
 
-	ret = mtk_smi_larb_get(pm->larbvenc);
-	if (ret) {
-		mtk_v4l2_err("mtk_smi_larb_get larb3 fail %d", ret);
-		goto clkerr;
+		for (i = 0; i < enc_clk->clk_num; i++) {
+			clk = enc_clk->clk_info[i].vcodec_clk;
+			ret = clk_prepare_enable(clk);
+			if (ret) {
+				mtk_v4l2_err("clk_prepare_enable %d %s fail %d",
+					     i, enc_clk->clk_info[i].clk_name,
+					     ret);
+				goto clkerr;
+			}
+		}
 	}
+
+	return;
+
+comp_clk_err:
+	for (i -= 1; i >= 0; i--)
+		clk_disable(enc_clk->clk_info[i].vcodec_clk);
 	return;
 
 clkerr:
@@ -112,20 +137,35 @@ void mtk_vcodec_enc_clock_on(struct mtk_vcodec_pm *pm)
 		clk_disable_unprepare(enc_clk->clk_info[i].vcodec_clk);
 }
 
-void mtk_vcodec_enc_clock_off(struct mtk_vcodec_pm *pm)
+void mtk_vcodec_enc_clock_off(struct mtk_vcodec_dev *dev, int core_id)
 {
-	struct mtk_vcodec_clk *enc_clk = &pm->venc_clk;
-	int i = 0;
+	struct mtk_venc_comp_dev *venc;
+	struct mtk_vcodec_pm *enc_pm;
+	struct mtk_vcodec_clk *enc_clk;
+	int i;
 
-	mtk_smi_larb_put(pm->larbvenc);
-	for (i = enc_clk->clk_num - 1; i >= 0; i--)
-		clk_disable_unprepare(enc_clk->clk_info[i].vcodec_clk);
+	if (dev->venc_pdata->hw_mode == VENC_FRAME_RACING_MODE) {
+		venc = (struct mtk_venc_comp_dev *)dev->enc_comp_dev[core_id];
+		enc_pm = &venc->pm;
+		enc_clk = &enc_pm->venc_clk;
+
+		for (i = enc_clk->clk_num - 1; i >= 0; i--)
+			clk_disable(enc_clk->clk_info[i].vcodec_clk);
+	} else {
+		enc_pm = &dev->pm;
+		enc_clk = &enc_pm->venc_clk;
+
+		for (i = enc_clk->clk_num - 1; i >= 0; i--)
+			clk_disable_unprepare(enc_clk->clk_info[i].vcodec_clk);
+	}
 }
 
 int mtk_venc_enable_comp_hw(struct mtk_vcodec_dev *dev)
 {
 	int i, ret;
 	struct mtk_venc_comp_dev *venc_comp;
+	struct mtk_vcodec_clk *enc_clk;
+	int j = 0;
 
 	/*
 	 * frame_racing mode needs power on all available component devices.
@@ -140,23 +180,38 @@ int mtk_venc_enable_comp_hw(struct mtk_vcodec_dev *dev)
 			mtk_v4l2_err("power on core[%d] fail %d", i, ret);
 				goto pw_err;
 		}
+
+		enc_clk = &venc_comp->pm.venc_clk;
+		for (j = 0; j < enc_clk->clk_num; j++) {
+			ret = clk_prepare(enc_clk->clk_info[j].vcodec_clk);
+			if (ret) {
+				mtk_v4l2_err("prepare clk [%s] fail %d",
+					     enc_clk->clk_info[j].clk_name,
+					     ret);
+				goto pw_err;
+			}
+		}
 	}
 	return 0;
 
 pw_err:
 	for (i -= 1; i >= 0; i--) {
 		venc_comp = (struct mtk_venc_comp_dev *)dev->enc_comp_dev[i];
+		enc_clk = &venc_comp->pm.venc_clk;
+
+		for (j -= 1; j >= 0; j--)
+			clk_unprepare(enc_clk->clk_info[j].vcodec_clk);
 
 		mtk_smi_larb_put(venc_comp->pm.larbvenc);
 	}
-
 	return ret;
 }
 
 int mtk_venc_disable_comp_hw(struct mtk_vcodec_dev *dev)
 {
-	int i;
+	int i, j;
 	struct mtk_venc_comp_dev *venc_comp;
+	struct mtk_vcodec_clk *enc_clk;
 
 	/*power off all available component device*/
 	for (i = 0; i < MTK_VENC_HW_MAX; i++) {
@@ -164,6 +219,11 @@ int mtk_venc_disable_comp_hw(struct mtk_vcodec_dev *dev)
 		if (!venc_comp)
 			return 0;
 
+		enc_clk = &venc_comp->pm.venc_clk;
+
+		for (j = enc_clk->clk_num - 1; j >= 0; j--)
+			clk_unprepare(enc_clk->clk_info[j].vcodec_clk);
+
 		mtk_smi_larb_put(venc_comp->pm.larbvenc);
 	}
 
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.h
index 2e76b858fed1..f3a819cc0e56 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.h
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.h
@@ -13,8 +13,8 @@ int mtk_vcodec_init_enc_pm(struct platform_device *pdev,
 			   struct mtk_vcodec_pm *pm);
 void mtk_vcodec_release_enc_pm(struct mtk_vcodec_pm *pm);
 
-void mtk_vcodec_enc_clock_on(struct mtk_vcodec_pm *pm);
-void mtk_vcodec_enc_clock_off(struct mtk_vcodec_pm *pm);
+void mtk_vcodec_enc_clock_on(struct mtk_vcodec_dev *dev, int core_id);
+void mtk_vcodec_enc_clock_off(struct mtk_vcodec_dev *dev, int core_id);
 
 int mtk_venc_enable_comp_hw(struct mtk_vcodec_dev *dev);
 int mtk_venc_disable_comp_hw(struct mtk_vcodec_dev *dev);
diff --git a/drivers/media/platform/mtk-vcodec/venc_drv_if.c b/drivers/media/platform/mtk-vcodec/venc_drv_if.c
index ce0bce811615..6cbdb7e30bb3 100644
--- a/drivers/media/platform/mtk-vcodec/venc_drv_if.c
+++ b/drivers/media/platform/mtk-vcodec/venc_drv_if.c
@@ -32,9 +32,7 @@ int venc_if_init(struct mtk_vcodec_ctx *ctx, unsigned int fourcc)
 	}
 
 	mtk_venc_lock(ctx);
-	mtk_vcodec_enc_clock_on(&ctx->dev->pm);
 	ret = ctx->enc_if->init(ctx);
-	mtk_vcodec_enc_clock_off(&ctx->dev->pm);
 	mtk_venc_unlock(ctx);
 
 	return ret;
@@ -46,9 +44,7 @@ int venc_if_set_param(struct mtk_vcodec_ctx *ctx,
 	int ret = 0;
 
 	mtk_venc_lock(ctx);
-	mtk_vcodec_enc_clock_on(&ctx->dev->pm);
 	ret = ctx->enc_if->set_param(ctx->drv_handle, type, in);
-	mtk_vcodec_enc_clock_off(&ctx->dev->pm);
 	mtk_venc_unlock(ctx);
 
 	return ret;
@@ -68,10 +64,10 @@ int venc_if_encode(struct mtk_vcodec_ctx *ctx,
 	ctx->dev->curr_ctx = ctx;
 	spin_unlock_irqrestore(&ctx->dev->irqlock, flags);
 
-	mtk_vcodec_enc_clock_on(&ctx->dev->pm);
+	mtk_vcodec_enc_clock_on(ctx->dev, 0);
 	ret = ctx->enc_if->encode(ctx->drv_handle, opt, frm_buf,
 				  bs_buf, result);
-	mtk_vcodec_enc_clock_off(&ctx->dev->pm);
+	mtk_vcodec_enc_clock_off(ctx->dev, 0);
 
 	spin_lock_irqsave(&ctx->dev->irqlock, flags);
 	ctx->dev->curr_ctx = NULL;
@@ -89,9 +85,7 @@ int venc_if_deinit(struct mtk_vcodec_ctx *ctx)
 		return 0;
 
 	mtk_venc_lock(ctx);
-	mtk_vcodec_enc_clock_on(&ctx->dev->pm);
 	ret = ctx->enc_if->deinit(ctx->drv_handle);
-	mtk_vcodec_enc_clock_off(&ctx->dev->pm);
 	mtk_venc_unlock(ctx);
 
 	ctx->drv_handle = NULL;
-- 
2.25.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH 5/9] media: mtk-vcodec: Rewrite venc clock interface

[Irui Wang]

Frame-racing mode need enable the specific core's clk, add an
another param for clock_on/clock_off interface.

Signed-off-by: Irui Wang <irui.wang@mediatek.com>
---
 .../platform/mtk-vcodec/mtk_vcodec_enc_pm.c   | 104 ++++++++++++++----
 .../platform/mtk-vcodec/mtk_vcodec_enc_pm.h   |   4 +-
 .../media/platform/mtk-vcodec/venc_drv_if.c   |  10 +-
 3 files changed, 86 insertions(+), 32 deletions(-)

diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
index d9df0e3701f1..bfb9e21bfd26 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
@@ -86,25 +86,50 @@ void mtk_vcodec_release_enc_pm(struct mtk_vcodec_pm *pm)
 }
 
 
-void mtk_vcodec_enc_clock_on(struct mtk_vcodec_pm *pm)
+void mtk_vcodec_enc_clock_on(struct mtk_vcodec_dev *dev, int core_id)
 {
-	struct mtk_vcodec_clk *enc_clk = &pm->venc_clk;
-	int ret, i = 0;
+	struct mtk_venc_comp_dev *venc;
+	struct mtk_vcodec_pm *enc_pm;
+	struct mtk_vcodec_clk *enc_clk;
+	struct clk		*clk;
+	int ret, i;
 
-	for (i = 0; i < enc_clk->clk_num; i++) {
-		ret = clk_prepare_enable(enc_clk->clk_info[i].vcodec_clk);
-		if (ret) {
-			mtk_v4l2_err("venc clk_prepare_enable %d %s fail %d", i,
-				enc_clk->clk_info[i].clk_name, ret);
-			goto clkerr;
+	if (dev->venc_pdata->hw_mode == VENC_FRAME_RACING_MODE) {
+		venc = (struct mtk_venc_comp_dev *)dev->enc_comp_dev[core_id];
+		enc_pm = &venc->pm;
+		enc_clk = &enc_pm->venc_clk;
+
+		for (i = 0; i < enc_clk->clk_num; i++) {
+			clk = enc_clk->clk_info[i].vcodec_clk;
+			ret = clk_enable(clk);
+			if (ret) {
+				mtk_v4l2_err("clk_enable %d %s fail %d", i,
+					     enc_clk->clk_info[i].clk_name,
+					     ret);
+				goto comp_clk_err;
+			}
 		}
-	}
+	} else {
+		enc_pm = &dev->pm;
+		enc_clk = &enc_pm->venc_clk;
 
-	ret = mtk_smi_larb_get(pm->larbvenc);
-	if (ret) {
-		mtk_v4l2_err("mtk_smi_larb_get larb3 fail %d", ret);
-		goto clkerr;
+		for (i = 0; i < enc_clk->clk_num; i++) {
+			clk = enc_clk->clk_info[i].vcodec_clk;
+			ret = clk_prepare_enable(clk);
+			if (ret) {
+				mtk_v4l2_err("clk_prepare_enable %d %s fail %d",
+					     i, enc_clk->clk_info[i].clk_name,
+					     ret);
+				goto clkerr;
+			}
+		}
 	}
+
+	return;
+
+comp_clk_err:
+	for (i -= 1; i >= 0; i--)
+		clk_disable(enc_clk->clk_info[i].vcodec_clk);
 	return;
 
 clkerr:
@@ -112,20 +137,35 @@ void mtk_vcodec_enc_clock_on(struct mtk_vcodec_pm *pm)
 		clk_disable_unprepare(enc_clk->clk_info[i].vcodec_clk);
 }
 
-void mtk_vcodec_enc_clock_off(struct mtk_vcodec_pm *pm)
+void mtk_vcodec_enc_clock_off(struct mtk_vcodec_dev *dev, int core_id)
 {
-	struct mtk_vcodec_clk *enc_clk = &pm->venc_clk;
-	int i = 0;
+	struct mtk_venc_comp_dev *venc;
+	struct mtk_vcodec_pm *enc_pm;
+	struct mtk_vcodec_clk *enc_clk;
+	int i;
 
-	mtk_smi_larb_put(pm->larbvenc);
-	for (i = enc_clk->clk_num - 1; i >= 0; i--)
-		clk_disable_unprepare(enc_clk->clk_info[i].vcodec_clk);
+	if (dev->venc_pdata->hw_mode == VENC_FRAME_RACING_MODE) {
+		venc = (struct mtk_venc_comp_dev *)dev->enc_comp_dev[core_id];
+		enc_pm = &venc->pm;
+		enc_clk = &enc_pm->venc_clk;
+
+		for (i = enc_clk->clk_num - 1; i >= 0; i--)
+			clk_disable(enc_clk->clk_info[i].vcodec_clk);
+	} else {
+		enc_pm = &dev->pm;
+		enc_clk = &enc_pm->venc_clk;
+
+		for (i = enc_clk->clk_num - 1; i >= 0; i--)
+			clk_disable_unprepare(enc_clk->clk_info[i].vcodec_clk);
+	}
 }
 
 int mtk_venc_enable_comp_hw(struct mtk_vcodec_dev *dev)
 {
 	int i, ret;
 	struct mtk_venc_comp_dev *venc_comp;
+	struct mtk_vcodec_clk *enc_clk;
+	int j = 0;
 
 	/*
 	 * frame_racing mode needs power on all available component devices.
@@ -140,23 +180,38 @@ int mtk_venc_enable_comp_hw(struct mtk_vcodec_dev *dev)
 			mtk_v4l2_err("power on core[%d] fail %d", i, ret);
 				goto pw_err;
 		}
+
+		enc_clk = &venc_comp->pm.venc_clk;
+		for (j = 0; j < enc_clk->clk_num; j++) {
+			ret = clk_prepare(enc_clk->clk_info[j].vcodec_clk);
+			if (ret) {
+				mtk_v4l2_err("prepare clk [%s] fail %d",
+					     enc_clk->clk_info[j].clk_name,
+					     ret);
+				goto pw_err;
+			}
+		}
 	}
 	return 0;
 
 pw_err:
 	for (i -= 1; i >= 0; i--) {
 		venc_comp = (struct mtk_venc_comp_dev *)dev->enc_comp_dev[i];
+		enc_clk = &venc_comp->pm.venc_clk;
+
+		for (j -= 1; j >= 0; j--)
+			clk_unprepare(enc_clk->clk_info[j].vcodec_clk);
 
 		mtk_smi_larb_put(venc_comp->pm.larbvenc);
 	}
-
 	return ret;
 }
 
 int mtk_venc_disable_comp_hw(struct mtk_vcodec_dev *dev)
 {
-	int i;
+	int i, j;
 	struct mtk_venc_comp_dev *venc_comp;
+	struct mtk_vcodec_clk *enc_clk;
 
 	/*power off all available component device*/
 	for (i = 0; i < MTK_VENC_HW_MAX; i++) {
@@ -164,6 +219,11 @@ int mtk_venc_disable_comp_hw(struct mtk_vcodec_dev *dev)
 		if (!venc_comp)
 			return 0;
 
+		enc_clk = &venc_comp->pm.venc_clk;
+
+		for (j = enc_clk->clk_num - 1; j >= 0; j--)
+			clk_unprepare(enc_clk->clk_info[j].vcodec_clk);
+
 		mtk_smi_larb_put(venc_comp->pm.larbvenc);
 	}
 
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.h
index 2e76b858fed1..f3a819cc0e56 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.h
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.h
@@ -13,8 +13,8 @@ int mtk_vcodec_init_enc_pm(struct platform_device *pdev,
 			   struct mtk_vcodec_pm *pm);
 void mtk_vcodec_release_enc_pm(struct mtk_vcodec_pm *pm);
 
-void mtk_vcodec_enc_clock_on(struct mtk_vcodec_pm *pm);
-void mtk_vcodec_enc_clock_off(struct mtk_vcodec_pm *pm);
+void mtk_vcodec_enc_clock_on(struct mtk_vcodec_dev *dev, int core_id);
+void mtk_vcodec_enc_clock_off(struct mtk_vcodec_dev *dev, int core_id);
 
 int mtk_venc_enable_comp_hw(struct mtk_vcodec_dev *dev);
 int mtk_venc_disable_comp_hw(struct mtk_vcodec_dev *dev);
diff --git a/drivers/media/platform/mtk-vcodec/venc_drv_if.c b/drivers/media/platform/mtk-vcodec/venc_drv_if.c
index ce0bce811615..6cbdb7e30bb3 100644
--- a/drivers/media/platform/mtk-vcodec/venc_drv_if.c
+++ b/drivers/media/platform/mtk-vcodec/venc_drv_if.c
@@ -32,9 +32,7 @@ int venc_if_init(struct mtk_vcodec_ctx *ctx, unsigned int fourcc)
 	}
 
 	mtk_venc_lock(ctx);
-	mtk_vcodec_enc_clock_on(&ctx->dev->pm);
 	ret = ctx->enc_if->init(ctx);
-	mtk_vcodec_enc_clock_off(&ctx->dev->pm);
 	mtk_venc_unlock(ctx);
 
 	return ret;
@@ -46,9 +44,7 @@ int venc_if_set_param(struct mtk_vcodec_ctx *ctx,
 	int ret = 0;
 
 	mtk_venc_lock(ctx);
-	mtk_vcodec_enc_clock_on(&ctx->dev->pm);
 	ret = ctx->enc_if->set_param(ctx->drv_handle, type, in);
-	mtk_vcodec_enc_clock_off(&ctx->dev->pm);
 	mtk_venc_unlock(ctx);
 
 	return ret;
@@ -68,10 +64,10 @@ int venc_if_encode(struct mtk_vcodec_ctx *ctx,
 	ctx->dev->curr_ctx = ctx;
 	spin_unlock_irqrestore(&ctx->dev->irqlock, flags);
 
-	mtk_vcodec_enc_clock_on(&ctx->dev->pm);
+	mtk_vcodec_enc_clock_on(ctx->dev, 0);
 	ret = ctx->enc_if->encode(ctx->drv_handle, opt, frm_buf,
 				  bs_buf, result);
-	mtk_vcodec_enc_clock_off(&ctx->dev->pm);
+	mtk_vcodec_enc_clock_off(ctx->dev, 0);
 
 	spin_lock_irqsave(&ctx->dev->irqlock, flags);
 	ctx->dev->curr_ctx = NULL;
@@ -89,9 +85,7 @@ int venc_if_deinit(struct mtk_vcodec_ctx *ctx)
 		return 0;
 
 	mtk_venc_lock(ctx);
-	mtk_vcodec_enc_clock_on(&ctx->dev->pm);
 	ret = ctx->enc_if->deinit(ctx->drv_handle);
-	mtk_vcodec_enc_clock_off(&ctx->dev->pm);
 	mtk_venc_unlock(ctx);
 
 	ctx->drv_handle = NULL;
-- 
2.25.1
_______________________________________________
Linux-mediatek mailing list
Linux-mediatek@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-mediatek

Re: [PATCH 5/9] media: mtk-vcodec: Rewrite venc clock interface

[Tzung-Bi Shih]

On Mon, Aug 16, 2021 at 06:59:30PM +0800, Irui Wang wrote:
> -void mtk_vcodec_enc_clock_on(struct mtk_vcodec_pm *pm)
> +void mtk_vcodec_enc_clock_on(struct mtk_vcodec_dev *dev, int core_id)
>  {
> -	struct mtk_vcodec_clk *enc_clk = &pm->venc_clk;
> -	int ret, i = 0;
> +	struct mtk_venc_comp_dev *venc;
> +	struct mtk_vcodec_pm *enc_pm;
> +	struct mtk_vcodec_clk *enc_clk;
> +	struct clk		*clk;
To be neat, remove the extra spaces.

> -	ret = mtk_smi_larb_get(pm->larbvenc);
> -	if (ret) {
> -		mtk_v4l2_err("mtk_smi_larb_get larb3 fail %d", ret);
> -		goto clkerr;
I may miss the context but why does it remove mtk_smi_larb_get()?

> -void mtk_vcodec_enc_clock_off(struct mtk_vcodec_pm *pm)
> +void mtk_vcodec_enc_clock_off(struct mtk_vcodec_dev *dev, int core_id)
>  {
> -	struct mtk_vcodec_clk *enc_clk = &pm->venc_clk;
> -	int i = 0;
> +	struct mtk_venc_comp_dev *venc;
> +	struct mtk_vcodec_pm *enc_pm;
> +	struct mtk_vcodec_clk *enc_clk;
> +	int i;
>  
> -	mtk_smi_larb_put(pm->larbvenc);
Same here.  Why does it remove mtk_smi_larb_put()?

>  int mtk_venc_enable_comp_hw(struct mtk_vcodec_dev *dev)
>  {
>  	int i, ret;
>  	struct mtk_venc_comp_dev *venc_comp;
> +	struct mtk_vcodec_clk *enc_clk;
> +	int j = 0;
It doesn't need to be initialized.  Can inline to "int i, ret;".

Re: [PATCH 5/9] media: mtk-vcodec: Rewrite venc clock interface

[Tzung-Bi Shih]

On Mon, Aug 16, 2021 at 06:59:30PM +0800, Irui Wang wrote:
> -void mtk_vcodec_enc_clock_on(struct mtk_vcodec_pm *pm)
> +void mtk_vcodec_enc_clock_on(struct mtk_vcodec_dev *dev, int core_id)
>  {
> -	struct mtk_vcodec_clk *enc_clk = &pm->venc_clk;
> -	int ret, i = 0;
> +	struct mtk_venc_comp_dev *venc;
> +	struct mtk_vcodec_pm *enc_pm;
> +	struct mtk_vcodec_clk *enc_clk;
> +	struct clk		*clk;
To be neat, remove the extra spaces.

> -	ret = mtk_smi_larb_get(pm->larbvenc);
> -	if (ret) {
> -		mtk_v4l2_err("mtk_smi_larb_get larb3 fail %d", ret);
> -		goto clkerr;
I may miss the context but why does it remove mtk_smi_larb_get()?

> -void mtk_vcodec_enc_clock_off(struct mtk_vcodec_pm *pm)
> +void mtk_vcodec_enc_clock_off(struct mtk_vcodec_dev *dev, int core_id)
>  {
> -	struct mtk_vcodec_clk *enc_clk = &pm->venc_clk;
> -	int i = 0;
> +	struct mtk_venc_comp_dev *venc;
> +	struct mtk_vcodec_pm *enc_pm;
> +	struct mtk_vcodec_clk *enc_clk;
> +	int i;
>  
> -	mtk_smi_larb_put(pm->larbvenc);
Same here.  Why does it remove mtk_smi_larb_put()?

>  int mtk_venc_enable_comp_hw(struct mtk_vcodec_dev *dev)
>  {
>  	int i, ret;
>  	struct mtk_venc_comp_dev *venc_comp;
> +	struct mtk_vcodec_clk *enc_clk;
> +	int j = 0;
It doesn't need to be initialized.  Can inline to "int i, ret;".

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH 5/9] media: mtk-vcodec: Rewrite venc clock interface

[Tzung-Bi Shih]

On Mon, Aug 16, 2021 at 06:59:30PM +0800, Irui Wang wrote:
> -void mtk_vcodec_enc_clock_on(struct mtk_vcodec_pm *pm)
> +void mtk_vcodec_enc_clock_on(struct mtk_vcodec_dev *dev, int core_id)
>  {
> -	struct mtk_vcodec_clk *enc_clk = &pm->venc_clk;
> -	int ret, i = 0;
> +	struct mtk_venc_comp_dev *venc;
> +	struct mtk_vcodec_pm *enc_pm;
> +	struct mtk_vcodec_clk *enc_clk;
> +	struct clk		*clk;
To be neat, remove the extra spaces.

> -	ret = mtk_smi_larb_get(pm->larbvenc);
> -	if (ret) {
> -		mtk_v4l2_err("mtk_smi_larb_get larb3 fail %d", ret);
> -		goto clkerr;
I may miss the context but why does it remove mtk_smi_larb_get()?

> -void mtk_vcodec_enc_clock_off(struct mtk_vcodec_pm *pm)
> +void mtk_vcodec_enc_clock_off(struct mtk_vcodec_dev *dev, int core_id)
>  {
> -	struct mtk_vcodec_clk *enc_clk = &pm->venc_clk;
> -	int i = 0;
> +	struct mtk_venc_comp_dev *venc;
> +	struct mtk_vcodec_pm *enc_pm;
> +	struct mtk_vcodec_clk *enc_clk;
> +	int i;
>  
> -	mtk_smi_larb_put(pm->larbvenc);
Same here.  Why does it remove mtk_smi_larb_put()?

>  int mtk_venc_enable_comp_hw(struct mtk_vcodec_dev *dev)
>  {
>  	int i, ret;
>  	struct mtk_venc_comp_dev *venc_comp;
> +	struct mtk_vcodec_clk *enc_clk;
> +	int j = 0;
It doesn't need to be initialized.  Can inline to "int i, ret;".

_______________________________________________
Linux-mediatek mailing list
Linux-mediatek@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-mediatek

[PATCH 6/9] media: mtk-vcodec: Add new venc drv interface for frame_racing mode

[Irui Wang]

Frame-racing mode encoding need more venc working buffers, it
will break the compatibility if we just add venc_vsi in AP-Kernel
but not in firmware, so add a new venc driver interface to
distinguish the sigle_core_mode and frame_racing mode.

The new driver interface can be used for different codecs in
the future.

Signed-off-by: Irui Wang <irui.wang@mediatek.com>
---
 drivers/media/platform/mtk-vcodec/Makefile    |   1 +
 .../platform/mtk-vcodec/mtk_vcodec_drv.h      |   1 +
 .../platform/mtk-vcodec/mtk_vcodec_util.c     |  19 +
 .../platform/mtk-vcodec/mtk_vcodec_util.h     |   4 +
 .../platform/mtk-vcodec/venc/venc_common_if.c | 629 ++++++++++++++++++
 .../media/platform/mtk-vcodec/venc_drv_if.c   |   5 +-
 .../media/platform/mtk-vcodec/venc_drv_if.h   |   1 +
 7 files changed, 659 insertions(+), 1 deletion(-)
 create mode 100644 drivers/media/platform/mtk-vcodec/venc/venc_common_if.c

diff --git a/drivers/media/platform/mtk-vcodec/Makefile b/drivers/media/platform/mtk-vcodec/Makefile
index 661d4afeb628..a42ce704bf04 100644
--- a/drivers/media/platform/mtk-vcodec/Makefile
+++ b/drivers/media/platform/mtk-vcodec/Makefile
@@ -15,6 +15,7 @@ mtk-vcodec-dec-y := vdec/vdec_h264_if.o \
 
 mtk-vcodec-enc-y := venc/venc_vp8_if.o \
 		venc/venc_h264_if.o \
+		venc/venc_common_if.o \
 		mtk_vcodec_enc.o \
 		mtk_vcodec_enc_drv.o \
 		mtk_vcodec_enc_pm.o \
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
index c160b9e505eb..1c6c1eca649f 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
@@ -352,6 +352,7 @@ struct mtk_vcodec_enc_pdata {
 };
 
 #define MTK_ENC_CTX_IS_EXT(ctx) ((ctx)->dev->venc_pdata->uses_ext)
+#define MTK_ENC_HW_MODE(ctx) ((ctx)->dev->venc_pdata->hw_mode)
 
 /**
  * struct mtk_vcodec_dev - driver data
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c
index ac5973b6735f..5057e2cba627 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c
@@ -9,6 +9,7 @@
 
 #include "mtk_vcodec_drv.h"
 #include "mtk_vcodec_util.h"
+#include "mtk_vcodec_enc_hw.h"
 
 /* For encoder, this will enable logs in venc/*/
 bool mtk_vcodec_dbg;
@@ -33,6 +34,24 @@ void __iomem *mtk_vcodec_get_reg_addr(struct mtk_vcodec_ctx *data,
 }
 EXPORT_SYMBOL(mtk_vcodec_get_reg_addr);
 
+void __iomem *mtk_get_venc_comp_reg_addr(struct mtk_vcodec_ctx *data,
+					 unsigned int hw_id)
+{
+	struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)data;
+	struct mtk_venc_comp_dev *venc;
+
+	if (!data || hw_id >= MTK_VENC_HW_MAX) {
+		mtk_v4l2_err("Invalid arguments, hw_id=%d", hw_id);
+		return NULL;
+	}
+	venc = (struct mtk_venc_comp_dev *)ctx->dev->enc_comp_dev[hw_id];
+	if (!venc)
+		return NULL;
+
+	return venc->reg_base;
+}
+EXPORT_SYMBOL(mtk_get_venc_comp_reg_addr);
+
 int mtk_vcodec_mem_alloc(struct mtk_vcodec_ctx *data,
 			struct mtk_vcodec_mem *mem)
 {
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
index b999d7b84ed1..0027747c0a27 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
@@ -75,6 +75,10 @@ extern bool mtk_vcodec_dbg;
 
 void __iomem *mtk_vcodec_get_reg_addr(struct mtk_vcodec_ctx *data,
 				unsigned int reg_idx);
+
+void __iomem *mtk_get_venc_comp_reg_addr(struct mtk_vcodec_ctx *data,
+					 unsigned int hw_id);
+
 int mtk_vcodec_mem_alloc(struct mtk_vcodec_ctx *data,
 				struct mtk_vcodec_mem *mem);
 void mtk_vcodec_mem_free(struct mtk_vcodec_ctx *data,
diff --git a/drivers/media/platform/mtk-vcodec/venc/venc_common_if.c b/drivers/media/platform/mtk-vcodec/venc/venc_common_if.c
new file mode 100644
index 000000000000..01b7d93241af
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/venc/venc_common_if.c
@@ -0,0 +1,629 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2021 MediaTek Inc.
+ */
+
+#include "../venc_vpu_if.h"
+#include "../venc_drv_base.h"
+#include "../mtk_vcodec_intr.h"
+#include "../mtk_vcodec_enc.h"
+#include "../mtk_vcodec_drv.h"
+#include "../mtk_vcodec_util.h"
+#include "../mtk_vcodec_enc_hw.h"
+
+static const char h264_filler_marker[] = {0x0, 0x0, 0x0, 0x1, 0xc};
+
+#define H264_FILLER_MARKER_SIZE ARRAY_SIZE(h264_filler_marker)
+#define VENC_PIC_BITSTREAM_BYTE_CNT 0x0098
+
+enum venc_vpu_work_buf {
+	VENC_VPU_WORK_BUF_RC_INFO_CORE0,
+	VENC_VPU_WORK_BUF_RC_INFO_CORE1,
+	VENC_VPU_WORK_BUF_FR_RC_INFO,
+	VENC_VPU_WORK_BUF_RC_CODE,
+	VENC_VPU_WORK_BUF_REC_LUMA,
+	VENC_VPU_WORK_BUF_REC_CHROMA,
+	VENC_VPU_WORK_BUF_REF_LUMA,
+	VENC_VPU_WORK_BUF_REF_CHROMA,
+	VENC_VPU_WORK_BUF_MV_INFO_1,
+	VENC_VPU_WORK_BUF_MV_INFO_2,
+	VENC_VPU_WORK_BUF_SKIP_FRAME,
+	VENC_VPU_WORK_BUF_MAX,
+};
+
+enum venc_frame_type {
+	VENC_IDR_FRM,
+	VENC_I_FRM,
+	VENC_P_FRM,
+	VENC_B_FRM,
+};
+
+enum venc_bs_mode {
+	VENC_BS_MODE_SPS,
+	VENC_BS_MODE_PPS,
+	VENC_BS_MODE_FRAME,
+};
+
+struct venc_vpu_buf {
+	u32 iova;
+	u32 vpua;
+	u32 size;
+};
+
+struct venc_vpu_config {
+	u32 input_fourcc;
+	u32 bitrate;
+	u32 pic_w;
+	u32 pic_h;
+	u32 buf_w;
+	u32 buf_h;
+	u32 gop_size;
+	u32 intra_period;
+	u32 framerate;
+	u32 profile;
+	u32 level;
+	u32 wfd;
+	u32 max_qp;
+	u32 min_qp;
+	u32 reserved[8];
+};
+
+struct venc_vsi {
+	struct venc_vpu_config config;
+	struct venc_vpu_buf work_bufs[VENC_VPU_WORK_BUF_MAX];
+};
+
+struct venc_common_inst {
+	void __iomem *hw_base[MTK_VENC_HW_MAX];
+	struct mtk_vcodec_mem work_bufs[VENC_VPU_WORK_BUF_MAX];
+	struct mtk_vcodec_mem pps_buf;
+	bool work_buf_allocated;
+	unsigned int frm_cnt;
+	unsigned int skip_frm_cnt;
+	unsigned int prepend_hdr;
+	struct venc_vpu_inst vpu_inst;
+	struct venc_vsi *vsi;
+	struct mtk_vcodec_ctx *ctx;
+};
+
+static inline u32 venc_read_reg(struct venc_common_inst *inst,
+				u32 addr, int hw_id)
+{
+	return readl(inst->hw_base[hw_id] + addr);
+}
+
+static unsigned int h264_get_profile(struct venc_common_inst *inst,
+				     unsigned int profile)
+{
+	switch (profile) {
+	case V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE:
+		return 66;
+	case V4L2_MPEG_VIDEO_H264_PROFILE_MAIN:
+		return 77;
+	case V4L2_MPEG_VIDEO_H264_PROFILE_HIGH:
+		return 100;
+	case V4L2_MPEG_VIDEO_H264_PROFILE_CONSTRAINED_BASELINE:
+		mtk_vcodec_err(inst, "unsupported CONSTRAINED_BASELINE");
+		return 0;
+	case V4L2_MPEG_VIDEO_H264_PROFILE_EXTENDED:
+		mtk_vcodec_err(inst, "unsupported EXTENDED");
+		return 0;
+	default:
+		mtk_vcodec_debug(inst, "unsupported profile %d", profile);
+		return 100;
+	}
+}
+
+static unsigned int h264_get_level(struct venc_common_inst *inst,
+				   unsigned int level)
+{
+	switch (level) {
+	case V4L2_MPEG_VIDEO_H264_LEVEL_1B:
+		mtk_vcodec_err(inst, "unsupported 1B");
+		return 0;
+	case V4L2_MPEG_VIDEO_H264_LEVEL_1_0:
+		return 10;
+	case V4L2_MPEG_VIDEO_H264_LEVEL_1_1:
+		return 11;
+	case V4L2_MPEG_VIDEO_H264_LEVEL_1_2:
+		return 12;
+	case V4L2_MPEG_VIDEO_H264_LEVEL_1_3:
+		return 13;
+	case V4L2_MPEG_VIDEO_H264_LEVEL_2_0:
+		return 20;
+	case V4L2_MPEG_VIDEO_H264_LEVEL_2_1:
+		return 21;
+	case V4L2_MPEG_VIDEO_H264_LEVEL_2_2:
+		return 22;
+	case V4L2_MPEG_VIDEO_H264_LEVEL_3_0:
+		return 30;
+	case V4L2_MPEG_VIDEO_H264_LEVEL_3_1:
+		return 31;
+	case V4L2_MPEG_VIDEO_H264_LEVEL_3_2:
+		return 32;
+	case V4L2_MPEG_VIDEO_H264_LEVEL_4_0:
+		return 40;
+	case V4L2_MPEG_VIDEO_H264_LEVEL_4_1:
+		return 41;
+	case V4L2_MPEG_VIDEO_H264_LEVEL_4_2:
+		return 42;
+	case V4L2_MPEG_VIDEO_H264_LEVEL_5_0:
+		return 50;
+	case V4L2_MPEG_VIDEO_H264_LEVEL_5_1:
+		return 51;
+	default:
+		mtk_vcodec_debug(inst, "unsupported level %d", level);
+		return 31;
+	}
+}
+
+static void venc_free_work_buf(struct venc_common_inst *inst)
+{
+	int i;
+
+	mtk_vcodec_debug_enter(inst);
+
+	for (i = 0; i < VENC_VPU_WORK_BUF_MAX; i++) {
+		if (i != VENC_VPU_WORK_BUF_SKIP_FRAME)
+			mtk_vcodec_mem_free(inst->ctx, &inst->work_bufs[i]);
+	}
+
+	mtk_vcodec_mem_free(inst->ctx, &inst->pps_buf);
+
+	mtk_vcodec_debug_leave(inst);
+}
+
+static int venc_alloc_work_buf(struct venc_common_inst *inst)
+{
+	int i;
+	int ret = 0;
+	struct venc_vpu_buf *wb = inst->vsi->work_bufs;
+
+	mtk_vcodec_debug_enter(inst);
+
+	for (i = 0; i < VENC_VPU_WORK_BUF_MAX; i++) {
+		/*
+		 * This 'wb' structure is set by VPU side and shared to AP for
+		 * buffer allocation and IO virtual addr mapping. For most of
+		 * the buffers, AP will allocate the buffer according to 'size'
+		 * field and store the IO virtual addr in 'iova' field. There
+		 * are two exceptions:
+		 * (1) RC_CODE buffer, it's pre-allocated in the VPU side, and
+		 * save the VPU addr in the 'vpua' field. The AP will translate
+		 * the VPU addr to the corresponding IO virtual addr and store
+		 * in 'iova' field for reg setting in VPU side.
+		 * (2) SKIP_FRAME buffer, it's pre-allocated in the VPU side,
+		 * and save the VPU addr in the 'vpua' field. The AP will
+		 * translate the VPU addr to the corresponding AP side virtual
+		 * address and do some memcpy access to move to bitstream buffer
+		 * assigned by v4l2 layer.
+		 */
+		inst->work_bufs[i].size = wb[i].size;
+		if (i == VENC_VPU_WORK_BUF_SKIP_FRAME) {
+			struct mtk_vcodec_fw *handler;
+
+			handler = inst->vpu_inst.ctx->dev->fw_handler;
+			inst->work_bufs[i].va =
+				mtk_vcodec_fw_map_dm_addr(handler, wb[i].vpua);
+			inst->work_bufs[i].dma_addr = 0;
+		} else {
+			ret = mtk_vcodec_mem_alloc(inst->ctx,
+						   &inst->work_bufs[i]);
+			if (ret) {
+				mtk_vcodec_err(inst,
+					       "cannot allocate buf %d", i);
+				goto err_alloc;
+			}
+			/*
+			 * This RC_CODE is pre-allocated by VPU and saved in VPU
+			 * addr. So we need use memcpy to copy RC_CODE from VPU
+			 * addr into IO virtual addr in 'iova' field for reg
+			 * setting in VPU side.
+			 */
+			if (i == VENC_VPU_WORK_BUF_RC_CODE) {
+				struct mtk_vcodec_fw *handler;
+				void *tmp_va;
+
+				handler = inst->vpu_inst.ctx->dev->fw_handler;
+				tmp_va = mtk_vcodec_fw_map_dm_addr(handler,
+								   wb[i].vpua);
+				memcpy(inst->work_bufs[i].va, tmp_va,
+				       wb[i].size);
+			}
+		}
+		wb[i].iova = inst->work_bufs[i].dma_addr;
+
+		mtk_vcodec_debug(inst,
+				 "work_buf[%d] va=0x%p iova=%pad size=%zu",
+				 i, inst->work_bufs[i].va,
+				 &inst->work_bufs[i].dma_addr,
+				 inst->work_bufs[i].size);
+	}
+
+	/* the pps_buf is used by AP side only */
+	inst->pps_buf.size = 128;
+	ret = mtk_vcodec_mem_alloc(inst->ctx, &inst->pps_buf);
+	if (ret) {
+		mtk_vcodec_err(inst, "cannot allocate pps_buf");
+		goto err_alloc;
+	}
+
+	mtk_vcodec_debug_leave(inst);
+
+	return ret;
+
+err_alloc:
+	venc_free_work_buf(inst);
+
+	return ret;
+}
+
+static unsigned int venc_wait_comp_done(struct venc_common_inst *inst,
+					unsigned int hw_id)
+{
+	unsigned int irq_status = 0;
+	struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)inst->ctx;
+
+	if (!mtk_vcodec_wait_for_done_ctx(ctx, MTK_INST_IRQ_RECEIVED,
+					  WAIT_INTR_TIMEOUT_MS)) {
+		irq_status = ctx->irq_status;
+		mtk_vcodec_debug(inst, "irq_status %x <-", irq_status);
+	}
+	return irq_status;
+}
+
+static int venc_frame_type(struct venc_common_inst *inst)
+{
+	if ((inst->vsi->config.gop_size != 0 &&
+	     (inst->frm_cnt % inst->vsi->config.gop_size) == 0) ||
+	    (inst->frm_cnt == 0 && inst->vsi->config.gop_size == 0)) {
+		/* IDR frame */
+		return VENC_IDR_FRM;
+	} else if ((inst->vsi->config.intra_period != 0 &&
+		    (inst->frm_cnt % inst->vsi->config.intra_period) == 0) ||
+		   (inst->frm_cnt == 0 &&
+		    inst->vsi->config.intra_period == 0)) {
+		/* I frame */
+		return VENC_I_FRM;
+	} else {
+		return VENC_P_FRM;  /* Note: B frames are not supported */
+	}
+}
+
+static int venc_encode_sps(struct venc_common_inst *inst,
+			   struct mtk_vcodec_mem *bs_buf,
+			   unsigned int *bs_size)
+{
+	int ret = 0;
+	unsigned int irq_status;
+
+	mtk_vcodec_debug_enter(inst);
+
+	ret = vpu_enc_encode(&inst->vpu_inst, VENC_BS_MODE_SPS, NULL,
+			     bs_buf, bs_size, NULL);
+	if (ret)
+		return ret;
+
+	irq_status = venc_wait_comp_done(inst, MTK_VENC_CORE0);
+	if (irq_status != MTK_VENC_IRQ_STATUS_SPS) {
+		mtk_vcodec_err(inst, "expect irq status %d",
+			       MTK_VENC_IRQ_STATUS_SPS);
+		return -EINVAL;
+	}
+
+	*bs_size = venc_read_reg(inst, VENC_PIC_BITSTREAM_BYTE_CNT,
+				 MTK_VENC_CORE0);
+	mtk_vcodec_debug(inst, "bs size %d <-", *bs_size);
+
+	return ret;
+}
+
+static int venc_encode_pps(struct venc_common_inst *inst,
+			   struct mtk_vcodec_mem *bs_buf,
+			   unsigned int *bs_size)
+{
+	int ret = 0;
+	unsigned int irq_status;
+
+	mtk_vcodec_debug_enter(inst);
+
+	ret = vpu_enc_encode(&inst->vpu_inst, VENC_BS_MODE_PPS, NULL,
+			     bs_buf, bs_size, NULL);
+	if (ret)
+		return ret;
+
+	irq_status = venc_wait_comp_done(inst, MTK_VENC_CORE0);
+	if (irq_status != MTK_VENC_IRQ_STATUS_PPS) {
+		mtk_vcodec_err(inst, "expect irq status %d",
+			       MTK_VENC_IRQ_STATUS_PPS);
+		return -EINVAL;
+	}
+
+	*bs_size = venc_read_reg(inst, VENC_PIC_BITSTREAM_BYTE_CNT,
+				 MTK_VENC_CORE0);
+	mtk_vcodec_debug(inst, "bs size %d <-", *bs_size);
+
+	return ret;
+}
+
+static int venc_encode_header(struct venc_common_inst *inst,
+			      struct mtk_vcodec_mem *bs_buf,
+			      unsigned int *bs_size)
+{
+	int ret = 0;
+	unsigned int bs_size_sps;
+	unsigned int bs_size_pps;
+
+	ret = venc_encode_sps(inst, bs_buf, &bs_size_sps);
+	if (ret)
+		return ret;
+
+	ret = venc_encode_pps(inst, &inst->pps_buf, &bs_size_pps);
+	if (ret)
+		return ret;
+
+	memcpy(bs_buf->va + bs_size_sps, inst->pps_buf.va, bs_size_pps);
+	*bs_size = bs_size_sps + bs_size_pps;
+
+	return ret;
+}
+
+static int venc_encode_frame(struct venc_common_inst *inst,
+			     struct venc_frm_buf *frm_buf,
+			     struct mtk_vcodec_mem *bs_buf,
+			     unsigned int *bs_size,
+			     int hw_id)
+{
+	int ret = 0;
+	struct venc_frame_info frame_info;
+
+	mtk_vcodec_debug_enter(inst);
+	mtk_vcodec_debug(inst, "frm_cnt = %d\n ", inst->frm_cnt);
+	frame_info.frm_count = inst->frm_cnt;
+	frame_info.skip_frm_count = inst->skip_frm_cnt;
+	frame_info.frm_type = venc_frame_type(inst);
+	mtk_vcodec_debug(inst, "frm_count = %d,skip_frm_count =%d,frm_type=%d.\n",
+			 frame_info.frm_count, frame_info.skip_frm_count,
+			 frame_info.frm_type);
+	ret = vpu_enc_encode(&inst->vpu_inst, VENC_BS_MODE_FRAME, frm_buf,
+			     bs_buf, bs_size, &frame_info);
+	if (ret)
+		return ret;
+
+	++inst->frm_cnt;
+	mtk_vcodec_debug(inst, "frm %d bs_size %d key_frm %d <-",
+			 inst->frm_cnt, *bs_size, inst->vpu_inst.is_key_frm);
+
+	return ret;
+}
+
+static void h264_encode_filler(struct venc_common_inst *inst, void *buf,
+			       int size)
+{
+	unsigned char *p = buf;
+
+	if (size < H264_FILLER_MARKER_SIZE) {
+		mtk_vcodec_err(inst, "filler size too small %d", size);
+		return;
+	}
+
+	memcpy(p, h264_filler_marker, ARRAY_SIZE(h264_filler_marker));
+	size -= H264_FILLER_MARKER_SIZE;
+	p += H264_FILLER_MARKER_SIZE;
+	memset(p, 0xff, size);
+}
+
+static int venc_init(struct mtk_vcodec_ctx *ctx)
+{
+	int i;
+	int ret = 0;
+	struct venc_common_inst *inst;
+
+	inst = kzalloc(sizeof(*inst), GFP_KERNEL);
+	if (!inst)
+		return -ENOMEM;
+
+	inst->ctx = ctx;
+	inst->vpu_inst.ctx = ctx;
+	inst->vpu_inst.id = SCP_IPI_VENC_H264;
+
+	mtk_vcodec_debug_enter(inst);
+
+	ret = vpu_enc_init(&inst->vpu_inst);
+
+	inst->vsi = (struct venc_vsi *)inst->vpu_inst.vsi;
+
+	for (i = 0; i < MTK_VENC_HW_MAX; i++)
+		inst->hw_base[i] = mtk_get_venc_comp_reg_addr(ctx, i);
+
+	mtk_vcodec_debug_leave(inst);
+
+	if (ret)
+		kfree(inst);
+	else
+		ctx->drv_handle = inst;
+
+	return ret;
+}
+
+static int venc_encode(void *handle,
+		       enum venc_start_opt opt,
+		       struct venc_frm_buf *frm_buf,
+		       struct mtk_vcodec_mem *bs_buf,
+		       struct venc_done_result *result)
+{
+	int ret;
+	struct venc_common_inst *inst = (struct venc_common_inst *)handle;
+	struct mtk_vcodec_ctx *ctx = inst->ctx;
+
+	switch (opt) {
+	case VENC_START_OPT_ENCODE_SEQUENCE_HEADER: {
+		unsigned int bs_size_hdr;
+
+		ret = venc_encode_header(inst, bs_buf, &bs_size_hdr);
+		if (ret) {
+			mtk_vcodec_err(inst, "encode header failed: %d", ret);
+			return ret;
+		}
+
+		result->bs_size = bs_size_hdr;
+		result->is_key_frm = false;
+
+		break;
+	}
+
+	case VENC_START_OPT_ENCODE_FRAME: {
+		int hdr_sz;
+		int hdr_sz_ext;
+		int filler_sz = 0;
+		const int bs_align = 128;
+		struct mtk_vcodec_mem tmp_bs_buf;
+		unsigned int bs_size_hdr;
+		unsigned int bs_size_frm;
+
+		if (!inst->prepend_hdr) {
+			ret = venc_encode_frame(inst, frm_buf, bs_buf,
+						&result->bs_size, ctx->hw_id);
+			if (ret) {
+				mtk_vcodec_err(inst, "encode frame failed: %d",
+					       ret);
+				return ret;
+			}
+
+			result->is_key_frm = inst->vpu_inst.is_key_frm;
+			break;
+		}
+
+		mtk_vcodec_debug(inst, "venc_encode_frame prepend SPS/PPS");
+
+		ret = venc_encode_header(inst, bs_buf, &bs_size_hdr);
+		if (ret) {
+			mtk_vcodec_err(inst, "encode prepend hdr failed: %d",
+				       ret);
+			return ret;
+		}
+
+		hdr_sz = bs_size_hdr;
+		hdr_sz_ext = (hdr_sz & (bs_align - 1));
+		if (hdr_sz_ext) {
+			filler_sz = bs_align - hdr_sz_ext;
+			if (hdr_sz_ext + H264_FILLER_MARKER_SIZE > bs_align)
+				filler_sz += bs_align;
+			h264_encode_filler(inst, bs_buf->va + hdr_sz,
+					   filler_sz);
+		}
+
+		tmp_bs_buf.va = bs_buf->va + hdr_sz + filler_sz;
+		tmp_bs_buf.dma_addr = bs_buf->dma_addr + hdr_sz + filler_sz;
+		tmp_bs_buf.size = bs_buf->size - (hdr_sz + filler_sz);
+
+		ret = venc_encode_frame(inst, frm_buf, &tmp_bs_buf,
+					&bs_size_frm, ctx->hw_id);
+
+		if (ret) {
+			mtk_vcodec_err(inst, "encode hdr frame failed: %d",
+				       ret);
+			return ret;
+		}
+
+		result->bs_size = hdr_sz + filler_sz + bs_size_frm;
+
+		mtk_vcodec_debug(inst, "hdr %d filler %d frame %d bs %d",
+				 hdr_sz, filler_sz, bs_size_frm,
+				 result->bs_size);
+
+		inst->prepend_hdr = 0;
+		result->is_key_frm = inst->vpu_inst.is_key_frm;
+		break;
+	}
+
+	default:
+		mtk_vcodec_err(inst, "venc_start_opt %d not supported", opt);
+		return -EINVAL;
+	}
+
+	return ret;
+}
+
+static int venc_set_param(void *handle,
+			  enum venc_set_param_type type,
+			  struct venc_enc_param *enc_prm)
+{
+	int ret = 0;
+	struct venc_common_inst *inst = (struct venc_common_inst *)handle;
+
+	mtk_vcodec_debug(inst, "->type=%d", type);
+
+	switch (type) {
+	case VENC_SET_PARAM_ENC:
+		inst->vsi->config.input_fourcc = enc_prm->input_yuv_fmt;
+		inst->vsi->config.bitrate = enc_prm->bitrate;
+		inst->vsi->config.pic_w = enc_prm->width;
+		inst->vsi->config.pic_h = enc_prm->height;
+		inst->vsi->config.buf_w = enc_prm->buf_width;
+		inst->vsi->config.buf_h = enc_prm->buf_height;
+		inst->vsi->config.gop_size = enc_prm->gop_size;
+		inst->vsi->config.framerate = enc_prm->frm_rate;
+		inst->vsi->config.intra_period = enc_prm->intra_period;
+		inst->vsi->config.profile =
+			h264_get_profile(inst, enc_prm->h264_profile);
+		inst->vsi->config.level =
+			h264_get_level(inst, enc_prm->h264_level);
+		inst->vsi->config.wfd = 0;
+
+		ret = vpu_enc_set_param(&inst->vpu_inst, type, enc_prm);
+		if (ret)
+			break;
+		if (inst->work_buf_allocated) {
+			venc_free_work_buf(inst);
+			inst->work_buf_allocated = false;
+		}
+		ret = venc_alloc_work_buf(inst);
+		if (ret)
+			break;
+		inst->work_buf_allocated = true;
+		break;
+
+	case VENC_SET_PARAM_PREPEND_HEADER:
+		inst->prepend_hdr = 1;
+		mtk_vcodec_debug(inst, "set prepend header mode");
+		break;
+	case VENC_SET_PARAM_FORCE_INTRA:
+	case VENC_SET_PARAM_GOP_SIZE:
+	case VENC_SET_PARAM_INTRA_PERIOD:
+		inst->frm_cnt = 0;
+		inst->skip_frm_cnt = 0;
+		fallthrough;
+	default:
+		ret = vpu_enc_set_param(&inst->vpu_inst, type, enc_prm);
+		break;
+	}
+
+	mtk_vcodec_debug_leave(inst);
+	return ret;
+}
+
+static int venc_deinit(void *handle)
+{
+	int ret = 0;
+	struct venc_common_inst *inst = (struct venc_common_inst *)handle;
+
+	mtk_vcodec_debug_enter(inst);
+
+	ret = vpu_enc_deinit(&inst->vpu_inst);
+
+	if (inst->work_buf_allocated)
+		venc_free_work_buf(inst);
+
+	mtk_vcodec_debug_leave(inst);
+	kfree(inst);
+
+	return ret;
+}
+
+const struct venc_common_if venc_if = {
+	.init = venc_init,
+	.encode = venc_encode,
+	.set_param = venc_set_param,
+	.deinit = venc_deinit,
+};
diff --git a/drivers/media/platform/mtk-vcodec/venc_drv_if.c b/drivers/media/platform/mtk-vcodec/venc_drv_if.c
index 6cbdb7e30bb3..93936d002bed 100644
--- a/drivers/media/platform/mtk-vcodec/venc_drv_if.c
+++ b/drivers/media/platform/mtk-vcodec/venc_drv_if.c
@@ -25,7 +25,10 @@ int venc_if_init(struct mtk_vcodec_ctx *ctx, unsigned int fourcc)
 		ctx->enc_if = &venc_vp8_if;
 		break;
 	case V4L2_PIX_FMT_H264:
-		ctx->enc_if = &venc_h264_if;
+		if (MTK_ENC_HW_MODE(ctx) == VENC_FRAME_RACING_MODE)
+			ctx->enc_if = &venc_if;
+		else
+			ctx->enc_if = &venc_h264_if;
 		break;
 	default:
 		return -EINVAL;
diff --git a/drivers/media/platform/mtk-vcodec/venc_drv_if.h b/drivers/media/platform/mtk-vcodec/venc_drv_if.h
index 0b04a1020873..8f384ba24ddf 100644
--- a/drivers/media/platform/mtk-vcodec/venc_drv_if.h
+++ b/drivers/media/platform/mtk-vcodec/venc_drv_if.h
@@ -125,6 +125,7 @@ struct venc_done_result {
 
 extern const struct venc_common_if venc_h264_if;
 extern const struct venc_common_if venc_vp8_if;
+extern const struct venc_common_if venc_if;
 
 /*
  * venc_if_init - Create the driver handle
-- 
2.25.1

[PATCH 6/9] media: mtk-vcodec: Add new venc drv interface for frame_racing mode

[Irui Wang]

Frame-racing mode encoding need more venc working buffers, it
will break the compatibility if we just add venc_vsi in AP-Kernel
but not in firmware, so add a new venc driver interface to
distinguish the sigle_core_mode and frame_racing mode.

The new driver interface can be used for different codecs in
the future.

Signed-off-by: Irui Wang <irui.wang@mediatek.com>
---
 drivers/media/platform/mtk-vcodec/Makefile    |   1 +
 .../platform/mtk-vcodec/mtk_vcodec_drv.h      |   1 +
 .../platform/mtk-vcodec/mtk_vcodec_util.c     |  19 +
 .../platform/mtk-vcodec/mtk_vcodec_util.h     |   4 +
 .../platform/mtk-vcodec/venc/venc_common_if.c | 629 ++++++++++++++++++
 .../media/platform/mtk-vcodec/venc_drv_if.c   |   5 +-
 .../media/platform/mtk-vcodec/venc_drv_if.h   |   1 +
 7 files changed, 659 insertions(+), 1 deletion(-)
 create mode 100644 drivers/media/platform/mtk-vcodec/venc/venc_common_if.c

diff --git a/drivers/media/platform/mtk-vcodec/Makefile b/drivers/media/platform/mtk-vcodec/Makefile
index 661d4afeb628..a42ce704bf04 100644
--- a/drivers/media/platform/mtk-vcodec/Makefile
+++ b/drivers/media/platform/mtk-vcodec/Makefile
@@ -15,6 +15,7 @@ mtk-vcodec-dec-y := vdec/vdec_h264_if.o \
 
 mtk-vcodec-enc-y := venc/venc_vp8_if.o \
 		venc/venc_h264_if.o \
+		venc/venc_common_if.o \
 		mtk_vcodec_enc.o \
 		mtk_vcodec_enc_drv.o \
 		mtk_vcodec_enc_pm.o \
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
index c160b9e505eb..1c6c1eca649f 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
@@ -352,6 +352,7 @@ struct mtk_vcodec_enc_pdata {
 };
 
 #define MTK_ENC_CTX_IS_EXT(ctx) ((ctx)->dev->venc_pdata->uses_ext)
+#define MTK_ENC_HW_MODE(ctx) ((ctx)->dev->venc_pdata->hw_mode)
 
 /**
  * struct mtk_vcodec_dev - driver data
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c
index ac5973b6735f..5057e2cba627 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c
@@ -9,6 +9,7 @@
 
 #include "mtk_vcodec_drv.h"
 #include "mtk_vcodec_util.h"
+#include "mtk_vcodec_enc_hw.h"
 
 /* For encoder, this will enable logs in venc/*/
 bool mtk_vcodec_dbg;
@@ -33,6 +34,24 @@ void __iomem *mtk_vcodec_get_reg_addr(struct mtk_vcodec_ctx *data,
 }
 EXPORT_SYMBOL(mtk_vcodec_get_reg_addr);
 
+void __iomem *mtk_get_venc_comp_reg_addr(struct mtk_vcodec_ctx *data,
+					 unsigned int hw_id)
+{
+	struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)data;
+	struct mtk_venc_comp_dev *venc;
+
+	if (!data || hw_id >= MTK_VENC_HW_MAX) {
+		mtk_v4l2_err("Invalid arguments, hw_id=%d", hw_id);
+		return NULL;
+	}
+	venc = (struct mtk_venc_comp_dev *)ctx->dev->enc_comp_dev[hw_id];
+	if (!venc)
+		return NULL;
+
+	return venc->reg_base;
+}
+EXPORT_SYMBOL(mtk_get_venc_comp_reg_addr);
+
 int mtk_vcodec_mem_alloc(struct mtk_vcodec_ctx *data,
 			struct mtk_vcodec_mem *mem)
 {
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
index b999d7b84ed1..0027747c0a27 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
@@ -75,6 +75,10 @@ extern bool mtk_vcodec_dbg;
 
 void __iomem *mtk_vcodec_get_reg_addr(struct mtk_vcodec_ctx *data,
 				unsigned int reg_idx);
+
+void __iomem *mtk_get_venc_comp_reg_addr(struct mtk_vcodec_ctx *data,
+					 unsigned int hw_id);
+
 int mtk_vcodec_mem_alloc(struct mtk_vcodec_ctx *data,
 				struct mtk_vcodec_mem *mem);
 void mtk_vcodec_mem_free(struct mtk_vcodec_ctx *data,
diff --git a/drivers/media/platform/mtk-vcodec/venc/venc_common_if.c b/drivers/media/platform/mtk-vcodec/venc/venc_common_if.c
new file mode 100644
index 000000000000..01b7d93241af
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/venc/venc_common_if.c
@@ -0,0 +1,629 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2021 MediaTek Inc.
+ */
+
+#include "../venc_vpu_if.h"
+#include "../venc_drv_base.h"
+#include "../mtk_vcodec_intr.h"
+#include "../mtk_vcodec_enc.h"
+#include "../mtk_vcodec_drv.h"
+#include "../mtk_vcodec_util.h"
+#include "../mtk_vcodec_enc_hw.h"
+
+static const char h264_filler_marker[] = {0x0, 0x0, 0x0, 0x1, 0xc};
+
+#define H264_FILLER_MARKER_SIZE ARRAY_SIZE(h264_filler_marker)
+#define VENC_PIC_BITSTREAM_BYTE_CNT 0x0098
+
+enum venc_vpu_work_buf {
+	VENC_VPU_WORK_BUF_RC_INFO_CORE0,
+	VENC_VPU_WORK_BUF_RC_INFO_CORE1,
+	VENC_VPU_WORK_BUF_FR_RC_INFO,
+	VENC_VPU_WORK_BUF_RC_CODE,
+	VENC_VPU_WORK_BUF_REC_LUMA,
+	VENC_VPU_WORK_BUF_REC_CHROMA,
+	VENC_VPU_WORK_BUF_REF_LUMA,
+	VENC_VPU_WORK_BUF_REF_CHROMA,
+	VENC_VPU_WORK_BUF_MV_INFO_1,
+	VENC_VPU_WORK_BUF_MV_INFO_2,
+	VENC_VPU_WORK_BUF_SKIP_FRAME,
+	VENC_VPU_WORK_BUF_MAX,
+};
+
+enum venc_frame_type {
+	VENC_IDR_FRM,
+	VENC_I_FRM,
+	VENC_P_FRM,
+	VENC_B_FRM,
+};
+
+enum venc_bs_mode {
+	VENC_BS_MODE_SPS,
+	VENC_BS_MODE_PPS,
+	VENC_BS_MODE_FRAME,
+};
+
+struct venc_vpu_buf {
+	u32 iova;
+	u32 vpua;
+	u32 size;
+};
+
+struct venc_vpu_config {
+	u32 input_fourcc;
+	u32 bitrate;
+	u32 pic_w;
+	u32 pic_h;
+	u32 buf_w;
+	u32 buf_h;
+	u32 gop_size;
+	u32 intra_period;
+	u32 framerate;
+	u32 profile;
+	u32 level;
+	u32 wfd;
+	u32 max_qp;
+	u32 min_qp;
+	u32 reserved[8];
+};
+
+struct venc_vsi {
+	struct venc_vpu_config config;
+	struct venc_vpu_buf work_bufs[VENC_VPU_WORK_BUF_MAX];
+};
+
+struct venc_common_inst {
+	void __iomem *hw_base[MTK_VENC_HW_MAX];
+	struct mtk_vcodec_mem work_bufs[VENC_VPU_WORK_BUF_MAX];
+	struct mtk_vcodec_mem pps_buf;
+	bool work_buf_allocated;
+	unsigned int frm_cnt;
+	unsigned int skip_frm_cnt;
+	unsigned int prepend_hdr;
+	struct venc_vpu_inst vpu_inst;
+	struct venc_vsi *vsi;
+	struct mtk_vcodec_ctx *ctx;
+};
+
+static inline u32 venc_read_reg(struct venc_common_inst *inst,
+				u32 addr, int hw_id)
+{
+	return readl(inst->hw_base[hw_id] + addr);
+}
+
+static unsigned int h264_get_profile(struct venc_common_inst *inst,
+				     unsigned int profile)
+{
+	switch (profile) {
+	case V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE:
+		return 66;
+	case V4L2_MPEG_VIDEO_H264_PROFILE_MAIN:
+		return 77;
+	case V4L2_MPEG_VIDEO_H264_PROFILE_HIGH:
+		return 100;
+	case V4L2_MPEG_VIDEO_H264_PROFILE_CONSTRAINED_BASELINE:
+		mtk_vcodec_err(inst, "unsupported CONSTRAINED_BASELINE");
+		return 0;
+	case V4L2_MPEG_VIDEO_H264_PROFILE_EXTENDED:
+		mtk_vcodec_err(inst, "unsupported EXTENDED");
+		return 0;
+	default:
+		mtk_vcodec_debug(inst, "unsupported profile %d", profile);
+		return 100;
+	}
+}
+
+static unsigned int h264_get_level(struct venc_common_inst *inst,
+				   unsigned int level)
+{
+	switch (level) {
+	case V4L2_MPEG_VIDEO_H264_LEVEL_1B:
+		mtk_vcodec_err(inst, "unsupported 1B");
+		return 0;
+	case V4L2_MPEG_VIDEO_H264_LEVEL_1_0:
+		return 10;
+	case V4L2_MPEG_VIDEO_H264_LEVEL_1_1:
+		return 11;
+	case V4L2_MPEG_VIDEO_H264_LEVEL_1_2:
+		return 12;
+	case V4L2_MPEG_VIDEO_H264_LEVEL_1_3:
+		return 13;
+	case V4L2_MPEG_VIDEO_H264_LEVEL_2_0:
+		return 20;
+	case V4L2_MPEG_VIDEO_H264_LEVEL_2_1:
+		return 21;
+	case V4L2_MPEG_VIDEO_H264_LEVEL_2_2:
+		return 22;
+	case V4L2_MPEG_VIDEO_H264_LEVEL_3_0:
+		return 30;
+	case V4L2_MPEG_VIDEO_H264_LEVEL_3_1:
+		return 31;
+	case V4L2_MPEG_VIDEO_H264_LEVEL_3_2:
+		return 32;
+	case V4L2_MPEG_VIDEO_H264_LEVEL_4_0:
+		return 40;
+	case V4L2_MPEG_VIDEO_H264_LEVEL_4_1:
+		return 41;
+	case V4L2_MPEG_VIDEO_H264_LEVEL_4_2:
+		return 42;
+	case V4L2_MPEG_VIDEO_H264_LEVEL_5_0:
+		return 50;
+	case V4L2_MPEG_VIDEO_H264_LEVEL_5_1:
+		return 51;
+	default:
+		mtk_vcodec_debug(inst, "unsupported level %d", level);
+		return 31;
+	}
+}
+
+static void venc_free_work_buf(struct venc_common_inst *inst)
+{
+	int i;
+
+	mtk_vcodec_debug_enter(inst);
+
+	for (i = 0; i < VENC_VPU_WORK_BUF_MAX; i++) {
+		if (i != VENC_VPU_WORK_BUF_SKIP_FRAME)
+			mtk_vcodec_mem_free(inst->ctx, &inst->work_bufs[i]);
+	}
+
+	mtk_vcodec_mem_free(inst->ctx, &inst->pps_buf);
+
+	mtk_vcodec_debug_leave(inst);
+}
+
+static int venc_alloc_work_buf(struct venc_common_inst *inst)
+{
+	int i;
+	int ret = 0;
+	struct venc_vpu_buf *wb = inst->vsi->work_bufs;
+
+	mtk_vcodec_debug_enter(inst);
+
+	for (i = 0; i < VENC_VPU_WORK_BUF_MAX; i++) {
+		/*
+		 * This 'wb' structure is set by VPU side and shared to AP for
+		 * buffer allocation and IO virtual addr mapping. For most of
+		 * the buffers, AP will allocate the buffer according to 'size'
+		 * field and store the IO virtual addr in 'iova' field. There
+		 * are two exceptions:
+		 * (1) RC_CODE buffer, it's pre-allocated in the VPU side, and
+		 * save the VPU addr in the 'vpua' field. The AP will translate
+		 * the VPU addr to the corresponding IO virtual addr and store
+		 * in 'iova' field for reg setting in VPU side.
+		 * (2) SKIP_FRAME buffer, it's pre-allocated in the VPU side,
+		 * and save the VPU addr in the 'vpua' field. The AP will
+		 * translate the VPU addr to the corresponding AP side virtual
+		 * address and do some memcpy access to move to bitstream buffer
+		 * assigned by v4l2 layer.
+		 */
+		inst->work_bufs[i].size = wb[i].size;
+		if (i == VENC_VPU_WORK_BUF_SKIP_FRAME) {
+			struct mtk_vcodec_fw *handler;
+
+			handler = inst->vpu_inst.ctx->dev->fw_handler;
+			inst->work_bufs[i].va =
+				mtk_vcodec_fw_map_dm_addr(handler, wb[i].vpua);
+			inst->work_bufs[i].dma_addr = 0;
+		} else {
+			ret = mtk_vcodec_mem_alloc(inst->ctx,
+						   &inst->work_bufs[i]);
+			if (ret) {
+				mtk_vcodec_err(inst,
+					       "cannot allocate buf %d", i);
+				goto err_alloc;
+			}
+			/*
+			 * This RC_CODE is pre-allocated by VPU and saved in VPU
+			 * addr. So we need use memcpy to copy RC_CODE from VPU
+			 * addr into IO virtual addr in 'iova' field for reg
+			 * setting in VPU side.
+			 */
+			if (i == VENC_VPU_WORK_BUF_RC_CODE) {
+				struct mtk_vcodec_fw *handler;
+				void *tmp_va;
+
+				handler = inst->vpu_inst.ctx->dev->fw_handler;
+				tmp_va = mtk_vcodec_fw_map_dm_addr(handler,
+								   wb[i].vpua);
+				memcpy(inst->work_bufs[i].va, tmp_va,
+				       wb[i].size);
+			}
+		}
+		wb[i].iova = inst->work_bufs[i].dma_addr;
+
+		mtk_vcodec_debug(inst,
+				 "work_buf[%d] va=0x%p iova=%pad size=%zu",
+				 i, inst->work_bufs[i].va,
+				 &inst->work_bufs[i].dma_addr,
+				 inst->work_bufs[i].size);
+	}
+
+	/* the pps_buf is used by AP side only */
+	inst->pps_buf.size = 128;
+	ret = mtk_vcodec_mem_alloc(inst->ctx, &inst->pps_buf);
+	if (ret) {
+		mtk_vcodec_err(inst, "cannot allocate pps_buf");
+		goto err_alloc;
+	}
+
+	mtk_vcodec_debug_leave(inst);
+
+	return ret;
+
+err_alloc:
+	venc_free_work_buf(inst);
+
+	return ret;
+}
+
+static unsigned int venc_wait_comp_done(struct venc_common_inst *inst,
+					unsigned int hw_id)
+{
+	unsigned int irq_status = 0;
+	struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)inst->ctx;
+
+	if (!mtk_vcodec_wait_for_done_ctx(ctx, MTK_INST_IRQ_RECEIVED,
+					  WAIT_INTR_TIMEOUT_MS)) {
+		irq_status = ctx->irq_status;
+		mtk_vcodec_debug(inst, "irq_status %x <-", irq_status);
+	}
+	return irq_status;
+}
+
+static int venc_frame_type(struct venc_common_inst *inst)
+{
+	if ((inst->vsi->config.gop_size != 0 &&
+	     (inst->frm_cnt % inst->vsi->config.gop_size) == 0) ||
+	    (inst->frm_cnt == 0 && inst->vsi->config.gop_size == 0)) {
+		/* IDR frame */
+		return VENC_IDR_FRM;
+	} else if ((inst->vsi->config.intra_period != 0 &&
+		    (inst->frm_cnt % inst->vsi->config.intra_period) == 0) ||
+		   (inst->frm_cnt == 0 &&
+		    inst->vsi->config.intra_period == 0)) {
+		/* I frame */
+		return VENC_I_FRM;
+	} else {
+		return VENC_P_FRM;  /* Note: B frames are not supported */
+	}
+}
+
+static int venc_encode_sps(struct venc_common_inst *inst,
+			   struct mtk_vcodec_mem *bs_buf,
+			   unsigned int *bs_size)
+{
+	int ret = 0;
+	unsigned int irq_status;
+
+	mtk_vcodec_debug_enter(inst);
+
+	ret = vpu_enc_encode(&inst->vpu_inst, VENC_BS_MODE_SPS, NULL,
+			     bs_buf, bs_size, NULL);
+	if (ret)
+		return ret;
+
+	irq_status = venc_wait_comp_done(inst, MTK_VENC_CORE0);
+	if (irq_status != MTK_VENC_IRQ_STATUS_SPS) {
+		mtk_vcodec_err(inst, "expect irq status %d",
+			       MTK_VENC_IRQ_STATUS_SPS);
+		return -EINVAL;
+	}
+
+	*bs_size = venc_read_reg(inst, VENC_PIC_BITSTREAM_BYTE_CNT,
+				 MTK_VENC_CORE0);
+	mtk_vcodec_debug(inst, "bs size %d <-", *bs_size);
+
+	return ret;
+}
+
+static int venc_encode_pps(struct venc_common_inst *inst,
+			   struct mtk_vcodec_mem *bs_buf,
+			   unsigned int *bs_size)
+{
+	int ret = 0;
+	unsigned int irq_status;
+
+	mtk_vcodec_debug_enter(inst);
+
+	ret = vpu_enc_encode(&inst->vpu_inst, VENC_BS_MODE_PPS, NULL,
+			     bs_buf, bs_size, NULL);
+	if (ret)
+		return ret;
+
+	irq_status = venc_wait_comp_done(inst, MTK_VENC_CORE0);
+	if (irq_status != MTK_VENC_IRQ_STATUS_PPS) {
+		mtk_vcodec_err(inst, "expect irq status %d",
+			       MTK_VENC_IRQ_STATUS_PPS);
+		return -EINVAL;
+	}
+
+	*bs_size = venc_read_reg(inst, VENC_PIC_BITSTREAM_BYTE_CNT,
+				 MTK_VENC_CORE0);
+	mtk_vcodec_debug(inst, "bs size %d <-", *bs_size);
+
+	return ret;
+}
+
+static int venc_encode_header(struct venc_common_inst *inst,
+			      struct mtk_vcodec_mem *bs_buf,
+			      unsigned int *bs_size)
+{
+	int ret = 0;
+	unsigned int bs_size_sps;
+	unsigned int bs_size_pps;
+
+	ret = venc_encode_sps(inst, bs_buf, &bs_size_sps);
+	if (ret)
+		return ret;
+
+	ret = venc_encode_pps(inst, &inst->pps_buf, &bs_size_pps);
+	if (ret)
+		return ret;
+
+	memcpy(bs_buf->va + bs_size_sps, inst->pps_buf.va, bs_size_pps);
+	*bs_size = bs_size_sps + bs_size_pps;
+
+	return ret;
+}
+
+static int venc_encode_frame(struct venc_common_inst *inst,
+			     struct venc_frm_buf *frm_buf,
+			     struct mtk_vcodec_mem *bs_buf,
+			     unsigned int *bs_size,
+			     int hw_id)
+{
+	int ret = 0;
+	struct venc_frame_info frame_info;
+
+	mtk_vcodec_debug_enter(inst);
+	mtk_vcodec_debug(inst, "frm_cnt = %d\n ", inst->frm_cnt);
+	frame_info.frm_count = inst->frm_cnt;
+	frame_info.skip_frm_count = inst->skip_frm_cnt;
+	frame_info.frm_type = venc_frame_type(inst);
+	mtk_vcodec_debug(inst, "frm_count = %d,skip_frm_count =%d,frm_type=%d.\n",
+			 frame_info.frm_count, frame_info.skip_frm_count,
+			 frame_info.frm_type);
+	ret = vpu_enc_encode(&inst->vpu_inst, VENC_BS_MODE_FRAME, frm_buf,
+			     bs_buf, bs_size, &frame_info);
+	if (ret)
+		return ret;
+
+	++inst->frm_cnt;
+	mtk_vcodec_debug(inst, "frm %d bs_size %d key_frm %d <-",
+			 inst->frm_cnt, *bs_size, inst->vpu_inst.is_key_frm);
+
+	return ret;
+}
+
+static void h264_encode_filler(struct venc_common_inst *inst, void *buf,
+			       int size)
+{
+	unsigned char *p = buf;
+
+	if (size < H264_FILLER_MARKER_SIZE) {
+		mtk_vcodec_err(inst, "filler size too small %d", size);
+		return;
+	}
+
+	memcpy(p, h264_filler_marker, ARRAY_SIZE(h264_filler_marker));
+	size -= H264_FILLER_MARKER_SIZE;
+	p += H264_FILLER_MARKER_SIZE;
+	memset(p, 0xff, size);
+}
+
+static int venc_init(struct mtk_vcodec_ctx *ctx)
+{
+	int i;
+	int ret = 0;
+	struct venc_common_inst *inst;
+
+	inst = kzalloc(sizeof(*inst), GFP_KERNEL);
+	if (!inst)
+		return -ENOMEM;
+
+	inst->ctx = ctx;
+	inst->vpu_inst.ctx = ctx;
+	inst->vpu_inst.id = SCP_IPI_VENC_H264;
+
+	mtk_vcodec_debug_enter(inst);
+
+	ret = vpu_enc_init(&inst->vpu_inst);
+
+	inst->vsi = (struct venc_vsi *)inst->vpu_inst.vsi;
+
+	for (i = 0; i < MTK_VENC_HW_MAX; i++)
+		inst->hw_base[i] = mtk_get_venc_comp_reg_addr(ctx, i);
+
+	mtk_vcodec_debug_leave(inst);
+
+	if (ret)
+		kfree(inst);
+	else
+		ctx->drv_handle = inst;
+
+	return ret;
+}
+
+static int venc_encode(void *handle,
+		       enum venc_start_opt opt,
+		       struct venc_frm_buf *frm_buf,
+		       struct mtk_vcodec_mem *bs_buf,
+		       struct venc_done_result *result)
+{
+	int ret;
+	struct venc_common_inst *inst = (struct venc_common_inst *)handle;
+	struct mtk_vcodec_ctx *ctx = inst->ctx;
+
+	switch (opt) {
+	case VENC_START_OPT_ENCODE_SEQUENCE_HEADER: {
+		unsigned int bs_size_hdr;
+
+		ret = venc_encode_header(inst, bs_buf, &bs_size_hdr);
+		if (ret) {
+			mtk_vcodec_err(inst, "encode header failed: %d", ret);
+			return ret;
+		}
+
+		result->bs_size = bs_size_hdr;
+		result->is_key_frm = false;
+
+		break;
+	}
+
+	case VENC_START_OPT_ENCODE_FRAME: {
+		int hdr_sz;
+		int hdr_sz_ext;
+		int filler_sz = 0;
+		const int bs_align = 128;
+		struct mtk_vcodec_mem tmp_bs_buf;
+		unsigned int bs_size_hdr;
+		unsigned int bs_size_frm;
+
+		if (!inst->prepend_hdr) {
+			ret = venc_encode_frame(inst, frm_buf, bs_buf,
+						&result->bs_size, ctx->hw_id);
+			if (ret) {
+				mtk_vcodec_err(inst, "encode frame failed: %d",
+					       ret);
+				return ret;
+			}
+
+			result->is_key_frm = inst->vpu_inst.is_key_frm;
+			break;
+		}
+
+		mtk_vcodec_debug(inst, "venc_encode_frame prepend SPS/PPS");
+
+		ret = venc_encode_header(inst, bs_buf, &bs_size_hdr);
+		if (ret) {
+			mtk_vcodec_err(inst, "encode prepend hdr failed: %d",
+				       ret);
+			return ret;
+		}
+
+		hdr_sz = bs_size_hdr;
+		hdr_sz_ext = (hdr_sz & (bs_align - 1));
+		if (hdr_sz_ext) {
+			filler_sz = bs_align - hdr_sz_ext;
+			if (hdr_sz_ext + H264_FILLER_MARKER_SIZE > bs_align)
+				filler_sz += bs_align;
+			h264_encode_filler(inst, bs_buf->va + hdr_sz,
+					   filler_sz);
+		}
+
+		tmp_bs_buf.va = bs_buf->va + hdr_sz + filler_sz;
+		tmp_bs_buf.dma_addr = bs_buf->dma_addr + hdr_sz + filler_sz;
+		tmp_bs_buf.size = bs_buf->size - (hdr_sz + filler_sz);
+
+		ret = venc_encode_frame(inst, frm_buf, &tmp_bs_buf,
+					&bs_size_frm, ctx->hw_id);
+
+		if (ret) {
+			mtk_vcodec_err(inst, "encode hdr frame failed: %d",
+				       ret);
+			return ret;
+		}
+
+		result->bs_size = hdr_sz + filler_sz + bs_size_frm;
+
+		mtk_vcodec_debug(inst, "hdr %d filler %d frame %d bs %d",
+				 hdr_sz, filler_sz, bs_size_frm,
+				 result->bs_size);
+
+		inst->prepend_hdr = 0;
+		result->is_key_frm = inst->vpu_inst.is_key_frm;
+		break;
+	}
+
+	default:
+		mtk_vcodec_err(inst, "venc_start_opt %d not supported", opt);
+		return -EINVAL;
+	}
+
+	return ret;
+}
+
+static int venc_set_param(void *handle,
+			  enum venc_set_param_type type,
+			  struct venc_enc_param *enc_prm)
+{
+	int ret = 0;
+	struct venc_common_inst *inst = (struct venc_common_inst *)handle;
+
+	mtk_vcodec_debug(inst, "->type=%d", type);
+
+	switch (type) {
+	case VENC_SET_PARAM_ENC:
+		inst->vsi->config.input_fourcc = enc_prm->input_yuv_fmt;
+		inst->vsi->config.bitrate = enc_prm->bitrate;
+		inst->vsi->config.pic_w = enc_prm->width;
+		inst->vsi->config.pic_h = enc_prm->height;
+		inst->vsi->config.buf_w = enc_prm->buf_width;
+		inst->vsi->config.buf_h = enc_prm->buf_height;
+		inst->vsi->config.gop_size = enc_prm->gop_size;
+		inst->vsi->config.framerate = enc_prm->frm_rate;
+		inst->vsi->config.intra_period = enc_prm->intra_period;
+		inst->vsi->config.profile =
+			h264_get_profile(inst, enc_prm->h264_profile);
+		inst->vsi->config.level =
+			h264_get_level(inst, enc_prm->h264_level);
+		inst->vsi->config.wfd = 0;
+
+		ret = vpu_enc_set_param(&inst->vpu_inst, type, enc_prm);
+		if (ret)
+			break;
+		if (inst->work_buf_allocated) {
+			venc_free_work_buf(inst);
+			inst->work_buf_allocated = false;
+		}
+		ret = venc_alloc_work_buf(inst);
+		if (ret)
+			break;
+		inst->work_buf_allocated = true;
+		break;
+
+	case VENC_SET_PARAM_PREPEND_HEADER:
+		inst->prepend_hdr = 1;
+		mtk_vcodec_debug(inst, "set prepend header mode");
+		break;
+	case VENC_SET_PARAM_FORCE_INTRA:
+	case VENC_SET_PARAM_GOP_SIZE:
+	case VENC_SET_PARAM_INTRA_PERIOD:
+		inst->frm_cnt = 0;
+		inst->skip_frm_cnt = 0;
+		fallthrough;
+	default:
+		ret = vpu_enc_set_param(&inst->vpu_inst, type, enc_prm);
+		break;
+	}
+
+	mtk_vcodec_debug_leave(inst);
+	return ret;
+}
+
+static int venc_deinit(void *handle)
+{
+	int ret = 0;
+	struct venc_common_inst *inst = (struct venc_common_inst *)handle;
+
+	mtk_vcodec_debug_enter(inst);
+
+	ret = vpu_enc_deinit(&inst->vpu_inst);
+
+	if (inst->work_buf_allocated)
+		venc_free_work_buf(inst);
+
+	mtk_vcodec_debug_leave(inst);
+	kfree(inst);
+
+	return ret;
+}
+
+const struct venc_common_if venc_if = {
+	.init = venc_init,
+	.encode = venc_encode,
+	.set_param = venc_set_param,
+	.deinit = venc_deinit,
+};
diff --git a/drivers/media/platform/mtk-vcodec/venc_drv_if.c b/drivers/media/platform/mtk-vcodec/venc_drv_if.c
index 6cbdb7e30bb3..93936d002bed 100644
--- a/drivers/media/platform/mtk-vcodec/venc_drv_if.c
+++ b/drivers/media/platform/mtk-vcodec/venc_drv_if.c
@@ -25,7 +25,10 @@ int venc_if_init(struct mtk_vcodec_ctx *ctx, unsigned int fourcc)
 		ctx->enc_if = &venc_vp8_if;
 		break;
 	case V4L2_PIX_FMT_H264:
-		ctx->enc_if = &venc_h264_if;
+		if (MTK_ENC_HW_MODE(ctx) == VENC_FRAME_RACING_MODE)
+			ctx->enc_if = &venc_if;
+		else
+			ctx->enc_if = &venc_h264_if;
 		break;
 	default:
 		return -EINVAL;
diff --git a/drivers/media/platform/mtk-vcodec/venc_drv_if.h b/drivers/media/platform/mtk-vcodec/venc_drv_if.h
index 0b04a1020873..8f384ba24ddf 100644
--- a/drivers/media/platform/mtk-vcodec/venc_drv_if.h
+++ b/drivers/media/platform/mtk-vcodec/venc_drv_if.h
@@ -125,6 +125,7 @@ struct venc_done_result {
 
 extern const struct venc_common_if venc_h264_if;
 extern const struct venc_common_if venc_vp8_if;
+extern const struct venc_common_if venc_if;
 
 /*
  * venc_if_init - Create the driver handle
-- 
2.25.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH 6/9] media: mtk-vcodec: Add new venc drv interface for frame_racing mode

[Irui Wang]

Frame-racing mode encoding need more venc working buffers, it
will break the compatibility if we just add venc_vsi in AP-Kernel
but not in firmware, so add a new venc driver interface to
distinguish the sigle_core_mode and frame_racing mode.

The new driver interface can be used for different codecs in
the future.

Signed-off-by: Irui Wang <irui.wang@mediatek.com>
---
 drivers/media/platform/mtk-vcodec/Makefile    |   1 +
 .../platform/mtk-vcodec/mtk_vcodec_drv.h      |   1 +
 .../platform/mtk-vcodec/mtk_vcodec_util.c     |  19 +
 .../platform/mtk-vcodec/mtk_vcodec_util.h     |   4 +
 .../platform/mtk-vcodec/venc/venc_common_if.c | 629 ++++++++++++++++++
 .../media/platform/mtk-vcodec/venc_drv_if.c   |   5 +-
 .../media/platform/mtk-vcodec/venc_drv_if.h   |   1 +
 7 files changed, 659 insertions(+), 1 deletion(-)
 create mode 100644 drivers/media/platform/mtk-vcodec/venc/venc_common_if.c

diff --git a/drivers/media/platform/mtk-vcodec/Makefile b/drivers/media/platform/mtk-vcodec/Makefile
index 661d4afeb628..a42ce704bf04 100644
--- a/drivers/media/platform/mtk-vcodec/Makefile
+++ b/drivers/media/platform/mtk-vcodec/Makefile
@@ -15,6 +15,7 @@ mtk-vcodec-dec-y := vdec/vdec_h264_if.o \
 
 mtk-vcodec-enc-y := venc/venc_vp8_if.o \
 		venc/venc_h264_if.o \
+		venc/venc_common_if.o \
 		mtk_vcodec_enc.o \
 		mtk_vcodec_enc_drv.o \
 		mtk_vcodec_enc_pm.o \
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
index c160b9e505eb..1c6c1eca649f 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
@@ -352,6 +352,7 @@ struct mtk_vcodec_enc_pdata {
 };
 
 #define MTK_ENC_CTX_IS_EXT(ctx) ((ctx)->dev->venc_pdata->uses_ext)
+#define MTK_ENC_HW_MODE(ctx) ((ctx)->dev->venc_pdata->hw_mode)
 
 /**
  * struct mtk_vcodec_dev - driver data
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c
index ac5973b6735f..5057e2cba627 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c
@@ -9,6 +9,7 @@
 
 #include "mtk_vcodec_drv.h"
 #include "mtk_vcodec_util.h"
+#include "mtk_vcodec_enc_hw.h"
 
 /* For encoder, this will enable logs in venc/*/
 bool mtk_vcodec_dbg;
@@ -33,6 +34,24 @@ void __iomem *mtk_vcodec_get_reg_addr(struct mtk_vcodec_ctx *data,
 }
 EXPORT_SYMBOL(mtk_vcodec_get_reg_addr);
 
+void __iomem *mtk_get_venc_comp_reg_addr(struct mtk_vcodec_ctx *data,
+					 unsigned int hw_id)
+{
+	struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)data;
+	struct mtk_venc_comp_dev *venc;
+
+	if (!data || hw_id >= MTK_VENC_HW_MAX) {
+		mtk_v4l2_err("Invalid arguments, hw_id=%d", hw_id);
+		return NULL;
+	}
+	venc = (struct mtk_venc_comp_dev *)ctx->dev->enc_comp_dev[hw_id];
+	if (!venc)
+		return NULL;
+
+	return venc->reg_base;
+}
+EXPORT_SYMBOL(mtk_get_venc_comp_reg_addr);
+
 int mtk_vcodec_mem_alloc(struct mtk_vcodec_ctx *data,
 			struct mtk_vcodec_mem *mem)
 {
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
index b999d7b84ed1..0027747c0a27 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
@@ -75,6 +75,10 @@ extern bool mtk_vcodec_dbg;
 
 void __iomem *mtk_vcodec_get_reg_addr(struct mtk_vcodec_ctx *data,
 				unsigned int reg_idx);
+
+void __iomem *mtk_get_venc_comp_reg_addr(struct mtk_vcodec_ctx *data,
+					 unsigned int hw_id);
+
 int mtk_vcodec_mem_alloc(struct mtk_vcodec_ctx *data,
 				struct mtk_vcodec_mem *mem);
 void mtk_vcodec_mem_free(struct mtk_vcodec_ctx *data,
diff --git a/drivers/media/platform/mtk-vcodec/venc/venc_common_if.c b/drivers/media/platform/mtk-vcodec/venc/venc_common_if.c
new file mode 100644
index 000000000000..01b7d93241af
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/venc/venc_common_if.c
@@ -0,0 +1,629 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2021 MediaTek Inc.
+ */
+
+#include "../venc_vpu_if.h"
+#include "../venc_drv_base.h"
+#include "../mtk_vcodec_intr.h"
+#include "../mtk_vcodec_enc.h"
+#include "../mtk_vcodec_drv.h"
+#include "../mtk_vcodec_util.h"
+#include "../mtk_vcodec_enc_hw.h"
+
+static const char h264_filler_marker[] = {0x0, 0x0, 0x0, 0x1, 0xc};
+
+#define H264_FILLER_MARKER_SIZE ARRAY_SIZE(h264_filler_marker)
+#define VENC_PIC_BITSTREAM_BYTE_CNT 0x0098
+
+enum venc_vpu_work_buf {
+	VENC_VPU_WORK_BUF_RC_INFO_CORE0,
+	VENC_VPU_WORK_BUF_RC_INFO_CORE1,
+	VENC_VPU_WORK_BUF_FR_RC_INFO,
+	VENC_VPU_WORK_BUF_RC_CODE,
+	VENC_VPU_WORK_BUF_REC_LUMA,
+	VENC_VPU_WORK_BUF_REC_CHROMA,
+	VENC_VPU_WORK_BUF_REF_LUMA,
+	VENC_VPU_WORK_BUF_REF_CHROMA,
+	VENC_VPU_WORK_BUF_MV_INFO_1,
+	VENC_VPU_WORK_BUF_MV_INFO_2,
+	VENC_VPU_WORK_BUF_SKIP_FRAME,
+	VENC_VPU_WORK_BUF_MAX,
+};
+
+enum venc_frame_type {
+	VENC_IDR_FRM,
+	VENC_I_FRM,
+	VENC_P_FRM,
+	VENC_B_FRM,
+};
+
+enum venc_bs_mode {
+	VENC_BS_MODE_SPS,
+	VENC_BS_MODE_PPS,
+	VENC_BS_MODE_FRAME,
+};
+
+struct venc_vpu_buf {
+	u32 iova;
+	u32 vpua;
+	u32 size;
+};
+
+struct venc_vpu_config {
+	u32 input_fourcc;
+	u32 bitrate;
+	u32 pic_w;
+	u32 pic_h;
+	u32 buf_w;
+	u32 buf_h;
+	u32 gop_size;
+	u32 intra_period;
+	u32 framerate;
+	u32 profile;
+	u32 level;
+	u32 wfd;
+	u32 max_qp;
+	u32 min_qp;
+	u32 reserved[8];
+};
+
+struct venc_vsi {
+	struct venc_vpu_config config;
+	struct venc_vpu_buf work_bufs[VENC_VPU_WORK_BUF_MAX];
+};
+
+struct venc_common_inst {
+	void __iomem *hw_base[MTK_VENC_HW_MAX];
+	struct mtk_vcodec_mem work_bufs[VENC_VPU_WORK_BUF_MAX];
+	struct mtk_vcodec_mem pps_buf;
+	bool work_buf_allocated;
+	unsigned int frm_cnt;
+	unsigned int skip_frm_cnt;
+	unsigned int prepend_hdr;
+	struct venc_vpu_inst vpu_inst;
+	struct venc_vsi *vsi;
+	struct mtk_vcodec_ctx *ctx;
+};
+
+static inline u32 venc_read_reg(struct venc_common_inst *inst,
+				u32 addr, int hw_id)
+{
+	return readl(inst->hw_base[hw_id] + addr);
+}
+
+static unsigned int h264_get_profile(struct venc_common_inst *inst,
+				     unsigned int profile)
+{
+	switch (profile) {
+	case V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE:
+		return 66;
+	case V4L2_MPEG_VIDEO_H264_PROFILE_MAIN:
+		return 77;
+	case V4L2_MPEG_VIDEO_H264_PROFILE_HIGH:
+		return 100;
+	case V4L2_MPEG_VIDEO_H264_PROFILE_CONSTRAINED_BASELINE:
+		mtk_vcodec_err(inst, "unsupported CONSTRAINED_BASELINE");
+		return 0;
+	case V4L2_MPEG_VIDEO_H264_PROFILE_EXTENDED:
+		mtk_vcodec_err(inst, "unsupported EXTENDED");
+		return 0;
+	default:
+		mtk_vcodec_debug(inst, "unsupported profile %d", profile);
+		return 100;
+	}
+}
+
+static unsigned int h264_get_level(struct venc_common_inst *inst,
+				   unsigned int level)
+{
+	switch (level) {
+	case V4L2_MPEG_VIDEO_H264_LEVEL_1B:
+		mtk_vcodec_err(inst, "unsupported 1B");
+		return 0;
+	case V4L2_MPEG_VIDEO_H264_LEVEL_1_0:
+		return 10;
+	case V4L2_MPEG_VIDEO_H264_LEVEL_1_1:
+		return 11;
+	case V4L2_MPEG_VIDEO_H264_LEVEL_1_2:
+		return 12;
+	case V4L2_MPEG_VIDEO_H264_LEVEL_1_3:
+		return 13;
+	case V4L2_MPEG_VIDEO_H264_LEVEL_2_0:
+		return 20;
+	case V4L2_MPEG_VIDEO_H264_LEVEL_2_1:
+		return 21;
+	case V4L2_MPEG_VIDEO_H264_LEVEL_2_2:
+		return 22;
+	case V4L2_MPEG_VIDEO_H264_LEVEL_3_0:
+		return 30;
+	case V4L2_MPEG_VIDEO_H264_LEVEL_3_1:
+		return 31;
+	case V4L2_MPEG_VIDEO_H264_LEVEL_3_2:
+		return 32;
+	case V4L2_MPEG_VIDEO_H264_LEVEL_4_0:
+		return 40;
+	case V4L2_MPEG_VIDEO_H264_LEVEL_4_1:
+		return 41;
+	case V4L2_MPEG_VIDEO_H264_LEVEL_4_2:
+		return 42;
+	case V4L2_MPEG_VIDEO_H264_LEVEL_5_0:
+		return 50;
+	case V4L2_MPEG_VIDEO_H264_LEVEL_5_1:
+		return 51;
+	default:
+		mtk_vcodec_debug(inst, "unsupported level %d", level);
+		return 31;
+	}
+}
+
+static void venc_free_work_buf(struct venc_common_inst *inst)
+{
+	int i;
+
+	mtk_vcodec_debug_enter(inst);
+
+	for (i = 0; i < VENC_VPU_WORK_BUF_MAX; i++) {
+		if (i != VENC_VPU_WORK_BUF_SKIP_FRAME)
+			mtk_vcodec_mem_free(inst->ctx, &inst->work_bufs[i]);
+	}
+
+	mtk_vcodec_mem_free(inst->ctx, &inst->pps_buf);
+
+	mtk_vcodec_debug_leave(inst);
+}
+
+static int venc_alloc_work_buf(struct venc_common_inst *inst)
+{
+	int i;
+	int ret = 0;
+	struct venc_vpu_buf *wb = inst->vsi->work_bufs;
+
+	mtk_vcodec_debug_enter(inst);
+
+	for (i = 0; i < VENC_VPU_WORK_BUF_MAX; i++) {
+		/*
+		 * This 'wb' structure is set by VPU side and shared to AP for
+		 * buffer allocation and IO virtual addr mapping. For most of
+		 * the buffers, AP will allocate the buffer according to 'size'
+		 * field and store the IO virtual addr in 'iova' field. There
+		 * are two exceptions:
+		 * (1) RC_CODE buffer, it's pre-allocated in the VPU side, and
+		 * save the VPU addr in the 'vpua' field. The AP will translate
+		 * the VPU addr to the corresponding IO virtual addr and store
+		 * in 'iova' field for reg setting in VPU side.
+		 * (2) SKIP_FRAME buffer, it's pre-allocated in the VPU side,
+		 * and save the VPU addr in the 'vpua' field. The AP will
+		 * translate the VPU addr to the corresponding AP side virtual
+		 * address and do some memcpy access to move to bitstream buffer
+		 * assigned by v4l2 layer.
+		 */
+		inst->work_bufs[i].size = wb[i].size;
+		if (i == VENC_VPU_WORK_BUF_SKIP_FRAME) {
+			struct mtk_vcodec_fw *handler;
+
+			handler = inst->vpu_inst.ctx->dev->fw_handler;
+			inst->work_bufs[i].va =
+				mtk_vcodec_fw_map_dm_addr(handler, wb[i].vpua);
+			inst->work_bufs[i].dma_addr = 0;
+		} else {
+			ret = mtk_vcodec_mem_alloc(inst->ctx,
+						   &inst->work_bufs[i]);
+			if (ret) {
+				mtk_vcodec_err(inst,
+					       "cannot allocate buf %d", i);
+				goto err_alloc;
+			}
+			/*
+			 * This RC_CODE is pre-allocated by VPU and saved in VPU
+			 * addr. So we need use memcpy to copy RC_CODE from VPU
+			 * addr into IO virtual addr in 'iova' field for reg
+			 * setting in VPU side.
+			 */
+			if (i == VENC_VPU_WORK_BUF_RC_CODE) {
+				struct mtk_vcodec_fw *handler;
+				void *tmp_va;
+
+				handler = inst->vpu_inst.ctx->dev->fw_handler;
+				tmp_va = mtk_vcodec_fw_map_dm_addr(handler,
+								   wb[i].vpua);
+				memcpy(inst->work_bufs[i].va, tmp_va,
+				       wb[i].size);
+			}
+		}
+		wb[i].iova = inst->work_bufs[i].dma_addr;
+
+		mtk_vcodec_debug(inst,
+				 "work_buf[%d] va=0x%p iova=%pad size=%zu",
+				 i, inst->work_bufs[i].va,
+				 &inst->work_bufs[i].dma_addr,
+				 inst->work_bufs[i].size);
+	}
+
+	/* the pps_buf is used by AP side only */
+	inst->pps_buf.size = 128;
+	ret = mtk_vcodec_mem_alloc(inst->ctx, &inst->pps_buf);
+	if (ret) {
+		mtk_vcodec_err(inst, "cannot allocate pps_buf");
+		goto err_alloc;
+	}
+
+	mtk_vcodec_debug_leave(inst);
+
+	return ret;
+
+err_alloc:
+	venc_free_work_buf(inst);
+
+	return ret;
+}
+
+static unsigned int venc_wait_comp_done(struct venc_common_inst *inst,
+					unsigned int hw_id)
+{
+	unsigned int irq_status = 0;
+	struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)inst->ctx;
+
+	if (!mtk_vcodec_wait_for_done_ctx(ctx, MTK_INST_IRQ_RECEIVED,
+					  WAIT_INTR_TIMEOUT_MS)) {
+		irq_status = ctx->irq_status;
+		mtk_vcodec_debug(inst, "irq_status %x <-", irq_status);
+	}
+	return irq_status;
+}
+
+static int venc_frame_type(struct venc_common_inst *inst)
+{
+	if ((inst->vsi->config.gop_size != 0 &&
+	     (inst->frm_cnt % inst->vsi->config.gop_size) == 0) ||
+	    (inst->frm_cnt == 0 && inst->vsi->config.gop_size == 0)) {
+		/* IDR frame */
+		return VENC_IDR_FRM;
+	} else if ((inst->vsi->config.intra_period != 0 &&
+		    (inst->frm_cnt % inst->vsi->config.intra_period) == 0) ||
+		   (inst->frm_cnt == 0 &&
+		    inst->vsi->config.intra_period == 0)) {
+		/* I frame */
+		return VENC_I_FRM;
+	} else {
+		return VENC_P_FRM;  /* Note: B frames are not supported */
+	}
+}
+
+static int venc_encode_sps(struct venc_common_inst *inst,
+			   struct mtk_vcodec_mem *bs_buf,
+			   unsigned int *bs_size)
+{
+	int ret = 0;
+	unsigned int irq_status;
+
+	mtk_vcodec_debug_enter(inst);
+
+	ret = vpu_enc_encode(&inst->vpu_inst, VENC_BS_MODE_SPS, NULL,
+			     bs_buf, bs_size, NULL);
+	if (ret)
+		return ret;
+
+	irq_status = venc_wait_comp_done(inst, MTK_VENC_CORE0);
+	if (irq_status != MTK_VENC_IRQ_STATUS_SPS) {
+		mtk_vcodec_err(inst, "expect irq status %d",
+			       MTK_VENC_IRQ_STATUS_SPS);
+		return -EINVAL;
+	}
+
+	*bs_size = venc_read_reg(inst, VENC_PIC_BITSTREAM_BYTE_CNT,
+				 MTK_VENC_CORE0);
+	mtk_vcodec_debug(inst, "bs size %d <-", *bs_size);
+
+	return ret;
+}
+
+static int venc_encode_pps(struct venc_common_inst *inst,
+			   struct mtk_vcodec_mem *bs_buf,
+			   unsigned int *bs_size)
+{
+	int ret = 0;
+	unsigned int irq_status;
+
+	mtk_vcodec_debug_enter(inst);
+
+	ret = vpu_enc_encode(&inst->vpu_inst, VENC_BS_MODE_PPS, NULL,
+			     bs_buf, bs_size, NULL);
+	if (ret)
+		return ret;
+
+	irq_status = venc_wait_comp_done(inst, MTK_VENC_CORE0);
+	if (irq_status != MTK_VENC_IRQ_STATUS_PPS) {
+		mtk_vcodec_err(inst, "expect irq status %d",
+			       MTK_VENC_IRQ_STATUS_PPS);
+		return -EINVAL;
+	}
+
+	*bs_size = venc_read_reg(inst, VENC_PIC_BITSTREAM_BYTE_CNT,
+				 MTK_VENC_CORE0);
+	mtk_vcodec_debug(inst, "bs size %d <-", *bs_size);
+
+	return ret;
+}
+
+static int venc_encode_header(struct venc_common_inst *inst,
+			      struct mtk_vcodec_mem *bs_buf,
+			      unsigned int *bs_size)
+{
+	int ret = 0;
+	unsigned int bs_size_sps;
+	unsigned int bs_size_pps;
+
+	ret = venc_encode_sps(inst, bs_buf, &bs_size_sps);
+	if (ret)
+		return ret;
+
+	ret = venc_encode_pps(inst, &inst->pps_buf, &bs_size_pps);
+	if (ret)
+		return ret;
+
+	memcpy(bs_buf->va + bs_size_sps, inst->pps_buf.va, bs_size_pps);
+	*bs_size = bs_size_sps + bs_size_pps;
+
+	return ret;
+}
+
+static int venc_encode_frame(struct venc_common_inst *inst,
+			     struct venc_frm_buf *frm_buf,
+			     struct mtk_vcodec_mem *bs_buf,
+			     unsigned int *bs_size,
+			     int hw_id)
+{
+	int ret = 0;
+	struct venc_frame_info frame_info;
+
+	mtk_vcodec_debug_enter(inst);
+	mtk_vcodec_debug(inst, "frm_cnt = %d\n ", inst->frm_cnt);
+	frame_info.frm_count = inst->frm_cnt;
+	frame_info.skip_frm_count = inst->skip_frm_cnt;
+	frame_info.frm_type = venc_frame_type(inst);
+	mtk_vcodec_debug(inst, "frm_count = %d,skip_frm_count =%d,frm_type=%d.\n",
+			 frame_info.frm_count, frame_info.skip_frm_count,
+			 frame_info.frm_type);
+	ret = vpu_enc_encode(&inst->vpu_inst, VENC_BS_MODE_FRAME, frm_buf,
+			     bs_buf, bs_size, &frame_info);
+	if (ret)
+		return ret;
+
+	++inst->frm_cnt;
+	mtk_vcodec_debug(inst, "frm %d bs_size %d key_frm %d <-",
+			 inst->frm_cnt, *bs_size, inst->vpu_inst.is_key_frm);
+
+	return ret;
+}
+
+static void h264_encode_filler(struct venc_common_inst *inst, void *buf,
+			       int size)
+{
+	unsigned char *p = buf;
+
+	if (size < H264_FILLER_MARKER_SIZE) {
+		mtk_vcodec_err(inst, "filler size too small %d", size);
+		return;
+	}
+
+	memcpy(p, h264_filler_marker, ARRAY_SIZE(h264_filler_marker));
+	size -= H264_FILLER_MARKER_SIZE;
+	p += H264_FILLER_MARKER_SIZE;
+	memset(p, 0xff, size);
+}
+
+static int venc_init(struct mtk_vcodec_ctx *ctx)
+{
+	int i;
+	int ret = 0;
+	struct venc_common_inst *inst;
+
+	inst = kzalloc(sizeof(*inst), GFP_KERNEL);
+	if (!inst)
+		return -ENOMEM;
+
+	inst->ctx = ctx;
+	inst->vpu_inst.ctx = ctx;
+	inst->vpu_inst.id = SCP_IPI_VENC_H264;
+
+	mtk_vcodec_debug_enter(inst);
+
+	ret = vpu_enc_init(&inst->vpu_inst);
+
+	inst->vsi = (struct venc_vsi *)inst->vpu_inst.vsi;
+
+	for (i = 0; i < MTK_VENC_HW_MAX; i++)
+		inst->hw_base[i] = mtk_get_venc_comp_reg_addr(ctx, i);
+
+	mtk_vcodec_debug_leave(inst);
+
+	if (ret)
+		kfree(inst);
+	else
+		ctx->drv_handle = inst;
+
+	return ret;
+}
+
+static int venc_encode(void *handle,
+		       enum venc_start_opt opt,
+		       struct venc_frm_buf *frm_buf,
+		       struct mtk_vcodec_mem *bs_buf,
+		       struct venc_done_result *result)
+{
+	int ret;
+	struct venc_common_inst *inst = (struct venc_common_inst *)handle;
+	struct mtk_vcodec_ctx *ctx = inst->ctx;
+
+	switch (opt) {
+	case VENC_START_OPT_ENCODE_SEQUENCE_HEADER: {
+		unsigned int bs_size_hdr;
+
+		ret = venc_encode_header(inst, bs_buf, &bs_size_hdr);
+		if (ret) {
+			mtk_vcodec_err(inst, "encode header failed: %d", ret);
+			return ret;
+		}
+
+		result->bs_size = bs_size_hdr;
+		result->is_key_frm = false;
+
+		break;
+	}
+
+	case VENC_START_OPT_ENCODE_FRAME: {
+		int hdr_sz;
+		int hdr_sz_ext;
+		int filler_sz = 0;
+		const int bs_align = 128;
+		struct mtk_vcodec_mem tmp_bs_buf;
+		unsigned int bs_size_hdr;
+		unsigned int bs_size_frm;
+
+		if (!inst->prepend_hdr) {
+			ret = venc_encode_frame(inst, frm_buf, bs_buf,
+						&result->bs_size, ctx->hw_id);
+			if (ret) {
+				mtk_vcodec_err(inst, "encode frame failed: %d",
+					       ret);
+				return ret;
+			}
+
+			result->is_key_frm = inst->vpu_inst.is_key_frm;
+			break;
+		}
+
+		mtk_vcodec_debug(inst, "venc_encode_frame prepend SPS/PPS");
+
+		ret = venc_encode_header(inst, bs_buf, &bs_size_hdr);
+		if (ret) {
+			mtk_vcodec_err(inst, "encode prepend hdr failed: %d",
+				       ret);
+			return ret;
+		}
+
+		hdr_sz = bs_size_hdr;
+		hdr_sz_ext = (hdr_sz & (bs_align - 1));
+		if (hdr_sz_ext) {
+			filler_sz = bs_align - hdr_sz_ext;
+			if (hdr_sz_ext + H264_FILLER_MARKER_SIZE > bs_align)
+				filler_sz += bs_align;
+			h264_encode_filler(inst, bs_buf->va + hdr_sz,
+					   filler_sz);
+		}
+
+		tmp_bs_buf.va = bs_buf->va + hdr_sz + filler_sz;
+		tmp_bs_buf.dma_addr = bs_buf->dma_addr + hdr_sz + filler_sz;
+		tmp_bs_buf.size = bs_buf->size - (hdr_sz + filler_sz);
+
+		ret = venc_encode_frame(inst, frm_buf, &tmp_bs_buf,
+					&bs_size_frm, ctx->hw_id);
+
+		if (ret) {
+			mtk_vcodec_err(inst, "encode hdr frame failed: %d",
+				       ret);
+			return ret;
+		}
+
+		result->bs_size = hdr_sz + filler_sz + bs_size_frm;
+
+		mtk_vcodec_debug(inst, "hdr %d filler %d frame %d bs %d",
+				 hdr_sz, filler_sz, bs_size_frm,
+				 result->bs_size);
+
+		inst->prepend_hdr = 0;
+		result->is_key_frm = inst->vpu_inst.is_key_frm;
+		break;
+	}
+
+	default:
+		mtk_vcodec_err(inst, "venc_start_opt %d not supported", opt);
+		return -EINVAL;
+	}
+
+	return ret;
+}
+
+static int venc_set_param(void *handle,
+			  enum venc_set_param_type type,
+			  struct venc_enc_param *enc_prm)
+{
+	int ret = 0;
+	struct venc_common_inst *inst = (struct venc_common_inst *)handle;
+
+	mtk_vcodec_debug(inst, "->type=%d", type);
+
+	switch (type) {
+	case VENC_SET_PARAM_ENC:
+		inst->vsi->config.input_fourcc = enc_prm->input_yuv_fmt;
+		inst->vsi->config.bitrate = enc_prm->bitrate;
+		inst->vsi->config.pic_w = enc_prm->width;
+		inst->vsi->config.pic_h = enc_prm->height;
+		inst->vsi->config.buf_w = enc_prm->buf_width;
+		inst->vsi->config.buf_h = enc_prm->buf_height;
+		inst->vsi->config.gop_size = enc_prm->gop_size;
+		inst->vsi->config.framerate = enc_prm->frm_rate;
+		inst->vsi->config.intra_period = enc_prm->intra_period;
+		inst->vsi->config.profile =
+			h264_get_profile(inst, enc_prm->h264_profile);
+		inst->vsi->config.level =
+			h264_get_level(inst, enc_prm->h264_level);
+		inst->vsi->config.wfd = 0;
+
+		ret = vpu_enc_set_param(&inst->vpu_inst, type, enc_prm);
+		if (ret)
+			break;
+		if (inst->work_buf_allocated) {
+			venc_free_work_buf(inst);
+			inst->work_buf_allocated = false;
+		}
+		ret = venc_alloc_work_buf(inst);
+		if (ret)
+			break;
+		inst->work_buf_allocated = true;
+		break;
+
+	case VENC_SET_PARAM_PREPEND_HEADER:
+		inst->prepend_hdr = 1;
+		mtk_vcodec_debug(inst, "set prepend header mode");
+		break;
+	case VENC_SET_PARAM_FORCE_INTRA:
+	case VENC_SET_PARAM_GOP_SIZE:
+	case VENC_SET_PARAM_INTRA_PERIOD:
+		inst->frm_cnt = 0;
+		inst->skip_frm_cnt = 0;
+		fallthrough;
+	default:
+		ret = vpu_enc_set_param(&inst->vpu_inst, type, enc_prm);
+		break;
+	}
+
+	mtk_vcodec_debug_leave(inst);
+	return ret;
+}
+
+static int venc_deinit(void *handle)
+{
+	int ret = 0;
+	struct venc_common_inst *inst = (struct venc_common_inst *)handle;
+
+	mtk_vcodec_debug_enter(inst);
+
+	ret = vpu_enc_deinit(&inst->vpu_inst);
+
+	if (inst->work_buf_allocated)
+		venc_free_work_buf(inst);
+
+	mtk_vcodec_debug_leave(inst);
+	kfree(inst);
+
+	return ret;
+}
+
+const struct venc_common_if venc_if = {
+	.init = venc_init,
+	.encode = venc_encode,
+	.set_param = venc_set_param,
+	.deinit = venc_deinit,
+};
diff --git a/drivers/media/platform/mtk-vcodec/venc_drv_if.c b/drivers/media/platform/mtk-vcodec/venc_drv_if.c
index 6cbdb7e30bb3..93936d002bed 100644
--- a/drivers/media/platform/mtk-vcodec/venc_drv_if.c
+++ b/drivers/media/platform/mtk-vcodec/venc_drv_if.c
@@ -25,7 +25,10 @@ int venc_if_init(struct mtk_vcodec_ctx *ctx, unsigned int fourcc)
 		ctx->enc_if = &venc_vp8_if;
 		break;
 	case V4L2_PIX_FMT_H264:
-		ctx->enc_if = &venc_h264_if;
+		if (MTK_ENC_HW_MODE(ctx) == VENC_FRAME_RACING_MODE)
+			ctx->enc_if = &venc_if;
+		else
+			ctx->enc_if = &venc_h264_if;
 		break;
 	default:
 		return -EINVAL;
diff --git a/drivers/media/platform/mtk-vcodec/venc_drv_if.h b/drivers/media/platform/mtk-vcodec/venc_drv_if.h
index 0b04a1020873..8f384ba24ddf 100644
--- a/drivers/media/platform/mtk-vcodec/venc_drv_if.h
+++ b/drivers/media/platform/mtk-vcodec/venc_drv_if.h
@@ -125,6 +125,7 @@ struct venc_done_result {
 
 extern const struct venc_common_if venc_h264_if;
 extern const struct venc_common_if venc_vp8_if;
+extern const struct venc_common_if venc_if;
 
 /*
  * venc_if_init - Create the driver handle
-- 
2.25.1
_______________________________________________
Linux-mediatek mailing list
Linux-mediatek@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-mediatek

Re: [PATCH 6/9] media: mtk-vcodec: Add new venc drv interface for frame_racing mode

[Tzung-Bi Shih]

On Mon, Aug 16, 2021 at 06:59:31PM +0800, Irui Wang wrote:
> Frame-racing mode encoding need more venc working buffers, it
> will break the compatibility if we just add venc_vsi in AP-Kernel
> but not in firmware, so add a new venc driver interface to
> distinguish the sigle_core_mode and frame_racing mode.
> 
> The new driver interface can be used for different codecs in
> the future.
The patch contains more code than the commit message explains that I couldn't understand.  Only find some nits.

> --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c
> +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c
> @@ -9,6 +9,7 @@
>  
>  #include "mtk_vcodec_drv.h"
>  #include "mtk_vcodec_util.h"
> +#include "mtk_vcodec_enc_hw.h"
Please try to maintain the order.

> +void __iomem *mtk_get_venc_comp_reg_addr(struct mtk_vcodec_ctx *data,
> +					 unsigned int hw_id)
> +{
> +	struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)data;
typeof(ctx) == typeof(data), rename the function argument 'data' to 'ctx'.

> +static int venc_alloc_work_buf(struct venc_common_inst *inst)
> +{
> +	int i;
> +	int ret = 0;
It doesn't need to be initialized.  See comment below.

> +	mtk_vcodec_debug_leave(inst);
> +
> +	return ret;
It is more clear if just return 0 here.  In the meantime, ret doesn't need to be initialized.

> +static unsigned int venc_wait_comp_done(struct venc_common_inst *inst,
> +					unsigned int hw_id)
> +{
> +	unsigned int irq_status = 0;
> +	struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)inst->ctx;
No need to cast.

> +static int venc_encode_sps(struct venc_common_inst *inst,
> +			   struct mtk_vcodec_mem *bs_buf,
> +			   unsigned int *bs_size)
> +{
> +	int ret = 0;
Same.

> +	*bs_size = venc_read_reg(inst, VENC_PIC_BITSTREAM_BYTE_CNT,
> +				 MTK_VENC_CORE0);
> +	mtk_vcodec_debug(inst, "bs size %d <-", *bs_size);
> +
> +	return ret;
Same.

> +static int venc_encode_pps(struct venc_common_inst *inst,
> +			   struct mtk_vcodec_mem *bs_buf,
> +			   unsigned int *bs_size)
> +{
> +	int ret = 0;
Same.

> +	*bs_size = venc_read_reg(inst, VENC_PIC_BITSTREAM_BYTE_CNT,
> +				 MTK_VENC_CORE0);
> +	mtk_vcodec_debug(inst, "bs size %d <-", *bs_size);
> +
> +	return ret;
Same.

> +static int venc_encode_header(struct venc_common_inst *inst,
> +			      struct mtk_vcodec_mem *bs_buf,
> +			      unsigned int *bs_size)
> +{
> +	int ret = 0;
Same.

> +	memcpy(bs_buf->va + bs_size_sps, inst->pps_buf.va, bs_size_pps);
> +	*bs_size = bs_size_sps + bs_size_pps;
> +
> +	return ret;
Same.

> +static int venc_encode_frame(struct venc_common_inst *inst,
> +			     struct venc_frm_buf *frm_buf,
> +			     struct mtk_vcodec_mem *bs_buf,
> +			     unsigned int *bs_size,
> +			     int hw_id)
> +{
> +	int ret = 0;
Same.

> +	++inst->frm_cnt;
> +	mtk_vcodec_debug(inst, "frm %d bs_size %d key_frm %d <-",
> +			 inst->frm_cnt, *bs_size, inst->vpu_inst.is_key_frm);
> +
> +	return ret;
Same.

> +static void h264_encode_filler(struct venc_common_inst *inst, void *buf,
> +			       int size)
> +{
> +	unsigned char *p = buf;
> +
> +	if (size < H264_FILLER_MARKER_SIZE) {
> +		mtk_vcodec_err(inst, "filler size too small %d", size);
> +		return;
> +	}
> +
> +	memcpy(p, h264_filler_marker, ARRAY_SIZE(h264_filler_marker));
Replace ARRAY_SIZE(h264_filler_marker) to H264_FILLER_MARKER_SIZE.

> +static int venc_init(struct mtk_vcodec_ctx *ctx)
> +{
> +	int i;
> +	int ret = 0;
> +	struct venc_common_inst *inst;
> +
> +	inst = kzalloc(sizeof(*inst), GFP_KERNEL);
> +	if (!inst)
> +		return -ENOMEM;
> +
> +	inst->ctx = ctx;
> +	inst->vpu_inst.ctx = ctx;
> +	inst->vpu_inst.id = SCP_IPI_VENC_H264;
> +
> +	mtk_vcodec_debug_enter(inst);
> +
> +	ret = vpu_enc_init(&inst->vpu_inst);
> +
> +	inst->vsi = (struct venc_vsi *)inst->vpu_inst.vsi;
No need to cast.

> +static int venc_encode(void *handle,
> +		       enum venc_start_opt opt,
> +		       struct venc_frm_buf *frm_buf,
> +		       struct mtk_vcodec_mem *bs_buf,
> +		       struct venc_done_result *result)
> +{
> +	int ret;
> +	struct venc_common_inst *inst = (struct venc_common_inst *)handle;
No need to cast.

> +	default:
> +		mtk_vcodec_err(inst, "venc_start_opt %d not supported", opt);
> +		return -EINVAL;
> +	}
> +
> +	return ret;
It is more clear if just return 0 here.

> +static int venc_set_param(void *handle,
> +			  enum venc_set_param_type type,
> +			  struct venc_enc_param *enc_prm)
> +{
> +	int ret = 0;
> +	struct venc_common_inst *inst = (struct venc_common_inst *)handle;
No need to cast.

> +static int venc_deinit(void *handle)
> +{
> +	int ret = 0;
Same.

> +	struct venc_common_inst *inst = (struct venc_common_inst *)handle;
No need to cast.

> +	mtk_vcodec_debug_leave(inst);
> +	kfree(inst);
> +
> +	return ret;
Same.

Re: [PATCH 6/9] media: mtk-vcodec: Add new venc drv interface for frame_racing mode

[Tzung-Bi Shih]

On Mon, Aug 16, 2021 at 06:59:31PM +0800, Irui Wang wrote:
> Frame-racing mode encoding need more venc working buffers, it
> will break the compatibility if we just add venc_vsi in AP-Kernel
> but not in firmware, so add a new venc driver interface to
> distinguish the sigle_core_mode and frame_racing mode.
> 
> The new driver interface can be used for different codecs in
> the future.
The patch contains more code than the commit message explains that I couldn't understand.  Only find some nits.

> --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c
> +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c
> @@ -9,6 +9,7 @@
>  
>  #include "mtk_vcodec_drv.h"
>  #include "mtk_vcodec_util.h"
> +#include "mtk_vcodec_enc_hw.h"
Please try to maintain the order.

> +void __iomem *mtk_get_venc_comp_reg_addr(struct mtk_vcodec_ctx *data,
> +					 unsigned int hw_id)
> +{
> +	struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)data;
typeof(ctx) == typeof(data), rename the function argument 'data' to 'ctx'.

> +static int venc_alloc_work_buf(struct venc_common_inst *inst)
> +{
> +	int i;
> +	int ret = 0;
It doesn't need to be initialized.  See comment below.

> +	mtk_vcodec_debug_leave(inst);
> +
> +	return ret;
It is more clear if just return 0 here.  In the meantime, ret doesn't need to be initialized.

> +static unsigned int venc_wait_comp_done(struct venc_common_inst *inst,
> +					unsigned int hw_id)
> +{
> +	unsigned int irq_status = 0;
> +	struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)inst->ctx;
No need to cast.

> +static int venc_encode_sps(struct venc_common_inst *inst,
> +			   struct mtk_vcodec_mem *bs_buf,
> +			   unsigned int *bs_size)
> +{
> +	int ret = 0;
Same.

> +	*bs_size = venc_read_reg(inst, VENC_PIC_BITSTREAM_BYTE_CNT,
> +				 MTK_VENC_CORE0);
> +	mtk_vcodec_debug(inst, "bs size %d <-", *bs_size);
> +
> +	return ret;
Same.

> +static int venc_encode_pps(struct venc_common_inst *inst,
> +			   struct mtk_vcodec_mem *bs_buf,
> +			   unsigned int *bs_size)
> +{
> +	int ret = 0;
Same.

> +	*bs_size = venc_read_reg(inst, VENC_PIC_BITSTREAM_BYTE_CNT,
> +				 MTK_VENC_CORE0);
> +	mtk_vcodec_debug(inst, "bs size %d <-", *bs_size);
> +
> +	return ret;
Same.

> +static int venc_encode_header(struct venc_common_inst *inst,
> +			      struct mtk_vcodec_mem *bs_buf,
> +			      unsigned int *bs_size)
> +{
> +	int ret = 0;
Same.

> +	memcpy(bs_buf->va + bs_size_sps, inst->pps_buf.va, bs_size_pps);
> +	*bs_size = bs_size_sps + bs_size_pps;
> +
> +	return ret;
Same.

> +static int venc_encode_frame(struct venc_common_inst *inst,
> +			     struct venc_frm_buf *frm_buf,
> +			     struct mtk_vcodec_mem *bs_buf,
> +			     unsigned int *bs_size,
> +			     int hw_id)
> +{
> +	int ret = 0;
Same.

> +	++inst->frm_cnt;
> +	mtk_vcodec_debug(inst, "frm %d bs_size %d key_frm %d <-",
> +			 inst->frm_cnt, *bs_size, inst->vpu_inst.is_key_frm);
> +
> +	return ret;
Same.

> +static void h264_encode_filler(struct venc_common_inst *inst, void *buf,
> +			       int size)
> +{
> +	unsigned char *p = buf;
> +
> +	if (size < H264_FILLER_MARKER_SIZE) {
> +		mtk_vcodec_err(inst, "filler size too small %d", size);
> +		return;
> +	}
> +
> +	memcpy(p, h264_filler_marker, ARRAY_SIZE(h264_filler_marker));
Replace ARRAY_SIZE(h264_filler_marker) to H264_FILLER_MARKER_SIZE.

> +static int venc_init(struct mtk_vcodec_ctx *ctx)
> +{
> +	int i;
> +	int ret = 0;
> +	struct venc_common_inst *inst;
> +
> +	inst = kzalloc(sizeof(*inst), GFP_KERNEL);
> +	if (!inst)
> +		return -ENOMEM;
> +
> +	inst->ctx = ctx;
> +	inst->vpu_inst.ctx = ctx;
> +	inst->vpu_inst.id = SCP_IPI_VENC_H264;
> +
> +	mtk_vcodec_debug_enter(inst);
> +
> +	ret = vpu_enc_init(&inst->vpu_inst);
> +
> +	inst->vsi = (struct venc_vsi *)inst->vpu_inst.vsi;
No need to cast.

> +static int venc_encode(void *handle,
> +		       enum venc_start_opt opt,
> +		       struct venc_frm_buf *frm_buf,
> +		       struct mtk_vcodec_mem *bs_buf,
> +		       struct venc_done_result *result)
> +{
> +	int ret;
> +	struct venc_common_inst *inst = (struct venc_common_inst *)handle;
No need to cast.

> +	default:
> +		mtk_vcodec_err(inst, "venc_start_opt %d not supported", opt);
> +		return -EINVAL;
> +	}
> +
> +	return ret;
It is more clear if just return 0 here.

> +static int venc_set_param(void *handle,
> +			  enum venc_set_param_type type,
> +			  struct venc_enc_param *enc_prm)
> +{
> +	int ret = 0;
> +	struct venc_common_inst *inst = (struct venc_common_inst *)handle;
No need to cast.

> +static int venc_deinit(void *handle)
> +{
> +	int ret = 0;
Same.

> +	struct venc_common_inst *inst = (struct venc_common_inst *)handle;
No need to cast.

> +	mtk_vcodec_debug_leave(inst);
> +	kfree(inst);
> +
> +	return ret;
Same.

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH 6/9] media: mtk-vcodec: Add new venc drv interface for frame_racing mode

[Tzung-Bi Shih]

On Mon, Aug 16, 2021 at 06:59:31PM +0800, Irui Wang wrote:
> Frame-racing mode encoding need more venc working buffers, it
> will break the compatibility if we just add venc_vsi in AP-Kernel
> but not in firmware, so add a new venc driver interface to
> distinguish the sigle_core_mode and frame_racing mode.
> 
> The new driver interface can be used for different codecs in
> the future.
The patch contains more code than the commit message explains that I couldn't understand.  Only find some nits.

> --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c
> +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c
> @@ -9,6 +9,7 @@
>  
>  #include "mtk_vcodec_drv.h"
>  #include "mtk_vcodec_util.h"
> +#include "mtk_vcodec_enc_hw.h"
Please try to maintain the order.

> +void __iomem *mtk_get_venc_comp_reg_addr(struct mtk_vcodec_ctx *data,
> +					 unsigned int hw_id)
> +{
> +	struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)data;
typeof(ctx) == typeof(data), rename the function argument 'data' to 'ctx'.

> +static int venc_alloc_work_buf(struct venc_common_inst *inst)
> +{
> +	int i;
> +	int ret = 0;
It doesn't need to be initialized.  See comment below.

> +	mtk_vcodec_debug_leave(inst);
> +
> +	return ret;
It is more clear if just return 0 here.  In the meantime, ret doesn't need to be initialized.

> +static unsigned int venc_wait_comp_done(struct venc_common_inst *inst,
> +					unsigned int hw_id)
> +{
> +	unsigned int irq_status = 0;
> +	struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)inst->ctx;
No need to cast.

> +static int venc_encode_sps(struct venc_common_inst *inst,
> +			   struct mtk_vcodec_mem *bs_buf,
> +			   unsigned int *bs_size)
> +{
> +	int ret = 0;
Same.

> +	*bs_size = venc_read_reg(inst, VENC_PIC_BITSTREAM_BYTE_CNT,
> +				 MTK_VENC_CORE0);
> +	mtk_vcodec_debug(inst, "bs size %d <-", *bs_size);
> +
> +	return ret;
Same.

> +static int venc_encode_pps(struct venc_common_inst *inst,
> +			   struct mtk_vcodec_mem *bs_buf,
> +			   unsigned int *bs_size)
> +{
> +	int ret = 0;
Same.

> +	*bs_size = venc_read_reg(inst, VENC_PIC_BITSTREAM_BYTE_CNT,
> +				 MTK_VENC_CORE0);
> +	mtk_vcodec_debug(inst, "bs size %d <-", *bs_size);
> +
> +	return ret;
Same.

> +static int venc_encode_header(struct venc_common_inst *inst,
> +			      struct mtk_vcodec_mem *bs_buf,
> +			      unsigned int *bs_size)
> +{
> +	int ret = 0;
Same.

> +	memcpy(bs_buf->va + bs_size_sps, inst->pps_buf.va, bs_size_pps);
> +	*bs_size = bs_size_sps + bs_size_pps;
> +
> +	return ret;
Same.

> +static int venc_encode_frame(struct venc_common_inst *inst,
> +			     struct venc_frm_buf *frm_buf,
> +			     struct mtk_vcodec_mem *bs_buf,
> +			     unsigned int *bs_size,
> +			     int hw_id)
> +{
> +	int ret = 0;
Same.

> +	++inst->frm_cnt;
> +	mtk_vcodec_debug(inst, "frm %d bs_size %d key_frm %d <-",
> +			 inst->frm_cnt, *bs_size, inst->vpu_inst.is_key_frm);
> +
> +	return ret;
Same.

> +static void h264_encode_filler(struct venc_common_inst *inst, void *buf,
> +			       int size)
> +{
> +	unsigned char *p = buf;
> +
> +	if (size < H264_FILLER_MARKER_SIZE) {
> +		mtk_vcodec_err(inst, "filler size too small %d", size);
> +		return;
> +	}
> +
> +	memcpy(p, h264_filler_marker, ARRAY_SIZE(h264_filler_marker));
Replace ARRAY_SIZE(h264_filler_marker) to H264_FILLER_MARKER_SIZE.

> +static int venc_init(struct mtk_vcodec_ctx *ctx)
> +{
> +	int i;
> +	int ret = 0;
> +	struct venc_common_inst *inst;
> +
> +	inst = kzalloc(sizeof(*inst), GFP_KERNEL);
> +	if (!inst)
> +		return -ENOMEM;
> +
> +	inst->ctx = ctx;
> +	inst->vpu_inst.ctx = ctx;
> +	inst->vpu_inst.id = SCP_IPI_VENC_H264;
> +
> +	mtk_vcodec_debug_enter(inst);
> +
> +	ret = vpu_enc_init(&inst->vpu_inst);
> +
> +	inst->vsi = (struct venc_vsi *)inst->vpu_inst.vsi;
No need to cast.

> +static int venc_encode(void *handle,
> +		       enum venc_start_opt opt,
> +		       struct venc_frm_buf *frm_buf,
> +		       struct mtk_vcodec_mem *bs_buf,
> +		       struct venc_done_result *result)
> +{
> +	int ret;
> +	struct venc_common_inst *inst = (struct venc_common_inst *)handle;
No need to cast.

> +	default:
> +		mtk_vcodec_err(inst, "venc_start_opt %d not supported", opt);
> +		return -EINVAL;
> +	}
> +
> +	return ret;
It is more clear if just return 0 here.

> +static int venc_set_param(void *handle,
> +			  enum venc_set_param_type type,
> +			  struct venc_enc_param *enc_prm)
> +{
> +	int ret = 0;
> +	struct venc_common_inst *inst = (struct venc_common_inst *)handle;
No need to cast.

> +static int venc_deinit(void *handle)
> +{
> +	int ret = 0;
Same.

> +	struct venc_common_inst *inst = (struct venc_common_inst *)handle;
No need to cast.

> +	mtk_vcodec_debug_leave(inst);
> +	kfree(inst);
> +
> +	return ret;
Same.

_______________________________________________
Linux-mediatek mailing list
Linux-mediatek@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-mediatek

[PATCH 7/9] media: mtk-vcodec: Add frame racing mode encode process

[Irui Wang]

The frame_racing mode encoding is try to use the two venc cores:
frame#0 use core#0, frame#1 use core#1, frame#2 use core#0...,

Lock the device and enabe the clock by used core, for sequence
header encoding, it always used core#0.

Signed-off-by: Irui Wang <irui.wang@mediatek.com>
---
 .../platform/mtk-vcodec/mtk_vcodec_drv.h      |  5 +-
 .../platform/mtk-vcodec/mtk_vcodec_enc.c      | 35 ++++----
 .../platform/mtk-vcodec/mtk_vcodec_enc.h      |  4 +-
 .../platform/mtk-vcodec/mtk_vcodec_enc_drv.c  |  6 +-
 .../platform/mtk-vcodec/venc/venc_common_if.c |  6 +-
 .../platform/mtk-vcodec/venc/venc_h264_if.c   |  6 +-
 .../platform/mtk-vcodec/venc/venc_vp8_if.c    |  2 +-
 .../media/platform/mtk-vcodec/venc_drv_if.c   | 85 ++++++++++++++-----
 .../media/platform/mtk-vcodec/venc_drv_if.h   |  4 +
 .../media/platform/mtk-vcodec/venc_vpu_if.c   | 11 ++-
 .../media/platform/mtk-vcodec/venc_vpu_if.h   |  3 +-
 11 files changed, 118 insertions(+), 49 deletions(-)

diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
index 1c6c1eca649f..7a9c2dd52b91 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
@@ -273,6 +273,7 @@ struct vdec_pic_info {
  * @decoded_frame_cnt: number of decoded frames
  * @lock: protect variables accessed by V4L2 threads and worker thread such as
  *	  mtk_video_dec_buf.
+ * @enc_idx: used to record encoded frame count
  */
 struct mtk_vcodec_ctx {
 	enum mtk_instance_type type;
@@ -313,6 +314,8 @@ struct mtk_vcodec_ctx {
 	int decoded_frame_cnt;
 	struct mutex lock;
 
+	int hw_id;
+	int enc_idx;
 };
 
 enum mtk_chip {
@@ -422,7 +425,7 @@ struct mtk_vcodec_dev {
 	int enc_irq;
 
 	struct mutex dec_mutex;
-	struct mutex enc_mutex;
+	struct mutex enc_mutex[MTK_VENC_HW_MAX];
 
 	struct mtk_vcodec_pm pm;
 	unsigned int dec_capability;
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
index d22f9c8fd063..6b37bd092fdf 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
@@ -1090,21 +1090,26 @@ static void mtk_venc_worker(struct work_struct *work)
 	if (enc_result.is_key_frm)
 		dst_buf->flags |= V4L2_BUF_FLAG_KEYFRAME;
 
-	if (ret) {
-		v4l2_m2m_buf_done(src_buf, VB2_BUF_STATE_ERROR);
-		dst_buf->vb2_buf.planes[0].bytesused = 0;
-		v4l2_m2m_buf_done(dst_buf, VB2_BUF_STATE_ERROR);
-		mtk_v4l2_err("venc_if_encode failed=%d", ret);
-	} else {
-		v4l2_m2m_buf_done(src_buf, VB2_BUF_STATE_DONE);
-		dst_buf->vb2_buf.planes[0].bytesused = enc_result.bs_size;
-		v4l2_m2m_buf_done(dst_buf, VB2_BUF_STATE_DONE);
-		mtk_v4l2_debug(2, "venc_if_encode bs size=%d",
-				 enc_result.bs_size);
+	if (ctx->dev->venc_pdata->hw_mode == VENC_SINGLE_CORE_MODE) {
+		if (ret) {
+			v4l2_m2m_buf_done(src_buf, VB2_BUF_STATE_ERROR);
+			dst_buf->vb2_buf.planes[0].bytesused = 0;
+			v4l2_m2m_buf_done(dst_buf, VB2_BUF_STATE_ERROR);
+			mtk_v4l2_err("venc_if_encode failed=%d", ret);
+		} else {
+			v4l2_m2m_buf_done(src_buf, VB2_BUF_STATE_DONE);
+			dst_buf->vb2_buf.planes[0].bytesused =
+					enc_result.bs_size;
+			v4l2_m2m_buf_done(dst_buf, VB2_BUF_STATE_DONE);
+			mtk_v4l2_debug(2, "venc_if_encode bs size=%d",
+				       enc_result.bs_size);
+		}
 	}
 
 	v4l2_m2m_job_finish(ctx->dev->m2m_dev_enc, ctx->m2m_ctx);
 
+	ctx->enc_idx++;
+
 	mtk_v4l2_debug(1, "<=== src_buf[%d] dst_buf[%d] venc_if_encode ret=%d Size=%u===>",
 			src_buf->vb2_buf.index, dst_buf->vb2_buf.index, ret,
 			enc_result.bs_size);
@@ -1305,19 +1310,19 @@ int mtk_vcodec_enc_queue_init(void *priv, struct vb2_queue *src_vq,
 	return vb2_queue_init(dst_vq);
 }
 
-int mtk_venc_unlock(struct mtk_vcodec_ctx *ctx)
+int mtk_venc_unlock(struct mtk_vcodec_ctx *ctx, int core_id)
 {
 	struct mtk_vcodec_dev *dev = ctx->dev;
 
-	mutex_unlock(&dev->enc_mutex);
+	mutex_unlock(&dev->enc_mutex[core_id]);
 	return 0;
 }
 
-int mtk_venc_lock(struct mtk_vcodec_ctx *ctx)
+int mtk_venc_lock(struct mtk_vcodec_ctx *ctx, int core_id)
 {
 	struct mtk_vcodec_dev *dev = ctx->dev;
 
-	mutex_lock(&dev->enc_mutex);
+	mutex_lock(&dev->enc_mutex[core_id]);
 	return 0;
 }
 
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h
index 30b6d7f22ed7..8d486f532025 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h
@@ -41,8 +41,8 @@ struct mtk_video_enc_buf {
 extern const struct v4l2_ioctl_ops mtk_venc_ioctl_ops;
 extern const struct v4l2_m2m_ops mtk_venc_m2m_ops;
 
-int mtk_venc_unlock(struct mtk_vcodec_ctx *ctx);
-int mtk_venc_lock(struct mtk_vcodec_ctx *ctx);
+int mtk_venc_unlock(struct mtk_vcodec_ctx *ctx, int core_id);
+int mtk_venc_lock(struct mtk_vcodec_ctx *ctx, int core_id);
 int mtk_vcodec_enc_queue_init(void *priv, struct vb2_queue *src_vq,
 			      struct vb2_queue *dst_vq);
 void mtk_vcodec_enc_release(struct mtk_vcodec_ctx *ctx);
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
index b78435e8223a..ff1f05a5ca28 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
@@ -320,7 +320,7 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
 	struct resource *res;
 	phandle rproc_phandle;
 	enum mtk_vcodec_fw_type fw_type;
-	int ret;
+	int ret, i;
 	struct component_match *match = NULL;
 
 	dev = devm_kzalloc(&pdev->dev, sizeof(*dev), GFP_KERNEL);
@@ -384,7 +384,9 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
 		}
 	}
 
-	mutex_init(&dev->enc_mutex);
+	for (i = 0; i < MTK_VENC_HW_MAX; i++)
+		mutex_init(&dev->enc_mutex[i]);
+
 	mutex_init(&dev->dev_mutex);
 	spin_lock_init(&dev->irqlock);
 
diff --git a/drivers/media/platform/mtk-vcodec/venc/venc_common_if.c b/drivers/media/platform/mtk-vcodec/venc/venc_common_if.c
index 01b7d93241af..1580bd1beaac 100644
--- a/drivers/media/platform/mtk-vcodec/venc/venc_common_if.c
+++ b/drivers/media/platform/mtk-vcodec/venc/venc_common_if.c
@@ -300,7 +300,7 @@ static int venc_encode_sps(struct venc_common_inst *inst,
 	mtk_vcodec_debug_enter(inst);
 
 	ret = vpu_enc_encode(&inst->vpu_inst, VENC_BS_MODE_SPS, NULL,
-			     bs_buf, bs_size, NULL);
+			     bs_buf, bs_size, NULL, MTK_VENC_CORE0);
 	if (ret)
 		return ret;
 
@@ -328,7 +328,7 @@ static int venc_encode_pps(struct venc_common_inst *inst,
 	mtk_vcodec_debug_enter(inst);
 
 	ret = vpu_enc_encode(&inst->vpu_inst, VENC_BS_MODE_PPS, NULL,
-			     bs_buf, bs_size, NULL);
+			     bs_buf, bs_size, NULL, MTK_VENC_CORE0);
 	if (ret)
 		return ret;
 
@@ -386,7 +386,7 @@ static int venc_encode_frame(struct venc_common_inst *inst,
 			 frame_info.frm_count, frame_info.skip_frm_count,
 			 frame_info.frm_type);
 	ret = vpu_enc_encode(&inst->vpu_inst, VENC_BS_MODE_FRAME, frm_buf,
-			     bs_buf, bs_size, &frame_info);
+			     bs_buf, bs_size, &frame_info, hw_id);
 	if (ret)
 		return ret;
 
diff --git a/drivers/media/platform/mtk-vcodec/venc/venc_h264_if.c b/drivers/media/platform/mtk-vcodec/venc/venc_h264_if.c
index b6a4f2074fa5..a90b06a3fe5a 100644
--- a/drivers/media/platform/mtk-vcodec/venc/venc_h264_if.c
+++ b/drivers/media/platform/mtk-vcodec/venc/venc_h264_if.c
@@ -368,7 +368,7 @@ static int h264_encode_sps(struct venc_h264_inst *inst,
 	mtk_vcodec_debug_enter(inst);
 
 	ret = vpu_enc_encode(&inst->vpu_inst, H264_BS_MODE_SPS, NULL,
-			     bs_buf, bs_size, NULL);
+			     bs_buf, bs_size, NULL, MTK_VENC_CORE0);
 	if (ret)
 		return ret;
 
@@ -395,7 +395,7 @@ static int h264_encode_pps(struct venc_h264_inst *inst,
 	mtk_vcodec_debug_enter(inst);
 
 	ret = vpu_enc_encode(&inst->vpu_inst, H264_BS_MODE_PPS, NULL,
-			     bs_buf, bs_size, NULL);
+			     bs_buf, bs_size, NULL, MTK_VENC_CORE0);
 	if (ret)
 		return ret;
 
@@ -452,7 +452,7 @@ static int h264_encode_frame(struct venc_h264_inst *inst,
 			 frame_info.frm_count, frame_info.skip_frm_count,
 			 frame_info.frm_type);
 	ret = vpu_enc_encode(&inst->vpu_inst, H264_BS_MODE_FRAME, frm_buf,
-			     bs_buf, bs_size, &frame_info);
+			     bs_buf, bs_size, &frame_info, MTK_VENC_CORE0);
 	if (ret)
 		return ret;
 
diff --git a/drivers/media/platform/mtk-vcodec/venc/venc_vp8_if.c b/drivers/media/platform/mtk-vcodec/venc/venc_vp8_if.c
index 8267a9c4fd25..a68bf53d7d81 100644
--- a/drivers/media/platform/mtk-vcodec/venc/venc_vp8_if.c
+++ b/drivers/media/platform/mtk-vcodec/venc/venc_vp8_if.c
@@ -303,7 +303,7 @@ static int vp8_enc_encode_frame(struct venc_vp8_inst *inst,
 	mtk_vcodec_debug(inst, "->frm_cnt=%d", inst->frm_cnt);
 
 	ret = vpu_enc_encode(&inst->vpu_inst, 0, frm_buf, bs_buf, bs_size,
-			     NULL);
+			     NULL, MTK_VENC_CORE0);
 	if (ret)
 		return ret;
 
diff --git a/drivers/media/platform/mtk-vcodec/venc_drv_if.c b/drivers/media/platform/mtk-vcodec/venc_drv_if.c
index 93936d002bed..09fcec05a655 100644
--- a/drivers/media/platform/mtk-vcodec/venc_drv_if.c
+++ b/drivers/media/platform/mtk-vcodec/venc_drv_if.c
@@ -15,6 +15,7 @@
 
 #include "mtk_vcodec_enc.h"
 #include "mtk_vcodec_enc_pm.h"
+#include "mtk_vcodec_enc_hw.h"
 
 int venc_if_init(struct mtk_vcodec_ctx *ctx, unsigned int fourcc)
 {
@@ -34,9 +35,9 @@ int venc_if_init(struct mtk_vcodec_ctx *ctx, unsigned int fourcc)
 		return -EINVAL;
 	}
 
-	mtk_venc_lock(ctx);
+	mtk_venc_lock(ctx, 0);
 	ret = ctx->enc_if->init(ctx);
-	mtk_venc_unlock(ctx);
+	mtk_venc_unlock(ctx, 0);
 
 	return ret;
 }
@@ -46,9 +47,9 @@ int venc_if_set_param(struct mtk_vcodec_ctx *ctx,
 {
 	int ret = 0;
 
-	mtk_venc_lock(ctx);
+	mtk_venc_lock(ctx, 0);
 	ret = ctx->enc_if->set_param(ctx->drv_handle, type, in);
-	mtk_venc_unlock(ctx);
+	mtk_venc_unlock(ctx, 0);
 
 	return ret;
 }
@@ -59,24 +60,12 @@ int venc_if_encode(struct mtk_vcodec_ctx *ctx,
 		   struct venc_done_result *result)
 {
 	int ret = 0;
-	unsigned long flags;
-
-	mtk_venc_lock(ctx);
 
-	spin_lock_irqsave(&ctx->dev->irqlock, flags);
-	ctx->dev->curr_ctx = ctx;
-	spin_unlock_irqrestore(&ctx->dev->irqlock, flags);
-
-	mtk_vcodec_enc_clock_on(ctx->dev, 0);
+	venc_encode_prepare(ctx, opt);
 	ret = ctx->enc_if->encode(ctx->drv_handle, opt, frm_buf,
 				  bs_buf, result);
-	mtk_vcodec_enc_clock_off(ctx->dev, 0);
-
-	spin_lock_irqsave(&ctx->dev->irqlock, flags);
-	ctx->dev->curr_ctx = NULL;
-	spin_unlock_irqrestore(&ctx->dev->irqlock, flags);
+	venc_encode_unprepare(ctx, opt);
 
-	mtk_venc_unlock(ctx);
 	return ret;
 }
 
@@ -87,11 +76,67 @@ int venc_if_deinit(struct mtk_vcodec_ctx *ctx)
 	if (!ctx->drv_handle)
 		return 0;
 
-	mtk_venc_lock(ctx);
+	mtk_venc_lock(ctx, 0);
 	ret = ctx->enc_if->deinit(ctx->drv_handle);
-	mtk_venc_unlock(ctx);
+	mtk_venc_unlock(ctx, 0);
 
 	ctx->drv_handle = NULL;
 
 	return ret;
 }
+
+void venc_encode_prepare(struct mtk_vcodec_ctx *ctx,
+			 enum venc_start_opt opt)
+{
+	unsigned long flags;
+	struct mtk_venc_comp_dev *venc;
+
+	if (ctx->dev->venc_pdata->hw_mode == VENC_FRAME_RACING_MODE) {
+		if (ctx->enc_idx & 0x01)
+			ctx->hw_id = MTK_VENC_CORE1;
+		else
+			ctx->hw_id = MTK_VENC_CORE0;
+	} else {
+		ctx->hw_id = MTK_VENC_CORE0;
+	}
+	mtk_venc_lock(ctx, ctx->hw_id);
+
+	spin_lock_irqsave(&ctx->dev->irqlock, flags);
+
+	if (ctx->dev->venc_pdata->hw_mode == VENC_FRAME_RACING_MODE) {
+		venc = ctx->dev->enc_comp_dev[ctx->hw_id];
+
+		venc->curr_ctx = ctx;
+	} else {
+		ctx->dev->curr_ctx = ctx;
+	}
+
+	spin_unlock_irqrestore(&ctx->dev->irqlock, flags);
+
+	mtk_vcodec_enc_clock_on(ctx->dev, ctx->hw_id);
+}
+
+void venc_encode_unprepare(struct mtk_vcodec_ctx *ctx,
+			   enum venc_start_opt opt)
+{
+	unsigned long flags;
+	struct mtk_venc_comp_dev *venc;
+
+	/*clock off and unlock after irq done*/
+	if (ctx->dev->venc_pdata->hw_mode == VENC_FRAME_RACING_MODE) {
+		if (opt == VENC_START_OPT_ENCODE_SEQUENCE_HEADER) {
+			mtk_vcodec_enc_clock_off(ctx->dev, ctx->hw_id);
+			spin_lock_irqsave(&ctx->dev->irqlock, flags);
+			venc = ctx->dev->enc_comp_dev[ctx->hw_id];
+			venc->curr_ctx = NULL;
+			spin_unlock_irqrestore(&ctx->dev->irqlock, flags);
+			mtk_venc_unlock(ctx, ctx->hw_id);
+		}
+	} else {
+		mtk_vcodec_enc_clock_off(ctx->dev, ctx->hw_id);
+		spin_lock_irqsave(&ctx->dev->irqlock, flags);
+		ctx->dev->curr_ctx = NULL;
+		spin_unlock_irqrestore(&ctx->dev->irqlock, flags);
+		mtk_venc_unlock(ctx, ctx->hw_id);
+	}
+}
diff --git a/drivers/media/platform/mtk-vcodec/venc_drv_if.h b/drivers/media/platform/mtk-vcodec/venc_drv_if.h
index 8f384ba24ddf..7bd1f437d8e3 100644
--- a/drivers/media/platform/mtk-vcodec/venc_drv_if.h
+++ b/drivers/media/platform/mtk-vcodec/venc_drv_if.h
@@ -168,4 +168,8 @@ int venc_if_encode(struct mtk_vcodec_ctx *ctx,
 		   struct mtk_vcodec_mem *bs_buf,
 		   struct venc_done_result *result);
 
+void venc_encode_prepare(struct mtk_vcodec_ctx *ctx,
+			 enum venc_start_opt opt);
+void venc_encode_unprepare(struct mtk_vcodec_ctx *ctx,
+			   enum venc_start_opt opt);
 #endif /* _VENC_DRV_IF_H_ */
diff --git a/drivers/media/platform/mtk-vcodec/venc_vpu_if.c b/drivers/media/platform/mtk-vcodec/venc_vpu_if.c
index be6d8790a41e..5bee5f1828f1 100644
--- a/drivers/media/platform/mtk-vcodec/venc_vpu_if.c
+++ b/drivers/media/platform/mtk-vcodec/venc_vpu_if.c
@@ -226,9 +226,12 @@ int vpu_enc_encode(struct venc_vpu_inst *vpu, unsigned int bs_mode,
 		   struct venc_frm_buf *frm_buf,
 		   struct mtk_vcodec_mem *bs_buf,
 		   unsigned int *bs_size,
-		   struct venc_frame_info *frame_info)
+		   struct venc_frame_info *frame_info,
+		   int hw_id)
 {
 	const bool is_ext = MTK_ENC_CTX_IS_EXT(vpu->ctx);
+	const int hw_mode = MTK_ENC_HW_MODE(vpu->ctx);
+
 	size_t msg_size = is_ext ?
 		sizeof(struct venc_ap_ipi_msg_enc_ext) :
 		sizeof(struct venc_ap_ipi_msg_enc);
@@ -262,6 +265,12 @@ int vpu_enc_encode(struct venc_vpu_inst *vpu, unsigned int bs_mode,
 		out.data[1] = frame_info->skip_frm_count;
 		out.data[2] = frame_info->frm_type;
 	}
+
+	if (hw_mode == VENC_FRAME_RACING_MODE) {
+		out.data_item = 4;
+		out.data[3] = hw_id;
+	}
+
 	if (vpu_enc_send_msg(vpu, &out, msg_size)) {
 		mtk_vcodec_err(vpu, "AP_IPIMSG_ENC_ENCODE %d fail",
 			       bs_mode);
diff --git a/drivers/media/platform/mtk-vcodec/venc_vpu_if.h b/drivers/media/platform/mtk-vcodec/venc_vpu_if.h
index f9be9cab7ff7..e55656781f6e 100644
--- a/drivers/media/platform/mtk-vcodec/venc_vpu_if.h
+++ b/drivers/media/platform/mtk-vcodec/venc_vpu_if.h
@@ -46,7 +46,8 @@ int vpu_enc_encode(struct venc_vpu_inst *vpu, unsigned int bs_mode,
 		   struct venc_frm_buf *frm_buf,
 		   struct mtk_vcodec_mem *bs_buf,
 		   unsigned int *bs_size,
-		   struct venc_frame_info *frame_info);
+		   struct venc_frame_info *frame_info,
+		   int hw_id);
 int vpu_enc_deinit(struct venc_vpu_inst *vpu);
 
 #endif
-- 
2.25.1

[PATCH 7/9] media: mtk-vcodec: Add frame racing mode encode process

[Irui Wang]

The frame_racing mode encoding is try to use the two venc cores:
frame#0 use core#0, frame#1 use core#1, frame#2 use core#0...,

Lock the device and enabe the clock by used core, for sequence
header encoding, it always used core#0.

Signed-off-by: Irui Wang <irui.wang@mediatek.com>
---
 .../platform/mtk-vcodec/mtk_vcodec_drv.h      |  5 +-
 .../platform/mtk-vcodec/mtk_vcodec_enc.c      | 35 ++++----
 .../platform/mtk-vcodec/mtk_vcodec_enc.h      |  4 +-
 .../platform/mtk-vcodec/mtk_vcodec_enc_drv.c  |  6 +-
 .../platform/mtk-vcodec/venc/venc_common_if.c |  6 +-
 .../platform/mtk-vcodec/venc/venc_h264_if.c   |  6 +-
 .../platform/mtk-vcodec/venc/venc_vp8_if.c    |  2 +-
 .../media/platform/mtk-vcodec/venc_drv_if.c   | 85 ++++++++++++++-----
 .../media/platform/mtk-vcodec/venc_drv_if.h   |  4 +
 .../media/platform/mtk-vcodec/venc_vpu_if.c   | 11 ++-
 .../media/platform/mtk-vcodec/venc_vpu_if.h   |  3 +-
 11 files changed, 118 insertions(+), 49 deletions(-)

diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
index 1c6c1eca649f..7a9c2dd52b91 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
@@ -273,6 +273,7 @@ struct vdec_pic_info {
  * @decoded_frame_cnt: number of decoded frames
  * @lock: protect variables accessed by V4L2 threads and worker thread such as
  *	  mtk_video_dec_buf.
+ * @enc_idx: used to record encoded frame count
  */
 struct mtk_vcodec_ctx {
 	enum mtk_instance_type type;
@@ -313,6 +314,8 @@ struct mtk_vcodec_ctx {
 	int decoded_frame_cnt;
 	struct mutex lock;
 
+	int hw_id;
+	int enc_idx;
 };
 
 enum mtk_chip {
@@ -422,7 +425,7 @@ struct mtk_vcodec_dev {
 	int enc_irq;
 
 	struct mutex dec_mutex;
-	struct mutex enc_mutex;
+	struct mutex enc_mutex[MTK_VENC_HW_MAX];
 
 	struct mtk_vcodec_pm pm;
 	unsigned int dec_capability;
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
index d22f9c8fd063..6b37bd092fdf 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
@@ -1090,21 +1090,26 @@ static void mtk_venc_worker(struct work_struct *work)
 	if (enc_result.is_key_frm)
 		dst_buf->flags |= V4L2_BUF_FLAG_KEYFRAME;
 
-	if (ret) {
-		v4l2_m2m_buf_done(src_buf, VB2_BUF_STATE_ERROR);
-		dst_buf->vb2_buf.planes[0].bytesused = 0;
-		v4l2_m2m_buf_done(dst_buf, VB2_BUF_STATE_ERROR);
-		mtk_v4l2_err("venc_if_encode failed=%d", ret);
-	} else {
-		v4l2_m2m_buf_done(src_buf, VB2_BUF_STATE_DONE);
-		dst_buf->vb2_buf.planes[0].bytesused = enc_result.bs_size;
-		v4l2_m2m_buf_done(dst_buf, VB2_BUF_STATE_DONE);
-		mtk_v4l2_debug(2, "venc_if_encode bs size=%d",
-				 enc_result.bs_size);
+	if (ctx->dev->venc_pdata->hw_mode == VENC_SINGLE_CORE_MODE) {
+		if (ret) {
+			v4l2_m2m_buf_done(src_buf, VB2_BUF_STATE_ERROR);
+			dst_buf->vb2_buf.planes[0].bytesused = 0;
+			v4l2_m2m_buf_done(dst_buf, VB2_BUF_STATE_ERROR);
+			mtk_v4l2_err("venc_if_encode failed=%d", ret);
+		} else {
+			v4l2_m2m_buf_done(src_buf, VB2_BUF_STATE_DONE);
+			dst_buf->vb2_buf.planes[0].bytesused =
+					enc_result.bs_size;
+			v4l2_m2m_buf_done(dst_buf, VB2_BUF_STATE_DONE);
+			mtk_v4l2_debug(2, "venc_if_encode bs size=%d",
+				       enc_result.bs_size);
+		}
 	}
 
 	v4l2_m2m_job_finish(ctx->dev->m2m_dev_enc, ctx->m2m_ctx);
 
+	ctx->enc_idx++;
+
 	mtk_v4l2_debug(1, "<=== src_buf[%d] dst_buf[%d] venc_if_encode ret=%d Size=%u===>",
 			src_buf->vb2_buf.index, dst_buf->vb2_buf.index, ret,
 			enc_result.bs_size);
@@ -1305,19 +1310,19 @@ int mtk_vcodec_enc_queue_init(void *priv, struct vb2_queue *src_vq,
 	return vb2_queue_init(dst_vq);
 }
 
-int mtk_venc_unlock(struct mtk_vcodec_ctx *ctx)
+int mtk_venc_unlock(struct mtk_vcodec_ctx *ctx, int core_id)
 {
 	struct mtk_vcodec_dev *dev = ctx->dev;
 
-	mutex_unlock(&dev->enc_mutex);
+	mutex_unlock(&dev->enc_mutex[core_id]);
 	return 0;
 }
 
-int mtk_venc_lock(struct mtk_vcodec_ctx *ctx)
+int mtk_venc_lock(struct mtk_vcodec_ctx *ctx, int core_id)
 {
 	struct mtk_vcodec_dev *dev = ctx->dev;
 
-	mutex_lock(&dev->enc_mutex);
+	mutex_lock(&dev->enc_mutex[core_id]);
 	return 0;
 }
 
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h
index 30b6d7f22ed7..8d486f532025 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h
@@ -41,8 +41,8 @@ struct mtk_video_enc_buf {
 extern const struct v4l2_ioctl_ops mtk_venc_ioctl_ops;
 extern const struct v4l2_m2m_ops mtk_venc_m2m_ops;
 
-int mtk_venc_unlock(struct mtk_vcodec_ctx *ctx);
-int mtk_venc_lock(struct mtk_vcodec_ctx *ctx);
+int mtk_venc_unlock(struct mtk_vcodec_ctx *ctx, int core_id);
+int mtk_venc_lock(struct mtk_vcodec_ctx *ctx, int core_id);
 int mtk_vcodec_enc_queue_init(void *priv, struct vb2_queue *src_vq,
 			      struct vb2_queue *dst_vq);
 void mtk_vcodec_enc_release(struct mtk_vcodec_ctx *ctx);
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
index b78435e8223a..ff1f05a5ca28 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
@@ -320,7 +320,7 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
 	struct resource *res;
 	phandle rproc_phandle;
 	enum mtk_vcodec_fw_type fw_type;
-	int ret;
+	int ret, i;
 	struct component_match *match = NULL;
 
 	dev = devm_kzalloc(&pdev->dev, sizeof(*dev), GFP_KERNEL);
@@ -384,7 +384,9 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
 		}
 	}
 
-	mutex_init(&dev->enc_mutex);
+	for (i = 0; i < MTK_VENC_HW_MAX; i++)
+		mutex_init(&dev->enc_mutex[i]);
+
 	mutex_init(&dev->dev_mutex);
 	spin_lock_init(&dev->irqlock);
 
diff --git a/drivers/media/platform/mtk-vcodec/venc/venc_common_if.c b/drivers/media/platform/mtk-vcodec/venc/venc_common_if.c
index 01b7d93241af..1580bd1beaac 100644
--- a/drivers/media/platform/mtk-vcodec/venc/venc_common_if.c
+++ b/drivers/media/platform/mtk-vcodec/venc/venc_common_if.c
@@ -300,7 +300,7 @@ static int venc_encode_sps(struct venc_common_inst *inst,
 	mtk_vcodec_debug_enter(inst);
 
 	ret = vpu_enc_encode(&inst->vpu_inst, VENC_BS_MODE_SPS, NULL,
-			     bs_buf, bs_size, NULL);
+			     bs_buf, bs_size, NULL, MTK_VENC_CORE0);
 	if (ret)
 		return ret;
 
@@ -328,7 +328,7 @@ static int venc_encode_pps(struct venc_common_inst *inst,
 	mtk_vcodec_debug_enter(inst);
 
 	ret = vpu_enc_encode(&inst->vpu_inst, VENC_BS_MODE_PPS, NULL,
-			     bs_buf, bs_size, NULL);
+			     bs_buf, bs_size, NULL, MTK_VENC_CORE0);
 	if (ret)
 		return ret;
 
@@ -386,7 +386,7 @@ static int venc_encode_frame(struct venc_common_inst *inst,
 			 frame_info.frm_count, frame_info.skip_frm_count,
 			 frame_info.frm_type);
 	ret = vpu_enc_encode(&inst->vpu_inst, VENC_BS_MODE_FRAME, frm_buf,
-			     bs_buf, bs_size, &frame_info);
+			     bs_buf, bs_size, &frame_info, hw_id);
 	if (ret)
 		return ret;
 
diff --git a/drivers/media/platform/mtk-vcodec/venc/venc_h264_if.c b/drivers/media/platform/mtk-vcodec/venc/venc_h264_if.c
index b6a4f2074fa5..a90b06a3fe5a 100644
--- a/drivers/media/platform/mtk-vcodec/venc/venc_h264_if.c
+++ b/drivers/media/platform/mtk-vcodec/venc/venc_h264_if.c
@@ -368,7 +368,7 @@ static int h264_encode_sps(struct venc_h264_inst *inst,
 	mtk_vcodec_debug_enter(inst);
 
 	ret = vpu_enc_encode(&inst->vpu_inst, H264_BS_MODE_SPS, NULL,
-			     bs_buf, bs_size, NULL);
+			     bs_buf, bs_size, NULL, MTK_VENC_CORE0);
 	if (ret)
 		return ret;
 
@@ -395,7 +395,7 @@ static int h264_encode_pps(struct venc_h264_inst *inst,
 	mtk_vcodec_debug_enter(inst);
 
 	ret = vpu_enc_encode(&inst->vpu_inst, H264_BS_MODE_PPS, NULL,
-			     bs_buf, bs_size, NULL);
+			     bs_buf, bs_size, NULL, MTK_VENC_CORE0);
 	if (ret)
 		return ret;
 
@@ -452,7 +452,7 @@ static int h264_encode_frame(struct venc_h264_inst *inst,
 			 frame_info.frm_count, frame_info.skip_frm_count,
 			 frame_info.frm_type);
 	ret = vpu_enc_encode(&inst->vpu_inst, H264_BS_MODE_FRAME, frm_buf,
-			     bs_buf, bs_size, &frame_info);
+			     bs_buf, bs_size, &frame_info, MTK_VENC_CORE0);
 	if (ret)
 		return ret;
 
diff --git a/drivers/media/platform/mtk-vcodec/venc/venc_vp8_if.c b/drivers/media/platform/mtk-vcodec/venc/venc_vp8_if.c
index 8267a9c4fd25..a68bf53d7d81 100644
--- a/drivers/media/platform/mtk-vcodec/venc/venc_vp8_if.c
+++ b/drivers/media/platform/mtk-vcodec/venc/venc_vp8_if.c
@@ -303,7 +303,7 @@ static int vp8_enc_encode_frame(struct venc_vp8_inst *inst,
 	mtk_vcodec_debug(inst, "->frm_cnt=%d", inst->frm_cnt);
 
 	ret = vpu_enc_encode(&inst->vpu_inst, 0, frm_buf, bs_buf, bs_size,
-			     NULL);
+			     NULL, MTK_VENC_CORE0);
 	if (ret)
 		return ret;
 
diff --git a/drivers/media/platform/mtk-vcodec/venc_drv_if.c b/drivers/media/platform/mtk-vcodec/venc_drv_if.c
index 93936d002bed..09fcec05a655 100644
--- a/drivers/media/platform/mtk-vcodec/venc_drv_if.c
+++ b/drivers/media/platform/mtk-vcodec/venc_drv_if.c
@@ -15,6 +15,7 @@
 
 #include "mtk_vcodec_enc.h"
 #include "mtk_vcodec_enc_pm.h"
+#include "mtk_vcodec_enc_hw.h"
 
 int venc_if_init(struct mtk_vcodec_ctx *ctx, unsigned int fourcc)
 {
@@ -34,9 +35,9 @@ int venc_if_init(struct mtk_vcodec_ctx *ctx, unsigned int fourcc)
 		return -EINVAL;
 	}
 
-	mtk_venc_lock(ctx);
+	mtk_venc_lock(ctx, 0);
 	ret = ctx->enc_if->init(ctx);
-	mtk_venc_unlock(ctx);
+	mtk_venc_unlock(ctx, 0);
 
 	return ret;
 }
@@ -46,9 +47,9 @@ int venc_if_set_param(struct mtk_vcodec_ctx *ctx,
 {
 	int ret = 0;
 
-	mtk_venc_lock(ctx);
+	mtk_venc_lock(ctx, 0);
 	ret = ctx->enc_if->set_param(ctx->drv_handle, type, in);
-	mtk_venc_unlock(ctx);
+	mtk_venc_unlock(ctx, 0);
 
 	return ret;
 }
@@ -59,24 +60,12 @@ int venc_if_encode(struct mtk_vcodec_ctx *ctx,
 		   struct venc_done_result *result)
 {
 	int ret = 0;
-	unsigned long flags;
-
-	mtk_venc_lock(ctx);
 
-	spin_lock_irqsave(&ctx->dev->irqlock, flags);
-	ctx->dev->curr_ctx = ctx;
-	spin_unlock_irqrestore(&ctx->dev->irqlock, flags);
-
-	mtk_vcodec_enc_clock_on(ctx->dev, 0);
+	venc_encode_prepare(ctx, opt);
 	ret = ctx->enc_if->encode(ctx->drv_handle, opt, frm_buf,
 				  bs_buf, result);
-	mtk_vcodec_enc_clock_off(ctx->dev, 0);
-
-	spin_lock_irqsave(&ctx->dev->irqlock, flags);
-	ctx->dev->curr_ctx = NULL;
-	spin_unlock_irqrestore(&ctx->dev->irqlock, flags);
+	venc_encode_unprepare(ctx, opt);
 
-	mtk_venc_unlock(ctx);
 	return ret;
 }
 
@@ -87,11 +76,67 @@ int venc_if_deinit(struct mtk_vcodec_ctx *ctx)
 	if (!ctx->drv_handle)
 		return 0;
 
-	mtk_venc_lock(ctx);
+	mtk_venc_lock(ctx, 0);
 	ret = ctx->enc_if->deinit(ctx->drv_handle);
-	mtk_venc_unlock(ctx);
+	mtk_venc_unlock(ctx, 0);
 
 	ctx->drv_handle = NULL;
 
 	return ret;
 }
+
+void venc_encode_prepare(struct mtk_vcodec_ctx *ctx,
+			 enum venc_start_opt opt)
+{
+	unsigned long flags;
+	struct mtk_venc_comp_dev *venc;
+
+	if (ctx->dev->venc_pdata->hw_mode == VENC_FRAME_RACING_MODE) {
+		if (ctx->enc_idx & 0x01)
+			ctx->hw_id = MTK_VENC_CORE1;
+		else
+			ctx->hw_id = MTK_VENC_CORE0;
+	} else {
+		ctx->hw_id = MTK_VENC_CORE0;
+	}
+	mtk_venc_lock(ctx, ctx->hw_id);
+
+	spin_lock_irqsave(&ctx->dev->irqlock, flags);
+
+	if (ctx->dev->venc_pdata->hw_mode == VENC_FRAME_RACING_MODE) {
+		venc = ctx->dev->enc_comp_dev[ctx->hw_id];
+
+		venc->curr_ctx = ctx;
+	} else {
+		ctx->dev->curr_ctx = ctx;
+	}
+
+	spin_unlock_irqrestore(&ctx->dev->irqlock, flags);
+
+	mtk_vcodec_enc_clock_on(ctx->dev, ctx->hw_id);
+}
+
+void venc_encode_unprepare(struct mtk_vcodec_ctx *ctx,
+			   enum venc_start_opt opt)
+{
+	unsigned long flags;
+	struct mtk_venc_comp_dev *venc;
+
+	/*clock off and unlock after irq done*/
+	if (ctx->dev->venc_pdata->hw_mode == VENC_FRAME_RACING_MODE) {
+		if (opt == VENC_START_OPT_ENCODE_SEQUENCE_HEADER) {
+			mtk_vcodec_enc_clock_off(ctx->dev, ctx->hw_id);
+			spin_lock_irqsave(&ctx->dev->irqlock, flags);
+			venc = ctx->dev->enc_comp_dev[ctx->hw_id];
+			venc->curr_ctx = NULL;
+			spin_unlock_irqrestore(&ctx->dev->irqlock, flags);
+			mtk_venc_unlock(ctx, ctx->hw_id);
+		}
+	} else {
+		mtk_vcodec_enc_clock_off(ctx->dev, ctx->hw_id);
+		spin_lock_irqsave(&ctx->dev->irqlock, flags);
+		ctx->dev->curr_ctx = NULL;
+		spin_unlock_irqrestore(&ctx->dev->irqlock, flags);
+		mtk_venc_unlock(ctx, ctx->hw_id);
+	}
+}
diff --git a/drivers/media/platform/mtk-vcodec/venc_drv_if.h b/drivers/media/platform/mtk-vcodec/venc_drv_if.h
index 8f384ba24ddf..7bd1f437d8e3 100644
--- a/drivers/media/platform/mtk-vcodec/venc_drv_if.h
+++ b/drivers/media/platform/mtk-vcodec/venc_drv_if.h
@@ -168,4 +168,8 @@ int venc_if_encode(struct mtk_vcodec_ctx *ctx,
 		   struct mtk_vcodec_mem *bs_buf,
 		   struct venc_done_result *result);
 
+void venc_encode_prepare(struct mtk_vcodec_ctx *ctx,
+			 enum venc_start_opt opt);
+void venc_encode_unprepare(struct mtk_vcodec_ctx *ctx,
+			   enum venc_start_opt opt);
 #endif /* _VENC_DRV_IF_H_ */
diff --git a/drivers/media/platform/mtk-vcodec/venc_vpu_if.c b/drivers/media/platform/mtk-vcodec/venc_vpu_if.c
index be6d8790a41e..5bee5f1828f1 100644
--- a/drivers/media/platform/mtk-vcodec/venc_vpu_if.c
+++ b/drivers/media/platform/mtk-vcodec/venc_vpu_if.c
@@ -226,9 +226,12 @@ int vpu_enc_encode(struct venc_vpu_inst *vpu, unsigned int bs_mode,
 		   struct venc_frm_buf *frm_buf,
 		   struct mtk_vcodec_mem *bs_buf,
 		   unsigned int *bs_size,
-		   struct venc_frame_info *frame_info)
+		   struct venc_frame_info *frame_info,
+		   int hw_id)
 {
 	const bool is_ext = MTK_ENC_CTX_IS_EXT(vpu->ctx);
+	const int hw_mode = MTK_ENC_HW_MODE(vpu->ctx);
+
 	size_t msg_size = is_ext ?
 		sizeof(struct venc_ap_ipi_msg_enc_ext) :
 		sizeof(struct venc_ap_ipi_msg_enc);
@@ -262,6 +265,12 @@ int vpu_enc_encode(struct venc_vpu_inst *vpu, unsigned int bs_mode,
 		out.data[1] = frame_info->skip_frm_count;
 		out.data[2] = frame_info->frm_type;
 	}
+
+	if (hw_mode == VENC_FRAME_RACING_MODE) {
+		out.data_item = 4;
+		out.data[3] = hw_id;
+	}
+
 	if (vpu_enc_send_msg(vpu, &out, msg_size)) {
 		mtk_vcodec_err(vpu, "AP_IPIMSG_ENC_ENCODE %d fail",
 			       bs_mode);
diff --git a/drivers/media/platform/mtk-vcodec/venc_vpu_if.h b/drivers/media/platform/mtk-vcodec/venc_vpu_if.h
index f9be9cab7ff7..e55656781f6e 100644
--- a/drivers/media/platform/mtk-vcodec/venc_vpu_if.h
+++ b/drivers/media/platform/mtk-vcodec/venc_vpu_if.h
@@ -46,7 +46,8 @@ int vpu_enc_encode(struct venc_vpu_inst *vpu, unsigned int bs_mode,
 		   struct venc_frm_buf *frm_buf,
 		   struct mtk_vcodec_mem *bs_buf,
 		   unsigned int *bs_size,
-		   struct venc_frame_info *frame_info);
+		   struct venc_frame_info *frame_info,
+		   int hw_id);
 int vpu_enc_deinit(struct venc_vpu_inst *vpu);
 
 #endif
-- 
2.25.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH 7/9] media: mtk-vcodec: Add frame racing mode encode process

[Irui Wang]

The frame_racing mode encoding is try to use the two venc cores:
frame#0 use core#0, frame#1 use core#1, frame#2 use core#0...,

Lock the device and enabe the clock by used core, for sequence
header encoding, it always used core#0.

Signed-off-by: Irui Wang <irui.wang@mediatek.com>
---
 .../platform/mtk-vcodec/mtk_vcodec_drv.h      |  5 +-
 .../platform/mtk-vcodec/mtk_vcodec_enc.c      | 35 ++++----
 .../platform/mtk-vcodec/mtk_vcodec_enc.h      |  4 +-
 .../platform/mtk-vcodec/mtk_vcodec_enc_drv.c  |  6 +-
 .../platform/mtk-vcodec/venc/venc_common_if.c |  6 +-
 .../platform/mtk-vcodec/venc/venc_h264_if.c   |  6 +-
 .../platform/mtk-vcodec/venc/venc_vp8_if.c    |  2 +-
 .../media/platform/mtk-vcodec/venc_drv_if.c   | 85 ++++++++++++++-----
 .../media/platform/mtk-vcodec/venc_drv_if.h   |  4 +
 .../media/platform/mtk-vcodec/venc_vpu_if.c   | 11 ++-
 .../media/platform/mtk-vcodec/venc_vpu_if.h   |  3 +-
 11 files changed, 118 insertions(+), 49 deletions(-)

diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
index 1c6c1eca649f..7a9c2dd52b91 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
@@ -273,6 +273,7 @@ struct vdec_pic_info {
  * @decoded_frame_cnt: number of decoded frames
  * @lock: protect variables accessed by V4L2 threads and worker thread such as
  *	  mtk_video_dec_buf.
+ * @enc_idx: used to record encoded frame count
  */
 struct mtk_vcodec_ctx {
 	enum mtk_instance_type type;
@@ -313,6 +314,8 @@ struct mtk_vcodec_ctx {
 	int decoded_frame_cnt;
 	struct mutex lock;
 
+	int hw_id;
+	int enc_idx;
 };
 
 enum mtk_chip {
@@ -422,7 +425,7 @@ struct mtk_vcodec_dev {
 	int enc_irq;
 
 	struct mutex dec_mutex;
-	struct mutex enc_mutex;
+	struct mutex enc_mutex[MTK_VENC_HW_MAX];
 
 	struct mtk_vcodec_pm pm;
 	unsigned int dec_capability;
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
index d22f9c8fd063..6b37bd092fdf 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
@@ -1090,21 +1090,26 @@ static void mtk_venc_worker(struct work_struct *work)
 	if (enc_result.is_key_frm)
 		dst_buf->flags |= V4L2_BUF_FLAG_KEYFRAME;
 
-	if (ret) {
-		v4l2_m2m_buf_done(src_buf, VB2_BUF_STATE_ERROR);
-		dst_buf->vb2_buf.planes[0].bytesused = 0;
-		v4l2_m2m_buf_done(dst_buf, VB2_BUF_STATE_ERROR);
-		mtk_v4l2_err("venc_if_encode failed=%d", ret);
-	} else {
-		v4l2_m2m_buf_done(src_buf, VB2_BUF_STATE_DONE);
-		dst_buf->vb2_buf.planes[0].bytesused = enc_result.bs_size;
-		v4l2_m2m_buf_done(dst_buf, VB2_BUF_STATE_DONE);
-		mtk_v4l2_debug(2, "venc_if_encode bs size=%d",
-				 enc_result.bs_size);
+	if (ctx->dev->venc_pdata->hw_mode == VENC_SINGLE_CORE_MODE) {
+		if (ret) {
+			v4l2_m2m_buf_done(src_buf, VB2_BUF_STATE_ERROR);
+			dst_buf->vb2_buf.planes[0].bytesused = 0;
+			v4l2_m2m_buf_done(dst_buf, VB2_BUF_STATE_ERROR);
+			mtk_v4l2_err("venc_if_encode failed=%d", ret);
+		} else {
+			v4l2_m2m_buf_done(src_buf, VB2_BUF_STATE_DONE);
+			dst_buf->vb2_buf.planes[0].bytesused =
+					enc_result.bs_size;
+			v4l2_m2m_buf_done(dst_buf, VB2_BUF_STATE_DONE);
+			mtk_v4l2_debug(2, "venc_if_encode bs size=%d",
+				       enc_result.bs_size);
+		}
 	}
 
 	v4l2_m2m_job_finish(ctx->dev->m2m_dev_enc, ctx->m2m_ctx);
 
+	ctx->enc_idx++;
+
 	mtk_v4l2_debug(1, "<=== src_buf[%d] dst_buf[%d] venc_if_encode ret=%d Size=%u===>",
 			src_buf->vb2_buf.index, dst_buf->vb2_buf.index, ret,
 			enc_result.bs_size);
@@ -1305,19 +1310,19 @@ int mtk_vcodec_enc_queue_init(void *priv, struct vb2_queue *src_vq,
 	return vb2_queue_init(dst_vq);
 }
 
-int mtk_venc_unlock(struct mtk_vcodec_ctx *ctx)
+int mtk_venc_unlock(struct mtk_vcodec_ctx *ctx, int core_id)
 {
 	struct mtk_vcodec_dev *dev = ctx->dev;
 
-	mutex_unlock(&dev->enc_mutex);
+	mutex_unlock(&dev->enc_mutex[core_id]);
 	return 0;
 }
 
-int mtk_venc_lock(struct mtk_vcodec_ctx *ctx)
+int mtk_venc_lock(struct mtk_vcodec_ctx *ctx, int core_id)
 {
 	struct mtk_vcodec_dev *dev = ctx->dev;
 
-	mutex_lock(&dev->enc_mutex);
+	mutex_lock(&dev->enc_mutex[core_id]);
 	return 0;
 }
 
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h
index 30b6d7f22ed7..8d486f532025 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h
@@ -41,8 +41,8 @@ struct mtk_video_enc_buf {
 extern const struct v4l2_ioctl_ops mtk_venc_ioctl_ops;
 extern const struct v4l2_m2m_ops mtk_venc_m2m_ops;
 
-int mtk_venc_unlock(struct mtk_vcodec_ctx *ctx);
-int mtk_venc_lock(struct mtk_vcodec_ctx *ctx);
+int mtk_venc_unlock(struct mtk_vcodec_ctx *ctx, int core_id);
+int mtk_venc_lock(struct mtk_vcodec_ctx *ctx, int core_id);
 int mtk_vcodec_enc_queue_init(void *priv, struct vb2_queue *src_vq,
 			      struct vb2_queue *dst_vq);
 void mtk_vcodec_enc_release(struct mtk_vcodec_ctx *ctx);
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
index b78435e8223a..ff1f05a5ca28 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
@@ -320,7 +320,7 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
 	struct resource *res;
 	phandle rproc_phandle;
 	enum mtk_vcodec_fw_type fw_type;
-	int ret;
+	int ret, i;
 	struct component_match *match = NULL;
 
 	dev = devm_kzalloc(&pdev->dev, sizeof(*dev), GFP_KERNEL);
@@ -384,7 +384,9 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
 		}
 	}
 
-	mutex_init(&dev->enc_mutex);
+	for (i = 0; i < MTK_VENC_HW_MAX; i++)
+		mutex_init(&dev->enc_mutex[i]);
+
 	mutex_init(&dev->dev_mutex);
 	spin_lock_init(&dev->irqlock);
 
diff --git a/drivers/media/platform/mtk-vcodec/venc/venc_common_if.c b/drivers/media/platform/mtk-vcodec/venc/venc_common_if.c
index 01b7d93241af..1580bd1beaac 100644
--- a/drivers/media/platform/mtk-vcodec/venc/venc_common_if.c
+++ b/drivers/media/platform/mtk-vcodec/venc/venc_common_if.c
@@ -300,7 +300,7 @@ static int venc_encode_sps(struct venc_common_inst *inst,
 	mtk_vcodec_debug_enter(inst);
 
 	ret = vpu_enc_encode(&inst->vpu_inst, VENC_BS_MODE_SPS, NULL,
-			     bs_buf, bs_size, NULL);
+			     bs_buf, bs_size, NULL, MTK_VENC_CORE0);
 	if (ret)
 		return ret;
 
@@ -328,7 +328,7 @@ static int venc_encode_pps(struct venc_common_inst *inst,
 	mtk_vcodec_debug_enter(inst);
 
 	ret = vpu_enc_encode(&inst->vpu_inst, VENC_BS_MODE_PPS, NULL,
-			     bs_buf, bs_size, NULL);
+			     bs_buf, bs_size, NULL, MTK_VENC_CORE0);
 	if (ret)
 		return ret;
 
@@ -386,7 +386,7 @@ static int venc_encode_frame(struct venc_common_inst *inst,
 			 frame_info.frm_count, frame_info.skip_frm_count,
 			 frame_info.frm_type);
 	ret = vpu_enc_encode(&inst->vpu_inst, VENC_BS_MODE_FRAME, frm_buf,
-			     bs_buf, bs_size, &frame_info);
+			     bs_buf, bs_size, &frame_info, hw_id);
 	if (ret)
 		return ret;
 
diff --git a/drivers/media/platform/mtk-vcodec/venc/venc_h264_if.c b/drivers/media/platform/mtk-vcodec/venc/venc_h264_if.c
index b6a4f2074fa5..a90b06a3fe5a 100644
--- a/drivers/media/platform/mtk-vcodec/venc/venc_h264_if.c
+++ b/drivers/media/platform/mtk-vcodec/venc/venc_h264_if.c
@@ -368,7 +368,7 @@ static int h264_encode_sps(struct venc_h264_inst *inst,
 	mtk_vcodec_debug_enter(inst);
 
 	ret = vpu_enc_encode(&inst->vpu_inst, H264_BS_MODE_SPS, NULL,
-			     bs_buf, bs_size, NULL);
+			     bs_buf, bs_size, NULL, MTK_VENC_CORE0);
 	if (ret)
 		return ret;
 
@@ -395,7 +395,7 @@ static int h264_encode_pps(struct venc_h264_inst *inst,
 	mtk_vcodec_debug_enter(inst);
 
 	ret = vpu_enc_encode(&inst->vpu_inst, H264_BS_MODE_PPS, NULL,
-			     bs_buf, bs_size, NULL);
+			     bs_buf, bs_size, NULL, MTK_VENC_CORE0);
 	if (ret)
 		return ret;
 
@@ -452,7 +452,7 @@ static int h264_encode_frame(struct venc_h264_inst *inst,
 			 frame_info.frm_count, frame_info.skip_frm_count,
 			 frame_info.frm_type);
 	ret = vpu_enc_encode(&inst->vpu_inst, H264_BS_MODE_FRAME, frm_buf,
-			     bs_buf, bs_size, &frame_info);
+			     bs_buf, bs_size, &frame_info, MTK_VENC_CORE0);
 	if (ret)
 		return ret;
 
diff --git a/drivers/media/platform/mtk-vcodec/venc/venc_vp8_if.c b/drivers/media/platform/mtk-vcodec/venc/venc_vp8_if.c
index 8267a9c4fd25..a68bf53d7d81 100644
--- a/drivers/media/platform/mtk-vcodec/venc/venc_vp8_if.c
+++ b/drivers/media/platform/mtk-vcodec/venc/venc_vp8_if.c
@@ -303,7 +303,7 @@ static int vp8_enc_encode_frame(struct venc_vp8_inst *inst,
 	mtk_vcodec_debug(inst, "->frm_cnt=%d", inst->frm_cnt);
 
 	ret = vpu_enc_encode(&inst->vpu_inst, 0, frm_buf, bs_buf, bs_size,
-			     NULL);
+			     NULL, MTK_VENC_CORE0);
 	if (ret)
 		return ret;
 
diff --git a/drivers/media/platform/mtk-vcodec/venc_drv_if.c b/drivers/media/platform/mtk-vcodec/venc_drv_if.c
index 93936d002bed..09fcec05a655 100644
--- a/drivers/media/platform/mtk-vcodec/venc_drv_if.c
+++ b/drivers/media/platform/mtk-vcodec/venc_drv_if.c
@@ -15,6 +15,7 @@
 
 #include "mtk_vcodec_enc.h"
 #include "mtk_vcodec_enc_pm.h"
+#include "mtk_vcodec_enc_hw.h"
 
 int venc_if_init(struct mtk_vcodec_ctx *ctx, unsigned int fourcc)
 {
@@ -34,9 +35,9 @@ int venc_if_init(struct mtk_vcodec_ctx *ctx, unsigned int fourcc)
 		return -EINVAL;
 	}
 
-	mtk_venc_lock(ctx);
+	mtk_venc_lock(ctx, 0);
 	ret = ctx->enc_if->init(ctx);
-	mtk_venc_unlock(ctx);
+	mtk_venc_unlock(ctx, 0);
 
 	return ret;
 }
@@ -46,9 +47,9 @@ int venc_if_set_param(struct mtk_vcodec_ctx *ctx,
 {
 	int ret = 0;
 
-	mtk_venc_lock(ctx);
+	mtk_venc_lock(ctx, 0);
 	ret = ctx->enc_if->set_param(ctx->drv_handle, type, in);
-	mtk_venc_unlock(ctx);
+	mtk_venc_unlock(ctx, 0);
 
 	return ret;
 }
@@ -59,24 +60,12 @@ int venc_if_encode(struct mtk_vcodec_ctx *ctx,
 		   struct venc_done_result *result)
 {
 	int ret = 0;
-	unsigned long flags;
-
-	mtk_venc_lock(ctx);
 
-	spin_lock_irqsave(&ctx->dev->irqlock, flags);
-	ctx->dev->curr_ctx = ctx;
-	spin_unlock_irqrestore(&ctx->dev->irqlock, flags);
-
-	mtk_vcodec_enc_clock_on(ctx->dev, 0);
+	venc_encode_prepare(ctx, opt);
 	ret = ctx->enc_if->encode(ctx->drv_handle, opt, frm_buf,
 				  bs_buf, result);
-	mtk_vcodec_enc_clock_off(ctx->dev, 0);
-
-	spin_lock_irqsave(&ctx->dev->irqlock, flags);
-	ctx->dev->curr_ctx = NULL;
-	spin_unlock_irqrestore(&ctx->dev->irqlock, flags);
+	venc_encode_unprepare(ctx, opt);
 
-	mtk_venc_unlock(ctx);
 	return ret;
 }
 
@@ -87,11 +76,67 @@ int venc_if_deinit(struct mtk_vcodec_ctx *ctx)
 	if (!ctx->drv_handle)
 		return 0;
 
-	mtk_venc_lock(ctx);
+	mtk_venc_lock(ctx, 0);
 	ret = ctx->enc_if->deinit(ctx->drv_handle);
-	mtk_venc_unlock(ctx);
+	mtk_venc_unlock(ctx, 0);
 
 	ctx->drv_handle = NULL;
 
 	return ret;
 }
+
+void venc_encode_prepare(struct mtk_vcodec_ctx *ctx,
+			 enum venc_start_opt opt)
+{
+	unsigned long flags;
+	struct mtk_venc_comp_dev *venc;
+
+	if (ctx->dev->venc_pdata->hw_mode == VENC_FRAME_RACING_MODE) {
+		if (ctx->enc_idx & 0x01)
+			ctx->hw_id = MTK_VENC_CORE1;
+		else
+			ctx->hw_id = MTK_VENC_CORE0;
+	} else {
+		ctx->hw_id = MTK_VENC_CORE0;
+	}
+	mtk_venc_lock(ctx, ctx->hw_id);
+
+	spin_lock_irqsave(&ctx->dev->irqlock, flags);
+
+	if (ctx->dev->venc_pdata->hw_mode == VENC_FRAME_RACING_MODE) {
+		venc = ctx->dev->enc_comp_dev[ctx->hw_id];
+
+		venc->curr_ctx = ctx;
+	} else {
+		ctx->dev->curr_ctx = ctx;
+	}
+
+	spin_unlock_irqrestore(&ctx->dev->irqlock, flags);
+
+	mtk_vcodec_enc_clock_on(ctx->dev, ctx->hw_id);
+}
+
+void venc_encode_unprepare(struct mtk_vcodec_ctx *ctx,
+			   enum venc_start_opt opt)
+{
+	unsigned long flags;
+	struct mtk_venc_comp_dev *venc;
+
+	/*clock off and unlock after irq done*/
+	if (ctx->dev->venc_pdata->hw_mode == VENC_FRAME_RACING_MODE) {
+		if (opt == VENC_START_OPT_ENCODE_SEQUENCE_HEADER) {
+			mtk_vcodec_enc_clock_off(ctx->dev, ctx->hw_id);
+			spin_lock_irqsave(&ctx->dev->irqlock, flags);
+			venc = ctx->dev->enc_comp_dev[ctx->hw_id];
+			venc->curr_ctx = NULL;
+			spin_unlock_irqrestore(&ctx->dev->irqlock, flags);
+			mtk_venc_unlock(ctx, ctx->hw_id);
+		}
+	} else {
+		mtk_vcodec_enc_clock_off(ctx->dev, ctx->hw_id);
+		spin_lock_irqsave(&ctx->dev->irqlock, flags);
+		ctx->dev->curr_ctx = NULL;
+		spin_unlock_irqrestore(&ctx->dev->irqlock, flags);
+		mtk_venc_unlock(ctx, ctx->hw_id);
+	}
+}
diff --git a/drivers/media/platform/mtk-vcodec/venc_drv_if.h b/drivers/media/platform/mtk-vcodec/venc_drv_if.h
index 8f384ba24ddf..7bd1f437d8e3 100644
--- a/drivers/media/platform/mtk-vcodec/venc_drv_if.h
+++ b/drivers/media/platform/mtk-vcodec/venc_drv_if.h
@@ -168,4 +168,8 @@ int venc_if_encode(struct mtk_vcodec_ctx *ctx,
 		   struct mtk_vcodec_mem *bs_buf,
 		   struct venc_done_result *result);
 
+void venc_encode_prepare(struct mtk_vcodec_ctx *ctx,
+			 enum venc_start_opt opt);
+void venc_encode_unprepare(struct mtk_vcodec_ctx *ctx,
+			   enum venc_start_opt opt);
 #endif /* _VENC_DRV_IF_H_ */
diff --git a/drivers/media/platform/mtk-vcodec/venc_vpu_if.c b/drivers/media/platform/mtk-vcodec/venc_vpu_if.c
index be6d8790a41e..5bee5f1828f1 100644
--- a/drivers/media/platform/mtk-vcodec/venc_vpu_if.c
+++ b/drivers/media/platform/mtk-vcodec/venc_vpu_if.c
@@ -226,9 +226,12 @@ int vpu_enc_encode(struct venc_vpu_inst *vpu, unsigned int bs_mode,
 		   struct venc_frm_buf *frm_buf,
 		   struct mtk_vcodec_mem *bs_buf,
 		   unsigned int *bs_size,
-		   struct venc_frame_info *frame_info)
+		   struct venc_frame_info *frame_info,
+		   int hw_id)
 {
 	const bool is_ext = MTK_ENC_CTX_IS_EXT(vpu->ctx);
+	const int hw_mode = MTK_ENC_HW_MODE(vpu->ctx);
+
 	size_t msg_size = is_ext ?
 		sizeof(struct venc_ap_ipi_msg_enc_ext) :
 		sizeof(struct venc_ap_ipi_msg_enc);
@@ -262,6 +265,12 @@ int vpu_enc_encode(struct venc_vpu_inst *vpu, unsigned int bs_mode,
 		out.data[1] = frame_info->skip_frm_count;
 		out.data[2] = frame_info->frm_type;
 	}
+
+	if (hw_mode == VENC_FRAME_RACING_MODE) {
+		out.data_item = 4;
+		out.data[3] = hw_id;
+	}
+
 	if (vpu_enc_send_msg(vpu, &out, msg_size)) {
 		mtk_vcodec_err(vpu, "AP_IPIMSG_ENC_ENCODE %d fail",
 			       bs_mode);
diff --git a/drivers/media/platform/mtk-vcodec/venc_vpu_if.h b/drivers/media/platform/mtk-vcodec/venc_vpu_if.h
index f9be9cab7ff7..e55656781f6e 100644
--- a/drivers/media/platform/mtk-vcodec/venc_vpu_if.h
+++ b/drivers/media/platform/mtk-vcodec/venc_vpu_if.h
@@ -46,7 +46,8 @@ int vpu_enc_encode(struct venc_vpu_inst *vpu, unsigned int bs_mode,
 		   struct venc_frm_buf *frm_buf,
 		   struct mtk_vcodec_mem *bs_buf,
 		   unsigned int *bs_size,
-		   struct venc_frame_info *frame_info);
+		   struct venc_frame_info *frame_info,
+		   int hw_id);
 int vpu_enc_deinit(struct venc_vpu_inst *vpu);
 
 #endif
-- 
2.25.1
_______________________________________________
Linux-mediatek mailing list
Linux-mediatek@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-mediatek

Re: [PATCH 7/9] media: mtk-vcodec: Add frame racing mode encode process

[Tzung-Bi Shih]

On Mon, Aug 16, 2021 at 06:59:32PM +0800, Irui Wang wrote:
> The frame_racing mode encoding is try to use the two venc cores:
s/is try/tries/

> frame#0 use core#0, frame#1 use core#1, frame#2 use core#0...,
s/use/uses/g

> Lock the device and enabe the clock by used core, for sequence
s/enabe/enable/

> header encoding, it always used core#0.
s/used/uses/

> --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
> +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
> @@ -273,6 +273,7 @@ struct vdec_pic_info {
>   * @decoded_frame_cnt: number of decoded frames
>   * @lock: protect variables accessed by V4L2 threads and worker thread such as
>   *	  mtk_video_dec_buf.
> + * @enc_idx: used to record encoded frame count
>   */
>  struct mtk_vcodec_ctx {
>  	enum mtk_instance_type type;
> @@ -313,6 +314,8 @@ struct mtk_vcodec_ctx {
>  	int decoded_frame_cnt;
>  	struct mutex lock;
>  
> +	int hw_id;
> +	int enc_idx;
hw_id lacks of kerneldoc which could introduce smatch warning.

> --- a/drivers/media/platform/mtk-vcodec/venc_drv_if.c
> +++ b/drivers/media/platform/mtk-vcodec/venc_drv_if.c
> @@ -15,6 +15,7 @@
>  
>  #include "mtk_vcodec_enc.h"
>  #include "mtk_vcodec_enc_pm.h"
> +#include "mtk_vcodec_enc_hw.h"
Please try to maintain the order.

> @@ -34,9 +35,9 @@ int venc_if_init(struct mtk_vcodec_ctx *ctx, unsigned int fourcc)
>  		return -EINVAL;
>  	}
>  
> -	mtk_venc_lock(ctx);
> +	mtk_venc_lock(ctx, 0);
Does it make more sense to use ctx->hw_id instead 0 (even if it is always 0 in the path)?

>  	ret = ctx->enc_if->init(ctx);
> -	mtk_venc_unlock(ctx);
> +	mtk_venc_unlock(ctx, 0);
Same.

> @@ -46,9 +47,9 @@ int venc_if_set_param(struct mtk_vcodec_ctx *ctx,
>  {
>  	int ret = 0;
>  
> -	mtk_venc_lock(ctx);
> +	mtk_venc_lock(ctx, 0);
Same.

>  	ret = ctx->enc_if->set_param(ctx->drv_handle, type, in);
> -	mtk_venc_unlock(ctx);
> +	mtk_venc_unlock(ctx, 0);
Same.

> @@ -87,11 +76,67 @@ int venc_if_deinit(struct mtk_vcodec_ctx *ctx)
>  	if (!ctx->drv_handle)
>  		return 0;
>  
> -	mtk_venc_lock(ctx);
> +	mtk_venc_lock(ctx, 0);
Same.

>  	ret = ctx->enc_if->deinit(ctx->drv_handle);
> -	mtk_venc_unlock(ctx);
> +	mtk_venc_unlock(ctx, 0);
Same.

> +void venc_encode_unprepare(struct mtk_vcodec_ctx *ctx,
> +			   enum venc_start_opt opt)
> +{
> +	unsigned long flags;
> +	struct mtk_venc_comp_dev *venc;
> +
> +	/*clock off and unlock after irq done*/
> +	if (ctx->dev->venc_pdata->hw_mode == VENC_FRAME_RACING_MODE) {
> +		if (opt == VENC_START_OPT_ENCODE_SEQUENCE_HEADER) {
> +			mtk_vcodec_enc_clock_off(ctx->dev, ctx->hw_id);
> +			spin_lock_irqsave(&ctx->dev->irqlock, flags);
> +			venc = ctx->dev->enc_comp_dev[ctx->hw_id];
> +			venc->curr_ctx = NULL;
> +			spin_unlock_irqrestore(&ctx->dev->irqlock, flags);
> +			mtk_venc_unlock(ctx, ctx->hw_id);
> +		}
> +	} else {
> +		mtk_vcodec_enc_clock_off(ctx->dev, ctx->hw_id);
> +		spin_lock_irqsave(&ctx->dev->irqlock, flags);
> +		ctx->dev->curr_ctx = NULL;
> +		spin_unlock_irqrestore(&ctx->dev->irqlock, flags);
> +		mtk_venc_unlock(ctx, ctx->hw_id);
The few statements are identical.  Should try to reuse them.

Re: [PATCH 7/9] media: mtk-vcodec: Add frame racing mode encode process

[Tzung-Bi Shih]

On Mon, Aug 16, 2021 at 06:59:32PM +0800, Irui Wang wrote:
> The frame_racing mode encoding is try to use the two venc cores:
s/is try/tries/

> frame#0 use core#0, frame#1 use core#1, frame#2 use core#0...,
s/use/uses/g

> Lock the device and enabe the clock by used core, for sequence
s/enabe/enable/

> header encoding, it always used core#0.
s/used/uses/

> --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
> +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
> @@ -273,6 +273,7 @@ struct vdec_pic_info {
>   * @decoded_frame_cnt: number of decoded frames
>   * @lock: protect variables accessed by V4L2 threads and worker thread such as
>   *	  mtk_video_dec_buf.
> + * @enc_idx: used to record encoded frame count
>   */
>  struct mtk_vcodec_ctx {
>  	enum mtk_instance_type type;
> @@ -313,6 +314,8 @@ struct mtk_vcodec_ctx {
>  	int decoded_frame_cnt;
>  	struct mutex lock;
>  
> +	int hw_id;
> +	int enc_idx;
hw_id lacks of kerneldoc which could introduce smatch warning.

> --- a/drivers/media/platform/mtk-vcodec/venc_drv_if.c
> +++ b/drivers/media/platform/mtk-vcodec/venc_drv_if.c
> @@ -15,6 +15,7 @@
>  
>  #include "mtk_vcodec_enc.h"
>  #include "mtk_vcodec_enc_pm.h"
> +#include "mtk_vcodec_enc_hw.h"
Please try to maintain the order.

> @@ -34,9 +35,9 @@ int venc_if_init(struct mtk_vcodec_ctx *ctx, unsigned int fourcc)
>  		return -EINVAL;
>  	}
>  
> -	mtk_venc_lock(ctx);
> +	mtk_venc_lock(ctx, 0);
Does it make more sense to use ctx->hw_id instead 0 (even if it is always 0 in the path)?

>  	ret = ctx->enc_if->init(ctx);
> -	mtk_venc_unlock(ctx);
> +	mtk_venc_unlock(ctx, 0);
Same.

> @@ -46,9 +47,9 @@ int venc_if_set_param(struct mtk_vcodec_ctx *ctx,
>  {
>  	int ret = 0;
>  
> -	mtk_venc_lock(ctx);
> +	mtk_venc_lock(ctx, 0);
Same.

>  	ret = ctx->enc_if->set_param(ctx->drv_handle, type, in);
> -	mtk_venc_unlock(ctx);
> +	mtk_venc_unlock(ctx, 0);
Same.

> @@ -87,11 +76,67 @@ int venc_if_deinit(struct mtk_vcodec_ctx *ctx)
>  	if (!ctx->drv_handle)
>  		return 0;
>  
> -	mtk_venc_lock(ctx);
> +	mtk_venc_lock(ctx, 0);
Same.

>  	ret = ctx->enc_if->deinit(ctx->drv_handle);
> -	mtk_venc_unlock(ctx);
> +	mtk_venc_unlock(ctx, 0);
Same.

> +void venc_encode_unprepare(struct mtk_vcodec_ctx *ctx,
> +			   enum venc_start_opt opt)
> +{
> +	unsigned long flags;
> +	struct mtk_venc_comp_dev *venc;
> +
> +	/*clock off and unlock after irq done*/
> +	if (ctx->dev->venc_pdata->hw_mode == VENC_FRAME_RACING_MODE) {
> +		if (opt == VENC_START_OPT_ENCODE_SEQUENCE_HEADER) {
> +			mtk_vcodec_enc_clock_off(ctx->dev, ctx->hw_id);
> +			spin_lock_irqsave(&ctx->dev->irqlock, flags);
> +			venc = ctx->dev->enc_comp_dev[ctx->hw_id];
> +			venc->curr_ctx = NULL;
> +			spin_unlock_irqrestore(&ctx->dev->irqlock, flags);
> +			mtk_venc_unlock(ctx, ctx->hw_id);
> +		}
> +	} else {
> +		mtk_vcodec_enc_clock_off(ctx->dev, ctx->hw_id);
> +		spin_lock_irqsave(&ctx->dev->irqlock, flags);
> +		ctx->dev->curr_ctx = NULL;
> +		spin_unlock_irqrestore(&ctx->dev->irqlock, flags);
> +		mtk_venc_unlock(ctx, ctx->hw_id);
The few statements are identical.  Should try to reuse them.

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH 7/9] media: mtk-vcodec: Add frame racing mode encode process

[Tzung-Bi Shih]

On Mon, Aug 16, 2021 at 06:59:32PM +0800, Irui Wang wrote:
> The frame_racing mode encoding is try to use the two venc cores:
s/is try/tries/

> frame#0 use core#0, frame#1 use core#1, frame#2 use core#0...,
s/use/uses/g

> Lock the device and enabe the clock by used core, for sequence
s/enabe/enable/

> header encoding, it always used core#0.
s/used/uses/

> --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
> +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
> @@ -273,6 +273,7 @@ struct vdec_pic_info {
>   * @decoded_frame_cnt: number of decoded frames
>   * @lock: protect variables accessed by V4L2 threads and worker thread such as
>   *	  mtk_video_dec_buf.
> + * @enc_idx: used to record encoded frame count
>   */
>  struct mtk_vcodec_ctx {
>  	enum mtk_instance_type type;
> @@ -313,6 +314,8 @@ struct mtk_vcodec_ctx {
>  	int decoded_frame_cnt;
>  	struct mutex lock;
>  
> +	int hw_id;
> +	int enc_idx;
hw_id lacks of kerneldoc which could introduce smatch warning.

> --- a/drivers/media/platform/mtk-vcodec/venc_drv_if.c
> +++ b/drivers/media/platform/mtk-vcodec/venc_drv_if.c
> @@ -15,6 +15,7 @@
>  
>  #include "mtk_vcodec_enc.h"
>  #include "mtk_vcodec_enc_pm.h"
> +#include "mtk_vcodec_enc_hw.h"
Please try to maintain the order.

> @@ -34,9 +35,9 @@ int venc_if_init(struct mtk_vcodec_ctx *ctx, unsigned int fourcc)
>  		return -EINVAL;
>  	}
>  
> -	mtk_venc_lock(ctx);
> +	mtk_venc_lock(ctx, 0);
Does it make more sense to use ctx->hw_id instead 0 (even if it is always 0 in the path)?

>  	ret = ctx->enc_if->init(ctx);
> -	mtk_venc_unlock(ctx);
> +	mtk_venc_unlock(ctx, 0);
Same.

> @@ -46,9 +47,9 @@ int venc_if_set_param(struct mtk_vcodec_ctx *ctx,
>  {
>  	int ret = 0;
>  
> -	mtk_venc_lock(ctx);
> +	mtk_venc_lock(ctx, 0);
Same.

>  	ret = ctx->enc_if->set_param(ctx->drv_handle, type, in);
> -	mtk_venc_unlock(ctx);
> +	mtk_venc_unlock(ctx, 0);
Same.

> @@ -87,11 +76,67 @@ int venc_if_deinit(struct mtk_vcodec_ctx *ctx)
>  	if (!ctx->drv_handle)
>  		return 0;
>  
> -	mtk_venc_lock(ctx);
> +	mtk_venc_lock(ctx, 0);
Same.

>  	ret = ctx->enc_if->deinit(ctx->drv_handle);
> -	mtk_venc_unlock(ctx);
> +	mtk_venc_unlock(ctx, 0);
Same.

> +void venc_encode_unprepare(struct mtk_vcodec_ctx *ctx,
> +			   enum venc_start_opt opt)
> +{
> +	unsigned long flags;
> +	struct mtk_venc_comp_dev *venc;
> +
> +	/*clock off and unlock after irq done*/
> +	if (ctx->dev->venc_pdata->hw_mode == VENC_FRAME_RACING_MODE) {
> +		if (opt == VENC_START_OPT_ENCODE_SEQUENCE_HEADER) {
> +			mtk_vcodec_enc_clock_off(ctx->dev, ctx->hw_id);
> +			spin_lock_irqsave(&ctx->dev->irqlock, flags);
> +			venc = ctx->dev->enc_comp_dev[ctx->hw_id];
> +			venc->curr_ctx = NULL;
> +			spin_unlock_irqrestore(&ctx->dev->irqlock, flags);
> +			mtk_venc_unlock(ctx, ctx->hw_id);
> +		}
> +	} else {
> +		mtk_vcodec_enc_clock_off(ctx->dev, ctx->hw_id);
> +		spin_lock_irqsave(&ctx->dev->irqlock, flags);
> +		ctx->dev->curr_ctx = NULL;
> +		spin_unlock_irqrestore(&ctx->dev->irqlock, flags);
> +		mtk_venc_unlock(ctx, ctx->hw_id);
The few statements are identical.  Should try to reuse them.

_______________________________________________
Linux-mediatek mailing list
Linux-mediatek@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-mediatek

[PATCH 8/9] media: mtk-vcodec: Return encode result to client

[Irui Wang]

For frame-racing mode encoding, there will no sync wait IRQ done
function, so the encode result can't return to client in
device_run. After device_run called to trigger encoding process,
finish it, so another scheduled job can be ran. When encoding
done, return the encode result to client in IRQ handler.

Signed-off-by: Irui Wang <irui.wang@mediatek.com>
---
 .../platform/mtk-vcodec/mtk_vcodec_drv.h      |  6 ++
 .../platform/mtk-vcodec/mtk_vcodec_enc.c      | 61 ++++++++++++++++++-
 .../platform/mtk-vcodec/mtk_vcodec_enc.h      |  4 +-
 .../platform/mtk-vcodec/mtk_vcodec_enc_hw.c   | 35 +++++++++++
 .../platform/mtk-vcodec/mtk_vcodec_util.h     |  1 +
 .../platform/mtk-vcodec/venc/venc_common_if.c |  9 +++
 .../media/platform/mtk-vcodec/venc_drv_if.h   |  2 +
 7 files changed, 115 insertions(+), 3 deletions(-)

diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
index 7a9c2dd52b91..c43321362896 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
@@ -274,6 +274,9 @@ struct vdec_pic_info {
  * @lock: protect variables accessed by V4L2 threads and worker thread such as
  *	  mtk_video_dec_buf.
  * @enc_idx: used to record encoded frame count
+ * @pfrm_buf: used to store current ctx's frame buffer information
+ * @pbs_buf: used to store current ctx's bitstream buffer information
+ * @hdr_size: used to store prepend header size
  */
 struct mtk_vcodec_ctx {
 	enum mtk_instance_type type;
@@ -316,6 +319,9 @@ struct mtk_vcodec_ctx {
 
 	int hw_id;
 	int enc_idx;
+	struct vb2_v4l2_buffer *pfrm_buf[MTK_VENC_HW_MAX];
+	struct vb2_v4l2_buffer *pbs_buf[MTK_VENC_HW_MAX];
+	unsigned int hdr_size;
 };
 
 enum mtk_chip {
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
index 6b37bd092fdf..de8e012f1b0e 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
@@ -867,6 +867,8 @@ static void vb2ops_venc_stop_streaming(struct vb2_queue *q)
 
 	mtk_v4l2_debug(2, "[%d]-> type=%d", ctx->id, q->type);
 
+	mtk_venc_lock_all(ctx);
+
 	if (q->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
 		while ((dst_buf = v4l2_m2m_dst_buf_remove(ctx->m2m_ctx))) {
 			dst_buf->vb2_buf.planes[0].bytesused = 0;
@@ -1014,10 +1016,13 @@ static int mtk_venc_param_change(struct mtk_vcodec_ctx *ctx)
 				ctx->id,
 				vb2_v4l2->vb2_buf.index,
 				mtk_buf->enc_params.force_intra);
-		if (mtk_buf->enc_params.force_intra)
+		if (mtk_buf->enc_params.force_intra) {
 			ret |= venc_if_set_param(ctx,
 						 VENC_SET_PARAM_FORCE_INTRA,
 						 NULL);
+
+			ctx->enc_idx = 0;
+		}
 	}
 
 	mtk_buf->param_change = MTK_ENCODE_PARAM_NONE;
@@ -1068,9 +1073,12 @@ static void mtk_venc_worker(struct work_struct *work)
 		frm_buf.fb_addr[i].size =
 				(size_t)src_buf->vb2_buf.planes[i].length;
 	}
+	frm_buf.src_addr = src_buf;
+
 	bs_buf.va = vb2_plane_vaddr(&dst_buf->vb2_buf, 0);
 	bs_buf.dma_addr = vb2_dma_contig_plane_dma_addr(&dst_buf->vb2_buf, 0);
 	bs_buf.size = (size_t)dst_buf->vb2_buf.planes[0].length;
+	bs_buf.buf = dst_buf;
 
 	mtk_v4l2_debug(2,
 			"Framebuf PA=%llx Size=0x%zx;PA=0x%llx Size=0x%zx;PA=0x%llx Size=%zu",
@@ -1119,6 +1127,8 @@ static void m2mops_venc_device_run(void *priv)
 {
 	struct mtk_vcodec_ctx *ctx = priv;
 
+	mtk_venc_param_change(ctx);
+
 	if ((ctx->q_data[MTK_Q_DATA_DST].fmt->fourcc == V4L2_PIX_FMT_H264) &&
 	    (ctx->state != MTK_STATE_HEADER)) {
 		/* encode h264 sps/pps header */
@@ -1127,7 +1137,6 @@ static void m2mops_venc_device_run(void *priv)
 		return;
 	}
 
-	mtk_venc_param_change(ctx);
 	queue_work(ctx->dev->encode_workqueue, &ctx->encode_work);
 }
 
@@ -1310,6 +1319,33 @@ int mtk_vcodec_enc_queue_init(void *priv, struct vb2_queue *src_vq,
 	return vb2_queue_init(dst_vq);
 }
 
+void mtk_venc_buf_done(struct mtk_vcodec_ctx *ctx, int core_id,
+		       unsigned int bs_size, bool time_out, bool key_frame)
+{
+	struct vb2_v4l2_buffer *src_vb2_v4l2 = NULL;
+	struct vb2_v4l2_buffer *dst_vb2_v4l2 = NULL;
+
+	/*
+	 * the frm_buf(src_buf) and bs_buf(dst_buf) can be obtained from ctx,
+	 * then put them to done list, user can get them by dqbuf call
+	 */
+	src_vb2_v4l2 = ctx->pfrm_buf[core_id];
+	dst_vb2_v4l2 = ctx->pbs_buf[core_id];
+
+	if (src_vb2_v4l2 && dst_vb2_v4l2) {
+		dst_vb2_v4l2->vb2_buf.timestamp =
+			src_vb2_v4l2->vb2_buf.timestamp;
+		dst_vb2_v4l2->timecode = src_vb2_v4l2->timecode;
+
+		if (key_frame)
+			dst_vb2_v4l2->flags |= V4L2_BUF_FLAG_KEYFRAME;
+
+		v4l2_m2m_buf_done(src_vb2_v4l2, VB2_BUF_STATE_DONE);
+		dst_vb2_v4l2->vb2_buf.planes[0].bytesused = bs_size;
+		v4l2_m2m_buf_done(dst_vb2_v4l2, VB2_BUF_STATE_DONE);
+	}
+}
+
 int mtk_venc_unlock(struct mtk_vcodec_ctx *ctx, int core_id)
 {
 	struct mtk_vcodec_dev *dev = ctx->dev;
@@ -1335,3 +1371,24 @@ void mtk_vcodec_enc_release(struct mtk_vcodec_ctx *ctx)
 
 	ctx->state = MTK_STATE_FREE;
 }
+
+void mtk_venc_lock_all(struct mtk_vcodec_ctx *ctx)
+{
+	unsigned int i;
+	struct mtk_vcodec_dev *dev = ctx->dev;
+
+	/*
+	 * For frame-racing mode encoding, there are may be bufs being encoded
+	 * when get the empty flush buffer or stop streaming, for example, the
+	 * buffer with LAST flag will return to client before the encoding
+	 * buffers, which will cause frame lost.
+	 * The encoder device mutex will be locked during encoding process,
+	 * when encode done, the mutex unlocked. So if all encoder device mutex
+	 * can be locked, which means there are no bufs being encoded at this
+	 * time, then the buffer with LAST flag can return to client properly.
+	 */
+	for (i = 0; i < MTK_VENC_HW_MAX; i++) {
+		mutex_lock(&dev->enc_mutex[i]);
+		mutex_unlock(&dev->enc_mutex[i]);
+	}
+}
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h
index 8d486f532025..1d603ccb2f3d 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h
@@ -48,5 +48,7 @@ int mtk_vcodec_enc_queue_init(void *priv, struct vb2_queue *src_vq,
 void mtk_vcodec_enc_release(struct mtk_vcodec_ctx *ctx);
 int mtk_vcodec_enc_ctrls_setup(struct mtk_vcodec_ctx *ctx);
 void mtk_vcodec_enc_set_default_params(struct mtk_vcodec_ctx *ctx);
-
+void mtk_venc_buf_done(struct mtk_vcodec_ctx *ctx, int core_id,
+		       unsigned int bs_size, bool time_out, bool key_frame);
+void mtk_venc_lock_all(struct mtk_vcodec_ctx *ctx);
 #endif /* _MTK_VCODEC_ENC_H_ */
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.c
index 2a9112b0561d..687b255c58e8 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.c
@@ -12,6 +12,17 @@
 #include "mtk_vcodec_enc_hw.h"
 #include "mtk_vcodec_enc_pm.h"
 #include "mtk_vcodec_enc.h"
+#define VENC_PIC_BITSTREAM_BYTE_CNT 0x0098
+#define VENC_PIC_FRM_TYPE	0x0010
+#define VENC_PIC_KEY_FRM	0x2
+
+/* Wake up core context wait_queue */
+static void mtk_venc_comp_wake_up_ctx(struct mtk_vcodec_ctx *ctx,
+				      unsigned int hw_id)
+{
+	ctx->int_cond = 1;
+	wake_up_interruptible(&ctx->queue);
+}
 
 static void clean_irq_status(unsigned int irq_status, void __iomem *addr)
 {
@@ -40,6 +51,9 @@ static irqreturn_t mtk_enc_comp_irq_handler(int irq, void *priv)
 	struct mtk_vcodec_ctx *ctx;
 	unsigned long flags;
 	void __iomem *addr;
+	unsigned int bs_size;
+	unsigned int frm_type;
+	bool is_key_frame = 0;
 
 	spin_lock_irqsave(&dev->master_dev->irqlock, flags);
 	ctx = dev->curr_ctx;
@@ -49,7 +63,28 @@ static irqreturn_t mtk_enc_comp_irq_handler(int irq, void *priv)
 
 	addr = dev->reg_base + MTK_VENC_IRQ_ACK_OFFSET;
 	ctx->irq_status = readl(dev->reg_base + MTK_VENC_IRQ_STATUS_OFFSET);
+	bs_size = readl(dev->reg_base + VENC_PIC_BITSTREAM_BYTE_CNT);
+	frm_type = readl(dev->reg_base + VENC_PIC_FRM_TYPE);
 	clean_irq_status(ctx->irq_status, addr);
+	if (ctx->irq_status & MTK_VENC_IRQ_STATUS_FRM) {
+		if (ctx->hdr_size != 0) {
+			bs_size += ctx->hdr_size;
+			ctx->hdr_size = 0;
+		}
+		if (frm_type & VENC_PIC_KEY_FRM)
+			is_key_frame = 1;
+
+		mtk_venc_buf_done(ctx, dev->core_id, bs_size, 0, is_key_frame);
+		mtk_vcodec_enc_clock_off(dev->master_dev, dev->core_id);
+
+		spin_lock_irqsave(&dev->master_dev->irqlock, flags);
+		dev->curr_ctx = NULL;
+		spin_unlock_irqrestore(&dev->master_dev->irqlock, flags);
+
+		mtk_venc_unlock(ctx, dev->core_id);
+	} else {
+		mtk_venc_comp_wake_up_ctx(ctx, dev->core_id);
+	}
 
 	return IRQ_HANDLED;
 }
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
index 0027747c0a27..7321b7537f20 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
@@ -15,6 +15,7 @@ struct mtk_vcodec_mem {
 	size_t size;
 	void *va;
 	dma_addr_t dma_addr;
+	void *buf;
 };
 
 struct mtk_vcodec_fb {
diff --git a/drivers/media/platform/mtk-vcodec/venc/venc_common_if.c b/drivers/media/platform/mtk-vcodec/venc/venc_common_if.c
index 1580bd1beaac..75b4e90232ae 100644
--- a/drivers/media/platform/mtk-vcodec/venc/venc_common_if.c
+++ b/drivers/media/platform/mtk-vcodec/venc/venc_common_if.c
@@ -481,6 +481,13 @@ static int venc_encode(void *handle,
 		unsigned int bs_size_hdr;
 		unsigned int bs_size_frm;
 
+		/*
+		 * the frm_buf and bs_buf need to be recorded into ctx,
+		 * when encoding done, the target buffer can be obtained
+		 */
+		ctx->pfrm_buf[ctx->hw_id] = frm_buf->src_addr;
+		ctx->pbs_buf[ctx->hw_id] = bs_buf->buf;
+
 		if (!inst->prepend_hdr) {
 			ret = venc_encode_frame(inst, frm_buf, bs_buf,
 						&result->bs_size, ctx->hw_id);
@@ -528,6 +535,8 @@ static int venc_encode(void *handle,
 
 		result->bs_size = hdr_sz + filler_sz + bs_size_frm;
 
+		ctx->hdr_size = hdr_sz + filler_sz;
+
 		mtk_vcodec_debug(inst, "hdr %d filler %d frame %d bs %d",
 				 hdr_sz, filler_sz, bs_size_frm,
 				 result->bs_size);
diff --git a/drivers/media/platform/mtk-vcodec/venc_drv_if.h b/drivers/media/platform/mtk-vcodec/venc_drv_if.h
index 7bd1f437d8e3..50824e718c91 100644
--- a/drivers/media/platform/mtk-vcodec/venc_drv_if.h
+++ b/drivers/media/platform/mtk-vcodec/venc_drv_if.h
@@ -108,9 +108,11 @@ struct venc_frame_info {
 /*
  * struct venc_frm_buf - frame buffer information used in venc_if_encode()
  * @fb_addr: plane frame buffer addresses
+ * @src_addr: vb2_v4l2_buffer src buffer address
  */
 struct venc_frm_buf {
 	struct mtk_vcodec_fb fb_addr[MTK_VCODEC_MAX_PLANES];
+	void *src_addr;
 };
 
 /*
-- 
2.25.1

[PATCH 8/9] media: mtk-vcodec: Return encode result to client

[Irui Wang]

For frame-racing mode encoding, there will no sync wait IRQ done
function, so the encode result can't return to client in
device_run. After device_run called to trigger encoding process,
finish it, so another scheduled job can be ran. When encoding
done, return the encode result to client in IRQ handler.

Signed-off-by: Irui Wang <irui.wang@mediatek.com>
---
 .../platform/mtk-vcodec/mtk_vcodec_drv.h      |  6 ++
 .../platform/mtk-vcodec/mtk_vcodec_enc.c      | 61 ++++++++++++++++++-
 .../platform/mtk-vcodec/mtk_vcodec_enc.h      |  4 +-
 .../platform/mtk-vcodec/mtk_vcodec_enc_hw.c   | 35 +++++++++++
 .../platform/mtk-vcodec/mtk_vcodec_util.h     |  1 +
 .../platform/mtk-vcodec/venc/venc_common_if.c |  9 +++
 .../media/platform/mtk-vcodec/venc_drv_if.h   |  2 +
 7 files changed, 115 insertions(+), 3 deletions(-)

diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
index 7a9c2dd52b91..c43321362896 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
@@ -274,6 +274,9 @@ struct vdec_pic_info {
  * @lock: protect variables accessed by V4L2 threads and worker thread such as
  *	  mtk_video_dec_buf.
  * @enc_idx: used to record encoded frame count
+ * @pfrm_buf: used to store current ctx's frame buffer information
+ * @pbs_buf: used to store current ctx's bitstream buffer information
+ * @hdr_size: used to store prepend header size
  */
 struct mtk_vcodec_ctx {
 	enum mtk_instance_type type;
@@ -316,6 +319,9 @@ struct mtk_vcodec_ctx {
 
 	int hw_id;
 	int enc_idx;
+	struct vb2_v4l2_buffer *pfrm_buf[MTK_VENC_HW_MAX];
+	struct vb2_v4l2_buffer *pbs_buf[MTK_VENC_HW_MAX];
+	unsigned int hdr_size;
 };
 
 enum mtk_chip {
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
index 6b37bd092fdf..de8e012f1b0e 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
@@ -867,6 +867,8 @@ static void vb2ops_venc_stop_streaming(struct vb2_queue *q)
 
 	mtk_v4l2_debug(2, "[%d]-> type=%d", ctx->id, q->type);
 
+	mtk_venc_lock_all(ctx);
+
 	if (q->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
 		while ((dst_buf = v4l2_m2m_dst_buf_remove(ctx->m2m_ctx))) {
 			dst_buf->vb2_buf.planes[0].bytesused = 0;
@@ -1014,10 +1016,13 @@ static int mtk_venc_param_change(struct mtk_vcodec_ctx *ctx)
 				ctx->id,
 				vb2_v4l2->vb2_buf.index,
 				mtk_buf->enc_params.force_intra);
-		if (mtk_buf->enc_params.force_intra)
+		if (mtk_buf->enc_params.force_intra) {
 			ret |= venc_if_set_param(ctx,
 						 VENC_SET_PARAM_FORCE_INTRA,
 						 NULL);
+
+			ctx->enc_idx = 0;
+		}
 	}
 
 	mtk_buf->param_change = MTK_ENCODE_PARAM_NONE;
@@ -1068,9 +1073,12 @@ static void mtk_venc_worker(struct work_struct *work)
 		frm_buf.fb_addr[i].size =
 				(size_t)src_buf->vb2_buf.planes[i].length;
 	}
+	frm_buf.src_addr = src_buf;
+
 	bs_buf.va = vb2_plane_vaddr(&dst_buf->vb2_buf, 0);
 	bs_buf.dma_addr = vb2_dma_contig_plane_dma_addr(&dst_buf->vb2_buf, 0);
 	bs_buf.size = (size_t)dst_buf->vb2_buf.planes[0].length;
+	bs_buf.buf = dst_buf;
 
 	mtk_v4l2_debug(2,
 			"Framebuf PA=%llx Size=0x%zx;PA=0x%llx Size=0x%zx;PA=0x%llx Size=%zu",
@@ -1119,6 +1127,8 @@ static void m2mops_venc_device_run(void *priv)
 {
 	struct mtk_vcodec_ctx *ctx = priv;
 
+	mtk_venc_param_change(ctx);
+
 	if ((ctx->q_data[MTK_Q_DATA_DST].fmt->fourcc == V4L2_PIX_FMT_H264) &&
 	    (ctx->state != MTK_STATE_HEADER)) {
 		/* encode h264 sps/pps header */
@@ -1127,7 +1137,6 @@ static void m2mops_venc_device_run(void *priv)
 		return;
 	}
 
-	mtk_venc_param_change(ctx);
 	queue_work(ctx->dev->encode_workqueue, &ctx->encode_work);
 }
 
@@ -1310,6 +1319,33 @@ int mtk_vcodec_enc_queue_init(void *priv, struct vb2_queue *src_vq,
 	return vb2_queue_init(dst_vq);
 }
 
+void mtk_venc_buf_done(struct mtk_vcodec_ctx *ctx, int core_id,
+		       unsigned int bs_size, bool time_out, bool key_frame)
+{
+	struct vb2_v4l2_buffer *src_vb2_v4l2 = NULL;
+	struct vb2_v4l2_buffer *dst_vb2_v4l2 = NULL;
+
+	/*
+	 * the frm_buf(src_buf) and bs_buf(dst_buf) can be obtained from ctx,
+	 * then put them to done list, user can get them by dqbuf call
+	 */
+	src_vb2_v4l2 = ctx->pfrm_buf[core_id];
+	dst_vb2_v4l2 = ctx->pbs_buf[core_id];
+
+	if (src_vb2_v4l2 && dst_vb2_v4l2) {
+		dst_vb2_v4l2->vb2_buf.timestamp =
+			src_vb2_v4l2->vb2_buf.timestamp;
+		dst_vb2_v4l2->timecode = src_vb2_v4l2->timecode;
+
+		if (key_frame)
+			dst_vb2_v4l2->flags |= V4L2_BUF_FLAG_KEYFRAME;
+
+		v4l2_m2m_buf_done(src_vb2_v4l2, VB2_BUF_STATE_DONE);
+		dst_vb2_v4l2->vb2_buf.planes[0].bytesused = bs_size;
+		v4l2_m2m_buf_done(dst_vb2_v4l2, VB2_BUF_STATE_DONE);
+	}
+}
+
 int mtk_venc_unlock(struct mtk_vcodec_ctx *ctx, int core_id)
 {
 	struct mtk_vcodec_dev *dev = ctx->dev;
@@ -1335,3 +1371,24 @@ void mtk_vcodec_enc_release(struct mtk_vcodec_ctx *ctx)
 
 	ctx->state = MTK_STATE_FREE;
 }
+
+void mtk_venc_lock_all(struct mtk_vcodec_ctx *ctx)
+{
+	unsigned int i;
+	struct mtk_vcodec_dev *dev = ctx->dev;
+
+	/*
+	 * For frame-racing mode encoding, there are may be bufs being encoded
+	 * when get the empty flush buffer or stop streaming, for example, the
+	 * buffer with LAST flag will return to client before the encoding
+	 * buffers, which will cause frame lost.
+	 * The encoder device mutex will be locked during encoding process,
+	 * when encode done, the mutex unlocked. So if all encoder device mutex
+	 * can be locked, which means there are no bufs being encoded at this
+	 * time, then the buffer with LAST flag can return to client properly.
+	 */
+	for (i = 0; i < MTK_VENC_HW_MAX; i++) {
+		mutex_lock(&dev->enc_mutex[i]);
+		mutex_unlock(&dev->enc_mutex[i]);
+	}
+}
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h
index 8d486f532025..1d603ccb2f3d 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h
@@ -48,5 +48,7 @@ int mtk_vcodec_enc_queue_init(void *priv, struct vb2_queue *src_vq,
 void mtk_vcodec_enc_release(struct mtk_vcodec_ctx *ctx);
 int mtk_vcodec_enc_ctrls_setup(struct mtk_vcodec_ctx *ctx);
 void mtk_vcodec_enc_set_default_params(struct mtk_vcodec_ctx *ctx);
-
+void mtk_venc_buf_done(struct mtk_vcodec_ctx *ctx, int core_id,
+		       unsigned int bs_size, bool time_out, bool key_frame);
+void mtk_venc_lock_all(struct mtk_vcodec_ctx *ctx);
 #endif /* _MTK_VCODEC_ENC_H_ */
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.c
index 2a9112b0561d..687b255c58e8 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.c
@@ -12,6 +12,17 @@
 #include "mtk_vcodec_enc_hw.h"
 #include "mtk_vcodec_enc_pm.h"
 #include "mtk_vcodec_enc.h"
+#define VENC_PIC_BITSTREAM_BYTE_CNT 0x0098
+#define VENC_PIC_FRM_TYPE	0x0010
+#define VENC_PIC_KEY_FRM	0x2
+
+/* Wake up core context wait_queue */
+static void mtk_venc_comp_wake_up_ctx(struct mtk_vcodec_ctx *ctx,
+				      unsigned int hw_id)
+{
+	ctx->int_cond = 1;
+	wake_up_interruptible(&ctx->queue);
+}
 
 static void clean_irq_status(unsigned int irq_status, void __iomem *addr)
 {
@@ -40,6 +51,9 @@ static irqreturn_t mtk_enc_comp_irq_handler(int irq, void *priv)
 	struct mtk_vcodec_ctx *ctx;
 	unsigned long flags;
 	void __iomem *addr;
+	unsigned int bs_size;
+	unsigned int frm_type;
+	bool is_key_frame = 0;
 
 	spin_lock_irqsave(&dev->master_dev->irqlock, flags);
 	ctx = dev->curr_ctx;
@@ -49,7 +63,28 @@ static irqreturn_t mtk_enc_comp_irq_handler(int irq, void *priv)
 
 	addr = dev->reg_base + MTK_VENC_IRQ_ACK_OFFSET;
 	ctx->irq_status = readl(dev->reg_base + MTK_VENC_IRQ_STATUS_OFFSET);
+	bs_size = readl(dev->reg_base + VENC_PIC_BITSTREAM_BYTE_CNT);
+	frm_type = readl(dev->reg_base + VENC_PIC_FRM_TYPE);
 	clean_irq_status(ctx->irq_status, addr);
+	if (ctx->irq_status & MTK_VENC_IRQ_STATUS_FRM) {
+		if (ctx->hdr_size != 0) {
+			bs_size += ctx->hdr_size;
+			ctx->hdr_size = 0;
+		}
+		if (frm_type & VENC_PIC_KEY_FRM)
+			is_key_frame = 1;
+
+		mtk_venc_buf_done(ctx, dev->core_id, bs_size, 0, is_key_frame);
+		mtk_vcodec_enc_clock_off(dev->master_dev, dev->core_id);
+
+		spin_lock_irqsave(&dev->master_dev->irqlock, flags);
+		dev->curr_ctx = NULL;
+		spin_unlock_irqrestore(&dev->master_dev->irqlock, flags);
+
+		mtk_venc_unlock(ctx, dev->core_id);
+	} else {
+		mtk_venc_comp_wake_up_ctx(ctx, dev->core_id);
+	}
 
 	return IRQ_HANDLED;
 }
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
index 0027747c0a27..7321b7537f20 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
@@ -15,6 +15,7 @@ struct mtk_vcodec_mem {
 	size_t size;
 	void *va;
 	dma_addr_t dma_addr;
+	void *buf;
 };
 
 struct mtk_vcodec_fb {
diff --git a/drivers/media/platform/mtk-vcodec/venc/venc_common_if.c b/drivers/media/platform/mtk-vcodec/venc/venc_common_if.c
index 1580bd1beaac..75b4e90232ae 100644
--- a/drivers/media/platform/mtk-vcodec/venc/venc_common_if.c
+++ b/drivers/media/platform/mtk-vcodec/venc/venc_common_if.c
@@ -481,6 +481,13 @@ static int venc_encode(void *handle,
 		unsigned int bs_size_hdr;
 		unsigned int bs_size_frm;
 
+		/*
+		 * the frm_buf and bs_buf need to be recorded into ctx,
+		 * when encoding done, the target buffer can be obtained
+		 */
+		ctx->pfrm_buf[ctx->hw_id] = frm_buf->src_addr;
+		ctx->pbs_buf[ctx->hw_id] = bs_buf->buf;
+
 		if (!inst->prepend_hdr) {
 			ret = venc_encode_frame(inst, frm_buf, bs_buf,
 						&result->bs_size, ctx->hw_id);
@@ -528,6 +535,8 @@ static int venc_encode(void *handle,
 
 		result->bs_size = hdr_sz + filler_sz + bs_size_frm;
 
+		ctx->hdr_size = hdr_sz + filler_sz;
+
 		mtk_vcodec_debug(inst, "hdr %d filler %d frame %d bs %d",
 				 hdr_sz, filler_sz, bs_size_frm,
 				 result->bs_size);
diff --git a/drivers/media/platform/mtk-vcodec/venc_drv_if.h b/drivers/media/platform/mtk-vcodec/venc_drv_if.h
index 7bd1f437d8e3..50824e718c91 100644
--- a/drivers/media/platform/mtk-vcodec/venc_drv_if.h
+++ b/drivers/media/platform/mtk-vcodec/venc_drv_if.h
@@ -108,9 +108,11 @@ struct venc_frame_info {
 /*
  * struct venc_frm_buf - frame buffer information used in venc_if_encode()
  * @fb_addr: plane frame buffer addresses
+ * @src_addr: vb2_v4l2_buffer src buffer address
  */
 struct venc_frm_buf {
 	struct mtk_vcodec_fb fb_addr[MTK_VCODEC_MAX_PLANES];
+	void *src_addr;
 };
 
 /*
-- 
2.25.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH 8/9] media: mtk-vcodec: Return encode result to client

[Irui Wang]

For frame-racing mode encoding, there will no sync wait IRQ done
function, so the encode result can't return to client in
device_run. After device_run called to trigger encoding process,
finish it, so another scheduled job can be ran. When encoding
done, return the encode result to client in IRQ handler.

Signed-off-by: Irui Wang <irui.wang@mediatek.com>
---
 .../platform/mtk-vcodec/mtk_vcodec_drv.h      |  6 ++
 .../platform/mtk-vcodec/mtk_vcodec_enc.c      | 61 ++++++++++++++++++-
 .../platform/mtk-vcodec/mtk_vcodec_enc.h      |  4 +-
 .../platform/mtk-vcodec/mtk_vcodec_enc_hw.c   | 35 +++++++++++
 .../platform/mtk-vcodec/mtk_vcodec_util.h     |  1 +
 .../platform/mtk-vcodec/venc/venc_common_if.c |  9 +++
 .../media/platform/mtk-vcodec/venc_drv_if.h   |  2 +
 7 files changed, 115 insertions(+), 3 deletions(-)

diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
index 7a9c2dd52b91..c43321362896 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
@@ -274,6 +274,9 @@ struct vdec_pic_info {
  * @lock: protect variables accessed by V4L2 threads and worker thread such as
  *	  mtk_video_dec_buf.
  * @enc_idx: used to record encoded frame count
+ * @pfrm_buf: used to store current ctx's frame buffer information
+ * @pbs_buf: used to store current ctx's bitstream buffer information
+ * @hdr_size: used to store prepend header size
  */
 struct mtk_vcodec_ctx {
 	enum mtk_instance_type type;
@@ -316,6 +319,9 @@ struct mtk_vcodec_ctx {
 
 	int hw_id;
 	int enc_idx;
+	struct vb2_v4l2_buffer *pfrm_buf[MTK_VENC_HW_MAX];
+	struct vb2_v4l2_buffer *pbs_buf[MTK_VENC_HW_MAX];
+	unsigned int hdr_size;
 };
 
 enum mtk_chip {
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
index 6b37bd092fdf..de8e012f1b0e 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
@@ -867,6 +867,8 @@ static void vb2ops_venc_stop_streaming(struct vb2_queue *q)
 
 	mtk_v4l2_debug(2, "[%d]-> type=%d", ctx->id, q->type);
 
+	mtk_venc_lock_all(ctx);
+
 	if (q->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
 		while ((dst_buf = v4l2_m2m_dst_buf_remove(ctx->m2m_ctx))) {
 			dst_buf->vb2_buf.planes[0].bytesused = 0;
@@ -1014,10 +1016,13 @@ static int mtk_venc_param_change(struct mtk_vcodec_ctx *ctx)
 				ctx->id,
 				vb2_v4l2->vb2_buf.index,
 				mtk_buf->enc_params.force_intra);
-		if (mtk_buf->enc_params.force_intra)
+		if (mtk_buf->enc_params.force_intra) {
 			ret |= venc_if_set_param(ctx,
 						 VENC_SET_PARAM_FORCE_INTRA,
 						 NULL);
+
+			ctx->enc_idx = 0;
+		}
 	}
 
 	mtk_buf->param_change = MTK_ENCODE_PARAM_NONE;
@@ -1068,9 +1073,12 @@ static void mtk_venc_worker(struct work_struct *work)
 		frm_buf.fb_addr[i].size =
 				(size_t)src_buf->vb2_buf.planes[i].length;
 	}
+	frm_buf.src_addr = src_buf;
+
 	bs_buf.va = vb2_plane_vaddr(&dst_buf->vb2_buf, 0);
 	bs_buf.dma_addr = vb2_dma_contig_plane_dma_addr(&dst_buf->vb2_buf, 0);
 	bs_buf.size = (size_t)dst_buf->vb2_buf.planes[0].length;
+	bs_buf.buf = dst_buf;
 
 	mtk_v4l2_debug(2,
 			"Framebuf PA=%llx Size=0x%zx;PA=0x%llx Size=0x%zx;PA=0x%llx Size=%zu",
@@ -1119,6 +1127,8 @@ static void m2mops_venc_device_run(void *priv)
 {
 	struct mtk_vcodec_ctx *ctx = priv;
 
+	mtk_venc_param_change(ctx);
+
 	if ((ctx->q_data[MTK_Q_DATA_DST].fmt->fourcc == V4L2_PIX_FMT_H264) &&
 	    (ctx->state != MTK_STATE_HEADER)) {
 		/* encode h264 sps/pps header */
@@ -1127,7 +1137,6 @@ static void m2mops_venc_device_run(void *priv)
 		return;
 	}
 
-	mtk_venc_param_change(ctx);
 	queue_work(ctx->dev->encode_workqueue, &ctx->encode_work);
 }
 
@@ -1310,6 +1319,33 @@ int mtk_vcodec_enc_queue_init(void *priv, struct vb2_queue *src_vq,
 	return vb2_queue_init(dst_vq);
 }
 
+void mtk_venc_buf_done(struct mtk_vcodec_ctx *ctx, int core_id,
+		       unsigned int bs_size, bool time_out, bool key_frame)
+{
+	struct vb2_v4l2_buffer *src_vb2_v4l2 = NULL;
+	struct vb2_v4l2_buffer *dst_vb2_v4l2 = NULL;
+
+	/*
+	 * the frm_buf(src_buf) and bs_buf(dst_buf) can be obtained from ctx,
+	 * then put them to done list, user can get them by dqbuf call
+	 */
+	src_vb2_v4l2 = ctx->pfrm_buf[core_id];
+	dst_vb2_v4l2 = ctx->pbs_buf[core_id];
+
+	if (src_vb2_v4l2 && dst_vb2_v4l2) {
+		dst_vb2_v4l2->vb2_buf.timestamp =
+			src_vb2_v4l2->vb2_buf.timestamp;
+		dst_vb2_v4l2->timecode = src_vb2_v4l2->timecode;
+
+		if (key_frame)
+			dst_vb2_v4l2->flags |= V4L2_BUF_FLAG_KEYFRAME;
+
+		v4l2_m2m_buf_done(src_vb2_v4l2, VB2_BUF_STATE_DONE);
+		dst_vb2_v4l2->vb2_buf.planes[0].bytesused = bs_size;
+		v4l2_m2m_buf_done(dst_vb2_v4l2, VB2_BUF_STATE_DONE);
+	}
+}
+
 int mtk_venc_unlock(struct mtk_vcodec_ctx *ctx, int core_id)
 {
 	struct mtk_vcodec_dev *dev = ctx->dev;
@@ -1335,3 +1371,24 @@ void mtk_vcodec_enc_release(struct mtk_vcodec_ctx *ctx)
 
 	ctx->state = MTK_STATE_FREE;
 }
+
+void mtk_venc_lock_all(struct mtk_vcodec_ctx *ctx)
+{
+	unsigned int i;
+	struct mtk_vcodec_dev *dev = ctx->dev;
+
+	/*
+	 * For frame-racing mode encoding, there are may be bufs being encoded
+	 * when get the empty flush buffer or stop streaming, for example, the
+	 * buffer with LAST flag will return to client before the encoding
+	 * buffers, which will cause frame lost.
+	 * The encoder device mutex will be locked during encoding process,
+	 * when encode done, the mutex unlocked. So if all encoder device mutex
+	 * can be locked, which means there are no bufs being encoded at this
+	 * time, then the buffer with LAST flag can return to client properly.
+	 */
+	for (i = 0; i < MTK_VENC_HW_MAX; i++) {
+		mutex_lock(&dev->enc_mutex[i]);
+		mutex_unlock(&dev->enc_mutex[i]);
+	}
+}
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h
index 8d486f532025..1d603ccb2f3d 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h
@@ -48,5 +48,7 @@ int mtk_vcodec_enc_queue_init(void *priv, struct vb2_queue *src_vq,
 void mtk_vcodec_enc_release(struct mtk_vcodec_ctx *ctx);
 int mtk_vcodec_enc_ctrls_setup(struct mtk_vcodec_ctx *ctx);
 void mtk_vcodec_enc_set_default_params(struct mtk_vcodec_ctx *ctx);
-
+void mtk_venc_buf_done(struct mtk_vcodec_ctx *ctx, int core_id,
+		       unsigned int bs_size, bool time_out, bool key_frame);
+void mtk_venc_lock_all(struct mtk_vcodec_ctx *ctx);
 #endif /* _MTK_VCODEC_ENC_H_ */
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.c
index 2a9112b0561d..687b255c58e8 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.c
@@ -12,6 +12,17 @@
 #include "mtk_vcodec_enc_hw.h"
 #include "mtk_vcodec_enc_pm.h"
 #include "mtk_vcodec_enc.h"
+#define VENC_PIC_BITSTREAM_BYTE_CNT 0x0098
+#define VENC_PIC_FRM_TYPE	0x0010
+#define VENC_PIC_KEY_FRM	0x2
+
+/* Wake up core context wait_queue */
+static void mtk_venc_comp_wake_up_ctx(struct mtk_vcodec_ctx *ctx,
+				      unsigned int hw_id)
+{
+	ctx->int_cond = 1;
+	wake_up_interruptible(&ctx->queue);
+}
 
 static void clean_irq_status(unsigned int irq_status, void __iomem *addr)
 {
@@ -40,6 +51,9 @@ static irqreturn_t mtk_enc_comp_irq_handler(int irq, void *priv)
 	struct mtk_vcodec_ctx *ctx;
 	unsigned long flags;
 	void __iomem *addr;
+	unsigned int bs_size;
+	unsigned int frm_type;
+	bool is_key_frame = 0;
 
 	spin_lock_irqsave(&dev->master_dev->irqlock, flags);
 	ctx = dev->curr_ctx;
@@ -49,7 +63,28 @@ static irqreturn_t mtk_enc_comp_irq_handler(int irq, void *priv)
 
 	addr = dev->reg_base + MTK_VENC_IRQ_ACK_OFFSET;
 	ctx->irq_status = readl(dev->reg_base + MTK_VENC_IRQ_STATUS_OFFSET);
+	bs_size = readl(dev->reg_base + VENC_PIC_BITSTREAM_BYTE_CNT);
+	frm_type = readl(dev->reg_base + VENC_PIC_FRM_TYPE);
 	clean_irq_status(ctx->irq_status, addr);
+	if (ctx->irq_status & MTK_VENC_IRQ_STATUS_FRM) {
+		if (ctx->hdr_size != 0) {
+			bs_size += ctx->hdr_size;
+			ctx->hdr_size = 0;
+		}
+		if (frm_type & VENC_PIC_KEY_FRM)
+			is_key_frame = 1;
+
+		mtk_venc_buf_done(ctx, dev->core_id, bs_size, 0, is_key_frame);
+		mtk_vcodec_enc_clock_off(dev->master_dev, dev->core_id);
+
+		spin_lock_irqsave(&dev->master_dev->irqlock, flags);
+		dev->curr_ctx = NULL;
+		spin_unlock_irqrestore(&dev->master_dev->irqlock, flags);
+
+		mtk_venc_unlock(ctx, dev->core_id);
+	} else {
+		mtk_venc_comp_wake_up_ctx(ctx, dev->core_id);
+	}
 
 	return IRQ_HANDLED;
 }
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
index 0027747c0a27..7321b7537f20 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
@@ -15,6 +15,7 @@ struct mtk_vcodec_mem {
 	size_t size;
 	void *va;
 	dma_addr_t dma_addr;
+	void *buf;
 };
 
 struct mtk_vcodec_fb {
diff --git a/drivers/media/platform/mtk-vcodec/venc/venc_common_if.c b/drivers/media/platform/mtk-vcodec/venc/venc_common_if.c
index 1580bd1beaac..75b4e90232ae 100644
--- a/drivers/media/platform/mtk-vcodec/venc/venc_common_if.c
+++ b/drivers/media/platform/mtk-vcodec/venc/venc_common_if.c
@@ -481,6 +481,13 @@ static int venc_encode(void *handle,
 		unsigned int bs_size_hdr;
 		unsigned int bs_size_frm;
 
+		/*
+		 * the frm_buf and bs_buf need to be recorded into ctx,
+		 * when encoding done, the target buffer can be obtained
+		 */
+		ctx->pfrm_buf[ctx->hw_id] = frm_buf->src_addr;
+		ctx->pbs_buf[ctx->hw_id] = bs_buf->buf;
+
 		if (!inst->prepend_hdr) {
 			ret = venc_encode_frame(inst, frm_buf, bs_buf,
 						&result->bs_size, ctx->hw_id);
@@ -528,6 +535,8 @@ static int venc_encode(void *handle,
 
 		result->bs_size = hdr_sz + filler_sz + bs_size_frm;
 
+		ctx->hdr_size = hdr_sz + filler_sz;
+
 		mtk_vcodec_debug(inst, "hdr %d filler %d frame %d bs %d",
 				 hdr_sz, filler_sz, bs_size_frm,
 				 result->bs_size);
diff --git a/drivers/media/platform/mtk-vcodec/venc_drv_if.h b/drivers/media/platform/mtk-vcodec/venc_drv_if.h
index 7bd1f437d8e3..50824e718c91 100644
--- a/drivers/media/platform/mtk-vcodec/venc_drv_if.h
+++ b/drivers/media/platform/mtk-vcodec/venc_drv_if.h
@@ -108,9 +108,11 @@ struct venc_frame_info {
 /*
  * struct venc_frm_buf - frame buffer information used in venc_if_encode()
  * @fb_addr: plane frame buffer addresses
+ * @src_addr: vb2_v4l2_buffer src buffer address
  */
 struct venc_frm_buf {
 	struct mtk_vcodec_fb fb_addr[MTK_VCODEC_MAX_PLANES];
+	void *src_addr;
 };
 
 /*
-- 
2.25.1
_______________________________________________
Linux-mediatek mailing list
Linux-mediatek@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-mediatek

[PATCH 9/9] media: mtk-vcodec: Add delayed worker for encode timeout

[Irui Wang]

When encoding timeout, a delayed timeout worker is needed because
of asynchronous encoding process, restore encoding process in
timeout worker.

Signed-off-by: Irui Wang <irui.wang@mediatek.com>
---
 .../platform/mtk-vcodec/mtk_vcodec_enc.c      | 12 ++++--
 .../platform/mtk-vcodec/mtk_vcodec_enc_hw.c   | 26 +++++++++++
 .../platform/mtk-vcodec/mtk_vcodec_enc_hw.h   |  2 +
 .../platform/mtk-vcodec/venc/venc_common_if.c | 43 +++++++++++++++++--
 4 files changed, 77 insertions(+), 6 deletions(-)

diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
index de8e012f1b0e..bc635436aa90 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
@@ -1340,9 +1340,15 @@ void mtk_venc_buf_done(struct mtk_vcodec_ctx *ctx, int core_id,
 		if (key_frame)
 			dst_vb2_v4l2->flags |= V4L2_BUF_FLAG_KEYFRAME;
 
-		v4l2_m2m_buf_done(src_vb2_v4l2, VB2_BUF_STATE_DONE);
-		dst_vb2_v4l2->vb2_buf.planes[0].bytesused = bs_size;
-		v4l2_m2m_buf_done(dst_vb2_v4l2, VB2_BUF_STATE_DONE);
+		if (time_out) {
+			v4l2_m2m_buf_done(src_vb2_v4l2, VB2_BUF_STATE_ERROR);
+			dst_vb2_v4l2->vb2_buf.planes[0].bytesused = 0;
+			v4l2_m2m_buf_done(dst_vb2_v4l2, VB2_BUF_STATE_ERROR);
+		} else {
+			v4l2_m2m_buf_done(src_vb2_v4l2, VB2_BUF_STATE_DONE);
+			dst_vb2_v4l2->vb2_buf.planes[0].bytesused = bs_size;
+			v4l2_m2m_buf_done(dst_vb2_v4l2, VB2_BUF_STATE_DONE);
+		}
 	}
 }
 
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.c
index 687b255c58e8..f6a5dfa6929f 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.c
@@ -16,6 +16,28 @@
 #define VENC_PIC_FRM_TYPE	0x0010
 #define VENC_PIC_KEY_FRM	0x2
 
+static void mtk_venc_timeout_worker(struct work_struct *work)
+{
+	struct mtk_venc_comp_dev *dev = container_of(work,
+						     struct mtk_venc_comp_dev,
+						     enc_timeout_worker.work);
+
+	struct mtk_vcodec_ctx *ctx;
+	unsigned long flags;
+
+	spin_lock_irqsave(&dev->master_dev->irqlock, flags);
+	ctx = dev->curr_ctx;
+
+	dev_err(&dev->plat_dev->dev, "core: %d timeout\n", dev->core_id);
+
+	mtk_venc_buf_done(ctx, dev->core_id, 0, 1, 0);
+
+	mtk_vcodec_enc_clock_off(dev->master_dev, dev->core_id);
+	dev->curr_ctx = NULL;
+	spin_unlock_irqrestore(&dev->master_dev->irqlock, flags);
+
+	mtk_venc_unlock(ctx, dev->core_id);
+}
 /* Wake up core context wait_queue */
 static void mtk_venc_comp_wake_up_ctx(struct mtk_vcodec_ctx *ctx,
 				      unsigned int hw_id)
@@ -67,6 +89,7 @@ static irqreturn_t mtk_enc_comp_irq_handler(int irq, void *priv)
 	frm_type = readl(dev->reg_base + VENC_PIC_FRM_TYPE);
 	clean_irq_status(ctx->irq_status, addr);
 	if (ctx->irq_status & MTK_VENC_IRQ_STATUS_FRM) {
+		cancel_delayed_work(&dev->enc_timeout_worker);
 		if (ctx->hdr_size != 0) {
 			bs_size += ctx->hdr_size;
 			ctx->hdr_size = 0;
@@ -182,6 +205,9 @@ static int mtk_venc_comp_probe(struct platform_device *pdev)
 
 	platform_set_drvdata(pdev, comp_dev);
 
+	INIT_DELAYED_WORK(&comp_dev->enc_timeout_worker,
+			  mtk_venc_timeout_worker);
+
 	ret = component_add(&pdev->dev, &mtk_venc_component_ops);
 	if (ret) {
 		dev_err(&pdev->dev, "Failed to add component: %d\n", ret);
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.h
index f9ae97e252dc..e69344e38ca6 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.h
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.h
@@ -19,6 +19,7 @@
  * @irq_status: venc hw irq status
  * @enc_irq: venc device irq
  * @core id: for venc core id: core#0, core#1...
+ * @enc_timeout_worker: venc timeout worker
  */
 struct mtk_venc_comp_dev {
 	struct platform_device *plat_dev;
@@ -31,6 +32,7 @@ struct mtk_venc_comp_dev {
 	unsigned int irq_status;
 	int enc_irq;
 	int core_id;
+	struct delayed_work enc_timeout_worker;
 };
 
 #endif /* _MTK_VCODEC_ENC_HW_H_ */
diff --git a/drivers/media/platform/mtk-vcodec/venc/venc_common_if.c b/drivers/media/platform/mtk-vcodec/venc/venc_common_if.c
index 75b4e90232ae..1aab84faadf0 100644
--- a/drivers/media/platform/mtk-vcodec/venc/venc_common_if.c
+++ b/drivers/media/platform/mtk-vcodec/venc/venc_common_if.c
@@ -10,11 +10,13 @@
 #include "../mtk_vcodec_drv.h"
 #include "../mtk_vcodec_util.h"
 #include "../mtk_vcodec_enc_hw.h"
+#include "../mtk_vcodec_enc_pm.h"
 
 static const char h264_filler_marker[] = {0x0, 0x0, 0x0, 0x1, 0xc};
 
 #define H264_FILLER_MARKER_SIZE ARRAY_SIZE(h264_filler_marker)
 #define VENC_PIC_BITSTREAM_BYTE_CNT 0x0098
+#define MTK_VENC_HW_TIMEOUT_MSEC 2000
 
 enum venc_vpu_work_buf {
 	VENC_VPU_WORK_BUF_RC_INFO_CORE0,
@@ -413,6 +415,34 @@ static void h264_encode_filler(struct venc_common_inst *inst, void *buf,
 	memset(p, 0xff, size);
 }
 
+static void init_schedule_work(struct mtk_vcodec_ctx *ctx)
+{
+	unsigned long timeout;
+	struct mtk_venc_comp_dev *venc;
+
+	venc = ctx->dev->enc_comp_dev[ctx->hw_id];
+	timeout = msecs_to_jiffies(MTK_VENC_HW_TIMEOUT_MSEC);
+	schedule_delayed_work(&venc->enc_timeout_worker, timeout);
+}
+
+static void cancel_schedule_work(struct mtk_vcodec_ctx *ctx)
+{
+	struct mtk_venc_comp_dev *venc;
+	unsigned long flags;
+
+	venc = ctx->dev->enc_comp_dev[ctx->hw_id];
+	cancel_delayed_work(&venc->enc_timeout_worker);
+
+	mtk_venc_buf_done(ctx, ctx->hw_id, 0, 1, 0);
+	mtk_vcodec_enc_clock_off(ctx->dev, ctx->hw_id);
+
+	spin_lock_irqsave(&ctx->dev->irqlock, flags);
+	venc->curr_ctx = NULL;
+	spin_unlock_irqrestore(&ctx->dev->irqlock, flags);
+
+	mtk_venc_unlock(ctx, ctx->hw_id);
+}
+
 static int venc_init(struct mtk_vcodec_ctx *ctx)
 {
 	int i;
@@ -489,12 +519,13 @@ static int venc_encode(void *handle,
 		ctx->pbs_buf[ctx->hw_id] = bs_buf->buf;
 
 		if (!inst->prepend_hdr) {
+			init_schedule_work(ctx);
 			ret = venc_encode_frame(inst, frm_buf, bs_buf,
 						&result->bs_size, ctx->hw_id);
 			if (ret) {
 				mtk_vcodec_err(inst, "encode frame failed: %d",
 					       ret);
-				return ret;
+				goto encode_err;
 			}
 
 			result->is_key_frm = inst->vpu_inst.is_key_frm;
@@ -503,11 +534,13 @@ static int venc_encode(void *handle,
 
 		mtk_vcodec_debug(inst, "venc_encode_frame prepend SPS/PPS");
 
+		init_schedule_work(ctx);
+
 		ret = venc_encode_header(inst, bs_buf, &bs_size_hdr);
 		if (ret) {
 			mtk_vcodec_err(inst, "encode prepend hdr failed: %d",
 				       ret);
-			return ret;
+			goto encode_err;
 		}
 
 		hdr_sz = bs_size_hdr;
@@ -530,7 +563,7 @@ static int venc_encode(void *handle,
 		if (ret) {
 			mtk_vcodec_err(inst, "encode hdr frame failed: %d",
 				       ret);
-			return ret;
+			goto encode_err;
 		}
 
 		result->bs_size = hdr_sz + filler_sz + bs_size_frm;
@@ -552,6 +585,10 @@ static int venc_encode(void *handle,
 	}
 
 	return ret;
+
+encode_err:
+	cancel_schedule_work(ctx);
+	return ret;
 }
 
 static int venc_set_param(void *handle,
-- 
2.25.1

[PATCH 9/9] media: mtk-vcodec: Add delayed worker for encode timeout

[Irui Wang]

When encoding timeout, a delayed timeout worker is needed because
of asynchronous encoding process, restore encoding process in
timeout worker.

Signed-off-by: Irui Wang <irui.wang@mediatek.com>
---
 .../platform/mtk-vcodec/mtk_vcodec_enc.c      | 12 ++++--
 .../platform/mtk-vcodec/mtk_vcodec_enc_hw.c   | 26 +++++++++++
 .../platform/mtk-vcodec/mtk_vcodec_enc_hw.h   |  2 +
 .../platform/mtk-vcodec/venc/venc_common_if.c | 43 +++++++++++++++++--
 4 files changed, 77 insertions(+), 6 deletions(-)

diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
index de8e012f1b0e..bc635436aa90 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
@@ -1340,9 +1340,15 @@ void mtk_venc_buf_done(struct mtk_vcodec_ctx *ctx, int core_id,
 		if (key_frame)
 			dst_vb2_v4l2->flags |= V4L2_BUF_FLAG_KEYFRAME;
 
-		v4l2_m2m_buf_done(src_vb2_v4l2, VB2_BUF_STATE_DONE);
-		dst_vb2_v4l2->vb2_buf.planes[0].bytesused = bs_size;
-		v4l2_m2m_buf_done(dst_vb2_v4l2, VB2_BUF_STATE_DONE);
+		if (time_out) {
+			v4l2_m2m_buf_done(src_vb2_v4l2, VB2_BUF_STATE_ERROR);
+			dst_vb2_v4l2->vb2_buf.planes[0].bytesused = 0;
+			v4l2_m2m_buf_done(dst_vb2_v4l2, VB2_BUF_STATE_ERROR);
+		} else {
+			v4l2_m2m_buf_done(src_vb2_v4l2, VB2_BUF_STATE_DONE);
+			dst_vb2_v4l2->vb2_buf.planes[0].bytesused = bs_size;
+			v4l2_m2m_buf_done(dst_vb2_v4l2, VB2_BUF_STATE_DONE);
+		}
 	}
 }
 
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.c
index 687b255c58e8..f6a5dfa6929f 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.c
@@ -16,6 +16,28 @@
 #define VENC_PIC_FRM_TYPE	0x0010
 #define VENC_PIC_KEY_FRM	0x2
 
+static void mtk_venc_timeout_worker(struct work_struct *work)
+{
+	struct mtk_venc_comp_dev *dev = container_of(work,
+						     struct mtk_venc_comp_dev,
+						     enc_timeout_worker.work);
+
+	struct mtk_vcodec_ctx *ctx;
+	unsigned long flags;
+
+	spin_lock_irqsave(&dev->master_dev->irqlock, flags);
+	ctx = dev->curr_ctx;
+
+	dev_err(&dev->plat_dev->dev, "core: %d timeout\n", dev->core_id);
+
+	mtk_venc_buf_done(ctx, dev->core_id, 0, 1, 0);
+
+	mtk_vcodec_enc_clock_off(dev->master_dev, dev->core_id);
+	dev->curr_ctx = NULL;
+	spin_unlock_irqrestore(&dev->master_dev->irqlock, flags);
+
+	mtk_venc_unlock(ctx, dev->core_id);
+}
 /* Wake up core context wait_queue */
 static void mtk_venc_comp_wake_up_ctx(struct mtk_vcodec_ctx *ctx,
 				      unsigned int hw_id)
@@ -67,6 +89,7 @@ static irqreturn_t mtk_enc_comp_irq_handler(int irq, void *priv)
 	frm_type = readl(dev->reg_base + VENC_PIC_FRM_TYPE);
 	clean_irq_status(ctx->irq_status, addr);
 	if (ctx->irq_status & MTK_VENC_IRQ_STATUS_FRM) {
+		cancel_delayed_work(&dev->enc_timeout_worker);
 		if (ctx->hdr_size != 0) {
 			bs_size += ctx->hdr_size;
 			ctx->hdr_size = 0;
@@ -182,6 +205,9 @@ static int mtk_venc_comp_probe(struct platform_device *pdev)
 
 	platform_set_drvdata(pdev, comp_dev);
 
+	INIT_DELAYED_WORK(&comp_dev->enc_timeout_worker,
+			  mtk_venc_timeout_worker);
+
 	ret = component_add(&pdev->dev, &mtk_venc_component_ops);
 	if (ret) {
 		dev_err(&pdev->dev, "Failed to add component: %d\n", ret);
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.h
index f9ae97e252dc..e69344e38ca6 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.h
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.h
@@ -19,6 +19,7 @@
  * @irq_status: venc hw irq status
  * @enc_irq: venc device irq
  * @core id: for venc core id: core#0, core#1...
+ * @enc_timeout_worker: venc timeout worker
  */
 struct mtk_venc_comp_dev {
 	struct platform_device *plat_dev;
@@ -31,6 +32,7 @@ struct mtk_venc_comp_dev {
 	unsigned int irq_status;
 	int enc_irq;
 	int core_id;
+	struct delayed_work enc_timeout_worker;
 };
 
 #endif /* _MTK_VCODEC_ENC_HW_H_ */
diff --git a/drivers/media/platform/mtk-vcodec/venc/venc_common_if.c b/drivers/media/platform/mtk-vcodec/venc/venc_common_if.c
index 75b4e90232ae..1aab84faadf0 100644
--- a/drivers/media/platform/mtk-vcodec/venc/venc_common_if.c
+++ b/drivers/media/platform/mtk-vcodec/venc/venc_common_if.c
@@ -10,11 +10,13 @@
 #include "../mtk_vcodec_drv.h"
 #include "../mtk_vcodec_util.h"
 #include "../mtk_vcodec_enc_hw.h"
+#include "../mtk_vcodec_enc_pm.h"
 
 static const char h264_filler_marker[] = {0x0, 0x0, 0x0, 0x1, 0xc};
 
 #define H264_FILLER_MARKER_SIZE ARRAY_SIZE(h264_filler_marker)
 #define VENC_PIC_BITSTREAM_BYTE_CNT 0x0098
+#define MTK_VENC_HW_TIMEOUT_MSEC 2000
 
 enum venc_vpu_work_buf {
 	VENC_VPU_WORK_BUF_RC_INFO_CORE0,
@@ -413,6 +415,34 @@ static void h264_encode_filler(struct venc_common_inst *inst, void *buf,
 	memset(p, 0xff, size);
 }
 
+static void init_schedule_work(struct mtk_vcodec_ctx *ctx)
+{
+	unsigned long timeout;
+	struct mtk_venc_comp_dev *venc;
+
+	venc = ctx->dev->enc_comp_dev[ctx->hw_id];
+	timeout = msecs_to_jiffies(MTK_VENC_HW_TIMEOUT_MSEC);
+	schedule_delayed_work(&venc->enc_timeout_worker, timeout);
+}
+
+static void cancel_schedule_work(struct mtk_vcodec_ctx *ctx)
+{
+	struct mtk_venc_comp_dev *venc;
+	unsigned long flags;
+
+	venc = ctx->dev->enc_comp_dev[ctx->hw_id];
+	cancel_delayed_work(&venc->enc_timeout_worker);
+
+	mtk_venc_buf_done(ctx, ctx->hw_id, 0, 1, 0);
+	mtk_vcodec_enc_clock_off(ctx->dev, ctx->hw_id);
+
+	spin_lock_irqsave(&ctx->dev->irqlock, flags);
+	venc->curr_ctx = NULL;
+	spin_unlock_irqrestore(&ctx->dev->irqlock, flags);
+
+	mtk_venc_unlock(ctx, ctx->hw_id);
+}
+
 static int venc_init(struct mtk_vcodec_ctx *ctx)
 {
 	int i;
@@ -489,12 +519,13 @@ static int venc_encode(void *handle,
 		ctx->pbs_buf[ctx->hw_id] = bs_buf->buf;
 
 		if (!inst->prepend_hdr) {
+			init_schedule_work(ctx);
 			ret = venc_encode_frame(inst, frm_buf, bs_buf,
 						&result->bs_size, ctx->hw_id);
 			if (ret) {
 				mtk_vcodec_err(inst, "encode frame failed: %d",
 					       ret);
-				return ret;
+				goto encode_err;
 			}
 
 			result->is_key_frm = inst->vpu_inst.is_key_frm;
@@ -503,11 +534,13 @@ static int venc_encode(void *handle,
 
 		mtk_vcodec_debug(inst, "venc_encode_frame prepend SPS/PPS");
 
+		init_schedule_work(ctx);
+
 		ret = venc_encode_header(inst, bs_buf, &bs_size_hdr);
 		if (ret) {
 			mtk_vcodec_err(inst, "encode prepend hdr failed: %d",
 				       ret);
-			return ret;
+			goto encode_err;
 		}
 
 		hdr_sz = bs_size_hdr;
@@ -530,7 +563,7 @@ static int venc_encode(void *handle,
 		if (ret) {
 			mtk_vcodec_err(inst, "encode hdr frame failed: %d",
 				       ret);
-			return ret;
+			goto encode_err;
 		}
 
 		result->bs_size = hdr_sz + filler_sz + bs_size_frm;
@@ -552,6 +585,10 @@ static int venc_encode(void *handle,
 	}
 
 	return ret;
+
+encode_err:
+	cancel_schedule_work(ctx);
+	return ret;
 }
 
 static int venc_set_param(void *handle,
-- 
2.25.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH 9/9] media: mtk-vcodec: Add delayed worker for encode timeout

[Irui Wang]

When encoding timeout, a delayed timeout worker is needed because
of asynchronous encoding process, restore encoding process in
timeout worker.

Signed-off-by: Irui Wang <irui.wang@mediatek.com>
---
 .../platform/mtk-vcodec/mtk_vcodec_enc.c      | 12 ++++--
 .../platform/mtk-vcodec/mtk_vcodec_enc_hw.c   | 26 +++++++++++
 .../platform/mtk-vcodec/mtk_vcodec_enc_hw.h   |  2 +
 .../platform/mtk-vcodec/venc/venc_common_if.c | 43 +++++++++++++++++--
 4 files changed, 77 insertions(+), 6 deletions(-)

diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
index de8e012f1b0e..bc635436aa90 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
@@ -1340,9 +1340,15 @@ void mtk_venc_buf_done(struct mtk_vcodec_ctx *ctx, int core_id,
 		if (key_frame)
 			dst_vb2_v4l2->flags |= V4L2_BUF_FLAG_KEYFRAME;
 
-		v4l2_m2m_buf_done(src_vb2_v4l2, VB2_BUF_STATE_DONE);
-		dst_vb2_v4l2->vb2_buf.planes[0].bytesused = bs_size;
-		v4l2_m2m_buf_done(dst_vb2_v4l2, VB2_BUF_STATE_DONE);
+		if (time_out) {
+			v4l2_m2m_buf_done(src_vb2_v4l2, VB2_BUF_STATE_ERROR);
+			dst_vb2_v4l2->vb2_buf.planes[0].bytesused = 0;
+			v4l2_m2m_buf_done(dst_vb2_v4l2, VB2_BUF_STATE_ERROR);
+		} else {
+			v4l2_m2m_buf_done(src_vb2_v4l2, VB2_BUF_STATE_DONE);
+			dst_vb2_v4l2->vb2_buf.planes[0].bytesused = bs_size;
+			v4l2_m2m_buf_done(dst_vb2_v4l2, VB2_BUF_STATE_DONE);
+		}
 	}
 }
 
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.c
index 687b255c58e8..f6a5dfa6929f 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.c
@@ -16,6 +16,28 @@
 #define VENC_PIC_FRM_TYPE	0x0010
 #define VENC_PIC_KEY_FRM	0x2
 
+static void mtk_venc_timeout_worker(struct work_struct *work)
+{
+	struct mtk_venc_comp_dev *dev = container_of(work,
+						     struct mtk_venc_comp_dev,
+						     enc_timeout_worker.work);
+
+	struct mtk_vcodec_ctx *ctx;
+	unsigned long flags;
+
+	spin_lock_irqsave(&dev->master_dev->irqlock, flags);
+	ctx = dev->curr_ctx;
+
+	dev_err(&dev->plat_dev->dev, "core: %d timeout\n", dev->core_id);
+
+	mtk_venc_buf_done(ctx, dev->core_id, 0, 1, 0);
+
+	mtk_vcodec_enc_clock_off(dev->master_dev, dev->core_id);
+	dev->curr_ctx = NULL;
+	spin_unlock_irqrestore(&dev->master_dev->irqlock, flags);
+
+	mtk_venc_unlock(ctx, dev->core_id);
+}
 /* Wake up core context wait_queue */
 static void mtk_venc_comp_wake_up_ctx(struct mtk_vcodec_ctx *ctx,
 				      unsigned int hw_id)
@@ -67,6 +89,7 @@ static irqreturn_t mtk_enc_comp_irq_handler(int irq, void *priv)
 	frm_type = readl(dev->reg_base + VENC_PIC_FRM_TYPE);
 	clean_irq_status(ctx->irq_status, addr);
 	if (ctx->irq_status & MTK_VENC_IRQ_STATUS_FRM) {
+		cancel_delayed_work(&dev->enc_timeout_worker);
 		if (ctx->hdr_size != 0) {
 			bs_size += ctx->hdr_size;
 			ctx->hdr_size = 0;
@@ -182,6 +205,9 @@ static int mtk_venc_comp_probe(struct platform_device *pdev)
 
 	platform_set_drvdata(pdev, comp_dev);
 
+	INIT_DELAYED_WORK(&comp_dev->enc_timeout_worker,
+			  mtk_venc_timeout_worker);
+
 	ret = component_add(&pdev->dev, &mtk_venc_component_ops);
 	if (ret) {
 		dev_err(&pdev->dev, "Failed to add component: %d\n", ret);
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.h
index f9ae97e252dc..e69344e38ca6 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.h
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.h
@@ -19,6 +19,7 @@
  * @irq_status: venc hw irq status
  * @enc_irq: venc device irq
  * @core id: for venc core id: core#0, core#1...
+ * @enc_timeout_worker: venc timeout worker
  */
 struct mtk_venc_comp_dev {
 	struct platform_device *plat_dev;
@@ -31,6 +32,7 @@ struct mtk_venc_comp_dev {
 	unsigned int irq_status;
 	int enc_irq;
 	int core_id;
+	struct delayed_work enc_timeout_worker;
 };
 
 #endif /* _MTK_VCODEC_ENC_HW_H_ */
diff --git a/drivers/media/platform/mtk-vcodec/venc/venc_common_if.c b/drivers/media/platform/mtk-vcodec/venc/venc_common_if.c
index 75b4e90232ae..1aab84faadf0 100644
--- a/drivers/media/platform/mtk-vcodec/venc/venc_common_if.c
+++ b/drivers/media/platform/mtk-vcodec/venc/venc_common_if.c
@@ -10,11 +10,13 @@
 #include "../mtk_vcodec_drv.h"
 #include "../mtk_vcodec_util.h"
 #include "../mtk_vcodec_enc_hw.h"
+#include "../mtk_vcodec_enc_pm.h"
 
 static const char h264_filler_marker[] = {0x0, 0x0, 0x0, 0x1, 0xc};
 
 #define H264_FILLER_MARKER_SIZE ARRAY_SIZE(h264_filler_marker)
 #define VENC_PIC_BITSTREAM_BYTE_CNT 0x0098
+#define MTK_VENC_HW_TIMEOUT_MSEC 2000
 
 enum venc_vpu_work_buf {
 	VENC_VPU_WORK_BUF_RC_INFO_CORE0,
@@ -413,6 +415,34 @@ static void h264_encode_filler(struct venc_common_inst *inst, void *buf,
 	memset(p, 0xff, size);
 }
 
+static void init_schedule_work(struct mtk_vcodec_ctx *ctx)
+{
+	unsigned long timeout;
+	struct mtk_venc_comp_dev *venc;
+
+	venc = ctx->dev->enc_comp_dev[ctx->hw_id];
+	timeout = msecs_to_jiffies(MTK_VENC_HW_TIMEOUT_MSEC);
+	schedule_delayed_work(&venc->enc_timeout_worker, timeout);
+}
+
+static void cancel_schedule_work(struct mtk_vcodec_ctx *ctx)
+{
+	struct mtk_venc_comp_dev *venc;
+	unsigned long flags;
+
+	venc = ctx->dev->enc_comp_dev[ctx->hw_id];
+	cancel_delayed_work(&venc->enc_timeout_worker);
+
+	mtk_venc_buf_done(ctx, ctx->hw_id, 0, 1, 0);
+	mtk_vcodec_enc_clock_off(ctx->dev, ctx->hw_id);
+
+	spin_lock_irqsave(&ctx->dev->irqlock, flags);
+	venc->curr_ctx = NULL;
+	spin_unlock_irqrestore(&ctx->dev->irqlock, flags);
+
+	mtk_venc_unlock(ctx, ctx->hw_id);
+}
+
 static int venc_init(struct mtk_vcodec_ctx *ctx)
 {
 	int i;
@@ -489,12 +519,13 @@ static int venc_encode(void *handle,
 		ctx->pbs_buf[ctx->hw_id] = bs_buf->buf;
 
 		if (!inst->prepend_hdr) {
+			init_schedule_work(ctx);
 			ret = venc_encode_frame(inst, frm_buf, bs_buf,
 						&result->bs_size, ctx->hw_id);
 			if (ret) {
 				mtk_vcodec_err(inst, "encode frame failed: %d",
 					       ret);
-				return ret;
+				goto encode_err;
 			}
 
 			result->is_key_frm = inst->vpu_inst.is_key_frm;
@@ -503,11 +534,13 @@ static int venc_encode(void *handle,
 
 		mtk_vcodec_debug(inst, "venc_encode_frame prepend SPS/PPS");
 
+		init_schedule_work(ctx);
+
 		ret = venc_encode_header(inst, bs_buf, &bs_size_hdr);
 		if (ret) {
 			mtk_vcodec_err(inst, "encode prepend hdr failed: %d",
 				       ret);
-			return ret;
+			goto encode_err;
 		}
 
 		hdr_sz = bs_size_hdr;
@@ -530,7 +563,7 @@ static int venc_encode(void *handle,
 		if (ret) {
 			mtk_vcodec_err(inst, "encode hdr frame failed: %d",
 				       ret);
-			return ret;
+			goto encode_err;
 		}
 
 		result->bs_size = hdr_sz + filler_sz + bs_size_frm;
@@ -552,6 +585,10 @@ static int venc_encode(void *handle,
 	}
 
 	return ret;
+
+encode_err:
+	cancel_schedule_work(ctx);
+	return ret;
 }
 
 static int venc_set_param(void *handle,
-- 
2.25.1
_______________________________________________
Linux-mediatek mailing list
Linux-mediatek@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-mediatek

Re: [PATCH 0/9] Enable two H264 encoder cores on MT8195

[AngeloGioacchino Del Regno]

Il 16/08/21 12:59, Irui Wang ha scritto:
> MT8195 has two H264 encoder cores, they have their own power-domains,
> clocks, interrupts, register base. The two H264 encoder cores can work
> together to achieve higher performance.
> 
> This series of patches is to use enable two h264 encoder cores.
> path[1..2]: use linux component framework to manage encoder hardware,
> user call "mt8195-vcodec-enc" driver can get the encoder master device,
> the encoding work is done by the two encoder core device. The hw_mode
> variable is added to distinguish from old platform, two encoder cores
> called "FRAME_RACING_MODE".
> 
> The hardware mode of two encoder cores work together(overlap, another
> word called) on MT8195 called "frame_racing_mode", the two encoder
> power-domains should be power on together while encoding, the encoding
> process look like this:
> 
>      VENC Core0 frm#0....frm#2....frm#4
>      VENC Core1  .frm#1....frm#3....frm#5
> 
> patch[3..5]: due to the component device, the master device has no
> power-domains/clocks properties in dtsi, so the power/clock init function
> can't use for "frame_racing_mode" device in master device probe process,
> it should be called in component device probe process. Power on the
> hardware power and clock on demand.
> 
> patch[6]: "frame_racing_mode" encoding need a new set of memory buffer
> for two encoder cores. For compatibility, we should new a encoder driver
> interface.
> 
> patch[7..9]: add "frame_racing_mode" encoding process:
> As-Is: Synchronous
> VIDIOC_QBUF#0 --> device_run(triger encoder) --> wait encoder IRQ -->
> encode done with result --> job_finish
> 
> VIDIOC_QBUF#1 --> device_run(triger encoder) --> wait encoder IRQ -->
> encode done with result --> job_finish
> ...
> 
> To-Be: Asynchronous
> VIDIOC_QBUF#0 --> device_run(triger encoder core0) --> job_finish
> ..VIDIOC_QBUF#1 --> device_run(triger encoder core1) --> job_finish
> (core0 may encode done here, return encode result to client)
> VIDIOC_QBUF#2 --> device_run(triger encoder core0) --> job_finish
> 
> Thers is no "wait encoder IRQ" synchronous call during "frame_racing_mode"
> encoding process, which can full use the two encoder cores to achieve
> higher performance.
> 
> Irui Wang (9):
>    dt-bindings: media: mtk-vcodec: Add binding for MT8195 two venc cores
>    media: mtk-vcodec: Use component framework to manage encoder hardware
>    media: mtk-vcodec: Rewrite venc power manage interface
>    media: mtk-vcodec: Add venc power on/off interface
>    media: mtk-vcodec: Rewrite venc clock interface
>    media: mtk-vcodec: Add new venc drv interface for frame_racing mode
>    media: mtk-vcodec: Add frame racing mode encode process
>    media: mtk-vcodec: Return encode result to client
>    media: mtk-vcodec: Add delayed worker for encode timeout
> 
>   .../bindings/media/mediatek-vcodec.txt        |   2 +
>   drivers/media/platform/mtk-vcodec/Makefile    |   2 +
>   .../platform/mtk-vcodec/mtk_vcodec_drv.h      |  34 +-
>   .../platform/mtk-vcodec/mtk_vcodec_enc.c      | 120 +++-
>   .../platform/mtk-vcodec/mtk_vcodec_enc.h      |  10 +-
>   .../platform/mtk-vcodec/mtk_vcodec_enc_drv.c  | 204 +++++-
>   .../platform/mtk-vcodec/mtk_vcodec_enc_hw.c   | 253 +++++++
>   .../platform/mtk-vcodec/mtk_vcodec_enc_hw.h   |  38 +
>   .../platform/mtk-vcodec/mtk_vcodec_enc_pm.c   | 213 ++++--
>   .../platform/mtk-vcodec/mtk_vcodec_enc_pm.h   |  13 +-
>   .../platform/mtk-vcodec/mtk_vcodec_util.c     |  19 +
>   .../platform/mtk-vcodec/mtk_vcodec_util.h     |   5 +
>   .../platform/mtk-vcodec/venc/venc_common_if.c | 675 ++++++++++++++++++
>   .../platform/mtk-vcodec/venc/venc_h264_if.c   |   6 +-
>   .../platform/mtk-vcodec/venc/venc_vp8_if.c    |   2 +-
>   .../media/platform/mtk-vcodec/venc_drv_if.c   |  96 ++-
>   .../media/platform/mtk-vcodec/venc_drv_if.h   |   7 +
>   .../media/platform/mtk-vcodec/venc_vpu_if.c   |  11 +-
>   .../media/platform/mtk-vcodec/venc_vpu_if.h   |   3 +-
>   19 files changed, 1564 insertions(+), 149 deletions(-)
>   create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.c
>   create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.h
>   create mode 100644 drivers/media/platform/mtk-vcodec/venc/venc_common_if.c
> 

Hello Irui,
we have some interest in this series; can you please rebase it over the latest
changes that were sent in series [1] [2]?

Also, please follow [2], where of_platform_populate was used in place of the
component framework (where applicable and where possible) on this series, as well.

Thanks!
- Angelo

[1]: https://patchwork.kernel.org/project/linux-mediatek/list/?series=579201
[2]: https://patchwork.kernel.org/project/linux-mediatek/list/?series=578467

_______________________________________________
Linux-mediatek mailing list
Linux-mediatek@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-mediatek

Re: [PATCH 0/9] Enable two H264 encoder cores on MT8195

[AngeloGioacchino Del Regno]

Il 16/08/21 12:59, Irui Wang ha scritto:
> MT8195 has two H264 encoder cores, they have their own power-domains,
> clocks, interrupts, register base. The two H264 encoder cores can work
> together to achieve higher performance.
> 
> This series of patches is to use enable two h264 encoder cores.
> path[1..2]: use linux component framework to manage encoder hardware,
> user call "mt8195-vcodec-enc" driver can get the encoder master device,
> the encoding work is done by the two encoder core device. The hw_mode
> variable is added to distinguish from old platform, two encoder cores
> called "FRAME_RACING_MODE".
> 
> The hardware mode of two encoder cores work together(overlap, another
> word called) on MT8195 called "frame_racing_mode", the two encoder
> power-domains should be power on together while encoding, the encoding
> process look like this:
> 
>      VENC Core0 frm#0....frm#2....frm#4
>      VENC Core1  .frm#1....frm#3....frm#5
> 
> patch[3..5]: due to the component device, the master device has no
> power-domains/clocks properties in dtsi, so the power/clock init function
> can't use for "frame_racing_mode" device in master device probe process,
> it should be called in component device probe process. Power on the
> hardware power and clock on demand.
> 
> patch[6]: "frame_racing_mode" encoding need a new set of memory buffer
> for two encoder cores. For compatibility, we should new a encoder driver
> interface.
> 
> patch[7..9]: add "frame_racing_mode" encoding process:
> As-Is: Synchronous
> VIDIOC_QBUF#0 --> device_run(triger encoder) --> wait encoder IRQ -->
> encode done with result --> job_finish
> 
> VIDIOC_QBUF#1 --> device_run(triger encoder) --> wait encoder IRQ -->
> encode done with result --> job_finish
> ...
> 
> To-Be: Asynchronous
> VIDIOC_QBUF#0 --> device_run(triger encoder core0) --> job_finish
> ..VIDIOC_QBUF#1 --> device_run(triger encoder core1) --> job_finish
> (core0 may encode done here, return encode result to client)
> VIDIOC_QBUF#2 --> device_run(triger encoder core0) --> job_finish
> 
> Thers is no "wait encoder IRQ" synchronous call during "frame_racing_mode"
> encoding process, which can full use the two encoder cores to achieve
> higher performance.
> 
> Irui Wang (9):
>    dt-bindings: media: mtk-vcodec: Add binding for MT8195 two venc cores
>    media: mtk-vcodec: Use component framework to manage encoder hardware
>    media: mtk-vcodec: Rewrite venc power manage interface
>    media: mtk-vcodec: Add venc power on/off interface
>    media: mtk-vcodec: Rewrite venc clock interface
>    media: mtk-vcodec: Add new venc drv interface for frame_racing mode
>    media: mtk-vcodec: Add frame racing mode encode process
>    media: mtk-vcodec: Return encode result to client
>    media: mtk-vcodec: Add delayed worker for encode timeout
> 
>   .../bindings/media/mediatek-vcodec.txt        |   2 +
>   drivers/media/platform/mtk-vcodec/Makefile    |   2 +
>   .../platform/mtk-vcodec/mtk_vcodec_drv.h      |  34 +-
>   .../platform/mtk-vcodec/mtk_vcodec_enc.c      | 120 +++-
>   .../platform/mtk-vcodec/mtk_vcodec_enc.h      |  10 +-
>   .../platform/mtk-vcodec/mtk_vcodec_enc_drv.c  | 204 +++++-
>   .../platform/mtk-vcodec/mtk_vcodec_enc_hw.c   | 253 +++++++
>   .../platform/mtk-vcodec/mtk_vcodec_enc_hw.h   |  38 +
>   .../platform/mtk-vcodec/mtk_vcodec_enc_pm.c   | 213 ++++--
>   .../platform/mtk-vcodec/mtk_vcodec_enc_pm.h   |  13 +-
>   .../platform/mtk-vcodec/mtk_vcodec_util.c     |  19 +
>   .../platform/mtk-vcodec/mtk_vcodec_util.h     |   5 +
>   .../platform/mtk-vcodec/venc/venc_common_if.c | 675 ++++++++++++++++++
>   .../platform/mtk-vcodec/venc/venc_h264_if.c   |   6 +-
>   .../platform/mtk-vcodec/venc/venc_vp8_if.c    |   2 +-
>   .../media/platform/mtk-vcodec/venc_drv_if.c   |  96 ++-
>   .../media/platform/mtk-vcodec/venc_drv_if.h   |   7 +
>   .../media/platform/mtk-vcodec/venc_vpu_if.c   |  11 +-
>   .../media/platform/mtk-vcodec/venc_vpu_if.h   |   3 +-
>   19 files changed, 1564 insertions(+), 149 deletions(-)
>   create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.c
>   create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.h
>   create mode 100644 drivers/media/platform/mtk-vcodec/venc/venc_common_if.c
> 

Hello Irui,
we have some interest in this series; can you please rebase it over the latest
changes that were sent in series [1] [2]?

Also, please follow [2], where of_platform_populate was used in place of the
component framework (where applicable and where possible) on this series, as well.

Thanks!
- Angelo

[1]: https://patchwork.kernel.org/project/linux-mediatek/list/?series=579201
[2]: https://patchwork.kernel.org/project/linux-mediatek/list/?series=578467

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH 0/9] Enable two H264 encoder cores on MT8195

[AngeloGioacchino Del Regno]

Il 16/08/21 12:59, Irui Wang ha scritto:
> MT8195 has two H264 encoder cores, they have their own power-domains,
> clocks, interrupts, register base. The two H264 encoder cores can work
> together to achieve higher performance.
> 
> This series of patches is to use enable two h264 encoder cores.
> path[1..2]: use linux component framework to manage encoder hardware,
> user call "mt8195-vcodec-enc" driver can get the encoder master device,
> the encoding work is done by the two encoder core device. The hw_mode
> variable is added to distinguish from old platform, two encoder cores
> called "FRAME_RACING_MODE".
> 
> The hardware mode of two encoder cores work together(overlap, another
> word called) on MT8195 called "frame_racing_mode", the two encoder
> power-domains should be power on together while encoding, the encoding
> process look like this:
> 
>      VENC Core0 frm#0....frm#2....frm#4
>      VENC Core1  .frm#1....frm#3....frm#5
> 
> patch[3..5]: due to the component device, the master device has no
> power-domains/clocks properties in dtsi, so the power/clock init function
> can't use for "frame_racing_mode" device in master device probe process,
> it should be called in component device probe process. Power on the
> hardware power and clock on demand.
> 
> patch[6]: "frame_racing_mode" encoding need a new set of memory buffer
> for two encoder cores. For compatibility, we should new a encoder driver
> interface.
> 
> patch[7..9]: add "frame_racing_mode" encoding process:
> As-Is: Synchronous
> VIDIOC_QBUF#0 --> device_run(triger encoder) --> wait encoder IRQ -->
> encode done with result --> job_finish
> 
> VIDIOC_QBUF#1 --> device_run(triger encoder) --> wait encoder IRQ -->
> encode done with result --> job_finish
> ...
> 
> To-Be: Asynchronous
> VIDIOC_QBUF#0 --> device_run(triger encoder core0) --> job_finish
> ..VIDIOC_QBUF#1 --> device_run(triger encoder core1) --> job_finish
> (core0 may encode done here, return encode result to client)
> VIDIOC_QBUF#2 --> device_run(triger encoder core0) --> job_finish
> 
> Thers is no "wait encoder IRQ" synchronous call during "frame_racing_mode"
> encoding process, which can full use the two encoder cores to achieve
> higher performance.
> 
> Irui Wang (9):
>    dt-bindings: media: mtk-vcodec: Add binding for MT8195 two venc cores
>    media: mtk-vcodec: Use component framework to manage encoder hardware
>    media: mtk-vcodec: Rewrite venc power manage interface
>    media: mtk-vcodec: Add venc power on/off interface
>    media: mtk-vcodec: Rewrite venc clock interface
>    media: mtk-vcodec: Add new venc drv interface for frame_racing mode
>    media: mtk-vcodec: Add frame racing mode encode process
>    media: mtk-vcodec: Return encode result to client
>    media: mtk-vcodec: Add delayed worker for encode timeout
> 
>   .../bindings/media/mediatek-vcodec.txt        |   2 +
>   drivers/media/platform/mtk-vcodec/Makefile    |   2 +
>   .../platform/mtk-vcodec/mtk_vcodec_drv.h      |  34 +-
>   .../platform/mtk-vcodec/mtk_vcodec_enc.c      | 120 +++-
>   .../platform/mtk-vcodec/mtk_vcodec_enc.h      |  10 +-
>   .../platform/mtk-vcodec/mtk_vcodec_enc_drv.c  | 204 +++++-
>   .../platform/mtk-vcodec/mtk_vcodec_enc_hw.c   | 253 +++++++
>   .../platform/mtk-vcodec/mtk_vcodec_enc_hw.h   |  38 +
>   .../platform/mtk-vcodec/mtk_vcodec_enc_pm.c   | 213 ++++--
>   .../platform/mtk-vcodec/mtk_vcodec_enc_pm.h   |  13 +-
>   .../platform/mtk-vcodec/mtk_vcodec_util.c     |  19 +
>   .../platform/mtk-vcodec/mtk_vcodec_util.h     |   5 +
>   .../platform/mtk-vcodec/venc/venc_common_if.c | 675 ++++++++++++++++++
>   .../platform/mtk-vcodec/venc/venc_h264_if.c   |   6 +-
>   .../platform/mtk-vcodec/venc/venc_vp8_if.c    |   2 +-
>   .../media/platform/mtk-vcodec/venc_drv_if.c   |  96 ++-
>   .../media/platform/mtk-vcodec/venc_drv_if.h   |   7 +
>   .../media/platform/mtk-vcodec/venc_vpu_if.c   |  11 +-
>   .../media/platform/mtk-vcodec/venc_vpu_if.h   |   3 +-
>   19 files changed, 1564 insertions(+), 149 deletions(-)
>   create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.c
>   create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.h
>   create mode 100644 drivers/media/platform/mtk-vcodec/venc/venc_common_if.c
> 

Hello Irui,
we have some interest in this series; can you please rebase it over the latest
changes that were sent in series [1] [2]?

Also, please follow [2], where of_platform_populate was used in place of the
component framework (where applicable and where possible) on this series, as well.

Thanks!
- Angelo

[1]: https://patchwork.kernel.org/project/linux-mediatek/list/?series=579201
[2]: https://patchwork.kernel.org/project/linux-mediatek/list/?series=578467