All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH v5, 0/8] Support H264 multi-core encoder on MT8195
@ 2022-07-29  3:51 ` Irui Wang
  0 siblings, 0 replies; 22+ messages in thread
From: Irui Wang @ 2022-07-29  3:51 UTC (permalink / raw)
  To: Hans Verkuil, angelogioacchino.delregno, Rob Herring,
	Mauro Carvalho Chehab, Matthias Brugger, Alexandre Courbot,
	Tiffany Lin, Andrew-CT Chen, Tzung-Bi Shih, Tomasz Figa,
	nicolas.dufresne
  Cc: Maoguang Meng, Longfei Wang, Yunfei Dong, Irui Wang, linux-media,
	devicetree, linux-kernel, linux-arm-kernel, linux-mediatek,
	Project_Global_Chrome_Upstream_Group

MT8195 has two H264 encoder hardware, named core0 and core1, this two
cores can encode two input frames separately at the same time to achieve
higher performance.

This series of patches are used to enable the two H264 encoder cores,
the difference between encoding process before and after enable two
cores is just like as below:
As-Is: Synchronous
V4L2_VIDIOC_QBUF#0 --> device_run(triger encoder) --> wait encoder IRQ
-->
encoding done with result --> job_finish
V4l2_VIDIOC_QBUF#1 --> device_run(triger encoder) --> wait encoder IRQ
-->
encoding done with result --> job_finish
...
To-Be: Asynchronous
V4L2_VIDIOC_QBUF#0 --> device_run(triger encoder) --> job_finish
..V4l2_VIDIOC_QBUF#1 --> device_run(triger encoder) --> job_finish
(venc core0 may encode done here, done the encoding result to client)
V4L2_VIDIOC_QBUF#2 --> device_run(triger encoder) --> job_finish.

---
changes compared with v4:
- reabse to the newer linux media stage tree.
- remove "mediatek,venc-multi-core" property since sub-device can
  be probed by of_platform_populate api directly.
- some modifications for patch v4's review comments.

changes compared with v3:
- rebase to the newer linux media stage.
- add a capability to indicate scp firmware support multi-core.
- probe core0 as main device, core1 as sub-device.

changes compared with v2:
- update venc core dt-bindings, add two new properties for current
  usage.
- parse venc multi_core mode from device tree.
- rebase to the newer linux media stage.

changes compared with v1:
- of_platform_populate was used in place of the component framework.
- new yaml file for venc cores.
- some modifications for patch v1's review comments.
---

Irui Wang (8):
  dt-bindings: media: mediatek: vcodec: Adds encoder cores dt-bindings
    for mt8195
  media: mediatek: vcodec: Enable venc dual core usage
  media: mediatek: vcodec: Refactor venc power manage function
  media: mediatek: vcodec: Add more extra processing for multi-core
    encoding
  media: mediatek: vcodec: Add venc power on/off function
  media: mediatek: vcodec: Refactor encoder clock on/off function
  media: mediatek: vcodec: Add multi-core encoding process
  media: mediatek: vcodec: Return encoding result in asynchronous mode

 .../media/mediatek,vcodec-encoder-core.yaml   | 218 ++++++++++++++++
 .../media/mediatek,vcodec-encoder.yaml        |   1 -
 .../media/platform/mediatek/vcodec/Makefile   |   4 +-
 .../platform/mediatek/vcodec/mtk_vcodec_drv.h |  28 +-
 .../platform/mediatek/vcodec/mtk_vcodec_enc.c | 113 ++++++--
 .../platform/mediatek/vcodec/mtk_vcodec_enc.h |  11 +-
 .../mediatek/vcodec/mtk_vcodec_enc_drv.c      |  44 +++-
 .../mediatek/vcodec/mtk_vcodec_enc_hw.c       | 156 +++++++++++
 .../mediatek/vcodec/mtk_vcodec_enc_hw.h       |  34 +++
 .../mediatek/vcodec/mtk_vcodec_enc_pm.c       | 178 +++++++++++--
 .../mediatek/vcodec/mtk_vcodec_enc_pm.h       |  11 +-
 .../mediatek/vcodec/mtk_vcodec_util.c         |  19 ++
 .../mediatek/vcodec/mtk_vcodec_util.h         |   3 +
 .../mediatek/vcodec/venc/venc_h264_if.c       | 243 ++++++++++++++----
 .../mediatek/vcodec/venc/venc_vp8_if.c        |   3 +-
 .../platform/mediatek/vcodec/venc_drv_if.c    |  75 ++++--
 .../platform/mediatek/vcodec/venc_drv_if.h    |   6 +
 .../platform/mediatek/vcodec/venc_vpu_if.c    |   9 +-
 .../platform/mediatek/vcodec/venc_vpu_if.h    |   3 +-
 19 files changed, 1028 insertions(+), 131 deletions(-)
 create mode 100644 Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml
 create mode 100644 drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.c
 create mode 100644 drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.h

-- 
2.18.0


^ permalink raw reply	[flat|nested] 22+ messages in thread
* [PATCH v5, 0/8] Support H264 multi-core encoder on MT8195
@ 2022-07-29  3:51 ` Irui Wang
  0 siblings, 0 replies; 22+ messages in thread
From: Irui Wang @ 2022-07-29  3:51 UTC (permalink / raw)
  To: Hans Verkuil, angelogioacchino.delregno, Rob Herring,
	Mauro Carvalho Chehab, Matthias Brugger, Alexandre Courbot,
	Tiffany Lin, Andrew-CT Chen, Tzung-Bi Shih, Tomasz Figa,
	nicolas.dufresne
  Cc: Maoguang Meng, Longfei Wang, Yunfei Dong, Irui Wang, linux-media,
	devicetree, linux-kernel, linux-arm-kernel, linux-mediatek,
	Project_Global_Chrome_Upstream_Group

MT8195 has two H264 encoder hardware, named core0 and core1, this two
cores can encode two input frames separately at the same time to achieve
higher performance.

This series of patches are used to enable the two H264 encoder cores,
the difference between encoding process before and after enable two
cores is just like as below:
As-Is: Synchronous
V4L2_VIDIOC_QBUF#0 --> device_run(triger encoder) --> wait encoder IRQ
-->
encoding done with result --> job_finish
V4l2_VIDIOC_QBUF#1 --> device_run(triger encoder) --> wait encoder IRQ
-->
encoding done with result --> job_finish
...
To-Be: Asynchronous
V4L2_VIDIOC_QBUF#0 --> device_run(triger encoder) --> job_finish
..V4l2_VIDIOC_QBUF#1 --> device_run(triger encoder) --> job_finish
(venc core0 may encode done here, done the encoding result to client)
V4L2_VIDIOC_QBUF#2 --> device_run(triger encoder) --> job_finish.

---
changes compared with v4:
- reabse to the newer linux media stage tree.
- remove "mediatek,venc-multi-core" property since sub-device can
  be probed by of_platform_populate api directly.
- some modifications for patch v4's review comments.

changes compared with v3:
- rebase to the newer linux media stage.
- add a capability to indicate scp firmware support multi-core.
- probe core0 as main device, core1 as sub-device.

changes compared with v2:
- update venc core dt-bindings, add two new properties for current
  usage.
- parse venc multi_core mode from device tree.
- rebase to the newer linux media stage.

changes compared with v1:
- of_platform_populate was used in place of the component framework.
- new yaml file for venc cores.
- some modifications for patch v1's review comments.
---

Irui Wang (8):
  dt-bindings: media: mediatek: vcodec: Adds encoder cores dt-bindings
    for mt8195
  media: mediatek: vcodec: Enable venc dual core usage
  media: mediatek: vcodec: Refactor venc power manage function
  media: mediatek: vcodec: Add more extra processing for multi-core
    encoding
  media: mediatek: vcodec: Add venc power on/off function
  media: mediatek: vcodec: Refactor encoder clock on/off function
  media: mediatek: vcodec: Add multi-core encoding process
  media: mediatek: vcodec: Return encoding result in asynchronous mode

 .../media/mediatek,vcodec-encoder-core.yaml   | 218 ++++++++++++++++
 .../media/mediatek,vcodec-encoder.yaml        |   1 -
 .../media/platform/mediatek/vcodec/Makefile   |   4 +-
 .../platform/mediatek/vcodec/mtk_vcodec_drv.h |  28 +-
 .../platform/mediatek/vcodec/mtk_vcodec_enc.c | 113 ++++++--
 .../platform/mediatek/vcodec/mtk_vcodec_enc.h |  11 +-
 .../mediatek/vcodec/mtk_vcodec_enc_drv.c      |  44 +++-
 .../mediatek/vcodec/mtk_vcodec_enc_hw.c       | 156 +++++++++++
 .../mediatek/vcodec/mtk_vcodec_enc_hw.h       |  34 +++
 .../mediatek/vcodec/mtk_vcodec_enc_pm.c       | 178 +++++++++++--
 .../mediatek/vcodec/mtk_vcodec_enc_pm.h       |  11 +-
 .../mediatek/vcodec/mtk_vcodec_util.c         |  19 ++
 .../mediatek/vcodec/mtk_vcodec_util.h         |   3 +
 .../mediatek/vcodec/venc/venc_h264_if.c       | 243 ++++++++++++++----
 .../mediatek/vcodec/venc/venc_vp8_if.c        |   3 +-
 .../platform/mediatek/vcodec/venc_drv_if.c    |  75 ++++--
 .../platform/mediatek/vcodec/venc_drv_if.h    |   6 +
 .../platform/mediatek/vcodec/venc_vpu_if.c    |   9 +-
 .../platform/mediatek/vcodec/venc_vpu_if.h    |   3 +-
 19 files changed, 1028 insertions(+), 131 deletions(-)
 create mode 100644 Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml
 create mode 100644 drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.c
 create mode 100644 drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.h

-- 
2.18.0


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 22+ messages in thread
* [PATCH v5, 1/8] dt-bindings: media: mediatek: vcodec: Adds encoder cores dt-bindings for mt8195
  2022-07-29  3:51 ` Irui Wang
@ 2022-07-29  3:51   ` Irui Wang
  -1 siblings, 0 replies; 22+ messages in thread
From: Irui Wang @ 2022-07-29  3:51 UTC (permalink / raw)
  To: Hans Verkuil, angelogioacchino.delregno, Rob Herring,
	Mauro Carvalho Chehab, Matthias Brugger, Alexandre Courbot,
	Tiffany Lin, Andrew-CT Chen, Tzung-Bi Shih, Tomasz Figa,
	nicolas.dufresne
  Cc: Maoguang Meng, Longfei Wang, Yunfei Dong, Irui Wang, linux-media,
	devicetree, linux-kernel, linux-arm-kernel, linux-mediatek,
	Project_Global_Chrome_Upstream_Group

mt8195 has two H264 encoder hardware, which are named core0 and core1.
The two encoder cores are independent, we can just enable one core to
do encoding or enable both of them to achieve higher performance. We
pick core0 as main device and core1 as its subdevice, it just a way to
to manage the two encoder hardware, because they are two equal encoder
hardware with the same function.

Signed-off-by: Irui Wang <irui.wang@mediatek.com>
---
 .../media/mediatek,vcodec-encoder-core.yaml   | 218 ++++++++++++++++++
 .../media/mediatek,vcodec-encoder.yaml        |   1 -
 2 files changed, 218 insertions(+), 1 deletion(-)
 create mode 100644 Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml

diff --git a/Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml b/Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml
new file mode 100644
index 000000000000..96e016f5000b
--- /dev/null
+++ b/Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml
@@ -0,0 +1,218 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+
+%YAML 1.2
+---
+$id: "http://devicetree.org/schemas/media/mediatek,vcodec-encoder-core.yaml#"
+$schema: "http://devicetree.org/meta-schemas/core.yaml#"
+
+title: MediaTek Video Encoder Accelerator With Multi Core
+
+maintainers:
+  - Irui Wang <irui.wang@mediatek.com>
+
+description: |
+  MediaTek Video Encoder is the video encoder hardware present in MediaTek
+  SoCs which supports high resolution encoding functionalities. To meet higher
+  encoder performance, there will be one or more encoder hardware inside SoC,
+  which named core0, core1, etc.. For example, mt8195 has two encoder hardware,
+  the two encoder cores block diagram, can check below.
+  --------------------------------------------------------------
+  Input frame  0     1     2     3     4     5     6
+               |     |     |     |     |     |     |
+               v     |     v     |     v     |     v
+           +-------+ | +-------+ | +-------+ | +-------+
+           | core0 | | | core0 | | | core0 | | | core0 |
+           +-------+ | +-------+ | +-------+ | +-------+
+               |     |     |     |     |     |     |
+               |     v     |     v     |     v     |
+               | +-------+ | +-------+ | +-------+ |
+               | | core1 | | | core1 | | | core1 | |
+               | +-------+ | +-------+ | +-------+ |
+               |     |     |     |     |     |     |
+               v     v     v     v     v     v     v    <parent>
+  --------------------------------------------------------------
+                            core || index               <child>
+                                 \/
+       +--------------------------------------------------+
+       |                     core0/core1                  |
+       |             enable/disable power/clk/irq         |
+       +--------------------------------------------------+
+  --------------------------------------------------------------
+  As above, there are two cores child devices, they are two encoder hardware
+  which can encode input frames in order. When start encoding, input frame 0
+  will be encoded by core0, and input frame 1 can be encoded by core1 even if
+  frame 0 has not been encoded done yet, after frame 0 encoded done, frame 2
+  will be encoded by core0, even input frames are encoded by core0 and odd
+  input frames are encoded by core1, these two encoder cores encode ench input
+  frames in this overlapping manner.
+
+properties:
+  compatible:
+    items:
+      - enum:
+          - mediatek,mt8195-vcodec-enc
+
+  reg:
+    maxItems: 1
+
+  mediatek,scp:
+    $ref: /schemas/types.yaml#/definitions/phandle
+    description: |
+      The node of system control processor (SCP), using
+      the remoteproc & rpmsg framework.
+
+  iommus:
+    minItems: 1
+    maxItems: 32
+    description: |
+      List of the hardware port in respective IOMMU block for current Socs.
+      Refer to bindings/iommu/mediatek,iommu.yaml.
+
+  interrupts:
+    maxItems: 1
+
+  clocks:
+    maxItems: 1
+
+  clock-names:
+    maxItems: 1
+
+  power-domains:
+    maxItems: 1
+
+  dma-ranges:
+    maxItems: 1
+    description: |
+      Describes the physical address space of IOMMU maps to memory.
+
+  "#address-cells":
+    const: 2
+
+  "#size-cells":
+    const: 2
+
+  ranges: true
+
+# Required child node:
+patternProperties:
+  "^venc-core@[0-9a-f]+$":
+    type: object
+    description: |
+      The video encoder core device node which should be added as subnodes to
+      the main venc node, it represents a encoder hardware.
+
+    properties:
+      compatible:
+        items:
+          - const: mediatek,mtk-venc-hw
+
+      reg:
+        maxItems: 1
+
+      mediatek,hw-id:
+        $ref: /schemas/types.yaml#/definitions/uint32
+        description: |
+          Current encoder core id. We use it to pick which one encoder core
+          will be used to encoding current input frame.
+
+      iommus:
+        minItems: 1
+        maxItems: 32
+        description: |
+          List of the hardware port in respective IOMMU block for current Socs.
+          Refer to bindings/iommu/mediatek,iommu.yaml.
+
+      interrupts:
+        maxItems: 1
+
+      clocks:
+        maxItems: 1
+
+      clock-names:
+        maxItems: 1
+
+      power-domains:
+        maxItems: 1
+
+    required:
+      - compatible
+      - reg
+      - mediatek,hw-id
+      - iommus
+      - interrupts
+      - clocks
+      - clock-names
+      - assigned-clocks
+      - assigned-clock-parents
+      - power-domains
+
+    additionalProperties: false
+
+required:
+  - compatible
+  - reg
+  - mediatek,scp
+  - iommus
+  - interrupts
+  - clocks
+  - clock-names
+  - dma-ranges
+
+additionalProperties: false
+
+examples:
+  - |
+    #include <dt-bindings/interrupt-controller/arm-gic.h>
+    #include <dt-bindings/memory/mt8195-memory-port.h>
+    #include <dt-bindings/interrupt-controller/irq.h>
+    #include <dt-bindings/clock/mt8195-clk.h>
+    #include <dt-bindings/power/mt8195-power.h>
+
+    soc {
+        #address-cells = <2>;
+        #size-cells = <2>;
+
+        venc: venc@1a020000 {
+            compatible = "mediatek,mt8195-vcodec-enc";
+            reg = <0 0x1a020000 0 0x10000>;
+            mediatek,scp = <&scp>;
+            iommus = <&iommu_vdo M4U_PORT_L19_VENC_RCPU>,
+                     <&iommu_vdo M4U_PORT_L19_VENC_REC>,
+                     <&iommu_vdo M4U_PORT_L19_VENC_BSDMA>,
+                     <&iommu_vdo M4U_PORT_L19_VENC_SV_COMV>,
+                     <&iommu_vdo M4U_PORT_L19_VENC_RD_COMV>,
+                     <&iommu_vdo M4U_PORT_L19_VENC_CUR_LUMA>,
+                     <&iommu_vdo M4U_PORT_L19_VENC_CUR_CHROMA>,
+                     <&iommu_vdo M4U_PORT_L19_VENC_REF_LUMA>,
+                     <&iommu_vdo M4U_PORT_L19_VENC_REF_CHROMA>;
+            interrupts = <GIC_SPI 341 IRQ_TYPE_LEVEL_HIGH 0>;
+            clocks = <&vencsys CLK_VENC_VENC>;
+            clock-names = "clk_venc";
+            power-domains = <&spm MT8195_POWER_DOMAIN_VENC>;
+            dma-ranges = <0x1 0x0 0x0 0x40000000 0x0 0xfff00000>;
+            #address-cells = <2>;
+            #size-cells = <2>;
+            ranges;
+
+            venc-core@1b020000 {
+                compatible = "mediatek,mtk-venc-hw";
+                reg = <0 0x1b020000 0 0x10000>;
+                mediatek,hw-id = <1>;
+                iommus = <&iommu_vpp M4U_PORT_L20_VENC_RCPU>,
+                         <&iommu_vpp M4U_PORT_L20_VENC_REC>,
+                         <&iommu_vpp M4U_PORT_L20_VENC_BSDMA>,
+                         <&iommu_vpp M4U_PORT_L20_VENC_SV_COMV>,
+                         <&iommu_vpp M4U_PORT_L20_VENC_RD_COMV>,
+                         <&iommu_vpp M4U_PORT_L20_VENC_CUR_LUMA>,
+                         <&iommu_vpp M4U_PORT_L20_VENC_CUR_CHROMA>,
+                         <&iommu_vpp M4U_PORT_L20_VENC_REF_LUMA>,
+                         <&iommu_vpp M4U_PORT_L20_VENC_REF_CHROMA>;
+                interrupts = <GIC_SPI 346 IRQ_TYPE_LEVEL_HIGH 0>;
+                clocks = <&vencsys_core1 CLK_VENC_CORE1_VENC>;
+                clock-names = "clk_venc_core1";
+                assigned-clocks = <&topckgen CLK_TOP_VENC>;
+                assigned-clock-parents = <&topckgen CLK_TOP_UNIVPLL_D4>;
+                power-domains = <&spm MT8195_POWER_DOMAIN_VENC_CORE1>;
+            };
+        };
+    };
diff --git a/Documentation/devicetree/bindings/media/mediatek,vcodec-encoder.yaml b/Documentation/devicetree/bindings/media/mediatek,vcodec-encoder.yaml
index d36fcca04cbc..11682659c4c4 100644
--- a/Documentation/devicetree/bindings/media/mediatek,vcodec-encoder.yaml
+++ b/Documentation/devicetree/bindings/media/mediatek,vcodec-encoder.yaml
@@ -21,7 +21,6 @@ properties:
       - mediatek,mt8173-vcodec-enc
       - mediatek,mt8183-vcodec-enc
       - mediatek,mt8192-vcodec-enc
-      - mediatek,mt8195-vcodec-enc
 
   reg:
     maxItems: 1
-- 
2.18.0


^ permalink raw reply related	[flat|nested] 22+ messages in thread
* [PATCH v5, 1/8] dt-bindings: media: mediatek: vcodec: Adds encoder cores dt-bindings for mt8195
@ 2022-07-29  3:51   ` Irui Wang
  0 siblings, 0 replies; 22+ messages in thread
From: Irui Wang @ 2022-07-29  3:51 UTC (permalink / raw)
  To: Hans Verkuil, angelogioacchino.delregno, Rob Herring,
	Mauro Carvalho Chehab, Matthias Brugger, Alexandre Courbot,
	Tiffany Lin, Andrew-CT Chen, Tzung-Bi Shih, Tomasz Figa,
	nicolas.dufresne
  Cc: Maoguang Meng, Longfei Wang, Yunfei Dong, Irui Wang, linux-media,
	devicetree, linux-kernel, linux-arm-kernel, linux-mediatek,
	Project_Global_Chrome_Upstream_Group

mt8195 has two H264 encoder hardware, which are named core0 and core1.
The two encoder cores are independent, we can just enable one core to
do encoding or enable both of them to achieve higher performance. We
pick core0 as main device and core1 as its subdevice, it just a way to
to manage the two encoder hardware, because they are two equal encoder
hardware with the same function.

Signed-off-by: Irui Wang <irui.wang@mediatek.com>
---
 .../media/mediatek,vcodec-encoder-core.yaml   | 218 ++++++++++++++++++
 .../media/mediatek,vcodec-encoder.yaml        |   1 -
 2 files changed, 218 insertions(+), 1 deletion(-)
 create mode 100644 Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml

diff --git a/Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml b/Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml
new file mode 100644
index 000000000000..96e016f5000b
--- /dev/null
+++ b/Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml
@@ -0,0 +1,218 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+
+%YAML 1.2
+---
+$id: "http://devicetree.org/schemas/media/mediatek,vcodec-encoder-core.yaml#"
+$schema: "http://devicetree.org/meta-schemas/core.yaml#"
+
+title: MediaTek Video Encoder Accelerator With Multi Core
+
+maintainers:
+  - Irui Wang <irui.wang@mediatek.com>
+
+description: |
+  MediaTek Video Encoder is the video encoder hardware present in MediaTek
+  SoCs which supports high resolution encoding functionalities. To meet higher
+  encoder performance, there will be one or more encoder hardware inside SoC,
+  which named core0, core1, etc.. For example, mt8195 has two encoder hardware,
+  the two encoder cores block diagram, can check below.
+  --------------------------------------------------------------
+  Input frame  0     1     2     3     4     5     6
+               |     |     |     |     |     |     |
+               v     |     v     |     v     |     v
+           +-------+ | +-------+ | +-------+ | +-------+
+           | core0 | | | core0 | | | core0 | | | core0 |
+           +-------+ | +-------+ | +-------+ | +-------+
+               |     |     |     |     |     |     |
+               |     v     |     v     |     v     |
+               | +-------+ | +-------+ | +-------+ |
+               | | core1 | | | core1 | | | core1 | |
+               | +-------+ | +-------+ | +-------+ |
+               |     |     |     |     |     |     |
+               v     v     v     v     v     v     v    <parent>
+  --------------------------------------------------------------
+                            core || index               <child>
+                                 \/
+       +--------------------------------------------------+
+       |                     core0/core1                  |
+       |             enable/disable power/clk/irq         |
+       +--------------------------------------------------+
+  --------------------------------------------------------------
+  As above, there are two cores child devices, they are two encoder hardware
+  which can encode input frames in order. When start encoding, input frame 0
+  will be encoded by core0, and input frame 1 can be encoded by core1 even if
+  frame 0 has not been encoded done yet, after frame 0 encoded done, frame 2
+  will be encoded by core0, even input frames are encoded by core0 and odd
+  input frames are encoded by core1, these two encoder cores encode ench input
+  frames in this overlapping manner.
+
+properties:
+  compatible:
+    items:
+      - enum:
+          - mediatek,mt8195-vcodec-enc
+
+  reg:
+    maxItems: 1
+
+  mediatek,scp:
+    $ref: /schemas/types.yaml#/definitions/phandle
+    description: |
+      The node of system control processor (SCP), using
+      the remoteproc & rpmsg framework.
+
+  iommus:
+    minItems: 1
+    maxItems: 32
+    description: |
+      List of the hardware port in respective IOMMU block for current Socs.
+      Refer to bindings/iommu/mediatek,iommu.yaml.
+
+  interrupts:
+    maxItems: 1
+
+  clocks:
+    maxItems: 1
+
+  clock-names:
+    maxItems: 1
+
+  power-domains:
+    maxItems: 1
+
+  dma-ranges:
+    maxItems: 1
+    description: |
+      Describes the physical address space of IOMMU maps to memory.
+
+  "#address-cells":
+    const: 2
+
+  "#size-cells":
+    const: 2
+
+  ranges: true
+
+# Required child node:
+patternProperties:
+  "^venc-core@[0-9a-f]+$":
+    type: object
+    description: |
+      The video encoder core device node which should be added as subnodes to
+      the main venc node, it represents a encoder hardware.
+
+    properties:
+      compatible:
+        items:
+          - const: mediatek,mtk-venc-hw
+
+      reg:
+        maxItems: 1
+
+      mediatek,hw-id:
+        $ref: /schemas/types.yaml#/definitions/uint32
+        description: |
+          Current encoder core id. We use it to pick which one encoder core
+          will be used to encoding current input frame.
+
+      iommus:
+        minItems: 1
+        maxItems: 32
+        description: |
+          List of the hardware port in respective IOMMU block for current Socs.
+          Refer to bindings/iommu/mediatek,iommu.yaml.
+
+      interrupts:
+        maxItems: 1
+
+      clocks:
+        maxItems: 1
+
+      clock-names:
+        maxItems: 1
+
+      power-domains:
+        maxItems: 1
+
+    required:
+      - compatible
+      - reg
+      - mediatek,hw-id
+      - iommus
+      - interrupts
+      - clocks
+      - clock-names
+      - assigned-clocks
+      - assigned-clock-parents
+      - power-domains
+
+    additionalProperties: false
+
+required:
+  - compatible
+  - reg
+  - mediatek,scp
+  - iommus
+  - interrupts
+  - clocks
+  - clock-names
+  - dma-ranges
+
+additionalProperties: false
+
+examples:
+  - |
+    #include <dt-bindings/interrupt-controller/arm-gic.h>
+    #include <dt-bindings/memory/mt8195-memory-port.h>
+    #include <dt-bindings/interrupt-controller/irq.h>
+    #include <dt-bindings/clock/mt8195-clk.h>
+    #include <dt-bindings/power/mt8195-power.h>
+
+    soc {
+        #address-cells = <2>;
+        #size-cells = <2>;
+
+        venc: venc@1a020000 {
+            compatible = "mediatek,mt8195-vcodec-enc";
+            reg = <0 0x1a020000 0 0x10000>;
+            mediatek,scp = <&scp>;
+            iommus = <&iommu_vdo M4U_PORT_L19_VENC_RCPU>,
+                     <&iommu_vdo M4U_PORT_L19_VENC_REC>,
+                     <&iommu_vdo M4U_PORT_L19_VENC_BSDMA>,
+                     <&iommu_vdo M4U_PORT_L19_VENC_SV_COMV>,
+                     <&iommu_vdo M4U_PORT_L19_VENC_RD_COMV>,
+                     <&iommu_vdo M4U_PORT_L19_VENC_CUR_LUMA>,
+                     <&iommu_vdo M4U_PORT_L19_VENC_CUR_CHROMA>,
+                     <&iommu_vdo M4U_PORT_L19_VENC_REF_LUMA>,
+                     <&iommu_vdo M4U_PORT_L19_VENC_REF_CHROMA>;
+            interrupts = <GIC_SPI 341 IRQ_TYPE_LEVEL_HIGH 0>;
+            clocks = <&vencsys CLK_VENC_VENC>;
+            clock-names = "clk_venc";
+            power-domains = <&spm MT8195_POWER_DOMAIN_VENC>;
+            dma-ranges = <0x1 0x0 0x0 0x40000000 0x0 0xfff00000>;
+            #address-cells = <2>;
+            #size-cells = <2>;
+            ranges;
+
+            venc-core@1b020000 {
+                compatible = "mediatek,mtk-venc-hw";
+                reg = <0 0x1b020000 0 0x10000>;
+                mediatek,hw-id = <1>;
+                iommus = <&iommu_vpp M4U_PORT_L20_VENC_RCPU>,
+                         <&iommu_vpp M4U_PORT_L20_VENC_REC>,
+                         <&iommu_vpp M4U_PORT_L20_VENC_BSDMA>,
+                         <&iommu_vpp M4U_PORT_L20_VENC_SV_COMV>,
+                         <&iommu_vpp M4U_PORT_L20_VENC_RD_COMV>,
+                         <&iommu_vpp M4U_PORT_L20_VENC_CUR_LUMA>,
+                         <&iommu_vpp M4U_PORT_L20_VENC_CUR_CHROMA>,
+                         <&iommu_vpp M4U_PORT_L20_VENC_REF_LUMA>,
+                         <&iommu_vpp M4U_PORT_L20_VENC_REF_CHROMA>;
+                interrupts = <GIC_SPI 346 IRQ_TYPE_LEVEL_HIGH 0>;
+                clocks = <&vencsys_core1 CLK_VENC_CORE1_VENC>;
+                clock-names = "clk_venc_core1";
+                assigned-clocks = <&topckgen CLK_TOP_VENC>;
+                assigned-clock-parents = <&topckgen CLK_TOP_UNIVPLL_D4>;
+                power-domains = <&spm MT8195_POWER_DOMAIN_VENC_CORE1>;
+            };
+        };
+    };
diff --git a/Documentation/devicetree/bindings/media/mediatek,vcodec-encoder.yaml b/Documentation/devicetree/bindings/media/mediatek,vcodec-encoder.yaml
index d36fcca04cbc..11682659c4c4 100644
--- a/Documentation/devicetree/bindings/media/mediatek,vcodec-encoder.yaml
+++ b/Documentation/devicetree/bindings/media/mediatek,vcodec-encoder.yaml
@@ -21,7 +21,6 @@ properties:
       - mediatek,mt8173-vcodec-enc
       - mediatek,mt8183-vcodec-enc
       - mediatek,mt8192-vcodec-enc
-      - mediatek,mt8195-vcodec-enc
 
   reg:
     maxItems: 1
-- 
2.18.0


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply related	[flat|nested] 22+ messages in thread
* [PATCH v5, 2/8] media: mediatek: vcodec: Enable venc dual core usage
  2022-07-29  3:51 ` Irui Wang
@ 2022-07-29  3:51   ` Irui Wang
  -1 siblings, 0 replies; 22+ messages in thread
From: Irui Wang @ 2022-07-29  3:51 UTC (permalink / raw)
  To: Hans Verkuil, angelogioacchino.delregno, Rob Herring,
	Mauro Carvalho Chehab, Matthias Brugger, Alexandre Courbot,
	Tiffany Lin, Andrew-CT Chen, Tzung-Bi Shih, Tomasz Figa,
	nicolas.dufresne
  Cc: Maoguang Meng, Longfei Wang, Yunfei Dong, Irui Wang, linux-media,
	devicetree, linux-kernel, linux-arm-kernel, linux-mediatek,
	Project_Global_Chrome_Upstream_Group

Adds new property to indicate whether the encoder has multiple cores.
Use of_platform_populate to probe each venc cores, the core device can
use the init_clk/request_irq helper to initialize their own power/clk/irq.

Signed-off-by: Irui Wang <irui.wang@mediatek.com>
---
 .../media/platform/mediatek/vcodec/Makefile   |   4 +-
 .../platform/mediatek/vcodec/mtk_vcodec_drv.h |  12 ++
 .../mediatek/vcodec/mtk_vcodec_enc_drv.c      |   6 +
 .../mediatek/vcodec/mtk_vcodec_enc_hw.c       | 136 ++++++++++++++++++
 .../mediatek/vcodec/mtk_vcodec_enc_hw.h       |  36 +++++
 5 files changed, 193 insertions(+), 1 deletion(-)
 create mode 100644 drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.c
 create mode 100644 drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.h

diff --git a/drivers/media/platform/mediatek/vcodec/Makefile b/drivers/media/platform/mediatek/vcodec/Makefile
index 93e7a343b5b0..ac068d88af29 100644
--- a/drivers/media/platform/mediatek/vcodec/Makefile
+++ b/drivers/media/platform/mediatek/vcodec/Makefile
@@ -3,7 +3,8 @@
 obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += mtk-vcodec-dec.o \
 				       mtk-vcodec-enc.o \
 				       mtk-vcodec-common.o \
-				       mtk-vcodec-dec-hw.o
+				       mtk-vcodec-dec-hw.o \
+				       mtk_vcodec_enc_hw.o
 
 mtk-vcodec-dec-y := vdec/vdec_h264_if.o \
 		vdec/vdec_vp8_if.o \
@@ -32,6 +33,7 @@ mtk-vcodec-enc-y := venc/venc_vp8_if.o \
 		venc_drv_if.o \
 		venc_vpu_if.o \
 
+mtk-vcodec-enc-hw-y := mtk_vcodec_enc_hw.o
 
 mtk-vcodec-common-y := mtk_vcodec_intr.o \
 		mtk_vcodec_util.o \
diff --git a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_drv.h b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_drv.h
index ef4584a46417..644021c5bbd4 100644
--- a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_drv.h
+++ b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_drv.h
@@ -95,6 +95,15 @@ enum mtk_fmt_type {
 	MTK_FMT_FRAME = 2,
 };
 
+/*
+ * enum mtk_venc_hw_id -- encoder hardware id
+ */
+enum mtk_venc_hw_id {
+	MTK_VENC_CORE_0 = 0,
+	MTK_VENC_CORE_1,
+	MTK_VENC_HW_MAX,
+};
+
 /*
  * enum mtk_vdec_hw_id - Hardware index used to separate
  *                         different hardware
@@ -464,6 +473,7 @@ struct mtk_vcodec_enc_pdata {
  * @dec_active_cnt: used to mark whether need to record register value
  * @vdec_racing_info: record register value
  * @dec_racing_info_mutex: mutex lock used for inner racing mode
+ * @enc_hw_dev: used to store venc core device
  */
 struct mtk_vcodec_dev {
 	struct v4l2_device v4l2_dev;
@@ -514,6 +524,8 @@ struct mtk_vcodec_dev {
 	u32 vdec_racing_info[132];
 	/* Protects access to vdec_racing_info data */
 	struct mutex dec_racing_info_mutex;
+
+	void *enc_hw_dev[MTK_VENC_HW_MAX];
 };
 
 static inline struct mtk_vcodec_ctx *fh_to_ctx(struct v4l2_fh *fh)
diff --git a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_drv.c b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_drv.c
index 95e8c29ccc65..9cb6ce1fa101 100644
--- a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_drv.c
+++ b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_drv.c
@@ -263,6 +263,12 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
 		goto err_enc_pm;
 	}
 
+	ret = of_platform_populate(pdev->dev.of_node, NULL, NULL, &pdev->dev);
+	if (ret) {
+		mtk_v4l2_err("Failed to populate children devices");
+		goto err_enc_pm;
+	}
+
 	pm_runtime_enable(&pdev->dev);
 
 	dev->reg_base[dev->venc_pdata->core_id] =
diff --git a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.c b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.c
new file mode 100644
index 000000000000..cc4938f027e0
--- /dev/null
+++ b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.c
@@ -0,0 +1,136 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2021 MediaTek Inc.
+ */
+
+#include <linux/interrupt.h>
+#include <linux/irq.h>
+#include <linux/module.h>
+#include <linux/of_platform.h>
+#include <linux/pm_runtime.h>
+#include <linux/slab.h>
+
+#include "mtk_vcodec_drv.h"
+#include "mtk_vcodec_enc.h"
+#include "mtk_vcodec_enc_hw.h"
+#include "mtk_vcodec_intr.h"
+
+static const struct of_device_id mtk_venc_hw_ids[] = {
+	{.compatible = "mediatek,mtk-venc-hw",},
+	{},
+};
+MODULE_DEVICE_TABLE(of, mtk_venc_hw_ids);
+
+static void clean_hw_irq_status(unsigned int irq_status, void __iomem *addr)
+{
+	if (irq_status & MTK_VENC_IRQ_STATUS_PAUSE)
+		writel(MTK_VENC_IRQ_STATUS_PAUSE, addr);
+
+	if (irq_status & MTK_VENC_IRQ_STATUS_SWITCH)
+		writel(MTK_VENC_IRQ_STATUS_SWITCH, addr);
+
+	if (irq_status & MTK_VENC_IRQ_STATUS_DRAM)
+		writel(MTK_VENC_IRQ_STATUS_DRAM, addr);
+
+	if (irq_status & MTK_VENC_IRQ_STATUS_SPS)
+		writel(MTK_VENC_IRQ_STATUS_SPS, addr);
+
+	if (irq_status & MTK_VENC_IRQ_STATUS_PPS)
+		writel(MTK_VENC_IRQ_STATUS_PPS, addr);
+
+	if (irq_status & MTK_VENC_IRQ_STATUS_FRM)
+		writel(MTK_VENC_IRQ_STATUS_FRM, addr);
+}
+
+static irqreturn_t mtk_enc_hw_irq_handler(int irq, void *priv)
+{
+	struct mtk_venc_hw_dev *dev = priv;
+	struct mtk_vcodec_ctx *ctx;
+	unsigned long flags;
+	void __iomem *addr;
+
+	spin_lock_irqsave(&dev->main_dev->irqlock, flags);
+	ctx = dev->curr_ctx;
+	spin_unlock_irqrestore(&dev->main_dev->irqlock, flags);
+	if (!ctx)
+		return IRQ_HANDLED;
+
+	mtk_v4l2_debug(1, "id=%d core :%d", ctx->id, dev->hw_id);
+
+	addr = dev->reg_base + MTK_VENC_IRQ_ACK_OFFSET;
+	ctx->irq_status = readl(dev->reg_base + MTK_VENC_IRQ_STATUS_OFFSET);
+	clean_hw_irq_status(ctx->irq_status, addr);
+
+	wake_up_ctx(ctx, MTK_INST_IRQ_RECEIVED, 0);
+	return IRQ_HANDLED;
+}
+
+static int mtk_venc_hw_probe(struct platform_device *pdev)
+{
+	struct device *dev = &pdev->dev;
+	struct mtk_venc_hw_dev *sub_core;
+	struct mtk_vcodec_dev *main_dev;
+	int ret;
+
+	if (!dev->parent)
+		return dev_err_probe(dev, -ENODEV,
+				     "No parent for venc core device\n");
+
+	main_dev = dev_get_drvdata(dev->parent);
+	if (!main_dev)
+		return dev_err_probe(dev, -EINVAL,
+				     "Failed to get parent driver data\n");
+
+	sub_core = devm_kzalloc(&pdev->dev, sizeof(*sub_core), GFP_KERNEL);
+	if (!sub_core)
+		return dev_err_probe(dev, -ENOMEM,
+				     "Failed to get alloc core data\n");
+
+	sub_core->plat_dev = pdev;
+
+	platform_set_drvdata(pdev, sub_core);
+
+	sub_core->reg_base = devm_platform_ioremap_resource(pdev, 0);
+	if (IS_ERR(sub_core->reg_base))
+		return dev_err_probe(dev, PTR_ERR(sub_core->reg_base),
+				     "Failed to get reg base\n");
+
+	sub_core->enc_irq = platform_get_irq(pdev, 0);
+	if (sub_core->enc_irq < 0)
+		return dev_err_probe(dev, -EINVAL,
+				     "Failed to get irq resource\n");
+
+	ret = devm_request_irq(dev, sub_core->enc_irq,
+			       mtk_enc_hw_irq_handler, 0,
+			       pdev->name, sub_core);
+	if (ret)
+		return dev_err_probe(dev, -EINVAL,
+				     "Failed to install sub_core->enc_irq %d\n",
+				     sub_core->enc_irq);
+
+	ret = of_property_read_u32(dev->of_node, "mediatek,hw-id",
+				   &sub_core->hw_id);
+
+	if (ret || sub_core->hw_id >= MTK_VENC_HW_MAX)
+		return dev_err_probe(dev, (ret ? ret : -EINVAL),
+				     "Cannot parse hardware id");
+
+	main_dev->enc_hw_dev[sub_core->hw_id] = sub_core;
+	sub_core->main_dev = main_dev;
+
+	dev_dbg(dev, "Venc core :%d probe done\n", sub_core->hw_id);
+
+	return 0;
+}
+
+static struct platform_driver mtk_venc_core_driver = {
+	.probe  = mtk_venc_hw_probe,
+	.driver = {
+		.name	 = "mtk-venc-core",
+		.of_match_table = mtk_venc_hw_ids,
+	},
+};
+module_platform_driver(mtk_venc_core_driver);
+
+MODULE_LICENSE("GPL");
+MODULE_DESCRIPTION("MediaTek video encoder core driver");
diff --git a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.h b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.h
new file mode 100644
index 000000000000..9daea9665659
--- /dev/null
+++ b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.h
@@ -0,0 +1,36 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2021 MediaTek Inc.
+ */
+
+#ifndef _MTK_VCODEC_ENC_HW_H_
+#define _MTK_VCODEC_ENC_HW_H_
+
+#include <linux/platform_device.h>
+#include "mtk_vcodec_drv.h"
+
+/**
+ * struct mtk_venc_hw_dev - driver data
+ * @plat_dev: platform_device
+ * @main_dev: main device
+ * @pm: power management data
+ * @curr_ctx: the context that is waiting for venc hardware
+ * @reg_base: mapped address of venc registers
+ * @irq_status: venc hardware irq status
+ * @enc_irq: venc device irq
+ * @hw_id: for venc hardware id: core#0, core#1...
+ */
+struct mtk_venc_hw_dev {
+	struct platform_device *plat_dev;
+	struct mtk_vcodec_dev *main_dev;
+
+	struct mtk_vcodec_pm pm;
+	struct mtk_vcodec_ctx *curr_ctx;
+
+	void __iomem *reg_base;
+	unsigned int irq_status;
+	int enc_irq;
+	enum mtk_venc_hw_id hw_id;
+};
+
+#endif /* _MTK_VCODEC_ENC_HW_H_ */
-- 
2.18.0


^ permalink raw reply related	[flat|nested] 22+ messages in thread
* [PATCH v5, 2/8] media: mediatek: vcodec: Enable venc dual core usage
@ 2022-07-29  3:51   ` Irui Wang
  0 siblings, 0 replies; 22+ messages in thread
From: Irui Wang @ 2022-07-29  3:51 UTC (permalink / raw)
  To: Hans Verkuil, angelogioacchino.delregno, Rob Herring,
	Mauro Carvalho Chehab, Matthias Brugger, Alexandre Courbot,
	Tiffany Lin, Andrew-CT Chen, Tzung-Bi Shih, Tomasz Figa,
	nicolas.dufresne
  Cc: Maoguang Meng, Longfei Wang, Yunfei Dong, Irui Wang, linux-media,
	devicetree, linux-kernel, linux-arm-kernel, linux-mediatek,
	Project_Global_Chrome_Upstream_Group

Adds new property to indicate whether the encoder has multiple cores.
Use of_platform_populate to probe each venc cores, the core device can
use the init_clk/request_irq helper to initialize their own power/clk/irq.

Signed-off-by: Irui Wang <irui.wang@mediatek.com>
---
 .../media/platform/mediatek/vcodec/Makefile   |   4 +-
 .../platform/mediatek/vcodec/mtk_vcodec_drv.h |  12 ++
 .../mediatek/vcodec/mtk_vcodec_enc_drv.c      |   6 +
 .../mediatek/vcodec/mtk_vcodec_enc_hw.c       | 136 ++++++++++++++++++
 .../mediatek/vcodec/mtk_vcodec_enc_hw.h       |  36 +++++
 5 files changed, 193 insertions(+), 1 deletion(-)
 create mode 100644 drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.c
 create mode 100644 drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.h

diff --git a/drivers/media/platform/mediatek/vcodec/Makefile b/drivers/media/platform/mediatek/vcodec/Makefile
index 93e7a343b5b0..ac068d88af29 100644
--- a/drivers/media/platform/mediatek/vcodec/Makefile
+++ b/drivers/media/platform/mediatek/vcodec/Makefile
@@ -3,7 +3,8 @@
 obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += mtk-vcodec-dec.o \
 				       mtk-vcodec-enc.o \
 				       mtk-vcodec-common.o \
-				       mtk-vcodec-dec-hw.o
+				       mtk-vcodec-dec-hw.o \
+				       mtk_vcodec_enc_hw.o
 
 mtk-vcodec-dec-y := vdec/vdec_h264_if.o \
 		vdec/vdec_vp8_if.o \
@@ -32,6 +33,7 @@ mtk-vcodec-enc-y := venc/venc_vp8_if.o \
 		venc_drv_if.o \
 		venc_vpu_if.o \
 
+mtk-vcodec-enc-hw-y := mtk_vcodec_enc_hw.o
 
 mtk-vcodec-common-y := mtk_vcodec_intr.o \
 		mtk_vcodec_util.o \
diff --git a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_drv.h b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_drv.h
index ef4584a46417..644021c5bbd4 100644
--- a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_drv.h
+++ b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_drv.h
@@ -95,6 +95,15 @@ enum mtk_fmt_type {
 	MTK_FMT_FRAME = 2,
 };
 
+/*
+ * enum mtk_venc_hw_id -- encoder hardware id
+ */
+enum mtk_venc_hw_id {
+	MTK_VENC_CORE_0 = 0,
+	MTK_VENC_CORE_1,
+	MTK_VENC_HW_MAX,
+};
+
 /*
  * enum mtk_vdec_hw_id - Hardware index used to separate
  *                         different hardware
@@ -464,6 +473,7 @@ struct mtk_vcodec_enc_pdata {
  * @dec_active_cnt: used to mark whether need to record register value
  * @vdec_racing_info: record register value
  * @dec_racing_info_mutex: mutex lock used for inner racing mode
+ * @enc_hw_dev: used to store venc core device
  */
 struct mtk_vcodec_dev {
 	struct v4l2_device v4l2_dev;
@@ -514,6 +524,8 @@ struct mtk_vcodec_dev {
 	u32 vdec_racing_info[132];
 	/* Protects access to vdec_racing_info data */
 	struct mutex dec_racing_info_mutex;
+
+	void *enc_hw_dev[MTK_VENC_HW_MAX];
 };
 
 static inline struct mtk_vcodec_ctx *fh_to_ctx(struct v4l2_fh *fh)
diff --git a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_drv.c b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_drv.c
index 95e8c29ccc65..9cb6ce1fa101 100644
--- a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_drv.c
+++ b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_drv.c
@@ -263,6 +263,12 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
 		goto err_enc_pm;
 	}
 
+	ret = of_platform_populate(pdev->dev.of_node, NULL, NULL, &pdev->dev);
+	if (ret) {
+		mtk_v4l2_err("Failed to populate children devices");
+		goto err_enc_pm;
+	}
+
 	pm_runtime_enable(&pdev->dev);
 
 	dev->reg_base[dev->venc_pdata->core_id] =
diff --git a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.c b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.c
new file mode 100644
index 000000000000..cc4938f027e0
--- /dev/null
+++ b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.c
@@ -0,0 +1,136 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2021 MediaTek Inc.
+ */
+
+#include <linux/interrupt.h>
+#include <linux/irq.h>
+#include <linux/module.h>
+#include <linux/of_platform.h>
+#include <linux/pm_runtime.h>
+#include <linux/slab.h>
+
+#include "mtk_vcodec_drv.h"
+#include "mtk_vcodec_enc.h"
+#include "mtk_vcodec_enc_hw.h"
+#include "mtk_vcodec_intr.h"
+
+static const struct of_device_id mtk_venc_hw_ids[] = {
+	{.compatible = "mediatek,mtk-venc-hw",},
+	{},
+};
+MODULE_DEVICE_TABLE(of, mtk_venc_hw_ids);
+
+static void clean_hw_irq_status(unsigned int irq_status, void __iomem *addr)
+{
+	if (irq_status & MTK_VENC_IRQ_STATUS_PAUSE)
+		writel(MTK_VENC_IRQ_STATUS_PAUSE, addr);
+
+	if (irq_status & MTK_VENC_IRQ_STATUS_SWITCH)
+		writel(MTK_VENC_IRQ_STATUS_SWITCH, addr);
+
+	if (irq_status & MTK_VENC_IRQ_STATUS_DRAM)
+		writel(MTK_VENC_IRQ_STATUS_DRAM, addr);
+
+	if (irq_status & MTK_VENC_IRQ_STATUS_SPS)
+		writel(MTK_VENC_IRQ_STATUS_SPS, addr);
+
+	if (irq_status & MTK_VENC_IRQ_STATUS_PPS)
+		writel(MTK_VENC_IRQ_STATUS_PPS, addr);
+
+	if (irq_status & MTK_VENC_IRQ_STATUS_FRM)
+		writel(MTK_VENC_IRQ_STATUS_FRM, addr);
+}
+
+static irqreturn_t mtk_enc_hw_irq_handler(int irq, void *priv)
+{
+	struct mtk_venc_hw_dev *dev = priv;
+	struct mtk_vcodec_ctx *ctx;
+	unsigned long flags;
+	void __iomem *addr;
+
+	spin_lock_irqsave(&dev->main_dev->irqlock, flags);
+	ctx = dev->curr_ctx;
+	spin_unlock_irqrestore(&dev->main_dev->irqlock, flags);
+	if (!ctx)
+		return IRQ_HANDLED;
+
+	mtk_v4l2_debug(1, "id=%d core :%d", ctx->id, dev->hw_id);
+
+	addr = dev->reg_base + MTK_VENC_IRQ_ACK_OFFSET;
+	ctx->irq_status = readl(dev->reg_base + MTK_VENC_IRQ_STATUS_OFFSET);
+	clean_hw_irq_status(ctx->irq_status, addr);
+
+	wake_up_ctx(ctx, MTK_INST_IRQ_RECEIVED, 0);
+	return IRQ_HANDLED;
+}
+
+static int mtk_venc_hw_probe(struct platform_device *pdev)
+{
+	struct device *dev = &pdev->dev;
+	struct mtk_venc_hw_dev *sub_core;
+	struct mtk_vcodec_dev *main_dev;
+	int ret;
+
+	if (!dev->parent)
+		return dev_err_probe(dev, -ENODEV,
+				     "No parent for venc core device\n");
+
+	main_dev = dev_get_drvdata(dev->parent);
+	if (!main_dev)
+		return dev_err_probe(dev, -EINVAL,
+				     "Failed to get parent driver data\n");
+
+	sub_core = devm_kzalloc(&pdev->dev, sizeof(*sub_core), GFP_KERNEL);
+	if (!sub_core)
+		return dev_err_probe(dev, -ENOMEM,
+				     "Failed to get alloc core data\n");
+
+	sub_core->plat_dev = pdev;
+
+	platform_set_drvdata(pdev, sub_core);
+
+	sub_core->reg_base = devm_platform_ioremap_resource(pdev, 0);
+	if (IS_ERR(sub_core->reg_base))
+		return dev_err_probe(dev, PTR_ERR(sub_core->reg_base),
+				     "Failed to get reg base\n");
+
+	sub_core->enc_irq = platform_get_irq(pdev, 0);
+	if (sub_core->enc_irq < 0)
+		return dev_err_probe(dev, -EINVAL,
+				     "Failed to get irq resource\n");
+
+	ret = devm_request_irq(dev, sub_core->enc_irq,
+			       mtk_enc_hw_irq_handler, 0,
+			       pdev->name, sub_core);
+	if (ret)
+		return dev_err_probe(dev, -EINVAL,
+				     "Failed to install sub_core->enc_irq %d\n",
+				     sub_core->enc_irq);
+
+	ret = of_property_read_u32(dev->of_node, "mediatek,hw-id",
+				   &sub_core->hw_id);
+
+	if (ret || sub_core->hw_id >= MTK_VENC_HW_MAX)
+		return dev_err_probe(dev, (ret ? ret : -EINVAL),
+				     "Cannot parse hardware id");
+
+	main_dev->enc_hw_dev[sub_core->hw_id] = sub_core;
+	sub_core->main_dev = main_dev;
+
+	dev_dbg(dev, "Venc core :%d probe done\n", sub_core->hw_id);
+
+	return 0;
+}
+
+static struct platform_driver mtk_venc_core_driver = {
+	.probe  = mtk_venc_hw_probe,
+	.driver = {
+		.name	 = "mtk-venc-core",
+		.of_match_table = mtk_venc_hw_ids,
+	},
+};
+module_platform_driver(mtk_venc_core_driver);
+
+MODULE_LICENSE("GPL");
+MODULE_DESCRIPTION("MediaTek video encoder core driver");
diff --git a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.h b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.h
new file mode 100644
index 000000000000..9daea9665659
--- /dev/null
+++ b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.h
@@ -0,0 +1,36 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2021 MediaTek Inc.
+ */
+
+#ifndef _MTK_VCODEC_ENC_HW_H_
+#define _MTK_VCODEC_ENC_HW_H_
+
+#include <linux/platform_device.h>
+#include "mtk_vcodec_drv.h"
+
+/**
+ * struct mtk_venc_hw_dev - driver data
+ * @plat_dev: platform_device
+ * @main_dev: main device
+ * @pm: power management data
+ * @curr_ctx: the context that is waiting for venc hardware
+ * @reg_base: mapped address of venc registers
+ * @irq_status: venc hardware irq status
+ * @enc_irq: venc device irq
+ * @hw_id: for venc hardware id: core#0, core#1...
+ */
+struct mtk_venc_hw_dev {
+	struct platform_device *plat_dev;
+	struct mtk_vcodec_dev *main_dev;
+
+	struct mtk_vcodec_pm pm;
+	struct mtk_vcodec_ctx *curr_ctx;
+
+	void __iomem *reg_base;
+	unsigned int irq_status;
+	int enc_irq;
+	enum mtk_venc_hw_id hw_id;
+};
+
+#endif /* _MTK_VCODEC_ENC_HW_H_ */
-- 
2.18.0


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply related	[flat|nested] 22+ messages in thread
* [PATCH v5, 3/8] media: mediatek: vcodec: Refactor venc power manage function
  2022-07-29  3:51 ` Irui Wang
@ 2022-07-29  3:51   ` Irui Wang
  -1 siblings, 0 replies; 22+ messages in thread
From: Irui Wang @ 2022-07-29  3:51 UTC (permalink / raw)
  To: Hans Verkuil, angelogioacchino.delregno, Rob Herring,
	Mauro Carvalho Chehab, Matthias Brugger, Alexandre Courbot,
	Tiffany Lin, Andrew-CT Chen, Tzung-Bi Shih, Tomasz Figa,
	nicolas.dufresne
  Cc: Maoguang Meng, Longfei Wang, Yunfei Dong, Irui Wang, linux-media,
	devicetree, linux-kernel, linux-arm-kernel, linux-mediatek,
	Project_Global_Chrome_Upstream_Group

The args "struct mtk_vcodec_dev *" doesn't appropriate for init_clk
functions because of sub-devices, sub-devices will init their own
"pm/clk" instead, so refactor the pm function with args
"platform_device *" and "mtk_vcodec_pm*".

Signed-off-by: Irui Wang <irui.wang@mediatek.com>
---
 .../media/platform/mediatek/vcodec/mtk_vcodec_enc_drv.c  | 2 +-
 .../media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.c   | 8 ++++++++
 .../media/platform/mediatek/vcodec/mtk_vcodec_enc_pm.c   | 9 +++------
 .../media/platform/mediatek/vcodec/mtk_vcodec_enc_pm.h   | 3 ++-
 4 files changed, 14 insertions(+), 8 deletions(-)

diff --git a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_drv.c b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_drv.c
index 9cb6ce1fa101..2e997b32a541 100644
--- a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_drv.c
+++ b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_drv.c
@@ -257,7 +257,7 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
 		return PTR_ERR(dev->fw_handler);
 
 	dev->venc_pdata = of_device_get_match_data(&pdev->dev);
-	ret = mtk_vcodec_init_enc_clk(dev);
+	ret = mtk_vcodec_init_enc_clk(dev->plat_dev, &dev->pm);
 	if (ret < 0) {
 		dev_err(&pdev->dev, "Failed to get mtk vcodec clock source!");
 		goto err_enc_pm;
diff --git a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.c b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.c
index cc4938f027e0..1ed4279925b7 100644
--- a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.c
+++ b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.c
@@ -13,6 +13,7 @@
 #include "mtk_vcodec_drv.h"
 #include "mtk_vcodec_enc.h"
 #include "mtk_vcodec_enc_hw.h"
+#include "mtk_vcodec_enc_pm.h"
 #include "mtk_vcodec_intr.h"
 
 static const struct of_device_id mtk_venc_hw_ids[] = {
@@ -115,6 +116,13 @@ static int mtk_venc_hw_probe(struct platform_device *pdev)
 		return dev_err_probe(dev, (ret ? ret : -EINVAL),
 				     "Cannot parse hardware id");
 
+	ret = mtk_vcodec_init_enc_clk(sub_core->plat_dev, &sub_core->pm);
+	if (ret < 0)
+		return dev_err_probe(dev, ret,
+				     "Failed to get venc core clock source!");
+
+	pm_runtime_enable(&pdev->dev);
+
 	main_dev->enc_hw_dev[sub_core->hw_id] = sub_core;
 	sub_core->main_dev = main_dev;
 
diff --git a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_pm.c b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_pm.c
index 7055954eb2af..75de5031d292 100644
--- a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_pm.c
+++ b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_pm.c
@@ -12,17 +12,13 @@
 #include "mtk_vcodec_enc_pm.h"
 #include "mtk_vcodec_util.h"
 
-int mtk_vcodec_init_enc_clk(struct mtk_vcodec_dev *mtkdev)
+int mtk_vcodec_init_enc_clk(struct platform_device *pdev,
+			    struct mtk_vcodec_pm *pm)
 {
-	struct platform_device *pdev;
-	struct mtk_vcodec_pm *pm;
 	struct mtk_vcodec_clk *enc_clk;
 	struct mtk_vcodec_clk_info *clk_info;
 	int ret, i;
 
-	pdev = mtkdev->plat_dev;
-	pm = &mtkdev->pm;
-	memset(pm, 0, sizeof(struct mtk_vcodec_pm));
 	pm->dev = &pdev->dev;
 	enc_clk = &pm->venc_clk;
 
@@ -58,6 +54,7 @@ int mtk_vcodec_init_enc_clk(struct mtk_vcodec_dev *mtkdev)
 
 	return 0;
 }
+EXPORT_SYMBOL_GPL(mtk_vcodec_init_enc_clk);
 
 void mtk_vcodec_enc_clock_on(struct mtk_vcodec_pm *pm)
 {
diff --git a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_pm.h b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_pm.h
index bc455cefc0cd..361dec5be47f 100644
--- a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_pm.h
+++ b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_pm.h
@@ -9,7 +9,8 @@
 
 #include "mtk_vcodec_drv.h"
 
-int mtk_vcodec_init_enc_clk(struct mtk_vcodec_dev *dev);
+int mtk_vcodec_init_enc_clk(struct platform_device *pdev,
+			    struct mtk_vcodec_pm *pm);
 
 void mtk_vcodec_enc_clock_on(struct mtk_vcodec_pm *pm);
 void mtk_vcodec_enc_clock_off(struct mtk_vcodec_pm *pm);
-- 
2.18.0


^ permalink raw reply related	[flat|nested] 22+ messages in thread
* [PATCH v5, 3/8] media: mediatek: vcodec: Refactor venc power manage function
@ 2022-07-29  3:51   ` Irui Wang
  0 siblings, 0 replies; 22+ messages in thread
From: Irui Wang @ 2022-07-29  3:51 UTC (permalink / raw)
  To: Hans Verkuil, angelogioacchino.delregno, Rob Herring,
	Mauro Carvalho Chehab, Matthias Brugger, Alexandre Courbot,
	Tiffany Lin, Andrew-CT Chen, Tzung-Bi Shih, Tomasz Figa,
	nicolas.dufresne
  Cc: Maoguang Meng, Longfei Wang, Yunfei Dong, Irui Wang, linux-media,
	devicetree, linux-kernel, linux-arm-kernel, linux-mediatek,
	Project_Global_Chrome_Upstream_Group

The args "struct mtk_vcodec_dev *" doesn't appropriate for init_clk
functions because of sub-devices, sub-devices will init their own
"pm/clk" instead, so refactor the pm function with args
"platform_device *" and "mtk_vcodec_pm*".

Signed-off-by: Irui Wang <irui.wang@mediatek.com>
---
 .../media/platform/mediatek/vcodec/mtk_vcodec_enc_drv.c  | 2 +-
 .../media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.c   | 8 ++++++++
 .../media/platform/mediatek/vcodec/mtk_vcodec_enc_pm.c   | 9 +++------
 .../media/platform/mediatek/vcodec/mtk_vcodec_enc_pm.h   | 3 ++-
 4 files changed, 14 insertions(+), 8 deletions(-)

diff --git a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_drv.c b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_drv.c
index 9cb6ce1fa101..2e997b32a541 100644
--- a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_drv.c
+++ b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_drv.c
@@ -257,7 +257,7 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
 		return PTR_ERR(dev->fw_handler);
 
 	dev->venc_pdata = of_device_get_match_data(&pdev->dev);
-	ret = mtk_vcodec_init_enc_clk(dev);
+	ret = mtk_vcodec_init_enc_clk(dev->plat_dev, &dev->pm);
 	if (ret < 0) {
 		dev_err(&pdev->dev, "Failed to get mtk vcodec clock source!");
 		goto err_enc_pm;
diff --git a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.c b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.c
index cc4938f027e0..1ed4279925b7 100644
--- a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.c
+++ b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.c
@@ -13,6 +13,7 @@
 #include "mtk_vcodec_drv.h"
 #include "mtk_vcodec_enc.h"
 #include "mtk_vcodec_enc_hw.h"
+#include "mtk_vcodec_enc_pm.h"
 #include "mtk_vcodec_intr.h"
 
 static const struct of_device_id mtk_venc_hw_ids[] = {
@@ -115,6 +116,13 @@ static int mtk_venc_hw_probe(struct platform_device *pdev)
 		return dev_err_probe(dev, (ret ? ret : -EINVAL),
 				     "Cannot parse hardware id");
 
+	ret = mtk_vcodec_init_enc_clk(sub_core->plat_dev, &sub_core->pm);
+	if (ret < 0)
+		return dev_err_probe(dev, ret,
+				     "Failed to get venc core clock source!");
+
+	pm_runtime_enable(&pdev->dev);
+
 	main_dev->enc_hw_dev[sub_core->hw_id] = sub_core;
 	sub_core->main_dev = main_dev;
 
diff --git a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_pm.c b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_pm.c
index 7055954eb2af..75de5031d292 100644
--- a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_pm.c
+++ b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_pm.c
@@ -12,17 +12,13 @@
 #include "mtk_vcodec_enc_pm.h"
 #include "mtk_vcodec_util.h"
 
-int mtk_vcodec_init_enc_clk(struct mtk_vcodec_dev *mtkdev)
+int mtk_vcodec_init_enc_clk(struct platform_device *pdev,
+			    struct mtk_vcodec_pm *pm)
 {
-	struct platform_device *pdev;
-	struct mtk_vcodec_pm *pm;
 	struct mtk_vcodec_clk *enc_clk;
 	struct mtk_vcodec_clk_info *clk_info;
 	int ret, i;
 
-	pdev = mtkdev->plat_dev;
-	pm = &mtkdev->pm;
-	memset(pm, 0, sizeof(struct mtk_vcodec_pm));
 	pm->dev = &pdev->dev;
 	enc_clk = &pm->venc_clk;
 
@@ -58,6 +54,7 @@ int mtk_vcodec_init_enc_clk(struct mtk_vcodec_dev *mtkdev)
 
 	return 0;
 }
+EXPORT_SYMBOL_GPL(mtk_vcodec_init_enc_clk);
 
 void mtk_vcodec_enc_clock_on(struct mtk_vcodec_pm *pm)
 {
diff --git a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_pm.h b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_pm.h
index bc455cefc0cd..361dec5be47f 100644
--- a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_pm.h
+++ b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_pm.h
@@ -9,7 +9,8 @@
 
 #include "mtk_vcodec_drv.h"
 
-int mtk_vcodec_init_enc_clk(struct mtk_vcodec_dev *dev);
+int mtk_vcodec_init_enc_clk(struct platform_device *pdev,
+			    struct mtk_vcodec_pm *pm);
 
 void mtk_vcodec_enc_clock_on(struct mtk_vcodec_pm *pm);
 void mtk_vcodec_enc_clock_off(struct mtk_vcodec_pm *pm);
-- 
2.18.0


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply related	[flat|nested] 22+ messages in thread
* [PATCH v5, 4/8] media: mediatek: vcodec: Add more extra processing for multi-core encoding
  2022-07-29  3:51 ` Irui Wang
@ 2022-07-29  3:51   ` Irui Wang
  -1 siblings, 0 replies; 22+ messages in thread
From: Irui Wang @ 2022-07-29  3:51 UTC (permalink / raw)
  To: Hans Verkuil, angelogioacchino.delregno, Rob Herring,
	Mauro Carvalho Chehab, Matthias Brugger, Alexandre Courbot,
	Tiffany Lin, Andrew-CT Chen, Tzung-Bi Shih, Tomasz Figa,
	nicolas.dufresne
  Cc: Maoguang Meng, Longfei Wang, Yunfei Dong, Irui Wang, linux-media,
	devicetree, linux-kernel, linux-arm-kernel, linux-mediatek,
	Project_Global_Chrome_Upstream_Group

Add a bit for indicating support multi-core encoding, because multi-core
encoding need more working buffers for encoder hardware. The working
buffers are allocated from kernel side then pass to scp firmware side
through shared memory, the struct definition must be kept align between
kernel and scp firmware side. New another shared memory struct for
multi-core encoding.

Signed-off-by: Irui Wang <irui.wang@mediatek.com>
---
 .../platform/mediatek/vcodec/mtk_vcodec_drv.h |   2 +
 .../mediatek/vcodec/mtk_vcodec_util.c         |  19 ++
 .../mediatek/vcodec/mtk_vcodec_util.h         |   2 +
 .../mediatek/vcodec/venc/venc_h264_if.c       | 207 +++++++++++++++---
 4 files changed, 195 insertions(+), 35 deletions(-)

diff --git a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_drv.h b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_drv.h
index 644021c5bbd4..250e9833b955 100644
--- a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_drv.h
+++ b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_drv.h
@@ -27,6 +27,8 @@
 #define WAIT_INTR_TIMEOUT_MS	1000
 #define IS_VDEC_LAT_ARCH(hw_arch) ((hw_arch) >= MTK_VDEC_LAT_SINGLE_CORE)
 #define IS_VDEC_INNER_RACING(capability) ((capability) & MTK_VCODEC_INNER_RACING)
+#define MTK_VENC_MULTICORE_ENABLE BIT(1)
+#define IS_VENC_MULTICORE(capability) ((capability) & MTK_VENC_MULTICORE_ENABLE)
 
 /*
  * enum mtk_hw_reg_idx - MTK hw register base index
diff --git a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_util.c b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_util.c
index ace78c4b5b9e..a4c2cbf4040e 100644
--- a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_util.c
+++ b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_util.c
@@ -11,6 +11,7 @@
 
 #include "mtk_vcodec_dec_hw.h"
 #include "mtk_vcodec_drv.h"
+#include "mtk_vcodec_enc_hw.h"
 #include "mtk_vcodec_util.h"
 
 void __iomem *mtk_vcodec_get_reg_addr(struct mtk_vcodec_ctx *data,
@@ -26,6 +27,24 @@ void __iomem *mtk_vcodec_get_reg_addr(struct mtk_vcodec_ctx *data,
 }
 EXPORT_SYMBOL(mtk_vcodec_get_reg_addr);
 
+void __iomem *mtk_venc_get_core_reg_addr(struct mtk_vcodec_ctx *ctx,
+					 int hw_id)
+{
+	struct mtk_venc_hw_dev *sub_core;
+
+	if (hw_id >= MTK_VENC_HW_MAX) {
+		mtk_v4l2_err("Invalid hw_id = %d", hw_id);
+		return NULL;
+	}
+
+	sub_core = (struct mtk_venc_hw_dev *)ctx->dev->enc_hw_dev[hw_id];
+	if (!sub_core)
+		return NULL;
+
+	return sub_core->reg_base;
+}
+EXPORT_SYMBOL(mtk_venc_get_core_reg_addr);
+
 int mtk_vcodec_mem_alloc(struct mtk_vcodec_ctx *data,
 			struct mtk_vcodec_mem *mem)
 {
diff --git a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_util.h b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_util.h
index 71956627a0e2..0033c53d5589 100644
--- a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_util.h
+++ b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_util.h
@@ -50,6 +50,8 @@ struct mtk_vcodec_dev;
 
 void __iomem *mtk_vcodec_get_reg_addr(struct mtk_vcodec_ctx *data,
 				unsigned int reg_idx);
+void __iomem *mtk_venc_get_core_reg_addr(struct mtk_vcodec_ctx *data,
+					 int hw_id);
 int mtk_vcodec_mem_alloc(struct mtk_vcodec_ctx *data,
 				struct mtk_vcodec_mem *mem);
 void mtk_vcodec_mem_free(struct mtk_vcodec_ctx *data,
diff --git a/drivers/media/platform/mediatek/vcodec/venc/venc_h264_if.c b/drivers/media/platform/mediatek/vcodec/venc/venc_h264_if.c
index 4d9b8798dffe..3a565929f2b0 100644
--- a/drivers/media/platform/mediatek/vcodec/venc/venc_h264_if.c
+++ b/drivers/media/platform/mediatek/vcodec/venc/venc_h264_if.c
@@ -50,6 +50,24 @@ enum venc_h264_vpu_work_buf {
 	VENC_H264_VPU_WORK_BUF_MAX,
 };
 
+/*
+ * enum venc_multi_core_work_buf - h264 multi core encoder buffer index
+ */
+enum venc_multi_core_work_buf {
+	VENC_MULTI_CORE_WORK_BUF_RC_INFO_CORE0,
+	VENC_MULTI_CORE_WORK_BUF_RC_CODE,
+	VENC_MULTI_CORE_WORK_BUF_REC_LUMA,
+	VENC_MULTI_CORE_WORK_BUF_REC_CHROMA,
+	VENC_MULTI_CORE_WORK_BUF_REF_LUMA,
+	VENC_MULTI_CORE_WORK_BUF_REF_CHROMA,
+	VENC_MULTI_CORE_WORK_BUF_MV_INFO_1,
+	VENC_MULTI_CORE_WORK_BUF_MV_INFO_2,
+	VENC_MULTI_CORE_WORK_BUF_SKIP_FRAME,
+	VENC_MULTI_CORE_WORK_BUF_RC_INFO_CORE1,
+	VENC_MULTI_CORE_WORK_BUF_FR_RC_INFO,
+	VENC_MULTI_CORE_WORK_BUF_MAX,
+};
+
 /*
  * enum venc_h264_bs_mode - for bs_mode argument in h264_enc_vpu_encode
  */
@@ -94,6 +112,47 @@ struct venc_h264_vpu_config {
 	u32 wfd;
 };
 
+/*
+ * struct venc_h264_vpu_config_ext - Structure for h264 encoder configuration
+ *                               AP-W/R : AP is writer/reader on this item
+ *                               VPU-W/R: VPU is write/reader on this item
+ * @input_fourcc: input fourcc
+ * @bitrate: target bitrate (in bps)
+ * @pic_w: picture width. Picture size is visible stream resolution, in pixels,
+ *         to be used for display purposes; must be smaller or equal to buffer
+ *         size.
+ * @pic_h: picture height
+ * @buf_w: buffer width. Buffer size is stream resolution in pixels aligned to
+ *         hardware requirements.
+ * @buf_h: buffer height
+ * @gop_size: group of picture size (idr frame)
+ * @intra_period: intra frame period
+ * @framerate: frame rate in fps
+ * @profile: as specified in standard
+ * @level: as specified in standard
+ * @wfd: WFD mode 1:on, 0:off
+ * @max_qp: max quant parameter
+ * @min_qp: min quant parameter
+ * @reserved: reserved configs
+ */
+struct venc_h264_vpu_config_ext {
+	u32 input_fourcc;
+	u32 bitrate;
+	u32 pic_w;
+	u32 pic_h;
+	u32 buf_w;
+	u32 buf_h;
+	u32 gop_size;
+	u32 intra_period;
+	u32 framerate;
+	u32 profile;
+	u32 level;
+	u32 wfd;
+	u32 max_qp;
+	u32 min_qp;
+	u32 reserved[8];
+};
+
 /*
  * struct venc_h264_vpu_buf - Structure for buffer information
  *                            AP-W/R : AP is writer/reader on this item
@@ -127,6 +186,17 @@ struct venc_h264_vsi {
 	struct venc_h264_vpu_buf work_bufs[VENC_H264_VPU_WORK_BUF_MAX];
 };
 
+/*
+ * struct venc_multi_core_vsi - Structure for VPU driver control and info share
+ *				Used for multi-core encode sharing
+ * @config: h264 encoder configuration
+ * @work_bufs: working buffer information in VPU side
+ */
+struct venc_multi_core_vsi {
+	struct venc_h264_vpu_config_ext config;
+	struct venc_h264_vpu_buf work_bufs[VENC_MULTI_CORE_WORK_BUF_MAX];
+};
+
 /*
  * struct venc_h264_inst - h264 encoder AP driver instance
  * @hw_base: h264 encoder hardware register base
@@ -140,11 +210,12 @@ struct venc_h264_vsi {
  * @vpu_inst: VPU instance to exchange information between AP and VPU
  * @vsi: driver structure allocated by VPU side and shared to AP side for
  *	 control and info share
+ * @core_vsi: used for multi-core encode info sharing.
  * @ctx: context for v4l2 layer integration
  */
 struct venc_h264_inst {
-	void __iomem *hw_base;
-	struct mtk_vcodec_mem work_bufs[VENC_H264_VPU_WORK_BUF_MAX];
+	void __iomem *hw_base[MTK_VENC_HW_MAX];
+	struct mtk_vcodec_mem work_bufs[VENC_MULTI_CORE_WORK_BUF_MAX];
 	struct mtk_vcodec_mem pps_buf;
 	bool work_buf_allocated;
 	unsigned int frm_cnt;
@@ -152,12 +223,13 @@ struct venc_h264_inst {
 	unsigned int prepend_hdr;
 	struct venc_vpu_inst vpu_inst;
 	struct venc_h264_vsi *vsi;
+	struct venc_multi_core_vsi *core_vsi;
 	struct mtk_vcodec_ctx *ctx;
 };
 
 static inline u32 h264_read_reg(struct venc_h264_inst *inst, u32 addr)
 {
-	return readl(inst->hw_base + addr);
+	return readl(inst->hw_base[MTK_VENC_CORE_0] + addr);
 }
 
 static unsigned int h264_get_profile(struct venc_h264_inst *inst,
@@ -227,14 +299,20 @@ static unsigned int h264_get_level(struct venc_h264_inst *inst,
 
 static void h264_enc_free_work_buf(struct venc_h264_inst *inst)
 {
-	int i;
+	struct mtk_vcodec_ctx *ctx = inst->ctx;
+	int i, max_work_buf;
 
 	mtk_vcodec_debug_enter(inst);
 
+	if (IS_VENC_MULTICORE(ctx->dev->enc_capability))
+		max_work_buf = VENC_MULTI_CORE_WORK_BUF_MAX;
+	else
+		max_work_buf = VENC_H264_VPU_WORK_BUF_MAX;
+
 	/* Except the SKIP_FRAME buffers,
 	 * other buffers need to be freed by AP.
 	 */
-	for (i = 0; i < VENC_H264_VPU_WORK_BUF_MAX; i++) {
+	for (i = 0; i < max_work_buf; i++) {
 		if (i != VENC_H264_VPU_WORK_BUF_SKIP_FRAME)
 			mtk_vcodec_mem_free(inst->ctx, &inst->work_bufs[i]);
 	}
@@ -246,13 +324,22 @@ static void h264_enc_free_work_buf(struct venc_h264_inst *inst)
 
 static int h264_enc_alloc_work_buf(struct venc_h264_inst *inst)
 {
-	int i;
+	struct mtk_vcodec_ctx *ctx = inst->ctx;
+	struct venc_h264_vpu_buf *wb;
+	int i, max_work_buf;
 	int ret = 0;
-	struct venc_h264_vpu_buf *wb = inst->vsi->work_bufs;
 
 	mtk_vcodec_debug_enter(inst);
 
-	for (i = 0; i < VENC_H264_VPU_WORK_BUF_MAX; i++) {
+	if (IS_VENC_MULTICORE(ctx->dev->enc_capability)) {
+		wb = inst->core_vsi->work_bufs;
+		max_work_buf = VENC_MULTI_CORE_WORK_BUF_MAX;
+	} else {
+		wb = inst->vsi->work_bufs;
+		max_work_buf = VENC_H264_VPU_WORK_BUF_MAX;
+	}
+
+	for (i = 0; i < max_work_buf; i++) {
 		/*
 		 * This 'wb' structure is set by VPU side and shared to AP for
 		 * buffer allocation and IO virtual addr mapping. For most of
@@ -342,22 +429,22 @@ static unsigned int h264_enc_wait_venc_done(struct venc_h264_inst *inst)
 	return irq_status;
 }
 
-static int h264_frame_type(struct venc_h264_inst *inst)
+static int h264_frame_type(unsigned int frm_cnt, unsigned int gop_size,
+			   unsigned int intra_period)
 {
-	if ((inst->vsi->config.gop_size != 0 &&
-	     (inst->frm_cnt % inst->vsi->config.gop_size) == 0) ||
-	    (inst->frm_cnt == 0 && inst->vsi->config.gop_size == 0)) {
+	if ((gop_size != 0 && (frm_cnt % gop_size) == 0) ||
+	    (frm_cnt == 0 && gop_size == 0)) {
 		/* IDR frame */
 		return VENC_H264_IDR_FRM;
-	} else if ((inst->vsi->config.intra_period != 0 &&
-		    (inst->frm_cnt % inst->vsi->config.intra_period) == 0) ||
-		   (inst->frm_cnt == 0 && inst->vsi->config.intra_period == 0)) {
+	} else if ((intra_period != 0 && (frm_cnt % intra_period) == 0) ||
+		   (frm_cnt == 0 && intra_period == 0)) {
 		/* I frame */
 		return VENC_H264_I_FRM;
 	} else {
 		return VENC_H264_P_FRM;  /* Note: B frames are not supported */
 	}
 }
+
 static int h264_encode_sps(struct venc_h264_inst *inst,
 			   struct mtk_vcodec_mem *bs_buf,
 			   unsigned int *bs_size)
@@ -437,15 +524,25 @@ static int h264_encode_frame(struct venc_h264_inst *inst,
 			     struct mtk_vcodec_mem *bs_buf,
 			     unsigned int *bs_size)
 {
-	int ret = 0;
-	unsigned int irq_status;
 	struct venc_frame_info frame_info;
+	struct mtk_vcodec_ctx *ctx = inst->ctx;
+	unsigned int irq_status, gop_size, intra_period;
+	int ret = 0;
 
 	mtk_vcodec_debug_enter(inst);
 	mtk_vcodec_debug(inst, "frm_cnt = %d\n ", inst->frm_cnt);
+
+	if (IS_VENC_MULTICORE(ctx->dev->enc_capability)) {
+		gop_size = inst->core_vsi->config.gop_size;
+		intra_period = inst->core_vsi->config.intra_period;
+	} else {
+		gop_size = inst->vsi->config.gop_size;
+		intra_period = inst->vsi->config.intra_period;
+	}
 	frame_info.frm_count = inst->frm_cnt;
 	frame_info.skip_frm_count = inst->skip_frm_cnt;
-	frame_info.frm_type = h264_frame_type(inst);
+	frame_info.frm_type = h264_frame_type(inst->frm_cnt, gop_size,
+					      intra_period);
 	mtk_vcodec_debug(inst, "frm_count = %d,skip_frm_count =%d,frm_type=%d.\n",
 			 frame_info.frm_count, frame_info.skip_frm_count,
 			 frame_info.frm_type);
@@ -501,8 +598,8 @@ static void h264_encode_filler(struct venc_h264_inst *inst, void *buf,
 static int h264_enc_init(struct mtk_vcodec_ctx *ctx)
 {
 	const bool is_ext = MTK_ENC_CTX_IS_EXT(ctx);
-	int ret = 0;
 	struct venc_h264_inst *inst;
+	int ret, i;
 
 	inst = kzalloc(sizeof(*inst), GFP_KERNEL);
 	if (!inst)
@@ -511,13 +608,23 @@ static int h264_enc_init(struct mtk_vcodec_ctx *ctx)
 	inst->ctx = ctx;
 	inst->vpu_inst.ctx = ctx;
 	inst->vpu_inst.id = is_ext ? SCP_IPI_VENC_H264 : IPI_VENC_H264;
-	inst->hw_base = mtk_vcodec_get_reg_addr(inst->ctx, VENC_SYS);
 
 	mtk_vcodec_debug_enter(inst);
 
 	ret = vpu_enc_init(&inst->vpu_inst);
 
-	inst->vsi = (struct venc_h264_vsi *)inst->vpu_inst.vsi;
+	inst->hw_base[0] = mtk_vcodec_get_reg_addr(inst->ctx, VENC_SYS);
+
+	if (IS_VENC_MULTICORE(ctx->dev->enc_capability)) {
+		inst->core_vsi =
+			(struct venc_multi_core_vsi *)inst->vpu_inst.vsi;
+
+		for (i = 1; i < MTK_VENC_HW_MAX; i++)
+			inst->hw_base[i] =
+				mtk_venc_get_core_reg_addr(inst->ctx, i);
+	} else {
+		inst->vsi = (struct venc_h264_vsi *)inst->vpu_inst.vsi;
+	}
 
 	mtk_vcodec_debug_leave(inst);
 
@@ -624,31 +731,61 @@ static int h264_enc_encode(void *handle,
 	return ret;
 }
 
+static void h264_enc_set_configs(struct venc_h264_inst *inst,
+				 struct venc_enc_param *enc_prm)
+{
+	inst->vsi->config.input_fourcc = enc_prm->input_yuv_fmt;
+	inst->vsi->config.bitrate = enc_prm->bitrate;
+	inst->vsi->config.pic_w = enc_prm->width;
+	inst->vsi->config.pic_h = enc_prm->height;
+	inst->vsi->config.buf_w = enc_prm->buf_width;
+	inst->vsi->config.buf_h = enc_prm->buf_height;
+	inst->vsi->config.gop_size = enc_prm->gop_size;
+	inst->vsi->config.framerate = enc_prm->frm_rate;
+	inst->vsi->config.intra_period = enc_prm->intra_period;
+	inst->vsi->config.profile =
+		h264_get_profile(inst, enc_prm->h264_profile);
+	inst->vsi->config.level =
+		h264_get_level(inst, enc_prm->h264_level);
+	inst->vsi->config.wfd = 0;
+}
+
+static void h264_enc_set_core_configs(struct venc_h264_inst *inst,
+				      struct venc_enc_param *enc_prm)
+{
+	inst->core_vsi->config.input_fourcc = enc_prm->input_yuv_fmt;
+	inst->core_vsi->config.bitrate = enc_prm->bitrate;
+	inst->core_vsi->config.pic_w = enc_prm->width;
+	inst->core_vsi->config.pic_h = enc_prm->height;
+	inst->core_vsi->config.buf_w = enc_prm->buf_width;
+	inst->core_vsi->config.buf_h = enc_prm->buf_height;
+	inst->core_vsi->config.gop_size = enc_prm->gop_size;
+	inst->core_vsi->config.framerate = enc_prm->frm_rate;
+	inst->core_vsi->config.intra_period = enc_prm->intra_period;
+	inst->core_vsi->config.profile =
+		h264_get_profile(inst, enc_prm->h264_profile);
+	inst->core_vsi->config.level =
+		h264_get_level(inst, enc_prm->h264_level);
+	inst->core_vsi->config.wfd = 0;
+}
+
 static int h264_enc_set_param(void *handle,
 			      enum venc_set_param_type type,
 			      struct venc_enc_param *enc_prm)
 {
 	int ret = 0;
 	struct venc_h264_inst *inst = (struct venc_h264_inst *)handle;
+	struct mtk_vcodec_ctx *ctx = inst->ctx;
 
 	mtk_vcodec_debug(inst, "->type=%d", type);
 
 	switch (type) {
 	case VENC_SET_PARAM_ENC:
-		inst->vsi->config.input_fourcc = enc_prm->input_yuv_fmt;
-		inst->vsi->config.bitrate = enc_prm->bitrate;
-		inst->vsi->config.pic_w = enc_prm->width;
-		inst->vsi->config.pic_h = enc_prm->height;
-		inst->vsi->config.buf_w = enc_prm->buf_width;
-		inst->vsi->config.buf_h = enc_prm->buf_height;
-		inst->vsi->config.gop_size = enc_prm->gop_size;
-		inst->vsi->config.framerate = enc_prm->frm_rate;
-		inst->vsi->config.intra_period = enc_prm->intra_period;
-		inst->vsi->config.profile =
-			h264_get_profile(inst, enc_prm->h264_profile);
-		inst->vsi->config.level =
-			h264_get_level(inst, enc_prm->h264_level);
-		inst->vsi->config.wfd = 0;
+		if (IS_VENC_MULTICORE(ctx->dev->enc_capability))
+			h264_enc_set_core_configs(inst, enc_prm);
+		else
+			h264_enc_set_configs(inst, enc_prm);
+
 		ret = vpu_enc_set_param(&inst->vpu_inst, type, enc_prm);
 		if (ret)
 			break;
-- 
2.18.0


^ permalink raw reply related	[flat|nested] 22+ messages in thread
* [PATCH v5, 4/8] media: mediatek: vcodec: Add more extra processing for multi-core encoding
@ 2022-07-29  3:51   ` Irui Wang
  0 siblings, 0 replies; 22+ messages in thread
From: Irui Wang @ 2022-07-29  3:51 UTC (permalink / raw)
  To: Hans Verkuil, angelogioacchino.delregno, Rob Herring,
	Mauro Carvalho Chehab, Matthias Brugger, Alexandre Courbot,
	Tiffany Lin, Andrew-CT Chen, Tzung-Bi Shih, Tomasz Figa,
	nicolas.dufresne
  Cc: Maoguang Meng, Longfei Wang, Yunfei Dong, Irui Wang, linux-media,
	devicetree, linux-kernel, linux-arm-kernel, linux-mediatek,
	Project_Global_Chrome_Upstream_Group

Add a bit for indicating support multi-core encoding, because multi-core
encoding need more working buffers for encoder hardware. The working
buffers are allocated from kernel side then pass to scp firmware side
through shared memory, the struct definition must be kept align between
kernel and scp firmware side. New another shared memory struct for
multi-core encoding.

Signed-off-by: Irui Wang <irui.wang@mediatek.com>
---
 .../platform/mediatek/vcodec/mtk_vcodec_drv.h |   2 +
 .../mediatek/vcodec/mtk_vcodec_util.c         |  19 ++
 .../mediatek/vcodec/mtk_vcodec_util.h         |   2 +
 .../mediatek/vcodec/venc/venc_h264_if.c       | 207 +++++++++++++++---
 4 files changed, 195 insertions(+), 35 deletions(-)

diff --git a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_drv.h b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_drv.h
index 644021c5bbd4..250e9833b955 100644
--- a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_drv.h
+++ b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_drv.h
@@ -27,6 +27,8 @@
 #define WAIT_INTR_TIMEOUT_MS	1000
 #define IS_VDEC_LAT_ARCH(hw_arch) ((hw_arch) >= MTK_VDEC_LAT_SINGLE_CORE)
 #define IS_VDEC_INNER_RACING(capability) ((capability) & MTK_VCODEC_INNER_RACING)
+#define MTK_VENC_MULTICORE_ENABLE BIT(1)
+#define IS_VENC_MULTICORE(capability) ((capability) & MTK_VENC_MULTICORE_ENABLE)
 
 /*
  * enum mtk_hw_reg_idx - MTK hw register base index
diff --git a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_util.c b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_util.c
index ace78c4b5b9e..a4c2cbf4040e 100644
--- a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_util.c
+++ b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_util.c
@@ -11,6 +11,7 @@
 
 #include "mtk_vcodec_dec_hw.h"
 #include "mtk_vcodec_drv.h"
+#include "mtk_vcodec_enc_hw.h"
 #include "mtk_vcodec_util.h"
 
 void __iomem *mtk_vcodec_get_reg_addr(struct mtk_vcodec_ctx *data,
@@ -26,6 +27,24 @@ void __iomem *mtk_vcodec_get_reg_addr(struct mtk_vcodec_ctx *data,
 }
 EXPORT_SYMBOL(mtk_vcodec_get_reg_addr);
 
+void __iomem *mtk_venc_get_core_reg_addr(struct mtk_vcodec_ctx *ctx,
+					 int hw_id)
+{
+	struct mtk_venc_hw_dev *sub_core;
+
+	if (hw_id >= MTK_VENC_HW_MAX) {
+		mtk_v4l2_err("Invalid hw_id = %d", hw_id);
+		return NULL;
+	}
+
+	sub_core = (struct mtk_venc_hw_dev *)ctx->dev->enc_hw_dev[hw_id];
+	if (!sub_core)
+		return NULL;
+
+	return sub_core->reg_base;
+}
+EXPORT_SYMBOL(mtk_venc_get_core_reg_addr);
+
 int mtk_vcodec_mem_alloc(struct mtk_vcodec_ctx *data,
 			struct mtk_vcodec_mem *mem)
 {
diff --git a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_util.h b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_util.h
index 71956627a0e2..0033c53d5589 100644
--- a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_util.h
+++ b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_util.h
@@ -50,6 +50,8 @@ struct mtk_vcodec_dev;
 
 void __iomem *mtk_vcodec_get_reg_addr(struct mtk_vcodec_ctx *data,
 				unsigned int reg_idx);
+void __iomem *mtk_venc_get_core_reg_addr(struct mtk_vcodec_ctx *data,
+					 int hw_id);
 int mtk_vcodec_mem_alloc(struct mtk_vcodec_ctx *data,
 				struct mtk_vcodec_mem *mem);
 void mtk_vcodec_mem_free(struct mtk_vcodec_ctx *data,
diff --git a/drivers/media/platform/mediatek/vcodec/venc/venc_h264_if.c b/drivers/media/platform/mediatek/vcodec/venc/venc_h264_if.c
index 4d9b8798dffe..3a565929f2b0 100644
--- a/drivers/media/platform/mediatek/vcodec/venc/venc_h264_if.c
+++ b/drivers/media/platform/mediatek/vcodec/venc/venc_h264_if.c
@@ -50,6 +50,24 @@ enum venc_h264_vpu_work_buf {
 	VENC_H264_VPU_WORK_BUF_MAX,
 };
 
+/*
+ * enum venc_multi_core_work_buf - h264 multi core encoder buffer index
+ */
+enum venc_multi_core_work_buf {
+	VENC_MULTI_CORE_WORK_BUF_RC_INFO_CORE0,
+	VENC_MULTI_CORE_WORK_BUF_RC_CODE,
+	VENC_MULTI_CORE_WORK_BUF_REC_LUMA,
+	VENC_MULTI_CORE_WORK_BUF_REC_CHROMA,
+	VENC_MULTI_CORE_WORK_BUF_REF_LUMA,
+	VENC_MULTI_CORE_WORK_BUF_REF_CHROMA,
+	VENC_MULTI_CORE_WORK_BUF_MV_INFO_1,
+	VENC_MULTI_CORE_WORK_BUF_MV_INFO_2,
+	VENC_MULTI_CORE_WORK_BUF_SKIP_FRAME,
+	VENC_MULTI_CORE_WORK_BUF_RC_INFO_CORE1,
+	VENC_MULTI_CORE_WORK_BUF_FR_RC_INFO,
+	VENC_MULTI_CORE_WORK_BUF_MAX,
+};
+
 /*
  * enum venc_h264_bs_mode - for bs_mode argument in h264_enc_vpu_encode
  */
@@ -94,6 +112,47 @@ struct venc_h264_vpu_config {
 	u32 wfd;
 };
 
+/*
+ * struct venc_h264_vpu_config_ext - Structure for h264 encoder configuration
+ *                               AP-W/R : AP is writer/reader on this item
+ *                               VPU-W/R: VPU is write/reader on this item
+ * @input_fourcc: input fourcc
+ * @bitrate: target bitrate (in bps)
+ * @pic_w: picture width. Picture size is visible stream resolution, in pixels,
+ *         to be used for display purposes; must be smaller or equal to buffer
+ *         size.
+ * @pic_h: picture height
+ * @buf_w: buffer width. Buffer size is stream resolution in pixels aligned to
+ *         hardware requirements.
+ * @buf_h: buffer height
+ * @gop_size: group of picture size (idr frame)
+ * @intra_period: intra frame period
+ * @framerate: frame rate in fps
+ * @profile: as specified in standard
+ * @level: as specified in standard
+ * @wfd: WFD mode 1:on, 0:off
+ * @max_qp: max quant parameter
+ * @min_qp: min quant parameter
+ * @reserved: reserved configs
+ */
+struct venc_h264_vpu_config_ext {
+	u32 input_fourcc;
+	u32 bitrate;
+	u32 pic_w;
+	u32 pic_h;
+	u32 buf_w;
+	u32 buf_h;
+	u32 gop_size;
+	u32 intra_period;
+	u32 framerate;
+	u32 profile;
+	u32 level;
+	u32 wfd;
+	u32 max_qp;
+	u32 min_qp;
+	u32 reserved[8];
+};
+
 /*
  * struct venc_h264_vpu_buf - Structure for buffer information
  *                            AP-W/R : AP is writer/reader on this item
@@ -127,6 +186,17 @@ struct venc_h264_vsi {
 	struct venc_h264_vpu_buf work_bufs[VENC_H264_VPU_WORK_BUF_MAX];
 };
 
+/*
+ * struct venc_multi_core_vsi - Structure for VPU driver control and info share
+ *				Used for multi-core encode sharing
+ * @config: h264 encoder configuration
+ * @work_bufs: working buffer information in VPU side
+ */
+struct venc_multi_core_vsi {
+	struct venc_h264_vpu_config_ext config;
+	struct venc_h264_vpu_buf work_bufs[VENC_MULTI_CORE_WORK_BUF_MAX];
+};
+
 /*
  * struct venc_h264_inst - h264 encoder AP driver instance
  * @hw_base: h264 encoder hardware register base
@@ -140,11 +210,12 @@ struct venc_h264_vsi {
  * @vpu_inst: VPU instance to exchange information between AP and VPU
  * @vsi: driver structure allocated by VPU side and shared to AP side for
  *	 control and info share
+ * @core_vsi: used for multi-core encode info sharing.
  * @ctx: context for v4l2 layer integration
  */
 struct venc_h264_inst {
-	void __iomem *hw_base;
-	struct mtk_vcodec_mem work_bufs[VENC_H264_VPU_WORK_BUF_MAX];
+	void __iomem *hw_base[MTK_VENC_HW_MAX];
+	struct mtk_vcodec_mem work_bufs[VENC_MULTI_CORE_WORK_BUF_MAX];
 	struct mtk_vcodec_mem pps_buf;
 	bool work_buf_allocated;
 	unsigned int frm_cnt;
@@ -152,12 +223,13 @@ struct venc_h264_inst {
 	unsigned int prepend_hdr;
 	struct venc_vpu_inst vpu_inst;
 	struct venc_h264_vsi *vsi;
+	struct venc_multi_core_vsi *core_vsi;
 	struct mtk_vcodec_ctx *ctx;
 };
 
 static inline u32 h264_read_reg(struct venc_h264_inst *inst, u32 addr)
 {
-	return readl(inst->hw_base + addr);
+	return readl(inst->hw_base[MTK_VENC_CORE_0] + addr);
 }
 
 static unsigned int h264_get_profile(struct venc_h264_inst *inst,
@@ -227,14 +299,20 @@ static unsigned int h264_get_level(struct venc_h264_inst *inst,
 
 static void h264_enc_free_work_buf(struct venc_h264_inst *inst)
 {
-	int i;
+	struct mtk_vcodec_ctx *ctx = inst->ctx;
+	int i, max_work_buf;
 
 	mtk_vcodec_debug_enter(inst);
 
+	if (IS_VENC_MULTICORE(ctx->dev->enc_capability))
+		max_work_buf = VENC_MULTI_CORE_WORK_BUF_MAX;
+	else
+		max_work_buf = VENC_H264_VPU_WORK_BUF_MAX;
+
 	/* Except the SKIP_FRAME buffers,
 	 * other buffers need to be freed by AP.
 	 */
-	for (i = 0; i < VENC_H264_VPU_WORK_BUF_MAX; i++) {
+	for (i = 0; i < max_work_buf; i++) {
 		if (i != VENC_H264_VPU_WORK_BUF_SKIP_FRAME)
 			mtk_vcodec_mem_free(inst->ctx, &inst->work_bufs[i]);
 	}
@@ -246,13 +324,22 @@ static void h264_enc_free_work_buf(struct venc_h264_inst *inst)
 
 static int h264_enc_alloc_work_buf(struct venc_h264_inst *inst)
 {
-	int i;
+	struct mtk_vcodec_ctx *ctx = inst->ctx;
+	struct venc_h264_vpu_buf *wb;
+	int i, max_work_buf;
 	int ret = 0;
-	struct venc_h264_vpu_buf *wb = inst->vsi->work_bufs;
 
 	mtk_vcodec_debug_enter(inst);
 
-	for (i = 0; i < VENC_H264_VPU_WORK_BUF_MAX; i++) {
+	if (IS_VENC_MULTICORE(ctx->dev->enc_capability)) {
+		wb = inst->core_vsi->work_bufs;
+		max_work_buf = VENC_MULTI_CORE_WORK_BUF_MAX;
+	} else {
+		wb = inst->vsi->work_bufs;
+		max_work_buf = VENC_H264_VPU_WORK_BUF_MAX;
+	}
+
+	for (i = 0; i < max_work_buf; i++) {
 		/*
 		 * This 'wb' structure is set by VPU side and shared to AP for
 		 * buffer allocation and IO virtual addr mapping. For most of
@@ -342,22 +429,22 @@ static unsigned int h264_enc_wait_venc_done(struct venc_h264_inst *inst)
 	return irq_status;
 }
 
-static int h264_frame_type(struct venc_h264_inst *inst)
+static int h264_frame_type(unsigned int frm_cnt, unsigned int gop_size,
+			   unsigned int intra_period)
 {
-	if ((inst->vsi->config.gop_size != 0 &&
-	     (inst->frm_cnt % inst->vsi->config.gop_size) == 0) ||
-	    (inst->frm_cnt == 0 && inst->vsi->config.gop_size == 0)) {
+	if ((gop_size != 0 && (frm_cnt % gop_size) == 0) ||
+	    (frm_cnt == 0 && gop_size == 0)) {
 		/* IDR frame */
 		return VENC_H264_IDR_FRM;
-	} else if ((inst->vsi->config.intra_period != 0 &&
-		    (inst->frm_cnt % inst->vsi->config.intra_period) == 0) ||
-		   (inst->frm_cnt == 0 && inst->vsi->config.intra_period == 0)) {
+	} else if ((intra_period != 0 && (frm_cnt % intra_period) == 0) ||
+		   (frm_cnt == 0 && intra_period == 0)) {
 		/* I frame */
 		return VENC_H264_I_FRM;
 	} else {
 		return VENC_H264_P_FRM;  /* Note: B frames are not supported */
 	}
 }
+
 static int h264_encode_sps(struct venc_h264_inst *inst,
 			   struct mtk_vcodec_mem *bs_buf,
 			   unsigned int *bs_size)
@@ -437,15 +524,25 @@ static int h264_encode_frame(struct venc_h264_inst *inst,
 			     struct mtk_vcodec_mem *bs_buf,
 			     unsigned int *bs_size)
 {
-	int ret = 0;
-	unsigned int irq_status;
 	struct venc_frame_info frame_info;
+	struct mtk_vcodec_ctx *ctx = inst->ctx;
+	unsigned int irq_status, gop_size, intra_period;
+	int ret = 0;
 
 	mtk_vcodec_debug_enter(inst);
 	mtk_vcodec_debug(inst, "frm_cnt = %d\n ", inst->frm_cnt);
+
+	if (IS_VENC_MULTICORE(ctx->dev->enc_capability)) {
+		gop_size = inst->core_vsi->config.gop_size;
+		intra_period = inst->core_vsi->config.intra_period;
+	} else {
+		gop_size = inst->vsi->config.gop_size;
+		intra_period = inst->vsi->config.intra_period;
+	}
 	frame_info.frm_count = inst->frm_cnt;
 	frame_info.skip_frm_count = inst->skip_frm_cnt;
-	frame_info.frm_type = h264_frame_type(inst);
+	frame_info.frm_type = h264_frame_type(inst->frm_cnt, gop_size,
+					      intra_period);
 	mtk_vcodec_debug(inst, "frm_count = %d,skip_frm_count =%d,frm_type=%d.\n",
 			 frame_info.frm_count, frame_info.skip_frm_count,
 			 frame_info.frm_type);
@@ -501,8 +598,8 @@ static void h264_encode_filler(struct venc_h264_inst *inst, void *buf,
 static int h264_enc_init(struct mtk_vcodec_ctx *ctx)
 {
 	const bool is_ext = MTK_ENC_CTX_IS_EXT(ctx);
-	int ret = 0;
 	struct venc_h264_inst *inst;
+	int ret, i;
 
 	inst = kzalloc(sizeof(*inst), GFP_KERNEL);
 	if (!inst)
@@ -511,13 +608,23 @@ static int h264_enc_init(struct mtk_vcodec_ctx *ctx)
 	inst->ctx = ctx;
 	inst->vpu_inst.ctx = ctx;
 	inst->vpu_inst.id = is_ext ? SCP_IPI_VENC_H264 : IPI_VENC_H264;
-	inst->hw_base = mtk_vcodec_get_reg_addr(inst->ctx, VENC_SYS);
 
 	mtk_vcodec_debug_enter(inst);
 
 	ret = vpu_enc_init(&inst->vpu_inst);
 
-	inst->vsi = (struct venc_h264_vsi *)inst->vpu_inst.vsi;
+	inst->hw_base[0] = mtk_vcodec_get_reg_addr(inst->ctx, VENC_SYS);
+
+	if (IS_VENC_MULTICORE(ctx->dev->enc_capability)) {
+		inst->core_vsi =
+			(struct venc_multi_core_vsi *)inst->vpu_inst.vsi;
+
+		for (i = 1; i < MTK_VENC_HW_MAX; i++)
+			inst->hw_base[i] =
+				mtk_venc_get_core_reg_addr(inst->ctx, i);
+	} else {
+		inst->vsi = (struct venc_h264_vsi *)inst->vpu_inst.vsi;
+	}
 
 	mtk_vcodec_debug_leave(inst);
 
@@ -624,31 +731,61 @@ static int h264_enc_encode(void *handle,
 	return ret;
 }
 
+static void h264_enc_set_configs(struct venc_h264_inst *inst,
+				 struct venc_enc_param *enc_prm)
+{
+	inst->vsi->config.input_fourcc = enc_prm->input_yuv_fmt;
+	inst->vsi->config.bitrate = enc_prm->bitrate;
+	inst->vsi->config.pic_w = enc_prm->width;
+	inst->vsi->config.pic_h = enc_prm->height;
+	inst->vsi->config.buf_w = enc_prm->buf_width;
+	inst->vsi->config.buf_h = enc_prm->buf_height;
+	inst->vsi->config.gop_size = enc_prm->gop_size;
+	inst->vsi->config.framerate = enc_prm->frm_rate;
+	inst->vsi->config.intra_period = enc_prm->intra_period;
+	inst->vsi->config.profile =
+		h264_get_profile(inst, enc_prm->h264_profile);
+	inst->vsi->config.level =
+		h264_get_level(inst, enc_prm->h264_level);
+	inst->vsi->config.wfd = 0;
+}
+
+static void h264_enc_set_core_configs(struct venc_h264_inst *inst,
+				      struct venc_enc_param *enc_prm)
+{
+	inst->core_vsi->config.input_fourcc = enc_prm->input_yuv_fmt;
+	inst->core_vsi->config.bitrate = enc_prm->bitrate;
+	inst->core_vsi->config.pic_w = enc_prm->width;
+	inst->core_vsi->config.pic_h = enc_prm->height;
+	inst->core_vsi->config.buf_w = enc_prm->buf_width;
+	inst->core_vsi->config.buf_h = enc_prm->buf_height;
+	inst->core_vsi->config.gop_size = enc_prm->gop_size;
+	inst->core_vsi->config.framerate = enc_prm->frm_rate;
+	inst->core_vsi->config.intra_period = enc_prm->intra_period;
+	inst->core_vsi->config.profile =
+		h264_get_profile(inst, enc_prm->h264_profile);
+	inst->core_vsi->config.level =
+		h264_get_level(inst, enc_prm->h264_level);
+	inst->core_vsi->config.wfd = 0;
+}
+
 static int h264_enc_set_param(void *handle,
 			      enum venc_set_param_type type,
 			      struct venc_enc_param *enc_prm)
 {
 	int ret = 0;
 	struct venc_h264_inst *inst = (struct venc_h264_inst *)handle;
+	struct mtk_vcodec_ctx *ctx = inst->ctx;
 
 	mtk_vcodec_debug(inst, "->type=%d", type);
 
 	switch (type) {
 	case VENC_SET_PARAM_ENC:
-		inst->vsi->config.input_fourcc = enc_prm->input_yuv_fmt;
-		inst->vsi->config.bitrate = enc_prm->bitrate;
-		inst->vsi->config.pic_w = enc_prm->width;
-		inst->vsi->config.pic_h = enc_prm->height;
-		inst->vsi->config.buf_w = enc_prm->buf_width;
-		inst->vsi->config.buf_h = enc_prm->buf_height;
-		inst->vsi->config.gop_size = enc_prm->gop_size;
-		inst->vsi->config.framerate = enc_prm->frm_rate;
-		inst->vsi->config.intra_period = enc_prm->intra_period;
-		inst->vsi->config.profile =
-			h264_get_profile(inst, enc_prm->h264_profile);
-		inst->vsi->config.level =
-			h264_get_level(inst, enc_prm->h264_level);
-		inst->vsi->config.wfd = 0;
+		if (IS_VENC_MULTICORE(ctx->dev->enc_capability))
+			h264_enc_set_core_configs(inst, enc_prm);
+		else
+			h264_enc_set_configs(inst, enc_prm);
+
 		ret = vpu_enc_set_param(&inst->vpu_inst, type, enc_prm);
 		if (ret)
 			break;
-- 
2.18.0


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply related	[flat|nested] 22+ messages in thread
* [PATCH v5, 5/8] media: mediatek: vcodec: Add venc power on/off function
  2022-07-29  3:51 ` Irui Wang
@ 2022-07-29  3:51   ` Irui Wang
  -1 siblings, 0 replies; 22+ messages in thread
From: Irui Wang @ 2022-07-29  3:51 UTC (permalink / raw)
  To: Hans Verkuil, angelogioacchino.delregno, Rob Herring,
	Mauro Carvalho Chehab, Matthias Brugger, Alexandre Courbot,
	Tiffany Lin, Andrew-CT Chen, Tzung-Bi Shih, Tomasz Figa,
	nicolas.dufresne
  Cc: Maoguang Meng, Longfei Wang, Yunfei Dong, Irui Wang, linux-media,
	devicetree, linux-kernel, linux-arm-kernel, linux-mediatek,
	Project_Global_Chrome_Upstream_Group

when enable multi-core encoding, all available encoder cores' power need
to be on/off, add new functions for encoder cores' power management.

Signed-off-by: Irui Wang <irui.wang@mediatek.com>
---
 .../platform/mediatek/vcodec/mtk_vcodec_enc.c | 31 +++----
 .../mediatek/vcodec/mtk_vcodec_enc_pm.c       | 83 +++++++++++++++++++
 .../mediatek/vcodec/mtk_vcodec_enc_pm.h       |  2 +
 .../platform/mediatek/vcodec/venc_drv_if.c    |  6 --
 4 files changed, 97 insertions(+), 25 deletions(-)

diff --git a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc.c b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc.c
index 25e816863597..ad20f729ef6f 100644
--- a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc.c
+++ b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc.c
@@ -8,10 +8,10 @@
 #include <media/v4l2-event.h>
 #include <media/v4l2-mem2mem.h>
 #include <media/videobuf2-dma-contig.h>
-#include <linux/pm_runtime.h>
 
 #include "mtk_vcodec_drv.h"
 #include "mtk_vcodec_enc.h"
+#include "mtk_vcodec_enc_pm.h"
 #include "mtk_vcodec_intr.h"
 #include "mtk_vcodec_util.h"
 #include "venc_drv_if.h"
@@ -877,8 +877,7 @@ static int vb2ops_venc_start_streaming(struct vb2_queue *q, unsigned int count)
 {
 	struct mtk_vcodec_ctx *ctx = vb2_get_drv_priv(q);
 	struct venc_enc_param param;
-	int ret, pm_ret;
-	int i;
+	int ret, i;
 
 	/* Once state turn into MTK_STATE_ABORT, we need stop_streaming
 	  * to clear it
@@ -897,18 +896,12 @@ static int vb2ops_venc_start_streaming(struct vb2_queue *q, unsigned int count)
 			return 0;
 	}
 
-	ret = pm_runtime_resume_and_get(&ctx->dev->plat_dev->dev);
-	if (ret < 0) {
-		mtk_v4l2_err("pm_runtime_resume_and_get fail %d", ret);
-		goto err_start_stream;
-	}
-
 	mtk_venc_set_param(ctx, &param);
 	ret = venc_if_set_param(ctx, VENC_SET_PARAM_ENC, &param);
 	if (ret) {
 		mtk_v4l2_err("venc_if_set_param failed=%d", ret);
 		ctx->state = MTK_STATE_ABORT;
-		goto err_set_param;
+		goto err_start_stream;
 	}
 	ctx->param_change = MTK_ENCODE_PARAM_NONE;
 
@@ -921,17 +914,19 @@ static int vb2ops_venc_start_streaming(struct vb2_queue *q, unsigned int count)
 		if (ret) {
 			mtk_v4l2_err("venc_if_set_param failed=%d", ret);
 			ctx->state = MTK_STATE_ABORT;
-			goto err_set_param;
+			goto err_start_stream;
 		}
 		ctx->state = MTK_STATE_HEADER;
 	}
 
-	return 0;
+	ret = mtk_vcodec_enc_power_on(ctx);
+	if (ret) {
+		mtk_v4l2_err("mtk_vcodec_enc_power_on failed=%d", ret);
+		ctx->state = MTK_STATE_ABORT;
+		goto err_start_stream;
+	}
 
-err_set_param:
-	pm_ret = pm_runtime_put(&ctx->dev->plat_dev->dev);
-	if (pm_ret < 0)
-		mtk_v4l2_err("pm_runtime_put fail %d", pm_ret);
+	return 0;
 
 err_start_stream:
 	for (i = 0; i < q->num_buffers; ++i) {
@@ -1016,9 +1011,7 @@ static void vb2ops_venc_stop_streaming(struct vb2_queue *q)
 	if (ret)
 		mtk_v4l2_err("venc_if_deinit failed=%d", ret);
 
-	ret = pm_runtime_put(&ctx->dev->plat_dev->dev);
-	if (ret < 0)
-		mtk_v4l2_err("pm_runtime_put fail %d", ret);
+	mtk_vcodec_enc_power_off(ctx);
 
 	ctx->state = MTK_STATE_FREE;
 }
diff --git a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_pm.c b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_pm.c
index 75de5031d292..213c3f50e9eb 100644
--- a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_pm.c
+++ b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_pm.c
@@ -9,6 +9,7 @@
 #include <linux/of_platform.h>
 #include <linux/pm_runtime.h>
 
+#include "mtk_vcodec_enc_hw.h"
 #include "mtk_vcodec_enc_pm.h"
 #include "mtk_vcodec_util.h"
 
@@ -56,6 +57,88 @@ int mtk_vcodec_init_enc_clk(struct platform_device *pdev,
 }
 EXPORT_SYMBOL_GPL(mtk_vcodec_init_enc_clk);
 
+static int mtk_enc_core_power_on(struct mtk_vcodec_ctx *ctx)
+{
+	struct mtk_venc_hw_dev *sub_core;
+	int ret, i;
+
+	/* multi-core encoding need power on all available cores */
+	for (i = 0; i < MTK_VENC_HW_MAX; i++) {
+		sub_core = (struct mtk_venc_hw_dev *)ctx->dev->enc_hw_dev[i];
+		if (!sub_core)
+			continue;
+
+		ret = pm_runtime_resume_and_get(&sub_core->plat_dev->dev);
+		if (ret) {
+			mtk_v4l2_err("power on sub_core[%d] fail %d", i, ret);
+			goto pm_on_fail;
+		}
+	}
+	return ret;
+
+pm_on_fail:
+	for (i -= 1; i >= 0; i--) {
+		sub_core = (struct mtk_venc_hw_dev *)ctx->dev->enc_hw_dev[i];
+		pm_runtime_put_sync(&sub_core->plat_dev->dev);
+	}
+	return ret;
+}
+
+int mtk_vcodec_enc_power_on(struct mtk_vcodec_ctx *ctx)
+{
+	struct mtk_vcodec_pm *pm = &ctx->dev->pm;
+	int ret;
+
+	ret = pm_runtime_resume_and_get(pm->dev);
+	if (ret) {
+		mtk_v4l2_err("pm_runtime_resume_and_get fail %d", ret);
+		return ret;
+	}
+
+	if (IS_VENC_MULTICORE(ctx->dev->enc_capability)) {
+		ret = mtk_enc_core_power_on(ctx);
+		if (ret) {
+			mtk_v4l2_err("mtk_enc_core_power_on fail %d", ret);
+			goto core_error;
+		}
+	}
+	return ret;
+
+core_error:
+	pm_runtime_put_sync(pm->dev);
+	return ret;
+}
+
+static void mtk_enc_core_power_off(struct mtk_vcodec_ctx *ctx)
+{
+	struct mtk_venc_hw_dev *sub_core;
+	int ret, i;
+
+	/* multi-core encoding need power off all available cores */
+	for (i = 0; i < MTK_VENC_HW_MAX; i++) {
+		sub_core = (struct mtk_venc_hw_dev *)ctx->dev->enc_hw_dev[i];
+		if (!sub_core)
+			continue;
+
+		ret = pm_runtime_put_sync(&sub_core->plat_dev->dev);
+		if (ret)
+			mtk_v4l2_err("power off sub_core[%d] fail %d", i, ret);
+	}
+}
+
+void mtk_vcodec_enc_power_off(struct mtk_vcodec_ctx *ctx)
+{
+	struct mtk_vcodec_pm *pm = &ctx->dev->pm;
+	int ret;
+
+	if (IS_VENC_MULTICORE(ctx->dev->enc_capability))
+		mtk_enc_core_power_off(ctx);
+
+	ret = pm_runtime_put_sync(pm->dev);
+	if (ret)
+		mtk_v4l2_err("pm_runtime_put_sync fail %d", ret);
+}
+
 void mtk_vcodec_enc_clock_on(struct mtk_vcodec_pm *pm)
 {
 	struct mtk_vcodec_clk *enc_clk = &pm->venc_clk;
diff --git a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_pm.h b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_pm.h
index 361dec5be47f..9065dec4ed4f 100644
--- a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_pm.h
+++ b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_pm.h
@@ -12,6 +12,8 @@
 int mtk_vcodec_init_enc_clk(struct platform_device *pdev,
 			    struct mtk_vcodec_pm *pm);
 
+int mtk_vcodec_enc_power_on(struct mtk_vcodec_ctx *ctx);
+void mtk_vcodec_enc_power_off(struct mtk_vcodec_ctx *ctx);
 void mtk_vcodec_enc_clock_on(struct mtk_vcodec_pm *pm);
 void mtk_vcodec_enc_clock_off(struct mtk_vcodec_pm *pm);
 
diff --git a/drivers/media/platform/mediatek/vcodec/venc_drv_if.c b/drivers/media/platform/mediatek/vcodec/venc_drv_if.c
index ce0bce811615..65a27e39ef5b 100644
--- a/drivers/media/platform/mediatek/vcodec/venc_drv_if.c
+++ b/drivers/media/platform/mediatek/vcodec/venc_drv_if.c
@@ -32,9 +32,7 @@ int venc_if_init(struct mtk_vcodec_ctx *ctx, unsigned int fourcc)
 	}
 
 	mtk_venc_lock(ctx);
-	mtk_vcodec_enc_clock_on(&ctx->dev->pm);
 	ret = ctx->enc_if->init(ctx);
-	mtk_vcodec_enc_clock_off(&ctx->dev->pm);
 	mtk_venc_unlock(ctx);
 
 	return ret;
@@ -46,9 +44,7 @@ int venc_if_set_param(struct mtk_vcodec_ctx *ctx,
 	int ret = 0;
 
 	mtk_venc_lock(ctx);
-	mtk_vcodec_enc_clock_on(&ctx->dev->pm);
 	ret = ctx->enc_if->set_param(ctx->drv_handle, type, in);
-	mtk_vcodec_enc_clock_off(&ctx->dev->pm);
 	mtk_venc_unlock(ctx);
 
 	return ret;
@@ -89,9 +85,7 @@ int venc_if_deinit(struct mtk_vcodec_ctx *ctx)
 		return 0;
 
 	mtk_venc_lock(ctx);
-	mtk_vcodec_enc_clock_on(&ctx->dev->pm);
 	ret = ctx->enc_if->deinit(ctx->drv_handle);
-	mtk_vcodec_enc_clock_off(&ctx->dev->pm);
 	mtk_venc_unlock(ctx);
 
 	ctx->drv_handle = NULL;
-- 
2.18.0


^ permalink raw reply related	[flat|nested] 22+ messages in thread
* [PATCH v5, 5/8] media: mediatek: vcodec: Add venc power on/off function
@ 2022-07-29  3:51   ` Irui Wang
  0 siblings, 0 replies; 22+ messages in thread
From: Irui Wang @ 2022-07-29  3:51 UTC (permalink / raw)
  To: Hans Verkuil, angelogioacchino.delregno, Rob Herring,
	Mauro Carvalho Chehab, Matthias Brugger, Alexandre Courbot,
	Tiffany Lin, Andrew-CT Chen, Tzung-Bi Shih, Tomasz Figa,
	nicolas.dufresne
  Cc: Maoguang Meng, Longfei Wang, Yunfei Dong, Irui Wang, linux-media,
	devicetree, linux-kernel, linux-arm-kernel, linux-mediatek,
	Project_Global_Chrome_Upstream_Group

when enable multi-core encoding, all available encoder cores' power need
to be on/off, add new functions for encoder cores' power management.

Signed-off-by: Irui Wang <irui.wang@mediatek.com>
---
 .../platform/mediatek/vcodec/mtk_vcodec_enc.c | 31 +++----
 .../mediatek/vcodec/mtk_vcodec_enc_pm.c       | 83 +++++++++++++++++++
 .../mediatek/vcodec/mtk_vcodec_enc_pm.h       |  2 +
 .../platform/mediatek/vcodec/venc_drv_if.c    |  6 --
 4 files changed, 97 insertions(+), 25 deletions(-)

diff --git a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc.c b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc.c
index 25e816863597..ad20f729ef6f 100644
--- a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc.c
+++ b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc.c
@@ -8,10 +8,10 @@
 #include <media/v4l2-event.h>
 #include <media/v4l2-mem2mem.h>
 #include <media/videobuf2-dma-contig.h>
-#include <linux/pm_runtime.h>
 
 #include "mtk_vcodec_drv.h"
 #include "mtk_vcodec_enc.h"
+#include "mtk_vcodec_enc_pm.h"
 #include "mtk_vcodec_intr.h"
 #include "mtk_vcodec_util.h"
 #include "venc_drv_if.h"
@@ -877,8 +877,7 @@ static int vb2ops_venc_start_streaming(struct vb2_queue *q, unsigned int count)
 {
 	struct mtk_vcodec_ctx *ctx = vb2_get_drv_priv(q);
 	struct venc_enc_param param;
-	int ret, pm_ret;
-	int i;
+	int ret, i;
 
 	/* Once state turn into MTK_STATE_ABORT, we need stop_streaming
 	  * to clear it
@@ -897,18 +896,12 @@ static int vb2ops_venc_start_streaming(struct vb2_queue *q, unsigned int count)
 			return 0;
 	}
 
-	ret = pm_runtime_resume_and_get(&ctx->dev->plat_dev->dev);
-	if (ret < 0) {
-		mtk_v4l2_err("pm_runtime_resume_and_get fail %d", ret);
-		goto err_start_stream;
-	}
-
 	mtk_venc_set_param(ctx, &param);
 	ret = venc_if_set_param(ctx, VENC_SET_PARAM_ENC, &param);
 	if (ret) {
 		mtk_v4l2_err("venc_if_set_param failed=%d", ret);
 		ctx->state = MTK_STATE_ABORT;
-		goto err_set_param;
+		goto err_start_stream;
 	}
 	ctx->param_change = MTK_ENCODE_PARAM_NONE;
 
@@ -921,17 +914,19 @@ static int vb2ops_venc_start_streaming(struct vb2_queue *q, unsigned int count)
 		if (ret) {
 			mtk_v4l2_err("venc_if_set_param failed=%d", ret);
 			ctx->state = MTK_STATE_ABORT;
-			goto err_set_param;
+			goto err_start_stream;
 		}
 		ctx->state = MTK_STATE_HEADER;
 	}
 
-	return 0;
+	ret = mtk_vcodec_enc_power_on(ctx);
+	if (ret) {
+		mtk_v4l2_err("mtk_vcodec_enc_power_on failed=%d", ret);
+		ctx->state = MTK_STATE_ABORT;
+		goto err_start_stream;
+	}
 
-err_set_param:
-	pm_ret = pm_runtime_put(&ctx->dev->plat_dev->dev);
-	if (pm_ret < 0)
-		mtk_v4l2_err("pm_runtime_put fail %d", pm_ret);
+	return 0;
 
 err_start_stream:
 	for (i = 0; i < q->num_buffers; ++i) {
@@ -1016,9 +1011,7 @@ static void vb2ops_venc_stop_streaming(struct vb2_queue *q)
 	if (ret)
 		mtk_v4l2_err("venc_if_deinit failed=%d", ret);
 
-	ret = pm_runtime_put(&ctx->dev->plat_dev->dev);
-	if (ret < 0)
-		mtk_v4l2_err("pm_runtime_put fail %d", ret);
+	mtk_vcodec_enc_power_off(ctx);
 
 	ctx->state = MTK_STATE_FREE;
 }
diff --git a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_pm.c b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_pm.c
index 75de5031d292..213c3f50e9eb 100644
--- a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_pm.c
+++ b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_pm.c
@@ -9,6 +9,7 @@
 #include <linux/of_platform.h>
 #include <linux/pm_runtime.h>
 
+#include "mtk_vcodec_enc_hw.h"
 #include "mtk_vcodec_enc_pm.h"
 #include "mtk_vcodec_util.h"
 
@@ -56,6 +57,88 @@ int mtk_vcodec_init_enc_clk(struct platform_device *pdev,
 }
 EXPORT_SYMBOL_GPL(mtk_vcodec_init_enc_clk);
 
+static int mtk_enc_core_power_on(struct mtk_vcodec_ctx *ctx)
+{
+	struct mtk_venc_hw_dev *sub_core;
+	int ret, i;
+
+	/* multi-core encoding need power on all available cores */
+	for (i = 0; i < MTK_VENC_HW_MAX; i++) {
+		sub_core = (struct mtk_venc_hw_dev *)ctx->dev->enc_hw_dev[i];
+		if (!sub_core)
+			continue;
+
+		ret = pm_runtime_resume_and_get(&sub_core->plat_dev->dev);
+		if (ret) {
+			mtk_v4l2_err("power on sub_core[%d] fail %d", i, ret);
+			goto pm_on_fail;
+		}
+	}
+	return ret;
+
+pm_on_fail:
+	for (i -= 1; i >= 0; i--) {
+		sub_core = (struct mtk_venc_hw_dev *)ctx->dev->enc_hw_dev[i];
+		pm_runtime_put_sync(&sub_core->plat_dev->dev);
+	}
+	return ret;
+}
+
+int mtk_vcodec_enc_power_on(struct mtk_vcodec_ctx *ctx)
+{
+	struct mtk_vcodec_pm *pm = &ctx->dev->pm;
+	int ret;
+
+	ret = pm_runtime_resume_and_get(pm->dev);
+	if (ret) {
+		mtk_v4l2_err("pm_runtime_resume_and_get fail %d", ret);
+		return ret;
+	}
+
+	if (IS_VENC_MULTICORE(ctx->dev->enc_capability)) {
+		ret = mtk_enc_core_power_on(ctx);
+		if (ret) {
+			mtk_v4l2_err("mtk_enc_core_power_on fail %d", ret);
+			goto core_error;
+		}
+	}
+	return ret;
+
+core_error:
+	pm_runtime_put_sync(pm->dev);
+	return ret;
+}
+
+static void mtk_enc_core_power_off(struct mtk_vcodec_ctx *ctx)
+{
+	struct mtk_venc_hw_dev *sub_core;
+	int ret, i;
+
+	/* multi-core encoding need power off all available cores */
+	for (i = 0; i < MTK_VENC_HW_MAX; i++) {
+		sub_core = (struct mtk_venc_hw_dev *)ctx->dev->enc_hw_dev[i];
+		if (!sub_core)
+			continue;
+
+		ret = pm_runtime_put_sync(&sub_core->plat_dev->dev);
+		if (ret)
+			mtk_v4l2_err("power off sub_core[%d] fail %d", i, ret);
+	}
+}
+
+void mtk_vcodec_enc_power_off(struct mtk_vcodec_ctx *ctx)
+{
+	struct mtk_vcodec_pm *pm = &ctx->dev->pm;
+	int ret;
+
+	if (IS_VENC_MULTICORE(ctx->dev->enc_capability))
+		mtk_enc_core_power_off(ctx);
+
+	ret = pm_runtime_put_sync(pm->dev);
+	if (ret)
+		mtk_v4l2_err("pm_runtime_put_sync fail %d", ret);
+}
+
 void mtk_vcodec_enc_clock_on(struct mtk_vcodec_pm *pm)
 {
 	struct mtk_vcodec_clk *enc_clk = &pm->venc_clk;
diff --git a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_pm.h b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_pm.h
index 361dec5be47f..9065dec4ed4f 100644
--- a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_pm.h
+++ b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_pm.h
@@ -12,6 +12,8 @@
 int mtk_vcodec_init_enc_clk(struct platform_device *pdev,
 			    struct mtk_vcodec_pm *pm);
 
+int mtk_vcodec_enc_power_on(struct mtk_vcodec_ctx *ctx);
+void mtk_vcodec_enc_power_off(struct mtk_vcodec_ctx *ctx);
 void mtk_vcodec_enc_clock_on(struct mtk_vcodec_pm *pm);
 void mtk_vcodec_enc_clock_off(struct mtk_vcodec_pm *pm);
 
diff --git a/drivers/media/platform/mediatek/vcodec/venc_drv_if.c b/drivers/media/platform/mediatek/vcodec/venc_drv_if.c
index ce0bce811615..65a27e39ef5b 100644
--- a/drivers/media/platform/mediatek/vcodec/venc_drv_if.c
+++ b/drivers/media/platform/mediatek/vcodec/venc_drv_if.c
@@ -32,9 +32,7 @@ int venc_if_init(struct mtk_vcodec_ctx *ctx, unsigned int fourcc)
 	}
 
 	mtk_venc_lock(ctx);
-	mtk_vcodec_enc_clock_on(&ctx->dev->pm);
 	ret = ctx->enc_if->init(ctx);
-	mtk_vcodec_enc_clock_off(&ctx->dev->pm);
 	mtk_venc_unlock(ctx);
 
 	return ret;
@@ -46,9 +44,7 @@ int venc_if_set_param(struct mtk_vcodec_ctx *ctx,
 	int ret = 0;
 
 	mtk_venc_lock(ctx);
-	mtk_vcodec_enc_clock_on(&ctx->dev->pm);
 	ret = ctx->enc_if->set_param(ctx->drv_handle, type, in);
-	mtk_vcodec_enc_clock_off(&ctx->dev->pm);
 	mtk_venc_unlock(ctx);
 
 	return ret;
@@ -89,9 +85,7 @@ int venc_if_deinit(struct mtk_vcodec_ctx *ctx)
 		return 0;
 
 	mtk_venc_lock(ctx);
-	mtk_vcodec_enc_clock_on(&ctx->dev->pm);
 	ret = ctx->enc_if->deinit(ctx->drv_handle);
-	mtk_vcodec_enc_clock_off(&ctx->dev->pm);
 	mtk_venc_unlock(ctx);
 
 	ctx->drv_handle = NULL;
-- 
2.18.0


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply related	[flat|nested] 22+ messages in thread
* [PATCH v5, 6/8] media: mediatek: vcodec: Refactor encoder clock on/off function
  2022-07-29  3:51 ` Irui Wang
@ 2022-07-29  3:51   ` Irui Wang
  -1 siblings, 0 replies; 22+ messages in thread
From: Irui Wang @ 2022-07-29  3:51 UTC (permalink / raw)
  To: Hans Verkuil, angelogioacchino.delregno, Rob Herring,
	Mauro Carvalho Chehab, Matthias Brugger, Alexandre Courbot,
	Tiffany Lin, Andrew-CT Chen, Tzung-Bi Shih, Tomasz Figa,
	nicolas.dufresne
  Cc: Maoguang Meng, Longfei Wang, Yunfei Dong, Irui Wang, linux-media,
	devicetree, linux-kernel, linux-arm-kernel, linux-mediatek,
	Project_Global_Chrome_Upstream_Group

when enable multi-core encoding, encoder cores use their own clock,
refactor clock management functions with used encoder hardware id.

Signed-off-by: Irui Wang <irui.wang@mediatek.com>
---
 .../mediatek/vcodec/mtk_vcodec_enc_pm.c       | 89 ++++++++++++++++---
 .../mediatek/vcodec/mtk_vcodec_enc_pm.h       |  6 +-
 .../platform/mediatek/vcodec/venc_drv_if.c    |  4 +-
 3 files changed, 84 insertions(+), 15 deletions(-)

diff --git a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_pm.c b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_pm.c
index 213c3f50e9eb..2f83aade779a 100644
--- a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_pm.c
+++ b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_pm.c
@@ -60,7 +60,9 @@ EXPORT_SYMBOL_GPL(mtk_vcodec_init_enc_clk);
 static int mtk_enc_core_power_on(struct mtk_vcodec_ctx *ctx)
 {
 	struct mtk_venc_hw_dev *sub_core;
+	struct mtk_vcodec_clk *clk;
 	int ret, i;
+	int j = 0;
 
 	/* multi-core encoding need power on all available cores */
 	for (i = 0; i < MTK_VENC_HW_MAX; i++) {
@@ -73,12 +75,27 @@ static int mtk_enc_core_power_on(struct mtk_vcodec_ctx *ctx)
 			mtk_v4l2_err("power on sub_core[%d] fail %d", i, ret);
 			goto pm_on_fail;
 		}
+
+		clk = &sub_core->pm.venc_clk;
+		for (j = 0; j < clk->clk_num; j++) {
+			ret = clk_prepare(clk->clk_info[j].vcodec_clk);
+			if (ret) {
+				mtk_v4l2_err("prepare clk [%s] fail %d",
+					     clk->clk_info[j].clk_name, ret);
+				goto pm_on_fail;
+			}
+		}
 	}
 	return ret;
 
 pm_on_fail:
 	for (i -= 1; i >= 0; i--) {
 		sub_core = (struct mtk_venc_hw_dev *)ctx->dev->enc_hw_dev[i];
+
+		clk = &sub_core->pm.venc_clk;
+		for (j -= 1; j >= 0; j--)
+			clk_unprepare(clk->clk_info[j].vcodec_clk);
+
 		pm_runtime_put_sync(&sub_core->plat_dev->dev);
 	}
 	return ret;
@@ -87,7 +104,9 @@ static int mtk_enc_core_power_on(struct mtk_vcodec_ctx *ctx)
 int mtk_vcodec_enc_power_on(struct mtk_vcodec_ctx *ctx)
 {
 	struct mtk_vcodec_pm *pm = &ctx->dev->pm;
+	struct mtk_vcodec_clk *clk;
 	int ret;
+	int i = 0;
 
 	ret = pm_runtime_resume_and_get(pm->dev);
 	if (ret) {
@@ -95,6 +114,16 @@ int mtk_vcodec_enc_power_on(struct mtk_vcodec_ctx *ctx)
 		return ret;
 	}
 
+	clk = &pm->venc_clk;
+	for (i = 0; i < clk->clk_num; i++) {
+		ret = clk_prepare(clk->clk_info[i].vcodec_clk);
+		if (ret) {
+			mtk_v4l2_err("prepare clk [%s] fail %d",
+				     clk->clk_info[i].clk_name, ret);
+			goto clk_error;
+		}
+	}
+
 	if (IS_VENC_MULTICORE(ctx->dev->enc_capability)) {
 		ret = mtk_enc_core_power_on(ctx);
 		if (ret) {
@@ -104,6 +133,9 @@ int mtk_vcodec_enc_power_on(struct mtk_vcodec_ctx *ctx)
 	}
 	return ret;
 
+clk_error:
+	for (i -= 1; i >= 0; i--)
+		clk_unprepare(clk->clk_info[i].vcodec_clk);
 core_error:
 	pm_runtime_put_sync(pm->dev);
 	return ret;
@@ -112,7 +144,8 @@ int mtk_vcodec_enc_power_on(struct mtk_vcodec_ctx *ctx)
 static void mtk_enc_core_power_off(struct mtk_vcodec_ctx *ctx)
 {
 	struct mtk_venc_hw_dev *sub_core;
-	int ret, i;
+	struct mtk_vcodec_clk *clk;
+	int ret, i, j;
 
 	/* multi-core encoding need power off all available cores */
 	for (i = 0; i < MTK_VENC_HW_MAX; i++) {
@@ -120,6 +153,10 @@ static void mtk_enc_core_power_off(struct mtk_vcodec_ctx *ctx)
 		if (!sub_core)
 			continue;
 
+		clk = &sub_core->pm.venc_clk;
+		for (j = clk->clk_num - 1; j >= 0; j--)
+			clk_unprepare(clk->clk_info[j].vcodec_clk);
+
 		ret = pm_runtime_put_sync(&sub_core->plat_dev->dev);
 		if (ret)
 			mtk_v4l2_err("power off sub_core[%d] fail %d", i, ret);
@@ -129,26 +166,44 @@ static void mtk_enc_core_power_off(struct mtk_vcodec_ctx *ctx)
 void mtk_vcodec_enc_power_off(struct mtk_vcodec_ctx *ctx)
 {
 	struct mtk_vcodec_pm *pm = &ctx->dev->pm;
-	int ret;
+	struct mtk_vcodec_clk *clk;
+	int ret, i;
 
 	if (IS_VENC_MULTICORE(ctx->dev->enc_capability))
 		mtk_enc_core_power_off(ctx);
 
+	clk = &pm->venc_clk;
+	for (i = clk->clk_num - 1; i >= 0; i--)
+		clk_unprepare(clk->clk_info[i].vcodec_clk);
+
 	ret = pm_runtime_put_sync(pm->dev);
 	if (ret)
 		mtk_v4l2_err("pm_runtime_put_sync fail %d", ret);
 }
 
-void mtk_vcodec_enc_clock_on(struct mtk_vcodec_pm *pm)
+void mtk_vcodec_enc_clock_on(struct mtk_vcodec_dev *dev,
+			     enum mtk_venc_hw_id hw_id)
 {
-	struct mtk_vcodec_clk *enc_clk = &pm->venc_clk;
+	struct mtk_venc_hw_dev *sub_core;
+	struct mtk_vcodec_clk *enc_clk;
+
 	int ret, i = 0;
 
+	if (hw_id == MTK_VENC_CORE_0) {
+		enc_clk = &dev->pm.venc_clk;
+	} else if (hw_id == MTK_VENC_CORE_1) {
+		sub_core = (struct mtk_venc_hw_dev *)dev->enc_hw_dev[hw_id];
+		enc_clk = &sub_core->pm.venc_clk;
+	} else {
+		mtk_v4l2_err("invalid hw id : %d", hw_id);
+		return;
+	}
+
 	for (i = 0; i < enc_clk->clk_num; i++) {
-		ret = clk_prepare_enable(enc_clk->clk_info[i].vcodec_clk);
+		ret = clk_enable(enc_clk->clk_info[i].vcodec_clk);
 		if (ret) {
-			mtk_v4l2_err("venc clk_prepare_enable %d %s fail %d", i,
-				enc_clk->clk_info[i].clk_name, ret);
+			mtk_v4l2_err("venc clk_enable %d %s fail %d", i,
+				     enc_clk->clk_info[i].clk_name, ret);
 			goto clkerr;
 		}
 	}
@@ -157,14 +212,26 @@ void mtk_vcodec_enc_clock_on(struct mtk_vcodec_pm *pm)
 
 clkerr:
 	for (i -= 1; i >= 0; i--)
-		clk_disable_unprepare(enc_clk->clk_info[i].vcodec_clk);
+		clk_disable(enc_clk->clk_info[i].vcodec_clk);
 }
 
-void mtk_vcodec_enc_clock_off(struct mtk_vcodec_pm *pm)
+void mtk_vcodec_enc_clock_off(struct mtk_vcodec_dev *dev,
+			      enum mtk_venc_hw_id hw_id)
 {
-	struct mtk_vcodec_clk *enc_clk = &pm->venc_clk;
+	struct mtk_venc_hw_dev *sub_core;
+	struct mtk_vcodec_clk *enc_clk;
 	int i = 0;
 
+	if (hw_id == MTK_VENC_CORE_0) {
+		enc_clk = &dev->pm.venc_clk;
+	} else if (hw_id == MTK_VENC_CORE_1) {
+		sub_core = (struct mtk_venc_hw_dev *)dev->enc_hw_dev[hw_id];
+		enc_clk = &sub_core->pm.venc_clk;
+	} else {
+		mtk_v4l2_err("invalid hw id : %d", hw_id);
+		return;
+	}
+
 	for (i = enc_clk->clk_num - 1; i >= 0; i--)
-		clk_disable_unprepare(enc_clk->clk_info[i].vcodec_clk);
+		clk_disable(enc_clk->clk_info[i].vcodec_clk);
 }
diff --git a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_pm.h b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_pm.h
index 9065dec4ed4f..a2906d2971ee 100644
--- a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_pm.h
+++ b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_pm.h
@@ -14,7 +14,9 @@ int mtk_vcodec_init_enc_clk(struct platform_device *pdev,
 
 int mtk_vcodec_enc_power_on(struct mtk_vcodec_ctx *ctx);
 void mtk_vcodec_enc_power_off(struct mtk_vcodec_ctx *ctx);
-void mtk_vcodec_enc_clock_on(struct mtk_vcodec_pm *pm);
-void mtk_vcodec_enc_clock_off(struct mtk_vcodec_pm *pm);
+void mtk_vcodec_enc_clock_on(struct mtk_vcodec_dev *dev,
+			     enum mtk_venc_hw_id hw_id);
+void mtk_vcodec_enc_clock_off(struct mtk_vcodec_dev *dev,
+			      enum mtk_venc_hw_id hw_id);
 
 #endif /* _MTK_VCODEC_ENC_PM_H_ */
diff --git a/drivers/media/platform/mediatek/vcodec/venc_drv_if.c b/drivers/media/platform/mediatek/vcodec/venc_drv_if.c
index 65a27e39ef5b..6cbdb7e30bb3 100644
--- a/drivers/media/platform/mediatek/vcodec/venc_drv_if.c
+++ b/drivers/media/platform/mediatek/vcodec/venc_drv_if.c
@@ -64,10 +64,10 @@ int venc_if_encode(struct mtk_vcodec_ctx *ctx,
 	ctx->dev->curr_ctx = ctx;
 	spin_unlock_irqrestore(&ctx->dev->irqlock, flags);
 
-	mtk_vcodec_enc_clock_on(&ctx->dev->pm);
+	mtk_vcodec_enc_clock_on(ctx->dev, 0);
 	ret = ctx->enc_if->encode(ctx->drv_handle, opt, frm_buf,
 				  bs_buf, result);
-	mtk_vcodec_enc_clock_off(&ctx->dev->pm);
+	mtk_vcodec_enc_clock_off(ctx->dev, 0);
 
 	spin_lock_irqsave(&ctx->dev->irqlock, flags);
 	ctx->dev->curr_ctx = NULL;
-- 
2.18.0


^ permalink raw reply related	[flat|nested] 22+ messages in thread
* [PATCH v5, 6/8] media: mediatek: vcodec: Refactor encoder clock on/off function
@ 2022-07-29  3:51   ` Irui Wang
  0 siblings, 0 replies; 22+ messages in thread
From: Irui Wang @ 2022-07-29  3:51 UTC (permalink / raw)
  To: Hans Verkuil, angelogioacchino.delregno, Rob Herring,
	Mauro Carvalho Chehab, Matthias Brugger, Alexandre Courbot,
	Tiffany Lin, Andrew-CT Chen, Tzung-Bi Shih, Tomasz Figa,
	nicolas.dufresne
  Cc: Maoguang Meng, Longfei Wang, Yunfei Dong, Irui Wang, linux-media,
	devicetree, linux-kernel, linux-arm-kernel, linux-mediatek,
	Project_Global_Chrome_Upstream_Group

when enable multi-core encoding, encoder cores use their own clock,
refactor clock management functions with used encoder hardware id.

Signed-off-by: Irui Wang <irui.wang@mediatek.com>
---
 .../mediatek/vcodec/mtk_vcodec_enc_pm.c       | 89 ++++++++++++++++---
 .../mediatek/vcodec/mtk_vcodec_enc_pm.h       |  6 +-
 .../platform/mediatek/vcodec/venc_drv_if.c    |  4 +-
 3 files changed, 84 insertions(+), 15 deletions(-)

diff --git a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_pm.c b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_pm.c
index 213c3f50e9eb..2f83aade779a 100644
--- a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_pm.c
+++ b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_pm.c
@@ -60,7 +60,9 @@ EXPORT_SYMBOL_GPL(mtk_vcodec_init_enc_clk);
 static int mtk_enc_core_power_on(struct mtk_vcodec_ctx *ctx)
 {
 	struct mtk_venc_hw_dev *sub_core;
+	struct mtk_vcodec_clk *clk;
 	int ret, i;
+	int j = 0;
 
 	/* multi-core encoding need power on all available cores */
 	for (i = 0; i < MTK_VENC_HW_MAX; i++) {
@@ -73,12 +75,27 @@ static int mtk_enc_core_power_on(struct mtk_vcodec_ctx *ctx)
 			mtk_v4l2_err("power on sub_core[%d] fail %d", i, ret);
 			goto pm_on_fail;
 		}
+
+		clk = &sub_core->pm.venc_clk;
+		for (j = 0; j < clk->clk_num; j++) {
+			ret = clk_prepare(clk->clk_info[j].vcodec_clk);
+			if (ret) {
+				mtk_v4l2_err("prepare clk [%s] fail %d",
+					     clk->clk_info[j].clk_name, ret);
+				goto pm_on_fail;
+			}
+		}
 	}
 	return ret;
 
 pm_on_fail:
 	for (i -= 1; i >= 0; i--) {
 		sub_core = (struct mtk_venc_hw_dev *)ctx->dev->enc_hw_dev[i];
+
+		clk = &sub_core->pm.venc_clk;
+		for (j -= 1; j >= 0; j--)
+			clk_unprepare(clk->clk_info[j].vcodec_clk);
+
 		pm_runtime_put_sync(&sub_core->plat_dev->dev);
 	}
 	return ret;
@@ -87,7 +104,9 @@ static int mtk_enc_core_power_on(struct mtk_vcodec_ctx *ctx)
 int mtk_vcodec_enc_power_on(struct mtk_vcodec_ctx *ctx)
 {
 	struct mtk_vcodec_pm *pm = &ctx->dev->pm;
+	struct mtk_vcodec_clk *clk;
 	int ret;
+	int i = 0;
 
 	ret = pm_runtime_resume_and_get(pm->dev);
 	if (ret) {
@@ -95,6 +114,16 @@ int mtk_vcodec_enc_power_on(struct mtk_vcodec_ctx *ctx)
 		return ret;
 	}
 
+	clk = &pm->venc_clk;
+	for (i = 0; i < clk->clk_num; i++) {
+		ret = clk_prepare(clk->clk_info[i].vcodec_clk);
+		if (ret) {
+			mtk_v4l2_err("prepare clk [%s] fail %d",
+				     clk->clk_info[i].clk_name, ret);
+			goto clk_error;
+		}
+	}
+
 	if (IS_VENC_MULTICORE(ctx->dev->enc_capability)) {
 		ret = mtk_enc_core_power_on(ctx);
 		if (ret) {
@@ -104,6 +133,9 @@ int mtk_vcodec_enc_power_on(struct mtk_vcodec_ctx *ctx)
 	}
 	return ret;
 
+clk_error:
+	for (i -= 1; i >= 0; i--)
+		clk_unprepare(clk->clk_info[i].vcodec_clk);
 core_error:
 	pm_runtime_put_sync(pm->dev);
 	return ret;
@@ -112,7 +144,8 @@ int mtk_vcodec_enc_power_on(struct mtk_vcodec_ctx *ctx)
 static void mtk_enc_core_power_off(struct mtk_vcodec_ctx *ctx)
 {
 	struct mtk_venc_hw_dev *sub_core;
-	int ret, i;
+	struct mtk_vcodec_clk *clk;
+	int ret, i, j;
 
 	/* multi-core encoding need power off all available cores */
 	for (i = 0; i < MTK_VENC_HW_MAX; i++) {
@@ -120,6 +153,10 @@ static void mtk_enc_core_power_off(struct mtk_vcodec_ctx *ctx)
 		if (!sub_core)
 			continue;
 
+		clk = &sub_core->pm.venc_clk;
+		for (j = clk->clk_num - 1; j >= 0; j--)
+			clk_unprepare(clk->clk_info[j].vcodec_clk);
+
 		ret = pm_runtime_put_sync(&sub_core->plat_dev->dev);
 		if (ret)
 			mtk_v4l2_err("power off sub_core[%d] fail %d", i, ret);
@@ -129,26 +166,44 @@ static void mtk_enc_core_power_off(struct mtk_vcodec_ctx *ctx)
 void mtk_vcodec_enc_power_off(struct mtk_vcodec_ctx *ctx)
 {
 	struct mtk_vcodec_pm *pm = &ctx->dev->pm;
-	int ret;
+	struct mtk_vcodec_clk *clk;
+	int ret, i;
 
 	if (IS_VENC_MULTICORE(ctx->dev->enc_capability))
 		mtk_enc_core_power_off(ctx);
 
+	clk = &pm->venc_clk;
+	for (i = clk->clk_num - 1; i >= 0; i--)
+		clk_unprepare(clk->clk_info[i].vcodec_clk);
+
 	ret = pm_runtime_put_sync(pm->dev);
 	if (ret)
 		mtk_v4l2_err("pm_runtime_put_sync fail %d", ret);
 }
 
-void mtk_vcodec_enc_clock_on(struct mtk_vcodec_pm *pm)
+void mtk_vcodec_enc_clock_on(struct mtk_vcodec_dev *dev,
+			     enum mtk_venc_hw_id hw_id)
 {
-	struct mtk_vcodec_clk *enc_clk = &pm->venc_clk;
+	struct mtk_venc_hw_dev *sub_core;
+	struct mtk_vcodec_clk *enc_clk;
+
 	int ret, i = 0;
 
+	if (hw_id == MTK_VENC_CORE_0) {
+		enc_clk = &dev->pm.venc_clk;
+	} else if (hw_id == MTK_VENC_CORE_1) {
+		sub_core = (struct mtk_venc_hw_dev *)dev->enc_hw_dev[hw_id];
+		enc_clk = &sub_core->pm.venc_clk;
+	} else {
+		mtk_v4l2_err("invalid hw id : %d", hw_id);
+		return;
+	}
+
 	for (i = 0; i < enc_clk->clk_num; i++) {
-		ret = clk_prepare_enable(enc_clk->clk_info[i].vcodec_clk);
+		ret = clk_enable(enc_clk->clk_info[i].vcodec_clk);
 		if (ret) {
-			mtk_v4l2_err("venc clk_prepare_enable %d %s fail %d", i,
-				enc_clk->clk_info[i].clk_name, ret);
+			mtk_v4l2_err("venc clk_enable %d %s fail %d", i,
+				     enc_clk->clk_info[i].clk_name, ret);
 			goto clkerr;
 		}
 	}
@@ -157,14 +212,26 @@ void mtk_vcodec_enc_clock_on(struct mtk_vcodec_pm *pm)
 
 clkerr:
 	for (i -= 1; i >= 0; i--)
-		clk_disable_unprepare(enc_clk->clk_info[i].vcodec_clk);
+		clk_disable(enc_clk->clk_info[i].vcodec_clk);
 }
 
-void mtk_vcodec_enc_clock_off(struct mtk_vcodec_pm *pm)
+void mtk_vcodec_enc_clock_off(struct mtk_vcodec_dev *dev,
+			      enum mtk_venc_hw_id hw_id)
 {
-	struct mtk_vcodec_clk *enc_clk = &pm->venc_clk;
+	struct mtk_venc_hw_dev *sub_core;
+	struct mtk_vcodec_clk *enc_clk;
 	int i = 0;
 
+	if (hw_id == MTK_VENC_CORE_0) {
+		enc_clk = &dev->pm.venc_clk;
+	} else if (hw_id == MTK_VENC_CORE_1) {
+		sub_core = (struct mtk_venc_hw_dev *)dev->enc_hw_dev[hw_id];
+		enc_clk = &sub_core->pm.venc_clk;
+	} else {
+		mtk_v4l2_err("invalid hw id : %d", hw_id);
+		return;
+	}
+
 	for (i = enc_clk->clk_num - 1; i >= 0; i--)
-		clk_disable_unprepare(enc_clk->clk_info[i].vcodec_clk);
+		clk_disable(enc_clk->clk_info[i].vcodec_clk);
 }
diff --git a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_pm.h b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_pm.h
index 9065dec4ed4f..a2906d2971ee 100644
--- a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_pm.h
+++ b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_pm.h
@@ -14,7 +14,9 @@ int mtk_vcodec_init_enc_clk(struct platform_device *pdev,
 
 int mtk_vcodec_enc_power_on(struct mtk_vcodec_ctx *ctx);
 void mtk_vcodec_enc_power_off(struct mtk_vcodec_ctx *ctx);
-void mtk_vcodec_enc_clock_on(struct mtk_vcodec_pm *pm);
-void mtk_vcodec_enc_clock_off(struct mtk_vcodec_pm *pm);
+void mtk_vcodec_enc_clock_on(struct mtk_vcodec_dev *dev,
+			     enum mtk_venc_hw_id hw_id);
+void mtk_vcodec_enc_clock_off(struct mtk_vcodec_dev *dev,
+			      enum mtk_venc_hw_id hw_id);
 
 #endif /* _MTK_VCODEC_ENC_PM_H_ */
diff --git a/drivers/media/platform/mediatek/vcodec/venc_drv_if.c b/drivers/media/platform/mediatek/vcodec/venc_drv_if.c
index 65a27e39ef5b..6cbdb7e30bb3 100644
--- a/drivers/media/platform/mediatek/vcodec/venc_drv_if.c
+++ b/drivers/media/platform/mediatek/vcodec/venc_drv_if.c
@@ -64,10 +64,10 @@ int venc_if_encode(struct mtk_vcodec_ctx *ctx,
 	ctx->dev->curr_ctx = ctx;
 	spin_unlock_irqrestore(&ctx->dev->irqlock, flags);
 
-	mtk_vcodec_enc_clock_on(&ctx->dev->pm);
+	mtk_vcodec_enc_clock_on(ctx->dev, 0);
 	ret = ctx->enc_if->encode(ctx->drv_handle, opt, frm_buf,
 				  bs_buf, result);
-	mtk_vcodec_enc_clock_off(&ctx->dev->pm);
+	mtk_vcodec_enc_clock_off(ctx->dev, 0);
 
 	spin_lock_irqsave(&ctx->dev->irqlock, flags);
 	ctx->dev->curr_ctx = NULL;
-- 
2.18.0


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply related	[flat|nested] 22+ messages in thread
* [PATCH v5, 7/8] media: mediatek: vcodec: Add multi-core encoding process
  2022-07-29  3:51 ` Irui Wang
@ 2022-07-29  3:51   ` Irui Wang
  -1 siblings, 0 replies; 22+ messages in thread
From: Irui Wang @ 2022-07-29  3:51 UTC (permalink / raw)
  To: Hans Verkuil, angelogioacchino.delregno, Rob Herring,
	Mauro Carvalho Chehab, Matthias Brugger, Alexandre Courbot,
	Tiffany Lin, Andrew-CT Chen, Tzung-Bi Shih, Tomasz Figa,
	nicolas.dufresne
  Cc: Maoguang Meng, Longfei Wang, Yunfei Dong, Irui Wang, linux-media,
	devicetree, linux-kernel, linux-arm-kernel, linux-mediatek,
	Project_Global_Chrome_Upstream_Group

MT8195 has two encoder hardware cores, when enable multi-core encoding,
it is possiable to trying to use the two cores. According to hardware
requirements, input frame 0 uses core0, frame 1 uses core1, then frame 2
uses core0, lock the device and enable clock by used core, and for
sequence header encoding, it always use core0.

Signed-off-by: Irui Wang <irui.wang@mediatek.com>
---
 .../platform/mediatek/vcodec/mtk_vcodec_drv.h |  8 ++-
 .../platform/mediatek/vcodec/mtk_vcodec_enc.c | 10 +--
 .../platform/mediatek/vcodec/mtk_vcodec_enc.h |  4 +-
 .../mediatek/vcodec/mtk_vcodec_enc_drv.c      |  8 ++-
 .../mediatek/vcodec/mtk_vcodec_enc_hw.c       |  7 +-
 .../mediatek/vcodec/mtk_vcodec_enc_hw.h       |  2 -
 .../mediatek/vcodec/venc/venc_h264_if.c       | 16 +++--
 .../mediatek/vcodec/venc/venc_vp8_if.c        |  3 +-
 .../platform/mediatek/vcodec/venc_drv_if.c    | 69 ++++++++++++++-----
 .../platform/mediatek/vcodec/venc_drv_if.h    |  4 ++
 .../platform/mediatek/vcodec/venc_vpu_if.c    |  9 ++-
 .../platform/mediatek/vcodec/venc_vpu_if.h    |  3 +-
 12 files changed, 100 insertions(+), 43 deletions(-)

diff --git a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_drv.h b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_drv.h
index 250e9833b955..7ffca004345c 100644
--- a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_drv.h
+++ b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_drv.h
@@ -289,6 +289,7 @@ struct vdec_pic_info {
  * @hw_id: hardware index used to identify different hardware.
  *
  * @msg_queue: msg queue used to store lat buffer information.
+ * @encoded_frame_cnt: number of encoded frames
  */
 struct mtk_vcodec_ctx {
 	enum mtk_instance_type type;
@@ -335,6 +336,8 @@ struct mtk_vcodec_ctx {
 	int hw_id;
 
 	struct vdec_msg_queue msg_queue;
+
+	int encoded_frame_cnt;
 };
 
 /*
@@ -476,6 +479,7 @@ struct mtk_vcodec_enc_pdata {
  * @vdec_racing_info: record register value
  * @dec_racing_info_mutex: mutex lock used for inner racing mode
  * @enc_hw_dev: used to store venc core device
+ * @curr_enc_ctx: used to store current used enc context
  */
 struct mtk_vcodec_dev {
 	struct v4l2_device v4l2_dev;
@@ -509,7 +513,8 @@ struct mtk_vcodec_dev {
 
 	/* decoder hardware mutex lock */
 	struct mutex dec_mutex[MTK_VDEC_HW_MAX];
-	struct mutex enc_mutex;
+	/* encoder hardware mutex lock */
+	struct mutex enc_mutex[MTK_VENC_HW_MAX];
 
 	struct mtk_vcodec_pm pm;
 	unsigned int dec_capability;
@@ -528,6 +533,7 @@ struct mtk_vcodec_dev {
 	struct mutex dec_racing_info_mutex;
 
 	void *enc_hw_dev[MTK_VENC_HW_MAX];
+	struct mtk_vcodec_ctx *curr_enc_ctx[MTK_VENC_HW_MAX];
 };
 
 static inline struct mtk_vcodec_ctx *fh_to_ctx(struct v4l2_fh *fh)
diff --git a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc.c b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc.c
index ad20f729ef6f..a2586485970b 100644
--- a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc.c
+++ b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc.c
@@ -1240,6 +1240,8 @@ static void mtk_venc_worker(struct work_struct *work)
 				 enc_result.bs_size);
 	}
 
+	ctx->encoded_frame_cnt++;
+
 	v4l2_m2m_job_finish(ctx->dev->m2m_dev_enc, ctx->m2m_ctx);
 
 	mtk_v4l2_debug(1, "<=== src_buf[%d] dst_buf[%d] venc_if_encode ret=%d Size=%u===>",
@@ -1448,19 +1450,19 @@ int mtk_vcodec_enc_queue_init(void *priv, struct vb2_queue *src_vq,
 	return vb2_queue_init(dst_vq);
 }
 
-int mtk_venc_unlock(struct mtk_vcodec_ctx *ctx)
+int mtk_venc_unlock(struct mtk_vcodec_ctx *ctx, int hw_id)
 {
 	struct mtk_vcodec_dev *dev = ctx->dev;
 
-	mutex_unlock(&dev->enc_mutex);
+	mutex_unlock(&dev->enc_mutex[hw_id]);
 	return 0;
 }
 
-int mtk_venc_lock(struct mtk_vcodec_ctx *ctx)
+int mtk_venc_lock(struct mtk_vcodec_ctx *ctx, int hw_id)
 {
 	struct mtk_vcodec_dev *dev = ctx->dev;
 
-	mutex_lock(&dev->enc_mutex);
+	mutex_lock(&dev->enc_mutex[hw_id]);
 	return 0;
 }
 
diff --git a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc.h b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc.h
index 513ee7993e34..29f5c8d1b59f 100644
--- a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc.h
+++ b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc.h
@@ -39,8 +39,8 @@ struct mtk_video_enc_buf {
 extern const struct v4l2_ioctl_ops mtk_venc_ioctl_ops;
 extern const struct v4l2_m2m_ops mtk_venc_m2m_ops;
 
-int mtk_venc_unlock(struct mtk_vcodec_ctx *ctx);
-int mtk_venc_lock(struct mtk_vcodec_ctx *ctx);
+int mtk_venc_unlock(struct mtk_vcodec_ctx *ctx, int hw_id);
+int mtk_venc_lock(struct mtk_vcodec_ctx *ctx, int hw_id);
 int mtk_vcodec_enc_queue_init(void *priv, struct vb2_queue *src_vq,
 			      struct vb2_queue *dst_vq);
 void mtk_vcodec_enc_release(struct mtk_vcodec_ctx *ctx);
diff --git a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_drv.c b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_drv.c
index 2e997b32a541..1c73bfeb80cb 100644
--- a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_drv.c
+++ b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_drv.c
@@ -91,7 +91,7 @@ static irqreturn_t mtk_vcodec_enc_irq_handler(int irq, void *priv)
 	void __iomem *addr;
 
 	spin_lock_irqsave(&dev->irqlock, flags);
-	ctx = dev->curr_ctx;
+	ctx = dev->curr_enc_ctx[MTK_VENC_CORE_0];
 	spin_unlock_irqrestore(&dev->irqlock, flags);
 
 	mtk_v4l2_debug(1, "id=%d coreid:%d", ctx->id, dev->venc_pdata->core_id);
@@ -231,7 +231,7 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
 	struct resource *res;
 	phandle rproc_phandle;
 	enum mtk_vcodec_fw_type fw_type;
-	int ret;
+	int ret, i;
 
 	dev = devm_kzalloc(&pdev->dev, sizeof(*dev), GFP_KERNEL);
 	if (!dev)
@@ -298,7 +298,9 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
 		goto err_res;
 	}
 
-	mutex_init(&dev->enc_mutex);
+	for (i = 0; i < MTK_VENC_HW_MAX; i++)
+		mutex_init(&dev->enc_mutex[i]);
+
 	mutex_init(&dev->dev_mutex);
 	spin_lock_init(&dev->irqlock);
 
diff --git a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.c b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.c
index 1ed4279925b7..6df5221b742f 100644
--- a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.c
+++ b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.c
@@ -46,13 +46,14 @@ static void clean_hw_irq_status(unsigned int irq_status, void __iomem *addr)
 static irqreturn_t mtk_enc_hw_irq_handler(int irq, void *priv)
 {
 	struct mtk_venc_hw_dev *dev = priv;
+	struct mtk_vcodec_dev *main_dev = dev->main_dev;
 	struct mtk_vcodec_ctx *ctx;
 	unsigned long flags;
 	void __iomem *addr;
 
-	spin_lock_irqsave(&dev->main_dev->irqlock, flags);
-	ctx = dev->curr_ctx;
-	spin_unlock_irqrestore(&dev->main_dev->irqlock, flags);
+	spin_lock_irqsave(&main_dev->irqlock, flags);
+	ctx = main_dev->curr_enc_ctx[dev->hw_id];
+	spin_unlock_irqrestore(&main_dev->irqlock, flags);
 	if (!ctx)
 		return IRQ_HANDLED;
 
diff --git a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.h b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.h
index 9daea9665659..e8dfd46b3b01 100644
--- a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.h
+++ b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.h
@@ -14,7 +14,6 @@
  * @plat_dev: platform_device
  * @main_dev: main device
  * @pm: power management data
- * @curr_ctx: the context that is waiting for venc hardware
  * @reg_base: mapped address of venc registers
  * @irq_status: venc hardware irq status
  * @enc_irq: venc device irq
@@ -25,7 +24,6 @@ struct mtk_venc_hw_dev {
 	struct mtk_vcodec_dev *main_dev;
 
 	struct mtk_vcodec_pm pm;
-	struct mtk_vcodec_ctx *curr_ctx;
 
 	void __iomem *reg_base;
 	unsigned int irq_status;
diff --git a/drivers/media/platform/mediatek/vcodec/venc/venc_h264_if.c b/drivers/media/platform/mediatek/vcodec/venc/venc_h264_if.c
index 3a565929f2b0..c114aa7dcd20 100644
--- a/drivers/media/platform/mediatek/vcodec/venc/venc_h264_if.c
+++ b/drivers/media/platform/mediatek/vcodec/venc/venc_h264_if.c
@@ -454,7 +454,8 @@ static int h264_encode_sps(struct venc_h264_inst *inst,
 
 	mtk_vcodec_debug_enter(inst);
 
-	ret = vpu_enc_encode(&inst->vpu_inst, H264_BS_MODE_SPS, NULL, bs_buf, NULL);
+	ret = vpu_enc_encode(&inst->vpu_inst, H264_BS_MODE_SPS, NULL,
+			     bs_buf, NULL, MTK_VENC_CORE_0);
 	if (ret)
 		return ret;
 
@@ -480,7 +481,8 @@ static int h264_encode_pps(struct venc_h264_inst *inst,
 
 	mtk_vcodec_debug_enter(inst);
 
-	ret = vpu_enc_encode(&inst->vpu_inst, H264_BS_MODE_PPS, NULL, bs_buf, NULL);
+	ret = vpu_enc_encode(&inst->vpu_inst, H264_BS_MODE_PPS, NULL,
+			     bs_buf, NULL, MTK_VENC_CORE_0);
 	if (ret)
 		return ret;
 
@@ -522,7 +524,8 @@ static int h264_encode_header(struct venc_h264_inst *inst,
 static int h264_encode_frame(struct venc_h264_inst *inst,
 			     struct venc_frm_buf *frm_buf,
 			     struct mtk_vcodec_mem *bs_buf,
-			     unsigned int *bs_size)
+			     unsigned int *bs_size,
+			     int hw_id)
 {
 	struct venc_frame_info frame_info;
 	struct mtk_vcodec_ctx *ctx = inst->ctx;
@@ -546,7 +549,8 @@ static int h264_encode_frame(struct venc_h264_inst *inst,
 	mtk_vcodec_debug(inst, "frm_count = %d,skip_frm_count =%d,frm_type=%d.\n",
 			 frame_info.frm_count, frame_info.skip_frm_count,
 			 frame_info.frm_type);
-	ret = vpu_enc_encode(&inst->vpu_inst, H264_BS_MODE_FRAME, frm_buf, bs_buf, &frame_info);
+	ret = vpu_enc_encode(&inst->vpu_inst, H264_BS_MODE_FRAME, frm_buf,
+			     bs_buf, &frame_info, hw_id);
 	if (ret)
 		return ret;
 
@@ -674,7 +678,7 @@ static int h264_enc_encode(void *handle,
 
 		if (!inst->prepend_hdr) {
 			ret = h264_encode_frame(inst, frm_buf, bs_buf,
-						&result->bs_size);
+						&result->bs_size, ctx->hw_id);
 			if (ret)
 				goto encode_err;
 			result->is_key_frm = inst->vpu_inst.is_key_frm;
@@ -702,7 +706,7 @@ static int h264_enc_encode(void *handle,
 		tmp_bs_buf.size = bs_buf->size - (hdr_sz + filler_sz);
 
 		ret = h264_encode_frame(inst, frm_buf, &tmp_bs_buf,
-					&bs_size_frm);
+					&bs_size_frm, ctx->hw_id);
 		if (ret)
 			goto encode_err;
 
diff --git a/drivers/media/platform/mediatek/vcodec/venc/venc_vp8_if.c b/drivers/media/platform/mediatek/vcodec/venc/venc_vp8_if.c
index 56ce58f761f1..9eed39a4faa2 100644
--- a/drivers/media/platform/mediatek/vcodec/venc/venc_vp8_if.c
+++ b/drivers/media/platform/mediatek/vcodec/venc/venc_vp8_if.c
@@ -302,7 +302,8 @@ static int vp8_enc_encode_frame(struct venc_vp8_inst *inst,
 
 	mtk_vcodec_debug(inst, "->frm_cnt=%d", inst->frm_cnt);
 
-	ret = vpu_enc_encode(&inst->vpu_inst, 0, frm_buf, bs_buf, NULL);
+	ret = vpu_enc_encode(&inst->vpu_inst, 0, frm_buf,
+			     bs_buf, NULL, MTK_VENC_CORE_0);
 	if (ret)
 		return ret;
 
diff --git a/drivers/media/platform/mediatek/vcodec/venc_drv_if.c b/drivers/media/platform/mediatek/vcodec/venc_drv_if.c
index 6cbdb7e30bb3..53fa9a91d3fa 100644
--- a/drivers/media/platform/mediatek/vcodec/venc_drv_if.c
+++ b/drivers/media/platform/mediatek/vcodec/venc_drv_if.c
@@ -16,6 +16,8 @@
 #include "mtk_vcodec_enc.h"
 #include "mtk_vcodec_enc_pm.h"
 
+#define ODD_VENC_FRAME 0x1
+
 int venc_if_init(struct mtk_vcodec_ctx *ctx, unsigned int fourcc)
 {
 	int ret = 0;
@@ -31,9 +33,9 @@ int venc_if_init(struct mtk_vcodec_ctx *ctx, unsigned int fourcc)
 		return -EINVAL;
 	}
 
-	mtk_venc_lock(ctx);
+	mtk_venc_lock(ctx, ctx->hw_id);
 	ret = ctx->enc_if->init(ctx);
-	mtk_venc_unlock(ctx);
+	mtk_venc_unlock(ctx, ctx->hw_id);
 
 	return ret;
 }
@@ -43,9 +45,9 @@ int venc_if_set_param(struct mtk_vcodec_ctx *ctx,
 {
 	int ret = 0;
 
-	mtk_venc_lock(ctx);
+	mtk_venc_lock(ctx, ctx->hw_id);
 	ret = ctx->enc_if->set_param(ctx->drv_handle, type, in);
-	mtk_venc_unlock(ctx);
+	mtk_venc_unlock(ctx, ctx->hw_id);
 
 	return ret;
 }
@@ -56,24 +58,13 @@ int venc_if_encode(struct mtk_vcodec_ctx *ctx,
 		   struct venc_done_result *result)
 {
 	int ret = 0;
-	unsigned long flags;
 
-	mtk_venc_lock(ctx);
+	venc_encode_prepare(ctx, opt);
 
-	spin_lock_irqsave(&ctx->dev->irqlock, flags);
-	ctx->dev->curr_ctx = ctx;
-	spin_unlock_irqrestore(&ctx->dev->irqlock, flags);
-
-	mtk_vcodec_enc_clock_on(ctx->dev, 0);
 	ret = ctx->enc_if->encode(ctx->drv_handle, opt, frm_buf,
 				  bs_buf, result);
-	mtk_vcodec_enc_clock_off(ctx->dev, 0);
-
-	spin_lock_irqsave(&ctx->dev->irqlock, flags);
-	ctx->dev->curr_ctx = NULL;
-	spin_unlock_irqrestore(&ctx->dev->irqlock, flags);
 
-	mtk_venc_unlock(ctx);
+	venc_encode_unprepare(ctx, opt, ret);
 	return ret;
 }
 
@@ -84,11 +75,51 @@ int venc_if_deinit(struct mtk_vcodec_ctx *ctx)
 	if (!ctx->drv_handle)
 		return 0;
 
-	mtk_venc_lock(ctx);
+	mtk_venc_lock(ctx, ctx->hw_id);
 	ret = ctx->enc_if->deinit(ctx->drv_handle);
-	mtk_venc_unlock(ctx);
+	mtk_venc_unlock(ctx, ctx->hw_id);
 
 	ctx->drv_handle = NULL;
 
 	return ret;
 }
+
+void venc_encode_prepare(struct mtk_vcodec_ctx *ctx,
+			 enum venc_start_opt opt)
+{
+	unsigned long flags;
+
+	if (IS_VENC_MULTICORE(ctx->dev->enc_capability)) {
+		if (ctx->encoded_frame_cnt & ODD_VENC_FRAME)
+			ctx->hw_id = MTK_VENC_CORE_1;
+		else
+			ctx->hw_id = MTK_VENC_CORE_0;
+	} else {
+		ctx->hw_id = MTK_VENC_CORE_0;
+	}
+
+	mtk_venc_lock(ctx, ctx->hw_id);
+
+	spin_lock_irqsave(&ctx->dev->irqlock, flags);
+	ctx->dev->curr_enc_ctx[ctx->hw_id] = ctx;
+	spin_unlock_irqrestore(&ctx->dev->irqlock, flags);
+
+	mtk_vcodec_enc_clock_on(ctx->dev, ctx->hw_id);
+}
+
+void venc_encode_unprepare(struct mtk_vcodec_ctx *ctx,
+			   enum venc_start_opt opt, int ret)
+{
+	unsigned long flags;
+
+	if (ret || !IS_VENC_MULTICORE(ctx->dev->enc_capability) ||
+	    opt == VENC_START_OPT_ENCODE_SEQUENCE_HEADER) {
+		mtk_vcodec_enc_clock_off(ctx->dev, ctx->hw_id);
+
+		spin_lock_irqsave(&ctx->dev->irqlock, flags);
+		ctx->dev->curr_enc_ctx[ctx->hw_id] = NULL;
+		spin_unlock_irqrestore(&ctx->dev->irqlock, flags);
+
+		mtk_venc_unlock(ctx, ctx->hw_id);
+	}
+}
diff --git a/drivers/media/platform/mediatek/vcodec/venc_drv_if.h b/drivers/media/platform/mediatek/vcodec/venc_drv_if.h
index 0b04a1020873..e676ccf6bd25 100644
--- a/drivers/media/platform/mediatek/vcodec/venc_drv_if.h
+++ b/drivers/media/platform/mediatek/vcodec/venc_drv_if.h
@@ -167,4 +167,8 @@ int venc_if_encode(struct mtk_vcodec_ctx *ctx,
 		   struct mtk_vcodec_mem *bs_buf,
 		   struct venc_done_result *result);
 
+void venc_encode_prepare(struct mtk_vcodec_ctx *ctx,
+			 enum venc_start_opt opt);
+void venc_encode_unprepare(struct mtk_vcodec_ctx *ctx,
+			   enum venc_start_opt opt, int ret);
 #endif /* _VENC_DRV_IF_H_ */
diff --git a/drivers/media/platform/mediatek/vcodec/venc_vpu_if.c b/drivers/media/platform/mediatek/vcodec/venc_vpu_if.c
index d3570c4c177d..7d6c32c304a1 100644
--- a/drivers/media/platform/mediatek/vcodec/venc_vpu_if.c
+++ b/drivers/media/platform/mediatek/vcodec/venc_vpu_if.c
@@ -225,7 +225,8 @@ int vpu_enc_set_param(struct venc_vpu_inst *vpu,
 int vpu_enc_encode(struct venc_vpu_inst *vpu, unsigned int bs_mode,
 		   struct venc_frm_buf *frm_buf,
 		   struct mtk_vcodec_mem *bs_buf,
-		   struct venc_frame_info *frame_info)
+		   struct venc_frame_info *frame_info,
+		   int hw_id)
 {
 	const bool is_ext = MTK_ENC_CTX_IS_EXT(vpu->ctx);
 	size_t msg_size = is_ext ?
@@ -261,6 +262,12 @@ int vpu_enc_encode(struct venc_vpu_inst *vpu, unsigned int bs_mode,
 		out.data[1] = frame_info->skip_frm_count;
 		out.data[2] = frame_info->frm_type;
 	}
+
+	if (IS_VENC_MULTICORE(vpu->ctx->dev->enc_capability)) {
+		out.data_item = 4;
+		out.data[3] = hw_id;
+	}
+
 	if (vpu_enc_send_msg(vpu, &out, msg_size)) {
 		mtk_vcodec_err(vpu, "AP_IPIMSG_ENC_ENCODE %d fail",
 			       bs_mode);
diff --git a/drivers/media/platform/mediatek/vcodec/venc_vpu_if.h b/drivers/media/platform/mediatek/vcodec/venc_vpu_if.h
index f83bc1b3f2bf..23efcea8d510 100644
--- a/drivers/media/platform/mediatek/vcodec/venc_vpu_if.h
+++ b/drivers/media/platform/mediatek/vcodec/venc_vpu_if.h
@@ -45,7 +45,8 @@ int vpu_enc_set_param(struct venc_vpu_inst *vpu,
 int vpu_enc_encode(struct venc_vpu_inst *vpu, unsigned int bs_mode,
 		   struct venc_frm_buf *frm_buf,
 		   struct mtk_vcodec_mem *bs_buf,
-		   struct venc_frame_info *frame_info);
+		   struct venc_frame_info *frame_info,
+		   int hw_id);
 int vpu_enc_deinit(struct venc_vpu_inst *vpu);
 
 #endif
-- 
2.18.0


^ permalink raw reply related	[flat|nested] 22+ messages in thread
* [PATCH v5, 7/8] media: mediatek: vcodec: Add multi-core encoding process
@ 2022-07-29  3:51   ` Irui Wang
  0 siblings, 0 replies; 22+ messages in thread
From: Irui Wang @ 2022-07-29  3:51 UTC (permalink / raw)
  To: Hans Verkuil, angelogioacchino.delregno, Rob Herring,
	Mauro Carvalho Chehab, Matthias Brugger, Alexandre Courbot,
	Tiffany Lin, Andrew-CT Chen, Tzung-Bi Shih, Tomasz Figa,
	nicolas.dufresne
  Cc: Maoguang Meng, Longfei Wang, Yunfei Dong, Irui Wang, linux-media,
	devicetree, linux-kernel, linux-arm-kernel, linux-mediatek,
	Project_Global_Chrome_Upstream_Group

MT8195 has two encoder hardware cores, when enable multi-core encoding,
it is possiable to trying to use the two cores. According to hardware
requirements, input frame 0 uses core0, frame 1 uses core1, then frame 2
uses core0, lock the device and enable clock by used core, and for
sequence header encoding, it always use core0.

Signed-off-by: Irui Wang <irui.wang@mediatek.com>
---
 .../platform/mediatek/vcodec/mtk_vcodec_drv.h |  8 ++-
 .../platform/mediatek/vcodec/mtk_vcodec_enc.c | 10 +--
 .../platform/mediatek/vcodec/mtk_vcodec_enc.h |  4 +-
 .../mediatek/vcodec/mtk_vcodec_enc_drv.c      |  8 ++-
 .../mediatek/vcodec/mtk_vcodec_enc_hw.c       |  7 +-
 .../mediatek/vcodec/mtk_vcodec_enc_hw.h       |  2 -
 .../mediatek/vcodec/venc/venc_h264_if.c       | 16 +++--
 .../mediatek/vcodec/venc/venc_vp8_if.c        |  3 +-
 .../platform/mediatek/vcodec/venc_drv_if.c    | 69 ++++++++++++++-----
 .../platform/mediatek/vcodec/venc_drv_if.h    |  4 ++
 .../platform/mediatek/vcodec/venc_vpu_if.c    |  9 ++-
 .../platform/mediatek/vcodec/venc_vpu_if.h    |  3 +-
 12 files changed, 100 insertions(+), 43 deletions(-)

diff --git a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_drv.h b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_drv.h
index 250e9833b955..7ffca004345c 100644
--- a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_drv.h
+++ b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_drv.h
@@ -289,6 +289,7 @@ struct vdec_pic_info {
  * @hw_id: hardware index used to identify different hardware.
  *
  * @msg_queue: msg queue used to store lat buffer information.
+ * @encoded_frame_cnt: number of encoded frames
  */
 struct mtk_vcodec_ctx {
 	enum mtk_instance_type type;
@@ -335,6 +336,8 @@ struct mtk_vcodec_ctx {
 	int hw_id;
 
 	struct vdec_msg_queue msg_queue;
+
+	int encoded_frame_cnt;
 };
 
 /*
@@ -476,6 +479,7 @@ struct mtk_vcodec_enc_pdata {
  * @vdec_racing_info: record register value
  * @dec_racing_info_mutex: mutex lock used for inner racing mode
  * @enc_hw_dev: used to store venc core device
+ * @curr_enc_ctx: used to store current used enc context
  */
 struct mtk_vcodec_dev {
 	struct v4l2_device v4l2_dev;
@@ -509,7 +513,8 @@ struct mtk_vcodec_dev {
 
 	/* decoder hardware mutex lock */
 	struct mutex dec_mutex[MTK_VDEC_HW_MAX];
-	struct mutex enc_mutex;
+	/* encoder hardware mutex lock */
+	struct mutex enc_mutex[MTK_VENC_HW_MAX];
 
 	struct mtk_vcodec_pm pm;
 	unsigned int dec_capability;
@@ -528,6 +533,7 @@ struct mtk_vcodec_dev {
 	struct mutex dec_racing_info_mutex;
 
 	void *enc_hw_dev[MTK_VENC_HW_MAX];
+	struct mtk_vcodec_ctx *curr_enc_ctx[MTK_VENC_HW_MAX];
 };
 
 static inline struct mtk_vcodec_ctx *fh_to_ctx(struct v4l2_fh *fh)
diff --git a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc.c b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc.c
index ad20f729ef6f..a2586485970b 100644
--- a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc.c
+++ b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc.c
@@ -1240,6 +1240,8 @@ static void mtk_venc_worker(struct work_struct *work)
 				 enc_result.bs_size);
 	}
 
+	ctx->encoded_frame_cnt++;
+
 	v4l2_m2m_job_finish(ctx->dev->m2m_dev_enc, ctx->m2m_ctx);
 
 	mtk_v4l2_debug(1, "<=== src_buf[%d] dst_buf[%d] venc_if_encode ret=%d Size=%u===>",
@@ -1448,19 +1450,19 @@ int mtk_vcodec_enc_queue_init(void *priv, struct vb2_queue *src_vq,
 	return vb2_queue_init(dst_vq);
 }
 
-int mtk_venc_unlock(struct mtk_vcodec_ctx *ctx)
+int mtk_venc_unlock(struct mtk_vcodec_ctx *ctx, int hw_id)
 {
 	struct mtk_vcodec_dev *dev = ctx->dev;
 
-	mutex_unlock(&dev->enc_mutex);
+	mutex_unlock(&dev->enc_mutex[hw_id]);
 	return 0;
 }
 
-int mtk_venc_lock(struct mtk_vcodec_ctx *ctx)
+int mtk_venc_lock(struct mtk_vcodec_ctx *ctx, int hw_id)
 {
 	struct mtk_vcodec_dev *dev = ctx->dev;
 
-	mutex_lock(&dev->enc_mutex);
+	mutex_lock(&dev->enc_mutex[hw_id]);
 	return 0;
 }
 
diff --git a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc.h b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc.h
index 513ee7993e34..29f5c8d1b59f 100644
--- a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc.h
+++ b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc.h
@@ -39,8 +39,8 @@ struct mtk_video_enc_buf {
 extern const struct v4l2_ioctl_ops mtk_venc_ioctl_ops;
 extern const struct v4l2_m2m_ops mtk_venc_m2m_ops;
 
-int mtk_venc_unlock(struct mtk_vcodec_ctx *ctx);
-int mtk_venc_lock(struct mtk_vcodec_ctx *ctx);
+int mtk_venc_unlock(struct mtk_vcodec_ctx *ctx, int hw_id);
+int mtk_venc_lock(struct mtk_vcodec_ctx *ctx, int hw_id);
 int mtk_vcodec_enc_queue_init(void *priv, struct vb2_queue *src_vq,
 			      struct vb2_queue *dst_vq);
 void mtk_vcodec_enc_release(struct mtk_vcodec_ctx *ctx);
diff --git a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_drv.c b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_drv.c
index 2e997b32a541..1c73bfeb80cb 100644
--- a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_drv.c
+++ b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_drv.c
@@ -91,7 +91,7 @@ static irqreturn_t mtk_vcodec_enc_irq_handler(int irq, void *priv)
 	void __iomem *addr;
 
 	spin_lock_irqsave(&dev->irqlock, flags);
-	ctx = dev->curr_ctx;
+	ctx = dev->curr_enc_ctx[MTK_VENC_CORE_0];
 	spin_unlock_irqrestore(&dev->irqlock, flags);
 
 	mtk_v4l2_debug(1, "id=%d coreid:%d", ctx->id, dev->venc_pdata->core_id);
@@ -231,7 +231,7 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
 	struct resource *res;
 	phandle rproc_phandle;
 	enum mtk_vcodec_fw_type fw_type;
-	int ret;
+	int ret, i;
 
 	dev = devm_kzalloc(&pdev->dev, sizeof(*dev), GFP_KERNEL);
 	if (!dev)
@@ -298,7 +298,9 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
 		goto err_res;
 	}
 
-	mutex_init(&dev->enc_mutex);
+	for (i = 0; i < MTK_VENC_HW_MAX; i++)
+		mutex_init(&dev->enc_mutex[i]);
+
 	mutex_init(&dev->dev_mutex);
 	spin_lock_init(&dev->irqlock);
 
diff --git a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.c b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.c
index 1ed4279925b7..6df5221b742f 100644
--- a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.c
+++ b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.c
@@ -46,13 +46,14 @@ static void clean_hw_irq_status(unsigned int irq_status, void __iomem *addr)
 static irqreturn_t mtk_enc_hw_irq_handler(int irq, void *priv)
 {
 	struct mtk_venc_hw_dev *dev = priv;
+	struct mtk_vcodec_dev *main_dev = dev->main_dev;
 	struct mtk_vcodec_ctx *ctx;
 	unsigned long flags;
 	void __iomem *addr;
 
-	spin_lock_irqsave(&dev->main_dev->irqlock, flags);
-	ctx = dev->curr_ctx;
-	spin_unlock_irqrestore(&dev->main_dev->irqlock, flags);
+	spin_lock_irqsave(&main_dev->irqlock, flags);
+	ctx = main_dev->curr_enc_ctx[dev->hw_id];
+	spin_unlock_irqrestore(&main_dev->irqlock, flags);
 	if (!ctx)
 		return IRQ_HANDLED;
 
diff --git a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.h b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.h
index 9daea9665659..e8dfd46b3b01 100644
--- a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.h
+++ b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.h
@@ -14,7 +14,6 @@
  * @plat_dev: platform_device
  * @main_dev: main device
  * @pm: power management data
- * @curr_ctx: the context that is waiting for venc hardware
  * @reg_base: mapped address of venc registers
  * @irq_status: venc hardware irq status
  * @enc_irq: venc device irq
@@ -25,7 +24,6 @@ struct mtk_venc_hw_dev {
 	struct mtk_vcodec_dev *main_dev;
 
 	struct mtk_vcodec_pm pm;
-	struct mtk_vcodec_ctx *curr_ctx;
 
 	void __iomem *reg_base;
 	unsigned int irq_status;
diff --git a/drivers/media/platform/mediatek/vcodec/venc/venc_h264_if.c b/drivers/media/platform/mediatek/vcodec/venc/venc_h264_if.c
index 3a565929f2b0..c114aa7dcd20 100644
--- a/drivers/media/platform/mediatek/vcodec/venc/venc_h264_if.c
+++ b/drivers/media/platform/mediatek/vcodec/venc/venc_h264_if.c
@@ -454,7 +454,8 @@ static int h264_encode_sps(struct venc_h264_inst *inst,
 
 	mtk_vcodec_debug_enter(inst);
 
-	ret = vpu_enc_encode(&inst->vpu_inst, H264_BS_MODE_SPS, NULL, bs_buf, NULL);
+	ret = vpu_enc_encode(&inst->vpu_inst, H264_BS_MODE_SPS, NULL,
+			     bs_buf, NULL, MTK_VENC_CORE_0);
 	if (ret)
 		return ret;
 
@@ -480,7 +481,8 @@ static int h264_encode_pps(struct venc_h264_inst *inst,
 
 	mtk_vcodec_debug_enter(inst);
 
-	ret = vpu_enc_encode(&inst->vpu_inst, H264_BS_MODE_PPS, NULL, bs_buf, NULL);
+	ret = vpu_enc_encode(&inst->vpu_inst, H264_BS_MODE_PPS, NULL,
+			     bs_buf, NULL, MTK_VENC_CORE_0);
 	if (ret)
 		return ret;
 
@@ -522,7 +524,8 @@ static int h264_encode_header(struct venc_h264_inst *inst,
 static int h264_encode_frame(struct venc_h264_inst *inst,
 			     struct venc_frm_buf *frm_buf,
 			     struct mtk_vcodec_mem *bs_buf,
-			     unsigned int *bs_size)
+			     unsigned int *bs_size,
+			     int hw_id)
 {
 	struct venc_frame_info frame_info;
 	struct mtk_vcodec_ctx *ctx = inst->ctx;
@@ -546,7 +549,8 @@ static int h264_encode_frame(struct venc_h264_inst *inst,
 	mtk_vcodec_debug(inst, "frm_count = %d,skip_frm_count =%d,frm_type=%d.\n",
 			 frame_info.frm_count, frame_info.skip_frm_count,
 			 frame_info.frm_type);
-	ret = vpu_enc_encode(&inst->vpu_inst, H264_BS_MODE_FRAME, frm_buf, bs_buf, &frame_info);
+	ret = vpu_enc_encode(&inst->vpu_inst, H264_BS_MODE_FRAME, frm_buf,
+			     bs_buf, &frame_info, hw_id);
 	if (ret)
 		return ret;
 
@@ -674,7 +678,7 @@ static int h264_enc_encode(void *handle,
 
 		if (!inst->prepend_hdr) {
 			ret = h264_encode_frame(inst, frm_buf, bs_buf,
-						&result->bs_size);
+						&result->bs_size, ctx->hw_id);
 			if (ret)
 				goto encode_err;
 			result->is_key_frm = inst->vpu_inst.is_key_frm;
@@ -702,7 +706,7 @@ static int h264_enc_encode(void *handle,
 		tmp_bs_buf.size = bs_buf->size - (hdr_sz + filler_sz);
 
 		ret = h264_encode_frame(inst, frm_buf, &tmp_bs_buf,
-					&bs_size_frm);
+					&bs_size_frm, ctx->hw_id);
 		if (ret)
 			goto encode_err;
 
diff --git a/drivers/media/platform/mediatek/vcodec/venc/venc_vp8_if.c b/drivers/media/platform/mediatek/vcodec/venc/venc_vp8_if.c
index 56ce58f761f1..9eed39a4faa2 100644
--- a/drivers/media/platform/mediatek/vcodec/venc/venc_vp8_if.c
+++ b/drivers/media/platform/mediatek/vcodec/venc/venc_vp8_if.c
@@ -302,7 +302,8 @@ static int vp8_enc_encode_frame(struct venc_vp8_inst *inst,
 
 	mtk_vcodec_debug(inst, "->frm_cnt=%d", inst->frm_cnt);
 
-	ret = vpu_enc_encode(&inst->vpu_inst, 0, frm_buf, bs_buf, NULL);
+	ret = vpu_enc_encode(&inst->vpu_inst, 0, frm_buf,
+			     bs_buf, NULL, MTK_VENC_CORE_0);
 	if (ret)
 		return ret;
 
diff --git a/drivers/media/platform/mediatek/vcodec/venc_drv_if.c b/drivers/media/platform/mediatek/vcodec/venc_drv_if.c
index 6cbdb7e30bb3..53fa9a91d3fa 100644
--- a/drivers/media/platform/mediatek/vcodec/venc_drv_if.c
+++ b/drivers/media/platform/mediatek/vcodec/venc_drv_if.c
@@ -16,6 +16,8 @@
 #include "mtk_vcodec_enc.h"
 #include "mtk_vcodec_enc_pm.h"
 
+#define ODD_VENC_FRAME 0x1
+
 int venc_if_init(struct mtk_vcodec_ctx *ctx, unsigned int fourcc)
 {
 	int ret = 0;
@@ -31,9 +33,9 @@ int venc_if_init(struct mtk_vcodec_ctx *ctx, unsigned int fourcc)
 		return -EINVAL;
 	}
 
-	mtk_venc_lock(ctx);
+	mtk_venc_lock(ctx, ctx->hw_id);
 	ret = ctx->enc_if->init(ctx);
-	mtk_venc_unlock(ctx);
+	mtk_venc_unlock(ctx, ctx->hw_id);
 
 	return ret;
 }
@@ -43,9 +45,9 @@ int venc_if_set_param(struct mtk_vcodec_ctx *ctx,
 {
 	int ret = 0;
 
-	mtk_venc_lock(ctx);
+	mtk_venc_lock(ctx, ctx->hw_id);
 	ret = ctx->enc_if->set_param(ctx->drv_handle, type, in);
-	mtk_venc_unlock(ctx);
+	mtk_venc_unlock(ctx, ctx->hw_id);
 
 	return ret;
 }
@@ -56,24 +58,13 @@ int venc_if_encode(struct mtk_vcodec_ctx *ctx,
 		   struct venc_done_result *result)
 {
 	int ret = 0;
-	unsigned long flags;
 
-	mtk_venc_lock(ctx);
+	venc_encode_prepare(ctx, opt);
 
-	spin_lock_irqsave(&ctx->dev->irqlock, flags);
-	ctx->dev->curr_ctx = ctx;
-	spin_unlock_irqrestore(&ctx->dev->irqlock, flags);
-
-	mtk_vcodec_enc_clock_on(ctx->dev, 0);
 	ret = ctx->enc_if->encode(ctx->drv_handle, opt, frm_buf,
 				  bs_buf, result);
-	mtk_vcodec_enc_clock_off(ctx->dev, 0);
-
-	spin_lock_irqsave(&ctx->dev->irqlock, flags);
-	ctx->dev->curr_ctx = NULL;
-	spin_unlock_irqrestore(&ctx->dev->irqlock, flags);
 
-	mtk_venc_unlock(ctx);
+	venc_encode_unprepare(ctx, opt, ret);
 	return ret;
 }
 
@@ -84,11 +75,51 @@ int venc_if_deinit(struct mtk_vcodec_ctx *ctx)
 	if (!ctx->drv_handle)
 		return 0;
 
-	mtk_venc_lock(ctx);
+	mtk_venc_lock(ctx, ctx->hw_id);
 	ret = ctx->enc_if->deinit(ctx->drv_handle);
-	mtk_venc_unlock(ctx);
+	mtk_venc_unlock(ctx, ctx->hw_id);
 
 	ctx->drv_handle = NULL;
 
 	return ret;
 }
+
+void venc_encode_prepare(struct mtk_vcodec_ctx *ctx,
+			 enum venc_start_opt opt)
+{
+	unsigned long flags;
+
+	if (IS_VENC_MULTICORE(ctx->dev->enc_capability)) {
+		if (ctx->encoded_frame_cnt & ODD_VENC_FRAME)
+			ctx->hw_id = MTK_VENC_CORE_1;
+		else
+			ctx->hw_id = MTK_VENC_CORE_0;
+	} else {
+		ctx->hw_id = MTK_VENC_CORE_0;
+	}
+
+	mtk_venc_lock(ctx, ctx->hw_id);
+
+	spin_lock_irqsave(&ctx->dev->irqlock, flags);
+	ctx->dev->curr_enc_ctx[ctx->hw_id] = ctx;
+	spin_unlock_irqrestore(&ctx->dev->irqlock, flags);
+
+	mtk_vcodec_enc_clock_on(ctx->dev, ctx->hw_id);
+}
+
+void venc_encode_unprepare(struct mtk_vcodec_ctx *ctx,
+			   enum venc_start_opt opt, int ret)
+{
+	unsigned long flags;
+
+	if (ret || !IS_VENC_MULTICORE(ctx->dev->enc_capability) ||
+	    opt == VENC_START_OPT_ENCODE_SEQUENCE_HEADER) {
+		mtk_vcodec_enc_clock_off(ctx->dev, ctx->hw_id);
+
+		spin_lock_irqsave(&ctx->dev->irqlock, flags);
+		ctx->dev->curr_enc_ctx[ctx->hw_id] = NULL;
+		spin_unlock_irqrestore(&ctx->dev->irqlock, flags);
+
+		mtk_venc_unlock(ctx, ctx->hw_id);
+	}
+}
diff --git a/drivers/media/platform/mediatek/vcodec/venc_drv_if.h b/drivers/media/platform/mediatek/vcodec/venc_drv_if.h
index 0b04a1020873..e676ccf6bd25 100644
--- a/drivers/media/platform/mediatek/vcodec/venc_drv_if.h
+++ b/drivers/media/platform/mediatek/vcodec/venc_drv_if.h
@@ -167,4 +167,8 @@ int venc_if_encode(struct mtk_vcodec_ctx *ctx,
 		   struct mtk_vcodec_mem *bs_buf,
 		   struct venc_done_result *result);
 
+void venc_encode_prepare(struct mtk_vcodec_ctx *ctx,
+			 enum venc_start_opt opt);
+void venc_encode_unprepare(struct mtk_vcodec_ctx *ctx,
+			   enum venc_start_opt opt, int ret);
 #endif /* _VENC_DRV_IF_H_ */
diff --git a/drivers/media/platform/mediatek/vcodec/venc_vpu_if.c b/drivers/media/platform/mediatek/vcodec/venc_vpu_if.c
index d3570c4c177d..7d6c32c304a1 100644
--- a/drivers/media/platform/mediatek/vcodec/venc_vpu_if.c
+++ b/drivers/media/platform/mediatek/vcodec/venc_vpu_if.c
@@ -225,7 +225,8 @@ int vpu_enc_set_param(struct venc_vpu_inst *vpu,
 int vpu_enc_encode(struct venc_vpu_inst *vpu, unsigned int bs_mode,
 		   struct venc_frm_buf *frm_buf,
 		   struct mtk_vcodec_mem *bs_buf,
-		   struct venc_frame_info *frame_info)
+		   struct venc_frame_info *frame_info,
+		   int hw_id)
 {
 	const bool is_ext = MTK_ENC_CTX_IS_EXT(vpu->ctx);
 	size_t msg_size = is_ext ?
@@ -261,6 +262,12 @@ int vpu_enc_encode(struct venc_vpu_inst *vpu, unsigned int bs_mode,
 		out.data[1] = frame_info->skip_frm_count;
 		out.data[2] = frame_info->frm_type;
 	}
+
+	if (IS_VENC_MULTICORE(vpu->ctx->dev->enc_capability)) {
+		out.data_item = 4;
+		out.data[3] = hw_id;
+	}
+
 	if (vpu_enc_send_msg(vpu, &out, msg_size)) {
 		mtk_vcodec_err(vpu, "AP_IPIMSG_ENC_ENCODE %d fail",
 			       bs_mode);
diff --git a/drivers/media/platform/mediatek/vcodec/venc_vpu_if.h b/drivers/media/platform/mediatek/vcodec/venc_vpu_if.h
index f83bc1b3f2bf..23efcea8d510 100644
--- a/drivers/media/platform/mediatek/vcodec/venc_vpu_if.h
+++ b/drivers/media/platform/mediatek/vcodec/venc_vpu_if.h
@@ -45,7 +45,8 @@ int vpu_enc_set_param(struct venc_vpu_inst *vpu,
 int vpu_enc_encode(struct venc_vpu_inst *vpu, unsigned int bs_mode,
 		   struct venc_frm_buf *frm_buf,
 		   struct mtk_vcodec_mem *bs_buf,
-		   struct venc_frame_info *frame_info);
+		   struct venc_frame_info *frame_info,
+		   int hw_id);
 int vpu_enc_deinit(struct venc_vpu_inst *vpu);
 
 #endif
-- 
2.18.0


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply related	[flat|nested] 22+ messages in thread
* [PATCH v5, 8/8] media: mediatek: vcodec: Return encoding result in asynchronous mode
  2022-07-29  3:51 ` Irui Wang
@ 2022-07-29  3:51   ` Irui Wang
  -1 siblings, 0 replies; 22+ messages in thread
From: Irui Wang @ 2022-07-29  3:51 UTC (permalink / raw)
  To: Hans Verkuil, angelogioacchino.delregno, Rob Herring,
	Mauro Carvalho Chehab, Matthias Brugger, Alexandre Courbot,
	Tiffany Lin, Andrew-CT Chen, Tzung-Bi Shih, Tomasz Figa,
	nicolas.dufresne
  Cc: Maoguang Meng, Longfei Wang, Yunfei Dong, Irui Wang, linux-media,
	devicetree, linux-kernel, linux-arm-kernel, linux-mediatek,
	Project_Global_Chrome_Upstream_Group

when enable multi-core encoding, the wait IRQ done synchronous function
should not be called, so the encoding result can't return to client in
device_run. Move the buffer done function in IRQ handler.

Signed-off-by: Irui Wang <irui.wang@mediatek.com>
---
 .../platform/mediatek/vcodec/mtk_vcodec_drv.h |  6 ++
 .../platform/mediatek/vcodec/mtk_vcodec_enc.c | 72 +++++++++++++++++--
 .../platform/mediatek/vcodec/mtk_vcodec_enc.h |  7 +-
 .../mediatek/vcodec/mtk_vcodec_enc_drv.c      | 28 +++++++-
 .../mediatek/vcodec/mtk_vcodec_enc_hw.c       | 13 +++-
 .../mediatek/vcodec/mtk_vcodec_enc_pm.c       |  1 +
 .../mediatek/vcodec/mtk_vcodec_util.h         |  1 +
 .../mediatek/vcodec/venc/venc_h264_if.c       | 20 ++++--
 .../platform/mediatek/vcodec/venc_drv_if.h    |  2 +
 9 files changed, 137 insertions(+), 13 deletions(-)

diff --git a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_drv.h b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_drv.h
index 7ffca004345c..50166fc68cb8 100644
--- a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_drv.h
+++ b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_drv.h
@@ -290,6 +290,9 @@ struct vdec_pic_info {
  *
  * @msg_queue: msg queue used to store lat buffer information.
  * @encoded_frame_cnt: number of encoded frames
+ * @pfrm_buf: used to store current ctx's frame buffer
+ * @pbs_buf: used to store current ctx's bitstream buffer
+ * @hdr_size: used to store prepend header size
  */
 struct mtk_vcodec_ctx {
 	enum mtk_instance_type type;
@@ -338,6 +341,9 @@ struct mtk_vcodec_ctx {
 	struct vdec_msg_queue msg_queue;
 
 	int encoded_frame_cnt;
+	struct vb2_v4l2_buffer *pfrm_buf[MTK_VENC_HW_MAX];
+	struct vb2_v4l2_buffer *pbs_buf[MTK_VENC_HW_MAX];
+	unsigned int hdr_size;
 };
 
 /*
diff --git a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc.c b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc.c
index a2586485970b..3778d802eed5 100644
--- a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc.c
+++ b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc.c
@@ -956,6 +956,8 @@ static void vb2ops_venc_stop_streaming(struct vb2_queue *q)
 
 	mtk_v4l2_debug(2, "[%d]-> type=%d", ctx->id, q->type);
 
+	mtk_venc_lock_all(ctx);
+
 	if (q->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
 		while ((dst_buf = v4l2_m2m_dst_buf_remove(ctx->m2m_ctx))) {
 			vb2_set_plane_payload(&dst_buf->vb2_buf, 0, 0);
@@ -1191,6 +1193,7 @@ static void mtk_venc_worker(struct work_struct *work)
 	 * is dequeued.
 	 */
 	if (src_buf == &ctx->empty_flush_buf.vb) {
+		mtk_venc_lock_all(ctx);
 		vb2_set_plane_payload(&dst_buf->vb2_buf, 0, 0);
 		dst_buf->flags |= V4L2_BUF_FLAG_LAST;
 		v4l2_m2m_buf_done(dst_buf, VB2_BUF_STATE_DONE);
@@ -1205,9 +1208,12 @@ static void mtk_venc_worker(struct work_struct *work)
 		frm_buf.fb_addr[i].size =
 				(size_t)src_buf->vb2_buf.planes[i].length;
 	}
+	frm_buf.frm_addr = src_buf;
+
 	bs_buf.va = vb2_plane_vaddr(&dst_buf->vb2_buf, 0);
 	bs_buf.dma_addr = vb2_dma_contig_plane_dma_addr(&dst_buf->vb2_buf, 0);
 	bs_buf.size = (size_t)dst_buf->vb2_buf.planes[0].length;
+	bs_buf.buf = dst_buf;
 
 	mtk_v4l2_debug(2,
 			"Framebuf PA=%llx Size=0x%zx;PA=0x%llx Size=0x%zx;PA=0x%llx Size=%zu",
@@ -1233,11 +1239,14 @@ static void mtk_venc_worker(struct work_struct *work)
 		v4l2_m2m_buf_done(dst_buf, VB2_BUF_STATE_ERROR);
 		mtk_v4l2_err("venc_if_encode failed=%d", ret);
 	} else {
-		v4l2_m2m_buf_done(src_buf, VB2_BUF_STATE_DONE);
-		vb2_set_plane_payload(&dst_buf->vb2_buf, 0, enc_result.bs_size);
-		v4l2_m2m_buf_done(dst_buf, VB2_BUF_STATE_DONE);
-		mtk_v4l2_debug(2, "venc_if_encode bs size=%d",
-				 enc_result.bs_size);
+		if (!IS_VENC_MULTICORE(ctx->dev->enc_capability)) {
+			v4l2_m2m_buf_done(src_buf, VB2_BUF_STATE_DONE);
+			vb2_set_plane_payload(&dst_buf->vb2_buf, 0,
+					      enc_result.bs_size);
+			v4l2_m2m_buf_done(dst_buf, VB2_BUF_STATE_DONE);
+			mtk_v4l2_debug(2, "venc_if_encode bs size=%d",
+				       enc_result.bs_size);
+		}
 	}
 
 	ctx->encoded_frame_cnt++;
@@ -1450,6 +1459,34 @@ int mtk_vcodec_enc_queue_init(void *priv, struct vb2_queue *src_vq,
 	return vb2_queue_init(dst_vq);
 }
 
+void mtk_venc_buf_done(struct mtk_vcodec_ctx *ctx, int hw_id,
+		       unsigned int bs_size, bool time_out, bool key_frame)
+{
+	struct vb2_v4l2_buffer *src_vb2_v4l2 = NULL;
+	struct vb2_v4l2_buffer *dst_vb2_v4l2 = NULL;
+
+	/*
+	 * the frm_buf(src_buf) and bs_buf(dst_buf) can be obtained from ctx,
+	 * then put them to done list, user can get them by dqbuf call
+	 */
+	src_vb2_v4l2 = ctx->pfrm_buf[hw_id];
+	dst_vb2_v4l2 = ctx->pbs_buf[hw_id];
+
+	if (src_vb2_v4l2 && dst_vb2_v4l2) {
+		dst_vb2_v4l2->vb2_buf.timestamp =
+			src_vb2_v4l2->vb2_buf.timestamp;
+		dst_vb2_v4l2->timecode = src_vb2_v4l2->timecode;
+
+		if (key_frame)
+			dst_vb2_v4l2->flags |= V4L2_BUF_FLAG_KEYFRAME;
+
+		v4l2_m2m_buf_done(src_vb2_v4l2, VB2_BUF_STATE_DONE);
+		vb2_set_plane_payload(&dst_vb2_v4l2->vb2_buf, 0, bs_size);
+		v4l2_m2m_buf_done(dst_vb2_v4l2, VB2_BUF_STATE_DONE);
+	}
+}
+EXPORT_SYMBOL_GPL(mtk_venc_buf_done);
+
 int mtk_venc_unlock(struct mtk_vcodec_ctx *ctx, int hw_id)
 {
 	struct mtk_vcodec_dev *dev = ctx->dev;
@@ -1457,6 +1494,7 @@ int mtk_venc_unlock(struct mtk_vcodec_ctx *ctx, int hw_id)
 	mutex_unlock(&dev->enc_mutex[hw_id]);
 	return 0;
 }
+EXPORT_SYMBOL_GPL(mtk_venc_unlock);
 
 int mtk_venc_lock(struct mtk_vcodec_ctx *ctx, int hw_id)
 {
@@ -1465,6 +1503,30 @@ int mtk_venc_lock(struct mtk_vcodec_ctx *ctx, int hw_id)
 	mutex_lock(&dev->enc_mutex[hw_id]);
 	return 0;
 }
+EXPORT_SYMBOL_GPL(mtk_venc_lock);
+
+void mtk_venc_lock_all(struct mtk_vcodec_ctx *ctx)
+{
+	unsigned int i;
+	struct mtk_vcodec_dev *dev = ctx->dev;
+
+	/*
+	 * For multi-core mode encoding, there are may be bufs being encoded
+	 * when get the empty flush buffer or stop streaming, for example, the
+	 * buffer with LAST flag will return to client before the encoding
+	 * buffers, which will cause frame lost.
+	 * The encoder device mutex will be locked during encoding process,
+	 * when encode done, the mutex unlocked. So if all encoder device mutex
+	 * can be locked, which means there are no bufs being encoded at this
+	 * time, then the buffer with LAST flag can return to client properly.
+	 */
+
+	for (i = 0; i < MTK_VENC_HW_MAX; i++) {
+		mutex_lock(&dev->enc_mutex[i]);
+		mutex_unlock(&dev->enc_mutex[i]);
+	}
+}
+EXPORT_SYMBOL_GPL(mtk_venc_lock_all);
 
 void mtk_vcodec_enc_release(struct mtk_vcodec_ctx *ctx)
 {
diff --git a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc.h b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc.h
index 29f5c8d1b59f..5ab17381c7ba 100644
--- a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc.h
+++ b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc.h
@@ -20,6 +20,9 @@
 
 #define MTK_VENC_IRQ_STATUS_OFFSET	0x05C
 #define MTK_VENC_IRQ_ACK_OFFSET	0x060
+#define VENC_PIC_BITSTREAM_BYTE_CNT 0x0098
+#define VENC_PIC_FRM_TYPE		0x0010
+#define VENC_PIC_KEY_FRM       0x2
 
 /**
  * struct mtk_video_enc_buf - Private data related to each VB2 buffer.
@@ -46,5 +49,7 @@ int mtk_vcodec_enc_queue_init(void *priv, struct vb2_queue *src_vq,
 void mtk_vcodec_enc_release(struct mtk_vcodec_ctx *ctx);
 int mtk_vcodec_enc_ctrls_setup(struct mtk_vcodec_ctx *ctx);
 void mtk_vcodec_enc_set_default_params(struct mtk_vcodec_ctx *ctx);
-
+void mtk_venc_buf_done(struct mtk_vcodec_ctx *ctx, int hw_id,
+		       unsigned int bs_size, bool time_out, bool key_frame);
+void mtk_venc_lock_all(struct mtk_vcodec_ctx *ctx);
 #endif /* _MTK_VCODEC_ENC_H_ */
diff --git a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_drv.c b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_drv.c
index 1c73bfeb80cb..165e95affe35 100644
--- a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_drv.c
+++ b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_drv.c
@@ -89,6 +89,9 @@ static irqreturn_t mtk_vcodec_enc_irq_handler(int irq, void *priv)
 	struct mtk_vcodec_ctx *ctx;
 	unsigned long flags;
 	void __iomem *addr;
+	unsigned int bs_size;
+	unsigned int frm_type;
+	bool is_key_frame = 0;
 
 	spin_lock_irqsave(&dev->irqlock, flags);
 	ctx = dev->curr_enc_ctx[MTK_VENC_CORE_0];
@@ -101,8 +104,32 @@ static irqreturn_t mtk_vcodec_enc_irq_handler(int irq, void *priv)
 	ctx->irq_status = readl(dev->reg_base[dev->venc_pdata->core_id] +
 				(MTK_VENC_IRQ_STATUS_OFFSET));
 
+	bs_size = readl(dev->reg_base[dev->venc_pdata->core_id] +
+			(VENC_PIC_BITSTREAM_BYTE_CNT));
+	frm_type = readl(dev->reg_base[dev->venc_pdata->core_id] +
+			 (VENC_PIC_FRM_TYPE));
+
 	clean_irq_status(ctx->irq_status, addr);
 
+	if (IS_VENC_MULTICORE(dev->enc_capability)) {
+		if (ctx->irq_status & MTK_VENC_IRQ_STATUS_FRM) {
+			if (ctx->hdr_size != 0) {
+				bs_size += ctx->hdr_size;
+				ctx->hdr_size = 0;
+			}
+
+			if (frm_type & VENC_PIC_KEY_FRM)
+				is_key_frame = 1;
+
+			mtk_venc_buf_done(ctx, 0, bs_size, 0, is_key_frame);
+			mtk_vcodec_enc_clock_off(dev, 0);
+			mtk_venc_unlock(ctx, 0);
+		} else {
+			wake_up_ctx(ctx, MTK_INST_IRQ_RECEIVED, 0);
+		}
+		return IRQ_HANDLED;
+	}
+
 	wake_up_ctx(ctx, MTK_INST_IRQ_RECEIVED, 0);
 	return IRQ_HANDLED;
 }
@@ -286,7 +313,6 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
 	}
 
 	dev->enc_irq = platform_get_irq(pdev, 0);
-	irq_set_status_flags(dev->enc_irq, IRQ_NOAUTOEN);
 	ret = devm_request_irq(&pdev->dev, dev->enc_irq,
 			       mtk_vcodec_enc_irq_handler,
 			       0, pdev->name, dev);
diff --git a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.c b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.c
index 6df5221b742f..0367e59b20a9 100644
--- a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.c
+++ b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.c
@@ -50,6 +50,9 @@ static irqreturn_t mtk_enc_hw_irq_handler(int irq, void *priv)
 	struct mtk_vcodec_ctx *ctx;
 	unsigned long flags;
 	void __iomem *addr;
+	unsigned int bs_size;
+	unsigned int frm_type;
+	bool is_key_frame = 0;
 
 	spin_lock_irqsave(&main_dev->irqlock, flags);
 	ctx = main_dev->curr_enc_ctx[dev->hw_id];
@@ -61,9 +64,17 @@ static irqreturn_t mtk_enc_hw_irq_handler(int irq, void *priv)
 
 	addr = dev->reg_base + MTK_VENC_IRQ_ACK_OFFSET;
 	ctx->irq_status = readl(dev->reg_base + MTK_VENC_IRQ_STATUS_OFFSET);
+	bs_size = readl(dev->reg_base + VENC_PIC_BITSTREAM_BYTE_CNT);
+	frm_type = readl(dev->reg_base + VENC_PIC_FRM_TYPE);
 	clean_hw_irq_status(ctx->irq_status, addr);
 
-	wake_up_ctx(ctx, MTK_INST_IRQ_RECEIVED, 0);
+	if (frm_type & VENC_PIC_KEY_FRM)
+		is_key_frame = 1;
+
+	mtk_venc_buf_done(ctx, dev->hw_id, bs_size, 0, is_key_frame);
+	mtk_vcodec_enc_clock_off(main_dev, dev->hw_id);
+	mtk_venc_unlock(ctx, dev->hw_id);
+
 	return IRQ_HANDLED;
 }
 
diff --git a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_pm.c b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_pm.c
index 2f83aade779a..af2968a8d2e5 100644
--- a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_pm.c
+++ b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_pm.c
@@ -235,3 +235,4 @@ void mtk_vcodec_enc_clock_off(struct mtk_vcodec_dev *dev,
 	for (i = enc_clk->clk_num - 1; i >= 0; i--)
 		clk_disable(enc_clk->clk_info[i].vcodec_clk);
 }
+EXPORT_SYMBOL_GPL(mtk_vcodec_enc_clock_off);
diff --git a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_util.h b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_util.h
index 0033c53d5589..cecf78d6d4c6 100644
--- a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_util.h
+++ b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_util.h
@@ -15,6 +15,7 @@ struct mtk_vcodec_mem {
 	size_t size;
 	void *va;
 	dma_addr_t dma_addr;
+	void *buf;
 };
 
 struct mtk_vcodec_fb {
diff --git a/drivers/media/platform/mediatek/vcodec/venc/venc_h264_if.c b/drivers/media/platform/mediatek/vcodec/venc/venc_h264_if.c
index c114aa7dcd20..3bd5c6f2fcf7 100644
--- a/drivers/media/platform/mediatek/vcodec/venc/venc_h264_if.c
+++ b/drivers/media/platform/mediatek/vcodec/venc/venc_h264_if.c
@@ -22,7 +22,6 @@
 static const char h264_filler_marker[] = {0x0, 0x0, 0x0, 0x1, 0xc};
 
 #define H264_FILLER_MARKER_SIZE ARRAY_SIZE(h264_filler_marker)
-#define VENC_PIC_BITSTREAM_BYTE_CNT 0x0098
 
 /*
  * enum venc_h264_frame_type - h264 encoder output bitstream frame type
@@ -568,6 +567,11 @@ static int h264_encode_frame(struct venc_h264_inst *inst,
 		return ret;
 	}
 
+	if (IS_VENC_MULTICORE(ctx->dev->enc_capability)) {
+		++inst->frm_cnt;
+		return ret;
+	}
+
 	irq_status = h264_enc_wait_venc_done(inst);
 	if (irq_status != MTK_VENC_IRQ_STATUS_FRM) {
 		mtk_vcodec_err(inst, "irq_status=%d failed", irq_status);
@@ -652,8 +656,6 @@ static int h264_enc_encode(void *handle,
 
 	mtk_vcodec_debug(inst, "opt %d ->", opt);
 
-	enable_irq(ctx->dev->enc_irq);
-
 	switch (opt) {
 	case VENC_START_OPT_ENCODE_SEQUENCE_HEADER: {
 		unsigned int bs_size_hdr;
@@ -676,6 +678,13 @@ static int h264_enc_encode(void *handle,
 		unsigned int bs_size_hdr;
 		unsigned int bs_size_frm;
 
+		/*
+		 * the frm_buf and bs_buf need to recorded into current ctx,
+		 * when encoding done, the target buffers can be got from ctx.
+		 */
+		ctx->pfrm_buf[ctx->hw_id] = frm_buf->frm_addr;
+		ctx->pbs_buf[ctx->hw_id] = bs_buf->buf;
+
 		if (!inst->prepend_hdr) {
 			ret = h264_encode_frame(inst, frm_buf, bs_buf,
 						&result->bs_size, ctx->hw_id);
@@ -710,7 +719,9 @@ static int h264_enc_encode(void *handle,
 		if (ret)
 			goto encode_err;
 
-		result->bs_size = hdr_sz + filler_sz + bs_size_frm;
+		ctx->hdr_size = hdr_sz + filler_sz;
+
+		result->bs_size = ctx->hdr_size + bs_size_frm;
 
 		mtk_vcodec_debug(inst, "hdr %d filler %d frame %d bs %d",
 				 hdr_sz, filler_sz, bs_size_frm,
@@ -729,7 +740,6 @@ static int h264_enc_encode(void *handle,
 
 encode_err:
 
-	disable_irq(ctx->dev->enc_irq);
 	mtk_vcodec_debug(inst, "opt %d <-", opt);
 
 	return ret;
diff --git a/drivers/media/platform/mediatek/vcodec/venc_drv_if.h b/drivers/media/platform/mediatek/vcodec/venc_drv_if.h
index e676ccf6bd25..7e24b7f573d7 100644
--- a/drivers/media/platform/mediatek/vcodec/venc_drv_if.h
+++ b/drivers/media/platform/mediatek/vcodec/venc_drv_if.h
@@ -108,9 +108,11 @@ struct venc_frame_info {
 /*
  * struct venc_frm_buf - frame buffer information used in venc_if_encode()
  * @fb_addr: plane frame buffer addresses
+ * @frm_addr: current v4l2 buffer address
  */
 struct venc_frm_buf {
 	struct mtk_vcodec_fb fb_addr[MTK_VCODEC_MAX_PLANES];
+	void *frm_addr;
 };
 
 /*
-- 
2.18.0


^ permalink raw reply related	[flat|nested] 22+ messages in thread
* [PATCH v5, 8/8] media: mediatek: vcodec: Return encoding result in asynchronous mode
@ 2022-07-29  3:51   ` Irui Wang
  0 siblings, 0 replies; 22+ messages in thread
From: Irui Wang @ 2022-07-29  3:51 UTC (permalink / raw)
  To: Hans Verkuil, angelogioacchino.delregno, Rob Herring,
	Mauro Carvalho Chehab, Matthias Brugger, Alexandre Courbot,
	Tiffany Lin, Andrew-CT Chen, Tzung-Bi Shih, Tomasz Figa,
	nicolas.dufresne
  Cc: Maoguang Meng, Longfei Wang, Yunfei Dong, Irui Wang, linux-media,
	devicetree, linux-kernel, linux-arm-kernel, linux-mediatek,
	Project_Global_Chrome_Upstream_Group

when enable multi-core encoding, the wait IRQ done synchronous function
should not be called, so the encoding result can't return to client in
device_run. Move the buffer done function in IRQ handler.

Signed-off-by: Irui Wang <irui.wang@mediatek.com>
---
 .../platform/mediatek/vcodec/mtk_vcodec_drv.h |  6 ++
 .../platform/mediatek/vcodec/mtk_vcodec_enc.c | 72 +++++++++++++++++--
 .../platform/mediatek/vcodec/mtk_vcodec_enc.h |  7 +-
 .../mediatek/vcodec/mtk_vcodec_enc_drv.c      | 28 +++++++-
 .../mediatek/vcodec/mtk_vcodec_enc_hw.c       | 13 +++-
 .../mediatek/vcodec/mtk_vcodec_enc_pm.c       |  1 +
 .../mediatek/vcodec/mtk_vcodec_util.h         |  1 +
 .../mediatek/vcodec/venc/venc_h264_if.c       | 20 ++++--
 .../platform/mediatek/vcodec/venc_drv_if.h    |  2 +
 9 files changed, 137 insertions(+), 13 deletions(-)

diff --git a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_drv.h b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_drv.h
index 7ffca004345c..50166fc68cb8 100644
--- a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_drv.h
+++ b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_drv.h
@@ -290,6 +290,9 @@ struct vdec_pic_info {
  *
  * @msg_queue: msg queue used to store lat buffer information.
  * @encoded_frame_cnt: number of encoded frames
+ * @pfrm_buf: used to store current ctx's frame buffer
+ * @pbs_buf: used to store current ctx's bitstream buffer
+ * @hdr_size: used to store prepend header size
  */
 struct mtk_vcodec_ctx {
 	enum mtk_instance_type type;
@@ -338,6 +341,9 @@ struct mtk_vcodec_ctx {
 	struct vdec_msg_queue msg_queue;
 
 	int encoded_frame_cnt;
+	struct vb2_v4l2_buffer *pfrm_buf[MTK_VENC_HW_MAX];
+	struct vb2_v4l2_buffer *pbs_buf[MTK_VENC_HW_MAX];
+	unsigned int hdr_size;
 };
 
 /*
diff --git a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc.c b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc.c
index a2586485970b..3778d802eed5 100644
--- a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc.c
+++ b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc.c
@@ -956,6 +956,8 @@ static void vb2ops_venc_stop_streaming(struct vb2_queue *q)
 
 	mtk_v4l2_debug(2, "[%d]-> type=%d", ctx->id, q->type);
 
+	mtk_venc_lock_all(ctx);
+
 	if (q->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
 		while ((dst_buf = v4l2_m2m_dst_buf_remove(ctx->m2m_ctx))) {
 			vb2_set_plane_payload(&dst_buf->vb2_buf, 0, 0);
@@ -1191,6 +1193,7 @@ static void mtk_venc_worker(struct work_struct *work)
 	 * is dequeued.
 	 */
 	if (src_buf == &ctx->empty_flush_buf.vb) {
+		mtk_venc_lock_all(ctx);
 		vb2_set_plane_payload(&dst_buf->vb2_buf, 0, 0);
 		dst_buf->flags |= V4L2_BUF_FLAG_LAST;
 		v4l2_m2m_buf_done(dst_buf, VB2_BUF_STATE_DONE);
@@ -1205,9 +1208,12 @@ static void mtk_venc_worker(struct work_struct *work)
 		frm_buf.fb_addr[i].size =
 				(size_t)src_buf->vb2_buf.planes[i].length;
 	}
+	frm_buf.frm_addr = src_buf;
+
 	bs_buf.va = vb2_plane_vaddr(&dst_buf->vb2_buf, 0);
 	bs_buf.dma_addr = vb2_dma_contig_plane_dma_addr(&dst_buf->vb2_buf, 0);
 	bs_buf.size = (size_t)dst_buf->vb2_buf.planes[0].length;
+	bs_buf.buf = dst_buf;
 
 	mtk_v4l2_debug(2,
 			"Framebuf PA=%llx Size=0x%zx;PA=0x%llx Size=0x%zx;PA=0x%llx Size=%zu",
@@ -1233,11 +1239,14 @@ static void mtk_venc_worker(struct work_struct *work)
 		v4l2_m2m_buf_done(dst_buf, VB2_BUF_STATE_ERROR);
 		mtk_v4l2_err("venc_if_encode failed=%d", ret);
 	} else {
-		v4l2_m2m_buf_done(src_buf, VB2_BUF_STATE_DONE);
-		vb2_set_plane_payload(&dst_buf->vb2_buf, 0, enc_result.bs_size);
-		v4l2_m2m_buf_done(dst_buf, VB2_BUF_STATE_DONE);
-		mtk_v4l2_debug(2, "venc_if_encode bs size=%d",
-				 enc_result.bs_size);
+		if (!IS_VENC_MULTICORE(ctx->dev->enc_capability)) {
+			v4l2_m2m_buf_done(src_buf, VB2_BUF_STATE_DONE);
+			vb2_set_plane_payload(&dst_buf->vb2_buf, 0,
+					      enc_result.bs_size);
+			v4l2_m2m_buf_done(dst_buf, VB2_BUF_STATE_DONE);
+			mtk_v4l2_debug(2, "venc_if_encode bs size=%d",
+				       enc_result.bs_size);
+		}
 	}
 
 	ctx->encoded_frame_cnt++;
@@ -1450,6 +1459,34 @@ int mtk_vcodec_enc_queue_init(void *priv, struct vb2_queue *src_vq,
 	return vb2_queue_init(dst_vq);
 }
 
+void mtk_venc_buf_done(struct mtk_vcodec_ctx *ctx, int hw_id,
+		       unsigned int bs_size, bool time_out, bool key_frame)
+{
+	struct vb2_v4l2_buffer *src_vb2_v4l2 = NULL;
+	struct vb2_v4l2_buffer *dst_vb2_v4l2 = NULL;
+
+	/*
+	 * the frm_buf(src_buf) and bs_buf(dst_buf) can be obtained from ctx,
+	 * then put them to done list, user can get them by dqbuf call
+	 */
+	src_vb2_v4l2 = ctx->pfrm_buf[hw_id];
+	dst_vb2_v4l2 = ctx->pbs_buf[hw_id];
+
+	if (src_vb2_v4l2 && dst_vb2_v4l2) {
+		dst_vb2_v4l2->vb2_buf.timestamp =
+			src_vb2_v4l2->vb2_buf.timestamp;
+		dst_vb2_v4l2->timecode = src_vb2_v4l2->timecode;
+
+		if (key_frame)
+			dst_vb2_v4l2->flags |= V4L2_BUF_FLAG_KEYFRAME;
+
+		v4l2_m2m_buf_done(src_vb2_v4l2, VB2_BUF_STATE_DONE);
+		vb2_set_plane_payload(&dst_vb2_v4l2->vb2_buf, 0, bs_size);
+		v4l2_m2m_buf_done(dst_vb2_v4l2, VB2_BUF_STATE_DONE);
+	}
+}
+EXPORT_SYMBOL_GPL(mtk_venc_buf_done);
+
 int mtk_venc_unlock(struct mtk_vcodec_ctx *ctx, int hw_id)
 {
 	struct mtk_vcodec_dev *dev = ctx->dev;
@@ -1457,6 +1494,7 @@ int mtk_venc_unlock(struct mtk_vcodec_ctx *ctx, int hw_id)
 	mutex_unlock(&dev->enc_mutex[hw_id]);
 	return 0;
 }
+EXPORT_SYMBOL_GPL(mtk_venc_unlock);
 
 int mtk_venc_lock(struct mtk_vcodec_ctx *ctx, int hw_id)
 {
@@ -1465,6 +1503,30 @@ int mtk_venc_lock(struct mtk_vcodec_ctx *ctx, int hw_id)
 	mutex_lock(&dev->enc_mutex[hw_id]);
 	return 0;
 }
+EXPORT_SYMBOL_GPL(mtk_venc_lock);
+
+void mtk_venc_lock_all(struct mtk_vcodec_ctx *ctx)
+{
+	unsigned int i;
+	struct mtk_vcodec_dev *dev = ctx->dev;
+
+	/*
+	 * For multi-core mode encoding, there are may be bufs being encoded
+	 * when get the empty flush buffer or stop streaming, for example, the
+	 * buffer with LAST flag will return to client before the encoding
+	 * buffers, which will cause frame lost.
+	 * The encoder device mutex will be locked during encoding process,
+	 * when encode done, the mutex unlocked. So if all encoder device mutex
+	 * can be locked, which means there are no bufs being encoded at this
+	 * time, then the buffer with LAST flag can return to client properly.
+	 */
+
+	for (i = 0; i < MTK_VENC_HW_MAX; i++) {
+		mutex_lock(&dev->enc_mutex[i]);
+		mutex_unlock(&dev->enc_mutex[i]);
+	}
+}
+EXPORT_SYMBOL_GPL(mtk_venc_lock_all);
 
 void mtk_vcodec_enc_release(struct mtk_vcodec_ctx *ctx)
 {
diff --git a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc.h b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc.h
index 29f5c8d1b59f..5ab17381c7ba 100644
--- a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc.h
+++ b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc.h
@@ -20,6 +20,9 @@
 
 #define MTK_VENC_IRQ_STATUS_OFFSET	0x05C
 #define MTK_VENC_IRQ_ACK_OFFSET	0x060
+#define VENC_PIC_BITSTREAM_BYTE_CNT 0x0098
+#define VENC_PIC_FRM_TYPE		0x0010
+#define VENC_PIC_KEY_FRM       0x2
 
 /**
  * struct mtk_video_enc_buf - Private data related to each VB2 buffer.
@@ -46,5 +49,7 @@ int mtk_vcodec_enc_queue_init(void *priv, struct vb2_queue *src_vq,
 void mtk_vcodec_enc_release(struct mtk_vcodec_ctx *ctx);
 int mtk_vcodec_enc_ctrls_setup(struct mtk_vcodec_ctx *ctx);
 void mtk_vcodec_enc_set_default_params(struct mtk_vcodec_ctx *ctx);
-
+void mtk_venc_buf_done(struct mtk_vcodec_ctx *ctx, int hw_id,
+		       unsigned int bs_size, bool time_out, bool key_frame);
+void mtk_venc_lock_all(struct mtk_vcodec_ctx *ctx);
 #endif /* _MTK_VCODEC_ENC_H_ */
diff --git a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_drv.c b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_drv.c
index 1c73bfeb80cb..165e95affe35 100644
--- a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_drv.c
+++ b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_drv.c
@@ -89,6 +89,9 @@ static irqreturn_t mtk_vcodec_enc_irq_handler(int irq, void *priv)
 	struct mtk_vcodec_ctx *ctx;
 	unsigned long flags;
 	void __iomem *addr;
+	unsigned int bs_size;
+	unsigned int frm_type;
+	bool is_key_frame = 0;
 
 	spin_lock_irqsave(&dev->irqlock, flags);
 	ctx = dev->curr_enc_ctx[MTK_VENC_CORE_0];
@@ -101,8 +104,32 @@ static irqreturn_t mtk_vcodec_enc_irq_handler(int irq, void *priv)
 	ctx->irq_status = readl(dev->reg_base[dev->venc_pdata->core_id] +
 				(MTK_VENC_IRQ_STATUS_OFFSET));
 
+	bs_size = readl(dev->reg_base[dev->venc_pdata->core_id] +
+			(VENC_PIC_BITSTREAM_BYTE_CNT));
+	frm_type = readl(dev->reg_base[dev->venc_pdata->core_id] +
+			 (VENC_PIC_FRM_TYPE));
+
 	clean_irq_status(ctx->irq_status, addr);
 
+	if (IS_VENC_MULTICORE(dev->enc_capability)) {
+		if (ctx->irq_status & MTK_VENC_IRQ_STATUS_FRM) {
+			if (ctx->hdr_size != 0) {
+				bs_size += ctx->hdr_size;
+				ctx->hdr_size = 0;
+			}
+
+			if (frm_type & VENC_PIC_KEY_FRM)
+				is_key_frame = 1;
+
+			mtk_venc_buf_done(ctx, 0, bs_size, 0, is_key_frame);
+			mtk_vcodec_enc_clock_off(dev, 0);
+			mtk_venc_unlock(ctx, 0);
+		} else {
+			wake_up_ctx(ctx, MTK_INST_IRQ_RECEIVED, 0);
+		}
+		return IRQ_HANDLED;
+	}
+
 	wake_up_ctx(ctx, MTK_INST_IRQ_RECEIVED, 0);
 	return IRQ_HANDLED;
 }
@@ -286,7 +313,6 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
 	}
 
 	dev->enc_irq = platform_get_irq(pdev, 0);
-	irq_set_status_flags(dev->enc_irq, IRQ_NOAUTOEN);
 	ret = devm_request_irq(&pdev->dev, dev->enc_irq,
 			       mtk_vcodec_enc_irq_handler,
 			       0, pdev->name, dev);
diff --git a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.c b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.c
index 6df5221b742f..0367e59b20a9 100644
--- a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.c
+++ b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.c
@@ -50,6 +50,9 @@ static irqreturn_t mtk_enc_hw_irq_handler(int irq, void *priv)
 	struct mtk_vcodec_ctx *ctx;
 	unsigned long flags;
 	void __iomem *addr;
+	unsigned int bs_size;
+	unsigned int frm_type;
+	bool is_key_frame = 0;
 
 	spin_lock_irqsave(&main_dev->irqlock, flags);
 	ctx = main_dev->curr_enc_ctx[dev->hw_id];
@@ -61,9 +64,17 @@ static irqreturn_t mtk_enc_hw_irq_handler(int irq, void *priv)
 
 	addr = dev->reg_base + MTK_VENC_IRQ_ACK_OFFSET;
 	ctx->irq_status = readl(dev->reg_base + MTK_VENC_IRQ_STATUS_OFFSET);
+	bs_size = readl(dev->reg_base + VENC_PIC_BITSTREAM_BYTE_CNT);
+	frm_type = readl(dev->reg_base + VENC_PIC_FRM_TYPE);
 	clean_hw_irq_status(ctx->irq_status, addr);
 
-	wake_up_ctx(ctx, MTK_INST_IRQ_RECEIVED, 0);
+	if (frm_type & VENC_PIC_KEY_FRM)
+		is_key_frame = 1;
+
+	mtk_venc_buf_done(ctx, dev->hw_id, bs_size, 0, is_key_frame);
+	mtk_vcodec_enc_clock_off(main_dev, dev->hw_id);
+	mtk_venc_unlock(ctx, dev->hw_id);
+
 	return IRQ_HANDLED;
 }
 
diff --git a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_pm.c b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_pm.c
index 2f83aade779a..af2968a8d2e5 100644
--- a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_pm.c
+++ b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_pm.c
@@ -235,3 +235,4 @@ void mtk_vcodec_enc_clock_off(struct mtk_vcodec_dev *dev,
 	for (i = enc_clk->clk_num - 1; i >= 0; i--)
 		clk_disable(enc_clk->clk_info[i].vcodec_clk);
 }
+EXPORT_SYMBOL_GPL(mtk_vcodec_enc_clock_off);
diff --git a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_util.h b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_util.h
index 0033c53d5589..cecf78d6d4c6 100644
--- a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_util.h
+++ b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_util.h
@@ -15,6 +15,7 @@ struct mtk_vcodec_mem {
 	size_t size;
 	void *va;
 	dma_addr_t dma_addr;
+	void *buf;
 };
 
 struct mtk_vcodec_fb {
diff --git a/drivers/media/platform/mediatek/vcodec/venc/venc_h264_if.c b/drivers/media/platform/mediatek/vcodec/venc/venc_h264_if.c
index c114aa7dcd20..3bd5c6f2fcf7 100644
--- a/drivers/media/platform/mediatek/vcodec/venc/venc_h264_if.c
+++ b/drivers/media/platform/mediatek/vcodec/venc/venc_h264_if.c
@@ -22,7 +22,6 @@
 static const char h264_filler_marker[] = {0x0, 0x0, 0x0, 0x1, 0xc};
 
 #define H264_FILLER_MARKER_SIZE ARRAY_SIZE(h264_filler_marker)
-#define VENC_PIC_BITSTREAM_BYTE_CNT 0x0098
 
 /*
  * enum venc_h264_frame_type - h264 encoder output bitstream frame type
@@ -568,6 +567,11 @@ static int h264_encode_frame(struct venc_h264_inst *inst,
 		return ret;
 	}
 
+	if (IS_VENC_MULTICORE(ctx->dev->enc_capability)) {
+		++inst->frm_cnt;
+		return ret;
+	}
+
 	irq_status = h264_enc_wait_venc_done(inst);
 	if (irq_status != MTK_VENC_IRQ_STATUS_FRM) {
 		mtk_vcodec_err(inst, "irq_status=%d failed", irq_status);
@@ -652,8 +656,6 @@ static int h264_enc_encode(void *handle,
 
 	mtk_vcodec_debug(inst, "opt %d ->", opt);
 
-	enable_irq(ctx->dev->enc_irq);
-
 	switch (opt) {
 	case VENC_START_OPT_ENCODE_SEQUENCE_HEADER: {
 		unsigned int bs_size_hdr;
@@ -676,6 +678,13 @@ static int h264_enc_encode(void *handle,
 		unsigned int bs_size_hdr;
 		unsigned int bs_size_frm;
 
+		/*
+		 * the frm_buf and bs_buf need to recorded into current ctx,
+		 * when encoding done, the target buffers can be got from ctx.
+		 */
+		ctx->pfrm_buf[ctx->hw_id] = frm_buf->frm_addr;
+		ctx->pbs_buf[ctx->hw_id] = bs_buf->buf;
+
 		if (!inst->prepend_hdr) {
 			ret = h264_encode_frame(inst, frm_buf, bs_buf,
 						&result->bs_size, ctx->hw_id);
@@ -710,7 +719,9 @@ static int h264_enc_encode(void *handle,
 		if (ret)
 			goto encode_err;
 
-		result->bs_size = hdr_sz + filler_sz + bs_size_frm;
+		ctx->hdr_size = hdr_sz + filler_sz;
+
+		result->bs_size = ctx->hdr_size + bs_size_frm;
 
 		mtk_vcodec_debug(inst, "hdr %d filler %d frame %d bs %d",
 				 hdr_sz, filler_sz, bs_size_frm,
@@ -729,7 +740,6 @@ static int h264_enc_encode(void *handle,
 
 encode_err:
 
-	disable_irq(ctx->dev->enc_irq);
 	mtk_vcodec_debug(inst, "opt %d <-", opt);
 
 	return ret;
diff --git a/drivers/media/platform/mediatek/vcodec/venc_drv_if.h b/drivers/media/platform/mediatek/vcodec/venc_drv_if.h
index e676ccf6bd25..7e24b7f573d7 100644
--- a/drivers/media/platform/mediatek/vcodec/venc_drv_if.h
+++ b/drivers/media/platform/mediatek/vcodec/venc_drv_if.h
@@ -108,9 +108,11 @@ struct venc_frame_info {
 /*
  * struct venc_frm_buf - frame buffer information used in venc_if_encode()
  * @fb_addr: plane frame buffer addresses
+ * @frm_addr: current v4l2 buffer address
  */
 struct venc_frm_buf {
 	struct mtk_vcodec_fb fb_addr[MTK_VCODEC_MAX_PLANES];
+	void *frm_addr;
 };
 
 /*
-- 
2.18.0


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply related	[flat|nested] 22+ messages in thread
* Re: [PATCH v5, 0/8] Support H264 multi-core encoder on MT8195
  2022-07-29  3:51 ` Irui Wang
@ 2022-07-29 15:08   ` Nicolas Dufresne
  -1 siblings, 0 replies; 22+ messages in thread
From: Nicolas Dufresne @ 2022-07-29 15:08 UTC (permalink / raw)
  To: Irui Wang, Hans Verkuil, angelogioacchino.delregno, Rob Herring,
	Mauro Carvalho Chehab, Matthias Brugger, Alexandre Courbot,
	Tiffany Lin, Andrew-CT Chen, Tzung-Bi Shih, Tomasz Figa
  Cc: Maoguang Meng, Longfei Wang, Yunfei Dong, linux-media,
	devicetree, linux-kernel, linux-arm-kernel, linux-mediatek,
	Project_Global_Chrome_Upstream_Group

Hi Irui,

Le vendredi 29 juillet 2022 à 11:51 +0800, Irui Wang a écrit :
> MT8195 has two H264 encoder hardware, named core0 and core1, this two
> cores can encode two input frames separately at the same time to achieve
> higher performance.

I suspect you could provide some performance numbers on that specific HW before
and after your change ? This could help integrator regarding what type of gain
they can expect of back-porting this feature. This is not strictly mandatory
though.

> 
> This series of patches are used to enable the two H264 encoder cores,
> the difference between encoding process before and after enable two
> cores is just like as below:
> As-Is: Synchronous
> V4L2_VIDIOC_QBUF#0 --> device_run(triger encoder) --> wait encoder IRQ
> -->
> encoding done with result --> job_finish
> V4l2_VIDIOC_QBUF#1 --> device_run(triger encoder) --> wait encoder IRQ
> -->
> encoding done with result --> job_finish
> ...
> To-Be: Asynchronous
> V4L2_VIDIOC_QBUF#0 --> device_run(triger encoder) --> job_finish
> ..V4l2_VIDIOC_QBUF#1 --> device_run(triger encoder) --> job_finish
> (venc core0 may encode done here, done the encoding result to client)
> V4L2_VIDIOC_QBUF#2 --> device_run(triger encoder) --> job_finish.
> 
> ---
> changes compared with v4:
> - reabse to the newer linux media stage tree.
> - remove "mediatek,venc-multi-core" property since sub-device can
>   be probed by of_platform_populate api directly.
> - some modifications for patch v4's review comments.
> 
> changes compared with v3:
> - rebase to the newer linux media stage.
> - add a capability to indicate scp firmware support multi-core.
> - probe core0 as main device, core1 as sub-device.
> 
> changes compared with v2:
> - update venc core dt-bindings, add two new properties for current
>   usage.
> - parse venc multi_core mode from device tree.
> - rebase to the newer linux media stage.
> 
> changes compared with v1:
> - of_platform_populate was used in place of the component framework.
> - new yaml file for venc cores.
> - some modifications for patch v1's review comments.
> ---
> 
> Irui Wang (8):
>   dt-bindings: media: mediatek: vcodec: Adds encoder cores dt-bindings
>     for mt8195
>   media: mediatek: vcodec: Enable venc dual core usage
>   media: mediatek: vcodec: Refactor venc power manage function
>   media: mediatek: vcodec: Add more extra processing for multi-core
>     encoding
>   media: mediatek: vcodec: Add venc power on/off function
>   media: mediatek: vcodec: Refactor encoder clock on/off function
>   media: mediatek: vcodec: Add multi-core encoding process
>   media: mediatek: vcodec: Return encoding result in asynchronous mode
> 
>  .../media/mediatek,vcodec-encoder-core.yaml   | 218 ++++++++++++++++
>  .../media/mediatek,vcodec-encoder.yaml        |   1 -
>  .../media/platform/mediatek/vcodec/Makefile   |   4 +-
>  .../platform/mediatek/vcodec/mtk_vcodec_drv.h |  28 +-
>  .../platform/mediatek/vcodec/mtk_vcodec_enc.c | 113 ++++++--
>  .../platform/mediatek/vcodec/mtk_vcodec_enc.h |  11 +-
>  .../mediatek/vcodec/mtk_vcodec_enc_drv.c      |  44 +++-
>  .../mediatek/vcodec/mtk_vcodec_enc_hw.c       | 156 +++++++++++
>  .../mediatek/vcodec/mtk_vcodec_enc_hw.h       |  34 +++
>  .../mediatek/vcodec/mtk_vcodec_enc_pm.c       | 178 +++++++++++--
>  .../mediatek/vcodec/mtk_vcodec_enc_pm.h       |  11 +-
>  .../mediatek/vcodec/mtk_vcodec_util.c         |  19 ++
>  .../mediatek/vcodec/mtk_vcodec_util.h         |   3 +
>  .../mediatek/vcodec/venc/venc_h264_if.c       | 243 ++++++++++++++----
>  .../mediatek/vcodec/venc/venc_vp8_if.c        |   3 +-
>  .../platform/mediatek/vcodec/venc_drv_if.c    |  75 ++++--
>  .../platform/mediatek/vcodec/venc_drv_if.h    |   6 +
>  .../platform/mediatek/vcodec/venc_vpu_if.c    |   9 +-
>  .../platform/mediatek/vcodec/venc_vpu_if.h    |   3 +-
>  19 files changed, 1028 insertions(+), 131 deletions(-)
>  create mode 100644 Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml
>  create mode 100644 drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.c
>  create mode 100644 drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.h
> 


^ permalink raw reply	[flat|nested] 22+ messages in thread
* Re: [PATCH v5, 0/8] Support H264 multi-core encoder on MT8195
@ 2022-07-29 15:08   ` Nicolas Dufresne
  0 siblings, 0 replies; 22+ messages in thread
From: Nicolas Dufresne @ 2022-07-29 15:08 UTC (permalink / raw)
  To: Irui Wang, Hans Verkuil, angelogioacchino.delregno, Rob Herring,
	Mauro Carvalho Chehab, Matthias Brugger, Alexandre Courbot,
	Tiffany Lin, Andrew-CT Chen, Tzung-Bi Shih, Tomasz Figa
  Cc: Maoguang Meng, Longfei Wang, Yunfei Dong, linux-media,
	devicetree, linux-kernel, linux-arm-kernel, linux-mediatek,
	Project_Global_Chrome_Upstream_Group

Hi Irui,

Le vendredi 29 juillet 2022 à 11:51 +0800, Irui Wang a écrit :
> MT8195 has two H264 encoder hardware, named core0 and core1, this two
> cores can encode two input frames separately at the same time to achieve
> higher performance.

I suspect you could provide some performance numbers on that specific HW before
and after your change ? This could help integrator regarding what type of gain
they can expect of back-porting this feature. This is not strictly mandatory
though.

> 
> This series of patches are used to enable the two H264 encoder cores,
> the difference between encoding process before and after enable two
> cores is just like as below:
> As-Is: Synchronous
> V4L2_VIDIOC_QBUF#0 --> device_run(triger encoder) --> wait encoder IRQ
> -->
> encoding done with result --> job_finish
> V4l2_VIDIOC_QBUF#1 --> device_run(triger encoder) --> wait encoder IRQ
> -->
> encoding done with result --> job_finish
> ...
> To-Be: Asynchronous
> V4L2_VIDIOC_QBUF#0 --> device_run(triger encoder) --> job_finish
> ..V4l2_VIDIOC_QBUF#1 --> device_run(triger encoder) --> job_finish
> (venc core0 may encode done here, done the encoding result to client)
> V4L2_VIDIOC_QBUF#2 --> device_run(triger encoder) --> job_finish.
> 
> ---
> changes compared with v4:
> - reabse to the newer linux media stage tree.
> - remove "mediatek,venc-multi-core" property since sub-device can
>   be probed by of_platform_populate api directly.
> - some modifications for patch v4's review comments.
> 
> changes compared with v3:
> - rebase to the newer linux media stage.
> - add a capability to indicate scp firmware support multi-core.
> - probe core0 as main device, core1 as sub-device.
> 
> changes compared with v2:
> - update venc core dt-bindings, add two new properties for current
>   usage.
> - parse venc multi_core mode from device tree.
> - rebase to the newer linux media stage.
> 
> changes compared with v1:
> - of_platform_populate was used in place of the component framework.
> - new yaml file for venc cores.
> - some modifications for patch v1's review comments.
> ---
> 
> Irui Wang (8):
>   dt-bindings: media: mediatek: vcodec: Adds encoder cores dt-bindings
>     for mt8195
>   media: mediatek: vcodec: Enable venc dual core usage
>   media: mediatek: vcodec: Refactor venc power manage function
>   media: mediatek: vcodec: Add more extra processing for multi-core
>     encoding
>   media: mediatek: vcodec: Add venc power on/off function
>   media: mediatek: vcodec: Refactor encoder clock on/off function
>   media: mediatek: vcodec: Add multi-core encoding process
>   media: mediatek: vcodec: Return encoding result in asynchronous mode
> 
>  .../media/mediatek,vcodec-encoder-core.yaml   | 218 ++++++++++++++++
>  .../media/mediatek,vcodec-encoder.yaml        |   1 -
>  .../media/platform/mediatek/vcodec/Makefile   |   4 +-
>  .../platform/mediatek/vcodec/mtk_vcodec_drv.h |  28 +-
>  .../platform/mediatek/vcodec/mtk_vcodec_enc.c | 113 ++++++--
>  .../platform/mediatek/vcodec/mtk_vcodec_enc.h |  11 +-
>  .../mediatek/vcodec/mtk_vcodec_enc_drv.c      |  44 +++-
>  .../mediatek/vcodec/mtk_vcodec_enc_hw.c       | 156 +++++++++++
>  .../mediatek/vcodec/mtk_vcodec_enc_hw.h       |  34 +++
>  .../mediatek/vcodec/mtk_vcodec_enc_pm.c       | 178 +++++++++++--
>  .../mediatek/vcodec/mtk_vcodec_enc_pm.h       |  11 +-
>  .../mediatek/vcodec/mtk_vcodec_util.c         |  19 ++
>  .../mediatek/vcodec/mtk_vcodec_util.h         |   3 +
>  .../mediatek/vcodec/venc/venc_h264_if.c       | 243 ++++++++++++++----
>  .../mediatek/vcodec/venc/venc_vp8_if.c        |   3 +-
>  .../platform/mediatek/vcodec/venc_drv_if.c    |  75 ++++--
>  .../platform/mediatek/vcodec/venc_drv_if.h    |   6 +
>  .../platform/mediatek/vcodec/venc_vpu_if.c    |   9 +-
>  .../platform/mediatek/vcodec/venc_vpu_if.h    |   3 +-
>  19 files changed, 1028 insertions(+), 131 deletions(-)
>  create mode 100644 Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml
>  create mode 100644 drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.c
>  create mode 100644 drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.h
> 


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 22+ messages in thread
* Re: [PATCH v5, 0/8] Support H264 multi-core encoder on MT8195
  2022-07-29 15:08   ` Nicolas Dufresne
@ 2022-08-09  2:14     ` Irui Wang
  -1 siblings, 0 replies; 22+ messages in thread
From: Irui Wang @ 2022-08-09  2:14 UTC (permalink / raw)
  To: Nicolas Dufresne, Hans Verkuil, angelogioacchino.delregno,
	Rob Herring, Mauro Carvalho Chehab, Matthias Brugger,
	Alexandre Courbot, Tiffany Lin, Andrew-CT Chen, Tzung-Bi Shih,
	Tomasz Figa
  Cc: Maoguang Meng, Longfei Wang, Yunfei Dong, linux-media,
	devicetree, linux-kernel, linux-arm-kernel, linux-mediatek,
	Project_Global_Chrome_Upstream_Group

Dear Nicolas,

Sorry for late, we have tested this change by chrome tast test before:

tast run $IP video.PlatformEncoding.v4l2_h264*

here are the average fps results:
Before:
180p: 1913
180_meet: 1935

360p: 554
360_meet: 571

720p: 159
720_meet: 189

After:
180p: 2236
180_meet: 2337

360p: 820
360_meet: 830

720p: 324
720_meet: 390

Thanks
BRs
On Fri, 2022-07-29 at 11:08 -0400, Nicolas Dufresne wrote:
> Hi Irui,
> 
> Le vendredi 29 juillet 2022 à 11:51 +0800, Irui Wang a écrit :
> > MT8195 has two H264 encoder hardware, named core0 and core1, this
> > two
> > cores can encode two input frames separately at the same time to
> > achieve
> > higher performance.
> 
> I suspect you could provide some performance numbers on that specific
> HW before
> and after your change ? This could help integrator regarding what
> type of gain
> they can expect of back-porting this feature. This is not strictly
> mandatory
> though.
> 
> > 
> > This series of patches are used to enable the two H264 encoder
> > cores,
> > the difference between encoding process before and after enable two
> > cores is just like as below:
> > As-Is: Synchronous
> > V4L2_VIDIOC_QBUF#0 --> device_run(triger encoder) --> wait encoder
> > IRQ
> > -->
> > encoding done with result --> job_finish
> > V4l2_VIDIOC_QBUF#1 --> device_run(triger encoder) --> wait encoder
> > IRQ
> > -->
> > encoding done with result --> job_finish
> > ...
> > To-Be: Asynchronous
> > V4L2_VIDIOC_QBUF#0 --> device_run(triger encoder) --> job_finish
> > ..V4l2_VIDIOC_QBUF#1 --> device_run(triger encoder) --> job_finish
> > (venc core0 may encode done here, done the encoding result to
> > client)
> > V4L2_VIDIOC_QBUF#2 --> device_run(triger encoder) --> job_finish.
> > 
> > ---
> > changes compared with v4:
> > - reabse to the newer linux media stage tree.
> > - remove "mediatek,venc-multi-core" property since sub-device can
> >   be probed by of_platform_populate api directly.
> > - some modifications for patch v4's review comments.
> > 
> > changes compared with v3:
> > - rebase to the newer linux media stage.
> > - add a capability to indicate scp firmware support multi-core.
> > - probe core0 as main device, core1 as sub-device.
> > 
> > changes compared with v2:
> > - update venc core dt-bindings, add two new properties for current
> >   usage.
> > - parse venc multi_core mode from device tree.
> > - rebase to the newer linux media stage.
> > 
> > changes compared with v1:
> > - of_platform_populate was used in place of the component
> > framework.
> > - new yaml file for venc cores.
> > - some modifications for patch v1's review comments.
> > ---
> > 
> > Irui Wang (8):
> >   dt-bindings: media: mediatek: vcodec: Adds encoder cores dt-
> > bindings
> >     for mt8195
> >   media: mediatek: vcodec: Enable venc dual core usage
> >   media: mediatek: vcodec: Refactor venc power manage function
> >   media: mediatek: vcodec: Add more extra processing for multi-core
> >     encoding
> >   media: mediatek: vcodec: Add venc power on/off function
> >   media: mediatek: vcodec: Refactor encoder clock on/off function
> >   media: mediatek: vcodec: Add multi-core encoding process
> >   media: mediatek: vcodec: Return encoding result in asynchronous
> > mode
> > 
> >  .../media/mediatek,vcodec-encoder-core.yaml   | 218
> > ++++++++++++++++
> >  .../media/mediatek,vcodec-encoder.yaml        |   1 -
> >  .../media/platform/mediatek/vcodec/Makefile   |   4 +-
> >  .../platform/mediatek/vcodec/mtk_vcodec_drv.h |  28 +-
> >  .../platform/mediatek/vcodec/mtk_vcodec_enc.c | 113 ++++++--
> >  .../platform/mediatek/vcodec/mtk_vcodec_enc.h |  11 +-
> >  .../mediatek/vcodec/mtk_vcodec_enc_drv.c      |  44 +++-
> >  .../mediatek/vcodec/mtk_vcodec_enc_hw.c       | 156 +++++++++++
> >  .../mediatek/vcodec/mtk_vcodec_enc_hw.h       |  34 +++
> >  .../mediatek/vcodec/mtk_vcodec_enc_pm.c       | 178 +++++++++++--
> >  .../mediatek/vcodec/mtk_vcodec_enc_pm.h       |  11 +-
> >  .../mediatek/vcodec/mtk_vcodec_util.c         |  19 ++
> >  .../mediatek/vcodec/mtk_vcodec_util.h         |   3 +
> >  .../mediatek/vcodec/venc/venc_h264_if.c       | 243
> > ++++++++++++++----
> >  .../mediatek/vcodec/venc/venc_vp8_if.c        |   3 +-
> >  .../platform/mediatek/vcodec/venc_drv_if.c    |  75 ++++--
> >  .../platform/mediatek/vcodec/venc_drv_if.h    |   6 +
> >  .../platform/mediatek/vcodec/venc_vpu_if.c    |   9 +-
> >  .../platform/mediatek/vcodec/venc_vpu_if.h    |   3 +-
> >  19 files changed, 1028 insertions(+), 131 deletions(-)
> >  create mode 100644
> > Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-
> > core.yaml
> >  create mode 100644
> > drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.c
> >  create mode 100644
> > drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.h
> > 
> 
> 


^ permalink raw reply	[flat|nested] 22+ messages in thread
* Re: [PATCH v5, 0/8] Support H264 multi-core encoder on MT8195
@ 2022-08-09  2:14     ` Irui Wang
  0 siblings, 0 replies; 22+ messages in thread
From: Irui Wang @ 2022-08-09  2:14 UTC (permalink / raw)
  To: Nicolas Dufresne, Hans Verkuil, angelogioacchino.delregno,
	Rob Herring, Mauro Carvalho Chehab, Matthias Brugger,
	Alexandre Courbot, Tiffany Lin, Andrew-CT Chen, Tzung-Bi Shih,
	Tomasz Figa
  Cc: Maoguang Meng, Longfei Wang, Yunfei Dong, linux-media,
	devicetree, linux-kernel, linux-arm-kernel, linux-mediatek,
	Project_Global_Chrome_Upstream_Group

Dear Nicolas,

Sorry for late, we have tested this change by chrome tast test before:

tast run $IP video.PlatformEncoding.v4l2_h264*

here are the average fps results:
Before:
180p: 1913
180_meet: 1935

360p: 554
360_meet: 571

720p: 159
720_meet: 189

After:
180p: 2236
180_meet: 2337

360p: 820
360_meet: 830

720p: 324
720_meet: 390

Thanks
BRs
On Fri, 2022-07-29 at 11:08 -0400, Nicolas Dufresne wrote:
> Hi Irui,
> 
> Le vendredi 29 juillet 2022 à 11:51 +0800, Irui Wang a écrit :
> > MT8195 has two H264 encoder hardware, named core0 and core1, this
> > two
> > cores can encode two input frames separately at the same time to
> > achieve
> > higher performance.
> 
> I suspect you could provide some performance numbers on that specific
> HW before
> and after your change ? This could help integrator regarding what
> type of gain
> they can expect of back-porting this feature. This is not strictly
> mandatory
> though.
> 
> > 
> > This series of patches are used to enable the two H264 encoder
> > cores,
> > the difference between encoding process before and after enable two
> > cores is just like as below:
> > As-Is: Synchronous
> > V4L2_VIDIOC_QBUF#0 --> device_run(triger encoder) --> wait encoder
> > IRQ
> > -->
> > encoding done with result --> job_finish
> > V4l2_VIDIOC_QBUF#1 --> device_run(triger encoder) --> wait encoder
> > IRQ
> > -->
> > encoding done with result --> job_finish
> > ...
> > To-Be: Asynchronous
> > V4L2_VIDIOC_QBUF#0 --> device_run(triger encoder) --> job_finish
> > ..V4l2_VIDIOC_QBUF#1 --> device_run(triger encoder) --> job_finish
> > (venc core0 may encode done here, done the encoding result to
> > client)
> > V4L2_VIDIOC_QBUF#2 --> device_run(triger encoder) --> job_finish.
> > 
> > ---
> > changes compared with v4:
> > - reabse to the newer linux media stage tree.
> > - remove "mediatek,venc-multi-core" property since sub-device can
> >   be probed by of_platform_populate api directly.
> > - some modifications for patch v4's review comments.
> > 
> > changes compared with v3:
> > - rebase to the newer linux media stage.
> > - add a capability to indicate scp firmware support multi-core.
> > - probe core0 as main device, core1 as sub-device.
> > 
> > changes compared with v2:
> > - update venc core dt-bindings, add two new properties for current
> >   usage.
> > - parse venc multi_core mode from device tree.
> > - rebase to the newer linux media stage.
> > 
> > changes compared with v1:
> > - of_platform_populate was used in place of the component
> > framework.
> > - new yaml file for venc cores.
> > - some modifications for patch v1's review comments.
> > ---
> > 
> > Irui Wang (8):
> >   dt-bindings: media: mediatek: vcodec: Adds encoder cores dt-
> > bindings
> >     for mt8195
> >   media: mediatek: vcodec: Enable venc dual core usage
> >   media: mediatek: vcodec: Refactor venc power manage function
> >   media: mediatek: vcodec: Add more extra processing for multi-core
> >     encoding
> >   media: mediatek: vcodec: Add venc power on/off function
> >   media: mediatek: vcodec: Refactor encoder clock on/off function
> >   media: mediatek: vcodec: Add multi-core encoding process
> >   media: mediatek: vcodec: Return encoding result in asynchronous
> > mode
> > 
> >  .../media/mediatek,vcodec-encoder-core.yaml   | 218
> > ++++++++++++++++
> >  .../media/mediatek,vcodec-encoder.yaml        |   1 -
> >  .../media/platform/mediatek/vcodec/Makefile   |   4 +-
> >  .../platform/mediatek/vcodec/mtk_vcodec_drv.h |  28 +-
> >  .../platform/mediatek/vcodec/mtk_vcodec_enc.c | 113 ++++++--
> >  .../platform/mediatek/vcodec/mtk_vcodec_enc.h |  11 +-
> >  .../mediatek/vcodec/mtk_vcodec_enc_drv.c      |  44 +++-
> >  .../mediatek/vcodec/mtk_vcodec_enc_hw.c       | 156 +++++++++++
> >  .../mediatek/vcodec/mtk_vcodec_enc_hw.h       |  34 +++
> >  .../mediatek/vcodec/mtk_vcodec_enc_pm.c       | 178 +++++++++++--
> >  .../mediatek/vcodec/mtk_vcodec_enc_pm.h       |  11 +-
> >  .../mediatek/vcodec/mtk_vcodec_util.c         |  19 ++
> >  .../mediatek/vcodec/mtk_vcodec_util.h         |   3 +
> >  .../mediatek/vcodec/venc/venc_h264_if.c       | 243
> > ++++++++++++++----
> >  .../mediatek/vcodec/venc/venc_vp8_if.c        |   3 +-
> >  .../platform/mediatek/vcodec/venc_drv_if.c    |  75 ++++--
> >  .../platform/mediatek/vcodec/venc_drv_if.h    |   6 +
> >  .../platform/mediatek/vcodec/venc_vpu_if.c    |   9 +-
> >  .../platform/mediatek/vcodec/venc_vpu_if.h    |   3 +-
> >  19 files changed, 1028 insertions(+), 131 deletions(-)
> >  create mode 100644
> > Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-
> > core.yaml
> >  create mode 100644
> > drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.c
> >  create mode 100644
> > drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.h
> > 
> 
> 


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 22+ messages in thread
end of thread, other threads:[~2022-08-09  2:26 UTC | newest]

Thread overview: 22+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-07-29  3:51 [PATCH v5, 0/8] Support H264 multi-core encoder on MT8195 Irui Wang
2022-07-29  3:51 ` Irui Wang
2022-07-29  3:51 ` [PATCH v5, 1/8] dt-bindings: media: mediatek: vcodec: Adds encoder cores dt-bindings for mt8195 Irui Wang
2022-07-29  3:51   ` Irui Wang
2022-07-29  3:51 ` [PATCH v5, 2/8] media: mediatek: vcodec: Enable venc dual core usage Irui Wang
2022-07-29  3:51   ` Irui Wang
2022-07-29  3:51 ` [PATCH v5, 3/8] media: mediatek: vcodec: Refactor venc power manage function Irui Wang
2022-07-29  3:51   ` Irui Wang
2022-07-29  3:51 ` [PATCH v5, 4/8] media: mediatek: vcodec: Add more extra processing for multi-core encoding Irui Wang
2022-07-29  3:51   ` Irui Wang
2022-07-29  3:51 ` [PATCH v5, 5/8] media: mediatek: vcodec: Add venc power on/off function Irui Wang
2022-07-29  3:51   ` Irui Wang
2022-07-29  3:51 ` [PATCH v5, 6/8] media: mediatek: vcodec: Refactor encoder clock " Irui Wang
2022-07-29  3:51   ` Irui Wang
2022-07-29  3:51 ` [PATCH v5, 7/8] media: mediatek: vcodec: Add multi-core encoding process Irui Wang
2022-07-29  3:51   ` Irui Wang
2022-07-29  3:51 ` [PATCH v5, 8/8] media: mediatek: vcodec: Return encoding result in asynchronous mode Irui Wang
2022-07-29  3:51   ` Irui Wang
2022-07-29 15:08 ` [PATCH v5, 0/8] Support H264 multi-core encoder on MT8195 Nicolas Dufresne
2022-07-29 15:08   ` Nicolas Dufresne
2022-08-09  2:14   ` Irui Wang
2022-08-09  2:14     ` Irui Wang

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.