* [PATCH v5 1/3] media: dt-binding: media: Add rk3568-vepu binding
2022-06-12 15:53 [PATCH v5 0/3] Enable JPEG Encoder on RK3566/RK3568 Nicolas Frattaroli
@ 2022-06-12 15:53 ` Nicolas Frattaroli
2022-06-14 21:02 ` Rob Herring
2022-06-12 15:53 ` [PATCH v5 2/3] media: hantro: Add support for RK356x encoder Nicolas Frattaroli
` (2 subsequent siblings)
3 siblings, 1 reply; 7+ messages in thread
From: Nicolas Frattaroli @ 2022-06-12 15:53 UTC (permalink / raw)
To: Ezequiel Garcia, Philipp Zabel, Mauro Carvalho Chehab,
Rob Herring, Krzysztof Kozlowski, Heiko Stuebner,
Nicolas Frattaroli
Cc: linux-media, linux-rockchip, devicetree, linux-arm-kernel, linux-kernel
The RK3568 and RK3566 have a Hantro VPU node solely dedicated to
encoding. This patch adds a new binding to describe it, as it
does not really fit the rockchip-vpu binding, since there is no
decoder.
Signed-off-by: Nicolas Frattaroli <frattaroli.nicolas@gmail.com>
---
.../bindings/media/rockchip,rk3568-vepu.yaml | 69 +++++++++++++++++++
MAINTAINERS | 1 +
2 files changed, 70 insertions(+)
create mode 100644 Documentation/devicetree/bindings/media/rockchip,rk3568-vepu.yaml
diff --git a/Documentation/devicetree/bindings/media/rockchip,rk3568-vepu.yaml b/Documentation/devicetree/bindings/media/rockchip,rk3568-vepu.yaml
new file mode 100644
index 000000000000..81b26eb4cd35
--- /dev/null
+++ b/Documentation/devicetree/bindings/media/rockchip,rk3568-vepu.yaml
@@ -0,0 +1,69 @@
+# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause)
+
+%YAML 1.2
+---
+$id: "http://devicetree.org/schemas/media/rockchip,rk3568-vepu.yaml#"
+$schema: "http://devicetree.org/meta-schemas/core.yaml#"
+
+title: Hantro G1 VPU encoders implemented on Rockchip SoCs
+
+maintainers:
+ - Nicolas Frattaroli <frattaroli.nicolas@gmail.com>
+
+description:
+ Hantro G1 video encode-only accelerators present on Rockchip SoCs.
+
+properties:
+ compatible:
+ enum:
+ - rockchip,rk3568-vepu
+
+ reg:
+ maxItems: 1
+
+ interrupts:
+ maxItems: 1
+
+ clocks:
+ maxItems: 2
+
+ clock-names:
+ items:
+ - const: aclk
+ - const: hclk
+
+ power-domains:
+ maxItems: 1
+
+ iommus:
+ maxItems: 1
+
+required:
+ - compatible
+ - reg
+ - interrupts
+ - clocks
+ - clock-names
+
+additionalProperties: false
+
+examples:
+ - |
+ #include <dt-bindings/clock/rk3568-cru.h>
+ #include <dt-bindings/interrupt-controller/arm-gic.h>
+ #include <dt-bindings/power/rk3568-power.h>
+
+ bus {
+ #address-cells = <2>;
+ #size-cells = <2>;
+
+ vepu: video-codec@fdee0000 {
+ compatible = "rockchip,rk3568-vepu";
+ reg = <0x0 0xfdee0000 0x0 0x800>;
+ interrupts = <GIC_SPI 64 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&cru ACLK_JENC>, <&cru HCLK_JENC>;
+ clock-names = "aclk", "hclk";
+ iommus = <&vepu_mmu>;
+ power-domains = <&power RK3568_PD_RGA>;
+ };
+ };
diff --git a/MAINTAINERS b/MAINTAINERS
index 475e28365385..03c5db88908c 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -8703,6 +8703,7 @@ L: linux-media@vger.kernel.org
L: linux-rockchip@lists.infradead.org
S: Maintained
F: Documentation/devicetree/bindings/media/nxp,imx8mq-vpu.yaml
+F: Documentation/devicetree/bindings/media/rockchip,rk3568-vepu.yaml
F: Documentation/devicetree/bindings/media/rockchip-vpu.yaml
F: drivers/staging/media/hantro/
--
2.36.1
^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [PATCH v5 1/3] media: dt-binding: media: Add rk3568-vepu binding
2022-06-12 15:53 ` [PATCH v5 1/3] media: dt-binding: media: Add rk3568-vepu binding Nicolas Frattaroli
@ 2022-06-14 21:02 ` Rob Herring
0 siblings, 0 replies; 7+ messages in thread
From: Rob Herring @ 2022-06-14 21:02 UTC (permalink / raw)
To: Nicolas Frattaroli
Cc: linux-kernel, Philipp Zabel, Krzysztof Kozlowski, linux-rockchip,
Mauro Carvalho Chehab, Ezequiel Garcia, devicetree,
linux-arm-kernel, Rob Herring, Heiko Stuebner, linux-media
On Sun, 12 Jun 2022 17:53:44 +0200, Nicolas Frattaroli wrote:
> The RK3568 and RK3566 have a Hantro VPU node solely dedicated to
> encoding. This patch adds a new binding to describe it, as it
> does not really fit the rockchip-vpu binding, since there is no
> decoder.
>
> Signed-off-by: Nicolas Frattaroli <frattaroli.nicolas@gmail.com>
> ---
> .../bindings/media/rockchip,rk3568-vepu.yaml | 69 +++++++++++++++++++
> MAINTAINERS | 1 +
> 2 files changed, 70 insertions(+)
> create mode 100644 Documentation/devicetree/bindings/media/rockchip,rk3568-vepu.yaml
>
Reviewed-by: Rob Herring <robh@kernel.org>
^ permalink raw reply [flat|nested] 7+ messages in thread
* [PATCH v5 2/3] media: hantro: Add support for RK356x encoder
2022-06-12 15:53 [PATCH v5 0/3] Enable JPEG Encoder on RK3566/RK3568 Nicolas Frattaroli
2022-06-12 15:53 ` [PATCH v5 1/3] media: dt-binding: media: Add rk3568-vepu binding Nicolas Frattaroli
@ 2022-06-12 15:53 ` Nicolas Frattaroli
2022-06-23 11:53 ` Ezequiel Garcia
2022-06-12 15:53 ` [PATCH v5 3/3] arm64: dts: rockchip: Add Hantro encoder node to rk356x Nicolas Frattaroli
2022-09-09 23:10 ` (subset) [PATCH v5 0/3] Enable JPEG Encoder on RK3566/RK3568 Heiko Stuebner
3 siblings, 1 reply; 7+ messages in thread
From: Nicolas Frattaroli @ 2022-06-12 15:53 UTC (permalink / raw)
To: Ezequiel Garcia, Philipp Zabel, Mauro Carvalho Chehab,
Greg Kroah-Hartman, Heiko Stuebner
Cc: Nicolas Frattaroli, linux-media, linux-rockchip, linux-staging,
linux-kernel, linux-arm-kernel
The RK3566 and RK3568 SoCs come with a small Hantro instance which is
solely dedicated to encoding. This patch adds the necessary structs to
the Hantro driver to allow the JPEG encoder of it to function.
Through some sleuthing through the vendor's MPP source code and after
closer inspection of the TRM, it was determined that the hardware likely
supports VP8 and H.264 as well.
Tested with the following GStreamer command:
gst-launch-1.0 videotestsrc ! v4l2jpegenc ! matroskamux ! \
filesink location=foo.mkv
Signed-off-by: Nicolas Frattaroli <frattaroli.nicolas@gmail.com>
---
drivers/staging/media/hantro/hantro_drv.c | 1 +
drivers/staging/media/hantro/hantro_hw.h | 1 +
.../staging/media/hantro/rockchip_vpu_hw.c | 25 +++++++++++++++++++
3 files changed, 27 insertions(+)
diff --git a/drivers/staging/media/hantro/hantro_drv.c b/drivers/staging/media/hantro/hantro_drv.c
index ac232b5f7825..1112e8d0c821 100644
--- a/drivers/staging/media/hantro/hantro_drv.c
+++ b/drivers/staging/media/hantro/hantro_drv.c
@@ -638,6 +638,7 @@ static const struct of_device_id of_hantro_match[] = {
{ .compatible = "rockchip,rk3288-vpu", .data = &rk3288_vpu_variant, },
{ .compatible = "rockchip,rk3328-vpu", .data = &rk3328_vpu_variant, },
{ .compatible = "rockchip,rk3399-vpu", .data = &rk3399_vpu_variant, },
+ { .compatible = "rockchip,rk3568-vepu", .data = &rk3568_vepu_variant, },
{ .compatible = "rockchip,rk3568-vpu", .data = &rk3568_vpu_variant, },
#endif
#ifdef CONFIG_VIDEO_HANTRO_IMX8M
diff --git a/drivers/staging/media/hantro/hantro_hw.h b/drivers/staging/media/hantro/hantro_hw.h
index 52a960f6fa4a..8c7eb5591446 100644
--- a/drivers/staging/media/hantro/hantro_hw.h
+++ b/drivers/staging/media/hantro/hantro_hw.h
@@ -306,6 +306,7 @@ extern const struct hantro_variant rk3066_vpu_variant;
extern const struct hantro_variant rk3288_vpu_variant;
extern const struct hantro_variant rk3328_vpu_variant;
extern const struct hantro_variant rk3399_vpu_variant;
+extern const struct hantro_variant rk3568_vepu_variant;
extern const struct hantro_variant rk3568_vpu_variant;
extern const struct hantro_variant sama5d4_vdec_variant;
extern const struct hantro_variant sunxi_vpu_variant;
diff --git a/drivers/staging/media/hantro/rockchip_vpu_hw.c b/drivers/staging/media/hantro/rockchip_vpu_hw.c
index fc96501f3bc8..b39813d53e57 100644
--- a/drivers/staging/media/hantro/rockchip_vpu_hw.c
+++ b/drivers/staging/media/hantro/rockchip_vpu_hw.c
@@ -417,6 +417,14 @@ static const struct hantro_codec_ops rk3399_vpu_codec_ops[] = {
},
};
+static const struct hantro_codec_ops rk3568_vepu_codec_ops[] = {
+ [HANTRO_MODE_JPEG_ENC] = {
+ .run = rockchip_vpu2_jpeg_enc_run,
+ .reset = rockchip_vpu2_enc_reset,
+ .done = rockchip_vpu2_jpeg_enc_done,
+ },
+};
+
/*
* VPU variant.
*/
@@ -439,6 +447,10 @@ static const struct hantro_irq rockchip_vpu2_irqs[] = {
{ "vdpu", rockchip_vpu2_vdpu_irq },
};
+static const struct hantro_irq rk3568_vepu_irqs[] = {
+ { "vepu", rockchip_vpu2_vepu_irq },
+};
+
static const char * const rk3066_vpu_clk_names[] = {
"aclk_vdpu", "hclk_vdpu",
"aclk_vepu", "hclk_vepu"
@@ -545,6 +557,19 @@ const struct hantro_variant rk3399_vpu_variant = {
.num_clocks = ARRAY_SIZE(rockchip_vpu_clk_names)
};
+const struct hantro_variant rk3568_vepu_variant = {
+ .enc_offset = 0x0,
+ .enc_fmts = rockchip_vpu_enc_fmts,
+ .num_enc_fmts = ARRAY_SIZE(rockchip_vpu_enc_fmts),
+ .codec = HANTRO_JPEG_ENCODER,
+ .codec_ops = rk3568_vepu_codec_ops,
+ .irqs = rk3568_vepu_irqs,
+ .num_irqs = ARRAY_SIZE(rk3568_vepu_irqs),
+ .init = rockchip_vpu_hw_init,
+ .clk_names = rockchip_vpu_clk_names,
+ .num_clocks = ARRAY_SIZE(rockchip_vpu_clk_names)
+};
+
const struct hantro_variant rk3568_vpu_variant = {
.dec_offset = 0x400,
.dec_fmts = rk3399_vpu_dec_fmts,
--
2.36.1
^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [PATCH v5 2/3] media: hantro: Add support for RK356x encoder
2022-06-12 15:53 ` [PATCH v5 2/3] media: hantro: Add support for RK356x encoder Nicolas Frattaroli
@ 2022-06-23 11:53 ` Ezequiel Garcia
0 siblings, 0 replies; 7+ messages in thread
From: Ezequiel Garcia @ 2022-06-23 11:53 UTC (permalink / raw)
To: Nicolas Frattaroli
Cc: Philipp Zabel, Mauro Carvalho Chehab, Greg Kroah-Hartman,
Heiko Stuebner, linux-media, linux-rockchip, linux-staging,
linux-kernel, linux-arm-kernel
Hi Nicolas,
On Sun, Jun 12, 2022 at 05:53:45PM +0200, Nicolas Frattaroli wrote:
> The RK3566 and RK3568 SoCs come with a small Hantro instance which is
> solely dedicated to encoding. This patch adds the necessary structs to
> the Hantro driver to allow the JPEG encoder of it to function.
>
> Through some sleuthing through the vendor's MPP source code and after
> closer inspection of the TRM, it was determined that the hardware likely
> supports VP8 and H.264 as well.
>
> Tested with the following GStreamer command:
>
> gst-launch-1.0 videotestsrc ! v4l2jpegenc ! matroskamux ! \
> filesink location=foo.mkv
>
> Signed-off-by: Nicolas Frattaroli <frattaroli.nicolas@gmail.com>
Reviewed-by: Ezequiel Garcia <ezequiel@vanguardiasur.com.ar>
Thanks,
Ezequiel
> ---
> drivers/staging/media/hantro/hantro_drv.c | 1 +
> drivers/staging/media/hantro/hantro_hw.h | 1 +
> .../staging/media/hantro/rockchip_vpu_hw.c | 25 +++++++++++++++++++
> 3 files changed, 27 insertions(+)
>
> diff --git a/drivers/staging/media/hantro/hantro_drv.c b/drivers/staging/media/hantro/hantro_drv.c
> index ac232b5f7825..1112e8d0c821 100644
> --- a/drivers/staging/media/hantro/hantro_drv.c
> +++ b/drivers/staging/media/hantro/hantro_drv.c
> @@ -638,6 +638,7 @@ static const struct of_device_id of_hantro_match[] = {
> { .compatible = "rockchip,rk3288-vpu", .data = &rk3288_vpu_variant, },
> { .compatible = "rockchip,rk3328-vpu", .data = &rk3328_vpu_variant, },
> { .compatible = "rockchip,rk3399-vpu", .data = &rk3399_vpu_variant, },
> + { .compatible = "rockchip,rk3568-vepu", .data = &rk3568_vepu_variant, },
> { .compatible = "rockchip,rk3568-vpu", .data = &rk3568_vpu_variant, },
> #endif
> #ifdef CONFIG_VIDEO_HANTRO_IMX8M
> diff --git a/drivers/staging/media/hantro/hantro_hw.h b/drivers/staging/media/hantro/hantro_hw.h
> index 52a960f6fa4a..8c7eb5591446 100644
> --- a/drivers/staging/media/hantro/hantro_hw.h
> +++ b/drivers/staging/media/hantro/hantro_hw.h
> @@ -306,6 +306,7 @@ extern const struct hantro_variant rk3066_vpu_variant;
> extern const struct hantro_variant rk3288_vpu_variant;
> extern const struct hantro_variant rk3328_vpu_variant;
> extern const struct hantro_variant rk3399_vpu_variant;
> +extern const struct hantro_variant rk3568_vepu_variant;
> extern const struct hantro_variant rk3568_vpu_variant;
> extern const struct hantro_variant sama5d4_vdec_variant;
> extern const struct hantro_variant sunxi_vpu_variant;
> diff --git a/drivers/staging/media/hantro/rockchip_vpu_hw.c b/drivers/staging/media/hantro/rockchip_vpu_hw.c
> index fc96501f3bc8..b39813d53e57 100644
> --- a/drivers/staging/media/hantro/rockchip_vpu_hw.c
> +++ b/drivers/staging/media/hantro/rockchip_vpu_hw.c
> @@ -417,6 +417,14 @@ static const struct hantro_codec_ops rk3399_vpu_codec_ops[] = {
> },
> };
>
> +static const struct hantro_codec_ops rk3568_vepu_codec_ops[] = {
> + [HANTRO_MODE_JPEG_ENC] = {
> + .run = rockchip_vpu2_jpeg_enc_run,
> + .reset = rockchip_vpu2_enc_reset,
> + .done = rockchip_vpu2_jpeg_enc_done,
> + },
> +};
> +
> /*
> * VPU variant.
> */
> @@ -439,6 +447,10 @@ static const struct hantro_irq rockchip_vpu2_irqs[] = {
> { "vdpu", rockchip_vpu2_vdpu_irq },
> };
>
> +static const struct hantro_irq rk3568_vepu_irqs[] = {
> + { "vepu", rockchip_vpu2_vepu_irq },
> +};
> +
> static const char * const rk3066_vpu_clk_names[] = {
> "aclk_vdpu", "hclk_vdpu",
> "aclk_vepu", "hclk_vepu"
> @@ -545,6 +557,19 @@ const struct hantro_variant rk3399_vpu_variant = {
> .num_clocks = ARRAY_SIZE(rockchip_vpu_clk_names)
> };
>
> +const struct hantro_variant rk3568_vepu_variant = {
> + .enc_offset = 0x0,
> + .enc_fmts = rockchip_vpu_enc_fmts,
> + .num_enc_fmts = ARRAY_SIZE(rockchip_vpu_enc_fmts),
> + .codec = HANTRO_JPEG_ENCODER,
> + .codec_ops = rk3568_vepu_codec_ops,
> + .irqs = rk3568_vepu_irqs,
> + .num_irqs = ARRAY_SIZE(rk3568_vepu_irqs),
> + .init = rockchip_vpu_hw_init,
> + .clk_names = rockchip_vpu_clk_names,
> + .num_clocks = ARRAY_SIZE(rockchip_vpu_clk_names)
> +};
> +
> const struct hantro_variant rk3568_vpu_variant = {
> .dec_offset = 0x400,
> .dec_fmts = rk3399_vpu_dec_fmts,
> --
> 2.36.1
>
^ permalink raw reply [flat|nested] 7+ messages in thread
* [PATCH v5 3/3] arm64: dts: rockchip: Add Hantro encoder node to rk356x
2022-06-12 15:53 [PATCH v5 0/3] Enable JPEG Encoder on RK3566/RK3568 Nicolas Frattaroli
2022-06-12 15:53 ` [PATCH v5 1/3] media: dt-binding: media: Add rk3568-vepu binding Nicolas Frattaroli
2022-06-12 15:53 ` [PATCH v5 2/3] media: hantro: Add support for RK356x encoder Nicolas Frattaroli
@ 2022-06-12 15:53 ` Nicolas Frattaroli
2022-09-09 23:10 ` (subset) [PATCH v5 0/3] Enable JPEG Encoder on RK3566/RK3568 Heiko Stuebner
3 siblings, 0 replies; 7+ messages in thread
From: Nicolas Frattaroli @ 2022-06-12 15:53 UTC (permalink / raw)
To: Rob Herring, Krzysztof Kozlowski, Heiko Stuebner
Cc: Nicolas Frattaroli, devicetree, linux-arm-kernel, linux-rockchip,
linux-kernel
The RK3566 and RK3568 come with a dedicated Hantro instance solely for
encoding. This patch adds a node for this to the device tree, along with
a node for its MMU.
Signed-off-by: Nicolas Frattaroli <frattaroli.nicolas@gmail.com>
---
arch/arm64/boot/dts/rockchip/rk356x.dtsi | 20 ++++++++++++++++++++
1 file changed, 20 insertions(+)
diff --git a/arch/arm64/boot/dts/rockchip/rk356x.dtsi b/arch/arm64/boot/dts/rockchip/rk356x.dtsi
index cc1c5a65c5e5..73a1403192e9 100644
--- a/arch/arm64/boot/dts/rockchip/rk356x.dtsi
+++ b/arch/arm64/boot/dts/rockchip/rk356x.dtsi
@@ -576,6 +576,26 @@ gpu: gpu@fde60000 {
status = "disabled";
};
+ vepu: video-codec@fdee0000 {
+ compatible = "rockchip,rk3568-vepu";
+ reg = <0x0 0xfdee0000 0x0 0x800>;
+ interrupts = <GIC_SPI 64 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&cru ACLK_JENC>, <&cru HCLK_JENC>;
+ clock-names = "aclk", "hclk";
+ iommus = <&vepu_mmu>;
+ power-domains = <&power RK3568_PD_RGA>;
+ };
+
+ vepu_mmu: iommu@fdee0800 {
+ compatible = "rockchip,rk3568-iommu";
+ reg = <0x0 0xfdee0800 0x0 0x40>;
+ interrupts = <GIC_SPI 63 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&cru ACLK_JENC>, <&cru HCLK_JENC>;
+ clock-names = "aclk", "iface";
+ power-domains = <&power RK3568_PD_RGA>;
+ #iommu-cells = <0>;
+ };
+
sdmmc2: mmc@fe000000 {
compatible = "rockchip,rk3568-dw-mshc", "rockchip,rk3288-dw-mshc";
reg = <0x0 0xfe000000 0x0 0x4000>;
--
2.36.1
^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: (subset) [PATCH v5 0/3] Enable JPEG Encoder on RK3566/RK3568
2022-06-12 15:53 [PATCH v5 0/3] Enable JPEG Encoder on RK3566/RK3568 Nicolas Frattaroli
` (2 preceding siblings ...)
2022-06-12 15:53 ` [PATCH v5 3/3] arm64: dts: rockchip: Add Hantro encoder node to rk356x Nicolas Frattaroli
@ 2022-09-09 23:10 ` Heiko Stuebner
3 siblings, 0 replies; 7+ messages in thread
From: Heiko Stuebner @ 2022-09-09 23:10 UTC (permalink / raw)
To: Rob Herring, Ezequiel Garcia, Liang Chen, Krzysztof Kozlowski,
Greg Kroah-Hartman, Philipp Zabel, Peter Geis,
Mauro Carvalho Chehab, Michael Riesch, Nicolas Frattaroli
Cc: Heiko Stuebner, linux-kernel, devicetree, linux-media,
linux-staging, linux-rockchip, linux-arm-kernel
On Sun, 12 Jun 2022 17:53:43 +0200, Nicolas Frattaroli wrote:
> this is v5 of my RK356x JPEG encoder patch set. It enables the Hantro
> hardware encoder of the RK3566 and RK3568 line of SoCs, with JPEG being
> the only format the driver currently supports encoding for.
>
> The first patch adds a new binding, the rockchip,rk3568-vepu binding, to
> describe this kind of hardware. The reason for going with a new binding
> instead of modifying the vpu binding as the previous versions did is
> that the vpu binding is getting quite long, and Ezequiel Garcia suggested
> (on IRC) that we could document encoder only instances in their own
> binding. This makes sense to me, especially considering that RK3588 will
> have more Hantro instances like this to document.
>
> [...]
Applied, thanks!
[3/3] arm64: dts: rockchip: Add Hantro encoder node to rk356x
commit: 03d86fb5a56919ccf47e1cc5861bb5452017ab93
Best regards,
--
Heiko Stuebner <heiko@sntech.de>
^ permalink raw reply [flat|nested] 7+ messages in thread