* [PATCH v4 1/2] media: dt-binding: Document rk3588’s VEPU121
2024-04-18 14:15 [PATCH v4 0/2] Enable JPEG encoding on rk3588 Emmanuel Gil Peyrot
@ 2024-04-18 14:15 ` Emmanuel Gil Peyrot
2024-04-18 14:15 ` [PATCH v4 2/2] arm64: dts: rockchip: Add one VEPU121 to rk3588 Emmanuel Gil Peyrot
2024-04-27 7:16 ` [PATCH v4 0/2] Enable JPEG encoding on rk3588 Jianfeng Liu
2 siblings, 0 replies; 6+ messages in thread
From: Emmanuel Gil Peyrot @ 2024-04-18 14:15 UTC (permalink / raw)
To: linux-kernel
Cc: Emmanuel Gil Peyrot, Ezequiel Garcia, Philipp Zabel,
Mauro Carvalho Chehab, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Heiko Stuebner, Joerg Roedel, Will Deacon,
Robin Murphy, Sebastian Reichel, Cristian Ciocaltea,
Dragan Simic, Shreeya Patel, Chris Morgan, Andy Yan,
Nicolas Frattaroli, linux-media, linux-rockchip, devicetree,
linux-arm-kernel, iommu, Conor Dooley
This encoder-only device is present four times on this SoC, and should
support everything the rk3568 vepu supports (so JPEG, H.264 and VP8
encoding).
According to the TRM[1], there is also the VEPU580 encoder which
supports H.264 and H.265, and various VDPU* decoders, of which only the
VDPU981 is currently supported. This patch describes only the VEPU121.
[1] https://github.com/FanX-Tek/rk3588-TRM-and-Datasheet
Signed-off-by: Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
Reviewed-by: Sebastian Reichel <sebastian.reichel@collabora.com>
Acked-by: Conor Dooley <conor.dooley@microchip.com>
---
.../devicetree/bindings/media/rockchip,rk3568-vepu.yaml | 8 ++++++--
1 file changed, 6 insertions(+), 2 deletions(-)
diff --git a/Documentation/devicetree/bindings/media/rockchip,rk3568-vepu.yaml b/Documentation/devicetree/bindings/media/rockchip,rk3568-vepu.yaml
index 9d90d8d0565a..4c6cb21da041 100644
--- a/Documentation/devicetree/bindings/media/rockchip,rk3568-vepu.yaml
+++ b/Documentation/devicetree/bindings/media/rockchip,rk3568-vepu.yaml
@@ -15,8 +15,12 @@ description:
properties:
compatible:
- enum:
- - rockchip,rk3568-vepu
+ oneOf:
+ - const: rockchip,rk3568-vepu
+ - items:
+ - enum:
+ - rockchip,rk3588-vepu121
+ - const: rockchip,rk3568-vepu
reg:
maxItems: 1
--
2.44.0
^ permalink raw reply related [flat|nested] 6+ messages in thread
* [PATCH v4 2/2] arm64: dts: rockchip: Add one VEPU121 to rk3588
2024-04-18 14:15 [PATCH v4 0/2] Enable JPEG encoding on rk3588 Emmanuel Gil Peyrot
2024-04-18 14:15 ` [PATCH v4 1/2] media: dt-binding: Document rk3588’s VEPU121 Emmanuel Gil Peyrot
@ 2024-04-18 14:15 ` Emmanuel Gil Peyrot
2024-04-27 7:16 ` [PATCH v4 0/2] Enable JPEG encoding on rk3588 Jianfeng Liu
2 siblings, 0 replies; 6+ messages in thread
From: Emmanuel Gil Peyrot @ 2024-04-18 14:15 UTC (permalink / raw)
To: linux-kernel
Cc: Emmanuel Gil Peyrot, Ezequiel Garcia, Philipp Zabel,
Mauro Carvalho Chehab, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Heiko Stuebner, Joerg Roedel, Will Deacon,
Robin Murphy, Sebastian Reichel, Cristian Ciocaltea,
Dragan Simic, Shreeya Patel, Chris Morgan, Andy Yan,
Nicolas Frattaroli, linux-media, linux-rockchip, devicetree,
linux-arm-kernel, iommu
The TRM (version 1.0 page 385) lists five VEPU121 cores, but only four
interrupts are listed (on page 24), and the driver would expose them all
as different video nodes so only one of them is exposed for now.
Signed-off-by: Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
Formerly-Reviewed-by: Sebastian Reichel <sebastian.reichel@collabora.com>
---
arch/arm64/boot/dts/rockchip/rk3588s.dtsi | 29 +++++++++++++++++++++++
1 file changed, 29 insertions(+)
diff --git a/arch/arm64/boot/dts/rockchip/rk3588s.dtsi b/arch/arm64/boot/dts/rockchip/rk3588s.dtsi
index 87b83c87bd55..5c6cc4cd81df 100644
--- a/arch/arm64/boot/dts/rockchip/rk3588s.dtsi
+++ b/arch/arm64/boot/dts/rockchip/rk3588s.dtsi
@@ -2488,6 +2488,35 @@ gpio4: gpio@fec50000 {
};
};
+ /*
+ * Currently only one of the four JPEG encoders is exposed, this
+ * reduces the optimal throughput by four.
+ *
+ * Once the driver is made to expose all four devices as a single video
+ * node, the rest can be enabled again, see the full patch here:
+ * https://lore.kernel.org/linux-media/20240327134115.424846-1-linkmauve@linkmauve.fr/T/#m3a6df0ba15e4af40b998b0ff2a02b0dd0d730c8e
+ */
+
+ jpeg_enc0: video-codec@fdba0000 {
+ compatible = "rockchip,rk3588-vepu121", "rockchip,rk3568-vepu";
+ reg = <0x0 0xfdba0000 0x0 0x800>;
+ interrupts = <GIC_SPI 122 IRQ_TYPE_LEVEL_HIGH 0>;
+ clocks = <&cru ACLK_JPEG_ENCODER0>, <&cru HCLK_JPEG_ENCODER0>;
+ clock-names = "aclk", "hclk";
+ iommus = <&jpeg_enc0_mmu>;
+ power-domains = <&power RK3588_PD_VDPU>;
+ };
+
+ jpeg_enc0_mmu: iommu@fdba0800 {
+ compatible = "rockchip,rk3588-iommu", "rockchip,rk3568-iommu";
+ reg = <0x0 0xfdba0800 0x0 0x40>;
+ interrupts = <GIC_SPI 121 IRQ_TYPE_LEVEL_HIGH 0>;
+ clocks = <&cru ACLK_JPEG_ENCODER0>, <&cru HCLK_JPEG_ENCODER0>;
+ clock-names = "aclk", "iface";
+ power-domains = <&power RK3588_PD_VDPU>;
+ #iommu-cells = <0>;
+ };
+
av1d: video-codec@fdc70000 {
compatible = "rockchip,rk3588-av1-vpu";
reg = <0x0 0xfdc70000 0x0 0x800>;
--
2.44.0
^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCH v4 0/2] Enable JPEG encoding on rk3588
2024-04-18 14:15 [PATCH v4 0/2] Enable JPEG encoding on rk3588 Emmanuel Gil Peyrot
2024-04-18 14:15 ` [PATCH v4 1/2] media: dt-binding: Document rk3588’s VEPU121 Emmanuel Gil Peyrot
2024-04-18 14:15 ` [PATCH v4 2/2] arm64: dts: rockchip: Add one VEPU121 to rk3588 Emmanuel Gil Peyrot
@ 2024-04-27 7:16 ` Jianfeng Liu
2024-04-28 17:01 ` Heiko Stübner
2 siblings, 1 reply; 6+ messages in thread
From: Jianfeng Liu @ 2024-04-27 7:16 UTC (permalink / raw)
To: linkmauve
Cc: andy.yan, conor+dt, cristian.ciocaltea, devicetree, dsimic,
ezequiel, frattaroli.nicolas, heiko, iommu, joro,
krzysztof.kozlowski+dt, linux-arm-kernel, linux-kernel,
linux-media, linux-rockchip, macromorgan, mchehab, p.zabel, robh,
robin.murphy, sebastian.reichel, shreeya.patel, will, sigmaris,
nicolas
Hi Emmanuel,
On Thu, 18 Apr 2024 16:15:04 +0200, Emmanuel Gil Peyrot wrote:
>- Only expose a single VEPU121, since the driver doesn’t yet support
> exposing them all as a single video node to userspace.
I'm also doing work with the hantro node on rk3588 which has both vdpu
and vepu. Discussions can be found here[1]. If I enable the jpeg encoder
at feb50000, would it cause any side effects if other jpeg encoders in
your patch are enabled? And what's the disadvantage of enabling multi
jpeg endoers in devicetree?
[1] https://patchwork.kernel.org/project/linux-rockchip/patch/20240418111002.83015-3-liujianfeng1994@gmail.com/
Best regards,
Jianfeng
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH v4 0/2] Enable JPEG encoding on rk3588
2024-04-27 7:16 ` [PATCH v4 0/2] Enable JPEG encoding on rk3588 Jianfeng Liu
@ 2024-04-28 17:01 ` Heiko Stübner
2024-04-28 17:29 ` Jianfeng Liu
0 siblings, 1 reply; 6+ messages in thread
From: Heiko Stübner @ 2024-04-28 17:01 UTC (permalink / raw)
To: linkmauve, Jianfeng Liu
Cc: andy.yan, conor+dt, cristian.ciocaltea, devicetree, dsimic,
ezequiel, frattaroli.nicolas, iommu, joro,
krzysztof.kozlowski+dt, linux-arm-kernel, linux-kernel,
linux-media, linux-rockchip, macromorgan, mchehab, p.zabel, robh,
robin.murphy, sebastian.reichel, shreeya.patel, will, sigmaris,
nicolas
Hi Jianfeng,
Am Samstag, 27. April 2024, 09:16:38 CEST schrieb Jianfeng Liu:
> Hi Emmanuel,
>
> On Thu, 18 Apr 2024 16:15:04 +0200, Emmanuel Gil Peyrot wrote:
> >- Only expose a single VEPU121, since the driver doesn’t yet support
> > exposing them all as a single video node to userspace.
> I'm also doing work with the hantro node on rk3588 which has both vdpu
> and vepu. Discussions can be found here[1]. If I enable the jpeg encoder
> at feb50000, would it cause any side effects if other jpeg encoders in
> your patch are enabled? And what's the disadvantage of enabling multi
> jpeg endoers in devicetree?
the basic problem is that exposing multiple jpeg encoders would require
userspace to do the scheduling. Which is bad.
I.e. all userspace programms would need to know the existence of
all jpeg encoders and then somehow negotiate how to use all of them
most efficiently.
Think multiple different programs that would need to negotiate to
spread across all of them in the best way.
Doing this in the kernel, we just expose one encoder (and queue) all
programs would just pile onto that one encoder and the kernel then
would be on the hook to do the scheduling - which the kernel can do
better (with relevant code implemented)
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH v4 0/2] Enable JPEG encoding on rk3588
2024-04-28 17:01 ` Heiko Stübner
@ 2024-04-28 17:29 ` Jianfeng Liu
0 siblings, 0 replies; 6+ messages in thread
From: Jianfeng Liu @ 2024-04-28 17:29 UTC (permalink / raw)
To: heiko
Cc: andy.yan, conor+dt, cristian.ciocaltea, devicetree, dsimic,
ezequiel, frattaroli.nicolas, iommu, joro,
krzysztof.kozlowski+dt, linkmauve, linux-arm-kernel,
linux-kernel, linux-media, linux-rockchip, liujianfeng1994,
macromorgan, mchehab, nicolas, p.zabel, robh, robin.murphy,
sebastian.reichel, shreeya.patel, sigmaris, will
Hi Heiko,
On Sun, 28 Apr 2024 19:01:38 +0200, Heiko Stübner wrote:
>the basic problem is that exposing multiple jpeg encoders would require
>userspace to do the scheduling. Which is bad.
>I.e. all userspace programms would need to know the existence of
>all jpeg encoders and then somehow negotiate how to use all of them
>most efficiently.
>
>Think multiple different programs that would need to negotiate to
>spread across all of them in the best way.
>
>Doing this in the kernel, we just expose one encoder (and queue) all
>programs would just pile onto that one encoder and the kernel then
>would be on the hook to do the scheduling - which the kernel can do
>better (with relevant code implemented)
Yeah let kernel do the scheduling is indeed better. And I'm happy to
hear this method.
So I will keep the vpu at feb50000 with jpeg endoder disabled until
multi encoder scheduling is implemented.
Best regards,
Jianfeng
^ permalink raw reply [flat|nested] 6+ messages in thread