* [PATCH 0/8] Additional features for Hantro HEVC
@ 2021-06-04 13:06 Benjamin Gaignard
2021-06-04 13:06 ` [PATCH 1/8] media: hantro: Trace hevc hw cycles performance register Benjamin Gaignard
` (8 more replies)
0 siblings, 9 replies; 21+ messages in thread
From: Benjamin Gaignard @ 2021-06-04 13:06 UTC (permalink / raw)
To: hverkuil, ezequiel, p.zabel, mchehab, shawnguo, s.hauer,
festevam, gregkh, mripard, paul.kocialkowski, wens,
jernej.skrabec, emil.l.velikov, andrzej.p, jc
Cc: kernel, linux-imx, linux-media, linux-rockchip, linux-arm-kernel,
linux-kernel, Benjamin Gaignard
Basic HEVC support has been added to Hantro driver in this pull request:
https://www.spinics.net/lists/linux-media/msg193744.html
Thanks to that it is now possible to support more features for this driver.
The first patch allow to log the hardware performance per macroblock.
The second patch makes the driver use compressed reference frames to
reduce memory bandwidth consumption.
Patches 3 to 5 allow to decode and produce 10-bits P010 frames.
Patch 6 make usage of G2 post processor to scale down the frames.
Patches 7 and 8 add the support of HEVC scaling matrix by adding a new
control.
All these patches enhance the HEVC support for Hantro (G2) hardware.
Unluckily they almost all touch the same pieces of code, where buffer
size, offset and addresses are set, so they have to be in this order.
They depend of the series pushed in this pull request:
https://www.spinics.net/lists/linux-media/msg193744.html
Benjamin
Benjamin Gaignard (8):
media: hantro: Trace hevc hw cycles performance register
media: hantro: Add support of compressed reference buffers
media: hantro: hevc: Allow 10-bits encoded streams
media: Add P010 video format
media: hantro: hevc: Allow to produce 10-bit frames
media: hantro: enumerate scaled output formats
media: hevc: Add scaling matrix control
media: hantro: Add scaling lists feature
.../media/v4l/ext-ctrls-codec.rst | 45 +++++
.../media/v4l/pixfmt-yuv-planar.rst | 8 +
.../media/v4l/vidioc-queryctrl.rst | 6 +
drivers/media/v4l2-core/v4l2-common.c | 1 +
drivers/media/v4l2-core/v4l2-ctrls-core.c | 6 +
drivers/media/v4l2-core/v4l2-ctrls-defs.c | 4 +
drivers/media/v4l2-core/v4l2-ioctl.c | 1 +
drivers/staging/media/hantro/hantro.h | 4 +
drivers/staging/media/hantro/hantro_drv.c | 32 +++-
.../staging/media/hantro/hantro_g2_hevc_dec.c | 175 ++++++++++++++++--
drivers/staging/media/hantro/hantro_g2_regs.h | 12 ++
drivers/staging/media/hantro/hantro_hevc.c | 60 +++++-
drivers/staging/media/hantro/hantro_hw.h | 7 +
drivers/staging/media/hantro/hantro_v4l2.c | 10 +-
drivers/staging/media/hantro/imx8m_vpu_hw.c | 6 +
drivers/staging/media/hantro/trace.h | 40 ++++
include/media/hevc-ctrls.h | 11 ++
include/uapi/linux/videodev2.h | 1 +
18 files changed, 407 insertions(+), 22 deletions(-)
create mode 100644 drivers/staging/media/hantro/trace.h
--
2.25.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 21+ messages in thread
* [PATCH 1/8] media: hantro: Trace hevc hw cycles performance register
2021-06-04 13:06 [PATCH 0/8] Additional features for Hantro HEVC Benjamin Gaignard
@ 2021-06-04 13:06 ` Benjamin Gaignard
2021-06-13 9:53 ` Hans Verkuil
2021-06-04 13:06 ` [PATCH 2/8] media: hantro: Add support of compressed reference buffers Benjamin Gaignard
` (7 subsequent siblings)
8 siblings, 1 reply; 21+ messages in thread
From: Benjamin Gaignard @ 2021-06-04 13:06 UTC (permalink / raw)
To: hverkuil, ezequiel, p.zabel, mchehab, shawnguo, s.hauer,
festevam, gregkh, mripard, paul.kocialkowski, wens,
jernej.skrabec, emil.l.velikov, andrzej.p, jc
Cc: kernel, linux-imx, linux-media, linux-rockchip, linux-arm-kernel,
linux-kernel, Benjamin Gaignard
After each hevc decoded frame trace the hardware performance.
It provides the number of hw cycles spend per decoded macroblock.
Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
---
drivers/staging/media/hantro/hantro_drv.c | 3 ++
.../staging/media/hantro/hantro_g2_hevc_dec.c | 16 ++++++++
drivers/staging/media/hantro/hantro_g2_regs.h | 1 +
drivers/staging/media/hantro/hantro_hw.h | 1 +
drivers/staging/media/hantro/imx8m_vpu_hw.c | 1 +
drivers/staging/media/hantro/trace.h | 40 +++++++++++++++++++
6 files changed, 62 insertions(+)
create mode 100644 drivers/staging/media/hantro/trace.h
diff --git a/drivers/staging/media/hantro/hantro_drv.c b/drivers/staging/media/hantro/hantro_drv.c
index dbc69ee0b562..6053c86b1c3f 100644
--- a/drivers/staging/media/hantro/hantro_drv.c
+++ b/drivers/staging/media/hantro/hantro_drv.c
@@ -28,6 +28,9 @@
#include "hantro.h"
#include "hantro_hw.h"
+#define CREATE_TRACE_POINTS
+#include "trace.h"
+
#define DRIVER_NAME "hantro-vpu"
int hantro_debug;
diff --git a/drivers/staging/media/hantro/hantro_g2_hevc_dec.c b/drivers/staging/media/hantro/hantro_g2_hevc_dec.c
index 340efb57fd18..89fac5146433 100644
--- a/drivers/staging/media/hantro/hantro_g2_hevc_dec.c
+++ b/drivers/staging/media/hantro/hantro_g2_hevc_dec.c
@@ -7,6 +7,7 @@
#include "hantro_hw.h"
#include "hantro_g2_regs.h"
+#include "trace.h"
#define HEVC_DEC_MODE 0xC
@@ -22,6 +23,21 @@ static inline void hantro_write_addr(struct hantro_dev *vpu,
vdpu_write(vpu, addr & 0xffffffff, offset);
}
+void hantro_g2_hevc_dec_done(struct hantro_ctx *ctx)
+{
+ const struct hantro_hevc_dec_ctrls *ctrls = &ctx->hevc_dec.ctrls;
+ const struct v4l2_ctrl_hevc_sps *sps = ctrls->sps;
+ struct hantro_dev *vpu = ctx->dev;
+ u32 hw_cycles = 0;
+ u32 mbs = (sps->pic_width_in_luma_samples *
+ sps->pic_height_in_luma_samples) >> 8;
+
+ if (mbs)
+ hw_cycles = vdpu_read(vpu, G2_HW_PERFORMANCE) / mbs;
+
+ trace_hantro_hevc_perf(ctx, hw_cycles);
+}
+
static void prepare_tile_info_buffer(struct hantro_ctx *ctx)
{
struct hantro_dev *vpu = ctx->dev;
diff --git a/drivers/staging/media/hantro/hantro_g2_regs.h b/drivers/staging/media/hantro/hantro_g2_regs.h
index bb22fa921914..17d84ec9c5c2 100644
--- a/drivers/staging/media/hantro/hantro_g2_regs.h
+++ b/drivers/staging/media/hantro/hantro_g2_regs.h
@@ -177,6 +177,7 @@
#define G2_REG_CONFIG_DEC_CLK_GATE_E BIT(16)
#define G2_REG_CONFIG_DEC_CLK_GATE_IDLE_E BIT(17)
+#define G2_HW_PERFORMANCE (G2_SWREG(63))
#define G2_ADDR_DST (G2_SWREG(65))
#define G2_REG_ADDR_REF(i) (G2_SWREG(67) + ((i) * 0x8))
#define G2_ADDR_DST_CHR (G2_SWREG(99))
diff --git a/drivers/staging/media/hantro/hantro_hw.h b/drivers/staging/media/hantro/hantro_hw.h
index 5737a7707944..8fa0aacb61cd 100644
--- a/drivers/staging/media/hantro/hantro_hw.h
+++ b/drivers/staging/media/hantro/hantro_hw.h
@@ -240,6 +240,7 @@ void hantro_h264_dec_exit(struct hantro_ctx *ctx);
int hantro_hevc_dec_init(struct hantro_ctx *ctx);
void hantro_hevc_dec_exit(struct hantro_ctx *ctx);
int hantro_g2_hevc_dec_run(struct hantro_ctx *ctx);
+void hantro_g2_hevc_dec_done(struct hantro_ctx *ctx);
int hantro_hevc_dec_prepare_run(struct hantro_ctx *ctx);
dma_addr_t hantro_hevc_get_ref_buf(struct hantro_ctx *ctx, int poc);
void hantro_hevc_ref_remove_unused(struct hantro_ctx *ctx);
diff --git a/drivers/staging/media/hantro/imx8m_vpu_hw.c b/drivers/staging/media/hantro/imx8m_vpu_hw.c
index ea919bfb9891..7e9e24bb5057 100644
--- a/drivers/staging/media/hantro/imx8m_vpu_hw.c
+++ b/drivers/staging/media/hantro/imx8m_vpu_hw.c
@@ -239,6 +239,7 @@ static const struct hantro_codec_ops imx8mq_vpu_g2_codec_ops[] = {
.reset = imx8m_vpu_g2_reset,
.init = hantro_hevc_dec_init,
.exit = hantro_hevc_dec_exit,
+ .done = hantro_g2_hevc_dec_done,
},
};
diff --git a/drivers/staging/media/hantro/trace.h b/drivers/staging/media/hantro/trace.h
new file mode 100644
index 000000000000..8abe5ddb4814
--- /dev/null
+++ b/drivers/staging/media/hantro/trace.h
@@ -0,0 +1,40 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#undef TRACE_SYSTEM
+#define TRACE_SYSTEM hantro
+
+#if !defined(__HANTRO_TRACE_H__) || defined(TRACE_HEADER_MULTI_READ)
+#define __HANTRO_TRACE_H__
+
+#include <linux/tracepoint.h>
+#include <media/videobuf2-v4l2.h>
+
+#include "hantro.h"
+
+TRACE_EVENT(hantro_hevc_perf,
+ TP_PROTO(struct hantro_ctx *ctx, u32 hw_cycles),
+
+ TP_ARGS(ctx, hw_cycles),
+
+ TP_STRUCT__entry(
+ __field(int, minor)
+ __field(u32, hw_cycles)
+ ),
+
+ TP_fast_assign(
+ __entry->minor = ctx->fh.vdev->minor;
+ __entry->hw_cycles = hw_cycles;
+ ),
+
+ TP_printk("minor = %d, %8d cycles / mb",
+ __entry->minor, __entry->hw_cycles)
+);
+
+#endif /* __HANTRO_TRACE_H__ */
+
+#undef TRACE_INCLUDE_PATH
+#define TRACE_INCLUDE_PATH ../../drivers/staging/media/hantro
+#undef TRACE_INCLUDE_FILE
+#define TRACE_INCLUDE_FILE trace
+
+/* This part must be outside protection */
+#include <trace/define_trace.h>
--
2.25.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 21+ messages in thread
* [PATCH 2/8] media: hantro: Add support of compressed reference buffers
2021-06-04 13:06 [PATCH 0/8] Additional features for Hantro HEVC Benjamin Gaignard
2021-06-04 13:06 ` [PATCH 1/8] media: hantro: Trace hevc hw cycles performance register Benjamin Gaignard
@ 2021-06-04 13:06 ` Benjamin Gaignard
2021-06-04 16:32 ` Ezequiel Garcia
2021-06-04 13:06 ` [PATCH 3/8] media: hantro: hevc: Allow 10-bits encoded streams Benjamin Gaignard
` (6 subsequent siblings)
8 siblings, 1 reply; 21+ messages in thread
From: Benjamin Gaignard @ 2021-06-04 13:06 UTC (permalink / raw)
To: hverkuil, ezequiel, p.zabel, mchehab, shawnguo, s.hauer,
festevam, gregkh, mripard, paul.kocialkowski, wens,
jernej.skrabec, emil.l.velikov, andrzej.p, jc
Cc: kernel, linux-imx, linux-media, linux-rockchip, linux-arm-kernel,
linux-kernel, Benjamin Gaignard
Reference frame compression is a feature added in G2 decoder to compress
frame buffers so that the bandwidth of storing/loading reference frames
can be reduced, especially when the resolution of decoded stream is of
high definition.
Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
---
.../staging/media/hantro/hantro_g2_hevc_dec.c | 20 ++++++++++--
drivers/staging/media/hantro/hantro_g2_regs.h | 4 +++
drivers/staging/media/hantro/hantro_hevc.c | 32 ++++++++++++++++++-
drivers/staging/media/hantro/hantro_hw.h | 2 ++
4 files changed, 54 insertions(+), 4 deletions(-)
diff --git a/drivers/staging/media/hantro/hantro_g2_hevc_dec.c b/drivers/staging/media/hantro/hantro_g2_hevc_dec.c
index 89fac5146433..9a715e803037 100644
--- a/drivers/staging/media/hantro/hantro_g2_hevc_dec.c
+++ b/drivers/staging/media/hantro/hantro_g2_hevc_dec.c
@@ -382,10 +382,12 @@ static int set_ref(struct hantro_ctx *ctx)
const struct v4l2_ctrl_hevc_pps *pps = ctrls->pps;
const struct v4l2_ctrl_hevc_decode_params *decode_params = ctrls->decode_params;
const struct v4l2_hevc_dpb_entry *dpb = decode_params->dpb;
- dma_addr_t luma_addr, chroma_addr, mv_addr = 0;
+ dma_addr_t luma_addr, chroma_addr, mv_addr, compress_luma_addr, compress_chroma_addr = 0;
struct hantro_dev *vpu = ctx->dev;
size_t cr_offset = hantro_hevc_chroma_offset(sps);
size_t mv_offset = hantro_hevc_motion_vectors_offset(sps);
+ size_t compress_luma_offset = hantro_hevc_luma_compress_offset(sps);
+ size_t compress_chroma_offset = hantro_hevc_chroma_compress_offset(sps);
u32 max_ref_frames;
u16 dpb_longterm_e;
static const struct hantro_reg cur_poc[] = {
@@ -460,6 +462,8 @@ static int set_ref(struct hantro_ctx *ctx)
chroma_addr = luma_addr + cr_offset;
mv_addr = luma_addr + mv_offset;
+ compress_luma_addr = luma_addr + compress_luma_offset;
+ compress_chroma_addr = luma_addr + compress_chroma_offset;
if (dpb[i].rps == V4L2_HEVC_DPB_ENTRY_RPS_LT_CURR)
dpb_longterm_e |= BIT(V4L2_HEVC_DPB_ENTRIES_NUM_MAX - 1 - i);
@@ -467,6 +471,8 @@ static int set_ref(struct hantro_ctx *ctx)
hantro_write_addr(vpu, G2_REG_ADDR_REF(i), luma_addr);
hantro_write_addr(vpu, G2_REG_CHR_REF(i), chroma_addr);
hantro_write_addr(vpu, G2_REG_DMV_REF(i), mv_addr);
+ hantro_write_addr(vpu, G2_COMP_ADDR_REF(i), compress_luma_addr);
+ hantro_write_addr(vpu, G2_COMP_CHR_REF(i), compress_chroma_addr);
}
luma_addr = hantro_hevc_get_ref_buf(ctx, decode_params->pic_order_cnt_val);
@@ -475,7 +481,11 @@ static int set_ref(struct hantro_ctx *ctx)
chroma_addr = luma_addr + cr_offset;
mv_addr = luma_addr + mv_offset;
+ compress_luma_addr = luma_addr + compress_luma_offset;
+ compress_chroma_addr = luma_addr + compress_chroma_offset;
+ hantro_write_addr(vpu, G2_COMP_ADDR_REF(i), compress_luma_addr);
+ hantro_write_addr(vpu, G2_COMP_CHR_REF(i), compress_chroma_addr);
hantro_write_addr(vpu, G2_REG_ADDR_REF(i), luma_addr);
hantro_write_addr(vpu, G2_REG_CHR_REF(i), chroma_addr);
hantro_write_addr(vpu, G2_REG_DMV_REF(i++), mv_addr);
@@ -483,6 +493,8 @@ static int set_ref(struct hantro_ctx *ctx)
hantro_write_addr(vpu, G2_ADDR_DST, luma_addr);
hantro_write_addr(vpu, G2_ADDR_DST_CHR, chroma_addr);
hantro_write_addr(vpu, G2_ADDR_DST_MV, mv_addr);
+ hantro_write_addr(vpu, G2_COMP_ADDR_DST, compress_luma_addr);
+ hantro_write_addr(vpu, G2_COMP_CHR, compress_chroma_addr);
hantro_hevc_ref_remove_unused(ctx);
@@ -490,6 +502,8 @@ static int set_ref(struct hantro_ctx *ctx)
hantro_write_addr(vpu, G2_REG_ADDR_REF(i), 0);
hantro_write_addr(vpu, G2_REG_CHR_REF(i), 0);
hantro_write_addr(vpu, G2_REG_DMV_REF(i), 0);
+ hantro_write_addr(vpu, G2_COMP_ADDR_REF(i), 0);
+ hantro_write_addr(vpu, G2_COMP_CHR_REF(i), 0);
}
hantro_reg_write(vpu, &g2_refer_lterm_e, dpb_longterm_e);
@@ -580,8 +594,8 @@ int hantro_g2_hevc_dec_run(struct hantro_ctx *ctx)
/* Don't disable output */
hantro_reg_write(vpu, &g2_out_dis, 0);
- /* Don't compress buffers */
- hantro_reg_write(vpu, &g2_ref_compress_bypass, 1);
+ /* Compress buffers */
+ hantro_reg_write(vpu, &g2_ref_compress_bypass, 0);
/* use NV12 as output format */
hantro_reg_write(vpu, &g2_out_rs_e, 1);
diff --git a/drivers/staging/media/hantro/hantro_g2_regs.h b/drivers/staging/media/hantro/hantro_g2_regs.h
index 17d84ec9c5c2..0414d92e3860 100644
--- a/drivers/staging/media/hantro/hantro_g2_regs.h
+++ b/drivers/staging/media/hantro/hantro_g2_regs.h
@@ -192,6 +192,10 @@
#define G2_TILE_FILTER (G2_SWREG(179))
#define G2_TILE_SAO (G2_SWREG(181))
#define G2_TILE_BSD (G2_SWREG(183))
+#define G2_COMP_ADDR_DST (G2_SWREG(190))
+#define G2_COMP_ADDR_REF(i) (G2_SWREG(192) + ((i) * 0x8))
+#define G2_COMP_CHR (G2_SWREG(224))
+#define G2_COMP_CHR_REF(i) (G2_SWREG(226) + ((i) * 0x8))
#define g2_strm_buffer_len G2_DEC_REG(258, 0, 0xffffffff)
#define g2_strm_start_offset G2_DEC_REG(259, 0, 0xffffffff)
diff --git a/drivers/staging/media/hantro/hantro_hevc.c b/drivers/staging/media/hantro/hantro_hevc.c
index 5347f5a41c2a..1b2da990fbf0 100644
--- a/drivers/staging/media/hantro/hantro_hevc.c
+++ b/drivers/staging/media/hantro/hantro_hevc.c
@@ -61,12 +61,42 @@ static size_t hantro_hevc_mv_size(const struct v4l2_ctrl_hevc_sps *sps)
return mv_size;
}
+size_t hantro_hevc_luma_compress_offset(const struct v4l2_ctrl_hevc_sps *sps)
+{
+ return hantro_hevc_motion_vectors_offset(sps) + hantro_hevc_mv_size(sps);
+}
+
+static size_t hantro_hevc_luma_compress_size(const struct v4l2_ctrl_hevc_sps *sps)
+{
+ u32 pic_width_in_cbsy =
+ round_up((sps->pic_width_in_luma_samples + 8 - 1) / 8, 16);
+ u32 pic_height_in_cbsy = (sps->pic_height_in_luma_samples + 8 - 1) / 8;
+
+ return round_up(pic_width_in_cbsy * pic_height_in_cbsy, 16);
+}
+
+size_t hantro_hevc_chroma_compress_offset(const struct v4l2_ctrl_hevc_sps *sps)
+{
+ return hantro_hevc_luma_compress_offset(sps) + hantro_hevc_luma_compress_size(sps);
+}
+
+static size_t hantro_hevc_chroma_compress_size(const struct v4l2_ctrl_hevc_sps *sps)
+{
+ u32 pic_width_in_cbsc =
+ round_up((sps->pic_width_in_luma_samples + 16 - 1) / 16, 16);
+ u32 pic_height_in_cbsc = (sps->pic_height_in_luma_samples / 2 + 4 - 1) / 4;
+
+ return round_up(pic_width_in_cbsc * pic_height_in_cbsc, 16);
+}
+
static size_t hantro_hevc_ref_size(struct hantro_ctx *ctx)
{
const struct hantro_hevc_dec_ctrls *ctrls = &ctx->hevc_dec.ctrls;
const struct v4l2_ctrl_hevc_sps *sps = ctrls->sps;
- return hantro_hevc_motion_vectors_offset(sps) + hantro_hevc_mv_size(sps);
+ return hantro_hevc_motion_vectors_offset(sps) + hantro_hevc_mv_size(sps) +
+ hantro_hevc_luma_compress_size(sps) +
+ hantro_hevc_chroma_compress_size(sps);
}
static void hantro_hevc_ref_free(struct hantro_ctx *ctx)
diff --git a/drivers/staging/media/hantro/hantro_hw.h b/drivers/staging/media/hantro/hantro_hw.h
index 8fa0aacb61cd..c5374cd74d66 100644
--- a/drivers/staging/media/hantro/hantro_hw.h
+++ b/drivers/staging/media/hantro/hantro_hw.h
@@ -246,6 +246,8 @@ dma_addr_t hantro_hevc_get_ref_buf(struct hantro_ctx *ctx, int poc);
void hantro_hevc_ref_remove_unused(struct hantro_ctx *ctx);
size_t hantro_hevc_chroma_offset(const struct v4l2_ctrl_hevc_sps *sps);
size_t hantro_hevc_motion_vectors_offset(const struct v4l2_ctrl_hevc_sps *sps);
+size_t hantro_hevc_luma_compress_offset(const struct v4l2_ctrl_hevc_sps *sps);
+size_t hantro_hevc_chroma_compress_offset(const struct v4l2_ctrl_hevc_sps *sps);
static inline size_t
hantro_h264_mv_size(unsigned int width, unsigned int height)
--
2.25.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 21+ messages in thread
* [PATCH 3/8] media: hantro: hevc: Allow 10-bits encoded streams
2021-06-04 13:06 [PATCH 0/8] Additional features for Hantro HEVC Benjamin Gaignard
2021-06-04 13:06 ` [PATCH 1/8] media: hantro: Trace hevc hw cycles performance register Benjamin Gaignard
2021-06-04 13:06 ` [PATCH 2/8] media: hantro: Add support of compressed reference buffers Benjamin Gaignard
@ 2021-06-04 13:06 ` Benjamin Gaignard
2021-06-04 13:06 ` [PATCH 4/8] media: Add P010 video format Benjamin Gaignard
` (5 subsequent siblings)
8 siblings, 0 replies; 21+ messages in thread
From: Benjamin Gaignard @ 2021-06-04 13:06 UTC (permalink / raw)
To: hverkuil, ezequiel, p.zabel, mchehab, shawnguo, s.hauer,
festevam, gregkh, mripard, paul.kocialkowski, wens,
jernej.skrabec, emil.l.velikov, andrzej.p, jc
Cc: kernel, linux-imx, linux-media, linux-rockchip, linux-arm-kernel,
linux-kernel, Benjamin Gaignard
Allow to use 10-bits encoded streams but force the output
to remain in 8-bits.
Add a function to get chroma offset for the output since
it may now not match with internal reference buffers chroma
offset.
Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
---
drivers/staging/media/hantro/hantro_drv.c | 5 ++--
.../staging/media/hantro/hantro_g2_hevc_dec.c | 29 +++++++++++++++----
drivers/staging/media/hantro/hantro_g2_regs.h | 1 +
drivers/staging/media/hantro/hantro_hevc.c | 5 ++--
4 files changed, 31 insertions(+), 9 deletions(-)
diff --git a/drivers/staging/media/hantro/hantro_drv.c b/drivers/staging/media/hantro/hantro_drv.c
index 6053c86b1c3f..43feb14bbbba 100644
--- a/drivers/staging/media/hantro/hantro_drv.c
+++ b/drivers/staging/media/hantro/hantro_drv.c
@@ -263,8 +263,9 @@ static int hantro_try_ctrl(struct v4l2_ctrl *ctrl)
if (sps->bit_depth_luma_minus8 != sps->bit_depth_chroma_minus8)
/* Luma and chroma bit depth mismatch */
return -EINVAL;
- if (sps->bit_depth_luma_minus8 != 0)
- /* Only 8-bit is supported */
+ if (sps->bit_depth_luma_minus8 != 0 &&
+ sps->bit_depth_luma_minus8 != 2)
+ /* Only 8-bit or 10-bit is supported */
return -EINVAL;
if (sps->flags & V4L2_HEVC_SPS_FLAG_SCALING_LIST_ENABLED)
/* No scaling support */
diff --git a/drivers/staging/media/hantro/hantro_g2_hevc_dec.c b/drivers/staging/media/hantro/hantro_g2_hevc_dec.c
index 9a715e803037..df147b69a31f 100644
--- a/drivers/staging/media/hantro/hantro_g2_hevc_dec.c
+++ b/drivers/staging/media/hantro/hantro_g2_hevc_dec.c
@@ -133,6 +133,16 @@ static void prepare_tile_info_buffer(struct hantro_ctx *ctx)
vpu_debug(1, "%s: no chroma!\n", __func__);
}
+static bool is_8bit_dst_format(struct hantro_ctx *ctx)
+{
+ switch (ctx->vpu_dst_fmt->fourcc) {
+ case V4L2_PIX_FMT_NV12:
+ return true;
+ default:
+ return false;
+ }
+}
+
static void set_params(struct hantro_ctx *ctx)
{
const struct hantro_hevc_dec_ctrls *ctrls = &ctx->hevc_dec.ctrls;
@@ -148,7 +158,8 @@ static void set_params(struct hantro_ctx *ctx)
hantro_reg_write(vpu, &g2_bit_depth_y_minus8, sps->bit_depth_luma_minus8);
hantro_reg_write(vpu, &g2_bit_depth_c_minus8, sps->bit_depth_chroma_minus8);
- hantro_reg_write(vpu, &g2_output_8_bits, 0);
+ hantro_reg_write(vpu, &g2_output_8_bits, 1);
+ hantro_reg_write(vpu, &g2_output_format, 0);
hantro_reg_write(vpu, &g2_hdr_skip_length, ctrls->hevc_hdr_skip_length);
@@ -511,13 +522,21 @@ static int set_ref(struct hantro_ctx *ctx)
return 0;
}
+static size_t hantro_hevc_output_chroma_offset(struct hantro_ctx *ctx)
+{
+ const struct hantro_hevc_dec_ctrls *ctrls = &ctx->hevc_dec.ctrls;
+ const struct v4l2_ctrl_hevc_sps *sps = ctrls->sps;
+ int bytes_per_pixel = is_8bit_dst_format(ctx) ? 1 : 2;
+
+ return sps->pic_width_in_luma_samples *
+ sps->pic_height_in_luma_samples * bytes_per_pixel;
+}
+
static void set_buffers(struct hantro_ctx *ctx)
{
struct vb2_v4l2_buffer *src_buf, *dst_buf;
struct hantro_dev *vpu = ctx->dev;
- const struct hantro_hevc_dec_ctrls *ctrls = &ctx->hevc_dec.ctrls;
- const struct v4l2_ctrl_hevc_sps *sps = ctrls->sps;
- size_t cr_offset = hantro_hevc_chroma_offset(sps);
+ size_t output_cr_offset = hantro_hevc_output_chroma_offset(ctx);
dma_addr_t src_dma, dst_dma;
u32 src_len, src_buf_len;
@@ -539,7 +558,7 @@ static void set_buffers(struct hantro_ctx *ctx)
dst_dma = hantro_get_dec_buf_addr(ctx, &dst_buf->vb2_buf);
hantro_write_addr(vpu, G2_RASTER_SCAN, dst_dma);
- hantro_write_addr(vpu, G2_RASTER_SCAN_CHR, dst_dma + cr_offset);
+ hantro_write_addr(vpu, G2_RASTER_SCAN_CHR, dst_dma + output_cr_offset);
hantro_write_addr(vpu, G2_ADDR_TILE_SIZE, ctx->hevc_dec.tile_sizes.dma);
hantro_write_addr(vpu, G2_TILE_FILTER, ctx->hevc_dec.tile_filter.dma);
hantro_write_addr(vpu, G2_TILE_SAO, ctx->hevc_dec.tile_sao.dma);
diff --git a/drivers/staging/media/hantro/hantro_g2_regs.h b/drivers/staging/media/hantro/hantro_g2_regs.h
index 0414d92e3860..941e5482d27b 100644
--- a/drivers/staging/media/hantro/hantro_g2_regs.h
+++ b/drivers/staging/media/hantro/hantro_g2_regs.h
@@ -77,6 +77,7 @@
#define g2_bit_depth_y_minus8 G2_DEC_REG(8, 6, 0x3)
#define g2_bit_depth_c_minus8 G2_DEC_REG(8, 4, 0x3)
#define g2_output_8_bits G2_DEC_REG(8, 3, 0x1)
+#define g2_output_format G2_DEC_REG(8, 0, 0x7)
#define g2_refidx1_active G2_DEC_REG(9, 19, 0x1f)
#define g2_refidx0_active G2_DEC_REG(9, 14, 0x1f)
diff --git a/drivers/staging/media/hantro/hantro_hevc.c b/drivers/staging/media/hantro/hantro_hevc.c
index 1b2da990fbf0..1d44ea69c930 100644
--- a/drivers/staging/media/hantro/hantro_hevc.c
+++ b/drivers/staging/media/hantro/hantro_hevc.c
@@ -195,6 +195,7 @@ static int tile_buffer_reallocate(struct hantro_ctx *ctx)
const struct v4l2_ctrl_hevc_sps *sps = ctrls->sps;
unsigned int num_tile_cols = pps->num_tile_columns_minus1 + 1;
unsigned int height64 = (sps->pic_height_in_luma_samples + 63) & ~63;
+ unsigned int pixel_depth = sps->bit_depth_luma_minus8 == 0 ? 8 : 10;
unsigned int size;
if (num_tile_cols <= 1 ||
@@ -223,7 +224,7 @@ static int tile_buffer_reallocate(struct hantro_ctx *ctx)
hevc_dec->tile_bsd.cpu = NULL;
}
- size = VERT_FILTER_RAM_SIZE * height64 * (num_tile_cols - 1);
+ size = (VERT_FILTER_RAM_SIZE * height64 * (num_tile_cols - 1) * pixel_depth) / 8;
hevc_dec->tile_filter.cpu = dma_alloc_coherent(vpu->dev, size,
&hevc_dec->tile_filter.dma,
GFP_KERNEL);
@@ -231,7 +232,7 @@ static int tile_buffer_reallocate(struct hantro_ctx *ctx)
goto err_free_tile_buffers;
hevc_dec->tile_filter.size = size;
- size = VERT_SAO_RAM_SIZE * height64 * (num_tile_cols - 1);
+ size = (VERT_SAO_RAM_SIZE * height64 * (num_tile_cols - 1) * pixel_depth) / 8;
hevc_dec->tile_sao.cpu = dma_alloc_coherent(vpu->dev, size,
&hevc_dec->tile_sao.dma,
GFP_KERNEL);
--
2.25.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 21+ messages in thread
* [PATCH 4/8] media: Add P010 video format
2021-06-04 13:06 [PATCH 0/8] Additional features for Hantro HEVC Benjamin Gaignard
` (2 preceding siblings ...)
2021-06-04 13:06 ` [PATCH 3/8] media: hantro: hevc: Allow 10-bits encoded streams Benjamin Gaignard
@ 2021-06-04 13:06 ` Benjamin Gaignard
2021-06-04 16:17 ` Nicolas Dufresne
2021-06-04 13:06 ` [PATCH 5/8] media: hantro: hevc: Allow to produce 10-bit frames Benjamin Gaignard
` (4 subsequent siblings)
8 siblings, 1 reply; 21+ messages in thread
From: Benjamin Gaignard @ 2021-06-04 13:06 UTC (permalink / raw)
To: hverkuil, ezequiel, p.zabel, mchehab, shawnguo, s.hauer,
festevam, gregkh, mripard, paul.kocialkowski, wens,
jernej.skrabec, emil.l.velikov, andrzej.p, jc
Cc: kernel, linux-imx, linux-media, linux-rockchip, linux-arm-kernel,
linux-kernel, Benjamin Gaignard
P010 is a YUV format with 10-bits per pixel with interleaved UV.
Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
---
.../userspace-api/media/v4l/pixfmt-yuv-planar.rst | 8 ++++++++
drivers/media/v4l2-core/v4l2-common.c | 1 +
drivers/media/v4l2-core/v4l2-ioctl.c | 1 +
include/uapi/linux/videodev2.h | 1 +
4 files changed, 11 insertions(+)
diff --git a/Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst b/Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst
index 090c091affd2..71fed70c03ec 100644
--- a/Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst
+++ b/Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst
@@ -100,6 +100,13 @@ All components are stored with the same number of bits per component.
- Cb, Cr
- No
- 64x32 macroblocks
+ * - V4L2_PIX_FMT_P010
+ - 'P010'
+ - 10
+ - 4:2:0
+ - Cb, Cr
+ - No
+ - Linear
Horizontal Z order
* - V4L2_PIX_FMT_NV12MT_16X16
@@ -171,6 +178,7 @@ horizontally.
.. _V4L2-PIX-FMT-NV21:
.. _V4L2-PIX-FMT-NV12M:
.. _V4L2-PIX-FMT-NV21M:
+.. _V4L2-PIX-FMT-P010:
NV12, NV21, NV12M and NV21M
---------------------------
diff --git a/drivers/media/v4l2-core/v4l2-common.c b/drivers/media/v4l2-core/v4l2-common.c
index 04af03285a20..37b5d82359dd 100644
--- a/drivers/media/v4l2-core/v4l2-common.c
+++ b/drivers/media/v4l2-core/v4l2-common.c
@@ -266,6 +266,7 @@ const struct v4l2_format_info *v4l2_format_info(u32 format)
{ .format = V4L2_PIX_FMT_NV61, .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 1, .comp_planes = 2, .bpp = { 1, 2, 0, 0 }, .hdiv = 2, .vdiv = 1 },
{ .format = V4L2_PIX_FMT_NV24, .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 1, .comp_planes = 2, .bpp = { 1, 2, 0, 0 }, .hdiv = 1, .vdiv = 1 },
{ .format = V4L2_PIX_FMT_NV42, .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 1, .comp_planes = 2, .bpp = { 1, 2, 0, 0 }, .hdiv = 1, .vdiv = 1 },
+ { .format = V4L2_PIX_FMT_P010, .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 1, .comp_planes = 2, .bpp = { 2, 2, 0, 0 }, .hdiv = 2, .vdiv = 1 },
{ .format = V4L2_PIX_FMT_YUV410, .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 1, .comp_planes = 3, .bpp = { 1, 1, 1, 0 }, .hdiv = 4, .vdiv = 4 },
{ .format = V4L2_PIX_FMT_YVU410, .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 1, .comp_planes = 3, .bpp = { 1, 1, 1, 0 }, .hdiv = 4, .vdiv = 4 },
diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
index 2673f51aafa4..6404d5b6e350 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -1282,6 +1282,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
case V4L2_PIX_FMT_NV61: descr = "Y/CrCb 4:2:2"; break;
case V4L2_PIX_FMT_NV24: descr = "Y/CbCr 4:4:4"; break;
case V4L2_PIX_FMT_NV42: descr = "Y/CrCb 4:4:4"; break;
+ case V4L2_PIX_FMT_P010: descr = "10-bit Y/CrCb 4:2:0"; break;
case V4L2_PIX_FMT_NV12M: descr = "Y/CbCr 4:2:0 (N-C)"; break;
case V4L2_PIX_FMT_NV21M: descr = "Y/CrCb 4:2:0 (N-C)"; break;
case V4L2_PIX_FMT_NV16M: descr = "Y/CbCr 4:2:2 (N-C)"; break;
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 9260791b8438..e5f7acde0730 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -602,6 +602,7 @@ struct v4l2_pix_format {
#define V4L2_PIX_FMT_NV24 v4l2_fourcc('N', 'V', '2', '4') /* 24 Y/CbCr 4:4:4 */
#define V4L2_PIX_FMT_NV42 v4l2_fourcc('N', 'V', '4', '2') /* 24 Y/CrCb 4:4:4 */
#define V4L2_PIX_FMT_HM12 v4l2_fourcc('H', 'M', '1', '2') /* 8 YUV 4:2:0 16x16 macroblocks */
+#define V4L2_PIX_FMT_P010 v4l2_fourcc('P', '0', '1', '0') /* 15 Y/CbCr 4:2:0 10-bit per pixel*/
/* two non contiguous planes - one Y, one Cr + Cb interleaved */
#define V4L2_PIX_FMT_NV12M v4l2_fourcc('N', 'M', '1', '2') /* 12 Y/CbCr 4:2:0 */
--
2.25.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 21+ messages in thread
* [PATCH 5/8] media: hantro: hevc: Allow to produce 10-bit frames
2021-06-04 13:06 [PATCH 0/8] Additional features for Hantro HEVC Benjamin Gaignard
` (3 preceding siblings ...)
2021-06-04 13:06 ` [PATCH 4/8] media: Add P010 video format Benjamin Gaignard
@ 2021-06-04 13:06 ` Benjamin Gaignard
2021-06-04 13:06 ` [PATCH 6/8] media: hantro: enumerate scaled output formats Benjamin Gaignard
` (3 subsequent siblings)
8 siblings, 0 replies; 21+ messages in thread
From: Benjamin Gaignard @ 2021-06-04 13:06 UTC (permalink / raw)
To: hverkuil, ezequiel, p.zabel, mchehab, shawnguo, s.hauer,
festevam, gregkh, mripard, paul.kocialkowski, wens,
jernej.skrabec, emil.l.velikov, andrzej.p, jc
Cc: kernel, linux-imx, linux-media, linux-rockchip, linux-arm-kernel,
linux-kernel, Benjamin Gaignard
If Hantro driver receive an 10-bit encoded bitstream allow it
to produce 10-bit frames.
Check that we are not try to produce 10-bit frames from a 8-bit
encoded bitstream.
Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
---
drivers/staging/media/hantro/hantro_drv.c | 18 ++++++++++++++++++
.../staging/media/hantro/hantro_g2_hevc_dec.c | 18 ++++++++++++++----
drivers/staging/media/hantro/hantro_hevc.c | 2 +-
drivers/staging/media/hantro/imx8m_vpu_hw.c | 4 ++++
4 files changed, 37 insertions(+), 5 deletions(-)
diff --git a/drivers/staging/media/hantro/hantro_drv.c b/drivers/staging/media/hantro/hantro_drv.c
index 43feb14bbbba..5e6609fa4143 100644
--- a/drivers/staging/media/hantro/hantro_drv.c
+++ b/drivers/staging/media/hantro/hantro_drv.c
@@ -243,6 +243,16 @@ queue_init(void *priv, struct vb2_queue *src_vq, struct vb2_queue *dst_vq)
return vb2_queue_init(dst_vq);
}
+static bool hantro_is_10bit_dst_format(struct hantro_ctx *ctx)
+{
+ switch (ctx->vpu_dst_fmt->fourcc) {
+ case V4L2_PIX_FMT_P010:
+ return true;
+ default:
+ return false;
+ }
+}
+
static int hantro_try_ctrl(struct v4l2_ctrl *ctrl)
{
if (ctrl->id == V4L2_CID_STATELESS_H264_SPS) {
@@ -259,6 +269,10 @@ static int hantro_try_ctrl(struct v4l2_ctrl *ctrl)
return -EINVAL;
} else if (ctrl->id == V4L2_CID_MPEG_VIDEO_HEVC_SPS) {
const struct v4l2_ctrl_hevc_sps *sps = ctrl->p_new.p_hevc_sps;
+ struct hantro_ctx *ctx;
+
+ ctx = container_of(ctrl->handler,
+ struct hantro_ctx, ctrl_handler);
if (sps->bit_depth_luma_minus8 != sps->bit_depth_chroma_minus8)
/* Luma and chroma bit depth mismatch */
@@ -270,6 +284,10 @@ static int hantro_try_ctrl(struct v4l2_ctrl *ctrl)
if (sps->flags & V4L2_HEVC_SPS_FLAG_SCALING_LIST_ENABLED)
/* No scaling support */
return -EINVAL;
+ if (sps->bit_depth_luma_minus8 == 0 &&
+ hantro_is_10bit_dst_format(ctx)) {
+ return -EINVAL;
+ }
}
return 0;
}
diff --git a/drivers/staging/media/hantro/hantro_g2_hevc_dec.c b/drivers/staging/media/hantro/hantro_g2_hevc_dec.c
index df147b69a31f..b171b28ec24c 100644
--- a/drivers/staging/media/hantro/hantro_g2_hevc_dec.c
+++ b/drivers/staging/media/hantro/hantro_g2_hevc_dec.c
@@ -143,6 +143,16 @@ static bool is_8bit_dst_format(struct hantro_ctx *ctx)
}
}
+static int get_dst_format(struct hantro_ctx *ctx)
+{
+ switch (ctx->vpu_dst_fmt->fourcc) {
+ case V4L2_PIX_FMT_P010:
+ return 0x1;
+ default:
+ return 0x0;
+ }
+}
+
static void set_params(struct hantro_ctx *ctx)
{
const struct hantro_hevc_dec_ctrls *ctrls = &ctx->hevc_dec.ctrls;
@@ -158,8 +168,8 @@ static void set_params(struct hantro_ctx *ctx)
hantro_reg_write(vpu, &g2_bit_depth_y_minus8, sps->bit_depth_luma_minus8);
hantro_reg_write(vpu, &g2_bit_depth_c_minus8, sps->bit_depth_chroma_minus8);
- hantro_reg_write(vpu, &g2_output_8_bits, 1);
- hantro_reg_write(vpu, &g2_output_format, 0);
+ hantro_reg_write(vpu, &g2_output_8_bits, is_8bit_dst_format(ctx));
+ hantro_reg_write(vpu, &g2_output_format, get_dst_format(ctx));
hantro_reg_write(vpu, &g2_hdr_skip_length, ctrls->hevc_hdr_skip_length);
@@ -529,7 +539,7 @@ static size_t hantro_hevc_output_chroma_offset(struct hantro_ctx *ctx)
int bytes_per_pixel = is_8bit_dst_format(ctx) ? 1 : 2;
return sps->pic_width_in_luma_samples *
- sps->pic_height_in_luma_samples * bytes_per_pixel;
+ sps->pic_height_in_luma_samples * bytes_per_pixel;
}
static void set_buffers(struct hantro_ctx *ctx)
@@ -616,7 +626,7 @@ int hantro_g2_hevc_dec_run(struct hantro_ctx *ctx)
/* Compress buffers */
hantro_reg_write(vpu, &g2_ref_compress_bypass, 0);
- /* use NV12 as output format */
+ /* Use raster-scan as output format */
hantro_reg_write(vpu, &g2_out_rs_e, 1);
/* Bus width and max burst */
diff --git a/drivers/staging/media/hantro/hantro_hevc.c b/drivers/staging/media/hantro/hantro_hevc.c
index 1d44ea69c930..ae9827dd7086 100644
--- a/drivers/staging/media/hantro/hantro_hevc.c
+++ b/drivers/staging/media/hantro/hantro_hevc.c
@@ -25,7 +25,7 @@
#define UNUSED_REF -1
-#define G2_ALIGN 16
+#define G2_ALIGN 16
size_t hantro_hevc_chroma_offset(const struct v4l2_ctrl_hevc_sps *sps)
{
diff --git a/drivers/staging/media/hantro/imx8m_vpu_hw.c b/drivers/staging/media/hantro/imx8m_vpu_hw.c
index 7e9e24bb5057..b2e85512455b 100644
--- a/drivers/staging/media/hantro/imx8m_vpu_hw.c
+++ b/drivers/staging/media/hantro/imx8m_vpu_hw.c
@@ -136,6 +136,10 @@ static const struct hantro_fmt imx8m_vpu_g2_dec_fmts[] = {
.fourcc = V4L2_PIX_FMT_NV12,
.codec_mode = HANTRO_MODE_NONE,
},
+ {
+ .fourcc = V4L2_PIX_FMT_P010,
+ .codec_mode = HANTRO_MODE_NONE,
+ },
{
.fourcc = V4L2_PIX_FMT_HEVC_SLICE,
.codec_mode = HANTRO_MODE_HEVC_DEC,
--
2.25.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 21+ messages in thread
* [PATCH 6/8] media: hantro: enumerate scaled output formats
2021-06-04 13:06 [PATCH 0/8] Additional features for Hantro HEVC Benjamin Gaignard
` (4 preceding siblings ...)
2021-06-04 13:06 ` [PATCH 5/8] media: hantro: hevc: Allow to produce 10-bit frames Benjamin Gaignard
@ 2021-06-04 13:06 ` Benjamin Gaignard
2021-06-04 13:06 ` [PATCH 7/8] media: hevc: Add scaling matrix control Benjamin Gaignard
` (2 subsequent siblings)
8 siblings, 0 replies; 21+ messages in thread
From: Benjamin Gaignard @ 2021-06-04 13:06 UTC (permalink / raw)
To: hverkuil, ezequiel, p.zabel, mchehab, shawnguo, s.hauer,
festevam, gregkh, mripard, paul.kocialkowski, wens,
jernej.skrabec, emil.l.velikov, andrzej.p, jc
Cc: kernel, linux-imx, linux-media, linux-rockchip, linux-arm-kernel,
linux-kernel, Benjamin Gaignard
When enumerating the output formats take care of the hardware scaling
capabilities.
For a given input size G2 hardware block is capable of down scale the
output by 2, 4 or 8 factor. When decoding 4K streams that to be could
helpful to save memory bandwidth.
Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
---
drivers/staging/media/hantro/hantro.h | 4 ++
.../staging/media/hantro/hantro_g2_hevc_dec.c | 46 ++++++++++++++++++-
drivers/staging/media/hantro/hantro_g2_regs.h | 6 +++
drivers/staging/media/hantro/hantro_hw.h | 1 +
drivers/staging/media/hantro/hantro_v4l2.c | 10 ++--
drivers/staging/media/hantro/imx8m_vpu_hw.c | 1 +
6 files changed, 63 insertions(+), 5 deletions(-)
diff --git a/drivers/staging/media/hantro/hantro.h b/drivers/staging/media/hantro/hantro.h
index a70c386de6f1..fc7377783787 100644
--- a/drivers/staging/media/hantro/hantro.h
+++ b/drivers/staging/media/hantro/hantro.h
@@ -70,6 +70,9 @@ struct hantro_irq {
* @reg_names: array of register range names
* @num_regs: number of register range names in the array
* @postproc_regs: &struct hantro_postproc_regs pointer
+ * @scaling: Set possible scaled output formats.
+ * Returns zero if OK, a negative value in error cases.
+ * Optional.
*/
struct hantro_variant {
unsigned int enc_offset;
@@ -91,6 +94,7 @@ struct hantro_variant {
const char * const *reg_names;
int num_regs;
const struct hantro_postproc_regs *postproc_regs;
+ int (*scaling)(struct hantro_ctx *ctx, struct v4l2_frmsizeenum *fsize);
};
/**
diff --git a/drivers/staging/media/hantro/hantro_g2_hevc_dec.c b/drivers/staging/media/hantro/hantro_g2_hevc_dec.c
index b171b28ec24c..741b029aef9e 100644
--- a/drivers/staging/media/hantro/hantro_g2_hevc_dec.c
+++ b/drivers/staging/media/hantro/hantro_g2_hevc_dec.c
@@ -396,6 +396,17 @@ static void set_ref_pic_list(struct hantro_ctx *ctx)
}
}
+static int down_scale_factor(struct hantro_ctx *ctx)
+{
+ const struct hantro_hevc_dec_ctrls *ctrls = &ctx->hevc_dec.ctrls;
+ const struct v4l2_ctrl_hevc_sps *sps = ctrls->sps;
+
+ if (sps->pic_width_in_luma_samples == ctx->dst_fmt.width)
+ return 0;
+
+ return DIV_ROUND_CLOSEST(sps->pic_width_in_luma_samples, ctx->dst_fmt.width);
+}
+
static int set_ref(struct hantro_ctx *ctx)
{
const struct hantro_hevc_dec_ctrls *ctrls = &ctx->hevc_dec.ctrls;
@@ -409,6 +420,7 @@ static int set_ref(struct hantro_ctx *ctx)
size_t mv_offset = hantro_hevc_motion_vectors_offset(sps);
size_t compress_luma_offset = hantro_hevc_luma_compress_offset(sps);
size_t compress_chroma_offset = hantro_hevc_chroma_compress_offset(sps);
+ int down_scale = down_scale_factor(ctx);
u32 max_ref_frames;
u16 dpb_longterm_e;
static const struct hantro_reg cur_poc[] = {
@@ -511,8 +523,18 @@ static int set_ref(struct hantro_ctx *ctx)
hantro_write_addr(vpu, G2_REG_CHR_REF(i), chroma_addr);
hantro_write_addr(vpu, G2_REG_DMV_REF(i++), mv_addr);
- hantro_write_addr(vpu, G2_ADDR_DST, luma_addr);
- hantro_write_addr(vpu, G2_ADDR_DST_CHR, chroma_addr);
+ if (down_scale) {
+ chroma_addr = luma_addr + (cr_offset >> down_scale);
+ hantro_reg_write(vpu, &g2_down_scale_e, 1);
+ hantro_reg_write(vpu, &g2_down_scale_y, down_scale >> 2);
+ hantro_reg_write(vpu, &g2_down_scale_x, down_scale >> 2);
+ hantro_write_addr(vpu, G2_DS_DST, luma_addr);
+ hantro_write_addr(vpu, G2_DS_DST_CHR, chroma_addr);
+ } else {
+ hantro_write_addr(vpu, G2_ADDR_DST, luma_addr);
+ hantro_write_addr(vpu, G2_ADDR_DST_CHR, chroma_addr);
+ }
+
hantro_write_addr(vpu, G2_ADDR_DST_MV, mv_addr);
hantro_write_addr(vpu, G2_COMP_ADDR_DST, compress_luma_addr);
hantro_write_addr(vpu, G2_COMP_CHR, compress_chroma_addr);
@@ -592,6 +614,26 @@ static void hantro_g2_check_idle(struct hantro_dev *vpu)
}
}
+int hantro_g2_hevc_dec_scaling(struct hantro_ctx *ctx,
+ struct v4l2_frmsizeenum *fsize)
+{
+ /**
+ * G2 scaler can scale down by 0, 2, 4 or 8
+ * use fsize->index has power of 2 diviser
+ **/
+ if (fsize->index > 3)
+ return -EINVAL;
+
+ if (!ctx->src_fmt.width || !ctx->src_fmt.height)
+ return -EINVAL;
+
+ fsize->type = V4L2_FRMSIZE_TYPE_DISCRETE;
+ fsize->discrete.width = ctx->src_fmt.width >> fsize->index;
+ fsize->discrete.height = ctx->src_fmt.height >> fsize->index;
+
+ return 0;
+}
+
int hantro_g2_hevc_dec_run(struct hantro_ctx *ctx)
{
struct hantro_dev *vpu = ctx->dev;
diff --git a/drivers/staging/media/hantro/hantro_g2_regs.h b/drivers/staging/media/hantro/hantro_g2_regs.h
index 941e5482d27b..54f3d78ce46e 100644
--- a/drivers/staging/media/hantro/hantro_g2_regs.h
+++ b/drivers/staging/media/hantro/hantro_g2_regs.h
@@ -174,6 +174,10 @@
#define g2_buswidth G2_DEC_REG(58, 8, 0x7)
#define g2_max_burst G2_DEC_REG(58, 0, 0xff)
+#define g2_down_scale_e G2_DEC_REG(184, 7, 0x1)
+#define g2_down_scale_y G2_DEC_REG(184, 2, 0x3)
+#define g2_down_scale_x G2_DEC_REG(184, 0, 0x3)
+
#define G2_REG_CONFIG G2_SWREG(58)
#define G2_REG_CONFIG_DEC_CLK_GATE_E BIT(16)
#define G2_REG_CONFIG_DEC_CLK_GATE_IDLE_E BIT(17)
@@ -193,6 +197,8 @@
#define G2_TILE_FILTER (G2_SWREG(179))
#define G2_TILE_SAO (G2_SWREG(181))
#define G2_TILE_BSD (G2_SWREG(183))
+#define G2_DS_DST (G2_SWREG(186))
+#define G2_DS_DST_CHR (G2_SWREG(188))
#define G2_COMP_ADDR_DST (G2_SWREG(190))
#define G2_COMP_ADDR_REF(i) (G2_SWREG(192) + ((i) * 0x8))
#define G2_COMP_CHR (G2_SWREG(224))
diff --git a/drivers/staging/media/hantro/hantro_hw.h b/drivers/staging/media/hantro/hantro_hw.h
index c5374cd74d66..2edb890f10af 100644
--- a/drivers/staging/media/hantro/hantro_hw.h
+++ b/drivers/staging/media/hantro/hantro_hw.h
@@ -241,6 +241,7 @@ int hantro_hevc_dec_init(struct hantro_ctx *ctx);
void hantro_hevc_dec_exit(struct hantro_ctx *ctx);
int hantro_g2_hevc_dec_run(struct hantro_ctx *ctx);
void hantro_g2_hevc_dec_done(struct hantro_ctx *ctx);
+int hantro_g2_hevc_dec_scaling(struct hantro_ctx *ctx, struct v4l2_frmsizeenum *fsize);
int hantro_hevc_dec_prepare_run(struct hantro_ctx *ctx);
dma_addr_t hantro_hevc_get_ref_buf(struct hantro_ctx *ctx, int poc);
void hantro_hevc_ref_remove_unused(struct hantro_ctx *ctx);
diff --git a/drivers/staging/media/hantro/hantro_v4l2.c b/drivers/staging/media/hantro/hantro_v4l2.c
index bcb0bdff4a9a..566b645f661d 100644
--- a/drivers/staging/media/hantro/hantro_v4l2.c
+++ b/drivers/staging/media/hantro/hantro_v4l2.c
@@ -118,7 +118,7 @@ static int vidioc_enum_framesizes(struct file *file, void *priv,
struct hantro_ctx *ctx = fh_to_ctx(priv);
const struct hantro_fmt *fmt;
- if (fsize->index != 0) {
+ if (fsize->index != 0 && !ctx->dev->variant->scaling) {
vpu_debug(0, "invalid frame size index (expected 0, got %d)\n",
fsize->index);
return -EINVAL;
@@ -131,9 +131,13 @@ static int vidioc_enum_framesizes(struct file *file, void *priv,
return -EINVAL;
}
- /* This only makes sense for coded formats */
- if (fmt->codec_mode == HANTRO_MODE_NONE)
+ /* For non-coded formats check if scaling is possible */
+ if (fmt->codec_mode == HANTRO_MODE_NONE) {
+ if (ctx->dev->variant->scaling)
+ return ctx->dev->variant->scaling(ctx, fsize);
+
return -EINVAL;
+ }
fsize->type = V4L2_FRMSIZE_TYPE_STEPWISE;
fsize->stepwise = fmt->frmsize;
diff --git a/drivers/staging/media/hantro/imx8m_vpu_hw.c b/drivers/staging/media/hantro/imx8m_vpu_hw.c
index b2e85512455b..7b2b1c8d9fab 100644
--- a/drivers/staging/media/hantro/imx8m_vpu_hw.c
+++ b/drivers/staging/media/hantro/imx8m_vpu_hw.c
@@ -293,4 +293,5 @@ const struct hantro_variant imx8mq_vpu_g2_variant = {
.num_irqs = ARRAY_SIZE(imx8mq_g2_irqs),
.clk_names = imx8mq_clk_names,
.num_clocks = ARRAY_SIZE(imx8mq_clk_names),
+ .scaling = hantro_g2_hevc_dec_scaling,
};
--
2.25.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 21+ messages in thread
* [PATCH 7/8] media: hevc: Add scaling matrix control
2021-06-04 13:06 [PATCH 0/8] Additional features for Hantro HEVC Benjamin Gaignard
` (5 preceding siblings ...)
2021-06-04 13:06 ` [PATCH 6/8] media: hantro: enumerate scaled output formats Benjamin Gaignard
@ 2021-06-04 13:06 ` Benjamin Gaignard
2021-06-06 7:49 ` Jernej Škrabec
2021-06-04 13:06 ` [PATCH 8/8] media: hantro: Add scaling lists feature Benjamin Gaignard
2021-06-04 16:49 ` [PATCH 0/8] Additional features for Hantro HEVC Ezequiel Garcia
8 siblings, 1 reply; 21+ messages in thread
From: Benjamin Gaignard @ 2021-06-04 13:06 UTC (permalink / raw)
To: hverkuil, ezequiel, p.zabel, mchehab, shawnguo, s.hauer,
festevam, gregkh, mripard, paul.kocialkowski, wens,
jernej.skrabec, emil.l.velikov, andrzej.p, jc
Cc: kernel, linux-imx, linux-media, linux-rockchip, linux-arm-kernel,
linux-kernel, Benjamin Gaignard
HEVC scaling lists are used for the scaling process for transform
coefficients.
V4L2_HEVC_SPS_FLAG_SCALING_LIST_ENABLED has to set when they are
encoded in the bitstream.
Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
---
.../media/v4l/ext-ctrls-codec.rst | 45 +++++++++++++++++++
.../media/v4l/vidioc-queryctrl.rst | 6 +++
drivers/media/v4l2-core/v4l2-ctrls-core.c | 6 +++
drivers/media/v4l2-core/v4l2-ctrls-defs.c | 4 ++
include/media/hevc-ctrls.h | 11 +++++
5 files changed, 72 insertions(+)
diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
index 9120c5bcaf90..a4512b7cb520 100644
--- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
+++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
@@ -3065,6 +3065,51 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
\normalsize
+``V4L2_CID_MPEG_VIDEO_HEVC_SCALING_MATRIX (struct)``
+ Specifies the HEVC scaling matrix parameters used for the scaling process
+ for transform coefficients.
+ These matrix and parameters are defined according to :ref:`hevc`.
+ They are described in section 7.4.5 "Scaling list data semantics" of
+ the specification.
+
+.. c:type:: v4l2_ctrl_hevc_scaling_matrix
+
+.. raw:: latex
+
+ \scriptsize
+
+.. tabularcolumns:: |p{5.4cm}|p{6.8cm}|p{5.1cm}|
+
+.. cssclass:: longtable
+
+.. flat-table:: struct v4l2_ctrl_hevc_slice_params
+ :header-rows: 0
+ :stub-columns: 0
+ :widths: 1 1 2
+
+ * - __u8
+ - ``scaling_list_4x4[6][16]``
+ -
+ * - __u8
+ - ``scaling_list_8x8[6][64]``
+ -
+ * - __u8
+ - ``scaling_list_16x16[6][64]``
+ -
+ * - __u8
+ - ``scaling_list_32x32[2][64]``
+ -
+ * - __u8
+ - ``scaling_list_dc_coef_16x16[6]``
+ -
+ * - __u8
+ - ``scaling_list_dc_coef_32x32[2]``
+ -
+
+.. raw:: latex
+
+ \normalsize
+
.. c:type:: v4l2_hevc_dpb_entry
.. raw:: latex
diff --git a/Documentation/userspace-api/media/v4l/vidioc-queryctrl.rst b/Documentation/userspace-api/media/v4l/vidioc-queryctrl.rst
index f9ecf6276129..2f491c17dd5d 100644
--- a/Documentation/userspace-api/media/v4l/vidioc-queryctrl.rst
+++ b/Documentation/userspace-api/media/v4l/vidioc-queryctrl.rst
@@ -495,6 +495,12 @@ See also the examples in :ref:`control`.
- n/a
- A struct :c:type:`v4l2_ctrl_hevc_slice_params`, containing HEVC
slice parameters for stateless video decoders.
+ * - ``V4L2_CTRL_TYPE_HEVC_SCALING_MATRIX``
+ - n/a
+ - n/a
+ - n/a
+ - A struct :c:type:`v4l2_ctrl_hevc_scaling_matrix`, containing HEVC
+ scaling matrix for stateless video decoders.
* - ``V4L2_CTRL_TYPE_VP8_FRAME``
- n/a
- n/a
diff --git a/drivers/media/v4l2-core/v4l2-ctrls-core.c b/drivers/media/v4l2-core/v4l2-ctrls-core.c
index c4b5082849b6..70adfc1b9c81 100644
--- a/drivers/media/v4l2-core/v4l2-ctrls-core.c
+++ b/drivers/media/v4l2-core/v4l2-ctrls-core.c
@@ -687,6 +687,9 @@ static int std_validate_compound(const struct v4l2_ctrl *ctrl, u32 idx,
break;
+ case V4L2_CTRL_TYPE_HEVC_SCALING_MATRIX:
+ break;
+
case V4L2_CTRL_TYPE_AREA:
area = p;
if (!area->width || !area->height)
@@ -1240,6 +1243,9 @@ static struct v4l2_ctrl *v4l2_ctrl_new(struct v4l2_ctrl_handler *hdl,
case V4L2_CTRL_TYPE_HEVC_SLICE_PARAMS:
elem_size = sizeof(struct v4l2_ctrl_hevc_slice_params);
break;
+ case V4L2_CTRL_TYPE_HEVC_SCALING_MATRIX:
+ elem_size = sizeof(struct v4l2_ctrl_hevc_scaling_matrix);
+ break;
case V4L2_CTRL_TYPE_HEVC_DECODE_PARAMS:
elem_size = sizeof(struct v4l2_ctrl_hevc_decode_params);
break;
diff --git a/drivers/media/v4l2-core/v4l2-ctrls-defs.c b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
index b6344bbf1e00..cb29c2a7fabe 100644
--- a/drivers/media/v4l2-core/v4l2-ctrls-defs.c
+++ b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
@@ -996,6 +996,7 @@ const char *v4l2_ctrl_get_name(u32 id)
case V4L2_CID_MPEG_VIDEO_HEVC_SPS: return "HEVC Sequence Parameter Set";
case V4L2_CID_MPEG_VIDEO_HEVC_PPS: return "HEVC Picture Parameter Set";
case V4L2_CID_MPEG_VIDEO_HEVC_SLICE_PARAMS: return "HEVC Slice Parameters";
+ case V4L2_CID_MPEG_VIDEO_HEVC_SCALING_MATRIX: return "HEVC Scaling Matrix";
case V4L2_CID_MPEG_VIDEO_HEVC_DECODE_PARAMS: return "HEVC Decode Parameters";
case V4L2_CID_MPEG_VIDEO_HEVC_DECODE_MODE: return "HEVC Decode Mode";
case V4L2_CID_MPEG_VIDEO_HEVC_START_CODE: return "HEVC Start Code";
@@ -1488,6 +1489,9 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type,
case V4L2_CID_MPEG_VIDEO_HEVC_SLICE_PARAMS:
*type = V4L2_CTRL_TYPE_HEVC_SLICE_PARAMS;
break;
+ case V4L2_CID_MPEG_VIDEO_HEVC_SCALING_MATRIX:
+ *type = V4L2_CTRL_TYPE_HEVC_SCALING_MATRIX;
+ break;
case V4L2_CID_MPEG_VIDEO_HEVC_DECODE_PARAMS:
*type = V4L2_CTRL_TYPE_HEVC_DECODE_PARAMS;
break;
diff --git a/include/media/hevc-ctrls.h b/include/media/hevc-ctrls.h
index dc964ff7cd29..da6189ef7ab4 100644
--- a/include/media/hevc-ctrls.h
+++ b/include/media/hevc-ctrls.h
@@ -19,6 +19,7 @@
#define V4L2_CID_MPEG_VIDEO_HEVC_SPS (V4L2_CID_CODEC_BASE + 1008)
#define V4L2_CID_MPEG_VIDEO_HEVC_PPS (V4L2_CID_CODEC_BASE + 1009)
#define V4L2_CID_MPEG_VIDEO_HEVC_SLICE_PARAMS (V4L2_CID_CODEC_BASE + 1010)
+#define V4L2_CID_MPEG_VIDEO_HEVC_SCALING_MATRIX (V4L2_CID_CODEC_BASE + 1011)
#define V4L2_CID_MPEG_VIDEO_HEVC_DECODE_PARAMS (V4L2_CID_CODEC_BASE + 1012)
#define V4L2_CID_MPEG_VIDEO_HEVC_DECODE_MODE (V4L2_CID_CODEC_BASE + 1015)
#define V4L2_CID_MPEG_VIDEO_HEVC_START_CODE (V4L2_CID_CODEC_BASE + 1016)
@@ -27,6 +28,7 @@
#define V4L2_CTRL_TYPE_HEVC_SPS 0x0120
#define V4L2_CTRL_TYPE_HEVC_PPS 0x0121
#define V4L2_CTRL_TYPE_HEVC_SLICE_PARAMS 0x0122
+#define V4L2_CTRL_TYPE_HEVC_SCALING_MATRIX 0x0123
#define V4L2_CTRL_TYPE_HEVC_DECODE_PARAMS 0x0124
enum v4l2_mpeg_video_hevc_decode_mode {
@@ -225,6 +227,15 @@ struct v4l2_ctrl_hevc_decode_params {
__u64 flags;
};
+struct v4l2_ctrl_hevc_scaling_matrix {
+ __u8 scaling_list_4x4[6][16];
+ __u8 scaling_list_8x8[6][64];
+ __u8 scaling_list_16x16[6][64];
+ __u8 scaling_list_32x32[2][64];
+ __u8 scaling_list_dc_coef_16x16[6];
+ __u8 scaling_list_dc_coef_32x32[2];
+};
+
/* MPEG-class control IDs specific to the Hantro driver as defined by V4L2 */
#define V4L2_CID_CODEC_HANTRO_BASE (V4L2_CTRL_CLASS_CODEC | 0x1200)
/*
--
2.25.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 21+ messages in thread
* [PATCH 8/8] media: hantro: Add scaling lists feature
2021-06-04 13:06 [PATCH 0/8] Additional features for Hantro HEVC Benjamin Gaignard
` (6 preceding siblings ...)
2021-06-04 13:06 ` [PATCH 7/8] media: hevc: Add scaling matrix control Benjamin Gaignard
@ 2021-06-04 13:06 ` Benjamin Gaignard
2021-06-04 16:49 ` [PATCH 0/8] Additional features for Hantro HEVC Ezequiel Garcia
8 siblings, 0 replies; 21+ messages in thread
From: Benjamin Gaignard @ 2021-06-04 13:06 UTC (permalink / raw)
To: hverkuil, ezequiel, p.zabel, mchehab, shawnguo, s.hauer,
festevam, gregkh, mripard, paul.kocialkowski, wens,
jernej.skrabec, emil.l.velikov, andrzej.p, jc
Cc: kernel, linux-imx, linux-media, linux-rockchip, linux-arm-kernel,
linux-kernel, Benjamin Gaignard
If the bitstream embedded scaling lists allow the driver to use
them for decode the frames.
The scaling lists are expected to be in raster scan order (i.e. not up
right diagonal scan order)
Allocate the memory needed to store lists.
Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
---
drivers/staging/media/hantro/hantro_drv.c | 8 +--
.../staging/media/hantro/hantro_g2_hevc_dec.c | 52 +++++++++++++++++++
drivers/staging/media/hantro/hantro_hevc.c | 21 ++++++++
drivers/staging/media/hantro/hantro_hw.h | 3 ++
4 files changed, 81 insertions(+), 3 deletions(-)
diff --git a/drivers/staging/media/hantro/hantro_drv.c b/drivers/staging/media/hantro/hantro_drv.c
index 5e6609fa4143..324619c78ada 100644
--- a/drivers/staging/media/hantro/hantro_drv.c
+++ b/drivers/staging/media/hantro/hantro_drv.c
@@ -281,9 +281,6 @@ static int hantro_try_ctrl(struct v4l2_ctrl *ctrl)
sps->bit_depth_luma_minus8 != 2)
/* Only 8-bit or 10-bit is supported */
return -EINVAL;
- if (sps->flags & V4L2_HEVC_SPS_FLAG_SCALING_LIST_ENABLED)
- /* No scaling support */
- return -EINVAL;
if (sps->bit_depth_luma_minus8 == 0 &&
hantro_is_10bit_dst_format(ctx)) {
return -EINVAL;
@@ -469,6 +466,11 @@ static const struct hantro_ctrl controls[] = {
.cfg = {
.id = V4L2_CID_MPEG_VIDEO_HEVC_DECODE_PARAMS,
},
+ }, {
+ .codec = HANTRO_HEVC_DECODER,
+ .cfg = {
+ .id = V4L2_CID_MPEG_VIDEO_HEVC_SCALING_MATRIX,
+ },
}, {
.codec = HANTRO_HEVC_DECODER,
.cfg = {
diff --git a/drivers/staging/media/hantro/hantro_g2_hevc_dec.c b/drivers/staging/media/hantro/hantro_g2_hevc_dec.c
index 741b029aef9e..f7be8629c3cc 100644
--- a/drivers/staging/media/hantro/hantro_g2_hevc_dec.c
+++ b/drivers/staging/media/hantro/hantro_g2_hevc_dec.c
@@ -597,6 +597,56 @@ static void set_buffers(struct hantro_ctx *ctx)
hantro_write_addr(vpu, G2_TILE_BSD, ctx->hevc_dec.tile_bsd.dma);
}
+static void prepare_scaling_list_buffer(struct hantro_ctx *ctx)
+{
+ struct hantro_dev *vpu = ctx->dev;
+ const struct hantro_hevc_dec_ctrls *ctrls = &ctx->hevc_dec.ctrls;
+ const struct v4l2_ctrl_hevc_scaling_matrix *sc = ctrls->scaling;
+ const struct v4l2_ctrl_hevc_sps *sps = ctrls->sps;
+ u8 *p = ((u8 *)ctx->hevc_dec.scaling_lists.cpu);
+ unsigned int scaling_list_enabled;
+ unsigned int i, j, k;
+
+ scaling_list_enabled = !!(sps->flags & V4L2_HEVC_SPS_FLAG_SCALING_LIST_ENABLED);
+ hantro_reg_write(vpu, &g2_scaling_list_e, scaling_list_enabled);
+
+ if (!scaling_list_enabled)
+ return;
+
+ for (i = 0; i < ARRAY_SIZE(sc->scaling_list_dc_coef_16x16); i++)
+ *p++ = sc->scaling_list_dc_coef_16x16[i];
+
+ for (i = 0; i < ARRAY_SIZE(sc->scaling_list_dc_coef_32x32); i++)
+ *p++ = sc->scaling_list_dc_coef_32x32[i];
+
+ /* 128-bit boundary */
+ p += 8;
+
+ /* write scaling lists column by column */
+
+ for (i = 0; i < 6; i++)
+ for (j = 0; j < 4; j++)
+ for (k = 0; k < 4; k++)
+ *p++ = sc->scaling_list_4x4[i][4 * k + j];
+
+ for (i = 0; i < 6; i++)
+ for (j = 0; j < 8; j++)
+ for (k = 0; k < 8; k++)
+ *p++ = sc->scaling_list_8x8[i][8 * k + j];
+
+ for (i = 0; i < 6; i++)
+ for (j = 0; j < 8; j++)
+ for (k = 0; k < 8; k++)
+ *p++ = sc->scaling_list_16x16[i][8 * k + j];
+
+ for (i = 0; i < 2; i++)
+ for (j = 0; j < 8; j++)
+ for (k = 0; k < 8; k++)
+ *p++ = sc->scaling_list_32x32[i][8 * k + j];
+
+ hantro_write_addr(vpu, HEVC_SCALING_LIST, ctx->hevc_dec.scaling_lists.dma);
+}
+
static void hantro_g2_check_idle(struct hantro_dev *vpu)
{
int i;
@@ -657,6 +707,8 @@ int hantro_g2_hevc_dec_run(struct hantro_ctx *ctx)
set_buffers(ctx);
prepare_tile_info_buffer(ctx);
+ prepare_scaling_list_buffer(ctx);
+
hantro_end_prepare_run(ctx);
hantro_reg_write(vpu, &g2_mode, HEVC_DEC_MODE);
diff --git a/drivers/staging/media/hantro/hantro_hevc.c b/drivers/staging/media/hantro/hantro_hevc.c
index ae9827dd7086..57c1ddeaf2db 100644
--- a/drivers/staging/media/hantro/hantro_hevc.c
+++ b/drivers/staging/media/hantro/hantro_hevc.c
@@ -20,6 +20,8 @@
/* tile border coefficients of filter */
#define VERT_SAO_RAM_SIZE 48 /* bytes per pixel */
+#define SCALING_LIST_SIZE (16 * 64)
+
#define MAX_TILE_COLS 20
#define MAX_TILE_ROWS 22
@@ -287,6 +289,11 @@ int hantro_hevc_dec_prepare_run(struct hantro_ctx *ctx)
if (WARN_ON(!ctrls->decode_params))
return -EINVAL;
+ ctrls->scaling =
+ hantro_get_ctrl(ctx, V4L2_CID_MPEG_VIDEO_HEVC_SCALING_MATRIX);
+ if (WARN_ON(!ctrls->scaling))
+ return -EINVAL;
+
ctrls->sps =
hantro_get_ctrl(ctx, V4L2_CID_MPEG_VIDEO_HEVC_SPS);
if (WARN_ON(!ctrls->sps))
@@ -315,6 +322,12 @@ void hantro_hevc_dec_exit(struct hantro_ctx *ctx)
hevc_dec->tile_sizes.dma);
hevc_dec->tile_sizes.cpu = NULL;
+ if (hevc_dec->scaling_lists.cpu)
+ dma_free_coherent(vpu->dev, hevc_dec->scaling_lists.size,
+ hevc_dec->scaling_lists.cpu,
+ hevc_dec->scaling_lists.dma);
+ hevc_dec->scaling_lists.cpu = NULL;
+
if (hevc_dec->tile_filter.cpu)
dma_free_coherent(vpu->dev, hevc_dec->tile_filter.size,
hevc_dec->tile_filter.cpu,
@@ -358,6 +371,14 @@ int hantro_hevc_dec_init(struct hantro_ctx *ctx)
hevc_dec->tile_sizes.size = size;
+ hevc_dec->scaling_lists.cpu = dma_alloc_coherent(vpu->dev, SCALING_LIST_SIZE,
+ &hevc_dec->scaling_lists.dma,
+ GFP_KERNEL);
+ if (!hevc_dec->scaling_lists.cpu)
+ return -ENOMEM;
+
+ hevc_dec->scaling_lists.size = SCALING_LIST_SIZE;
+
hantro_hevc_ref_init(ctx);
return 0;
diff --git a/drivers/staging/media/hantro/hantro_hw.h b/drivers/staging/media/hantro/hantro_hw.h
index 2edb890f10af..88add18b1bad 100644
--- a/drivers/staging/media/hantro/hantro_hw.h
+++ b/drivers/staging/media/hantro/hantro_hw.h
@@ -108,6 +108,7 @@ struct hantro_h264_dec_hw_ctx {
*/
struct hantro_hevc_dec_ctrls {
const struct v4l2_ctrl_hevc_decode_params *decode_params;
+ const struct v4l2_ctrl_hevc_scaling_matrix *scaling;
const struct v4l2_ctrl_hevc_sps *sps;
const struct v4l2_ctrl_hevc_pps *pps;
u32 hevc_hdr_skip_length;
@@ -120,6 +121,7 @@ struct hantro_hevc_dec_ctrls {
* @tile_sao: Tile SAO buffer
* @tile_bsd: Tile BSD control buffer
* @ref_bufs: Internal reference buffers
+ * @scaling_lists: Scaling lists buffer
* @ref_bufs_poc: Internal reference buffers picture order count
* @ref_bufs_used: Bitfield of used reference buffers
* @ctrls: V4L2 controls attached to a run
@@ -131,6 +133,7 @@ struct hantro_hevc_dec_hw_ctx {
struct hantro_aux_buf tile_sao;
struct hantro_aux_buf tile_bsd;
struct hantro_aux_buf ref_bufs[NUM_REF_PICTURES];
+ struct hantro_aux_buf scaling_lists;
int ref_bufs_poc[NUM_REF_PICTURES];
u32 ref_bufs_used;
struct hantro_hevc_dec_ctrls ctrls;
--
2.25.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 21+ messages in thread
* Re: [PATCH 4/8] media: Add P010 video format
2021-06-04 13:06 ` [PATCH 4/8] media: Add P010 video format Benjamin Gaignard
@ 2021-06-04 16:17 ` Nicolas Dufresne
2021-06-07 11:31 ` Benjamin Gaignard
0 siblings, 1 reply; 21+ messages in thread
From: Nicolas Dufresne @ 2021-06-04 16:17 UTC (permalink / raw)
To: Benjamin Gaignard, hverkuil, ezequiel, p.zabel, mchehab,
shawnguo, s.hauer, festevam, gregkh, mripard, paul.kocialkowski,
wens, jernej.skrabec, emil.l.velikov, andrzej.p, jc
Cc: kernel, linux-imx, linux-media, linux-rockchip, linux-arm-kernel,
linux-kernel
Le vendredi 04 juin 2021 à 15:06 +0200, Benjamin Gaignard a écrit :
> P010 is a YUV format with 10-bits per pixel with interleaved UV.
>
> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
> ---
> .../userspace-api/media/v4l/pixfmt-yuv-planar.rst | 8 ++++++++
> drivers/media/v4l2-core/v4l2-common.c | 1 +
> drivers/media/v4l2-core/v4l2-ioctl.c | 1 +
> include/uapi/linux/videodev2.h | 1 +
> 4 files changed, 11 insertions(+)
>
> diff --git a/Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst b/Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst
> index 090c091affd2..71fed70c03ec 100644
> --- a/Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst
> +++ b/Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst
> @@ -100,6 +100,13 @@ All components are stored with the same number of bits per component.
> - Cb, Cr
> - No
> - 64x32 macroblocks
> + * - V4L2_PIX_FMT_P010
> + - 'P010'
> + - 10
> + - 4:2:0
> + - Cb, Cr
> + - No
> + - Linear
>
> Horizontal Z order
> * - V4L2_PIX_FMT_NV12MT_16X16
> @@ -171,6 +178,7 @@ horizontally.
> .. _V4L2-PIX-FMT-NV21:
> .. _V4L2-PIX-FMT-NV12M:
> .. _V4L2-PIX-FMT-NV21M:
> +.. _V4L2-PIX-FMT-P010:
The NV12/21 documentation is not sufficient to describe this format. While it
shares the layout (two planes Y and interleaved UV), it does not share the
packing. In this case, assuming this is P010 (and not the P010 the Rockchip
tried to upstreamed previously), each 10bit worth of pixel data would be pakced
into 16 bits with the least significant 6 bit being padding bits.
>
> NV12, NV21, NV12M and NV21M
> ---------------------------
> diff --git a/drivers/media/v4l2-core/v4l2-common.c b/drivers/media/v4l2-core/v4l2-common.c
> index 04af03285a20..37b5d82359dd 100644
> --- a/drivers/media/v4l2-core/v4l2-common.c
> +++ b/drivers/media/v4l2-core/v4l2-common.c
> @@ -266,6 +266,7 @@ const struct v4l2_format_info *v4l2_format_info(u32 format)
> { .format = V4L2_PIX_FMT_NV61, .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 1, .comp_planes = 2, .bpp = { 1, 2, 0, 0 }, .hdiv = 2, .vdiv = 1 },
> { .format = V4L2_PIX_FMT_NV24, .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 1, .comp_planes = 2, .bpp = { 1, 2, 0, 0 }, .hdiv = 1, .vdiv = 1 },
> { .format = V4L2_PIX_FMT_NV42, .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 1, .comp_planes = 2, .bpp = { 1, 2, 0, 0 }, .hdiv = 1, .vdiv = 1 },
> + { .format = V4L2_PIX_FMT_P010, .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 1, .comp_planes = 2, .bpp = { 2, 2, 0, 0 }, .hdiv = 2, .vdiv = 1 },
>
> { .format = V4L2_PIX_FMT_YUV410, .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 1, .comp_planes = 3, .bpp = { 1, 1, 1, 0 }, .hdiv = 4, .vdiv = 4 },
> { .format = V4L2_PIX_FMT_YVU410, .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 1, .comp_planes = 3, .bpp = { 1, 1, 1, 0 }, .hdiv = 4, .vdiv = 4 },
> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
> index 2673f51aafa4..6404d5b6e350 100644
> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> @@ -1282,6 +1282,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
> case V4L2_PIX_FMT_NV61: descr = "Y/CrCb 4:2:2"; break;
> case V4L2_PIX_FMT_NV24: descr = "Y/CbCr 4:4:4"; break;
> case V4L2_PIX_FMT_NV42: descr = "Y/CrCb 4:4:4"; break;
> + case V4L2_PIX_FMT_P010: descr = "10-bit Y/CrCb 4:2:0"; break;
> case V4L2_PIX_FMT_NV12M: descr = "Y/CbCr 4:2:0 (N-C)"; break;
> case V4L2_PIX_FMT_NV21M: descr = "Y/CrCb 4:2:0 (N-C)"; break;
> case V4L2_PIX_FMT_NV16M: descr = "Y/CbCr 4:2:2 (N-C)"; break;
> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> index 9260791b8438..e5f7acde0730 100644
> --- a/include/uapi/linux/videodev2.h
> +++ b/include/uapi/linux/videodev2.h
> @@ -602,6 +602,7 @@ struct v4l2_pix_format {
> #define V4L2_PIX_FMT_NV24 v4l2_fourcc('N', 'V', '2', '4') /* 24 Y/CbCr 4:4:4 */
> #define V4L2_PIX_FMT_NV42 v4l2_fourcc('N', 'V', '4', '2') /* 24 Y/CrCb 4:4:4 */
> #define V4L2_PIX_FMT_HM12 v4l2_fourcc('H', 'M', '1', '2') /* 8 YUV 4:2:0 16x16 macroblocks */
> +#define V4L2_PIX_FMT_P010 v4l2_fourcc('P', '0', '1', '0') /* 15 Y/CbCr 4:2:0 10-bit per pixel*/
>
> /* two non contiguous planes - one Y, one Cr + Cb interleaved */
> #define V4L2_PIX_FMT_NV12M v4l2_fourcc('N', 'M', '1', '2') /* 12 Y/CbCr 4:2:0 */
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH 2/8] media: hantro: Add support of compressed reference buffers
2021-06-04 13:06 ` [PATCH 2/8] media: hantro: Add support of compressed reference buffers Benjamin Gaignard
@ 2021-06-04 16:32 ` Ezequiel Garcia
2021-06-07 13:27 ` Benjamin Gaignard
0 siblings, 1 reply; 21+ messages in thread
From: Ezequiel Garcia @ 2021-06-04 16:32 UTC (permalink / raw)
To: Benjamin Gaignard, hverkuil, p.zabel, mchehab, shawnguo, s.hauer,
festevam, gregkh, mripard, paul.kocialkowski, wens,
jernej.skrabec, emil.l.velikov, andrzej.p, jc
Cc: kernel, linux-imx, linux-media, linux-rockchip, linux-arm-kernel,
linux-kernel
Hi Benjamin,
Thanks for the patches, they looks really nice!
On Fri, 2021-06-04 at 15:06 +0200, Benjamin Gaignard wrote:
> Reference frame compression is a feature added in G2 decoder to compress
> frame buffers so that the bandwidth of storing/loading reference frames
> can be reduced, especially when the resolution of decoded stream is of
> high definition.
>
Can you do some tracing using the performance counters in patch 1,
with and without this patch, while we are here, it would be interesting
to run some experiments.
If I understand correctly, the G2 HEVC engine is producing tiled buffers,
which are then post-processed to NV12.
So this compression feature is compressing the tiled buffers,
the NV12 are still uncompressed. If this is the case, could add
some comments somewhere about the different sets of buffers involved?
Also, I believe I mentioned this before, is there a plan
to be able to expose the tiled buffers to userspace (i.e. without
the NV12 linear post-processing)? I think the display controller
can take compressed tiled-buffers.
> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
> ---
> .../staging/media/hantro/hantro_g2_hevc_dec.c | 20 ++++++++++--
> drivers/staging/media/hantro/hantro_g2_regs.h | 4 +++
> drivers/staging/media/hantro/hantro_hevc.c | 32 ++++++++++++++++++-
> drivers/staging/media/hantro/hantro_hw.h | 2 ++
> 4 files changed, 54 insertions(+), 4 deletions(-)
>
> diff --git a/drivers/staging/media/hantro/hantro_g2_hevc_dec.c b/drivers/staging/media/hantro/hantro_g2_hevc_dec.c
> index 89fac5146433..9a715e803037 100644
> --- a/drivers/staging/media/hantro/hantro_g2_hevc_dec.c
> +++ b/drivers/staging/media/hantro/hantro_g2_hevc_dec.c
> @@ -382,10 +382,12 @@ static int set_ref(struct hantro_ctx *ctx)
> const struct v4l2_ctrl_hevc_pps *pps = ctrls->pps;
> const struct v4l2_ctrl_hevc_decode_params *decode_params = ctrls->decode_params;
> const struct v4l2_hevc_dpb_entry *dpb = decode_params->dpb;
> - dma_addr_t luma_addr, chroma_addr, mv_addr = 0;
> + dma_addr_t luma_addr, chroma_addr, mv_addr, compress_luma_addr, compress_chroma_addr = 0;
> struct hantro_dev *vpu = ctx->dev;
> size_t cr_offset = hantro_hevc_chroma_offset(sps);
> size_t mv_offset = hantro_hevc_motion_vectors_offset(sps);
> + size_t compress_luma_offset = hantro_hevc_luma_compress_offset(sps);
> + size_t compress_chroma_offset = hantro_hevc_chroma_compress_offset(sps);
> u32 max_ref_frames;
> u16 dpb_longterm_e;
> static const struct hantro_reg cur_poc[] = {
> @@ -460,6 +462,8 @@ static int set_ref(struct hantro_ctx *ctx)
>
> chroma_addr = luma_addr + cr_offset;
> mv_addr = luma_addr + mv_offset;
> + compress_luma_addr = luma_addr + compress_luma_offset;
> + compress_chroma_addr = luma_addr + compress_chroma_offset;
>
> if (dpb[i].rps == V4L2_HEVC_DPB_ENTRY_RPS_LT_CURR)
> dpb_longterm_e |= BIT(V4L2_HEVC_DPB_ENTRIES_NUM_MAX - 1 - i);
> @@ -467,6 +471,8 @@ static int set_ref(struct hantro_ctx *ctx)
> hantro_write_addr(vpu, G2_REG_ADDR_REF(i), luma_addr);
> hantro_write_addr(vpu, G2_REG_CHR_REF(i), chroma_addr);
> hantro_write_addr(vpu, G2_REG_DMV_REF(i), mv_addr);
> + hantro_write_addr(vpu, G2_COMP_ADDR_REF(i), compress_luma_addr);
> + hantro_write_addr(vpu, G2_COMP_CHR_REF(i), compress_chroma_addr);
I'm a bit confused here, you are seeing luma_addr and also compress_luma_addr
for references??
I'm not saying this is wrong, just would like some comments in the code
so the intention is a bit more clear and I can understand what's going on.
> }
>
> luma_addr = hantro_hevc_get_ref_buf(ctx, decode_params->pic_order_cnt_val);
> @@ -475,7 +481,11 @@ static int set_ref(struct hantro_ctx *ctx)
>
> chroma_addr = luma_addr + cr_offset;
> mv_addr = luma_addr + mv_offset;
> + compress_luma_addr = luma_addr + compress_luma_offset;
> + compress_chroma_addr = luma_addr + compress_chroma_offset;
>
> + hantro_write_addr(vpu, G2_COMP_ADDR_REF(i), compress_luma_addr);
> + hantro_write_addr(vpu, G2_COMP_CHR_REF(i), compress_chroma_addr);
> hantro_write_addr(vpu, G2_REG_ADDR_REF(i), luma_addr);
> hantro_write_addr(vpu, G2_REG_CHR_REF(i), chroma_addr);
> hantro_write_addr(vpu, G2_REG_DMV_REF(i++), mv_addr);
> @@ -483,6 +493,8 @@ static int set_ref(struct hantro_ctx *ctx)
> hantro_write_addr(vpu, G2_ADDR_DST, luma_addr);
> hantro_write_addr(vpu, G2_ADDR_DST_CHR, chroma_addr);
> hantro_write_addr(vpu, G2_ADDR_DST_MV, mv_addr);
> + hantro_write_addr(vpu, G2_COMP_ADDR_DST, compress_luma_addr);
> + hantro_write_addr(vpu, G2_COMP_CHR, compress_chroma_addr);
>
> hantro_hevc_ref_remove_unused(ctx);
>
> @@ -490,6 +502,8 @@ static int set_ref(struct hantro_ctx *ctx)
> hantro_write_addr(vpu, G2_REG_ADDR_REF(i), 0);
> hantro_write_addr(vpu, G2_REG_CHR_REF(i), 0);
> hantro_write_addr(vpu, G2_REG_DMV_REF(i), 0);
> + hantro_write_addr(vpu, G2_COMP_ADDR_REF(i), 0);
> + hantro_write_addr(vpu, G2_COMP_CHR_REF(i), 0);
> }
>
> hantro_reg_write(vpu, &g2_refer_lterm_e, dpb_longterm_e);
> @@ -580,8 +594,8 @@ int hantro_g2_hevc_dec_run(struct hantro_ctx *ctx)
> /* Don't disable output */
> hantro_reg_write(vpu, &g2_out_dis, 0);
>
> - /* Don't compress buffers */
> - hantro_reg_write(vpu, &g2_ref_compress_bypass, 1);
> + /* Compress buffers */
> + hantro_reg_write(vpu, &g2_ref_compress_bypass, 0);
>
> /* use NV12 as output format */
> hantro_reg_write(vpu, &g2_out_rs_e, 1);
> diff --git a/drivers/staging/media/hantro/hantro_g2_regs.h b/drivers/staging/media/hantro/hantro_g2_regs.h
> index 17d84ec9c5c2..0414d92e3860 100644
> --- a/drivers/staging/media/hantro/hantro_g2_regs.h
> +++ b/drivers/staging/media/hantro/hantro_g2_regs.h
> @@ -192,6 +192,10 @@
> #define G2_TILE_FILTER (G2_SWREG(179))
> #define G2_TILE_SAO (G2_SWREG(181))
> #define G2_TILE_BSD (G2_SWREG(183))
> +#define G2_COMP_ADDR_DST (G2_SWREG(190))
> +#define G2_COMP_ADDR_REF(i) (G2_SWREG(192) + ((i) * 0x8))
> +#define G2_COMP_CHR (G2_SWREG(224))
> +#define G2_COMP_CHR_REF(i) (G2_SWREG(226) + ((i) * 0x8))
>
> #define g2_strm_buffer_len G2_DEC_REG(258, 0, 0xffffffff)
> #define g2_strm_start_offset G2_DEC_REG(259, 0, 0xffffffff)
> diff --git a/drivers/staging/media/hantro/hantro_hevc.c b/drivers/staging/media/hantro/hantro_hevc.c
> index 5347f5a41c2a..1b2da990fbf0 100644
> --- a/drivers/staging/media/hantro/hantro_hevc.c
> +++ b/drivers/staging/media/hantro/hantro_hevc.c
> @@ -61,12 +61,42 @@ static size_t hantro_hevc_mv_size(const struct v4l2_ctrl_hevc_sps *sps)
> return mv_size;
> }
>
> +size_t hantro_hevc_luma_compress_offset(const struct v4l2_ctrl_hevc_sps *sps)
> +{
> + return hantro_hevc_motion_vectors_offset(sps) + hantro_hevc_mv_size(sps);
> +}
> +
> +static size_t hantro_hevc_luma_compress_size(const struct v4l2_ctrl_hevc_sps *sps)
> +{
> + u32 pic_width_in_cbsy =
> + round_up((sps->pic_width_in_luma_samples + 8 - 1) / 8, 16);
> + u32 pic_height_in_cbsy = (sps->pic_height_in_luma_samples + 8 - 1) / 8;
> +
> + return round_up(pic_width_in_cbsy * pic_height_in_cbsy, 16);
> +}
> +
> +size_t hantro_hevc_chroma_compress_offset(const struct v4l2_ctrl_hevc_sps *sps)
> +{
> + return hantro_hevc_luma_compress_offset(sps) + hantro_hevc_luma_compress_size(sps);
> +}
> +
> +static size_t hantro_hevc_chroma_compress_size(const struct v4l2_ctrl_hevc_sps *sps)
> +{
> + u32 pic_width_in_cbsc =
> + round_up((sps->pic_width_in_luma_samples + 16 - 1) / 16, 16);
> + u32 pic_height_in_cbsc = (sps->pic_height_in_luma_samples / 2 + 4 - 1) / 4;
> +
Can we get some #defines for these magic numbers?
> + return round_up(pic_width_in_cbsc * pic_height_in_cbsc, 16);
> +}
> +
> static size_t hantro_hevc_ref_size(struct hantro_ctx *ctx)
> {
> const struct hantro_hevc_dec_ctrls *ctrls = &ctx->hevc_dec.ctrls;
> const struct v4l2_ctrl_hevc_sps *sps = ctrls->sps;
>
> - return hantro_hevc_motion_vectors_offset(sps) + hantro_hevc_mv_size(sps);
> + return hantro_hevc_motion_vectors_offset(sps) + hantro_hevc_mv_size(sps) +
> + hantro_hevc_luma_compress_size(sps) +
> + hantro_hevc_chroma_compress_size(sps);
> }
>
> static void hantro_hevc_ref_free(struct hantro_ctx *ctx)
> diff --git a/drivers/staging/media/hantro/hantro_hw.h b/drivers/staging/media/hantro/hantro_hw.h
> index 8fa0aacb61cd..c5374cd74d66 100644
> --- a/drivers/staging/media/hantro/hantro_hw.h
> +++ b/drivers/staging/media/hantro/hantro_hw.h
> @@ -246,6 +246,8 @@ dma_addr_t hantro_hevc_get_ref_buf(struct hantro_ctx *ctx, int poc);
> void hantro_hevc_ref_remove_unused(struct hantro_ctx *ctx);
> size_t hantro_hevc_chroma_offset(const struct v4l2_ctrl_hevc_sps *sps);
> size_t hantro_hevc_motion_vectors_offset(const struct v4l2_ctrl_hevc_sps *sps);
> +size_t hantro_hevc_luma_compress_offset(const struct v4l2_ctrl_hevc_sps *sps);
> +size_t hantro_hevc_chroma_compress_offset(const struct v4l2_ctrl_hevc_sps *sps);
>
> static inline size_t
> hantro_h264_mv_size(unsigned int width, unsigned int height)
Thanks,
Ezequiel
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH 0/8] Additional features for Hantro HEVC
2021-06-04 13:06 [PATCH 0/8] Additional features for Hantro HEVC Benjamin Gaignard
` (7 preceding siblings ...)
2021-06-04 13:06 ` [PATCH 8/8] media: hantro: Add scaling lists feature Benjamin Gaignard
@ 2021-06-04 16:49 ` Ezequiel Garcia
2021-06-07 11:20 ` Benjamin Gaignard
8 siblings, 1 reply; 21+ messages in thread
From: Ezequiel Garcia @ 2021-06-04 16:49 UTC (permalink / raw)
To: Benjamin Gaignard, hverkuil, p.zabel, mchehab, shawnguo, s.hauer,
festevam, gregkh, mripard, paul.kocialkowski, wens,
jernej.skrabec, emil.l.velikov, andrzej.p, jc
Cc: kernel, linux-imx, linux-media, linux-rockchip, linux-arm-kernel,
linux-kernel
Hi Benjamin,
Thanks for posting this so quickly.
On Fri, 2021-06-04 at 15:06 +0200, Benjamin Gaignard wrote:
> Basic HEVC support has been added to Hantro driver in this pull request:
> https://www.spinics.net/lists/linux-media/msg193744.html
>
> Thanks to that it is now possible to support more features for this driver.
>
> The first patch allow to log the hardware performance per macroblock.
> The second patch makes the driver use compressed reference frames to
> reduce memory bandwidth consumption.
As I commented, it would be nice to do some measurements here
> Patches 3 to 5 allow to decode and produce 10-bits P010 frames.
I suppose this means that some 10-bit test vectors in Fluster HEVC
test suite are now passing?
> Patch 6 make usage of G2 post processor to scale down the frames.
> Patches 7 and 8 add the support of HEVC scaling matrix by adding a new
> control.
>
Ditto, the test vectors with scaling lists should now pass, any chance
you post the test suite run before and after this series, just for reference?
Thanks again,
Ezequiel
> All these patches enhance the HEVC support for Hantro (G2) hardware.
> Unluckily they almost all touch the same pieces of code, where buffer
> size, offset and addresses are set, so they have to be in this order.
> They depend of the series pushed in this pull request:
> https://www.spinics.net/lists/linux-media/msg193744.html
>
> Benjamin
>
> Benjamin Gaignard (8):
> media: hantro: Trace hevc hw cycles performance register
> media: hantro: Add support of compressed reference buffers
> media: hantro: hevc: Allow 10-bits encoded streams
> media: Add P010 video format
> media: hantro: hevc: Allow to produce 10-bit frames
> media: hantro: enumerate scaled output formats
> media: hevc: Add scaling matrix control
> media: hantro: Add scaling lists feature
>
> .../media/v4l/ext-ctrls-codec.rst | 45 +++++
> .../media/v4l/pixfmt-yuv-planar.rst | 8 +
> .../media/v4l/vidioc-queryctrl.rst | 6 +
> drivers/media/v4l2-core/v4l2-common.c | 1 +
> drivers/media/v4l2-core/v4l2-ctrls-core.c | 6 +
> drivers/media/v4l2-core/v4l2-ctrls-defs.c | 4 +
> drivers/media/v4l2-core/v4l2-ioctl.c | 1 +
> drivers/staging/media/hantro/hantro.h | 4 +
> drivers/staging/media/hantro/hantro_drv.c | 32 +++-
> .../staging/media/hantro/hantro_g2_hevc_dec.c | 175 ++++++++++++++++--
> drivers/staging/media/hantro/hantro_g2_regs.h | 12 ++
> drivers/staging/media/hantro/hantro_hevc.c | 60 +++++-
> drivers/staging/media/hantro/hantro_hw.h | 7 +
> drivers/staging/media/hantro/hantro_v4l2.c | 10 +-
> drivers/staging/media/hantro/imx8m_vpu_hw.c | 6 +
> drivers/staging/media/hantro/trace.h | 40 ++++
> include/media/hevc-ctrls.h | 11 ++
> include/uapi/linux/videodev2.h | 1 +
> 18 files changed, 407 insertions(+), 22 deletions(-)
> create mode 100644 drivers/staging/media/hantro/trace.h
>
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH 7/8] media: hevc: Add scaling matrix control
2021-06-04 13:06 ` [PATCH 7/8] media: hevc: Add scaling matrix control Benjamin Gaignard
@ 2021-06-06 7:49 ` Jernej Škrabec
2021-06-07 7:23 ` Benjamin Gaignard
0 siblings, 1 reply; 21+ messages in thread
From: Jernej Škrabec @ 2021-06-06 7:49 UTC (permalink / raw)
To: hverkuil, ezequiel, p.zabel, mchehab, shawnguo, s.hauer,
festevam, gregkh, mripard, paul.kocialkowski, wens,
jernej.skrabec, emil.l.velikov, andrzej.p, jc, Benjamin Gaignard
Cc: kernel, linux-imx, linux-media, linux-rockchip, linux-arm-kernel,
linux-kernel, Benjamin Gaignard
Hi!
Dne petek, 04. junij 2021 ob 15:06:18 CEST je Benjamin Gaignard napisal(a):
> HEVC scaling lists are used for the scaling process for transform
> coefficients.
> V4L2_HEVC_SPS_FLAG_SCALING_LIST_ENABLED has to set when they are
> encoded in the bitstream.
>
> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
> ---
> .../media/v4l/ext-ctrls-codec.rst | 45 +++++++++++++++++++
> .../media/v4l/vidioc-queryctrl.rst | 6 +++
> drivers/media/v4l2-core/v4l2-ctrls-core.c | 6 +++
> drivers/media/v4l2-core/v4l2-ctrls-defs.c | 4 ++
> include/media/hevc-ctrls.h | 11 +++++
> 5 files changed, 72 insertions(+)
>
> diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst b/
Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
> index 9120c5bcaf90..a4512b7cb520 100644
> --- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
> +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
> @@ -3065,6 +3065,51 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
>
> \normalsize
>
> +``V4L2_CID_MPEG_VIDEO_HEVC_SCALING_MATRIX (struct)``
> + Specifies the HEVC scaling matrix parameters used for the scaling
process
> + for transform coefficients.
> + These matrix and parameters are defined according to :ref:`hevc`.
> + They are described in section 7.4.5 "Scaling list data semantics" of
> + the specification.
> +
> +.. c:type:: v4l2_ctrl_hevc_scaling_matrix
> +
> +.. raw:: latex
> +
> + \scriptsize
> +
> +.. tabularcolumns:: |p{5.4cm}|p{6.8cm}|p{5.1cm}|
> +
> +.. cssclass:: longtable
> +
> +.. flat-table:: struct v4l2_ctrl_hevc_slice_params
^ copy paste error? It should be v4l2_ctrl_hevc_scaling_matrix.
Best regards,
Jernej
> + :header-rows: 0
> + :stub-columns: 0
> + :widths: 1 1 2
> +
> + * - __u8
> + - ``scaling_list_4x4[6][16]``
> + -
> + * - __u8
> + - ``scaling_list_8x8[6][64]``
> + -
> + * - __u8
> + - ``scaling_list_16x16[6][64]``
> + -
> + * - __u8
> + - ``scaling_list_32x32[2][64]``
> + -
> + * - __u8
> + - ``scaling_list_dc_coef_16x16[6]``
> + -
> + * - __u8
> + - ``scaling_list_dc_coef_32x32[2]``
> + -
> +
> +.. raw:: latex
> +
> + \normalsize
> +
> .. c:type:: v4l2_hevc_dpb_entry
>
> .. raw:: latex
> diff --git a/Documentation/userspace-api/media/v4l/vidioc-queryctrl.rst b/
Documentation/userspace-api/media/v4l/vidioc-queryctrl.rst
> index f9ecf6276129..2f491c17dd5d 100644
> --- a/Documentation/userspace-api/media/v4l/vidioc-queryctrl.rst
> +++ b/Documentation/userspace-api/media/v4l/vidioc-queryctrl.rst
> @@ -495,6 +495,12 @@ See also the examples in :ref:`control`.
> - n/a
> - A struct :c:type:`v4l2_ctrl_hevc_slice_params`, containing HEVC
> slice parameters for stateless video decoders.
> + * - ``V4L2_CTRL_TYPE_HEVC_SCALING_MATRIX``
> + - n/a
> + - n/a
> + - n/a
> + - A struct :c:type:`v4l2_ctrl_hevc_scaling_matrix`, containing HEVC
> + scaling matrix for stateless video decoders.
> * - ``V4L2_CTRL_TYPE_VP8_FRAME``
> - n/a
> - n/a
> diff --git a/drivers/media/v4l2-core/v4l2-ctrls-core.c b/drivers/media/v4l2-
core/v4l2-ctrls-core.c
> index c4b5082849b6..70adfc1b9c81 100644
> --- a/drivers/media/v4l2-core/v4l2-ctrls-core.c
> +++ b/drivers/media/v4l2-core/v4l2-ctrls-core.c
> @@ -687,6 +687,9 @@ static int std_validate_compound(const struct v4l2_ctrl
*ctrl, u32 idx,
>
> break;
>
> + case V4L2_CTRL_TYPE_HEVC_SCALING_MATRIX:
> + break;
> +
> case V4L2_CTRL_TYPE_AREA:
> area = p;
> if (!area->width || !area->height)
> @@ -1240,6 +1243,9 @@ static struct v4l2_ctrl *v4l2_ctrl_new(struct
v4l2_ctrl_handler *hdl,
> case V4L2_CTRL_TYPE_HEVC_SLICE_PARAMS:
> elem_size = sizeof(struct v4l2_ctrl_hevc_slice_params);
> break;
> + case V4L2_CTRL_TYPE_HEVC_SCALING_MATRIX:
> + elem_size = sizeof(struct
v4l2_ctrl_hevc_scaling_matrix);
> + break;
> case V4L2_CTRL_TYPE_HEVC_DECODE_PARAMS:
> elem_size = sizeof(struct
v4l2_ctrl_hevc_decode_params);
> break;
> diff --git a/drivers/media/v4l2-core/v4l2-ctrls-defs.c b/drivers/media/v4l2-
core/v4l2-ctrls-defs.c
> index b6344bbf1e00..cb29c2a7fabe 100644
> --- a/drivers/media/v4l2-core/v4l2-ctrls-defs.c
> +++ b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
> @@ -996,6 +996,7 @@ const char *v4l2_ctrl_get_name(u32 id)
> case V4L2_CID_MPEG_VIDEO_HEVC_SPS:
return "HEVC Sequence Parameter Set";
> case V4L2_CID_MPEG_VIDEO_HEVC_PPS:
return "HEVC Picture Parameter Set";
> case V4L2_CID_MPEG_VIDEO_HEVC_SLICE_PARAMS: return
"HEVC Slice Parameters";
> + case V4L2_CID_MPEG_VIDEO_HEVC_SCALING_MATRIX:
return "HEVC Scaling Matrix";
> case V4L2_CID_MPEG_VIDEO_HEVC_DECODE_PARAMS:
return "HEVC Decode Parameters";
> case V4L2_CID_MPEG_VIDEO_HEVC_DECODE_MODE: return "HEVC
Decode Mode";
> case V4L2_CID_MPEG_VIDEO_HEVC_START_CODE: return
"HEVC Start Code";
> @@ -1488,6 +1489,9 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum
v4l2_ctrl_type *type,
> case V4L2_CID_MPEG_VIDEO_HEVC_SLICE_PARAMS:
> *type = V4L2_CTRL_TYPE_HEVC_SLICE_PARAMS;
> break;
> + case V4L2_CID_MPEG_VIDEO_HEVC_SCALING_MATRIX:
> + *type = V4L2_CTRL_TYPE_HEVC_SCALING_MATRIX;
> + break;
> case V4L2_CID_MPEG_VIDEO_HEVC_DECODE_PARAMS:
> *type = V4L2_CTRL_TYPE_HEVC_DECODE_PARAMS;
> break;
> diff --git a/include/media/hevc-ctrls.h b/include/media/hevc-ctrls.h
> index dc964ff7cd29..da6189ef7ab4 100644
> --- a/include/media/hevc-ctrls.h
> +++ b/include/media/hevc-ctrls.h
> @@ -19,6 +19,7 @@
> #define V4L2_CID_MPEG_VIDEO_HEVC_SPS (V4L2_CID_CODEC_BASE +
1008)
> #define V4L2_CID_MPEG_VIDEO_HEVC_PPS (V4L2_CID_CODEC_BASE +
1009)
> #define V4L2_CID_MPEG_VIDEO_HEVC_SLICE_PARAMS (V4L2_CID_CODEC_BASE +
1010)
> +#define V4L2_CID_MPEG_VIDEO_HEVC_SCALING_MATRIX (V4L2_CID_CODEC_BASE +
1011)
> #define V4L2_CID_MPEG_VIDEO_HEVC_DECODE_PARAMS (V4L2_CID_CODEC_BASE +
1012)
> #define V4L2_CID_MPEG_VIDEO_HEVC_DECODE_MODE (V4L2_CID_CODEC_BASE +
1015)
> #define V4L2_CID_MPEG_VIDEO_HEVC_START_CODE (V4L2_CID_CODEC_BASE +
1016)
> @@ -27,6 +28,7 @@
> #define V4L2_CTRL_TYPE_HEVC_SPS 0x0120
> #define V4L2_CTRL_TYPE_HEVC_PPS 0x0121
> #define V4L2_CTRL_TYPE_HEVC_SLICE_PARAMS 0x0122
> +#define V4L2_CTRL_TYPE_HEVC_SCALING_MATRIX 0x0123
> #define V4L2_CTRL_TYPE_HEVC_DECODE_PARAMS 0x0124
>
> enum v4l2_mpeg_video_hevc_decode_mode {
> @@ -225,6 +227,15 @@ struct v4l2_ctrl_hevc_decode_params {
> __u64 flags;
> };
>
> +struct v4l2_ctrl_hevc_scaling_matrix {
> + __u8 scaling_list_4x4[6][16];
> + __u8 scaling_list_8x8[6][64];
> + __u8 scaling_list_16x16[6][64];
> + __u8 scaling_list_32x32[2][64];
> + __u8 scaling_list_dc_coef_16x16[6];
> + __u8 scaling_list_dc_coef_32x32[2];
> +};
> +
> /* MPEG-class control IDs specific to the Hantro driver as defined by V4L2
*/
> #define V4L2_CID_CODEC_HANTRO_BASE
(V4L2_CTRL_CLASS_CODEC | 0x1200)
> /*
>
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH 7/8] media: hevc: Add scaling matrix control
2021-06-06 7:49 ` Jernej Škrabec
@ 2021-06-07 7:23 ` Benjamin Gaignard
0 siblings, 0 replies; 21+ messages in thread
From: Benjamin Gaignard @ 2021-06-07 7:23 UTC (permalink / raw)
To: Jernej Škrabec, hverkuil, ezequiel, p.zabel, mchehab,
shawnguo, s.hauer, festevam, gregkh, mripard, paul.kocialkowski,
wens, jernej.skrabec, emil.l.velikov, andrzej.p, jc
Cc: kernel, linux-imx, linux-media, linux-rockchip, linux-arm-kernel,
linux-kernel
Le 06/06/2021 à 09:49, Jernej Škrabec a écrit :
> Hi!
>
> Dne petek, 04. junij 2021 ob 15:06:18 CEST je Benjamin Gaignard napisal(a):
>> HEVC scaling lists are used for the scaling process for transform
>> coefficients.
>> V4L2_HEVC_SPS_FLAG_SCALING_LIST_ENABLED has to set when they are
>> encoded in the bitstream.
>>
>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>> ---
>> .../media/v4l/ext-ctrls-codec.rst | 45 +++++++++++++++++++
>> .../media/v4l/vidioc-queryctrl.rst | 6 +++
>> drivers/media/v4l2-core/v4l2-ctrls-core.c | 6 +++
>> drivers/media/v4l2-core/v4l2-ctrls-defs.c | 4 ++
>> include/media/hevc-ctrls.h | 11 +++++
>> 5 files changed, 72 insertions(+)
>>
>> diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst b/
> Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
>> index 9120c5bcaf90..a4512b7cb520 100644
>> --- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
>> +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
>> @@ -3065,6 +3065,51 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
>>
>> \normalsize
>>
>> +``V4L2_CID_MPEG_VIDEO_HEVC_SCALING_MATRIX (struct)``
>> + Specifies the HEVC scaling matrix parameters used for the scaling
> process
>> + for transform coefficients.
>> + These matrix and parameters are defined according to :ref:`hevc`.
>> + They are described in section 7.4.5 "Scaling list data semantics" of
>> + the specification.
>> +
>> +.. c:type:: v4l2_ctrl_hevc_scaling_matrix
>> +
>> +.. raw:: latex
>> +
>> + \scriptsize
>> +
>> +.. tabularcolumns:: |p{5.4cm}|p{6.8cm}|p{5.1cm}|
>> +
>> +.. cssclass:: longtable
>> +
>> +.. flat-table:: struct v4l2_ctrl_hevc_slice_params
> ^ copy paste error? It should be v4l2_ctrl_hevc_scaling_matrix.
Yes I will fix it in the next version.
Thanks
Benjamin
>
> Best regards,
> Jernej
>
>> + :header-rows: 0
>> + :stub-columns: 0
>> + :widths: 1 1 2
>> +
>> + * - __u8
>> + - ``scaling_list_4x4[6][16]``
>> + -
>> + * - __u8
>> + - ``scaling_list_8x8[6][64]``
>> + -
>> + * - __u8
>> + - ``scaling_list_16x16[6][64]``
>> + -
>> + * - __u8
>> + - ``scaling_list_32x32[2][64]``
>> + -
>> + * - __u8
>> + - ``scaling_list_dc_coef_16x16[6]``
>> + -
>> + * - __u8
>> + - ``scaling_list_dc_coef_32x32[2]``
>> + -
>> +
>> +.. raw:: latex
>> +
>> + \normalsize
>> +
>> .. c:type:: v4l2_hevc_dpb_entry
>>
>> .. raw:: latex
>> diff --git a/Documentation/userspace-api/media/v4l/vidioc-queryctrl.rst b/
> Documentation/userspace-api/media/v4l/vidioc-queryctrl.rst
>> index f9ecf6276129..2f491c17dd5d 100644
>> --- a/Documentation/userspace-api/media/v4l/vidioc-queryctrl.rst
>> +++ b/Documentation/userspace-api/media/v4l/vidioc-queryctrl.rst
>> @@ -495,6 +495,12 @@ See also the examples in :ref:`control`.
>> - n/a
>> - A struct :c:type:`v4l2_ctrl_hevc_slice_params`, containing HEVC
>> slice parameters for stateless video decoders.
>> + * - ``V4L2_CTRL_TYPE_HEVC_SCALING_MATRIX``
>> + - n/a
>> + - n/a
>> + - n/a
>> + - A struct :c:type:`v4l2_ctrl_hevc_scaling_matrix`, containing HEVC
>> + scaling matrix for stateless video decoders.
>> * - ``V4L2_CTRL_TYPE_VP8_FRAME``
>> - n/a
>> - n/a
>> diff --git a/drivers/media/v4l2-core/v4l2-ctrls-core.c b/drivers/media/v4l2-
> core/v4l2-ctrls-core.c
>> index c4b5082849b6..70adfc1b9c81 100644
>> --- a/drivers/media/v4l2-core/v4l2-ctrls-core.c
>> +++ b/drivers/media/v4l2-core/v4l2-ctrls-core.c
>> @@ -687,6 +687,9 @@ static int std_validate_compound(const struct v4l2_ctrl
> *ctrl, u32 idx,
>>
>> break;
>>
>> + case V4L2_CTRL_TYPE_HEVC_SCALING_MATRIX:
>> + break;
>> +
>> case V4L2_CTRL_TYPE_AREA:
>> area = p;
>> if (!area->width || !area->height)
>> @@ -1240,6 +1243,9 @@ static struct v4l2_ctrl *v4l2_ctrl_new(struct
> v4l2_ctrl_handler *hdl,
>> case V4L2_CTRL_TYPE_HEVC_SLICE_PARAMS:
>> elem_size = sizeof(struct v4l2_ctrl_hevc_slice_params);
>> break;
>> + case V4L2_CTRL_TYPE_HEVC_SCALING_MATRIX:
>> + elem_size = sizeof(struct
> v4l2_ctrl_hevc_scaling_matrix);
>> + break;
>> case V4L2_CTRL_TYPE_HEVC_DECODE_PARAMS:
>> elem_size = sizeof(struct
> v4l2_ctrl_hevc_decode_params);
>> break;
>> diff --git a/drivers/media/v4l2-core/v4l2-ctrls-defs.c b/drivers/media/v4l2-
> core/v4l2-ctrls-defs.c
>> index b6344bbf1e00..cb29c2a7fabe 100644
>> --- a/drivers/media/v4l2-core/v4l2-ctrls-defs.c
>> +++ b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
>> @@ -996,6 +996,7 @@ const char *v4l2_ctrl_get_name(u32 id)
>> case V4L2_CID_MPEG_VIDEO_HEVC_SPS:
> return "HEVC Sequence Parameter Set";
>> case V4L2_CID_MPEG_VIDEO_HEVC_PPS:
> return "HEVC Picture Parameter Set";
>> case V4L2_CID_MPEG_VIDEO_HEVC_SLICE_PARAMS: return
> "HEVC Slice Parameters";
>> + case V4L2_CID_MPEG_VIDEO_HEVC_SCALING_MATRIX:
> return "HEVC Scaling Matrix";
>> case V4L2_CID_MPEG_VIDEO_HEVC_DECODE_PARAMS:
> return "HEVC Decode Parameters";
>> case V4L2_CID_MPEG_VIDEO_HEVC_DECODE_MODE: return "HEVC
> Decode Mode";
>> case V4L2_CID_MPEG_VIDEO_HEVC_START_CODE: return
> "HEVC Start Code";
>> @@ -1488,6 +1489,9 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum
> v4l2_ctrl_type *type,
>> case V4L2_CID_MPEG_VIDEO_HEVC_SLICE_PARAMS:
>> *type = V4L2_CTRL_TYPE_HEVC_SLICE_PARAMS;
>> break;
>> + case V4L2_CID_MPEG_VIDEO_HEVC_SCALING_MATRIX:
>> + *type = V4L2_CTRL_TYPE_HEVC_SCALING_MATRIX;
>> + break;
>> case V4L2_CID_MPEG_VIDEO_HEVC_DECODE_PARAMS:
>> *type = V4L2_CTRL_TYPE_HEVC_DECODE_PARAMS;
>> break;
>> diff --git a/include/media/hevc-ctrls.h b/include/media/hevc-ctrls.h
>> index dc964ff7cd29..da6189ef7ab4 100644
>> --- a/include/media/hevc-ctrls.h
>> +++ b/include/media/hevc-ctrls.h
>> @@ -19,6 +19,7 @@
>> #define V4L2_CID_MPEG_VIDEO_HEVC_SPS (V4L2_CID_CODEC_BASE +
> 1008)
>> #define V4L2_CID_MPEG_VIDEO_HEVC_PPS (V4L2_CID_CODEC_BASE +
> 1009)
>> #define V4L2_CID_MPEG_VIDEO_HEVC_SLICE_PARAMS (V4L2_CID_CODEC_BASE +
> 1010)
>> +#define V4L2_CID_MPEG_VIDEO_HEVC_SCALING_MATRIX (V4L2_CID_CODEC_BASE +
> 1011)
>> #define V4L2_CID_MPEG_VIDEO_HEVC_DECODE_PARAMS (V4L2_CID_CODEC_BASE +
> 1012)
>> #define V4L2_CID_MPEG_VIDEO_HEVC_DECODE_MODE (V4L2_CID_CODEC_BASE +
> 1015)
>> #define V4L2_CID_MPEG_VIDEO_HEVC_START_CODE (V4L2_CID_CODEC_BASE +
> 1016)
>> @@ -27,6 +28,7 @@
>> #define V4L2_CTRL_TYPE_HEVC_SPS 0x0120
>> #define V4L2_CTRL_TYPE_HEVC_PPS 0x0121
>> #define V4L2_CTRL_TYPE_HEVC_SLICE_PARAMS 0x0122
>> +#define V4L2_CTRL_TYPE_HEVC_SCALING_MATRIX 0x0123
>> #define V4L2_CTRL_TYPE_HEVC_DECODE_PARAMS 0x0124
>>
>> enum v4l2_mpeg_video_hevc_decode_mode {
>> @@ -225,6 +227,15 @@ struct v4l2_ctrl_hevc_decode_params {
>> __u64 flags;
>> };
>>
>> +struct v4l2_ctrl_hevc_scaling_matrix {
>> + __u8 scaling_list_4x4[6][16];
>> + __u8 scaling_list_8x8[6][64];
>> + __u8 scaling_list_16x16[6][64];
>> + __u8 scaling_list_32x32[2][64];
>> + __u8 scaling_list_dc_coef_16x16[6];
>> + __u8 scaling_list_dc_coef_32x32[2];
>> +};
>> +
>> /* MPEG-class control IDs specific to the Hantro driver as defined by V4L2
> */
>> #define V4L2_CID_CODEC_HANTRO_BASE
> (V4L2_CTRL_CLASS_CODEC | 0x1200)
>> /*
>>
>
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH 0/8] Additional features for Hantro HEVC
2021-06-04 16:49 ` [PATCH 0/8] Additional features for Hantro HEVC Ezequiel Garcia
@ 2021-06-07 11:20 ` Benjamin Gaignard
0 siblings, 0 replies; 21+ messages in thread
From: Benjamin Gaignard @ 2021-06-07 11:20 UTC (permalink / raw)
To: Ezequiel Garcia, hverkuil, p.zabel, mchehab, shawnguo, s.hauer,
festevam, gregkh, mripard, paul.kocialkowski, wens,
jernej.skrabec, emil.l.velikov, andrzej.p, jc
Cc: kernel, linux-imx, linux-media, linux-rockchip, linux-arm-kernel,
linux-kernel
Le 04/06/2021 à 18:49, Ezequiel Garcia a écrit :
> Hi Benjamin,
>
> Thanks for posting this so quickly.
>
> On Fri, 2021-06-04 at 15:06 +0200, Benjamin Gaignard wrote:
>> Basic HEVC support has been added to Hantro driver in this pull request:
>> https://www.spinics.net/lists/linux-media/msg193744.html
>>
>> Thanks to that it is now possible to support more features for this driver.
>>
>> The first patch allow to log the hardware performance per macroblock.
>> The second patch makes the driver use compressed reference frames to
>> reduce memory bandwidth consumption.
> As I commented, it would be nice to do some measurements here
The performances measured by the first patch (the number of macroblock per cycle) remain
the same because the goal of using compressed frames is to save memory bandwidth.
That doesn't impact what is measure by the Hantro hardware block itself.
To give you an example that is the result of decoding one of the fluster reference file.
gst-launch-1.0 filesrc location=AMP_A_Samsung_7.hevc ! h265parse ! v4l2slh265dec ! fakesink
cat /sys/kernel/debug/tracing/trace
#
# entries-in-buffer/entries-written: 17/17 #P:4
#
# _-----=> irqs-off
# / _----=> need-resched
# | / _---=> hardirq/softirq
# || / _--=> preempt-depth
# ||| / delay
# TASK-PID CPU# |||| TIMESTAMP FUNCTION
# | | | |||| | |
<idle>-0 [000] d.h1 143.745124: hantro_hevc_perf: minor = 1, 180 cycles / mb
<idle>-0 [000] d.h1 143.758086: hantro_hevc_perf: minor = 1, 172 cycles / mb
<idle>-0 [000] d.h1 143.770829: hantro_hevc_perf: minor = 1, 167 cycles / mb
<idle>-0 [000] d.h1 143.782757: hantro_hevc_perf: minor = 1, 166 cycles / mb
<idle>-0 [000] d.h1 143.794866: hantro_hevc_perf: minor = 1, 165 cycles / mb
<idle>-0 [000] d.h1 143.808012: hantro_hevc_perf: minor = 1, 165 cycles / mb
<idle>-0 [000] d.h1 143.818225: hantro_hevc_perf: minor = 1, 166 cycles / mb
<idle>-0 [000] d.h1 143.828063: hantro_hevc_perf: minor = 1, 165 cycles / mb
<idle>-0 [000] d.h1 143.838217: hantro_hevc_perf: minor = 1, 165 cycles / mb
<idle>-0 [000] d.h1 143.848728: hantro_hevc_perf: minor = 1, 172 cycles / mb
<idle>-0 [000] d.h1 143.858677: hantro_hevc_perf: minor = 1, 167 cycles / mb
<idle>-0 [000] d.h1 143.868576: hantro_hevc_perf: minor = 1, 166 cycles / mb
<idle>-0 [000] d.h1 143.878431: hantro_hevc_perf: minor = 1, 165 cycles / mb
<idle>-0 [000] d.h1 143.888684: hantro_hevc_perf: minor = 1, 165 cycles / mb
<idle>-0 [000] d.h1 143.898827: hantro_hevc_perf: minor = 1, 166 cycles / mb
<idle>-0 [000] d.h1 143.908645: hantro_hevc_perf: minor = 1, 165 cycles / mb
<idle>-0 [000] d.h1 143.918772: hantro_hevc_perf: minor = 1, 165 cycles / mb
>
>> Patches 3 to 5 allow to decode and produce 10-bits P010 frames.
> I suppose this means that some 10-bit test vectors in Fluster HEVC
> test suite are now passing?
>
>> Patch 6 make usage of G2 post processor to scale down the frames.
>> Patches 7 and 8 add the support of HEVC scaling matrix by adding a new
>> control.
>>
> Ditto, the test vectors with scaling lists should now pass, any chance
> you post the test suite run before and after this series, just for reference?
Files like SLIST_A_Sony_5, SLIST_B_Sony_9, SLIST_C_Sony_4, SLIST_D_Sony_9 or
DBLK_A_MAIN10_VIXS_4 are now decoded by the driver.
fluster score is 70/147.
I think it could be better but, in few cases, it seems that the last frames
isn't retrieved by the userland stack but that need to be verify and debug
in an other series ;-)
Benjamin
>
> Thanks again,
> Ezequiel
>
>> All these patches enhance the HEVC support for Hantro (G2) hardware.
>> Unluckily they almost all touch the same pieces of code, where buffer
>> size, offset and addresses are set, so they have to be in this order.
>> They depend of the series pushed in this pull request:
>> https://www.spinics.net/lists/linux-media/msg193744.html
>>
>> Benjamin
>>
>> Benjamin Gaignard (8):
>> media: hantro: Trace hevc hw cycles performance register
>> media: hantro: Add support of compressed reference buffers
>> media: hantro: hevc: Allow 10-bits encoded streams
>> media: Add P010 video format
>> media: hantro: hevc: Allow to produce 10-bit frames
>> media: hantro: enumerate scaled output formats
>> media: hevc: Add scaling matrix control
>> media: hantro: Add scaling lists feature
>>
>> .../media/v4l/ext-ctrls-codec.rst | 45 +++++
>> .../media/v4l/pixfmt-yuv-planar.rst | 8 +
>> .../media/v4l/vidioc-queryctrl.rst | 6 +
>> drivers/media/v4l2-core/v4l2-common.c | 1 +
>> drivers/media/v4l2-core/v4l2-ctrls-core.c | 6 +
>> drivers/media/v4l2-core/v4l2-ctrls-defs.c | 4 +
>> drivers/media/v4l2-core/v4l2-ioctl.c | 1 +
>> drivers/staging/media/hantro/hantro.h | 4 +
>> drivers/staging/media/hantro/hantro_drv.c | 32 +++-
>> .../staging/media/hantro/hantro_g2_hevc_dec.c | 175 ++++++++++++++++--
>> drivers/staging/media/hantro/hantro_g2_regs.h | 12 ++
>> drivers/staging/media/hantro/hantro_hevc.c | 60 +++++-
>> drivers/staging/media/hantro/hantro_hw.h | 7 +
>> drivers/staging/media/hantro/hantro_v4l2.c | 10 +-
>> drivers/staging/media/hantro/imx8m_vpu_hw.c | 6 +
>> drivers/staging/media/hantro/trace.h | 40 ++++
>> include/media/hevc-ctrls.h | 11 ++
>> include/uapi/linux/videodev2.h | 1 +
>> 18 files changed, 407 insertions(+), 22 deletions(-)
>> create mode 100644 drivers/staging/media/hantro/trace.h
>>
>
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH 4/8] media: Add P010 video format
2021-06-04 16:17 ` Nicolas Dufresne
@ 2021-06-07 11:31 ` Benjamin Gaignard
2021-06-09 17:58 ` Nicolas Dufresne
0 siblings, 1 reply; 21+ messages in thread
From: Benjamin Gaignard @ 2021-06-07 11:31 UTC (permalink / raw)
To: Nicolas Dufresne, hverkuil, ezequiel, p.zabel, mchehab, shawnguo,
s.hauer, festevam, gregkh, mripard, paul.kocialkowski, wens,
jernej.skrabec, emil.l.velikov, andrzej.p, jc
Cc: kernel, linux-imx, linux-media, linux-rockchip, linux-arm-kernel,
linux-kernel
Le 04/06/2021 à 18:17, Nicolas Dufresne a écrit :
> Le vendredi 04 juin 2021 à 15:06 +0200, Benjamin Gaignard a écrit :
>> P010 is a YUV format with 10-bits per pixel with interleaved UV.
>>
>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>> ---
>> .../userspace-api/media/v4l/pixfmt-yuv-planar.rst | 8 ++++++++
>> drivers/media/v4l2-core/v4l2-common.c | 1 +
>> drivers/media/v4l2-core/v4l2-ioctl.c | 1 +
>> include/uapi/linux/videodev2.h | 1 +
>> 4 files changed, 11 insertions(+)
>>
>> diff --git a/Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst b/Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst
>> index 090c091affd2..71fed70c03ec 100644
>> --- a/Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst
>> +++ b/Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst
>> @@ -100,6 +100,13 @@ All components are stored with the same number of bits per component.
>> - Cb, Cr
>> - No
>> - 64x32 macroblocks
>> + * - V4L2_PIX_FMT_P010
>> + - 'P010'
>> + - 10
>> + - 4:2:0
>> + - Cb, Cr
>> + - No
>> + - Linear
>>
>> Horizontal Z order
>> * - V4L2_PIX_FMT_NV12MT_16X16
>> @@ -171,6 +178,7 @@ horizontally.
>> .. _V4L2-PIX-FMT-NV21:
>> .. _V4L2-PIX-FMT-NV12M:
>> .. _V4L2-PIX-FMT-NV21M:
>> +.. _V4L2-PIX-FMT-P010:
> The NV12/21 documentation is not sufficient to describe this format. While it
> shares the layout (two planes Y and interleaved UV), it does not share the
> packing. In this case, assuming this is P010 (and not the P010 the Rockchip
> tried to upstreamed previously), each 10bit worth of pixel data would be pakced
> into 16 bits with the least significant 6 bit being padding bits.
Yes it 10 bits packed in 16 bits. I was think that the 'Bits per component' columns
was referring to the number of encoded bits.
So like this is it fine ? or does something else than the bit per component need to be changed ?
* - V4L2_PIX_FMT_P010
- 'P010'
- 16
- 4:2:0
- Cb, Cr
- No
- Linear
Regards,
Benjamin
>
>>
>> NV12, NV21, NV12M and NV21M
>> ---------------------------
>> diff --git a/drivers/media/v4l2-core/v4l2-common.c b/drivers/media/v4l2-core/v4l2-common.c
>> index 04af03285a20..37b5d82359dd 100644
>> --- a/drivers/media/v4l2-core/v4l2-common.c
>> +++ b/drivers/media/v4l2-core/v4l2-common.c
>> @@ -266,6 +266,7 @@ const struct v4l2_format_info *v4l2_format_info(u32 format)
>> { .format = V4L2_PIX_FMT_NV61, .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 1, .comp_planes = 2, .bpp = { 1, 2, 0, 0 }, .hdiv = 2, .vdiv = 1 },
>> { .format = V4L2_PIX_FMT_NV24, .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 1, .comp_planes = 2, .bpp = { 1, 2, 0, 0 }, .hdiv = 1, .vdiv = 1 },
>> { .format = V4L2_PIX_FMT_NV42, .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 1, .comp_planes = 2, .bpp = { 1, 2, 0, 0 }, .hdiv = 1, .vdiv = 1 },
>> + { .format = V4L2_PIX_FMT_P010, .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 1, .comp_planes = 2, .bpp = { 2, 2, 0, 0 }, .hdiv = 2, .vdiv = 1 },
>>
>> { .format = V4L2_PIX_FMT_YUV410, .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 1, .comp_planes = 3, .bpp = { 1, 1, 1, 0 }, .hdiv = 4, .vdiv = 4 },
>> { .format = V4L2_PIX_FMT_YVU410, .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 1, .comp_planes = 3, .bpp = { 1, 1, 1, 0 }, .hdiv = 4, .vdiv = 4 },
>> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
>> index 2673f51aafa4..6404d5b6e350 100644
>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>> @@ -1282,6 +1282,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
>> case V4L2_PIX_FMT_NV61: descr = "Y/CrCb 4:2:2"; break;
>> case V4L2_PIX_FMT_NV24: descr = "Y/CbCr 4:4:4"; break;
>> case V4L2_PIX_FMT_NV42: descr = "Y/CrCb 4:4:4"; break;
>> + case V4L2_PIX_FMT_P010: descr = "10-bit Y/CrCb 4:2:0"; break;
>> case V4L2_PIX_FMT_NV12M: descr = "Y/CbCr 4:2:0 (N-C)"; break;
>> case V4L2_PIX_FMT_NV21M: descr = "Y/CrCb 4:2:0 (N-C)"; break;
>> case V4L2_PIX_FMT_NV16M: descr = "Y/CbCr 4:2:2 (N-C)"; break;
>> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
>> index 9260791b8438..e5f7acde0730 100644
>> --- a/include/uapi/linux/videodev2.h
>> +++ b/include/uapi/linux/videodev2.h
>> @@ -602,6 +602,7 @@ struct v4l2_pix_format {
>> #define V4L2_PIX_FMT_NV24 v4l2_fourcc('N', 'V', '2', '4') /* 24 Y/CbCr 4:4:4 */
>> #define V4L2_PIX_FMT_NV42 v4l2_fourcc('N', 'V', '4', '2') /* 24 Y/CrCb 4:4:4 */
>> #define V4L2_PIX_FMT_HM12 v4l2_fourcc('H', 'M', '1', '2') /* 8 YUV 4:2:0 16x16 macroblocks */
>> +#define V4L2_PIX_FMT_P010 v4l2_fourcc('P', '0', '1', '0') /* 15 Y/CbCr 4:2:0 10-bit per pixel*/
>>
>> /* two non contiguous planes - one Y, one Cr + Cb interleaved */
>> #define V4L2_PIX_FMT_NV12M v4l2_fourcc('N', 'M', '1', '2') /* 12 Y/CbCr 4:2:0 */
>
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH 2/8] media: hantro: Add support of compressed reference buffers
2021-06-04 16:32 ` Ezequiel Garcia
@ 2021-06-07 13:27 ` Benjamin Gaignard
0 siblings, 0 replies; 21+ messages in thread
From: Benjamin Gaignard @ 2021-06-07 13:27 UTC (permalink / raw)
To: Ezequiel Garcia, hverkuil, p.zabel, mchehab, shawnguo, s.hauer,
festevam, gregkh, mripard, paul.kocialkowski, wens,
jernej.skrabec, emil.l.velikov, andrzej.p, jc
Cc: kernel, linux-imx, linux-media, linux-rockchip, linux-arm-kernel,
linux-kernel
Le 04/06/2021 à 18:32, Ezequiel Garcia a écrit :
> Hi Benjamin,
>
> Thanks for the patches, they looks really nice!
>
> On Fri, 2021-06-04 at 15:06 +0200, Benjamin Gaignard wrote:
>> Reference frame compression is a feature added in G2 decoder to compress
>> frame buffers so that the bandwidth of storing/loading reference frames
>> can be reduced, especially when the resolution of decoded stream is of
>> high definition.
>>
> Can you do some tracing using the performance counters in patch 1,
> with and without this patch, while we are here, it would be interesting
> to run some experiments.
>
> If I understand correctly, the G2 HEVC engine is producing tiled buffers,
> which are then post-processed to NV12.
>
> So this compression feature is compressing the tiled buffers,
> the NV12 are still uncompressed. If this is the case, could add
> some comments somewhere about the different sets of buffers involved?
I will do that
>
> Also, I believe I mentioned this before, is there a plan
> to be able to expose the tiled buffers to userspace (i.e. without
> the NV12 linear post-processing)? I think the display controller
> can take compressed tiled-buffers.
Yes it could be possible to use vb2 buffers instead of internally
allocated buffers but that would means define a new format/modifier
before doing that.
>
>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>> ---
>> .../staging/media/hantro/hantro_g2_hevc_dec.c | 20 ++++++++++--
>> drivers/staging/media/hantro/hantro_g2_regs.h | 4 +++
>> drivers/staging/media/hantro/hantro_hevc.c | 32 ++++++++++++++++++-
>> drivers/staging/media/hantro/hantro_hw.h | 2 ++
>> 4 files changed, 54 insertions(+), 4 deletions(-)
>>
>> diff --git a/drivers/staging/media/hantro/hantro_g2_hevc_dec.c b/drivers/staging/media/hantro/hantro_g2_hevc_dec.c
>> index 89fac5146433..9a715e803037 100644
>> --- a/drivers/staging/media/hantro/hantro_g2_hevc_dec.c
>> +++ b/drivers/staging/media/hantro/hantro_g2_hevc_dec.c
>> @@ -382,10 +382,12 @@ static int set_ref(struct hantro_ctx *ctx)
>> const struct v4l2_ctrl_hevc_pps *pps = ctrls->pps;
>> const struct v4l2_ctrl_hevc_decode_params *decode_params = ctrls->decode_params;
>> const struct v4l2_hevc_dpb_entry *dpb = decode_params->dpb;
>> - dma_addr_t luma_addr, chroma_addr, mv_addr = 0;
>> + dma_addr_t luma_addr, chroma_addr, mv_addr, compress_luma_addr, compress_chroma_addr = 0;
>> struct hantro_dev *vpu = ctx->dev;
>> size_t cr_offset = hantro_hevc_chroma_offset(sps);
>> size_t mv_offset = hantro_hevc_motion_vectors_offset(sps);
>> + size_t compress_luma_offset = hantro_hevc_luma_compress_offset(sps);
>> + size_t compress_chroma_offset = hantro_hevc_chroma_compress_offset(sps);
>> u32 max_ref_frames;
>> u16 dpb_longterm_e;
>> static const struct hantro_reg cur_poc[] = {
>> @@ -460,6 +462,8 @@ static int set_ref(struct hantro_ctx *ctx)
>>
>> chroma_addr = luma_addr + cr_offset;
>> mv_addr = luma_addr + mv_offset;
>> + compress_luma_addr = luma_addr + compress_luma_offset;
>> + compress_chroma_addr = luma_addr + compress_chroma_offset;
>>
>> if (dpb[i].rps == V4L2_HEVC_DPB_ENTRY_RPS_LT_CURR)
>> dpb_longterm_e |= BIT(V4L2_HEVC_DPB_ENTRIES_NUM_MAX - 1 - i);
>> @@ -467,6 +471,8 @@ static int set_ref(struct hantro_ctx *ctx)
>> hantro_write_addr(vpu, G2_REG_ADDR_REF(i), luma_addr);
>> hantro_write_addr(vpu, G2_REG_CHR_REF(i), chroma_addr);
>> hantro_write_addr(vpu, G2_REG_DMV_REF(i), mv_addr);
>> + hantro_write_addr(vpu, G2_COMP_ADDR_REF(i), compress_luma_addr);
>> + hantro_write_addr(vpu, G2_COMP_CHR_REF(i), compress_chroma_addr);
> I'm a bit confused here, you are seeing luma_addr and also compress_luma_addr
> for references??
Yes both are need to make this work. It also done like that in vendor stack.
>
> I'm not saying this is wrong, just would like some comments in the code
> so the intention is a bit more clear and I can understand what's going on.
I will add comments to clarify that point.
Benjamin
>
>> }
>>
>> luma_addr = hantro_hevc_get_ref_buf(ctx, decode_params->pic_order_cnt_val);
>> @@ -475,7 +481,11 @@ static int set_ref(struct hantro_ctx *ctx)
>>
>> chroma_addr = luma_addr + cr_offset;
>> mv_addr = luma_addr + mv_offset;
>> + compress_luma_addr = luma_addr + compress_luma_offset;
>> + compress_chroma_addr = luma_addr + compress_chroma_offset;
>>
>> + hantro_write_addr(vpu, G2_COMP_ADDR_REF(i), compress_luma_addr);
>> + hantro_write_addr(vpu, G2_COMP_CHR_REF(i), compress_chroma_addr);
>> hantro_write_addr(vpu, G2_REG_ADDR_REF(i), luma_addr);
>> hantro_write_addr(vpu, G2_REG_CHR_REF(i), chroma_addr);
>> hantro_write_addr(vpu, G2_REG_DMV_REF(i++), mv_addr);
>> @@ -483,6 +493,8 @@ static int set_ref(struct hantro_ctx *ctx)
>> hantro_write_addr(vpu, G2_ADDR_DST, luma_addr);
>> hantro_write_addr(vpu, G2_ADDR_DST_CHR, chroma_addr);
>> hantro_write_addr(vpu, G2_ADDR_DST_MV, mv_addr);
>> + hantro_write_addr(vpu, G2_COMP_ADDR_DST, compress_luma_addr);
>> + hantro_write_addr(vpu, G2_COMP_CHR, compress_chroma_addr);
>>
>> hantro_hevc_ref_remove_unused(ctx);
>>
>> @@ -490,6 +502,8 @@ static int set_ref(struct hantro_ctx *ctx)
>> hantro_write_addr(vpu, G2_REG_ADDR_REF(i), 0);
>> hantro_write_addr(vpu, G2_REG_CHR_REF(i), 0);
>> hantro_write_addr(vpu, G2_REG_DMV_REF(i), 0);
>> + hantro_write_addr(vpu, G2_COMP_ADDR_REF(i), 0);
>> + hantro_write_addr(vpu, G2_COMP_CHR_REF(i), 0);
>> }
>>
>> hantro_reg_write(vpu, &g2_refer_lterm_e, dpb_longterm_e);
>> @@ -580,8 +594,8 @@ int hantro_g2_hevc_dec_run(struct hantro_ctx *ctx)
>> /* Don't disable output */
>> hantro_reg_write(vpu, &g2_out_dis, 0);
>>
>> - /* Don't compress buffers */
>> - hantro_reg_write(vpu, &g2_ref_compress_bypass, 1);
>> + /* Compress buffers */
>> + hantro_reg_write(vpu, &g2_ref_compress_bypass, 0);
>>
>> /* use NV12 as output format */
>> hantro_reg_write(vpu, &g2_out_rs_e, 1);
>> diff --git a/drivers/staging/media/hantro/hantro_g2_regs.h b/drivers/staging/media/hantro/hantro_g2_regs.h
>> index 17d84ec9c5c2..0414d92e3860 100644
>> --- a/drivers/staging/media/hantro/hantro_g2_regs.h
>> +++ b/drivers/staging/media/hantro/hantro_g2_regs.h
>> @@ -192,6 +192,10 @@
>> #define G2_TILE_FILTER (G2_SWREG(179))
>> #define G2_TILE_SAO (G2_SWREG(181))
>> #define G2_TILE_BSD (G2_SWREG(183))
>> +#define G2_COMP_ADDR_DST (G2_SWREG(190))
>> +#define G2_COMP_ADDR_REF(i) (G2_SWREG(192) + ((i) * 0x8))
>> +#define G2_COMP_CHR (G2_SWREG(224))
>> +#define G2_COMP_CHR_REF(i) (G2_SWREG(226) + ((i) * 0x8))
>>
>> #define g2_strm_buffer_len G2_DEC_REG(258, 0, 0xffffffff)
>> #define g2_strm_start_offset G2_DEC_REG(259, 0, 0xffffffff)
>> diff --git a/drivers/staging/media/hantro/hantro_hevc.c b/drivers/staging/media/hantro/hantro_hevc.c
>> index 5347f5a41c2a..1b2da990fbf0 100644
>> --- a/drivers/staging/media/hantro/hantro_hevc.c
>> +++ b/drivers/staging/media/hantro/hantro_hevc.c
>> @@ -61,12 +61,42 @@ static size_t hantro_hevc_mv_size(const struct v4l2_ctrl_hevc_sps *sps)
>> return mv_size;
>> }
>>
>> +size_t hantro_hevc_luma_compress_offset(const struct v4l2_ctrl_hevc_sps *sps)
>> +{
>> + return hantro_hevc_motion_vectors_offset(sps) + hantro_hevc_mv_size(sps);
>> +}
>> +
>> +static size_t hantro_hevc_luma_compress_size(const struct v4l2_ctrl_hevc_sps *sps)
>> +{
>> + u32 pic_width_in_cbsy =
>> + round_up((sps->pic_width_in_luma_samples + 8 - 1) / 8, 16);
>> + u32 pic_height_in_cbsy = (sps->pic_height_in_luma_samples + 8 - 1) / 8;
>> +
>> + return round_up(pic_width_in_cbsy * pic_height_in_cbsy, 16);
>> +}
>> +
>> +size_t hantro_hevc_chroma_compress_offset(const struct v4l2_ctrl_hevc_sps *sps)
>> +{
>> + return hantro_hevc_luma_compress_offset(sps) + hantro_hevc_luma_compress_size(sps);
>> +}
>> +
>> +static size_t hantro_hevc_chroma_compress_size(const struct v4l2_ctrl_hevc_sps *sps)
>> +{
>> + u32 pic_width_in_cbsc =
>> + round_up((sps->pic_width_in_luma_samples + 16 - 1) / 16, 16);
>> + u32 pic_height_in_cbsc = (sps->pic_height_in_luma_samples / 2 + 4 - 1) / 4;
>> +
> Can we get some #defines for these magic numbers?
>
>> + return round_up(pic_width_in_cbsc * pic_height_in_cbsc, 16);
>> +}
>> +
>> static size_t hantro_hevc_ref_size(struct hantro_ctx *ctx)
>> {
>> const struct hantro_hevc_dec_ctrls *ctrls = &ctx->hevc_dec.ctrls;
>> const struct v4l2_ctrl_hevc_sps *sps = ctrls->sps;
>>
>> - return hantro_hevc_motion_vectors_offset(sps) + hantro_hevc_mv_size(sps);
>> + return hantro_hevc_motion_vectors_offset(sps) + hantro_hevc_mv_size(sps) +
>> + hantro_hevc_luma_compress_size(sps) +
>> + hantro_hevc_chroma_compress_size(sps);
>> }
>>
>> static void hantro_hevc_ref_free(struct hantro_ctx *ctx)
>> diff --git a/drivers/staging/media/hantro/hantro_hw.h b/drivers/staging/media/hantro/hantro_hw.h
>> index 8fa0aacb61cd..c5374cd74d66 100644
>> --- a/drivers/staging/media/hantro/hantro_hw.h
>> +++ b/drivers/staging/media/hantro/hantro_hw.h
>> @@ -246,6 +246,8 @@ dma_addr_t hantro_hevc_get_ref_buf(struct hantro_ctx *ctx, int poc);
>> void hantro_hevc_ref_remove_unused(struct hantro_ctx *ctx);
>> size_t hantro_hevc_chroma_offset(const struct v4l2_ctrl_hevc_sps *sps);
>> size_t hantro_hevc_motion_vectors_offset(const struct v4l2_ctrl_hevc_sps *sps);
>> +size_t hantro_hevc_luma_compress_offset(const struct v4l2_ctrl_hevc_sps *sps);
>> +size_t hantro_hevc_chroma_compress_offset(const struct v4l2_ctrl_hevc_sps *sps);
>>
>> static inline size_t
>> hantro_h264_mv_size(unsigned int width, unsigned int height)
> Thanks,
> Ezequiel
>
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH 4/8] media: Add P010 video format
2021-06-07 11:31 ` Benjamin Gaignard
@ 2021-06-09 17:58 ` Nicolas Dufresne
0 siblings, 0 replies; 21+ messages in thread
From: Nicolas Dufresne @ 2021-06-09 17:58 UTC (permalink / raw)
To: Benjamin Gaignard, hverkuil, ezequiel, p.zabel, mchehab,
shawnguo, s.hauer, festevam, gregkh, mripard, paul.kocialkowski,
wens, jernej.skrabec, emil.l.velikov, andrzej.p, jc
Cc: kernel, linux-imx, linux-media, linux-rockchip, linux-arm-kernel,
linux-kernel
Le lundi 07 juin 2021 à 13:31 +0200, Benjamin Gaignard a écrit :
> Le 04/06/2021 à 18:17, Nicolas Dufresne a écrit :
> > Le vendredi 04 juin 2021 à 15:06 +0200, Benjamin Gaignard a écrit :
> > > P010 is a YUV format with 10-bits per pixel with interleaved UV.
> > >
> > > Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
> > > ---
> > > .../userspace-api/media/v4l/pixfmt-yuv-planar.rst | 8 ++++++++
> > > drivers/media/v4l2-core/v4l2-common.c | 1 +
> > > drivers/media/v4l2-core/v4l2-ioctl.c | 1 +
> > > include/uapi/linux/videodev2.h | 1 +
> > > 4 files changed, 11 insertions(+)
> > >
> > > diff --git a/Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst b/Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst
> > > index 090c091affd2..71fed70c03ec 100644
> > > --- a/Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst
> > > +++ b/Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst
> > > @@ -100,6 +100,13 @@ All components are stored with the same number of bits per component.
> > > - Cb, Cr
> > > - No
> > > - 64x32 macroblocks
> > > + * - V4L2_PIX_FMT_P010
> > > + - 'P010'
> > > + - 10
> > > + - 4:2:0
> > > + - Cb, Cr
> > > + - No
> > > + - Linear
> > >
> > > Horizontal Z order
> > > * - V4L2_PIX_FMT_NV12MT_16X16
> > > @@ -171,6 +178,7 @@ horizontally.
> > > .. _V4L2-PIX-FMT-NV21:
> > > .. _V4L2-PIX-FMT-NV12M:
> > > .. _V4L2-PIX-FMT-NV21M:
> > > +.. _V4L2-PIX-FMT-P010:
> > The NV12/21 documentation is not sufficient to describe this format. While it
> > shares the layout (two planes Y and interleaved UV), it does not share the
> > packing. In this case, assuming this is P010 (and not the P010 the Rockchip
> > tried to upstreamed previously), each 10bit worth of pixel data would be pakced
> > into 16 bits with the least significant 6 bit being padding bits.
>
> Yes it 10 bits packed in 16 bits. I was think that the 'Bits per component' columns
> was referring to the number of encoded bits.
> So like this is it fine ? or does something else than the bit per component need to be changed ?
> * - V4L2_PIX_FMT_P010
> - 'P010'
> - 16
> - 4:2:0
> - Cb, Cr
> - No
> - Linear
I believe we also need a table, similar to what we do for BGRx and other padded
format in order to document were the padding is located. Specially, with 16bit
formats, which could also have a big engian variants if handled per 16bit words.
>
> Regards,
> Benjamin
>
> >
> > >
> > > NV12, NV21, NV12M and NV21M
> > > ---------------------------
> > > diff --git a/drivers/media/v4l2-core/v4l2-common.c b/drivers/media/v4l2-core/v4l2-common.c
> > > index 04af03285a20..37b5d82359dd 100644
> > > --- a/drivers/media/v4l2-core/v4l2-common.c
> > > +++ b/drivers/media/v4l2-core/v4l2-common.c
> > > @@ -266,6 +266,7 @@ const struct v4l2_format_info *v4l2_format_info(u32 format)
> > > { .format = V4L2_PIX_FMT_NV61, .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 1, .comp_planes = 2, .bpp = { 1, 2, 0, 0 }, .hdiv = 2, .vdiv = 1 },
> > > { .format = V4L2_PIX_FMT_NV24, .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 1, .comp_planes = 2, .bpp = { 1, 2, 0, 0 }, .hdiv = 1, .vdiv = 1 },
> > > { .format = V4L2_PIX_FMT_NV42, .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 1, .comp_planes = 2, .bpp = { 1, 2, 0, 0 }, .hdiv = 1, .vdiv = 1 },
> > > + { .format = V4L2_PIX_FMT_P010, .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 1, .comp_planes = 2, .bpp = { 2, 2, 0, 0 }, .hdiv = 2, .vdiv = 1 },
> > >
> > > { .format = V4L2_PIX_FMT_YUV410, .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 1, .comp_planes = 3, .bpp = { 1, 1, 1, 0 }, .hdiv = 4, .vdiv = 4 },
> > > { .format = V4L2_PIX_FMT_YVU410, .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 1, .comp_planes = 3, .bpp = { 1, 1, 1, 0 }, .hdiv = 4, .vdiv = 4 },
> > > diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
> > > index 2673f51aafa4..6404d5b6e350 100644
> > > --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> > > +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> > > @@ -1282,6 +1282,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
> > > case V4L2_PIX_FMT_NV61: descr = "Y/CrCb 4:2:2"; break;
> > > case V4L2_PIX_FMT_NV24: descr = "Y/CbCr 4:4:4"; break;
> > > case V4L2_PIX_FMT_NV42: descr = "Y/CrCb 4:4:4"; break;
> > > + case V4L2_PIX_FMT_P010: descr = "10-bit Y/CrCb 4:2:0"; break;
> > > case V4L2_PIX_FMT_NV12M: descr = "Y/CbCr 4:2:0 (N-C)"; break;
> > > case V4L2_PIX_FMT_NV21M: descr = "Y/CrCb 4:2:0 (N-C)"; break;
> > > case V4L2_PIX_FMT_NV16M: descr = "Y/CbCr 4:2:2 (N-C)"; break;
> > > diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> > > index 9260791b8438..e5f7acde0730 100644
> > > --- a/include/uapi/linux/videodev2.h
> > > +++ b/include/uapi/linux/videodev2.h
> > > @@ -602,6 +602,7 @@ struct v4l2_pix_format {
> > > #define V4L2_PIX_FMT_NV24 v4l2_fourcc('N', 'V', '2', '4') /* 24 Y/CbCr 4:4:4 */
> > > #define V4L2_PIX_FMT_NV42 v4l2_fourcc('N', 'V', '4', '2') /* 24 Y/CrCb 4:4:4 */
> > > #define V4L2_PIX_FMT_HM12 v4l2_fourcc('H', 'M', '1', '2') /* 8 YUV 4:2:0 16x16 macroblocks */
> > > +#define V4L2_PIX_FMT_P010 v4l2_fourcc('P', '0', '1', '0') /* 15 Y/CbCr 4:2:0 10-bit per pixel*/
> > >
> > > /* two non contiguous planes - one Y, one Cr + Cb interleaved */
> > > #define V4L2_PIX_FMT_NV12M v4l2_fourcc('N', 'M', '1', '2') /* 12 Y/CbCr 4:2:0 */
> >
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH 1/8] media: hantro: Trace hevc hw cycles performance register
2021-06-04 13:06 ` [PATCH 1/8] media: hantro: Trace hevc hw cycles performance register Benjamin Gaignard
@ 2021-06-13 9:53 ` Hans Verkuil
2021-06-13 9:55 ` Hans Verkuil
0 siblings, 1 reply; 21+ messages in thread
From: Hans Verkuil @ 2021-06-13 9:53 UTC (permalink / raw)
To: Benjamin Gaignard, ezequiel, p.zabel, mchehab, shawnguo, s.hauer,
festevam, gregkh, mripard, paul.kocialkowski, wens,
jernej.skrabec, emil.l.velikov, andrzej.p, jc
Cc: kernel, linux-imx, linux-media, linux-rockchip, linux-arm-kernel,
linux-kernel
On 04/06/2021 15:06, Benjamin Gaignard wrote:
> After each hevc decoded frame trace the hardware performance.
> It provides the number of hw cycles spend per decoded macroblock.
>
> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
> ---
> drivers/staging/media/hantro/hantro_drv.c | 3 ++
> .../staging/media/hantro/hantro_g2_hevc_dec.c | 16 ++++++++
> drivers/staging/media/hantro/hantro_g2_regs.h | 1 +
> drivers/staging/media/hantro/hantro_hw.h | 1 +
> drivers/staging/media/hantro/imx8m_vpu_hw.c | 1 +
> drivers/staging/media/hantro/trace.h | 40 +++++++++++++++++++
Can you rename this to hantro_trace.h? I prefer to have the driver prefix in the name.
Regards,
Hans
> 6 files changed, 62 insertions(+)
> create mode 100644 drivers/staging/media/hantro/trace.h
>
> diff --git a/drivers/staging/media/hantro/hantro_drv.c b/drivers/staging/media/hantro/hantro_drv.c
> index dbc69ee0b562..6053c86b1c3f 100644
> --- a/drivers/staging/media/hantro/hantro_drv.c
> +++ b/drivers/staging/media/hantro/hantro_drv.c
> @@ -28,6 +28,9 @@
> #include "hantro.h"
> #include "hantro_hw.h"
>
> +#define CREATE_TRACE_POINTS
> +#include "trace.h"
> +
> #define DRIVER_NAME "hantro-vpu"
>
> int hantro_debug;
> diff --git a/drivers/staging/media/hantro/hantro_g2_hevc_dec.c b/drivers/staging/media/hantro/hantro_g2_hevc_dec.c
> index 340efb57fd18..89fac5146433 100644
> --- a/drivers/staging/media/hantro/hantro_g2_hevc_dec.c
> +++ b/drivers/staging/media/hantro/hantro_g2_hevc_dec.c
> @@ -7,6 +7,7 @@
>
> #include "hantro_hw.h"
> #include "hantro_g2_regs.h"
> +#include "trace.h"
>
> #define HEVC_DEC_MODE 0xC
>
> @@ -22,6 +23,21 @@ static inline void hantro_write_addr(struct hantro_dev *vpu,
> vdpu_write(vpu, addr & 0xffffffff, offset);
> }
>
> +void hantro_g2_hevc_dec_done(struct hantro_ctx *ctx)
> +{
> + const struct hantro_hevc_dec_ctrls *ctrls = &ctx->hevc_dec.ctrls;
> + const struct v4l2_ctrl_hevc_sps *sps = ctrls->sps;
> + struct hantro_dev *vpu = ctx->dev;
> + u32 hw_cycles = 0;
> + u32 mbs = (sps->pic_width_in_luma_samples *
> + sps->pic_height_in_luma_samples) >> 8;
> +
> + if (mbs)
> + hw_cycles = vdpu_read(vpu, G2_HW_PERFORMANCE) / mbs;
> +
> + trace_hantro_hevc_perf(ctx, hw_cycles);
> +}
> +
> static void prepare_tile_info_buffer(struct hantro_ctx *ctx)
> {
> struct hantro_dev *vpu = ctx->dev;
> diff --git a/drivers/staging/media/hantro/hantro_g2_regs.h b/drivers/staging/media/hantro/hantro_g2_regs.h
> index bb22fa921914..17d84ec9c5c2 100644
> --- a/drivers/staging/media/hantro/hantro_g2_regs.h
> +++ b/drivers/staging/media/hantro/hantro_g2_regs.h
> @@ -177,6 +177,7 @@
> #define G2_REG_CONFIG_DEC_CLK_GATE_E BIT(16)
> #define G2_REG_CONFIG_DEC_CLK_GATE_IDLE_E BIT(17)
>
> +#define G2_HW_PERFORMANCE (G2_SWREG(63))
> #define G2_ADDR_DST (G2_SWREG(65))
> #define G2_REG_ADDR_REF(i) (G2_SWREG(67) + ((i) * 0x8))
> #define G2_ADDR_DST_CHR (G2_SWREG(99))
> diff --git a/drivers/staging/media/hantro/hantro_hw.h b/drivers/staging/media/hantro/hantro_hw.h
> index 5737a7707944..8fa0aacb61cd 100644
> --- a/drivers/staging/media/hantro/hantro_hw.h
> +++ b/drivers/staging/media/hantro/hantro_hw.h
> @@ -240,6 +240,7 @@ void hantro_h264_dec_exit(struct hantro_ctx *ctx);
> int hantro_hevc_dec_init(struct hantro_ctx *ctx);
> void hantro_hevc_dec_exit(struct hantro_ctx *ctx);
> int hantro_g2_hevc_dec_run(struct hantro_ctx *ctx);
> +void hantro_g2_hevc_dec_done(struct hantro_ctx *ctx);
> int hantro_hevc_dec_prepare_run(struct hantro_ctx *ctx);
> dma_addr_t hantro_hevc_get_ref_buf(struct hantro_ctx *ctx, int poc);
> void hantro_hevc_ref_remove_unused(struct hantro_ctx *ctx);
> diff --git a/drivers/staging/media/hantro/imx8m_vpu_hw.c b/drivers/staging/media/hantro/imx8m_vpu_hw.c
> index ea919bfb9891..7e9e24bb5057 100644
> --- a/drivers/staging/media/hantro/imx8m_vpu_hw.c
> +++ b/drivers/staging/media/hantro/imx8m_vpu_hw.c
> @@ -239,6 +239,7 @@ static const struct hantro_codec_ops imx8mq_vpu_g2_codec_ops[] = {
> .reset = imx8m_vpu_g2_reset,
> .init = hantro_hevc_dec_init,
> .exit = hantro_hevc_dec_exit,
> + .done = hantro_g2_hevc_dec_done,
> },
> };
>
> diff --git a/drivers/staging/media/hantro/trace.h b/drivers/staging/media/hantro/trace.h
> new file mode 100644
> index 000000000000..8abe5ddb4814
> --- /dev/null
> +++ b/drivers/staging/media/hantro/trace.h
> @@ -0,0 +1,40 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +#undef TRACE_SYSTEM
> +#define TRACE_SYSTEM hantro
> +
> +#if !defined(__HANTRO_TRACE_H__) || defined(TRACE_HEADER_MULTI_READ)
> +#define __HANTRO_TRACE_H__
> +
> +#include <linux/tracepoint.h>
> +#include <media/videobuf2-v4l2.h>
> +
> +#include "hantro.h"
> +
> +TRACE_EVENT(hantro_hevc_perf,
> + TP_PROTO(struct hantro_ctx *ctx, u32 hw_cycles),
> +
> + TP_ARGS(ctx, hw_cycles),
> +
> + TP_STRUCT__entry(
> + __field(int, minor)
> + __field(u32, hw_cycles)
> + ),
> +
> + TP_fast_assign(
> + __entry->minor = ctx->fh.vdev->minor;
> + __entry->hw_cycles = hw_cycles;
> + ),
> +
> + TP_printk("minor = %d, %8d cycles / mb",
> + __entry->minor, __entry->hw_cycles)
> +);
> +
> +#endif /* __HANTRO_TRACE_H__ */
> +
> +#undef TRACE_INCLUDE_PATH
> +#define TRACE_INCLUDE_PATH ../../drivers/staging/media/hantro
> +#undef TRACE_INCLUDE_FILE
> +#define TRACE_INCLUDE_FILE trace
> +
> +/* This part must be outside protection */
> +#include <trace/define_trace.h>
>
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH 1/8] media: hantro: Trace hevc hw cycles performance register
2021-06-13 9:53 ` Hans Verkuil
@ 2021-06-13 9:55 ` Hans Verkuil
2021-06-14 7:45 ` Benjamin Gaignard
0 siblings, 1 reply; 21+ messages in thread
From: Hans Verkuil @ 2021-06-13 9:55 UTC (permalink / raw)
To: Benjamin Gaignard, ezequiel, p.zabel, mchehab, shawnguo, s.hauer,
festevam, gregkh, mripard, paul.kocialkowski, wens,
jernej.skrabec, emil.l.velikov, andrzej.p, jc
Cc: kernel, linux-imx, linux-media, linux-rockchip, linux-arm-kernel,
linux-kernel
On 13/06/2021 11:53, Hans Verkuil wrote:
> On 04/06/2021 15:06, Benjamin Gaignard wrote:
>> After each hevc decoded frame trace the hardware performance.
>> It provides the number of hw cycles spend per decoded macroblock.
>>
>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>> ---
>> drivers/staging/media/hantro/hantro_drv.c | 3 ++
>> .../staging/media/hantro/hantro_g2_hevc_dec.c | 16 ++++++++
>> drivers/staging/media/hantro/hantro_g2_regs.h | 1 +
>> drivers/staging/media/hantro/hantro_hw.h | 1 +
>> drivers/staging/media/hantro/imx8m_vpu_hw.c | 1 +
>> drivers/staging/media/hantro/trace.h | 40 +++++++++++++++++++
>
> Can you rename this to hantro_trace.h? I prefer to have the driver prefix in the name.
Ah, I accidentally replied to the v1, but the same comment applies to v2.
Regards,
Hans
>
> Regards,
>
> Hans
>
>> 6 files changed, 62 insertions(+)
>> create mode 100644 drivers/staging/media/hantro/trace.h
>>
>> diff --git a/drivers/staging/media/hantro/hantro_drv.c b/drivers/staging/media/hantro/hantro_drv.c
>> index dbc69ee0b562..6053c86b1c3f 100644
>> --- a/drivers/staging/media/hantro/hantro_drv.c
>> +++ b/drivers/staging/media/hantro/hantro_drv.c
>> @@ -28,6 +28,9 @@
>> #include "hantro.h"
>> #include "hantro_hw.h"
>>
>> +#define CREATE_TRACE_POINTS
>> +#include "trace.h"
>> +
>> #define DRIVER_NAME "hantro-vpu"
>>
>> int hantro_debug;
>> diff --git a/drivers/staging/media/hantro/hantro_g2_hevc_dec.c b/drivers/staging/media/hantro/hantro_g2_hevc_dec.c
>> index 340efb57fd18..89fac5146433 100644
>> --- a/drivers/staging/media/hantro/hantro_g2_hevc_dec.c
>> +++ b/drivers/staging/media/hantro/hantro_g2_hevc_dec.c
>> @@ -7,6 +7,7 @@
>>
>> #include "hantro_hw.h"
>> #include "hantro_g2_regs.h"
>> +#include "trace.h"
>>
>> #define HEVC_DEC_MODE 0xC
>>
>> @@ -22,6 +23,21 @@ static inline void hantro_write_addr(struct hantro_dev *vpu,
>> vdpu_write(vpu, addr & 0xffffffff, offset);
>> }
>>
>> +void hantro_g2_hevc_dec_done(struct hantro_ctx *ctx)
>> +{
>> + const struct hantro_hevc_dec_ctrls *ctrls = &ctx->hevc_dec.ctrls;
>> + const struct v4l2_ctrl_hevc_sps *sps = ctrls->sps;
>> + struct hantro_dev *vpu = ctx->dev;
>> + u32 hw_cycles = 0;
>> + u32 mbs = (sps->pic_width_in_luma_samples *
>> + sps->pic_height_in_luma_samples) >> 8;
>> +
>> + if (mbs)
>> + hw_cycles = vdpu_read(vpu, G2_HW_PERFORMANCE) / mbs;
>> +
>> + trace_hantro_hevc_perf(ctx, hw_cycles);
>> +}
>> +
>> static void prepare_tile_info_buffer(struct hantro_ctx *ctx)
>> {
>> struct hantro_dev *vpu = ctx->dev;
>> diff --git a/drivers/staging/media/hantro/hantro_g2_regs.h b/drivers/staging/media/hantro/hantro_g2_regs.h
>> index bb22fa921914..17d84ec9c5c2 100644
>> --- a/drivers/staging/media/hantro/hantro_g2_regs.h
>> +++ b/drivers/staging/media/hantro/hantro_g2_regs.h
>> @@ -177,6 +177,7 @@
>> #define G2_REG_CONFIG_DEC_CLK_GATE_E BIT(16)
>> #define G2_REG_CONFIG_DEC_CLK_GATE_IDLE_E BIT(17)
>>
>> +#define G2_HW_PERFORMANCE (G2_SWREG(63))
>> #define G2_ADDR_DST (G2_SWREG(65))
>> #define G2_REG_ADDR_REF(i) (G2_SWREG(67) + ((i) * 0x8))
>> #define G2_ADDR_DST_CHR (G2_SWREG(99))
>> diff --git a/drivers/staging/media/hantro/hantro_hw.h b/drivers/staging/media/hantro/hantro_hw.h
>> index 5737a7707944..8fa0aacb61cd 100644
>> --- a/drivers/staging/media/hantro/hantro_hw.h
>> +++ b/drivers/staging/media/hantro/hantro_hw.h
>> @@ -240,6 +240,7 @@ void hantro_h264_dec_exit(struct hantro_ctx *ctx);
>> int hantro_hevc_dec_init(struct hantro_ctx *ctx);
>> void hantro_hevc_dec_exit(struct hantro_ctx *ctx);
>> int hantro_g2_hevc_dec_run(struct hantro_ctx *ctx);
>> +void hantro_g2_hevc_dec_done(struct hantro_ctx *ctx);
>> int hantro_hevc_dec_prepare_run(struct hantro_ctx *ctx);
>> dma_addr_t hantro_hevc_get_ref_buf(struct hantro_ctx *ctx, int poc);
>> void hantro_hevc_ref_remove_unused(struct hantro_ctx *ctx);
>> diff --git a/drivers/staging/media/hantro/imx8m_vpu_hw.c b/drivers/staging/media/hantro/imx8m_vpu_hw.c
>> index ea919bfb9891..7e9e24bb5057 100644
>> --- a/drivers/staging/media/hantro/imx8m_vpu_hw.c
>> +++ b/drivers/staging/media/hantro/imx8m_vpu_hw.c
>> @@ -239,6 +239,7 @@ static const struct hantro_codec_ops imx8mq_vpu_g2_codec_ops[] = {
>> .reset = imx8m_vpu_g2_reset,
>> .init = hantro_hevc_dec_init,
>> .exit = hantro_hevc_dec_exit,
>> + .done = hantro_g2_hevc_dec_done,
>> },
>> };
>>
>> diff --git a/drivers/staging/media/hantro/trace.h b/drivers/staging/media/hantro/trace.h
>> new file mode 100644
>> index 000000000000..8abe5ddb4814
>> --- /dev/null
>> +++ b/drivers/staging/media/hantro/trace.h
>> @@ -0,0 +1,40 @@
>> +/* SPDX-License-Identifier: GPL-2.0 */
>> +#undef TRACE_SYSTEM
>> +#define TRACE_SYSTEM hantro
>> +
>> +#if !defined(__HANTRO_TRACE_H__) || defined(TRACE_HEADER_MULTI_READ)
>> +#define __HANTRO_TRACE_H__
>> +
>> +#include <linux/tracepoint.h>
>> +#include <media/videobuf2-v4l2.h>
>> +
>> +#include "hantro.h"
>> +
>> +TRACE_EVENT(hantro_hevc_perf,
>> + TP_PROTO(struct hantro_ctx *ctx, u32 hw_cycles),
>> +
>> + TP_ARGS(ctx, hw_cycles),
>> +
>> + TP_STRUCT__entry(
>> + __field(int, minor)
>> + __field(u32, hw_cycles)
>> + ),
>> +
>> + TP_fast_assign(
>> + __entry->minor = ctx->fh.vdev->minor;
>> + __entry->hw_cycles = hw_cycles;
>> + ),
>> +
>> + TP_printk("minor = %d, %8d cycles / mb",
>> + __entry->minor, __entry->hw_cycles)
>> +);
>> +
>> +#endif /* __HANTRO_TRACE_H__ */
>> +
>> +#undef TRACE_INCLUDE_PATH
>> +#define TRACE_INCLUDE_PATH ../../drivers/staging/media/hantro
>> +#undef TRACE_INCLUDE_FILE
>> +#define TRACE_INCLUDE_FILE trace
>> +
>> +/* This part must be outside protection */
>> +#include <trace/define_trace.h>
>>
>
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH 1/8] media: hantro: Trace hevc hw cycles performance register
2021-06-13 9:55 ` Hans Verkuil
@ 2021-06-14 7:45 ` Benjamin Gaignard
0 siblings, 0 replies; 21+ messages in thread
From: Benjamin Gaignard @ 2021-06-14 7:45 UTC (permalink / raw)
To: Hans Verkuil, ezequiel, p.zabel, mchehab, shawnguo, s.hauer,
festevam, gregkh, mripard, paul.kocialkowski, wens,
jernej.skrabec, emil.l.velikov, andrzej.p, jc
Cc: kernel, linux-imx, linux-media, linux-rockchip, linux-arm-kernel,
linux-kernel
Le 13/06/2021 à 11:55, Hans Verkuil a écrit :
> On 13/06/2021 11:53, Hans Verkuil wrote:
>> On 04/06/2021 15:06, Benjamin Gaignard wrote:
>>> After each hevc decoded frame trace the hardware performance.
>>> It provides the number of hw cycles spend per decoded macroblock.
>>>
>>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>>> ---
>>> drivers/staging/media/hantro/hantro_drv.c | 3 ++
>>> .../staging/media/hantro/hantro_g2_hevc_dec.c | 16 ++++++++
>>> drivers/staging/media/hantro/hantro_g2_regs.h | 1 +
>>> drivers/staging/media/hantro/hantro_hw.h | 1 +
>>> drivers/staging/media/hantro/imx8m_vpu_hw.c | 1 +
>>> drivers/staging/media/hantro/trace.h | 40 +++++++++++++++++++
>> Can you rename this to hantro_trace.h? I prefer to have the driver prefix in the name.
> Ah, I accidentally replied to the v1, but the same comment applies to v2.
I have fix that, it will be in v3.
I just wait for more feedback before send it.
Thanks,
Benjamin
>
> Regards,
>
> Hans
>
>> Regards,
>>
>> Hans
>>
>>> 6 files changed, 62 insertions(+)
>>> create mode 100644 drivers/staging/media/hantro/trace.h
>>>
>>> diff --git a/drivers/staging/media/hantro/hantro_drv.c b/drivers/staging/media/hantro/hantro_drv.c
>>> index dbc69ee0b562..6053c86b1c3f 100644
>>> --- a/drivers/staging/media/hantro/hantro_drv.c
>>> +++ b/drivers/staging/media/hantro/hantro_drv.c
>>> @@ -28,6 +28,9 @@
>>> #include "hantro.h"
>>> #include "hantro_hw.h"
>>>
>>> +#define CREATE_TRACE_POINTS
>>> +#include "trace.h"
>>> +
>>> #define DRIVER_NAME "hantro-vpu"
>>>
>>> int hantro_debug;
>>> diff --git a/drivers/staging/media/hantro/hantro_g2_hevc_dec.c b/drivers/staging/media/hantro/hantro_g2_hevc_dec.c
>>> index 340efb57fd18..89fac5146433 100644
>>> --- a/drivers/staging/media/hantro/hantro_g2_hevc_dec.c
>>> +++ b/drivers/staging/media/hantro/hantro_g2_hevc_dec.c
>>> @@ -7,6 +7,7 @@
>>>
>>> #include "hantro_hw.h"
>>> #include "hantro_g2_regs.h"
>>> +#include "trace.h"
>>>
>>> #define HEVC_DEC_MODE 0xC
>>>
>>> @@ -22,6 +23,21 @@ static inline void hantro_write_addr(struct hantro_dev *vpu,
>>> vdpu_write(vpu, addr & 0xffffffff, offset);
>>> }
>>>
>>> +void hantro_g2_hevc_dec_done(struct hantro_ctx *ctx)
>>> +{
>>> + const struct hantro_hevc_dec_ctrls *ctrls = &ctx->hevc_dec.ctrls;
>>> + const struct v4l2_ctrl_hevc_sps *sps = ctrls->sps;
>>> + struct hantro_dev *vpu = ctx->dev;
>>> + u32 hw_cycles = 0;
>>> + u32 mbs = (sps->pic_width_in_luma_samples *
>>> + sps->pic_height_in_luma_samples) >> 8;
>>> +
>>> + if (mbs)
>>> + hw_cycles = vdpu_read(vpu, G2_HW_PERFORMANCE) / mbs;
>>> +
>>> + trace_hantro_hevc_perf(ctx, hw_cycles);
>>> +}
>>> +
>>> static void prepare_tile_info_buffer(struct hantro_ctx *ctx)
>>> {
>>> struct hantro_dev *vpu = ctx->dev;
>>> diff --git a/drivers/staging/media/hantro/hantro_g2_regs.h b/drivers/staging/media/hantro/hantro_g2_regs.h
>>> index bb22fa921914..17d84ec9c5c2 100644
>>> --- a/drivers/staging/media/hantro/hantro_g2_regs.h
>>> +++ b/drivers/staging/media/hantro/hantro_g2_regs.h
>>> @@ -177,6 +177,7 @@
>>> #define G2_REG_CONFIG_DEC_CLK_GATE_E BIT(16)
>>> #define G2_REG_CONFIG_DEC_CLK_GATE_IDLE_E BIT(17)
>>>
>>> +#define G2_HW_PERFORMANCE (G2_SWREG(63))
>>> #define G2_ADDR_DST (G2_SWREG(65))
>>> #define G2_REG_ADDR_REF(i) (G2_SWREG(67) + ((i) * 0x8))
>>> #define G2_ADDR_DST_CHR (G2_SWREG(99))
>>> diff --git a/drivers/staging/media/hantro/hantro_hw.h b/drivers/staging/media/hantro/hantro_hw.h
>>> index 5737a7707944..8fa0aacb61cd 100644
>>> --- a/drivers/staging/media/hantro/hantro_hw.h
>>> +++ b/drivers/staging/media/hantro/hantro_hw.h
>>> @@ -240,6 +240,7 @@ void hantro_h264_dec_exit(struct hantro_ctx *ctx);
>>> int hantro_hevc_dec_init(struct hantro_ctx *ctx);
>>> void hantro_hevc_dec_exit(struct hantro_ctx *ctx);
>>> int hantro_g2_hevc_dec_run(struct hantro_ctx *ctx);
>>> +void hantro_g2_hevc_dec_done(struct hantro_ctx *ctx);
>>> int hantro_hevc_dec_prepare_run(struct hantro_ctx *ctx);
>>> dma_addr_t hantro_hevc_get_ref_buf(struct hantro_ctx *ctx, int poc);
>>> void hantro_hevc_ref_remove_unused(struct hantro_ctx *ctx);
>>> diff --git a/drivers/staging/media/hantro/imx8m_vpu_hw.c b/drivers/staging/media/hantro/imx8m_vpu_hw.c
>>> index ea919bfb9891..7e9e24bb5057 100644
>>> --- a/drivers/staging/media/hantro/imx8m_vpu_hw.c
>>> +++ b/drivers/staging/media/hantro/imx8m_vpu_hw.c
>>> @@ -239,6 +239,7 @@ static const struct hantro_codec_ops imx8mq_vpu_g2_codec_ops[] = {
>>> .reset = imx8m_vpu_g2_reset,
>>> .init = hantro_hevc_dec_init,
>>> .exit = hantro_hevc_dec_exit,
>>> + .done = hantro_g2_hevc_dec_done,
>>> },
>>> };
>>>
>>> diff --git a/drivers/staging/media/hantro/trace.h b/drivers/staging/media/hantro/trace.h
>>> new file mode 100644
>>> index 000000000000..8abe5ddb4814
>>> --- /dev/null
>>> +++ b/drivers/staging/media/hantro/trace.h
>>> @@ -0,0 +1,40 @@
>>> +/* SPDX-License-Identifier: GPL-2.0 */
>>> +#undef TRACE_SYSTEM
>>> +#define TRACE_SYSTEM hantro
>>> +
>>> +#if !defined(__HANTRO_TRACE_H__) || defined(TRACE_HEADER_MULTI_READ)
>>> +#define __HANTRO_TRACE_H__
>>> +
>>> +#include <linux/tracepoint.h>
>>> +#include <media/videobuf2-v4l2.h>
>>> +
>>> +#include "hantro.h"
>>> +
>>> +TRACE_EVENT(hantro_hevc_perf,
>>> + TP_PROTO(struct hantro_ctx *ctx, u32 hw_cycles),
>>> +
>>> + TP_ARGS(ctx, hw_cycles),
>>> +
>>> + TP_STRUCT__entry(
>>> + __field(int, minor)
>>> + __field(u32, hw_cycles)
>>> + ),
>>> +
>>> + TP_fast_assign(
>>> + __entry->minor = ctx->fh.vdev->minor;
>>> + __entry->hw_cycles = hw_cycles;
>>> + ),
>>> +
>>> + TP_printk("minor = %d, %8d cycles / mb",
>>> + __entry->minor, __entry->hw_cycles)
>>> +);
>>> +
>>> +#endif /* __HANTRO_TRACE_H__ */
>>> +
>>> +#undef TRACE_INCLUDE_PATH
>>> +#define TRACE_INCLUDE_PATH ../../drivers/staging/media/hantro
>>> +#undef TRACE_INCLUDE_FILE
>>> +#define TRACE_INCLUDE_FILE trace
>>> +
>>> +/* This part must be outside protection */
>>> +#include <trace/define_trace.h>
>>>
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 21+ messages in thread
end of thread, other threads:[~2021-06-14 7:49 UTC | newest]
Thread overview: 21+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-06-04 13:06 [PATCH 0/8] Additional features for Hantro HEVC Benjamin Gaignard
2021-06-04 13:06 ` [PATCH 1/8] media: hantro: Trace hevc hw cycles performance register Benjamin Gaignard
2021-06-13 9:53 ` Hans Verkuil
2021-06-13 9:55 ` Hans Verkuil
2021-06-14 7:45 ` Benjamin Gaignard
2021-06-04 13:06 ` [PATCH 2/8] media: hantro: Add support of compressed reference buffers Benjamin Gaignard
2021-06-04 16:32 ` Ezequiel Garcia
2021-06-07 13:27 ` Benjamin Gaignard
2021-06-04 13:06 ` [PATCH 3/8] media: hantro: hevc: Allow 10-bits encoded streams Benjamin Gaignard
2021-06-04 13:06 ` [PATCH 4/8] media: Add P010 video format Benjamin Gaignard
2021-06-04 16:17 ` Nicolas Dufresne
2021-06-07 11:31 ` Benjamin Gaignard
2021-06-09 17:58 ` Nicolas Dufresne
2021-06-04 13:06 ` [PATCH 5/8] media: hantro: hevc: Allow to produce 10-bit frames Benjamin Gaignard
2021-06-04 13:06 ` [PATCH 6/8] media: hantro: enumerate scaled output formats Benjamin Gaignard
2021-06-04 13:06 ` [PATCH 7/8] media: hevc: Add scaling matrix control Benjamin Gaignard
2021-06-06 7:49 ` Jernej Škrabec
2021-06-07 7:23 ` Benjamin Gaignard
2021-06-04 13:06 ` [PATCH 8/8] media: hantro: Add scaling lists feature Benjamin Gaignard
2021-06-04 16:49 ` [PATCH 0/8] Additional features for Hantro HEVC Ezequiel Garcia
2021-06-07 11:20 ` Benjamin Gaignard
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).