linux-rockchip.lists.infradead.org archive mirror
 help / color / mirror / Atom feed
* [PATCH] media: verisilicon: Fix null pointer dereference in try_fmt
@ 2023-05-16  9:12 Michael Tretter
  2023-05-23 14:22 ` Diederik de Haas
  2023-05-25 14:38 ` Nicolas Dufresne
  0 siblings, 2 replies; 6+ messages in thread
From: Michael Tretter @ 2023-05-16  9:12 UTC (permalink / raw)
  To: ezequiel, p.zabel, mchehab, hverkuil-cisco, benjamin.gaignard
  Cc: linux-media, linux-rockchip, kernel, m.tretter

Since commit db6f68b51e5c ("media: verisilicon: Do not set context
src/dst formats in reset functions"), vpu_src_fmt is not set in the
reset function, but only set in hantro_set_fmt_out, which calls
hantro_try_fmt before setting the format. Therefore, hantro_try_fmt
might be called with vpu_src_fmt still being null.

Add a test if the format is actually set before checking the format.

Signed-off-by: Michael Tretter <m.tretter@pengutronix.de>
Fixes: db6f68b51e5c ("media: verisilicon: Do not set context src/dst formats in reset functions")
---
 drivers/media/platform/verisilicon/hantro_v4l2.c | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/drivers/media/platform/verisilicon/hantro_v4l2.c b/drivers/media/platform/verisilicon/hantro_v4l2.c
index 835518534e3b..ec37d2646fde 100644
--- a/drivers/media/platform/verisilicon/hantro_v4l2.c
+++ b/drivers/media/platform/verisilicon/hantro_v4l2.c
@@ -313,17 +313,20 @@ static int hantro_try_fmt(const struct hantro_ctx *ctx,
 		/* Fill remaining fields */
 		v4l2_fill_pixfmt_mp(pix_mp, fmt->fourcc, pix_mp->width,
 				    pix_mp->height);
-		if (ctx->vpu_src_fmt->fourcc == V4L2_PIX_FMT_H264_SLICE &&
+		if (ctx->vpu_src_fmt &&
+		    ctx->vpu_src_fmt->fourcc == V4L2_PIX_FMT_H264_SLICE &&
 		    !hantro_needs_postproc(ctx, fmt))
 			pix_mp->plane_fmt[0].sizeimage +=
 				hantro_h264_mv_size(pix_mp->width,
 						    pix_mp->height);
-		else if (ctx->vpu_src_fmt->fourcc == V4L2_PIX_FMT_VP9_FRAME &&
+		else if (ctx->vpu_src_fmt &&
+			 ctx->vpu_src_fmt->fourcc == V4L2_PIX_FMT_VP9_FRAME &&
 			 !hantro_needs_postproc(ctx, fmt))
 			pix_mp->plane_fmt[0].sizeimage +=
 				hantro_vp9_mv_size(pix_mp->width,
 						   pix_mp->height);
-		else if (ctx->vpu_src_fmt->fourcc == V4L2_PIX_FMT_HEVC_SLICE &&
+		else if (ctx->vpu_src_fmt &&
+			 ctx->vpu_src_fmt->fourcc == V4L2_PIX_FMT_HEVC_SLICE &&
 			 !hantro_needs_postproc(ctx, fmt))
 			pix_mp->plane_fmt[0].sizeimage +=
 				hantro_hevc_mv_size(pix_mp->width,
-- 
2.39.2


_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip

^ permalink raw reply related	[flat|nested] 6+ messages in thread

end of thread, other threads:[~2023-05-25 15:17 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-05-16  9:12 [PATCH] media: verisilicon: Fix null pointer dereference in try_fmt Michael Tretter
2023-05-23 14:22 ` Diederik de Haas
2023-05-25 14:36   ` Nicolas Dufresne
2023-05-25 15:16     ` Diederik de Haas
2023-05-25 14:38 ` Nicolas Dufresne
2023-05-25 14:46   ` Hans Verkuil

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).