All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH] media: verisilicon: Fix crash when probing encoder
@ 2023-04-13  7:03 ` Benjamin Gaignard
  0 siblings, 0 replies; 4+ messages in thread
From: Benjamin Gaignard @ 2023-04-13  7:03 UTC (permalink / raw)
  To: ezequiel, p.zabel, mchehab, m.szyprowski
  Cc: linux-media, linux-rockchip, linux-kernel, kernel, Benjamin Gaignard

ctx->vpu_dst_fmt is no more initialized before calling hantro_try_fmt()
so assigne it to vpu_fmt led to crash the kernel.
Like for decoder case use 'fmt' as vpu format for encoder and clean up
the code.

Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
Tested-by: Marek Szyprowski <m.szyprowski@samsung.com>
Fixes: db6f68b51e5c ("media: verisilicon: Do not set context src/dst formats in reset functions")
---
 drivers/media/platform/verisilicon/hantro_v4l2.c | 8 +++-----
 1 file changed, 3 insertions(+), 5 deletions(-)

diff --git a/drivers/media/platform/verisilicon/hantro_v4l2.c b/drivers/media/platform/verisilicon/hantro_v4l2.c
index 8f1414085f47..51f528f36241 100644
--- a/drivers/media/platform/verisilicon/hantro_v4l2.c
+++ b/drivers/media/platform/verisilicon/hantro_v4l2.c
@@ -293,13 +293,11 @@ static int hantro_try_fmt(const struct hantro_ctx *ctx,
 		pix_mp->pixelformat = fmt->fourcc;
 	}
 
+	vpu_fmt = fmt;
+
 	if (coded) {
 		pix_mp->num_planes = 1;
-		vpu_fmt = fmt;
-	} else if (ctx->is_encoder) {
-		vpu_fmt = ctx->vpu_dst_fmt;
-	} else {
-		vpu_fmt = fmt;
+	} else if (!ctx->is_encoder) {
 		/*
 		 * Width/height on the CAPTURE end of a decoder are ignored and
 		 * replaced by the OUTPUT ones.
-- 
2.34.1


^ permalink raw reply related	[flat|nested] 4+ messages in thread
* [PATCH] media: verisilicon: Fix crash when probing encoder
@ 2023-04-13  7:03 ` Benjamin Gaignard
  0 siblings, 0 replies; 4+ messages in thread
From: Benjamin Gaignard @ 2023-04-13  7:03 UTC (permalink / raw)
  To: ezequiel, p.zabel, mchehab, m.szyprowski
  Cc: linux-media, linux-rockchip, linux-kernel, kernel, Benjamin Gaignard

ctx->vpu_dst_fmt is no more initialized before calling hantro_try_fmt()
so assigne it to vpu_fmt led to crash the kernel.
Like for decoder case use 'fmt' as vpu format for encoder and clean up
the code.

Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
Tested-by: Marek Szyprowski <m.szyprowski@samsung.com>
Fixes: db6f68b51e5c ("media: verisilicon: Do not set context src/dst formats in reset functions")
---
 drivers/media/platform/verisilicon/hantro_v4l2.c | 8 +++-----
 1 file changed, 3 insertions(+), 5 deletions(-)

diff --git a/drivers/media/platform/verisilicon/hantro_v4l2.c b/drivers/media/platform/verisilicon/hantro_v4l2.c
index 8f1414085f47..51f528f36241 100644
--- a/drivers/media/platform/verisilicon/hantro_v4l2.c
+++ b/drivers/media/platform/verisilicon/hantro_v4l2.c
@@ -293,13 +293,11 @@ static int hantro_try_fmt(const struct hantro_ctx *ctx,
 		pix_mp->pixelformat = fmt->fourcc;
 	}
 
+	vpu_fmt = fmt;
+
 	if (coded) {
 		pix_mp->num_planes = 1;
-		vpu_fmt = fmt;
-	} else if (ctx->is_encoder) {
-		vpu_fmt = ctx->vpu_dst_fmt;
-	} else {
-		vpu_fmt = fmt;
+	} else if (!ctx->is_encoder) {
 		/*
 		 * Width/height on the CAPTURE end of a decoder are ignored and
 		 * replaced by the OUTPUT ones.
-- 
2.34.1


_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip

^ permalink raw reply related	[flat|nested] 4+ messages in thread
* Re: [PATCH] media: verisilicon: Fix crash when probing encoder
  2023-04-13  7:03 ` Benjamin Gaignard
@ 2023-04-13 19:45   ` Nicolas Dufresne
  -1 siblings, 0 replies; 4+ messages in thread
From: Nicolas Dufresne @ 2023-04-13 19:45 UTC (permalink / raw)
  To: Benjamin Gaignard, ezequiel, p.zabel, mchehab, m.szyprowski
  Cc: linux-media, linux-rockchip, linux-kernel, kernel

Le jeudi 13 avril 2023 à 09:03 +0200, Benjamin Gaignard a écrit :
> ctx->vpu_dst_fmt is no more initialized before calling hantro_try_fmt()
> so assigne it to vpu_fmt led to crash the kernel.
> Like for decoder case use 'fmt' as vpu format for encoder and clean up
> the code.
> 
> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
> Tested-by: Marek Szyprowski <m.szyprowski@samsung.com>
> Fixes: db6f68b51e5c ("media: verisilicon: Do not set context src/dst formats in reset functions")

Reviewed-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>

> ---
>  drivers/media/platform/verisilicon/hantro_v4l2.c | 8 +++-----
>  1 file changed, 3 insertions(+), 5 deletions(-)
> 
> diff --git a/drivers/media/platform/verisilicon/hantro_v4l2.c b/drivers/media/platform/verisilicon/hantro_v4l2.c
> index 8f1414085f47..51f528f36241 100644
> --- a/drivers/media/platform/verisilicon/hantro_v4l2.c
> +++ b/drivers/media/platform/verisilicon/hantro_v4l2.c
> @@ -293,13 +293,11 @@ static int hantro_try_fmt(const struct hantro_ctx *ctx,
>  		pix_mp->pixelformat = fmt->fourcc;
>  	}
>  
> +	vpu_fmt = fmt;
> +
>  	if (coded) {
>  		pix_mp->num_planes = 1;
> -		vpu_fmt = fmt;
> -	} else if (ctx->is_encoder) {
> -		vpu_fmt = ctx->vpu_dst_fmt;
> -	} else {
> -		vpu_fmt = fmt;
> +	} else if (!ctx->is_encoder) {
>  		/*
>  		 * Width/height on the CAPTURE end of a decoder are ignored and
>  		 * replaced by the OUTPUT ones.
> -- 
> 2.34.1
> 
> 


^ permalink raw reply	[flat|nested] 4+ messages in thread
* Re: [PATCH] media: verisilicon: Fix crash when probing encoder
@ 2023-04-13 19:45   ` Nicolas Dufresne
  0 siblings, 0 replies; 4+ messages in thread
From: Nicolas Dufresne @ 2023-04-13 19:45 UTC (permalink / raw)
  To: Benjamin Gaignard, ezequiel, p.zabel, mchehab, m.szyprowski
  Cc: linux-media, linux-rockchip, linux-kernel, kernel

Le jeudi 13 avril 2023 à 09:03 +0200, Benjamin Gaignard a écrit :
> ctx->vpu_dst_fmt is no more initialized before calling hantro_try_fmt()
> so assigne it to vpu_fmt led to crash the kernel.
> Like for decoder case use 'fmt' as vpu format for encoder and clean up
> the code.
> 
> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
> Tested-by: Marek Szyprowski <m.szyprowski@samsung.com>
> Fixes: db6f68b51e5c ("media: verisilicon: Do not set context src/dst formats in reset functions")

Reviewed-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>

> ---
>  drivers/media/platform/verisilicon/hantro_v4l2.c | 8 +++-----
>  1 file changed, 3 insertions(+), 5 deletions(-)
> 
> diff --git a/drivers/media/platform/verisilicon/hantro_v4l2.c b/drivers/media/platform/verisilicon/hantro_v4l2.c
> index 8f1414085f47..51f528f36241 100644
> --- a/drivers/media/platform/verisilicon/hantro_v4l2.c
> +++ b/drivers/media/platform/verisilicon/hantro_v4l2.c
> @@ -293,13 +293,11 @@ static int hantro_try_fmt(const struct hantro_ctx *ctx,
>  		pix_mp->pixelformat = fmt->fourcc;
>  	}
>  
> +	vpu_fmt = fmt;
> +
>  	if (coded) {
>  		pix_mp->num_planes = 1;
> -		vpu_fmt = fmt;
> -	} else if (ctx->is_encoder) {
> -		vpu_fmt = ctx->vpu_dst_fmt;
> -	} else {
> -		vpu_fmt = fmt;
> +	} else if (!ctx->is_encoder) {
>  		/*
>  		 * Width/height on the CAPTURE end of a decoder are ignored and
>  		 * replaced by the OUTPUT ones.
> -- 
> 2.34.1
> 
> 


_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip

^ permalink raw reply	[flat|nested] 4+ messages in thread
end of thread, other threads:[~2023-04-13 19:45 UTC | newest]

Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-04-13  7:03 [PATCH] media: verisilicon: Fix crash when probing encoder Benjamin Gaignard
2023-04-13  7:03 ` Benjamin Gaignard
2023-04-13 19:45 ` Nicolas Dufresne
2023-04-13 19:45   ` Nicolas Dufresne

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.