[PATCH] video: fbdev: i740fb: Check the argument of i740_calc_vclk()

[PATCH] video: fbdev: i740fb: Check the argument of i740_calc_vclk()

[Zheyu Ma]

Since the user can control the arguments of the ioctl() from the user
space, under special arguments that may result in a divide-by-zero bug.

If the user provides an improper 'pixclock' value that makes the argumet
of i740_calc_vclk() less than 'I740_RFREQ_FIX', it will cause a
divide-by-zero bug in:
    drivers/video/fbdev/i740fb.c:353 p_best = min(15, ilog2(I740_MAX_VCO_FREQ / (freq / I740_RFREQ_FIX)));

The following log can reveal it:

divide error: 0000 [#1] PREEMPT SMP KASAN PTI
RIP: 0010:i740_calc_vclk drivers/video/fbdev/i740fb.c:353 [inline]
RIP: 0010:i740fb_decode_var drivers/video/fbdev/i740fb.c:646 [inline]
RIP: 0010:i740fb_set_par+0x163f/0x3b70 drivers/video/fbdev/i740fb.c:742
Call Trace:
 fb_set_var+0x604/0xeb0 drivers/video/fbdev/core/fbmem.c:1034
 do_fb_ioctl+0x234/0x670 drivers/video/fbdev/core/fbmem.c:1110
 fb_ioctl+0xdd/0x130 drivers/video/fbdev/core/fbmem.c:1189

Fix this by checking the argument of i740_calc_vclk() first.

Signed-off-by: Zheyu Ma <zheyuma97@gmail.com>
---
 drivers/video/fbdev/i740fb.c | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/drivers/video/fbdev/i740fb.c b/drivers/video/fbdev/i740fb.c
index 09dd85553d4f..7f09a0daaaa2 100644
--- a/drivers/video/fbdev/i740fb.c
+++ b/drivers/video/fbdev/i740fb.c
@@ -400,7 +400,7 @@ static int i740fb_decode_var(const struct fb_var_screeninfo *var,
 	u32 xres, right, hslen, left, xtotal;
 	u32 yres, lower, vslen, upper, ytotal;
 	u32 vxres, xoffset, vyres, yoffset;
-	u32 bpp, base, dacspeed24, mem;
+	u32 bpp, base, dacspeed24, mem, freq;
 	u8 r7;
 	int i;
 
@@ -643,7 +643,12 @@ static int i740fb_decode_var(const struct fb_var_screeninfo *var,
 	par->atc[VGA_ATC_OVERSCAN] = 0;
 
 	/* Calculate VCLK that most closely matches the requested dot clock */
-	i740_calc_vclk((((u32)1e9) / var->pixclock) * (u32)(1e3), par);
+	freq = (((u32)1e9) / var->pixclock) * (u32)(1e3);
+	if (freq < I740_RFREQ_FIX) {
+		fb_dbg(info, "invalid pixclock\n");
+		freq = I740_RFREQ_FIX;
+	}
+	i740_calc_vclk(freq, par);
 
 	/* Since we program the clocks ourselves, always use VCLK2. */
 	par->misc |= 0x0C;
-- 
2.25.1

Re: [PATCH] video: fbdev: i740fb: Check the argument of i740_calc_vclk()

[Helge Deller]

On 8/3/22 11:24, Zheyu Ma wrote:
> Since the user can control the arguments of the ioctl() from the user
> space, under special arguments that may result in a divide-by-zero bug.
>
> If the user provides an improper 'pixclock' value that makes the argumet
> of i740_calc_vclk() less than 'I740_RFREQ_FIX', it will cause a
> divide-by-zero bug in:
>     drivers/video/fbdev/i740fb.c:353 p_best = min(15, ilog2(I740_MAX_VCO_FREQ / (freq / I740_RFREQ_FIX)));
>
> The following log can reveal it:
>
> divide error: 0000 [#1] PREEMPT SMP KASAN PTI
> RIP: 0010:i740_calc_vclk drivers/video/fbdev/i740fb.c:353 [inline]
> RIP: 0010:i740fb_decode_var drivers/video/fbdev/i740fb.c:646 [inline]
> RIP: 0010:i740fb_set_par+0x163f/0x3b70 drivers/video/fbdev/i740fb.c:742
> Call Trace:
>  fb_set_var+0x604/0xeb0 drivers/video/fbdev/core/fbmem.c:1034
>  do_fb_ioctl+0x234/0x670 drivers/video/fbdev/core/fbmem.c:1110
>  fb_ioctl+0xdd/0x130 drivers/video/fbdev/core/fbmem.c:1189
>
> Fix this by checking the argument of i740_calc_vclk() first.
>
> Signed-off-by: Zheyu Ma <zheyuma97@gmail.com>

applied to fbdev git tree.

Thanks!
Helge

> ---
>  drivers/video/fbdev/i740fb.c | 9 +++++++--
>  1 file changed, 7 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/video/fbdev/i740fb.c b/drivers/video/fbdev/i740fb.c
> index 09dd85553d4f..7f09a0daaaa2 100644
> --- a/drivers/video/fbdev/i740fb.c
> +++ b/drivers/video/fbdev/i740fb.c
> @@ -400,7 +400,7 @@ static int i740fb_decode_var(const struct fb_var_screeninfo *var,
>  	u32 xres, right, hslen, left, xtotal;
>  	u32 yres, lower, vslen, upper, ytotal;
>  	u32 vxres, xoffset, vyres, yoffset;
> -	u32 bpp, base, dacspeed24, mem;
> +	u32 bpp, base, dacspeed24, mem, freq;
>  	u8 r7;
>  	int i;
>
> @@ -643,7 +643,12 @@ static int i740fb_decode_var(const struct fb_var_screeninfo *var,
>  	par->atc[VGA_ATC_OVERSCAN] = 0;
>
>  	/* Calculate VCLK that most closely matches the requested dot clock */
> -	i740_calc_vclk((((u32)1e9) / var->pixclock) * (u32)(1e3), par);
> +	freq = (((u32)1e9) / var->pixclock) * (u32)(1e3);
> +	if (freq < I740_RFREQ_FIX) {
> +		fb_dbg(info, "invalid pixclock\n");
> +		freq = I740_RFREQ_FIX;
> +	}
> +	i740_calc_vclk(freq, par);
>
>  	/* Since we program the clocks ourselves, always use VCLK2. */
>  	par->misc |= 0x0C;