Re: [syzbot] BUG: unable to handle kernel paging request in vga16fb_fillrect

From: Dan Carpenter <dan.carpenter@oracle.com>
To: Tetsuo Handa <penguin-kernel@i-love.sakura.ne.jp>
Cc: Geert Uytterhoeven <geert@linux-m68k.org>,
	Randy Dunlap <rdunlap@infradead.org>,
	syzbot <syzbot+04168c8063cfdde1db5e@syzkaller.appspotmail.com>,
	Andrew Morton <akpm@linux-foundation.org>,
	Bartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com>,
	Colin King <colin.king@canonical.com>,
	DRI Development <dri-devel@lists.freedesktop.org>,
	Linux Fbdev development list <linux-fbdev@vger.kernel.org>,
	Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
	Masahiro Yamada <masahiroy@kernel.org>,
	syzkaller-bugs@googlegroups.com
Subject: Re: [syzbot] BUG: unable to handle kernel paging request in vga16fb_fillrect
Date: Mon, 30 Aug 2021 17:29:32 +0300	[thread overview]
Message-ID: <20210830142932.GJ12231@kadam> (raw)
In-Reply-To: <03d0f549-9731-8b06-1393-60d4bef27884@i-love.sakura.ne.jp>

On Mon, Aug 30, 2021 at 11:25:51PM +0900, Tetsuo Handa wrote:
> On 2021/08/30 22:47, Dan Carpenter wrote:
> > On Mon, Aug 30, 2021 at 10:37:31PM +0900, Tetsuo Handa wrote:
> >> On 2021/08/30 22:00, Dan Carpenter wrote:
> >>>>> diff --git a/drivers/video/fbdev/vga16fb.c b/drivers/video/fbdev/vga16fb.c
> >>>>> index e2757ff1c23d..e483a3f5fd47 100644
> >>>>> --- a/drivers/video/fbdev/vga16fb.c
> >>>>> +++ b/drivers/video/fbdev/vga16fb.c
> >>>>> @@ -403,7 +403,7 @@ static int vga16fb_check_var(struct fb_var_screeninfo *var,
> >>>>>
> >>>>>         if (yres > vyres)
> >>>>>                 vyres = yres;
> >>>>> -       if (vxres * vyres > maxmem) {
> >>>>> +       if ((u64) vxres * vyres > (u64) maxmem) {
> >>>>
> >>>> Mindlessly changing the sizes is not the solution.
> >>>> Please use e.g. the array_size() helper from <linux/overflow.h>
> >>>> instead.
> >>>
> >>> On a 64bit system the array_size() macro is going to do the exact same
> >>> casts?  But I do think this code would be easier to understand if the
> >>> integer overflow check were pull out separately and done first:
> >>>
> >>> 	if (array_size(vxres, vyres) >= UINT_MAX)
> >>> 		return -EINVAL;
> >>
> >> This is wrong. array_size() returns ULONG_MAX on 64bits upon overflow and
> >> returns UINT_MAX on 32bits upon overflow. However, UINT_MAX is a valid
> >> value without overflow (e.g. vxres == UINT_MAX / 15 && vyres == 15).
> > 
> > Huh...  I just assumed we didn't allow resolutions that high.
> 
> Of course, we don't allow resolutions that high. ;-)
> 
> Since I don't know possible max resolutions, I chose UINT_MAX + 1 as a common
> limit for returning -EINVAL. Unless overflow happens, vga16fb_check_var() will
> return -ENOMEM on such high resolutions.
> 
> > 
> >> Comparing like "> (u64) UINT_MAX" is to detect only overflow.
> >>
> > 
> > Of course, that doesn't work on 32 bit systems.  Also the cast isn't
> > required because of type promotion.
> 
> Indeed, "> UINT_MAX" seems to work on both 32bits and 64bits.

Sorry, for the confusion.  I'm talking about array_size() which is
size_t.  Your approach using unsigned long long works.

regards,
dan carpenter