All of lore.kernel.org
 help / color / mirror / Atom feed
From: Dan Carpenter <dan.carpenter@oracle.com>
To: Gustavo Padovan <gustavo@padovan.org>
Cc: David Airlie <airlied@linux.ie>,
	kernel-janitors@vger.kernel.org, dri-devel@lists.freedesktop.org
Subject: [PATCH v2] drm/dumb-buffers: Integer overflow in drm_mode_create_ioctl()
Date: Wed, 16 May 2018 14:00:26 +0000	[thread overview]
Message-ID: <20180516140026.GA19340@mwanda> (raw)
In-Reply-To: <152585393757.3513.738158010667924495@mail.alporthouse.com>

There is a comment here which says that DIV_ROUND_UP() and that's where
the problem comes from.  Say you pick:

	args->bpp = UINT_MAX - 7;
	args->width = 4;
	args->height = 1;

The integer overflow in DIV_ROUND_UP() means "cpp" is UINT_MAX / 8 and
because of how we picked args->width that means cpp < UINT_MAX / 4.

I've fixed it by preventing the integer overflow in DIV_ROUND_UP().  I
removed the check for !cpp because it's not possible after this change.
I also changed all the 0xffffffffU references to U32_MAX.

Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com>
---
v2:  additional cleanups

diff --git a/drivers/gpu/drm/drm_dumb_buffers.c b/drivers/gpu/drm/drm_dumb_buffers.c
index 39ac15ce4702..9e2ae02f31e0 100644
--- a/drivers/gpu/drm/drm_dumb_buffers.c
+++ b/drivers/gpu/drm/drm_dumb_buffers.c
@@ -65,12 +65,13 @@ int drm_mode_create_dumb_ioctl(struct drm_device *dev,
 		return -EINVAL;
 
 	/* overflow checks for 32bit size calculations */
-	/* NOTE: DIV_ROUND_UP() can overflow */
+	if (args->bpp > U32_MAX - 8)
+		return -EINVAL;
 	cpp = DIV_ROUND_UP(args->bpp, 8);
-	if (!cpp || cpp > 0xffffffffU / args->width)
+	if (cpp > U32_MAX / args->width)
 		return -EINVAL;
 	stride = cpp * args->width;
-	if (args->height > 0xffffffffU / stride)
+	if (args->height > U32_MAX / stride)
 		return -EINVAL;
 
 	/* test for wrap-around */

WARNING: multiple messages have this Message-ID (diff)
From: Dan Carpenter <dan.carpenter@oracle.com>
To: Gustavo Padovan <gustavo@padovan.org>
Cc: David Airlie <airlied@linux.ie>,
	kernel-janitors@vger.kernel.org, dri-devel@lists.freedesktop.org
Subject: [PATCH v2] drm/dumb-buffers: Integer overflow in drm_mode_create_ioctl()
Date: Wed, 16 May 2018 17:00:26 +0300	[thread overview]
Message-ID: <20180516140026.GA19340@mwanda> (raw)
In-Reply-To: <152585393757.3513.738158010667924495@mail.alporthouse.com>

There is a comment here which says that DIV_ROUND_UP() and that's where
the problem comes from.  Say you pick:

	args->bpp = UINT_MAX - 7;
	args->width = 4;
	args->height = 1;

The integer overflow in DIV_ROUND_UP() means "cpp" is UINT_MAX / 8 and
because of how we picked args->width that means cpp < UINT_MAX / 4.

I've fixed it by preventing the integer overflow in DIV_ROUND_UP().  I
removed the check for !cpp because it's not possible after this change.
I also changed all the 0xffffffffU references to U32_MAX.

Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com>
---
v2:  additional cleanups

diff --git a/drivers/gpu/drm/drm_dumb_buffers.c b/drivers/gpu/drm/drm_dumb_buffers.c
index 39ac15ce4702..9e2ae02f31e0 100644
--- a/drivers/gpu/drm/drm_dumb_buffers.c
+++ b/drivers/gpu/drm/drm_dumb_buffers.c
@@ -65,12 +65,13 @@ int drm_mode_create_dumb_ioctl(struct drm_device *dev,
 		return -EINVAL;
 
 	/* overflow checks for 32bit size calculations */
-	/* NOTE: DIV_ROUND_UP() can overflow */
+	if (args->bpp > U32_MAX - 8)
+		return -EINVAL;
 	cpp = DIV_ROUND_UP(args->bpp, 8);
-	if (!cpp || cpp > 0xffffffffU / args->width)
+	if (cpp > U32_MAX / args->width)
 		return -EINVAL;
 	stride = cpp * args->width;
-	if (args->height > 0xffffffffU / stride)
+	if (args->height > U32_MAX / stride)
 		return -EINVAL;
 
 	/* test for wrap-around */
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

  parent reply	other threads:[~2018-05-16 14:00 UTC|newest]

Thread overview: 24+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-05-09  7:22 [PATCH] drm/dumb-buffers: Integer overflow in drm_mode_create_ioctl() Dan Carpenter
2018-05-09  7:22 ` Dan Carpenter
2018-05-09  8:12 ` [PATCH v2] " Dan Carpenter
2018-05-09  8:12   ` Dan Carpenter
2018-05-09  8:18   ` Chris Wilson
2018-05-09  8:18     ` Chris Wilson
2018-05-09 11:59     ` Dan Carpenter
2018-05-09 11:59       ` Dan Carpenter
2018-05-16 14:00     ` Dan Carpenter [this message]
2018-05-16 14:00       ` Dan Carpenter
2018-05-16 14:26       ` Chris Wilson
2018-05-16 14:26         ` Chris Wilson
2018-05-16 14:52         ` Dan Carpenter
2018-05-16 14:52           ` Dan Carpenter
2018-05-16 14:56           ` Chris Wilson
2018-05-16 15:15             ` Dan Carpenter
2018-05-16 15:15               ` Dan Carpenter
2018-05-16 15:25               ` Chris Wilson
2018-05-16 15:41               ` Dan Carpenter
2018-05-16 15:41                 ` Dan Carpenter
2018-05-16 15:57       ` Daniel Vetter
2018-05-16 15:57         ` Daniel Vetter
2018-05-09  8:23 ` [PATCH] " Daniel Vetter
2018-05-09  8:23   ` Daniel Vetter

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20180516140026.GA19340@mwanda \
    --to=dan.carpenter@oracle.com \
    --cc=airlied@linux.ie \
    --cc=dri-devel@lists.freedesktop.org \
    --cc=gustavo@padovan.org \
    --cc=kernel-janitors@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.