From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752882AbaHXRSN (ORCPT ); Sun, 24 Aug 2014 13:18:13 -0400 Received: from mail-ig0-f182.google.com ([209.85.213.182]:54292 "EHLO mail-ig0-f182.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752479AbaHXRSM (ORCPT ); Sun, 24 Aug 2014 13:18:12 -0400 MIME-Version: 1.0 In-Reply-To: References:

Date: Sun, 24 Aug 2014 19:18:11 +0200 Message-ID: Subject: Re: drm_mode_create_dumb_ioctl: divide error From: David Herrmann To: Tommi Rantala Cc: "dri-devel@lists.freedesktop.org" , trinity@vger.kernel.org, LKML , David Airlie , Dave Jones Content-Type: text/plain; charset=UTF-8 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hi On Sun, Aug 24, 2014 at 7:12 PM, Tommi Rantala wrote: > (gdb) info locals > cpp = 0 > stride = 0 > size = > > (gdb) print /x *(struct drm_mode_create_dumb *)data > $13 = { > height = 0xffffffff, > width = 0xffffffff, > bpp = 0xffffffff, > flags = 0xffffffff, > handle = 0xffffffff, > pitch = 0xffffffff, > size = 0xffffffffffffffff > } Thanks a lot for digging into this. Looks like DIV_ROUND_UP() can overflow... *check* Yes, it's defined as: #define DIV_ROUND_UP(n,d) (((n) + (d) - 1) / (d)) Kinda unexpected that a function called "DIV" can overflow, but makes sense here. Obvious fix is to test "cpp" for zero. I will send it to dri-devel. Thanks! David From mboxrd@z Thu Jan 1 00:00:00 1970 From: David Herrmann Subject: Re: drm_mode_create_dumb_ioctl: divide error Date: Sun, 24 Aug 2014 19:18:11 +0200 Message-ID: References:

Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=qF67AGZhhULTaefq0j/vwGab/fJ1Ki3uSfWMl1RqQB4=; b=zqECdDyxDWcGCFE4i2SkaZHt//6Z2MMB2YEP/AAaQ2BICHJ3CENBhqnoO16qabyJTl B3ML8CHjUzFyxEI9/JMTgvZyDpkWQqm0erb6ZcJygwO1Q3/RdrMQZi+S4y6WPurQMzaB DYjF8BS19m5BA6fQUs4E8Irp5TwFrfZWB2GpP5RqR7nO7dGmRxDlSyAP0HMCOw83KOTO 4FuvDByHSc9+6kv5QPElASUZuceP4n0QUlQpePu90kkZuoYaRGzx5Cx+IigPJ1vJ9CXu eOP2KQd++xl4AXC9QOwLWKIr8GHGHe7jvgPCZ6jJ1vrbt6BU/DB+APvZIZuswir9Bqpa xK0g== In-Reply-To: List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" To: Tommi Rantala Cc: Dave Jones , LKML , "dri-devel@lists.freedesktop.org" , trinity@vger.kernel.org Hi On Sun, Aug 24, 2014 at 7:12 PM, Tommi Rantala wrote: > (gdb) info locals > cpp = 0 > stride = 0 > size = > > (gdb) print /x *(struct drm_mode_create_dumb *)data > $13 = { > height = 0xffffffff, > width = 0xffffffff, > bpp = 0xffffffff, > flags = 0xffffffff, > handle = 0xffffffff, > pitch = 0xffffffff, > size = 0xffffffffffffffff > } Thanks a lot for digging into this. Looks like DIV_ROUND_UP() can overflow... *check* Yes, it's defined as: #define DIV_ROUND_UP(n,d) (((n) + (d) - 1) / (d)) Kinda unexpected that a function called "DIV" can overflow, but makes sense here. Obvious fix is to test "cpp" for zero. I will send it to dri-devel. Thanks! David