dri-devel Archive on lore.kernel.org
 help / color / Atom feed
From: bugzilla-daemon@bugzilla.kernel.org
To: dri-devel@lists.freedesktop.org
Subject: [Bug 207383] [Regression] 5.7 amdgpu/polaris11 gpf: amdgpu_atomic_commit_tail
Date: Tue, 21 Jul 2020 20:33:54 +0000
Message-ID: <bug-207383-2300-mFL0I6lM2i@https.bugzilla.kernel.org/> (raw)
In-Reply-To: <bug-207383-2300@https.bugzilla.kernel.org/>

https://bugzilla.kernel.org/show_bug.cgi?id=207383

--- Comment #76 from mnrzk@protonmail.com ---
(In reply to Kees Cook from comment #75)
> Hi!
> 
> First, let me say sorry for all the work my patch has caused! It seems like
> it might be tickling another (previously dormant) bug in the gpu driver.
> 
> 
> (In reply to mnrzk from comment #30)
> > I've been looking at this bug for a while now and I'll try to share what
> > I've found about it.
> > 
> > In some conditions, when amdgpu_dm_atomic_commit_tail calls
> > dm_atomic_get_new_state, dm_atomic_get_new_state returns a struct
> > dm_atomic_state* with an garbage context pointer.
> > 
> > I've also found that this bug exclusively occurs when commit_work is on the
> > workqueue. After forcing drm_atomic_helper_commit to run all of the commits
> > without adding to the workqueue and running the OS, the issue seems to have
> > disappeared. The system was stable for at least 1.5 hours before I manually
> > shut it down (meanwhile it has usually crashed within 30-45 minutes).
> > 
> > Perhaps there's some sort of race condition occurring after commit_work is
> > queued?
> 
> If it helps to explain what's happening in 3202fa62f, the kernel memory
> allocator is moving it's free pointer from offset 0 to the middle of the
> object. That means that when the memory is freed, it writes 8 bytes to join
> the newly freed memory into the allocator's freelist. That always happened,
> but after 3202fa62f it began writing it in the middle, not offset 0. If the
> work queue is trying to use freed memory, and before it didn't notice the
> first 8 bytes getting written, now it appears to notice the overwrite... but
> that still means something is freeing memory before it should.
> 
> Finding that might be a real trick. :( However, if you've suffered through
> all those bisections, I wonder if you can try one other thing, which is to
> compile the kernel with KASAN:
> 
> CONFIG_KASAN=y
> CONFIG_KASAN_GENERIC=y
> CONFIG_KASAN_OUTLINE=y
> CONFIG_KASAN_STACK=y
> CONFIG_KASAN_VMALLOC=y
> 
> This will make things _slow_, which might mean the use-after-free race may
> never trigger. *However* it's possible that it'll catch a bad behavior
> before it even needs to get hit in a race that triggers the behavior you're
> seeing. (And note that swapping CONFIG_KASAN_OUTLINE=y for
> CONFIG_KASAN_INLINE=y might speed things up, but the kernel image gets
> bigger).
> 
> I'm going to try to read the work queue code for the driver and see if
> anything obvious stands out...

Actually this makes perfect sense, struct dm_atomic_state* dm_state has
two components, base (a struct containing a struct drm_atomic_state*) and
context (a struct dc_state*). Reading through the code of
amdgpu_dm_atomic_commit_tail, I see that dm_state->base is never used.

If my understanding is correct, base would have previously been filled with
the freelist pointer (since it's the first 8 bytes). Now since the freelist
pointer is being put in the middle (rounded to the nearest sizeof(void*),
 or 8 bytes), it's being put in the last 8 bytes of *dm_state
(or dm_state->context).

I'll place a void* for padding in the middle of struct dm_atomic_state* and
if my hypothesis is correct, the padding will be filled with garbage data
instead of context and the bug should be fixed. Of course, there would
still be a use-after-free bug in the code which may cause other issues in
the future so I wouldn't really consider it a solution.

Regarding KASAN, I've tried compiling the kernel with KASAN enabled and
from my experience, the bug did not trigger after actively using the system
for 3 hours and leaving it on for 12 hours. This was almost a month ago
though so maybe I'll try again with different KASAN options (i.e.
CONFIG_KASAN_INLINE=y). If anyone has any more tips on getting KASAN to run
faster, I'll be glad to hear them.

-- 
You are receiving this mail because:
You are watching the assignee of the bug.
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

  parent reply index

Thread overview: 118+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-04-21  9:51 [Bug 207383] New: [Regression] 5.7-rc: " bugzilla-daemon
2020-04-21  9:57 ` [Bug 207383] " bugzilla-daemon
2020-04-21 10:04 ` bugzilla-daemon
2020-04-23  4:59 ` bugzilla-daemon
2020-04-27 19:24 ` bugzilla-daemon
2020-04-27 19:42 ` bugzilla-daemon
2020-04-27 19:43 ` bugzilla-daemon
2020-05-01  8:20 ` bugzilla-daemon
2020-05-01  8:28 ` bugzilla-daemon
2020-05-02 16:03 ` bugzilla-daemon
2020-05-03 15:10 ` bugzilla-daemon
2020-05-05  4:23 ` bugzilla-daemon
2020-05-06 17:46 ` bugzilla-daemon
2020-05-06 22:06 ` bugzilla-daemon
2020-06-03  0:04 ` [Bug 207383] [Regression] 5.7 " bugzilla-daemon
2020-06-21  7:01 ` bugzilla-daemon
2020-06-22 15:20 ` bugzilla-daemon
2020-06-22 17:44 ` bugzilla-daemon
2020-06-22 17:57 ` bugzilla-daemon
2020-06-22 19:36 ` bugzilla-daemon
2020-06-22 20:00 ` bugzilla-daemon
2020-06-23 15:36 ` bugzilla-daemon
2020-06-23 23:41 ` bugzilla-daemon
2020-06-24  8:55 ` bugzilla-daemon
2020-06-27  4:37 ` bugzilla-daemon
2020-06-27  4:38 ` bugzilla-daemon
2020-06-27  5:16 ` bugzilla-daemon
2020-06-27  6:08 ` bugzilla-daemon
2020-06-27  7:07 ` bugzilla-daemon
2020-06-27 22:26 ` bugzilla-daemon
2020-06-28  1:12 ` bugzilla-daemon
2020-06-28 10:48 ` bugzilla-daemon
2020-06-28 15:30 ` bugzilla-daemon
2020-06-29  7:39 ` bugzilla-daemon
2020-06-29 22:09 ` bugzilla-daemon
2020-07-01 19:08 ` bugzilla-daemon
2020-07-04 19:57 ` bugzilla-daemon
2020-07-04 20:13 ` bugzilla-daemon
2020-07-05 16:58 ` bugzilla-daemon
2020-07-05 22:08 ` bugzilla-daemon
2020-07-06 16:24 ` bugzilla-daemon
2020-07-06 23:57 ` bugzilla-daemon
2020-07-07  0:37 ` bugzilla-daemon
2020-07-07  3:01 ` bugzilla-daemon
2020-07-07 11:01 ` bugzilla-daemon
2020-07-07 12:43 ` bugzilla-daemon
2020-07-07 15:27 ` bugzilla-daemon
2020-07-07 19:05 ` bugzilla-daemon
2020-07-08  0:25 ` bugzilla-daemon
2020-07-08  1:25 ` bugzilla-daemon
2020-07-08 20:16 ` bugzilla-daemon
2020-07-08 20:17 ` bugzilla-daemon
2020-07-09  7:45 ` bugzilla-daemon
2020-07-10  7:23 ` bugzilla-daemon
2020-07-10  7:36 ` bugzilla-daemon
2020-07-10  8:10 ` bugzilla-daemon
2020-07-10 10:55 ` bugzilla-daemon
2020-07-10 11:25 ` bugzilla-daemon
2020-07-10 14:31 ` bugzilla-daemon
2020-07-12  5:20 ` bugzilla-daemon
2020-07-12  5:47 ` bugzilla-daemon
2020-07-12  7:47 ` bugzilla-daemon
2020-07-14 23:36 ` bugzilla-daemon
2020-07-15 16:49 ` bugzilla-daemon
2020-07-15 17:12 ` bugzilla-daemon
2020-07-16  2:12 ` bugzilla-daemon
2020-07-16  6:37 ` bugzilla-daemon
2020-07-16  9:35 ` bugzilla-daemon
2020-07-16 10:24 ` bugzilla-daemon
2020-07-16 10:30 ` bugzilla-daemon
2020-07-16 10:32 ` bugzilla-daemon
2020-07-17 12:39 ` bugzilla-daemon
2020-07-20  2:20 ` bugzilla-daemon
2020-07-21 16:40 ` bugzilla-daemon
2020-07-21 16:57 ` bugzilla-daemon
2020-07-21 19:32 ` bugzilla-daemon
2020-07-21 20:33 ` bugzilla-daemon [this message]
2020-07-21 20:49 ` bugzilla-daemon
2020-07-21 20:56 ` bugzilla-daemon
2020-07-21 21:16 ` bugzilla-daemon
2020-07-22  2:03 ` bugzilla-daemon
2020-07-22  2:05 ` bugzilla-daemon
2020-07-22  3:37 ` bugzilla-daemon
2020-07-22  7:27 ` bugzilla-daemon
2020-07-22 13:04 ` bugzilla-daemon
2020-07-23  0:48 ` bugzilla-daemon
2020-07-23  5:46 ` bugzilla-daemon
2020-07-23 21:30 ` bugzilla-daemon
2020-07-23 21:34 ` bugzilla-daemon
2020-07-24  7:18 ` bugzilla-daemon
2020-07-24  7:24 ` bugzilla-daemon
2020-07-24 19:08 ` bugzilla-daemon
2020-07-24 21:00 ` bugzilla-daemon
2020-07-25  2:38 ` bugzilla-daemon
2020-07-26  6:47 ` bugzilla-daemon
2020-07-26 18:40 ` bugzilla-daemon
2020-07-26 19:55 ` bugzilla-daemon
2020-07-26 22:52 ` bugzilla-daemon
2020-07-26 23:30 ` bugzilla-daemon
2020-07-26 23:52 ` bugzilla-daemon
2020-07-27  6:11 ` bugzilla-daemon
2020-07-27 16:55 ` bugzilla-daemon
2020-07-28  2:29 ` bugzilla-daemon
2020-07-28  3:21 ` bugzilla-daemon
2020-07-28  3:39 ` bugzilla-daemon
2020-07-28  7:14 ` bugzilla-daemon
2020-07-29  2:33 ` bugzilla-daemon
2020-07-29  6:41 ` bugzilla-daemon
2020-07-29 16:02 ` bugzilla-daemon
2020-07-29 16:37 ` bugzilla-daemon
2020-07-29 16:45 ` bugzilla-daemon
2020-07-29 20:32 ` bugzilla-daemon
2020-07-31 16:38 ` bugzilla-daemon
2020-08-02  1:40 ` bugzilla-daemon
2020-08-02 13:06 ` bugzilla-daemon
2020-08-03 13:51 ` bugzilla-daemon
2020-08-05 16:10 ` bugzilla-daemon
2020-08-17  5:45 ` bugzilla-daemon

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=bug-207383-2300-mFL0I6lM2i@https.bugzilla.kernel.org/ \
    --to=bugzilla-daemon@bugzilla.kernel.org \
    --cc=dri-devel@lists.freedesktop.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

dri-devel Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/dri-devel/0 dri-devel/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 dri-devel dri-devel/ https://lore.kernel.org/dri-devel \
		dri-devel@lists.freedesktop.org
	public-inbox-index dri-devel

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.freedesktop.lists.dri-devel


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git