From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752960AbdLHKOf (ORCPT ); Fri, 8 Dec 2017 05:14:35 -0500 Received: from bombadil.infradead.org ([65.50.211.133]:56882 "EHLO bombadil.infradead.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752312AbdLHKOa (ORCPT ); Fri, 8 Dec 2017 05:14:30 -0500 Date: Fri, 8 Dec 2017 11:14:16 +0100 From: Peter Zijlstra To: LKML , DRI Development , Intel Graphics Development , Tvrtko Ursulin , Marta Lofstedt , Byungchul Park , Ingo Molnar , Tejun Heo , Kees Cook , Thomas Gleixner , Shaohua Li , Andrew Morton , Jens Axboe , Greg Kroah-Hartman , Jonathan Corbet , Oleg Nesterov , Daniel Vetter Subject: Re: [Intel-gfx] [PATCH] kthread: finer-grained lockdep/cross-release completion Message-ID: <20171208101416.hu4uwan3jgcbb6dw@hirez.programming.kicks-ass.net> References: <20171207100849.407-1-daniel.vetter@ffwll.ch> <20171207122255.zi5ishny24k66ik7@hirez.programming.kicks-ass.net> <20171207145828.z52kjbrrlcl75m7m@phenom.ffwll.local> <20171207195709.cxcil57boc6czdot@hirez.programming.kicks-ass.net> <20171207205657.wkt6xj3e2opegqeq@phenom.ffwll.local> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20171207205657.wkt6xj3e2opegqeq@phenom.ffwll.local> User-Agent: NeoMutt/20170609 (1.8.3) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Thu, Dec 07, 2017 at 09:56:57PM +0100, Daniel Vetter wrote: > On Thu, Dec 07, 2017 at 08:57:09PM +0100, Peter Zijlstra wrote: > > Is what it says I suppose. Now I don't know enough about that i915 code > > to say if that breadcrumbs_signal thread can ever trigger a fault or > > not. I got properly lost in that dma_fence callback maze. > > > > You're saying not? > > Our own kthread, no. At least a tons of run on our CI with the kthread > patch applied shut up lockdep splats for good. And since we have all the > i915 kthreads still with the same lockdep_map even with the patch applied, > since they are all created in the same function, I think that's pretty > solid evidence. > > [There's also really no reasonable reason for it to fault, but I trust > automated tools more to check this stuff than my own brain. The test suite > we're running is fairly nasty and does all kinds of corner case > thrashing. Note that the dma_fence callbacks can be provideded by any > other driver (think multi-gpu desktops and stuff), but the contract is > that they must be able to handle hardirq context. Faulting's definitely > not on the table.] OK, good. > The problem lockdep seems to complain about is that some random other > kthread could fault, end up in the i915 fault handler, and get stuck until > i915_reset_device is done doing what it needs to do. But as long as that > kthread is in turn not providing a service that i915_reset_device needs, I > don't see how that can deadlock. And if we have that case (there was > definitely plenty of that stuff that cross-release uncovered in our code, > we had to shuffle a bunch of allocations and things out from under > dev->struct_mutex), then there should be another lock or completion that > closes the loop again. Indeed so. > > (also, that comment near need_resched() doesn't make sense to me) > > I assume you mean the one in intel_breadcrumbs_signaler(). The hw design > is somewhat screwed up and depends upon ridiculously low interrupt > servicing time. We get there by essentially implementing something like > netdev polled mode, from irq context. Like net polling if the scheduler > gets pissed at us we stop and dump it all into a kthread. From a latency > and ops/sec pov a gpu is pretty close to networking sometimes. > > [Note: I just have a rough idea what the code is supposed to do, I didn't > write/review/debug that one.] The thing is though; that calling schedule() from an RT thread doesn't help anything if it goes running instantly again. And looking more; that uses the waitqueue code 'creatively' it doesn't actually have a condition to wait on, so wtf is it doing with a waitqueue?