From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.9 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id F2677C43215 for ; Wed, 20 Nov 2019 21:36:47 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id B7BA820C01 for ; Wed, 20 Nov 2019 21:36:47 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="WnYCpyrj" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726658AbfKTVgq (ORCPT ); Wed, 20 Nov 2019 16:36:46 -0500 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:42320 "EHLO us-smtp-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1725842AbfKTVgq (ORCPT ); Wed, 20 Nov 2019 16:36:46 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1574285804; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0pAJwOUYZlIukE0BbqgWGnjQga+s55dRnSCojvuuOg8=; b=WnYCpyrj2aGLa/LNN6Okn1S8iOzMMNmvquuG2JtT+CBMhvUrbgVCbCegIHZzOekGTE14Yk UtoLD/wjyetuhgDnO/goi9y8zvm69lceRC3rLfP0UItYE/I3o0UC+gHRk3dsKTjzLaOHJG qmeA5oUw9GP9y67PUSr11oLETKv60YY= Received: from mail-qk1-f200.google.com (mail-qk1-f200.google.com [209.85.222.200]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-280-Iiwn5MR4MmGcYHxnp2GAPA-1; Wed, 20 Nov 2019 16:36:43 -0500 Received: by mail-qk1-f200.google.com with SMTP id a186so595217qkb.18 for ; Wed, 20 Nov 2019 13:36:43 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=BEOpo1eAW+6WweEGc5o46P0sIScU4OqKVUN6KWPXGDE=; b=bSbqep5FlS/vT2b22OUjnrvZqN8mdzmuk347oekVDFydj6/0a4F8UjRq8Mkf0XVFPD Wm8jV55n+B7rzRiTwUIoeEvVXa/ZNvAf8Oy7U+a9gf6CQPfAcvJK/bT4OIAIgJJBoBwG d1jzudMWJWu2G7b35WLE6iF5VLama+K6pIpwoQaoII2OHvX7ra16USwvplGVVnRkk66w o2E0ls4EmLaE6R9VENrYI5RpVyJwmW7WH7DAWhuROerMTi+72/WRjhpppKkXoflu+j2K 7VeLjB9z8b+C9JDaaNbcXZut5PKoMx0Oeshyd5m6k/mhbzWVLCOG3acdyMjI1fOGeDHY OuUw== X-Gm-Message-State: APjAAAXzTDAsb41Pk7ms/v26qwh+yb2OV4qSY5om+sqtz+0+ENvjPS/E XWWD37E2PmuilPS5REEnzBQXEVczsJC4PzX/YMgwwu2YSZSmzXNbL5HuHOU0J0PTIhDFyHGavXf 08rSmDMcJ1sDdwPgs03wyexfnS5fjg+XsZ9N+Towv X-Received: by 2002:a0c:baad:: with SMTP id x45mr4716317qvf.230.1574285802807; Wed, 20 Nov 2019 13:36:42 -0800 (PST) X-Google-Smtp-Source: APXvYqyekfP3DQaxdB5Z3roInhLmB3w1CVawg+LlvI/EnwQJNFYBD1Fs7D2MZDTcOhlIQfBzDzvxUKzBQtFDGEGE1aM= X-Received: by 2002:a0c:baad:: with SMTP id x45mr4716288qvf.230.1574285802452; Wed, 20 Nov 2019 13:36:42 -0800 (PST) MIME-Version: 1.0 References: <20191120112212.GA11621@lahna.fi.intel.com> <20191120115127.GD11621@lahna.fi.intel.com> <20191120120913.GE11621@lahna.fi.intel.com> <20191120151542.GH11621@lahna.fi.intel.com> <20191120155301.GL11621@lahna.fi.intel.com> <20191120162306.GM11621@lahna.fi.intel.com> In-Reply-To: <20191120162306.GM11621@lahna.fi.intel.com> From: Karol Herbst Date: Wed, 20 Nov 2019 22:36:31 +0100 Message-ID: Subject: Re: [PATCH v4] pci: prevent putting nvidia GPUs into lower device states on certain intel bridges To: Mika Westerberg Cc: "Rafael J. Wysocki" , Bjorn Helgaas , LKML , Lyude Paul , "Rafael J . Wysocki" , Linux PCI , Linux PM , dri-devel , nouveau , Dave Airlie , Mario Limonciello X-MC-Unique: Iiwn5MR4MmGcYHxnp2GAPA-1 X-Mimecast-Spam-Score: 0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org with the branch and patch applied: https://gist.githubusercontent.com/karolherbst/03c4c8141b0fa292d781badfa186= 479e/raw/5c62640afbc57d6e69ea924c338bd2836e770d02/gistfile1.txt On Wed, Nov 20, 2019 at 5:23 PM Mika Westerberg wrote: > > On Wed, Nov 20, 2019 at 05:53:07PM +0200, Mika Westerberg wrote: > > On Wed, Nov 20, 2019 at 04:37:14PM +0100, Karol Herbst wrote: > > > On Wed, Nov 20, 2019 at 4:15 PM Mika Westerberg > > > wrote: > > > > > > > > On Wed, Nov 20, 2019 at 01:11:52PM +0100, Karol Herbst wrote: > > > > > On Wed, Nov 20, 2019 at 1:09 PM Mika Westerberg > > > > > wrote: > > > > > > > > > > > > On Wed, Nov 20, 2019 at 12:58:00PM +0100, Karol Herbst wrote: > > > > > > > overall, what I really want to know is, _why_ does it work on= windows? > > > > > > > > > > > > So do I ;-) > > > > > > > > > > > > > Or what are we doing differently on Linux so that it doesn't = work? If > > > > > > > anybody has any idea on how we could dig into this and figure= it out > > > > > > > on this level, this would probably allow us to get closer to = the root > > > > > > > cause? no? > > > > > > > > > > > > Have you tried to use the acpi_rev_override parameter in your s= ystem and > > > > > > does it have any effect? > > > > > > > > > > > > Also did you try to trace the ACPI _ON/_OFF() methods? I think = that > > > > > > should hopefully reveal something. > > > > > > > > > > > > > > > > I think I did in the past and it seemed to have worked, there is = just > > > > > one big issue with this: it's a Dell specific workaround afaik, a= nd > > > > > this issue plagues not just Dell, but we've seen it on HP and Len= ovo > > > > > laptops as well, and I've heard about users having the same issue= s on > > > > > Asus and MSI laptops as well. > > > > > > > > Maybe it is not a workaround at all but instead it simply determine= s > > > > whether the system supports RTD3 or something like that (IIRC Windo= ws 8 > > > > started supporting it). Maybe Dell added check for Linux because at= that > > > > time Linux did not support it. > > > > > > > > > > the point is, it's not checking it by default, so by default you stil= l > > > run into the windows 8 codepath. > > > > Well you can add the quirk to acpi_rev_dmi_table[] so it goes to that > > path by default. There are a bunch of similar entries for Dell machines= . > > > > Of course this does not help the non-Dell users so we would still need > > to figure out the root cause. > > I think I asked you to test the PCIe delay patch and it did not help but > I wonder if it helps if we increase the delay. As an experiment could > you try Bjorn's pci/pm branch. The last two commits are for the delay. > > If you could pull that branch and apply the following patch on top and > give it a try? Then post the dmesg somewhere so we can see whether it > did the delay at all. > > diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c > index 1f319b1175da..1ad6f1372ed5 100644 > --- a/drivers/pci/pci.c > +++ b/drivers/pci/pci.c > @@ -4697,12 +4697,7 @@ void pci_bridge_wait_for_secondary_bus(struct pci_= dev *dev) > return; > } > > - /* Take d3cold_delay requirements into account */ > - delay =3D pci_bus_max_d3cold_delay(dev->subordinate); > - if (!delay) { > - up_read(&pci_bus_sem); > - return; > - } > + delay =3D 500; > > child =3D list_first_entry(&dev->subordinate->devices, struct pci= _dev, > bus_list); > @@ -4715,7 +4710,7 @@ void pci_bridge_wait_for_secondary_bus(struct pci_d= ev *dev) > * management for them (see pci_bridge_d3_possible()). > */ > if (!pci_is_pcie(dev)) { > - pci_dbg(dev, "waiting %d ms for secondary bus\n", 1000 + = delay); > + pci_info(dev, "waiting %d ms for secondary bus\n", 1000 += delay); > msleep(1000 + delay); > return; > } > @@ -4741,10 +4736,10 @@ void pci_bridge_wait_for_secondary_bus(struct pci= _dev *dev) > return; > > if (pcie_get_speed_cap(dev) <=3D PCIE_SPEED_5_0GT) { > - pci_dbg(dev, "waiting %d ms for downstream link\n", delay= ); > + pci_info(dev, "waiting %d ms for downstream link\n", dela= y); > msleep(delay); > } else { > - pci_dbg(dev, "waiting %d ms for downstream link, after ac= tivation\n", > + pci_info(dev, "waiting %d ms for downstream link, after a= ctivation\n", > delay); > if (!pcie_wait_for_link_delay(dev, true, delay)) { > /* Did not train, no need to wait any further */ > @@ -4753,7 +4748,7 @@ void pci_bridge_wait_for_secondary_bus(struct pci_d= ev *dev) > } > > if (!pci_device_is_present(child)) { > - pci_dbg(child, "waiting additional %d ms to become access= ible\n", delay); > + pci_info(child, "waiting additional %d ms to become acces= sible\n", delay); > msleep(delay); > } > } > From mboxrd@z Thu Jan 1 00:00:00 1970 From: Karol Herbst Subject: Re: [PATCH v4] pci: prevent putting nvidia GPUs into lower device states on certain intel bridges Date: Wed, 20 Nov 2019 22:36:31 +0100 Message-ID: References: <20191120112212.GA11621@lahna.fi.intel.com> <20191120115127.GD11621@lahna.fi.intel.com> <20191120120913.GE11621@lahna.fi.intel.com> <20191120151542.GH11621@lahna.fi.intel.com> <20191120155301.GL11621@lahna.fi.intel.com> <20191120162306.GM11621@lahna.fi.intel.com> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Return-path: In-Reply-To: <20191120162306.GM11621-3PARRvDOhMZrdx17CPfAsdBPR1lH4CV8@public.gmane.org> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: nouveau-bounces-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW@public.gmane.org Sender: "Nouveau" To: Mika Westerberg Cc: "Rafael J. Wysocki" , Linux PCI , Linux PM , "Rafael J . Wysocki" , LKML , dri-devel , Mario Limonciello , Bjorn Helgaas , nouveau List-Id: nouveau.vger.kernel.org d2l0aCB0aGUgYnJhbmNoIGFuZCBwYXRjaCBhcHBsaWVkOgpodHRwczovL2dpc3QuZ2l0aHVidXNl cmNvbnRlbnQuY29tL2thcm9saGVyYnN0LzAzYzRjODE0MWIwZmEyOTJkNzgxYmFkZmExODY0Nzll L3Jhdy81YzYyNjQwYWZiYzU3ZDZlNjllYTkyNGMzMzhiZDI4MzZlNzcwZDAyL2dpc3RmaWxlMS50 eHQKCk9uIFdlZCwgTm92IDIwLCAyMDE5IGF0IDU6MjMgUE0gTWlrYSBXZXN0ZXJiZXJnCjxtaWth Lndlc3RlcmJlcmdAaW50ZWwuY29tPiB3cm90ZToKPgo+IE9uIFdlZCwgTm92IDIwLCAyMDE5IGF0 IDA1OjUzOjA3UE0gKzAyMDAsIE1pa2EgV2VzdGVyYmVyZyB3cm90ZToKPiA+IE9uIFdlZCwgTm92 IDIwLCAyMDE5IGF0IDA0OjM3OjE0UE0gKzAxMDAsIEthcm9sIEhlcmJzdCB3cm90ZToKPiA+ID4g T24gV2VkLCBOb3YgMjAsIDIwMTkgYXQgNDoxNSBQTSBNaWthIFdlc3RlcmJlcmcKPiA+ID4gPG1p a2Eud2VzdGVyYmVyZ0BpbnRlbC5jb20+IHdyb3RlOgo+ID4gPiA+Cj4gPiA+ID4gT24gV2VkLCBO b3YgMjAsIDIwMTkgYXQgMDE6MTE6NTJQTSArMDEwMCwgS2Fyb2wgSGVyYnN0IHdyb3RlOgo+ID4g PiA+ID4gT24gV2VkLCBOb3YgMjAsIDIwMTkgYXQgMTowOSBQTSBNaWthIFdlc3RlcmJlcmcKPiA+ ID4gPiA+IDxtaWthLndlc3RlcmJlcmdAaW50ZWwuY29tPiB3cm90ZToKPiA+ID4gPiA+ID4KPiA+ ID4gPiA+ID4gT24gV2VkLCBOb3YgMjAsIDIwMTkgYXQgMTI6NTg6MDBQTSArMDEwMCwgS2Fyb2wg SGVyYnN0IHdyb3RlOgo+ID4gPiA+ID4gPiA+IG92ZXJhbGwsIHdoYXQgSSByZWFsbHkgd2FudCB0 byBrbm93IGlzLCBfd2h5XyBkb2VzIGl0IHdvcmsgb24gd2luZG93cz8KPiA+ID4gPiA+ID4KPiA+ ID4gPiA+ID4gU28gZG8gSSA7LSkKPiA+ID4gPiA+ID4KPiA+ID4gPiA+ID4gPiBPciB3aGF0IGFy ZSB3ZSBkb2luZyBkaWZmZXJlbnRseSBvbiBMaW51eCBzbyB0aGF0IGl0IGRvZXNuJ3Qgd29yaz8g SWYKPiA+ID4gPiA+ID4gPiBhbnlib2R5IGhhcyBhbnkgaWRlYSBvbiBob3cgd2UgY291bGQgZGln IGludG8gdGhpcyBhbmQgZmlndXJlIGl0IG91dAo+ID4gPiA+ID4gPiA+IG9uIHRoaXMgbGV2ZWws IHRoaXMgd291bGQgcHJvYmFibHkgYWxsb3cgdXMgdG8gZ2V0IGNsb3NlciB0byB0aGUgcm9vdAo+ ID4gPiA+ID4gPiA+IGNhdXNlPyBubz8KPiA+ID4gPiA+ID4KPiA+ID4gPiA+ID4gSGF2ZSB5b3Ug dHJpZWQgdG8gdXNlIHRoZSBhY3BpX3Jldl9vdmVycmlkZSBwYXJhbWV0ZXIgaW4geW91ciBzeXN0 ZW0gYW5kCj4gPiA+ID4gPiA+IGRvZXMgaXQgaGF2ZSBhbnkgZWZmZWN0Pwo+ID4gPiA+ID4gPgo+ ID4gPiA+ID4gPiBBbHNvIGRpZCB5b3UgdHJ5IHRvIHRyYWNlIHRoZSBBQ1BJIF9PTi9fT0ZGKCkg bWV0aG9kcz8gSSB0aGluayB0aGF0Cj4gPiA+ID4gPiA+IHNob3VsZCBob3BlZnVsbHkgcmV2ZWFs IHNvbWV0aGluZy4KPiA+ID4gPiA+ID4KPiA+ID4gPiA+Cj4gPiA+ID4gPiBJIHRoaW5rIEkgZGlk IGluIHRoZSBwYXN0IGFuZCBpdCBzZWVtZWQgdG8gaGF2ZSB3b3JrZWQsIHRoZXJlIGlzIGp1c3QK PiA+ID4gPiA+IG9uZSBiaWcgaXNzdWUgd2l0aCB0aGlzOiBpdCdzIGEgRGVsbCBzcGVjaWZpYyB3 b3JrYXJvdW5kIGFmYWlrLCBhbmQKPiA+ID4gPiA+IHRoaXMgaXNzdWUgcGxhZ3VlcyBub3QganVz dCBEZWxsLCBidXQgd2UndmUgc2VlbiBpdCBvbiBIUCBhbmQgTGVub3ZvCj4gPiA+ID4gPiBsYXB0 b3BzIGFzIHdlbGwsIGFuZCBJJ3ZlIGhlYXJkIGFib3V0IHVzZXJzIGhhdmluZyB0aGUgc2FtZSBp c3N1ZXMgb24KPiA+ID4gPiA+IEFzdXMgYW5kIE1TSSBsYXB0b3BzIGFzIHdlbGwuCj4gPiA+ID4K PiA+ID4gPiBNYXliZSBpdCBpcyBub3QgYSB3b3JrYXJvdW5kIGF0IGFsbCBidXQgaW5zdGVhZCBp dCBzaW1wbHkgZGV0ZXJtaW5lcwo+ID4gPiA+IHdoZXRoZXIgdGhlIHN5c3RlbSBzdXBwb3J0cyBS VEQzIG9yIHNvbWV0aGluZyBsaWtlIHRoYXQgKElJUkMgV2luZG93cyA4Cj4gPiA+ID4gc3RhcnRl ZCBzdXBwb3J0aW5nIGl0KS4gTWF5YmUgRGVsbCBhZGRlZCBjaGVjayBmb3IgTGludXggYmVjYXVz ZSBhdCB0aGF0Cj4gPiA+ID4gdGltZSBMaW51eCBkaWQgbm90IHN1cHBvcnQgaXQuCj4gPiA+ID4K PiA+ID4KPiA+ID4gdGhlIHBvaW50IGlzLCBpdCdzIG5vdCBjaGVja2luZyBpdCBieSBkZWZhdWx0 LCBzbyBieSBkZWZhdWx0IHlvdSBzdGlsbAo+ID4gPiBydW4gaW50byB0aGUgd2luZG93cyA4IGNv ZGVwYXRoLgo+ID4KPiA+IFdlbGwgeW91IGNhbiBhZGQgdGhlIHF1aXJrIHRvIGFjcGlfcmV2X2Rt aV90YWJsZVtdIHNvIGl0IGdvZXMgdG8gdGhhdAo+ID4gcGF0aCBieSBkZWZhdWx0LiBUaGVyZSBh cmUgYSBidW5jaCBvZiBzaW1pbGFyIGVudHJpZXMgZm9yIERlbGwgbWFjaGluZXMuCj4gPgo+ID4g T2YgY291cnNlIHRoaXMgZG9lcyBub3QgaGVscCB0aGUgbm9uLURlbGwgdXNlcnMgc28gd2Ugd291 bGQgc3RpbGwgbmVlZAo+ID4gdG8gZmlndXJlIG91dCB0aGUgcm9vdCBjYXVzZS4KPgo+IEkgdGhp bmsgSSBhc2tlZCB5b3UgdG8gdGVzdCB0aGUgUENJZSBkZWxheSBwYXRjaCBhbmQgaXQgZGlkIG5v dCBoZWxwIGJ1dAo+IEkgd29uZGVyIGlmIGl0IGhlbHBzIGlmIHdlIGluY3JlYXNlIHRoZSBkZWxh eS4gQXMgYW4gZXhwZXJpbWVudCBjb3VsZAo+IHlvdSB0cnkgQmpvcm4ncyBwY2kvcG0gYnJhbmNo LiBUaGUgbGFzdCB0d28gY29tbWl0cyBhcmUgZm9yIHRoZSBkZWxheS4KPgo+IElmIHlvdSBjb3Vs ZCBwdWxsIHRoYXQgYnJhbmNoIGFuZCBhcHBseSB0aGUgZm9sbG93aW5nIHBhdGNoIG9uIHRvcCBh bmQKPiBnaXZlIGl0IGEgdHJ5PyBUaGVuIHBvc3QgdGhlIGRtZXNnIHNvbWV3aGVyZSBzbyB3ZSBj YW4gc2VlIHdoZXRoZXIgaXQKPiBkaWQgdGhlIGRlbGF5IGF0IGFsbC4KPgo+IGRpZmYgLS1naXQg YS9kcml2ZXJzL3BjaS9wY2kuYyBiL2RyaXZlcnMvcGNpL3BjaS5jCj4gaW5kZXggMWYzMTliMTE3 NWRhLi4xYWQ2ZjEzNzJlZDUgMTAwNjQ0Cj4gLS0tIGEvZHJpdmVycy9wY2kvcGNpLmMKPiArKysg Yi9kcml2ZXJzL3BjaS9wY2kuYwo+IEBAIC00Njk3LDEyICs0Njk3LDcgQEAgdm9pZCBwY2lfYnJp ZGdlX3dhaXRfZm9yX3NlY29uZGFyeV9idXMoc3RydWN0IHBjaV9kZXYgKmRldikKPiAgICAgICAg ICAgICAgICAgcmV0dXJuOwo+ICAgICAgICAgfQo+Cj4gLSAgICAgICAvKiBUYWtlIGQzY29sZF9k ZWxheSByZXF1aXJlbWVudHMgaW50byBhY2NvdW50ICovCj4gLSAgICAgICBkZWxheSA9IHBjaV9i dXNfbWF4X2QzY29sZF9kZWxheShkZXYtPnN1Ym9yZGluYXRlKTsKPiAtICAgICAgIGlmICghZGVs YXkpIHsKPiAtICAgICAgICAgICAgICAgdXBfcmVhZCgmcGNpX2J1c19zZW0pOwo+IC0gICAgICAg ICAgICAgICByZXR1cm47Cj4gLSAgICAgICB9Cj4gKyAgICAgICBkZWxheSA9IDUwMDsKPgo+ICAg ICAgICAgY2hpbGQgPSBsaXN0X2ZpcnN0X2VudHJ5KCZkZXYtPnN1Ym9yZGluYXRlLT5kZXZpY2Vz LCBzdHJ1Y3QgcGNpX2RldiwKPiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBidXNf bGlzdCk7Cj4gQEAgLTQ3MTUsNyArNDcxMCw3IEBAIHZvaWQgcGNpX2JyaWRnZV93YWl0X2Zvcl9z ZWNvbmRhcnlfYnVzKHN0cnVjdCBwY2lfZGV2ICpkZXYpCj4gICAgICAgICAgKiBtYW5hZ2VtZW50 IGZvciB0aGVtIChzZWUgcGNpX2JyaWRnZV9kM19wb3NzaWJsZSgpKS4KPiAgICAgICAgICAqLwo+ ICAgICAgICAgaWYgKCFwY2lfaXNfcGNpZShkZXYpKSB7Cj4gLSAgICAgICAgICAgICAgIHBjaV9k YmcoZGV2LCAid2FpdGluZyAlZCBtcyBmb3Igc2Vjb25kYXJ5IGJ1c1xuIiwgMTAwMCArIGRlbGF5 KTsKPiArICAgICAgICAgICAgICAgcGNpX2luZm8oZGV2LCAid2FpdGluZyAlZCBtcyBmb3Igc2Vj b25kYXJ5IGJ1c1xuIiwgMTAwMCArIGRlbGF5KTsKPiAgICAgICAgICAgICAgICAgbXNsZWVwKDEw MDAgKyBkZWxheSk7Cj4gICAgICAgICAgICAgICAgIHJldHVybjsKPiAgICAgICAgIH0KPiBAQCAt NDc0MSwxMCArNDczNiwxMCBAQCB2b2lkIHBjaV9icmlkZ2Vfd2FpdF9mb3Jfc2Vjb25kYXJ5X2J1 cyhzdHJ1Y3QgcGNpX2RldiAqZGV2KQo+ICAgICAgICAgICAgICAgICByZXR1cm47Cj4KPiAgICAg ICAgIGlmIChwY2llX2dldF9zcGVlZF9jYXAoZGV2KSA8PSBQQ0lFX1NQRUVEXzVfMEdUKSB7Cj4g LSAgICAgICAgICAgICAgIHBjaV9kYmcoZGV2LCAid2FpdGluZyAlZCBtcyBmb3IgZG93bnN0cmVh bSBsaW5rXG4iLCBkZWxheSk7Cj4gKyAgICAgICAgICAgICAgIHBjaV9pbmZvKGRldiwgIndhaXRp bmcgJWQgbXMgZm9yIGRvd25zdHJlYW0gbGlua1xuIiwgZGVsYXkpOwo+ICAgICAgICAgICAgICAg ICBtc2xlZXAoZGVsYXkpOwo+ICAgICAgICAgfSBlbHNlIHsKPiAtICAgICAgICAgICAgICAgcGNp X2RiZyhkZXYsICJ3YWl0aW5nICVkIG1zIGZvciBkb3duc3RyZWFtIGxpbmssIGFmdGVyIGFjdGl2 YXRpb25cbiIsCj4gKyAgICAgICAgICAgICAgIHBjaV9pbmZvKGRldiwgIndhaXRpbmcgJWQgbXMg Zm9yIGRvd25zdHJlYW0gbGluaywgYWZ0ZXIgYWN0aXZhdGlvblxuIiwKPiAgICAgICAgICAgICAg ICAgICAgICAgICBkZWxheSk7Cj4gICAgICAgICAgICAgICAgIGlmICghcGNpZV93YWl0X2Zvcl9s aW5rX2RlbGF5KGRldiwgdHJ1ZSwgZGVsYXkpKSB7Cj4gICAgICAgICAgICAgICAgICAgICAgICAg LyogRGlkIG5vdCB0cmFpbiwgbm8gbmVlZCB0byB3YWl0IGFueSBmdXJ0aGVyICovCj4gQEAgLTQ3 NTMsNyArNDc0OCw3IEBAIHZvaWQgcGNpX2JyaWRnZV93YWl0X2Zvcl9zZWNvbmRhcnlfYnVzKHN0 cnVjdCBwY2lfZGV2ICpkZXYpCj4gICAgICAgICB9Cj4KPiAgICAgICAgIGlmICghcGNpX2Rldmlj ZV9pc19wcmVzZW50KGNoaWxkKSkgewo+IC0gICAgICAgICAgICAgICBwY2lfZGJnKGNoaWxkLCAi d2FpdGluZyBhZGRpdGlvbmFsICVkIG1zIHRvIGJlY29tZSBhY2Nlc3NpYmxlXG4iLCBkZWxheSk7 Cj4gKyAgICAgICAgICAgICAgIHBjaV9pbmZvKGNoaWxkLCAid2FpdGluZyBhZGRpdGlvbmFsICVk IG1zIHRvIGJlY29tZSBhY2Nlc3NpYmxlXG4iLCBkZWxheSk7Cj4gICAgICAgICAgICAgICAgIG1z bGVlcChkZWxheSk7Cj4gICAgICAgICB9Cj4gIH0KPgoKX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX18KTm91dmVhdSBtYWlsaW5nIGxpc3QKTm91dmVhdUBsaXN0 cy5mcmVlZGVza3RvcC5vcmcKaHR0cHM6Ly9saXN0cy5mcmVlZGVza3RvcC5vcmcvbWFpbG1hbi9s aXN0aW5mby9ub3V2ZWF1 From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.8 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5A8B8C432C0 for ; Wed, 20 Nov 2019 21:36:51 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 32D4D20855 for ; Wed, 20 Nov 2019 21:36:50 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 32D4D20855 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id E0F606EA27; Wed, 20 Nov 2019 21:36:47 +0000 (UTC) Received: from us-smtp-delivery-1.mimecast.com (us-smtp-1.mimecast.com [207.211.31.81]) by gabe.freedesktop.org (Postfix) with ESMTPS id E789A6E9E3 for ; Wed, 20 Nov 2019 21:36:45 +0000 (UTC) Received: from mail-qv1-f71.google.com (mail-qv1-f71.google.com [209.85.219.71]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-267-2_ud3neqMbejJiDRSmaKpQ-1; Wed, 20 Nov 2019 16:36:43 -0500 Received: by mail-qv1-f71.google.com with SMTP id m43so749724qvc.17 for ; Wed, 20 Nov 2019 13:36:43 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=BEOpo1eAW+6WweEGc5o46P0sIScU4OqKVUN6KWPXGDE=; b=Iz1q9FQ1IhmICjQhMuWX7KgEa22CwvsT/CzqLxwiGKEcWwrM/9dP/rzPbo03YBcj7i Hv47nJaNVsUlAyM08btZOdv1kHq9+XEQVZuzLhn2saIgN2wiHx81MeXyc85PZ46aJ9xD twUN1wM3ZhI8spJTbJTQNwWEQDSAuE2lkKE9wMtgMLp/JL3YJrJTwBEgC0y+yj4Bo7Tt Lrp3ZhcDwoIELu4lPaO0fzoIBWZCeWczRnrFIJRiETZpws4/sy22LwdMvJS7sPIbb58r BH/i2eO3s3mdDoCx0yto8afNbd8rLgQUhMKP1Ln2cYSlWxfF+cbSqPa3ChacMMQ5zEau MQ+g== X-Gm-Message-State: APjAAAWH27uBNJ34GGnjTr9GlfZ3OmaoB16nIci9G/D3DuR+CYaWCw+E EaPBu3LHX/eoYDG9tp0zcvy3z/pLbOCDgSIpNXlojGIdxE+6/vTHDr92hwy2R72OtFyzgrbzmJZ Ivx141MU43l6/1a/8sguLtojy68lxCIldeRmGpVN2NLJa X-Received: by 2002:a0c:baad:: with SMTP id x45mr4716314qvf.230.1574285802805; Wed, 20 Nov 2019 13:36:42 -0800 (PST) X-Google-Smtp-Source: APXvYqyekfP3DQaxdB5Z3roInhLmB3w1CVawg+LlvI/EnwQJNFYBD1Fs7D2MZDTcOhlIQfBzDzvxUKzBQtFDGEGE1aM= X-Received: by 2002:a0c:baad:: with SMTP id x45mr4716288qvf.230.1574285802452; Wed, 20 Nov 2019 13:36:42 -0800 (PST) MIME-Version: 1.0 References: <20191120112212.GA11621@lahna.fi.intel.com> <20191120115127.GD11621@lahna.fi.intel.com> <20191120120913.GE11621@lahna.fi.intel.com> <20191120151542.GH11621@lahna.fi.intel.com> <20191120155301.GL11621@lahna.fi.intel.com> <20191120162306.GM11621@lahna.fi.intel.com> In-Reply-To: <20191120162306.GM11621@lahna.fi.intel.com> From: Karol Herbst Date: Wed, 20 Nov 2019 22:36:31 +0100 Message-ID: Subject: Re: [PATCH v4] pci: prevent putting nvidia GPUs into lower device states on certain intel bridges To: Mika Westerberg X-MC-Unique: 2_ud3neqMbejJiDRSmaKpQ-1 X-Mimecast-Spam-Score: 0 X-Mailman-Original-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1574285804; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0pAJwOUYZlIukE0BbqgWGnjQga+s55dRnSCojvuuOg8=; b=WnYCpyrj2aGLa/LNN6Okn1S8iOzMMNmvquuG2JtT+CBMhvUrbgVCbCegIHZzOekGTE14Yk UtoLD/wjyetuhgDnO/goi9y8zvm69lceRC3rLfP0UItYE/I3o0UC+gHRk3dsKTjzLaOHJG qmeA5oUw9GP9y67PUSr11oLETKv60YY= X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: "Rafael J. Wysocki" , Linux PCI , Linux PM , "Rafael J . Wysocki" , LKML , dri-devel , Mario Limonciello , Bjorn Helgaas , nouveau Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Message-ID: <20191120213631.prYqiJEzftxcP-SnAhpK4F7JVcRZ7EfmQpk5RlNrSyo@z> d2l0aCB0aGUgYnJhbmNoIGFuZCBwYXRjaCBhcHBsaWVkOgpodHRwczovL2dpc3QuZ2l0aHVidXNl cmNvbnRlbnQuY29tL2thcm9saGVyYnN0LzAzYzRjODE0MWIwZmEyOTJkNzgxYmFkZmExODY0Nzll L3Jhdy81YzYyNjQwYWZiYzU3ZDZlNjllYTkyNGMzMzhiZDI4MzZlNzcwZDAyL2dpc3RmaWxlMS50 eHQKCk9uIFdlZCwgTm92IDIwLCAyMDE5IGF0IDU6MjMgUE0gTWlrYSBXZXN0ZXJiZXJnCjxtaWth Lndlc3RlcmJlcmdAaW50ZWwuY29tPiB3cm90ZToKPgo+IE9uIFdlZCwgTm92IDIwLCAyMDE5IGF0 IDA1OjUzOjA3UE0gKzAyMDAsIE1pa2EgV2VzdGVyYmVyZyB3cm90ZToKPiA+IE9uIFdlZCwgTm92 IDIwLCAyMDE5IGF0IDA0OjM3OjE0UE0gKzAxMDAsIEthcm9sIEhlcmJzdCB3cm90ZToKPiA+ID4g T24gV2VkLCBOb3YgMjAsIDIwMTkgYXQgNDoxNSBQTSBNaWthIFdlc3RlcmJlcmcKPiA+ID4gPG1p a2Eud2VzdGVyYmVyZ0BpbnRlbC5jb20+IHdyb3RlOgo+ID4gPiA+Cj4gPiA+ID4gT24gV2VkLCBO b3YgMjAsIDIwMTkgYXQgMDE6MTE6NTJQTSArMDEwMCwgS2Fyb2wgSGVyYnN0IHdyb3RlOgo+ID4g PiA+ID4gT24gV2VkLCBOb3YgMjAsIDIwMTkgYXQgMTowOSBQTSBNaWthIFdlc3RlcmJlcmcKPiA+ ID4gPiA+IDxtaWthLndlc3RlcmJlcmdAaW50ZWwuY29tPiB3cm90ZToKPiA+ID4gPiA+ID4KPiA+ ID4gPiA+ID4gT24gV2VkLCBOb3YgMjAsIDIwMTkgYXQgMTI6NTg6MDBQTSArMDEwMCwgS2Fyb2wg SGVyYnN0IHdyb3RlOgo+ID4gPiA+ID4gPiA+IG92ZXJhbGwsIHdoYXQgSSByZWFsbHkgd2FudCB0 byBrbm93IGlzLCBfd2h5XyBkb2VzIGl0IHdvcmsgb24gd2luZG93cz8KPiA+ID4gPiA+ID4KPiA+ ID4gPiA+ID4gU28gZG8gSSA7LSkKPiA+ID4gPiA+ID4KPiA+ID4gPiA+ID4gPiBPciB3aGF0IGFy ZSB3ZSBkb2luZyBkaWZmZXJlbnRseSBvbiBMaW51eCBzbyB0aGF0IGl0IGRvZXNuJ3Qgd29yaz8g SWYKPiA+ID4gPiA+ID4gPiBhbnlib2R5IGhhcyBhbnkgaWRlYSBvbiBob3cgd2UgY291bGQgZGln IGludG8gdGhpcyBhbmQgZmlndXJlIGl0IG91dAo+ID4gPiA+ID4gPiA+IG9uIHRoaXMgbGV2ZWws IHRoaXMgd291bGQgcHJvYmFibHkgYWxsb3cgdXMgdG8gZ2V0IGNsb3NlciB0byB0aGUgcm9vdAo+ ID4gPiA+ID4gPiA+IGNhdXNlPyBubz8KPiA+ID4gPiA+ID4KPiA+ID4gPiA+ID4gSGF2ZSB5b3Ug dHJpZWQgdG8gdXNlIHRoZSBhY3BpX3Jldl9vdmVycmlkZSBwYXJhbWV0ZXIgaW4geW91ciBzeXN0 ZW0gYW5kCj4gPiA+ID4gPiA+IGRvZXMgaXQgaGF2ZSBhbnkgZWZmZWN0Pwo+ID4gPiA+ID4gPgo+ ID4gPiA+ID4gPiBBbHNvIGRpZCB5b3UgdHJ5IHRvIHRyYWNlIHRoZSBBQ1BJIF9PTi9fT0ZGKCkg bWV0aG9kcz8gSSB0aGluayB0aGF0Cj4gPiA+ID4gPiA+IHNob3VsZCBob3BlZnVsbHkgcmV2ZWFs IHNvbWV0aGluZy4KPiA+ID4gPiA+ID4KPiA+ID4gPiA+Cj4gPiA+ID4gPiBJIHRoaW5rIEkgZGlk IGluIHRoZSBwYXN0IGFuZCBpdCBzZWVtZWQgdG8gaGF2ZSB3b3JrZWQsIHRoZXJlIGlzIGp1c3QK PiA+ID4gPiA+IG9uZSBiaWcgaXNzdWUgd2l0aCB0aGlzOiBpdCdzIGEgRGVsbCBzcGVjaWZpYyB3 b3JrYXJvdW5kIGFmYWlrLCBhbmQKPiA+ID4gPiA+IHRoaXMgaXNzdWUgcGxhZ3VlcyBub3QganVz dCBEZWxsLCBidXQgd2UndmUgc2VlbiBpdCBvbiBIUCBhbmQgTGVub3ZvCj4gPiA+ID4gPiBsYXB0 b3BzIGFzIHdlbGwsIGFuZCBJJ3ZlIGhlYXJkIGFib3V0IHVzZXJzIGhhdmluZyB0aGUgc2FtZSBp c3N1ZXMgb24KPiA+ID4gPiA+IEFzdXMgYW5kIE1TSSBsYXB0b3BzIGFzIHdlbGwuCj4gPiA+ID4K PiA+ID4gPiBNYXliZSBpdCBpcyBub3QgYSB3b3JrYXJvdW5kIGF0IGFsbCBidXQgaW5zdGVhZCBp dCBzaW1wbHkgZGV0ZXJtaW5lcwo+ID4gPiA+IHdoZXRoZXIgdGhlIHN5c3RlbSBzdXBwb3J0cyBS VEQzIG9yIHNvbWV0aGluZyBsaWtlIHRoYXQgKElJUkMgV2luZG93cyA4Cj4gPiA+ID4gc3RhcnRl ZCBzdXBwb3J0aW5nIGl0KS4gTWF5YmUgRGVsbCBhZGRlZCBjaGVjayBmb3IgTGludXggYmVjYXVz ZSBhdCB0aGF0Cj4gPiA+ID4gdGltZSBMaW51eCBkaWQgbm90IHN1cHBvcnQgaXQuCj4gPiA+ID4K PiA+ID4KPiA+ID4gdGhlIHBvaW50IGlzLCBpdCdzIG5vdCBjaGVja2luZyBpdCBieSBkZWZhdWx0 LCBzbyBieSBkZWZhdWx0IHlvdSBzdGlsbAo+ID4gPiBydW4gaW50byB0aGUgd2luZG93cyA4IGNv ZGVwYXRoLgo+ID4KPiA+IFdlbGwgeW91IGNhbiBhZGQgdGhlIHF1aXJrIHRvIGFjcGlfcmV2X2Rt aV90YWJsZVtdIHNvIGl0IGdvZXMgdG8gdGhhdAo+ID4gcGF0aCBieSBkZWZhdWx0LiBUaGVyZSBh cmUgYSBidW5jaCBvZiBzaW1pbGFyIGVudHJpZXMgZm9yIERlbGwgbWFjaGluZXMuCj4gPgo+ID4g T2YgY291cnNlIHRoaXMgZG9lcyBub3QgaGVscCB0aGUgbm9uLURlbGwgdXNlcnMgc28gd2Ugd291 bGQgc3RpbGwgbmVlZAo+ID4gdG8gZmlndXJlIG91dCB0aGUgcm9vdCBjYXVzZS4KPgo+IEkgdGhp bmsgSSBhc2tlZCB5b3UgdG8gdGVzdCB0aGUgUENJZSBkZWxheSBwYXRjaCBhbmQgaXQgZGlkIG5v dCBoZWxwIGJ1dAo+IEkgd29uZGVyIGlmIGl0IGhlbHBzIGlmIHdlIGluY3JlYXNlIHRoZSBkZWxh eS4gQXMgYW4gZXhwZXJpbWVudCBjb3VsZAo+IHlvdSB0cnkgQmpvcm4ncyBwY2kvcG0gYnJhbmNo LiBUaGUgbGFzdCB0d28gY29tbWl0cyBhcmUgZm9yIHRoZSBkZWxheS4KPgo+IElmIHlvdSBjb3Vs ZCBwdWxsIHRoYXQgYnJhbmNoIGFuZCBhcHBseSB0aGUgZm9sbG93aW5nIHBhdGNoIG9uIHRvcCBh bmQKPiBnaXZlIGl0IGEgdHJ5PyBUaGVuIHBvc3QgdGhlIGRtZXNnIHNvbWV3aGVyZSBzbyB3ZSBj YW4gc2VlIHdoZXRoZXIgaXQKPiBkaWQgdGhlIGRlbGF5IGF0IGFsbC4KPgo+IGRpZmYgLS1naXQg YS9kcml2ZXJzL3BjaS9wY2kuYyBiL2RyaXZlcnMvcGNpL3BjaS5jCj4gaW5kZXggMWYzMTliMTE3 NWRhLi4xYWQ2ZjEzNzJlZDUgMTAwNjQ0Cj4gLS0tIGEvZHJpdmVycy9wY2kvcGNpLmMKPiArKysg Yi9kcml2ZXJzL3BjaS9wY2kuYwo+IEBAIC00Njk3LDEyICs0Njk3LDcgQEAgdm9pZCBwY2lfYnJp ZGdlX3dhaXRfZm9yX3NlY29uZGFyeV9idXMoc3RydWN0IHBjaV9kZXYgKmRldikKPiAgICAgICAg ICAgICAgICAgcmV0dXJuOwo+ICAgICAgICAgfQo+Cj4gLSAgICAgICAvKiBUYWtlIGQzY29sZF9k ZWxheSByZXF1aXJlbWVudHMgaW50byBhY2NvdW50ICovCj4gLSAgICAgICBkZWxheSA9IHBjaV9i dXNfbWF4X2QzY29sZF9kZWxheShkZXYtPnN1Ym9yZGluYXRlKTsKPiAtICAgICAgIGlmICghZGVs YXkpIHsKPiAtICAgICAgICAgICAgICAgdXBfcmVhZCgmcGNpX2J1c19zZW0pOwo+IC0gICAgICAg ICAgICAgICByZXR1cm47Cj4gLSAgICAgICB9Cj4gKyAgICAgICBkZWxheSA9IDUwMDsKPgo+ICAg ICAgICAgY2hpbGQgPSBsaXN0X2ZpcnN0X2VudHJ5KCZkZXYtPnN1Ym9yZGluYXRlLT5kZXZpY2Vz LCBzdHJ1Y3QgcGNpX2RldiwKPiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBidXNf bGlzdCk7Cj4gQEAgLTQ3MTUsNyArNDcxMCw3IEBAIHZvaWQgcGNpX2JyaWRnZV93YWl0X2Zvcl9z ZWNvbmRhcnlfYnVzKHN0cnVjdCBwY2lfZGV2ICpkZXYpCj4gICAgICAgICAgKiBtYW5hZ2VtZW50 IGZvciB0aGVtIChzZWUgcGNpX2JyaWRnZV9kM19wb3NzaWJsZSgpKS4KPiAgICAgICAgICAqLwo+ ICAgICAgICAgaWYgKCFwY2lfaXNfcGNpZShkZXYpKSB7Cj4gLSAgICAgICAgICAgICAgIHBjaV9k YmcoZGV2LCAid2FpdGluZyAlZCBtcyBmb3Igc2Vjb25kYXJ5IGJ1c1xuIiwgMTAwMCArIGRlbGF5 KTsKPiArICAgICAgICAgICAgICAgcGNpX2luZm8oZGV2LCAid2FpdGluZyAlZCBtcyBmb3Igc2Vj b25kYXJ5IGJ1c1xuIiwgMTAwMCArIGRlbGF5KTsKPiAgICAgICAgICAgICAgICAgbXNsZWVwKDEw MDAgKyBkZWxheSk7Cj4gICAgICAgICAgICAgICAgIHJldHVybjsKPiAgICAgICAgIH0KPiBAQCAt NDc0MSwxMCArNDczNiwxMCBAQCB2b2lkIHBjaV9icmlkZ2Vfd2FpdF9mb3Jfc2Vjb25kYXJ5X2J1 cyhzdHJ1Y3QgcGNpX2RldiAqZGV2KQo+ICAgICAgICAgICAgICAgICByZXR1cm47Cj4KPiAgICAg ICAgIGlmIChwY2llX2dldF9zcGVlZF9jYXAoZGV2KSA8PSBQQ0lFX1NQRUVEXzVfMEdUKSB7Cj4g LSAgICAgICAgICAgICAgIHBjaV9kYmcoZGV2LCAid2FpdGluZyAlZCBtcyBmb3IgZG93bnN0cmVh bSBsaW5rXG4iLCBkZWxheSk7Cj4gKyAgICAgICAgICAgICAgIHBjaV9pbmZvKGRldiwgIndhaXRp bmcgJWQgbXMgZm9yIGRvd25zdHJlYW0gbGlua1xuIiwgZGVsYXkpOwo+ICAgICAgICAgICAgICAg ICBtc2xlZXAoZGVsYXkpOwo+ICAgICAgICAgfSBlbHNlIHsKPiAtICAgICAgICAgICAgICAgcGNp X2RiZyhkZXYsICJ3YWl0aW5nICVkIG1zIGZvciBkb3duc3RyZWFtIGxpbmssIGFmdGVyIGFjdGl2 YXRpb25cbiIsCj4gKyAgICAgICAgICAgICAgIHBjaV9pbmZvKGRldiwgIndhaXRpbmcgJWQgbXMg Zm9yIGRvd25zdHJlYW0gbGluaywgYWZ0ZXIgYWN0aXZhdGlvblxuIiwKPiAgICAgICAgICAgICAg ICAgICAgICAgICBkZWxheSk7Cj4gICAgICAgICAgICAgICAgIGlmICghcGNpZV93YWl0X2Zvcl9s aW5rX2RlbGF5KGRldiwgdHJ1ZSwgZGVsYXkpKSB7Cj4gICAgICAgICAgICAgICAgICAgICAgICAg LyogRGlkIG5vdCB0cmFpbiwgbm8gbmVlZCB0byB3YWl0IGFueSBmdXJ0aGVyICovCj4gQEAgLTQ3 NTMsNyArNDc0OCw3IEBAIHZvaWQgcGNpX2JyaWRnZV93YWl0X2Zvcl9zZWNvbmRhcnlfYnVzKHN0 cnVjdCBwY2lfZGV2ICpkZXYpCj4gICAgICAgICB9Cj4KPiAgICAgICAgIGlmICghcGNpX2Rldmlj ZV9pc19wcmVzZW50KGNoaWxkKSkgewo+IC0gICAgICAgICAgICAgICBwY2lfZGJnKGNoaWxkLCAi d2FpdGluZyBhZGRpdGlvbmFsICVkIG1zIHRvIGJlY29tZSBhY2Nlc3NpYmxlXG4iLCBkZWxheSk7 Cj4gKyAgICAgICAgICAgICAgIHBjaV9pbmZvKGNoaWxkLCAid2FpdGluZyBhZGRpdGlvbmFsICVk IG1zIHRvIGJlY29tZSBhY2Nlc3NpYmxlXG4iLCBkZWxheSk7Cj4gICAgICAgICAgICAgICAgIG1z bGVlcChkZWxheSk7Cj4gICAgICAgICB9Cj4gIH0KPgoKX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX18KZHJpLWRldmVsIG1haWxpbmcgbGlzdApkcmktZGV2ZWxA bGlzdHMuZnJlZWRlc2t0b3Aub3JnCmh0dHBzOi8vbGlzdHMuZnJlZWRlc2t0b3Aub3JnL21haWxt YW4vbGlzdGluZm8vZHJpLWRldmVs