From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.1 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A90C2C432C3 for ; Thu, 21 Nov 2019 15:43:39 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 82219206B6 for ; Thu, 21 Nov 2019 15:43:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1574351019; bh=JdSx8VgthvKm/DukO4rs0RA86l4rKTii5jKc/DpVSSM=; h=References:In-Reply-To:From:Date:Subject:To:Cc:List-ID:From; b=Hhf42oN4w95FSVb2CYzrz9ZBBA/+3GllWni9nPpeeOdtK/dnMnsoIpgDtWmfX9B4/ t3rvuQysjXz8RAgUOEbhWHW7Cvnc2wTM14lb6XW1kIwbYkPNaVUHojQx2LyMZ89UeP VjzhJEFR5mC2rkzfTsM+RjAlUGCu/IkLlZqUEUoA= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726984AbfKUPni (ORCPT ); Thu, 21 Nov 2019 10:43:38 -0500 Received: from mail-ot1-f66.google.com ([209.85.210.66]:37137 "EHLO mail-ot1-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726293AbfKUPni (ORCPT ); Thu, 21 Nov 2019 10:43:38 -0500 Received: by mail-ot1-f66.google.com with SMTP id d5so3325616otp.4; Thu, 21 Nov 2019 07:43:36 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=O6BtIcSR6Sz2/7y/lGE3fC0q1eR/CKJYxQr5CTA/9Uc=; b=pOhfT9kJ9HEyHZ+7YqsMjsJ+4osY8nBU/WgE0IKnWh6msl4uQx+4Sg2jK3q6qcW2CQ crFwN/1Te5TvAzC+5eM22SdDiZ6yqSYk1t8AV0SbW9foKMsHy2dblhmfT6eJUHKJFOAS bic8yDNUhdLRvTgKRY3zjly7b6D+mTUO9Bn7dtHLeXOTphZuWgF+D9V/pWBelIlX3BCf EVQRqQXzMFEUohiSvdFVb0moxpsFdohGCpFOwiohtPUD/Jiou5L6a7D0tBXijd5HC9PJ SbEMk01zd2cFJl4M84EXr+0VPtO+DFSX+3aRS8eRjwdg02C6mzimAJfdisMJulc/g7Sj LqjA== X-Gm-Message-State: APjAAAX0m4KP023mktn65lPnA2waSXnC0potUZmLfg71FltJCjK4dvH0 g2KAWSff9YjlOpiEKJJH6zEfgCjD3bPkOzteYYk= X-Google-Smtp-Source: APXvYqxszDM90W/tRIlxbpBS9iYycKQwHojBugtimrAfg57FfxPfoUZyiQ/a8iP2L0RYZnuBEcgjDJESRlnjbPGrO6M= X-Received: by 2002:a05:6830:232a:: with SMTP id q10mr7175465otg.262.1574351015588; Thu, 21 Nov 2019 07:43:35 -0800 (PST) MIME-Version: 1.0 References: <20191120151542.GH11621@lahna.fi.intel.com> <20191120155301.GL11621@lahna.fi.intel.com> <20191121112821.GU11621@lahna.fi.intel.com> <20191121114610.GW11621@lahna.fi.intel.com> <20191121125236.GX11621@lahna.fi.intel.com> In-Reply-To: <20191121125236.GX11621@lahna.fi.intel.com> From: "Rafael J. Wysocki" Date: Thu, 21 Nov 2019 16:43:24 +0100 Message-ID: Subject: Re: [PATCH v4] pci: prevent putting nvidia GPUs into lower device states on certain intel bridges To: Mika Westerberg Cc: "Rafael J. Wysocki" , Karol Herbst , Bjorn Helgaas , LKML , Lyude Paul , "Rafael J . Wysocki" , Linux PCI , Linux PM , dri-devel , nouveau , Dave Airlie , Mario Limonciello Content-Type: text/plain; charset="UTF-8" Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Thu, Nov 21, 2019 at 1:52 PM Mika Westerberg wrote: > > On Thu, Nov 21, 2019 at 01:46:14PM +0200, Mika Westerberg wrote: > > On Thu, Nov 21, 2019 at 12:34:22PM +0100, Rafael J. Wysocki wrote: > > > On Thu, Nov 21, 2019 at 12:28 PM Mika Westerberg > > > wrote: > > > > > > > > On Wed, Nov 20, 2019 at 11:29:33PM +0100, Rafael J. Wysocki wrote: > > > > > > last week or so I found systems where the GPU was under the "PCI > > > > > > Express Root Port" (name from lspci) and on those systems all of that > > > > > > seems to work. So I am wondering if it's indeed just the 0x1901 one, > > > > > > which also explains Mikas case that Thunderbolt stuff works as devices > > > > > > never get populated under this particular bridge controller, but under > > > > > > those "Root Port"s > > > > > > > > > > It always is a PCIe port, but its location within the SoC may matter. > > > > > > > > Exactly. Intel hardware has PCIe ports on CPU side (these are called > > > > PEG, PCI Express Graphics, ports), and the PCH side. I think the IP is > > > > still the same. > > > > > > > > > Also some custom AML-based power management is involved and that may > > > > > be making specific assumptions on the configuration of the SoC and the > > > > > GPU at the time of its invocation which unfortunately are not known to > > > > > us. > > > > > > > > > > However, it looks like the AML invoked to power down the GPU from > > > > > acpi_pci_set_power_state() gets confused if it is not in PCI D0 at > > > > > that point, so it looks like that AML tries to access device memory on > > > > > the GPU (beyond the PCI config space) or similar which is not > > > > > accessible in PCI power states below D0. > > > > > > > > Or the PCI config space of the GPU when the parent root port is in D3hot > > > > (as it is the case here). Also then the GPU config space is not > > > > accessible. > > > > > > Why would the parent port be in D3hot at that point? Wouldn't that be > > > a suspend ordering violation? > > > > No. We put the GPU into D3hot first, OK Does this involve any AML, like a _PS3 under the GPU object? > > then the root port and then turn > > off the power resource (which is attached to the root port) resulting > > the topology entering D3cold. > > I don't see that happening in the AML though. Which AML do you mean, specifically? The _OFF method for the root port's _PR3 power resource or something else? > Basically the difference is that when Windows 7 or Linux (the _REV==5 > check) then we directly do link disable whereas in Windows 8+ we invoke > LKDS() method that puts the link into L2/L3. None of the fields they > access seem to touch the GPU itself. So that may be where the problem is. Putting the downstream component into PCI D[1-3] is expected to put the link into L1, so I'm not sure how that plays with the later attempt to put it into L2/L3 Ready. Also, L2/L3 Ready is expected to be transient, so finally power should be removed somehow. > LKDS() for the first PEG port looks like this: > > P0L2 = One > Sleep (0x10) > Local0 = Zero > While (P0L2) > { > If ((Local0 > 0x04)) > { > Break > } > > Sleep (0x10) > Local0++ > } > > One thing that comes to mind is that the loop can end even if P0L2 is > not cleared as it does only 5 iterations with 16 ms sleep between. Maybe > Sleep() is implemented differently in Windows? I mean Linux may be > "faster" here and return prematurely and if we leave the port into D0 > this does not happen, or something. I'm just throwing out ideas :) But this actually works for the downstream component in D0, doesn't it? Also, if the downstream component is in D0, the port actually should stay in D0 too, so what would happen with the $subject patch applied? From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Rafael J. Wysocki" Subject: Re: [PATCH v4] pci: prevent putting nvidia GPUs into lower device states on certain intel bridges Date: Thu, 21 Nov 2019 16:43:24 +0100 Message-ID: References: <20191120151542.GH11621@lahna.fi.intel.com> <20191120155301.GL11621@lahna.fi.intel.com> <20191121112821.GU11621@lahna.fi.intel.com> <20191121114610.GW11621@lahna.fi.intel.com> <20191121125236.GX11621@lahna.fi.intel.com> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Return-path: In-Reply-To: <20191121125236.GX11621-3PARRvDOhMZrdx17CPfAsdBPR1lH4CV8@public.gmane.org> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: nouveau-bounces-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW@public.gmane.org Sender: "Nouveau" To: Mika Westerberg Cc: "Rafael J. Wysocki" , Linux PCI , Linux PM , "Rafael J . Wysocki" , LKML , dri-devel , Mario Limonciello , Bjorn Helgaas , nouveau List-Id: nouveau.vger.kernel.org T24gVGh1LCBOb3YgMjEsIDIwMTkgYXQgMTo1MiBQTSBNaWthIFdlc3RlcmJlcmcKPG1pa2Eud2Vz dGVyYmVyZ0BpbnRlbC5jb20+IHdyb3RlOgo+Cj4gT24gVGh1LCBOb3YgMjEsIDIwMTkgYXQgMDE6 NDY6MTRQTSArMDIwMCwgTWlrYSBXZXN0ZXJiZXJnIHdyb3RlOgo+ID4gT24gVGh1LCBOb3YgMjEs IDIwMTkgYXQgMTI6MzQ6MjJQTSArMDEwMCwgUmFmYWVsIEouIFd5c29ja2kgd3JvdGU6Cj4gPiA+ IE9uIFRodSwgTm92IDIxLCAyMDE5IGF0IDEyOjI4IFBNIE1pa2EgV2VzdGVyYmVyZwo+ID4gPiA8 bWlrYS53ZXN0ZXJiZXJnQGludGVsLmNvbT4gd3JvdGU6Cj4gPiA+ID4KPiA+ID4gPiBPbiBXZWQs IE5vdiAyMCwgMjAxOSBhdCAxMToyOTozM1BNICswMTAwLCBSYWZhZWwgSi4gV3lzb2NraSB3cm90 ZToKPiA+ID4gPiA+ID4gbGFzdCB3ZWVrIG9yIHNvIEkgZm91bmQgc3lzdGVtcyB3aGVyZSB0aGUg R1BVIHdhcyB1bmRlciB0aGUgIlBDSQo+ID4gPiA+ID4gPiBFeHByZXNzIFJvb3QgUG9ydCIgKG5h bWUgZnJvbSBsc3BjaSkgYW5kIG9uIHRob3NlIHN5c3RlbXMgYWxsIG9mIHRoYXQKPiA+ID4gPiA+ ID4gc2VlbXMgdG8gd29yay4gU28gSSBhbSB3b25kZXJpbmcgaWYgaXQncyBpbmRlZWQganVzdCB0 aGUgMHgxOTAxIG9uZSwKPiA+ID4gPiA+ID4gd2hpY2ggYWxzbyBleHBsYWlucyBNaWthcyBjYXNl IHRoYXQgVGh1bmRlcmJvbHQgc3R1ZmYgd29ya3MgYXMgZGV2aWNlcwo+ID4gPiA+ID4gPiBuZXZl ciBnZXQgcG9wdWxhdGVkIHVuZGVyIHRoaXMgcGFydGljdWxhciBicmlkZ2UgY29udHJvbGxlciwg YnV0IHVuZGVyCj4gPiA+ID4gPiA+IHRob3NlICJSb290IFBvcnQicwo+ID4gPiA+ID4KPiA+ID4g PiA+IEl0IGFsd2F5cyBpcyBhIFBDSWUgcG9ydCwgYnV0IGl0cyBsb2NhdGlvbiB3aXRoaW4gdGhl IFNvQyBtYXkgbWF0dGVyLgo+ID4gPiA+Cj4gPiA+ID4gRXhhY3RseS4gSW50ZWwgaGFyZHdhcmUg aGFzIFBDSWUgcG9ydHMgb24gQ1BVIHNpZGUgKHRoZXNlIGFyZSBjYWxsZWQKPiA+ID4gPiBQRUcs IFBDSSBFeHByZXNzIEdyYXBoaWNzLCBwb3J0cyksIGFuZCB0aGUgUENIIHNpZGUuIEkgdGhpbmsg dGhlIElQIGlzCj4gPiA+ID4gc3RpbGwgdGhlIHNhbWUuCj4gPiA+ID4KPiA+ID4gPiA+IEFsc28g c29tZSBjdXN0b20gQU1MLWJhc2VkIHBvd2VyIG1hbmFnZW1lbnQgaXMgaW52b2x2ZWQgYW5kIHRo YXQgbWF5Cj4gPiA+ID4gPiBiZSBtYWtpbmcgc3BlY2lmaWMgYXNzdW1wdGlvbnMgb24gdGhlIGNv bmZpZ3VyYXRpb24gb2YgdGhlIFNvQyBhbmQgdGhlCj4gPiA+ID4gPiBHUFUgYXQgdGhlIHRpbWUg b2YgaXRzIGludm9jYXRpb24gd2hpY2ggdW5mb3J0dW5hdGVseSBhcmUgbm90IGtub3duIHRvCj4g PiA+ID4gPiB1cy4KPiA+ID4gPiA+Cj4gPiA+ID4gPiBIb3dldmVyLCBpdCBsb29rcyBsaWtlIHRo ZSBBTUwgaW52b2tlZCB0byBwb3dlciBkb3duIHRoZSBHUFUgZnJvbQo+ID4gPiA+ID4gYWNwaV9w Y2lfc2V0X3Bvd2VyX3N0YXRlKCkgZ2V0cyBjb25mdXNlZCBpZiBpdCBpcyBub3QgaW4gUENJIEQw IGF0Cj4gPiA+ID4gPiB0aGF0IHBvaW50LCBzbyBpdCBsb29rcyBsaWtlIHRoYXQgQU1MIHRyaWVz IHRvIGFjY2VzcyBkZXZpY2UgbWVtb3J5IG9uCj4gPiA+ID4gPiB0aGUgR1BVIChiZXlvbmQgdGhl IFBDSSBjb25maWcgc3BhY2UpIG9yIHNpbWlsYXIgd2hpY2ggaXMgbm90Cj4gPiA+ID4gPiBhY2Nl c3NpYmxlIGluIFBDSSBwb3dlciBzdGF0ZXMgYmVsb3cgRDAuCj4gPiA+ID4KPiA+ID4gPiBPciB0 aGUgUENJIGNvbmZpZyBzcGFjZSBvZiB0aGUgR1BVIHdoZW4gdGhlIHBhcmVudCByb290IHBvcnQg aXMgaW4gRDNob3QKPiA+ID4gPiAoYXMgaXQgaXMgdGhlIGNhc2UgaGVyZSkuIEFsc28gdGhlbiB0 aGUgR1BVIGNvbmZpZyBzcGFjZSBpcyBub3QKPiA+ID4gPiBhY2Nlc3NpYmxlLgo+ID4gPgo+ID4g PiBXaHkgd291bGQgdGhlIHBhcmVudCBwb3J0IGJlIGluIEQzaG90IGF0IHRoYXQgcG9pbnQ/ICBX b3VsZG4ndCB0aGF0IGJlCj4gPiA+IGEgc3VzcGVuZCBvcmRlcmluZyB2aW9sYXRpb24/Cj4gPgo+ ID4gTm8uIFdlIHB1dCB0aGUgR1BVIGludG8gRDNob3QgZmlyc3QsCgpPSwoKRG9lcyB0aGlzIGlu dm9sdmUgYW55IEFNTCwgbGlrZSBhIF9QUzMgdW5kZXIgdGhlIEdQVSBvYmplY3Q/Cgo+ID4gdGhl biB0aGUgcm9vdCBwb3J0IGFuZCB0aGVuIHR1cm4KPiA+IG9mZiB0aGUgcG93ZXIgcmVzb3VyY2Ug KHdoaWNoIGlzIGF0dGFjaGVkIHRvIHRoZSByb290IHBvcnQpIHJlc3VsdGluZwo+ID4gdGhlIHRv cG9sb2d5IGVudGVyaW5nIEQzY29sZC4KPgo+IEkgZG9uJ3Qgc2VlIHRoYXQgaGFwcGVuaW5nIGlu IHRoZSBBTUwgdGhvdWdoLgoKV2hpY2ggQU1MIGRvIHlvdSBtZWFuLCBzcGVjaWZpY2FsbHk/ICBU aGUgX09GRiBtZXRob2QgZm9yIHRoZSByb290CnBvcnQncyBfUFIzIHBvd2VyIHJlc291cmNlIG9y IHNvbWV0aGluZyBlbHNlPwoKPiBCYXNpY2FsbHkgdGhlIGRpZmZlcmVuY2UgaXMgdGhhdCB3aGVu IFdpbmRvd3MgNyBvciBMaW51eCAodGhlIF9SRVY9PTUKPiBjaGVjaykgdGhlbiB3ZSBkaXJlY3Rs eSBkbyBsaW5rIGRpc2FibGUgd2hlcmVhcyBpbiBXaW5kb3dzIDgrIHdlIGludm9rZQo+IExLRFMo KSBtZXRob2QgdGhhdCBwdXRzIHRoZSBsaW5rIGludG8gTDIvTDMuIE5vbmUgb2YgdGhlIGZpZWxk cyB0aGV5Cj4gYWNjZXNzIHNlZW0gdG8gdG91Y2ggdGhlIEdQVSBpdHNlbGYuCgpTbyB0aGF0IG1h eSBiZSB3aGVyZSB0aGUgcHJvYmxlbSBpcy4KClB1dHRpbmcgdGhlIGRvd25zdHJlYW0gY29tcG9u ZW50IGludG8gUENJIERbMS0zXSBpcyBleHBlY3RlZCB0byBwdXQKdGhlIGxpbmsgaW50byBMMSwg c28gSSdtIG5vdCBzdXJlIGhvdyB0aGF0IHBsYXlzIHdpdGggdGhlIGxhdGVyCmF0dGVtcHQgdG8g cHV0IGl0IGludG8gTDIvTDMgUmVhZHkuCgpBbHNvLCBMMi9MMyBSZWFkeSBpcyBleHBlY3RlZCB0 byBiZSB0cmFuc2llbnQsIHNvIGZpbmFsbHkgcG93ZXIgc2hvdWxkCmJlIHJlbW92ZWQgc29tZWhv dy4KCj4gTEtEUygpIGZvciB0aGUgZmlyc3QgUEVHIHBvcnQgbG9va3MgbGlrZSB0aGlzOgo+Cj4g ICAgUDBMMiA9IE9uZQo+ICAgIFNsZWVwICgweDEwKQo+ICAgIExvY2FsMCA9IFplcm8KPiAgICBX aGlsZSAoUDBMMikKPiAgICB7Cj4gICAgICAgICBJZiAoKExvY2FsMCA+IDB4MDQpKQo+ICAgICAg ICAgewo+ICAgICAgICAgICAgIEJyZWFrCj4gICAgICAgICB9Cj4KPiAgICAgICAgIFNsZWVwICgw eDEwKQo+ICAgICAgICAgTG9jYWwwKysKPiAgICB9Cj4KPiBPbmUgdGhpbmcgdGhhdCBjb21lcyB0 byBtaW5kIGlzIHRoYXQgdGhlIGxvb3AgY2FuIGVuZCBldmVuIGlmIFAwTDIgaXMKPiBub3QgY2xl YXJlZCBhcyBpdCBkb2VzIG9ubHkgNSBpdGVyYXRpb25zIHdpdGggMTYgbXMgc2xlZXAgYmV0d2Vl bi4gTWF5YmUKPiBTbGVlcCgpIGlzIGltcGxlbWVudGVkIGRpZmZlcmVudGx5IGluIFdpbmRvd3M/ IEkgbWVhbiBMaW51eCBtYXkgYmUKPiAiZmFzdGVyIiBoZXJlIGFuZCByZXR1cm4gcHJlbWF0dXJl bHkgYW5kIGlmIHdlIGxlYXZlIHRoZSBwb3J0IGludG8gRDAKPiB0aGlzIGRvZXMgbm90IGhhcHBl biwgb3Igc29tZXRoaW5nLiBJJ20ganVzdCB0aHJvd2luZyBvdXQgaWRlYXMgOikKCkJ1dCB0aGlz IGFjdHVhbGx5IHdvcmtzIGZvciB0aGUgZG93bnN0cmVhbSBjb21wb25lbnQgaW4gRDAsIGRvZXNu J3QgaXQ/CgpBbHNvLCBpZiB0aGUgZG93bnN0cmVhbSBjb21wb25lbnQgaXMgaW4gRDAsIHRoZSBw b3J0IGFjdHVhbGx5IHNob3VsZApzdGF5IGluIEQwIHRvbywgc28gd2hhdCB3b3VsZCBoYXBwZW4g d2l0aCB0aGUgJHN1YmplY3QgcGF0Y2ggYXBwbGllZD8KX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX18KTm91dmVhdSBtYWlsaW5nIGxpc3QKTm91dmVhdUBsaXN0 cy5mcmVlZGVza3RvcC5vcmcKaHR0cHM6Ly9saXN0cy5mcmVlZGVza3RvcC5vcmcvbWFpbG1hbi9s aXN0aW5mby9ub3V2ZWF1 From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.0 required=3.0 tests=MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4D359C432C3 for ; Thu, 21 Nov 2019 15:43:39 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 2A59D2071C for ; Thu, 21 Nov 2019 15:43:39 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2A59D2071C Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id B72856F38F; Thu, 21 Nov 2019 15:43:37 +0000 (UTC) Received: from mail-ot1-f68.google.com (mail-ot1-f68.google.com [209.85.210.68]) by gabe.freedesktop.org (Postfix) with ESMTPS id AD05E6E1AA; Thu, 21 Nov 2019 15:43:36 +0000 (UTC) Received: by mail-ot1-f68.google.com with SMTP id 23so1530900otf.2; Thu, 21 Nov 2019 07:43:36 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=O6BtIcSR6Sz2/7y/lGE3fC0q1eR/CKJYxQr5CTA/9Uc=; b=WM2p9Ep9EWTXySvBbSi03+XTvXd8tis2W0ysfLth5+f7IAVCv1YiQONt6QpoU3xCon Opx/YdBCXiqbSStrNzpQyL0b1aoEfYnVzqEQe4lwDqyF8sx4xQQ3aEwBa5jxwJwZSsMY wMbnvN3+c2CZBGQXKz2tQbz2XEiVzKIHFuTvNeP+4yElGn52qdX84hYYNNBZ0JhUwA0q nbFrVxtUZc50X1l8mZXCZDwxpTxFDAGLvCCxO+kaBJpu3UD3zszhQG555XjHA+SmojWv Wi/N/tz/hqtPn+T4WV3xffjdYLRCGgQi8Db94wcHT027glaC26TyljDQD50O2It2r1pR qrYg== X-Gm-Message-State: APjAAAWF46+5JEsjapLKQz6XDM7hvstQQW8E8krG2JWMuD1ELufMQswO NQiMBBzcRb5+YflixIZJBPa5DP2VxKjq8T6UTxA= X-Google-Smtp-Source: APXvYqxszDM90W/tRIlxbpBS9iYycKQwHojBugtimrAfg57FfxPfoUZyiQ/a8iP2L0RYZnuBEcgjDJESRlnjbPGrO6M= X-Received: by 2002:a05:6830:232a:: with SMTP id q10mr7175465otg.262.1574351015588; Thu, 21 Nov 2019 07:43:35 -0800 (PST) MIME-Version: 1.0 References: <20191120151542.GH11621@lahna.fi.intel.com> <20191120155301.GL11621@lahna.fi.intel.com> <20191121112821.GU11621@lahna.fi.intel.com> <20191121114610.GW11621@lahna.fi.intel.com> <20191121125236.GX11621@lahna.fi.intel.com> In-Reply-To: <20191121125236.GX11621@lahna.fi.intel.com> From: "Rafael J. Wysocki" Date: Thu, 21 Nov 2019 16:43:24 +0100 Message-ID: Subject: Re: [PATCH v4] pci: prevent putting nvidia GPUs into lower device states on certain intel bridges To: Mika Westerberg X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Karol Herbst , "Rafael J. Wysocki" , Linux PCI , Linux PM , "Rafael J . Wysocki" , LKML , dri-devel , Mario Limonciello , Bjorn Helgaas , nouveau Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Message-ID: <20191121154324.ydaSWL-WFi_FjOElwXO7UwZ_pmYljXteRxERLPgM7rs@z> T24gVGh1LCBOb3YgMjEsIDIwMTkgYXQgMTo1MiBQTSBNaWthIFdlc3RlcmJlcmcKPG1pa2Eud2Vz dGVyYmVyZ0BpbnRlbC5jb20+IHdyb3RlOgo+Cj4gT24gVGh1LCBOb3YgMjEsIDIwMTkgYXQgMDE6 NDY6MTRQTSArMDIwMCwgTWlrYSBXZXN0ZXJiZXJnIHdyb3RlOgo+ID4gT24gVGh1LCBOb3YgMjEs IDIwMTkgYXQgMTI6MzQ6MjJQTSArMDEwMCwgUmFmYWVsIEouIFd5c29ja2kgd3JvdGU6Cj4gPiA+ IE9uIFRodSwgTm92IDIxLCAyMDE5IGF0IDEyOjI4IFBNIE1pa2EgV2VzdGVyYmVyZwo+ID4gPiA8 bWlrYS53ZXN0ZXJiZXJnQGludGVsLmNvbT4gd3JvdGU6Cj4gPiA+ID4KPiA+ID4gPiBPbiBXZWQs IE5vdiAyMCwgMjAxOSBhdCAxMToyOTozM1BNICswMTAwLCBSYWZhZWwgSi4gV3lzb2NraSB3cm90 ZToKPiA+ID4gPiA+ID4gbGFzdCB3ZWVrIG9yIHNvIEkgZm91bmQgc3lzdGVtcyB3aGVyZSB0aGUg R1BVIHdhcyB1bmRlciB0aGUgIlBDSQo+ID4gPiA+ID4gPiBFeHByZXNzIFJvb3QgUG9ydCIgKG5h bWUgZnJvbSBsc3BjaSkgYW5kIG9uIHRob3NlIHN5c3RlbXMgYWxsIG9mIHRoYXQKPiA+ID4gPiA+ ID4gc2VlbXMgdG8gd29yay4gU28gSSBhbSB3b25kZXJpbmcgaWYgaXQncyBpbmRlZWQganVzdCB0 aGUgMHgxOTAxIG9uZSwKPiA+ID4gPiA+ID4gd2hpY2ggYWxzbyBleHBsYWlucyBNaWthcyBjYXNl IHRoYXQgVGh1bmRlcmJvbHQgc3R1ZmYgd29ya3MgYXMgZGV2aWNlcwo+ID4gPiA+ID4gPiBuZXZl ciBnZXQgcG9wdWxhdGVkIHVuZGVyIHRoaXMgcGFydGljdWxhciBicmlkZ2UgY29udHJvbGxlciwg YnV0IHVuZGVyCj4gPiA+ID4gPiA+IHRob3NlICJSb290IFBvcnQicwo+ID4gPiA+ID4KPiA+ID4g PiA+IEl0IGFsd2F5cyBpcyBhIFBDSWUgcG9ydCwgYnV0IGl0cyBsb2NhdGlvbiB3aXRoaW4gdGhl IFNvQyBtYXkgbWF0dGVyLgo+ID4gPiA+Cj4gPiA+ID4gRXhhY3RseS4gSW50ZWwgaGFyZHdhcmUg aGFzIFBDSWUgcG9ydHMgb24gQ1BVIHNpZGUgKHRoZXNlIGFyZSBjYWxsZWQKPiA+ID4gPiBQRUcs IFBDSSBFeHByZXNzIEdyYXBoaWNzLCBwb3J0cyksIGFuZCB0aGUgUENIIHNpZGUuIEkgdGhpbmsg dGhlIElQIGlzCj4gPiA+ID4gc3RpbGwgdGhlIHNhbWUuCj4gPiA+ID4KPiA+ID4gPiA+IEFsc28g c29tZSBjdXN0b20gQU1MLWJhc2VkIHBvd2VyIG1hbmFnZW1lbnQgaXMgaW52b2x2ZWQgYW5kIHRo YXQgbWF5Cj4gPiA+ID4gPiBiZSBtYWtpbmcgc3BlY2lmaWMgYXNzdW1wdGlvbnMgb24gdGhlIGNv bmZpZ3VyYXRpb24gb2YgdGhlIFNvQyBhbmQgdGhlCj4gPiA+ID4gPiBHUFUgYXQgdGhlIHRpbWUg b2YgaXRzIGludm9jYXRpb24gd2hpY2ggdW5mb3J0dW5hdGVseSBhcmUgbm90IGtub3duIHRvCj4g PiA+ID4gPiB1cy4KPiA+ID4gPiA+Cj4gPiA+ID4gPiBIb3dldmVyLCBpdCBsb29rcyBsaWtlIHRo ZSBBTUwgaW52b2tlZCB0byBwb3dlciBkb3duIHRoZSBHUFUgZnJvbQo+ID4gPiA+ID4gYWNwaV9w Y2lfc2V0X3Bvd2VyX3N0YXRlKCkgZ2V0cyBjb25mdXNlZCBpZiBpdCBpcyBub3QgaW4gUENJIEQw IGF0Cj4gPiA+ID4gPiB0aGF0IHBvaW50LCBzbyBpdCBsb29rcyBsaWtlIHRoYXQgQU1MIHRyaWVz IHRvIGFjY2VzcyBkZXZpY2UgbWVtb3J5IG9uCj4gPiA+ID4gPiB0aGUgR1BVIChiZXlvbmQgdGhl IFBDSSBjb25maWcgc3BhY2UpIG9yIHNpbWlsYXIgd2hpY2ggaXMgbm90Cj4gPiA+ID4gPiBhY2Nl c3NpYmxlIGluIFBDSSBwb3dlciBzdGF0ZXMgYmVsb3cgRDAuCj4gPiA+ID4KPiA+ID4gPiBPciB0 aGUgUENJIGNvbmZpZyBzcGFjZSBvZiB0aGUgR1BVIHdoZW4gdGhlIHBhcmVudCByb290IHBvcnQg aXMgaW4gRDNob3QKPiA+ID4gPiAoYXMgaXQgaXMgdGhlIGNhc2UgaGVyZSkuIEFsc28gdGhlbiB0 aGUgR1BVIGNvbmZpZyBzcGFjZSBpcyBub3QKPiA+ID4gPiBhY2Nlc3NpYmxlLgo+ID4gPgo+ID4g PiBXaHkgd291bGQgdGhlIHBhcmVudCBwb3J0IGJlIGluIEQzaG90IGF0IHRoYXQgcG9pbnQ/ICBX b3VsZG4ndCB0aGF0IGJlCj4gPiA+IGEgc3VzcGVuZCBvcmRlcmluZyB2aW9sYXRpb24/Cj4gPgo+ ID4gTm8uIFdlIHB1dCB0aGUgR1BVIGludG8gRDNob3QgZmlyc3QsCgpPSwoKRG9lcyB0aGlzIGlu dm9sdmUgYW55IEFNTCwgbGlrZSBhIF9QUzMgdW5kZXIgdGhlIEdQVSBvYmplY3Q/Cgo+ID4gdGhl biB0aGUgcm9vdCBwb3J0IGFuZCB0aGVuIHR1cm4KPiA+IG9mZiB0aGUgcG93ZXIgcmVzb3VyY2Ug KHdoaWNoIGlzIGF0dGFjaGVkIHRvIHRoZSByb290IHBvcnQpIHJlc3VsdGluZwo+ID4gdGhlIHRv cG9sb2d5IGVudGVyaW5nIEQzY29sZC4KPgo+IEkgZG9uJ3Qgc2VlIHRoYXQgaGFwcGVuaW5nIGlu IHRoZSBBTUwgdGhvdWdoLgoKV2hpY2ggQU1MIGRvIHlvdSBtZWFuLCBzcGVjaWZpY2FsbHk/ICBU aGUgX09GRiBtZXRob2QgZm9yIHRoZSByb290CnBvcnQncyBfUFIzIHBvd2VyIHJlc291cmNlIG9y IHNvbWV0aGluZyBlbHNlPwoKPiBCYXNpY2FsbHkgdGhlIGRpZmZlcmVuY2UgaXMgdGhhdCB3aGVu IFdpbmRvd3MgNyBvciBMaW51eCAodGhlIF9SRVY9PTUKPiBjaGVjaykgdGhlbiB3ZSBkaXJlY3Rs eSBkbyBsaW5rIGRpc2FibGUgd2hlcmVhcyBpbiBXaW5kb3dzIDgrIHdlIGludm9rZQo+IExLRFMo KSBtZXRob2QgdGhhdCBwdXRzIHRoZSBsaW5rIGludG8gTDIvTDMuIE5vbmUgb2YgdGhlIGZpZWxk cyB0aGV5Cj4gYWNjZXNzIHNlZW0gdG8gdG91Y2ggdGhlIEdQVSBpdHNlbGYuCgpTbyB0aGF0IG1h eSBiZSB3aGVyZSB0aGUgcHJvYmxlbSBpcy4KClB1dHRpbmcgdGhlIGRvd25zdHJlYW0gY29tcG9u ZW50IGludG8gUENJIERbMS0zXSBpcyBleHBlY3RlZCB0byBwdXQKdGhlIGxpbmsgaW50byBMMSwg c28gSSdtIG5vdCBzdXJlIGhvdyB0aGF0IHBsYXlzIHdpdGggdGhlIGxhdGVyCmF0dGVtcHQgdG8g cHV0IGl0IGludG8gTDIvTDMgUmVhZHkuCgpBbHNvLCBMMi9MMyBSZWFkeSBpcyBleHBlY3RlZCB0 byBiZSB0cmFuc2llbnQsIHNvIGZpbmFsbHkgcG93ZXIgc2hvdWxkCmJlIHJlbW92ZWQgc29tZWhv dy4KCj4gTEtEUygpIGZvciB0aGUgZmlyc3QgUEVHIHBvcnQgbG9va3MgbGlrZSB0aGlzOgo+Cj4g ICAgUDBMMiA9IE9uZQo+ICAgIFNsZWVwICgweDEwKQo+ICAgIExvY2FsMCA9IFplcm8KPiAgICBX aGlsZSAoUDBMMikKPiAgICB7Cj4gICAgICAgICBJZiAoKExvY2FsMCA+IDB4MDQpKQo+ICAgICAg ICAgewo+ICAgICAgICAgICAgIEJyZWFrCj4gICAgICAgICB9Cj4KPiAgICAgICAgIFNsZWVwICgw eDEwKQo+ICAgICAgICAgTG9jYWwwKysKPiAgICB9Cj4KPiBPbmUgdGhpbmcgdGhhdCBjb21lcyB0 byBtaW5kIGlzIHRoYXQgdGhlIGxvb3AgY2FuIGVuZCBldmVuIGlmIFAwTDIgaXMKPiBub3QgY2xl YXJlZCBhcyBpdCBkb2VzIG9ubHkgNSBpdGVyYXRpb25zIHdpdGggMTYgbXMgc2xlZXAgYmV0d2Vl bi4gTWF5YmUKPiBTbGVlcCgpIGlzIGltcGxlbWVudGVkIGRpZmZlcmVudGx5IGluIFdpbmRvd3M/ IEkgbWVhbiBMaW51eCBtYXkgYmUKPiAiZmFzdGVyIiBoZXJlIGFuZCByZXR1cm4gcHJlbWF0dXJl bHkgYW5kIGlmIHdlIGxlYXZlIHRoZSBwb3J0IGludG8gRDAKPiB0aGlzIGRvZXMgbm90IGhhcHBl biwgb3Igc29tZXRoaW5nLiBJJ20ganVzdCB0aHJvd2luZyBvdXQgaWRlYXMgOikKCkJ1dCB0aGlz IGFjdHVhbGx5IHdvcmtzIGZvciB0aGUgZG93bnN0cmVhbSBjb21wb25lbnQgaW4gRDAsIGRvZXNu J3QgaXQ/CgpBbHNvLCBpZiB0aGUgZG93bnN0cmVhbSBjb21wb25lbnQgaXMgaW4gRDAsIHRoZSBw b3J0IGFjdHVhbGx5IHNob3VsZApzdGF5IGluIEQwIHRvbywgc28gd2hhdCB3b3VsZCBoYXBwZW4g d2l0aCB0aGUgJHN1YmplY3QgcGF0Y2ggYXBwbGllZD8KX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX18KZHJpLWRldmVsIG1haWxpbmcgbGlzdApkcmktZGV2ZWxA bGlzdHMuZnJlZWRlc2t0b3Aub3JnCmh0dHBzOi8vbGlzdHMuZnJlZWRlc2t0b3Aub3JnL21haWxt YW4vbGlzdGluZm8vZHJpLWRldmVs