From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.8 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E0351C432C0 for ; Wed, 20 Nov 2019 15:37:35 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id AEE0520709 for ; Wed, 20 Nov 2019 15:37:35 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="MCM6LDn3" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729156AbfKTPhe (ORCPT ); Wed, 20 Nov 2019 10:37:34 -0500 Received: from us-smtp-2.mimecast.com ([205.139.110.61]:39584 "EHLO us-smtp-delivery-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1728445AbfKTPhb (ORCPT ); Wed, 20 Nov 2019 10:37:31 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1574264249; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/C9gUt9cyA76AzLuivaEWyzrhcrqWJ0lBzJ30P+LXYs=; b=MCM6LDn32Q4K+NrSFNN2985PKABcP3GJjhJKWil5gIxAFLAKvYmcazSxpXEXlnjG0dls/6 bL9SUGO46RRfgREq4+SFLGPd0Y/WeAkqSLPCiz8PbVUTSuY9eOiDNuier06wrNyibzCzD+ kjA4RBTohe/Q0fVrQjIITsxRONwfBY4= Received: from mail-qt1-f200.google.com (mail-qt1-f200.google.com [209.85.160.200]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-276-iVvNkIJQM3K309Al5z9gbw-1; Wed, 20 Nov 2019 10:37:26 -0500 Received: by mail-qt1-f200.google.com with SMTP id g5so104442qtc.5 for ; Wed, 20 Nov 2019 07:37:26 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=BksgvfivWT5oEKOsVtU2jjsSwTJ8rcltyxFUj+I5xNY=; b=f+J4UjHlEXwHGe8BtE0Cg5uYfP/65j2+h24qfXznAMatOKC9f8Q6ghqTen+J2TmwAp SIINYwQHjZkpRSpvYUM3hueNoAhuVMLYCkZZv8eofQrwcOgzePyGWJqCPjMehdHbt2Bk +7H/3Fc6DLmPvFj/zZOcUzzl00f+x5dOjf0h3hbsUnhbZG9GIQiHI6VJWzhC40JvWnT9 tt7YvRh56QEdmaCEMAGdzQ9TdIkI4pHgaeUgW8fO9TUON7Xf7Ua48AqdXr5KoORfjfAz v0l5YUiidjiCFjtNajGOyhs3tK/rinHrCf5moPtyNBjp9wrjHuwQvWSmJFzsXuynGDdE F+mw== X-Gm-Message-State: APjAAAXrXTcFfw4Nm5apcD60dfgUgdZV9ro44rqK9syDpD3DBUolILtr 1j3+5T/GMC8UhweOVPcsNNmbfwtlPGSlI2VcVywTmNLnyuHTNeWD4wlRXEutmLKt/b2880N9pOt Jresxcthl0WgfGE24PAKhRrpaw6cwgo8JsG7I8qQE X-Received: by 2002:a37:6811:: with SMTP id d17mr3082060qkc.102.1574264245968; Wed, 20 Nov 2019 07:37:25 -0800 (PST) X-Google-Smtp-Source: APXvYqzt0K/ijAMXYP756+k75llJT5zszZ47OfQZKFovlD4ZwaMaT169d/nRHiPSAwI9CqrdON3hSsUsT8pA85i0bCE= X-Received: by 2002:a37:6811:: with SMTP id d17mr3082027qkc.102.1574264245623; Wed, 20 Nov 2019 07:37:25 -0800 (PST) MIME-Version: 1.0 References: <20191119214955.GA223696@google.com> <20191120101816.GX11621@lahna.fi.intel.com> <20191120112212.GA11621@lahna.fi.intel.com> <20191120115127.GD11621@lahna.fi.intel.com> <20191120120913.GE11621@lahna.fi.intel.com> <20191120151542.GH11621@lahna.fi.intel.com> In-Reply-To: <20191120151542.GH11621@lahna.fi.intel.com> From: Karol Herbst Date: Wed, 20 Nov 2019 16:37:14 +0100 Message-ID: Subject: Re: [PATCH v4] pci: prevent putting nvidia GPUs into lower device states on certain intel bridges To: Mika Westerberg Cc: "Rafael J. Wysocki" , Bjorn Helgaas , LKML , Lyude Paul , "Rafael J . Wysocki" , Linux PCI , Linux PM , dri-devel , nouveau , Dave Airlie , Mario Limonciello X-MC-Unique: iVvNkIJQM3K309Al5z9gbw-1 X-Mimecast-Spam-Score: 0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, Nov 20, 2019 at 4:15 PM Mika Westerberg wrote: > > On Wed, Nov 20, 2019 at 01:11:52PM +0100, Karol Herbst wrote: > > On Wed, Nov 20, 2019 at 1:09 PM Mika Westerberg > > wrote: > > > > > > On Wed, Nov 20, 2019 at 12:58:00PM +0100, Karol Herbst wrote: > > > > overall, what I really want to know is, _why_ does it work on windo= ws? > > > > > > So do I ;-) > > > > > > > Or what are we doing differently on Linux so that it doesn't work? = If > > > > anybody has any idea on how we could dig into this and figure it ou= t > > > > on this level, this would probably allow us to get closer to the ro= ot > > > > cause? no? > > > > > > Have you tried to use the acpi_rev_override parameter in your system = and > > > does it have any effect? > > > > > > Also did you try to trace the ACPI _ON/_OFF() methods? I think that > > > should hopefully reveal something. > > > > > > > I think I did in the past and it seemed to have worked, there is just > > one big issue with this: it's a Dell specific workaround afaik, and > > this issue plagues not just Dell, but we've seen it on HP and Lenovo > > laptops as well, and I've heard about users having the same issues on > > Asus and MSI laptops as well. > > Maybe it is not a workaround at all but instead it simply determines > whether the system supports RTD3 or something like that (IIRC Windows 8 > started supporting it). Maybe Dell added check for Linux because at that > time Linux did not support it. > the point is, it's not checking it by default, so by default you still run into the windows 8 codepath. > In case RTD3 is supported it invokes LKDS() which probably does the L2 > or L3 entry and this is for some reason does not work the same way in > Linux than it does with Windows 8+. > > I don't remember if this happens only with nouveau or with the > proprietary driver as well but looking at the nouveau runtime PM suspend > hook (assuming I'm looking at the correct code): > > static int > nouveau_pmops_runtime_suspend(struct device *dev) > { > struct pci_dev *pdev =3D to_pci_dev(dev); > struct drm_device *drm_dev =3D pci_get_drvdata(pdev); > int ret; > > if (!nouveau_pmops_runtime()) { > pm_runtime_forbid(dev); > return -EBUSY; > } > > nouveau_switcheroo_optimus_dsm(); > ret =3D nouveau_do_suspend(drm_dev, true); > pci_save_state(pdev); > pci_disable_device(pdev); > pci_ignore_hotplug(pdev); > pci_set_power_state(pdev, PCI_D3cold); > drm_dev->switch_power_state =3D DRM_SWITCH_POWER_DYNAMIC_OFF; > return ret; > } > > Normally PCI drivers leave the PCI bus PM things to PCI core but here > the driver does these. So I wonder if it makes any difference if we let > the core handle all that: > > static int > nouveau_pmops_runtime_suspend(struct device *dev) > { > struct pci_dev *pdev =3D to_pci_dev(dev); > struct drm_device *drm_dev =3D pci_get_drvdata(pdev); > int ret; > > if (!nouveau_pmops_runtime()) { > pm_runtime_forbid(dev); > return -EBUSY; > } > > nouveau_switcheroo_optimus_dsm(); > ret =3D nouveau_do_suspend(drm_dev, true); > pci_ignore_hotplug(pdev); > drm_dev->switch_power_state =3D DRM_SWITCH_POWER_DYNAMIC_OFF; > return ret; > } > > and similar for the nouveau_pmops_runtime_resume(). > yeah, I tried that at some point and it didn't help either. The reason we call those from inside Nouveau is to support systems pre _PR where nouveau invokes custom _DSM calls on its own. We could potentially check for that though. From mboxrd@z Thu Jan 1 00:00:00 1970 From: Karol Herbst Subject: Re: [PATCH v4] pci: prevent putting nvidia GPUs into lower device states on certain intel bridges Date: Wed, 20 Nov 2019 16:37:14 +0100 Message-ID: References: <20191119214955.GA223696@google.com> <20191120101816.GX11621@lahna.fi.intel.com> <20191120112212.GA11621@lahna.fi.intel.com> <20191120115127.GD11621@lahna.fi.intel.com> <20191120120913.GE11621@lahna.fi.intel.com> <20191120151542.GH11621@lahna.fi.intel.com> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Return-path: In-Reply-To: <20191120151542.GH11621@lahna.fi.intel.com> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" To: Mika Westerberg Cc: "Rafael J. Wysocki" , Linux PCI , Linux PM , "Rafael J . Wysocki" , LKML , dri-devel , Mario Limonciello , Bjorn Helgaas , nouveau List-Id: nouveau.vger.kernel.org T24gV2VkLCBOb3YgMjAsIDIwMTkgYXQgNDoxNSBQTSBNaWthIFdlc3RlcmJlcmcKPG1pa2Eud2Vz dGVyYmVyZ0BpbnRlbC5jb20+IHdyb3RlOgo+Cj4gT24gV2VkLCBOb3YgMjAsIDIwMTkgYXQgMDE6 MTE6NTJQTSArMDEwMCwgS2Fyb2wgSGVyYnN0IHdyb3RlOgo+ID4gT24gV2VkLCBOb3YgMjAsIDIw MTkgYXQgMTowOSBQTSBNaWthIFdlc3RlcmJlcmcKPiA+IDxtaWthLndlc3RlcmJlcmdAaW50ZWwu Y29tPiB3cm90ZToKPiA+ID4KPiA+ID4gT24gV2VkLCBOb3YgMjAsIDIwMTkgYXQgMTI6NTg6MDBQ TSArMDEwMCwgS2Fyb2wgSGVyYnN0IHdyb3RlOgo+ID4gPiA+IG92ZXJhbGwsIHdoYXQgSSByZWFs bHkgd2FudCB0byBrbm93IGlzLCBfd2h5XyBkb2VzIGl0IHdvcmsgb24gd2luZG93cz8KPiA+ID4K PiA+ID4gU28gZG8gSSA7LSkKPiA+ID4KPiA+ID4gPiBPciB3aGF0IGFyZSB3ZSBkb2luZyBkaWZm ZXJlbnRseSBvbiBMaW51eCBzbyB0aGF0IGl0IGRvZXNuJ3Qgd29yaz8gSWYKPiA+ID4gPiBhbnli b2R5IGhhcyBhbnkgaWRlYSBvbiBob3cgd2UgY291bGQgZGlnIGludG8gdGhpcyBhbmQgZmlndXJl IGl0IG91dAo+ID4gPiA+IG9uIHRoaXMgbGV2ZWwsIHRoaXMgd291bGQgcHJvYmFibHkgYWxsb3cg dXMgdG8gZ2V0IGNsb3NlciB0byB0aGUgcm9vdAo+ID4gPiA+IGNhdXNlPyBubz8KPiA+ID4KPiA+ ID4gSGF2ZSB5b3UgdHJpZWQgdG8gdXNlIHRoZSBhY3BpX3Jldl9vdmVycmlkZSBwYXJhbWV0ZXIg aW4geW91ciBzeXN0ZW0gYW5kCj4gPiA+IGRvZXMgaXQgaGF2ZSBhbnkgZWZmZWN0Pwo+ID4gPgo+ ID4gPiBBbHNvIGRpZCB5b3UgdHJ5IHRvIHRyYWNlIHRoZSBBQ1BJIF9PTi9fT0ZGKCkgbWV0aG9k cz8gSSB0aGluayB0aGF0Cj4gPiA+IHNob3VsZCBob3BlZnVsbHkgcmV2ZWFsIHNvbWV0aGluZy4K PiA+ID4KPiA+Cj4gPiBJIHRoaW5rIEkgZGlkIGluIHRoZSBwYXN0IGFuZCBpdCBzZWVtZWQgdG8g aGF2ZSB3b3JrZWQsIHRoZXJlIGlzIGp1c3QKPiA+IG9uZSBiaWcgaXNzdWUgd2l0aCB0aGlzOiBp dCdzIGEgRGVsbCBzcGVjaWZpYyB3b3JrYXJvdW5kIGFmYWlrLCBhbmQKPiA+IHRoaXMgaXNzdWUg cGxhZ3VlcyBub3QganVzdCBEZWxsLCBidXQgd2UndmUgc2VlbiBpdCBvbiBIUCBhbmQgTGVub3Zv Cj4gPiBsYXB0b3BzIGFzIHdlbGwsIGFuZCBJJ3ZlIGhlYXJkIGFib3V0IHVzZXJzIGhhdmluZyB0 aGUgc2FtZSBpc3N1ZXMgb24KPiA+IEFzdXMgYW5kIE1TSSBsYXB0b3BzIGFzIHdlbGwuCj4KPiBN YXliZSBpdCBpcyBub3QgYSB3b3JrYXJvdW5kIGF0IGFsbCBidXQgaW5zdGVhZCBpdCBzaW1wbHkg ZGV0ZXJtaW5lcwo+IHdoZXRoZXIgdGhlIHN5c3RlbSBzdXBwb3J0cyBSVEQzIG9yIHNvbWV0aGlu ZyBsaWtlIHRoYXQgKElJUkMgV2luZG93cyA4Cj4gc3RhcnRlZCBzdXBwb3J0aW5nIGl0KS4gTWF5 YmUgRGVsbCBhZGRlZCBjaGVjayBmb3IgTGludXggYmVjYXVzZSBhdCB0aGF0Cj4gdGltZSBMaW51 eCBkaWQgbm90IHN1cHBvcnQgaXQuCj4KCnRoZSBwb2ludCBpcywgaXQncyBub3QgY2hlY2tpbmcg aXQgYnkgZGVmYXVsdCwgc28gYnkgZGVmYXVsdCB5b3Ugc3RpbGwKcnVuIGludG8gdGhlIHdpbmRv d3MgOCBjb2RlcGF0aC4KCj4gSW4gY2FzZSBSVEQzIGlzIHN1cHBvcnRlZCBpdCBpbnZva2VzIExL RFMoKSB3aGljaCBwcm9iYWJseSBkb2VzIHRoZSBMMgo+IG9yIEwzIGVudHJ5IGFuZCB0aGlzIGlz IGZvciBzb21lIHJlYXNvbiBkb2VzIG5vdCB3b3JrIHRoZSBzYW1lIHdheSBpbgo+IExpbnV4IHRo YW4gaXQgZG9lcyB3aXRoIFdpbmRvd3MgOCsuCj4KPiBJIGRvbid0IHJlbWVtYmVyIGlmIHRoaXMg aGFwcGVucyBvbmx5IHdpdGggbm91dmVhdSBvciB3aXRoIHRoZQo+IHByb3ByaWV0YXJ5IGRyaXZl ciBhcyB3ZWxsIGJ1dCBsb29raW5nIGF0IHRoZSBub3V2ZWF1IHJ1bnRpbWUgUE0gc3VzcGVuZAo+ IGhvb2sgKGFzc3VtaW5nIEknbSBsb29raW5nIGF0IHRoZSBjb3JyZWN0IGNvZGUpOgo+Cj4gc3Rh dGljIGludAo+IG5vdXZlYXVfcG1vcHNfcnVudGltZV9zdXNwZW5kKHN0cnVjdCBkZXZpY2UgKmRl dikKPiB7Cj4gICAgICAgICBzdHJ1Y3QgcGNpX2RldiAqcGRldiA9IHRvX3BjaV9kZXYoZGV2KTsK PiAgICAgICAgIHN0cnVjdCBkcm1fZGV2aWNlICpkcm1fZGV2ID0gcGNpX2dldF9kcnZkYXRhKHBk ZXYpOwo+ICAgICAgICAgaW50IHJldDsKPgo+ICAgICAgICAgaWYgKCFub3V2ZWF1X3Btb3BzX3J1 bnRpbWUoKSkgewo+ICAgICAgICAgICAgICAgICBwbV9ydW50aW1lX2ZvcmJpZChkZXYpOwo+ICAg ICAgICAgICAgICAgICByZXR1cm4gLUVCVVNZOwo+ICAgICAgICAgfQo+Cj4gICAgICAgICBub3V2 ZWF1X3N3aXRjaGVyb29fb3B0aW11c19kc20oKTsKPiAgICAgICAgIHJldCA9IG5vdXZlYXVfZG9f c3VzcGVuZChkcm1fZGV2LCB0cnVlKTsKPiAgICAgICAgIHBjaV9zYXZlX3N0YXRlKHBkZXYpOwo+ ICAgICAgICAgcGNpX2Rpc2FibGVfZGV2aWNlKHBkZXYpOwo+ICAgICAgICAgcGNpX2lnbm9yZV9o b3RwbHVnKHBkZXYpOwo+ICAgICAgICAgcGNpX3NldF9wb3dlcl9zdGF0ZShwZGV2LCBQQ0lfRDNj b2xkKTsKPiAgICAgICAgIGRybV9kZXYtPnN3aXRjaF9wb3dlcl9zdGF0ZSA9IERSTV9TV0lUQ0hf UE9XRVJfRFlOQU1JQ19PRkY7Cj4gICAgICAgICByZXR1cm4gcmV0Owo+IH0KPgo+IE5vcm1hbGx5 IFBDSSBkcml2ZXJzIGxlYXZlIHRoZSBQQ0kgYnVzIFBNIHRoaW5ncyB0byBQQ0kgY29yZSBidXQg aGVyZQo+IHRoZSBkcml2ZXIgZG9lcyB0aGVzZS4gU28gSSB3b25kZXIgaWYgaXQgbWFrZXMgYW55 IGRpZmZlcmVuY2UgaWYgd2UgbGV0Cj4gdGhlIGNvcmUgaGFuZGxlIGFsbCB0aGF0Ogo+Cj4gc3Rh dGljIGludAo+IG5vdXZlYXVfcG1vcHNfcnVudGltZV9zdXNwZW5kKHN0cnVjdCBkZXZpY2UgKmRl dikKPiB7Cj4gICAgICAgICBzdHJ1Y3QgcGNpX2RldiAqcGRldiA9IHRvX3BjaV9kZXYoZGV2KTsK PiAgICAgICAgIHN0cnVjdCBkcm1fZGV2aWNlICpkcm1fZGV2ID0gcGNpX2dldF9kcnZkYXRhKHBk ZXYpOwo+ICAgICAgICAgaW50IHJldDsKPgo+ICAgICAgICAgaWYgKCFub3V2ZWF1X3Btb3BzX3J1 bnRpbWUoKSkgewo+ICAgICAgICAgICAgICAgICBwbV9ydW50aW1lX2ZvcmJpZChkZXYpOwo+ICAg ICAgICAgICAgICAgICByZXR1cm4gLUVCVVNZOwo+ICAgICAgICAgfQo+Cj4gICAgICAgICBub3V2 ZWF1X3N3aXRjaGVyb29fb3B0aW11c19kc20oKTsKPiAgICAgICAgIHJldCA9IG5vdXZlYXVfZG9f c3VzcGVuZChkcm1fZGV2LCB0cnVlKTsKPiAgICAgICAgIHBjaV9pZ25vcmVfaG90cGx1ZyhwZGV2 KTsKPiAgICAgICAgIGRybV9kZXYtPnN3aXRjaF9wb3dlcl9zdGF0ZSA9IERSTV9TV0lUQ0hfUE9X RVJfRFlOQU1JQ19PRkY7Cj4gICAgICAgICByZXR1cm4gcmV0Owo+IH0KPgo+IGFuZCBzaW1pbGFy IGZvciB0aGUgbm91dmVhdV9wbW9wc19ydW50aW1lX3Jlc3VtZSgpLgo+Cgp5ZWFoLCBJIHRyaWVk IHRoYXQgYXQgc29tZSBwb2ludCBhbmQgaXQgZGlkbid0IGhlbHAgZWl0aGVyLiBUaGUgcmVhc29u CndlIGNhbGwgdGhvc2UgZnJvbSBpbnNpZGUgTm91dmVhdSBpcyB0byBzdXBwb3J0IHN5c3RlbXMg cHJlIF9QUiB3aGVyZQpub3V2ZWF1IGludm9rZXMgY3VzdG9tIF9EU00gY2FsbHMgb24gaXRzIG93 bi4gV2UgY291bGQgcG90ZW50aWFsbHkKY2hlY2sgZm9yIHRoYXQgdGhvdWdoLgoKX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KZHJpLWRldmVsIG1haWxpbmcg bGlzdApkcmktZGV2ZWxAbGlzdHMuZnJlZWRlc2t0b3Aub3JnCmh0dHBzOi8vbGlzdHMuZnJlZWRl c2t0b3Aub3JnL21haWxtYW4vbGlzdGluZm8vZHJpLWRldmVs