From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.8 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 83A11C432C3 for ; Wed, 27 Nov 2019 11:51:59 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 5891D2053B for ; Wed, 27 Nov 2019 11:51:59 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="iFRDAi7X" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727017AbfK0Lv6 (ORCPT ); Wed, 27 Nov 2019 06:51:58 -0500 Received: from us-smtp-1.mimecast.com ([207.211.31.81]:52734 "EHLO us-smtp-delivery-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1726556AbfK0Lv6 (ORCPT ); Wed, 27 Nov 2019 06:51:58 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1574855517; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/XhIGMtsi7KjLuMFks8eIn7a7OR+crqiPZAFOU/Qxss=; b=iFRDAi7XhSv2bcOJ+Ffw0LowCrf4yFK4HhrIA73B7bKD8SE1LqZVID+ilrKeX0QLAu0BdY RbvvGZmX4WAEXIG4nXqlq4SFOJDImwK65lEAQrMkOPecvTKLqCXfYT6h3y2i6+QKOUAWf/ GunixtYqp8VOss5xH1bWyY/hOGHle48= Received: from mail-qt1-f198.google.com (mail-qt1-f198.google.com [209.85.160.198]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-39-lyJ-O4l_OLaQEc0p2RyDOg-1; Wed, 27 Nov 2019 06:51:54 -0500 Received: by mail-qt1-f198.google.com with SMTP id x21so14644838qtp.1 for ; Wed, 27 Nov 2019 03:51:54 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=zfYAUmGvWJZJSRiVsjUka5mWwDiEaeMITYpQOX7mwXY=; b=MDcsB/K7LsorhhpG6DHZI04rQra7jCcbXwLZr6kSQ4tPheMsbTDBq3TpHp3yNg6bm+ gsAvSHSxPzZUdWgshB6nFR/5V3NN5+GP4Dr0fGsjjlBY3eCysdXhfen+RPCZ5nw9hvl/ TsDxWYPi0YjPl2DD4hwWJIqEaZygrju4u74bZhSqqIrwBQ7o/V+gZAIywtM3v4mAW/hZ SKsBec64F445BIDpkrqb92XlEbeMPLtQERe7F2DGAJMr9muBFKdfmgTkqi+oRbJ6sfD8 6MaisNMG1c+ZjhwLmAp5q4zEBWurk7X0C9qBeFCBponWbbZ7qxxJGpVHrguIoPmqU5wk VlPg== X-Gm-Message-State: APjAAAU9AssXAVML6Px0Z28AiCvu12gzAHgwcYkiiUaATHtPYQtawrr/ U0YIZZYtCYHt7NVSaASPNoGjPAAoQuWHM+uN4D4aIfNMsBwiXwl/oCT7GUvJ7TF9tFr8SOlyCZN nHSDOfafX0biwoHzlD5C0t11X0KLwwHMHjsEEdW0X X-Received: by 2002:a37:9083:: with SMTP id s125mr3828282qkd.192.1574855513971; Wed, 27 Nov 2019 03:51:53 -0800 (PST) X-Google-Smtp-Source: APXvYqxZYMA+GCGq805VHWy/zJ5Esuj29dPvonXDjlfISqurrPh5cY/HxAxHE6xgDQ+gYrWiq10cFGm+kgXM2WTFJbE= X-Received: by 2002:a37:9083:: with SMTP id s125mr3828257qkd.192.1574855513663; Wed, 27 Nov 2019 03:51:53 -0800 (PST) MIME-Version: 1.0 References: <20191121112821.GU11621@lahna.fi.intel.com> <20191121114610.GW11621@lahna.fi.intel.com> <20191127114856.GZ11621@lahna.fi.intel.com> In-Reply-To: <20191127114856.GZ11621@lahna.fi.intel.com> From: Karol Herbst Date: Wed, 27 Nov 2019 12:51:42 +0100 Message-ID: Subject: Re: [PATCH v4] pci: prevent putting nvidia GPUs into lower device states on certain intel bridges To: Mika Westerberg Cc: Lyude Paul , "Rafael J. Wysocki" , Bjorn Helgaas , LKML , "Rafael J . Wysocki" , Linux PCI , Linux PM , dri-devel , nouveau , Dave Airlie , Mario Limonciello X-MC-Unique: lyJ-O4l_OLaQEc0p2RyDOg-1 X-Mimecast-Spam-Score: 0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, Nov 27, 2019 at 12:49 PM Mika Westerberg wrote: > > On Tue, Nov 26, 2019 at 06:10:36PM -0500, Lyude Paul wrote: > > Hey-this is almost certainly not the right place in this thread to resp= ond, > > but this thread has gotten so deep evolution can't push the subject fur= ther to > > the right, heh. So I'll just respond here. > > :) > > > I've been following this and helping out Karol with testing here and th= ere. > > They had me test Bjorn's PCI branch on the X1 Extreme 2nd generation, w= hich > > has a turing GPU and 8086:1901 PCI bridge. > > > > I was about to say "the patch fixed things, hooray!" but it seems that = after > > trying runtime suspend/resume a couple times things fall apart again: > > You mean $subject patch, no? > no, I told Lyude to test the pci/pm branch as the runpm errors we saw on that machine looked different. Some BAR error the GPU reported after it got resumed, so I was wondering if the delays were helping with that. But after some cycles it still caused the same issue, that the GPU disappeared. Later testing also showed that my patch also didn't seem to help with this error sadly :/ > > [ 686.883247] nouveau 0000:01:00.0: DRM: suspending object tree... > > [ 752.866484] ACPI Error: Aborting method \_SB.PCI0.PEG0.PEGP.NVPO due= to previous error (AE_AML_LOOP_TIMEOUT) (20190816/psparse-529) > > [ 752.866508] ACPI Error: Aborting method \_SB.PCI0.PGON due to previo= us error (AE_AML_LOOP_TIMEOUT) (20190816/psparse-529) > > [ 752.866521] ACPI Error: Aborting method \_SB.PCI0.PEG0.PG00._ON due = to previous error (AE_AML_LOOP_TIMEOUT) (20190816/psparse-529) > > This is probably the culprit. The same AML code fails to properly turn > on the device. > > Is acpidump from this system available somewhere? > From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.8 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0D843C43215 for ; Wed, 27 Nov 2019 11:52:02 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E0C792053B for ; Wed, 27 Nov 2019 11:52:01 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E0C792053B Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 0F5296E528; Wed, 27 Nov 2019 11:52:01 +0000 (UTC) Received: from us-smtp-delivery-1.mimecast.com (us-smtp-2.mimecast.com [205.139.110.61]) by gabe.freedesktop.org (Postfix) with ESMTPS id 218706E526 for ; Wed, 27 Nov 2019 11:51:59 +0000 (UTC) Received: from mail-qt1-f199.google.com (mail-qt1-f199.google.com [209.85.160.199]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-136-C4t1wTtcN8WzE3t1NhBqww-1; Wed, 27 Nov 2019 06:51:54 -0500 Received: by mail-qt1-f199.google.com with SMTP id g13so14571668qtq.16 for ; Wed, 27 Nov 2019 03:51:54 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=zfYAUmGvWJZJSRiVsjUka5mWwDiEaeMITYpQOX7mwXY=; b=dkPykpZnnHUuRDtvR1sMTuEgacAjkIcZc5LRPexJmH9+7tjIjPG8m6lfxH3F5BIzVZ YD+fTWPhD8b24aO9zHNxyKypS4DZHzYzhdasPaw36+byQa4zIMGQMwCSzRXv4dYAv5Mt i+o+mjngJdUSnbBt91q/4LMI/MCpdANsl/rTYMFUFh01/q05A2QPW8CiYMvPTw0gIOY1 5WiUkZ5Ms0Lb/x2WS03EsZbfMIiFVyXRrx/Ec7vCAUSPSxYOkRR95/J1ABsoFDXEh2or IgD8HoW4j7SoRzSH/Dxq1t18qWD8j88dKQUsJ40iBny86xEujz07uCMA4yocCm3BHy/j sA+A== X-Gm-Message-State: APjAAAUDiQHwSKBAiKTxCoM5UvLnCK3660+6Y9OioEfJEduX1e1kO6/B rsACgRfDDGHld/qQRFl53kWP3YdwiMylbOzlLLDrhSYBA9+GYS9UDY/SmY+UtAkkn+oHnwwBZwx 7m9zqbkcsF1zp3iCs5pnbMTUeuXMb60HRMP6kWrlBF+ft X-Received: by 2002:a37:9083:: with SMTP id s125mr3828285qkd.192.1574855513971; Wed, 27 Nov 2019 03:51:53 -0800 (PST) X-Google-Smtp-Source: APXvYqxZYMA+GCGq805VHWy/zJ5Esuj29dPvonXDjlfISqurrPh5cY/HxAxHE6xgDQ+gYrWiq10cFGm+kgXM2WTFJbE= X-Received: by 2002:a37:9083:: with SMTP id s125mr3828257qkd.192.1574855513663; Wed, 27 Nov 2019 03:51:53 -0800 (PST) MIME-Version: 1.0 References: <20191121112821.GU11621@lahna.fi.intel.com> <20191121114610.GW11621@lahna.fi.intel.com> <20191127114856.GZ11621@lahna.fi.intel.com> In-Reply-To: <20191127114856.GZ11621@lahna.fi.intel.com> From: Karol Herbst Date: Wed, 27 Nov 2019 12:51:42 +0100 Message-ID: Subject: Re: [PATCH v4] pci: prevent putting nvidia GPUs into lower device states on certain intel bridges To: Mika Westerberg X-MC-Unique: C4t1wTtcN8WzE3t1NhBqww-1 X-Mimecast-Spam-Score: 0 X-Mailman-Original-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1574855518; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/XhIGMtsi7KjLuMFks8eIn7a7OR+crqiPZAFOU/Qxss=; b=NtbC7q1/8VLDUM6dTDoUqMOHf0yXHvQN4Sq0apY2bQegfMS2SqdIz2D1H3MnuPte9sAiuT PAVvcsCFKkVuNRsSsQl29uusTzbjH4mClHEi08JTSTIUt/iINSFNJDFzJpsF0zK6LHHixZ ULSK9aYdulOHRu1OGw62c1Q5ItU0c5U= X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: "Rafael J. Wysocki" , Linux PCI , Linux PM , "Rafael J . Wysocki" , LKML , dri-devel , Mario Limonciello , Bjorn Helgaas , nouveau Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Message-ID: <20191127115142.AxsEUOVaRtl8jZ7lBc0XinK5jXJc8s0rJslf0yzzSYg@z> T24gV2VkLCBOb3YgMjcsIDIwMTkgYXQgMTI6NDkgUE0gTWlrYSBXZXN0ZXJiZXJnCjxtaWthLndl c3RlcmJlcmdAaW50ZWwuY29tPiB3cm90ZToKPgo+IE9uIFR1ZSwgTm92IDI2LCAyMDE5IGF0IDA2 OjEwOjM2UE0gLTA1MDAsIEx5dWRlIFBhdWwgd3JvdGU6Cj4gPiBIZXktdGhpcyBpcyBhbG1vc3Qg Y2VydGFpbmx5IG5vdCB0aGUgcmlnaHQgcGxhY2UgaW4gdGhpcyB0aHJlYWQgdG8gcmVzcG9uZCwK PiA+IGJ1dCB0aGlzIHRocmVhZCBoYXMgZ290dGVuIHNvIGRlZXAgZXZvbHV0aW9uIGNhbid0IHB1 c2ggdGhlIHN1YmplY3QgZnVydGhlciB0bwo+ID4gdGhlIHJpZ2h0LCBoZWguIFNvIEknbGwganVz dCByZXNwb25kIGhlcmUuCj4KPiA6KQo+Cj4gPiBJJ3ZlIGJlZW4gZm9sbG93aW5nIHRoaXMgYW5k IGhlbHBpbmcgb3V0IEthcm9sIHdpdGggdGVzdGluZyBoZXJlIGFuZCB0aGVyZS4KPiA+IFRoZXkg aGFkIG1lIHRlc3QgQmpvcm4ncyBQQ0kgYnJhbmNoIG9uIHRoZSBYMSBFeHRyZW1lIDJuZCBnZW5l cmF0aW9uLCB3aGljaAo+ID4gaGFzIGEgdHVyaW5nIEdQVSBhbmQgODA4NjoxOTAxIFBDSSBicmlk Z2UuCj4gPgo+ID4gSSB3YXMgYWJvdXQgdG8gc2F5ICJ0aGUgcGF0Y2ggZml4ZWQgdGhpbmdzLCBo b29yYXkhIiBidXQgaXQgc2VlbXMgdGhhdCBhZnRlcgo+ID4gdHJ5aW5nIHJ1bnRpbWUgc3VzcGVu ZC9yZXN1bWUgYSBjb3VwbGUgdGltZXMgdGhpbmdzIGZhbGwgYXBhcnQgYWdhaW46Cj4KPiBZb3Ug bWVhbiAkc3ViamVjdCBwYXRjaCwgbm8/Cj4KCm5vLCBJIHRvbGQgTHl1ZGUgdG8gdGVzdCB0aGUg cGNpL3BtIGJyYW5jaCBhcyB0aGUgcnVucG0gZXJyb3JzIHdlIHNhdwpvbiB0aGF0IG1hY2hpbmUg bG9va2VkIGRpZmZlcmVudC4gU29tZSBCQVIgZXJyb3IgdGhlIEdQVSByZXBvcnRlZAphZnRlciBp dCBnb3QgcmVzdW1lZCwgc28gSSB3YXMgd29uZGVyaW5nIGlmIHRoZSBkZWxheXMgd2VyZSBoZWxw aW5nCndpdGggdGhhdC4gQnV0IGFmdGVyIHNvbWUgY3ljbGVzIGl0IHN0aWxsIGNhdXNlZCB0aGUg c2FtZSBpc3N1ZSwgdGhhdAp0aGUgR1BVIGRpc2FwcGVhcmVkLiBMYXRlciB0ZXN0aW5nIGFsc28g c2hvd2VkIHRoYXQgbXkgcGF0Y2ggYWxzbwpkaWRuJ3Qgc2VlbSB0byBoZWxwIHdpdGggdGhpcyBl cnJvciBzYWRseSA6LwoKPiA+IFsgIDY4Ni44ODMyNDddIG5vdXZlYXUgMDAwMDowMTowMC4wOiBE Uk06IHN1c3BlbmRpbmcgb2JqZWN0IHRyZWUuLi4KPiA+IFsgIDc1Mi44NjY0ODRdIEFDUEkgRXJy b3I6IEFib3J0aW5nIG1ldGhvZCBcX1NCLlBDSTAuUEVHMC5QRUdQLk5WUE8gZHVlIHRvIHByZXZp b3VzIGVycm9yIChBRV9BTUxfTE9PUF9USU1FT1VUKSAoMjAxOTA4MTYvcHNwYXJzZS01MjkpCj4g PiBbICA3NTIuODY2NTA4XSBBQ1BJIEVycm9yOiBBYm9ydGluZyBtZXRob2QgXF9TQi5QQ0kwLlBH T04gZHVlIHRvIHByZXZpb3VzIGVycm9yIChBRV9BTUxfTE9PUF9USU1FT1VUKSAoMjAxOTA4MTYv cHNwYXJzZS01MjkpCj4gPiBbICA3NTIuODY2NTIxXSBBQ1BJIEVycm9yOiBBYm9ydGluZyBtZXRo b2QgXF9TQi5QQ0kwLlBFRzAuUEcwMC5fT04gZHVlIHRvIHByZXZpb3VzIGVycm9yIChBRV9BTUxf TE9PUF9USU1FT1VUKSAoMjAxOTA4MTYvcHNwYXJzZS01MjkpCj4KPiBUaGlzIGlzIHByb2JhYmx5 IHRoZSBjdWxwcml0LiBUaGUgc2FtZSBBTUwgY29kZSBmYWlscyB0byBwcm9wZXJseSB0dXJuCj4g b24gdGhlIGRldmljZS4KPgo+IElzIGFjcGlkdW1wIGZyb20gdGhpcyBzeXN0ZW0gYXZhaWxhYmxl IHNvbWV3aGVyZT8KPgoKX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX18KZHJpLWRldmVsIG1haWxpbmcgbGlzdApkcmktZGV2ZWxAbGlzdHMuZnJlZWRlc2t0b3Au b3JnCmh0dHBzOi8vbGlzdHMuZnJlZWRlc2t0b3Aub3JnL21haWxtYW4vbGlzdGluZm8vZHJpLWRl dmVs