From: Lyude Paul <lyude@redhat.com>
To: "Rafael J. Wysocki" <rafael@kernel.org>,
Karol Herbst <kherbst@redhat.com>
Cc: Mika Westerberg <mika.westerberg@intel.com>,
Bjorn Helgaas <helgaas@kernel.org>,
LKML <linux-kernel@vger.kernel.org>,
"Rafael J . Wysocki" <rjw@rjwysocki.net>,
Linux PCI <linux-pci@vger.kernel.org>,
Linux PM <linux-pm@vger.kernel.org>,
dri-devel <dri-devel@lists.freedesktop.org>,
nouveau <nouveau@lists.freedesktop.org>,
Dave Airlie <airlied@gmail.com>,
Mario Limonciello <Mario.Limonciello@dell.com>
Subject: Re: [PATCH v4] pci: prevent putting nvidia GPUs into lower device states on certain intel bridges
Date: Tue, 26 Nov 2019 18:10:36 -0500 [thread overview]
Message-ID: <e0eeddf4214f54dfac08e428dfb30cbd39f20680.camel@redhat.com> (raw)
In-Reply-To: <CAJZ5v0jbh7jz+YQcw-gC5ztmMOc4E9+KFBCy4VGRsRFxBw-gnw@mail.gmail.com>
[big snip]
> There is a sysfs attribute called "d3cold_allowed" which can be used
> for "blocking" D3cold, so can you please retest using that?
>
Hey-this is almost certainly not the right place in this thread to respond,
but this thread has gotten so deep evolution can't push the subject further to
the right, heh. So I'll just respond here.
I've been following this and helping out Karol with testing here and there.
They had me test Bjorn's PCI branch on the X1 Extreme 2nd generation, which
has a turing GPU and 8086:1901 PCI bridge.
I was about to say "the patch fixed things, hooray!" but it seems that after
trying runtime suspend/resume a couple times things fall apart again:
[ 27.680433] nouveau 0000:01:00.0: enabling device (0000 -> 0003)
[ 27.680578] nouveau 0000:01:00.0: NVIDIA TU117 (167000a1)
[ 27.763967] nouveau 0000:01:00.0: bios: version 90.17.20.00.16
[ 27.764664] nouveau 0000:01:00.0: fb: 4096 MiB GDDR5
[ 27.806115] vga_switcheroo: enabled
[ 27.806221] [TTM] Zone kernel: Available graphics memory: 16244510 KiB
[ 27.806222] [TTM] Zone dma32: Available graphics memory: 2097152 KiB
[ 27.806222] [TTM] Initializing pool allocator
[ 27.806224] [TTM] Initializing DMA pool allocator
[ 27.806249] nouveau 0000:01:00.0: DRM: VRAM: 4096 MiB
[ 27.806249] nouveau 0000:01:00.0: DRM: GART: 536870912 MiB
[ 27.806250] nouveau 0000:01:00.0: DRM: BIT table 'A' not found
[ 27.806251] nouveau 0000:01:00.0: DRM: BIT table 'L' not found
[ 27.806251] nouveau 0000:01:00.0: DRM: TMDS table version 2.0
[ 27.806252] nouveau 0000:01:00.0: DRM: DCB version 4.1
[ 27.806253] nouveau 0000:01:00.0: DRM: DCB outp 00: 02800f66 04600020
[ 27.806253] nouveau 0000:01:00.0: DRM: DCB outp 01: 02011f52 00020010
[ 27.806254] nouveau 0000:01:00.0: DRM: DCB outp 02: 01022f36 04600010
[ 27.806254] nouveau 0000:01:00.0: DRM: DCB outp 03: 01033f46 04600020
[ 27.806255] nouveau 0000:01:00.0: DRM: DCB conn 00: 00020047
[ 27.806255] nouveau 0000:01:00.0: DRM: DCB conn 01: 00010161
[ 27.806256] nouveau 0000:01:00.0: DRM: DCB conn 02: 00001248
[ 27.806256] nouveau 0000:01:00.0: DRM: DCB conn 03: 00002348
[ 27.806257] nouveau 0000:01:00.0: DRM: Pointer to flat panel table invalid
[ 27.806415] nouveau 0000:01:00.0: DRM: failed to create kernel channel, -22
[ 27.806530] nouveau 0000:01:00.0: DRM: MM: using COPY for buffer copies
[ 28.114808] nouveau 0000:01:00.0: DRM: unknown connector type 48
[ 28.114943] nouveau 0000:01:00.0: DRM: unknown connector type 48
[ 28.115026] [drm] Supports vblank timestamp caching Rev 2 (21.10.2013).
[ 28.115027] [drm] Driver supports precise vblank timestamp query.
[ 28.116611] [drm] Cannot find any crtc or sizes
[ 28.117452] [drm] Initialized nouveau 1.3.1 20120801 for 0000:01:00.0 on minor 1
[ 28.118074] [drm] Cannot find any crtc or sizes
[ 28.119523] [drm] Cannot find any crtc or sizes
[ 34.081503] nouveau 0000:01:00.0: DRM: suspending console...
[ 34.081508] nouveau 0000:01:00.0: DRM: suspending display...
[ 34.081528] nouveau 0000:01:00.0: DRM: evicting buffers...
[ 34.081531] nouveau 0000:01:00.0: DRM: waiting for kernel channels to go idle...
[ 34.081551] nouveau 0000:01:00.0: DRM: suspending fence...
[ 34.091173] nouveau 0000:01:00.0: DRM: suspending object tree...
[ 37.729746] nouveau 0000:01:00.0: DRM: resuming object tree...
[ 38.042076] nouveau 0000:01:00.0: DRM: resuming fence...
[ 38.042167] nouveau 0000:01:00.0: DRM: resuming display...
[ 38.042175] nouveau 0000:01:00.0: DRM: resuming console...
[ 44.309325] nouveau 0000:01:00.0: DRM: suspending console...
[ 44.309331] nouveau 0000:01:00.0: DRM: suspending display...
[ 44.309349] nouveau 0000:01:00.0: DRM: evicting buffers...
[ 44.309352] nouveau 0000:01:00.0: DRM: waiting for kernel channels to go idle...
[ 44.309371] nouveau 0000:01:00.0: DRM: suspending fence...
[ 44.318938] nouveau 0000:01:00.0: DRM: suspending object tree...
[ 76.577644] nouveau 0000:01:00.0: DRM: resuming object tree...
[ 76.890266] nouveau 0000:01:00.0: DRM: resuming fence...
[ 76.890362] nouveau 0000:01:00.0: DRM: resuming display...
[ 76.890379] nouveau 0000:01:00.0: DRM: resuming console...
[ 82.721356] nouveau 0000:01:00.0: DRM: suspending console...
[ 82.721361] nouveau 0000:01:00.0: DRM: suspending display...
[ 82.721380] nouveau 0000:01:00.0: DRM: evicting buffers...
[ 82.721383] nouveau 0000:01:00.0: DRM: waiting for kernel channels to go idle...
[ 82.721403] nouveau 0000:01:00.0: DRM: suspending fence...
[ 82.730483] nouveau 0000:01:00.0: DRM: suspending object tree...
[ 681.369950] nouveau 0000:01:00.0: DRM: resuming object tree...
[ 681.690464] nouveau 0000:01:00.0: DRM: resuming fence...
[ 681.690555] nouveau 0000:01:00.0: DRM: resuming display...
[ 681.690568] nouveau 0000:01:00.0: DRM: resuming console...
[ 686.873629] nouveau 0000:01:00.0: DRM: suspending console...
[ 686.873634] nouveau 0000:01:00.0: DRM: suspending display...
[ 686.873653] nouveau 0000:01:00.0: DRM: evicting buffers...
[ 686.873656] nouveau 0000:01:00.0: DRM: waiting for kernel channels to go idle...
[ 686.873676] nouveau 0000:01:00.0: DRM: suspending fence...
[ 686.883247] nouveau 0000:01:00.0: DRM: suspending object tree...
[ 752.866484] ACPI Error: Aborting method \_SB.PCI0.PEG0.PEGP.NVPO due to previous error (AE_AML_LOOP_TIMEOUT) (20190816/psparse-529)
[ 752.866508] ACPI Error: Aborting method \_SB.PCI0.PGON due to previous error (AE_AML_LOOP_TIMEOUT) (20190816/psparse-529)
[ 752.866521] ACPI Error: Aborting method \_SB.PCI0.PEG0.PG00._ON due to previous error (AE_AML_LOOP_TIMEOUT) (20190816/psparse-529)
[ 752.866542] acpi device:00: Failed to change power state to D0
[ 754.209030] video LNXVIDEO:00: Cannot transition to power state D0 for parent in (unknown)
[ 755.848894] nouveau 0000:01:00.0: not ready 1023ms after Switch to D0; waiting
[ 756.936876] nouveau 0000:01:00.0: not ready 2047ms after Switch to D0; waiting
[ 759.048849] nouveau 0000:01:00.0: not ready 4095ms after Switch to D0; waiting
[ 763.208807] nouveau 0000:01:00.0: not ready 8191ms after Switch to D0; waiting
[ 771.912692] nouveau 0000:01:00.0: not ready 16383ms after Switch to D0; waiting
[ 788.808505] nouveau 0000:01:00.0: not ready 32767ms after Switch to D0; waiting
752.866542 is where I start trying to resume the GPU. lspci -nn:
00:00.0 Host bridge [0600]: Intel Corporation Device [8086:3e20] (rev 0d)
00:01.0 PCI bridge [0604]: Intel Corporation Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor PCIe Controller (x16) [8086:1901] (rev 0d)
00:02.0 VGA compatible controller [0300]: Intel Corporation UHD Graphics 630 (Mobile) [8086:3e9b] (rev 02)
00:04.0 Signal processing controller [1180]: Intel Corporation Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor Thermal Subsystem [8086:1903] (rev 0d)
00:08.0 System peripheral [0880]: Intel Corporation Xeon E3-1200 v5/v6 / E3-1500 v5 / 6th/7th/8th Gen Core Processor Gaussian Mixture Model [8086:1911]
00:12.0 Signal processing controller [1180]: Intel Corporation Cannon Lake PCH Thermal Controller [8086:a379] (rev 10)
00:14.0 USB controller [0c03]: Intel Corporation Cannon Lake PCH USB 3.1 xHCI Host Controller [8086:a36d] (rev 10)
00:14.2 RAM memory [0500]: Intel Corporation Cannon Lake PCH Shared SRAM [8086:a36f] (rev 10)
00:15.0 Serial bus controller [0c80]: Intel Corporation Cannon Lake PCH Serial IO I2C Controller #0 [8086:a368] (rev 10)
00:16.0 Communication controller [0780]: Intel Corporation Cannon Lake PCH HECI Controller [8086:a360] (rev 10)
00:1b.0 PCI bridge [0604]: Intel Corporation Cannon Lake PCH PCI Express Root Port #17 [8086:a340] (rev f0)
00:1b.4 PCI bridge [0604]: Intel Corporation Cannon Lake PCH PCI Express Root Port #21 [8086:a32c] (rev f0)
00:1c.0 PCI bridge [0604]: Intel Corporation Cannon Lake PCH PCI Express Root Port #1 [8086:a338] (rev f0)
00:1d.0 PCI bridge [0604]: Intel Corporation Cannon Lake PCH PCI Express Root Port #9 [8086:a330] (rev f0)
00:1d.6 PCI bridge [0604]: Intel Corporation Cannon Lake PCH PCI Express Root Port #15 [8086:a336] (rev f0)
00:1e.0 Communication controller [0780]: Intel Corporation Device [8086:a328] (rev 10)
00:1f.0 ISA bridge [0601]: Intel Corporation Device [8086:a30e] (rev 10)
00:1f.3 Audio device [0403]: Intel Corporation Cannon Lake PCH cAVS [8086:a348] (rev 10)
00:1f.4 SMBus [0c05]: Intel Corporation Cannon Lake PCH SMBus Controller [8086:a323] (rev 10)
00:1f.5 Serial bus controller [0c80]: Intel Corporation Cannon Lake PCH SPI Controller [8086:a324] (rev 10)
00:1f.6 Ethernet controller [0200]: Intel Corporation Ethernet Connection (7) I219-LM [8086:15bb] (rev 10)
01:00.0 VGA compatible controller [0300]: NVIDIA Corporation Device [10de:1f91] (rev a1)
01:00.1 Audio device [0403]: NVIDIA Corporation Device [10de:10fa] (rev a1)
02:00.0 Non-Volatile memory controller [0108]: Samsung Electronics Co Ltd NVMe SSD Controller SM981/PM981/PM983 [144d:a808]
04:00.0 PCI bridge [0604]: Intel Corporation JHL7540 Thunderbolt 3 Bridge [Titan Ridge 4C 2018] [8086:15ea] (rev 06)
05:00.0 PCI bridge [0604]: Intel Corporation JHL7540 Thunderbolt 3 Bridge [Titan Ridge 4C 2018] [8086:15ea] (rev 06)
05:01.0 PCI bridge [0604]: Intel Corporation JHL7540 Thunderbolt 3 Bridge [Titan Ridge 4C 2018] [8086:15ea] (rev 06)
05:02.0 PCI bridge [0604]: Intel Corporation JHL7540 Thunderbolt 3 Bridge [Titan Ridge 4C 2018] [8086:15ea] (rev 06)
05:04.0 PCI bridge [0604]: Intel Corporation JHL7540 Thunderbolt 3 Bridge [Titan Ridge 4C 2018] [8086:15ea] (rev 06)
06:00.0 System peripheral [0880]: Intel Corporation JHL7540 Thunderbolt 3 NHI [Titan Ridge 4C 2018] [8086:15eb] (rev 06)
2c:00.0 USB controller [0c03]: Intel Corporation JHL7540 Thunderbolt 3 USB Controller [Titan Ridge 4C 2018] [8086:15ec] (rev 06)
52:00.0 Network controller [0280]: Intel Corporation Wi-Fi 6 AX200 [8086:2723] (rev 1a)
53:00.0 Unassigned class [ff00]: Realtek Semiconductor Co., Ltd. RTS525A PCI Express Card Reader [10ec:525a] (rev 01)
--
Cheers,
Lyude Paul
next prev parent reply other threads:[~2019-11-26 23:10 UTC|newest]
Thread overview: 67+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-10-17 12:19 [PATCH v4] pci: prevent putting nvidia GPUs into lower device states on certain intel bridges Karol Herbst
2019-11-14 19:17 ` Karol Herbst
2019-11-19 20:06 ` Dave Airlie
2019-11-19 21:49 ` Bjorn Helgaas
2019-11-19 22:26 ` Karol Herbst
2019-11-19 22:57 ` Bjorn Helgaas
2019-11-20 10:18 ` Mika Westerberg
2019-11-20 10:52 ` Rafael J. Wysocki
2019-11-20 11:22 ` Mika Westerberg
2019-11-20 11:48 ` Rafael J. Wysocki
2019-11-20 11:51 ` Karol Herbst
2019-11-20 12:06 ` Rafael J. Wysocki
2019-11-20 12:09 ` Karol Herbst
2019-11-20 12:14 ` Rafael J. Wysocki
2019-11-20 12:19 ` Karol Herbst
2019-11-20 12:11 ` Rafael J. Wysocki
2019-11-20 11:51 ` Mika Westerberg
2019-11-20 11:54 ` Karol Herbst
2019-11-20 11:58 ` Karol Herbst
2019-11-20 12:09 ` Mika Westerberg
2019-11-20 12:11 ` Karol Herbst
2019-11-20 15:15 ` Mika Westerberg
2019-11-20 15:37 ` Karol Herbst
2019-11-20 15:53 ` Mika Westerberg
2019-11-20 16:23 ` Mika Westerberg
2019-11-20 21:36 ` Karol Herbst
2019-11-21 10:14 ` Mika Westerberg
2019-11-21 11:03 ` Rafael J. Wysocki
2019-11-21 11:08 ` Rafael J. Wysocki
2019-11-21 11:15 ` Rafael J. Wysocki
2019-11-21 11:17 ` Mika Westerberg
2019-11-21 11:31 ` Rafael J. Wysocki
2019-11-20 21:37 ` Rafael J. Wysocki
2019-11-20 21:40 ` Karol Herbst
2019-11-20 22:29 ` Rafael J. Wysocki
2019-11-21 11:28 ` Mika Westerberg
2019-11-21 11:34 ` Rafael J. Wysocki
2019-11-21 11:46 ` Mika Westerberg
2019-11-21 12:52 ` Mika Westerberg
2019-11-21 12:56 ` Karol Herbst
2019-11-21 15:43 ` Rafael J. Wysocki
2019-11-21 19:49 ` Mika Westerberg
2019-11-21 22:39 ` Rafael J. Wysocki
2019-11-21 22:50 ` Karol Herbst
2019-11-22 0:13 ` Karol Herbst
2019-11-22 9:07 ` Rafael J. Wysocki
2019-11-22 11:30 ` Karol Herbst
2019-11-22 10:36 ` Mika Westerberg
2019-11-22 11:30 ` Rafael J. Wysocki
2019-11-22 11:34 ` Karol Herbst
2019-11-22 11:54 ` Rafael J. Wysocki
2019-11-22 11:52 ` Mika Westerberg
2019-11-22 12:15 ` Rafael J. Wysocki
2019-11-21 12:52 ` Karol Herbst
2019-11-21 15:47 ` Rafael J. Wysocki
2019-11-21 16:06 ` Karol Herbst
2019-11-21 16:39 ` Rafael J. Wysocki
2019-11-26 23:10 ` Lyude Paul [this message]
2019-11-27 11:48 ` Mika Westerberg
2019-11-27 11:51 ` Karol Herbst
2019-11-27 19:51 ` Lyude Paul
2019-12-09 11:17 ` Karol Herbst
2019-12-09 11:38 ` Rafael J. Wysocki
2019-12-09 12:24 ` Karol Herbst
2019-12-10 19:58 ` Dave Airlie
2019-12-10 20:49 ` Karol Herbst
2020-01-13 15:31 ` Karol Herbst
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=e0eeddf4214f54dfac08e428dfb30cbd39f20680.camel@redhat.com \
--to=lyude@redhat.com \
--cc=Mario.Limonciello@dell.com \
--cc=airlied@gmail.com \
--cc=dri-devel@lists.freedesktop.org \
--cc=helgaas@kernel.org \
--cc=kherbst@redhat.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-pci@vger.kernel.org \
--cc=linux-pm@vger.kernel.org \
--cc=mika.westerberg@intel.com \
--cc=nouveau@lists.freedesktop.org \
--cc=rafael@kernel.org \
--cc=rjw@rjwysocki.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).