* Kernel 6.7 regression doesn't boot if using AMD eGPU
@ 2024-04-13 22:04 Eric Wagner
2024-04-14 0:01 ` Bagas Sanjaya
` (2 more replies)
0 siblings, 3 replies; 17+ messages in thread
From: Eric Wagner @ 2024-04-13 22:04 UTC (permalink / raw)
To: Joerg Roedel, Will Deacon
Cc: Suravee Suthikulpanit, Robin Murphy, iommu, linux-kernel
[-- Attachment #1.1: Type: text/plain, Size: 717 bytes --]
On my Thinkpad T14s G3 AMD (Ryzen 7 6850U) laptop connected to an AMD RX
580 in Akitio Node Thunderbolt 3 eGPU. Booting with the eGPU connected
hangs on kernels 6.7 and 6.8, but worked on 6.6. For debugging, I find that
adding the kernel parameter amd_iommu=off seems to fix the issue and allows
booting with the eGPU on 6.7.
I tried bisecting the issue between 6.6 and 6.7 and ended up with:
"e8cca466a84a75f8ff2a7a31173c99ee6d1c59d2 is the first bad commit" in the
attached. This seems to indicate an amd iommu issue.
Two others also reported the same issue on AMD Ryzen 7 7840 with AMD RX
6000 connected as eGPU (https://gitlab.freedesktop.org/drm/amd/-/issues/3182
).
Let me know if you need more information.
[-- Attachment #1.2: Type: text/html, Size: 884 bytes --]
[-- Attachment #2: bisect.log --]
[-- Type: text/x-log, Size: 6298 bytes --]
Bisecting: 366 revisions left to test after this (roughly 9 steps)
[74e9347ebc5be452935fe4f3eddb150aa5a6f4fe] Merge tag 'loongarch-fixes-6.6-3' of git://git.kernel.org/pub/scm/linux/kernel/git/chenhuacai/linux-loongson
Bisecting: 182 revisions left to test after this (roughly 8 steps)
[f6176471542d991137543af2ef1c18dae3286079] Merge tag 'mtd/fixes-for-6.6-rc7' of git://git.kernel.org/pub/scm/linux/kernel/git/mtd/linux
Bisecting: 87 revisions left to test after this (roughly 7 steps)
[fe3cfe869d5e0453754cf2b4c75110276b5e8527] Merge tag 'phy-fixes-6.6' of git://git.kernel.org/pub/scm/linux/kernel/git/phy/linux-phy
Bisecting: 43 revisions left to test after this (roughly 6 steps)
[c76c067e488ccd55734c3e750799caf2c5956db6] s390/pci: Use dma-iommu layer
Bisecting: 27 revisions left to test after this (roughly 5 steps)
[aa5cabc4ce8e6b45d170d162dc54b1bac1767c47] Merge tag 'arm-smmu-updates' of git://git.kernel.org/pub/scm/linux/kernel/git/will/linux into arm/smmu
Bisecting: 14 revisions left to test after this (roughly 4 steps)
[bbc70e0aec287e164344b1a071bd46466a4f29b3] iommu/dart: Remove the force_bypass variable
Bisecting: 9 revisions left to test after this (roughly 3 steps)
[e82c175e63229ea495a0a0b5305a98b5b6ee5346] Revert "iommu/vt-d: Remove unused function"
Bisecting: 5 revisions left to test after this (roughly 2 steps)
[92bce97f0c341d3037b0f364b6839483f6a41cae] s390/pci: Fix reset of IOMMU software counters
Bisecting: 3 revisions left to test after this (roughly 2 steps)
[3613047280ec42a4e1350fdc1a6dd161ff4008cc] Merge tag 'v6.6-rc7' into core
Bisecting: 2 revisions left to test after this (roughly 1 step)
[f7da9c081517daba70f9f9342e09d7a6322ba323] iommu/tegra-smmu: Drop unnecessary error check for for debugfs_create_dir()
Bisecting: 1 revision left to test after this (roughly 1 step)
[9e13ec61de2a51195b122a79461431d8cb99d7b5] iommu/virtio: Add __counted_by for struct viommu_request and use struct_size()
Bisecting: 0 revisions left to test after this (roughly 0 steps)
[6e6c6d6bc6c96c2477ddfea24a121eb5ee12b7a3] iommu: Avoid unnecessary cache invalidations
e8cca466a84a75f8ff2a7a31173c99ee6d1c59d2 is the first bad commit
commit e8cca466a84a75f8ff2a7a31173c99ee6d1c59d2
Merge: 6e6c6d6bc6 f7da9c0815 aa5cabc4ce 9e13ec61de e82c175e63 cedc811c76 3613047280 92bce97f0c
Author: Joerg Roedel <jroedel@suse.de>
Date: Fri Oct 27 09:13:40 2023 +0200
Merge branches 'iommu/fixes', 'arm/tegra', 'arm/smmu', 'virtio', 'x86/vt-d', 'x86/amd', 'core' and 's390' into next
Documentation/admin-guide/kernel-parameters.txt | 9 +-
.../devicetree/bindings/iommu/arm,smmu.yaml | 2 +
arch/arm/configs/multi_v7_defconfig | 1 -
arch/arm/configs/tegra_defconfig | 1 -
arch/powerpc/kernel/iommu.c | 53 +-
arch/s390/include/asm/pci.h | 11 -
arch/s390/include/asm/pci_clp.h | 3 +
arch/s390/include/asm/pci_dma.h | 121 +--
arch/s390/pci/Makefile | 2 +-
arch/s390/pci/pci.c | 35 +-
arch/s390/pci/pci_bus.c | 5 -
arch/s390/pci/pci_debug.c | 12 +-
arch/s390/pci/pci_dma.c | 746 ---------------
arch/s390/pci/pci_event.c | 17 +-
arch/s390/pci/pci_sysfs.c | 19 +-
drivers/iommu/Kconfig | 15 +-
drivers/iommu/Makefile | 1 -
drivers/iommu/amd/Kconfig | 9 -
drivers/iommu/amd/Makefile | 1 -
drivers/iommu/amd/amd_iommu.h | 35 +-
drivers/iommu/amd/amd_iommu_types.h | 52 +-
drivers/iommu/amd/init.c | 117 +--
drivers/iommu/amd/io_pgtable_v2.c | 8 +-
drivers/iommu/amd/iommu.c | 577 +++++-------
drivers/iommu/amd/iommu_v2.c | 996 ---------------------
drivers/iommu/apple-dart.c | 138 +--
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 71 +-
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 251 +++---
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 17 +-
drivers/iommu/arm/arm-smmu/arm-smmu-qcom.c | 2 +
drivers/iommu/arm/arm-smmu/qcom_iommu.c | 45 +-
drivers/iommu/dma-iommu.c | 200 ++++-
drivers/iommu/exynos-iommu.c | 83 +-
drivers/iommu/fsl_pamu_domain.c | 41 +-
drivers/iommu/intel/debugfs.c | 215 ++++-
drivers/iommu/intel/iommu.c | 19 +-
drivers/iommu/intel/iommu.h | 14 +
drivers/iommu/iommu.c | 455 +++++-----
drivers/iommu/iommufd/selftest.c | 30 +-
drivers/iommu/iova.c | 95 +-
drivers/iommu/ipmmu-vmsa.c | 72 +-
drivers/iommu/msm_iommu.c | 35 +-
drivers/iommu/mtk_iommu.c | 35 +-
drivers/iommu/mtk_iommu_v1.c | 28 +-
drivers/iommu/omap-iommu.c | 69 +-
drivers/iommu/omap-iommu.h | 2 +-
drivers/iommu/rockchip-iommu.c | 59 +-
drivers/iommu/s390-iommu.c | 424 ++++++++-
drivers/iommu/sprd-iommu.c | 36 +-
drivers/iommu/sun50i-iommu.c | 80 +-
drivers/iommu/tegra-gart.c | 371 --------
drivers/iommu/tegra-smmu.c | 58 +-
drivers/iommu/virtio-iommu.c | 4 +-
drivers/memory/tegra/mc.c | 34 -
drivers/memory/tegra/tegra20.c | 28 -
include/linux/amd-iommu.h | 120 ---
include/linux/iommu.h | 38 +-
include/soc/tegra/mc.h | 26 -
58 files changed, 2138 insertions(+), 3905 deletions(-)
delete mode 100644 arch/s390/pci/pci_dma.c
delete mode 100644 drivers/iommu/amd/iommu_v2.c
delete mode 100644 drivers/iommu/tegra-gart.c
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: Kernel 6.7 regression doesn't boot if using AMD eGPU
2024-04-13 22:04 Kernel 6.7 regression doesn't boot if using AMD eGPU Eric Wagner
@ 2024-04-14 0:01 ` Bagas Sanjaya
2024-04-15 8:04 ` Joerg Roedel
2024-04-15 16:30 ` Jason Gunthorpe
2 siblings, 0 replies; 17+ messages in thread
From: Bagas Sanjaya @ 2024-04-14 0:01 UTC (permalink / raw)
To: Eric Wagner, Joerg Roedel, Will Deacon
Cc: Suravee Suthikulpanit, Robin Murphy, iommu, linux-kernel,
Linux Regressions
[-- Attachment #1: Type: text/plain, Size: 3512 bytes --]
On Sat, Apr 13, 2024 at 06:04:12PM -0400, Eric Wagner wrote:
> On my Thinkpad T14s G3 AMD (Ryzen 7 6850U) laptop connected to an AMD RX
> 580 in Akitio Node Thunderbolt 3 eGPU. Booting with the eGPU connected
> hangs on kernels 6.7 and 6.8, but worked on 6.6. For debugging, I find that
> adding the kernel parameter amd_iommu=off seems to fix the issue and allows
> booting with the eGPU on 6.7.
>
> I tried bisecting the issue between 6.6 and 6.7 and ended up with:
> "e8cca466a84a75f8ff2a7a31173c99ee6d1c59d2 is the first bad commit" in the
> attached. This seems to indicate an amd iommu issue.
>
> Two others also reported the same issue on AMD Ryzen 7 7840 with AMD RX
> 6000 connected as eGPU (https://gitlab.freedesktop.org/drm/amd/-/issues/3182
> ).
>
> Let me know if you need more information.
> Bisecting: 366 revisions left to test after this (roughly 9 steps)
> [74e9347ebc5be452935fe4f3eddb150aa5a6f4fe] Merge tag 'loongarch-fixes-6.6-3' of git://git.kernel.org/pub/scm/linux/kernel/git/chenhuacai/linux-loongson
> Bisecting: 182 revisions left to test after this (roughly 8 steps)
> [f6176471542d991137543af2ef1c18dae3286079] Merge tag 'mtd/fixes-for-6.6-rc7' of git://git.kernel.org/pub/scm/linux/kernel/git/mtd/linux
> Bisecting: 87 revisions left to test after this (roughly 7 steps)
> [fe3cfe869d5e0453754cf2b4c75110276b5e8527] Merge tag 'phy-fixes-6.6' of git://git.kernel.org/pub/scm/linux/kernel/git/phy/linux-phy
> Bisecting: 43 revisions left to test after this (roughly 6 steps)
> [c76c067e488ccd55734c3e750799caf2c5956db6] s390/pci: Use dma-iommu layer
> Bisecting: 27 revisions left to test after this (roughly 5 steps)
> [aa5cabc4ce8e6b45d170d162dc54b1bac1767c47] Merge tag 'arm-smmu-updates' of git://git.kernel.org/pub/scm/linux/kernel/git/will/linux into arm/smmu
> Bisecting: 14 revisions left to test after this (roughly 4 steps)
> [bbc70e0aec287e164344b1a071bd46466a4f29b3] iommu/dart: Remove the force_bypass variable
> Bisecting: 9 revisions left to test after this (roughly 3 steps)
> [e82c175e63229ea495a0a0b5305a98b5b6ee5346] Revert "iommu/vt-d: Remove unused function"
> Bisecting: 5 revisions left to test after this (roughly 2 steps)
> [92bce97f0c341d3037b0f364b6839483f6a41cae] s390/pci: Fix reset of IOMMU software counters
> Bisecting: 3 revisions left to test after this (roughly 2 steps)
> [3613047280ec42a4e1350fdc1a6dd161ff4008cc] Merge tag 'v6.6-rc7' into core
> Bisecting: 2 revisions left to test after this (roughly 1 step)
> [f7da9c081517daba70f9f9342e09d7a6322ba323] iommu/tegra-smmu: Drop unnecessary error check for for debugfs_create_dir()
> Bisecting: 1 revision left to test after this (roughly 1 step)
> [9e13ec61de2a51195b122a79461431d8cb99d7b5] iommu/virtio: Add __counted_by for struct viommu_request and use struct_size()
> Bisecting: 0 revisions left to test after this (roughly 0 steps)
> [6e6c6d6bc6c96c2477ddfea24a121eb5ee12b7a3] iommu: Avoid unnecessary cache invalidations
> e8cca466a84a75f8ff2a7a31173c99ee6d1c59d2 is the first bad commit
> commit e8cca466a84a75f8ff2a7a31173c99ee6d1c59d2
> Merge: 6e6c6d6bc6 f7da9c0815 aa5cabc4ce 9e13ec61de e82c175e63 cedc811c76 3613047280 92bce97f0c
> Author: Joerg Roedel <jroedel@suse.de>
> Date: Fri Oct 27 09:13:40 2023 +0200
>
> Merge branches 'iommu/fixes', 'arm/tegra', 'arm/smmu', 'virtio', 'x86/vt-d', 'x86/amd', 'core' and 's390' into next
>
Also Cc: regressions ML.
--
An old man doll... just what I always wanted! - Clara
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: Kernel 6.7 regression doesn't boot if using AMD eGPU
2024-04-13 22:04 Kernel 6.7 regression doesn't boot if using AMD eGPU Eric Wagner
2024-04-14 0:01 ` Bagas Sanjaya
@ 2024-04-15 8:04 ` Joerg Roedel
2024-04-15 16:30 ` Jason Gunthorpe
2 siblings, 0 replies; 17+ messages in thread
From: Joerg Roedel @ 2024-04-15 8:04 UTC (permalink / raw)
To: Eric Wagner
Cc: Will Deacon, Suravee Suthikulpanit, Robin Murphy, iommu,
linux-kernel, Vasant Hegde
Also adding Vasant.
On Sat, Apr 13, 2024 at 06:04:12PM -0400, Eric Wagner wrote:
> On my Thinkpad T14s G3 AMD (Ryzen 7 6850U) laptop connected to an AMD RX 580 in
> Akitio Node Thunderbolt 3 eGPU. Booting with the eGPU connected hangs on
> kernels 6.7 and 6.8, but worked on 6.6. For debugging, I find that adding the
> kernel parameter amd_iommu=off seems to fix the issue and allows booting with
> the eGPU on 6.7.
Do you have any way of getting the boot log of the hang? It is hard to
debug this without further data on where it hang and what happens
before.
Regards,
Joerg
>
> I tried bisecting the issue between 6.6 and 6.7 and ended up with:
> "e8cca466a84a75f8ff2a7a31173c99ee6d1c59d2 is the first bad commit" in the
> attached. This seems to indicate an amd iommu issue.
>
> Two others also reported the same issue on AMD Ryzen 7 7840 with AMD RX 6000
> connected as eGPU (https://gitlab.freedesktop.org/drm/amd/-/issues/3182).
>
> Let me know if you need more information.
> Bisecting: 366 revisions left to test after this (roughly 9 steps)
> [74e9347ebc5be452935fe4f3eddb150aa5a6f4fe] Merge tag 'loongarch-fixes-6.6-3' of git://git.kernel.org/pub/scm/linux/kernel/git/chenhuacai/linux-loongson
> Bisecting: 182 revisions left to test after this (roughly 8 steps)
> [f6176471542d991137543af2ef1c18dae3286079] Merge tag 'mtd/fixes-for-6.6-rc7' of git://git.kernel.org/pub/scm/linux/kernel/git/mtd/linux
> Bisecting: 87 revisions left to test after this (roughly 7 steps)
> [fe3cfe869d5e0453754cf2b4c75110276b5e8527] Merge tag 'phy-fixes-6.6' of git://git.kernel.org/pub/scm/linux/kernel/git/phy/linux-phy
> Bisecting: 43 revisions left to test after this (roughly 6 steps)
> [c76c067e488ccd55734c3e750799caf2c5956db6] s390/pci: Use dma-iommu layer
> Bisecting: 27 revisions left to test after this (roughly 5 steps)
> [aa5cabc4ce8e6b45d170d162dc54b1bac1767c47] Merge tag 'arm-smmu-updates' of git://git.kernel.org/pub/scm/linux/kernel/git/will/linux into arm/smmu
> Bisecting: 14 revisions left to test after this (roughly 4 steps)
> [bbc70e0aec287e164344b1a071bd46466a4f29b3] iommu/dart: Remove the force_bypass variable
> Bisecting: 9 revisions left to test after this (roughly 3 steps)
> [e82c175e63229ea495a0a0b5305a98b5b6ee5346] Revert "iommu/vt-d: Remove unused function"
> Bisecting: 5 revisions left to test after this (roughly 2 steps)
> [92bce97f0c341d3037b0f364b6839483f6a41cae] s390/pci: Fix reset of IOMMU software counters
> Bisecting: 3 revisions left to test after this (roughly 2 steps)
> [3613047280ec42a4e1350fdc1a6dd161ff4008cc] Merge tag 'v6.6-rc7' into core
> Bisecting: 2 revisions left to test after this (roughly 1 step)
> [f7da9c081517daba70f9f9342e09d7a6322ba323] iommu/tegra-smmu: Drop unnecessary error check for for debugfs_create_dir()
> Bisecting: 1 revision left to test after this (roughly 1 step)
> [9e13ec61de2a51195b122a79461431d8cb99d7b5] iommu/virtio: Add __counted_by for struct viommu_request and use struct_size()
> Bisecting: 0 revisions left to test after this (roughly 0 steps)
> [6e6c6d6bc6c96c2477ddfea24a121eb5ee12b7a3] iommu: Avoid unnecessary cache invalidations
> e8cca466a84a75f8ff2a7a31173c99ee6d1c59d2 is the first bad commit
> commit e8cca466a84a75f8ff2a7a31173c99ee6d1c59d2
> Merge: 6e6c6d6bc6 f7da9c0815 aa5cabc4ce 9e13ec61de e82c175e63 cedc811c76 3613047280 92bce97f0c
> Author: Joerg Roedel <jroedel@suse.de>
> Date: Fri Oct 27 09:13:40 2023 +0200
>
> Merge branches 'iommu/fixes', 'arm/tegra', 'arm/smmu', 'virtio', 'x86/vt-d', 'x86/amd', 'core' and 's390' into next
>
> Documentation/admin-guide/kernel-parameters.txt | 9 +-
> .../devicetree/bindings/iommu/arm,smmu.yaml | 2 +
> arch/arm/configs/multi_v7_defconfig | 1 -
> arch/arm/configs/tegra_defconfig | 1 -
> arch/powerpc/kernel/iommu.c | 53 +-
> arch/s390/include/asm/pci.h | 11 -
> arch/s390/include/asm/pci_clp.h | 3 +
> arch/s390/include/asm/pci_dma.h | 121 +--
> arch/s390/pci/Makefile | 2 +-
> arch/s390/pci/pci.c | 35 +-
> arch/s390/pci/pci_bus.c | 5 -
> arch/s390/pci/pci_debug.c | 12 +-
> arch/s390/pci/pci_dma.c | 746 ---------------
> arch/s390/pci/pci_event.c | 17 +-
> arch/s390/pci/pci_sysfs.c | 19 +-
> drivers/iommu/Kconfig | 15 +-
> drivers/iommu/Makefile | 1 -
> drivers/iommu/amd/Kconfig | 9 -
> drivers/iommu/amd/Makefile | 1 -
> drivers/iommu/amd/amd_iommu.h | 35 +-
> drivers/iommu/amd/amd_iommu_types.h | 52 +-
> drivers/iommu/amd/init.c | 117 +--
> drivers/iommu/amd/io_pgtable_v2.c | 8 +-
> drivers/iommu/amd/iommu.c | 577 +++++-------
> drivers/iommu/amd/iommu_v2.c | 996 ---------------------
> drivers/iommu/apple-dart.c | 138 +--
> drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 71 +-
> drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 251 +++---
> drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 17 +-
> drivers/iommu/arm/arm-smmu/arm-smmu-qcom.c | 2 +
> drivers/iommu/arm/arm-smmu/qcom_iommu.c | 45 +-
> drivers/iommu/dma-iommu.c | 200 ++++-
> drivers/iommu/exynos-iommu.c | 83 +-
> drivers/iommu/fsl_pamu_domain.c | 41 +-
> drivers/iommu/intel/debugfs.c | 215 ++++-
> drivers/iommu/intel/iommu.c | 19 +-
> drivers/iommu/intel/iommu.h | 14 +
> drivers/iommu/iommu.c | 455 +++++-----
> drivers/iommu/iommufd/selftest.c | 30 +-
> drivers/iommu/iova.c | 95 +-
> drivers/iommu/ipmmu-vmsa.c | 72 +-
> drivers/iommu/msm_iommu.c | 35 +-
> drivers/iommu/mtk_iommu.c | 35 +-
> drivers/iommu/mtk_iommu_v1.c | 28 +-
> drivers/iommu/omap-iommu.c | 69 +-
> drivers/iommu/omap-iommu.h | 2 +-
> drivers/iommu/rockchip-iommu.c | 59 +-
> drivers/iommu/s390-iommu.c | 424 ++++++++-
> drivers/iommu/sprd-iommu.c | 36 +-
> drivers/iommu/sun50i-iommu.c | 80 +-
> drivers/iommu/tegra-gart.c | 371 --------
> drivers/iommu/tegra-smmu.c | 58 +-
> drivers/iommu/virtio-iommu.c | 4 +-
> drivers/memory/tegra/mc.c | 34 -
> drivers/memory/tegra/tegra20.c | 28 -
> include/linux/amd-iommu.h | 120 ---
> include/linux/iommu.h | 38 +-
> include/soc/tegra/mc.h | 26 -
> 58 files changed, 2138 insertions(+), 3905 deletions(-)
> delete mode 100644 arch/s390/pci/pci_dma.c
> delete mode 100644 drivers/iommu/amd/iommu_v2.c
> delete mode 100644 drivers/iommu/tegra-gart.c
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: Kernel 6.7 regression doesn't boot if using AMD eGPU
2024-04-13 22:04 Kernel 6.7 regression doesn't boot if using AMD eGPU Eric Wagner
2024-04-14 0:01 ` Bagas Sanjaya
2024-04-15 8:04 ` Joerg Roedel
@ 2024-04-15 16:30 ` Jason Gunthorpe
2024-04-15 18:57 ` Eric Wagner
2 siblings, 1 reply; 17+ messages in thread
From: Jason Gunthorpe @ 2024-04-15 16:30 UTC (permalink / raw)
To: Eric Wagner
Cc: Joerg Roedel, Will Deacon, Suravee Suthikulpanit, Robin Murphy,
iommu, linux-kernel
On Sat, Apr 13, 2024 at 06:04:12PM -0400, Eric Wagner wrote:
> On my Thinkpad T14s G3 AMD (Ryzen 7 6850U) laptop connected to an AMD
> RX 580 in Akitio Node Thunderbolt 3 eGPU. Booting with the eGPU
> connected hangs on kernels 6.7 and 6.8, but worked on 6.6. For
> debugging, I find that adding the kernel parameter amd_iommu=off seems
> to fix the issue and allows booting with the eGPU on 6.7.
> I tried bisecting the issue between 6.6 and 6.7 and ended up with:
> "e8cca466a84a75f8ff2a7a31173c99ee6d1c59d2 is the first bad commit" in
> the attached. This seems to indicate an amd iommu issue.
> Two others also reported the same issue on AMD Ryzen 7 7840 with AMD RX
> 6000 connected as eGPU
> ([1]https://gitlab.freedesktop.org/drm/amd/-/issues/3182).
> Let me know if you need more information.
>
> References
>
> 1. https://gitlab.freedesktop.org/drm/amd/-/issues/3182
> Bisecting: 366 revisions left to test after this (roughly 9 steps)
> [74e9347ebc5be452935fe4f3eddb150aa5a6f4fe] Merge tag 'loongarch-fixes-6.6-3' of git://git.kernel.org/pub/scm/linux/kernel/git/chenhuacai/linux-loongson
> Bisecting: 182 revisions left to test after this (roughly 8 steps)
> [f6176471542d991137543af2ef1c18dae3286079] Merge tag 'mtd/fixes-for-6.6-rc7' of git://git.kernel.org/pub/scm/linux/kernel/git/mtd/linux
> Bisecting: 87 revisions left to test after this (roughly 7 steps)
> [fe3cfe869d5e0453754cf2b4c75110276b5e8527] Merge tag 'phy-fixes-6.6' of git://git.kernel.org/pub/scm/linux/kernel/git/phy/linux-phy
> Bisecting: 43 revisions left to test after this (roughly 6 steps)
> [c76c067e488ccd55734c3e750799caf2c5956db6] s390/pci: Use dma-iommu layer
> Bisecting: 27 revisions left to test after this (roughly 5 steps)
> [aa5cabc4ce8e6b45d170d162dc54b1bac1767c47] Merge tag 'arm-smmu-updates' of git://git.kernel.org/pub/scm/linux/kernel/git/will/linux into arm/smmu
> Bisecting: 14 revisions left to test after this (roughly 4 steps)
> [bbc70e0aec287e164344b1a071bd46466a4f29b3] iommu/dart: Remove the force_bypass variable
> Bisecting: 9 revisions left to test after this (roughly 3 steps)
> [e82c175e63229ea495a0a0b5305a98b5b6ee5346] Revert "iommu/vt-d: Remove unused function"
> Bisecting: 5 revisions left to test after this (roughly 2 steps)
> [92bce97f0c341d3037b0f364b6839483f6a41cae] s390/pci: Fix reset of IOMMU software counters
> Bisecting: 3 revisions left to test after this (roughly 2 steps)
> [3613047280ec42a4e1350fdc1a6dd161ff4008cc] Merge tag 'v6.6-rc7' into core
> Bisecting: 2 revisions left to test after this (roughly 1 step)
> [f7da9c081517daba70f9f9342e09d7a6322ba323] iommu/tegra-smmu: Drop unnecessary error check for for debugfs_create_dir()
> Bisecting: 1 revision left to test after this (roughly 1 step)
> [9e13ec61de2a51195b122a79461431d8cb99d7b5] iommu/virtio: Add __counted_by for struct viommu_request and use struct_size()
> Bisecting: 0 revisions left to test after this (roughly 0 steps)
> [6e6c6d6bc6c96c2477ddfea24a121eb5ee12b7a3] iommu: Avoid unnecessary cache invalidations
> e8cca466a84a75f8ff2a7a31173c99ee6d1c59d2 is the first bad commit
> commit e8cca466a84a75f8ff2a7a31173c99ee6d1c59d2
> Merge: 6e6c6d6bc6 f7da9c0815 aa5cabc4ce 9e13ec61de e82c175e63 cedc811c76 3613047280 92bce97f0c
> Author: Joerg Roedel <jroedel@suse.de>
> Date: Fri Oct 27 09:13:40 2023 +0200
>
> Merge branches 'iommu/fixes', 'arm/tegra', 'arm/smmu', 'virtio', 'x86/vt-d', 'x86/amd', 'core' and 's390' into next
Do you have the good/bad log on this? It doesn't look like bisect
tested enough stuff to really conclude the merge is the bad thing, at
a minimum it should be testing all the bases of the merge. Do you have
--first-parent set or something?
I would test cedc811c76778bdef91d405717acee0de54d8db5 (x86/amd) and
3613047280ec42a4e1350fdc1a6dd161ff4008cc (core) directly. Most likely
cedc will be bad problem.
If one of them is bad then restart the bisection with that as the bad
and 6e6c6d6bc6 as the good.
(or run bisect again with e8cca466a84a75f8ff2a7a31173c99ee6d1c59d2 as
the bad and 6e6c6d6bc6c96c2477ddfea24a121eb5ee12b7a3 as the good
without --first-parent)
Jason
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: Kernel 6.7 regression doesn't boot if using AMD eGPU
2024-04-15 16:30 ` Jason Gunthorpe
@ 2024-04-15 18:57 ` Eric Wagner
2024-04-15 21:44 ` Robin Murphy
0 siblings, 1 reply; 17+ messages in thread
From: Eric Wagner @ 2024-04-15 18:57 UTC (permalink / raw)
To: Jason Gunthorpe
Cc: Joerg Roedel, Will Deacon, Suravee Suthikulpanit, Robin Murphy,
iommu, linux-kernel
[-- Attachment #1.1: Type: text/plain, Size: 5081 bytes --]
Apologies if I made a mistake in the first bisect, I'm new to kernel
debugging.
I tested cedc811c76778bdef91d405717acee0de54d8db5 (x86/amd) and
3613047280ec42a4e1350fdc1a6dd161ff4008cc (core) directly and both were good.
Then I ran git bisect again with e8cca466a84a75f8ff2a7a31173c99ee6d1c59d2
as the bad and 6e6c6d6bc6c96c2477ddfea24a121eb5ee12b7a3 as the good and the
bisect log is attached. It ended up at the same commit as before.
I've also attached a picture of the boot screen that occurs when it hangs.
0000:05:00.0 is the PCIe bus address of the RX 580 eGPU that's causing the
problem.
On Mon, Apr 15, 2024 at 12:30 PM Jason Gunthorpe <jgg@ziepe.ca> wrote:
> On Sat, Apr 13, 2024 at 06:04:12PM -0400, Eric Wagner wrote:
> > On my Thinkpad T14s G3 AMD (Ryzen 7 6850U) laptop connected to an AMD
> > RX 580 in Akitio Node Thunderbolt 3 eGPU. Booting with the eGPU
> > connected hangs on kernels 6.7 and 6.8, but worked on 6.6. For
> > debugging, I find that adding the kernel parameter amd_iommu=off seems
> > to fix the issue and allows booting with the eGPU on 6.7.
> > I tried bisecting the issue between 6.6 and 6.7 and ended up with:
> > "e8cca466a84a75f8ff2a7a31173c99ee6d1c59d2 is the first bad commit" in
> > the attached. This seems to indicate an amd iommu issue.
> > Two others also reported the same issue on AMD Ryzen 7 7840 with AMD
> RX
> > 6000 connected as eGPU
> > ([1]https://gitlab.freedesktop.org/drm/amd/-/issues/3182).
> > Let me know if you need more information.
> >
> > References
> >
> > 1. https://gitlab.freedesktop.org/drm/amd/-/issues/3182
>
> > Bisecting: 366 revisions left to test after this (roughly 9 steps)
> > [74e9347ebc5be452935fe4f3eddb150aa5a6f4fe] Merge tag
> 'loongarch-fixes-6.6-3' of git://
> git.kernel.org/pub/scm/linux/kernel/git/chenhuacai/linux-loongson
> > Bisecting: 182 revisions left to test after this (roughly 8 steps)
> > [f6176471542d991137543af2ef1c18dae3286079] Merge tag
> 'mtd/fixes-for-6.6-rc7' of git://
> git.kernel.org/pub/scm/linux/kernel/git/mtd/linux
> > Bisecting: 87 revisions left to test after this (roughly 7 steps)
> > [fe3cfe869d5e0453754cf2b4c75110276b5e8527] Merge tag 'phy-fixes-6.6' of
> git://git.kernel.org/pub/scm/linux/kernel/git/phy/linux-phy
> > Bisecting: 43 revisions left to test after this (roughly 6 steps)
> > [c76c067e488ccd55734c3e750799caf2c5956db6] s390/pci: Use dma-iommu layer
> > Bisecting: 27 revisions left to test after this (roughly 5 steps)
> > [aa5cabc4ce8e6b45d170d162dc54b1bac1767c47] Merge tag 'arm-smmu-updates'
> of git://git.kernel.org/pub/scm/linux/kernel/git/will/linux into arm/smmu
> > Bisecting: 14 revisions left to test after this (roughly 4 steps)
> > [bbc70e0aec287e164344b1a071bd46466a4f29b3] iommu/dart: Remove the
> force_bypass variable
> > Bisecting: 9 revisions left to test after this (roughly 3 steps)
> > [e82c175e63229ea495a0a0b5305a98b5b6ee5346] Revert "iommu/vt-d: Remove
> unused function"
> > Bisecting: 5 revisions left to test after this (roughly 2 steps)
> > [92bce97f0c341d3037b0f364b6839483f6a41cae] s390/pci: Fix reset of IOMMU
> software counters
> > Bisecting: 3 revisions left to test after this (roughly 2 steps)
> > [3613047280ec42a4e1350fdc1a6dd161ff4008cc] Merge tag 'v6.6-rc7' into core
> > Bisecting: 2 revisions left to test after this (roughly 1 step)
> > [f7da9c081517daba70f9f9342e09d7a6322ba323] iommu/tegra-smmu: Drop
> unnecessary error check for for debugfs_create_dir()
> > Bisecting: 1 revision left to test after this (roughly 1 step)
> > [9e13ec61de2a51195b122a79461431d8cb99d7b5] iommu/virtio: Add
> __counted_by for struct viommu_request and use struct_size()
> > Bisecting: 0 revisions left to test after this (roughly 0 steps)
> > [6e6c6d6bc6c96c2477ddfea24a121eb5ee12b7a3] iommu: Avoid unnecessary
> cache invalidations
> > e8cca466a84a75f8ff2a7a31173c99ee6d1c59d2 is the first bad commit
> > commit e8cca466a84a75f8ff2a7a31173c99ee6d1c59d2
> > Merge: 6e6c6d6bc6 f7da9c0815 aa5cabc4ce 9e13ec61de e82c175e63 cedc811c76
> 3613047280 92bce97f0c
> > Author: Joerg Roedel <jroedel@suse.de>
> > Date: Fri Oct 27 09:13:40 2023 +0200
> >
> > Merge branches 'iommu/fixes', 'arm/tegra', 'arm/smmu', 'virtio',
> 'x86/vt-d', 'x86/amd', 'core' and 's390' into next
>
> Do you have the good/bad log on this? It doesn't look like bisect
> tested enough stuff to really conclude the merge is the bad thing, at
> a minimum it should be testing all the bases of the merge. Do you have
> --first-parent set or something?
>
> I would test cedc811c76778bdef91d405717acee0de54d8db5 (x86/amd) and
> 3613047280ec42a4e1350fdc1a6dd161ff4008cc (core) directly. Most likely
> cedc will be bad problem.
>
> If one of them is bad then restart the bisection with that as the bad
> and 6e6c6d6bc6 as the good.
>
> (or run bisect again with e8cca466a84a75f8ff2a7a31173c99ee6d1c59d2 as
> the bad and 6e6c6d6bc6c96c2477ddfea24a121eb5ee12b7a3 as the good
> without --first-parent)
>
> Jason
>
[-- Attachment #1.2: Type: text/html, Size: 6512 bytes --]
[-- Attachment #2: bisect2.log --]
[-- Type: text/x-log, Size: 1945 bytes --]
git bisect start
# bad: [e8cca466a84a75f8ff2a7a31173c99ee6d1c59d2] Merge branches 'iommu/fixes', 'arm/tegra', 'arm/smmu', 'virtio', 'x86/vt-d', 'x86/amd', 'core' and 's390' into next
git bisect bad e8cca466a84a75f8ff2a7a31173c99ee6d1c59d2
# good: [6e6c6d6bc6c96c2477ddfea24a121eb5ee12b7a3] iommu: Avoid unnecessary cache invalidations
git bisect good 6e6c6d6bc6c96c2477ddfea24a121eb5ee12b7a3
# good: [482feb5c649261cd2a7ad02e4ca63c159d6ec795] iommu/dart: Call apple_dart_finalize_domain() as part of alloc_paging()
git bisect good 482feb5c649261cd2a7ad02e4ca63c159d6ec795
# good: [cedc811c76778bdef91d405717acee0de54d8db5] iommu/amd: Remove DMA_FQ type from domain allocation path
git bisect good cedc811c76778bdef91d405717acee0de54d8db5
# good: [aa5cabc4ce8e6b45d170d162dc54b1bac1767c47] Merge tag 'arm-smmu-updates' of git://git.kernel.org/pub/scm/linux/kernel/git/will/linux into arm/smmu
git bisect good aa5cabc4ce8e6b45d170d162dc54b1bac1767c47
# good: [92bce97f0c341d3037b0f364b6839483f6a41cae] s390/pci: Fix reset of IOMMU software counters
git bisect good 92bce97f0c341d3037b0f364b6839483f6a41cae
# good: [e82c175e63229ea495a0a0b5305a98b5b6ee5346] Revert "iommu/vt-d: Remove unused function"
git bisect good e82c175e63229ea495a0a0b5305a98b5b6ee5346
# good: [3613047280ec42a4e1350fdc1a6dd161ff4008cc] Merge tag 'v6.6-rc7' into core
git bisect good 3613047280ec42a4e1350fdc1a6dd161ff4008cc
# good: [f7da9c081517daba70f9f9342e09d7a6322ba323] iommu/tegra-smmu: Drop unnecessary error check for for debugfs_create_dir()
git bisect good f7da9c081517daba70f9f9342e09d7a6322ba323
# good: [9e13ec61de2a51195b122a79461431d8cb99d7b5] iommu/virtio: Add __counted_by for struct viommu_request and use struct_size()
git bisect good 9e13ec61de2a51195b122a79461431d8cb99d7b5
# first bad commit: [e8cca466a84a75f8ff2a7a31173c99ee6d1c59d2] Merge branches 'iommu/fixes', 'arm/tegra', 'arm/smmu', 'virtio', 'x86/vt-d', 'x86/amd', 'core' and 's390' into next
[-- Attachment #3: 20240415_133212.jpg --]
[-- Type: image/jpeg, Size: 2084419 bytes --]
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: Kernel 6.7 regression doesn't boot if using AMD eGPU
2024-04-15 18:57 ` Eric Wagner
@ 2024-04-15 21:44 ` Robin Murphy
2024-04-16 0:39 ` Jason Gunthorpe
0 siblings, 1 reply; 17+ messages in thread
From: Robin Murphy @ 2024-04-15 21:44 UTC (permalink / raw)
To: Eric Wagner, Jason Gunthorpe
Cc: Joerg Roedel, Will Deacon, Suravee Suthikulpanit, iommu, linux-kernel
On 2024-04-15 7:57 pm, Eric Wagner wrote:
> Apologies if I made a mistake in the first bisect, I'm new to kernel
> debugging.
>
> I tested cedc811c76778bdef91d405717acee0de54d8db5 (x86/amd) and
> 3613047280ec42a4e1350fdc1a6dd161ff4008cc (core) directly and both were good.
> Then I ran git bisect again with e8cca466a84a75f8ff2a7a31173c99ee6d1c59d2
> as the bad and 6e6c6d6bc6c96c2477ddfea24a121eb5ee12b7a3 as the good and the
> bisect log is attached. It ended up at the same commit as before.
>
> I've also attached a picture of the boot screen that occurs when it hangs.
> 0000:05:00.0 is the PCIe bus address of the RX 580 eGPU that's causing the
> problem.
Looks like 59ddce4418da483 probably broke things most - prior to that,
the fact that it's behind a Thunderbolt port would have always taken
precedence and forced IOMMU_DOMAIN_DMA regardless of what the driver may
have wanted to say, whereas now we ask the driver first, then complain
that it conflicts with the untrusted status and ultimately don't
configure the IOMMU at all. Meanwhile the GPU driver presumably goes on
to believe it's using dma-direct with no IOMMU present, resulting in
fireworks when its traffic reaches the IOMMU. Great :(
However the other notable thing that also happened between 6.6 and 6.7
was the removal of the AMD iommu_v2 code, so there's some possibility
that the GPU driver still may have only been working before due to that
also subverting the default domain with its own identity domain, so
whether it would actually work again with
iommu_get_default_domain_type() sorted out is yet another question... As
a first step I'd test the quick hack below, but be prepared for things
to still break slightly differently.
Cheers,
Robin.
----->8-----
diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
index 996e79dc582d..063e1eb32fbd 100644
--- a/drivers/iommu/iommu.c
+++ b/drivers/iommu/iommu.c
@@ -1774,7 +1774,7 @@ static int iommu_get_default_domain_type(struct
iommu_group *group,
untrusted,
"Device is not trusted, but driver is overriding group %u to %s,
refusing to probe.\n",
group->id, iommu_domain_type_str(driver_type));
- return -1;
+ //return -1;
}
driver_type = IOMMU_DOMAIN_DMA;
}
^ permalink raw reply related [flat|nested] 17+ messages in thread
* Re: Kernel 6.7 regression doesn't boot if using AMD eGPU
2024-04-15 21:44 ` Robin Murphy
@ 2024-04-16 0:39 ` Jason Gunthorpe
[not found] ` <CAHudX3x-ofB=-K6UwWpf4r7Ge9AQbvLQ2qq2C7N-R5LG2qMz7Q@mail.gmail.com>
2024-04-16 11:25 ` Robin Murphy
0 siblings, 2 replies; 17+ messages in thread
From: Jason Gunthorpe @ 2024-04-16 0:39 UTC (permalink / raw)
To: Robin Murphy, Vasant Hegde
Cc: Eric Wagner, Joerg Roedel, Will Deacon, Suravee Suthikulpanit,
iommu, linux-kernel
On Mon, Apr 15, 2024 at 10:44:34PM +0100, Robin Murphy wrote:
> On 2024-04-15 7:57 pm, Eric Wagner wrote:
> > Apologies if I made a mistake in the first bisect, I'm new to kernel
> > debugging.
> >
> > I tested cedc811c76778bdef91d405717acee0de54d8db5 (x86/amd) and
> > 3613047280ec42a4e1350fdc1a6dd161ff4008cc (core) directly and both were good.
> > Then I ran git bisect again with e8cca466a84a75f8ff2a7a31173c99ee6d1c59d2
> > as the bad and 6e6c6d6bc6c96c2477ddfea24a121eb5ee12b7a3 as the good and the
> > bisect log is attached. It ended up at the same commit as before.
> >
> > I've also attached a picture of the boot screen that occurs when it hangs.
> > 0000:05:00.0 is the PCIe bus address of the RX 580 eGPU that's causing the
> > problem.
>
> Looks like 59ddce4418da483 probably broke things most - prior to that, the
> fact that it's behind a Thunderbolt port would have always taken precedence
> and forced IOMMU_DOMAIN_DMA regardless of what the driver may have wanted to
> saywhereas now we ask the driver first, then complain that it conflicts
> with the untrusted status and ultimately don't configure the IOMMU
> at all.
Yes, if the driver wants to force a domain type it should be
forced. Driver knows best. Eg AMD forces IDENTITY when the HW/driver
is incapable of supporting otherwise.
> Meanwhile the GPU driver presumably goes on to believe it's using dma-direct
> with no IOMMU present, resulting in fireworks when its traffic reaches the
> IOMMU. Great :(
I wonder where is the missing error handling.. iommu probe failure
should not go on to allow driver attach, there is no guarentee any DMA
works now that many iommus are booting up in BLOCKED.
> However the other notable thing that also happened between 6.6 and 6.7 was
> the removal of the AMD iommu_v2 code, so there's some possibility that the
> GPU driver still may have only been working before due to that also
Most likely it is the above change interacting with this patch when
they are both combined in the merge:
commit 92e2bd56a5f9fc44313fda802a43a63cc2a9c8f6
Author: Vasant Hegde <vasant.hegde@amd.com>
Date: Thu Sep 21 09:21:45 2023 +0000
iommu/amd: Introduce iommu_dev_data.flags to track device capabilities
@@ -2471,7 +2481,7 @@ static int amd_iommu_def_domain_type(struct device *dev)
* and require remapping.
* - SNP is enabled, because it prohibits DTE[Mode]=0.
*/
- if (dev_data->iommu_v2 &&
+ if (pdev_pasid_supported(dev_data) &&
!cc_platform_has(CC_ATTR_MEM_ENCRYPT) &&
!amd_iommu_snp_en) {
return IOMMU_DOMAIN_IDENTITY;
Which, IIRC, was intended to be temporary to work around limitations
in the DTE programming logic within the driver. Previously iommu_v2 as
a module option that Eric probably doesn't set, I guess.
The below will probably make it boot, but Vasant should check what
happens if PASID is eventually attached too.
diff --git a/drivers/iommu/amd/iommu.c b/drivers/iommu/amd/iommu.c
index d35c1b8c8e65ce..f3da6a5b6cb1cb 100644
--- a/drivers/iommu/amd/iommu.c
+++ b/drivers/iommu/amd/iommu.c
@@ -2758,11 +2758,16 @@ static int amd_iommu_def_domain_type(struct device *dev)
* and require remapping.
* - SNP is enabled, because it prohibits DTE[Mode]=0.
*/
- if (pdev_pasid_supported(dev_data) &&
- !cc_platform_has(CC_ATTR_MEM_ENCRYPT) &&
- !amd_iommu_snp_en) {
+ if (!cc_platform_has(CC_ATTR_MEM_ENCRYPT) && !amd_iommu_snp_en)
+ return IOMMU_DOMAIN_IDENTITY;
+
+ /*
+ * For now driver limitations prevent enabling PASID as a paging domain
+ * on the RID together.
+ */
+ if (dev_is_pci(dev) && !to_pci_dev(dev)->untrusted &&
+ pdev_pasid_supported(dev_data))
return IOMMU_DOMAIN_IDENTITY;
- }
return 0;
}
Jason
^ permalink raw reply related [flat|nested] 17+ messages in thread
* Re: Kernel 6.7 regression doesn't boot if using AMD eGPU
[not found] ` <CAHudX3x-ofB=-K6UwWpf4r7Ge9AQbvLQ2qq2C7N-R5LG2qMz7Q@mail.gmail.com>
@ 2024-04-16 10:53 ` Vasant Hegde
2024-04-16 11:39 ` Jason Gunthorpe
2024-04-17 2:16 ` Eric Wagner
0 siblings, 2 replies; 17+ messages in thread
From: Vasant Hegde @ 2024-04-16 10:53 UTC (permalink / raw)
To: Eric Wagner, Jason Gunthorpe
Cc: Robin Murphy, Vasant Hegde, Joerg Roedel, Will Deacon,
Suravee Suthikulpanit, iommu, linux-kernel
Hi Eric,
On 4/16/2024 7:42 AM, Eric Wagner wrote:
> On Mon, Apr 15, 2024 at 5:44 PM Robin Murphy <robin.murphy@arm.com
> <mailto:robin.murphy@arm.com>> wrote:
>
> As a first step I'd test the quick hack below, but be prepared for things
> to still break slightly differently.
>
> Cheers,
> Robin.
>
> ----->8-----
> diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
> index 996e79dc582d..063e1eb32fbd 100644
> --- a/drivers/iommu/iommu.c
> +++ b/drivers/iommu/iommu.c
> @@ -1774,7 +1774,7 @@ static int iommu_get_default_domain_type(struct
> iommu_group *group,
> untrusted,
> "Device is not trusted, but driver is
> overriding group %u to %s,
> refusing to probe.\n",
> group->id, iommu_domain_type_str(driver_type));
> - return -1;
> + //return -1;
> }
> driver_type = IOMMU_DOMAIN_DMA;
> }
>
> This worked and got the system booting for me.
Thanks for testing.
IIUC eGPU is behind Thunderbolt and hence IOMMU treated it as 'untrusted device'.
AMD driver tries to allocate "IDENTITY" domain for GPU devices.
iommu_get_default_domain_type() return -1 as it expects IOMMU_DOMAIN_DMA for
untrusted device.
Can you please attach lspci -vvv output? Also /proc/cmdline output.
>
> On Mon, Apr 15, 2024 at 8:39 PM Jason Gunthorpe <jgg@ziepe.ca
> <mailto:jgg@ziepe.ca>> wrote:
>
> On Mon, Apr 15, 2024 at 10:44:34PM +0100, Robin Murphy wrote:
> > On 2024-04-15 7:57 pm, Eric Wagner wrote:
> > > Apologies if I made a mistake in the first bisect, I'm new to kernel
> > > debugging.
> > >
> > > I tested cedc811c76778bdef91d405717acee0de54d8db5 (x86/amd) and
> > > 3613047280ec42a4e1350fdc1a6dd161ff4008cc (core) directly and both were
> good.
> > > Then I ran git bisect again with e8cca466a84a75f8ff2a7a31173c99ee6d1c59d2
> > > as the bad and 6e6c6d6bc6c96c2477ddfea24a121eb5ee12b7a3 as the good and the
> > > bisect log is attached. It ended up at the same commit as before.
> > >
> > > I've also attached a picture of the boot screen that occurs when it hangs.
> > > 0000:05:00.0 is the PCIe bus address of the RX 580 eGPU that's causing the
> > > problem.
> >
> > Looks like 59ddce4418da483 probably broke things most - prior to that, the
> > fact that it's behind a Thunderbolt port would have always taken precedence
> > and forced IOMMU_DOMAIN_DMA regardless of what the driver may have wanted to
> > saywhereas now we ask the driver first, then complain that it conflicts
> > with the untrusted status and ultimately don't configure the IOMMU
> > at all.
>
> Yes, if the driver wants to force a domain type it should be
> forced. Driver knows best. Eg AMD forces IDENTITY when the HW/driver
> is incapable of supporting otherwise.
@Jason,
Looks like before commit 59ddce4418da483, core IOMMU layer was enforcing
'IOMMU_DOMAIN_DMA' for untrusted device. If its trusted device then it was
letting HW IOMMU driver to pick best domain type.
Did we change that flow? Are we expecting HW IOMMU driver to handler untrusted
devices as well?
>
> > Meanwhile the GPU driver presumably goes on to believe it's using dma-direct
> > with no IOMMU present, resulting in fireworks when its traffic reaches the
> > IOMMU. Great :(
>
> I wonder where is the missing error handling.. iommu probe failure
> should not go on to allow driver attach, there is no guarentee any DMA
> works now that many iommus are booting up in BLOCKED.
Looking into code path, in failure path we cleanup device, not group. May be
that's causing issue? Not sure where it failed. If I manage to find some system
I will try to debug further.
>
> > However the other notable thing that also happened between 6.6 and 6.7 was
> > the removal of the AMD iommu_v2 code, so there's some possibility that the
> > GPU driver still may have only been working before due to that also
>
> Most likely it is the above change interacting with this patch when
> they are both combined in the merge:
>
> commit 92e2bd56a5f9fc44313fda802a43a63cc2a9c8f6
> Author: Vasant Hegde <vasant.hegde@amd.com <mailto:vasant.hegde@amd.com>>
> Date: Thu Sep 21 09:21:45 2023 +0000
>
> iommu/amd: Introduce iommu_dev_data.flags to track device capabilities
>
> @@ -2471,7 +2481,7 @@ static int amd_iommu_def_domain_type(struct device *dev)
> * and require remapping.
> * - SNP is enabled, because it prohibits DTE[Mode]=0.
> */
> - if (dev_data->iommu_v2 &&
> + if (pdev_pasid_supported(dev_data) &&
> !cc_platform_has(CC_ATTR_MEM_ENCRYPT) &&
> !amd_iommu_snp_en) {
> return IOMMU_DOMAIN_IDENTITY;
>
> Which, IIRC, was intended to be temporary to work around limitations
> in the DTE programming logic within the driver. Previously iommu_v2 as
> a module option that Eric probably doesn't set, I guess.
>
> The below will probably make it boot, but Vasant should check what
> happens if PASID is eventually attached too.
>
> diff --git a/drivers/iommu/amd/iommu.c b/drivers/iommu/amd/iommu.c
> index d35c1b8c8e65ce..f3da6a5b6cb1cb 100644
> --- a/drivers/iommu/amd/iommu.c
> +++ b/drivers/iommu/amd/iommu.c
> @@ -2758,11 +2758,16 @@ static int amd_iommu_def_domain_type(struct device *dev)
> * and require remapping.
> * - SNP is enabled, because it prohibits DTE[Mode]=0.
> */
> - if (pdev_pasid_supported(dev_data) &&
> - !cc_platform_has(CC_ATTR_MEM_ENCRYPT) &&
> - !amd_iommu_snp_en) {
> + if (!cc_platform_has(CC_ATTR_MEM_ENCRYPT) && !amd_iommu_snp_en)
> + return IOMMU_DOMAIN_IDENTITY;
> +
> + /*
> + * For now driver limitations prevent enabling PASID as a paging domain
> + * on the RID together.
> + */
> + if (dev_is_pci(dev) && !to_pci_dev(dev)->untrusted &&
> + pdev_pasid_supported(dev_data))
> return IOMMU_DOMAIN_IDENTITY;
> - }
>
> return 0;
> }
>
> Jason
>
>
> As it booted ok with Robin's patch above, these changes to
> drivers/iommu/amd/iommu.c didn't seem to make a difference for me. I was testing
> with amd iommu v2 off in the kernel config and I also have TSME enabled in the
> BIOS if that matters.
TMSE is transparent to OS. So its fine.
-Vasant
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: Kernel 6.7 regression doesn't boot if using AMD eGPU
2024-04-16 0:39 ` Jason Gunthorpe
[not found] ` <CAHudX3x-ofB=-K6UwWpf4r7Ge9AQbvLQ2qq2C7N-R5LG2qMz7Q@mail.gmail.com>
@ 2024-04-16 11:25 ` Robin Murphy
2024-04-16 11:49 ` Jason Gunthorpe
2024-04-16 12:44 ` Vasant Hegde
1 sibling, 2 replies; 17+ messages in thread
From: Robin Murphy @ 2024-04-16 11:25 UTC (permalink / raw)
To: Jason Gunthorpe, Vasant Hegde
Cc: Eric Wagner, Joerg Roedel, Will Deacon, Suravee Suthikulpanit,
iommu, linux-kernel
On 2024-04-16 1:39 am, Jason Gunthorpe wrote:
> On Mon, Apr 15, 2024 at 10:44:34PM +0100, Robin Murphy wrote:
>> On 2024-04-15 7:57 pm, Eric Wagner wrote:
>>> Apologies if I made a mistake in the first bisect, I'm new to kernel
>>> debugging.
>>>
>>> I tested cedc811c76778bdef91d405717acee0de54d8db5 (x86/amd) and
>>> 3613047280ec42a4e1350fdc1a6dd161ff4008cc (core) directly and both were good.
>>> Then I ran git bisect again with e8cca466a84a75f8ff2a7a31173c99ee6d1c59d2
>>> as the bad and 6e6c6d6bc6c96c2477ddfea24a121eb5ee12b7a3 as the good and the
>>> bisect log is attached. It ended up at the same commit as before.
>>>
>>> I've also attached a picture of the boot screen that occurs when it hangs.
>>> 0000:05:00.0 is the PCIe bus address of the RX 580 eGPU that's causing the
>>> problem.
>>
>> Looks like 59ddce4418da483 probably broke things most - prior to that, the
>> fact that it's behind a Thunderbolt port would have always taken precedence
>> and forced IOMMU_DOMAIN_DMA regardless of what the driver may have wanted to
>> saywhereas now we ask the driver first, then complain that it conflicts
>> with the untrusted status and ultimately don't configure the IOMMU
>> at all.
>
> Yes, if the driver wants to force a domain type it should be
> forced. Driver knows best. Eg AMD forces IDENTITY when the HW/driver
> is incapable of supporting otherwise.
No, in the case of AMD it only forces identity if it thinks the device
might want to use PASIDs (because of the architectural limitation that
the RID always operates in GPA space so can't have its own independent
translation).
Either way, though, there's really little sense to that argument - if
enforcing strict translation *might* compromise the device's
functionality, we should instead go out of our way to ensure it's
definitely as broken as possible? I fail to see how that can be
justified as useful or desirable behaviour.
>> Meanwhile the GPU driver presumably goes on to believe it's using dma-direct
>> with no IOMMU present, resulting in fireworks when its traffic reaches the
>> IOMMU. Great :(
>
> I wonder where is the missing error handling.. iommu probe failure
> should not go on to allow driver attach, there is no guarentee any DMA
> works now that many iommus are booting up in BLOCKED.
What do you mean error handling? After you spent a year rewriting the
probing code to your own grand design, don't suggest you don't even know
how it fundamentally works...
"Failing" iommu_probe_device is merely how we tell ourselves that we're
not interested in a device, and consequently tell the rest of the kernel
it doesn't have an IOMMU (via device_iommu_mapped() returning false).
This is normal and expected for devices which legitimately have no IOMMU
in the first place; conversely we don't do a great deal for unexpected
failures since those typically represent system-fatal conditions
whatever we might try to do. We've never had much of a notion of
expected failures when an IOMMU *is* present, but even then, denying any
trace of the IOMMU and removing ourselves from the picture is clearly
not the ideal way to approach that. We're running off a bus notifier (or
even later), so ultimately our return value is meaningless; at that
point the device already exists and has been added to its bus, we can't
undo that.
However it looks to be even more fun if failure occurs in *deferred*
default domain creation via bus_iommu_probe(), since then we give up and
dismiss the entire IOMMU. Except the x86 drivers ignore the return from
iommu_device_register(), so further hilarity ensues...
I think I've now satisfied myself that a simple fix for the core code is
appropriate and will write that up now; one other thing I couldn't quite
figure out is whether the AMD driver somehow prevents PASIDs being used
while the group is attached to a non-identity (and non-nested) domain -
that's probably one for Vasant to confirm.
Thanks,
Robin.
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: Kernel 6.7 regression doesn't boot if using AMD eGPU
2024-04-16 10:53 ` Vasant Hegde
@ 2024-04-16 11:39 ` Jason Gunthorpe
2024-04-17 2:16 ` Eric Wagner
1 sibling, 0 replies; 17+ messages in thread
From: Jason Gunthorpe @ 2024-04-16 11:39 UTC (permalink / raw)
To: Vasant Hegde
Cc: Eric Wagner, Robin Murphy, Vasant Hegde, Joerg Roedel,
Will Deacon, Suravee Suthikulpanit, iommu, linux-kernel
On Tue, Apr 16, 2024 at 04:23:38PM +0530, Vasant Hegde wrote:
> > Yes, if the driver wants to force a domain type it should be
> > forced. Driver knows best. Eg AMD forces IDENTITY when the HW/driver
> > is incapable of supporting otherwise.
>
>
> @Jason,
>
> Looks like before commit 59ddce4418da483, core IOMMU layer was enforcing
> 'IOMMU_DOMAIN_DMA' for untrusted device. If its trusted device then it was
> letting HW IOMMU driver to pick best domain type.
If the driver wants to force identity because paging doesn't work then
yes it needs to figure something out..
Really the drivers should not be designed to do this, they need to
accommodate paging domains in all cases if things are going to work
correctly. The def_domain callback should be a last resort.
> > diff --git a/drivers/iommu/amd/iommu.c b/drivers/iommu/amd/iommu.c
> > index d35c1b8c8e65ce..f3da6a5b6cb1cb 100644
> > --- a/drivers/iommu/amd/iommu.c
> > +++ b/drivers/iommu/amd/iommu.c
> > @@ -2758,11 +2758,16 @@ static int amd_iommu_def_domain_type(struct device *dev)
> > * and require remapping.
> > * - SNP is enabled, because it prohibits DTE[Mode]=0.
> > */
> > - if (pdev_pasid_supported(dev_data) &&
> > - !cc_platform_has(CC_ATTR_MEM_ENCRYPT) &&
> > - !amd_iommu_snp_en) {
> > + if (!cc_platform_has(CC_ATTR_MEM_ENCRYPT) && !amd_iommu_snp_en)
> > + return IOMMU_DOMAIN_IDENTITY;
> > +
> > + /*
> > + * For now driver limitations prevent enabling PASID as a paging domain
> > + * on the RID together.
> > + */
> > + if (dev_is_pci(dev) && !to_pci_dev(dev)->untrusted &&
> > + pdev_pasid_supported(dev_data))
> > return IOMMU_DOMAIN_IDENTITY;
> > - }
> >
> > return 0;
> > }
> >
> >
> > As it booted ok with Robin's patch above, these changes to
> > drivers/iommu/amd/iommu.c didn't seem to make a difference for me. I was
> > testing with amd iommu v2 off in the kernel config and I also have TSME
> > enabled in the BIOS if that matters.
There must be a mistake in the above then, it would be good to sort it
out because something like that is the right fix.
Jason
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: Kernel 6.7 regression doesn't boot if using AMD eGPU
2024-04-16 11:25 ` Robin Murphy
@ 2024-04-16 11:49 ` Jason Gunthorpe
2024-04-17 8:48 ` Vasant Hegde
2024-04-16 12:44 ` Vasant Hegde
1 sibling, 1 reply; 17+ messages in thread
From: Jason Gunthorpe @ 2024-04-16 11:49 UTC (permalink / raw)
To: Robin Murphy
Cc: Vasant Hegde, Eric Wagner, Joerg Roedel, Will Deacon,
Suravee Suthikulpanit, iommu, linux-kernel
On Tue, Apr 16, 2024 at 12:25:52PM +0100, Robin Murphy wrote:
> On 2024-04-16 1:39 am, Jason Gunthorpe wrote:
> > On Mon, Apr 15, 2024 at 10:44:34PM +0100, Robin Murphy wrote:
> > > On 2024-04-15 7:57 pm, Eric Wagner wrote:
> > > > Apologies if I made a mistake in the first bisect, I'm new to kernel
> > > > debugging.
> > > >
> > > > I tested cedc811c76778bdef91d405717acee0de54d8db5 (x86/amd) and
> > > > 3613047280ec42a4e1350fdc1a6dd161ff4008cc (core) directly and both were good.
> > > > Then I ran git bisect again with e8cca466a84a75f8ff2a7a31173c99ee6d1c59d2
> > > > as the bad and 6e6c6d6bc6c96c2477ddfea24a121eb5ee12b7a3 as the good and the
> > > > bisect log is attached. It ended up at the same commit as before.
> > > >
> > > > I've also attached a picture of the boot screen that occurs when it hangs.
> > > > 0000:05:00.0 is the PCIe bus address of the RX 580 eGPU that's causing the
> > > > problem.
> > >
> > > Looks like 59ddce4418da483 probably broke things most - prior to that, the
> > > fact that it's behind a Thunderbolt port would have always taken precedence
> > > and forced IOMMU_DOMAIN_DMA regardless of what the driver may have wanted to
> > > saywhereas now we ask the driver first, then complain that it conflicts
> > > with the untrusted status and ultimately don't configure the IOMMU
> > > at all.
> >
> > Yes, if the driver wants to force a domain type it should be
> > forced. Driver knows best. Eg AMD forces IDENTITY when the HW/driver
> > is incapable of supporting otherwise.
>
> No, in the case of AMD it only forces identity if it thinks the device might
> want to use PASIDs (because of the architectural limitation that the RID
> always operates in GPA space so can't have its own independent translation).
AMD forces this because it doesn't yet have a way to automatically
choose it's v1/v2 page table format during alloc domain. It is just a
SW bug.
The CC/SNP limitation is also a SW bug but is more fatal as it can't
even attach a v1 page table in this mode.
> Either way, though, there's really little sense to that argument - if
> enforcing strict translation *might* compromise the device's functionality,
> we should instead go out of our way to ensure it's definitely as broken as
> possible? I fail to see how that can be justified as useful or desirable
> behaviour.
For SNP cases the attach of a DMA domain will fail, so yes, moving the
failure earlier and giving a clear message is desirable.
> "Failing" iommu_probe_device is merely how we tell ourselves that we're not
> interested in a device, and consequently tell the rest of the kernel it
> doesn't have an IOMMU (via device_iommu_mapped() returning false).
Probing failing with ENODEV means the device has no iommu and the rest
of the code should assume DMA direct will work.
Probing failing with any other error code means the device has an
iommu and it couldn't be setup. DMA direct probably won't work today.
If you want all failure codes to mean the device is safe for DMA
direct then we need to try and attach the IDENTITY domain on various
probe failure paths too.
> I think I've now satisfied myself that a simple fix for the core code is
> appropriate and will write that up now; one other thing I couldn't
> quite
It really doesn't match the design here.
Jason
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: Kernel 6.7 regression doesn't boot if using AMD eGPU
2024-04-16 11:25 ` Robin Murphy
2024-04-16 11:49 ` Jason Gunthorpe
@ 2024-04-16 12:44 ` Vasant Hegde
2024-04-17 10:36 ` Robin Murphy
1 sibling, 1 reply; 17+ messages in thread
From: Vasant Hegde @ 2024-04-16 12:44 UTC (permalink / raw)
To: Robin Murphy, Jason Gunthorpe
Cc: Eric Wagner, Joerg Roedel, Will Deacon, Suravee Suthikulpanit,
iommu, linux-kernel
Robin,
On 4/16/2024 4:55 PM, Robin Murphy wrote:
> On 2024-04-16 1:39 am, Jason Gunthorpe wrote:
>> On Mon, Apr 15, 2024 at 10:44:34PM +0100, Robin Murphy wrote:
>>> On 2024-04-15 7:57 pm, Eric Wagner wrote:
>>>> Apologies if I made a mistake in the first bisect, I'm new to kernel
>>>> debugging.
>>>>
>>>> I tested cedc811c76778bdef91d405717acee0de54d8db5 (x86/amd) and
>>>> 3613047280ec42a4e1350fdc1a6dd161ff4008cc (core) directly and both were good.
>>>> Then I ran git bisect again with e8cca466a84a75f8ff2a7a31173c99ee6d1c59d2
>>>> as the bad and 6e6c6d6bc6c96c2477ddfea24a121eb5ee12b7a3 as the good and the
>>>> bisect log is attached. It ended up at the same commit as before.
>>>>
>>>> I've also attached a picture of the boot screen that occurs when it hangs.
>>>> 0000:05:00.0 is the PCIe bus address of the RX 580 eGPU that's causing the
>>>> problem.
.../...
>
> "Failing" iommu_probe_device is merely how we tell ourselves that we're not
> interested in a device, and consequently tell the rest of the kernel it doesn't
> have an IOMMU (via device_iommu_mapped() returning false). This is normal and
> expected for devices which legitimately have no IOMMU in the first place;
> conversely we don't do a great deal for unexpected failures since those
> typically represent system-fatal conditions whatever we might try to do. We've
> never had much of a notion of expected failures when an IOMMU *is* present, but
> even then, denying any trace of the IOMMU and removing ourselves from the
> picture is clearly not the ideal way to approach that. We're running off a bus
> notifier (or even later), so ultimately our return value is meaningless; at that
> point the device already exists and has been added to its bus, we can't undo that.
>
> However it looks to be even more fun if failure occurs in *deferred* default
> domain creation via bus_iommu_probe(), since then we give up and dismiss the
> entire IOMMU. Except the x86 drivers ignore the return from
> iommu_device_register(), so further hilarity ensues...
>
> I think I've now satisfied myself that a simple fix for the core code is
> appropriate and will write that up now; one other thing I couldn't quite figure
> out is whether the AMD driver somehow prevents PASIDs being used while the group
> is attached to a non-identity (and non-nested) domain - that's probably one for
> Vasant to confirm.
AMD driver supports PASID with below domain type :
- Identity domain
- DMA translation mode (DMA and DMA_FQ) with AMD v2 page table
(amd_iommu=pgtbl_v2).
Currently amd_iommu_def_domain_type() tries to put PASID capable devices in
identity domain mode. This is something to fix. Its in my TODO list. I will try
to get into it soon.
Hope this clarifies.
-Vasant
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: Kernel 6.7 regression doesn't boot if using AMD eGPU
2024-04-16 10:53 ` Vasant Hegde
2024-04-16 11:39 ` Jason Gunthorpe
@ 2024-04-17 2:16 ` Eric Wagner
2024-04-17 8:57 ` Vasant Hegde
1 sibling, 1 reply; 17+ messages in thread
From: Eric Wagner @ 2024-04-17 2:16 UTC (permalink / raw)
To: Vasant Hegde
Cc: Jason Gunthorpe, Robin Murphy, Vasant Hegde, Joerg Roedel,
Will Deacon, Suravee Suthikulpanit, iommu, linux-kernel
[-- Attachment #1: Type: text/plain, Size: 7427 bytes --]
On Tue, Apr 16, 2024 at 6:53 AM Vasant Hegde <vashegde@amd.com> wrote:
>
> Hi Eric,
>
>
> On 4/16/2024 7:42 AM, Eric Wagner wrote:
> > On Mon, Apr 15, 2024 at 5:44 PM Robin Murphy <robin.murphy@arm.com
> > <mailto:robin.murphy@arm.com>> wrote:
> >
> > As a first step I'd test the quick hack below, but be prepared for things
> > to still break slightly differently.
> >
> > Cheers,
> > Robin.
> >
> > ----->8-----
> > diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
> > index 996e79dc582d..063e1eb32fbd 100644
> > --- a/drivers/iommu/iommu.c
> > +++ b/drivers/iommu/iommu.c
> > @@ -1774,7 +1774,7 @@ static int iommu_get_default_domain_type(struct
> > iommu_group *group,
> > untrusted,
> > "Device is not trusted, but driver is
> > overriding group %u to %s,
> > refusing to probe.\n",
> > group->id, iommu_domain_type_str(driver_type));
> > - return -1;
> > + //return -1;
> > }
> > driver_type = IOMMU_DOMAIN_DMA;
> > }
> >
> > This worked and got the system booting for me.
>
> Thanks for testing.
>
> IIUC eGPU is behind Thunderbolt and hence IOMMU treated it as 'untrusted device'.
>
> AMD driver tries to allocate "IDENTITY" domain for GPU devices.
> iommu_get_default_domain_type() return -1 as it expects IOMMU_DOMAIN_DMA for
> untrusted device.
>
> Can you please attach lspci -vvv output? Also /proc/cmdline output.
>
>
> >
> > On Mon, Apr 15, 2024 at 8:39 PM Jason Gunthorpe <jgg@ziepe.ca
> > <mailto:jgg@ziepe.ca>> wrote:
> >
> > On Mon, Apr 15, 2024 at 10:44:34PM +0100, Robin Murphy wrote:
> > > On 2024-04-15 7:57 pm, Eric Wagner wrote:
> > > > Apologies if I made a mistake in the first bisect, I'm new to kernel
> > > > debugging.
> > > >
> > > > I tested cedc811c76778bdef91d405717acee0de54d8db5 (x86/amd) and
> > > > 3613047280ec42a4e1350fdc1a6dd161ff4008cc (core) directly and both were
> > good.
> > > > Then I ran git bisect again with e8cca466a84a75f8ff2a7a31173c99ee6d1c59d2
> > > > as the bad and 6e6c6d6bc6c96c2477ddfea24a121eb5ee12b7a3 as the good and the
> > > > bisect log is attached. It ended up at the same commit as before.
> > > >
> > > > I've also attached a picture of the boot screen that occurs when it hangs.
> > > > 0000:05:00.0 is the PCIe bus address of the RX 580 eGPU that's causing the
> > > > problem.
> > >
> > > Looks like 59ddce4418da483 probably broke things most - prior to that, the
> > > fact that it's behind a Thunderbolt port would have always taken precedence
> > > and forced IOMMU_DOMAIN_DMA regardless of what the driver may have wanted to
> > > saywhereas now we ask the driver first, then complain that it conflicts
> > > with the untrusted status and ultimately don't configure the IOMMU
> > > at all.
> >
> > Yes, if the driver wants to force a domain type it should be
> > forced. Driver knows best. Eg AMD forces IDENTITY when the HW/driver
> > is incapable of supporting otherwise.
>
>
> @Jason,
>
> Looks like before commit 59ddce4418da483, core IOMMU layer was enforcing
> 'IOMMU_DOMAIN_DMA' for untrusted device. If its trusted device then it was
> letting HW IOMMU driver to pick best domain type.
>
> Did we change that flow? Are we expecting HW IOMMU driver to handler untrusted
> devices as well?
>
> >
> > > Meanwhile the GPU driver presumably goes on to believe it's using dma-direct
> > > with no IOMMU present, resulting in fireworks when its traffic reaches the
> > > IOMMU. Great :(
> >
> > I wonder where is the missing error handling.. iommu probe failure
> > should not go on to allow driver attach, there is no guarentee any DMA
> > works now that many iommus are booting up in BLOCKED.
>
>
> Looking into code path, in failure path we cleanup device, not group. May be
> that's causing issue? Not sure where it failed. If I manage to find some system
> I will try to debug further.
>
>
>
> >
> > > However the other notable thing that also happened between 6.6 and 6.7 was
> > > the removal of the AMD iommu_v2 code, so there's some possibility that the
> > > GPU driver still may have only been working before due to that also
> >
> > Most likely it is the above change interacting with this patch when
> > they are both combined in the merge:
> >
> > commit 92e2bd56a5f9fc44313fda802a43a63cc2a9c8f6
> > Author: Vasant Hegde <vasant.hegde@amd.com <mailto:vasant.hegde@amd.com>>
> > Date: Thu Sep 21 09:21:45 2023 +0000
> >
> > iommu/amd: Introduce iommu_dev_data.flags to track device capabilities
> >
> > @@ -2471,7 +2481,7 @@ static int amd_iommu_def_domain_type(struct device *dev)
> > * and require remapping.
> > * - SNP is enabled, because it prohibits DTE[Mode]=0.
> > */
> > - if (dev_data->iommu_v2 &&
> > + if (pdev_pasid_supported(dev_data) &&
> > !cc_platform_has(CC_ATTR_MEM_ENCRYPT) &&
> > !amd_iommu_snp_en) {
> > return IOMMU_DOMAIN_IDENTITY;
> >
> > Which, IIRC, was intended to be temporary to work around limitations
> > in the DTE programming logic within the driver. Previously iommu_v2 as
> > a module option that Eric probably doesn't set, I guess.
> >
> > The below will probably make it boot, but Vasant should check what
> > happens if PASID is eventually attached too.
> >
> > diff --git a/drivers/iommu/amd/iommu.c b/drivers/iommu/amd/iommu.c
> > index d35c1b8c8e65ce..f3da6a5b6cb1cb 100644
> > --- a/drivers/iommu/amd/iommu.c
> > +++ b/drivers/iommu/amd/iommu.c
> > @@ -2758,11 +2758,16 @@ static int amd_iommu_def_domain_type(struct device *dev)
> > * and require remapping.
> > * - SNP is enabled, because it prohibits DTE[Mode]=0.
> > */
> > - if (pdev_pasid_supported(dev_data) &&
> > - !cc_platform_has(CC_ATTR_MEM_ENCRYPT) &&
> > - !amd_iommu_snp_en) {
> > + if (!cc_platform_has(CC_ATTR_MEM_ENCRYPT) && !amd_iommu_snp_en)
> > + return IOMMU_DOMAIN_IDENTITY;
> > +
> > + /*
> > + * For now driver limitations prevent enabling PASID as a paging domain
> > + * on the RID together.
> > + */
> > + if (dev_is_pci(dev) && !to_pci_dev(dev)->untrusted &&
> > + pdev_pasid_supported(dev_data))
> > return IOMMU_DOMAIN_IDENTITY;
> > - }
> >
> > return 0;
> > }
> >
> > Jason
> >
> >
> > As it booted ok with Robin's patch above, these changes to
> > drivers/iommu/amd/iommu.c didn't seem to make a difference for me. I was testing
> > with amd iommu v2 off in the kernel config and I also have TSME enabled in the
> > BIOS if that matters.
>
> TMSE is transparent to OS. So its fine.
>
> -Vasant
>
Output of lspci -vvv and /proc/cmdline attached.
-Eric
[-- Attachment #2: cmdline.log --]
[-- Type: text/x-log, Size: 83 bytes --]
BOOT_IMAGE=/vmlinuz root=UUID=f660f1a3-1034-4a95-a310-3d7199ea58a5 ro quiet splash
[-- Attachment #3: lspci.log --]
[-- Type: text/x-log, Size: 84184 bytes --]
00:00.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 14b5 (rev 01)
Subsystem: Lenovo Device 50b4
Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
00:00.2 IOMMU: Advanced Micro Devices, Inc. [AMD] Device 14b6
Subsystem: Lenovo Device 50b4
Control: I/O- Mem- BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0
Interrupt: pin A routed to IRQ 25
Capabilities: [40] Secure device <?>
Capabilities: [64] MSI: Enable+ Count=1/4 Maskable- 64bit+
Address: 00000000fee05000 Data: 0020
Capabilities: [74] HyperTransport: MSI Mapping Enable+ Fixed+
00:01.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 14b7 (rev 01)
Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
IOMMU group: 0
00:02.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 14b7 (rev 01)
Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
IOMMU group: 1
00:02.2 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 14ba (prog-if 00 [Normal decode])
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 32 bytes
Interrupt: pin ? routed to IRQ 35
IOMMU group: 2
Bus: primary=00, secondary=01, subordinate=01, sec-latency=0
I/O behind bridge: 0000f000-00000fff [disabled]
Memory behind bridge: 98000000-981fffff [size=2M]
Prefetchable memory behind bridge: 00000000fff00000-00000000000fffff [disabled]
Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort+ <SERR- <PERR-
BridgeCtl: Parity- SERR+ NoISA- VGA- VGA16- MAbort- >Reset- FastB2B-
PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
Capabilities: [50] Power Management version 3
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [58] Express (v2) Root Port (Slot+), MSI 00
DevCap: MaxPayload 256 bytes, PhantFunc 0
ExtTag+ RBE+
DevCtl: CorrErr- NonFatalErr- FatalErr- UnsupReq-
RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop+
MaxPayload 128 bytes, MaxReadReq 512 bytes
DevSta: CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr- TransPend-
LnkCap: Port #4, Speed 16GT/s, Width x1, ASPM L1, Exit Latency L1 <64us
ClockPM- Surprise- LLActRep+ BwNot+ ASPMOptComp+
LnkCtl: ASPM L1 Enabled; RCB 64 bytes, Disabled- CommClk+
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 5GT/s (downgraded), Width x1 (ok)
TrErr- Train- SlotClk+ DLActive+ BWMgmt+ ABWMgmt-
SltCap: AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surprise-
Slot #0, PowerLimit 75.000W; Interlock- NoCompl+
SltCtl: Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg-
Control: AttnInd Unknown, PwrInd Unknown, Power- Interlock-
SltSta: Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet+ Interlock-
Changed: MRL- PresDet- LinkState+
RootCap: CRSVisible+
RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna+ CRSVisible+
RootSta: PME ReqID 0000, PMEStatus- PMEPending-
DevCap2: Completion Timeout: Range ABCD, TimeoutDis+ NROPrPrP- LTR+
10BitTagComp+ 10BitTagReq+ OBFF Not Supported, ExtFmt+ EETLPPrefix+, MaxEETLPPrefixes 1
EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit-
FRS- LN System CLS Not Supported, TPHComp+ ExtTPHComp- ARIFwd+
AtomicOpsCap: Routing+ 32bit+ 64bit+ 128bitCAS-
DevCtl2: Completion Timeout: 65ms to 210ms, TimeoutDis- LTR+ OBFF Disabled, ARIFwd-
AtomicOpsCtl: ReqEn- EgressBlck-
LnkCap2: Supported Link Speeds: 2.5-16GT/s, Crosslink- Retimer+ 2Retimers+ DRS-
LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
Compliance De-emphasis: -6dB
LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete- EqualizationPhase1-
EqualizationPhase2- EqualizationPhase3- LinkEqualizationRequest-
Retimer- 2Retimers- CrosslinkRes: unsupported
Capabilities: [a0] MSI: Enable+ Count=1/1 Maskable- 64bit+
Address: 00000000fee00000 Data: 0000
Capabilities: [c0] Subsystem: Lenovo Device 50b4
Capabilities: [c8] HyperTransport: MSI Mapping Enable+ Fixed+
Capabilities: [100 v1] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
Capabilities: [150 v2] Advanced Error Reporting
UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UESvrt: DLP- SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP- ECRC- UnsupReq- ACSViol-
CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr-
CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+
AERCap: First Error Pointer: 00, ECRCGenCap+ ECRCGenEn- ECRCChkCap+ ECRCChkEn-
MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap-
HeaderLog: 00000000 00000000 00000000 00000000
RootCmd: CERptEn- NFERptEn- FERptEn-
RootSta: CERcvd- MultCERcvd- UERcvd- MultUERcvd-
FirstFatal- NonFatalMsg- FatalMsg- IntMsg 0
ErrorSrc: ERR_COR: 0000 ERR_FATAL/NONFATAL: 0000
Capabilities: [270 v1] Secondary PCI Express
LnkCtl3: LnkEquIntrruptEn- PerformEqu-
LaneErrStat: 0
Capabilities: [2a0 v1] Access Control Services
ACSCap: SrcValid+ TransBlk+ ReqRedir+ CmpltRedir+ UpstreamFwd+ EgressCtrl- DirectTrans+
ACSCtl: SrcValid+ TransBlk- ReqRedir+ CmpltRedir+ UpstreamFwd+ EgressCtrl- DirectTrans-
Capabilities: [370 v1] L1 PM Substates
L1SubCap: PCI-PM_L1.2+ PCI-PM_L1.1+ ASPM_L1.2+ ASPM_L1.1+ L1_PM_Substates+
PortCommonModeRestoreTime=70us PortTPowerOnTime=150us
L1SubCtl1: PCI-PM_L1.2+ PCI-PM_L1.1+ ASPM_L1.2+ ASPM_L1.1+
T_CommonMode=70us LTR1.2_Threshold=226304ns
L1SubCtl2: T_PwrOn=150us
Capabilities: [400 v1] Data Link Feature <?>
Capabilities: [410 v1] Physical Layer 16.0 GT/s <?>
Capabilities: [440 v1] Lane Margining at the Receiver <?>
Kernel driver in use: pcieport
00:02.4 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 14ba (prog-if 00 [Normal decode])
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 32 bytes
Interrupt: pin ? routed to IRQ 36
IOMMU group: 3
Bus: primary=00, secondary=02, subordinate=02, sec-latency=0
I/O behind bridge: 0000f000-00000fff [disabled]
Memory behind bridge: 98a00000-98afffff [size=1M]
Prefetchable memory behind bridge: 00000000fff00000-00000000000fffff [disabled]
Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort+ <SERR- <PERR-
BridgeCtl: Parity- SERR+ NoISA- VGA- VGA16- MAbort- >Reset- FastB2B-
PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
Capabilities: [50] Power Management version 3
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [58] Express (v2) Root Port (Slot+), MSI 00
DevCap: MaxPayload 256 bytes, PhantFunc 0
ExtTag+ RBE+
DevCtl: CorrErr- NonFatalErr- FatalErr- UnsupReq-
RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop+
MaxPayload 256 bytes, MaxReadReq 512 bytes
DevSta: CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr- TransPend-
LnkCap: Port #0, Speed 16GT/s, Width x4, ASPM L1, Exit Latency L1 <64us
ClockPM- Surprise- LLActRep+ BwNot+ ASPMOptComp+
LnkCtl: ASPM L1 Enabled; RCB 64 bytes, Disabled- CommClk+
ExtSynch+ ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 16GT/s (ok), Width x4 (ok)
TrErr- Train- SlotClk+ DLActive+ BWMgmt+ ABWMgmt-
SltCap: AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surprise-
Slot #0, PowerLimit 75.000W; Interlock- NoCompl+
SltCtl: Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg-
Control: AttnInd Unknown, PwrInd Unknown, Power- Interlock-
SltSta: Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet+ Interlock-
Changed: MRL- PresDet- LinkState+
RootCap: CRSVisible+
RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna+ CRSVisible+
RootSta: PME ReqID 0000, PMEStatus- PMEPending-
DevCap2: Completion Timeout: Range ABCD, TimeoutDis+ NROPrPrP- LTR+
10BitTagComp+ 10BitTagReq+ OBFF Not Supported, ExtFmt+ EETLPPrefix+, MaxEETLPPrefixes 1
EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit-
FRS- LN System CLS Not Supported, TPHComp+ ExtTPHComp- ARIFwd+
AtomicOpsCap: Routing+ 32bit+ 64bit+ 128bitCAS-
DevCtl2: Completion Timeout: 65ms to 210ms, TimeoutDis- LTR+ OBFF Disabled, ARIFwd+
AtomicOpsCtl: ReqEn- EgressBlck-
LnkCap2: Supported Link Speeds: 2.5-16GT/s, Crosslink- Retimer+ 2Retimers+ DRS-
LnkCtl2: Target Link Speed: 16GT/s, EnterCompliance- SpeedDis-
Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
Compliance De-emphasis: -6dB
LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete+ EqualizationPhase1+
EqualizationPhase2+ EqualizationPhase3+ LinkEqualizationRequest-
Retimer- 2Retimers- CrosslinkRes: unsupported
Capabilities: [a0] MSI: Enable+ Count=1/1 Maskable- 64bit+
Address: 00000000fee00000 Data: 0000
Capabilities: [c0] Subsystem: Lenovo Device 50b4
Capabilities: [c8] HyperTransport: MSI Mapping Enable+ Fixed+
Capabilities: [100 v1] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
Capabilities: [150 v2] Advanced Error Reporting
UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UESvrt: DLP- SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP- ECRC- UnsupReq- ACSViol-
CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr-
CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+
AERCap: First Error Pointer: 00, ECRCGenCap+ ECRCGenEn- ECRCChkCap+ ECRCChkEn-
MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap-
HeaderLog: 00000000 00000000 00000000 00000000
RootCmd: CERptEn- NFERptEn- FERptEn-
RootSta: CERcvd- MultCERcvd- UERcvd- MultUERcvd-
FirstFatal- NonFatalMsg- FatalMsg- IntMsg 0
ErrorSrc: ERR_COR: 0000 ERR_FATAL/NONFATAL: 0000
Capabilities: [270 v1] Secondary PCI Express
LnkCtl3: LnkEquIntrruptEn- PerformEqu-
LaneErrStat: 0
Capabilities: [2a0 v1] Access Control Services
ACSCap: SrcValid+ TransBlk+ ReqRedir+ CmpltRedir+ UpstreamFwd+ EgressCtrl- DirectTrans+
ACSCtl: SrcValid+ TransBlk- ReqRedir+ CmpltRedir+ UpstreamFwd+ EgressCtrl- DirectTrans-
Capabilities: [370 v1] L1 PM Substates
L1SubCap: PCI-PM_L1.2+ PCI-PM_L1.1+ ASPM_L1.2+ ASPM_L1.1+ L1_PM_Substates+
PortCommonModeRestoreTime=10us PortTPowerOnTime=150us
L1SubCtl1: PCI-PM_L1.2+ PCI-PM_L1.1+ ASPM_L1.2+ ASPM_L1.1+
T_CommonMode=10us LTR1.2_Threshold=166912ns
L1SubCtl2: T_PwrOn=150us
Capabilities: [400 v1] Data Link Feature <?>
Capabilities: [410 v1] Physical Layer 16.0 GT/s <?>
Capabilities: [440 v1] Lane Margining at the Receiver <?>
Kernel driver in use: pcieport
00:03.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 14b7 (rev 01)
Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
IOMMU group: 4
00:04.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 14b7 (rev 01)
Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
IOMMU group: 5
00:04.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 14cd (prog-if 00 [Normal decode])
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 32 bytes
Interrupt: pin ? routed to IRQ 37
IOMMU group: 5
Bus: primary=00, secondary=03, subordinate=32, sec-latency=0
I/O behind bridge: 00002000-00003fff [size=8K]
Memory behind bridge: 80000000-97ffffff [size=384M]
Prefetchable memory behind bridge: 0000000900000000-0000000a7fffffff [size=6G]
Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort+ <SERR- <PERR-
BridgeCtl: Parity- SERR+ NoISA- VGA- VGA16- MAbort- >Reset- FastB2B-
PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
Capabilities: [50] Power Management version 3
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [58] Express (v2) Root Port (Slot+), MSI 00
DevCap: MaxPayload 128 bytes, PhantFunc 0
ExtTag+ RBE+
DevCtl: CorrErr- NonFatalErr- FatalErr- UnsupReq-
RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop+
MaxPayload 128 bytes, MaxReadReq 512 bytes
DevSta: CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr- TransPend-
LnkCap: Port #0, Speed 2.5GT/s, Width x1, ASPM L1, Exit Latency L1 <4us
ClockPM- Surprise- LLActRep+ BwNot- ASPMOptComp+
LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk+
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 2.5GT/s (ok), Width x1 (ok)
TrErr- Train- SlotClk+ DLActive+ BWMgmt- ABWMgmt-
SltCap: AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug+ Surprise+
Slot #0, PowerLimit 0.000W; Interlock- NoCompl+
SltCtl: Enable: AttnBtn- PwrFlt- MRL- PresDet+ CmdCplt- HPIrq+ LinkChg+
Control: AttnInd Unknown, PwrInd Unknown, Power- Interlock-
SltSta: Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet+ Interlock-
Changed: MRL- PresDet- LinkState-
RootCap: CRSVisible+
RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna+ CRSVisible+
RootSta: PME ReqID 0000, PMEStatus- PMEPending-
DevCap2: Completion Timeout: Range ABCD, TimeoutDis+ NROPrPrP- LTR+
10BitTagComp+ 10BitTagReq+ OBFF Not Supported, ExtFmt+ EETLPPrefix+, MaxEETLPPrefixes 1
EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit-
FRS- LN System CLS Not Supported, TPHComp+ ExtTPHComp- ARIFwd+
AtomicOpsCap: Routing+ 32bit+ 64bit+ 128bitCAS-
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR+ OBFF Disabled, ARIFwd-
AtomicOpsCtl: ReqEn- EgressBlck-
LnkCap2: Supported Link Speeds: 2.5GT/s, Crosslink- Retimer- 2Retimers- DRS-
LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-
Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
Compliance De-emphasis: -6dB
LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete- EqualizationPhase1-
EqualizationPhase2- EqualizationPhase3- LinkEqualizationRequest-
Retimer- 2Retimers- CrosslinkRes: unsupported
Capabilities: [a0] MSI: Enable+ Count=1/1 Maskable- 64bit+
Address: 00000000fee00000 Data: 0000
Capabilities: [c0] Subsystem: Advanced Micro Devices, Inc. [AMD] Device 1453
Capabilities: [c8] HyperTransport: MSI Mapping Enable+ Fixed+
Capabilities: [100 v1] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
Capabilities: [150 v2] Advanced Error Reporting
UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr-
CEMsk: RxErr- BadTLP- BadDLLP+ Rollover- Timeout- AdvNonFatalErr+
AERCap: First Error Pointer: 00, ECRCGenCap+ ECRCGenEn- ECRCChkCap+ ECRCChkEn-
MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap-
HeaderLog: 00000000 00000000 00000000 00000000
RootCmd: CERptEn- NFERptEn- FERptEn-
RootSta: CERcvd- MultCERcvd- UERcvd- MultUERcvd-
FirstFatal- NonFatalMsg- FatalMsg- IntMsg 0
ErrorSrc: ERR_COR: 0000 ERR_FATAL/NONFATAL: 0000
Capabilities: [270 v1] Secondary PCI Express
LnkCtl3: LnkEquIntrruptEn- PerformEqu-
LaneErrStat: 0
Capabilities: [400 v1] Data Link Feature <?>
Kernel driver in use: pcieport
00:08.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 14b7 (rev 01)
Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
IOMMU group: 6
00:08.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 14b9 (rev 10) (prog-if 00 [Normal decode])
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 32 bytes
Interrupt: pin A routed to IRQ 38
IOMMU group: 7
Bus: primary=00, secondary=33, subordinate=33, sec-latency=0
I/O behind bridge: 00001000-00001fff [size=4K]
Memory behind bridge: 98600000-989fffff [size=4M]
Prefetchable memory behind bridge: 00000008e0000000-00000008f01fffff [size=258M]
Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
BridgeCtl: Parity- SERR+ NoISA- VGA+ VGA16- MAbort- >Reset- FastB2B-
PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
Capabilities: [50] Power Management version 3
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [58] Express (v2) Root Port (Slot-), MSI 00
DevCap: MaxPayload 512 bytes, PhantFunc 0
ExtTag+ RBE+
DevCtl: CorrErr- NonFatalErr- FatalErr- UnsupReq-
RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop+
MaxPayload 128 bytes, MaxReadReq 512 bytes
DevSta: CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr- TransPend-
LnkCap: Port #0, Speed 16GT/s, Width x16, ASPM L0s L1, Exit Latency L0s <64ns, L1 <1us
ClockPM- Surprise- LLActRep+ BwNot+ ASPMOptComp+
LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk+
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 16GT/s (ok), Width x16 (ok)
TrErr- Train- SlotClk+ DLActive+ BWMgmt+ ABWMgmt-
RootCap: CRSVisible+
RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna+ CRSVisible+
RootSta: PME ReqID 0000, PMEStatus- PMEPending-
DevCap2: Completion Timeout: Not Supported, TimeoutDis- NROPrPrP- LTR-
10BitTagComp+ 10BitTagReq- OBFF Not Supported, ExtFmt+ EETLPPrefix+, MaxEETLPPrefixes 4
EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit-
FRS- LN System CLS Not Supported, TPHComp- ExtTPHComp- ARIFwd-
AtomicOpsCap: Routing+ 32bit- 64bit- 128bitCAS-
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR- OBFF Disabled, ARIFwd-
AtomicOpsCtl: ReqEn- EgressBlck-
LnkCap2: Supported Link Speeds: 2.5-16GT/s, Crosslink- Retimer+ 2Retimers+ DRS-
LnkCtl2: Target Link Speed: 16GT/s, EnterCompliance- SpeedDis-
Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
Compliance De-emphasis: -6dB
LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete+ EqualizationPhase1+
EqualizationPhase2+ EqualizationPhase3+ LinkEqualizationRequest-
Retimer- 2Retimers- CrosslinkRes: unsupported
Capabilities: [a0] MSI: Enable+ Count=1/1 Maskable- 64bit+
Address: 00000000fee00000 Data: 0000
Capabilities: [c0] Subsystem: Device 50b4:17aa
Capabilities: [100 v1] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
Capabilities: [270 v1] Secondary PCI Express
LnkCtl3: LnkEquIntrruptEn- PerformEqu-
LaneErrStat: 0
Capabilities: [2a0 v1] Access Control Services
ACSCap: SrcValid+ TransBlk+ ReqRedir+ CmpltRedir+ UpstreamFwd+ EgressCtrl- DirectTrans+
ACSCtl: SrcValid+ TransBlk- ReqRedir+ CmpltRedir+ UpstreamFwd+ EgressCtrl- DirectTrans-
Capabilities: [400 v1] Data Link Feature <?>
Capabilities: [410 v1] Physical Layer 16.0 GT/s <?>
Capabilities: [450 v1] Lane Margining at the Receiver <?>
Kernel driver in use: pcieport
00:08.3 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 14b9 (rev 10) (prog-if 00 [Normal decode])
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 32 bytes
Interrupt: pin A routed to IRQ 39
IOMMU group: 8
Bus: primary=00, secondary=34, subordinate=34, sec-latency=0
I/O behind bridge: 0000f000-00000fff [disabled]
Memory behind bridge: 98200000-985fffff [size=4M]
Prefetchable memory behind bridge: 00000000fff00000-00000000000fffff [disabled]
Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
BridgeCtl: Parity- SERR+ NoISA- VGA- VGA16- MAbort- >Reset- FastB2B-
PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
Capabilities: [50] Power Management version 3
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [58] Express (v2) Root Port (Slot-), MSI 00
DevCap: MaxPayload 512 bytes, PhantFunc 0
ExtTag+ RBE+
DevCtl: CorrErr- NonFatalErr- FatalErr- UnsupReq-
RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop+
MaxPayload 128 bytes, MaxReadReq 512 bytes
DevSta: CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr- TransPend-
LnkCap: Port #0, Speed 16GT/s, Width x16, ASPM L0s L1, Exit Latency L0s <64ns, L1 <1us
ClockPM- Surprise- LLActRep+ BwNot+ ASPMOptComp+
LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk+
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 16GT/s (ok), Width x16 (ok)
TrErr- Train- SlotClk+ DLActive+ BWMgmt+ ABWMgmt-
RootCap: CRSVisible+
RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna+ CRSVisible+
RootSta: PME ReqID 0000, PMEStatus- PMEPending-
DevCap2: Completion Timeout: Not Supported, TimeoutDis- NROPrPrP- LTR-
10BitTagComp+ 10BitTagReq- OBFF Not Supported, ExtFmt- EETLPPrefix-
EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit-
FRS- LN System CLS Not Supported, TPHComp- ExtTPHComp- ARIFwd-
AtomicOpsCap: Routing- 32bit- 64bit- 128bitCAS-
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR- OBFF Disabled, ARIFwd-
AtomicOpsCtl: ReqEn- EgressBlck-
LnkCap2: Supported Link Speeds: 2.5-16GT/s, Crosslink- Retimer+ 2Retimers+ DRS-
LnkCtl2: Target Link Speed: 16GT/s, EnterCompliance- SpeedDis-
Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
Compliance De-emphasis: -6dB
LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete+ EqualizationPhase1+
EqualizationPhase2+ EqualizationPhase3+ LinkEqualizationRequest-
Retimer- 2Retimers- CrosslinkRes: unsupported
Capabilities: [a0] MSI: Enable+ Count=1/1 Maskable- 64bit+
Address: 00000000fee00000 Data: 0000
Capabilities: [c0] Subsystem: Device 50b4:17aa
Capabilities: [100 v1] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
Capabilities: [150 v2] Advanced Error Reporting
UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr-
CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+
AERCap: First Error Pointer: 00, ECRCGenCap- ECRCGenEn- ECRCChkCap- ECRCChkEn-
MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap-
HeaderLog: 00000000 00000000 00000000 00000000
RootCmd: CERptEn- NFERptEn- FERptEn-
RootSta: CERcvd- MultCERcvd- UERcvd- MultUERcvd-
FirstFatal- NonFatalMsg- FatalMsg- IntMsg 0
ErrorSrc: ERR_COR: 0000 ERR_FATAL/NONFATAL: 0000
Capabilities: [270 v1] Secondary PCI Express
LnkCtl3: LnkEquIntrruptEn- PerformEqu-
LaneErrStat: 0
Capabilities: [2a0 v1] Access Control Services
ACSCap: SrcValid+ TransBlk+ ReqRedir+ CmpltRedir+ UpstreamFwd+ EgressCtrl- DirectTrans+
ACSCtl: SrcValid+ TransBlk- ReqRedir+ CmpltRedir+ UpstreamFwd+ EgressCtrl- DirectTrans-
Capabilities: [400 v1] Data Link Feature <?>
Capabilities: [410 v1] Physical Layer 16.0 GT/s <?>
Capabilities: [450 v1] Lane Margining at the Receiver <?>
Kernel driver in use: pcieport
00:14.0 SMBus: Advanced Micro Devices, Inc. [AMD] FCH SMBus Controller (rev 71)
Subsystem: Lenovo FCH SMBus Controller
Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
Status: Cap- 66MHz+ UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
IOMMU group: 9
Kernel driver in use: piix4_smbus
Kernel modules: i2c_piix4
00:14.3 ISA bridge: Advanced Micro Devices, Inc. [AMD] FCH LPC Bridge (rev 51)
Subsystem: Lenovo FCH LPC Bridge
Control: I/O+ Mem+ BusMaster+ SpecCycle+ MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
Status: Cap- 66MHz+ UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0
IOMMU group: 9
00:18.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1679
Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
IOMMU group: 10
00:18.1 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 167a
Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
IOMMU group: 10
00:18.2 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 167b
Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
IOMMU group: 10
00:18.3 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 167c
Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
IOMMU group: 10
Kernel driver in use: k10temp
Kernel modules: k10temp
00:18.4 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 167d
Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
IOMMU group: 10
00:18.5 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 167e
Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
IOMMU group: 10
00:18.6 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 167f
Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
IOMMU group: 10
00:18.7 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1680
Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
IOMMU group: 10
01:00.0 Network controller: Qualcomm Atheros QCNFA765 (rev 01)
Subsystem: Lenovo Device 9309
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 32 bytes
Interrupt: pin ? routed to IRQ 96
IOMMU group: 11
Region 0: Memory at 98000000 (64-bit, non-prefetchable) [size=2M]
Capabilities: [40] Power Management version 3
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [50] MSI: Enable+ Count=32/32 Maskable+ 64bit-
Address: fee00000 Data: 0000
Masking: fe023c00 Pending: 00000000
Capabilities: [70] Express (v2) Endpoint, MSI 00
DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s unlimited, L1 unlimited
ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset- SlotPowerLimit 75.000W
DevCtl: CorrErr- NonFatalErr- FatalErr- UnsupReq-
RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+
MaxPayload 128 bytes, MaxReadReq 512 bytes
DevSta: CorrErr+ NonFatalErr- FatalErr- UnsupReq+ AuxPwr+ TransPend-
LnkCap: Port #0, Speed 8GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <1us, L1 <64us
ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp+
LnkCtl: ASPM L1 Enabled; RCB 64 bytes, Disabled- CommClk+
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 5GT/s (downgraded), Width x1 (ok)
TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
DevCap2: Completion Timeout: Range ABCD, TimeoutDis+ NROPrPrP- LTR+
10BitTagComp- 10BitTagReq- OBFF Not Supported, ExtFmt- EETLPPrefix-
EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit-
FRS- TPHComp+ ExtTPHComp-
AtomicOpsCap: 32bit- 64bit- 128bitCAS-
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR+ OBFF Disabled,
AtomicOpsCtl: ReqEn-
LnkCap2: Supported Link Speeds: 2.5-8GT/s, Crosslink- Retimer- 2Retimers- DRS-
LnkCtl2: Target Link Speed: 8GT/s, EnterCompliance- SpeedDis-
Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
Compliance De-emphasis: -6dB
LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete- EqualizationPhase1-
EqualizationPhase2- EqualizationPhase3- LinkEqualizationRequest-
Retimer- 2Retimers- CrosslinkRes: unsupported
Capabilities: [100 v2] Advanced Error Reporting
UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+
CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+
AERCap: First Error Pointer: 00, ECRCGenCap+ ECRCGenEn- ECRCChkCap+ ECRCChkEn-
MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap-
HeaderLog: 00000000 00000000 00000000 00000000
Capabilities: [148 v1] Secondary PCI Express
LnkCtl3: LnkEquIntrruptEn- PerformEqu-
LaneErrStat: 0
Capabilities: [158 v1] Transaction Processing Hints
No steering table available
Capabilities: [1e4 v1] Latency Tolerance Reporting
Max snoop latency: 1048576ns
Max no snoop latency: 1048576ns
Capabilities: [1ec v1] L1 PM Substates
L1SubCap: PCI-PM_L1.2+ PCI-PM_L1.1+ ASPM_L1.2+ ASPM_L1.1+ L1_PM_Substates+
PortCommonModeRestoreTime=70us PortTPowerOnTime=0us
L1SubCtl1: PCI-PM_L1.2+ PCI-PM_L1.1+ ASPM_L1.2+ ASPM_L1.1+
T_CommonMode=0us LTR1.2_Threshold=226304ns
L1SubCtl2: T_PwrOn=150us
Kernel driver in use: ath11k_pci
Kernel modules: ath11k_pci
02:00.0 Non-Volatile memory controller: Samsung Electronics Co Ltd NVMe SSD Controller PM9A1/PM9A3/980PRO (prog-if 02 [NVM Express])
Subsystem: Samsung Electronics Co Ltd NVMe SSD Controller PM9A1/PM9A3/980PRO
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 32 bytes
Interrupt: pin A routed to IRQ 43
IOMMU group: 12
Region 0: Memory at 98a00000 (64-bit, non-prefetchable) [size=16K]
Capabilities: [40] Power Management version 3
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [50] MSI: Enable- Count=1/32 Maskable- 64bit+
Address: 0000000000000000 Data: 0000
Capabilities: [70] Express (v2) Endpoint, MSI 00
DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s unlimited, L1 unlimited
ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset+ SlotPowerLimit 75.000W
DevCtl: CorrErr- NonFatalErr- FatalErr- UnsupReq-
RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop+ FLReset-
MaxPayload 256 bytes, MaxReadReq 512 bytes
DevSta: CorrErr+ NonFatalErr- FatalErr- UnsupReq+ AuxPwr- TransPend-
LnkCap: Port #0, Speed 16GT/s, Width x4, ASPM L1, Exit Latency L1 <64us
ClockPM+ Surprise- LLActRep- BwNot- ASPMOptComp+
LnkCtl: ASPM L1 Enabled; RCB 64 bytes, Disabled- CommClk+
ExtSynch+ ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 16GT/s (ok), Width x4 (ok)
TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
DevCap2: Completion Timeout: Range ABCD, TimeoutDis+ NROPrPrP- LTR+
10BitTagComp+ 10BitTagReq- OBFF Not Supported, ExtFmt- EETLPPrefix-
EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit-
FRS- TPHComp- ExtTPHComp-
AtomicOpsCap: 32bit- 64bit- 128bitCAS-
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR+ OBFF Disabled,
AtomicOpsCtl: ReqEn-
LnkCap2: Supported Link Speeds: 2.5-16GT/s, Crosslink- Retimer+ 2Retimers+ DRS-
LnkCtl2: Target Link Speed: 16GT/s, EnterCompliance- SpeedDis-
Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
Compliance De-emphasis: -6dB
LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete+ EqualizationPhase1+
EqualizationPhase2+ EqualizationPhase3+ LinkEqualizationRequest-
Retimer- 2Retimers- CrosslinkRes: Upstream Port
Capabilities: [b0] MSI-X: Enable+ Count=130 Masked-
Vector table: BAR=0 offset=00003000
PBA: BAR=0 offset=00002000
Capabilities: [100 v2] Advanced Error Reporting
UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+
CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+
AERCap: First Error Pointer: 00, ECRCGenCap+ ECRCGenEn- ECRCChkCap+ ECRCChkEn-
MultHdrRecCap+ MultHdrRecEn- TLPPfxPres- HdrLogCap-
HeaderLog: 00000000 00000000 00000000 00000000
Capabilities: [168 v1] Alternative Routing-ID Interpretation (ARI)
ARICap: MFVC- ACS-, Next Function: 0
ARICtl: MFVC- ACS-, Function Group: 0
Capabilities: [178 v1] Secondary PCI Express
LnkCtl3: LnkEquIntrruptEn- PerformEqu-
LaneErrStat: 0
Capabilities: [198 v1] Physical Layer 16.0 GT/s <?>
Capabilities: [1bc v1] Lane Margining at the Receiver <?>
Capabilities: [214 v1] Latency Tolerance Reporting
Max snoop latency: 1048576ns
Max no snoop latency: 1048576ns
Capabilities: [21c v1] L1 PM Substates
L1SubCap: PCI-PM_L1.2+ PCI-PM_L1.1+ ASPM_L1.2+ ASPM_L1.1+ L1_PM_Substates+
PortCommonModeRestoreTime=10us PortTPowerOnTime=10us
L1SubCtl1: PCI-PM_L1.2+ PCI-PM_L1.1+ ASPM_L1.2+ ASPM_L1.1+
T_CommonMode=0us LTR1.2_Threshold=166912ns
L1SubCtl2: T_PwrOn=150us
Capabilities: [3a0 v1] Data Link Feature <?>
Kernel driver in use: nvme
Kernel modules: nvme
03:00.0 PCI bridge: Intel Corporation DSL6340 Thunderbolt 3 Bridge [Alpine Ridge 2C 2015] (prog-if 00 [Normal decode])
Physical Slot: 0
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 32 bytes
Interrupt: pin A routed to IRQ 40
IOMMU group: 5
Bus: primary=03, secondary=04, subordinate=05, sec-latency=0
I/O behind bridge: 00002000-00002fff [size=4K]
Memory behind bridge: 80000000-800fffff [size=1M]
Prefetchable memory behind bridge: 0000000900000000-0000000a7fffffff [size=6G]
Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
BridgeCtl: Parity- SERR+ NoISA- VGA- VGA16- MAbort- >Reset- FastB2B-
PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
Capabilities: [80] Power Management version 3
Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=375mA PME(D0+,D1+,D2+,D3hot+,D3cold+)
Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [88] MSI: Enable- Count=1/1 Maskable- 64bit+
Address: 0000000000000000 Data: 0000
Capabilities: [ac] Subsystem: Akitio DSL6340 Thunderbolt 3 Bridge [Alpine Ridge 2C 2015]
Capabilities: [c0] Express (v2) Upstream Port, MSI 00
DevCap: MaxPayload 128 bytes, PhantFunc 0
ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ SlotPowerLimit 0.000W
DevCtl: CorrErr- NonFatalErr- FatalErr- UnsupReq-
RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop+
MaxPayload 128 bytes, MaxReadReq 512 bytes
DevSta: CorrErr+ NonFatalErr- FatalErr- UnsupReq+ AuxPwr+ TransPend-
LnkCap: Port #0, Speed 2.5GT/s, Width x4, ASPM L0s L1, Exit Latency L0s <2us, L1 <4us
ClockPM+ Surprise- LLActRep- BwNot- ASPMOptComp+
LnkCtl: ASPM Disabled; Disabled- CommClk+
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 2.5GT/s (ok), Width x4 (ok)
TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
DevCap2: Completion Timeout: Not Supported, TimeoutDis- NROPrPrP- LTR+
10BitTagComp- 10BitTagReq- OBFF Not Supported, ExtFmt- EETLPPrefix-
EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit-
FRS-
AtomicOpsCap: Routing-
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR+ OBFF Disabled,
AtomicOpsCtl: EgressBlck-
LnkCap2: Supported Link Speeds: 2.5-8GT/s, Crosslink- Retimer- 2Retimers- DRS-
LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-
Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
Compliance De-emphasis: -6dB
LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete- EqualizationPhase1-
EqualizationPhase2- EqualizationPhase3- LinkEqualizationRequest-
Retimer- 2Retimers- CrosslinkRes: unsupported
Capabilities: [100 v1] Device Serial Number 7b-5c-00-d1-b3-e9-07-00
Capabilities: [200 v1] Advanced Error Reporting
UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UESvrt: DLP+ SDES- TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+
CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+
AERCap: First Error Pointer: 00, ECRCGenCap- ECRCGenEn- ECRCChkCap- ECRCChkEn-
MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap-
HeaderLog: 00000000 00000000 00000000 00000000
Capabilities: [300 v1] Virtual Channel
Caps: LPEVC=0 RefClk=100ns PATEntryBits=1
Arb: Fixed- WRR32- WRR64- WRR128-
Ctrl: ArbSelect=Fixed
Status: InProgress-
VC0: Caps: PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
Arb: Fixed+ WRR32- WRR64- WRR128- TWRR128- WRR256-
Ctrl: Enable+ ID=0 ArbSelect=Fixed TC/VC=ff
Status: NegoPending- InProgress-
Capabilities: [400 v1] Power Budgeting <?>
Capabilities: [500 v1] Vendor Specific Information: ID=1234 Rev=1 Len=0d8 <?>
Capabilities: [600 v1] Latency Tolerance Reporting
Max snoop latency: 0ns
Max no snoop latency: 0ns
Capabilities: [700 v1] Secondary PCI Express
LnkCtl3: LnkEquIntrruptEn- PerformEqu-
LaneErrStat: 0
Kernel driver in use: pcieport
04:01.0 PCI bridge: Intel Corporation DSL6340 Thunderbolt 3 Bridge [Alpine Ridge 2C 2015] (prog-if 00 [Normal decode])
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 32 bytes
Interrupt: pin A routed to IRQ 42
IOMMU group: 5
Bus: primary=04, secondary=05, subordinate=05, sec-latency=0
I/O behind bridge: 00002000-00002fff [size=4K]
Memory behind bridge: 80000000-800fffff [size=1M]
Prefetchable memory behind bridge: 0000000900000000-0000000a7fffffff [size=6G]
Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
BridgeCtl: Parity- SERR+ NoISA- VGA- VGA16- MAbort- >Reset- FastB2B-
PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
Capabilities: [80] Power Management version 3
Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=375mA PME(D0+,D1+,D2+,D3hot+,D3cold+)
Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [88] MSI: Enable+ Count=1/1 Maskable- 64bit+
Address: 00000000fee00000 Data: 0000
Capabilities: [ac] Subsystem: Akitio DSL6340 Thunderbolt 3 Bridge [Alpine Ridge 2C 2015]
Capabilities: [c0] Express (v2) Downstream Port (Slot+), MSI 00
DevCap: MaxPayload 128 bytes, PhantFunc 0
ExtTag+ RBE+
DevCtl: CorrErr- NonFatalErr- FatalErr- UnsupReq-
RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop+
MaxPayload 128 bytes, MaxReadReq 512 bytes
DevSta: CorrErr+ NonFatalErr- FatalErr- UnsupReq- AuxPwr+ TransPend-
LnkCap: Port #1, Speed 8GT/s, Width x4, ASPM L0s L1, Exit Latency L0s <2us, L1 <4us
ClockPM- Surprise- LLActRep+ BwNot+ ASPMOptComp+
LnkCtl: ASPM Disabled; Disabled- CommClk+
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 2.5GT/s (downgraded), Width x4 (ok)
TrErr- Train- SlotClk+ DLActive+ BWMgmt+ ABWMgmt+
SltCap: AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surprise-
Slot #0, PowerLimit 0.000W; Interlock- NoCompl+
SltCtl: Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg-
Control: AttnInd Unknown, PwrInd Unknown, Power- Interlock-
SltSta: Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet+ Interlock-
Changed: MRL- PresDet+ LinkState+
DevCap2: Completion Timeout: Not Supported, TimeoutDis- NROPrPrP- LTR+
10BitTagComp- 10BitTagReq- OBFF Not Supported, ExtFmt- EETLPPrefix-
EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit-
FRS- ARIFwd-
AtomicOpsCap: Routing-
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR+ OBFF Disabled, ARIFwd-
AtomicOpsCtl: EgressBlck-
LnkCap2: Supported Link Speeds: 2.5-8GT/s, Crosslink- Retimer- 2Retimers- DRS-
LnkCtl2: Target Link Speed: 8GT/s, EnterCompliance- SpeedDis-, Selectable De-emphasis: -3.5dB
Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
Compliance De-emphasis: -6dB
LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete+ EqualizationPhase1+
EqualizationPhase2+ EqualizationPhase3+ LinkEqualizationRequest-
Retimer- 2Retimers- CrosslinkRes: unsupported
Capabilities: [100 v1] Device Serial Number 7b-5c-00-d1-b3-e9-07-00
Capabilities: [200 v1] Advanced Error Reporting
UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UESvrt: DLP+ SDES- TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
CESta: RxErr- BadTLP- BadDLLP+ Rollover- Timeout- AdvNonFatalErr-
CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+
AERCap: First Error Pointer: 00, ECRCGenCap- ECRCGenEn- ECRCChkCap- ECRCChkEn-
MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap-
HeaderLog: 00000000 00000000 00000000 00000000
Capabilities: [300 v1] Virtual Channel
Caps: LPEVC=0 RefClk=100ns PATEntryBits=1
Arb: Fixed- WRR32- WRR64- WRR128-
Ctrl: ArbSelect=Fixed
Status: InProgress-
VC0: Caps: PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
Arb: Fixed+ WRR32- WRR64- WRR128- TWRR128- WRR256-
Ctrl: Enable+ ID=0 ArbSelect=Fixed TC/VC=ff
Status: NegoPending- InProgress-
Capabilities: [400 v1] Power Budgeting <?>
Capabilities: [500 v1] Vendor Specific Information: ID=1234 Rev=1 Len=0d8 <?>
Capabilities: [700 v1] Secondary PCI Express
LnkCtl3: LnkEquIntrruptEn- PerformEqu-
LaneErrStat: LaneErr at lane: 0 1 2 3
Kernel driver in use: pcieport
05:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere [Radeon RX 470/480/570/570X/580/580X/590] (rev e7) (prog-if 00 [VGA controller])
Subsystem: Sapphire Technology Limited Nitro+ Radeon RX 570/580/590
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 32 bytes
Interrupt: pin A routed to IRQ 71
IOMMU group: 5
Region 0: Memory at 900000000 (64-bit, prefetchable) [size=4G]
Region 2: Memory at a00000000 (64-bit, prefetchable) [size=2M]
Region 4: I/O ports at 2000 [size=256]
Region 5: Memory at 80000000 (32-bit, non-prefetchable) [size=256K]
Expansion ROM at 80060000 [disabled] [size=128K]
Capabilities: [48] Vendor Specific Information: Len=08 <?>
Capabilities: [50] Power Management version 3
Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA PME(D0-,D1+,D2+,D3hot+,D3cold+)
Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [58] Express (v2) Legacy Endpoint, MSI 00
DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s <4us, L1 unlimited
ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset-
DevCtl: CorrErr- NonFatalErr- FatalErr- UnsupReq-
RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop+
MaxPayload 128 bytes, MaxReadReq 512 bytes
DevSta: CorrErr+ NonFatalErr- FatalErr- UnsupReq+ AuxPwr- TransPend-
LnkCap: Port #0, Speed 8GT/s, Width x16, ASPM L1, Exit Latency L1 <1us
ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp+
LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk+
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 2.5GT/s (downgraded), Width x4 (downgraded)
TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
DevCap2: Completion Timeout: Not Supported, TimeoutDis- NROPrPrP- LTR+
10BitTagComp- 10BitTagReq- OBFF Not Supported, ExtFmt+ EETLPPrefix+, MaxEETLPPrefixes 1
EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit-
FRS-
AtomicOpsCap: 32bit+ 64bit+ 128bitCAS-
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR+ OBFF Disabled,
AtomicOpsCtl: ReqEn-
LnkCap2: Supported Link Speeds: 2.5-8GT/s, Crosslink- Retimer- 2Retimers- DRS-
LnkCtl2: Target Link Speed: 8GT/s, EnterCompliance- SpeedDis-
Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
Compliance De-emphasis: -6dB
LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete+ EqualizationPhase1+
EqualizationPhase2+ EqualizationPhase3+ LinkEqualizationRequest+
Retimer- 2Retimers- CrosslinkRes: unsupported
Capabilities: [a0] MSI: Enable+ Count=1/1 Maskable- 64bit+
Address: 00000000fee00000 Data: 0000
Capabilities: [100 v1] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
Capabilities: [150 v2] Advanced Error Reporting
UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+
CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+
AERCap: First Error Pointer: 00, ECRCGenCap+ ECRCGenEn- ECRCChkCap+ ECRCChkEn-
MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap-
HeaderLog: 00000000 00000000 00000000 00000000
Capabilities: [200 v1] Physical Resizable BAR
BAR 0: current size: 4GB, supported: 256MB 512MB 1GB 2GB 4GB
Capabilities: [270 v1] Secondary PCI Express
LnkCtl3: LnkEquIntrruptEn- PerformEqu-
LaneErrStat: 0
Capabilities: [2b0 v1] Address Translation Service (ATS)
ATSCap: Invalidate Queue Depth: 00
ATSCtl: Enable-, Smallest Translation Unit: 00
Capabilities: [2c0 v1] Page Request Interface (PRI)
PRICtl: Enable+ Reset-
PRISta: RF- UPRGI- Stopped+
Page Request Capacity: 00000020, Page Request Allocation: 00000020
Capabilities: [2d0 v1] Process Address Space ID (PASID)
PASIDCap: Exec+ Priv+, Max PASID Width: 10
PASIDCtl: Enable- Exec- Priv-
Capabilities: [320 v1] Latency Tolerance Reporting
Max snoop latency: 0ns
Max no snoop latency: 0ns
Capabilities: [328 v1] Alternative Routing-ID Interpretation (ARI)
ARICap: MFVC- ACS-, Next Function: 1
ARICtl: MFVC- ACS-, Function Group: 0
Capabilities: [370 v1] L1 PM Substates
L1SubCap: PCI-PM_L1.2+ PCI-PM_L1.1+ ASPM_L1.2+ ASPM_L1.1+ L1_PM_Substates+
PortCommonModeRestoreTime=0us PortTPowerOnTime=170us
L1SubCtl1: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2- ASPM_L1.1-
T_CommonMode=0us LTR1.2_Threshold=0ns
L1SubCtl2: T_PwrOn=10us
Kernel driver in use: amdgpu
Kernel modules: amdgpu
05:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere HDMI Audio [Radeon RX 470/480 / 570/580/590]
Subsystem: Sapphire Technology Limited Ellesmere HDMI Audio [Radeon RX 470/480 / 570/580/590]
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 32 bytes
Interrupt: pin B routed to IRQ 93
IOMMU group: 5
Region 0: Memory at 80040000 (64-bit, non-prefetchable) [size=16K]
Capabilities: [48] Vendor Specific Information: Len=08 <?>
Capabilities: [50] Power Management version 3
Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [58] Express (v2) Legacy Endpoint, MSI 00
DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s <4us, L1 unlimited
ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset-
DevCtl: CorrErr- NonFatalErr- FatalErr- UnsupReq-
RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop+
MaxPayload 128 bytes, MaxReadReq 512 bytes
DevSta: CorrErr+ NonFatalErr- FatalErr- UnsupReq+ AuxPwr- TransPend-
LnkCap: Port #0, Speed 8GT/s, Width x16, ASPM L1, Exit Latency L1 <1us
ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp+
LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk+
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 2.5GT/s (downgraded), Width x4 (downgraded)
TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
DevCap2: Completion Timeout: Not Supported, TimeoutDis- NROPrPrP- LTR+
10BitTagComp- 10BitTagReq- OBFF Not Supported, ExtFmt+ EETLPPrefix+, MaxEETLPPrefixes 1
EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit-
FRS-
AtomicOpsCap: 32bit+ 64bit+ 128bitCAS-
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR- OBFF Disabled,
AtomicOpsCtl: ReqEn-
LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete- EqualizationPhase1-
EqualizationPhase2- EqualizationPhase3- LinkEqualizationRequest-
Retimer- 2Retimers- CrosslinkRes: unsupported
Capabilities: [a0] MSI: Enable+ Count=1/1 Maskable- 64bit+
Address: 00000000fee00000 Data: 0000
Capabilities: [100 v1] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
Capabilities: [150 v2] Advanced Error Reporting
UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+
CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+
AERCap: First Error Pointer: 00, ECRCGenCap+ ECRCGenEn- ECRCChkCap+ ECRCChkEn-
MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap-
HeaderLog: 00000000 00000000 00000000 00000000
Capabilities: [328 v1] Alternative Routing-ID Interpretation (ARI)
ARICap: MFVC- ACS-, Next Function: 0
ARICtl: MFVC- ACS-, Function Group: 0
Kernel driver in use: snd_hda_intel
Kernel modules: snd_hda_intel
33:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Rembrandt (rev d1) (prog-if 00 [VGA controller])
Subsystem: Lenovo Device 50b4
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 32 bytes
Interrupt: pin A routed to IRQ 47
IOMMU group: 13
Region 0: Memory at 8e0000000 (64-bit, prefetchable) [size=256M]
Region 2: Memory at 8f0000000 (64-bit, prefetchable) [size=2M]
Region 4: I/O ports at 1000 [size=256]
Region 5: Memory at 98900000 (32-bit, non-prefetchable) [size=512K]
Capabilities: [48] Vendor Specific Information: Len=08 <?>
Capabilities: [50] Power Management version 3
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1+,D2+,D3hot+,D3cold+)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [64] Express (v2) Legacy Endpoint, MSI 00
DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s <4us, L1 unlimited
ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset-
DevCtl: CorrErr- NonFatalErr- FatalErr- UnsupReq-
RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop+
MaxPayload 128 bytes, MaxReadReq 512 bytes
DevSta: CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr- TransPend-
LnkCap: Port #0, Speed 16GT/s, Width x16, ASPM L0s L1, Exit Latency L0s <64ns, L1 <1us
ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp+
LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk+
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 16GT/s (ok), Width x16 (ok)
TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
DevCap2: Completion Timeout: Range ABCD, TimeoutDis+ NROPrPrP- LTR-
10BitTagComp+ 10BitTagReq- OBFF Not Supported, ExtFmt+ EETLPPrefix+, MaxEETLPPrefixes 1
EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit-
FRS-
AtomicOpsCap: 32bit+ 64bit+ 128bitCAS-
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR- OBFF Disabled,
AtomicOpsCtl: ReqEn+
LnkCap2: Supported Link Speeds: 2.5-16GT/s, Crosslink- Retimer+ 2Retimers+ DRS-
LnkCtl2: Target Link Speed: 16GT/s, EnterCompliance- SpeedDis-
Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
Compliance De-emphasis: -6dB
LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete+ EqualizationPhase1+
EqualizationPhase2+ EqualizationPhase3+ LinkEqualizationRequest-
Retimer- 2Retimers- CrosslinkRes: unsupported
Capabilities: [a0] MSI: Enable- Count=1/4 Maskable- 64bit+
Address: 0000000000000000 Data: 0000
Capabilities: [c0] MSI-X: Enable+ Count=4 Masked-
Vector table: BAR=5 offset=00042000
PBA: BAR=5 offset=00043000
Capabilities: [100 v1] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
Capabilities: [270 v1] Secondary PCI Express
LnkCtl3: LnkEquIntrruptEn- PerformEqu-
LaneErrStat: 0
Capabilities: [2a0 v1] Access Control Services
ACSCap: SrcValid- TransBlk- ReqRedir- CmpltRedir- UpstreamFwd- EgressCtrl- DirectTrans-
ACSCtl: SrcValid- TransBlk- ReqRedir- CmpltRedir- UpstreamFwd- EgressCtrl- DirectTrans-
Capabilities: [2b0 v1] Address Translation Service (ATS)
ATSCap: Invalidate Queue Depth: 00
ATSCtl: Enable+, Smallest Translation Unit: 00
Capabilities: [2c0 v1] Page Request Interface (PRI)
PRICtl: Enable+ Reset-
PRISta: RF- UPRGI- Stopped+
Page Request Capacity: 00000100, Page Request Allocation: 00000020
Capabilities: [2d0 v1] Process Address Space ID (PASID)
PASIDCap: Exec+ Priv+, Max PASID Width: 10
PASIDCtl: Enable+ Exec- Priv-
Capabilities: [410 v1] Physical Layer 16.0 GT/s <?>
Capabilities: [450 v1] Lane Margining at the Receiver <?>
Kernel driver in use: amdgpu
Kernel modules: amdgpu
33:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Device 1640
Subsystem: Lenovo Device 50b4
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 32 bytes
Interrupt: pin B routed to IRQ 94
IOMMU group: 14
Region 0: Memory at 989c8000 (32-bit, non-prefetchable) [size=16K]
Capabilities: [48] Vendor Specific Information: Len=08 <?>
Capabilities: [50] Power Management version 3
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1+,D2+,D3hot+,D3cold+)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [64] Express (v2) Legacy Endpoint, MSI 00
DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s <4us, L1 unlimited
ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset-
DevCtl: CorrErr- NonFatalErr- FatalErr- UnsupReq-
RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop+
MaxPayload 128 bytes, MaxReadReq 512 bytes
DevSta: CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr- TransPend-
LnkCap: Port #0, Speed 16GT/s, Width x16, ASPM L0s L1, Exit Latency L0s <64ns, L1 <1us
ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp+
LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk+
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 16GT/s (ok), Width x16 (ok)
TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
DevCap2: Completion Timeout: Range ABCD, TimeoutDis+ NROPrPrP- LTR-
10BitTagComp+ 10BitTagReq- OBFF Not Supported, ExtFmt+ EETLPPrefix+, MaxEETLPPrefixes 1
EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit-
FRS-
AtomicOpsCap: 32bit- 64bit- 128bitCAS-
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR- OBFF Disabled,
AtomicOpsCtl: ReqEn+
LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete- EqualizationPhase1-
EqualizationPhase2- EqualizationPhase3- LinkEqualizationRequest-
Retimer- 2Retimers- CrosslinkRes: unsupported
Capabilities: [a0] MSI: Enable+ Count=1/1 Maskable- 64bit+
Address: 00000000fee00000 Data: 0000
Capabilities: [100 v1] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
Capabilities: [2a0 v1] Access Control Services
ACSCap: SrcValid- TransBlk- ReqRedir- CmpltRedir- UpstreamFwd- EgressCtrl- DirectTrans-
ACSCtl: SrcValid- TransBlk- ReqRedir- CmpltRedir- UpstreamFwd- EgressCtrl- DirectTrans-
Kernel driver in use: snd_hda_intel
Kernel modules: snd_hda_intel
33:00.2 Encryption controller: Advanced Micro Devices, Inc. [AMD] VanGogh PSP/CCP
Subsystem: Lenovo VanGogh PSP/CCP
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 32 bytes
Interrupt: pin C routed to IRQ 24
IOMMU group: 15
Region 2: Memory at 98800000 (32-bit, non-prefetchable) [size=1M]
Region 5: Memory at 989cc000 (32-bit, non-prefetchable) [size=8K]
Capabilities: [48] Vendor Specific Information: Len=08 <?>
Capabilities: [50] Power Management version 3
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [64] Express (v2) Endpoint, MSI 00
DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s <4us, L1 unlimited
ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset- SlotPowerLimit 0.000W
DevCtl: CorrErr- NonFatalErr- FatalErr- UnsupReq-
RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop+
MaxPayload 128 bytes, MaxReadReq 512 bytes
DevSta: CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr- TransPend-
LnkCap: Port #0, Speed 16GT/s, Width x16, ASPM L0s L1, Exit Latency L0s <64ns, L1 <1us
ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp+
LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk+
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 16GT/s (ok), Width x16 (ok)
TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
DevCap2: Completion Timeout: Range ABCD, TimeoutDis+ NROPrPrP- LTR-
10BitTagComp+ 10BitTagReq- OBFF Not Supported, ExtFmt+ EETLPPrefix+, MaxEETLPPrefixes 1
EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit-
FRS- TPHComp- ExtTPHComp-
AtomicOpsCap: 32bit- 64bit- 128bitCAS-
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR- OBFF Disabled,
AtomicOpsCtl: ReqEn-
LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete- EqualizationPhase1-
EqualizationPhase2- EqualizationPhase3- LinkEqualizationRequest-
Retimer- 2Retimers- CrosslinkRes: unsupported
Capabilities: [a0] MSI: Enable- Count=1/2 Maskable- 64bit+
Address: 0000000000000000 Data: 0000
Capabilities: [c0] MSI-X: Enable+ Count=2 Masked-
Vector table: BAR=5 offset=00000000
PBA: BAR=5 offset=00001000
Capabilities: [100 v1] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
Capabilities: [2a0 v1] Access Control Services
ACSCap: SrcValid- TransBlk- ReqRedir- CmpltRedir- UpstreamFwd- EgressCtrl- DirectTrans-
ACSCtl: SrcValid- TransBlk- ReqRedir- CmpltRedir- UpstreamFwd- EgressCtrl- DirectTrans-
Kernel driver in use: ccp
Kernel modules: ccp
33:00.3 USB controller: Advanced Micro Devices, Inc. [AMD] Device 161d (prog-if 30 [XHCI])
Subsystem: Lenovo Device 50b4
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 32 bytes
Interrupt: pin D routed to IRQ 46
IOMMU group: 16
Region 0: Memory at 98600000 (64-bit, non-prefetchable) [size=1M]
Capabilities: [48] Vendor Specific Information: Len=08 <?>
Capabilities: [50] Power Management version 3
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [64] Express (v2) Endpoint, MSI 00
DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s <4us, L1 unlimited
ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset- SlotPowerLimit 0.000W
DevCtl: CorrErr- NonFatalErr- FatalErr- UnsupReq-
RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop+
MaxPayload 128 bytes, MaxReadReq 512 bytes
DevSta: CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr- TransPend-
LnkCap: Port #0, Speed 16GT/s, Width x16, ASPM L0s L1, Exit Latency L0s <64ns, L1 <1us
ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp+
LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk+
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 16GT/s (ok), Width x16 (ok)
TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
DevCap2: Completion Timeout: Range ABCD, TimeoutDis+ NROPrPrP- LTR-
10BitTagComp+ 10BitTagReq- OBFF Not Supported, ExtFmt+ EETLPPrefix+, MaxEETLPPrefixes 1
EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit-
FRS- TPHComp- ExtTPHComp-
AtomicOpsCap: 32bit- 64bit- 128bitCAS-
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR- OBFF Disabled,
AtomicOpsCtl: ReqEn-
LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete- EqualizationPhase1-
EqualizationPhase2- EqualizationPhase3- LinkEqualizationRequest-
Retimer- 2Retimers- CrosslinkRes: unsupported
Capabilities: [a0] MSI: Enable+ Count=1/1 Maskable- 64bit+
Address: 00000000fee00000 Data: 0000
Capabilities: [c0] MSI-X: Enable- Count=1 Masked-
Vector table: BAR=0 offset=000fe000
PBA: BAR=0 offset=000ff000
Capabilities: [100 v1] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
Capabilities: [2a0 v1] Access Control Services
ACSCap: SrcValid- TransBlk- ReqRedir- CmpltRedir- UpstreamFwd- EgressCtrl- DirectTrans-
ACSCtl: SrcValid- TransBlk- ReqRedir- CmpltRedir- UpstreamFwd- EgressCtrl- DirectTrans-
Kernel driver in use: xhci_hcd
Kernel modules: xhci_pci
33:00.4 USB controller: Advanced Micro Devices, Inc. [AMD] Device 161e (prog-if 30 [XHCI])
Subsystem: Lenovo Device 50b4
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 32 bytes
Interrupt: pin A routed to IRQ 48
IOMMU group: 17
Region 0: Memory at 98700000 (64-bit, non-prefetchable) [size=1M]
Capabilities: [48] Vendor Specific Information: Len=08 <?>
Capabilities: [50] Power Management version 3
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [64] Express (v2) Endpoint, MSI 00
DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s <4us, L1 unlimited
ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset- SlotPowerLimit 0.000W
DevCtl: CorrErr- NonFatalErr- FatalErr- UnsupReq-
RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop+
MaxPayload 128 bytes, MaxReadReq 512 bytes
DevSta: CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr- TransPend-
LnkCap: Port #0, Speed 16GT/s, Width x16, ASPM L0s L1, Exit Latency L0s <64ns, L1 <1us
ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp+
LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk+
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 16GT/s (ok), Width x16 (ok)
TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
DevCap2: Completion Timeout: Range ABCD, TimeoutDis+ NROPrPrP- LTR-
10BitTagComp+ 10BitTagReq- OBFF Not Supported, ExtFmt+ EETLPPrefix+, MaxEETLPPrefixes 1
EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit-
FRS- TPHComp- ExtTPHComp-
AtomicOpsCap: 32bit- 64bit- 128bitCAS-
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR- OBFF Disabled,
AtomicOpsCtl: ReqEn-
LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete- EqualizationPhase1-
EqualizationPhase2- EqualizationPhase3- LinkEqualizationRequest-
Retimer- 2Retimers- CrosslinkRes: unsupported
Capabilities: [a0] MSI: Enable+ Count=1/1 Maskable- 64bit+
Address: 00000000fee00000 Data: 0000
Capabilities: [c0] MSI-X: Enable- Count=1 Masked-
Vector table: BAR=0 offset=000fe000
PBA: BAR=0 offset=000ff000
Capabilities: [100 v1] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
Capabilities: [2a0 v1] Access Control Services
ACSCap: SrcValid- TransBlk- ReqRedir- CmpltRedir- UpstreamFwd- EgressCtrl- DirectTrans-
ACSCtl: SrcValid- TransBlk- ReqRedir- CmpltRedir- UpstreamFwd- EgressCtrl- DirectTrans-
Kernel driver in use: xhci_hcd
Kernel modules: xhci_pci
33:00.5 Multimedia controller: Advanced Micro Devices, Inc. [AMD] Raven/Raven2/FireFlight/Renoir Audio Processor (rev 60)
Subsystem: Lenovo Raven/Raven2/FireFlight/Renoir Audio Processor
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 32 bytes
Interrupt: pin B routed to IRQ 92
IOMMU group: 18
Region 0: Memory at 98980000 (32-bit, non-prefetchable) [size=256K]
Capabilities: [48] Vendor Specific Information: Len=08 <?>
Capabilities: [50] Power Management version 3
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [64] Express (v2) Endpoint, MSI 00
DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s <4us, L1 unlimited
ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset- SlotPowerLimit 0.000W
DevCtl: CorrErr- NonFatalErr- FatalErr- UnsupReq-
RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop+
MaxPayload 128 bytes, MaxReadReq 512 bytes
DevSta: CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr- TransPend-
LnkCap: Port #0, Speed 16GT/s, Width x16, ASPM L0s L1, Exit Latency L0s <64ns, L1 <1us
ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp+
LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk+
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 16GT/s (ok), Width x16 (ok)
TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
DevCap2: Completion Timeout: Range ABCD, TimeoutDis+ NROPrPrP- LTR-
10BitTagComp+ 10BitTagReq- OBFF Not Supported, ExtFmt+ EETLPPrefix+, MaxEETLPPrefixes 1
EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit-
FRS- TPHComp- ExtTPHComp-
AtomicOpsCap: 32bit- 64bit- 128bitCAS-
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR- OBFF Disabled,
AtomicOpsCtl: ReqEn-
LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete- EqualizationPhase1-
EqualizationPhase2- EqualizationPhase3- LinkEqualizationRequest-
Retimer- 2Retimers- CrosslinkRes: unsupported
Capabilities: [a0] MSI: Enable- Count=1/1 Maskable- 64bit+
Address: 0000000000000000 Data: 0000
Capabilities: [100 v1] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
Capabilities: [2a0 v1] Access Control Services
ACSCap: SrcValid- TransBlk- ReqRedir- CmpltRedir- UpstreamFwd- EgressCtrl- DirectTrans-
ACSCtl: SrcValid- TransBlk- ReqRedir- CmpltRedir- UpstreamFwd- EgressCtrl- DirectTrans-
Kernel driver in use: snd_pci_acp6x
Kernel modules: snd_pci_acp3x, snd_rn_pci_acp3x, snd_pci_acp5x, snd_pci_acp6x, snd_acp_pci, snd_rpl_pci_acp6x, snd_pci_ps, snd_sof_amd_renoir, snd_sof_amd_rembrandt
33:00.6 Audio device: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 10h-1fh) HD Audio Controller
Subsystem: Lenovo Family 17h (Models 10h-1fh) HD Audio Controller
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 32 bytes
Interrupt: pin C routed to IRQ 95
IOMMU group: 19
Region 0: Memory at 989c0000 (32-bit, non-prefetchable) [size=32K]
Capabilities: [48] Vendor Specific Information: Len=08 <?>
Capabilities: [50] Power Management version 3
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [64] Express (v2) Endpoint, MSI 00
DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s <4us, L1 unlimited
ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset- SlotPowerLimit 0.000W
DevCtl: CorrErr- NonFatalErr- FatalErr- UnsupReq-
RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop+
MaxPayload 128 bytes, MaxReadReq 512 bytes
DevSta: CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr- TransPend-
LnkCap: Port #0, Speed 16GT/s, Width x16, ASPM L0s L1, Exit Latency L0s <64ns, L1 <1us
ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp+
LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk+
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 16GT/s (ok), Width x16 (ok)
TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
DevCap2: Completion Timeout: Range ABCD, TimeoutDis+ NROPrPrP- LTR-
10BitTagComp+ 10BitTagReq- OBFF Not Supported, ExtFmt+ EETLPPrefix+, MaxEETLPPrefixes 1
EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit-
FRS- TPHComp- ExtTPHComp-
AtomicOpsCap: 32bit- 64bit- 128bitCAS-
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR- OBFF Disabled,
AtomicOpsCtl: ReqEn-
LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete- EqualizationPhase1-
EqualizationPhase2- EqualizationPhase3- LinkEqualizationRequest-
Retimer- 2Retimers- CrosslinkRes: unsupported
Capabilities: [a0] MSI: Enable+ Count=1/1 Maskable- 64bit+
Address: 00000000fee00000 Data: 0000
Capabilities: [100 v1] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
Capabilities: [2a0 v1] Access Control Services
ACSCap: SrcValid- TransBlk- ReqRedir- CmpltRedir- UpstreamFwd- EgressCtrl- DirectTrans-
ACSCtl: SrcValid- TransBlk- ReqRedir- CmpltRedir- UpstreamFwd- EgressCtrl- DirectTrans-
Kernel driver in use: snd_hda_intel
Kernel modules: snd_hda_intel
34:00.0 USB controller: Advanced Micro Devices, Inc. [AMD] Device 161f (prog-if 30 [XHCI])
Subsystem: Lenovo Device 50b4
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 32 bytes
Interrupt: pin A routed to IRQ 50
IOMMU group: 20
Region 0: Memory at 98200000 (64-bit, non-prefetchable) [size=1M]
Capabilities: [48] Vendor Specific Information: Len=08 <?>
Capabilities: [50] Power Management version 3
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [64] Express (v2) Endpoint, MSI 00
DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s <4us, L1 unlimited
ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset- SlotPowerLimit 0.000W
DevCtl: CorrErr- NonFatalErr- FatalErr- UnsupReq-
RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop+
MaxPayload 128 bytes, MaxReadReq 512 bytes
DevSta: CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr- TransPend-
LnkCap: Port #0, Speed 16GT/s, Width x16, ASPM L0s L1, Exit Latency L0s <64ns, L1 <1us
ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp+
LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk+
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 16GT/s (ok), Width x16 (ok)
TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
DevCap2: Completion Timeout: Range ABCD, TimeoutDis+ NROPrPrP- LTR-
10BitTagComp+ 10BitTagReq- OBFF Not Supported, ExtFmt- EETLPPrefix-
EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit-
FRS- TPHComp- ExtTPHComp-
AtomicOpsCap: 32bit- 64bit- 128bitCAS-
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR- OBFF Disabled,
AtomicOpsCtl: ReqEn-
LnkCap2: Supported Link Speeds: 2.5-16GT/s, Crosslink- Retimer+ 2Retimers+ DRS-
LnkCtl2: Target Link Speed: 16GT/s, EnterCompliance- SpeedDis-
Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
Compliance De-emphasis: -6dB
LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete+ EqualizationPhase1+
EqualizationPhase2+ EqualizationPhase3+ LinkEqualizationRequest-
Retimer- 2Retimers- CrosslinkRes: unsupported
Capabilities: [a0] MSI: Enable+ Count=1/1 Maskable- 64bit+
Address: 00000000fee00000 Data: 0000
Capabilities: [c0] MSI-X: Enable- Count=1 Masked-
Vector table: BAR=0 offset=000fe000
PBA: BAR=0 offset=000ff000
Capabilities: [100 v1] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
Capabilities: [270 v1] Secondary PCI Express
LnkCtl3: LnkEquIntrruptEn- PerformEqu-
LaneErrStat: 0
Capabilities: [2a0 v1] Access Control Services
ACSCap: SrcValid- TransBlk- ReqRedir- CmpltRedir- UpstreamFwd- EgressCtrl- DirectTrans-
ACSCtl: SrcValid- TransBlk- ReqRedir- CmpltRedir- UpstreamFwd- EgressCtrl- DirectTrans-
Capabilities: [410 v1] Physical Layer 16.0 GT/s <?>
Capabilities: [450 v1] Lane Margining at the Receiver <?>
Kernel driver in use: xhci_hcd
Kernel modules: xhci_pci
34:00.3 USB controller: Advanced Micro Devices, Inc. [AMD] Device 15d6 (prog-if 30 [XHCI])
Subsystem: Advanced Micro Devices, Inc. [AMD] Device 15d6
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 32 bytes
Interrupt: pin C routed to IRQ 51
IOMMU group: 21
Region 0: Memory at 98300000 (64-bit, non-prefetchable) [size=1M]
Capabilities: [48] Vendor Specific Information: Len=08 <?>
Capabilities: [50] Power Management version 3
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [64] Express (v2) Endpoint, MSI 00
DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s <4us, L1 unlimited
ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset- SlotPowerLimit 0.000W
DevCtl: CorrErr- NonFatalErr- FatalErr- UnsupReq-
RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop+
MaxPayload 128 bytes, MaxReadReq 512 bytes
DevSta: CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr- TransPend-
LnkCap: Port #0, Speed 16GT/s, Width x16, ASPM L0s L1, Exit Latency L0s <64ns, L1 <1us
ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp+
LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk+
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 16GT/s (ok), Width x16 (ok)
TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
DevCap2: Completion Timeout: Range ABCD, TimeoutDis+ NROPrPrP- LTR-
10BitTagComp+ 10BitTagReq- OBFF Not Supported, ExtFmt- EETLPPrefix-
EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit-
FRS- TPHComp- ExtTPHComp-
AtomicOpsCap: 32bit- 64bit- 128bitCAS-
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR- OBFF Disabled,
AtomicOpsCtl: ReqEn-
LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete- EqualizationPhase1-
EqualizationPhase2- EqualizationPhase3- LinkEqualizationRequest-
Retimer- 2Retimers- CrosslinkRes: unsupported
Capabilities: [a0] MSI: Enable+ Count=1/1 Maskable- 64bit+
Address: 00000000fee00000 Data: 0000
Capabilities: [c0] MSI-X: Enable- Count=1 Masked-
Vector table: BAR=0 offset=000fe000
PBA: BAR=0 offset=000ff000
Capabilities: [100 v1] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
Capabilities: [2a0 v1] Access Control Services
ACSCap: SrcValid- TransBlk- ReqRedir- CmpltRedir- UpstreamFwd- EgressCtrl- DirectTrans-
ACSCtl: SrcValid- TransBlk- ReqRedir- CmpltRedir- UpstreamFwd- EgressCtrl- DirectTrans-
Kernel driver in use: xhci_hcd
Kernel modules: xhci_pci
34:00.4 USB controller: Advanced Micro Devices, Inc. [AMD] Device 15d7 (prog-if 30 [XHCI])
Subsystem: Advanced Micro Devices, Inc. [AMD] Device 15d7
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 32 bytes
Interrupt: pin D routed to IRQ 52
IOMMU group: 22
Region 0: Memory at 98400000 (64-bit, non-prefetchable) [size=1M]
Capabilities: [48] Vendor Specific Information: Len=08 <?>
Capabilities: [50] Power Management version 3
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [64] Express (v2) Endpoint, MSI 00
DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s <4us, L1 unlimited
ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset- SlotPowerLimit 0.000W
DevCtl: CorrErr- NonFatalErr- FatalErr- UnsupReq-
RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop+
MaxPayload 128 bytes, MaxReadReq 512 bytes
DevSta: CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr- TransPend-
LnkCap: Port #0, Speed 16GT/s, Width x16, ASPM L0s L1, Exit Latency L0s <64ns, L1 <1us
ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp+
LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk+
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 16GT/s (ok), Width x16 (ok)
TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
DevCap2: Completion Timeout: Range ABCD, TimeoutDis+ NROPrPrP- LTR-
10BitTagComp+ 10BitTagReq- OBFF Not Supported, ExtFmt- EETLPPrefix-
EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit-
FRS- TPHComp- ExtTPHComp-
AtomicOpsCap: 32bit- 64bit- 128bitCAS-
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR- OBFF Disabled,
AtomicOpsCtl: ReqEn-
LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete- EqualizationPhase1-
EqualizationPhase2- EqualizationPhase3- LinkEqualizationRequest-
Retimer- 2Retimers- CrosslinkRes: unsupported
Capabilities: [a0] MSI: Enable+ Count=1/1 Maskable- 64bit+
Address: 00000000fee00000 Data: 0000
Capabilities: [c0] MSI-X: Enable- Count=1 Masked-
Vector table: BAR=0 offset=000fe000
PBA: BAR=0 offset=000ff000
Capabilities: [100 v1] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
Capabilities: [2a0 v1] Access Control Services
ACSCap: SrcValid- TransBlk- ReqRedir- CmpltRedir- UpstreamFwd- EgressCtrl- DirectTrans-
ACSCtl: SrcValid- TransBlk- ReqRedir- CmpltRedir- UpstreamFwd- EgressCtrl- DirectTrans-
Kernel driver in use: xhci_hcd
Kernel modules: xhci_pci
34:00.6 USB controller: Advanced Micro Devices, Inc. [AMD] Device 162f (prog-if 40 [USB4 Host Interface])
Subsystem: Advanced Micro Devices, Inc. [AMD] Device 162f
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 32 bytes
Interrupt: pin B routed to IRQ 75
IOMMU group: 23
Region 0: Memory at 98500000 (64-bit, non-prefetchable) [size=512K]
Capabilities: [48] Vendor Specific Information: Len=08 <?>
Capabilities: [50] Power Management version 3
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [64] Express (v2) Endpoint, MSI 00
DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s <4us, L1 unlimited
ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset- SlotPowerLimit 0.000W
DevCtl: CorrErr- NonFatalErr- FatalErr- UnsupReq-
RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop+
MaxPayload 128 bytes, MaxReadReq 512 bytes
DevSta: CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr- TransPend-
LnkCap: Port #0, Speed 16GT/s, Width x16, ASPM L0s L1, Exit Latency L0s <64ns, L1 <1us
ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp+
LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk+
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 16GT/s (ok), Width x16 (ok)
TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
DevCap2: Completion Timeout: Range ABCD, TimeoutDis+ NROPrPrP- LTR-
10BitTagComp+ 10BitTagReq- OBFF Not Supported, ExtFmt- EETLPPrefix-
EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit-
FRS- TPHComp- ExtTPHComp-
AtomicOpsCap: 32bit- 64bit- 128bitCAS-
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR- OBFF Disabled,
AtomicOpsCtl: ReqEn-
LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete- EqualizationPhase1-
EqualizationPhase2- EqualizationPhase3- LinkEqualizationRequest-
Retimer- 2Retimers- CrosslinkRes: unsupported
Capabilities: [a0] MSI: Enable- Count=1/16 Maskable- 64bit+
Address: 0000000000000000 Data: 0000
Capabilities: [c0] MSI-X: Enable+ Count=16 Masked-
Vector table: BAR=0 offset=0007e000
PBA: BAR=0 offset=0007f000
Capabilities: [100 v1] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
Capabilities: [2a0 v1] Access Control Services
ACSCap: SrcValid- TransBlk- ReqRedir- CmpltRedir- UpstreamFwd- EgressCtrl- DirectTrans-
ACSCtl: SrcValid- TransBlk- ReqRedir- CmpltRedir- UpstreamFwd- EgressCtrl- DirectTrans-
Kernel driver in use: thunderbolt
Kernel modules: thunderbolt
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: Kernel 6.7 regression doesn't boot if using AMD eGPU
2024-04-16 11:49 ` Jason Gunthorpe
@ 2024-04-17 8:48 ` Vasant Hegde
0 siblings, 0 replies; 17+ messages in thread
From: Vasant Hegde @ 2024-04-17 8:48 UTC (permalink / raw)
To: Jason Gunthorpe, Robin Murphy
Cc: Vasant Hegde, Eric Wagner, Joerg Roedel, Will Deacon,
Suravee Suthikulpanit, iommu, linux-kernel
Hi,
On 4/16/2024 5:19 PM, Jason Gunthorpe wrote:
> On Tue, Apr 16, 2024 at 12:25:52PM +0100, Robin Murphy wrote:
>> On 2024-04-16 1:39 am, Jason Gunthorpe wrote:
>>> On Mon, Apr 15, 2024 at 10:44:34PM +0100, Robin Murphy wrote:
>>>> On 2024-04-15 7:57 pm, Eric Wagner wrote:
>>>>> Apologies if I made a mistake in the first bisect, I'm new to kernel
>>>>> debugging.
>>>>>
>>>>> I tested cedc811c76778bdef91d405717acee0de54d8db5 (x86/amd) and
>>>>> 3613047280ec42a4e1350fdc1a6dd161ff4008cc (core) directly and both were good.
>>>>> Then I ran git bisect again with e8cca466a84a75f8ff2a7a31173c99ee6d1c59d2
>>>>> as the bad and 6e6c6d6bc6c96c2477ddfea24a121eb5ee12b7a3 as the good and the
>>>>> bisect log is attached. It ended up at the same commit as before.
>>>>>
>>>>> I've also attached a picture of the boot screen that occurs when it hangs.
>>>>> 0000:05:00.0 is the PCIe bus address of the RX 580 eGPU that's causing the
>>>>> problem.
>>>>
>>>> Looks like 59ddce4418da483 probably broke things most - prior to that, the
>>>> fact that it's behind a Thunderbolt port would have always taken precedence
>>>> and forced IOMMU_DOMAIN_DMA regardless of what the driver may have wanted to
>>>> saywhereas now we ask the driver first, then complain that it conflicts
>>>> with the untrusted status and ultimately don't configure the IOMMU
>>>> at all.
>>>
>>> Yes, if the driver wants to force a domain type it should be
>>> forced. Driver knows best. Eg AMD forces IDENTITY when the HW/driver
>>> is incapable of supporting otherwise.
>>
>> No, in the case of AMD it only forces identity if it thinks the device might
>> want to use PASIDs (because of the architectural limitation that the RID
>> always operates in GPA space so can't have its own independent translation).
>
> AMD forces this because it doesn't yet have a way to automatically
> choose it's v1/v2 page table format during alloc domain. It is just a
> SW bug.
Yes. This will be fixed.
>
> The CC/SNP limitation is also a SW bug but is more fatal as it can't
> even attach a v1 page table in this mode.
Memory Encryption needs Encryption bit support. So we enforce Paging mode (Both
AMD v1 and v2 page table format works fine).
SNP is hardware limitation. When SNP is enabled then IOMMU must be configured
with v1 page table. It won't support Identity mapping.
>
>> Either way, though, there's really little sense to that argument - if
>> enforcing strict translation *might* compromise the device's functionality,
>> we should instead go out of our way to ensure it's definitely as broken as
>> possible? I fail to see how that can be justified as useful or desirable
>> behaviour.
>
> For SNP cases the attach of a DMA domain will fail, so yes, moving the
> failure earlier and giving a clear message is desirable.
Its handled during initialization itself (iommu_snp_enable()). If IOMMU is not
configured with V1 page table we don't enable SNP.
-Vasant
>
>> "Failing" iommu_probe_device is merely how we tell ourselves that we're not
>> interested in a device, and consequently tell the rest of the kernel it
>> doesn't have an IOMMU (via device_iommu_mapped() returning false).
>
> Probing failing with ENODEV means the device has no iommu and the rest
> of the code should assume DMA direct will work.
>
> Probing failing with any other error code means the device has an
> iommu and it couldn't be setup. DMA direct probably won't work today.
>
> If you want all failure codes to mean the device is safe for DMA
> direct then we need to try and attach the IDENTITY domain on various
> probe failure paths too.
>
>> I think I've now satisfied myself that a simple fix for the core code is
>> appropriate and will write that up now; one other thing I couldn't
>> quite
>
> It really doesn't match the design here.
>
> Jason
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: Kernel 6.7 regression doesn't boot if using AMD eGPU
2024-04-17 2:16 ` Eric Wagner
@ 2024-04-17 8:57 ` Vasant Hegde
0 siblings, 0 replies; 17+ messages in thread
From: Vasant Hegde @ 2024-04-17 8:57 UTC (permalink / raw)
To: Eric Wagner
Cc: Jason Gunthorpe, Robin Murphy, Vasant Hegde, Joerg Roedel,
Will Deacon, Suravee Suthikulpanit, iommu, linux-kernel
On 4/17/2024 7:46 AM, Eric Wagner wrote:
> On Tue, Apr 16, 2024 at 6:53 AM Vasant Hegde <vashegde@amd.com> wrote:
>>
>> Hi Eric,
>>
>
.../...
>>>
>>>
>>> As it booted ok with Robin's patch above, these changes to
>>> drivers/iommu/amd/iommu.c didn't seem to make a difference for me. I was testing
>>> with amd iommu v2 off in the kernel config and I also have TSME enabled in the
>>> BIOS if that matters.
>>
>> TMSE is transparent to OS. So its fine.
>>
>> -Vasant
>>
> Output of lspci -vvv and /proc/cmdline attached.
Thanks Eric.
-Vasant
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: Kernel 6.7 regression doesn't boot if using AMD eGPU
2024-04-16 12:44 ` Vasant Hegde
@ 2024-04-17 10:36 ` Robin Murphy
2024-04-18 5:01 ` Vasant Hegde
0 siblings, 1 reply; 17+ messages in thread
From: Robin Murphy @ 2024-04-17 10:36 UTC (permalink / raw)
To: Vasant Hegde, Jason Gunthorpe
Cc: Eric Wagner, Joerg Roedel, Will Deacon, Suravee Suthikulpanit,
iommu, linux-kernel
On 2024-04-16 1:44 pm, Vasant Hegde wrote:
> Robin,
>
> On 4/16/2024 4:55 PM, Robin Murphy wrote:
>> On 2024-04-16 1:39 am, Jason Gunthorpe wrote:
>>> On Mon, Apr 15, 2024 at 10:44:34PM +0100, Robin Murphy wrote:
>>>> On 2024-04-15 7:57 pm, Eric Wagner wrote:
>>>>> Apologies if I made a mistake in the first bisect, I'm new to kernel
>>>>> debugging.
>>>>>
>>>>> I tested cedc811c76778bdef91d405717acee0de54d8db5 (x86/amd) and
>>>>> 3613047280ec42a4e1350fdc1a6dd161ff4008cc (core) directly and both
>>>>> were good.
>>>>> Then I ran git bisect again with
>>>>> e8cca466a84a75f8ff2a7a31173c99ee6d1c59d2
>>>>> as the bad and 6e6c6d6bc6c96c2477ddfea24a121eb5ee12b7a3 as the good
>>>>> and the
>>>>> bisect log is attached. It ended up at the same commit as before.
>>>>>
>>>>> I've also attached a picture of the boot screen that occurs when it
>>>>> hangs.
>>>>> 0000:05:00.0 is the PCIe bus address of the RX 580 eGPU that's
>>>>> causing the
>>>>> problem.
>
> .../...
>
>>
>> "Failing" iommu_probe_device is merely how we tell ourselves that
>> we're not interested in a device, and consequently tell the rest of
>> the kernel it doesn't have an IOMMU (via device_iommu_mapped()
>> returning false). This is normal and expected for devices which
>> legitimately have no IOMMU in the first place; conversely we don't do
>> a great deal for unexpected failures since those typically represent
>> system-fatal conditions whatever we might try to do. We've never had
>> much of a notion of expected failures when an IOMMU *is* present, but
>> even then, denying any trace of the IOMMU and removing ourselves from
>> the picture is clearly not the ideal way to approach that. We're
>> running off a bus notifier (or even later), so ultimately our return
>> value is meaningless; at that point the device already exists and has
>> been added to its bus, we can't undo that.
>>
>> However it looks to be even more fun if failure occurs in *deferred*
>> default domain creation via bus_iommu_probe(), since then we give up
>> and dismiss the entire IOMMU. Except the x86 drivers ignore the return
>> from iommu_device_register(), so further hilarity ensues...
>>
>> I think I've now satisfied myself that a simple fix for the core code
>> is appropriate and will write that up now; one other thing I couldn't
>> quite figure out is whether the AMD driver somehow prevents PASIDs
>> being used while the group is attached to a non-identity (and
>> non-nested) domain - that's probably one for Vasant to confirm.
>
> AMD driver supports PASID with below domain type :
> - Identity domain
> - DMA translation mode (DMA and DMA_FQ) with AMD v2 page table
> (amd_iommu=pgtbl_v2).
>
>
> Currently amd_iommu_def_domain_type() tries to put PASID capable devices
> in identity domain mode. This is something to fix. Its in my TODO list.
> I will try to get into it soon.
>
> Hope this clarifies.
Ooh, I see you now have GIoV to allow that similarly to how SMMUv3 does
it - that wasn't in the older version of the spec that I've previously
been referring to :)
Can you confirm there's no hardware actually been made to the older
spec, supporting v2 and PASIDs but *not* having GIoV? Otherwise, I think
you'll still have the problem that if you use the GPA-SPA translation in
the DTE to implement IOMMU_DOMAIN_DMA for the RID, it makes all the
PASID GVA-GPA mappings useless for host SVA.
Cheers,
Robin.
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: Kernel 6.7 regression doesn't boot if using AMD eGPU
2024-04-17 10:36 ` Robin Murphy
@ 2024-04-18 5:01 ` Vasant Hegde
0 siblings, 0 replies; 17+ messages in thread
From: Vasant Hegde @ 2024-04-18 5:01 UTC (permalink / raw)
To: Robin Murphy, Jason Gunthorpe
Cc: Eric Wagner, Joerg Roedel, Will Deacon, Suravee Suthikulpanit,
iommu, linux-kernel
Hi Robin,
On 4/17/2024 4:06 PM, Robin Murphy wrote:
> On 2024-04-16 1:44 pm, Vasant Hegde wrote:
>> Robin,
>>
>> On 4/16/2024 4:55 PM, Robin Murphy wrote:
>>> On 2024-04-16 1:39 am, Jason Gunthorpe wrote:
>>>> On Mon, Apr 15, 2024 at 10:44:34PM +0100, Robin Murphy wrote:
>>>>> On 2024-04-15 7:57 pm, Eric Wagner wrote:
>>>>>> Apologies if I made a mistake in the first bisect, I'm new to kernel
>>>>>> debugging.
>>>>>>
>>>>>> I tested cedc811c76778bdef91d405717acee0de54d8db5 (x86/amd) and
>>>>>> 3613047280ec42a4e1350fdc1a6dd161ff4008cc (core) directly and both were good.
>>>>>> Then I ran git bisect again with e8cca466a84a75f8ff2a7a31173c99ee6d1c59d2
>>>>>> as the bad and 6e6c6d6bc6c96c2477ddfea24a121eb5ee12b7a3 as the good and the
>>>>>> bisect log is attached. It ended up at the same commit as before.
>>>>>>
>>>>>> I've also attached a picture of the boot screen that occurs when it hangs.
>>>>>> 0000:05:00.0 is the PCIe bus address of the RX 580 eGPU that's causing the
>>>>>> problem.
>>
>> .../...
>>
>>>
>>> "Failing" iommu_probe_device is merely how we tell ourselves that we're not
>>> interested in a device, and consequently tell the rest of the kernel it
>>> doesn't have an IOMMU (via device_iommu_mapped() returning false). This is
>>> normal and expected for devices which legitimately have no IOMMU in the first
>>> place; conversely we don't do a great deal for unexpected failures since
>>> those typically represent system-fatal conditions whatever we might try to
>>> do. We've never had much of a notion of expected failures when an IOMMU *is*
>>> present, but even then, denying any trace of the IOMMU and removing ourselves
>>> from the picture is clearly not the ideal way to approach that. We're running
>>> off a bus notifier (or even later), so ultimately our return value is
>>> meaningless; at that point the device already exists and has been added to
>>> its bus, we can't undo that.
>>>
>>> However it looks to be even more fun if failure occurs in *deferred* default
>>> domain creation via bus_iommu_probe(), since then we give up and dismiss the
>>> entire IOMMU. Except the x86 drivers ignore the return from
>>> iommu_device_register(), so further hilarity ensues...
>>>
>>> I think I've now satisfied myself that a simple fix for the core code is
>>> appropriate and will write that up now; one other thing I couldn't quite
>>> figure out is whether the AMD driver somehow prevents PASIDs being used while
>>> the group is attached to a non-identity (and non-nested) domain - that's
>>> probably one for Vasant to confirm.
>>
>> AMD driver supports PASID with below domain type :
>> - Identity domain
>> - DMA translation mode (DMA and DMA_FQ) with AMD v2 page table
>> (amd_iommu=pgtbl_v2).
>>
>>
>> Currently amd_iommu_def_domain_type() tries to put PASID capable devices in
>> identity domain mode. This is something to fix. Its in my TODO list. I will
>> try to get into it soon.
>>
>> Hope this clarifies.
>
> Ooh, I see you now have GIoV to allow that similarly to how SMMUv3 does it -
> that wasn't in the older version of the spec that I've previously been referring
> to :)
Right. This got added later.
>
> Can you confirm there's no hardware actually been made to the older spec,
> supporting v2 and PASIDs but *not* having GIoV? Otherwise, I think you'll still
> have the problem that if you use the GPA-SPA translation in the DTE to implement
> IOMMU_DOMAIN_DMA for the RID, it makes all the PASID GVA-GPA mappings useless
> for host SVA.
I believe we did made HW with old spec. Fortunately we have sufficient feature
bit to detect those feature support. I will have to carefully tweak the
amd_iommu_def_domain_type().
-Vasant
^ permalink raw reply [flat|nested] 17+ messages in thread
end of thread, other threads:[~2024-04-18 5:01 UTC | newest]
Thread overview: 17+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2024-04-13 22:04 Kernel 6.7 regression doesn't boot if using AMD eGPU Eric Wagner
2024-04-14 0:01 ` Bagas Sanjaya
2024-04-15 8:04 ` Joerg Roedel
2024-04-15 16:30 ` Jason Gunthorpe
2024-04-15 18:57 ` Eric Wagner
2024-04-15 21:44 ` Robin Murphy
2024-04-16 0:39 ` Jason Gunthorpe
[not found] ` <CAHudX3x-ofB=-K6UwWpf4r7Ge9AQbvLQ2qq2C7N-R5LG2qMz7Q@mail.gmail.com>
2024-04-16 10:53 ` Vasant Hegde
2024-04-16 11:39 ` Jason Gunthorpe
2024-04-17 2:16 ` Eric Wagner
2024-04-17 8:57 ` Vasant Hegde
2024-04-16 11:25 ` Robin Murphy
2024-04-16 11:49 ` Jason Gunthorpe
2024-04-17 8:48 ` Vasant Hegde
2024-04-16 12:44 ` Vasant Hegde
2024-04-17 10:36 ` Robin Murphy
2024-04-18 5:01 ` Vasant Hegde
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.