All of lore.kernel.org
 help / color / mirror / Atom feed
* Re: [PATCH] drm/radeon: insist on 32-bit DMA for Cedar on PPC64/PPC64LE
       [not found]   ` <20180222215101.3962-1-bcrocker-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
@ 2018-02-22 16:56     ` Alex Deucher
  2018-02-22 22:52     ` Ben Crocker
  1 sibling, 0 replies; 13+ messages in thread
From: Alex Deucher @ 2018-02-22 16:56 UTC (permalink / raw)
  To: Ben Crocker; +Cc: amd-gfx list, Maling list - DRI developers

On Thu, Feb 22, 2018 at 4:51 PM, Ben Crocker <bcrocker@redhat.com> wrote:
> In radeon_device_init, set the need_dma32 flag for Cedar chips
> (e.g. FirePro 2270).  This fixes, or at least works around, a bug
> on PowerPC exposed by last year's commits
>
> 8e3f1b1d8255105f31556aacf8aeb6071b00d469 (Russell Currey)
>
> and
>
> 253fd51e2f533552ae35a0c661705da6c4842c1b (Alistair Popple)
>
> which enabled the 64-bit DMA iommu bypass.
>
> This caused the device to freeze, in some cases unrecoverably, and is
> the subject of several bug reports internal to Red Hat.
>
> Signed-off-by: Ben Crocker <bcrocker@redhat.com>
> ---
>  drivers/gpu/drm/radeon/radeon_device.c | 5 +++++
>  1 file changed, 5 insertions(+)
>
> diff --git a/drivers/gpu/drm/radeon/radeon_device.c b/drivers/gpu/drm/radeon/radeon_device.c
> index ffc10cadcf34..ba7435bcd208 100644
> --- a/drivers/gpu/drm/radeon/radeon_device.c
> +++ b/drivers/gpu/drm/radeon/radeon_device.c
> @@ -1397,6 +1397,11 @@ int radeon_device_init(struct radeon_device *rdev,
>         if ((rdev->flags & RADEON_IS_PCI) &&
>             (rdev->family <= CHIP_RS740))
>                 rdev->need_dma32 = true;
> +#ifdef CONFIG_PPC64
> +       if ((rdev->flags & (RADEON_IS_PCI | RADEON_IS_PCIE)) &&
> +           (rdev->family == CHIP_CEDAR))
> +               rdev->need_dma32 = true;
> +#endif

You can drop the PCI and PCIE checks and just check the family.

Alex

>
>         dma_bits = rdev->need_dma32 ? 32 : 40;
>         r = pci_set_dma_mask(rdev->pdev, DMA_BIT_MASK(dma_bits));
> --
> 2.13.6
>
> _______________________________________________
> amd-gfx mailing list
> amd-gfx@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/amd-gfx
_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

^ permalink raw reply	[flat|nested] 13+ messages in thread

* [PATCH] drm/radeon: insist on 32-bit DMA for Cedar on PPC64/PPC64LE
       [not found] <to=<20180221234154.6877-1-bcrocker@redhat.com>
@ 2018-02-22 21:51 ` Ben Crocker
       [not found]   ` <20180222215101.3962-1-bcrocker-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
  0 siblings, 1 reply; 13+ messages in thread
From: Ben Crocker @ 2018-02-22 21:51 UTC (permalink / raw)
  To: dri-devel, amd-gfx; +Cc: Ben Crocker

In radeon_device_init, set the need_dma32 flag for Cedar chips
(e.g. FirePro 2270).  This fixes, or at least works around, a bug
on PowerPC exposed by last year's commits

8e3f1b1d8255105f31556aacf8aeb6071b00d469 (Russell Currey)

and

253fd51e2f533552ae35a0c661705da6c4842c1b (Alistair Popple)

which enabled the 64-bit DMA iommu bypass.

This caused the device to freeze, in some cases unrecoverably, and is
the subject of several bug reports internal to Red Hat.

Signed-off-by: Ben Crocker <bcrocker@redhat.com>
---
 drivers/gpu/drm/radeon/radeon_device.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/drivers/gpu/drm/radeon/radeon_device.c b/drivers/gpu/drm/radeon/radeon_device.c
index ffc10cadcf34..ba7435bcd208 100644
--- a/drivers/gpu/drm/radeon/radeon_device.c
+++ b/drivers/gpu/drm/radeon/radeon_device.c
@@ -1397,6 +1397,11 @@ int radeon_device_init(struct radeon_device *rdev,
 	if ((rdev->flags & RADEON_IS_PCI) &&
 	    (rdev->family <= CHIP_RS740))
 		rdev->need_dma32 = true;
+#ifdef CONFIG_PPC64
+	if ((rdev->flags & (RADEON_IS_PCI | RADEON_IS_PCIE)) &&
+	    (rdev->family == CHIP_CEDAR))
+		rdev->need_dma32 = true;
+#endif
 
 	dma_bits = rdev->need_dma32 ? 32 : 40;
 	r = pci_set_dma_mask(rdev->pdev, DMA_BIT_MASK(dma_bits));
-- 
2.13.6

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply related	[flat|nested] 13+ messages in thread

* [PATCH] drm/radeon: insist on 32-bit DMA for Cedar on PPC64/PPC64LE
       [not found]   ` <20180222215101.3962-1-bcrocker-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
  2018-02-22 16:56     ` Alex Deucher
@ 2018-02-22 22:52     ` Ben Crocker
  2018-02-23  4:38       ` Alex Deucher
  1 sibling, 1 reply; 13+ messages in thread
From: Ben Crocker @ 2018-02-22 22:52 UTC (permalink / raw)
  To: dri-devel-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW,
	amd-gfx-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW
  Cc: Ben Crocker

In radeon_device_init, set the need_dma32 flag for Cedar chips
(e.g. FirePro 2270).  This fixes, or at least works around, a bug
on PowerPC exposed by last year's commits

8e3f1b1d8255105f31556aacf8aeb6071b00d469 (Russell Currey)

and

253fd51e2f533552ae35a0c661705da6c4842c1b (Alistair Popple)

which enabled the 64-bit DMA iommu bypass.

This caused the device to freeze, in some cases unrecoverably, and is
the subject of several bug reports internal to Red Hat.

Signed-off-by: Ben Crocker <bcrocker@redhat.com>
---
 drivers/gpu/drm/radeon/radeon_device.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/drivers/gpu/drm/radeon/radeon_device.c b/drivers/gpu/drm/radeon/radeon_device.c
index ffc10cadcf34..32b577c776b9 100644
--- a/drivers/gpu/drm/radeon/radeon_device.c
+++ b/drivers/gpu/drm/radeon/radeon_device.c
@@ -1397,6 +1397,10 @@ int radeon_device_init(struct radeon_device *rdev,
 	if ((rdev->flags & RADEON_IS_PCI) &&
 	    (rdev->family <= CHIP_RS740))
 		rdev->need_dma32 = true;
+#ifdef CONFIG_PPC64
+	if (rdev->family == CHIP_CEDAR)
+		rdev->need_dma32 = true;
+#endif
 
 	dma_bits = rdev->need_dma32 ? 32 : 40;
 	r = pci_set_dma_mask(rdev->pdev, DMA_BIT_MASK(dma_bits));
-- 
2.13.6

_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

^ permalink raw reply related	[flat|nested] 13+ messages in thread

* Re: [PATCH] drm/radeon: insist on 32-bit DMA for Cedar on PPC64/PPC64LE
  2018-02-22 22:52     ` Ben Crocker
@ 2018-02-23  4:38       ` Alex Deucher
  0 siblings, 0 replies; 13+ messages in thread
From: Alex Deucher @ 2018-02-23  4:38 UTC (permalink / raw)
  To: Ben Crocker; +Cc: amd-gfx list, Maling list - DRI developers

On Thu, Feb 22, 2018 at 5:52 PM, Ben Crocker <bcrocker@redhat.com> wrote:
> In radeon_device_init, set the need_dma32 flag for Cedar chips
> (e.g. FirePro 2270).  This fixes, or at least works around, a bug
> on PowerPC exposed by last year's commits
>
> 8e3f1b1d8255105f31556aacf8aeb6071b00d469 (Russell Currey)
>
> and
>
> 253fd51e2f533552ae35a0c661705da6c4842c1b (Alistair Popple)
>
> which enabled the 64-bit DMA iommu bypass.
>
> This caused the device to freeze, in some cases unrecoverably, and is
> the subject of several bug reports internal to Red Hat.
>
> Signed-off-by: Ben Crocker <bcrocker@redhat.com>

Applied. thanks!

Alex

> ---
>  drivers/gpu/drm/radeon/radeon_device.c | 4 ++++
>  1 file changed, 4 insertions(+)
>
> diff --git a/drivers/gpu/drm/radeon/radeon_device.c b/drivers/gpu/drm/radeon/radeon_device.c
> index ffc10cadcf34..32b577c776b9 100644
> --- a/drivers/gpu/drm/radeon/radeon_device.c
> +++ b/drivers/gpu/drm/radeon/radeon_device.c
> @@ -1397,6 +1397,10 @@ int radeon_device_init(struct radeon_device *rdev,
>         if ((rdev->flags & RADEON_IS_PCI) &&
>             (rdev->family <= CHIP_RS740))
>                 rdev->need_dma32 = true;
> +#ifdef CONFIG_PPC64
> +       if (rdev->family == CHIP_CEDAR)
> +               rdev->need_dma32 = true;
> +#endif
>
>         dma_bits = rdev->need_dma32 ? 32 : 40;
>         r = pci_set_dma_mask(rdev->pdev, DMA_BIT_MASK(dma_bits));
> --
> 2.13.6
>
> _______________________________________________
> dri-devel mailing list
> dri-devel@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/dri-devel
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [PATCH] drm/radeon: insist on 32-bit DMA for Cedar on PPC64/PPC64LE
  2018-02-23 16:02               ` Ben Crocker
@ 2018-02-23 16:37                 ` Christian König
  0 siblings, 0 replies; 13+ messages in thread
From: Christian König @ 2018-02-23 16:37 UTC (permalink / raw)
  To: Ben Crocker
  Cc: Michel Dänzer, Maling list - DRI developers, amd-gfx list


[-- Attachment #1.1: Type: text/plain, Size: 2694 bytes --]

Hi Ben,

> I mentioned the HD 5450 purely as a matter of archeological interest.
Ok, in that case looking at the lspci -tv output doesn't tell us 
anything new.

> I suspected, but couldn't prove, that the problem might have something
> to do with the bridge (IBM POWER8 Host Bridge (PHB3)).

In that case you would see the problem with the caicos as well.

Or are there issues random enough that there could actually be some 
problem with the caicos as well and we haven't noticed it so far?

See it is really really odd that this should only happen with the cedar. 
On the other hand if it works for now, I don't see much issue having 
this workaround.

Regards,
Christian.

Am 23.02.2018 um 17:02 schrieb Ben Crocker:
> Hi Christian, Michel, Alex, et al.,
>
> I mentioned the HD 5450 purely as a matter of archeological interest.
>
> Back to the FirePro 2270 and Embedded Radeon E6465:
>
> I've attached text from both "lspci -tv" and "lspci -v."
> Actually I'm attaching a couple of different "lspci -v" outputs, one
> with the FirePro 2270 in place and one with the E6465 in (the same) place.
>
> I suspected, but couldn't prove, that the problem might have something
> to do with the bridge (IBM POWER8 Host Bridge (PHB3)).
>
> But I want to reiterate:
>
>   * Cedar GPU -> problem (before patch, that is);
>   * Caicos GPU in the same slot -> no problem
>
>
> -- Ben
>
> P.S.  Alex, thanks for applying the patch so expeditiously!
>
>
> On Fri, Feb 23, 2018 at 2:45 AM, Christian König 
> <ckoenig.leichtzumerken@gmail.com 
> <mailto:ckoenig.leichtzumerken@gmail.com>> wrote:
>
>     Am 22.02.2018 um 18:56 schrieb Michel Dänzer:
>
>         On 2018-02-22 06:37 PM, Ben Crocker wrote:
>
>             One of my colleagues did discover a "Radeon HG 5450 PCI"
>             from February
>             2010 which did, apparently, have a Cedar GPU and very
>             definitely had a
>             (plain old) PCI connector.
>
>         There must be a PCIe-to-PCI bridge on that board. The GPU
>         itself is
>         always PCIe, and treated accordingly by the driver.
>
>
>     Ben, just an educated guess but is this one the one which is
>     failing to work correctly?
>
>     Cause the PCIe bus interface is pretty much identical over all
>     generations of the last decade or so. Only the newest Vega10
>     generation is a bit different.
>
>     So I strongly thing that this isn't related to the device being a
>     Cedar at all, but rather that you have a bridge above it which
>     doesn't correctly handle 64bit transfers.
>
>     Can you please send and "lspci -t" of both the working and the
>     problematic devices?
>
>     Thanks,
>     Christian.
>
>


[-- Attachment #1.2: Type: text/html, Size: 4696 bytes --]

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [PATCH] drm/radeon: insist on 32-bit DMA for Cedar on PPC64/PPC64LE
  2018-02-23  7:45             ` Christian König
@ 2018-02-23 16:02               ` Ben Crocker
  2018-02-23 16:37                 ` Christian König
  0 siblings, 1 reply; 13+ messages in thread
From: Ben Crocker @ 2018-02-23 16:02 UTC (permalink / raw)
  To: Christian Koenig
  Cc: Michel Dänzer, Maling list - DRI developers, amd-gfx list


[-- Attachment #1.1: Type: text/plain, Size: 1803 bytes --]

Hi Christian, Michel, Alex, et al.,

I mentioned the HD 5450 purely as a matter of archeological interest.

Back to the FirePro 2270 and Embedded Radeon E6465:

I've attached text from both "lspci -tv" and "lspci -v."
Actually I'm attaching a couple of different "lspci -v" outputs, one
with the FirePro 2270 in place and one with the E6465 in (the same) place.

I suspected, but couldn't prove, that the problem might have something
to do with the bridge (IBM POWER8 Host Bridge (PHB3)).

But I want to reiterate:

   - Cedar GPU -> problem (before patch, that is);
   - Caicos GPU in the same slot -> no problem


-- Ben

P.S.  Alex, thanks for applying the patch so expeditiously!


On Fri, Feb 23, 2018 at 2:45 AM, Christian König <
ckoenig.leichtzumerken@gmail.com> wrote:

> Am 22.02.2018 um 18:56 schrieb Michel Dänzer:
>
>> On 2018-02-22 06:37 PM, Ben Crocker wrote:
>>
>>> One of my colleagues did discover a "Radeon HG 5450 PCI" from February
>>> 2010 which did, apparently, have a Cedar GPU and very definitely had a
>>> (plain old) PCI connector.
>>>
>> There must be a PCIe-to-PCI bridge on that board. The GPU itself is
>> always PCIe, and treated accordingly by the driver.
>>
>
> Ben, just an educated guess but is this one the one which is failing to
> work correctly?
>
> Cause the PCIe bus interface is pretty much identical over all generations
> of the last decade or so. Only the newest Vega10 generation is a bit
> different.
>
> So I strongly thing that this isn't related to the device being a Cedar at
> all, but rather that you have a bridge above it which doesn't correctly
> handle 64bit transfers.
>
> Can you please send and "lspci -t" of both the working and the problematic
> devices?
>
> Thanks,
> Christian.
>

[-- Attachment #1.2: Type: text/html, Size: 2485 bytes --]

[-- Attachment #2: lspci-tv.txt --]
[-- Type: text/plain, Size: 1474 bytes --]

-+-[0045:00]---00.0-[01]----00.0  Texas Instruments TUSB73x0 SuperSpeed USB 3.0 xHCI Host Controller
 +-[0044:00]---00.0-[01]--
 +-[0040:00]---00.0-[01]--
 +-[0005:00]---00.0-[01-0f]----00.0-[02-0f]--+-01.0-[03]----00.0  Texas Instruments TUSB73x0 SuperSpeed USB 3.0 xHCI Host Controller
 |                                           +-08.0-[04]----00.0  IBM PCI-E IPR SAS Adapter (ASIC)
 |                                           +-09.0-[05]--+-00.0  Broadcom Limited NetXtreme BCM5719 Gigabit Ethernet PCIe
 |                                           |            +-00.1  Broadcom Limited NetXtreme BCM5719 Gigabit Ethernet PCIe
 |                                           |            +-00.2  Broadcom Limited NetXtreme BCM5719 Gigabit Ethernet PCIe
 |                                           |            \-00.3  Broadcom Limited NetXtreme BCM5719 Gigabit Ethernet PCIe
 |                                           +-10.0-[06-0a]--
 |                                           \-11.0-[0b-0f]--
 +-[0004:00]---00.0-[01]--+-00.0  Advanced Micro Devices, Inc. [AMD/ATI] Cedar GL [FirePro 2270]
 |                        \-00.1  Advanced Micro Devices, Inc. [AMD/ATI] Cedar HDMI Audio [Radeon HD 5400/6300/7300 Series]
 +-[0001:00]---00.0-[01-0d]----00.0-[02-0d]--+-01.0-[03-07]--
 |                                           +-08.0-[08]----00.0  IBM PCI-E IPR SAS Adapter (ASIC)
 |                                           \-09.0-[09-0d]--
 \-[0000:00]---00.0-[01]--

[-- Attachment #3: lspci-v-cedar.txt --]
[-- Type: text/plain, Size: 23247 bytes --]

0000:00:00.0 PCI bridge: IBM POWER8 Host Bridge (PHB3) (prog-if 00 [Normal decode])
	Flags: bus master, fast devsel, latency 0, NUMA node 0
	Bus: primary=00, secondary=01, subordinate=01, sec-latency=0
	I/O behind bridge: None
	Memory behind bridge: None
	Prefetchable memory behind bridge: None
	Capabilities: [40] Power Management version 3
	Capabilities: [48] Express Root Port (Slot-), MSI 00
	Capabilities: [100] Advanced Error Reporting
	Capabilities: [148] #19
	Kernel modules: shpchp

0001:00:00.0 PCI bridge: IBM POWER8 Host Bridge (PHB3) (prog-if 00 [Normal decode])
	Flags: bus master, fast devsel, latency 0, NUMA node 0
	Bus: primary=00, secondary=01, subordinate=0d, sec-latency=0
	I/O behind bridge: None
	Memory behind bridge: 80000000-ffefffff [size=2047M]
	Prefetchable memory behind bridge: 0000210000000000-000021fdffffffff [size=1016G]
	Capabilities: [40] Power Management version 3
	Capabilities: [48] Express Root Port (Slot-), MSI 00
	Capabilities: [100] Advanced Error Reporting
	Capabilities: [148] #19
	Kernel modules: shpchp

0001:01:00.0 PCI bridge: PLX Technology, Inc. PEX 8732 32-lane, 8-Port PCI Express Gen 3 (8.0 GT/s) Switch (rev ca) (prog-if 00 [Normal decode])
	Flags: bus master, fast devsel, latency 0, IRQ 493, NUMA node 0
	Memory at 3fe082800000 (32-bit, non-prefetchable) [size=256K]
	Bus: primary=01, secondary=02, subordinate=0d, sec-latency=0
	I/O behind bridge: None
	Memory behind bridge: 80000000-ffefffff [size=2047M]
	Prefetchable memory behind bridge: 0000210000000000-000021fdffffffff [size=1016G]
	Capabilities: [40] Power Management version 3
	Capabilities: [48] MSI: Enable- Count=1/8 Maskable+ 64bit+
	Capabilities: [68] Express Upstream Port, MSI 00
	Capabilities: [a4] Subsystem: PLX Technology, Inc. PEX 8732 32-lane, 8-Port PCI Express Gen 3 (8.0 GT/s) Switch
	Capabilities: [100] Device Serial Number ca-87-00-10-07-08-28-15
	Capabilities: [fb4] Advanced Error Reporting
	Capabilities: [138] Power Budgeting <?>
	Capabilities: [10c] #19
	Capabilities: [148] Virtual Channel
	Capabilities: [e00] #12
	Capabilities: [b00] Latency Tolerance Reporting
	Capabilities: [b70] Vendor Specific Information: ID=0001 Rev=0 Len=010 <?>
	Kernel modules: shpchp

0001:02:01.0 PCI bridge: PLX Technology, Inc. PEX 8732 32-lane, 8-Port PCI Express Gen 3 (8.0 GT/s) Switch (rev ca) (prog-if 00 [Normal decode])
	Flags: bus master, fast devsel, latency 0, IRQ 494, NUMA node 0
	Bus: primary=02, secondary=03, subordinate=07, sec-latency=0
	I/O behind bridge: None
	Memory behind bridge: 80000000-807fffff [size=8M]
	Prefetchable memory behind bridge: 0000210000000000-00002100ffffffff [size=4G]
	Capabilities: [40] Power Management version 3
	Capabilities: [48] MSI: Enable- Count=1/8 Maskable+ 64bit+
	Capabilities: [68] Express Downstream Port (Slot+), MSI 00
	Capabilities: [a4] Subsystem: PLX Technology, Inc. PEX 8732 32-lane, 8-Port PCI Express Gen 3 (8.0 GT/s) Switch
	Capabilities: [100] Device Serial Number ca-87-00-10-b5-df-0e-00
	Capabilities: [fb4] Advanced Error Reporting
	Capabilities: [138] Power Budgeting <?>
	Capabilities: [10c] #19
	Capabilities: [148] Virtual Channel
	Capabilities: [e00] #12
	Capabilities: [f24] Access Control Services
	Capabilities: [b70] Vendor Specific Information: ID=0001 Rev=0 Len=010 <?>
	Kernel modules: shpchp

0001:02:08.0 PCI bridge: PLX Technology, Inc. PEX 8732 32-lane, 8-Port PCI Express Gen 3 (8.0 GT/s) Switch (rev ca) (prog-if 00 [Normal decode])
	Flags: bus master, fast devsel, latency 0, IRQ 493, NUMA node 0
	Bus: primary=02, secondary=08, subordinate=08, sec-latency=0
	I/O behind bridge: None
	Memory behind bridge: 80800000-80ffffff [size=8M]
	Prefetchable memory behind bridge: 0000210100000000-00002101ffffffff [size=4G]
	Capabilities: [40] Power Management version 3
	Capabilities: [48] MSI: Enable- Count=1/8 Maskable+ 64bit+
	Capabilities: [68] Express Downstream Port (Slot+), MSI 00
	Capabilities: [a4] Subsystem: PLX Technology, Inc. PEX 8732 32-lane, 8-Port PCI Express Gen 3 (8.0 GT/s) Switch
	Capabilities: [100] Device Serial Number ca-87-00-10-b5-df-0e-00
	Capabilities: [fb4] Advanced Error Reporting
	Capabilities: [138] Power Budgeting <?>
	Capabilities: [10c] #19
	Capabilities: [148] Virtual Channel
	Capabilities: [e00] #12
	Capabilities: [f24] Access Control Services
	Capabilities: [b70] Vendor Specific Information: ID=0001 Rev=0 Len=010 <?>
	Kernel modules: shpchp

0001:02:09.0 PCI bridge: PLX Technology, Inc. PEX 8732 32-lane, 8-Port PCI Express Gen 3 (8.0 GT/s) Switch (rev ca) (prog-if 00 [Normal decode])
	Flags: bus master, fast devsel, latency 0, IRQ 494, NUMA node 0
	Bus: primary=02, secondary=09, subordinate=0d, sec-latency=0
	I/O behind bridge: None
	Memory behind bridge: 81000000-817fffff [size=8M]
	Prefetchable memory behind bridge: 0000210200000000-00002102ffffffff [size=4G]
	Capabilities: [40] Power Management version 3
	Capabilities: [48] MSI: Enable- Count=1/8 Maskable+ 64bit+
	Capabilities: [68] Express Downstream Port (Slot+), MSI 00
	Capabilities: [a4] Subsystem: PLX Technology, Inc. PEX 8732 32-lane, 8-Port PCI Express Gen 3 (8.0 GT/s) Switch
	Capabilities: [100] Device Serial Number ca-87-00-10-b5-df-0e-00
	Capabilities: [fb4] Advanced Error Reporting
	Capabilities: [138] Power Budgeting <?>
	Capabilities: [10c] #19
	Capabilities: [148] Virtual Channel
	Capabilities: [e00] #12
	Capabilities: [f24] Access Control Services
	Capabilities: [b70] Vendor Specific Information: ID=0001 Rev=0 Len=010 <?>
	Kernel modules: shpchp

0001:08:00.0 RAID bus controller: IBM PCI-E IPR SAS Adapter (ASIC) (rev 02)
	Subsystem: IBM PCIe3 x8 Cache SAS RAID Internal Adapter 6Gb (57D8)
	Flags: bus master, fast devsel, latency 0, NUMA node 0
	Memory at 3fe080820000 (64-bit, non-prefetchable) [size=64K]
	Memory at 3fe080830000 (64-bit, non-prefetchable) [size=64K]
	[virtual] Expansion ROM at 3fe080800000 [disabled] [size=128K]
	Capabilities: [40] Power Management version 3
	Capabilities: [48] MSI: Enable- Count=1/1 Maskable+ 64bit+
	Capabilities: [60] Express Endpoint, MSI 00
	Capabilities: [a4] MSI-X: Enable+ Count=16 Masked-
	Capabilities: [100] Advanced Error Reporting
	Capabilities: [188] Alternative Routing-ID Interpretation (ARI)
	Capabilities: [190] #19
	Kernel driver in use: ipr
	Kernel modules: ipr

0004:00:00.0 PCI bridge: IBM POWER8 Host Bridge (PHB3) (prog-if 00 [Normal decode])
	Flags: bus master, fast devsel, latency 0, NUMA node 1
	Bus: primary=00, secondary=01, subordinate=01, sec-latency=0
	I/O behind bridge: None
	Memory behind bridge: 80000000-ffefffff [size=2047M]
	Prefetchable memory behind bridge: 0000240000000000-000024fdffffffff [size=1016G]
	Capabilities: [40] Power Management version 3
	Capabilities: [48] Express Root Port (Slot-), MSI 00
	Capabilities: [100] Advanced Error Reporting
	Capabilities: [148] #19
	Kernel modules: shpchp

0004:01:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Cedar GL [FirePro 2270] (prog-if 00 [VGA controller])
	Subsystem: Advanced Micro Devices, Inc. [AMD/ATI] Device 0126
	Flags: bus master, fast devsel, latency 0, IRQ 398, NUMA node 1
	Memory at 240000000000 (64-bit, prefetchable) [size=256M]
	Memory at 3fe200000000 (64-bit, non-prefetchable) [size=128K]
	I/O ports at <unassigned> [disabled]
	Expansion ROM at 3fe200020000 [disabled] [size=128K]
	Capabilities: [50] Power Management version 3
	Capabilities: [58] Express Legacy Endpoint, MSI 00
	Capabilities: [a0] MSI: Enable+ Count=1/1 Maskable- 64bit+
	Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
	Capabilities: [150] Advanced Error Reporting
	Kernel driver in use: radeon
	Kernel modules: radeon

0004:01:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Cedar HDMI Audio [Radeon HD 5400/6300/7300 Series]
	Subsystem: Advanced Micro Devices, Inc. [AMD/ATI] Cedar HDMI Audio [Radeon HD 5400/6300/7300 Series]
	Flags: fast devsel, IRQ 482, NUMA node 1
	Memory at 3fe200040000 (64-bit, non-prefetchable) [disabled] [size=16K]
	Capabilities: [50] Power Management version 3
	Capabilities: [58] Express Legacy Endpoint, MSI 00
	Capabilities: [a0] MSI: Enable- Count=1/1 Maskable- 64bit+
	Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
	Capabilities: [150] Advanced Error Reporting

0005:00:00.0 PCI bridge: IBM POWER8 Host Bridge (PHB3) (prog-if 00 [Normal decode])
	Flags: bus master, fast devsel, latency 0, NUMA node 1
	Bus: primary=00, secondary=01, subordinate=0f, sec-latency=0
	I/O behind bridge: None
	Memory behind bridge: 80000000-ffefffff [size=2047M]
	Prefetchable memory behind bridge: 0000250000000000-000025fdffffffff [size=1016G]
	Capabilities: [40] Power Management version 3
	Capabilities: [48] Express Root Port (Slot-), MSI 00
	Capabilities: [100] Advanced Error Reporting
	Capabilities: [148] #19
	Kernel modules: shpchp

0005:01:00.0 PCI bridge: PLX Technology, Inc. PEX 8748 48-Lane, 12-Port PCI Express Gen 3 (8 GT/s) Switch, 27 x 27mm FCBGA (rev ca) (prog-if 00 [Normal decode])
	Flags: bus master, fast devsel, latency 0, IRQ 477, NUMA node 1
	Memory at 3fe283800000 (32-bit, non-prefetchable) [size=256K]
	Bus: primary=01, secondary=02, subordinate=0f, sec-latency=0
	I/O behind bridge: None
	Memory behind bridge: 80000000-ffefffff [size=2047M]
	Prefetchable memory behind bridge: 0000250000000000-000025fdffffffff [size=1016G]
	Capabilities: [40] Power Management version 3
	Capabilities: [48] MSI: Enable- Count=1/8 Maskable+ 64bit+
	Capabilities: [68] Express Upstream Port, MSI 00
	Capabilities: [a4] Subsystem: PLX Technology, Inc. PEX 8748 48-Lane, 12-Port PCI Express Gen 3 (8 GT/s) Switch, 27 x 27mm FCBGA
	Capabilities: [100] Device Serial Number ca-87-00-10-07-08-28-15
	Capabilities: [fb4] Advanced Error Reporting
	Capabilities: [138] Power Budgeting <?>
	Capabilities: [10c] #19
	Capabilities: [148] Virtual Channel
	Capabilities: [e00] #12
	Capabilities: [b00] Latency Tolerance Reporting
	Capabilities: [b70] Vendor Specific Information: ID=0001 Rev=0 Len=010 <?>
	Kernel modules: shpchp

0005:02:01.0 PCI bridge: PLX Technology, Inc. PEX 8748 48-Lane, 12-Port PCI Express Gen 3 (8 GT/s) Switch, 27 x 27mm FCBGA (rev ca) (prog-if 00 [Normal decode])
	Flags: bus master, fast devsel, latency 0, IRQ 478, NUMA node 1
	Bus: primary=02, secondary=03, subordinate=03, sec-latency=0
	I/O behind bridge: None
	Memory behind bridge: 80000000-807fffff [size=8M]
	Prefetchable memory behind bridge: None
	Capabilities: [40] Power Management version 3
	Capabilities: [48] MSI: Enable- Count=1/8 Maskable+ 64bit+
	Capabilities: [68] Express Downstream Port (Slot-), MSI 00
	Capabilities: [a4] Subsystem: PLX Technology, Inc. PEX 8748 48-Lane, 12-Port PCI Express Gen 3 (8 GT/s) Switch, 27 x 27mm FCBGA
	Capabilities: [100] Device Serial Number ca-87-00-10-b5-df-0e-00
	Capabilities: [fb4] Advanced Error Reporting
	Capabilities: [138] Power Budgeting <?>
	Capabilities: [10c] #19
	Capabilities: [148] Virtual Channel
	Capabilities: [e00] #12
	Capabilities: [f24] Access Control Services
	Capabilities: [b70] Vendor Specific Information: ID=0001 Rev=0 Len=010 <?>
	Kernel modules: shpchp

0005:02:08.0 PCI bridge: PLX Technology, Inc. PEX 8748 48-Lane, 12-Port PCI Express Gen 3 (8 GT/s) Switch, 27 x 27mm FCBGA (rev ca) (prog-if 00 [Normal decode])
	Flags: bus master, fast devsel, latency 0, IRQ 477, NUMA node 1
	Bus: primary=02, secondary=04, subordinate=04, sec-latency=0
	I/O behind bridge: None
	Memory behind bridge: 80800000-80ffffff [size=8M]
	Prefetchable memory behind bridge: 0000250000000000-00002500ffffffff [size=4G]
	Capabilities: [40] Power Management version 3
	Capabilities: [48] MSI: Enable- Count=1/8 Maskable+ 64bit+
	Capabilities: [68] Express Downstream Port (Slot+), MSI 00
	Capabilities: [a4] Subsystem: PLX Technology, Inc. PEX 8748 48-Lane, 12-Port PCI Express Gen 3 (8 GT/s) Switch, 27 x 27mm FCBGA
	Capabilities: [100] Device Serial Number ca-87-00-10-b5-df-0e-00
	Capabilities: [fb4] Advanced Error Reporting
	Capabilities: [138] Power Budgeting <?>
	Capabilities: [10c] #19
	Capabilities: [148] Virtual Channel
	Capabilities: [e00] #12
	Capabilities: [f24] Access Control Services
	Capabilities: [b70] Vendor Specific Information: ID=0001 Rev=0 Len=010 <?>
	Kernel modules: shpchp

0005:02:09.0 PCI bridge: PLX Technology, Inc. PEX 8748 48-Lane, 12-Port PCI Express Gen 3 (8 GT/s) Switch, 27 x 27mm FCBGA (rev ca) (prog-if 00 [Normal decode])
	Flags: bus master, fast devsel, latency 0, IRQ 478, NUMA node 1
	Bus: primary=02, secondary=05, subordinate=05, sec-latency=0
	I/O behind bridge: None
	Memory behind bridge: 81000000-817fffff [size=8M]
	Prefetchable memory behind bridge: 0000250100000000-00002501ffffffff [size=4G]
	Capabilities: [40] Power Management version 3
	Capabilities: [48] MSI: Enable- Count=1/8 Maskable+ 64bit+
	Capabilities: [68] Express Downstream Port (Slot+), MSI 00
	Capabilities: [a4] Subsystem: PLX Technology, Inc. PEX 8748 48-Lane, 12-Port PCI Express Gen 3 (8 GT/s) Switch, 27 x 27mm FCBGA
	Capabilities: [100] Device Serial Number ca-87-00-10-b5-df-0e-00
	Capabilities: [fb4] Advanced Error Reporting
	Capabilities: [138] Power Budgeting <?>
	Capabilities: [10c] #19
	Capabilities: [148] Virtual Channel
	Capabilities: [e00] #12
	Capabilities: [f24] Access Control Services
	Capabilities: [b70] Vendor Specific Information: ID=0001 Rev=0 Len=010 <?>
	Kernel modules: shpchp

0005:02:10.0 PCI bridge: PLX Technology, Inc. PEX 8748 48-Lane, 12-Port PCI Express Gen 3 (8 GT/s) Switch, 27 x 27mm FCBGA (rev ca) (prog-if 00 [Normal decode])
	Flags: bus master, fast devsel, latency 0, IRQ 477, NUMA node 1
	Bus: primary=02, secondary=06, subordinate=0a, sec-latency=0
	I/O behind bridge: None
	Memory behind bridge: 81800000-81ffffff [size=8M]
	Prefetchable memory behind bridge: 0000250200000000-00002502ffffffff [size=4G]
	Capabilities: [40] Power Management version 3
	Capabilities: [48] MSI: Enable- Count=1/8 Maskable+ 64bit+
	Capabilities: [68] Express Downstream Port (Slot+), MSI 00
	Capabilities: [a4] Subsystem: PLX Technology, Inc. PEX 8748 48-Lane, 12-Port PCI Express Gen 3 (8 GT/s) Switch, 27 x 27mm FCBGA
	Capabilities: [100] Device Serial Number ca-87-00-10-b5-df-0e-00
	Capabilities: [fb4] Advanced Error Reporting
	Capabilities: [138] Power Budgeting <?>
	Capabilities: [10c] #19
	Capabilities: [148] Virtual Channel
	Capabilities: [e00] #12
	Capabilities: [f24] Access Control Services
	Capabilities: [b70] Vendor Specific Information: ID=0001 Rev=0 Len=010 <?>
	Kernel modules: shpchp

0005:02:11.0 PCI bridge: PLX Technology, Inc. PEX 8748 48-Lane, 12-Port PCI Express Gen 3 (8 GT/s) Switch, 27 x 27mm FCBGA (rev ca) (prog-if 00 [Normal decode])
	Flags: bus master, fast devsel, latency 0, IRQ 478, NUMA node 1
	Bus: primary=02, secondary=0b, subordinate=0f, sec-latency=0
	I/O behind bridge: None
	Memory behind bridge: 82000000-827fffff [size=8M]
	Prefetchable memory behind bridge: 0000250300000000-00002503ffffffff [size=4G]
	Capabilities: [40] Power Management version 3
	Capabilities: [48] MSI: Enable- Count=1/8 Maskable+ 64bit+
	Capabilities: [68] Express Downstream Port (Slot+), MSI 00
	Capabilities: [a4] Subsystem: PLX Technology, Inc. PEX 8748 48-Lane, 12-Port PCI Express Gen 3 (8 GT/s) Switch, 27 x 27mm FCBGA
	Capabilities: [100] Device Serial Number ca-87-00-10-b5-df-0e-00
	Capabilities: [fb4] Advanced Error Reporting
	Capabilities: [138] Power Budgeting <?>
	Capabilities: [10c] #19
	Capabilities: [148] Virtual Channel
	Capabilities: [e00] #12
	Capabilities: [f24] Access Control Services
	Capabilities: [b70] Vendor Specific Information: ID=0001 Rev=0 Len=010 <?>
	Kernel modules: shpchp

0005:03:00.0 USB controller: Texas Instruments TUSB73x0 SuperSpeed USB 3.0 xHCI Host Controller (rev 02) (prog-if 30 [XHCI])
	Subsystem: IBM S824 (8286-42A)
	Flags: bus master, fast devsel, latency 0, IRQ 478, NUMA node 1
	Memory at 3fe280000000 (64-bit, non-prefetchable) [size=64K]
	Memory at 3fe280010000 (64-bit, non-prefetchable) [size=8K]
	Capabilities: [40] Power Management version 3
	Capabilities: [48] MSI: Enable- Count=1/8 Maskable- 64bit+
	Capabilities: [70] Express Endpoint, MSI 00
	Capabilities: [c0] MSI-X: Enable+ Count=8 Masked-
	Capabilities: [100] Advanced Error Reporting
	Capabilities: [150] Device Serial Number 08-00-28-00-00-20-00-00
	Kernel driver in use: xhci_hcd

0005:04:00.0 RAID bus controller: IBM PCI-E IPR SAS Adapter (ASIC) (rev 02)
	Subsystem: IBM PCIe3 x8 Cache SAS RAID Internal Adapter 6Gb (57D8)
	Flags: bus master, fast devsel, latency 0, NUMA node 1
	Memory at 3fe280820000 (64-bit, non-prefetchable) [size=64K]
	Memory at 3fe280830000 (64-bit, non-prefetchable) [size=64K]
	[virtual] Expansion ROM at 3fe280800000 [disabled] [size=128K]
	Capabilities: [40] Power Management version 3
	Capabilities: [48] MSI: Enable- Count=1/1 Maskable+ 64bit+
	Capabilities: [60] Express Endpoint, MSI 00
	Capabilities: [a4] MSI-X: Enable+ Count=16 Masked-
	Capabilities: [100] Advanced Error Reporting
	Capabilities: [188] Alternative Routing-ID Interpretation (ARI)
	Capabilities: [190] #19
	Kernel driver in use: ipr
	Kernel modules: ipr

0005:05:00.0 Ethernet controller: Broadcom Limited NetXtreme BCM5719 Gigabit Ethernet PCIe (rev 01)
	Subsystem: IBM FC 5260/5899 4-port 1 GbE Adapter for Power
	Flags: bus master, fast devsel, latency 0, IRQ 478, NUMA node 1
	Memory at 250100000000 (64-bit, prefetchable) [size=64K]
	Memory at 250100010000 (64-bit, prefetchable) [size=64K]
	Memory at 250100020000 (64-bit, prefetchable) [size=64K]
	[virtual] Expansion ROM at 3fe281000000 [disabled] [size=512K]
	Capabilities: [48] Power Management version 3
	Capabilities: [50] Vital Product Data
	Capabilities: [58] MSI: Enable- Count=1/8 Maskable- 64bit+
	Capabilities: [a0] MSI-X: Enable+ Count=17 Masked-
	Capabilities: [ac] Express Endpoint, MSI 00
	Capabilities: [100] Advanced Error Reporting
	Capabilities: [13c] Device Serial Number 00-00-40-f2-e9-5a-e6-5c
	Capabilities: [150] Power Budgeting <?>
	Capabilities: [160] Virtual Channel
	Capabilities: [230] Transaction Processing Hints
	Kernel driver in use: tg3
	Kernel modules: tg3

0005:05:00.1 Ethernet controller: Broadcom Limited NetXtreme BCM5719 Gigabit Ethernet PCIe (rev 01)
	Subsystem: IBM FC 5260/5899 4-port 1 GbE Adapter for Power
	Flags: bus master, fast devsel, latency 0, IRQ 479, NUMA node 1
	Memory at 250100030000 (64-bit, prefetchable) [size=64K]
	Memory at 250100040000 (64-bit, prefetchable) [size=64K]
	Memory at 250100050000 (64-bit, prefetchable) [size=64K]
	[virtual] Expansion ROM at 3fe281080000 [disabled] [size=512K]
	Capabilities: [48] Power Management version 3
	Capabilities: [50] Vital Product Data
	Capabilities: [58] MSI: Enable- Count=1/8 Maskable- 64bit+
	Capabilities: [a0] MSI-X: Enable+ Count=17 Masked-
	Capabilities: [ac] Express Endpoint, MSI 00
	Capabilities: [100] Advanced Error Reporting
	Capabilities: [13c] Device Serial Number 00-00-40-f2-e9-5a-e6-5d
	Capabilities: [150] Power Budgeting <?>
	Capabilities: [160] Virtual Channel
	Capabilities: [230] Transaction Processing Hints
	Kernel driver in use: tg3
	Kernel modules: tg3

0005:05:00.2 Ethernet controller: Broadcom Limited NetXtreme BCM5719 Gigabit Ethernet PCIe (rev 01)
	Subsystem: IBM FC 5260/5899 4-port 1 GbE Adapter for Power
	Flags: bus master, fast devsel, latency 0, IRQ 478, NUMA node 1
	Memory at 250100060000 (64-bit, prefetchable) [size=64K]
	Memory at 250100070000 (64-bit, prefetchable) [size=64K]
	Memory at 250100080000 (64-bit, prefetchable) [size=64K]
	[virtual] Expansion ROM at 3fe281100000 [disabled] [size=512K]
	Capabilities: [48] Power Management version 3
	Capabilities: [50] Vital Product Data
	Capabilities: [58] MSI: Enable- Count=1/8 Maskable- 64bit+
	Capabilities: [a0] MSI-X: Enable+ Count=17 Masked-
	Capabilities: [ac] Express Endpoint, MSI 00
	Capabilities: [100] Advanced Error Reporting
	Capabilities: [13c] Device Serial Number 00-00-40-f2-e9-5a-e6-5e
	Capabilities: [150] Power Budgeting <?>
	Capabilities: [160] Virtual Channel
	Capabilities: [230] Transaction Processing Hints
	Kernel driver in use: tg3
	Kernel modules: tg3

0005:05:00.3 Ethernet controller: Broadcom Limited NetXtreme BCM5719 Gigabit Ethernet PCIe (rev 01)
	Subsystem: IBM FC 5260/5899 4-port 1 GbE Adapter for Power
	Flags: bus master, fast devsel, latency 0, IRQ 479, NUMA node 1
	Memory at 250100090000 (64-bit, prefetchable) [size=64K]
	Memory at 2501000a0000 (64-bit, prefetchable) [size=64K]
	Memory at 2501000b0000 (64-bit, prefetchable) [size=64K]
	[virtual] Expansion ROM at 3fe281180000 [disabled] [size=512K]
	Capabilities: [48] Power Management version 3
	Capabilities: [50] Vital Product Data
	Capabilities: [58] MSI: Enable- Count=1/8 Maskable- 64bit+
	Capabilities: [a0] MSI-X: Enable+ Count=17 Masked-
	Capabilities: [ac] Express Endpoint, MSI 00
	Capabilities: [100] Advanced Error Reporting
	Capabilities: [13c] Device Serial Number 00-00-40-f2-e9-5a-e6-5f
	Capabilities: [150] Power Budgeting <?>
	Capabilities: [160] Virtual Channel
	Capabilities: [230] Transaction Processing Hints
	Kernel driver in use: tg3
	Kernel modules: tg3

0040:00:00.0 PCI bridge: IBM POWER8 Host Bridge (PHB3) (prog-if 00 [Normal decode])
	Flags: bus master, fast devsel, latency 0, NUMA node 16
	Bus: primary=00, secondary=01, subordinate=01, sec-latency=0
	I/O behind bridge: None
	Memory behind bridge: None
	Prefetchable memory behind bridge: None
	Capabilities: [40] Power Management version 3
	Capabilities: [48] Express Root Port (Slot-), MSI 00
	Capabilities: [100] Advanced Error Reporting
	Capabilities: [148] #19
	Kernel modules: shpchp

0044:00:00.0 PCI bridge: IBM POWER8 Host Bridge (PHB3) (prog-if 00 [Normal decode])
	Flags: bus master, fast devsel, latency 0, NUMA node 17
	Bus: primary=00, secondary=01, subordinate=01, sec-latency=0
	I/O behind bridge: None
	Memory behind bridge: None
	Prefetchable memory behind bridge: None
	Capabilities: [40] Power Management version 3
	Capabilities: [48] Express Root Port (Slot-), MSI 00
	Capabilities: [100] Advanced Error Reporting
	Capabilities: [148] #19
	Kernel modules: shpchp

0045:00:00.0 PCI bridge: IBM POWER8 Host Bridge (PHB3) (prog-if 00 [Normal decode])
	Flags: bus master, fast devsel, latency 0, NUMA node 17
	Bus: primary=00, secondary=01, subordinate=01, sec-latency=0
	I/O behind bridge: None
	Memory behind bridge: 80000000-ffefffff [size=2047M]
	Prefetchable memory behind bridge: None
	Capabilities: [40] Power Management version 3
	Capabilities: [48] Express Root Port (Slot-), MSI 00
	Capabilities: [100] Advanced Error Reporting
	Capabilities: [148] #19
	Kernel modules: shpchp

0045:01:00.0 USB controller: Texas Instruments TUSB73x0 SuperSpeed USB 3.0 xHCI Host Controller (rev 02) (prog-if 30 [XHCI])
	Subsystem: IBM Device 049e
	Flags: bus master, fast devsel, latency 0, IRQ 221, NUMA node 17
	Memory at 3ff280000000 (64-bit, non-prefetchable) [size=64K]
	Memory at 3ff280010000 (64-bit, non-prefetchable) [size=8K]
	Capabilities: [40] Power Management version 3
	Capabilities: [48] MSI: Enable- Count=1/8 Maskable- 64bit+
	Capabilities: [70] Express Endpoint, MSI 00
	Capabilities: [c0] MSI-X: Enable+ Count=8 Masked-
	Capabilities: [100] Advanced Error Reporting
	Capabilities: [150] Device Serial Number 08-00-28-00-00-20-00-00
	Kernel driver in use: xhci_hcd


[-- Attachment #4: lspci-v-caicos.txt --]
[-- Type: text/plain, Size: 23324 bytes --]

0000:00:00.0 PCI bridge: IBM POWER8 Host Bridge (PHB3) (prog-if 00 [Normal decode])
	Flags: bus master, fast devsel, latency 0, NUMA node 0
	Bus: primary=00, secondary=01, subordinate=01, sec-latency=0
	I/O behind bridge: None
	Memory behind bridge: None
	Prefetchable memory behind bridge: None
	Capabilities: [40] Power Management version 3
	Capabilities: [48] Express Root Port (Slot-), MSI 00
	Capabilities: [100] Advanced Error Reporting
	Capabilities: [148] #19
	Kernel modules: shpchp

0001:00:00.0 PCI bridge: IBM POWER8 Host Bridge (PHB3) (prog-if 00 [Normal decode])
	Flags: bus master, fast devsel, latency 0, NUMA node 0
	Bus: primary=00, secondary=01, subordinate=0d, sec-latency=0
	I/O behind bridge: None
	Memory behind bridge: 80000000-ffefffff [size=2047M]
	Prefetchable memory behind bridge: 0000210000000000-000021fdffffffff [size=1016G]
	Capabilities: [40] Power Management version 3
	Capabilities: [48] Express Root Port (Slot-), MSI 00
	Capabilities: [100] Advanced Error Reporting
	Capabilities: [148] #19
	Kernel modules: shpchp

0001:01:00.0 PCI bridge: PLX Technology, Inc. PEX 8732 32-lane, 8-Port PCI Express Gen 3 (8.0 GT/s) Switch (rev ca) (prog-if 00 [Normal decode])
	Flags: bus master, fast devsel, latency 0, IRQ 493, NUMA node 0
	Memory at 3fe082800000 (32-bit, non-prefetchable) [size=256K]
	Bus: primary=01, secondary=02, subordinate=0d, sec-latency=0
	I/O behind bridge: None
	Memory behind bridge: 80000000-ffefffff [size=2047M]
	Prefetchable memory behind bridge: 0000210000000000-000021fdffffffff [size=1016G]
	Capabilities: [40] Power Management version 3
	Capabilities: [48] MSI: Enable- Count=1/8 Maskable+ 64bit+
	Capabilities: [68] Express Upstream Port, MSI 00
	Capabilities: [a4] Subsystem: PLX Technology, Inc. PEX 8732 32-lane, 8-Port PCI Express Gen 3 (8.0 GT/s) Switch
	Capabilities: [100] Device Serial Number ca-87-00-10-07-08-28-15
	Capabilities: [fb4] Advanced Error Reporting
	Capabilities: [138] Power Budgeting <?>
	Capabilities: [10c] #19
	Capabilities: [148] Virtual Channel
	Capabilities: [e00] #12
	Capabilities: [b00] Latency Tolerance Reporting
	Capabilities: [b70] Vendor Specific Information: ID=0001 Rev=0 Len=010 <?>
	Kernel modules: shpchp

0001:02:01.0 PCI bridge: PLX Technology, Inc. PEX 8732 32-lane, 8-Port PCI Express Gen 3 (8.0 GT/s) Switch (rev ca) (prog-if 00 [Normal decode])
	Flags: bus master, fast devsel, latency 0, IRQ 494, NUMA node 0
	Bus: primary=02, secondary=03, subordinate=07, sec-latency=0
	I/O behind bridge: None
	Memory behind bridge: 80000000-807fffff [size=8M]
	Prefetchable memory behind bridge: 0000210000000000-00002100ffffffff [size=4G]
	Capabilities: [40] Power Management version 3
	Capabilities: [48] MSI: Enable- Count=1/8 Maskable+ 64bit+
	Capabilities: [68] Express Downstream Port (Slot+), MSI 00
	Capabilities: [a4] Subsystem: PLX Technology, Inc. PEX 8732 32-lane, 8-Port PCI Express Gen 3 (8.0 GT/s) Switch
	Capabilities: [100] Device Serial Number ca-87-00-10-b5-df-0e-00
	Capabilities: [fb4] Advanced Error Reporting
	Capabilities: [138] Power Budgeting <?>
	Capabilities: [10c] #19
	Capabilities: [148] Virtual Channel
	Capabilities: [e00] #12
	Capabilities: [f24] Access Control Services
	Capabilities: [b70] Vendor Specific Information: ID=0001 Rev=0 Len=010 <?>
	Kernel modules: shpchp

0001:02:08.0 PCI bridge: PLX Technology, Inc. PEX 8732 32-lane, 8-Port PCI Express Gen 3 (8.0 GT/s) Switch (rev ca) (prog-if 00 [Normal decode])
	Flags: bus master, fast devsel, latency 0, IRQ 493, NUMA node 0
	Bus: primary=02, secondary=08, subordinate=08, sec-latency=0
	I/O behind bridge: None
	Memory behind bridge: 80800000-80ffffff [size=8M]
	Prefetchable memory behind bridge: 0000210100000000-00002101ffffffff [size=4G]
	Capabilities: [40] Power Management version 3
	Capabilities: [48] MSI: Enable- Count=1/8 Maskable+ 64bit+
	Capabilities: [68] Express Downstream Port (Slot+), MSI 00
	Capabilities: [a4] Subsystem: PLX Technology, Inc. PEX 8732 32-lane, 8-Port PCI Express Gen 3 (8.0 GT/s) Switch
	Capabilities: [100] Device Serial Number ca-87-00-10-b5-df-0e-00
	Capabilities: [fb4] Advanced Error Reporting
	Capabilities: [138] Power Budgeting <?>
	Capabilities: [10c] #19
	Capabilities: [148] Virtual Channel
	Capabilities: [e00] #12
	Capabilities: [f24] Access Control Services
	Capabilities: [b70] Vendor Specific Information: ID=0001 Rev=0 Len=010 <?>
	Kernel modules: shpchp

0001:02:09.0 PCI bridge: PLX Technology, Inc. PEX 8732 32-lane, 8-Port PCI Express Gen 3 (8.0 GT/s) Switch (rev ca) (prog-if 00 [Normal decode])
	Flags: bus master, fast devsel, latency 0, IRQ 494, NUMA node 0
	Bus: primary=02, secondary=09, subordinate=0d, sec-latency=0
	I/O behind bridge: None
	Memory behind bridge: 81000000-817fffff [size=8M]
	Prefetchable memory behind bridge: 0000210200000000-00002102ffffffff [size=4G]
	Capabilities: [40] Power Management version 3
	Capabilities: [48] MSI: Enable- Count=1/8 Maskable+ 64bit+
	Capabilities: [68] Express Downstream Port (Slot+), MSI 00
	Capabilities: [a4] Subsystem: PLX Technology, Inc. PEX 8732 32-lane, 8-Port PCI Express Gen 3 (8.0 GT/s) Switch
	Capabilities: [100] Device Serial Number ca-87-00-10-b5-df-0e-00
	Capabilities: [fb4] Advanced Error Reporting
	Capabilities: [138] Power Budgeting <?>
	Capabilities: [10c] #19
	Capabilities: [148] Virtual Channel
	Capabilities: [e00] #12
	Capabilities: [f24] Access Control Services
	Capabilities: [b70] Vendor Specific Information: ID=0001 Rev=0 Len=010 <?>
	Kernel modules: shpchp

0001:08:00.0 RAID bus controller: IBM PCI-E IPR SAS Adapter (ASIC) (rev 02)
	Subsystem: IBM PCIe3 x8 Cache SAS RAID Internal Adapter 6Gb (57D8)
	Flags: bus master, fast devsel, latency 0, NUMA node 0
	Memory at 3fe080820000 (64-bit, non-prefetchable) [size=64K]
	Memory at 3fe080830000 (64-bit, non-prefetchable) [size=64K]
	[virtual] Expansion ROM at 3fe080800000 [disabled] [size=128K]
	Capabilities: [40] Power Management version 3
	Capabilities: [48] MSI: Enable- Count=1/1 Maskable+ 64bit+
	Capabilities: [60] Express Endpoint, MSI 00
	Capabilities: [a4] MSI-X: Enable+ Count=16 Masked-
	Capabilities: [100] Advanced Error Reporting
	Capabilities: [188] Alternative Routing-ID Interpretation (ARI)
	Capabilities: [190] #19
	Kernel driver in use: ipr
	Kernel modules: ipr

0004:00:00.0 PCI bridge: IBM POWER8 Host Bridge (PHB3) (prog-if 00 [Normal decode])
	Flags: bus master, fast devsel, latency 0, NUMA node 1
	Bus: primary=00, secondary=01, subordinate=01, sec-latency=0
	I/O behind bridge: None
	Memory behind bridge: 80000000-ffefffff [size=2047M]
	Prefetchable memory behind bridge: 0000240000000000-000024fdffffffff [size=1016G]
	Capabilities: [40] Power Management version 3
	Capabilities: [48] Express Root Port (Slot-), MSI 00
	Capabilities: [100] Advanced Error Reporting
	Capabilities: [148] #19
	Kernel modules: shpchp

0004:01:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Caicos [Radeon HD 6450/7450/8450 / R5 230 OEM] (prog-if 00 [VGA controller])
	Subsystem: Advanced Micro Devices, Inc. [AMD/ATI] Device 2120
	Flags: bus master, fast devsel, latency 0, IRQ 398, NUMA node 1
	Memory at 240000000000 (64-bit, prefetchable) [size=256M]
	Memory at 3fe200000000 (64-bit, non-prefetchable) [size=128K]
	I/O ports at <unassigned> [disabled]
	Expansion ROM at 3fe200020000 [disabled] [size=128K]
	Capabilities: [50] Power Management version 3
	Capabilities: [58] Express Legacy Endpoint, MSI 00
	Capabilities: [a0] MSI: Enable+ Count=1/1 Maskable- 64bit+
	Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
	Capabilities: [150] Advanced Error Reporting
	Kernel driver in use: radeon
	Kernel modules: radeon

0004:01:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Caicos HDMI Audio [Radeon HD 6450 / 7450/8450/8490 OEM / R5 230/235/235X OEM]
	Subsystem: Advanced Micro Devices, Inc. [AMD/ATI] Caicos HDMI Audio [Radeon HD 6450 / 7450/8450/8490 OEM / R5 230/235/235X OEM]
	Flags: fast devsel, IRQ 482, NUMA node 1
	Memory at 3fe200040000 (64-bit, non-prefetchable) [disabled] [size=16K]
	Capabilities: [50] Power Management version 3
	Capabilities: [58] Express Legacy Endpoint, MSI 00
	Capabilities: [a0] MSI: Enable- Count=1/1 Maskable- 64bit+
	Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
	Capabilities: [150] Advanced Error Reporting

0005:00:00.0 PCI bridge: IBM POWER8 Host Bridge (PHB3) (prog-if 00 [Normal decode])
	Flags: bus master, fast devsel, latency 0, NUMA node 1
	Bus: primary=00, secondary=01, subordinate=0f, sec-latency=0
	I/O behind bridge: None
	Memory behind bridge: 80000000-ffefffff [size=2047M]
	Prefetchable memory behind bridge: 0000250000000000-000025fdffffffff [size=1016G]
	Capabilities: [40] Power Management version 3
	Capabilities: [48] Express Root Port (Slot-), MSI 00
	Capabilities: [100] Advanced Error Reporting
	Capabilities: [148] #19
	Kernel modules: shpchp

0005:01:00.0 PCI bridge: PLX Technology, Inc. PEX 8748 48-Lane, 12-Port PCI Express Gen 3 (8 GT/s) Switch, 27 x 27mm FCBGA (rev ca) (prog-if 00 [Normal decode])
	Flags: bus master, fast devsel, latency 0, IRQ 477, NUMA node 1
	Memory at 3fe283800000 (32-bit, non-prefetchable) [size=256K]
	Bus: primary=01, secondary=02, subordinate=0f, sec-latency=0
	I/O behind bridge: None
	Memory behind bridge: 80000000-ffefffff [size=2047M]
	Prefetchable memory behind bridge: 0000250000000000-000025fdffffffff [size=1016G]
	Capabilities: [40] Power Management version 3
	Capabilities: [48] MSI: Enable- Count=1/8 Maskable+ 64bit+
	Capabilities: [68] Express Upstream Port, MSI 00
	Capabilities: [a4] Subsystem: PLX Technology, Inc. PEX 8748 48-Lane, 12-Port PCI Express Gen 3 (8 GT/s) Switch, 27 x 27mm FCBGA
	Capabilities: [100] Device Serial Number ca-87-00-10-07-08-28-15
	Capabilities: [fb4] Advanced Error Reporting
	Capabilities: [138] Power Budgeting <?>
	Capabilities: [10c] #19
	Capabilities: [148] Virtual Channel
	Capabilities: [e00] #12
	Capabilities: [b00] Latency Tolerance Reporting
	Capabilities: [b70] Vendor Specific Information: ID=0001 Rev=0 Len=010 <?>
	Kernel modules: shpchp

0005:02:01.0 PCI bridge: PLX Technology, Inc. PEX 8748 48-Lane, 12-Port PCI Express Gen 3 (8 GT/s) Switch, 27 x 27mm FCBGA (rev ca) (prog-if 00 [Normal decode])
	Flags: bus master, fast devsel, latency 0, IRQ 478, NUMA node 1
	Bus: primary=02, secondary=03, subordinate=03, sec-latency=0
	I/O behind bridge: None
	Memory behind bridge: 80000000-807fffff [size=8M]
	Prefetchable memory behind bridge: None
	Capabilities: [40] Power Management version 3
	Capabilities: [48] MSI: Enable- Count=1/8 Maskable+ 64bit+
	Capabilities: [68] Express Downstream Port (Slot-), MSI 00
	Capabilities: [a4] Subsystem: PLX Technology, Inc. PEX 8748 48-Lane, 12-Port PCI Express Gen 3 (8 GT/s) Switch, 27 x 27mm FCBGA
	Capabilities: [100] Device Serial Number ca-87-00-10-b5-df-0e-00
	Capabilities: [fb4] Advanced Error Reporting
	Capabilities: [138] Power Budgeting <?>
	Capabilities: [10c] #19
	Capabilities: [148] Virtual Channel
	Capabilities: [e00] #12
	Capabilities: [f24] Access Control Services
	Capabilities: [b70] Vendor Specific Information: ID=0001 Rev=0 Len=010 <?>
	Kernel modules: shpchp

0005:02:08.0 PCI bridge: PLX Technology, Inc. PEX 8748 48-Lane, 12-Port PCI Express Gen 3 (8 GT/s) Switch, 27 x 27mm FCBGA (rev ca) (prog-if 00 [Normal decode])
	Flags: bus master, fast devsel, latency 0, IRQ 477, NUMA node 1
	Bus: primary=02, secondary=04, subordinate=04, sec-latency=0
	I/O behind bridge: None
	Memory behind bridge: 80800000-80ffffff [size=8M]
	Prefetchable memory behind bridge: 0000250000000000-00002500ffffffff [size=4G]
	Capabilities: [40] Power Management version 3
	Capabilities: [48] MSI: Enable- Count=1/8 Maskable+ 64bit+
	Capabilities: [68] Express Downstream Port (Slot+), MSI 00
	Capabilities: [a4] Subsystem: PLX Technology, Inc. PEX 8748 48-Lane, 12-Port PCI Express Gen 3 (8 GT/s) Switch, 27 x 27mm FCBGA
	Capabilities: [100] Device Serial Number ca-87-00-10-b5-df-0e-00
	Capabilities: [fb4] Advanced Error Reporting
	Capabilities: [138] Power Budgeting <?>
	Capabilities: [10c] #19
	Capabilities: [148] Virtual Channel
	Capabilities: [e00] #12
	Capabilities: [f24] Access Control Services
	Capabilities: [b70] Vendor Specific Information: ID=0001 Rev=0 Len=010 <?>
	Kernel modules: shpchp

0005:02:09.0 PCI bridge: PLX Technology, Inc. PEX 8748 48-Lane, 12-Port PCI Express Gen 3 (8 GT/s) Switch, 27 x 27mm FCBGA (rev ca) (prog-if 00 [Normal decode])
	Flags: bus master, fast devsel, latency 0, IRQ 478, NUMA node 1
	Bus: primary=02, secondary=05, subordinate=05, sec-latency=0
	I/O behind bridge: None
	Memory behind bridge: 81000000-817fffff [size=8M]
	Prefetchable memory behind bridge: 0000250100000000-00002501ffffffff [size=4G]
	Capabilities: [40] Power Management version 3
	Capabilities: [48] MSI: Enable- Count=1/8 Maskable+ 64bit+
	Capabilities: [68] Express Downstream Port (Slot+), MSI 00
	Capabilities: [a4] Subsystem: PLX Technology, Inc. PEX 8748 48-Lane, 12-Port PCI Express Gen 3 (8 GT/s) Switch, 27 x 27mm FCBGA
	Capabilities: [100] Device Serial Number ca-87-00-10-b5-df-0e-00
	Capabilities: [fb4] Advanced Error Reporting
	Capabilities: [138] Power Budgeting <?>
	Capabilities: [10c] #19
	Capabilities: [148] Virtual Channel
	Capabilities: [e00] #12
	Capabilities: [f24] Access Control Services
	Capabilities: [b70] Vendor Specific Information: ID=0001 Rev=0 Len=010 <?>
	Kernel modules: shpchp

0005:02:10.0 PCI bridge: PLX Technology, Inc. PEX 8748 48-Lane, 12-Port PCI Express Gen 3 (8 GT/s) Switch, 27 x 27mm FCBGA (rev ca) (prog-if 00 [Normal decode])
	Flags: bus master, fast devsel, latency 0, IRQ 477, NUMA node 1
	Bus: primary=02, secondary=06, subordinate=0a, sec-latency=0
	I/O behind bridge: None
	Memory behind bridge: 81800000-81ffffff [size=8M]
	Prefetchable memory behind bridge: 0000250200000000-00002502ffffffff [size=4G]
	Capabilities: [40] Power Management version 3
	Capabilities: [48] MSI: Enable- Count=1/8 Maskable+ 64bit+
	Capabilities: [68] Express Downstream Port (Slot+), MSI 00
	Capabilities: [a4] Subsystem: PLX Technology, Inc. PEX 8748 48-Lane, 12-Port PCI Express Gen 3 (8 GT/s) Switch, 27 x 27mm FCBGA
	Capabilities: [100] Device Serial Number ca-87-00-10-b5-df-0e-00
	Capabilities: [fb4] Advanced Error Reporting
	Capabilities: [138] Power Budgeting <?>
	Capabilities: [10c] #19
	Capabilities: [148] Virtual Channel
	Capabilities: [e00] #12
	Capabilities: [f24] Access Control Services
	Capabilities: [b70] Vendor Specific Information: ID=0001 Rev=0 Len=010 <?>
	Kernel modules: shpchp

0005:02:11.0 PCI bridge: PLX Technology, Inc. PEX 8748 48-Lane, 12-Port PCI Express Gen 3 (8 GT/s) Switch, 27 x 27mm FCBGA (rev ca) (prog-if 00 [Normal decode])
	Flags: bus master, fast devsel, latency 0, IRQ 478, NUMA node 1
	Bus: primary=02, secondary=0b, subordinate=0f, sec-latency=0
	I/O behind bridge: None
	Memory behind bridge: 82000000-827fffff [size=8M]
	Prefetchable memory behind bridge: 0000250300000000-00002503ffffffff [size=4G]
	Capabilities: [40] Power Management version 3
	Capabilities: [48] MSI: Enable- Count=1/8 Maskable+ 64bit+
	Capabilities: [68] Express Downstream Port (Slot+), MSI 00
	Capabilities: [a4] Subsystem: PLX Technology, Inc. PEX 8748 48-Lane, 12-Port PCI Express Gen 3 (8 GT/s) Switch, 27 x 27mm FCBGA
	Capabilities: [100] Device Serial Number ca-87-00-10-b5-df-0e-00
	Capabilities: [fb4] Advanced Error Reporting
	Capabilities: [138] Power Budgeting <?>
	Capabilities: [10c] #19
	Capabilities: [148] Virtual Channel
	Capabilities: [e00] #12
	Capabilities: [f24] Access Control Services
	Capabilities: [b70] Vendor Specific Information: ID=0001 Rev=0 Len=010 <?>
	Kernel modules: shpchp

0005:03:00.0 USB controller: Texas Instruments TUSB73x0 SuperSpeed USB 3.0 xHCI Host Controller (rev 02) (prog-if 30 [XHCI])
	Subsystem: IBM S824 (8286-42A)
	Flags: bus master, fast devsel, latency 0, IRQ 478, NUMA node 1
	Memory at 3fe280000000 (64-bit, non-prefetchable) [size=64K]
	Memory at 3fe280010000 (64-bit, non-prefetchable) [size=8K]
	Capabilities: [40] Power Management version 3
	Capabilities: [48] MSI: Enable- Count=1/8 Maskable- 64bit+
	Capabilities: [70] Express Endpoint, MSI 00
	Capabilities: [c0] MSI-X: Enable+ Count=8 Masked-
	Capabilities: [100] Advanced Error Reporting
	Capabilities: [150] Device Serial Number 08-00-28-00-00-20-00-00
	Kernel driver in use: xhci_hcd

0005:04:00.0 RAID bus controller: IBM PCI-E IPR SAS Adapter (ASIC) (rev 02)
	Subsystem: IBM PCIe3 x8 Cache SAS RAID Internal Adapter 6Gb (57D8)
	Flags: bus master, fast devsel, latency 0, NUMA node 1
	Memory at 3fe280820000 (64-bit, non-prefetchable) [size=64K]
	Memory at 3fe280830000 (64-bit, non-prefetchable) [size=64K]
	[virtual] Expansion ROM at 3fe280800000 [disabled] [size=128K]
	Capabilities: [40] Power Management version 3
	Capabilities: [48] MSI: Enable- Count=1/1 Maskable+ 64bit+
	Capabilities: [60] Express Endpoint, MSI 00
	Capabilities: [a4] MSI-X: Enable+ Count=16 Masked-
	Capabilities: [100] Advanced Error Reporting
	Capabilities: [188] Alternative Routing-ID Interpretation (ARI)
	Capabilities: [190] #19
	Kernel driver in use: ipr
	Kernel modules: ipr

0005:05:00.0 Ethernet controller: Broadcom Limited NetXtreme BCM5719 Gigabit Ethernet PCIe (rev 01)
	Subsystem: IBM FC 5260/5899 4-port 1 GbE Adapter for Power
	Flags: bus master, fast devsel, latency 0, IRQ 478, NUMA node 1
	Memory at 250100000000 (64-bit, prefetchable) [size=64K]
	Memory at 250100010000 (64-bit, prefetchable) [size=64K]
	Memory at 250100020000 (64-bit, prefetchable) [size=64K]
	[virtual] Expansion ROM at 3fe281000000 [disabled] [size=512K]
	Capabilities: [48] Power Management version 3
	Capabilities: [50] Vital Product Data
	Capabilities: [58] MSI: Enable- Count=1/8 Maskable- 64bit+
	Capabilities: [a0] MSI-X: Enable+ Count=17 Masked-
	Capabilities: [ac] Express Endpoint, MSI 00
	Capabilities: [100] Advanced Error Reporting
	Capabilities: [13c] Device Serial Number 00-00-40-f2-e9-5a-e6-5c
	Capabilities: [150] Power Budgeting <?>
	Capabilities: [160] Virtual Channel
	Capabilities: [230] Transaction Processing Hints
	Kernel driver in use: tg3
	Kernel modules: tg3

0005:05:00.1 Ethernet controller: Broadcom Limited NetXtreme BCM5719 Gigabit Ethernet PCIe (rev 01)
	Subsystem: IBM FC 5260/5899 4-port 1 GbE Adapter for Power
	Flags: bus master, fast devsel, latency 0, IRQ 479, NUMA node 1
	Memory at 250100030000 (64-bit, prefetchable) [size=64K]
	Memory at 250100040000 (64-bit, prefetchable) [size=64K]
	Memory at 250100050000 (64-bit, prefetchable) [size=64K]
	[virtual] Expansion ROM at 3fe281080000 [disabled] [size=512K]
	Capabilities: [48] Power Management version 3
	Capabilities: [50] Vital Product Data
	Capabilities: [58] MSI: Enable- Count=1/8 Maskable- 64bit+
	Capabilities: [a0] MSI-X: Enable+ Count=17 Masked-
	Capabilities: [ac] Express Endpoint, MSI 00
	Capabilities: [100] Advanced Error Reporting
	Capabilities: [13c] Device Serial Number 00-00-40-f2-e9-5a-e6-5d
	Capabilities: [150] Power Budgeting <?>
	Capabilities: [160] Virtual Channel
	Capabilities: [230] Transaction Processing Hints
	Kernel driver in use: tg3
	Kernel modules: tg3

0005:05:00.2 Ethernet controller: Broadcom Limited NetXtreme BCM5719 Gigabit Ethernet PCIe (rev 01)
	Subsystem: IBM FC 5260/5899 4-port 1 GbE Adapter for Power
	Flags: bus master, fast devsel, latency 0, IRQ 478, NUMA node 1
	Memory at 250100060000 (64-bit, prefetchable) [size=64K]
	Memory at 250100070000 (64-bit, prefetchable) [size=64K]
	Memory at 250100080000 (64-bit, prefetchable) [size=64K]
	[virtual] Expansion ROM at 3fe281100000 [disabled] [size=512K]
	Capabilities: [48] Power Management version 3
	Capabilities: [50] Vital Product Data
	Capabilities: [58] MSI: Enable- Count=1/8 Maskable- 64bit+
	Capabilities: [a0] MSI-X: Enable+ Count=17 Masked-
	Capabilities: [ac] Express Endpoint, MSI 00
	Capabilities: [100] Advanced Error Reporting
	Capabilities: [13c] Device Serial Number 00-00-40-f2-e9-5a-e6-5e
	Capabilities: [150] Power Budgeting <?>
	Capabilities: [160] Virtual Channel
	Capabilities: [230] Transaction Processing Hints
	Kernel driver in use: tg3
	Kernel modules: tg3

0005:05:00.3 Ethernet controller: Broadcom Limited NetXtreme BCM5719 Gigabit Ethernet PCIe (rev 01)
	Subsystem: IBM FC 5260/5899 4-port 1 GbE Adapter for Power
	Flags: bus master, fast devsel, latency 0, IRQ 479, NUMA node 1
	Memory at 250100090000 (64-bit, prefetchable) [size=64K]
	Memory at 2501000a0000 (64-bit, prefetchable) [size=64K]
	Memory at 2501000b0000 (64-bit, prefetchable) [size=64K]
	[virtual] Expansion ROM at 3fe281180000 [disabled] [size=512K]
	Capabilities: [48] Power Management version 3
	Capabilities: [50] Vital Product Data
	Capabilities: [58] MSI: Enable- Count=1/8 Maskable- 64bit+
	Capabilities: [a0] MSI-X: Enable+ Count=17 Masked-
	Capabilities: [ac] Express Endpoint, MSI 00
	Capabilities: [100] Advanced Error Reporting
	Capabilities: [13c] Device Serial Number 00-00-40-f2-e9-5a-e6-5f
	Capabilities: [150] Power Budgeting <?>
	Capabilities: [160] Virtual Channel
	Capabilities: [230] Transaction Processing Hints
	Kernel driver in use: tg3
	Kernel modules: tg3

0040:00:00.0 PCI bridge: IBM POWER8 Host Bridge (PHB3) (prog-if 00 [Normal decode])
	Flags: bus master, fast devsel, latency 0, NUMA node 16
	Bus: primary=00, secondary=01, subordinate=01, sec-latency=0
	I/O behind bridge: None
	Memory behind bridge: None
	Prefetchable memory behind bridge: None
	Capabilities: [40] Power Management version 3
	Capabilities: [48] Express Root Port (Slot-), MSI 00
	Capabilities: [100] Advanced Error Reporting
	Capabilities: [148] #19
	Kernel modules: shpchp

0044:00:00.0 PCI bridge: IBM POWER8 Host Bridge (PHB3) (prog-if 00 [Normal decode])
	Flags: bus master, fast devsel, latency 0, NUMA node 17
	Bus: primary=00, secondary=01, subordinate=01, sec-latency=0
	I/O behind bridge: None
	Memory behind bridge: None
	Prefetchable memory behind bridge: None
	Capabilities: [40] Power Management version 3
	Capabilities: [48] Express Root Port (Slot-), MSI 00
	Capabilities: [100] Advanced Error Reporting
	Capabilities: [148] #19
	Kernel modules: shpchp

0045:00:00.0 PCI bridge: IBM POWER8 Host Bridge (PHB3) (prog-if 00 [Normal decode])
	Flags: bus master, fast devsel, latency 0, NUMA node 17
	Bus: primary=00, secondary=01, subordinate=01, sec-latency=0
	I/O behind bridge: None
	Memory behind bridge: 80000000-ffefffff [size=2047M]
	Prefetchable memory behind bridge: None
	Capabilities: [40] Power Management version 3
	Capabilities: [48] Express Root Port (Slot-), MSI 00
	Capabilities: [100] Advanced Error Reporting
	Capabilities: [148] #19
	Kernel modules: shpchp

0045:01:00.0 USB controller: Texas Instruments TUSB73x0 SuperSpeed USB 3.0 xHCI Host Controller (rev 02) (prog-if 30 [XHCI])
	Subsystem: IBM Device 049e
	Flags: bus master, fast devsel, latency 0, IRQ 221, NUMA node 17
	Memory at 3ff280000000 (64-bit, non-prefetchable) [size=64K]
	Memory at 3ff280010000 (64-bit, non-prefetchable) [size=8K]
	Capabilities: [40] Power Management version 3
	Capabilities: [48] MSI: Enable- Count=1/8 Maskable- 64bit+
	Capabilities: [70] Express Endpoint, MSI 00
	Capabilities: [c0] MSI-X: Enable+ Count=8 Masked-
	Capabilities: [100] Advanced Error Reporting
	Capabilities: [150] Device Serial Number 08-00-28-00-00-20-00-00
	Kernel driver in use: xhci_hcd


[-- Attachment #5: Type: text/plain, Size: 160 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [PATCH] drm/radeon: insist on 32-bit DMA for Cedar on PPC64/PPC64LE
       [not found]           ` <1bcf5b27-aa25-deb6-85e9-161dd038276b-otUistvHUpPR7s880joybQ@public.gmane.org>
@ 2018-02-23  7:45             ` Christian König
  2018-02-23 16:02               ` Ben Crocker
  0 siblings, 1 reply; 13+ messages in thread
From: Christian König @ 2018-02-23  7:45 UTC (permalink / raw)
  To: Michel Dänzer, Ben Crocker
  Cc: Maling list - DRI developers, amd-gfx list

Am 22.02.2018 um 18:56 schrieb Michel Dänzer:
> On 2018-02-22 06:37 PM, Ben Crocker wrote:
>> One of my colleagues did discover a "Radeon HG 5450 PCI" from February
>> 2010 which did, apparently, have a Cedar GPU and very definitely had a
>> (plain old) PCI connector.
> There must be a PCIe-to-PCI bridge on that board. The GPU itself is
> always PCIe, and treated accordingly by the driver.

Ben, just an educated guess but is this one the one which is failing to 
work correctly?

Cause the PCIe bus interface is pretty much identical over all 
generations of the last decade or so. Only the newest Vega10 generation 
is a bit different.

So I strongly thing that this isn't related to the device being a Cedar 
at all, but rather that you have a bridge above it which doesn't 
correctly handle 64bit transfers.

Can you please send and "lspci -t" of both the working and the 
problematic devices?

Thanks,
Christian.
_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [PATCH] drm/radeon: insist on 32-bit DMA for Cedar on PPC64/PPC64LE
       [not found]                 ` <CALk+eReghDHuv23-62pD0PbdBt6p+g_XeE+_c2gRdkoaOQh+CQ-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
@ 2018-02-23  1:39                   ` Ben Crocker
  0 siblings, 0 replies; 13+ messages in thread
From: Ben Crocker @ 2018-02-23  1:39 UTC (permalink / raw)
  To: Michel Dänzer, Alex Deucher, Christian Koenig, amd-gfx list
  Cc: Ben Crocker


[-- Attachment #1.1: Type: text/plain, Size: 1420 bytes --]

That's what IBM is most interested in at this juncture:
FirePro 2270
Embedded Radeon E6465

The E6465 already worked without this patch.


On Thu, Feb 22, 2018 at 6:39 PM, Ben Crocker <bcrocker-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> wrote:

> Alex asked what other GPUs I've tested on, and the answer is: E6465
> (Caicos GPU).
>
> -- Ben
>
>
> On Thu, Feb 22, 2018 at 1:33 PM, Ben Crocker <bcrocker-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> wrote:
>
>> I think that last version I sent a few minutes ago is final form, now.
>> Thanks for all your help!
>>
>> If there's anything you could do to expedite posting this patch to the
>> main repo, that would be GREATLY appreciated!
>>
>> -- Thanks,
>>   Ben
>>
>>
>> On Thu, Feb 22, 2018 at 12:56 PM, Michel Dänzer <michel-otUistvHUpPR7s880joybQ@public.gmane.org>
>> wrote:
>>
>>> On 2018-02-22 06:37 PM, Ben Crocker wrote:
>>> > One of my colleagues did discover a "Radeon HG 5450 PCI" from February
>>> > 2010 which did, apparently, have a Cedar GPU and very definitely had a
>>> > (plain old) PCI connector.
>>>
>>> There must be a PCIe-to-PCI bridge on that board. The GPU itself is
>>> always PCIe, and treated accordingly by the driver.
>>>
>>>
>>> --
>>> Earthling Michel Dänzer               |               http://www.amd.com
>>> Libre software enthusiast             |             Mesa and X developer
>>>
>>
>>
>

[-- Attachment #1.2: Type: text/html, Size: 2981 bytes --]

[-- Attachment #2: Type: text/plain, Size: 154 bytes --]

_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [PATCH] drm/radeon: insist on 32-bit DMA for Cedar on PPC64/PPC64LE
       [not found]             ` <CALk+eRc+EOry_G6Ovy0Z-pvE7jQWy5gSx+1kpHA4q3A0oEZs1w-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
@ 2018-02-22 23:39               ` Ben Crocker
       [not found]                 ` <CALk+eReghDHuv23-62pD0PbdBt6p+g_XeE+_c2gRdkoaOQh+CQ-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
  0 siblings, 1 reply; 13+ messages in thread
From: Ben Crocker @ 2018-02-22 23:39 UTC (permalink / raw)
  To: Michel Dänzer, Alex Deucher, Christian Koenig, amd-gfx list
  Cc: Ben Crocker


[-- Attachment #1.1: Type: text/plain, Size: 1125 bytes --]

Alex asked what other GPUs I've tested on, and the answer is: E6465 (Caicos
GPU).

-- Ben


On Thu, Feb 22, 2018 at 1:33 PM, Ben Crocker <bcrocker-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> wrote:

> I think that last version I sent a few minutes ago is final form, now.
> Thanks for all your help!
>
> If there's anything you could do to expedite posting this patch to the
> main repo, that would be GREATLY appreciated!
>
> -- Thanks,
>   Ben
>
>
> On Thu, Feb 22, 2018 at 12:56 PM, Michel Dänzer <michel-otUistvHUpPR7s880joybQ@public.gmane.org>
> wrote:
>
>> On 2018-02-22 06:37 PM, Ben Crocker wrote:
>> > One of my colleagues did discover a "Radeon HG 5450 PCI" from February
>> > 2010 which did, apparently, have a Cedar GPU and very definitely had a
>> > (plain old) PCI connector.
>>
>> There must be a PCIe-to-PCI bridge on that board. The GPU itself is
>> always PCIe, and treated accordingly by the driver.
>>
>>
>> --
>> Earthling Michel Dänzer               |               http://www.amd.com
>> Libre software enthusiast             |             Mesa and X developer
>>
>
>

[-- Attachment #1.2: Type: text/html, Size: 2059 bytes --]

[-- Attachment #2: Type: text/plain, Size: 154 bytes --]

_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

^ permalink raw reply	[flat|nested] 13+ messages in thread

* [PATCH] drm/radeon: insist on 32-bit DMA for Cedar on PPC64/PPC64LE
  2018-02-21 23:41 [PATCH] drm/radeon: insist on 32-bit DMA for Cedar Ben Crocker
@ 2018-02-22 21:57 ` Ben Crocker
  2018-02-22 17:04   ` Michel Dänzer
  0 siblings, 1 reply; 13+ messages in thread
From: Ben Crocker @ 2018-02-22 21:57 UTC (permalink / raw)
  To: dri-devel, amd-gfx; +Cc: Ben Crocker

In radeon_device_init, set the need_dma32 flag for Cedar chips
(e.g. FirePro 2270).  This fixes, or at least works around, a bug
on PowerPC exposed by last year's commits

8e3f1b1d8255105f31556aacf8aeb6071b00d469 (Russell Currey)

and

253fd51e2f533552ae35a0c661705da6c4842c1b (Alistair Popple)

which enabled the 64-bit DMA iommu bypass.

This caused the device to freeze, in some cases unrecoverably, and is
the subject of several bug reports internal to Red Hat.

Signed-off-by: Ben Crocker <bcrocker@redhat.com>
---
 drivers/gpu/drm/radeon/radeon_device.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/drivers/gpu/drm/radeon/radeon_device.c b/drivers/gpu/drm/radeon/radeon_device.c
index ffc10cadcf34..ba7435bcd208 100644
--- a/drivers/gpu/drm/radeon/radeon_device.c
+++ b/drivers/gpu/drm/radeon/radeon_device.c
@@ -1397,6 +1397,11 @@ int radeon_device_init(struct radeon_device *rdev,
 	if ((rdev->flags & RADEON_IS_PCI) &&
 	    (rdev->family <= CHIP_RS740))
 		rdev->need_dma32 = true;
+#ifdef CONFIG_PPC64
+	if ((rdev->flags & (RADEON_IS_PCI | RADEON_IS_PCIE)) &&
+	    (rdev->family == CHIP_CEDAR))
+		rdev->need_dma32 = true;
+#endif
 
 	dma_bits = rdev->need_dma32 ? 32 : 40;
 	r = pci_set_dma_mask(rdev->pdev, DMA_BIT_MASK(dma_bits));
-- 
2.13.6

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply related	[flat|nested] 13+ messages in thread

* Re: [PATCH] drm/radeon: insist on 32-bit DMA for Cedar on PPC64/PPC64LE
       [not found]       ` <CALk+eRew=1nHZXRtXR430eFStOiQFUDgrp8VfsBAfcp8W9qAOA-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
@ 2018-02-22 17:56         ` Michel Dänzer
       [not found]           ` <CALk+eRc+EOry_G6Ovy0Z-pvE7jQWy5gSx+1kpHA4q3A0oEZs1w@mail.gmail.com>
       [not found]           ` <1bcf5b27-aa25-deb6-85e9-161dd038276b-otUistvHUpPR7s880joybQ@public.gmane.org>
  0 siblings, 2 replies; 13+ messages in thread
From: Michel Dänzer @ 2018-02-22 17:56 UTC (permalink / raw)
  To: Ben Crocker; +Cc: amd-gfx list, Maling list - DRI developers

On 2018-02-22 06:37 PM, Ben Crocker wrote:
> One of my colleagues did discover a "Radeon HG 5450 PCI" from February
> 2010 which did, apparently, have a Cedar GPU and very definitely had a
> (plain old) PCI connector.

There must be a PCIe-to-PCI bridge on that board. The GPU itself is
always PCIe, and treated accordingly by the driver.


-- 
Earthling Michel Dänzer               |               http://www.amd.com
Libre software enthusiast             |             Mesa and X developer
_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [PATCH] drm/radeon: insist on 32-bit DMA for Cedar on PPC64/PPC64LE
  2018-02-22 17:04   ` Michel Dänzer
@ 2018-02-22 17:37     ` Ben Crocker
       [not found]       ` <CALk+eRew=1nHZXRtXR430eFStOiQFUDgrp8VfsBAfcp8W9qAOA-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
  0 siblings, 1 reply; 13+ messages in thread
From: Ben Crocker @ 2018-02-22 17:37 UTC (permalink / raw)
  To: Michel Dänzer; +Cc: Maling list - DRI developers, amd-gfx list


[-- Attachment #1.1: Type: text/plain, Size: 2063 bytes --]

One of my colleagues did discover a "Radeon HG 5450 PCI" from February
2010 which did, apparently, have a Cedar GPU and very definitely had a
(plain old) PCI connector.

But I take your point, and will just drop the PCI/PCIE check.


On Thu, Feb 22, 2018 at 12:04 PM, Michel Dänzer <michel@daenzer.net> wrote:

> On 2018-02-22 10:57 PM, Ben Crocker wrote:
> > In radeon_device_init, set the need_dma32 flag for Cedar chips
> > (e.g. FirePro 2270).  This fixes, or at least works around, a bug
> > on PowerPC exposed by last year's commits
> >
> > 8e3f1b1d8255105f31556aacf8aeb6071b00d469 (Russell Currey)
> >
> > and
> >
> > 253fd51e2f533552ae35a0c661705da6c4842c1b (Alistair Popple)
> >
> > which enabled the 64-bit DMA iommu bypass.
> >
> > This caused the device to freeze, in some cases unrecoverably, and is
> > the subject of several bug reports internal to Red Hat.
> >
> > Signed-off-by: Ben Crocker <bcrocker@redhat.com>
> > ---
> >  drivers/gpu/drm/radeon/radeon_device.c | 5 +++++
> >  1 file changed, 5 insertions(+)
> >
> > diff --git a/drivers/gpu/drm/radeon/radeon_device.c
> b/drivers/gpu/drm/radeon/radeon_device.c
> > index ffc10cadcf34..ba7435bcd208 100644
> > --- a/drivers/gpu/drm/radeon/radeon_device.c
> > +++ b/drivers/gpu/drm/radeon/radeon_device.c
> > @@ -1397,6 +1397,11 @@ int radeon_device_init(struct radeon_device *rdev,
> >       if ((rdev->flags & RADEON_IS_PCI) &&
> >           (rdev->family <= CHIP_RS740))
> >               rdev->need_dma32 = true;
> > +#ifdef CONFIG_PPC64
> > +     if ((rdev->flags & (RADEON_IS_PCI | RADEON_IS_PCIE)) &&
>
> Cedar is (like all Radeons from the last decade) always PCIe, so this
> flags check is redundant FWIW.
>
>
> --
> Earthling Michel Dänzer               |               http://www.amd.com
> Libre software enthusiast             |             Mesa and X developer
> _______________________________________________
> amd-gfx mailing list
> amd-gfx@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/amd-gfx
>

[-- Attachment #1.2: Type: text/html, Size: 3084 bytes --]

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [PATCH] drm/radeon: insist on 32-bit DMA for Cedar on PPC64/PPC64LE
  2018-02-22 21:57 ` [PATCH] drm/radeon: insist on 32-bit DMA for Cedar on PPC64/PPC64LE Ben Crocker
@ 2018-02-22 17:04   ` Michel Dänzer
  2018-02-22 17:37     ` Ben Crocker
  0 siblings, 1 reply; 13+ messages in thread
From: Michel Dänzer @ 2018-02-22 17:04 UTC (permalink / raw)
  To: Ben Crocker; +Cc: amd-gfx, dri-devel

On 2018-02-22 10:57 PM, Ben Crocker wrote:
> In radeon_device_init, set the need_dma32 flag for Cedar chips
> (e.g. FirePro 2270).  This fixes, or at least works around, a bug
> on PowerPC exposed by last year's commits
> 
> 8e3f1b1d8255105f31556aacf8aeb6071b00d469 (Russell Currey)
> 
> and
> 
> 253fd51e2f533552ae35a0c661705da6c4842c1b (Alistair Popple)
> 
> which enabled the 64-bit DMA iommu bypass.
> 
> This caused the device to freeze, in some cases unrecoverably, and is
> the subject of several bug reports internal to Red Hat.
> 
> Signed-off-by: Ben Crocker <bcrocker@redhat.com>
> ---
>  drivers/gpu/drm/radeon/radeon_device.c | 5 +++++
>  1 file changed, 5 insertions(+)
> 
> diff --git a/drivers/gpu/drm/radeon/radeon_device.c b/drivers/gpu/drm/radeon/radeon_device.c
> index ffc10cadcf34..ba7435bcd208 100644
> --- a/drivers/gpu/drm/radeon/radeon_device.c
> +++ b/drivers/gpu/drm/radeon/radeon_device.c
> @@ -1397,6 +1397,11 @@ int radeon_device_init(struct radeon_device *rdev,
>  	if ((rdev->flags & RADEON_IS_PCI) &&
>  	    (rdev->family <= CHIP_RS740))
>  		rdev->need_dma32 = true;
> +#ifdef CONFIG_PPC64
> +	if ((rdev->flags & (RADEON_IS_PCI | RADEON_IS_PCIE)) &&

Cedar is (like all Radeons from the last decade) always PCIe, so this
flags check is redundant FWIW.


-- 
Earthling Michel Dänzer               |               http://www.amd.com
Libre software enthusiast             |             Mesa and X developer
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 13+ messages in thread

end of thread, other threads:[~2018-02-23 16:37 UTC | newest]

Thread overview: 13+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
     [not found] <to=<20180221234154.6877-1-bcrocker@redhat.com>
2018-02-22 21:51 ` [PATCH] drm/radeon: insist on 32-bit DMA for Cedar on PPC64/PPC64LE Ben Crocker
     [not found]   ` <20180222215101.3962-1-bcrocker-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
2018-02-22 16:56     ` Alex Deucher
2018-02-22 22:52     ` Ben Crocker
2018-02-23  4:38       ` Alex Deucher
2018-02-21 23:41 [PATCH] drm/radeon: insist on 32-bit DMA for Cedar Ben Crocker
2018-02-22 21:57 ` [PATCH] drm/radeon: insist on 32-bit DMA for Cedar on PPC64/PPC64LE Ben Crocker
2018-02-22 17:04   ` Michel Dänzer
2018-02-22 17:37     ` Ben Crocker
     [not found]       ` <CALk+eRew=1nHZXRtXR430eFStOiQFUDgrp8VfsBAfcp8W9qAOA-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2018-02-22 17:56         ` Michel Dänzer
     [not found]           ` <CALk+eRc+EOry_G6Ovy0Z-pvE7jQWy5gSx+1kpHA4q3A0oEZs1w@mail.gmail.com>
     [not found]             ` <CALk+eRc+EOry_G6Ovy0Z-pvE7jQWy5gSx+1kpHA4q3A0oEZs1w-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2018-02-22 23:39               ` Ben Crocker
     [not found]                 ` <CALk+eReghDHuv23-62pD0PbdBt6p+g_XeE+_c2gRdkoaOQh+CQ-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2018-02-23  1:39                   ` Ben Crocker
     [not found]           ` <1bcf5b27-aa25-deb6-85e9-161dd038276b-otUistvHUpPR7s880joybQ@public.gmane.org>
2018-02-23  7:45             ` Christian König
2018-02-23 16:02               ` Ben Crocker
2018-02-23 16:37                 ` Christian König

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.