From: Chris Packham <Chris.Packham@alliedtelesis.co.nz>
To: Thomas Petazzoni <thomas.petazzoni@bootlin.com>,
Jason Cooper <jason@lakedaemon.net>,
Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>,
"Bjorn Helgaas" <bhelgaas@google.com>,
"linux-pci@vger.kernel.org" <linux-pci@vger.kernel.org>,
"linux-arm-kernel@lists.infradead.org"
<linux-arm-kernel@lists.infradead.org>,
"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
Subject: Kirkwood PCI Express and bridges
Date: Fri, 21 Jun 2019 04:03:27 +0000 [thread overview]
Message-ID: <403548ec3a7543b08ca32e47a1465e70@svr-chch-ex1.atlnz.lc> (raw)
Hi All,
I'm in the process of updating the kernel version used on our products
from 4.4 -> 5.1.
We have one product that uses a Kirkwood CPU, IDT PCI bridge and Marvell
Switch ASIC. The Switch ASIC presents as multiple PCI devices.
The hardware setup looks like this
__________
[ Kirkwood ] --- [ IDT 5T5 ] ---+--- | |
+--- | Switch |
+--- | |
+--- |__________|
On the 4.4 based kernel things are fine
[root@awplus flash]# lspci -t
-[0000:00]---01.0-[01-06]----00.0-[02-06]--+-02.0-[03]----00.0
+-03.0-[04]----00.0
+-04.0-[05]----00.0
\-05.0-[06]----00.0
But on the 5.1 based kernel things get a little weird
[root@awplus flash]# lspci -t
-[0000:00]---01.0-[01-06]--+-00.0-[02-06]--
+-01.0
+-02.0-[02-06]--
+-03.0-[02-06]--
+-04.0-[02-06]--
+-05.0-[02-06]--
+-06.0-[02-06]--
+-07.0-[02-06]--
+-08.0-[02-06]--
+-09.0-[02-06]--
+-0a.0-[02-06]--
+-0b.0-[02-06]--
+-0c.0-[02-06]--
+-0d.0-[02-06]--
+-0e.0-[02-06]--
+-0f.0-[02-06]--
+-10.0-[02-06]--
+-11.0-[02-06]--
+-12.0-[02-06]--
+-13.0-[02-06]--
+-14.0-[02-06]--
+-15.0-[02-06]--
+-16.0-[02-06]--
+-17.0-[02-06]--
+-18.0-[02-06]--
+-19.0-[02-06]--
+-1a.0-[02-06]--
+-1b.0-[02-06]--
+-1c.0-[02-06]--
+-1d.0-[02-06]--
+-1e.0-[02-06]--
\-1f.0-[02-06]--+-02.0-[03]----00.0
+-03.0-[04]----00.0
+-04.0-[05]----00.0
\-05.0-[06]----00.0
I'll start bisecting to see where things started going wrong. I just
wondered if this rings any bells for anyone.
The startup output also seems to be quite unhappy
Detected board: alliedtelesis,SBx81GC40
Booting into Linux kernel ...
** 143 printk messages dropped **
pci 0000:01:19.0: PME# supported from D0 D3hot D3cold
pci 0000:01:1a.0: [111d:803c] type 01 class 0x060400
pci 0000:01:1a.0: PME# supported from D0 D3hot D3cold
pci 0000:01:1b.0: [111d:803c] type 01 class 0x060400
pci 0000:01:1b.0: PME# supported from D0 D3hot D3cold
pci 0000:01:1c.0: [111d:803c] type 01 class 0x060400
pci 0000:01:1c.0: PME# supported from D0 D3hot D3cold
pci 0000:01:1d.0: [111d:803c] type 01 class 0x060400
pci 0000:01:1d.0: PME# supported from D0 D3hot D3cold
pci 0000:01:1e.0: [111d:803c] type 01 class 0x060400
pci 0000:01:1e.0: PME# supported from D0 D3hot D3cold
pci 0000:01:1f.0: [111d:803c] type 01 class 0x060400
pci 0000:01:1f.0: PME# supported from D0 D3hot D3cold
PCI: bus1: Fast back to back transfers disabled
pci 0000:01:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring
pci 0000:01:02.0: bridge configuration invalid ([bus 00-00]), reconfiguring
pci 0000:01:03.0: bridge configuration invalid ([bus 00-00]), reconfiguring
pci 0000:01:04.0: bridge configuration invalid ([bus 00-00]), reconfiguring
pci 0000:01:05.0: bridge configuration invalid ([bus 00-00]), reconfiguring
pci 0000:01:06.0: bridge configuration invalid ([bus 00-00]), reconfiguring
pci 0000:01:07.0: bridge configuration invalid ([bus 00-00]), reconfiguring
pci 0000:01:08.0: bridge configuration invalid ([bus 00-00]), reconfiguring
pci 0000:01:09.0: bridge configuration invalid ([bus 00-00]), reconfiguring
pci 0000:01:0a.0: bridge configuration invalid ([bus 00-00]), reconfiguring
pci 0000:01:0b.0: bridge configuration invalid ([bus 00-00]), reconfiguring
pci 0000:01:0c.0: bridge configuration invalid ([bus 00-00]), reconfiguring
pci 0000:01:0d.0: bridge configuration invalid ([bus 00-00]), reconfiguring
pci 0000:01:0e.0: bridge configuration invalid ([bus 00-00]), reconfiguring
pci 0000:01:0f.0: bridge configuration invalid ([bus 00-00]), reconfiguring
pci 0000:01:10.0: bridge configuration invalid ([bus 00-00]), reconfiguring
pci 0000:01:11.0: bridge configuration invalid ([bus 00-00]), reconfiguring
pci 0000:01:12.0: bridge configuration invalid ([bus 00-00]), reconfiguring
pci 0000:01:13.0: bridge configuration invalid ([bus 00-00]), reconfiguring
pci 0000:01:14.0: bridge configuration invalid ([bus 00-00]), reconfiguring
pci 0000:01:15.0: bridge configuration invalid ([bus 00-00]), reconfiguring
pci 0000:01:16.0: bridge configuration invalid ([bus 00-00]), reconfiguring
pci 0000:01:17.0: bridge configuration invalid ([bus 00-00]), reconfiguring
pci 0000:01:18.0: bridge configuration invalid ([bus 00-00]), reconfiguring
pci 0000:01:19.0: bridge configuration invalid ([bus 00-00]), reconfiguring
pci 0000:01:1a.0: bridge configuration invalid ([bus 00-00]), reconfiguring
pci 0000:01:1b.0: bridge configuration invalid ([bus 00-00]), reconfiguring
pci 0000:01:1c.0: bridge configuration invalid ([bus 00-00]), reconfiguring
pci 0000:01:1d.0: bridge configuration invalid ([bus 00-00]), reconfiguring
pci 0000:01:1e.0: bridge configuration invalid ([bus 00-00]), reconfiguring
pci 0000:01:1f.0: bridge configuration invalid ([bus 00-00]), reconfiguring
pci 0000:02:02.0: [111d:803c] type 01 class 0x060400
pci 0000:02:02.0: enabling Extended Tags
pci 0000:02:02.0: PME# supported from D0 D3hot D3cold
pci 0000:02:03.0: [111d:803c] type 01 class 0x060400
pci 0000:02:03.0: enabling Extended Tags
pci 0000:02:03.0: PME# supported from D0 D3hot D3cold
pci 0000:02:04.0: [111d:803c] type 01 class 0x060400
pci 0000:02:04.0: enabling Extended Tags
pci 0000:02:04.0: PME# supported from D0 D3hot D3cold
pci 0000:02:05.0: [111d:803c] type 01 class 0x060400
pci 0000:02:05.0: enabling Extended Tags
pci 0000:02:05.0: PME# supported from D0 D3hot D3cold
PCI: bus2: Fast back to back transfers disabled
pci 0000:02:02.0: bridge configuration invalid ([bus 00-00]), reconfiguring
pci 0000:02:03.0: bridge configuration invalid ([bus 00-00]), reconfiguring
pci 0000:02:04.0: bridge configuration invalid ([bus 00-00]), reconfiguring
pci 0000:02:05.0: bridge configuration invalid ([bus 00-00]), reconfiguring
pci 0000:03:00.0: [11ab:e023] type 00 class 0x058000
pci 0000:03:00.0: reg 0x10: [mem 0xd0000000-0xd00fffff 64bit pref]
pci 0000:03:00.0: reg 0x18: [mem 0x00000000-0x03ffffff 64bit]
PCI: bus3: Fast back to back transfers disabled
pci_bus 0000:03: busn_res: [bus 03-ff] end is updated to 03
pci 0000:04:00.0: [11ab:e023] type 00 class 0x058000
pci 0000:04:00.0: reg 0x10: [mem 0xd0000000-0xd00fffff 64bit pref]
pci 0000:04:00.0: reg 0x18: [mem 0x00000000-0x03ffffff 64bit]
PCI: bus4: Fast back to back transfers disabled
pci_bus 0000:04: busn_res: [bus 04-ff] end is updated to 04
pci 0000:05:00.0: [11ab:e023] type 00 class 0x058000
pci 0000:05:00.0: reg 0x10: [mem 0xd0000000-0xd00fffff 64bit pref]
pci 0000:05:00.0: reg 0x18: [mem 0x00000000-0x03ffffff 64bit]
PCI: bus5: Fast back to back transfers disabled
pci_bus 0000:05: busn_res: [bus 05-ff] end is updated to 05
pci 0000:06:00.0: [11ab:e023] type 00 class 0x058000
pci 0000:06:00.0: reg 0x10: [mem 0xd0000000-0xd00fffff 64bit pref]
pci 0000:06:00.0: reg 0x18: [mem 0x00000000-0x03ffffff 64bit]
PCI: bus6: Fast back to back transfers disabled
pci_bus 0000:06: busn_res: [bus 06-ff] end is updated to 06
pci_bus 0000:02: busn_res: [bus 02-ff] end is updated to 06
pci_bus 0000:01: busn_res: [bus 01-ff] end is updated to 06
pci 0000:00:01.0: BAR 8: no space for [mem size 0x1c000000]
pci 0000:00:01.0: BAR 8: failed to assign [mem size 0x1c000000]
pci 0000:00:01.0: BAR 6: assigned [mem 0xe0000000-0xe00007ff pref]
pci 0000:01:01.0: BAR 2: no space for [mem size 0x08000000]
pci 0000:01:01.0: BAR 2: failed to assign [mem size 0x08000000]
pci 0000:01:00.0: BAR 8: no space for [mem size 0x10000000]
pci 0000:01:00.0: BAR 8: failed to assign [mem size 0x10000000]
pci 0000:01:00.0: BAR 9: no space for [mem size 0x00400000 64bit pref]
pci 0000:01:00.0: BAR 9: failed to assign [mem size 0x00400000 64bit pref]
pci 0000:01:01.0: BAR 0: no space for [mem size 0x00100000 64bit pref]
pci 0000:01:01.0: BAR 0: failed to assign [mem size 0x00100000 64bit pref]
pci 0000:02:02.0: BAR 8: no space for [mem size 0x04000000]
pci 0000:02:02.0: BAR 8: failed to assign [mem size 0x04000000]
pci 0000:02:03.0: BAR 8: no space for [mem size 0x04000000]
pci 0000:02:03.0: BAR 8: failed to assign [mem size 0x04000000]
pci 0000:02:04.0: BAR 8: no space for [mem size 0x04000000]
pci 0000:02:04.0: BAR 8: failed to assign [mem size 0x04000000]
pci 0000:02:05.0: BAR 8: no space for [mem size 0x04000000]
pci 0000:02:05.0: BAR 8: failed to assign [mem size 0x04000000]
pci 0000:02:02.0: BAR 9: no space for [mem size 0x00100000 64bit pref]
pci 0000:02:02.0: BAR 9: failed to assign [mem size 0x00100000 64bit pref]
pci 0000:02:03.0: BAR 9: no space for [mem size 0x00100000 64bit pref]
pci 0000:02:03.0: BAR 9: failed to assign [mem size 0x00100000 64bit pref]
pci 0000:02:04.0: BAR 9: no space for [mem size 0x00100000 64bit pref]
pci 0000:02:04.0: BAR 9: failed to assign [mem size 0x00100000 64bit pref]
pci 0000:02:05.0: BAR 9: no space for [mem size 0x00100000 64bit pref]
pci 0000:02:05.0: BAR 9: failed to assign [mem size 0x00100000 64bit pref]
pci 0000:03:00.0: BAR 2: no space for [mem size 0x04000000 64bit]
pci 0000:03:00.0: BAR 2: failed to assign [mem size 0x04000000 64bit]
pci 0000:03:00.0: BAR 0: no space for [mem size 0x00100000 64bit pref]
pci 0000:03:00.0: BAR 0: failed to assign [mem size 0x00100000 64bit pref]
pci 0000:02:02.0: PCI bridge to [bus 03]
pci 0000:04:00.0: BAR 2: no space for [mem size 0x04000000 64bit]
pci 0000:04:00.0: BAR 2: failed to assign [mem size 0x04000000 64bit]
pci 0000:04:00.0: BAR 0: no space for [mem size 0x00100000 64bit pref]
pci 0000:04:00.0: BAR 0: failed to assign [mem size 0x00100000 64bit pref]
pci 0000:02:03.0: PCI bridge to [bus 04]
pci 0000:05:00.0: BAR 2: no space for [mem size 0x04000000 64bit]
pci 0000:05:00.0: BAR 2: failed to assign [mem size 0x04000000 64bit]
pci 0000:05:00.0: BAR 0: no space for [mem size 0x00100000 64bit pref]
pci 0000:05:00.0: BAR 0: failed to assign [mem size 0x00100000 64bit pref]
pci 0000:02:04.0: PCI bridge to [bus 05]
pci 0000:06:00.0: BAR 2: no space for [mem size 0x04000000 64bit]
pci 0000:06:00.0: BAR 2: failed to assign [mem size 0x04000000 64bit]
pci 0000:06:00.0: BAR 0: no space for [mem size 0x00100000 64bit pref]
pci 0000:06:00.0: BAR 0: failed to assign [mem size 0x00100000 64bit pref]
pci 0000:02:05.0: PCI bridge to [bus 06]
pci 0000:01:00.0: PCI bridge to [bus 02-06]
pci 0000:00:01.0: PCI bridge to [bus 01-06]
next reply other threads:[~2019-06-21 4:03 UTC|newest]
Thread overview: 4+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-06-21 4:03 Chris Packham [this message]
2019-06-21 5:33 ` Kirkwood PCI Express and bridges Thomas Petazzoni
2019-06-24 4:08 ` Chris Packham
2019-06-25 2:05 ` Chris Packham
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=403548ec3a7543b08ca32e47a1465e70@svr-chch-ex1.atlnz.lc \
--to=chris.packham@alliedtelesis.co.nz \
--cc=bhelgaas@google.com \
--cc=jason@lakedaemon.net \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-pci@vger.kernel.org \
--cc=lorenzo.pieralisi@arm.com \
--cc=thomas.petazzoni@bootlin.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).