archive mirror
 help / color / mirror / Atom feed
From: Bjorn Helgaas <>
To: "Pali Rohár" <>
Cc: "Stefan Chulski" <>,
	"Bjorn Helgaas" <>,
	"Thomas Petazzoni" <>,
	"Marek Behún" <>, "Stefan Roese" <>,
	"Phil Sutter" <>, "Mario Six" <>,
	"Lorenzo Pieralisi" <>,
	"" <>
Subject: Re: [EXT] Re: pci mvebu issue (memory controller)
Date: Wed, 2 Jun 2021 14:14:55 -0500	[thread overview]
Message-ID: <20210602191455.GA2038253@bjorn-Precision-5520> (raw)
In-Reply-To: <20210602110703.ymdt6nxsjl7e6glk@pali>

On Wed, Jun 02, 2021 at 01:07:03PM +0200, Pali Rohár wrote:

> In configuration with *bad* suffix is used U-Boot which does not ignore
> PCIe device Memory controller and configure it when U-Boot initialize.
> In this configuration loaded kernel is unable to initialize wifi cards.
> In configuration with *ok* suffix is U-Boot explicitly patched to
> ignores PCIe device Memory controller and loaded kernel can use wifi
> cards without any issue.
> In both configurations is used same kernel version. As I wrote in
> previous emails kernel already ignores and hides Memory controller PCIe
> device, so lspci does not see it.
> In attachment I'm sending dmesg and lspci outputs from Linux and pci
> output from U-Boot.
> What is suspicious for me is that this Memory controller device is at
> the same bus as wifi card. PCIe is "point to point", so at the other end
> of link should be only one device... Therefore I'm not sure if kernel
> can handle such thing like "two PCIe devices" at other end of PCIe link.
> Could you look at attached logs if you see something suspicious here? Or
> if you need other logs (either from U-Boot or kernel) please let me
> know.
> Note that U-Boot does not see PCIe Bridge as it is emulated only by
> kernel. So U-Boot enumerates buses from zero and kernel from one (as
> kernel's bus zero is for emulated PCIe Bridges).

I've lost track of what the problem is or what patch we're evaluating.

Here's what I see from dmesg/lspci/uboot:

  # dmesg (both bad/ok) and lspci:
  00:01.0 [11ab:6820] Root Port to [bus 01]
  00:02.0 [11ab:6820] Root Port to [bus 02]
  00:03.0 [11ab:6820] Root Port to [bus 03]
  01:00.0 [168c:002e] Atheros AR9287 NIC
  02:00.0 [168c:0046] Atheros QCA9984 NIC
  03:00.0 [168c:003c] Atheros QCA986x/988x NIC

The above looks perfectly reasonable.

  # uboot (bad):
  00.00.00 [11ab:6820] memory controller
  00.01.00 [168c:002e] NIC
  01.00.00 [11ab:6820] memory controller
  01.01.00 [168c:0046] NIC
  02.00.00 [11ab:6820] memory controller
  02.01.00 [168c:003c] NIC

The above looks dubious at best.  Bus 00 clearly must be a root bus
because bus 00 can never be a bridge's secondary bus.

Either buses 01 and 02 need to also be root buses (e.g., if we had
three host bridges, one leading to bus 00, another to bus 01, and
another to bus 02), OR there must be Root Ports that act as bridges
leading from bus 00 to bus 01 and bus 02.  The "memory controllers"
are vendor/device ID [11ab:6820], which Linux thinks are Root Ports,
so I assume they are really Root Ports (or some emulation of them).

It's *possible* to have both a Root Port and a NIC on bus 0, as shown
here.  However, the NIC would have to be a Root Complex integrated
Endpoint, and this NIC ([168c:002e]) is not one of those.  It's a
garden-variety PCIe legacy endpoint connected by a link.  So this NIC
cannot actually be on bus 00.

All these NICs are PCIe legacy endpoints with links, so they all must
have a Root Port leading to them.  So this topology is not really

  # uboot (ok):
  00.00.00 [168c:002e] NIC
  01.00.00 [168c:0046] NIC
  02.00.00 [168c:003c] NIC

This topology is impossible from a PCI perspective because there's no
way to get from bus 00 to bus 01 or 02.

       reply	other threads:[~2021-06-02 19:14 UTC|newest]

Thread overview: 9+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
     [not found] <20210602110703.ymdt6nxsjl7e6glk@pali>
2021-06-02 19:14 ` Bjorn Helgaas [this message]
2021-06-02 20:39   ` [EXT] Re: pci mvebu issue (memory controller) Pali Rohár
2021-06-02 21:01     ` Bjorn Helgaas
2021-06-02 21:13       ` Pali Rohár
2021-06-02 21:59         ` Marek Behún
2021-02-09 13:17 Marek Behún
2021-02-10  8:54 ` Thomas Petazzoni
2021-02-10 13:59   ` [EXT] " Stefan Chulski
2021-02-19 17:44     ` Pali Rohár
2021-03-04 18:29       ` Bjorn Helgaas
2021-11-01 18:07         ` Jason Gunthorpe

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20210602191455.GA2038253@bjorn-Precision-5520 \ \ \ \ \ \ \ \ \ \ \ \

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).