Re: [PATCH v3 04/10] PCI/RCEC: Add pcie_walk_rcec() to walk associated RCiEPs

From: Bjorn Helgaas <helgaas@kernel.org>
To: Sean V Kelley <sean.v.kelley@intel.com>
Cc: "Zhuo, Qiuxu" <qiuxu.zhuo@intel.com>,
	Jonathan.Cameron@huawei.com, rjw@rjwysocki.net,
	sathyanarayanan.kuppuswamy@linux.intel.com, "Raj,
	Ashok" <ashok.raj@intel.com>, "Luck, Tony" <tony.luck@intel.com>,
	linux-pci@vger.kernel.org, bhelgaas@google.com,
	linux-kernel@vger.kernel.org
Subject: Re: [PATCH v3 04/10] PCI/RCEC: Add pcie_walk_rcec() to walk associated RCiEPs
Date: Wed, 16 Sep 2020 17:49:42 -0500	[thread overview]
Message-ID: <20200916224942.GA1594177@bjorn-Precision-5520> (raw)
In-Reply-To: <7B04CA9A-7332-4001-963B-E56642044F5D@intel.com>

On Mon, Sep 14, 2020 at 09:55:53AM -0700, Sean V Kelley wrote:
> On 11 Sep 2020, at 17:50, Bjorn Helgaas wrote:
> > On Fri, Sep 11, 2020 at 04:16:03PM -0700, Sean V Kelley wrote:

> > > I’ve done some experimenting with this approach, and I think
> > > there may be a problem of just walking the busses during
> > > enumeration pci_init_capabilities(). One problem is where one
> > > has an RCEC on a root bus: 6a(00.4) and an RCiEP on another root
> > > bus: 6b(00.0).  They will never find each other in this approach
> > > through a normal pci_bus_walk() call using their respective
> > > root_bus.
> > > 
> > > >  +-[0000:6b]-+-00.0
> > > >  |           +-00.1
> > > >  |           +-00.2
> > > >  |           \-00.3
> > > >  +-[0000:6a]-+-00.0
> > > >  |           +-00.1
> > > >  |           +-00.2
> > > >  |           \-00.4
> > 
> > Wow, is that even allowed?
> > 
> > There's no bridge from 0000:6a to 0000:6b, so we will not scan 0000:6b
> > unless we find a host bridge with _CRS where 6b is the first bus
> > number below the bridge.  I think that means this would have to be
> > described in ACPI as two separate root bridges:
> > 
> >   ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 6a])
> >   ACPI: PCI Root Bridge [PCI1] (domain 0000 [bus 6b])
> 
> Otherwise, the RCEC Associated Endpoint Extended Capabilities would have to
> have explicitly mentioned a bridge?

I just meant that the enumeration algorithm starts with a PNP0A03
device and searches the root bus from its _CRS, descending under any
bridges it finds.  There's no PCI-to-PCI bridge from 6a to 6b (if
there *were* such a bridge, 6b would not be a root bridge).

> > I *guess* maybe it's allowed by the PCIe spec to have an RCEC and
> > associated RCiEPs on separate root buses?  It seems awfully strange
> > and not in character for PCIe, but I guess I can't point to language
> > that prohibits it.
> 
> Yes, it should be possible.

Ugh :)