Re: [PATCH] arm64: dts: marvell: armada-37xx: Set linux,pci-domain to zero

[Rob Herring <robh+dt@kernel.org>]

On Thu, Apr 15, 2021 at 3:45 AM Marek Behun <marek.behun@nic.cz> wrote:
>
> On Thu, 15 Apr 2021 10:36:40 +0200
> Pali Rohár <pali@kernel.org> wrote:
>
> > On Tuesday 13 April 2021 13:17:29 Rob Herring wrote:
> > > On Mon, Apr 12, 2021 at 7:41 AM Pali Rohár <pali@kernel.org> wrote:
> > > >
> > > > Since commit 526a76991b7b ("PCI: aardvark: Implement driver 'remove'
> > > > function and allow to build it as module") PCIe controller driver for
> > > > Armada 37xx can be dynamically loaded and unloaded at runtime. Also driver
> > > > allows dynamic binding and unbinding of PCIe controller device.
> > > >
> > > > Kernel PCI subsystem assigns by default dynamically allocated PCI domain
> > > > number (starting from zero) for this PCIe controller every time when device
> > > > is bound. So PCI domain changes after every unbind / bind operation.
> > >
> > > PCI host bridges as a module are relatively new, so seems likely a bug to me.
> >
> > Why a bug? It is there since 5.10 and it is working.

I mean historically, the PCI subsystem didn't even support host
bridges as a module. They weren't even proper drivers and it was all
arch specific code. Most of the host bridge drivers are still built-in
only. This seems like a small detail that was easily overlooked.
unbind is not a well tested path.

> > > > Alternative way for assigning PCI domain number is to use static allocated
> > > > numbers defined in Device Tree. This option has requirement that every PCI
> > > > controller in system must have defined PCI bus number in Device Tree.
> > >
> > > That seems entirely pointless from a DT point of view with a single PCI bridge.
> >
> > If domain id is not specified in DT then kernel uses counter and assigns
> > counter++. So it is not pointless if we want to have stable domain id.
>
> What Rob is trying to say is that
> - the bug is that kernel assigns counter++
> - device-tree should not be used to fix problems with how kernel does
>   things
> - if a device has only one PCIe controller, it is pointless to define
>   it's pci-domain. If there were multiple controllers, then it would
>   make sense, but there is only one

Yes. I think what we want here is a domain bitmap rather than a
counter and we assign the lowest free bit. That could also allow for
handling a mixture of fixed domain numbers and dynamically assigned
ones.

You could create scenarios where the numbers change on you, but it
wouldn't be any different than say plugging in USB serial adapters.
You get the same ttyUSBx device when you re-attach unless there's been
other ttyUSBx devices attached/detached.

Rob