Re: [PATCH v2 19/27] pci: PCIe driver for Marvell Armada 370/XP systems

[Bjorn Helgaas <bhelgaas@google.com>]

On Thu, Jan 31, 2013 at 9:04 AM, Thomas Petazzoni
<thomas.petazzoni@free-electrons.com> wrote:
> Dear Bjorn Helgaas,
>
> On Wed, 30 Jan 2013 11:52:15 -0700, Bjorn Helgaas wrote:
>
>> The secondary/subordinate registers effectively define a bus number
>> aperture that tells the bridge which transactions to claim and forward
>> downstream.  When enumerating devices, we may update the subordinate
>> bus number to widen the aperture so we can enumerate an arbitrary tree
>> behind the bridge.  When we're finished, we'll probably narrow it by
>> updating the subordinate again, so the unused bus number space can be
>> used for other bridges.  I don't know the exact details of the
>> algorithm, and they're likely to change anyway, but pci_scan_bridge()
>> is where most of it happens.
>>
>> It looks like your current system doesn't support trees below the
>> bridges, but hopefully we can make it so the generic enumeration
>> algorithms still work.
>
> The PCI-to-PCI bridge specification says that the Primary Bus Number
> Register, Secondary Bus Number Register and Subordinate Bus Number
> Register of the PCI configuration space of a PCI-to-PCI bridge should
> all be set to 0 after reset.
>
> Until now, I was forcing a specific value of the Secondary Bus Number
> and Subordinate Bus Number (1 for my first bridge, 2 for my second
> bridge, etc.).
>
> Following you're recommendation, I've changed this, and left those
> values initialized to 0 by default, in order to let Linux set correct
> values. Yes, Linux does assign appropriate values in the Secondary Bus
> Number Register. But before that Linux also complains loudly that the
> bridge configuration is invalid:
>
> pci 0000:00:01.0: bridge configuration invalid ([bus 00-00]), reconfiguring
> pci 0000:00:02.0: bridge configuration invalid ([bus 00-00]), reconfiguring
> pci 0000:00:03.0: bridge configuration invalid ([bus 00-00]), reconfiguring
> pci 0000:00:04.0: bridge configuration invalid ([bus 00-00]), reconfiguring
> pci 0000:00:05.0: bridge configuration invalid ([bus 00-00]), reconfiguring
> pci 0000:00:06.0: bridge configuration invalid ([bus 00-00]), reconfiguring

Linux makes the unwarranted assumption that the PCI hierarchy has
already been configured by firmware.  If the only problem is the
messages above, I think we could just rework the message so it doesn't
look like an error.  I would guess that we probably also see the same
distressing message when we hot-add a card with a bridge on it,
because firmware won't have initialized the bridge.

My rule of thumb is that I like to note something in dmesg about the
initial configuration of bus/mem/io apertures and BARs, as well as
indications when we update them.  That way, the dmesg log should
contain enough information to debug most enumeration and configuration
defects.  pci_scan_bridge() is somewhat lacking in this regard.

Bjorn