RE: [PATCH] PCI: controller: dwc: Make PCI_IMX6 depend on PCIEPORTBUS

[Richard Zhu <hongxing.zhu@nxp.com>]

> -----Original Message-----
> From: Andrey Smirnov [mailto:andrew.smirnov@gmail.com]
> Sent: 2018年12月8日 7:57
> To: niklas.cassel@linaro.org
> Cc: Lucas Stach <l.stach@pengutronix.de>; linux-pci@vger.kernel.org; Bjorn
> Helgaas <bhelgaas@google.com>; Chris Healy <cphealy@gmail.com>;
> Leonard Crestez <leonard.crestez@nxp.com>; Aisheng Dong
> <aisheng.dong@nxp.com>; Richard Zhu <hongxing.zhu@nxp.com>;
> dl-linux-imx <linux-imx@nxp.com>; linux-arm-kernel
> <linux-arm-kernel@lists.infradead.org>; linux-kernel
> <linux-kernel@vger.kernel.org>
> Subject: Re: [PATCH] PCI: controller: dwc: Make PCI_IMX6 depend on
> PCIEPORTBUS
> 
> On Fri, Dec 7, 2018 at 5:11 AM Niklas Cassel <niklas.cassel@linaro.org>
> wrote:
> >
> > On Thu, Dec 06, 2018 at 08:55:13PM -0800, Andrey Smirnov wrote:
> > > On Thu, Dec 6, 2018 at 2:28 AM Lucas Stach <l.stach@pengutronix.de>
> wrote:
> > > >
> > > > Am Mittwoch, den 05.12.2018, 23:45 -0800 schrieb Andrey Smirnov:
> > > > > Building a kernel with CONFIG_PCI_IMX6=y, but
> > > > > CONFIG_PCIEPORTBUS=n produces a system where built-in PCIE
> > > > > bridge (16c3:abcd) isn't bound to pcieport driver. This, in
> > > > > turn, results in a PCIE bus that is capable of enumerating
> > > > > attached PCIE device, but lacks functional interrupt support.
> > > >
> > > > This is odd. AFAIK PCI port services are a totally optional thing
> > > > and them being absent should not lead to a non-functional PCI bus.
> > > > So I would really like to see some deeper analysis what is going on here.
> > > >
> > >
> > > AFAICT, this is due to pcieport driver enabling MSI of the bridge
> > > device (16c3:abcd) via pcie_port_device_register() ->
> > > pcie_init_service_irqs() -> pcie_port_enable_irq_vec() -> etc.
> > >
> > > I did an experiment on a i.MX8MQ/PCIE -> i210 setup I have: I
> > > disabled CONFIG_PCIEPORTBUS and hacked igb_main.c enough to make
> the
> > > i210 driver believe it should fall back onto legacy interrupts. Even
> > > without pcieport present in the system, i210 worked as expected via
> > > legacy interrupts, which seems to collaborate my conjecture above.
> > >
> > > Thanks,
> > > Andrey Smirnov
> >
> > IIUC PCIEPORTBUS should not be needed for MSIs to work, it is only
> > needed if you want e.g. PME or AER.
> >
> > The difference is that if PCIEPORTBUS is enabled, a MSI irq vector
> > will be allocated for the Root Complex itself, so that it can send an
> > irq when e.g. AER has detected an error.
> >
> >
> > If we disregard that MSI handling is currently broken on DWC PCIe:
> >
> https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmar
> >
> c.info%2F%3Fl%3Dlinux-pci%26m%3D154214986924244%26w%3D2&amp;d
> ata=02%7C
> >
> 01%7Chongxing.zhu%40nxp.com%7Cd0e75c24b7504c3f395508d65c9fc1a3%
> 7C686ea
> >
> 1d3bc2b4c6fa92cd99c5c301635%7C0%7C1%7C636798238552985486&amp;
> sdata=rwd
> > edF0SijZqkFrpSHFq7uLmXeftGag0pJyT9XC4NOc%3D&amp;reserved=0
> > It is very possible to have MSIs on dragonboard 820c, which also uses
> > the DWC PCIe controller, without having PCIEPORTBUS selected:
> >
> > # zcat /proc/config.gz | grep -E "PCIE_QCOM|PCIEPORTBUS"
> > # CONFIG_PCIEPORTBUS is not set
> > CONFIG_PCIE_QCOM=y
> >
> >
> > # lspci -v -s 0000:00:00.0
> > 0000:00:00.0 PCI bridge: Qualcomm Device 0104 (prog-if 00 [Normal
> decode])
> >         ...
> >         Capabilities: [50] MSI: Enable- Count=1/32 Maskable+ 64bit+
> >
> > # lspci -v -s 0000:01:00.0
> > 0000:01:00.0 Network controller: Qualcomm Atheros QCA6174 802.11ac
> Wireless Network Adapter (rev 32)
> >         ...
> >         Capabilities: [50] MSI: Enable+ Count=1/8 Maskable+ 64bit-
> >
> >
> > # cat /proc/interrupts | grep MSI
> >  70:       5620          0          0          0   PCI-MSI
> 524288 Edge      ath10k_pci
> >
> > So perhaps this is a bug specific to imx6?
> >
> 
> Yeah, that seems entirely plausible. I reached out to NXP via one of the
> support channels to clarify. I'll report if I hear back from them.
> 
[Richard Zhu] Did some tests.
The MSI_EN of the MSI CAP of RC should be set to 1 on iMX6 PCIe.
Otherwise, the MSI wouldn't be triggered although the device is present and the MSIs are assigned.

> Thanks,
> Andrey Smirnov