Re: [PATCH] PCI: dwc: skip MSI init if MSIs have been explicitly disabled

[Tim Harvey <tharvey@gateworks.com>]

On Mon, Mar 4, 2019 at 12:18 PM Marc Zyngier <marc.zyngier@arm.com> wrote:
>
> Hi both,
>
> On Mon, 04 Mar 2019 19:39:45 +0000,
> Lucas Stach <l.stach@pengutronix.de> wrote:
> >
> > Hi Bjorn,
> >
> > Am Montag, den 04.03.2019, 13:25 -0600 schrieb Bjorn Helgaas:
> > > [+cc Marc, in case you have any suggestions]
> > >
> > > Hi Lucas,
> > >
> > > Nit: this has already been merged, but next time please make the
> > > subject line match the convention wrt capitalization.  This is not a
> > > PCI-specific idea; all it takes is "git log --oneline <file>", and
> > > that should be common practice anywhere in the kernel.
> >
> > Sorry about that. I didn't really notice the capitalization, but
> > focused on the prefix.
> >
> > > On Wed, Feb 27, 2019 at 05:52:19PM +0100, Lucas Stach wrote:
> > > > Since 7c5925afbc58 (PCI: dwc: Move MSI IRQs allocation to IRQ domains
> > > > hierarchical API) the MSI init claims one of the controller IRQs as a
> > > > chained IRQ line for the MSI controller. On some designs, like the i.MX6,
> > > > this line is shared with a PCIe legacy IRQ. When the line is claimed for
> > > > the MSI domain, any device trying to use this legacy IRQs will fail to
> > > > request this IRQ line.
> > > >
> > > > As MSI and legacy IRQs are already mutually exclusive on the DWC core,
> > > > as the core won't forward any legacy IRQs once any MSI has been enabled,
> > > > users wishing to use legacy IRQs already need to explictly disable MSI
> > > > support (usually via the pci=nomsi kernel commandline option). To avoid
> > > > any issues with MSI conflicting with legacy IRQs, just skip all of the
> > > > DWC MSI initalization, including the IRQ line claim, when MSI is disabled.
> > >
> > > Does this mean that if we have a device that uses legacy IRQs, the
> > > user has to figure out to boot with "pci=nomsi"?
> >
> > As long as there is only a single device connected and there are no
> > port services things will work. If port services are active, those will
> > start to use MSIs, breaking legacy IRQs in the process.
> >
> > I've asked Synopsys if there is a workaround for this, but it seems
> > that the core is working "as designed" with no workaround for this icky
> > behavior.
>
> Is this the general DWC controller behaviour? Or something that is
> specific to a given implementation? I can't believe someone actually
> thought this is an acceptable behaviour...
>
> Gustavo, can you please check with your HW colleagues and let
> everybody know what's the official Synopsys position on this?
>
> >
> > > I don't like kernel command line parameters.  If we need that
> > > parameter to make devices with legacy IRQs work, what happens without
> > > the parameter?  How obvious is it that the fix is to use "pci=nomsi"?
> >
> > Totally non-obvious currently. Maybe this warrants a warning in the
> > kernel log.
> >
> > > Is it impossible for Linux to figure this out and make it work
> > > automatically?
> > >
> > > If we can't do it automatically, fine, maybe we have to live with the
> > > parameter.  But if there's any way we can avoid it, we should.
> >
> > I don't think there is a practical way to make this work
> > automatically.
> >
> > At PCIe enumeration time we can only know if all devices in the
> > hierarchy are able to use MSIs, but this doesn't mean the device driver
> > will use MSIs. This is something we only get to know at driver probe
> > time, at which point it may already be too late, as another device
> > might already use some MSIs, so we can't revert to legacy IRQs only
> > mode.
>
> It is almost as the default is backward. It feels like it should
> default to legacy interrupts, and only enable MSIs if the user
> actually asks for it. We should always be able to provide legacy
> interrupts, as this is the usual fallback for most drivers.
>

Marc,

I would agree with 'should default to legacy interrupts' as that works
everywhere AFAIK and likely allows for improved performance.

Tim