Re: [PATCH RESEND] PCI: disable runtime PM for PLX switches

[Alexander Fomichev <fomichev.ru@gmail.com>]

On Tue, Apr 23, 2019 at 04:53:40PM -0500, Bjorn Helgaas wrote:
> On Mon, Apr 15, 2019 at 09:15:54AM -0500, Bjorn Helgaas wrote:
> > This says it's a resend, but I don't see a previous posting; maybe it was
> > HTML and rejected by the mailing list?
> > 
The first post was niether rejected nor accepted in ML. So I added
"resend" tag in case it appears in the archive finally.

> > On Mon, Apr 15, 2019 at 04:59:03PM +0300, Alexander Fomichev wrote:
> > > PLX switches have an issue that their internal registers become inaccessible
> > > when runtime PM is enabled. Therefore PLX service tools can't communicate
> > > with the hardware. A kernel option "pcie_port_pm=off" can be used as a
> > > workaround. But it affects all the devices.
> > > So this solution is to add PLX switch devices to the quirk list for
> > > disabling runtime PM only for them.
> > 
> > I assume the problem is actually that the config space registers are
> > inaccessible when the device is in D3hot?
> 
> Reading this again, I realize you said "internal registers".  I don't
> know whether that actually means config space registers (which
> *should* work even when the device is in D3hot (see the PCIe reference
> below and PCI Power Management Spec r1.2, sec 5.4.1)), or MMIO
> registers (which are not expected to work while in D3hot).
> 
> If the service tools read MMIO registers, presumably that goes through
> some driver that should be able to manage runtime PM.  Or, if there's
> no driver, I think your service tool could prevent runtime power
> management by changing /sys/devices/.../power/control to "on" (see
> Documentation/power/runtime_pm.txt).
> 
You're right. Config space registers are accessible. The driver can't
read/write MMIO registers (Device-Specific Registers as they're called
by Broadcom).

> Please repost this with more details.
> 
> > I think config space access is supposed to work when a device is in D3hot
> > (see PCIe r4.0, sec 5.3.1.4).
> > 
> > If it doesn't work, wouldn't that mean that we couldn't even bring the
> > device *out* of D3hot, since that requires a config write?
> > 
> > If this is really the problem, it'd be nice to identify this specifically
> > instead of piggy-backing on the "is_hotplug_bridge" thing, which might be
> > coincidentally related, but also carries other meanings.
> > 
The proposed patch was a sort of workaround. If this approach is not
acceptable then it needs more research how to change PLX driver that it
can play with PM to access MMIO registers.

Regards,
  Alexander