PME while runtime suspend

PME while runtime suspend

[Loic Poulain]

Hi Bjorn,

Trying to support runtime suspend in a driver, which puts the device
in D3hot and wait either for host/driver initiated resume
(runtime_get), or device initiated resume (PME).

But, given that old change: 42eca2302146 ("PCI: Don't touch card regs
after runtime suspend D3")

PME that was enabled from pci_finish_runtime_suspend() is not enabled
anymore for almost all drivers in case of runtime-suspend. The only
way to enable this is by calling pci_wake_from_d3() from the PCI device
driver's runtime_suspend() callback, but this function fails if the
device wake_up is not enabled, which makes sense since it targets
system-wide sleep wake-up (and wake-up is user/distro policy).

So is there a proper way to allow PME while the device is runtime
suspended, without having to tell the user to enabled 'unrelated' wake_up
capability?

Regards,
Loic

Re: PME while runtime suspend

[Bjorn Helgaas]

[+cc Rafael, Dave (author of 42eca2302146), Vaibhav, linux-pm]

On Fri, Feb 26, 2021 at 11:37:12AM +0100, Loic Poulain wrote:
> Hi Bjorn,
> 
> Trying to support runtime suspend in a driver, which puts the device
> in D3hot and wait either for host/driver initiated resume
> (runtime_get), or device initiated resume (PME).
> 
> But, given that old change: 42eca2302146 ("PCI: Don't touch card regs
> after runtime suspend D3")
> 
> PME that was enabled from pci_finish_runtime_suspend() is not enabled
> anymore for almost all drivers in case of runtime-suspend. The only
> way to enable this is by calling pci_wake_from_d3() from the PCI device
> driver's runtime_suspend() callback, but this function fails if the
> device wake_up is not enabled, which makes sense since it targets
> system-wide sleep wake-up (and wake-up is user/distro policy).
> 
> So is there a proper way to allow PME while the device is runtime
> suspended, without having to tell the user to enabled 'unrelated' wake_up
> capability?

pci_pm_runtime_suspend() calls pci_finish_runtime_suspend(), which
enables wake-up, unless "pci_dev->state_saved".  IIUC we should be
enabling wake-up unless the driver has called pci_save_state() itself.

So I infer that your driver does call pci_save_state() and the PCI
core does not enable wake-up.  Right?

Why does your driver call pci_save_state()?  In most cases I don't
think drivers should need to do that themselves because the PCI core
will do it for them.  E.g., see Vaibhav's recent eb6779d4c505 ("e1000:
use generic power management") [1]

Bjorn

[1] https://git.kernel.org/linus/eb6779d4c505

Re: PME while runtime suspend

[Loic Poulain]

Hi Bjorn,

On Fri, 26 Feb 2021 at 23:22, Bjorn Helgaas <helgaas@kernel.org> wrote:
>
> [+cc Rafael, Dave (author of 42eca2302146), Vaibhav, linux-pm]
>
> On Fri, Feb 26, 2021 at 11:37:12AM +0100, Loic Poulain wrote:
> > Hi Bjorn,
> >
> > Trying to support runtime suspend in a driver, which puts the device
> > in D3hot and wait either for host/driver initiated resume
> > (runtime_get), or device initiated resume (PME).
> >
> > But, given that old change: 42eca2302146 ("PCI: Don't touch card regs
> > after runtime suspend D3")
> >
> > PME that was enabled from pci_finish_runtime_suspend() is not enabled
> > anymore for almost all drivers in case of runtime-suspend. The only
> > way to enable this is by calling pci_wake_from_d3() from the PCI device
> > driver's runtime_suspend() callback, but this function fails if the
> > device wake_up is not enabled, which makes sense since it targets
> > system-wide sleep wake-up (and wake-up is user/distro policy).
> >
> > So is there a proper way to allow PME while the device is runtime
> > suspended, without having to tell the user to enabled 'unrelated' wake_up
> > capability?
>
> pci_pm_runtime_suspend() calls pci_finish_runtime_suspend(), which
> enables wake-up, unless "pci_dev->state_saved".  IIUC we should be
> enabling wake-up unless the driver has called pci_save_state() itself.
>
> So I infer that your driver does call pci_save_state() and the PCI
> core does not enable wake-up.  Right?

Right.



>
> Why does your driver call pci_save_state()?  In most cases I don't
> think drivers should need to do that themselves because the PCI core
> will do it for them.  E.g., see Vaibhav's recent eb6779d4c505 ("e1000:
> use generic power management") [1]

Thanks for the pointer, I was storing the PCI state in order to
restore it when the device is crashing and lose its PCI context. But I
can do that one time once the device is initialized. I've applied the
same changes as you pointed, and it works as expected.

Thanks,
Loic

Re: PME while runtime suspend

[Bjorn Helgaas]

On Tue, Mar 02, 2021 at 03:52:05PM +0100, Loic Poulain wrote:
> Hi Bjorn,
> 
> On Fri, 26 Feb 2021 at 23:22, Bjorn Helgaas <helgaas@kernel.org> wrote:
> >
> > [+cc Rafael, Dave (author of 42eca2302146), Vaibhav, linux-pm]
> >
> > On Fri, Feb 26, 2021 at 11:37:12AM +0100, Loic Poulain wrote:
> > > Hi Bjorn,
> > >
> > > Trying to support runtime suspend in a driver, which puts the device
> > > in D3hot and wait either for host/driver initiated resume
> > > (runtime_get), or device initiated resume (PME).
> > >
> > > But, given that old change: 42eca2302146 ("PCI: Don't touch card regs
> > > after runtime suspend D3")
> > >
> > > PME that was enabled from pci_finish_runtime_suspend() is not enabled
> > > anymore for almost all drivers in case of runtime-suspend. The only
> > > way to enable this is by calling pci_wake_from_d3() from the PCI device
> > > driver's runtime_suspend() callback, but this function fails if the
> > > device wake_up is not enabled, which makes sense since it targets
> > > system-wide sleep wake-up (and wake-up is user/distro policy).
> > >
> > > So is there a proper way to allow PME while the device is runtime
> > > suspended, without having to tell the user to enabled 'unrelated' wake_up
> > > capability?
> >
> > pci_pm_runtime_suspend() calls pci_finish_runtime_suspend(), which
> > enables wake-up, unless "pci_dev->state_saved".  IIUC we should be
> > enabling wake-up unless the driver has called pci_save_state() itself.
> >
> > So I infer that your driver does call pci_save_state() and the PCI
> > core does not enable wake-up.  Right?
> 
> Right.
> 
> > Why does your driver call pci_save_state()?  In most cases I don't
> > think drivers should need to do that themselves because the PCI core
> > will do it for them.  E.g., see Vaibhav's recent eb6779d4c505 ("e1000:
> > use generic power management") [1]
> 
> Thanks for the pointer, I was storing the PCI state in order to
> restore it when the device is crashing and lose its PCI context. But I
> can do that one time once the device is initialized. I've applied the
> same changes as you pointed, and it works as expected.

Great, thanks for letting us know!  It's always nice when you can
*remove* code and things work better!

Bjorn