From: Bjorn Helgaas <helgaas@kernel.org> To: Dexuan Cui <decui@microsoft.com> Cc: "Rafael J . Wysocki" <rafael.j.wysocki@intel.com>, Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>, Michael Kelley <mikelley@microsoft.com>, Sasha Levin <Alexander.Levin@microsoft.com>, Haiyang Zhang <haiyangz@microsoft.com>, KY Srinivasan <kys@microsoft.com>, Stephen Hemminger <sthemmin@microsoft.com>, olaf@aepfle.de, apw@canonical.com, jasowang@redhat.com, vkuznets@redhat.com, marcelo.cerri@canonical.com, jackm@mellanox.com, linux-pci@vger.kernel.org, linux-hyperv@vger.kernel.org, linux-kernel@vger.kernel.org, driverdev-devel@linuxdriverproject.org, Bjorn Helgaas <bhelgaas@google.com>, stable@vger.kernel.org Subject: [PATCH 1/7] PCI/PM: Always return devices to D0 when thawing Date: Mon, 14 Oct 2019 18:00:10 -0500 [thread overview] Message-ID: <20191014230016.240912-2-helgaas@kernel.org> (raw) In-Reply-To: <20191014230016.240912-1-helgaas@kernel.org> From: Dexuan Cui <decui@microsoft.com> pci_pm_thaw_noirq() is supposed to return the device to D0 and restore its configuration registers, but previously it only did that for devices whose drivers implemented the new power management ops. Hibernation, e.g., via "echo disk > /sys/power/state", involves freezing devices, creating a hibernation image, thawing devices, writing the image, and powering off. The fact that thawing did not return devices with legacy power management to D0 caused errors, e.g., in this path: pci_pm_thaw_noirq if (pci_has_legacy_pm_support(pci_dev)) # true for Mellanox VF driver return pci_legacy_resume_early(dev) # ... legacy PM skips the rest pci_set_power_state(pci_dev, PCI_D0) pci_restore_state(pci_dev) pci_pm_thaw if (pci_has_legacy_pm_support(pci_dev)) pci_legacy_resume drv->resume mlx4_resume ... pci_enable_msix_range ... if (dev->current_state != PCI_D0) # <--- return -EINVAL; which caused these warnings: mlx4_core a6d1:00:02.0: INTx is not supported in multi-function mode, aborting PM: dpm_run_callback(): pci_pm_thaw+0x0/0xd7 returns -95 PM: Device a6d1:00:02.0 failed to thaw: error -95 Return devices to D0 and restore config registers for all devices, not just those whose drivers support new power management. [bhelgaas: also call pci_restore_state() before pci_legacy_resume_early(), update comment, add stable tag, commit log] Link: https://lore.kernel.org/r/KU1P153MB016637CAEAD346F0AA8E3801BFAD0@KU1P153MB0166.APCP153.PROD.OUTLOOK.COM Signed-off-by: Dexuan Cui <decui@microsoft.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Cc: stable@vger.kernel.org # v4.13+ --- drivers/pci/pci-driver.c | 17 +++++++++++------ 1 file changed, 11 insertions(+), 6 deletions(-) diff --git a/drivers/pci/pci-driver.c b/drivers/pci/pci-driver.c index a8124e47bf6e..d4ac8ce8c1f9 100644 --- a/drivers/pci/pci-driver.c +++ b/drivers/pci/pci-driver.c @@ -1076,17 +1076,22 @@ static int pci_pm_thaw_noirq(struct device *dev) return error; } - if (pci_has_legacy_pm_support(pci_dev)) - return pci_legacy_resume_early(dev); - /* - * pci_restore_state() requires the device to be in D0 (because of MSI - * restoration among other things), so force it into D0 in case the - * driver's "freeze" callbacks put it into a low-power state directly. + * Both the legacy ->resume_early() and the new pm->thaw_noirq() + * callbacks assume the device has been returned to D0 and its + * config state has been restored. + * + * In addition, pci_restore_state() restores MSI-X state in MMIO + * space, which requires the device to be in D0, so return it to D0 + * in case the driver's "freeze" callbacks put it into a low-power + * state. */ pci_set_power_state(pci_dev, PCI_D0); pci_restore_state(pci_dev); + if (pci_has_legacy_pm_support(pci_dev)) + return pci_legacy_resume_early(dev); + if (drv && drv->pm && drv->pm->thaw_noirq) error = drv->pm->thaw_noirq(dev); -- 2.23.0.700.g56cf767bdb-goog
WARNING: multiple messages have this Message-ID (diff)
From: Bjorn Helgaas <helgaas@kernel.org> To: Dexuan Cui <decui@microsoft.com> Cc: driverdev-devel@linuxdriverproject.org, olaf@aepfle.de, Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>, Stephen Hemminger <sthemmin@microsoft.com>, jackm@mellanox.com, Haiyang Zhang <haiyangz@microsoft.com>, "Rafael J . Wysocki" <rafael.j.wysocki@intel.com>, linux-hyperv@vger.kernel.org, stable@vger.kernel.org, Michael Kelley <mikelley@microsoft.com>, Sasha Levin <Alexander.Levin@microsoft.com>, marcelo.cerri@canonical.com, linux-pci@vger.kernel.org, apw@canonical.com, vkuznets@redhat.com, Bjorn Helgaas <bhelgaas@google.com>, jasowang@redhat.com, linux-kernel@vger.kernel.org Subject: [PATCH 1/7] PCI/PM: Always return devices to D0 when thawing Date: Mon, 14 Oct 2019 18:00:10 -0500 [thread overview] Message-ID: <20191014230016.240912-2-helgaas@kernel.org> (raw) In-Reply-To: <20191014230016.240912-1-helgaas@kernel.org> From: Dexuan Cui <decui@microsoft.com> pci_pm_thaw_noirq() is supposed to return the device to D0 and restore its configuration registers, but previously it only did that for devices whose drivers implemented the new power management ops. Hibernation, e.g., via "echo disk > /sys/power/state", involves freezing devices, creating a hibernation image, thawing devices, writing the image, and powering off. The fact that thawing did not return devices with legacy power management to D0 caused errors, e.g., in this path: pci_pm_thaw_noirq if (pci_has_legacy_pm_support(pci_dev)) # true for Mellanox VF driver return pci_legacy_resume_early(dev) # ... legacy PM skips the rest pci_set_power_state(pci_dev, PCI_D0) pci_restore_state(pci_dev) pci_pm_thaw if (pci_has_legacy_pm_support(pci_dev)) pci_legacy_resume drv->resume mlx4_resume ... pci_enable_msix_range ... if (dev->current_state != PCI_D0) # <--- return -EINVAL; which caused these warnings: mlx4_core a6d1:00:02.0: INTx is not supported in multi-function mode, aborting PM: dpm_run_callback(): pci_pm_thaw+0x0/0xd7 returns -95 PM: Device a6d1:00:02.0 failed to thaw: error -95 Return devices to D0 and restore config registers for all devices, not just those whose drivers support new power management. [bhelgaas: also call pci_restore_state() before pci_legacy_resume_early(), update comment, add stable tag, commit log] Link: https://lore.kernel.org/r/KU1P153MB016637CAEAD346F0AA8E3801BFAD0@KU1P153MB0166.APCP153.PROD.OUTLOOK.COM Signed-off-by: Dexuan Cui <decui@microsoft.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Cc: stable@vger.kernel.org # v4.13+ --- drivers/pci/pci-driver.c | 17 +++++++++++------ 1 file changed, 11 insertions(+), 6 deletions(-) diff --git a/drivers/pci/pci-driver.c b/drivers/pci/pci-driver.c index a8124e47bf6e..d4ac8ce8c1f9 100644 --- a/drivers/pci/pci-driver.c +++ b/drivers/pci/pci-driver.c @@ -1076,17 +1076,22 @@ static int pci_pm_thaw_noirq(struct device *dev) return error; } - if (pci_has_legacy_pm_support(pci_dev)) - return pci_legacy_resume_early(dev); - /* - * pci_restore_state() requires the device to be in D0 (because of MSI - * restoration among other things), so force it into D0 in case the - * driver's "freeze" callbacks put it into a low-power state directly. + * Both the legacy ->resume_early() and the new pm->thaw_noirq() + * callbacks assume the device has been returned to D0 and its + * config state has been restored. + * + * In addition, pci_restore_state() restores MSI-X state in MMIO + * space, which requires the device to be in D0, so return it to D0 + * in case the driver's "freeze" callbacks put it into a low-power + * state. */ pci_set_power_state(pci_dev, PCI_D0); pci_restore_state(pci_dev); + if (pci_has_legacy_pm_support(pci_dev)) + return pci_legacy_resume_early(dev); + if (drv && drv->pm && drv->pm->thaw_noirq) error = drv->pm->thaw_noirq(dev); -- 2.23.0.700.g56cf767bdb-goog _______________________________________________ devel mailing list devel@linuxdriverproject.org http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel
next prev parent reply other threads:[~2019-10-14 23:01 UTC|newest] Thread overview: 36+ messages / expand[flat|nested] mbox.gz Atom feed top 2019-10-14 23:00 [PATCH v3 0/7] PCI: PM: Move to D0 before calling pci_legacy_resume_early() Bjorn Helgaas 2019-10-14 23:00 ` Bjorn Helgaas 2019-10-14 23:00 ` Bjorn Helgaas [this message] 2019-10-14 23:00 ` [PATCH 1/7] PCI/PM: Always return devices to D0 when thawing Bjorn Helgaas 2019-10-15 17:15 ` Rafael J. Wysocki 2019-10-15 17:15 ` Rafael J. Wysocki 2019-10-14 23:00 ` [PATCH 2/7] PCI/PM: Correct pci_pm_thaw_noirq() documentation Bjorn Helgaas 2019-10-14 23:00 ` Bjorn Helgaas 2019-10-15 17:17 ` Rafael J. Wysocki 2019-10-15 17:17 ` Rafael J. Wysocki 2019-10-14 23:00 ` [PATCH 3/7] PCI/PM: Clear PCIe PME Status even for legacy power management Bjorn Helgaas 2019-10-14 23:00 ` Bjorn Helgaas 2019-10-15 17:20 ` Rafael J. Wysocki 2019-10-15 17:20 ` Rafael J. Wysocki 2019-10-14 23:00 ` [PATCH 4/7] PCI/PM: Run resume fixups before disabling wakeup events Bjorn Helgaas 2019-10-14 23:00 ` Bjorn Helgaas 2019-10-15 17:21 ` Rafael J. Wysocki 2019-10-15 17:21 ` Rafael J. Wysocki 2019-10-14 23:00 ` [PATCH 5/7] PCI/PM: Make power management op coding style consistent Bjorn Helgaas 2019-10-14 23:00 ` Bjorn Helgaas 2019-10-15 17:22 ` Rafael J. Wysocki 2019-10-15 17:22 ` Rafael J. Wysocki 2019-10-16 13:50 ` Dan Carpenter 2019-10-16 13:50 ` Dan Carpenter 2019-10-14 23:00 ` [PATCH 6/7] PCI/PM: Wrap long lines in documentation Bjorn Helgaas 2019-10-14 23:00 ` Bjorn Helgaas 2019-10-15 17:23 ` Rafael J. Wysocki 2019-10-15 17:23 ` Rafael J. Wysocki 2019-10-14 23:00 ` [PATCH 7/7] PCI/MSI: Move power state check out of pci_msi_supported() Bjorn Helgaas 2019-10-14 23:00 ` Bjorn Helgaas 2019-10-15 17:24 ` Rafael J. Wysocki 2019-10-15 17:24 ` Rafael J. Wysocki 2019-10-15 18:24 ` [PATCH v3 0/7] PCI: PM: Move to D0 before calling pci_legacy_resume_early() Dexuan Cui 2019-10-15 18:24 ` Dexuan Cui 2019-10-15 18:42 ` Bjorn Helgaas 2019-10-15 18:42 ` Bjorn Helgaas
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20191014230016.240912-2-helgaas@kernel.org \ --to=helgaas@kernel.org \ --cc=Alexander.Levin@microsoft.com \ --cc=apw@canonical.com \ --cc=bhelgaas@google.com \ --cc=decui@microsoft.com \ --cc=driverdev-devel@linuxdriverproject.org \ --cc=haiyangz@microsoft.com \ --cc=jackm@mellanox.com \ --cc=jasowang@redhat.com \ --cc=kys@microsoft.com \ --cc=linux-hyperv@vger.kernel.org \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-pci@vger.kernel.org \ --cc=lorenzo.pieralisi@arm.com \ --cc=marcelo.cerri@canonical.com \ --cc=mikelley@microsoft.com \ --cc=olaf@aepfle.de \ --cc=rafael.j.wysocki@intel.com \ --cc=stable@vger.kernel.org \ --cc=sthemmin@microsoft.com \ --cc=vkuznets@redhat.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.