From: Matthias Kaehlcke <mka@chromium.org>
To: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Cc: Dhruva Gole <d-gole@ti.com>,
lpieralisi@kernel.org, robh@kernel.org, andersson@kernel.org,
konrad.dybcio@linaro.org, kw@linux.com, bhelgaas@google.com,
linux-pci@vger.kernel.org, linux-arm-msm@vger.kernel.org,
linux-kernel@vger.kernel.org, quic_krichai@quicinc.com,
johan+linaro@kernel.org, steev@kali.org
Subject: Re: [PATCH 1/1] PCI: qcom: Add support for system suspend and resume
Date: Mon, 9 Jan 2023 22:41:59 +0000 [thread overview]
Message-ID: <Y7yYN1yGxqquR1FI@google.com> (raw)
In-Reply-To: <20230106190252.GA485076@thinkpad>
On Sat, Jan 07, 2023 at 12:32:52AM +0530, Manivannan Sadhasivam wrote:
> On Fri, Jan 06, 2023 at 06:17:19PM +0000, Matthias Kaehlcke wrote:
> > On Thu, Jan 05, 2023 at 07:06:39PM +0530, Manivannan Sadhasivam wrote:
> > > On Tue, Jan 03, 2023 at 04:46:11PM +0530, Dhruva Gole wrote:
> > > >
> > > >
> > > > On 03/01/23 13:19, Manivannan Sadhasivam wrote:
> > > > > During the system suspend, vote for minimal interconnect bandwidth and
> > > > > also turn OFF the resources like clock and PHY if there are no active
> > > > > devices connected to the controller. For the controllers with active
> > > > > devices, the resources are kept ON as removing the resources will
> > > > > trigger access violation during the late end of suspend cycle as kernel
> > > > > tries to access the config space of PCIe devices to mask the MSIs.
> > > > >
> > > > > Also, it is not desirable to put the link into L2/L3 state as that
> > > > > implies VDD supply will be removed and the devices may go into powerdown
> > > > > state. This will affect the lifetime of storage devices like NVMe.
> > > > >
> > > > > And finally, during resume, turn ON the resources if the controller was
> > > > > truly suspended (resources OFF) and update the interconnect bandwidth
> > > > > based on PCIe Gen speed.
> > > > >
> > > > > Suggested-by: Krishna chaitanya chundru <quic_krichai@quicinc.com>
> > > > > Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
> > > > > ---
> > > >
> > > > Nice to have another driver added to the list of system suspend
> > > > support!
> > > >
> > > > Acked-by: Dhruva Gole <d-gole@ti.com>
> > > >
> > > > > drivers/pci/controller/dwc/pcie-qcom.c | 52 ++++++++++++++++++++++++++
> > > > > 1 file changed, 52 insertions(+)
> > > > >
> > > > > diff --git a/drivers/pci/controller/dwc/pcie-qcom.c b/drivers/pci/controller/dwc/pcie-qcom.c
> > > > > index 5696e327795b..48810f1f2dba 100644
> > > > > --- a/drivers/pci/controller/dwc/pcie-qcom.c
> > > > > +++ b/drivers/pci/controller/dwc/pcie-qcom.c
> > > > > @@ -227,6 +227,7 @@ struct qcom_pcie {
> > > > > struct gpio_desc *reset;
> > > > > struct icc_path *icc_mem;
> > > > > const struct qcom_pcie_cfg *cfg;qcom_pcie_icc_update
> > > > > + bool suspended;
> > > > > };
> > > > > #define to_qcom_pcie(x) dev_get_drvdata((x)->dev)
> > > > > @@ -1835,6 +1836,52 @@ static int qcom_pcie_remove(struct platform_device *pdev)
> > > > > return 0;
> > > > > }
> > > > > +static int qcom_pcie_suspend_noirq(struct device *dev)
> > > > > +{
> > > > > + struct qcom_pcie *pcie = dev_get_drvdata(dev);
> > > > > + int ret;
> > > > > +
> > > > > + ret = icc_set_bw(pcie->icc_mem, 0, 0);
> > > > > + if (ret) {
> > > > > + dev_err(pcie->pci->dev, "Failed to set interconnect bandwidth: %d\n", ret);
> > > > > + return ret;
> > > > > + }
> > > > > +
> > > > > + /*
> > > > > + * Turn OFF the resources only for controllers without active PCIe devices. For controllers
> > > > > + * with active devices, the resources are kept ON and the link is expected to be in L0/L1
> > > > > + * (sub)states.
> > > > > + *
> > > > > + * Turning OFF the resources for controllers with active PCIe devices will trigger access
> > > > > + * violation during the end of the suspend cycle, as kernel tries to access the PCIe devices
> > > > > + * config space for masking MSIs.
> > > > > + *
> > > > > + * Also, it is not desirable to put the link into L2/L3 state as that implies VDD supply
> > > > > + * will be removed and the devices may go into powerdown state. This will affect the
> > > > > + * lifetime of the storage devices like NVMe.
> > > > > + */
> > > > > + if (!dw_pcie_link_up(pcie->pci)) {
> > > > > + qcom_pcie_host_deinit(&pcie->pci->pp);
> > > > > + pcie->suspended = true;
> > > > > + }
> > > > > +
> > > > > + return 0;
> > > > > +}
> > > > > +
> > > > > +static int qcom_pcie_resume_noirq(struct device *dev)
> > > > > +{
> > > > > + struct qcom_pcie *pcie = dev_get_drvdata(dev);
> > > > > +
> > > > > + if (pcie->suspended) {
> > > > > + qcom_pcie_host_init(&pcie->pci->pp);
> > > > > + pcie->suspended = false;
> > > > > + }
> > > > > +
> > > > > + qcom_pcie_icc_update(pcie);
> > > > > +
> > > > > + return 0;
> > > > > +}
> > > > > +
> > > > > static const struct of_device_id qcom_pcie_match[] = {
> > > > > { .compatible = "qcom,pcie-apq8064", .data = &cfg_2_1_0 },
> > > > > { .compatible = "qcom,pcie-apq8084", .data = &cfg_1_0_0 },
> > > > > @@ -1870,12 +1917,17 @@ DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_QCOM, 0x0302, qcom_fixup_class);
> > > > > DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_QCOM, 0x1000, qcom_fixup_class);
> > > > > DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_QCOM, 0x1001, qcom_fixup_class);
> > > > > +static const struct dev_pm_ops qcom_pcie_pm_ops = {
> > > > > + NOIRQ_SYSTEM_SLEEP_PM_OPS(qcom_pcie_suspend_noirq, qcom_pcie_resume_noirq)
> > > > > +};
> > > > > +
> > > > > static struct platform_driver qcom_pcie_driver = {
> > > > > .probe = qcom_pcie_probe,
> > > > > .remove = qcom_pcie_remove,
> > > > > .driver = {
> > > > > .name = "qcom-pcie",
> > > > > .of_match_table = qcom_pcie_match,
> > > > > + .pm = &qcom_pcie_pm_ops,
> > > > > },
> > > > > };
> > > > > module_platform_driver(qcom_pcie_driver);
> > > >
> > > > Out of curiosity, were you able to measure how much power you were able
> > > > to save after adding suspend support for PCIe? I don't know if clock
> > > > gating really saves much amount of power, but yeah its true that we
> > > > can't really cut off the power domain entirely in this case.
> > > >
> > >
> > > I did not measure the power consumption and I agree that we won't save much
> > > power with setting icc bandwidth to 0. But it is better to have something
> > > than nothing. And in the coming days, I have plans to look into other power
> > > saving measures also.
> >
> > On a sc7280 system I see a reduction of ~30mW with this patch when no PCI
> > card is plugged in. The reduction seems to come from powering the PHY down.
> >
>
> Thanks a lot for testing!
>
> > Interestingly on that system power consumption during suspend (without this
> > patch) is ~30mW higher *without* a PCI card vs. with a card. Maybe the PHY
> > doesn't enter a low power mode when no card is plugged in?
>
> Yeah, both PHY and controllers are never put into low power mode even if there
> are no devices connected. I don't know if the low power mode is possible at
> all with PHY.
It's still interesting that the PHY apparently at least enters a *lower* power
mode when a card is plugged in, the extra 30mW are only seen without a card.
next prev parent reply other threads:[~2023-01-09 22:43 UTC|newest]
Thread overview: 9+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-01-03 7:49 [PATCH 0/1] PCI: qcom: Add support for system suspend and resume Manivannan Sadhasivam
2023-01-03 7:49 ` [PATCH 1/1] " Manivannan Sadhasivam
2023-01-03 11:16 ` Dhruva Gole
2023-01-05 13:36 ` Manivannan Sadhasivam
2023-01-06 18:17 ` Matthias Kaehlcke
2023-01-06 19:02 ` Manivannan Sadhasivam
2023-01-09 22:41 ` Matthias Kaehlcke [this message]
2023-01-03 13:16 ` Johan Hovold
2023-01-05 13:33 ` Manivannan Sadhasivam
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=Y7yYN1yGxqquR1FI@google.com \
--to=mka@chromium.org \
--cc=andersson@kernel.org \
--cc=bhelgaas@google.com \
--cc=d-gole@ti.com \
--cc=johan+linaro@kernel.org \
--cc=konrad.dybcio@linaro.org \
--cc=kw@linux.com \
--cc=linux-arm-msm@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-pci@vger.kernel.org \
--cc=lpieralisi@kernel.org \
--cc=manivannan.sadhasivam@linaro.org \
--cc=quic_krichai@quicinc.com \
--cc=robh@kernel.org \
--cc=steev@kali.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).