[PATCH 0/7] PCI: layerscape: Add power management support

[PATCH 0/7] PCI: layerscape: Add power management support

[Zhiqiang Hou]

From: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>

This patch series is to add PCIe power management support for NXP
Layerscape platfroms.

Hou Zhiqiang (7):
  PCI: dwc: Fix a bug of the case dw_pci->ops is NULL
  PCI: layerscape: Change to use the DWC common link-up check function
  dt-bindings: pci: layerscape-pci: Add a optional property big-endian
  arm64: dts: layerscape: Add big-endian property for PCIe nodes
  dt-bindings: pci: layerscape-pci: Update the description of SCFG
    property
  dts: arm64: ls1043a: Add SCFG phandle for PCIe nodes
  PCI: layerscape: Add power management support

 .../bindings/pci/layerscape-pci.txt           |   6 +-
 .../arm64/boot/dts/freescale/fsl-ls1012a.dtsi |   1 +
 .../arm64/boot/dts/freescale/fsl-ls1043a.dtsi |   6 +
 .../arm64/boot/dts/freescale/fsl-ls1046a.dtsi |   3 +
 drivers/pci/controller/dwc/pci-layerscape.c   | 473 ++++++++++++++----
 drivers/pci/controller/dwc/pcie-designware.c  |  12 +-
 drivers/pci/controller/dwc/pcie-designware.h  |   1 +
 7 files changed, 388 insertions(+), 114 deletions(-)

-- 
2.17.1

[PATCH 1/7] PCI: dwc: Fix a bug of the case dw_pci->ops is NULL

[Zhiqiang Hou]

From: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>

The dw_pci->ops may be a NULL, and fix it by adding one more check.

Signed-off-by: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
---
 drivers/pci/controller/dwc/pcie-designware.c | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/drivers/pci/controller/dwc/pcie-designware.c b/drivers/pci/controller/dwc/pcie-designware.c
index b723e0cc41fb..bdf8938da9cd 100644
--- a/drivers/pci/controller/dwc/pcie-designware.c
+++ b/drivers/pci/controller/dwc/pcie-designware.c
@@ -140,7 +140,7 @@ u32 dw_pcie_read_dbi(struct dw_pcie *pci, u32 reg, size_t size)
 	int ret;
 	u32 val;
 
-	if (pci->ops->read_dbi)
+	if (pci->ops && pci->ops->read_dbi)
 		return pci->ops->read_dbi(pci, pci->dbi_base, reg, size);
 
 	ret = dw_pcie_read(pci->dbi_base + reg, size, &val);
@@ -155,7 +155,7 @@ void dw_pcie_write_dbi(struct dw_pcie *pci, u32 reg, size_t size, u32 val)
 {
 	int ret;
 
-	if (pci->ops->write_dbi) {
+	if (pci->ops && pci->ops->write_dbi) {
 		pci->ops->write_dbi(pci, pci->dbi_base, reg, size, val);
 		return;
 	}
@@ -200,7 +200,7 @@ u32 dw_pcie_read_atu(struct dw_pcie *pci, u32 reg, size_t size)
 	int ret;
 	u32 val;
 
-	if (pci->ops->read_dbi)
+	if (pci->ops && pci->ops->read_dbi)
 		return pci->ops->read_dbi(pci, pci->atu_base, reg, size);
 
 	ret = dw_pcie_read(pci->atu_base + reg, size, &val);
@@ -214,7 +214,7 @@ void dw_pcie_write_atu(struct dw_pcie *pci, u32 reg, size_t size, u32 val)
 {
 	int ret;
 
-	if (pci->ops->write_dbi) {
+	if (pci->ops && pci->ops->write_dbi) {
 		pci->ops->write_dbi(pci, pci->atu_base, reg, size, val);
 		return;
 	}
@@ -283,7 +283,7 @@ void dw_pcie_prog_outbound_atu(struct dw_pcie *pci, int index, int type,
 {
 	u32 retries, val;
 
-	if (pci->ops->cpu_addr_fixup)
+	if (pci->ops && pci->ops->cpu_addr_fixup)
 		cpu_addr = pci->ops->cpu_addr_fixup(pci, cpu_addr);
 
 	if (pci->iatu_unroll_enabled) {
@@ -470,7 +470,7 @@ int dw_pcie_link_up(struct dw_pcie *pci)
 {
 	u32 val;
 
-	if (pci->ops->link_up)
+	if (pci->ops && pci->ops->link_up)
 		return pci->ops->link_up(pci);
 
 	val = readl(pci->dbi_base + PCIE_PORT_DEBUG1);
-- 
2.17.1

RE: [PATCH 1/7] PCI: dwc: Fix a bug of the case dw_pci->ops is NULL

[Gustavo Pimentel]

Hi Hou,

On Mon, Sep 7, 2020 at 6:37:55, Zhiqiang Hou <Zhiqiang.Hou@nxp.com> 
wrote:

> From: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
> 
> The dw_pci->ops may be a NULL, and fix it by adding one more check.
> 
> Signed-off-by: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
> ---
>  drivers/pci/controller/dwc/pcie-designware.c | 12 ++++++------
>  1 file changed, 6 insertions(+), 6 deletions(-)
> 
> diff --git a/drivers/pci/controller/dwc/pcie-designware.c b/drivers/pci/controller/dwc/pcie-designware.c
> index b723e0cc41fb..bdf8938da9cd 100644
> --- a/drivers/pci/controller/dwc/pcie-designware.c
> +++ b/drivers/pci/controller/dwc/pcie-designware.c
> @@ -140,7 +140,7 @@ u32 dw_pcie_read_dbi(struct dw_pcie *pci, u32 reg, size_t size)
>  	int ret;
>  	u32 val;
>  
> -	if (pci->ops->read_dbi)
> +	if (pci->ops && pci->ops->read_dbi)
>  		return pci->ops->read_dbi(pci, pci->dbi_base, reg, size);
>  
>  	ret = dw_pcie_read(pci->dbi_base + reg, size, &val);
> @@ -155,7 +155,7 @@ void dw_pcie_write_dbi(struct dw_pcie *pci, u32 reg, size_t size, u32 val)
>  {
>  	int ret;
>  
> -	if (pci->ops->write_dbi) {
> +	if (pci->ops && pci->ops->write_dbi) {
>  		pci->ops->write_dbi(pci, pci->dbi_base, reg, size, val);
>  		return;
>  	}
> @@ -200,7 +200,7 @@ u32 dw_pcie_read_atu(struct dw_pcie *pci, u32 reg, size_t size)
>  	int ret;
>  	u32 val;
>  
> -	if (pci->ops->read_dbi)
> +	if (pci->ops && pci->ops->read_dbi)
>  		return pci->ops->read_dbi(pci, pci->atu_base, reg, size);
>  
>  	ret = dw_pcie_read(pci->atu_base + reg, size, &val);
> @@ -214,7 +214,7 @@ void dw_pcie_write_atu(struct dw_pcie *pci, u32 reg, size_t size, u32 val)
>  {
>  	int ret;
>  
> -	if (pci->ops->write_dbi) {
> +	if (pci->ops && pci->ops->write_dbi) {
>  		pci->ops->write_dbi(pci, pci->atu_base, reg, size, val);
>  		return;
>  	}
> @@ -283,7 +283,7 @@ void dw_pcie_prog_outbound_atu(struct dw_pcie *pci, int index, int type,
>  {
>  	u32 retries, val;
>  
> -	if (pci->ops->cpu_addr_fixup)
> +	if (pci->ops && pci->ops->cpu_addr_fixup)
>  		cpu_addr = pci->ops->cpu_addr_fixup(pci, cpu_addr);
>  
>  	if (pci->iatu_unroll_enabled) {
> @@ -470,7 +470,7 @@ int dw_pcie_link_up(struct dw_pcie *pci)
>  {
>  	u32 val;
>  
> -	if (pci->ops->link_up)
> +	if (pci->ops && pci->ops->link_up)
>  		return pci->ops->link_up(pci);
>  
>  	val = readl(pci->dbi_base + PCIE_PORT_DEBUG1);
> -- 
> 2.17.1

Looks good to me.

Acked-by: Gustavo Pimentel <gustavo.pimentel@synopsys.com>

RE: [PATCH 1/7] PCI: dwc: Fix a bug of the case dw_pci->ops is NULL

[Z.q. Hou]

Hi Gustavo,

Thanks a lot for your review and ack!

Regards,
Zhiqiang

> -----Original Message-----
> From: Gustavo Pimentel <Gustavo.Pimentel@synopsys.com>
> Sent: 2020年9月9日 17:29
> To: Z.q. Hou <zhiqiang.hou@nxp.com>; linux-pci@vger.kernel.org;
> devicetree@vger.kernel.org; linux-kernel@vger.kernel.org;
> bhelgaas@google.com; robh+dt@kernel.org; shawnguo@kernel.org; Leo Li
> <leoyang.li@nxp.com>; lorenzo.pieralisi@arm.com
> Cc: M.h. Lian <minghuan.lian@nxp.com>; Mingkai Hu
> <mingkai.hu@nxp.com>; Roy Zang <roy.zang@nxp.com>
> Subject: RE: [PATCH 1/7] PCI: dwc: Fix a bug of the case dw_pci->ops is NULL
> 
> Hi Hou,
> 
> On Mon, Sep 7, 2020 at 6:37:55, Zhiqiang Hou <Zhiqiang.Hou@nxp.com>
> wrote:
> 
> > From: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
> >
> > The dw_pci->ops may be a NULL, and fix it by adding one more check.
> >
> > Signed-off-by: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
> > ---
> >  drivers/pci/controller/dwc/pcie-designware.c | 12 ++++++------
> >  1 file changed, 6 insertions(+), 6 deletions(-)
> >
> > diff --git a/drivers/pci/controller/dwc/pcie-designware.c
> > b/drivers/pci/controller/dwc/pcie-designware.c
> > index b723e0cc41fb..bdf8938da9cd 100644
> > --- a/drivers/pci/controller/dwc/pcie-designware.c
> > +++ b/drivers/pci/controller/dwc/pcie-designware.c
> > @@ -140,7 +140,7 @@ u32 dw_pcie_read_dbi(struct dw_pcie *pci, u32
> reg, size_t size)
> >  	int ret;
> >  	u32 val;
> >
> > -	if (pci->ops->read_dbi)
> > +	if (pci->ops && pci->ops->read_dbi)
> >  		return pci->ops->read_dbi(pci, pci->dbi_base, reg, size);
> >
> >  	ret = dw_pcie_read(pci->dbi_base + reg, size, &val); @@ -155,7
> > +155,7 @@ void dw_pcie_write_dbi(struct dw_pcie *pci, u32 reg, size_t
> > size, u32 val)  {
> >  	int ret;
> >
> > -	if (pci->ops->write_dbi) {
> > +	if (pci->ops && pci->ops->write_dbi) {
> >  		pci->ops->write_dbi(pci, pci->dbi_base, reg, size, val);
> >  		return;
> >  	}
> > @@ -200,7 +200,7 @@ u32 dw_pcie_read_atu(struct dw_pcie *pci, u32
> reg, size_t size)
> >  	int ret;
> >  	u32 val;
> >
> > -	if (pci->ops->read_dbi)
> > +	if (pci->ops && pci->ops->read_dbi)
> >  		return pci->ops->read_dbi(pci, pci->atu_base, reg, size);
> >
> >  	ret = dw_pcie_read(pci->atu_base + reg, size, &val); @@ -214,7
> > +214,7 @@ void dw_pcie_write_atu(struct dw_pcie *pci, u32 reg, size_t
> > size, u32 val)  {
> >  	int ret;
> >
> > -	if (pci->ops->write_dbi) {
> > +	if (pci->ops && pci->ops->write_dbi) {
> >  		pci->ops->write_dbi(pci, pci->atu_base, reg, size, val);
> >  		return;
> >  	}
> > @@ -283,7 +283,7 @@ void dw_pcie_prog_outbound_atu(struct dw_pcie
> > *pci, int index, int type,  {
> >  	u32 retries, val;
> >
> > -	if (pci->ops->cpu_addr_fixup)
> > +	if (pci->ops && pci->ops->cpu_addr_fixup)
> >  		cpu_addr = pci->ops->cpu_addr_fixup(pci, cpu_addr);
> >
> >  	if (pci->iatu_unroll_enabled) {
> > @@ -470,7 +470,7 @@ int dw_pcie_link_up(struct dw_pcie *pci)  {
> >  	u32 val;
> >
> > -	if (pci->ops->link_up)
> > +	if (pci->ops && pci->ops->link_up)
> >  		return pci->ops->link_up(pci);
> >
> >  	val = readl(pci->dbi_base + PCIE_PORT_DEBUG1);
> > --
> > 2.17.1
> 
> Looks good to me.
> 
> Acked-by: Gustavo Pimentel <gustavo.pimentel@synopsys.com>
> 
> 
> 

Re: [PATCH 1/7] PCI: dwc: Fix a bug of the case dw_pci->ops is NULL

[Rob Herring]

On Mon, Sep 07, 2020 at 01:37:55PM +0800, Zhiqiang Hou wrote:
> From: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
> 
> The dw_pci->ops may be a NULL, and fix it by adding one more check.
> 
> Signed-off-by: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
> ---
>  drivers/pci/controller/dwc/pcie-designware.c | 12 ++++++------
>  1 file changed, 6 insertions(+), 6 deletions(-)

Reviewed-by: Rob Herring <robh@kernel.org>

Note that this may conflict with my 40 patch clean-up series.

Rob

RE: [PATCH 1/7] PCI: dwc: Fix a bug of the case dw_pci->ops is NULL

[Z.q. Hou]

Hi Rob,

Thanks a lot for your review!

Regards,
Zhiqiang

> -----Original Message-----
> From: Rob Herring <robh@kernel.org>
> Sent: 2020年9月15日 9:16
> To: Z.q. Hou <zhiqiang.hou@nxp.com>
> Cc: linux-pci@vger.kernel.org; devicetree@vger.kernel.org;
> linux-kernel@vger.kernel.org; bhelgaas@google.com;
> shawnguo@kernel.org; Leo Li <leoyang.li@nxp.com>;
> lorenzo.pieralisi@arm.com; gustavo.pimentel@synopsys.com; M.h. Lian
> <minghuan.lian@nxp.com>; Mingkai Hu <mingkai.hu@nxp.com>; Roy Zang
> <roy.zang@nxp.com>
> Subject: Re: [PATCH 1/7] PCI: dwc: Fix a bug of the case dw_pci->ops is NULL
> 
> On Mon, Sep 07, 2020 at 01:37:55PM +0800, Zhiqiang Hou wrote:
> > From: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
> >
> > The dw_pci->ops may be a NULL, and fix it by adding one more check.
> >
> > Signed-off-by: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
> > ---
> >  drivers/pci/controller/dwc/pcie-designware.c | 12 ++++++------
> >  1 file changed, 6 insertions(+), 6 deletions(-)
> 
> Reviewed-by: Rob Herring <robh@kernel.org>
> 
> Note that this may conflict with my 40 patch clean-up series.
> 
> Rob

[PATCH 2/7] PCI: layerscape: Change to use the DWC common link-up check function

[Zhiqiang Hou]

From: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>

The current Layerscape PCIe driver directly uses the physical layer
LTSSM code to check the link-up state, which treats the > L0 states
as link-up. This is not correct, since there is not explicit map
between link-up state and LTSSM. So this patch changes to use the
DWC common link-up check function.

Signed-off-by: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
---
 drivers/pci/controller/dwc/pci-layerscape.c | 141 ++------------------
 1 file changed, 10 insertions(+), 131 deletions(-)

diff --git a/drivers/pci/controller/dwc/pci-layerscape.c b/drivers/pci/controller/dwc/pci-layerscape.c
index f24f79a70d9a..be404c16bcbe 100644
--- a/drivers/pci/controller/dwc/pci-layerscape.c
+++ b/drivers/pci/controller/dwc/pci-layerscape.c
@@ -22,12 +22,6 @@
 
 #include "pcie-designware.h"
 
-/* PEX1/2 Misc Ports Status Register */
-#define SCFG_PEXMSCPORTSR(pex_idx)	(0x94 + (pex_idx) * 4)
-#define LTSSM_STATE_SHIFT	20
-#define LTSSM_STATE_MASK	0x3f
-#define LTSSM_PCIE_L0		0x11 /* L0 state */
-
 /* PEX Internal Configuration Registers */
 #define PCIE_STRFMR1		0x71c /* Symbol Timer & Filter Mask Register1 */
 #define PCIE_ABSERR		0x8d0 /* Bridge Slave Error Response Register */
@@ -36,19 +30,12 @@
 #define PCIE_IATU_NUM		6
 
 struct ls_pcie_drvdata {
-	u32 lut_offset;
-	u32 ltssm_shift;
-	u32 lut_dbg;
 	const struct dw_pcie_host_ops *ops;
-	const struct dw_pcie_ops *dw_pcie_ops;
 };
 
 struct ls_pcie {
 	struct dw_pcie *pci;
-	void __iomem *lut;
-	struct regmap *scfg;
 	const struct ls_pcie_drvdata *drvdata;
-	int index;
 };
 
 #define to_ls_pcie(x)	dev_get_drvdata((x)->dev)
@@ -91,38 +78,6 @@ static void ls_pcie_disable_outbound_atus(struct ls_pcie *pcie)
 		dw_pcie_disable_atu(pcie->pci, i, DW_PCIE_REGION_OUTBOUND);
 }
 
-static int ls1021_pcie_link_up(struct dw_pcie *pci)
-{
-	u32 state;
-	struct ls_pcie *pcie = to_ls_pcie(pci);
-
-	if (!pcie->scfg)
-		return 0;
-
-	regmap_read(pcie->scfg, SCFG_PEXMSCPORTSR(pcie->index), &state);
-	state = (state >> LTSSM_STATE_SHIFT) & LTSSM_STATE_MASK;
-
-	if (state < LTSSM_PCIE_L0)
-		return 0;
-
-	return 1;
-}
-
-static int ls_pcie_link_up(struct dw_pcie *pci)
-{
-	struct ls_pcie *pcie = to_ls_pcie(pci);
-	u32 state;
-
-	state = (ioread32(pcie->lut + pcie->drvdata->lut_dbg) >>
-		 pcie->drvdata->ltssm_shift) &
-		 LTSSM_STATE_MASK;
-
-	if (state < LTSSM_PCIE_L0)
-		return 0;
-
-	return 1;
-}
-
 /* Forward error response of outbound non-posted requests */
 static void ls_pcie_fix_error_response(struct ls_pcie *pcie)
 {
@@ -155,33 +110,6 @@ static int ls_pcie_host_init(struct pcie_port *pp)
 	return 0;
 }
 
-static int ls1021_pcie_host_init(struct pcie_port *pp)
-{
-	struct dw_pcie *pci = to_dw_pcie_from_pp(pp);
-	struct ls_pcie *pcie = to_ls_pcie(pci);
-	struct device *dev = pci->dev;
-	u32 index[2];
-	int ret;
-
-	pcie->scfg = syscon_regmap_lookup_by_phandle(dev->of_node,
-						     "fsl,pcie-scfg");
-	if (IS_ERR(pcie->scfg)) {
-		ret = PTR_ERR(pcie->scfg);
-		dev_err(dev, "No syscfg phandle specified\n");
-		pcie->scfg = NULL;
-		return ret;
-	}
-
-	if (of_property_read_u32_array(dev->of_node,
-				       "fsl,pcie-scfg", index, 2)) {
-		pcie->scfg = NULL;
-		return -EINVAL;
-	}
-	pcie->index = index[1];
-
-	return ls_pcie_host_init(pp);
-}
-
 static int ls_pcie_msi_host_init(struct pcie_port *pp)
 {
 	struct dw_pcie *pci = to_dw_pcie_from_pp(pp);
@@ -205,71 +133,25 @@ static int ls_pcie_msi_host_init(struct pcie_port *pp)
 	return 0;
 }
 
-static const struct dw_pcie_host_ops ls1021_pcie_host_ops = {
-	.host_init = ls1021_pcie_host_init,
-	.msi_host_init = ls_pcie_msi_host_init,
-};
-
 static const struct dw_pcie_host_ops ls_pcie_host_ops = {
 	.host_init = ls_pcie_host_init,
 	.msi_host_init = ls_pcie_msi_host_init,
 };
 
-static const struct dw_pcie_ops dw_ls1021_pcie_ops = {
-	.link_up = ls1021_pcie_link_up,
-};
-
-static const struct dw_pcie_ops dw_ls_pcie_ops = {
-	.link_up = ls_pcie_link_up,
-};
-
-static const struct ls_pcie_drvdata ls1021_drvdata = {
-	.ops = &ls1021_pcie_host_ops,
-	.dw_pcie_ops = &dw_ls1021_pcie_ops,
-};
-
-static const struct ls_pcie_drvdata ls1043_drvdata = {
-	.lut_offset = 0x10000,
-	.ltssm_shift = 24,
-	.lut_dbg = 0x7fc,
-	.ops = &ls_pcie_host_ops,
-	.dw_pcie_ops = &dw_ls_pcie_ops,
-};
-
-static const struct ls_pcie_drvdata ls1046_drvdata = {
-	.lut_offset = 0x80000,
-	.ltssm_shift = 24,
-	.lut_dbg = 0x407fc,
-	.ops = &ls_pcie_host_ops,
-	.dw_pcie_ops = &dw_ls_pcie_ops,
-};
-
-static const struct ls_pcie_drvdata ls2080_drvdata = {
-	.lut_offset = 0x80000,
-	.ltssm_shift = 0,
-	.lut_dbg = 0x7fc,
-	.ops = &ls_pcie_host_ops,
-	.dw_pcie_ops = &dw_ls_pcie_ops,
-};
-
-static const struct ls_pcie_drvdata ls2088_drvdata = {
-	.lut_offset = 0x80000,
-	.ltssm_shift = 0,
-	.lut_dbg = 0x407fc,
+static const struct ls_pcie_drvdata layerscape_drvdata = {
 	.ops = &ls_pcie_host_ops,
-	.dw_pcie_ops = &dw_ls_pcie_ops,
 };
 
 static const struct of_device_id ls_pcie_of_match[] = {
-	{ .compatible = "fsl,ls1012a-pcie", .data = &ls1046_drvdata },
-	{ .compatible = "fsl,ls1021a-pcie", .data = &ls1021_drvdata },
-	{ .compatible = "fsl,ls1028a-pcie", .data = &ls2088_drvdata },
-	{ .compatible = "fsl,ls1043a-pcie", .data = &ls1043_drvdata },
-	{ .compatible = "fsl,ls1046a-pcie", .data = &ls1046_drvdata },
-	{ .compatible = "fsl,ls2080a-pcie", .data = &ls2080_drvdata },
-	{ .compatible = "fsl,ls2085a-pcie", .data = &ls2080_drvdata },
-	{ .compatible = "fsl,ls2088a-pcie", .data = &ls2088_drvdata },
-	{ .compatible = "fsl,ls1088a-pcie", .data = &ls2088_drvdata },
+	{ .compatible = "fsl,ls1012a-pcie", .data = &layerscape_drvdata },
+	{ .compatible = "fsl,ls1021a-pcie", .data = &layerscape_drvdata },
+	{ .compatible = "fsl,ls1028a-pcie", .data = &layerscape_drvdata },
+	{ .compatible = "fsl,ls1043a-pcie", .data = &layerscape_drvdata },
+	{ .compatible = "fsl,ls1046a-pcie", .data = &layerscape_drvdata },
+	{ .compatible = "fsl,ls2080a-pcie", .data = &layerscape_drvdata },
+	{ .compatible = "fsl,ls2085a-pcie", .data = &layerscape_drvdata },
+	{ .compatible = "fsl,ls2088a-pcie", .data = &layerscape_drvdata },
+	{ .compatible = "fsl,ls1088a-pcie", .data = &layerscape_drvdata },
 	{ },
 };
 
@@ -310,7 +192,6 @@ static int __init ls_pcie_probe(struct platform_device *pdev)
 	pcie->drvdata = of_device_get_match_data(dev);
 
 	pci->dev = dev;
-	pci->ops = pcie->drvdata->dw_pcie_ops;
 
 	pcie->pci = pci;
 
@@ -319,8 +200,6 @@ static int __init ls_pcie_probe(struct platform_device *pdev)
 	if (IS_ERR(pci->dbi_base))
 		return PTR_ERR(pci->dbi_base);
 
-	pcie->lut = pci->dbi_base + pcie->drvdata->lut_offset;
-
 	if (!ls_pcie_is_bridge(pcie))
 		return -ENODEV;
 
-- 
2.17.1

Re: [PATCH 2/7] PCI: layerscape: Change to use the DWC common link-up check function

[Rob Herring]

On Mon, Sep 07, 2020 at 01:37:56PM +0800, Zhiqiang Hou wrote:
> From: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
> 
> The current Layerscape PCIe driver directly uses the physical layer
> LTSSM code to check the link-up state, which treats the > L0 states
> as link-up. This is not correct, since there is not explicit map
> between link-up state and LTSSM. So this patch changes to use the
> DWC common link-up check function.
> 
> Signed-off-by: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
> ---
>  drivers/pci/controller/dwc/pci-layerscape.c | 141 ++------------------
>  1 file changed, 10 insertions(+), 131 deletions(-)

IIRC, the common function uses a debug register. I've been wondering do 
the common PCIe config space registers not work on DWC? If you have an 
answer, that would be great for some potential additional cleanups.

Either way,

Reviewed-by: Rob Herring <robh@kernel.org>

RE: [PATCH 2/7] PCI: layerscape: Change to use the DWC common link-up check function

[Z.q. Hou]

Hi Rob,

Thanks a lot for your review!

> -----Original Message-----
> From: Rob Herring <robh@kernel.org>
> Sent: 2020年9月15日 9:20
> To: Z.q. Hou <zhiqiang.hou@nxp.com>
> Cc: linux-pci@vger.kernel.org; devicetree@vger.kernel.org;
> linux-kernel@vger.kernel.org; bhelgaas@google.com;
> shawnguo@kernel.org; Leo Li <leoyang.li@nxp.com>;
> lorenzo.pieralisi@arm.com; gustavo.pimentel@synopsys.com; M.h. Lian
> <minghuan.lian@nxp.com>; Mingkai Hu <mingkai.hu@nxp.com>; Roy Zang
> <roy.zang@nxp.com>
> Subject: Re: [PATCH 2/7] PCI: layerscape: Change to use the DWC common
> link-up check function
> 
> On Mon, Sep 07, 2020 at 01:37:56PM +0800, Zhiqiang Hou wrote:
> > From: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
> >
> > The current Layerscape PCIe driver directly uses the physical layer
> > LTSSM code to check the link-up state, which treats the > L0 states as
> > link-up. This is not correct, since there is not explicit map between
> > link-up state and LTSSM. So this patch changes to use the DWC common
> > link-up check function.
> >
> > Signed-off-by: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
> > ---
> >  drivers/pci/controller/dwc/pci-layerscape.c | 141
> > ++------------------
> >  1 file changed, 10 insertions(+), 131 deletions(-)
> 
> IIRC, the common function uses a debug register. I've been wondering do
> the common PCIe config space registers not work on DWC? If you have an
> answer, that would be great for some potential additional cleanups.
> 

You're right it uses a port debug register, but I'm not sure if the Link status
Register of common PCIe config space works or not on DWC.

Gustavo, can you help answer this query?

Regards,
Zhiqiang

> Either way,
> 
> Reviewed-by: Rob Herring <robh@kernel.org>

[PATCH 3/7] dt-bindings: pci: layerscape-pci: Add a optional property big-endian

[Zhiqiang Hou]

From: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>

This property is to indicate the endianness when accessing the
PEX_LUT and PF register block, so if these registers are
implemented in big-endian, specify this property.

Signed-off-by: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
---
 Documentation/devicetree/bindings/pci/layerscape-pci.txt | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/Documentation/devicetree/bindings/pci/layerscape-pci.txt b/Documentation/devicetree/bindings/pci/layerscape-pci.txt
index 99a386ea691c..2236d3f3089b 100644
--- a/Documentation/devicetree/bindings/pci/layerscape-pci.txt
+++ b/Documentation/devicetree/bindings/pci/layerscape-pci.txt
@@ -37,6 +37,10 @@ Required properties:
   of the data transferred from/to the IP block. This can avoid the software
   cache flush/invalid actions, and improve the performance significantly.
 
+Optional properties:
+- big-endian: If the PEX_LUT and PF register block is in big-endian, specify
+  this property.
+
 Example:
 
 	pcie@3400000 {
-- 
2.17.1

Re: [PATCH 3/7] dt-bindings: pci: layerscape-pci: Add a optional property big-endian

[Rob Herring]

On Mon, 07 Sep 2020 13:37:57 +0800, Zhiqiang Hou wrote:
> From: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
> 
> This property is to indicate the endianness when accessing the
> PEX_LUT and PF register block, so if these registers are
> implemented in big-endian, specify this property.
> 
> Signed-off-by: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
> ---
>  Documentation/devicetree/bindings/pci/layerscape-pci.txt | 4 ++++
>  1 file changed, 4 insertions(+)
> 

Acked-by: Rob Herring <robh@kernel.org>

RE: [PATCH 3/7] dt-bindings: pci: layerscape-pci: Add a optional property big-endian

[Z.q. Hou]

Hi Rob,

Thanks a lot for your review and ack!

Regards,
Zhiqiang

> -----Original Message-----
> From: Rob Herring <robh@kernel.org>
> Sent: 2020年9月15日 9:31
> To: Z.q. Hou <zhiqiang.hou@nxp.com>
> Cc: bhelgaas@google.com; linux-kernel@vger.kernel.org;
> shawnguo@kernel.org; Leo Li <leoyang.li@nxp.com>;
> linux-pci@vger.kernel.org; M.h. Lian <minghuan.lian@nxp.com>;
> robh+dt@kernel.org; gustavo.pimentel@synopsys.com;
> lorenzo.pieralisi@arm.com; Roy Zang <roy.zang@nxp.com>; Mingkai Hu
> <mingkai.hu@nxp.com>; devicetree@vger.kernel.org
> Subject: Re: [PATCH 3/7] dt-bindings: pci: layerscape-pci: Add a optional
> property big-endian
> 
> On Mon, 07 Sep 2020 13:37:57 +0800, Zhiqiang Hou wrote:
> > From: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
> >
> > This property is to indicate the endianness when accessing the PEX_LUT
> > and PF register block, so if these registers are implemented in
> > big-endian, specify this property.
> >
> > Signed-off-by: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
> > ---
> >  Documentation/devicetree/bindings/pci/layerscape-pci.txt | 4 ++++
> >  1 file changed, 4 insertions(+)
> >
> 
> Acked-by: Rob Herring <robh@kernel.org>

[PATCH 4/7] arm64: dts: layerscape: Add big-endian property for PCIe nodes

[Zhiqiang Hou]

From: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>

Add the big-endian property for LS1012A, LS1043A and LS1046A
PCIe devicetree nodes.

Signed-off-by: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
---
 arch/arm64/boot/dts/freescale/fsl-ls1012a.dtsi | 1 +
 arch/arm64/boot/dts/freescale/fsl-ls1043a.dtsi | 3 +++
 arch/arm64/boot/dts/freescale/fsl-ls1046a.dtsi | 3 +++
 3 files changed, 7 insertions(+)

diff --git a/arch/arm64/boot/dts/freescale/fsl-ls1012a.dtsi b/arch/arm64/boot/dts/freescale/fsl-ls1012a.dtsi
index ff19ec415b60..77a0d401c177 100644
--- a/arch/arm64/boot/dts/freescale/fsl-ls1012a.dtsi
+++ b/arch/arm64/boot/dts/freescale/fsl-ls1012a.dtsi
@@ -511,6 +511,7 @@
 					<0000 0 0 2 &gic 0 111 IRQ_TYPE_LEVEL_HIGH>,
 					<0000 0 0 3 &gic 0 112 IRQ_TYPE_LEVEL_HIGH>,
 					<0000 0 0 4 &gic 0 113 IRQ_TYPE_LEVEL_HIGH>;
+			big-endian;
 			status = "disabled";
 		};
 
diff --git a/arch/arm64/boot/dts/freescale/fsl-ls1043a.dtsi b/arch/arm64/boot/dts/freescale/fsl-ls1043a.dtsi
index 5c2e370f6316..70e07612da12 100644
--- a/arch/arm64/boot/dts/freescale/fsl-ls1043a.dtsi
+++ b/arch/arm64/boot/dts/freescale/fsl-ls1043a.dtsi
@@ -837,6 +837,7 @@
 					<0000 0 0 2 &gic 0 111 0x4>,
 					<0000 0 0 3 &gic 0 112 0x4>,
 					<0000 0 0 4 &gic 0 113 0x4>;
+			big-endian;
 			status = "disabled";
 		};
 
@@ -863,6 +864,7 @@
 					<0000 0 0 2 &gic 0 121 0x4>,
 					<0000 0 0 3 &gic 0 122 0x4>,
 					<0000 0 0 4 &gic 0 123 0x4>;
+			big-endian;
 			status = "disabled";
 		};
 
@@ -889,6 +891,7 @@
 					<0000 0 0 2 &gic 0 155 0x4>,
 					<0000 0 0 3 &gic 0 156 0x4>,
 					<0000 0 0 4 &gic 0 157 0x4>;
+			big-endian;
 			status = "disabled";
 		};
 
diff --git a/arch/arm64/boot/dts/freescale/fsl-ls1046a.dtsi b/arch/arm64/boot/dts/freescale/fsl-ls1046a.dtsi
index 0246d975a206..efcbfe8e5e1d 100644
--- a/arch/arm64/boot/dts/freescale/fsl-ls1046a.dtsi
+++ b/arch/arm64/boot/dts/freescale/fsl-ls1046a.dtsi
@@ -741,6 +741,7 @@
 					<0000 0 0 2 &gic GIC_SPI 110 IRQ_TYPE_LEVEL_HIGH>,
 					<0000 0 0 3 &gic GIC_SPI 110 IRQ_TYPE_LEVEL_HIGH>,
 					<0000 0 0 4 &gic GIC_SPI 110 IRQ_TYPE_LEVEL_HIGH>;
+			big-endian;
 			status = "disabled";
 		};
 
@@ -777,6 +778,7 @@
 					<0000 0 0 2 &gic GIC_SPI 120 IRQ_TYPE_LEVEL_HIGH>,
 					<0000 0 0 3 &gic GIC_SPI 120 IRQ_TYPE_LEVEL_HIGH>,
 					<0000 0 0 4 &gic GIC_SPI 120 IRQ_TYPE_LEVEL_HIGH>;
+			big-endian;
 			status = "disabled";
 		};
 
@@ -813,6 +815,7 @@
 					<0000 0 0 2 &gic GIC_SPI 154 IRQ_TYPE_LEVEL_HIGH>,
 					<0000 0 0 3 &gic GIC_SPI 154 IRQ_TYPE_LEVEL_HIGH>,
 					<0000 0 0 4 &gic GIC_SPI 154 IRQ_TYPE_LEVEL_HIGH>;
+			big-endian;
 			status = "disabled";
 		};
 
-- 
2.17.1

[PATCH 5/7] dt-bindings: pci: layerscape-pci: Update the description of SCFG property

[Zhiqiang Hou]

From: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>

Update the description of the second entry of 'fsl,pcie-scfg' property,
as the LS1043A PCIe controller also has some control registers in SCFG
block, while it has 3 controllers.

Signed-off-by: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
---
 Documentation/devicetree/bindings/pci/layerscape-pci.txt | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Documentation/devicetree/bindings/pci/layerscape-pci.txt b/Documentation/devicetree/bindings/pci/layerscape-pci.txt
index 2236d3f3089b..e992ec712bf6 100644
--- a/Documentation/devicetree/bindings/pci/layerscape-pci.txt
+++ b/Documentation/devicetree/bindings/pci/layerscape-pci.txt
@@ -31,7 +31,7 @@ Required properties:
   "intr": The interrupt that is asserted for controller interrupts
 - fsl,pcie-scfg: Must include two entries.
   The first entry must be a link to the SCFG device node
-  The second entry must be '0' or '1' based on physical PCIe controller index.
+  The second entry is the physical PCIe controller index starting from '0'.
   This is used to get SCFG PEXN registers
 - dma-coherent: Indicates that the hardware IP block can ensure the coherency
   of the data transferred from/to the IP block. This can avoid the software
-- 
2.17.1

Re: [PATCH 5/7] dt-bindings: pci: layerscape-pci: Update the description of SCFG property

[Rob Herring]

On Mon, 07 Sep 2020 13:37:59 +0800, Zhiqiang Hou wrote:
> From: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
> 
> Update the description of the second entry of 'fsl,pcie-scfg' property,
> as the LS1043A PCIe controller also has some control registers in SCFG
> block, while it has 3 controllers.
> 
> Signed-off-by: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
> ---
>  Documentation/devicetree/bindings/pci/layerscape-pci.txt | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 

Acked-by: Rob Herring <robh@kernel.org>

RE: [PATCH 5/7] dt-bindings: pci: layerscape-pci: Update the description of SCFG property

[Z.q. Hou]

Hi Rob,

Thanks a lot for your review and ack!

Regards,
Zhiqiang

> -----Original Message-----
> From: Rob Herring <robh@kernel.org>
> Sent: 2020年9月15日 9:31
> To: Z.q. Hou <zhiqiang.hou@nxp.com>
> Cc: linux-pci@vger.kernel.org; linux-kernel@vger.kernel.org; M.h. Lian
> <minghuan.lian@nxp.com>; devicetree@vger.kernel.org;
> gustavo.pimentel@synopsys.com; robh+dt@kernel.org; Mingkai Hu
> <mingkai.hu@nxp.com>; Roy Zang <roy.zang@nxp.com>;
> shawnguo@kernel.org; Leo Li <leoyang.li@nxp.com>;
> bhelgaas@google.com; lorenzo.pieralisi@arm.com
> Subject: Re: [PATCH 5/7] dt-bindings: pci: layerscape-pci: Update the
> description of SCFG property
> 
> On Mon, 07 Sep 2020 13:37:59 +0800, Zhiqiang Hou wrote:
> > From: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
> >
> > Update the description of the second entry of 'fsl,pcie-scfg'
> > property, as the LS1043A PCIe controller also has some control
> > registers in SCFG block, while it has 3 controllers.
> >
> > Signed-off-by: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
> > ---
> >  Documentation/devicetree/bindings/pci/layerscape-pci.txt | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> >
> 
> Acked-by: Rob Herring <robh@kernel.org>

[PATCH 6/7] dts: arm64: ls1043a: Add SCFG phandle for PCIe nodes

[Zhiqiang Hou]

From: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>

The LS1043A PCIe controller has some control registers
in SCFG block, so add the SCFG phandle for each PCIe
controller DT node.

Signed-off-by: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
---
 arch/arm64/boot/dts/freescale/fsl-ls1043a.dtsi | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/arch/arm64/boot/dts/freescale/fsl-ls1043a.dtsi b/arch/arm64/boot/dts/freescale/fsl-ls1043a.dtsi
index 70e07612da12..30ccf1fdb851 100644
--- a/arch/arm64/boot/dts/freescale/fsl-ls1043a.dtsi
+++ b/arch/arm64/boot/dts/freescale/fsl-ls1043a.dtsi
@@ -822,6 +822,7 @@
 			interrupts = <0 118 0x4>, /* controller interrupt */
 				     <0 117 0x4>; /* PME interrupt */
 			interrupt-names = "intr", "pme";
+			fsl,pcie-scfg = <&scfg 0>;
 			#address-cells = <3>;
 			#size-cells = <2>;
 			device_type = "pci";
@@ -849,6 +850,7 @@
 			interrupts = <0 128 0x4>,
 				     <0 127 0x4>;
 			interrupt-names = "intr", "pme";
+			fsl,pcie-scfg = <&scfg 1>;
 			#address-cells = <3>;
 			#size-cells = <2>;
 			device_type = "pci";
@@ -876,6 +878,7 @@
 			interrupts = <0 162 0x4>,
 				     <0 161 0x4>;
 			interrupt-names = "intr", "pme";
+			fsl,pcie-scfg = <&scfg 2>;
 			#address-cells = <3>;
 			#size-cells = <2>;
 			device_type = "pci";
-- 
2.17.1

Re: [PATCH 6/7] dts: arm64: ls1043a: Add SCFG phandle for PCIe nodes

[Shawn Guo]

On Mon, Sep 07, 2020 at 01:38:00PM +0800, Zhiqiang Hou wrote:
> From: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
> 
> The LS1043A PCIe controller has some control registers
> in SCFG block, so add the SCFG phandle for each PCIe
> controller DT node.
> 
> Signed-off-by: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>

'arm64: dts: ...' for subject prefix.

Shawn

RE: [PATCH 6/7] dts: arm64: ls1043a: Add SCFG phandle for PCIe nodes

[Z.q. Hou]

Hi Shawn,

Thanks a lot for your comments!

> -----Original Message-----
> From: Shawn Guo <shawnguo@kernel.org>
> Sent: 2020年9月21日 21:17
> To: Z.q. Hou <zhiqiang.hou@nxp.com>
> Cc: linux-pci@vger.kernel.org; devicetree@vger.kernel.org;
> linux-kernel@vger.kernel.org; bhelgaas@google.com; robh+dt@kernel.org;
> Leo Li <leoyang.li@nxp.com>; lorenzo.pieralisi@arm.com;
> gustavo.pimentel@synopsys.com; M.h. Lian <minghuan.lian@nxp.com>;
> Mingkai Hu <mingkai.hu@nxp.com>; Roy Zang <roy.zang@nxp.com>
> Subject: Re: [PATCH 6/7] dts: arm64: ls1043a: Add SCFG phandle for PCIe
> nodes
> 
> On Mon, Sep 07, 2020 at 01:38:00PM +0800, Zhiqiang Hou wrote:
> > From: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
> >
> > The LS1043A PCIe controller has some control registers in SCFG block,
> > so add the SCFG phandle for each PCIe controller DT node.
> >
> > Signed-off-by: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
> 
> 'arm64: dts: ...' for subject prefix.

Will correct it in next version.

Regards,
Zhiqiang

> 
> Shawn

[PATCH 7/7] PCI: layerscape: Add power management support

[Zhiqiang Hou]

From: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>

Add PME_Turn_Off/PME_TO_Ack handshake sequence, and finally
put the PCIe controller into D3 state after the L2/L3 ready
state transition process completion.

Signed-off-by: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
---
 drivers/pci/controller/dwc/pci-layerscape.c  | 384 ++++++++++++++++++-
 drivers/pci/controller/dwc/pcie-designware.h |   1 +
 2 files changed, 383 insertions(+), 2 deletions(-)

diff --git a/drivers/pci/controller/dwc/pci-layerscape.c b/drivers/pci/controller/dwc/pci-layerscape.c
index be404c16bcbe..ca9ea07f77c1 100644
--- a/drivers/pci/controller/dwc/pci-layerscape.c
+++ b/drivers/pci/controller/dwc/pci-layerscape.c
@@ -3,13 +3,16 @@
  * PCIe host controller driver for Freescale Layerscape SoCs
  *
  * Copyright (C) 2014 Freescale Semiconductor.
+ * Copyright 2020 NXP
  *
  * Author: Minghuan Lian <Minghuan.Lian@freescale.com>
  */
 
+#include <linux/delay.h>
 #include <linux/kernel.h>
 #include <linux/interrupt.h>
 #include <linux/init.h>
+#include <linux/iopoll.h>
 #include <linux/of_pci.h>
 #include <linux/of_platform.h>
 #include <linux/of_irq.h>
@@ -27,17 +30,66 @@
 #define PCIE_ABSERR		0x8d0 /* Bridge Slave Error Response Register */
 #define PCIE_ABSERR_SETTING	0x9401 /* Forward error of non-posted request */
 
+#define PCIE_PM_SCR		0x44
+#define PCIE_PM_SCR_PMEPS_D0	0x0
+#define PCIE_PM_SCR_PMEPS_D3	0x3
+
+#define PCIE_LNKCTL		0x80  /* PCIe link ctrl Register */
+
+/* PF Message Command Register */
+#define LS_PCIE_PF_MCR		0x2c
+#define PF_MCR_PTOMR		BIT(0)
+#define PF_MCR_EXL2S		BIT(1)
+
+/* LS1021A PEXn PM Write Control Register */
+#define SCFG_PEXPMWRCR(idx)	(0x5c + (idx) * 0x64)
+#define PMXMTTURNOFF		BIT(31)
+#define SCFG_PEXSFTRSTCR	0x190
+#define PEXSR(idx)		BIT(idx)
+
+/* LS1043A PEX PME control register */
+#define SCFG_PEXPMECR		0x144
+#define PEXPME(idx)		BIT(31 - (idx) * 4)
+
+/* LS1043A PEX LUT debug register */
+#define LS_PCIE_LDBG	0x7fc
+#define LDBG_SR		BIT(30)
+#define LDBG_WE		BIT(31)
+
 #define PCIE_IATU_NUM		6
 
+#define LS_PCIE_IS_L2(v)	\
+	(((v) & PORT_LOGIC_LTSSM_STATE_MASK) == PORT_LOGIC_LTSSM_STATE_L2)
+
+struct ls_pcie;
+
+struct ls_pcie_host_pm_ops {
+	int (*pm_init)(struct ls_pcie *pcie);
+	void (*send_turn_off_message)(struct ls_pcie *pcie);
+	void (*exit_from_l2)(struct ls_pcie *pcie);
+};
+
 struct ls_pcie_drvdata {
+	const u32 pf_off;
+	const u32 lut_off;
 	const struct dw_pcie_host_ops *ops;
+	const struct ls_pcie_host_pm_ops *pm_ops;
 };
 
 struct ls_pcie {
 	struct dw_pcie *pci;
 	const struct ls_pcie_drvdata *drvdata;
+	void __iomem *pf_base;
+	void __iomem *lut_base;
+	bool big_endian;
+	bool ep_presence;
+	bool pm_support;
+	struct regmap *scfg;
+	int index;
 };
 
+#define ls_pcie_lut_readl_addr(addr)	ls_pcie_lut_readl(pcie, addr)
+#define ls_pcie_pf_readl_addr(addr)	ls_pcie_pf_readl(pcie, addr)
 #define to_ls_pcie(x)	dev_get_drvdata((x)->dev)
 
 static bool ls_pcie_is_bridge(struct ls_pcie *pcie)
@@ -86,6 +138,208 @@ static void ls_pcie_fix_error_response(struct ls_pcie *pcie)
 	iowrite32(PCIE_ABSERR_SETTING, pci->dbi_base + PCIE_ABSERR);
 }
 
+static u32 ls_pcie_lut_readl(struct ls_pcie *pcie, u32 off)
+{
+	if (pcie->big_endian)
+		return ioread32be(pcie->lut_base + off);
+
+	return ioread32(pcie->lut_base + off);
+}
+
+static void ls_pcie_lut_writel(struct ls_pcie *pcie, u32 off, u32 val)
+{
+	if (pcie->big_endian)
+		return iowrite32be(val, pcie->lut_base + off);
+
+	return iowrite32(val, pcie->lut_base + off);
+
+}
+
+static u32 ls_pcie_pf_readl(struct ls_pcie *pcie, u32 off)
+{
+	if (pcie->big_endian)
+		return ioread32be(pcie->pf_base + off);
+
+	return ioread32(pcie->pf_base + off);
+}
+
+static void ls_pcie_pf_writel(struct ls_pcie *pcie, u32 off, u32 val)
+{
+	if (pcie->big_endian)
+		return iowrite32be(val, pcie->pf_base + off);
+
+	return iowrite32(val, pcie->pf_base + off);
+
+}
+
+static void ls_pcie_send_turnoff_msg(struct ls_pcie *pcie)
+{
+	u32 val;
+	int ret;
+
+	val = ls_pcie_pf_readl(pcie, LS_PCIE_PF_MCR);
+	val |= PF_MCR_PTOMR;
+	ls_pcie_pf_writel(pcie, LS_PCIE_PF_MCR, val);
+
+	ret = readx_poll_timeout(ls_pcie_pf_readl_addr, LS_PCIE_PF_MCR,
+				 val, !(val & PF_MCR_PTOMR), 100, 10000);
+	if (ret)
+		dev_info(pcie->pci->dev, "poll turn off message timeout\n");
+}
+
+static void ls1021a_pcie_send_turnoff_msg(struct ls_pcie *pcie)
+{
+	u32 val;
+
+	if (!pcie->scfg) {
+		dev_dbg(pcie->pci->dev, "SYSCFG is NULL\n");
+		return;
+	}
+
+	/* Send Turn_off message */
+	regmap_read(pcie->scfg, SCFG_PEXPMWRCR(pcie->index), &val);
+	val |= PMXMTTURNOFF;
+	regmap_write(pcie->scfg, SCFG_PEXPMWRCR(pcie->index), val);
+
+	mdelay(10);
+
+	/* Clear Turn_off message */
+	regmap_read(pcie->scfg, SCFG_PEXPMWRCR(pcie->index), &val);
+	val &= ~PMXMTTURNOFF;
+	regmap_write(pcie->scfg, SCFG_PEXPMWRCR(pcie->index), val);
+}
+
+static void ls1043a_pcie_send_turnoff_msg(struct ls_pcie *pcie)
+{
+	u32 val;
+
+	if (!pcie->scfg) {
+		dev_dbg(pcie->pci->dev, "SYSCFG is NULL\n");
+		return;
+	}
+
+	/* Send Turn_off message */
+	regmap_read(pcie->scfg, SCFG_PEXPMECR, &val);
+	val |= PEXPME(pcie->index);
+	regmap_write(pcie->scfg, SCFG_PEXPMECR, val);
+
+	mdelay(10);
+
+	/* Clear Turn_off message */
+	regmap_read(pcie->scfg, SCFG_PEXPMECR, &val);
+	val &= ~PEXPME(pcie->index);
+	regmap_write(pcie->scfg, SCFG_PEXPMECR, val);
+}
+
+static void ls_pcie_exit_from_l2(struct ls_pcie *pcie)
+{
+	u32 val;
+	int ret;
+
+	val = ls_pcie_pf_readl(pcie, LS_PCIE_PF_MCR);
+	val |= PF_MCR_EXL2S;
+	ls_pcie_pf_writel(pcie, LS_PCIE_PF_MCR, val);
+
+	ret = readx_poll_timeout(ls_pcie_pf_readl_addr, LS_PCIE_PF_MCR,
+				 val, !(val & PF_MCR_EXL2S), 100, 10000);
+	if (ret)
+		dev_info(pcie->pci->dev, "poll exit L2 state timeout\n");
+}
+
+static void ls_pcie_retrain_link(struct ls_pcie *pcie)
+{
+	struct dw_pcie *pci = pcie->pci;
+	u32 val;
+
+	val = dw_pcie_readw_dbi(pci, PCIE_LNKCTL);
+	val |= PCI_EXP_LNKCTL_RL;
+	dw_pcie_writew_dbi(pci, PCIE_LNKCTL, val);
+}
+
+static void ls1021a_pcie_exit_from_l2(struct ls_pcie *pcie)
+{
+	u32 val;
+
+	regmap_read(pcie->scfg, SCFG_PEXSFTRSTCR, &val);
+	val |= PEXSR(pcie->index);
+	regmap_write(pcie->scfg, SCFG_PEXSFTRSTCR, val);
+
+	regmap_read(pcie->scfg, SCFG_PEXSFTRSTCR, &val);
+	val &= ~PEXSR(pcie->index);
+	regmap_write(pcie->scfg, SCFG_PEXSFTRSTCR, val);
+
+	mdelay(1);
+
+	ls_pcie_retrain_link(pcie);
+}
+static void ls1043a_pcie_exit_from_l2(struct ls_pcie *pcie)
+{
+	u32 val;
+
+	val = ls_pcie_lut_readl(pcie, LS_PCIE_LDBG);
+	val |= LDBG_WE;
+	ls_pcie_lut_writel(pcie, LS_PCIE_LDBG, val);
+
+	val = ls_pcie_lut_readl(pcie, LS_PCIE_LDBG);
+	val |= LDBG_SR;
+	ls_pcie_lut_writel(pcie, LS_PCIE_LDBG, val);
+
+	val = ls_pcie_lut_readl(pcie, LS_PCIE_LDBG);
+	val &= ~LDBG_SR;
+	ls_pcie_lut_writel(pcie, LS_PCIE_LDBG, val);
+
+	val = ls_pcie_lut_readl(pcie, LS_PCIE_LDBG);
+	val &= ~LDBG_WE;
+	ls_pcie_lut_writel(pcie, LS_PCIE_LDBG, val);
+
+	mdelay(1);
+
+	ls_pcie_retrain_link(pcie);
+}
+
+static int ls1021a_pcie_pm_init(struct ls_pcie *pcie)
+{
+	struct device *dev = pcie->pci->dev;
+	u32 index[2];
+	int ret;
+
+	pcie->scfg = syscon_regmap_lookup_by_phandle(dev->of_node,
+						     "fsl,pcie-scfg");
+	if (IS_ERR(pcie->scfg)) {
+		ret = PTR_ERR(pcie->scfg);
+		dev_err(dev, "No syscfg phandle specified\n");
+		pcie->scfg = NULL;
+		return ret;
+	}
+
+	ret = of_property_read_u32_array(dev->of_node, "fsl,pcie-scfg",
+					 index, 2);
+	if (ret) {
+		pcie->scfg = NULL;
+		return ret;
+	}
+
+	pcie->index = index[1];
+
+	return 0;
+}
+
+static int ls_pcie_pm_init(struct ls_pcie *pcie)
+{
+	return 0;
+}
+
+static void ls_pcie_set_dstate(struct ls_pcie *pcie, u32 dstate)
+{
+	struct dw_pcie *pci = pcie->pci;
+	u32 val;
+
+	val = dw_pcie_readw_dbi(pci, PCIE_PM_SCR);
+	val &= ~PCI_PM_CTRL_STATE_MASK;
+	val |= dstate;
+	dw_pcie_writew_dbi(pci, PCIE_PM_SCR, val);
+}
+
 static int ls_pcie_host_init(struct pcie_port *pp)
 {
 	struct dw_pcie *pci = to_dw_pcie_from_pp(pp);
@@ -133,20 +387,52 @@ static int ls_pcie_msi_host_init(struct pcie_port *pp)
 	return 0;
 }
 
+static struct ls_pcie_host_pm_ops ls1021a_pcie_host_pm_ops = {
+	.pm_init = &ls1021a_pcie_pm_init,
+	.send_turn_off_message = &ls1021a_pcie_send_turnoff_msg,
+	.exit_from_l2 = &ls1021a_pcie_exit_from_l2,
+};
+
+static struct ls_pcie_host_pm_ops ls1043a_pcie_host_pm_ops = {
+	.pm_init = &ls1021a_pcie_pm_init,
+	.send_turn_off_message = &ls1043a_pcie_send_turnoff_msg,
+	.exit_from_l2 = &ls1043a_pcie_exit_from_l2,
+};
+
+static struct ls_pcie_host_pm_ops ls_pcie_host_pm_ops = {
+	.pm_init = &ls_pcie_pm_init,
+	.send_turn_off_message = &ls_pcie_send_turnoff_msg,
+	.exit_from_l2 = &ls_pcie_exit_from_l2,
+};
+
 static const struct dw_pcie_host_ops ls_pcie_host_ops = {
 	.host_init = ls_pcie_host_init,
 	.msi_host_init = ls_pcie_msi_host_init,
 };
 
+static const struct ls_pcie_drvdata ls1021a_drvdata = {
+	.ops = &ls_pcie_host_ops,
+	.pm_ops = &ls1021a_pcie_host_pm_ops,
+};
+
+static const struct ls_pcie_drvdata ls1043a_drvdata = {
+	.ops = &ls_pcie_host_ops,
+	.lut_off = 0x10000,
+	.pm_ops = &ls1043a_pcie_host_pm_ops,
+};
+
 static const struct ls_pcie_drvdata layerscape_drvdata = {
 	.ops = &ls_pcie_host_ops,
+	.lut_off = 0x80000,
+	.pf_off = 0xc0000,
+	.pm_ops = &ls_pcie_host_pm_ops,
 };
 
 static const struct of_device_id ls_pcie_of_match[] = {
 	{ .compatible = "fsl,ls1012a-pcie", .data = &layerscape_drvdata },
-	{ .compatible = "fsl,ls1021a-pcie", .data = &layerscape_drvdata },
+	{ .compatible = "fsl,ls1021a-pcie", .data = &ls1021a_drvdata },
 	{ .compatible = "fsl,ls1028a-pcie", .data = &layerscape_drvdata },
-	{ .compatible = "fsl,ls1043a-pcie", .data = &layerscape_drvdata },
+	{ .compatible = "fsl,ls1043a-pcie", .data = &ls1043a_drvdata },
 	{ .compatible = "fsl,ls1046a-pcie", .data = &layerscape_drvdata },
 	{ .compatible = "fsl,ls2080a-pcie", .data = &layerscape_drvdata },
 	{ .compatible = "fsl,ls2085a-pcie", .data = &layerscape_drvdata },
@@ -170,6 +456,15 @@ static int __init ls_add_pcie_port(struct ls_pcie *pcie)
 		return ret;
 	}
 
+	if (dw_pcie_link_up(pci)) {
+		dev_dbg(pci->dev, "Endpoint is present\n");
+		pcie->ep_presence = true;
+	}
+
+	if (pcie->drvdata->pm_ops && pcie->drvdata->pm_ops->pm_init &&
+	    !pcie->drvdata->pm_ops->pm_init(pcie))
+		pcie->pm_support = true;
+
 	return 0;
 }
 
@@ -200,6 +495,14 @@ static int __init ls_pcie_probe(struct platform_device *pdev)
 	if (IS_ERR(pci->dbi_base))
 		return PTR_ERR(pci->dbi_base);
 
+	pcie->big_endian = of_property_read_bool(dev->of_node, "big-endian");
+
+	if (pcie->drvdata->lut_off)
+		pcie->lut_base = pci->dbi_base + pcie->drvdata->lut_off;
+
+	if (pcie->drvdata->pf_off)
+		pcie->pf_base = pci->dbi_base + pcie->drvdata->pf_off;
+
 	if (!ls_pcie_is_bridge(pcie))
 		return -ENODEV;
 
@@ -212,11 +515,88 @@ static int __init ls_pcie_probe(struct platform_device *pdev)
 	return 0;
 }
 
+static bool ls_pcie_pm_check(struct ls_pcie *pcie)
+{
+	if (!pcie->ep_presence) {
+		dev_dbg(pcie->pci->dev, "Endpoint isn't present\n");
+		return false;
+	}
+
+	if (!pcie->pm_support)
+		return false;
+
+	return true;
+}
+
+#ifdef CONFIG_PM_SLEEP
+static int ls_pcie_suspend_noirq(struct device *dev)
+{
+	struct ls_pcie *pcie = dev_get_drvdata(dev);
+	struct dw_pcie *pci = pcie->pci;
+	u32 val;
+	int ret;
+
+	if (!ls_pcie_pm_check(pcie))
+		return 0;
+
+	pcie->drvdata->pm_ops->send_turn_off_message(pcie);
+
+	/* 10ms timeout to check L2 ready */
+	ret = readl_poll_timeout(pci->dbi_base + PCIE_PORT_DEBUG0,
+				 val, LS_PCIE_IS_L2(val), 100, 10000);
+	if (ret) {
+		dev_err(dev, "PCIe link enter L2 timeout! ltssm = 0x%x\n", val);
+		return ret;
+	}
+
+	ls_pcie_set_dstate(pcie, PCIE_PM_SCR_PMEPS_D3);
+
+	return 0;
+}
+
+static int ls_pcie_resume_noirq(struct device *dev)
+{
+	struct ls_pcie *pcie = dev_get_drvdata(dev);
+	struct dw_pcie *pci = pcie->pci;
+	int ret;
+
+	if (!ls_pcie_pm_check(pcie))
+		return 0;
+
+	ls_pcie_set_dstate(pcie, PCIE_PM_SCR_PMEPS_D0);
+
+	pcie->drvdata->pm_ops->exit_from_l2(pcie);
+
+	/* delay 10ms to access EP */
+	mdelay(10);
+
+	ret = ls_pcie_host_init(&pci->pp);
+	if (ret) {
+		dev_err(dev, "ls_pcie_host_init failed! ret = 0x%x\n", ret);
+		return ret;
+	}
+
+	ret = dw_pcie_wait_for_link(pci);
+	if (ret) {
+		dev_err(dev, "wait link up timeout! ret = 0x%x\n", ret);
+		return ret;
+	}
+
+	return 0;
+}
+#endif /* CONFIG_PM_SLEEP */
+
+static const struct dev_pm_ops ls_pcie_pm_ops = {
+	SET_NOIRQ_SYSTEM_SLEEP_PM_OPS(ls_pcie_suspend_noirq,
+				      ls_pcie_resume_noirq)
+};
+
 static struct platform_driver ls_pcie_driver = {
 	.driver = {
 		.name = "layerscape-pcie",
 		.of_match_table = ls_pcie_of_match,
 		.suppress_bind_attrs = true,
+		.pm = &ls_pcie_pm_ops,
 	},
 };
 builtin_platform_driver_probe(ls_pcie_driver, ls_pcie_probe);
diff --git a/drivers/pci/controller/dwc/pcie-designware.h b/drivers/pci/controller/dwc/pcie-designware.h
index f911760dcc69..1f2fd03d1eba 100644
--- a/drivers/pci/controller/dwc/pcie-designware.h
+++ b/drivers/pci/controller/dwc/pcie-designware.h
@@ -46,6 +46,7 @@
 #define PCIE_PORT_DEBUG0		0x728
 #define PORT_LOGIC_LTSSM_STATE_MASK	0x1f
 #define PORT_LOGIC_LTSSM_STATE_L0	0x11
+#define PORT_LOGIC_LTSSM_STATE_L2	0x15
 #define PCIE_PORT_DEBUG1		0x72C
 #define PCIE_PORT_DEBUG1_LINK_UP		BIT(4)
 #define PCIE_PORT_DEBUG1_LINK_IN_TRAINING	BIT(29)
-- 
2.17.1

Re: [PATCH 7/7] PCI: layerscape: Add power management support

[Rob Herring]

On Mon, Sep 07, 2020 at 01:38:01PM +0800, Zhiqiang Hou wrote:
> From: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
> 
> Add PME_Turn_Off/PME_TO_Ack handshake sequence, and finally
> put the PCIe controller into D3 state after the L2/L3 ready
> state transition process completion.
> 
> Signed-off-by: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
> ---
>  drivers/pci/controller/dwc/pci-layerscape.c  | 384 ++++++++++++++++++-
>  drivers/pci/controller/dwc/pcie-designware.h |   1 +
>  2 files changed, 383 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/pci/controller/dwc/pci-layerscape.c b/drivers/pci/controller/dwc/pci-layerscape.c
> index be404c16bcbe..ca9ea07f77c1 100644
> --- a/drivers/pci/controller/dwc/pci-layerscape.c
> +++ b/drivers/pci/controller/dwc/pci-layerscape.c
> @@ -3,13 +3,16 @@
>   * PCIe host controller driver for Freescale Layerscape SoCs
>   *
>   * Copyright (C) 2014 Freescale Semiconductor.
> + * Copyright 2020 NXP
>   *
>   * Author: Minghuan Lian <Minghuan.Lian@freescale.com>
>   */
>  
> +#include <linux/delay.h>
>  #include <linux/kernel.h>
>  #include <linux/interrupt.h>
>  #include <linux/init.h>
> +#include <linux/iopoll.h>
>  #include <linux/of_pci.h>
>  #include <linux/of_platform.h>
>  #include <linux/of_irq.h>
> @@ -27,17 +30,66 @@
>  #define PCIE_ABSERR		0x8d0 /* Bridge Slave Error Response Register */
>  #define PCIE_ABSERR_SETTING	0x9401 /* Forward error of non-posted request */
>  
> +#define PCIE_PM_SCR		0x44
> +#define PCIE_PM_SCR_PMEPS_D0	0x0
> +#define PCIE_PM_SCR_PMEPS_D3	0x3
> +
> +#define PCIE_LNKCTL		0x80  /* PCIe link ctrl Register */

Use the common defines.


> +
> +/* PF Message Command Register */
> +#define LS_PCIE_PF_MCR		0x2c
> +#define PF_MCR_PTOMR		BIT(0)
> +#define PF_MCR_EXL2S		BIT(1)
> +
> +/* LS1021A PEXn PM Write Control Register */
> +#define SCFG_PEXPMWRCR(idx)	(0x5c + (idx) * 0x64)
> +#define PMXMTTURNOFF		BIT(31)
> +#define SCFG_PEXSFTRSTCR	0x190
> +#define PEXSR(idx)		BIT(idx)
> +
> +/* LS1043A PEX PME control register */
> +#define SCFG_PEXPMECR		0x144
> +#define PEXPME(idx)		BIT(31 - (idx) * 4)
> +
> +/* LS1043A PEX LUT debug register */
> +#define LS_PCIE_LDBG	0x7fc
> +#define LDBG_SR		BIT(30)
> +#define LDBG_WE		BIT(31)
> +
>  #define PCIE_IATU_NUM		6
>  
> +#define LS_PCIE_IS_L2(v)	\
> +	(((v) & PORT_LOGIC_LTSSM_STATE_MASK) == PORT_LOGIC_LTSSM_STATE_L2)
> +
> +struct ls_pcie;
> +
> +struct ls_pcie_host_pm_ops {
> +	int (*pm_init)(struct ls_pcie *pcie);
> +	void (*send_turn_off_message)(struct ls_pcie *pcie);
> +	void (*exit_from_l2)(struct ls_pcie *pcie);
> +};
> +
>  struct ls_pcie_drvdata {
> +	const u32 pf_off;
> +	const u32 lut_off;
>  	const struct dw_pcie_host_ops *ops;
> +	const struct ls_pcie_host_pm_ops *pm_ops;
>  };
>  
>  struct ls_pcie {
>  	struct dw_pcie *pci;
>  	const struct ls_pcie_drvdata *drvdata;
> +	void __iomem *pf_base;
> +	void __iomem *lut_base;
> +	bool big_endian;
> +	bool ep_presence;
> +	bool pm_support;
> +	struct regmap *scfg;
> +	int index;
>  };
>  
> +#define ls_pcie_lut_readl_addr(addr)	ls_pcie_lut_readl(pcie, addr)
> +#define ls_pcie_pf_readl_addr(addr)	ls_pcie_pf_readl(pcie, addr)
>  #define to_ls_pcie(x)	dev_get_drvdata((x)->dev)
>  
>  static bool ls_pcie_is_bridge(struct ls_pcie *pcie)
> @@ -86,6 +138,208 @@ static void ls_pcie_fix_error_response(struct ls_pcie *pcie)
>  	iowrite32(PCIE_ABSERR_SETTING, pci->dbi_base + PCIE_ABSERR);
>  }
>  
> +static u32 ls_pcie_lut_readl(struct ls_pcie *pcie, u32 off)
> +{
> +	if (pcie->big_endian)
> +		return ioread32be(pcie->lut_base + off);
> +
> +	return ioread32(pcie->lut_base + off);
> +}
> +
> +static void ls_pcie_lut_writel(struct ls_pcie *pcie, u32 off, u32 val)
> +{
> +	if (pcie->big_endian)
> +		return iowrite32be(val, pcie->lut_base + off);
> +
> +	return iowrite32(val, pcie->lut_base + off);
> +
> +}
> +
> +static u32 ls_pcie_pf_readl(struct ls_pcie *pcie, u32 off)
> +{
> +	if (pcie->big_endian)
> +		return ioread32be(pcie->pf_base + off);
> +
> +	return ioread32(pcie->pf_base + off);
> +}
> +
> +static void ls_pcie_pf_writel(struct ls_pcie *pcie, u32 off, u32 val)
> +{
> +	if (pcie->big_endian)
> +		return iowrite32be(val, pcie->pf_base + off);
> +
> +	return iowrite32(val, pcie->pf_base + off);
> +
> +}
> +
> +static void ls_pcie_send_turnoff_msg(struct ls_pcie *pcie)
> +{
> +	u32 val;
> +	int ret;
> +
> +	val = ls_pcie_pf_readl(pcie, LS_PCIE_PF_MCR);
> +	val |= PF_MCR_PTOMR;
> +	ls_pcie_pf_writel(pcie, LS_PCIE_PF_MCR, val);
> +
> +	ret = readx_poll_timeout(ls_pcie_pf_readl_addr, LS_PCIE_PF_MCR,
> +				 val, !(val & PF_MCR_PTOMR), 100, 10000);
> +	if (ret)
> +		dev_info(pcie->pci->dev, "poll turn off message timeout\n");
> +}
> +
> +static void ls1021a_pcie_send_turnoff_msg(struct ls_pcie *pcie)
> +{
> +	u32 val;
> +
> +	if (!pcie->scfg) {
> +		dev_dbg(pcie->pci->dev, "SYSCFG is NULL\n");
> +		return;
> +	}
> +
> +	/* Send Turn_off message */
> +	regmap_read(pcie->scfg, SCFG_PEXPMWRCR(pcie->index), &val);
> +	val |= PMXMTTURNOFF;
> +	regmap_write(pcie->scfg, SCFG_PEXPMWRCR(pcie->index), val);
> +
> +	mdelay(10);
> +
> +	/* Clear Turn_off message */
> +	regmap_read(pcie->scfg, SCFG_PEXPMWRCR(pcie->index), &val);
> +	val &= ~PMXMTTURNOFF;
> +	regmap_write(pcie->scfg, SCFG_PEXPMWRCR(pcie->index), val);
> +}
> +
> +static void ls1043a_pcie_send_turnoff_msg(struct ls_pcie *pcie)
> +{
> +	u32 val;
> +
> +	if (!pcie->scfg) {
> +		dev_dbg(pcie->pci->dev, "SYSCFG is NULL\n");
> +		return;
> +	}
> +
> +	/* Send Turn_off message */
> +	regmap_read(pcie->scfg, SCFG_PEXPMECR, &val);
> +	val |= PEXPME(pcie->index);
> +	regmap_write(pcie->scfg, SCFG_PEXPMECR, val);
> +
> +	mdelay(10);
> +
> +	/* Clear Turn_off message */
> +	regmap_read(pcie->scfg, SCFG_PEXPMECR, &val);
> +	val &= ~PEXPME(pcie->index);
> +	regmap_write(pcie->scfg, SCFG_PEXPMECR, val);
> +}
> +
> +static void ls_pcie_exit_from_l2(struct ls_pcie *pcie)
> +{
> +	u32 val;
> +	int ret;
> +
> +	val = ls_pcie_pf_readl(pcie, LS_PCIE_PF_MCR);
> +	val |= PF_MCR_EXL2S;
> +	ls_pcie_pf_writel(pcie, LS_PCIE_PF_MCR, val);
> +
> +	ret = readx_poll_timeout(ls_pcie_pf_readl_addr, LS_PCIE_PF_MCR,
> +				 val, !(val & PF_MCR_EXL2S), 100, 10000);
> +	if (ret)
> +		dev_info(pcie->pci->dev, "poll exit L2 state timeout\n");
> +}
> +
> +static void ls_pcie_retrain_link(struct ls_pcie *pcie)
> +{
> +	struct dw_pcie *pci = pcie->pci;
> +	u32 val;
> +
> +	val = dw_pcie_readw_dbi(pci, PCIE_LNKCTL);
> +	val |= PCI_EXP_LNKCTL_RL;
> +	dw_pcie_writew_dbi(pci, PCIE_LNKCTL, val);
> +}
> +
> +static void ls1021a_pcie_exit_from_l2(struct ls_pcie *pcie)
> +{
> +	u32 val;
> +
> +	regmap_read(pcie->scfg, SCFG_PEXSFTRSTCR, &val);
> +	val |= PEXSR(pcie->index);
> +	regmap_write(pcie->scfg, SCFG_PEXSFTRSTCR, val);
> +
> +	regmap_read(pcie->scfg, SCFG_PEXSFTRSTCR, &val);
> +	val &= ~PEXSR(pcie->index);
> +	regmap_write(pcie->scfg, SCFG_PEXSFTRSTCR, val);
> +
> +	mdelay(1);
> +
> +	ls_pcie_retrain_link(pcie);
> +}
> +static void ls1043a_pcie_exit_from_l2(struct ls_pcie *pcie)
> +{
> +	u32 val;
> +
> +	val = ls_pcie_lut_readl(pcie, LS_PCIE_LDBG);
> +	val |= LDBG_WE;
> +	ls_pcie_lut_writel(pcie, LS_PCIE_LDBG, val);
> +
> +	val = ls_pcie_lut_readl(pcie, LS_PCIE_LDBG);
> +	val |= LDBG_SR;
> +	ls_pcie_lut_writel(pcie, LS_PCIE_LDBG, val);
> +
> +	val = ls_pcie_lut_readl(pcie, LS_PCIE_LDBG);
> +	val &= ~LDBG_SR;
> +	ls_pcie_lut_writel(pcie, LS_PCIE_LDBG, val);
> +
> +	val = ls_pcie_lut_readl(pcie, LS_PCIE_LDBG);
> +	val &= ~LDBG_WE;
> +	ls_pcie_lut_writel(pcie, LS_PCIE_LDBG, val);
> +
> +	mdelay(1);
> +
> +	ls_pcie_retrain_link(pcie);
> +}
> +
> +static int ls1021a_pcie_pm_init(struct ls_pcie *pcie)
> +{
> +	struct device *dev = pcie->pci->dev;
> +	u32 index[2];
> +	int ret;
> +
> +	pcie->scfg = syscon_regmap_lookup_by_phandle(dev->of_node,
> +						     "fsl,pcie-scfg");
> +	if (IS_ERR(pcie->scfg)) {
> +		ret = PTR_ERR(pcie->scfg);
> +		dev_err(dev, "No syscfg phandle specified\n");
> +		pcie->scfg = NULL;
> +		return ret;
> +	}
> +
> +	ret = of_property_read_u32_array(dev->of_node, "fsl,pcie-scfg",
> +					 index, 2);
> +	if (ret) {
> +		pcie->scfg = NULL;
> +		return ret;
> +	}
> +
> +	pcie->index = index[1];
> +
> +	return 0;
> +}
> +
> +static int ls_pcie_pm_init(struct ls_pcie *pcie)
> +{
> +	return 0;
> +}
> +
> +static void ls_pcie_set_dstate(struct ls_pcie *pcie, u32 dstate)
> +{
> +	struct dw_pcie *pci = pcie->pci;
> +	u32 val;
> +
> +	val = dw_pcie_readw_dbi(pci, PCIE_PM_SCR);
> +	val &= ~PCI_PM_CTRL_STATE_MASK;
> +	val |= dstate;
> +	dw_pcie_writew_dbi(pci, PCIE_PM_SCR, val);
> +}
> +
>  static int ls_pcie_host_init(struct pcie_port *pp)
>  {
>  	struct dw_pcie *pci = to_dw_pcie_from_pp(pp);
> @@ -133,20 +387,52 @@ static int ls_pcie_msi_host_init(struct pcie_port *pp)
>  	return 0;
>  }
>  
> +static struct ls_pcie_host_pm_ops ls1021a_pcie_host_pm_ops = {
> +	.pm_init = &ls1021a_pcie_pm_init,
> +	.send_turn_off_message = &ls1021a_pcie_send_turnoff_msg,
> +	.exit_from_l2 = &ls1021a_pcie_exit_from_l2,
> +};
> +
> +static struct ls_pcie_host_pm_ops ls1043a_pcie_host_pm_ops = {
> +	.pm_init = &ls1021a_pcie_pm_init,
> +	.send_turn_off_message = &ls1043a_pcie_send_turnoff_msg,
> +	.exit_from_l2 = &ls1043a_pcie_exit_from_l2,
> +};
> +
> +static struct ls_pcie_host_pm_ops ls_pcie_host_pm_ops = {
> +	.pm_init = &ls_pcie_pm_init,
> +	.send_turn_off_message = &ls_pcie_send_turnoff_msg,
> +	.exit_from_l2 = &ls_pcie_exit_from_l2,
> +};
> +
>  static const struct dw_pcie_host_ops ls_pcie_host_ops = {
>  	.host_init = ls_pcie_host_init,
>  	.msi_host_init = ls_pcie_msi_host_init,
>  };
>  
> +static const struct ls_pcie_drvdata ls1021a_drvdata = {
> +	.ops = &ls_pcie_host_ops,
> +	.pm_ops = &ls1021a_pcie_host_pm_ops,
> +};
> +
> +static const struct ls_pcie_drvdata ls1043a_drvdata = {
> +	.ops = &ls_pcie_host_ops,
> +	.lut_off = 0x10000,
> +	.pm_ops = &ls1043a_pcie_host_pm_ops,
> +};
> +
>  static const struct ls_pcie_drvdata layerscape_drvdata = {
>  	.ops = &ls_pcie_host_ops,
> +	.lut_off = 0x80000,
> +	.pf_off = 0xc0000,
> +	.pm_ops = &ls_pcie_host_pm_ops,
>  };
>  
>  static const struct of_device_id ls_pcie_of_match[] = {
>  	{ .compatible = "fsl,ls1012a-pcie", .data = &layerscape_drvdata },
> -	{ .compatible = "fsl,ls1021a-pcie", .data = &layerscape_drvdata },
> +	{ .compatible = "fsl,ls1021a-pcie", .data = &ls1021a_drvdata },
>  	{ .compatible = "fsl,ls1028a-pcie", .data = &layerscape_drvdata },
> -	{ .compatible = "fsl,ls1043a-pcie", .data = &layerscape_drvdata },
> +	{ .compatible = "fsl,ls1043a-pcie", .data = &ls1043a_drvdata },
>  	{ .compatible = "fsl,ls1046a-pcie", .data = &layerscape_drvdata },
>  	{ .compatible = "fsl,ls2080a-pcie", .data = &layerscape_drvdata },
>  	{ .compatible = "fsl,ls2085a-pcie", .data = &layerscape_drvdata },
> @@ -170,6 +456,15 @@ static int __init ls_add_pcie_port(struct ls_pcie *pcie)
>  		return ret;
>  	}
>  
> +	if (dw_pcie_link_up(pci)) {
> +		dev_dbg(pci->dev, "Endpoint is present\n");
> +		pcie->ep_presence = true;
> +	}
> +
> +	if (pcie->drvdata->pm_ops && pcie->drvdata->pm_ops->pm_init &&
> +	    !pcie->drvdata->pm_ops->pm_init(pcie))
> +		pcie->pm_support = true;
> +
>  	return 0;
>  }
>  
> @@ -200,6 +495,14 @@ static int __init ls_pcie_probe(struct platform_device *pdev)
>  	if (IS_ERR(pci->dbi_base))
>  		return PTR_ERR(pci->dbi_base);
>  
> +	pcie->big_endian = of_property_read_bool(dev->of_node, "big-endian");
> +
> +	if (pcie->drvdata->lut_off)
> +		pcie->lut_base = pci->dbi_base + pcie->drvdata->lut_off;
> +
> +	if (pcie->drvdata->pf_off)
> +		pcie->pf_base = pci->dbi_base + pcie->drvdata->pf_off;
> +
>  	if (!ls_pcie_is_bridge(pcie))
>  		return -ENODEV;
>  
> @@ -212,11 +515,88 @@ static int __init ls_pcie_probe(struct platform_device *pdev)
>  	return 0;
>  }
>  
> +static bool ls_pcie_pm_check(struct ls_pcie *pcie)
> +{
> +	if (!pcie->ep_presence) {
> +		dev_dbg(pcie->pci->dev, "Endpoint isn't present\n");
> +		return false;
> +	}
> +
> +	if (!pcie->pm_support)
> +		return false;
> +
> +	return true;
> +}
> +
> +#ifdef CONFIG_PM_SLEEP
> +static int ls_pcie_suspend_noirq(struct device *dev)

Why is noirq variant needed? Most drivers don't need this.

> +{
> +	struct ls_pcie *pcie = dev_get_drvdata(dev);
> +	struct dw_pcie *pci = pcie->pci;
> +	u32 val;
> +	int ret;
> +
> +	if (!ls_pcie_pm_check(pcie))
> +		return 0;
> +
> +	pcie->drvdata->pm_ops->send_turn_off_message(pcie);
> +
> +	/* 10ms timeout to check L2 ready */
> +	ret = readl_poll_timeout(pci->dbi_base + PCIE_PORT_DEBUG0,
> +				 val, LS_PCIE_IS_L2(val), 100, 10000);
> +	if (ret) {
> +		dev_err(dev, "PCIe link enter L2 timeout! ltssm = 0x%x\n", val);
> +		return ret;
> +	}
> +
> +	ls_pcie_set_dstate(pcie, PCIE_PM_SCR_PMEPS_D3);
> +
> +	return 0;
> +}
> +
> +static int ls_pcie_resume_noirq(struct device *dev)
> +{
> +	struct ls_pcie *pcie = dev_get_drvdata(dev);
> +	struct dw_pcie *pci = pcie->pci;
> +	int ret;
> +
> +	if (!ls_pcie_pm_check(pcie))
> +		return 0;
> +
> +	ls_pcie_set_dstate(pcie, PCIE_PM_SCR_PMEPS_D0);
> +
> +	pcie->drvdata->pm_ops->exit_from_l2(pcie);
> +
> +	/* delay 10ms to access EP */
> +	mdelay(10);
> +
> +	ret = ls_pcie_host_init(&pci->pp);
> +	if (ret) {
> +		dev_err(dev, "ls_pcie_host_init failed! ret = 0x%x\n", ret);
> +		return ret;
> +	}
> +
> +	ret = dw_pcie_wait_for_link(pci);
> +	if (ret) {
> +		dev_err(dev, "wait link up timeout! ret = 0x%x\n", ret);
> +		return ret;
> +	}
> +
> +	return 0;
> +}
> +#endif /* CONFIG_PM_SLEEP */
> +
> +static const struct dev_pm_ops ls_pcie_pm_ops = {
> +	SET_NOIRQ_SYSTEM_SLEEP_PM_OPS(ls_pcie_suspend_noirq,
> +				      ls_pcie_resume_noirq)
> +};
> +
>  static struct platform_driver ls_pcie_driver = {
>  	.driver = {
>  		.name = "layerscape-pcie",
>  		.of_match_table = ls_pcie_of_match,
>  		.suppress_bind_attrs = true,
> +		.pm = &ls_pcie_pm_ops,
>  	},
>  };
>  builtin_platform_driver_probe(ls_pcie_driver, ls_pcie_probe);
> diff --git a/drivers/pci/controller/dwc/pcie-designware.h b/drivers/pci/controller/dwc/pcie-designware.h
> index f911760dcc69..1f2fd03d1eba 100644
> --- a/drivers/pci/controller/dwc/pcie-designware.h
> +++ b/drivers/pci/controller/dwc/pcie-designware.h
> @@ -46,6 +46,7 @@
>  #define PCIE_PORT_DEBUG0		0x728
>  #define PORT_LOGIC_LTSSM_STATE_MASK	0x1f
>  #define PORT_LOGIC_LTSSM_STATE_L0	0x11
> +#define PORT_LOGIC_LTSSM_STATE_L2	0x15
>  #define PCIE_PORT_DEBUG1		0x72C
>  #define PCIE_PORT_DEBUG1_LINK_UP		BIT(4)
>  #define PCIE_PORT_DEBUG1_LINK_IN_TRAINING	BIT(29)
> -- 
> 2.17.1
> 

RE: [PATCH 7/7] PCI: layerscape: Add power management support

[Z.q. Hou]

Hi Rob,

Thanks a lot for your comments!

> -----Original Message-----
> From: Rob Herring <robh@kernel.org>
> Sent: 2020年9月15日 9:30
> To: Z.q. Hou <zhiqiang.hou@nxp.com>
> Cc: linux-pci@vger.kernel.org; devicetree@vger.kernel.org;
> linux-kernel@vger.kernel.org; bhelgaas@google.com;
> shawnguo@kernel.org; Leo Li <leoyang.li@nxp.com>;
> lorenzo.pieralisi@arm.com; gustavo.pimentel@synopsys.com; M.h. Lian
> <minghuan.lian@nxp.com>; Mingkai Hu <mingkai.hu@nxp.com>; Roy Zang
> <roy.zang@nxp.com>
> Subject: Re: [PATCH 7/7] PCI: layerscape: Add power management support
> 
> On Mon, Sep 07, 2020 at 01:38:01PM +0800, Zhiqiang Hou wrote:
> > From: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
> >
> > Add PME_Turn_Off/PME_TO_Ack handshake sequence, and finally put the
> > PCIe controller into D3 state after the L2/L3 ready state transition
> > process completion.
> >
> > Signed-off-by: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
> > ---
> >  drivers/pci/controller/dwc/pci-layerscape.c  | 384
> ++++++++++++++++++-
> >  drivers/pci/controller/dwc/pcie-designware.h |   1 +
> >  2 files changed, 383 insertions(+), 2 deletions(-)
> >
> > diff --git a/drivers/pci/controller/dwc/pci-layerscape.c
> > b/drivers/pci/controller/dwc/pci-layerscape.c
> > index be404c16bcbe..ca9ea07f77c1 100644
> > --- a/drivers/pci/controller/dwc/pci-layerscape.c
> > +++ b/drivers/pci/controller/dwc/pci-layerscape.c
> > @@ -3,13 +3,16 @@
> >   * PCIe host controller driver for Freescale Layerscape SoCs
> >   *
> >   * Copyright (C) 2014 Freescale Semiconductor.
> > + * Copyright 2020 NXP
> >   *
> >   * Author: Minghuan Lian <Minghuan.Lian@freescale.com>
> >   */
> >
> > +#include <linux/delay.h>
> >  #include <linux/kernel.h>
> >  #include <linux/interrupt.h>
> >  #include <linux/init.h>
> > +#include <linux/iopoll.h>
> >  #include <linux/of_pci.h>
> >  #include <linux/of_platform.h>
> >  #include <linux/of_irq.h>
> > @@ -27,17 +30,66 @@
> >  #define PCIE_ABSERR		0x8d0 /* Bridge Slave Error Response
> Register */
> >  #define PCIE_ABSERR_SETTING	0x9401 /* Forward error of
> non-posted request */
> >
> > +#define PCIE_PM_SCR		0x44
> > +#define PCIE_PM_SCR_PMEPS_D0	0x0
> > +#define PCIE_PM_SCR_PMEPS_D3	0x3
> > +
> > +#define PCIE_LNKCTL		0x80  /* PCIe link ctrl Register */
> 
> Use the common defines.
> 

OK, will change in next version.

> 
> > +
> > +/* PF Message Command Register */
> > +#define LS_PCIE_PF_MCR		0x2c
> > +#define PF_MCR_PTOMR		BIT(0)
> > +#define PF_MCR_EXL2S		BIT(1)
> > +
> > +/* LS1021A PEXn PM Write Control Register */
> > +#define SCFG_PEXPMWRCR(idx)	(0x5c + (idx) * 0x64)
> > +#define PMXMTTURNOFF		BIT(31)
> > +#define SCFG_PEXSFTRSTCR	0x190
> > +#define PEXSR(idx)		BIT(idx)
> > +
> > +/* LS1043A PEX PME control register */
> > +#define SCFG_PEXPMECR		0x144
> > +#define PEXPME(idx)		BIT(31 - (idx) * 4)
> > +
> > +/* LS1043A PEX LUT debug register */
> > +#define LS_PCIE_LDBG	0x7fc
> > +#define LDBG_SR		BIT(30)
> > +#define LDBG_WE		BIT(31)
> > +
> >  #define PCIE_IATU_NUM		6
> >
> > +#define LS_PCIE_IS_L2(v)	\
> > +	(((v) & PORT_LOGIC_LTSSM_STATE_MASK) ==
> PORT_LOGIC_LTSSM_STATE_L2)
> > +
> > +struct ls_pcie;
> > +
> > +struct ls_pcie_host_pm_ops {
> > +	int (*pm_init)(struct ls_pcie *pcie);
> > +	void (*send_turn_off_message)(struct ls_pcie *pcie);
> > +	void (*exit_from_l2)(struct ls_pcie *pcie); };
> > +
> >  struct ls_pcie_drvdata {
> > +	const u32 pf_off;
> > +	const u32 lut_off;
> >  	const struct dw_pcie_host_ops *ops;
> > +	const struct ls_pcie_host_pm_ops *pm_ops;
> >  };
> >
> >  struct ls_pcie {
> >  	struct dw_pcie *pci;
> >  	const struct ls_pcie_drvdata *drvdata;
> > +	void __iomem *pf_base;
> > +	void __iomem *lut_base;
> > +	bool big_endian;
> > +	bool ep_presence;
> > +	bool pm_support;
> > +	struct regmap *scfg;
> > +	int index;
> >  };
> >
> > +#define ls_pcie_lut_readl_addr(addr)	ls_pcie_lut_readl(pcie, addr)
> > +#define ls_pcie_pf_readl_addr(addr)	ls_pcie_pf_readl(pcie, addr)
> >  #define to_ls_pcie(x)	dev_get_drvdata((x)->dev)
> >
> >  static bool ls_pcie_is_bridge(struct ls_pcie *pcie) @@ -86,6 +138,208
> > @@ static void ls_pcie_fix_error_response(struct ls_pcie *pcie)
> >  	iowrite32(PCIE_ABSERR_SETTING, pci->dbi_base + PCIE_ABSERR);  }
> >
> > +static u32 ls_pcie_lut_readl(struct ls_pcie *pcie, u32 off) {
> > +	if (pcie->big_endian)
> > +		return ioread32be(pcie->lut_base + off);
> > +
> > +	return ioread32(pcie->lut_base + off); }
> > +
> > +static void ls_pcie_lut_writel(struct ls_pcie *pcie, u32 off, u32
> > +val) {
> > +	if (pcie->big_endian)
> > +		return iowrite32be(val, pcie->lut_base + off);
> > +
> > +	return iowrite32(val, pcie->lut_base + off);
> > +
> > +}
> > +
> > +static u32 ls_pcie_pf_readl(struct ls_pcie *pcie, u32 off) {
> > +	if (pcie->big_endian)
> > +		return ioread32be(pcie->pf_base + off);
> > +
> > +	return ioread32(pcie->pf_base + off); }
> > +
> > +static void ls_pcie_pf_writel(struct ls_pcie *pcie, u32 off, u32 val)
> > +{
> > +	if (pcie->big_endian)
> > +		return iowrite32be(val, pcie->pf_base + off);
> > +
> > +	return iowrite32(val, pcie->pf_base + off);
> > +
> > +}
> > +
> > +static void ls_pcie_send_turnoff_msg(struct ls_pcie *pcie) {
> > +	u32 val;
> > +	int ret;
> > +
> > +	val = ls_pcie_pf_readl(pcie, LS_PCIE_PF_MCR);
> > +	val |= PF_MCR_PTOMR;
> > +	ls_pcie_pf_writel(pcie, LS_PCIE_PF_MCR, val);
> > +
> > +	ret = readx_poll_timeout(ls_pcie_pf_readl_addr, LS_PCIE_PF_MCR,
> > +				 val, !(val & PF_MCR_PTOMR), 100, 10000);
> > +	if (ret)
> > +		dev_info(pcie->pci->dev, "poll turn off message timeout\n"); }
> > +
> > +static void ls1021a_pcie_send_turnoff_msg(struct ls_pcie *pcie) {
> > +	u32 val;
> > +
> > +	if (!pcie->scfg) {
> > +		dev_dbg(pcie->pci->dev, "SYSCFG is NULL\n");
> > +		return;
> > +	}
> > +
> > +	/* Send Turn_off message */
> > +	regmap_read(pcie->scfg, SCFG_PEXPMWRCR(pcie->index), &val);
> > +	val |= PMXMTTURNOFF;
> > +	regmap_write(pcie->scfg, SCFG_PEXPMWRCR(pcie->index), val);
> > +
> > +	mdelay(10);
> > +
> > +	/* Clear Turn_off message */
> > +	regmap_read(pcie->scfg, SCFG_PEXPMWRCR(pcie->index), &val);
> > +	val &= ~PMXMTTURNOFF;
> > +	regmap_write(pcie->scfg, SCFG_PEXPMWRCR(pcie->index), val); }
> > +
> > +static void ls1043a_pcie_send_turnoff_msg(struct ls_pcie *pcie) {
> > +	u32 val;
> > +
> > +	if (!pcie->scfg) {
> > +		dev_dbg(pcie->pci->dev, "SYSCFG is NULL\n");
> > +		return;
> > +	}
> > +
> > +	/* Send Turn_off message */
> > +	regmap_read(pcie->scfg, SCFG_PEXPMECR, &val);
> > +	val |= PEXPME(pcie->index);
> > +	regmap_write(pcie->scfg, SCFG_PEXPMECR, val);
> > +
> > +	mdelay(10);
> > +
> > +	/* Clear Turn_off message */
> > +	regmap_read(pcie->scfg, SCFG_PEXPMECR, &val);
> > +	val &= ~PEXPME(pcie->index);
> > +	regmap_write(pcie->scfg, SCFG_PEXPMECR, val); }
> > +
> > +static void ls_pcie_exit_from_l2(struct ls_pcie *pcie) {
> > +	u32 val;
> > +	int ret;
> > +
> > +	val = ls_pcie_pf_readl(pcie, LS_PCIE_PF_MCR);
> > +	val |= PF_MCR_EXL2S;
> > +	ls_pcie_pf_writel(pcie, LS_PCIE_PF_MCR, val);
> > +
> > +	ret = readx_poll_timeout(ls_pcie_pf_readl_addr, LS_PCIE_PF_MCR,
> > +				 val, !(val & PF_MCR_EXL2S), 100, 10000);
> > +	if (ret)
> > +		dev_info(pcie->pci->dev, "poll exit L2 state timeout\n"); }
> > +
> > +static void ls_pcie_retrain_link(struct ls_pcie *pcie) {
> > +	struct dw_pcie *pci = pcie->pci;
> > +	u32 val;
> > +
> > +	val = dw_pcie_readw_dbi(pci, PCIE_LNKCTL);
> > +	val |= PCI_EXP_LNKCTL_RL;
> > +	dw_pcie_writew_dbi(pci, PCIE_LNKCTL, val); }
> > +
> > +static void ls1021a_pcie_exit_from_l2(struct ls_pcie *pcie) {
> > +	u32 val;
> > +
> > +	regmap_read(pcie->scfg, SCFG_PEXSFTRSTCR, &val);
> > +	val |= PEXSR(pcie->index);
> > +	regmap_write(pcie->scfg, SCFG_PEXSFTRSTCR, val);
> > +
> > +	regmap_read(pcie->scfg, SCFG_PEXSFTRSTCR, &val);
> > +	val &= ~PEXSR(pcie->index);
> > +	regmap_write(pcie->scfg, SCFG_PEXSFTRSTCR, val);
> > +
> > +	mdelay(1);
> > +
> > +	ls_pcie_retrain_link(pcie);
> > +}
> > +static void ls1043a_pcie_exit_from_l2(struct ls_pcie *pcie) {
> > +	u32 val;
> > +
> > +	val = ls_pcie_lut_readl(pcie, LS_PCIE_LDBG);
> > +	val |= LDBG_WE;
> > +	ls_pcie_lut_writel(pcie, LS_PCIE_LDBG, val);
> > +
> > +	val = ls_pcie_lut_readl(pcie, LS_PCIE_LDBG);
> > +	val |= LDBG_SR;
> > +	ls_pcie_lut_writel(pcie, LS_PCIE_LDBG, val);
> > +
> > +	val = ls_pcie_lut_readl(pcie, LS_PCIE_LDBG);
> > +	val &= ~LDBG_SR;
> > +	ls_pcie_lut_writel(pcie, LS_PCIE_LDBG, val);
> > +
> > +	val = ls_pcie_lut_readl(pcie, LS_PCIE_LDBG);
> > +	val &= ~LDBG_WE;
> > +	ls_pcie_lut_writel(pcie, LS_PCIE_LDBG, val);
> > +
> > +	mdelay(1);
> > +
> > +	ls_pcie_retrain_link(pcie);
> > +}
> > +
> > +static int ls1021a_pcie_pm_init(struct ls_pcie *pcie) {
> > +	struct device *dev = pcie->pci->dev;
> > +	u32 index[2];
> > +	int ret;
> > +
> > +	pcie->scfg = syscon_regmap_lookup_by_phandle(dev->of_node,
> > +						     "fsl,pcie-scfg");
> > +	if (IS_ERR(pcie->scfg)) {
> > +		ret = PTR_ERR(pcie->scfg);
> > +		dev_err(dev, "No syscfg phandle specified\n");
> > +		pcie->scfg = NULL;
> > +		return ret;
> > +	}
> > +
> > +	ret = of_property_read_u32_array(dev->of_node, "fsl,pcie-scfg",
> > +					 index, 2);
> > +	if (ret) {
> > +		pcie->scfg = NULL;
> > +		return ret;
> > +	}
> > +
> > +	pcie->index = index[1];
> > +
> > +	return 0;
> > +}
> > +
> > +static int ls_pcie_pm_init(struct ls_pcie *pcie) {
> > +	return 0;
> > +}
> > +
> > +static void ls_pcie_set_dstate(struct ls_pcie *pcie, u32 dstate) {
> > +	struct dw_pcie *pci = pcie->pci;
> > +	u32 val;
> > +
> > +	val = dw_pcie_readw_dbi(pci, PCIE_PM_SCR);
> > +	val &= ~PCI_PM_CTRL_STATE_MASK;
> > +	val |= dstate;
> > +	dw_pcie_writew_dbi(pci, PCIE_PM_SCR, val); }
> > +
> >  static int ls_pcie_host_init(struct pcie_port *pp)  {
> >  	struct dw_pcie *pci = to_dw_pcie_from_pp(pp); @@ -133,20 +387,52
> @@
> > static int ls_pcie_msi_host_init(struct pcie_port *pp)
> >  	return 0;
> >  }
> >
> > +static struct ls_pcie_host_pm_ops ls1021a_pcie_host_pm_ops = {
> > +	.pm_init = &ls1021a_pcie_pm_init,
> > +	.send_turn_off_message = &ls1021a_pcie_send_turnoff_msg,
> > +	.exit_from_l2 = &ls1021a_pcie_exit_from_l2, };
> > +
> > +static struct ls_pcie_host_pm_ops ls1043a_pcie_host_pm_ops = {
> > +	.pm_init = &ls1021a_pcie_pm_init,
> > +	.send_turn_off_message = &ls1043a_pcie_send_turnoff_msg,
> > +	.exit_from_l2 = &ls1043a_pcie_exit_from_l2, };
> > +
> > +static struct ls_pcie_host_pm_ops ls_pcie_host_pm_ops = {
> > +	.pm_init = &ls_pcie_pm_init,
> > +	.send_turn_off_message = &ls_pcie_send_turnoff_msg,
> > +	.exit_from_l2 = &ls_pcie_exit_from_l2, };
> > +
> >  static const struct dw_pcie_host_ops ls_pcie_host_ops = {
> >  	.host_init = ls_pcie_host_init,
> >  	.msi_host_init = ls_pcie_msi_host_init,  };
> >
> > +static const struct ls_pcie_drvdata ls1021a_drvdata = {
> > +	.ops = &ls_pcie_host_ops,
> > +	.pm_ops = &ls1021a_pcie_host_pm_ops, };
> > +
> > +static const struct ls_pcie_drvdata ls1043a_drvdata = {
> > +	.ops = &ls_pcie_host_ops,
> > +	.lut_off = 0x10000,
> > +	.pm_ops = &ls1043a_pcie_host_pm_ops, };
> > +
> >  static const struct ls_pcie_drvdata layerscape_drvdata = {
> >  	.ops = &ls_pcie_host_ops,
> > +	.lut_off = 0x80000,
> > +	.pf_off = 0xc0000,
> > +	.pm_ops = &ls_pcie_host_pm_ops,
> >  };
> >
> >  static const struct of_device_id ls_pcie_of_match[] = {
> >  	{ .compatible = "fsl,ls1012a-pcie", .data = &layerscape_drvdata },
> > -	{ .compatible = "fsl,ls1021a-pcie", .data = &layerscape_drvdata },
> > +	{ .compatible = "fsl,ls1021a-pcie", .data = &ls1021a_drvdata },
> >  	{ .compatible = "fsl,ls1028a-pcie", .data = &layerscape_drvdata },
> > -	{ .compatible = "fsl,ls1043a-pcie", .data = &layerscape_drvdata },
> > +	{ .compatible = "fsl,ls1043a-pcie", .data = &ls1043a_drvdata },
> >  	{ .compatible = "fsl,ls1046a-pcie", .data = &layerscape_drvdata },
> >  	{ .compatible = "fsl,ls2080a-pcie", .data = &layerscape_drvdata },
> >  	{ .compatible = "fsl,ls2085a-pcie", .data = &layerscape_drvdata },
> > @@ -170,6 +456,15 @@ static int __init ls_add_pcie_port(struct ls_pcie
> *pcie)
> >  		return ret;
> >  	}
> >
> > +	if (dw_pcie_link_up(pci)) {
> > +		dev_dbg(pci->dev, "Endpoint is present\n");
> > +		pcie->ep_presence = true;
> > +	}
> > +
> > +	if (pcie->drvdata->pm_ops && pcie->drvdata->pm_ops->pm_init &&
> > +	    !pcie->drvdata->pm_ops->pm_init(pcie))
> > +		pcie->pm_support = true;
> > +
> >  	return 0;
> >  }
> >
> > @@ -200,6 +495,14 @@ static int __init ls_pcie_probe(struct
> platform_device *pdev)
> >  	if (IS_ERR(pci->dbi_base))
> >  		return PTR_ERR(pci->dbi_base);
> >
> > +	pcie->big_endian = of_property_read_bool(dev->of_node,
> > +"big-endian");
> > +
> > +	if (pcie->drvdata->lut_off)
> > +		pcie->lut_base = pci->dbi_base + pcie->drvdata->lut_off;
> > +
> > +	if (pcie->drvdata->pf_off)
> > +		pcie->pf_base = pci->dbi_base + pcie->drvdata->pf_off;
> > +
> >  	if (!ls_pcie_is_bridge(pcie))
> >  		return -ENODEV;
> >
> > @@ -212,11 +515,88 @@ static int __init ls_pcie_probe(struct
> platform_device *pdev)
> >  	return 0;
> >  }
> >
> > +static bool ls_pcie_pm_check(struct ls_pcie *pcie) {
> > +	if (!pcie->ep_presence) {
> > +		dev_dbg(pcie->pci->dev, "Endpoint isn't present\n");
> > +		return false;
> > +	}
> > +
> > +	if (!pcie->pm_support)
> > +		return false;
> > +
> > +	return true;
> > +}
> > +
> > +#ifdef CONFIG_PM_SLEEP
> > +static int ls_pcie_suspend_noirq(struct device *dev)
> 
> Why is noirq variant needed? Most drivers don't need this.

Actually all PCIe RC drivers use the noirq variant, the reason
is the pci_pm_suspend_noirq() will call the pci_save_state().

Regards,
Zhiqiang

> 
> > +{
> > +	struct ls_pcie *pcie = dev_get_drvdata(dev);
> > +	struct dw_pcie *pci = pcie->pci;
> > +	u32 val;
> > +	int ret;
> > +
> > +	if (!ls_pcie_pm_check(pcie))
> > +		return 0;
> > +
> > +	pcie->drvdata->pm_ops->send_turn_off_message(pcie);
> > +
> > +	/* 10ms timeout to check L2 ready */
> > +	ret = readl_poll_timeout(pci->dbi_base + PCIE_PORT_DEBUG0,
> > +				 val, LS_PCIE_IS_L2(val), 100, 10000);
> > +	if (ret) {
> > +		dev_err(dev, "PCIe link enter L2 timeout! ltssm = 0x%x\n", val);
> > +		return ret;
> > +	}
> > +
> > +	ls_pcie_set_dstate(pcie, PCIE_PM_SCR_PMEPS_D3);
> > +
> > +	return 0;
> > +}
> > +
> > +static int ls_pcie_resume_noirq(struct device *dev) {
> > +	struct ls_pcie *pcie = dev_get_drvdata(dev);
> > +	struct dw_pcie *pci = pcie->pci;
> > +	int ret;
> > +
> > +	if (!ls_pcie_pm_check(pcie))
> > +		return 0;
> > +
> > +	ls_pcie_set_dstate(pcie, PCIE_PM_SCR_PMEPS_D0);
> > +
> > +	pcie->drvdata->pm_ops->exit_from_l2(pcie);
> > +
> > +	/* delay 10ms to access EP */
> > +	mdelay(10);
> > +
> > +	ret = ls_pcie_host_init(&pci->pp);
> > +	if (ret) {
> > +		dev_err(dev, "ls_pcie_host_init failed! ret = 0x%x\n", ret);
> > +		return ret;
> > +	}
> > +
> > +	ret = dw_pcie_wait_for_link(pci);
> > +	if (ret) {
> > +		dev_err(dev, "wait link up timeout! ret = 0x%x\n", ret);
> > +		return ret;
> > +	}
> > +
> > +	return 0;
> > +}
> > +#endif /* CONFIG_PM_SLEEP */
> > +
> > +static const struct dev_pm_ops ls_pcie_pm_ops = {
> > +	SET_NOIRQ_SYSTEM_SLEEP_PM_OPS(ls_pcie_suspend_noirq,
> > +				      ls_pcie_resume_noirq)
> > +};
> > +
> >  static struct platform_driver ls_pcie_driver = {
> >  	.driver = {
> >  		.name = "layerscape-pcie",
> >  		.of_match_table = ls_pcie_of_match,
> >  		.suppress_bind_attrs = true,
> > +		.pm = &ls_pcie_pm_ops,
> >  	},
> >  };
> >  builtin_platform_driver_probe(ls_pcie_driver, ls_pcie_probe); diff
> > --git a/drivers/pci/controller/dwc/pcie-designware.h
> > b/drivers/pci/controller/dwc/pcie-designware.h
> > index f911760dcc69..1f2fd03d1eba 100644
> > --- a/drivers/pci/controller/dwc/pcie-designware.h
> > +++ b/drivers/pci/controller/dwc/pcie-designware.h
> > @@ -46,6 +46,7 @@
> >  #define PCIE_PORT_DEBUG0		0x728
> >  #define PORT_LOGIC_LTSSM_STATE_MASK	0x1f
> >  #define PORT_LOGIC_LTSSM_STATE_L0	0x11
> > +#define PORT_LOGIC_LTSSM_STATE_L2	0x15
> >  #define PCIE_PORT_DEBUG1		0x72C
> >  #define PCIE_PORT_DEBUG1_LINK_UP		BIT(4)
> >  #define PCIE_PORT_DEBUG1_LINK_IN_TRAINING	BIT(29)
> > --
> > 2.17.1
> >

Re: [PATCH 0/7] PCI: layerscape: Add power management support

[Lorenzo Pieralisi]

On Mon, Sep 07, 2020 at 01:37:54PM +0800, Zhiqiang Hou wrote:
> From: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
> 
> This patch series is to add PCIe power management support for NXP
> Layerscape platfroms.
> 
> Hou Zhiqiang (7):
>   PCI: dwc: Fix a bug of the case dw_pci->ops is NULL
>   PCI: layerscape: Change to use the DWC common link-up check function
>   dt-bindings: pci: layerscape-pci: Add a optional property big-endian
>   arm64: dts: layerscape: Add big-endian property for PCIe nodes
>   dt-bindings: pci: layerscape-pci: Update the description of SCFG
>     property
>   dts: arm64: ls1043a: Add SCFG phandle for PCIe nodes
>   PCI: layerscape: Add power management support
> 
>  .../bindings/pci/layerscape-pci.txt           |   6 +-
>  .../arm64/boot/dts/freescale/fsl-ls1012a.dtsi |   1 +
>  .../arm64/boot/dts/freescale/fsl-ls1043a.dtsi |   6 +
>  .../arm64/boot/dts/freescale/fsl-ls1046a.dtsi |   3 +
>  drivers/pci/controller/dwc/pci-layerscape.c   | 473 ++++++++++++++----
>  drivers/pci/controller/dwc/pcie-designware.c  |  12 +-
>  drivers/pci/controller/dwc/pcie-designware.h  |   1 +
>  7 files changed, 388 insertions(+), 114 deletions(-)

I don't know which patches are still applicable, I will mark this
series as superseded since you will have to rebase it anyway - please
let me know what's the plan.

Thanks,
Lorenzo

RE: [PATCH 0/7] PCI: layerscape: Add power management support

[Z.q. Hou]

Hi Lorenzo,

> -----Original Message-----
> From: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
> Sent: 2021年3月23日 19:15
> To: Z.q. Hou <zhiqiang.hou@nxp.com>
> Cc: linux-pci@vger.kernel.org; devicetree@vger.kernel.org;
> linux-kernel@vger.kernel.org; bhelgaas@google.com; robh+dt@kernel.org;
> shawnguo@kernel.org; Leo Li <leoyang.li@nxp.com>;
> gustavo.pimentel@synopsys.com; M.h. Lian <minghuan.lian@nxp.com>;
> Mingkai Hu <mingkai.hu@nxp.com>; Roy Zang <roy.zang@nxp.com>
> Subject: Re: [PATCH 0/7] PCI: layerscape: Add power management support
> 
> On Mon, Sep 07, 2020 at 01:37:54PM +0800, Zhiqiang Hou wrote:
> > From: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
> >
> > This patch series is to add PCIe power management support for NXP
> > Layerscape platfroms.
> >
> > Hou Zhiqiang (7):
> >   PCI: dwc: Fix a bug of the case dw_pci->ops is NULL
> >   PCI: layerscape: Change to use the DWC common link-up check
> function
> >   dt-bindings: pci: layerscape-pci: Add a optional property big-endian
> >   arm64: dts: layerscape: Add big-endian property for PCIe nodes
> >   dt-bindings: pci: layerscape-pci: Update the description of SCFG
> >     property
> >   dts: arm64: ls1043a: Add SCFG phandle for PCIe nodes
> >   PCI: layerscape: Add power management support
> >
> >  .../bindings/pci/layerscape-pci.txt           |   6 +-
> >  .../arm64/boot/dts/freescale/fsl-ls1012a.dtsi |   1 +
> >  .../arm64/boot/dts/freescale/fsl-ls1043a.dtsi |   6 +
> >  .../arm64/boot/dts/freescale/fsl-ls1046a.dtsi |   3 +
> >  drivers/pci/controller/dwc/pci-layerscape.c   | 473
> ++++++++++++++----
> >  drivers/pci/controller/dwc/pcie-designware.c  |  12 +-
> >  drivers/pci/controller/dwc/pcie-designware.h  |   1 +
> >  7 files changed, 388 insertions(+), 114 deletions(-)
> 
> I don't know which patches are still applicable, I will mark this series as
> superseded since you will have to rebase it anyway - please let me know
> what's the plan.

I'll rebase this series on the latest base.

Thanks,
Zhiqiang

> 
> Thanks,
> Lorenzo