[PATCH RESEND v4 00/15] PCI: dwc: Add hw version and dma-ranges support

[PATCH RESEND v4 00/15] PCI: dwc: Add hw version and dma-ranges support

[Serge Semin]

This patchset is a second one in the series created in the framework of
my Baikal-T1 PCIe/eDMA-related work:

[1: In-progress v5] PCI: dwc: Various fixes and cleanups
Link: https://lore.kernel.org/linux-pci/20220624143428.8334-1-Sergey.Semin@baikalelectronics.ru/
[2: In-progress v4] PCI: dwc: Add hw version and dma-ranges support
Link: ---you are looking at it---
[3: In-progress v3] PCI: dwc: Add extended YAML-schema and Baikal-T1 support
Link: https://lore.kernel.org/linux-pci/20220610085706.15741-1-Sergey.Semin@baikalelectronics.ru/
[4: In-progress v3] dmaengine: dw-edma: Add RP/EP local DMA support
Link: https://lore.kernel.org/linux-pci/20220610091459.17612-1-Sergey.Semin@baikalelectronics.ru

Note it is very recommended to merge the patchsets in the same order as
they are listed in the set above in order to have them applied smoothly.
Nothing prevents them from being reviewed synchronously though.

Originally the patches submitted in this patchset were a part of the series:
Link: https://lore.kernel.org/linux-pci/20220503214638.1895-1-Sergey.Semin@baikalelectronics.ru/
but due to the reviewers requests the series was expanded to about 30
patches which made it too bulky for a comfortable review. So I decided to
split it up into two patchsets: 2. and 3. in the table above.

Regarding the series content. This patchset is about adding new features
to the DW PCIe core, Root Port and Endpoint drivers. First we suggest to
add a more verbose link-up log message. Really printing link generation
and width would be much more informative than just "link up". Then a
series of IP-core version-related patches go, like using a native FourCC
version representation, adding the IP-core auto-detection, adding a better
structured IP-core version/type interface and finally dropping manual
IP-core version setups from the platforms which are supposed to have it
auto-detected. After that the platform-specific host de-initialization
method is introduced. It's unused in the framework of this patchset but
will be utilized in the next one (see the table above). A series of iATU
optimizations, cleanups and new features goes afterwards. In particular we
suggest to drop some redundant enumerations, add iATU regions size
detection procedure and then use the regions parameters to verify the
requested by the platform iATU ranges/dma-ranges settings. After that the
dma-ranges property support is added for the DW PCIe Host controllers.

Link: https://lore.kernel.org/linux-pci/20220324013734.18234-1-Sergey.Semin@baikalelectronics.ru/
Changelog v2:
- Test the error condition first and return straight away if it comes true
  in the link up waiting and link state logging method (@Joe).
- Move the dw_pcie_region_type enumeration removal patch to being applied
  before the IB/OB iATU windows setup simplification patch (@Rob).
- Move the iATU region selection procedure into a helper function (@Rob).
- Rebase from kernel v5.17 onto v5.18-rc3 since the later kernel has
  already DT bindings converted. (@Rob)
- Simplify the iATU region selection procedure by recalculating the base
  address only if the space is unrolled. The iATU viewport base address
  will be saved in the pci->atu_base field.
- Move the IP-core version detection procedure call from
  dw_pcie_ep_init_complete() to dw_pcie_ep_init().
- Add a new patch: "PCI: dwc: Detect iATU settings after getting
  "addr_space" resource."

Link: https://lore.kernel.org/linux-pci/20220503225104.12108-1-Sergey.Semin@baikalelectronics.ru
Changelog v3:
- Fix pcie-tegra194-acpi.c driver to using the new macros names.
  (@Manivannan)
- Drop in/outbound iATU window size alignment constraint. (@Manivannan)
- Detach this series of patches into a dedicated patchset.
- Rebase onto kernel v5.18.

Link: https://lore.kernel.org/linux-pci/20220610084444.14549-1-Sergey.Semin@baikalelectronics.ru/
Changelog v4:
- Just resend.
- Rebase onto the kernel v5.19-rcX.

Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>
Cc: Alexey Malahov <Alexey.Malahov@baikalelectronics.ru>
Cc: Pavel Parkhomenko <Pavel.Parkhomenko@baikalelectronics.ru>
Cc: "Krzysztof Wilczyński" <kw@linux.com>
Cc: Frank Li <Frank.Li@nxp.com>
Cc: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Cc: Rob Herring <robh+dt@kernel.org>
Cc: linux-pci@vger.kernel.org
Cc: devicetree@vger.kernel.org
Cc: linux-kernel@vger.kernel.org

Serge Semin (15):
  PCI: dwc: Add more verbose link-up message
  PCI: dwc: Detect iATU settings after getting "addr_space" resource
  PCI: dwc: Convert to using native IP-core versions representation
  PCI: dwc: Add IP-core version detection procedure
  PCI: dwc: Introduce Synopsys IP-core versions/types interface
  PCI: intel-gw: Drop manual DW PCIe controller version setup
  PCI: tegra194: Drop manual DW PCIe controller version setup
  PCI: dwc: Add host de-initialization callback
  PCI: dwc: Drop inbound iATU types enumeration - dw_pcie_as_type
  PCI: dwc: Drop iATU regions enumeration - dw_pcie_region_type
  PCI: dwc: Simplify in/outbound iATU setup methods
  PCI: dwc: Add iATU regions size detection procedure
  PCI: dwc: Verify in/out regions against iATU constraints
  PCI: dwc: Check iATU in/outbound ranges setup methods status
  PCI: dwc: Introduce dma-ranges property support for RC-host

 drivers/pci/controller/dwc/pci-keystone.c     |  12 +-
 .../pci/controller/dwc/pcie-designware-ep.c   |  40 +-
 .../pci/controller/dwc/pcie-designware-host.c | 202 +++++---
 drivers/pci/controller/dwc/pcie-designware.c  | 461 ++++++++----------
 drivers/pci/controller/dwc/pcie-designware.h  | 123 ++---
 drivers/pci/controller/dwc/pcie-intel-gw.c    |  22 +-
 .../pci/controller/dwc/pcie-tegra194-acpi.c   |   7 +-
 drivers/pci/controller/dwc/pcie-tegra194.c    |   1 -
 8 files changed, 461 insertions(+), 407 deletions(-)

-- 
2.35.1

[PATCH RESEND v4 01/15] PCI: dwc: Add more verbose link-up message

[Serge Semin]

Printing just "link up" isn't that much informative especially when it
comes to working with the PCI Express bus. Even if the link is up, due to
multiple reasons the bus performance can degrade to slower speeds or to
narrower width than both Root Port and its partner is capable of. In that
case it would be handy to know the link specifications as early as
possible. So let's add a more verbose message to the busy-wait link-state
method, which will contain the link speed generation and the PCIe bus
width in case if the link up state is discovered. Otherwise an error will
be printed to the system log.

Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>
Reviewed-by: Rob Herring <robh@kernel.org>

---

Changelog v2:
- Test the error condition first and return straight away if it comes true.
  The typical return is better to be unindented (@Joe).
---
 drivers/pci/controller/dwc/pcie-designware.c | 22 ++++++++++++++------
 1 file changed, 16 insertions(+), 6 deletions(-)

diff --git a/drivers/pci/controller/dwc/pcie-designware.c b/drivers/pci/controller/dwc/pcie-designware.c
index ce01187947c9..e66d16a86168 100644
--- a/drivers/pci/controller/dwc/pcie-designware.c
+++ b/drivers/pci/controller/dwc/pcie-designware.c
@@ -524,20 +524,30 @@ void dw_pcie_disable_atu(struct dw_pcie *pci, int index,
 
 int dw_pcie_wait_for_link(struct dw_pcie *pci)
 {
+	u32 offset, val;
 	int retries;
 
 	/* Check if the link is up or not */
 	for (retries = 0; retries < LINK_WAIT_MAX_RETRIES; retries++) {
-		if (dw_pcie_link_up(pci)) {
-			dev_info(pci->dev, "Link up\n");
-			return 0;
-		}
+		if (dw_pcie_link_up(pci))
+			break;
+
 		usleep_range(LINK_WAIT_USLEEP_MIN, LINK_WAIT_USLEEP_MAX);
 	}
 
-	dev_info(pci->dev, "Phy link never came up\n");
+	if (retries >= LINK_WAIT_MAX_RETRIES) {
+		dev_err(pci->dev, "Phy link never came up\n");
+		return -ETIMEDOUT;
+	}
+
+	offset = dw_pcie_find_capability(pci, PCI_CAP_ID_EXP);
+	val = dw_pcie_readw_dbi(pci, offset + PCI_EXP_LNKSTA);
 
-	return -ETIMEDOUT;
+	dev_info(pci->dev, "PCIe Gen.%u x%u link up\n",
+		 FIELD_GET(PCI_EXP_LNKSTA_CLS, val),
+		 FIELD_GET(PCI_EXP_LNKSTA_NLW, val));
+
+	return 0;
 }
 EXPORT_SYMBOL_GPL(dw_pcie_wait_for_link);
 
-- 
2.35.1

Re: [PATCH RESEND v4 01/15] PCI: dwc: Add more verbose link-up message

[Manivannan Sadhasivam]

On Fri, Jun 24, 2022 at 05:39:33PM +0300, Serge Semin wrote:
> Printing just "link up" isn't that much informative especially when it
> comes to working with the PCI Express bus. Even if the link is up, due to
> multiple reasons the bus performance can degrade to slower speeds or to
> narrower width than both Root Port and its partner is capable of. In that
> case it would be handy to know the link specifications as early as
> possible. So let's add a more verbose message to the busy-wait link-state
> method, which will contain the link speed generation and the PCIe bus
> width in case if the link up state is discovered. Otherwise an error will
> be printed to the system log.
> 
> Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>

Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>

Thanks,
Mani

> Reviewed-by: Rob Herring <robh@kernel.org>
> 
> ---
> 
> Changelog v2:
> - Test the error condition first and return straight away if it comes true.
>   The typical return is better to be unindented (@Joe).
> ---
>  drivers/pci/controller/dwc/pcie-designware.c | 22 ++++++++++++++------
>  1 file changed, 16 insertions(+), 6 deletions(-)
> 
> diff --git a/drivers/pci/controller/dwc/pcie-designware.c b/drivers/pci/controller/dwc/pcie-designware.c
> index ce01187947c9..e66d16a86168 100644
> --- a/drivers/pci/controller/dwc/pcie-designware.c
> +++ b/drivers/pci/controller/dwc/pcie-designware.c
> @@ -524,20 +524,30 @@ void dw_pcie_disable_atu(struct dw_pcie *pci, int index,
>  
>  int dw_pcie_wait_for_link(struct dw_pcie *pci)
>  {
> +	u32 offset, val;
>  	int retries;
>  
>  	/* Check if the link is up or not */
>  	for (retries = 0; retries < LINK_WAIT_MAX_RETRIES; retries++) {
> -		if (dw_pcie_link_up(pci)) {
> -			dev_info(pci->dev, "Link up\n");
> -			return 0;
> -		}
> +		if (dw_pcie_link_up(pci))
> +			break;
> +
>  		usleep_range(LINK_WAIT_USLEEP_MIN, LINK_WAIT_USLEEP_MAX);
>  	}
>  
> -	dev_info(pci->dev, "Phy link never came up\n");
> +	if (retries >= LINK_WAIT_MAX_RETRIES) {
> +		dev_err(pci->dev, "Phy link never came up\n");
> +		return -ETIMEDOUT;
> +	}
> +
> +	offset = dw_pcie_find_capability(pci, PCI_CAP_ID_EXP);
> +	val = dw_pcie_readw_dbi(pci, offset + PCI_EXP_LNKSTA);
>  
> -	return -ETIMEDOUT;
> +	dev_info(pci->dev, "PCIe Gen.%u x%u link up\n",
> +		 FIELD_GET(PCI_EXP_LNKSTA_CLS, val),
> +		 FIELD_GET(PCI_EXP_LNKSTA_NLW, val));
> +
> +	return 0;
>  }
>  EXPORT_SYMBOL_GPL(dw_pcie_wait_for_link);
>  
> -- 
> 2.35.1
> 

-- 
மணிவண்ணன் சதாசிவம்

[PATCH RESEND v4 02/15] PCI: dwc: Detect iATU settings after getting "addr_space" resource

[Serge Semin]

The iATU detection procedure was introduced in the commit 281f1f99cf3a
("PCI: dwc: Detect number of iATU windows"). A bit later the procedure
execution was moved to Host/EP-specific methods in the framework of commit
8bcca2658558 ("PCI: dwc: Move iATU detection earlier"). The later
modification wasn't done in the most optimal way since the "addr_space"
CSR region resource doesn't depend on anything detected in the
dw_pcie_iatu_detect() method. Thus the detection can be postponed to be
executed after the resource request which can fail and make the detection
pointless. It will be also helpful for the dw_pcie_ep_init() method
readability since we are about to add the IP-core version and eDMA module
(a bit later) detection procedures.

Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>
Reviewed-by: Rob Herring <robh@kernel.org>

---

Changelog v2:
- This is a new patch added on v2 iteration of the series.
---
 drivers/pci/controller/dwc/pcie-designware-ep.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/pci/controller/dwc/pcie-designware-ep.c b/drivers/pci/controller/dwc/pcie-designware-ep.c
index 15b8059544e3..1e35542d6f72 100644
--- a/drivers/pci/controller/dwc/pcie-designware-ep.c
+++ b/drivers/pci/controller/dwc/pcie-designware-ep.c
@@ -704,8 +704,6 @@ int dw_pcie_ep_init(struct dw_pcie_ep *ep)
 		}
 	}
 
-	dw_pcie_iatu_detect(pci);
-
 	res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "addr_space");
 	if (!res)
 		return -EINVAL;
@@ -713,6 +711,8 @@ int dw_pcie_ep_init(struct dw_pcie_ep *ep)
 	ep->phys_base = res->start;
 	ep->addr_size = resource_size(res);
 
+	dw_pcie_iatu_detect(pci);
+
 	ep->ib_window_map = devm_kcalloc(dev,
 					 BITS_TO_LONGS(pci->num_ib_windows),
 					 sizeof(long),
-- 
2.35.1

Re: [PATCH RESEND v4 02/15] PCI: dwc: Detect iATU settings after getting "addr_space" resource

[Bjorn Helgaas]

On Fri, Jun 24, 2022 at 05:39:34PM +0300, Serge Semin wrote:
> The iATU detection procedure was introduced in the commit 281f1f99cf3a
> ("PCI: dwc: Detect number of iATU windows"). A bit later the procedure
> execution was moved to Host/EP-specific methods in the framework of commit
> 8bcca2658558 ("PCI: dwc: Move iATU detection earlier"). The later
> modification wasn't done in the most optimal way since the "addr_space"
> CSR region resource doesn't depend on anything detected in the
> dw_pcie_iatu_detect() method. Thus the detection can be postponed to be
> executed after the resource request which can fail and make the detection
> pointless. It will be also helpful for the dw_pcie_ep_init() method
> readability since we are about to add the IP-core version and eDMA module
> (a bit later) detection procedures.

The patch is fine, but IMO the commit log is overkill.  It's enough to
say that we do some work (dw_pcie_iatu_detect()) earlier than
necessary, so we can move it later.  All the history makes it seem
like this is more complicated than it really is.

Something like this is enough:

    Previously, dw_pcie_ep_init() did:

      dw_pcie_iatu_detect(pci);
      res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "addr_space");
      if (!res)
        return -EINVAL;

    The platform_get_resource_byname() can fail, and dw_pcie_iatu_detect()
    doesn't depend on the "addr_space" resource, so delay it until afterwards,
    i.e.,

      platform_get_resource_byname(pdev, IORESOURCE_MEM, "addr_space");
      dw_pcie_iatu_detect(pci);

No need to repost for this.

> Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>
> Reviewed-by: Rob Herring <robh@kernel.org>
> 
> ---
> 
> Changelog v2:
> - This is a new patch added on v2 iteration of the series.
> ---
>  drivers/pci/controller/dwc/pcie-designware-ep.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/pci/controller/dwc/pcie-designware-ep.c b/drivers/pci/controller/dwc/pcie-designware-ep.c
> index 15b8059544e3..1e35542d6f72 100644
> --- a/drivers/pci/controller/dwc/pcie-designware-ep.c
> +++ b/drivers/pci/controller/dwc/pcie-designware-ep.c
> @@ -704,8 +704,6 @@ int dw_pcie_ep_init(struct dw_pcie_ep *ep)
>  		}
>  	}
>  
> -	dw_pcie_iatu_detect(pci);
> -
>  	res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "addr_space");
>  	if (!res)
>  		return -EINVAL;
> @@ -713,6 +711,8 @@ int dw_pcie_ep_init(struct dw_pcie_ep *ep)
>  	ep->phys_base = res->start;
>  	ep->addr_size = resource_size(res);
>  
> +	dw_pcie_iatu_detect(pci);
> +
>  	ep->ib_window_map = devm_kcalloc(dev,
>  					 BITS_TO_LONGS(pci->num_ib_windows),
>  					 sizeof(long),
> -- 
> 2.35.1
> 

Re: [PATCH RESEND v4 02/15] PCI: dwc: Detect iATU settings after getting "addr_space" resource

[Manivannan Sadhasivam]

On Fri, Jun 24, 2022 at 05:39:34PM +0300, Serge Semin wrote:
> The iATU detection procedure was introduced in the commit 281f1f99cf3a
> ("PCI: dwc: Detect number of iATU windows"). A bit later the procedure
> execution was moved to Host/EP-specific methods in the framework of commit
> 8bcca2658558 ("PCI: dwc: Move iATU detection earlier"). The later
> modification wasn't done in the most optimal way since the "addr_space"
> CSR region resource doesn't depend on anything detected in the
> dw_pcie_iatu_detect() method. Thus the detection can be postponed to be
> executed after the resource request which can fail and make the detection
> pointless. It will be also helpful for the dw_pcie_ep_init() method
> readability since we are about to add the IP-core version and eDMA module
> (a bit later) detection procedures.
> 
> Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>

Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>

Thanks,
Mani

> Reviewed-by: Rob Herring <robh@kernel.org>
> 
> ---
> 
> Changelog v2:
> - This is a new patch added on v2 iteration of the series.
> ---
>  drivers/pci/controller/dwc/pcie-designware-ep.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/pci/controller/dwc/pcie-designware-ep.c b/drivers/pci/controller/dwc/pcie-designware-ep.c
> index 15b8059544e3..1e35542d6f72 100644
> --- a/drivers/pci/controller/dwc/pcie-designware-ep.c
> +++ b/drivers/pci/controller/dwc/pcie-designware-ep.c
> @@ -704,8 +704,6 @@ int dw_pcie_ep_init(struct dw_pcie_ep *ep)
>  		}
>  	}
>  
> -	dw_pcie_iatu_detect(pci);
> -
>  	res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "addr_space");
>  	if (!res)
>  		return -EINVAL;
> @@ -713,6 +711,8 @@ int dw_pcie_ep_init(struct dw_pcie_ep *ep)
>  	ep->phys_base = res->start;
>  	ep->addr_size = resource_size(res);
>  
> +	dw_pcie_iatu_detect(pci);
> +
>  	ep->ib_window_map = devm_kcalloc(dev,
>  					 BITS_TO_LONGS(pci->num_ib_windows),
>  					 sizeof(long),
> -- 
> 2.35.1
> 

-- 
மணிவண்ணன் சதாசிவம்

[PATCH RESEND v4 03/15] PCI: dwc: Convert to using native IP-core versions representation

[Serge Semin]

Since DWC PCIe v4.70a the controller version can be read from the
PORT_LOGIC.PCIE_VERSION_OFF register. Version is represented in the FourCC
format [1]. It's standard versioning approach for the Synopsys DWC
IP-cores. Moreover some of the DWC kernel drivers already make use of it
to fixup version-dependent functionality (See DWC USB3, Stmicro STMMAC or
recent DW SPI driver). In order to preserve the standard version
representation and prevent the data conversion back and forth, we suggest
to preserve the native version representation in the DWC PCIe driver too
in the same way as it has already been done in the rest of the DWC
drivers. IP-core version reading from the CSR will be introduced in the
next commit together with a simple macro-based API to use it.

[1] https://en.wikipedia.org/wiki/FourCC

Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>
Reviewed-by: Rob Herring <robh@kernel.org>
---
 drivers/pci/controller/dwc/pci-keystone.c    | 12 ++++++------
 drivers/pci/controller/dwc/pcie-designware.c |  8 ++++----
 drivers/pci/controller/dwc/pcie-designware.h | 10 +++++++++-
 drivers/pci/controller/dwc/pcie-intel-gw.c   |  4 ++--
 drivers/pci/controller/dwc/pcie-tegra194.c   |  2 +-
 5 files changed, 22 insertions(+), 14 deletions(-)

diff --git a/drivers/pci/controller/dwc/pci-keystone.c b/drivers/pci/controller/dwc/pci-keystone.c
index c3d88aa27dd4..c4ab3d775a18 100644
--- a/drivers/pci/controller/dwc/pci-keystone.c
+++ b/drivers/pci/controller/dwc/pci-keystone.c
@@ -109,7 +109,7 @@ struct ks_pcie_of_data {
 	enum dw_pcie_device_mode mode;
 	const struct dw_pcie_host_ops *host_ops;
 	const struct dw_pcie_ep_ops *ep_ops;
-	unsigned int version;
+	u32 version;
 };
 
 struct keystone_pcie {
@@ -1069,19 +1069,19 @@ static int ks_pcie_am654_set_mode(struct device *dev,
 
 static const struct ks_pcie_of_data ks_pcie_rc_of_data = {
 	.host_ops = &ks_pcie_host_ops,
-	.version = 0x365A,
+	.version = DW_PCIE_VER_365A,
 };
 
 static const struct ks_pcie_of_data ks_pcie_am654_rc_of_data = {
 	.host_ops = &ks_pcie_am654_host_ops,
 	.mode = DW_PCIE_RC_TYPE,
-	.version = 0x490A,
+	.version = DW_PCIE_VER_490A,
 };
 
 static const struct ks_pcie_of_data ks_pcie_am654_ep_of_data = {
 	.ep_ops = &ks_pcie_am654_ep_ops,
 	.mode = DW_PCIE_EP_TYPE,
-	.version = 0x490A,
+	.version = DW_PCIE_VER_490A,
 };
 
 static const struct of_device_id ks_pcie_of_match[] = {
@@ -1114,12 +1114,12 @@ static int __init ks_pcie_probe(struct platform_device *pdev)
 	struct device_link **link;
 	struct gpio_desc *gpiod;
 	struct resource *res;
-	unsigned int version;
 	void __iomem *base;
 	u32 num_viewport;
 	struct phy **phy;
 	u32 num_lanes;
 	char name[10];
+	u32 version;
 	int ret;
 	int irq;
 	int i;
@@ -1233,7 +1233,7 @@ static int __init ks_pcie_probe(struct platform_device *pdev)
 		goto err_get_sync;
 	}
 
-	if (pci->version >= 0x480A)
+	if (pci->version >= DW_PCIE_VER_480A)
 		ret = ks_pcie_am654_set_mode(dev, mode);
 	else
 		ret = ks_pcie_set_mode(dev);
diff --git a/drivers/pci/controller/dwc/pcie-designware.c b/drivers/pci/controller/dwc/pcie-designware.c
index e66d16a86168..f10a7d5d94e8 100644
--- a/drivers/pci/controller/dwc/pcie-designware.c
+++ b/drivers/pci/controller/dwc/pcie-designware.c
@@ -289,7 +289,7 @@ static void dw_pcie_prog_outbound_atu_unroll(struct dw_pcie *pci, u8 func_no,
 	val = type | PCIE_ATU_FUNC_NUM(func_no);
 	if (upper_32_bits(limit_addr) > upper_32_bits(cpu_addr))
 		val |= PCIE_ATU_INCREASE_REGION_SIZE;
-	if (pci->version == 0x490A)
+	if (pci->version == DW_PCIE_VER_490A)
 		val = dw_pcie_enable_ecrc(val);
 	dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_REGION_CTRL1, val);
 	dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_REGION_CTRL2,
@@ -336,7 +336,7 @@ static void __dw_pcie_prog_outbound_atu(struct dw_pcie *pci, u8 func_no,
 			   upper_32_bits(cpu_addr));
 	dw_pcie_writel_dbi(pci, PCIE_ATU_LIMIT,
 			   lower_32_bits(limit_addr));
-	if (pci->version >= 0x460A)
+	if (pci->version >= DW_PCIE_VER_460A)
 		dw_pcie_writel_dbi(pci, PCIE_ATU_UPPER_LIMIT,
 				   upper_32_bits(limit_addr));
 	dw_pcie_writel_dbi(pci, PCIE_ATU_LOWER_TARGET,
@@ -345,9 +345,9 @@ static void __dw_pcie_prog_outbound_atu(struct dw_pcie *pci, u8 func_no,
 			   upper_32_bits(pci_addr));
 	val = type | PCIE_ATU_FUNC_NUM(func_no);
 	if (upper_32_bits(limit_addr) > upper_32_bits(cpu_addr) &&
-	    pci->version >= 0x460A)
+	    pci->version >= DW_PCIE_VER_460A)
 		val |= PCIE_ATU_INCREASE_REGION_SIZE;
-	if (pci->version == 0x490A)
+	if (pci->version == DW_PCIE_VER_490A)
 		val = dw_pcie_enable_ecrc(val);
 	dw_pcie_writel_dbi(pci, PCIE_ATU_CR1, val);
 	dw_pcie_writel_dbi(pci, PCIE_ATU_CR2, PCIE_ATU_ENABLE);
diff --git a/drivers/pci/controller/dwc/pcie-designware.h b/drivers/pci/controller/dwc/pcie-designware.h
index 32df3ebccf19..6b81530fb2ca 100644
--- a/drivers/pci/controller/dwc/pcie-designware.h
+++ b/drivers/pci/controller/dwc/pcie-designware.h
@@ -20,6 +20,14 @@
 #include <linux/pci-epc.h>
 #include <linux/pci-epf.h>
 
+/* DWC PCIe IP-core versions (native support since v4.70a) */
+#define DW_PCIE_VER_365A		0x3336352a
+#define DW_PCIE_VER_460A		0x3436302a
+#define DW_PCIE_VER_470A		0x3437302a
+#define DW_PCIE_VER_480A		0x3438302a
+#define DW_PCIE_VER_490A		0x3439302a
+#define DW_PCIE_VER_520A		0x3532302a
+
 /* Parameters for the waiting for link up routine */
 #define LINK_WAIT_MAX_RETRIES		10
 #define LINK_WAIT_USLEEP_MIN		90000
@@ -270,7 +278,7 @@ struct dw_pcie {
 	struct dw_pcie_rp	pp;
 	struct dw_pcie_ep	ep;
 	const struct dw_pcie_ops *ops;
-	unsigned int		version;
+	u32			version;
 	int			num_lanes;
 	int			link_gen;
 	u8			n_fts[2];
diff --git a/drivers/pci/controller/dwc/pcie-intel-gw.c b/drivers/pci/controller/dwc/pcie-intel-gw.c
index 07bc54886d71..371b5aa189d1 100644
--- a/drivers/pci/controller/dwc/pcie-intel-gw.c
+++ b/drivers/pci/controller/dwc/pcie-intel-gw.c
@@ -59,7 +59,7 @@
 #define RESET_INTERVAL_MS		100
 
 struct intel_pcie_soc {
-	unsigned int	pcie_ver;
+	u32	pcie_ver;
 };
 
 struct intel_pcie {
@@ -395,7 +395,7 @@ static const struct dw_pcie_host_ops intel_pcie_dw_ops = {
 };
 
 static const struct intel_pcie_soc pcie_data = {
-	.pcie_ver =		0x520A,
+	.pcie_ver =		DW_PCIE_VER_520A,
 };
 
 static int intel_pcie_probe(struct platform_device *pdev)
diff --git a/drivers/pci/controller/dwc/pcie-tegra194.c b/drivers/pci/controller/dwc/pcie-tegra194.c
index 65135f5c4a4a..f24b30b7454f 100644
--- a/drivers/pci/controller/dwc/pcie-tegra194.c
+++ b/drivers/pci/controller/dwc/pcie-tegra194.c
@@ -1979,7 +1979,7 @@ static int tegra194_pcie_probe(struct platform_device *pdev)
 	pci->ops = &tegra_dw_pcie_ops;
 	pci->n_fts[0] = N_FTS_VAL;
 	pci->n_fts[1] = FTS_VAL;
-	pci->version = 0x490A;
+	pci->version = DW_PCIE_VER_490A;
 
 	pp = &pci->pp;
 	pp->num_vectors = MAX_MSI_IRQS;
-- 
2.35.1

Re: [PATCH RESEND v4 03/15] PCI: dwc: Convert to using native IP-core versions representation

[Bjorn Helgaas]

On Fri, Jun 24, 2022 at 05:39:35PM +0300, Serge Semin wrote:
> Since DWC PCIe v4.70a the controller version can be read from the
> PORT_LOGIC.PCIE_VERSION_OFF register. Version is represented in the FourCC
> format [1]. It's standard versioning approach for the Synopsys DWC
> IP-cores. Moreover some of the DWC kernel drivers already make use of it
> to fixup version-dependent functionality (See DWC USB3, Stmicro STMMAC or
> recent DW SPI driver).

These references to other drivers might be useful, but without a
function name or file name, I can't easily find them.

> In order to preserve the standard version
> representation and prevent the data conversion back and forth, we suggest
> to preserve the native version representation in the DWC PCIe driver too
> in the same way as it has already been done in the rest of the DWC
> drivers. IP-core version reading from the CSR will be introduced in the
> next commit together with a simple macro-based API to use it.
> 
> [1] https://en.wikipedia.org/wiki/FourCC

Re: [PATCH RESEND v4 03/15] PCI: dwc: Convert to using native IP-core versions representation

[Ben Dooks]

On 28/07/2022 16:24, Bjorn Helgaas wrote:
> On Fri, Jun 24, 2022 at 05:39:35PM +0300, Serge Semin wrote:
>> Since DWC PCIe v4.70a the controller version can be read from the
>> PORT_LOGIC.PCIE_VERSION_OFF register. Version is represented in the FourCC
>> format [1]. It's standard versioning approach for the Synopsys DWC
>> IP-cores. Moreover some of the DWC kernel drivers already make use of it
>> to fixup version-dependent functionality (See DWC USB3, Stmicro STMMAC or
>> recent DW SPI driver).
> 
> These references to other drivers might be useful, but without a
> function name or file name, I can't easily find them.
> 
>> In order to preserve the standard version
>> representation and prevent the data conversion back and forth, we suggest
>> to preserve the native version representation in the DWC PCIe driver too
>> in the same way as it has already been done in the rest of the DWC
>> drivers. IP-core version reading from the CSR will be introduced in the
>> next commit together with a simple macro-based API to use it.
>>
>> [1] https://en.wikipedia.org/wiki/FourCC
> 

I'm currently looking at a OF based dw-apb-timers-pwm driver, so also
would like to follow this.

-- 
Ben Dooks				http://www.codethink.co.uk/
Senior Engineer				Codethink - Providing Genius

https://www.codethink.co.uk/privacy.html

Re: [PATCH RESEND v4 03/15] PCI: dwc: Convert to using native IP-core versions representation

[Bjorn Helgaas]

On Thu, Jul 28, 2022 at 04:31:17PM +0100, Ben Dooks wrote:
> On 28/07/2022 16:24, Bjorn Helgaas wrote:
> > On Fri, Jun 24, 2022 at 05:39:35PM +0300, Serge Semin wrote:
> > > Since DWC PCIe v4.70a the controller version can be read from the
> > > PORT_LOGIC.PCIE_VERSION_OFF register. Version is represented in the FourCC
> > > format [1]. It's standard versioning approach for the Synopsys DWC
> > > IP-cores. Moreover some of the DWC kernel drivers already make use of it
> > > to fixup version-dependent functionality (See DWC USB3, Stmicro STMMAC or
> > > recent DW SPI driver).
> > 
> > These references to other drivers might be useful, but without a
> > function name or file name, I can't easily find them.
> > 
> > > In order to preserve the standard version
> > > representation and prevent the data conversion back and forth, we suggest
> > > to preserve the native version representation in the DWC PCIe driver too
> > > in the same way as it has already been done in the rest of the DWC
> > > drivers. IP-core version reading from the CSR will be introduced in the
> > > next commit together with a simple macro-based API to use it.
> > > 
> > > [1] https://en.wikipedia.org/wiki/FourCC
> 
> I'm currently looking at a OF based dw-apb-timers-pwm driver, so also
> would like to follow this.

FWIW, the breadcrumbs I found are:

  dwc3_core_is_valid()
  stmmac_hwif_init()
  dw_spi_hw_init()

Re: [PATCH RESEND v4 03/15] PCI: dwc: Convert to using native IP-core versions representation

[Serge Semin]

On Thu, Jul 28, 2022 at 10:24:18AM -0500, Bjorn Helgaas wrote:
> On Fri, Jun 24, 2022 at 05:39:35PM +0300, Serge Semin wrote:
> > Since DWC PCIe v4.70a the controller version can be read from the
> > PORT_LOGIC.PCIE_VERSION_OFF register. Version is represented in the FourCC
> > format [1]. It's standard versioning approach for the Synopsys DWC
> > IP-cores. Moreover some of the DWC kernel drivers already make use of it
> > to fixup version-dependent functionality (See DWC USB3, Stmicro STMMAC or
> > recent DW SPI driver).
> 

> These references to other drivers might be useful, but without a
> function name or file name, I can't easily find them.

1. DW APB SSI driver, defined in drivers/spi/spi-dw.h:
#define DW_HSSI_102A                    0x3130322a
#define dw_spi_ip_is(_dws, _ip) ...
#define dw_spi_ver_is(_dws, _ip, _ver) ...
#define dw_spi_ver_is_ge(_dws, _ip, _ver) ...

2. DWC USB3 driver, defined in drivers/usb/dwc3/core.h
#define DWC3_IP                 0x5533
#define DWC31_IP                0x3331
#define DWC32_IP                0x3332
#define DWC3_REVISION_173A      0x5533173a
#define DWC3_REVISION_175A      0x5533175a
...
#define DWC31_REVISION_110A     0x3131302a
#define DWC31_REVISION_120A     0x3132302a
...
#define DWC3_IP_IS(_ip) ...
#define DWC3_VER_IS(_ip, _ver) ...
#define DWC3_VER_IS_PRIOR(_ip, _ver) ...
#define DWC3_VER_IS_WITHIN(_ip, _from, _to) ...
#define DWC3_VER_TYPE_IS_WITHIN(_ip, _ver, _from, _to) ...

Regarding the STMMAC (DW MAC/GMAC/xGMAC) driver. I've harried up to
claim it has the native IP-core versioning support. The current kernel
driver doesn't have it. Instead I have it implemented in my local
repo, but it isn't ready to be submitted yet. I need some more time to
finish my DW GMAC/xGMAC work first.

Ideally we could have created a common interface for all the drivers.
I thought about it first when I was initially creating the patchsets.
But now I don't time left for this at all. The review process's taken
way much more time than I had/planned to spend for the PCIe-patches.
So what I can do at this stage is to provide minor fixes if required.

-Sergey

> 
> > In order to preserve the standard version
> > representation and prevent the data conversion back and forth, we suggest
> > to preserve the native version representation in the DWC PCIe driver too
> > in the same way as it has already been done in the rest of the DWC
> > drivers. IP-core version reading from the CSR will be introduced in the
> > next commit together with a simple macro-based API to use it.
> > 
> > [1] https://en.wikipedia.org/wiki/FourCC

Re: [PATCH RESEND v4 03/15] PCI: dwc: Convert to using native IP-core versions representation

[Manivannan Sadhasivam]

On Fri, Jun 24, 2022 at 05:39:35PM +0300, Serge Semin wrote:
> Since DWC PCIe v4.70a the controller version can be read from the
> PORT_LOGIC.PCIE_VERSION_OFF register. Version is represented in the FourCC
> format [1]. It's standard versioning approach for the Synopsys DWC
> IP-cores. Moreover some of the DWC kernel drivers already make use of it
> to fixup version-dependent functionality (See DWC USB3, Stmicro STMMAC or
> recent DW SPI driver). In order to preserve the standard version
> representation and prevent the data conversion back and forth, we suggest
> to preserve the native version representation in the DWC PCIe driver too
> in the same way as it has already been done in the rest of the DWC
> drivers. IP-core version reading from the CSR will be introduced in the
> next commit together with a simple macro-based API to use it.
> 
> [1] https://en.wikipedia.org/wiki/FourCC
> 
> Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>
> Reviewed-by: Rob Herring <robh@kernel.org>

Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>

Thanks,
Mani

> ---
>  drivers/pci/controller/dwc/pci-keystone.c    | 12 ++++++------
>  drivers/pci/controller/dwc/pcie-designware.c |  8 ++++----
>  drivers/pci/controller/dwc/pcie-designware.h | 10 +++++++++-
>  drivers/pci/controller/dwc/pcie-intel-gw.c   |  4 ++--
>  drivers/pci/controller/dwc/pcie-tegra194.c   |  2 +-
>  5 files changed, 22 insertions(+), 14 deletions(-)
> 
> diff --git a/drivers/pci/controller/dwc/pci-keystone.c b/drivers/pci/controller/dwc/pci-keystone.c
> index c3d88aa27dd4..c4ab3d775a18 100644
> --- a/drivers/pci/controller/dwc/pci-keystone.c
> +++ b/drivers/pci/controller/dwc/pci-keystone.c
> @@ -109,7 +109,7 @@ struct ks_pcie_of_data {
>  	enum dw_pcie_device_mode mode;
>  	const struct dw_pcie_host_ops *host_ops;
>  	const struct dw_pcie_ep_ops *ep_ops;
> -	unsigned int version;
> +	u32 version;
>  };
>  
>  struct keystone_pcie {
> @@ -1069,19 +1069,19 @@ static int ks_pcie_am654_set_mode(struct device *dev,
>  
>  static const struct ks_pcie_of_data ks_pcie_rc_of_data = {
>  	.host_ops = &ks_pcie_host_ops,
> -	.version = 0x365A,
> +	.version = DW_PCIE_VER_365A,
>  };
>  
>  static const struct ks_pcie_of_data ks_pcie_am654_rc_of_data = {
>  	.host_ops = &ks_pcie_am654_host_ops,
>  	.mode = DW_PCIE_RC_TYPE,
> -	.version = 0x490A,
> +	.version = DW_PCIE_VER_490A,
>  };
>  
>  static const struct ks_pcie_of_data ks_pcie_am654_ep_of_data = {
>  	.ep_ops = &ks_pcie_am654_ep_ops,
>  	.mode = DW_PCIE_EP_TYPE,
> -	.version = 0x490A,
> +	.version = DW_PCIE_VER_490A,
>  };
>  
>  static const struct of_device_id ks_pcie_of_match[] = {
> @@ -1114,12 +1114,12 @@ static int __init ks_pcie_probe(struct platform_device *pdev)
>  	struct device_link **link;
>  	struct gpio_desc *gpiod;
>  	struct resource *res;
> -	unsigned int version;
>  	void __iomem *base;
>  	u32 num_viewport;
>  	struct phy **phy;
>  	u32 num_lanes;
>  	char name[10];
> +	u32 version;
>  	int ret;
>  	int irq;
>  	int i;
> @@ -1233,7 +1233,7 @@ static int __init ks_pcie_probe(struct platform_device *pdev)
>  		goto err_get_sync;
>  	}
>  
> -	if (pci->version >= 0x480A)
> +	if (pci->version >= DW_PCIE_VER_480A)
>  		ret = ks_pcie_am654_set_mode(dev, mode);
>  	else
>  		ret = ks_pcie_set_mode(dev);
> diff --git a/drivers/pci/controller/dwc/pcie-designware.c b/drivers/pci/controller/dwc/pcie-designware.c
> index e66d16a86168..f10a7d5d94e8 100644
> --- a/drivers/pci/controller/dwc/pcie-designware.c
> +++ b/drivers/pci/controller/dwc/pcie-designware.c
> @@ -289,7 +289,7 @@ static void dw_pcie_prog_outbound_atu_unroll(struct dw_pcie *pci, u8 func_no,
>  	val = type | PCIE_ATU_FUNC_NUM(func_no);
>  	if (upper_32_bits(limit_addr) > upper_32_bits(cpu_addr))
>  		val |= PCIE_ATU_INCREASE_REGION_SIZE;
> -	if (pci->version == 0x490A)
> +	if (pci->version == DW_PCIE_VER_490A)
>  		val = dw_pcie_enable_ecrc(val);
>  	dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_REGION_CTRL1, val);
>  	dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_REGION_CTRL2,
> @@ -336,7 +336,7 @@ static void __dw_pcie_prog_outbound_atu(struct dw_pcie *pci, u8 func_no,
>  			   upper_32_bits(cpu_addr));
>  	dw_pcie_writel_dbi(pci, PCIE_ATU_LIMIT,
>  			   lower_32_bits(limit_addr));
> -	if (pci->version >= 0x460A)
> +	if (pci->version >= DW_PCIE_VER_460A)
>  		dw_pcie_writel_dbi(pci, PCIE_ATU_UPPER_LIMIT,
>  				   upper_32_bits(limit_addr));
>  	dw_pcie_writel_dbi(pci, PCIE_ATU_LOWER_TARGET,
> @@ -345,9 +345,9 @@ static void __dw_pcie_prog_outbound_atu(struct dw_pcie *pci, u8 func_no,
>  			   upper_32_bits(pci_addr));
>  	val = type | PCIE_ATU_FUNC_NUM(func_no);
>  	if (upper_32_bits(limit_addr) > upper_32_bits(cpu_addr) &&
> -	    pci->version >= 0x460A)
> +	    pci->version >= DW_PCIE_VER_460A)
>  		val |= PCIE_ATU_INCREASE_REGION_SIZE;
> -	if (pci->version == 0x490A)
> +	if (pci->version == DW_PCIE_VER_490A)
>  		val = dw_pcie_enable_ecrc(val);
>  	dw_pcie_writel_dbi(pci, PCIE_ATU_CR1, val);
>  	dw_pcie_writel_dbi(pci, PCIE_ATU_CR2, PCIE_ATU_ENABLE);
> diff --git a/drivers/pci/controller/dwc/pcie-designware.h b/drivers/pci/controller/dwc/pcie-designware.h
> index 32df3ebccf19..6b81530fb2ca 100644
> --- a/drivers/pci/controller/dwc/pcie-designware.h
> +++ b/drivers/pci/controller/dwc/pcie-designware.h
> @@ -20,6 +20,14 @@
>  #include <linux/pci-epc.h>
>  #include <linux/pci-epf.h>
>  
> +/* DWC PCIe IP-core versions (native support since v4.70a) */
> +#define DW_PCIE_VER_365A		0x3336352a
> +#define DW_PCIE_VER_460A		0x3436302a
> +#define DW_PCIE_VER_470A		0x3437302a
> +#define DW_PCIE_VER_480A		0x3438302a
> +#define DW_PCIE_VER_490A		0x3439302a
> +#define DW_PCIE_VER_520A		0x3532302a
> +
>  /* Parameters for the waiting for link up routine */
>  #define LINK_WAIT_MAX_RETRIES		10
>  #define LINK_WAIT_USLEEP_MIN		90000
> @@ -270,7 +278,7 @@ struct dw_pcie {
>  	struct dw_pcie_rp	pp;
>  	struct dw_pcie_ep	ep;
>  	const struct dw_pcie_ops *ops;
> -	unsigned int		version;
> +	u32			version;
>  	int			num_lanes;
>  	int			link_gen;
>  	u8			n_fts[2];
> diff --git a/drivers/pci/controller/dwc/pcie-intel-gw.c b/drivers/pci/controller/dwc/pcie-intel-gw.c
> index 07bc54886d71..371b5aa189d1 100644
> --- a/drivers/pci/controller/dwc/pcie-intel-gw.c
> +++ b/drivers/pci/controller/dwc/pcie-intel-gw.c
> @@ -59,7 +59,7 @@
>  #define RESET_INTERVAL_MS		100
>  
>  struct intel_pcie_soc {
> -	unsigned int	pcie_ver;
> +	u32	pcie_ver;
>  };
>  
>  struct intel_pcie {
> @@ -395,7 +395,7 @@ static const struct dw_pcie_host_ops intel_pcie_dw_ops = {
>  };
>  
>  static const struct intel_pcie_soc pcie_data = {
> -	.pcie_ver =		0x520A,
> +	.pcie_ver =		DW_PCIE_VER_520A,
>  };
>  
>  static int intel_pcie_probe(struct platform_device *pdev)
> diff --git a/drivers/pci/controller/dwc/pcie-tegra194.c b/drivers/pci/controller/dwc/pcie-tegra194.c
> index 65135f5c4a4a..f24b30b7454f 100644
> --- a/drivers/pci/controller/dwc/pcie-tegra194.c
> +++ b/drivers/pci/controller/dwc/pcie-tegra194.c
> @@ -1979,7 +1979,7 @@ static int tegra194_pcie_probe(struct platform_device *pdev)
>  	pci->ops = &tegra_dw_pcie_ops;
>  	pci->n_fts[0] = N_FTS_VAL;
>  	pci->n_fts[1] = FTS_VAL;
> -	pci->version = 0x490A;
> +	pci->version = DW_PCIE_VER_490A;
>  
>  	pp = &pci->pp;
>  	pp->num_vectors = MAX_MSI_IRQS;
> -- 
> 2.35.1
> 

-- 
மணிவண்ணன் சதாசிவம்

[PATCH RESEND v4 04/15] PCI: dwc: Add IP-core version detection procedure

[Serge Semin]

Since DWC PCIe v4.70a the controller version and version type can be read
from the PORT_LOGIC.PCIE_VERSION_OFF and PORT_LOGIC.PCIE_VERSION_TYPE_OFF
registers respectively. Seeing the generic code has got version-dependent
parts let's use these registers to find out the controller version.  The
detection procedure is executed for both RC and EP modes right after the
platform-specific initialization. We can't do that earlier since the
glue-drivers can perform the DBI-related setups there including the bus
reference clocks activation, without which the CSRs just can't be read.

Note the CSRs content is zero on the older DWC PCIe controller. In that
case we have no choice but to rely on the platform setup.

Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>
Reviewed-by: Rob Herring <robh@kernel.org>

---

Changelog v2:
- Move the IP-core version detection procedure call from
  dw_pcie_ep_init_complete() to dw_pcie_ep_init().
---
 .../pci/controller/dwc/pcie-designware-ep.c   |  2 ++
 .../pci/controller/dwc/pcie-designware-host.c |  2 ++
 drivers/pci/controller/dwc/pcie-designware.c  | 24 +++++++++++++++++++
 drivers/pci/controller/dwc/pcie-designware.h  |  6 +++++
 4 files changed, 34 insertions(+)

diff --git a/drivers/pci/controller/dwc/pcie-designware-ep.c b/drivers/pci/controller/dwc/pcie-designware-ep.c
index 1e35542d6f72..ffbd3af6d65a 100644
--- a/drivers/pci/controller/dwc/pcie-designware-ep.c
+++ b/drivers/pci/controller/dwc/pcie-designware-ep.c
@@ -711,6 +711,8 @@ int dw_pcie_ep_init(struct dw_pcie_ep *ep)
 	ep->phys_base = res->start;
 	ep->addr_size = resource_size(res);
 
+	dw_pcie_version_detect(pci);
+
 	dw_pcie_iatu_detect(pci);
 
 	ep->ib_window_map = devm_kcalloc(dev,
diff --git a/drivers/pci/controller/dwc/pcie-designware-host.c b/drivers/pci/controller/dwc/pcie-designware-host.c
index 95256434913f..b1437b37140f 100644
--- a/drivers/pci/controller/dwc/pcie-designware-host.c
+++ b/drivers/pci/controller/dwc/pcie-designware-host.c
@@ -405,6 +405,8 @@ int dw_pcie_host_init(struct dw_pcie_rp *pp)
 		}
 	}
 
+	dw_pcie_version_detect(pci);
+
 	dw_pcie_iatu_detect(pci);
 
 	dw_pcie_setup_rc(pp);
diff --git a/drivers/pci/controller/dwc/pcie-designware.c b/drivers/pci/controller/dwc/pcie-designware.c
index f10a7d5d94e8..cbb36ccaa48b 100644
--- a/drivers/pci/controller/dwc/pcie-designware.c
+++ b/drivers/pci/controller/dwc/pcie-designware.c
@@ -16,6 +16,30 @@
 #include "../../pci.h"
 #include "pcie-designware.h"
 
+void dw_pcie_version_detect(struct dw_pcie *pci)
+{
+	u32 ver;
+
+	/* The content of the CSR is zero on DWC PCIe older than v4.70a */
+	ver = dw_pcie_readl_dbi(pci, PCIE_VERSION_NUMBER);
+	if (!ver)
+		return;
+
+	if (pci->version && pci->version != ver)
+		dev_warn(pci->dev, "Versions don't match (%08x != %08x)\n",
+			 pci->version, ver);
+	else
+		pci->version = ver;
+
+	ver = dw_pcie_readl_dbi(pci, PCIE_VERSION_TYPE);
+
+	if (pci->type && pci->type != ver)
+		dev_warn(pci->dev, "Types don't match (%08x != %08x)\n",
+			 pci->type, ver);
+	else
+		pci->type = ver;
+}
+
 /*
  * These interfaces resemble the pci_find_*capability() interfaces, but these
  * are for configuring host controllers, which are bridges *to* PCI devices but
diff --git a/drivers/pci/controller/dwc/pcie-designware.h b/drivers/pci/controller/dwc/pcie-designware.h
index 6b81530fb2ca..7899808bdbc6 100644
--- a/drivers/pci/controller/dwc/pcie-designware.h
+++ b/drivers/pci/controller/dwc/pcie-designware.h
@@ -85,6 +85,9 @@
 #define PCIE_PORT_MULTI_LANE_CTRL	0x8C0
 #define PORT_MLTI_UPCFG_SUPPORT		BIT(7)
 
+#define PCIE_VERSION_NUMBER		0x8F8
+#define PCIE_VERSION_TYPE		0x8FC
+
 #define PCIE_ATU_VIEWPORT		0x900
 #define PCIE_ATU_REGION_INBOUND		BIT(31)
 #define PCIE_ATU_REGION_OUTBOUND	0
@@ -279,6 +282,7 @@ struct dw_pcie {
 	struct dw_pcie_ep	ep;
 	const struct dw_pcie_ops *ops;
 	u32			version;
+	u32			type;
 	int			num_lanes;
 	int			link_gen;
 	u8			n_fts[2];
@@ -290,6 +294,8 @@ struct dw_pcie {
 #define to_dw_pcie_from_ep(endpoint)   \
 		container_of((endpoint), struct dw_pcie, ep)
 
+void dw_pcie_version_detect(struct dw_pcie *pci);
+
 u8 dw_pcie_find_capability(struct dw_pcie *pci, u8 cap);
 u16 dw_pcie_find_ext_capability(struct dw_pcie *pci, u8 cap);
 
-- 
2.35.1

Re: [PATCH RESEND v4 04/15] PCI: dwc: Add IP-core version detection procedure

[Manivannan Sadhasivam]

On Fri, Jun 24, 2022 at 05:39:36PM +0300, Serge Semin wrote:
> Since DWC PCIe v4.70a the controller version and version type can be read
> from the PORT_LOGIC.PCIE_VERSION_OFF and PORT_LOGIC.PCIE_VERSION_TYPE_OFF
> registers respectively. Seeing the generic code has got version-dependent
> parts let's use these registers to find out the controller version.  The
> detection procedure is executed for both RC and EP modes right after the
> platform-specific initialization. We can't do that earlier since the
> glue-drivers can perform the DBI-related setups there including the bus
> reference clocks activation, without which the CSRs just can't be read.
> 
> Note the CSRs content is zero on the older DWC PCIe controller. In that
> case we have no choice but to rely on the platform setup.
> 
> Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>

Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>

> Reviewed-by: Rob Herring <robh@kernel.org>
> 
> ---
> 
> Changelog v2:
> - Move the IP-core version detection procedure call from
>   dw_pcie_ep_init_complete() to dw_pcie_ep_init().
> ---
>  .../pci/controller/dwc/pcie-designware-ep.c   |  2 ++
>  .../pci/controller/dwc/pcie-designware-host.c |  2 ++
>  drivers/pci/controller/dwc/pcie-designware.c  | 24 +++++++++++++++++++
>  drivers/pci/controller/dwc/pcie-designware.h  |  6 +++++
>  4 files changed, 34 insertions(+)
> 
> diff --git a/drivers/pci/controller/dwc/pcie-designware-ep.c b/drivers/pci/controller/dwc/pcie-designware-ep.c
> index 1e35542d6f72..ffbd3af6d65a 100644
> --- a/drivers/pci/controller/dwc/pcie-designware-ep.c
> +++ b/drivers/pci/controller/dwc/pcie-designware-ep.c
> @@ -711,6 +711,8 @@ int dw_pcie_ep_init(struct dw_pcie_ep *ep)
>  	ep->phys_base = res->start;
>  	ep->addr_size = resource_size(res);
>  
> +	dw_pcie_version_detect(pci);
> +

There is still an ongoing debate about moving all DBI accesses to
init_complete. But this is fine atm.

Thanks,
Mani

>  	dw_pcie_iatu_detect(pci);
>  
>  	ep->ib_window_map = devm_kcalloc(dev,
> diff --git a/drivers/pci/controller/dwc/pcie-designware-host.c b/drivers/pci/controller/dwc/pcie-designware-host.c
> index 95256434913f..b1437b37140f 100644
> --- a/drivers/pci/controller/dwc/pcie-designware-host.c
> +++ b/drivers/pci/controller/dwc/pcie-designware-host.c
> @@ -405,6 +405,8 @@ int dw_pcie_host_init(struct dw_pcie_rp *pp)
>  		}
>  	}
>  
> +	dw_pcie_version_detect(pci);
> +
>  	dw_pcie_iatu_detect(pci);
>  
>  	dw_pcie_setup_rc(pp);
> diff --git a/drivers/pci/controller/dwc/pcie-designware.c b/drivers/pci/controller/dwc/pcie-designware.c
> index f10a7d5d94e8..cbb36ccaa48b 100644
> --- a/drivers/pci/controller/dwc/pcie-designware.c
> +++ b/drivers/pci/controller/dwc/pcie-designware.c
> @@ -16,6 +16,30 @@
>  #include "../../pci.h"
>  #include "pcie-designware.h"
>  
> +void dw_pcie_version_detect(struct dw_pcie *pci)
> +{
> +	u32 ver;
> +
> +	/* The content of the CSR is zero on DWC PCIe older than v4.70a */
> +	ver = dw_pcie_readl_dbi(pci, PCIE_VERSION_NUMBER);
> +	if (!ver)
> +		return;
> +
> +	if (pci->version && pci->version != ver)
> +		dev_warn(pci->dev, "Versions don't match (%08x != %08x)\n",
> +			 pci->version, ver);
> +	else
> +		pci->version = ver;
> +
> +	ver = dw_pcie_readl_dbi(pci, PCIE_VERSION_TYPE);
> +
> +	if (pci->type && pci->type != ver)
> +		dev_warn(pci->dev, "Types don't match (%08x != %08x)\n",
> +			 pci->type, ver);
> +	else
> +		pci->type = ver;
> +}
> +
>  /*
>   * These interfaces resemble the pci_find_*capability() interfaces, but these
>   * are for configuring host controllers, which are bridges *to* PCI devices but
> diff --git a/drivers/pci/controller/dwc/pcie-designware.h b/drivers/pci/controller/dwc/pcie-designware.h
> index 6b81530fb2ca..7899808bdbc6 100644
> --- a/drivers/pci/controller/dwc/pcie-designware.h
> +++ b/drivers/pci/controller/dwc/pcie-designware.h
> @@ -85,6 +85,9 @@
>  #define PCIE_PORT_MULTI_LANE_CTRL	0x8C0
>  #define PORT_MLTI_UPCFG_SUPPORT		BIT(7)
>  
> +#define PCIE_VERSION_NUMBER		0x8F8
> +#define PCIE_VERSION_TYPE		0x8FC
> +
>  #define PCIE_ATU_VIEWPORT		0x900
>  #define PCIE_ATU_REGION_INBOUND		BIT(31)
>  #define PCIE_ATU_REGION_OUTBOUND	0
> @@ -279,6 +282,7 @@ struct dw_pcie {
>  	struct dw_pcie_ep	ep;
>  	const struct dw_pcie_ops *ops;
>  	u32			version;
> +	u32			type;
>  	int			num_lanes;
>  	int			link_gen;
>  	u8			n_fts[2];
> @@ -290,6 +294,8 @@ struct dw_pcie {
>  #define to_dw_pcie_from_ep(endpoint)   \
>  		container_of((endpoint), struct dw_pcie, ep)
>  
> +void dw_pcie_version_detect(struct dw_pcie *pci);
> +
>  u8 dw_pcie_find_capability(struct dw_pcie *pci, u8 cap);
>  u16 dw_pcie_find_ext_capability(struct dw_pcie *pci, u8 cap);
>  
> -- 
> 2.35.1
> 

-- 
மணிவண்ணன் சதாசிவம்

Re: [PATCH RESEND v4 04/15] PCI: dwc: Add IP-core version detection procedure

[Bjorn Helgaas]

On Mon, Aug 01, 2022 at 06:42:19PM +0530, Manivannan Sadhasivam wrote:
> On Fri, Jun 24, 2022 at 05:39:36PM +0300, Serge Semin wrote:
> > Since DWC PCIe v4.70a the controller version and version type can be read
> > from the PORT_LOGIC.PCIE_VERSION_OFF and PORT_LOGIC.PCIE_VERSION_TYPE_OFF
> > registers respectively. Seeing the generic code has got version-dependent
> > parts let's use these registers to find out the controller version.  The
> > detection procedure is executed for both RC and EP modes right after the
> > platform-specific initialization. We can't do that earlier since the
> > glue-drivers can perform the DBI-related setups there including the bus
> > reference clocks activation, without which the CSRs just can't be read.
> > 
> > Note the CSRs content is zero on the older DWC PCIe controller. In that
> > case we have no choice but to rely on the platform setup.
> > 
> > Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>
> 
> Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
> 
> > Reviewed-by: Rob Herring <robh@kernel.org>

> > @@ -711,6 +711,8 @@ int dw_pcie_ep_init(struct dw_pcie_ep *ep)
> >  	ep->phys_base = res->start;
> >  	ep->addr_size = resource_size(res);
> >  
> > +	dw_pcie_version_detect(pci);
> > +
> 
> There is still an ongoing debate about moving all DBI accesses to
> init_complete. But this is fine atm.

Well, if I understand it correctly, e966f7390da9 ("PCI: dwc: Refactor
core initialization code for EP mode") claims that all DBI accesses
should be in dw_pcie_ep_init_complete(), so it's not so much a debate
as a discussion about how best to achieve that.

But you're right, we can fix that up later if necessary.

> >  	dw_pcie_iatu_detect(pci);

Re: [PATCH RESEND v4 04/15] PCI: dwc: Add IP-core version detection procedure

[Manivannan Sadhasivam]

On Mon, Aug 01, 2022 at 03:06:06PM -0500, Bjorn Helgaas wrote:
> On Mon, Aug 01, 2022 at 06:42:19PM +0530, Manivannan Sadhasivam wrote:
> > On Fri, Jun 24, 2022 at 05:39:36PM +0300, Serge Semin wrote:
> > > Since DWC PCIe v4.70a the controller version and version type can be read
> > > from the PORT_LOGIC.PCIE_VERSION_OFF and PORT_LOGIC.PCIE_VERSION_TYPE_OFF
> > > registers respectively. Seeing the generic code has got version-dependent
> > > parts let's use these registers to find out the controller version.  The
> > > detection procedure is executed for both RC and EP modes right after the
> > > platform-specific initialization. We can't do that earlier since the
> > > glue-drivers can perform the DBI-related setups there including the bus
> > > reference clocks activation, without which the CSRs just can't be read.
> > > 
> > > Note the CSRs content is zero on the older DWC PCIe controller. In that
> > > case we have no choice but to rely on the platform setup.
> > > 
> > > Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>
> > 
> > Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
> > 
> > > Reviewed-by: Rob Herring <robh@kernel.org>
> 
> > > @@ -711,6 +711,8 @@ int dw_pcie_ep_init(struct dw_pcie_ep *ep)
> > >  	ep->phys_base = res->start;
> > >  	ep->addr_size = resource_size(res);
> > >  
> > > +	dw_pcie_version_detect(pci);
> > > +
> > 
> > There is still an ongoing debate about moving all DBI accesses to
> > init_complete. But this is fine atm.
> 
> Well, if I understand it correctly, e966f7390da9 ("PCI: dwc: Refactor
> core initialization code for EP mode") claims that all DBI accesses
> should be in dw_pcie_ep_init_complete(), so it's not so much a debate
> as a discussion about how best to achieve that.
> 

Glad to know that we are on the same page. Let's continue the discussion in
that thread.

Thanks,
Mani

> But you're right, we can fix that up later if necessary.
> 
> > >  	dw_pcie_iatu_detect(pci);

-- 
மணிவண்ணன் சதாசிவம்

[PATCH RESEND v4 05/15] PCI: dwc: Introduce Synopsys IP-core versions/types interface

[Serge Semin]

Instead of manual DW PCIe data version field comparison let's use a handy
macro-based interface in order to shorten out the statements, simplify the
corresponding parts, improve the code readability and maintainability in
perspective when more complex version-based dependencies need to
implemented. Similar approaches have already been implemented in the DWC
USB3 and DW SPI drivers (though with some IP-core evolution specifics).

Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>
Reviewed-by: Rob Herring <robh@kernel.org>
---
 drivers/pci/controller/dwc/pci-keystone.c    |  2 +-
 drivers/pci/controller/dwc/pcie-designware.c |  8 ++++----
 drivers/pci/controller/dwc/pcie-designware.h | 15 +++++++++++++++
 3 files changed, 20 insertions(+), 5 deletions(-)

diff --git a/drivers/pci/controller/dwc/pci-keystone.c b/drivers/pci/controller/dwc/pci-keystone.c
index c4ab3d775a18..2a9bbde224af 100644
--- a/drivers/pci/controller/dwc/pci-keystone.c
+++ b/drivers/pci/controller/dwc/pci-keystone.c
@@ -1233,7 +1233,7 @@ static int __init ks_pcie_probe(struct platform_device *pdev)
 		goto err_get_sync;
 	}
 
-	if (pci->version >= DW_PCIE_VER_480A)
+	if (dw_pcie_ver_is_ge(pci, 480A))
 		ret = ks_pcie_am654_set_mode(dev, mode);
 	else
 		ret = ks_pcie_set_mode(dev);
diff --git a/drivers/pci/controller/dwc/pcie-designware.c b/drivers/pci/controller/dwc/pcie-designware.c
index cbb36ccaa48b..bd575ad32bc4 100644
--- a/drivers/pci/controller/dwc/pcie-designware.c
+++ b/drivers/pci/controller/dwc/pcie-designware.c
@@ -313,7 +313,7 @@ static void dw_pcie_prog_outbound_atu_unroll(struct dw_pcie *pci, u8 func_no,
 	val = type | PCIE_ATU_FUNC_NUM(func_no);
 	if (upper_32_bits(limit_addr) > upper_32_bits(cpu_addr))
 		val |= PCIE_ATU_INCREASE_REGION_SIZE;
-	if (pci->version == DW_PCIE_VER_490A)
+	if (dw_pcie_ver_is(pci, 490A))
 		val = dw_pcie_enable_ecrc(val);
 	dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_REGION_CTRL1, val);
 	dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_REGION_CTRL2,
@@ -360,7 +360,7 @@ static void __dw_pcie_prog_outbound_atu(struct dw_pcie *pci, u8 func_no,
 			   upper_32_bits(cpu_addr));
 	dw_pcie_writel_dbi(pci, PCIE_ATU_LIMIT,
 			   lower_32_bits(limit_addr));
-	if (pci->version >= DW_PCIE_VER_460A)
+	if (dw_pcie_ver_is_ge(pci, 460A))
 		dw_pcie_writel_dbi(pci, PCIE_ATU_UPPER_LIMIT,
 				   upper_32_bits(limit_addr));
 	dw_pcie_writel_dbi(pci, PCIE_ATU_LOWER_TARGET,
@@ -369,9 +369,9 @@ static void __dw_pcie_prog_outbound_atu(struct dw_pcie *pci, u8 func_no,
 			   upper_32_bits(pci_addr));
 	val = type | PCIE_ATU_FUNC_NUM(func_no);
 	if (upper_32_bits(limit_addr) > upper_32_bits(cpu_addr) &&
-	    pci->version >= DW_PCIE_VER_460A)
+	    dw_pcie_ver_is_ge(pci, 460A))
 		val |= PCIE_ATU_INCREASE_REGION_SIZE;
-	if (pci->version == DW_PCIE_VER_490A)
+	if (dw_pcie_ver_is(pci, 490A))
 		val = dw_pcie_enable_ecrc(val);
 	dw_pcie_writel_dbi(pci, PCIE_ATU_CR1, val);
 	dw_pcie_writel_dbi(pci, PCIE_ATU_CR2, PCIE_ATU_ENABLE);
diff --git a/drivers/pci/controller/dwc/pcie-designware.h b/drivers/pci/controller/dwc/pcie-designware.h
index 7899808bdbc6..d247f227464c 100644
--- a/drivers/pci/controller/dwc/pcie-designware.h
+++ b/drivers/pci/controller/dwc/pcie-designware.h
@@ -28,6 +28,21 @@
 #define DW_PCIE_VER_490A		0x3439302a
 #define DW_PCIE_VER_520A		0x3532302a
 
+#define __dw_pcie_ver_cmp(_pci, _ver, _op) \
+	((_pci)->version _op DW_PCIE_VER_ ## _ver)
+
+#define dw_pcie_ver_is(_pci, _ver) __dw_pcie_ver_cmp(_pci, _ver, ==)
+
+#define dw_pcie_ver_is_ge(_pci, _ver) __dw_pcie_ver_cmp(_pci, _ver, >=)
+
+#define dw_pcie_ver_type_is(_pci, _ver, _type) \
+	(__dw_pcie_ver_cmp(_pci, _ver, ==) && \
+	 __dw_pcie_ver_cmp(_pci, TYPE_ ## _type, ==))
+
+#define dw_pcie_ver_type_is_ge(_pci, _ver, _type) \
+	(__dw_pcie_ver_cmp(_pci, _ver, ==) && \
+	 __dw_pcie_ver_cmp(_pci, TYPE_ ## _type, >=))
+
 /* Parameters for the waiting for link up routine */
 #define LINK_WAIT_MAX_RETRIES		10
 #define LINK_WAIT_USLEEP_MIN		90000
-- 
2.35.1

Re: [PATCH RESEND v4 05/15] PCI: dwc: Introduce Synopsys IP-core versions/types interface

[Manivannan Sadhasivam]

On Fri, Jun 24, 2022 at 05:39:37PM +0300, Serge Semin wrote:
> Instead of manual DW PCIe data version field comparison let's use a handy
> macro-based interface in order to shorten out the statements, simplify the
> corresponding parts, improve the code readability and maintainability in
> perspective when more complex version-based dependencies need to
> implemented. Similar approaches have already been implemented in the DWC
> USB3 and DW SPI drivers (though with some IP-core evolution specifics).
> 
> Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>
> Reviewed-by: Rob Herring <robh@kernel.org>
> ---
>  drivers/pci/controller/dwc/pci-keystone.c    |  2 +-
>  drivers/pci/controller/dwc/pcie-designware.c |  8 ++++----
>  drivers/pci/controller/dwc/pcie-designware.h | 15 +++++++++++++++
>  3 files changed, 20 insertions(+), 5 deletions(-)
> 
> diff --git a/drivers/pci/controller/dwc/pci-keystone.c b/drivers/pci/controller/dwc/pci-keystone.c
> index c4ab3d775a18..2a9bbde224af 100644
> --- a/drivers/pci/controller/dwc/pci-keystone.c
> +++ b/drivers/pci/controller/dwc/pci-keystone.c
> @@ -1233,7 +1233,7 @@ static int __init ks_pcie_probe(struct platform_device *pdev)
>  		goto err_get_sync;
>  	}
>  
> -	if (pci->version >= DW_PCIE_VER_480A)
> +	if (dw_pcie_ver_is_ge(pci, 480A))
>  		ret = ks_pcie_am654_set_mode(dev, mode);
>  	else
>  		ret = ks_pcie_set_mode(dev);
> diff --git a/drivers/pci/controller/dwc/pcie-designware.c b/drivers/pci/controller/dwc/pcie-designware.c
> index cbb36ccaa48b..bd575ad32bc4 100644
> --- a/drivers/pci/controller/dwc/pcie-designware.c
> +++ b/drivers/pci/controller/dwc/pcie-designware.c
> @@ -313,7 +313,7 @@ static void dw_pcie_prog_outbound_atu_unroll(struct dw_pcie *pci, u8 func_no,
>  	val = type | PCIE_ATU_FUNC_NUM(func_no);
>  	if (upper_32_bits(limit_addr) > upper_32_bits(cpu_addr))
>  		val |= PCIE_ATU_INCREASE_REGION_SIZE;
> -	if (pci->version == DW_PCIE_VER_490A)
> +	if (dw_pcie_ver_is(pci, 490A))
>  		val = dw_pcie_enable_ecrc(val);
>  	dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_REGION_CTRL1, val);
>  	dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_REGION_CTRL2,
> @@ -360,7 +360,7 @@ static void __dw_pcie_prog_outbound_atu(struct dw_pcie *pci, u8 func_no,
>  			   upper_32_bits(cpu_addr));
>  	dw_pcie_writel_dbi(pci, PCIE_ATU_LIMIT,
>  			   lower_32_bits(limit_addr));
> -	if (pci->version >= DW_PCIE_VER_460A)
> +	if (dw_pcie_ver_is_ge(pci, 460A))
>  		dw_pcie_writel_dbi(pci, PCIE_ATU_UPPER_LIMIT,
>  				   upper_32_bits(limit_addr));
>  	dw_pcie_writel_dbi(pci, PCIE_ATU_LOWER_TARGET,
> @@ -369,9 +369,9 @@ static void __dw_pcie_prog_outbound_atu(struct dw_pcie *pci, u8 func_no,
>  			   upper_32_bits(pci_addr));
>  	val = type | PCIE_ATU_FUNC_NUM(func_no);
>  	if (upper_32_bits(limit_addr) > upper_32_bits(cpu_addr) &&
> -	    pci->version >= DW_PCIE_VER_460A)
> +	    dw_pcie_ver_is_ge(pci, 460A))
>  		val |= PCIE_ATU_INCREASE_REGION_SIZE;
> -	if (pci->version == DW_PCIE_VER_490A)
> +	if (dw_pcie_ver_is(pci, 490A))
>  		val = dw_pcie_enable_ecrc(val);
>  	dw_pcie_writel_dbi(pci, PCIE_ATU_CR1, val);
>  	dw_pcie_writel_dbi(pci, PCIE_ATU_CR2, PCIE_ATU_ENABLE);
> diff --git a/drivers/pci/controller/dwc/pcie-designware.h b/drivers/pci/controller/dwc/pcie-designware.h
> index 7899808bdbc6..d247f227464c 100644
> --- a/drivers/pci/controller/dwc/pcie-designware.h
> +++ b/drivers/pci/controller/dwc/pcie-designware.h
> @@ -28,6 +28,21 @@
>  #define DW_PCIE_VER_490A		0x3439302a
>  #define DW_PCIE_VER_520A		0x3532302a
>  
> +#define __dw_pcie_ver_cmp(_pci, _ver, _op) \
> +	((_pci)->version _op DW_PCIE_VER_ ## _ver)
> +
> +#define dw_pcie_ver_is(_pci, _ver) __dw_pcie_ver_cmp(_pci, _ver, ==)
> +
> +#define dw_pcie_ver_is_ge(_pci, _ver) __dw_pcie_ver_cmp(_pci, _ver, >=)
> +

The macro names are not very intuitive. But I cannot come with anything better.

> +#define dw_pcie_ver_type_is(_pci, _ver, _type) \
> +	(__dw_pcie_ver_cmp(_pci, _ver, ==) && \
> +	 __dw_pcie_ver_cmp(_pci, TYPE_ ## _type, ==))
> +
> +#define dw_pcie_ver_type_is_ge(_pci, _ver, _type) \
> +	(__dw_pcie_ver_cmp(_pci, _ver, ==) && \
> +	 __dw_pcie_ver_cmp(_pci, TYPE_ ## _type, >=))
> +

Are these macros used anywhere?

Thanks,
Mani

>  /* Parameters for the waiting for link up routine */
>  #define LINK_WAIT_MAX_RETRIES		10
>  #define LINK_WAIT_USLEEP_MIN		90000
> -- 
> 2.35.1
> 

-- 
மணிவண்ணன் சதாசிவம்

[PATCH RESEND v4 06/15] PCI: intel-gw: Drop manual DW PCIe controller version setup

[Serge Semin]

Since the DW PCIe common code now supports the IP-core version
auto-detection there is no point manually setting the version up for the
controllers newer than v4.70a. In particular Intel GW PCIe platform code
can be set free from the manual version setup, which as a positive side
effect causes the private device data removal too.

Suggested-by: Rob Herring <robh@kernel.org>
Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>

---

Folks, I don't have Intel GW PCIe hw instance to test it out. Could you
please make sure this patch doesn't brake anything?

Changelog v3:
- This is a new patch create as a result of the discussion:
  https://lore.kernel.org/linux-pci/20220503214638.1895-6-Sergey.Semin@baikalelectronics.ru/
---
 drivers/pci/controller/dwc/pcie-intel-gw.c | 16 +---------------
 1 file changed, 1 insertion(+), 15 deletions(-)

diff --git a/drivers/pci/controller/dwc/pcie-intel-gw.c b/drivers/pci/controller/dwc/pcie-intel-gw.c
index 371b5aa189d1..a44f685ec94d 100644
--- a/drivers/pci/controller/dwc/pcie-intel-gw.c
+++ b/drivers/pci/controller/dwc/pcie-intel-gw.c
@@ -58,10 +58,6 @@
 #define BUS_IATU_OFFSET			SZ_256M
 #define RESET_INTERVAL_MS		100
 
-struct intel_pcie_soc {
-	u32	pcie_ver;
-};
-
 struct intel_pcie {
 	struct dw_pcie		pci;
 	void __iomem		*app_base;
@@ -394,13 +390,8 @@ static const struct dw_pcie_host_ops intel_pcie_dw_ops = {
 	.host_init =		intel_pcie_rc_init,
 };
 
-static const struct intel_pcie_soc pcie_data = {
-	.pcie_ver =		DW_PCIE_VER_520A,
-};
-
 static int intel_pcie_probe(struct platform_device *pdev)
 {
-	const struct intel_pcie_soc *data;
 	struct device *dev = &pdev->dev;
 	struct intel_pcie *pcie;
 	struct dw_pcie_rp *pp;
@@ -424,12 +415,7 @@ static int intel_pcie_probe(struct platform_device *pdev)
 	if (ret)
 		return ret;
 
-	data = device_get_match_data(dev);
-	if (!data)
-		return -ENODEV;
-
 	pci->ops = &intel_pcie_ops;
-	pci->version = data->pcie_ver;
 	pp->ops = &intel_pcie_dw_ops;
 
 	ret = dw_pcie_host_init(pp);
@@ -447,7 +433,7 @@ static const struct dev_pm_ops intel_pcie_pm_ops = {
 };
 
 static const struct of_device_id of_intel_pcie_match[] = {
-	{ .compatible = "intel,lgm-pcie", .data = &pcie_data },
+	{ .compatible = "intel,lgm-pcie" },
 	{}
 };
 
-- 
2.35.1

Re: [PATCH RESEND v4 06/15] PCI: intel-gw: Drop manual DW PCIe controller version setup

[Manivannan Sadhasivam]

On Fri, Jun 24, 2022 at 05:39:38PM +0300, Serge Semin wrote:
> Since the DW PCIe common code now supports the IP-core version
> auto-detection there is no point manually setting the version up for the
> controllers newer than v4.70a. In particular Intel GW PCIe platform code
> can be set free from the manual version setup, which as a positive side
> effect causes the private device data removal too.
> 
> Suggested-by: Rob Herring <robh@kernel.org>
> Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>

I don't have a hardware to test. But going by the DWC spec, this looks good to
me.

CCed the author of this driver, "Dilip Kota" in case he can confirm.

Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>

Thanks,
Mani

> 
> ---
> 
> Folks, I don't have Intel GW PCIe hw instance to test it out. Could you
> please make sure this patch doesn't brake anything?
> 
> Changelog v3:
> - This is a new patch create as a result of the discussion:
>   https://lore.kernel.org/linux-pci/20220503214638.1895-6-Sergey.Semin@baikalelectronics.ru/
> ---
>  drivers/pci/controller/dwc/pcie-intel-gw.c | 16 +---------------
>  1 file changed, 1 insertion(+), 15 deletions(-)
> 
> diff --git a/drivers/pci/controller/dwc/pcie-intel-gw.c b/drivers/pci/controller/dwc/pcie-intel-gw.c
> index 371b5aa189d1..a44f685ec94d 100644
> --- a/drivers/pci/controller/dwc/pcie-intel-gw.c
> +++ b/drivers/pci/controller/dwc/pcie-intel-gw.c
> @@ -58,10 +58,6 @@
>  #define BUS_IATU_OFFSET			SZ_256M
>  #define RESET_INTERVAL_MS		100
>  
> -struct intel_pcie_soc {
> -	u32	pcie_ver;
> -};
> -
>  struct intel_pcie {
>  	struct dw_pcie		pci;
>  	void __iomem		*app_base;
> @@ -394,13 +390,8 @@ static const struct dw_pcie_host_ops intel_pcie_dw_ops = {
>  	.host_init =		intel_pcie_rc_init,
>  };
>  
> -static const struct intel_pcie_soc pcie_data = {
> -	.pcie_ver =		DW_PCIE_VER_520A,
> -};
> -
>  static int intel_pcie_probe(struct platform_device *pdev)
>  {
> -	const struct intel_pcie_soc *data;
>  	struct device *dev = &pdev->dev;
>  	struct intel_pcie *pcie;
>  	struct dw_pcie_rp *pp;
> @@ -424,12 +415,7 @@ static int intel_pcie_probe(struct platform_device *pdev)
>  	if (ret)
>  		return ret;
>  
> -	data = device_get_match_data(dev);
> -	if (!data)
> -		return -ENODEV;
> -
>  	pci->ops = &intel_pcie_ops;
> -	pci->version = data->pcie_ver;
>  	pp->ops = &intel_pcie_dw_ops;
>  
>  	ret = dw_pcie_host_init(pp);
> @@ -447,7 +433,7 @@ static const struct dev_pm_ops intel_pcie_pm_ops = {
>  };
>  
>  static const struct of_device_id of_intel_pcie_match[] = {
> -	{ .compatible = "intel,lgm-pcie", .data = &pcie_data },
> +	{ .compatible = "intel,lgm-pcie" },
>  	{}
>  };
>  
> -- 
> 2.35.1
> 

-- 
மணிவண்ணன் சதாசிவம்

[PATCH RESEND v4 07/15] PCI: tegra194: Drop manual DW PCIe controller version setup

[Serge Semin]

Since the DW PCIe common code now supports the IP-core version
auto-detection there is no point in manually setting the version up for the
controllers newer than v4.70a. Seeing Tegra 194 PCIe Host and EP
controllers are based on the DW PCIe v4.90a IP-core we can freely drop the
dw_pcie.version field initialization.

Suggested-by: Rob Herring <robh@kernel.org>
Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>

---

Folks, I don't have Tegra 194 PCIe hw instance to test it out. Could you
please make sure this patch doesn't brake anything?

Changelog v3:
- This is a new patch create as a result of the discussion:
  https://lore.kernel.org/linux-pci/20220503214638.1895-6-Sergey.Semin@baikalelectronics.ru/
---
 drivers/pci/controller/dwc/pcie-tegra194.c | 1 -
 1 file changed, 1 deletion(-)

diff --git a/drivers/pci/controller/dwc/pcie-tegra194.c b/drivers/pci/controller/dwc/pcie-tegra194.c
index f24b30b7454f..e497e6de8d15 100644
--- a/drivers/pci/controller/dwc/pcie-tegra194.c
+++ b/drivers/pci/controller/dwc/pcie-tegra194.c
@@ -1979,7 +1979,6 @@ static int tegra194_pcie_probe(struct platform_device *pdev)
 	pci->ops = &tegra_dw_pcie_ops;
 	pci->n_fts[0] = N_FTS_VAL;
 	pci->n_fts[1] = FTS_VAL;
-	pci->version = DW_PCIE_VER_490A;
 
 	pp = &pci->pp;
 	pp->num_vectors = MAX_MSI_IRQS;
-- 
2.35.1

Re: [PATCH RESEND v4 07/15] PCI: tegra194: Drop manual DW PCIe controller version setup

[Vidya Sagar]

On 6/24/2022 8:09 PM, Serge Semin wrote:
> External email: Use caution opening links or attachments
> 
> 
> Since the DW PCIe common code now supports the IP-core version
> auto-detection there is no point in manually setting the version up for the
> controllers newer than v4.70a. Seeing Tegra 194 PCIe Host and EP
> controllers are based on the DW PCIe v4.90a IP-core we can freely drop the
> dw_pcie.version field initialization.
> 
> Suggested-by: Rob Herring <robh@kernel.org>
> Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>
> 
> ---
> 
> Folks, I don't have Tegra 194 PCIe hw instance to test it out. Could you
> please make sure this patch doesn't brake anything?

Hi,
I tried to apply the series 
https://patchwork.kernel.org/project/linux-pci/list/?series=653624 on 
top of linux-next and ran into conflicts. Could you please tell me the 
minimum set of patches to be taken?

Thanks,
Vidya Sagar

> 
> Changelog v3:
> - This is a new patch create as a result of the discussion:
>    https://lore.kernel.org/linux-pci/20220503214638.1895-6-Sergey.Semin@baikalelectronics.ru/
> ---
>   drivers/pci/controller/dwc/pcie-tegra194.c | 1 -
>   1 file changed, 1 deletion(-)
> 
> diff --git a/drivers/pci/controller/dwc/pcie-tegra194.c b/drivers/pci/controller/dwc/pcie-tegra194.c
> index f24b30b7454f..e497e6de8d15 100644
> --- a/drivers/pci/controller/dwc/pcie-tegra194.c
> +++ b/drivers/pci/controller/dwc/pcie-tegra194.c
> @@ -1979,7 +1979,6 @@ static int tegra194_pcie_probe(struct platform_device *pdev)
>          pci->ops = &tegra_dw_pcie_ops;
>          pci->n_fts[0] = N_FTS_VAL;
>          pci->n_fts[1] = FTS_VAL;
> -       pci->version = DW_PCIE_VER_490A;
> 
>          pp = &pci->pp;
>          pp->num_vectors = MAX_MSI_IRQS;
> --
> 2.35.1
> 

Re: [PATCH RESEND v4 07/15] PCI: tegra194: Drop manual DW PCIe controller version setup

[Serge Semin]

On Mon, Jun 27, 2022 at 01:29:33PM +0530, Vidya Sagar wrote:
> 
> 
> On 6/24/2022 8:09 PM, Serge Semin wrote:
> > External email: Use caution opening links or attachments
> > 
> > 
> > Since the DW PCIe common code now supports the IP-core version
> > auto-detection there is no point in manually setting the version up for the
> > controllers newer than v4.70a. Seeing Tegra 194 PCIe Host and EP
> > controllers are based on the DW PCIe v4.90a IP-core we can freely drop the
> > dw_pcie.version field initialization.
> > 
> > Suggested-by: Rob Herring <robh@kernel.org>
> > Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>
> > 
> > ---
> > 
> > Folks, I don't have Tegra 194 PCIe hw instance to test it out. Could you
> > please make sure this patch doesn't brake anything?
> 

> Hi,
> I tried to apply the series
> https://patchwork.kernel.org/project/linux-pci/list/?series=653624 on top of
> linux-next and ran into conflicts. Could you please tell me the minimum set
> of patches to be taken?

Hi.
As the cover letter says it is supposed to be applied on top of the
next series *:
https://lore.kernel.org/linux-pci/20220624143428.8334-1-Sergey.Semin@baikalelectronics.ru/
* Currently latest patchset version is v5.

Then - the resent series what you are referring to.

-Sergey

> 
> Thanks,
> Vidya Sagar
> 
> > 
> > Changelog v3:
> > - This is a new patch create as a result of the discussion:
> >    https://lore.kernel.org/linux-pci/20220503214638.1895-6-Sergey.Semin@baikalelectronics.ru/
> > ---
> >   drivers/pci/controller/dwc/pcie-tegra194.c | 1 -
> >   1 file changed, 1 deletion(-)
> > 
> > diff --git a/drivers/pci/controller/dwc/pcie-tegra194.c b/drivers/pci/controller/dwc/pcie-tegra194.c
> > index f24b30b7454f..e497e6de8d15 100644
> > --- a/drivers/pci/controller/dwc/pcie-tegra194.c
> > +++ b/drivers/pci/controller/dwc/pcie-tegra194.c
> > @@ -1979,7 +1979,6 @@ static int tegra194_pcie_probe(struct platform_device *pdev)
> >          pci->ops = &tegra_dw_pcie_ops;
> >          pci->n_fts[0] = N_FTS_VAL;
> >          pci->n_fts[1] = FTS_VAL;
> > -       pci->version = DW_PCIE_VER_490A;
> > 
> >          pp = &pci->pp;
> >          pp->num_vectors = MAX_MSI_IRQS;
> > --
> > 2.35.1
> > 

Re: [PATCH RESEND v4 07/15] PCI: tegra194: Drop manual DW PCIe controller version setup

[Manivannan Sadhasivam]

On Fri, Jun 24, 2022 at 05:39:39PM +0300, Serge Semin wrote:
> Since the DW PCIe common code now supports the IP-core version
> auto-detection there is no point in manually setting the version up for the
> controllers newer than v4.70a. Seeing Tegra 194 PCIe Host and EP
> controllers are based on the DW PCIe v4.90a IP-core we can freely drop the
> dw_pcie.version field initialization.
> 
> Suggested-by: Rob Herring <robh@kernel.org>
> Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>

Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>

Thanks,
Mani

> 
> ---
> 
> Folks, I don't have Tegra 194 PCIe hw instance to test it out. Could you
> please make sure this patch doesn't brake anything?
> 
> Changelog v3:
> - This is a new patch create as a result of the discussion:
>   https://lore.kernel.org/linux-pci/20220503214638.1895-6-Sergey.Semin@baikalelectronics.ru/
> ---
>  drivers/pci/controller/dwc/pcie-tegra194.c | 1 -
>  1 file changed, 1 deletion(-)
> 
> diff --git a/drivers/pci/controller/dwc/pcie-tegra194.c b/drivers/pci/controller/dwc/pcie-tegra194.c
> index f24b30b7454f..e497e6de8d15 100644
> --- a/drivers/pci/controller/dwc/pcie-tegra194.c
> +++ b/drivers/pci/controller/dwc/pcie-tegra194.c
> @@ -1979,7 +1979,6 @@ static int tegra194_pcie_probe(struct platform_device *pdev)
>  	pci->ops = &tegra_dw_pcie_ops;
>  	pci->n_fts[0] = N_FTS_VAL;
>  	pci->n_fts[1] = FTS_VAL;
> -	pci->version = DW_PCIE_VER_490A;
>  
>  	pp = &pci->pp;
>  	pp->num_vectors = MAX_MSI_IRQS;
> -- 
> 2.35.1
> 

-- 
மணிவண்ணன் சதாசிவம்

[PATCH RESEND v4 08/15] PCI: dwc: Add host de-initialization callback

[Serge Semin]

Seeing the platform-specific DW PCIe host-initialization is performed from
within the generic dw_pcie_host_init() method by means of the dedicated
dw_pcie_ops.host_init() callback, there must be declared an antagonist
which would perform the corresponding cleanups. Let's add such callback
then. It will be called in the dw_pcie_host_deinit() method and in the
cleanup-on-error path in the dw_pcie_host_init() function.

Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>
---
 .../pci/controller/dwc/pcie-designware-host.c | 21 ++++++++++++++-----
 drivers/pci/controller/dwc/pcie-designware.h  |  1 +
 2 files changed, 17 insertions(+), 5 deletions(-)

diff --git a/drivers/pci/controller/dwc/pcie-designware-host.c b/drivers/pci/controller/dwc/pcie-designware-host.c
index b1437b37140f..4f984c845b59 100644
--- a/drivers/pci/controller/dwc/pcie-designware-host.c
+++ b/drivers/pci/controller/dwc/pcie-designware-host.c
@@ -354,13 +354,14 @@ int dw_pcie_host_init(struct dw_pcie_rp *pp)
 			pp->num_vectors = MSI_DEF_NUM_VECTORS;
 		} else if (pp->num_vectors > MAX_MSI_IRQS) {
 			dev_err(dev, "Invalid number of vectors\n");
-			return -EINVAL;
+			ret = -EINVAL;
+			goto err_deinit_host;
 		}
 
 		if (pp->ops->msi_host_init) {
 			ret = pp->ops->msi_host_init(pp);
 			if (ret < 0)
-				return ret;
+				goto err_deinit_host;
 		} else if (pp->has_msi_ctrl) {
 			u32 ctrl, num_ctrls;
 
@@ -372,8 +373,10 @@ int dw_pcie_host_init(struct dw_pcie_rp *pp)
 				pp->msi_irq = platform_get_irq_byname_optional(pdev, "msi");
 				if (pp->msi_irq < 0) {
 					pp->msi_irq = platform_get_irq(pdev, 0);
-					if (pp->msi_irq < 0)
-						return pp->msi_irq;
+					if (pp->msi_irq < 0) {
+						ret = pp->msi_irq;
+						goto err_deinit_host;
+					}
 				}
 			}
 
@@ -381,7 +384,7 @@ int dw_pcie_host_init(struct dw_pcie_rp *pp)
 
 			ret = dw_pcie_allocate_domains(pp);
 			if (ret)
-				return ret;
+				goto err_deinit_host;
 
 			if (pp->msi_irq > 0)
 				irq_set_chained_handler_and_data(pp->msi_irq,
@@ -434,6 +437,11 @@ int dw_pcie_host_init(struct dw_pcie_rp *pp)
 err_free_msi:
 	if (pp->has_msi_ctrl)
 		dw_pcie_free_msi(pp);
+
+err_deinit_host:
+	if (pp->ops->host_deinit)
+		pp->ops->host_deinit(pp);
+
 	return ret;
 }
 EXPORT_SYMBOL_GPL(dw_pcie_host_init);
@@ -449,6 +457,9 @@ void dw_pcie_host_deinit(struct dw_pcie_rp *pp)
 
 	if (pp->has_msi_ctrl)
 		dw_pcie_free_msi(pp);
+
+	if (pp->ops->host_deinit)
+		pp->ops->host_deinit(pp);
 }
 EXPORT_SYMBOL_GPL(dw_pcie_host_deinit);
 
diff --git a/drivers/pci/controller/dwc/pcie-designware.h b/drivers/pci/controller/dwc/pcie-designware.h
index d247f227464c..7f1c00fa084d 100644
--- a/drivers/pci/controller/dwc/pcie-designware.h
+++ b/drivers/pci/controller/dwc/pcie-designware.h
@@ -200,6 +200,7 @@ enum dw_pcie_device_mode {
 
 struct dw_pcie_host_ops {
 	int (*host_init)(struct dw_pcie_rp *pp);
+	void (*host_deinit)(struct dw_pcie_rp *pp);
 	int (*msi_host_init)(struct dw_pcie_rp *pp);
 };
 
-- 
2.35.1

Re: [PATCH RESEND v4 08/15] PCI: dwc: Add host de-initialization callback

[Manivannan Sadhasivam]

On Fri, Jun 24, 2022 at 05:39:40PM +0300, Serge Semin wrote:
> Seeing the platform-specific DW PCIe host-initialization is performed from
> within the generic dw_pcie_host_init() method by means of the dedicated
> dw_pcie_ops.host_init() callback, there must be declared an antagonist
> which would perform the corresponding cleanups. Let's add such callback
> then. It will be called in the dw_pcie_host_deinit() method and in the
> cleanup-on-error path in the dw_pcie_host_init() function.
> 
> Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>

Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>

Thanks,
Mani

> ---
>  .../pci/controller/dwc/pcie-designware-host.c | 21 ++++++++++++++-----
>  drivers/pci/controller/dwc/pcie-designware.h  |  1 +
>  2 files changed, 17 insertions(+), 5 deletions(-)
> 
> diff --git a/drivers/pci/controller/dwc/pcie-designware-host.c b/drivers/pci/controller/dwc/pcie-designware-host.c
> index b1437b37140f..4f984c845b59 100644
> --- a/drivers/pci/controller/dwc/pcie-designware-host.c
> +++ b/drivers/pci/controller/dwc/pcie-designware-host.c
> @@ -354,13 +354,14 @@ int dw_pcie_host_init(struct dw_pcie_rp *pp)
>  			pp->num_vectors = MSI_DEF_NUM_VECTORS;
>  		} else if (pp->num_vectors > MAX_MSI_IRQS) {
>  			dev_err(dev, "Invalid number of vectors\n");
> -			return -EINVAL;
> +			ret = -EINVAL;
> +			goto err_deinit_host;
>  		}
>  
>  		if (pp->ops->msi_host_init) {
>  			ret = pp->ops->msi_host_init(pp);
>  			if (ret < 0)
> -				return ret;
> +				goto err_deinit_host;
>  		} else if (pp->has_msi_ctrl) {
>  			u32 ctrl, num_ctrls;
>  
> @@ -372,8 +373,10 @@ int dw_pcie_host_init(struct dw_pcie_rp *pp)
>  				pp->msi_irq = platform_get_irq_byname_optional(pdev, "msi");
>  				if (pp->msi_irq < 0) {
>  					pp->msi_irq = platform_get_irq(pdev, 0);
> -					if (pp->msi_irq < 0)
> -						return pp->msi_irq;
> +					if (pp->msi_irq < 0) {
> +						ret = pp->msi_irq;
> +						goto err_deinit_host;
> +					}
>  				}
>  			}
>  
> @@ -381,7 +384,7 @@ int dw_pcie_host_init(struct dw_pcie_rp *pp)
>  
>  			ret = dw_pcie_allocate_domains(pp);
>  			if (ret)
> -				return ret;
> +				goto err_deinit_host;
>  
>  			if (pp->msi_irq > 0)
>  				irq_set_chained_handler_and_data(pp->msi_irq,
> @@ -434,6 +437,11 @@ int dw_pcie_host_init(struct dw_pcie_rp *pp)
>  err_free_msi:
>  	if (pp->has_msi_ctrl)
>  		dw_pcie_free_msi(pp);
> +
> +err_deinit_host:
> +	if (pp->ops->host_deinit)
> +		pp->ops->host_deinit(pp);
> +
>  	return ret;
>  }
>  EXPORT_SYMBOL_GPL(dw_pcie_host_init);
> @@ -449,6 +457,9 @@ void dw_pcie_host_deinit(struct dw_pcie_rp *pp)
>  
>  	if (pp->has_msi_ctrl)
>  		dw_pcie_free_msi(pp);
> +
> +	if (pp->ops->host_deinit)
> +		pp->ops->host_deinit(pp);
>  }
>  EXPORT_SYMBOL_GPL(dw_pcie_host_deinit);
>  
> diff --git a/drivers/pci/controller/dwc/pcie-designware.h b/drivers/pci/controller/dwc/pcie-designware.h
> index d247f227464c..7f1c00fa084d 100644
> --- a/drivers/pci/controller/dwc/pcie-designware.h
> +++ b/drivers/pci/controller/dwc/pcie-designware.h
> @@ -200,6 +200,7 @@ enum dw_pcie_device_mode {
>  
>  struct dw_pcie_host_ops {
>  	int (*host_init)(struct dw_pcie_rp *pp);
> +	void (*host_deinit)(struct dw_pcie_rp *pp);
>  	int (*msi_host_init)(struct dw_pcie_rp *pp);
>  };
>  
> -- 
> 2.35.1
> 

-- 
மணிவண்ணன் சதாசிவம்

[PATCH RESEND v4 09/15] PCI: dwc: Drop inbound iATU types enumeration - dw_pcie_as_type

[Serge Semin]

There is no point in having an enumeration declared in the driver for the
PCIe end-point. First of all it's redundant since the driver already has a
set of macro declared which describe the available in/out iATU types, thus
having an addition abstraction just needlessly complicates the code.
Secondly checking the passed iATU type for validity within a single driver
is pointless since the driver is supposed to be consistent by its nature.
Finally the outbound iATU type isn't encoded by the denoted enumeration,
thus giving a false impression that the in and out iATU types are
unrelated while they are the same. So to speak let's drop the redundant
dw_pcie_as_type enumeration replacing it with the direct iATU type usage.

While at it, since we are touching the iATU inbound regions config methods
anyway, let's fix the arguments order so the type would be followed by the
address-related parameters. Thus the inbound and outbound iATU setup
methods will look alike. That shall improve the code readability a bit.

Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>
Reviewed-by: Rob Herring <robh@kernel.org>
---
 .../pci/controller/dwc/pcie-designware-ep.c   | 21 +++++------
 drivers/pci/controller/dwc/pcie-designware.c  | 35 +++----------------
 drivers/pci/controller/dwc/pcie-designware.h  |  9 +----
 3 files changed, 15 insertions(+), 50 deletions(-)

diff --git a/drivers/pci/controller/dwc/pcie-designware-ep.c b/drivers/pci/controller/dwc/pcie-designware-ep.c
index ffbd3af6d65a..5a158813f687 100644
--- a/drivers/pci/controller/dwc/pcie-designware-ep.c
+++ b/drivers/pci/controller/dwc/pcie-designware-ep.c
@@ -154,9 +154,8 @@ static int dw_pcie_ep_write_header(struct pci_epc *epc, u8 func_no, u8 vfunc_no,
 	return 0;
 }
 
-static int dw_pcie_ep_inbound_atu(struct dw_pcie_ep *ep, u8 func_no,
-				  enum pci_barno bar, dma_addr_t cpu_addr,
-				  enum dw_pcie_as_type as_type)
+static int dw_pcie_ep_inbound_atu(struct dw_pcie_ep *ep, u8 func_no, int type,
+				  dma_addr_t cpu_addr, enum pci_barno bar)
 {
 	int ret;
 	u32 free_win;
@@ -168,8 +167,8 @@ static int dw_pcie_ep_inbound_atu(struct dw_pcie_ep *ep, u8 func_no,
 		return -EINVAL;
 	}
 
-	ret = dw_pcie_prog_inbound_atu(pci, func_no, free_win, bar, cpu_addr,
-				       as_type);
+	ret = dw_pcie_prog_inbound_atu(pci, func_no, free_win, type,
+				       cpu_addr, bar);
 	if (ret < 0) {
 		dev_err(pci->dev, "Failed to program IB window\n");
 		return ret;
@@ -221,27 +220,25 @@ static void dw_pcie_ep_clear_bar(struct pci_epc *epc, u8 func_no, u8 vfunc_no,
 static int dw_pcie_ep_set_bar(struct pci_epc *epc, u8 func_no, u8 vfunc_no,
 			      struct pci_epf_bar *epf_bar)
 {
-	int ret;
 	struct dw_pcie_ep *ep = epc_get_drvdata(epc);
 	struct dw_pcie *pci = to_dw_pcie_from_ep(ep);
 	enum pci_barno bar = epf_bar->barno;
 	size_t size = epf_bar->size;
 	int flags = epf_bar->flags;
-	enum dw_pcie_as_type as_type;
-	u32 reg;
 	unsigned int func_offset = 0;
+	int ret, type;
+	u32 reg;
 
 	func_offset = dw_pcie_ep_func_select(ep, func_no);
 
 	reg = PCI_BASE_ADDRESS_0 + (4 * bar) + func_offset;
 
 	if (!(flags & PCI_BASE_ADDRESS_SPACE))
-		as_type = DW_PCIE_AS_MEM;
+		type = PCIE_ATU_TYPE_MEM;
 	else
-		as_type = DW_PCIE_AS_IO;
+		type = PCIE_ATU_TYPE_IO;
 
-	ret = dw_pcie_ep_inbound_atu(ep, func_no, bar,
-				     epf_bar->phys_addr, as_type);
+	ret = dw_pcie_ep_inbound_atu(ep, func_no, type, epf_bar->phys_addr, bar);
 	if (ret)
 		return ret;
 
diff --git a/drivers/pci/controller/dwc/pcie-designware.c b/drivers/pci/controller/dwc/pcie-designware.c
index bd575ad32bc4..330575182712 100644
--- a/drivers/pci/controller/dwc/pcie-designware.c
+++ b/drivers/pci/controller/dwc/pcie-designware.c
@@ -421,10 +421,9 @@ static void dw_pcie_writel_ib_unroll(struct dw_pcie *pci, u32 index, u32 reg,
 }
 
 static int dw_pcie_prog_inbound_atu_unroll(struct dw_pcie *pci, u8 func_no,
-					   int index, int bar, u64 cpu_addr,
-					   enum dw_pcie_as_type as_type)
+					   int index, int type,
+					   u64 cpu_addr, u8 bar)
 {
-	int type;
 	u32 retries, val;
 
 	dw_pcie_writel_ib_unroll(pci, index, PCIE_ATU_UNR_LOWER_TARGET,
@@ -432,17 +431,6 @@ static int dw_pcie_prog_inbound_atu_unroll(struct dw_pcie *pci, u8 func_no,
 	dw_pcie_writel_ib_unroll(pci, index, PCIE_ATU_UNR_UPPER_TARGET,
 				 upper_32_bits(cpu_addr));
 
-	switch (as_type) {
-	case DW_PCIE_AS_MEM:
-		type = PCIE_ATU_TYPE_MEM;
-		break;
-	case DW_PCIE_AS_IO:
-		type = PCIE_ATU_TYPE_IO;
-		break;
-	default:
-		return -EINVAL;
-	}
-
 	dw_pcie_writel_ib_unroll(pci, index, PCIE_ATU_UNR_REGION_CTRL1, type |
 				 PCIE_ATU_FUNC_NUM(func_no));
 	dw_pcie_writel_ib_unroll(pci, index, PCIE_ATU_UNR_REGION_CTRL2,
@@ -468,32 +456,19 @@ static int dw_pcie_prog_inbound_atu_unroll(struct dw_pcie *pci, u8 func_no,
 }
 
 int dw_pcie_prog_inbound_atu(struct dw_pcie *pci, u8 func_no, int index,
-			     int bar, u64 cpu_addr,
-			     enum dw_pcie_as_type as_type)
+			     int type, u64 cpu_addr, u8 bar)
 {
-	int type;
 	u32 retries, val;
 
 	if (pci->iatu_unroll_enabled)
-		return dw_pcie_prog_inbound_atu_unroll(pci, func_no, index, bar,
-						       cpu_addr, as_type);
+		return dw_pcie_prog_inbound_atu_unroll(pci, func_no, index, type,
+						       cpu_addr, bar);
 
 	dw_pcie_writel_dbi(pci, PCIE_ATU_VIEWPORT, PCIE_ATU_REGION_INBOUND |
 			   index);
 	dw_pcie_writel_dbi(pci, PCIE_ATU_LOWER_TARGET, lower_32_bits(cpu_addr));
 	dw_pcie_writel_dbi(pci, PCIE_ATU_UPPER_TARGET, upper_32_bits(cpu_addr));
 
-	switch (as_type) {
-	case DW_PCIE_AS_MEM:
-		type = PCIE_ATU_TYPE_MEM;
-		break;
-	case DW_PCIE_AS_IO:
-		type = PCIE_ATU_TYPE_IO;
-		break;
-	default:
-		return -EINVAL;
-	}
-
 	dw_pcie_writel_dbi(pci, PCIE_ATU_CR1, type |
 			   PCIE_ATU_FUNC_NUM(func_no));
 	dw_pcie_writel_dbi(pci, PCIE_ATU_CR2, PCIE_ATU_ENABLE |
diff --git a/drivers/pci/controller/dwc/pcie-designware.h b/drivers/pci/controller/dwc/pcie-designware.h
index 7f1c00fa084d..c63ace3c3f25 100644
--- a/drivers/pci/controller/dwc/pcie-designware.h
+++ b/drivers/pci/controller/dwc/pcie-designware.h
@@ -228,12 +228,6 @@ struct dw_pcie_rp {
 	DECLARE_BITMAP(msi_irq_in_use, MAX_MSI_IRQS);
 };
 
-enum dw_pcie_as_type {
-	DW_PCIE_AS_UNKNOWN,
-	DW_PCIE_AS_MEM,
-	DW_PCIE_AS_IO,
-};
-
 struct dw_pcie_ep_ops {
 	void	(*ep_init)(struct dw_pcie_ep *ep);
 	int	(*raise_irq)(struct dw_pcie_ep *ep, u8 func_no,
@@ -331,8 +325,7 @@ void dw_pcie_prog_ep_outbound_atu(struct dw_pcie *pci, u8 func_no, int index,
 				  int type, u64 cpu_addr, u64 pci_addr,
 				  u64 size);
 int dw_pcie_prog_inbound_atu(struct dw_pcie *pci, u8 func_no, int index,
-			     int bar, u64 cpu_addr,
-			     enum dw_pcie_as_type as_type);
+			     int type, u64 cpu_addr, u8 bar);
 void dw_pcie_disable_atu(struct dw_pcie *pci, int index,
 			 enum dw_pcie_region_type type);
 void dw_pcie_setup(struct dw_pcie *pci);
-- 
2.35.1

Re: [PATCH RESEND v4 09/15] PCI: dwc: Drop inbound iATU types enumeration - dw_pcie_as_type

[Manivannan Sadhasivam]

On Fri, Jun 24, 2022 at 05:39:41PM +0300, Serge Semin wrote:
> There is no point in having an enumeration declared in the driver for the
> PCIe end-point. First of all it's redundant since the driver already has a
> set of macro declared which describe the available in/out iATU types, thus
> having an addition abstraction just needlessly complicates the code.
> Secondly checking the passed iATU type for validity within a single driver
> is pointless since the driver is supposed to be consistent by its nature.
> Finally the outbound iATU type isn't encoded by the denoted enumeration,
> thus giving a false impression that the in and out iATU types are
> unrelated while they are the same. So to speak let's drop the redundant
> dw_pcie_as_type enumeration replacing it with the direct iATU type usage.
> 
> While at it, since we are touching the iATU inbound regions config methods
> anyway, let's fix the arguments order so the type would be followed by the
> address-related parameters. Thus the inbound and outbound iATU setup
> methods will look alike. That shall improve the code readability a bit.
> 
> Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>
> Reviewed-by: Rob Herring <robh@kernel.org>

Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>

Thanks,
Mani

> ---
>  .../pci/controller/dwc/pcie-designware-ep.c   | 21 +++++------
>  drivers/pci/controller/dwc/pcie-designware.c  | 35 +++----------------
>  drivers/pci/controller/dwc/pcie-designware.h  |  9 +----
>  3 files changed, 15 insertions(+), 50 deletions(-)
> 
> diff --git a/drivers/pci/controller/dwc/pcie-designware-ep.c b/drivers/pci/controller/dwc/pcie-designware-ep.c
> index ffbd3af6d65a..5a158813f687 100644
> --- a/drivers/pci/controller/dwc/pcie-designware-ep.c
> +++ b/drivers/pci/controller/dwc/pcie-designware-ep.c
> @@ -154,9 +154,8 @@ static int dw_pcie_ep_write_header(struct pci_epc *epc, u8 func_no, u8 vfunc_no,
>  	return 0;
>  }
>  
> -static int dw_pcie_ep_inbound_atu(struct dw_pcie_ep *ep, u8 func_no,
> -				  enum pci_barno bar, dma_addr_t cpu_addr,
> -				  enum dw_pcie_as_type as_type)
> +static int dw_pcie_ep_inbound_atu(struct dw_pcie_ep *ep, u8 func_no, int type,
> +				  dma_addr_t cpu_addr, enum pci_barno bar)
>  {
>  	int ret;
>  	u32 free_win;
> @@ -168,8 +167,8 @@ static int dw_pcie_ep_inbound_atu(struct dw_pcie_ep *ep, u8 func_no,
>  		return -EINVAL;
>  	}
>  
> -	ret = dw_pcie_prog_inbound_atu(pci, func_no, free_win, bar, cpu_addr,
> -				       as_type);
> +	ret = dw_pcie_prog_inbound_atu(pci, func_no, free_win, type,
> +				       cpu_addr, bar);
>  	if (ret < 0) {
>  		dev_err(pci->dev, "Failed to program IB window\n");
>  		return ret;
> @@ -221,27 +220,25 @@ static void dw_pcie_ep_clear_bar(struct pci_epc *epc, u8 func_no, u8 vfunc_no,
>  static int dw_pcie_ep_set_bar(struct pci_epc *epc, u8 func_no, u8 vfunc_no,
>  			      struct pci_epf_bar *epf_bar)
>  {
> -	int ret;
>  	struct dw_pcie_ep *ep = epc_get_drvdata(epc);
>  	struct dw_pcie *pci = to_dw_pcie_from_ep(ep);
>  	enum pci_barno bar = epf_bar->barno;
>  	size_t size = epf_bar->size;
>  	int flags = epf_bar->flags;
> -	enum dw_pcie_as_type as_type;
> -	u32 reg;
>  	unsigned int func_offset = 0;
> +	int ret, type;
> +	u32 reg;
>  
>  	func_offset = dw_pcie_ep_func_select(ep, func_no);
>  
>  	reg = PCI_BASE_ADDRESS_0 + (4 * bar) + func_offset;
>  
>  	if (!(flags & PCI_BASE_ADDRESS_SPACE))
> -		as_type = DW_PCIE_AS_MEM;
> +		type = PCIE_ATU_TYPE_MEM;
>  	else
> -		as_type = DW_PCIE_AS_IO;
> +		type = PCIE_ATU_TYPE_IO;
>  
> -	ret = dw_pcie_ep_inbound_atu(ep, func_no, bar,
> -				     epf_bar->phys_addr, as_type);
> +	ret = dw_pcie_ep_inbound_atu(ep, func_no, type, epf_bar->phys_addr, bar);
>  	if (ret)
>  		return ret;
>  
> diff --git a/drivers/pci/controller/dwc/pcie-designware.c b/drivers/pci/controller/dwc/pcie-designware.c
> index bd575ad32bc4..330575182712 100644
> --- a/drivers/pci/controller/dwc/pcie-designware.c
> +++ b/drivers/pci/controller/dwc/pcie-designware.c
> @@ -421,10 +421,9 @@ static void dw_pcie_writel_ib_unroll(struct dw_pcie *pci, u32 index, u32 reg,
>  }
>  
>  static int dw_pcie_prog_inbound_atu_unroll(struct dw_pcie *pci, u8 func_no,
> -					   int index, int bar, u64 cpu_addr,
> -					   enum dw_pcie_as_type as_type)
> +					   int index, int type,
> +					   u64 cpu_addr, u8 bar)
>  {
> -	int type;
>  	u32 retries, val;
>  
>  	dw_pcie_writel_ib_unroll(pci, index, PCIE_ATU_UNR_LOWER_TARGET,
> @@ -432,17 +431,6 @@ static int dw_pcie_prog_inbound_atu_unroll(struct dw_pcie *pci, u8 func_no,
>  	dw_pcie_writel_ib_unroll(pci, index, PCIE_ATU_UNR_UPPER_TARGET,
>  				 upper_32_bits(cpu_addr));
>  
> -	switch (as_type) {
> -	case DW_PCIE_AS_MEM:
> -		type = PCIE_ATU_TYPE_MEM;
> -		break;
> -	case DW_PCIE_AS_IO:
> -		type = PCIE_ATU_TYPE_IO;
> -		break;
> -	default:
> -		return -EINVAL;
> -	}
> -
>  	dw_pcie_writel_ib_unroll(pci, index, PCIE_ATU_UNR_REGION_CTRL1, type |
>  				 PCIE_ATU_FUNC_NUM(func_no));
>  	dw_pcie_writel_ib_unroll(pci, index, PCIE_ATU_UNR_REGION_CTRL2,
> @@ -468,32 +456,19 @@ static int dw_pcie_prog_inbound_atu_unroll(struct dw_pcie *pci, u8 func_no,
>  }
>  
>  int dw_pcie_prog_inbound_atu(struct dw_pcie *pci, u8 func_no, int index,
> -			     int bar, u64 cpu_addr,
> -			     enum dw_pcie_as_type as_type)
> +			     int type, u64 cpu_addr, u8 bar)
>  {
> -	int type;
>  	u32 retries, val;
>  
>  	if (pci->iatu_unroll_enabled)
> -		return dw_pcie_prog_inbound_atu_unroll(pci, func_no, index, bar,
> -						       cpu_addr, as_type);
> +		return dw_pcie_prog_inbound_atu_unroll(pci, func_no, index, type,
> +						       cpu_addr, bar);
>  
>  	dw_pcie_writel_dbi(pci, PCIE_ATU_VIEWPORT, PCIE_ATU_REGION_INBOUND |
>  			   index);
>  	dw_pcie_writel_dbi(pci, PCIE_ATU_LOWER_TARGET, lower_32_bits(cpu_addr));
>  	dw_pcie_writel_dbi(pci, PCIE_ATU_UPPER_TARGET, upper_32_bits(cpu_addr));
>  
> -	switch (as_type) {
> -	case DW_PCIE_AS_MEM:
> -		type = PCIE_ATU_TYPE_MEM;
> -		break;
> -	case DW_PCIE_AS_IO:
> -		type = PCIE_ATU_TYPE_IO;
> -		break;
> -	default:
> -		return -EINVAL;
> -	}
> -
>  	dw_pcie_writel_dbi(pci, PCIE_ATU_CR1, type |
>  			   PCIE_ATU_FUNC_NUM(func_no));
>  	dw_pcie_writel_dbi(pci, PCIE_ATU_CR2, PCIE_ATU_ENABLE |
> diff --git a/drivers/pci/controller/dwc/pcie-designware.h b/drivers/pci/controller/dwc/pcie-designware.h
> index 7f1c00fa084d..c63ace3c3f25 100644
> --- a/drivers/pci/controller/dwc/pcie-designware.h
> +++ b/drivers/pci/controller/dwc/pcie-designware.h
> @@ -228,12 +228,6 @@ struct dw_pcie_rp {
>  	DECLARE_BITMAP(msi_irq_in_use, MAX_MSI_IRQS);
>  };
>  
> -enum dw_pcie_as_type {
> -	DW_PCIE_AS_UNKNOWN,
> -	DW_PCIE_AS_MEM,
> -	DW_PCIE_AS_IO,
> -};
> -
>  struct dw_pcie_ep_ops {
>  	void	(*ep_init)(struct dw_pcie_ep *ep);
>  	int	(*raise_irq)(struct dw_pcie_ep *ep, u8 func_no,
> @@ -331,8 +325,7 @@ void dw_pcie_prog_ep_outbound_atu(struct dw_pcie *pci, u8 func_no, int index,
>  				  int type, u64 cpu_addr, u64 pci_addr,
>  				  u64 size);
>  int dw_pcie_prog_inbound_atu(struct dw_pcie *pci, u8 func_no, int index,
> -			     int bar, u64 cpu_addr,
> -			     enum dw_pcie_as_type as_type);
> +			     int type, u64 cpu_addr, u8 bar);
>  void dw_pcie_disable_atu(struct dw_pcie *pci, int index,
>  			 enum dw_pcie_region_type type);
>  void dw_pcie_setup(struct dw_pcie *pci);
> -- 
> 2.35.1
> 

-- 
மணிவண்ணன் சதாசிவம்

[PATCH RESEND v4 10/15] PCI: dwc: Drop iATU regions enumeration - dw_pcie_region_type

[Serge Semin]

There is no point in having the dw_pcie_region_type enumeration for almost
the same reasons as it was stated for dw_pcie_as_type. First of all it's
redundant since the driver already has a set of the macros declared which
describe the possible inbound and outbound iATU regions. Having an
addition abstraction just needlessly complicates the code. Secondly
checking the region type passed to the dw_pcie_disable_atu() method for
validity is pointless since the erroneous situation is just ignored in the
current method implementation. So to speak let's drop the redundant
dw_pcie_region_type enumeration replacing it with the direct iATU
direction macro usage.

Since the dw_pcie_disable_atu() method now directly accepts the
in-/outbound iATU region direction instead of the abstract region type we
need to change the argument name and the arguments order. The later change
makes the function prototype looking more logical since the passed index
indicates an iATU window within the regions with the corresponding
direction.

Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>
Reviewed-by: Rob Herring <robh@kernel.org>

---

Changelog v2:
- Move this patch to being applied before the IB/OB iATU windows setup
  simplification patch (@Rob).
---
 .../pci/controller/dwc/pcie-designware-ep.c   |  4 +--
 .../pci/controller/dwc/pcie-designware-host.c |  2 +-
 drivers/pci/controller/dwc/pcie-designware.c  | 28 +++++--------------
 drivers/pci/controller/dwc/pcie-designware.h  | 13 ++-------
 4 files changed, 13 insertions(+), 34 deletions(-)

diff --git a/drivers/pci/controller/dwc/pcie-designware-ep.c b/drivers/pci/controller/dwc/pcie-designware-ep.c
index 5a158813f687..2e91222f7c98 100644
--- a/drivers/pci/controller/dwc/pcie-designware-ep.c
+++ b/drivers/pci/controller/dwc/pcie-designware-ep.c
@@ -212,7 +212,7 @@ static void dw_pcie_ep_clear_bar(struct pci_epc *epc, u8 func_no, u8 vfunc_no,
 
 	__dw_pcie_ep_reset_bar(pci, func_no, bar, epf_bar->flags);
 
-	dw_pcie_disable_atu(pci, atu_index, DW_PCIE_REGION_INBOUND);
+	dw_pcie_disable_atu(pci, PCIE_ATU_REGION_DIR_IB, atu_index);
 	clear_bit(atu_index, ep->ib_window_map);
 	ep->epf_bar[bar] = NULL;
 }
@@ -286,7 +286,7 @@ static void dw_pcie_ep_unmap_addr(struct pci_epc *epc, u8 func_no, u8 vfunc_no,
 	if (ret < 0)
 		return;
 
-	dw_pcie_disable_atu(pci, atu_index, DW_PCIE_REGION_OUTBOUND);
+	dw_pcie_disable_atu(pci, PCIE_ATU_REGION_DIR_OB, atu_index);
 	clear_bit(atu_index, ep->ob_window_map);
 }
 
diff --git a/drivers/pci/controller/dwc/pcie-designware-host.c b/drivers/pci/controller/dwc/pcie-designware-host.c
index 4f984c845b59..e0a2819608c6 100644
--- a/drivers/pci/controller/dwc/pcie-designware-host.c
+++ b/drivers/pci/controller/dwc/pcie-designware-host.c
@@ -618,7 +618,7 @@ void dw_pcie_setup_rc(struct dw_pcie_rp *pp)
 		 * multiple matches
 		 */
 		for (i = 0; i < pci->num_ob_windows; i++)
-			dw_pcie_disable_atu(pci, i, DW_PCIE_REGION_OUTBOUND);
+			dw_pcie_disable_atu(pci, PCIE_ATU_REGION_DIR_OB, i);
 
 		/* Get last memory resource entry */
 		resource_list_for_each_entry(entry, &pp->bridge->windows) {
diff --git a/drivers/pci/controller/dwc/pcie-designware.c b/drivers/pci/controller/dwc/pcie-designware.c
index 330575182712..a90d3f6ce50c 100644
--- a/drivers/pci/controller/dwc/pcie-designware.c
+++ b/drivers/pci/controller/dwc/pcie-designware.c
@@ -353,7 +353,7 @@ static void __dw_pcie_prog_outbound_atu(struct dw_pcie *pci, u8 func_no,
 	limit_addr = cpu_addr + size - 1;
 
 	dw_pcie_writel_dbi(pci, PCIE_ATU_VIEWPORT,
-			   PCIE_ATU_REGION_OUTBOUND | index);
+			   PCIE_ATU_REGION_DIR_OB | index);
 	dw_pcie_writel_dbi(pci, PCIE_ATU_LOWER_BASE,
 			   lower_32_bits(cpu_addr));
 	dw_pcie_writel_dbi(pci, PCIE_ATU_UPPER_BASE,
@@ -464,7 +464,7 @@ int dw_pcie_prog_inbound_atu(struct dw_pcie *pci, u8 func_no, int index,
 		return dw_pcie_prog_inbound_atu_unroll(pci, func_no, index, type,
 						       cpu_addr, bar);
 
-	dw_pcie_writel_dbi(pci, PCIE_ATU_VIEWPORT, PCIE_ATU_REGION_INBOUND |
+	dw_pcie_writel_dbi(pci, PCIE_ATU_VIEWPORT, PCIE_ATU_REGION_DIR_IB |
 			   index);
 	dw_pcie_writel_dbi(pci, PCIE_ATU_LOWER_TARGET, lower_32_bits(cpu_addr));
 	dw_pcie_writel_dbi(pci, PCIE_ATU_UPPER_TARGET, upper_32_bits(cpu_addr));
@@ -491,24 +491,10 @@ int dw_pcie_prog_inbound_atu(struct dw_pcie *pci, u8 func_no, int index,
 	return -EBUSY;
 }
 
-void dw_pcie_disable_atu(struct dw_pcie *pci, int index,
-			 enum dw_pcie_region_type type)
+void dw_pcie_disable_atu(struct dw_pcie *pci, u32 dir, int index)
 {
-	u32 region;
-
-	switch (type) {
-	case DW_PCIE_REGION_INBOUND:
-		region = PCIE_ATU_REGION_INBOUND;
-		break;
-	case DW_PCIE_REGION_OUTBOUND:
-		region = PCIE_ATU_REGION_OUTBOUND;
-		break;
-	default:
-		return;
-	}
-
 	if (pci->iatu_unroll_enabled) {
-		if (region == PCIE_ATU_REGION_INBOUND) {
+		if (dir == PCIE_ATU_REGION_DIR_IB) {
 			dw_pcie_writel_ib_unroll(pci, index, PCIE_ATU_UNR_REGION_CTRL2,
 						 ~(u32)PCIE_ATU_ENABLE);
 		} else {
@@ -516,7 +502,7 @@ void dw_pcie_disable_atu(struct dw_pcie *pci, int index,
 						 ~(u32)PCIE_ATU_ENABLE);
 		}
 	} else {
-		dw_pcie_writel_dbi(pci, PCIE_ATU_VIEWPORT, region | index);
+		dw_pcie_writel_dbi(pci, PCIE_ATU_VIEWPORT, dir | index);
 		dw_pcie_writel_dbi(pci, PCIE_ATU_CR2, ~(u32)PCIE_ATU_ENABLE);
 	}
 }
@@ -661,7 +647,7 @@ static void dw_pcie_iatu_detect_regions(struct dw_pcie *pci)
 	max_region = dw_pcie_readl_dbi(pci, PCIE_ATU_VIEWPORT) + 1;
 
 	for (i = 0; i < max_region; i++) {
-		dw_pcie_writel_dbi(pci, PCIE_ATU_VIEWPORT, PCIE_ATU_REGION_OUTBOUND | i);
+		dw_pcie_writel_dbi(pci, PCIE_ATU_VIEWPORT, PCIE_ATU_REGION_DIR_OB | i);
 		dw_pcie_writel_dbi(pci, PCIE_ATU_LOWER_TARGET, 0x11110000);
 		val = dw_pcie_readl_dbi(pci, PCIE_ATU_LOWER_TARGET);
 		if (val == 0x11110000)
@@ -671,7 +657,7 @@ static void dw_pcie_iatu_detect_regions(struct dw_pcie *pci)
 	}
 
 	for (i = 0; i < max_region; i++) {
-		dw_pcie_writel_dbi(pci, PCIE_ATU_VIEWPORT, PCIE_ATU_REGION_INBOUND | i);
+		dw_pcie_writel_dbi(pci, PCIE_ATU_VIEWPORT, PCIE_ATU_REGION_DIR_IB | i);
 		dw_pcie_writel_dbi(pci, PCIE_ATU_LOWER_TARGET, 0x11110000);
 		val = dw_pcie_readl_dbi(pci, PCIE_ATU_LOWER_TARGET);
 		if (val == 0x11110000)
diff --git a/drivers/pci/controller/dwc/pcie-designware.h b/drivers/pci/controller/dwc/pcie-designware.h
index c63ace3c3f25..72d185ff72f3 100644
--- a/drivers/pci/controller/dwc/pcie-designware.h
+++ b/drivers/pci/controller/dwc/pcie-designware.h
@@ -104,8 +104,8 @@
 #define PCIE_VERSION_TYPE		0x8FC
 
 #define PCIE_ATU_VIEWPORT		0x900
-#define PCIE_ATU_REGION_INBOUND		BIT(31)
-#define PCIE_ATU_REGION_OUTBOUND	0
+#define PCIE_ATU_REGION_DIR_IB		BIT(31)
+#define PCIE_ATU_REGION_DIR_OB		0
 #define PCIE_ATU_CR1			0x904
 #define PCIE_ATU_INCREASE_REGION_SIZE	BIT(13)
 #define PCIE_ATU_TYPE_MEM		0x0
@@ -185,12 +185,6 @@ struct dw_pcie;
 struct dw_pcie_rp;
 struct dw_pcie_ep;
 
-enum dw_pcie_region_type {
-	DW_PCIE_REGION_UNKNOWN,
-	DW_PCIE_REGION_INBOUND,
-	DW_PCIE_REGION_OUTBOUND,
-};
-
 enum dw_pcie_device_mode {
 	DW_PCIE_UNKNOWN_TYPE,
 	DW_PCIE_EP_TYPE,
@@ -326,8 +320,7 @@ void dw_pcie_prog_ep_outbound_atu(struct dw_pcie *pci, u8 func_no, int index,
 				  u64 size);
 int dw_pcie_prog_inbound_atu(struct dw_pcie *pci, u8 func_no, int index,
 			     int type, u64 cpu_addr, u8 bar);
-void dw_pcie_disable_atu(struct dw_pcie *pci, int index,
-			 enum dw_pcie_region_type type);
+void dw_pcie_disable_atu(struct dw_pcie *pci, u32 dir, int index);
 void dw_pcie_setup(struct dw_pcie *pci);
 void dw_pcie_iatu_detect(struct dw_pcie *pci);
 
-- 
2.35.1

Re: [PATCH RESEND v4 10/15] PCI: dwc: Drop iATU regions enumeration - dw_pcie_region_type

[Manivannan Sadhasivam]

On Fri, Jun 24, 2022 at 05:39:42PM +0300, Serge Semin wrote:
> There is no point in having the dw_pcie_region_type enumeration for almost
> the same reasons as it was stated for dw_pcie_as_type. First of all it's
> redundant since the driver already has a set of the macros declared which
> describe the possible inbound and outbound iATU regions. Having an
> addition abstraction just needlessly complicates the code. Secondly
> checking the region type passed to the dw_pcie_disable_atu() method for
> validity is pointless since the erroneous situation is just ignored in the
> current method implementation. So to speak let's drop the redundant
> dw_pcie_region_type enumeration replacing it with the direct iATU
> direction macro usage.
> 
> Since the dw_pcie_disable_atu() method now directly accepts the
> in-/outbound iATU region direction instead of the abstract region type we
> need to change the argument name and the arguments order. The later change
> makes the function prototype looking more logical since the passed index
> indicates an iATU window within the regions with the corresponding
> direction.
> 
> Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>

Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>

Thanks,
Mani

> Reviewed-by: Rob Herring <robh@kernel.org>
> 
> ---
> 
> Changelog v2:
> - Move this patch to being applied before the IB/OB iATU windows setup
>   simplification patch (@Rob).
> ---
>  .../pci/controller/dwc/pcie-designware-ep.c   |  4 +--
>  .../pci/controller/dwc/pcie-designware-host.c |  2 +-
>  drivers/pci/controller/dwc/pcie-designware.c  | 28 +++++--------------
>  drivers/pci/controller/dwc/pcie-designware.h  | 13 ++-------
>  4 files changed, 13 insertions(+), 34 deletions(-)
> 
> diff --git a/drivers/pci/controller/dwc/pcie-designware-ep.c b/drivers/pci/controller/dwc/pcie-designware-ep.c
> index 5a158813f687..2e91222f7c98 100644
> --- a/drivers/pci/controller/dwc/pcie-designware-ep.c
> +++ b/drivers/pci/controller/dwc/pcie-designware-ep.c
> @@ -212,7 +212,7 @@ static void dw_pcie_ep_clear_bar(struct pci_epc *epc, u8 func_no, u8 vfunc_no,
>  
>  	__dw_pcie_ep_reset_bar(pci, func_no, bar, epf_bar->flags);
>  
> -	dw_pcie_disable_atu(pci, atu_index, DW_PCIE_REGION_INBOUND);
> +	dw_pcie_disable_atu(pci, PCIE_ATU_REGION_DIR_IB, atu_index);
>  	clear_bit(atu_index, ep->ib_window_map);
>  	ep->epf_bar[bar] = NULL;
>  }
> @@ -286,7 +286,7 @@ static void dw_pcie_ep_unmap_addr(struct pci_epc *epc, u8 func_no, u8 vfunc_no,
>  	if (ret < 0)
>  		return;
>  
> -	dw_pcie_disable_atu(pci, atu_index, DW_PCIE_REGION_OUTBOUND);
> +	dw_pcie_disable_atu(pci, PCIE_ATU_REGION_DIR_OB, atu_index);
>  	clear_bit(atu_index, ep->ob_window_map);
>  }
>  
> diff --git a/drivers/pci/controller/dwc/pcie-designware-host.c b/drivers/pci/controller/dwc/pcie-designware-host.c
> index 4f984c845b59..e0a2819608c6 100644
> --- a/drivers/pci/controller/dwc/pcie-designware-host.c
> +++ b/drivers/pci/controller/dwc/pcie-designware-host.c
> @@ -618,7 +618,7 @@ void dw_pcie_setup_rc(struct dw_pcie_rp *pp)
>  		 * multiple matches
>  		 */
>  		for (i = 0; i < pci->num_ob_windows; i++)
> -			dw_pcie_disable_atu(pci, i, DW_PCIE_REGION_OUTBOUND);
> +			dw_pcie_disable_atu(pci, PCIE_ATU_REGION_DIR_OB, i);
>  
>  		/* Get last memory resource entry */
>  		resource_list_for_each_entry(entry, &pp->bridge->windows) {
> diff --git a/drivers/pci/controller/dwc/pcie-designware.c b/drivers/pci/controller/dwc/pcie-designware.c
> index 330575182712..a90d3f6ce50c 100644
> --- a/drivers/pci/controller/dwc/pcie-designware.c
> +++ b/drivers/pci/controller/dwc/pcie-designware.c
> @@ -353,7 +353,7 @@ static void __dw_pcie_prog_outbound_atu(struct dw_pcie *pci, u8 func_no,
>  	limit_addr = cpu_addr + size - 1;
>  
>  	dw_pcie_writel_dbi(pci, PCIE_ATU_VIEWPORT,
> -			   PCIE_ATU_REGION_OUTBOUND | index);
> +			   PCIE_ATU_REGION_DIR_OB | index);
>  	dw_pcie_writel_dbi(pci, PCIE_ATU_LOWER_BASE,
>  			   lower_32_bits(cpu_addr));
>  	dw_pcie_writel_dbi(pci, PCIE_ATU_UPPER_BASE,
> @@ -464,7 +464,7 @@ int dw_pcie_prog_inbound_atu(struct dw_pcie *pci, u8 func_no, int index,
>  		return dw_pcie_prog_inbound_atu_unroll(pci, func_no, index, type,
>  						       cpu_addr, bar);
>  
> -	dw_pcie_writel_dbi(pci, PCIE_ATU_VIEWPORT, PCIE_ATU_REGION_INBOUND |
> +	dw_pcie_writel_dbi(pci, PCIE_ATU_VIEWPORT, PCIE_ATU_REGION_DIR_IB |
>  			   index);
>  	dw_pcie_writel_dbi(pci, PCIE_ATU_LOWER_TARGET, lower_32_bits(cpu_addr));
>  	dw_pcie_writel_dbi(pci, PCIE_ATU_UPPER_TARGET, upper_32_bits(cpu_addr));
> @@ -491,24 +491,10 @@ int dw_pcie_prog_inbound_atu(struct dw_pcie *pci, u8 func_no, int index,
>  	return -EBUSY;
>  }
>  
> -void dw_pcie_disable_atu(struct dw_pcie *pci, int index,
> -			 enum dw_pcie_region_type type)
> +void dw_pcie_disable_atu(struct dw_pcie *pci, u32 dir, int index)
>  {
> -	u32 region;
> -
> -	switch (type) {
> -	case DW_PCIE_REGION_INBOUND:
> -		region = PCIE_ATU_REGION_INBOUND;
> -		break;
> -	case DW_PCIE_REGION_OUTBOUND:
> -		region = PCIE_ATU_REGION_OUTBOUND;
> -		break;
> -	default:
> -		return;
> -	}
> -
>  	if (pci->iatu_unroll_enabled) {
> -		if (region == PCIE_ATU_REGION_INBOUND) {
> +		if (dir == PCIE_ATU_REGION_DIR_IB) {
>  			dw_pcie_writel_ib_unroll(pci, index, PCIE_ATU_UNR_REGION_CTRL2,
>  						 ~(u32)PCIE_ATU_ENABLE);
>  		} else {
> @@ -516,7 +502,7 @@ void dw_pcie_disable_atu(struct dw_pcie *pci, int index,
>  						 ~(u32)PCIE_ATU_ENABLE);
>  		}
>  	} else {
> -		dw_pcie_writel_dbi(pci, PCIE_ATU_VIEWPORT, region | index);
> +		dw_pcie_writel_dbi(pci, PCIE_ATU_VIEWPORT, dir | index);
>  		dw_pcie_writel_dbi(pci, PCIE_ATU_CR2, ~(u32)PCIE_ATU_ENABLE);
>  	}
>  }
> @@ -661,7 +647,7 @@ static void dw_pcie_iatu_detect_regions(struct dw_pcie *pci)
>  	max_region = dw_pcie_readl_dbi(pci, PCIE_ATU_VIEWPORT) + 1;
>  
>  	for (i = 0; i < max_region; i++) {
> -		dw_pcie_writel_dbi(pci, PCIE_ATU_VIEWPORT, PCIE_ATU_REGION_OUTBOUND | i);
> +		dw_pcie_writel_dbi(pci, PCIE_ATU_VIEWPORT, PCIE_ATU_REGION_DIR_OB | i);
>  		dw_pcie_writel_dbi(pci, PCIE_ATU_LOWER_TARGET, 0x11110000);
>  		val = dw_pcie_readl_dbi(pci, PCIE_ATU_LOWER_TARGET);
>  		if (val == 0x11110000)
> @@ -671,7 +657,7 @@ static void dw_pcie_iatu_detect_regions(struct dw_pcie *pci)
>  	}
>  
>  	for (i = 0; i < max_region; i++) {
> -		dw_pcie_writel_dbi(pci, PCIE_ATU_VIEWPORT, PCIE_ATU_REGION_INBOUND | i);
> +		dw_pcie_writel_dbi(pci, PCIE_ATU_VIEWPORT, PCIE_ATU_REGION_DIR_IB | i);
>  		dw_pcie_writel_dbi(pci, PCIE_ATU_LOWER_TARGET, 0x11110000);
>  		val = dw_pcie_readl_dbi(pci, PCIE_ATU_LOWER_TARGET);
>  		if (val == 0x11110000)
> diff --git a/drivers/pci/controller/dwc/pcie-designware.h b/drivers/pci/controller/dwc/pcie-designware.h
> index c63ace3c3f25..72d185ff72f3 100644
> --- a/drivers/pci/controller/dwc/pcie-designware.h
> +++ b/drivers/pci/controller/dwc/pcie-designware.h
> @@ -104,8 +104,8 @@
>  #define PCIE_VERSION_TYPE		0x8FC
>  
>  #define PCIE_ATU_VIEWPORT		0x900
> -#define PCIE_ATU_REGION_INBOUND		BIT(31)
> -#define PCIE_ATU_REGION_OUTBOUND	0
> +#define PCIE_ATU_REGION_DIR_IB		BIT(31)
> +#define PCIE_ATU_REGION_DIR_OB		0
>  #define PCIE_ATU_CR1			0x904
>  #define PCIE_ATU_INCREASE_REGION_SIZE	BIT(13)
>  #define PCIE_ATU_TYPE_MEM		0x0
> @@ -185,12 +185,6 @@ struct dw_pcie;
>  struct dw_pcie_rp;
>  struct dw_pcie_ep;
>  
> -enum dw_pcie_region_type {
> -	DW_PCIE_REGION_UNKNOWN,
> -	DW_PCIE_REGION_INBOUND,
> -	DW_PCIE_REGION_OUTBOUND,
> -};
> -
>  enum dw_pcie_device_mode {
>  	DW_PCIE_UNKNOWN_TYPE,
>  	DW_PCIE_EP_TYPE,
> @@ -326,8 +320,7 @@ void dw_pcie_prog_ep_outbound_atu(struct dw_pcie *pci, u8 func_no, int index,
>  				  u64 size);
>  int dw_pcie_prog_inbound_atu(struct dw_pcie *pci, u8 func_no, int index,
>  			     int type, u64 cpu_addr, u8 bar);
> -void dw_pcie_disable_atu(struct dw_pcie *pci, int index,
> -			 enum dw_pcie_region_type type);
> +void dw_pcie_disable_atu(struct dw_pcie *pci, u32 dir, int index);
>  void dw_pcie_setup(struct dw_pcie *pci);
>  void dw_pcie_iatu_detect(struct dw_pcie *pci);
>  
> -- 
> 2.35.1
> 

-- 
மணிவண்ணன் சதாசிவம்

[PATCH RESEND v4 11/15] PCI: dwc: Simplify in/outbound iATU setup methods

[Serge Semin]

From maintainability and scalability points of view it has been wrong to
use different iATU inbound and outbound regions accessors for the viewport
and unrolled versions of the iATU CSRs mapping. Seeing the particular iATU
region-wise registers layout is almost fully compatible for different
IP-core versions, there were no much points in splitting the code up that
way since it was possible to implement a common windows setup methods for
both viewport and unrolled iATU CSRs spaces. While what we can observe in
the current driver implementation of these methods, is a lot of code
duplication, which consequently worsen the code readability,
maintainability and scalability. Note the current implementation is a bit
more performant than the one suggested in this commit since it implies
having less MMIO accesses. But the gain just doesn't worth having the
denoted difficulties especially seeing the iATU setup methods are mainly
called on the DW PCIe controller and peripheral devices initialization
stage.

Here we suggest to move the iATU viewport and unrolled CSR access
specifics in the dw_pcie_readl_atu() and dw_pcie_writel_atu() method, and
convert the dw_pcie_prog_outbound_atu() and
dw_pcie_prog_{ep_}inbound_atu() functions to being generic instead of
having a different methods for each viewport and unrolled types of iATU
CSRs mapping. Nothing complex really. First of all the dw_pcie_readl_atu()
and dw_pcie_writel_atu() are converted to accept relative iATU CSRs
address together with the iATU region direction (inbound or outbound) and
region index. If DW PCIe controller doesn't have the unrolled iATU CSRs
space, then the accessors will need to activate a iATU viewport based on
the specified direction and index, otherwise a base address for the
corresponding region CSRs will be calculated by means of the
PCIE_ATU_UNROLL_BASE() macro. The CSRs macro have been modified in
accordance with that logic in the pcie-designware.h header file.

The rest of the changes in this commit just concern converting the iATU
in-/out-bound setup methods and iATU regions detection procedure to be
compatible with the new accessors semantics. As a result we've dropped the
no more required dw_pcie_prog_outbound_atu_unroll(),
dw_pcie_prog_inbound_atu_unroll() and dw_pcie_iatu_detect_regions_unroll()
methods.

Note aside with the denoted code improvements, there is an additional
positive side effect of this change. If at some point an atomic iATU
configs setup procedure is required, it will be possible to be done with
no much effort just by adding the synchronization into the
dw_pcie_readl_atu() and dw_pcie_writel_atu() accessors.

Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>

---

Note the pcie-tegra194-acpi.c driver has been fixed here to have it
utilizing the macros introduced in this patch. Judging by the code
semantics Tegra 194 ACPI has weird iATU mapping. It's clearly unrolled but
the CSRs are accessed over the viewport offsets. This couldn't have been
designed more confusing...

Changelog v2:
- Move the iATU region selection procedure into a helper function (@Rob).
- Simplify the iATU region selection procedure by recalculating the base
  address only if the space is unrolled. The iATU viewport base address
  is saved in the pci->atu_base field from now.

Changelog v3:
- Fix pcie-tegra194-acpi.c driver to using the new macros names.
  (@Manivannan)
---
 drivers/pci/controller/dwc/pcie-designware.c  | 293 ++++++------------
 drivers/pci/controller/dwc/pcie-designware.h  |  48 ++-
 .../pci/controller/dwc/pcie-tegra194-acpi.c   |   7 +-
 3 files changed, 111 insertions(+), 237 deletions(-)

diff --git a/drivers/pci/controller/dwc/pcie-designware.c b/drivers/pci/controller/dwc/pcie-designware.c
index a90d3f6ce50c..f2aa65d02a6c 100644
--- a/drivers/pci/controller/dwc/pcie-designware.c
+++ b/drivers/pci/controller/dwc/pcie-designware.c
@@ -205,48 +205,64 @@ void dw_pcie_write_dbi2(struct dw_pcie *pci, u32 reg, size_t size, u32 val)
 		dev_err(pci->dev, "write DBI address failed\n");
 }
 
-static u32 dw_pcie_readl_atu(struct dw_pcie *pci, u32 reg)
+static inline void __iomem *dw_pcie_select_atu(struct dw_pcie *pci, u32 dir,
+					       u32 index)
 {
+	void __iomem *base = pci->atu_base;
+
+	if (pci->iatu_unroll_enabled)
+		base += PCIE_ATU_UNROLL_BASE(dir, index);
+	else
+		dw_pcie_writel_dbi(pci, PCIE_ATU_VIEWPORT, dir | index);
+
+	return base;
+}
+
+static u32 dw_pcie_readl_atu(struct dw_pcie *pci, u32 dir, u32 index, u32 reg)
+{
+	void __iomem *base;
 	int ret;
 	u32 val;
 
+	base = dw_pcie_select_atu(pci, dir, index);
+
 	if (pci->ops && pci->ops->read_dbi)
-		return pci->ops->read_dbi(pci, pci->atu_base, reg, 4);
+		return pci->ops->read_dbi(pci, base, reg, 4);
 
-	ret = dw_pcie_read(pci->atu_base + reg, 4, &val);
+	ret = dw_pcie_read(base + reg, 4, &val);
 	if (ret)
 		dev_err(pci->dev, "Read ATU address failed\n");
 
 	return val;
 }
 
-static void dw_pcie_writel_atu(struct dw_pcie *pci, u32 reg, u32 val)
+static void dw_pcie_writel_atu(struct dw_pcie *pci, u32 dir, u32 index,
+			       u32 reg, u32 val)
 {
+	void __iomem *base;
 	int ret;
 
+	base = dw_pcie_select_atu(pci, dir, index);
+
 	if (pci->ops && pci->ops->write_dbi) {
-		pci->ops->write_dbi(pci, pci->atu_base, reg, 4, val);
+		pci->ops->write_dbi(pci, base, reg, 4, val);
 		return;
 	}
 
-	ret = dw_pcie_write(pci->atu_base + reg, 4, val);
+	ret = dw_pcie_write(base + reg, 4, val);
 	if (ret)
 		dev_err(pci->dev, "Write ATU address failed\n");
 }
 
-static u32 dw_pcie_readl_ob_unroll(struct dw_pcie *pci, u32 index, u32 reg)
+static inline u32 dw_pcie_readl_atu_ob(struct dw_pcie *pci, u32 index, u32 reg)
 {
-	u32 offset = PCIE_GET_ATU_OUTB_UNR_REG_OFFSET(index);
-
-	return dw_pcie_readl_atu(pci, offset + reg);
+	return dw_pcie_readl_atu(pci, PCIE_ATU_REGION_DIR_OB, index, reg);
 }
 
-static void dw_pcie_writel_ob_unroll(struct dw_pcie *pci, u32 index, u32 reg,
-				     u32 val)
+static inline void dw_pcie_writel_atu_ob(struct dw_pcie *pci, u32 index, u32 reg,
+					 u32 val)
 {
-	u32 offset = PCIE_GET_ATU_OUTB_UNR_REG_OFFSET(index);
-
-	dw_pcie_writel_atu(pci, offset + reg, val);
+	dw_pcie_writel_atu(pci, PCIE_ATU_REGION_DIR_OB, index, reg, val);
 }
 
 static inline u32 dw_pcie_enable_ecrc(u32 val)
@@ -290,50 +306,6 @@ static inline u32 dw_pcie_enable_ecrc(u32 val)
 	return val | PCIE_ATU_TD;
 }
 
-static void dw_pcie_prog_outbound_atu_unroll(struct dw_pcie *pci, u8 func_no,
-					     int index, int type,
-					     u64 cpu_addr, u64 pci_addr,
-					     u64 size)
-{
-	u32 retries, val;
-	u64 limit_addr = cpu_addr + size - 1;
-
-	dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_LOWER_BASE,
-				 lower_32_bits(cpu_addr));
-	dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_UPPER_BASE,
-				 upper_32_bits(cpu_addr));
-	dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_LOWER_LIMIT,
-				 lower_32_bits(limit_addr));
-	dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_UPPER_LIMIT,
-				 upper_32_bits(limit_addr));
-	dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_LOWER_TARGET,
-				 lower_32_bits(pci_addr));
-	dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_UPPER_TARGET,
-				 upper_32_bits(pci_addr));
-	val = type | PCIE_ATU_FUNC_NUM(func_no);
-	if (upper_32_bits(limit_addr) > upper_32_bits(cpu_addr))
-		val |= PCIE_ATU_INCREASE_REGION_SIZE;
-	if (dw_pcie_ver_is(pci, 490A))
-		val = dw_pcie_enable_ecrc(val);
-	dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_REGION_CTRL1, val);
-	dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_REGION_CTRL2,
-				 PCIE_ATU_ENABLE);
-
-	/*
-	 * Make sure ATU enable takes effect before any subsequent config
-	 * and I/O accesses.
-	 */
-	for (retries = 0; retries < LINK_WAIT_MAX_IATU_RETRIES; retries++) {
-		val = dw_pcie_readl_ob_unroll(pci, index,
-					      PCIE_ATU_UNR_REGION_CTRL2);
-		if (val & PCIE_ATU_ENABLE)
-			return;
-
-		mdelay(LINK_WAIT_IATU);
-	}
-	dev_err(pci->dev, "Outbound iATU is not being enabled\n");
-}
-
 static void __dw_pcie_prog_outbound_atu(struct dw_pcie *pci, u8 func_no,
 					int index, int type, u64 cpu_addr,
 					u64 pci_addr, u64 size)
@@ -344,49 +316,46 @@ static void __dw_pcie_prog_outbound_atu(struct dw_pcie *pci, u8 func_no,
 	if (pci->ops && pci->ops->cpu_addr_fixup)
 		cpu_addr = pci->ops->cpu_addr_fixup(pci, cpu_addr);
 
-	if (pci->iatu_unroll_enabled) {
-		dw_pcie_prog_outbound_atu_unroll(pci, func_no, index, type,
-						 cpu_addr, pci_addr, size);
-		return;
-	}
-
 	limit_addr = cpu_addr + size - 1;
 
-	dw_pcie_writel_dbi(pci, PCIE_ATU_VIEWPORT,
-			   PCIE_ATU_REGION_DIR_OB | index);
-	dw_pcie_writel_dbi(pci, PCIE_ATU_LOWER_BASE,
-			   lower_32_bits(cpu_addr));
-	dw_pcie_writel_dbi(pci, PCIE_ATU_UPPER_BASE,
-			   upper_32_bits(cpu_addr));
-	dw_pcie_writel_dbi(pci, PCIE_ATU_LIMIT,
-			   lower_32_bits(limit_addr));
+	dw_pcie_writel_atu_ob(pci, index, PCIE_ATU_LOWER_BASE,
+			      lower_32_bits(cpu_addr));
+	dw_pcie_writel_atu_ob(pci, index, PCIE_ATU_UPPER_BASE,
+			      upper_32_bits(cpu_addr));
+
+	dw_pcie_writel_atu_ob(pci, index, PCIE_ATU_LIMIT,
+			      lower_32_bits(limit_addr));
 	if (dw_pcie_ver_is_ge(pci, 460A))
-		dw_pcie_writel_dbi(pci, PCIE_ATU_UPPER_LIMIT,
-				   upper_32_bits(limit_addr));
-	dw_pcie_writel_dbi(pci, PCIE_ATU_LOWER_TARGET,
-			   lower_32_bits(pci_addr));
-	dw_pcie_writel_dbi(pci, PCIE_ATU_UPPER_TARGET,
-			   upper_32_bits(pci_addr));
+		dw_pcie_writel_atu_ob(pci, index, PCIE_ATU_UPPER_LIMIT,
+				      upper_32_bits(limit_addr));
+
+	dw_pcie_writel_atu_ob(pci, index, PCIE_ATU_LOWER_TARGET,
+			      lower_32_bits(pci_addr));
+	dw_pcie_writel_atu_ob(pci, index, PCIE_ATU_UPPER_TARGET,
+			      upper_32_bits(pci_addr));
+
 	val = type | PCIE_ATU_FUNC_NUM(func_no);
 	if (upper_32_bits(limit_addr) > upper_32_bits(cpu_addr) &&
 	    dw_pcie_ver_is_ge(pci, 460A))
 		val |= PCIE_ATU_INCREASE_REGION_SIZE;
 	if (dw_pcie_ver_is(pci, 490A))
 		val = dw_pcie_enable_ecrc(val);
-	dw_pcie_writel_dbi(pci, PCIE_ATU_CR1, val);
-	dw_pcie_writel_dbi(pci, PCIE_ATU_CR2, PCIE_ATU_ENABLE);
+	dw_pcie_writel_atu_ob(pci, index, PCIE_ATU_REGION_CTRL1, val);
+
+	dw_pcie_writel_atu_ob(pci, index, PCIE_ATU_REGION_CTRL2, PCIE_ATU_ENABLE);
 
 	/*
 	 * Make sure ATU enable takes effect before any subsequent config
 	 * and I/O accesses.
 	 */
 	for (retries = 0; retries < LINK_WAIT_MAX_IATU_RETRIES; retries++) {
-		val = dw_pcie_readl_dbi(pci, PCIE_ATU_CR2);
+		val = dw_pcie_readl_atu_ob(pci, index, PCIE_ATU_REGION_CTRL2);
 		if (val & PCIE_ATU_ENABLE)
 			return;
 
 		mdelay(LINK_WAIT_IATU);
 	}
+
 	dev_err(pci->dev, "Outbound iATU is not being enabled\n");
 }
 
@@ -405,54 +374,15 @@ void dw_pcie_prog_ep_outbound_atu(struct dw_pcie *pci, u8 func_no, int index,
 				    cpu_addr, pci_addr, size);
 }
 
-static u32 dw_pcie_readl_ib_unroll(struct dw_pcie *pci, u32 index, u32 reg)
-{
-	u32 offset = PCIE_GET_ATU_INB_UNR_REG_OFFSET(index);
-
-	return dw_pcie_readl_atu(pci, offset + reg);
-}
-
-static void dw_pcie_writel_ib_unroll(struct dw_pcie *pci, u32 index, u32 reg,
-				     u32 val)
+static inline u32 dw_pcie_readl_atu_ib(struct dw_pcie *pci, u32 index, u32 reg)
 {
-	u32 offset = PCIE_GET_ATU_INB_UNR_REG_OFFSET(index);
-
-	dw_pcie_writel_atu(pci, offset + reg, val);
+	return dw_pcie_readl_atu(pci, PCIE_ATU_REGION_DIR_IB, index, reg);
 }
 
-static int dw_pcie_prog_inbound_atu_unroll(struct dw_pcie *pci, u8 func_no,
-					   int index, int type,
-					   u64 cpu_addr, u8 bar)
+static inline void dw_pcie_writel_atu_ib(struct dw_pcie *pci, u32 index, u32 reg,
+					 u32 val)
 {
-	u32 retries, val;
-
-	dw_pcie_writel_ib_unroll(pci, index, PCIE_ATU_UNR_LOWER_TARGET,
-				 lower_32_bits(cpu_addr));
-	dw_pcie_writel_ib_unroll(pci, index, PCIE_ATU_UNR_UPPER_TARGET,
-				 upper_32_bits(cpu_addr));
-
-	dw_pcie_writel_ib_unroll(pci, index, PCIE_ATU_UNR_REGION_CTRL1, type |
-				 PCIE_ATU_FUNC_NUM(func_no));
-	dw_pcie_writel_ib_unroll(pci, index, PCIE_ATU_UNR_REGION_CTRL2,
-				 PCIE_ATU_FUNC_NUM_MATCH_EN |
-				 PCIE_ATU_ENABLE |
-				 PCIE_ATU_BAR_MODE_ENABLE | (bar << 8));
-
-	/*
-	 * Make sure ATU enable takes effect before any subsequent config
-	 * and I/O accesses.
-	 */
-	for (retries = 0; retries < LINK_WAIT_MAX_IATU_RETRIES; retries++) {
-		val = dw_pcie_readl_ib_unroll(pci, index,
-					      PCIE_ATU_UNR_REGION_CTRL2);
-		if (val & PCIE_ATU_ENABLE)
-			return 0;
-
-		mdelay(LINK_WAIT_IATU);
-	}
-	dev_err(pci->dev, "Inbound iATU is not being enabled\n");
-
-	return -EBUSY;
+	dw_pcie_writel_atu(pci, PCIE_ATU_REGION_DIR_IB, index, reg, val);
 }
 
 int dw_pcie_prog_inbound_atu(struct dw_pcie *pci, u8 func_no, int index,
@@ -460,51 +390,37 @@ int dw_pcie_prog_inbound_atu(struct dw_pcie *pci, u8 func_no, int index,
 {
 	u32 retries, val;
 
-	if (pci->iatu_unroll_enabled)
-		return dw_pcie_prog_inbound_atu_unroll(pci, func_no, index, type,
-						       cpu_addr, bar);
+	dw_pcie_writel_atu_ib(pci, index, PCIE_ATU_LOWER_TARGET,
+			      lower_32_bits(cpu_addr));
+	dw_pcie_writel_atu_ib(pci, index, PCIE_ATU_UPPER_TARGET,
+			      upper_32_bits(cpu_addr));
 
-	dw_pcie_writel_dbi(pci, PCIE_ATU_VIEWPORT, PCIE_ATU_REGION_DIR_IB |
-			   index);
-	dw_pcie_writel_dbi(pci, PCIE_ATU_LOWER_TARGET, lower_32_bits(cpu_addr));
-	dw_pcie_writel_dbi(pci, PCIE_ATU_UPPER_TARGET, upper_32_bits(cpu_addr));
-
-	dw_pcie_writel_dbi(pci, PCIE_ATU_CR1, type |
-			   PCIE_ATU_FUNC_NUM(func_no));
-	dw_pcie_writel_dbi(pci, PCIE_ATU_CR2, PCIE_ATU_ENABLE |
-			   PCIE_ATU_FUNC_NUM_MATCH_EN |
-			   PCIE_ATU_BAR_MODE_ENABLE | (bar << 8));
+	dw_pcie_writel_atu_ib(pci, index, PCIE_ATU_REGION_CTRL1, type |
+			      PCIE_ATU_FUNC_NUM(func_no));
+	dw_pcie_writel_atu_ib(pci, index, PCIE_ATU_REGION_CTRL2,
+			      PCIE_ATU_ENABLE | PCIE_ATU_FUNC_NUM_MATCH_EN |
+			      PCIE_ATU_BAR_MODE_ENABLE | (bar << 8));
 
 	/*
 	 * Make sure ATU enable takes effect before any subsequent config
 	 * and I/O accesses.
 	 */
 	for (retries = 0; retries < LINK_WAIT_MAX_IATU_RETRIES; retries++) {
-		val = dw_pcie_readl_dbi(pci, PCIE_ATU_CR2);
+		val = dw_pcie_readl_atu_ib(pci, index, PCIE_ATU_REGION_CTRL2);
 		if (val & PCIE_ATU_ENABLE)
 			return 0;
 
 		mdelay(LINK_WAIT_IATU);
 	}
+
 	dev_err(pci->dev, "Inbound iATU is not being enabled\n");
 
-	return -EBUSY;
+	return -ETIMEDOUT;
 }
 
 void dw_pcie_disable_atu(struct dw_pcie *pci, u32 dir, int index)
 {
-	if (pci->iatu_unroll_enabled) {
-		if (dir == PCIE_ATU_REGION_DIR_IB) {
-			dw_pcie_writel_ib_unroll(pci, index, PCIE_ATU_UNR_REGION_CTRL2,
-						 ~(u32)PCIE_ATU_ENABLE);
-		} else {
-			dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_REGION_CTRL2,
-						 ~(u32)PCIE_ATU_ENABLE);
-		}
-	} else {
-		dw_pcie_writel_dbi(pci, PCIE_ATU_VIEWPORT, dir | index);
-		dw_pcie_writel_dbi(pci, PCIE_ATU_CR2, ~(u32)PCIE_ATU_ENABLE);
-	}
+	dw_pcie_writel_atu(pci, dir, index, PCIE_ATU_REGION_CTRL2, 0);
 }
 
 int dw_pcie_wait_for_link(struct dw_pcie *pci)
@@ -606,63 +522,29 @@ static bool dw_pcie_iatu_unroll_enabled(struct dw_pcie *pci)
 	return false;
 }
 
-static void dw_pcie_iatu_detect_regions_unroll(struct dw_pcie *pci)
-{
-	int max_region, i, ob = 0, ib = 0;
-	u32 val;
-
-	max_region = min((int)pci->atu_size / 512, 256);
-
-	for (i = 0; i < max_region; i++) {
-		dw_pcie_writel_ob_unroll(pci, i, PCIE_ATU_UNR_LOWER_TARGET,
-					0x11110000);
-
-		val = dw_pcie_readl_ob_unroll(pci, i, PCIE_ATU_UNR_LOWER_TARGET);
-		if (val == 0x11110000)
-			ob++;
-		else
-			break;
-	}
-
-	for (i = 0; i < max_region; i++) {
-		dw_pcie_writel_ib_unroll(pci, i, PCIE_ATU_UNR_LOWER_TARGET,
-					0x11110000);
-
-		val = dw_pcie_readl_ib_unroll(pci, i, PCIE_ATU_UNR_LOWER_TARGET);
-		if (val == 0x11110000)
-			ib++;
-		else
-			break;
-	}
-	pci->num_ib_windows = ib;
-	pci->num_ob_windows = ob;
-}
-
 static void dw_pcie_iatu_detect_regions(struct dw_pcie *pci)
 {
-	int max_region, i, ob = 0, ib = 0;
+	int max_region, ob, ib;
 	u32 val;
 
-	dw_pcie_writel_dbi(pci, PCIE_ATU_VIEWPORT, 0xFF);
-	max_region = dw_pcie_readl_dbi(pci, PCIE_ATU_VIEWPORT) + 1;
+	if (pci->iatu_unroll_enabled) {
+		max_region = min((int)pci->atu_size / 512, 256);
+	} else {
+		dw_pcie_writel_dbi(pci, PCIE_ATU_VIEWPORT, 0xFF);
+		max_region = dw_pcie_readl_dbi(pci, PCIE_ATU_VIEWPORT) + 1;
+	}
 
-	for (i = 0; i < max_region; i++) {
-		dw_pcie_writel_dbi(pci, PCIE_ATU_VIEWPORT, PCIE_ATU_REGION_DIR_OB | i);
-		dw_pcie_writel_dbi(pci, PCIE_ATU_LOWER_TARGET, 0x11110000);
-		val = dw_pcie_readl_dbi(pci, PCIE_ATU_LOWER_TARGET);
-		if (val == 0x11110000)
-			ob++;
-		else
+	for (ob = 0; ob < max_region; ob++) {
+		dw_pcie_writel_atu_ob(pci, ob, PCIE_ATU_LOWER_TARGET, 0x11110000);
+		val = dw_pcie_readl_atu_ob(pci, ob, PCIE_ATU_LOWER_TARGET);
+		if (val != 0x11110000)
 			break;
 	}
 
-	for (i = 0; i < max_region; i++) {
-		dw_pcie_writel_dbi(pci, PCIE_ATU_VIEWPORT, PCIE_ATU_REGION_DIR_IB | i);
-		dw_pcie_writel_dbi(pci, PCIE_ATU_LOWER_TARGET, 0x11110000);
-		val = dw_pcie_readl_dbi(pci, PCIE_ATU_LOWER_TARGET);
-		if (val == 0x11110000)
-			ib++;
-		else
+	for (ib = 0; ib < max_region; ib++) {
+		dw_pcie_writel_atu_ib(pci, ib, PCIE_ATU_LOWER_TARGET, 0x11110000);
+		val = dw_pcie_readl_atu_ib(pci, ib, PCIE_ATU_LOWER_TARGET);
+		if (val != 0x11110000)
 			break;
 	}
 
@@ -690,12 +572,13 @@ void dw_pcie_iatu_detect(struct dw_pcie *pci)
 		if (!pci->atu_size)
 			/* Pick a minimal default, enough for 8 in and 8 out windows */
 			pci->atu_size = SZ_4K;
-
-		dw_pcie_iatu_detect_regions_unroll(pci);
 	} else {
-		dw_pcie_iatu_detect_regions(pci);
+		pci->atu_base = pci->dbi_base + PCIE_ATU_VIEWPORT_BASE;
+		pci->atu_size = PCIE_ATU_VIEWPORT_SIZE;
 	}
 
+	dw_pcie_iatu_detect_regions(pci);
+
 	dev_info(pci->dev, "iATU unroll: %s\n", pci->iatu_unroll_enabled ?
 		"enabled" : "disabled");
 
diff --git a/drivers/pci/controller/dwc/pcie-designware.h b/drivers/pci/controller/dwc/pcie-designware.h
index 72d185ff72f3..c18f0db09b31 100644
--- a/drivers/pci/controller/dwc/pcie-designware.h
+++ b/drivers/pci/controller/dwc/pcie-designware.h
@@ -103,10 +103,21 @@
 #define PCIE_VERSION_NUMBER		0x8F8
 #define PCIE_VERSION_TYPE		0x8FC
 
+/*
+ * iATU inbound and outbound windows CSRs. Before the IP-core v4.80a each
+ * iATU region CSRs had been indirectly accessible by means of the dedicated
+ * viewport selector. The iATU/eDMA CSRs space was re-designed in DWC PCIe
+ * v4.80a in a way so the viewport was unrolled into the directly accessible
+ * iATU/eDMA CSRs space.
+ */
 #define PCIE_ATU_VIEWPORT		0x900
 #define PCIE_ATU_REGION_DIR_IB		BIT(31)
 #define PCIE_ATU_REGION_DIR_OB		0
-#define PCIE_ATU_CR1			0x904
+#define PCIE_ATU_VIEWPORT_BASE		0x904
+#define PCIE_ATU_UNROLL_BASE(dir, index) \
+	(((index) << 9) | ((dir == PCIE_ATU_REGION_DIR_IB) ? BIT(8) : 0))
+#define PCIE_ATU_VIEWPORT_SIZE		0x2C
+#define PCIE_ATU_REGION_CTRL1		0x000
 #define PCIE_ATU_INCREASE_REGION_SIZE	BIT(13)
 #define PCIE_ATU_TYPE_MEM		0x0
 #define PCIE_ATU_TYPE_IO		0x2
@@ -114,19 +125,19 @@
 #define PCIE_ATU_TYPE_CFG1		0x5
 #define PCIE_ATU_TD			BIT(8)
 #define PCIE_ATU_FUNC_NUM(pf)           ((pf) << 20)
-#define PCIE_ATU_CR2			0x908
+#define PCIE_ATU_REGION_CTRL2		0x004
 #define PCIE_ATU_ENABLE			BIT(31)
 #define PCIE_ATU_BAR_MODE_ENABLE	BIT(30)
 #define PCIE_ATU_FUNC_NUM_MATCH_EN      BIT(19)
-#define PCIE_ATU_LOWER_BASE		0x90C
-#define PCIE_ATU_UPPER_BASE		0x910
-#define PCIE_ATU_LIMIT			0x914
-#define PCIE_ATU_LOWER_TARGET		0x918
+#define PCIE_ATU_LOWER_BASE		0x008
+#define PCIE_ATU_UPPER_BASE		0x00C
+#define PCIE_ATU_LIMIT			0x010
+#define PCIE_ATU_LOWER_TARGET		0x014
 #define PCIE_ATU_BUS(x)			FIELD_PREP(GENMASK(31, 24), x)
 #define PCIE_ATU_DEV(x)			FIELD_PREP(GENMASK(23, 19), x)
 #define PCIE_ATU_FUNC(x)		FIELD_PREP(GENMASK(18, 16), x)
-#define PCIE_ATU_UPPER_TARGET		0x91C
-#define PCIE_ATU_UPPER_LIMIT		0x924
+#define PCIE_ATU_UPPER_TARGET		0x018
+#define PCIE_ATU_UPPER_LIMIT		0x020
 
 #define PCIE_MISC_CONTROL_1_OFF		0x8BC
 #define PCIE_DBI_RO_WR_EN		BIT(0)
@@ -143,19 +154,6 @@
 
 #define PCIE_PL_CHK_REG_ERR_ADDR			0xB28
 
-/*
- * iATU Unroll-specific register definitions
- * From 4.80 core version the address translation will be made by unroll
- */
-#define PCIE_ATU_UNR_REGION_CTRL1	0x00
-#define PCIE_ATU_UNR_REGION_CTRL2	0x04
-#define PCIE_ATU_UNR_LOWER_BASE		0x08
-#define PCIE_ATU_UNR_UPPER_BASE		0x0C
-#define PCIE_ATU_UNR_LOWER_LIMIT	0x10
-#define PCIE_ATU_UNR_LOWER_TARGET	0x14
-#define PCIE_ATU_UNR_UPPER_TARGET	0x18
-#define PCIE_ATU_UNR_UPPER_LIMIT	0x20
-
 /*
  * The default address offset between dbi_base and atu_base. Root controller
  * drivers are not required to initialize atu_base if the offset matches this
@@ -164,13 +162,6 @@
  */
 #define DEFAULT_DBI_ATU_OFFSET (0x3 << 20)
 
-/* Register address builder */
-#define PCIE_GET_ATU_OUTB_UNR_REG_OFFSET(region) \
-		((region) << 9)
-
-#define PCIE_GET_ATU_INB_UNR_REG_OFFSET(region) \
-		(((region) << 9) | BIT(8))
-
 #define MAX_MSI_IRQS			256
 #define MAX_MSI_IRQS_PER_CTRL		32
 #define MAX_MSI_CTRLS			(MAX_MSI_IRQS / MAX_MSI_IRQS_PER_CTRL)
@@ -277,7 +268,6 @@ struct dw_pcie {
 	struct device		*dev;
 	void __iomem		*dbi_base;
 	void __iomem		*dbi_base2;
-	/* Used when iatu_unroll_enabled is true */
 	void __iomem		*atu_base;
 	size_t			atu_size;
 	u32			num_ib_windows;
diff --git a/drivers/pci/controller/dwc/pcie-tegra194-acpi.c b/drivers/pci/controller/dwc/pcie-tegra194-acpi.c
index c2de6ed4d86f..55f61914a986 100644
--- a/drivers/pci/controller/dwc/pcie-tegra194-acpi.c
+++ b/drivers/pci/controller/dwc/pcie-tegra194-acpi.c
@@ -39,7 +39,8 @@ static int tegra194_acpi_init(struct pci_config_window *cfg)
 static void atu_reg_write(struct tegra194_pcie_ecam *pcie_ecam, int index,
 			  u32 val, u32 reg)
 {
-	u32 offset = PCIE_GET_ATU_OUTB_UNR_REG_OFFSET(index);
+	u32 offset = PCIE_ATU_UNROLL_BASE(PCIE_ATU_REGION_DIR_OB, index) +
+		     PCIE_ATU_VIEWPORT_BASE;
 
 	writel(val, pcie_ecam->iatu_base + offset + reg);
 }
@@ -58,8 +59,8 @@ static void program_outbound_atu(struct tegra194_pcie_ecam *pcie_ecam,
 		      PCIE_ATU_LIMIT);
 	atu_reg_write(pcie_ecam, index, upper_32_bits(pci_addr),
 		      PCIE_ATU_UPPER_TARGET);
-	atu_reg_write(pcie_ecam, index, type, PCIE_ATU_CR1);
-	atu_reg_write(pcie_ecam, index, PCIE_ATU_ENABLE, PCIE_ATU_CR2);
+	atu_reg_write(pcie_ecam, index, type, PCIE_ATU_REGION_CTRL1);
+	atu_reg_write(pcie_ecam, index, PCIE_ATU_ENABLE, PCIE_ATU_REGION_CTRL2);
 }
 
 static void __iomem *tegra194_map_bus(struct pci_bus *bus,
-- 
2.35.1

Re: [PATCH RESEND v4 11/15] PCI: dwc: Simplify in/outbound iATU setup methods

[Manivannan Sadhasivam]

On Fri, Jun 24, 2022 at 05:39:43PM +0300, Serge Semin wrote:
> From maintainability and scalability points of view it has been wrong to
> use different iATU inbound and outbound regions accessors for the viewport
> and unrolled versions of the iATU CSRs mapping. Seeing the particular iATU
> region-wise registers layout is almost fully compatible for different
> IP-core versions, there were no much points in splitting the code up that
> way since it was possible to implement a common windows setup methods for
> both viewport and unrolled iATU CSRs spaces. While what we can observe in
> the current driver implementation of these methods, is a lot of code
> duplication, which consequently worsen the code readability,
> maintainability and scalability. Note the current implementation is a bit
> more performant than the one suggested in this commit since it implies
> having less MMIO accesses. But the gain just doesn't worth having the
> denoted difficulties especially seeing the iATU setup methods are mainly
> called on the DW PCIe controller and peripheral devices initialization
> stage.
> 
> Here we suggest to move the iATU viewport and unrolled CSR access
> specifics in the dw_pcie_readl_atu() and dw_pcie_writel_atu() method, and
> convert the dw_pcie_prog_outbound_atu() and
> dw_pcie_prog_{ep_}inbound_atu() functions to being generic instead of
> having a different methods for each viewport and unrolled types of iATU
> CSRs mapping. Nothing complex really. First of all the dw_pcie_readl_atu()
> and dw_pcie_writel_atu() are converted to accept relative iATU CSRs
> address together with the iATU region direction (inbound or outbound) and
> region index. If DW PCIe controller doesn't have the unrolled iATU CSRs
> space, then the accessors will need to activate a iATU viewport based on
> the specified direction and index, otherwise a base address for the
> corresponding region CSRs will be calculated by means of the
> PCIE_ATU_UNROLL_BASE() macro. The CSRs macro have been modified in
> accordance with that logic in the pcie-designware.h header file.
> 
> The rest of the changes in this commit just concern converting the iATU
> in-/out-bound setup methods and iATU regions detection procedure to be
> compatible with the new accessors semantics. As a result we've dropped the
> no more required dw_pcie_prog_outbound_atu_unroll(),
> dw_pcie_prog_inbound_atu_unroll() and dw_pcie_iatu_detect_regions_unroll()
> methods.
> 
> Note aside with the denoted code improvements, there is an additional
> positive side effect of this change. If at some point an atomic iATU
> configs setup procedure is required, it will be possible to be done with
> no much effort just by adding the synchronization into the
> dw_pcie_readl_atu() and dw_pcie_writel_atu() accessors.
> 
> Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>

One nitpick mentioned below. With that fixed,

Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>

> 
> ---
> 
> Note the pcie-tegra194-acpi.c driver has been fixed here to have it
> utilizing the macros introduced in this patch. Judging by the code
> semantics Tegra 194 ACPI has weird iATU mapping. It's clearly unrolled but
> the CSRs are accessed over the viewport offsets. This couldn't have been
> designed more confusing...
> 
> Changelog v2:
> - Move the iATU region selection procedure into a helper function (@Rob).
> - Simplify the iATU region selection procedure by recalculating the base
>   address only if the space is unrolled. The iATU viewport base address
>   is saved in the pci->atu_base field from now.
> 
> Changelog v3:
> - Fix pcie-tegra194-acpi.c driver to using the new macros names.
>   (@Manivannan)
> ---
>  drivers/pci/controller/dwc/pcie-designware.c  | 293 ++++++------------
>  drivers/pci/controller/dwc/pcie-designware.h  |  48 ++-
>  .../pci/controller/dwc/pcie-tegra194-acpi.c   |   7 +-
>  3 files changed, 111 insertions(+), 237 deletions(-)
> 
> diff --git a/drivers/pci/controller/dwc/pcie-designware.c b/drivers/pci/controller/dwc/pcie-designware.c
> index a90d3f6ce50c..f2aa65d02a6c 100644
> --- a/drivers/pci/controller/dwc/pcie-designware.c
> +++ b/drivers/pci/controller/dwc/pcie-designware.c
> @@ -205,48 +205,64 @@ void dw_pcie_write_dbi2(struct dw_pcie *pci, u32 reg, size_t size, u32 val)
>  		dev_err(pci->dev, "write DBI address failed\n");
>  }
>  
> -static u32 dw_pcie_readl_atu(struct dw_pcie *pci, u32 reg)
> +static inline void __iomem *dw_pcie_select_atu(struct dw_pcie *pci, u32 dir,

This could be renamed to "dw_pcie_get_atu_base()" since we are anyway getting
the base address of iATU.

Thanks,
Mani

> +					       u32 index)
>  {
> +	void __iomem *base = pci->atu_base;
> +
> +	if (pci->iatu_unroll_enabled)
> +		base += PCIE_ATU_UNROLL_BASE(dir, index);
> +	else
> +		dw_pcie_writel_dbi(pci, PCIE_ATU_VIEWPORT, dir | index);
> +
> +	return base;
> +}
> +
> +static u32 dw_pcie_readl_atu(struct dw_pcie *pci, u32 dir, u32 index, u32 reg)
> +{
> +	void __iomem *base;
>  	int ret;
>  	u32 val;
>  
> +	base = dw_pcie_select_atu(pci, dir, index);
> +
>  	if (pci->ops && pci->ops->read_dbi)
> -		return pci->ops->read_dbi(pci, pci->atu_base, reg, 4);
> +		return pci->ops->read_dbi(pci, base, reg, 4);
>  
> -	ret = dw_pcie_read(pci->atu_base + reg, 4, &val);
> +	ret = dw_pcie_read(base + reg, 4, &val);
>  	if (ret)
>  		dev_err(pci->dev, "Read ATU address failed\n");
>  
>  	return val;
>  }
>  
> -static void dw_pcie_writel_atu(struct dw_pcie *pci, u32 reg, u32 val)
> +static void dw_pcie_writel_atu(struct dw_pcie *pci, u32 dir, u32 index,
> +			       u32 reg, u32 val)
>  {
> +	void __iomem *base;
>  	int ret;
>  
> +	base = dw_pcie_select_atu(pci, dir, index);
> +
>  	if (pci->ops && pci->ops->write_dbi) {
> -		pci->ops->write_dbi(pci, pci->atu_base, reg, 4, val);
> +		pci->ops->write_dbi(pci, base, reg, 4, val);
>  		return;
>  	}
>  
> -	ret = dw_pcie_write(pci->atu_base + reg, 4, val);
> +	ret = dw_pcie_write(base + reg, 4, val);
>  	if (ret)
>  		dev_err(pci->dev, "Write ATU address failed\n");
>  }
>  
> -static u32 dw_pcie_readl_ob_unroll(struct dw_pcie *pci, u32 index, u32 reg)
> +static inline u32 dw_pcie_readl_atu_ob(struct dw_pcie *pci, u32 index, u32 reg)
>  {
> -	u32 offset = PCIE_GET_ATU_OUTB_UNR_REG_OFFSET(index);
> -
> -	return dw_pcie_readl_atu(pci, offset + reg);
> +	return dw_pcie_readl_atu(pci, PCIE_ATU_REGION_DIR_OB, index, reg);
>  }
>  
> -static void dw_pcie_writel_ob_unroll(struct dw_pcie *pci, u32 index, u32 reg,
> -				     u32 val)
> +static inline void dw_pcie_writel_atu_ob(struct dw_pcie *pci, u32 index, u32 reg,
> +					 u32 val)
>  {
> -	u32 offset = PCIE_GET_ATU_OUTB_UNR_REG_OFFSET(index);
> -
> -	dw_pcie_writel_atu(pci, offset + reg, val);
> +	dw_pcie_writel_atu(pci, PCIE_ATU_REGION_DIR_OB, index, reg, val);
>  }
>  
>  static inline u32 dw_pcie_enable_ecrc(u32 val)
> @@ -290,50 +306,6 @@ static inline u32 dw_pcie_enable_ecrc(u32 val)
>  	return val | PCIE_ATU_TD;
>  }
>  
> -static void dw_pcie_prog_outbound_atu_unroll(struct dw_pcie *pci, u8 func_no,
> -					     int index, int type,
> -					     u64 cpu_addr, u64 pci_addr,
> -					     u64 size)
> -{
> -	u32 retries, val;
> -	u64 limit_addr = cpu_addr + size - 1;
> -
> -	dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_LOWER_BASE,
> -				 lower_32_bits(cpu_addr));
> -	dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_UPPER_BASE,
> -				 upper_32_bits(cpu_addr));
> -	dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_LOWER_LIMIT,
> -				 lower_32_bits(limit_addr));
> -	dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_UPPER_LIMIT,
> -				 upper_32_bits(limit_addr));
> -	dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_LOWER_TARGET,
> -				 lower_32_bits(pci_addr));
> -	dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_UPPER_TARGET,
> -				 upper_32_bits(pci_addr));
> -	val = type | PCIE_ATU_FUNC_NUM(func_no);
> -	if (upper_32_bits(limit_addr) > upper_32_bits(cpu_addr))
> -		val |= PCIE_ATU_INCREASE_REGION_SIZE;
> -	if (dw_pcie_ver_is(pci, 490A))
> -		val = dw_pcie_enable_ecrc(val);
> -	dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_REGION_CTRL1, val);
> -	dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_REGION_CTRL2,
> -				 PCIE_ATU_ENABLE);
> -
> -	/*
> -	 * Make sure ATU enable takes effect before any subsequent config
> -	 * and I/O accesses.
> -	 */
> -	for (retries = 0; retries < LINK_WAIT_MAX_IATU_RETRIES; retries++) {
> -		val = dw_pcie_readl_ob_unroll(pci, index,
> -					      PCIE_ATU_UNR_REGION_CTRL2);
> -		if (val & PCIE_ATU_ENABLE)
> -			return;
> -
> -		mdelay(LINK_WAIT_IATU);
> -	}
> -	dev_err(pci->dev, "Outbound iATU is not being enabled\n");
> -}
> -
>  static void __dw_pcie_prog_outbound_atu(struct dw_pcie *pci, u8 func_no,
>  					int index, int type, u64 cpu_addr,
>  					u64 pci_addr, u64 size)
> @@ -344,49 +316,46 @@ static void __dw_pcie_prog_outbound_atu(struct dw_pcie *pci, u8 func_no,
>  	if (pci->ops && pci->ops->cpu_addr_fixup)
>  		cpu_addr = pci->ops->cpu_addr_fixup(pci, cpu_addr);
>  
> -	if (pci->iatu_unroll_enabled) {
> -		dw_pcie_prog_outbound_atu_unroll(pci, func_no, index, type,
> -						 cpu_addr, pci_addr, size);
> -		return;
> -	}
> -
>  	limit_addr = cpu_addr + size - 1;
>  
> -	dw_pcie_writel_dbi(pci, PCIE_ATU_VIEWPORT,
> -			   PCIE_ATU_REGION_DIR_OB | index);
> -	dw_pcie_writel_dbi(pci, PCIE_ATU_LOWER_BASE,
> -			   lower_32_bits(cpu_addr));
> -	dw_pcie_writel_dbi(pci, PCIE_ATU_UPPER_BASE,
> -			   upper_32_bits(cpu_addr));
> -	dw_pcie_writel_dbi(pci, PCIE_ATU_LIMIT,
> -			   lower_32_bits(limit_addr));
> +	dw_pcie_writel_atu_ob(pci, index, PCIE_ATU_LOWER_BASE,
> +			      lower_32_bits(cpu_addr));
> +	dw_pcie_writel_atu_ob(pci, index, PCIE_ATU_UPPER_BASE,
> +			      upper_32_bits(cpu_addr));
> +
> +	dw_pcie_writel_atu_ob(pci, index, PCIE_ATU_LIMIT,
> +			      lower_32_bits(limit_addr));
>  	if (dw_pcie_ver_is_ge(pci, 460A))
> -		dw_pcie_writel_dbi(pci, PCIE_ATU_UPPER_LIMIT,
> -				   upper_32_bits(limit_addr));
> -	dw_pcie_writel_dbi(pci, PCIE_ATU_LOWER_TARGET,
> -			   lower_32_bits(pci_addr));
> -	dw_pcie_writel_dbi(pci, PCIE_ATU_UPPER_TARGET,
> -			   upper_32_bits(pci_addr));
> +		dw_pcie_writel_atu_ob(pci, index, PCIE_ATU_UPPER_LIMIT,
> +				      upper_32_bits(limit_addr));
> +
> +	dw_pcie_writel_atu_ob(pci, index, PCIE_ATU_LOWER_TARGET,
> +			      lower_32_bits(pci_addr));
> +	dw_pcie_writel_atu_ob(pci, index, PCIE_ATU_UPPER_TARGET,
> +			      upper_32_bits(pci_addr));
> +
>  	val = type | PCIE_ATU_FUNC_NUM(func_no);
>  	if (upper_32_bits(limit_addr) > upper_32_bits(cpu_addr) &&
>  	    dw_pcie_ver_is_ge(pci, 460A))
>  		val |= PCIE_ATU_INCREASE_REGION_SIZE;
>  	if (dw_pcie_ver_is(pci, 490A))
>  		val = dw_pcie_enable_ecrc(val);
> -	dw_pcie_writel_dbi(pci, PCIE_ATU_CR1, val);
> -	dw_pcie_writel_dbi(pci, PCIE_ATU_CR2, PCIE_ATU_ENABLE);
> +	dw_pcie_writel_atu_ob(pci, index, PCIE_ATU_REGION_CTRL1, val);
> +
> +	dw_pcie_writel_atu_ob(pci, index, PCIE_ATU_REGION_CTRL2, PCIE_ATU_ENABLE);
>  
>  	/*
>  	 * Make sure ATU enable takes effect before any subsequent config
>  	 * and I/O accesses.
>  	 */
>  	for (retries = 0; retries < LINK_WAIT_MAX_IATU_RETRIES; retries++) {
> -		val = dw_pcie_readl_dbi(pci, PCIE_ATU_CR2);
> +		val = dw_pcie_readl_atu_ob(pci, index, PCIE_ATU_REGION_CTRL2);
>  		if (val & PCIE_ATU_ENABLE)
>  			return;
>  
>  		mdelay(LINK_WAIT_IATU);
>  	}
> +
>  	dev_err(pci->dev, "Outbound iATU is not being enabled\n");
>  }
>  
> @@ -405,54 +374,15 @@ void dw_pcie_prog_ep_outbound_atu(struct dw_pcie *pci, u8 func_no, int index,
>  				    cpu_addr, pci_addr, size);
>  }
>  
> -static u32 dw_pcie_readl_ib_unroll(struct dw_pcie *pci, u32 index, u32 reg)
> -{
> -	u32 offset = PCIE_GET_ATU_INB_UNR_REG_OFFSET(index);
> -
> -	return dw_pcie_readl_atu(pci, offset + reg);
> -}
> -
> -static void dw_pcie_writel_ib_unroll(struct dw_pcie *pci, u32 index, u32 reg,
> -				     u32 val)
> +static inline u32 dw_pcie_readl_atu_ib(struct dw_pcie *pci, u32 index, u32 reg)
>  {
> -	u32 offset = PCIE_GET_ATU_INB_UNR_REG_OFFSET(index);
> -
> -	dw_pcie_writel_atu(pci, offset + reg, val);
> +	return dw_pcie_readl_atu(pci, PCIE_ATU_REGION_DIR_IB, index, reg);
>  }
>  
> -static int dw_pcie_prog_inbound_atu_unroll(struct dw_pcie *pci, u8 func_no,
> -					   int index, int type,
> -					   u64 cpu_addr, u8 bar)
> +static inline void dw_pcie_writel_atu_ib(struct dw_pcie *pci, u32 index, u32 reg,
> +					 u32 val)
>  {
> -	u32 retries, val;
> -
> -	dw_pcie_writel_ib_unroll(pci, index, PCIE_ATU_UNR_LOWER_TARGET,
> -				 lower_32_bits(cpu_addr));
> -	dw_pcie_writel_ib_unroll(pci, index, PCIE_ATU_UNR_UPPER_TARGET,
> -				 upper_32_bits(cpu_addr));
> -
> -	dw_pcie_writel_ib_unroll(pci, index, PCIE_ATU_UNR_REGION_CTRL1, type |
> -				 PCIE_ATU_FUNC_NUM(func_no));
> -	dw_pcie_writel_ib_unroll(pci, index, PCIE_ATU_UNR_REGION_CTRL2,
> -				 PCIE_ATU_FUNC_NUM_MATCH_EN |
> -				 PCIE_ATU_ENABLE |
> -				 PCIE_ATU_BAR_MODE_ENABLE | (bar << 8));
> -
> -	/*
> -	 * Make sure ATU enable takes effect before any subsequent config
> -	 * and I/O accesses.
> -	 */
> -	for (retries = 0; retries < LINK_WAIT_MAX_IATU_RETRIES; retries++) {
> -		val = dw_pcie_readl_ib_unroll(pci, index,
> -					      PCIE_ATU_UNR_REGION_CTRL2);
> -		if (val & PCIE_ATU_ENABLE)
> -			return 0;
> -
> -		mdelay(LINK_WAIT_IATU);
> -	}
> -	dev_err(pci->dev, "Inbound iATU is not being enabled\n");
> -
> -	return -EBUSY;
> +	dw_pcie_writel_atu(pci, PCIE_ATU_REGION_DIR_IB, index, reg, val);
>  }
>  
>  int dw_pcie_prog_inbound_atu(struct dw_pcie *pci, u8 func_no, int index,
> @@ -460,51 +390,37 @@ int dw_pcie_prog_inbound_atu(struct dw_pcie *pci, u8 func_no, int index,
>  {
>  	u32 retries, val;
>  
> -	if (pci->iatu_unroll_enabled)
> -		return dw_pcie_prog_inbound_atu_unroll(pci, func_no, index, type,
> -						       cpu_addr, bar);
> +	dw_pcie_writel_atu_ib(pci, index, PCIE_ATU_LOWER_TARGET,
> +			      lower_32_bits(cpu_addr));
> +	dw_pcie_writel_atu_ib(pci, index, PCIE_ATU_UPPER_TARGET,
> +			      upper_32_bits(cpu_addr));
>  
> -	dw_pcie_writel_dbi(pci, PCIE_ATU_VIEWPORT, PCIE_ATU_REGION_DIR_IB |
> -			   index);
> -	dw_pcie_writel_dbi(pci, PCIE_ATU_LOWER_TARGET, lower_32_bits(cpu_addr));
> -	dw_pcie_writel_dbi(pci, PCIE_ATU_UPPER_TARGET, upper_32_bits(cpu_addr));
> -
> -	dw_pcie_writel_dbi(pci, PCIE_ATU_CR1, type |
> -			   PCIE_ATU_FUNC_NUM(func_no));
> -	dw_pcie_writel_dbi(pci, PCIE_ATU_CR2, PCIE_ATU_ENABLE |
> -			   PCIE_ATU_FUNC_NUM_MATCH_EN |
> -			   PCIE_ATU_BAR_MODE_ENABLE | (bar << 8));
> +	dw_pcie_writel_atu_ib(pci, index, PCIE_ATU_REGION_CTRL1, type |
> +			      PCIE_ATU_FUNC_NUM(func_no));
> +	dw_pcie_writel_atu_ib(pci, index, PCIE_ATU_REGION_CTRL2,
> +			      PCIE_ATU_ENABLE | PCIE_ATU_FUNC_NUM_MATCH_EN |
> +			      PCIE_ATU_BAR_MODE_ENABLE | (bar << 8));
>  
>  	/*
>  	 * Make sure ATU enable takes effect before any subsequent config
>  	 * and I/O accesses.
>  	 */
>  	for (retries = 0; retries < LINK_WAIT_MAX_IATU_RETRIES; retries++) {
> -		val = dw_pcie_readl_dbi(pci, PCIE_ATU_CR2);
> +		val = dw_pcie_readl_atu_ib(pci, index, PCIE_ATU_REGION_CTRL2);
>  		if (val & PCIE_ATU_ENABLE)
>  			return 0;
>  
>  		mdelay(LINK_WAIT_IATU);
>  	}
> +
>  	dev_err(pci->dev, "Inbound iATU is not being enabled\n");
>  
> -	return -EBUSY;
> +	return -ETIMEDOUT;
>  }
>  
>  void dw_pcie_disable_atu(struct dw_pcie *pci, u32 dir, int index)
>  {
> -	if (pci->iatu_unroll_enabled) {
> -		if (dir == PCIE_ATU_REGION_DIR_IB) {
> -			dw_pcie_writel_ib_unroll(pci, index, PCIE_ATU_UNR_REGION_CTRL2,
> -						 ~(u32)PCIE_ATU_ENABLE);
> -		} else {
> -			dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_REGION_CTRL2,
> -						 ~(u32)PCIE_ATU_ENABLE);
> -		}
> -	} else {
> -		dw_pcie_writel_dbi(pci, PCIE_ATU_VIEWPORT, dir | index);
> -		dw_pcie_writel_dbi(pci, PCIE_ATU_CR2, ~(u32)PCIE_ATU_ENABLE);
> -	}
> +	dw_pcie_writel_atu(pci, dir, index, PCIE_ATU_REGION_CTRL2, 0);
>  }
>  
>  int dw_pcie_wait_for_link(struct dw_pcie *pci)
> @@ -606,63 +522,29 @@ static bool dw_pcie_iatu_unroll_enabled(struct dw_pcie *pci)
>  	return false;
>  }
>  
> -static void dw_pcie_iatu_detect_regions_unroll(struct dw_pcie *pci)
> -{
> -	int max_region, i, ob = 0, ib = 0;
> -	u32 val;
> -
> -	max_region = min((int)pci->atu_size / 512, 256);
> -
> -	for (i = 0; i < max_region; i++) {
> -		dw_pcie_writel_ob_unroll(pci, i, PCIE_ATU_UNR_LOWER_TARGET,
> -					0x11110000);
> -
> -		val = dw_pcie_readl_ob_unroll(pci, i, PCIE_ATU_UNR_LOWER_TARGET);
> -		if (val == 0x11110000)
> -			ob++;
> -		else
> -			break;
> -	}
> -
> -	for (i = 0; i < max_region; i++) {
> -		dw_pcie_writel_ib_unroll(pci, i, PCIE_ATU_UNR_LOWER_TARGET,
> -					0x11110000);
> -
> -		val = dw_pcie_readl_ib_unroll(pci, i, PCIE_ATU_UNR_LOWER_TARGET);
> -		if (val == 0x11110000)
> -			ib++;
> -		else
> -			break;
> -	}
> -	pci->num_ib_windows = ib;
> -	pci->num_ob_windows = ob;
> -}
> -
>  static void dw_pcie_iatu_detect_regions(struct dw_pcie *pci)
>  {
> -	int max_region, i, ob = 0, ib = 0;
> +	int max_region, ob, ib;
>  	u32 val;
>  
> -	dw_pcie_writel_dbi(pci, PCIE_ATU_VIEWPORT, 0xFF);
> -	max_region = dw_pcie_readl_dbi(pci, PCIE_ATU_VIEWPORT) + 1;
> +	if (pci->iatu_unroll_enabled) {
> +		max_region = min((int)pci->atu_size / 512, 256);
> +	} else {
> +		dw_pcie_writel_dbi(pci, PCIE_ATU_VIEWPORT, 0xFF);
> +		max_region = dw_pcie_readl_dbi(pci, PCIE_ATU_VIEWPORT) + 1;
> +	}
>  
> -	for (i = 0; i < max_region; i++) {
> -		dw_pcie_writel_dbi(pci, PCIE_ATU_VIEWPORT, PCIE_ATU_REGION_DIR_OB | i);
> -		dw_pcie_writel_dbi(pci, PCIE_ATU_LOWER_TARGET, 0x11110000);
> -		val = dw_pcie_readl_dbi(pci, PCIE_ATU_LOWER_TARGET);
> -		if (val == 0x11110000)
> -			ob++;
> -		else
> +	for (ob = 0; ob < max_region; ob++) {
> +		dw_pcie_writel_atu_ob(pci, ob, PCIE_ATU_LOWER_TARGET, 0x11110000);
> +		val = dw_pcie_readl_atu_ob(pci, ob, PCIE_ATU_LOWER_TARGET);
> +		if (val != 0x11110000)
>  			break;
>  	}
>  
> -	for (i = 0; i < max_region; i++) {
> -		dw_pcie_writel_dbi(pci, PCIE_ATU_VIEWPORT, PCIE_ATU_REGION_DIR_IB | i);
> -		dw_pcie_writel_dbi(pci, PCIE_ATU_LOWER_TARGET, 0x11110000);
> -		val = dw_pcie_readl_dbi(pci, PCIE_ATU_LOWER_TARGET);
> -		if (val == 0x11110000)
> -			ib++;
> -		else
> +	for (ib = 0; ib < max_region; ib++) {
> +		dw_pcie_writel_atu_ib(pci, ib, PCIE_ATU_LOWER_TARGET, 0x11110000);
> +		val = dw_pcie_readl_atu_ib(pci, ib, PCIE_ATU_LOWER_TARGET);
> +		if (val != 0x11110000)
>  			break;
>  	}
>  
> @@ -690,12 +572,13 @@ void dw_pcie_iatu_detect(struct dw_pcie *pci)
>  		if (!pci->atu_size)
>  			/* Pick a minimal default, enough for 8 in and 8 out windows */
>  			pci->atu_size = SZ_4K;
> -
> -		dw_pcie_iatu_detect_regions_unroll(pci);
>  	} else {
> -		dw_pcie_iatu_detect_regions(pci);
> +		pci->atu_base = pci->dbi_base + PCIE_ATU_VIEWPORT_BASE;
> +		pci->atu_size = PCIE_ATU_VIEWPORT_SIZE;
>  	}
>  
> +	dw_pcie_iatu_detect_regions(pci);
> +
>  	dev_info(pci->dev, "iATU unroll: %s\n", pci->iatu_unroll_enabled ?
>  		"enabled" : "disabled");
>  
> diff --git a/drivers/pci/controller/dwc/pcie-designware.h b/drivers/pci/controller/dwc/pcie-designware.h
> index 72d185ff72f3..c18f0db09b31 100644
> --- a/drivers/pci/controller/dwc/pcie-designware.h
> +++ b/drivers/pci/controller/dwc/pcie-designware.h
> @@ -103,10 +103,21 @@
>  #define PCIE_VERSION_NUMBER		0x8F8
>  #define PCIE_VERSION_TYPE		0x8FC
>  
> +/*
> + * iATU inbound and outbound windows CSRs. Before the IP-core v4.80a each
> + * iATU region CSRs had been indirectly accessible by means of the dedicated
> + * viewport selector. The iATU/eDMA CSRs space was re-designed in DWC PCIe
> + * v4.80a in a way so the viewport was unrolled into the directly accessible
> + * iATU/eDMA CSRs space.
> + */
>  #define PCIE_ATU_VIEWPORT		0x900
>  #define PCIE_ATU_REGION_DIR_IB		BIT(31)
>  #define PCIE_ATU_REGION_DIR_OB		0
> -#define PCIE_ATU_CR1			0x904
> +#define PCIE_ATU_VIEWPORT_BASE		0x904
> +#define PCIE_ATU_UNROLL_BASE(dir, index) \
> +	(((index) << 9) | ((dir == PCIE_ATU_REGION_DIR_IB) ? BIT(8) : 0))
> +#define PCIE_ATU_VIEWPORT_SIZE		0x2C
> +#define PCIE_ATU_REGION_CTRL1		0x000
>  #define PCIE_ATU_INCREASE_REGION_SIZE	BIT(13)
>  #define PCIE_ATU_TYPE_MEM		0x0
>  #define PCIE_ATU_TYPE_IO		0x2
> @@ -114,19 +125,19 @@
>  #define PCIE_ATU_TYPE_CFG1		0x5
>  #define PCIE_ATU_TD			BIT(8)
>  #define PCIE_ATU_FUNC_NUM(pf)           ((pf) << 20)
> -#define PCIE_ATU_CR2			0x908
> +#define PCIE_ATU_REGION_CTRL2		0x004
>  #define PCIE_ATU_ENABLE			BIT(31)
>  #define PCIE_ATU_BAR_MODE_ENABLE	BIT(30)
>  #define PCIE_ATU_FUNC_NUM_MATCH_EN      BIT(19)
> -#define PCIE_ATU_LOWER_BASE		0x90C
> -#define PCIE_ATU_UPPER_BASE		0x910
> -#define PCIE_ATU_LIMIT			0x914
> -#define PCIE_ATU_LOWER_TARGET		0x918
> +#define PCIE_ATU_LOWER_BASE		0x008
> +#define PCIE_ATU_UPPER_BASE		0x00C
> +#define PCIE_ATU_LIMIT			0x010
> +#define PCIE_ATU_LOWER_TARGET		0x014
>  #define PCIE_ATU_BUS(x)			FIELD_PREP(GENMASK(31, 24), x)
>  #define PCIE_ATU_DEV(x)			FIELD_PREP(GENMASK(23, 19), x)
>  #define PCIE_ATU_FUNC(x)		FIELD_PREP(GENMASK(18, 16), x)
> -#define PCIE_ATU_UPPER_TARGET		0x91C
> -#define PCIE_ATU_UPPER_LIMIT		0x924
> +#define PCIE_ATU_UPPER_TARGET		0x018
> +#define PCIE_ATU_UPPER_LIMIT		0x020
>  
>  #define PCIE_MISC_CONTROL_1_OFF		0x8BC
>  #define PCIE_DBI_RO_WR_EN		BIT(0)
> @@ -143,19 +154,6 @@
>  
>  #define PCIE_PL_CHK_REG_ERR_ADDR			0xB28
>  
> -/*
> - * iATU Unroll-specific register definitions
> - * From 4.80 core version the address translation will be made by unroll
> - */
> -#define PCIE_ATU_UNR_REGION_CTRL1	0x00
> -#define PCIE_ATU_UNR_REGION_CTRL2	0x04
> -#define PCIE_ATU_UNR_LOWER_BASE		0x08
> -#define PCIE_ATU_UNR_UPPER_BASE		0x0C
> -#define PCIE_ATU_UNR_LOWER_LIMIT	0x10
> -#define PCIE_ATU_UNR_LOWER_TARGET	0x14
> -#define PCIE_ATU_UNR_UPPER_TARGET	0x18
> -#define PCIE_ATU_UNR_UPPER_LIMIT	0x20
> -
>  /*
>   * The default address offset between dbi_base and atu_base. Root controller
>   * drivers are not required to initialize atu_base if the offset matches this
> @@ -164,13 +162,6 @@
>   */
>  #define DEFAULT_DBI_ATU_OFFSET (0x3 << 20)
>  
> -/* Register address builder */
> -#define PCIE_GET_ATU_OUTB_UNR_REG_OFFSET(region) \
> -		((region) << 9)
> -
> -#define PCIE_GET_ATU_INB_UNR_REG_OFFSET(region) \
> -		(((region) << 9) | BIT(8))
> -
>  #define MAX_MSI_IRQS			256
>  #define MAX_MSI_IRQS_PER_CTRL		32
>  #define MAX_MSI_CTRLS			(MAX_MSI_IRQS / MAX_MSI_IRQS_PER_CTRL)
> @@ -277,7 +268,6 @@ struct dw_pcie {
>  	struct device		*dev;
>  	void __iomem		*dbi_base;
>  	void __iomem		*dbi_base2;
> -	/* Used when iatu_unroll_enabled is true */
>  	void __iomem		*atu_base;
>  	size_t			atu_size;
>  	u32			num_ib_windows;
> diff --git a/drivers/pci/controller/dwc/pcie-tegra194-acpi.c b/drivers/pci/controller/dwc/pcie-tegra194-acpi.c
> index c2de6ed4d86f..55f61914a986 100644
> --- a/drivers/pci/controller/dwc/pcie-tegra194-acpi.c
> +++ b/drivers/pci/controller/dwc/pcie-tegra194-acpi.c
> @@ -39,7 +39,8 @@ static int tegra194_acpi_init(struct pci_config_window *cfg)
>  static void atu_reg_write(struct tegra194_pcie_ecam *pcie_ecam, int index,
>  			  u32 val, u32 reg)
>  {
> -	u32 offset = PCIE_GET_ATU_OUTB_UNR_REG_OFFSET(index);
> +	u32 offset = PCIE_ATU_UNROLL_BASE(PCIE_ATU_REGION_DIR_OB, index) +
> +		     PCIE_ATU_VIEWPORT_BASE;
>  
>  	writel(val, pcie_ecam->iatu_base + offset + reg);
>  }
> @@ -58,8 +59,8 @@ static void program_outbound_atu(struct tegra194_pcie_ecam *pcie_ecam,
>  		      PCIE_ATU_LIMIT);
>  	atu_reg_write(pcie_ecam, index, upper_32_bits(pci_addr),
>  		      PCIE_ATU_UPPER_TARGET);
> -	atu_reg_write(pcie_ecam, index, type, PCIE_ATU_CR1);
> -	atu_reg_write(pcie_ecam, index, PCIE_ATU_ENABLE, PCIE_ATU_CR2);
> +	atu_reg_write(pcie_ecam, index, type, PCIE_ATU_REGION_CTRL1);
> +	atu_reg_write(pcie_ecam, index, PCIE_ATU_ENABLE, PCIE_ATU_REGION_CTRL2);
>  }
>  
>  static void __iomem *tegra194_map_bus(struct pci_bus *bus,
> -- 
> 2.35.1
> 

-- 
மணிவண்ணன் சதாசிவம்

Re: [PATCH RESEND v4 11/15] PCI: dwc: Simplify in/outbound iATU setup methods

[Bjorn Helgaas]

On Mon, Aug 01, 2022 at 07:20:57PM +0530, Manivannan Sadhasivam wrote:
> On Fri, Jun 24, 2022 at 05:39:43PM +0300, Serge Semin wrote:
> > From maintainability and scalability points of view it has been wrong to
> > use different iATU inbound and outbound regions accessors for the viewport
> > and unrolled versions of the iATU CSRs mapping. Seeing the particular iATU
> > region-wise registers layout is almost fully compatible for different
> > IP-core versions, there were no much points in splitting the code up that
> > way since it was possible to implement a common windows setup methods for
> > both viewport and unrolled iATU CSRs spaces. While what we can observe in
> > the current driver implementation of these methods, is a lot of code
> > duplication, which consequently worsen the code readability,
> > maintainability and scalability. Note the current implementation is a bit
> > more performant than the one suggested in this commit since it implies
> > having less MMIO accesses. But the gain just doesn't worth having the
> > denoted difficulties especially seeing the iATU setup methods are mainly
> > called on the DW PCIe controller and peripheral devices initialization
> > stage.
> > 
> > Here we suggest to move the iATU viewport and unrolled CSR access
> > specifics in the dw_pcie_readl_atu() and dw_pcie_writel_atu() method, and
> > convert the dw_pcie_prog_outbound_atu() and
> > dw_pcie_prog_{ep_}inbound_atu() functions to being generic instead of
> > having a different methods for each viewport and unrolled types of iATU
> > CSRs mapping. Nothing complex really. First of all the dw_pcie_readl_atu()
> > and dw_pcie_writel_atu() are converted to accept relative iATU CSRs
> > address together with the iATU region direction (inbound or outbound) and
> > region index. If DW PCIe controller doesn't have the unrolled iATU CSRs
> > space, then the accessors will need to activate a iATU viewport based on
> > the specified direction and index, otherwise a base address for the
> > corresponding region CSRs will be calculated by means of the
> > PCIE_ATU_UNROLL_BASE() macro. The CSRs macro have been modified in
> > accordance with that logic in the pcie-designware.h header file.
> > 
> > The rest of the changes in this commit just concern converting the iATU
> > in-/out-bound setup methods and iATU regions detection procedure to be
> > compatible with the new accessors semantics. As a result we've dropped the
> > no more required dw_pcie_prog_outbound_atu_unroll(),
> > dw_pcie_prog_inbound_atu_unroll() and dw_pcie_iatu_detect_regions_unroll()
> > methods.
> > 
> > Note aside with the denoted code improvements, there is an additional
> > positive side effect of this change. If at some point an atomic iATU
> > configs setup procedure is required, it will be possible to be done with
> > no much effort just by adding the synchronization into the
> > dw_pcie_readl_atu() and dw_pcie_writel_atu() accessors.
> > 
> > Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>
> 
> One nitpick mentioned below. With that fixed,
> 
> Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>

> > +static inline void __iomem *dw_pcie_select_atu(struct dw_pcie *pci, u32 dir,
> 
> This could be renamed to "dw_pcie_get_atu_base()" since we are anyway getting
> the base address of iATU.

I can see it both ways.  It definitely returns a base address, so
"get_atu_base" makes sense.  But it also writes PCIE_ATU_VIEWPORT, and
"select_atu" hints at that side effect while "get_atu_base" does not.

> > +					       u32 index)
> >  {
> > +	void __iomem *base = pci->atu_base;
> > +
> > +	if (pci->iatu_unroll_enabled)
> > +		base += PCIE_ATU_UNROLL_BASE(dir, index);
> > +	else
> > +		dw_pcie_writel_dbi(pci, PCIE_ATU_VIEWPORT, dir | index);
> > +
> > +	return base;
> > +}

[PATCH RESEND v4 12/15] PCI: dwc: Add iATU regions size detection procedure

[Serge Semin]

Depending on the DWC PCIe RC/EP/DM IP-core configuration parameters the
controllers can be equipped not only with various number of inbound and
outbound iATU windows, but with varied regions settings like alignment
(which is also the minimum window size), minimum and maximum sizes. So to
speak if internal ATU is enabled for the denoted IP-cores then the former
settings will be defined by the CX_ATU_MIN_REGION_SIZE parameter while the
later one will be determined by the CX_ATU_MAX_REGION_SIZE configuration
parameter. Anyway having these parameters used in the driver will help to
verify whether the requested inbound or outbound memory mappings can be
fully created. Currently the driver doesn't perform any corresponding
checking.

Note 1. The extended iATU regions have been supported since DWC PCIe
v4.60a. There is no need in testing the upper limit register availability
for the older cores.

Note 2. The regions alignment is determined with using the fls() method
since the lower four bits of the ATU Limit register can be occupied with
the Circular Buffer Increment setting, which can be initialized with
zeros.

The (dma-)ranges verification will be added a bit later in one of the next
commits.

Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>
Reviewed-by: Rob Herring <robh@kernel.org>
---
 drivers/pci/controller/dwc/pcie-designware.c | 33 +++++++++++++++++---
 drivers/pci/controller/dwc/pcie-designware.h |  2 ++
 2 files changed, 31 insertions(+), 4 deletions(-)

diff --git a/drivers/pci/controller/dwc/pcie-designware.c b/drivers/pci/controller/dwc/pcie-designware.c
index f2aa65d02a6c..776752891d11 100644
--- a/drivers/pci/controller/dwc/pcie-designware.c
+++ b/drivers/pci/controller/dwc/pcie-designware.c
@@ -8,9 +8,11 @@
  * Author: Jingoo Han <jg1.han@samsung.com>
  */
 
+#include <linux/bitops.h>
 #include <linux/delay.h>
 #include <linux/of.h>
 #include <linux/of_platform.h>
+#include <linux/sizes.h>
 #include <linux/types.h>
 
 #include "../../pci.h"
@@ -525,7 +527,8 @@ static bool dw_pcie_iatu_unroll_enabled(struct dw_pcie *pci)
 static void dw_pcie_iatu_detect_regions(struct dw_pcie *pci)
 {
 	int max_region, ob, ib;
-	u32 val;
+	u32 val, min, dir;
+	u64 max;
 
 	if (pci->iatu_unroll_enabled) {
 		max_region = min((int)pci->atu_size / 512, 256);
@@ -548,8 +551,29 @@ static void dw_pcie_iatu_detect_regions(struct dw_pcie *pci)
 			break;
 	}
 
-	pci->num_ib_windows = ib;
+	if (ob) {
+		dir = PCIE_ATU_REGION_DIR_OB;
+	} else if (ib) {
+		dir = PCIE_ATU_REGION_DIR_IB;
+	} else {
+		dev_err(pci->dev, "No iATU regions found\n");
+		return;
+	}
+
+	dw_pcie_writel_atu(pci, dir, 0, PCIE_ATU_LIMIT, 0x0);
+	min = dw_pcie_readl_atu(pci, dir, 0, PCIE_ATU_LIMIT);
+
+	if (dw_pcie_ver_is_ge(pci, 460A)) {
+		dw_pcie_writel_atu(pci, dir, 0, PCIE_ATU_UPPER_LIMIT, 0xFFFFFFFF);
+		max = dw_pcie_readl_atu(pci, dir, 0, PCIE_ATU_UPPER_LIMIT);
+	} else {
+		max = 0;
+	}
+
 	pci->num_ob_windows = ob;
+	pci->num_ib_windows = ib;
+	pci->region_align = 1 << fls(min);
+	pci->region_limit = (max << 32) | (SZ_4G - 1);
 }
 
 void dw_pcie_iatu_detect(struct dw_pcie *pci)
@@ -582,8 +606,9 @@ void dw_pcie_iatu_detect(struct dw_pcie *pci)
 	dev_info(pci->dev, "iATU unroll: %s\n", pci->iatu_unroll_enabled ?
 		"enabled" : "disabled");
 
-	dev_info(pci->dev, "Detected iATU regions: %u outbound, %u inbound\n",
-		 pci->num_ob_windows, pci->num_ib_windows);
+	dev_info(pci->dev, "iATU regions: %u ob, %u ib, align %uK, limit %lluG\n",
+		 pci->num_ob_windows, pci->num_ib_windows,
+		 pci->region_align / SZ_1K, (pci->region_limit + 1) / SZ_1G);
 }
 
 void dw_pcie_setup(struct dw_pcie *pci)
diff --git a/drivers/pci/controller/dwc/pcie-designware.h b/drivers/pci/controller/dwc/pcie-designware.h
index c18f0db09b31..25c86771c810 100644
--- a/drivers/pci/controller/dwc/pcie-designware.h
+++ b/drivers/pci/controller/dwc/pcie-designware.h
@@ -272,6 +272,8 @@ struct dw_pcie {
 	size_t			atu_size;
 	u32			num_ib_windows;
 	u32			num_ob_windows;
+	u32			region_align;
+	u64			region_limit;
 	struct dw_pcie_rp	pp;
 	struct dw_pcie_ep	ep;
 	const struct dw_pcie_ops *ops;
-- 
2.35.1

Re: [PATCH RESEND v4 12/15] PCI: dwc: Add iATU regions size detection procedure

[Manivannan Sadhasivam]

On Fri, Jun 24, 2022 at 05:39:44PM +0300, Serge Semin wrote:
> Depending on the DWC PCIe RC/EP/DM IP-core configuration parameters the
> controllers can be equipped not only with various number of inbound and
> outbound iATU windows, but with varied regions settings like alignment
> (which is also the minimum window size), minimum and maximum sizes. So to
> speak if internal ATU is enabled for the denoted IP-cores then the former
> settings will be defined by the CX_ATU_MIN_REGION_SIZE parameter while the
> later one will be determined by the CX_ATU_MAX_REGION_SIZE configuration
> parameter. Anyway having these parameters used in the driver will help to
> verify whether the requested inbound or outbound memory mappings can be
> fully created. Currently the driver doesn't perform any corresponding
> checking.
> 
> Note 1. The extended iATU regions have been supported since DWC PCIe
> v4.60a. There is no need in testing the upper limit register availability
> for the older cores.
> 
> Note 2. The regions alignment is determined with using the fls() method
> since the lower four bits of the ATU Limit register can be occupied with
> the Circular Buffer Increment setting, which can be initialized with
> zeros.
> 
> The (dma-)ranges verification will be added a bit later in one of the next
> commits.
> 
> Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>

Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>

Thanks,
Mani

> Reviewed-by: Rob Herring <robh@kernel.org>
> ---
>  drivers/pci/controller/dwc/pcie-designware.c | 33 +++++++++++++++++---
>  drivers/pci/controller/dwc/pcie-designware.h |  2 ++
>  2 files changed, 31 insertions(+), 4 deletions(-)
> 
> diff --git a/drivers/pci/controller/dwc/pcie-designware.c b/drivers/pci/controller/dwc/pcie-designware.c
> index f2aa65d02a6c..776752891d11 100644
> --- a/drivers/pci/controller/dwc/pcie-designware.c
> +++ b/drivers/pci/controller/dwc/pcie-designware.c
> @@ -8,9 +8,11 @@
>   * Author: Jingoo Han <jg1.han@samsung.com>
>   */
>  
> +#include <linux/bitops.h>
>  #include <linux/delay.h>
>  #include <linux/of.h>
>  #include <linux/of_platform.h>
> +#include <linux/sizes.h>
>  #include <linux/types.h>
>  
>  #include "../../pci.h"
> @@ -525,7 +527,8 @@ static bool dw_pcie_iatu_unroll_enabled(struct dw_pcie *pci)
>  static void dw_pcie_iatu_detect_regions(struct dw_pcie *pci)
>  {
>  	int max_region, ob, ib;
> -	u32 val;
> +	u32 val, min, dir;
> +	u64 max;
>  
>  	if (pci->iatu_unroll_enabled) {
>  		max_region = min((int)pci->atu_size / 512, 256);
> @@ -548,8 +551,29 @@ static void dw_pcie_iatu_detect_regions(struct dw_pcie *pci)
>  			break;
>  	}
>  
> -	pci->num_ib_windows = ib;
> +	if (ob) {
> +		dir = PCIE_ATU_REGION_DIR_OB;
> +	} else if (ib) {
> +		dir = PCIE_ATU_REGION_DIR_IB;
> +	} else {
> +		dev_err(pci->dev, "No iATU regions found\n");
> +		return;
> +	}
> +
> +	dw_pcie_writel_atu(pci, dir, 0, PCIE_ATU_LIMIT, 0x0);
> +	min = dw_pcie_readl_atu(pci, dir, 0, PCIE_ATU_LIMIT);
> +
> +	if (dw_pcie_ver_is_ge(pci, 460A)) {
> +		dw_pcie_writel_atu(pci, dir, 0, PCIE_ATU_UPPER_LIMIT, 0xFFFFFFFF);
> +		max = dw_pcie_readl_atu(pci, dir, 0, PCIE_ATU_UPPER_LIMIT);
> +	} else {
> +		max = 0;
> +	}
> +
>  	pci->num_ob_windows = ob;
> +	pci->num_ib_windows = ib;
> +	pci->region_align = 1 << fls(min);
> +	pci->region_limit = (max << 32) | (SZ_4G - 1);
>  }
>  
>  void dw_pcie_iatu_detect(struct dw_pcie *pci)
> @@ -582,8 +606,9 @@ void dw_pcie_iatu_detect(struct dw_pcie *pci)
>  	dev_info(pci->dev, "iATU unroll: %s\n", pci->iatu_unroll_enabled ?
>  		"enabled" : "disabled");
>  
> -	dev_info(pci->dev, "Detected iATU regions: %u outbound, %u inbound\n",
> -		 pci->num_ob_windows, pci->num_ib_windows);
> +	dev_info(pci->dev, "iATU regions: %u ob, %u ib, align %uK, limit %lluG\n",
> +		 pci->num_ob_windows, pci->num_ib_windows,
> +		 pci->region_align / SZ_1K, (pci->region_limit + 1) / SZ_1G);
>  }
>  
>  void dw_pcie_setup(struct dw_pcie *pci)
> diff --git a/drivers/pci/controller/dwc/pcie-designware.h b/drivers/pci/controller/dwc/pcie-designware.h
> index c18f0db09b31..25c86771c810 100644
> --- a/drivers/pci/controller/dwc/pcie-designware.h
> +++ b/drivers/pci/controller/dwc/pcie-designware.h
> @@ -272,6 +272,8 @@ struct dw_pcie {
>  	size_t			atu_size;
>  	u32			num_ib_windows;
>  	u32			num_ob_windows;
> +	u32			region_align;
> +	u64			region_limit;
>  	struct dw_pcie_rp	pp;
>  	struct dw_pcie_ep	ep;
>  	const struct dw_pcie_ops *ops;
> -- 
> 2.35.1
> 

-- 
மணிவண்ணன் சதாசிவம்

[PATCH RESEND v4 13/15] PCI: dwc: Verify in/out regions against iATU constraints

[Serge Semin]

Since the DWC PCIe driver private data now contains the iATU inbound and
outbound regions constraints info like alignment, minimum and maximum
limits, we can use them to make the in- and outbound iATU regions setup
methods more strict to the ranges a callee tries to specify.  That will
give us the safer dw_pcie_prog_outbound_atu(),
dw_pcie_prog_ep_outbound_atu() and dw_pcie_prog_inbound_atu() functions.

First of all let's update the outbound ATU entries setup methods to
returning the operation status. The methods will fail either in case if
the range is failed to be activated or the passed region doesn't fulfill
iATU constraints. Secondly the passed to the
dw_pcie_prog_{ep_}outbound_atu() methods region-related parameters are
verified against the detected iATU regions constraints. In particular the
region limit address must not overflow the lower/upper limit CSR RW-fields
otherwise the specified range will be just silently clamped. That
verification will also protect the code from having u64 type overflow.
Secondly let's make sure base address (CPU-address), target address
(PCI-address) and size are properly aligned. Unaligned ranges will be
silently aligned down (addresses) and up (limit) on writing the values to
the corresponding registers, which in it turn may lead to unpredictable
results like ranges virtual overlap. Finally the CPU-address alignment
needs to be verified in the dw_pcie_prog_inbound_atu() method too as the
DWC PCIe RC/EP registers manual demands seeing the lower bits of the in-
and outbound iATU base address are always zeros.

Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>
Reviewed-by: Rob Herring <robh@kernel.org>

---

Changelog v3:
- Drop outbound iATU window size alignment constraint. (@Manivannan)
---
 drivers/pci/controller/dwc/pcie-designware.c | 38 +++++++++++++-------
 drivers/pci/controller/dwc/pcie-designware.h | 10 +++---
 2 files changed, 29 insertions(+), 19 deletions(-)

diff --git a/drivers/pci/controller/dwc/pcie-designware.c b/drivers/pci/controller/dwc/pcie-designware.c
index 776752891d11..9c622b635fdd 100644
--- a/drivers/pci/controller/dwc/pcie-designware.c
+++ b/drivers/pci/controller/dwc/pcie-designware.c
@@ -8,6 +8,7 @@
  * Author: Jingoo Han <jg1.han@samsung.com>
  */
 
+#include <linux/align.h>
 #include <linux/bitops.h>
 #include <linux/delay.h>
 #include <linux/of.h>
@@ -308,9 +309,9 @@ static inline u32 dw_pcie_enable_ecrc(u32 val)
 	return val | PCIE_ATU_TD;
 }
 
-static void __dw_pcie_prog_outbound_atu(struct dw_pcie *pci, u8 func_no,
-					int index, int type, u64 cpu_addr,
-					u64 pci_addr, u64 size)
+static int __dw_pcie_prog_outbound_atu(struct dw_pcie *pci, u8 func_no,
+				       int index, int type, u64 cpu_addr,
+				       u64 pci_addr, u64 size)
 {
 	u32 retries, val;
 	u64 limit_addr;
@@ -320,6 +321,12 @@ static void __dw_pcie_prog_outbound_atu(struct dw_pcie *pci, u8 func_no,
 
 	limit_addr = cpu_addr + size - 1;
 
+	if ((limit_addr & ~pci->region_limit) != (cpu_addr & ~pci->region_limit) ||
+	    !IS_ALIGNED(cpu_addr, pci->region_align) ||
+	    !IS_ALIGNED(pci_addr, pci->region_align) || !size) {
+		return -EINVAL;
+	}
+
 	dw_pcie_writel_atu_ob(pci, index, PCIE_ATU_LOWER_BASE,
 			      lower_32_bits(cpu_addr));
 	dw_pcie_writel_atu_ob(pci, index, PCIE_ATU_UPPER_BASE,
@@ -353,27 +360,29 @@ static void __dw_pcie_prog_outbound_atu(struct dw_pcie *pci, u8 func_no,
 	for (retries = 0; retries < LINK_WAIT_MAX_IATU_RETRIES; retries++) {
 		val = dw_pcie_readl_atu_ob(pci, index, PCIE_ATU_REGION_CTRL2);
 		if (val & PCIE_ATU_ENABLE)
-			return;
+			return 0;
 
 		mdelay(LINK_WAIT_IATU);
 	}
 
 	dev_err(pci->dev, "Outbound iATU is not being enabled\n");
+
+	return -ETIMEDOUT;
 }
 
-void dw_pcie_prog_outbound_atu(struct dw_pcie *pci, int index, int type,
-			       u64 cpu_addr, u64 pci_addr, u64 size)
+int dw_pcie_prog_outbound_atu(struct dw_pcie *pci, int index, int type,
+			      u64 cpu_addr, u64 pci_addr, u64 size)
 {
-	__dw_pcie_prog_outbound_atu(pci, 0, index, type,
-				    cpu_addr, pci_addr, size);
+	return __dw_pcie_prog_outbound_atu(pci, 0, index, type,
+					   cpu_addr, pci_addr, size);
 }
 
-void dw_pcie_prog_ep_outbound_atu(struct dw_pcie *pci, u8 func_no, int index,
-				  int type, u64 cpu_addr, u64 pci_addr,
-				  u64 size)
+int dw_pcie_prog_ep_outbound_atu(struct dw_pcie *pci, u8 func_no, int index,
+				 int type, u64 cpu_addr, u64 pci_addr,
+				 u64 size)
 {
-	__dw_pcie_prog_outbound_atu(pci, func_no, index, type,
-				    cpu_addr, pci_addr, size);
+	return __dw_pcie_prog_outbound_atu(pci, func_no, index, type,
+					   cpu_addr, pci_addr, size);
 }
 
 static inline u32 dw_pcie_readl_atu_ib(struct dw_pcie *pci, u32 index, u32 reg)
@@ -392,6 +401,9 @@ int dw_pcie_prog_inbound_atu(struct dw_pcie *pci, u8 func_no, int index,
 {
 	u32 retries, val;
 
+	if (!IS_ALIGNED(cpu_addr, pci->region_align))
+		return -EINVAL;
+
 	dw_pcie_writel_atu_ib(pci, index, PCIE_ATU_LOWER_TARGET,
 			      lower_32_bits(cpu_addr));
 	dw_pcie_writel_atu_ib(pci, index, PCIE_ATU_UPPER_TARGET,
diff --git a/drivers/pci/controller/dwc/pcie-designware.h b/drivers/pci/controller/dwc/pcie-designware.h
index 25c86771c810..60f1ddc54933 100644
--- a/drivers/pci/controller/dwc/pcie-designware.h
+++ b/drivers/pci/controller/dwc/pcie-designware.h
@@ -304,12 +304,10 @@ void dw_pcie_write_dbi2(struct dw_pcie *pci, u32 reg, size_t size, u32 val);
 int dw_pcie_link_up(struct dw_pcie *pci);
 void dw_pcie_upconfig_setup(struct dw_pcie *pci);
 int dw_pcie_wait_for_link(struct dw_pcie *pci);
-void dw_pcie_prog_outbound_atu(struct dw_pcie *pci, int index,
-			       int type, u64 cpu_addr, u64 pci_addr,
-			       u64 size);
-void dw_pcie_prog_ep_outbound_atu(struct dw_pcie *pci, u8 func_no, int index,
-				  int type, u64 cpu_addr, u64 pci_addr,
-				  u64 size);
+int dw_pcie_prog_outbound_atu(struct dw_pcie *pci, int index, int type,
+			      u64 cpu_addr, u64 pci_addr, u64 size);
+int dw_pcie_prog_ep_outbound_atu(struct dw_pcie *pci, u8 func_no, int index,
+				 int type, u64 cpu_addr, u64 pci_addr, u64 size);
 int dw_pcie_prog_inbound_atu(struct dw_pcie *pci, u8 func_no, int index,
 			     int type, u64 cpu_addr, u8 bar);
 void dw_pcie_disable_atu(struct dw_pcie *pci, u32 dir, int index);
-- 
2.35.1

Re: [PATCH RESEND v4 13/15] PCI: dwc: Verify in/out regions against iATU constraints

[Manivannan Sadhasivam]

On Fri, Jun 24, 2022 at 05:39:45PM +0300, Serge Semin wrote:
> Since the DWC PCIe driver private data now contains the iATU inbound and
> outbound regions constraints info like alignment, minimum and maximum
> limits, we can use them to make the in- and outbound iATU regions setup
> methods more strict to the ranges a callee tries to specify.  That will
> give us the safer dw_pcie_prog_outbound_atu(),
> dw_pcie_prog_ep_outbound_atu() and dw_pcie_prog_inbound_atu() functions.
> 
> First of all let's update the outbound ATU entries setup methods to
> returning the operation status. The methods will fail either in case if
> the range is failed to be activated or the passed region doesn't fulfill
> iATU constraints. Secondly the passed to the
> dw_pcie_prog_{ep_}outbound_atu() methods region-related parameters are
> verified against the detected iATU regions constraints. In particular the
> region limit address must not overflow the lower/upper limit CSR RW-fields
> otherwise the specified range will be just silently clamped. That
> verification will also protect the code from having u64 type overflow.
> Secondly let's make sure base address (CPU-address), target address
> (PCI-address) and size are properly aligned. Unaligned ranges will be
> silently aligned down (addresses) and up (limit) on writing the values to
> the corresponding registers, which in it turn may lead to unpredictable
> results like ranges virtual overlap. Finally the CPU-address alignment
> needs to be verified in the dw_pcie_prog_inbound_atu() method too as the
> DWC PCIe RC/EP registers manual demands seeing the lower bits of the in-
> and outbound iATU base address are always zeros.
> 
> Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>

Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>

Thanks,
Mani

> Reviewed-by: Rob Herring <robh@kernel.org>
> 
> ---
> 
> Changelog v3:
> - Drop outbound iATU window size alignment constraint. (@Manivannan)
> ---
>  drivers/pci/controller/dwc/pcie-designware.c | 38 +++++++++++++-------
>  drivers/pci/controller/dwc/pcie-designware.h | 10 +++---
>  2 files changed, 29 insertions(+), 19 deletions(-)
> 
> diff --git a/drivers/pci/controller/dwc/pcie-designware.c b/drivers/pci/controller/dwc/pcie-designware.c
> index 776752891d11..9c622b635fdd 100644
> --- a/drivers/pci/controller/dwc/pcie-designware.c
> +++ b/drivers/pci/controller/dwc/pcie-designware.c
> @@ -8,6 +8,7 @@
>   * Author: Jingoo Han <jg1.han@samsung.com>
>   */
>  
> +#include <linux/align.h>
>  #include <linux/bitops.h>
>  #include <linux/delay.h>
>  #include <linux/of.h>
> @@ -308,9 +309,9 @@ static inline u32 dw_pcie_enable_ecrc(u32 val)
>  	return val | PCIE_ATU_TD;
>  }
>  
> -static void __dw_pcie_prog_outbound_atu(struct dw_pcie *pci, u8 func_no,
> -					int index, int type, u64 cpu_addr,
> -					u64 pci_addr, u64 size)
> +static int __dw_pcie_prog_outbound_atu(struct dw_pcie *pci, u8 func_no,
> +				       int index, int type, u64 cpu_addr,
> +				       u64 pci_addr, u64 size)
>  {
>  	u32 retries, val;
>  	u64 limit_addr;
> @@ -320,6 +321,12 @@ static void __dw_pcie_prog_outbound_atu(struct dw_pcie *pci, u8 func_no,
>  
>  	limit_addr = cpu_addr + size - 1;
>  
> +	if ((limit_addr & ~pci->region_limit) != (cpu_addr & ~pci->region_limit) ||
> +	    !IS_ALIGNED(cpu_addr, pci->region_align) ||
> +	    !IS_ALIGNED(pci_addr, pci->region_align) || !size) {
> +		return -EINVAL;
> +	}
> +
>  	dw_pcie_writel_atu_ob(pci, index, PCIE_ATU_LOWER_BASE,
>  			      lower_32_bits(cpu_addr));
>  	dw_pcie_writel_atu_ob(pci, index, PCIE_ATU_UPPER_BASE,
> @@ -353,27 +360,29 @@ static void __dw_pcie_prog_outbound_atu(struct dw_pcie *pci, u8 func_no,
>  	for (retries = 0; retries < LINK_WAIT_MAX_IATU_RETRIES; retries++) {
>  		val = dw_pcie_readl_atu_ob(pci, index, PCIE_ATU_REGION_CTRL2);
>  		if (val & PCIE_ATU_ENABLE)
> -			return;
> +			return 0;
>  
>  		mdelay(LINK_WAIT_IATU);
>  	}
>  
>  	dev_err(pci->dev, "Outbound iATU is not being enabled\n");
> +
> +	return -ETIMEDOUT;
>  }
>  
> -void dw_pcie_prog_outbound_atu(struct dw_pcie *pci, int index, int type,
> -			       u64 cpu_addr, u64 pci_addr, u64 size)
> +int dw_pcie_prog_outbound_atu(struct dw_pcie *pci, int index, int type,
> +			      u64 cpu_addr, u64 pci_addr, u64 size)
>  {
> -	__dw_pcie_prog_outbound_atu(pci, 0, index, type,
> -				    cpu_addr, pci_addr, size);
> +	return __dw_pcie_prog_outbound_atu(pci, 0, index, type,
> +					   cpu_addr, pci_addr, size);
>  }
>  
> -void dw_pcie_prog_ep_outbound_atu(struct dw_pcie *pci, u8 func_no, int index,
> -				  int type, u64 cpu_addr, u64 pci_addr,
> -				  u64 size)
> +int dw_pcie_prog_ep_outbound_atu(struct dw_pcie *pci, u8 func_no, int index,
> +				 int type, u64 cpu_addr, u64 pci_addr,
> +				 u64 size)
>  {
> -	__dw_pcie_prog_outbound_atu(pci, func_no, index, type,
> -				    cpu_addr, pci_addr, size);
> +	return __dw_pcie_prog_outbound_atu(pci, func_no, index, type,
> +					   cpu_addr, pci_addr, size);
>  }
>  
>  static inline u32 dw_pcie_readl_atu_ib(struct dw_pcie *pci, u32 index, u32 reg)
> @@ -392,6 +401,9 @@ int dw_pcie_prog_inbound_atu(struct dw_pcie *pci, u8 func_no, int index,
>  {
>  	u32 retries, val;
>  
> +	if (!IS_ALIGNED(cpu_addr, pci->region_align))
> +		return -EINVAL;
> +
>  	dw_pcie_writel_atu_ib(pci, index, PCIE_ATU_LOWER_TARGET,
>  			      lower_32_bits(cpu_addr));
>  	dw_pcie_writel_atu_ib(pci, index, PCIE_ATU_UPPER_TARGET,
> diff --git a/drivers/pci/controller/dwc/pcie-designware.h b/drivers/pci/controller/dwc/pcie-designware.h
> index 25c86771c810..60f1ddc54933 100644
> --- a/drivers/pci/controller/dwc/pcie-designware.h
> +++ b/drivers/pci/controller/dwc/pcie-designware.h
> @@ -304,12 +304,10 @@ void dw_pcie_write_dbi2(struct dw_pcie *pci, u32 reg, size_t size, u32 val);
>  int dw_pcie_link_up(struct dw_pcie *pci);
>  void dw_pcie_upconfig_setup(struct dw_pcie *pci);
>  int dw_pcie_wait_for_link(struct dw_pcie *pci);
> -void dw_pcie_prog_outbound_atu(struct dw_pcie *pci, int index,
> -			       int type, u64 cpu_addr, u64 pci_addr,
> -			       u64 size);
> -void dw_pcie_prog_ep_outbound_atu(struct dw_pcie *pci, u8 func_no, int index,
> -				  int type, u64 cpu_addr, u64 pci_addr,
> -				  u64 size);
> +int dw_pcie_prog_outbound_atu(struct dw_pcie *pci, int index, int type,
> +			      u64 cpu_addr, u64 pci_addr, u64 size);
> +int dw_pcie_prog_ep_outbound_atu(struct dw_pcie *pci, u8 func_no, int index,
> +				 int type, u64 cpu_addr, u64 pci_addr, u64 size);
>  int dw_pcie_prog_inbound_atu(struct dw_pcie *pci, u8 func_no, int index,
>  			     int type, u64 cpu_addr, u8 bar);
>  void dw_pcie_disable_atu(struct dw_pcie *pci, u32 dir, int index);
> -- 
> 2.35.1
> 

-- 
மணிவண்ணன் சதாசிவம்

[PATCH RESEND v4 14/15] PCI: dwc: Check iATU in/outbound ranges setup methods status

[Serge Semin]

Let's make the DWC PCIe RC/EP safer and more verbose for the invalid or
failed inbound and outbound iATU windows setups. Needless to say that
silently ignoring iATU regions setup errors may cause unpredictable
errors. For instance if for some reason a cfg or IO window fails to be
activated, then any CFG/IO requested won't reach target PCIe devices and
the corresponding accessors will return platform-specific random values.

First of all we need to convert dw_pcie_ep_outbound_atu() method to check
whether the specified outbound iATU range is successfully setup. That
method is called by the pci_epc_ops.map_addr callback. Thus we'll make the
EP-specific CPU->PCIe memory mappings saver.

Secondly since the iATU outbound range programming method now returns the
operation status, it will be handy to take that status into account in the
pci_ops.{map_bus,read,write} methods. Thus any failed mapping will be
immediately noticeable by the PCIe CFG operations requesters.

Finally we need to convert the dw_pcie_setup_rc() method to returning the
operation status, since the iATU outbound ranges setup procedure may now
fail. It will be especially handy in case if the DW PCIe RC DT-node has
invalid/unsupported (dma-)ranges property. Note since the suggested
modification causes having too wide code indentation, it is reasonable
from maintainability and readability points of view to move the outbound
ranges setup procedure in the separate function.

Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>
Reviewed-by: Rob Herring <robh@kernel.org>
---
 .../pci/controller/dwc/pcie-designware-ep.c   |   9 +-
 .../pci/controller/dwc/pcie-designware-host.c | 153 ++++++++++++------
 drivers/pci/controller/dwc/pcie-designware.h  |   5 +-
 drivers/pci/controller/dwc/pcie-intel-gw.c    |   6 +-
 4 files changed, 114 insertions(+), 59 deletions(-)

diff --git a/drivers/pci/controller/dwc/pcie-designware-ep.c b/drivers/pci/controller/dwc/pcie-designware-ep.c
index 2e91222f7c98..627c4b69878c 100644
--- a/drivers/pci/controller/dwc/pcie-designware-ep.c
+++ b/drivers/pci/controller/dwc/pcie-designware-ep.c
@@ -184,8 +184,9 @@ static int dw_pcie_ep_outbound_atu(struct dw_pcie_ep *ep, u8 func_no,
 				   phys_addr_t phys_addr,
 				   u64 pci_addr, size_t size)
 {
-	u32 free_win;
 	struct dw_pcie *pci = to_dw_pcie_from_ep(ep);
+	u32 free_win;
+	int ret;
 
 	free_win = find_first_zero_bit(ep->ob_window_map, pci->num_ob_windows);
 	if (free_win >= pci->num_ob_windows) {
@@ -193,8 +194,10 @@ static int dw_pcie_ep_outbound_atu(struct dw_pcie_ep *ep, u8 func_no,
 		return -EINVAL;
 	}
 
-	dw_pcie_prog_ep_outbound_atu(pci, func_no, free_win, PCIE_ATU_TYPE_MEM,
-				     phys_addr, pci_addr, size);
+	ret = dw_pcie_prog_ep_outbound_atu(pci, func_no, free_win, PCIE_ATU_TYPE_MEM,
+					   phys_addr, pci_addr, size);
+	if (ret)
+		return ret;
 
 	set_bit(free_win, ep->ob_window_map);
 	ep->outbound_addr[free_win] = phys_addr;
diff --git a/drivers/pci/controller/dwc/pcie-designware-host.c b/drivers/pci/controller/dwc/pcie-designware-host.c
index e0a2819608c6..6993ce9e856d 100644
--- a/drivers/pci/controller/dwc/pcie-designware-host.c
+++ b/drivers/pci/controller/dwc/pcie-designware-host.c
@@ -412,7 +412,9 @@ int dw_pcie_host_init(struct dw_pcie_rp *pp)
 
 	dw_pcie_iatu_detect(pci);
 
-	dw_pcie_setup_rc(pp);
+	ret = dw_pcie_setup_rc(pp);
+	if (ret)
+		goto err_free_msi;
 
 	if (!dw_pcie_link_up(pci)) {
 		ret = dw_pcie_start_link(pci);
@@ -466,10 +468,10 @@ EXPORT_SYMBOL_GPL(dw_pcie_host_deinit);
 static void __iomem *dw_pcie_other_conf_map_bus(struct pci_bus *bus,
 						unsigned int devfn, int where)
 {
-	int type;
-	u32 busdev;
 	struct dw_pcie_rp *pp = bus->sysdata;
 	struct dw_pcie *pci = to_dw_pcie_from_pp(pp);
+	int type, ret;
+	u32 busdev;
 
 	/*
 	 * Checking whether the link is up here is a last line of defense
@@ -490,8 +492,10 @@ static void __iomem *dw_pcie_other_conf_map_bus(struct pci_bus *bus,
 	else
 		type = PCIE_ATU_TYPE_CFG1;
 
-
-	dw_pcie_prog_outbound_atu(pci, 0, type, pp->cfg0_base, busdev, pp->cfg0_size);
+	ret = dw_pcie_prog_outbound_atu(pci, 0, type, pp->cfg0_base, busdev,
+					pp->cfg0_size);
+	if (ret)
+		return NULL;
 
 	return pp->va_cfg0_base + where;
 }
@@ -499,33 +503,45 @@ static void __iomem *dw_pcie_other_conf_map_bus(struct pci_bus *bus,
 static int dw_pcie_rd_other_conf(struct pci_bus *bus, unsigned int devfn,
 				 int where, int size, u32 *val)
 {
-	int ret;
 	struct dw_pcie_rp *pp = bus->sysdata;
 	struct dw_pcie *pci = to_dw_pcie_from_pp(pp);
+	int ret;
 
 	ret = pci_generic_config_read(bus, devfn, where, size, val);
+	if (ret != PCIBIOS_SUCCESSFUL)
+		return ret;
 
-	if (!ret && pp->cfg0_io_shared)
-		dw_pcie_prog_outbound_atu(pci, 0, PCIE_ATU_TYPE_IO, pp->io_base,
-					  pp->io_bus_addr, pp->io_size);
+	if (pp->cfg0_io_shared) {
+		ret = dw_pcie_prog_outbound_atu(pci, 0, PCIE_ATU_TYPE_IO,
+						pp->io_base, pp->io_bus_addr,
+						pp->io_size);
+		if (ret)
+			return PCIBIOS_SET_FAILED;
+	}
 
-	return ret;
+	return PCIBIOS_SUCCESSFUL;
 }
 
 static int dw_pcie_wr_other_conf(struct pci_bus *bus, unsigned int devfn,
 				 int where, int size, u32 val)
 {
-	int ret;
 	struct dw_pcie_rp *pp = bus->sysdata;
 	struct dw_pcie *pci = to_dw_pcie_from_pp(pp);
+	int ret;
 
 	ret = pci_generic_config_write(bus, devfn, where, size, val);
+	if (ret != PCIBIOS_SUCCESSFUL)
+		return ret;
 
-	if (!ret && pp->cfg0_io_shared)
-		dw_pcie_prog_outbound_atu(pci, 0, PCIE_ATU_TYPE_IO, pp->io_base,
-					  pp->io_bus_addr, pp->io_size);
+	if (pp->cfg0_io_shared) {
+		ret = dw_pcie_prog_outbound_atu(pci, 0, PCIE_ATU_TYPE_IO,
+						pp->io_base, pp->io_bus_addr,
+						pp->io_size);
+		if (ret)
+			return PCIBIOS_SET_FAILED;
+	}
 
-	return ret;
+	return PCIBIOS_SUCCESSFUL;
 }
 
 static struct pci_ops dw_child_pcie_ops = {
@@ -552,10 +568,72 @@ static struct pci_ops dw_pcie_ops = {
 	.write = pci_generic_config_write,
 };
 
-void dw_pcie_setup_rc(struct dw_pcie_rp *pp)
+static int dw_pcie_iatu_setup(struct dw_pcie_rp *pp)
 {
-	u32 val, ctrl, num_ctrls;
 	struct dw_pcie *pci = to_dw_pcie_from_pp(pp);
+	struct resource_entry *entry;
+	int i, ret;
+
+	/* Note the very first outbound ATU is used for CFG IOs */
+	if (!pci->num_ob_windows) {
+		dev_err(pci->dev, "No outbound iATU found\n");
+		return -EINVAL;
+	}
+
+	/*
+	 * Ensure all outbound windows are disabled before proceeding with
+	 * the MEM/IO ranges setups.
+	 */
+	for (i = 0; i < pci->num_ob_windows; i++)
+		dw_pcie_disable_atu(pci, PCIE_ATU_REGION_DIR_OB, i);
+
+	i = 0;
+	resource_list_for_each_entry(entry, &pp->bridge->windows) {
+		if (resource_type(entry->res) != IORESOURCE_MEM)
+			continue;
+
+		if (pci->num_ob_windows <= ++i)
+			break;
+
+		ret = dw_pcie_prog_outbound_atu(pci, i, PCIE_ATU_TYPE_MEM,
+						entry->res->start,
+						entry->res->start - entry->offset,
+						resource_size(entry->res));
+		if (ret) {
+			dev_err(pci->dev, "Failed to set MEM range %pr\n",
+				entry->res);
+			return ret;
+		}
+	}
+
+	if (pp->io_size) {
+		if (pci->num_ob_windows > ++i) {
+			ret = dw_pcie_prog_outbound_atu(pci, i, PCIE_ATU_TYPE_IO,
+							pp->io_base,
+							pp->io_bus_addr,
+							pp->io_size);
+			if (ret) {
+				dev_err(pci->dev, "Failed to set IO range %pr\n",
+					entry->res);
+				return ret;
+			}
+		} else {
+			pp->cfg0_io_shared = true;
+		}
+	}
+
+	if (pci->num_ob_windows <= i)
+		dev_warn(pci->dev, "Resources exceed number of ATU entries (%d)\n",
+			 pci->num_ob_windows);
+
+	return 0;
+}
+
+int dw_pcie_setup_rc(struct dw_pcie_rp *pp)
+{
+	struct dw_pcie *pci = to_dw_pcie_from_pp(pp);
+	u32 val, ctrl, num_ctrls;
+	int ret;
 
 	/*
 	 * Enable DBI read-only registers for writing/updating configuration.
@@ -610,42 +688,9 @@ void dw_pcie_setup_rc(struct dw_pcie_rp *pp)
 	 * ATU, so we should not program the ATU here.
 	 */
 	if (pp->bridge->child_ops == &dw_child_pcie_ops) {
-		int i, atu_idx = 0;
-		struct resource_entry *entry;
-
-		/*
-		 * Ensure all outbound windows are disabled so there are
-		 * multiple matches
-		 */
-		for (i = 0; i < pci->num_ob_windows; i++)
-			dw_pcie_disable_atu(pci, PCIE_ATU_REGION_DIR_OB, i);
-
-		/* Get last memory resource entry */
-		resource_list_for_each_entry(entry, &pp->bridge->windows) {
-			if (resource_type(entry->res) != IORESOURCE_MEM)
-				continue;
-
-			if (pci->num_ob_windows <= ++atu_idx)
-				break;
-
-			dw_pcie_prog_outbound_atu(pci, atu_idx,
-						  PCIE_ATU_TYPE_MEM, entry->res->start,
-						  entry->res->start - entry->offset,
-						  resource_size(entry->res));
-		}
-
-		if (pp->io_size) {
-			if (pci->num_ob_windows > ++atu_idx)
-				dw_pcie_prog_outbound_atu(pci, atu_idx,
-							  PCIE_ATU_TYPE_IO, pp->io_base,
-							  pp->io_bus_addr, pp->io_size);
-			else
-				pp->cfg0_io_shared = true;
-		}
-
-		if (pci->num_ob_windows <= atu_idx)
-			dev_warn(dev, "Resources exceed number of ATU entries (%d)\n",
-				 pci->num_ob_windows);
+		ret = dw_pcie_iatu_setup(pp);
+		if (ret)
+			return ret;
 	}
 
 	dw_pcie_writel_dbi(pci, PCI_BASE_ADDRESS_0, 0);
@@ -658,5 +703,7 @@ void dw_pcie_setup_rc(struct dw_pcie_rp *pp)
 	dw_pcie_writel_dbi(pci, PCIE_LINK_WIDTH_SPEED_CONTROL, val);
 
 	dw_pcie_dbi_ro_wr_dis(pci);
+
+	return 0;
 }
 EXPORT_SYMBOL_GPL(dw_pcie_setup_rc);
diff --git a/drivers/pci/controller/dwc/pcie-designware.h b/drivers/pci/controller/dwc/pcie-designware.h
index 60f1ddc54933..c3e73ed9aff5 100644
--- a/drivers/pci/controller/dwc/pcie-designware.h
+++ b/drivers/pci/controller/dwc/pcie-designware.h
@@ -387,7 +387,7 @@ static inline void dw_pcie_stop_link(struct dw_pcie *pci)
 
 #ifdef CONFIG_PCIE_DW_HOST
 irqreturn_t dw_handle_msi_irq(struct dw_pcie_rp *pp);
-void dw_pcie_setup_rc(struct dw_pcie_rp *pp);
+int dw_pcie_setup_rc(struct dw_pcie_rp *pp);
 int dw_pcie_host_init(struct dw_pcie_rp *pp);
 void dw_pcie_host_deinit(struct dw_pcie_rp *pp);
 int dw_pcie_allocate_domains(struct dw_pcie_rp *pp);
@@ -399,8 +399,9 @@ static inline irqreturn_t dw_handle_msi_irq(struct dw_pcie_rp *pp)
 	return IRQ_NONE;
 }
 
-static inline void dw_pcie_setup_rc(struct dw_pcie_rp *pp)
+static inline int dw_pcie_setup_rc(struct dw_pcie_rp *pp)
 {
+	return 0;
 }
 
 static inline int dw_pcie_host_init(struct dw_pcie_rp *pp)
diff --git a/drivers/pci/controller/dwc/pcie-intel-gw.c b/drivers/pci/controller/dwc/pcie-intel-gw.c
index a44f685ec94d..c3481200e86a 100644
--- a/drivers/pci/controller/dwc/pcie-intel-gw.c
+++ b/drivers/pci/controller/dwc/pcie-intel-gw.c
@@ -302,7 +302,11 @@ static int intel_pcie_host_setup(struct intel_pcie *pcie)
 	intel_pcie_ltssm_disable(pcie);
 	intel_pcie_link_setup(pcie);
 	intel_pcie_init_n_fts(pci);
-	dw_pcie_setup_rc(&pci->pp);
+
+	ret = dw_pcie_setup_rc(&pci->pp);
+	if (ret)
+		goto app_init_err;
+
 	dw_pcie_upconfig_setup(pci);
 
 	intel_pcie_device_rst_deassert(pcie);
-- 
2.35.1

Re: [PATCH RESEND v4 14/15] PCI: dwc: Check iATU in/outbound ranges setup methods status

[Manivannan Sadhasivam]

On Fri, Jun 24, 2022 at 05:39:46PM +0300, Serge Semin wrote:
> Let's make the DWC PCIe RC/EP safer and more verbose for the invalid or
> failed inbound and outbound iATU windows setups. Needless to say that
> silently ignoring iATU regions setup errors may cause unpredictable
> errors. For instance if for some reason a cfg or IO window fails to be
> activated, then any CFG/IO requested won't reach target PCIe devices and
> the corresponding accessors will return platform-specific random values.
> 
> First of all we need to convert dw_pcie_ep_outbound_atu() method to check
> whether the specified outbound iATU range is successfully setup. That
> method is called by the pci_epc_ops.map_addr callback. Thus we'll make the
> EP-specific CPU->PCIe memory mappings saver.
> 
> Secondly since the iATU outbound range programming method now returns the
> operation status, it will be handy to take that status into account in the
> pci_ops.{map_bus,read,write} methods. Thus any failed mapping will be
> immediately noticeable by the PCIe CFG operations requesters.
> 
> Finally we need to convert the dw_pcie_setup_rc() method to returning the
> operation status, since the iATU outbound ranges setup procedure may now
> fail. It will be especially handy in case if the DW PCIe RC DT-node has
> invalid/unsupported (dma-)ranges property. Note since the suggested
> modification causes having too wide code indentation, it is reasonable
> from maintainability and readability points of view to move the outbound
> ranges setup procedure in the separate function.
> 
> Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>

Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>

Thanks,
Mani

> Reviewed-by: Rob Herring <robh@kernel.org>
> ---
>  .../pci/controller/dwc/pcie-designware-ep.c   |   9 +-
>  .../pci/controller/dwc/pcie-designware-host.c | 153 ++++++++++++------
>  drivers/pci/controller/dwc/pcie-designware.h  |   5 +-
>  drivers/pci/controller/dwc/pcie-intel-gw.c    |   6 +-
>  4 files changed, 114 insertions(+), 59 deletions(-)
> 
> diff --git a/drivers/pci/controller/dwc/pcie-designware-ep.c b/drivers/pci/controller/dwc/pcie-designware-ep.c
> index 2e91222f7c98..627c4b69878c 100644
> --- a/drivers/pci/controller/dwc/pcie-designware-ep.c
> +++ b/drivers/pci/controller/dwc/pcie-designware-ep.c
> @@ -184,8 +184,9 @@ static int dw_pcie_ep_outbound_atu(struct dw_pcie_ep *ep, u8 func_no,
>  				   phys_addr_t phys_addr,
>  				   u64 pci_addr, size_t size)
>  {
> -	u32 free_win;
>  	struct dw_pcie *pci = to_dw_pcie_from_ep(ep);
> +	u32 free_win;
> +	int ret;
>  
>  	free_win = find_first_zero_bit(ep->ob_window_map, pci->num_ob_windows);
>  	if (free_win >= pci->num_ob_windows) {
> @@ -193,8 +194,10 @@ static int dw_pcie_ep_outbound_atu(struct dw_pcie_ep *ep, u8 func_no,
>  		return -EINVAL;
>  	}
>  
> -	dw_pcie_prog_ep_outbound_atu(pci, func_no, free_win, PCIE_ATU_TYPE_MEM,
> -				     phys_addr, pci_addr, size);
> +	ret = dw_pcie_prog_ep_outbound_atu(pci, func_no, free_win, PCIE_ATU_TYPE_MEM,
> +					   phys_addr, pci_addr, size);
> +	if (ret)
> +		return ret;
>  
>  	set_bit(free_win, ep->ob_window_map);
>  	ep->outbound_addr[free_win] = phys_addr;
> diff --git a/drivers/pci/controller/dwc/pcie-designware-host.c b/drivers/pci/controller/dwc/pcie-designware-host.c
> index e0a2819608c6..6993ce9e856d 100644
> --- a/drivers/pci/controller/dwc/pcie-designware-host.c
> +++ b/drivers/pci/controller/dwc/pcie-designware-host.c
> @@ -412,7 +412,9 @@ int dw_pcie_host_init(struct dw_pcie_rp *pp)
>  
>  	dw_pcie_iatu_detect(pci);
>  
> -	dw_pcie_setup_rc(pp);
> +	ret = dw_pcie_setup_rc(pp);
> +	if (ret)
> +		goto err_free_msi;
>  
>  	if (!dw_pcie_link_up(pci)) {
>  		ret = dw_pcie_start_link(pci);
> @@ -466,10 +468,10 @@ EXPORT_SYMBOL_GPL(dw_pcie_host_deinit);
>  static void __iomem *dw_pcie_other_conf_map_bus(struct pci_bus *bus,
>  						unsigned int devfn, int where)
>  {
> -	int type;
> -	u32 busdev;
>  	struct dw_pcie_rp *pp = bus->sysdata;
>  	struct dw_pcie *pci = to_dw_pcie_from_pp(pp);
> +	int type, ret;
> +	u32 busdev;
>  
>  	/*
>  	 * Checking whether the link is up here is a last line of defense
> @@ -490,8 +492,10 @@ static void __iomem *dw_pcie_other_conf_map_bus(struct pci_bus *bus,
>  	else
>  		type = PCIE_ATU_TYPE_CFG1;
>  
> -
> -	dw_pcie_prog_outbound_atu(pci, 0, type, pp->cfg0_base, busdev, pp->cfg0_size);
> +	ret = dw_pcie_prog_outbound_atu(pci, 0, type, pp->cfg0_base, busdev,
> +					pp->cfg0_size);
> +	if (ret)
> +		return NULL;
>  
>  	return pp->va_cfg0_base + where;
>  }
> @@ -499,33 +503,45 @@ static void __iomem *dw_pcie_other_conf_map_bus(struct pci_bus *bus,
>  static int dw_pcie_rd_other_conf(struct pci_bus *bus, unsigned int devfn,
>  				 int where, int size, u32 *val)
>  {
> -	int ret;
>  	struct dw_pcie_rp *pp = bus->sysdata;
>  	struct dw_pcie *pci = to_dw_pcie_from_pp(pp);
> +	int ret;
>  
>  	ret = pci_generic_config_read(bus, devfn, where, size, val);
> +	if (ret != PCIBIOS_SUCCESSFUL)
> +		return ret;
>  
> -	if (!ret && pp->cfg0_io_shared)
> -		dw_pcie_prog_outbound_atu(pci, 0, PCIE_ATU_TYPE_IO, pp->io_base,
> -					  pp->io_bus_addr, pp->io_size);
> +	if (pp->cfg0_io_shared) {
> +		ret = dw_pcie_prog_outbound_atu(pci, 0, PCIE_ATU_TYPE_IO,
> +						pp->io_base, pp->io_bus_addr,
> +						pp->io_size);
> +		if (ret)
> +			return PCIBIOS_SET_FAILED;
> +	}
>  
> -	return ret;
> +	return PCIBIOS_SUCCESSFUL;
>  }
>  
>  static int dw_pcie_wr_other_conf(struct pci_bus *bus, unsigned int devfn,
>  				 int where, int size, u32 val)
>  {
> -	int ret;
>  	struct dw_pcie_rp *pp = bus->sysdata;
>  	struct dw_pcie *pci = to_dw_pcie_from_pp(pp);
> +	int ret;
>  
>  	ret = pci_generic_config_write(bus, devfn, where, size, val);
> +	if (ret != PCIBIOS_SUCCESSFUL)
> +		return ret;
>  
> -	if (!ret && pp->cfg0_io_shared)
> -		dw_pcie_prog_outbound_atu(pci, 0, PCIE_ATU_TYPE_IO, pp->io_base,
> -					  pp->io_bus_addr, pp->io_size);
> +	if (pp->cfg0_io_shared) {
> +		ret = dw_pcie_prog_outbound_atu(pci, 0, PCIE_ATU_TYPE_IO,
> +						pp->io_base, pp->io_bus_addr,
> +						pp->io_size);
> +		if (ret)
> +			return PCIBIOS_SET_FAILED;
> +	}
>  
> -	return ret;
> +	return PCIBIOS_SUCCESSFUL;
>  }
>  
>  static struct pci_ops dw_child_pcie_ops = {
> @@ -552,10 +568,72 @@ static struct pci_ops dw_pcie_ops = {
>  	.write = pci_generic_config_write,
>  };
>  
> -void dw_pcie_setup_rc(struct dw_pcie_rp *pp)
> +static int dw_pcie_iatu_setup(struct dw_pcie_rp *pp)
>  {
> -	u32 val, ctrl, num_ctrls;
>  	struct dw_pcie *pci = to_dw_pcie_from_pp(pp);
> +	struct resource_entry *entry;
> +	int i, ret;
> +
> +	/* Note the very first outbound ATU is used for CFG IOs */
> +	if (!pci->num_ob_windows) {
> +		dev_err(pci->dev, "No outbound iATU found\n");
> +		return -EINVAL;
> +	}
> +
> +	/*
> +	 * Ensure all outbound windows are disabled before proceeding with
> +	 * the MEM/IO ranges setups.
> +	 */
> +	for (i = 0; i < pci->num_ob_windows; i++)
> +		dw_pcie_disable_atu(pci, PCIE_ATU_REGION_DIR_OB, i);
> +
> +	i = 0;
> +	resource_list_for_each_entry(entry, &pp->bridge->windows) {
> +		if (resource_type(entry->res) != IORESOURCE_MEM)
> +			continue;
> +
> +		if (pci->num_ob_windows <= ++i)
> +			break;
> +
> +		ret = dw_pcie_prog_outbound_atu(pci, i, PCIE_ATU_TYPE_MEM,
> +						entry->res->start,
> +						entry->res->start - entry->offset,
> +						resource_size(entry->res));
> +		if (ret) {
> +			dev_err(pci->dev, "Failed to set MEM range %pr\n",
> +				entry->res);
> +			return ret;
> +		}
> +	}
> +
> +	if (pp->io_size) {
> +		if (pci->num_ob_windows > ++i) {
> +			ret = dw_pcie_prog_outbound_atu(pci, i, PCIE_ATU_TYPE_IO,
> +							pp->io_base,
> +							pp->io_bus_addr,
> +							pp->io_size);
> +			if (ret) {
> +				dev_err(pci->dev, "Failed to set IO range %pr\n",
> +					entry->res);
> +				return ret;
> +			}
> +		} else {
> +			pp->cfg0_io_shared = true;
> +		}
> +	}
> +
> +	if (pci->num_ob_windows <= i)
> +		dev_warn(pci->dev, "Resources exceed number of ATU entries (%d)\n",
> +			 pci->num_ob_windows);
> +
> +	return 0;
> +}
> +
> +int dw_pcie_setup_rc(struct dw_pcie_rp *pp)
> +{
> +	struct dw_pcie *pci = to_dw_pcie_from_pp(pp);
> +	u32 val, ctrl, num_ctrls;
> +	int ret;
>  
>  	/*
>  	 * Enable DBI read-only registers for writing/updating configuration.
> @@ -610,42 +688,9 @@ void dw_pcie_setup_rc(struct dw_pcie_rp *pp)
>  	 * ATU, so we should not program the ATU here.
>  	 */
>  	if (pp->bridge->child_ops == &dw_child_pcie_ops) {
> -		int i, atu_idx = 0;
> -		struct resource_entry *entry;
> -
> -		/*
> -		 * Ensure all outbound windows are disabled so there are
> -		 * multiple matches
> -		 */
> -		for (i = 0; i < pci->num_ob_windows; i++)
> -			dw_pcie_disable_atu(pci, PCIE_ATU_REGION_DIR_OB, i);
> -
> -		/* Get last memory resource entry */
> -		resource_list_for_each_entry(entry, &pp->bridge->windows) {
> -			if (resource_type(entry->res) != IORESOURCE_MEM)
> -				continue;
> -
> -			if (pci->num_ob_windows <= ++atu_idx)
> -				break;
> -
> -			dw_pcie_prog_outbound_atu(pci, atu_idx,
> -						  PCIE_ATU_TYPE_MEM, entry->res->start,
> -						  entry->res->start - entry->offset,
> -						  resource_size(entry->res));
> -		}
> -
> -		if (pp->io_size) {
> -			if (pci->num_ob_windows > ++atu_idx)
> -				dw_pcie_prog_outbound_atu(pci, atu_idx,
> -							  PCIE_ATU_TYPE_IO, pp->io_base,
> -							  pp->io_bus_addr, pp->io_size);
> -			else
> -				pp->cfg0_io_shared = true;
> -		}
> -
> -		if (pci->num_ob_windows <= atu_idx)
> -			dev_warn(dev, "Resources exceed number of ATU entries (%d)\n",
> -				 pci->num_ob_windows);
> +		ret = dw_pcie_iatu_setup(pp);
> +		if (ret)
> +			return ret;
>  	}
>  
>  	dw_pcie_writel_dbi(pci, PCI_BASE_ADDRESS_0, 0);
> @@ -658,5 +703,7 @@ void dw_pcie_setup_rc(struct dw_pcie_rp *pp)
>  	dw_pcie_writel_dbi(pci, PCIE_LINK_WIDTH_SPEED_CONTROL, val);
>  
>  	dw_pcie_dbi_ro_wr_dis(pci);
> +
> +	return 0;
>  }
>  EXPORT_SYMBOL_GPL(dw_pcie_setup_rc);
> diff --git a/drivers/pci/controller/dwc/pcie-designware.h b/drivers/pci/controller/dwc/pcie-designware.h
> index 60f1ddc54933..c3e73ed9aff5 100644
> --- a/drivers/pci/controller/dwc/pcie-designware.h
> +++ b/drivers/pci/controller/dwc/pcie-designware.h
> @@ -387,7 +387,7 @@ static inline void dw_pcie_stop_link(struct dw_pcie *pci)
>  
>  #ifdef CONFIG_PCIE_DW_HOST
>  irqreturn_t dw_handle_msi_irq(struct dw_pcie_rp *pp);
> -void dw_pcie_setup_rc(struct dw_pcie_rp *pp);
> +int dw_pcie_setup_rc(struct dw_pcie_rp *pp);
>  int dw_pcie_host_init(struct dw_pcie_rp *pp);
>  void dw_pcie_host_deinit(struct dw_pcie_rp *pp);
>  int dw_pcie_allocate_domains(struct dw_pcie_rp *pp);
> @@ -399,8 +399,9 @@ static inline irqreturn_t dw_handle_msi_irq(struct dw_pcie_rp *pp)
>  	return IRQ_NONE;
>  }
>  
> -static inline void dw_pcie_setup_rc(struct dw_pcie_rp *pp)
> +static inline int dw_pcie_setup_rc(struct dw_pcie_rp *pp)
>  {
> +	return 0;
>  }
>  
>  static inline int dw_pcie_host_init(struct dw_pcie_rp *pp)
> diff --git a/drivers/pci/controller/dwc/pcie-intel-gw.c b/drivers/pci/controller/dwc/pcie-intel-gw.c
> index a44f685ec94d..c3481200e86a 100644
> --- a/drivers/pci/controller/dwc/pcie-intel-gw.c
> +++ b/drivers/pci/controller/dwc/pcie-intel-gw.c
> @@ -302,7 +302,11 @@ static int intel_pcie_host_setup(struct intel_pcie *pcie)
>  	intel_pcie_ltssm_disable(pcie);
>  	intel_pcie_link_setup(pcie);
>  	intel_pcie_init_n_fts(pci);
> -	dw_pcie_setup_rc(&pci->pp);
> +
> +	ret = dw_pcie_setup_rc(&pci->pp);
> +	if (ret)
> +		goto app_init_err;
> +
>  	dw_pcie_upconfig_setup(pci);
>  
>  	intel_pcie_device_rst_deassert(pcie);
> -- 
> 2.35.1
> 

-- 
மணிவண்ணன் சதாசிவம்

Re: [PATCH RESEND v4 14/15] PCI: dwc: Check iATU in/outbound ranges setup methods status

[Bjorn Helgaas]

On Mon, Aug 01, 2022 at 07:24:57PM +0530, Manivannan Sadhasivam wrote:
> On Fri, Jun 24, 2022 at 05:39:46PM +0300, Serge Semin wrote:
> > Let's make the DWC PCIe RC/EP safer and more verbose for the invalid or
> > failed inbound and outbound iATU windows setups. Needless to say that
> > silently ignoring iATU regions setup errors may cause unpredictable
> > errors. For instance if for some reason a cfg or IO window fails to be
> > activated, then any CFG/IO requested won't reach target PCIe devices and
> > the corresponding accessors will return platform-specific random values.
> > 
> > First of all we need to convert dw_pcie_ep_outbound_atu() method to check
> > whether the specified outbound iATU range is successfully setup. That
> > method is called by the pci_epc_ops.map_addr callback. Thus we'll make the
> > EP-specific CPU->PCIe memory mappings saver.
> > 
> > Secondly since the iATU outbound range programming method now returns the
> > operation status, it will be handy to take that status into account in the
> > pci_ops.{map_bus,read,write} methods. Thus any failed mapping will be
> > immediately noticeable by the PCIe CFG operations requesters.
> > 
> > Finally we need to convert the dw_pcie_setup_rc() method to returning the
> > operation status, since the iATU outbound ranges setup procedure may now
> > fail. It will be especially handy in case if the DW PCIe RC DT-node has
> > invalid/unsupported (dma-)ranges property. Note since the suggested
> > modification causes having too wide code indentation, it is reasonable
> > from maintainability and readability points of view to move the outbound
> > ranges setup procedure in the separate function.
> > 
> > Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>
> 
> Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>

Thank you very much for all your reviews, Manivannan!  I added your
reviewed-by to the appropriate commits.

[PATCH RESEND v4 15/15] PCI: dwc: Introduce dma-ranges property support for RC-host

[Serge Semin]

In accordance with the generic PCIe Root Port DT-bindings the "dma-ranges"
property has the same format as the "ranges" property. The only difference
is in their semantics. The "dma-ranges" property describes the PCIe-to-CPU
memory mapping in opposite to the CPU-to-PCIe mapping of the "ranges"
property. Even though the DW PCIe controllers are normally equipped with
the internal Address Translation Unit which inbound and outbound tables
can be used to implement both properties semantics, it was surprising for
me to discover that the host-related part of the DW PCIe driver currently
supports the "ranges" property only while the "dma-ranges" windows are
just ignored. Having the "dma-ranges" supported in the driver would be
very handy for the platforms, that don't tolerate the 1:1 CPU-PCIe memory
mapping and require a customized PCIe memory layout. So let's fix that by
introducing the "dma-ranges" property support.

First of all we suggest to rename the dw_pcie_prog_inbound_atu() method to
dw_pcie_prog_ep_inbound_atu() and create a new version of the
dw_pcie_prog_inbound_atu() function. Thus we'll have two methods for the
RC and EP controllers respectively in the same way as it has been
developed for the outbound ATU setup methods.

Secondly aside with the memory window index and type the new
dw_pcie_prog_inbound_atu() function will accept CPU address, PCIe address
and size as its arguments. These parameters define the PCIe and CPU memory
ranges which will be used to setup the respective inbound ATU mapping. The
passed parameters need to be verified against the ATU ranges constraints
in the same way as it is done for the outbound ranges.

Finally the DMA-ranges detected for the PCIe controller need to be
converted to the inbound ATU entries during the host controller
initialization procedure. It will be done in the framework of the
dw_pcie_iatu_setup() method. Note before setting the inbound ranges up we
need to disable all the inbound ATU entries in order to prevent unexpected
PCIe TLPs translations defined by some third party software like
bootloaders.

Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>
Reviewed-by: Rob Herring <robh@kernel.org>

---

Changelog v3:
- Drop inbound iATU window size alignment constraint. (@Manivannan)
---
 .../pci/controller/dwc/pcie-designware-ep.c   |  4 +-
 .../pci/controller/dwc/pcie-designware-host.c | 32 ++++++++++-
 drivers/pci/controller/dwc/pcie-designware.c  | 56 ++++++++++++++++++-
 drivers/pci/controller/dwc/pcie-designware.h  |  6 +-
 4 files changed, 89 insertions(+), 9 deletions(-)

diff --git a/drivers/pci/controller/dwc/pcie-designware-ep.c b/drivers/pci/controller/dwc/pcie-designware-ep.c
index 627c4b69878c..441feff1917a 100644
--- a/drivers/pci/controller/dwc/pcie-designware-ep.c
+++ b/drivers/pci/controller/dwc/pcie-designware-ep.c
@@ -167,8 +167,8 @@ static int dw_pcie_ep_inbound_atu(struct dw_pcie_ep *ep, u8 func_no, int type,
 		return -EINVAL;
 	}
 
-	ret = dw_pcie_prog_inbound_atu(pci, func_no, free_win, type,
-				       cpu_addr, bar);
+	ret = dw_pcie_prog_ep_inbound_atu(pci, func_no, free_win, type,
+					  cpu_addr, bar);
 	if (ret < 0) {
 		dev_err(pci->dev, "Failed to program IB window\n");
 		return ret;
diff --git a/drivers/pci/controller/dwc/pcie-designware-host.c b/drivers/pci/controller/dwc/pcie-designware-host.c
index 6993ce9e856d..2fbe9dc11634 100644
--- a/drivers/pci/controller/dwc/pcie-designware-host.c
+++ b/drivers/pci/controller/dwc/pcie-designware-host.c
@@ -581,12 +581,15 @@ static int dw_pcie_iatu_setup(struct dw_pcie_rp *pp)
 	}
 
 	/*
-	 * Ensure all outbound windows are disabled before proceeding with
-	 * the MEM/IO ranges setups.
+	 * Ensure all out/inbound windows are disabled before proceeding with
+	 * the MEM/IO (dma-)ranges setups.
 	 */
 	for (i = 0; i < pci->num_ob_windows; i++)
 		dw_pcie_disable_atu(pci, PCIE_ATU_REGION_DIR_OB, i);
 
+	for (i = 0; i < pci->num_ib_windows; i++)
+		dw_pcie_disable_atu(pci, PCIE_ATU_REGION_DIR_IB, i);
+
 	i = 0;
 	resource_list_for_each_entry(entry, &pp->bridge->windows) {
 		if (resource_type(entry->res) != IORESOURCE_MEM)
@@ -623,9 +626,32 @@ static int dw_pcie_iatu_setup(struct dw_pcie_rp *pp)
 	}
 
 	if (pci->num_ob_windows <= i)
-		dev_warn(pci->dev, "Resources exceed number of ATU entries (%d)\n",
+		dev_warn(pci->dev, "Ranges exceed outbound iATU size (%d)\n",
 			 pci->num_ob_windows);
 
+	i = 0;
+	resource_list_for_each_entry(entry, &pp->bridge->dma_ranges) {
+		if (resource_type(entry->res) != IORESOURCE_MEM)
+			continue;
+
+		if (pci->num_ib_windows <= i)
+			break;
+
+		ret = dw_pcie_prog_inbound_atu(pci, i++, PCIE_ATU_TYPE_MEM,
+					       entry->res->start,
+					       entry->res->start - entry->offset,
+					       resource_size(entry->res));
+		if (ret) {
+			dev_err(pci->dev, "Failed to set DMA range %pr\n",
+				entry->res);
+			return ret;
+		}
+	}
+
+	if (pci->num_ib_windows <= i)
+		dev_warn(pci->dev, "Dma-ranges exceed inbound iATU size (%u)\n",
+			 pci->num_ib_windows);
+
 	return 0;
 }
 
diff --git a/drivers/pci/controller/dwc/pcie-designware.c b/drivers/pci/controller/dwc/pcie-designware.c
index 9c622b635fdd..7a5be3c4f8e0 100644
--- a/drivers/pci/controller/dwc/pcie-designware.c
+++ b/drivers/pci/controller/dwc/pcie-designware.c
@@ -396,8 +396,60 @@ static inline void dw_pcie_writel_atu_ib(struct dw_pcie *pci, u32 index, u32 reg
 	dw_pcie_writel_atu(pci, PCIE_ATU_REGION_DIR_IB, index, reg, val);
 }
 
-int dw_pcie_prog_inbound_atu(struct dw_pcie *pci, u8 func_no, int index,
-			     int type, u64 cpu_addr, u8 bar)
+int dw_pcie_prog_inbound_atu(struct dw_pcie *pci, int index, int type,
+			     u64 cpu_addr, u64 pci_addr, u64 size)
+{
+	u64 limit_addr = pci_addr + size - 1;
+	u32 retries, val;
+
+	if ((limit_addr & ~pci->region_limit) != (pci_addr & ~pci->region_limit) ||
+	    !IS_ALIGNED(cpu_addr, pci->region_align) ||
+	    !IS_ALIGNED(pci_addr, pci->region_align) || !size) {
+		return -EINVAL;
+	}
+
+	dw_pcie_writel_atu_ib(pci, index, PCIE_ATU_LOWER_BASE,
+			      lower_32_bits(pci_addr));
+	dw_pcie_writel_atu_ib(pci, index, PCIE_ATU_UPPER_BASE,
+			      upper_32_bits(pci_addr));
+
+	dw_pcie_writel_atu_ib(pci, index, PCIE_ATU_LIMIT,
+			      lower_32_bits(limit_addr));
+	if (dw_pcie_ver_is_ge(pci, 460A))
+		dw_pcie_writel_atu_ib(pci, index, PCIE_ATU_UPPER_LIMIT,
+				      upper_32_bits(limit_addr));
+
+	dw_pcie_writel_atu_ib(pci, index, PCIE_ATU_LOWER_TARGET,
+			      lower_32_bits(cpu_addr));
+	dw_pcie_writel_atu_ib(pci, index, PCIE_ATU_UPPER_TARGET,
+			      upper_32_bits(cpu_addr));
+
+	val = type;
+	if (upper_32_bits(limit_addr) > upper_32_bits(pci_addr) &&
+	    dw_pcie_ver_is_ge(pci, 460A))
+		val |= PCIE_ATU_INCREASE_REGION_SIZE;
+	dw_pcie_writel_atu_ib(pci, index, PCIE_ATU_REGION_CTRL1, val);
+	dw_pcie_writel_atu_ib(pci, index, PCIE_ATU_REGION_CTRL2, PCIE_ATU_ENABLE);
+
+	/*
+	 * Make sure ATU enable takes effect before any subsequent config
+	 * and I/O accesses.
+	 */
+	for (retries = 0; retries < LINK_WAIT_MAX_IATU_RETRIES; retries++) {
+		val = dw_pcie_readl_atu_ib(pci, index, PCIE_ATU_REGION_CTRL2);
+		if (val & PCIE_ATU_ENABLE)
+			return 0;
+
+		mdelay(LINK_WAIT_IATU);
+	}
+
+	dev_err(pci->dev, "Inbound iATU is not being enabled\n");
+
+	return -ETIMEDOUT;
+}
+
+int dw_pcie_prog_ep_inbound_atu(struct dw_pcie *pci, u8 func_no, int index,
+				int type, u64 cpu_addr, u8 bar)
 {
 	u32 retries, val;
 
diff --git a/drivers/pci/controller/dwc/pcie-designware.h b/drivers/pci/controller/dwc/pcie-designware.h
index c3e73ed9aff5..5954e8cf9eec 100644
--- a/drivers/pci/controller/dwc/pcie-designware.h
+++ b/drivers/pci/controller/dwc/pcie-designware.h
@@ -308,8 +308,10 @@ int dw_pcie_prog_outbound_atu(struct dw_pcie *pci, int index, int type,
 			      u64 cpu_addr, u64 pci_addr, u64 size);
 int dw_pcie_prog_ep_outbound_atu(struct dw_pcie *pci, u8 func_no, int index,
 				 int type, u64 cpu_addr, u64 pci_addr, u64 size);
-int dw_pcie_prog_inbound_atu(struct dw_pcie *pci, u8 func_no, int index,
-			     int type, u64 cpu_addr, u8 bar);
+int dw_pcie_prog_inbound_atu(struct dw_pcie *pci, int index, int type,
+			     u64 cpu_addr, u64 pci_addr, u64 size);
+int dw_pcie_prog_ep_inbound_atu(struct dw_pcie *pci, u8 func_no, int index,
+				int type, u64 cpu_addr, u8 bar);
 void dw_pcie_disable_atu(struct dw_pcie *pci, u32 dir, int index);
 void dw_pcie_setup(struct dw_pcie *pci);
 void dw_pcie_iatu_detect(struct dw_pcie *pci);
-- 
2.35.1

Re: [PATCH RESEND v4 15/15] PCI: dwc: Introduce dma-ranges property support for RC-host

[Bjorn Helgaas]

On Fri, Jun 24, 2022 at 05:39:47PM +0300, Serge Semin wrote:
> In accordance with the generic PCIe Root Port DT-bindings the "dma-ranges"
> property has the same format as the "ranges" property. The only difference
> is in their semantics. The "dma-ranges" property describes the PCIe-to-CPU
> memory mapping in opposite to the CPU-to-PCIe mapping of the "ranges"
> property. Even though the DW PCIe controllers are normally equipped with
> the internal Address Translation Unit which inbound and outbound tables
> can be used to implement both properties semantics, it was surprising for
> me to discover that the host-related part of the DW PCIe driver currently
> supports the "ranges" property only while the "dma-ranges" windows are
> just ignored. Having the "dma-ranges" supported in the driver would be
> very handy for the platforms, that don't tolerate the 1:1 CPU-PCIe memory
> mapping and require a customized PCIe memory layout. So let's fix that by
> introducing the "dma-ranges" property support.

Do we have a platform that requires this yet?  Or does this fix a bug?

I see that dw_pcie_host_init() calls devm_pci_alloc_host_bridge(),
which eventually parses "dma-ranges", but I don't see any DWC DT
bindings that use it yet.

I'm not clear on what value this adds today.

Bjorn

Re: [PATCH RESEND v4 15/15] PCI: dwc: Introduce dma-ranges property support for RC-host

[Serge Semin]

On Thu, Jul 28, 2022 at 05:11:20PM -0500, Bjorn Helgaas wrote:
> On Fri, Jun 24, 2022 at 05:39:47PM +0300, Serge Semin wrote:
> > In accordance with the generic PCIe Root Port DT-bindings the "dma-ranges"
> > property has the same format as the "ranges" property. The only difference
> > is in their semantics. The "dma-ranges" property describes the PCIe-to-CPU
> > memory mapping in opposite to the CPU-to-PCIe mapping of the "ranges"
> > property. Even though the DW PCIe controllers are normally equipped with
> > the internal Address Translation Unit which inbound and outbound tables
> > can be used to implement both properties semantics, it was surprising for
> > me to discover that the host-related part of the DW PCIe driver currently
> > supports the "ranges" property only while the "dma-ranges" windows are
> > just ignored. Having the "dma-ranges" supported in the driver would be
> > very handy for the platforms, that don't tolerate the 1:1 CPU-PCIe memory
> > mapping and require a customized PCIe memory layout. So let's fix that by
> > introducing the "dma-ranges" property support.
> 

> Do we have a platform that requires this yet?  Or does this fix a bug?
> 
> I see that dw_pcie_host_init() calls devm_pci_alloc_host_bridge(),
> which eventually parses "dma-ranges", but I don't see any DWC DT
> bindings that use it yet.
> 
> I'm not clear on what value this adds today.

There are several points of having this supported.
First of all, generic PCIe DT-bindings permit having the dma-ranges
specified for the PCIe RCs. If so having it unsupported by the driver
just breaks the bindings or at least makes it incomplete.
Second, the main point of this patchset is to add the dma-ranges
support.) Especially seeing some other PCIe RC drivers do have it
supported too.
Finally. It is required for our platform (and for all the platforms
with similar issues). The problem is that the outbound source window
base address (on CPU-side) is size-unaligned. It resides at the 128MB
base address (size is somewhat about ~335MB). In case of the
one-on-one CPU->PCI mapping the peripherals with relatively big BARs
(at least of 256MB) and which need the BARs having size-aligned memory
won't be supported. So we had to remap the PCIe space to the
size-aligned base address. But in its turn that caused the PCIe-CPU
memory overlap. So PCIe DMA stopped working for the overlapped memory
due to the implicit P2P transactions. In order to fix that we had to
add the dma-ranges support to the DW PCIe driver and use it to remap
the overlapped memory. So please add this patch to the repo. We really
need it.

-Sergey

> 
> Bjorn

Re: [PATCH RESEND v4 15/15] PCI: dwc: Introduce dma-ranges property support for RC-host

[Bjorn Helgaas]

On Fri, Jul 29, 2022 at 07:52:20AM +0300, Serge Semin wrote:
> On Thu, Jul 28, 2022 at 05:11:20PM -0500, Bjorn Helgaas wrote:
> > On Fri, Jun 24, 2022 at 05:39:47PM +0300, Serge Semin wrote:
> > > In accordance with the generic PCIe Root Port DT-bindings the "dma-ranges"
> > > property has the same format as the "ranges" property. The only difference
> > > is in their semantics. The "dma-ranges" property describes the PCIe-to-CPU
> > > memory mapping in opposite to the CPU-to-PCIe mapping of the "ranges"
> > > property. Even though the DW PCIe controllers are normally equipped with
> > > the internal Address Translation Unit which inbound and outbound tables
> > > can be used to implement both properties semantics, it was surprising for
> > > me to discover that the host-related part of the DW PCIe driver currently
> > > supports the "ranges" property only while the "dma-ranges" windows are
> > > just ignored. Having the "dma-ranges" supported in the driver would be
> > > very handy for the platforms, that don't tolerate the 1:1 CPU-PCIe memory
> > > mapping and require a customized PCIe memory layout. So let's fix that by
> > > introducing the "dma-ranges" property support.
> 
> > Do we have a platform that requires this yet?  Or does this fix a bug?
> > 
> > I see that dw_pcie_host_init() calls devm_pci_alloc_host_bridge(),
> > which eventually parses "dma-ranges", but I don't see any DWC DT
> > bindings that use it yet.
> > 
> > I'm not clear on what value this adds today.
> 
> There are several points of having this supported.
> First of all, generic PCIe DT-bindings permit having the dma-ranges
> specified for the PCIe RCs. If so having it unsupported by the driver
> just breaks the bindings or at least makes it incomplete.

Are there bindings in the tree that are broken and will be fixed by
this?

> Second, the main point of this patchset is to add the dma-ranges
> support.) Especially seeing some other PCIe RC drivers do have it
> supported too.
> Finally. It is required for our platform (and for all the platforms
> with similar issues). The problem is that the outbound source window
> base address (on CPU-side) is size-unaligned. It resides at the 128MB
> base address (size is somewhat about ~335MB). In case of the
> one-on-one CPU->PCI mapping the peripherals with relatively big BARs
> (at least of 256MB) and which need the BARs having size-aligned memory
> won't be supported. So we had to remap the PCIe space to the
> size-aligned base address. But in its turn that caused the PCIe-CPU
> memory overlap. So PCIe DMA stopped working for the overlapped memory
> due to the implicit P2P transactions. In order to fix that we had to
> add the dma-ranges support to the DW PCIe driver and use it to remap
> the overlapped memory. So please add this patch to the repo. We really
> need it.

Does the above apply to the pending Baikal-T1 driver?  If so, let's
just include this patch in that series.  Then we'll have a user of
this functionality and we'll be able to exercise and test this code.

Bjorn

Re: [PATCH RESEND v4 15/15] PCI: dwc: Introduce dma-ranges property support for RC-host

[Serge Semin]

On Fri, Jul 29, 2022 at 06:33:45AM -0500, Bjorn Helgaas wrote:
> On Fri, Jul 29, 2022 at 07:52:20AM +0300, Serge Semin wrote:
> > On Thu, Jul 28, 2022 at 05:11:20PM -0500, Bjorn Helgaas wrote:
> > > On Fri, Jun 24, 2022 at 05:39:47PM +0300, Serge Semin wrote:
> > > > In accordance with the generic PCIe Root Port DT-bindings the "dma-ranges"
> > > > property has the same format as the "ranges" property. The only difference
> > > > is in their semantics. The "dma-ranges" property describes the PCIe-to-CPU
> > > > memory mapping in opposite to the CPU-to-PCIe mapping of the "ranges"
> > > > property. Even though the DW PCIe controllers are normally equipped with
> > > > the internal Address Translation Unit which inbound and outbound tables
> > > > can be used to implement both properties semantics, it was surprising for
> > > > me to discover that the host-related part of the DW PCIe driver currently
> > > > supports the "ranges" property only while the "dma-ranges" windows are
> > > > just ignored. Having the "dma-ranges" supported in the driver would be
> > > > very handy for the platforms, that don't tolerate the 1:1 CPU-PCIe memory
> > > > mapping and require a customized PCIe memory layout. So let's fix that by
> > > > introducing the "dma-ranges" property support.
> > 
> > > Do we have a platform that requires this yet?  Or does this fix a bug?
> > > 
> > > I see that dw_pcie_host_init() calls devm_pci_alloc_host_bridge(),
> > > which eventually parses "dma-ranges", but I don't see any DWC DT
> > > bindings that use it yet.
> > > 
> > > I'm not clear on what value this adds today.
> > 
> > There are several points of having this supported.
> > First of all, generic PCIe DT-bindings permit having the dma-ranges
> > specified for the PCIe RCs. If so having it unsupported by the driver
> > just breaks the bindings or at least makes it incomplete.
> 

> Are there bindings in the tree that are broken and will be fixed by
> this?

Shortly speaking: explicitly none of them are broken, but implicitly most
of them are broken.)

As I said the generic DT-bindings permit having the dma-ranges
property specified for the PCIe RCs:
Link: https://github.com/robherring/dt-schema/blob/master/schemas/pci/pci-bus.yaml#L47

So all the PCIe RC bindings can be equipped with the dma-ranges
property. At least dtschema check tool won't print any error if the
property is specified for any of the currently available PCIe RC
device including DW PCIe RC. But if the dma-ranges aren't supported by
the driver having it specified won't give any expected effect. In case
of the DW PCIe RC device setting the dma-ranges property up won't
change anything because without this patch the dma-ranges won't be
accordingly translated to the inbound iATU settings.

> 
> > Second, the main point of this patchset is to add the dma-ranges
> > support.) Especially seeing some other PCIe RC drivers do have it
> > supported too.
> > Finally. It is required for our platform (and for all the platforms
> > with similar issues). The problem is that the outbound source window
> > base address (on CPU-side) is size-unaligned. It resides at the 128MB
> > base address (size is somewhat about ~335MB). In case of the
> > one-on-one CPU->PCI mapping the peripherals with relatively big BARs
> > (at least of 256MB) and which need the BARs having size-aligned memory
> > won't be supported. So we had to remap the PCIe space to the
> > size-aligned base address. But in its turn that caused the PCIe-CPU
> > memory overlap. So PCIe DMA stopped working for the overlapped memory
> > due to the implicit P2P transactions. In order to fix that we had to
> > add the dma-ranges support to the DW PCIe driver and use it to remap
> > the overlapped memory. So please add this patch to the repo. We really
> > need it.
> 

> Does the above apply to the pending Baikal-T1 driver? 

The problem described above is indeed specific to the Baikal-T1
platform.

> If so, let's
> just include this patch in that series.  Then we'll have a user of
> this functionality and we'll be able to exercise and test this code.

Basically it can be tested out on any platform, not only on ours. You
don't even need to have the problem described above. All you need is a
DW PCIe RC-device with the inbound iATU windows support, this patch
and a dts file with the dma-ranges specified for the corresponding DT
device node.

Secondly even if you do move this patch to that series it likely won't
be tested by anyone but me, which I and our customer have already done
many times here. By postponing the patch merge in you'll just postpone
the feature being utilized by the rest of the kernel users and nothing
else. I am sure it's working at least in our case. Let's add it to the
kernel on this cycle so in case of unexpected problems they could be
fixed in the next rc-stages.

-Sergey

> 
> Bjorn

Re: [PATCH RESEND v4 15/15] PCI: dwc: Introduce dma-ranges property support for RC-host

[Manivannan Sadhasivam]

On Fri, Jun 24, 2022 at 05:39:47PM +0300, Serge Semin wrote:
> In accordance with the generic PCIe Root Port DT-bindings the "dma-ranges"
> property has the same format as the "ranges" property. The only difference
> is in their semantics. The "dma-ranges" property describes the PCIe-to-CPU
> memory mapping in opposite to the CPU-to-PCIe mapping of the "ranges"
> property. Even though the DW PCIe controllers are normally equipped with
> the internal Address Translation Unit which inbound and outbound tables
> can be used to implement both properties semantics, it was surprising for
> me to discover that the host-related part of the DW PCIe driver currently
> supports the "ranges" property only while the "dma-ranges" windows are
> just ignored. Having the "dma-ranges" supported in the driver would be
> very handy for the platforms, that don't tolerate the 1:1 CPU-PCIe memory
> mapping and require a customized PCIe memory layout. So let's fix that by
> introducing the "dma-ranges" property support.
> 
> First of all we suggest to rename the dw_pcie_prog_inbound_atu() method to
> dw_pcie_prog_ep_inbound_atu() and create a new version of the
> dw_pcie_prog_inbound_atu() function. Thus we'll have two methods for the
> RC and EP controllers respectively in the same way as it has been
> developed for the outbound ATU setup methods.
> 
> Secondly aside with the memory window index and type the new
> dw_pcie_prog_inbound_atu() function will accept CPU address, PCIe address
> and size as its arguments. These parameters define the PCIe and CPU memory
> ranges which will be used to setup the respective inbound ATU mapping. The
> passed parameters need to be verified against the ATU ranges constraints
> in the same way as it is done for the outbound ranges.
> 
> Finally the DMA-ranges detected for the PCIe controller need to be
> converted to the inbound ATU entries during the host controller
> initialization procedure. It will be done in the framework of the
> dw_pcie_iatu_setup() method. Note before setting the inbound ranges up we
> need to disable all the inbound ATU entries in order to prevent unexpected
> PCIe TLPs translations defined by some third party software like
> bootloaders.
> 
> Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>

Small nitpick below,

Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>

Thanks,
Mani

> Reviewed-by: Rob Herring <robh@kernel.org>
> 
> ---
> 
> Changelog v3:
> - Drop inbound iATU window size alignment constraint. (@Manivannan)
> ---
>  .../pci/controller/dwc/pcie-designware-ep.c   |  4 +-
>  .../pci/controller/dwc/pcie-designware-host.c | 32 ++++++++++-
>  drivers/pci/controller/dwc/pcie-designware.c  | 56 ++++++++++++++++++-
>  drivers/pci/controller/dwc/pcie-designware.h  |  6 +-
>  4 files changed, 89 insertions(+), 9 deletions(-)
> 
> diff --git a/drivers/pci/controller/dwc/pcie-designware-ep.c b/drivers/pci/controller/dwc/pcie-designware-ep.c
> index 627c4b69878c..441feff1917a 100644
> --- a/drivers/pci/controller/dwc/pcie-designware-ep.c
> +++ b/drivers/pci/controller/dwc/pcie-designware-ep.c
> @@ -167,8 +167,8 @@ static int dw_pcie_ep_inbound_atu(struct dw_pcie_ep *ep, u8 func_no, int type,
>  		return -EINVAL;
>  	}
>  
> -	ret = dw_pcie_prog_inbound_atu(pci, func_no, free_win, type,
> -				       cpu_addr, bar);
> +	ret = dw_pcie_prog_ep_inbound_atu(pci, func_no, free_win, type,
> +					  cpu_addr, bar);
>  	if (ret < 0) {
>  		dev_err(pci->dev, "Failed to program IB window\n");
>  		return ret;
> diff --git a/drivers/pci/controller/dwc/pcie-designware-host.c b/drivers/pci/controller/dwc/pcie-designware-host.c
> index 6993ce9e856d..2fbe9dc11634 100644
> --- a/drivers/pci/controller/dwc/pcie-designware-host.c
> +++ b/drivers/pci/controller/dwc/pcie-designware-host.c
> @@ -581,12 +581,15 @@ static int dw_pcie_iatu_setup(struct dw_pcie_rp *pp)
>  	}
>  
>  	/*
> -	 * Ensure all outbound windows are disabled before proceeding with
> -	 * the MEM/IO ranges setups.
> +	 * Ensure all out/inbound windows are disabled before proceeding with
> +	 * the MEM/IO (dma-)ranges setups.
>  	 */
>  	for (i = 0; i < pci->num_ob_windows; i++)
>  		dw_pcie_disable_atu(pci, PCIE_ATU_REGION_DIR_OB, i);
>  
> +	for (i = 0; i < pci->num_ib_windows; i++)
> +		dw_pcie_disable_atu(pci, PCIE_ATU_REGION_DIR_IB, i);
> +
>  	i = 0;
>  	resource_list_for_each_entry(entry, &pp->bridge->windows) {
>  		if (resource_type(entry->res) != IORESOURCE_MEM)
> @@ -623,9 +626,32 @@ static int dw_pcie_iatu_setup(struct dw_pcie_rp *pp)
>  	}
>  
>  	if (pci->num_ob_windows <= i)
> -		dev_warn(pci->dev, "Resources exceed number of ATU entries (%d)\n",
> +		dev_warn(pci->dev, "Ranges exceed outbound iATU size (%d)\n",
>  			 pci->num_ob_windows);
>  

I think a comment here explaining what's going on below would be helpful in the
future.

> +	i = 0;
> +	resource_list_for_each_entry(entry, &pp->bridge->dma_ranges) {
> +		if (resource_type(entry->res) != IORESOURCE_MEM)
> +			continue;
> +
> +		if (pci->num_ib_windows <= i)
> +			break;
> +
> +		ret = dw_pcie_prog_inbound_atu(pci, i++, PCIE_ATU_TYPE_MEM,
> +					       entry->res->start,
> +					       entry->res->start - entry->offset,
> +					       resource_size(entry->res));
> +		if (ret) {
> +			dev_err(pci->dev, "Failed to set DMA range %pr\n",
> +				entry->res);
> +			return ret;
> +		}
> +	}
> +
> +	if (pci->num_ib_windows <= i)
> +		dev_warn(pci->dev, "Dma-ranges exceed inbound iATU size (%u)\n",

"dma-ranges"

> +			 pci->num_ib_windows);
> +
>  	return 0;
>  }
>  
> diff --git a/drivers/pci/controller/dwc/pcie-designware.c b/drivers/pci/controller/dwc/pcie-designware.c
> index 9c622b635fdd..7a5be3c4f8e0 100644
> --- a/drivers/pci/controller/dwc/pcie-designware.c
> +++ b/drivers/pci/controller/dwc/pcie-designware.c
> @@ -396,8 +396,60 @@ static inline void dw_pcie_writel_atu_ib(struct dw_pcie *pci, u32 index, u32 reg
>  	dw_pcie_writel_atu(pci, PCIE_ATU_REGION_DIR_IB, index, reg, val);
>  }
>  
> -int dw_pcie_prog_inbound_atu(struct dw_pcie *pci, u8 func_no, int index,
> -			     int type, u64 cpu_addr, u8 bar)
> +int dw_pcie_prog_inbound_atu(struct dw_pcie *pci, int index, int type,
> +			     u64 cpu_addr, u64 pci_addr, u64 size)
> +{
> +	u64 limit_addr = pci_addr + size - 1;
> +	u32 retries, val;
> +
> +	if ((limit_addr & ~pci->region_limit) != (pci_addr & ~pci->region_limit) ||
> +	    !IS_ALIGNED(cpu_addr, pci->region_align) ||
> +	    !IS_ALIGNED(pci_addr, pci->region_align) || !size) {
> +		return -EINVAL;
> +	}
> +
> +	dw_pcie_writel_atu_ib(pci, index, PCIE_ATU_LOWER_BASE,
> +			      lower_32_bits(pci_addr));
> +	dw_pcie_writel_atu_ib(pci, index, PCIE_ATU_UPPER_BASE,
> +			      upper_32_bits(pci_addr));
> +
> +	dw_pcie_writel_atu_ib(pci, index, PCIE_ATU_LIMIT,
> +			      lower_32_bits(limit_addr));
> +	if (dw_pcie_ver_is_ge(pci, 460A))
> +		dw_pcie_writel_atu_ib(pci, index, PCIE_ATU_UPPER_LIMIT,
> +				      upper_32_bits(limit_addr));
> +
> +	dw_pcie_writel_atu_ib(pci, index, PCIE_ATU_LOWER_TARGET,
> +			      lower_32_bits(cpu_addr));
> +	dw_pcie_writel_atu_ib(pci, index, PCIE_ATU_UPPER_TARGET,
> +			      upper_32_bits(cpu_addr));
> +
> +	val = type;
> +	if (upper_32_bits(limit_addr) > upper_32_bits(pci_addr) &&
> +	    dw_pcie_ver_is_ge(pci, 460A))
> +		val |= PCIE_ATU_INCREASE_REGION_SIZE;
> +	dw_pcie_writel_atu_ib(pci, index, PCIE_ATU_REGION_CTRL1, val);
> +	dw_pcie_writel_atu_ib(pci, index, PCIE_ATU_REGION_CTRL2, PCIE_ATU_ENABLE);
> +
> +	/*
> +	 * Make sure ATU enable takes effect before any subsequent config
> +	 * and I/O accesses.
> +	 */
> +	for (retries = 0; retries < LINK_WAIT_MAX_IATU_RETRIES; retries++) {
> +		val = dw_pcie_readl_atu_ib(pci, index, PCIE_ATU_REGION_CTRL2);
> +		if (val & PCIE_ATU_ENABLE)
> +			return 0;
> +
> +		mdelay(LINK_WAIT_IATU);
> +	}
> +
> +	dev_err(pci->dev, "Inbound iATU is not being enabled\n");
> +
> +	return -ETIMEDOUT;
> +}
> +
> +int dw_pcie_prog_ep_inbound_atu(struct dw_pcie *pci, u8 func_no, int index,
> +				int type, u64 cpu_addr, u8 bar)
>  {
>  	u32 retries, val;
>  
> diff --git a/drivers/pci/controller/dwc/pcie-designware.h b/drivers/pci/controller/dwc/pcie-designware.h
> index c3e73ed9aff5..5954e8cf9eec 100644
> --- a/drivers/pci/controller/dwc/pcie-designware.h
> +++ b/drivers/pci/controller/dwc/pcie-designware.h
> @@ -308,8 +308,10 @@ int dw_pcie_prog_outbound_atu(struct dw_pcie *pci, int index, int type,
>  			      u64 cpu_addr, u64 pci_addr, u64 size);
>  int dw_pcie_prog_ep_outbound_atu(struct dw_pcie *pci, u8 func_no, int index,
>  				 int type, u64 cpu_addr, u64 pci_addr, u64 size);
> -int dw_pcie_prog_inbound_atu(struct dw_pcie *pci, u8 func_no, int index,
> -			     int type, u64 cpu_addr, u8 bar);
> +int dw_pcie_prog_inbound_atu(struct dw_pcie *pci, int index, int type,
> +			     u64 cpu_addr, u64 pci_addr, u64 size);
> +int dw_pcie_prog_ep_inbound_atu(struct dw_pcie *pci, u8 func_no, int index,
> +				int type, u64 cpu_addr, u8 bar);
>  void dw_pcie_disable_atu(struct dw_pcie *pci, u32 dir, int index);
>  void dw_pcie_setup(struct dw_pcie *pci);
>  void dw_pcie_iatu_detect(struct dw_pcie *pci);
> -- 
> 2.35.1
> 

-- 
மணிவண்ணன் சதாசிவம்

Re: [PATCH RESEND v4 15/15] PCI: dwc: Introduce dma-ranges property support for RC-host

[Serge Semin]

On Mon, Aug 01, 2022 at 07:30:47PM +0530, Manivannan Sadhasivam wrote:
> On Fri, Jun 24, 2022 at 05:39:47PM +0300, Serge Semin wrote:
> > In accordance with the generic PCIe Root Port DT-bindings the "dma-ranges"
> > property has the same format as the "ranges" property. The only difference
> > is in their semantics. The "dma-ranges" property describes the PCIe-to-CPU
> > memory mapping in opposite to the CPU-to-PCIe mapping of the "ranges"
> > property. Even though the DW PCIe controllers are normally equipped with
> > the internal Address Translation Unit which inbound and outbound tables
> > can be used to implement both properties semantics, it was surprising for
> > me to discover that the host-related part of the DW PCIe driver currently
> > supports the "ranges" property only while the "dma-ranges" windows are
> > just ignored. Having the "dma-ranges" supported in the driver would be
> > very handy for the platforms, that don't tolerate the 1:1 CPU-PCIe memory
> > mapping and require a customized PCIe memory layout. So let's fix that by
> > introducing the "dma-ranges" property support.
> > 
> > First of all we suggest to rename the dw_pcie_prog_inbound_atu() method to
> > dw_pcie_prog_ep_inbound_atu() and create a new version of the
> > dw_pcie_prog_inbound_atu() function. Thus we'll have two methods for the
> > RC and EP controllers respectively in the same way as it has been
> > developed for the outbound ATU setup methods.
> > 
> > Secondly aside with the memory window index and type the new
> > dw_pcie_prog_inbound_atu() function will accept CPU address, PCIe address
> > and size as its arguments. These parameters define the PCIe and CPU memory
> > ranges which will be used to setup the respective inbound ATU mapping. The
> > passed parameters need to be verified against the ATU ranges constraints
> > in the same way as it is done for the outbound ranges.
> > 
> > Finally the DMA-ranges detected for the PCIe controller need to be
> > converted to the inbound ATU entries during the host controller
> > initialization procedure. It will be done in the framework of the
> > dw_pcie_iatu_setup() method. Note before setting the inbound ranges up we
> > need to disable all the inbound ATU entries in order to prevent unexpected
> > PCIe TLPs translations defined by some third party software like
> > bootloaders.
> > 
> > Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>
> 
> Small nitpick below,
> 
> Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
> 
> Thanks,
> Mani
> 
> > Reviewed-by: Rob Herring <robh@kernel.org>
> > 
> > ---
> > 
> > Changelog v3:
> > - Drop inbound iATU window size alignment constraint. (@Manivannan)
> > ---
> >  .../pci/controller/dwc/pcie-designware-ep.c   |  4 +-
> >  .../pci/controller/dwc/pcie-designware-host.c | 32 ++++++++++-
> >  drivers/pci/controller/dwc/pcie-designware.c  | 56 ++++++++++++++++++-
> >  drivers/pci/controller/dwc/pcie-designware.h  |  6 +-
> >  4 files changed, 89 insertions(+), 9 deletions(-)
> > 
> > diff --git a/drivers/pci/controller/dwc/pcie-designware-ep.c b/drivers/pci/controller/dwc/pcie-designware-ep.c
> > index 627c4b69878c..441feff1917a 100644
> > --- a/drivers/pci/controller/dwc/pcie-designware-ep.c
> > +++ b/drivers/pci/controller/dwc/pcie-designware-ep.c
> > @@ -167,8 +167,8 @@ static int dw_pcie_ep_inbound_atu(struct dw_pcie_ep *ep, u8 func_no, int type,
> >  		return -EINVAL;
> >  	}
> >  
> > -	ret = dw_pcie_prog_inbound_atu(pci, func_no, free_win, type,
> > -				       cpu_addr, bar);
> > +	ret = dw_pcie_prog_ep_inbound_atu(pci, func_no, free_win, type,
> > +					  cpu_addr, bar);
> >  	if (ret < 0) {
> >  		dev_err(pci->dev, "Failed to program IB window\n");
> >  		return ret;
> > diff --git a/drivers/pci/controller/dwc/pcie-designware-host.c b/drivers/pci/controller/dwc/pcie-designware-host.c
> > index 6993ce9e856d..2fbe9dc11634 100644
> > --- a/drivers/pci/controller/dwc/pcie-designware-host.c
> > +++ b/drivers/pci/controller/dwc/pcie-designware-host.c
> > @@ -581,12 +581,15 @@ static int dw_pcie_iatu_setup(struct dw_pcie_rp *pp)
> >  	}
> >  
> >  	/*
> > -	 * Ensure all outbound windows are disabled before proceeding with
> > -	 * the MEM/IO ranges setups.
> > +	 * Ensure all out/inbound windows are disabled before proceeding with
> > +	 * the MEM/IO (dma-)ranges setups.
> >  	 */
> >  	for (i = 0; i < pci->num_ob_windows; i++)
> >  		dw_pcie_disable_atu(pci, PCIE_ATU_REGION_DIR_OB, i);
> >  
> > +	for (i = 0; i < pci->num_ib_windows; i++)
> > +		dw_pcie_disable_atu(pci, PCIE_ATU_REGION_DIR_IB, i);
> > +
> >  	i = 0;
> >  	resource_list_for_each_entry(entry, &pp->bridge->windows) {
> >  		if (resource_type(entry->res) != IORESOURCE_MEM)
> > @@ -623,9 +626,32 @@ static int dw_pcie_iatu_setup(struct dw_pcie_rp *pp)
> >  	}
> >  
> >  	if (pci->num_ob_windows <= i)
> > -		dev_warn(pci->dev, "Resources exceed number of ATU entries (%d)\n",
> > +		dev_warn(pci->dev, "Ranges exceed outbound iATU size (%d)\n",
> >  			 pci->num_ob_windows);
> >  
> 

> I think a comment here explaining what's going on below would be helpful in the
> future.

Both ranges and dma-ranges are initialized in the same way. Moreover
the method is pretty much coherent with self-explained name. So adding
an additional comment here doesn't seem much required.

> 
> > +	i = 0;
> > +	resource_list_for_each_entry(entry, &pp->bridge->dma_ranges) {
> > +		if (resource_type(entry->res) != IORESOURCE_MEM)
> > +			continue;
> > +
> > +		if (pci->num_ib_windows <= i)
> > +			break;
> > +
> > +		ret = dw_pcie_prog_inbound_atu(pci, i++, PCIE_ATU_TYPE_MEM,
> > +					       entry->res->start,
> > +					       entry->res->start - entry->offset,
> > +					       resource_size(entry->res));
> > +		if (ret) {
> > +			dev_err(pci->dev, "Failed to set DMA range %pr\n",
> > +				entry->res);
> > +			return ret;
> > +		}
> > +	}
> > +
> > +	if (pci->num_ib_windows <= i)
> > +		dev_warn(pci->dev, "Dma-ranges exceed inbound iATU size (%u)\n",
> 

> "dma-ranges"

I was referring to the similar message printed in case of the number
of specified ranges goes out of bounds.

-Sergey

> 
> > +			 pci->num_ib_windows);
> > +
> >  	return 0;
> >  }
> >  
> > diff --git a/drivers/pci/controller/dwc/pcie-designware.c b/drivers/pci/controller/dwc/pcie-designware.c
> > index 9c622b635fdd..7a5be3c4f8e0 100644
> > --- a/drivers/pci/controller/dwc/pcie-designware.c
> > +++ b/drivers/pci/controller/dwc/pcie-designware.c
> > @@ -396,8 +396,60 @@ static inline void dw_pcie_writel_atu_ib(struct dw_pcie *pci, u32 index, u32 reg
> >  	dw_pcie_writel_atu(pci, PCIE_ATU_REGION_DIR_IB, index, reg, val);
> >  }
> >  
> > -int dw_pcie_prog_inbound_atu(struct dw_pcie *pci, u8 func_no, int index,
> > -			     int type, u64 cpu_addr, u8 bar)
> > +int dw_pcie_prog_inbound_atu(struct dw_pcie *pci, int index, int type,
> > +			     u64 cpu_addr, u64 pci_addr, u64 size)
> > +{
> > +	u64 limit_addr = pci_addr + size - 1;
> > +	u32 retries, val;
> > +
> > +	if ((limit_addr & ~pci->region_limit) != (pci_addr & ~pci->region_limit) ||
> > +	    !IS_ALIGNED(cpu_addr, pci->region_align) ||
> > +	    !IS_ALIGNED(pci_addr, pci->region_align) || !size) {
> > +		return -EINVAL;
> > +	}
> > +
> > +	dw_pcie_writel_atu_ib(pci, index, PCIE_ATU_LOWER_BASE,
> > +			      lower_32_bits(pci_addr));
> > +	dw_pcie_writel_atu_ib(pci, index, PCIE_ATU_UPPER_BASE,
> > +			      upper_32_bits(pci_addr));
> > +
> > +	dw_pcie_writel_atu_ib(pci, index, PCIE_ATU_LIMIT,
> > +			      lower_32_bits(limit_addr));
> > +	if (dw_pcie_ver_is_ge(pci, 460A))
> > +		dw_pcie_writel_atu_ib(pci, index, PCIE_ATU_UPPER_LIMIT,
> > +				      upper_32_bits(limit_addr));
> > +
> > +	dw_pcie_writel_atu_ib(pci, index, PCIE_ATU_LOWER_TARGET,
> > +			      lower_32_bits(cpu_addr));
> > +	dw_pcie_writel_atu_ib(pci, index, PCIE_ATU_UPPER_TARGET,
> > +			      upper_32_bits(cpu_addr));
> > +
> > +	val = type;
> > +	if (upper_32_bits(limit_addr) > upper_32_bits(pci_addr) &&
> > +	    dw_pcie_ver_is_ge(pci, 460A))
> > +		val |= PCIE_ATU_INCREASE_REGION_SIZE;
> > +	dw_pcie_writel_atu_ib(pci, index, PCIE_ATU_REGION_CTRL1, val);
> > +	dw_pcie_writel_atu_ib(pci, index, PCIE_ATU_REGION_CTRL2, PCIE_ATU_ENABLE);
> > +
> > +	/*
> > +	 * Make sure ATU enable takes effect before any subsequent config
> > +	 * and I/O accesses.
> > +	 */
> > +	for (retries = 0; retries < LINK_WAIT_MAX_IATU_RETRIES; retries++) {
> > +		val = dw_pcie_readl_atu_ib(pci, index, PCIE_ATU_REGION_CTRL2);
> > +		if (val & PCIE_ATU_ENABLE)
> > +			return 0;
> > +
> > +		mdelay(LINK_WAIT_IATU);
> > +	}
> > +
> > +	dev_err(pci->dev, "Inbound iATU is not being enabled\n");
> > +
> > +	return -ETIMEDOUT;
> > +}
> > +
> > +int dw_pcie_prog_ep_inbound_atu(struct dw_pcie *pci, u8 func_no, int index,
> > +				int type, u64 cpu_addr, u8 bar)
> >  {
> >  	u32 retries, val;
> >  
> > diff --git a/drivers/pci/controller/dwc/pcie-designware.h b/drivers/pci/controller/dwc/pcie-designware.h
> > index c3e73ed9aff5..5954e8cf9eec 100644
> > --- a/drivers/pci/controller/dwc/pcie-designware.h
> > +++ b/drivers/pci/controller/dwc/pcie-designware.h
> > @@ -308,8 +308,10 @@ int dw_pcie_prog_outbound_atu(struct dw_pcie *pci, int index, int type,
> >  			      u64 cpu_addr, u64 pci_addr, u64 size);
> >  int dw_pcie_prog_ep_outbound_atu(struct dw_pcie *pci, u8 func_no, int index,
> >  				 int type, u64 cpu_addr, u64 pci_addr, u64 size);
> > -int dw_pcie_prog_inbound_atu(struct dw_pcie *pci, u8 func_no, int index,
> > -			     int type, u64 cpu_addr, u8 bar);
> > +int dw_pcie_prog_inbound_atu(struct dw_pcie *pci, int index, int type,
> > +			     u64 cpu_addr, u64 pci_addr, u64 size);
> > +int dw_pcie_prog_ep_inbound_atu(struct dw_pcie *pci, u8 func_no, int index,
> > +				int type, u64 cpu_addr, u8 bar);
> >  void dw_pcie_disable_atu(struct dw_pcie *pci, u32 dir, int index);
> >  void dw_pcie_setup(struct dw_pcie *pci);
> >  void dw_pcie_iatu_detect(struct dw_pcie *pci);
> > -- 
> > 2.35.1
> > 
> 
> -- 
> மணிவண்ணன் சதாசிவம்

Re: [PATCH RESEND v4 00/15] PCI: dwc: Add hw version and dma-ranges support

[Serge Semin]

Hi Bjorn.

If you are back from vacation yet, could you please consider this patchset
merging in into your repo in the branch 'pci/edma' with Fank Li eDMA
updates?
Link: https://git.kernel.org/pub/scm/linux/kernel/git/helgaas/pci.git/log/?h=pci/edma

Note this must be done before the series:
Link: https://git.kernel.org/pub/scm/linux/kernel/git/helgaas/pci.git/log/?h=pci%2Fctrl%2Fdwc-fixes
content is moved there.

-Sergey

On Fri, Jun 24, 2022 at 05:39:32PM +0300, Serge Semin wrote:
> This patchset is a second one in the series created in the framework of
> my Baikal-T1 PCIe/eDMA-related work:
> 
> [1: In-progress v5] PCI: dwc: Various fixes and cleanups
> Link: https://lore.kernel.org/linux-pci/20220624143428.8334-1-Sergey.Semin@baikalelectronics.ru/
> [2: In-progress v4] PCI: dwc: Add hw version and dma-ranges support
> Link: ---you are looking at it---
> [3: In-progress v3] PCI: dwc: Add extended YAML-schema and Baikal-T1 support
> Link: https://lore.kernel.org/linux-pci/20220610085706.15741-1-Sergey.Semin@baikalelectronics.ru/
> [4: In-progress v3] dmaengine: dw-edma: Add RP/EP local DMA support
> Link: https://lore.kernel.org/linux-pci/20220610091459.17612-1-Sergey.Semin@baikalelectronics.ru
> 
> Note it is very recommended to merge the patchsets in the same order as
> they are listed in the set above in order to have them applied smoothly.
> Nothing prevents them from being reviewed synchronously though.
> 
> Originally the patches submitted in this patchset were a part of the series:
> Link: https://lore.kernel.org/linux-pci/20220503214638.1895-1-Sergey.Semin@baikalelectronics.ru/
> but due to the reviewers requests the series was expanded to about 30
> patches which made it too bulky for a comfortable review. So I decided to
> split it up into two patchsets: 2. and 3. in the table above.
> 
> Regarding the series content. This patchset is about adding new features
> to the DW PCIe core, Root Port and Endpoint drivers. First we suggest to
> add a more verbose link-up log message. Really printing link generation
> and width would be much more informative than just "link up". Then a
> series of IP-core version-related patches go, like using a native FourCC
> version representation, adding the IP-core auto-detection, adding a better
> structured IP-core version/type interface and finally dropping manual
> IP-core version setups from the platforms which are supposed to have it
> auto-detected. After that the platform-specific host de-initialization
> method is introduced. It's unused in the framework of this patchset but
> will be utilized in the next one (see the table above). A series of iATU
> optimizations, cleanups and new features goes afterwards. In particular we
> suggest to drop some redundant enumerations, add iATU regions size
> detection procedure and then use the regions parameters to verify the
> requested by the platform iATU ranges/dma-ranges settings. After that the
> dma-ranges property support is added for the DW PCIe Host controllers.
> 
> Link: https://lore.kernel.org/linux-pci/20220324013734.18234-1-Sergey.Semin@baikalelectronics.ru/
> Changelog v2:
> - Test the error condition first and return straight away if it comes true
>   in the link up waiting and link state logging method (@Joe).
> - Move the dw_pcie_region_type enumeration removal patch to being applied
>   before the IB/OB iATU windows setup simplification patch (@Rob).
> - Move the iATU region selection procedure into a helper function (@Rob).
> - Rebase from kernel v5.17 onto v5.18-rc3 since the later kernel has
>   already DT bindings converted. (@Rob)
> - Simplify the iATU region selection procedure by recalculating the base
>   address only if the space is unrolled. The iATU viewport base address
>   will be saved in the pci->atu_base field.
> - Move the IP-core version detection procedure call from
>   dw_pcie_ep_init_complete() to dw_pcie_ep_init().
> - Add a new patch: "PCI: dwc: Detect iATU settings after getting
>   "addr_space" resource."
> 
> Link: https://lore.kernel.org/linux-pci/20220503225104.12108-1-Sergey.Semin@baikalelectronics.ru
> Changelog v3:
> - Fix pcie-tegra194-acpi.c driver to using the new macros names.
>   (@Manivannan)
> - Drop in/outbound iATU window size alignment constraint. (@Manivannan)
> - Detach this series of patches into a dedicated patchset.
> - Rebase onto kernel v5.18.
> 
> Link: https://lore.kernel.org/linux-pci/20220610084444.14549-1-Sergey.Semin@baikalelectronics.ru/
> Changelog v4:
> - Just resend.
> - Rebase onto the kernel v5.19-rcX.
> 
> Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>
> Cc: Alexey Malahov <Alexey.Malahov@baikalelectronics.ru>
> Cc: Pavel Parkhomenko <Pavel.Parkhomenko@baikalelectronics.ru>
> Cc: "Krzysztof Wilczyński" <kw@linux.com>
> Cc: Frank Li <Frank.Li@nxp.com>
> Cc: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
> Cc: Rob Herring <robh+dt@kernel.org>
> Cc: linux-pci@vger.kernel.org
> Cc: devicetree@vger.kernel.org
> Cc: linux-kernel@vger.kernel.org
> 
> Serge Semin (15):
>   PCI: dwc: Add more verbose link-up message
>   PCI: dwc: Detect iATU settings after getting "addr_space" resource
>   PCI: dwc: Convert to using native IP-core versions representation
>   PCI: dwc: Add IP-core version detection procedure
>   PCI: dwc: Introduce Synopsys IP-core versions/types interface
>   PCI: intel-gw: Drop manual DW PCIe controller version setup
>   PCI: tegra194: Drop manual DW PCIe controller version setup
>   PCI: dwc: Add host de-initialization callback
>   PCI: dwc: Drop inbound iATU types enumeration - dw_pcie_as_type
>   PCI: dwc: Drop iATU regions enumeration - dw_pcie_region_type
>   PCI: dwc: Simplify in/outbound iATU setup methods
>   PCI: dwc: Add iATU regions size detection procedure
>   PCI: dwc: Verify in/out regions against iATU constraints
>   PCI: dwc: Check iATU in/outbound ranges setup methods status
>   PCI: dwc: Introduce dma-ranges property support for RC-host
> 
>  drivers/pci/controller/dwc/pci-keystone.c     |  12 +-
>  .../pci/controller/dwc/pcie-designware-ep.c   |  40 +-
>  .../pci/controller/dwc/pcie-designware-host.c | 202 +++++---
>  drivers/pci/controller/dwc/pcie-designware.c  | 461 ++++++++----------
>  drivers/pci/controller/dwc/pcie-designware.h  | 123 ++---
>  drivers/pci/controller/dwc/pcie-intel-gw.c    |  22 +-
>  .../pci/controller/dwc/pcie-tegra194-acpi.c   |   7 +-
>  drivers/pci/controller/dwc/pcie-tegra194.c    |   1 -
>  8 files changed, 461 insertions(+), 407 deletions(-)
> 
> -- 
> 2.35.1
> 

Re: [PATCH RESEND v4 00/15] PCI: dwc: Add hw version and dma-ranges support

[Bjorn Helgaas]

On Fri, Jun 24, 2022 at 05:39:32PM +0300, Serge Semin wrote:
> This patchset is a second one in the series created in the framework of
> my Baikal-T1 PCIe/eDMA-related work:
> 
> [1: In-progress v5] PCI: dwc: Various fixes and cleanups
> Link: https://lore.kernel.org/linux-pci/20220624143428.8334-1-Sergey.Semin@baikalelectronics.ru/
> [2: In-progress v4] PCI: dwc: Add hw version and dma-ranges support
> Link: ---you are looking at it---

https://lore.kernel.org/r/20220624143947.8991-1-Sergey.Semin@baikalelectronics.ru

> [3: In-progress v3] PCI: dwc: Add extended YAML-schema and Baikal-T1 support
> Link: https://lore.kernel.org/linux-pci/20220610085706.15741-1-Sergey.Semin@baikalelectronics.ru/
> [4: In-progress v3] dmaengine: dw-edma: Add RP/EP local DMA support
> Link: https://lore.kernel.org/linux-pci/20220610091459.17612-1-Sergey.Semin@baikalelectronics.ru

Hi Serge, is the above the latest and greatest list?  The 3rd series
doesn't apply cleanly for me:

  05:40:34 ~/linux (main)$ git checkout -b wip/serge v5.19-rc1
  Switched to a new branch 'wip/serge'

  # fetch 1: PCI: dwc: Various fixes and cleanups
  05:40:45 ~/linux (wip/serge)$ b4 am -om/ https://lore.kernel.org/linux-pci/20220624143428.8334-1-Sergey.Semin@baikalelectronics.ru/
  Analyzing 37 messages in the thread
  Checking attestation on all messages, may take a moment...
  ---
    ✓ [PATCH v5 1/18] PCI: dwc: Stop link in the host init error and de-initialization
    ✓ [PATCH v5 2/18] PCI: dwc: Add unroll iATU space support to the regions disable method
    ✓ [PATCH v5 3/18] PCI: dwc: Disable outbound windows for controllers with iATU
    ✓ [PATCH v5 4/18] PCI: dwc: Set INCREASE_REGION_SIZE flag based on limit address
    ✓ [PATCH v5 5/18] PCI: dwc: Deallocate EPC memory on EP init error
      + Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org> (✓ DKIM/linaro.org)
    ✓ [PATCH v5 6/18] PCI: dwc: Enable CDM-check independently from the num_lanes value
      + Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org> (✓ DKIM/linaro.org)
    ✓ [PATCH v5 7/18] PCI: dwc: Add braces to the multi-line if-else statements
    ✓ [PATCH v5 8/18] PCI: dwc: Add trailing new-line literals to the log messages
    ✓ [PATCH v5 9/18] PCI: dwc: Discard IP-core version checking on unrolled iATU detection
    ✓ [PATCH v5 10/18] PCI: dwc: Convert Link-up status method to using dw_pcie_readl_dbi()
    ✓ [PATCH v5 11/18] PCI: dwc: Organize local variables usage
      + Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org> (✓ DKIM/linaro.org)
    ✓ [PATCH v5 12/18] PCI: dwc: Re-use local pointer to the resource data
      + Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org> (✓ DKIM/linaro.org)
    ✓ [PATCH v5 13/18] PCI: dwc: Add start_link/stop_link inliners
      + Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org> (✓ DKIM/linaro.org)
    ✓ [PATCH v5 14/18] PCI: dwc: Move io_cfg_atu_shared to the Root Port descriptor
      + Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org> (✓ DKIM/linaro.org)
    ✓ [PATCH v5 15/18] PCI: dwc: Add dw_ prefix to the pcie_port structure name
      + Acked-by: Jesper Nilsson <jesper.nilsson@axis.com> (✓ DKIM/axis.com)
      + Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org> (✓ DKIM/linaro.org)
      + Reviewed-by: Neil Armstrong <narmstrong@baylibre.com> (✓ DKIM/baylibre-com.20210112.gappssmtp.com)
    ✓ [PATCH v5 16/18] PCI: dwc-plat: Simplify the probe method return value handling
    ✓ [PATCH v5 17/18] PCI: dwc-plat: Discard unused regmap pointer
    ✓ [PATCH v5 18/18] PCI: dwc-plat: Drop dw_plat_pcie_of_match forward declaration
    ---
    ✓ Signed: DKIM/baikalelectronics.ru
  ---
  Total patches: 18
  ---
  NOTE: some trailers ignored due to from/email mismatches:
      ! Trailer: Link: https://lore.kernel.org/linux-pci/20220616152048.gcqacgs2ed66vsl4@mobilestation/
       Msg From: Serge Semin <fancer.lancer@gmail.com>
  NOTE: Rerun with -S to apply them anyway
  ---
  Cover: m/v5_20220624_sergey_semin_pci_dwc_various_fixes_and_cleanups.cover
   Link: https://lore.kernel.org/r/20220624143428.8334-1-Sergey.Semin@baikalelectronics.ru
   Base: applies clean to current tree
	 git am m/v5_20220624_sergey_semin_pci_dwc_various_fixes_and_cleanups.mbx

  # apply 1: PCI: dwc: Various fixes and cleanups
  05:41:24 ~/linux (wip/serge)$ git am m/v5_20220624_sergey_semin_pci_dwc_various_fixes_and_cleanups.mbx
  Applying: PCI: dwc: Stop link in the host init error and de-initialization
  Applying: PCI: dwc: Add unroll iATU space support to the regions disable method
  Applying: PCI: dwc: Disable outbound windows for controllers with iATU
  Applying: PCI: dwc: Set INCREASE_REGION_SIZE flag based on limit address
  Applying: PCI: dwc: Deallocate EPC memory on EP init error
  Applying: PCI: dwc: Enable CDM-check independently from the num_lanes value
  Applying: PCI: dwc: Add braces to the multi-line if-else statements
  Applying: PCI: dwc: Add trailing new-line literals to the log messages
  Applying: PCI: dwc: Discard IP-core version checking on unrolled iATU detection
  Applying: PCI: dwc: Convert Link-up status method to using dw_pcie_readl_dbi()
  Applying: PCI: dwc: Organize local variables usage
  Applying: PCI: dwc: Re-use local pointer to the resource data
  Applying: PCI: dwc: Add start_link/stop_link inliners
  Applying: PCI: dwc: Move io_cfg_atu_shared to the Root Port descriptor
  Applying: PCI: dwc: Add dw_ prefix to the pcie_port structure name
  Applying: PCI: dwc-plat: Simplify the probe method return value handling
  Applying: PCI: dwc-plat: Discard unused regmap pointer
  Applying: PCI: dwc-plat: Drop dw_plat_pcie_of_match forward declaration

  # fetch 2: PCI: dwc: Add hw version and dma-ranges support
  05:41:46 ~/linux (wip/serge)$ b4 am -om/ https://lore.kernel.org/r/20220624143947.8991-1-Sergey.Semin@baikalelectronics.ru
  Looking up https://lore.kernel.org/r/20220624143947.8991-1-Sergey.Semin%40baikalelectronics.ru
  Analyzing 19 messages in the thread
  Checking attestation on all messages, may take a moment...
  ---
    ✓ [PATCH v4 1/15] PCI: dwc: Add more verbose link-up message
    ✓ [PATCH v4 2/15] PCI: dwc: Detect iATU settings after getting "addr_space" resource
    ✓ [PATCH v4 3/15] PCI: dwc: Convert to using native IP-core versions representation
    ✓ [PATCH v4 4/15] PCI: dwc: Add IP-core version detection procedure
    ✓ [PATCH v4 5/15] PCI: dwc: Introduce Synopsys IP-core versions/types interface
    ✓ [PATCH v4 6/15] PCI: intel-gw: Drop manual DW PCIe controller version setup
    ✓ [PATCH v4 7/15] PCI: tegra194: Drop manual DW PCIe controller version setup
    ✓ [PATCH v4 8/15] PCI: dwc: Add host de-initialization callback
    ✓ [PATCH v4 9/15] PCI: dwc: Drop inbound iATU types enumeration - dw_pcie_as_type
    ✓ [PATCH v4 10/15] PCI: dwc: Drop iATU regions enumeration - dw_pcie_region_type
    ✓ [PATCH v4 11/15] PCI: dwc: Simplify in/outbound iATU setup methods
    ✓ [PATCH v4 12/15] PCI: dwc: Add iATU regions size detection procedure
    ✓ [PATCH v4 13/15] PCI: dwc: Verify in/out regions against iATU constraints
    ✓ [PATCH v4 14/15] PCI: dwc: Check iATU in/outbound ranges setup methods status
    ✓ [PATCH v4 15/15] PCI: dwc: Introduce dma-ranges property support for RC-host
    ---
    ✓ Signed: DKIM/baikalelectronics.ru
  ---
  Total patches: 15
  ---
  NOTE: some trailers ignored due to from/email mismatches:
      ! Trailer: Link: https://git.kernel.org/pub/scm/linux/kernel/git/helgaas/pci.git/log/?h=pci%2Fctrl%2Fdwc-fixes
       Msg From: Serge Semin <fancer.lancer@gmail.com>
      ! Trailer: Link: https://git.kernel.org/pub/scm/linux/kernel/git/helgaas/pci.git/log/?h=pci/edma
       Msg From: Serge Semin <fancer.lancer@gmail.com>
  NOTE: Rerun with -S to apply them anyway
  ---
  Cover: m/v4_20220624_sergey_semin_pci_dwc_add_hw_version_and_dma_ranges_support.cover
   Link: https://lore.kernel.org/r/20220624143947.8991-1-Sergey.Semin@baikalelectronics.ru
   Base: applies clean to current tree
	 git am m/v4_20220624_sergey_semin_pci_dwc_add_hw_version_and_dma_ranges_support.mbx

  # apply 2: PCI: dwc: Add hw version and dma-ranges support
  05:42:05 ~/linux (wip/serge)$ git am m/v4_20220624_sergey_semin_pci_dwc_add_hw_version_and_dma_ranges_support.mbx
  Applying: PCI: dwc: Add more verbose link-up message
  Applying: PCI: dwc: Detect iATU settings after getting "addr_space" resource
  Applying: PCI: dwc: Convert to using native IP-core versions representation
  Applying: PCI: dwc: Add IP-core version detection procedure
  Applying: PCI: dwc: Introduce Synopsys IP-core versions/types interface
  Applying: PCI: intel-gw: Drop manual DW PCIe controller version setup
  Applying: PCI: tegra194: Drop manual DW PCIe controller version setup
  Applying: PCI: dwc: Add host de-initialization callback
  Applying: PCI: dwc: Drop inbound iATU types enumeration - dw_pcie_as_type
  Applying: PCI: dwc: Drop iATU regions enumeration - dw_pcie_region_type
  Applying: PCI: dwc: Simplify in/outbound iATU setup methods
  Applying: PCI: dwc: Add iATU regions size detection procedure
  Applying: PCI: dwc: Verify in/out regions against iATU constraints
  Applying: PCI: dwc: Check iATU in/outbound ranges setup methods status
  Applying: PCI: dwc: Introduce dma-ranges property support for RC-host

  # fetch 3: PCI: dwc: Add extended YAML-schema and Baikal-T1 support
  05:42:40 ~/linux (wip/serge)$ b4 am -om/ https://lore.kernel.org/linux-pci/20220610085706.15741-1-Sergey.Semin@baikalelectronics.ru/
  Analyzing 51 messages in the thread
  Checking attestation on all messages, may take a moment...
  ---
    ✓ [PATCH v3 1/17] dt-bindings: PCI: dwc: Detach common RP/EP DT bindings
    ✓ [PATCH v3 2/17] dt-bindings: PCI: dwc: Remove bus node from the examples
      + Reviewed-by: Rob Herring <robh@kernel.org>
    ✓ [PATCH v3 3/17] dt-bindings: PCI: dwc: Add phys/phy-names common properties
    ✓ [PATCH v3 4/17] dt-bindings: PCI: dwc: Add max-link-speed common property
    ✓ [PATCH v3 5/17] dt-bindings: PCI: dwc: Stop selecting generic bindings by default
    ✓ [PATCH v3 6/17] dt-bindings: PCI: dwc: Add max-functions EP property
      + Reviewed-by: Rob Herring <robh@kernel.org>
    ✓ [PATCH v3 7/17] dt-bindings: PCI: dwc: Add interrupts/interrupt-names common properties
    ✓ [PATCH v3 8/17] dt-bindings: PCI: dwc: Add reg/reg-names common properties
    ✓ [PATCH v3 9/17] dt-bindings: PCI: dwc: Add clocks/resets common properties
    ✓ [PATCH v3 10/17] dt-bindings: PCI: dwc: Add dma-coherent property
    ✓ [PATCH v3 11/17] dt-bindings: PCI: dwc: Apply common schema to Rockchip DW PCIe nodes
    ✓ [PATCH v3 12/17] dt-bindings: PCI: dwc: Add Baikal-T1 PCIe Root Port bindings
    ✓ [PATCH v3 13/17] PCI: dwc: Introduce generic controller capabilities interface
      + Reviewed-by: Rob Herring <robh@kernel.org>
    ✓ [PATCH v3 14/17] PCI: dwc: Introduce generic resources getter
      + Reviewed-by: Rob Herring <robh@kernel.org>
    ✓ [PATCH v3 15/17] PCI: dwc: Combine iATU detection procedures
      + Reviewed-by: Rob Herring <robh@kernel.org>
    ✓ [PATCH v3 16/17] PCI: dwc: Introduce generic platform clocks and resets
    ✓ [PATCH v3 17/17] PCI: dwc: Add Baikal-T1 PCIe controller support
    ---
    ✓ Signed: DKIM/baikalelectronics.ru
  ---
  Total patches: 17
  ---
  NOTE: some trailers ignored due to from/email mismatches:
      ! Trailer: Link: https://lore.kernel.org/linux-pci/20220610091459.17612-23-Sergey.Semin@baikalelectronics.ru/
       Msg From: Serge Semin <fancer.lancer@gmail.com>
      ! Trailer: Link: https://lore.kernel.org/linux-pci/20220610091459.17612-1-Sergey.Semin@baikalelectronics.ru/
       Msg From: Serge Semin <fancer.lancer@gmail.com>
  NOTE: Rerun with -S to apply them anyway
  ---
  Cover: m/v3_20220610_sergey_semin_pci_dwc_add_generic_resources_and_baikal_t1_support.cover
   Link: https://lore.kernel.org/r/20220610085706.15741-1-Sergey.Semin@baikalelectronics.ru
   Base: not specified
	 git am m/v3_20220610_sergey_semin_pci_dwc_add_generic_resources_and_baikal_t1_support.mbx

  # apply 3: PCI: dwc: Add extended YAML-schema and Baikal-T1 support
  05:42:58 ~/linux (wip/serge)$ git am m/v3_20220610_sergey_semin_pci_dwc_add_generic_resources_and_baikal_t1_support.mbx
  Applying: dt-bindings: PCI: dwc: Detach common RP/EP DT bindings
  error: patch failed: Documentation/devicetree/bindings/pci/snps,dw-pcie-ep.yaml:36
  error: Documentation/devicetree/bindings/pci/snps,dw-pcie-ep.yaml: patch does not apply
  error: patch failed: Documentation/devicetree/bindings/pci/snps,dw-pcie.yaml:37
  error: Documentation/devicetree/bindings/pci/snps,dw-pcie.yaml: patch does not apply
  Patch failed at 0001 dt-bindings: PCI: dwc: Detach common RP/EP DT bindings
  hint: Use 'git am --show-current-patch' to see the failed patch
  When you have resolved this problem, run "git am --continue".
  If you prefer to skip this patch, run "git am --skip" instead.
  To restore the original branch and stop patching, run "git am --abort".
  05:43:07 ~/linux (wip/serge|AM 1/17)$

Re: [PATCH RESEND v4 00/15] PCI: dwc: Add hw version and dma-ranges support

[Serge Semin]

Hi Bjorn

On Wed, Jul 27, 2022 at 05:49:04PM -0500, Bjorn Helgaas wrote:
> On Fri, Jun 24, 2022 at 05:39:32PM +0300, Serge Semin wrote:
> > This patchset is a second one in the series created in the framework of
> > my Baikal-T1 PCIe/eDMA-related work:
> > 
> > [1: In-progress v5] PCI: dwc: Various fixes and cleanups
> > Link: https://lore.kernel.org/linux-pci/20220624143428.8334-1-Sergey.Semin@baikalelectronics.ru/
> > [2: In-progress v4] PCI: dwc: Add hw version and dma-ranges support
> > Link: ---you are looking at it---
> 
> https://lore.kernel.org/r/20220624143947.8991-1-Sergey.Semin@baikalelectronics.ru
> 
> > [3: In-progress v3] PCI: dwc: Add extended YAML-schema and Baikal-T1 support
> > Link: https://lore.kernel.org/linux-pci/20220610085706.15741-1-Sergey.Semin@baikalelectronics.ru/
> > [4: In-progress v3] dmaengine: dw-edma: Add RP/EP local DMA support
> > Link: https://lore.kernel.org/linux-pci/20220610091459.17612-1-Sergey.Semin@baikalelectronics.ru
> 

> Hi Serge, is the above the latest and greatest list?  The 3rd series
> doesn't apply cleanly for me:

Thanks for getting back to my patchsets. The 3rd series is still under
review (see below for details). Here is the summary regarding the
patchsets state:

[1: Done v5] PCI: dwc: Various fixes and cleanups
The latest lore
Link: https://lore.kernel.org/linux-pci/20220624143428.8334-1-Sergey.Semin@baikalelectronics.ru/
You've already merged it in here:
Link: https://git.kernel.org/pub/scm/linux/kernel/git/helgaas/pci.git/log/?h=pci/ctrl/dwc

[2: Done v4] PCI: dwc: Add hw version and dma-ranges support
The latest lore
Link: https://lore.kernel.org/linux-pci/20220624143947.8991-1-Sergey.Semin@baikalelectronics.ru

[3: In-progress v3] PCI: dwc: Add extended YAML-schema and Baikal-T1 support
The latest lore
Link: https://lore.kernel.org/linux-pci/20220610085706.15741-1-Sergey.Semin@baikalelectronics.ru/
This series, you are having difficulties to apply, is still under Rob'
review. I've addressed all your comments locally in my repo, but Rob'
didn't respond on some of my questions (there are several principal
aspects of the dt-bindings patches he didn't like). First he used to
be in vacation, then he just missed several of my email messages, but
responded on another ones. Pings didn't work either. Then I thought
the merge-window was opened so I has stopped bothering him for some
time.) So I'll resend the series as is one more time today then.
Hopefully he'll get back to it soon.

[4: Done v3] dmaengine: dw-edma: Add RP/EP local DMA support
The latest lore
Link: https://lore.kernel.org/linux-pci/20220610091459.17612-1-Sergey.Semin@baikalelectronics.ru/
The series review is done. It has even got Vinod' Ab tag. But I need
to resend it (will do this today) since it needs to be rebased on the
latest Frank Li patchset. Frank series is in your repo already:
Link: https://git.kernel.org/pub/scm/linux/kernel/git/helgaas/pci.git/log/?h=pci/ctrl/dwc-edma

So if you aren't going to merge the DW PCIe/eDMA related series in on this
merge window, please collect my and Frank work in a single branch of
yours (pci/ctrl/dwc-edma), since my patchset #4 depends on his series
and the patchsets #3, #2 and #1 described above.

-Sergey

> 
>   05:40:34 ~/linux (main)$ git checkout -b wip/serge v5.19-rc1
>   Switched to a new branch 'wip/serge'
> 
>   # fetch 1: PCI: dwc: Various fixes and cleanups
>   05:40:45 ~/linux (wip/serge)$ b4 am -om/ https://lore.kernel.org/linux-pci/20220624143428.8334-1-Sergey.Semin@baikalelectronics.ru/
>   Analyzing 37 messages in the thread
>   Checking attestation on all messages, may take a moment...
>   ---
>     ✓ [PATCH v5 1/18] PCI: dwc: Stop link in the host init error and de-initialization
>     ✓ [PATCH v5 2/18] PCI: dwc: Add unroll iATU space support to the regions disable method
>     ✓ [PATCH v5 3/18] PCI: dwc: Disable outbound windows for controllers with iATU
>     ✓ [PATCH v5 4/18] PCI: dwc: Set INCREASE_REGION_SIZE flag based on limit address
>     ✓ [PATCH v5 5/18] PCI: dwc: Deallocate EPC memory on EP init error
>       + Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org> (✓ DKIM/linaro.org)
>     ✓ [PATCH v5 6/18] PCI: dwc: Enable CDM-check independently from the num_lanes value
>       + Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org> (✓ DKIM/linaro.org)
>     ✓ [PATCH v5 7/18] PCI: dwc: Add braces to the multi-line if-else statements
>     ✓ [PATCH v5 8/18] PCI: dwc: Add trailing new-line literals to the log messages
>     ✓ [PATCH v5 9/18] PCI: dwc: Discard IP-core version checking on unrolled iATU detection
>     ✓ [PATCH v5 10/18] PCI: dwc: Convert Link-up status method to using dw_pcie_readl_dbi()
>     ✓ [PATCH v5 11/18] PCI: dwc: Organize local variables usage
>       + Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org> (✓ DKIM/linaro.org)
>     ✓ [PATCH v5 12/18] PCI: dwc: Re-use local pointer to the resource data
>       + Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org> (✓ DKIM/linaro.org)
>     ✓ [PATCH v5 13/18] PCI: dwc: Add start_link/stop_link inliners
>       + Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org> (✓ DKIM/linaro.org)
>     ✓ [PATCH v5 14/18] PCI: dwc: Move io_cfg_atu_shared to the Root Port descriptor
>       + Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org> (✓ DKIM/linaro.org)
>     ✓ [PATCH v5 15/18] PCI: dwc: Add dw_ prefix to the pcie_port structure name
>       + Acked-by: Jesper Nilsson <jesper.nilsson@axis.com> (✓ DKIM/axis.com)
>       + Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org> (✓ DKIM/linaro.org)
>       + Reviewed-by: Neil Armstrong <narmstrong@baylibre.com> (✓ DKIM/baylibre-com.20210112.gappssmtp.com)
>     ✓ [PATCH v5 16/18] PCI: dwc-plat: Simplify the probe method return value handling
>     ✓ [PATCH v5 17/18] PCI: dwc-plat: Discard unused regmap pointer
>     ✓ [PATCH v5 18/18] PCI: dwc-plat: Drop dw_plat_pcie_of_match forward declaration
>     ---
>     ✓ Signed: DKIM/baikalelectronics.ru
>   ---
>   Total patches: 18
>   ---
>   NOTE: some trailers ignored due to from/email mismatches:
>       ! Trailer: Link: https://lore.kernel.org/linux-pci/20220616152048.gcqacgs2ed66vsl4@mobilestation/
>        Msg From: Serge Semin <fancer.lancer@gmail.com>
>   NOTE: Rerun with -S to apply them anyway
>   ---
>   Cover: m/v5_20220624_sergey_semin_pci_dwc_various_fixes_and_cleanups.cover
>    Link: https://lore.kernel.org/r/20220624143428.8334-1-Sergey.Semin@baikalelectronics.ru
>    Base: applies clean to current tree
> 	 git am m/v5_20220624_sergey_semin_pci_dwc_various_fixes_and_cleanups.mbx
> 
>   # apply 1: PCI: dwc: Various fixes and cleanups
>   05:41:24 ~/linux (wip/serge)$ git am m/v5_20220624_sergey_semin_pci_dwc_various_fixes_and_cleanups.mbx
>   Applying: PCI: dwc: Stop link in the host init error and de-initialization
>   Applying: PCI: dwc: Add unroll iATU space support to the regions disable method
>   Applying: PCI: dwc: Disable outbound windows for controllers with iATU
>   Applying: PCI: dwc: Set INCREASE_REGION_SIZE flag based on limit address
>   Applying: PCI: dwc: Deallocate EPC memory on EP init error
>   Applying: PCI: dwc: Enable CDM-check independently from the num_lanes value
>   Applying: PCI: dwc: Add braces to the multi-line if-else statements
>   Applying: PCI: dwc: Add trailing new-line literals to the log messages
>   Applying: PCI: dwc: Discard IP-core version checking on unrolled iATU detection
>   Applying: PCI: dwc: Convert Link-up status method to using dw_pcie_readl_dbi()
>   Applying: PCI: dwc: Organize local variables usage
>   Applying: PCI: dwc: Re-use local pointer to the resource data
>   Applying: PCI: dwc: Add start_link/stop_link inliners
>   Applying: PCI: dwc: Move io_cfg_atu_shared to the Root Port descriptor
>   Applying: PCI: dwc: Add dw_ prefix to the pcie_port structure name
>   Applying: PCI: dwc-plat: Simplify the probe method return value handling
>   Applying: PCI: dwc-plat: Discard unused regmap pointer
>   Applying: PCI: dwc-plat: Drop dw_plat_pcie_of_match forward declaration
> 
>   # fetch 2: PCI: dwc: Add hw version and dma-ranges support
>   05:41:46 ~/linux (wip/serge)$ b4 am -om/ https://lore.kernel.org/r/20220624143947.8991-1-Sergey.Semin@baikalelectronics.ru
>   Looking up https://lore.kernel.org/r/20220624143947.8991-1-Sergey.Semin%40baikalelectronics.ru
>   Analyzing 19 messages in the thread
>   Checking attestation on all messages, may take a moment...
>   ---
>     ✓ [PATCH v4 1/15] PCI: dwc: Add more verbose link-up message
>     ✓ [PATCH v4 2/15] PCI: dwc: Detect iATU settings after getting "addr_space" resource
>     ✓ [PATCH v4 3/15] PCI: dwc: Convert to using native IP-core versions representation
>     ✓ [PATCH v4 4/15] PCI: dwc: Add IP-core version detection procedure
>     ✓ [PATCH v4 5/15] PCI: dwc: Introduce Synopsys IP-core versions/types interface
>     ✓ [PATCH v4 6/15] PCI: intel-gw: Drop manual DW PCIe controller version setup
>     ✓ [PATCH v4 7/15] PCI: tegra194: Drop manual DW PCIe controller version setup
>     ✓ [PATCH v4 8/15] PCI: dwc: Add host de-initialization callback
>     ✓ [PATCH v4 9/15] PCI: dwc: Drop inbound iATU types enumeration - dw_pcie_as_type
>     ✓ [PATCH v4 10/15] PCI: dwc: Drop iATU regions enumeration - dw_pcie_region_type
>     ✓ [PATCH v4 11/15] PCI: dwc: Simplify in/outbound iATU setup methods
>     ✓ [PATCH v4 12/15] PCI: dwc: Add iATU regions size detection procedure
>     ✓ [PATCH v4 13/15] PCI: dwc: Verify in/out regions against iATU constraints
>     ✓ [PATCH v4 14/15] PCI: dwc: Check iATU in/outbound ranges setup methods status
>     ✓ [PATCH v4 15/15] PCI: dwc: Introduce dma-ranges property support for RC-host
>     ---
>     ✓ Signed: DKIM/baikalelectronics.ru
>   ---
>   Total patches: 15
>   ---
>   NOTE: some trailers ignored due to from/email mismatches:
>       ! Trailer: Link: https://git.kernel.org/pub/scm/linux/kernel/git/helgaas/pci.git/log/?h=pci%2Fctrl%2Fdwc-fixes
>        Msg From: Serge Semin <fancer.lancer@gmail.com>
>       ! Trailer: Link: https://git.kernel.org/pub/scm/linux/kernel/git/helgaas/pci.git/log/?h=pci/edma
>        Msg From: Serge Semin <fancer.lancer@gmail.com>
>   NOTE: Rerun with -S to apply them anyway
>   ---
>   Cover: m/v4_20220624_sergey_semin_pci_dwc_add_hw_version_and_dma_ranges_support.cover
>    Link: https://lore.kernel.org/r/20220624143947.8991-1-Sergey.Semin@baikalelectronics.ru
>    Base: applies clean to current tree
> 	 git am m/v4_20220624_sergey_semin_pci_dwc_add_hw_version_and_dma_ranges_support.mbx
> 
>   # apply 2: PCI: dwc: Add hw version and dma-ranges support
>   05:42:05 ~/linux (wip/serge)$ git am m/v4_20220624_sergey_semin_pci_dwc_add_hw_version_and_dma_ranges_support.mbx
>   Applying: PCI: dwc: Add more verbose link-up message
>   Applying: PCI: dwc: Detect iATU settings after getting "addr_space" resource
>   Applying: PCI: dwc: Convert to using native IP-core versions representation
>   Applying: PCI: dwc: Add IP-core version detection procedure
>   Applying: PCI: dwc: Introduce Synopsys IP-core versions/types interface
>   Applying: PCI: intel-gw: Drop manual DW PCIe controller version setup
>   Applying: PCI: tegra194: Drop manual DW PCIe controller version setup
>   Applying: PCI: dwc: Add host de-initialization callback
>   Applying: PCI: dwc: Drop inbound iATU types enumeration - dw_pcie_as_type
>   Applying: PCI: dwc: Drop iATU regions enumeration - dw_pcie_region_type
>   Applying: PCI: dwc: Simplify in/outbound iATU setup methods
>   Applying: PCI: dwc: Add iATU regions size detection procedure
>   Applying: PCI: dwc: Verify in/out regions against iATU constraints
>   Applying: PCI: dwc: Check iATU in/outbound ranges setup methods status
>   Applying: PCI: dwc: Introduce dma-ranges property support for RC-host
> 
>   # fetch 3: PCI: dwc: Add extended YAML-schema and Baikal-T1 support
>   05:42:40 ~/linux (wip/serge)$ b4 am -om/ https://lore.kernel.org/linux-pci/20220610085706.15741-1-Sergey.Semin@baikalelectronics.ru/
>   Analyzing 51 messages in the thread
>   Checking attestation on all messages, may take a moment...
>   ---
>     ✓ [PATCH v3 1/17] dt-bindings: PCI: dwc: Detach common RP/EP DT bindings
>     ✓ [PATCH v3 2/17] dt-bindings: PCI: dwc: Remove bus node from the examples
>       + Reviewed-by: Rob Herring <robh@kernel.org>
>     ✓ [PATCH v3 3/17] dt-bindings: PCI: dwc: Add phys/phy-names common properties
>     ✓ [PATCH v3 4/17] dt-bindings: PCI: dwc: Add max-link-speed common property
>     ✓ [PATCH v3 5/17] dt-bindings: PCI: dwc: Stop selecting generic bindings by default
>     ✓ [PATCH v3 6/17] dt-bindings: PCI: dwc: Add max-functions EP property
>       + Reviewed-by: Rob Herring <robh@kernel.org>
>     ✓ [PATCH v3 7/17] dt-bindings: PCI: dwc: Add interrupts/interrupt-names common properties
>     ✓ [PATCH v3 8/17] dt-bindings: PCI: dwc: Add reg/reg-names common properties
>     ✓ [PATCH v3 9/17] dt-bindings: PCI: dwc: Add clocks/resets common properties
>     ✓ [PATCH v3 10/17] dt-bindings: PCI: dwc: Add dma-coherent property
>     ✓ [PATCH v3 11/17] dt-bindings: PCI: dwc: Apply common schema to Rockchip DW PCIe nodes
>     ✓ [PATCH v3 12/17] dt-bindings: PCI: dwc: Add Baikal-T1 PCIe Root Port bindings
>     ✓ [PATCH v3 13/17] PCI: dwc: Introduce generic controller capabilities interface
>       + Reviewed-by: Rob Herring <robh@kernel.org>
>     ✓ [PATCH v3 14/17] PCI: dwc: Introduce generic resources getter
>       + Reviewed-by: Rob Herring <robh@kernel.org>
>     ✓ [PATCH v3 15/17] PCI: dwc: Combine iATU detection procedures
>       + Reviewed-by: Rob Herring <robh@kernel.org>
>     ✓ [PATCH v3 16/17] PCI: dwc: Introduce generic platform clocks and resets
>     ✓ [PATCH v3 17/17] PCI: dwc: Add Baikal-T1 PCIe controller support
>     ---
>     ✓ Signed: DKIM/baikalelectronics.ru
>   ---
>   Total patches: 17
>   ---
>   NOTE: some trailers ignored due to from/email mismatches:
>       ! Trailer: Link: https://lore.kernel.org/linux-pci/20220610091459.17612-23-Sergey.Semin@baikalelectronics.ru/
>        Msg From: Serge Semin <fancer.lancer@gmail.com>
>       ! Trailer: Link: https://lore.kernel.org/linux-pci/20220610091459.17612-1-Sergey.Semin@baikalelectronics.ru/
>        Msg From: Serge Semin <fancer.lancer@gmail.com>
>   NOTE: Rerun with -S to apply them anyway
>   ---
>   Cover: m/v3_20220610_sergey_semin_pci_dwc_add_generic_resources_and_baikal_t1_support.cover
>    Link: https://lore.kernel.org/r/20220610085706.15741-1-Sergey.Semin@baikalelectronics.ru
>    Base: not specified
> 	 git am m/v3_20220610_sergey_semin_pci_dwc_add_generic_resources_and_baikal_t1_support.mbx
> 
>   # apply 3: PCI: dwc: Add extended YAML-schema and Baikal-T1 support
>   05:42:58 ~/linux (wip/serge)$ git am m/v3_20220610_sergey_semin_pci_dwc_add_generic_resources_and_baikal_t1_support.mbx
>   Applying: dt-bindings: PCI: dwc: Detach common RP/EP DT bindings
>   error: patch failed: Documentation/devicetree/bindings/pci/snps,dw-pcie-ep.yaml:36
>   error: Documentation/devicetree/bindings/pci/snps,dw-pcie-ep.yaml: patch does not apply
>   error: patch failed: Documentation/devicetree/bindings/pci/snps,dw-pcie.yaml:37
>   error: Documentation/devicetree/bindings/pci/snps,dw-pcie.yaml: patch does not apply
>   Patch failed at 0001 dt-bindings: PCI: dwc: Detach common RP/EP DT bindings
>   hint: Use 'git am --show-current-patch' to see the failed patch
>   When you have resolved this problem, run "git am --continue".
>   If you prefer to skip this patch, run "git am --skip" instead.
>   To restore the original branch and stop patching, run "git am --abort".
>   05:43:07 ~/linux (wip/serge|AM 1/17)$

Re: [PATCH RESEND v4 00/15] PCI: dwc: Add hw version and dma-ranges support

[Bjorn Helgaas]

On Fri, Jun 24, 2022 at 05:39:32PM +0300, Serge Semin wrote:
> This patchset is a second one in the series created in the framework of
> my Baikal-T1 PCIe/eDMA-related work:

> Serge Semin (15):
>   PCI: dwc: Add more verbose link-up message
>   PCI: dwc: Detect iATU settings after getting "addr_space" resource
>   PCI: dwc: Convert to using native IP-core versions representation
>   PCI: dwc: Add IP-core version detection procedure
>   PCI: dwc: Introduce Synopsys IP-core versions/types interface
>   PCI: intel-gw: Drop manual DW PCIe controller version setup
>   PCI: tegra194: Drop manual DW PCIe controller version setup
>   PCI: dwc: Add host de-initialization callback
>   PCI: dwc: Drop inbound iATU types enumeration - dw_pcie_as_type
>   PCI: dwc: Drop iATU regions enumeration - dw_pcie_region_type
>   PCI: dwc: Simplify in/outbound iATU setup methods
>   PCI: dwc: Add iATU regions size detection procedure
>   PCI: dwc: Verify in/out regions against iATU constraints
>   PCI: dwc: Check iATU in/outbound ranges setup methods status

I applied the above to pci/ctrl/dwc for v5.20, thanks!

>   PCI: dwc: Introduce dma-ranges property support for RC-host

I deferred this one for now because the current value isn't clear yet.
If we have a user for it, I'll be glad to add it.

Bjorn

Re: [PATCH RESEND v4 00/15] PCI: dwc: Add hw version and dma-ranges support

[Lorenzo Pieralisi]

On Thu, Jul 28, 2022 at 09:36:45PM -0500, Bjorn Helgaas wrote:
> On Fri, Jun 24, 2022 at 05:39:32PM +0300, Serge Semin wrote:
> > This patchset is a second one in the series created in the framework of
> > my Baikal-T1 PCIe/eDMA-related work:
> 
> > Serge Semin (15):
> >   PCI: dwc: Add more verbose link-up message
> >   PCI: dwc: Detect iATU settings after getting "addr_space" resource
> >   PCI: dwc: Convert to using native IP-core versions representation
> >   PCI: dwc: Add IP-core version detection procedure
> >   PCI: dwc: Introduce Synopsys IP-core versions/types interface
> >   PCI: intel-gw: Drop manual DW PCIe controller version setup
> >   PCI: tegra194: Drop manual DW PCIe controller version setup
> >   PCI: dwc: Add host de-initialization callback
> >   PCI: dwc: Drop inbound iATU types enumeration - dw_pcie_as_type
> >   PCI: dwc: Drop iATU regions enumeration - dw_pcie_region_type
> >   PCI: dwc: Simplify in/outbound iATU setup methods
> >   PCI: dwc: Add iATU regions size detection procedure
> >   PCI: dwc: Verify in/out regions against iATU constraints
> >   PCI: dwc: Check iATU in/outbound ranges setup methods status
> 
> I applied the above to pci/ctrl/dwc for v5.20, thanks!
> 
> >   PCI: dwc: Introduce dma-ranges property support for RC-host
> 
> I deferred this one for now because the current value isn't clear yet.
> If we have a user for it, I'll be glad to add it.

This one is still deferred and I agree with Bjorn it should be part of
a series that actually requires it - it would also simplify the review.

Lorenzo

Re: [PATCH RESEND v4 00/15] PCI: dwc: Add hw version and dma-ranges support

[Serge Semin]

On Wed, Sep 28, 2022 at 10:10:52AM +0200, Lorenzo Pieralisi wrote:
> On Thu, Jul 28, 2022 at 09:36:45PM -0500, Bjorn Helgaas wrote:
> > On Fri, Jun 24, 2022 at 05:39:32PM +0300, Serge Semin wrote:
> > > This patchset is a second one in the series created in the framework of
> > > my Baikal-T1 PCIe/eDMA-related work:
> > 
> > > Serge Semin (15):
> > >   PCI: dwc: Add more verbose link-up message
> > >   PCI: dwc: Detect iATU settings after getting "addr_space" resource
> > >   PCI: dwc: Convert to using native IP-core versions representation
> > >   PCI: dwc: Add IP-core version detection procedure
> > >   PCI: dwc: Introduce Synopsys IP-core versions/types interface
> > >   PCI: intel-gw: Drop manual DW PCIe controller version setup
> > >   PCI: tegra194: Drop manual DW PCIe controller version setup
> > >   PCI: dwc: Add host de-initialization callback
> > >   PCI: dwc: Drop inbound iATU types enumeration - dw_pcie_as_type
> > >   PCI: dwc: Drop iATU regions enumeration - dw_pcie_region_type
> > >   PCI: dwc: Simplify in/outbound iATU setup methods
> > >   PCI: dwc: Add iATU regions size detection procedure
> > >   PCI: dwc: Verify in/out regions against iATU constraints
> > >   PCI: dwc: Check iATU in/outbound ranges setup methods status
> > 
> > I applied the above to pci/ctrl/dwc for v5.20, thanks!
> > 
> > >   PCI: dwc: Introduce dma-ranges property support for RC-host
> > 
> > I deferred this one for now because the current value isn't clear yet.
> > If we have a user for it, I'll be glad to add it.
> 

> This one is still deferred and I agree with Bjorn it should be part of
> a series that actually requires it - it would also simplify the review.

Have already moved it to the next patchset:
https://lore.kernel.org/linux-pci/20220822184701.25246-1-Sergey.Semin@baikalelectronics.ru/
See the patch in there:
https://lore.kernel.org/linux-pci/20220822184701.25246-16-Sergey.Semin@baikalelectronics.ru/

-Sergey

> 
> Lorenzo