[PATCH v5 00/18] dwc MSI fixes, ARTPEC-6 EP mode support, ARTPEC-7 SoC support

[PATCH v5 00/18] dwc MSI fixes, ARTPEC-6 EP mode support, ARTPEC-7 SoC support

[Niklas Cassel]

This is a series that adds:
- PCI endpoint mode support in the ARTPEC-6 driver.
- ARTPEC-7 SoC support in the ARTPEC-6 driver (the SoCs are very similar).
- Small fixes for MSI in designware-ep and designware-host,
  needed to get endpoint mode support working for ARTPEC-6.
- Cleanups in pci-dra7xx to better prepare for endpoint mode in other
  DWC based PCIe drivers.

New in V5:
-Replaced ep->page_size with epc->mem->page_size in patch
 ("PCI: designware-ep: Pre-allocate memory for MSI in dw_pcie_ep_init") and
 in patch ("PCI: designware-ep: Add generic function for raising MSI irq").
-Included a new patch in this patch series:
 ("PCI: dwc: artpec6: Deassert the core before waiting for PHY").

Niklas Cassel (18):
  PCI: dwc: Use the DMA-API to get the MSI address
  PCI: designware-ep: dw_pcie_ep_set_msi() should only set MMC bits
  PCI: designware-ep: Read-only registers need DBI_RO_WR_EN to be
    writable
  PCI: designware-ep: Pre-allocate memory for MSI in dw_pcie_ep_init
  PCI: designware-ep: Remove static keyword from dw_pcie_ep_reset_bar()
  PCI: designware-ep: Add generic function for raising MSI irq
  PCI: dwc: dra7xx: Refactor Kconfig and Makefile handling for host/ep
    mode
  PCI: dwc: dra7xx: Assign pp->ops in dra7xx_add_pcie_port() rather than
    in probe
  PCI: dwc: dra7xx: Help compiler to remove unused code
  PCI: dwc: artpec6: Remove unused defines
  PCI: dwc: artpec6: Use BIT and GENMASK macros
  PCI: dwc: artpec6: Split artpec6_pcie_establish_link() into smaller
    functions
  bindings: PCI: artpec: Add support for endpoint mode
  PCI: dwc: artpec6: Add support for endpoint mode
  PCI: dwc: Make cpu_addr_fixup take struct dw_pcie as argument
  PCI: dwc: artpec6: Deassert the core before waiting for PHY
  bindings: PCI: artpec: Add support for the ARTPEC-7 SoC
  PCI: dwc: artpec6: Add support for the ARTPEC-7 SoC

 .../devicetree/bindings/pci/axis,artpec6-pcie.txt  |   5 +-
 drivers/pci/dwc/Kconfig                            |  68 +--
 drivers/pci/dwc/Makefile                           |   4 +-
 drivers/pci/dwc/pci-dra7xx.c                       |  27 +-
 drivers/pci/dwc/pcie-artpec6.c                     | 470 ++++++++++++++++++---
 drivers/pci/dwc/pcie-designware-ep.c               |  59 ++-
 drivers/pci/dwc/pcie-designware-host.c             |  15 +-
 drivers/pci/dwc/pcie-designware.c                  |   2 +-
 drivers/pci/dwc/pcie-designware.h                  |  22 +-
 9 files changed, 554 insertions(+), 118 deletions(-)

-- 
2.14.2

[PATCH v5 01/18] PCI: dwc: Use the DMA-API to get the MSI address

[Niklas Cassel]

Use the DMA-API to get the MSI address. This address will be written to
our PCI config space and to the register which determines which AXI
address the DWC IP will spoof for incoming MSI irqs.

Since it is a PCIe endpoint device, rather than the CPU, that is supposed
to write to the MSI address, the proper way to get the MSI address is by
using the DMA API, not by using virt_to_phys().

Using virt_to_phys() might work on some systems, but using the DMA API
should work on all systems.

This is essentially the same thing as allocating a buffer in a driver
to which the endpoint will write to. To do this, we use the DMA API.

Signed-off-by: Niklas Cassel <niklas.cassel@axis.com>
---
 drivers/pci/dwc/pcie-designware-host.c | 15 ++++++++++++---
 drivers/pci/dwc/pcie-designware.h      |  3 ++-
 2 files changed, 14 insertions(+), 4 deletions(-)

diff --git a/drivers/pci/dwc/pcie-designware-host.c b/drivers/pci/dwc/pcie-designware-host.c
index 81e2157a7cfb..33b52fe98a01 100644
--- a/drivers/pci/dwc/pcie-designware-host.c
+++ b/drivers/pci/dwc/pcie-designware-host.c
@@ -83,10 +83,19 @@ irqreturn_t dw_handle_msi_irq(struct pcie_port *pp)
 
 void dw_pcie_msi_init(struct pcie_port *pp)
 {
+	struct dw_pcie *pci = to_dw_pcie_from_pp(pp);
+	struct device *dev = pci->dev;
+	struct page *page;
 	u64 msi_target;
 
-	pp->msi_data = __get_free_pages(GFP_KERNEL, 0);
-	msi_target = virt_to_phys((void *)pp->msi_data);
+	page = alloc_page(GFP_KERNEL | GFP_DMA32);
+	pp->msi_data = dma_map_page(dev, page, 0, PAGE_SIZE, DMA_FROM_DEVICE);
+	if (dma_mapping_error(dev, pp->msi_data)) {
+		dev_err(dev, "failed to map MSI data\n");
+		__free_page(page);
+		return;
+	}
+	msi_target = (u64)pp->msi_data;
 
 	/* program the msi_data */
 	dw_pcie_wr_own_conf(pp, PCIE_MSI_ADDR_LO, 4,
@@ -187,7 +196,7 @@ static void dw_msi_setup_msg(struct pcie_port *pp, unsigned int irq, u32 pos)
 	if (pp->ops->get_msi_addr)
 		msi_target = pp->ops->get_msi_addr(pp);
 	else
-		msi_target = virt_to_phys((void *)pp->msi_data);
+		msi_target = (u64)pp->msi_data;
 
 	msg.address_lo = (u32)(msi_target & 0xffffffff);
 	msg.address_hi = (u32)(msi_target >> 32 & 0xffffffff);
diff --git a/drivers/pci/dwc/pcie-designware.h b/drivers/pci/dwc/pcie-designware.h
index e5d9d77b778e..ecdede68522a 100644
--- a/drivers/pci/dwc/pcie-designware.h
+++ b/drivers/pci/dwc/pcie-designware.h
@@ -14,6 +14,7 @@
 #ifndef _PCIE_DESIGNWARE_H
 #define _PCIE_DESIGNWARE_H
 
+#include <linux/dma-mapping.h>
 #include <linux/irq.h>
 #include <linux/msi.h>
 #include <linux/pci.h>
@@ -168,7 +169,7 @@ struct pcie_port {
 	const struct dw_pcie_host_ops *ops;
 	int			msi_irq;
 	struct irq_domain	*irq_domain;
-	unsigned long		msi_data;
+	dma_addr_t		msi_data;
 	DECLARE_BITMAP(msi_irq_in_use, MAX_MSI_IRQS);
 };
 
-- 
2.14.2

[PATCH v5 02/18] PCI: designware-ep: dw_pcie_ep_set_msi() should only set MMC bits

[Niklas Cassel]

Previously, dw_pcie_ep_set_msi() wrote all bits in the Message Control
register, thus overwriting the PCI_MSI_FLAGS_64BIT bit.
By clearing the PCI_MSI_FLAGS_64BIT bit, we break MSI
on systems where the RC has set a 64 bit MSI address.
Fix dw_pcie_ep_set_msi() so that it only sets MMC bits.

Signed-off-by: Niklas Cassel <niklas.cassel@axis.com>
---
 drivers/pci/dwc/pcie-designware-ep.c | 4 +++-
 drivers/pci/dwc/pcie-designware.h    | 1 +
 2 files changed, 4 insertions(+), 1 deletion(-)

diff --git a/drivers/pci/dwc/pcie-designware-ep.c b/drivers/pci/dwc/pcie-designware-ep.c
index d53d5f168363..c92ab87fd660 100644
--- a/drivers/pci/dwc/pcie-designware-ep.c
+++ b/drivers/pci/dwc/pcie-designware-ep.c
@@ -220,7 +220,9 @@ static int dw_pcie_ep_set_msi(struct pci_epc *epc, u8 encode_int)
 	struct dw_pcie_ep *ep = epc_get_drvdata(epc);
 	struct dw_pcie *pci = to_dw_pcie_from_ep(ep);
 
-	val = (encode_int << MSI_CAP_MMC_SHIFT);
+	val = dw_pcie_readw_dbi(pci, MSI_MESSAGE_CONTROL);
+	val &= ~MSI_CAP_MMC_MASK;
+	val |= (encode_int << MSI_CAP_MMC_SHIFT) & MSI_CAP_MMC_MASK;
 	dw_pcie_writew_dbi(pci, MSI_MESSAGE_CONTROL, val);
 
 	return 0;
diff --git a/drivers/pci/dwc/pcie-designware.h b/drivers/pci/dwc/pcie-designware.h
index ecdede68522a..9aaf0cd04dd6 100644
--- a/drivers/pci/dwc/pcie-designware.h
+++ b/drivers/pci/dwc/pcie-designware.h
@@ -101,6 +101,7 @@
 
 #define MSI_MESSAGE_CONTROL		0x52
 #define MSI_CAP_MMC_SHIFT		1
+#define MSI_CAP_MMC_MASK		(7 << MSI_CAP_MMC_SHIFT)
 #define MSI_CAP_MME_SHIFT		4
 #define MSI_CAP_MME_MASK		(7 << MSI_CAP_MME_SHIFT)
 #define MSI_MESSAGE_ADDR_L32		0x54
-- 
2.14.2

[PATCH v5 03/18] PCI: designware-ep: Read-only registers need DBI_RO_WR_EN to be writable

[Niklas Cassel]

Certain registers that pcie-designware-ep tries to write to are read-only
registers. However, these registers can become read/write if we first
enable the DBI_RO_WR_EN bit. Set/unset the DBI_RO_WR_EN bit before/after
writing these registers.

Signed-off-by: Niklas Cassel <niklas.cassel@axis.com>
---
 drivers/pci/dwc/pcie-designware-ep.c | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/drivers/pci/dwc/pcie-designware-ep.c b/drivers/pci/dwc/pcie-designware-ep.c
index c92ab87fd660..3fb34be99715 100644
--- a/drivers/pci/dwc/pcie-designware-ep.c
+++ b/drivers/pci/dwc/pcie-designware-ep.c
@@ -35,8 +35,10 @@ static void dw_pcie_ep_reset_bar(struct dw_pcie *pci, enum pci_barno bar)
 	u32 reg;
 
 	reg = PCI_BASE_ADDRESS_0 + (4 * bar);
+	dw_pcie_dbi_ro_wr_en(pci);
 	dw_pcie_writel_dbi2(pci, reg, 0x0);
 	dw_pcie_writel_dbi(pci, reg, 0x0);
+	dw_pcie_dbi_ro_wr_dis(pci);
 }
 
 static int dw_pcie_ep_write_header(struct pci_epc *epc,
@@ -45,6 +47,7 @@ static int dw_pcie_ep_write_header(struct pci_epc *epc,
 	struct dw_pcie_ep *ep = epc_get_drvdata(epc);
 	struct dw_pcie *pci = to_dw_pcie_from_ep(ep);
 
+	dw_pcie_dbi_ro_wr_en(pci);
 	dw_pcie_writew_dbi(pci, PCI_VENDOR_ID, hdr->vendorid);
 	dw_pcie_writew_dbi(pci, PCI_DEVICE_ID, hdr->deviceid);
 	dw_pcie_writeb_dbi(pci, PCI_REVISION_ID, hdr->revid);
@@ -58,6 +61,7 @@ static int dw_pcie_ep_write_header(struct pci_epc *epc,
 	dw_pcie_writew_dbi(pci, PCI_SUBSYSTEM_ID, hdr->subsys_id);
 	dw_pcie_writeb_dbi(pci, PCI_INTERRUPT_PIN,
 			   hdr->interrupt_pin);
+	dw_pcie_dbi_ro_wr_dis(pci);
 
 	return 0;
 }
@@ -142,8 +146,10 @@ static int dw_pcie_ep_set_bar(struct pci_epc *epc, enum pci_barno bar,
 	if (ret)
 		return ret;
 
+	dw_pcie_dbi_ro_wr_en(pci);
 	dw_pcie_writel_dbi2(pci, reg, size - 1);
 	dw_pcie_writel_dbi(pci, reg, flags);
+	dw_pcie_dbi_ro_wr_dis(pci);
 
 	return 0;
 }
@@ -223,7 +229,9 @@ static int dw_pcie_ep_set_msi(struct pci_epc *epc, u8 encode_int)
 	val = dw_pcie_readw_dbi(pci, MSI_MESSAGE_CONTROL);
 	val &= ~MSI_CAP_MMC_MASK;
 	val |= (encode_int << MSI_CAP_MMC_SHIFT) & MSI_CAP_MMC_MASK;
+	dw_pcie_dbi_ro_wr_en(pci);
 	dw_pcie_writew_dbi(pci, MSI_MESSAGE_CONTROL, val);
+	dw_pcie_dbi_ro_wr_dis(pci);
 
 	return 0;
 }
-- 
2.14.2

[PATCH v5 04/18] PCI: designware-ep: Pre-allocate memory for MSI in dw_pcie_ep_init

[Niklas Cassel]

Certain SoCs need to map the MSI address in raise_irq.
To map an address, you first need to call pci_epc_mem_alloc_addr(),
however, pci_epc_mem_alloc_addr() calls ioremap() (which can sleep).

Since raise_irq is only called from atomic context, we can't call
pci_epc_mem_alloc_addr() from raise_irq.

Pre-allocate a page in dw_pcie_ep_init(), so that this page can later
be used to map/unmap the MSI address in raise_irq.

Signed-off-by: Niklas Cassel <niklas.cassel@axis.com>
---
 drivers/pci/dwc/pcie-designware-ep.c | 10 ++++++++++
 drivers/pci/dwc/pcie-designware.h    |  2 ++
 2 files changed, 12 insertions(+)

diff --git a/drivers/pci/dwc/pcie-designware-ep.c b/drivers/pci/dwc/pcie-designware-ep.c
index 3fb34be99715..8d8019cdff2a 100644
--- a/drivers/pci/dwc/pcie-designware-ep.c
+++ b/drivers/pci/dwc/pcie-designware-ep.c
@@ -286,6 +286,9 @@ void dw_pcie_ep_exit(struct dw_pcie_ep *ep)
 {
 	struct pci_epc *epc = ep->epc;
 
+	pci_epc_mem_free_addr(epc, ep->msi_mem_phys, ep->msi_mem,
+			      epc->mem->page_size);
+
 	pci_epc_mem_exit(epc);
 }
 
@@ -341,6 +344,13 @@ int dw_pcie_ep_init(struct dw_pcie_ep *ep)
 		return ret;
 	}
 
+	ep->msi_mem = pci_epc_mem_alloc_addr(epc, &ep->msi_mem_phys,
+					     epc->mem->page_size);
+	if (!ep->msi_mem) {
+		dev_err(dev, "Failed to reserve memory for MSI\n");
+		return -ENOMEM;
+	}
+
 	ep->epc = epc;
 	epc_set_drvdata(epc, ep);
 	dw_pcie_setup(pci);
diff --git a/drivers/pci/dwc/pcie-designware.h b/drivers/pci/dwc/pcie-designware.h
index 9aaf0cd04dd6..5a1da459eda5 100644
--- a/drivers/pci/dwc/pcie-designware.h
+++ b/drivers/pci/dwc/pcie-designware.h
@@ -198,6 +198,8 @@ struct dw_pcie_ep {
 	unsigned long		ob_window_map;
 	u32			num_ib_windows;
 	u32			num_ob_windows;
+	void __iomem		*msi_mem;
+	phys_addr_t		msi_mem_phys;
 };
 
 struct dw_pcie_ops {
-- 
2.14.2

[PATCH v5 05/18] PCI: designware-ep: Remove static keyword from dw_pcie_ep_reset_bar()

[Niklas Cassel]

Remove the static keyword from dw_pcie_ep_reset_bar() so that
pci-dra7xx.c does not need its own copy of dw_pcie_ep_reset_bar().

Signed-off-by: Niklas Cassel <niklas.cassel@axis.com>
Acked-by: Kishon Vijay Abraham I <kishon@ti.com>
---
 drivers/pci/dwc/pci-dra7xx.c         | 9 ---------
 drivers/pci/dwc/pcie-designware-ep.c | 2 +-
 drivers/pci/dwc/pcie-designware.h    | 5 +++++
 3 files changed, 6 insertions(+), 10 deletions(-)

diff --git a/drivers/pci/dwc/pci-dra7xx.c b/drivers/pci/dwc/pci-dra7xx.c
index e77a4ceed74c..d330b7d86ce3 100644
--- a/drivers/pci/dwc/pci-dra7xx.c
+++ b/drivers/pci/dwc/pci-dra7xx.c
@@ -337,15 +337,6 @@ static irqreturn_t dra7xx_pcie_irq_handler(int irq, void *arg)
 	return IRQ_HANDLED;
 }
 
-static void dw_pcie_ep_reset_bar(struct dw_pcie *pci, enum pci_barno bar)
-{
-	u32 reg;
-
-	reg = PCI_BASE_ADDRESS_0 + (4 * bar);
-	dw_pcie_writel_dbi2(pci, reg, 0x0);
-	dw_pcie_writel_dbi(pci, reg, 0x0);
-}
-
 static void dra7xx_pcie_ep_init(struct dw_pcie_ep *ep)
 {
 	struct dw_pcie *pci = to_dw_pcie_from_ep(ep);
diff --git a/drivers/pci/dwc/pcie-designware-ep.c b/drivers/pci/dwc/pcie-designware-ep.c
index 8d8019cdff2a..700ed2f4becf 100644
--- a/drivers/pci/dwc/pcie-designware-ep.c
+++ b/drivers/pci/dwc/pcie-designware-ep.c
@@ -30,7 +30,7 @@ void dw_pcie_ep_linkup(struct dw_pcie_ep *ep)
 	pci_epc_linkup(epc);
 }
 
-static void dw_pcie_ep_reset_bar(struct dw_pcie *pci, enum pci_barno bar)
+void dw_pcie_ep_reset_bar(struct dw_pcie *pci, enum pci_barno bar)
 {
 	u32 reg;
 
diff --git a/drivers/pci/dwc/pcie-designware.h b/drivers/pci/dwc/pcie-designware.h
index 5a1da459eda5..37dfad8d003f 100644
--- a/drivers/pci/dwc/pcie-designware.h
+++ b/drivers/pci/dwc/pcie-designware.h
@@ -338,6 +338,7 @@ static inline int dw_pcie_host_init(struct pcie_port *pp)
 void dw_pcie_ep_linkup(struct dw_pcie_ep *ep);
 int dw_pcie_ep_init(struct dw_pcie_ep *ep);
 void dw_pcie_ep_exit(struct dw_pcie_ep *ep);
+void dw_pcie_ep_reset_bar(struct dw_pcie *pci, enum pci_barno bar);
 #else
 static inline void dw_pcie_ep_linkup(struct dw_pcie_ep *ep)
 {
@@ -351,5 +352,9 @@ static inline int dw_pcie_ep_init(struct dw_pcie_ep *ep)
 static inline void dw_pcie_ep_exit(struct dw_pcie_ep *ep)
 {
 }
+
+static inline void dw_pcie_ep_reset_bar(struct dw_pcie *pci, enum pci_barno bar)
+{
+}
 #endif
 #endif /* _PCIE_DESIGNWARE_H */
-- 
2.14.2

[PATCH v5 06/18] PCI: designware-ep: Add generic function for raising MSI irq

[Niklas Cassel]

Add a generic function for raising MSI irqs that can be used by all
DWC based controllers.

Note that certain controllers, like DRA7xx, have a special convenience
register for raising MSI irqs that doesn't require you to explicitly map
the MSI address. Therefore, it is likely that certain drivers will
not use this generic function, even if they can.

Signed-off-by: Niklas Cassel <niklas.cassel@axis.com>
---
 drivers/pci/dwc/pcie-designware-ep.c | 35 +++++++++++++++++++++++++++++++++++
 drivers/pci/dwc/pcie-designware.h    |  9 +++++++++
 2 files changed, 44 insertions(+)

diff --git a/drivers/pci/dwc/pcie-designware-ep.c b/drivers/pci/dwc/pcie-designware-ep.c
index 700ed2f4becf..c5aa1cac5041 100644
--- a/drivers/pci/dwc/pcie-designware-ep.c
+++ b/drivers/pci/dwc/pcie-designware-ep.c
@@ -282,6 +282,41 @@ static const struct pci_epc_ops epc_ops = {
 	.stop			= dw_pcie_ep_stop,
 };
 
+int dw_pcie_ep_raise_msi_irq(struct dw_pcie_ep *ep,
+			     u8 interrupt_num)
+{
+	struct dw_pcie *pci = to_dw_pcie_from_ep(ep);
+	struct pci_epc *epc = ep->epc;
+	u16 msg_ctrl, msg_data;
+	u32 msg_addr_lower, msg_addr_upper;
+	u64 msg_addr;
+	bool has_upper;
+	int ret;
+
+	/* Raise MSI per the PCI Local Bus Specification Revision 3.0, 6.8.1. */
+	msg_ctrl = dw_pcie_readw_dbi(pci, MSI_MESSAGE_CONTROL);
+	has_upper = !!(msg_ctrl & PCI_MSI_FLAGS_64BIT);
+	msg_addr_lower = dw_pcie_readl_dbi(pci, MSI_MESSAGE_ADDR_L32);
+	if (has_upper) {
+		msg_addr_upper = dw_pcie_readl_dbi(pci, MSI_MESSAGE_ADDR_U32);
+		msg_data = dw_pcie_readw_dbi(pci, MSI_MESSAGE_DATA_64);
+	} else {
+		msg_addr_upper = 0;
+		msg_data = dw_pcie_readw_dbi(pci, MSI_MESSAGE_DATA_32);
+	}
+	msg_addr = ((u64) msg_addr_upper) << 32 | msg_addr_lower;
+	ret = dw_pcie_ep_map_addr(epc, ep->msi_mem_phys, msg_addr,
+				  epc->mem->page_size);
+	if (ret)
+		return ret;
+
+	writel(msg_data | (interrupt_num - 1), ep->msi_mem);
+
+	dw_pcie_ep_unmap_addr(epc, ep->msi_mem_phys);
+
+	return 0;
+}
+
 void dw_pcie_ep_exit(struct dw_pcie_ep *ep)
 {
 	struct pci_epc *epc = ep->epc;
diff --git a/drivers/pci/dwc/pcie-designware.h b/drivers/pci/dwc/pcie-designware.h
index 37dfad8d003f..24edac035160 100644
--- a/drivers/pci/dwc/pcie-designware.h
+++ b/drivers/pci/dwc/pcie-designware.h
@@ -106,6 +106,8 @@
 #define MSI_CAP_MME_MASK		(7 << MSI_CAP_MME_SHIFT)
 #define MSI_MESSAGE_ADDR_L32		0x54
 #define MSI_MESSAGE_ADDR_U32		0x58
+#define MSI_MESSAGE_DATA_32		0x58
+#define MSI_MESSAGE_DATA_64		0x5C
 
 /*
  * Maximum number of MSI IRQs can be 256 per controller. But keep
@@ -338,6 +340,7 @@ static inline int dw_pcie_host_init(struct pcie_port *pp)
 void dw_pcie_ep_linkup(struct dw_pcie_ep *ep);
 int dw_pcie_ep_init(struct dw_pcie_ep *ep);
 void dw_pcie_ep_exit(struct dw_pcie_ep *ep);
+int dw_pcie_ep_raise_msi_irq(struct dw_pcie_ep *ep, u8 interrupt_num);
 void dw_pcie_ep_reset_bar(struct dw_pcie *pci, enum pci_barno bar);
 #else
 static inline void dw_pcie_ep_linkup(struct dw_pcie_ep *ep)
@@ -353,6 +356,12 @@ static inline void dw_pcie_ep_exit(struct dw_pcie_ep *ep)
 {
 }
 
+static inline int dw_pcie_ep_raise_msi_irq(struct dw_pcie_ep *ep,
+					   u8 interrupt_num)
+{
+	return 0;
+}
+
 static inline void dw_pcie_ep_reset_bar(struct dw_pcie *pci, enum pci_barno bar)
 {
 }
-- 
2.14.2

[PATCH v5 07/18] PCI: dwc: dra7xx: Refactor Kconfig and Makefile handling for host/ep mode

[Niklas Cassel]

Refactor the Kconfig and Makefile handling for host/ep mode, since
the previous handling was a bit unorthodox and would have been a bit
bloated once more DWC based controllers added support for ep mode.

Signed-off-by: Niklas Cassel <niklas.cassel@axis.com>
Acked-by: Kishon Vijay Abraham I <kishon@ti.com>
---
 drivers/pci/dwc/Kconfig  | 45 ++++++++++++++++++++++-----------------------
 drivers/pci/dwc/Makefile |  4 +---
 2 files changed, 23 insertions(+), 26 deletions(-)

diff --git a/drivers/pci/dwc/Kconfig b/drivers/pci/dwc/Kconfig
index 113e09440f85..3954353e3e2e 100644
--- a/drivers/pci/dwc/Kconfig
+++ b/drivers/pci/dwc/Kconfig
@@ -15,39 +15,38 @@ config PCIE_DW_EP
 	select PCIE_DW
 
 config PCI_DRA7XX
-	bool "TI DRA7xx PCIe controller"
-	depends on SOC_DRA7XX || COMPILE_TEST
-	depends on (PCI && PCI_MSI_IRQ_DOMAIN) || PCI_ENDPOINT
-	depends on OF && HAS_IOMEM && TI_PIPE3
-	help
-	 Enables support for the PCIe controller in the DRA7xx SoC. There
-	 are two instances of PCIe controller in DRA7xx. This controller can
-	 work either as EP or RC. In order to enable host-specific features
-	 PCI_DRA7XX_HOST must be selected and in order to enable device-
-	 specific features PCI_DRA7XX_EP must be selected. This uses
-	 the DesignWare core.
-
-if PCI_DRA7XX
+	bool
 
 config PCI_DRA7XX_HOST
-	bool "PCI DRA7xx Host Mode"
-	depends on PCI
-	depends on PCI_MSI_IRQ_DOMAIN
+	bool "TI DRA7xx PCIe controller Host Mode"
+	depends on SOC_DRA7XX || COMPILE_TEST
+	depends on PCI && PCI_MSI_IRQ_DOMAIN
+	depends on OF && HAS_IOMEM && TI_PIPE3
 	select PCIE_DW_HOST
+	select PCI_DRA7XX
 	default y
 	help
-	 Enables support for the PCIe controller in the DRA7xx SoC to work in
-	 host mode.
+	  Enables support for the PCIe controller in the DRA7xx SoC to work in
+	  host mode. There are two instances of PCIe controller in DRA7xx.
+	  This controller can work either as EP or RC. In order to enable
+	  host-specific features PCI_DRA7XX_HOST must be selected and in order
+	  to enable device-specific features PCI_DRA7XX_EP must be selected.
+	  This uses the DesignWare core.
 
 config PCI_DRA7XX_EP
-	bool "PCI DRA7xx Endpoint Mode"
+	bool "TI DRA7xx PCIe controller Endpoint Mode"
+	depends on SOC_DRA7XX || COMPILE_TEST
 	depends on PCI_ENDPOINT
+	depends on OF && HAS_IOMEM && TI_PIPE3
 	select PCIE_DW_EP
+	select PCI_DRA7XX
 	help
-	 Enables support for the PCIe controller in the DRA7xx SoC to work in
-	 endpoint mode.
-
-endif
+	  Enables support for the PCIe controller in the DRA7xx SoC to work in
+	  endpoint mode. There are two instances of PCIe controller in DRA7xx.
+	  This controller can work either as EP or RC. In order to enable
+	  host-specific features PCI_DRA7XX_HOST must be selected and in order
+	  to enable device-specific features PCI_DRA7XX_EP must be selected.
+	  This uses the DesignWare core.
 
 config PCIE_DW_PLAT
 	bool "Platform bus based DesignWare PCIe Controller"
diff --git a/drivers/pci/dwc/Makefile b/drivers/pci/dwc/Makefile
index 41ba499c96ee..a9d8a6fb48e3 100644
--- a/drivers/pci/dwc/Makefile
+++ b/drivers/pci/dwc/Makefile
@@ -3,9 +3,7 @@ obj-$(CONFIG_PCIE_DW) += pcie-designware.o
 obj-$(CONFIG_PCIE_DW_HOST) += pcie-designware-host.o
 obj-$(CONFIG_PCIE_DW_EP) += pcie-designware-ep.o
 obj-$(CONFIG_PCIE_DW_PLAT) += pcie-designware-plat.o
-ifneq ($(filter y,$(CONFIG_PCI_DRA7XX_HOST) $(CONFIG_PCI_DRA7XX_EP)),)
-        obj-$(CONFIG_PCI_DRA7XX) += pci-dra7xx.o
-endif
+obj-$(CONFIG_PCI_DRA7XX) += pci-dra7xx.o
 obj-$(CONFIG_PCI_EXYNOS) += pci-exynos.o
 obj-$(CONFIG_PCI_IMX6) += pci-imx6.o
 obj-$(CONFIG_PCIE_SPEAR13XX) += pcie-spear13xx.o
-- 
2.14.2

[PATCH v5 08/18] PCI: dwc: dra7xx: Assign pp->ops in dra7xx_add_pcie_port() rather than in probe

[Niklas Cassel]

Assign pp->ops in *_add_pcie_port() to match how it is done in other
drivers like exynos, imx7, keystone, armada8k, artpec6, designware-plat,
hisi, kirin and spear13xx.

This is probably a remainder since when dev and ops were assigned as
members to pp. Since we now assign them as members to struct dw_pcie,
the pp->ops assignment should definitely be in dra7xx_add_pcie_port().

This is done so that the compiler (in a later commit) can remove more
code when enabling only one of the two supported modes (host/ep) in
the dra7xx driver.

Signed-off-by: Niklas Cassel <niklas.cassel@axis.com>
Acked-by: Kishon Vijay Abraham I <kishon@ti.com>
---
 drivers/pci/dwc/pci-dra7xx.c | 6 ++----
 1 file changed, 2 insertions(+), 4 deletions(-)

diff --git a/drivers/pci/dwc/pci-dra7xx.c b/drivers/pci/dwc/pci-dra7xx.c
index d330b7d86ce3..07c74ae3614e 100644
--- a/drivers/pci/dwc/pci-dra7xx.c
+++ b/drivers/pci/dwc/pci-dra7xx.c
@@ -461,6 +461,8 @@ static int __init dra7xx_add_pcie_port(struct dra7xx_pcie *dra7xx,
 	if (!pci->dbi_base)
 		return -ENOMEM;
 
+	pp->ops = &dra7xx_pcie_host_ops;
+
 	ret = dw_pcie_host_init(pp);
 	if (ret) {
 		dev_err(dev, "failed to initialize host\n");
@@ -590,7 +592,6 @@ static int __init dra7xx_pcie_probe(struct platform_device *pdev)
 	void __iomem *base;
 	struct resource *res;
 	struct dw_pcie *pci;
-	struct pcie_port *pp;
 	struct dra7xx_pcie *dra7xx;
 	struct device *dev = &pdev->dev;
 	struct device_node *np = dev->of_node;
@@ -618,9 +619,6 @@ static int __init dra7xx_pcie_probe(struct platform_device *pdev)
 	pci->dev = dev;
 	pci->ops = &dw_pcie_ops;
 
-	pp = &pci->pp;
-	pp->ops = &dra7xx_pcie_host_ops;
-
 	irq = platform_get_irq(pdev, 0);
 	if (irq < 0) {
 		dev_err(dev, "missing IRQ resource: %d\n", irq);
-- 
2.14.2

[PATCH v5 09/18] PCI: dwc: dra7xx: Help compiler to remove unused code

[Niklas Cassel]

The dra7xx driver supports both host and ep mode.
When enabling support for only one of the modes, help the compiler
to remove code for the mode that we have not enabled in the driver.

By adding if (!IS_ENABLED(CONFIG_PCI_DRA7XX_HOST)) return -ENODEV;
anything after that statement will get silently dropped by the compiler,
including static functions and structures that are referenced indirectly
from there.

Suggested-by: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Niklas Cassel <niklas.cassel@axis.com>
---
 drivers/pci/dwc/pci-dra7xx.c | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/drivers/pci/dwc/pci-dra7xx.c b/drivers/pci/dwc/pci-dra7xx.c
index 07c74ae3614e..224ff8affdce 100644
--- a/drivers/pci/dwc/pci-dra7xx.c
+++ b/drivers/pci/dwc/pci-dra7xx.c
@@ -694,6 +694,11 @@ static int __init dra7xx_pcie_probe(struct platform_device *pdev)
 
 	switch (mode) {
 	case DW_PCIE_RC_TYPE:
+		if (!IS_ENABLED(CONFIG_PCI_DRA7XX_HOST)) {
+			ret = -ENODEV;
+			goto err_gpio;
+		}
+
 		dra7xx_pcie_writel(dra7xx, PCIECTRL_TI_CONF_DEVICE_TYPE,
 				   DEVICE_TYPE_RC);
 		ret = dra7xx_add_pcie_port(dra7xx, pdev);
@@ -701,6 +706,11 @@ static int __init dra7xx_pcie_probe(struct platform_device *pdev)
 			goto err_gpio;
 		break;
 	case DW_PCIE_EP_TYPE:
+		if (!IS_ENABLED(CONFIG_PCI_DRA7XX_EP)) {
+			ret = -ENODEV;
+			goto err_gpio;
+		}
+
 		dra7xx_pcie_writel(dra7xx, PCIECTRL_TI_CONF_DEVICE_TYPE,
 				   DEVICE_TYPE_EP);
 
-- 
2.14.2

[PATCH v5 10/18] PCI: dwc: artpec6: Remove unused defines

[Niklas Cassel]

Commit b015b37e6693 ("PCI: artpec6: Stop enabling writes to
DBI read-only registers") removed the only write using these
defines, but it did not remove the defines.
Remove the defines since they are now unused.

Signed-off-by: Niklas Cassel <niklas.cassel@axis.com>
---
 drivers/pci/dwc/pcie-artpec6.c | 3 ---
 1 file changed, 3 deletions(-)

diff --git a/drivers/pci/dwc/pcie-artpec6.c b/drivers/pci/dwc/pcie-artpec6.c
index 6653619db6a1..4b8ef266dc2f 100644
--- a/drivers/pci/dwc/pcie-artpec6.c
+++ b/drivers/pci/dwc/pcie-artpec6.c
@@ -37,9 +37,6 @@ struct artpec6_pcie {
 #define PCIE_PHY_DEBUG_R0		(PL_OFFSET + 0x28)
 #define PCIE_PHY_DEBUG_R1		(PL_OFFSET + 0x2c)
 
-#define MISC_CONTROL_1_OFF		(PL_OFFSET + 0x1bc)
-#define  DBI_RO_WR_EN			1
-
 /* ARTPEC-6 specific registers */
 #define PCIECFG				0x18
 #define  PCIECFG_DBG_OEN		(1 << 24)
-- 
2.14.2

[PATCH v5 11/18] PCI: dwc: artpec6: Use BIT and GENMASK macros

[Niklas Cassel]

Use BIT and GENMASK macros to improve readability.

Signed-off-by: Niklas Cassel <niklas.cassel@axis.com>
---
 drivers/pci/dwc/pcie-artpec6.c | 34 +++++++++++++++++-----------------
 1 file changed, 17 insertions(+), 17 deletions(-)

diff --git a/drivers/pci/dwc/pcie-artpec6.c b/drivers/pci/dwc/pcie-artpec6.c
index 4b8ef266dc2f..18075e0fab80 100644
--- a/drivers/pci/dwc/pcie-artpec6.c
+++ b/drivers/pci/dwc/pcie-artpec6.c
@@ -39,28 +39,28 @@ struct artpec6_pcie {
 
 /* ARTPEC-6 specific registers */
 #define PCIECFG				0x18
-#define  PCIECFG_DBG_OEN		(1 << 24)
-#define  PCIECFG_CORE_RESET_REQ		(1 << 21)
-#define  PCIECFG_LTSSM_ENABLE		(1 << 20)
-#define  PCIECFG_CLKREQ_B		(1 << 11)
-#define  PCIECFG_REFCLK_ENABLE		(1 << 10)
-#define  PCIECFG_PLL_ENABLE		(1 << 9)
-#define  PCIECFG_PCLK_ENABLE		(1 << 8)
-#define  PCIECFG_RISRCREN		(1 << 4)
-#define  PCIECFG_MODE_TX_DRV_EN		(1 << 3)
-#define  PCIECFG_CISRREN		(1 << 2)
-#define  PCIECFG_MACRO_ENABLE		(1 << 0)
+#define  PCIECFG_DBG_OEN		BIT(24)
+#define  PCIECFG_CORE_RESET_REQ		BIT(21)
+#define  PCIECFG_LTSSM_ENABLE		BIT(20)
+#define  PCIECFG_CLKREQ_B		BIT(11)
+#define  PCIECFG_REFCLK_ENABLE		BIT(10)
+#define  PCIECFG_PLL_ENABLE		BIT(9)
+#define  PCIECFG_PCLK_ENABLE		BIT(8)
+#define  PCIECFG_RISRCREN		BIT(4)
+#define  PCIECFG_MODE_TX_DRV_EN		BIT(3)
+#define  PCIECFG_CISRREN		BIT(2)
+#define  PCIECFG_MACRO_ENABLE		BIT(0)
 
 #define NOCCFG				0x40
-#define NOCCFG_ENABLE_CLK_PCIE		(1 << 4)
-#define NOCCFG_POWER_PCIE_IDLEACK	(1 << 3)
-#define NOCCFG_POWER_PCIE_IDLE		(1 << 2)
-#define NOCCFG_POWER_PCIE_IDLEREQ	(1 << 1)
+#define  NOCCFG_ENABLE_CLK_PCIE		BIT(4)
+#define  NOCCFG_POWER_PCIE_IDLEACK	BIT(3)
+#define  NOCCFG_POWER_PCIE_IDLE		BIT(2)
+#define  NOCCFG_POWER_PCIE_IDLEREQ	BIT(1)
 
 #define PHY_STATUS			0x118
-#define PHY_COSPLLLOCK			(1 << 0)
+#define  PHY_COSPLLLOCK			BIT(0)
 
-#define ARTPEC6_CPU_TO_BUS_ADDR		0x0fffffff
+#define ARTPEC6_CPU_TO_BUS_ADDR		GENMASK(27, 0)
 
 static u32 artpec6_pcie_readl(struct artpec6_pcie *artpec6_pcie, u32 offset)
 {
-- 
2.14.2

[PATCH v5 12/18] PCI: dwc: artpec6: Split artpec6_pcie_establish_link() into smaller functions

[Niklas Cassel]

Split artpec6_pcie_establish_link() into smaller functions
to better match other drivers such as dra7xx and imx6.
This is also done to prepare for endpoint mode support.

Signed-off-by: Niklas Cassel <niklas.cassel@axis.com>
---
 drivers/pci/dwc/pcie-artpec6.c | 55 ++++++++++++++++++++++--------------------
 1 file changed, 29 insertions(+), 26 deletions(-)

diff --git a/drivers/pci/dwc/pcie-artpec6.c b/drivers/pci/dwc/pcie-artpec6.c
index 18075e0fab80..b2783b475c2a 100644
--- a/drivers/pci/dwc/pcie-artpec6.c
+++ b/drivers/pci/dwc/pcie-artpec6.c
@@ -34,8 +34,6 @@ struct artpec6_pcie {
 
 /* PCIe Port Logic registers (memory-mapped) */
 #define PL_OFFSET			0x700
-#define PCIE_PHY_DEBUG_R0		(PL_OFFSET + 0x28)
-#define PCIE_PHY_DEBUG_R1		(PL_OFFSET + 0x2c)
 
 /* ARTPEC-6 specific registers */
 #define PCIECFG				0x18
@@ -80,18 +78,23 @@ static u64 artpec6_pcie_cpu_addr_fixup(u64 pci_addr)
 	return pci_addr & ARTPEC6_CPU_TO_BUS_ADDR;
 }
 
-static int artpec6_pcie_establish_link(struct artpec6_pcie *artpec6_pcie)
+static int artpec6_pcie_establish_link(struct dw_pcie *pci)
 {
-	struct dw_pcie *pci = artpec6_pcie->pci;
-	struct pcie_port *pp = &pci->pp;
+	struct artpec6_pcie *artpec6_pcie = to_artpec6_pcie(pci);
 	u32 val;
-	unsigned int retries;
 
-	/* Hold DW core in reset */
 	val = artpec6_pcie_readl(artpec6_pcie, PCIECFG);
-	val |= PCIECFG_CORE_RESET_REQ;
+	val |= PCIECFG_LTSSM_ENABLE;
 	artpec6_pcie_writel(artpec6_pcie, PCIECFG, val);
 
+	return 0;
+}
+
+static void artpec6_pcie_init_phy(struct artpec6_pcie *artpec6_pcie)
+{
+	u32 val;
+	unsigned int retries;
+
 	val = artpec6_pcie_readl(artpec6_pcie, PCIECFG);
 	val |=  PCIECFG_RISRCREN |	/* Receiver term. 50 Ohm */
 		PCIECFG_MODE_TX_DRV_EN |
@@ -131,30 +134,25 @@ static int artpec6_pcie_establish_link(struct artpec6_pcie *artpec6_pcie)
 		val = readl(artpec6_pcie->phy_base + PHY_STATUS);
 		retries--;
 	} while (retries && !(val & PHY_COSPLLLOCK));
+}
+
+static void artpec6_pcie_assert_core_reset(struct artpec6_pcie *artpec6_pcie)
+{
+	u32 val;
 
-	/* Take DW core out of reset */
 	val = artpec6_pcie_readl(artpec6_pcie, PCIECFG);
-	val &= ~PCIECFG_CORE_RESET_REQ;
+	val |= PCIECFG_CORE_RESET_REQ;
 	artpec6_pcie_writel(artpec6_pcie, PCIECFG, val);
-	usleep_range(100, 200);
+}
 
-	/* setup root complex */
-	dw_pcie_setup_rc(pp);
+static void artpec6_pcie_deassert_core_reset(struct artpec6_pcie *artpec6_pcie)
+{
+	u32 val;
 
-	/* assert LTSSM enable */
 	val = artpec6_pcie_readl(artpec6_pcie, PCIECFG);
-	val |= PCIECFG_LTSSM_ENABLE;
+	val &= ~PCIECFG_CORE_RESET_REQ;
 	artpec6_pcie_writel(artpec6_pcie, PCIECFG, val);
-
-	/* check if the link is up or not */
-	if (!dw_pcie_wait_for_link(pci))
-		return 0;
-
-	dev_dbg(pci->dev, "DEBUG_R0: 0x%08x, DEBUG_R1: 0x%08x\n",
-		dw_pcie_readl_dbi(pci, PCIE_PHY_DEBUG_R0),
-		dw_pcie_readl_dbi(pci, PCIE_PHY_DEBUG_R1));
-
-	return -ETIMEDOUT;
+	usleep_range(100, 200);
 }
 
 static void artpec6_pcie_enable_interrupts(struct artpec6_pcie *artpec6_pcie)
@@ -171,7 +169,12 @@ static int artpec6_pcie_host_init(struct pcie_port *pp)
 	struct dw_pcie *pci = to_dw_pcie_from_pp(pp);
 	struct artpec6_pcie *artpec6_pcie = to_artpec6_pcie(pci);
 
-	artpec6_pcie_establish_link(artpec6_pcie);
+	artpec6_pcie_assert_core_reset(artpec6_pcie);
+	artpec6_pcie_init_phy(artpec6_pcie);
+	artpec6_pcie_deassert_core_reset(artpec6_pcie);
+	dw_pcie_setup_rc(pp);
+	artpec6_pcie_establish_link(pci);
+	dw_pcie_wait_for_link(pci);
 	artpec6_pcie_enable_interrupts(artpec6_pcie);
 
 	return 0;
-- 
2.14.2

[PATCH v5 13/18] bindings: PCI: artpec: Add support for endpoint mode

[Niklas Cassel]

The PCIe controller integrated in ARTPEC-6 SoCs is capable of operating in
endpoint mode. Add endpoint mode support to the artpec6 driver.

Signed-off-by: Niklas Cassel <niklas.cassel@axis.com>
Acked-by: Rob Herring <robh@kernel.org>
---
 Documentation/devicetree/bindings/pci/axis,artpec6-pcie.txt | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/Documentation/devicetree/bindings/pci/axis,artpec6-pcie.txt b/Documentation/devicetree/bindings/pci/axis,artpec6-pcie.txt
index 4e4aee4439ea..33eef7ae5a23 100644
--- a/Documentation/devicetree/bindings/pci/axis,artpec6-pcie.txt
+++ b/Documentation/devicetree/bindings/pci/axis,artpec6-pcie.txt
@@ -4,7 +4,8 @@ This PCIe host controller is based on the Synopsys DesignWare PCIe IP
 and thus inherits all the common properties defined in designware-pcie.txt.
 
 Required properties:
-- compatible: "axis,artpec6-pcie", "snps,dw-pcie"
+- compatible: "axis,artpec6-pcie", "snps,dw-pcie" for ARTPEC-6 in RC mode;
+	      "axis,artpec6-pcie-ep", "snps,dw-pcie" for ARTPEC-6 in EP mode;
 - reg: base addresses and lengths of the PCIe controller (DBI),
 	the PHY controller, and configuration address space.
 - reg-names: Must include the following entries:
-- 
2.14.2

[PATCH v5 13/18] bindings: PCI: artpec: Add support for endpoint mode

[Niklas Cassel]

The PCIe controller integrated in ARTPEC-6 SoCs is capable of operating in
endpoint mode. Add endpoint mode support to the artpec6 driver.

Signed-off-by: Niklas Cassel <niklas.cassel-VrBV9hrLPhE@public.gmane.org>
Acked-by: Rob Herring <robh-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>
---
 Documentation/devicetree/bindings/pci/axis,artpec6-pcie.txt | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/Documentation/devicetree/bindings/pci/axis,artpec6-pcie.txt b/Documentation/devicetree/bindings/pci/axis,artpec6-pcie.txt
index 4e4aee4439ea..33eef7ae5a23 100644
--- a/Documentation/devicetree/bindings/pci/axis,artpec6-pcie.txt
+++ b/Documentation/devicetree/bindings/pci/axis,artpec6-pcie.txt
@@ -4,7 +4,8 @@ This PCIe host controller is based on the Synopsys DesignWare PCIe IP
 and thus inherits all the common properties defined in designware-pcie.txt.
 
 Required properties:
-- compatible: "axis,artpec6-pcie", "snps,dw-pcie"
+- compatible: "axis,artpec6-pcie", "snps,dw-pcie" for ARTPEC-6 in RC mode;
+	      "axis,artpec6-pcie-ep", "snps,dw-pcie" for ARTPEC-6 in EP mode;
 - reg: base addresses and lengths of the PCIe controller (DBI),
 	the PHY controller, and configuration address space.
 - reg-names: Must include the following entries:
-- 
2.14.2

--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v5 14/18] PCI: dwc: artpec6: Add support for endpoint mode

[Niklas Cassel]

The PCIe controller integrated in ARTPEC-6 SoCs is capable of operating in
endpoint mode. Add endpoint mode support to the artpec6 driver.

Signed-off-by: Niklas Cassel <niklas.cassel@axis.com>
---
 drivers/pci/dwc/Kconfig        |  23 +++++--
 drivers/pci/dwc/pcie-artpec6.c | 152 +++++++++++++++++++++++++++++++++++++++--
 2 files changed, 164 insertions(+), 11 deletions(-)

diff --git a/drivers/pci/dwc/Kconfig b/drivers/pci/dwc/Kconfig
index 3954353e3e2e..0fb96c7754de 100644
--- a/drivers/pci/dwc/Kconfig
+++ b/drivers/pci/dwc/Kconfig
@@ -148,15 +148,28 @@ config PCIE_ARMADA_8K
 	  DesignWare core functions to implement the driver.
 
 config PCIE_ARTPEC6
-	bool "Axis ARTPEC-6 PCIe controller"
-	depends on PCI
+	bool
+
+config PCIE_ARTPEC6_HOST
+	bool "Axis ARTPEC-6 PCIe controller Host Mode"
 	depends on MACH_ARTPEC6
-	depends on PCI_MSI_IRQ_DOMAIN
+	depends on PCI && PCI_MSI_IRQ_DOMAIN
 	select PCIEPORTBUS
 	select PCIE_DW_HOST
+	select PCIE_ARTPEC6
+	help
+	  Enables support for the PCIe controller in the ARTPEC-6 SoC to work in
+	  host mode. This uses the DesignWare core.
+
+config PCIE_ARTPEC6_EP
+	bool "Axis ARTPEC-6 PCIe controller Endpoint Mode"
+	depends on MACH_ARTPEC6
+	depends on PCI_ENDPOINT
+	select PCIE_DW_EP
+	select PCIE_ARTPEC6
 	help
-	  Say Y here to enable PCIe controller support on Axis ARTPEC-6
-	  SoCs.  This PCIe controller uses the DesignWare core.
+	  Enables support for the PCIe controller in the ARTPEC-6 SoC to work in
+	  endpoint mode. This uses the DesignWare core.
 
 config PCIE_KIRIN
 	depends on OF && ARM64
diff --git a/drivers/pci/dwc/pcie-artpec6.c b/drivers/pci/dwc/pcie-artpec6.c
index b2783b475c2a..e7de4e4649eb 100644
--- a/drivers/pci/dwc/pcie-artpec6.c
+++ b/drivers/pci/dwc/pcie-artpec6.c
@@ -13,6 +13,7 @@
 #include <linux/delay.h>
 #include <linux/kernel.h>
 #include <linux/init.h>
+#include <linux/of_device.h>
 #include <linux/pci.h>
 #include <linux/platform_device.h>
 #include <linux/resource.h>
@@ -30,8 +31,15 @@ struct artpec6_pcie {
 	struct dw_pcie		*pci;
 	struct regmap		*regmap;	/* DT axis,syscon-pcie */
 	void __iomem		*phy_base;	/* DT phy */
+	enum dw_pcie_device_mode mode;
 };
 
+struct artpec_pcie_of_data {
+	enum dw_pcie_device_mode mode;
+};
+
+static const struct of_device_id artpec6_pcie_of_match[];
+
 /* PCIe Port Logic registers (memory-mapped) */
 #define PL_OFFSET			0x700
 
@@ -40,6 +48,7 @@ struct artpec6_pcie {
 #define  PCIECFG_DBG_OEN		BIT(24)
 #define  PCIECFG_CORE_RESET_REQ		BIT(21)
 #define  PCIECFG_LTSSM_ENABLE		BIT(20)
+#define  PCIECFG_DEVICE_TYPE_MASK	GENMASK(19, 16)
 #define  PCIECFG_CLKREQ_B		BIT(11)
 #define  PCIECFG_REFCLK_ENABLE		BIT(10)
 #define  PCIECFG_PLL_ENABLE		BIT(9)
@@ -90,6 +99,22 @@ static int artpec6_pcie_establish_link(struct dw_pcie *pci)
 	return 0;
 }
 
+static void artpec6_pcie_stop_link(struct dw_pcie *pci)
+{
+	struct artpec6_pcie *artpec6_pcie = to_artpec6_pcie(pci);
+	u32 val;
+
+	val = artpec6_pcie_readl(artpec6_pcie, PCIECFG);
+	val &= ~PCIECFG_LTSSM_ENABLE;
+	artpec6_pcie_writel(artpec6_pcie, PCIECFG, val);
+}
+
+static const struct dw_pcie_ops dw_pcie_ops = {
+	.cpu_addr_fixup = artpec6_pcie_cpu_addr_fixup,
+	.start_link = artpec6_pcie_establish_link,
+	.stop_link = artpec6_pcie_stop_link,
+};
+
 static void artpec6_pcie_init_phy(struct artpec6_pcie *artpec6_pcie)
 {
 	u32 val;
@@ -230,10 +255,76 @@ static int artpec6_add_pcie_port(struct artpec6_pcie *artpec6_pcie,
 	return 0;
 }
 
-static const struct dw_pcie_ops dw_pcie_ops = {
-	.cpu_addr_fixup = artpec6_pcie_cpu_addr_fixup,
+static void artpec6_pcie_ep_init(struct dw_pcie_ep *ep)
+{
+	struct dw_pcie *pci = to_dw_pcie_from_ep(ep);
+	struct artpec6_pcie *artpec6_pcie = to_artpec6_pcie(pci);
+	enum pci_barno bar;
+
+	artpec6_pcie_assert_core_reset(artpec6_pcie);
+	artpec6_pcie_init_phy(artpec6_pcie);
+	artpec6_pcie_deassert_core_reset(artpec6_pcie);
+
+	for (bar = BAR_0; bar <= BAR_5; bar++)
+		dw_pcie_ep_reset_bar(pci, bar);
+}
+
+static int artpec6_pcie_raise_irq(struct dw_pcie_ep *ep,
+				  enum pci_epc_irq_type type, u8 interrupt_num)
+{
+	struct dw_pcie *pci = to_dw_pcie_from_ep(ep);
+
+	switch (type) {
+	case PCI_EPC_IRQ_LEGACY:
+		dev_err(pci->dev, "EP cannot trigger legacy IRQs\n");
+		return -EINVAL;
+	case PCI_EPC_IRQ_MSI:
+		return dw_pcie_ep_raise_msi_irq(ep, interrupt_num);
+	default:
+		dev_err(pci->dev, "UNKNOWN IRQ type\n");
+	}
+
+	return 0;
+}
+
+static struct dw_pcie_ep_ops pcie_ep_ops = {
+	.ep_init = artpec6_pcie_ep_init,
+	.raise_irq = artpec6_pcie_raise_irq,
 };
 
+static int artpec6_add_pcie_ep(struct artpec6_pcie *artpec6_pcie,
+			       struct platform_device *pdev)
+{
+	int ret;
+	struct dw_pcie_ep *ep;
+	struct resource *res;
+	struct device *dev = &pdev->dev;
+	struct dw_pcie *pci = artpec6_pcie->pci;
+
+	ep = &pci->ep;
+	ep->ops = &pcie_ep_ops;
+
+	res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "dbi2");
+	pci->dbi_base2 = devm_ioremap(dev, res->start, resource_size(res));
+	if (IS_ERR(pci->dbi_base2))
+		return PTR_ERR(pci->dbi_base2);
+
+	res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "addr_space");
+	if (!res)
+		return -EINVAL;
+
+	ep->phys_base = res->start;
+	ep->addr_size = resource_size(res);
+
+	ret = dw_pcie_ep_init(ep);
+	if (ret) {
+		dev_err(dev, "failed to initialize endpoint\n");
+		return ret;
+	}
+
+	return 0;
+}
+
 static int artpec6_pcie_probe(struct platform_device *pdev)
 {
 	struct device *dev = &pdev->dev;
@@ -242,6 +333,16 @@ static int artpec6_pcie_probe(struct platform_device *pdev)
 	struct resource *dbi_base;
 	struct resource *phy_base;
 	int ret;
+	const struct of_device_id *match;
+	const struct artpec_pcie_of_data *data;
+	enum dw_pcie_device_mode mode;
+
+	match = of_match_device(artpec6_pcie_of_match, dev);
+	if (!match)
+		return -EINVAL;
+
+	data = (struct artpec_pcie_of_data *)match->data;
+	mode = (enum dw_pcie_device_mode)data->mode;
 
 	artpec6_pcie = devm_kzalloc(dev, sizeof(*artpec6_pcie), GFP_KERNEL);
 	if (!artpec6_pcie)
@@ -255,6 +356,7 @@ static int artpec6_pcie_probe(struct platform_device *pdev)
 	pci->ops = &dw_pcie_ops;
 
 	artpec6_pcie->pci = pci;
+	artpec6_pcie->mode = mode;
 
 	dbi_base = platform_get_resource_byname(pdev, IORESOURCE_MEM, "dbi");
 	pci->dbi_base = devm_ioremap_resource(dev, dbi_base);
@@ -274,15 +376,53 @@ static int artpec6_pcie_probe(struct platform_device *pdev)
 
 	platform_set_drvdata(pdev, artpec6_pcie);
 
-	ret = artpec6_add_pcie_port(artpec6_pcie, pdev);
-	if (ret < 0)
-		return ret;
+	switch (artpec6_pcie->mode) {
+	case DW_PCIE_RC_TYPE:
+		if (!IS_ENABLED(CONFIG_PCIE_ARTPEC6_HOST))
+			return -ENODEV;
+
+		ret = artpec6_add_pcie_port(artpec6_pcie, pdev);
+		if (ret < 0)
+			return ret;
+		break;
+	case DW_PCIE_EP_TYPE: {
+		u32 val;
+
+		if (!IS_ENABLED(CONFIG_PCIE_ARTPEC6_EP))
+			return -ENODEV;
+
+		val = artpec6_pcie_readl(artpec6_pcie, PCIECFG);
+		val &= ~PCIECFG_DEVICE_TYPE_MASK;
+		artpec6_pcie_writel(artpec6_pcie, PCIECFG, val);
+		ret = artpec6_add_pcie_ep(artpec6_pcie, pdev);
+		if (ret < 0)
+			return ret;
+		break;
+	}
+	default:
+		dev_err(dev, "INVALID device type %d\n", artpec6_pcie->mode);
+	}
 
 	return 0;
 }
 
+static const struct artpec_pcie_of_data artpec6_pcie_rc_of_data = {
+	.mode = DW_PCIE_RC_TYPE,
+};
+
+static const struct artpec_pcie_of_data artpec6_pcie_ep_of_data = {
+	.mode = DW_PCIE_EP_TYPE,
+};
+
 static const struct of_device_id artpec6_pcie_of_match[] = {
-	{ .compatible = "axis,artpec6-pcie", },
+	{
+		.compatible = "axis,artpec6-pcie",
+		.data = &artpec6_pcie_rc_of_data,
+	},
+	{
+		.compatible = "axis,artpec6-pcie-ep",
+		.data = &artpec6_pcie_ep_of_data,
+	},
 	{},
 };
 
-- 
2.14.2

[PATCH v5 15/18] PCI: dwc: Make cpu_addr_fixup take struct dw_pcie as argument

[Niklas Cassel]

There is no need to hard code the cpu to bus address fixup mask.
The PCIe controller has a global address on the AXI bus, however,
from the perspective of the PCIe controller, its base starts at 0x0,
so the local address is 0x0. To get the bus address, simply subtract
the global address from the cpu address. The global address is taken
from device tree.

Also for ARTPEC-7, hard coding the cpu to bus address fixup mask is
not possible, since it uses a High Address Bits Look Up Table, which
means that it can, at runtime, map the PCIe window to an arbitrary
address in the 32-bit address space.

This also fixes a bug for ARTPEC-6, where the cpu to bus address
fixup mask previously was off by one (GENMASK(27, 0), rather than
GENMASK(28, 0)). This is another reason to calculate the mask by
using values from device tree.

Signed-off-by: Niklas Cassel <niklas.cassel@axis.com>
---
 drivers/pci/dwc/pci-dra7xx.c      |  2 +-
 drivers/pci/dwc/pcie-artpec6.c    | 18 ++++++++++++++----
 drivers/pci/dwc/pcie-designware.c |  2 +-
 drivers/pci/dwc/pcie-designware.h |  2 +-
 4 files changed, 17 insertions(+), 7 deletions(-)

diff --git a/drivers/pci/dwc/pci-dra7xx.c b/drivers/pci/dwc/pci-dra7xx.c
index 224ff8affdce..89d87844abb3 100644
--- a/drivers/pci/dwc/pci-dra7xx.c
+++ b/drivers/pci/dwc/pci-dra7xx.c
@@ -110,7 +110,7 @@ static inline void dra7xx_pcie_writel(struct dra7xx_pcie *pcie, u32 offset,
 	writel(value, pcie->base + offset);
 }
 
-static u64 dra7xx_pcie_cpu_addr_fixup(u64 pci_addr)
+static u64 dra7xx_pcie_cpu_addr_fixup(struct dw_pcie *pci, u64 pci_addr)
 {
 	return pci_addr & DRA7XX_CPU_TO_BUS_ADDR;
 }
diff --git a/drivers/pci/dwc/pcie-artpec6.c b/drivers/pci/dwc/pcie-artpec6.c
index e7de4e4649eb..318a2bd0d97e 100644
--- a/drivers/pci/dwc/pcie-artpec6.c
+++ b/drivers/pci/dwc/pcie-artpec6.c
@@ -67,8 +67,6 @@ static const struct of_device_id artpec6_pcie_of_match[];
 #define PHY_STATUS			0x118
 #define  PHY_COSPLLLOCK			BIT(0)
 
-#define ARTPEC6_CPU_TO_BUS_ADDR		GENMASK(27, 0)
-
 static u32 artpec6_pcie_readl(struct artpec6_pcie *artpec6_pcie, u32 offset)
 {
 	u32 val;
@@ -82,9 +80,21 @@ static void artpec6_pcie_writel(struct artpec6_pcie *artpec6_pcie, u32 offset, u
 	regmap_write(artpec6_pcie->regmap, offset, val);
 }
 
-static u64 artpec6_pcie_cpu_addr_fixup(u64 pci_addr)
+static u64 artpec6_pcie_cpu_addr_fixup(struct dw_pcie *pci, u64 pci_addr)
 {
-	return pci_addr & ARTPEC6_CPU_TO_BUS_ADDR;
+	struct artpec6_pcie *artpec6_pcie = to_artpec6_pcie(pci);
+	struct pcie_port *pp = &pci->pp;
+	struct dw_pcie_ep *ep = &pci->ep;
+
+	switch (artpec6_pcie->mode) {
+	case DW_PCIE_RC_TYPE:
+		return pci_addr - pp->cfg0_base;
+	case DW_PCIE_EP_TYPE:
+		return pci_addr - ep->phys_base;
+	default:
+		dev_err(pci->dev, "UNKNOWN device type\n");
+	}
+	return pci_addr;
 }
 
 static int artpec6_pcie_establish_link(struct dw_pcie *pci)
diff --git a/drivers/pci/dwc/pcie-designware.c b/drivers/pci/dwc/pcie-designware.c
index 88abdddee2ad..800be7a4f087 100644
--- a/drivers/pci/dwc/pcie-designware.c
+++ b/drivers/pci/dwc/pcie-designware.c
@@ -149,7 +149,7 @@ void dw_pcie_prog_outbound_atu(struct dw_pcie *pci, int index, int type,
 	u32 retries, val;
 
 	if (pci->ops->cpu_addr_fixup)
-		cpu_addr = pci->ops->cpu_addr_fixup(cpu_addr);
+		cpu_addr = pci->ops->cpu_addr_fixup(pci, cpu_addr);
 
 	if (pci->iatu_unroll_enabled) {
 		dw_pcie_prog_outbound_atu_unroll(pci, index, type, cpu_addr,
diff --git a/drivers/pci/dwc/pcie-designware.h b/drivers/pci/dwc/pcie-designware.h
index 24edac035160..cca5a81c1c74 100644
--- a/drivers/pci/dwc/pcie-designware.h
+++ b/drivers/pci/dwc/pcie-designware.h
@@ -205,7 +205,7 @@ struct dw_pcie_ep {
 };
 
 struct dw_pcie_ops {
-	u64	(*cpu_addr_fixup)(u64 cpu_addr);
+	u64	(*cpu_addr_fixup)(struct dw_pcie *pcie, u64 cpu_addr);
 	u32	(*read_dbi)(struct dw_pcie *pcie, void __iomem *base, u32 reg,
 			    size_t size);
 	void	(*write_dbi)(struct dw_pcie *pcie, void __iomem *base, u32 reg,
-- 
2.14.2

[PATCH v5 16/18] PCI: dwc: artpec6: Deassert the core before waiting for PHY

[Niklas Cassel]

Waiting for the PHY while the core was held in reset worked for artpec6,
but for artpec7, in order to read the required registers, the core has to
be out of reset.
Refactor the code so we always wait for the PHY after the core has been
deasserted, since this works for both artpec6 and artpec7.

Signed-off-by: Niklas Cassel <niklas.cassel@axis.com>
---
 drivers/pci/dwc/pcie-artpec6.c | 45 +++++++++++++++++++++++++++---------------
 1 file changed, 29 insertions(+), 16 deletions(-)

diff --git a/drivers/pci/dwc/pcie-artpec6.c b/drivers/pci/dwc/pcie-artpec6.c
index 318a2bd0d97e..064c5a93ea80 100644
--- a/drivers/pci/dwc/pcie-artpec6.c
+++ b/drivers/pci/dwc/pcie-artpec6.c
@@ -125,11 +125,37 @@ static const struct dw_pcie_ops dw_pcie_ops = {
 	.stop_link = artpec6_pcie_stop_link,
 };
 
-static void artpec6_pcie_init_phy(struct artpec6_pcie *artpec6_pcie)
+static void artpec6_pcie_wait_for_phy(struct artpec6_pcie *artpec6_pcie)
 {
+	struct dw_pcie *pci = artpec6_pcie->pci;
+	struct device *dev = pci->dev;
 	u32 val;
 	unsigned int retries;
 
+	retries = 50;
+	do {
+		usleep_range(1000, 2000);
+		val = artpec6_pcie_readl(artpec6_pcie, NOCCFG);
+		retries--;
+	} while (retries &&
+		(val & (NOCCFG_POWER_PCIE_IDLEACK | NOCCFG_POWER_PCIE_IDLE)));
+	if (!retries)
+		dev_err(dev, "PCIe clock manager did not leave idle state\n");
+
+	retries = 50;
+	do {
+		usleep_range(1000, 2000);
+		val = readl(artpec6_pcie->phy_base + PHY_STATUS);
+		retries--;
+	} while (retries && !(val & PHY_COSPLLLOCK));
+	if (!retries)
+		dev_err(dev, "PHY PLL did not lock\n");
+}
+
+static void artpec6_pcie_init_phy(struct artpec6_pcie *artpec6_pcie)
+{
+	u32 val;
+
 	val = artpec6_pcie_readl(artpec6_pcie, PCIECFG);
 	val |=  PCIECFG_RISRCREN |	/* Receiver term. 50 Ohm */
 		PCIECFG_MODE_TX_DRV_EN |
@@ -154,21 +180,6 @@ static void artpec6_pcie_init_phy(struct artpec6_pcie *artpec6_pcie)
 	val = artpec6_pcie_readl(artpec6_pcie, NOCCFG);
 	val &= ~NOCCFG_POWER_PCIE_IDLEREQ;
 	artpec6_pcie_writel(artpec6_pcie, NOCCFG, val);
-
-	retries = 50;
-	do {
-		usleep_range(1000, 2000);
-		val = artpec6_pcie_readl(artpec6_pcie, NOCCFG);
-		retries--;
-	} while (retries &&
-		(val & (NOCCFG_POWER_PCIE_IDLEACK | NOCCFG_POWER_PCIE_IDLE)));
-
-	retries = 50;
-	do {
-		usleep_range(1000, 2000);
-		val = readl(artpec6_pcie->phy_base + PHY_STATUS);
-		retries--;
-	} while (retries && !(val & PHY_COSPLLLOCK));
 }
 
 static void artpec6_pcie_assert_core_reset(struct artpec6_pcie *artpec6_pcie)
@@ -207,6 +218,7 @@ static int artpec6_pcie_host_init(struct pcie_port *pp)
 	artpec6_pcie_assert_core_reset(artpec6_pcie);
 	artpec6_pcie_init_phy(artpec6_pcie);
 	artpec6_pcie_deassert_core_reset(artpec6_pcie);
+	artpec6_pcie_wait_for_phy(artpec6_pcie);
 	dw_pcie_setup_rc(pp);
 	artpec6_pcie_establish_link(pci);
 	dw_pcie_wait_for_link(pci);
@@ -274,6 +286,7 @@ static void artpec6_pcie_ep_init(struct dw_pcie_ep *ep)
 	artpec6_pcie_assert_core_reset(artpec6_pcie);
 	artpec6_pcie_init_phy(artpec6_pcie);
 	artpec6_pcie_deassert_core_reset(artpec6_pcie);
+	artpec6_pcie_wait_for_phy(artpec6_pcie);
 
 	for (bar = BAR_0; bar <= BAR_5; bar++)
 		dw_pcie_ep_reset_bar(pci, bar);
-- 
2.14.2

[PATCH v5 17/18] bindings: PCI: artpec: Add support for the ARTPEC-7 SoC

[Niklas Cassel]

Add support for the ARTPEC-7 SoC in the artpec6 driver.
The ARTPEC-6 SoC and the ARTPEC-7 SoC are very similar.
Unfortunately, some fields in the PCIECFG and PCIESTAT
register have changed.

Signed-off-by: Niklas Cassel <niklas.cassel@axis.com>
Acked-by: Rob Herring <robh@kernel.org>
---
 Documentation/devicetree/bindings/pci/axis,artpec6-pcie.txt | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/Documentation/devicetree/bindings/pci/axis,artpec6-pcie.txt b/Documentation/devicetree/bindings/pci/axis,artpec6-pcie.txt
index 33eef7ae5a23..979dc7b6cfe8 100644
--- a/Documentation/devicetree/bindings/pci/axis,artpec6-pcie.txt
+++ b/Documentation/devicetree/bindings/pci/axis,artpec6-pcie.txt
@@ -6,6 +6,8 @@ and thus inherits all the common properties defined in designware-pcie.txt.
 Required properties:
 - compatible: "axis,artpec6-pcie", "snps,dw-pcie" for ARTPEC-6 in RC mode;
 	      "axis,artpec6-pcie-ep", "snps,dw-pcie" for ARTPEC-6 in EP mode;
+	      "axis,artpec7-pcie", "snps,dw-pcie" for ARTPEC-7 in RC mode;
+	      "axis,artpec7-pcie-ep", "snps,dw-pcie" for ARTPEC-7 in EP mode;
 - reg: base addresses and lengths of the PCIe controller (DBI),
 	the PHY controller, and configuration address space.
 - reg-names: Must include the following entries:
-- 
2.14.2

[PATCH v5 18/18] PCI: dwc: artpec6: Add support for the ARTPEC-7 SoC

[Niklas Cassel]

Add support for	the ARTPEC-7 SoC in the	artpec6	driver.
The ARTPEC-6 SoC and the ARTPEC-7 SoC are very similar.
Unfortunately, some fields in the PCIECFG and PCIESTAT
register have changed.

Signed-off-by: Niklas Cassel <niklas.cassel@axis.com>
---
 drivers/pci/dwc/pcie-artpec6.c | 187 ++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 183 insertions(+), 4 deletions(-)

diff --git a/drivers/pci/dwc/pcie-artpec6.c b/drivers/pci/dwc/pcie-artpec6.c
index 064c5a93ea80..312f21b6e013 100644
--- a/drivers/pci/dwc/pcie-artpec6.c
+++ b/drivers/pci/dwc/pcie-artpec6.c
@@ -27,14 +27,21 @@
 
 #define to_artpec6_pcie(x)	dev_get_drvdata((x)->dev)
 
+enum artpec_pcie_variants {
+	ARTPEC6,
+	ARTPEC7,
+};
+
 struct artpec6_pcie {
 	struct dw_pcie		*pci;
 	struct regmap		*regmap;	/* DT axis,syscon-pcie */
 	void __iomem		*phy_base;	/* DT phy */
+	enum artpec_pcie_variants variant;
 	enum dw_pcie_device_mode mode;
 };
 
 struct artpec_pcie_of_data {
+	enum artpec_pcie_variants variant;
 	enum dw_pcie_device_mode mode;
 };
 
@@ -43,6 +50,13 @@ static const struct of_device_id artpec6_pcie_of_match[];
 /* PCIe Port Logic registers (memory-mapped) */
 #define PL_OFFSET			0x700
 
+#define ACK_F_ASPM_CTRL_OFF		(PL_OFFSET + 0xc)
+#define ACK_N_FTS_MASK			GENMASK(15, 8)
+#define ACK_N_FTS(x)			(((x) << 8) & ACK_N_FTS_MASK)
+
+#define FAST_TRAINING_SEQ_MASK		GENMASK(7, 0)
+#define FAST_TRAINING_SEQ(x)		(((x) << 0) & FAST_TRAINING_SEQ_MASK)
+
 /* ARTPEC-6 specific registers */
 #define PCIECFG				0x18
 #define  PCIECFG_DBG_OEN		BIT(24)
@@ -57,6 +71,13 @@ static const struct of_device_id artpec6_pcie_of_match[];
 #define  PCIECFG_MODE_TX_DRV_EN		BIT(3)
 #define  PCIECFG_CISRREN		BIT(2)
 #define  PCIECFG_MACRO_ENABLE		BIT(0)
+/* ARTPEC-7 specific fields */
+#define  PCIECFG_REFCLKSEL		BIT(23)
+#define  PCIECFG_NOC_RESET		BIT(3)
+
+#define PCIESTAT			0x1c
+/* ARTPEC-7 specific fields */
+#define  PCIESTAT_EXTREFCLK		BIT(3)
 
 #define NOCCFG				0x40
 #define  NOCCFG_ENABLE_CLK_PCIE		BIT(4)
@@ -67,6 +88,12 @@ static const struct of_device_id artpec6_pcie_of_match[];
 #define PHY_STATUS			0x118
 #define  PHY_COSPLLLOCK			BIT(0)
 
+#define PHY_TX_ASIC_OUT			0x4040
+#define  PHY_TX_ASIC_OUT_TX_ACK		BIT(0)
+
+#define PHY_RX_ASIC_OUT			0x405c
+#define  PHY_RX_ASIC_OUT_ACK		BIT(0)
+
 static u32 artpec6_pcie_readl(struct artpec6_pcie *artpec6_pcie, u32 offset)
 {
 	u32 val;
@@ -125,7 +152,7 @@ static const struct dw_pcie_ops dw_pcie_ops = {
 	.stop_link = artpec6_pcie_stop_link,
 };
 
-static void artpec6_pcie_wait_for_phy(struct artpec6_pcie *artpec6_pcie)
+static void artpec6_pcie_wait_for_phy_a6(struct artpec6_pcie *artpec6_pcie)
 {
 	struct dw_pcie *pci = artpec6_pcie->pci;
 	struct device *dev = pci->dev;
@@ -152,7 +179,49 @@ static void artpec6_pcie_wait_for_phy(struct artpec6_pcie *artpec6_pcie)
 		dev_err(dev, "PHY PLL did not lock\n");
 }
 
-static void artpec6_pcie_init_phy(struct artpec6_pcie *artpec6_pcie)
+static void artpec6_pcie_wait_for_phy_a7(struct artpec6_pcie *artpec6_pcie)
+{
+	struct dw_pcie *pci = artpec6_pcie->pci;
+	struct device *dev = pci->dev;
+	u32 val;
+	u16 phy_status_tx, phy_status_rx;
+	unsigned int retries;
+
+	retries = 50;
+	do {
+		usleep_range(1000, 2000);
+		val = artpec6_pcie_readl(artpec6_pcie, NOCCFG);
+		retries--;
+	} while (retries &&
+		(val & (NOCCFG_POWER_PCIE_IDLEACK | NOCCFG_POWER_PCIE_IDLE)));
+	if (!retries)
+		dev_err(dev, "PCIe clock manager did not leave idle state\n");
+
+	retries = 50;
+	do {
+		usleep_range(1000, 2000);
+		phy_status_tx = readw(artpec6_pcie->phy_base + PHY_TX_ASIC_OUT);
+		phy_status_rx = readw(artpec6_pcie->phy_base + PHY_RX_ASIC_OUT);
+		retries--;
+	} while (retries && ((phy_status_tx & PHY_TX_ASIC_OUT_TX_ACK) ||
+				(phy_status_rx & PHY_RX_ASIC_OUT_ACK)));
+	if (!retries)
+		dev_err(dev, "PHY did not enter Pn state\n");
+}
+
+static void artpec6_pcie_wait_for_phy(struct artpec6_pcie *artpec6_pcie)
+{
+	switch (artpec6_pcie->variant) {
+	case ARTPEC6:
+		artpec6_pcie_wait_for_phy_a6(artpec6_pcie);
+		break;
+	case ARTPEC7:
+		artpec6_pcie_wait_for_phy_a7(artpec6_pcie);
+		break;
+	}
+}
+
+static void artpec6_pcie_init_phy_a6(struct artpec6_pcie *artpec6_pcie)
 {
 	u32 val;
 
@@ -182,12 +251,90 @@ static void artpec6_pcie_init_phy(struct artpec6_pcie *artpec6_pcie)
 	artpec6_pcie_writel(artpec6_pcie, NOCCFG, val);
 }
 
+static void artpec6_pcie_init_phy_a7(struct artpec6_pcie *artpec6_pcie)
+{
+	struct dw_pcie *pci = artpec6_pcie->pci;
+	u32 val;
+	bool extrefclk;
+
+	/* Check if external reference clock is connected */
+	val = artpec6_pcie_readl(artpec6_pcie, PCIESTAT);
+	extrefclk = !!(val & PCIESTAT_EXTREFCLK);
+	dev_dbg(pci->dev, "Using reference clock: %s\n",
+		extrefclk ? "external" : "internal");
+
+	val = artpec6_pcie_readl(artpec6_pcie, PCIECFG);
+	val |=  PCIECFG_RISRCREN |	/* Receiver term. 50 Ohm */
+		PCIECFG_PCLK_ENABLE;
+	if (extrefclk)
+		val |= PCIECFG_REFCLKSEL;
+	else
+		val &= ~PCIECFG_REFCLKSEL;
+	artpec6_pcie_writel(artpec6_pcie, PCIECFG, val);
+	usleep_range(10, 20);
+
+	val = artpec6_pcie_readl(artpec6_pcie, NOCCFG);
+	val |= NOCCFG_ENABLE_CLK_PCIE;
+	artpec6_pcie_writel(artpec6_pcie, NOCCFG, val);
+	usleep_range(20, 30);
+
+	val = artpec6_pcie_readl(artpec6_pcie, NOCCFG);
+	val &= ~NOCCFG_POWER_PCIE_IDLEREQ;
+	artpec6_pcie_writel(artpec6_pcie, NOCCFG, val);
+}
+
+static void artpec6_pcie_init_phy(struct artpec6_pcie *artpec6_pcie)
+{
+	switch (artpec6_pcie->variant) {
+	case ARTPEC6:
+		artpec6_pcie_init_phy_a6(artpec6_pcie);
+		break;
+	case ARTPEC7:
+		artpec6_pcie_init_phy_a7(artpec6_pcie);
+		break;
+	}
+}
+
+static void artpec6_pcie_set_nfts(struct artpec6_pcie *artpec6_pcie)
+{
+	struct dw_pcie *pci = artpec6_pcie->pci;
+	u32 val;
+
+	if (artpec6_pcie->variant != ARTPEC7)
+		return;
+
+	/*
+	 * Increase the N_FTS (Number of Fast Training Sequences)
+	 * to be transmitted when transitioning from L0s to L0.
+	 */
+	val = dw_pcie_readl_dbi(pci, ACK_F_ASPM_CTRL_OFF);
+	val &= ~ACK_N_FTS_MASK;
+	val |= ACK_N_FTS(180);
+	dw_pcie_writel_dbi(pci, ACK_F_ASPM_CTRL_OFF, val);
+
+	/*
+	 * Set the Number of Fast Training Sequences that the core
+	 * advertises as its N_FTS during Gen2 or Gen3 link training.
+	 */
+	val = dw_pcie_readl_dbi(pci, PCIE_LINK_WIDTH_SPEED_CONTROL);
+	val &= ~FAST_TRAINING_SEQ_MASK;
+	val |= FAST_TRAINING_SEQ(180);
+	dw_pcie_writel_dbi(pci, PCIE_LINK_WIDTH_SPEED_CONTROL, val);
+}
+
 static void artpec6_pcie_assert_core_reset(struct artpec6_pcie *artpec6_pcie)
 {
 	u32 val;
 
 	val = artpec6_pcie_readl(artpec6_pcie, PCIECFG);
-	val |= PCIECFG_CORE_RESET_REQ;
+	switch (artpec6_pcie->variant) {
+	case ARTPEC6:
+		val |= PCIECFG_CORE_RESET_REQ;
+		break;
+	case ARTPEC7:
+		val &= ~PCIECFG_NOC_RESET;
+		break;
+	}
 	artpec6_pcie_writel(artpec6_pcie, PCIECFG, val);
 }
 
@@ -196,7 +343,14 @@ static void artpec6_pcie_deassert_core_reset(struct artpec6_pcie *artpec6_pcie)
 	u32 val;
 
 	val = artpec6_pcie_readl(artpec6_pcie, PCIECFG);
-	val &= ~PCIECFG_CORE_RESET_REQ;
+	switch (artpec6_pcie->variant) {
+	case ARTPEC6:
+		val &= ~PCIECFG_CORE_RESET_REQ;
+		break;
+	case ARTPEC7:
+		val |= PCIECFG_NOC_RESET;
+		break;
+	}
 	artpec6_pcie_writel(artpec6_pcie, PCIECFG, val);
 	usleep_range(100, 200);
 }
@@ -219,6 +373,7 @@ static int artpec6_pcie_host_init(struct pcie_port *pp)
 	artpec6_pcie_init_phy(artpec6_pcie);
 	artpec6_pcie_deassert_core_reset(artpec6_pcie);
 	artpec6_pcie_wait_for_phy(artpec6_pcie);
+	artpec6_pcie_set_nfts(artpec6_pcie);
 	dw_pcie_setup_rc(pp);
 	artpec6_pcie_establish_link(pci);
 	dw_pcie_wait_for_link(pci);
@@ -287,6 +442,7 @@ static void artpec6_pcie_ep_init(struct dw_pcie_ep *ep)
 	artpec6_pcie_init_phy(artpec6_pcie);
 	artpec6_pcie_deassert_core_reset(artpec6_pcie);
 	artpec6_pcie_wait_for_phy(artpec6_pcie);
+	artpec6_pcie_set_nfts(artpec6_pcie);
 
 	for (bar = BAR_0; bar <= BAR_5; bar++)
 		dw_pcie_ep_reset_bar(pci, bar);
@@ -358,6 +514,7 @@ static int artpec6_pcie_probe(struct platform_device *pdev)
 	int ret;
 	const struct of_device_id *match;
 	const struct artpec_pcie_of_data *data;
+	enum artpec_pcie_variants variant;
 	enum dw_pcie_device_mode mode;
 
 	match = of_match_device(artpec6_pcie_of_match, dev);
@@ -365,6 +522,7 @@ static int artpec6_pcie_probe(struct platform_device *pdev)
 		return -EINVAL;
 
 	data = (struct artpec_pcie_of_data *)match->data;
+	variant = (enum artpec_pcie_variants)data->variant;
 	mode = (enum dw_pcie_device_mode)data->mode;
 
 	artpec6_pcie = devm_kzalloc(dev, sizeof(*artpec6_pcie), GFP_KERNEL);
@@ -379,6 +537,7 @@ static int artpec6_pcie_probe(struct platform_device *pdev)
 	pci->ops = &dw_pcie_ops;
 
 	artpec6_pcie->pci = pci;
+	artpec6_pcie->variant = variant;
 	artpec6_pcie->mode = mode;
 
 	dbi_base = platform_get_resource_byname(pdev, IORESOURCE_MEM, "dbi");
@@ -430,10 +589,22 @@ static int artpec6_pcie_probe(struct platform_device *pdev)
 }
 
 static const struct artpec_pcie_of_data artpec6_pcie_rc_of_data = {
+	.variant = ARTPEC6,
 	.mode = DW_PCIE_RC_TYPE,
 };
 
 static const struct artpec_pcie_of_data artpec6_pcie_ep_of_data = {
+	.variant = ARTPEC6,
+	.mode = DW_PCIE_EP_TYPE,
+};
+
+static const struct artpec_pcie_of_data artpec7_pcie_rc_of_data = {
+	.variant = ARTPEC7,
+	.mode = DW_PCIE_RC_TYPE,
+};
+
+static const struct artpec_pcie_of_data artpec7_pcie_ep_of_data = {
+	.variant = ARTPEC7,
 	.mode = DW_PCIE_EP_TYPE,
 };
 
@@ -446,6 +617,14 @@ static const struct of_device_id artpec6_pcie_of_match[] = {
 		.compatible = "axis,artpec6-pcie-ep",
 		.data = &artpec6_pcie_ep_of_data,
 	},
+	{
+		.compatible = "axis,artpec7-pcie",
+		.data = &artpec7_pcie_rc_of_data,
+	},
+	{
+		.compatible = "axis,artpec7-pcie-ep",
+		.data = &artpec7_pcie_ep_of_data,
+	},
 	{},
 };
 
-- 
2.14.2

Re: [PATCH v5 01/18] PCI: dwc: Use the DMA-API to get the MSI address

[Lorenzo Pieralisi]

Jingoo, Joao,

I am expecting your testing on the series and ACKs on the dwc related
patches please, according to v4 review - I will mark them as needs
review/ACK waiting for you to chime in.

v4 thread:

https://patchwork.ozlabs.org/patch/833882/

Thanks,
Lorenzo

On Mon, Nov 20, 2017 at 02:32:04PM +0100, Niklas Cassel wrote:
> Use the DMA-API to get the MSI address. This address will be written to
> our PCI config space and to the register which determines which AXI
> address the DWC IP will spoof for incoming MSI irqs.
> 
> Since it is a PCIe endpoint device, rather than the CPU, that is supposed
> to write to the MSI address, the proper way to get the MSI address is by
> using the DMA API, not by using virt_to_phys().
> 
> Using virt_to_phys() might work on some systems, but using the DMA API
> should work on all systems.
> 
> This is essentially the same thing as allocating a buffer in a driver
> to which the endpoint will write to. To do this, we use the DMA API.
> 
> Signed-off-by: Niklas Cassel <niklas.cassel@axis.com>
> ---
>  drivers/pci/dwc/pcie-designware-host.c | 15 ++++++++++++---
>  drivers/pci/dwc/pcie-designware.h      |  3 ++-
>  2 files changed, 14 insertions(+), 4 deletions(-)
> 
> diff --git a/drivers/pci/dwc/pcie-designware-host.c b/drivers/pci/dwc/pcie-designware-host.c
> index 81e2157a7cfb..33b52fe98a01 100644
> --- a/drivers/pci/dwc/pcie-designware-host.c
> +++ b/drivers/pci/dwc/pcie-designware-host.c
> @@ -83,10 +83,19 @@ irqreturn_t dw_handle_msi_irq(struct pcie_port *pp)
>  
>  void dw_pcie_msi_init(struct pcie_port *pp)
>  {
> +	struct dw_pcie *pci = to_dw_pcie_from_pp(pp);
> +	struct device *dev = pci->dev;
> +	struct page *page;
>  	u64 msi_target;
>  
> -	pp->msi_data = __get_free_pages(GFP_KERNEL, 0);
> -	msi_target = virt_to_phys((void *)pp->msi_data);
> +	page = alloc_page(GFP_KERNEL | GFP_DMA32);
> +	pp->msi_data = dma_map_page(dev, page, 0, PAGE_SIZE, DMA_FROM_DEVICE);
> +	if (dma_mapping_error(dev, pp->msi_data)) {
> +		dev_err(dev, "failed to map MSI data\n");
> +		__free_page(page);
> +		return;
> +	}
> +	msi_target = (u64)pp->msi_data;
>  
>  	/* program the msi_data */
>  	dw_pcie_wr_own_conf(pp, PCIE_MSI_ADDR_LO, 4,
> @@ -187,7 +196,7 @@ static void dw_msi_setup_msg(struct pcie_port *pp, unsigned int irq, u32 pos)
>  	if (pp->ops->get_msi_addr)
>  		msi_target = pp->ops->get_msi_addr(pp);
>  	else
> -		msi_target = virt_to_phys((void *)pp->msi_data);
> +		msi_target = (u64)pp->msi_data;
>  
>  	msg.address_lo = (u32)(msi_target & 0xffffffff);
>  	msg.address_hi = (u32)(msi_target >> 32 & 0xffffffff);
> diff --git a/drivers/pci/dwc/pcie-designware.h b/drivers/pci/dwc/pcie-designware.h
> index e5d9d77b778e..ecdede68522a 100644
> --- a/drivers/pci/dwc/pcie-designware.h
> +++ b/drivers/pci/dwc/pcie-designware.h
> @@ -14,6 +14,7 @@
>  #ifndef _PCIE_DESIGNWARE_H
>  #define _PCIE_DESIGNWARE_H
>  
> +#include <linux/dma-mapping.h>
>  #include <linux/irq.h>
>  #include <linux/msi.h>
>  #include <linux/pci.h>
> @@ -168,7 +169,7 @@ struct pcie_port {
>  	const struct dw_pcie_host_ops *ops;
>  	int			msi_irq;
>  	struct irq_domain	*irq_domain;
> -	unsigned long		msi_data;
> +	dma_addr_t		msi_data;
>  	DECLARE_BITMAP(msi_irq_in_use, MAX_MSI_IRQS);
>  };
>  
> -- 
> 2.14.2
> 

Re: [PATCH v5 01/18] PCI: dwc: Use the DMA-API to get the MSI address

[Niklas Cassel]

On Thu, Nov 30, 2017 at 03:28:43PM +0000, Lorenzo Pieralisi wrote:
> Jingoo, Joao,
> 
> I am expecting your testing on the series and ACKs on the dwc related
> patches please, according to v4 review - I will mark them as needs
> review/ACK waiting for you to chime in.
> 
> v4 thread:
> 
> https://patchwork.ozlabs.org/patch/833882/
> 

Hello Lorenzo, Jingoo, Joao,

Tomorrow another 2 weeks has passed.
V1 of this patch series was posted on 2017-10-13.

I'm a bit worried that this patch series will not make it to linux-next
in time for this patch series to be included in the 4.16 pull request.

Here is the V5 patch series:
https://patchwork.ozlabs.org/project/linux-pci/list/?series=14364

Perhaps any maintainer of a designware based PCIe driver could
help out and test this patch series?

Regards,
Niklas

Re: [PATCH v5 01/18] PCI: dwc: Use the DMA-API to get the MSI address

[Lorenzo Pieralisi]

Jingoo, Joao,

Niklas has a point. I can't apply this series without your testing
and review, so please do test/ack if you deem that suitable.

On Wed, Dec 13, 2017 at 02:59:54PM +0100, Niklas Cassel wrote:
> On Thu, Nov 30, 2017 at 03:28:43PM +0000, Lorenzo Pieralisi wrote:
> > Jingoo, Joao,
> > 
> > I am expecting your testing on the series and ACKs on the dwc related
> > patches please, according to v4 review - I will mark them as needs
> > review/ACK waiting for you to chime in.
> > 
> > v4 thread:
> > 
> > https://patchwork.ozlabs.org/patch/833882/
> > 
> 
> Hello Lorenzo, Jingoo, Joao,
> 
> Tomorrow another 2 weeks has passed.
> V1 of this patch series was posted on 2017-10-13.

I understand - I need their review/testing to move forward with this
series and I strongly encourage them to chime in - as soon as possible.

> I'm a bit worried that this patch series will not make it to linux-next
> in time for this patch series to be included in the 4.16 pull request.

We should be able to do it if they reply within a reasonable timeframe.

Thanks,
Lorenzo

> Here is the V5 patch series:
> https://patchwork.ozlabs.org/project/linux-pci/list/?series=14364
> 
> Perhaps any maintainer of a designware based PCIe driver could
> help out and test this patch series?
> 
> Regards,
> Niklas

Re: [PATCH v5 01/18] PCI: dwc: Use the DMA-API to get the MSI address

[Joao Pinto]

Hi Niklas,

Às 1:59 PM de 12/13/2017, Niklas Cassel escreveu:
> On Thu, Nov 30, 2017 at 03:28:43PM +0000, Lorenzo Pieralisi wrote:
>> Jingoo, Joao,
>>
>> I am expecting your testing on the series and ACKs on the dwc related
>> patches please, according to v4 review - I will mark them as needs
>> review/ACK waiting for you to chime in.
>>
>> v4 thread:
>>
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__patchwork.ozlabs.org_patch_833882_&d=DwIBAg&c=DPL6_X_6JkXFx7AXWqB0tg&r=s2fO0hii0OGNOv9qQy_HRXy-xAJUD1NNoEcc3io_kx0&m=CQAV8rCgm2jd7m3iSWL5vBbGTqSc7yN3N7zeQDz-ZUY&s=mqyDi45qm4qE0jcm0KviYAoLHmMXMMIWEW-gAR0RKOw&e=
>>
> 
> Hello Lorenzo, Jingoo, Joao,
> 
> Tomorrow another 2 weeks has passed.
> V1 of this patch series was posted on 2017-10-13.

Sorry, I have been tight up with a debug session and not able to check this out.
Adding Gustavo in CC that is now also working in PCI software development.
I am going to check the code ASAP and we will test it as soon as the debug is
finished.

Thanks.

> 
> I'm a bit worried that this patch series will not make it to linux-next
> in time for this patch series to be included in the 4.16 pull request.
> 
> Here is the V5 patch series:
> https://urldefense.proofpoint.com/v2/url?u=https-3A__patchwork.ozlabs.org_project_linux-2Dpci_list_-3Fseries-3D14364&d=DwIBAg&c=DPL6_X_6JkXFx7AXWqB0tg&r=s2fO0hii0OGNOv9qQy_HRXy-xAJUD1NNoEcc3io_kx0&m=CQAV8rCgm2jd7m3iSWL5vBbGTqSc7yN3N7zeQDz-ZUY&s=qXtUHFXK7_nCndQob0I0qDByBdJsL7rRjZ9cx7c847U&e=
> 
> Perhaps any maintainer of a designware based PCIe driver could
> help out and test this patch series?
> 
> Regards,
> Niklas
> 

Re: [PATCH v5 01/18] PCI: dwc: Use the DMA-API to get the MSI address

[Gustavo Pimentel]

Hi Niklas and Lorenzo,

I'm going to work on PCI software development now as told by João and I will
test your code now.

I was retrieving the patches through the patchwork
https://patchwork.ozlabs.org/project/linux-pci/list/?submitter=65580 and I
notice that its missing the patch 13 and 17, is this right?

If not, can you give me both patches location?

Thanks.


On 13/12/2017 17:21, Joao Pinto wrote:
> Hi Niklas,
>
> Às 1:59 PM de 12/13/2017, Niklas Cassel escreveu:
>> On Thu, Nov 30, 2017 at 03:28:43PM +0000, Lorenzo Pieralisi wrote:
>>> Jingoo, Joao,
>>>
>>> I am expecting your testing on the series and ACKs on the dwc related
>>> patches please, according to v4 review - I will mark them as needs
>>> review/ACK waiting for you to chime in.
>>>
>>> v4 thread:
>>>
>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__patchwork.ozlabs.org_patch_833882_&d=DwIBAg&c=DPL6_X_6JkXFx7AXWqB0tg&r=s2fO0hii0OGNOv9qQy_HRXy-xAJUD1NNoEcc3io_kx0&m=CQAV8rCgm2jd7m3iSWL5vBbGTqSc7yN3N7zeQDz-ZUY&s=mqyDi45qm4qE0jcm0KviYAoLHmMXMMIWEW-gAR0RKOw&e=
>>>
>> Hello Lorenzo, Jingoo, Joao,
>>
>> Tomorrow another 2 weeks has passed.
>> V1 of this patch series was posted on 2017-10-13.
> Sorry, I have been tight up with a debug session and not able to check this out.
> Adding Gustavo in CC that is now also working in PCI software development.
> I am going to check the code ASAP and we will test it as soon as the debug is
> finished.
>
> Thanks.
>
>> I'm a bit worried that this patch series will not make it to linux-next
>> in time for this patch series to be included in the 4.16 pull request.
>>
>> Here is the V5 patch series:
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__patchwork.ozlabs.org_project_linux-2Dpci_list_-3Fseries-3D14364&d=DwIBAg&c=DPL6_X_6JkXFx7AXWqB0tg&r=s2fO0hii0OGNOv9qQy_HRXy-xAJUD1NNoEcc3io_kx0&m=CQAV8rCgm2jd7m3iSWL5vBbGTqSc7yN3N7zeQDz-ZUY&s=qXtUHFXK7_nCndQob0I0qDByBdJsL7rRjZ9cx7c847U&e=
>>
>> Perhaps any maintainer of a designware based PCIe driver could
>> help out and test this patch series?
>>
>> Regards,
>> Niklas
>>

Re: [PATCH v5 01/18] PCI: dwc: Use the DMA-API to get the MSI address

[Lorenzo Pieralisi]

On Thu, Dec 14, 2017 at 12:16:38PM +0000, Gustavo Pimentel wrote:
> Hi Niklas and Lorenzo,
> 
> I'm going to work on PCI software development now as told by Joao and I will
> test your code now.
> 
> I was retrieving the patches through the patchwork
> https://patchwork.ozlabs.org/project/linux-pci/list/?submitter=65580 and I
> notice that its missing the patch 13 and 17, is this right?

No, you just have to refine the filter, I marked the DT bindings as
ready to be applied that's why they do not show up (and by the way
you can test the series even without those two patches - that are just
DT bindings).

Thanks,
Lorenzo

> If not, can you give me both patches location?
> 
> Thanks.
> 
> 
> On 13/12/2017 17:21, Joao Pinto wrote:
> > Hi Niklas,
> >
> > ??s 1:59 PM de 12/13/2017, Niklas Cassel escreveu:
> >> On Thu, Nov 30, 2017 at 03:28:43PM +0000, Lorenzo Pieralisi wrote:
> >>> Jingoo, Joao,
> >>>
> >>> I am expecting your testing on the series and ACKs on the dwc related
> >>> patches please, according to v4 review - I will mark them as needs
> >>> review/ACK waiting for you to chime in.
> >>>
> >>> v4 thread:
> >>>
> >>> https://urldefense.proofpoint.com/v2/url?u=https-3A__patchwork.ozlabs.org_patch_833882_&d=DwIBAg&c=DPL6_X_6JkXFx7AXWqB0tg&r=s2fO0hii0OGNOv9qQy_HRXy-xAJUD1NNoEcc3io_kx0&m=CQAV8rCgm2jd7m3iSWL5vBbGTqSc7yN3N7zeQDz-ZUY&s=mqyDi45qm4qE0jcm0KviYAoLHmMXMMIWEW-gAR0RKOw&e=
> >>>
> >> Hello Lorenzo, Jingoo, Joao,
> >>
> >> Tomorrow another 2 weeks has passed.
> >> V1 of this patch series was posted on 2017-10-13.
> > Sorry, I have been tight up with a debug session and not able to check this out.
> > Adding Gustavo in CC that is now also working in PCI software development.
> > I am going to check the code ASAP and we will test it as soon as the debug is
> > finished.
> >
> > Thanks.
> >
> >> I'm a bit worried that this patch series will not make it to linux-next
> >> in time for this patch series to be included in the 4.16 pull request.
> >>
> >> Here is the V5 patch series:
> >> https://urldefense.proofpoint.com/v2/url?u=https-3A__patchwork.ozlabs.org_project_linux-2Dpci_list_-3Fseries-3D14364&d=DwIBAg&c=DPL6_X_6JkXFx7AXWqB0tg&r=s2fO0hii0OGNOv9qQy_HRXy-xAJUD1NNoEcc3io_kx0&m=CQAV8rCgm2jd7m3iSWL5vBbGTqSc7yN3N7zeQDz-ZUY&s=qXtUHFXK7_nCndQob0I0qDByBdJsL7rRjZ9cx7c847U&e=
> >>
> >> Perhaps any maintainer of a designware based PCIe driver could
> >> help out and test this patch series?
> >>
> >> Regards,
> >> Niklas
> >>
> 
> 

Re: [PATCH v5 01/18] PCI: dwc: Use the DMA-API to get the MSI address

[Gustavo Pimentel]

Ok. I will compile and test it now.

Thanks.


On 14/12/2017 12:22, Lorenzo Pieralisi wrote:
> On Thu, Dec 14, 2017 at 12:16:38PM +0000, Gustavo Pimentel wrote:
>> Hi Niklas and Lorenzo,
>>
>> I'm going to work on PCI software development now as told by Joao and I will
>> test your code now.
>>
>> I was retrieving the patches through the patchwork
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__patchwork.ozlabs.org_project_linux-2Dpci_list_-3Fsubmitter-3D65580&d=DwIBAg&c=DPL6_X_6JkXFx7AXWqB0tg&r=bkWxpLoW-f-E3EdiDCCa0_h0PicsViasSlvIpzZvPxs&m=paO6WYeoTil-HWIJVBhwaWcIZP7r95MZNRzKMN6kprU&s=yjtNhJKPJdsHD_A52xWSD2g_YNHuLhO1YADKsPaPsio&e= and I
>> notice that its missing the patch 13 and 17, is this right?
> No, you just have to refine the filter, I marked the DT bindings as
> ready to be applied that's why they do not show up (and by the way
> you can test the series even without those two patches - that are just
> DT bindings).
>
> Thanks,
> Lorenzo
>
>> If not, can you give me both patches location?
>>
>> Thanks.
>>
>>
>> On 13/12/2017 17:21, Joao Pinto wrote:
>>> Hi Niklas,
>>>
>>> ??s 1:59 PM de 12/13/2017, Niklas Cassel escreveu:
>>>> On Thu, Nov 30, 2017 at 03:28:43PM +0000, Lorenzo Pieralisi wrote:
>>>>> Jingoo, Joao,
>>>>>
>>>>> I am expecting your testing on the series and ACKs on the dwc related
>>>>> patches please, according to v4 review - I will mark them as needs
>>>>> review/ACK waiting for you to chime in.
>>>>>
>>>>> v4 thread:
>>>>>
>>>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__patchwork.ozlabs.org_patch_833882_&d=DwIBAg&c=DPL6_X_6JkXFx7AXWqB0tg&r=s2fO0hii0OGNOv9qQy_HRXy-xAJUD1NNoEcc3io_kx0&m=CQAV8rCgm2jd7m3iSWL5vBbGTqSc7yN3N7zeQDz-ZUY&s=mqyDi45qm4qE0jcm0KviYAoLHmMXMMIWEW-gAR0RKOw&e=
>>>>>
>>>> Hello Lorenzo, Jingoo, Joao,
>>>>
>>>> Tomorrow another 2 weeks has passed.
>>>> V1 of this patch series was posted on 2017-10-13.
>>> Sorry, I have been tight up with a debug session and not able to check this out.
>>> Adding Gustavo in CC that is now also working in PCI software development.
>>> I am going to check the code ASAP and we will test it as soon as the debug is
>>> finished.
>>>
>>> Thanks.
>>>
>>>> I'm a bit worried that this patch series will not make it to linux-next
>>>> in time for this patch series to be included in the 4.16 pull request.
>>>>
>>>> Here is the V5 patch series:
>>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__patchwork.ozlabs.org_project_linux-2Dpci_list_-3Fseries-3D14364&d=DwIBAg&c=DPL6_X_6JkXFx7AXWqB0tg&r=s2fO0hii0OGNOv9qQy_HRXy-xAJUD1NNoEcc3io_kx0&m=CQAV8rCgm2jd7m3iSWL5vBbGTqSc7yN3N7zeQDz-ZUY&s=qXtUHFXK7_nCndQob0I0qDByBdJsL7rRjZ9cx7c847U&e=
>>>>
>>>> Perhaps any maintainer of a designware based PCIe driver could
>>>> help out and test this patch series?
>>>>
>>>> Regards,
>>>> Niklas
>>>>
>>

Re: [PATCH v5 01/18] PCI: dwc: Use the DMA-API to get the MSI address

[Lorenzo Pieralisi]

Hi Gustavo,

On Thu, Dec 14, 2017 at 12:38:04PM +0000, Gustavo Pimentel wrote:
> Ok. I will compile and test it now.

Have you managed to retrieve the patches and test them ?

Thank you,
Lorenzo

> Thanks.
> 
> 
> On 14/12/2017 12:22, Lorenzo Pieralisi wrote:
> > On Thu, Dec 14, 2017 at 12:16:38PM +0000, Gustavo Pimentel wrote:
> >> Hi Niklas and Lorenzo,
> >>
> >> I'm going to work on PCI software development now as told by Joao and I will
> >> test your code now.
> >>
> >> I was retrieving the patches through the patchwork
> >> https://urldefense.proofpoint.com/v2/url?u=https-3A__patchwork.ozlabs.org_project_linux-2Dpci_list_-3Fsubmitter-3D65580&d=DwIBAg&c=DPL6_X_6JkXFx7AXWqB0tg&r=bkWxpLoW-f-E3EdiDCCa0_h0PicsViasSlvIpzZvPxs&m=paO6WYeoTil-HWIJVBhwaWcIZP7r95MZNRzKMN6kprU&s=yjtNhJKPJdsHD_A52xWSD2g_YNHuLhO1YADKsPaPsio&e= and I
> >> notice that its missing the patch 13 and 17, is this right?
> > No, you just have to refine the filter, I marked the DT bindings as
> > ready to be applied that's why they do not show up (and by the way
> > you can test the series even without those two patches - that are just
> > DT bindings).
> >
> > Thanks,
> > Lorenzo
> >
> >> If not, can you give me both patches location?
> >>
> >> Thanks.
> >>
> >>
> >> On 13/12/2017 17:21, Joao Pinto wrote:
> >>> Hi Niklas,
> >>>
> >>> ??s 1:59 PM de 12/13/2017, Niklas Cassel escreveu:
> >>>> On Thu, Nov 30, 2017 at 03:28:43PM +0000, Lorenzo Pieralisi wrote:
> >>>>> Jingoo, Joao,
> >>>>>
> >>>>> I am expecting your testing on the series and ACKs on the dwc related
> >>>>> patches please, according to v4 review - I will mark them as needs
> >>>>> review/ACK waiting for you to chime in.
> >>>>>
> >>>>> v4 thread:
> >>>>>
> >>>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__patchwork.ozlabs.org_patch_833882_&d=DwIBAg&c=DPL6_X_6JkXFx7AXWqB0tg&r=s2fO0hii0OGNOv9qQy_HRXy-xAJUD1NNoEcc3io_kx0&m=CQAV8rCgm2jd7m3iSWL5vBbGTqSc7yN3N7zeQDz-ZUY&s=mqyDi45qm4qE0jcm0KviYAoLHmMXMMIWEW-gAR0RKOw&e=
> >>>>>
> >>>> Hello Lorenzo, Jingoo, Joao,
> >>>>
> >>>> Tomorrow another 2 weeks has passed.
> >>>> V1 of this patch series was posted on 2017-10-13.
> >>> Sorry, I have been tight up with a debug session and not able to check this out.
> >>> Adding Gustavo in CC that is now also working in PCI software development.
> >>> I am going to check the code ASAP and we will test it as soon as the debug is
> >>> finished.
> >>>
> >>> Thanks.
> >>>
> >>>> I'm a bit worried that this patch series will not make it to linux-next
> >>>> in time for this patch series to be included in the 4.16 pull request.
> >>>>
> >>>> Here is the V5 patch series:
> >>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__patchwork.ozlabs.org_project_linux-2Dpci_list_-3Fseries-3D14364&d=DwIBAg&c=DPL6_X_6JkXFx7AXWqB0tg&r=s2fO0hii0OGNOv9qQy_HRXy-xAJUD1NNoEcc3io_kx0&m=CQAV8rCgm2jd7m3iSWL5vBbGTqSc7yN3N7zeQDz-ZUY&s=qXtUHFXK7_nCndQob0I0qDByBdJsL7rRjZ9cx7c847U&e=
> >>>>
> >>>> Perhaps any maintainer of a designware based PCIe driver could
> >>>> help out and test this patch series?
> >>>>
> >>>> Regards,
> >>>> Niklas
> >>>>
> >>
> 
> 

Re: [PATCH v5 01/18] PCI: dwc: Use the DMA-API to get the MSI address

[Gustavo Pimentel]

Hi Lorenzo,

Yes, I have retrieve patches and performed some basic tests without having any
problems till now.

However, I'm still trying to test the changes with our automated system. It's
being very difficult to find a time slot available for running this tests.

I hope in this week I could give you go/not go flag. Sorry the delay...

Regards,

Gustavo


On 18/12/2017 15:57, Lorenzo Pieralisi wrote:
> Hi Gustavo,
>
> On Thu, Dec 14, 2017 at 12:38:04PM +0000, Gustavo Pimentel wrote:
>> Ok. I will compile and test it now.
> Have you managed to retrieve the patches and test them ?
>
> Thank you,
> Lorenzo
>
>> Thanks.
>>
>>
>> On 14/12/2017 12:22, Lorenzo Pieralisi wrote:
>>> On Thu, Dec 14, 2017 at 12:16:38PM +0000, Gustavo Pimentel wrote:
>>>> Hi Niklas and Lorenzo,
>>>>
>>>> I'm going to work on PCI software development now as told by Joao and I will
>>>> test your code now.
>>>>
>>>> I was retrieving the patches through the patchwork
>>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__patchwork.ozlabs.org_project_linux-2Dpci_list_-3Fsubmitter-3D65580&d=DwIBAg&c=DPL6_X_6JkXFx7AXWqB0tg&r=bkWxpLoW-f-E3EdiDCCa0_h0PicsViasSlvIpzZvPxs&m=paO6WYeoTil-HWIJVBhwaWcIZP7r95MZNRzKMN6kprU&s=yjtNhJKPJdsHD_A52xWSD2g_YNHuLhO1YADKsPaPsio&e= and I
>>>> notice that its missing the patch 13 and 17, is this right?
>>> No, you just have to refine the filter, I marked the DT bindings as
>>> ready to be applied that's why they do not show up (and by the way
>>> you can test the series even without those two patches - that are just
>>> DT bindings).
>>>
>>> Thanks,
>>> Lorenzo
>>>
>>>> If not, can you give me both patches location?
>>>>
>>>> Thanks.
>>>>
>>>>
>>>> On 13/12/2017 17:21, Joao Pinto wrote:
>>>>> Hi Niklas,
>>>>>
>>>>> ??s 1:59 PM de 12/13/2017, Niklas Cassel escreveu:
>>>>>> On Thu, Nov 30, 2017 at 03:28:43PM +0000, Lorenzo Pieralisi wrote:
>>>>>>> Jingoo, Joao,
>>>>>>>
>>>>>>> I am expecting your testing on the series and ACKs on the dwc related
>>>>>>> patches please, according to v4 review - I will mark them as needs
>>>>>>> review/ACK waiting for you to chime in.
>>>>>>>
>>>>>>> v4 thread:
>>>>>>>
>>>>>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__patchwork.ozlabs.org_patch_833882_&d=DwIBAg&c=DPL6_X_6JkXFx7AXWqB0tg&r=s2fO0hii0OGNOv9qQy_HRXy-xAJUD1NNoEcc3io_kx0&m=CQAV8rCgm2jd7m3iSWL5vBbGTqSc7yN3N7zeQDz-ZUY&s=mqyDi45qm4qE0jcm0KviYAoLHmMXMMIWEW-gAR0RKOw&e=
>>>>>>>
>>>>>> Hello Lorenzo, Jingoo, Joao,
>>>>>>
>>>>>> Tomorrow another 2 weeks has passed.
>>>>>> V1 of this patch series was posted on 2017-10-13.
>>>>> Sorry, I have been tight up with a debug session and not able to check this out.
>>>>> Adding Gustavo in CC that is now also working in PCI software development.
>>>>> I am going to check the code ASAP and we will test it as soon as the debug is
>>>>> finished.
>>>>>
>>>>> Thanks.
>>>>>
>>>>>> I'm a bit worried that this patch series will not make it to linux-next
>>>>>> in time for this patch series to be included in the 4.16 pull request.
>>>>>>
>>>>>> Here is the V5 patch series:
>>>>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__patchwork.ozlabs.org_project_linux-2Dpci_list_-3Fseries-3D14364&d=DwIBAg&c=DPL6_X_6JkXFx7AXWqB0tg&r=s2fO0hii0OGNOv9qQy_HRXy-xAJUD1NNoEcc3io_kx0&m=CQAV8rCgm2jd7m3iSWL5vBbGTqSc7yN3N7zeQDz-ZUY&s=qXtUHFXK7_nCndQob0I0qDByBdJsL7rRjZ9cx7c847U&e=
>>>>>>
>>>>>> Perhaps any maintainer of a designware based PCIe driver could
>>>>>> help out and test this patch series?
>>>>>>
>>>>>> Regards,
>>>>>> Niklas
>>>>>>
>>

Re: [PATCH v5 15/18] PCI: dwc: Make cpu_addr_fixup take struct dw_pcie as argument

[Lorenzo Pieralisi]

On Mon, Nov 20, 2017 at 02:32:18PM +0100, Niklas Cassel wrote:
> There is no need to hard code the cpu to bus address fixup mask.

There is no need or hardcoding it is broken ? There is a difference
between those two.

> The PCIe controller has a global address on the AXI bus, however,
> from the perspective of the PCIe controller, its base starts at 0x0,
> so the local address is 0x0. To get the bus address, simply subtract
> the global address from the cpu address. The global address is taken
> from device tree.

I do not understand this paragraph, I would kindly ask you and Kishon to
explain please this patch and

commit a660083eb06c ("PCI: dwc: designware: Add new *ops* for CPU addr
fixup")

What the cpu_addr_fixup() hook is supposed to do ? And what does the
"addr_space" property represent ?

> Also for ARTPEC-7, hard coding the cpu to bus address fixup mask is
> not possible, since it uses a High Address Bits Look Up Table, which
> means that it can, at runtime, map the PCIe window to an arbitrary
> address in the 32-bit address space.

But you are not changing the ARTPEC-7 mechanism, are you ? I do not
understand this paragraph - I see no change for ARTPEC-7 in this patch.

> This also fixes a bug for ARTPEC-6, where the cpu to bus address
> fixup mask previously was off by one (GENMASK(27, 0), rather than
> GENMASK(28, 0)). This is another reason to calculate the mask by
> using values from device tree.

What this patch does (and how the cpu_addr_fixup() mechanism works)
is not clear to me, please explain, we can rewrite the commit log
with a clear explanation then.

Thanks,
Lorenzo

> Signed-off-by: Niklas Cassel <niklas.cassel@axis.com>
> ---
>  drivers/pci/dwc/pci-dra7xx.c      |  2 +-
>  drivers/pci/dwc/pcie-artpec6.c    | 18 ++++++++++++++----
>  drivers/pci/dwc/pcie-designware.c |  2 +-
>  drivers/pci/dwc/pcie-designware.h |  2 +-
>  4 files changed, 17 insertions(+), 7 deletions(-)
> 
> diff --git a/drivers/pci/dwc/pci-dra7xx.c b/drivers/pci/dwc/pci-dra7xx.c
> index 224ff8affdce..89d87844abb3 100644
> --- a/drivers/pci/dwc/pci-dra7xx.c
> +++ b/drivers/pci/dwc/pci-dra7xx.c
> @@ -110,7 +110,7 @@ static inline void dra7xx_pcie_writel(struct dra7xx_pcie *pcie, u32 offset,
>  	writel(value, pcie->base + offset);
>  }
>  
> -static u64 dra7xx_pcie_cpu_addr_fixup(u64 pci_addr)
> +static u64 dra7xx_pcie_cpu_addr_fixup(struct dw_pcie *pci, u64 pci_addr)
>  {
>  	return pci_addr & DRA7XX_CPU_TO_BUS_ADDR;
>  }
> diff --git a/drivers/pci/dwc/pcie-artpec6.c b/drivers/pci/dwc/pcie-artpec6.c
> index e7de4e4649eb..318a2bd0d97e 100644
> --- a/drivers/pci/dwc/pcie-artpec6.c
> +++ b/drivers/pci/dwc/pcie-artpec6.c
> @@ -67,8 +67,6 @@ static const struct of_device_id artpec6_pcie_of_match[];
>  #define PHY_STATUS			0x118
>  #define  PHY_COSPLLLOCK			BIT(0)
>  
> -#define ARTPEC6_CPU_TO_BUS_ADDR		GENMASK(27, 0)
> -
>  static u32 artpec6_pcie_readl(struct artpec6_pcie *artpec6_pcie, u32 offset)
>  {
>  	u32 val;
> @@ -82,9 +80,21 @@ static void artpec6_pcie_writel(struct artpec6_pcie *artpec6_pcie, u32 offset, u
>  	regmap_write(artpec6_pcie->regmap, offset, val);
>  }
>  
> -static u64 artpec6_pcie_cpu_addr_fixup(u64 pci_addr)
> +static u64 artpec6_pcie_cpu_addr_fixup(struct dw_pcie *pci, u64 pci_addr)
>  {
> -	return pci_addr & ARTPEC6_CPU_TO_BUS_ADDR;
> +	struct artpec6_pcie *artpec6_pcie = to_artpec6_pcie(pci);
> +	struct pcie_port *pp = &pci->pp;
> +	struct dw_pcie_ep *ep = &pci->ep;
> +
> +	switch (artpec6_pcie->mode) {
> +	case DW_PCIE_RC_TYPE:
> +		return pci_addr - pp->cfg0_base;
> +	case DW_PCIE_EP_TYPE:
> +		return pci_addr - ep->phys_base;
> +	default:
> +		dev_err(pci->dev, "UNKNOWN device type\n");
> +	}
> +	return pci_addr;
>  }
>  
>  static int artpec6_pcie_establish_link(struct dw_pcie *pci)
> diff --git a/drivers/pci/dwc/pcie-designware.c b/drivers/pci/dwc/pcie-designware.c
> index 88abdddee2ad..800be7a4f087 100644
> --- a/drivers/pci/dwc/pcie-designware.c
> +++ b/drivers/pci/dwc/pcie-designware.c
> @@ -149,7 +149,7 @@ void dw_pcie_prog_outbound_atu(struct dw_pcie *pci, int index, int type,
>  	u32 retries, val;
>  
>  	if (pci->ops->cpu_addr_fixup)
> -		cpu_addr = pci->ops->cpu_addr_fixup(cpu_addr);
> +		cpu_addr = pci->ops->cpu_addr_fixup(pci, cpu_addr);
>  
>  	if (pci->iatu_unroll_enabled) {
>  		dw_pcie_prog_outbound_atu_unroll(pci, index, type, cpu_addr,
> diff --git a/drivers/pci/dwc/pcie-designware.h b/drivers/pci/dwc/pcie-designware.h
> index 24edac035160..cca5a81c1c74 100644
> --- a/drivers/pci/dwc/pcie-designware.h
> +++ b/drivers/pci/dwc/pcie-designware.h
> @@ -205,7 +205,7 @@ struct dw_pcie_ep {
>  };
>  
>  struct dw_pcie_ops {
> -	u64	(*cpu_addr_fixup)(u64 cpu_addr);
> +	u64	(*cpu_addr_fixup)(struct dw_pcie *pcie, u64 cpu_addr);
>  	u32	(*read_dbi)(struct dw_pcie *pcie, void __iomem *base, u32 reg,
>  			    size_t size);
>  	void	(*write_dbi)(struct dw_pcie *pcie, void __iomem *base, u32 reg,
> -- 
> 2.14.2
> 

Re: [PATCH v5 15/18] PCI: dwc: Make cpu_addr_fixup take struct dw_pcie as argument

[Niklas Cassel]

On Mon, Dec 18, 2017 at 06:10:58PM +0000, Lorenzo Pieralisi wrote:
> On Mon, Nov 20, 2017 at 02:32:18PM +0100, Niklas Cassel wrote:
> > There is no need to hard code the cpu to bus address fixup mask.
> 
> There is no need or hardcoding it is broken ? There is a difference
> between those two.

Well, both :P

The current hardcoding for ARTPEC-6: GENMASK(27, 0), is wrong.
Hardcoding the correct mask GENMASK(28, 0) would be sufficient
for ARTPEC-6.

However, the ARTPEC-7 is a 32-bit CPU (without LPAE), but has
a 64-bit bus. So the ARTPEC-7 can have more devices than fits
in the 32-bit range, therefore a lookup table is placed between
the bus and the CPU, so you can choose what devices to map.
This mapping, which decides what devices to map, is currently
done at boot via devicetree.
So, on ARTPEC-7, it is not possible to hardcode a CPU_TO_BUS
mask, since the PCIe controller's address is only known via DT.

Having a hardcoded value is arguably wrong. It should probably
have been a DT property called something like "cpu-bus-fixup-mask".
However, this property is not needed since we already have
pp->cfg0_base and ep->phys_base, which is derived from already
existing DT properties.

By using pp->cfg0_base and ep->phys_base, we avoid hardcoding a mask
in the driver. This should work for ARTPEC-6, DRA7xx and ARTPEC-7.
I have not changed the code in DRA7xx though, since their existing
code works, but if they want, they could use the same logic as
artpec6_pcie_cpu_addr_fixup, and thus remove their hardcoded mask.

> 
> > The PCIe controller has a global address on the AXI bus, however,
> > from the perspective of the PCIe controller, its base starts at 0x0,
> > so the local address is 0x0. To get the bus address, simply subtract
> > the global address from the cpu address. The global address is taken
> > from device tree.
> 
> I do not understand this paragraph, I would kindly ask you and Kishon to
> explain please this patch and
> 
> commit a660083eb06c ("PCI: dwc: designware: Add new *ops* for CPU addr
> fixup")
> 
> What the cpu_addr_fixup() hook is supposed to do ? And what does the
> "addr_space" property represent ?

The ARTPEC-6 and DRA7xx SoCs both has an interconnect configured is a way
where the local address of the PCIe controller is 0x0.

For ARTPEC-6 the global address of the PCIe controller is 0xf8050000,
so if the CPU writes to address 0xf8050008, from the perspective of the
PCIe controller, it will see a write to address 0x8.

This is normally not a problem, but when reading/writing from an endpoint,
we go via the outbound iATU. The outbound iATU has to be programmed with
a base address (reads/writes in the range [base address + limit] will be
subject to translation).

However, since the address the PCIe controller sees, in reality is
"0xf8050000 + the address the PCIe controller sees", we need to subtract
the global address before programming the base address in the outbound
iATU.

Kishon explains the same thing in commit f4c55c5a3f7f68c0.

For this patch, I tried make the commit message understandable,
without going into too much detail, but there is probably still
room for improvement.

> 
> > Also for ARTPEC-7, hard coding the cpu to bus address fixup mask is
> > not possible, since it uses a High Address Bits Look Up Table, which
> > means that it can, at runtime, map the PCIe window to an arbitrary
> > address in the 32-bit address space.
> 
> But you are not changing the ARTPEC-7 mechanism, are you ? I do not
> understand this paragraph - I see no change for ARTPEC-7 in this patch.

Hopefully this is clarified by the comments above.

> 
> > This also fixes a bug for ARTPEC-6, where the cpu to bus address
> > fixup mask previously was off by one (GENMASK(27, 0), rather than
> > GENMASK(28, 0)). This is another reason to calculate the mask by
> > using values from device tree.
> 
> What this patch does (and how the cpu_addr_fixup() mechanism works)
> is not clear to me, please explain, we can rewrite the commit log
> with a clear explanation then.

Hopefully this is clarified by the comments above.


Regards,
Niklas

Re: [PATCH v5 01/18] PCI: dwc: Use the DMA-API to get the MSI address

[Lorenzo Pieralisi]

On Mon, Nov 20, 2017 at 02:32:04PM +0100, Niklas Cassel wrote:
> Use the DMA-API to get the MSI address. This address will be written to
> our PCI config space and to the register which determines which AXI
> address the DWC IP will spoof for incoming MSI irqs.
> 
> Since it is a PCIe endpoint device, rather than the CPU, that is supposed
> to write to the MSI address, the proper way to get the MSI address is by
> using the DMA API, not by using virt_to_phys().
> 
> Using virt_to_phys() might work on some systems, but using the DMA API
> should work on all systems.
> 
> This is essentially the same thing as allocating a buffer in a driver
> to which the endpoint will write to. To do this, we use the DMA API.
> 
> Signed-off-by: Niklas Cassel <niklas.cassel@axis.com>
> ---
>  drivers/pci/dwc/pcie-designware-host.c | 15 ++++++++++++---
>  drivers/pci/dwc/pcie-designware.h      |  3 ++-
>  2 files changed, 14 insertions(+), 4 deletions(-)
> 
> diff --git a/drivers/pci/dwc/pcie-designware-host.c b/drivers/pci/dwc/pcie-designware-host.c
> index 81e2157a7cfb..33b52fe98a01 100644
> --- a/drivers/pci/dwc/pcie-designware-host.c
> +++ b/drivers/pci/dwc/pcie-designware-host.c
> @@ -83,10 +83,19 @@ irqreturn_t dw_handle_msi_irq(struct pcie_port *pp)
>  
>  void dw_pcie_msi_init(struct pcie_port *pp)
>  {
> +	struct dw_pcie *pci = to_dw_pcie_from_pp(pp);
> +	struct device *dev = pci->dev;
> +	struct page *page;
>  	u64 msi_target;
>  
> -	pp->msi_data = __get_free_pages(GFP_KERNEL, 0);
> -	msi_target = virt_to_phys((void *)pp->msi_data);
> +	page = alloc_page(GFP_KERNEL | GFP_DMA32);

See this thread about GFP_DMA32:

https://patchwork.ozlabs.org/patch/834864/

I need to look back at this set earlier versions, I do not know why
you change the allocation flags but GFP_DMA32 may not provide what
you need - I think you should either remove it or provide a
justification for it given the discussion above.

Lorenzo

> +	pp->msi_data = dma_map_page(dev, page, 0, PAGE_SIZE, DMA_FROM_DEVICE);
> +	if (dma_mapping_error(dev, pp->msi_data)) {
> +		dev_err(dev, "failed to map MSI data\n");
> +		__free_page(page);
> +		return;
> +	}
> +	msi_target = (u64)pp->msi_data;
>  
>  	/* program the msi_data */
>  	dw_pcie_wr_own_conf(pp, PCIE_MSI_ADDR_LO, 4,
> @@ -187,7 +196,7 @@ static void dw_msi_setup_msg(struct pcie_port *pp, unsigned int irq, u32 pos)
>  	if (pp->ops->get_msi_addr)
>  		msi_target = pp->ops->get_msi_addr(pp);
>  	else
> -		msi_target = virt_to_phys((void *)pp->msi_data);
> +		msi_target = (u64)pp->msi_data;
>  
>  	msg.address_lo = (u32)(msi_target & 0xffffffff);
>  	msg.address_hi = (u32)(msi_target >> 32 & 0xffffffff);
> diff --git a/drivers/pci/dwc/pcie-designware.h b/drivers/pci/dwc/pcie-designware.h
> index e5d9d77b778e..ecdede68522a 100644
> --- a/drivers/pci/dwc/pcie-designware.h
> +++ b/drivers/pci/dwc/pcie-designware.h
> @@ -14,6 +14,7 @@
>  #ifndef _PCIE_DESIGNWARE_H
>  #define _PCIE_DESIGNWARE_H
>  
> +#include <linux/dma-mapping.h>
>  #include <linux/irq.h>
>  #include <linux/msi.h>
>  #include <linux/pci.h>
> @@ -168,7 +169,7 @@ struct pcie_port {
>  	const struct dw_pcie_host_ops *ops;
>  	int			msi_irq;
>  	struct irq_domain	*irq_domain;
> -	unsigned long		msi_data;
> +	dma_addr_t		msi_data;
>  	DECLARE_BITMAP(msi_irq_in_use, MAX_MSI_IRQS);
>  };
>  
> -- 
> 2.14.2
> 

Re: [PATCH v5 15/18] PCI: dwc: Make cpu_addr_fixup take struct dw_pcie as argument

[Lorenzo Pieralisi]

On Mon, Dec 18, 2017 at 10:15:43PM +0100, Niklas Cassel wrote:
> On Mon, Dec 18, 2017 at 06:10:58PM +0000, Lorenzo Pieralisi wrote:
> > On Mon, Nov 20, 2017 at 02:32:18PM +0100, Niklas Cassel wrote:
> > > There is no need to hard code the cpu to bus address fixup mask.
> > 
> > There is no need or hardcoding it is broken ? There is a difference
> > between those two.
> 
> Well, both :P
> 
> The current hardcoding for ARTPEC-6: GENMASK(27, 0), is wrong.
> Hardcoding the correct mask GENMASK(28, 0) would be sufficient
> for ARTPEC-6.
> 
> However, the ARTPEC-7 is a 32-bit CPU (without LPAE), but has
> a 64-bit bus. So the ARTPEC-7 can have more devices than fits
> in the 32-bit range, therefore a lookup table is placed between
> the bus and the CPU, so you can choose what devices to map.
> This mapping, which decides what devices to map, is currently
> done at boot via devicetree.
> So, on ARTPEC-7, it is not possible to hardcode a CPU_TO_BUS
> mask, since the PCIe controller's address is only known via DT.
> 
> Having a hardcoded value is arguably wrong. It should probably
> have been a DT property called something like "cpu-bus-fixup-mask".
> However, this property is not needed since we already have
> pp->cfg0_base and ep->phys_base, which is derived from already
> existing DT properties.
> 
> By using pp->cfg0_base and ep->phys_base, we avoid hardcoding a mask
> in the driver. This should work for ARTPEC-6, DRA7xx and ARTPEC-7.
> I have not changed the code in DRA7xx though, since their existing
> code works, but if they want, they could use the same logic as
> artpec6_pcie_cpu_addr_fixup, and thus remove their hardcoded mask.
> 
> > 
> > > The PCIe controller has a global address on the AXI bus, however,
> > > from the perspective of the PCIe controller, its base starts at 0x0,
> > > so the local address is 0x0. To get the bus address, simply subtract
> > > the global address from the cpu address. The global address is taken
> > > from device tree.
> > 
> > I do not understand this paragraph, I would kindly ask you and Kishon to
> > explain please this patch and
> > 
> > commit a660083eb06c ("PCI: dwc: designware: Add new *ops* for CPU addr
> > fixup")
> > 
> > What the cpu_addr_fixup() hook is supposed to do ? And what does the
> > "addr_space" property represent ?
> 
> The ARTPEC-6 and DRA7xx SoCs both has an interconnect configured is a way
> where the local address of the PCIe controller is 0x0.
> 
> For ARTPEC-6 the global address of the PCIe controller is 0xf8050000,
> so if the CPU writes to address 0xf8050008, from the perspective of the
> PCIe controller, it will see a write to address 0x8.
> 
> This is normally not a problem, but when reading/writing from an endpoint,
> we go via the outbound iATU. The outbound iATU has to be programmed with
> a base address (reads/writes in the range [base address + limit] will be
> subject to translation).
> 
> However, since the address the PCIe controller sees, in reality is
> "0xf8050000 + the address the PCIe controller sees", we need to subtract
> the global address before programming the base address in the outbound
> iATU.
> 
> Kishon explains the same thing in commit f4c55c5a3f7f68c0.
> 
> For this patch, I tried make the commit message understandable,
> without going into too much detail, but there is probably still
> room for improvement.
> 
> > 
> > > Also for ARTPEC-7, hard coding the cpu to bus address fixup mask is
> > > not possible, since it uses a High Address Bits Look Up Table, which
> > > means that it can, at runtime, map the PCIe window to an arbitrary
> > > address in the 32-bit address space.
> > 
> > But you are not changing the ARTPEC-7 mechanism, are you ? I do not
> > understand this paragraph - I see no change for ARTPEC-7 in this patch.
> 
> Hopefully this is clarified by the comments above.
> 
> > 
> > > This also fixes a bug for ARTPEC-6, where the cpu to bus address
> > > fixup mask previously was off by one (GENMASK(27, 0), rather than
> > > GENMASK(28, 0)). This is another reason to calculate the mask by
> > > using values from device tree.
> > 
> > What this patch does (and how the cpu_addr_fixup() mechanism works)
> > is not clear to me, please explain, we can rewrite the commit log
> > with a clear explanation then.
> 
> Hopefully this is clarified by the comments above.

It is, thank you - it would be good to convert this thread into commit
log format, you can send it as an update to this patch.

Thanks,
Lorenzo

Re: [PATCH v5 01/18] PCI: dwc: Use the DMA-API to get the MSI address

[Lorenzo Pieralisi]

On Tue, Dec 19, 2017 at 12:45:30PM +0000, Gustavo Pimentel wrote:
>    Tested-By: Gustavo Pimentel <gustavo.pimentel@synopsys.com>
>    Using an arc board with USB and SATA end points with a root complex PCIe
>    IP core version 5.00, I ran the the following tests:
>     - I checked that all end points were listed correctly;
>     - I checked that no PCI related errors was present in dmesg;
>     - I copied a file from arc to the USB flash drive and checked the
>    checksum;
>     - I copied a file from USB flash drive to the ARC and checked the
>    checksum;
>     - I copied a file from arc to the SATA disk and checked the checksum;
>     - I copied a file from SATA disk to the arc and checked the checksum;
>     - I copied a file from USB flash drive to the SATA disk and checked the
>    checksum;
>     - I copied a file from SATA disk to the USB flash drive and checked the
>    checksum;
>     - I checked that no PCI related errors was present in dmesg;
> 
>    Everything seemed normal to me.

Thank you very much. I would still need ACKs from the respective
maintainers (Joao/Jingoo/Kishon) in order to push this upstream please.

Thanks,
Lorenzo

>    On 18/12/2017 16:11, Gustavo Pimentel wrote:
> 
>  Hi Lorenzo,
> 
>  Yes, I have retrieve patches and performed some basic tests without having any
>  problems till now.
> 
>  However, I'm still trying to test the changes with our automated system. It's
>  being very difficult to find a time slot available for running this tests.
> 
>  I hope in this week I could give you go/not go flag. Sorry the delay...
> 
>  Regards,
> 
>  Gustavo
> 
> 
>  On 18/12/2017 15:57, Lorenzo Pieralisi wrote:
> 
>  Hi Gustavo,
> 
>  On Thu, Dec 14, 2017 at 12:38:04PM +0000, Gustavo Pimentel wrote:
> 
>  Ok. I will compile and test it now.
> 
>  Have you managed to retrieve the patches and test them ?
> 
>  Thank you,
>  Lorenzo
> 
> 
>  Thanks.
> 
> 
>  On 14/12/2017 12:22, Lorenzo Pieralisi wrote:
> 
>  On Thu, Dec 14, 2017 at 12:16:38PM +0000, Gustavo Pimentel wrote:
> 
>  Hi Niklas and Lorenzo,
> 
>  I'm going to work on PCI software development now as told by Joao and I will
>  test your code now.
> 
>  I was retrieving the patches through the patchwork
>  https://urldefense.proofpoint.com/v2/url?u=https-3A__patchwork.ozlabs.org_project_linux-2Dpci_list_-3Fsubmitter-3D65580&d=DwIBAg&c=DPL6_X_6JkXFx7AXWqB0tg&r=bkWxpLoW-f-E3EdiDCCa0_h0PicsViasSlvIpzZvPxs&m=paO6WYeoTil-HWIJVBhwaWcIZP7r95MZNRzKMN6kprU&s=yjtNhJKPJdsHD_A52xWSD2g_YNHuLhO1YADKsPaPsio&e= and I
>  notice that its missing the patch 13 and 17, is this right?
> 
>  No, you just have to refine the filter, I marked the DT bindings as
>  ready to be applied that's why they do not show up (and by the way
>  you can test the series even without those two patches - that are just
>  DT bindings).
> 
>  Thanks,
>  Lorenzo
> 
> 
>  If not, can you give me both patches location?
> 
>  Thanks.
> 
> 
>  On 13/12/2017 17:21, Joao Pinto wrote:
> 
>  Hi Niklas,
> 
>  ??s 1:59 PM de 12/13/2017, Niklas Cassel escreveu:
> 
>  On Thu, Nov 30, 2017 at 03:28:43PM +0000, Lorenzo Pieralisi wrote:
> 
>  Jingoo, Joao,
> 
>  I am expecting your testing on the series and ACKs on the dwc related
>  patches please, according to v4 review - I will mark them as needs
>  review/ACK waiting for you to chime in.
> 
>  v4 thread:
> 
>  https://urldefense.proofpoint.com/v2/url?u=https-3A__patchwork.ozlabs.org_patch_833882_&d=DwIBAg&c=DPL6_X_6JkXFx7AXWqB0tg&r=s2fO0hii0OGNOv9qQy_HRXy-xAJUD1NNoEcc3io_kx0&m=CQAV8rCgm2jd7m3iSWL5vBbGTqSc7yN3N7zeQDz-ZUY&s=mqyDi45qm4qE0jcm0KviYAoLHmMXMMIWEW-gAR0RKOw&e=
> 
> 
>  Hello Lorenzo, Jingoo, Joao,
> 
>  Tomorrow another 2 weeks has passed.
>  V1 of this patch series was posted on 2017-10-13.
> 
>  Sorry, I have been tight up with a debug session and not able to check this out.
>  Adding Gustavo in CC that is now also working in PCI software development.
>  I am going to check the code ASAP and we will test it as soon as the debug is
>  finished.
> 
>  Thanks.
> 
> 
>  I'm a bit worried that this patch series will not make it to linux-next
>  in time for this patch series to be included in the 4.16 pull request.
> 
>  Here is the V5 patch series:
>  https://urldefense.proofpoint.com/v2/url?u=https-3A__patchwork.ozlabs.org_project_linux-2Dpci_list_-3Fseries-3D14364&d=DwIBAg&c=DPL6_X_6JkXFx7AXWqB0tg&r=s2fO0hii0OGNOv9qQy_HRXy-xAJUD1NNoEcc3io_kx0&m=CQAV8rCgm2jd7m3iSWL5vBbGTqSc7yN3N7zeQDz-ZUY&s=qXtUHFXK7_nCndQob0I0qDByBdJsL7rRjZ9cx7c847U&e=
> 
>  Perhaps any maintainer of a designware based PCIe driver could
>  help out and test this patch series?
> 
>  Regards,
>  Niklas

Re: [PATCH v5 01/18] PCI: dwc: Use the DMA-API to get the MSI address

[Niklas Cassel]

On Tue, Dec 19, 2017 at 10:19:18AM +0000, Lorenzo Pieralisi wrote:
> On Mon, Nov 20, 2017 at 02:32:04PM +0100, Niklas Cassel wrote:
> > Use the DMA-API to get the MSI address. This address will be written to
> > our PCI config space and to the register which determines which AXI
> > address the DWC IP will spoof for incoming MSI irqs.
> > 
> > Since it is a PCIe endpoint device, rather than the CPU, that is supposed
> > to write to the MSI address, the proper way to get the MSI address is by
> > using the DMA API, not by using virt_to_phys().
> > 
> > Using virt_to_phys() might work on some systems, but using the DMA API
> > should work on all systems.
> > 
> > This is essentially the same thing as allocating a buffer in a driver
> > to which the endpoint will write to. To do this, we use the DMA API.
> > 
> > Signed-off-by: Niklas Cassel <niklas.cassel@axis.com>
> > ---
> >  drivers/pci/dwc/pcie-designware-host.c | 15 ++++++++++++---
> >  drivers/pci/dwc/pcie-designware.h      |  3 ++-
> >  2 files changed, 14 insertions(+), 4 deletions(-)
> > 
> > diff --git a/drivers/pci/dwc/pcie-designware-host.c b/drivers/pci/dwc/pcie-designware-host.c
> > index 81e2157a7cfb..33b52fe98a01 100644
> > --- a/drivers/pci/dwc/pcie-designware-host.c
> > +++ b/drivers/pci/dwc/pcie-designware-host.c
> > @@ -83,10 +83,19 @@ irqreturn_t dw_handle_msi_irq(struct pcie_port *pp)
> >  
> >  void dw_pcie_msi_init(struct pcie_port *pp)
> >  {
> > +	struct dw_pcie *pci = to_dw_pcie_from_pp(pp);
> > +	struct device *dev = pci->dev;
> > +	struct page *page;
> >  	u64 msi_target;
> >  
> > -	pp->msi_data = __get_free_pages(GFP_KERNEL, 0);
> > -	msi_target = virt_to_phys((void *)pp->msi_data);
> > +	page = alloc_page(GFP_KERNEL | GFP_DMA32);
> 
> See this thread about GFP_DMA32:
> 
> https://patchwork.ozlabs.org/patch/834864/
> 
> I need to look back at this set earlier versions, I do not know why
> you change the allocation flags but GFP_DMA32 may not provide what
> you need - I think you should either remove it or provide a
> justification for it given the discussion above.
> 
> Lorenzo

Hello Lorenzo,

I agree, if we want to change this so that the MSI address is guaranteed
to be in the first 4 GB (since some PCIe endpoints only support 32-bit
MSI), we should do so in a separate patch.

Further more, according to the discussion you linked, any such patch
should consider using GFP_DMA instead of GFP_DMA32, since the
ZONE_DMA32 to ZONE_DMA fallback (in case CONFIG_ZONE_DMA32 is not
set) seems to be broken (and has been for several years).

I will submit a new patch set version where I drop GFP_DMA32.


Regards,
Niklas

Re: [PATCH v5 01/18] PCI: dwc: Use the DMA-API to get the MSI address

[Niklas Cassel]

On Tue, Dec 19, 2017 at 12:45:30PM +0000, Gustavo Pimentel wrote:
> Tested-By: Gustavo Pimentel <gustavo.pimentel@synopsys.com>
> 
> Using an arc board with USB and SATA end points with a root complex PCIe IP core
> version 5.00, I ran the the following tests:
>  - I checked that all end points were listed correctly;
>  - I checked that no PCI related errors was present in dmesg;
>  - I copied a file from arc to the USB flash drive and checked the checksum;
>  - I copied a file from USB flash drive to the ARC and checked the checksum;
>  - I copied a file from arc to the SATA disk and checked the checksum;
>  - I copied a file from SATA disk to the arc and checked the checksum;
>  - I copied a file from USB flash drive to the SATA disk and checked the checksum;
>  - I copied a file from SATA disk to the USB flash drive and checked the checksum;
>  - I checked that no PCI related errors was present in dmesg;
> 
> Everything seemed normal to me.

Hello Gustavo,

Thanks a lot for testing.

FYI: I submitted a new patch set to address Lorenzo's review comments,
however, I do not think that we need to redo all the testing, since
now we are using the same GFP flags for MSI as we did before, which
obviously worked.

https://marc.info/?l=devicetree&m=151372652827788&w=2

Regards,
Niklas