[PATCH v10 0/2] PCI: keembay: Add support for Intel Keem Bay

[PATCH v10 0/2] PCI: keembay: Add support for Intel Keem Bay

[srikanth.thokala]

From: Srikanth Thokala <srikanth.thokala@intel.com>

Hi,

The first patch is to document DT bindings for Keem Bay PCIe controller
for both Root Complex and Endpoint modes.

The second patch is the driver file, a glue driver. Keem Bay PCIe
controller is based on DesignWare PCIe IP.

The patch was tested with Keem Bay evaluation module board, with B0
stepping.

Kindly review.

Thanks!
Srikanth

Changes since v9:
- Add 'Reviewed-by: Krzysztof Wilczyński <kw@linux.com>', thanks to Krzysztof.
- Rebased to v5.13-rc5.

Changes since v8:
- Use chained IRQ to handle MSIs, as suggested by Lorenzo Pieralisi.
- Rename *_setup_irq() to *_setup_msi_irq() to be more meaningful.
- Move *_setup_msi_irq() call to *_add_pcie_port() to make it invoke
  only when controller is in Root Complex mode. In Endpoint mode,
  IRQ will be setup by the respective driver which will be based on
  PCIe End Point Framework.

Changes since v7:
- Rename keembay_pcie_ltssm_enable() to align to its functionality.
- Fix other minor comments from "Krzysztof Wilczyński <kw@linux.com>"

Changes since v6:
- Arrange SoB in chronological order.
- Alphabetized and modified status of entry in MAINTAINERS.
- Added a comment to specify the PCIe spec section about the delay.

Changes since v5:
- Rebased to v5.11-rc4.
- Updated maintainers to add myself in DT binding documents.
- Fix checkpatch issues.

Changes since v4:
- Rebased to v5.11-rc1 and retest.

Changes since v3:
- Add Reviewed-by: Rob Herring <robh@kernel.org> tag in dt-bindings
  patch.
- Remove the keembay_pcie_{readl,writel} wrappers. And replace them with
  readl() and writel().
- Remove the dead code related to unused irqs.
- Remove unused definition for unused irqs.
- In keembay_pcie_ep_init(), initialize enabled interrupts to known state.
- Rebased to next-20201215.

Changes since v2:
- In keembay_pcie_probe(), use return keembay_pcie_add_pcie_port(pcie,
  pdev); statement and remove return 0; at the end of the function.

Changes since v1:
- In dt-bindings patch.
  - Fixed indent warning for compatible property.
  - Rename interrupt-names to pcie, pcie_ev, pcie_err and
    pcie_mem_access, similar to the name used in datasheet.
  - Remove device_type, #address-cells and #size-cells property.
  - Remove num-viewport, num-ib-windows and num-ob-windows property.
  - Replace additionalProperties with unevaluatedProperties, for RC
    only.
  - Add dbi2 and atu property.
  - Remove description for regs and interrupts property.
  - Change enum value for num-lanes to 1 and 2 only.
- In driver patch.
  - In Kconfig file, remove dependency on ARM64.
  - Add new define, PCIE_REGS_PCIE_SII_LINK_UP.
  - Remove PCIE_DBI2_MASK.
  - In struct keembay_pcie, declare pci member as struct, not pointer.
    And remove irq number members.
  - Rename and rework keembay_pcie_establish_link(), to
    keembay_pcie_start_link().
  - Remove unneeded BAR disable steps.
  - Remove unused interrupt handlers; keembay_pcie_ev_irq_handler(),
    keembay_pcie_err_irq_handler().
  - Remove keembay_pcie_enable_interrupts().
  - Rework keembay_pcie_setup_irq() and call it from
    keembay_pcie_probe().
  - Remove keembay_pcie_host_init() and make keembay_pcie_host_ops
    empty.
  - Keep and rework keembay_pcie_add_pcie_port() a little.
  - Remove keembay_pcie_add_pcie_ep() and call dw_pcie_ep_init() from
    keembay_pcie_probe().
  - In keembay_pcie_probe(), remove dbi setup as it will be handled in
    dwc common code.
  - In keembay_pcie_link_up(), use return (val &
    PCIE_REGS_PCIE_SII_LINK_UP) == PCIE_REGS_PCIE_SII_LINK_UP.
  - In keembay_pcie_ep_raise_irq(), rework error message for
    PCI_EPC_IRQ_LEGACY and default cases.
- Rebased to next-20201124, that has dwc pci refactoring,
  https://lore.kernel.org/linux-pci/20201105211159.1814485-1-robh@kernel.org/.

Srikanth Thokala (2):
  dt-bindings: PCI: Add Intel Keem Bay PCIe controller
  PCI: keembay: Add support for Intel Keem Bay

 .../bindings/pci/intel,keembay-pcie-ep.yaml   |  69 +++
 .../bindings/pci/intel,keembay-pcie.yaml      |  97 ++++
 MAINTAINERS                                   |   7 +
 drivers/pci/controller/dwc/Kconfig            |  28 ++
 drivers/pci/controller/dwc/Makefile           |   1 +
 drivers/pci/controller/dwc/pcie-keembay.c     | 451 ++++++++++++++++++
 6 files changed, 653 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/pci/intel,keembay-pcie-ep.yaml
 create mode 100644 Documentation/devicetree/bindings/pci/intel,keembay-pcie.yaml
 create mode 100644 drivers/pci/controller/dwc/pcie-keembay.c


base-commit: 614124bea77e452aa6df7a8714e8bc820b489922
-- 
2.17.1

[PATCH v10 1/2] dt-bindings: PCI: Add Intel Keem Bay PCIe controller

[srikanth.thokala]

From: Srikanth Thokala <srikanth.thokala@intel.com>

Document DT bindings for PCIe controller found on Intel Keem Bay SoC.

Signed-off-by: Wan Ahmad Zainie <wan.ahmad.zainie.wan.mohamad@intel.com>
Reviewed-by: Rob Herring <robh@kernel.org>
Signed-off-by: Srikanth Thokala <srikanth.thokala@intel.com>
---
 .../bindings/pci/intel,keembay-pcie-ep.yaml   | 69 +++++++++++++
 .../bindings/pci/intel,keembay-pcie.yaml      | 97 +++++++++++++++++++
 2 files changed, 166 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/pci/intel,keembay-pcie-ep.yaml
 create mode 100644 Documentation/devicetree/bindings/pci/intel,keembay-pcie.yaml

diff --git a/Documentation/devicetree/bindings/pci/intel,keembay-pcie-ep.yaml b/Documentation/devicetree/bindings/pci/intel,keembay-pcie-ep.yaml
new file mode 100644
index 000000000000..e87ff27526ff
--- /dev/null
+++ b/Documentation/devicetree/bindings/pci/intel,keembay-pcie-ep.yaml
@@ -0,0 +1,69 @@
+# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: "http://devicetree.org/schemas/pci/intel,keembay-pcie-ep.yaml#"
+$schema: "http://devicetree.org/meta-schemas/core.yaml#"
+
+title: Intel Keem Bay PCIe controller Endpoint mode
+
+maintainers:
+  - Wan Ahmad Zainie <wan.ahmad.zainie.wan.mohamad@intel.com>
+  - Srikanth Thokala <srikanth.thokala@intel.com>
+
+properties:
+  compatible:
+    const: intel,keembay-pcie-ep
+
+  reg:
+    maxItems: 5
+
+  reg-names:
+    items:
+      - const: dbi
+      - const: dbi2
+      - const: atu
+      - const: addr_space
+      - const: apb
+
+  interrupts:
+    maxItems: 4
+
+  interrupt-names:
+    items:
+      - const: pcie
+      - const: pcie_ev
+      - const: pcie_err
+      - const: pcie_mem_access
+
+  num-lanes:
+    description: Number of lanes to use.
+    enum: [ 1, 2 ]
+
+required:
+  - compatible
+  - reg
+  - reg-names
+  - interrupts
+  - interrupt-names
+
+additionalProperties: false
+
+examples:
+  - |
+    #include <dt-bindings/interrupt-controller/arm-gic.h>
+    #include <dt-bindings/interrupt-controller/irq.h>
+    pcie-ep@37000000 {
+          compatible = "intel,keembay-pcie-ep";
+          reg = <0x37000000 0x00001000>,
+                <0x37100000 0x00001000>,
+                <0x37300000 0x00001000>,
+                <0x36000000 0x01000000>,
+                <0x37800000 0x00000200>;
+          reg-names = "dbi", "dbi2", "atu", "addr_space", "apb";
+          interrupts = <GIC_SPI 107 IRQ_TYPE_LEVEL_HIGH>,
+                       <GIC_SPI 108 IRQ_TYPE_EDGE_RISING>,
+                       <GIC_SPI 109 IRQ_TYPE_LEVEL_HIGH>,
+                       <GIC_SPI 110 IRQ_TYPE_LEVEL_HIGH>;
+          interrupt-names = "pcie", "pcie_ev", "pcie_err", "pcie_mem_access";
+          num-lanes = <2>;
+    };
diff --git a/Documentation/devicetree/bindings/pci/intel,keembay-pcie.yaml b/Documentation/devicetree/bindings/pci/intel,keembay-pcie.yaml
new file mode 100644
index 000000000000..ed4400c9ac09
--- /dev/null
+++ b/Documentation/devicetree/bindings/pci/intel,keembay-pcie.yaml
@@ -0,0 +1,97 @@
+# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: "http://devicetree.org/schemas/pci/intel,keembay-pcie.yaml#"
+$schema: "http://devicetree.org/meta-schemas/core.yaml#"
+
+title: Intel Keem Bay PCIe controller Root Complex mode
+
+maintainers:
+  - Wan Ahmad Zainie <wan.ahmad.zainie.wan.mohamad@intel.com>
+  - Srikanth Thokala <srikanth.thokala@intel.com>
+
+allOf:
+  - $ref: /schemas/pci/pci-bus.yaml#
+
+properties:
+  compatible:
+    const: intel,keembay-pcie
+
+  ranges:
+    maxItems: 1
+
+  reset-gpios:
+    maxItems: 1
+
+  reg:
+    maxItems: 4
+
+  reg-names:
+    items:
+      - const: dbi
+      - const: atu
+      - const: config
+      - const: apb
+
+  clocks:
+    maxItems: 2
+
+  clock-names:
+    items:
+      - const: master
+      - const: aux
+
+  interrupts:
+    maxItems: 3
+
+  interrupt-names:
+    items:
+      - const: pcie
+      - const: pcie_ev
+      - const: pcie_err
+
+  num-lanes:
+    description: Number of lanes to use.
+    enum: [ 1, 2 ]
+
+required:
+  - compatible
+  - reg
+  - reg-names
+  - ranges
+  - clocks
+  - clock-names
+  - interrupts
+  - interrupt-names
+  - reset-gpios
+
+unevaluatedProperties: false
+
+examples:
+  - |
+    #include <dt-bindings/interrupt-controller/arm-gic.h>
+    #include <dt-bindings/interrupt-controller/irq.h>
+    #include <dt-bindings/gpio/gpio.h>
+    #define KEEM_BAY_A53_PCIE
+    #define KEEM_BAY_A53_AUX_PCIE
+    pcie@37000000 {
+          compatible = "intel,keembay-pcie";
+          reg = <0x37000000 0x00001000>,
+                <0x37300000 0x00001000>,
+                <0x36e00000 0x00200000>,
+                <0x37800000 0x00000200>;
+          reg-names = "dbi", "atu", "config", "apb";
+          #address-cells = <3>;
+          #size-cells = <2>;
+          device_type = "pci";
+          ranges = <0x02000000 0 0x36000000 0x36000000 0 0x00e00000>;
+          interrupts = <GIC_SPI 107 IRQ_TYPE_LEVEL_HIGH>,
+                       <GIC_SPI 108 IRQ_TYPE_LEVEL_HIGH>,
+                       <GIC_SPI 109 IRQ_TYPE_LEVEL_HIGH>;
+          interrupt-names = "pcie", "pcie_ev", "pcie_err";
+          clocks = <&scmi_clk KEEM_BAY_A53_PCIE>,
+                   <&scmi_clk KEEM_BAY_A53_AUX_PCIE>;
+          clock-names = "master", "aux";
+          reset-gpios = <&pca2 9 GPIO_ACTIVE_LOW>;
+          num-lanes = <2>;
+    };
-- 
2.17.1

[PATCH v10 2/2] PCI: keembay: Add support for Intel Keem Bay

[srikanth.thokala]

From: Srikanth Thokala <srikanth.thokala@intel.com>

Add driver for Intel Keem Bay SoC PCIe controller. This controller
is based on DesignWare PCIe core.

In Root Complex mode, only internal reference clock is possible for
Keem Bay A0. For Keem Bay B0, external reference clock can be used
and will be the default configuration. Currently, keembay_pcie_of_data
structure has one member. It will be expanded later to handle this
difference.

Endpoint mode link initialization is handled by the boot firmware.

Signed-off-by: Wan Ahmad Zainie <wan.ahmad.zainie.wan.mohamad@intel.com>
Acked-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: Srikanth Thokala <srikanth.thokala@intel.com>
Reviewed-by: Krzysztof Wilczyński <kw@linux.com>
---
 MAINTAINERS                               |   7 +
 drivers/pci/controller/dwc/Kconfig        |  28 ++
 drivers/pci/controller/dwc/Makefile       |   1 +
 drivers/pci/controller/dwc/pcie-keembay.c | 451 ++++++++++++++++++++++
 4 files changed, 487 insertions(+)
 create mode 100644 drivers/pci/controller/dwc/pcie-keembay.c

diff --git a/MAINTAINERS b/MAINTAINERS
index b706dd20ff2b..40d1fa0b0a0d 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -14244,6 +14244,13 @@ S:	Maintained
 F:	Documentation/devicetree/bindings/pci/hisilicon-histb-pcie.txt
 F:	drivers/pci/controller/dwc/pcie-histb.c
 
+PCIE DRIVER FOR INTEL KEEM BAY
+M:	Srikanth Thokala <srikanth.thokala@intel.com>
+L:	linux-pci@vger.kernel.org
+S:	Supported
+F:	Documentation/devicetree/bindings/pci/intel,keembay-pcie*
+F:	drivers/pci/controller/dwc/pcie-keembay.c
+
 PCIE DRIVER FOR MEDIATEK
 M:	Ryder Lee <ryder.lee@mediatek.com>
 M:	Jianjun Wang <jianjun.wang@mediatek.com>
diff --git a/drivers/pci/controller/dwc/Kconfig b/drivers/pci/controller/dwc/Kconfig
index 423d35872ce4..04430ddde8c4 100644
--- a/drivers/pci/controller/dwc/Kconfig
+++ b/drivers/pci/controller/dwc/Kconfig
@@ -225,6 +225,34 @@ config PCIE_INTEL_GW
 	  The PCIe controller uses the DesignWare core plus Intel-specific
 	  hardware wrappers.
 
+config PCIE_KEEMBAY
+	bool
+
+config PCIE_KEEMBAY_HOST
+	bool "Intel Keem Bay PCIe controller - Host mode"
+	depends on ARCH_KEEMBAY || COMPILE_TEST
+	depends on PCI && PCI_MSI_IRQ_DOMAIN
+	select PCIE_DW_HOST
+	select PCIE_KEEMBAY
+	help
+	  Say 'Y' here to enable support for the PCIe controller in Keem Bay
+	  to work in host mode.
+	  The PCIe controller is based on DesignWare Hardware and uses
+	  DesignWare core functions.
+
+config PCIE_KEEMBAY_EP
+	bool "Intel Keem Bay PCIe controller - Endpoint mode"
+	depends on ARCH_KEEMBAY || COMPILE_TEST
+	depends on PCI && PCI_MSI_IRQ_DOMAIN
+	depends on PCI_ENDPOINT
+	select PCIE_DW_EP
+	select PCIE_KEEMBAY
+	help
+	  Say 'Y' here to enable support for the PCIe controller in Keem Bay
+	  to work in endpoint mode.
+	  The PCIe controller is based on DesignWare Hardware and uses
+	  DesignWare core functions.
+
 config PCIE_KIRIN
 	depends on OF && (ARM64 || COMPILE_TEST)
 	bool "HiSilicon Kirin series SoCs PCIe controllers"
diff --git a/drivers/pci/controller/dwc/Makefile b/drivers/pci/controller/dwc/Makefile
index eca805c1a023..6932292b3f22 100644
--- a/drivers/pci/controller/dwc/Makefile
+++ b/drivers/pci/controller/dwc/Makefile
@@ -15,6 +15,7 @@ obj-$(CONFIG_PCIE_QCOM) += pcie-qcom.o
 obj-$(CONFIG_PCIE_ARMADA_8K) += pcie-armada8k.o
 obj-$(CONFIG_PCIE_ARTPEC6) += pcie-artpec6.o
 obj-$(CONFIG_PCIE_INTEL_GW) += pcie-intel-gw.o
+obj-$(CONFIG_PCIE_KEEMBAY) += pcie-keembay.o
 obj-$(CONFIG_PCIE_KIRIN) += pcie-kirin.o
 obj-$(CONFIG_PCIE_HISI_STB) += pcie-histb.o
 obj-$(CONFIG_PCI_MESON) += pci-meson.o
diff --git a/drivers/pci/controller/dwc/pcie-keembay.c b/drivers/pci/controller/dwc/pcie-keembay.c
new file mode 100644
index 000000000000..91ebb461c8a3
--- /dev/null
+++ b/drivers/pci/controller/dwc/pcie-keembay.c
@@ -0,0 +1,451 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * PCIe controller driver for Intel Keem Bay
+ * Copyright (C) 2020 Intel Corporation
+ */
+
+#include <linux/bitfield.h>
+#include <linux/bits.h>
+#include <linux/clk.h>
+#include <linux/delay.h>
+#include <linux/err.h>
+#include <linux/gpio/consumer.h>
+#include <linux/init.h>
+#include <linux/iopoll.h>
+#include <linux/irqchip/chained_irq.h>
+#include <linux/kernel.h>
+#include <linux/mod_devicetable.h>
+#include <linux/pci.h>
+#include <linux/platform_device.h>
+#include <linux/property.h>
+
+#include "pcie-designware.h"
+
+/* PCIE_REGS_APB_SLV Registers */
+#define PCIE_REGS_PCIE_CFG		0x0004
+#define  PCIE_DEVICE_TYPE		BIT(8)
+#define  PCIE_RSTN			BIT(0)
+#define PCIE_REGS_PCIE_APP_CNTRL	0x0008
+#define  APP_LTSSM_ENABLE		BIT(0)
+#define PCIE_REGS_INTERRUPT_ENABLE	0x0028
+#define  MSI_CTRL_INT_EN		BIT(8)
+#define  EDMA_INT_EN			GENMASK(7, 0)
+#define PCIE_REGS_INTERRUPT_STATUS	0x002c
+#define  MSI_CTRL_INT			BIT(8)
+#define PCIE_REGS_PCIE_SII_PM_STATE	0x00b0
+#define  SMLH_LINK_UP			BIT(19)
+#define  RDLH_LINK_UP			BIT(8)
+#define  PCIE_REGS_PCIE_SII_LINK_UP	(SMLH_LINK_UP | RDLH_LINK_UP)
+#define PCIE_REGS_PCIE_PHY_CNTL		0x0164
+#define  PHY0_SRAM_BYPASS		BIT(8)
+#define PCIE_REGS_PCIE_PHY_STAT		0x0168
+#define  PHY0_MPLLA_STATE		BIT(1)
+#define PCIE_REGS_LJPLL_STA		0x016c
+#define  LJPLL_LOCK			BIT(0)
+#define PCIE_REGS_LJPLL_CNTRL_0		0x0170
+#define  LJPLL_EN			BIT(29)
+#define  LJPLL_FOUT_EN			GENMASK(24, 21)
+#define PCIE_REGS_LJPLL_CNTRL_2		0x0178
+#define  LJPLL_REF_DIV			GENMASK(17, 12)
+#define  LJPLL_FB_DIV			GENMASK(11, 0)
+#define PCIE_REGS_LJPLL_CNTRL_3		0x017c
+#define  LJPLL_POST_DIV3A		GENMASK(24, 22)
+#define  LJPLL_POST_DIV2A		GENMASK(18, 16)
+
+#define PERST_DELAY_US		1000
+#define AUX_CLK_RATE_HZ		24000000
+
+struct keembay_pcie {
+	struct dw_pcie		pci;
+	void __iomem		*apb_base;
+	enum dw_pcie_device_mode mode;
+
+	struct clk		*clk_master;
+	struct clk		*clk_aux;
+	struct gpio_desc	*reset;
+};
+
+struct keembay_pcie_of_data {
+	enum dw_pcie_device_mode mode;
+};
+
+static void keembay_ep_reset_assert(struct keembay_pcie *pcie)
+{
+	gpiod_set_value_cansleep(pcie->reset, 1);
+	usleep_range(PERST_DELAY_US, PERST_DELAY_US + 500);
+}
+
+static void keembay_ep_reset_deassert(struct keembay_pcie *pcie)
+{
+	/*
+	 * Ensure that PERST# is asserted for a minimum of 100ms.
+	 *
+	 * For more details, refer to PCI Express Card Electromechanical
+	 * Specification Revision 1.1, Table-2.4.
+	 */
+	msleep(100);
+
+	gpiod_set_value_cansleep(pcie->reset, 0);
+	usleep_range(PERST_DELAY_US, PERST_DELAY_US + 500);
+}
+
+static void keembay_pcie_ltssm_set(struct keembay_pcie *pcie, bool enable)
+{
+	u32 val;
+
+	val = readl(pcie->apb_base + PCIE_REGS_PCIE_APP_CNTRL);
+	if (enable)
+		val |= APP_LTSSM_ENABLE;
+	else
+		val &= ~APP_LTSSM_ENABLE;
+	writel(val, pcie->apb_base + PCIE_REGS_PCIE_APP_CNTRL);
+}
+
+static int keembay_pcie_link_up(struct dw_pcie *pci)
+{
+	struct keembay_pcie *pcie = dev_get_drvdata(pci->dev);
+	u32 val;
+
+	val = readl(pcie->apb_base + PCIE_REGS_PCIE_SII_PM_STATE);
+
+	return (val & PCIE_REGS_PCIE_SII_LINK_UP) == PCIE_REGS_PCIE_SII_LINK_UP;
+}
+
+static int keembay_pcie_start_link(struct dw_pcie *pci)
+{
+	struct keembay_pcie *pcie = dev_get_drvdata(pci->dev);
+	u32 val;
+	int ret;
+
+	if (pcie->mode == DW_PCIE_EP_TYPE)
+		return 0;
+
+	keembay_pcie_ltssm_set(pcie, false);
+
+	ret = readl_poll_timeout(pcie->apb_base + PCIE_REGS_PCIE_PHY_STAT,
+				 val, val & PHY0_MPLLA_STATE, 20,
+				 500 * USEC_PER_MSEC);
+	if (ret) {
+		dev_err(pci->dev, "MPLLA is not locked\n");
+		return ret;
+	}
+
+	keembay_pcie_ltssm_set(pcie, true);
+
+	return 0;
+}
+
+static void keembay_pcie_stop_link(struct dw_pcie *pci)
+{
+	struct keembay_pcie *pcie = dev_get_drvdata(pci->dev);
+
+	keembay_pcie_ltssm_set(pcie, false);
+}
+
+static const struct dw_pcie_ops keembay_pcie_ops = {
+	.link_up	= keembay_pcie_link_up,
+	.start_link	= keembay_pcie_start_link,
+	.stop_link	= keembay_pcie_stop_link,
+};
+
+static inline struct clk *keembay_pcie_probe_clock(struct device *dev,
+						   const char *id, u64 rate)
+{
+	struct clk *clk;
+	int ret;
+
+	clk = devm_clk_get(dev, id);
+	if (IS_ERR(clk))
+		return clk;
+
+	if (rate) {
+		ret = clk_set_rate(clk, rate);
+		if (ret)
+			return ERR_PTR(ret);
+	}
+
+	ret = clk_prepare_enable(clk);
+	if (ret)
+		return ERR_PTR(ret);
+
+	ret = devm_add_action_or_reset(dev,
+				       (void(*)(void *))clk_disable_unprepare,
+				       clk);
+	if (ret)
+		return ERR_PTR(ret);
+
+	return clk;
+}
+
+static int keembay_pcie_probe_clocks(struct keembay_pcie *pcie)
+{
+	struct dw_pcie *pci = &pcie->pci;
+	struct device *dev = pci->dev;
+
+	pcie->clk_master = keembay_pcie_probe_clock(dev, "master", 0);
+	if (IS_ERR(pcie->clk_master))
+		return dev_err_probe(dev, PTR_ERR(pcie->clk_master),
+				     "Failed to enable master clock");
+
+	pcie->clk_aux = keembay_pcie_probe_clock(dev, "aux", AUX_CLK_RATE_HZ);
+	if (IS_ERR(pcie->clk_aux))
+		return dev_err_probe(dev, PTR_ERR(pcie->clk_aux),
+				     "Failed to enable auxiliary clock");
+
+	return 0;
+}
+
+/*
+ * Initialize the internal PCIe PLL in Host mode.
+ * See the following sections in Keem Bay data book,
+ * (1) 6.4.6.1 PCIe Subsystem Example Initialization,
+ * (2) 6.8 PCIe Low Jitter PLL for Ref Clk Generation.
+ */
+static int keembay_pcie_pll_init(struct keembay_pcie *pcie)
+{
+	struct dw_pcie *pci = &pcie->pci;
+	u32 val;
+	int ret;
+
+	val = FIELD_PREP(LJPLL_REF_DIV, 0) | FIELD_PREP(LJPLL_FB_DIV, 0x32);
+	writel(val, pcie->apb_base + PCIE_REGS_LJPLL_CNTRL_2);
+
+	val = FIELD_PREP(LJPLL_POST_DIV3A, 0x2) |
+		FIELD_PREP(LJPLL_POST_DIV2A, 0x2);
+	writel(val, pcie->apb_base + PCIE_REGS_LJPLL_CNTRL_3);
+
+	val = FIELD_PREP(LJPLL_EN, 0x1) | FIELD_PREP(LJPLL_FOUT_EN, 0xc);
+	writel(val, pcie->apb_base + PCIE_REGS_LJPLL_CNTRL_0);
+
+	ret = readl_poll_timeout(pcie->apb_base + PCIE_REGS_LJPLL_STA,
+				 val, val & LJPLL_LOCK, 20,
+				 500 * USEC_PER_MSEC);
+	if (ret)
+		dev_err(pci->dev, "Low jitter PLL is not locked\n");
+
+	return ret;
+}
+
+static void keembay_pcie_msi_irq_handler(struct irq_desc *desc)
+{
+	struct keembay_pcie *pcie = irq_desc_get_handler_data(desc);
+	struct irq_chip *chip = irq_desc_get_chip(desc);
+	u32 val, mask, status;
+	struct pcie_port *pp;
+
+	chained_irq_enter(chip, desc);
+
+	pp = &pcie->pci.pp;
+	val = readl(pcie->apb_base + PCIE_REGS_INTERRUPT_STATUS);
+	mask = readl(pcie->apb_base + PCIE_REGS_INTERRUPT_ENABLE);
+
+	status = val & mask;
+
+	if (status & MSI_CTRL_INT) {
+		dw_handle_msi_irq(pp);
+		writel(status, pcie->apb_base + PCIE_REGS_INTERRUPT_STATUS);
+	}
+
+	chained_irq_exit(chip, desc);
+}
+
+static int keembay_pcie_setup_msi_irq(struct keembay_pcie *pcie)
+{
+	struct dw_pcie *pci = &pcie->pci;
+	struct device *dev = pci->dev;
+	struct platform_device *pdev = to_platform_device(dev);
+	int irq;
+
+	irq = platform_get_irq_byname(pdev, "pcie");
+	if (irq < 0)
+		return irq;
+
+	irq_set_chained_handler_and_data(irq, keembay_pcie_msi_irq_handler,
+					 pcie);
+
+	return 0;
+}
+
+static void keembay_pcie_ep_init(struct dw_pcie_ep *ep)
+{
+	struct dw_pcie *pci = to_dw_pcie_from_ep(ep);
+	struct keembay_pcie *pcie = dev_get_drvdata(pci->dev);
+
+	writel(EDMA_INT_EN, pcie->apb_base + PCIE_REGS_INTERRUPT_ENABLE);
+}
+
+static int keembay_pcie_ep_raise_irq(struct dw_pcie_ep *ep, u8 func_no,
+				     enum pci_epc_irq_type type,
+				     u16 interrupt_num)
+{
+	struct dw_pcie *pci = to_dw_pcie_from_ep(ep);
+
+	switch (type) {
+	case PCI_EPC_IRQ_LEGACY:
+		/* Legacy interrupts are not supported in Keem Bay */
+		dev_err(pci->dev, "Legacy IRQ is not supported\n");
+		return -EINVAL;
+	case PCI_EPC_IRQ_MSI:
+		return dw_pcie_ep_raise_msi_irq(ep, func_no, interrupt_num);
+	case PCI_EPC_IRQ_MSIX:
+		return dw_pcie_ep_raise_msix_irq(ep, func_no, interrupt_num);
+	default:
+		dev_err(pci->dev, "Unknown IRQ type %d\n", type);
+		return -EINVAL;
+	}
+}
+
+static const struct pci_epc_features keembay_pcie_epc_features = {
+	.linkup_notifier	= false,
+	.msi_capable		= true,
+	.msix_capable		= true,
+	.reserved_bar		= BIT(BAR_1) | BIT(BAR_3) | BIT(BAR_5),
+	.bar_fixed_64bit	= BIT(BAR_0) | BIT(BAR_2) | BIT(BAR_4),
+	.align			= SZ_16K,
+};
+
+static const struct pci_epc_features *
+keembay_pcie_get_features(struct dw_pcie_ep *ep)
+{
+	return &keembay_pcie_epc_features;
+}
+
+static const struct dw_pcie_ep_ops keembay_pcie_ep_ops = {
+	.ep_init	= keembay_pcie_ep_init,
+	.raise_irq	= keembay_pcie_ep_raise_irq,
+	.get_features	= keembay_pcie_get_features,
+};
+
+static const struct dw_pcie_host_ops keembay_pcie_host_ops = {
+};
+
+static int keembay_pcie_add_pcie_port(struct keembay_pcie *pcie,
+				      struct platform_device *pdev)
+{
+	struct dw_pcie *pci = &pcie->pci;
+	struct pcie_port *pp = &pci->pp;
+	struct device *dev = &pdev->dev;
+	u32 val;
+	int ret;
+
+	pp->ops = &keembay_pcie_host_ops;
+	pp->msi_irq = -ENODEV;
+
+	ret = keembay_pcie_setup_msi_irq(pcie);
+	if (ret)
+		return ret;
+
+	pcie->reset = devm_gpiod_get(dev, "reset", GPIOD_OUT_HIGH);
+	if (IS_ERR(pcie->reset))
+		return PTR_ERR(pcie->reset);
+
+	ret = keembay_pcie_probe_clocks(pcie);
+	if (ret)
+		return ret;
+
+	val = readl(pcie->apb_base + PCIE_REGS_PCIE_PHY_CNTL);
+	val |= PHY0_SRAM_BYPASS;
+	writel(val, pcie->apb_base + PCIE_REGS_PCIE_PHY_CNTL);
+
+	writel(PCIE_DEVICE_TYPE, pcie->apb_base + PCIE_REGS_PCIE_CFG);
+
+	ret = keembay_pcie_pll_init(pcie);
+	if (ret)
+		return ret;
+
+	val = readl(pcie->apb_base + PCIE_REGS_PCIE_CFG);
+	writel(val | PCIE_RSTN, pcie->apb_base + PCIE_REGS_PCIE_CFG);
+	keembay_ep_reset_deassert(pcie);
+
+	ret = dw_pcie_host_init(pp);
+	if (ret) {
+		keembay_ep_reset_assert(pcie);
+		dev_err(dev, "Failed to initialize host: %d\n", ret);
+		return ret;
+	}
+
+	val = readl(pcie->apb_base + PCIE_REGS_INTERRUPT_ENABLE);
+	if (IS_ENABLED(CONFIG_PCI_MSI))
+		val |= MSI_CTRL_INT_EN;
+	writel(val, pcie->apb_base + PCIE_REGS_INTERRUPT_ENABLE);
+
+	return 0;
+}
+
+static int keembay_pcie_probe(struct platform_device *pdev)
+{
+	const struct keembay_pcie_of_data *data;
+	struct device *dev = &pdev->dev;
+	struct keembay_pcie *pcie;
+	struct dw_pcie *pci;
+	enum dw_pcie_device_mode mode;
+
+	data = device_get_match_data(dev);
+	if (!data)
+		return -ENODEV;
+
+	mode = (enum dw_pcie_device_mode)data->mode;
+
+	pcie = devm_kzalloc(dev, sizeof(*pcie), GFP_KERNEL);
+	if (!pcie)
+		return -ENOMEM;
+
+	pci = &pcie->pci;
+	pci->dev = dev;
+	pci->ops = &keembay_pcie_ops;
+
+	pcie->mode = mode;
+
+	pcie->apb_base = devm_platform_ioremap_resource_byname(pdev, "apb");
+	if (IS_ERR(pcie->apb_base))
+		return PTR_ERR(pcie->apb_base);
+
+	platform_set_drvdata(pdev, pcie);
+
+	switch (pcie->mode) {
+	case DW_PCIE_RC_TYPE:
+		if (!IS_ENABLED(CONFIG_PCIE_KEEMBAY_HOST))
+			return -ENODEV;
+
+		return keembay_pcie_add_pcie_port(pcie, pdev);
+	case DW_PCIE_EP_TYPE:
+		if (!IS_ENABLED(CONFIG_PCIE_KEEMBAY_EP))
+			return -ENODEV;
+
+		pci->ep.ops = &keembay_pcie_ep_ops;
+		return dw_pcie_ep_init(&pci->ep);
+	default:
+		dev_err(dev, "Invalid device type %d\n", pcie->mode);
+		return -ENODEV;
+	}
+}
+
+static const struct keembay_pcie_of_data keembay_pcie_rc_of_data = {
+	.mode = DW_PCIE_RC_TYPE,
+};
+
+static const struct keembay_pcie_of_data keembay_pcie_ep_of_data = {
+	.mode = DW_PCIE_EP_TYPE,
+};
+
+static const struct of_device_id keembay_pcie_of_match[] = {
+	{
+		.compatible = "intel,keembay-pcie",
+		.data = &keembay_pcie_rc_of_data,
+	},
+	{
+		.compatible = "intel,keembay-pcie-ep",
+		.data = &keembay_pcie_ep_of_data,
+	},
+	{}
+};
+
+static struct platform_driver keembay_pcie_driver = {
+	.driver = {
+		.name = "keembay-pcie",
+		.of_match_table = keembay_pcie_of_match,
+		.suppress_bind_attrs = true,
+	},
+	.probe  = keembay_pcie_probe,
+};
+builtin_platform_driver(keembay_pcie_driver);
-- 
2.17.1

RE: [PATCH v10 0/2] PCI: keembay: Add support for Intel Keem Bay

[Thokala, Srikanth]

Hi Lorenzo Pieralisi and Bjorn Helgaas,

> -----Original Message-----
> From: Thokala, Srikanth <srikanth.thokala@intel.com>
> Sent: Monday, June 7, 2021 9:11 PM
> To: robh+dt@kernel.org; lorenzo.pieralisi@arm.com
> Cc: linux-pci@vger.kernel.org; devicetree@vger.kernel.org;
> andriy.shevchenko@linux.intel.com; mgross@linux.intel.com; Raja
> Subramanian, Lakshmi Bai <lakshmi.bai.raja.subramanian@intel.com>;
> Sangannavar, Mallikarjunappa <mallikarjunappa.sangannavar@intel.com>;
> kw@linux.com; Thokala, Srikanth <srikanth.thokala@intel.com>
> Subject: [PATCH v10 0/2] PCI: keembay: Add support for Intel Keem Bay
> 
> From: Srikanth Thokala <srikanth.thokala@intel.com>
> 
> Hi,
> 
> The first patch is to document DT bindings for Keem Bay PCIe controller
> for both Root Complex and Endpoint modes.
> 
> The second patch is the driver file, a glue driver. Keem Bay PCIe
> controller is based on DesignWare PCIe IP.
> 
> The patch was tested with Keem Bay evaluation module board, with B0
> stepping.
> 
> Kindly review.

Kindly review the patch series.
I have got 'Reviewed-by' for both patches in the series and please let
me know if you have any comments that I need to address further.

Thanks!
Srikanth

> 
> Thanks!
> Srikanth
> 
> Changes since v9:
> - Add 'Reviewed-by: Krzysztof Wilczyński <kw@linux.com>', thanks to
> Krzysztof.
> - Rebased to v5.13-rc5.
> 
> Changes since v8:
> - Use chained IRQ to handle MSIs, as suggested by Lorenzo Pieralisi.
> - Rename *_setup_irq() to *_setup_msi_irq() to be more meaningful.
> - Move *_setup_msi_irq() call to *_add_pcie_port() to make it invoke
>   only when controller is in Root Complex mode. In Endpoint mode,
>   IRQ will be setup by the respective driver which will be based on
>   PCIe End Point Framework.
> 
> Changes since v7:
> - Rename keembay_pcie_ltssm_enable() to align to its functionality.
> - Fix other minor comments from "Krzysztof Wilczyński <kw@linux.com>"
> 
> Changes since v6:
> - Arrange SoB in chronological order.
> - Alphabetized and modified status of entry in MAINTAINERS.
> - Added a comment to specify the PCIe spec section about the delay.
> 
> Changes since v5:
> - Rebased to v5.11-rc4.
> - Updated maintainers to add myself in DT binding documents.
> - Fix checkpatch issues.
> 
> Changes since v4:
> - Rebased to v5.11-rc1 and retest.
> 
> Changes since v3:
> - Add Reviewed-by: Rob Herring <robh@kernel.org> tag in dt-bindings
>   patch.
> - Remove the keembay_pcie_{readl,writel} wrappers. And replace them with
>   readl() and writel().
> - Remove the dead code related to unused irqs.
> - Remove unused definition for unused irqs.
> - In keembay_pcie_ep_init(), initialize enabled interrupts to known state.
> - Rebased to next-20201215.
> 
> Changes since v2:
> - In keembay_pcie_probe(), use return keembay_pcie_add_pcie_port(pcie,
>   pdev); statement and remove return 0; at the end of the function.
> 
> Changes since v1:
> - In dt-bindings patch.
>   - Fixed indent warning for compatible property.
>   - Rename interrupt-names to pcie, pcie_ev, pcie_err and
>     pcie_mem_access, similar to the name used in datasheet.
>   - Remove device_type, #address-cells and #size-cells property.
>   - Remove num-viewport, num-ib-windows and num-ob-windows property.
>   - Replace additionalProperties with unevaluatedProperties, for RC
>     only.
>   - Add dbi2 and atu property.
>   - Remove description for regs and interrupts property.
>   - Change enum value for num-lanes to 1 and 2 only.
> - In driver patch.
>   - In Kconfig file, remove dependency on ARM64.
>   - Add new define, PCIE_REGS_PCIE_SII_LINK_UP.
>   - Remove PCIE_DBI2_MASK.
>   - In struct keembay_pcie, declare pci member as struct, not pointer.
>     And remove irq number members.
>   - Rename and rework keembay_pcie_establish_link(), to
>     keembay_pcie_start_link().
>   - Remove unneeded BAR disable steps.
>   - Remove unused interrupt handlers; keembay_pcie_ev_irq_handler(),
>     keembay_pcie_err_irq_handler().
>   - Remove keembay_pcie_enable_interrupts().
>   - Rework keembay_pcie_setup_irq() and call it from
>     keembay_pcie_probe().
>   - Remove keembay_pcie_host_init() and make keembay_pcie_host_ops
>     empty.
>   - Keep and rework keembay_pcie_add_pcie_port() a little.
>   - Remove keembay_pcie_add_pcie_ep() and call dw_pcie_ep_init() from
>     keembay_pcie_probe().
>   - In keembay_pcie_probe(), remove dbi setup as it will be handled in
>     dwc common code.
>   - In keembay_pcie_link_up(), use return (val &
>     PCIE_REGS_PCIE_SII_LINK_UP) == PCIE_REGS_PCIE_SII_LINK_UP.
>   - In keembay_pcie_ep_raise_irq(), rework error message for
>     PCI_EPC_IRQ_LEGACY and default cases.
> - Rebased to next-20201124, that has dwc pci refactoring,
>   https://lore.kernel.org/linux-pci/20201105211159.1814485-1-
> robh@kernel.org/.
> 
> Srikanth Thokala (2):
>   dt-bindings: PCI: Add Intel Keem Bay PCIe controller
>   PCI: keembay: Add support for Intel Keem Bay
> 
>  .../bindings/pci/intel,keembay-pcie-ep.yaml   |  69 +++
>  .../bindings/pci/intel,keembay-pcie.yaml      |  97 ++++
>  MAINTAINERS                                   |   7 +
>  drivers/pci/controller/dwc/Kconfig            |  28 ++
>  drivers/pci/controller/dwc/Makefile           |   1 +
>  drivers/pci/controller/dwc/pcie-keembay.c     | 451 ++++++++++++++++++
>  6 files changed, 653 insertions(+)
>  create mode 100644 Documentation/devicetree/bindings/pci/intel,keembay-
> pcie-ep.yaml
>  create mode 100644 Documentation/devicetree/bindings/pci/intel,keembay-
> pcie.yaml
>  create mode 100644 drivers/pci/controller/dwc/pcie-keembay.c
> 
> 
> base-commit: 614124bea77e452aa6df7a8714e8bc820b489922
> --
> 2.17.1

Re: [PATCH v10 2/2] PCI: keembay: Add support for Intel Keem Bay

[Rob Herring]

On Mon, Jun 7, 2021 at 1:47 AM <srikanth.thokala@intel.com> wrote:
>
> From: Srikanth Thokala <srikanth.thokala@intel.com>
>
> Add driver for Intel Keem Bay SoC PCIe controller. This controller
> is based on DesignWare PCIe core.
>
> In Root Complex mode, only internal reference clock is possible for
> Keem Bay A0. For Keem Bay B0, external reference clock can be used
> and will be the default configuration. Currently, keembay_pcie_of_data
> structure has one member. It will be expanded later to handle this
> difference.
>
> Endpoint mode link initialization is handled by the boot firmware.
>
> Signed-off-by: Wan Ahmad Zainie <wan.ahmad.zainie.wan.mohamad@intel.com>
> Acked-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
> Signed-off-by: Srikanth Thokala <srikanth.thokala@intel.com>
> Reviewed-by: Krzysztof Wilczyński <kw@linux.com>
> ---
>  MAINTAINERS                               |   7 +
>  drivers/pci/controller/dwc/Kconfig        |  28 ++
>  drivers/pci/controller/dwc/Makefile       |   1 +
>  drivers/pci/controller/dwc/pcie-keembay.c | 451 ++++++++++++++++++++++
>  4 files changed, 487 insertions(+)
>  create mode 100644 drivers/pci/controller/dwc/pcie-keembay.c

Reviewed-by: Rob Herring <robh@kernel.org>

RE: [PATCH v10 2/2] PCI: keembay: Add support for Intel Keem Bay

[Thokala, Srikanth]

Hi Rob,

> -----Original Message-----
> From: Rob Herring <robh+dt@kernel.org>
> Sent: Wednesday, June 16, 2021 2:39 AM
> To: Thokala, Srikanth <srikanth.thokala@intel.com>
> Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>; PCI <linux-
> pci@vger.kernel.org>; devicetree@vger.kernel.org; Andy Shevchenko
> <andriy.shevchenko@linux.intel.com>; Mark Gross <mgross@linux.intel.com>;
> Raja Subramanian, Lakshmi Bai <lakshmi.bai.raja.subramanian@intel.com>;
> Sangannavar, Mallikarjunappa <mallikarjunappa.sangannavar@intel.com>;
> Krzysztof Wilczynski <kw@linux.com>
> Subject: Re: [PATCH v10 2/2] PCI: keembay: Add support for Intel Keem Bay
> 
> On Mon, Jun 7, 2021 at 1:47 AM <srikanth.thokala@intel.com> wrote:
> >
> > From: Srikanth Thokala <srikanth.thokala@intel.com>
> >
> > Add driver for Intel Keem Bay SoC PCIe controller. This controller
> > is based on DesignWare PCIe core.
> >
> > In Root Complex mode, only internal reference clock is possible for
> > Keem Bay A0. For Keem Bay B0, external reference clock can be used
> > and will be the default configuration. Currently, keembay_pcie_of_data
> > structure has one member. It will be expanded later to handle this
> > difference.
> >
> > Endpoint mode link initialization is handled by the boot firmware.
> >
> > Signed-off-by: Wan Ahmad Zainie <wan.ahmad.zainie.wan.mohamad@intel.com>
> > Acked-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
> > Signed-off-by: Srikanth Thokala <srikanth.thokala@intel.com>
> > Reviewed-by: Krzysztof Wilczyński <kw@linux.com>
> > ---
> >  MAINTAINERS                               |   7 +
> >  drivers/pci/controller/dwc/Kconfig        |  28 ++
> >  drivers/pci/controller/dwc/Makefile       |   1 +
> >  drivers/pci/controller/dwc/pcie-keembay.c | 451 ++++++++++++++++++++++
> >  4 files changed, 487 insertions(+)
> >  create mode 100644 drivers/pci/controller/dwc/pcie-keembay.c
> 
> Reviewed-by: Rob Herring <robh@kernel.org>

Thank you, Rob, for the "Reviewed-by".

Srikanth

Re: [PATCH v10 2/2] PCI: keembay: Add support for Intel Keem Bay

[Lorenzo Pieralisi]

[+Marc]

On Mon, Jun 07, 2021 at 09:10:44PM +0530, srikanth.thokala@intel.com wrote:
[...]

> +static void keembay_pcie_msi_irq_handler(struct irq_desc *desc)
> +{
> +	struct keembay_pcie *pcie = irq_desc_get_handler_data(desc);
> +	struct irq_chip *chip = irq_desc_get_chip(desc);
> +	u32 val, mask, status;
> +	struct pcie_port *pp;
> +
> +	chained_irq_enter(chip, desc);
> +
> +	pp = &pcie->pci.pp;
> +	val = readl(pcie->apb_base + PCIE_REGS_INTERRUPT_STATUS);
> +	mask = readl(pcie->apb_base + PCIE_REGS_INTERRUPT_ENABLE);
> +
> +	status = val & mask;
> +
> +	if (status & MSI_CTRL_INT) {
> +		dw_handle_msi_irq(pp);
> +		writel(status, pcie->apb_base + PCIE_REGS_INTERRUPT_STATUS);
> +	}
> +
> +	chained_irq_exit(chip, desc);
> +}
> +
> +static int keembay_pcie_setup_msi_irq(struct keembay_pcie *pcie)
> +{
> +	struct dw_pcie *pci = &pcie->pci;
> +	struct device *dev = pci->dev;
> +	struct platform_device *pdev = to_platform_device(dev);
> +	int irq;
> +
> +	irq = platform_get_irq_byname(pdev, "pcie");
> +	if (irq < 0)
> +		return irq;
> +
> +	irq_set_chained_handler_and_data(irq, keembay_pcie_msi_irq_handler,
> +					 pcie);
> +

Ok this is yet another DWC MSI incantation and given that Marc worked
hard consolidating them let's have a look before we merge it.

IIUC - this IP relies on the DWC logic to handle MSIs + custom
registers to detect a pending MSI IRQ because the logic in
dw_chained_msi_irq() is *not* enough so you have to register
a driver specific chained handler. This looks similar to the dra7xx
driver MSI handling but I am not entirely convinced it is needed.

I assume this code in keembay_pcie_msi_irq_handler() is required
owing to additional IP logic on top of the standard DWC IP, in
particular the PCIE_REGS_INTERRUPT_STATUS write to "clear" the
IRQ.

We need more insights before merging it so please provide them.

pp = &pcie->pci.pp;
val = readl(pcie->apb_base + PCIE_REGS_INTERRUPT_STATUS);
mask = readl(pcie->apb_base + PCIE_REGS_INTERRUPT_ENABLE);

status = val & mask;

if (status & MSI_CTRL_INT) {
	dw_handle_msi_irq(pp);
	writel(status, pcie->apb_base + PCIE_REGS_INTERRUPT_STATUS);
}

Thanks,
Lorenzo

> +static void keembay_pcie_ep_init(struct dw_pcie_ep *ep)
> +{
> +	struct dw_pcie *pci = to_dw_pcie_from_ep(ep);
> +	struct keembay_pcie *pcie = dev_get_drvdata(pci->dev);
> +
> +	writel(EDMA_INT_EN, pcie->apb_base + PCIE_REGS_INTERRUPT_ENABLE);
> +}
> +
> +static int keembay_pcie_ep_raise_irq(struct dw_pcie_ep *ep, u8 func_no,
> +				     enum pci_epc_irq_type type,
> +				     u16 interrupt_num)
> +{
> +	struct dw_pcie *pci = to_dw_pcie_from_ep(ep);
> +
> +	switch (type) {
> +	case PCI_EPC_IRQ_LEGACY:
> +		/* Legacy interrupts are not supported in Keem Bay */
> +		dev_err(pci->dev, "Legacy IRQ is not supported\n");
> +		return -EINVAL;
> +	case PCI_EPC_IRQ_MSI:
> +		return dw_pcie_ep_raise_msi_irq(ep, func_no, interrupt_num);
> +	case PCI_EPC_IRQ_MSIX:
> +		return dw_pcie_ep_raise_msix_irq(ep, func_no, interrupt_num);
> +	default:
> +		dev_err(pci->dev, "Unknown IRQ type %d\n", type);
> +		return -EINVAL;
> +	}
> +}
> +
> +static const struct pci_epc_features keembay_pcie_epc_features = {
> +	.linkup_notifier	= false,
> +	.msi_capable		= true,
> +	.msix_capable		= true,
> +	.reserved_bar		= BIT(BAR_1) | BIT(BAR_3) | BIT(BAR_5),
> +	.bar_fixed_64bit	= BIT(BAR_0) | BIT(BAR_2) | BIT(BAR_4),
> +	.align			= SZ_16K,
> +};
> +
> +static const struct pci_epc_features *
> +keembay_pcie_get_features(struct dw_pcie_ep *ep)
> +{
> +	return &keembay_pcie_epc_features;
> +}
> +
> +static const struct dw_pcie_ep_ops keembay_pcie_ep_ops = {
> +	.ep_init	= keembay_pcie_ep_init,
> +	.raise_irq	= keembay_pcie_ep_raise_irq,
> +	.get_features	= keembay_pcie_get_features,
> +};
> +
> +static const struct dw_pcie_host_ops keembay_pcie_host_ops = {
> +};
> +
> +static int keembay_pcie_add_pcie_port(struct keembay_pcie *pcie,
> +				      struct platform_device *pdev)
> +{
> +	struct dw_pcie *pci = &pcie->pci;
> +	struct pcie_port *pp = &pci->pp;
> +	struct device *dev = &pdev->dev;
> +	u32 val;
> +	int ret;
> +
> +	pp->ops = &keembay_pcie_host_ops;
> +	pp->msi_irq = -ENODEV;
> +
> +	ret = keembay_pcie_setup_msi_irq(pcie);
> +	if (ret)
> +		return ret;
> +
> +	pcie->reset = devm_gpiod_get(dev, "reset", GPIOD_OUT_HIGH);
> +	if (IS_ERR(pcie->reset))
> +		return PTR_ERR(pcie->reset);
> +
> +	ret = keembay_pcie_probe_clocks(pcie);
> +	if (ret)
> +		return ret;
> +
> +	val = readl(pcie->apb_base + PCIE_REGS_PCIE_PHY_CNTL);
> +	val |= PHY0_SRAM_BYPASS;
> +	writel(val, pcie->apb_base + PCIE_REGS_PCIE_PHY_CNTL);
> +
> +	writel(PCIE_DEVICE_TYPE, pcie->apb_base + PCIE_REGS_PCIE_CFG);
> +
> +	ret = keembay_pcie_pll_init(pcie);
> +	if (ret)
> +		return ret;
> +
> +	val = readl(pcie->apb_base + PCIE_REGS_PCIE_CFG);
> +	writel(val | PCIE_RSTN, pcie->apb_base + PCIE_REGS_PCIE_CFG);
> +	keembay_ep_reset_deassert(pcie);
> +
> +	ret = dw_pcie_host_init(pp);
> +	if (ret) {
> +		keembay_ep_reset_assert(pcie);
> +		dev_err(dev, "Failed to initialize host: %d\n", ret);
> +		return ret;
> +	}
> +
> +	val = readl(pcie->apb_base + PCIE_REGS_INTERRUPT_ENABLE);
> +	if (IS_ENABLED(CONFIG_PCI_MSI))
> +		val |= MSI_CTRL_INT_EN;
> +	writel(val, pcie->apb_base + PCIE_REGS_INTERRUPT_ENABLE);
> +
> +	return 0;
> +}
> +
> +static int keembay_pcie_probe(struct platform_device *pdev)
> +{
> +	const struct keembay_pcie_of_data *data;
> +	struct device *dev = &pdev->dev;
> +	struct keembay_pcie *pcie;
> +	struct dw_pcie *pci;
> +	enum dw_pcie_device_mode mode;
> +
> +	data = device_get_match_data(dev);
> +	if (!data)
> +		return -ENODEV;
> +
> +	mode = (enum dw_pcie_device_mode)data->mode;
> +
> +	pcie = devm_kzalloc(dev, sizeof(*pcie), GFP_KERNEL);
> +	if (!pcie)
> +		return -ENOMEM;
> +
> +	pci = &pcie->pci;
> +	pci->dev = dev;
> +	pci->ops = &keembay_pcie_ops;
> +
> +	pcie->mode = mode;
> +
> +	pcie->apb_base = devm_platform_ioremap_resource_byname(pdev, "apb");
> +	if (IS_ERR(pcie->apb_base))
> +		return PTR_ERR(pcie->apb_base);
> +
> +	platform_set_drvdata(pdev, pcie);
> +
> +	switch (pcie->mode) {
> +	case DW_PCIE_RC_TYPE:
> +		if (!IS_ENABLED(CONFIG_PCIE_KEEMBAY_HOST))
> +			return -ENODEV;
> +
> +		return keembay_pcie_add_pcie_port(pcie, pdev);
> +	case DW_PCIE_EP_TYPE:
> +		if (!IS_ENABLED(CONFIG_PCIE_KEEMBAY_EP))
> +			return -ENODEV;
> +
> +		pci->ep.ops = &keembay_pcie_ep_ops;
> +		return dw_pcie_ep_init(&pci->ep);
> +	default:
> +		dev_err(dev, "Invalid device type %d\n", pcie->mode);
> +		return -ENODEV;
> +	}
> +}
> +
> +static const struct keembay_pcie_of_data keembay_pcie_rc_of_data = {
> +	.mode = DW_PCIE_RC_TYPE,
> +};
> +
> +static const struct keembay_pcie_of_data keembay_pcie_ep_of_data = {
> +	.mode = DW_PCIE_EP_TYPE,
> +};
> +
> +static const struct of_device_id keembay_pcie_of_match[] = {
> +	{
> +		.compatible = "intel,keembay-pcie",
> +		.data = &keembay_pcie_rc_of_data,
> +	},
> +	{
> +		.compatible = "intel,keembay-pcie-ep",
> +		.data = &keembay_pcie_ep_of_data,
> +	},
> +	{}
> +};
> +
> +static struct platform_driver keembay_pcie_driver = {
> +	.driver = {
> +		.name = "keembay-pcie",
> +		.of_match_table = keembay_pcie_of_match,
> +		.suppress_bind_attrs = true,
> +	},
> +	.probe  = keembay_pcie_probe,
> +};
> +builtin_platform_driver(keembay_pcie_driver);
> -- 
> 2.17.1
> 

RE: [PATCH v10 2/2] PCI: keembay: Add support for Intel Keem Bay

[Thokala, Srikanth]

Hi Lorenzo,

> -----Original Message-----
> From: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
> Sent: Monday, June 21, 2021 10:23 PM
> To: Thokala, Srikanth <srikanth.thokala@intel.com>; maz@kernel.org
> Cc: robh+dt@kernel.org; linux-pci@vger.kernel.org;
> devicetree@vger.kernel.org; andriy.shevchenko@linux.intel.com;
> mgross@linux.intel.com; Raja Subramanian, Lakshmi Bai
> <lakshmi.bai.raja.subramanian@intel.com>; Sangannavar, Mallikarjunappa
> <mallikarjunappa.sangannavar@intel.com>; kw@linux.com
> Subject: Re: [PATCH v10 2/2] PCI: keembay: Add support for Intel Keem Bay
> 
> [+Marc]
> 
> On Mon, Jun 07, 2021 at 09:10:44PM +0530, srikanth.thokala@intel.com
> wrote:
> [...]
> 
> > +static void keembay_pcie_msi_irq_handler(struct irq_desc *desc)
> > +{
> > +	struct keembay_pcie *pcie = irq_desc_get_handler_data(desc);
> > +	struct irq_chip *chip = irq_desc_get_chip(desc);
> > +	u32 val, mask, status;
> > +	struct pcie_port *pp;
> > +
> > +	chained_irq_enter(chip, desc);
> > +
> > +	pp = &pcie->pci.pp;
> > +	val = readl(pcie->apb_base + PCIE_REGS_INTERRUPT_STATUS);
> > +	mask = readl(pcie->apb_base + PCIE_REGS_INTERRUPT_ENABLE);
> > +
> > +	status = val & mask;
> > +
> > +	if (status & MSI_CTRL_INT) {
> > +		dw_handle_msi_irq(pp);
> > +		writel(status, pcie->apb_base + PCIE_REGS_INTERRUPT_STATUS);
> > +	}
> > +
> > +	chained_irq_exit(chip, desc);
> > +}
> > +
> > +static int keembay_pcie_setup_msi_irq(struct keembay_pcie *pcie)
> > +{
> > +	struct dw_pcie *pci = &pcie->pci;
> > +	struct device *dev = pci->dev;
> > +	struct platform_device *pdev = to_platform_device(dev);
> > +	int irq;
> > +
> > +	irq = platform_get_irq_byname(pdev, "pcie");
> > +	if (irq < 0)
> > +		return irq;
> > +
> > +	irq_set_chained_handler_and_data(irq, keembay_pcie_msi_irq_handler,
> > +					 pcie);
> > +
> 
> Ok this is yet another DWC MSI incantation and given that Marc worked
> hard consolidating them let's have a look before we merge it.
> 
> IIUC - this IP relies on the DWC logic to handle MSIs + custom
> registers to detect a pending MSI IRQ because the logic in
> dw_chained_msi_irq() is *not* enough so you have to register
> a driver specific chained handler. This looks similar to the dra7xx
> driver MSI handling but I am not entirely convinced it is needed.
> 
> I assume this code in keembay_pcie_msi_irq_handler() is required
> owing to additional IP logic on top of the standard DWC IP, in
> particular the PCIE_REGS_INTERRUPT_STATUS write to "clear" the
> IRQ.
> 
> We need more insights before merging it so please provide them.
> 
> pp = &pcie->pci.pp;
> val = readl(pcie->apb_base + PCIE_REGS_INTERRUPT_STATUS);
> mask = readl(pcie->apb_base + PCIE_REGS_INTERRUPT_ENABLE);
> 
> status = val & mask;
> 
> if (status & MSI_CTRL_INT) {
> 	dw_handle_msi_irq(pp);
> 	writel(status, pcie->apb_base + PCIE_REGS_INTERRUPT_STATUS);
> }

Yes, your understanding is correct.

Additional registers PCIE_REGS_INTERRUPT_ENABLE/_STATUS are provided
by IP to control the interrupts.

To receive MSI interrupts, SW must enable bit '8' of _ENABLE register.
And once a MSI is raised by the End point, bit '8' of _STATUS register
will be set and it needs to be cleared after servicing the interrupt.

Thanks!
Srikanth

> 
> Thanks,
> Lorenzo
> 
> > +static void keembay_pcie_ep_init(struct dw_pcie_ep *ep)
> > +{
> > +	struct dw_pcie *pci = to_dw_pcie_from_ep(ep);
> > +	struct keembay_pcie *pcie = dev_get_drvdata(pci->dev);
> > +
> > +	writel(EDMA_INT_EN, pcie->apb_base + PCIE_REGS_INTERRUPT_ENABLE);
> > +}
> > +
> > +static int keembay_pcie_ep_raise_irq(struct dw_pcie_ep *ep, u8 func_no,
> > +				     enum pci_epc_irq_type type,
> > +				     u16 interrupt_num)
> > +{
> > +	struct dw_pcie *pci = to_dw_pcie_from_ep(ep);
> > +
> > +	switch (type) {
> > +	case PCI_EPC_IRQ_LEGACY:
> > +		/* Legacy interrupts are not supported in Keem Bay */
> > +		dev_err(pci->dev, "Legacy IRQ is not supported\n");
> > +		return -EINVAL;
> > +	case PCI_EPC_IRQ_MSI:
> > +		return dw_pcie_ep_raise_msi_irq(ep, func_no, interrupt_num);
> > +	case PCI_EPC_IRQ_MSIX:
> > +		return dw_pcie_ep_raise_msix_irq(ep, func_no, interrupt_num);
> > +	default:
> > +		dev_err(pci->dev, "Unknown IRQ type %d\n", type);
> > +		return -EINVAL;
> > +	}
> > +}
> > +
> > +static const struct pci_epc_features keembay_pcie_epc_features = {
> > +	.linkup_notifier	= false,
> > +	.msi_capable		= true,
> > +	.msix_capable		= true,
> > +	.reserved_bar		= BIT(BAR_1) | BIT(BAR_3) | BIT(BAR_5),
> > +	.bar_fixed_64bit	= BIT(BAR_0) | BIT(BAR_2) | BIT(BAR_4),
> > +	.align			= SZ_16K,
> > +};
> > +
> > +static const struct pci_epc_features *
> > +keembay_pcie_get_features(struct dw_pcie_ep *ep)
> > +{
> > +	return &keembay_pcie_epc_features;
> > +}
> > +
> > +static const struct dw_pcie_ep_ops keembay_pcie_ep_ops = {
> > +	.ep_init	= keembay_pcie_ep_init,
> > +	.raise_irq	= keembay_pcie_ep_raise_irq,
> > +	.get_features	= keembay_pcie_get_features,
> > +};
> > +
> > +static const struct dw_pcie_host_ops keembay_pcie_host_ops = {
> > +};
> > +
> > +static int keembay_pcie_add_pcie_port(struct keembay_pcie *pcie,
> > +				      struct platform_device *pdev)
> > +{
> > +	struct dw_pcie *pci = &pcie->pci;
> > +	struct pcie_port *pp = &pci->pp;
> > +	struct device *dev = &pdev->dev;
> > +	u32 val;
> > +	int ret;
> > +
> > +	pp->ops = &keembay_pcie_host_ops;
> > +	pp->msi_irq = -ENODEV;
> > +
> > +	ret = keembay_pcie_setup_msi_irq(pcie);
> > +	if (ret)
> > +		return ret;
> > +
> > +	pcie->reset = devm_gpiod_get(dev, "reset", GPIOD_OUT_HIGH);
> > +	if (IS_ERR(pcie->reset))
> > +		return PTR_ERR(pcie->reset);
> > +
> > +	ret = keembay_pcie_probe_clocks(pcie);
> > +	if (ret)
> > +		return ret;
> > +
> > +	val = readl(pcie->apb_base + PCIE_REGS_PCIE_PHY_CNTL);
> > +	val |= PHY0_SRAM_BYPASS;
> > +	writel(val, pcie->apb_base + PCIE_REGS_PCIE_PHY_CNTL);
> > +
> > +	writel(PCIE_DEVICE_TYPE, pcie->apb_base + PCIE_REGS_PCIE_CFG);
> > +
> > +	ret = keembay_pcie_pll_init(pcie);
> > +	if (ret)
> > +		return ret;
> > +
> > +	val = readl(pcie->apb_base + PCIE_REGS_PCIE_CFG);
> > +	writel(val | PCIE_RSTN, pcie->apb_base + PCIE_REGS_PCIE_CFG);
> > +	keembay_ep_reset_deassert(pcie);
> > +
> > +	ret = dw_pcie_host_init(pp);
> > +	if (ret) {
> > +		keembay_ep_reset_assert(pcie);
> > +		dev_err(dev, "Failed to initialize host: %d\n", ret);
> > +		return ret;
> > +	}
> > +
> > +	val = readl(pcie->apb_base + PCIE_REGS_INTERRUPT_ENABLE);
> > +	if (IS_ENABLED(CONFIG_PCI_MSI))
> > +		val |= MSI_CTRL_INT_EN;
> > +	writel(val, pcie->apb_base + PCIE_REGS_INTERRUPT_ENABLE);
> > +
> > +	return 0;
> > +}
> > +
> > +static int keembay_pcie_probe(struct platform_device *pdev)
> > +{
> > +	const struct keembay_pcie_of_data *data;
> > +	struct device *dev = &pdev->dev;
> > +	struct keembay_pcie *pcie;
> > +	struct dw_pcie *pci;
> > +	enum dw_pcie_device_mode mode;
> > +
> > +	data = device_get_match_data(dev);
> > +	if (!data)
> > +		return -ENODEV;
> > +
> > +	mode = (enum dw_pcie_device_mode)data->mode;
> > +
> > +	pcie = devm_kzalloc(dev, sizeof(*pcie), GFP_KERNEL);
> > +	if (!pcie)
> > +		return -ENOMEM;
> > +
> > +	pci = &pcie->pci;
> > +	pci->dev = dev;
> > +	pci->ops = &keembay_pcie_ops;
> > +
> > +	pcie->mode = mode;
> > +
> > +	pcie->apb_base = devm_platform_ioremap_resource_byname(pdev, "apb");
> > +	if (IS_ERR(pcie->apb_base))
> > +		return PTR_ERR(pcie->apb_base);
> > +
> > +	platform_set_drvdata(pdev, pcie);
> > +
> > +	switch (pcie->mode) {
> > +	case DW_PCIE_RC_TYPE:
> > +		if (!IS_ENABLED(CONFIG_PCIE_KEEMBAY_HOST))
> > +			return -ENODEV;
> > +
> > +		return keembay_pcie_add_pcie_port(pcie, pdev);
> > +	case DW_PCIE_EP_TYPE:
> > +		if (!IS_ENABLED(CONFIG_PCIE_KEEMBAY_EP))
> > +			return -ENODEV;
> > +
> > +		pci->ep.ops = &keembay_pcie_ep_ops;
> > +		return dw_pcie_ep_init(&pci->ep);
> > +	default:
> > +		dev_err(dev, "Invalid device type %d\n", pcie->mode);
> > +		return -ENODEV;
> > +	}
> > +}
> > +
> > +static const struct keembay_pcie_of_data keembay_pcie_rc_of_data = {
> > +	.mode = DW_PCIE_RC_TYPE,
> > +};
> > +
> > +static const struct keembay_pcie_of_data keembay_pcie_ep_of_data = {
> > +	.mode = DW_PCIE_EP_TYPE,
> > +};
> > +
> > +static const struct of_device_id keembay_pcie_of_match[] = {
> > +	{
> > +		.compatible = "intel,keembay-pcie",
> > +		.data = &keembay_pcie_rc_of_data,
> > +	},
> > +	{
> > +		.compatible = "intel,keembay-pcie-ep",
> > +		.data = &keembay_pcie_ep_of_data,
> > +	},
> > +	{}
> > +};
> > +
> > +static struct platform_driver keembay_pcie_driver = {
> > +	.driver = {
> > +		.name = "keembay-pcie",
> > +		.of_match_table = keembay_pcie_of_match,
> > +		.suppress_bind_attrs = true,
> > +	},
> > +	.probe  = keembay_pcie_probe,
> > +};
> > +builtin_platform_driver(keembay_pcie_driver);
> > --
> > 2.17.1
> >

RE: [PATCH v10 2/2] PCI: keembay: Add support for Intel Keem Bay

[Thokala, Srikanth]

Hi Lorenzo and Marc,

> -----Original Message-----
> From: Thokala, Srikanth
> Sent: Friday, June 25, 2021 8:54 AM
> To: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>; maz@kernel.org
> Cc: robh+dt@kernel.org; linux-pci@vger.kernel.org;
> devicetree@vger.kernel.org; andriy.shevchenko@linux.intel.com;
> mgross@linux.intel.com; Raja Subramanian, Lakshmi Bai
> <lakshmi.bai.raja.subramanian@intel.com>; Sangannavar, Mallikarjunappa
> <mallikarjunappa.sangannavar@intel.com>; kw@linux.com
> Subject: RE: [PATCH v10 2/2] PCI: keembay: Add support for Intel Keem
> Bay
> 
> Hi Lorenzo,
> 
> > -----Original Message-----
> > From: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
> > Sent: Monday, June 21, 2021 10:23 PM
> > To: Thokala, Srikanth <srikanth.thokala@intel.com>; maz@kernel.org
> > Cc: robh+dt@kernel.org; linux-pci@vger.kernel.org;
> > devicetree@vger.kernel.org; andriy.shevchenko@linux.intel.com;
> > mgross@linux.intel.com; Raja Subramanian, Lakshmi Bai
> > <lakshmi.bai.raja.subramanian@intel.com>; Sangannavar,
> Mallikarjunappa
> > <mallikarjunappa.sangannavar@intel.com>; kw@linux.com
> > Subject: Re: [PATCH v10 2/2] PCI: keembay: Add support for Intel Keem
> Bay
> >
> > [+Marc]
> >
> > On Mon, Jun 07, 2021 at 09:10:44PM +0530, srikanth.thokala@intel.com
> > wrote:
> > [...]
> >
> > > +static void keembay_pcie_msi_irq_handler(struct irq_desc *desc)
> > > +{
> > > +	struct keembay_pcie *pcie = irq_desc_get_handler_data(desc);
> > > +	struct irq_chip *chip = irq_desc_get_chip(desc);
> > > +	u32 val, mask, status;
> > > +	struct pcie_port *pp;
> > > +
> > > +	chained_irq_enter(chip, desc);
> > > +
> > > +	pp = &pcie->pci.pp;
> > > +	val = readl(pcie->apb_base + PCIE_REGS_INTERRUPT_STATUS);
> > > +	mask = readl(pcie->apb_base + PCIE_REGS_INTERRUPT_ENABLE);
> > > +
> > > +	status = val & mask;
> > > +
> > > +	if (status & MSI_CTRL_INT) {
> > > +		dw_handle_msi_irq(pp);
> > > +		writel(status, pcie->apb_base +
> PCIE_REGS_INTERRUPT_STATUS);
> > > +	}
> > > +
> > > +	chained_irq_exit(chip, desc);
> > > +}
> > > +
> > > +static int keembay_pcie_setup_msi_irq(struct keembay_pcie *pcie)
> > > +{
> > > +	struct dw_pcie *pci = &pcie->pci;
> > > +	struct device *dev = pci->dev;
> > > +	struct platform_device *pdev = to_platform_device(dev);
> > > +	int irq;
> > > +
> > > +	irq = platform_get_irq_byname(pdev, "pcie");
> > > +	if (irq < 0)
> > > +		return irq;
> > > +
> > > +	irq_set_chained_handler_and_data(irq,
> keembay_pcie_msi_irq_handler,
> > > +					 pcie);
> > > +
> >
> > Ok this is yet another DWC MSI incantation and given that Marc worked
> > hard consolidating them let's have a look before we merge it.
> >
> > IIUC - this IP relies on the DWC logic to handle MSIs + custom
> > registers to detect a pending MSI IRQ because the logic in
> > dw_chained_msi_irq() is *not* enough so you have to register
> > a driver specific chained handler. This looks similar to the dra7xx
> > driver MSI handling but I am not entirely convinced it is needed.
> >
> > I assume this code in keembay_pcie_msi_irq_handler() is required
> > owing to additional IP logic on top of the standard DWC IP, in
> > particular the PCIE_REGS_INTERRUPT_STATUS write to "clear" the
> > IRQ.
> >
> > We need more insights before merging it so please provide them.
> >
> > pp = &pcie->pci.pp;
> > val = readl(pcie->apb_base + PCIE_REGS_INTERRUPT_STATUS);
> > mask = readl(pcie->apb_base + PCIE_REGS_INTERRUPT_ENABLE);
> >
> > status = val & mask;
> >
> > if (status & MSI_CTRL_INT) {
> > 	dw_handle_msi_irq(pp);
> > 	writel(status, pcie->apb_base + PCIE_REGS_INTERRUPT_STATUS);
> > }
> 
> Yes, your understanding is correct.
> 
> Additional registers PCIE_REGS_INTERRUPT_ENABLE/_STATUS are provided
> by IP to control the interrupts.
> 
> To receive MSI interrupts, SW must enable bit '8' of _ENABLE register.
> And once a MSI is raised by the End point, bit '8' of _STATUS register
> will be set and it needs to be cleared after servicing the interrupt.

Hope I have provided all the necessary information.
Kindly feedback.

Thanks!
Srikanth

> 
> Thanks!
> Srikanth
> 
> >
> > Thanks,
> > Lorenzo
> >
> > > +static void keembay_pcie_ep_init(struct dw_pcie_ep *ep)
> > > +{
> > > +	struct dw_pcie *pci = to_dw_pcie_from_ep(ep);
> > > +	struct keembay_pcie *pcie = dev_get_drvdata(pci->dev);
> > > +
> > > +	writel(EDMA_INT_EN, pcie->apb_base + PCIE_REGS_INTERRUPT_ENABLE);
> > > +}
> > > +
> > > +static int keembay_pcie_ep_raise_irq(struct dw_pcie_ep *ep, u8
> func_no,
> > > +				     enum pci_epc_irq_type type,
> > > +				     u16 interrupt_num)
> > > +{
> > > +	struct dw_pcie *pci = to_dw_pcie_from_ep(ep);
> > > +
> > > +	switch (type) {
> > > +	case PCI_EPC_IRQ_LEGACY:
> > > +		/* Legacy interrupts are not supported in Keem Bay */
> > > +		dev_err(pci->dev, "Legacy IRQ is not supported\n");
> > > +		return -EINVAL;
> > > +	case PCI_EPC_IRQ_MSI:
> > > +		return dw_pcie_ep_raise_msi_irq(ep, func_no,
> interrupt_num);
> > > +	case PCI_EPC_IRQ_MSIX:
> > > +		return dw_pcie_ep_raise_msix_irq(ep, func_no,
> interrupt_num);
> > > +	default:
> > > +		dev_err(pci->dev, "Unknown IRQ type %d\n", type);
> > > +		return -EINVAL;
> > > +	}
> > > +}
> > > +
> > > +static const struct pci_epc_features keembay_pcie_epc_features = {
> > > +	.linkup_notifier	= false,
> > > +	.msi_capable		= true,
> > > +	.msix_capable		= true,
> > > +	.reserved_bar		= BIT(BAR_1) | BIT(BAR_3) | BIT(BAR_5),
> > > +	.bar_fixed_64bit	= BIT(BAR_0) | BIT(BAR_2) | BIT(BAR_4),
> > > +	.align			= SZ_16K,
> > > +};
> > > +
> > > +static const struct pci_epc_features *
> > > +keembay_pcie_get_features(struct dw_pcie_ep *ep)
> > > +{
> > > +	return &keembay_pcie_epc_features;
> > > +}
> > > +
> > > +static const struct dw_pcie_ep_ops keembay_pcie_ep_ops = {
> > > +	.ep_init	= keembay_pcie_ep_init,
> > > +	.raise_irq	= keembay_pcie_ep_raise_irq,
> > > +	.get_features	= keembay_pcie_get_features,
> > > +};
> > > +
> > > +static const struct dw_pcie_host_ops keembay_pcie_host_ops = {
> > > +};
> > > +
> > > +static int keembay_pcie_add_pcie_port(struct keembay_pcie *pcie,
> > > +				      struct platform_device *pdev)
> > > +{
> > > +	struct dw_pcie *pci = &pcie->pci;
> > > +	struct pcie_port *pp = &pci->pp;
> > > +	struct device *dev = &pdev->dev;
> > > +	u32 val;
> > > +	int ret;
> > > +
> > > +	pp->ops = &keembay_pcie_host_ops;
> > > +	pp->msi_irq = -ENODEV;
> > > +
> > > +	ret = keembay_pcie_setup_msi_irq(pcie);
> > > +	if (ret)
> > > +		return ret;
> > > +
> > > +	pcie->reset = devm_gpiod_get(dev, "reset", GPIOD_OUT_HIGH);
> > > +	if (IS_ERR(pcie->reset))
> > > +		return PTR_ERR(pcie->reset);
> > > +
> > > +	ret = keembay_pcie_probe_clocks(pcie);
> > > +	if (ret)
> > > +		return ret;
> > > +
> > > +	val = readl(pcie->apb_base + PCIE_REGS_PCIE_PHY_CNTL);
> > > +	val |= PHY0_SRAM_BYPASS;
> > > +	writel(val, pcie->apb_base + PCIE_REGS_PCIE_PHY_CNTL);
> > > +
> > > +	writel(PCIE_DEVICE_TYPE, pcie->apb_base + PCIE_REGS_PCIE_CFG);
> > > +
> > > +	ret = keembay_pcie_pll_init(pcie);
> > > +	if (ret)
> > > +		return ret;
> > > +
> > > +	val = readl(pcie->apb_base + PCIE_REGS_PCIE_CFG);
> > > +	writel(val | PCIE_RSTN, pcie->apb_base + PCIE_REGS_PCIE_CFG);
> > > +	keembay_ep_reset_deassert(pcie);
> > > +
> > > +	ret = dw_pcie_host_init(pp);
> > > +	if (ret) {
> > > +		keembay_ep_reset_assert(pcie);
> > > +		dev_err(dev, "Failed to initialize host: %d\n", ret);
> > > +		return ret;
> > > +	}
> > > +
> > > +	val = readl(pcie->apb_base + PCIE_REGS_INTERRUPT_ENABLE);
> > > +	if (IS_ENABLED(CONFIG_PCI_MSI))
> > > +		val |= MSI_CTRL_INT_EN;
> > > +	writel(val, pcie->apb_base + PCIE_REGS_INTERRUPT_ENABLE);
> > > +
> > > +	return 0;
> > > +}
> > > +
> > > +static int keembay_pcie_probe(struct platform_device *pdev)
> > > +{
> > > +	const struct keembay_pcie_of_data *data;
> > > +	struct device *dev = &pdev->dev;
> > > +	struct keembay_pcie *pcie;
> > > +	struct dw_pcie *pci;
> > > +	enum dw_pcie_device_mode mode;
> > > +
> > > +	data = device_get_match_data(dev);
> > > +	if (!data)
> > > +		return -ENODEV;
> > > +
> > > +	mode = (enum dw_pcie_device_mode)data->mode;
> > > +
> > > +	pcie = devm_kzalloc(dev, sizeof(*pcie), GFP_KERNEL);
> > > +	if (!pcie)
> > > +		return -ENOMEM;
> > > +
> > > +	pci = &pcie->pci;
> > > +	pci->dev = dev;
> > > +	pci->ops = &keembay_pcie_ops;
> > > +
> > > +	pcie->mode = mode;
> > > +
> > > +	pcie->apb_base = devm_platform_ioremap_resource_byname(pdev,
> "apb");
> > > +	if (IS_ERR(pcie->apb_base))
> > > +		return PTR_ERR(pcie->apb_base);
> > > +
> > > +	platform_set_drvdata(pdev, pcie);
> > > +
> > > +	switch (pcie->mode) {
> > > +	case DW_PCIE_RC_TYPE:
> > > +		if (!IS_ENABLED(CONFIG_PCIE_KEEMBAY_HOST))
> > > +			return -ENODEV;
> > > +
> > > +		return keembay_pcie_add_pcie_port(pcie, pdev);
> > > +	case DW_PCIE_EP_TYPE:
> > > +		if (!IS_ENABLED(CONFIG_PCIE_KEEMBAY_EP))
> > > +			return -ENODEV;
> > > +
> > > +		pci->ep.ops = &keembay_pcie_ep_ops;
> > > +		return dw_pcie_ep_init(&pci->ep);
> > > +	default:
> > > +		dev_err(dev, "Invalid device type %d\n", pcie->mode);
> > > +		return -ENODEV;
> > > +	}
> > > +}
> > > +
> > > +static const struct keembay_pcie_of_data keembay_pcie_rc_of_data =
> {
> > > +	.mode = DW_PCIE_RC_TYPE,
> > > +};
> > > +
> > > +static const struct keembay_pcie_of_data keembay_pcie_ep_of_data =
> {
> > > +	.mode = DW_PCIE_EP_TYPE,
> > > +};
> > > +
> > > +static const struct of_device_id keembay_pcie_of_match[] = {
> > > +	{
> > > +		.compatible = "intel,keembay-pcie",
> > > +		.data = &keembay_pcie_rc_of_data,
> > > +	},
> > > +	{
> > > +		.compatible = "intel,keembay-pcie-ep",
> > > +		.data = &keembay_pcie_ep_of_data,
> > > +	},
> > > +	{}
> > > +};
> > > +
> > > +static struct platform_driver keembay_pcie_driver = {
> > > +	.driver = {
> > > +		.name = "keembay-pcie",
> > > +		.of_match_table = keembay_pcie_of_match,
> > > +		.suppress_bind_attrs = true,
> > > +	},
> > > +	.probe  = keembay_pcie_probe,
> > > +};
> > > +builtin_platform_driver(keembay_pcie_driver);
> > > --
> > > 2.17.1
> > >

RE: [PATCH v10 2/2] PCI: keembay: Add support for Intel Keem Bay

[Thokala, Srikanth]

Hi Lorenzo and Marc,

> -----Original Message-----
> From: Thokala, Srikanth
> Sent: Wednesday, July 7, 2021 5:25 PM
> To: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>; maz@kernel.org
> Cc: robh+dt@kernel.org; linux-pci@vger.kernel.org;
> devicetree@vger.kernel.org; andriy.shevchenko@linux.intel.com;
> mgross@linux.intel.com; Raja Subramanian, Lakshmi Bai
> <lakshmi.bai.raja.subramanian@intel.com>; Sangannavar, Mallikarjunappa
> <mallikarjunappa.sangannavar@intel.com>; kw@linux.com
> Subject: RE: [PATCH v10 2/2] PCI: keembay: Add support for Intel Keem
> Bay
> 
> Hi Lorenzo and Marc,
> 
> > -----Original Message-----
> > From: Thokala, Srikanth
> > Sent: Friday, June 25, 2021 8:54 AM
> > To: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>; maz@kernel.org
> > Cc: robh+dt@kernel.org; linux-pci@vger.kernel.org;
> > devicetree@vger.kernel.org; andriy.shevchenko@linux.intel.com;
> > mgross@linux.intel.com; Raja Subramanian, Lakshmi Bai
> > <lakshmi.bai.raja.subramanian@intel.com>; Sangannavar,
> Mallikarjunappa
> > <mallikarjunappa.sangannavar@intel.com>; kw@linux.com
> > Subject: RE: [PATCH v10 2/2] PCI: keembay: Add support for Intel Keem
> > Bay
> >
> > Hi Lorenzo,
> >
> > > -----Original Message-----
> > > From: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
> > > Sent: Monday, June 21, 2021 10:23 PM
> > > To: Thokala, Srikanth <srikanth.thokala@intel.com>; maz@kernel.org
> > > Cc: robh+dt@kernel.org; linux-pci@vger.kernel.org;
> > > devicetree@vger.kernel.org; andriy.shevchenko@linux.intel.com;
> > > mgross@linux.intel.com; Raja Subramanian, Lakshmi Bai
> > > <lakshmi.bai.raja.subramanian@intel.com>; Sangannavar,
> > Mallikarjunappa
> > > <mallikarjunappa.sangannavar@intel.com>; kw@linux.com
> > > Subject: Re: [PATCH v10 2/2] PCI: keembay: Add support for Intel
> Keem
> > Bay
> > >
> > > [+Marc]
> > >
> > > On Mon, Jun 07, 2021 at 09:10:44PM +0530,
> srikanth.thokala@intel.com
> > > wrote:
> > > [...]
> > >
> > > > +static void keembay_pcie_msi_irq_handler(struct irq_desc *desc)
> > > > +{
> > > > +	struct keembay_pcie *pcie =
> irq_desc_get_handler_data(desc);
> > > > +	struct irq_chip *chip = irq_desc_get_chip(desc);
> > > > +	u32 val, mask, status;
> > > > +	struct pcie_port *pp;
> > > > +
> > > > +	chained_irq_enter(chip, desc);
> > > > +
> > > > +	pp = &pcie->pci.pp;
> > > > +	val = readl(pcie->apb_base + PCIE_REGS_INTERRUPT_STATUS);
> > > > +	mask = readl(pcie->apb_base + PCIE_REGS_INTERRUPT_ENABLE);
> > > > +
> > > > +	status = val & mask;
> > > > +
> > > > +	if (status & MSI_CTRL_INT) {
> > > > +		dw_handle_msi_irq(pp);
> > > > +		writel(status, pcie->apb_base +
> > PCIE_REGS_INTERRUPT_STATUS);
> > > > +	}
> > > > +
> > > > +	chained_irq_exit(chip, desc);
> > > > +}
> > > > +
> > > > +static int keembay_pcie_setup_msi_irq(struct keembay_pcie *pcie)
> > > > +{
> > > > +	struct dw_pcie *pci = &pcie->pci;
> > > > +	struct device *dev = pci->dev;
> > > > +	struct platform_device *pdev = to_platform_device(dev);
> > > > +	int irq;
> > > > +
> > > > +	irq = platform_get_irq_byname(pdev, "pcie");
> > > > +	if (irq < 0)
> > > > +		return irq;
> > > > +
> > > > +	irq_set_chained_handler_and_data(irq,
> > keembay_pcie_msi_irq_handler,
> > > > +					 pcie);
> > > > +
> > >
> > > Ok this is yet another DWC MSI incantation and given that Marc
> worked
> > > hard consolidating them let's have a look before we merge it.
> > >
> > > IIUC - this IP relies on the DWC logic to handle MSIs + custom
> > > registers to detect a pending MSI IRQ because the logic in
> > > dw_chained_msi_irq() is *not* enough so you have to register
> > > a driver specific chained handler. This looks similar to the dra7xx
> > > driver MSI handling but I am not entirely convinced it is needed.
> > >
> > > I assume this code in keembay_pcie_msi_irq_handler() is required
> > > owing to additional IP logic on top of the standard DWC IP, in
> > > particular the PCIE_REGS_INTERRUPT_STATUS write to "clear" the
> > > IRQ.
> > >
> > > We need more insights before merging it so please provide them.
> > >
> > > pp = &pcie->pci.pp;
> > > val = readl(pcie->apb_base + PCIE_REGS_INTERRUPT_STATUS);
> > > mask = readl(pcie->apb_base + PCIE_REGS_INTERRUPT_ENABLE);
> > >
> > > status = val & mask;
> > >
> > > if (status & MSI_CTRL_INT) {
> > > 	dw_handle_msi_irq(pp);
> > > 	writel(status, pcie->apb_base + PCIE_REGS_INTERRUPT_STATUS);
> > > }
> >
> > Yes, your understanding is correct.
> >
> > Additional registers PCIE_REGS_INTERRUPT_ENABLE/_STATUS are provided
> > by IP to control the interrupts.
> >
> > To receive MSI interrupts, SW must enable bit '8' of _ENABLE
> register.
> > And once a MSI is raised by the End point, bit '8' of _STATUS
> register
> > will be set and it needs to be cleared after servicing the interrupt.
> 
> Hope I have provided all the necessary information.
> Kindly feedback.

Kindly feedback.

Thanks!
Srikanth

> 
> Thanks!
> Srikanth
> 
> >
> > Thanks!
> > Srikanth
> >
> > >
> > > Thanks,
> > > Lorenzo
> > >
> > > > +static void keembay_pcie_ep_init(struct dw_pcie_ep *ep)
> > > > +{
> > > > +	struct dw_pcie *pci = to_dw_pcie_from_ep(ep);
> > > > +	struct keembay_pcie *pcie = dev_get_drvdata(pci->dev);
> > > > +
> > > > +	writel(EDMA_INT_EN, pcie->apb_base +
> PCIE_REGS_INTERRUPT_ENABLE);
> > > > +}
> > > > +
> > > > +static int keembay_pcie_ep_raise_irq(struct dw_pcie_ep *ep, u8
> > func_no,
> > > > +				     enum pci_epc_irq_type type,
> > > > +				     u16 interrupt_num)
> > > > +{
> > > > +	struct dw_pcie *pci = to_dw_pcie_from_ep(ep);
> > > > +
> > > > +	switch (type) {
> > > > +	case PCI_EPC_IRQ_LEGACY:
> > > > +		/* Legacy interrupts are not supported in Keem Bay */
> > > > +		dev_err(pci->dev, "Legacy IRQ is not supported\n");
> > > > +		return -EINVAL;
> > > > +	case PCI_EPC_IRQ_MSI:
> > > > +		return dw_pcie_ep_raise_msi_irq(ep, func_no,
> > interrupt_num);
> > > > +	case PCI_EPC_IRQ_MSIX:
> > > > +		return dw_pcie_ep_raise_msix_irq(ep, func_no,
> > interrupt_num);
> > > > +	default:
> > > > +		dev_err(pci->dev, "Unknown IRQ type %d\n", type);
> > > > +		return -EINVAL;
> > > > +	}
> > > > +}
> > > > +
> > > > +static const struct pci_epc_features keembay_pcie_epc_features =
> {
> > > > +	.linkup_notifier	= false,
> > > > +	.msi_capable		= true,
> > > > +	.msix_capable		= true,
> > > > +	.reserved_bar		= BIT(BAR_1) | BIT(BAR_3) |
> BIT(BAR_5),
> > > > +	.bar_fixed_64bit	= BIT(BAR_0) | BIT(BAR_2) | BIT(BAR_4),
> > > > +	.align			= SZ_16K,
> > > > +};
> > > > +
> > > > +static const struct pci_epc_features *
> > > > +keembay_pcie_get_features(struct dw_pcie_ep *ep)
> > > > +{
> > > > +	return &keembay_pcie_epc_features;
> > > > +}
> > > > +
> > > > +static const struct dw_pcie_ep_ops keembay_pcie_ep_ops = {
> > > > +	.ep_init	= keembay_pcie_ep_init,
> > > > +	.raise_irq	= keembay_pcie_ep_raise_irq,
> > > > +	.get_features	= keembay_pcie_get_features,
> > > > +};
> > > > +
> > > > +static const struct dw_pcie_host_ops keembay_pcie_host_ops = {
> > > > +};
> > > > +
> > > > +static int keembay_pcie_add_pcie_port(struct keembay_pcie *pcie,
> > > > +				      struct platform_device *pdev)
> > > > +{
> > > > +	struct dw_pcie *pci = &pcie->pci;
> > > > +	struct pcie_port *pp = &pci->pp;
> > > > +	struct device *dev = &pdev->dev;
> > > > +	u32 val;
> > > > +	int ret;
> > > > +
> > > > +	pp->ops = &keembay_pcie_host_ops;
> > > > +	pp->msi_irq = -ENODEV;
> > > > +
> > > > +	ret = keembay_pcie_setup_msi_irq(pcie);
> > > > +	if (ret)
> > > > +		return ret;
> > > > +
> > > > +	pcie->reset = devm_gpiod_get(dev, "reset", GPIOD_OUT_HIGH);
> > > > +	if (IS_ERR(pcie->reset))
> > > > +		return PTR_ERR(pcie->reset);
> > > > +
> > > > +	ret = keembay_pcie_probe_clocks(pcie);
> > > > +	if (ret)
> > > > +		return ret;
> > > > +
> > > > +	val = readl(pcie->apb_base + PCIE_REGS_PCIE_PHY_CNTL);
> > > > +	val |= PHY0_SRAM_BYPASS;
> > > > +	writel(val, pcie->apb_base + PCIE_REGS_PCIE_PHY_CNTL);
> > > > +
> > > > +	writel(PCIE_DEVICE_TYPE, pcie->apb_base +
> PCIE_REGS_PCIE_CFG);
> > > > +
> > > > +	ret = keembay_pcie_pll_init(pcie);
> > > > +	if (ret)
> > > > +		return ret;
> > > > +
> > > > +	val = readl(pcie->apb_base + PCIE_REGS_PCIE_CFG);
> > > > +	writel(val | PCIE_RSTN, pcie->apb_base +
> PCIE_REGS_PCIE_CFG);
> > > > +	keembay_ep_reset_deassert(pcie);
> > > > +
> > > > +	ret = dw_pcie_host_init(pp);
> > > > +	if (ret) {
> > > > +		keembay_ep_reset_assert(pcie);
> > > > +		dev_err(dev, "Failed to initialize host: %d\n", ret);
> > > > +		return ret;
> > > > +	}
> > > > +
> > > > +	val = readl(pcie->apb_base + PCIE_REGS_INTERRUPT_ENABLE);
> > > > +	if (IS_ENABLED(CONFIG_PCI_MSI))
> > > > +		val |= MSI_CTRL_INT_EN;
> > > > +	writel(val, pcie->apb_base + PCIE_REGS_INTERRUPT_ENABLE);
> > > > +
> > > > +	return 0;
> > > > +}
> > > > +
> > > > +static int keembay_pcie_probe(struct platform_device *pdev)
> > > > +{
> > > > +	const struct keembay_pcie_of_data *data;
> > > > +	struct device *dev = &pdev->dev;
> > > > +	struct keembay_pcie *pcie;
> > > > +	struct dw_pcie *pci;
> > > > +	enum dw_pcie_device_mode mode;
> > > > +
> > > > +	data = device_get_match_data(dev);
> > > > +	if (!data)
> > > > +		return -ENODEV;
> > > > +
> > > > +	mode = (enum dw_pcie_device_mode)data->mode;
> > > > +
> > > > +	pcie = devm_kzalloc(dev, sizeof(*pcie), GFP_KERNEL);
> > > > +	if (!pcie)
> > > > +		return -ENOMEM;
> > > > +
> > > > +	pci = &pcie->pci;
> > > > +	pci->dev = dev;
> > > > +	pci->ops = &keembay_pcie_ops;
> > > > +
> > > > +	pcie->mode = mode;
> > > > +
> > > > +	pcie->apb_base =
> devm_platform_ioremap_resource_byname(pdev,
> > "apb");
> > > > +	if (IS_ERR(pcie->apb_base))
> > > > +		return PTR_ERR(pcie->apb_base);
> > > > +
> > > > +	platform_set_drvdata(pdev, pcie);
> > > > +
> > > > +	switch (pcie->mode) {
> > > > +	case DW_PCIE_RC_TYPE:
> > > > +		if (!IS_ENABLED(CONFIG_PCIE_KEEMBAY_HOST))
> > > > +			return -ENODEV;
> > > > +
> > > > +		return keembay_pcie_add_pcie_port(pcie, pdev);
> > > > +	case DW_PCIE_EP_TYPE:
> > > > +		if (!IS_ENABLED(CONFIG_PCIE_KEEMBAY_EP))
> > > > +			return -ENODEV;
> > > > +
> > > > +		pci->ep.ops = &keembay_pcie_ep_ops;
> > > > +		return dw_pcie_ep_init(&pci->ep);
> > > > +	default:
> > > > +		dev_err(dev, "Invalid device type %d\n", pcie->mode);
> > > > +		return -ENODEV;
> > > > +	}
> > > > +}
> > > > +
> > > > +static const struct keembay_pcie_of_data keembay_pcie_rc_of_data
> =
> > {
> > > > +	.mode = DW_PCIE_RC_TYPE,
> > > > +};
> > > > +
> > > > +static const struct keembay_pcie_of_data keembay_pcie_ep_of_data
> =
> > {
> > > > +	.mode = DW_PCIE_EP_TYPE,
> > > > +};
> > > > +
> > > > +static const struct of_device_id keembay_pcie_of_match[] = {
> > > > +	{
> > > > +		.compatible = "intel,keembay-pcie",
> > > > +		.data = &keembay_pcie_rc_of_data,
> > > > +	},
> > > > +	{
> > > > +		.compatible = "intel,keembay-pcie-ep",
> > > > +		.data = &keembay_pcie_ep_of_data,
> > > > +	},
> > > > +	{}
> > > > +};
> > > > +
> > > > +static struct platform_driver keembay_pcie_driver = {
> > > > +	.driver = {
> > > > +		.name = "keembay-pcie",
> > > > +		.of_match_table = keembay_pcie_of_match,
> > > > +		.suppress_bind_attrs = true,
> > > > +	},
> > > > +	.probe  = keembay_pcie_probe,
> > > > +};
> > > > +builtin_platform_driver(keembay_pcie_driver);
> > > > --
> > > > 2.17.1
> > > >

Re: [PATCH v10 2/2] PCI: keembay: Add support for Intel Keem Bay

[Marc Zyngier]

On 2021-06-25 04:23, Thokala, Srikanth wrote:
> Hi Lorenzo,
> 
>> -----Original Message-----
>> From: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
>> Sent: Monday, June 21, 2021 10:23 PM
>> To: Thokala, Srikanth <srikanth.thokala@intel.com>; maz@kernel.org
>> Cc: robh+dt@kernel.org; linux-pci@vger.kernel.org;
>> devicetree@vger.kernel.org; andriy.shevchenko@linux.intel.com;
>> mgross@linux.intel.com; Raja Subramanian, Lakshmi Bai
>> <lakshmi.bai.raja.subramanian@intel.com>; Sangannavar, Mallikarjunappa
>> <mallikarjunappa.sangannavar@intel.com>; kw@linux.com
>> Subject: Re: [PATCH v10 2/2] PCI: keembay: Add support for Intel Keem 
>> Bay
>> 
>> [+Marc]
>> 
>> On Mon, Jun 07, 2021 at 09:10:44PM +0530, srikanth.thokala@intel.com
>> wrote:
>> [...]
>> 
>> > +static void keembay_pcie_msi_irq_handler(struct irq_desc *desc)
>> > +{
>> > +	struct keembay_pcie *pcie = irq_desc_get_handler_data(desc);
>> > +	struct irq_chip *chip = irq_desc_get_chip(desc);
>> > +	u32 val, mask, status;
>> > +	struct pcie_port *pp;
>> > +
>> > +	chained_irq_enter(chip, desc);
>> > +
>> > +	pp = &pcie->pci.pp;
>> > +	val = readl(pcie->apb_base + PCIE_REGS_INTERRUPT_STATUS);
>> > +	mask = readl(pcie->apb_base + PCIE_REGS_INTERRUPT_ENABLE);
>> > +
>> > +	status = val & mask;
>> > +
>> > +	if (status & MSI_CTRL_INT) {
>> > +		dw_handle_msi_irq(pp);
>> > +		writel(status, pcie->apb_base + PCIE_REGS_INTERRUPT_STATUS);
>> > +	}
>> > +
>> > +	chained_irq_exit(chip, desc);
>> > +}
>> > +
>> > +static int keembay_pcie_setup_msi_irq(struct keembay_pcie *pcie)
>> > +{
>> > +	struct dw_pcie *pci = &pcie->pci;
>> > +	struct device *dev = pci->dev;
>> > +	struct platform_device *pdev = to_platform_device(dev);
>> > +	int irq;
>> > +
>> > +	irq = platform_get_irq_byname(pdev, "pcie");
>> > +	if (irq < 0)
>> > +		return irq;
>> > +
>> > +	irq_set_chained_handler_and_data(irq, keembay_pcie_msi_irq_handler,
>> > +					 pcie);
>> > +
>> 
>> Ok this is yet another DWC MSI incantation and given that Marc worked
>> hard consolidating them let's have a look before we merge it.
>> 
>> IIUC - this IP relies on the DWC logic to handle MSIs + custom
>> registers to detect a pending MSI IRQ because the logic in
>> dw_chained_msi_irq() is *not* enough so you have to register
>> a driver specific chained handler. This looks similar to the dra7xx
>> driver MSI handling but I am not entirely convinced it is needed.
>> 
>> I assume this code in keembay_pcie_msi_irq_handler() is required
>> owing to additional IP logic on top of the standard DWC IP, in
>> particular the PCIE_REGS_INTERRUPT_STATUS write to "clear" the
>> IRQ.
>> 
>> We need more insights before merging it so please provide them.
>> 
>> pp = &pcie->pci.pp;
>> val = readl(pcie->apb_base + PCIE_REGS_INTERRUPT_STATUS);
>> mask = readl(pcie->apb_base + PCIE_REGS_INTERRUPT_ENABLE);
>> 
>> status = val & mask;
>> 
>> if (status & MSI_CTRL_INT) {
>> 	dw_handle_msi_irq(pp);
>> 	writel(status, pcie->apb_base + PCIE_REGS_INTERRUPT_STATUS);
>> }
> 
> Yes, your understanding is correct.
> 
> Additional registers PCIE_REGS_INTERRUPT_ENABLE/_STATUS are provided
> by IP to control the interrupts.
> 
> To receive MSI interrupts, SW must enable bit '8' of _ENABLE register.
> And once a MSI is raised by the End point, bit '8' of _STATUS register
> will be set and it needs to be cleared after servicing the interrupt.

What isn't clear here is whether the other bits that are written back
are significant and have a side effect. If only bit 8 is required,
shouldn't you *only* write this bit back?

Only you can know the answer to this, but it would be good if you
could actually document this deviation from the already wonky
DWC infrastructure.

Thanks,

         M.
-- 
Jazz is not dead. It just smells funny...

RE: [PATCH v10 2/2] PCI: keembay: Add support for Intel Keem Bay

[Thokala, Srikanth]

Hi Marc,

> -----Original Message-----
> From: Marc Zyngier <maz@kernel.org>
> Sent: Monday, July 26, 2021 1:30 PM
> To: Thokala, Srikanth <srikanth.thokala@intel.com>
> Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>; robh+dt@kernel.org;
> linux-pci@vger.kernel.org; devicetree@vger.kernel.org;
> andriy.shevchenko@linux.intel.com; mgross@linux.intel.com; Raja
> Subramanian, Lakshmi Bai <lakshmi.bai.raja.subramanian@intel.com>;
> Sangannavar, Mallikarjunappa <mallikarjunappa.sangannavar@intel.com>;
> kw@linux.com
> Subject: Re: [PATCH v10 2/2] PCI: keembay: Add support for Intel Keem
> Bay
> 
> On 2021-06-25 04:23, Thokala, Srikanth wrote:
> > Hi Lorenzo,
> >
> >> -----Original Message-----
> >> From: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
> >> Sent: Monday, June 21, 2021 10:23 PM
> >> To: Thokala, Srikanth <srikanth.thokala@intel.com>; maz@kernel.org
> >> Cc: robh+dt@kernel.org; linux-pci@vger.kernel.org;
> >> devicetree@vger.kernel.org; andriy.shevchenko@linux.intel.com;
> >> mgross@linux.intel.com; Raja Subramanian, Lakshmi Bai
> >> <lakshmi.bai.raja.subramanian@intel.com>; Sangannavar,
> Mallikarjunappa
> >> <mallikarjunappa.sangannavar@intel.com>; kw@linux.com
> >> Subject: Re: [PATCH v10 2/2] PCI: keembay: Add support for Intel
> Keem
> >> Bay
> >>
> >> [+Marc]
> >>
> >> On Mon, Jun 07, 2021 at 09:10:44PM +0530, srikanth.thokala@intel.com
> >> wrote:
> >> [...]
> >>
> >> > +static void keembay_pcie_msi_irq_handler(struct irq_desc *desc)
> >> > +{
> >> > +	struct keembay_pcie *pcie =
> irq_desc_get_handler_data(desc);
> >> > +	struct irq_chip *chip = irq_desc_get_chip(desc);
> >> > +	u32 val, mask, status;
> >> > +	struct pcie_port *pp;
> >> > +
> >> > +	chained_irq_enter(chip, desc);
> >> > +
> >> > +	pp = &pcie->pci.pp;
> >> > +	val = readl(pcie->apb_base + PCIE_REGS_INTERRUPT_STATUS);
> >> > +	mask = readl(pcie->apb_base + PCIE_REGS_INTERRUPT_ENABLE);
> >> > +
> >> > +	status = val & mask;
> >> > +
> >> > +	if (status & MSI_CTRL_INT) {
> >> > +		dw_handle_msi_irq(pp);
> >> > +		writel(status, pcie->apb_base +
> PCIE_REGS_INTERRUPT_STATUS);
> >> > +	}
> >> > +
> >> > +	chained_irq_exit(chip, desc);
> >> > +}
> >> > +
> >> > +static int keembay_pcie_setup_msi_irq(struct keembay_pcie *pcie)
> >> > +{
> >> > +	struct dw_pcie *pci = &pcie->pci;
> >> > +	struct device *dev = pci->dev;
> >> > +	struct platform_device *pdev = to_platform_device(dev);
> >> > +	int irq;
> >> > +
> >> > +	irq = platform_get_irq_byname(pdev, "pcie");
> >> > +	if (irq < 0)
> >> > +		return irq;
> >> > +
> >> > +	irq_set_chained_handler_and_data(irq,
> keembay_pcie_msi_irq_handler,
> >> > +					 pcie);
> >> > +
> >>
> >> Ok this is yet another DWC MSI incantation and given that Marc
> worked
> >> hard consolidating them let's have a look before we merge it.
> >>
> >> IIUC - this IP relies on the DWC logic to handle MSIs + custom
> >> registers to detect a pending MSI IRQ because the logic in
> >> dw_chained_msi_irq() is *not* enough so you have to register
> >> a driver specific chained handler. This looks similar to the dra7xx
> >> driver MSI handling but I am not entirely convinced it is needed.
> >>
> >> I assume this code in keembay_pcie_msi_irq_handler() is required
> >> owing to additional IP logic on top of the standard DWC IP, in
> >> particular the PCIE_REGS_INTERRUPT_STATUS write to "clear" the
> >> IRQ.
> >>
> >> We need more insights before merging it so please provide them.
> >>
> >> pp = &pcie->pci.pp;
> >> val = readl(pcie->apb_base + PCIE_REGS_INTERRUPT_STATUS);
> >> mask = readl(pcie->apb_base + PCIE_REGS_INTERRUPT_ENABLE);
> >>
> >> status = val & mask;
> >>
> >> if (status & MSI_CTRL_INT) {
> >> 	dw_handle_msi_irq(pp);
> >> 	writel(status, pcie->apb_base + PCIE_REGS_INTERRUPT_STATUS);
> >> }
> >
> > Yes, your understanding is correct.
> >
> > Additional registers PCIE_REGS_INTERRUPT_ENABLE/_STATUS are provided
> > by IP to control the interrupts.
> >
> > To receive MSI interrupts, SW must enable bit '8' of _ENABLE
> register.
> > And once a MSI is raised by the End point, bit '8' of _STATUS
> register
> > will be set and it needs to be cleared after servicing the interrupt.
> 
> What isn't clear here is whether the other bits that are written back
> are significant and have a side effect. If only bit 8 is required,
> shouldn't you *only* write this bit back?
> 
> Only you can know the answer to this, but it would be good if you
> could actually document this deviation from the already wonky
> DWC infrastructure.

SW will only unmask MSI Interrupt i.e. bit '8' in the 'INTERRUPT_ENABLE'
register during the Root Port initialization, other bits of this register
are not significant in this mode.

Thanks!
Srikanth

> 
> Thanks,
> 
>          M.
> --
> Jazz is not dead. It just smells funny...