linux-pci.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [PATCH V4 00/28] Enable Tegra PCIe root port features
@ 2019-05-16  5:52 Manikanta Maddireddy
  2019-05-16  5:52 ` [PATCH V4 01/28] soc/tegra: pmc: Export tegra_powergate_power_on() Manikanta Maddireddy
                   ` (29 more replies)
  0 siblings, 30 replies; 72+ messages in thread
From: Manikanta Maddireddy @ 2019-05-16  5:52 UTC (permalink / raw)
  To: thierry.reding, bhelgaas, robh+dt, mark.rutland, jonathanh,
	lorenzo.pieralisi, vidyas
  Cc: linux-tegra, linux-pci, devicetree, Manikanta Maddireddy

This series of patches adds,
- Tegra root port features like Gen2, AER, etc
- Power and perf optimizations
- Fixes like "power up sequence", "dev_err prints", etc

This series of patches are tested on Tegra186 based Jetson-TX2, Tegra210
based Jetson-TX1, T124 based Jetson-TK1 platforms, Tegra20 and Tegra30
platforms.

Manikanta Maddireddy (28):
  soc/tegra: pmc: Export tegra_powergate_power_on()
  PCI: tegra: Handle failure cases in tegra_pcie_power_on()
  PCI: tegra: Rearrange Tegra PCIe driver functions
  PCI: tegra: Mask AFI_INTR in runtime suspend
  PCI: tegra: Fix PCIe host power up sequence
  PCI: tegra: Add PCIe Gen2 link speed support
  PCI: tegra: Advertise PCIe Advanced Error Reporting (AER) capability
  PCI: tegra: Program UPHY electrical settings for Tegra210
  PCI: tegra: Enable opportunistic UpdateFC and ACK
  PCI: tegra: Disable AFI dynamic clock gating
  PCI: tegra: Process pending DLL transactions before entering L1 or L2
  PCI: tegra: Enable PCIe xclk clock clamping
  PCI: tegra: Increase the deskew retry time
  PCI: tegra: Add SW fixup for RAW violations
  PCI: tegra: Update flow control timer frequency in Tegra210
  PCI: tegra: Set target speed as Gen1 before starting LTSSM
  PCI: tegra: Fix PLLE power down issue due to CLKREQ# signal
  PCI: tegra: Program AFI_CACHE* registers only for Tegra20
  PCI: tegra: Change PRSNT_SENSE IRQ log to debug
  PCI: tegra: Use legacy IRQ for port service drivers
  PCI: tegra: Add AFI_PEX2_CTRL reg offset as part of soc struct
  PCI: tegra: Access endpoint config only if PCIe link is up
  dt-bindings: pci: tegra: Document PCIe DPD pinctrl optional prop
  arm64: tegra: Add PEX DPD states as pinctrl properties
  PCI: tegra: Put PEX CLK & BIAS pads in DPD mode
  PCI: Add DT binding for "reset-gpios" property
  PCI: tegra: Add support for GPIO based PERST#
  PCI: tegra: Change link retry log level to debug

 .../bindings/pci/nvidia,tegra20-pcie.txt      |   8 +
 Documentation/devicetree/bindings/pci/pci.txt |   3 +
 arch/arm64/boot/dts/nvidia/tegra210.dtsi      |  19 +
 drivers/pci/controller/pci-tegra.c            | 615 +++++++++++++++---
 drivers/soc/tegra/pmc.c                       |   1 +
 5 files changed, 566 insertions(+), 80 deletions(-)

-- 
2.17.1


^ permalink raw reply	[flat|nested] 72+ messages in thread

* [PATCH V4 01/28] soc/tegra: pmc: Export tegra_powergate_power_on()
  2019-05-16  5:52 [PATCH V4 00/28] Enable Tegra PCIe root port features Manikanta Maddireddy
@ 2019-05-16  5:52 ` Manikanta Maddireddy
  2019-05-16  5:52 ` [PATCH V4 02/28] PCI: tegra: Handle failure cases in tegra_pcie_power_on() Manikanta Maddireddy
                   ` (28 subsequent siblings)
  29 siblings, 0 replies; 72+ messages in thread
From: Manikanta Maddireddy @ 2019-05-16  5:52 UTC (permalink / raw)
  To: thierry.reding, bhelgaas, robh+dt, mark.rutland, jonathanh,
	lorenzo.pieralisi, vidyas
  Cc: linux-tegra, linux-pci, devicetree, Manikanta Maddireddy

tegra_powergate_sequence_power_up() powers up partition and also enables
clock & reset. However, if a controller like PCIe have multiple clocks
& resets and they need to be enabled in a sequence, driver must use
standalone function tegra_powergate_power_on() to power up partition.

Export tegra_powergate_power_on() to allow Tegra controller drivers to
unpower gate partition independent to clock & reset.

Signed-off-by: Manikanta Maddireddy <mmaddireddy@nvidia.com>
Acked-by: Thierry Reding <treding@nvidia.com>
---
V4: No change

V3: No change

V2: No change

 drivers/soc/tegra/pmc.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/drivers/soc/tegra/pmc.c b/drivers/soc/tegra/pmc.c
index 0c5f79528e5f..cb3de81348bd 100644
--- a/drivers/soc/tegra/pmc.c
+++ b/drivers/soc/tegra/pmc.c
@@ -701,6 +701,7 @@ int tegra_powergate_power_on(unsigned int id)
 
 	return tegra_powergate_set(pmc, id, true);
 }
+EXPORT_SYMBOL(tegra_powergate_power_on);
 
 /**
  * tegra_powergate_power_off() - power off partition
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 72+ messages in thread

* [PATCH V4 02/28] PCI: tegra: Handle failure cases in tegra_pcie_power_on()
  2019-05-16  5:52 [PATCH V4 00/28] Enable Tegra PCIe root port features Manikanta Maddireddy
  2019-05-16  5:52 ` [PATCH V4 01/28] soc/tegra: pmc: Export tegra_powergate_power_on() Manikanta Maddireddy
@ 2019-05-16  5:52 ` Manikanta Maddireddy
  2019-05-16  5:52 ` [PATCH V4 03/28] PCI: tegra: Rearrange Tegra PCIe driver functions Manikanta Maddireddy
                   ` (27 subsequent siblings)
  29 siblings, 0 replies; 72+ messages in thread
From: Manikanta Maddireddy @ 2019-05-16  5:52 UTC (permalink / raw)
  To: thierry.reding, bhelgaas, robh+dt, mark.rutland, jonathanh,
	lorenzo.pieralisi, vidyas
  Cc: linux-tegra, linux-pci, devicetree, Manikanta Maddireddy

Unroll the PCIe power on sequence if any one of the steps fail in
tegra_pcie_power_on().

Signed-off-by: Manikanta Maddireddy <mmaddireddy@nvidia.com>
Acked-by: Thierry Reding <treding@nvidia.com>
---
V4: No change

V3: No change

V2: New patch to handle error cleanup in tegra_pcie_power_on().

 drivers/pci/controller/pci-tegra.c | 26 +++++++++++++++++++++-----
 1 file changed, 21 insertions(+), 5 deletions(-)

diff --git a/drivers/pci/controller/pci-tegra.c b/drivers/pci/controller/pci-tegra.c
index f4f53d092e00..8235d937951b 100644
--- a/drivers/pci/controller/pci-tegra.c
+++ b/drivers/pci/controller/pci-tegra.c
@@ -1052,7 +1052,7 @@ static int tegra_pcie_power_on(struct tegra_pcie *pcie)
 		err = clk_prepare_enable(pcie->pex_clk);
 		if (err) {
 			dev_err(dev, "failed to enable PEX clock: %d\n", err);
-			return err;
+			goto regulator_disable;
 		}
 		reset_control_deassert(pcie->pex_rst);
 	} else {
@@ -1061,7 +1061,7 @@ static int tegra_pcie_power_on(struct tegra_pcie *pcie)
 							pcie->pex_rst);
 		if (err) {
 			dev_err(dev, "powerup sequence failed: %d\n", err);
-			return err;
+			goto regulator_disable;
 		}
 	}
 
@@ -1070,24 +1070,40 @@ static int tegra_pcie_power_on(struct tegra_pcie *pcie)
 	err = clk_prepare_enable(pcie->afi_clk);
 	if (err < 0) {
 		dev_err(dev, "failed to enable AFI clock: %d\n", err);
-		return err;
+		goto powergate;
 	}
 
 	if (soc->has_cml_clk) {
 		err = clk_prepare_enable(pcie->cml_clk);
 		if (err < 0) {
 			dev_err(dev, "failed to enable CML clock: %d\n", err);
-			return err;
+			goto disable_afi_clk;
 		}
 	}
 
 	err = clk_prepare_enable(pcie->pll_e);
 	if (err < 0) {
 		dev_err(dev, "failed to enable PLLE clock: %d\n", err);
-		return err;
+		goto disable_cml_clk;
 	}
 
 	return 0;
+
+disable_cml_clk:
+	if (soc->has_cml_clk)
+		clk_disable_unprepare(pcie->cml_clk);
+disable_afi_clk:
+	clk_disable_unprepare(pcie->afi_clk);
+powergate:
+	reset_control_assert(pcie->afi_rst);
+	reset_control_assert(pcie->pex_rst);
+	clk_disable_unprepare(pcie->pex_clk);
+	if (!dev->pm_domain)
+		tegra_powergate_power_off(TEGRA_POWERGATE_PCIE);
+regulator_disable:
+	regulator_bulk_disable(pcie->num_supplies, pcie->supplies);
+
+	return err;
 }
 
 static int tegra_pcie_clocks_get(struct tegra_pcie *pcie)
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 72+ messages in thread

* [PATCH V4 03/28] PCI: tegra: Rearrange Tegra PCIe driver functions
  2019-05-16  5:52 [PATCH V4 00/28] Enable Tegra PCIe root port features Manikanta Maddireddy
  2019-05-16  5:52 ` [PATCH V4 01/28] soc/tegra: pmc: Export tegra_powergate_power_on() Manikanta Maddireddy
  2019-05-16  5:52 ` [PATCH V4 02/28] PCI: tegra: Handle failure cases in tegra_pcie_power_on() Manikanta Maddireddy
@ 2019-05-16  5:52 ` Manikanta Maddireddy
  2019-05-16  5:52 ` [PATCH V4 04/28] PCI: tegra: Mask AFI_INTR in runtime suspend Manikanta Maddireddy
                   ` (26 subsequent siblings)
  29 siblings, 0 replies; 72+ messages in thread
From: Manikanta Maddireddy @ 2019-05-16  5:52 UTC (permalink / raw)
  To: thierry.reding, bhelgaas, robh+dt, mark.rutland, jonathanh,
	lorenzo.pieralisi, vidyas
  Cc: linux-tegra, linux-pci, devicetree, Manikanta Maddireddy

Tegra PCIe has register spec for,
 - AXI to FPCI(AFI) bridge
 - Multiple PCIe root ports
 - PCIe PHY
 - PCIe pad control

Rearrange Tegra PCIe driver functions such that each function programs
required module only.
 - tegra_pcie_enable_controller(): Program AFI module and enable PCIe
controller.
 - tegra_pcie_phy_power_on(): Bring up PCIe PHY.
 - tegra_pcie_apply_pad_settings(): Program PCIe REFCLK pad settings.
 - tegra_pcie_enable_ports(): Program each root port and bring up PCIe
link.

Signed-off-by: Manikanta Maddireddy <mmaddireddy@nvidia.com>
Acked-by: Thierry Reding <treding@nvidia.com>
---
V4: No change

V3: No change

V2: This is new patch in V2

 drivers/pci/controller/pci-tegra.c | 70 +++++++++++++-----------------
 1 file changed, 30 insertions(+), 40 deletions(-)

diff --git a/drivers/pci/controller/pci-tegra.c b/drivers/pci/controller/pci-tegra.c
index 8235d937951b..bb3c0af9c830 100644
--- a/drivers/pci/controller/pci-tegra.c
+++ b/drivers/pci/controller/pci-tegra.c
@@ -852,7 +852,6 @@ static int tegra_pcie_port_phy_power_off(struct tegra_pcie_port *port)
 static int tegra_pcie_phy_power_on(struct tegra_pcie *pcie)
 {
 	struct device *dev = pcie->dev;
-	const struct tegra_pcie_soc *soc = pcie->soc;
 	struct tegra_pcie_port *port;
 	int err;
 
@@ -878,12 +877,6 @@ static int tegra_pcie_phy_power_on(struct tegra_pcie *pcie)
 		}
 	}
 
-	/* Configure the reference clock driver */
-	pads_writel(pcie, soc->pads_refclk_cfg0, PADS_REFCLK_CFG0);
-
-	if (soc->num_ports > 2)
-		pads_writel(pcie, soc->pads_refclk_cfg1, PADS_REFCLK_CFG1);
-
 	return 0;
 }
 
@@ -918,13 +911,11 @@ static int tegra_pcie_phy_power_off(struct tegra_pcie *pcie)
 	return 0;
 }
 
-static int tegra_pcie_enable_controller(struct tegra_pcie *pcie)
+static void tegra_pcie_enable_controller(struct tegra_pcie *pcie)
 {
-	struct device *dev = pcie->dev;
 	const struct tegra_pcie_soc *soc = pcie->soc;
 	struct tegra_pcie_port *port;
 	unsigned long value;
-	int err;
 
 	/* enable PLL power down */
 	if (pcie->phy) {
@@ -958,14 +949,6 @@ static int tegra_pcie_enable_controller(struct tegra_pcie *pcie)
 		afi_writel(pcie, value, AFI_FUSE);
 	}
 
-	if (soc->program_uphy) {
-		err = tegra_pcie_phy_power_on(pcie);
-		if (err < 0) {
-			dev_err(dev, "failed to power on PHY(s): %d\n", err);
-			return err;
-		}
-	}
-
 	/* take the PCIe interface module out of reset */
 	reset_control_deassert(pcie->pcie_xrst);
 
@@ -989,22 +972,6 @@ static int tegra_pcie_enable_controller(struct tegra_pcie *pcie)
 
 	/* disable all exceptions */
 	afi_writel(pcie, 0, AFI_FPCI_ERROR_MASKS);
-
-	return 0;
-}
-
-static void tegra_pcie_disable_controller(struct tegra_pcie *pcie)
-{
-	int err;
-
-	reset_control_assert(pcie->pcie_xrst);
-
-	if (pcie->soc->program_uphy) {
-		err = tegra_pcie_phy_power_off(pcie);
-		if (err < 0)
-			dev_err(pcie->dev, "failed to power off PHY(s): %d\n",
-				err);
-	}
 }
 
 static void tegra_pcie_power_off(struct tegra_pcie *pcie)
@@ -1106,6 +1073,17 @@ static int tegra_pcie_power_on(struct tegra_pcie *pcie)
 	return err;
 }
 
+static void tegra_pcie_apply_pad_settings(struct tegra_pcie *pcie)
+{
+	const struct tegra_pcie_soc *soc = pcie->soc;
+
+	/* Configure the reference clock driver */
+	pads_writel(pcie, soc->pads_refclk_cfg0, PADS_REFCLK_CFG0);
+
+	if (soc->num_ports > 2)
+		pads_writel(pcie, soc->pads_refclk_cfg1, PADS_REFCLK_CFG1);
+}
+
 static int tegra_pcie_clocks_get(struct tegra_pcie *pcie)
 {
 	struct device *dev = pcie->dev;
@@ -2482,16 +2460,23 @@ static int __maybe_unused tegra_pcie_pm_suspend(struct device *dev)
 {
 	struct tegra_pcie *pcie = dev_get_drvdata(dev);
 	struct tegra_pcie_port *port;
+	int err;
 
 	list_for_each_entry(port, &pcie->ports, list)
 		tegra_pcie_pme_turnoff(port);
 
 	tegra_pcie_disable_ports(pcie);
 
+	if (pcie->soc->program_uphy) {
+		err = tegra_pcie_phy_power_off(pcie);
+		if (err < 0)
+			dev_err(dev, "failed to power off PHY(s): %d\n", err);
+	}
+
 	if (IS_ENABLED(CONFIG_PCI_MSI))
 		tegra_pcie_disable_msi(pcie);
 
-	tegra_pcie_disable_controller(pcie);
+	reset_control_assert(pcie->pcie_xrst);
 	tegra_pcie_power_off(pcie);
 
 	return 0;
@@ -2507,16 +2492,21 @@ static int __maybe_unused tegra_pcie_pm_resume(struct device *dev)
 		dev_err(dev, "tegra pcie power on fail: %d\n", err);
 		return err;
 	}
-	err = tegra_pcie_enable_controller(pcie);
-	if (err) {
-		dev_err(dev, "tegra pcie controller enable fail: %d\n", err);
-		goto poweroff;
-	}
+	tegra_pcie_enable_controller(pcie);
 	tegra_pcie_setup_translations(pcie);
 
 	if (IS_ENABLED(CONFIG_PCI_MSI))
 		tegra_pcie_enable_msi(pcie);
 
+	if (pcie->soc->program_uphy) {
+		err = tegra_pcie_phy_power_on(pcie);
+		if (err < 0) {
+			dev_err(dev, "failed to power on PHY(s): %d\n", err);
+			goto poweroff;
+		}
+	}
+
+	tegra_pcie_apply_pad_settings(pcie);
 	tegra_pcie_enable_ports(pcie);
 
 	return 0;
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 72+ messages in thread

* [PATCH V4 04/28] PCI: tegra: Mask AFI_INTR in runtime suspend
  2019-05-16  5:52 [PATCH V4 00/28] Enable Tegra PCIe root port features Manikanta Maddireddy
                   ` (2 preceding siblings ...)
  2019-05-16  5:52 ` [PATCH V4 03/28] PCI: tegra: Rearrange Tegra PCIe driver functions Manikanta Maddireddy
@ 2019-05-16  5:52 ` Manikanta Maddireddy
  2019-06-04 13:08   ` Thierry Reding
  2019-05-16  5:52 ` [PATCH V4 05/28] PCI: tegra: Fix PCIe host power up sequence Manikanta Maddireddy
                   ` (25 subsequent siblings)
  29 siblings, 1 reply; 72+ messages in thread
From: Manikanta Maddireddy @ 2019-05-16  5:52 UTC (permalink / raw)
  To: thierry.reding, bhelgaas, robh+dt, mark.rutland, jonathanh,
	lorenzo.pieralisi, vidyas
  Cc: linux-tegra, linux-pci, devicetree, Manikanta Maddireddy

AFI_INTR is unmasked in tegra_pcie_enable_controller(), mask it to avoid
unwanted interrupts raised by AFI after pex_rst is asserted.

Following sequence triggers such scenario,
 - tegra_pcie_remove() triggers runtime suspend
 - pex_rst is asserted in runtime suspend
 - PRSNT_MAP bit field in RP_PRIV_MISC register changes from EP_PRSNT to
   EP_ABSNT
 - This is sensed by AFI and triggers "Slot present pin change" interrupt
 - tegra_pcie_isr() function accesses AFI register when runtime suspend
   is going through power off sequence

rmmod pci-tegra
 pci_generic_config_write32: 108 callbacks suppressed
 pci_bus 0002:00: 2-byte config write to 0002:00:02.0 offset 0x4c may corrupt adjacent RW1C bits
 pci_bus 0002:00: 2-byte config write to 0002:00:02.0 offset 0x9c may corrupt adjacent RW1C bits
 pci_bus 0002:00: 2-byte config write to 0002:00:02.0 offset 0x88 may corrupt adjacent RW1C bits
 pci_bus 0002:00: 2-byte config write to 0002:00:02.0 offset 0x90 may corrupt adjacent RW1C bits
 pci_bus 0002:00: 2-byte config write to 0002:00:02.0 offset 0x4 may corrupt adjacent RW1C bits
 igb 0002:04:00.1: removed PHC on enP2p4s0f1
 igb 0002:04:00.0: removed PHC on enP2p4s0f0
 pci_bus 0002:00: 2-byte config write to 0002:00:01.0 offset 0x4c may corrupt adjacent RW1C bits
 pci_bus 0002:00: 2-byte config write to 0002:00:01.0 offset 0x9c may corrupt adjacent RW1C bits
 pci_bus 0002:00: 2-byte config write to 0002:00:01.0 offset 0x88 may corrupt adjacent RW1C bits
 pci_bus 0002:00: 2-byte config write to 0002:00:01.0 offset 0x90 may corrupt adjacent RW1C bits
 pci_bus 0002:00: 2-byte config write to 0002:00:01.0 offset 0x4 may corrupt adjacent RW1C bits
 rcu: INFO: rcu_preempt self-detected stall on CPU
 SError Interrupt on CPU0, code 0xbf000002 -- SError
 CPU: 0 PID: 0 Comm: swapper/0 Tainted: G        W         5.1.0-rc3-next-20190405-00027-gcd8110499e6f-dirty #42
 Hardware name: NVIDIA Jetson TX1 Developer Kit (DT)
 pstate: 20000085 (nzCv daIf -PAN -UAO)
 pc : tegra_pcie_isr+0x58/0x178 [pci_tegra]
 lr : tegra_pcie_isr+0x40/0x178 [pci_tegra]
 sp : ffff000010003da0
 x29: ffff000010003da0 x28: 0000000000000000
 x27: ffff8000f9e61000 x26: ffff000010fbf420
 x25: ffff000011427f93 x24: ffff8000fa600410
 x23: ffff00001129d000 x22: ffff00001129d000
 x21: ffff8000f18bf3c0 x20: 0000000000000070
 x19: 00000000ffffffff x18: 0000000000000000
 x17: 0000000000000000 x16: 0000000000000000
 x15: 0000000000000000 x14: ffff000008d40a48
 x13: ffff000008d40a30 x12: ffff000008d40a20
 x11: ffff000008d40a10 x10: ffff000008d40a00
 x9 : ffff000008d409e8 x8 : ffff000008d40ae8
 x7 : ffff000008d40ad0 x6 : ffff000010003e58
 x5 : ffff8000fac00248 x4 : 0000000000000000
 x3 : ffff000008d40b08 x2 : fffffffffffffff8
 x1 : ffff000008d3f4e8 x0 : 00000000ffffffff
 Kernel panic - not syncing: Asynchronous SError Interrupt
 CPU: 0 PID: 0 Comm: swapper/0 Tainted: G        W         5.1.0-rc3-next-20190405-00027-gcd8110499e6f-dirty #42
 Hardware name: NVIDIA Jetson TX1 Developer Kit (DT)
 Call trace:
  dump_backtrace+0x0/0x158
  show_stack+0x14/0x20
  dump_stack+0xa8/0xcc
  panic+0x140/0x2f4
  nmi_panic+0x6c/0x70
  arm64_serror_panic+0x74/0x80
  __pte_error+0x0/0x28
  el1_error+0x84/0xf8
  tegra_pcie_isr+0x58/0x178 [pci_tegra]
  __handle_irq_event_percpu+0x70/0x198
  handle_irq_event_percpu+0x34/0x88
  handle_irq_event+0x48/0x78
  handle_fasteoi_irq+0xb4/0x190
  generic_handle_irq+0x24/0x38
  __handle_domain_irq+0x5c/0xb8
  gic_handle_irq+0x58/0xa8
  el1_irq+0xb8/0x180
  cpuidle_enter_state+0x138/0x358
  cpuidle_enter+0x18/0x20
  call_cpuidle+0x1c/0x48
  do_idle+0x230/0x2d0
  cpu_startup_entry+0x20/0x28
  rest_init+0xd4/0xe0
  arch_call_rest_init+0xc/0x14
  start_kernel+0x444/0x470

Signed-off-by: Manikanta Maddireddy <mmaddireddy@nvidia.com>
---
V4: No change

V3:
* Update the commit log and comment to reflect why this fix is required
* MSI interrupt is not disabled

V2: This is new patch in V2

 drivers/pci/controller/pci-tegra.c | 14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/drivers/pci/controller/pci-tegra.c b/drivers/pci/controller/pci-tegra.c
index bb3c0af9c830..d0e517f084a1 100644
--- a/drivers/pci/controller/pci-tegra.c
+++ b/drivers/pci/controller/pci-tegra.c
@@ -1622,6 +1622,15 @@ static int tegra_pcie_disable_msi(struct tegra_pcie *pcie)
 	return 0;
 }
 
+static void tegra_pcie_disable_interrupts(struct tegra_pcie *pcie)
+{
+	u32 value;
+
+	value = afi_readl(pcie, AFI_INTR_MASK);
+	value &= ~AFI_INTR_MASK_INT_MASK;
+	afi_writel(pcie, value, AFI_INTR_MASK);
+}
+
 static int tegra_pcie_get_xbar_config(struct tegra_pcie *pcie, u32 lanes,
 				      u32 *xbar)
 {
@@ -2466,6 +2475,11 @@ static int __maybe_unused tegra_pcie_pm_suspend(struct device *dev)
 		tegra_pcie_pme_turnoff(port);
 
 	tegra_pcie_disable_ports(pcie);
+	/*
+	 * AFI_INTR is unmasked in tegra_pcie_enable_controller(), mask it to
+	 * avoid unwanted interrupts raised by AFI after pex_rst is asserted.
+	 */
+	tegra_pcie_disable_interrupts(pcie);
 
 	if (pcie->soc->program_uphy) {
 		err = tegra_pcie_phy_power_off(pcie);
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 72+ messages in thread

* [PATCH V4 05/28] PCI: tegra: Fix PCIe host power up sequence
  2019-05-16  5:52 [PATCH V4 00/28] Enable Tegra PCIe root port features Manikanta Maddireddy
                   ` (3 preceding siblings ...)
  2019-05-16  5:52 ` [PATCH V4 04/28] PCI: tegra: Mask AFI_INTR in runtime suspend Manikanta Maddireddy
@ 2019-05-16  5:52 ` Manikanta Maddireddy
  2019-05-16  5:52 ` [PATCH V4 06/28] PCI: tegra: Add PCIe Gen2 link speed support Manikanta Maddireddy
                   ` (24 subsequent siblings)
  29 siblings, 0 replies; 72+ messages in thread
From: Manikanta Maddireddy @ 2019-05-16  5:52 UTC (permalink / raw)
  To: thierry.reding, bhelgaas, robh+dt, mark.rutland, jonathanh,
	lorenzo.pieralisi, vidyas
  Cc: linux-tegra, linux-pci, devicetree, Manikanta Maddireddy

PCIe host power up sequence involves programming AFI(AXI to FPCI bridge)
registers first and then PCIe registers. Otherwise AFI register settings
may not latch to PCIe IP.

PCIe root port starts LTSSM as soon as PCIe xrst is deasserted.
So deassert PCIe xrst after programming PCIe registers.

Modify PCIe power up sequence as follows,
 - Power ungate PCIe partition
 - Enable AFI clock
 - Deassert AFI reset
 - Program AFI registers
 - Enable PCIe clock
 - Deassert PCIe reset
 - Program PCIe PHY
 - Program PCIe pad control registers
 - Program PCIe root port registers
 - Deassert PCIe xrst to start LTSSM

Signed-off-by: Manikanta Maddireddy <mmaddireddy@nvidia.com>
Acked-by: Thierry Reding <treding@nvidia.com>
---
V4: No change

V3: No change

V2: Error cleanup changes are moved to new patch and only sequence
correction is done in this patch.

 drivers/pci/controller/pci-tegra.c | 52 +++++++++++++++++-------------
 1 file changed, 30 insertions(+), 22 deletions(-)

diff --git a/drivers/pci/controller/pci-tegra.c b/drivers/pci/controller/pci-tegra.c
index d0e517f084a1..015add4e345b 100644
--- a/drivers/pci/controller/pci-tegra.c
+++ b/drivers/pci/controller/pci-tegra.c
@@ -949,9 +949,6 @@ static void tegra_pcie_enable_controller(struct tegra_pcie *pcie)
 		afi_writel(pcie, value, AFI_FUSE);
 	}
 
-	/* take the PCIe interface module out of reset */
-	reset_control_deassert(pcie->pcie_xrst);
-
 	/* finally enable PCIe */
 	value = afi_readl(pcie, AFI_CONFIGURATION);
 	value |= AFI_CONFIGURATION_EN_FPCI;
@@ -981,13 +978,11 @@ static void tegra_pcie_power_off(struct tegra_pcie *pcie)
 	int err;
 
 	reset_control_assert(pcie->afi_rst);
-	reset_control_assert(pcie->pex_rst);
 
 	clk_disable_unprepare(pcie->pll_e);
 	if (soc->has_cml_clk)
 		clk_disable_unprepare(pcie->cml_clk);
 	clk_disable_unprepare(pcie->afi_clk);
-	clk_disable_unprepare(pcie->pex_clk);
 
 	if (!dev->pm_domain)
 		tegra_powergate_power_off(TEGRA_POWERGATE_PCIE);
@@ -1015,25 +1010,19 @@ static int tegra_pcie_power_on(struct tegra_pcie *pcie)
 	if (err < 0)
 		dev_err(dev, "failed to enable regulators: %d\n", err);
 
-	if (dev->pm_domain) {
-		err = clk_prepare_enable(pcie->pex_clk);
+	if (!dev->pm_domain) {
+		err = tegra_powergate_power_on(TEGRA_POWERGATE_PCIE);
 		if (err) {
-			dev_err(dev, "failed to enable PEX clock: %d\n", err);
+			dev_err(dev, "failed to power ungate: %d\n", err);
 			goto regulator_disable;
 		}
-		reset_control_deassert(pcie->pex_rst);
-	} else {
-		err = tegra_powergate_sequence_power_up(TEGRA_POWERGATE_PCIE,
-							pcie->pex_clk,
-							pcie->pex_rst);
+		err = tegra_powergate_remove_clamping(TEGRA_POWERGATE_PCIE);
 		if (err) {
-			dev_err(dev, "powerup sequence failed: %d\n", err);
-			goto regulator_disable;
+			dev_err(dev, "failed to remove clamp: %d\n", err);
+			goto powergate;
 		}
 	}
 
-	reset_control_deassert(pcie->afi_rst);
-
 	err = clk_prepare_enable(pcie->afi_clk);
 	if (err < 0) {
 		dev_err(dev, "failed to enable AFI clock: %d\n", err);
@@ -1054,6 +1043,8 @@ static int tegra_pcie_power_on(struct tegra_pcie *pcie)
 		goto disable_cml_clk;
 	}
 
+	reset_control_deassert(pcie->afi_rst);
+
 	return 0;
 
 disable_cml_clk:
@@ -1062,9 +1053,6 @@ static int tegra_pcie_power_on(struct tegra_pcie *pcie)
 disable_afi_clk:
 	clk_disable_unprepare(pcie->afi_clk);
 powergate:
-	reset_control_assert(pcie->afi_rst);
-	reset_control_assert(pcie->pex_rst);
-	clk_disable_unprepare(pcie->pex_clk);
 	if (!dev->pm_domain)
 		tegra_powergate_power_off(TEGRA_POWERGATE_PCIE);
 regulator_disable:
@@ -2111,7 +2099,12 @@ static void tegra_pcie_enable_ports(struct tegra_pcie *pcie)
 			 port->index, port->lanes);
 
 		tegra_pcie_port_enable(port);
+	}
 
+	/* Start LTSSM from Tegra side */
+	reset_control_deassert(pcie->pcie_xrst);
+
+	list_for_each_entry_safe(port, tmp, &pcie->ports, list) {
 		if (tegra_pcie_port_check_link(port))
 			continue;
 
@@ -2126,6 +2119,8 @@ static void tegra_pcie_disable_ports(struct tegra_pcie *pcie)
 {
 	struct tegra_pcie_port *port, *tmp;
 
+	reset_control_assert(pcie->pcie_xrst);
+
 	list_for_each_entry_safe(port, tmp, &pcie->ports, list)
 		tegra_pcie_port_disable(port);
 }
@@ -2487,10 +2482,12 @@ static int __maybe_unused tegra_pcie_pm_suspend(struct device *dev)
 			dev_err(dev, "failed to power off PHY(s): %d\n", err);
 	}
 
+	reset_control_assert(pcie->pex_rst);
+	clk_disable_unprepare(pcie->pex_clk);
+
 	if (IS_ENABLED(CONFIG_PCI_MSI))
 		tegra_pcie_disable_msi(pcie);
 
-	reset_control_assert(pcie->pcie_xrst);
 	tegra_pcie_power_off(pcie);
 
 	return 0;
@@ -2512,11 +2509,19 @@ static int __maybe_unused tegra_pcie_pm_resume(struct device *dev)
 	if (IS_ENABLED(CONFIG_PCI_MSI))
 		tegra_pcie_enable_msi(pcie);
 
+	err = clk_prepare_enable(pcie->pex_clk);
+	if (err) {
+		dev_err(dev, "failed to enable PEX clock: %d\n", err);
+		goto poweroff;
+	}
+
+	reset_control_deassert(pcie->pex_rst);
+
 	if (pcie->soc->program_uphy) {
 		err = tegra_pcie_phy_power_on(pcie);
 		if (err < 0) {
 			dev_err(dev, "failed to power on PHY(s): %d\n", err);
-			goto poweroff;
+			goto disable_pex_clk;
 		}
 	}
 
@@ -2525,6 +2530,9 @@ static int __maybe_unused tegra_pcie_pm_resume(struct device *dev)
 
 	return 0;
 
+disable_pex_clk:
+	reset_control_assert(pcie->pex_rst);
+	clk_disable_unprepare(pcie->pex_clk);
 poweroff:
 	tegra_pcie_power_off(pcie);
 
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 72+ messages in thread

* [PATCH V4 06/28] PCI: tegra: Add PCIe Gen2 link speed support
  2019-05-16  5:52 [PATCH V4 00/28] Enable Tegra PCIe root port features Manikanta Maddireddy
                   ` (4 preceding siblings ...)
  2019-05-16  5:52 ` [PATCH V4 05/28] PCI: tegra: Fix PCIe host power up sequence Manikanta Maddireddy
@ 2019-05-16  5:52 ` Manikanta Maddireddy
  2019-05-16  5:52 ` [PATCH V4 07/28] PCI: tegra: Advertise PCIe Advanced Error Reporting (AER) capability Manikanta Maddireddy
                   ` (23 subsequent siblings)
  29 siblings, 0 replies; 72+ messages in thread
From: Manikanta Maddireddy @ 2019-05-16  5:52 UTC (permalink / raw)
  To: thierry.reding, bhelgaas, robh+dt, mark.rutland, jonathanh,
	lorenzo.pieralisi, vidyas
  Cc: linux-tegra, linux-pci, devicetree, Manikanta Maddireddy

Tegra124, Tegra132, Tegra210 and Tegra186 support Gen2 link speed. After
PCIe link is up in Gen1, set target link speed as Gen2 and retrain link.
Link switches to Gen2 speed if Gen2 capable end point is connected, else
link stays in Gen1.

Per PCIe 4.0r0.9 sec 7.6.3.7 implementation note, driver need to wait for
PCIe LTSSM to come back from recovery before retraining the link.

Signed-off-by: Manikanta Maddireddy <mmaddireddy@nvidia.com>
Acked-by: Thierry Reding <treding@nvidia.com>
---
V4: No change

V3: Added blank line after each while loop.

V2: Changed "for loop" to "while", to make it compact and handled coding
style comments.

 drivers/pci/controller/pci-tegra.c | 64 ++++++++++++++++++++++++++++++
 1 file changed, 64 insertions(+)

diff --git a/drivers/pci/controller/pci-tegra.c b/drivers/pci/controller/pci-tegra.c
index 015add4e345b..cd542a8af618 100644
--- a/drivers/pci/controller/pci-tegra.c
+++ b/drivers/pci/controller/pci-tegra.c
@@ -191,6 +191,8 @@
 #define  RP_LINK_CONTROL_STATUS_DL_LINK_ACTIVE	0x20000000
 #define  RP_LINK_CONTROL_STATUS_LINKSTAT_MASK	0x3fff0000
 
+#define RP_LINK_CONTROL_STATUS_2		0x000000b0
+
 #define PADS_CTL_SEL		0x0000009c
 
 #define PADS_CTL		0x000000a0
@@ -226,6 +228,7 @@
 #define PADS_REFCLK_CFG_DRVI_SHIFT		12 /* 15:12 */
 
 #define PME_ACK_TIMEOUT 10000
+#define LINK_RETRAIN_TIMEOUT 100000 /* in usec */
 
 struct tegra_msi {
 	struct msi_controller chip;
@@ -2089,6 +2092,64 @@ static bool tegra_pcie_port_check_link(struct tegra_pcie_port *port)
 	return false;
 }
 
+static void tegra_pcie_change_link_speed(struct tegra_pcie *pcie)
+{
+	struct device *dev = pcie->dev;
+	struct tegra_pcie_port *port, *tmp;
+	ktime_t deadline;
+	u32 value;
+
+	list_for_each_entry_safe(port, tmp, &pcie->ports, list) {
+		/*
+		 * "Supported Link Speeds Vector" in "Link Capabilities 2"
+		 * is not supported by Tegra. tegra_pcie_change_link_speed()
+		 * is called only for Tegra chips which support Gen2.
+		 * So there no harm if supported link speed is not verified.
+		 */
+		value = readl(port->base + RP_LINK_CONTROL_STATUS_2);
+		value &= ~PCI_EXP_LNKSTA_CLS;
+		value |= PCI_EXP_LNKSTA_CLS_5_0GB;
+		writel(value, port->base + RP_LINK_CONTROL_STATUS_2);
+
+		/*
+		 * Poll until link comes back from recovery to avoid race
+		 * condition.
+		 */
+		deadline = ktime_add_us(ktime_get(), LINK_RETRAIN_TIMEOUT);
+
+		while (ktime_before(ktime_get(), deadline)) {
+			value = readl(port->base + RP_LINK_CONTROL_STATUS);
+			if ((value & PCI_EXP_LNKSTA_LT) == 0)
+				break;
+
+			usleep_range(2000, 3000);
+		}
+
+		if (value & PCI_EXP_LNKSTA_LT)
+			dev_warn(dev, "PCIe port %u link is in recovery\n",
+				 port->index);
+
+		/* Retrain the link */
+		value = readl(port->base + RP_LINK_CONTROL_STATUS);
+		value |= PCI_EXP_LNKCTL_RL;
+		writel(value, port->base + RP_LINK_CONTROL_STATUS);
+
+		deadline = ktime_add_us(ktime_get(), LINK_RETRAIN_TIMEOUT);
+
+		while (ktime_before(ktime_get(), deadline)) {
+			value = readl(port->base + RP_LINK_CONTROL_STATUS);
+			if ((value & PCI_EXP_LNKSTA_LT) == 0)
+				break;
+
+			usleep_range(2000, 3000);
+		}
+
+		if (value & PCI_EXP_LNKSTA_LT)
+			dev_err(dev, "failed to retrain link of port %u\n",
+				port->index);
+	}
+}
+
 static void tegra_pcie_enable_ports(struct tegra_pcie *pcie)
 {
 	struct device *dev = pcie->dev;
@@ -2113,6 +2174,9 @@ static void tegra_pcie_enable_ports(struct tegra_pcie *pcie)
 		tegra_pcie_port_disable(port);
 		tegra_pcie_port_free(port);
 	}
+
+	if (pcie->soc->has_gen2)
+		tegra_pcie_change_link_speed(pcie);
 }
 
 static void tegra_pcie_disable_ports(struct tegra_pcie *pcie)
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 72+ messages in thread

* [PATCH V4 07/28] PCI: tegra: Advertise PCIe Advanced Error Reporting (AER) capability
  2019-05-16  5:52 [PATCH V4 00/28] Enable Tegra PCIe root port features Manikanta Maddireddy
                   ` (5 preceding siblings ...)
  2019-05-16  5:52 ` [PATCH V4 06/28] PCI: tegra: Add PCIe Gen2 link speed support Manikanta Maddireddy
@ 2019-05-16  5:52 ` Manikanta Maddireddy
  2019-05-16  5:52 ` [PATCH V4 08/28] PCI: tegra: Program UPHY electrical settings for Tegra210 Manikanta Maddireddy
                   ` (22 subsequent siblings)
  29 siblings, 0 replies; 72+ messages in thread
From: Manikanta Maddireddy @ 2019-05-16  5:52 UTC (permalink / raw)
  To: thierry.reding, bhelgaas, robh+dt, mark.rutland, jonathanh,
	lorenzo.pieralisi, vidyas
  Cc: linux-tegra, linux-pci, devicetree, Manikanta Maddireddy

Default root port setting hides AER capability. This patch enables the
advertisement of AER capability by root port.

Signed-off-by: Manikanta Maddireddy <mmaddireddy@nvidia.com>
Acked-by: Thierry Reding <treding@nvidia.com>
---
V4: No change

V3: No change

V2: No change

 drivers/pci/controller/pci-tegra.c | 15 +++++++++++++++
 1 file changed, 15 insertions(+)

diff --git a/drivers/pci/controller/pci-tegra.c b/drivers/pci/controller/pci-tegra.c
index cd542a8af618..e402627bd221 100644
--- a/drivers/pci/controller/pci-tegra.c
+++ b/drivers/pci/controller/pci-tegra.c
@@ -180,6 +180,9 @@
 #define RP_VEND_XP	0x00000f00
 #define  RP_VEND_XP_DL_UP	(1 << 30)
 
+#define RP_VEND_CTL1	0x00000f48
+#define  RP_VEND_CTL1_ERPT	(1 << 13)
+
 #define RP_VEND_CTL2 0x00000fa8
 #define  RP_VEND_CTL2_PCA_ENABLE (1 << 7)
 
@@ -479,6 +482,16 @@ static void tegra_pcie_port_reset(struct tegra_pcie_port *port)
 	afi_writel(port->pcie, value, ctrl);
 }
 
+static void tegra_pcie_enable_rp_features(struct tegra_pcie_port *port)
+{
+	u32 value;
+
+	/* Enable AER capability */
+	value = readl(port->base + RP_VEND_CTL1);
+	value |= RP_VEND_CTL1_ERPT;
+	writel(value, port->base + RP_VEND_CTL1);
+}
+
 static void tegra_pcie_port_enable(struct tegra_pcie_port *port)
 {
 	unsigned long ctrl = tegra_pcie_port_get_pex_ctrl(port);
@@ -503,6 +516,8 @@ static void tegra_pcie_port_enable(struct tegra_pcie_port *port)
 		value |= RP_VEND_CTL2_PCA_ENABLE;
 		writel(value, port->base + RP_VEND_CTL2);
 	}
+
+	tegra_pcie_enable_rp_features(port);
 }
 
 static void tegra_pcie_port_disable(struct tegra_pcie_port *port)
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 72+ messages in thread

* [PATCH V4 08/28] PCI: tegra: Program UPHY electrical settings for Tegra210
  2019-05-16  5:52 [PATCH V4 00/28] Enable Tegra PCIe root port features Manikanta Maddireddy
                   ` (6 preceding siblings ...)
  2019-05-16  5:52 ` [PATCH V4 07/28] PCI: tegra: Advertise PCIe Advanced Error Reporting (AER) capability Manikanta Maddireddy
@ 2019-05-16  5:52 ` Manikanta Maddireddy
  2019-05-16  5:52 ` [PATCH V4 09/28] PCI: tegra: Enable opportunistic UpdateFC and ACK Manikanta Maddireddy
                   ` (21 subsequent siblings)
  29 siblings, 0 replies; 72+ messages in thread
From: Manikanta Maddireddy @ 2019-05-16  5:52 UTC (permalink / raw)
  To: thierry.reding, bhelgaas, robh+dt, mark.rutland, jonathanh,
	lorenzo.pieralisi, vidyas
  Cc: linux-tegra, linux-pci, devicetree, Manikanta Maddireddy

UPHY electrical programming guidelines are documented in Tegra210 TRM.
Program these electrical settings for proper eye diagram in Gen1 and Gen2
link speeds.

Signed-off-by: Manikanta Maddireddy <mmaddireddy@nvidia.com>
Acked-by: Thierry Reding <treding@nvidia.com>
---
V4: No change

V3: No change

V2: Addressed coding style comments

 drivers/pci/controller/pci-tegra.c | 107 +++++++++++++++++++++++++++++
 1 file changed, 107 insertions(+)

diff --git a/drivers/pci/controller/pci-tegra.c b/drivers/pci/controller/pci-tegra.c
index e402627bd221..76d913ef5bf4 100644
--- a/drivers/pci/controller/pci-tegra.c
+++ b/drivers/pci/controller/pci-tegra.c
@@ -177,6 +177,32 @@
 
 #define AFI_PEXBIAS_CTRL_0		0x168
 
+#define RP_ECTL_2_R1	0x00000e84
+#define  RP_ECTL_2_R1_RX_CTLE_1C_MASK		0xffff
+
+#define RP_ECTL_4_R1	0x00000e8c
+#define  RP_ECTL_4_R1_RX_CDR_CTRL_1C_MASK	(0xffff << 16)
+#define  RP_ECTL_4_R1_RX_CDR_CTRL_1C_SHIFT	16
+
+#define RP_ECTL_5_R1	0x00000e90
+#define  RP_ECTL_5_R1_RX_EQ_CTRL_L_1C_MASK	0xffffffff
+
+#define RP_ECTL_6_R1	0x00000e94
+#define  RP_ECTL_6_R1_RX_EQ_CTRL_H_1C_MASK	0xffffffff
+
+#define RP_ECTL_2_R2	0x00000ea4
+#define  RP_ECTL_2_R2_RX_CTLE_1C_MASK	0xffff
+
+#define RP_ECTL_4_R2	0x00000eac
+#define  RP_ECTL_4_R2_RX_CDR_CTRL_1C_MASK	(0xffff << 16)
+#define  RP_ECTL_4_R2_RX_CDR_CTRL_1C_SHIFT	16
+
+#define RP_ECTL_5_R2	0x00000eb0
+#define  RP_ECTL_5_R2_RX_EQ_CTRL_L_1C_MASK	0xffffffff
+
+#define RP_ECTL_6_R2	0x00000eb4
+#define  RP_ECTL_6_R2_RX_EQ_CTRL_H_1C_MASK	0xffffffff
+
 #define RP_VEND_XP	0x00000f00
 #define  RP_VEND_XP_DL_UP	(1 << 30)
 
@@ -266,6 +292,19 @@ struct tegra_pcie_soc {
 	bool has_gen2;
 	bool force_pca_enable;
 	bool program_uphy;
+	struct {
+		struct {
+			u32 rp_ectl_2_r1;
+			u32 rp_ectl_4_r1;
+			u32 rp_ectl_5_r1;
+			u32 rp_ectl_6_r1;
+			u32 rp_ectl_2_r2;
+			u32 rp_ectl_4_r2;
+			u32 rp_ectl_5_r2;
+			u32 rp_ectl_6_r2;
+		} regs;
+		bool enable;
+	} ectl;
 };
 
 static inline struct tegra_msi *to_tegra_msi(struct msi_controller *chip)
@@ -492,6 +531,54 @@ static void tegra_pcie_enable_rp_features(struct tegra_pcie_port *port)
 	writel(value, port->base + RP_VEND_CTL1);
 }
 
+static void tegra_pcie_program_ectl_settings(struct tegra_pcie_port *port)
+{
+	const struct tegra_pcie_soc *soc = port->pcie->soc;
+	u32 value;
+
+	value = readl(port->base + RP_ECTL_2_R1);
+	value &= ~RP_ECTL_2_R1_RX_CTLE_1C_MASK;
+	value |= soc->ectl.regs.rp_ectl_2_r1;
+	writel(value, port->base + RP_ECTL_2_R1);
+
+	value = readl(port->base + RP_ECTL_4_R1);
+	value &= ~RP_ECTL_4_R1_RX_CDR_CTRL_1C_MASK;
+	value |= soc->ectl.regs.rp_ectl_4_r1 <<
+				RP_ECTL_4_R1_RX_CDR_CTRL_1C_SHIFT;
+	writel(value, port->base + RP_ECTL_4_R1);
+
+	value = readl(port->base + RP_ECTL_5_R1);
+	value &= ~RP_ECTL_5_R1_RX_EQ_CTRL_L_1C_MASK;
+	value |= soc->ectl.regs.rp_ectl_5_r1;
+	writel(value, port->base + RP_ECTL_5_R1);
+
+	value = readl(port->base + RP_ECTL_6_R1);
+	value &= ~RP_ECTL_6_R1_RX_EQ_CTRL_H_1C_MASK;
+	value |= soc->ectl.regs.rp_ectl_6_r1;
+	writel(value, port->base + RP_ECTL_6_R1);
+
+	value = readl(port->base + RP_ECTL_2_R2);
+	value &= ~RP_ECTL_2_R2_RX_CTLE_1C_MASK;
+	value |= soc->ectl.regs.rp_ectl_2_r2;
+	writel(value, port->base + RP_ECTL_2_R2);
+
+	value = readl(port->base + RP_ECTL_4_R2);
+	value &= ~RP_ECTL_4_R2_RX_CDR_CTRL_1C_MASK;
+	value |= soc->ectl.regs.rp_ectl_4_r2 <<
+				RP_ECTL_4_R2_RX_CDR_CTRL_1C_SHIFT;
+	writel(value, port->base + RP_ECTL_4_R2);
+
+	value = readl(port->base + RP_ECTL_5_R2);
+	value &= ~RP_ECTL_5_R2_RX_EQ_CTRL_L_1C_MASK;
+	value |= soc->ectl.regs.rp_ectl_5_r2;
+	writel(value, port->base + RP_ECTL_5_R2);
+
+	value = readl(port->base + RP_ECTL_6_R2);
+	value &= ~RP_ECTL_6_R2_RX_EQ_CTRL_H_1C_MASK;
+	value |= soc->ectl.regs.rp_ectl_6_r2;
+	writel(value, port->base + RP_ECTL_6_R2);
+}
+
 static void tegra_pcie_port_enable(struct tegra_pcie_port *port)
 {
 	unsigned long ctrl = tegra_pcie_port_get_pex_ctrl(port);
@@ -518,6 +605,9 @@ static void tegra_pcie_port_enable(struct tegra_pcie_port *port)
 	}
 
 	tegra_pcie_enable_rp_features(port);
+
+	if (soc->ectl.enable)
+		tegra_pcie_program_ectl_settings(port);
 }
 
 static void tegra_pcie_port_disable(struct tegra_pcie_port *port)
@@ -2223,6 +2313,7 @@ static const struct tegra_pcie_soc tegra20_pcie = {
 	.has_gen2 = false,
 	.force_pca_enable = false,
 	.program_uphy = true,
+	.ectl.enable = false,
 };
 
 static const struct tegra_pcie_port_soc tegra30_pcie_ports[] = {
@@ -2246,6 +2337,7 @@ static const struct tegra_pcie_soc tegra30_pcie = {
 	.has_gen2 = false,
 	.force_pca_enable = false,
 	.program_uphy = true,
+	.ectl.enable = false,
 };
 
 static const struct tegra_pcie_soc tegra124_pcie = {
@@ -2262,6 +2354,7 @@ static const struct tegra_pcie_soc tegra124_pcie = {
 	.has_gen2 = true,
 	.force_pca_enable = false,
 	.program_uphy = true,
+	.ectl.enable = false,
 };
 
 static const struct tegra_pcie_soc tegra210_pcie = {
@@ -2278,6 +2371,19 @@ static const struct tegra_pcie_soc tegra210_pcie = {
 	.has_gen2 = true,
 	.force_pca_enable = true,
 	.program_uphy = true,
+	.ectl = {
+		.regs = {
+			.rp_ectl_2_r1 = 0x0000000f,
+			.rp_ectl_4_r1 = 0x00000067,
+			.rp_ectl_5_r1 = 0x55010000,
+			.rp_ectl_6_r1 = 0x00000001,
+			.rp_ectl_2_r2 = 0x0000008f,
+			.rp_ectl_4_r2 = 0x000000c7,
+			.rp_ectl_5_r2 = 0x55010000,
+			.rp_ectl_6_r2 = 0x00000001,
+		},
+		.enable = true,
+	},
 };
 
 static const struct tegra_pcie_port_soc tegra186_pcie_ports[] = {
@@ -2301,6 +2407,7 @@ static const struct tegra_pcie_soc tegra186_pcie = {
 	.has_gen2 = true,
 	.force_pca_enable = false,
 	.program_uphy = false,
+	.ectl.enable = false,
 };
 
 static const struct of_device_id tegra_pcie_of_match[] = {
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 72+ messages in thread

* [PATCH V4 09/28] PCI: tegra: Enable opportunistic UpdateFC and ACK
  2019-05-16  5:52 [PATCH V4 00/28] Enable Tegra PCIe root port features Manikanta Maddireddy
                   ` (7 preceding siblings ...)
  2019-05-16  5:52 ` [PATCH V4 08/28] PCI: tegra: Program UPHY electrical settings for Tegra210 Manikanta Maddireddy
@ 2019-05-16  5:52 ` Manikanta Maddireddy
  2019-05-16  5:52 ` [PATCH V4 10/28] PCI: tegra: Disable AFI dynamic clock gating Manikanta Maddireddy
                   ` (20 subsequent siblings)
  29 siblings, 0 replies; 72+ messages in thread
From: Manikanta Maddireddy @ 2019-05-16  5:52 UTC (permalink / raw)
  To: thierry.reding, bhelgaas, robh+dt, mark.rutland, jonathanh,
	lorenzo.pieralisi, vidyas
  Cc: linux-tegra, linux-pci, devicetree, Manikanta Maddireddy

Enable opportunistic UpdateFC and ACK to allow data link layer send
pending ACKs and UpdateFC packets when link is idle instead of waiting
for timers to expire. This improves the PCIe performance due to better
utilization of PCIe bandwidth.

Signed-off-by: Manikanta Maddireddy <mmaddireddy@nvidia.com>
Acked-by: Thierry Reding <treding@nvidia.com>
---
V4: No change

V3: No change

V2: No change

 drivers/pci/controller/pci-tegra.c | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/drivers/pci/controller/pci-tegra.c b/drivers/pci/controller/pci-tegra.c
index 76d913ef5bf4..b1e2e398b4e8 100644
--- a/drivers/pci/controller/pci-tegra.c
+++ b/drivers/pci/controller/pci-tegra.c
@@ -204,7 +204,9 @@
 #define  RP_ECTL_6_R2_RX_EQ_CTRL_H_1C_MASK	0xffffffff
 
 #define RP_VEND_XP	0x00000f00
-#define  RP_VEND_XP_DL_UP	(1 << 30)
+#define  RP_VEND_XP_DL_UP			(1 << 30)
+#define  RP_VEND_XP_OPPORTUNISTIC_ACK		(1 << 27)
+#define  RP_VEND_XP_OPPORTUNISTIC_UPDATEFC	(1 << 28)
 
 #define RP_VEND_CTL1	0x00000f48
 #define  RP_VEND_CTL1_ERPT	(1 << 13)
@@ -529,6 +531,12 @@ static void tegra_pcie_enable_rp_features(struct tegra_pcie_port *port)
 	value = readl(port->base + RP_VEND_CTL1);
 	value |= RP_VEND_CTL1_ERPT;
 	writel(value, port->base + RP_VEND_CTL1);
+
+	/* Optimal settings to enhance bandwidth */
+	value = readl(port->base + RP_VEND_XP);
+	value |= RP_VEND_XP_OPPORTUNISTIC_ACK;
+	value |= RP_VEND_XP_OPPORTUNISTIC_UPDATEFC;
+	writel(value, port->base + RP_VEND_XP);
 }
 
 static void tegra_pcie_program_ectl_settings(struct tegra_pcie_port *port)
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 72+ messages in thread

* [PATCH V4 10/28] PCI: tegra: Disable AFI dynamic clock gating
  2019-05-16  5:52 [PATCH V4 00/28] Enable Tegra PCIe root port features Manikanta Maddireddy
                   ` (8 preceding siblings ...)
  2019-05-16  5:52 ` [PATCH V4 09/28] PCI: tegra: Enable opportunistic UpdateFC and ACK Manikanta Maddireddy
@ 2019-05-16  5:52 ` Manikanta Maddireddy
  2019-05-16  5:52 ` [PATCH V4 11/28] PCI: tegra: Process pending DLL transactions before entering L1 or L2 Manikanta Maddireddy
                   ` (19 subsequent siblings)
  29 siblings, 0 replies; 72+ messages in thread
From: Manikanta Maddireddy @ 2019-05-16  5:52 UTC (permalink / raw)
  To: thierry.reding, bhelgaas, robh+dt, mark.rutland, jonathanh,
	lorenzo.pieralisi, vidyas
  Cc: linux-tegra, linux-pci, devicetree, Manikanta Maddireddy

Outstanding write counter in AFI is used to generate idle signal to
dynamically gate the AFI clock. When there are 32 outstanding writes
from AFI to memory, the outstanding write counter overflows and
indicates that there are "0" outstanding write transactions.

When memory controller is under heavy load, write completions to AFI
gets delayed and AFI write counter overflows. This causes AFI clock gating
even when there are outstanding transactions towards memory controller
resulting in system hang.

Disable dynamic clock gating of AFI clock to avoid system hang.

CLKEN_OVERRIDE bit is not defined in Tegra20 and Tegra30, however
programming this bit doesn't cause any side effects. Program this
bit for all Tegra SoCs to avoid conditional check.

Signed-off-by: Manikanta Maddireddy <mmaddireddy@nvidia.com>
Acked-by: Thierry Reding <treding@nvidia.com>
---
V4: No change

V3: No change

V2: No change

 drivers/pci/controller/pci-tegra.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/drivers/pci/controller/pci-tegra.c b/drivers/pci/controller/pci-tegra.c
index b1e2e398b4e8..17c6d858ddc6 100644
--- a/drivers/pci/controller/pci-tegra.c
+++ b/drivers/pci/controller/pci-tegra.c
@@ -95,7 +95,8 @@
 #define AFI_MSI_EN_VEC7		0xa8
 
 #define AFI_CONFIGURATION		0xac
-#define  AFI_CONFIGURATION_EN_FPCI	(1 << 0)
+#define  AFI_CONFIGURATION_EN_FPCI		(1 << 0)
+#define  AFI_CONFIGURATION_CLKEN_OVERRIDE	(1 << 31)
 
 #define AFI_FPCI_ERROR_MASKS	0xb0
 
@@ -1065,9 +1066,10 @@ static void tegra_pcie_enable_controller(struct tegra_pcie *pcie)
 		afi_writel(pcie, value, AFI_FUSE);
 	}
 
-	/* finally enable PCIe */
+	/* Disable AFI dynamic clock gating and enable PCIe */
 	value = afi_readl(pcie, AFI_CONFIGURATION);
 	value |= AFI_CONFIGURATION_EN_FPCI;
+	value |= AFI_CONFIGURATION_CLKEN_OVERRIDE;
 	afi_writel(pcie, value, AFI_CONFIGURATION);
 
 	value = AFI_INTR_EN_INI_SLVERR | AFI_INTR_EN_INI_DECERR |
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 72+ messages in thread

* [PATCH V4 11/28] PCI: tegra: Process pending DLL transactions before entering L1 or L2
  2019-05-16  5:52 [PATCH V4 00/28] Enable Tegra PCIe root port features Manikanta Maddireddy
                   ` (9 preceding siblings ...)
  2019-05-16  5:52 ` [PATCH V4 10/28] PCI: tegra: Disable AFI dynamic clock gating Manikanta Maddireddy
@ 2019-05-16  5:52 ` Manikanta Maddireddy
  2019-05-16  5:52 ` [PATCH V4 12/28] PCI: tegra: Enable PCIe xclk clock clamping Manikanta Maddireddy
                   ` (18 subsequent siblings)
  29 siblings, 0 replies; 72+ messages in thread
From: Manikanta Maddireddy @ 2019-05-16  5:52 UTC (permalink / raw)
  To: thierry.reding, bhelgaas, robh+dt, mark.rutland, jonathanh,
	lorenzo.pieralisi, vidyas
  Cc: linux-tegra, linux-pci, devicetree, Manikanta Maddireddy

PM message are truncated while entering L1 or L2, which is resulting in
receiver errors. Set the required bit to finish processing DLLP before
link enter L1 or L2.

Signed-off-by: Manikanta Maddireddy <mmaddireddy@nvidia.com>
Acked-by: Thierry Reding <treding@nvidia.com>
---
V4: No change

V3: No change

V2: No change

 drivers/pci/controller/pci-tegra.c | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/drivers/pci/controller/pci-tegra.c b/drivers/pci/controller/pci-tegra.c
index 17c6d858ddc6..d3da03a10e04 100644
--- a/drivers/pci/controller/pci-tegra.c
+++ b/drivers/pci/controller/pci-tegra.c
@@ -212,6 +212,9 @@
 #define RP_VEND_CTL1	0x00000f48
 #define  RP_VEND_CTL1_ERPT	(1 << 13)
 
+#define RP_VEND_XP_BIST	0x00000f4c
+#define  RP_VEND_XP_BIST_GOTO_L1_L2_AFTER_DLLP_DONE	(1 << 28)
+
 #define RP_VEND_CTL2 0x00000fa8
 #define  RP_VEND_CTL2_PCA_ENABLE (1 << 7)
 
@@ -538,6 +541,14 @@ static void tegra_pcie_enable_rp_features(struct tegra_pcie_port *port)
 	value |= RP_VEND_XP_OPPORTUNISTIC_ACK;
 	value |= RP_VEND_XP_OPPORTUNISTIC_UPDATEFC;
 	writel(value, port->base + RP_VEND_XP);
+
+	/*
+	 * LTSSM will wait for DLLP to finish before entering L1 or L2,
+	 * to avoid truncation of PM messages which results in receiver errors
+	 */
+	value = readl(port->base + RP_VEND_XP_BIST);
+	value |= RP_VEND_XP_BIST_GOTO_L1_L2_AFTER_DLLP_DONE;
+	writel(value, port->base + RP_VEND_XP_BIST);
 }
 
 static void tegra_pcie_program_ectl_settings(struct tegra_pcie_port *port)
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 72+ messages in thread

* [PATCH V4 12/28] PCI: tegra: Enable PCIe xclk clock clamping
  2019-05-16  5:52 [PATCH V4 00/28] Enable Tegra PCIe root port features Manikanta Maddireddy
                   ` (10 preceding siblings ...)
  2019-05-16  5:52 ` [PATCH V4 11/28] PCI: tegra: Process pending DLL transactions before entering L1 or L2 Manikanta Maddireddy
@ 2019-05-16  5:52 ` Manikanta Maddireddy
  2019-05-16  5:52 ` [PATCH V4 13/28] PCI: tegra: Increase the deskew retry time Manikanta Maddireddy
                   ` (17 subsequent siblings)
  29 siblings, 0 replies; 72+ messages in thread
From: Manikanta Maddireddy @ 2019-05-16  5:52 UTC (permalink / raw)
  To: thierry.reding, bhelgaas, robh+dt, mark.rutland, jonathanh,
	lorenzo.pieralisi, vidyas
  Cc: linux-tegra, linux-pci, devicetree, Manikanta Maddireddy

Enable xclk clock clamping when entering L1. Clamp threshold will
determine the time spent waiting for clock module to turn on xclk after
signaling it. Default threshold value in Tegra124 and Tegra210 is not
enough to turn on xclk clock. Increase the clamp threshold to meet the
clock module timing in Tegra124 and Tegra210. Default threshold value is
enough in Tegra20, Tegra30 and Tegra186.

Signed-off-by: Manikanta Maddireddy <mmaddireddy@nvidia.com>
Acked-by: Thierry Reding <treding@nvidia.com>
---
V4: No change

V3: No change

V2: Took care of typos in commit log and coding style comments.

 drivers/pci/controller/pci-tegra.c | 30 ++++++++++++++++++++++++++++--
 1 file changed, 28 insertions(+), 2 deletions(-)

diff --git a/drivers/pci/controller/pci-tegra.c b/drivers/pci/controller/pci-tegra.c
index d3da03a10e04..96cd75821872 100644
--- a/drivers/pci/controller/pci-tegra.c
+++ b/drivers/pci/controller/pci-tegra.c
@@ -219,8 +219,14 @@
 #define  RP_VEND_CTL2_PCA_ENABLE (1 << 7)
 
 #define RP_PRIV_MISC	0x00000fe0
-#define  RP_PRIV_MISC_PRSNT_MAP_EP_PRSNT (0xe << 0)
-#define  RP_PRIV_MISC_PRSNT_MAP_EP_ABSNT (0xf << 0)
+#define  RP_PRIV_MISC_PRSNT_MAP_EP_PRSNT		(0xe << 0)
+#define  RP_PRIV_MISC_PRSNT_MAP_EP_ABSNT		(0xf << 0)
+#define  RP_PRIV_MISC_CTLR_CLK_CLAMP_THRESHOLD_MASK	(0x7f << 16)
+#define  RP_PRIV_MISC_CTLR_CLK_CLAMP_THRESHOLD		(0xf << 16)
+#define  RP_PRIV_MISC_CTLR_CLK_CLAMP_ENABLE		(1 << 23)
+#define  RP_PRIV_MISC_TMS_CLK_CLAMP_THRESHOLD_MASK	(0x7f << 24)
+#define  RP_PRIV_MISC_TMS_CLK_CLAMP_THRESHOLD		(0xf << 24)
+#define  RP_PRIV_MISC_TMS_CLK_CLAMP_ENABLE		(1 << 31)
 
 #define RP_LINK_CONTROL_STATUS			0x00000090
 #define  RP_LINK_CONTROL_STATUS_DL_LINK_ACTIVE	0x20000000
@@ -298,6 +304,7 @@ struct tegra_pcie_soc {
 	bool has_gen2;
 	bool force_pca_enable;
 	bool program_uphy;
+	bool update_clamp_threshold;
 	struct {
 		struct {
 			u32 rp_ectl_2_r1;
@@ -529,6 +536,7 @@ static void tegra_pcie_port_reset(struct tegra_pcie_port *port)
 
 static void tegra_pcie_enable_rp_features(struct tegra_pcie_port *port)
 {
+	const struct tegra_pcie_soc *soc = port->pcie->soc;
 	u32 value;
 
 	/* Enable AER capability */
@@ -549,6 +557,19 @@ static void tegra_pcie_enable_rp_features(struct tegra_pcie_port *port)
 	value = readl(port->base + RP_VEND_XP_BIST);
 	value |= RP_VEND_XP_BIST_GOTO_L1_L2_AFTER_DLLP_DONE;
 	writel(value, port->base + RP_VEND_XP_BIST);
+
+	value = readl(port->base + RP_PRIV_MISC);
+	value |= RP_PRIV_MISC_CTLR_CLK_CLAMP_ENABLE;
+	value |= RP_PRIV_MISC_TMS_CLK_CLAMP_ENABLE;
+
+	if (soc->update_clamp_threshold) {
+		value &= ~(RP_PRIV_MISC_CTLR_CLK_CLAMP_THRESHOLD_MASK |
+				RP_PRIV_MISC_TMS_CLK_CLAMP_THRESHOLD_MASK);
+		value |= RP_PRIV_MISC_CTLR_CLK_CLAMP_THRESHOLD |
+			RP_PRIV_MISC_TMS_CLK_CLAMP_THRESHOLD;
+	}
+
+	writel(value, port->base + RP_PRIV_MISC);
 }
 
 static void tegra_pcie_program_ectl_settings(struct tegra_pcie_port *port)
@@ -2334,6 +2355,7 @@ static const struct tegra_pcie_soc tegra20_pcie = {
 	.has_gen2 = false,
 	.force_pca_enable = false,
 	.program_uphy = true,
+	.update_clamp_threshold = false,
 	.ectl.enable = false,
 };
 
@@ -2358,6 +2380,7 @@ static const struct tegra_pcie_soc tegra30_pcie = {
 	.has_gen2 = false,
 	.force_pca_enable = false,
 	.program_uphy = true,
+	.update_clamp_threshold = false,
 	.ectl.enable = false,
 };
 
@@ -2375,6 +2398,7 @@ static const struct tegra_pcie_soc tegra124_pcie = {
 	.has_gen2 = true,
 	.force_pca_enable = false,
 	.program_uphy = true,
+	.update_clamp_threshold = true,
 	.ectl.enable = false,
 };
 
@@ -2392,6 +2416,7 @@ static const struct tegra_pcie_soc tegra210_pcie = {
 	.has_gen2 = true,
 	.force_pca_enable = true,
 	.program_uphy = true,
+	.update_clamp_threshold = true,
 	.ectl = {
 		.regs = {
 			.rp_ectl_2_r1 = 0x0000000f,
@@ -2428,6 +2453,7 @@ static const struct tegra_pcie_soc tegra186_pcie = {
 	.has_gen2 = true,
 	.force_pca_enable = false,
 	.program_uphy = false,
+	.update_clamp_threshold = false,
 	.ectl.enable = false,
 };
 
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 72+ messages in thread

* [PATCH V4 13/28] PCI: tegra: Increase the deskew retry time
  2019-05-16  5:52 [PATCH V4 00/28] Enable Tegra PCIe root port features Manikanta Maddireddy
                   ` (11 preceding siblings ...)
  2019-05-16  5:52 ` [PATCH V4 12/28] PCI: tegra: Enable PCIe xclk clock clamping Manikanta Maddireddy
@ 2019-05-16  5:52 ` Manikanta Maddireddy
  2019-05-16  5:52 ` [PATCH V4 14/28] PCI: tegra: Add SW fixup for RAW violations Manikanta Maddireddy
                   ` (16 subsequent siblings)
  29 siblings, 0 replies; 72+ messages in thread
From: Manikanta Maddireddy @ 2019-05-16  5:52 UTC (permalink / raw)
  To: thierry.reding, bhelgaas, robh+dt, mark.rutland, jonathanh,
	lorenzo.pieralisi, vidyas
  Cc: linux-tegra, linux-pci, devicetree, Manikanta Maddireddy

Sometimes link speed change from Gen2 to Gen1 fails due to instability
in deskew logic on lane-0 in Tegra210. Increase the deskew retry time
to resolve this issue.

Signed-off-by: Manikanta Maddireddy <mmaddireddy@nvidia.com>
Acked-by: Thierry Reding <treding@nvidia.com>
---
V4: No change

V3: No change

V2: Took care of typos in commit log and coding style comments.

 drivers/pci/controller/pci-tegra.c | 30 ++++++++++++++++++++++++++++++
 1 file changed, 30 insertions(+)

diff --git a/drivers/pci/controller/pci-tegra.c b/drivers/pci/controller/pci-tegra.c
index 96cd75821872..9c28f1d9f177 100644
--- a/drivers/pci/controller/pci-tegra.c
+++ b/drivers/pci/controller/pci-tegra.c
@@ -209,6 +209,10 @@
 #define  RP_VEND_XP_OPPORTUNISTIC_ACK		(1 << 27)
 #define  RP_VEND_XP_OPPORTUNISTIC_UPDATEFC	(1 << 28)
 
+#define RP_VEND_CTL0	0x00000f44
+#define  RP_VEND_CTL0_DSK_RST_PULSE_WIDTH_MASK	(0xf << 12)
+#define  RP_VEND_CTL0_DSK_RST_PULSE_WIDTH	(0x9 << 12)
+
 #define RP_VEND_CTL1	0x00000f48
 #define  RP_VEND_CTL1_ERPT	(1 << 13)
 
@@ -305,6 +309,7 @@ struct tegra_pcie_soc {
 	bool force_pca_enable;
 	bool program_uphy;
 	bool update_clamp_threshold;
+	bool program_deskew_time;
 	struct {
 		struct {
 			u32 rp_ectl_2_r1;
@@ -620,6 +625,24 @@ static void tegra_pcie_program_ectl_settings(struct tegra_pcie_port *port)
 	writel(value, port->base + RP_ECTL_6_R2);
 }
 
+static void tegra_pcie_apply_sw_fixup(struct tegra_pcie_port *port)
+{
+	const struct tegra_pcie_soc *soc = port->pcie->soc;
+	u32 value;
+
+	/*
+	 * Sometimes link speed change from Gen2 to Gen1 fails due to
+	 * instability in deskew logic on lane-0. Increase the deskew
+	 * retry time to resolve this issue.
+	 */
+	if (soc->program_deskew_time) {
+		value = readl(port->base + RP_VEND_CTL0);
+		value &= ~RP_VEND_CTL0_DSK_RST_PULSE_WIDTH_MASK;
+		value |= RP_VEND_CTL0_DSK_RST_PULSE_WIDTH;
+		writel(value, port->base + RP_VEND_CTL0);
+	}
+}
+
 static void tegra_pcie_port_enable(struct tegra_pcie_port *port)
 {
 	unsigned long ctrl = tegra_pcie_port_get_pex_ctrl(port);
@@ -649,6 +672,8 @@ static void tegra_pcie_port_enable(struct tegra_pcie_port *port)
 
 	if (soc->ectl.enable)
 		tegra_pcie_program_ectl_settings(port);
+
+	tegra_pcie_apply_sw_fixup(port);
 }
 
 static void tegra_pcie_port_disable(struct tegra_pcie_port *port)
@@ -2356,6 +2381,7 @@ static const struct tegra_pcie_soc tegra20_pcie = {
 	.force_pca_enable = false,
 	.program_uphy = true,
 	.update_clamp_threshold = false,
+	.program_deskew_time = false,
 	.ectl.enable = false,
 };
 
@@ -2381,6 +2407,7 @@ static const struct tegra_pcie_soc tegra30_pcie = {
 	.force_pca_enable = false,
 	.program_uphy = true,
 	.update_clamp_threshold = false,
+	.program_deskew_time = false,
 	.ectl.enable = false,
 };
 
@@ -2399,6 +2426,7 @@ static const struct tegra_pcie_soc tegra124_pcie = {
 	.force_pca_enable = false,
 	.program_uphy = true,
 	.update_clamp_threshold = true,
+	.program_deskew_time = false,
 	.ectl.enable = false,
 };
 
@@ -2417,6 +2445,7 @@ static const struct tegra_pcie_soc tegra210_pcie = {
 	.force_pca_enable = true,
 	.program_uphy = true,
 	.update_clamp_threshold = true,
+	.program_deskew_time = true,
 	.ectl = {
 		.regs = {
 			.rp_ectl_2_r1 = 0x0000000f,
@@ -2454,6 +2483,7 @@ static const struct tegra_pcie_soc tegra186_pcie = {
 	.force_pca_enable = false,
 	.program_uphy = false,
 	.update_clamp_threshold = false,
+	.program_deskew_time = false,
 	.ectl.enable = false,
 };
 
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 72+ messages in thread

* [PATCH V4 14/28] PCI: tegra: Add SW fixup for RAW violations
  2019-05-16  5:52 [PATCH V4 00/28] Enable Tegra PCIe root port features Manikanta Maddireddy
                   ` (12 preceding siblings ...)
  2019-05-16  5:52 ` [PATCH V4 13/28] PCI: tegra: Increase the deskew retry time Manikanta Maddireddy
@ 2019-05-16  5:52 ` Manikanta Maddireddy
  2019-05-16  5:52 ` [PATCH V4 15/28] PCI: tegra: Update flow control timer frequency in Tegra210 Manikanta Maddireddy
                   ` (15 subsequent siblings)
  29 siblings, 0 replies; 72+ messages in thread
From: Manikanta Maddireddy @ 2019-05-16  5:52 UTC (permalink / raw)
  To: thierry.reding, bhelgaas, robh+dt, mark.rutland, jonathanh,
	lorenzo.pieralisi, vidyas
  Cc: linux-tegra, linux-pci, devicetree, Manikanta Maddireddy

The logic which blocks read requests till AFI gets ACK for all outstanding
writes from memory controller does not behave correctly when number of
outstanding writes become more than 32 in Tegra124 and Tegra132.

SW fixup is to prevent writes from accumulating more than 32 by,
 - limiting outstanding posted writes to 14
 - modifying Gen1 and Gen2 UpdateFC timer frequency

UpdateFC timer frequency is equal to twice the value of register content
in nsec. These settings are recommended after stress testing with different
values.

Signed-off-by: Manikanta Maddireddy <mmaddireddy@nvidia.com>
Acked-by: Thierry Reding <treding@nvidia.com>
---
V4: No change

V3: No change

V2: Changed update_fc_val to update_fc_threshold

 drivers/pci/controller/pci-tegra.c | 34 ++++++++++++++++++++++++++++++
 1 file changed, 34 insertions(+)

diff --git a/drivers/pci/controller/pci-tegra.c b/drivers/pci/controller/pci-tegra.c
index 9c28f1d9f177..be39f28a7a28 100644
--- a/drivers/pci/controller/pci-tegra.c
+++ b/drivers/pci/controller/pci-tegra.c
@@ -178,6 +178,13 @@
 
 #define AFI_PEXBIAS_CTRL_0		0x168
 
+#define RP_PRIV_XP_DL		0x00000494
+#define  RP_PRIV_XP_DL_GEN2_UPD_FC_TSHOLD	(0x1ff << 1)
+
+#define RP_RX_HDR_LIMIT		0x00000e00
+#define  RP_RX_HDR_LIMIT_PW_MASK	(0xff << 8)
+#define  RP_RX_HDR_LIMIT_PW		(0x0e << 8)
+
 #define RP_ECTL_2_R1	0x00000e84
 #define  RP_ECTL_2_R1_RX_CTLE_1C_MASK		0xffff
 
@@ -208,6 +215,7 @@
 #define  RP_VEND_XP_DL_UP			(1 << 30)
 #define  RP_VEND_XP_OPPORTUNISTIC_ACK		(1 << 27)
 #define  RP_VEND_XP_OPPORTUNISTIC_UPDATEFC	(1 << 28)
+#define  RP_VEND_XP_UPDATE_FC_THRESHOLD_MASK	(0xff << 18)
 
 #define RP_VEND_CTL0	0x00000f44
 #define  RP_VEND_CTL0_DSK_RST_PULSE_WIDTH_MASK	(0xf << 12)
@@ -301,6 +309,7 @@ struct tegra_pcie_soc {
 	u32 tx_ref_sel;
 	u32 pads_refclk_cfg0;
 	u32 pads_refclk_cfg1;
+	u32 update_fc_threshold;
 	bool has_pex_clkreq_en;
 	bool has_pex_bias_ctrl;
 	bool has_intr_prsnt_sense;
@@ -310,6 +319,7 @@ struct tegra_pcie_soc {
 	bool program_uphy;
 	bool update_clamp_threshold;
 	bool program_deskew_time;
+	bool raw_violation_fixup;
 	struct {
 		struct {
 			u32 rp_ectl_2_r1;
@@ -641,6 +651,23 @@ static void tegra_pcie_apply_sw_fixup(struct tegra_pcie_port *port)
 		value |= RP_VEND_CTL0_DSK_RST_PULSE_WIDTH;
 		writel(value, port->base + RP_VEND_CTL0);
 	}
+
+	/* Fixup for read after write violation. */
+	if (soc->raw_violation_fixup) {
+		value = readl(port->base + RP_RX_HDR_LIMIT);
+		value &= ~RP_RX_HDR_LIMIT_PW_MASK;
+		value |= RP_RX_HDR_LIMIT_PW;
+		writel(value, port->base + RP_RX_HDR_LIMIT);
+
+		value = readl(port->base + RP_PRIV_XP_DL);
+		value |= RP_PRIV_XP_DL_GEN2_UPD_FC_TSHOLD;
+		writel(value, port->base + RP_PRIV_XP_DL);
+
+		value = readl(port->base + RP_VEND_XP);
+		value &= ~RP_VEND_XP_UPDATE_FC_THRESHOLD_MASK;
+		value |= soc->update_fc_threshold;
+		writel(value, port->base + RP_VEND_XP);
+	}
 }
 
 static void tegra_pcie_port_enable(struct tegra_pcie_port *port)
@@ -2382,6 +2409,7 @@ static const struct tegra_pcie_soc tegra20_pcie = {
 	.program_uphy = true,
 	.update_clamp_threshold = false,
 	.program_deskew_time = false,
+	.raw_violation_fixup = false,
 	.ectl.enable = false,
 };
 
@@ -2408,6 +2436,7 @@ static const struct tegra_pcie_soc tegra30_pcie = {
 	.program_uphy = true,
 	.update_clamp_threshold = false,
 	.program_deskew_time = false,
+	.raw_violation_fixup = false,
 	.ectl.enable = false,
 };
 
@@ -2418,6 +2447,8 @@ static const struct tegra_pcie_soc tegra124_pcie = {
 	.pads_pll_ctl = PADS_PLL_CTL_TEGRA30,
 	.tx_ref_sel = PADS_PLL_CTL_TXCLKREF_BUF_EN,
 	.pads_refclk_cfg0 = 0x44ac44ac,
+	/* FC threshold is bit[25:18] */
+	.update_fc_threshold = 0x03fc0000,
 	.has_pex_clkreq_en = true,
 	.has_pex_bias_ctrl = true,
 	.has_intr_prsnt_sense = true,
@@ -2427,6 +2458,7 @@ static const struct tegra_pcie_soc tegra124_pcie = {
 	.program_uphy = true,
 	.update_clamp_threshold = true,
 	.program_deskew_time = false,
+	.raw_violation_fixup = true,
 	.ectl.enable = false,
 };
 
@@ -2446,6 +2478,7 @@ static const struct tegra_pcie_soc tegra210_pcie = {
 	.program_uphy = true,
 	.update_clamp_threshold = true,
 	.program_deskew_time = true,
+	.raw_violation_fixup = false,
 	.ectl = {
 		.regs = {
 			.rp_ectl_2_r1 = 0x0000000f,
@@ -2484,6 +2517,7 @@ static const struct tegra_pcie_soc tegra186_pcie = {
 	.program_uphy = false,
 	.update_clamp_threshold = false,
 	.program_deskew_time = false,
+	.raw_violation_fixup = false,
 	.ectl.enable = false,
 };
 
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 72+ messages in thread

* [PATCH V4 15/28] PCI: tegra: Update flow control timer frequency in Tegra210
  2019-05-16  5:52 [PATCH V4 00/28] Enable Tegra PCIe root port features Manikanta Maddireddy
                   ` (13 preceding siblings ...)
  2019-05-16  5:52 ` [PATCH V4 14/28] PCI: tegra: Add SW fixup for RAW violations Manikanta Maddireddy
@ 2019-05-16  5:52 ` Manikanta Maddireddy
  2019-05-16  5:52 ` [PATCH V4 16/28] PCI: tegra: Set target speed as Gen1 before starting LTSSM Manikanta Maddireddy
                   ` (14 subsequent siblings)
  29 siblings, 0 replies; 72+ messages in thread
From: Manikanta Maddireddy @ 2019-05-16  5:52 UTC (permalink / raw)
  To: thierry.reding, bhelgaas, robh+dt, mark.rutland, jonathanh,
	lorenzo.pieralisi, vidyas
  Cc: linux-tegra, linux-pci, devicetree, Manikanta Maddireddy

Recommended UpdateFC threshold in Tegra210 is 0x60 for best performance
of x1 link. Setting this to 0x60 provides the best balance between number
of UpdateFC packets and read data sent over the link.

UpdateFC timer frequency is equal to twice the value of register content
in nsec, i.e (2 * 0x60) = 192 nsec.

Signed-off-by: Manikanta Maddireddy <mmaddireddy@nvidia.com>
Acked-by: Thierry Reding <treding@nvidia.com>
---
V4: No change

V3: No change

V2: Changed update_fc_val to update_fc_threshold

 drivers/pci/controller/pci-tegra.c | 15 +++++++++++++++
 1 file changed, 15 insertions(+)

diff --git a/drivers/pci/controller/pci-tegra.c b/drivers/pci/controller/pci-tegra.c
index be39f28a7a28..16d4d3cba3ff 100644
--- a/drivers/pci/controller/pci-tegra.c
+++ b/drivers/pci/controller/pci-tegra.c
@@ -320,6 +320,7 @@ struct tegra_pcie_soc {
 	bool update_clamp_threshold;
 	bool program_deskew_time;
 	bool raw_violation_fixup;
+	bool update_fc_timer;
 	struct {
 		struct {
 			u32 rp_ectl_2_r1;
@@ -668,6 +669,13 @@ static void tegra_pcie_apply_sw_fixup(struct tegra_pcie_port *port)
 		value |= soc->update_fc_threshold;
 		writel(value, port->base + RP_VEND_XP);
 	}
+
+	if (soc->update_fc_timer) {
+		value = readl(port->base + RP_VEND_XP);
+		value &= ~RP_VEND_XP_UPDATE_FC_THRESHOLD_MASK;
+		value |= soc->update_fc_threshold;
+		writel(value, port->base + RP_VEND_XP);
+	}
 }
 
 static void tegra_pcie_port_enable(struct tegra_pcie_port *port)
@@ -2410,6 +2418,7 @@ static const struct tegra_pcie_soc tegra20_pcie = {
 	.update_clamp_threshold = false,
 	.program_deskew_time = false,
 	.raw_violation_fixup = false,
+	.update_fc_timer = false,
 	.ectl.enable = false,
 };
 
@@ -2437,6 +2446,7 @@ static const struct tegra_pcie_soc tegra30_pcie = {
 	.update_clamp_threshold = false,
 	.program_deskew_time = false,
 	.raw_violation_fixup = false,
+	.update_fc_timer = false,
 	.ectl.enable = false,
 };
 
@@ -2459,6 +2469,7 @@ static const struct tegra_pcie_soc tegra124_pcie = {
 	.update_clamp_threshold = true,
 	.program_deskew_time = false,
 	.raw_violation_fixup = true,
+	.update_fc_timer = false,
 	.ectl.enable = false,
 };
 
@@ -2469,6 +2480,8 @@ static const struct tegra_pcie_soc tegra210_pcie = {
 	.pads_pll_ctl = PADS_PLL_CTL_TEGRA30,
 	.tx_ref_sel = PADS_PLL_CTL_TXCLKREF_BUF_EN,
 	.pads_refclk_cfg0 = 0x90b890b8,
+	/* FC threshold is bit[25:18] */
+	.update_fc_threshold = 0x01800000,
 	.has_pex_clkreq_en = true,
 	.has_pex_bias_ctrl = true,
 	.has_intr_prsnt_sense = true,
@@ -2479,6 +2492,7 @@ static const struct tegra_pcie_soc tegra210_pcie = {
 	.update_clamp_threshold = true,
 	.program_deskew_time = true,
 	.raw_violation_fixup = false,
+	.update_fc_timer = true,
 	.ectl = {
 		.regs = {
 			.rp_ectl_2_r1 = 0x0000000f,
@@ -2518,6 +2532,7 @@ static const struct tegra_pcie_soc tegra186_pcie = {
 	.update_clamp_threshold = false,
 	.program_deskew_time = false,
 	.raw_violation_fixup = false,
+	.update_fc_timer = false,
 	.ectl.enable = false,
 };
 
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 72+ messages in thread

* [PATCH V4 16/28] PCI: tegra: Set target speed as Gen1 before starting LTSSM
  2019-05-16  5:52 [PATCH V4 00/28] Enable Tegra PCIe root port features Manikanta Maddireddy
                   ` (14 preceding siblings ...)
  2019-05-16  5:52 ` [PATCH V4 15/28] PCI: tegra: Update flow control timer frequency in Tegra210 Manikanta Maddireddy
@ 2019-05-16  5:52 ` Manikanta Maddireddy
  2019-05-16  5:52 ` [PATCH V4 17/28] PCI: tegra: Fix PLLE power down issue due to CLKREQ# signal Manikanta Maddireddy
                   ` (13 subsequent siblings)
  29 siblings, 0 replies; 72+ messages in thread
From: Manikanta Maddireddy @ 2019-05-16  5:52 UTC (permalink / raw)
  To: thierry.reding, bhelgaas, robh+dt, mark.rutland, jonathanh,
	lorenzo.pieralisi, vidyas
  Cc: linux-tegra, linux-pci, devicetree, Manikanta Maddireddy

PCIe link up fails with few legacy endpoints if root port advertises both
Gen-1 and Gen-2 speeds in Tegra. This is because link number negotiation
fails if both Gen1 & Gen2 are advertised. Tegra doesn't retry link up by
advertising only Gen1. Hence, the strategy followed here is to initially
advertise only Gen-1 and after link is up, retrain link to Gen-2 speed.

Tegra doesn't support HW autonomous speed change. Link comes up in Gen1
even if Gen2 is advertised, so there is no downside of this change.

This behavior is observed with following two PCIe devices on Tegra,
 - Fusion HDTV 5 Express card
 - IOGear SIL - PCIE - SATA card

Signed-off-by: Manikanta Maddireddy <mmaddireddy@nvidia.com>
Acked-by: Thierry Reding <treding@nvidia.com>
---
V4: No change

V3: No change

V2: Updated commit log to reflect why this issue is observed on Tegra with
these particular cards

 drivers/pci/controller/pci-tegra.c | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/drivers/pci/controller/pci-tegra.c b/drivers/pci/controller/pci-tegra.c
index 16d4d3cba3ff..9ee111062ab7 100644
--- a/drivers/pci/controller/pci-tegra.c
+++ b/drivers/pci/controller/pci-tegra.c
@@ -676,6 +676,17 @@ static void tegra_pcie_apply_sw_fixup(struct tegra_pcie_port *port)
 		value |= soc->update_fc_threshold;
 		writel(value, port->base + RP_VEND_XP);
 	}
+
+	/*
+	 * PCIe link doesn't come up with few legacy PCIe endpoints if
+	 * root port advertises both Gen-1 and Gen-2 speeds in Tegra.
+	 * Hence, the strategy followed here is to initially advertise
+	 * only Gen-1 and after link is up, retrain link to Gen-2 speed
+	 */
+	value = readl(port->base + RP_LINK_CONTROL_STATUS_2);
+	value &= ~PCI_EXP_LNKSTA_CLS;
+	value |= PCI_EXP_LNKSTA_CLS_2_5GB;
+	writel(value, port->base + RP_LINK_CONTROL_STATUS_2);
 }
 
 static void tegra_pcie_port_enable(struct tegra_pcie_port *port)
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 72+ messages in thread

* [PATCH V4 17/28] PCI: tegra: Fix PLLE power down issue due to CLKREQ# signal
  2019-05-16  5:52 [PATCH V4 00/28] Enable Tegra PCIe root port features Manikanta Maddireddy
                   ` (15 preceding siblings ...)
  2019-05-16  5:52 ` [PATCH V4 16/28] PCI: tegra: Set target speed as Gen1 before starting LTSSM Manikanta Maddireddy
@ 2019-05-16  5:52 ` Manikanta Maddireddy
  2019-05-16  5:52 ` [PATCH V4 18/28] PCI: tegra: Program AFI_CACHE* registers only for Tegra20 Manikanta Maddireddy
                   ` (12 subsequent siblings)
  29 siblings, 0 replies; 72+ messages in thread
From: Manikanta Maddireddy @ 2019-05-16  5:52 UTC (permalink / raw)
  To: thierry.reding, bhelgaas, robh+dt, mark.rutland, jonathanh,
	lorenzo.pieralisi, vidyas
  Cc: linux-tegra, linux-pci, devicetree, Manikanta Maddireddy

Disable controllers which failed to link up and configure CLKREQ# signals
of these controllers as GPIO. This is required to avoid CLKREQ# signal of
inactive controllers interfering with PLLE power down sequence.

PCIE_CLKREQ_GPIO bits are defined only in Tegra186, however programming
these bits in other SoCs doesn't cause any side effects. Program these
bits for all Tegra SoCs to avoid conditional check.

Signed-off-by: Manikanta Maddireddy <mmaddireddy@nvidia.com>
Acked-by: Thierry Reding <treding@nvidia.com>
---
V4: No change

V3: No change

V2: Corrected the comment in driver

 drivers/pci/controller/pci-tegra.c | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/drivers/pci/controller/pci-tegra.c b/drivers/pci/controller/pci-tegra.c
index 9ee111062ab7..4767af9f3b88 100644
--- a/drivers/pci/controller/pci-tegra.c
+++ b/drivers/pci/controller/pci-tegra.c
@@ -160,6 +160,8 @@
 #define  AFI_PCIE_CONFIG_SM2TMS0_XBAR_CONFIG_211	(0x1 << 20)
 #define  AFI_PCIE_CONFIG_SM2TMS0_XBAR_CONFIG_411	(0x2 << 20)
 #define  AFI_PCIE_CONFIG_SM2TMS0_XBAR_CONFIG_111	(0x2 << 20)
+#define  AFI_PCIE_CONFIG_PCIE_CLKREQ_GPIO(x)		(1 << ((x) + 29))
+#define  AFI_PCIE_CONFIG_PCIE_CLKREQ_GPIO_ALL		(0x7 << 29)
 
 #define AFI_FUSE			0x104
 #define  AFI_FUSE_PCIE_T0_GEN2_DIS	(1 << 2)
@@ -741,6 +743,12 @@ static void tegra_pcie_port_disable(struct tegra_pcie_port *port)
 
 	value &= ~AFI_PEX_CTRL_REFCLK_EN;
 	afi_writel(port->pcie, value, ctrl);
+
+	/* disable PCIe port and set CLKREQ# as GPIO to allow PLLE power down */
+	value = afi_readl(port->pcie, AFI_PCIE_CONFIG);
+	value |= AFI_PCIE_CONFIG_PCIE_DISABLE(port->index);
+	value |= AFI_PCIE_CONFIG_PCIE_CLKREQ_GPIO(port->index);
+	afi_writel(port->pcie, value, AFI_PCIE_CONFIG);
 }
 
 static void tegra_pcie_port_free(struct tegra_pcie_port *port)
@@ -1153,9 +1161,12 @@ static void tegra_pcie_enable_controller(struct tegra_pcie *pcie)
 	value = afi_readl(pcie, AFI_PCIE_CONFIG);
 	value &= ~AFI_PCIE_CONFIG_SM2TMS0_XBAR_CONFIG_MASK;
 	value |= AFI_PCIE_CONFIG_PCIE_DISABLE_ALL | pcie->xbar_config;
+	value |= AFI_PCIE_CONFIG_PCIE_CLKREQ_GPIO_ALL;
 
-	list_for_each_entry(port, &pcie->ports, list)
+	list_for_each_entry(port, &pcie->ports, list) {
 		value &= ~AFI_PCIE_CONFIG_PCIE_DISABLE(port->index);
+		value &= ~AFI_PCIE_CONFIG_PCIE_CLKREQ_GPIO(port->index);
+	}
 
 	afi_writel(pcie, value, AFI_PCIE_CONFIG);
 
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 72+ messages in thread

* [PATCH V4 18/28] PCI: tegra: Program AFI_CACHE* registers only for Tegra20
  2019-05-16  5:52 [PATCH V4 00/28] Enable Tegra PCIe root port features Manikanta Maddireddy
                   ` (16 preceding siblings ...)
  2019-05-16  5:52 ` [PATCH V4 17/28] PCI: tegra: Fix PLLE power down issue due to CLKREQ# signal Manikanta Maddireddy
@ 2019-05-16  5:52 ` Manikanta Maddireddy
  2019-05-16  5:52 ` [PATCH V4 19/28] PCI: tegra: Change PRSNT_SENSE IRQ log to debug Manikanta Maddireddy
                   ` (11 subsequent siblings)
  29 siblings, 0 replies; 72+ messages in thread
From: Manikanta Maddireddy @ 2019-05-16  5:52 UTC (permalink / raw)
  To: thierry.reding, bhelgaas, robh+dt, mark.rutland, jonathanh,
	lorenzo.pieralisi, vidyas
  Cc: linux-tegra, linux-pci, devicetree, Manikanta Maddireddy

Cacheable upstream transactions are supported in Tegra20 and Tegra186 only.
AFI_CACHE* registers are available in Tegra20 to support cacheable upstream
transactions. In Tegra186, AFI_AXCACHE register is defined instead of
AFI_CACHE* to be in line with its MSS design. Therefore, program AFI_CACHE*
registers only for Tegra20.

Signed-off-by: Manikanta Maddireddy <mmaddireddy@nvidia.com>
Acked-by: Thierry Reding <treding@nvidia.com>
---
V4: No change

V3: Initialized has_cache_bars variable for each soc data structure.

V2: Used soc variable for comparision instead of compatible string.

 drivers/pci/controller/pci-tegra.c | 18 +++++++++++++-----
 1 file changed, 13 insertions(+), 5 deletions(-)

diff --git a/drivers/pci/controller/pci-tegra.c b/drivers/pci/controller/pci-tegra.c
index 4767af9f3b88..0f02f54f8a61 100644
--- a/drivers/pci/controller/pci-tegra.c
+++ b/drivers/pci/controller/pci-tegra.c
@@ -323,6 +323,7 @@ struct tegra_pcie_soc {
 	bool program_deskew_time;
 	bool raw_violation_fixup;
 	bool update_fc_timer;
+	bool has_cache_bars;
 	struct {
 		struct {
 			u32 rp_ectl_2_r1;
@@ -932,11 +933,13 @@ static void tegra_pcie_setup_translations(struct tegra_pcie *pcie)
 	afi_writel(pcie, 0, AFI_AXI_BAR5_SZ);
 	afi_writel(pcie, 0, AFI_FPCI_BAR5);
 
-	/* map all upstream transactions as uncached */
-	afi_writel(pcie, 0, AFI_CACHE_BAR0_ST);
-	afi_writel(pcie, 0, AFI_CACHE_BAR0_SZ);
-	afi_writel(pcie, 0, AFI_CACHE_BAR1_ST);
-	afi_writel(pcie, 0, AFI_CACHE_BAR1_SZ);
+	if (pcie->soc->has_cache_bars) {
+		/* map all upstream transactions as uncached */
+		afi_writel(pcie, 0, AFI_CACHE_BAR0_ST);
+		afi_writel(pcie, 0, AFI_CACHE_BAR0_SZ);
+		afi_writel(pcie, 0, AFI_CACHE_BAR1_ST);
+		afi_writel(pcie, 0, AFI_CACHE_BAR1_SZ);
+	}
 
 	/* MSI translations are setup only when needed */
 	afi_writel(pcie, 0, AFI_MSI_FPCI_BAR_ST);
@@ -2441,6 +2444,7 @@ static const struct tegra_pcie_soc tegra20_pcie = {
 	.program_deskew_time = false,
 	.raw_violation_fixup = false,
 	.update_fc_timer = false,
+	.has_cache_bars = true,
 	.ectl.enable = false,
 };
 
@@ -2469,6 +2473,7 @@ static const struct tegra_pcie_soc tegra30_pcie = {
 	.program_deskew_time = false,
 	.raw_violation_fixup = false,
 	.update_fc_timer = false,
+	.has_cache_bars = false,
 	.ectl.enable = false,
 };
 
@@ -2492,6 +2497,7 @@ static const struct tegra_pcie_soc tegra124_pcie = {
 	.program_deskew_time = false,
 	.raw_violation_fixup = true,
 	.update_fc_timer = false,
+	.has_cache_bars = false,
 	.ectl.enable = false,
 };
 
@@ -2515,6 +2521,7 @@ static const struct tegra_pcie_soc tegra210_pcie = {
 	.program_deskew_time = true,
 	.raw_violation_fixup = false,
 	.update_fc_timer = true,
+	.has_cache_bars = false,
 	.ectl = {
 		.regs = {
 			.rp_ectl_2_r1 = 0x0000000f,
@@ -2555,6 +2562,7 @@ static const struct tegra_pcie_soc tegra186_pcie = {
 	.program_deskew_time = false,
 	.raw_violation_fixup = false,
 	.update_fc_timer = false,
+	.has_cache_bars = false,
 	.ectl.enable = false,
 };
 
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 72+ messages in thread

* [PATCH V4 19/28] PCI: tegra: Change PRSNT_SENSE IRQ log to debug
  2019-05-16  5:52 [PATCH V4 00/28] Enable Tegra PCIe root port features Manikanta Maddireddy
                   ` (17 preceding siblings ...)
  2019-05-16  5:52 ` [PATCH V4 18/28] PCI: tegra: Program AFI_CACHE* registers only for Tegra20 Manikanta Maddireddy
@ 2019-05-16  5:52 ` Manikanta Maddireddy
  2019-05-16  5:52 ` [PATCH V4 20/28] PCI: tegra: Use legacy IRQ for port service drivers Manikanta Maddireddy
                   ` (10 subsequent siblings)
  29 siblings, 0 replies; 72+ messages in thread
From: Manikanta Maddireddy @ 2019-05-16  5:52 UTC (permalink / raw)
  To: thierry.reding, bhelgaas, robh+dt, mark.rutland, jonathanh,
	lorenzo.pieralisi, vidyas
  Cc: linux-tegra, linux-pci, devicetree, Manikanta Maddireddy

PRSNT_MAP bit field is programmed to update the slot present status.
PRSNT_SENSE IRQ is triggered when this bit field is programmed, which is
not an error. Add a new if condition to trap PRSNT_SENSE code and print it
with debug log level.

Signed-off-by: Manikanta Maddireddy <mmaddireddy@nvidia.com>
Acked-by: Thierry Reding <treding@nvidia.com>
---
V4: No change

V3: Correct typos in commit log

V2: If-else to switch-case conversion patch is dropped, this patch is
rebased to stay with if-else statements

 drivers/pci/controller/pci-tegra.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/pci/controller/pci-tegra.c b/drivers/pci/controller/pci-tegra.c
index 0f02f54f8a61..ac57c5badd9b 100644
--- a/drivers/pci/controller/pci-tegra.c
+++ b/drivers/pci/controller/pci-tegra.c
@@ -864,7 +864,7 @@ static irqreturn_t tegra_pcie_isr(int irq, void *arg)
 	 * do not pollute kernel log with master abort reports since they
 	 * happen a lot during enumeration
 	 */
-	if (code == AFI_INTR_MASTER_ABORT)
+	if (code == AFI_INTR_MASTER_ABORT || code == AFI_INTR_PE_PRSNT_SENSE)
 		dev_dbg(dev, "%s, signature: %08x\n", err_msg[code], signature);
 	else
 		dev_err(dev, "%s, signature: %08x\n", err_msg[code], signature);
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 72+ messages in thread

* [PATCH V4 20/28] PCI: tegra: Use legacy IRQ for port service drivers
  2019-05-16  5:52 [PATCH V4 00/28] Enable Tegra PCIe root port features Manikanta Maddireddy
                   ` (18 preceding siblings ...)
  2019-05-16  5:52 ` [PATCH V4 19/28] PCI: tegra: Change PRSNT_SENSE IRQ log to debug Manikanta Maddireddy
@ 2019-05-16  5:52 ` Manikanta Maddireddy
  2019-05-20 20:37   ` Bjorn Helgaas
  2019-05-16  5:53 ` [PATCH V4 21/28] PCI: tegra: Add AFI_PEX2_CTRL reg offset as part of soc struct Manikanta Maddireddy
                   ` (9 subsequent siblings)
  29 siblings, 1 reply; 72+ messages in thread
From: Manikanta Maddireddy @ 2019-05-16  5:52 UTC (permalink / raw)
  To: thierry.reding, bhelgaas, robh+dt, mark.rutland, jonathanh,
	lorenzo.pieralisi, vidyas
  Cc: linux-tegra, linux-pci, devicetree, Manikanta Maddireddy

Tegra signals PCIe services like AER, PME, etc. over legacy IRQ line.
By default, service drivers register interrupt routine over MSI IRQ line.
Use pcie_pme_disable_msi() function to disable MSI for service drivers.

PME and AER interrupts registered to MSI without this change,
cat /proc/interrupts | grep -i pci
36: 21 0 0 0 0 0 GICv2 104 Level       PCIE
37: 35 0 0 0 0 0 GICv2 105 Level       Tegra PCIe MSI
76: 0  0 0 0 0 0 Tegra PCIe MSI 0 Edge PCIe PME, aerdrv, PCIe BW notif

PME and AER interrupts registered to legacy IRQ with this change,
cat /proc/interrupts | grep -i pci
36: 33 0 0 0 0 0 GICv2 104 Level      PCIE, PCIe PME, aerdrv, PCIe BW notif
37: 52 0 0 0 0 0 GICv2 105 Level      Tegra PCIe MSI

Signed-off-by: Manikanta Maddireddy <mmaddireddy@nvidia.com>
Acked-by: Thierry Reding <treding@nvidia.com>
---
V4: No change

V3: Corrected typo in commit log

V2: No change

 drivers/pci/controller/pci-tegra.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/drivers/pci/controller/pci-tegra.c b/drivers/pci/controller/pci-tegra.c
index ac57c5badd9b..0024bc42b400 100644
--- a/drivers/pci/controller/pci-tegra.c
+++ b/drivers/pci/controller/pci-tegra.c
@@ -41,6 +41,7 @@
 #include <soc/tegra/pmc.h>
 
 #include "../pci.h"
+#include "../pcie/portdrv.h"
 
 #define INT_PCI_MSI_NR (8 * 32)
 
@@ -2725,6 +2726,9 @@ static int tegra_pcie_probe(struct platform_device *pdev)
 		goto put_resources;
 	}
 
+	/* Switch to legacy IRQ for PCIe services like AER, PME*/
+	pcie_pme_disable_msi();
+
 	pm_runtime_enable(pcie->dev);
 	err = pm_runtime_get_sync(pcie->dev);
 	if (err) {
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 72+ messages in thread

* [PATCH V4 21/28] PCI: tegra: Add AFI_PEX2_CTRL reg offset as part of soc struct
  2019-05-16  5:52 [PATCH V4 00/28] Enable Tegra PCIe root port features Manikanta Maddireddy
                   ` (19 preceding siblings ...)
  2019-05-16  5:52 ` [PATCH V4 20/28] PCI: tegra: Use legacy IRQ for port service drivers Manikanta Maddireddy
@ 2019-05-16  5:53 ` Manikanta Maddireddy
  2019-05-16  5:53 ` [PATCH V4 22/28] PCI: tegra: Access endpoint config only if PCIe link is up Manikanta Maddireddy
                   ` (8 subsequent siblings)
  29 siblings, 0 replies; 72+ messages in thread
From: Manikanta Maddireddy @ 2019-05-16  5:53 UTC (permalink / raw)
  To: thierry.reding, bhelgaas, robh+dt, mark.rutland, jonathanh,
	lorenzo.pieralisi, vidyas
  Cc: linux-tegra, linux-pci, devicetree, Manikanta Maddireddy

Tegra186 and Tegra30 have three PCIe root ports. AFI_PEX2_CTRL register
is defined for third root port. Offset of this register in Tegra186 is
different from Tegra30, so add offset as part of soc data structure.

Signed-off-by: Manikanta Maddireddy <mmaddireddy@nvidia.com>
Acked-by: Thierry Reding <treding@nvidia.com>
---
V4: No change

V3: No change

V2: No change

 drivers/pci/controller/pci-tegra.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/drivers/pci/controller/pci-tegra.c b/drivers/pci/controller/pci-tegra.c
index 0024bc42b400..d20c88a79e00 100644
--- a/drivers/pci/controller/pci-tegra.c
+++ b/drivers/pci/controller/pci-tegra.c
@@ -169,7 +169,6 @@
 
 #define AFI_PEX0_CTRL			0x110
 #define AFI_PEX1_CTRL			0x118
-#define AFI_PEX2_CTRL			0x128
 #define  AFI_PEX_CTRL_RST		(1 << 0)
 #define  AFI_PEX_CTRL_CLKREQ_EN		(1 << 1)
 #define  AFI_PEX_CTRL_REFCLK_EN		(1 << 3)
@@ -308,6 +307,7 @@ struct tegra_pcie_soc {
 	unsigned int num_ports;
 	const struct tegra_pcie_port_soc *ports;
 	unsigned int msi_base_shift;
+	unsigned long afi_pex2_ctrl;
 	u32 pads_pll_ctl;
 	u32 tx_ref_sel;
 	u32 pads_refclk_cfg0;
@@ -518,6 +518,7 @@ static struct pci_ops tegra_pcie_ops = {
 
 static unsigned long tegra_pcie_port_get_pex_ctrl(struct tegra_pcie_port *port)
 {
+	const struct tegra_pcie_soc *soc = port->pcie->soc;
 	unsigned long ret = 0;
 
 	switch (port->index) {
@@ -530,7 +531,7 @@ static unsigned long tegra_pcie_port_get_pex_ctrl(struct tegra_pcie_port *port)
 		break;
 
 	case 2:
-		ret = AFI_PEX2_CTRL;
+		ret = soc->afi_pex2_ctrl;
 		break;
 	}
 
@@ -2431,6 +2432,7 @@ static const struct tegra_pcie_soc tegra20_pcie = {
 	.num_ports = 2,
 	.ports = tegra20_pcie_ports,
 	.msi_base_shift = 0,
+	.afi_pex2_ctrl = 0x128,
 	.pads_pll_ctl = PADS_PLL_CTL_TEGRA20,
 	.tx_ref_sel = PADS_PLL_CTL_TXCLKREF_DIV10,
 	.pads_refclk_cfg0 = 0xfa5cfa5c,
@@ -2548,6 +2550,7 @@ static const struct tegra_pcie_soc tegra186_pcie = {
 	.num_ports = 3,
 	.ports = tegra186_pcie_ports,
 	.msi_base_shift = 8,
+	.afi_pex2_ctrl = 0x19c,
 	.pads_pll_ctl = PADS_PLL_CTL_TEGRA30,
 	.tx_ref_sel = PADS_PLL_CTL_TXCLKREF_BUF_EN,
 	.pads_refclk_cfg0 = 0x80b880b8,
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 72+ messages in thread

* [PATCH V4 22/28] PCI: tegra: Access endpoint config only if PCIe link is up
  2019-05-16  5:52 [PATCH V4 00/28] Enable Tegra PCIe root port features Manikanta Maddireddy
                   ` (20 preceding siblings ...)
  2019-05-16  5:53 ` [PATCH V4 21/28] PCI: tegra: Add AFI_PEX2_CTRL reg offset as part of soc struct Manikanta Maddireddy
@ 2019-05-16  5:53 ` Manikanta Maddireddy
  2019-06-04 13:14   ` Thierry Reding
  2019-05-16  5:53 ` [PATCH V4 23/28] dt-bindings: pci: tegra: Document PCIe DPD pinctrl optional prop Manikanta Maddireddy
                   ` (7 subsequent siblings)
  29 siblings, 1 reply; 72+ messages in thread
From: Manikanta Maddireddy @ 2019-05-16  5:53 UTC (permalink / raw)
  To: thierry.reding, bhelgaas, robh+dt, mark.rutland, jonathanh,
	lorenzo.pieralisi, vidyas
  Cc: linux-tegra, linux-pci, devicetree, Manikanta Maddireddy

Few endpoints like Wi-Fi supports power on/off and to leverage that
root port must support hot-plug and hot-unplug. Tegra PCIe doesn't
support hot-plug and hot-unplug, however it supports endpoint power
on/off feature as follows,
 - Power off sequence:
   - Transition of PCIe link to L2
   - Power off endpoint
   - Leave root port in power up state with the link in L2
 - Power on sequence:
   - Power on endpoint
   - Apply hot reset to get PCIe link up

PCIe client driver stops accessing PCIe endpoint config and BAR registers
after endpoint is powered off. However, software applications like x11
server or lspci can access endpoint config registers in which case
host controller raises "response decoding" errors. To avoid this scenario,
add PCIe link up check in config read and write callback functions before
accessing endpoint config registers.

Signed-off-by: Manikanta Maddireddy <mmaddireddy@nvidia.com>
---
V4: No change

V3: Update the commit log with explanation for the need of this patch

V2: Change tegra_pcie_link_status() to tegra_pcie_link_up()

 drivers/pci/controller/pci-tegra.c | 38 ++++++++++++++++++++++++++++++
 1 file changed, 38 insertions(+)

diff --git a/drivers/pci/controller/pci-tegra.c b/drivers/pci/controller/pci-tegra.c
index d20c88a79e00..33f4dfab9e35 100644
--- a/drivers/pci/controller/pci-tegra.c
+++ b/drivers/pci/controller/pci-tegra.c
@@ -428,6 +428,14 @@ static inline u32 pads_readl(struct tegra_pcie *pcie, unsigned long offset)
 	return readl(pcie->pads + offset);
 }
 
+static bool tegra_pcie_link_up(struct tegra_pcie_port *port)
+{
+	u32 value;
+
+	value = readl(port->base + RP_LINK_CONTROL_STATUS);
+	return !!(value & RP_LINK_CONTROL_STATUS_DL_LINK_ACTIVE);
+}
+
 /*
  * The configuration space mapping on Tegra is somewhat similar to the ECAM
  * defined by PCIe. However it deviates a bit in how the 4 bits for extended
@@ -493,20 +501,50 @@ static void __iomem *tegra_pcie_map_bus(struct pci_bus *bus,
 static int tegra_pcie_config_read(struct pci_bus *bus, unsigned int devfn,
 				  int where, int size, u32 *value)
 {
+	struct tegra_pcie *pcie = bus->sysdata;
+	struct pci_dev *bridge;
+	struct tegra_pcie_port *port;
+
 	if (bus->number == 0)
 		return pci_generic_config_read32(bus, devfn, where, size,
 						 value);
 
+	bridge = pcie_find_root_port(bus->self);
+
+	list_for_each_entry(port, &pcie->ports, list)
+		if (port->index + 1 == PCI_SLOT(bridge->devfn))
+			break;
+
+	/* If there is no link, then there is no device */
+	if (!tegra_pcie_link_up(port)) {
+		*value = 0xffffffff;
+		return PCIBIOS_DEVICE_NOT_FOUND;
+	}
+
 	return pci_generic_config_read(bus, devfn, where, size, value);
 }
 
 static int tegra_pcie_config_write(struct pci_bus *bus, unsigned int devfn,
 				   int where, int size, u32 value)
 {
+	struct tegra_pcie *pcie = bus->sysdata;
+	struct tegra_pcie_port *port;
+	struct pci_dev *bridge;
+
 	if (bus->number == 0)
 		return pci_generic_config_write32(bus, devfn, where, size,
 						  value);
 
+	bridge = pcie_find_root_port(bus->self);
+
+	list_for_each_entry(port, &pcie->ports, list)
+		if (port->index + 1 == PCI_SLOT(bridge->devfn))
+			break;
+
+	/* If there is no link, then there is no device */
+	if (!tegra_pcie_link_up(port))
+		return PCIBIOS_DEVICE_NOT_FOUND;
+
 	return pci_generic_config_write(bus, devfn, where, size, value);
 }
 
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 72+ messages in thread

* [PATCH V4 23/28] dt-bindings: pci: tegra: Document PCIe DPD pinctrl optional prop
  2019-05-16  5:52 [PATCH V4 00/28] Enable Tegra PCIe root port features Manikanta Maddireddy
                   ` (21 preceding siblings ...)
  2019-05-16  5:53 ` [PATCH V4 22/28] PCI: tegra: Access endpoint config only if PCIe link is up Manikanta Maddireddy
@ 2019-05-16  5:53 ` Manikanta Maddireddy
  2019-05-16  5:53 ` [PATCH V4 24/28] arm64: tegra: Add PEX DPD states as pinctrl properties Manikanta Maddireddy
                   ` (6 subsequent siblings)
  29 siblings, 0 replies; 72+ messages in thread
From: Manikanta Maddireddy @ 2019-05-16  5:53 UTC (permalink / raw)
  To: thierry.reding, bhelgaas, robh+dt, mark.rutland, jonathanh,
	lorenzo.pieralisi, vidyas
  Cc: linux-tegra, linux-pci, devicetree, Manikanta Maddireddy

Document PCIe DPD pinctrl optional property to put PEX clk & BIAS pads
in low power mode.

Signed-off-by: Manikanta Maddireddy <mmaddireddy@nvidia.com>
Reviewed-by: Rob Herring <robh@kernel.org>
Acked-by: Thierry Reding <treding@nvidia.com>
---
V4: No change

V3: No change

V2: Using standard pinctrl names, default and idle

 .../devicetree/bindings/pci/nvidia,tegra20-pcie.txt       | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/Documentation/devicetree/bindings/pci/nvidia,tegra20-pcie.txt b/Documentation/devicetree/bindings/pci/nvidia,tegra20-pcie.txt
index 145a4f04194f..7939bca47861 100644
--- a/Documentation/devicetree/bindings/pci/nvidia,tegra20-pcie.txt
+++ b/Documentation/devicetree/bindings/pci/nvidia,tegra20-pcie.txt
@@ -65,6 +65,14 @@ Required properties:
   - afi
   - pcie_x
 
+Optional properties:
+- pinctrl-names: A list of pinctrl state names. Must contain the following
+  entries:
+  - "default": active state, puts PCIe I/O out of deep power down state
+  - "idle": puts PCIe I/O into deep power down state
+- pinctrl-0: phandle for the default/active state of pin configurations.
+- pinctrl-1: phandle for the idle state of pin configurations.
+
 Required properties on Tegra124 and later (deprecated):
 - phys: Must contain an entry for each entry in phy-names.
 - phy-names: Must include the following entries:
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 72+ messages in thread

* [PATCH V4 24/28] arm64: tegra: Add PEX DPD states as pinctrl properties
  2019-05-16  5:52 [PATCH V4 00/28] Enable Tegra PCIe root port features Manikanta Maddireddy
                   ` (22 preceding siblings ...)
  2019-05-16  5:53 ` [PATCH V4 23/28] dt-bindings: pci: tegra: Document PCIe DPD pinctrl optional prop Manikanta Maddireddy
@ 2019-05-16  5:53 ` Manikanta Maddireddy
  2019-05-16  5:53 ` [PATCH V4 25/28] PCI: tegra: Put PEX CLK & BIAS pads in DPD mode Manikanta Maddireddy
                   ` (5 subsequent siblings)
  29 siblings, 0 replies; 72+ messages in thread
From: Manikanta Maddireddy @ 2019-05-16  5:53 UTC (permalink / raw)
  To: thierry.reding, bhelgaas, robh+dt, mark.rutland, jonathanh,
	lorenzo.pieralisi, vidyas
  Cc: linux-tegra, linux-pci, devicetree, Manikanta Maddireddy

Add PEX deep power down states as pinctrl properties to set in PCIe driver.
In Tegra210, BIAS pads are not in power down mode when clamps are applied.
To set the pads in DPD, pass the PEX DPD states as pinctrl properties to
PCIe driver.

Signed-off-by: Manikanta Maddireddy <mmaddireddy@nvidia.com>
---
V4: No change

V3: No change

V2: Using standard pinctrl names, default and idle

 arch/arm64/boot/dts/nvidia/tegra210.dtsi | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)

diff --git a/arch/arm64/boot/dts/nvidia/tegra210.dtsi b/arch/arm64/boot/dts/nvidia/tegra210.dtsi
index a550c0a4d572..3899c54ea28f 100644
--- a/arch/arm64/boot/dts/nvidia/tegra210.dtsi
+++ b/arch/arm64/boot/dts/nvidia/tegra210.dtsi
@@ -48,6 +48,11 @@
 			 <&tegra_car 72>,
 			 <&tegra_car 74>;
 		reset-names = "pex", "afi", "pcie_x";
+
+		pinctrl-names = "default", "idle";
+		pinctrl-0 = <&pex_dpd_disable>;
+		pinctrl-1 = <&pex_dpd_enable>;
+
 		status = "disabled";
 
 		pci@1,0 {
@@ -848,6 +853,20 @@
 			pins = "sdmmc3";
 			power-source = <TEGRA_IO_PAD_VOLTAGE_1V8>;
 		};
+
+		pex_dpd_disable: pex_en {
+			pex-dpd-disable {
+				pins = "pex-bias", "pex-clk1", "pex-clk2";
+				low-power-disable;
+			};
+		};
+
+		pex_dpd_enable: pex_dis {
+			pex-dpd-enable {
+				pins = "pex-bias", "pex-clk1", "pex-clk2";
+				low-power-enable;
+			};
+		};
 	};
 
 	fuse@7000f800 {
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 72+ messages in thread

* [PATCH V4 25/28] PCI: tegra: Put PEX CLK & BIAS pads in DPD mode
  2019-05-16  5:52 [PATCH V4 00/28] Enable Tegra PCIe root port features Manikanta Maddireddy
                   ` (23 preceding siblings ...)
  2019-05-16  5:53 ` [PATCH V4 24/28] arm64: tegra: Add PEX DPD states as pinctrl properties Manikanta Maddireddy
@ 2019-05-16  5:53 ` Manikanta Maddireddy
  2019-05-16  5:53 ` [PATCH V4 26/28] PCI: Add DT binding for "reset-gpios" property Manikanta Maddireddy
                   ` (4 subsequent siblings)
  29 siblings, 0 replies; 72+ messages in thread
From: Manikanta Maddireddy @ 2019-05-16  5:53 UTC (permalink / raw)
  To: thierry.reding, bhelgaas, robh+dt, mark.rutland, jonathanh,
	lorenzo.pieralisi, vidyas
  Cc: linux-tegra, linux-pci, devicetree, Manikanta Maddireddy

In Tegra210 AFI design has clamp value for the BIAS pad as 0, which keeps
the bias pad in non power down mode. This is leading to power consumption
of 2 mW in BIAS pad, even if the PCIe partition is powergated. To avoid
unnecessary power consumption, put PEX CLK & BIAS pads in deep power down
mode when PCIe partition is power gated.

Signed-off-by: Manikanta Maddireddy <mmaddireddy@nvidia.com>
Acked-by: Thierry Reding <treding@nvidia.com>
---
V4: No change

V3: No change

V2: Using standard pinctrl functions to apply default and idle states

 drivers/pci/controller/pci-tegra.c | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/drivers/pci/controller/pci-tegra.c b/drivers/pci/controller/pci-tegra.c
index 33f4dfab9e35..06b99fcbf382 100644
--- a/drivers/pci/controller/pci-tegra.c
+++ b/drivers/pci/controller/pci-tegra.c
@@ -30,6 +30,7 @@
 #include <linux/of_platform.h>
 #include <linux/pci.h>
 #include <linux/phy/phy.h>
+#include <linux/pinctrl/consumer.h>
 #include <linux/platform_device.h>
 #include <linux/reset.h>
 #include <linux/sizes.h>
@@ -2875,6 +2876,7 @@ static int __maybe_unused tegra_pcie_pm_suspend(struct device *dev)
 	if (IS_ENABLED(CONFIG_PCI_MSI))
 		tegra_pcie_disable_msi(pcie);
 
+	pinctrl_pm_select_idle_state(dev);
 	tegra_pcie_power_off(pcie);
 
 	return 0;
@@ -2890,6 +2892,13 @@ static int __maybe_unused tegra_pcie_pm_resume(struct device *dev)
 		dev_err(dev, "tegra pcie power on fail: %d\n", err);
 		return err;
 	}
+
+	err = pinctrl_pm_select_default_state(dev);
+	if (err < 0) {
+		dev_err(dev, "failed to disable PCIe IO DPD: %d\n", err);
+		goto poweroff;
+	}
+
 	tegra_pcie_enable_controller(pcie);
 	tegra_pcie_setup_translations(pcie);
 
@@ -2899,7 +2908,7 @@ static int __maybe_unused tegra_pcie_pm_resume(struct device *dev)
 	err = clk_prepare_enable(pcie->pex_clk);
 	if (err) {
 		dev_err(dev, "failed to enable PEX clock: %d\n", err);
-		goto poweroff;
+		goto pex_dpd_enable;
 	}
 
 	reset_control_deassert(pcie->pex_rst);
@@ -2920,6 +2929,8 @@ static int __maybe_unused tegra_pcie_pm_resume(struct device *dev)
 disable_pex_clk:
 	reset_control_assert(pcie->pex_rst);
 	clk_disable_unprepare(pcie->pex_clk);
+pex_dpd_enable:
+	pinctrl_pm_select_idle_state(dev);
 poweroff:
 	tegra_pcie_power_off(pcie);
 
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 72+ messages in thread

* [PATCH V4 26/28] PCI: Add DT binding for "reset-gpios" property
  2019-05-16  5:52 [PATCH V4 00/28] Enable Tegra PCIe root port features Manikanta Maddireddy
                   ` (24 preceding siblings ...)
  2019-05-16  5:53 ` [PATCH V4 25/28] PCI: tegra: Put PEX CLK & BIAS pads in DPD mode Manikanta Maddireddy
@ 2019-05-16  5:53 ` Manikanta Maddireddy
  2019-06-17 11:30   ` Thierry Reding
  2019-05-16  5:53 ` [PATCH V4 27/28] PCI: tegra: Add support for GPIO based PERST# Manikanta Maddireddy
                   ` (3 subsequent siblings)
  29 siblings, 1 reply; 72+ messages in thread
From: Manikanta Maddireddy @ 2019-05-16  5:53 UTC (permalink / raw)
  To: thierry.reding, bhelgaas, robh+dt, mark.rutland, jonathanh,
	lorenzo.pieralisi, vidyas
  Cc: linux-tegra, linux-pci, devicetree, Manikanta Maddireddy

Add DT binding for "reset-gpios" property which supports GPIO based PERST#
signal.

Signed-off-by: Manikanta Maddireddy <mmaddireddy@nvidia.com>
Reviewed-by: Rob Herring <robh@kernel.org>
Acked-by: Thierry Reding <treding@nvidia.com>
---
V4: No change

V3: Moved to common pci binding doc

V2: Using standard "reset-gpio" property

 Documentation/devicetree/bindings/pci/pci.txt | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/Documentation/devicetree/bindings/pci/pci.txt b/Documentation/devicetree/bindings/pci/pci.txt
index c77981c5dd18..79124898aa5b 100644
--- a/Documentation/devicetree/bindings/pci/pci.txt
+++ b/Documentation/devicetree/bindings/pci/pci.txt
@@ -24,3 +24,6 @@ driver implementation may support the following properties:
    unsupported link speed, for instance, trying to do training for
    unsupported link speed, etc.  Must be '4' for gen4, '3' for gen3, '2'
    for gen2, and '1' for gen1. Any other values are invalid.
+- reset-gpios:
+   If present this property specifies PERST# GPIO. Host drivers can parse the
+   GPIO and apply fundamental reset to endpoints.
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 72+ messages in thread

* [PATCH V4 27/28] PCI: tegra: Add support for GPIO based PERST#
  2019-05-16  5:52 [PATCH V4 00/28] Enable Tegra PCIe root port features Manikanta Maddireddy
                   ` (25 preceding siblings ...)
  2019-05-16  5:53 ` [PATCH V4 26/28] PCI: Add DT binding for "reset-gpios" property Manikanta Maddireddy
@ 2019-05-16  5:53 ` Manikanta Maddireddy
  2019-06-04 13:22   ` Thierry Reding
  2019-05-16  5:53 ` [PATCH V4 28/28] PCI: tegra: Change link retry log level to debug Manikanta Maddireddy
                   ` (2 subsequent siblings)
  29 siblings, 1 reply; 72+ messages in thread
From: Manikanta Maddireddy @ 2019-05-16  5:53 UTC (permalink / raw)
  To: thierry.reding, bhelgaas, robh+dt, mark.rutland, jonathanh,
	lorenzo.pieralisi, vidyas
  Cc: linux-tegra, linux-pci, devicetree, Manikanta Maddireddy

Add support for GPIO based PERST# signal. GPIO number comes from per port
PCIe device tree node.

Signed-off-by: Manikanta Maddireddy <mmaddireddy@nvidia.com>
---
V4: Using devm_gpiod_get_from_of_node() to get reset-gpios

V3: Using helper function to get reset-gpios

V2: Using standard "reset-gpio" property

 drivers/pci/controller/pci-tegra.c | 41 +++++++++++++++++++++++++-----
 1 file changed, 35 insertions(+), 6 deletions(-)

diff --git a/drivers/pci/controller/pci-tegra.c b/drivers/pci/controller/pci-tegra.c
index 06b99fcbf382..09b4d384ba38 100644
--- a/drivers/pci/controller/pci-tegra.c
+++ b/drivers/pci/controller/pci-tegra.c
@@ -17,6 +17,7 @@
 #include <linux/debugfs.h>
 #include <linux/delay.h>
 #include <linux/export.h>
+#include <linux/gpio.h>
 #include <linux/interrupt.h>
 #include <linux/iopoll.h>
 #include <linux/irq.h>
@@ -400,6 +401,8 @@ struct tegra_pcie_port {
 	unsigned int lanes;
 
 	struct phy **phys;
+
+	struct gpio_desc *reset_gpiod;
 };
 
 struct tegra_pcie_bus {
@@ -583,15 +586,23 @@ static void tegra_pcie_port_reset(struct tegra_pcie_port *port)
 	unsigned long value;
 
 	/* pulse reset signal */
-	value = afi_readl(port->pcie, ctrl);
-	value &= ~AFI_PEX_CTRL_RST;
-	afi_writel(port->pcie, value, ctrl);
+	if (port->reset_gpiod) {
+		gpiod_set_value(port->reset_gpiod, 0);
+	} else {
+		value = afi_readl(port->pcie, ctrl);
+		value &= ~AFI_PEX_CTRL_RST;
+		afi_writel(port->pcie, value, ctrl);
+	}
 
 	usleep_range(1000, 2000);
 
-	value = afi_readl(port->pcie, ctrl);
-	value |= AFI_PEX_CTRL_RST;
-	afi_writel(port->pcie, value, ctrl);
+	if (port->reset_gpiod) {
+		gpiod_set_value(port->reset_gpiod, 1);
+	} else {
+		value = afi_readl(port->pcie, ctrl);
+		value |= AFI_PEX_CTRL_RST;
+		afi_writel(port->pcie, value, ctrl);
+	}
 }
 
 static void tegra_pcie_enable_rp_features(struct tegra_pcie_port *port)
@@ -2238,6 +2249,7 @@ static int tegra_pcie_parse_dt(struct tegra_pcie *pcie)
 		struct tegra_pcie_port *rp;
 		unsigned int index;
 		u32 value;
+		char *label;
 
 		err = of_pci_get_devfn(port);
 		if (err < 0) {
@@ -2296,6 +2308,23 @@ static int tegra_pcie_parse_dt(struct tegra_pcie *pcie)
 		if (IS_ERR(rp->base))
 			return PTR_ERR(rp->base);
 
+		label = kasprintf(GFP_KERNEL, "pex-reset-%u", index);
+		if (!label) {
+			dev_err(dev, "failed to create reset GPIO label\n");
+			return -ENOMEM;
+		}
+
+		rp->reset_gpiod = devm_gpiod_get_from_of_node(dev, port,
+							      "reset-gpios", 0,
+							      GPIOD_OUT_LOW,
+							      label);
+		kfree(label);
+		if (IS_ERR(rp->reset_gpiod)) {
+			err = PTR_ERR(rp->reset_gpiod);
+			dev_err(dev, "failed to get reset GPIO: %d\n", err);
+			return err;
+		}
+
 		list_add_tail(&rp->list, &pcie->ports);
 	}
 
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 72+ messages in thread

* [PATCH V4 28/28] PCI: tegra: Change link retry log level to debug
  2019-05-16  5:52 [PATCH V4 00/28] Enable Tegra PCIe root port features Manikanta Maddireddy
                   ` (26 preceding siblings ...)
  2019-05-16  5:53 ` [PATCH V4 27/28] PCI: tegra: Add support for GPIO based PERST# Manikanta Maddireddy
@ 2019-05-16  5:53 ` Manikanta Maddireddy
  2019-06-04 13:22   ` Thierry Reding
  2019-05-16 13:12 ` [PATCH V4 00/28] Enable Tegra PCIe root port features Bjorn Helgaas
  2019-06-10  4:45 ` Manikanta Maddireddy
  29 siblings, 1 reply; 72+ messages in thread
From: Manikanta Maddireddy @ 2019-05-16  5:53 UTC (permalink / raw)
  To: thierry.reding, bhelgaas, robh+dt, mark.rutland, jonathanh,
	lorenzo.pieralisi, vidyas
  Cc: linux-tegra, linux-pci, devicetree, Manikanta Maddireddy

Driver checks for link up three times before giving up, each retry attempt
is printed as an error. Letting users know that PCIe link is down and in the
process of being brought up again is for debug, not an error condition.

Signed-off-by: Manikanta Maddireddy <mmaddireddy@nvidia.com>
---
V4: No change

V3: Changed dev_err to dev_dbg

V2: Updated commit log

 drivers/pci/controller/pci-tegra.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/pci/controller/pci-tegra.c b/drivers/pci/controller/pci-tegra.c
index 09b4d384ba38..e9420d87363e 100644
--- a/drivers/pci/controller/pci-tegra.c
+++ b/drivers/pci/controller/pci-tegra.c
@@ -2372,7 +2372,7 @@ static bool tegra_pcie_port_check_link(struct tegra_pcie_port *port)
 		} while (--timeout);
 
 		if (!timeout) {
-			dev_err(dev, "link %u down, retrying\n", port->index);
+			dev_dbg(dev, "link %u down, retrying\n", port->index);
 			goto retry;
 		}
 
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 72+ messages in thread

* Re: [PATCH V4 00/28] Enable Tegra PCIe root port features
  2019-05-16  5:52 [PATCH V4 00/28] Enable Tegra PCIe root port features Manikanta Maddireddy
                   ` (27 preceding siblings ...)
  2019-05-16  5:53 ` [PATCH V4 28/28] PCI: tegra: Change link retry log level to debug Manikanta Maddireddy
@ 2019-05-16 13:12 ` Bjorn Helgaas
  2019-05-17  8:38   ` Manikanta Maddireddy
  2019-06-10  4:45 ` Manikanta Maddireddy
  29 siblings, 1 reply; 72+ messages in thread
From: Bjorn Helgaas @ 2019-05-16 13:12 UTC (permalink / raw)
  To: Manikanta Maddireddy
  Cc: thierry.reding, robh+dt, mark.rutland, jonathanh,
	lorenzo.pieralisi, vidyas, linux-tegra, linux-pci, devicetree

On Thu, May 16, 2019 at 11:22:39AM +0530, Manikanta Maddireddy wrote:
> This series of patches adds,
> - Tegra root port features like Gen2, AER, etc
> - Power and perf optimizations
> - Fixes like "power up sequence", "dev_err prints", etc

Please:

  1) Put the brakes on.  You posted v3 of these 30 patches on May 13
     and v4 on May 16.  There's no hurry; the merge window is still
     open and nothing will be added to -next until at least next week.
     If you space these out a little, people will have time to digest
     them.

  2) Mention in the cover letter what changed between v3 and v4 so
     people know where to spend their effort.

> This series of patches are tested on Tegra186 based Jetson-TX2, Tegra210
> based Jetson-TX1, T124 based Jetson-TK1 platforms, Tegra20 and Tegra30
> platforms.
> 
> Manikanta Maddireddy (28):
>   soc/tegra: pmc: Export tegra_powergate_power_on()
>   PCI: tegra: Handle failure cases in tegra_pcie_power_on()
>   PCI: tegra: Rearrange Tegra PCIe driver functions
>   PCI: tegra: Mask AFI_INTR in runtime suspend
>   PCI: tegra: Fix PCIe host power up sequence
>   PCI: tegra: Add PCIe Gen2 link speed support
>   PCI: tegra: Advertise PCIe Advanced Error Reporting (AER) capability
>   PCI: tegra: Program UPHY electrical settings for Tegra210
>   PCI: tegra: Enable opportunistic UpdateFC and ACK
>   PCI: tegra: Disable AFI dynamic clock gating
>   PCI: tegra: Process pending DLL transactions before entering L1 or L2
>   PCI: tegra: Enable PCIe xclk clock clamping
>   PCI: tegra: Increase the deskew retry time
>   PCI: tegra: Add SW fixup for RAW violations
>   PCI: tegra: Update flow control timer frequency in Tegra210
>   PCI: tegra: Set target speed as Gen1 before starting LTSSM
>   PCI: tegra: Fix PLLE power down issue due to CLKREQ# signal
>   PCI: tegra: Program AFI_CACHE* registers only for Tegra20
>   PCI: tegra: Change PRSNT_SENSE IRQ log to debug
>   PCI: tegra: Use legacy IRQ for port service drivers
>   PCI: tegra: Add AFI_PEX2_CTRL reg offset as part of soc struct
>   PCI: tegra: Access endpoint config only if PCIe link is up
>   dt-bindings: pci: tegra: Document PCIe DPD pinctrl optional prop
>   arm64: tegra: Add PEX DPD states as pinctrl properties
>   PCI: tegra: Put PEX CLK & BIAS pads in DPD mode
>   PCI: Add DT binding for "reset-gpios" property
>   PCI: tegra: Add support for GPIO based PERST#
>   PCI: tegra: Change link retry log level to debug
> 
>  .../bindings/pci/nvidia,tegra20-pcie.txt      |   8 +
>  Documentation/devicetree/bindings/pci/pci.txt |   3 +
>  arch/arm64/boot/dts/nvidia/tegra210.dtsi      |  19 +
>  drivers/pci/controller/pci-tegra.c            | 615 +++++++++++++++---
>  drivers/soc/tegra/pmc.c                       |   1 +
>  5 files changed, 566 insertions(+), 80 deletions(-)
> 
> -- 
> 2.17.1
> 

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: [PATCH V4 00/28] Enable Tegra PCIe root port features
  2019-05-16 13:12 ` [PATCH V4 00/28] Enable Tegra PCIe root port features Bjorn Helgaas
@ 2019-05-17  8:38   ` Manikanta Maddireddy
  0 siblings, 0 replies; 72+ messages in thread
From: Manikanta Maddireddy @ 2019-05-17  8:38 UTC (permalink / raw)
  To: Bjorn Helgaas
  Cc: thierry.reding, robh+dt, mark.rutland, jonathanh,
	lorenzo.pieralisi, vidyas, linux-tegra, linux-pci, devicetree



On 16-May-19 6:42 PM, Bjorn Helgaas wrote:
> On Thu, May 16, 2019 at 11:22:39AM +0530, Manikanta Maddireddy wrote:
>> This series of patches adds,
>> - Tegra root port features like Gen2, AER, etc
>> - Power and perf optimizations
>> - Fixes like "power up sequence", "dev_err prints", etc
> Please:
>
>   1) Put the brakes on.  You posted v3 of these 30 patches on May 13
>      and v4 on May 16.  There's no hurry; the merge window is still
>      open and nothing will be added to -next until at least next week.
>      If you space these out a little, people will have time to digest
>      them.
>
>   2) Mention in the cover letter what changed between v3 and v4 so
>      people know where to spend their effort.

Reason for sending v4 quickly is because one of the patches deviated from
Rob & Thierry's comments in v2. To address this I published v4 and marked
v3 series as superseded. 

I will follow these two points from next time, meanwhile updating the
changes from v3 to v4 below,

Changes from v3 to v4:
 - Patch [V3,27/29] is dropped
 - Patch [V3,28/29]: devm_gpiod_get_from_of_node() is directly used in
   pci-tegra driver instead of of_get_pci* wrapper function defined in
   Patch [V3,27/29].

Manikanta 

>> This series of patches are tested on Tegra186 based Jetson-TX2, Tegra210
>> based Jetson-TX1, T124 based Jetson-TK1 platforms, Tegra20 and Tegra30
>> platforms.
>>
>> Manikanta Maddireddy (28):
>>   soc/tegra: pmc: Export tegra_powergate_power_on()
>>   PCI: tegra: Handle failure cases in tegra_pcie_power_on()
>>   PCI: tegra: Rearrange Tegra PCIe driver functions
>>   PCI: tegra: Mask AFI_INTR in runtime suspend
>>   PCI: tegra: Fix PCIe host power up sequence
>>   PCI: tegra: Add PCIe Gen2 link speed support
>>   PCI: tegra: Advertise PCIe Advanced Error Reporting (AER) capability
>>   PCI: tegra: Program UPHY electrical settings for Tegra210
>>   PCI: tegra: Enable opportunistic UpdateFC and ACK
>>   PCI: tegra: Disable AFI dynamic clock gating
>>   PCI: tegra: Process pending DLL transactions before entering L1 or L2
>>   PCI: tegra: Enable PCIe xclk clock clamping
>>   PCI: tegra: Increase the deskew retry time
>>   PCI: tegra: Add SW fixup for RAW violations
>>   PCI: tegra: Update flow control timer frequency in Tegra210
>>   PCI: tegra: Set target speed as Gen1 before starting LTSSM
>>   PCI: tegra: Fix PLLE power down issue due to CLKREQ# signal
>>   PCI: tegra: Program AFI_CACHE* registers only for Tegra20
>>   PCI: tegra: Change PRSNT_SENSE IRQ log to debug
>>   PCI: tegra: Use legacy IRQ for port service drivers
>>   PCI: tegra: Add AFI_PEX2_CTRL reg offset as part of soc struct
>>   PCI: tegra: Access endpoint config only if PCIe link is up
>>   dt-bindings: pci: tegra: Document PCIe DPD pinctrl optional prop
>>   arm64: tegra: Add PEX DPD states as pinctrl properties
>>   PCI: tegra: Put PEX CLK & BIAS pads in DPD mode
>>   PCI: Add DT binding for "reset-gpios" property
>>   PCI: tegra: Add support for GPIO based PERST#
>>   PCI: tegra: Change link retry log level to debug
>>
>>  .../bindings/pci/nvidia,tegra20-pcie.txt      |   8 +
>>  Documentation/devicetree/bindings/pci/pci.txt |   3 +
>>  arch/arm64/boot/dts/nvidia/tegra210.dtsi      |  19 +
>>  drivers/pci/controller/pci-tegra.c            | 615 +++++++++++++++---
>>  drivers/soc/tegra/pmc.c                       |   1 +
>>  5 files changed, 566 insertions(+), 80 deletions(-)
>>
>> -- 
>> 2.17.1
>>


^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: [PATCH V4 20/28] PCI: tegra: Use legacy IRQ for port service drivers
  2019-05-16  5:52 ` [PATCH V4 20/28] PCI: tegra: Use legacy IRQ for port service drivers Manikanta Maddireddy
@ 2019-05-20 20:37   ` Bjorn Helgaas
  2019-05-21  9:07     ` Manikanta Maddireddy
  0 siblings, 1 reply; 72+ messages in thread
From: Bjorn Helgaas @ 2019-05-20 20:37 UTC (permalink / raw)
  To: Manikanta Maddireddy
  Cc: thierry.reding, robh+dt, mark.rutland, jonathanh,
	lorenzo.pieralisi, vidyas, linux-tegra, linux-pci, devicetree

On Thu, May 16, 2019 at 11:22:59AM +0530, Manikanta Maddireddy wrote:
> Tegra signals PCIe services like AER, PME, etc. over legacy IRQ line.
> By default, service drivers register interrupt routine over MSI IRQ line.
> Use pcie_pme_disable_msi() function to disable MSI for service drivers.

I think this device is not quite spec-compliant:

  https://lore.kernel.org/linux-pci/20190520175729.GC49425@google.com/

and you should work around this with a quirk that sets pdev->no_msi so
we don't use MSI for it at all.

> PME and AER interrupts registered to MSI without this change,
> cat /proc/interrupts | grep -i pci
> 36: 21 0 0 0 0 0 GICv2 104 Level       PCIE
> 37: 35 0 0 0 0 0 GICv2 105 Level       Tegra PCIe MSI
> 76: 0  0 0 0 0 0 Tegra PCIe MSI 0 Edge PCIe PME, aerdrv, PCIe BW notif
> 
> PME and AER interrupts registered to legacy IRQ with this change,
> cat /proc/interrupts | grep -i pci
> 36: 33 0 0 0 0 0 GICv2 104 Level      PCIE, PCIe PME, aerdrv, PCIe BW notif
> 37: 52 0 0 0 0 0 GICv2 105 Level      Tegra PCIe MSI
> 
> Signed-off-by: Manikanta Maddireddy <mmaddireddy@nvidia.com>
> Acked-by: Thierry Reding <treding@nvidia.com>
> ---
> V4: No change
> 
> V3: Corrected typo in commit log
> 
> V2: No change
> 
>  drivers/pci/controller/pci-tegra.c | 4 ++++
>  1 file changed, 4 insertions(+)
> 
> diff --git a/drivers/pci/controller/pci-tegra.c b/drivers/pci/controller/pci-tegra.c
> index ac57c5badd9b..0024bc42b400 100644
> --- a/drivers/pci/controller/pci-tegra.c
> +++ b/drivers/pci/controller/pci-tegra.c
> @@ -41,6 +41,7 @@
>  #include <soc/tegra/pmc.h>
>  
>  #include "../pci.h"
> +#include "../pcie/portdrv.h"
>  
>  #define INT_PCI_MSI_NR (8 * 32)
>  
> @@ -2725,6 +2726,9 @@ static int tegra_pcie_probe(struct platform_device *pdev)
>  		goto put_resources;
>  	}
>  
> +	/* Switch to legacy IRQ for PCIe services like AER, PME*/
> +	pcie_pme_disable_msi();
> +
>  	pm_runtime_enable(pcie->dev);
>  	err = pm_runtime_get_sync(pcie->dev);
>  	if (err) {
> -- 
> 2.17.1
> 

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: [PATCH V4 20/28] PCI: tegra: Use legacy IRQ for port service drivers
  2019-05-20 20:37   ` Bjorn Helgaas
@ 2019-05-21  9:07     ` Manikanta Maddireddy
  0 siblings, 0 replies; 72+ messages in thread
From: Manikanta Maddireddy @ 2019-05-21  9:07 UTC (permalink / raw)
  To: Bjorn Helgaas
  Cc: thierry.reding, robh+dt, mark.rutland, jonathanh,
	lorenzo.pieralisi, vidyas, linux-tegra, linux-pci, devicetree



On 21-May-19 2:07 AM, Bjorn Helgaas wrote:
> On Thu, May 16, 2019 at 11:22:59AM +0530, Manikanta Maddireddy wrote:
>> Tegra signals PCIe services like AER, PME, etc. over legacy IRQ line.
>> By default, service drivers register interrupt routine over MSI IRQ line.
>> Use pcie_pme_disable_msi() function to disable MSI for service drivers.
> I think this device is not quite spec-compliant:
>
>   https://lore.kernel.org/linux-pci/20190520175729.GC49425@google.com/
>
> and you should work around this with a quirk that sets pdev->no_msi so
> we don't use MSI for it at all.

OK, I will update in next version. 

Manikanta

>
>> PME and AER interrupts registered to MSI without this change,
>> cat /proc/interrupts | grep -i pci
>> 36: 21 0 0 0 0 0 GICv2 104 Level       PCIE
>> 37: 35 0 0 0 0 0 GICv2 105 Level       Tegra PCIe MSI
>> 76: 0  0 0 0 0 0 Tegra PCIe MSI 0 Edge PCIe PME, aerdrv, PCIe BW notif
>>
>> PME and AER interrupts registered to legacy IRQ with this change,
>> cat /proc/interrupts | grep -i pci
>> 36: 33 0 0 0 0 0 GICv2 104 Level      PCIE, PCIe PME, aerdrv, PCIe BW notif
>> 37: 52 0 0 0 0 0 GICv2 105 Level      Tegra PCIe MSI
>>
>> Signed-off-by: Manikanta Maddireddy <mmaddireddy@nvidia.com>
>> Acked-by: Thierry Reding <treding@nvidia.com>
>> ---
>> V4: No change
>>
>> V3: Corrected typo in commit log
>>
>> V2: No change
>>
>>  drivers/pci/controller/pci-tegra.c | 4 ++++
>>  1 file changed, 4 insertions(+)
>>
>> diff --git a/drivers/pci/controller/pci-tegra.c b/drivers/pci/controller/pci-tegra.c
>> index ac57c5badd9b..0024bc42b400 100644
>> --- a/drivers/pci/controller/pci-tegra.c
>> +++ b/drivers/pci/controller/pci-tegra.c
>> @@ -41,6 +41,7 @@
>>  #include <soc/tegra/pmc.h>
>>  
>>  #include "../pci.h"
>> +#include "../pcie/portdrv.h"
>>  
>>  #define INT_PCI_MSI_NR (8 * 32)
>>  
>> @@ -2725,6 +2726,9 @@ static int tegra_pcie_probe(struct platform_device *pdev)
>>  		goto put_resources;
>>  	}
>>  
>> +	/* Switch to legacy IRQ for PCIe services like AER, PME*/
>> +	pcie_pme_disable_msi();
>> +
>>  	pm_runtime_enable(pcie->dev);
>>  	err = pm_runtime_get_sync(pcie->dev);
>>  	if (err) {
>> -- 
>> 2.17.1
>>


^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: [PATCH V4 04/28] PCI: tegra: Mask AFI_INTR in runtime suspend
  2019-05-16  5:52 ` [PATCH V4 04/28] PCI: tegra: Mask AFI_INTR in runtime suspend Manikanta Maddireddy
@ 2019-06-04 13:08   ` Thierry Reding
  0 siblings, 0 replies; 72+ messages in thread
From: Thierry Reding @ 2019-06-04 13:08 UTC (permalink / raw)
  To: Manikanta Maddireddy
  Cc: bhelgaas, robh+dt, mark.rutland, jonathanh, lorenzo.pieralisi,
	vidyas, linux-tegra, linux-pci, devicetree

[-- Attachment #1: Type: text/plain, Size: 5697 bytes --]

On Thu, May 16, 2019 at 11:22:43AM +0530, Manikanta Maddireddy wrote:
> AFI_INTR is unmasked in tegra_pcie_enable_controller(), mask it to avoid
> unwanted interrupts raised by AFI after pex_rst is asserted.
> 
> Following sequence triggers such scenario,
>  - tegra_pcie_remove() triggers runtime suspend
>  - pex_rst is asserted in runtime suspend
>  - PRSNT_MAP bit field in RP_PRIV_MISC register changes from EP_PRSNT to
>    EP_ABSNT
>  - This is sensed by AFI and triggers "Slot present pin change" interrupt
>  - tegra_pcie_isr() function accesses AFI register when runtime suspend
>    is going through power off sequence
> 
> rmmod pci-tegra
>  pci_generic_config_write32: 108 callbacks suppressed
>  pci_bus 0002:00: 2-byte config write to 0002:00:02.0 offset 0x4c may corrupt adjacent RW1C bits
>  pci_bus 0002:00: 2-byte config write to 0002:00:02.0 offset 0x9c may corrupt adjacent RW1C bits
>  pci_bus 0002:00: 2-byte config write to 0002:00:02.0 offset 0x88 may corrupt adjacent RW1C bits
>  pci_bus 0002:00: 2-byte config write to 0002:00:02.0 offset 0x90 may corrupt adjacent RW1C bits
>  pci_bus 0002:00: 2-byte config write to 0002:00:02.0 offset 0x4 may corrupt adjacent RW1C bits
>  igb 0002:04:00.1: removed PHC on enP2p4s0f1
>  igb 0002:04:00.0: removed PHC on enP2p4s0f0
>  pci_bus 0002:00: 2-byte config write to 0002:00:01.0 offset 0x4c may corrupt adjacent RW1C bits
>  pci_bus 0002:00: 2-byte config write to 0002:00:01.0 offset 0x9c may corrupt adjacent RW1C bits
>  pci_bus 0002:00: 2-byte config write to 0002:00:01.0 offset 0x88 may corrupt adjacent RW1C bits
>  pci_bus 0002:00: 2-byte config write to 0002:00:01.0 offset 0x90 may corrupt adjacent RW1C bits
>  pci_bus 0002:00: 2-byte config write to 0002:00:01.0 offset 0x4 may corrupt adjacent RW1C bits
>  rcu: INFO: rcu_preempt self-detected stall on CPU
>  SError Interrupt on CPU0, code 0xbf000002 -- SError
>  CPU: 0 PID: 0 Comm: swapper/0 Tainted: G        W         5.1.0-rc3-next-20190405-00027-gcd8110499e6f-dirty #42
>  Hardware name: NVIDIA Jetson TX1 Developer Kit (DT)
>  pstate: 20000085 (nzCv daIf -PAN -UAO)
>  pc : tegra_pcie_isr+0x58/0x178 [pci_tegra]
>  lr : tegra_pcie_isr+0x40/0x178 [pci_tegra]
>  sp : ffff000010003da0
>  x29: ffff000010003da0 x28: 0000000000000000
>  x27: ffff8000f9e61000 x26: ffff000010fbf420
>  x25: ffff000011427f93 x24: ffff8000fa600410
>  x23: ffff00001129d000 x22: ffff00001129d000
>  x21: ffff8000f18bf3c0 x20: 0000000000000070
>  x19: 00000000ffffffff x18: 0000000000000000
>  x17: 0000000000000000 x16: 0000000000000000
>  x15: 0000000000000000 x14: ffff000008d40a48
>  x13: ffff000008d40a30 x12: ffff000008d40a20
>  x11: ffff000008d40a10 x10: ffff000008d40a00
>  x9 : ffff000008d409e8 x8 : ffff000008d40ae8
>  x7 : ffff000008d40ad0 x6 : ffff000010003e58
>  x5 : ffff8000fac00248 x4 : 0000000000000000
>  x3 : ffff000008d40b08 x2 : fffffffffffffff8
>  x1 : ffff000008d3f4e8 x0 : 00000000ffffffff
>  Kernel panic - not syncing: Asynchronous SError Interrupt
>  CPU: 0 PID: 0 Comm: swapper/0 Tainted: G        W         5.1.0-rc3-next-20190405-00027-gcd8110499e6f-dirty #42
>  Hardware name: NVIDIA Jetson TX1 Developer Kit (DT)
>  Call trace:
>   dump_backtrace+0x0/0x158
>   show_stack+0x14/0x20
>   dump_stack+0xa8/0xcc
>   panic+0x140/0x2f4
>   nmi_panic+0x6c/0x70
>   arm64_serror_panic+0x74/0x80
>   __pte_error+0x0/0x28
>   el1_error+0x84/0xf8
>   tegra_pcie_isr+0x58/0x178 [pci_tegra]
>   __handle_irq_event_percpu+0x70/0x198
>   handle_irq_event_percpu+0x34/0x88
>   handle_irq_event+0x48/0x78
>   handle_fasteoi_irq+0xb4/0x190
>   generic_handle_irq+0x24/0x38
>   __handle_domain_irq+0x5c/0xb8
>   gic_handle_irq+0x58/0xa8
>   el1_irq+0xb8/0x180
>   cpuidle_enter_state+0x138/0x358
>   cpuidle_enter+0x18/0x20
>   call_cpuidle+0x1c/0x48
>   do_idle+0x230/0x2d0
>   cpu_startup_entry+0x20/0x28
>   rest_init+0xd4/0xe0
>   arch_call_rest_init+0xc/0x14
>   start_kernel+0x444/0x470
> 
> Signed-off-by: Manikanta Maddireddy <mmaddireddy@nvidia.com>
> ---
> V4: No change
> 
> V3:
> * Update the commit log and comment to reflect why this fix is required
> * MSI interrupt is not disabled
> 
> V2: This is new patch in V2
> 
>  drivers/pci/controller/pci-tegra.c | 14 ++++++++++++++
>  1 file changed, 14 insertions(+)
> 
> diff --git a/drivers/pci/controller/pci-tegra.c b/drivers/pci/controller/pci-tegra.c
> index bb3c0af9c830..d0e517f084a1 100644
> --- a/drivers/pci/controller/pci-tegra.c
> +++ b/drivers/pci/controller/pci-tegra.c
> @@ -1622,6 +1622,15 @@ static int tegra_pcie_disable_msi(struct tegra_pcie *pcie)
>  	return 0;
>  }
>  
> +static void tegra_pcie_disable_interrupts(struct tegra_pcie *pcie)
> +{
> +	u32 value;
> +
> +	value = afi_readl(pcie, AFI_INTR_MASK);
> +	value &= ~AFI_INTR_MASK_INT_MASK;
> +	afi_writel(pcie, value, AFI_INTR_MASK);
> +}
> +
>  static int tegra_pcie_get_xbar_config(struct tegra_pcie *pcie, u32 lanes,
>  				      u32 *xbar)
>  {
> @@ -2466,6 +2475,11 @@ static int __maybe_unused tegra_pcie_pm_suspend(struct device *dev)
>  		tegra_pcie_pme_turnoff(port);
>  
>  	tegra_pcie_disable_ports(pcie);
> +	/*

Use a blank line before block-style comments (and typically comments in
general) for better readability.

Other than than:

Acked-by: Thierry Reding <treding@nvidia.com>

> +	 * AFI_INTR is unmasked in tegra_pcie_enable_controller(), mask it to
> +	 * avoid unwanted interrupts raised by AFI after pex_rst is asserted.
> +	 */
> +	tegra_pcie_disable_interrupts(pcie);
>  
>  	if (pcie->soc->program_uphy) {
>  		err = tegra_pcie_phy_power_off(pcie);
> -- 
> 2.17.1
> 

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: [PATCH V4 22/28] PCI: tegra: Access endpoint config only if PCIe link is up
  2019-05-16  5:53 ` [PATCH V4 22/28] PCI: tegra: Access endpoint config only if PCIe link is up Manikanta Maddireddy
@ 2019-06-04 13:14   ` Thierry Reding
  2019-06-04 14:10     ` Manikanta Maddireddy
  0 siblings, 1 reply; 72+ messages in thread
From: Thierry Reding @ 2019-06-04 13:14 UTC (permalink / raw)
  To: Manikanta Maddireddy
  Cc: bhelgaas, robh+dt, mark.rutland, jonathanh, lorenzo.pieralisi,
	vidyas, linux-tegra, linux-pci, devicetree

[-- Attachment #1: Type: text/plain, Size: 4306 bytes --]

On Thu, May 16, 2019 at 11:23:01AM +0530, Manikanta Maddireddy wrote:
> Few endpoints like Wi-Fi supports power on/off and to leverage that
> root port must support hot-plug and hot-unplug. Tegra PCIe doesn't
> support hot-plug and hot-unplug, however it supports endpoint power
> on/off feature as follows,
>  - Power off sequence:
>    - Transition of PCIe link to L2
>    - Power off endpoint
>    - Leave root port in power up state with the link in L2
>  - Power on sequence:
>    - Power on endpoint
>    - Apply hot reset to get PCIe link up
> 
> PCIe client driver stops accessing PCIe endpoint config and BAR registers
> after endpoint is powered off. However, software applications like x11
> server or lspci can access endpoint config registers in which case
> host controller raises "response decoding" errors. To avoid this scenario,
> add PCIe link up check in config read and write callback functions before
> accessing endpoint config registers.
> 
> Signed-off-by: Manikanta Maddireddy <mmaddireddy@nvidia.com>
> ---
> V4: No change
> 
> V3: Update the commit log with explanation for the need of this patch
> 
> V2: Change tegra_pcie_link_status() to tegra_pcie_link_up()
> 
>  drivers/pci/controller/pci-tegra.c | 38 ++++++++++++++++++++++++++++++
>  1 file changed, 38 insertions(+)

This still doesn't look right to me conceptually. If somebody wants to
access the PCI devices after the kernel has powered them off, why can't
we just power the devices back on so that we allow userspace to properly
access the devices?

Or if that's not what we want, shouldn't we add something to the core
PCI infrastructure to let us deal with this? It seems like this is some
general problem that would apply to every PCI device and host bridge
driver. Having each driver implement this logic separately doesn't seem
like a good idea to me.

Thierry

> diff --git a/drivers/pci/controller/pci-tegra.c b/drivers/pci/controller/pci-tegra.c
> index d20c88a79e00..33f4dfab9e35 100644
> --- a/drivers/pci/controller/pci-tegra.c
> +++ b/drivers/pci/controller/pci-tegra.c
> @@ -428,6 +428,14 @@ static inline u32 pads_readl(struct tegra_pcie *pcie, unsigned long offset)
>  	return readl(pcie->pads + offset);
>  }
>  
> +static bool tegra_pcie_link_up(struct tegra_pcie_port *port)
> +{
> +	u32 value;
> +
> +	value = readl(port->base + RP_LINK_CONTROL_STATUS);
> +	return !!(value & RP_LINK_CONTROL_STATUS_DL_LINK_ACTIVE);
> +}
> +
>  /*
>   * The configuration space mapping on Tegra is somewhat similar to the ECAM
>   * defined by PCIe. However it deviates a bit in how the 4 bits for extended
> @@ -493,20 +501,50 @@ static void __iomem *tegra_pcie_map_bus(struct pci_bus *bus,
>  static int tegra_pcie_config_read(struct pci_bus *bus, unsigned int devfn,
>  				  int where, int size, u32 *value)
>  {
> +	struct tegra_pcie *pcie = bus->sysdata;
> +	struct pci_dev *bridge;
> +	struct tegra_pcie_port *port;
> +
>  	if (bus->number == 0)
>  		return pci_generic_config_read32(bus, devfn, where, size,
>  						 value);
>  
> +	bridge = pcie_find_root_port(bus->self);
> +
> +	list_for_each_entry(port, &pcie->ports, list)
> +		if (port->index + 1 == PCI_SLOT(bridge->devfn))
> +			break;
> +
> +	/* If there is no link, then there is no device */
> +	if (!tegra_pcie_link_up(port)) {
> +		*value = 0xffffffff;
> +		return PCIBIOS_DEVICE_NOT_FOUND;
> +	}
> +
>  	return pci_generic_config_read(bus, devfn, where, size, value);
>  }
>  
>  static int tegra_pcie_config_write(struct pci_bus *bus, unsigned int devfn,
>  				   int where, int size, u32 value)
>  {
> +	struct tegra_pcie *pcie = bus->sysdata;
> +	struct tegra_pcie_port *port;
> +	struct pci_dev *bridge;
> +
>  	if (bus->number == 0)
>  		return pci_generic_config_write32(bus, devfn, where, size,
>  						  value);
>  
> +	bridge = pcie_find_root_port(bus->self);
> +
> +	list_for_each_entry(port, &pcie->ports, list)
> +		if (port->index + 1 == PCI_SLOT(bridge->devfn))
> +			break;
> +
> +	/* If there is no link, then there is no device */
> +	if (!tegra_pcie_link_up(port))
> +		return PCIBIOS_DEVICE_NOT_FOUND;
> +
>  	return pci_generic_config_write(bus, devfn, where, size, value);
>  }
>  
> -- 
> 2.17.1
> 

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: [PATCH V4 27/28] PCI: tegra: Add support for GPIO based PERST#
  2019-05-16  5:53 ` [PATCH V4 27/28] PCI: tegra: Add support for GPIO based PERST# Manikanta Maddireddy
@ 2019-06-04 13:22   ` Thierry Reding
  2019-06-13 15:24     ` Lorenzo Pieralisi
  0 siblings, 1 reply; 72+ messages in thread
From: Thierry Reding @ 2019-06-04 13:22 UTC (permalink / raw)
  To: Manikanta Maddireddy
  Cc: bhelgaas, robh+dt, mark.rutland, jonathanh, lorenzo.pieralisi,
	vidyas, linux-tegra, linux-pci, devicetree

[-- Attachment #1: Type: text/plain, Size: 3968 bytes --]

On Thu, May 16, 2019 at 11:23:06AM +0530, Manikanta Maddireddy wrote:
> Add support for GPIO based PERST# signal. GPIO number comes from per port
> PCIe device tree node.
> 
> Signed-off-by: Manikanta Maddireddy <mmaddireddy@nvidia.com>
> ---
> V4: Using devm_gpiod_get_from_of_node() to get reset-gpios
> 
> V3: Using helper function to get reset-gpios
> 
> V2: Using standard "reset-gpio" property
> 
>  drivers/pci/controller/pci-tegra.c | 41 +++++++++++++++++++++++++-----
>  1 file changed, 35 insertions(+), 6 deletions(-)
> 
> diff --git a/drivers/pci/controller/pci-tegra.c b/drivers/pci/controller/pci-tegra.c
> index 06b99fcbf382..09b4d384ba38 100644
> --- a/drivers/pci/controller/pci-tegra.c
> +++ b/drivers/pci/controller/pci-tegra.c
> @@ -17,6 +17,7 @@
>  #include <linux/debugfs.h>
>  #include <linux/delay.h>
>  #include <linux/export.h>
> +#include <linux/gpio.h>
>  #include <linux/interrupt.h>
>  #include <linux/iopoll.h>
>  #include <linux/irq.h>
> @@ -400,6 +401,8 @@ struct tegra_pcie_port {
>  	unsigned int lanes;
>  
>  	struct phy **phys;
> +
> +	struct gpio_desc *reset_gpiod;

Nit: I'd leave away the "d" at the end there. Or perhaps even the _gpio
suffix entirely. But it's fine either way.

>  };
>  
>  struct tegra_pcie_bus {
> @@ -583,15 +586,23 @@ static void tegra_pcie_port_reset(struct tegra_pcie_port *port)
>  	unsigned long value;
>  
>  	/* pulse reset signal */
> -	value = afi_readl(port->pcie, ctrl);
> -	value &= ~AFI_PEX_CTRL_RST;
> -	afi_writel(port->pcie, value, ctrl);
> +	if (port->reset_gpiod) {
> +		gpiod_set_value(port->reset_gpiod, 0);

So is this actually deasserting the reset pin, or is it asserting a
low-active reset? I think it's the latter, because ...

> +	} else {
> +		value = afi_readl(port->pcie, ctrl);
> +		value &= ~AFI_PEX_CTRL_RST;
> +		afi_writel(port->pcie, value, ctrl);
> +	}
>  
>  	usleep_range(1000, 2000);
>  
> -	value = afi_readl(port->pcie, ctrl);
> -	value |= AFI_PEX_CTRL_RST;
> -	afi_writel(port->pcie, value, ctrl);
> +	if (port->reset_gpiod) {
> +		gpiod_set_value(port->reset_gpiod, 1);

After this the port should be functional, right? I think it'd be better
to reverse the logic here and move the polarity of the GPIO into device
tree. gpiod_set_value() takes care of inverting the level internally if
the GPIO is marked as low-active in DT.

The end result is obviously the same, but it makes the usage much
clearer. If somebody want to write a DT for their board, they will look
at the schematics and see a low-active reset line and may be tempted to
describe it as such in DT, but with your current code that would be
exactly the wrong way around.

> +	} else {
> +		value = afi_readl(port->pcie, ctrl);
> +		value |= AFI_PEX_CTRL_RST;
> +		afi_writel(port->pcie, value, ctrl);
> +	}
>  }
>  
>  static void tegra_pcie_enable_rp_features(struct tegra_pcie_port *port)
> @@ -2238,6 +2249,7 @@ static int tegra_pcie_parse_dt(struct tegra_pcie *pcie)
>  		struct tegra_pcie_port *rp;
>  		unsigned int index;
>  		u32 value;
> +		char *label;
>  
>  		err = of_pci_get_devfn(port);
>  		if (err < 0) {
> @@ -2296,6 +2308,23 @@ static int tegra_pcie_parse_dt(struct tegra_pcie *pcie)
>  		if (IS_ERR(rp->base))
>  			return PTR_ERR(rp->base);
>  
> +		label = kasprintf(GFP_KERNEL, "pex-reset-%u", index);

devm_kasprintf()?

Thierry

> +		if (!label) {
> +			dev_err(dev, "failed to create reset GPIO label\n");
> +			return -ENOMEM;
> +		}
> +
> +		rp->reset_gpiod = devm_gpiod_get_from_of_node(dev, port,
> +							      "reset-gpios", 0,
> +							      GPIOD_OUT_LOW,
> +							      label);
> +		kfree(label);
> +		if (IS_ERR(rp->reset_gpiod)) {
> +			err = PTR_ERR(rp->reset_gpiod);
> +			dev_err(dev, "failed to get reset GPIO: %d\n", err);
> +			return err;
> +		}
> +
>  		list_add_tail(&rp->list, &pcie->ports);
>  	}
>  
> -- 
> 2.17.1
> 

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: [PATCH V4 28/28] PCI: tegra: Change link retry log level to debug
  2019-05-16  5:53 ` [PATCH V4 28/28] PCI: tegra: Change link retry log level to debug Manikanta Maddireddy
@ 2019-06-04 13:22   ` Thierry Reding
  0 siblings, 0 replies; 72+ messages in thread
From: Thierry Reding @ 2019-06-04 13:22 UTC (permalink / raw)
  To: Manikanta Maddireddy
  Cc: bhelgaas, robh+dt, mark.rutland, jonathanh, lorenzo.pieralisi,
	vidyas, linux-tegra, linux-pci, devicetree

[-- Attachment #1: Type: text/plain, Size: 613 bytes --]

On Thu, May 16, 2019 at 11:23:07AM +0530, Manikanta Maddireddy wrote:
> Driver checks for link up three times before giving up, each retry attempt
> is printed as an error. Letting users know that PCIe link is down and in the
> process of being brought up again is for debug, not an error condition.
> 
> Signed-off-by: Manikanta Maddireddy <mmaddireddy@nvidia.com>
> ---
> V4: No change
> 
> V3: Changed dev_err to dev_dbg
> 
> V2: Updated commit log
> 
>  drivers/pci/controller/pci-tegra.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)

Acked-by: Thierry Reding <treding@nvidia.com>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: [PATCH V4 22/28] PCI: tegra: Access endpoint config only if PCIe link is up
  2019-06-04 13:14   ` Thierry Reding
@ 2019-06-04 14:10     ` Manikanta Maddireddy
  2019-06-10  4:38       ` Manikanta Maddireddy
  0 siblings, 1 reply; 72+ messages in thread
From: Manikanta Maddireddy @ 2019-06-04 14:10 UTC (permalink / raw)
  To: Thierry Reding
  Cc: bhelgaas, robh+dt, mark.rutland, jonathanh, lorenzo.pieralisi,
	vidyas, linux-tegra, linux-pci, devicetree



On 04-Jun-19 6:44 PM, Thierry Reding wrote:
> On Thu, May 16, 2019 at 11:23:01AM +0530, Manikanta Maddireddy wrote:
>> Few endpoints like Wi-Fi supports power on/off and to leverage that
>> root port must support hot-plug and hot-unplug. Tegra PCIe doesn't
>> support hot-plug and hot-unplug, however it supports endpoint power
>> on/off feature as follows,
>>  - Power off sequence:
>>    - Transition of PCIe link to L2
>>    - Power off endpoint
>>    - Leave root port in power up state with the link in L2
>>  - Power on sequence:
>>    - Power on endpoint
>>    - Apply hot reset to get PCIe link up
>>
>> PCIe client driver stops accessing PCIe endpoint config and BAR registers
>> after endpoint is powered off. However, software applications like x11
>> server or lspci can access endpoint config registers in which case
>> host controller raises "response decoding" errors. To avoid this scenario,
>> add PCIe link up check in config read and write callback functions before
>> accessing endpoint config registers.
>>
>> Signed-off-by: Manikanta Maddireddy <mmaddireddy@nvidia.com>
>> ---
>> V4: No change
>>
>> V3: Update the commit log with explanation for the need of this patch
>>
>> V2: Change tegra_pcie_link_status() to tegra_pcie_link_up()
>>
>>  drivers/pci/controller/pci-tegra.c | 38 ++++++++++++++++++++++++++++++
>>  1 file changed, 38 insertions(+)
> This still doesn't look right to me conceptually. If somebody wants to
> access the PCI devices after the kernel has powered them off, why can't
> we just power the devices back on so that we allow userspace to properly
> access the devices?

1. WiFi devices provides power-off feature for power saving in mobiles.
When WiFi is turned off we shouldn't power on the HW back without user
turning it back on.
2. When ever user process tries to access config space, it'll end up
in these functions. We cannot have is_powered_on check in config read/write
callbacks.
3. WiFi power on/off is device specific feature, we shouldn't handle it
in PCI subsystem or host controller driver.

>
> Or if that's not what we want, shouldn't we add something to the core
> PCI infrastructure to let us deal with this? It seems like this is some
> general problem that would apply to every PCI device and host bridge
> driver. Having each driver implement this logic separately doesn't seem
> like a good idea to me.
>
> Thierry

This should be handled by hotplug feature, whenever endpoint is powered-off/
removed from the slot, hot unplug event should take care of it. Unfortunately
Tegra PCIe doesn't support hotplug feature.

Manikanta

>> diff --git a/drivers/pci/controller/pci-tegra.c b/drivers/pci/controller/pci-tegra.c
>> index d20c88a79e00..33f4dfab9e35 100644
>> --- a/drivers/pci/controller/pci-tegra.c
>> +++ b/drivers/pci/controller/pci-tegra.c
>> @@ -428,6 +428,14 @@ static inline u32 pads_readl(struct tegra_pcie *pcie, unsigned long offset)
>>  	return readl(pcie->pads + offset);
>>  }
>>  
>> +static bool tegra_pcie_link_up(struct tegra_pcie_port *port)
>> +{
>> +	u32 value;
>> +
>> +	value = readl(port->base + RP_LINK_CONTROL_STATUS);
>> +	return !!(value & RP_LINK_CONTROL_STATUS_DL_LINK_ACTIVE);
>> +}
>> +
>>  /*
>>   * The configuration space mapping on Tegra is somewhat similar to the ECAM
>>   * defined by PCIe. However it deviates a bit in how the 4 bits for extended
>> @@ -493,20 +501,50 @@ static void __iomem *tegra_pcie_map_bus(struct pci_bus *bus,
>>  static int tegra_pcie_config_read(struct pci_bus *bus, unsigned int devfn,
>>  				  int where, int size, u32 *value)
>>  {
>> +	struct tegra_pcie *pcie = bus->sysdata;
>> +	struct pci_dev *bridge;
>> +	struct tegra_pcie_port *port;
>> +
>>  	if (bus->number == 0)
>>  		return pci_generic_config_read32(bus, devfn, where, size,
>>  						 value);
>>  
>> +	bridge = pcie_find_root_port(bus->self);
>> +
>> +	list_for_each_entry(port, &pcie->ports, list)
>> +		if (port->index + 1 == PCI_SLOT(bridge->devfn))
>> +			break;
>> +
>> +	/* If there is no link, then there is no device */
>> +	if (!tegra_pcie_link_up(port)) {
>> +		*value = 0xffffffff;
>> +		return PCIBIOS_DEVICE_NOT_FOUND;
>> +	}
>> +
>>  	return pci_generic_config_read(bus, devfn, where, size, value);
>>  }
>>  
>>  static int tegra_pcie_config_write(struct pci_bus *bus, unsigned int devfn,
>>  				   int where, int size, u32 value)
>>  {
>> +	struct tegra_pcie *pcie = bus->sysdata;
>> +	struct tegra_pcie_port *port;
>> +	struct pci_dev *bridge;
>> +
>>  	if (bus->number == 0)
>>  		return pci_generic_config_write32(bus, devfn, where, size,
>>  						  value);
>>  
>> +	bridge = pcie_find_root_port(bus->self);
>> +
>> +	list_for_each_entry(port, &pcie->ports, list)
>> +		if (port->index + 1 == PCI_SLOT(bridge->devfn))
>> +			break;
>> +
>> +	/* If there is no link, then there is no device */
>> +	if (!tegra_pcie_link_up(port))
>> +		return PCIBIOS_DEVICE_NOT_FOUND;
>> +
>>  	return pci_generic_config_write(bus, devfn, where, size, value);
>>  }
>>  
>> -- 
>> 2.17.1
>>


^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: [PATCH V4 22/28] PCI: tegra: Access endpoint config only if PCIe link is up
  2019-06-04 14:10     ` Manikanta Maddireddy
@ 2019-06-10  4:38       ` Manikanta Maddireddy
  2019-06-13 14:39         ` Lorenzo Pieralisi
  0 siblings, 1 reply; 72+ messages in thread
From: Manikanta Maddireddy @ 2019-06-10  4:38 UTC (permalink / raw)
  To: Thierry Reding, bhelgaas
  Cc: robh+dt, mark.rutland, jonathanh, lorenzo.pieralisi, vidyas,
	linux-tegra, linux-pci, devicetree



On 04-Jun-19 7:40 PM, Manikanta Maddireddy wrote:
>
> On 04-Jun-19 6:44 PM, Thierry Reding wrote:
>> On Thu, May 16, 2019 at 11:23:01AM +0530, Manikanta Maddireddy wrote:
>>> Few endpoints like Wi-Fi supports power on/off and to leverage that
>>> root port must support hot-plug and hot-unplug. Tegra PCIe doesn't
>>> support hot-plug and hot-unplug, however it supports endpoint power
>>> on/off feature as follows,
>>>  - Power off sequence:
>>>    - Transition of PCIe link to L2
>>>    - Power off endpoint
>>>    - Leave root port in power up state with the link in L2
>>>  - Power on sequence:
>>>    - Power on endpoint
>>>    - Apply hot reset to get PCIe link up
>>>
>>> PCIe client driver stops accessing PCIe endpoint config and BAR registers
>>> after endpoint is powered off. However, software applications like x11
>>> server or lspci can access endpoint config registers in which case
>>> host controller raises "response decoding" errors. To avoid this scenario,
>>> add PCIe link up check in config read and write callback functions before
>>> accessing endpoint config registers.
>>>
>>> Signed-off-by: Manikanta Maddireddy <mmaddireddy@nvidia.com>
>>> ---
>>> V4: No change
>>>
>>> V3: Update the commit log with explanation for the need of this patch
>>>
>>> V2: Change tegra_pcie_link_status() to tegra_pcie_link_up()
>>>
>>>  drivers/pci/controller/pci-tegra.c | 38 ++++++++++++++++++++++++++++++
>>>  1 file changed, 38 insertions(+)
>> This still doesn't look right to me conceptually. If somebody wants to
>> access the PCI devices after the kernel has powered them off, why can't
>> we just power the devices back on so that we allow userspace to properly
>> access the devices?
> 1. WiFi devices provides power-off feature for power saving in mobiles.
> When WiFi is turned off we shouldn't power on the HW back without user
> turning it back on.
> 2. When ever user process tries to access config space, it'll end up
> in these functions. We cannot have is_powered_on check in config read/write
> callbacks.
> 3. WiFi power on/off is device specific feature, we shouldn't handle it
> in PCI subsystem or host controller driver.
>
>> Or if that's not what we want, shouldn't we add something to the core
>> PCI infrastructure to let us deal with this? It seems like this is some
>> general problem that would apply to every PCI device and host bridge
>> driver. Having each driver implement this logic separately doesn't seem
>> like a good idea to me.
>>
>> Thierry
> This should be handled by hotplug feature, whenever endpoint is powered-off/
> removed from the slot, hot unplug event should take care of it. Unfortunately
> Tegra PCIe doesn't support hotplug feature.
>
> Manikanta

Hi Bjorn,

I thought about your comment in https://patchwork.ozlabs.org/patch/1084204/ again.
What if I add link up check in tegra_pcie_isr() and make "response decoding error"
as debug print? EP Config access will happen when link is down, but "Response
decoding error" print comes only if debug log is enabled. This way we can avoid
race issue in config accessors and we get prints when debug logs are enabled.

Thierry,
Please share your inputs as well.

Manikanta
 

>>> diff --git a/drivers/pci/controller/pci-tegra.c b/drivers/pci/controller/pci-tegra.c
>>> index d20c88a79e00..33f4dfab9e35 100644
>>> --- a/drivers/pci/controller/pci-tegra.c
>>> +++ b/drivers/pci/controller/pci-tegra.c
>>> @@ -428,6 +428,14 @@ static inline u32 pads_readl(struct tegra_pcie *pcie, unsigned long offset)
>>>  	return readl(pcie->pads + offset);
>>>  }
>>>  
>>> +static bool tegra_pcie_link_up(struct tegra_pcie_port *port)
>>> +{
>>> +	u32 value;
>>> +
>>> +	value = readl(port->base + RP_LINK_CONTROL_STATUS);
>>> +	return !!(value & RP_LINK_CONTROL_STATUS_DL_LINK_ACTIVE);
>>> +}
>>> +
>>>  /*
>>>   * The configuration space mapping on Tegra is somewhat similar to the ECAM
>>>   * defined by PCIe. However it deviates a bit in how the 4 bits for extended
>>> @@ -493,20 +501,50 @@ static void __iomem *tegra_pcie_map_bus(struct pci_bus *bus,
>>>  static int tegra_pcie_config_read(struct pci_bus *bus, unsigned int devfn,
>>>  				  int where, int size, u32 *value)
>>>  {
>>> +	struct tegra_pcie *pcie = bus->sysdata;
>>> +	struct pci_dev *bridge;
>>> +	struct tegra_pcie_port *port;
>>> +
>>>  	if (bus->number == 0)
>>>  		return pci_generic_config_read32(bus, devfn, where, size,
>>>  						 value);
>>>  
>>> +	bridge = pcie_find_root_port(bus->self);
>>> +
>>> +	list_for_each_entry(port, &pcie->ports, list)
>>> +		if (port->index + 1 == PCI_SLOT(bridge->devfn))
>>> +			break;
>>> +
>>> +	/* If there is no link, then there is no device */
>>> +	if (!tegra_pcie_link_up(port)) {
>>> +		*value = 0xffffffff;
>>> +		return PCIBIOS_DEVICE_NOT_FOUND;
>>> +	}
>>> +
>>>  	return pci_generic_config_read(bus, devfn, where, size, value);
>>>  }
>>>  
>>>  static int tegra_pcie_config_write(struct pci_bus *bus, unsigned int devfn,
>>>  				   int where, int size, u32 value)
>>>  {
>>> +	struct tegra_pcie *pcie = bus->sysdata;
>>> +	struct tegra_pcie_port *port;
>>> +	struct pci_dev *bridge;
>>> +
>>>  	if (bus->number == 0)
>>>  		return pci_generic_config_write32(bus, devfn, where, size,
>>>  						  value);
>>>  
>>> +	bridge = pcie_find_root_port(bus->self);
>>> +
>>> +	list_for_each_entry(port, &pcie->ports, list)
>>> +		if (port->index + 1 == PCI_SLOT(bridge->devfn))
>>> +			break;
>>> +
>>> +	/* If there is no link, then there is no device */
>>> +	if (!tegra_pcie_link_up(port))
>>> +		return PCIBIOS_DEVICE_NOT_FOUND;
>>> +
>>>  	return pci_generic_config_write(bus, devfn, where, size, value);
>>>  }
>>>  
>>> -- 
>>> 2.17.1
>>>


^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: [PATCH V4 00/28] Enable Tegra PCIe root port features
  2019-05-16  5:52 [PATCH V4 00/28] Enable Tegra PCIe root port features Manikanta Maddireddy
                   ` (28 preceding siblings ...)
  2019-05-16 13:12 ` [PATCH V4 00/28] Enable Tegra PCIe root port features Bjorn Helgaas
@ 2019-06-10  4:45 ` Manikanta Maddireddy
  2019-06-10 17:33   ` Lorenzo Pieralisi
  29 siblings, 1 reply; 72+ messages in thread
From: Manikanta Maddireddy @ 2019-06-10  4:45 UTC (permalink / raw)
  To: lorenzo.pieralisi
  Cc: thierry.reding, bhelgaas, robh+dt, mark.rutland, jonathanh,
	vidyas, linux-tegra, linux-pci, devicetree

Hi Lorenzo,

Thierry Ack'ed most of the patches, I am planning to address the review
comments for remaining two patches and publish V5. If you can review the
series, I will consolidate both the comments and address in V5.

Manikanta


On 16-May-19 11:22 AM, Manikanta Maddireddy wrote:
> This series of patches adds,
> - Tegra root port features like Gen2, AER, etc
> - Power and perf optimizations
> - Fixes like "power up sequence", "dev_err prints", etc
>
> This series of patches are tested on Tegra186 based Jetson-TX2, Tegra210
> based Jetson-TX1, T124 based Jetson-TK1 platforms, Tegra20 and Tegra30
> platforms.
>
> Manikanta Maddireddy (28):
>   soc/tegra: pmc: Export tegra_powergate_power_on()
>   PCI: tegra: Handle failure cases in tegra_pcie_power_on()
>   PCI: tegra: Rearrange Tegra PCIe driver functions
>   PCI: tegra: Mask AFI_INTR in runtime suspend
>   PCI: tegra: Fix PCIe host power up sequence
>   PCI: tegra: Add PCIe Gen2 link speed support
>   PCI: tegra: Advertise PCIe Advanced Error Reporting (AER) capability
>   PCI: tegra: Program UPHY electrical settings for Tegra210
>   PCI: tegra: Enable opportunistic UpdateFC and ACK
>   PCI: tegra: Disable AFI dynamic clock gating
>   PCI: tegra: Process pending DLL transactions before entering L1 or L2
>   PCI: tegra: Enable PCIe xclk clock clamping
>   PCI: tegra: Increase the deskew retry time
>   PCI: tegra: Add SW fixup for RAW violations
>   PCI: tegra: Update flow control timer frequency in Tegra210
>   PCI: tegra: Set target speed as Gen1 before starting LTSSM
>   PCI: tegra: Fix PLLE power down issue due to CLKREQ# signal
>   PCI: tegra: Program AFI_CACHE* registers only for Tegra20
>   PCI: tegra: Change PRSNT_SENSE IRQ log to debug
>   PCI: tegra: Use legacy IRQ for port service drivers
>   PCI: tegra: Add AFI_PEX2_CTRL reg offset as part of soc struct
>   PCI: tegra: Access endpoint config only if PCIe link is up
>   dt-bindings: pci: tegra: Document PCIe DPD pinctrl optional prop
>   arm64: tegra: Add PEX DPD states as pinctrl properties
>   PCI: tegra: Put PEX CLK & BIAS pads in DPD mode
>   PCI: Add DT binding for "reset-gpios" property
>   PCI: tegra: Add support for GPIO based PERST#
>   PCI: tegra: Change link retry log level to debug
>
>  .../bindings/pci/nvidia,tegra20-pcie.txt      |   8 +
>  Documentation/devicetree/bindings/pci/pci.txt |   3 +
>  arch/arm64/boot/dts/nvidia/tegra210.dtsi      |  19 +
>  drivers/pci/controller/pci-tegra.c            | 615 +++++++++++++++---
>  drivers/soc/tegra/pmc.c                       |   1 +
>  5 files changed, 566 insertions(+), 80 deletions(-)
>


^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: [PATCH V4 00/28] Enable Tegra PCIe root port features
  2019-06-10  4:45 ` Manikanta Maddireddy
@ 2019-06-10 17:33   ` Lorenzo Pieralisi
  0 siblings, 0 replies; 72+ messages in thread
From: Lorenzo Pieralisi @ 2019-06-10 17:33 UTC (permalink / raw)
  To: Manikanta Maddireddy
  Cc: thierry.reding, bhelgaas, robh+dt, mark.rutland, jonathanh,
	vidyas, linux-tegra, linux-pci, devicetree

On Mon, Jun 10, 2019 at 10:15:07AM +0530, Manikanta Maddireddy wrote:
> Hi Lorenzo,
> 
> Thierry Ack'ed most of the patches, I am planning to address the review
> comments for remaining two patches and publish V5. If you can review the
> series, I will consolidate both the comments and address in V5.

It will take me some time to get to this series but it is on my
radar, start preparing v5 but wait before posting it, I should
be able to comment shortly.

Lorenzo

> Manikanta
> 
> 
> On 16-May-19 11:22 AM, Manikanta Maddireddy wrote:
> > This series of patches adds,
> > - Tegra root port features like Gen2, AER, etc
> > - Power and perf optimizations
> > - Fixes like "power up sequence", "dev_err prints", etc
> >
> > This series of patches are tested on Tegra186 based Jetson-TX2, Tegra210
> > based Jetson-TX1, T124 based Jetson-TK1 platforms, Tegra20 and Tegra30
> > platforms.
> >
> > Manikanta Maddireddy (28):
> >   soc/tegra: pmc: Export tegra_powergate_power_on()
> >   PCI: tegra: Handle failure cases in tegra_pcie_power_on()
> >   PCI: tegra: Rearrange Tegra PCIe driver functions
> >   PCI: tegra: Mask AFI_INTR in runtime suspend
> >   PCI: tegra: Fix PCIe host power up sequence
> >   PCI: tegra: Add PCIe Gen2 link speed support
> >   PCI: tegra: Advertise PCIe Advanced Error Reporting (AER) capability
> >   PCI: tegra: Program UPHY electrical settings for Tegra210
> >   PCI: tegra: Enable opportunistic UpdateFC and ACK
> >   PCI: tegra: Disable AFI dynamic clock gating
> >   PCI: tegra: Process pending DLL transactions before entering L1 or L2
> >   PCI: tegra: Enable PCIe xclk clock clamping
> >   PCI: tegra: Increase the deskew retry time
> >   PCI: tegra: Add SW fixup for RAW violations
> >   PCI: tegra: Update flow control timer frequency in Tegra210
> >   PCI: tegra: Set target speed as Gen1 before starting LTSSM
> >   PCI: tegra: Fix PLLE power down issue due to CLKREQ# signal
> >   PCI: tegra: Program AFI_CACHE* registers only for Tegra20
> >   PCI: tegra: Change PRSNT_SENSE IRQ log to debug
> >   PCI: tegra: Use legacy IRQ for port service drivers
> >   PCI: tegra: Add AFI_PEX2_CTRL reg offset as part of soc struct
> >   PCI: tegra: Access endpoint config only if PCIe link is up
> >   dt-bindings: pci: tegra: Document PCIe DPD pinctrl optional prop
> >   arm64: tegra: Add PEX DPD states as pinctrl properties
> >   PCI: tegra: Put PEX CLK & BIAS pads in DPD mode
> >   PCI: Add DT binding for "reset-gpios" property
> >   PCI: tegra: Add support for GPIO based PERST#
> >   PCI: tegra: Change link retry log level to debug
> >
> >  .../bindings/pci/nvidia,tegra20-pcie.txt      |   8 +
> >  Documentation/devicetree/bindings/pci/pci.txt |   3 +
> >  arch/arm64/boot/dts/nvidia/tegra210.dtsi      |  19 +
> >  drivers/pci/controller/pci-tegra.c            | 615 +++++++++++++++---
> >  drivers/soc/tegra/pmc.c                       |   1 +
> >  5 files changed, 566 insertions(+), 80 deletions(-)
> >
> 

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: [PATCH V4 22/28] PCI: tegra: Access endpoint config only if PCIe link is up
  2019-06-10  4:38       ` Manikanta Maddireddy
@ 2019-06-13 14:39         ` Lorenzo Pieralisi
  2019-06-13 15:42           ` Thierry Reding
  0 siblings, 1 reply; 72+ messages in thread
From: Lorenzo Pieralisi @ 2019-06-13 14:39 UTC (permalink / raw)
  To: Manikanta Maddireddy
  Cc: Thierry Reding, bhelgaas, robh+dt, mark.rutland, jonathanh,
	vidyas, linux-tegra, linux-pci, devicetree

On Mon, Jun 10, 2019 at 10:08:16AM +0530, Manikanta Maddireddy wrote:
> 
> 
> On 04-Jun-19 7:40 PM, Manikanta Maddireddy wrote:
> >
> > On 04-Jun-19 6:44 PM, Thierry Reding wrote:
> >> On Thu, May 16, 2019 at 11:23:01AM +0530, Manikanta Maddireddy wrote:
> >>> Few endpoints like Wi-Fi supports power on/off and to leverage that
> >>> root port must support hot-plug and hot-unplug. Tegra PCIe doesn't
> >>> support hot-plug and hot-unplug, however it supports endpoint power
> >>> on/off feature as follows,
> >>>  - Power off sequence:
> >>>    - Transition of PCIe link to L2
> >>>    - Power off endpoint
> >>>    - Leave root port in power up state with the link in L2
> >>>  - Power on sequence:
> >>>    - Power on endpoint
> >>>    - Apply hot reset to get PCIe link up
> >>>
> >>> PCIe client driver stops accessing PCIe endpoint config and BAR registers
> >>> after endpoint is powered off. However, software applications like x11
> >>> server or lspci can access endpoint config registers in which case
> >>> host controller raises "response decoding" errors. To avoid this scenario,
> >>> add PCIe link up check in config read and write callback functions before
> >>> accessing endpoint config registers.
> >>>
> >>> Signed-off-by: Manikanta Maddireddy <mmaddireddy@nvidia.com>
> >>> ---
> >>> V4: No change
> >>>
> >>> V3: Update the commit log with explanation for the need of this patch
> >>>
> >>> V2: Change tegra_pcie_link_status() to tegra_pcie_link_up()
> >>>
> >>>  drivers/pci/controller/pci-tegra.c | 38 ++++++++++++++++++++++++++++++
> >>>  1 file changed, 38 insertions(+)
> >> This still doesn't look right to me conceptually. If somebody wants to
> >> access the PCI devices after the kernel has powered them off, why can't
> >> we just power the devices back on so that we allow userspace to properly
> >> access the devices?
> > 1. WiFi devices provides power-off feature for power saving in mobiles.
> > When WiFi is turned off we shouldn't power on the HW back without user
> > turning it back on.
> > 2. When ever user process tries to access config space, it'll end up
> > in these functions. We cannot have is_powered_on check in config read/write
> > callbacks.
> > 3. WiFi power on/off is device specific feature, we shouldn't handle it
> > in PCI subsystem or host controller driver.
> >
> >> Or if that's not what we want, shouldn't we add something to the core
> >> PCI infrastructure to let us deal with this? It seems like this is some
> >> general problem that would apply to every PCI device and host bridge
> >> driver. Having each driver implement this logic separately doesn't seem
> >> like a good idea to me.
> >>
> >> Thierry
> > This should be handled by hotplug feature, whenever endpoint is powered-off/
> > removed from the slot, hot unplug event should take care of it. Unfortunately
> > Tegra PCIe doesn't support hotplug feature.
> >
> > Manikanta
> 
> Hi Bjorn,
> 
> I thought about your comment in
> https://patchwork.ozlabs.org/patch/1084204/ again.  What if I add link
> up check in tegra_pcie_isr() and make "response decoding error" as
> debug print? EP Config access will happen when link is down, but
> "Response decoding error" print comes only if debug log is enabled.
> This way we can avoid race issue in config accessors and we get prints
> when debug logs are enabled.

I still do not see what you are actually solving. This patch should
be dropped.

Thanks,
Lorenzo

> Thierry,
> Please share your inputs as well.
> 
> Manikanta
>  
> 
> >>> diff --git a/drivers/pci/controller/pci-tegra.c b/drivers/pci/controller/pci-tegra.c
> >>> index d20c88a79e00..33f4dfab9e35 100644
> >>> --- a/drivers/pci/controller/pci-tegra.c
> >>> +++ b/drivers/pci/controller/pci-tegra.c
> >>> @@ -428,6 +428,14 @@ static inline u32 pads_readl(struct tegra_pcie *pcie, unsigned long offset)
> >>>  	return readl(pcie->pads + offset);
> >>>  }
> >>>  
> >>> +static bool tegra_pcie_link_up(struct tegra_pcie_port *port)
> >>> +{
> >>> +	u32 value;
> >>> +
> >>> +	value = readl(port->base + RP_LINK_CONTROL_STATUS);
> >>> +	return !!(value & RP_LINK_CONTROL_STATUS_DL_LINK_ACTIVE);
> >>> +}
> >>> +
> >>>  /*
> >>>   * The configuration space mapping on Tegra is somewhat similar to the ECAM
> >>>   * defined by PCIe. However it deviates a bit in how the 4 bits for extended
> >>> @@ -493,20 +501,50 @@ static void __iomem *tegra_pcie_map_bus(struct pci_bus *bus,
> >>>  static int tegra_pcie_config_read(struct pci_bus *bus, unsigned int devfn,
> >>>  				  int where, int size, u32 *value)
> >>>  {
> >>> +	struct tegra_pcie *pcie = bus->sysdata;
> >>> +	struct pci_dev *bridge;
> >>> +	struct tegra_pcie_port *port;
> >>> +
> >>>  	if (bus->number == 0)
> >>>  		return pci_generic_config_read32(bus, devfn, where, size,
> >>>  						 value);
> >>>  
> >>> +	bridge = pcie_find_root_port(bus->self);
> >>> +
> >>> +	list_for_each_entry(port, &pcie->ports, list)
> >>> +		if (port->index + 1 == PCI_SLOT(bridge->devfn))
> >>> +			break;
> >>> +
> >>> +	/* If there is no link, then there is no device */
> >>> +	if (!tegra_pcie_link_up(port)) {
> >>> +		*value = 0xffffffff;
> >>> +		return PCIBIOS_DEVICE_NOT_FOUND;
> >>> +	}
> >>> +
> >>>  	return pci_generic_config_read(bus, devfn, where, size, value);
> >>>  }
> >>>  
> >>>  static int tegra_pcie_config_write(struct pci_bus *bus, unsigned int devfn,
> >>>  				   int where, int size, u32 value)
> >>>  {
> >>> +	struct tegra_pcie *pcie = bus->sysdata;
> >>> +	struct tegra_pcie_port *port;
> >>> +	struct pci_dev *bridge;
> >>> +
> >>>  	if (bus->number == 0)
> >>>  		return pci_generic_config_write32(bus, devfn, where, size,
> >>>  						  value);
> >>>  
> >>> +	bridge = pcie_find_root_port(bus->self);
> >>> +
> >>> +	list_for_each_entry(port, &pcie->ports, list)
> >>> +		if (port->index + 1 == PCI_SLOT(bridge->devfn))
> >>> +			break;
> >>> +
> >>> +	/* If there is no link, then there is no device */
> >>> +	if (!tegra_pcie_link_up(port))
> >>> +		return PCIBIOS_DEVICE_NOT_FOUND;
> >>> +
> >>>  	return pci_generic_config_write(bus, devfn, where, size, value);
> >>>  }
> >>>  
> >>> -- 
> >>> 2.17.1
> >>>
> 

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: [PATCH V4 27/28] PCI: tegra: Add support for GPIO based PERST#
  2019-06-04 13:22   ` Thierry Reding
@ 2019-06-13 15:24     ` Lorenzo Pieralisi
  2019-06-14 10:37       ` Manikanta Maddireddy
  0 siblings, 1 reply; 72+ messages in thread
From: Lorenzo Pieralisi @ 2019-06-13 15:24 UTC (permalink / raw)
  To: Thierry Reding
  Cc: Manikanta Maddireddy, bhelgaas, robh+dt, mark.rutland, jonathanh,
	vidyas, linux-tegra, linux-pci, devicetree

On Tue, Jun 04, 2019 at 03:22:33PM +0200, Thierry Reding wrote:

[...]

> > +	} else {
> > +		value = afi_readl(port->pcie, ctrl);
> > +		value &= ~AFI_PEX_CTRL_RST;
> > +		afi_writel(port->pcie, value, ctrl);
> > +	}
> >  
> >  	usleep_range(1000, 2000);
> >  
> > -	value = afi_readl(port->pcie, ctrl);
> > -	value |= AFI_PEX_CTRL_RST;
> > -	afi_writel(port->pcie, value, ctrl);
> > +	if (port->reset_gpiod) {
> > +		gpiod_set_value(port->reset_gpiod, 1);
> 
> After this the port should be functional, right? I think it'd be better
> to reverse the logic here and move the polarity of the GPIO into device
> tree. gpiod_set_value() takes care of inverting the level internally if
> the GPIO is marked as low-active in DT.
> 
> The end result is obviously the same, but it makes the usage much
> clearer. If somebody want to write a DT for their board, they will look
> at the schematics and see a low-active reset line and may be tempted to
> describe it as such in DT, but with your current code that would be
> exactly the wrong way around.

I agree with Thierry here, you should change the logic.

Question: what's the advantage of adding GPIO reset support if that's
architected already in port registers ? I am pretty sure there is a
reason behind it (and forgive me the dumb question) and I would like to
have it written in the commit log.

Thanks,
Lorenzo

> > +	} else {
> > +		value = afi_readl(port->pcie, ctrl);
> > +		value |= AFI_PEX_CTRL_RST;
> > +		afi_writel(port->pcie, value, ctrl);
> > +	}
> >  }
> >  
> >  static void tegra_pcie_enable_rp_features(struct tegra_pcie_port *port)
> > @@ -2238,6 +2249,7 @@ static int tegra_pcie_parse_dt(struct tegra_pcie *pcie)
> >  		struct tegra_pcie_port *rp;
> >  		unsigned int index;
> >  		u32 value;
> > +		char *label;
> >  
> >  		err = of_pci_get_devfn(port);
> >  		if (err < 0) {
> > @@ -2296,6 +2308,23 @@ static int tegra_pcie_parse_dt(struct tegra_pcie *pcie)
> >  		if (IS_ERR(rp->base))
> >  			return PTR_ERR(rp->base);
> >  
> > +		label = kasprintf(GFP_KERNEL, "pex-reset-%u", index);
> 
> devm_kasprintf()?
> 
> Thierry
> 
> > +		if (!label) {
> > +			dev_err(dev, "failed to create reset GPIO label\n");
> > +			return -ENOMEM;
> > +		}
> > +
> > +		rp->reset_gpiod = devm_gpiod_get_from_of_node(dev, port,
> > +							      "reset-gpios", 0,
> > +							      GPIOD_OUT_LOW,
> > +							      label);
> > +		kfree(label);
> > +		if (IS_ERR(rp->reset_gpiod)) {
> > +			err = PTR_ERR(rp->reset_gpiod);
> > +			dev_err(dev, "failed to get reset GPIO: %d\n", err);
> > +			return err;
> > +		}
> > +
> >  		list_add_tail(&rp->list, &pcie->ports);
> >  	}
> >  
> > -- 
> > 2.17.1
> > 



^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: [PATCH V4 22/28] PCI: tegra: Access endpoint config only if PCIe link is up
  2019-06-13 14:39         ` Lorenzo Pieralisi
@ 2019-06-13 15:42           ` Thierry Reding
  2019-06-17 10:01             ` Manikanta Maddireddy
  0 siblings, 1 reply; 72+ messages in thread
From: Thierry Reding @ 2019-06-13 15:42 UTC (permalink / raw)
  To: Lorenzo Pieralisi
  Cc: Manikanta Maddireddy, bhelgaas, robh+dt, mark.rutland, jonathanh,
	vidyas, linux-tegra, linux-pci, devicetree

[-- Attachment #1: Type: text/plain, Size: 9257 bytes --]

On Thu, Jun 13, 2019 at 03:39:46PM +0100, Lorenzo Pieralisi wrote:
> On Mon, Jun 10, 2019 at 10:08:16AM +0530, Manikanta Maddireddy wrote:
> > 
> > 
> > On 04-Jun-19 7:40 PM, Manikanta Maddireddy wrote:
> > >
> > > On 04-Jun-19 6:44 PM, Thierry Reding wrote:
> > >> On Thu, May 16, 2019 at 11:23:01AM +0530, Manikanta Maddireddy wrote:
> > >>> Few endpoints like Wi-Fi supports power on/off and to leverage that
> > >>> root port must support hot-plug and hot-unplug. Tegra PCIe doesn't
> > >>> support hot-plug and hot-unplug, however it supports endpoint power
> > >>> on/off feature as follows,
> > >>>  - Power off sequence:
> > >>>    - Transition of PCIe link to L2
> > >>>    - Power off endpoint
> > >>>    - Leave root port in power up state with the link in L2
> > >>>  - Power on sequence:
> > >>>    - Power on endpoint
> > >>>    - Apply hot reset to get PCIe link up
> > >>>
> > >>> PCIe client driver stops accessing PCIe endpoint config and BAR registers
> > >>> after endpoint is powered off. However, software applications like x11
> > >>> server or lspci can access endpoint config registers in which case
> > >>> host controller raises "response decoding" errors. To avoid this scenario,
> > >>> add PCIe link up check in config read and write callback functions before
> > >>> accessing endpoint config registers.
> > >>>
> > >>> Signed-off-by: Manikanta Maddireddy <mmaddireddy@nvidia.com>
> > >>> ---
> > >>> V4: No change
> > >>>
> > >>> V3: Update the commit log with explanation for the need of this patch
> > >>>
> > >>> V2: Change tegra_pcie_link_status() to tegra_pcie_link_up()
> > >>>
> > >>>  drivers/pci/controller/pci-tegra.c | 38 ++++++++++++++++++++++++++++++
> > >>>  1 file changed, 38 insertions(+)
> > >> This still doesn't look right to me conceptually. If somebody wants to
> > >> access the PCI devices after the kernel has powered them off, why can't
> > >> we just power the devices back on so that we allow userspace to properly
> > >> access the devices?
> > > 1. WiFi devices provides power-off feature for power saving in mobiles.
> > > When WiFi is turned off we shouldn't power on the HW back without user
> > > turning it back on.
> > > 2. When ever user process tries to access config space, it'll end up
> > > in these functions. We cannot have is_powered_on check in config read/write
> > > callbacks.
> > > 3. WiFi power on/off is device specific feature, we shouldn't handle it
> > > in PCI subsystem or host controller driver.
> > >
> > >> Or if that's not what we want, shouldn't we add something to the core
> > >> PCI infrastructure to let us deal with this? It seems like this is some
> > >> general problem that would apply to every PCI device and host bridge
> > >> driver. Having each driver implement this logic separately doesn't seem
> > >> like a good idea to me.
> > >>
> > >> Thierry
> > > This should be handled by hotplug feature, whenever endpoint is powered-off/
> > > removed from the slot, hot unplug event should take care of it. Unfortunately
> > > Tegra PCIe doesn't support hotplug feature.
> > >
> > > Manikanta
> > 
> > Hi Bjorn,
> > 
> > I thought about your comment in
> > https://patchwork.ozlabs.org/patch/1084204/ again.  What if I add link
> > up check in tegra_pcie_isr() and make "response decoding error" as
> > debug print? EP Config access will happen when link is down, but
> > "Response decoding error" print comes only if debug log is enabled.
> > This way we can avoid race issue in config accessors and we get prints
> > when debug logs are enabled.
> 
> I still do not see what you are actually solving. This patch should
> be dropped.

The problem that Manikanta is trying to solve here occurs in this
situation (Manikanta, correct me if I've got this wrong): on some
setups, a WiFi module connected over PCI will toggle a power GPIO as
part of runtime suspend. This effectively causes the module to disappear
from the PCI bus (i.e. it can no longer be accessed until the power GPIO
is toggled again).

This is fine from a kernel point of view because the kernel keeps track
of what devices are suspended. However, userspace will occasionally try
to read the configuration space access of all devices, and since it
doesn't have any knowledge about the suspend state of these devices, it
doesn't know which ones to leave alone. I think this happens when the
X.Org server is running.

One thing that Manikanta and I had discussed was that perhaps the device
should be hot-unplugged when it goes into this low-power state. However,
we don't support hotplug on Tegra210 where this is needed, so we'd need
some sort of software-induced hot-unplug. However, this low power state
is entered when the WiFi interface is taken down (i.e. ip link set dev
<interface> down). If we were to remove the PCI device in that case, it
means that the interface goes away completely, which is completely
unexpected from a user's perspective. After all, taking a link down and
up may be something that scripts are doing all the time. They'd fall
over if after taking the interface down, the interface completely
disappears.

It's also not entirely clear to me how we get the device back onto the
bus again after it is in low power. If we hot-unplug the device, then
the driver will be unbound. Presumably the driver is what's controlling
the power GPIO, so there won't be any entity that can be used to bring
the chip back to life. Unless we deal with that power GPIO elsewhere
(rfkill switch perhaps?).

Perhaps one other way to deal with this would be to track the suspend
state of devices and then have the code that implements the PCI access
from userspace refuse accesses to devices that are asleep. I suppose
this is somewhat of an odd use-case because traditionally I guess PCI
devices never power down to a state where their configuration space can
no longer be accessed. At least that's what would explain why this has
never been an issue before. Or perhaps it has?

The last resort would be to just never put the WiFi chip into that low
power mode, though I'm not exactly sure what that means for the power
consumption on the affected systems.

Manikanta, can you fill in some of the blanks above?

Thierry

> > Thierry,
> > Please share your inputs as well.
> > 
> > Manikanta
> >  
> > 
> > >>> diff --git a/drivers/pci/controller/pci-tegra.c b/drivers/pci/controller/pci-tegra.c
> > >>> index d20c88a79e00..33f4dfab9e35 100644
> > >>> --- a/drivers/pci/controller/pci-tegra.c
> > >>> +++ b/drivers/pci/controller/pci-tegra.c
> > >>> @@ -428,6 +428,14 @@ static inline u32 pads_readl(struct tegra_pcie *pcie, unsigned long offset)
> > >>>  	return readl(pcie->pads + offset);
> > >>>  }
> > >>>  
> > >>> +static bool tegra_pcie_link_up(struct tegra_pcie_port *port)
> > >>> +{
> > >>> +	u32 value;
> > >>> +
> > >>> +	value = readl(port->base + RP_LINK_CONTROL_STATUS);
> > >>> +	return !!(value & RP_LINK_CONTROL_STATUS_DL_LINK_ACTIVE);
> > >>> +}
> > >>> +
> > >>>  /*
> > >>>   * The configuration space mapping on Tegra is somewhat similar to the ECAM
> > >>>   * defined by PCIe. However it deviates a bit in how the 4 bits for extended
> > >>> @@ -493,20 +501,50 @@ static void __iomem *tegra_pcie_map_bus(struct pci_bus *bus,
> > >>>  static int tegra_pcie_config_read(struct pci_bus *bus, unsigned int devfn,
> > >>>  				  int where, int size, u32 *value)
> > >>>  {
> > >>> +	struct tegra_pcie *pcie = bus->sysdata;
> > >>> +	struct pci_dev *bridge;
> > >>> +	struct tegra_pcie_port *port;
> > >>> +
> > >>>  	if (bus->number == 0)
> > >>>  		return pci_generic_config_read32(bus, devfn, where, size,
> > >>>  						 value);
> > >>>  
> > >>> +	bridge = pcie_find_root_port(bus->self);
> > >>> +
> > >>> +	list_for_each_entry(port, &pcie->ports, list)
> > >>> +		if (port->index + 1 == PCI_SLOT(bridge->devfn))
> > >>> +			break;
> > >>> +
> > >>> +	/* If there is no link, then there is no device */
> > >>> +	if (!tegra_pcie_link_up(port)) {
> > >>> +		*value = 0xffffffff;
> > >>> +		return PCIBIOS_DEVICE_NOT_FOUND;
> > >>> +	}
> > >>> +
> > >>>  	return pci_generic_config_read(bus, devfn, where, size, value);
> > >>>  }
> > >>>  
> > >>>  static int tegra_pcie_config_write(struct pci_bus *bus, unsigned int devfn,
> > >>>  				   int where, int size, u32 value)
> > >>>  {
> > >>> +	struct tegra_pcie *pcie = bus->sysdata;
> > >>> +	struct tegra_pcie_port *port;
> > >>> +	struct pci_dev *bridge;
> > >>> +
> > >>>  	if (bus->number == 0)
> > >>>  		return pci_generic_config_write32(bus, devfn, where, size,
> > >>>  						  value);
> > >>>  
> > >>> +	bridge = pcie_find_root_port(bus->self);
> > >>> +
> > >>> +	list_for_each_entry(port, &pcie->ports, list)
> > >>> +		if (port->index + 1 == PCI_SLOT(bridge->devfn))
> > >>> +			break;
> > >>> +
> > >>> +	/* If there is no link, then there is no device */
> > >>> +	if (!tegra_pcie_link_up(port))
> > >>> +		return PCIBIOS_DEVICE_NOT_FOUND;
> > >>> +
> > >>>  	return pci_generic_config_write(bus, devfn, where, size, value);
> > >>>  }
> > >>>  
> > >>> -- 
> > >>> 2.17.1
> > >>>
> > 

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: [PATCH V4 27/28] PCI: tegra: Add support for GPIO based PERST#
  2019-06-13 15:24     ` Lorenzo Pieralisi
@ 2019-06-14 10:37       ` Manikanta Maddireddy
  2019-06-14 14:32         ` Lorenzo Pieralisi
  0 siblings, 1 reply; 72+ messages in thread
From: Manikanta Maddireddy @ 2019-06-14 10:37 UTC (permalink / raw)
  To: Lorenzo Pieralisi, Thierry Reding
  Cc: bhelgaas, robh+dt, mark.rutland, jonathanh, vidyas, linux-tegra,
	linux-pci, devicetree



On 13-Jun-19 8:54 PM, Lorenzo Pieralisi wrote:
> On Tue, Jun 04, 2019 at 03:22:33PM +0200, Thierry Reding wrote:
>
> [...]
>
>>> +	} else {
>>> +		value = afi_readl(port->pcie, ctrl);
>>> +		value &= ~AFI_PEX_CTRL_RST;
>>> +		afi_writel(port->pcie, value, ctrl);
>>> +	}
>>>  
>>>  	usleep_range(1000, 2000);
>>>  
>>> -	value = afi_readl(port->pcie, ctrl);
>>> -	value |= AFI_PEX_CTRL_RST;
>>> -	afi_writel(port->pcie, value, ctrl);
>>> +	if (port->reset_gpiod) {
>>> +		gpiod_set_value(port->reset_gpiod, 1);
>> After this the port should be functional, right? I think it'd be better
>> to reverse the logic here and move the polarity of the GPIO into device
>> tree. gpiod_set_value() takes care of inverting the level internally if
>> the GPIO is marked as low-active in DT.
>>
>> The end result is obviously the same, but it makes the usage much
>> clearer. If somebody want to write a DT for their board, they will look
>> at the schematics and see a low-active reset line and may be tempted to
>> describe it as such in DT, but with your current code that would be
>> exactly the wrong way around.
> I agree with Thierry here, you should change the logic.
>
> Question: what's the advantage of adding GPIO reset support if that's
> architected already in port registers ? I am pretty sure there is a
> reason behind it (and forgive me the dumb question) and I would like to
> have it written in the commit log.
>
> Thanks,
> Lorenzo

Each PCIe controller has a dedicated SFIO pin to support PERST# signal. Port register
can control only this particular SFIO pin. However, in one of the Nvidia platform,
instead of using PCIe SFIO pin, different gpio is routed PCIe slot. This happened
because of a confusion in IO ball naming convention. To support this particular
platform, driver has provide gpio support. I will update the commit log in V5.

Manikanta

>
>>> +	} else {
>>> +		value = afi_readl(port->pcie, ctrl);
>>> +		value |= AFI_PEX_CTRL_RST;
>>> +		afi_writel(port->pcie, value, ctrl);
>>> +	}
>>>  }
>>>  
>>>  static void tegra_pcie_enable_rp_features(struct tegra_pcie_port *port)
>>> @@ -2238,6 +2249,7 @@ static int tegra_pcie_parse_dt(struct tegra_pcie *pcie)
>>>  		struct tegra_pcie_port *rp;
>>>  		unsigned int index;
>>>  		u32 value;
>>> +		char *label;
>>>  
>>>  		err = of_pci_get_devfn(port);
>>>  		if (err < 0) {
>>> @@ -2296,6 +2308,23 @@ static int tegra_pcie_parse_dt(struct tegra_pcie *pcie)
>>>  		if (IS_ERR(rp->base))
>>>  			return PTR_ERR(rp->base);
>>>  
>>> +		label = kasprintf(GFP_KERNEL, "pex-reset-%u", index);
>> devm_kasprintf()?
>>
>> Thierry
>>
>>> +		if (!label) {
>>> +			dev_err(dev, "failed to create reset GPIO label\n");
>>> +			return -ENOMEM;
>>> +		}
>>> +
>>> +		rp->reset_gpiod = devm_gpiod_get_from_of_node(dev, port,
>>> +							      "reset-gpios", 0,
>>> +							      GPIOD_OUT_LOW,
>>> +							      label);
>>> +		kfree(label);
>>> +		if (IS_ERR(rp->reset_gpiod)) {
>>> +			err = PTR_ERR(rp->reset_gpiod);
>>> +			dev_err(dev, "failed to get reset GPIO: %d\n", err);
>>> +			return err;
>>> +		}
>>> +
>>>  		list_add_tail(&rp->list, &pcie->ports);
>>>  	}
>>>  
>>> -- 
>>> 2.17.1
>>>
>


^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: [PATCH V4 27/28] PCI: tegra: Add support for GPIO based PERST#
  2019-06-14 10:37       ` Manikanta Maddireddy
@ 2019-06-14 14:32         ` Lorenzo Pieralisi
  2019-06-14 14:38           ` Manikanta Maddireddy
  0 siblings, 1 reply; 72+ messages in thread
From: Lorenzo Pieralisi @ 2019-06-14 14:32 UTC (permalink / raw)
  To: Manikanta Maddireddy
  Cc: Thierry Reding, bhelgaas, robh+dt, mark.rutland, jonathanh,
	vidyas, linux-tegra, linux-pci, devicetree

On Fri, Jun 14, 2019 at 04:07:35PM +0530, Manikanta Maddireddy wrote:
> 
> 
> On 13-Jun-19 8:54 PM, Lorenzo Pieralisi wrote:
> > On Tue, Jun 04, 2019 at 03:22:33PM +0200, Thierry Reding wrote:
> >
> > [...]
> >
> >>> +	} else {
> >>> +		value = afi_readl(port->pcie, ctrl);
> >>> +		value &= ~AFI_PEX_CTRL_RST;
> >>> +		afi_writel(port->pcie, value, ctrl);
> >>> +	}
> >>>  
> >>>  	usleep_range(1000, 2000);
> >>>  
> >>> -	value = afi_readl(port->pcie, ctrl);
> >>> -	value |= AFI_PEX_CTRL_RST;
> >>> -	afi_writel(port->pcie, value, ctrl);
> >>> +	if (port->reset_gpiod) {
> >>> +		gpiod_set_value(port->reset_gpiod, 1);
> >> After this the port should be functional, right? I think it'd be better
> >> to reverse the logic here and move the polarity of the GPIO into device
> >> tree. gpiod_set_value() takes care of inverting the level internally if
> >> the GPIO is marked as low-active in DT.
> >>
> >> The end result is obviously the same, but it makes the usage much
> >> clearer. If somebody want to write a DT for their board, they will look
> >> at the schematics and see a low-active reset line and may be tempted to
> >> describe it as such in DT, but with your current code that would be
> >> exactly the wrong way around.
> > I agree with Thierry here, you should change the logic.
> >
> > Question: what's the advantage of adding GPIO reset support if that's
> > architected already in port registers ? I am pretty sure there is a
> > reason behind it (and forgive me the dumb question) and I would like to
> > have it written in the commit log.
> >
> > Thanks,
> > Lorenzo
> 
> Each PCIe controller has a dedicated SFIO pin to support PERST#
> signal. Port register can control only this particular SFIO pin.
> However, in one of the Nvidia platform, instead of using PCIe SFIO
> pin, different gpio is routed PCIe slot. This happened because of a
> confusion in IO ball naming convention. To support this particular
> platform, driver has provide gpio support. I will update the commit
> log in V5.

What happens on that platform where you trigger reset through a port
register with :

value = afi_readl(port->pcie, ctrl);
value |= AFI_PEX_CTRL_RST;
afi_writel(port->pcie, value, ctrl);

(imagine the DT is not updated for instance or on current
mainline) ?

Lorenzo

> Manikanta
> 
> >
> >>> +	} else {
> >>> +		value = afi_readl(port->pcie, ctrl);
> >>> +		value |= AFI_PEX_CTRL_RST;
> >>> +		afi_writel(port->pcie, value, ctrl);
> >>> +	}
> >>>  }
> >>>  
> >>>  static void tegra_pcie_enable_rp_features(struct tegra_pcie_port *port)
> >>> @@ -2238,6 +2249,7 @@ static int tegra_pcie_parse_dt(struct tegra_pcie *pcie)
> >>>  		struct tegra_pcie_port *rp;
> >>>  		unsigned int index;
> >>>  		u32 value;
> >>> +		char *label;
> >>>  
> >>>  		err = of_pci_get_devfn(port);
> >>>  		if (err < 0) {
> >>> @@ -2296,6 +2308,23 @@ static int tegra_pcie_parse_dt(struct tegra_pcie *pcie)
> >>>  		if (IS_ERR(rp->base))
> >>>  			return PTR_ERR(rp->base);
> >>>  
> >>> +		label = kasprintf(GFP_KERNEL, "pex-reset-%u", index);
> >> devm_kasprintf()?
> >>
> >> Thierry
> >>
> >>> +		if (!label) {
> >>> +			dev_err(dev, "failed to create reset GPIO label\n");
> >>> +			return -ENOMEM;
> >>> +		}
> >>> +
> >>> +		rp->reset_gpiod = devm_gpiod_get_from_of_node(dev, port,
> >>> +							      "reset-gpios", 0,
> >>> +							      GPIOD_OUT_LOW,
> >>> +							      label);
> >>> +		kfree(label);
> >>> +		if (IS_ERR(rp->reset_gpiod)) {
> >>> +			err = PTR_ERR(rp->reset_gpiod);
> >>> +			dev_err(dev, "failed to get reset GPIO: %d\n", err);
> >>> +			return err;
> >>> +		}
> >>> +
> >>>  		list_add_tail(&rp->list, &pcie->ports);
> >>>  	}
> >>>  
> >>> -- 
> >>> 2.17.1
> >>>
> >
> 

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: [PATCH V4 27/28] PCI: tegra: Add support for GPIO based PERST#
  2019-06-14 14:32         ` Lorenzo Pieralisi
@ 2019-06-14 14:38           ` Manikanta Maddireddy
  2019-06-14 14:50             ` Lorenzo Pieralisi
  0 siblings, 1 reply; 72+ messages in thread
From: Manikanta Maddireddy @ 2019-06-14 14:38 UTC (permalink / raw)
  To: Lorenzo Pieralisi
  Cc: Thierry Reding, bhelgaas, robh+dt, mark.rutland, jonathanh,
	vidyas, linux-tegra, linux-pci, devicetree



On 14-Jun-19 8:02 PM, Lorenzo Pieralisi wrote:
> On Fri, Jun 14, 2019 at 04:07:35PM +0530, Manikanta Maddireddy wrote:
>>
>> On 13-Jun-19 8:54 PM, Lorenzo Pieralisi wrote:
>>> On Tue, Jun 04, 2019 at 03:22:33PM +0200, Thierry Reding wrote:
>>>
>>> [...]
>>>
>>>>> +	} else {
>>>>> +		value = afi_readl(port->pcie, ctrl);
>>>>> +		value &= ~AFI_PEX_CTRL_RST;
>>>>> +		afi_writel(port->pcie, value, ctrl);
>>>>> +	}
>>>>>  
>>>>>  	usleep_range(1000, 2000);
>>>>>  
>>>>> -	value = afi_readl(port->pcie, ctrl);
>>>>> -	value |= AFI_PEX_CTRL_RST;
>>>>> -	afi_writel(port->pcie, value, ctrl);
>>>>> +	if (port->reset_gpiod) {
>>>>> +		gpiod_set_value(port->reset_gpiod, 1);
>>>> After this the port should be functional, right? I think it'd be better
>>>> to reverse the logic here and move the polarity of the GPIO into device
>>>> tree. gpiod_set_value() takes care of inverting the level internally if
>>>> the GPIO is marked as low-active in DT.
>>>>
>>>> The end result is obviously the same, but it makes the usage much
>>>> clearer. If somebody want to write a DT for their board, they will look
>>>> at the schematics and see a low-active reset line and may be tempted to
>>>> describe it as such in DT, but with your current code that would be
>>>> exactly the wrong way around.
>>> I agree with Thierry here, you should change the logic.
>>>
>>> Question: what's the advantage of adding GPIO reset support if that's
>>> architected already in port registers ? I am pretty sure there is a
>>> reason behind it (and forgive me the dumb question) and I would like to
>>> have it written in the commit log.
>>>
>>> Thanks,
>>> Lorenzo
>> Each PCIe controller has a dedicated SFIO pin to support PERST#
>> signal. Port register can control only this particular SFIO pin.
>> However, in one of the Nvidia platform, instead of using PCIe SFIO
>> pin, different gpio is routed PCIe slot. This happened because of a
>> confusion in IO ball naming convention. To support this particular
>> platform, driver has provide gpio support. I will update the commit
>> log in V5.
> What happens on that platform where you trigger reset through a port
> register with :
>
> value = afi_readl(port->pcie, ctrl);
> value |= AFI_PEX_CTRL_RST;
> afi_writel(port->pcie, value, ctrl);
>
> (imagine the DT is not updated for instance or on current
> mainline) ?
>
> Lorenzo

Lets take an example of PCIe controller-0, SFIO ball name which is controlled
by the port-0 register is PEX_L0_RST. It will deassert PEX_L0_RST SFIO line
but it doesn't go to PCIe slot, so fundamental reset(PERST# deassert) is not
applied to the endpoint connected to that slot.

Manikanta

>> Manikanta
>>
>>>>> +	} else {
>>>>> +		value = afi_readl(port->pcie, ctrl);
>>>>> +		value |= AFI_PEX_CTRL_RST;
>>>>> +		afi_writel(port->pcie, value, ctrl);
>>>>> +	}
>>>>>  }
>>>>>  
>>>>>  static void tegra_pcie_enable_rp_features(struct tegra_pcie_port *port)
>>>>> @@ -2238,6 +2249,7 @@ static int tegra_pcie_parse_dt(struct tegra_pcie *pcie)
>>>>>  		struct tegra_pcie_port *rp;
>>>>>  		unsigned int index;
>>>>>  		u32 value;
>>>>> +		char *label;
>>>>>  
>>>>>  		err = of_pci_get_devfn(port);
>>>>>  		if (err < 0) {
>>>>> @@ -2296,6 +2308,23 @@ static int tegra_pcie_parse_dt(struct tegra_pcie *pcie)
>>>>>  		if (IS_ERR(rp->base))
>>>>>  			return PTR_ERR(rp->base);
>>>>>  
>>>>> +		label = kasprintf(GFP_KERNEL, "pex-reset-%u", index);
>>>> devm_kasprintf()?
>>>>
>>>> Thierry
>>>>
>>>>> +		if (!label) {
>>>>> +			dev_err(dev, "failed to create reset GPIO label\n");
>>>>> +			return -ENOMEM;
>>>>> +		}
>>>>> +
>>>>> +		rp->reset_gpiod = devm_gpiod_get_from_of_node(dev, port,
>>>>> +							      "reset-gpios", 0,
>>>>> +							      GPIOD_OUT_LOW,
>>>>> +							      label);
>>>>> +		kfree(label);
>>>>> +		if (IS_ERR(rp->reset_gpiod)) {
>>>>> +			err = PTR_ERR(rp->reset_gpiod);
>>>>> +			dev_err(dev, "failed to get reset GPIO: %d\n", err);
>>>>> +			return err;
>>>>> +		}
>>>>> +
>>>>>  		list_add_tail(&rp->list, &pcie->ports);
>>>>>  	}
>>>>>  
>>>>> -- 
>>>>> 2.17.1
>>>>>


^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: [PATCH V4 27/28] PCI: tegra: Add support for GPIO based PERST#
  2019-06-14 14:38           ` Manikanta Maddireddy
@ 2019-06-14 14:50             ` Lorenzo Pieralisi
  2019-06-14 14:56               ` Manikanta Maddireddy
  2019-06-14 15:23               ` Thierry Reding
  0 siblings, 2 replies; 72+ messages in thread
From: Lorenzo Pieralisi @ 2019-06-14 14:50 UTC (permalink / raw)
  To: Manikanta Maddireddy
  Cc: Thierry Reding, bhelgaas, robh+dt, mark.rutland, jonathanh,
	vidyas, linux-tegra, linux-pci, devicetree

On Fri, Jun 14, 2019 at 08:08:26PM +0530, Manikanta Maddireddy wrote:
> 
> 
> On 14-Jun-19 8:02 PM, Lorenzo Pieralisi wrote:
> > On Fri, Jun 14, 2019 at 04:07:35PM +0530, Manikanta Maddireddy wrote:
> >>
> >> On 13-Jun-19 8:54 PM, Lorenzo Pieralisi wrote:
> >>> On Tue, Jun 04, 2019 at 03:22:33PM +0200, Thierry Reding wrote:
> >>>
> >>> [...]
> >>>
> >>>>> +	} else {
> >>>>> +		value = afi_readl(port->pcie, ctrl);
> >>>>> +		value &= ~AFI_PEX_CTRL_RST;
> >>>>> +		afi_writel(port->pcie, value, ctrl);
> >>>>> +	}
> >>>>>  
> >>>>>  	usleep_range(1000, 2000);
> >>>>>  
> >>>>> -	value = afi_readl(port->pcie, ctrl);
> >>>>> -	value |= AFI_PEX_CTRL_RST;
> >>>>> -	afi_writel(port->pcie, value, ctrl);
> >>>>> +	if (port->reset_gpiod) {
> >>>>> +		gpiod_set_value(port->reset_gpiod, 1);
> >>>> After this the port should be functional, right? I think it'd be better
> >>>> to reverse the logic here and move the polarity of the GPIO into device
> >>>> tree. gpiod_set_value() takes care of inverting the level internally if
> >>>> the GPIO is marked as low-active in DT.
> >>>>
> >>>> The end result is obviously the same, but it makes the usage much
> >>>> clearer. If somebody want to write a DT for their board, they will look
> >>>> at the schematics and see a low-active reset line and may be tempted to
> >>>> describe it as such in DT, but with your current code that would be
> >>>> exactly the wrong way around.
> >>> I agree with Thierry here, you should change the logic.
> >>>
> >>> Question: what's the advantage of adding GPIO reset support if that's
> >>> architected already in port registers ? I am pretty sure there is a
> >>> reason behind it (and forgive me the dumb question) and I would like to
> >>> have it written in the commit log.
> >>>
> >>> Thanks,
> >>> Lorenzo
> >> Each PCIe controller has a dedicated SFIO pin to support PERST#
> >> signal. Port register can control only this particular SFIO pin.
> >> However, in one of the Nvidia platform, instead of using PCIe SFIO
> >> pin, different gpio is routed PCIe slot. This happened because of a
> >> confusion in IO ball naming convention. To support this particular
> >> platform, driver has provide gpio support. I will update the commit
> >> log in V5.
> > What happens on that platform where you trigger reset through a port
> > register with :
> >
> > value = afi_readl(port->pcie, ctrl);
> > value |= AFI_PEX_CTRL_RST;
> > afi_writel(port->pcie, value, ctrl);
> >
> > (imagine the DT is not updated for instance or on current
> > mainline) ?
> >
> > Lorenzo
> 
> Lets take an example of PCIe controller-0, SFIO ball name which is
> controlled by the port-0 register is PEX_L0_RST. It will deassert
> PEX_L0_RST SFIO line but it doesn't go to PCIe slot, so fundamental
> reset(PERST# deassert) is not applied to the endpoint connected to
> that slot.

That's the point I am making, if the reset is not applied nothing
will work (provided PEX_L0_RST does not do any damage either).

For the platform in question you should make reset-gpios mandatory and
fail if not present (instead of toggling the wrong reset line) there is
no chance the driver can work without that property AFAICS.

Lorenzo

> 
> 
> Manikanta
> 
> >> Manikanta
> >>
> >>>>> +	} else {
> >>>>> +		value = afi_readl(port->pcie, ctrl);
> >>>>> +		value |= AFI_PEX_CTRL_RST;
> >>>>> +		afi_writel(port->pcie, value, ctrl);
> >>>>> +	}
> >>>>>  }
> >>>>>  
> >>>>>  static void tegra_pcie_enable_rp_features(struct tegra_pcie_port *port)
> >>>>> @@ -2238,6 +2249,7 @@ static int tegra_pcie_parse_dt(struct tegra_pcie *pcie)
> >>>>>  		struct tegra_pcie_port *rp;
> >>>>>  		unsigned int index;
> >>>>>  		u32 value;
> >>>>> +		char *label;
> >>>>>  
> >>>>>  		err = of_pci_get_devfn(port);
> >>>>>  		if (err < 0) {
> >>>>> @@ -2296,6 +2308,23 @@ static int tegra_pcie_parse_dt(struct tegra_pcie *pcie)
> >>>>>  		if (IS_ERR(rp->base))
> >>>>>  			return PTR_ERR(rp->base);
> >>>>>  
> >>>>> +		label = kasprintf(GFP_KERNEL, "pex-reset-%u", index);
> >>>> devm_kasprintf()?
> >>>>
> >>>> Thierry
> >>>>
> >>>>> +		if (!label) {
> >>>>> +			dev_err(dev, "failed to create reset GPIO label\n");
> >>>>> +			return -ENOMEM;
> >>>>> +		}
> >>>>> +
> >>>>> +		rp->reset_gpiod = devm_gpiod_get_from_of_node(dev, port,
> >>>>> +							      "reset-gpios", 0,
> >>>>> +							      GPIOD_OUT_LOW,
> >>>>> +							      label);
> >>>>> +		kfree(label);
> >>>>> +		if (IS_ERR(rp->reset_gpiod)) {
> >>>>> +			err = PTR_ERR(rp->reset_gpiod);
> >>>>> +			dev_err(dev, "failed to get reset GPIO: %d\n", err);
> >>>>> +			return err;
> >>>>> +		}
> >>>>> +
> >>>>>  		list_add_tail(&rp->list, &pcie->ports);
> >>>>>  	}
> >>>>>  
> >>>>> -- 
> >>>>> 2.17.1
> >>>>>
> 

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: [PATCH V4 27/28] PCI: tegra: Add support for GPIO based PERST#
  2019-06-14 14:50             ` Lorenzo Pieralisi
@ 2019-06-14 14:56               ` Manikanta Maddireddy
  2019-06-14 15:23               ` Thierry Reding
  1 sibling, 0 replies; 72+ messages in thread
From: Manikanta Maddireddy @ 2019-06-14 14:56 UTC (permalink / raw)
  To: Lorenzo Pieralisi
  Cc: Thierry Reding, bhelgaas, robh+dt, mark.rutland, jonathanh,
	vidyas, linux-tegra, linux-pci, devicetree



On 14-Jun-19 8:20 PM, Lorenzo Pieralisi wrote:
> On Fri, Jun 14, 2019 at 08:08:26PM +0530, Manikanta Maddireddy wrote:
>>
>> On 14-Jun-19 8:02 PM, Lorenzo Pieralisi wrote:
>>> On Fri, Jun 14, 2019 at 04:07:35PM +0530, Manikanta Maddireddy wrote:
>>>> On 13-Jun-19 8:54 PM, Lorenzo Pieralisi wrote:
>>>>> On Tue, Jun 04, 2019 at 03:22:33PM +0200, Thierry Reding wrote:
>>>>>
>>>>> [...]
>>>>>
>>>>>>> +	} else {
>>>>>>> +		value = afi_readl(port->pcie, ctrl);
>>>>>>> +		value &= ~AFI_PEX_CTRL_RST;
>>>>>>> +		afi_writel(port->pcie, value, ctrl);
>>>>>>> +	}
>>>>>>>  
>>>>>>>  	usleep_range(1000, 2000);
>>>>>>>  
>>>>>>> -	value = afi_readl(port->pcie, ctrl);
>>>>>>> -	value |= AFI_PEX_CTRL_RST;
>>>>>>> -	afi_writel(port->pcie, value, ctrl);
>>>>>>> +	if (port->reset_gpiod) {
>>>>>>> +		gpiod_set_value(port->reset_gpiod, 1);
>>>>>> After this the port should be functional, right? I think it'd be better
>>>>>> to reverse the logic here and move the polarity of the GPIO into device
>>>>>> tree. gpiod_set_value() takes care of inverting the level internally if
>>>>>> the GPIO is marked as low-active in DT.
>>>>>>
>>>>>> The end result is obviously the same, but it makes the usage much
>>>>>> clearer. If somebody want to write a DT for their board, they will look
>>>>>> at the schematics and see a low-active reset line and may be tempted to
>>>>>> describe it as such in DT, but with your current code that would be
>>>>>> exactly the wrong way around.
>>>>> I agree with Thierry here, you should change the logic.
>>>>>
>>>>> Question: what's the advantage of adding GPIO reset support if that's
>>>>> architected already in port registers ? I am pretty sure there is a
>>>>> reason behind it (and forgive me the dumb question) and I would like to
>>>>> have it written in the commit log.
>>>>>
>>>>> Thanks,
>>>>> Lorenzo
>>>> Each PCIe controller has a dedicated SFIO pin to support PERST#
>>>> signal. Port register can control only this particular SFIO pin.
>>>> However, in one of the Nvidia platform, instead of using PCIe SFIO
>>>> pin, different gpio is routed PCIe slot. This happened because of a
>>>> confusion in IO ball naming convention. To support this particular
>>>> platform, driver has provide gpio support. I will update the commit
>>>> log in V5.
>>> What happens on that platform where you trigger reset through a port
>>> register with :
>>>
>>> value = afi_readl(port->pcie, ctrl);
>>> value |= AFI_PEX_CTRL_RST;
>>> afi_writel(port->pcie, value, ctrl);
>>>
>>> (imagine the DT is not updated for instance or on current
>>> mainline) ?
>>>
>>> Lorenzo
>> Lets take an example of PCIe controller-0, SFIO ball name which is
>> controlled by the port-0 register is PEX_L0_RST. It will deassert
>> PEX_L0_RST SFIO line but it doesn't go to PCIe slot, so fundamental
>> reset(PERST# deassert) is not applied to the endpoint connected to
>> that slot.
> That's the point I am making, if the reset is not applied nothing
> will work (provided PEX_L0_RST does not do any damage either).
>
> For the platform in question you should make reset-gpios mandatory and
> fail if not present (instead of toggling the wrong reset line) there is
> no chance the driver can work without that property AFAICS.
>
> Lorenzo

In upstream kernel, device tree file is not available for the platform in
question. That is the reason for not send device tree patch as part of 
this series.

Manikanta

>>
>> Manikanta
>>
>>>> Manikanta
>>>>
>>>>>>> +	} else {
>>>>>>> +		value = afi_readl(port->pcie, ctrl);
>>>>>>> +		value |= AFI_PEX_CTRL_RST;
>>>>>>> +		afi_writel(port->pcie, value, ctrl);
>>>>>>> +	}
>>>>>>>  }
>>>>>>>  
>>>>>>>  static void tegra_pcie_enable_rp_features(struct tegra_pcie_port *port)
>>>>>>> @@ -2238,6 +2249,7 @@ static int tegra_pcie_parse_dt(struct tegra_pcie *pcie)
>>>>>>>  		struct tegra_pcie_port *rp;
>>>>>>>  		unsigned int index;
>>>>>>>  		u32 value;
>>>>>>> +		char *label;
>>>>>>>  
>>>>>>>  		err = of_pci_get_devfn(port);
>>>>>>>  		if (err < 0) {
>>>>>>> @@ -2296,6 +2308,23 @@ static int tegra_pcie_parse_dt(struct tegra_pcie *pcie)
>>>>>>>  		if (IS_ERR(rp->base))
>>>>>>>  			return PTR_ERR(rp->base);
>>>>>>>  
>>>>>>> +		label = kasprintf(GFP_KERNEL, "pex-reset-%u", index);
>>>>>> devm_kasprintf()?
>>>>>>
>>>>>> Thierry
>>>>>>
>>>>>>> +		if (!label) {
>>>>>>> +			dev_err(dev, "failed to create reset GPIO label\n");
>>>>>>> +			return -ENOMEM;
>>>>>>> +		}
>>>>>>> +
>>>>>>> +		rp->reset_gpiod = devm_gpiod_get_from_of_node(dev, port,
>>>>>>> +							      "reset-gpios", 0,
>>>>>>> +							      GPIOD_OUT_LOW,
>>>>>>> +							      label);
>>>>>>> +		kfree(label);
>>>>>>> +		if (IS_ERR(rp->reset_gpiod)) {
>>>>>>> +			err = PTR_ERR(rp->reset_gpiod);
>>>>>>> +			dev_err(dev, "failed to get reset GPIO: %d\n", err);
>>>>>>> +			return err;
>>>>>>> +		}
>>>>>>> +
>>>>>>>  		list_add_tail(&rp->list, &pcie->ports);
>>>>>>>  	}
>>>>>>>  
>>>>>>> -- 
>>>>>>> 2.17.1
>>>>>>>


^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: [PATCH V4 27/28] PCI: tegra: Add support for GPIO based PERST#
  2019-06-14 14:50             ` Lorenzo Pieralisi
  2019-06-14 14:56               ` Manikanta Maddireddy
@ 2019-06-14 15:23               ` Thierry Reding
  2019-06-14 15:59                 ` Lorenzo Pieralisi
  1 sibling, 1 reply; 72+ messages in thread
From: Thierry Reding @ 2019-06-14 15:23 UTC (permalink / raw)
  To: Lorenzo Pieralisi
  Cc: Manikanta Maddireddy, bhelgaas, robh+dt, mark.rutland, jonathanh,
	vidyas, linux-tegra, linux-pci, devicetree

[-- Attachment #1: Type: text/plain, Size: 3677 bytes --]

On Fri, Jun 14, 2019 at 03:50:23PM +0100, Lorenzo Pieralisi wrote:
> On Fri, Jun 14, 2019 at 08:08:26PM +0530, Manikanta Maddireddy wrote:
> > 
> > 
> > On 14-Jun-19 8:02 PM, Lorenzo Pieralisi wrote:
> > > On Fri, Jun 14, 2019 at 04:07:35PM +0530, Manikanta Maddireddy wrote:
> > >>
> > >> On 13-Jun-19 8:54 PM, Lorenzo Pieralisi wrote:
> > >>> On Tue, Jun 04, 2019 at 03:22:33PM +0200, Thierry Reding wrote:
> > >>>
> > >>> [...]
> > >>>
> > >>>>> +	} else {
> > >>>>> +		value = afi_readl(port->pcie, ctrl);
> > >>>>> +		value &= ~AFI_PEX_CTRL_RST;
> > >>>>> +		afi_writel(port->pcie, value, ctrl);
> > >>>>> +	}
> > >>>>>  
> > >>>>>  	usleep_range(1000, 2000);
> > >>>>>  
> > >>>>> -	value = afi_readl(port->pcie, ctrl);
> > >>>>> -	value |= AFI_PEX_CTRL_RST;
> > >>>>> -	afi_writel(port->pcie, value, ctrl);
> > >>>>> +	if (port->reset_gpiod) {
> > >>>>> +		gpiod_set_value(port->reset_gpiod, 1);
> > >>>> After this the port should be functional, right? I think it'd be better
> > >>>> to reverse the logic here and move the polarity of the GPIO into device
> > >>>> tree. gpiod_set_value() takes care of inverting the level internally if
> > >>>> the GPIO is marked as low-active in DT.
> > >>>>
> > >>>> The end result is obviously the same, but it makes the usage much
> > >>>> clearer. If somebody want to write a DT for their board, they will look
> > >>>> at the schematics and see a low-active reset line and may be tempted to
> > >>>> describe it as such in DT, but with your current code that would be
> > >>>> exactly the wrong way around.
> > >>> I agree with Thierry here, you should change the logic.
> > >>>
> > >>> Question: what's the advantage of adding GPIO reset support if that's
> > >>> architected already in port registers ? I am pretty sure there is a
> > >>> reason behind it (and forgive me the dumb question) and I would like to
> > >>> have it written in the commit log.
> > >>>
> > >>> Thanks,
> > >>> Lorenzo
> > >> Each PCIe controller has a dedicated SFIO pin to support PERST#
> > >> signal. Port register can control only this particular SFIO pin.
> > >> However, in one of the Nvidia platform, instead of using PCIe SFIO
> > >> pin, different gpio is routed PCIe slot. This happened because of a
> > >> confusion in IO ball naming convention. To support this particular
> > >> platform, driver has provide gpio support. I will update the commit
> > >> log in V5.
> > > What happens on that platform where you trigger reset through a port
> > > register with :
> > >
> > > value = afi_readl(port->pcie, ctrl);
> > > value |= AFI_PEX_CTRL_RST;
> > > afi_writel(port->pcie, value, ctrl);
> > >
> > > (imagine the DT is not updated for instance or on current
> > > mainline) ?
> > >
> > > Lorenzo
> > 
> > Lets take an example of PCIe controller-0, SFIO ball name which is
> > controlled by the port-0 register is PEX_L0_RST. It will deassert
> > PEX_L0_RST SFIO line but it doesn't go to PCIe slot, so fundamental
> > reset(PERST# deassert) is not applied to the endpoint connected to
> > that slot.
> 
> That's the point I am making, if the reset is not applied nothing
> will work (provided PEX_L0_RST does not do any damage either).
> 
> For the platform in question you should make reset-gpios mandatory and
> fail if not present (instead of toggling the wrong reset line) there is
> no chance the driver can work without that property AFAICS.

I'm not sure I understand what you're proposing here. Are you suggesting
that we put a check in the driver to see if we're running on a specific
board and then fail if the reset-gpios are not there?

Thierry

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: [PATCH V4 27/28] PCI: tegra: Add support for GPIO based PERST#
  2019-06-14 15:23               ` Thierry Reding
@ 2019-06-14 15:59                 ` Lorenzo Pieralisi
  2019-06-14 16:30                   ` Manikanta Maddireddy
  0 siblings, 1 reply; 72+ messages in thread
From: Lorenzo Pieralisi @ 2019-06-14 15:59 UTC (permalink / raw)
  To: Thierry Reding
  Cc: Manikanta Maddireddy, bhelgaas, robh+dt, mark.rutland, jonathanh,
	vidyas, linux-tegra, linux-pci, devicetree

On Fri, Jun 14, 2019 at 05:23:04PM +0200, Thierry Reding wrote:
> On Fri, Jun 14, 2019 at 03:50:23PM +0100, Lorenzo Pieralisi wrote:
> > On Fri, Jun 14, 2019 at 08:08:26PM +0530, Manikanta Maddireddy wrote:
> > > 
> > > 
> > > On 14-Jun-19 8:02 PM, Lorenzo Pieralisi wrote:
> > > > On Fri, Jun 14, 2019 at 04:07:35PM +0530, Manikanta Maddireddy wrote:
> > > >>
> > > >> On 13-Jun-19 8:54 PM, Lorenzo Pieralisi wrote:
> > > >>> On Tue, Jun 04, 2019 at 03:22:33PM +0200, Thierry Reding wrote:
> > > >>>
> > > >>> [...]
> > > >>>
> > > >>>>> +	} else {
> > > >>>>> +		value = afi_readl(port->pcie, ctrl);
> > > >>>>> +		value &= ~AFI_PEX_CTRL_RST;
> > > >>>>> +		afi_writel(port->pcie, value, ctrl);
> > > >>>>> +	}
> > > >>>>>  
> > > >>>>>  	usleep_range(1000, 2000);
> > > >>>>>  
> > > >>>>> -	value = afi_readl(port->pcie, ctrl);
> > > >>>>> -	value |= AFI_PEX_CTRL_RST;
> > > >>>>> -	afi_writel(port->pcie, value, ctrl);
> > > >>>>> +	if (port->reset_gpiod) {
> > > >>>>> +		gpiod_set_value(port->reset_gpiod, 1);
> > > >>>> After this the port should be functional, right? I think it'd be better
> > > >>>> to reverse the logic here and move the polarity of the GPIO into device
> > > >>>> tree. gpiod_set_value() takes care of inverting the level internally if
> > > >>>> the GPIO is marked as low-active in DT.
> > > >>>>
> > > >>>> The end result is obviously the same, but it makes the usage much
> > > >>>> clearer. If somebody want to write a DT for their board, they will look
> > > >>>> at the schematics and see a low-active reset line and may be tempted to
> > > >>>> describe it as such in DT, but with your current code that would be
> > > >>>> exactly the wrong way around.
> > > >>> I agree with Thierry here, you should change the logic.
> > > >>>
> > > >>> Question: what's the advantage of adding GPIO reset support if that's
> > > >>> architected already in port registers ? I am pretty sure there is a
> > > >>> reason behind it (and forgive me the dumb question) and I would like to
> > > >>> have it written in the commit log.
> > > >>>
> > > >>> Thanks,
> > > >>> Lorenzo
> > > >> Each PCIe controller has a dedicated SFIO pin to support PERST#
> > > >> signal. Port register can control only this particular SFIO pin.
> > > >> However, in one of the Nvidia platform, instead of using PCIe SFIO
> > > >> pin, different gpio is routed PCIe slot. This happened because of a
> > > >> confusion in IO ball naming convention. To support this particular
> > > >> platform, driver has provide gpio support. I will update the commit
> > > >> log in V5.
> > > > What happens on that platform where you trigger reset through a port
> > > > register with :
> > > >
> > > > value = afi_readl(port->pcie, ctrl);
> > > > value |= AFI_PEX_CTRL_RST;
> > > > afi_writel(port->pcie, value, ctrl);
> > > >
> > > > (imagine the DT is not updated for instance or on current
> > > > mainline) ?
> > > >
> > > > Lorenzo
> > > 
> > > Lets take an example of PCIe controller-0, SFIO ball name which is
> > > controlled by the port-0 register is PEX_L0_RST. It will deassert
> > > PEX_L0_RST SFIO line but it doesn't go to PCIe slot, so fundamental
> > > reset(PERST# deassert) is not applied to the endpoint connected to
> > > that slot.
> > 
> > That's the point I am making, if the reset is not applied nothing
> > will work (provided PEX_L0_RST does not do any damage either).
> > 
> > For the platform in question you should make reset-gpios mandatory and
> > fail if not present (instead of toggling the wrong reset line) there is
> > no chance the driver can work without that property AFAICS.
> 
> I'm not sure I understand what you're proposing here. Are you suggesting
> that we put a check in the driver to see if we're running on a specific
> board and then fail if the reset-gpios are not there?

I am just trying to understand what this patch does. By reading it again
it looks like it makes GPIO PERST# reset mandatory for all platforms
supported by this driver (because if the driver does not grab an handle
to the GPIO tegra_pcie_parse_dt() fails), if I read the code correctly,
apologies if not.

Which makes me question the check:

	if (port->reset_gpiod) {
		gpiod_set_value(port->reset_gpiod, 0);

in tegra_pcie_port_reset(), if we are there port->reset_gpiod can't be
NULL or I am missing something and also make:

	} else {
		value = afi_readl(port->pcie, ctrl);
		value &= ~AFI_PEX_CTRL_RST;
		afi_writel(port->pcie, value, ctrl);
	}

path dead code.

Is this GPIO based #PERST a per-platform requirement or you want
to update the driver to always use GPIO based #PERST ?

And if it is a per-platform requirement I assume that a missing
DT property describing the GPIO #PERST should cause a probe failure,
not a fallback to port registers reset (which may have unintended
consequences).

From the commit log it is not clear what this patch does and for what
reason it does it but it should be, let's define it here and update the
log accordingly for everyone's benefit.

Lorenzo

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: [PATCH V4 27/28] PCI: tegra: Add support for GPIO based PERST#
  2019-06-14 15:59                 ` Lorenzo Pieralisi
@ 2019-06-14 16:30                   ` Manikanta Maddireddy
  2019-06-14 16:53                     ` Lorenzo Pieralisi
  0 siblings, 1 reply; 72+ messages in thread
From: Manikanta Maddireddy @ 2019-06-14 16:30 UTC (permalink / raw)
  To: Lorenzo Pieralisi, Thierry Reding
  Cc: bhelgaas, robh+dt, mark.rutland, jonathanh, vidyas, linux-tegra,
	linux-pci, devicetree



On 14-Jun-19 9:29 PM, Lorenzo Pieralisi wrote:
> On Fri, Jun 14, 2019 at 05:23:04PM +0200, Thierry Reding wrote:
>> On Fri, Jun 14, 2019 at 03:50:23PM +0100, Lorenzo Pieralisi wrote:
>>> On Fri, Jun 14, 2019 at 08:08:26PM +0530, Manikanta Maddireddy wrote:
>>>>
>>>> On 14-Jun-19 8:02 PM, Lorenzo Pieralisi wrote:
>>>>> On Fri, Jun 14, 2019 at 04:07:35PM +0530, Manikanta Maddireddy wrote:
>>>>>> On 13-Jun-19 8:54 PM, Lorenzo Pieralisi wrote:
>>>>>>> On Tue, Jun 04, 2019 at 03:22:33PM +0200, Thierry Reding wrote:
>>>>>>>
>>>>>>> [...]
>>>>>>>
>>>>>>>>> +	} else {
>>>>>>>>> +		value = afi_readl(port->pcie, ctrl);
>>>>>>>>> +		value &= ~AFI_PEX_CTRL_RST;
>>>>>>>>> +		afi_writel(port->pcie, value, ctrl);
>>>>>>>>> +	}
>>>>>>>>>  
>>>>>>>>>  	usleep_range(1000, 2000);
>>>>>>>>>  
>>>>>>>>> -	value = afi_readl(port->pcie, ctrl);
>>>>>>>>> -	value |= AFI_PEX_CTRL_RST;
>>>>>>>>> -	afi_writel(port->pcie, value, ctrl);
>>>>>>>>> +	if (port->reset_gpiod) {
>>>>>>>>> +		gpiod_set_value(port->reset_gpiod, 1);
>>>>>>>> After this the port should be functional, right? I think it'd be better
>>>>>>>> to reverse the logic here and move the polarity of the GPIO into device
>>>>>>>> tree. gpiod_set_value() takes care of inverting the level internally if
>>>>>>>> the GPIO is marked as low-active in DT.
>>>>>>>>
>>>>>>>> The end result is obviously the same, but it makes the usage much
>>>>>>>> clearer. If somebody want to write a DT for their board, they will look
>>>>>>>> at the schematics and see a low-active reset line and may be tempted to
>>>>>>>> describe it as such in DT, but with your current code that would be
>>>>>>>> exactly the wrong way around.
>>>>>>> I agree with Thierry here, you should change the logic.
>>>>>>>
>>>>>>> Question: what's the advantage of adding GPIO reset support if that's
>>>>>>> architected already in port registers ? I am pretty sure there is a
>>>>>>> reason behind it (and forgive me the dumb question) and I would like to
>>>>>>> have it written in the commit log.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Lorenzo
>>>>>> Each PCIe controller has a dedicated SFIO pin to support PERST#
>>>>>> signal. Port register can control only this particular SFIO pin.
>>>>>> However, in one of the Nvidia platform, instead of using PCIe SFIO
>>>>>> pin, different gpio is routed PCIe slot. This happened because of a
>>>>>> confusion in IO ball naming convention. To support this particular
>>>>>> platform, driver has provide gpio support. I will update the commit
>>>>>> log in V5.
>>>>> What happens on that platform where you trigger reset through a port
>>>>> register with :
>>>>>
>>>>> value = afi_readl(port->pcie, ctrl);
>>>>> value |= AFI_PEX_CTRL_RST;
>>>>> afi_writel(port->pcie, value, ctrl);
>>>>>
>>>>> (imagine the DT is not updated for instance or on current
>>>>> mainline) ?
>>>>>
>>>>> Lorenzo
>>>> Lets take an example of PCIe controller-0, SFIO ball name which is
>>>> controlled by the port-0 register is PEX_L0_RST. It will deassert
>>>> PEX_L0_RST SFIO line but it doesn't go to PCIe slot, so fundamental
>>>> reset(PERST# deassert) is not applied to the endpoint connected to
>>>> that slot.
>>> That's the point I am making, if the reset is not applied nothing
>>> will work (provided PEX_L0_RST does not do any damage either).
>>>
>>> For the platform in question you should make reset-gpios mandatory and
>>> fail if not present (instead of toggling the wrong reset line) there is
>>> no chance the driver can work without that property AFAICS.
>> I'm not sure I understand what you're proposing here. Are you suggesting
>> that we put a check in the driver to see if we're running on a specific
>> board and then fail if the reset-gpios are not there?
> I am just trying to understand what this patch does. By reading it again
> it looks like it makes GPIO PERST# reset mandatory for all platforms
> supported by this driver (because if the driver does not grab an handle
> to the GPIO tegra_pcie_parse_dt() fails), if I read the code correctly,
> apologies if not.
>
> Which makes me question the check:
>
> 	if (port->reset_gpiod) {
> 		gpiod_set_value(port->reset_gpiod, 0);
>
> in tegra_pcie_port_reset(), if we are there port->reset_gpiod can't be
> NULL or I am missing something and also make:
>
> 	} else {
> 		value = afi_readl(port->pcie, ctrl);
> 		value &= ~AFI_PEX_CTRL_RST;
> 		afi_writel(port->pcie, value, ctrl);
> 	}
>
> path dead code.
>
> Is this GPIO based #PERST a per-platform requirement or you want
> to update the driver to always use GPIO based #PERST ?
>
> And if it is a per-platform requirement I assume that a missing
> DT property describing the GPIO #PERST should cause a probe failure,
> not a fallback to port registers reset (which may have unintended
> consequences).
>
> From the commit log it is not clear what this patch does and for what
> reason it does it but it should be, let's define it here and update the
> log accordingly for everyone's benefit.
>
> Lorenzo

GPIO based PERST# is per-platform requirement.
If DT prop is not present, then devm_gpiod_get_from_of_node() returns
NULL gpio_desc.

struct gpio_desc *gpiod_get_from_of_node(struct device_node *node,
                                         const char *propname, int index,
                                         enum gpiod_flags dflags,
                                         const char *label)
{
        struct gpio_desc *desc;
        unsigned long lflags = 0;
        enum of_gpio_flags flags;
        bool active_low = false;
        bool single_ended = false;
        bool open_drain = false;
        bool transitory = false;
        int ret;

        desc = of_get_named_gpiod_flags(node, propname,
                                        index, &flags);

        if (!desc || IS_ERR(desc)) {
*/* If it is not there, just return NULL */****if (PTR_ERR(desc) == -ENOENT)****return NULL;*
                return desc;
        }
	...

}

Manikanta



^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: [PATCH V4 27/28] PCI: tegra: Add support for GPIO based PERST#
  2019-06-14 16:30                   ` Manikanta Maddireddy
@ 2019-06-14 16:53                     ` Lorenzo Pieralisi
  2019-06-14 17:23                       ` Manikanta Maddireddy
  2019-06-17 11:26                       ` Thierry Reding
  0 siblings, 2 replies; 72+ messages in thread
From: Lorenzo Pieralisi @ 2019-06-14 16:53 UTC (permalink / raw)
  To: Manikanta Maddireddy
  Cc: Thierry Reding, bhelgaas, robh+dt, mark.rutland, jonathanh,
	vidyas, linux-tegra, linux-pci, devicetree

On Fri, Jun 14, 2019 at 10:00:49PM +0530, Manikanta Maddireddy wrote:

[...]

> GPIO based PERST# is per-platform requirement.
> If DT prop is not present, then devm_gpiod_get_from_of_node() returns
> NULL gpio_desc.
> 
> struct gpio_desc *gpiod_get_from_of_node(struct device_node *node,
>                                          const char *propname, int index,
>                                          enum gpiod_flags dflags,
>                                          const char *label)
> {
>         struct gpio_desc *desc;
>         unsigned long lflags = 0;
>         enum of_gpio_flags flags;
>         bool active_low = false;
>         bool single_ended = false;
>         bool open_drain = false;
>         bool transitory = false;
>         int ret;
> 
>         desc = of_get_named_gpiod_flags(node, propname,
>                                         index, &flags);
> 
>         if (!desc || IS_ERR(desc)) {
> */* If it is not there, just return NULL */****if (PTR_ERR(desc) == -ENOENT)****return NULL;*
>                 return desc;
>         }
> 	...
> 
> }

Ok. My point then is that you have no way to enforce this requirement on
platforms that actually need it, I do not even know if there is a
way you can do it (I was thinking along the lines of using a
compatible string to detect whether the GPIO #PERST reset is mandatory)
but maybe this is not even a SOC property.

Maybe what I am asking is overkill, I just wanted to understand.

I was just asking a question to understand how you handle the case
where a GPIO pin definition is missing in DT for a platform that
actually needs it, the driver will probe but nothing will work.

It would be good to describe this and capture it in the commit log.

Thanks,
Lorenzo

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: [PATCH V4 27/28] PCI: tegra: Add support for GPIO based PERST#
  2019-06-14 16:53                     ` Lorenzo Pieralisi
@ 2019-06-14 17:23                       ` Manikanta Maddireddy
  2019-06-17  9:48                         ` Lorenzo Pieralisi
  2019-06-17 11:29                         ` Thierry Reding
  2019-06-17 11:26                       ` Thierry Reding
  1 sibling, 2 replies; 72+ messages in thread
From: Manikanta Maddireddy @ 2019-06-14 17:23 UTC (permalink / raw)
  To: Lorenzo Pieralisi
  Cc: Thierry Reding, bhelgaas, robh+dt, mark.rutland, jonathanh,
	vidyas, linux-tegra, linux-pci, devicetree



On 14-Jun-19 10:23 PM, Lorenzo Pieralisi wrote:
> On Fri, Jun 14, 2019 at 10:00:49PM +0530, Manikanta Maddireddy wrote:
>
> [...]
>
>> GPIO based PERST# is per-platform requirement.
>> If DT prop is not present, then devm_gpiod_get_from_of_node() returns
>> NULL gpio_desc.
>>
>> struct gpio_desc *gpiod_get_from_of_node(struct device_node *node,
>>                                          const char *propname, int index,
>>                                          enum gpiod_flags dflags,
>>                                          const char *label)
>> {
>>         struct gpio_desc *desc;
>>         unsigned long lflags = 0;
>>         enum of_gpio_flags flags;
>>         bool active_low = false;
>>         bool single_ended = false;
>>         bool open_drain = false;
>>         bool transitory = false;
>>         int ret;
>>
>>         desc = of_get_named_gpiod_flags(node, propname,
>>                                         index, &flags);
>>
>>         if (!desc || IS_ERR(desc)) {
>> */* If it is not there, just return NULL */****if (PTR_ERR(desc) == -ENOENT)****return NULL;*
>>                 return desc;
>>         }
>> 	...
>>
>> }
> Ok. My point then is that you have no way to enforce this requirement on
> platforms that actually need it, I do not even know if there is a
> way you can do it (I was thinking along the lines of using a
> compatible string to detect whether the GPIO #PERST reset is mandatory)
> but maybe this is not even a SOC property.
>
> Maybe what I am asking is overkill, I just wanted to understand.
>
> I was just asking a question to understand how you handle the case
> where a GPIO pin definition is missing in DT for a platform that
> actually needs it, the driver will probe but nothing will work.
>
> It would be good to describe this and capture it in the commit log.
>
> Thanks,
> Lorenzo

I can't think of a easy way to enforce this requirement. As you said
compatible string is per SOC, so we can't use it for a platform.
This issue is present on only one platform, so it is hard to miss the
DT property. That is the reason for publishing this patch with out this
enforcement in driver.

I thought for changing PERST# to GPIO for all platform, but testing is
a tedious job. Also I don't have Tegra20 and Tegra30 platforms.

Do you want me to drop the patch or update the limitation in the commit
log?

Manikanta



^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: [PATCH V4 27/28] PCI: tegra: Add support for GPIO based PERST#
  2019-06-14 17:23                       ` Manikanta Maddireddy
@ 2019-06-17  9:48                         ` Lorenzo Pieralisi
  2019-06-17 10:27                           ` Manikanta Maddireddy
  2019-06-17 11:29                         ` Thierry Reding
  1 sibling, 1 reply; 72+ messages in thread
From: Lorenzo Pieralisi @ 2019-06-17  9:48 UTC (permalink / raw)
  To: Manikanta Maddireddy
  Cc: Thierry Reding, bhelgaas, robh+dt, mark.rutland, jonathanh,
	vidyas, linux-tegra, linux-pci, devicetree

On Fri, Jun 14, 2019 at 10:53:13PM +0530, Manikanta Maddireddy wrote:

[...]

> > Ok. My point then is that you have no way to enforce this requirement on
> > platforms that actually need it, I do not even know if there is a
> > way you can do it (I was thinking along the lines of using a
> > compatible string to detect whether the GPIO #PERST reset is mandatory)
> > but maybe this is not even a SOC property.
> >
> > Maybe what I am asking is overkill, I just wanted to understand.
> >
> > I was just asking a question to understand how you handle the case
> > where a GPIO pin definition is missing in DT for a platform that
> > actually needs it, the driver will probe but nothing will work.
> >
> > It would be good to describe this and capture it in the commit log.
> >
> > Thanks,
> > Lorenzo
> 
> I can't think of a easy way to enforce this requirement. As you said
> compatible string is per SOC, so we can't use it for a platform.
> This issue is present on only one platform, so it is hard to miss the
> DT property. That is the reason for publishing this patch with out this
> enforcement in driver.
> 
> I thought for changing PERST# to GPIO for all platform, but testing is
> a tedious job. Also I don't have Tegra20 and Tegra30 platforms.

I can't help with that.

> Do you want me to drop the patch or update the limitation in the commit
> log?

It is Thierry's call, if he is OK with it fine by me, please do
update the commit log, it will help everybody understand.

Lorenzo

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: [PATCH V4 22/28] PCI: tegra: Access endpoint config only if PCIe link is up
  2019-06-13 15:42           ` Thierry Reding
@ 2019-06-17 10:01             ` Manikanta Maddireddy
  2019-06-17 11:47               ` Thierry Reding
  0 siblings, 1 reply; 72+ messages in thread
From: Manikanta Maddireddy @ 2019-06-17 10:01 UTC (permalink / raw)
  To: Thierry Reding, Lorenzo Pieralisi
  Cc: bhelgaas, robh+dt, mark.rutland, jonathanh, vidyas, linux-tegra,
	linux-pci, devicetree



On 13-Jun-19 9:12 PM, Thierry Reding wrote:
> On Thu, Jun 13, 2019 at 03:39:46PM +0100, Lorenzo Pieralisi wrote:
>> On Mon, Jun 10, 2019 at 10:08:16AM +0530, Manikanta Maddireddy wrote:
>>>
>>> On 04-Jun-19 7:40 PM, Manikanta Maddireddy wrote:
>>>> On 04-Jun-19 6:44 PM, Thierry Reding wrote:
>>>>> On Thu, May 16, 2019 at 11:23:01AM +0530, Manikanta Maddireddy wrote:
>>>>>> Few endpoints like Wi-Fi supports power on/off and to leverage that
>>>>>> root port must support hot-plug and hot-unplug. Tegra PCIe doesn't
>>>>>> support hot-plug and hot-unplug, however it supports endpoint power
>>>>>> on/off feature as follows,
>>>>>>  - Power off sequence:
>>>>>>    - Transition of PCIe link to L2
>>>>>>    - Power off endpoint
>>>>>>    - Leave root port in power up state with the link in L2
>>>>>>  - Power on sequence:
>>>>>>    - Power on endpoint
>>>>>>    - Apply hot reset to get PCIe link up
>>>>>>
>>>>>> PCIe client driver stops accessing PCIe endpoint config and BAR registers
>>>>>> after endpoint is powered off. However, software applications like x11
>>>>>> server or lspci can access endpoint config registers in which case
>>>>>> host controller raises "response decoding" errors. To avoid this scenario,
>>>>>> add PCIe link up check in config read and write callback functions before
>>>>>> accessing endpoint config registers.
>>>>>>
>>>>>> Signed-off-by: Manikanta Maddireddy <mmaddireddy@nvidia.com>
>>>>>> ---
>>>>>> V4: No change
>>>>>>
>>>>>> V3: Update the commit log with explanation for the need of this patch
>>>>>>
>>>>>> V2: Change tegra_pcie_link_status() to tegra_pcie_link_up()
>>>>>>
>>>>>>  drivers/pci/controller/pci-tegra.c | 38 ++++++++++++++++++++++++++++++
>>>>>>  1 file changed, 38 insertions(+)
>>>>> This still doesn't look right to me conceptually. If somebody wants to
>>>>> access the PCI devices after the kernel has powered them off, why can't
>>>>> we just power the devices back on so that we allow userspace to properly
>>>>> access the devices?
>>>> 1. WiFi devices provides power-off feature for power saving in mobiles.
>>>> When WiFi is turned off we shouldn't power on the HW back without user
>>>> turning it back on.
>>>> 2. When ever user process tries to access config space, it'll end up
>>>> in these functions. We cannot have is_powered_on check in config read/write
>>>> callbacks.
>>>> 3. WiFi power on/off is device specific feature, we shouldn't handle it
>>>> in PCI subsystem or host controller driver.
>>>>
>>>>> Or if that's not what we want, shouldn't we add something to the core
>>>>> PCI infrastructure to let us deal with this? It seems like this is some
>>>>> general problem that would apply to every PCI device and host bridge
>>>>> driver. Having each driver implement this logic separately doesn't seem
>>>>> like a good idea to me.
>>>>>
>>>>> Thierry
>>>> This should be handled by hotplug feature, whenever endpoint is powered-off/
>>>> removed from the slot, hot unplug event should take care of it. Unfortunately
>>>> Tegra PCIe doesn't support hotplug feature.
>>>>
>>>> Manikanta
>>> Hi Bjorn,
>>>
>>> I thought about your comment in
>>> https://patchwork.ozlabs.org/patch/1084204/ again.  What if I add link
>>> up check in tegra_pcie_isr() and make "response decoding error" as
>>> debug print? EP Config access will happen when link is down, but
>>> "Response decoding error" print comes only if debug log is enabled.
>>> This way we can avoid race issue in config accessors and we get prints
>>> when debug logs are enabled.
>> I still do not see what you are actually solving. This patch should
>> be dropped.
> The problem that Manikanta is trying to solve here occurs in this
> situation (Manikanta, correct me if I've got this wrong): on some
> setups, a WiFi module connected over PCI will toggle a power GPIO as
> part of runtime suspend. This effectively causes the module to disappear
> from the PCI bus (i.e. it can no longer be accessed until the power GPIO
> is toggled again).

GPIO is toggled as part of WiFi on/off, can be triggered from network manager UI.

>
> This is fine from a kernel point of view because the kernel keeps track
> of what devices are suspended. However, userspace will occasionally try
> to read the configuration space access of all devices, and since it
> doesn't have any knowledge about the suspend state of these devices, it
> doesn't know which ones to leave alone. I think this happens when the
> X.Org server is running.

This is fine from a kernel point of view because PCI client driver
doesn't initiate any PCIe transaction until network interface
is up during WiFi on.

>
> One thing that Manikanta and I had discussed was that perhaps the device
> should be hot-unplugged when it goes into this low-power state. However,
> we don't support hotplug on Tegra210 where this is needed, so we'd need
> some sort of software-induced hot-unplug. However, this low power state
> is entered when the WiFi interface is taken down (i.e. ip link set dev
> <interface> down). If we were to remove the PCI device in that case, it
> means that the interface goes away completely, which is completely
> unexpected from a user's perspective. After all, taking a link down and
> up may be something that scripts are doing all the time. They'd fall
> over if after taking the interface down, the interface completely
> disappears.
>
> It's also not entirely clear to me how we get the device back onto the
> bus again after it is in low power. If we hot-unplug the device, then
> the driver will be unbound. Presumably the driver is what's controlling
> the power GPIO, so there won't be any entity that can be used to bring
> the chip back to life. Unless we deal with that power GPIO elsewhere
> (rfkill switch perhaps?).

Correct, rfkill switch should handle the GPIO.
Sequence will be,
 - WiFi ON
   - rfkill switch enables the WiFi GPIO
   - Tegra PCIe receives hot plug event
   - Tegra PCIe hot plug driver rescans PCI bus and enumerates the device
   - PCI client driver is probed, which will create network interface
 - WiFi OFF
   - rfkill switch disables the WiFi GPIO
   - Tegra PCIe receives hot unplug event
   - Tegra PCIe hot plug driver removes PCI devices under the bus
   - PCI client driver remove is executed, which will remove network interface

We don't need current patch in this case because PCI device is not present
in the PCI hierarchy, so there cannot be EP config access with link down.
However Tegra doesn't support hot plug and unplug events. I am not sure
if we have any software based hot plug event trigger.

I will drop current patch and pursue if above sequence can be
implemented for Tegra.

Manikanta

>
> Perhaps one other way to deal with this would be to track the suspend
> state of devices and then have the code that implements the PCI access
> from userspace refuse accesses to devices that are asleep. I suppose
> this is somewhat of an odd use-case because traditionally I guess PCI
> devices never power down to a state where their configuration space can
> no longer be accessed. At least that's what would explain why this has
> never been an issue before. Or perhaps it has?
>
> The last resort would be to just never put the WiFi chip into that low
> power mode, though I'm not exactly sure what that means for the power
> consumption on the affected systems.
>
> Manikanta, can you fill in some of the blanks above?
>
> Thierry
>>> Thierry,
>>> Please share your inputs as well.
>>>
>>> Manikanta
>>>  
>>>
>>>>>> diff --git a/drivers/pci/controller/pci-tegra.c b/drivers/pci/controller/pci-tegra.c
>>>>>> index d20c88a79e00..33f4dfab9e35 100644
>>>>>> --- a/drivers/pci/controller/pci-tegra.c
>>>>>> +++ b/drivers/pci/controller/pci-tegra.c
>>>>>> @@ -428,6 +428,14 @@ static inline u32 pads_readl(struct tegra_pcie *pcie, unsigned long offset)
>>>>>>  	return readl(pcie->pads + offset);
>>>>>>  }
>>>>>>  
>>>>>> +static bool tegra_pcie_link_up(struct tegra_pcie_port *port)
>>>>>> +{
>>>>>> +	u32 value;
>>>>>> +
>>>>>> +	value = readl(port->base + RP_LINK_CONTROL_STATUS);
>>>>>> +	return !!(value & RP_LINK_CONTROL_STATUS_DL_LINK_ACTIVE);
>>>>>> +}
>>>>>> +
>>>>>>  /*
>>>>>>   * The configuration space mapping on Tegra is somewhat similar to the ECAM
>>>>>>   * defined by PCIe. However it deviates a bit in how the 4 bits for extended
>>>>>> @@ -493,20 +501,50 @@ static void __iomem *tegra_pcie_map_bus(struct pci_bus *bus,
>>>>>>  static int tegra_pcie_config_read(struct pci_bus *bus, unsigned int devfn,
>>>>>>  				  int where, int size, u32 *value)
>>>>>>  {
>>>>>> +	struct tegra_pcie *pcie = bus->sysdata;
>>>>>> +	struct pci_dev *bridge;
>>>>>> +	struct tegra_pcie_port *port;
>>>>>> +
>>>>>>  	if (bus->number == 0)
>>>>>>  		return pci_generic_config_read32(bus, devfn, where, size,
>>>>>>  						 value);
>>>>>>  
>>>>>> +	bridge = pcie_find_root_port(bus->self);
>>>>>> +
>>>>>> +	list_for_each_entry(port, &pcie->ports, list)
>>>>>> +		if (port->index + 1 == PCI_SLOT(bridge->devfn))
>>>>>> +			break;
>>>>>> +
>>>>>> +	/* If there is no link, then there is no device */
>>>>>> +	if (!tegra_pcie_link_up(port)) {
>>>>>> +		*value = 0xffffffff;
>>>>>> +		return PCIBIOS_DEVICE_NOT_FOUND;
>>>>>> +	}
>>>>>> +
>>>>>>  	return pci_generic_config_read(bus, devfn, where, size, value);
>>>>>>  }
>>>>>>  
>>>>>>  static int tegra_pcie_config_write(struct pci_bus *bus, unsigned int devfn,
>>>>>>  				   int where, int size, u32 value)
>>>>>>  {
>>>>>> +	struct tegra_pcie *pcie = bus->sysdata;
>>>>>> +	struct tegra_pcie_port *port;
>>>>>> +	struct pci_dev *bridge;
>>>>>> +
>>>>>>  	if (bus->number == 0)
>>>>>>  		return pci_generic_config_write32(bus, devfn, where, size,
>>>>>>  						  value);
>>>>>>  
>>>>>> +	bridge = pcie_find_root_port(bus->self);
>>>>>> +
>>>>>> +	list_for_each_entry(port, &pcie->ports, list)
>>>>>> +		if (port->index + 1 == PCI_SLOT(bridge->devfn))
>>>>>> +			break;
>>>>>> +
>>>>>> +	/* If there is no link, then there is no device */
>>>>>> +	if (!tegra_pcie_link_up(port))
>>>>>> +		return PCIBIOS_DEVICE_NOT_FOUND;
>>>>>> +
>>>>>>  	return pci_generic_config_write(bus, devfn, where, size, value);
>>>>>>  }
>>>>>>  
>>>>>> -- 
>>>>>> 2.17.1
>>>>>>


^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: [PATCH V4 27/28] PCI: tegra: Add support for GPIO based PERST#
  2019-06-17  9:48                         ` Lorenzo Pieralisi
@ 2019-06-17 10:27                           ` Manikanta Maddireddy
  2019-06-17 10:39                             ` Lorenzo Pieralisi
  0 siblings, 1 reply; 72+ messages in thread
From: Manikanta Maddireddy @ 2019-06-17 10:27 UTC (permalink / raw)
  To: Lorenzo Pieralisi
  Cc: Thierry Reding, bhelgaas, robh+dt, mark.rutland, jonathanh,
	vidyas, linux-tegra, linux-pci, devicetree



On 17-Jun-19 3:18 PM, Lorenzo Pieralisi wrote:
> On Fri, Jun 14, 2019 at 10:53:13PM +0530, Manikanta Maddireddy wrote:
>
> [...]
>
>>> Ok. My point then is that you have no way to enforce this requirement on
>>> platforms that actually need it, I do not even know if there is a
>>> way you can do it (I was thinking along the lines of using a
>>> compatible string to detect whether the GPIO #PERST reset is mandatory)
>>> but maybe this is not even a SOC property.
>>>
>>> Maybe what I am asking is overkill, I just wanted to understand.
>>>
>>> I was just asking a question to understand how you handle the case
>>> where a GPIO pin definition is missing in DT for a platform that
>>> actually needs it, the driver will probe but nothing will work.
>>>
>>> It would be good to describe this and capture it in the commit log.
>>>
>>> Thanks,
>>> Lorenzo
>> I can't think of a easy way to enforce this requirement. As you said
>> compatible string is per SOC, so we can't use it for a platform.
>> This issue is present on only one platform, so it is hard to miss the
>> DT property. That is the reason for publishing this patch with out this
>> enforcement in driver.
>>
>> I thought for changing PERST# to GPIO for all platform, but testing is
>> a tedious job. Also I don't have Tegra20 and Tegra30 platforms.
> I can't help with that.
>
>> Do you want me to drop the patch or update the limitation in the commit
>> log?
> It is Thierry's call, if he is OK with it fine by me, please do
> update the commit log, it will help everybody understand.
>
> Lorenzo

Sure, I will update the commit log in V5.
Please let me know if you completed reviewing this series, I will
send V5 addressing review comments in this patch.

Manikanta



^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: [PATCH V4 27/28] PCI: tegra: Add support for GPIO based PERST#
  2019-06-17 10:27                           ` Manikanta Maddireddy
@ 2019-06-17 10:39                             ` Lorenzo Pieralisi
  0 siblings, 0 replies; 72+ messages in thread
From: Lorenzo Pieralisi @ 2019-06-17 10:39 UTC (permalink / raw)
  To: Manikanta Maddireddy
  Cc: Thierry Reding, bhelgaas, robh+dt, mark.rutland, jonathanh,
	vidyas, linux-tegra, linux-pci, devicetree

On Mon, Jun 17, 2019 at 03:57:09PM +0530, Manikanta Maddireddy wrote:
> 
> 
> On 17-Jun-19 3:18 PM, Lorenzo Pieralisi wrote:
> > On Fri, Jun 14, 2019 at 10:53:13PM +0530, Manikanta Maddireddy wrote:
> >
> > [...]
> >
> >>> Ok. My point then is that you have no way to enforce this requirement on
> >>> platforms that actually need it, I do not even know if there is a
> >>> way you can do it (I was thinking along the lines of using a
> >>> compatible string to detect whether the GPIO #PERST reset is mandatory)
> >>> but maybe this is not even a SOC property.
> >>>
> >>> Maybe what I am asking is overkill, I just wanted to understand.
> >>>
> >>> I was just asking a question to understand how you handle the case
> >>> where a GPIO pin definition is missing in DT for a platform that
> >>> actually needs it, the driver will probe but nothing will work.
> >>>
> >>> It would be good to describe this and capture it in the commit log.
> >>>
> >>> Thanks,
> >>> Lorenzo
> >> I can't think of a easy way to enforce this requirement. As you said
> >> compatible string is per SOC, so we can't use it for a platform.
> >> This issue is present on only one platform, so it is hard to miss the
> >> DT property. That is the reason for publishing this patch with out this
> >> enforcement in driver.
> >>
> >> I thought for changing PERST# to GPIO for all platform, but testing is
> >> a tedious job. Also I don't have Tegra20 and Tegra30 platforms.
> > I can't help with that.
> >
> >> Do you want me to drop the patch or update the limitation in the commit
> >> log?
> > It is Thierry's call, if he is OK with it fine by me, please do
> > update the commit log, it will help everybody understand.
> >
> > Lorenzo
> 
> Sure, I will update the commit log in V5.
> Please let me know if you completed reviewing this series, I will
> send V5 addressing review comments in this patch.

Post v5, we should be able to get it in v5.3, thanks.

Lorenzo

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: [PATCH V4 27/28] PCI: tegra: Add support for GPIO based PERST#
  2019-06-14 16:53                     ` Lorenzo Pieralisi
  2019-06-14 17:23                       ` Manikanta Maddireddy
@ 2019-06-17 11:26                       ` Thierry Reding
  1 sibling, 0 replies; 72+ messages in thread
From: Thierry Reding @ 2019-06-17 11:26 UTC (permalink / raw)
  To: Lorenzo Pieralisi
  Cc: Manikanta Maddireddy, bhelgaas, robh+dt, mark.rutland, jonathanh,
	vidyas, linux-tegra, linux-pci, devicetree

[-- Attachment #1: Type: text/plain, Size: 3321 bytes --]

On Fri, Jun 14, 2019 at 05:53:53PM +0100, Lorenzo Pieralisi wrote:
> On Fri, Jun 14, 2019 at 10:00:49PM +0530, Manikanta Maddireddy wrote:
> 
> [...]
> 
> > GPIO based PERST# is per-platform requirement.
> > If DT prop is not present, then devm_gpiod_get_from_of_node() returns
> > NULL gpio_desc.
> > 
> > struct gpio_desc *gpiod_get_from_of_node(struct device_node *node,
> >                                          const char *propname, int index,
> >                                          enum gpiod_flags dflags,
> >                                          const char *label)
> > {
> >         struct gpio_desc *desc;
> >         unsigned long lflags = 0;
> >         enum of_gpio_flags flags;
> >         bool active_low = false;
> >         bool single_ended = false;
> >         bool open_drain = false;
> >         bool transitory = false;
> >         int ret;
> > 
> >         desc = of_get_named_gpiod_flags(node, propname,
> >                                         index, &flags);
> > 
> >         if (!desc || IS_ERR(desc)) {
> > */* If it is not there, just return NULL */****if (PTR_ERR(desc) == -ENOENT)****return NULL;*
> >                 return desc;
> >         }
> > 	...
> > 
> > }
> 
> Ok. My point then is that you have no way to enforce this requirement on
> platforms that actually need it, I do not even know if there is a
> way you can do it (I was thinking along the lines of using a
> compatible string to detect whether the GPIO #PERST reset is mandatory)
> but maybe this is not even a SOC property.

So this is definitely not an SoC property. From what Manikanta said, the
only reason why we need this is because on one particular design (that
we know of), the PCIe #PERST signal was connected to a pin that does not
originate from the PCIe controller. This happened by accident.

> Maybe what I am asking is overkill, I just wanted to understand.
> 
> I was just asking a question to understand how you handle the case
> where a GPIO pin definition is missing in DT for a platform that
> actually needs it, the driver will probe but nothing will work.

I think we should handle this the way we handle other GPIOs as well. If
some device requires a GPIO to be toggled to put it into or take it out
of reset, then that's something that needs to be described in DT. In
this case it just so happens that typically we don't need to worry about
it because the signals are properly connected and then the PCIe
controller will do the right with the reset. For the one design where
it doesn't work the reset GPIO is a workaround and it's board-specific
knowledge that the engineer writing the DT will have to know. I suspect
that if the same accident happened on another board, then as part of
writing the DT somebody would have noticed that we need an external pin
to be hooked up because the SFIO doesn't work.

> It would be good to describe this and capture it in the commit log.

Agreed. Let's be more verbose about the situation where this is required
and make it very clear that this is a workaround for a board design
mistake that shouldn't be necessary if best practices are followed.

Maybe add that to both the commit message and the device tree bindings
to make it difficult for people to miss.

Thierry

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: [PATCH V4 27/28] PCI: tegra: Add support for GPIO based PERST#
  2019-06-14 17:23                       ` Manikanta Maddireddy
  2019-06-17  9:48                         ` Lorenzo Pieralisi
@ 2019-06-17 11:29                         ` Thierry Reding
  1 sibling, 0 replies; 72+ messages in thread
From: Thierry Reding @ 2019-06-17 11:29 UTC (permalink / raw)
  To: Manikanta Maddireddy
  Cc: Lorenzo Pieralisi, bhelgaas, robh+dt, mark.rutland, jonathanh,
	vidyas, linux-tegra, linux-pci, devicetree

[-- Attachment #1: Type: text/plain, Size: 2943 bytes --]

On Fri, Jun 14, 2019 at 10:53:13PM +0530, Manikanta Maddireddy wrote:
> 
> 
> On 14-Jun-19 10:23 PM, Lorenzo Pieralisi wrote:
> > On Fri, Jun 14, 2019 at 10:00:49PM +0530, Manikanta Maddireddy wrote:
> >
> > [...]
> >
> >> GPIO based PERST# is per-platform requirement.
> >> If DT prop is not present, then devm_gpiod_get_from_of_node() returns
> >> NULL gpio_desc.
> >>
> >> struct gpio_desc *gpiod_get_from_of_node(struct device_node *node,
> >>                                          const char *propname, int index,
> >>                                          enum gpiod_flags dflags,
> >>                                          const char *label)
> >> {
> >>         struct gpio_desc *desc;
> >>         unsigned long lflags = 0;
> >>         enum of_gpio_flags flags;
> >>         bool active_low = false;
> >>         bool single_ended = false;
> >>         bool open_drain = false;
> >>         bool transitory = false;
> >>         int ret;
> >>
> >>         desc = of_get_named_gpiod_flags(node, propname,
> >>                                         index, &flags);
> >>
> >>         if (!desc || IS_ERR(desc)) {
> >> */* If it is not there, just return NULL */****if (PTR_ERR(desc) == -ENOENT)****return NULL;*
> >>                 return desc;
> >>         }
> >> 	...
> >>
> >> }
> > Ok. My point then is that you have no way to enforce this requirement on
> > platforms that actually need it, I do not even know if there is a
> > way you can do it (I was thinking along the lines of using a
> > compatible string to detect whether the GPIO #PERST reset is mandatory)
> > but maybe this is not even a SOC property.
> >
> > Maybe what I am asking is overkill, I just wanted to understand.
> >
> > I was just asking a question to understand how you handle the case
> > where a GPIO pin definition is missing in DT for a platform that
> > actually needs it, the driver will probe but nothing will work.
> >
> > It would be good to describe this and capture it in the commit log.
> >
> > Thanks,
> > Lorenzo
> 
> I can't think of a easy way to enforce this requirement. As you said
> compatible string is per SOC, so we can't use it for a platform.
> This issue is present on only one platform, so it is hard to miss the
> DT property. That is the reason for publishing this patch with out this
> enforcement in driver.
> 
> I thought for changing PERST# to GPIO for all platform, but testing is
> a tedious job. Also I don't have Tegra20 and Tegra30 platforms.

Yeah, let's not go that way. The standard way to do this is to use the
SFIO and let the PCIe controller and driver handle this. It's working
just fine on all platforms currently supported upstream. Using direct
GPIO for PERST# is a workaround, so let's not proliferate unless it is
absolutely necessary.

With an updated commit message, this is:

Acked-by: Thierry Reding <treding@nvidia.com>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: [PATCH V4 26/28] PCI: Add DT binding for "reset-gpios" property
  2019-05-16  5:53 ` [PATCH V4 26/28] PCI: Add DT binding for "reset-gpios" property Manikanta Maddireddy
@ 2019-06-17 11:30   ` Thierry Reding
  2019-06-17 11:38     ` Manikanta Maddireddy
  0 siblings, 1 reply; 72+ messages in thread
From: Thierry Reding @ 2019-06-17 11:30 UTC (permalink / raw)
  To: Manikanta Maddireddy
  Cc: bhelgaas, robh+dt, mark.rutland, jonathanh, lorenzo.pieralisi,
	vidyas, linux-tegra, linux-pci, devicetree

[-- Attachment #1: Type: text/plain, Size: 1430 bytes --]

On Thu, May 16, 2019 at 11:23:05AM +0530, Manikanta Maddireddy wrote:
> Add DT binding for "reset-gpios" property which supports GPIO based PERST#
> signal.
> 
> Signed-off-by: Manikanta Maddireddy <mmaddireddy@nvidia.com>
> Reviewed-by: Rob Herring <robh@kernel.org>
> Acked-by: Thierry Reding <treding@nvidia.com>
> ---
> V4: No change
> 
> V3: Moved to common pci binding doc
> 
> V2: Using standard "reset-gpio" property
> 
>  Documentation/devicetree/bindings/pci/pci.txt | 3 +++
>  1 file changed, 3 insertions(+)
> 
> diff --git a/Documentation/devicetree/bindings/pci/pci.txt b/Documentation/devicetree/bindings/pci/pci.txt
> index c77981c5dd18..79124898aa5b 100644
> --- a/Documentation/devicetree/bindings/pci/pci.txt
> +++ b/Documentation/devicetree/bindings/pci/pci.txt
> @@ -24,3 +24,6 @@ driver implementation may support the following properties:
>     unsupported link speed, for instance, trying to do training for
>     unsupported link speed, etc.  Must be '4' for gen4, '3' for gen3, '2'
>     for gen2, and '1' for gen1. Any other values are invalid.
> +- reset-gpios:
> +   If present this property specifies PERST# GPIO. Host drivers can parse the
> +   GPIO and apply fundamental reset to endpoints.

As mentioned in patch 27/28, maybe mention here that this is only a
workaround for bad board designs and that it shouldn't be necessary in
the majority of cases.

Thierry

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: [PATCH V4 26/28] PCI: Add DT binding for "reset-gpios" property
  2019-06-17 11:30   ` Thierry Reding
@ 2019-06-17 11:38     ` Manikanta Maddireddy
  2019-06-17 11:48       ` Thierry Reding
  0 siblings, 1 reply; 72+ messages in thread
From: Manikanta Maddireddy @ 2019-06-17 11:38 UTC (permalink / raw)
  To: Thierry Reding
  Cc: bhelgaas, robh+dt, mark.rutland, jonathanh, lorenzo.pieralisi,
	vidyas, linux-tegra, linux-pci, devicetree



On 17-Jun-19 5:00 PM, Thierry Reding wrote:
> On Thu, May 16, 2019 at 11:23:05AM +0530, Manikanta Maddireddy wrote:
>> Add DT binding for "reset-gpios" property which supports GPIO based PERST#
>> signal.
>>
>> Signed-off-by: Manikanta Maddireddy <mmaddireddy@nvidia.com>
>> Reviewed-by: Rob Herring <robh@kernel.org>
>> Acked-by: Thierry Reding <treding@nvidia.com>
>> ---
>> V4: No change
>>
>> V3: Moved to common pci binding doc
>>
>> V2: Using standard "reset-gpio" property
>>
>>  Documentation/devicetree/bindings/pci/pci.txt | 3 +++
>>  1 file changed, 3 insertions(+)
>>
>> diff --git a/Documentation/devicetree/bindings/pci/pci.txt b/Documentation/devicetree/bindings/pci/pci.txt
>> index c77981c5dd18..79124898aa5b 100644
>> --- a/Documentation/devicetree/bindings/pci/pci.txt
>> +++ b/Documentation/devicetree/bindings/pci/pci.txt
>> @@ -24,3 +24,6 @@ driver implementation may support the following properties:
>>     unsupported link speed, for instance, trying to do training for
>>     unsupported link speed, etc.  Must be '4' for gen4, '3' for gen3, '2'
>>     for gen2, and '1' for gen1. Any other values are invalid.
>> +- reset-gpios:
>> +   If present this property specifies PERST# GPIO. Host drivers can parse the
>> +   GPIO and apply fundamental reset to endpoints.
> As mentioned in patch 27/28, maybe mention here that this is only a
> workaround for bad board designs and that it shouldn't be necessary in
> the majority of cases.
>
> Thierry

This is common DT binding doc, I cannot add Tegra specific here.
reset-gpios is common DT prop, so Rob asked me to add it in common file.

Manikanta 


^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: [PATCH V4 22/28] PCI: tegra: Access endpoint config only if PCIe link is up
  2019-06-17 10:01             ` Manikanta Maddireddy
@ 2019-06-17 11:47               ` Thierry Reding
  2019-06-17 19:30                 ` Bjorn Helgaas
  0 siblings, 1 reply; 72+ messages in thread
From: Thierry Reding @ 2019-06-17 11:47 UTC (permalink / raw)
  To: Manikanta Maddireddy
  Cc: Lorenzo Pieralisi, bhelgaas, robh+dt, mark.rutland, jonathanh,
	vidyas, linux-tegra, linux-pci, devicetree

[-- Attachment #1: Type: text/plain, Size: 12403 bytes --]

On Mon, Jun 17, 2019 at 03:31:38PM +0530, Manikanta Maddireddy wrote:
> 
> 
> On 13-Jun-19 9:12 PM, Thierry Reding wrote:
> > On Thu, Jun 13, 2019 at 03:39:46PM +0100, Lorenzo Pieralisi wrote:
> >> On Mon, Jun 10, 2019 at 10:08:16AM +0530, Manikanta Maddireddy wrote:
> >>>
> >>> On 04-Jun-19 7:40 PM, Manikanta Maddireddy wrote:
> >>>> On 04-Jun-19 6:44 PM, Thierry Reding wrote:
> >>>>> On Thu, May 16, 2019 at 11:23:01AM +0530, Manikanta Maddireddy wrote:
> >>>>>> Few endpoints like Wi-Fi supports power on/off and to leverage that
> >>>>>> root port must support hot-plug and hot-unplug. Tegra PCIe doesn't
> >>>>>> support hot-plug and hot-unplug, however it supports endpoint power
> >>>>>> on/off feature as follows,
> >>>>>>  - Power off sequence:
> >>>>>>    - Transition of PCIe link to L2
> >>>>>>    - Power off endpoint
> >>>>>>    - Leave root port in power up state with the link in L2
> >>>>>>  - Power on sequence:
> >>>>>>    - Power on endpoint
> >>>>>>    - Apply hot reset to get PCIe link up
> >>>>>>
> >>>>>> PCIe client driver stops accessing PCIe endpoint config and BAR registers
> >>>>>> after endpoint is powered off. However, software applications like x11
> >>>>>> server or lspci can access endpoint config registers in which case
> >>>>>> host controller raises "response decoding" errors. To avoid this scenario,
> >>>>>> add PCIe link up check in config read and write callback functions before
> >>>>>> accessing endpoint config registers.
> >>>>>>
> >>>>>> Signed-off-by: Manikanta Maddireddy <mmaddireddy@nvidia.com>
> >>>>>> ---
> >>>>>> V4: No change
> >>>>>>
> >>>>>> V3: Update the commit log with explanation for the need of this patch
> >>>>>>
> >>>>>> V2: Change tegra_pcie_link_status() to tegra_pcie_link_up()
> >>>>>>
> >>>>>>  drivers/pci/controller/pci-tegra.c | 38 ++++++++++++++++++++++++++++++
> >>>>>>  1 file changed, 38 insertions(+)
> >>>>> This still doesn't look right to me conceptually. If somebody wants to
> >>>>> access the PCI devices after the kernel has powered them off, why can't
> >>>>> we just power the devices back on so that we allow userspace to properly
> >>>>> access the devices?
> >>>> 1. WiFi devices provides power-off feature for power saving in mobiles.
> >>>> When WiFi is turned off we shouldn't power on the HW back without user
> >>>> turning it back on.
> >>>> 2. When ever user process tries to access config space, it'll end up
> >>>> in these functions. We cannot have is_powered_on check in config read/write
> >>>> callbacks.
> >>>> 3. WiFi power on/off is device specific feature, we shouldn't handle it
> >>>> in PCI subsystem or host controller driver.
> >>>>
> >>>>> Or if that's not what we want, shouldn't we add something to the core
> >>>>> PCI infrastructure to let us deal with this? It seems like this is some
> >>>>> general problem that would apply to every PCI device and host bridge
> >>>>> driver. Having each driver implement this logic separately doesn't seem
> >>>>> like a good idea to me.
> >>>>>
> >>>>> Thierry
> >>>> This should be handled by hotplug feature, whenever endpoint is powered-off/
> >>>> removed from the slot, hot unplug event should take care of it. Unfortunately
> >>>> Tegra PCIe doesn't support hotplug feature.
> >>>>
> >>>> Manikanta
> >>> Hi Bjorn,
> >>>
> >>> I thought about your comment in
> >>> https://patchwork.ozlabs.org/patch/1084204/ again.  What if I add link
> >>> up check in tegra_pcie_isr() and make "response decoding error" as
> >>> debug print? EP Config access will happen when link is down, but
> >>> "Response decoding error" print comes only if debug log is enabled.
> >>> This way we can avoid race issue in config accessors and we get prints
> >>> when debug logs are enabled.
> >> I still do not see what you are actually solving. This patch should
> >> be dropped.
> > The problem that Manikanta is trying to solve here occurs in this
> > situation (Manikanta, correct me if I've got this wrong): on some
> > setups, a WiFi module connected over PCI will toggle a power GPIO as
> > part of runtime suspend. This effectively causes the module to disappear
> > from the PCI bus (i.e. it can no longer be accessed until the power GPIO
> > is toggled again).
> 
> GPIO is toggled as part of WiFi on/off, can be triggered from network manager UI.
> 
> >
> > This is fine from a kernel point of view because the kernel keeps track
> > of what devices are suspended. However, userspace will occasionally try
> > to read the configuration space access of all devices, and since it
> > doesn't have any knowledge about the suspend state of these devices, it
> > doesn't know which ones to leave alone. I think this happens when the
> > X.Org server is running.
> 
> This is fine from a kernel point of view because PCI client driver
> doesn't initiate any PCIe transaction until network interface
> is up during WiFi on.
> 
> >
> > One thing that Manikanta and I had discussed was that perhaps the device
> > should be hot-unplugged when it goes into this low-power state. However,
> > we don't support hotplug on Tegra210 where this is needed, so we'd need
> > some sort of software-induced hot-unplug. However, this low power state
> > is entered when the WiFi interface is taken down (i.e. ip link set dev
> > <interface> down). If we were to remove the PCI device in that case, it
> > means that the interface goes away completely, which is completely
> > unexpected from a user's perspective. After all, taking a link down and
> > up may be something that scripts are doing all the time. They'd fall
> > over if after taking the interface down, the interface completely
> > disappears.
> >
> > It's also not entirely clear to me how we get the device back onto the
> > bus again after it is in low power. If we hot-unplug the device, then
> > the driver will be unbound. Presumably the driver is what's controlling
> > the power GPIO, so there won't be any entity that can be used to bring
> > the chip back to life. Unless we deal with that power GPIO elsewhere
> > (rfkill switch perhaps?).
> 
> Correct, rfkill switch should handle the GPIO.
> Sequence will be,
>  - WiFi ON
>    - rfkill switch enables the WiFi GPIO
>    - Tegra PCIe receives hot plug event
>    - Tegra PCIe hot plug driver rescans PCI bus and enumerates the device
>    - PCI client driver is probed, which will create network interface
>  - WiFi OFF
>    - rfkill switch disables the WiFi GPIO
>    - Tegra PCIe receives hot unplug event
>    - Tegra PCIe hot plug driver removes PCI devices under the bus
>    - PCI client driver remove is executed, which will remove network interface
> 
> We don't need current patch in this case because PCI device is not present
> in the PCI hierarchy, so there cannot be EP config access with link down.
> However Tegra doesn't support hot plug and unplug events. I am not sure
> if we have any software based hot plug event trigger.
> 
> I will drop current patch and pursue if above sequence can be
> implemented for Tegra.

I just recalled that we have these messages in the kernel log:

	# dmesg | grep tegra-pcie
	[    1.055761] tegra-pcie 1003000.pcie: 4x1, 1x1 configuration
	[    2.745764] tegra-pcie 1003000.pcie: 4x1, 1x1 configuration
	[    2.753073] tegra-pcie 1003000.pcie: probing port 0, using 4 lanes
	[    2.761334] tegra-pcie 1003000.pcie: Slot present pin change, signature: 00000008
	[    3.177607] tegra-pcie 1003000.pcie: link 0 down, retrying
	[    3.585605] tegra-pcie 1003000.pcie: link 0 down, retrying
	[    3.993606] tegra-pcie 1003000.pcie: link 0 down, retrying
	[    4.001214] tegra-pcie 1003000.pcie: link 0 down, ignoring
	[    4.006733] tegra-pcie 1003000.pcie: probing port 1, using 1 lanes
	[    4.015042] tegra-pcie 1003000.pcie: Slot present pin change, signature: 00000000
	[    4.031177] tegra-pcie 1003000.pcie: PCI host bridge to bus 0000:00

These "slot present pin change" message do look a lot like hotplug
related messages. Could we perhaps use those to our advantage for this
case? Do you see these when you run on the platform where WiFi is
enabled/disabled using rfkill?

Given that rfkill is completely decoupled from PCI, I don't see how we
would trigger any software-based hotplug mechanism. Perhaps one thing
that we could do is the equivalent of this:

	# echo 1 > /sys/bus/pci/rescan

from some script that's perhaps tied to the rfkill somehow. I'm not sure
if that's possible, or generic enough.

Thierry

> > Perhaps one other way to deal with this would be to track the suspend
> > state of devices and then have the code that implements the PCI access
> > from userspace refuse accesses to devices that are asleep. I suppose
> > this is somewhat of an odd use-case because traditionally I guess PCI
> > devices never power down to a state where their configuration space can
> > no longer be accessed. At least that's what would explain why this has
> > never been an issue before. Or perhaps it has?
> >
> > The last resort would be to just never put the WiFi chip into that low
> > power mode, though I'm not exactly sure what that means for the power
> > consumption on the affected systems.
> >
> > Manikanta, can you fill in some of the blanks above?
> >
> > Thierry
> >>> Thierry,
> >>> Please share your inputs as well.
> >>>
> >>> Manikanta
> >>>  
> >>>
> >>>>>> diff --git a/drivers/pci/controller/pci-tegra.c b/drivers/pci/controller/pci-tegra.c
> >>>>>> index d20c88a79e00..33f4dfab9e35 100644
> >>>>>> --- a/drivers/pci/controller/pci-tegra.c
> >>>>>> +++ b/drivers/pci/controller/pci-tegra.c
> >>>>>> @@ -428,6 +428,14 @@ static inline u32 pads_readl(struct tegra_pcie *pcie, unsigned long offset)
> >>>>>>  	return readl(pcie->pads + offset);
> >>>>>>  }
> >>>>>>  
> >>>>>> +static bool tegra_pcie_link_up(struct tegra_pcie_port *port)
> >>>>>> +{
> >>>>>> +	u32 value;
> >>>>>> +
> >>>>>> +	value = readl(port->base + RP_LINK_CONTROL_STATUS);
> >>>>>> +	return !!(value & RP_LINK_CONTROL_STATUS_DL_LINK_ACTIVE);
> >>>>>> +}
> >>>>>> +
> >>>>>>  /*
> >>>>>>   * The configuration space mapping on Tegra is somewhat similar to the ECAM
> >>>>>>   * defined by PCIe. However it deviates a bit in how the 4 bits for extended
> >>>>>> @@ -493,20 +501,50 @@ static void __iomem *tegra_pcie_map_bus(struct pci_bus *bus,
> >>>>>>  static int tegra_pcie_config_read(struct pci_bus *bus, unsigned int devfn,
> >>>>>>  				  int where, int size, u32 *value)
> >>>>>>  {
> >>>>>> +	struct tegra_pcie *pcie = bus->sysdata;
> >>>>>> +	struct pci_dev *bridge;
> >>>>>> +	struct tegra_pcie_port *port;
> >>>>>> +
> >>>>>>  	if (bus->number == 0)
> >>>>>>  		return pci_generic_config_read32(bus, devfn, where, size,
> >>>>>>  						 value);
> >>>>>>  
> >>>>>> +	bridge = pcie_find_root_port(bus->self);
> >>>>>> +
> >>>>>> +	list_for_each_entry(port, &pcie->ports, list)
> >>>>>> +		if (port->index + 1 == PCI_SLOT(bridge->devfn))
> >>>>>> +			break;
> >>>>>> +
> >>>>>> +	/* If there is no link, then there is no device */
> >>>>>> +	if (!tegra_pcie_link_up(port)) {
> >>>>>> +		*value = 0xffffffff;
> >>>>>> +		return PCIBIOS_DEVICE_NOT_FOUND;
> >>>>>> +	}
> >>>>>> +
> >>>>>>  	return pci_generic_config_read(bus, devfn, where, size, value);
> >>>>>>  }
> >>>>>>  
> >>>>>>  static int tegra_pcie_config_write(struct pci_bus *bus, unsigned int devfn,
> >>>>>>  				   int where, int size, u32 value)
> >>>>>>  {
> >>>>>> +	struct tegra_pcie *pcie = bus->sysdata;
> >>>>>> +	struct tegra_pcie_port *port;
> >>>>>> +	struct pci_dev *bridge;
> >>>>>> +
> >>>>>>  	if (bus->number == 0)
> >>>>>>  		return pci_generic_config_write32(bus, devfn, where, size,
> >>>>>>  						  value);
> >>>>>>  
> >>>>>> +	bridge = pcie_find_root_port(bus->self);
> >>>>>> +
> >>>>>> +	list_for_each_entry(port, &pcie->ports, list)
> >>>>>> +		if (port->index + 1 == PCI_SLOT(bridge->devfn))
> >>>>>> +			break;
> >>>>>> +
> >>>>>> +	/* If there is no link, then there is no device */
> >>>>>> +	if (!tegra_pcie_link_up(port))
> >>>>>> +		return PCIBIOS_DEVICE_NOT_FOUND;
> >>>>>> +
> >>>>>>  	return pci_generic_config_write(bus, devfn, where, size, value);
> >>>>>>  }
> >>>>>>  
> >>>>>> -- 
> >>>>>> 2.17.1
> >>>>>>
> 

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: [PATCH V4 26/28] PCI: Add DT binding for "reset-gpios" property
  2019-06-17 11:38     ` Manikanta Maddireddy
@ 2019-06-17 11:48       ` Thierry Reding
  0 siblings, 0 replies; 72+ messages in thread
From: Thierry Reding @ 2019-06-17 11:48 UTC (permalink / raw)
  To: Manikanta Maddireddy
  Cc: bhelgaas, robh+dt, mark.rutland, jonathanh, lorenzo.pieralisi,
	vidyas, linux-tegra, linux-pci, devicetree

[-- Attachment #1: Type: text/plain, Size: 1887 bytes --]

On Mon, Jun 17, 2019 at 05:08:45PM +0530, Manikanta Maddireddy wrote:
> 
> 
> On 17-Jun-19 5:00 PM, Thierry Reding wrote:
> > On Thu, May 16, 2019 at 11:23:05AM +0530, Manikanta Maddireddy wrote:
> >> Add DT binding for "reset-gpios" property which supports GPIO based PERST#
> >> signal.
> >>
> >> Signed-off-by: Manikanta Maddireddy <mmaddireddy@nvidia.com>
> >> Reviewed-by: Rob Herring <robh@kernel.org>
> >> Acked-by: Thierry Reding <treding@nvidia.com>
> >> ---
> >> V4: No change
> >>
> >> V3: Moved to common pci binding doc
> >>
> >> V2: Using standard "reset-gpio" property
> >>
> >>  Documentation/devicetree/bindings/pci/pci.txt | 3 +++
> >>  1 file changed, 3 insertions(+)
> >>
> >> diff --git a/Documentation/devicetree/bindings/pci/pci.txt b/Documentation/devicetree/bindings/pci/pci.txt
> >> index c77981c5dd18..79124898aa5b 100644
> >> --- a/Documentation/devicetree/bindings/pci/pci.txt
> >> +++ b/Documentation/devicetree/bindings/pci/pci.txt
> >> @@ -24,3 +24,6 @@ driver implementation may support the following properties:
> >>     unsupported link speed, for instance, trying to do training for
> >>     unsupported link speed, etc.  Must be '4' for gen4, '3' for gen3, '2'
> >>     for gen2, and '1' for gen1. Any other values are invalid.
> >> +- reset-gpios:
> >> +   If present this property specifies PERST# GPIO. Host drivers can parse the
> >> +   GPIO and apply fundamental reset to endpoints.
> > As mentioned in patch 27/28, maybe mention here that this is only a
> > workaround for bad board designs and that it shouldn't be necessary in
> > the majority of cases.
> >
> > Thierry
> 
> This is common DT binding doc, I cannot add Tegra specific here.
> reset-gpios is common DT prop, so Rob asked me to add it in common file.

Ah, indeed. Alright, let's document it in the Tegra driver patch, then.

Thierry

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: [PATCH V4 22/28] PCI: tegra: Access endpoint config only if PCIe link is up
  2019-06-17 11:47               ` Thierry Reding
@ 2019-06-17 19:30                 ` Bjorn Helgaas
  2019-06-18  5:36                   ` Manikanta Maddireddy
  0 siblings, 1 reply; 72+ messages in thread
From: Bjorn Helgaas @ 2019-06-17 19:30 UTC (permalink / raw)
  To: Thierry Reding
  Cc: Manikanta Maddireddy, Lorenzo Pieralisi, robh+dt, mark.rutland,
	jonathanh, vidyas, linux-tegra, linux-pci, devicetree, linux-pm,
	Rafael J. Wysocki

[+cc Rafael, linux-pm, in case they have insights on how rfkill works]

On Mon, Jun 17, 2019 at 01:47:45PM +0200, Thierry Reding wrote:
> On Mon, Jun 17, 2019 at 03:31:38PM +0530, Manikanta Maddireddy wrote:
> > On 13-Jun-19 9:12 PM, Thierry Reding wrote:
> > > On Thu, Jun 13, 2019 at 03:39:46PM +0100, Lorenzo Pieralisi wrote:
> > >> On Mon, Jun 10, 2019 at 10:08:16AM +0530, Manikanta Maddireddy wrote:
> > >>> On 04-Jun-19 7:40 PM, Manikanta Maddireddy wrote:
> > >>>> On 04-Jun-19 6:44 PM, Thierry Reding wrote:
> > >>>>> On Thu, May 16, 2019 at 11:23:01AM +0530, Manikanta Maddireddy wrote:
> > >>>>>> Few endpoints like Wi-Fi supports power on/off and to leverage that
> > >>>>>> root port must support hot-plug and hot-unplug. Tegra PCIe doesn't
> > >>>>>> support hot-plug and hot-unplug, however it supports endpoint power
> > >>>>>> on/off feature as follows,
> > >>>>>>  - Power off sequence:
> > >>>>>>    - Transition of PCIe link to L2
> > >>>>>>    - Power off endpoint
> > >>>>>>    - Leave root port in power up state with the link in L2
> > >>>>>>  - Power on sequence:
> > >>>>>>    - Power on endpoint
> > >>>>>>    - Apply hot reset to get PCIe link up
> > >>>>>>
> > >>>>>> PCIe client driver stops accessing PCIe endpoint config and
> > >>>>>> BAR registers after endpoint is powered off. However,
> > >>>>>> software applications like x11 server or lspci can access
> > >>>>>> endpoint config registers in which case host controller
> > >>>>>> raises "response decoding" errors. To avoid this scenario,
> > >>>>>> add PCIe link up check in config read and write callback
> > >>>>>> functions before accessing endpoint config registers.

> > >>>>>> Signed-off-by: Manikanta Maddireddy <mmaddireddy@nvidia.com>
> > >>>>>> ---
> > >>>>>> V4: No change
> > >>>>>>
> > >>>>>> V3: Update the commit log with explanation for the need of this patch
> > >>>>>>
> > >>>>>> V2: Change tegra_pcie_link_status() to tegra_pcie_link_up()
> > >>>>>>
> > >>>>>>  drivers/pci/controller/pci-tegra.c | 38 ++++++++++++++++++++++++++++++
> > >>>>>>  1 file changed, 38 insertions(+)

> > >>>>> This still doesn't look right to me conceptually. If
> > >>>>> somebody wants to access the PCI devices after the kernel
> > >>>>> has powered them off, why can't we just power the devices
> > >>>>> back on so that we allow userspace to properly access the
> > >>>>> devices?

> > >>>> 1. WiFi devices provides power-off feature for power saving
> > >>>> in mobiles.  When WiFi is turned off we shouldn't power on
> > >>>> the HW back without user turning it back on.

> > >>>> 2. When ever user process tries to access config space, it'll
> > >>>> end up in these functions. We cannot have is_powered_on check
> > >>>> in config read/write callbacks.

> > >>>> 3. WiFi power on/off is device specific feature, we shouldn't
> > >>>> handle it in PCI subsystem or host controller driver.

> > >>>>
> > >>>>> Or if that's not what we want, shouldn't we add something to
> > >>>>> the core PCI infrastructure to let us deal with this? It
> > >>>>> seems like this is some general problem that would apply to
> > >>>>> every PCI device and host bridge driver. Having each driver
> > >>>>> implement this logic separately doesn't seem like a good
> > >>>>> idea to me.

> > >>>> This should be handled by hotplug feature, whenever endpoint
> > >>>> is powered-off/ removed from the slot, hot unplug event
> > >>>> should take care of it. Unfortunately Tegra PCIe doesn't
> > >>>> support hotplug feature.

> > >>> I thought about your comment in
> > >>> https://patchwork.ozlabs.org/patch/1084204/ again.  What if I
> > >>> add link up check in tegra_pcie_isr() and make "response
> > >>> decoding error" as debug print? EP Config access will happen
> > >>> when link is down, but "Response decoding error" print comes
> > >>> only if debug log is enabled.  This way we can avoid race
> > >>> issue in config accessors and we get prints when debug logs
> > >>> are enabled.

> > > The problem that Manikanta is trying to solve here occurs in
> > > this situation (Manikanta, correct me if I've got this wrong):
> > > on some setups, a WiFi module connected over PCI will toggle a
> > > power GPIO as part of runtime suspend. This effectively causes
> > > the module to disappear from the PCI bus (i.e. it can no longer
> > > be accessed until the power GPIO is toggled again).
> > 
> > GPIO is toggled as part of WiFi on/off, can be triggered from
> > network manager UI.

> > > This is fine from a kernel point of view because the kernel keeps track
> > > of what devices are suspended. However, userspace will occasionally try
> > > to read the configuration space access of all devices, and since it
> > > doesn't have any knowledge about the suspend state of these devices, it
> > > doesn't know which ones to leave alone. I think this happens when the
> > > X.Org server is running.
> > 
> > This is fine from a kernel point of view because PCI client driver
> > doesn't initiate any PCIe transaction until network interface
> > is up during WiFi on.
> > 
> > > One thing that Manikanta and I had discussed was that perhaps
> > > the device should be hot-unplugged when it goes into this
> > > low-power state. However, we don't support hotplug on Tegra210
> > > where this is needed, so we'd need some sort of software-induced
> > > hot-unplug. However, this low power state is entered when the
> > > WiFi interface is taken down (i.e. ip link set dev <interface>
> > > down). If we were to remove the PCI device in that case, it
> > > means that the interface goes away completely, which is
> > > completely unexpected from a user's perspective. After all,
> > > taking a link down and up may be something that scripts are
> > > doing all the time. They'd fall over if after taking the
> > > interface down, the interface completely disappears.

> > > It's also not entirely clear to me how we get the device back
> > > onto the bus again after it is in low power. If we hot-unplug
> > > the device, then the driver will be unbound. Presumably the
> > > driver is what's controlling the power GPIO, so there won't be
> > > any entity that can be used to bring the chip back to life.
> > > Unless we deal with that power GPIO elsewhere (rfkill switch
> > > perhaps?).

> > Correct, rfkill switch should handle the GPIO.
> > Sequence will be,
> >  - WiFi ON
> >    - rfkill switch enables the WiFi GPIO
> >    - Tegra PCIe receives hot plug event
> >    - Tegra PCIe hot plug driver rescans PCI bus and enumerates the device
> >    - PCI client driver is probed, which will create network interface
> >  - WiFi OFF
> >    - rfkill switch disables the WiFi GPIO
> >    - Tegra PCIe receives hot unplug event
> >    - Tegra PCIe hot plug driver removes PCI devices under the bus
> >    - PCI client driver remove is executed, which will remove
> >      network interface

> > We don't need current patch in this case because PCI device is not
> > present in the PCI hierarchy, so there cannot be EP config access
> > with link down.  However Tegra doesn't support hot plug and unplug
> > events. I am not sure if we have any software based hot plug event
> > trigger.

> > I will drop current patch and pursue if above sequence can be
> > implemented for Tegra.
> 
> I just recalled that we have these messages in the kernel log:
> 
> 	# dmesg | grep tegra-pcie
> 	[    1.055761] tegra-pcie 1003000.pcie: 4x1, 1x1 configuration
> 	[    2.745764] tegra-pcie 1003000.pcie: 4x1, 1x1 configuration
> 	[    2.753073] tegra-pcie 1003000.pcie: probing port 0, using 4 lanes
> 	[    2.761334] tegra-pcie 1003000.pcie: Slot present pin change, signature: 00000008
> 	[    3.177607] tegra-pcie 1003000.pcie: link 0 down, retrying
> 	[    3.585605] tegra-pcie 1003000.pcie: link 0 down, retrying
> 	[    3.993606] tegra-pcie 1003000.pcie: link 0 down, retrying
> 	[    4.001214] tegra-pcie 1003000.pcie: link 0 down, ignoring
> 	[    4.006733] tegra-pcie 1003000.pcie: probing port 1, using 1 lanes
> 	[    4.015042] tegra-pcie 1003000.pcie: Slot present pin change, signature: 00000000
> 	[    4.031177] tegra-pcie 1003000.pcie: PCI host bridge to bus 0000:00
> 
> These "slot present pin change" message do look a lot like hotplug
> related messages. Could we perhaps use those to our advantage for this
> case? Do you see these when you run on the platform where WiFi is
> enabled/disabled using rfkill?
> 
> Given that rfkill is completely decoupled from PCI, I don't see how we
> would trigger any software-based hotplug mechanism. Perhaps one thing
> that we could do is the equivalent of this:
> 
> 	# echo 1 > /sys/bus/pci/rescan
> 
> from some script that's perhaps tied to the rfkill somehow. I'm not sure
> if that's possible, or generic enough.

How does rfkill work?  It sounds like it completely removes power from
the wifi device, putting it in D3cold.  Is there any software
notification other than the "Slot present pin change" (which looks
like a Tegra-specific thing)?

If the device is in D3cold, it won't respond to any PCI transactions,
and there's no standard PCI mechanism to wake it up.  Probably the
cleanest way to handle this is to make it a hot-unplug.

If this were an ACPI system, the rfkill might be visible as some sort
of ACPI power management event, and there might be a corresponding way
for software to bring the device back to D0 temporarily.  That would
make lspci and X config reads work.  But I don't think this system has
ACPI.

> > > Perhaps one other way to deal with this would be to track the
> > > suspend state of devices and then have the code that implements
> > > the PCI access from userspace refuse accesses to devices that
> > > are asleep. I suppose this is somewhat of an odd use-case
> > > because traditionally I guess PCI devices never power down to a
> > > state where their configuration space can no longer be accessed.
> > > At least that's what would explain why this has never been an
> > > issue before. Or perhaps it has?
> > >
> > > The last resort would be to just never put the WiFi chip into
> > > that low power mode, though I'm not exactly sure what that means
> > > for the power consumption on the affected systems.

> > >>>>>> diff --git a/drivers/pci/controller/pci-tegra.c b/drivers/pci/controller/pci-tegra.c
> > >>>>>> index d20c88a79e00..33f4dfab9e35 100644
> > >>>>>> --- a/drivers/pci/controller/pci-tegra.c
> > >>>>>> +++ b/drivers/pci/controller/pci-tegra.c
> > >>>>>> @@ -428,6 +428,14 @@ static inline u32 pads_readl(struct tegra_pcie *pcie, unsigned long offset)
> > >>>>>>  	return readl(pcie->pads + offset);
> > >>>>>>  }
> > >>>>>>  
> > >>>>>> +static bool tegra_pcie_link_up(struct tegra_pcie_port *port)
> > >>>>>> +{
> > >>>>>> +	u32 value;
> > >>>>>> +
> > >>>>>> +	value = readl(port->base + RP_LINK_CONTROL_STATUS);
> > >>>>>> +	return !!(value & RP_LINK_CONTROL_STATUS_DL_LINK_ACTIVE);
> > >>>>>> +}
> > >>>>>> +
> > >>>>>>  /*
> > >>>>>>   * The configuration space mapping on Tegra is somewhat similar to the ECAM
> > >>>>>>   * defined by PCIe. However it deviates a bit in how the 4 bits for extended
> > >>>>>> @@ -493,20 +501,50 @@ static void __iomem *tegra_pcie_map_bus(struct pci_bus *bus,
> > >>>>>>  static int tegra_pcie_config_read(struct pci_bus *bus, unsigned int devfn,
> > >>>>>>  				  int where, int size, u32 *value)
> > >>>>>>  {
> > >>>>>> +	struct tegra_pcie *pcie = bus->sysdata;
> > >>>>>> +	struct pci_dev *bridge;
> > >>>>>> +	struct tegra_pcie_port *port;
> > >>>>>> +
> > >>>>>>  	if (bus->number == 0)
> > >>>>>>  		return pci_generic_config_read32(bus, devfn, where, size,
> > >>>>>>  						 value);
> > >>>>>>  
> > >>>>>> +	bridge = pcie_find_root_port(bus->self);
> > >>>>>> +
> > >>>>>> +	list_for_each_entry(port, &pcie->ports, list)
> > >>>>>> +		if (port->index + 1 == PCI_SLOT(bridge->devfn))
> > >>>>>> +			break;
> > >>>>>> +
> > >>>>>> +	/* If there is no link, then there is no device */
> > >>>>>> +	if (!tegra_pcie_link_up(port)) {
> > >>>>>> +		*value = 0xffffffff;
> > >>>>>> +		return PCIBIOS_DEVICE_NOT_FOUND;
> > >>>>>> +	}
> > >>>>>> +
> > >>>>>>  	return pci_generic_config_read(bus, devfn, where, size, value);
> > >>>>>>  }
> > >>>>>>  
> > >>>>>>  static int tegra_pcie_config_write(struct pci_bus *bus, unsigned int devfn,
> > >>>>>>  				   int where, int size, u32 value)
> > >>>>>>  {
> > >>>>>> +	struct tegra_pcie *pcie = bus->sysdata;
> > >>>>>> +	struct tegra_pcie_port *port;
> > >>>>>> +	struct pci_dev *bridge;
> > >>>>>> +
> > >>>>>>  	if (bus->number == 0)
> > >>>>>>  		return pci_generic_config_write32(bus, devfn, where, size,
> > >>>>>>  						  value);
> > >>>>>>  
> > >>>>>> +	bridge = pcie_find_root_port(bus->self);
> > >>>>>> +
> > >>>>>> +	list_for_each_entry(port, &pcie->ports, list)
> > >>>>>> +		if (port->index + 1 == PCI_SLOT(bridge->devfn))
> > >>>>>> +			break;
> > >>>>>> +
> > >>>>>> +	/* If there is no link, then there is no device */
> > >>>>>> +	if (!tegra_pcie_link_up(port))
> > >>>>>> +		return PCIBIOS_DEVICE_NOT_FOUND;
> > >>>>>> +
> > >>>>>>  	return pci_generic_config_write(bus, devfn, where, size, value);
> > >>>>>>  }
> > >>>>>>  
> > >>>>>> -- 
> > >>>>>> 2.17.1
> > >>>>>>
> > 



^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: [PATCH V4 22/28] PCI: tegra: Access endpoint config only if PCIe link is up
  2019-06-17 19:30                 ` Bjorn Helgaas
@ 2019-06-18  5:36                   ` Manikanta Maddireddy
  2019-06-18 10:49                     ` Thierry Reding
  0 siblings, 1 reply; 72+ messages in thread
From: Manikanta Maddireddy @ 2019-06-18  5:36 UTC (permalink / raw)
  To: Bjorn Helgaas, Thierry Reding
  Cc: Lorenzo Pieralisi, robh+dt, mark.rutland, jonathanh, vidyas,
	linux-tegra, linux-pci, devicetree, linux-pm, Rafael J. Wysocki


On 18-Jun-19 1:00 AM, Bjorn Helgaas wrote:
> [+cc Rafael, linux-pm, in case they have insights on how rfkill works]
>
> On Mon, Jun 17, 2019 at 01:47:45PM +0200, Thierry Reding wrote:
>> On Mon, Jun 17, 2019 at 03:31:38PM +0530, Manikanta Maddireddy wrote:
>>> On 13-Jun-19 9:12 PM, Thierry Reding wrote:
>>>> On Thu, Jun 13, 2019 at 03:39:46PM +0100, Lorenzo Pieralisi wrote:
>>>>> On Mon, Jun 10, 2019 at 10:08:16AM +0530, Manikanta Maddireddy wrote:
>>>>>> On 04-Jun-19 7:40 PM, Manikanta Maddireddy wrote:
>>>>>>> On 04-Jun-19 6:44 PM, Thierry Reding wrote:
>>>>>>>> On Thu, May 16, 2019 at 11:23:01AM +0530, Manikanta Maddireddy wrote:
>>>>>>>>> Few endpoints like Wi-Fi supports power on/off and to leverage that
>>>>>>>>> root port must support hot-plug and hot-unplug. Tegra PCIe doesn't
>>>>>>>>> support hot-plug and hot-unplug, however it supports endpoint power
>>>>>>>>> on/off feature as follows,
>>>>>>>>>  - Power off sequence:
>>>>>>>>>    - Transition of PCIe link to L2
>>>>>>>>>    - Power off endpoint
>>>>>>>>>    - Leave root port in power up state with the link in L2
>>>>>>>>>  - Power on sequence:
>>>>>>>>>    - Power on endpoint
>>>>>>>>>    - Apply hot reset to get PCIe link up
>>>>>>>>>
>>>>>>>>> PCIe client driver stops accessing PCIe endpoint config and
>>>>>>>>> BAR registers after endpoint is powered off. However,
>>>>>>>>> software applications like x11 server or lspci can access
>>>>>>>>> endpoint config registers in which case host controller
>>>>>>>>> raises "response decoding" errors. To avoid this scenario,
>>>>>>>>> add PCIe link up check in config read and write callback
>>>>>>>>> functions before accessing endpoint config registers.
>>>>>>>>> Signed-off-by: Manikanta Maddireddy <mmaddireddy@nvidia.com>
>>>>>>>>> ---
>>>>>>>>> V4: No change
>>>>>>>>>
>>>>>>>>> V3: Update the commit log with explanation for the need of this patch
>>>>>>>>>
>>>>>>>>> V2: Change tegra_pcie_link_status() to tegra_pcie_link_up()
>>>>>>>>>
>>>>>>>>>  drivers/pci/controller/pci-tegra.c | 38 ++++++++++++++++++++++++++++++
>>>>>>>>>  1 file changed, 38 insertions(+)
>>>>>>>> This still doesn't look right to me conceptually. If
>>>>>>>> somebody wants to access the PCI devices after the kernel
>>>>>>>> has powered them off, why can't we just power the devices
>>>>>>>> back on so that we allow userspace to properly access the
>>>>>>>> devices?
>>>>>>> 1. WiFi devices provides power-off feature for power saving
>>>>>>> in mobiles.  When WiFi is turned off we shouldn't power on
>>>>>>> the HW back without user turning it back on.
>>>>>>> 2. When ever user process tries to access config space, it'll
>>>>>>> end up in these functions. We cannot have is_powered_on check
>>>>>>> in config read/write callbacks.
>>>>>>> 3. WiFi power on/off is device specific feature, we shouldn't
>>>>>>> handle it in PCI subsystem or host controller driver.
>>>>>>>> Or if that's not what we want, shouldn't we add something to
>>>>>>>> the core PCI infrastructure to let us deal with this? It
>>>>>>>> seems like this is some general problem that would apply to
>>>>>>>> every PCI device and host bridge driver. Having each driver
>>>>>>>> implement this logic separately doesn't seem like a good
>>>>>>>> idea to me.
>>>>>>> This should be handled by hotplug feature, whenever endpoint
>>>>>>> is powered-off/ removed from the slot, hot unplug event
>>>>>>> should take care of it. Unfortunately Tegra PCIe doesn't
>>>>>>> support hotplug feature.
>>>>>> I thought about your comment in
>>>>>> https://patchwork.ozlabs.org/patch/1084204/ again.  What if I
>>>>>> add link up check in tegra_pcie_isr() and make "response
>>>>>> decoding error" as debug print? EP Config access will happen
>>>>>> when link is down, but "Response decoding error" print comes
>>>>>> only if debug log is enabled.  This way we can avoid race
>>>>>> issue in config accessors and we get prints when debug logs
>>>>>> are enabled.
>>>> The problem that Manikanta is trying to solve here occurs in
>>>> this situation (Manikanta, correct me if I've got this wrong):
>>>> on some setups, a WiFi module connected over PCI will toggle a
>>>> power GPIO as part of runtime suspend. This effectively causes
>>>> the module to disappear from the PCI bus (i.e. it can no longer
>>>> be accessed until the power GPIO is toggled again).
>>> GPIO is toggled as part of WiFi on/off, can be triggered from
>>> network manager UI.
>>>> This is fine from a kernel point of view because the kernel keeps track
>>>> of what devices are suspended. However, userspace will occasionally try
>>>> to read the configuration space access of all devices, and since it
>>>> doesn't have any knowledge about the suspend state of these devices, it
>>>> doesn't know which ones to leave alone. I think this happens when the
>>>> X.Org server is running.
>>> This is fine from a kernel point of view because PCI client driver
>>> doesn't initiate any PCIe transaction until network interface
>>> is up during WiFi on.
>>>
>>>> One thing that Manikanta and I had discussed was that perhaps
>>>> the device should be hot-unplugged when it goes into this
>>>> low-power state. However, we don't support hotplug on Tegra210
>>>> where this is needed, so we'd need some sort of software-induced
>>>> hot-unplug. However, this low power state is entered when the
>>>> WiFi interface is taken down (i.e. ip link set dev <interface>
>>>> down). If we were to remove the PCI device in that case, it
>>>> means that the interface goes away completely, which is
>>>> completely unexpected from a user's perspective. After all,
>>>> taking a link down and up may be something that scripts are
>>>> doing all the time. They'd fall over if after taking the
>>>> interface down, the interface completely disappears.
>>>> It's also not entirely clear to me how we get the device back
>>>> onto the bus again after it is in low power. If we hot-unplug
>>>> the device, then the driver will be unbound. Presumably the
>>>> driver is what's controlling the power GPIO, so there won't be
>>>> any entity that can be used to bring the chip back to life.
>>>> Unless we deal with that power GPIO elsewhere (rfkill switch
>>>> perhaps?).
>>> Correct, rfkill switch should handle the GPIO.
>>> Sequence will be,
>>>  - WiFi ON
>>>    - rfkill switch enables the WiFi GPIO
>>>    - Tegra PCIe receives hot plug event
>>>    - Tegra PCIe hot plug driver rescans PCI bus and enumerates the device
>>>    - PCI client driver is probed, which will create network interface
>>>  - WiFi OFF
>>>    - rfkill switch disables the WiFi GPIO
>>>    - Tegra PCIe receives hot unplug event
>>>    - Tegra PCIe hot plug driver removes PCI devices under the bus
>>>    - PCI client driver remove is executed, which will remove
>>>      network interface
>>> We don't need current patch in this case because PCI device is not
>>> present in the PCI hierarchy, so there cannot be EP config access
>>> with link down.  However Tegra doesn't support hot plug and unplug
>>> events. I am not sure if we have any software based hot plug event
>>> trigger.
>>> I will drop current patch and pursue if above sequence can be
>>> implemented for Tegra.
>> I just recalled that we have these messages in the kernel log:
>>
>> 	# dmesg | grep tegra-pcie
>> 	[    1.055761] tegra-pcie 1003000.pcie: 4x1, 1x1 configuration
>> 	[    2.745764] tegra-pcie 1003000.pcie: 4x1, 1x1 configuration
>> 	[    2.753073] tegra-pcie 1003000.pcie: probing port 0, using 4 lanes
>> 	[    2.761334] tegra-pcie 1003000.pcie: Slot present pin change, signature: 00000008
>> 	[    3.177607] tegra-pcie 1003000.pcie: link 0 down, retrying
>> 	[    3.585605] tegra-pcie 1003000.pcie: link 0 down, retrying
>> 	[    3.993606] tegra-pcie 1003000.pcie: link 0 down, retrying
>> 	[    4.001214] tegra-pcie 1003000.pcie: link 0 down, ignoring
>> 	[    4.006733] tegra-pcie 1003000.pcie: probing port 1, using 1 lanes
>> 	[    4.015042] tegra-pcie 1003000.pcie: Slot present pin change, signature: 00000000
>> 	[    4.031177] tegra-pcie 1003000.pcie: PCI host bridge to bus 0000:00
>>
>> These "slot present pin change" message do look a lot like hotplug
>> related messages. Could we perhaps use those to our advantage for this
>> case? Do you see these when you run on the platform where WiFi is
>> enabled/disabled using rfkill?
>>
>> Given that rfkill is completely decoupled from PCI, I don't see how we
>> would trigger any software-based hotplug mechanism. Perhaps one thing
>> that we could do is the equivalent of this:
>>
>> 	# echo 1 > /sys/bus/pci/rescan
>>
>> from some script that's perhaps tied to the rfkill somehow. I'm not sure
>> if that's possible, or generic enough.
> How does rfkill work?  It sounds like it completely removes power from
> the wifi device, putting it in D3cold.  Is there any software
> notification other than the "Slot present pin change" (which looks
> like a Tegra-specific thing)?
>
> If the device is in D3cold, it won't respond to any PCI transactions,
> and there's no standard PCI mechanism to wake it up.  Probably the
> cleanest way to handle this is to make it a hot-unplug.
>
> If this were an ACPI system, the rfkill might be visible as some sort
> of ACPI power management event, and there might be a corresponding way
> for software to bring the device back to D0 temporarily.  That would
> make lspci and X config reads work.  But I don't think this system has
> ACPI.

"Slot present pin change" interrupt is triggered based on the programming
of PRSNT_MAP bit field in PCIE2_RP_PRIV_MISC controller register and
they are not triggered when EP is hot plugged/unplugged. Tegra PCIe
controller doesn't have capability to detect EP hot plug and unplug.
Consider that WiFi off equivalent to SW aware EP hot unplug event.

The rfkill subsystem provides a generic interface for disabling any radio
transmitter in the system. WiFi M.2 form factor cards provide W_DISABLE
GPIO to control the radio transmitter and I have seen some cards provide
control to turn off complete chip through this GPIO. Here we are talking
about second case where device is put in D3cold state. This GPIO can be
registered to rfkill subsystem and rfkill commands (like "rfkill unblock
wifi") can be used to turn on radio transmitter during WiFi on.


Manikanta

>>>> Perhaps one other way to deal with this would be to track the
>>>> suspend state of devices and then have the code that implements
>>>> the PCI access from userspace refuse accesses to devices that
>>>> are asleep. I suppose this is somewhat of an odd use-case
>>>> because traditionally I guess PCI devices never power down to a
>>>> state where their configuration space can no longer be accessed.
>>>> At least that's what would explain why this has never been an
>>>> issue before. Or perhaps it has?
>>>>
>>>> The last resort would be to just never put the WiFi chip into
>>>> that low power mode, though I'm not exactly sure what that means
>>>> for the power consumption on the affected systems.
>>>>>>>>> diff --git a/drivers/pci/controller/pci-tegra.c b/drivers/pci/controller/pci-tegra.c
>>>>>>>>> index d20c88a79e00..33f4dfab9e35 100644
>>>>>>>>> --- a/drivers/pci/controller/pci-tegra.c
>>>>>>>>> +++ b/drivers/pci/controller/pci-tegra.c
>>>>>>>>> @@ -428,6 +428,14 @@ static inline u32 pads_readl(struct tegra_pcie *pcie, unsigned long offset)
>>>>>>>>>  	return readl(pcie->pads + offset);
>>>>>>>>>  }
>>>>>>>>>  
>>>>>>>>> +static bool tegra_pcie_link_up(struct tegra_pcie_port *port)
>>>>>>>>> +{
>>>>>>>>> +	u32 value;
>>>>>>>>> +
>>>>>>>>> +	value = readl(port->base + RP_LINK_CONTROL_STATUS);
>>>>>>>>> +	return !!(value & RP_LINK_CONTROL_STATUS_DL_LINK_ACTIVE);
>>>>>>>>> +}
>>>>>>>>> +
>>>>>>>>>  /*
>>>>>>>>>   * The configuration space mapping on Tegra is somewhat similar to the ECAM
>>>>>>>>>   * defined by PCIe. However it deviates a bit in how the 4 bits for extended
>>>>>>>>> @@ -493,20 +501,50 @@ static void __iomem *tegra_pcie_map_bus(struct pci_bus *bus,
>>>>>>>>>  static int tegra_pcie_config_read(struct pci_bus *bus, unsigned int devfn,
>>>>>>>>>  				  int where, int size, u32 *value)
>>>>>>>>>  {
>>>>>>>>> +	struct tegra_pcie *pcie = bus->sysdata;
>>>>>>>>> +	struct pci_dev *bridge;
>>>>>>>>> +	struct tegra_pcie_port *port;
>>>>>>>>> +
>>>>>>>>>  	if (bus->number == 0)
>>>>>>>>>  		return pci_generic_config_read32(bus, devfn, where, size,
>>>>>>>>>  						 value);
>>>>>>>>>  
>>>>>>>>> +	bridge = pcie_find_root_port(bus->self);
>>>>>>>>> +
>>>>>>>>> +	list_for_each_entry(port, &pcie->ports, list)
>>>>>>>>> +		if (port->index + 1 == PCI_SLOT(bridge->devfn))
>>>>>>>>> +			break;
>>>>>>>>> +
>>>>>>>>> +	/* If there is no link, then there is no device */
>>>>>>>>> +	if (!tegra_pcie_link_up(port)) {
>>>>>>>>> +		*value = 0xffffffff;
>>>>>>>>> +		return PCIBIOS_DEVICE_NOT_FOUND;
>>>>>>>>> +	}
>>>>>>>>> +
>>>>>>>>>  	return pci_generic_config_read(bus, devfn, where, size, value);
>>>>>>>>>  }
>>>>>>>>>  
>>>>>>>>>  static int tegra_pcie_config_write(struct pci_bus *bus, unsigned int devfn,
>>>>>>>>>  				   int where, int size, u32 value)
>>>>>>>>>  {
>>>>>>>>> +	struct tegra_pcie *pcie = bus->sysdata;
>>>>>>>>> +	struct tegra_pcie_port *port;
>>>>>>>>> +	struct pci_dev *bridge;
>>>>>>>>> +
>>>>>>>>>  	if (bus->number == 0)
>>>>>>>>>  		return pci_generic_config_write32(bus, devfn, where, size,
>>>>>>>>>  						  value);
>>>>>>>>>  
>>>>>>>>> +	bridge = pcie_find_root_port(bus->self);
>>>>>>>>> +
>>>>>>>>> +	list_for_each_entry(port, &pcie->ports, list)
>>>>>>>>> +		if (port->index + 1 == PCI_SLOT(bridge->devfn))
>>>>>>>>> +			break;
>>>>>>>>> +
>>>>>>>>> +	/* If there is no link, then there is no device */
>>>>>>>>> +	if (!tegra_pcie_link_up(port))
>>>>>>>>> +		return PCIBIOS_DEVICE_NOT_FOUND;
>>>>>>>>> +
>>>>>>>>>  	return pci_generic_config_write(bus, devfn, where, size, value);
>>>>>>>>>  }
>>>>>>>>>  
>>>>>>>>> -- 
>>>>>>>>> 2.17.1
>>>>>>>>>
>


^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: [PATCH V4 22/28] PCI: tegra: Access endpoint config only if PCIe link is up
  2019-06-18  5:36                   ` Manikanta Maddireddy
@ 2019-06-18 10:49                     ` Thierry Reding
  2019-06-18 12:32                       ` Johannes Berg
  0 siblings, 1 reply; 72+ messages in thread
From: Thierry Reding @ 2019-06-18 10:49 UTC (permalink / raw)
  To: Manikanta Maddireddy
  Cc: Bjorn Helgaas, Lorenzo Pieralisi, robh+dt, mark.rutland,
	jonathanh, vidyas, linux-tegra, linux-pci, devicetree, linux-pm,
	Rafael J. Wysocki, Johannes Berg, linux-wireless

[-- Attachment #1: Type: text/plain, Size: 12279 bytes --]

On Tue, Jun 18, 2019 at 11:06:44AM +0530, Manikanta Maddireddy wrote:
> 
> On 18-Jun-19 1:00 AM, Bjorn Helgaas wrote:
> > [+cc Rafael, linux-pm, in case they have insights on how rfkill works]
> >
> > On Mon, Jun 17, 2019 at 01:47:45PM +0200, Thierry Reding wrote:
> >> On Mon, Jun 17, 2019 at 03:31:38PM +0530, Manikanta Maddireddy wrote:
> >>> On 13-Jun-19 9:12 PM, Thierry Reding wrote:
> >>>> On Thu, Jun 13, 2019 at 03:39:46PM +0100, Lorenzo Pieralisi wrote:
> >>>>> On Mon, Jun 10, 2019 at 10:08:16AM +0530, Manikanta Maddireddy wrote:
> >>>>>> On 04-Jun-19 7:40 PM, Manikanta Maddireddy wrote:
> >>>>>>> On 04-Jun-19 6:44 PM, Thierry Reding wrote:
> >>>>>>>> On Thu, May 16, 2019 at 11:23:01AM +0530, Manikanta Maddireddy wrote:
> >>>>>>>>> Few endpoints like Wi-Fi supports power on/off and to leverage that
> >>>>>>>>> root port must support hot-plug and hot-unplug. Tegra PCIe doesn't
> >>>>>>>>> support hot-plug and hot-unplug, however it supports endpoint power
> >>>>>>>>> on/off feature as follows,
> >>>>>>>>>  - Power off sequence:
> >>>>>>>>>    - Transition of PCIe link to L2
> >>>>>>>>>    - Power off endpoint
> >>>>>>>>>    - Leave root port in power up state with the link in L2
> >>>>>>>>>  - Power on sequence:
> >>>>>>>>>    - Power on endpoint
> >>>>>>>>>    - Apply hot reset to get PCIe link up
> >>>>>>>>>
> >>>>>>>>> PCIe client driver stops accessing PCIe endpoint config and
> >>>>>>>>> BAR registers after endpoint is powered off. However,
> >>>>>>>>> software applications like x11 server or lspci can access
> >>>>>>>>> endpoint config registers in which case host controller
> >>>>>>>>> raises "response decoding" errors. To avoid this scenario,
> >>>>>>>>> add PCIe link up check in config read and write callback
> >>>>>>>>> functions before accessing endpoint config registers.
> >>>>>>>>> Signed-off-by: Manikanta Maddireddy <mmaddireddy@nvidia.com>
> >>>>>>>>> ---
> >>>>>>>>> V4: No change
> >>>>>>>>>
> >>>>>>>>> V3: Update the commit log with explanation for the need of this patch
> >>>>>>>>>
> >>>>>>>>> V2: Change tegra_pcie_link_status() to tegra_pcie_link_up()
> >>>>>>>>>
> >>>>>>>>>  drivers/pci/controller/pci-tegra.c | 38 ++++++++++++++++++++++++++++++
> >>>>>>>>>  1 file changed, 38 insertions(+)
> >>>>>>>> This still doesn't look right to me conceptually. If
> >>>>>>>> somebody wants to access the PCI devices after the kernel
> >>>>>>>> has powered them off, why can't we just power the devices
> >>>>>>>> back on so that we allow userspace to properly access the
> >>>>>>>> devices?
> >>>>>>> 1. WiFi devices provides power-off feature for power saving
> >>>>>>> in mobiles.  When WiFi is turned off we shouldn't power on
> >>>>>>> the HW back without user turning it back on.
> >>>>>>> 2. When ever user process tries to access config space, it'll
> >>>>>>> end up in these functions. We cannot have is_powered_on check
> >>>>>>> in config read/write callbacks.
> >>>>>>> 3. WiFi power on/off is device specific feature, we shouldn't
> >>>>>>> handle it in PCI subsystem or host controller driver.
> >>>>>>>> Or if that's not what we want, shouldn't we add something to
> >>>>>>>> the core PCI infrastructure to let us deal with this? It
> >>>>>>>> seems like this is some general problem that would apply to
> >>>>>>>> every PCI device and host bridge driver. Having each driver
> >>>>>>>> implement this logic separately doesn't seem like a good
> >>>>>>>> idea to me.
> >>>>>>> This should be handled by hotplug feature, whenever endpoint
> >>>>>>> is powered-off/ removed from the slot, hot unplug event
> >>>>>>> should take care of it. Unfortunately Tegra PCIe doesn't
> >>>>>>> support hotplug feature.
> >>>>>> I thought about your comment in
> >>>>>> https://patchwork.ozlabs.org/patch/1084204/ again.  What if I
> >>>>>> add link up check in tegra_pcie_isr() and make "response
> >>>>>> decoding error" as debug print? EP Config access will happen
> >>>>>> when link is down, but "Response decoding error" print comes
> >>>>>> only if debug log is enabled.  This way we can avoid race
> >>>>>> issue in config accessors and we get prints when debug logs
> >>>>>> are enabled.
> >>>> The problem that Manikanta is trying to solve here occurs in
> >>>> this situation (Manikanta, correct me if I've got this wrong):
> >>>> on some setups, a WiFi module connected over PCI will toggle a
> >>>> power GPIO as part of runtime suspend. This effectively causes
> >>>> the module to disappear from the PCI bus (i.e. it can no longer
> >>>> be accessed until the power GPIO is toggled again).
> >>> GPIO is toggled as part of WiFi on/off, can be triggered from
> >>> network manager UI.
> >>>> This is fine from a kernel point of view because the kernel keeps track
> >>>> of what devices are suspended. However, userspace will occasionally try
> >>>> to read the configuration space access of all devices, and since it
> >>>> doesn't have any knowledge about the suspend state of these devices, it
> >>>> doesn't know which ones to leave alone. I think this happens when the
> >>>> X.Org server is running.
> >>> This is fine from a kernel point of view because PCI client driver
> >>> doesn't initiate any PCIe transaction until network interface
> >>> is up during WiFi on.
> >>>
> >>>> One thing that Manikanta and I had discussed was that perhaps
> >>>> the device should be hot-unplugged when it goes into this
> >>>> low-power state. However, we don't support hotplug on Tegra210
> >>>> where this is needed, so we'd need some sort of software-induced
> >>>> hot-unplug. However, this low power state is entered when the
> >>>> WiFi interface is taken down (i.e. ip link set dev <interface>
> >>>> down). If we were to remove the PCI device in that case, it
> >>>> means that the interface goes away completely, which is
> >>>> completely unexpected from a user's perspective. After all,
> >>>> taking a link down and up may be something that scripts are
> >>>> doing all the time. They'd fall over if after taking the
> >>>> interface down, the interface completely disappears.
> >>>> It's also not entirely clear to me how we get the device back
> >>>> onto the bus again after it is in low power. If we hot-unplug
> >>>> the device, then the driver will be unbound. Presumably the
> >>>> driver is what's controlling the power GPIO, so there won't be
> >>>> any entity that can be used to bring the chip back to life.
> >>>> Unless we deal with that power GPIO elsewhere (rfkill switch
> >>>> perhaps?).
> >>> Correct, rfkill switch should handle the GPIO.
> >>> Sequence will be,
> >>>  - WiFi ON
> >>>    - rfkill switch enables the WiFi GPIO
> >>>    - Tegra PCIe receives hot plug event
> >>>    - Tegra PCIe hot plug driver rescans PCI bus and enumerates the device
> >>>    - PCI client driver is probed, which will create network interface
> >>>  - WiFi OFF
> >>>    - rfkill switch disables the WiFi GPIO
> >>>    - Tegra PCIe receives hot unplug event
> >>>    - Tegra PCIe hot plug driver removes PCI devices under the bus
> >>>    - PCI client driver remove is executed, which will remove
> >>>      network interface
> >>> We don't need current patch in this case because PCI device is not
> >>> present in the PCI hierarchy, so there cannot be EP config access
> >>> with link down.  However Tegra doesn't support hot plug and unplug
> >>> events. I am not sure if we have any software based hot plug event
> >>> trigger.
> >>> I will drop current patch and pursue if above sequence can be
> >>> implemented for Tegra.
> >> I just recalled that we have these messages in the kernel log:
> >>
> >> 	# dmesg | grep tegra-pcie
> >> 	[    1.055761] tegra-pcie 1003000.pcie: 4x1, 1x1 configuration
> >> 	[    2.745764] tegra-pcie 1003000.pcie: 4x1, 1x1 configuration
> >> 	[    2.753073] tegra-pcie 1003000.pcie: probing port 0, using 4 lanes
> >> 	[    2.761334] tegra-pcie 1003000.pcie: Slot present pin change, signature: 00000008
> >> 	[    3.177607] tegra-pcie 1003000.pcie: link 0 down, retrying
> >> 	[    3.585605] tegra-pcie 1003000.pcie: link 0 down, retrying
> >> 	[    3.993606] tegra-pcie 1003000.pcie: link 0 down, retrying
> >> 	[    4.001214] tegra-pcie 1003000.pcie: link 0 down, ignoring
> >> 	[    4.006733] tegra-pcie 1003000.pcie: probing port 1, using 1 lanes
> >> 	[    4.015042] tegra-pcie 1003000.pcie: Slot present pin change, signature: 00000000
> >> 	[    4.031177] tegra-pcie 1003000.pcie: PCI host bridge to bus 0000:00
> >>
> >> These "slot present pin change" message do look a lot like hotplug
> >> related messages. Could we perhaps use those to our advantage for this
> >> case? Do you see these when you run on the platform where WiFi is
> >> enabled/disabled using rfkill?
> >>
> >> Given that rfkill is completely decoupled from PCI, I don't see how we
> >> would trigger any software-based hotplug mechanism. Perhaps one thing
> >> that we could do is the equivalent of this:
> >>
> >> 	# echo 1 > /sys/bus/pci/rescan
> >>
> >> from some script that's perhaps tied to the rfkill somehow. I'm not sure
> >> if that's possible, or generic enough.
> > How does rfkill work?  It sounds like it completely removes power from
> > the wifi device, putting it in D3cold.  Is there any software
> > notification other than the "Slot present pin change" (which looks
> > like a Tegra-specific thing)?
> >
> > If the device is in D3cold, it won't respond to any PCI transactions,
> > and there's no standard PCI mechanism to wake it up.  Probably the
> > cleanest way to handle this is to make it a hot-unplug.
> >
> > If this were an ACPI system, the rfkill might be visible as some sort
> > of ACPI power management event, and there might be a corresponding way
> > for software to bring the device back to D0 temporarily.  That would
> > make lspci and X config reads work.  But I don't think this system has
> > ACPI.
> 
> "Slot present pin change" interrupt is triggered based on the programming
> of PRSNT_MAP bit field in PCIE2_RP_PRIV_MISC controller register and
> they are not triggered when EP is hot plugged/unplugged. Tegra PCIe
> controller doesn't have capability to detect EP hot plug and unplug.
> Consider that WiFi off equivalent to SW aware EP hot unplug event.
> 
> The rfkill subsystem provides a generic interface for disabling any radio
> transmitter in the system. WiFi M.2 form factor cards provide W_DISABLE
> GPIO to control the radio transmitter and I have seen some cards provide
> control to turn off complete chip through this GPIO. Here we are talking
> about second case where device is put in D3cold state. This GPIO can be
> registered to rfkill subsystem and rfkill commands (like "rfkill unblock
> wifi") can be used to turn on radio transmitter during WiFi on.

Perhaps what we need here is some sort of mechanism to make rfkill and
the PCI host controller interoperate? I could imagine for example that
the PCI host controller would get a new "rfkill" property in device
tree that points at the rfkill device via phandle.

The driver could then get a reference to it using something like:

	rfkill = rfkill_get(dev);
	if (IS_ERR(rfkill)) {
		...
	}

and register for notification:

	err = rfkill_subscribe(rfkill, callback);
	if (err < 0) {
		...
	}

rfkill_unsubscribe() and rfkill_put() would then be used upon driver
unload to detach from the rfkill.

I noticed that there's an rfkill-gpio driver (net/rfkill/rfkill-gpio.c)
that already does pretty much everything that we need, except that it
doesn't support DT yet, but I suspect that that's pretty easy to add.

Johannes, any thoughts on this. In a nutshell what we're trying to solve
here is devices that get removed from/added to PCI based on an rfkill-
type of device. The difference to other implementations is that we have
no way of detecting when the device has gone away (PCI hotplug does not
work). So we'd need some software-triggered mechanism to let the PCI
host controller know when the device is presumably going away or being
added back, so that the PCI bus can be rescanned and the PCI device
removed or added at that point).

Thierry

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: [PATCH V4 22/28] PCI: tegra: Access endpoint config only if PCIe link is up
  2019-06-18 10:49                     ` Thierry Reding
@ 2019-06-18 12:32                       ` Johannes Berg
  2019-06-18 13:40                         ` Thierry Reding
  2019-06-19 13:38                         ` Bjorn Helgaas
  0 siblings, 2 replies; 72+ messages in thread
From: Johannes Berg @ 2019-06-18 12:32 UTC (permalink / raw)
  To: Thierry Reding, Manikanta Maddireddy
  Cc: Bjorn Helgaas, Lorenzo Pieralisi, robh+dt, mark.rutland,
	jonathanh, vidyas, linux-tegra, linux-pci, devicetree, linux-pm,
	Rafael J. Wysocki, linux-wireless

I got to this thread really late I guess :-)

On Tue, 2019-06-18 at 12:49 +0200, Thierry Reding wrote:

> > > > > > > > > 1. WiFi devices provides power-off feature for power saving
> > > > > > > > > in mobiles.  When WiFi is turned off we shouldn't power on
> > > > > > > > > the HW back without user turning it back on.

But why would you disconnect the PCIe device just to power it down?!

> > > > > > The problem that Manikanta is trying to solve here occurs in
> > > > > > this situation (Manikanta, correct me if I've got this wrong):
> > > > > > on some setups, a WiFi module connected over PCI will toggle a
> > > > > > power GPIO as part of runtime suspend. This effectively causes
> > > > > > the module to disappear from the PCI bus (i.e. it can no longer
> > > > > > be accessed until the power GPIO is toggled again).
> > > > > 
> > > > > GPIO is toggled as part of WiFi on/off, can be triggered from
> > > > > network manager UI.

That's kinda icky, IMHO.

> > > > > Correct, rfkill switch should handle the GPIO.
> > > > > Sequence will be,
> > > > >  - WiFi ON
> > > > >    - rfkill switch enables the WiFi GPIO
> > > > >    - Tegra PCIe receives hot plug event
> > > > >    - Tegra PCIe hot plug driver rescans PCI bus and enumerates the device
> > > > >    - PCI client driver is probed, which will create network interface
> > > > >  - WiFi OFF
> > > > >    - rfkill switch disables the WiFi GPIO
> > > > >    - Tegra PCIe receives hot unplug event
> > > > >    - Tegra PCIe hot plug driver removes PCI devices under the bus
> > > > >    - PCI client driver remove is executed, which will remove
> > > > >      network interface
> > > > > We don't need current patch in this case because PCI device is not
> > > > > present in the PCI hierarchy, so there cannot be EP config access
> > > > > with link down.  However Tegra doesn't support hot plug and unplug
> > > > > events. I am not sure if we have any software based hot plug event
> > > > > trigger.

Looks reasonable to me.

I guess if you absolutely know in software when the device is present or
not, you don't need "real" PCIe hotplug, just need to tickle the
software right?

> > > How does rfkill work?  It sounds like it completely removes power from
> > > the wifi device, putting it in D3cold.  Is there any software
> > > notification other than the "Slot present pin change" (which looks
> > > like a Tegra-specific thing)?

Well, they said above it's a GPIO that controls it, so the software
already knows and doesn't really need an event?

> > The rfkill subsystem provides a generic interface for disabling any radio
> > transmitter in the system. WiFi M.2 form factor cards provide W_DISABLE
> > GPIO to control the radio transmitter

But it depends on the hardware how this is handled, Intel NICs for
example just trigger an IRQ to the host and don't turn off much, for
them the W_DISABLE pin is just a GPIO in input mode, with edge triggered
interrupt to the driver.

> > and I have seen some cards provide
> > control to turn off complete chip through this GPIO. 

I never heard of this. Which NICs are we talking about?

> Perhaps what we need here is some sort of mechanism to make rfkill and
> the PCI host controller interoperate? I could imagine for example that
> the PCI host controller would get a new "rfkill" property in device
> tree that points at the rfkill device via phandle.

But you don't know which the rfkill device is, do you?

I mean, fundamentally, you just have a GPIO that turns on and off the
W_DISABLE pin. NICs will not generally disappear from the bus when
that's turned on, so you need a NIC driver integration.

I guess you also have an rfkill-gpio driver assigned to this GPIO, which
gets assigned there via DT/platform code?

Ah, but then I guess you could have a phandle in the DT or so that ties
the W_DISABLE-GPIO with the PCIe slot that it controls.

> The driver could then get a reference to it using something like:
> 
> 	rfkill = rfkill_get(dev);
> 	if (IS_ERR(rfkill)) {
> 		...
> 	}
> 
> and register for notification:
> 
> 	err = rfkill_subscribe(rfkill, callback);
> 	if (err < 0) {
> 		...
> 	}
> 
> rfkill_unsubscribe() and rfkill_put() would then be used upon driver
> unload to detach from the rfkill.

This I don't understand.

> I noticed that there's an rfkill-gpio driver (net/rfkill/rfkill-gpio.c)
> that already does pretty much everything that we need, except that it
> doesn't support DT yet, but I suspect that that's pretty easy to add.

Oh, good point, no DT support here - so how *do* you actually
instantiate the rfkill today??

> Johannes, any thoughts on this. In a nutshell what we're trying to solve
> here is devices that get removed from/added to PCI based on an rfkill-
> type of device. The difference to other implementations is that we have
> no way of detecting when the device has gone away (PCI hotplug does not
> work). So we'd need some software-triggered mechanism to let the PCI
> host controller know when the device is presumably going away or being
> added back, so that the PCI bus can be rescanned and the PCI device
> removed or added at that point).

Right.

So, I'm not even sure we need the *driver* to do anything other than say
"I know the device will drop off the bus when rfkill is enabled", right?


But do we actually need rfkill to be involved here?

I mean, let's say first we make rfkill-gpio DT-aware, rather than just
ACPI. This should be simple. Then it drives a GPIO (it can actually
drive two and a clock, not sure I know why).

Now, next we need something that says that the device should be treated
as hotplug/unplug. We could make this in the driver somehow like you
suggested, but that seems like a lot of effort?

Couldn't we put this into the *GPIO* subsystem instead?

I mean - conceivably there could be GPIOs that just power down a device
for example. Not even through something like W_DISABLE, but just having
a GPIO hooked up to a transistor on the voltage pin of the device. That
would have very similar semantics?

So why not just attach the PCIe device/port to the GPIO, and have the
GPIO implementation here call the detach/attach (or detach/rescan?) when
they are toggled?

Not that I'd mind having it in rfkill! But it seems like a special case
to have it there, when you can do so much more with GPIOs.

johannes


^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: [PATCH V4 22/28] PCI: tegra: Access endpoint config only if PCIe link is up
  2019-06-18 12:32                       ` Johannes Berg
@ 2019-06-18 13:40                         ` Thierry Reding
  2019-06-18 14:48                           ` Johannes Berg
  2019-06-19 13:38                         ` Bjorn Helgaas
  1 sibling, 1 reply; 72+ messages in thread
From: Thierry Reding @ 2019-06-18 13:40 UTC (permalink / raw)
  To: Johannes Berg
  Cc: Manikanta Maddireddy, Bjorn Helgaas, Lorenzo Pieralisi, robh+dt,
	mark.rutland, jonathanh, vidyas, linux-tegra, linux-pci,
	devicetree, linux-pm, Rafael J. Wysocki, linux-wireless

[-- Attachment #1: Type: text/plain, Size: 12716 bytes --]

On Tue, Jun 18, 2019 at 02:32:59PM +0200, Johannes Berg wrote:
> I got to this thread really late I guess :-)
> 
> On Tue, 2019-06-18 at 12:49 +0200, Thierry Reding wrote:
> 
> > > > > > > > > > 1. WiFi devices provides power-off feature for power saving
> > > > > > > > > > in mobiles.  When WiFi is turned off we shouldn't power on
> > > > > > > > > > the HW back without user turning it back on.
> 
> But why would you disconnect the PCIe device just to power it down?!

It's a side-effect of asserting that W_DISABLE pin that the bus link
basically goes down. We've had a similar case recently, one that we
haven't quite solved either, where an RTL8169 Ethernet controller is
hooked up to a GPIO that controls the ISOLATEB (I think that was the
name) pin. If that pin is asserted, according to the documentation,
the device stops sampling/driving the PCI signals. So for all intents
and purposes it becomes disconnected.

We could kind of deal with this if the ISOLATEB was deasserted at probe
time, because that would mean that the device is at least enumerated on
PCI. Then when we go into some power down mode (for example when the
interface is taken down), the NIC driver could assert the GPIO and on
resuming from the power down mode deassert it again. Logically the
device would stay around, we just couldn't talk to it over PCI until the
driver has deasserted the ISOLATEB GPIO.

The problem is that it's not exactly defined what the status of the pin
would be at probe time. If it is asserted, the NIC will never show up on
the PCI bus and hence no driver would be registered that could deassert
the ISOLATEB signal. Well, unless we somehow created a "placeholder" PCI
device based on a device tree node (containing a reference to the GPIO)
so that the device would be enumerated (and probed) regardless of the
PCI link. There's no infrastructure to do that currently, but perhaps
worth investigating.

I think the W_DISABLE is somewhat similar. From what Manikanta was
saying, the PCI link also goes down when the pin is asserted, so we
loose any means of communicating with it over PCI.

The issue that Manikanta was trying to solve with this particular patch
was that since the PCI device is part of the PCI device hierarchy, some
userspace tools (X server, for example) will see it and try to discover
whether it's a GPU or not. This in turn causes errors from the PCI host
controller because it's trying to access a device behind a link that's
down. That, I assume, could also happen for the ISOLATEB case that I was
describing above, though it hasn't been brought up, I think.

> > > > > > > The problem that Manikanta is trying to solve here occurs in
> > > > > > > this situation (Manikanta, correct me if I've got this wrong):
> > > > > > > on some setups, a WiFi module connected over PCI will toggle a
> > > > > > > power GPIO as part of runtime suspend. This effectively causes
> > > > > > > the module to disappear from the PCI bus (i.e. it can no longer
> > > > > > > be accessed until the power GPIO is toggled again).
> > > > > > 
> > > > > > GPIO is toggled as part of WiFi on/off, can be triggered from
> > > > > > network manager UI.
> 
> That's kinda icky, IMHO.

Isn't that kind of the point of rfkill? I seem to remember having a
notebook where this was done exactly the same way. There was also a
button/switch that you could push which would result in the WiFi device
either going away completely or at the least loosing the WiFi link. It
seems like that's exactly what Manikanta is describing.

> > > > > > Correct, rfkill switch should handle the GPIO.
> > > > > > Sequence will be,
> > > > > >  - WiFi ON
> > > > > >    - rfkill switch enables the WiFi GPIO
> > > > > >    - Tegra PCIe receives hot plug event
> > > > > >    - Tegra PCIe hot plug driver rescans PCI bus and enumerates the device
> > > > > >    - PCI client driver is probed, which will create network interface
> > > > > >  - WiFi OFF
> > > > > >    - rfkill switch disables the WiFi GPIO
> > > > > >    - Tegra PCIe receives hot unplug event
> > > > > >    - Tegra PCIe hot plug driver removes PCI devices under the bus
> > > > > >    - PCI client driver remove is executed, which will remove
> > > > > >      network interface
> > > > > > We don't need current patch in this case because PCI device is not
> > > > > > present in the PCI hierarchy, so there cannot be EP config access
> > > > > > with link down.  However Tegra doesn't support hot plug and unplug
> > > > > > events. I am not sure if we have any software based hot plug event
> > > > > > trigger.
> 
> Looks reasonable to me.
> 
> I guess if you absolutely know in software when the device is present or
> not, you don't need "real" PCIe hotplug, just need to tickle the
> software right?

Right.

> > > > How does rfkill work?  It sounds like it completely removes power from
> > > > the wifi device, putting it in D3cold.  Is there any software
> > > > notification other than the "Slot present pin change" (which looks
> > > > like a Tegra-specific thing)?
> 
> Well, they said above it's a GPIO that controls it, so the software
> already knows and doesn't really need an event?

We still need to communicate from rfkill to the PCI host controller that
something happened, since they are two different entities.

> > > The rfkill subsystem provides a generic interface for disabling any radio
> > > transmitter in the system. WiFi M.2 form factor cards provide W_DISABLE
> > > GPIO to control the radio transmitter
> 
> But it depends on the hardware how this is handled, Intel NICs for
> example just trigger an IRQ to the host and don't turn off much, for
> them the W_DISABLE pin is just a GPIO in input mode, with edge triggered
> interrupt to the driver.

Okay, so does this mean you have some input device connected to the WiFi
device that will be used (without software intervention) to disable the
transmitter and then the WiFi device will signal using the W_DISABLE pin
that the transmitter was indeed disabled?

> > > and I have seen some cards provide
> > > control to turn off complete chip through this GPIO. 
> 
> I never heard of this. Which NICs are we talking about?
> 
> > Perhaps what we need here is some sort of mechanism to make rfkill and
> > the PCI host controller interoperate? I could imagine for example that
> > the PCI host controller would get a new "rfkill" property in device
> > tree that points at the rfkill device via phandle.
> 
> But you don't know which the rfkill device is, do you?
> 
> I mean, fundamentally, you just have a GPIO that turns on and off the
> W_DISABLE pin. NICs will not generally disappear from the bus when
> that's turned on, so you need a NIC driver integration.

I think that's the main problem that we're trying to solve. In our case
it does seem like the device completely disappears from the bus.

> I guess you also have an rfkill-gpio driver assigned to this GPIO, which
> gets assigned there via DT/platform code?

Yes, I think that's correct. Manikanta, please confirm.

> Ah, but then I guess you could have a phandle in the DT or so that ties
> the W_DISABLE-GPIO with the PCIe slot that it controls.

Right, that's what I was thinking.

> > The driver could then get a reference to it using something like:
> > 
> > 	rfkill = rfkill_get(dev);
> > 	if (IS_ERR(rfkill)) {
> > 		...
> > 	}
> > 
> > and register for notification:
> > 
> > 	err = rfkill_subscribe(rfkill, callback);
> > 	if (err < 0) {
> > 		...
> > 	}
> > 
> > rfkill_unsubscribe() and rfkill_put() would then be used upon driver
> > unload to detach from the rfkill.
> 
> This I don't understand.

This was just an example of what I was imagining. The network driver
would get an rfkill (looked up via device tree phandle) and subscribe to
receive events from it, so that it could be notified when the rfkill is
"blocked" and rescan the bus to get the WiFi device unplugged. Once
unblocked it would be notified again and rescan the bus so that the
device would reappear.

> > I noticed that there's an rfkill-gpio driver (net/rfkill/rfkill-gpio.c)
> > that already does pretty much everything that we need, except that it
> > doesn't support DT yet, but I suspect that that's pretty easy to add.
> 
> Oh, good point, no DT support here - so how *do* you actually
> instantiate the rfkill today??

I suspect that we've got downstream patches for that. The patch here is
part of a series to upstream support for this. I haven't seen the patch
for rfkill-gpio, but perhaps that's queued for later.

> > Johannes, any thoughts on this. In a nutshell what we're trying to solve
> > here is devices that get removed from/added to PCI based on an rfkill-
> > type of device. The difference to other implementations is that we have
> > no way of detecting when the device has gone away (PCI hotplug does not
> > work). So we'd need some software-triggered mechanism to let the PCI
> > host controller know when the device is presumably going away or being
> > added back, so that the PCI bus can be rescanned and the PCI device
> > removed or added at that point).
> 
> Right.
> 
> So, I'm not even sure we need the *driver* to do anything other than say
> "I know the device will drop off the bus when rfkill is enabled", right?
> 
> 
> But do we actually need rfkill to be involved here?
> 
> I mean, let's say first we make rfkill-gpio DT-aware, rather than just
> ACPI. This should be simple. Then it drives a GPIO (it can actually
> drive two and a clock, not sure I know why).
> 
> Now, next we need something that says that the device should be treated
> as hotplug/unplug. We could make this in the driver somehow like you
> suggested, but that seems like a lot of effort?
> 
> Couldn't we put this into the *GPIO* subsystem instead?
> 
> I mean - conceivably there could be GPIOs that just power down a device
> for example. Not even through something like W_DISABLE, but just having
> a GPIO hooked up to a transistor on the voltage pin of the device. That
> would have very similar semantics?
> 
> So why not just attach the PCIe device/port to the GPIO, and have the
> GPIO implementation here call the detach/attach (or detach/rescan?) when
> they are toggled?
> 
> Not that I'd mind having it in rfkill! But it seems like a special case
> to have it there, when you can do so much more with GPIOs.

Yeah, that's where things become a little muddy. For the ISOLATEB case
there was initially a similar proposal. The problem is that on one hand
we can have different semantics for these pins. On one platform this
could be a kind of "power" GPIO, on others it could be ISOLATE/DISABLE,
and on yet others it would be more like a reset. In order to make the
PCIe port aware of the differences we'd have to expose multiple GPIOs in
DT for context.

The other problem with this is that, in order to avoid the chicken-and-
egg problem, we need to associate these GPIOs with the root ports,
because those are the only ones that exist at probe time. All downstream
devices may not be available because the power/reset/disable pin is not
asserted/deasserted yet. Now, you could potentially have a switch in the
downstream hierarchy, so it becomes completely unclear what exact device
the GPIO is associated with.

Related to that, a GPIO like this is really only useful if you can make
use of it. For example you want to assert/deassert this GPIO in order to
put the WiFi/Ethernet/whatever device into a low-power mode when it is
not used, right? But in order to do so, the driver for that device needs
to be able to handle the GPIO, because it is the only one that knows the
right point in time to toggle it. Conversely, if this was associated
with the root port, the only point in time where the root port driver
could toggle it is on a suspend/resume of the entire bus, which makes it
rather useless.

But then we're back to square one where we basically have to associate
the GPIO with the specific device. I think that's the right thing to do
because, well, that's what reality is. The GPIO is directly routed to a
pin on the chip. It's not something that goes over the PCI connector or
anything. However, we're also back to the chicken-and-egg problem since
without toggling the GPIO the device might not even get enumerated.

rfkill-gpio has the advantage that it decouples this and gets us out of
the chicken-and-egg situation. It also has fairly well-defined semantics
and fits the use-case, so it's a very appealing option.

Thierry

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: [PATCH V4 22/28] PCI: tegra: Access endpoint config only if PCIe link is up
  2019-06-18 13:40                         ` Thierry Reding
@ 2019-06-18 14:48                           ` Johannes Berg
  0 siblings, 0 replies; 72+ messages in thread
From: Johannes Berg @ 2019-06-18 14:48 UTC (permalink / raw)
  To: Thierry Reding
  Cc: Manikanta Maddireddy, Bjorn Helgaas, Lorenzo Pieralisi, robh+dt,
	mark.rutland, jonathanh, vidyas, linux-tegra, linux-pci,
	devicetree, linux-pm, Rafael J. Wysocki, linux-wireless

On Tue, 2019-06-18 at 15:40 +0200, Thierry Reding wrote:
> On Tue, Jun 18, 2019 at 02:32:59PM +0200, Johannes Berg wrote:
> > I got to this thread really late I guess :-)
> > 
> > On Tue, 2019-06-18 at 12:49 +0200, Thierry Reding wrote:
> > 
> > > > > > > > > > > 1. WiFi devices provides power-off feature for power saving
> > > > > > > > > > > in mobiles.  When WiFi is turned off we shouldn't power on
> > > > > > > > > > > the HW back without user turning it back on.
> > 
> > But why would you disconnect the PCIe device just to power it down?!
> 
> It's a side-effect of asserting that W_DISABLE pin that the bus link
> basically goes down. We've had a similar case recently, one that we
> haven't quite solved either, where an RTL8169 Ethernet controller is
> hooked up to a GPIO that controls the ISOLATEB (I think that was the
> name) pin. If that pin is asserted, according to the documentation,
> the device stops sampling/driving the PCI signals. So for all intents
> and purposes it becomes disconnected.

Right.

> We could kind of deal with this if the ISOLATEB was deasserted at probe
> time, because that would mean that the device is at least enumerated on
> PCI. Then when we go into some power down mode (for example when the
> interface is taken down), the NIC driver could assert the GPIO and on
> resuming from the power down mode deassert it again. Logically the
> device would stay around, we just couldn't talk to it over PCI until the
> driver has deasserted the ISOLATEB GPIO.
> 
> The problem is that it's not exactly defined what the status of the pin
> would be at probe time. If it is asserted, the NIC will never show up on
> the PCI bus and hence no driver would be registered that could deassert
> the ISOLATEB signal. Well, unless we somehow created a "placeholder" PCI
> device based on a device tree node (containing a reference to the GPIO)
> so that the device would be enumerated (and probed) regardless of the
> PCI link. There's no infrastructure to do that currently, but perhaps
> worth investigating.
> 
> I think the W_DISABLE is somewhat similar. From what Manikanta was
> saying, the PCI link also goes down when the pin is asserted, so we
> loose any means of communicating with it over PCI.
> 
> The issue that Manikanta was trying to solve with this particular patch
> was that since the PCI device is part of the PCI device hierarchy, some
> userspace tools (X server, for example) will see it and try to discover
> whether it's a GPU or not. This in turn causes errors from the PCI host
> controller because it's trying to access a device behind a link that's
> down. That, I assume, could also happen for the ISOLATEB case that I was
> describing above, though it hasn't been brought up, I think.

Agree, sounds like it.

> > > > > > > > The problem that Manikanta is trying to solve here occurs in
> > > > > > > > this situation (Manikanta, correct me if I've got this wrong):
> > > > > > > > on some setups, a WiFi module connected over PCI will toggle a
> > > > > > > > power GPIO as part of runtime suspend. This effectively causes
> > > > > > > > the module to disappear from the PCI bus (i.e. it can no longer
> > > > > > > > be accessed until the power GPIO is toggled again).
> > > > > > > 
> > > > > > > GPIO is toggled as part of WiFi on/off, can be triggered from
> > > > > > > network manager UI.
> > 
> > That's kinda icky, IMHO.
> 
> Isn't that kind of the point of rfkill? I seem to remember having a
> notebook where this was done exactly the same way. There was also a
> button/switch that you could push which would result in the WiFi device
> either going away completely or at the least loosing the WiFi link. It
> seems like that's exactly what Manikanta is describing.

Right.

So ... rfkill has some terminology issues sometimes. Let me clarify what
typically happens.

Usually, you have the rfkill instance "wiphy-xyz" which is on the
wireless NIC (through cfg80211). This has two things:

 1) software rfkill *control*
 2) hardware rfkill *reporting*

So a device like iwlwifi has software rfkill control through rfkill
"wiphy-phy0", which is really just implemented as "bring down all the
netdevs etc.". And then you have "HW rfkill reporting", where our NIC
just reports the current status of the W_DISABLE pin. This again causes
the software to turn off all the netdevs etc.

This is how it looks like for the *device* side.

Now, for the *platform* side, which we're really looking at now, we
usually have the rfkill be a bit differently:

hardware rfkill *reporting* isn't used at all here.

software rfkill *control*, instead of directly controlling the netdevs
etc. like the instances in cfg80211 do, this just controls the GPIO.


Now, in a typical platform with an Intel NIC, you have something like

platform rfkill device
 --> controls GPIO
  W_DISABLE pin
   --> reported as HW rfkill
    --> rfkill-wiphy0 device

In some other platforms, you literally just have a hardware button:

hardware button
 --> controls GPIO
  ... as before ...

Now, this all works great, but is basically software only, just using
the hardware pins as a communication mechanism.


Now, with some platforms, and particularly with BT USB devices where
I've seen this a lot (but never saw it with PCIe before), you have what
you're describing here, that the device just drops off the bus for an
rfkill.

Again, though, the actual cause of this might be a GPIO control (through
an rfkill instance) or something else like a literal hardware button
(not uncommon for Bluetooth).


> > Well, they said above it's a GPIO that controls it, so the software
> > already knows and doesn't really need an event?
> 
> We still need to communicate from rfkill to the PCI host controller that
> something happened, since they are two different entities.

Yeah, but the question is if we really need it from *rfkill* rather than
the GPIO as I described below?

> > > > The rfkill subsystem provides a generic interface for disabling any radio
> > > > transmitter in the system. WiFi M.2 form factor cards provide W_DISABLE
> > > > GPIO to control the radio transmitter
> > 
> > But it depends on the hardware how this is handled, Intel NICs for
> > example just trigger an IRQ to the host and don't turn off much, for
> > them the W_DISABLE pin is just a GPIO in input mode, with edge triggered
> > interrupt to the driver.
> 
> Okay, so does this mean you have some input device connected to the WiFi
> device that will be used (without software intervention) to disable the
> transmitter and then the WiFi device will signal using the W_DISABLE pin
> that the transmitter was indeed disabled?

See above.

> This was just an example of what I was imagining. The network driver
> would get an rfkill (looked up via device tree phandle) and subscribe to
> receive events from it, so that it could be notified when the rfkill is
> "blocked" and rescan the bus to get the WiFi device unplugged. Once
> unblocked it would be notified again and rescan the bus so that the
> device would reappear.

Ok. Not sure we need much involvement of the driver and/or the rfkill
even though.

> > I mean, let's say first we make rfkill-gpio DT-aware, rather than just
> > ACPI. This should be simple. Then it drives a GPIO (it can actually
> > drive two and a clock, not sure I know why).
> > 
> > Now, next we need something that says that the device should be treated
> > as hotplug/unplug. We could make this in the driver somehow like you
> > suggested, but that seems like a lot of effort?
> > 
> > Couldn't we put this into the *GPIO* subsystem instead?
> > 
> > I mean - conceivably there could be GPIOs that just power down a device
> > for example. Not even through something like W_DISABLE, but just having
> > a GPIO hooked up to a transistor on the voltage pin of the device. That
> > would have very similar semantics?
> > 
> > So why not just attach the PCIe device/port to the GPIO, and have the
> > GPIO implementation here call the detach/attach (or detach/rescan?) when
> > they are toggled?
> > 
> > Not that I'd mind having it in rfkill! But it seems like a special case
> > to have it there, when you can do so much more with GPIOs.
> 
> Yeah, that's where things become a little muddy. For the ISOLATEB case
> there was initially a similar proposal. 

OK.

> The problem is that on one hand
> we can have different semantics for these pins. On one platform this
> could be a kind of "power" GPIO, on others it could be ISOLATE/DISABLE,
> and on yet others it would be more like a reset. In order to make the
> PCIe port aware of the differences we'd have to expose multiple GPIOs in
> DT for context.

Right, but does it matter?

I mean - does this have a different impact on the software? It seems to
me one way or the other all you really need to do is hot-unplug a device
on the right signal one way, and rescan for devices on the other signal?

> The other problem with this is that, in order to avoid the chicken-and-
> egg problem, we need to associate these GPIOs with the root ports,
> because those are the only ones that exist at probe time. All downstream
> devices may not be available because the power/reset/disable pin is not
> asserted/deasserted yet. Now, you could potentially have a switch in the
> downstream hierarchy, so it becomes completely unclear what exact device
> the GPIO is associated with.

Hmm, sort of I guess. I think you need *both* associations really.

On the "disappear" transition, you need it to be linked to a very
specific device. On the "appear" transition, you need the rescan of the
root port right?

> Related to that, a GPIO like this is really only useful if you can make
> use of it. For example you want to assert/deassert this GPIO in order to
> put the WiFi/Ethernet/whatever device into a low-power mode when it is
> not used, right? But in order to do so, the driver for that device needs
> to be able to handle the GPIO, because it is the only one that knows the
> right point in time to toggle it. Conversely, if this was associated
> with the root port, the only point in time where the root port driver
> could toggle it is on a suspend/resume of the entire bus, which makes it
> rather useless.

Depends. If you're talking about rfkill, you have a completely separate
rfkill-gpio device (the "platform rfkill device" I was talking about
earlier), and the driver for the actual wifi NIC isn't actually involved
at all.

> But then we're back to square one where we basically have to associate
> the GPIO with the specific device. I think that's the right thing to do
> because, well, that's what reality is. The GPIO is directly routed to a
> pin on the chip. It's not something that goes over the PCI connector or
> anything. However, we're also back to the chicken-and-egg problem since
> without toggling the GPIO the device might not even get enumerated.
> 
> rfkill-gpio has the advantage that it decouples this and gets us out of
> the chicken-and-egg situation. It also has fairly well-defined semantics
> and fits the use-case, so it's a very appealing option.

Sure, I get that, but I still don't understand why we should link it to
the rfkill rather than the GPIO?

I mean, if we have these things in the platform/DT:

 WIFI-WDISABLE-GPIO
 WIFI-NIC-PCIE
 PCIE-ROOT-PORT
 PLATFORM-WIFI-RFKILL

then we'd describe the

  PLATFORM-WIFI-RFKILL as an rfkill-gpio using the WIFI-WDISABLE-GPIO

and make some sort of link:

 WIFI-WDISABLE-GPIO --enable-rescan-- PCIE-ROOT-PORT
                    --unplug-device-- WIFI-NIC-PCIE

or not?

johannes


^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: [PATCH V4 22/28] PCI: tegra: Access endpoint config only if PCIe link is up
  2019-06-18 12:32                       ` Johannes Berg
  2019-06-18 13:40                         ` Thierry Reding
@ 2019-06-19 13:38                         ` Bjorn Helgaas
  2019-06-19 13:40                           ` Johannes Berg
  1 sibling, 1 reply; 72+ messages in thread
From: Bjorn Helgaas @ 2019-06-19 13:38 UTC (permalink / raw)
  To: Johannes Berg
  Cc: Thierry Reding, Manikanta Maddireddy, Lorenzo Pieralisi, robh+dt,
	mark.rutland, jonathanh, vidyas, linux-tegra, linux-pci,
	devicetree, linux-pm, Rafael J. Wysocki, linux-wireless

On Tue, Jun 18, 2019 at 02:32:59PM +0200, Johannes Berg wrote:
> On Tue, 2019-06-18 at 12:49 +0200, Thierry Reding wrote:
> 
> > > > > > > > > > 1. WiFi devices provides power-off feature for power saving
> > > > > > > > > > in mobiles.  When WiFi is turned off we shouldn't power on
> > > > > > > > > > the HW back without user turning it back on.
> 
> But why would you disconnect the PCIe device just to power it down?!
> 
> > > > > > > The problem that Manikanta is trying to solve here occurs in
> > > > > > > this situation (Manikanta, correct me if I've got this wrong):
> > > > > > > on some setups, a WiFi module connected over PCI will toggle a
> > > > > > > power GPIO as part of runtime suspend. This effectively causes
> > > > > > > the module to disappear from the PCI bus (i.e. it can no longer
> > > > > > > be accessed until the power GPIO is toggled again).
> > > > > > 
> > > > > > GPIO is toggled as part of WiFi on/off, can be triggered from
> > > > > > network manager UI.
> 
> That's kinda icky, IMHO.
> 
> > > > > > Correct, rfkill switch should handle the GPIO.
> > > > > > Sequence will be,
> > > > > >  - WiFi ON
> > > > > >    - rfkill switch enables the WiFi GPIO
> > > > > >    - Tegra PCIe receives hot plug event
> > > > > >    - Tegra PCIe hot plug driver rescans PCI bus and enumerates the device
> > > > > >    - PCI client driver is probed, which will create network interface
> > > > > >  - WiFi OFF
> > > > > >    - rfkill switch disables the WiFi GPIO
> > > > > >    - Tegra PCIe receives hot unplug event
> > > > > >    - Tegra PCIe hot plug driver removes PCI devices under the bus
> > > > > >    - PCI client driver remove is executed, which will remove
> > > > > >      network interface
> > > > > > We don't need current patch in this case because PCI device is not
> > > > > > present in the PCI hierarchy, so there cannot be EP config access
> > > > > > with link down.  However Tegra doesn't support hot plug and unplug
> > > > > > events. I am not sure if we have any software based hot plug event
> > > > > > trigger.
> 
> Looks reasonable to me.
> 
> I guess if you absolutely know in software when the device is
> present or not, you don't need "real" PCIe hotplug, just need to
> tickle the software right?
> 
> > > > How does rfkill work?  It sounds like it completely removes
> > > > power from the wifi device, putting it in D3cold.  Is there
> > > > any software notification other than the "Slot present pin
> > > > change" (which looks like a Tegra-specific thing)?
> 
> Well, they said above it's a GPIO that controls it, so the software
> already knows and doesn't really need an event?

Forgive my ignorance about rfkill.  At least in this Tegra case, it
sounds like rfkill basically controls a power switch for the entire
device, i.e., it doesn't merely turn off the radio portion of the
device; it puts the entire PCI device in D3cold.

Is rfkill integrated with the power management subsystem?  E.g., when
lspci or X tries to read config space via pci_read_config(), does the
pci_config_pm_runtime_get() in that path wake up the device?

IMO, if the struct pci_dev exists, we should be able to rely on the
device actually being accessible (possibly after bringing it back to
D0).  If rfkill only turns off the radio, leaving the PCI interface
active, that would be fine -- in that case generic PCI things like
lspci would work normally and it would be up to the driver to manage
network-related things.

But if rfkill turns off PCI interface and the power management
subsystem can't wake it up, I think we should unbind the driver and
remove the pci_dev, so it wouldn't appear in lspci at all.

Bjorn

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: [PATCH V4 22/28] PCI: tegra: Access endpoint config only if PCIe link is up
  2019-06-19 13:38                         ` Bjorn Helgaas
@ 2019-06-19 13:40                           ` Johannes Berg
  0 siblings, 0 replies; 72+ messages in thread
From: Johannes Berg @ 2019-06-19 13:40 UTC (permalink / raw)
  To: Bjorn Helgaas
  Cc: Thierry Reding, Manikanta Maddireddy, Lorenzo Pieralisi, robh+dt,
	mark.rutland, jonathanh, vidyas, linux-tegra, linux-pci,
	devicetree, linux-pm, Rafael J. Wysocki, linux-wireless

On Wed, 2019-06-19 at 08:38 -0500, Bjorn Helgaas wrote:

> > > > > How does rfkill work?  It sounds like it completely removes
> > > > > power from the wifi device, putting it in D3cold.  Is there
> > > > > any software notification other than the "Slot present pin
> > > > > change" (which looks like a Tegra-specific thing)?
> > 
> > Well, they said above it's a GPIO that controls it, so the software
> > already knows and doesn't really need an event?
> 
> Forgive my ignorance about rfkill.  At least in this Tegra case, it
> sounds like rfkill basically controls a power switch for the entire
> device, i.e., it doesn't merely turn off the radio portion of the
> device; it puts the entire PCI device in D3cold.

Sort of. The actual (hardware) implementation seems a bit more
complicated than a "power switch", but yes, that's the effect of it.

> Is rfkill integrated with the power management subsystem?  E.g., when
> lspci or X tries to read config space via pci_read_config(), does the
> pci_config_pm_runtime_get() in that path wake up the device?

No, that's the problem at hand AFAICT.

> IMO, if the struct pci_dev exists, we should be able to rely on the
> device actually being accessible (possibly after bringing it back to
> D0).  If rfkill only turns off the radio, leaving the PCI interface
> active, that would be fine -- in that case generic PCI things like
> lspci would work normally and it would be up to the driver to manage
> network-related things.
> 
> But if rfkill turns off PCI interface and the power management
> subsystem can't wake it up, I think we should unbind the driver and
> remove the pci_dev, so it wouldn't appear in lspci at all.

Right. That's being suggested here, but since the platform has no actual
hardware hotplug, that needs to be implemented in software.

The question at hand is *how* to actually achieve that.

I'm kind of arguing that it's not rfkill that achieves it, but the
underlying GPIO that toggles the device, since that GPIO could also be
bound to something other than an rfkill-gpio instance.

johannes


^ permalink raw reply	[flat|nested] 72+ messages in thread

end of thread, other threads:[~2019-06-19 13:41 UTC | newest]

Thread overview: 72+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-05-16  5:52 [PATCH V4 00/28] Enable Tegra PCIe root port features Manikanta Maddireddy
2019-05-16  5:52 ` [PATCH V4 01/28] soc/tegra: pmc: Export tegra_powergate_power_on() Manikanta Maddireddy
2019-05-16  5:52 ` [PATCH V4 02/28] PCI: tegra: Handle failure cases in tegra_pcie_power_on() Manikanta Maddireddy
2019-05-16  5:52 ` [PATCH V4 03/28] PCI: tegra: Rearrange Tegra PCIe driver functions Manikanta Maddireddy
2019-05-16  5:52 ` [PATCH V4 04/28] PCI: tegra: Mask AFI_INTR in runtime suspend Manikanta Maddireddy
2019-06-04 13:08   ` Thierry Reding
2019-05-16  5:52 ` [PATCH V4 05/28] PCI: tegra: Fix PCIe host power up sequence Manikanta Maddireddy
2019-05-16  5:52 ` [PATCH V4 06/28] PCI: tegra: Add PCIe Gen2 link speed support Manikanta Maddireddy
2019-05-16  5:52 ` [PATCH V4 07/28] PCI: tegra: Advertise PCIe Advanced Error Reporting (AER) capability Manikanta Maddireddy
2019-05-16  5:52 ` [PATCH V4 08/28] PCI: tegra: Program UPHY electrical settings for Tegra210 Manikanta Maddireddy
2019-05-16  5:52 ` [PATCH V4 09/28] PCI: tegra: Enable opportunistic UpdateFC and ACK Manikanta Maddireddy
2019-05-16  5:52 ` [PATCH V4 10/28] PCI: tegra: Disable AFI dynamic clock gating Manikanta Maddireddy
2019-05-16  5:52 ` [PATCH V4 11/28] PCI: tegra: Process pending DLL transactions before entering L1 or L2 Manikanta Maddireddy
2019-05-16  5:52 ` [PATCH V4 12/28] PCI: tegra: Enable PCIe xclk clock clamping Manikanta Maddireddy
2019-05-16  5:52 ` [PATCH V4 13/28] PCI: tegra: Increase the deskew retry time Manikanta Maddireddy
2019-05-16  5:52 ` [PATCH V4 14/28] PCI: tegra: Add SW fixup for RAW violations Manikanta Maddireddy
2019-05-16  5:52 ` [PATCH V4 15/28] PCI: tegra: Update flow control timer frequency in Tegra210 Manikanta Maddireddy
2019-05-16  5:52 ` [PATCH V4 16/28] PCI: tegra: Set target speed as Gen1 before starting LTSSM Manikanta Maddireddy
2019-05-16  5:52 ` [PATCH V4 17/28] PCI: tegra: Fix PLLE power down issue due to CLKREQ# signal Manikanta Maddireddy
2019-05-16  5:52 ` [PATCH V4 18/28] PCI: tegra: Program AFI_CACHE* registers only for Tegra20 Manikanta Maddireddy
2019-05-16  5:52 ` [PATCH V4 19/28] PCI: tegra: Change PRSNT_SENSE IRQ log to debug Manikanta Maddireddy
2019-05-16  5:52 ` [PATCH V4 20/28] PCI: tegra: Use legacy IRQ for port service drivers Manikanta Maddireddy
2019-05-20 20:37   ` Bjorn Helgaas
2019-05-21  9:07     ` Manikanta Maddireddy
2019-05-16  5:53 ` [PATCH V4 21/28] PCI: tegra: Add AFI_PEX2_CTRL reg offset as part of soc struct Manikanta Maddireddy
2019-05-16  5:53 ` [PATCH V4 22/28] PCI: tegra: Access endpoint config only if PCIe link is up Manikanta Maddireddy
2019-06-04 13:14   ` Thierry Reding
2019-06-04 14:10     ` Manikanta Maddireddy
2019-06-10  4:38       ` Manikanta Maddireddy
2019-06-13 14:39         ` Lorenzo Pieralisi
2019-06-13 15:42           ` Thierry Reding
2019-06-17 10:01             ` Manikanta Maddireddy
2019-06-17 11:47               ` Thierry Reding
2019-06-17 19:30                 ` Bjorn Helgaas
2019-06-18  5:36                   ` Manikanta Maddireddy
2019-06-18 10:49                     ` Thierry Reding
2019-06-18 12:32                       ` Johannes Berg
2019-06-18 13:40                         ` Thierry Reding
2019-06-18 14:48                           ` Johannes Berg
2019-06-19 13:38                         ` Bjorn Helgaas
2019-06-19 13:40                           ` Johannes Berg
2019-05-16  5:53 ` [PATCH V4 23/28] dt-bindings: pci: tegra: Document PCIe DPD pinctrl optional prop Manikanta Maddireddy
2019-05-16  5:53 ` [PATCH V4 24/28] arm64: tegra: Add PEX DPD states as pinctrl properties Manikanta Maddireddy
2019-05-16  5:53 ` [PATCH V4 25/28] PCI: tegra: Put PEX CLK & BIAS pads in DPD mode Manikanta Maddireddy
2019-05-16  5:53 ` [PATCH V4 26/28] PCI: Add DT binding for "reset-gpios" property Manikanta Maddireddy
2019-06-17 11:30   ` Thierry Reding
2019-06-17 11:38     ` Manikanta Maddireddy
2019-06-17 11:48       ` Thierry Reding
2019-05-16  5:53 ` [PATCH V4 27/28] PCI: tegra: Add support for GPIO based PERST# Manikanta Maddireddy
2019-06-04 13:22   ` Thierry Reding
2019-06-13 15:24     ` Lorenzo Pieralisi
2019-06-14 10:37       ` Manikanta Maddireddy
2019-06-14 14:32         ` Lorenzo Pieralisi
2019-06-14 14:38           ` Manikanta Maddireddy
2019-06-14 14:50             ` Lorenzo Pieralisi
2019-06-14 14:56               ` Manikanta Maddireddy
2019-06-14 15:23               ` Thierry Reding
2019-06-14 15:59                 ` Lorenzo Pieralisi
2019-06-14 16:30                   ` Manikanta Maddireddy
2019-06-14 16:53                     ` Lorenzo Pieralisi
2019-06-14 17:23                       ` Manikanta Maddireddy
2019-06-17  9:48                         ` Lorenzo Pieralisi
2019-06-17 10:27                           ` Manikanta Maddireddy
2019-06-17 10:39                             ` Lorenzo Pieralisi
2019-06-17 11:29                         ` Thierry Reding
2019-06-17 11:26                       ` Thierry Reding
2019-05-16  5:53 ` [PATCH V4 28/28] PCI: tegra: Change link retry log level to debug Manikanta Maddireddy
2019-06-04 13:22   ` Thierry Reding
2019-05-16 13:12 ` [PATCH V4 00/28] Enable Tegra PCIe root port features Bjorn Helgaas
2019-05-17  8:38   ` Manikanta Maddireddy
2019-06-10  4:45 ` Manikanta Maddireddy
2019-06-10 17:33   ` Lorenzo Pieralisi

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).