[PATCH 0/7] PCI: aardvark controller fixes BATCH 3

[PATCH 0/7] PCI: aardvark controller fixes BATCH 3

[Marek Behún]

Dear Lorenzo,

since we have dropped some patches from the second batch, I thought that
we could instead replace them with another 7 small changes, that we can
hopefully get applied in this cycle.

Description:
- patch 1 just adds a small explanation to pci-bridge-emul, no functional
  change
- patches 2 and 3 add support for some more registers on emulated bridge
- patch 4 is taken from the MSI patches we dropped from batch 2, but since
  it only clears MSIs and adds a useful macro, and makes no changes that
  depend on the API change request by Marc, I think we can add it now
  (especially since patch 5 uses this new macro)
- patches 5-7 make driver unbind work better

Marek

Pali Rohár (7):
  PCI: pci-bridge-emul: Add description for class_revision field
  PCI: pci-bridge-emul: Add definitions for missing capabilities
    registers
  PCI: aardvark: Add support for DEVCAP2, DEVCTL2, LNKCAP2 and LNKCTL2
    registers on emulated bridge
  PCI: aardvark: Clear all MSIs at setup
  PCI: aardvark: Disable bus mastering and mask all interrupts when
    unbinding driver
  PCI: aardvark: Free config space for emulated root bridge when
    unbinding driver to fix memory leak
  PCI: aardvark: Reset PCIe card and disable PHY at driver unbind

 drivers/pci/controller/pci-aardvark.c | 65 ++++++++++++++++++++++++---
 drivers/pci/pci-bridge-emul.c         | 45 ++++++++++++++++++-
 2 files changed, 103 insertions(+), 7 deletions(-)

-- 
2.32.0

[PATCH 1/7] PCI: pci-bridge-emul: Add description for class_revision field

[Marek Behún]

From: Pali Rohár <pali@kernel.org>

The current assignment to the class_revision member

  class_revision |= cpu_to_le32(PCI_CLASS_BRIDGE_PCI << 16);

can make the reader think that class is at high 16 bits of the member and
revision at low 16 bits.

In reality, class is at high 24 bits, but the class for PCI Bridge Normal
Decode is PCI_CLASS_BRIDGE_PCI << 8.

Change the assignment and add a comment to make this clearer.

Signed-off-by: Pali Rohár <pali@kernel.org>
Signed-off-by: Marek Behún <kabel@kernel.org>
---
 drivers/pci/pci-bridge-emul.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/drivers/pci/pci-bridge-emul.c b/drivers/pci/pci-bridge-emul.c
index db97cddfc85e..a4af1a533d71 100644
--- a/drivers/pci/pci-bridge-emul.c
+++ b/drivers/pci/pci-bridge-emul.c
@@ -265,7 +265,11 @@ int pci_bridge_emul_init(struct pci_bridge_emul *bridge,
 {
 	BUILD_BUG_ON(sizeof(bridge->conf) != PCI_BRIDGE_CONF_END);
 
-	bridge->conf.class_revision |= cpu_to_le32(PCI_CLASS_BRIDGE_PCI << 16);
+	/*
+	 * class_revision: Class is high 24 bits and revision is low 8 bit of this member,
+	 * while class for PCI Bridge Normal Decode has the 24-bit value: PCI_CLASS_BRIDGE_PCI << 8
+	 */
+	bridge->conf.class_revision |= cpu_to_le32((PCI_CLASS_BRIDGE_PCI << 8) << 8);
 	bridge->conf.header_type = PCI_HEADER_TYPE_BRIDGE;
 	bridge->conf.cache_line_size = 0x10;
 	bridge->conf.status = cpu_to_le16(PCI_STATUS_CAP_LIST);
-- 
2.32.0

[PATCH 2/7] PCI: pci-bridge-emul: Add definitions for missing capabilities registers

[Marek Behún]

From: Pali Rohár <pali@kernel.org>

pci-bridge-emul driver already allocates buffer for capabilities up to the
PCI_EXP_SLTSTA2 register, but does not define bit access behavior for these
registers. Add these missing definitions.

Signed-off-by: Pali Rohár <pali@kernel.org>
Signed-off-by: Marek Behún <kabel@kernel.org>
Cc: stable@vger.kernel.org
---
 drivers/pci/pci-bridge-emul.c | 39 +++++++++++++++++++++++++++++++++++
 1 file changed, 39 insertions(+)

diff --git a/drivers/pci/pci-bridge-emul.c b/drivers/pci/pci-bridge-emul.c
index a4af1a533d71..aa3320e3c469 100644
--- a/drivers/pci/pci-bridge-emul.c
+++ b/drivers/pci/pci-bridge-emul.c
@@ -251,6 +251,45 @@ struct pci_bridge_reg_behavior pcie_cap_regs_behavior[PCI_CAP_PCIE_SIZEOF / 4] =
 		.ro = GENMASK(15, 0) | PCI_EXP_RTSTA_PENDING,
 		.w1c = PCI_EXP_RTSTA_PME,
 	},
+
+	[PCI_EXP_DEVCAP2 / 4] = {
+		/* Device capabilities 2 register has reserved bits [30:27]. */
+		.ro = BIT(31) | GENMASK(26, 0),
+	},
+
+	[PCI_EXP_DEVCTL2 / 4] = {
+		/*
+		 * Device control 2 register is RW.
+		 *
+		 * Device status 2 register is reserved.
+		 */
+		.rw = GENMASK(15, 0),
+	},
+
+	[PCI_EXP_LNKCAP2 / 4] = {
+		/* Link capabilities 2 register has reserved bits [30:25] and 0. */
+		.ro = BIT(31) | GENMASK(24, 1),
+	},
+
+	[PCI_EXP_LNKCTL2 / 4] = {
+		/*
+		 * Link control 2 register is RW.
+		 *
+		 * Link status 2 register has bits 5, 15 W1C;
+		 * bits 10, 11 reserved and others are RO.
+		 */
+		.rw = GENMASK(15, 0),
+		.w1c = (BIT(15) | BIT(5)) << 16,
+		.ro = (GENMASK(14, 12) | GENMASK(9, 6) | GENMASK(4, 0)) << 16,
+	},
+
+	[PCI_EXP_SLTCAP2 / 4] = {
+		/* Slot capabilities 2 register is reserved. */
+	},
+
+	[PCI_EXP_SLTCTL2 / 4] = {
+		/* Both Slot control 2 and Slot status 2 registers are reserved. */
+	},
 };
 
 /*
-- 
2.32.0

Re: [PATCH 2/7] PCI: pci-bridge-emul: Add definitions for missing capabilities registers

[Lorenzo Pieralisi]

On Sun, Oct 31, 2021 at 07:12:28PM +0100, Marek Behún wrote:
> From: Pali Rohár <pali@kernel.org>
> 
> pci-bridge-emul driver already allocates buffer for capabilities up to the
> PCI_EXP_SLTSTA2 register, but does not define bit access behavior for these
> registers. Add these missing definitions.
> 
> Signed-off-by: Pali Rohár <pali@kernel.org>
> Signed-off-by: Marek Behún <kabel@kernel.org>
> Cc: stable@vger.kernel.org

Is this tag in preparation for something else ? I don't even think this
is a fix per-se.

Lorenzo

> ---
>  drivers/pci/pci-bridge-emul.c | 39 +++++++++++++++++++++++++++++++++++
>  1 file changed, 39 insertions(+)
> 
> diff --git a/drivers/pci/pci-bridge-emul.c b/drivers/pci/pci-bridge-emul.c
> index a4af1a533d71..aa3320e3c469 100644
> --- a/drivers/pci/pci-bridge-emul.c
> +++ b/drivers/pci/pci-bridge-emul.c
> @@ -251,6 +251,45 @@ struct pci_bridge_reg_behavior pcie_cap_regs_behavior[PCI_CAP_PCIE_SIZEOF / 4] =
>  		.ro = GENMASK(15, 0) | PCI_EXP_RTSTA_PENDING,
>  		.w1c = PCI_EXP_RTSTA_PME,
>  	},
> +
> +	[PCI_EXP_DEVCAP2 / 4] = {
> +		/* Device capabilities 2 register has reserved bits [30:27]. */
> +		.ro = BIT(31) | GENMASK(26, 0),
> +	},
> +
> +	[PCI_EXP_DEVCTL2 / 4] = {
> +		/*
> +		 * Device control 2 register is RW.
> +		 *
> +		 * Device status 2 register is reserved.
> +		 */
> +		.rw = GENMASK(15, 0),
> +	},
> +
> +	[PCI_EXP_LNKCAP2 / 4] = {
> +		/* Link capabilities 2 register has reserved bits [30:25] and 0. */
> +		.ro = BIT(31) | GENMASK(24, 1),
> +	},
> +
> +	[PCI_EXP_LNKCTL2 / 4] = {
> +		/*
> +		 * Link control 2 register is RW.
> +		 *
> +		 * Link status 2 register has bits 5, 15 W1C;
> +		 * bits 10, 11 reserved and others are RO.
> +		 */
> +		.rw = GENMASK(15, 0),
> +		.w1c = (BIT(15) | BIT(5)) << 16,
> +		.ro = (GENMASK(14, 12) | GENMASK(9, 6) | GENMASK(4, 0)) << 16,
> +	},
> +
> +	[PCI_EXP_SLTCAP2 / 4] = {
> +		/* Slot capabilities 2 register is reserved. */
> +	},
> +
> +	[PCI_EXP_SLTCTL2 / 4] = {
> +		/* Both Slot control 2 and Slot status 2 registers are reserved. */
> +	},
>  };
>  
>  /*
> -- 
> 2.32.0
> 

[PATCH 3/7] PCI: aardvark: Add support for DEVCAP2, DEVCTL2, LNKCAP2 and LNKCTL2 registers on emulated bridge

[Marek Behún]

From: Pali Rohár <pali@kernel.org>

PCI aardvark hardware supports access to DEVCAP2, DEVCTL2, LNKCAP2 and
LNKCTL2 configuration registers of PCIe core via PCIE_CORE_PCIEXP_CAP.
Export them via emulated software root bridge.

Signed-off-by: Pali Rohár <pali@kernel.org>
Signed-off-by: Marek Behún <kabel@kernel.org>
Cc: stable@vger.kernel.org
---
 drivers/pci/controller/pci-aardvark.c | 15 +++++++++++----
 1 file changed, 11 insertions(+), 4 deletions(-)

diff --git a/drivers/pci/controller/pci-aardvark.c b/drivers/pci/controller/pci-aardvark.c
index c5300d49807a..25af189a1052 100644
--- a/drivers/pci/controller/pci-aardvark.c
+++ b/drivers/pci/controller/pci-aardvark.c
@@ -884,8 +884,13 @@ advk_pci_bridge_emul_pcie_conf_read(struct pci_bridge_emul *bridge,
 	case PCI_CAP_LIST_ID:
 	case PCI_EXP_DEVCAP:
 	case PCI_EXP_DEVCTL:
+	case PCI_EXP_DEVCAP2:
+	case PCI_EXP_DEVCTL2:
+	case PCI_EXP_LNKCAP2:
+	case PCI_EXP_LNKCTL2:
 		*value = advk_readl(pcie, PCIE_CORE_PCIEXP_CAP + reg);
 		return PCI_BRIDGE_EMUL_HANDLED;
+
 	default:
 		return PCI_BRIDGE_EMUL_NOT_HANDLED;
 	}
@@ -899,10 +904,6 @@ advk_pci_bridge_emul_pcie_conf_write(struct pci_bridge_emul *bridge,
 	struct advk_pcie *pcie = bridge->data;
 
 	switch (reg) {
-	case PCI_EXP_DEVCTL:
-		advk_writel(pcie, new, PCIE_CORE_PCIEXP_CAP + reg);
-		break;
-
 	case PCI_EXP_LNKCTL:
 		advk_writel(pcie, new, PCIE_CORE_PCIEXP_CAP + reg);
 		if (new & PCI_EXP_LNKCTL_RL)
@@ -924,6 +925,12 @@ advk_pci_bridge_emul_pcie_conf_write(struct pci_bridge_emul *bridge,
 		advk_writel(pcie, new, PCIE_ISR0_REG);
 		break;
 
+	case PCI_EXP_DEVCTL:
+	case PCI_EXP_DEVCTL2:
+	case PCI_EXP_LNKCTL2:
+		advk_writel(pcie, new, PCIE_CORE_PCIEXP_CAP + reg);
+		break;
+
 	default:
 		break;
 	}
-- 
2.32.0

[PATCH 4/7] PCI: aardvark: Clear all MSIs at setup

[Marek Behún]

From: Pali Rohár <pali@kernel.org>

We already clear all the other interrupts (ISR0, ISR1, HOST_CTRL_INT).

Define a new macro PCIE_MSI_ALL_MASK and do the same clearing for MSIs,
to ensure that we don't start receiving spurious interrupts.

Use this new mask in advk_pcie_handle_msi();

Fixes: 8c39d710363c ("PCI: aardvark: Add Aardvark PCI host controller driver")
Signed-off-by: Pali Rohár <pali@kernel.org>
Signed-off-by: Marek Behún <kabel@kernel.org>
Cc: stable@vger.kernel.org
---
 drivers/pci/controller/pci-aardvark.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/drivers/pci/controller/pci-aardvark.c b/drivers/pci/controller/pci-aardvark.c
index 25af189a1052..71ce9f02d596 100644
--- a/drivers/pci/controller/pci-aardvark.c
+++ b/drivers/pci/controller/pci-aardvark.c
@@ -116,6 +116,7 @@
 #define PCIE_MSI_ADDR_HIGH_REG			(CONTROL_BASE_ADDR + 0x54)
 #define PCIE_MSI_STATUS_REG			(CONTROL_BASE_ADDR + 0x58)
 #define PCIE_MSI_MASK_REG			(CONTROL_BASE_ADDR + 0x5C)
+#define     PCIE_MSI_ALL_MASK			GENMASK(31, 0)
 #define PCIE_MSI_PAYLOAD_REG			(CONTROL_BASE_ADDR + 0x9C)
 #define     PCIE_MSI_DATA_MASK			GENMASK(15, 0)
 
@@ -571,6 +572,7 @@ static void advk_pcie_setup_hw(struct advk_pcie *pcie)
 	advk_writel(pcie, reg, PCIE_CORE_CTRL2_REG);
 
 	/* Clear all interrupts */
+	advk_writel(pcie, PCIE_MSI_ALL_MASK, PCIE_MSI_STATUS_REG);
 	advk_writel(pcie, PCIE_ISR0_ALL_MASK, PCIE_ISR0_REG);
 	advk_writel(pcie, PCIE_ISR1_ALL_MASK, PCIE_ISR1_REG);
 	advk_writel(pcie, PCIE_IRQ_ALL_MASK, HOST_CTRL_INT_STATUS_REG);
@@ -583,7 +585,7 @@ static void advk_pcie_setup_hw(struct advk_pcie *pcie)
 	advk_writel(pcie, PCIE_ISR1_ALL_MASK, PCIE_ISR1_MASK_REG);
 
 	/* Unmask all MSIs */
-	advk_writel(pcie, 0, PCIE_MSI_MASK_REG);
+	advk_writel(pcie, ~(u32)PCIE_MSI_ALL_MASK, PCIE_MSI_MASK_REG);
 
 	/* Enable summary interrupt for GIC SPI source */
 	reg = PCIE_IRQ_ALL_MASK & (~PCIE_IRQ_ENABLE_INTS_MASK);
@@ -1399,7 +1401,7 @@ static void advk_pcie_handle_msi(struct advk_pcie *pcie)
 
 	msi_mask = advk_readl(pcie, PCIE_MSI_MASK_REG);
 	msi_val = advk_readl(pcie, PCIE_MSI_STATUS_REG);
-	msi_status = msi_val & ~msi_mask;
+	msi_status = msi_val & ((~msi_mask) & PCIE_MSI_ALL_MASK);
 
 	for (msi_idx = 0; msi_idx < MSI_IRQ_NUM; msi_idx++) {
 		if (!(BIT(msi_idx) & msi_status))
-- 
2.32.0

[PATCH 5/7] PCI: aardvark: Disable bus mastering and mask all interrupts when unbinding driver

[Marek Behún]

From: Pali Rohár <pali@kernel.org>

Ensure that after driver unbinding PCIe cards are not be able to forward
memory and I/O requests in the upstream direction and that no interrupt can
be triggered.

Fixes: 526a76991b7b ("PCI: aardvark: Implement driver 'remove' function and allow to build it as module")
Signed-off-by: Pali Rohár <pali@kernel.org>
Signed-off-by: Marek Behún <kabel@kernel.org>
Cc: stable@vger.kernel.org
---
 drivers/pci/controller/pci-aardvark.c | 29 +++++++++++++++++++++++++++
 1 file changed, 29 insertions(+)

diff --git a/drivers/pci/controller/pci-aardvark.c b/drivers/pci/controller/pci-aardvark.c
index 71ce9f02d596..08b34accfe2f 100644
--- a/drivers/pci/controller/pci-aardvark.c
+++ b/drivers/pci/controller/pci-aardvark.c
@@ -1695,13 +1695,42 @@ static int advk_pcie_remove(struct platform_device *pdev)
 {
 	struct advk_pcie *pcie = platform_get_drvdata(pdev);
 	struct pci_host_bridge *bridge = pci_host_bridge_from_priv(pcie);
+	u32 val;
 	int i;
 
+	/* Remove PCI bus with all devices */
 	pci_lock_rescan_remove();
 	pci_stop_root_bus(bridge->bus);
 	pci_remove_root_bus(bridge->bus);
 	pci_unlock_rescan_remove();
 
+	/* Disable Root Bridge I/O space, memory space and bus mastering */
+	val = advk_readl(pcie, PCIE_CORE_CMD_STATUS_REG);
+	val &= ~(PCI_COMMAND_IO | PCI_COMMAND_MEMORY | PCI_COMMAND_MASTER);
+	advk_writel(pcie, val, PCIE_CORE_CMD_STATUS_REG);
+
+	/* Disable MSI */
+	val = advk_readl(pcie, PCIE_CORE_CTRL2_REG);
+	val &= ~PCIE_CORE_CTRL2_MSI_ENABLE;
+	advk_writel(pcie, val, PCIE_CORE_CTRL2_REG);
+
+	/* Clear MSI address */
+	advk_writel(pcie, 0, PCIE_MSI_ADDR_LOW_REG);
+	advk_writel(pcie, 0, PCIE_MSI_ADDR_HIGH_REG);
+
+	/* Mask all interrupts */
+	advk_writel(pcie, PCIE_MSI_ALL_MASK, PCIE_MSI_MASK_REG);
+	advk_writel(pcie, PCIE_ISR0_ALL_MASK, PCIE_ISR0_MASK_REG);
+	advk_writel(pcie, PCIE_ISR1_ALL_MASK, PCIE_ISR1_MASK_REG);
+	advk_writel(pcie, PCIE_IRQ_ALL_MASK, HOST_CTRL_INT_MASK_REG);
+
+	/* Clear all interrupts */
+	advk_writel(pcie, PCIE_MSI_ALL_MASK, PCIE_MSI_STATUS_REG);
+	advk_writel(pcie, PCIE_ISR0_ALL_MASK, PCIE_ISR0_REG);
+	advk_writel(pcie, PCIE_ISR1_ALL_MASK, PCIE_ISR1_REG);
+	advk_writel(pcie, PCIE_IRQ_ALL_MASK, HOST_CTRL_INT_STATUS_REG);
+
+	/* Remove IRQ domains */
 	advk_pcie_remove_msi_irq_domain(pcie);
 	advk_pcie_remove_irq_domain(pcie);
 
-- 
2.32.0

Re: [PATCH 5/7] PCI: aardvark: Disable bus mastering and mask all interrupts when unbinding driver

[Lorenzo Pieralisi]

On Sun, Oct 31, 2021 at 07:12:31PM +0100, Marek Behún wrote:
> From: Pali Rohár <pali@kernel.org>
> 
> Ensure that after driver unbinding PCIe cards are not be able to forward
> memory and I/O requests in the upstream direction and that no interrupt can
> be triggered.

Two actions - likely to require two patches.

> Fixes: 526a76991b7b ("PCI: aardvark: Implement driver 'remove' function and allow to build it as module")
> Signed-off-by: Pali Rohár <pali@kernel.org>
> Signed-off-by: Marek Behún <kabel@kernel.org>
> Cc: stable@vger.kernel.org

You keep adding stable tags, I suppose because you have fixes on top
that will need to go to stable, as said multiple times please let's
not jump the gun, let's fix (if this is fixing anything) mainline
first.

Lorenzo

>  drivers/pci/controller/pci-aardvark.c | 29 +++++++++++++++++++++++++++
>  1 file changed, 29 insertions(+)
> 
> diff --git a/drivers/pci/controller/pci-aardvark.c b/drivers/pci/controller/pci-aardvark.c
> index 71ce9f02d596..08b34accfe2f 100644
> --- a/drivers/pci/controller/pci-aardvark.c
> +++ b/drivers/pci/controller/pci-aardvark.c
> @@ -1695,13 +1695,42 @@ static int advk_pcie_remove(struct platform_device *pdev)
>  {
>  	struct advk_pcie *pcie = platform_get_drvdata(pdev);
>  	struct pci_host_bridge *bridge = pci_host_bridge_from_priv(pcie);
> +	u32 val;
>  	int i;
>  
> +	/* Remove PCI bus with all devices */
>  	pci_lock_rescan_remove();
>  	pci_stop_root_bus(bridge->bus);
>  	pci_remove_root_bus(bridge->bus);
>  	pci_unlock_rescan_remove();
>  
> +	/* Disable Root Bridge I/O space, memory space and bus mastering */
> +	val = advk_readl(pcie, PCIE_CORE_CMD_STATUS_REG);
> +	val &= ~(PCI_COMMAND_IO | PCI_COMMAND_MEMORY | PCI_COMMAND_MASTER);
> +	advk_writel(pcie, val, PCIE_CORE_CMD_STATUS_REG);
> +
> +	/* Disable MSI */
> +	val = advk_readl(pcie, PCIE_CORE_CTRL2_REG);
> +	val &= ~PCIE_CORE_CTRL2_MSI_ENABLE;
> +	advk_writel(pcie, val, PCIE_CORE_CTRL2_REG);
> +
> +	/* Clear MSI address */
> +	advk_writel(pcie, 0, PCIE_MSI_ADDR_LOW_REG);
> +	advk_writel(pcie, 0, PCIE_MSI_ADDR_HIGH_REG);
> +
> +	/* Mask all interrupts */
> +	advk_writel(pcie, PCIE_MSI_ALL_MASK, PCIE_MSI_MASK_REG);
> +	advk_writel(pcie, PCIE_ISR0_ALL_MASK, PCIE_ISR0_MASK_REG);
> +	advk_writel(pcie, PCIE_ISR1_ALL_MASK, PCIE_ISR1_MASK_REG);
> +	advk_writel(pcie, PCIE_IRQ_ALL_MASK, HOST_CTRL_INT_MASK_REG);
> +
> +	/* Clear all interrupts */
> +	advk_writel(pcie, PCIE_MSI_ALL_MASK, PCIE_MSI_STATUS_REG);
> +	advk_writel(pcie, PCIE_ISR0_ALL_MASK, PCIE_ISR0_REG);
> +	advk_writel(pcie, PCIE_ISR1_ALL_MASK, PCIE_ISR1_REG);
> +	advk_writel(pcie, PCIE_IRQ_ALL_MASK, HOST_CTRL_INT_STATUS_REG);
> +
> +	/* Remove IRQ domains */
>  	advk_pcie_remove_msi_irq_domain(pcie);
>  	advk_pcie_remove_irq_domain(pcie);
>  
> -- 
> 2.32.0
> 

[PATCH 6/7] PCI: aardvark: Free config space for emulated root bridge when unbinding driver to fix memory leak

[Marek Behún]

From: Pali Rohár <pali@kernel.org>

Do it after disabling and masking all interrupts, since aardvark interrupt
handler accesses config space of emulated root bridge.

Fixes: 526a76991b7b ("PCI: aardvark: Implement driver 'remove' function and allow to build it as module")
Signed-off-by: Pali Rohár <pali@kernel.org>
Signed-off-by: Marek Behún <kabel@kernel.org>
Cc: stable@vger.kernel.org
---
 drivers/pci/controller/pci-aardvark.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/drivers/pci/controller/pci-aardvark.c b/drivers/pci/controller/pci-aardvark.c
index 08b34accfe2f..b3d89cb449b6 100644
--- a/drivers/pci/controller/pci-aardvark.c
+++ b/drivers/pci/controller/pci-aardvark.c
@@ -1734,6 +1734,9 @@ static int advk_pcie_remove(struct platform_device *pdev)
 	advk_pcie_remove_msi_irq_domain(pcie);
 	advk_pcie_remove_irq_domain(pcie);
 
+	/* Free config space for emulated root bridge */
+	pci_bridge_emul_cleanup(&pcie->bridge);
+
 	/* Disable outbound address windows mapping */
 	for (i = 0; i < OB_WIN_COUNT; i++)
 		advk_pcie_disable_ob_win(pcie, i);
-- 
2.32.0

[PATCH 7/7] PCI: aardvark: Reset PCIe card and disable PHY at driver unbind

[Marek Behún]

From: Pali Rohár <pali@kernel.org>

When unbinding driver, assert PERST# signal which prepares PCIe card for
power down. Then disable link training and PHY.

Fixes: 526a76991b7b ("PCI: aardvark: Implement driver 'remove' function and allow to build it as module")
Signed-off-by: Pali Rohár <pali@kernel.org>
Signed-off-by: Marek Behún <kabel@kernel.org>
Cc: stable@vger.kernel.org
---
 drivers/pci/controller/pci-aardvark.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/drivers/pci/controller/pci-aardvark.c b/drivers/pci/controller/pci-aardvark.c
index b3d89cb449b6..2a82c4652c28 100644
--- a/drivers/pci/controller/pci-aardvark.c
+++ b/drivers/pci/controller/pci-aardvark.c
@@ -1737,10 +1737,22 @@ static int advk_pcie_remove(struct platform_device *pdev)
 	/* Free config space for emulated root bridge */
 	pci_bridge_emul_cleanup(&pcie->bridge);
 
+	/* Assert PERST# signal which prepares PCIe card for power down */
+	if (pcie->reset_gpio)
+		gpiod_set_value_cansleep(pcie->reset_gpio, 1);
+
+	/* Disable link training */
+	val = advk_readl(pcie, PCIE_CORE_CTRL0_REG);
+	val &= ~LINK_TRAINING_EN;
+	advk_writel(pcie, val, PCIE_CORE_CTRL0_REG);
+
 	/* Disable outbound address windows mapping */
 	for (i = 0; i < OB_WIN_COUNT; i++)
 		advk_pcie_disable_ob_win(pcie, i);
 
+	/* Disable phy */
+	advk_pcie_disable_phy(pcie);
+
 	return 0;
 }
 
-- 
2.32.0

Re: [PATCH 7/7] PCI: aardvark: Reset PCIe card and disable PHY at driver unbind

[Lorenzo Pieralisi]

On Sun, Oct 31, 2021 at 07:12:33PM +0100, Marek Behún wrote:
> From: Pali Rohár <pali@kernel.org>
> 
> When unbinding driver, assert PERST# signal which prepares PCIe card for
> power down. Then disable link training and PHY.

This reads as three actions. If we carry them out as a single patch we
have to explain why they are related and what problem they are solving
as a _single_ commit.

Otherwise we have to split this patch into three and explain each of
them as a separate fix.

I understand it is tempting to coalesce missing code in one single
change but every commit must implement a single logical change.

Thanks,
Lorenzo

> Fixes: 526a76991b7b ("PCI: aardvark: Implement driver 'remove' function and allow to build it as module")
> Signed-off-by: Pali Rohár <pali@kernel.org>
> Signed-off-by: Marek Behún <kabel@kernel.org>
> Cc: stable@vger.kernel.org
> ---
>  drivers/pci/controller/pci-aardvark.c | 12 ++++++++++++
>  1 file changed, 12 insertions(+)
> 
> diff --git a/drivers/pci/controller/pci-aardvark.c b/drivers/pci/controller/pci-aardvark.c
> index b3d89cb449b6..2a82c4652c28 100644
> --- a/drivers/pci/controller/pci-aardvark.c
> +++ b/drivers/pci/controller/pci-aardvark.c
> @@ -1737,10 +1737,22 @@ static int advk_pcie_remove(struct platform_device *pdev)
>  	/* Free config space for emulated root bridge */
>  	pci_bridge_emul_cleanup(&pcie->bridge);
>  
> +	/* Assert PERST# signal which prepares PCIe card for power down */
> +	if (pcie->reset_gpio)
> +		gpiod_set_value_cansleep(pcie->reset_gpio, 1);
> +
> +	/* Disable link training */
> +	val = advk_readl(pcie, PCIE_CORE_CTRL0_REG);
> +	val &= ~LINK_TRAINING_EN;
> +	advk_writel(pcie, val, PCIE_CORE_CTRL0_REG);
> +
>  	/* Disable outbound address windows mapping */
>  	for (i = 0; i < OB_WIN_COUNT; i++)
>  		advk_pcie_disable_ob_win(pcie, i);
>  
> +	/* Disable phy */
> +	advk_pcie_disable_phy(pcie);
> +
>  	return 0;
>  }
>  
> -- 
> 2.32.0
> 

Re: [PATCH 7/7] PCI: aardvark: Reset PCIe card and disable PHY at driver unbind

[Marek Behún]

On Mon, 29 Nov 2021 16:40:43 +0000
Lorenzo Pieralisi <lorenzo.pieralisi@arm.com> wrote:

> On Sun, Oct 31, 2021 at 07:12:33PM +0100, Marek Behún wrote:
> > From: Pali Rohár <pali@kernel.org>
> > 
> > When unbinding driver, assert PERST# signal which prepares PCIe card for
> > power down. Then disable link training and PHY.  
> 
> This reads as three actions. If we carry them out as a single patch we
> have to explain why they are related and what problem they are solving
> as a _single_ commit.
> 
> Otherwise we have to split this patch into three and explain each of
> them as a separate fix.
> 
> I understand it is tempting to coalesce missing code in one single
> change but every commit must implement a single logical change.

Hi Lorenzo,

this is a fix for driver remove function. Although each of these things
could be introduced in separate commits, IMO it doesn't make sense to
split it. It should have been done this way in the first place when the
driver removal support was introduced. I guess we could rewrite the
commit message to:

  PCI: aardvark: Disable controller entirely at driver unbind

  Add the following to driver unbind to disable the controller entirely:
  - asserting PERST# signal
  - disabling link training
  - disable PHY

Would this be okay?

Marek

Re: [PATCH 7/7] PCI: aardvark: Reset PCIe card and disable PHY at driver unbind

[Lorenzo Pieralisi]

On Mon, Nov 29, 2021 at 06:15:53PM +0100, Marek Behún wrote:
> On Mon, 29 Nov 2021 16:40:43 +0000
> Lorenzo Pieralisi <lorenzo.pieralisi@arm.com> wrote:
> 
> > On Sun, Oct 31, 2021 at 07:12:33PM +0100, Marek Behún wrote:
> > > From: Pali Rohár <pali@kernel.org>
> > > 
> > > When unbinding driver, assert PERST# signal which prepares PCIe card for
> > > power down. Then disable link training and PHY.  
> > 
> > This reads as three actions. If we carry them out as a single patch we
> > have to explain why they are related and what problem they are solving
> > as a _single_ commit.
> > 
> > Otherwise we have to split this patch into three and explain each of
> > them as a separate fix.
> > 
> > I understand it is tempting to coalesce missing code in one single
> > change but every commit must implement a single logical change.
> 
> Hi Lorenzo,
> 
> this is a fix for driver remove function. Although each of these things
> could be introduced in separate commits, IMO it doesn't make sense to
> split it. It should have been done this way in the first place when the
> driver removal support was introduced. I guess we could rewrite the
> commit message to:
> 
>   PCI: aardvark: Disable controller entirely at driver unbind

"PCI: aardvark: Fix the controller disabling sequence"

>   Add the following to driver unbind to disable the controller entirely:
>   - asserting PERST# signal
>   - disabling link training
>   - disable PHY
> 
> Would this be okay?

Yes, that's what I meant. I would describe the change in its entirety
not as three fixes - it makes sense to have one single patch as long
as we describe it properly.

Thanks,
Lorenzo

Re: [PATCH 7/7] PCI: aardvark: Reset PCIe card and disable PHY at driver unbind

[Marek Behún]

On Tue, 30 Nov 2021 10:31:57 +0000
Lorenzo Pieralisi <lorenzo.pieralisi@arm.com> wrote:

> On Mon, Nov 29, 2021 at 06:15:53PM +0100, Marek Behún wrote:
> > On Mon, 29 Nov 2021 16:40:43 +0000
> > Lorenzo Pieralisi <lorenzo.pieralisi@arm.com> wrote:
> >   
> > > On Sun, Oct 31, 2021 at 07:12:33PM +0100, Marek Behún wrote:  
> > > > From: Pali Rohár <pali@kernel.org>
> > > > 
> > > > When unbinding driver, assert PERST# signal which prepares PCIe card for
> > > > power down. Then disable link training and PHY.    
> > > 
> > > This reads as three actions. If we carry them out as a single patch we
> > > have to explain why they are related and what problem they are solving
> > > as a _single_ commit.
> > > 
> > > Otherwise we have to split this patch into three and explain each of
> > > them as a separate fix.
> > > 
> > > I understand it is tempting to coalesce missing code in one single
> > > change but every commit must implement a single logical change.  
> > 
> > Hi Lorenzo,
> > 
> > this is a fix for driver remove function. Although each of these things
> > could be introduced in separate commits, IMO it doesn't make sense to
> > split it. It should have been done this way in the first place when the
> > driver removal support was introduced. I guess we could rewrite the
> > commit message to:
> > 
> >   PCI: aardvark: Disable controller entirely at driver unbind  
> 
> "PCI: aardvark: Fix the controller disabling sequence"
> 
> >   Add the following to driver unbind to disable the controller entirely:
> >   - asserting PERST# signal
> >   - disabling link training
> >   - disable PHY
> > 
> > Would this be okay?  
> 
> Yes, that's what I meant. I would describe the change in its entirety
> not as three fixes - it makes sense to have one single patch as long
> as we describe it properly.

After I went through it again, I think it would be better to split it
into 3 patches. I am going to do that this way.

Marek