[PATCH v4 00/11] PCI: aardvark controller fixes BATCH 3

[PATCH v4 00/11] PCI: aardvark controller fixes BATCH 3

[Marek Behún]

Dear Lorenzo,

as you requested on IRC, I added more explanation to commit logs of the
last 3 patches.

Changes since v3:
- updated commit messages of patches 9, 10 and 11

Changes since v2:
- updated the second patch, updated definitions of registers
  PCI_EXP_DEVCAP2 and PCI_EXP_DEVCTL2

Changes since v1:
- removed fixes / stable tags
- split the patches as you first suggested, since it makes more sense
  IMO
- changed some commit messages a little

Marek

Pali Rohár (11):
  PCI: pci-bridge-emul: Add description for class_revision field
  PCI: pci-bridge-emul: Add definitions for missing capabilities
    registers
  PCI: aardvark: Add support for DEVCAP2, DEVCTL2, LNKCAP2 and LNKCTL2
    registers on emulated bridge
  PCI: aardvark: Clear all MSIs at setup
  PCI: aardvark: Comment actions in driver remove method
  PCI: aardvark: Disable bus mastering when unbinding driver
  PCI: aardvark: Mask all interrupts when unbinding driver
  PCI: aardvark: Fix memory leak in driver unbind
  PCI: aardvark: Assert PERST# when unbinding driver
  PCI: aardvark: Disable link training when unbinding driver
  PCI: aardvark: Disable common PHY when unbinding driver

 drivers/pci/controller/pci-aardvark.c | 65 ++++++++++++++++++++++++---
 drivers/pci/pci-bridge-emul.c         | 49 +++++++++++++++++++-
 2 files changed, 107 insertions(+), 7 deletions(-)

-- 
2.32.0

[PATCH v4 01/11] PCI: pci-bridge-emul: Add description for class_revision field

[Marek Behún]

From: Pali Rohár <pali@kernel.org>

The current assignment to the class_revision member

  class_revision |= cpu_to_le32(PCI_CLASS_BRIDGE_PCI << 16);

can make the reader think that class is at high 16 bits of the member and
revision at low 16 bits.

In reality, class is at high 24 bits, but the class for PCI Bridge Normal
Decode is PCI_CLASS_BRIDGE_PCI << 8.

Change the assignment and add a comment to make this clearer.

Signed-off-by: Pali Rohár <pali@kernel.org>
Signed-off-by: Marek Behún <kabel@kernel.org>
---
 drivers/pci/pci-bridge-emul.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/drivers/pci/pci-bridge-emul.c b/drivers/pci/pci-bridge-emul.c
index db97cddfc85e..a4af1a533d71 100644
--- a/drivers/pci/pci-bridge-emul.c
+++ b/drivers/pci/pci-bridge-emul.c
@@ -265,7 +265,11 @@ int pci_bridge_emul_init(struct pci_bridge_emul *bridge,
 {
 	BUILD_BUG_ON(sizeof(bridge->conf) != PCI_BRIDGE_CONF_END);
 
-	bridge->conf.class_revision |= cpu_to_le32(PCI_CLASS_BRIDGE_PCI << 16);
+	/*
+	 * class_revision: Class is high 24 bits and revision is low 8 bit of this member,
+	 * while class for PCI Bridge Normal Decode has the 24-bit value: PCI_CLASS_BRIDGE_PCI << 8
+	 */
+	bridge->conf.class_revision |= cpu_to_le32((PCI_CLASS_BRIDGE_PCI << 8) << 8);
 	bridge->conf.header_type = PCI_HEADER_TYPE_BRIDGE;
 	bridge->conf.cache_line_size = 0x10;
 	bridge->conf.status = cpu_to_le16(PCI_STATUS_CAP_LIST);
-- 
2.32.0

Re: [PATCH v4 01/11] PCI: pci-bridge-emul: Add description for class_revision field

[Bjorn Helgaas]

On Tue, Nov 30, 2021 at 06:29:03PM +0100, Marek Behún wrote:
> From: Pali Rohár <pali@kernel.org>
> 
> The current assignment to the class_revision member
> 
>   class_revision |= cpu_to_le32(PCI_CLASS_BRIDGE_PCI << 16);
> 
> can make the reader think that class is at high 16 bits of the member and
> revision at low 16 bits.
> 
> In reality, class is at high 24 bits, but the class for PCI Bridge Normal
> Decode is PCI_CLASS_BRIDGE_PCI << 8.
> 
> Change the assignment and add a comment to make this clearer.
> 
> Signed-off-by: Pali Rohár <pali@kernel.org>
> Signed-off-by: Marek Behún <kabel@kernel.org>
> ---
>  drivers/pci/pci-bridge-emul.c | 6 +++++-
>  1 file changed, 5 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/pci/pci-bridge-emul.c b/drivers/pci/pci-bridge-emul.c
> index db97cddfc85e..a4af1a533d71 100644
> --- a/drivers/pci/pci-bridge-emul.c
> +++ b/drivers/pci/pci-bridge-emul.c
> @@ -265,7 +265,11 @@ int pci_bridge_emul_init(struct pci_bridge_emul *bridge,
>  {
>  	BUILD_BUG_ON(sizeof(bridge->conf) != PCI_BRIDGE_CONF_END);
>  
> -	bridge->conf.class_revision |= cpu_to_le32(PCI_CLASS_BRIDGE_PCI << 16);
> +	/*
> +	 * class_revision: Class is high 24 bits and revision is low 8 bit of this member,
> +	 * while class for PCI Bridge Normal Decode has the 24-bit value: PCI_CLASS_BRIDGE_PCI << 8
> +	 */

Can you please re-wrap this comment so it fits in 80 columns like the
rest of the file?

I'd do something with the assignment, too.  It's hard to read when it
wraps, especially since at 80 columns it splits the "<<" in half.

I assume from the commit log that this is purely a readability change,
not a fix, right?

> +	bridge->conf.class_revision |= cpu_to_le32((PCI_CLASS_BRIDGE_PCI << 8) << 8);
>  	bridge->conf.header_type = PCI_HEADER_TYPE_BRIDGE;
>  	bridge->conf.cache_line_size = 0x10;
>  	bridge->conf.status = cpu_to_le16(PCI_STATUS_CAP_LIST);
> -- 
> 2.32.0
> 

Re: [PATCH v4 01/11] PCI: pci-bridge-emul: Add description for class_revision field

[Marek Behún]

On Fri, 3 Dec 2021 10:36:49 -0600
Bjorn Helgaas <helgaas@kernel.org> wrote:

> On Tue, Nov 30, 2021 at 06:29:03PM +0100, Marek Behún wrote:
> > From: Pali Rohár <pali@kernel.org>
> > 
> > The current assignment to the class_revision member
> > 
> >   class_revision |= cpu_to_le32(PCI_CLASS_BRIDGE_PCI << 16);
> > 
> > can make the reader think that class is at high 16 bits of the member and
> > revision at low 16 bits.
> > 
> > In reality, class is at high 24 bits, but the class for PCI Bridge Normal
> > Decode is PCI_CLASS_BRIDGE_PCI << 8.
> > 
> > Change the assignment and add a comment to make this clearer.
> > 
> > Signed-off-by: Pali Rohár <pali@kernel.org>
> > Signed-off-by: Marek Behún <kabel@kernel.org>
> > ---
> >  drivers/pci/pci-bridge-emul.c | 6 +++++-
> >  1 file changed, 5 insertions(+), 1 deletion(-)
> > 
> > diff --git a/drivers/pci/pci-bridge-emul.c b/drivers/pci/pci-bridge-emul.c
> > index db97cddfc85e..a4af1a533d71 100644
> > --- a/drivers/pci/pci-bridge-emul.c
> > +++ b/drivers/pci/pci-bridge-emul.c
> > @@ -265,7 +265,11 @@ int pci_bridge_emul_init(struct pci_bridge_emul *bridge,
> >  {
> >  	BUILD_BUG_ON(sizeof(bridge->conf) != PCI_BRIDGE_CONF_END);
> >  
> > -	bridge->conf.class_revision |= cpu_to_le32(PCI_CLASS_BRIDGE_PCI << 16);
> > +	/*
> > +	 * class_revision: Class is high 24 bits and revision is low 8 bit of this member,
> > +	 * while class for PCI Bridge Normal Decode has the 24-bit value: PCI_CLASS_BRIDGE_PCI << 8
> > +	 */  
> 
> Can you please re-wrap this comment so it fits in 80 columns like the
> rest of the file?

Bjorn, Lorenzo already applied this patches to his branch. Should I
send him a fixup, or try to persuade him to rebase? :)

> I'd do something with the assignment, too.  It's hard to read when it
> wraps, especially since at 80 columns it splits the "<<" in half.
> 
> I assume from the commit log that this is purely a readability change,
> not a fix, right?

Yes, you are right. No functional change.

[PATCH v4 02/11] PCI: pci-bridge-emul: Add definitions for missing capabilities registers

[Marek Behún]

From: Pali Rohár <pali@kernel.org>

pci-bridge-emul driver already allocates buffer for capabilities up to the
PCI_EXP_SLTSTA2 register, but does not define bit access behavior for these
registers. Add these missing definitions.

Signed-off-by: Pali Rohár <pali@kernel.org>
Signed-off-by: Marek Behún <kabel@kernel.org>
---
 drivers/pci/pci-bridge-emul.c | 43 +++++++++++++++++++++++++++++++++++
 1 file changed, 43 insertions(+)

diff --git a/drivers/pci/pci-bridge-emul.c b/drivers/pci/pci-bridge-emul.c
index a4af1a533d71..0d1177e52a43 100644
--- a/drivers/pci/pci-bridge-emul.c
+++ b/drivers/pci/pci-bridge-emul.c
@@ -251,6 +251,49 @@ struct pci_bridge_reg_behavior pcie_cap_regs_behavior[PCI_CAP_PCIE_SIZEOF / 4] =
 		.ro = GENMASK(15, 0) | PCI_EXP_RTSTA_PENDING,
 		.w1c = PCI_EXP_RTSTA_PME,
 	},
+
+	[PCI_EXP_DEVCAP2 / 4] = {
+		/*
+		 * Device capabilities 2 register has reserved bits [30:27].
+		 * Also bits [26:24] are reserved for non-upstream ports.
+		 */
+		.ro = BIT(31) | GENMASK(23, 0),
+	},
+
+	[PCI_EXP_DEVCTL2 / 4] = {
+		/*
+		 * Device control 2 register is RW. Bit 11 is reserved for
+		 * non-upstream ports.
+		 *
+		 * Device status 2 register is reserved.
+		 */
+		.rw = GENMASK(15, 12) | GENMASK(10, 0),
+	},
+
+	[PCI_EXP_LNKCAP2 / 4] = {
+		/* Link capabilities 2 register has reserved bits [30:25] and 0. */
+		.ro = BIT(31) | GENMASK(24, 1),
+	},
+
+	[PCI_EXP_LNKCTL2 / 4] = {
+		/*
+		 * Link control 2 register is RW.
+		 *
+		 * Link status 2 register has bits 5, 15 W1C;
+		 * bits 10, 11 reserved and others are RO.
+		 */
+		.rw = GENMASK(15, 0),
+		.w1c = (BIT(15) | BIT(5)) << 16,
+		.ro = (GENMASK(14, 12) | GENMASK(9, 6) | GENMASK(4, 0)) << 16,
+	},
+
+	[PCI_EXP_SLTCAP2 / 4] = {
+		/* Slot capabilities 2 register is reserved. */
+	},
+
+	[PCI_EXP_SLTCTL2 / 4] = {
+		/* Both Slot control 2 and Slot status 2 registers are reserved. */
+	},
 };
 
 /*
-- 
2.32.0

Re: [PATCH v4 02/11] PCI: pci-bridge-emul: Add definitions for missing capabilities registers

[Bjorn Helgaas]

On Tue, Nov 30, 2021 at 06:29:04PM +0100, Marek Behún wrote:
> From: Pali Rohár <pali@kernel.org>
> 
> pci-bridge-emul driver already allocates buffer for capabilities up to the
> PCI_EXP_SLTSTA2 register, but does not define bit access behavior for these
> registers. Add these missing definitions.
> 
> Signed-off-by: Pali Rohár <pali@kernel.org>
> Signed-off-by: Marek Behún <kabel@kernel.org>
> ---
>  drivers/pci/pci-bridge-emul.c | 43 +++++++++++++++++++++++++++++++++++
>  1 file changed, 43 insertions(+)
> 
> diff --git a/drivers/pci/pci-bridge-emul.c b/drivers/pci/pci-bridge-emul.c
> index a4af1a533d71..0d1177e52a43 100644
> --- a/drivers/pci/pci-bridge-emul.c
> +++ b/drivers/pci/pci-bridge-emul.c
> @@ -251,6 +251,49 @@ struct pci_bridge_reg_behavior pcie_cap_regs_behavior[PCI_CAP_PCIE_SIZEOF / 4] =
>  		.ro = GENMASK(15, 0) | PCI_EXP_RTSTA_PENDING,
>  		.w1c = PCI_EXP_RTSTA_PME,
>  	},
> +
> +	[PCI_EXP_DEVCAP2 / 4] = {
> +		/*
> +		 * Device capabilities 2 register has reserved bits [30:27].
> +		 * Also bits [26:24] are reserved for non-upstream ports.
> +		 */
> +		.ro = BIT(31) | GENMASK(23, 0),
> +	},
> +
> +	[PCI_EXP_DEVCTL2 / 4] = {
> +		/*
> +		 * Device control 2 register is RW. Bit 11 is reserved for
> +		 * non-upstream ports.
> +		 *
> +		 * Device status 2 register is reserved.
> +		 */
> +		.rw = GENMASK(15, 12) | GENMASK(10, 0),
> +	},
> +
> +	[PCI_EXP_LNKCAP2 / 4] = {
> +		/* Link capabilities 2 register has reserved bits [30:25] and 0. */

Rewrap (also below).

> +		.ro = BIT(31) | GENMASK(24, 1),
> +	},
> +
> +	[PCI_EXP_LNKCTL2 / 4] = {
> +		/*
> +		 * Link control 2 register is RW.
> +		 *
> +		 * Link status 2 register has bits 5, 15 W1C;
> +		 * bits 10, 11 reserved and others are RO.
> +		 */
> +		.rw = GENMASK(15, 0),
> +		.w1c = (BIT(15) | BIT(5)) << 16,
> +		.ro = (GENMASK(14, 12) | GENMASK(9, 6) | GENMASK(4, 0)) << 16,
> +	},
> +
> +	[PCI_EXP_SLTCAP2 / 4] = {
> +		/* Slot capabilities 2 register is reserved. */
> +	},
> +
> +	[PCI_EXP_SLTCTL2 / 4] = {
> +		/* Both Slot control 2 and Slot status 2 registers are reserved. */
> +	},
>  };
>  
>  /*
> -- 
> 2.32.0
> 

[PATCH v4 03/11] PCI: aardvark: Add support for DEVCAP2, DEVCTL2, LNKCAP2 and LNKCTL2 registers on emulated bridge

[Marek Behún]

From: Pali Rohár <pali@kernel.org>

PCI aardvark hardware supports access to DEVCAP2, DEVCTL2, LNKCAP2 and
LNKCTL2 configuration registers of PCIe core via PCIE_CORE_PCIEXP_CAP.
Export them via emulated software root bridge.

Signed-off-by: Pali Rohár <pali@kernel.org>
Signed-off-by: Marek Behún <kabel@kernel.org>
---
 drivers/pci/controller/pci-aardvark.c | 15 +++++++++++----
 1 file changed, 11 insertions(+), 4 deletions(-)

diff --git a/drivers/pci/controller/pci-aardvark.c b/drivers/pci/controller/pci-aardvark.c
index c5300d49807a..25af189a1052 100644
--- a/drivers/pci/controller/pci-aardvark.c
+++ b/drivers/pci/controller/pci-aardvark.c
@@ -884,8 +884,13 @@ advk_pci_bridge_emul_pcie_conf_read(struct pci_bridge_emul *bridge,
 	case PCI_CAP_LIST_ID:
 	case PCI_EXP_DEVCAP:
 	case PCI_EXP_DEVCTL:
+	case PCI_EXP_DEVCAP2:
+	case PCI_EXP_DEVCTL2:
+	case PCI_EXP_LNKCAP2:
+	case PCI_EXP_LNKCTL2:
 		*value = advk_readl(pcie, PCIE_CORE_PCIEXP_CAP + reg);
 		return PCI_BRIDGE_EMUL_HANDLED;
+
 	default:
 		return PCI_BRIDGE_EMUL_NOT_HANDLED;
 	}
@@ -899,10 +904,6 @@ advk_pci_bridge_emul_pcie_conf_write(struct pci_bridge_emul *bridge,
 	struct advk_pcie *pcie = bridge->data;
 
 	switch (reg) {
-	case PCI_EXP_DEVCTL:
-		advk_writel(pcie, new, PCIE_CORE_PCIEXP_CAP + reg);
-		break;
-
 	case PCI_EXP_LNKCTL:
 		advk_writel(pcie, new, PCIE_CORE_PCIEXP_CAP + reg);
 		if (new & PCI_EXP_LNKCTL_RL)
@@ -924,6 +925,12 @@ advk_pci_bridge_emul_pcie_conf_write(struct pci_bridge_emul *bridge,
 		advk_writel(pcie, new, PCIE_ISR0_REG);
 		break;
 
+	case PCI_EXP_DEVCTL:
+	case PCI_EXP_DEVCTL2:
+	case PCI_EXP_LNKCTL2:
+		advk_writel(pcie, new, PCIE_CORE_PCIEXP_CAP + reg);
+		break;
+
 	default:
 		break;
 	}
-- 
2.32.0

[PATCH v4 04/11] PCI: aardvark: Clear all MSIs at setup

[Marek Behún]

From: Pali Rohár <pali@kernel.org>

We already clear all the other interrupts (ISR0, ISR1, HOST_CTRL_INT).

Define a new macro PCIE_MSI_ALL_MASK and do the same clearing for MSIs,
to ensure that we don't start receiving spurious interrupts.

Use this new mask in advk_pcie_handle_msi();

Signed-off-by: Pali Rohár <pali@kernel.org>
Signed-off-by: Marek Behún <kabel@kernel.org>
---
 drivers/pci/controller/pci-aardvark.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/drivers/pci/controller/pci-aardvark.c b/drivers/pci/controller/pci-aardvark.c
index 25af189a1052..71ce9f02d596 100644
--- a/drivers/pci/controller/pci-aardvark.c
+++ b/drivers/pci/controller/pci-aardvark.c
@@ -116,6 +116,7 @@
 #define PCIE_MSI_ADDR_HIGH_REG			(CONTROL_BASE_ADDR + 0x54)
 #define PCIE_MSI_STATUS_REG			(CONTROL_BASE_ADDR + 0x58)
 #define PCIE_MSI_MASK_REG			(CONTROL_BASE_ADDR + 0x5C)
+#define     PCIE_MSI_ALL_MASK			GENMASK(31, 0)
 #define PCIE_MSI_PAYLOAD_REG			(CONTROL_BASE_ADDR + 0x9C)
 #define     PCIE_MSI_DATA_MASK			GENMASK(15, 0)
 
@@ -571,6 +572,7 @@ static void advk_pcie_setup_hw(struct advk_pcie *pcie)
 	advk_writel(pcie, reg, PCIE_CORE_CTRL2_REG);
 
 	/* Clear all interrupts */
+	advk_writel(pcie, PCIE_MSI_ALL_MASK, PCIE_MSI_STATUS_REG);
 	advk_writel(pcie, PCIE_ISR0_ALL_MASK, PCIE_ISR0_REG);
 	advk_writel(pcie, PCIE_ISR1_ALL_MASK, PCIE_ISR1_REG);
 	advk_writel(pcie, PCIE_IRQ_ALL_MASK, HOST_CTRL_INT_STATUS_REG);
@@ -583,7 +585,7 @@ static void advk_pcie_setup_hw(struct advk_pcie *pcie)
 	advk_writel(pcie, PCIE_ISR1_ALL_MASK, PCIE_ISR1_MASK_REG);
 
 	/* Unmask all MSIs */
-	advk_writel(pcie, 0, PCIE_MSI_MASK_REG);
+	advk_writel(pcie, ~(u32)PCIE_MSI_ALL_MASK, PCIE_MSI_MASK_REG);
 
 	/* Enable summary interrupt for GIC SPI source */
 	reg = PCIE_IRQ_ALL_MASK & (~PCIE_IRQ_ENABLE_INTS_MASK);
@@ -1399,7 +1401,7 @@ static void advk_pcie_handle_msi(struct advk_pcie *pcie)
 
 	msi_mask = advk_readl(pcie, PCIE_MSI_MASK_REG);
 	msi_val = advk_readl(pcie, PCIE_MSI_STATUS_REG);
-	msi_status = msi_val & ~msi_mask;
+	msi_status = msi_val & ((~msi_mask) & PCIE_MSI_ALL_MASK);
 
 	for (msi_idx = 0; msi_idx < MSI_IRQ_NUM; msi_idx++) {
 		if (!(BIT(msi_idx) & msi_status))
-- 
2.32.0

[PATCH v4 05/11] PCI: aardvark: Comment actions in driver remove method

[Marek Behún]

From: Pali Rohár <pali@kernel.org>

Add two more comments into the advk_pcie_remove() method.

Signed-off-by: Pali Rohár <pali@kernel.org>
Signed-off-by: Marek Behún <kabel@kernel.org>
---
 drivers/pci/controller/pci-aardvark.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/drivers/pci/controller/pci-aardvark.c b/drivers/pci/controller/pci-aardvark.c
index 71ce9f02d596..6348584c33be 100644
--- a/drivers/pci/controller/pci-aardvark.c
+++ b/drivers/pci/controller/pci-aardvark.c
@@ -1697,11 +1697,13 @@ static int advk_pcie_remove(struct platform_device *pdev)
 	struct pci_host_bridge *bridge = pci_host_bridge_from_priv(pcie);
 	int i;
 
+	/* Remove PCI bus with all devices */
 	pci_lock_rescan_remove();
 	pci_stop_root_bus(bridge->bus);
 	pci_remove_root_bus(bridge->bus);
 	pci_unlock_rescan_remove();
 
+	/* Remove IRQ domains */
 	advk_pcie_remove_msi_irq_domain(pcie);
 	advk_pcie_remove_irq_domain(pcie);
 
-- 
2.32.0

[PATCH v4 06/11] PCI: aardvark: Disable bus mastering when unbinding driver

[Marek Behún]

From: Pali Rohár <pali@kernel.org>

Ensure that after driver unbind PCIe cards are not able to forward
memory and I/O requests in the upstream direction.

Signed-off-by: Pali Rohár <pali@kernel.org>
Signed-off-by: Marek Behún <kabel@kernel.org>
---
 drivers/pci/controller/pci-aardvark.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/drivers/pci/controller/pci-aardvark.c b/drivers/pci/controller/pci-aardvark.c
index 6348584c33be..12eae05f3d10 100644
--- a/drivers/pci/controller/pci-aardvark.c
+++ b/drivers/pci/controller/pci-aardvark.c
@@ -1695,6 +1695,7 @@ static int advk_pcie_remove(struct platform_device *pdev)
 {
 	struct advk_pcie *pcie = platform_get_drvdata(pdev);
 	struct pci_host_bridge *bridge = pci_host_bridge_from_priv(pcie);
+	u32 val;
 	int i;
 
 	/* Remove PCI bus with all devices */
@@ -1703,6 +1704,11 @@ static int advk_pcie_remove(struct platform_device *pdev)
 	pci_remove_root_bus(bridge->bus);
 	pci_unlock_rescan_remove();
 
+	/* Disable Root Bridge I/O space, memory space and bus mastering */
+	val = advk_readl(pcie, PCIE_CORE_CMD_STATUS_REG);
+	val &= ~(PCI_COMMAND_IO | PCI_COMMAND_MEMORY | PCI_COMMAND_MASTER);
+	advk_writel(pcie, val, PCIE_CORE_CMD_STATUS_REG);
+
 	/* Remove IRQ domains */
 	advk_pcie_remove_msi_irq_domain(pcie);
 	advk_pcie_remove_irq_domain(pcie);
-- 
2.32.0

[PATCH v4 07/11] PCI: aardvark: Mask all interrupts when unbinding driver

[Marek Behún]

From: Pali Rohár <pali@kernel.org>

Ensure that no interrupt can be triggered after driver unbind.

Signed-off-by: Pali Rohár <pali@kernel.org>
Signed-off-by: Marek Behún <kabel@kernel.org>
---
 drivers/pci/controller/pci-aardvark.c | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)

diff --git a/drivers/pci/controller/pci-aardvark.c b/drivers/pci/controller/pci-aardvark.c
index 12eae05f3d10..08b34accfe2f 100644
--- a/drivers/pci/controller/pci-aardvark.c
+++ b/drivers/pci/controller/pci-aardvark.c
@@ -1709,6 +1709,27 @@ static int advk_pcie_remove(struct platform_device *pdev)
 	val &= ~(PCI_COMMAND_IO | PCI_COMMAND_MEMORY | PCI_COMMAND_MASTER);
 	advk_writel(pcie, val, PCIE_CORE_CMD_STATUS_REG);
 
+	/* Disable MSI */
+	val = advk_readl(pcie, PCIE_CORE_CTRL2_REG);
+	val &= ~PCIE_CORE_CTRL2_MSI_ENABLE;
+	advk_writel(pcie, val, PCIE_CORE_CTRL2_REG);
+
+	/* Clear MSI address */
+	advk_writel(pcie, 0, PCIE_MSI_ADDR_LOW_REG);
+	advk_writel(pcie, 0, PCIE_MSI_ADDR_HIGH_REG);
+
+	/* Mask all interrupts */
+	advk_writel(pcie, PCIE_MSI_ALL_MASK, PCIE_MSI_MASK_REG);
+	advk_writel(pcie, PCIE_ISR0_ALL_MASK, PCIE_ISR0_MASK_REG);
+	advk_writel(pcie, PCIE_ISR1_ALL_MASK, PCIE_ISR1_MASK_REG);
+	advk_writel(pcie, PCIE_IRQ_ALL_MASK, HOST_CTRL_INT_MASK_REG);
+
+	/* Clear all interrupts */
+	advk_writel(pcie, PCIE_MSI_ALL_MASK, PCIE_MSI_STATUS_REG);
+	advk_writel(pcie, PCIE_ISR0_ALL_MASK, PCIE_ISR0_REG);
+	advk_writel(pcie, PCIE_ISR1_ALL_MASK, PCIE_ISR1_REG);
+	advk_writel(pcie, PCIE_IRQ_ALL_MASK, HOST_CTRL_INT_STATUS_REG);
+
 	/* Remove IRQ domains */
 	advk_pcie_remove_msi_irq_domain(pcie);
 	advk_pcie_remove_irq_domain(pcie);
-- 
2.32.0

[PATCH v4 08/11] PCI: aardvark: Fix memory leak in driver unbind

[Marek Behún]

From: Pali Rohár <pali@kernel.org>

Free config space for emulated root bridge when unbinding driver to fix
memory leak. Do it after disabling and masking all interrupts, since
aardvark interrupt handler accesses config space of emulated root
bridge.

Signed-off-by: Pali Rohár <pali@kernel.org>
Signed-off-by: Marek Behún <kabel@kernel.org>
---
 drivers/pci/controller/pci-aardvark.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/drivers/pci/controller/pci-aardvark.c b/drivers/pci/controller/pci-aardvark.c
index 08b34accfe2f..b3d89cb449b6 100644
--- a/drivers/pci/controller/pci-aardvark.c
+++ b/drivers/pci/controller/pci-aardvark.c
@@ -1734,6 +1734,9 @@ static int advk_pcie_remove(struct platform_device *pdev)
 	advk_pcie_remove_msi_irq_domain(pcie);
 	advk_pcie_remove_irq_domain(pcie);
 
+	/* Free config space for emulated root bridge */
+	pci_bridge_emul_cleanup(&pcie->bridge);
+
 	/* Disable outbound address windows mapping */
 	for (i = 0; i < OB_WIN_COUNT; i++)
 		advk_pcie_disable_ob_win(pcie, i);
-- 
2.32.0

[PATCH v4 09/11] PCI: aardvark: Assert PERST# when unbinding driver

[Marek Behún]

From: Pali Rohár <pali@kernel.org>

Put the PCIe card into reset by asserting PERST# signal when unbinding
driver. It doesn't make sense to leave the card working if it can't
communicate with the host. This should also save some power.

Signed-off-by: Pali Rohár <pali@kernel.org>
Signed-off-by: Marek Behún <kabel@kernel.org>
---
 drivers/pci/controller/pci-aardvark.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/drivers/pci/controller/pci-aardvark.c b/drivers/pci/controller/pci-aardvark.c
index b3d89cb449b6..271ebecee965 100644
--- a/drivers/pci/controller/pci-aardvark.c
+++ b/drivers/pci/controller/pci-aardvark.c
@@ -1737,6 +1737,10 @@ static int advk_pcie_remove(struct platform_device *pdev)
 	/* Free config space for emulated root bridge */
 	pci_bridge_emul_cleanup(&pcie->bridge);
 
+	/* Assert PERST# signal which prepares PCIe card for power down */
+	if (pcie->reset_gpio)
+		gpiod_set_value_cansleep(pcie->reset_gpio, 1);
+
 	/* Disable outbound address windows mapping */
 	for (i = 0; i < OB_WIN_COUNT; i++)
 		advk_pcie_disable_ob_win(pcie, i);
-- 
2.32.0

[PATCH v4 10/11] PCI: aardvark: Disable link training when unbinding driver

[Marek Behún]

From: Pali Rohár <pali@kernel.org>

Disable link training circuit in driver unbind sequence. We want to
leave link training in the same state as it was before the driver was
probed.

Signed-off-by: Pali Rohár <pali@kernel.org>
Signed-off-by: Marek Behún <kabel@kernel.org>
---
 drivers/pci/controller/pci-aardvark.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/drivers/pci/controller/pci-aardvark.c b/drivers/pci/controller/pci-aardvark.c
index 271ebecee965..e5c88f1c177b 100644
--- a/drivers/pci/controller/pci-aardvark.c
+++ b/drivers/pci/controller/pci-aardvark.c
@@ -1741,6 +1741,11 @@ static int advk_pcie_remove(struct platform_device *pdev)
 	if (pcie->reset_gpio)
 		gpiod_set_value_cansleep(pcie->reset_gpio, 1);
 
+	/* Disable link training */
+	val = advk_readl(pcie, PCIE_CORE_CTRL0_REG);
+	val &= ~LINK_TRAINING_EN;
+	advk_writel(pcie, val, PCIE_CORE_CTRL0_REG);
+
 	/* Disable outbound address windows mapping */
 	for (i = 0; i < OB_WIN_COUNT; i++)
 		advk_pcie_disable_ob_win(pcie, i);
-- 
2.32.0

[PATCH v4 11/11] PCI: aardvark: Disable common PHY when unbinding driver

[Marek Behún]

From: Pali Rohár <pali@kernel.org>

Disable the PCIe PHY when unbinding driver. This should save some power.

Signed-off-by: Pali Rohár <pali@kernel.org>
Signed-off-by: Marek Behún <kabel@kernel.org>
---
 drivers/pci/controller/pci-aardvark.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/drivers/pci/controller/pci-aardvark.c b/drivers/pci/controller/pci-aardvark.c
index e5c88f1c177b..2a82c4652c28 100644
--- a/drivers/pci/controller/pci-aardvark.c
+++ b/drivers/pci/controller/pci-aardvark.c
@@ -1750,6 +1750,9 @@ static int advk_pcie_remove(struct platform_device *pdev)
 	for (i = 0; i < OB_WIN_COUNT; i++)
 		advk_pcie_disable_ob_win(pcie, i);
 
+	/* Disable phy */
+	advk_pcie_disable_phy(pcie);
+
 	return 0;
 }
 
-- 
2.32.0

Re: [PATCH v4 00/11] PCI: aardvark controller fixes BATCH 3

[Lorenzo Pieralisi]

On Tue, 30 Nov 2021 18:29:02 +0100, Marek Behún wrote:
> as you requested on IRC, I added more explanation to commit logs of the
> last 3 patches.
> 
> Changes since v3:
> - updated commit messages of patches 9, 10 and 11
> 
> Changes since v2:
> - updated the second patch, updated definitions of registers
>   PCI_EXP_DEVCAP2 and PCI_EXP_DEVCTL2
> 
> [...]

Applied to pci/aardvark, thanks!

[01/11] PCI: pci-bridge-emul: Add description for class_revision field
        https://git.kernel.org/lpieralisi/pci/c/9319230ac1
[02/11] PCI: pci-bridge-emul: Add definitions for missing capabilities registers
        https://git.kernel.org/lpieralisi/pci/c/8ea673a8b3
[03/11] PCI: aardvark: Add support for DEVCAP2, DEVCTL2, LNKCAP2 and LNKCTL2 registers on emulated bridge
        https://git.kernel.org/lpieralisi/pci/c/1d3e170344
[04/11] PCI: aardvark: Clear all MSIs at setup
        https://git.kernel.org/lpieralisi/pci/c/7d8dc1f7cd
[05/11] PCI: aardvark: Comment actions in driver remove method
        https://git.kernel.org/lpieralisi/pci/c/a4ca7948e1
[06/11] PCI: aardvark: Disable bus mastering when unbinding driver
        https://git.kernel.org/lpieralisi/pci/c/a46f2f6dd4
[07/11] PCI: aardvark: Mask all interrupts when unbinding driver
        https://git.kernel.org/lpieralisi/pci/c/13bcdf07cb
[08/11] PCI: aardvark: Fix memory leak in driver unbind
        https://git.kernel.org/lpieralisi/pci/c/2f040a17f5
[09/11] PCI: aardvark: Assert PERST# when unbinding driver
        https://git.kernel.org/lpieralisi/pci/c/1f54391be8
[10/11] PCI: aardvark: Disable link training when unbinding driver
        https://git.kernel.org/lpieralisi/pci/c/759dec2e3d
[11/11] PCI: aardvark: Disable common PHY when unbinding driver
        https://git.kernel.org/lpieralisi/pci/c/fdbbe242c1

Thanks,
Lorenzo