[PATCH pci-fixes 0/2] PCI Aardvark controller fixes

[PATCH pci-fixes 0/2] PCI Aardvark controller fixes

[Marek Behún]

Hello Lorenzo, Bjorn,

here are two fixes for pci-aardvark controller.

Marek

Marek Behún (1):
  Revert "PCI: aardvark: Fix support for PCI_ROM_ADDRESS1 on emulated
    bridge"

Pali Rohár (1):
  PCI: aardvark: Fix checking for MEM resource type

 drivers/pci/controller/pci-aardvark.c | 15 ++-------------
 1 file changed, 2 insertions(+), 13 deletions(-)

-- 
2.32.0

[PATCH pci-fixes 1/2] PCI: aardvark: Fix checking for MEM resource type

[Marek Behún]

From: Pali Rohár <pali@kernel.org>

IORESOURCE_MEM_64 is not type but type flag. Remove incorrect check for
type IORESOURCE_MEM_64.

Fixes: 64f160e19e92 ("PCI: aardvark: Configure PCIe resources from 'ranges' DT property")
Signed-off-by: Pali Rohár <pali@kernel.org>
Signed-off-by: Marek Behún <kabel@kernel.org>
---
 drivers/pci/controller/pci-aardvark.c | 6 ++----
 1 file changed, 2 insertions(+), 4 deletions(-)

diff --git a/drivers/pci/controller/pci-aardvark.c b/drivers/pci/controller/pci-aardvark.c
index c5300d49807a..baa62cdcaab4 100644
--- a/drivers/pci/controller/pci-aardvark.c
+++ b/drivers/pci/controller/pci-aardvark.c
@@ -1544,8 +1544,7 @@ static int advk_pcie_probe(struct platform_device *pdev)
 		 * only PIO for issuing configuration transfers which does
 		 * not use PCIe window configuration.
 		 */
-		if (type != IORESOURCE_MEM && type != IORESOURCE_MEM_64 &&
-		    type != IORESOURCE_IO)
+		if (type != IORESOURCE_MEM && type != IORESOURCE_IO)
 			continue;
 
 		/*
@@ -1553,8 +1552,7 @@ static int advk_pcie_probe(struct platform_device *pdev)
 		 * configuration is set to transparent memory access so it
 		 * does not need window configuration.
 		 */
-		if ((type == IORESOURCE_MEM || type == IORESOURCE_MEM_64) &&
-		    entry->offset == 0)
+		if (type == IORESOURCE_MEM && entry->offset == 0)
 			continue;
 
 		/*
-- 
2.32.0

[PATCH pci-fixes 2/2] Revert "PCI: aardvark: Fix support for PCI_ROM_ADDRESS1 on emulated bridge"

[Marek Behún]

This reverts commit 239edf686c14a9ff926dec2f350289ed7adfefe2.

PCI Bridge which represents aardvark's PCIe Root Port has Expansion ROM
Base Address register at offset 0x30, but its meaning is different than
PCI's Expansion ROM BAR register, although the layout is the same.
(This is why we thought it does the same thing.)

First: there is no ROM (or part of BootROM) in the A3720 SOC dedicated
for PCIe Root Port (or controller in RC mode) containing executable code
that would initialize the Root Port, suitable for execution in
bootloader (this is how Expansion ROM BAR is used on x86).

Second: in A3720 spec the register (address D0070030) is not documented
at all for Root Complex mode, but similar to other BAR registers, it has
an "entangled partner" in register D0075920, which does address
translation for the BAR in D0070030:
- the BAR register sets the address from the view of PCIe bus
- the translation register sets the address from the view of the CPU

The other BAR registers also have this entangled partner, and they
can be used to:
- in RC mode: address-checking on the receive side of the RC (they
  can define address ranges for memory accesses from remote Endpoints
  to the RC)
- in Endpoint mode: allow the remote CPU to access memory on A3720

The Expansion ROM BAR has only the Endpoint part documented, but from
the similarities we think that it can also be used in RC mode in that
way.

So either Expansion ROM BAR has different meaning (if the hypothesis
above is true), or we don't know it's meaning (since it is not
documented for RC mode).

Remove the register from the emulated bridge accessing functions.

Fixes: 239edf686c14 ("PCI: aardvark: Fix support for PCI_ROM_ADDRESS1 on emulated bridge")
Signed-off-by: Marek Behún <kabel@kernel.org>
---
 drivers/pci/controller/pci-aardvark.c | 9 ---------
 1 file changed, 9 deletions(-)

diff --git a/drivers/pci/controller/pci-aardvark.c b/drivers/pci/controller/pci-aardvark.c
index baa62cdcaab4..e3001b3b3293 100644
--- a/drivers/pci/controller/pci-aardvark.c
+++ b/drivers/pci/controller/pci-aardvark.c
@@ -32,7 +32,6 @@
 #define PCIE_CORE_DEV_ID_REG					0x0
 #define PCIE_CORE_CMD_STATUS_REG				0x4
 #define PCIE_CORE_DEV_REV_REG					0x8
-#define PCIE_CORE_EXP_ROM_BAR_REG				0x30
 #define PCIE_CORE_PCIEXP_CAP					0xc0
 #define PCIE_CORE_ERR_CAPCTL_REG				0x118
 #define     PCIE_CORE_ERR_CAPCTL_ECRC_CHK_TX			BIT(5)
@@ -774,10 +773,6 @@ advk_pci_bridge_emul_base_conf_read(struct pci_bridge_emul *bridge,
 		*value = advk_readl(pcie, PCIE_CORE_CMD_STATUS_REG);
 		return PCI_BRIDGE_EMUL_HANDLED;
 
-	case PCI_ROM_ADDRESS1:
-		*value = advk_readl(pcie, PCIE_CORE_EXP_ROM_BAR_REG);
-		return PCI_BRIDGE_EMUL_HANDLED;
-
 	case PCI_INTERRUPT_LINE: {
 		/*
 		 * From the whole 32bit register we support reading from HW only
@@ -810,10 +805,6 @@ advk_pci_bridge_emul_base_conf_write(struct pci_bridge_emul *bridge,
 		advk_writel(pcie, new, PCIE_CORE_CMD_STATUS_REG);
 		break;
 
-	case PCI_ROM_ADDRESS1:
-		advk_writel(pcie, new, PCIE_CORE_EXP_ROM_BAR_REG);
-		break;
-
 	case PCI_INTERRUPT_LINE:
 		if (mask & (PCI_BRIDGE_CTL_BUS_RESET << 16)) {
 			u32 val = advk_readl(pcie, PCIE_CORE_CTRL1_REG);
-- 
2.32.0

Re: [PATCH pci-fixes 2/2] Revert "PCI: aardvark: Fix support for PCI_ROM_ADDRESS1 on emulated bridge"

[Lorenzo Pieralisi]

On Thu, Nov 25, 2021 at 05:01:48PM +0100, Marek Behún wrote:
> This reverts commit 239edf686c14a9ff926dec2f350289ed7adfefe2.
> 
> PCI Bridge which represents aardvark's PCIe Root Port has Expansion ROM
> Base Address register at offset 0x30, but its meaning is different than
> PCI's Expansion ROM BAR register, although the layout is the same.
> (This is why we thought it does the same thing.)
> 
> First: there is no ROM (or part of BootROM) in the A3720 SOC dedicated
> for PCIe Root Port (or controller in RC mode) containing executable code
> that would initialize the Root Port, suitable for execution in
> bootloader (this is how Expansion ROM BAR is used on x86).
> 
> Second: in A3720 spec the register (address D0070030) is not documented
> at all for Root Complex mode, but similar to other BAR registers, it has
> an "entangled partner" in register D0075920, which does address
> translation for the BAR in D0070030:
> - the BAR register sets the address from the view of PCIe bus
> - the translation register sets the address from the view of the CPU
> 
> The other BAR registers also have this entangled partner, and they
> can be used to:
> - in RC mode: address-checking on the receive side of the RC (they
>   can define address ranges for memory accesses from remote Endpoints
>   to the RC)
> - in Endpoint mode: allow the remote CPU to access memory on A3720
> 
> The Expansion ROM BAR has only the Endpoint part documented, but from
> the similarities we think that it can also be used in RC mode in that
> way.
> 
> So either Expansion ROM BAR has different meaning (if the hypothesis
> above is true), or we don't know it's meaning (since it is not
> documented for RC mode).
> 
> Remove the register from the emulated bridge accessing functions.
> 
> Fixes: 239edf686c14 ("PCI: aardvark: Fix support for PCI_ROM_ADDRESS1 on emulated bridge")
> Signed-off-by: Marek Behún <kabel@kernel.org>
> ---
>  drivers/pci/controller/pci-aardvark.c | 9 ---------
>  1 file changed, 9 deletions(-)

Bjorn,

this reverts a commit we merged the last merge window so it is
a candidate for one of the upcoming -rcX.

Thanks,
Lorenzo

> diff --git a/drivers/pci/controller/pci-aardvark.c b/drivers/pci/controller/pci-aardvark.c
> index baa62cdcaab4..e3001b3b3293 100644
> --- a/drivers/pci/controller/pci-aardvark.c
> +++ b/drivers/pci/controller/pci-aardvark.c
> @@ -32,7 +32,6 @@
>  #define PCIE_CORE_DEV_ID_REG					0x0
>  #define PCIE_CORE_CMD_STATUS_REG				0x4
>  #define PCIE_CORE_DEV_REV_REG					0x8
> -#define PCIE_CORE_EXP_ROM_BAR_REG				0x30
>  #define PCIE_CORE_PCIEXP_CAP					0xc0
>  #define PCIE_CORE_ERR_CAPCTL_REG				0x118
>  #define     PCIE_CORE_ERR_CAPCTL_ECRC_CHK_TX			BIT(5)
> @@ -774,10 +773,6 @@ advk_pci_bridge_emul_base_conf_read(struct pci_bridge_emul *bridge,
>  		*value = advk_readl(pcie, PCIE_CORE_CMD_STATUS_REG);
>  		return PCI_BRIDGE_EMUL_HANDLED;
>  
> -	case PCI_ROM_ADDRESS1:
> -		*value = advk_readl(pcie, PCIE_CORE_EXP_ROM_BAR_REG);
> -		return PCI_BRIDGE_EMUL_HANDLED;
> -
>  	case PCI_INTERRUPT_LINE: {
>  		/*
>  		 * From the whole 32bit register we support reading from HW only
> @@ -810,10 +805,6 @@ advk_pci_bridge_emul_base_conf_write(struct pci_bridge_emul *bridge,
>  		advk_writel(pcie, new, PCIE_CORE_CMD_STATUS_REG);
>  		break;
>  
> -	case PCI_ROM_ADDRESS1:
> -		advk_writel(pcie, new, PCIE_CORE_EXP_ROM_BAR_REG);
> -		break;
> -
>  	case PCI_INTERRUPT_LINE:
>  		if (mask & (PCI_BRIDGE_CTL_BUS_RESET << 16)) {
>  			u32 val = advk_readl(pcie, PCIE_CORE_CTRL1_REG);
> -- 
> 2.32.0
> 

Re: [PATCH pci-fixes 2/2] Revert "PCI: aardvark: Fix support for PCI_ROM_ADDRESS1 on emulated bridge"

[Pali Rohár]

On Thursday 25 November 2021 17:01:48 Marek Behún wrote:
> This reverts commit 239edf686c14a9ff926dec2f350289ed7adfefe2.
> 
> PCI Bridge which represents aardvark's PCIe Root Port has Expansion ROM
> Base Address register at offset 0x30, but its meaning is different than
> PCI's Expansion ROM BAR register, although the layout is the same.
> (This is why we thought it does the same thing.)
> 
> First: there is no ROM (or part of BootROM) in the A3720 SOC dedicated
> for PCIe Root Port (or controller in RC mode) containing executable code
> that would initialize the Root Port, suitable for execution in
> bootloader (this is how Expansion ROM BAR is used on x86).
> 
> Second: in A3720 spec the register (address D0070030) is not documented
> at all for Root Complex mode, but similar to other BAR registers, it has
> an "entangled partner" in register D0075920, which does address
> translation for the BAR in D0070030:
> - the BAR register sets the address from the view of PCIe bus
> - the translation register sets the address from the view of the CPU
> 
> The other BAR registers also have this entangled partner, and they
> can be used to:
> - in RC mode: address-checking on the receive side of the RC (they
>   can define address ranges for memory accesses from remote Endpoints
>   to the RC)
> - in Endpoint mode: allow the remote CPU to access memory on A3720
> 
> The Expansion ROM BAR has only the Endpoint part documented, but from
> the similarities we think that it can also be used in RC mode in that
> way.
> 
> So either Expansion ROM BAR has different meaning (if the hypothesis
> above is true), or we don't know it's meaning (since it is not
> documented for RC mode).
> 
> Remove the register from the emulated bridge accessing functions.
> 
> Fixes: 239edf686c14 ("PCI: aardvark: Fix support for PCI_ROM_ADDRESS1 on emulated bridge")
> Signed-off-by: Marek Behún <kabel@kernel.org>

Seems that there is missing my tag, so I'm adding:

Reviewed-by: Pali Rohár <pali@kernel.org>

It is really a trap if device has register names and offsets same as in
PCIe spec, also syntax/content of registers is same, but then semantic,
meaning and usage is totally different.

> ---
>  drivers/pci/controller/pci-aardvark.c | 9 ---------
>  1 file changed, 9 deletions(-)
> 
> diff --git a/drivers/pci/controller/pci-aardvark.c b/drivers/pci/controller/pci-aardvark.c
> index baa62cdcaab4..e3001b3b3293 100644
> --- a/drivers/pci/controller/pci-aardvark.c
> +++ b/drivers/pci/controller/pci-aardvark.c
> @@ -32,7 +32,6 @@
>  #define PCIE_CORE_DEV_ID_REG					0x0
>  #define PCIE_CORE_CMD_STATUS_REG				0x4
>  #define PCIE_CORE_DEV_REV_REG					0x8
> -#define PCIE_CORE_EXP_ROM_BAR_REG				0x30
>  #define PCIE_CORE_PCIEXP_CAP					0xc0
>  #define PCIE_CORE_ERR_CAPCTL_REG				0x118
>  #define     PCIE_CORE_ERR_CAPCTL_ECRC_CHK_TX			BIT(5)
> @@ -774,10 +773,6 @@ advk_pci_bridge_emul_base_conf_read(struct pci_bridge_emul *bridge,
>  		*value = advk_readl(pcie, PCIE_CORE_CMD_STATUS_REG);
>  		return PCI_BRIDGE_EMUL_HANDLED;
>  
> -	case PCI_ROM_ADDRESS1:
> -		*value = advk_readl(pcie, PCIE_CORE_EXP_ROM_BAR_REG);
> -		return PCI_BRIDGE_EMUL_HANDLED;
> -
>  	case PCI_INTERRUPT_LINE: {
>  		/*
>  		 * From the whole 32bit register we support reading from HW only
> @@ -810,10 +805,6 @@ advk_pci_bridge_emul_base_conf_write(struct pci_bridge_emul *bridge,
>  		advk_writel(pcie, new, PCIE_CORE_CMD_STATUS_REG);
>  		break;
>  
> -	case PCI_ROM_ADDRESS1:
> -		advk_writel(pcie, new, PCIE_CORE_EXP_ROM_BAR_REG);
> -		break;
> -
>  	case PCI_INTERRUPT_LINE:
>  		if (mask & (PCI_BRIDGE_CTL_BUS_RESET << 16)) {
>  			u32 val = advk_readl(pcie, PCIE_CORE_CTRL1_REG);
> -- 
> 2.32.0
> 

Re: [PATCH pci-fixes 2/2] Revert "PCI: aardvark: Fix support for PCI_ROM_ADDRESS1 on emulated bridge"

[Bjorn Helgaas]

On Tue, Nov 30, 2021 at 11:29:37AM +0000, Lorenzo Pieralisi wrote:
> On Thu, Nov 25, 2021 at 05:01:48PM +0100, Marek Behún wrote:
> > This reverts commit 239edf686c14a9ff926dec2f350289ed7adfefe2.
> > 
> > PCI Bridge which represents aardvark's PCIe Root Port has Expansion ROM
> > Base Address register at offset 0x30, but its meaning is different than
> > PCI's Expansion ROM BAR register, although the layout is the same.
> > (This is why we thought it does the same thing.)
> > 
> > First: there is no ROM (or part of BootROM) in the A3720 SOC dedicated
> > for PCIe Root Port (or controller in RC mode) containing executable code
> > that would initialize the Root Port, suitable for execution in
> > bootloader (this is how Expansion ROM BAR is used on x86).
> > 
> > Second: in A3720 spec the register (address D0070030) is not documented
> > at all for Root Complex mode, but similar to other BAR registers, it has
> > an "entangled partner" in register D0075920, which does address
> > translation for the BAR in D0070030:
> > - the BAR register sets the address from the view of PCIe bus
> > - the translation register sets the address from the view of the CPU
> > 
> > The other BAR registers also have this entangled partner, and they
> > can be used to:
> > - in RC mode: address-checking on the receive side of the RC (they
> >   can define address ranges for memory accesses from remote Endpoints
> >   to the RC)
> > - in Endpoint mode: allow the remote CPU to access memory on A3720
> > 
> > The Expansion ROM BAR has only the Endpoint part documented, but from
> > the similarities we think that it can also be used in RC mode in that
> > way.
> > 
> > So either Expansion ROM BAR has different meaning (if the hypothesis
> > above is true), or we don't know it's meaning (since it is not
> > documented for RC mode).
> > 
> > Remove the register from the emulated bridge accessing functions.
> > 
> > Fixes: 239edf686c14 ("PCI: aardvark: Fix support for PCI_ROM_ADDRESS1 on emulated bridge")
> > Signed-off-by: Marek Behún <kabel@kernel.org>
> > ---
> >  drivers/pci/controller/pci-aardvark.c | 9 ---------
> >  1 file changed, 9 deletions(-)
> 
> Bjorn,
> 
> this reverts a commit we merged the last merge window so it is
> a candidate for one of the upcoming -rcX.

Sure, happy to apply the revert.

What problem does the revert fix?  I assume 239edf686c14 ("PCI:
aardvark: Fix support for PCI_ROM_ADDRESS1 on emulated bridge") broke
something, but the commit log for the revert doesn't say *what*.  How
would one notice that something broke?

> > diff --git a/drivers/pci/controller/pci-aardvark.c b/drivers/pci/controller/pci-aardvark.c
> > index baa62cdcaab4..e3001b3b3293 100644
> > --- a/drivers/pci/controller/pci-aardvark.c
> > +++ b/drivers/pci/controller/pci-aardvark.c
> > @@ -32,7 +32,6 @@
> >  #define PCIE_CORE_DEV_ID_REG					0x0
> >  #define PCIE_CORE_CMD_STATUS_REG				0x4
> >  #define PCIE_CORE_DEV_REV_REG					0x8
> > -#define PCIE_CORE_EXP_ROM_BAR_REG				0x30
> >  #define PCIE_CORE_PCIEXP_CAP					0xc0
> >  #define PCIE_CORE_ERR_CAPCTL_REG				0x118
> >  #define     PCIE_CORE_ERR_CAPCTL_ECRC_CHK_TX			BIT(5)
> > @@ -774,10 +773,6 @@ advk_pci_bridge_emul_base_conf_read(struct pci_bridge_emul *bridge,
> >  		*value = advk_readl(pcie, PCIE_CORE_CMD_STATUS_REG);
> >  		return PCI_BRIDGE_EMUL_HANDLED;
> >  
> > -	case PCI_ROM_ADDRESS1:
> > -		*value = advk_readl(pcie, PCIE_CORE_EXP_ROM_BAR_REG);
> > -		return PCI_BRIDGE_EMUL_HANDLED;
> > -
> >  	case PCI_INTERRUPT_LINE: {
> >  		/*
> >  		 * From the whole 32bit register we support reading from HW only
> > @@ -810,10 +805,6 @@ advk_pci_bridge_emul_base_conf_write(struct pci_bridge_emul *bridge,
> >  		advk_writel(pcie, new, PCIE_CORE_CMD_STATUS_REG);
> >  		break;
> >  
> > -	case PCI_ROM_ADDRESS1:
> > -		advk_writel(pcie, new, PCIE_CORE_EXP_ROM_BAR_REG);
> > -		break;
> > -
> >  	case PCI_INTERRUPT_LINE:
> >  		if (mask & (PCI_BRIDGE_CTL_BUS_RESET << 16)) {
> >  			u32 val = advk_readl(pcie, PCIE_CORE_CTRL1_REG);
> > -- 
> > 2.32.0
> > 

Re: [PATCH pci-fixes 2/2] Revert "PCI: aardvark: Fix support for PCI_ROM_ADDRESS1 on emulated bridge"

[Marek Behún]

On Tue, 30 Nov 2021 19:53:08 -0600
Bjorn Helgaas <helgaas@kernel.org> wrote:

> On Tue, Nov 30, 2021 at 11:29:37AM +0000, Lorenzo Pieralisi wrote:
> > On Thu, Nov 25, 2021 at 05:01:48PM +0100, Marek Behún wrote:  
> > > This reverts commit 239edf686c14a9ff926dec2f350289ed7adfefe2.
> > > 
> > > PCI Bridge which represents aardvark's PCIe Root Port has Expansion ROM
> > > Base Address register at offset 0x30, but its meaning is different than
> > > PCI's Expansion ROM BAR register, although the layout is the same.
> > > (This is why we thought it does the same thing.)
> > > 
> > > First: there is no ROM (or part of BootROM) in the A3720 SOC dedicated
> > > for PCIe Root Port (or controller in RC mode) containing executable code
> > > that would initialize the Root Port, suitable for execution in
> > > bootloader (this is how Expansion ROM BAR is used on x86).
> > > 
> > > Second: in A3720 spec the register (address D0070030) is not documented
> > > at all for Root Complex mode, but similar to other BAR registers, it has
> > > an "entangled partner" in register D0075920, which does address
> > > translation for the BAR in D0070030:
> > > - the BAR register sets the address from the view of PCIe bus
> > > - the translation register sets the address from the view of the CPU
> > > 
> > > The other BAR registers also have this entangled partner, and they
> > > can be used to:
> > > - in RC mode: address-checking on the receive side of the RC (they
> > >   can define address ranges for memory accesses from remote Endpoints
> > >   to the RC)
> > > - in Endpoint mode: allow the remote CPU to access memory on A3720
> > > 
> > > The Expansion ROM BAR has only the Endpoint part documented, but from
> > > the similarities we think that it can also be used in RC mode in that
> > > way.
> > > 
> > > So either Expansion ROM BAR has different meaning (if the hypothesis
> > > above is true), or we don't know it's meaning (since it is not
> > > documented for RC mode).
> > > 
> > > Remove the register from the emulated bridge accessing functions.
> > > 
> > > Fixes: 239edf686c14 ("PCI: aardvark: Fix support for PCI_ROM_ADDRESS1 on emulated bridge")
> > > Signed-off-by: Marek Behún <kabel@kernel.org>
> > > ---
> > >  drivers/pci/controller/pci-aardvark.c | 9 ---------
> > >  1 file changed, 9 deletions(-)  
> > 
> > Bjorn,
> > 
> > this reverts a commit we merged the last merge window so it is
> > a candidate for one of the upcoming -rcX.  
> 
> Sure, happy to apply the revert.
> 
> What problem does the revert fix?  I assume 239edf686c14 ("PCI:
> aardvark: Fix support for PCI_ROM_ADDRESS1 on emulated bridge") broke
> something, but the commit log for the revert doesn't say *what*.  How
> would one notice that something broke?

Hello Bjorn,

It doesn't break any real functionality that I know of, although it
might, since the register is read pci/probe.c pci_setup_device()
(pci_read_bases()).

But allowing the access to the register when it has different meaning
is wrong in a similar sense that a memory leak is wrong (a memory leak
also does not necessarily cause real problems, if it is small, but
still we should fix it).

Marek

Re: [PATCH pci-fixes 2/2] Revert "PCI: aardvark: Fix support for PCI_ROM_ADDRESS1 on emulated bridge"

[Bjorn Helgaas]

On Wed, Dec 01, 2021 at 10:50:45AM +0100, Marek Behún wrote:
> On Tue, 30 Nov 2021 19:53:08 -0600
> Bjorn Helgaas <helgaas@kernel.org> wrote:
> 
> > On Tue, Nov 30, 2021 at 11:29:37AM +0000, Lorenzo Pieralisi wrote:
> > > On Thu, Nov 25, 2021 at 05:01:48PM +0100, Marek Behún wrote:  
> > > > This reverts commit 239edf686c14a9ff926dec2f350289ed7adfefe2.
> > > > 
> > > > PCI Bridge which represents aardvark's PCIe Root Port has Expansion ROM
> > > > Base Address register at offset 0x30, but its meaning is different than
> > > > PCI's Expansion ROM BAR register, although the layout is the same.
> > > > (This is why we thought it does the same thing.)
> > > > 
> > > > First: there is no ROM (or part of BootROM) in the A3720 SOC dedicated
> > > > for PCIe Root Port (or controller in RC mode) containing executable code
> > > > that would initialize the Root Port, suitable for execution in
> > > > bootloader (this is how Expansion ROM BAR is used on x86).
> > > > 
> > > > Second: in A3720 spec the register (address D0070030) is not documented
> > > > at all for Root Complex mode, but similar to other BAR registers, it has
> > > > an "entangled partner" in register D0075920, which does address
> > > > translation for the BAR in D0070030:
> > > > - the BAR register sets the address from the view of PCIe bus
> > > > - the translation register sets the address from the view of the CPU
> > > > 
> > > > The other BAR registers also have this entangled partner, and they
> > > > can be used to:
> > > > - in RC mode: address-checking on the receive side of the RC (they
> > > >   can define address ranges for memory accesses from remote Endpoints
> > > >   to the RC)
> > > > - in Endpoint mode: allow the remote CPU to access memory on A3720
> > > > 
> > > > The Expansion ROM BAR has only the Endpoint part documented, but from
> > > > the similarities we think that it can also be used in RC mode in that
> > > > way.
> > > > 
> > > > So either Expansion ROM BAR has different meaning (if the hypothesis
> > > > above is true), or we don't know it's meaning (since it is not
> > > > documented for RC mode).
> > > > 
> > > > Remove the register from the emulated bridge accessing functions.
> > > > 
> > > > Fixes: 239edf686c14 ("PCI: aardvark: Fix support for PCI_ROM_ADDRESS1 on emulated bridge")
> > > > Signed-off-by: Marek Behún <kabel@kernel.org>
> > > > ---
> > > >  drivers/pci/controller/pci-aardvark.c | 9 ---------
> > > >  1 file changed, 9 deletions(-)  
> > > 
> > > Bjorn,
> > > 
> > > this reverts a commit we merged the last merge window so it is
> > > a candidate for one of the upcoming -rcX.  
> > 
> > Sure, happy to apply the revert.
> > 
> > What problem does the revert fix?  I assume 239edf686c14 ("PCI:
> > aardvark: Fix support for PCI_ROM_ADDRESS1 on emulated bridge") broke
> > something, but the commit log for the revert doesn't say *what*.  How
> > would one notice that something broke?
> 
> Hello Bjorn,
> 
> It doesn't break any real functionality that I know of, although it
> might, since the register is read pci/probe.c pci_setup_device()
> (pci_read_bases()).
> 
> But allowing the access to the register when it has different meaning
> is wrong in a similar sense that a memory leak is wrong (a memory leak
> also does not necessarily cause real problems, if it is small, but
> still we should fix it).

What is the new information that led you to conclude that 239edf686c14
is wrong?  Apparently you originally thought the bridge had a ROM BAR,
but later decided it didn't, based on *something*?  New observation?
New understanding of the spec?

I need to be able to explain why we should merge this after the merge
window, and I'm having a hard time extracting that from the commit
log.

Bjorn

Re: [PATCH pci-fixes 2/2] Revert "PCI: aardvark: Fix support for PCI_ROM_ADDRESS1 on emulated bridge"

[Marek Behún]

On Wed, 1 Dec 2021 06:35:18 -0600
Bjorn Helgaas <helgaas@kernel.org> wrote:

> On Wed, Dec 01, 2021 at 10:50:45AM +0100, Marek Behún wrote:
> > On Tue, 30 Nov 2021 19:53:08 -0600
> > Bjorn Helgaas <helgaas@kernel.org> wrote:
> >   
> > > On Tue, Nov 30, 2021 at 11:29:37AM +0000, Lorenzo Pieralisi wrote:  
> > > > On Thu, Nov 25, 2021 at 05:01:48PM +0100, Marek Behún wrote:    
> > > > > This reverts commit 239edf686c14a9ff926dec2f350289ed7adfefe2.
> > > > > 
> > > > > PCI Bridge which represents aardvark's PCIe Root Port has Expansion ROM
> > > > > Base Address register at offset 0x30, but its meaning is different than
> > > > > PCI's Expansion ROM BAR register, although the layout is the same.
> > > > > (This is why we thought it does the same thing.)
> > > > > 
> > > > > First: there is no ROM (or part of BootROM) in the A3720 SOC dedicated
> > > > > for PCIe Root Port (or controller in RC mode) containing executable code
> > > > > that would initialize the Root Port, suitable for execution in
> > > > > bootloader (this is how Expansion ROM BAR is used on x86).
> > > > > 
> > > > > Second: in A3720 spec the register (address D0070030) is not documented
> > > > > at all for Root Complex mode, but similar to other BAR registers, it has
> > > > > an "entangled partner" in register D0075920, which does address
> > > > > translation for the BAR in D0070030:
> > > > > - the BAR register sets the address from the view of PCIe bus
> > > > > - the translation register sets the address from the view of the CPU
> > > > > 
> > > > > The other BAR registers also have this entangled partner, and they
> > > > > can be used to:
> > > > > - in RC mode: address-checking on the receive side of the RC (they
> > > > >   can define address ranges for memory accesses from remote Endpoints
> > > > >   to the RC)
> > > > > - in Endpoint mode: allow the remote CPU to access memory on A3720
> > > > > 
> > > > > The Expansion ROM BAR has only the Endpoint part documented, but from
> > > > > the similarities we think that it can also be used in RC mode in that
> > > > > way.
> > > > > 
> > > > > So either Expansion ROM BAR has different meaning (if the hypothesis
> > > > > above is true), or we don't know it's meaning (since it is not
> > > > > documented for RC mode).
> > > > > 
> > > > > Remove the register from the emulated bridge accessing functions.
> > > > > 
> > > > > Fixes: 239edf686c14 ("PCI: aardvark: Fix support for PCI_ROM_ADDRESS1 on emulated bridge")
> > > > > Signed-off-by: Marek Behún <kabel@kernel.org>
> > > > > ---
> > > > >  drivers/pci/controller/pci-aardvark.c | 9 ---------
> > > > >  1 file changed, 9 deletions(-)    
> > > > 
> > > > Bjorn,
> > > > 
> > > > this reverts a commit we merged the last merge window so it is
> > > > a candidate for one of the upcoming -rcX.    
> > > 
> > > Sure, happy to apply the revert.
> > > 
> > > What problem does the revert fix?  I assume 239edf686c14 ("PCI:
> > > aardvark: Fix support for PCI_ROM_ADDRESS1 on emulated bridge") broke
> > > something, but the commit log for the revert doesn't say *what*.  How
> > > would one notice that something broke?  
> > 
> > Hello Bjorn,
> > 
> > It doesn't break any real functionality that I know of, although it
> > might, since the register is read pci/probe.c pci_setup_device()
> > (pci_read_bases()).
> > 
> > But allowing the access to the register when it has different meaning
> > is wrong in a similar sense that a memory leak is wrong (a memory leak
> > also does not necessarily cause real problems, if it is small, but
> > still we should fix it).  
> 
> What is the new information that led you to conclude that 239edf686c14
> is wrong?  Apparently you originally thought the bridge had a ROM BAR,
> but later decided it didn't, based on *something*?  New observation?
> New understanding of the spec?

Hi Bjorn,

The new observation is that although the register is defined in
register list (together with it's layour), it is not documented in RC
mode, although other BAR registers are (and their meaning is something
different from standard BAR registers in RC mode). Combined with the
fact that there is no ROM containing executable code in the SOC (which
we knew even before, but thought that maybe it could be somehow also
implemented), we concluded that this register has different meaning from
standard Expansion ROM BAR.

Marek

> I need to be able to explain why we should merge this after the merge
> window, and I'm having a hard time extracting that from the commit
> log.
> 
> Bjorn

Re: [PATCH pci-fixes 2/2] Revert "PCI: aardvark: Fix support for PCI_ROM_ADDRESS1 on emulated bridge"

[Bjorn Helgaas]

On Wed, Dec 01, 2021 at 06:23:24PM +0100, Marek Behún wrote:
> On Wed, 1 Dec 2021 06:35:18 -0600
> Bjorn Helgaas <helgaas@kernel.org> wrote:
> 
> > On Wed, Dec 01, 2021 at 10:50:45AM +0100, Marek Behún wrote:
> > > On Tue, 30 Nov 2021 19:53:08 -0600
> > > Bjorn Helgaas <helgaas@kernel.org> wrote:
> > >   
> > > > On Tue, Nov 30, 2021 at 11:29:37AM +0000, Lorenzo Pieralisi wrote:  
> > > > > On Thu, Nov 25, 2021 at 05:01:48PM +0100, Marek Behún wrote:    
> > > > > > This reverts commit 239edf686c14a9ff926dec2f350289ed7adfefe2.
> > > > > > 
> > > > > > PCI Bridge which represents aardvark's PCIe Root Port has Expansion ROM
> > > > > > Base Address register at offset 0x30, but its meaning is different than
> > > > > > PCI's Expansion ROM BAR register, although the layout is the same.
> > > > > > (This is why we thought it does the same thing.)
> > > > > > 
> > > > > > First: there is no ROM (or part of BootROM) in the A3720 SOC dedicated
> > > > > > for PCIe Root Port (or controller in RC mode) containing executable code
> > > > > > that would initialize the Root Port, suitable for execution in
> > > > > > bootloader (this is how Expansion ROM BAR is used on x86).
> > > > > > 
> > > > > > Second: in A3720 spec the register (address D0070030) is not documented
> > > > > > at all for Root Complex mode, but similar to other BAR registers, it has
> > > > > > an "entangled partner" in register D0075920, which does address
> > > > > > translation for the BAR in D0070030:
> > > > > > - the BAR register sets the address from the view of PCIe bus
> > > > > > - the translation register sets the address from the view of the CPU
> > > > > > 
> > > > > > The other BAR registers also have this entangled partner, and they
> > > > > > can be used to:
> > > > > > - in RC mode: address-checking on the receive side of the RC (they
> > > > > >   can define address ranges for memory accesses from remote Endpoints
> > > > > >   to the RC)
> > > > > > - in Endpoint mode: allow the remote CPU to access memory on A3720
> > > > > > 
> > > > > > The Expansion ROM BAR has only the Endpoint part documented, but from
> > > > > > the similarities we think that it can also be used in RC mode in that
> > > > > > way.
> > > > > > 
> > > > > > So either Expansion ROM BAR has different meaning (if the hypothesis
> > > > > > above is true), or we don't know it's meaning (since it is not
> > > > > > documented for RC mode).
> > > > > > 
> > > > > > Remove the register from the emulated bridge accessing functions.
> > > > > > 
> > > > > > Fixes: 239edf686c14 ("PCI: aardvark: Fix support for PCI_ROM_ADDRESS1 on emulated bridge")
> > > > > > Signed-off-by: Marek Behún <kabel@kernel.org>
> > > > > > ---
> > > > > >  drivers/pci/controller/pci-aardvark.c | 9 ---------
> > > > > >  1 file changed, 9 deletions(-)    
> > > > > 
> > > > > Bjorn,
> > > > > 
> > > > > this reverts a commit we merged the last merge window so it is
> > > > > a candidate for one of the upcoming -rcX.    
> > > > 
> > > > Sure, happy to apply the revert.
> > > > 
> > > > What problem does the revert fix?  I assume 239edf686c14 ("PCI:
> > > > aardvark: Fix support for PCI_ROM_ADDRESS1 on emulated bridge") broke
> > > > something, but the commit log for the revert doesn't say *what*.  How
> > > > would one notice that something broke?  
> > > 
> > > Hello Bjorn,
> > > 
> > > It doesn't break any real functionality that I know of, although it
> > > might, since the register is read pci/probe.c pci_setup_device()
> > > (pci_read_bases()).
> > > 
> > > But allowing the access to the register when it has different meaning
> > > is wrong in a similar sense that a memory leak is wrong (a memory leak
> > > also does not necessarily cause real problems, if it is small, but
> > > still we should fix it).  
> > 
> > What is the new information that led you to conclude that 239edf686c14
> > is wrong?  Apparently you originally thought the bridge had a ROM BAR,
> > but later decided it didn't, based on *something*?  New observation?
> > New understanding of the spec?
> 
> Hi Bjorn,
> 
> The new observation is that although the register is defined in
> register list (together with it's layour), it is not documented in RC
> mode, although other BAR registers are (and their meaning is something
> different from standard BAR registers in RC mode). Combined with the
> fact that there is no ROM containing executable code in the SOC (which
> we knew even before, but thought that maybe it could be somehow also
> implemented), we concluded that this register has different meaning from
> standard Expansion ROM BAR.

Thanks, applied to for-linus for v5.16!

Bjorn

Re: [PATCH pci-fixes 2/2] Revert "PCI: aardvark: Fix support for PCI_ROM_ADDRESS1 on emulated bridge"

[Bjorn Helgaas]

On Thu, Nov 25, 2021 at 05:01:48PM +0100, Marek Behún wrote:
> This reverts commit 239edf686c14a9ff926dec2f350289ed7adfefe2.
> 
> PCI Bridge which represents aardvark's PCIe Root Port has Expansion ROM
> Base Address register at offset 0x30, but its meaning is different than
> PCI's Expansion ROM BAR register, although the layout is the same.
> (This is why we thought it does the same thing.)
> 
> First: there is no ROM (or part of BootROM) in the A3720 SOC dedicated
> for PCIe Root Port (or controller in RC mode) containing executable code
> that would initialize the Root Port, suitable for execution in
> bootloader (this is how Expansion ROM BAR is used on x86).
> 
> Second: in A3720 spec the register (address D0070030) is not documented
> at all for Root Complex mode, but similar to other BAR registers, it has
> an "entangled partner" in register D0075920, which does address
> translation for the BAR in D0070030:
> - the BAR register sets the address from the view of PCIe bus
> - the translation register sets the address from the view of the CPU
> 
> The other BAR registers also have this entangled partner, and they
> can be used to:
> - in RC mode: address-checking on the receive side of the RC (they
>   can define address ranges for memory accesses from remote Endpoints
>   to the RC)
> - in Endpoint mode: allow the remote CPU to access memory on A3720
> 
> The Expansion ROM BAR has only the Endpoint part documented, but from
> the similarities we think that it can also be used in RC mode in that
> way.
> 
> So either Expansion ROM BAR has different meaning (if the hypothesis
> above is true), or we don't know it's meaning (since it is not
> documented for RC mode).
> 
> Remove the register from the emulated bridge accessing functions.
> 
> Fixes: 239edf686c14 ("PCI: aardvark: Fix support for PCI_ROM_ADDRESS1 on emulated bridge")
> Signed-off-by: Marek Behún <kabel@kernel.org>

Applied this patch (2/2) to for-linus for v5.16, thanks!

> ---
>  drivers/pci/controller/pci-aardvark.c | 9 ---------
>  1 file changed, 9 deletions(-)
> 
> diff --git a/drivers/pci/controller/pci-aardvark.c b/drivers/pci/controller/pci-aardvark.c
> index baa62cdcaab4..e3001b3b3293 100644
> --- a/drivers/pci/controller/pci-aardvark.c
> +++ b/drivers/pci/controller/pci-aardvark.c
> @@ -32,7 +32,6 @@
>  #define PCIE_CORE_DEV_ID_REG					0x0
>  #define PCIE_CORE_CMD_STATUS_REG				0x4
>  #define PCIE_CORE_DEV_REV_REG					0x8
> -#define PCIE_CORE_EXP_ROM_BAR_REG				0x30
>  #define PCIE_CORE_PCIEXP_CAP					0xc0
>  #define PCIE_CORE_ERR_CAPCTL_REG				0x118
>  #define     PCIE_CORE_ERR_CAPCTL_ECRC_CHK_TX			BIT(5)
> @@ -774,10 +773,6 @@ advk_pci_bridge_emul_base_conf_read(struct pci_bridge_emul *bridge,
>  		*value = advk_readl(pcie, PCIE_CORE_CMD_STATUS_REG);
>  		return PCI_BRIDGE_EMUL_HANDLED;
>  
> -	case PCI_ROM_ADDRESS1:
> -		*value = advk_readl(pcie, PCIE_CORE_EXP_ROM_BAR_REG);
> -		return PCI_BRIDGE_EMUL_HANDLED;
> -
>  	case PCI_INTERRUPT_LINE: {
>  		/*
>  		 * From the whole 32bit register we support reading from HW only
> @@ -810,10 +805,6 @@ advk_pci_bridge_emul_base_conf_write(struct pci_bridge_emul *bridge,
>  		advk_writel(pcie, new, PCIE_CORE_CMD_STATUS_REG);
>  		break;
>  
> -	case PCI_ROM_ADDRESS1:
> -		advk_writel(pcie, new, PCIE_CORE_EXP_ROM_BAR_REG);
> -		break;
> -
>  	case PCI_INTERRUPT_LINE:
>  		if (mask & (PCI_BRIDGE_CTL_BUS_RESET << 16)) {
>  			u32 val = advk_readl(pcie, PCIE_CORE_CTRL1_REG);
> -- 
> 2.32.0
> 

Re: (subset) [PATCH pci-fixes 0/2] PCI Aardvark controller fixes

[Lorenzo Pieralisi]

On Thu, 25 Nov 2021 17:01:46 +0100, Marek Behún wrote:
> here are two fixes for pci-aardvark controller.
> 
> Marek
> 
> Marek Behún (1):
>   Revert "PCI: aardvark: Fix support for PCI_ROM_ADDRESS1 on emulated
>     bridge"
> 
> [...]

Applied to pci/aardvark, thanks!

[1/2] PCI: aardvark: Fix checking for MEM resource type
      https://git.kernel.org/lpieralisi/pci/c/2070b2ddea

Thanks,
Lorenzo