[PATCH v3 0/2] PCI: mediatek: enable whole MMIO range and enlarge the PCIe2AHB window size

[PATCH v3 0/2] PCI: mediatek: enable whole MMIO range and enlarge the PCIe2AHB window size

[honghui.zhang]

From: Honghui Zhang <honghui.zhang@mediatek.com>

Two patches:
patch 1 enable whole MMIO range which also fix the complain of scripts/coccinelle/api/resource_size.cocci
patch 2 enlarge the PCIe2AHB window size to support fully access of 4GB DRAM from EP DMA.

v3:
 - update the changlog title for patch1 and update commit message following Bjorn's suggestion
 - move the "|" into the previous line.

v2:
 - Fix the checkpatch complains for patch 1.
 - update the commit message and change title of patch 1 for changelog conventions.
 - Add patch 2.

Honghui Zhang (2):
  PCI: mediatek: Enable the whole memory mapped IO range
  PCI: mediatek: Enlarge PCIe2AHB window size to support 4GB DRAM

 drivers/pci/controller/pcie-mediatek.c | 13 +++++++++----
 1 file changed, 9 insertions(+), 4 deletions(-)

-- 
2.6.4


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH v3 1/2] PCI: mediatek: Enable the whole memory mapped IO range

[honghui.zhang]

From: Honghui Zhang <honghui.zhang@mediatek.com>

Mediatek's HW assigned a bus address range(typically start from
0x2000_0000 to 0x2fff_ffff for both mt2712 and mt7622) for PCIe usage.
This bus address range is called memory mapped IO range, when CPU or
other HW access those address, PCIe RC HW should response to this
access. Normally the RC will translate those access request to TLPs
and send to corresponding EP side. It's like the total memory address
resource which could be allocated by EP and RC's BARs.

Although those address range is available for allocated, but it should
be enabled by the PCIE_AHB_TRANS_BASE register, what size will be
enabled is determined by AHB2PCIE_SIZE bits in this register.

In previous code we did not enable the full size of HW assigned address
range, if the EP's BAR requested size is bigger than the size we enabled
and smaller than the HW available size. The access request which target
at these un-enabled address will be blocked by RC, and EP side will
never get those TLPs.

Previous code never run into a system error in production because even
half of those range(128MB) is bigger enough for typical EP device's BAR
request(4MB).

But all those HW assigned bus range should be enabled. And it's Okay to
do that. RC will never forward a request to EP when this request is not
suitable for EP's BAR range.

Using resource_size(mem) instead of mem->end - mem->start to fix this,
since the MMIO window size for both MT2712 and MT7622 are all
0x1000_0000, this change will change the values of fls(size) from
fls(0xfff_ffff) to fls(0x1000_0000) and calcalate the whole memory
mapped IO range size.

This change also eliminate the following complain generated by
scripts/coccinelle/api/resource_size.cocci:

pcie-mediatek.c:720:13-16: WARNING: Suspicious code. resource_size is maybe missing with mem

Signed-off-by: Honghui Zhang <honghui.zhang@mediatek.com>
---
 drivers/pci/controller/pcie-mediatek.c | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)

diff --git a/drivers/pci/controller/pcie-mediatek.c b/drivers/pci/controller/pcie-mediatek.c
index 55e471c..c42fe5c 100644
--- a/drivers/pci/controller/pcie-mediatek.c
+++ b/drivers/pci/controller/pcie-mediatek.c
@@ -654,7 +654,6 @@ static int mtk_pcie_startup_port_v2(struct mtk_pcie_port *port)
 	struct resource *mem = &pcie->mem;
 	const struct mtk_pcie_soc *soc = port->pcie->soc;
 	u32 val;
-	size_t size;
 	int err;
 
 	/* MT7622 platforms need to enable LTSSM and ASPM from PCIe subsys */
@@ -706,8 +705,8 @@ static int mtk_pcie_startup_port_v2(struct mtk_pcie_port *port)
 		mtk_pcie_enable_msi(port);
 
 	/* Set AHB to PCIe translation windows */
-	size = mem->end - mem->start;
-	val = lower_32_bits(mem->start) | AHB2PCIE_SIZE(fls(size));
+	val = lower_32_bits(mem->start) |
+	      AHB2PCIE_SIZE(fls(resource_size(mem)));
 	writel(val, port->base + PCIE_AHB_TRANS_BASE0_L);
 
 	val = upper_32_bits(mem->start);
-- 
2.6.4


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH v3 2/2] PCI: mediatek: Enlarge PCIe2AHB window size to support 4GB DRAM

[honghui.zhang]

From: Honghui Zhang <honghui.zhang@mediatek.com>

The PCIE_AXI_WINDOW0 defines the translate window size for the request
from EP side. Request outside of this window will be treated as
unsupported request.

Enlarge this window size from fls(0xffffffff) to 2^33 to support 8GB
translate address range then EP DMA is capable of fully access 4GB
DRAM range(physical DRAM is start from 0x40000000).

Reported-by: Bjorn Helgaas <bhelgaas@google.com>
Signed-off-by: Honghui Zhang <honghui.zhang@mediatek.com>
---
 drivers/pci/controller/pcie-mediatek.c | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/drivers/pci/controller/pcie-mediatek.c b/drivers/pci/controller/pcie-mediatek.c
index c42fe5c..0b6c728 100644
--- a/drivers/pci/controller/pcie-mediatek.c
+++ b/drivers/pci/controller/pcie-mediatek.c
@@ -90,6 +90,12 @@
 #define AHB2PCIE_SIZE(x)	((x) & GENMASK(4, 0))
 #define PCIE_AXI_WINDOW0	0x448
 #define WIN_ENABLE		BIT(7)
+/*
+ * Define PCIe to AHB window size as 2^33 to support max 8GB address space
+ * translate, support least 4GB DRAM size access from EP DMA(physical DRAM
+ * start from 0x40000000).
+ */
+#define PCIE2AHB_SIZE	0x21
 
 /* PCIe V2 configuration transaction header */
 #define PCIE_CFG_HEADER0	0x460
@@ -713,7 +719,7 @@ static int mtk_pcie_startup_port_v2(struct mtk_pcie_port *port)
 	writel(val, port->base + PCIE_AHB_TRANS_BASE0_H);
 
 	/* Set PCIe to AXI translation memory space.*/
-	val = fls(0xffffffff) | WIN_ENABLE;
+	val = PCIE2AHB_SIZE | WIN_ENABLE;
 	writel(val, port->base + PCIE_AXI_WINDOW0);
 
 	return 0;
-- 
2.6.4


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v3 2/2] PCI: mediatek: Enlarge PCIe2AHB window size to support 4GB DRAM

[Lorenzo Pieralisi]

On Fri, Feb 01, 2019 at 01:36:07PM +0800, honghui.zhang@mediatek.com wrote:
> From: Honghui Zhang <honghui.zhang@mediatek.com>
> 
> The PCIE_AXI_WINDOW0 defines the translate window size for the request
> from EP side. Request outside of this window will be treated as
> unsupported request.
> 
> Enlarge this window size from fls(0xffffffff) to 2^33 to support 8GB
> translate address range then EP DMA is capable of fully access 4GB
> DRAM range(physical DRAM is start from 0x40000000).

I have rewritten both patches logs with the aim of merging them even if
it is quite late in the cycle, first you have to explain something to
me.

fls(0xffffffff) = 0x1f, which by your logic -> 2^31

What does it mean given what you say above ? That PCI devices can't
do _any_ DMA in the current setting (given the DRAM start address) ?

Lorenzo

> Reported-by: Bjorn Helgaas <bhelgaas@google.com>
> Signed-off-by: Honghui Zhang <honghui.zhang@mediatek.com>
> ---
>  drivers/pci/controller/pcie-mediatek.c | 8 +++++++-
>  1 file changed, 7 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/pci/controller/pcie-mediatek.c b/drivers/pci/controller/pcie-mediatek.c
> index c42fe5c..0b6c728 100644
> --- a/drivers/pci/controller/pcie-mediatek.c
> +++ b/drivers/pci/controller/pcie-mediatek.c
> @@ -90,6 +90,12 @@
>  #define AHB2PCIE_SIZE(x)	((x) & GENMASK(4, 0))
>  #define PCIE_AXI_WINDOW0	0x448
>  #define WIN_ENABLE		BIT(7)
> +/*
> + * Define PCIe to AHB window size as 2^33 to support max 8GB address space
> + * translate, support least 4GB DRAM size access from EP DMA(physical DRAM
> + * start from 0x40000000).
> + */
> +#define PCIE2AHB_SIZE	0x21
>  
>  /* PCIe V2 configuration transaction header */
>  #define PCIE_CFG_HEADER0	0x460
> @@ -713,7 +719,7 @@ static int mtk_pcie_startup_port_v2(struct mtk_pcie_port *port)
>  	writel(val, port->base + PCIE_AHB_TRANS_BASE0_H);
>  
>  	/* Set PCIe to AXI translation memory space.*/
> -	val = fls(0xffffffff) | WIN_ENABLE;
> +	val = PCIE2AHB_SIZE | WIN_ENABLE;
>  	writel(val, port->base + PCIE_AXI_WINDOW0);
>  
>  	return 0;
> -- 
> 2.6.4
> 

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v3 2/2] PCI: mediatek: Enlarge PCIe2AHB window size to support 4GB DRAM

[Honghui Zhang]

On Thu, 2019-02-28 at 17:42 +0000, Lorenzo Pieralisi wrote:
> On Fri, Feb 01, 2019 at 01:36:07PM +0800, honghui.zhang@mediatek.com wrote:
> > From: Honghui Zhang <honghui.zhang@mediatek.com>
> > 
> > The PCIE_AXI_WINDOW0 defines the translate window size for the request
> > from EP side. Request outside of this window will be treated as
> > unsupported request.
> > 
> > Enlarge this window size from fls(0xffffffff) to 2^33 to support 8GB
> > translate address range then EP DMA is capable of fully access 4GB
> > DRAM range(physical DRAM is start from 0x40000000).
> 
> I have rewritten both patches logs with the aim of merging them even if
> it is quite late in the cycle, first you have to explain something to
> me.
> 

Thanks very much for this.

> fls(0xffffffff) = 0x1f, which by your logic -> 2^31
> 
> What does it mean given what you say above ? That PCI devices can't
> do _any_ DMA in the current setting (given the DRAM start address) ?
> 

I'm afraid so.
From the HW datasheet I got from our HW designer, the description for
this pcie2axi_win_size filed is
" Possible values are 12 to 36 which means 2^12 to 2^36 bytes, leaving
this filed to 0 causes window to be disabled."

Current setting set the window size as 2^31, which means the request
from EP side could only access the address range from 0 to 0x8000_0000.
Considering the DRAM start from 0x4000_0000, that means only the first
1GB(0x4000_0000 ~ 0x8000_0000) could be accessed by EP side DMA.

This has not run into an error for our current usage, I guess because
MT2712 and MT7622 have several type boards, most of are only have 2GB
physical memory, and our test sample is not bigger enough to cover the
case that EP DMA will access fully DRAM. Or most EP device does not have
an built-in DMA engine, they may relay on the host side's MMIO(memory
mapped IO) operations.
 
Take MT2712 as example, in arch/arm64/boot/dts/mediatek/mt2712-evb.dts,
the physical memory size is defined as 0x80000000 and is described by
below node:

memory@400000000 {
	device_type = "memory";
	reg = <0 0x400000000 0 0x800000000>
}

Thanks.

> Lorenzo
> 
> > Reported-by: Bjorn Helgaas <bhelgaas@google.com>
> > Signed-off-by: Honghui Zhang <honghui.zhang@mediatek.com>
> > ---
> >  drivers/pci/controller/pcie-mediatek.c | 8 +++++++-
> >  1 file changed, 7 insertions(+), 1 deletion(-)
> > 
> > diff --git a/drivers/pci/controller/pcie-mediatek.c b/drivers/pci/controller/pcie-mediatek.c
> > index c42fe5c..0b6c728 100644
> > --- a/drivers/pci/controller/pcie-mediatek.c
> > +++ b/drivers/pci/controller/pcie-mediatek.c
> > @@ -90,6 +90,12 @@
> >  #define AHB2PCIE_SIZE(x)	((x) & GENMASK(4, 0))
> >  #define PCIE_AXI_WINDOW0	0x448
> >  #define WIN_ENABLE		BIT(7)
> > +/*
> > + * Define PCIe to AHB window size as 2^33 to support max 8GB address space
> > + * translate, support least 4GB DRAM size access from EP DMA(physical DRAM
> > + * start from 0x40000000).
> > + */
> > +#define PCIE2AHB_SIZE	0x21
> >  
> >  /* PCIe V2 configuration transaction header */
> >  #define PCIE_CFG_HEADER0	0x460
> > @@ -713,7 +719,7 @@ static int mtk_pcie_startup_port_v2(struct mtk_pcie_port *port)
> >  	writel(val, port->base + PCIE_AHB_TRANS_BASE0_H);
> >  
> >  	/* Set PCIe to AXI translation memory space.*/
> > -	val = fls(0xffffffff) | WIN_ENABLE;
> > +	val = PCIE2AHB_SIZE | WIN_ENABLE;
> >  	writel(val, port->base + PCIE_AXI_WINDOW0);
> >  
> >  	return 0;
> > -- 
> > 2.6.4
> > 



_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [SPAM][PATCH v3 0/2] PCI: mediatek: enable whole MMIO range and enlarge the PCIe2AHB window size

[Ryder Lee]

On Fri, 2019-02-01 at 13:36 +0800, Honghui Zhang (张洪辉) wrote:
> From: Honghui Zhang <honghui.zhang@mediatek.com>
> 
> Two patches:
> patch 1 enable whole MMIO range which also fix the complain of scripts/coccinelle/api/resource_size.cocci
> patch 2 enlarge the PCIe2AHB window size to support fully access of 4GB DRAM from EP DMA.
> 
> v3:
>  - update the changlog title for patch1 and update commit message following Bjorn's suggestion
>  - move the "|" into the previous line.
> 
> v2:
>  - Fix the checkpatch complains for patch 1.
>  - update the commit message and change title of patch 1 for changelog conventions.
>  - Add patch 2.
> 
> Honghui Zhang (2):
>   PCI: mediatek: Enable the whole memory mapped IO range
>   PCI: mediatek: Enlarge PCIe2AHB window size to support 4GB DRAM
> 
>  drivers/pci/controller/pcie-mediatek.c | 13 +++++++++----
>  1 file changed, 9 insertions(+), 4 deletions(-)
> 

For the series:

Acked-by: Ryder Lee <ryder.lee@mediatek.com>


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v3 0/2] PCI: mediatek: enable whole MMIO range and enlarge the PCIe2AHB window size

[Lorenzo Pieralisi]

On Fri, Feb 01, 2019 at 01:36:05PM +0800, honghui.zhang@mediatek.com wrote:
> From: Honghui Zhang <honghui.zhang@mediatek.com>
> 
> Two patches:
> patch 1 enable whole MMIO range which also fix the complain of scripts/coccinelle/api/resource_size.cocci
> patch 2 enlarge the PCIe2AHB window size to support fully access of 4GB DRAM from EP DMA.
> 
> v3:
>  - update the changlog title for patch1 and update commit message following Bjorn's suggestion
>  - move the "|" into the previous line.
> 
> v2:
>  - Fix the checkpatch complains for patch 1.
>  - update the commit message and change title of patch 1 for changelog conventions.
>  - Add patch 2.
> 
> Honghui Zhang (2):
>   PCI: mediatek: Enable the whole memory mapped IO range
>   PCI: mediatek: Enlarge PCIe2AHB window size to support 4GB DRAM
> 
>  drivers/pci/controller/pcie-mediatek.c | 13 +++++++++----
>  1 file changed, 9 insertions(+), 4 deletions(-)

I have basically rewritten the commit logs and applied the
patches to pci/mediatek for v5.1 (pending testing), please
have a look.

Lorenzo

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel