Re: [PATCH v8 07/21] iommu/io-pgtable-arm-v7s: Extend MediaTek 4GB Mode

[Yong Wu <yong.wu@mediatek.com>]

On Wed, 2019-07-10 at 15:36 +0100, Will Deacon wrote:
> On Sat, Jun 29, 2019 at 10:09:13AM +0800, Yong Wu wrote:
> > MediaTek extend the arm v7s descriptor to support the dram over 4GB.
> > 
> > In the mt2712 and mt8173, it's called "4GB mode", the physical address
> > is from 0x4000_0000 to 0x1_3fff_ffff, but from EMI point of view, it
> > is remapped to high address from 0x1_0000_0000 to 0x1_ffff_ffff, the
> > bit32 is always enabled. thus, in the M4U, we always enable the bit9
> > for all PTEs which means to enable bit32 of physical address.
> > 
> > but in mt8183, M4U support the dram from 0x4000_0000 to 0x3_ffff_ffff
> > which isn't remaped. We extend the PTEs: the bit9 represent bit32 of
> > PA and the bit4 represent bit33 of PA. Meanwhile the iova still is
> > 32bits.
> 
> What happens if bit4 is set in the pte for mt2712 or mt8173? Perhaps the

bit4 is ignored in mt2712 and mt8173(No effect).

> io-pgtable backend should be allowing oas > 32 when
> IO_PGTABLE_QUIRK_ARM_MTK_4GB is set, and then enforcing that itself.

About oas, It looks the oas doesn't work in current the v7s. 

How about I add a new simple preparing patch like this(copy from
io-pgtable-arm.c)?

==========================================
--- a/drivers/iommu/io-pgtable-arm-v7s.c
+++ b/drivers/iommu/io-pgtable-arm-v7s.c
@@ -495,7 +495,8 @@ static int arm_v7s_map(struct io_pgtable_ops *ops,
unsigned long iova,
        if (!(prot & (IOMMU_READ | IOMMU_WRITE)))
                return 0;

-       if (WARN_ON(upper_32_bits(iova) || upper_32_bits(paddr)))
+       if (WARN_ON(iova >= (1ULL << data->iop.cfg.ias) ||
+                   paddr >= (1ULL << data->iop.cfg.oas)))
                return -ERANGE;

===============================================

Then, change the oas in MTK 4GB mode, like this:

================================================
--- a/drivers/iommu/io-pgtable-arm-v7s.c
+++ b/drivers/iommu/io-pgtable-arm-v7s.c
@@ -721,7 +721,9 @@ static struct io_pgtable
*arm_v7s_alloc_pgtable(struct io_pgtable_cfg *cfg,
 {
        struct arm_v7s_io_pgtable *data;

-       if (cfg->ias > ARM_V7S_ADDR_BITS || cfg->oas >
ARM_V7S_ADDR_BITS)
+       if (cfg->ias > ARM_V7S_ADDR_BITS ||
+           (cfg->oas > ARM_V7S_ADDR_BITS &&
+            !(cfg->quirks & IO_PGTABLE_QUIRK_ARM_MTK_4GB)))
                return NULL;

        if (cfg->quirks & ~(IO_PGTABLE_QUIRK_ARM_NS |

--- a/drivers/iommu/mtk_iommu.c
+++ b/drivers/iommu/mtk_iommu.c
@@ -274,7 +274,7 @@ static int mtk_iommu_domain_finalise(struct
mtk_iommu_domain *dom)
                        IO_PGTABLE_QUIRK_TLBI_ON_MAP,
                .pgsize_bitmap = mtk_iommu_ops.pgsize_bitmap,
                .ias = 32,
-               .oas = 32,
+               .oas = 34,
                .tlb = &mtk_iommu_gather_ops,
                .iommu_dev = data->dev,
        };
================================================


> 
> > In order to unify code, in the "4GB mode", we add the bit32 for the
> > physical address manually in our driver.
> > 
> > Correspondingly, Adding bit32 and bit33 for the PA in the iova_to_phys
> > has to been moved into v7s.
> > 
> > Regarding whether the pagetable address could be over 4GB, the mt8183
> > support it while the previous mt8173 don't. thus keep it as is.
> > 
> > Signed-off-by: Yong Wu <yong.wu@mediatek.com>
> > Reviewed-by: Robin Murphy <robin.murphy@arm.com>
> > Reviewed-by: Evan Green <evgreen@chromium.org>
> > ---
> > Comparing with the previous one:
> > 1). Add a new patch "iommu/mediatek: Fix iova_to_phys PA start for 4GB
> > mode" before this one. Thus rebase it.
> > A little difference: in the 4gb mode, we add bit32 for PA. and the PA got
> > from iova_to_phys always have bit32 here, thus we should adjust it to locate
> > the valid pa.
> > 2). Add this code suggested from Evan.
> >  if (!data->plat_data->has_4gb_mode)
> > 	       data->enable_4GB = false;
> > ---
> >  drivers/iommu/io-pgtable-arm-v7s.c | 31 ++++++++++++++++++++++++-------
> >  drivers/iommu/mtk_iommu.c          | 29 ++++++++++++++++++-----------
> >  drivers/iommu/mtk_iommu.h          |  1 +
> >  3 files changed, 43 insertions(+), 18 deletions(-)
> > 
> > diff --git a/drivers/iommu/io-pgtable-arm-v7s.c b/drivers/iommu/io-pgtable-arm-v7s.c
> > index 94c38db..4077822 100644
> > --- a/drivers/iommu/io-pgtable-arm-v7s.c
> > +++ b/drivers/iommu/io-pgtable-arm-v7s.c
> > @@ -123,7 +123,9 @@
> >  #define ARM_V7S_TEX_MASK		0x7
> >  #define ARM_V7S_ATTR_TEX(val)		(((val) & ARM_V7S_TEX_MASK) << ARM_V7S_TEX_SHIFT)
> >  
> > -#define ARM_V7S_ATTR_MTK_4GB		BIT(9) /* MTK extend it for 4GB mode */
> > +/* MediaTek extend the two bits below for over 4GB mode */
> > +#define ARM_V7S_ATTR_MTK_PA_BIT32	BIT(9)
> > +#define ARM_V7S_ATTR_MTK_PA_BIT33	BIT(4)
> >  
> >  /* *well, except for TEX on level 2 large pages, of course :( */
> >  #define ARM_V7S_CONT_PAGE_TEX_SHIFT	6
> > @@ -190,13 +192,22 @@ static dma_addr_t __arm_v7s_dma_addr(void *pages)
> >  static arm_v7s_iopte paddr_to_iopte(phys_addr_t paddr, int lvl,
> >  				    struct io_pgtable_cfg *cfg)
> >  {
> > -	return paddr & ARM_V7S_LVL_MASK(lvl);
> > +	arm_v7s_iopte pte = paddr & ARM_V7S_LVL_MASK(lvl);
> > +
> > +	if (cfg->quirks & IO_PGTABLE_QUIRK_ARM_MTK_4GB) {
> > +		if (paddr & BIT_ULL(32))
> > +			pte |= ARM_V7S_ATTR_MTK_PA_BIT32;
> > +		if (paddr & BIT_ULL(33))
> > +			pte |= ARM_V7S_ATTR_MTK_PA_BIT33;
> > +	}
> > +	return pte;
> >  }
> >  
> >  static phys_addr_t iopte_to_paddr(arm_v7s_iopte pte, int lvl,
> >  				  struct io_pgtable_cfg *cfg)
> >  {
> >  	arm_v7s_iopte mask;
> > +	phys_addr_t paddr;
> >  
> >  	if (ARM_V7S_PTE_IS_TABLE(pte, lvl))
> >  		mask = ARM_V7S_TABLE_MASK;
> > @@ -205,7 +216,14 @@ static phys_addr_t iopte_to_paddr(arm_v7s_iopte pte, int lvl,
> >  	else
> >  		mask = ARM_V7S_LVL_MASK(lvl);
> >  
> > -	return pte & mask;
> > +	paddr = pte & mask;
> > +	if (cfg->quirks & IO_PGTABLE_QUIRK_ARM_MTK_4GB) {
> > +		if (pte & ARM_V7S_ATTR_MTK_PA_BIT32)
> > +			paddr |= BIT_ULL(32);
> > +		if (pte & ARM_V7S_ATTR_MTK_PA_BIT33)
> > +			paddr |= BIT_ULL(33);
> > +	}
> > +	return paddr;
> 
> I think this relies on CONFIG_PHYS_ADDR_T_64BIT, which isn't always set on
> 32-bit ARM.

This was discussed at [1]. Robin commented that this is not needed and
build won't complain about this.

[1]
http://lists.infradead.org/pipermail/linux-mediatek/2018-November/015688.html

> 
> Will