Re: [PATCH v4 3/3] iommu/io-pgtable-arm-v7s: Request DMA32 memory, and improve debugging

From: Vlastimil Babka <vbabka@suse.cz>
To: Nicolas Boichat <drinkcat@chromium.org>,
	Will Deacon <will.deacon@arm.com>
Cc: Robin Murphy <robin.murphy@arm.com>,
	Joerg Roedel <joro@8bytes.org>, Christoph Lameter <cl@linux.com>,
	Pekka Enberg <penberg@kernel.org>,
	David Rientjes <rientjes@google.com>,
	Joonsoo Kim <iamjoonsoo.kim@lge.com>,
	Andrew Morton <akpm@linux-foundation.org>,
	Michal Hocko <mhocko@suse.com>,
	Mel Gorman <mgorman@techsingularity.net>,
	Levin Alexander <Alexander.Levin@microsoft.com>,
	Huaisheng Ye <yehs1@lenovo.com>,
	Mike Rapoport <rppt@linux.vnet.ibm.com>,
	linux-arm-kernel@lists.infradead.org,
	iommu@lists.linux-foundation.org, linux-kernel@vger.kernel.org,
	linux-mm@kvack.org, Yong Wu <yong.wu@mediatek.com>,
	Matthias Brugger <matthias.bgg@gmail.com>,
	Tomasz Figa <tfiga@google.com>,
	yingjoe.chen@mediatek.com, hch@infradead.org,
	Matthew Wilcox <willy@infradead.org>
Subject: Re: [PATCH v4 3/3] iommu/io-pgtable-arm-v7s: Request DMA32 memory, and improve debugging
Date: Wed, 5 Dec 2018 15:43:52 +0100	[thread overview]
Message-ID: <cf005731-5f37-4aba-519f-7d7d4da04ec7@suse.cz> (raw)
In-Reply-To: <20181205054828.183476-4-drinkcat@chromium.org>

On 12/5/18 6:48 AM, Nicolas Boichat wrote:
> IOMMUs using ARMv7 short-descriptor format require page tables
> (level 1 and 2) to be allocated within the first 4GB of RAM, even
> on 64-bit systems.
> 
> For level 1/2 pages, ensure GFP_DMA32 is used if CONFIG_ZONE_DMA32
> is defined (e.g. on arm64 platforms).
> 
> For level 2 pages, allocate a slab cache in SLAB_CACHE_DMA32.
> 
> Also, print an error when the physical address does not fit in
> 32-bit, to make debugging easier in the future.
> 
> Fixes: ad67f5a6545f ("arm64: replace ZONE_DMA with ZONE_DMA32")
> Signed-off-by: Nicolas Boichat <drinkcat@chromium.org>
> ---
> 
> Changes since v2:
>  - Commit message
> 
> (v3 used the page_frag approach)
> 
>  drivers/iommu/io-pgtable-arm-v7s.c | 20 ++++++++++++++++----
>  1 file changed, 16 insertions(+), 4 deletions(-)
> 
> diff --git a/drivers/iommu/io-pgtable-arm-v7s.c b/drivers/iommu/io-pgtable-arm-v7s.c
> index 445c3bde04800c..996f7b6d00b44a 100644
> --- a/drivers/iommu/io-pgtable-arm-v7s.c
> +++ b/drivers/iommu/io-pgtable-arm-v7s.c
> @@ -161,6 +161,14 @@
>  
>  #define ARM_V7S_TCR_PD1			BIT(5)
>  
> +#ifdef CONFIG_ZONE_DMA32
> +#define ARM_V7S_TABLE_GFP_DMA GFP_DMA32
> +#define ARM_V7S_TABLE_SLAB_CACHE SLAB_CACHE_DMA32
> +#else
> +#define ARM_V7S_TABLE_GFP_DMA GFP_DMA
> +#define ARM_V7S_TABLE_SLAB_CACHE SLAB_CACHE_DMA
> +#endif
> +
>  typedef u32 arm_v7s_iopte;
>  
>  static bool selftest_running;
> @@ -198,13 +206,17 @@ static void *__arm_v7s_alloc_table(int lvl, gfp_t gfp,
>  	void *table = NULL;
>  
>  	if (lvl == 1)
> -		table = (void *)__get_dma_pages(__GFP_ZERO, get_order(size));
> +		table = (void *)__get_free_pages(
> +			__GFP_ZERO | ARM_V7S_TABLE_GFP_DMA, get_order(size));
>  	else if (lvl == 2)
> -		table = kmem_cache_zalloc(data->l2_tables, gfp | GFP_DMA);
> +		table = kmem_cache_zalloc(data->l2_tables,
> +					  gfp | ARM_V7S_TABLE_GFP_DMA);

So as I've explained in 2/3, you don't need ARM_V7S_TABLE_GFP_DMA here
(and then you don't need to adjust the slab warnings).

>  	phys = virt_to_phys(table);
> -	if (phys != (arm_v7s_iopte)phys)
> +	if (phys != (arm_v7s_iopte)phys) {
>  		/* Doesn't fit in PTE */
> +		dev_err(dev, "Page table does not fit in PTE: %pa", &phys);
>  		goto out_free;
> +	}
>  	if (table && !(cfg->quirks & IO_PGTABLE_QUIRK_NO_DMA)) {
>  		dma = dma_map_single(dev, table, size, DMA_TO_DEVICE);
>  		if (dma_mapping_error(dev, dma))
> @@ -737,7 +749,7 @@ static struct io_pgtable *arm_v7s_alloc_pgtable(struct io_pgtable_cfg *cfg,
>  	data->l2_tables = kmem_cache_create("io-pgtable_armv7s_l2",
>  					    ARM_V7S_TABLE_SIZE(2),
>  					    ARM_V7S_TABLE_SIZE(2),
> -					    SLAB_CACHE_DMA, NULL);
> +					    ARM_V7S_TABLE_SLAB_CACHE, NULL);
>  	if (!data->l2_tables)
>  		goto out_free_data;
>  
> 