From: Robin Murphy <robin.murphy-5wv7dgnIgG8@public.gmane.org> To: will.deacon-5wv7dgnIgG8@public.gmane.org Cc: iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org, linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org Subject: [PATCH] iommu/io-pgtable-arm: Make allocations NUMA-aware Date: Mon, 21 May 2018 19:12:40 +0100 [thread overview] Message-ID: <b745ecc2acd6aad20b5c3776ecd58022a9019c91.1526926225.git.robin.murphy@arm.com> (raw) We would generally expect pagetables to be read by the IOMMU more than written by the CPU, so in NUMA systems it would be preferable to avoid the IOMMU making cross-node pagetable walks if possible. We already have a handle on the IOMMU device for the sake of coherency management, so it's trivial to grab the appropriate NUMA node when allocating new pagetable pages. Note that we drop the semantics of alloc_pages_exact(), but that's fine since they have never been necessary: the only time we're allocating more than one page is for stage 2 top-level concatenation, but since that is based on the number of IPA bits, the size is always some exact power of two anyway. Signed-off-by: Robin Murphy <robin.murphy-5wv7dgnIgG8@public.gmane.org> --- drivers/iommu/io-pgtable-arm.c | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/drivers/iommu/io-pgtable-arm.c b/drivers/iommu/io-pgtable-arm.c index 39c2a056da21..e80ca386c5b4 100644 --- a/drivers/iommu/io-pgtable-arm.c +++ b/drivers/iommu/io-pgtable-arm.c @@ -231,12 +231,16 @@ static void *__arm_lpae_alloc_pages(size_t size, gfp_t gfp, struct io_pgtable_cfg *cfg) { struct device *dev = cfg->iommu_dev; + int order = get_order(size); + struct page *p; dma_addr_t dma; - void *pages = alloc_pages_exact(size, gfp | __GFP_ZERO); + void *pages; - if (!pages) + p = alloc_pages_node(dev_to_node(dev), gfp | __GFP_ZERO, order); + if (!p) return NULL; + pages = page_address(p); if (!(cfg->quirks & IO_PGTABLE_QUIRK_NO_DMA)) { dma = dma_map_single(dev, pages, size, DMA_TO_DEVICE); if (dma_mapping_error(dev, dma)) @@ -256,7 +260,7 @@ static void *__arm_lpae_alloc_pages(size_t size, gfp_t gfp, dev_err(dev, "Cannot accommodate DMA translation for IOMMU page tables\n"); dma_unmap_single(dev, dma, size, DMA_TO_DEVICE); out_free: - free_pages_exact(pages, size); + __free_pages(p, order); return NULL; } @@ -266,7 +270,7 @@ static void __arm_lpae_free_pages(void *pages, size_t size, if (!(cfg->quirks & IO_PGTABLE_QUIRK_NO_DMA)) dma_unmap_single(cfg->iommu_dev, __arm_lpae_dma_addr(pages), size, DMA_TO_DEVICE); - free_pages_exact(pages, size); + free_pages((unsigned long)pages, get_order(size)); } static void __arm_lpae_sync_pte(arm_lpae_iopte *ptep, -- 2.17.0.dirty
WARNING: multiple messages have this Message-ID (diff)
From: robin.murphy@arm.com (Robin Murphy) To: linux-arm-kernel@lists.infradead.org Subject: [PATCH] iommu/io-pgtable-arm: Make allocations NUMA-aware Date: Mon, 21 May 2018 19:12:40 +0100 [thread overview] Message-ID: <b745ecc2acd6aad20b5c3776ecd58022a9019c91.1526926225.git.robin.murphy@arm.com> (raw) We would generally expect pagetables to be read by the IOMMU more than written by the CPU, so in NUMA systems it would be preferable to avoid the IOMMU making cross-node pagetable walks if possible. We already have a handle on the IOMMU device for the sake of coherency management, so it's trivial to grab the appropriate NUMA node when allocating new pagetable pages. Note that we drop the semantics of alloc_pages_exact(), but that's fine since they have never been necessary: the only time we're allocating more than one page is for stage 2 top-level concatenation, but since that is based on the number of IPA bits, the size is always some exact power of two anyway. Signed-off-by: Robin Murphy <robin.murphy@arm.com> --- drivers/iommu/io-pgtable-arm.c | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/drivers/iommu/io-pgtable-arm.c b/drivers/iommu/io-pgtable-arm.c index 39c2a056da21..e80ca386c5b4 100644 --- a/drivers/iommu/io-pgtable-arm.c +++ b/drivers/iommu/io-pgtable-arm.c @@ -231,12 +231,16 @@ static void *__arm_lpae_alloc_pages(size_t size, gfp_t gfp, struct io_pgtable_cfg *cfg) { struct device *dev = cfg->iommu_dev; + int order = get_order(size); + struct page *p; dma_addr_t dma; - void *pages = alloc_pages_exact(size, gfp | __GFP_ZERO); + void *pages; - if (!pages) + p = alloc_pages_node(dev_to_node(dev), gfp | __GFP_ZERO, order); + if (!p) return NULL; + pages = page_address(p); if (!(cfg->quirks & IO_PGTABLE_QUIRK_NO_DMA)) { dma = dma_map_single(dev, pages, size, DMA_TO_DEVICE); if (dma_mapping_error(dev, dma)) @@ -256,7 +260,7 @@ static void *__arm_lpae_alloc_pages(size_t size, gfp_t gfp, dev_err(dev, "Cannot accommodate DMA translation for IOMMU page tables\n"); dma_unmap_single(dev, dma, size, DMA_TO_DEVICE); out_free: - free_pages_exact(pages, size); + __free_pages(p, order); return NULL; } @@ -266,7 +270,7 @@ static void __arm_lpae_free_pages(void *pages, size_t size, if (!(cfg->quirks & IO_PGTABLE_QUIRK_NO_DMA)) dma_unmap_single(cfg->iommu_dev, __arm_lpae_dma_addr(pages), size, DMA_TO_DEVICE); - free_pages_exact(pages, size); + free_pages((unsigned long)pages, get_order(size)); } static void __arm_lpae_sync_pte(arm_lpae_iopte *ptep, -- 2.17.0.dirty
next reply other threads:[~2018-05-21 18:12 UTC|newest] Thread overview: 6+ messages / expand[flat|nested] mbox.gz Atom feed top 2018-05-21 18:12 Robin Murphy [this message] 2018-05-21 18:12 ` [PATCH] iommu/io-pgtable-arm: Make allocations NUMA-aware Robin Murphy [not found] ` <b745ecc2acd6aad20b5c3776ecd58022a9019c91.1526926225.git.robin.murphy-5wv7dgnIgG8@public.gmane.org> 2018-05-21 18:47 ` Will Deacon 2018-05-21 18:47 ` Will Deacon [not found] ` <20180521184717.GA24025-5wv7dgnIgG8@public.gmane.org> 2018-05-22 10:58 ` Robin Murphy 2018-05-22 10:58 ` Robin Murphy
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=b745ecc2acd6aad20b5c3776ecd58022a9019c91.1526926225.git.robin.murphy@arm.com \ --to=robin.murphy-5wv7dgnigg8@public.gmane.org \ --cc=iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org \ --cc=linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org \ --cc=will.deacon-5wv7dgnIgG8@public.gmane.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.