From: Jichao Zou <zoujc@motorola.com> To: "akpm@linux-foundation.org" <akpm@linux-foundation.org>, "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>, "linux-mm@kvack.org" <linux-mm@kvack.org>, "minchan@kernel.org" <minchan@kernel.org>, "david@redhat.com" <david@redhat.com>, "song.bao.hua@hisilicon.com" <song.bao.hua@hisilicon.com>, "hch@lst.de" <hch@lst.de>, "m.szyprowski@samsung.com" <m.szyprowski@samsung.com>, "robin.murphy@arm.com" <robin.murphy@arm.com>, "iommu@lists.linux-foundation.org" <iommu@lists.linux-foundation.org>, JianQi Yang <yangj@motorola.com>, Yanjune Tian <tianyje@motorola.com> Subject: An cma optimization patch is used for cma_[alloc|free]. Date: Fri, 13 Aug 2021 07:00:07 +0000 [thread overview] Message-ID: <HK0PR03MB4177A44D9DAD3302251B0EC8D5FA9@HK0PR03MB4177.apcprd03.prod.outlook.com> (raw) [-- Attachment #1: Type: text/plain, Size: 399 bytes --] Pre-allocate CMA memory that configured in device tree, this greatly improves the CMA memory allocation efficiency, cma_[alloc|free] is less than 1ms, old way is took a few ms to tens or hundreds ms. Thanks. Best Regards, Zou Jichao 邹纪超 Advisory Engineer, SW BSP MBG ROW SW BJ PF BSP (CN) Motorola Mobility, A Lenovo Company motorola.com [-- Attachment #2: 0001-cma-optimize-cma-allocation.patch --] [-- Type: application/octet-stream, Size: 4988 bytes --] From a82dbc46e2343c394d3edcabcbc73ea6e9f403da Mon Sep 17 00:00:00 2001 From: Jichao Zou <zoujc@motorola.com> Date: Thu, 12 Aug 2021 18:20:21 +0800 Subject: [PATCH] cma:optimize cma allocation. Pre-allocate CMA memory that configured in device tree, this greatly improves the CMA memory allocation efficiency, cma_[alloc|free] is less than 1ms, old way is took a few ms to tens or hundreds ms. Signed-off-by: Jichao Zou <zoujc@motorola.com> --- include/linux/cma.h | 3 ++- kernel/dma/contiguous.c | 2 +- mm/cma.c | 35 ++++++++++++++++++++++++++++++++--- mm/cma.h | 1 + 4 files changed, 36 insertions(+), 5 deletions(-) diff --git a/include/linux/cma.h b/include/linux/cma.h index 53fd8c3cdbd0..68bc147a82a7 100644 --- a/include/linux/cma.h +++ b/include/linux/cma.h @@ -43,7 +43,8 @@ static inline int __init cma_declare_contiguous(phys_addr_t base, extern int cma_init_reserved_mem(phys_addr_t base, phys_addr_t size, unsigned int order_per_bit, const char *name, - struct cma **res_cma); + struct cma **res_cma, + unsigned long node); extern struct page *cma_alloc(struct cma *cma, unsigned long count, unsigned int align, bool no_warn); extern bool cma_release(struct cma *cma, const struct page *pages, unsigned long count); diff --git a/kernel/dma/contiguous.c b/kernel/dma/contiguous.c index 3d63d91cba5c..d77c2745244c 100644 --- a/kernel/dma/contiguous.c +++ b/kernel/dma/contiguous.c @@ -421,7 +421,7 @@ static int __init rmem_cma_setup(struct reserved_mem *rmem) return -EINVAL; } - err = cma_init_reserved_mem(rmem->base, rmem->size, 0, rmem->name, &cma); + err = cma_init_reserved_mem(rmem->base, rmem->size, 0, rmem->name, &cma, node); if (err) { pr_err("Reserved memory: unable to setup CMA region\n"); return err; diff --git a/mm/cma.c b/mm/cma.c index 995e15480937..c5682d03c5e9 100644 --- a/mm/cma.c +++ b/mm/cma.c @@ -32,6 +32,7 @@ #include <linux/io.h> #include <linux/kmemleak.h> #include <trace/events/cma.h> +#include <linux/of_fdt.h> #include "cma.h" @@ -124,6 +125,17 @@ static void __init cma_activate_area(struct cma *cma) INIT_HLIST_HEAD(&cma->mem_head); spin_lock_init(&cma->mem_head_lock); #endif + if (cma->preallocated_cma) { + struct acr_info info = {0}; + + pfn = base_pfn; + if (!alloc_contig_range(pfn, pfn + cma->count, MIGRATE_CMA, GFP_KERNEL, &info)) { + pr_info("CMA area %s be pre-allocated successfully\n", cma->name); + } else { + cma->preallocated_cma = false; + pr_err("CMA area %s be pre-allocated failure\n", cma->name); + } + } return; @@ -159,13 +171,15 @@ core_initcall(cma_init_reserved_areas); * the area will be set to "cmaN", where N is a running counter of * used areas. * @res_cma: Pointer to store the created cma region. + * @node: CMA memory dtsi node. * * This function creates custom contiguous area from already reserved memory. */ int __init cma_init_reserved_mem(phys_addr_t base, phys_addr_t size, unsigned int order_per_bit, const char *name, - struct cma **res_cma) + struct cma **res_cma, + unsigned long node) { struct cma *cma; phys_addr_t alignment; @@ -204,6 +218,9 @@ int __init cma_init_reserved_mem(phys_addr_t base, phys_addr_t size, cma->base_pfn = PFN_DOWN(base); cma->count = size >> PAGE_SHIFT; cma->order_per_bit = order_per_bit; + if (node) + cma->preallocated_cma = of_get_flat_dt_prop(node, "linux,preallocated-cma", NULL); + *res_cma = cma; cma_area_count++; totalcma_pages += (size / PAGE_SIZE); @@ -369,7 +386,7 @@ int __init cma_declare_contiguous_nid(phys_addr_t base, base = addr; } - ret = cma_init_reserved_mem(base, size, order_per_bit, name, res_cma); + ret = cma_init_reserved_mem(base, size, order_per_bit, name, res_cma, 0); if (ret) goto free_mem; @@ -471,6 +488,16 @@ struct page *cma_alloc(struct cma *cma, unsigned long count, spin_unlock_irq(&cma->lock); pfn = cma->base_pfn + (bitmap_no << cma->order_per_bit); + + /* + * cma bitmap should ensure that pfn is in the cma. + */ + if (cma->preallocated_cma) { + BUG_ON(pfn + count > cma->base_pfn + cma->count); + page = pfn_to_page(pfn); + ret = 0; + break; + } ret = alloc_contig_range(pfn, pfn + count, MIGRATE_CMA, GFP_KERNEL | (no_warn ? __GFP_NOWARN : 0)); @@ -551,7 +578,9 @@ bool cma_release(struct cma *cma, const struct page *pages, VM_BUG_ON(pfn + count > cma->base_pfn + cma->count); - free_contig_range(pfn, count); + if (!cma->preallocated_cma) + free_contig_range(pfn, count); + cma_clear_bitmap(cma, pfn, count); trace_cma_release(cma->name, pfn, pages, count); diff --git a/mm/cma.h b/mm/cma.h index 2c775877eae2..1778cb0e68c4 100644 --- a/mm/cma.h +++ b/mm/cma.h @@ -30,6 +30,7 @@ struct cma { /* kobject requires dynamic object */ struct cma_kobject *cma_kobj; #endif + bool preallocated_cma; }; extern struct cma cma_areas[MAX_CMA_AREAS]; -- 2.25.1
WARNING: multiple messages have this Message-ID (diff)
From: Jichao Zou <zoujc@motorola.com> To: "akpm@linux-foundation.org" <akpm@linux-foundation.org>, "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>, "linux-mm@kvack.org" <linux-mm@kvack.org>, "minchan@kernel.org" <minchan@kernel.org>, "david@redhat.com" <david@redhat.com>, "song.bao.hua@hisilicon.com" <song.bao.hua@hisilicon.com>, "hch@lst.de" <hch@lst.de>, "m.szyprowski@samsung.com" <m.szyprowski@samsung.com>, "robin.murphy@arm.com" <robin.murphy@arm.com>, "iommu@lists.linux-foundation.org" <iommu@lists.linux-foundation.org>, JianQi Yang <yangj@motorola.com>, Yanjune Tian <tianyje@motorola.com> Subject: An cma optimization patch is used for cma_[alloc|free]. Date: Fri, 13 Aug 2021 07:00:07 +0000 [thread overview] Message-ID: <HK0PR03MB4177A44D9DAD3302251B0EC8D5FA9@HK0PR03MB4177.apcprd03.prod.outlook.com> (raw) [-- Attachment #1: Type: text/plain, Size: 399 bytes --] Pre-allocate CMA memory that configured in device tree, this greatly improves the CMA memory allocation efficiency, cma_[alloc|free] is less than 1ms, old way is took a few ms to tens or hundreds ms. Thanks. Best Regards, Zou Jichao 邹纪超 Advisory Engineer, SW BSP MBG ROW SW BJ PF BSP (CN) Motorola Mobility, A Lenovo Company motorola.com [-- Attachment #2: 0001-cma-optimize-cma-allocation.patch --] [-- Type: application/octet-stream, Size: 4988 bytes --] From a82dbc46e2343c394d3edcabcbc73ea6e9f403da Mon Sep 17 00:00:00 2001 From: Jichao Zou <zoujc@motorola.com> Date: Thu, 12 Aug 2021 18:20:21 +0800 Subject: [PATCH] cma:optimize cma allocation. Pre-allocate CMA memory that configured in device tree, this greatly improves the CMA memory allocation efficiency, cma_[alloc|free] is less than 1ms, old way is took a few ms to tens or hundreds ms. Signed-off-by: Jichao Zou <zoujc@motorola.com> --- include/linux/cma.h | 3 ++- kernel/dma/contiguous.c | 2 +- mm/cma.c | 35 ++++++++++++++++++++++++++++++++--- mm/cma.h | 1 + 4 files changed, 36 insertions(+), 5 deletions(-) diff --git a/include/linux/cma.h b/include/linux/cma.h index 53fd8c3cdbd0..68bc147a82a7 100644 --- a/include/linux/cma.h +++ b/include/linux/cma.h @@ -43,7 +43,8 @@ static inline int __init cma_declare_contiguous(phys_addr_t base, extern int cma_init_reserved_mem(phys_addr_t base, phys_addr_t size, unsigned int order_per_bit, const char *name, - struct cma **res_cma); + struct cma **res_cma, + unsigned long node); extern struct page *cma_alloc(struct cma *cma, unsigned long count, unsigned int align, bool no_warn); extern bool cma_release(struct cma *cma, const struct page *pages, unsigned long count); diff --git a/kernel/dma/contiguous.c b/kernel/dma/contiguous.c index 3d63d91cba5c..d77c2745244c 100644 --- a/kernel/dma/contiguous.c +++ b/kernel/dma/contiguous.c @@ -421,7 +421,7 @@ static int __init rmem_cma_setup(struct reserved_mem *rmem) return -EINVAL; } - err = cma_init_reserved_mem(rmem->base, rmem->size, 0, rmem->name, &cma); + err = cma_init_reserved_mem(rmem->base, rmem->size, 0, rmem->name, &cma, node); if (err) { pr_err("Reserved memory: unable to setup CMA region\n"); return err; diff --git a/mm/cma.c b/mm/cma.c index 995e15480937..c5682d03c5e9 100644 --- a/mm/cma.c +++ b/mm/cma.c @@ -32,6 +32,7 @@ #include <linux/io.h> #include <linux/kmemleak.h> #include <trace/events/cma.h> +#include <linux/of_fdt.h> #include "cma.h" @@ -124,6 +125,17 @@ static void __init cma_activate_area(struct cma *cma) INIT_HLIST_HEAD(&cma->mem_head); spin_lock_init(&cma->mem_head_lock); #endif + if (cma->preallocated_cma) { + struct acr_info info = {0}; + + pfn = base_pfn; + if (!alloc_contig_range(pfn, pfn + cma->count, MIGRATE_CMA, GFP_KERNEL, &info)) { + pr_info("CMA area %s be pre-allocated successfully\n", cma->name); + } else { + cma->preallocated_cma = false; + pr_err("CMA area %s be pre-allocated failure\n", cma->name); + } + } return; @@ -159,13 +171,15 @@ core_initcall(cma_init_reserved_areas); * the area will be set to "cmaN", where N is a running counter of * used areas. * @res_cma: Pointer to store the created cma region. + * @node: CMA memory dtsi node. * * This function creates custom contiguous area from already reserved memory. */ int __init cma_init_reserved_mem(phys_addr_t base, phys_addr_t size, unsigned int order_per_bit, const char *name, - struct cma **res_cma) + struct cma **res_cma, + unsigned long node) { struct cma *cma; phys_addr_t alignment; @@ -204,6 +218,9 @@ int __init cma_init_reserved_mem(phys_addr_t base, phys_addr_t size, cma->base_pfn = PFN_DOWN(base); cma->count = size >> PAGE_SHIFT; cma->order_per_bit = order_per_bit; + if (node) + cma->preallocated_cma = of_get_flat_dt_prop(node, "linux,preallocated-cma", NULL); + *res_cma = cma; cma_area_count++; totalcma_pages += (size / PAGE_SIZE); @@ -369,7 +386,7 @@ int __init cma_declare_contiguous_nid(phys_addr_t base, base = addr; } - ret = cma_init_reserved_mem(base, size, order_per_bit, name, res_cma); + ret = cma_init_reserved_mem(base, size, order_per_bit, name, res_cma, 0); if (ret) goto free_mem; @@ -471,6 +488,16 @@ struct page *cma_alloc(struct cma *cma, unsigned long count, spin_unlock_irq(&cma->lock); pfn = cma->base_pfn + (bitmap_no << cma->order_per_bit); + + /* + * cma bitmap should ensure that pfn is in the cma. + */ + if (cma->preallocated_cma) { + BUG_ON(pfn + count > cma->base_pfn + cma->count); + page = pfn_to_page(pfn); + ret = 0; + break; + } ret = alloc_contig_range(pfn, pfn + count, MIGRATE_CMA, GFP_KERNEL | (no_warn ? __GFP_NOWARN : 0)); @@ -551,7 +578,9 @@ bool cma_release(struct cma *cma, const struct page *pages, VM_BUG_ON(pfn + count > cma->base_pfn + cma->count); - free_contig_range(pfn, count); + if (!cma->preallocated_cma) + free_contig_range(pfn, count); + cma_clear_bitmap(cma, pfn, count); trace_cma_release(cma->name, pfn, pages, count); diff --git a/mm/cma.h b/mm/cma.h index 2c775877eae2..1778cb0e68c4 100644 --- a/mm/cma.h +++ b/mm/cma.h @@ -30,6 +30,7 @@ struct cma { /* kobject requires dynamic object */ struct cma_kobject *cma_kobj; #endif + bool preallocated_cma; }; extern struct cma cma_areas[MAX_CMA_AREAS]; -- 2.25.1 [-- Attachment #3: Type: text/plain, Size: 156 bytes --] _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
next reply other threads:[~2021-08-13 7:00 UTC|newest] Thread overview: 14+ messages / expand[flat|nested] mbox.gz Atom feed top 2021-08-13 7:00 Jichao Zou [this message] 2021-08-13 7:00 ` An cma optimization patch is used for cma_[alloc|free] Jichao Zou 2021-08-13 7:45 ` David Hildenbrand 2021-08-13 7:45 ` David Hildenbrand 2021-08-13 8:27 ` 回复: [External]Re: " Jichao Zou 2021-08-13 8:27 ` Jichao Zou 2021-08-13 9:15 ` Robin Murphy 2021-08-13 9:15 ` Robin Murphy 2021-08-13 9:46 ` 回复: " Jichao Zou 2021-08-13 9:46 ` Jichao Zou 2021-08-13 10:08 ` Robin Murphy 2021-08-13 10:08 ` Robin Murphy 2021-08-13 11:26 ` 回复: " Jichao Zou 2021-08-13 11:26 ` Jichao Zou
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=HK0PR03MB4177A44D9DAD3302251B0EC8D5FA9@HK0PR03MB4177.apcprd03.prod.outlook.com \ --to=zoujc@motorola.com \ --cc=akpm@linux-foundation.org \ --cc=david@redhat.com \ --cc=hch@lst.de \ --cc=iommu@lists.linux-foundation.org \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-mm@kvack.org \ --cc=m.szyprowski@samsung.com \ --cc=minchan@kernel.org \ --cc=robin.murphy@arm.com \ --cc=song.bao.hua@hisilicon.com \ --cc=tianyje@motorola.com \ --cc=yangj@motorola.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.