All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH 0/3] support per-numa CMA for ARM server
@ 2020-06-03  2:42 ` Barry Song
  0 siblings, 0 replies; 64+ messages in thread
From: Barry Song @ 2020-06-03  2:42 UTC (permalink / raw)
  To: hch, m.szyprowski, robin.murphy, catalin.marinas
  Cc: iommu, linux-arm-kernel, linux-kernel, linuxarm,
	Jonathan.Cameron, john.garry, prime.zeng, Barry Song

Right now, smmu is using dma_alloc_coherent() to get memory to save queues
and tables. Typically, on ARM64 server, there is a default CMA located at
node0, which could be far away from node2, node3 etc.
Saving queues and tables remotely will increase the latency of ARM SMMU
significantly. For example, when SMMU is at node2 and the default global
CMA is at node0, after sending a CMD_SYNC in an empty command queue, we
have to wait more than 550ns for the completion of the command CMD_SYNC.
However, if we save them locally, we only need to wait for 240ns.

with per-numa CMA, smmu will get memory from local numa node to save command
queues and page tables. that means dma_unmap latency will be shrunk much.

Meanwhile, when iommu.passthrough is on, device drivers which call dma_
alloc_coherent() will also get local memory and avoid the travel between
numa nodes.

Barry Song (3):
  dma-direct: provide the ability to reserve per-numa CMA
  arm64: mm: reserve hugetlb CMA after numa_init
  arm64: mm: reserve per-numa CMA after numa_init

 arch/arm64/mm/init.c           | 12 ++++++----
 include/linux/dma-contiguous.h |  4 ++++
 kernel/dma/Kconfig             | 10 ++++++++
 kernel/dma/contiguous.c        | 43 +++++++++++++++++++++++++++++++++-
 4 files changed, 63 insertions(+), 6 deletions(-)

-- 
2.23.0



^ permalink raw reply	[flat|nested] 64+ messages in thread
* Re: [PATCH 1/3] dma-direct: provide the ability to reserve per-numa CMA
@ 2020-06-04  7:17 kernel test robot
  0 siblings, 0 replies; 64+ messages in thread
From: kernel test robot @ 2020-06-04  7:17 UTC (permalink / raw)
  To: kbuild

[-- Attachment #1: Type: text/plain, Size: 7185 bytes --]

CC: kbuild-all(a)lists.01.org
In-Reply-To: <20200603024231.61748-2-song.bao.hua@hisilicon.com>
References: <20200603024231.61748-2-song.bao.hua@hisilicon.com>
TO: Barry Song <song.bao.hua@hisilicon.com>
TO: hch(a)lst.de
TO: m.szyprowski(a)samsung.com
TO: robin.murphy(a)arm.com
TO: catalin.marinas(a)arm.com
CC: iommu(a)lists.linux-foundation.org
CC: linux-arm-kernel(a)lists.infradead.org
CC: linux-kernel(a)vger.kernel.org
CC: linuxarm(a)huawei.com
CC: Jonathan.Cameron(a)huawei.com
CC: john.garry(a)huawei.com

Hi Barry,

I love your patch! Perhaps something to improve:

[auto build test WARNING on arm64/for-next/core]
[also build test WARNING on linus/master v5.7]
[cannot apply to next-20200603]
[if your patch is applied to the wrong git tree, please drop us a note to help
improve the system. BTW, we also suggest to use '--base' option to specify the
base tree in git format-patch, please see https://stackoverflow.com/a/37406982]

url:    https://github.com/0day-ci/linux/commits/Barry-Song/support-per-numa-CMA-for-ARM-server/20200603-104821
base:   https://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-next/core
:::::: branch date: 28 hours ago
:::::: commit date: 28 hours ago
config: x86_64-randconfig-m001-20200603 (attached as .config)
compiler: gcc-9 (Debian 9.3.0-13) 9.3.0

If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot <lkp@intel.com>
Reported-by: Dan Carpenter <dan.carpenter@oracle.com>

smatch warnings:
kernel/dma/contiguous.c:274 dma_alloc_contiguous() warn: variable dereferenced before check 'dev' (see line 272)

# https://github.com/0day-ci/linux/commit/adb919e972c1cac3d8b11905d5258d23d3aac6a4
git remote add linux-review https://github.com/0day-ci/linux
git remote update linux-review
git checkout adb919e972c1cac3d8b11905d5258d23d3aac6a4
vim +/dev +274 kernel/dma/contiguous.c

de9e14eebf33a6 drivers/base/dma-contiguous.c Marek Szyprowski  2014-10-13  251  
b1d2dc009dece4 kernel/dma/contiguous.c       Nicolin Chen      2019-05-23  252  /**
b1d2dc009dece4 kernel/dma/contiguous.c       Nicolin Chen      2019-05-23  253   * dma_alloc_contiguous() - allocate contiguous pages
b1d2dc009dece4 kernel/dma/contiguous.c       Nicolin Chen      2019-05-23  254   * @dev:   Pointer to device for which the allocation is performed.
b1d2dc009dece4 kernel/dma/contiguous.c       Nicolin Chen      2019-05-23  255   * @size:  Requested allocation size.
b1d2dc009dece4 kernel/dma/contiguous.c       Nicolin Chen      2019-05-23  256   * @gfp:   Allocation flags.
b1d2dc009dece4 kernel/dma/contiguous.c       Nicolin Chen      2019-05-23  257   *
b1d2dc009dece4 kernel/dma/contiguous.c       Nicolin Chen      2019-05-23  258   * This function allocates contiguous memory buffer for specified device. It
b1d2dc009dece4 kernel/dma/contiguous.c       Nicolin Chen      2019-05-23  259   * first tries to use device specific contiguous memory area if available or
adb919e972c1ca kernel/dma/contiguous.c       Barry Song        2020-06-03  260   * the per-numa ones and default global one, then tries a fallback allocation
adb919e972c1ca kernel/dma/contiguous.c       Barry Song        2020-06-03  261   * of normal pages. per-numa memory areas don't support address limit
bd2e75633c8012 kernel/dma/contiguous.c       Nicolin Chen      2019-05-23  262   *
bd2e75633c8012 kernel/dma/contiguous.c       Nicolin Chen      2019-05-23  263   * Note that it byapss one-page size of allocations from the global area as
bd2e75633c8012 kernel/dma/contiguous.c       Nicolin Chen      2019-05-23  264   * the addresses within one page are always contiguous, so there is no need
bd2e75633c8012 kernel/dma/contiguous.c       Nicolin Chen      2019-05-23  265   * to waste CMA pages for that kind; it also helps reduce fragmentations.
b1d2dc009dece4 kernel/dma/contiguous.c       Nicolin Chen      2019-05-23  266   */
b1d2dc009dece4 kernel/dma/contiguous.c       Nicolin Chen      2019-05-23  267  struct page *dma_alloc_contiguous(struct device *dev, size_t size, gfp_t gfp)
b1d2dc009dece4 kernel/dma/contiguous.c       Nicolin Chen      2019-05-23  268  {
90ae409f9eb3bc kernel/dma/contiguous.c       Christoph Hellwig 2019-08-20  269  	size_t count = size >> PAGE_SHIFT;
b1d2dc009dece4 kernel/dma/contiguous.c       Nicolin Chen      2019-05-23  270  	struct page *page = NULL;
bd2e75633c8012 kernel/dma/contiguous.c       Nicolin Chen      2019-05-23  271  	struct cma *cma = NULL;
adb919e972c1ca kernel/dma/contiguous.c       Barry Song        2020-06-03 @272  	int nid = dev_to_node(dev);
bd2e75633c8012 kernel/dma/contiguous.c       Nicolin Chen      2019-05-23  273  
bd2e75633c8012 kernel/dma/contiguous.c       Nicolin Chen      2019-05-23 @274  	if (dev && dev->cma_area)
bd2e75633c8012 kernel/dma/contiguous.c       Nicolin Chen      2019-05-23  275  		cma = dev->cma_area;
adb919e972c1ca kernel/dma/contiguous.c       Barry Song        2020-06-03  276  	else if ((nid != NUMA_NO_NODE) && dma_contiguous_pernuma_area[nid]
adb919e972c1ca kernel/dma/contiguous.c       Barry Song        2020-06-03  277  		&& !(gfp & (GFP_DMA | GFP_DMA32)))
adb919e972c1ca kernel/dma/contiguous.c       Barry Song        2020-06-03  278  		cma = dma_contiguous_pernuma_area[nid];
bd2e75633c8012 kernel/dma/contiguous.c       Nicolin Chen      2019-05-23  279  	else if (count > 1)
bd2e75633c8012 kernel/dma/contiguous.c       Nicolin Chen      2019-05-23  280  		cma = dma_contiguous_default_area;
b1d2dc009dece4 kernel/dma/contiguous.c       Nicolin Chen      2019-05-23  281  
b1d2dc009dece4 kernel/dma/contiguous.c       Nicolin Chen      2019-05-23  282  	/* CMA can be used only in the context which permits sleeping */
b1d2dc009dece4 kernel/dma/contiguous.c       Nicolin Chen      2019-05-23  283  	if (cma && gfpflags_allow_blocking(gfp)) {
90ae409f9eb3bc kernel/dma/contiguous.c       Christoph Hellwig 2019-08-20  284  		size_t align = get_order(size);
c6622a425acd1d kernel/dma/contiguous.c       Nicolin Chen      2019-07-26  285  		size_t cma_align = min_t(size_t, align, CONFIG_CMA_ALIGNMENT);
c6622a425acd1d kernel/dma/contiguous.c       Nicolin Chen      2019-07-26  286  
c6622a425acd1d kernel/dma/contiguous.c       Nicolin Chen      2019-07-26  287  		page = cma_alloc(cma, count, cma_align, gfp & __GFP_NOWARN);
b1d2dc009dece4 kernel/dma/contiguous.c       Nicolin Chen      2019-05-23  288  	}
b1d2dc009dece4 kernel/dma/contiguous.c       Nicolin Chen      2019-05-23  289  
b1d2dc009dece4 kernel/dma/contiguous.c       Nicolin Chen      2019-05-23  290  	return page;
b1d2dc009dece4 kernel/dma/contiguous.c       Nicolin Chen      2019-05-23  291  }
b1d2dc009dece4 kernel/dma/contiguous.c       Nicolin Chen      2019-05-23  292  

:::::: The code at line 274 was first introduced by commit
:::::: bd2e75633c8012fc8a7431c82fda66237133bf7e dma-contiguous: use fallback alloc_pages for single pages

:::::: TO: Nicolin Chen <nicoleotsuka@gmail.com>
:::::: CC: Christoph Hellwig <hch@lst.de>

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org

[-- Attachment #2: config.gz --]
[-- Type: application/gzip, Size: 32337 bytes --]

^ permalink raw reply	[flat|nested] 64+ messages in thread

end of thread, other threads:[~2020-06-09 15:34 UTC | newest]

Thread overview: 64+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-06-03  2:42 [PATCH 0/3] support per-numa CMA for ARM server Barry Song
2020-06-03  2:42 ` Barry Song
2020-06-03  2:42 ` Barry Song
2020-06-03  2:42 ` [PATCH 1/3] dma-direct: provide the ability to reserve per-numa CMA Barry Song
2020-06-03  2:42   ` Barry Song
2020-06-03  2:42   ` Barry Song
2020-06-03  6:55   ` kbuild test robot
2020-06-03  6:55     ` kbuild test robot
2020-06-03  6:55     ` kbuild test robot
2020-06-03  6:55     ` kbuild test robot
2020-06-03  7:18   ` kbuild test robot
2020-06-03  7:18     ` kbuild test robot
2020-06-03  7:18     ` kbuild test robot
2020-06-03  7:18     ` kbuild test robot
2020-06-03  7:18   ` [RFC PATCH] dma-direct: dma_contiguous_pernuma_area[] can be static kbuild test robot
2020-06-03  7:18     ` kbuild test robot
2020-06-03  7:18     ` kbuild test robot
2020-06-03  7:18     ` kbuild test robot
2020-06-04 11:36   ` [PATCH 1/3] dma-direct: provide the ability to reserve per-numa CMA Dan Carpenter
2020-06-04 11:36     ` Dan Carpenter
2020-06-04 11:36     ` Dan Carpenter
2020-06-04 11:36     ` Dan Carpenter
2020-06-04 11:36     ` Dan Carpenter
2020-06-05  6:04     ` Song Bao Hua (Barry Song)
2020-06-05  6:04       ` Song Bao Hua
2020-06-05  6:04       ` Song Bao Hua (Barry Song)
2020-06-05  6:04       ` Song Bao Hua (Barry Song)
2020-06-05  8:57       ` Dan Carpenter
2020-06-05  8:57         ` Dan Carpenter
2020-06-05  8:57         ` Dan Carpenter
2020-06-05  8:57         ` Dan Carpenter
2020-06-05  8:57         ` Dan Carpenter
2020-06-06  3:46         ` [kbuild-all] " Philip Li
2020-06-06  3:46           ` Philip Li
2020-06-06  3:46           ` [kbuild-all] " Philip Li
2020-06-06  3:46           ` Philip Li
2020-06-06  3:46           ` Philip Li
2020-06-06 10:15           ` Song Bao Hua (Barry Song)
2020-06-06 10:15             ` Song Bao Hua
2020-06-06 10:15             ` [kbuild-all] " Song Bao Hua (Barry Song)
2020-06-06 10:15             ` Song Bao Hua (Barry Song)
2020-06-05 13:57   ` kernel test robot
2020-06-05 13:57     ` kernel test robot
2020-06-05 13:57     ` kernel test robot
2020-06-05 13:57     ` kernel test robot
2020-06-03  2:42 ` [PATCH 2/3] arm64: mm: reserve hugetlb CMA after numa_init Barry Song
2020-06-03  2:42   ` Barry Song
2020-06-03  2:42   ` Barry Song
2020-06-03  3:22   ` Roman Gushchin
2020-06-03  3:22     ` Roman Gushchin
2020-06-03  3:22     ` Roman Gushchin via iommu
2020-06-07 20:14     ` Matthias Brugger
2020-06-07 20:14       ` Matthias Brugger
2020-06-07 20:14       ` Matthias Brugger
2020-06-08  0:50       ` Song Bao Hua (Barry Song)
2020-06-08  0:50         ` Song Bao Hua (Barry Song)
2020-06-08  0:50         ` Song Bao Hua (Barry Song)
2020-06-09 15:33         ` Matthias Brugger
2020-06-09 15:33           ` Matthias Brugger
2020-06-09 15:33           ` Matthias Brugger
2020-06-03  2:42 ` [PATCH 3/3] arm64: mm: reserve per-numa " Barry Song
2020-06-03  2:42   ` Barry Song
2020-06-03  2:42   ` Barry Song
2020-06-04  7:17 [PATCH 1/3] dma-direct: provide the ability to reserve per-numa CMA kernel test robot

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.