* [PATCH v2 0/2] Optimize dma_*_from_contiguous calls @ 2019-05-06 22:33 Nicolin Chen 2019-05-06 22:33 ` [PATCH v2 1/2] dma-contiguous: Abstract dma_{alloc, free}_contiguous() Nicolin Chen ` (2 more replies) 0 siblings, 3 replies; 22+ messages in thread From: Nicolin Chen @ 2019-05-06 22:33 UTC (permalink / raw) To: hch, robin.murphy, m.szyprowski Cc: chris, linux-xtensa, keescook, sfr, tony, catalin.marinas, joro, will.deacon, linux, iommu, linux-kernel, jcmvbkbc, wsa+renesas, akpm, treding, dwmw2, iamjoonsoo.kim, linux-arm-kernel [ Per discussion at v1, we decide to add two new functions and start replacing callers one by one. For this series, it only touches the dma-direct part. And instead of merging two PATCHes, I still keep them separate so that we may easily revert PATCH-2 if anything bad happens as last time -- PATCH-1 is supposed to be a safe cleanup. ] This series of patches try to optimize dma_*_from_contiguous calls: PATCH-1 abstracts two new functions and applies to dma-direct.c file. PATCH-2 saves single pages and reduce fragmentations from CMA area. Please check their commit messages for detail changelog. Nicolin Chen (2): dma-contiguous: Abstract dma_{alloc,free}_contiguous() dma-contiguous: Use fallback alloc_pages for single pages include/linux/dma-contiguous.h | 10 ++++++ kernel/dma/contiguous.c | 57 ++++++++++++++++++++++++++++++++++ kernel/dma/direct.c | 24 +++----------- 3 files changed, 71 insertions(+), 20 deletions(-) -- 2.17.1 _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 22+ messages in thread
* [PATCH v2 1/2] dma-contiguous: Abstract dma_{alloc, free}_contiguous() 2019-05-06 22:33 [PATCH v2 0/2] Optimize dma_*_from_contiguous calls Nicolin Chen @ 2019-05-06 22:33 ` Nicolin Chen 2019-05-24 1:52 ` dann frazier 2019-05-06 22:33 ` [PATCH v2 2/2] dma-contiguous: Use fallback alloc_pages for single pages Nicolin Chen 2019-05-08 12:52 ` [PATCH v2 0/2] Optimize dma_*_from_contiguous calls Christoph Hellwig 2 siblings, 1 reply; 22+ messages in thread From: Nicolin Chen @ 2019-05-06 22:33 UTC (permalink / raw) To: hch, robin.murphy, m.szyprowski Cc: chris, linux-xtensa, keescook, sfr, tony, catalin.marinas, joro, will.deacon, linux, iommu, linux-kernel, jcmvbkbc, wsa+renesas, akpm, treding, dwmw2, iamjoonsoo.kim, linux-arm-kernel Both dma_alloc_from_contiguous() and dma_release_from_contiguous() are very simply implemented, but requiring callers to pass certain parameters like count and align, and taking a boolean parameter to check __GFP_NOWARN in the allocation flags. So every function call duplicates similar work: /* A piece of example */ unsigned long order = get_order(size); size_t count = size >> PAGE_SHIFT; page = dma_alloc_from_contiguous(dev, count, order, gfp & __GFP_NOWARN); [...] dma_release_from_contiguous(dev, page, size >> PAGE_SHIFT); Additionally, as CMA can be used only in the context which permits sleeping, most of callers do a gfpflags_allow_blocking() check and a corresponding fallback allocation of normal pages upon any false result: /* A piece of example */ if (gfpflags_allow_blocking(flag)) page = dma_alloc_from_contiguous(); if (!page) page = alloc_pages(); [...] if (!dma_release_from_contiguous(dev, page, count)) __free_pages(page, get_order(size)); So this patch simplifies those function calls by abstracting these operations into the two new functions: dma_{alloc,free}_contiguous. As some callers of dma_{alloc,release}_from_contiguous() might be complicated, this patch just implements these two new functions to kernel/dma/direct.c only as an initial step. Suggested-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Nicolin Chen <nicoleotsuka@gmail.com> --- Changelog v1->v2: * Added new functions beside the old ones so we can replace callers one by one later. * Applied new functions to dma/direct.c only, because it's the best example caller to apply and should be safe with the new functions. include/linux/dma-contiguous.h | 10 +++++++ kernel/dma/contiguous.c | 48 ++++++++++++++++++++++++++++++++++ kernel/dma/direct.c | 24 +++-------------- 3 files changed, 62 insertions(+), 20 deletions(-) diff --git a/include/linux/dma-contiguous.h b/include/linux/dma-contiguous.h index f247e8aa5e3d..dacbdcb91a89 100644 --- a/include/linux/dma-contiguous.h +++ b/include/linux/dma-contiguous.h @@ -115,6 +115,8 @@ struct page *dma_alloc_from_contiguous(struct device *dev, size_t count, unsigned int order, bool no_warn); bool dma_release_from_contiguous(struct device *dev, struct page *pages, int count); +struct page *dma_alloc_contiguous(struct device *dev, size_t size, gfp_t gfp); +void dma_free_contiguous(struct device *dev, struct page *page, size_t size); #else @@ -157,6 +159,14 @@ bool dma_release_from_contiguous(struct device *dev, struct page *pages, return false; } +struct page *dma_alloc_contiguous(struct device *dev, size_t size, gfp_t gfp) +{ + return NULL; +} + +static inline +void dma_free_contiguous(struct device *dev, struct page *page, size_t size) { } + #endif #endif diff --git a/kernel/dma/contiguous.c b/kernel/dma/contiguous.c index b2a87905846d..21f39a6cb04f 100644 --- a/kernel/dma/contiguous.c +++ b/kernel/dma/contiguous.c @@ -214,6 +214,54 @@ bool dma_release_from_contiguous(struct device *dev, struct page *pages, return cma_release(dev_get_cma_area(dev), pages, count); } +/** + * dma_alloc_contiguous() - allocate contiguous pages + * @dev: Pointer to device for which the allocation is performed. + * @size: Requested allocation size. + * @gfp: Allocation flags. + * + * This function allocates contiguous memory buffer for specified device. It + * first tries to use device specific contiguous memory area if available or + * the default global one, then tries a fallback allocation of normal pages. + */ +struct page *dma_alloc_contiguous(struct device *dev, size_t size, gfp_t gfp) +{ + int node = dev ? dev_to_node(dev) : NUMA_NO_NODE; + size_t count = PAGE_ALIGN(size) >> PAGE_SHIFT; + size_t align = get_order(PAGE_ALIGN(size)); + struct cma *cma = dev_get_cma_area(dev); + struct page *page = NULL; + + /* CMA can be used only in the context which permits sleeping */ + if (cma && gfpflags_allow_blocking(gfp)) { + align = min_t(size_t, align, CONFIG_CMA_ALIGNMENT); + page = cma_alloc(cma, count, align, gfp & __GFP_NOWARN); + } + + /* Fallback allocation of normal pages */ + if (!page) + page = alloc_pages_node(node, gfp, align); + + return page; +} + +/** + * dma_free_contiguous() - release allocated pages + * @dev: Pointer to device for which the pages were allocated. + * @page: Pointer to the allocated pages. + * @size: Size of allocated pages. + * + * This function releases memory allocated by dma_alloc_contiguous(). As the + * cma_release returns false when provided pages do not belong to contiguous + * area and true otherwise, this function then does a fallback __free_pages() + * upon a false-return. + */ +void dma_free_contiguous(struct device *dev, struct page *page, size_t size) +{ + if (!cma_release(dev_get_cma_area(dev), page, size >> PAGE_SHIFT)) + __free_pages(page, get_order(size)); +} + /* * Support for reserved memory regions defined in device tree */ diff --git a/kernel/dma/direct.c b/kernel/dma/direct.c index 2c2772e9702a..0816c1e8b05a 100644 --- a/kernel/dma/direct.c +++ b/kernel/dma/direct.c @@ -96,8 +96,6 @@ static bool dma_coherent_ok(struct device *dev, phys_addr_t phys, size_t size) struct page *__dma_direct_alloc_pages(struct device *dev, size_t size, dma_addr_t *dma_handle, gfp_t gfp, unsigned long attrs) { - unsigned int count = PAGE_ALIGN(size) >> PAGE_SHIFT; - int page_order = get_order(size); struct page *page = NULL; u64 phys_mask; @@ -109,20 +107,9 @@ struct page *__dma_direct_alloc_pages(struct device *dev, size_t size, gfp |= __dma_direct_optimal_gfp_mask(dev, dev->coherent_dma_mask, &phys_mask); again: - /* CMA can be used only in the context which permits sleeping */ - if (gfpflags_allow_blocking(gfp)) { - page = dma_alloc_from_contiguous(dev, count, page_order, - gfp & __GFP_NOWARN); - if (page && !dma_coherent_ok(dev, page_to_phys(page), size)) { - dma_release_from_contiguous(dev, page, count); - page = NULL; - } - } - if (!page) - page = alloc_pages_node(dev_to_node(dev), gfp, page_order); - + page = dma_alloc_contiguous(dev, size, gfp); if (page && !dma_coherent_ok(dev, page_to_phys(page), size)) { - __free_pages(page, page_order); + dma_free_contiguous(dev, page, size); page = NULL; if (IS_ENABLED(CONFIG_ZONE_DMA32) && @@ -154,7 +141,7 @@ void *dma_direct_alloc_pages(struct device *dev, size_t size, if (PageHighMem(page)) { /* * Depending on the cma= arguments and per-arch setup - * dma_alloc_from_contiguous could return highmem pages. + * dma_alloc_contiguous could return highmem pages. * Without remapping there is no way to return them here, * so log an error and fail. */ @@ -176,10 +163,7 @@ void *dma_direct_alloc_pages(struct device *dev, size_t size, void __dma_direct_free_pages(struct device *dev, size_t size, struct page *page) { - unsigned int count = PAGE_ALIGN(size) >> PAGE_SHIFT; - - if (!dma_release_from_contiguous(dev, page, count)) - __free_pages(page, get_order(size)); + dma_free_contiguous(dev, page, size); } void dma_direct_free_pages(struct device *dev, size_t size, void *cpu_addr, -- 2.17.1 _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply related [flat|nested] 22+ messages in thread
* Re: [PATCH v2 1/2] dma-contiguous: Abstract dma_{alloc, free}_contiguous() 2019-05-06 22:33 ` [PATCH v2 1/2] dma-contiguous: Abstract dma_{alloc, free}_contiguous() Nicolin Chen @ 2019-05-24 1:52 ` dann frazier 2019-05-24 2:59 ` dann frazier 0 siblings, 1 reply; 22+ messages in thread From: dann frazier @ 2019-05-24 1:52 UTC (permalink / raw) To: Nicolin Chen Cc: chris, keescook, linux-xtensa, tony, Catalin Marinas, joro, Will Deacon, linux, linux-kernel, jcmvbkbc, iommu, dwmw2, linux-arm-kernel, wsa+renesas, sfr, akpm, treding, Robin Murphy, Christoph Hellwig, iamjoonsoo.kim, Marek Szyprowski On Mon, May 6, 2019 at 4:35 PM Nicolin Chen <nicoleotsuka@gmail.com> wrote: > > Both dma_alloc_from_contiguous() and dma_release_from_contiguous() > are very simply implemented, but requiring callers to pass certain > parameters like count and align, and taking a boolean parameter to > check __GFP_NOWARN in the allocation flags. So every function call > duplicates similar work: > /* A piece of example */ > unsigned long order = get_order(size); > size_t count = size >> PAGE_SHIFT; > page = dma_alloc_from_contiguous(dev, count, order, gfp & __GFP_NOWARN); > [...] > dma_release_from_contiguous(dev, page, size >> PAGE_SHIFT); > > Additionally, as CMA can be used only in the context which permits > sleeping, most of callers do a gfpflags_allow_blocking() check and > a corresponding fallback allocation of normal pages upon any false > result: > /* A piece of example */ > if (gfpflags_allow_blocking(flag)) > page = dma_alloc_from_contiguous(); > if (!page) > page = alloc_pages(); > [...] > if (!dma_release_from_contiguous(dev, page, count)) > __free_pages(page, get_order(size)); > > So this patch simplifies those function calls by abstracting these > operations into the two new functions: dma_{alloc,free}_contiguous. > > As some callers of dma_{alloc,release}_from_contiguous() might be > complicated, this patch just implements these two new functions to > kernel/dma/direct.c only as an initial step. > > Suggested-by: Christoph Hellwig <hch@lst.de> > Signed-off-by: Nicolin Chen <nicoleotsuka@gmail.com> > --- > Changelog > v1->v2: > * Added new functions beside the old ones so we can replace callers > one by one later. > * Applied new functions to dma/direct.c only, because it's the best > example caller to apply and should be safe with the new functions. > > include/linux/dma-contiguous.h | 10 +++++++ > kernel/dma/contiguous.c | 48 ++++++++++++++++++++++++++++++++++ > kernel/dma/direct.c | 24 +++-------------- > 3 files changed, 62 insertions(+), 20 deletions(-) > > diff --git a/include/linux/dma-contiguous.h b/include/linux/dma-contiguous.h > index f247e8aa5e3d..dacbdcb91a89 100644 > --- a/include/linux/dma-contiguous.h > +++ b/include/linux/dma-contiguous.h > @@ -115,6 +115,8 @@ struct page *dma_alloc_from_contiguous(struct device *dev, size_t count, > unsigned int order, bool no_warn); > bool dma_release_from_contiguous(struct device *dev, struct page *pages, > int count); > +struct page *dma_alloc_contiguous(struct device *dev, size_t size, gfp_t gfp); > +void dma_free_contiguous(struct device *dev, struct page *page, size_t size); > > #else > > @@ -157,6 +159,14 @@ bool dma_release_from_contiguous(struct device *dev, struct page *pages, > return false; > } > > +struct page *dma_alloc_contiguous(struct device *dev, size_t size, gfp_t gfp) > +{ > + return NULL; > +} > + > +static inline > +void dma_free_contiguous(struct device *dev, struct page *page, size_t size) { } > + > #endif > > #endif > diff --git a/kernel/dma/contiguous.c b/kernel/dma/contiguous.c > index b2a87905846d..21f39a6cb04f 100644 > --- a/kernel/dma/contiguous.c > +++ b/kernel/dma/contiguous.c > @@ -214,6 +214,54 @@ bool dma_release_from_contiguous(struct device *dev, struct page *pages, > return cma_release(dev_get_cma_area(dev), pages, count); > } This breaks the build for me if CONFIG_DMA_CMA=n: LD [M] fs/9p/9p.o ld: fs/9p/vfs_inode.o: in function `dma_alloc_contiguous': vfs_inode.c:(.text+0xa60): multiple definition of `dma_alloc_contiguous'; fs/9p/vfs_super.o:vfs_super.c:(.text+0x500): first defined here Do the following insertions need to be under an #ifdef CONFIG_DMA_CMA ? -dann > +/** > + * dma_alloc_contiguous() - allocate contiguous pages > + * @dev: Pointer to device for which the allocation is performed. > + * @size: Requested allocation size. > + * @gfp: Allocation flags. > + * > + * This function allocates contiguous memory buffer for specified device. It > + * first tries to use device specific contiguous memory area if available or > + * the default global one, then tries a fallback allocation of normal pages. > + */ > +struct page *dma_alloc_contiguous(struct device *dev, size_t size, gfp_t gfp) > +{ > + int node = dev ? dev_to_node(dev) : NUMA_NO_NODE; > + size_t count = PAGE_ALIGN(size) >> PAGE_SHIFT; > + size_t align = get_order(PAGE_ALIGN(size)); > + struct cma *cma = dev_get_cma_area(dev); > + struct page *page = NULL; > + > + /* CMA can be used only in the context which permits sleeping */ > + if (cma && gfpflags_allow_blocking(gfp)) { > + align = min_t(size_t, align, CONFIG_CMA_ALIGNMENT); > + page = cma_alloc(cma, count, align, gfp & __GFP_NOWARN); > + } > + > + /* Fallback allocation of normal pages */ > + if (!page) > + page = alloc_pages_node(node, gfp, align); > + > + return page; > +} > + > +/** > + * dma_free_contiguous() - release allocated pages > + * @dev: Pointer to device for which the pages were allocated. > + * @page: Pointer to the allocated pages. > + * @size: Size of allocated pages. > + * > + * This function releases memory allocated by dma_alloc_contiguous(). As the > + * cma_release returns false when provided pages do not belong to contiguous > + * area and true otherwise, this function then does a fallback __free_pages() > + * upon a false-return. > + */ > +void dma_free_contiguous(struct device *dev, struct page *page, size_t size) > +{ > + if (!cma_release(dev_get_cma_area(dev), page, size >> PAGE_SHIFT)) > + __free_pages(page, get_order(size)); > +} > + > /* > * Support for reserved memory regions defined in device tree > */ > diff --git a/kernel/dma/direct.c b/kernel/dma/direct.c > index 2c2772e9702a..0816c1e8b05a 100644 > --- a/kernel/dma/direct.c > +++ b/kernel/dma/direct.c > @@ -96,8 +96,6 @@ static bool dma_coherent_ok(struct device *dev, phys_addr_t phys, size_t size) > struct page *__dma_direct_alloc_pages(struct device *dev, size_t size, > dma_addr_t *dma_handle, gfp_t gfp, unsigned long attrs) > { > - unsigned int count = PAGE_ALIGN(size) >> PAGE_SHIFT; > - int page_order = get_order(size); > struct page *page = NULL; > u64 phys_mask; > > @@ -109,20 +107,9 @@ struct page *__dma_direct_alloc_pages(struct device *dev, size_t size, > gfp |= __dma_direct_optimal_gfp_mask(dev, dev->coherent_dma_mask, > &phys_mask); > again: > - /* CMA can be used only in the context which permits sleeping */ > - if (gfpflags_allow_blocking(gfp)) { > - page = dma_alloc_from_contiguous(dev, count, page_order, > - gfp & __GFP_NOWARN); > - if (page && !dma_coherent_ok(dev, page_to_phys(page), size)) { > - dma_release_from_contiguous(dev, page, count); > - page = NULL; > - } > - } > - if (!page) > - page = alloc_pages_node(dev_to_node(dev), gfp, page_order); > - > + page = dma_alloc_contiguous(dev, size, gfp); > if (page && !dma_coherent_ok(dev, page_to_phys(page), size)) { > - __free_pages(page, page_order); > + dma_free_contiguous(dev, page, size); > page = NULL; > > if (IS_ENABLED(CONFIG_ZONE_DMA32) && > @@ -154,7 +141,7 @@ void *dma_direct_alloc_pages(struct device *dev, size_t size, > if (PageHighMem(page)) { > /* > * Depending on the cma= arguments and per-arch setup > - * dma_alloc_from_contiguous could return highmem pages. > + * dma_alloc_contiguous could return highmem pages. > * Without remapping there is no way to return them here, > * so log an error and fail. > */ > @@ -176,10 +163,7 @@ void *dma_direct_alloc_pages(struct device *dev, size_t size, > > void __dma_direct_free_pages(struct device *dev, size_t size, struct page *page) > { > - unsigned int count = PAGE_ALIGN(size) >> PAGE_SHIFT; > - > - if (!dma_release_from_contiguous(dev, page, count)) > - __free_pages(page, get_order(size)); > + dma_free_contiguous(dev, page, size); > } > > void dma_direct_free_pages(struct device *dev, size_t size, void *cpu_addr, > -- > 2.17.1 > > > _______________________________________________ > linux-arm-kernel mailing list > linux-arm-kernel@lists.infradead.org > http://lists.infradead.org/mailman/listinfo/linux-arm-kernel _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH v2 1/2] dma-contiguous: Abstract dma_{alloc, free}_contiguous() 2019-05-24 1:52 ` dann frazier @ 2019-05-24 2:59 ` dann frazier 2019-05-24 3:49 ` Nicolin Chen 0 siblings, 1 reply; 22+ messages in thread From: dann frazier @ 2019-05-24 2:59 UTC (permalink / raw) To: Nicolin Chen Cc: chris, keescook, linux-xtensa, tony, Catalin Marinas, joro, Will Deacon, linux, linux-kernel, jcmvbkbc, iommu, dwmw2, linux-arm-kernel, wsa+renesas, sfr, akpm, treding, Robin Murphy, Christoph Hellwig, iamjoonsoo.kim, Marek Szyprowski On Thu, May 23, 2019 at 7:52 PM dann frazier <dann.frazier@canonical.com> wrote: > > On Mon, May 6, 2019 at 4:35 PM Nicolin Chen <nicoleotsuka@gmail.com> wrote: > > > > Both dma_alloc_from_contiguous() and dma_release_from_contiguous() > > are very simply implemented, but requiring callers to pass certain > > parameters like count and align, and taking a boolean parameter to > > check __GFP_NOWARN in the allocation flags. So every function call > > duplicates similar work: > > /* A piece of example */ > > unsigned long order = get_order(size); > > size_t count = size >> PAGE_SHIFT; > > page = dma_alloc_from_contiguous(dev, count, order, gfp & __GFP_NOWARN); > > [...] > > dma_release_from_contiguous(dev, page, size >> PAGE_SHIFT); > > > > Additionally, as CMA can be used only in the context which permits > > sleeping, most of callers do a gfpflags_allow_blocking() check and > > a corresponding fallback allocation of normal pages upon any false > > result: > > /* A piece of example */ > > if (gfpflags_allow_blocking(flag)) > > page = dma_alloc_from_contiguous(); > > if (!page) > > page = alloc_pages(); > > [...] > > if (!dma_release_from_contiguous(dev, page, count)) > > __free_pages(page, get_order(size)); > > > > So this patch simplifies those function calls by abstracting these > > operations into the two new functions: dma_{alloc,free}_contiguous. > > > > As some callers of dma_{alloc,release}_from_contiguous() might be > > complicated, this patch just implements these two new functions to > > kernel/dma/direct.c only as an initial step. > > > > Suggested-by: Christoph Hellwig <hch@lst.de> > > Signed-off-by: Nicolin Chen <nicoleotsuka@gmail.com> > > --- > > Changelog > > v1->v2: > > * Added new functions beside the old ones so we can replace callers > > one by one later. > > * Applied new functions to dma/direct.c only, because it's the best > > example caller to apply and should be safe with the new functions. > > > > include/linux/dma-contiguous.h | 10 +++++++ > > kernel/dma/contiguous.c | 48 ++++++++++++++++++++++++++++++++++ > > kernel/dma/direct.c | 24 +++-------------- > > 3 files changed, 62 insertions(+), 20 deletions(-) > > > > diff --git a/include/linux/dma-contiguous.h b/include/linux/dma-contiguous.h > > index f247e8aa5e3d..dacbdcb91a89 100644 > > --- a/include/linux/dma-contiguous.h > > +++ b/include/linux/dma-contiguous.h > > @@ -115,6 +115,8 @@ struct page *dma_alloc_from_contiguous(struct device *dev, size_t count, > > unsigned int order, bool no_warn); > > bool dma_release_from_contiguous(struct device *dev, struct page *pages, > > int count); > > +struct page *dma_alloc_contiguous(struct device *dev, size_t size, gfp_t gfp); > > +void dma_free_contiguous(struct device *dev, struct page *page, size_t size); > > > > #else > > > > @@ -157,6 +159,14 @@ bool dma_release_from_contiguous(struct device *dev, struct page *pages, > > return false; > > } > > > > +struct page *dma_alloc_contiguous(struct device *dev, size_t size, gfp_t gfp) > > +{ > > + return NULL; > > +} > > + > > +static inline > > +void dma_free_contiguous(struct device *dev, struct page *page, size_t size) { } > > + > > #endif > > > > #endif > > diff --git a/kernel/dma/contiguous.c b/kernel/dma/contiguous.c > > index b2a87905846d..21f39a6cb04f 100644 > > --- a/kernel/dma/contiguous.c > > +++ b/kernel/dma/contiguous.c > > @@ -214,6 +214,54 @@ bool dma_release_from_contiguous(struct device *dev, struct page *pages, > > return cma_release(dev_get_cma_area(dev), pages, count); > > } > > This breaks the build for me if CONFIG_DMA_CMA=n: > > LD [M] fs/9p/9p.o > ld: fs/9p/vfs_inode.o: in function `dma_alloc_contiguous': > vfs_inode.c:(.text+0xa60): multiple definition of > `dma_alloc_contiguous'; fs/9p/vfs_super.o:vfs_super.c:(.text+0x500): > first defined here > > Do the following insertions need to be under an #ifdef CONFIG_DMA_CMA ? Ah, no - the problem is actually a missing "static inline" in the !CONFIG_DMA_CMA version of dma_alloc_contiguous(). -dann > > +/** > > + * dma_alloc_contiguous() - allocate contiguous pages > > + * @dev: Pointer to device for which the allocation is performed. > > + * @size: Requested allocation size. > > + * @gfp: Allocation flags. > > + * > > + * This function allocates contiguous memory buffer for specified device. It > > + * first tries to use device specific contiguous memory area if available or > > + * the default global one, then tries a fallback allocation of normal pages. > > + */ > > +struct page *dma_alloc_contiguous(struct device *dev, size_t size, gfp_t gfp) > > +{ > > + int node = dev ? dev_to_node(dev) : NUMA_NO_NODE; > > + size_t count = PAGE_ALIGN(size) >> PAGE_SHIFT; > > + size_t align = get_order(PAGE_ALIGN(size)); > > + struct cma *cma = dev_get_cma_area(dev); > > + struct page *page = NULL; > > + > > + /* CMA can be used only in the context which permits sleeping */ > > + if (cma && gfpflags_allow_blocking(gfp)) { > > + align = min_t(size_t, align, CONFIG_CMA_ALIGNMENT); > > + page = cma_alloc(cma, count, align, gfp & __GFP_NOWARN); > > + } > > + > > + /* Fallback allocation of normal pages */ > > + if (!page) > > + page = alloc_pages_node(node, gfp, align); > > + > > + return page; > > +} > > + > > +/** > > + * dma_free_contiguous() - release allocated pages > > + * @dev: Pointer to device for which the pages were allocated. > > + * @page: Pointer to the allocated pages. > > + * @size: Size of allocated pages. > > + * > > + * This function releases memory allocated by dma_alloc_contiguous(). As the > > + * cma_release returns false when provided pages do not belong to contiguous > > + * area and true otherwise, this function then does a fallback __free_pages() > > + * upon a false-return. > > + */ > > +void dma_free_contiguous(struct device *dev, struct page *page, size_t size) > > +{ > > + if (!cma_release(dev_get_cma_area(dev), page, size >> PAGE_SHIFT)) > > + __free_pages(page, get_order(size)); > > +} > > + > > /* > > * Support for reserved memory regions defined in device tree > > */ > > diff --git a/kernel/dma/direct.c b/kernel/dma/direct.c > > index 2c2772e9702a..0816c1e8b05a 100644 > > --- a/kernel/dma/direct.c > > +++ b/kernel/dma/direct.c > > @@ -96,8 +96,6 @@ static bool dma_coherent_ok(struct device *dev, phys_addr_t phys, size_t size) > > struct page *__dma_direct_alloc_pages(struct device *dev, size_t size, > > dma_addr_t *dma_handle, gfp_t gfp, unsigned long attrs) > > { > > - unsigned int count = PAGE_ALIGN(size) >> PAGE_SHIFT; > > - int page_order = get_order(size); > > struct page *page = NULL; > > u64 phys_mask; > > > > @@ -109,20 +107,9 @@ struct page *__dma_direct_alloc_pages(struct device *dev, size_t size, > > gfp |= __dma_direct_optimal_gfp_mask(dev, dev->coherent_dma_mask, > > &phys_mask); > > again: > > - /* CMA can be used only in the context which permits sleeping */ > > - if (gfpflags_allow_blocking(gfp)) { > > - page = dma_alloc_from_contiguous(dev, count, page_order, > > - gfp & __GFP_NOWARN); > > - if (page && !dma_coherent_ok(dev, page_to_phys(page), size)) { > > - dma_release_from_contiguous(dev, page, count); > > - page = NULL; > > - } > > - } > > - if (!page) > > - page = alloc_pages_node(dev_to_node(dev), gfp, page_order); > > - > > + page = dma_alloc_contiguous(dev, size, gfp); > > if (page && !dma_coherent_ok(dev, page_to_phys(page), size)) { > > - __free_pages(page, page_order); > > + dma_free_contiguous(dev, page, size); > > page = NULL; > > > > if (IS_ENABLED(CONFIG_ZONE_DMA32) && > > @@ -154,7 +141,7 @@ void *dma_direct_alloc_pages(struct device *dev, size_t size, > > if (PageHighMem(page)) { > > /* > > * Depending on the cma= arguments and per-arch setup > > - * dma_alloc_from_contiguous could return highmem pages. > > + * dma_alloc_contiguous could return highmem pages. > > * Without remapping there is no way to return them here, > > * so log an error and fail. > > */ > > @@ -176,10 +163,7 @@ void *dma_direct_alloc_pages(struct device *dev, size_t size, > > > > void __dma_direct_free_pages(struct device *dev, size_t size, struct page *page) > > { > > - unsigned int count = PAGE_ALIGN(size) >> PAGE_SHIFT; > > - > > - if (!dma_release_from_contiguous(dev, page, count)) > > - __free_pages(page, get_order(size)); > > + dma_free_contiguous(dev, page, size); > > } > > > > void dma_direct_free_pages(struct device *dev, size_t size, void *cpu_addr, > > -- > > 2.17.1 > > > > > > _______________________________________________ > > linux-arm-kernel mailing list > > linux-arm-kernel@lists.infradead.org > > http://lists.infradead.org/mailman/listinfo/linux-arm-kernel _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH v2 1/2] dma-contiguous: Abstract dma_{alloc, free}_contiguous() 2019-05-24 2:59 ` dann frazier @ 2019-05-24 3:49 ` Nicolin Chen 0 siblings, 0 replies; 22+ messages in thread From: Nicolin Chen @ 2019-05-24 3:49 UTC (permalink / raw) To: dann frazier Cc: chris, keescook, linux-xtensa, tony, Catalin Marinas, joro, Will Deacon, linux, linux-kernel, jcmvbkbc, iommu, dwmw2, linux-arm-kernel, wsa+renesas, sfr, akpm, treding, Robin Murphy, Christoph Hellwig, iamjoonsoo.kim, Marek Szyprowski On Thu, May 23, 2019 at 08:59:30PM -0600, dann frazier wrote: > > > diff --git a/kernel/dma/contiguous.c b/kernel/dma/contiguous.c > > > index b2a87905846d..21f39a6cb04f 100644 > > > --- a/kernel/dma/contiguous.c > > > +++ b/kernel/dma/contiguous.c > > > @@ -214,6 +214,54 @@ bool dma_release_from_contiguous(struct device *dev, struct page *pages, > > > return cma_release(dev_get_cma_area(dev), pages, count); > > > } > > > > This breaks the build for me if CONFIG_DMA_CMA=n: > > > > LD [M] fs/9p/9p.o > > ld: fs/9p/vfs_inode.o: in function `dma_alloc_contiguous': > > vfs_inode.c:(.text+0xa60): multiple definition of > > `dma_alloc_contiguous'; fs/9p/vfs_super.o:vfs_super.c:(.text+0x500): > > first defined here > > > > Do the following insertions need to be under an #ifdef CONFIG_DMA_CMA ? > > Ah, no - the problem is actually a missing "static inline" in the > !CONFIG_DMA_CMA version of dma_alloc_contiguous(). Yea, I saw it. Thanks for the testing and pointing it out. Sending v3..... _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 22+ messages in thread
* [PATCH v2 2/2] dma-contiguous: Use fallback alloc_pages for single pages 2019-05-06 22:33 [PATCH v2 0/2] Optimize dma_*_from_contiguous calls Nicolin Chen 2019-05-06 22:33 ` [PATCH v2 1/2] dma-contiguous: Abstract dma_{alloc, free}_contiguous() Nicolin Chen @ 2019-05-06 22:33 ` Nicolin Chen 2019-08-23 12:49 ` Masahiro Yamada 2019-05-08 12:52 ` [PATCH v2 0/2] Optimize dma_*_from_contiguous calls Christoph Hellwig 2 siblings, 1 reply; 22+ messages in thread From: Nicolin Chen @ 2019-05-06 22:33 UTC (permalink / raw) To: hch, robin.murphy, m.szyprowski Cc: chris, linux-xtensa, keescook, sfr, tony, catalin.marinas, joro, will.deacon, linux, iommu, linux-kernel, jcmvbkbc, wsa+renesas, akpm, treding, dwmw2, iamjoonsoo.kim, linux-arm-kernel The addresses within a single page are always contiguous, so it's not so necessary to always allocate one single page from CMA area. Since the CMA area has a limited predefined size of space, it may run out of space in heavy use cases, where there might be quite a lot CMA pages being allocated for single pages. However, there is also a concern that a device might care where a page comes from -- it might expect the page from CMA area and act differently if the page doesn't. This patch tries to use the fallback alloc_pages path, instead of one-page size allocations from the global CMA area in case that a device does not have its own CMA area. This'd save resources from the CMA global area for more CMA allocations, and also reduce CMA fragmentations resulted from trivial allocations. Signed-off-by: Nicolin Chen <nicoleotsuka@gmail.com> --- kernel/dma/contiguous.c | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/kernel/dma/contiguous.c b/kernel/dma/contiguous.c index 21f39a6cb04f..6914b92d5c88 100644 --- a/kernel/dma/contiguous.c +++ b/kernel/dma/contiguous.c @@ -223,14 +223,23 @@ bool dma_release_from_contiguous(struct device *dev, struct page *pages, * This function allocates contiguous memory buffer for specified device. It * first tries to use device specific contiguous memory area if available or * the default global one, then tries a fallback allocation of normal pages. + * + * Note that it byapss one-page size of allocations from the global area as + * the addresses within one page are always contiguous, so there is no need + * to waste CMA pages for that kind; it also helps reduce fragmentations. */ struct page *dma_alloc_contiguous(struct device *dev, size_t size, gfp_t gfp) { int node = dev ? dev_to_node(dev) : NUMA_NO_NODE; size_t count = PAGE_ALIGN(size) >> PAGE_SHIFT; size_t align = get_order(PAGE_ALIGN(size)); - struct cma *cma = dev_get_cma_area(dev); struct page *page = NULL; + struct cma *cma = NULL; + + if (dev && dev->cma_area) + cma = dev->cma_area; + else if (count > 1) + cma = dma_contiguous_default_area; /* CMA can be used only in the context which permits sleeping */ if (cma && gfpflags_allow_blocking(gfp)) { -- 2.17.1 _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply related [flat|nested] 22+ messages in thread
* Re: [PATCH v2 2/2] dma-contiguous: Use fallback alloc_pages for single pages 2019-05-06 22:33 ` [PATCH v2 2/2] dma-contiguous: Use fallback alloc_pages for single pages Nicolin Chen @ 2019-08-23 12:49 ` Masahiro Yamada 2019-08-23 12:56 ` Masahiro Yamada 2019-08-23 22:11 ` Nicolin Chen 0 siblings, 2 replies; 22+ messages in thread From: Masahiro Yamada @ 2019-08-23 12:49 UTC (permalink / raw) To: Nicolin Chen, Christoph Hellwig Cc: Chris Zankel, linux-xtensa, Kees Cook, Stephen Rothwell, Tony Lindgren, Catalin Marinas, Joerg Roedel, Will Deacon, Russell King, iommu, Linux Kernel Mailing List, Max Filippov, Wolfram Sang, iamjoonsoo.kim, Andrew Morton, Thierry Reding, Robin Murphy, David Woodhouse, linux-arm-kernel, Marek Szyprowski On Tue, May 7, 2019 at 7:36 AM Nicolin Chen <nicoleotsuka@gmail.com> wrote: > > The addresses within a single page are always contiguous, so it's > not so necessary to always allocate one single page from CMA area. > Since the CMA area has a limited predefined size of space, it may > run out of space in heavy use cases, where there might be quite a > lot CMA pages being allocated for single pages. > > However, there is also a concern that a device might care where a > page comes from -- it might expect the page from CMA area and act > differently if the page doesn't. > > This patch tries to use the fallback alloc_pages path, instead of > one-page size allocations from the global CMA area in case that a > device does not have its own CMA area. This'd save resources from > the CMA global area for more CMA allocations, and also reduce CMA > fragmentations resulted from trivial allocations. > > Signed-off-by: Nicolin Chen <nicoleotsuka@gmail.com> This commit (bd2e75633c8012fc8a7431c82fda66237133bf7e) broke the DMA for my MMC driver in the following way: [ 1.876755] mmc0: ADMA error [ 1.883385] mmc0: sdhci: ============ SDHCI REGISTER DUMP =========== [ 1.889834] mmc0: sdhci: Sys addr: 0x00000000 | Version: 0x00000002 [ 1.896284] mmc0: sdhci: Blk size: 0x00007200 | Blk cnt: 0x00000001 [ 1.902733] mmc0: sdhci: Argument: 0x00000000 | Trn mode: 0x00000013 [ 1.909182] mmc0: sdhci: Present: 0x01ff02f6 | Host ctl: 0x00000019 [ 1.915631] mmc0: sdhci: Power: 0x0000000b | Blk gap: 0x00000000 [ 1.922081] mmc0: sdhci: Wake-up: 0x00000000 | Clock: 0x0000fa07 [ 1.928530] mmc0: sdhci: Timeout: 0x0000000b | Int stat: 0x00000001 [ 1.934981] mmc0: sdhci: Int enab: 0x03ff008b | Sig enab: 0x03ff008b [ 1.941429] mmc0: sdhci: ACmd stat: 0x00000000 | Slot int: 0x00000001 [ 1.947880] mmc0: sdhci: Caps: 0x546ec800 | Caps_1: 0x00000000 [ 1.954329] mmc0: sdhci: Cmd: 0x0000083a | Max curr: 0x00000000 [ 1.960778] mmc0: sdhci: Resp[0]: 0x00000900 | Resp[1]: 0xffffffff [ 1.967229] mmc0: sdhci: Resp[2]: 0x320f5903 | Resp[3]: 0x3fd05e00 [ 1.973678] mmc0: sdhci: Host ctl2: 0x00000000 [ 1.978125] mmc0: sdhci: ADMA Err: 0x00000001 | ADMA Ptr: 0x000000013965b200 [ 1.985271] mmc0: sdhci: ============================================ [ 1.991758] mmc0: error -5 whilst initialising MMC card [ 1.991913] 43fb0000.uart: ttyS1 at MMIO 0x43fb0000 (irq = 0, base_baud = 768000) is a 16550A [ 2.011011] hctosys: unable to open rtc device (rtc0) [ 2.017694] Freeing unused kernel memory: 2368K [ 2.027131] Run /init as init process Starting syslogd: OK Starting klogd: OK Initializing random number generator... [ 2.074399] random: dd: uninitialized urandom read (512 bytes read) done. Starting network: OK [ 2.109593] mmc0: ADMA error [ 2.112488] mmc0: sdhci: ============ SDHCI REGISTER DUMP =========== [ 2.118941] mmc0: sdhci: Sys addr: 0x00000000 | Version: 0x00000002 [ 2.125389] mmc0: sdhci: Blk size: 0x00007200 | Blk cnt: 0x00000001 [ 2.131840] mmc0: sdhci: Argument: 0x00000000 | Trn mode: 0x00000013 [ 2.138289] mmc0: sdhci: Present: 0x01ff02f6 | Host ctl: 0x00000019 [ 2.144738] mmc0: sdhci: Power: 0x0000000b | Blk gap: 0x00000000 [ 2.151188] mmc0: sdhci: Wake-up: 0x00000000 | Clock: 0x00004e47 [ 2.157637] mmc0: sdhci: Timeout: 0x0000000b | Int stat: 0x00000001 [ 2.164087] mmc0: sdhci: Int enab: 0x03ff008b | Sig enab: 0x03ff008b [ 2.170536] mmc0: sdhci: ACmd stat: 0x00000000 | Slot int: 0x00000001 [ 2.176987] mmc0: sdhci: Caps: 0x546ec800 | Caps_1: 0x00000000 [ 2.183435] mmc0: sdhci: Cmd: 0x0000083a | Max curr: 0x00000000 [ 2.189886] mmc0: sdhci: Resp[0]: 0x00000900 | Resp[1]: 0xffffffff [ 2.196335] mmc0: sdhci: Resp[2]: 0x320f5903 | Resp[3]: 0x3fd05e00 [ 2.202784] mmc0: sdhci: Host ctl2: 0x00000000 [ 2.207232] mmc0: sdhci: ADMA Err: 0x00000001 | ADMA Ptr: 0x000000013965b200 [ 2.214379] mmc0: sdhci: ============================================ [ 2.220881] mmc0: error -5 whilst initialising MMC card Welcome to Buildroot buildroot login: [ 2.332786] mmc0: ADMA error [ 2.335668] mmc0: sdhci: ============ SDHCI REGISTER DUMP =========== [ 2.342119] mmc0: sdhci: Sys addr: 0x00000000 | Version: 0x00000002 [ 2.348568] mmc0: sdhci: Blk size: 0x00007200 | Blk cnt: 0x00000001 [ 2.355018] mmc0: sdhci: Argument: 0x00000000 | Trn mode: 0x00000013 [ 2.361468] mmc0: sdhci: Present: 0x01ff02f6 | Host ctl: 0x00000019 [ 2.367917] mmc0: sdhci: Power: 0x0000000b | Blk gap: 0x00000000 [ 2.374367] mmc0: sdhci: Wake-up: 0x00000000 | Clock: 0x0000f447 [ 2.380816] mmc0: sdhci: Timeout: 0x0000000b | Int stat: 0x00000001 [ 2.387267] mmc0: sdhci: Int enab: 0x03ff008b | Sig enab: 0x03ff008b [ 2.393716] mmc0: sdhci: ACmd stat: 0x00000000 | Slot int: 0x00000001 [ 2.400166] mmc0: sdhci: Caps: 0x546ec800 | Caps_1: 0x00000000 [ 2.406615] mmc0: sdhci: Cmd: 0x0000083a | Max curr: 0x00000000 [ 2.413065] mmc0: sdhci: Resp[0]: 0x00000900 | Resp[1]: 0xffffffff [ 2.419515] mmc0: sdhci: Resp[2]: 0x320f5903 | Resp[3]: 0x3fd05e00 [ 2.425963] mmc0: sdhci: Host ctl2: 0x00000000 [ 2.430412] mmc0: sdhci: ADMA Err: 0x00000001 | ADMA Ptr: 0x000000013965b200 [ 2.437557] mmc0: sdhci: ============================================ [ 2.444031] mmc0: error -5 whilst initialising MMC card [ 2.572203] mmc0: ADMA error [ 2.575089] mmc0: sdhci: ============ SDHCI REGISTER DUMP =========== [ 2.581540] mmc0: sdhci: Sys addr: 0x00000000 | Version: 0x00000002 [ 2.587989] mmc0: sdhci: Blk size: 0x00007200 | Blk cnt: 0x00000001 [ 2.594439] mmc0: sdhci: Argument: 0x00000000 | Trn mode: 0x00000013 [ 2.600889] mmc0: sdhci: Present: 0x01ef02f6 | Host ctl: 0x00000019 [ 2.607339] mmc0: sdhci: Power: 0x0000000b | Blk gap: 0x00000000 [ 2.613788] mmc0: sdhci: Wake-up: 0x00000000 | Clock: 0x0000e8c7 [ 2.620237] mmc0: sdhci: Timeout: 0x0000000b | Int stat: 0x00000001 [ 2.626686] mmc0: sdhci: Int enab: 0x03ff008b | Sig enab: 0x03ff008b [ 2.633137] mmc0: sdhci: ACmd stat: 0x00000000 | Slot int: 0x00000001 [ 2.639586] mmc0: sdhci: Caps: 0x546ec800 | Caps_1: 0x00000000 [ 2.646036] mmc0: sdhci: Cmd: 0x0000083a | Max curr: 0x00000000 [ 2.652485] mmc0: sdhci: Resp[0]: 0x00000900 | Resp[1]: 0xffffffff [ 2.658936] mmc0: sdhci: Resp[2]: 0x320f5903 | Resp[3]: 0x3fd05e00 [ 2.665384] mmc0: sdhci: Host ctl2: 0x00000000 [ 2.669832] mmc0: sdhci: ADMA Err: 0x00000001 | ADMA Ptr: 0x000000013965b200 [ 2.676979] mmc0: sdhci: ============================================ [ 2.683450] mmc0: error -5 whilst initialising MMC card CTRL-A Z for help | 115200 8N1 | NOR | Minicom 2.7.1 | VT102 | Offline | ttyUSB0 Reverting this commit fixed the problem. -- Best Regards Masahiro Yamada _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH v2 2/2] dma-contiguous: Use fallback alloc_pages for single pages 2019-08-23 12:49 ` Masahiro Yamada @ 2019-08-23 12:56 ` Masahiro Yamada 2019-08-25 1:10 ` Christoph Hellwig 2019-08-23 22:11 ` Nicolin Chen 1 sibling, 1 reply; 22+ messages in thread From: Masahiro Yamada @ 2019-08-23 12:56 UTC (permalink / raw) To: Nicolin Chen, Christoph Hellwig, linux-mmc, Ulf Hansson, Adrian Hunter Cc: Chris Zankel, linux-xtensa, Kees Cook, Stephen Rothwell, Tony Lindgren, Catalin Marinas, Joerg Roedel, Will Deacon, Russell King, iommu, Linux Kernel Mailing List, Max Filippov, Wolfram Sang, iamjoonsoo.kim, Andrew Morton, Thierry Reding, Robin Murphy, David Woodhouse, linux-arm-kernel, Marek Szyprowski + linux-mmc, Ulf Hansson, Adrian Hunter, ADMA of SDHCI is not working since bd2e75633c8012fc8a7431c82fda66237133bf7e Did anybody see the same problem? Masahiro On Fri, Aug 23, 2019 at 9:49 PM Masahiro Yamada <yamada.masahiro@socionext.com> wrote: > > On Tue, May 7, 2019 at 7:36 AM Nicolin Chen <nicoleotsuka@gmail.com> wrote: > > > > The addresses within a single page are always contiguous, so it's > > not so necessary to always allocate one single page from CMA area. > > Since the CMA area has a limited predefined size of space, it may > > run out of space in heavy use cases, where there might be quite a > > lot CMA pages being allocated for single pages. > > > > However, there is also a concern that a device might care where a > > page comes from -- it might expect the page from CMA area and act > > differently if the page doesn't. > > > > This patch tries to use the fallback alloc_pages path, instead of > > one-page size allocations from the global CMA area in case that a > > device does not have its own CMA area. This'd save resources from > > the CMA global area for more CMA allocations, and also reduce CMA > > fragmentations resulted from trivial allocations. > > > > Signed-off-by: Nicolin Chen <nicoleotsuka@gmail.com> > > > This commit (bd2e75633c8012fc8a7431c82fda66237133bf7e) > broke the DMA for my MMC driver in the following way: > > > > > [ 1.876755] mmc0: ADMA error > [ 1.883385] mmc0: sdhci: ============ SDHCI REGISTER DUMP =========== > [ 1.889834] mmc0: sdhci: Sys addr: 0x00000000 | Version: 0x00000002 > [ 1.896284] mmc0: sdhci: Blk size: 0x00007200 | Blk cnt: 0x00000001 > [ 1.902733] mmc0: sdhci: Argument: 0x00000000 | Trn mode: 0x00000013 > [ 1.909182] mmc0: sdhci: Present: 0x01ff02f6 | Host ctl: 0x00000019 > [ 1.915631] mmc0: sdhci: Power: 0x0000000b | Blk gap: 0x00000000 > [ 1.922081] mmc0: sdhci: Wake-up: 0x00000000 | Clock: 0x0000fa07 > [ 1.928530] mmc0: sdhci: Timeout: 0x0000000b | Int stat: 0x00000001 > [ 1.934981] mmc0: sdhci: Int enab: 0x03ff008b | Sig enab: 0x03ff008b > [ 1.941429] mmc0: sdhci: ACmd stat: 0x00000000 | Slot int: 0x00000001 > [ 1.947880] mmc0: sdhci: Caps: 0x546ec800 | Caps_1: 0x00000000 > [ 1.954329] mmc0: sdhci: Cmd: 0x0000083a | Max curr: 0x00000000 > [ 1.960778] mmc0: sdhci: Resp[0]: 0x00000900 | Resp[1]: 0xffffffff > [ 1.967229] mmc0: sdhci: Resp[2]: 0x320f5903 | Resp[3]: 0x3fd05e00 > [ 1.973678] mmc0: sdhci: Host ctl2: 0x00000000 > [ 1.978125] mmc0: sdhci: ADMA Err: 0x00000001 | ADMA Ptr: 0x000000013965b200 > [ 1.985271] mmc0: sdhci: ============================================ > [ 1.991758] mmc0: error -5 whilst initialising MMC card > [ 1.991913] 43fb0000.uart: ttyS1 at MMIO 0x43fb0000 (irq = 0, > base_baud = 768000) is a 16550A > [ 2.011011] hctosys: unable to open rtc device (rtc0) > [ 2.017694] Freeing unused kernel memory: 2368K > [ 2.027131] Run /init as init process > Starting syslogd: OK > Starting klogd: OK > Initializing random number generator... [ 2.074399] random: dd: > uninitialized urandom read (512 bytes read) > done. > Starting network: OK > [ 2.109593] mmc0: ADMA error > [ 2.112488] mmc0: sdhci: ============ SDHCI REGISTER DUMP =========== > [ 2.118941] mmc0: sdhci: Sys addr: 0x00000000 | Version: 0x00000002 > [ 2.125389] mmc0: sdhci: Blk size: 0x00007200 | Blk cnt: 0x00000001 > [ 2.131840] mmc0: sdhci: Argument: 0x00000000 | Trn mode: 0x00000013 > [ 2.138289] mmc0: sdhci: Present: 0x01ff02f6 | Host ctl: 0x00000019 > [ 2.144738] mmc0: sdhci: Power: 0x0000000b | Blk gap: 0x00000000 > [ 2.151188] mmc0: sdhci: Wake-up: 0x00000000 | Clock: 0x00004e47 > [ 2.157637] mmc0: sdhci: Timeout: 0x0000000b | Int stat: 0x00000001 > [ 2.164087] mmc0: sdhci: Int enab: 0x03ff008b | Sig enab: 0x03ff008b > [ 2.170536] mmc0: sdhci: ACmd stat: 0x00000000 | Slot int: 0x00000001 > [ 2.176987] mmc0: sdhci: Caps: 0x546ec800 | Caps_1: 0x00000000 > [ 2.183435] mmc0: sdhci: Cmd: 0x0000083a | Max curr: 0x00000000 > [ 2.189886] mmc0: sdhci: Resp[0]: 0x00000900 | Resp[1]: 0xffffffff > [ 2.196335] mmc0: sdhci: Resp[2]: 0x320f5903 | Resp[3]: 0x3fd05e00 > [ 2.202784] mmc0: sdhci: Host ctl2: 0x00000000 > [ 2.207232] mmc0: sdhci: ADMA Err: 0x00000001 | ADMA Ptr: 0x000000013965b200 > [ 2.214379] mmc0: sdhci: ============================================ > > [ 2.220881] mmc0: error -5 whilst initialising MMC card > Welcome to Buildroot > buildroot login: [ 2.332786] mmc0: ADMA error > [ 2.335668] mmc0: sdhci: ============ SDHCI REGISTER DUMP =========== > [ 2.342119] mmc0: sdhci: Sys addr: 0x00000000 | Version: 0x00000002 > [ 2.348568] mmc0: sdhci: Blk size: 0x00007200 | Blk cnt: 0x00000001 > [ 2.355018] mmc0: sdhci: Argument: 0x00000000 | Trn mode: 0x00000013 > [ 2.361468] mmc0: sdhci: Present: 0x01ff02f6 | Host ctl: 0x00000019 > [ 2.367917] mmc0: sdhci: Power: 0x0000000b | Blk gap: 0x00000000 > [ 2.374367] mmc0: sdhci: Wake-up: 0x00000000 | Clock: 0x0000f447 > [ 2.380816] mmc0: sdhci: Timeout: 0x0000000b | Int stat: 0x00000001 > [ 2.387267] mmc0: sdhci: Int enab: 0x03ff008b | Sig enab: 0x03ff008b > [ 2.393716] mmc0: sdhci: ACmd stat: 0x00000000 | Slot int: 0x00000001 > [ 2.400166] mmc0: sdhci: Caps: 0x546ec800 | Caps_1: 0x00000000 > [ 2.406615] mmc0: sdhci: Cmd: 0x0000083a | Max curr: 0x00000000 > [ 2.413065] mmc0: sdhci: Resp[0]: 0x00000900 | Resp[1]: 0xffffffff > [ 2.419515] mmc0: sdhci: Resp[2]: 0x320f5903 | Resp[3]: 0x3fd05e00 > [ 2.425963] mmc0: sdhci: Host ctl2: 0x00000000 > [ 2.430412] mmc0: sdhci: ADMA Err: 0x00000001 | ADMA Ptr: 0x000000013965b200 > [ 2.437557] mmc0: sdhci: ============================================ > [ 2.444031] mmc0: error -5 whilst initialising MMC card > [ 2.572203] mmc0: ADMA error > [ 2.575089] mmc0: sdhci: ============ SDHCI REGISTER DUMP =========== > [ 2.581540] mmc0: sdhci: Sys addr: 0x00000000 | Version: 0x00000002 > [ 2.587989] mmc0: sdhci: Blk size: 0x00007200 | Blk cnt: 0x00000001 > [ 2.594439] mmc0: sdhci: Argument: 0x00000000 | Trn mode: 0x00000013 > [ 2.600889] mmc0: sdhci: Present: 0x01ef02f6 | Host ctl: 0x00000019 > [ 2.607339] mmc0: sdhci: Power: 0x0000000b | Blk gap: 0x00000000 > [ 2.613788] mmc0: sdhci: Wake-up: 0x00000000 | Clock: 0x0000e8c7 > [ 2.620237] mmc0: sdhci: Timeout: 0x0000000b | Int stat: 0x00000001 > [ 2.626686] mmc0: sdhci: Int enab: 0x03ff008b | Sig enab: 0x03ff008b > [ 2.633137] mmc0: sdhci: ACmd stat: 0x00000000 | Slot int: 0x00000001 > [ 2.639586] mmc0: sdhci: Caps: 0x546ec800 | Caps_1: 0x00000000 > [ 2.646036] mmc0: sdhci: Cmd: 0x0000083a | Max curr: 0x00000000 > [ 2.652485] mmc0: sdhci: Resp[0]: 0x00000900 | Resp[1]: 0xffffffff > [ 2.658936] mmc0: sdhci: Resp[2]: 0x320f5903 | Resp[3]: 0x3fd05e00 > [ 2.665384] mmc0: sdhci: Host ctl2: 0x00000000 > [ 2.669832] mmc0: sdhci: ADMA Err: 0x00000001 | ADMA Ptr: 0x000000013965b200 > [ 2.676979] mmc0: sdhci: ============================================ > [ 2.683450] mmc0: error -5 whilst initialising MMC card > > CTRL-A Z for help | 115200 8N1 | NOR | Minicom 2.7.1 | VT102 | Offline > | ttyUSB0 > > Reverting this commit fixed the problem. > > > > -- > Best Regards > Masahiro Yamada -- Best Regards Masahiro Yamada _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH v2 2/2] dma-contiguous: Use fallback alloc_pages for single pages 2019-08-23 12:56 ` Masahiro Yamada @ 2019-08-25 1:10 ` Christoph Hellwig 2019-08-26 2:05 ` Masahiro Yamada 0 siblings, 1 reply; 22+ messages in thread From: Christoph Hellwig @ 2019-08-25 1:10 UTC (permalink / raw) To: Masahiro Yamada Cc: Ulf Hansson, Tony Lindgren, Catalin Marinas, Will Deacon, Linux Kernel Mailing List, Max Filippov, Christoph Hellwig, Marek Szyprowski, Stephen Rothwell, Joerg Roedel, Russell King, Thierry Reding, linux-xtensa, Kees Cook, Nicolin Chen, Andrew Morton, linux-arm-kernel, Chris Zankel, Wolfram Sang, Robin Murphy, linux-mmc, Adrian Hunter, iommu, iamjoonsoo.kim, David Woodhouse On Fri, Aug 23, 2019 at 09:56:52PM +0900, Masahiro Yamada wrote: > + linux-mmc, Ulf Hansson, Adrian Hunter, > > > ADMA of SDHCI is not working > since bd2e75633c8012fc8a7431c82fda66237133bf7e Does it work for you with this commit: http://git.infradead.org/users/hch/dma-mapping.git/commitdiff/90ae409f9eb3bcaf38688f9ec22375816053a08e _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH v2 2/2] dma-contiguous: Use fallback alloc_pages for single pages 2019-08-25 1:10 ` Christoph Hellwig @ 2019-08-26 2:05 ` Masahiro Yamada 2019-08-26 7:33 ` Christoph Hellwig 0 siblings, 1 reply; 22+ messages in thread From: Masahiro Yamada @ 2019-08-26 2:05 UTC (permalink / raw) To: Christoph Hellwig Cc: Ulf Hansson, Tony Lindgren, Catalin Marinas, Will Deacon, Linux Kernel Mailing List, Max Filippov, Marek Szyprowski, Stephen Rothwell, Joerg Roedel, Russell King, Thierry Reding, linux-xtensa, Kees Cook, Nicolin Chen, Andrew Morton, linux-arm-kernel, Chris Zankel, Wolfram Sang, David Woodhouse, linux-mmc, Adrian Hunter, iommu, iamjoonsoo.kim, Robin Murphy Christoph, On Sun, Aug 25, 2019 at 10:10 AM Christoph Hellwig <hch@lst.de> wrote: > > On Fri, Aug 23, 2019 at 09:56:52PM +0900, Masahiro Yamada wrote: > > + linux-mmc, Ulf Hansson, Adrian Hunter, > > > > > > ADMA of SDHCI is not working > > since bd2e75633c8012fc8a7431c82fda66237133bf7e > > Does it work for you with this commit: > > http://git.infradead.org/users/hch/dma-mapping.git/commitdiff/90ae409f9eb3bcaf38688f9ec22375816053a08e This is included in v5.3-rc6 so I tested it. No, it did not fix the problem. -- Best Regards Masahiro Yamada _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH v2 2/2] dma-contiguous: Use fallback alloc_pages for single pages 2019-08-26 2:05 ` Masahiro Yamada @ 2019-08-26 7:33 ` Christoph Hellwig 2019-08-27 7:45 ` Masahiro Yamada 0 siblings, 1 reply; 22+ messages in thread From: Christoph Hellwig @ 2019-08-26 7:33 UTC (permalink / raw) To: Masahiro Yamada Cc: Ulf Hansson, Tony Lindgren, Catalin Marinas, Will Deacon, Linux Kernel Mailing List, Max Filippov, Christoph Hellwig, Marek Szyprowski, Stephen Rothwell, Joerg Roedel, Russell King, Thierry Reding, linux-xtensa, Kees Cook, Nicolin Chen, Andrew Morton, linux-arm-kernel, Chris Zankel, Wolfram Sang, Robin Murphy, linux-mmc, Adrian Hunter, iommu, iamjoonsoo.kim, David Woodhouse On Mon, Aug 26, 2019 at 11:05:00AM +0900, Masahiro Yamada wrote: > This is included in v5.3-rc6 > so I tested it. So there is no allocation failure, but you get I/O errors later? Does the device use a device-private CMA area? Does it work with Linux 5.2 if CONFIG_DMA_CMA is disabled? _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH v2 2/2] dma-contiguous: Use fallback alloc_pages for single pages 2019-08-26 7:33 ` Christoph Hellwig @ 2019-08-27 7:45 ` Masahiro Yamada 2019-08-27 7:50 ` Christoph Hellwig 0 siblings, 1 reply; 22+ messages in thread From: Masahiro Yamada @ 2019-08-27 7:45 UTC (permalink / raw) To: Christoph Hellwig Cc: Ulf Hansson, Tony Lindgren, Catalin Marinas, Will Deacon, Linux Kernel Mailing List, Max Filippov, Marek Szyprowski, Stephen Rothwell, Joerg Roedel, Russell King, Thierry Reding, linux-xtensa, Kees Cook, Nicolin Chen, Andrew Morton, linux-arm-kernel, Chris Zankel, Wolfram Sang, David Woodhouse, linux-mmc, Adrian Hunter, iommu, iamjoonsoo.kim, Robin Murphy On Mon, Aug 26, 2019 at 4:33 PM Christoph Hellwig <hch@lst.de> wrote: > > On Mon, Aug 26, 2019 at 11:05:00AM +0900, Masahiro Yamada wrote: > > This is included in v5.3-rc6 > > so I tested it. > > So there is no allocation failure, but you get I/O errors later? Right. > > Does the device use a device-private CMA area? Not sure. My driver is drivers/mmc/host/sdhci-cadence.c It reuses routines in drivers/mmc/host/sdhci.c > Does it work with Linux > 5.2 if CONFIG_DMA_CMA is disabled? No. 5.2 + disable CONFIG_DMA_CMA failed in the same way. -- Best Regards Masahiro Yamada _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH v2 2/2] dma-contiguous: Use fallback alloc_pages for single pages 2019-08-27 7:45 ` Masahiro Yamada @ 2019-08-27 7:50 ` Christoph Hellwig 2019-08-27 9:03 ` Masahiro Yamada 0 siblings, 1 reply; 22+ messages in thread From: Christoph Hellwig @ 2019-08-27 7:50 UTC (permalink / raw) To: Masahiro Yamada Cc: Ulf Hansson, Tony Lindgren, Catalin Marinas, Will Deacon, Linux Kernel Mailing List, Max Filippov, Christoph Hellwig, Marek Szyprowski, Stephen Rothwell, Joerg Roedel, Russell King, Thierry Reding, linux-xtensa, Kees Cook, Nicolin Chen, Andrew Morton, linux-arm-kernel, Chris Zankel, Wolfram Sang, Robin Murphy, linux-mmc, Adrian Hunter, iommu, iamjoonsoo.kim, David Woodhouse On Tue, Aug 27, 2019 at 04:45:20PM +0900, Masahiro Yamada wrote: > On Mon, Aug 26, 2019 at 4:33 PM Christoph Hellwig <hch@lst.de> wrote: > > > > On Mon, Aug 26, 2019 at 11:05:00AM +0900, Masahiro Yamada wrote: > > > This is included in v5.3-rc6 > > > so I tested it. > > > > So there is no allocation failure, but you get I/O errors later? > > Right. > > > > > Does the device use a device-private CMA area? > > Not sure. > My driver is drivers/mmc/host/sdhci-cadence.c > It reuses routines in drivers/mmc/host/sdhci.c > > > > > Does it work with Linux > > 5.2 if CONFIG_DMA_CMA is disabled? > > No. > 5.2 + disable CONFIG_DMA_CMA > failed in the same way. So it seems like the device wants CMA memory. I guess the patch below will fix it, but that isn't the solution. Can you try it to confirm? In the end it probably assumes a dma mask it doesn't set that the CMA memory satisfies or something similar. diff --git a/kernel/dma/contiguous.c b/kernel/dma/contiguous.c index 69cfb4345388..bd2f24aa7f19 100644 --- a/kernel/dma/contiguous.c +++ b/kernel/dma/contiguous.c @@ -236,7 +236,7 @@ struct page *dma_alloc_contiguous(struct device *dev, size_t size, gfp_t gfp) if (dev && dev->cma_area) cma = dev->cma_area; - else if (count > 1) + else cma = dma_contiguous_default_area; /* CMA can be used only in the context which permits sleeping */ _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply related [flat|nested] 22+ messages in thread
* Re: [PATCH v2 2/2] dma-contiguous: Use fallback alloc_pages for single pages 2019-08-27 7:50 ` Christoph Hellwig @ 2019-08-27 9:03 ` Masahiro Yamada 2019-08-27 11:55 ` Christoph Hellwig 0 siblings, 1 reply; 22+ messages in thread From: Masahiro Yamada @ 2019-08-27 9:03 UTC (permalink / raw) To: Christoph Hellwig Cc: Ulf Hansson, Tony Lindgren, Catalin Marinas, Will Deacon, Linux Kernel Mailing List, Max Filippov, Marek Szyprowski, Stephen Rothwell, Joerg Roedel, Russell King, Thierry Reding, linux-xtensa, Kees Cook, Nicolin Chen, Andrew Morton, linux-arm-kernel, Chris Zankel, Wolfram Sang, David Woodhouse, linux-mmc, Adrian Hunter, iommu, iamjoonsoo.kim, Robin Murphy On Tue, Aug 27, 2019 at 4:50 PM Christoph Hellwig <hch@lst.de> wrote: > > On Tue, Aug 27, 2019 at 04:45:20PM +0900, Masahiro Yamada wrote: > > On Mon, Aug 26, 2019 at 4:33 PM Christoph Hellwig <hch@lst.de> wrote: > > > > > > On Mon, Aug 26, 2019 at 11:05:00AM +0900, Masahiro Yamada wrote: > > > > This is included in v5.3-rc6 > > > > so I tested it. > > > > > > So there is no allocation failure, but you get I/O errors later? > > > > Right. > > > > > > > > Does the device use a device-private CMA area? > > > > Not sure. > > My driver is drivers/mmc/host/sdhci-cadence.c > > It reuses routines in drivers/mmc/host/sdhci.c > > > > > > > > > Does it work with Linux > > > 5.2 if CONFIG_DMA_CMA is disabled? > > > > No. > > 5.2 + disable CONFIG_DMA_CMA > > failed in the same way. > > So it seems like the device wants CMA memory. I guess the patch > below will fix it, but that isn't the solution. Can you try it > to confirm? In the end it probably assumes a dma mask it doesn't > set that the CMA memory satisfies or something similar. Thanks for the pointer. > diff --git a/kernel/dma/contiguous.c b/kernel/dma/contiguous.c > index 69cfb4345388..bd2f24aa7f19 100644 > --- a/kernel/dma/contiguous.c > +++ b/kernel/dma/contiguous.c > @@ -236,7 +236,7 @@ struct page *dma_alloc_contiguous(struct device *dev, size_t size, gfp_t gfp) > > if (dev && dev->cma_area) > cma = dev->cma_area; > - else if (count > 1) > + else > cma = dma_contiguous_default_area; > > /* CMA can be used only in the context which permits sleeping */ Yes, this makes my driver working again when CONFIG_DMA_CMA=y. If I apply the following, my driver gets back working irrespective of CONFIG_DMA_CMA. diff --git a/drivers/mmc/host/sdhci-cadence.c b/drivers/mmc/host/sdhci-cadence.c index 163d1cf4367e..504459395c39 100644 --- a/drivers/mmc/host/sdhci-cadence.c +++ b/drivers/mmc/host/sdhci-cadence.c @@ -237,6 +237,7 @@ static const struct sdhci_ops sdhci_cdns_ops = { static const struct sdhci_pltfm_data sdhci_cdns_pltfm_data = { .ops = &sdhci_cdns_ops, + .quirks2 = SDHCI_QUIRK2_BROKEN_64_BIT_DMA, }; static int sdhci_cdns_set_tune_val(struct sdhci_host *host, unsigned int val) -- Best Regards Masahiro Yamada _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply related [flat|nested] 22+ messages in thread
* Re: [PATCH v2 2/2] dma-contiguous: Use fallback alloc_pages for single pages 2019-08-27 9:03 ` Masahiro Yamada @ 2019-08-27 11:55 ` Christoph Hellwig 2019-08-28 10:53 ` Masahiro Yamada 0 siblings, 1 reply; 22+ messages in thread From: Christoph Hellwig @ 2019-08-27 11:55 UTC (permalink / raw) To: Masahiro Yamada Cc: Ulf Hansson, Tony Lindgren, Catalin Marinas, Will Deacon, Linux Kernel Mailing List, Max Filippov, Christoph Hellwig, Marek Szyprowski, Stephen Rothwell, Joerg Roedel, Russell King, Thierry Reding, linux-xtensa, Kees Cook, Nicolin Chen, Andrew Morton, linux-arm-kernel, Chris Zankel, Wolfram Sang, Robin Murphy, linux-mmc, Adrian Hunter, iommu, iamjoonsoo.kim, David Woodhouse On Tue, Aug 27, 2019 at 06:03:14PM +0900, Masahiro Yamada wrote: > Yes, this makes my driver working again > when CONFIG_DMA_CMA=y. > > > If I apply the following, my driver gets back working > irrespective of CONFIG_DMA_CMA. That sounds a lot like the device simply isn't 64-bit DMA capable, and previously always got CMA allocations under the limit it actually supported. I suggest that you submit this quirk to the mmc maintainers. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH v2 2/2] dma-contiguous: Use fallback alloc_pages for single pages 2019-08-27 11:55 ` Christoph Hellwig @ 2019-08-28 10:53 ` Masahiro Yamada 2019-08-28 12:23 ` Masahiro Yamada 0 siblings, 1 reply; 22+ messages in thread From: Masahiro Yamada @ 2019-08-28 10:53 UTC (permalink / raw) To: Christoph Hellwig Cc: Ulf Hansson, Tony Lindgren, Catalin Marinas, Will Deacon, Linux Kernel Mailing List, Max Filippov, Marek Szyprowski, Stephen Rothwell, Joerg Roedel, Russell King, Thierry Reding, linux-xtensa, Kees Cook, Nicolin Chen, Andrew Morton, linux-arm-kernel, Chris Zankel, Wolfram Sang, David Woodhouse, linux-mmc, Adrian Hunter, iommu, iamjoonsoo.kim, Robin Murphy Hi Christoph, On Tue, Aug 27, 2019 at 8:55 PM Christoph Hellwig <hch@lst.de> wrote: > > On Tue, Aug 27, 2019 at 06:03:14PM +0900, Masahiro Yamada wrote: > > Yes, this makes my driver working again > > when CONFIG_DMA_CMA=y. > > > > > > If I apply the following, my driver gets back working > > irrespective of CONFIG_DMA_CMA. > > That sounds a lot like the device simply isn't 64-bit DMA capable, and > previously always got CMA allocations under the limit it actually > supported. I suggest that you submit this quirk to the mmc maintainers. I tested v5.2 and my MMC host controller works with dma_address that exceeds 32-bit physical address. So, I believe my MMC device is 64-bit DMA capable. I am still looking into the code to find out what was changed. -- Best Regards Masahiro Yamada _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH v2 2/2] dma-contiguous: Use fallback alloc_pages for single pages 2019-08-28 10:53 ` Masahiro Yamada @ 2019-08-28 12:23 ` Masahiro Yamada 2019-08-29 11:45 ` Masahiro Yamada 0 siblings, 1 reply; 22+ messages in thread From: Masahiro Yamada @ 2019-08-28 12:23 UTC (permalink / raw) To: Christoph Hellwig Cc: Ulf Hansson, Tony Lindgren, Catalin Marinas, Will Deacon, Linux Kernel Mailing List, Max Filippov, Marek Szyprowski, Stephen Rothwell, Joerg Roedel, Russell King, Thierry Reding, linux-xtensa, Kees Cook, Nicolin Chen, Andrew Morton, linux-arm-kernel, Chris Zankel, Wolfram Sang, David Woodhouse, linux-mmc, Adrian Hunter, iommu, iamjoonsoo.kim, Robin Murphy On Wed, Aug 28, 2019 at 7:53 PM Masahiro Yamada <yamada.masahiro@socionext.com> wrote: > > Hi Christoph, > > On Tue, Aug 27, 2019 at 8:55 PM Christoph Hellwig <hch@lst.de> wrote: > > > > On Tue, Aug 27, 2019 at 06:03:14PM +0900, Masahiro Yamada wrote: > > > Yes, this makes my driver working again > > > when CONFIG_DMA_CMA=y. > > > > > > > > > If I apply the following, my driver gets back working > > > irrespective of CONFIG_DMA_CMA. > > > > That sounds a lot like the device simply isn't 64-bit DMA capable, and > > previously always got CMA allocations under the limit it actually > > supported. I suggest that you submit this quirk to the mmc maintainers. > > > I tested v5.2 and my MMC host controller works with > dma_address that exceeds 32-bit physical address. > > So, I believe my MMC device is 64-bit DMA capable. > > I am still looking into the code > to find out what was changed. I retract this comment. Prior to bd2e75633c8012fc8a7431c82fda66237133bf7e, the descriptor table for ADMA is placed within the 32-bit phys address range, not exceeds the 32-bit limit. Probably, my device is not 64-bit capable. I will talk to the hardware engineer, and check the hardware spec just in case. Thanks. -- Best Regards Masahiro Yamada _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH v2 2/2] dma-contiguous: Use fallback alloc_pages for single pages 2019-08-28 12:23 ` Masahiro Yamada @ 2019-08-29 11:45 ` Masahiro Yamada 0 siblings, 0 replies; 22+ messages in thread From: Masahiro Yamada @ 2019-08-29 11:45 UTC (permalink / raw) To: Christoph Hellwig Cc: Ulf Hansson, Tony Lindgren, Catalin Marinas, Will Deacon, Linux Kernel Mailing List, Max Filippov, Marek Szyprowski, Stephen Rothwell, Joerg Roedel, Russell King, Thierry Reding, linux-xtensa, Kees Cook, Nicolin Chen, Andrew Morton, linux-arm-kernel, Chris Zankel, Wolfram Sang, David Woodhouse, linux-mmc, Adrian Hunter, iommu, iamjoonsoo.kim, Robin Murphy On Wed, Aug 28, 2019 at 9:23 PM Masahiro Yamada <yamada.masahiro@socionext.com> wrote: > > On Wed, Aug 28, 2019 at 7:53 PM Masahiro Yamada > <yamada.masahiro@socionext.com> wrote: > > > > Hi Christoph, > > > > On Tue, Aug 27, 2019 at 8:55 PM Christoph Hellwig <hch@lst.de> wrote: > > > > > > On Tue, Aug 27, 2019 at 06:03:14PM +0900, Masahiro Yamada wrote: > > > > Yes, this makes my driver working again > > > > when CONFIG_DMA_CMA=y. > > > > > > > > > > > > If I apply the following, my driver gets back working > > > > irrespective of CONFIG_DMA_CMA. > > > > > > That sounds a lot like the device simply isn't 64-bit DMA capable, and > > > previously always got CMA allocations under the limit it actually > > > supported. I suggest that you submit this quirk to the mmc maintainers. > > > > > > I tested v5.2 and my MMC host controller works with > > dma_address that exceeds 32-bit physical address. > > > > So, I believe my MMC device is 64-bit DMA capable. > > > > I am still looking into the code > > to find out what was changed. > > > I retract this comment. > > Prior to bd2e75633c8012fc8a7431c82fda66237133bf7e, > the descriptor table for ADMA is placed within the > 32-bit phys address range, not exceeds the 32-bit limit. > > Probably, my device is not 64-bit capable. > > I will talk to the hardware engineer, > and check the hardware spec just in case. > After looking more into my hardware, I found out how to fix my driver: https://lore.kernel.org/patchwork/patch/1121600/ -- Best Regards Masahiro Yamada _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH v2 2/2] dma-contiguous: Use fallback alloc_pages for single pages 2019-08-23 12:49 ` Masahiro Yamada 2019-08-23 12:56 ` Masahiro Yamada @ 2019-08-23 22:11 ` Nicolin Chen 2019-08-26 1:56 ` Masahiro Yamada 1 sibling, 1 reply; 22+ messages in thread From: Nicolin Chen @ 2019-08-23 22:11 UTC (permalink / raw) To: Masahiro Yamada Cc: Tony Lindgren, Catalin Marinas, Will Deacon, Max Filippov, Christoph Hellwig, Marek Szyprowski, Stephen Rothwell, Joerg Roedel, Russell King, Thierry Reding, linux-xtensa, Kees Cook, Andrew Morton, linux-arm-kernel, Chris Zankel, Wolfram Sang, Robin Murphy, Linux Kernel Mailing List, iommu, iamjoonsoo.kim, David Woodhouse On Fri, Aug 23, 2019 at 09:49:46PM +0900, Masahiro Yamada wrote: > On Tue, May 7, 2019 at 7:36 AM Nicolin Chen <nicoleotsuka@gmail.com> wrote: > > > > The addresses within a single page are always contiguous, so it's > > not so necessary to always allocate one single page from CMA area. > > Since the CMA area has a limited predefined size of space, it may > > run out of space in heavy use cases, where there might be quite a > > lot CMA pages being allocated for single pages. > > > > However, there is also a concern that a device might care where a > > page comes from -- it might expect the page from CMA area and act > > differently if the page doesn't. > > > > This patch tries to use the fallback alloc_pages path, instead of > > one-page size allocations from the global CMA area in case that a > > device does not have its own CMA area. This'd save resources from > > the CMA global area for more CMA allocations, and also reduce CMA > > fragmentations resulted from trivial allocations. > > > > Signed-off-by: Nicolin Chen <nicoleotsuka@gmail.com> > > > This commit (bd2e75633c8012fc8a7431c82fda66237133bf7e) > broke the DMA for my MMC driver in the following way: > > > > > [ 1.876755] mmc0: ADMA error > [ 1.883385] mmc0: sdhci: ============ SDHCI REGISTER DUMP =========== > [ 1.889834] mmc0: sdhci: Sys addr: 0x00000000 | Version: 0x00000002 > [ 1.896284] mmc0: sdhci: Blk size: 0x00007200 | Blk cnt: 0x00000001 > [ 1.902733] mmc0: sdhci: Argument: 0x00000000 | Trn mode: 0x00000013 > [ 1.909182] mmc0: sdhci: Present: 0x01ff02f6 | Host ctl: 0x00000019 > [ 1.915631] mmc0: sdhci: Power: 0x0000000b | Blk gap: 0x00000000 > [ 1.922081] mmc0: sdhci: Wake-up: 0x00000000 | Clock: 0x0000fa07 > [ 1.928530] mmc0: sdhci: Timeout: 0x0000000b | Int stat: 0x00000001 > [ 1.934981] mmc0: sdhci: Int enab: 0x03ff008b | Sig enab: 0x03ff008b > [ 1.941429] mmc0: sdhci: ACmd stat: 0x00000000 | Slot int: 0x00000001 > [ 1.947880] mmc0: sdhci: Caps: 0x546ec800 | Caps_1: 0x00000000 > [ 1.954329] mmc0: sdhci: Cmd: 0x0000083a | Max curr: 0x00000000 > [ 1.960778] mmc0: sdhci: Resp[0]: 0x00000900 | Resp[1]: 0xffffffff > [ 1.967229] mmc0: sdhci: Resp[2]: 0x320f5903 | Resp[3]: 0x3fd05e00 > [ 1.973678] mmc0: sdhci: Host ctl2: 0x00000000 > [ 1.978125] mmc0: sdhci: ADMA Err: 0x00000001 | ADMA Ptr: 0x000000013965b200 > [ 1.985271] mmc0: sdhci: ============================================ > [ 1.991758] mmc0: error -5 whilst initialising MMC card > [ 1.991913] 43fb0000.uart: ttyS1 at MMIO 0x43fb0000 (irq = 0, > base_baud = 768000) is a 16550A > [ 2.011011] hctosys: unable to open rtc device (rtc0) > [ 2.017694] Freeing unused kernel memory: 2368K > [ 2.027131] Run /init as init process > Starting syslogd: OK > Starting klogd: OK > Initializing random number generator... [ 2.074399] random: dd: > uninitialized urandom read (512 bytes read) > done. > Starting network: OK > [ 2.109593] mmc0: ADMA error > [ 2.112488] mmc0: sdhci: ============ SDHCI REGISTER DUMP =========== > [ 2.118941] mmc0: sdhci: Sys addr: 0x00000000 | Version: 0x00000002 > [ 2.125389] mmc0: sdhci: Blk size: 0x00007200 | Blk cnt: 0x00000001 > [ 2.131840] mmc0: sdhci: Argument: 0x00000000 | Trn mode: 0x00000013 > [ 2.138289] mmc0: sdhci: Present: 0x01ff02f6 | Host ctl: 0x00000019 > [ 2.144738] mmc0: sdhci: Power: 0x0000000b | Blk gap: 0x00000000 > [ 2.151188] mmc0: sdhci: Wake-up: 0x00000000 | Clock: 0x00004e47 > [ 2.157637] mmc0: sdhci: Timeout: 0x0000000b | Int stat: 0x00000001 > [ 2.164087] mmc0: sdhci: Int enab: 0x03ff008b | Sig enab: 0x03ff008b > [ 2.170536] mmc0: sdhci: ACmd stat: 0x00000000 | Slot int: 0x00000001 > [ 2.176987] mmc0: sdhci: Caps: 0x546ec800 | Caps_1: 0x00000000 > [ 2.183435] mmc0: sdhci: Cmd: 0x0000083a | Max curr: 0x00000000 > [ 2.189886] mmc0: sdhci: Resp[0]: 0x00000900 | Resp[1]: 0xffffffff > [ 2.196335] mmc0: sdhci: Resp[2]: 0x320f5903 | Resp[3]: 0x3fd05e00 > [ 2.202784] mmc0: sdhci: Host ctl2: 0x00000000 > [ 2.207232] mmc0: sdhci: ADMA Err: 0x00000001 | ADMA Ptr: 0x000000013965b200 > [ 2.214379] mmc0: sdhci: ============================================ > > [ 2.220881] mmc0: error -5 whilst initialising MMC card > Welcome to Buildroot > buildroot login: [ 2.332786] mmc0: ADMA error > [ 2.335668] mmc0: sdhci: ============ SDHCI REGISTER DUMP =========== > [ 2.342119] mmc0: sdhci: Sys addr: 0x00000000 | Version: 0x00000002 > [ 2.348568] mmc0: sdhci: Blk size: 0x00007200 | Blk cnt: 0x00000001 > [ 2.355018] mmc0: sdhci: Argument: 0x00000000 | Trn mode: 0x00000013 > [ 2.361468] mmc0: sdhci: Present: 0x01ff02f6 | Host ctl: 0x00000019 > [ 2.367917] mmc0: sdhci: Power: 0x0000000b | Blk gap: 0x00000000 > [ 2.374367] mmc0: sdhci: Wake-up: 0x00000000 | Clock: 0x0000f447 > [ 2.380816] mmc0: sdhci: Timeout: 0x0000000b | Int stat: 0x00000001 > [ 2.387267] mmc0: sdhci: Int enab: 0x03ff008b | Sig enab: 0x03ff008b > [ 2.393716] mmc0: sdhci: ACmd stat: 0x00000000 | Slot int: 0x00000001 > [ 2.400166] mmc0: sdhci: Caps: 0x546ec800 | Caps_1: 0x00000000 > [ 2.406615] mmc0: sdhci: Cmd: 0x0000083a | Max curr: 0x00000000 > [ 2.413065] mmc0: sdhci: Resp[0]: 0x00000900 | Resp[1]: 0xffffffff > [ 2.419515] mmc0: sdhci: Resp[2]: 0x320f5903 | Resp[3]: 0x3fd05e00 > [ 2.425963] mmc0: sdhci: Host ctl2: 0x00000000 > [ 2.430412] mmc0: sdhci: ADMA Err: 0x00000001 | ADMA Ptr: 0x000000013965b200 > [ 2.437557] mmc0: sdhci: ============================================ > [ 2.444031] mmc0: error -5 whilst initialising MMC card > [ 2.572203] mmc0: ADMA error > [ 2.575089] mmc0: sdhci: ============ SDHCI REGISTER DUMP =========== > [ 2.581540] mmc0: sdhci: Sys addr: 0x00000000 | Version: 0x00000002 > [ 2.587989] mmc0: sdhci: Blk size: 0x00007200 | Blk cnt: 0x00000001 > [ 2.594439] mmc0: sdhci: Argument: 0x00000000 | Trn mode: 0x00000013 > [ 2.600889] mmc0: sdhci: Present: 0x01ef02f6 | Host ctl: 0x00000019 > [ 2.607339] mmc0: sdhci: Power: 0x0000000b | Blk gap: 0x00000000 > [ 2.613788] mmc0: sdhci: Wake-up: 0x00000000 | Clock: 0x0000e8c7 > [ 2.620237] mmc0: sdhci: Timeout: 0x0000000b | Int stat: 0x00000001 > [ 2.626686] mmc0: sdhci: Int enab: 0x03ff008b | Sig enab: 0x03ff008b > [ 2.633137] mmc0: sdhci: ACmd stat: 0x00000000 | Slot int: 0x00000001 > [ 2.639586] mmc0: sdhci: Caps: 0x546ec800 | Caps_1: 0x00000000 > [ 2.646036] mmc0: sdhci: Cmd: 0x0000083a | Max curr: 0x00000000 > [ 2.652485] mmc0: sdhci: Resp[0]: 0x00000900 | Resp[1]: 0xffffffff > [ 2.658936] mmc0: sdhci: Resp[2]: 0x320f5903 | Resp[3]: 0x3fd05e00 > [ 2.665384] mmc0: sdhci: Host ctl2: 0x00000000 > [ 2.669832] mmc0: sdhci: ADMA Err: 0x00000001 | ADMA Ptr: 0x000000013965b200 > [ 2.676979] mmc0: sdhci: ============================================ > [ 2.683450] mmc0: error -5 whilst initialising MMC card > > CTRL-A Z for help | 115200 8N1 | NOR | Minicom 2.7.1 | VT102 | Offline > | ttyUSB0 > > Reverting this commit fixed the problem. We are having another problem with the new API and Christoph submitted a patch at: https://lkml.org/lkml/2019/8/20/86 Would it be possible for you to test to see if it can fix? We can revert my fallback change after all, if Christoph's patch doesn't work for you either. Thanks Nicolin _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH v2 2/2] dma-contiguous: Use fallback alloc_pages for single pages 2019-08-23 22:11 ` Nicolin Chen @ 2019-08-26 1:56 ` Masahiro Yamada 0 siblings, 0 replies; 22+ messages in thread From: Masahiro Yamada @ 2019-08-26 1:56 UTC (permalink / raw) To: Nicolin Chen Cc: Tony Lindgren, Catalin Marinas, Will Deacon, Max Filippov, Christoph Hellwig, Marek Szyprowski, Stephen Rothwell, Joerg Roedel, Russell King, Thierry Reding, linux-xtensa, Kees Cook, Andrew Morton, linux-arm-kernel, Chris Zankel, Wolfram Sang, Robin Murphy, Linux Kernel Mailing List, iommu, iamjoonsoo.kim, David Woodhouse Hi Nicolin, On Sat, Aug 24, 2019 at 7:10 AM Nicolin Chen <nicoleotsuka@gmail.com> wrote: > > On Fri, Aug 23, 2019 at 09:49:46PM +0900, Masahiro Yamada wrote: > > > > Reverting this commit fixed the problem. > > We are having another problem with the new API and Christoph > submitted a patch at: https://lkml.org/lkml/2019/8/20/86 > > Would it be possible for you to test to see if it can fix? It is included in 5.3-rc6 I tested 5.3-rc6 in on my board, but I still see the same DMA fauilure. Masahiro > We can revert my fallback change after all, if Christoph's > patch doesn't work for you either. > > Thanks > Nicolin -- Best Regards Masahiro Yamada _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH v2 0/2] Optimize dma_*_from_contiguous calls 2019-05-06 22:33 [PATCH v2 0/2] Optimize dma_*_from_contiguous calls Nicolin Chen 2019-05-06 22:33 ` [PATCH v2 1/2] dma-contiguous: Abstract dma_{alloc, free}_contiguous() Nicolin Chen 2019-05-06 22:33 ` [PATCH v2 2/2] dma-contiguous: Use fallback alloc_pages for single pages Nicolin Chen @ 2019-05-08 12:52 ` Christoph Hellwig 2019-05-08 18:08 ` Nicolin Chen 2 siblings, 1 reply; 22+ messages in thread From: Christoph Hellwig @ 2019-05-08 12:52 UTC (permalink / raw) To: Nicolin Chen Cc: tony, catalin.marinas, will.deacon, jcmvbkbc, hch, m.szyprowski, sfr, joro, linux, treding, linux-xtensa, keescook, akpm, linux-arm-kernel, chris, wsa+renesas, robin.murphy, linux-kernel, iommu, iamjoonsoo.kim, dwmw2 Hi Nicolin, modulo a trivial comment typo I found this looks fine to me. I plan to apply it with that fixed up around -rc2 time when I open the dma mapping tree opens for the the 5.3 merge window, unless someone finds an issue until then. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH v2 0/2] Optimize dma_*_from_contiguous calls 2019-05-08 12:52 ` [PATCH v2 0/2] Optimize dma_*_from_contiguous calls Christoph Hellwig @ 2019-05-08 18:08 ` Nicolin Chen 0 siblings, 0 replies; 22+ messages in thread From: Nicolin Chen @ 2019-05-08 18:08 UTC (permalink / raw) To: Christoph Hellwig Cc: chris, linux-xtensa, keescook, sfr, tony, catalin.marinas, joro, will.deacon, linux, iommu, linux-kernel, jcmvbkbc, wsa+renesas, iamjoonsoo.kim, akpm, treding, robin.murphy, dwmw2, linux-arm-kernel, m.szyprowski On Wed, May 08, 2019 at 02:52:54PM +0200, Christoph Hellwig wrote: > modulo a trivial comment typo I found this looks fine to me. I plan > to apply it with that fixed up around -rc2 time when I open the > dma mapping tree opens for the the 5.3 merge window, unless someone > finds an issue until then. Thanks for the help all the way. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 22+ messages in thread
end of thread, other threads:[~2019-08-29 11:49 UTC | newest] Thread overview: 22+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2019-05-06 22:33 [PATCH v2 0/2] Optimize dma_*_from_contiguous calls Nicolin Chen 2019-05-06 22:33 ` [PATCH v2 1/2] dma-contiguous: Abstract dma_{alloc, free}_contiguous() Nicolin Chen 2019-05-24 1:52 ` dann frazier 2019-05-24 2:59 ` dann frazier 2019-05-24 3:49 ` Nicolin Chen 2019-05-06 22:33 ` [PATCH v2 2/2] dma-contiguous: Use fallback alloc_pages for single pages Nicolin Chen 2019-08-23 12:49 ` Masahiro Yamada 2019-08-23 12:56 ` Masahiro Yamada 2019-08-25 1:10 ` Christoph Hellwig 2019-08-26 2:05 ` Masahiro Yamada 2019-08-26 7:33 ` Christoph Hellwig 2019-08-27 7:45 ` Masahiro Yamada 2019-08-27 7:50 ` Christoph Hellwig 2019-08-27 9:03 ` Masahiro Yamada 2019-08-27 11:55 ` Christoph Hellwig 2019-08-28 10:53 ` Masahiro Yamada 2019-08-28 12:23 ` Masahiro Yamada 2019-08-29 11:45 ` Masahiro Yamada 2019-08-23 22:11 ` Nicolin Chen 2019-08-26 1:56 ` Masahiro Yamada 2019-05-08 12:52 ` [PATCH v2 0/2] Optimize dma_*_from_contiguous calls Christoph Hellwig 2019-05-08 18:08 ` Nicolin Chen
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).