From: Alexey Brodkin <Alexey.Brodkin@synopsys.com>
To: "hch@lst.de" <hch@lst.de>
Cc: "deanbo422@gmail.com" <deanbo422@gmail.com>,
"linux-sh@vger.kernel.org" <linux-sh@vger.kernel.org>,
"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
"nios2-dev@lists.rocketboards.org"
<nios2-dev@lists.rocketboards.org>,
"linux-xtensa@linux-xtensa.org" <linux-xtensa@linux-xtensa.org>,
"linux-m68k@lists.linux-m68k.org"
<linux-m68k@lists.linux-m68k.org>,
"linux-alpha@vger.kernel.org" <linux-alpha@vger.kernel.org>,
"linux-hexagon@vger.kernel.org" <linux-hexagon@vger.kernel.org>,
"linux-snps-arc@lists.infradead.org"
<linux-snps-arc@lists.infradead.org>,
"iommu@lists.linux-foundation.org"
<iommu@lists.linux-foundation.org>,
"green.hu@gmail.com" <green.hu@gmail.com>,
"openrisc@lists.librecores.org" <openrisc@lists.librecores.org>,
"linux-arm-kernel@lists.infradead.org"
<linux-arm-kernel@lists.infradead.org>,
"monstr@monstr.eu" <monstr@monstr.eu>,
"linux-parisc@vger.kernel.org" <linux-parisc@vger.kernel.org>,
"linux-c6x-dev@linux-c6x.org" <linux-c6x-dev@linux-c6x.org>,
"linux-arch@vger.kernel.org" <linux-arch@vger.kernel.org>,
"sparclinux@vger.kernel.org" <sparclinux@vger.kernel.org>
Subject: Re: [PATCH 02/20] dma-mapping: provide a generic dma-noncoherent implementation
Date: Fri, 18 May 2018 13:03:46 +0000 [thread overview]
Message-ID: <bad125dff49f6e49c895e818c9d1abb346a46e8e.camel@synopsys.com> (raw)
Message-ID: <20180518130346.gfArSRWn5-yvoxLeE1AwB1H3urwCaqv6SExrNNraMY8@z> (raw)
In-Reply-To: <20180511075945.16548-3-hch@lst.de>
Hi Christoph,
On Fri, 2018-05-11 at 09:59 +0200, Christoph Hellwig wrote:
[snip]
There seems to be one subtle issue with map/unmap code.
While investigating problems on ARC I added instrumentation as below:
---------------------------------------->8------------------------------------
--- a/arch/arc/mm/dma.c
+++ b/arch/arc/mm/dma.c
@@ -152,14 +152,37 @@ static void _dma_cache_sync(struct device *dev, phys_addr_t paddr, size_t size,
}
}
+static const char *dir_to_str(enum dma_data_direction dir)
+{
+ switch (dir) {
+ case DMA_BIDIRECTIONAL: return "DMA_BIDIRECTIONAL";
+ case DMA_TO_DEVICE: return "DMA_TO_DEVICE";
+ case DMA_FROM_DEVICE: return "DMA_FROM_DEVICE";
+ case DMA_NONE: return "DMA_NONE";
+ default: return "WRONG_VALUE!";
+ }
+}
+
void arch_sync_dma_for_device(struct device *dev, phys_addr_t paddr,
size_t size, enum dma_data_direction dir)
{
+ if (dir != DMA_TO_DEVICE){
+ dump_stack();
+ printk(" *** %s@%d: DMA direction is %s instead of %s\n",
+ __func__, __LINE__, dir_to_str(dir), dir_to_str(DMA_TO_DEVICE));
+ }
+
return _dma_cache_sync(dev, paddr, size, dir);
}
void arch_sync_dma_for_cpu(struct device *dev, phys_addr_t paddr,
size_t size, enum dma_data_direction dir)
{
+ if (dir != DMA_FROM_DEVICE) {
+ dump_stack();
+ printk(" *** %s@%d: DMA direction is %s instead of %s\n",
+ __func__, __LINE__, dir_to_str(dir), dir_to_str(DMA_FROM_DEVICE));
+ }
+
return _dma_cache_sync(dev, paddr, size, dir);
}
---------------------------------------->8------------------------------------
And with that I noticed a bit unexpected output, see below:
---------------------------------------->8------------------------------------
Stack Trace:
arc_unwind_core.constprop.1+0xd4/0xf8
dump_stack+0x68/0x80
arch_sync_dma_for_device+0x34/0xc4
dma_noncoherent_map_sg+0x80/0x94
__dw_mci_start_request+0x1ee/0x868
dw_mci_request+0x17e/0x1c8
mmc_wait_for_req+0x106/0x1ac
mmc_app_sd_status+0x108/0x130
mmc_sd_setup_card+0xc6/0x2e8
mmc_attach_sd+0x1b6/0x394
mmc_rescan+0x2f4/0x3bc
process_one_work+0x194/0x348
worker_thread+0xf2/0x478
kthread+0x120/0x13c
ret_from_fork+0x18/0x1c
*** arch_sync_dma_for_device@172: DMA direction is DMA_FROM_DEVICE instead of DMA_TO_DEVICE
...
Stack Trace:
arc_unwind_core.constprop.1+0xd4/0xf8
dump_stack+0x68/0x80
arch_sync_dma_for_device+0x34/0xc4
dma_noncoherent_map_page+0x86/0x8c
usb_hcd_map_urb_for_dma+0x49e/0x53c
usb_hcd_submit_urb+0x43c/0x8c4
usb_control_msg+0xbe/0x16c
hub_port_init+0x5e0/0xb0c
hub_event+0x4e6/0x1164
process_one_work+0x194/0x348
worker_thread+0xf2/0x478
kthread+0x120/0x13c
ret_from_fork+0x18/0x1c
mmcblk0: p1 p2
*** arch_sync_dma_for_device@172: DMA direction is DMA_FROM_DEVICE instead of DMA_TO_DEVICE
...
and quite some more of the similar
...
---------------------------------------->8------------------------------------
In case of MMC/DW_MCI (AKA DesignWare MobileStorage controller) that's an execution flow:
1) __dw_mci_start_request()
2) dw_mci_pre_dma_transfer()
3) dma_map_sg(..., mmc_get_dma_dir(data))
Note mmc_get_dma_dir() is just "data->flags & MMC_DATA_WRITE ? DMA_TO_DEVICE : DMA_FROM_DEVICE".
I.e. if we're preparing for sending data dma_noncoherent_map_sg() will have DMA_TO_DEVICE which
is quite OK for passing to dma_noncoherent_sync_sg_for_device() but in case of reading we'll have
DMA_FROM_DEVICE which we'll pass to dma_noncoherent_sync_sg_for_device() in dma_noncoherent_map_sg().
I'd say this is not entirely correct because IMHO arch_sync_dma_for_cpu() is supposed to only be used
in case of DMA_FROM_DEVICE and arch_sync_dma_for_device() only in case of DMA_TO_DEVICE.
> +static dma_addr_t dma_noncoherent_map_page(struct device *dev, struct page *page,
> + unsigned long offset, size_t size, enum dma_data_direction dir,
> + unsigned long attrs)
> +{
> + dma_addr_t addr;
> +
> + addr = dma_direct_map_page(dev, page, offset, size, dir, attrs);
> + if (!dma_mapping_error(dev, addr) && !(attrs & DMA_ATTR_SKIP_CPU_SYNC))
> + arch_sync_dma_for_device(dev, page_to_phys(page), size, dir);
> + return addr;
> +}
> +
> +static int dma_noncoherent_map_sg(struct device *dev, struct scatterlist *sgl,
> + int nents, enum dma_data_direction dir, unsigned long attrs)
> +{
> + nents = dma_direct_map_sg(dev, sgl, nents, dir, attrs);
> + if (nents > 0 && !(attrs & DMA_ATTR_SKIP_CPU_SYNC))
> + dma_noncoherent_sync_sg_for_device(dev, sgl, nents, dir);
> + return nents;
> +}
The same is for unmap functions.
My guess is we need to respect direction in map/unmap functions and use
either dma_noncoherent_sync_single_for_cpu(..., DMA_FROM_DEVICE) or
dma_noncoherent_sync_single_for_device(...,DMA_TO_DEVICE).
> +static void dma_noncoherent_unmap_page(struct device *dev, dma_addr_t addr,
> + size_t size, enum dma_data_direction dir, unsigned long attrs)
> +{
> + if (!(attrs & DMA_ATTR_SKIP_CPU_SYNC))
> + dma_noncoherent_sync_single_for_cpu(dev, addr, size, dir);
> +}
> +
> +static void dma_noncoherent_unmap_sg(struct device *dev, struct scatterlist *sgl,
> + int nents, enum dma_data_direction dir, unsigned long attrs)
> +{
> + if (!(attrs & DMA_ATTR_SKIP_CPU_SYNC))
> + dma_noncoherent_sync_sg_for_cpu(dev, sgl, nents, dir);
> +}
> +#endif
But the real fix of my problem is:
---------------------------------------->8------------------------------------
--- a/lib/dma-noncoherent.c
+++ b/lib/dma-noncoherent.c
@@ -35,7 +35,7 @@ static dma_addr_t dma_noncoherent_map_page(struct device *dev, struct page *page
addr = dma_direct_map_page(dev, page, offset, size, dir, attrs);
if (!dma_mapping_error(dev, addr) && !(attrs & DMA_ATTR_SKIP_CPU_SYNC))
- arch_sync_dma_for_device(dev, page_to_phys(page), size, dir);
+ arch_sync_dma_for_device(dev, page_to_phys(page) + offset, size, dir);
return addr;
}
---------------------------------------->8------------------------------------
You seem to lost an offset in the page so if we happen to have a buffer not aligned to
a page boundary then we were obviously corrupting data outside our data :)
-Alexey
next prev parent reply other threads:[~2018-05-18 13:03 UTC|newest]
Thread overview: 78+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-05-11 7:59 common non-cache coherent direct dma mapping ops Christoph Hellwig
2018-05-11 7:59 ` Christoph Hellwig
2018-05-11 7:59 ` [PATCH 01/20] dma-mapping: simplify Kconfig dependencies Christoph Hellwig
2018-05-11 7:59 ` Christoph Hellwig
[not found] ` <20180511075945.16548-1-hch-jcswGhMUV9g@public.gmane.org>
2018-05-11 7:59 ` [PATCH 02/20] dma-mapping: provide a generic dma-noncoherent implementation Christoph Hellwig
2018-05-11 7:59 ` Christoph Hellwig
[not found] ` <20180511075945.16548-3-hch-jcswGhMUV9g@public.gmane.org>
2018-05-18 13:03 ` Alexey Brodkin [this message]
2018-05-18 13:03 ` Alexey Brodkin
[not found] ` <bad125dff49f6e49c895e818c9d1abb346a46e8e.camel-HKixBCOQz3hWk0Htik3J/w@public.gmane.org>
2018-05-18 13:27 ` hch-jcswGhMUV9g
2018-05-18 13:27 ` hch
[not found] ` <20180518132731.GA31125-jcswGhMUV9g@public.gmane.org>
2018-05-18 14:13 ` Alexey Brodkin
2018-05-18 14:13 ` Alexey Brodkin
2018-05-18 17:28 ` Vineet Gupta
2018-05-18 17:28 ` Vineet Gupta
2018-05-18 17:20 ` dma_sync_*_for_cpu and direction=TO_DEVICE (was Re: [PATCH 02/20] dma-mapping: provide a generic dma-noncoherent implementation) Vineet Gupta
2018-05-18 17:20 ` Vineet Gupta
2018-05-18 17:50 ` Russell King - ARM Linux
2018-05-18 17:50 ` Russell King - ARM Linux
[not found] ` <20180518175004.GF17671-l+eeeJia6m9URfEZ8mYm6t73F7V6hmMc@public.gmane.org>
2018-05-18 19:57 ` Alexey Brodkin
2018-05-18 19:57 ` Alexey Brodkin
[not found] ` <182840dedb4890a88c672b1c5d556920bf89a8fb.camel-HKixBCOQz3hWk0Htik3J/w@public.gmane.org>
2018-05-18 21:33 ` Russell King - ARM Linux
2018-05-18 21:33 ` Russell King - ARM Linux
2018-05-18 20:35 ` Vineet Gupta
2018-05-18 20:35 ` Vineet Gupta
2018-05-18 21:55 ` Russell King - ARM Linux
2018-05-18 21:55 ` Russell King - ARM Linux
2018-05-18 20:05 ` [PATCH 02/20] dma-mapping: provide a generic dma-noncoherent implementation Helge Deller
2018-05-18 20:05 ` Helge Deller
[not found] ` <0c5d27e9-2799-eb38-8b09-47a04c48b5c7-Mmb7MZpHnFY@public.gmane.org>
2018-05-19 6:38 ` hch-jcswGhMUV9g
2018-05-19 6:38 ` hch
2018-05-11 7:59 ` [PATCH 03/20] arc: use generic dma_noncoherent_ops Christoph Hellwig
2018-05-11 7:59 ` Christoph Hellwig
[not found] ` <20180511075945.16548-4-hch-jcswGhMUV9g@public.gmane.org>
2018-05-11 12:44 ` Alexey Brodkin
2018-05-11 12:44 ` Alexey Brodkin
2018-05-11 7:59 ` [PATCH 04/20] arm-nommu: " Christoph Hellwig
2018-05-11 7:59 ` Christoph Hellwig
[not found] ` <20180511075945.16548-5-hch-jcswGhMUV9g@public.gmane.org>
2018-05-11 9:11 ` Russell King - ARM Linux
2018-05-11 9:11 ` Russell King - ARM Linux
[not found] ` <20180511091114.GA16141-l+eeeJia6m9URfEZ8mYm6t73F7V6hmMc@public.gmane.org>
2018-05-22 11:53 ` Christoph Hellwig
2018-05-22 11:53 ` Christoph Hellwig
2018-05-11 13:56 ` John Garry
2018-05-11 13:56 ` John Garry
2018-05-11 7:59 ` [PATCH 05/20] c6x: " Christoph Hellwig
2018-05-11 7:59 ` Christoph Hellwig
[not found] ` <20180511075945.16548-6-hch-jcswGhMUV9g@public.gmane.org>
2018-05-15 0:25 ` [Linux-c6x-dev] " Mark Salter
2018-05-15 0:25 ` Mark Salter
2018-05-11 7:59 ` [PATCH 06/20] hexagon: " Christoph Hellwig
2018-05-11 7:59 ` Christoph Hellwig
2018-05-11 7:59 ` [PATCH 07/20] m68k: " Christoph Hellwig
2018-05-11 7:59 ` Christoph Hellwig
2018-05-11 7:59 ` [PATCH 08/20] microblaze: " Christoph Hellwig
2018-05-11 7:59 ` Christoph Hellwig
2018-05-11 7:59 ` [PATCH 09/20] microblaze: remove the consistent_sync and consistent_sync_page Christoph Hellwig
2018-05-11 7:59 ` Christoph Hellwig
2018-05-11 7:59 ` [PATCH 10/20] nds32: use generic dma_noncoherent_ops Christoph Hellwig
2018-05-11 7:59 ` Christoph Hellwig
2018-05-11 7:59 ` [PATCH 11/20] nios2: " Christoph Hellwig
2018-05-11 7:59 ` Christoph Hellwig
2018-05-11 7:59 ` [PATCH 12/20] openrisc: " Christoph Hellwig
2018-05-11 7:59 ` Christoph Hellwig
2018-05-11 7:59 ` [PATCH 13/20] sh: simplify get_arch_dma_ops Christoph Hellwig
2018-05-11 7:59 ` Christoph Hellwig
2018-05-11 7:59 ` [PATCH 14/20] sh: introduce a sh_cacheop_vaddr helper Christoph Hellwig
2018-05-11 7:59 ` Christoph Hellwig
2018-05-11 7:59 ` [PATCH 15/20] sh: use dma_direct_ops for the CONFIG_DMA_COHERENT case Christoph Hellwig
2018-05-11 7:59 ` Christoph Hellwig
2018-05-11 7:59 ` [PATCH 16/20] mm: split arch/sh/mm/consistent.c Christoph Hellwig
2018-05-11 7:59 ` Christoph Hellwig
2018-05-11 7:59 ` [PATCH 17/20] sh: use generic dma_noncoherent_ops Christoph Hellwig
2018-05-11 7:59 ` Christoph Hellwig
2018-05-11 7:59 ` [PATCH 18/20] xtensa: " Christoph Hellwig
2018-05-11 7:59 ` Christoph Hellwig
2018-05-11 7:59 ` [PATCH 19/20] sparc: " Christoph Hellwig
2018-05-11 7:59 ` Christoph Hellwig
2018-05-11 7:59 ` [PATCH 20/20] parisc: " Christoph Hellwig
2018-05-11 7:59 ` Christoph Hellwig
2018-05-13 13:26 ` common non-cache coherent direct dma mapping ops Helge Deller
2018-05-13 13:26 ` Helge Deller
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=bad125dff49f6e49c895e818c9d1abb346a46e8e.camel@synopsys.com \
--to=alexey.brodkin@synopsys.com \
--cc=deanbo422@gmail.com \
--cc=green.hu@gmail.com \
--cc=hch@lst.de \
--cc=iommu@lists.linux-foundation.org \
--cc=linux-alpha@vger.kernel.org \
--cc=linux-arch@vger.kernel.org \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-c6x-dev@linux-c6x.org \
--cc=linux-hexagon@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-m68k@lists.linux-m68k.org \
--cc=linux-parisc@vger.kernel.org \
--cc=linux-sh@vger.kernel.org \
--cc=linux-snps-arc@lists.infradead.org \
--cc=linux-xtensa@linux-xtensa.org \
--cc=monstr@monstr.eu \
--cc=nios2-dev@lists.rocketboards.org \
--cc=openrisc@lists.librecores.org \
--cc=sparclinux@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).