From: Laura Abbott <labbott-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
To: Robin Murphy <robin.murphy-5wv7dgnIgG8@public.gmane.org>,
Yong Wu <yong.wu-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>,
labbott-rxtnV0ftBwyoClj4AeEUq9i2O/JbrIOy@public.gmane.org
Cc: laurent.pinchart+renesas-ryLnwIuWjnjg/C1BVhZhaw@public.gmane.org,
catalin.marinas-5wv7dgnIgG8@public.gmane.org,
will.deacon-5wv7dgnIgG8@public.gmane.org,
tiffany.lin-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org,
Tomasz Figa <tfiga-F7+t8E8rja9g9hUCZPvPmw@public.gmane.org>,
iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org,
djkurtz-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org,
thunder.leizhen-hv44wF8Li93QT0dZR+AlfA@public.gmane.org,
yingjoe.chen-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org,
treding-DDmLM1+adcrQT0dZR+AlfA@public.gmane.org,
linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org
Subject: Re: [PATCH v6 2/3] arm64: Add IOMMU dma_ops
Date: Wed, 4 Nov 2015 09:35:26 -0800 [thread overview]
Message-ID: <563A41DE.2010902@redhat.com> (raw)
In-Reply-To: <563A0419.9070100-5wv7dgnIgG8@public.gmane.org>
On 11/04/2015 05:11 AM, Robin Murphy wrote:
> On 04/11/15 08:39, Yong Wu wrote:
>> On Thu, 2015-10-01 at 20:13 +0100, Robin Murphy wrote:
>>> Taking some inspiration from the arch/arm code, implement the
>>> arch-specific side of the DMA mapping ops using the new IOMMU-DMA layer.
>> [...]
>>> +static void *__iommu_alloc_attrs(struct device *dev, size_t size,
>>> + dma_addr_t *handle, gfp_t gfp,
>>> + struct dma_attrs *attrs)
>>> +{
>>> + bool coherent = is_device_dma_coherent(dev);
>>> + int ioprot = dma_direction_to_prot(DMA_BIDIRECTIONAL, coherent);
>>> + void *addr;
>>> +
>>> + if (WARN(!dev, "cannot create IOMMU mapping for unknown device\n"))
>>> + return NULL;
>>> + /*
>>> + * Some drivers rely on this, and we probably don't want the
>>> + * possibility of stale kernel data being read by devices anyway.
>>> + */
>>> + gfp |= __GFP_ZERO;
>>> +
>>> + if (gfp & __GFP_WAIT) {
>>> + struct page **pages;
>>> + pgprot_t prot = __get_dma_pgprot(attrs, PAGE_KERNEL, coherent);
>>> +
>>> + pages = iommu_dma_alloc(dev, size, gfp, ioprot, handle,
>>> + flush_page);
>>> + if (!pages)
>>> + return NULL;
>>> +
>>> + addr = dma_common_pages_remap(pages, size, VM_USERMAP, prot,
>>> + __builtin_return_address(0));
>>> + if (!addr)
>>> + iommu_dma_free(dev, pages, size, handle);
>>> + } else {
>>> + struct page *page;
>>> + /*
>>> + * In atomic context we can't remap anything, so we'll only
>>> + * get the virtually contiguous buffer we need by way of a
>>> + * physically contiguous allocation.
>>> + */
>>> + if (coherent) {
>>> + page = alloc_pages(gfp, get_order(size));
>>> + addr = page ? page_address(page) : NULL;
>>> + } else {
>>> + addr = __alloc_from_pool(size, &page, gfp);
>>> + }
>>> + if (!addr)
>>> + return NULL;
>>> +
>>> + *handle = iommu_dma_map_page(dev, page, 0, size, ioprot);
>>> + if (iommu_dma_mapping_error(dev, *handle)) {
>>> + if (coherent)
>>> + __free_pages(page, get_order(size));
>>> + else
>>> + __free_from_pool(addr, size);
>>> + addr = NULL;
>>> + }
>>> + }
>>> + return addr;
>>> +}
>>> +
>>> +static void __iommu_free_attrs(struct device *dev, size_t size, void *cpu_addr,
>>> + dma_addr_t handle, struct dma_attrs *attrs)
>>> +{
>>> + /*
>>> + * @cpu_addr will be one of 3 things depending on how it was allocated:
>>> + * - A remapped array of pages from iommu_dma_alloc(), for all
>>> + * non-atomic allocations.
>>> + * - A non-cacheable alias from the atomic pool, for atomic
>>> + * allocations by non-coherent devices.
>>> + * - A normal lowmem address, for atomic allocations by
>>> + * coherent devices.
>>> + * Hence how dodgy the below logic looks...
>>> + */
>>> + if (__in_atomic_pool(cpu_addr, size)) {
>>> + iommu_dma_unmap_page(dev, handle, size, 0, NULL);
>>> + __free_from_pool(cpu_addr, size);
>>> + } else if (is_vmalloc_addr(cpu_addr)){
>>> + struct vm_struct *area = find_vm_area(cpu_addr);
>>> +
>>> + if (WARN_ON(!area || !area->pages))
>>> + return;
>>> + iommu_dma_free(dev, area->pages, size, &handle);
>>> + dma_common_free_remap(cpu_addr, size, VM_USERMAP);
>>
>> Hi Robin,
>> We get a WARN issue while the size is not aligned here.
>>
>> The WARN log is:
>> [ 206.852002] WARNING: CPU: 0 PID: 23329
>> at /mnt/host/source/src/third_party/kernel/v3.18/mm/vmalloc.c:65
>> vunmap_page_range+0x190/0x1b4()
>> [ 206.864438] Modules linked in: nls_iso8859_1 nls_cp437 vfat fat
>> rfcomm i2c_dev uinput dm9601 uvcvideo btmrvl_sdio mwifiex_sdio mwifiex
>> btmrvl bluetooth zram fuse cfg80211 nf_conntrack_ipv6 nf_defrag_ipv6
>> ip6table_filter ip6_tables cdc_ether usbnet mii joydev snd_seq_midi
>> snd_seq_midi_event snd_rawmidi snd_seq snd_seq_device ppp_async
>> ppp_generic slhc tun
>> [ 206.902983] CPU: 0 PID: 23329 Comm: chrome Not tainted 3.18.0 #17
>> [ 206.910430] Hardware name: Mediatek Oak rev3 board (DT)
>> [ 206.920018] Call trace:
>> [ 206.925537] [<ffffffc000208c00>] dump_backtrace+0x0/0x140
>> [ 206.931905] [<ffffffc000208d5c>] show_stack+0x1c/0x28
>> [ 206.939158] [<ffffffc000870f80>] dump_stack+0x74/0x94
>> [ 206.947459] [<ffffffc0002219a4>] warn_slowpath_common+0x90/0xb8
>> [ 206.954100] [<ffffffc000221b58>] warn_slowpath_null+0x34/0x44
>> [ 206.961537] [<ffffffc000321358>] vunmap_page_range+0x18c/0x1b4
>> [ 206.967630] [<ffffffc0003213e4>] unmap_kernel_range+0x2c/0x78
>> [ 206.976977] [<ffffffc000582224>] dma_common_free_remap+0x68/0x80
>> [ 206.983581] [<ffffffc000217260>] __iommu_free_attrs+0x14c/0x160
>> [ 206.989646] [<ffffffc00066fc1c>] mtk_vcodec_mem_free+0xa0/0x15c
>> [ 206.996481] [<ffffffc00067e278>] vp9_free_work_buf+0x54/0x70
>> [ 207.002260] [<ffffffc00067f168>] vdec_vp9_deinit+0x7c/0xe8
>> [ 207.008134] [<ffffffc0006787d8>] vdec_if_deinit+0x84/0xec
>> [ 207.013820] [<ffffffc000677898>] mtk_vcodec_vdec_release+0x54/0x6c
>> [ 207.020672] [<ffffffc000673e3c>] fops_vcodec_release+0x7c/0xf8
>> [ 207.026607] [<ffffffc000652b78>] v4l2_release+0x3c/0x84
>> [ 207.031824] [<ffffffc00033b218>] __fput+0xf8/0x1c0
>> [ 207.036599] [<ffffffc00033b350>] ____fput+0x1c/0x2c
>> [ 207.041454] [<ffffffc00023ed78>] task_work_run+0xb0/0xd4
>> [ 207.046756] [<ffffffc00020872c>] do_notify_resume+0x54/0x6c
>>
>>
>> From the log I get in this fail case, the size of unmap here is
>> 0x10080, and its map size of dma_common_pages_remap in
>> __iommu_alloc_attrs is 0x10080, and the corresponding dma-map size is
>> 0x11000(after iova_align). I think all the parameters of map and unmap
>> are good, it look like not a DMA issue. but I don't know why we get this
>> warning.
>> Have you met this problem and give us some advices, Thanks.
>>
>> (If we add PAGE_ALIGN for the size in dma_alloc and dma_free, It is OK.)
>
> OK, having dug into this it looks like the root cause comes from some asymmetry in the common code: dma_common_pages remap() just passes the size through to get_vm_area_caller(), and the first thing that does is to page-align it. On the other hand, neither dma_common_free_remap() nor unmap_kernel_range() does anything with the size, so we wind up giving an unaligned end address to vunmap_page_range() and messing up the vmalloc page tables.
>
> I wonder if dma_common_free_remap() should be page-aligning the size to match expectations (i.e. make it correctly unmap any request the other functions happily mapped), or conversely, perhaps both the map and unmap functions should have a WARN_ON(size & PAGE_MASK) to enforce being called as actually intended. Laura?
>
Based on what I've found, the DMA mapping API needs to be able to handle unaligned sizes
gracefully so I don't think a warn is appropriate. I was aligning at the higher level but
it would be best for dma_common_free_remap to align as well.
> Either way, I'll send out a patch to make the arm64 side deal with it explicitly.
>
> Robin.
>
Thanks,
Laura
next prev parent reply other threads:[~2015-11-04 17:35 UTC|newest]
Thread overview: 38+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-10-01 19:13 [PATCH v6 0/3] arm64: IOMMU-backed DMA mapping Robin Murphy
[not found] ` <cover.1443718557.git.robin.murphy-5wv7dgnIgG8@public.gmane.org>
2015-10-01 19:13 ` [PATCH v6 1/3] iommu: Implement common IOMMU ops for " Robin Murphy
[not found] ` <ab8e1caa40d6da1afa4a49f30242ef4e6e1f17df.1443718557.git.robin.murphy-5wv7dgnIgG8@public.gmane.org>
2015-10-26 13:44 ` Yong Wu
2015-10-26 16:55 ` Robin Murphy
2015-10-30 1:17 ` Daniel Kurtz
2015-10-30 14:09 ` Joerg Roedel
[not found] ` <20151030140923.GJ27420-zLv9SwRftAIdnm+yROfE0A@public.gmane.org>
2015-10-30 18:18 ` Mark Hounschell
2015-10-30 14:27 ` Robin Murphy
2015-11-02 13:11 ` Daniel Kurtz
2015-11-02 13:43 ` Tomasz Figa
2015-11-03 17:41 ` Robin Murphy
2015-11-03 18:40 ` Russell King - ARM Linux
2015-11-04 5:15 ` Tomasz Figa
[not found] ` <CAAFQd5COY-dvBE73R=sUWoGfXR9CvgurGchYgXB6y9eqQ=BBUQ-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2015-11-04 9:10 ` Russell King - ARM Linux
2015-11-04 5:12 ` Tomasz Figa
[not found] ` <CAAFQd5A4TcvkDMFezqEpkfWL+7yO2v=Hm=twk=p-NpADPpvqEQ-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2015-11-04 9:27 ` Russell King - ARM Linux
2015-11-04 9:48 ` Tomasz Figa
[not found] ` <CAAFQd5ApSFC6Pm4tDhZbJOVZ7szCx=diKUtGXq=M9a5Y_4qzOQ-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2015-11-04 10:50 ` Russell King - ARM Linux
2015-11-09 13:11 ` Robin Murphy
2015-11-17 12:02 ` Marek Szyprowski
2015-10-01 19:13 ` [PATCH v6 2/3] arm64: Add IOMMU dma_ops Robin Murphy
2015-10-07 9:03 ` Anup Patel
[not found] ` <CAAhSdy2tpAfH+i=1axDkmRqZixsbVhd-_9VGvpyQ=5e06v=Kpg-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2015-10-07 16:36 ` Robin Murphy
2015-10-07 17:40 ` Anup Patel
[not found] ` <80cb035144a2648a5d94eb1fec3336f17ad249f1.1443718557.git.robin.murphy-5wv7dgnIgG8@public.gmane.org>
2015-10-06 11:00 ` Yong Wu
2015-10-07 16:07 ` Robin Murphy
[not found] ` <56154349.8040101-5wv7dgnIgG8@public.gmane.org>
2015-10-09 5:44 ` Yong Wu
2015-10-14 11:47 ` Joerg Roedel
2015-10-14 13:35 ` Catalin Marinas
[not found] ` <20151014133538.GG4239-M2fw3Uu6cmfZROr8t4l/smS4ubULX0JqMm0uRHvK7Nw@public.gmane.org>
2015-10-14 16:34 ` Robin Murphy
2015-11-04 8:39 ` Yong Wu
2015-11-04 13:11 ` Robin Murphy
[not found] ` <563A0419.9070100-5wv7dgnIgG8@public.gmane.org>
2015-11-04 17:35 ` Laura Abbott [this message]
2015-10-01 19:14 ` [PATCH v6 3/3] arm64: Hook up " Robin Murphy
2015-10-13 12:12 ` [PATCH v6 0/3] arm64: IOMMU-backed DMA mapping Robin Murphy
[not found] ` <561CF53E.7000809-5wv7dgnIgG8@public.gmane.org>
2015-10-14 11:50 ` joro-zLv9SwRftAIdnm+yROfE0A
[not found] ` <20151014115013.GM27420-zLv9SwRftAIdnm+yROfE0A@public.gmane.org>
2015-10-14 18:19 ` Robin Murphy
2015-10-15 15:04 ` Joerg Roedel
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=563A41DE.2010902@redhat.com \
--to=labbott-h+wxahxf7alqt0dzr+alfa@public.gmane.org \
--cc=catalin.marinas-5wv7dgnIgG8@public.gmane.org \
--cc=djkurtz-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org \
--cc=iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org \
--cc=labbott-rxtnV0ftBwyoClj4AeEUq9i2O/JbrIOy@public.gmane.org \
--cc=laurent.pinchart+renesas-ryLnwIuWjnjg/C1BVhZhaw@public.gmane.org \
--cc=linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org \
--cc=robin.murphy-5wv7dgnIgG8@public.gmane.org \
--cc=tfiga-F7+t8E8rja9g9hUCZPvPmw@public.gmane.org \
--cc=thunder.leizhen-hv44wF8Li93QT0dZR+AlfA@public.gmane.org \
--cc=tiffany.lin-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org \
--cc=treding-DDmLM1+adcrQT0dZR+AlfA@public.gmane.org \
--cc=will.deacon-5wv7dgnIgG8@public.gmane.org \
--cc=yingjoe.chen-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org \
--cc=yong.wu-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).