From: Russell King - ARM Linux <linux@arm.linux.org.uk>
To: Joerg Roedel <joro@8bytes.org>
Cc: linaro-mm-sig@lists.linaro.org, linux-kernel@vger.kernel.org,
Arnd Bergmann <arnd@arndb.de>,
linux-arm-kernel@lists.infradead.org
Subject: Re: [RFC] ARM DMA mapping TODO, v1
Date: Thu, 28 Apr 2011 13:42:42 +0100 [thread overview]
Message-ID: <20110428124242.GJ17290@n2100.arm.linux.org.uk> (raw)
In-Reply-To: <20110428122508.GC13402@8bytes.org>
On Thu, Apr 28, 2011 at 02:25:09PM +0200, Joerg Roedel wrote:
> On Thu, Apr 28, 2011 at 12:01:29PM +0100, Russell King - ARM Linux wrote:
>
> > dma_addr_t dma_map_page(struct device *dev, struct page *page, size_t offset,
> > size_t size, enum dma_data_direction dir)
> > {
> > struct dma_map_ops *ops = get_dma_ops(dev);
> > dma_addr_t addr;
> >
> > BUG_ON(!valid_dma_direction(dir));
> > if (ops->flags & DMA_MANAGE_CACHE || !dev->dma_cache_coherent)
> > __dma_page_cpu_to_dev(page, offset, size, dir);
> > addr = ops->map_page(dev, page, offset, size, dir, NULL);
> > debug_dma_map_page(dev, page, offset, size, dir, addr, false);
> >
> > return addr;
> > }
> >
> > Things like swiotlb and dmabounce would not set DMA_MANAGE_CACHE in
> > ops->flags, but real iommus and the standard no-iommu implementations
> > would be required to set it to ensure that data is visible in memory
> > for CPUs which have DMA incoherent caches.
>
> Do we need flags for that? A flag is necessary if the cache-management
> differs between IOMMU implementations on the same platform. If
> cache-management is only specific to the platform (or architecture) then
> it does make more sense to just call the function without flag checking
> and every platform with coherent DMA just implements these as static
> inline noops.
Sigh. You're not seeing the point.
There is _no_ point doing the cache management _if_ we're using something
like dmabounce or swiotlb, as we'll be using memcpy() at some point with
the buffer. Moreover, dmabounce or swiotlb may have to do its own cache
management _after_ that memcpy() to ensure that the page cache requirements
are met.
Doing DMA cache management for dmabounce or swiotlb will result in
unnecessary overhead - and as we can see from the MMC discussions,
it has a _significant_ performance impact.
Think about it. If you're using dmabounce, but still do the cache
management:
1. you flush the data out of the CPU cache back to memory.
2. you allocate new memory using dma_alloc_coherent() for the DMA buffer
which is accessible to the device.
3. you memcpy() the data out of the buffer you just flushed into the
DMA buffer - this re-fills the cache, evicting entries which may
otherwise be hot due to the cache fill policy.
Step 1 is entirely unnecessary and is just a complete and utter waste of
CPU resources.
WARNING: multiple messages have this Message-ID (diff)
From: linux@arm.linux.org.uk (Russell King - ARM Linux)
To: linux-arm-kernel@lists.infradead.org
Subject: [RFC] ARM DMA mapping TODO, v1
Date: Thu, 28 Apr 2011 13:42:42 +0100 [thread overview]
Message-ID: <20110428124242.GJ17290@n2100.arm.linux.org.uk> (raw)
In-Reply-To: <20110428122508.GC13402@8bytes.org>
On Thu, Apr 28, 2011 at 02:25:09PM +0200, Joerg Roedel wrote:
> On Thu, Apr 28, 2011 at 12:01:29PM +0100, Russell King - ARM Linux wrote:
>
> > dma_addr_t dma_map_page(struct device *dev, struct page *page, size_t offset,
> > size_t size, enum dma_data_direction dir)
> > {
> > struct dma_map_ops *ops = get_dma_ops(dev);
> > dma_addr_t addr;
> >
> > BUG_ON(!valid_dma_direction(dir));
> > if (ops->flags & DMA_MANAGE_CACHE || !dev->dma_cache_coherent)
> > __dma_page_cpu_to_dev(page, offset, size, dir);
> > addr = ops->map_page(dev, page, offset, size, dir, NULL);
> > debug_dma_map_page(dev, page, offset, size, dir, addr, false);
> >
> > return addr;
> > }
> >
> > Things like swiotlb and dmabounce would not set DMA_MANAGE_CACHE in
> > ops->flags, but real iommus and the standard no-iommu implementations
> > would be required to set it to ensure that data is visible in memory
> > for CPUs which have DMA incoherent caches.
>
> Do we need flags for that? A flag is necessary if the cache-management
> differs between IOMMU implementations on the same platform. If
> cache-management is only specific to the platform (or architecture) then
> it does make more sense to just call the function without flag checking
> and every platform with coherent DMA just implements these as static
> inline noops.
Sigh. You're not seeing the point.
There is _no_ point doing the cache management _if_ we're using something
like dmabounce or swiotlb, as we'll be using memcpy() at some point with
the buffer. Moreover, dmabounce or swiotlb may have to do its own cache
management _after_ that memcpy() to ensure that the page cache requirements
are met.
Doing DMA cache management for dmabounce or swiotlb will result in
unnecessary overhead - and as we can see from the MMC discussions,
it has a _significant_ performance impact.
Think about it. If you're using dmabounce, but still do the cache
management:
1. you flush the data out of the CPU cache back to memory.
2. you allocate new memory using dma_alloc_coherent() for the DMA buffer
which is accessible to the device.
3. you memcpy() the data out of the buffer you just flushed into the
DMA buffer - this re-fills the cache, evicting entries which may
otherwise be hot due to the cache fill policy.
Step 1 is entirely unnecessary and is just a complete and utter waste of
CPU resources.
next prev parent reply other threads:[~2011-04-28 12:42 UTC|newest]
Thread overview: 196+ messages / expand[flat|nested] mbox.gz Atom feed top
2011-04-21 19:29 [RFC] ARM DMA mapping TODO, v1 Arnd Bergmann
2011-04-21 19:29 ` Arnd Bergmann
2011-04-21 20:09 ` [Linaro-mm-sig] " Jesse Barnes
2011-04-21 20:09 ` Jesse Barnes
2011-04-21 21:52 ` Zach Pfeffer
2011-04-21 21:52 ` Zach Pfeffer
2011-04-22 0:34 ` KyongHo Cho
2011-04-22 0:34 ` KyongHo Cho
2011-04-26 14:29 ` Arnd Bergmann
2011-04-26 14:29 ` Arnd Bergmann
2011-04-26 14:28 ` Arnd Bergmann
2011-04-26 14:28 ` Arnd Bergmann
2011-04-26 14:26 ` Arnd Bergmann
2011-04-26 14:26 ` Arnd Bergmann
2011-04-26 15:39 ` Jesse Barnes
2011-04-26 15:39 ` Jesse Barnes
2011-04-27 7:35 ` Russell King - ARM Linux
2011-04-27 7:35 ` Russell King - ARM Linux
2011-04-27 8:56 ` Arnd Bergmann
2011-04-27 8:56 ` Arnd Bergmann
2011-04-27 9:09 ` Russell King - ARM Linux
2011-04-27 9:09 ` Russell King - ARM Linux
2011-04-27 11:02 ` Arnd Bergmann
2011-04-27 11:02 ` Arnd Bergmann
2011-04-27 16:16 ` [Linaro-mm-sig] " Alex Deucher
2011-04-27 16:16 ` Alex Deucher
2011-04-27 17:44 ` Anca Emanuel
2011-04-27 17:44 ` Anca Emanuel
2011-04-27 20:27 ` Russell King - ARM Linux
2011-04-27 20:27 ` Russell King - ARM Linux
2011-04-27 20:16 ` Russell King - ARM Linux
2011-04-27 20:16 ` Russell King - ARM Linux
2011-04-27 20:21 ` Arnd Bergmann
2011-04-27 20:21 ` Arnd Bergmann
2011-04-27 20:26 ` Russell King - ARM Linux
2011-04-27 20:26 ` Russell King - ARM Linux
2011-04-27 20:48 ` Arnd Bergmann
2011-04-27 20:48 ` Arnd Bergmann
2011-04-27 21:41 ` Benjamin Herrenschmidt
2011-04-27 21:41 ` Benjamin Herrenschmidt
2011-04-28 9:30 ` Russell King - ARM Linux
2011-04-28 9:30 ` Russell King - ARM Linux
2011-04-28 21:07 ` Benjamin Herrenschmidt
2011-04-28 21:07 ` Benjamin Herrenschmidt
2011-04-29 11:26 ` Arnd Bergmann
2011-04-29 11:26 ` Arnd Bergmann
2011-04-29 11:47 ` Benjamin Herrenschmidt
2011-04-29 11:47 ` Benjamin Herrenschmidt
2011-04-29 11:56 ` Alan Cox
2011-04-29 11:56 ` Alan Cox
2011-04-29 22:51 ` Benjamin Herrenschmidt
2011-04-29 22:51 ` Benjamin Herrenschmidt
2011-04-29 12:06 ` [Linaro-mm-sig] " Thomas Hellstrom
2011-04-29 12:06 ` Thomas Hellstrom
2011-04-29 13:34 ` Jerome Glisse
2011-04-29 13:34 ` Jerome Glisse
2011-04-29 22:55 ` Benjamin Herrenschmidt
2011-04-29 22:55 ` Benjamin Herrenschmidt
2011-04-29 22:53 ` Benjamin Herrenschmidt
2011-04-29 22:53 ` Benjamin Herrenschmidt
2011-04-27 10:51 ` Marek Szyprowski
2011-04-27 10:51 ` Marek Szyprowski
2011-04-27 21:37 ` Benjamin Herrenschmidt
2011-04-27 21:37 ` Benjamin Herrenschmidt
2011-04-28 6:40 ` [Linaro-mm-sig] " Arnd Bergmann
2011-04-28 6:40 ` Arnd Bergmann
2011-04-28 6:46 ` FUJITA Tomonori
2011-04-28 6:46 ` FUJITA Tomonori
2011-04-28 9:37 ` Russell King - ARM Linux
2011-04-28 9:37 ` Russell King - ARM Linux
2011-04-28 10:32 ` [Linaro-mm-sig] " Marek Szyprowski
2011-04-28 10:32 ` Marek Szyprowski
2011-04-28 10:51 ` Russell King - ARM Linux
2011-04-28 10:51 ` Russell King - ARM Linux
2011-04-28 12:28 ` Arnd Bergmann
2011-04-28 12:28 ` Arnd Bergmann
2011-04-28 13:15 ` Russell King - ARM Linux
2011-04-28 13:15 ` Russell King - ARM Linux
2011-04-28 14:29 ` Arnd Bergmann
2011-04-28 14:29 ` Arnd Bergmann
2011-04-28 14:34 ` Russell King - ARM Linux
2011-04-28 14:34 ` Russell King - ARM Linux
2011-04-28 14:39 ` Arnd Bergmann
2011-04-28 14:39 ` Arnd Bergmann
2011-04-28 14:58 ` Russell King - ARM Linux
2011-04-28 14:58 ` Russell King - ARM Linux
2011-04-28 19:37 ` Jerome Glisse
2011-04-28 19:37 ` Jerome Glisse
2011-04-29 0:29 ` Benjamin Herrenschmidt
2011-04-29 0:29 ` Benjamin Herrenschmidt
2011-04-29 5:50 ` Thomas Hellstrom
2011-04-29 5:50 ` Thomas Hellstrom
2011-04-29 7:35 ` Benjamin Herrenschmidt
2011-04-29 7:35 ` Benjamin Herrenschmidt
2011-04-29 10:55 ` Thomas Hellstrom
2011-04-29 10:55 ` Thomas Hellstrom
2011-04-29 22:50 ` Benjamin Herrenschmidt
2011-04-29 22:50 ` Benjamin Herrenschmidt
2011-04-29 16:27 ` Jesse Barnes
2011-04-29 16:27 ` Jesse Barnes
2011-04-29 22:46 ` Benjamin Herrenschmidt
2011-04-29 22:46 ` Benjamin Herrenschmidt
2011-04-30 2:45 ` Jesse Barnes
2011-04-30 2:45 ` Jesse Barnes
2011-04-29 7:59 ` Russell King - ARM Linux
2011-04-29 7:59 ` Russell King - ARM Linux
2011-04-29 16:32 ` Jesse Barnes
2011-04-29 16:32 ` Jesse Barnes
2011-04-29 18:29 ` Arnd Bergmann
2011-04-29 18:29 ` Arnd Bergmann
2011-04-29 22:15 ` Russell King - ARM Linux
2011-04-29 22:15 ` Russell King - ARM Linux
2011-05-02 4:42 ` David Brown
2011-05-02 4:42 ` David Brown
2011-05-02 11:26 ` Arnd Bergmann
2011-05-02 11:26 ` Arnd Bergmann
2011-04-29 22:37 ` Benjamin Herrenschmidt
2011-04-29 22:37 ` Benjamin Herrenschmidt
2011-04-29 13:42 ` Joerg Roedel
2011-04-29 13:42 ` Joerg Roedel
2011-04-29 14:19 ` Jerome Glisse
2011-04-29 14:19 ` Jerome Glisse
2011-04-29 15:37 ` Jordan Crouse
2011-04-29 15:37 ` Jordan Crouse
2011-04-28 14:38 ` FUJITA Tomonori
2011-04-28 14:38 ` FUJITA Tomonori
2011-04-29 0:25 ` Benjamin Herrenschmidt
2011-04-29 0:25 ` Benjamin Herrenschmidt
2011-04-29 11:21 ` Arnd Bergmann
2011-04-29 11:21 ` Arnd Bergmann
2011-04-28 10:41 ` Joerg Roedel
2011-04-28 10:41 ` Joerg Roedel
2011-04-28 11:01 ` Russell King - ARM Linux
2011-04-28 11:01 ` Russell King - ARM Linux
2011-04-28 12:25 ` Joerg Roedel
2011-04-28 12:25 ` Joerg Roedel
2011-04-28 12:42 ` Russell King - ARM Linux [this message]
2011-04-28 12:42 ` Russell King - ARM Linux
2011-04-28 12:59 ` Joerg Roedel
2011-04-28 12:59 ` Joerg Roedel
2011-04-28 13:02 ` Arnd Bergmann
2011-04-28 13:02 ` Arnd Bergmann
2011-04-28 13:19 ` Russell King - ARM Linux
2011-04-28 13:19 ` Russell King - ARM Linux
2011-04-28 13:56 ` Joerg Roedel
2011-04-28 13:56 ` Joerg Roedel
2011-04-28 14:30 ` Russell King - ARM Linux
2011-04-28 14:30 ` Russell King - ARM Linux
2011-04-27 9:52 ` Catalin Marinas
2011-04-27 9:52 ` Catalin Marinas
2011-04-27 10:43 ` Arnd Bergmann
2011-04-27 10:43 ` Arnd Bergmann
2011-04-27 11:08 ` Catalin Marinas
2011-04-27 11:08 ` Catalin Marinas
2011-04-28 0:15 ` Valdis.Kletnieks
2011-04-28 0:15 ` Valdis.Kletnieks at vt.edu
2011-04-28 8:27 ` Catalin Marinas
2011-04-28 8:27 ` Catalin Marinas
2011-04-28 12:12 ` Arnd Bergmann
2011-04-28 12:12 ` Arnd Bergmann
2011-04-28 12:36 ` Russell King - ARM Linux
2011-04-28 12:36 ` Russell King - ARM Linux
2011-04-28 12:48 ` Arnd Bergmann
2011-04-28 12:48 ` Arnd Bergmann
2011-05-03 14:45 ` Dave Martin
2011-05-03 14:45 ` Dave Martin
2011-04-29 15:41 ` [Linaro-mm-sig] " Arnd Bergmann
2011-04-29 15:41 ` Arnd Bergmann
2011-04-29 16:42 ` Catalin Marinas
2011-04-29 16:42 ` Catalin Marinas
2011-05-03 15:05 ` [Linaro-mm-sig] " Laurent Pinchart
2011-05-03 15:05 ` Laurent Pinchart
2011-05-03 15:31 ` Arnd Bergmann
2011-05-03 15:31 ` Arnd Bergmann
2011-04-27 14:06 ` FUJITA Tomonori
2011-04-27 14:06 ` FUJITA Tomonori
2011-04-27 14:29 ` Catalin Marinas
2011-04-27 14:29 ` Catalin Marinas
2011-04-27 14:34 ` FUJITA Tomonori
2011-04-27 14:34 ` FUJITA Tomonori
2011-04-27 20:29 ` Russell King - ARM Linux
2011-04-27 20:29 ` Russell King - ARM Linux
2011-04-27 21:45 ` Benjamin Herrenschmidt
2011-04-27 21:45 ` Benjamin Herrenschmidt
2011-04-28 7:24 ` [Linaro-mm-sig] " KyongHo Cho
2011-04-28 7:24 ` KyongHo Cho
2011-04-28 8:31 ` Catalin Marinas
2011-04-28 8:31 ` Catalin Marinas
2011-04-27 21:31 ` Benjamin Herrenschmidt
2011-04-27 21:31 ` Benjamin Herrenschmidt
2011-04-28 9:42 ` Russell King - ARM Linux
2011-04-28 9:42 ` Russell King - ARM Linux
2011-04-28 10:27 ` Joerg Roedel
2011-04-28 10:27 ` Joerg Roedel
2011-04-28 12:15 ` Arnd Bergmann
2011-04-28 12:15 ` Arnd Bergmann
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20110428124242.GJ17290@n2100.arm.linux.org.uk \
--to=linux@arm.linux.org.uk \
--cc=arnd@arndb.de \
--cc=joro@8bytes.org \
--cc=linaro-mm-sig@lists.linaro.org \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-kernel@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.