[Question about DMA] Consistent memory?

[Question about DMA] Consistent memory?

[Masahiro Yamada]

Hi.

I am new to the Linux DMA APIs.

First, I started by reading Documentation/DMA-API.txt,
but I am confused with the term "consistent memory".

Please help me understand the document correctly.


The DMA-API.txt says as follows:

----------------------->8--------------------------------------------
void *
dma_alloc_coherent(struct device *dev, size_t size,
    dma_addr_t *dma_handle, gfp_t flag)

Consistent memory is memory for which a write by either the device or
the processor can immediately be read by the processor or device
without having to worry about caching effects.  (You may however need
to make sure to flush the processor's write buffers before telling
devices to read that memory.)
------------------------8<--------------------------------------------


As far as I understand the the cited sentence, for the memory
to be consistent, DMA controllers must be connected to
DRAM through some special hardware that keeps the memory
coherency, such as SCU.  I assume the system like Fig.1


Fig.1

|------|  |------| |-----|
| CPU0 |  | CPU1 | | DMA |
|------|  |------| |-----|
   |         |        |
   |         |        |
|------|  |------| |-----|
| L1-C |  | L1-C | | ACP |
|------|  |------| |-----|
   |         |        |
|------------------------|
|    Snoop Control Unit  |
|------------------------|
             |
|------------------------|
|         L2-cache       |
|------------------------|
             |
|------------------------|
|           DRAM         |
|------------------------|

(ACP = accelerator coherency port)



But, I think such a system is rare.

At least on my SoC (ARM SoC), DMA controllers
for NAND, MMC, etc. are directly connected to the DRAM
like Fig.2.

So, cache operations must be explicitly done
by software before/after DMAs are kicked.
(I think this is very normal.)

Fig.2

|------|  |------| |-----|
| CPU0 |  | CPU1 | | DMA |
|------|  |------| |-----|
   |         |        |
   |         |        |
|------|  |------|    |
| L1-C |  | L1-C |    |
|------|  |------|    |
   |         |        |
|------------------|  |
|Snoop Control Unit|  |
|------------------|  |
         |            |
|------------------|  |
|   L2-cache       |  |
|------------------|  |
         |            |
|--------------------------|
|           DRAM           |
|--------------------------|


In a system like Fig.2, is the memory non-consistent?

As long as I read DMA-API.txt, it is non-consistent.
There is no consistent memory on my SoC.

But, not only dma_alloc_noncoherent, but also dma_alloc_coherent()
returns a memory region on my SoC.  I am confused...



-- 
Best Regards
Masahiro Yamada

Re: [Question about DMA] Consistent memory?

[Mike Looijmans]

On 31-12-15 08:50, Masahiro Yamada wrote:
> Hi.
>
> I am new to the Linux DMA APIs.
>
> First, I started by reading Documentation/DMA-API.txt,
> but I am confused with the term "consistent memory".
>
> Please help me understand the document correctly.
>
>
> The DMA-API.txt says as follows:
>
> ----------------------->8--------------------------------------------
> void *
> dma_alloc_coherent(struct device *dev, size_t size,
>      dma_addr_t *dma_handle, gfp_t flag)
>
> Consistent memory is memory for which a write by either the device or
> the processor can immediately be read by the processor or device
> without having to worry about caching effects.  (You may however need
> to make sure to flush the processor's write buffers before telling
> devices to read that memory.)
> ------------------------8<--------------------------------------------
>
>
> As far as I understand the the cited sentence, for the memory
> to be consistent, DMA controllers must be connected to
> DRAM through some special hardware that keeps the memory
> coherency, such as SCU.  I assume the system like Fig.1
>
>
> Fig.1
>
> |------|  |------| |-----|
> | CPU0 |  | CPU1 | | DMA |
> |------|  |------| |-----|
>     |         |        |
>     |         |        |
> |------|  |------| |-----|
> | L1-C |  | L1-C | | ACP |
> |------|  |------| |-----|
>     |         |        |
> |------------------------|
> |    Snoop Control Unit  |
> |------------------------|
>               |
> |------------------------|
> |         L2-cache       |
> |------------------------|
>               |
> |------------------------|
> |           DRAM         |
> |------------------------|
>
> (ACP = accelerator coherency port)
>
>
>
> But, I think such a system is rare.
>
> At least on my SoC (ARM SoC), DMA controllers
> for NAND, MMC, etc. are directly connected to the DRAM
> like Fig.2.
>
> So, cache operations must be explicitly done
> by software before/after DMAs are kicked.
> (I think this is very normal.)
>
> Fig.2
>
> |------|  |------| |-----|
> | CPU0 |  | CPU1 | | DMA |
> |------|  |------| |-----|
>     |         |        |
>     |         |        |
> |------|  |------|    |
> | L1-C |  | L1-C |    |
> |------|  |------|    |
>     |         |        |
> |------------------|  |
> |Snoop Control Unit|  |
> |------------------|  |
>           |            |
> |------------------|  |
> |   L2-cache       |  |
> |------------------|  |
>           |            |
> |--------------------------|
> |           DRAM           |
> |--------------------------|
>
>
> In a system like Fig.2, is the memory non-consistent?
>
> As long as I read DMA-API.txt, it is non-consistent.
> There is no consistent memory on my SoC.
>
> But, not only dma_alloc_noncoherent, but also dma_alloc_coherent()
> returns a memory region on my SoC.  I am confused...
>

On systems like this, dma_alloc_coherent() will simply return 
non-cacheable memory. Since the cache is not being used, the memory is 
coherent.

If you mmap the DMA buffer to user space, this will have performance 
implications for the userspace program. If you use the region as a 
"bounce" buffer, this is usually faster than using the streaming API.

-- 
Mike Looijmans

Re: [Question about DMA] Consistent memory?

[One Thousand Gnomes]

On Thu, 31 Dec 2015 16:50:54 +0900
> But, I think such a system is rare.

Actually its quite normal for some vendors processors but not others.

> At least on my SoC (ARM SoC), DMA controllers
> for NAND, MMC, etc. are directly connected to the DRAM
> like Fig.2.
> 
> So, cache operations must be explicitly done
> by software before/after DMAs are kicked.
> (I think this is very normal.)

For ARM certainly.

> 
> Fig.2
> 
> |------|  |------| |-----|
> | CPU0 |  | CPU1 | | DMA |
> |------|  |------| |-----|
>    |         |        |
>    |         |        |
> |------|  |------|    |
> | L1-C |  | L1-C |    |
> |------|  |------|    |
>    |         |        |
> |------------------|  |
> |Snoop Control Unit|  |
> |------------------|  |
>          |            |
> |------------------|  |
> |   L2-cache       |  |
> |------------------|  |
>          |            |
> |--------------------------|
> |           DRAM           |
> |--------------------------|
> 
> 
> In a system like Fig.2, is the memory non-consistent?

dma_alloc_coherent will always provide you with coherent memory. On a
machine with good cache interfaces it will provide you with normal
memory. On some systems it may be memory from a special window, in other
cases it will fall back to providing uncached memory for this.

If the platform genuinely cannot support this (even by marking those areas
uncacheable) then it will fail the allocation.

What it does mean is that you need to use non-coherent mappings when
accessing a lot of data. On hardware without proper cache coherency it
may be quite expensive to access coherent memory.

Alan

Re: [Question about DMA] Consistent memory?

[Masahiro Yamada]

Hi Alan, Mike,

Thanks for your help!


2015-12-31 19:25 GMT+09:00 One Thousand Gnomes <gnomes@lxorguk.ukuu.org.uk>:

>>
>> In a system like Fig.2, is the memory non-consistent?
>
> dma_alloc_coherent will always provide you with coherent memory. On a
> machine with good cache interfaces it will provide you with normal
> memory. On some systems it may be memory from a special window, in other
> cases it will fall back to providing uncached memory for this.
>
> If the platform genuinely cannot support this (even by marking those areas
> uncacheable) then it will fail the allocation.
>
> What it does mean is that you need to use non-coherent mappings when
> accessing a lot of data. On hardware without proper cache coherency it
> may be quite expensive to access coherent memory.


Now, it is clearer to me.
The following is what I understood.
(Please point out if I am wrong.)


I think, roughly, there are two ways for handling DMA:
(At first, I was so confused that I was thinking about [1] and [2] mixed.)



[1] DMA-coherent buffers

Allocate buffers with dma_alloc_coherent()
and just have access to the buffers without cache synchronization.

There is no need to call dma_sync_single_for_*().



[2] Streaming DMA

Allocate buffers with kmalloc() or friends,
and then map them for DMA with dma_map_single().

The buffers are cached, so they are non-consitent
unless there exists hardware assist such as
Cache Coherency Interconnect.

The drivers must invoke cache operations
by calling dma_sync_single_for_*().




Is there any guideline about which way should be used in drivers?

I think, if the buffer size is small, [1] is more efficient
because it need not invoke cache operations.

If the buffer is large, [2] seems better because
the cost of uncached memory access gets more expensive
than that of cache operations.

(If devices are connected to the memory controller
via Cache Coherency Interconnect, [1] always works very well.
But drivers should be written in a portable way, so
such a hardware implementation should not be expected.)

I am not sure about the border line between [1] and [2], though...



BTW, I am studying the DMA APIs in order to write a new
MMC host driver for my ARM SoC.


I grepped under drivers/mmc/host, and
I found many drivers call dma_alloc_coherent(),
but there are also some drivers that use dma_map_single().




-- 
Best Regards
Masahiro Yamada

Re: [Question about DMA] Consistent memory?

[Mike Looijmans]

On 31-12-2015 15:57, Masahiro Yamada wrote:
> Hi Alan, Mike,
>
> Thanks for your help!
>
>
> 2015-12-31 19:25 GMT+09:00 One Thousand Gnomes <gnomes@lxorguk.ukuu.org.uk>:
>
>>>
>>> In a system like Fig.2, is the memory non-consistent?
>>
>> dma_alloc_coherent will always provide you with coherent memory. On a
>> machine with good cache interfaces it will provide you with normal
>> memory. On some systems it may be memory from a special window, in other
>> cases it will fall back to providing uncached memory for this.
>>
>> If the platform genuinely cannot support this (even by marking those areas
>> uncacheable) then it will fail the allocation.
>>
>> What it does mean is that you need to use non-coherent mappings when
>> accessing a lot of data. On hardware without proper cache coherency it
>> may be quite expensive to access coherent memory.
>
>
> Now, it is clearer to me.
> The following is what I understood.
> (Please point out if I am wrong.)
>
>
> I think, roughly, there are two ways for handling DMA:
> (At first, I was so confused that I was thinking about [1] and [2] mixed.)
>
>
>
> [1] DMA-coherent buffers
>
> Allocate buffers with dma_alloc_coherent()
> and just have access to the buffers without cache synchronization.
>
> There is no need to call dma_sync_single_for_*().
>
>
>
> [2] Streaming DMA
>
> Allocate buffers with kmalloc() or friends,
> and then map them for DMA with dma_map_single().
>
> The buffers are cached, so they are non-consitent
> unless there exists hardware assist such as
> Cache Coherency Interconnect.
>
> The drivers must invoke cache operations
> by calling dma_sync_single_for_*().
>
>
>
>
> Is there any guideline about which way should be used in drivers?
>
> I think, if the buffer size is small, [1] is more efficient
> because it need not invoke cache operations.
>
> If the buffer is large, [2] seems better because
> the cost of uncached memory access gets more expensive
> than that of cache operations.

There's no difference in choice for large or small blocks. The dma_sync 
functions take linear time (as function of block size) to do their 
thing, larger buffers take longer to flush.

On the Zynq (also ARM, with a choice of coherency connections) I 
measured that the dma_sync operations took only slightly less time than 
simply copying the data.

If the action taken on the buffer after the DMA completion is to copy it 
to (of from) a user buffer, you should use dma_coherent calls. That's 
what I meant by "bounce buffers".

If you plan to DMA data straight to/from userspace, you'll need the 
dma_sync methods. (On coherent systems, the dma_sync methods become no-ops).

> (If devices are connected to the memory controller
> via Cache Coherency Interconnect, [1] always works very well.
> But drivers should be written in a portable way, so
> such a hardware implementation should not be expected.)
>
> I am not sure about the border line between [1] and [2], though...
>
>
>
> BTW, I am studying the DMA APIs in order to write a new
> MMC host driver for my ARM SoC.
>
>
> I grepped under drivers/mmc/host, and
> I found many drivers call dma_alloc_coherent(),
> but there are also some drivers that use dma_map_single().

If I recall correctly, most MMC controllers have their own 
scatter-gather DMA controller and copy data straight to/from userspace 
buffers.

-- 
Mike Looijmans

Re: [Question about DMA] Consistent memory?

[Russell King - ARM Linux]

On Thu, Dec 31, 2015 at 11:57:55PM +0900, Masahiro Yamada wrote:
> [1] DMA-coherent buffers
> 
> Allocate buffers with dma_alloc_coherent()
> and just have access to the buffers without cache synchronization.
> 
> There is no need to call dma_sync_single_for_*().

dma_sync_single_for_*() is part of the streaming API and should never
be used with DMA-coherent buffers.

> [2] Streaming DMA
> 
> Allocate buffers with kmalloc() or friends,
> and then map them for DMA with dma_map_single().
> 
> The buffers are cached, so they are non-consitent
> unless there exists hardware assist such as
> Cache Coherency Interconnect.
> 
> The drivers must invoke cache operations
> by calling dma_sync_single_for_*().

I have a problem with that last statement.  There is no "must".  One
way to look at the DMA API is that you're using the various calls to
transfer ownership (and access right) of the buffer between the CPU
and the DMA device.

So, dma_map_single() transfers ownership from the CPU to the DMA
device, as does dma_sync_single_for_device().  dma_unmap_single()
and dma_sync_single_for_cpu() transfers ownership from the DMA
device to the CPU.

If you intend to allocate a buffer, and then perform DMA on it, you
just need to allocate, use dma_map_single(), and then kick the DMA.
Once DMA has completed, use dma_unmap_single() before touching the
buffer.

If you intend to inspect the contents of the buffer during DMA, then
use dma_sync_single_for_cpu() before reading the buffer.  This
ensures that when you read from the buffer, you see up-to-date data.
You strictly don't need to use dma_sync_single_for_device() prior
to resuming DMA.

However, you must use dma_unmap_single() before you free the memory.

> I think, if the buffer size is small, [1] is more efficient
> because it need not invoke cache operations.
> 
> If the buffer is large, [2] seems better because
> the cost of uncached memory access gets more expensive
> than that of cache operations.

It doesn't always follow.  Coherent memory is only available in page
sized chunks, so aren't really "small buffers".

Generally, coherent memory is used for things like DMA descriptor ring
buffers, where we need simultaneous access by both the DMA device and
CPU (the DMA device updates descriptors as it processes them, the CPU
can inspect and queue new descriptors as the DMA device processes them.)
Network devices do this a lot.

The DMA API streaming interfaces tend to be used with buffers which are
allocated "out of control" of the driver - if we take the network device
example, the network packet buffers will be mapped and unmapped using
the streaming API.

With a different example, video capture, there's different trade offs.
A video capture buffer may be very large (8MB for a 1080p frame.)
Flushing the cache over 8MB of data is very inefficient, and it's
probably more performant to use DMA coherent memory instead, even
more so if you don't actually intend for the CPU to access it - eg,
you're passing the frame to another hardware block for further
processing.

> I grepped under drivers/mmc/host, and
> I found many drivers call dma_alloc_coherent(),
> but there are also some drivers that use dma_map_single().

Yes - you're probably seeing the pattern I mentioned above - DMA
descriptors on coherent memory, the data buffers being passed in
to the driver from elsewhere, and mapped using the streaming API.

Hope this is helpful.

-- 
RMK's Patch system: http://www.arm.linux.org.uk/developer/patches/
FTTC broadband for 0.8mile line: currently at 9.6Mbps down 400kbps up
according to speedtest.net.

Re: [Question about DMA] Consistent memory?

[Russell King - ARM Linux]

On Thu, Dec 31, 2015 at 04:50:54PM +0900, Masahiro Yamada wrote:
> Hi.
> 
> I am new to the Linux DMA APIs.
> 
> First, I started by reading Documentation/DMA-API.txt,
> but I am confused with the term "consistent memory".

Just read "coherent memory" instead - the documentation confusingly uses
the two terms to refer to the same thing.  I think there was a patch a
while back to replace "consistent" with "coherent" in this document,
though I'm not sure what happened to it.

I think you have answers to your other points by others in this thread.

Thanks.

-- 
RMK's Patch system: http://www.arm.linux.org.uk/developer/patches/
FTTC broadband for 0.8mile line: currently at 9.6Mbps down 400kbps up
according to speedtest.net.

Re: [Question about DMA] Consistent memory?

[James Bottomley]

On Sat, 2016-01-02 at 10:39 +0000, Russell King - ARM Linux wrote:
> On Thu, Dec 31, 2015 at 04:50:54PM +0900, Masahiro Yamada wrote:
> > Hi.
> > 
> > I am new to the Linux DMA APIs.
> > 
> > First, I started by reading Documentation/DMA-API.txt,
> > but I am confused with the term "consistent memory".
> 
> Just read "coherent memory" instead - the documentation confusingly 
> uses the two terms to refer to the same thing.  I think there was a 
> patch a while back to replace "consistent" with "coherent" in this 
> document, though I'm not sure what happened to it.

It's an standards issue.  The Document was originally based on the PCI
DMA API.  All the PCI standards documentation refers to "consistent
memory" instead of "coherent memory".  The original DMA API was
designed for PA-RISC and its standards documentation refers to
"coherent memory" hence the confusion.  The two terms are equivalent,
but there's no real way of removing either without someone reading the
actual specs and wondering what the other term means.

James

Re: [Question about DMA] Consistent memory?

[Russell King - ARM Linux]

On Sat, Jan 02, 2016 at 08:17:51AM -0800, James Bottomley wrote:
> On Sat, 2016-01-02 at 10:39 +0000, Russell King - ARM Linux wrote:
> > On Thu, Dec 31, 2015 at 04:50:54PM +0900, Masahiro Yamada wrote:
> > > Hi.
> > > 
> > > I am new to the Linux DMA APIs.
> > > 
> > > First, I started by reading Documentation/DMA-API.txt,
> > > but I am confused with the term "consistent memory".
> > 
> > Just read "coherent memory" instead - the documentation confusingly 
> > uses the two terms to refer to the same thing.  I think there was a 
> > patch a while back to replace "consistent" with "coherent" in this 
> > document, though I'm not sure what happened to it.
> 
> It's an standards issue.  The Document was originally based on the PCI
> DMA API.  All the PCI standards documentation refers to "consistent
> memory" instead of "coherent memory".  The original DMA API was
> designed for PA-RISC and its standards documentation refers to
> "coherent memory" hence the confusion.  The two terms are equivalent,
> but there's no real way of removing either without someone reading the
> actual specs and wondering what the other term means.

May it be an idea to add a footnote explaining that the two terms
are interchangable and equivalent then - this is not the first time
that people have asked questions about it, and I suspect that unless
something is done, there will be a continuing stream of questions.

-- 
RMK's Patch system: http://www.arm.linux.org.uk/developer/patches/
FTTC broadband for 0.8mile line: currently at 9.6Mbps down 400kbps up
according to speedtest.net.

Re: [Question about DMA] Consistent memory?

[Mike Looijmans]

On 2-1-2016 11:39, Russell King - ARM Linux wrote:
> On Thu, Dec 31, 2015 at 04:50:54PM +0900, Masahiro Yamada wrote:
>> Hi.
>>
>> I am new to the Linux DMA APIs.
>>
>> First, I started by reading Documentation/DMA-API.txt,
>> but I am confused with the term "consistent memory".
>
> Just read "coherent memory" instead - the documentation confusingly uses
> the two terms to refer to the same thing.  I think there was a patch a
> while back to replace "consistent" with "coherent" in this document,
> though I'm not sure what happened to it.

I wrote that patch. I never got any comments on it, so either I didn't 
post it to the right people, or no one really cares:
http://www.kernelhub.org/?msg=747166&p=2

I still think that if the kernel methods all have "coherent" in their 
name, we should use the word "coherent" in the documentation as well, 
and not confuse people even further. So I'd happily repost that patch.


-- 
Mike Looijmans

Re: [Question about DMA] Consistent memory?

[James Bottomley]

On Sat, 2016-01-02 at 19:35 +0100, Mike Looijmans wrote:
> On 2-1-2016 11:39, Russell King - ARM Linux wrote:
> > On Thu, Dec 31, 2015 at 04:50:54PM +0900, Masahiro Yamada wrote:
> > > Hi.
> > > 
> > > I am new to the Linux DMA APIs.
> > > 
> > > First, I started by reading Documentation/DMA-API.txt,
> > > but I am confused with the term "consistent memory".
> > 
> > Just read "coherent memory" instead - the documentation confusingly
> > uses
> > the two terms to refer to the same thing.  I think there was a
> > patch a
> > while back to replace "consistent" with "coherent" in this
> > document,
> > though I'm not sure what happened to it.
> 
> I wrote that patch. I never got any comments on it, so either I
> didn't 
> post it to the right people, or no one really cares:
> http://www.kernelhub.org/?msg=747166&p=2
> 
> I still think that if the kernel methods all have "coherent" in their
> name, we should use the word "coherent" in the documentation as well,
> and not confuse people even further. So I'd happily repost that 
> patch.

They don't: the PCI API still uses consistent:

asm-generic/pci-dma-compat.h:pci_alloc_consistent(struct pci_dev *hwdev, size_t size,
asm-generic/pci-dma-compat.h:pci_zalloc_consistent(struct pci_dev *hwdev, size_t size,
asm-generic/pci-dma-compat.h:pci_free_consistent(struct pci_dev *hwdev, size_t size,
asm-generic/pci-dma-compat.h:static inline int pci_set_consistent_dma_mask(struct pci_dev *dev, u64 mask)
linux/pci.h:/* kmem_cache style wrapper around pci_alloc_consistent() */
linux/pci.h:static inline int pci_set_consistent_dma_mask(struct pci_dev *dev, u64 mask)

These are named based on the PCI specification.

James