cache maintenance for device memory

cache maintenance for device memory

[Random Guy]

Hi,

I have a device with a piece of RAM. That memory is accessible to CPU.
I write a driver for that device and it maps device memory directly to
userspace processes. This mapping is cacheable. So far so good.

Now I'd like to flush or invalidate cache on specific parts of that memory.
If it was just normal system memory I could use streaming DMA API:
dma_map_*/dma_sync_*/dma_unmap_*. But the device memory is not
a part of system memory and so there are no struct page for its pages,
and thus streaming DMA support functions don't work.

Is there anything wrong with my setup?
What should I use to maintain caches for device memory?

TIA

cache maintenance for device memory

[Random Guy]

Hi syed,

thanks for your response.

On Fri, Dec 4, 2015 at 9:16 AM, sk.syed2 <sk.syed2@gmail.com> wrote:
>> I have a device with a piece of RAM. That memory is accessible to CPU.
>> I write a driver for that device and it maps device memory directly to
>> userspace processes. This mapping is cacheable. So far so good.
>>
>    How did you map this device memory? Using ioremap? What platform is this?

No, I provide mmap for the device file and then I do remap_pfn_range
using vma->vm_pgoff as an offset into the device memory block.

That's custom R&D ARM-based thing.

>> Now I'd like to flush or invalidate cache on specific parts of that memory.
>> If it was just normal system memory I could use streaming DMA API:
>> dma_map_*/dma_sync_*/dma_unmap_*. But the device memory is not
>> a part of system memory and so there are no struct page for its pages,
>> and thus streaming DMA support functions don't work.
>
> Once you map the io memory, you get VAs for that memory and you should be able
> to use dma_sync APIs.

I've looked at the following function calls:
arm_dma_sync_single_for_cpu
-> dma_cache_maint_page
-> __dma_page_dev_to_cpu
and I see them mapping DMA handle to struct page and using that struct page.
I'm sure that will not work, because as I said the device memory is not covered
by struct page's.

cache maintenance for device memory

[Random Guy]

On Thu, Dec 3, 2015 at 1:29 PM, Random Guy
<random.guy.from.russia@gmail.com> wrote:
> I have a device with a piece of RAM. That memory is accessible to CPU.
> I write a driver for that device and it maps device memory directly to
> userspace processes. This mapping is cacheable. So far so good.

Ok, I did some research and all drivers that I've found that map memory
to userspace do it with caching disabled.

After some more reading, especially
  http://events.linuxfoundation.org/sites/events/files/slides/20140429-dma.pdf
I conclude that cacheable mapping of device memory to userspace is
not supported in linux at all.

Is that correct?

TIA

cache maintenance for device memory

[Random Guy]

On Fri, Dec 4, 2015 at 5:12 PM, Random Guy
<random.guy.from.russia@gmail.com> wrote:
> Ok, I did some research and all drivers that I've found that map memory
> to userspace do it with caching disabled.

correction: 'that map device memory to userspace'.