cma_remap when using dma_alloc_attr :- DMA_ATTR_NO_KERNEL_MAPPING

cma_remap when using dma_alloc_attr :- DMA_ATTR_NO_KERNEL_MAPPING

[Pankaj Suryawanshi]

[-- Attachment #1: Type: text/plain, Size: 2549 bytes --]

Hello,

When we allocate cma memory using dma_alloc_attr using
DMA_ATTR_NO_KERNEL_MAPPING attribute. It will return physical address
without virtual mapping and thats the use case of this attribute. but lets
say some vpu/gpu drivers required virtual mapping of some part of the
allocation. then we dont have anything to remap that allocated memory to
virtual memory. and in 32-bit system it difficult for devices like android
to work all the time with virtual mapping, it degrade the performance.

For Example :

Lets say 4k video allocation required 300MB cma memory but not required
virtual mapping for all the 300MB, its require only 20MB virtually mapped
at some specific use case/point of video, and unmap virtual mapping after
uses, at that time this functions will be useful, it works like ioremap()
for cma_alloc() using dma apis.

/*
         * function call(s) to create virtual map of given physical memory
         * range [base, base+size) of CMA memory.
*/
void *cma_remap(__u32 base, __u32 size)
{
        struct page *page = phys_to_page(base);
        void *virt;

        pr_debug("cma: request to map 0x%08x for size 0x%08x\n",
                        base, size);

        size = PAGE_ALIGN(size);

        pgprot_t prot = get_dma_pgprot(DMA_ATTR, PAGE_KERNEL);

        if (PageHighMem(page)){
                virt = dma_alloc_remap(page, size, GFP_KERNEL, prot,
__builtin_return_address(0));
        }
        else
        {
                dma_remap(page, size, prot);
                virt = page_address(page);
        }

        if (!virt)
                pr_err("\x1b[31m" " cma: failed to map 0x%08x" "\x1b[0m\n",
                                base);
        else
                pr_debug("cma: 0x%08x is virtually mapped to 0x%08x\n",
                                base, (__u32) virt);

        return virt;
}

/*
         * function call(s) to remove virtual map of given virtual memory
         * range [virt, virt+size) of CMA memory.
*/

void cma_unmap(void *virt, __u32 size)
{
        size = PAGE_ALIGN(size);
        unsigned long pfn = virt_to_pfn(virt);
        struct page *page = pfn_to_page(pfn);

                if (PageHighMem(page))
                        dma_free_remap(virt, size);
                else
                        dma_remap(page, size, PAGE_KERNEL);

        pr_debug(" cma: virtual address 0x%08x is unmapped\n",
                        (__u32) virt);
}

This functions should be added in arch/arm/mm/dma-mapping.c file.

Please let me know if i am missing anything.

Regards,
Pankaj

[-- Attachment #2: Type: text/html, Size: 3117 bytes --]

Re: cma_remap when using dma_alloc_attr :- DMA_ATTR_NO_KERNEL_MAPPING

[Robin Murphy]

On 12/07/2019 19:30, Pankaj Suryawanshi wrote:
> Hello,
> 
> When we allocate cma memory using dma_alloc_attr using
> DMA_ATTR_NO_KERNEL_MAPPING attribute. It will return physical address
> without virtual mapping and thats the use case of this attribute. but lets
> say some vpu/gpu drivers required virtual mapping of some part of the
> allocation. then we dont have anything to remap that allocated memory to
> virtual memory. and in 32-bit system it difficult for devices like android
> to work all the time with virtual mapping, it degrade the performance.
> 
> For Example :
> 
> Lets say 4k video allocation required 300MB cma memory but not required
> virtual mapping for all the 300MB, its require only 20MB virtually mapped
> at some specific use case/point of video, and unmap virtual mapping after
> uses, at that time this functions will be useful, it works like ioremap()
> for cma_alloc() using dma apis.

Hmm, is there any significant reason that this case couldn't be handled 
with just get_vm_area() plus dma_mmap_attrs(). I know it's only 
*intended* for userspace mappings, but since the basic machinery is there...

Robin.

> /*
>           * function call(s) to create virtual map of given physical memory
>           * range [base, base+size) of CMA memory.
> */
> void *cma_remap(__u32 base, __u32 size)
> {
>          struct page *page = phys_to_page(base);
>          void *virt;
> 
>          pr_debug("cma: request to map 0x%08x for size 0x%08x\n",
>                          base, size);
> 
>          size = PAGE_ALIGN(size);
> 
>          pgprot_t prot = get_dma_pgprot(DMA_ATTR, PAGE_KERNEL);
> 
>          if (PageHighMem(page)){
>                  virt = dma_alloc_remap(page, size, GFP_KERNEL, prot,
> __builtin_return_address(0));
>          }
>          else
>          {
>                  dma_remap(page, size, prot);
>                  virt = page_address(page);
>          }
> 
>          if (!virt)
>                  pr_err("\x1b[31m" " cma: failed to map 0x%08x" "\x1b[0m\n",
>                                  base);
>          else
>                  pr_debug("cma: 0x%08x is virtually mapped to 0x%08x\n",
>                                  base, (__u32) virt);
> 
>          return virt;
> }
> 
> /*
>           * function call(s) to remove virtual map of given virtual memory
>           * range [virt, virt+size) of CMA memory.
> */
> 
> void cma_unmap(void *virt, __u32 size)
> {
>          size = PAGE_ALIGN(size);
>          unsigned long pfn = virt_to_pfn(virt);
>          struct page *page = pfn_to_page(pfn);
> 
>                  if (PageHighMem(page))
>                          dma_free_remap(virt, size);
>                  else
>                          dma_remap(page, size, PAGE_KERNEL);
> 
>          pr_debug(" cma: virtual address 0x%08x is unmapped\n",
>                          (__u32) virt);
> }
> 
> This functions should be added in arch/arm/mm/dma-mapping.c file.
> 
> Please let me know if i am missing anything.
> 
> Regards,
> Pankaj
> 

Re: cma_remap when using dma_alloc_attr :- DMA_ATTR_NO_KERNEL_MAPPING

[Christoph Hellwig]

On Tue, Jul 16, 2019 at 01:02:19PM +0100, Robin Murphy wrote:
>> Lets say 4k video allocation required 300MB cma memory but not required
>> virtual mapping for all the 300MB, its require only 20MB virtually mapped
>> at some specific use case/point of video, and unmap virtual mapping after
>> uses, at that time this functions will be useful, it works like ioremap()
>> for cma_alloc() using dma apis.
>
> Hmm, is there any significant reason that this case couldn't be handled 
> with just get_vm_area() plus dma_mmap_attrs(). I know it's only *intended* 
> for userspace mappings, but since the basic machinery is there...

Because the dma helper really are a black box abstraction.

That being said DMA_ATTR_NO_KERNEL_MAPPING and DMA_ATTR_NON_CONSISTENT
have been a constant pain in the b**t.  I've been toying with replacing
them with a dma_alloc_pages or similar abstraction that just returns
a struct page that is guaranteed to be dma addressable by the passed
in device.  Then the driver can call dma_map_page / dma_unmap_page /
dma_sync_* on it at well.  This would replace DMA_ATTR_NON_CONSISTENT
with a sensible API, and also DMA_ATTR_NO_KERNEL_MAPPING when called
with PageHighmem, while providing an easy to understand API and
something that can easily be fed into the various page based APIs
in the kernel.

That being said until we get arm moved over the common dma direct
and dma-iommu code, and x86 fully moved over to dma-iommu it just
seems way too much work to even get it into the various architectures
that matter, never mind all the fringe IOMMUs.  So for now I've just
been trying to contain the DMA_ATTR_NON_CONSISTENT and
DMA_ATTR_NO_KERNEL_MAPPING in fewer places while also killing bogus
or pointless users of these APIs.

Re: cma_remap when using dma_alloc_attr :- DMA_ATTR_NO_KERNEL_MAPPING

[Pankaj Suryawanshi]

On Tue, Jul 16, 2019 at 5:40 PM Christoph Hellwig <hch@lst.de> wrote:
>
> On Tue, Jul 16, 2019 at 01:02:19PM +0100, Robin Murphy wrote:
> >> Lets say 4k video allocation required 300MB cma memory but not required
> >> virtual mapping for all the 300MB, its require only 20MB virtually mapped
> >> at some specific use case/point of video, and unmap virtual mapping after
> >> uses, at that time this functions will be useful, it works like ioremap()
> >> for cma_alloc() using dma apis.
> >
> > Hmm, is there any significant reason that this case couldn't be handled
> > with just get_vm_area() plus dma_mmap_attrs(). I know it's only *intended*
> > for userspace mappings, but since the basic machinery is there...
>
> Because the dma helper really are a black box abstraction.
>
> That being said DMA_ATTR_NO_KERNEL_MAPPING and DMA_ATTR_NON_CONSISTENT
> have been a constant pain in the b**t.  I've been toying with replacing
> them with a dma_alloc_pages or similar abstraction that just returns
> a struct page that is guaranteed to be dma addressable by the passed
> in device.  Then the driver can call dma_map_page / dma_unmap_page /
> dma_sync_* on it at well.  This would replace DMA_ATTR_NON_CONSISTENT
> with a sensible API, and also DMA_ATTR_NO_KERNEL_MAPPING when called
> with PageHighmem, while providing an easy to understand API and
> something that can easily be fed into the various page based APIs
> in the kernel.
>
> That being said until we get arm moved over the common dma direct
> and dma-iommu code, and x86 fully moved over to dma-iommu it just
> seems way too much work to even get it into the various architectures
> that matter, never mind all the fringe IOMMUs.  So for now I've just
> been trying to contain the DMA_ATTR_NON_CONSISTENT and
> DMA_ATTR_NO_KERNEL_MAPPING in fewer places while also killing bogus
> or pointless users of these APIs.


I agree with you Christoph, users want page based api, which is useful
in many scenarios, but
As of now i think we have to move with this type of api which is
useful when dma_alloc (for cma )call with DMA_ATTR_NO_KERNEL_MAPPING,
and mapped again to kernel space, this api is useful mostly for 32-bit
system which has 4GB of limited virtual memory (its very less for
android devices) as we have already dma_mmap_attr() for user space
mapping.
This api is also useful for one who directly want to use cma_alloc()
in their own drivers. For example ion-cma.c.
Please let me know if any recommendation/suggestion/improvement required ?

Regards,
Pankaj