[PATCH] dma-direct: Export dma_direct_alloc() and dma_direct_free()

[PATCH] dma-direct: Export dma_direct_alloc() and dma_direct_free()

[Thierry Reding]

From: Thierry Reding <treding@nvidia.com>

Drivers that are built as modules may want to use these functions, so
make them available like the rest of the functions.

Signed-off-by: Thierry Reding <treding@nvidia.com>
---
 kernel/dma/direct.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/kernel/dma/direct.c b/kernel/dma/direct.c
index 25bd19974223..f14b7ced4592 100644
--- a/kernel/dma/direct.c
+++ b/kernel/dma/direct.c
@@ -200,6 +200,7 @@ void *dma_direct_alloc(struct device *dev, size_t size,
 		return arch_dma_alloc(dev, size, dma_handle, gfp, attrs);
 	return dma_direct_alloc_pages(dev, size, dma_handle, gfp, attrs);
 }
+EXPORT_SYMBOL(dma_direct_alloc);
 
 void dma_direct_free(struct device *dev, size_t size,
 		void *cpu_addr, dma_addr_t dma_addr, unsigned long attrs)
@@ -209,6 +210,7 @@ void dma_direct_free(struct device *dev, size_t size,
 	else
 		dma_direct_free_pages(dev, size, cpu_addr, dma_addr, attrs);
 }
+EXPORT_SYMBOL(dma_direct_free);
 
 #if defined(CONFIG_ARCH_HAS_SYNC_DMA_FOR_DEVICE) || \
     defined(CONFIG_SWIOTLB)
-- 
2.19.1

Re: [PATCH] dma-direct: Export dma_direct_alloc() and dma_direct_free()

[Christoph Hellwig]

On Tue, Feb 05, 2019 at 12:06:02PM +0100, Thierry Reding wrote:
> From: Thierry Reding <treding@nvidia.com>
> 
> Drivers that are built as modules may want to use these functions, so
> make them available like the rest of the functions.
> 
> Signed-off-by: Thierry Reding <treding@nvidia.com>

How do they want to use these functions?  The proper way is to
call dma_alloc_coherent / dma_alloc_free and let the DMA API
handle the rest.

Re: [PATCH] dma-direct: Export dma_direct_alloc() and dma_direct_free()

[Thierry Reding]

[-- Attachment #1: Type: text/plain, Size: 1580 bytes --]

On Tue, Feb 05, 2019 at 05:10:36PM +0100, Christoph Hellwig wrote:
> On Tue, Feb 05, 2019 at 12:06:02PM +0100, Thierry Reding wrote:
> > From: Thierry Reding <treding@nvidia.com>
> > 
> > Drivers that are built as modules may want to use these functions, so
> > make them available like the rest of the functions.
> > 
> > Signed-off-by: Thierry Reding <treding@nvidia.com>
> 
> How do they want to use these functions?  The proper way is to
> call dma_alloc_coherent / dma_alloc_free and let the DMA API
> handle the rest.

This is again for the host1x driver and Tegra DRM where we had a similar
situation a little while ago where we needed to explicitly detach from a
DMA/IOMMU mapping.

I ran into a similar situation on Tegra186 (which has an ARM SMMU). The
host1x driver needs to allocate memory for a push buffer that commands
are written to. This push buffer is mapped translated through the SMMU
using direct IOMMU API usage. We use the same domain to also map command
buffers that userspace can pass in.

The problem is that if I use dma_alloc_coherent(), then the memory will
already be mapped via the SMMU at that point and then the driver, not
knowing that memory has already been mapped, will attempt to map an IOVA
which will cause an SMMU fault when the host1x tries to access the
memory.

I didn't find an equivalent to arm_iommu_detach_device() for non-ARM,
but then stumbled across this and thought it was rather convenient for
these cases. If there's a better way to deal with this situation, I'd be
happy to do so.

Thierry

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH] dma-direct: Export dma_direct_alloc() and dma_direct_free()

[Christoph Hellwig]

On Tue, Feb 05, 2019 at 05:20:57PM +0100, Thierry Reding wrote:
> The problem is that if I use dma_alloc_coherent(), then the memory will
> already be mapped via the SMMU at that point and then the driver, not
> knowing that memory has already been mapped, will attempt to map an IOVA
> which will cause an SMMU fault when the host1x tries to access the
> memory.
> 
> I didn't find an equivalent to arm_iommu_detach_device() for non-ARM,
> but then stumbled across this and thought it was rather convenient for
> these cases. If there's a better way to deal with this situation, I'd be
> happy to do so.

So you basically do a dma_direct_alloc + iommu_map?  Can you send me
a pointer to your code?  Maybe we need to add a proper IOMMU-layer
API for that.

Re: [PATCH] dma-direct: Export dma_direct_alloc() and dma_direct_free()

[Thierry Reding]

[-- Attachment #1: Type: text/plain, Size: 1010 bytes --]

On Tue, Feb 05, 2019 at 05:38:37PM +0100, Christoph Hellwig wrote:
> On Tue, Feb 05, 2019 at 05:20:57PM +0100, Thierry Reding wrote:
> > The problem is that if I use dma_alloc_coherent(), then the memory will
> > already be mapped via the SMMU at that point and then the driver, not
> > knowing that memory has already been mapped, will attempt to map an IOVA
> > which will cause an SMMU fault when the host1x tries to access the
> > memory.
> > 
> > I didn't find an equivalent to arm_iommu_detach_device() for non-ARM,
> > but then stumbled across this and thought it was rather convenient for
> > these cases. If there's a better way to deal with this situation, I'd be
> > happy to do so.
> 
> So you basically do a dma_direct_alloc + iommu_map?  Can you send me
> a pointer to your code?  Maybe we need to add a proper IOMMU-layer
> API for that.

Sure, here you go:

	https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/tree/drivers/gpu/host1x/cdma.c#n106

Thierry

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH] dma-direct: Export dma_direct_alloc() and dma_direct_free()

[Christoph Hellwig]

On Tue, Feb 05, 2019 at 06:56:11PM +0100, Thierry Reding wrote:
> Sure, here you go:
> 
> 	https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/tree/drivers/gpu/host1x/cdma.c#n106

Yes, I think we I can come up with a nicer helper for that.

Re: [PATCH] dma-direct: Export dma_direct_alloc() and dma_direct_free()

[Thierry Reding]

[-- Attachment #1: Type: text/plain, Size: 820 bytes --]

On Tue, Feb 05, 2019 at 07:02:18PM +0100, Christoph Hellwig wrote:
> On Tue, Feb 05, 2019 at 06:56:11PM +0100, Thierry Reding wrote:
> > Sure, here you go:
> > 
> > 	https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/tree/drivers/gpu/host1x/cdma.c#n106
> 
> Yes, I think we I can come up with a nicer helper for that.

One thing I could also try is to remove direct IOMMU API usage at least
from the host1x driver. I think this might work nowadays.

For Tegra DRM we still need direct IOMMU API usage because we need to be
able to map into an IOMMU domain without knowing the struct device *
that we're mapping for (it could be needed by any of up to four display
controllers). For host1x we always only have one struct device *, so the
DMA mapping API should be good enough.

Thierry

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH] dma-direct: Export dma_direct_alloc() and dma_direct_free()

[Christoph Hellwig]

On Tue, Feb 05, 2019 at 11:29:12PM +0100, Thierry Reding wrote:
> > > 	https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/tree/drivers/gpu/host1x/cdma.c#n106
> > 
> > Yes, I think we I can come up with a nicer helper for that.
> 
> One thing I could also try is to remove direct IOMMU API usage at least
> from the host1x driver. I think this might work nowadays.
> 
> For Tegra DRM we still need direct IOMMU API usage because we need to be
> able to map into an IOMMU domain without knowing the struct device *
> that we're mapping for (it could be needed by any of up to four display
> controllers). For host1x we always only have one struct device *, so the
> DMA mapping API should be good enough.

If you can convert it to plain DMA API usage, please do.

I did look into an IOMMU API memory allocator, and while we can do it
easily for coherent devices, we need some arch hooks for non-coherent
device support.  With a pending series from me we have those for arm64,
but 32-bit arm support will require a lot more work first.

Re: [PATCH] dma-direct: Export dma_direct_alloc() and dma_direct_free()

[Marek Szyprowski]

Hi Thierry,

On 2019-02-05 23:29, Thierry Reding wrote:
> On Tue, Feb 05, 2019 at 07:02:18PM +0100, Christoph Hellwig wrote:
>> On Tue, Feb 05, 2019 at 06:56:11PM +0100, Thierry Reding wrote:
>>> Sure, here you go:
>>>
>>> 	https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/tree/drivers/gpu/host1x/cdma.c#n106
>> Yes, I think we I can come up with a nicer helper for that.
> One thing I could also try is to remove direct IOMMU API usage at least
> from the host1x driver. I think this might work nowadays.
>
> For Tegra DRM we still need direct IOMMU API usage because we need to be
> able to map into an IOMMU domain without knowing the struct device *
> that we're mapping for (it could be needed by any of up to four display
> controllers). For host1x we always only have one struct device *, so the
> DMA mapping API should be good enough.

In case of Tegra DRM you may try to use Exynos DRM approach. Exynos DRM
also consists of more than one DMA capable device and uses one common
IOMMU address space for them. It is achieved by attaching all those
devices to the IOMMU domain of the first CRTC device that has been
registered. Then that device is used for DMA-mapping calls. It is not
very elegant, but works fine and allows to use standard DMA-mapping calls.

Best regards
-- 
Marek Szyprowski, PhD
Samsung R&D Institute Poland