LKML Archive on lore.kernel.org
 help / color / Atom feed
* [RFC PATCH] iommu/dma: Allow drivers to reserve an iova range
@ 2020-02-14 22:58 Isaac J. Manjarres
  2020-02-17  8:01 ` Christoph Hellwig
  2020-02-19  9:46 ` Joerg Roedel
  0 siblings, 2 replies; 9+ messages in thread
From: Isaac J. Manjarres @ 2020-02-14 22:58 UTC (permalink / raw)
  To: iommu, linux-kernel
  Cc: Liam Mark, joro, pratikp, kernel-team, Isaac J. Manjarres

From: Liam Mark <lmark@codeaurora.org>

Some devices have a memory map which contains gaps or holes.
In order for the device to have as much IOVA space as possible,
allow its driver to inform the DMA-IOMMU layer that it should
not allocate addresses from these holes.

Change-Id: I15bd1d313d889c2572d0eb2adecf6bebde3267f7
Signed-off-by: Isaac J. Manjarres <isaacm@codeaurora.org>
---
 drivers/iommu/dma-iommu.c | 28 ++++++++++++++++++++++++++++
 include/linux/dma-iommu.h |  9 +++++++++
 2 files changed, 37 insertions(+)

diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c
index a2e96a5..3b83e1a 100644
--- a/drivers/iommu/dma-iommu.c
+++ b/drivers/iommu/dma-iommu.c
@@ -368,6 +368,34 @@ static int iommu_dma_deferred_attach(struct device *dev,
 	return 0;
 }
 
+/*
+ * Should be called prior to using dma-apis
+ */
+int iommu_dma_reserve_iova(struct device *dev, dma_addr_t base,
+			   u64 size)
+{
+	struct iommu_domain *domain;
+	struct iommu_dma_cookie *cookie;
+	struct iova_domain *iovad;
+	unsigned long pfn_lo, pfn_hi;
+
+	domain = iommu_get_domain_for_dev(dev);
+	if (!domain || !domain->iova_cookie)
+		return -EINVAL;
+
+	cookie = domain->iova_cookie;
+	iovad = &cookie->iovad;
+
+	/* iova will be freed automatically by put_iova_domain() */
+	pfn_lo = iova_pfn(iovad, base);
+	pfn_hi = iova_pfn(iovad, base + size - 1);
+	if (!reserve_iova(iovad, pfn_lo, pfn_hi))
+		return -EINVAL;
+
+	return 0;
+}
+EXPORT_SYMBOL(iommu_dma_reserve_iova);
+
 /**
  * dma_info_to_prot - Translate DMA API directions and attributes to IOMMU API
  *                    page flags.
diff --git a/include/linux/dma-iommu.h b/include/linux/dma-iommu.h
index 2112f21..79eef7c 100644
--- a/include/linux/dma-iommu.h
+++ b/include/linux/dma-iommu.h
@@ -37,6 +37,9 @@ void iommu_dma_compose_msi_msg(struct msi_desc *desc,
 
 void iommu_dma_get_resv_regions(struct device *dev, struct list_head *list);
 
+int iommu_dma_reserve_iova(struct device *dev, dma_addr_t base,
+			   u64 size);
+
 #else /* CONFIG_IOMMU_DMA */
 
 struct iommu_domain;
@@ -78,5 +81,11 @@ static inline void iommu_dma_get_resv_regions(struct device *dev, struct list_he
 {
 }
 
+static inline int iommu_dma_reserve_iova(struct device *dev, dma_addr_t base,
+					 u64 size)
+{
+	return -ENODEV;
+}
+
 #endif	/* CONFIG_IOMMU_DMA */
 #endif	/* __DMA_IOMMU_H */
-- 
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [RFC PATCH] iommu/dma: Allow drivers to reserve an iova range
  2020-02-14 22:58 [RFC PATCH] iommu/dma: Allow drivers to reserve an iova range Isaac J. Manjarres
@ 2020-02-17  8:01 ` Christoph Hellwig
  2020-02-17 15:50   ` Robin Murphy
  2020-02-19  9:46 ` Joerg Roedel
  1 sibling, 1 reply; 9+ messages in thread
From: Christoph Hellwig @ 2020-02-17  8:01 UTC (permalink / raw)
  To: Isaac J. Manjarres
  Cc: iommu, linux-kernel, Liam Mark, joro, pratikp, kernel-team

On Fri, Feb 14, 2020 at 02:58:16PM -0800, Isaac J. Manjarres wrote:
> From: Liam Mark <lmark@codeaurora.org>
> 
> Some devices have a memory map which contains gaps or holes.
> In order for the device to have as much IOVA space as possible,
> allow its driver to inform the DMA-IOMMU layer that it should
> not allocate addresses from these holes.

Layering violation.  dma-iommu is the translation layer between the
DMA API and the IOMMU API.  And calls into it from drivers performing
DMA mappings need to go through the DMA API (and be documented there).

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [RFC PATCH] iommu/dma: Allow drivers to reserve an iova range
  2020-02-17  8:01 ` Christoph Hellwig
@ 2020-02-17 15:50   ` Robin Murphy
  2020-02-19  1:57     ` isaacm
  0 siblings, 1 reply; 9+ messages in thread
From: Robin Murphy @ 2020-02-17 15:50 UTC (permalink / raw)
  To: Christoph Hellwig, Isaac J. Manjarres
  Cc: pratikp, linux-kernel, Liam Mark, iommu, kernel-team

On 17/02/2020 8:01 am, Christoph Hellwig wrote:
> On Fri, Feb 14, 2020 at 02:58:16PM -0800, Isaac J. Manjarres wrote:
>> From: Liam Mark <lmark@codeaurora.org>
>>
>> Some devices have a memory map which contains gaps or holes.
>> In order for the device to have as much IOVA space as possible,
>> allow its driver to inform the DMA-IOMMU layer that it should
>> not allocate addresses from these holes.
> 
> Layering violation.  dma-iommu is the translation layer between the
> DMA API and the IOMMU API.  And calls into it from drivers performing
> DMA mappings need to go through the DMA API (and be documented there).

+1

More than that, though, we already have "holes in the address space" 
support for the sake of PCI host bridge windows - assuming this is the 
same kind of thing (i.e. the holes are between memory regions and other 
resources in PA space, so are only relevant once address translation 
comes into the picture), then this is IOMMU API level stuff, so even a 
DMA API level interface would be inappropriate.

Robin.

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [RFC PATCH] iommu/dma: Allow drivers to reserve an iova range
  2020-02-17 15:50   ` Robin Murphy
@ 2020-02-19  1:57     ` isaacm
  2020-02-19 11:15       ` Will Deacon
  0 siblings, 1 reply; 9+ messages in thread
From: isaacm @ 2020-02-19  1:57 UTC (permalink / raw)
  To: Robin Murphy
  Cc: Christoph Hellwig, pratikp, linux-kernel, Liam Mark, iommu, kernel-team

On 2020-02-17 07:50, Robin Murphy wrote:
> On 17/02/2020 8:01 am, Christoph Hellwig wrote:
>> On Fri, Feb 14, 2020 at 02:58:16PM -0800, Isaac J. Manjarres wrote:
>>> From: Liam Mark <lmark@codeaurora.org>
>>> 
>>> Some devices have a memory map which contains gaps or holes.
>>> In order for the device to have as much IOVA space as possible,
>>> allow its driver to inform the DMA-IOMMU layer that it should
>>> not allocate addresses from these holes.
>> 
>> Layering violation.  dma-iommu is the translation layer between the
>> DMA API and the IOMMU API.  And calls into it from drivers performing
>> DMA mappings need to go through the DMA API (and be documented there).
> 
> +1
> 
> More than that, though, we already have "holes in the address space"
> support for the sake of PCI host bridge windows - assuming this is the
> same kind of thing (i.e. the holes are between memory regions and
> other resources in PA space, so are only relevant once address
> translation comes into the picture), then this is IOMMU API level
To make sure that we're on the same page, this support alludes to the 
handling in
dma-iommu.c that reserves portions of the IOVA space for the PCI host 
bridge windows,
correct? If so, then yes, this is similar.
> stuff, so even a DMA API level interface would be inappropriate.
Does this mean that the driver should be managing the IOVA space and 
mappings for this device using the IOMMU API? If so, is the rationale 
for this because the device driver can have the information of what IOVA 
ranges can and cannot be used? Shouldn't there be a generic way of 
informing an IOMMU driver about these reserved ranges? Perhaps through a 
device tree property, instead of deferring this type of management to 
the driver?
> 
> Robin.

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [RFC PATCH] iommu/dma: Allow drivers to reserve an iova range
  2020-02-14 22:58 [RFC PATCH] iommu/dma: Allow drivers to reserve an iova range Isaac J. Manjarres
  2020-02-17  8:01 ` Christoph Hellwig
@ 2020-02-19  9:46 ` Joerg Roedel
  1 sibling, 0 replies; 9+ messages in thread
From: Joerg Roedel @ 2020-02-19  9:46 UTC (permalink / raw)
  To: Isaac J. Manjarres; +Cc: iommu, linux-kernel, Liam Mark, pratikp, kernel-team

On Fri, Feb 14, 2020 at 02:58:16PM -0800, Isaac J. Manjarres wrote:
> From: Liam Mark <lmark@codeaurora.org>
> 
> Some devices have a memory map which contains gaps or holes.
> In order for the device to have as much IOVA space as possible,
> allow its driver to inform the DMA-IOMMU layer that it should
> not allocate addresses from these holes.
> 
> Change-Id: I15bd1d313d889c2572d0eb2adecf6bebde3267f7
> Signed-off-by: Isaac J. Manjarres <isaacm@codeaurora.org>

Ideally this is something put into the IOMMU firmware table by the
platform firmware. If its not there, a quirk is the best way to handle
this.

Regards,

	Joerg

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [RFC PATCH] iommu/dma: Allow drivers to reserve an iova range
  2020-02-19  1:57     ` isaacm
@ 2020-02-19 11:15       ` Will Deacon
  2020-02-19 20:06         ` isaacm
  0 siblings, 1 reply; 9+ messages in thread
From: Will Deacon @ 2020-02-19 11:15 UTC (permalink / raw)
  To: isaacm
  Cc: Robin Murphy, Christoph Hellwig, pratikp, linux-kernel,
	Liam Mark, iommu, kernel-team

On Tue, Feb 18, 2020 at 05:57:18PM -0800, isaacm@codeaurora.org wrote:
> On 2020-02-17 07:50, Robin Murphy wrote:
> > On 17/02/2020 8:01 am, Christoph Hellwig wrote:
> > > On Fri, Feb 14, 2020 at 02:58:16PM -0800, Isaac J. Manjarres wrote:
> > > > From: Liam Mark <lmark@codeaurora.org>
> > > > 
> > > > Some devices have a memory map which contains gaps or holes.
> > > > In order for the device to have as much IOVA space as possible,
> > > > allow its driver to inform the DMA-IOMMU layer that it should
> > > > not allocate addresses from these holes.
> > > 
> > > Layering violation.  dma-iommu is the translation layer between the
> > > DMA API and the IOMMU API.  And calls into it from drivers performing
> > > DMA mappings need to go through the DMA API (and be documented there).
> > 
> > +1
> > 
> > More than that, though, we already have "holes in the address space"
> > support for the sake of PCI host bridge windows - assuming this is the
> > same kind of thing (i.e. the holes are between memory regions and
> > other resources in PA space, so are only relevant once address
> > translation comes into the picture), then this is IOMMU API level
> To make sure that we're on the same page, this support alludes to the
> handling in
> dma-iommu.c that reserves portions of the IOVA space for the PCI host bridge
> windows,
> correct? If so, then yes, this is similar.
> > stuff, so even a DMA API level interface would be inappropriate.
> Does this mean that the driver should be managing the IOVA space and
> mappings for this device using the IOMMU API? If so, is the rationale for
> this because the device driver can have the information of what IOVA ranges
> can and cannot be used? Shouldn't there be a generic way of informing an
> IOMMU driver about these reserved ranges? Perhaps through a device tree
> property, instead of deferring this type of management to the driver?

Before we dive into designing that, can you please clarify whether the
reserved IOVA range applies to all DMA masters mastering through a
particular SMMU, or whether it's just about one specific master? I was
assuming the former, but wanted to be sure.

Thanks,

Will

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [RFC PATCH] iommu/dma: Allow drivers to reserve an iova range
  2020-02-19 11:15       ` Will Deacon
@ 2020-02-19 20:06         ` isaacm
  2020-02-20  8:45           ` Will Deacon
  0 siblings, 1 reply; 9+ messages in thread
From: isaacm @ 2020-02-19 20:06 UTC (permalink / raw)
  To: Will Deacon
  Cc: Robin Murphy, Christoph Hellwig, pratikp, linux-kernel,
	Liam Mark, iommu, kernel-team

On 2020-02-19 03:15, Will Deacon wrote:
> On Tue, Feb 18, 2020 at 05:57:18PM -0800, isaacm@codeaurora.org wrote:
>> On 2020-02-17 07:50, Robin Murphy wrote:
>> > On 17/02/2020 8:01 am, Christoph Hellwig wrote:
>> > > On Fri, Feb 14, 2020 at 02:58:16PM -0800, Isaac J. Manjarres wrote:
>> > > > From: Liam Mark <lmark@codeaurora.org>
>> > > >
>> > > > Some devices have a memory map which contains gaps or holes.
>> > > > In order for the device to have as much IOVA space as possible,
>> > > > allow its driver to inform the DMA-IOMMU layer that it should
>> > > > not allocate addresses from these holes.
>> > >
>> > > Layering violation.  dma-iommu is the translation layer between the
>> > > DMA API and the IOMMU API.  And calls into it from drivers performing
>> > > DMA mappings need to go through the DMA API (and be documented there).
>> >
>> > +1
>> >
>> > More than that, though, we already have "holes in the address space"
>> > support for the sake of PCI host bridge windows - assuming this is the
>> > same kind of thing (i.e. the holes are between memory regions and
>> > other resources in PA space, so are only relevant once address
>> > translation comes into the picture), then this is IOMMU API level
>> To make sure that we're on the same page, this support alludes to the
>> handling in
>> dma-iommu.c that reserves portions of the IOVA space for the PCI host 
>> bridge
>> windows,
>> correct? If so, then yes, this is similar.
>> > stuff, so even a DMA API level interface would be inappropriate.
>> Does this mean that the driver should be managing the IOVA space and
>> mappings for this device using the IOMMU API? If so, is the rationale 
>> for
>> this because the device driver can have the information of what IOVA 
>> ranges
>> can and cannot be used? Shouldn't there be a generic way of informing 
>> an
>> IOMMU driver about these reserved ranges? Perhaps through a device 
>> tree
>> property, instead of deferring this type of management to the driver?
> 
> Before we dive into designing that, can you please clarify whether the
> reserved IOVA range applies to all DMA masters mastering through a
> particular SMMU, or whether it's just about one specific master? I was
> assuming the former, but wanted to be sure.
> 
This situation currently applies to one master.
> Thanks,
> 
> Will

Thanks,
Isaac

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [RFC PATCH] iommu/dma: Allow drivers to reserve an iova range
  2020-02-19 20:06         ` isaacm
@ 2020-02-20  8:45           ` Will Deacon
  2020-02-20 18:57             ` Pratik Patel
  0 siblings, 1 reply; 9+ messages in thread
From: Will Deacon @ 2020-02-20  8:45 UTC (permalink / raw)
  To: isaacm
  Cc: Robin Murphy, Christoph Hellwig, pratikp, linux-kernel,
	Liam Mark, iommu, kernel-team

On Wed, Feb 19, 2020 at 12:06:28PM -0800, isaacm@codeaurora.org wrote:
> On 2020-02-19 03:15, Will Deacon wrote:
> > On Tue, Feb 18, 2020 at 05:57:18PM -0800, isaacm@codeaurora.org wrote:
> > > Does this mean that the driver should be managing the IOVA space and
> > > mappings for this device using the IOMMU API? If so, is the
> > > rationale for
> > > this because the device driver can have the information of what IOVA
> > > ranges
> > > can and cannot be used? Shouldn't there be a generic way of
> > > informing an
> > > IOMMU driver about these reserved ranges? Perhaps through a device
> > > tree
> > > property, instead of deferring this type of management to the driver?
> > 
> > Before we dive into designing that, can you please clarify whether the
> > reserved IOVA range applies to all DMA masters mastering through a
> > particular SMMU, or whether it's just about one specific master? I was
> > assuming the former, but wanted to be sure.
> > 
> This situation currently applies to one master.

Interesting. Is it problematic if the range is reserved for all masters
sharing that SMMU?

Will

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [RFC PATCH] iommu/dma: Allow drivers to reserve an iova range
  2020-02-20  8:45           ` Will Deacon
@ 2020-02-20 18:57             ` Pratik Patel
  0 siblings, 0 replies; 9+ messages in thread
From: Pratik Patel @ 2020-02-20 18:57 UTC (permalink / raw)
  To: Will Deacon
  Cc: isaacm, Robin Murphy, Christoph Hellwig, linux-kernel, Liam Mark,
	iommu, kernel-team

On 02/20  08:45 am, Will Deacon wrote:
> On Wed, Feb 19, 2020 at 12:06:28PM -0800, isaacm@codeaurora.org wrote:
> > On 2020-02-19 03:15, Will Deacon wrote:
> > > On Tue, Feb 18, 2020 at 05:57:18PM -0800, isaacm@codeaurora.org wrote:
> > > > Does this mean that the driver should be managing the IOVA space and
> > > > mappings for this device using the IOMMU API? If so, is the
> > > > rationale for
> > > > this because the device driver can have the information of what IOVA
> > > > ranges
> > > > can and cannot be used? Shouldn't there be a generic way of
> > > > informing an
> > > > IOMMU driver about these reserved ranges? Perhaps through a device
> > > > tree
> > > > property, instead of deferring this type of management to the driver?
> > > 
> > > Before we dive into designing that, can you please clarify whether the
> > > reserved IOVA range applies to all DMA masters mastering through a
> > > particular SMMU, or whether it's just about one specific master? I was
> > > assuming the former, but wanted to be sure.
> > > 
> > This situation currently applies to one master.
> 
> Interesting. Is it problematic if the range is reserved for all masters
> sharing that SMMU?
> 
Yes, that would be an overkill. It certainly is useful and in
some cases quite helpful to not waste that range of IOVA space for
other masters on the same SMMU that can use it.

-- 
The Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
a Linux Foundation Collaborative Project 

^ permalink raw reply	[flat|nested] 9+ messages in thread

end of thread, back to index

Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-02-14 22:58 [RFC PATCH] iommu/dma: Allow drivers to reserve an iova range Isaac J. Manjarres
2020-02-17  8:01 ` Christoph Hellwig
2020-02-17 15:50   ` Robin Murphy
2020-02-19  1:57     ` isaacm
2020-02-19 11:15       ` Will Deacon
2020-02-19 20:06         ` isaacm
2020-02-20  8:45           ` Will Deacon
2020-02-20 18:57             ` Pratik Patel
2020-02-19  9:46 ` Joerg Roedel

LKML Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/lkml/0 lkml/git/0.git
	git clone --mirror https://lore.kernel.org/lkml/1 lkml/git/1.git
	git clone --mirror https://lore.kernel.org/lkml/2 lkml/git/2.git
	git clone --mirror https://lore.kernel.org/lkml/3 lkml/git/3.git
	git clone --mirror https://lore.kernel.org/lkml/4 lkml/git/4.git
	git clone --mirror https://lore.kernel.org/lkml/5 lkml/git/5.git
	git clone --mirror https://lore.kernel.org/lkml/6 lkml/git/6.git
	git clone --mirror https://lore.kernel.org/lkml/7 lkml/git/7.git
	git clone --mirror https://lore.kernel.org/lkml/8 lkml/git/8.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 lkml lkml/ https://lore.kernel.org/lkml \
		linux-kernel@vger.kernel.org
	public-inbox-index lkml

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-kernel


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git