* [PATCH v7 1/7] iommu/vt-d: Don't switch off swiotlb if use direct dma
2019-08-23 7:17 [PATCH v7 0/7] iommu: Bounce page for untrusted devices Lu Baolu
@ 2019-08-23 7:17 ` Lu Baolu
2019-08-23 8:39 ` Joerg Roedel
2019-08-23 7:17 ` [PATCH v7 2/7] PCI: Add dev_is_untrusted helper Lu Baolu
` (5 subsequent siblings)
6 siblings, 1 reply; 11+ messages in thread
From: Lu Baolu @ 2019-08-23 7:17 UTC (permalink / raw)
To: David Woodhouse, Joerg Roedel, Bjorn Helgaas, Christoph Hellwig
Cc: Juergen Gross, kevin.tian, Stefano Stabellini, ashok.raj,
Konrad Rzeszutek Wilk, alan.cox, Jonathan Corbet, Robin Murphy,
Steven Rostedt, linux-kernel, iommu, pengfei.xu, Ingo Molnar,
jacob.jun.pan, Greg Kroah-Hartman, Boris Ostrovsky,
mika.westerberg
The direct dma implementation depends on swiotlb. Hence, don't
switch off swiotlb since direct dma interfaces are used in this
driver.
Cc: Ashok Raj <ashok.raj@intel.com>
Cc: Jacob Pan <jacob.jun.pan@linux.intel.com>
Cc: Kevin Tian <kevin.tian@intel.com>
Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
---
drivers/iommu/Kconfig | 1 +
drivers/iommu/intel-iommu.c | 6 ------
2 files changed, 1 insertion(+), 6 deletions(-)
diff --git a/drivers/iommu/Kconfig b/drivers/iommu/Kconfig
index e15cdcd8cb3c..a4ddeade8ac4 100644
--- a/drivers/iommu/Kconfig
+++ b/drivers/iommu/Kconfig
@@ -182,6 +182,7 @@ config INTEL_IOMMU
select IOMMU_IOVA
select NEED_DMA_MAP_STATE
select DMAR_TABLE
+ select SWIOTLB
help
DMA remapping (DMAR) devices support enables independent address
translations for Direct Memory Access (DMA) from devices.
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 12d094d08c0a..8316e57f047c 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -4569,9 +4569,6 @@ static int __init platform_optin_force_iommu(void)
iommu_identity_mapping |= IDENTMAP_ALL;
dmar_disabled = 0;
-#if defined(CONFIG_X86) && defined(CONFIG_SWIOTLB)
- swiotlb = 0;
-#endif
no_iommu = 0;
return 1;
@@ -4710,9 +4707,6 @@ int __init intel_iommu_init(void)
}
up_write(&dmar_global_lock);
-#if defined(CONFIG_X86) && defined(CONFIG_SWIOTLB)
- swiotlb = 0;
-#endif
dma_ops = &intel_dma_ops;
init_iommu_pm_ops();
--
2.17.1
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply related [flat|nested] 11+ messages in thread
* Re: [PATCH v7 1/7] iommu/vt-d: Don't switch off swiotlb if use direct dma
2019-08-23 7:17 ` [PATCH v7 1/7] iommu/vt-d: Don't switch off swiotlb if use direct dma Lu Baolu
@ 2019-08-23 8:39 ` Joerg Roedel
2019-08-24 2:17 ` Lu Baolu
0 siblings, 1 reply; 11+ messages in thread
From: Joerg Roedel @ 2019-08-23 8:39 UTC (permalink / raw)
To: Lu Baolu
Cc: alan.cox, Christoph Hellwig, Stefano Stabellini, ashok.raj,
Jonathan Corbet, pengfei.xu, Ingo Molnar, David Woodhouse,
kevin.tian, Konrad Rzeszutek Wilk, Steven Rostedt, Bjorn Helgaas,
Boris Ostrovsky, mika.westerberg, Juergen Gross,
Greg Kroah-Hartman, linux-kernel, iommu, jacob.jun.pan,
Robin Murphy
On Fri, Aug 23, 2019 at 03:17:29PM +0800, Lu Baolu wrote:
> --- a/drivers/iommu/intel-iommu.c
> +++ b/drivers/iommu/intel-iommu.c
> @@ -4569,9 +4569,6 @@ static int __init platform_optin_force_iommu(void)
> iommu_identity_mapping |= IDENTMAP_ALL;
>
> dmar_disabled = 0;
> -#if defined(CONFIG_X86) && defined(CONFIG_SWIOTLB)
> - swiotlb = 0;
> -#endif
> no_iommu = 0;
>
> return 1;
> @@ -4710,9 +4707,6 @@ int __init intel_iommu_init(void)
> }
> up_write(&dmar_global_lock);
>
> -#if defined(CONFIG_X86) && defined(CONFIG_SWIOTLB)
> - swiotlb = 0;
> -#endif
So this will cause the 64MB SWIOTLB aperture to be allocated even when
there will never be an untrusted device in the system, right? I guess
this will break some kdump setups as they need to resize their low
memory allocations to make room for the aperture because of this
patch-set.
But I also don't see a way around this for now as untrusted devices are
usually hotplugged and might not be present at boot. So we can't make
the decision about the allocation at boot time.
But this mechanism needs to be moved to the dma-iommu implementation at
some point, and then we should allocate the bounce memory pages
on-demand. We can easily do this in page-size chunks and map them
together with iommu page-tables. This way we don't need to pre-allocate
a large memory-chunk at boot.
Regards,
Joerg
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH v7 1/7] iommu/vt-d: Don't switch off swiotlb if use direct dma
2019-08-23 8:39 ` Joerg Roedel
@ 2019-08-24 2:17 ` Lu Baolu
2019-08-30 10:31 ` Joerg Roedel
0 siblings, 1 reply; 11+ messages in thread
From: Lu Baolu @ 2019-08-24 2:17 UTC (permalink / raw)
To: Joerg Roedel
Cc: alan.cox, Christoph Hellwig, Stefano Stabellini, ashok.raj,
Jonathan Corbet, pengfei.xu, Ingo Molnar, David Woodhouse,
kevin.tian, Konrad Rzeszutek Wilk, Steven Rostedt, Bjorn Helgaas,
Boris Ostrovsky, mika.westerberg, Juergen Gross,
Greg Kroah-Hartman, linux-kernel, iommu, jacob.jun.pan,
Robin Murphy
Hi Joerg,
On 8/23/19 4:39 PM, Joerg Roedel wrote:
> On Fri, Aug 23, 2019 at 03:17:29PM +0800, Lu Baolu wrote:
>> --- a/drivers/iommu/intel-iommu.c
>> +++ b/drivers/iommu/intel-iommu.c
>> @@ -4569,9 +4569,6 @@ static int __init platform_optin_force_iommu(void)
>> iommu_identity_mapping |= IDENTMAP_ALL;
>>
>> dmar_disabled = 0;
>> -#if defined(CONFIG_X86) && defined(CONFIG_SWIOTLB)
>> - swiotlb = 0;
>> -#endif
>> no_iommu = 0;
>>
>> return 1;
>> @@ -4710,9 +4707,6 @@ int __init intel_iommu_init(void)
>> }
>> up_write(&dmar_global_lock);
>>
>> -#if defined(CONFIG_X86) && defined(CONFIG_SWIOTLB)
>> - swiotlb = 0;
>> -#endif
>
> So this will cause the 64MB SWIOTLB aperture to be allocated even when
> there will never be an untrusted device in the system, right? I guess
> this will break some kdump setups as they need to resize their low
> memory allocations to make room for the aperture because of this
> patch-set.
Yes, you are right. I didn't consider the kdump case.
>
> But I also don't see a way around this for now as untrusted devices are
> usually hotplugged and might not be present at boot. So we can't make
> the decision about the allocation at boot time.
If a system has any external port, through which an untrusted device
might be connected, the external port itself should be marked as an
untrusted device, and all devices beneath it just inherit this
attribution.
So during iommu driver initialization, we can easily know whether the
system has (or potentially has) untrusted devices by iterating the
device tree. I will add such check in the next version if no objections.
>
> But this mechanism needs to be moved to the dma-iommu implementation at
> some point, and then we should allocate the bounce memory pages
> on-demand. We can easily do this in page-size chunks and map them
> together with iommu page-tables. This way we don't need to pre-allocate
> a large memory-chunk at boot.
>
> Regards,
>
> Joerg
Best regards,
Baolu
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH v7 1/7] iommu/vt-d: Don't switch off swiotlb if use direct dma
2019-08-24 2:17 ` Lu Baolu
@ 2019-08-30 10:31 ` Joerg Roedel
0 siblings, 0 replies; 11+ messages in thread
From: Joerg Roedel @ 2019-08-30 10:31 UTC (permalink / raw)
To: Lu Baolu
Cc: alan.cox, Christoph Hellwig, Stefano Stabellini, ashok.raj,
Jonathan Corbet, pengfei.xu, Ingo Molnar, David Woodhouse,
kevin.tian, Konrad Rzeszutek Wilk, Steven Rostedt, Bjorn Helgaas,
Boris Ostrovsky, mika.westerberg, Juergen Gross,
Greg Kroah-Hartman, linux-kernel, iommu, jacob.jun.pan,
Robin Murphy
On Sat, Aug 24, 2019 at 10:17:30AM +0800, Lu Baolu wrote:
> If a system has any external port, through which an untrusted device
> might be connected, the external port itself should be marked as an
> untrusted device, and all devices beneath it just inherit this
> attribution.
Okay, makes sense.
> So during iommu driver initialization, we can easily know whether the
> system has (or potentially has) untrusted devices by iterating the
> device tree. I will add such check in the next version if no objections.
Sounds good, thanks Baolu.
Joerg
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply [flat|nested] 11+ messages in thread
* [PATCH v7 2/7] PCI: Add dev_is_untrusted helper
2019-08-23 7:17 [PATCH v7 0/7] iommu: Bounce page for untrusted devices Lu Baolu
2019-08-23 7:17 ` [PATCH v7 1/7] iommu/vt-d: Don't switch off swiotlb if use direct dma Lu Baolu
@ 2019-08-23 7:17 ` Lu Baolu
2019-08-23 7:17 ` [PATCH v7 3/7] swiotlb: Split size parameter to map/unmap APIs Lu Baolu
` (4 subsequent siblings)
6 siblings, 0 replies; 11+ messages in thread
From: Lu Baolu @ 2019-08-23 7:17 UTC (permalink / raw)
To: David Woodhouse, Joerg Roedel, Bjorn Helgaas, Christoph Hellwig
Cc: Juergen Gross, kevin.tian, Stefano Stabellini, ashok.raj,
Konrad Rzeszutek Wilk, alan.cox, Jonathan Corbet, Robin Murphy,
Steven Rostedt, linux-kernel, iommu, pengfei.xu, Ingo Molnar,
jacob.jun.pan, Greg Kroah-Hartman, Boris Ostrovsky,
mika.westerberg
There are several places in the kernel where it is necessary to
check whether a device is a pci untrusted device. Add a helper
to simplify the callers.
Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
---
include/linux/pci.h | 2 ++
1 file changed, 2 insertions(+)
diff --git a/include/linux/pci.h b/include/linux/pci.h
index 82e4cd1b7ac3..6c107eb381ac 100644
--- a/include/linux/pci.h
+++ b/include/linux/pci.h
@@ -1029,6 +1029,7 @@ void pcibios_setup_bridge(struct pci_bus *bus, unsigned long type);
void pci_sort_breadthfirst(void);
#define dev_is_pci(d) ((d)->bus == &pci_bus_type)
#define dev_is_pf(d) ((dev_is_pci(d) ? to_pci_dev(d)->is_physfn : false))
+#define dev_is_untrusted(d) ((dev_is_pci(d) ? to_pci_dev(d)->untrusted : false))
/* Generic PCI functions exported to card drivers */
@@ -1768,6 +1769,7 @@ static inline struct pci_dev *pci_dev_get(struct pci_dev *dev) { return NULL; }
#define dev_is_pci(d) (false)
#define dev_is_pf(d) (false)
+#define dev_is_untrusted(d) (false)
static inline bool pci_acs_enabled(struct pci_dev *pdev, u16 acs_flags)
{ return false; }
static inline int pci_irqd_intx_xlate(struct irq_domain *d,
--
2.17.1
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH v7 3/7] swiotlb: Split size parameter to map/unmap APIs
2019-08-23 7:17 [PATCH v7 0/7] iommu: Bounce page for untrusted devices Lu Baolu
2019-08-23 7:17 ` [PATCH v7 1/7] iommu/vt-d: Don't switch off swiotlb if use direct dma Lu Baolu
2019-08-23 7:17 ` [PATCH v7 2/7] PCI: Add dev_is_untrusted helper Lu Baolu
@ 2019-08-23 7:17 ` Lu Baolu
2019-08-23 7:17 ` [PATCH v7 4/7] swiotlb: Zero out bounce buffer for untrusted device Lu Baolu
` (3 subsequent siblings)
6 siblings, 0 replies; 11+ messages in thread
From: Lu Baolu @ 2019-08-23 7:17 UTC (permalink / raw)
To: David Woodhouse, Joerg Roedel, Bjorn Helgaas, Christoph Hellwig
Cc: Juergen Gross, kevin.tian, Stefano Stabellini, ashok.raj,
Konrad Rzeszutek Wilk, alan.cox, Jonathan Corbet, Robin Murphy,
Steven Rostedt, linux-kernel, iommu, pengfei.xu, Ingo Molnar,
jacob.jun.pan, Greg Kroah-Hartman, Boris Ostrovsky,
mika.westerberg
This splits the size parameter to swiotlb_tbl_map_single() and
swiotlb_tbl_unmap_single() into an alloc_size and a mapping_size
parameter, where the latter one is rounded up to the iommu page
size.
Suggested-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
---
drivers/xen/swiotlb-xen.c | 8 ++++----
include/linux/swiotlb.h | 8 ++++++--
kernel/dma/direct.c | 2 +-
kernel/dma/swiotlb.c | 30 +++++++++++++++++++-----------
4 files changed, 30 insertions(+), 18 deletions(-)
diff --git a/drivers/xen/swiotlb-xen.c b/drivers/xen/swiotlb-xen.c
index ae1df496bf38..adcabd9473eb 100644
--- a/drivers/xen/swiotlb-xen.c
+++ b/drivers/xen/swiotlb-xen.c
@@ -386,8 +386,8 @@ static dma_addr_t xen_swiotlb_map_page(struct device *dev, struct page *page,
*/
trace_swiotlb_bounced(dev, dev_addr, size, swiotlb_force);
- map = swiotlb_tbl_map_single(dev, start_dma_addr, phys, size, dir,
- attrs);
+ map = swiotlb_tbl_map_single(dev, start_dma_addr, phys,
+ size, size, dir, attrs);
if (map == (phys_addr_t)DMA_MAPPING_ERROR)
return DMA_MAPPING_ERROR;
@@ -397,7 +397,7 @@ static dma_addr_t xen_swiotlb_map_page(struct device *dev, struct page *page,
* Ensure that the address returned is DMA'ble
*/
if (unlikely(!dma_capable(dev, dev_addr, size))) {
- swiotlb_tbl_unmap_single(dev, map, size, dir,
+ swiotlb_tbl_unmap_single(dev, map, size, size, dir,
attrs | DMA_ATTR_SKIP_CPU_SYNC);
return DMA_MAPPING_ERROR;
}
@@ -433,7 +433,7 @@ static void xen_unmap_single(struct device *hwdev, dma_addr_t dev_addr,
/* NOTE: We use dev_addr here, not paddr! */
if (is_xen_swiotlb_buffer(dev_addr))
- swiotlb_tbl_unmap_single(hwdev, paddr, size, dir, attrs);
+ swiotlb_tbl_unmap_single(hwdev, paddr, size, size, dir, attrs);
}
static void xen_swiotlb_unmap_page(struct device *hwdev, dma_addr_t dev_addr,
diff --git a/include/linux/swiotlb.h b/include/linux/swiotlb.h
index 361f62bb4a8e..cde3dc18e21a 100644
--- a/include/linux/swiotlb.h
+++ b/include/linux/swiotlb.h
@@ -46,13 +46,17 @@ enum dma_sync_target {
extern phys_addr_t swiotlb_tbl_map_single(struct device *hwdev,
dma_addr_t tbl_dma_addr,
- phys_addr_t phys, size_t size,
+ phys_addr_t phys,
+ size_t mapping_size,
+ size_t alloc_size,
enum dma_data_direction dir,
unsigned long attrs);
extern void swiotlb_tbl_unmap_single(struct device *hwdev,
phys_addr_t tlb_addr,
- size_t size, enum dma_data_direction dir,
+ size_t mapping_size,
+ size_t alloc_size,
+ enum dma_data_direction dir,
unsigned long attrs);
extern void swiotlb_tbl_sync_single(struct device *hwdev,
diff --git a/kernel/dma/direct.c b/kernel/dma/direct.c
index 795c9b095d75..a7f2a0163426 100644
--- a/kernel/dma/direct.c
+++ b/kernel/dma/direct.c
@@ -297,7 +297,7 @@ void dma_direct_unmap_page(struct device *dev, dma_addr_t addr,
dma_direct_sync_single_for_cpu(dev, addr, size, dir);
if (unlikely(is_swiotlb_buffer(phys)))
- swiotlb_tbl_unmap_single(dev, phys, size, dir, attrs);
+ swiotlb_tbl_unmap_single(dev, phys, size, size, dir, attrs);
}
EXPORT_SYMBOL(dma_direct_unmap_page);
diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
index 9de232229063..89066efa3840 100644
--- a/kernel/dma/swiotlb.c
+++ b/kernel/dma/swiotlb.c
@@ -444,7 +444,9 @@ static void swiotlb_bounce(phys_addr_t orig_addr, phys_addr_t tlb_addr,
phys_addr_t swiotlb_tbl_map_single(struct device *hwdev,
dma_addr_t tbl_dma_addr,
- phys_addr_t orig_addr, size_t size,
+ phys_addr_t orig_addr,
+ size_t mapping_size,
+ size_t alloc_size,
enum dma_data_direction dir,
unsigned long attrs)
{
@@ -464,6 +466,9 @@ phys_addr_t swiotlb_tbl_map_single(struct device *hwdev,
pr_warn_once("%s is active and system is using DMA bounce buffers\n",
sme_active() ? "SME" : "SEV");
+ if (WARN_ON(mapping_size > alloc_size))
+ return (phys_addr_t)DMA_MAPPING_ERROR;
+
mask = dma_get_seg_boundary(hwdev);
tbl_dma_addr &= mask;
@@ -481,8 +486,8 @@ phys_addr_t swiotlb_tbl_map_single(struct device *hwdev,
* For mappings greater than or equal to a page, we limit the stride
* (and hence alignment) to a page size.
*/
- nslots = ALIGN(size, 1 << IO_TLB_SHIFT) >> IO_TLB_SHIFT;
- if (size >= PAGE_SIZE)
+ nslots = ALIGN(alloc_size, 1 << IO_TLB_SHIFT) >> IO_TLB_SHIFT;
+ if (alloc_size >= PAGE_SIZE)
stride = (1 << (PAGE_SHIFT - IO_TLB_SHIFT));
else
stride = 1;
@@ -547,7 +552,7 @@ phys_addr_t swiotlb_tbl_map_single(struct device *hwdev,
spin_unlock_irqrestore(&io_tlb_lock, flags);
if (!(attrs & DMA_ATTR_NO_WARN) && printk_ratelimit())
dev_warn(hwdev, "swiotlb buffer is full (sz: %zd bytes), total %lu (slots), used %lu (slots)\n",
- size, io_tlb_nslabs, tmp_io_tlb_used);
+ alloc_size, io_tlb_nslabs, tmp_io_tlb_used);
return (phys_addr_t)DMA_MAPPING_ERROR;
found:
io_tlb_used += nslots;
@@ -562,7 +567,7 @@ phys_addr_t swiotlb_tbl_map_single(struct device *hwdev,
io_tlb_orig_addr[index+i] = orig_addr + (i << IO_TLB_SHIFT);
if (!(attrs & DMA_ATTR_SKIP_CPU_SYNC) &&
(dir == DMA_TO_DEVICE || dir == DMA_BIDIRECTIONAL))
- swiotlb_bounce(orig_addr, tlb_addr, size, DMA_TO_DEVICE);
+ swiotlb_bounce(orig_addr, tlb_addr, mapping_size, DMA_TO_DEVICE);
return tlb_addr;
}
@@ -571,21 +576,24 @@ phys_addr_t swiotlb_tbl_map_single(struct device *hwdev,
* tlb_addr is the physical address of the bounce buffer to unmap.
*/
void swiotlb_tbl_unmap_single(struct device *hwdev, phys_addr_t tlb_addr,
- size_t size, enum dma_data_direction dir,
- unsigned long attrs)
+ size_t mapping_size, size_t alloc_size,
+ enum dma_data_direction dir, unsigned long attrs)
{
unsigned long flags;
- int i, count, nslots = ALIGN(size, 1 << IO_TLB_SHIFT) >> IO_TLB_SHIFT;
+ int i, count, nslots = ALIGN(alloc_size, 1 << IO_TLB_SHIFT) >> IO_TLB_SHIFT;
int index = (tlb_addr - io_tlb_start) >> IO_TLB_SHIFT;
phys_addr_t orig_addr = io_tlb_orig_addr[index];
+ if (WARN_ON(mapping_size > alloc_size))
+ return;
+
/*
* First, sync the memory before unmapping the entry
*/
if (orig_addr != INVALID_PHYS_ADDR &&
!(attrs & DMA_ATTR_SKIP_CPU_SYNC) &&
((dir == DMA_FROM_DEVICE) || (dir == DMA_BIDIRECTIONAL)))
- swiotlb_bounce(orig_addr, tlb_addr, size, DMA_FROM_DEVICE);
+ swiotlb_bounce(orig_addr, tlb_addr, mapping_size, DMA_FROM_DEVICE);
/*
* Return the buffer to the free list by setting the corresponding
@@ -665,14 +673,14 @@ bool swiotlb_map(struct device *dev, phys_addr_t *phys, dma_addr_t *dma_addr,
/* Oh well, have to allocate and map a bounce buffer. */
*phys = swiotlb_tbl_map_single(dev, __phys_to_dma(dev, io_tlb_start),
- *phys, size, dir, attrs);
+ *phys, size, size, dir, attrs);
if (*phys == (phys_addr_t)DMA_MAPPING_ERROR)
return false;
/* Ensure that the address returned is DMA'ble */
*dma_addr = __phys_to_dma(dev, *phys);
if (unlikely(!dma_capable(dev, *dma_addr, size))) {
- swiotlb_tbl_unmap_single(dev, *phys, size, dir,
+ swiotlb_tbl_unmap_single(dev, *phys, size, size, dir,
attrs | DMA_ATTR_SKIP_CPU_SYNC);
return false;
}
--
2.17.1
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH v7 4/7] swiotlb: Zero out bounce buffer for untrusted device
2019-08-23 7:17 [PATCH v7 0/7] iommu: Bounce page for untrusted devices Lu Baolu
` (2 preceding siblings ...)
2019-08-23 7:17 ` [PATCH v7 3/7] swiotlb: Split size parameter to map/unmap APIs Lu Baolu
@ 2019-08-23 7:17 ` Lu Baolu
2019-08-23 7:17 ` [PATCH v7 5/7] iommu/vt-d: Check whether device requires bounce buffer Lu Baolu
` (2 subsequent siblings)
6 siblings, 0 replies; 11+ messages in thread
From: Lu Baolu @ 2019-08-23 7:17 UTC (permalink / raw)
To: David Woodhouse, Joerg Roedel, Bjorn Helgaas, Christoph Hellwig
Cc: Juergen Gross, kevin.tian, Stefano Stabellini, ashok.raj,
Konrad Rzeszutek Wilk, alan.cox, Jonathan Corbet, Robin Murphy,
Steven Rostedt, linux-kernel, iommu, pengfei.xu, Ingo Molnar,
jacob.jun.pan, Greg Kroah-Hartman, Boris Ostrovsky,
mika.westerberg
This is necessary to avoid exposing valid kernel data to any
malicious device.
Suggested-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
---
kernel/dma/swiotlb.c | 16 +++++++++++++++-
1 file changed, 15 insertions(+), 1 deletion(-)
diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
index 89066efa3840..04bea5a87462 100644
--- a/kernel/dma/swiotlb.c
+++ b/kernel/dma/swiotlb.c
@@ -35,6 +35,7 @@
#include <linux/scatterlist.h>
#include <linux/mem_encrypt.h>
#include <linux/set_memory.h>
+#include <linux/pci.h>
#ifdef CONFIG_DEBUG_FS
#include <linux/debugfs.h>
#endif
@@ -458,6 +459,8 @@ phys_addr_t swiotlb_tbl_map_single(struct device *hwdev,
unsigned long offset_slots;
unsigned long max_slots;
unsigned long tmp_io_tlb_used;
+ void *zero_addr;
+ size_t zero_size;
if (no_iotlb_memory)
panic("Can not allocate SWIOTLB buffer earlier and can't now provide you with the DMA bounce buffer");
@@ -565,9 +568,20 @@ phys_addr_t swiotlb_tbl_map_single(struct device *hwdev,
*/
for (i = 0; i < nslots; i++)
io_tlb_orig_addr[index+i] = orig_addr + (i << IO_TLB_SHIFT);
+
+ zero_addr = phys_to_virt(tlb_addr);
+ zero_size = alloc_size;
+
if (!(attrs & DMA_ATTR_SKIP_CPU_SYNC) &&
- (dir == DMA_TO_DEVICE || dir == DMA_BIDIRECTIONAL))
+ (dir == DMA_TO_DEVICE || dir == DMA_BIDIRECTIONAL)) {
swiotlb_bounce(orig_addr, tlb_addr, mapping_size, DMA_TO_DEVICE);
+ zero_addr += mapping_size;
+ zero_size -= mapping_size;
+ }
+
+ /* Zero out the bounce buffer if the consumer is untrusted. */
+ if (dev_is_untrusted(hwdev) && zero_size)
+ memset(zero_addr, 0, zero_size);
return tlb_addr;
}
--
2.17.1
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH v7 5/7] iommu/vt-d: Check whether device requires bounce buffer
2019-08-23 7:17 [PATCH v7 0/7] iommu: Bounce page for untrusted devices Lu Baolu
` (3 preceding siblings ...)
2019-08-23 7:17 ` [PATCH v7 4/7] swiotlb: Zero out bounce buffer for untrusted device Lu Baolu
@ 2019-08-23 7:17 ` Lu Baolu
2019-08-23 7:17 ` [PATCH v7 6/7] iommu/vt-d: Add trace events for device dma map/unmap Lu Baolu
2019-08-23 7:17 ` [PATCH v7 7/7] iommu/vt-d: Use bounce buffer for untrusted devices Lu Baolu
6 siblings, 0 replies; 11+ messages in thread
From: Lu Baolu @ 2019-08-23 7:17 UTC (permalink / raw)
To: David Woodhouse, Joerg Roedel, Bjorn Helgaas, Christoph Hellwig
Cc: Juergen Gross, kevin.tian, Stefano Stabellini, ashok.raj,
Konrad Rzeszutek Wilk, alan.cox, Jonathan Corbet, Robin Murphy,
Steven Rostedt, linux-kernel, iommu, pengfei.xu, Ingo Molnar,
jacob.jun.pan, Greg Kroah-Hartman, Boris Ostrovsky,
mika.westerberg
This adds a helper to check whether a device needs to
use bounce buffer. It also provides a boot time option
to disable the bounce buffer. Users can use this to
prevent the iommu driver from using the bounce buffer
for performance gain.
Cc: Ashok Raj <ashok.raj@intel.com>
Cc: Jacob Pan <jacob.jun.pan@linux.intel.com>
Cc: Kevin Tian <kevin.tian@intel.com>
Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
Tested-by: Xu Pengfei <pengfei.xu@intel.com>
Tested-by: Mika Westerberg <mika.westerberg@intel.com>
---
Documentation/admin-guide/kernel-parameters.txt | 5 +++++
drivers/iommu/intel-iommu.c | 6 ++++++
2 files changed, 11 insertions(+)
diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index 47d981a86e2f..aaca73080097 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -1732,6 +1732,11 @@
Note that using this option lowers the security
provided by tboot because it makes the system
vulnerable to DMA attacks.
+ nobounce [Default off]
+ Disable bounce buffer for unstrusted devices such as
+ the Thunderbolt devices. This will treat the untrusted
+ devices as the trusted ones, hence might expose security
+ risks of DMA attacks.
intel_idle.max_cstate= [KNL,HW,ACPI,X86]
0 disables intel_idle and fall back on acpi_idle.
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 8316e57f047c..b1ab3526e6fa 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -360,6 +360,7 @@ static int dmar_forcedac;
static int intel_iommu_strict;
static int intel_iommu_superpage = 1;
static int iommu_identity_mapping;
+static int intel_no_bounce;
#define IDENTMAP_ALL 1
#define IDENTMAP_GFX 2
@@ -373,6 +374,8 @@ EXPORT_SYMBOL_GPL(intel_iommu_gfx_mapped);
static DEFINE_SPINLOCK(device_domain_lock);
static LIST_HEAD(device_domain_list);
+#define device_needs_bounce(d) (!intel_no_bounce && dev_is_untrusted(d))
+
/*
* Iterate over elements in device_domain_list and call the specified
* callback @fn against each element.
@@ -455,6 +458,9 @@ static int __init intel_iommu_setup(char *str)
printk(KERN_INFO
"Intel-IOMMU: not forcing on after tboot. This could expose security risk for tboot\n");
intel_iommu_tboot_noforce = 1;
+ } else if (!strncmp(str, "nobounce", 8)) {
+ pr_info("Intel-IOMMU: No bounce buffer. This could expose security risks of DMA attacks\n");
+ intel_no_bounce = 1;
}
str += strcspn(str, ",");
--
2.17.1
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH v7 6/7] iommu/vt-d: Add trace events for device dma map/unmap
2019-08-23 7:17 [PATCH v7 0/7] iommu: Bounce page for untrusted devices Lu Baolu
` (4 preceding siblings ...)
2019-08-23 7:17 ` [PATCH v7 5/7] iommu/vt-d: Check whether device requires bounce buffer Lu Baolu
@ 2019-08-23 7:17 ` Lu Baolu
2019-08-23 7:17 ` [PATCH v7 7/7] iommu/vt-d: Use bounce buffer for untrusted devices Lu Baolu
6 siblings, 0 replies; 11+ messages in thread
From: Lu Baolu @ 2019-08-23 7:17 UTC (permalink / raw)
To: David Woodhouse, Joerg Roedel, Bjorn Helgaas, Christoph Hellwig
Cc: Juergen Gross, kevin.tian, Stefano Stabellini, ashok.raj,
Konrad Rzeszutek Wilk, alan.cox, Jonathan Corbet, Robin Murphy,
Steven Rostedt, linux-kernel, iommu, pengfei.xu, Ingo Molnar,
jacob.jun.pan, Greg Kroah-Hartman, Boris Ostrovsky,
mika.westerberg
This adds trace support for the Intel IOMMU driver. It
also declares some events which could be used to trace
the events when an IOVA is being mapped or unmapped in
a domain.
Cc: Ashok Raj <ashok.raj@intel.com>
Cc: Jacob Pan <jacob.jun.pan@linux.intel.com>
Cc: Kevin Tian <kevin.tian@intel.com>
Signed-off-by: Mika Westerberg <mika.westerberg@linux.intel.com>
Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
Reviewed-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
---
drivers/iommu/Makefile | 1 +
drivers/iommu/intel-trace.c | 14 +++++
include/trace/events/intel_iommu.h | 84 ++++++++++++++++++++++++++++++
3 files changed, 99 insertions(+)
create mode 100644 drivers/iommu/intel-trace.c
create mode 100644 include/trace/events/intel_iommu.h
diff --git a/drivers/iommu/Makefile b/drivers/iommu/Makefile
index f13f36ae1af6..bfe27b2755bd 100644
--- a/drivers/iommu/Makefile
+++ b/drivers/iommu/Makefile
@@ -17,6 +17,7 @@ obj-$(CONFIG_ARM_SMMU) += arm-smmu.o
obj-$(CONFIG_ARM_SMMU_V3) += arm-smmu-v3.o
obj-$(CONFIG_DMAR_TABLE) += dmar.o
obj-$(CONFIG_INTEL_IOMMU) += intel-iommu.o intel-pasid.o
+obj-$(CONFIG_INTEL_IOMMU) += intel-trace.o
obj-$(CONFIG_INTEL_IOMMU_DEBUGFS) += intel-iommu-debugfs.o
obj-$(CONFIG_INTEL_IOMMU_SVM) += intel-svm.o
obj-$(CONFIG_IPMMU_VMSA) += ipmmu-vmsa.o
diff --git a/drivers/iommu/intel-trace.c b/drivers/iommu/intel-trace.c
new file mode 100644
index 000000000000..bfb6a6e37a88
--- /dev/null
+++ b/drivers/iommu/intel-trace.c
@@ -0,0 +1,14 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Intel IOMMU trace support
+ *
+ * Copyright (C) 2019 Intel Corporation
+ *
+ * Author: Lu Baolu <baolu.lu@linux.intel.com>
+ */
+
+#include <linux/string.h>
+#include <linux/types.h>
+
+#define CREATE_TRACE_POINTS
+#include <trace/events/intel_iommu.h>
diff --git a/include/trace/events/intel_iommu.h b/include/trace/events/intel_iommu.h
new file mode 100644
index 000000000000..9c28e6cae86f
--- /dev/null
+++ b/include/trace/events/intel_iommu.h
@@ -0,0 +1,84 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Intel IOMMU trace support
+ *
+ * Copyright (C) 2019 Intel Corporation
+ *
+ * Author: Lu Baolu <baolu.lu@linux.intel.com>
+ */
+#ifdef CONFIG_INTEL_IOMMU
+#undef TRACE_SYSTEM
+#define TRACE_SYSTEM intel_iommu
+
+#if !defined(_TRACE_INTEL_IOMMU_H) || defined(TRACE_HEADER_MULTI_READ)
+#define _TRACE_INTEL_IOMMU_H
+
+#include <linux/tracepoint.h>
+#include <linux/intel-iommu.h>
+
+DECLARE_EVENT_CLASS(dma_map,
+ TP_PROTO(struct device *dev, dma_addr_t dev_addr, phys_addr_t phys_addr,
+ size_t size),
+
+ TP_ARGS(dev, dev_addr, phys_addr, size),
+
+ TP_STRUCT__entry(
+ __string(dev_name, dev_name(dev))
+ __field(dma_addr_t, dev_addr)
+ __field(phys_addr_t, phys_addr)
+ __field(size_t, size)
+ ),
+
+ TP_fast_assign(
+ __assign_str(dev_name, dev_name(dev));
+ __entry->dev_addr = dev_addr;
+ __entry->phys_addr = phys_addr;
+ __entry->size = size;
+ ),
+
+ TP_printk("dev=%s dev_addr=0x%llx phys_addr=0x%llx size=%zu",
+ __get_str(dev_name),
+ (unsigned long long)__entry->dev_addr,
+ (unsigned long long)__entry->phys_addr,
+ __entry->size)
+);
+
+DEFINE_EVENT(dma_map, bounce_map_single,
+ TP_PROTO(struct device *dev, dma_addr_t dev_addr, phys_addr_t phys_addr,
+ size_t size),
+ TP_ARGS(dev, dev_addr, phys_addr, size)
+);
+
+DECLARE_EVENT_CLASS(dma_unmap,
+ TP_PROTO(struct device *dev, dma_addr_t dev_addr, size_t size),
+
+ TP_ARGS(dev, dev_addr, size),
+
+ TP_STRUCT__entry(
+ __string(dev_name, dev_name(dev))
+ __field(dma_addr_t, dev_addr)
+ __field(size_t, size)
+ ),
+
+ TP_fast_assign(
+ __assign_str(dev_name, dev_name(dev));
+ __entry->dev_addr = dev_addr;
+ __entry->size = size;
+ ),
+
+ TP_printk("dev=%s dev_addr=0x%llx size=%zu",
+ __get_str(dev_name),
+ (unsigned long long)__entry->dev_addr,
+ __entry->size)
+);
+
+DEFINE_EVENT(dma_unmap, bounce_unmap_single,
+ TP_PROTO(struct device *dev, dma_addr_t dev_addr, size_t size),
+ TP_ARGS(dev, dev_addr, size)
+);
+
+#endif /* _TRACE_INTEL_IOMMU_H */
+
+/* This part must be outside protection */
+#include <trace/define_trace.h>
+#endif /* CONFIG_INTEL_IOMMU */
--
2.17.1
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH v7 7/7] iommu/vt-d: Use bounce buffer for untrusted devices
2019-08-23 7:17 [PATCH v7 0/7] iommu: Bounce page for untrusted devices Lu Baolu
` (5 preceding siblings ...)
2019-08-23 7:17 ` [PATCH v7 6/7] iommu/vt-d: Add trace events for device dma map/unmap Lu Baolu
@ 2019-08-23 7:17 ` Lu Baolu
6 siblings, 0 replies; 11+ messages in thread
From: Lu Baolu @ 2019-08-23 7:17 UTC (permalink / raw)
To: David Woodhouse, Joerg Roedel, Bjorn Helgaas, Christoph Hellwig
Cc: Juergen Gross, kevin.tian, Stefano Stabellini, ashok.raj,
Konrad Rzeszutek Wilk, alan.cox, Jonathan Corbet, Robin Murphy,
Steven Rostedt, linux-kernel, iommu, pengfei.xu, Ingo Molnar,
jacob.jun.pan, Greg Kroah-Hartman, Boris Ostrovsky,
mika.westerberg
The Intel VT-d hardware uses paging for DMA remapping.
The minimum mapped window is a page size. The device
drivers may map buffers not filling the whole IOMMU
window. This allows the device to access to possibly
unrelated memory and a malicious device could exploit
this to perform DMA attacks. To address this, the
Intel IOMMU driver will use bounce pages for those
buffers which don't fill whole IOMMU pages.
Cc: Ashok Raj <ashok.raj@intel.com>
Cc: Jacob Pan <jacob.jun.pan@linux.intel.com>
Cc: Kevin Tian <kevin.tian@intel.com>
Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
Tested-by: Xu Pengfei <pengfei.xu@intel.com>
Tested-by: Mika Westerberg <mika.westerberg@intel.com>
---
drivers/iommu/intel-iommu.c | 256 ++++++++++++++++++++++++++++++++++++
1 file changed, 256 insertions(+)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index b1ab3526e6fa..1f15c6f5880f 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -41,9 +41,11 @@
#include <linux/dma-direct.h>
#include <linux/crash_dump.h>
#include <linux/numa.h>
+#include <linux/swiotlb.h>
#include <asm/irq_remapping.h>
#include <asm/cacheflush.h>
#include <asm/iommu.h>
+#include <trace/events/intel_iommu.h>
#include "irq_remapping.h"
#include "intel-pasid.h"
@@ -3746,6 +3748,252 @@ static const struct dma_map_ops intel_dma_ops = {
.dma_supported = dma_direct_supported,
};
+static inline phys_addr_t
+iova_to_tlb_addr(struct dmar_domain *domain, dma_addr_t addr)
+{
+ struct dma_pte *pte;
+ int level = 0;
+ u64 phys = 0;
+
+ pte = pfn_to_dma_pte(domain, addr >> VTD_PAGE_SHIFT, &level);
+ if (pte)
+ phys = dma_pte_addr(pte);
+
+ return phys;
+}
+
+static void
+bounce_sync_single(struct device *dev, dma_addr_t addr, size_t size,
+ enum dma_data_direction dir, enum dma_sync_target target)
+{
+ struct dmar_domain *domain;
+ phys_addr_t tlb_addr;
+
+ domain = find_domain(dev);
+ if (WARN_ON(!domain))
+ return;
+
+ tlb_addr = iova_to_tlb_addr(domain, addr);
+ if (is_swiotlb_buffer(tlb_addr))
+ swiotlb_tbl_sync_single(dev, tlb_addr, size, dir, target);
+}
+
+static dma_addr_t
+bounce_map_single(struct device *dev, phys_addr_t paddr, size_t size,
+ enum dma_data_direction dir, unsigned long attrs,
+ u64 dma_mask)
+{
+ size_t aligned_size = ALIGN(size, VTD_PAGE_SIZE);
+ struct dmar_domain *domain;
+ struct intel_iommu *iommu;
+ unsigned long iova_pfn;
+ unsigned long nrpages;
+ phys_addr_t tlb_addr;
+ int prot = 0;
+ int ret;
+
+ domain = find_domain(dev);
+ if (WARN_ON(dir == DMA_NONE || !domain))
+ return DMA_MAPPING_ERROR;
+
+ iommu = domain_get_iommu(domain);
+ if (WARN_ON(!iommu))
+ return DMA_MAPPING_ERROR;
+
+ nrpages = aligned_nrpages(0, size);
+ iova_pfn = intel_alloc_iova(dev, domain,
+ dma_to_mm_pfn(nrpages), dma_mask);
+ if (!iova_pfn)
+ return DMA_MAPPING_ERROR;
+
+ /*
+ * Check if DMAR supports zero-length reads on write only
+ * mappings..
+ */
+ if (dir == DMA_TO_DEVICE || dir == DMA_BIDIRECTIONAL ||
+ !cap_zlr(iommu->cap))
+ prot |= DMA_PTE_READ;
+ if (dir == DMA_FROM_DEVICE || dir == DMA_BIDIRECTIONAL)
+ prot |= DMA_PTE_WRITE;
+
+ /*
+ * If both the physical buffer start address and size are
+ * page aligned, we don't need to use a bounce page.
+ */
+ if (!IS_ALIGNED(paddr | size, VTD_PAGE_SIZE)) {
+ tlb_addr = swiotlb_tbl_map_single(dev,
+ __phys_to_dma(dev, io_tlb_start),
+ paddr, size, aligned_size, dir, attrs);
+ if (tlb_addr == DMA_MAPPING_ERROR)
+ goto swiotlb_error;
+ } else {
+ tlb_addr = paddr;
+ }
+
+ ret = domain_pfn_mapping(domain, mm_to_dma_pfn(iova_pfn),
+ tlb_addr >> VTD_PAGE_SHIFT, nrpages, prot);
+ if (ret)
+ goto mapping_error;
+
+ trace_bounce_map_single(dev, iova_pfn << PAGE_SHIFT, paddr, size);
+
+ return (phys_addr_t)iova_pfn << PAGE_SHIFT;
+
+mapping_error:
+ if (is_swiotlb_buffer(tlb_addr))
+ swiotlb_tbl_unmap_single(dev, tlb_addr, size,
+ aligned_size, dir, attrs);
+swiotlb_error:
+ free_iova_fast(&domain->iovad, iova_pfn, dma_to_mm_pfn(nrpages));
+ dev_err(dev, "Device bounce map: %zx@%llx dir %d --- failed\n",
+ size, (unsigned long long)paddr, dir);
+
+ return DMA_MAPPING_ERROR;
+}
+
+static void
+bounce_unmap_single(struct device *dev, dma_addr_t dev_addr, size_t size,
+ enum dma_data_direction dir, unsigned long attrs)
+{
+ size_t aligned_size = ALIGN(size, VTD_PAGE_SIZE);
+ struct dmar_domain *domain;
+ phys_addr_t tlb_addr;
+
+ domain = find_domain(dev);
+ if (WARN_ON(!domain))
+ return;
+
+ tlb_addr = iova_to_tlb_addr(domain, dev_addr);
+ if (WARN_ON(!tlb_addr))
+ return;
+
+ intel_unmap(dev, dev_addr, size);
+ if (is_swiotlb_buffer(tlb_addr))
+ swiotlb_tbl_unmap_single(dev, tlb_addr, size,
+ aligned_size, dir, attrs);
+
+ trace_bounce_unmap_single(dev, dev_addr, size);
+}
+
+static dma_addr_t
+bounce_map_page(struct device *dev, struct page *page, unsigned long offset,
+ size_t size, enum dma_data_direction dir, unsigned long attrs)
+{
+ return bounce_map_single(dev, page_to_phys(page) + offset,
+ size, dir, attrs, *dev->dma_mask);
+}
+
+static dma_addr_t
+bounce_map_resource(struct device *dev, phys_addr_t phys_addr, size_t size,
+ enum dma_data_direction dir, unsigned long attrs)
+{
+ return bounce_map_single(dev, phys_addr, size,
+ dir, attrs, *dev->dma_mask);
+}
+
+static void
+bounce_unmap_page(struct device *dev, dma_addr_t dev_addr, size_t size,
+ enum dma_data_direction dir, unsigned long attrs)
+{
+ bounce_unmap_single(dev, dev_addr, size, dir, attrs);
+}
+
+static void
+bounce_unmap_resource(struct device *dev, dma_addr_t dev_addr, size_t size,
+ enum dma_data_direction dir, unsigned long attrs)
+{
+ bounce_unmap_single(dev, dev_addr, size, dir, attrs);
+}
+
+static void
+bounce_unmap_sg(struct device *dev, struct scatterlist *sglist, int nelems,
+ enum dma_data_direction dir, unsigned long attrs)
+{
+ struct scatterlist *sg;
+ int i;
+
+ for_each_sg(sglist, sg, nelems, i)
+ bounce_unmap_page(dev, sg->dma_address,
+ sg_dma_len(sg), dir, attrs);
+}
+
+static int
+bounce_map_sg(struct device *dev, struct scatterlist *sglist, int nelems,
+ enum dma_data_direction dir, unsigned long attrs)
+{
+ int i;
+ struct scatterlist *sg;
+
+ for_each_sg(sglist, sg, nelems, i) {
+ sg->dma_address = bounce_map_page(dev, sg_page(sg),
+ sg->offset, sg->length,
+ dir, attrs);
+ if (sg->dma_address == DMA_MAPPING_ERROR)
+ goto out_unmap;
+ sg_dma_len(sg) = sg->length;
+ }
+
+ return nelems;
+
+out_unmap:
+ bounce_unmap_sg(dev, sglist, i, dir, attrs | DMA_ATTR_SKIP_CPU_SYNC);
+ return 0;
+}
+
+static void
+bounce_sync_single_for_cpu(struct device *dev, dma_addr_t addr,
+ size_t size, enum dma_data_direction dir)
+{
+ bounce_sync_single(dev, addr, size, dir, SYNC_FOR_CPU);
+}
+
+static void
+bounce_sync_single_for_device(struct device *dev, dma_addr_t addr,
+ size_t size, enum dma_data_direction dir)
+{
+ bounce_sync_single(dev, addr, size, dir, SYNC_FOR_DEVICE);
+}
+
+static void
+bounce_sync_sg_for_cpu(struct device *dev, struct scatterlist *sglist,
+ int nelems, enum dma_data_direction dir)
+{
+ struct scatterlist *sg;
+ int i;
+
+ for_each_sg(sglist, sg, nelems, i)
+ bounce_sync_single(dev, sg_dma_address(sg),
+ sg_dma_len(sg), dir, SYNC_FOR_CPU);
+}
+
+static void
+bounce_sync_sg_for_device(struct device *dev, struct scatterlist *sglist,
+ int nelems, enum dma_data_direction dir)
+{
+ struct scatterlist *sg;
+ int i;
+
+ for_each_sg(sglist, sg, nelems, i)
+ bounce_sync_single(dev, sg_dma_address(sg),
+ sg_dma_len(sg), dir, SYNC_FOR_DEVICE);
+}
+
+static const struct dma_map_ops bounce_dma_ops = {
+ .alloc = intel_alloc_coherent,
+ .free = intel_free_coherent,
+ .map_sg = bounce_map_sg,
+ .unmap_sg = bounce_unmap_sg,
+ .map_page = bounce_map_page,
+ .unmap_page = bounce_unmap_page,
+ .sync_single_for_cpu = bounce_sync_single_for_cpu,
+ .sync_single_for_device = bounce_sync_single_for_device,
+ .sync_sg_for_cpu = bounce_sync_sg_for_cpu,
+ .sync_sg_for_device = bounce_sync_sg_for_device,
+ .map_resource = bounce_map_resource,
+ .unmap_resource = bounce_unmap_resource,
+ .dma_supported = dma_direct_supported,
+};
+
static inline int iommu_domain_cache_init(void)
{
int ret = 0;
@@ -5309,6 +5557,11 @@ static int intel_iommu_add_device(struct device *dev)
}
}
+ if (device_needs_bounce(dev)) {
+ dev_info(dev, "Use Intel IOMMU bounce page dma_ops\n");
+ set_dma_ops(dev, &bounce_dma_ops);
+ }
+
return 0;
}
@@ -5326,6 +5579,9 @@ static void intel_iommu_remove_device(struct device *dev)
iommu_group_remove_device(dev);
iommu_device_unlink(&iommu->iommu, dev);
+
+ if (device_needs_bounce(dev))
+ set_dma_ops(dev, NULL);
}
static void intel_iommu_get_resv_regions(struct device *device,
--
2.17.1
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply related [flat|nested] 11+ messages in thread