* swiotlb cleanups
@ 2021-02-07 16:09 Christoph Hellwig
2021-02-07 16:09 ` [PATCH 1/8] powerpc/svm: stop using io_tlb_start Christoph Hellwig
` (7 more replies)
0 siblings, 8 replies; 11+ messages in thread
From: Christoph Hellwig @ 2021-02-07 16:09 UTC (permalink / raw)
To: Konrad Rzeszutek Wilk
Cc: Michael Ellerman, Dongli Zhang, Claire Chang, xen-devel,
linuxppc-dev, iommu
Hi Konrad,
this series contains a bunch of swiotlb cleanups, mostly to reduce the
amount of internals exposed to code outside of swiotlb.c, which should
helper to prepare for supporting multiple different bounce buffer pools.
^ permalink raw reply [flat|nested] 11+ messages in thread
* [PATCH 1/8] powerpc/svm: stop using io_tlb_start
2021-02-07 16:09 swiotlb cleanups Christoph Hellwig
@ 2021-02-07 16:09 ` Christoph Hellwig
2021-02-07 16:09 ` [PATCH 2/8] xen-swiotlb: use is_swiotlb_buffer in is_xen_swiotlb_buffer Christoph Hellwig
` (6 subsequent siblings)
7 siblings, 0 replies; 11+ messages in thread
From: Christoph Hellwig @ 2021-02-07 16:09 UTC (permalink / raw)
To: Konrad Rzeszutek Wilk
Cc: Michael Ellerman, Dongli Zhang, Claire Chang, xen-devel,
linuxppc-dev, iommu
Use the local variable that is passed to swiotlb_init_with_tbl for
freeing the memory in the failure case to isolate the code a little
better from swiotlb internals.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
arch/powerpc/platforms/pseries/svm.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/arch/powerpc/platforms/pseries/svm.c b/arch/powerpc/platforms/pseries/svm.c
index 7b739cc7a8a93e..b9968ac7cc0789 100644
--- a/arch/powerpc/platforms/pseries/svm.c
+++ b/arch/powerpc/platforms/pseries/svm.c
@@ -56,7 +56,7 @@ void __init svm_swiotlb_init(void)
return;
if (io_tlb_start)
- memblock_free_early(io_tlb_start,
+ memblock_free_early(__pa(vstart),
PAGE_ALIGN(io_tlb_nslabs << IO_TLB_SHIFT));
panic("SVM: Cannot allocate SWIOTLB buffer");
}
--
2.29.2
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH 2/8] xen-swiotlb: use is_swiotlb_buffer in is_xen_swiotlb_buffer
2021-02-07 16:09 swiotlb cleanups Christoph Hellwig
2021-02-07 16:09 ` [PATCH 1/8] powerpc/svm: stop using io_tlb_start Christoph Hellwig
@ 2021-02-07 16:09 ` Christoph Hellwig
2021-02-19 20:43 ` Konrad Rzeszutek Wilk
2021-02-07 16:09 ` [PATCH 3/8] xen-swiotlb: use io_tlb_end in xen_swiotlb_dma_supported Christoph Hellwig
` (5 subsequent siblings)
7 siblings, 1 reply; 11+ messages in thread
From: Christoph Hellwig @ 2021-02-07 16:09 UTC (permalink / raw)
To: Konrad Rzeszutek Wilk
Cc: Michael Ellerman, Dongli Zhang, Claire Chang, xen-devel,
linuxppc-dev, iommu
Use the is_swiotlb_buffer to check if a physical address is
a swiotlb buffer. This works because xen-swiotlb does use the
same buffer as the main swiotlb code, and xen_io_tlb_{start,end}
are just the addresses for it that went through phys_to_virt.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
drivers/xen/swiotlb-xen.c | 6 ++----
1 file changed, 2 insertions(+), 4 deletions(-)
diff --git a/drivers/xen/swiotlb-xen.c b/drivers/xen/swiotlb-xen.c
index 2b385c1b4a99cb..a4026822a889f7 100644
--- a/drivers/xen/swiotlb-xen.c
+++ b/drivers/xen/swiotlb-xen.c
@@ -111,10 +111,8 @@ static int is_xen_swiotlb_buffer(struct device *dev, dma_addr_t dma_addr)
* have the same virtual address as another address
* in our domain. Therefore _only_ check address within our domain.
*/
- if (pfn_valid(PFN_DOWN(paddr))) {
- return paddr >= virt_to_phys(xen_io_tlb_start) &&
- paddr < virt_to_phys(xen_io_tlb_end);
- }
+ if (pfn_valid(PFN_DOWN(paddr)))
+ return is_swiotlb_buffer(paddr);
return 0;
}
--
2.29.2
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH 3/8] xen-swiotlb: use io_tlb_end in xen_swiotlb_dma_supported
2021-02-07 16:09 swiotlb cleanups Christoph Hellwig
2021-02-07 16:09 ` [PATCH 1/8] powerpc/svm: stop using io_tlb_start Christoph Hellwig
2021-02-07 16:09 ` [PATCH 2/8] xen-swiotlb: use is_swiotlb_buffer in is_xen_swiotlb_buffer Christoph Hellwig
@ 2021-02-07 16:09 ` Christoph Hellwig
2021-02-19 21:00 ` Konrad Rzeszutek Wilk
2021-02-07 16:09 ` [PATCH 4/8] xen-swiotlb: remove xen_set_nslabs Christoph Hellwig
` (4 subsequent siblings)
7 siblings, 1 reply; 11+ messages in thread
From: Christoph Hellwig @ 2021-02-07 16:09 UTC (permalink / raw)
To: Konrad Rzeszutek Wilk
Cc: Michael Ellerman, Dongli Zhang, Claire Chang, xen-devel,
linuxppc-dev, iommu
Use the existing variable that holds the physical address for
xen_io_tlb_end to simplify xen_swiotlb_dma_supported a bit, and remove
the otherwise unused xen_io_tlb_end variable and the xen_virt_to_bus
helper.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
drivers/xen/swiotlb-xen.c | 10 ++--------
1 file changed, 2 insertions(+), 8 deletions(-)
diff --git a/drivers/xen/swiotlb-xen.c b/drivers/xen/swiotlb-xen.c
index a4026822a889f7..4298f74a083985 100644
--- a/drivers/xen/swiotlb-xen.c
+++ b/drivers/xen/swiotlb-xen.c
@@ -46,7 +46,7 @@
* API.
*/
-static char *xen_io_tlb_start, *xen_io_tlb_end;
+static char *xen_io_tlb_start;
static unsigned long xen_io_tlb_nslabs;
/*
* Quick lookup value of the bus address of the IOTLB.
@@ -82,11 +82,6 @@ static inline phys_addr_t xen_dma_to_phys(struct device *dev,
return xen_bus_to_phys(dev, dma_to_phys(dev, dma_addr));
}
-static inline dma_addr_t xen_virt_to_bus(struct device *dev, void *address)
-{
- return xen_phys_to_dma(dev, virt_to_phys(address));
-}
-
static inline int range_straddles_page_boundary(phys_addr_t p, size_t size)
{
unsigned long next_bfn, xen_pfn = XEN_PFN_DOWN(p);
@@ -250,7 +245,6 @@ int __ref xen_swiotlb_init(int verbose, bool early)
rc = swiotlb_late_init_with_tbl(xen_io_tlb_start, xen_io_tlb_nslabs);
end:
- xen_io_tlb_end = xen_io_tlb_start + bytes;
if (!rc)
swiotlb_set_max_segment(PAGE_SIZE);
@@ -558,7 +552,7 @@ xen_swiotlb_sync_sg_for_device(struct device *dev, struct scatterlist *sgl,
static int
xen_swiotlb_dma_supported(struct device *hwdev, u64 mask)
{
- return xen_virt_to_bus(hwdev, xen_io_tlb_end - 1) <= mask;
+ return xen_phys_to_dma(hwdev, io_tlb_end - 1) <= mask;
}
const struct dma_map_ops xen_swiotlb_dma_ops = {
--
2.29.2
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH 4/8] xen-swiotlb: remove xen_set_nslabs
2021-02-07 16:09 swiotlb cleanups Christoph Hellwig
` (2 preceding siblings ...)
2021-02-07 16:09 ` [PATCH 3/8] xen-swiotlb: use io_tlb_end in xen_swiotlb_dma_supported Christoph Hellwig
@ 2021-02-07 16:09 ` Christoph Hellwig
2021-02-07 16:09 ` [PATCH 5/8] xen-swiotlb: remove xen_io_tlb_start and xen_io_tlb_nslabs Christoph Hellwig
` (3 subsequent siblings)
7 siblings, 0 replies; 11+ messages in thread
From: Christoph Hellwig @ 2021-02-07 16:09 UTC (permalink / raw)
To: Konrad Rzeszutek Wilk
Cc: Michael Ellerman, Dongli Zhang, Claire Chang, xen-devel,
linuxppc-dev, iommu
The xen_set_nslabs function is a little weird, as it has just one
caller, that caller passes a global variable as the argument,
which is then overriden in the function and a derivative of it
returned. Just add a cpp symbol for the default size using a readable
constant and open code the remaining three lines in the caller.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
drivers/xen/swiotlb-xen.c | 19 +++++++------------
1 file changed, 7 insertions(+), 12 deletions(-)
diff --git a/drivers/xen/swiotlb-xen.c b/drivers/xen/swiotlb-xen.c
index 4298f74a083985..57f8d5fadc1fcd 100644
--- a/drivers/xen/swiotlb-xen.c
+++ b/drivers/xen/swiotlb-xen.c
@@ -138,16 +138,6 @@ xen_swiotlb_fixup(void *buf, size_t size, unsigned long nslabs)
} while (i < nslabs);
return 0;
}
-static unsigned long xen_set_nslabs(unsigned long nr_tbl)
-{
- if (!nr_tbl) {
- xen_io_tlb_nslabs = (64 * 1024 * 1024 >> IO_TLB_SHIFT);
- xen_io_tlb_nslabs = ALIGN(xen_io_tlb_nslabs, IO_TLB_SEGSIZE);
- } else
- xen_io_tlb_nslabs = nr_tbl;
-
- return xen_io_tlb_nslabs << IO_TLB_SHIFT;
-}
enum xen_swiotlb_err {
XEN_SWIOTLB_UNKNOWN = 0,
@@ -170,6 +160,9 @@ static const char *xen_swiotlb_error(enum xen_swiotlb_err err)
}
return "";
}
+
+#define DEFAULT_NSLABS ALIGN(SZ_64M >> IO_TLB_SHIFT, IO_TLB_SEGSIZE)
+
int __ref xen_swiotlb_init(int verbose, bool early)
{
unsigned long bytes, order;
@@ -179,8 +172,10 @@ int __ref xen_swiotlb_init(int verbose, bool early)
xen_io_tlb_nslabs = swiotlb_nr_tbl();
retry:
- bytes = xen_set_nslabs(xen_io_tlb_nslabs);
- order = get_order(xen_io_tlb_nslabs << IO_TLB_SHIFT);
+ if (!xen_io_tlb_nslabs)
+ xen_io_tlb_nslabs = DEFAULT_NSLABS;
+ bytes = xen_io_tlb_nslabs << IO_TLB_SHIFT;
+ order = get_order(bytes);
/*
* IO TLB memory already allocated. Just use it.
--
2.29.2
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH 5/8] xen-swiotlb: remove xen_io_tlb_start and xen_io_tlb_nslabs
2021-02-07 16:09 swiotlb cleanups Christoph Hellwig
` (3 preceding siblings ...)
2021-02-07 16:09 ` [PATCH 4/8] xen-swiotlb: remove xen_set_nslabs Christoph Hellwig
@ 2021-02-07 16:09 ` Christoph Hellwig
2021-02-07 16:09 ` [PATCH 6/8] swiotlb: lift the double initialization protection from xen-swiotlb Christoph Hellwig
` (2 subsequent siblings)
7 siblings, 0 replies; 11+ messages in thread
From: Christoph Hellwig @ 2021-02-07 16:09 UTC (permalink / raw)
To: Konrad Rzeszutek Wilk
Cc: Michael Ellerman, Dongli Zhang, Claire Chang, xen-devel,
linuxppc-dev, iommu
The xen_io_tlb_start and xen_io_tlb_nslabs variables ar now only used in
xen_swiotlb_init, so replace them with local variables.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
drivers/xen/swiotlb-xen.c | 57 +++++++++++++++++----------------------
1 file changed, 25 insertions(+), 32 deletions(-)
diff --git a/drivers/xen/swiotlb-xen.c b/drivers/xen/swiotlb-xen.c
index 57f8d5fadc1fcd..6e0f2c5ecd1a2f 100644
--- a/drivers/xen/swiotlb-xen.c
+++ b/drivers/xen/swiotlb-xen.c
@@ -40,14 +40,7 @@
#include <trace/events/swiotlb.h>
#define MAX_DMA_BITS 32
-/*
- * Used to do a quick range check in swiotlb_tbl_unmap_single and
- * swiotlb_tbl_sync_single_*, to see if the memory was in fact allocated by this
- * API.
- */
-static char *xen_io_tlb_start;
-static unsigned long xen_io_tlb_nslabs;
/*
* Quick lookup value of the bus address of the IOTLB.
*/
@@ -169,75 +162,75 @@ int __ref xen_swiotlb_init(int verbose, bool early)
int rc = -ENOMEM;
enum xen_swiotlb_err m_ret = XEN_SWIOTLB_UNKNOWN;
unsigned int repeat = 3;
+ char *start;
+ unsigned long nslabs;
- xen_io_tlb_nslabs = swiotlb_nr_tbl();
+ nslabs = swiotlb_nr_tbl();
retry:
- if (!xen_io_tlb_nslabs)
- xen_io_tlb_nslabs = DEFAULT_NSLABS;
- bytes = xen_io_tlb_nslabs << IO_TLB_SHIFT;
+ if (!nslabs)
+ nslabs = DEFAULT_NSLABS;
+ bytes = nslabs << IO_TLB_SHIFT;
order = get_order(bytes);
/*
* IO TLB memory already allocated. Just use it.
*/
- if (io_tlb_start != 0) {
- xen_io_tlb_start = phys_to_virt(io_tlb_start);
+ if (io_tlb_start != 0)
goto end;
- }
/*
* Get IO TLB memory from any location.
*/
if (early) {
- xen_io_tlb_start = memblock_alloc(PAGE_ALIGN(bytes),
+ start = memblock_alloc(PAGE_ALIGN(bytes),
PAGE_SIZE);
- if (!xen_io_tlb_start)
+ if (!start)
panic("%s: Failed to allocate %lu bytes align=0x%lx\n",
__func__, PAGE_ALIGN(bytes), PAGE_SIZE);
} else {
#define SLABS_PER_PAGE (1 << (PAGE_SHIFT - IO_TLB_SHIFT))
#define IO_TLB_MIN_SLABS ((1<<20) >> IO_TLB_SHIFT)
while ((SLABS_PER_PAGE << order) > IO_TLB_MIN_SLABS) {
- xen_io_tlb_start = (void *)xen_get_swiotlb_free_pages(order);
- if (xen_io_tlb_start)
+ start = (void *)xen_get_swiotlb_free_pages(order);
+ if (start)
break;
order--;
}
if (order != get_order(bytes)) {
pr_warn("Warning: only able to allocate %ld MB for software IO TLB\n",
(PAGE_SIZE << order) >> 20);
- xen_io_tlb_nslabs = SLABS_PER_PAGE << order;
- bytes = xen_io_tlb_nslabs << IO_TLB_SHIFT;
+ nslabs = SLABS_PER_PAGE << order;
+ bytes = nslabs << IO_TLB_SHIFT;
}
}
- if (!xen_io_tlb_start) {
+ if (!start) {
m_ret = XEN_SWIOTLB_ENOMEM;
goto error;
}
/*
* And replace that memory with pages under 4GB.
*/
- rc = xen_swiotlb_fixup(xen_io_tlb_start,
+ rc = xen_swiotlb_fixup(start,
bytes,
- xen_io_tlb_nslabs);
+ nslabs);
if (rc) {
if (early)
- memblock_free(__pa(xen_io_tlb_start),
+ memblock_free(__pa(start),
PAGE_ALIGN(bytes));
else {
- free_pages((unsigned long)xen_io_tlb_start, order);
- xen_io_tlb_start = NULL;
+ free_pages((unsigned long)start, order);
+ start = NULL;
}
m_ret = XEN_SWIOTLB_EFIXUP;
goto error;
}
if (early) {
- if (swiotlb_init_with_tbl(xen_io_tlb_start, xen_io_tlb_nslabs,
+ if (swiotlb_init_with_tbl(start, nslabs,
verbose))
panic("Cannot allocate SWIOTLB buffer");
rc = 0;
} else
- rc = swiotlb_late_init_with_tbl(xen_io_tlb_start, xen_io_tlb_nslabs);
+ rc = swiotlb_late_init_with_tbl(start, nslabs);
end:
if (!rc)
@@ -246,17 +239,17 @@ int __ref xen_swiotlb_init(int verbose, bool early)
return rc;
error:
if (repeat--) {
- xen_io_tlb_nslabs = max(1024UL, /* Min is 2MB */
- (xen_io_tlb_nslabs >> 1));
+ nslabs = max(1024UL, /* Min is 2MB */
+ (nslabs >> 1));
pr_info("Lowering to %luMB\n",
- (xen_io_tlb_nslabs << IO_TLB_SHIFT) >> 20);
+ (nslabs << IO_TLB_SHIFT) >> 20);
goto retry;
}
pr_err("%s (rc:%d)\n", xen_swiotlb_error(m_ret), rc);
if (early)
panic("%s (rc:%d)", xen_swiotlb_error(m_ret), rc);
else
- free_pages((unsigned long)xen_io_tlb_start, order);
+ free_pages((unsigned long)start, order);
return rc;
}
--
2.29.2
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH 6/8] swiotlb: lift the double initialization protection from xen-swiotlb
2021-02-07 16:09 swiotlb cleanups Christoph Hellwig
` (4 preceding siblings ...)
2021-02-07 16:09 ` [PATCH 5/8] xen-swiotlb: remove xen_io_tlb_start and xen_io_tlb_nslabs Christoph Hellwig
@ 2021-02-07 16:09 ` Christoph Hellwig
2021-02-07 16:09 ` [PATCH 7/8] xen-swiotlb: split xen_swiotlb_init Christoph Hellwig
2021-02-07 16:09 ` [PATCH 8/8] xen-swiotlb: remove the unused size argument from xen_swiotlb_fixup Christoph Hellwig
7 siblings, 0 replies; 11+ messages in thread
From: Christoph Hellwig @ 2021-02-07 16:09 UTC (permalink / raw)
To: Konrad Rzeszutek Wilk
Cc: Michael Ellerman, Dongli Zhang, Claire Chang, xen-devel,
linuxppc-dev, iommu
Lift the double initialization protection from xen-swiotlb to the core
code to avoid exposing too many swiotlb internals. Also upgrade the
check to a warning as it should not happen.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
drivers/xen/swiotlb-xen.c | 7 -------
kernel/dma/swiotlb.c | 8 ++++++++
2 files changed, 8 insertions(+), 7 deletions(-)
diff --git a/drivers/xen/swiotlb-xen.c b/drivers/xen/swiotlb-xen.c
index 6e0f2c5ecd1a2f..e6c8556e879ee6 100644
--- a/drivers/xen/swiotlb-xen.c
+++ b/drivers/xen/swiotlb-xen.c
@@ -172,12 +172,6 @@ int __ref xen_swiotlb_init(int verbose, bool early)
bytes = nslabs << IO_TLB_SHIFT;
order = get_order(bytes);
- /*
- * IO TLB memory already allocated. Just use it.
- */
- if (io_tlb_start != 0)
- goto end;
-
/*
* Get IO TLB memory from any location.
*/
@@ -232,7 +226,6 @@ int __ref xen_swiotlb_init(int verbose, bool early)
} else
rc = swiotlb_late_init_with_tbl(start, nslabs);
-end:
if (!rc)
swiotlb_set_max_segment(PAGE_SIZE);
diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
index adcc3c87e10078..a3737961ae5769 100644
--- a/kernel/dma/swiotlb.c
+++ b/kernel/dma/swiotlb.c
@@ -224,6 +224,10 @@ int __init swiotlb_init_with_tbl(char *tlb, unsigned long nslabs, int verbose)
unsigned long i, bytes;
size_t alloc_size;
+ /* protect against double initialization */
+ if (WARN_ON_ONCE(io_tlb_start))
+ return -ENOMEM;
+
bytes = nslabs << IO_TLB_SHIFT;
io_tlb_nslabs = nslabs;
@@ -355,6 +359,10 @@ swiotlb_late_init_with_tbl(char *tlb, unsigned long nslabs)
{
unsigned long i, bytes;
+ /* protect against double initialization */
+ if (WARN_ON_ONCE(io_tlb_start))
+ return -ENOMEM;
+
bytes = nslabs << IO_TLB_SHIFT;
io_tlb_nslabs = nslabs;
--
2.29.2
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH 7/8] xen-swiotlb: split xen_swiotlb_init
2021-02-07 16:09 swiotlb cleanups Christoph Hellwig
` (5 preceding siblings ...)
2021-02-07 16:09 ` [PATCH 6/8] swiotlb: lift the double initialization protection from xen-swiotlb Christoph Hellwig
@ 2021-02-07 16:09 ` Christoph Hellwig
2021-02-07 16:09 ` [PATCH 8/8] xen-swiotlb: remove the unused size argument from xen_swiotlb_fixup Christoph Hellwig
7 siblings, 0 replies; 11+ messages in thread
From: Christoph Hellwig @ 2021-02-07 16:09 UTC (permalink / raw)
To: Konrad Rzeszutek Wilk
Cc: Michael Ellerman, Dongli Zhang, Claire Chang, xen-devel,
linuxppc-dev, iommu
Split xen_swiotlb_init into a normal an an early case. That makes both
much simpler and more readable, and also allows marking the early
code as __init and x86-only.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
arch/arm/xen/mm.c | 2 +-
arch/x86/xen/pci-swiotlb-xen.c | 4 +-
drivers/xen/swiotlb-xen.c | 124 +++++++++++++++++++--------------
include/xen/swiotlb-xen.h | 3 +-
4 files changed, 75 insertions(+), 58 deletions(-)
diff --git a/arch/arm/xen/mm.c b/arch/arm/xen/mm.c
index 467fa225c3d0ed..aae950cd053fea 100644
--- a/arch/arm/xen/mm.c
+++ b/arch/arm/xen/mm.c
@@ -140,7 +140,7 @@ static int __init xen_mm_init(void)
struct gnttab_cache_flush cflush;
if (!xen_initial_domain())
return 0;
- xen_swiotlb_init(1, false);
+ xen_swiotlb_init();
cflush.op = 0;
cflush.a.dev_bus_addr = 0;
diff --git a/arch/x86/xen/pci-swiotlb-xen.c b/arch/x86/xen/pci-swiotlb-xen.c
index 19ae3e4fe4e98e..54f9aa7e845739 100644
--- a/arch/x86/xen/pci-swiotlb-xen.c
+++ b/arch/x86/xen/pci-swiotlb-xen.c
@@ -59,7 +59,7 @@ int __init pci_xen_swiotlb_detect(void)
void __init pci_xen_swiotlb_init(void)
{
if (xen_swiotlb) {
- xen_swiotlb_init(1, true /* early */);
+ xen_swiotlb_init_early();
dma_ops = &xen_swiotlb_dma_ops;
#ifdef CONFIG_PCI
@@ -76,7 +76,7 @@ int pci_xen_swiotlb_init_late(void)
if (xen_swiotlb)
return 0;
- rc = xen_swiotlb_init(1, false /* late */);
+ rc = xen_swiotlb_init();
if (rc)
return rc;
diff --git a/drivers/xen/swiotlb-xen.c b/drivers/xen/swiotlb-xen.c
index e6c8556e879ee6..b2d9e77059bf5a 100644
--- a/drivers/xen/swiotlb-xen.c
+++ b/drivers/xen/swiotlb-xen.c
@@ -156,96 +156,112 @@ static const char *xen_swiotlb_error(enum xen_swiotlb_err err)
#define DEFAULT_NSLABS ALIGN(SZ_64M >> IO_TLB_SHIFT, IO_TLB_SEGSIZE)
-int __ref xen_swiotlb_init(int verbose, bool early)
+int __ref xen_swiotlb_init(void)
{
- unsigned long bytes, order;
- int rc = -ENOMEM;
enum xen_swiotlb_err m_ret = XEN_SWIOTLB_UNKNOWN;
+ unsigned long nslabs, bytes, order;
unsigned int repeat = 3;
+ int rc = -ENOMEM;
char *start;
- unsigned long nslabs;
nslabs = swiotlb_nr_tbl();
-retry:
if (!nslabs)
nslabs = DEFAULT_NSLABS;
+retry:
+ m_ret = XEN_SWIOTLB_ENOMEM;
bytes = nslabs << IO_TLB_SHIFT;
order = get_order(bytes);
/*
* Get IO TLB memory from any location.
*/
- if (early) {
- start = memblock_alloc(PAGE_ALIGN(bytes),
- PAGE_SIZE);
- if (!start)
- panic("%s: Failed to allocate %lu bytes align=0x%lx\n",
- __func__, PAGE_ALIGN(bytes), PAGE_SIZE);
- } else {
#define SLABS_PER_PAGE (1 << (PAGE_SHIFT - IO_TLB_SHIFT))
#define IO_TLB_MIN_SLABS ((1<<20) >> IO_TLB_SHIFT)
- while ((SLABS_PER_PAGE << order) > IO_TLB_MIN_SLABS) {
- start = (void *)xen_get_swiotlb_free_pages(order);
- if (start)
- break;
- order--;
- }
- if (order != get_order(bytes)) {
- pr_warn("Warning: only able to allocate %ld MB for software IO TLB\n",
- (PAGE_SIZE << order) >> 20);
- nslabs = SLABS_PER_PAGE << order;
- bytes = nslabs << IO_TLB_SHIFT;
- }
+ while ((SLABS_PER_PAGE << order) > IO_TLB_MIN_SLABS) {
+ start = (void *)xen_get_swiotlb_free_pages(order);
+ if (start)
+ break;
+ order--;
}
- if (!start) {
- m_ret = XEN_SWIOTLB_ENOMEM;
+ if (!start)
goto error;
+ if (order != get_order(bytes)) {
+ pr_warn("Warning: only able to allocate %ld MB for software IO TLB\n",
+ (PAGE_SIZE << order) >> 20);
+ nslabs = SLABS_PER_PAGE << order;
+ bytes = nslabs << IO_TLB_SHIFT;
}
+
/*
* And replace that memory with pages under 4GB.
*/
- rc = xen_swiotlb_fixup(start,
- bytes,
- nslabs);
+ rc = xen_swiotlb_fixup(start, bytes, nslabs);
if (rc) {
- if (early)
- memblock_free(__pa(start),
- PAGE_ALIGN(bytes));
- else {
- free_pages((unsigned long)start, order);
- start = NULL;
- }
+ free_pages((unsigned long)start, order);
m_ret = XEN_SWIOTLB_EFIXUP;
goto error;
}
- if (early) {
- if (swiotlb_init_with_tbl(start, nslabs,
- verbose))
- panic("Cannot allocate SWIOTLB buffer");
- rc = 0;
- } else
- rc = swiotlb_late_init_with_tbl(start, nslabs);
-
- if (!rc)
- swiotlb_set_max_segment(PAGE_SIZE);
-
- return rc;
+ rc = swiotlb_late_init_with_tbl(start, nslabs);
+ if (rc)
+ return rc;
+ swiotlb_set_max_segment(PAGE_SIZE);
+ return 0;
error:
if (repeat--) {
- nslabs = max(1024UL, /* Min is 2MB */
- (nslabs >> 1));
+ /* Min is 2MB */
+ nslabs = max(1024UL, (nslabs >> 1));
pr_info("Lowering to %luMB\n",
(nslabs << IO_TLB_SHIFT) >> 20);
goto retry;
}
pr_err("%s (rc:%d)\n", xen_swiotlb_error(m_ret), rc);
- if (early)
- panic("%s (rc:%d)", xen_swiotlb_error(m_ret), rc);
- else
- free_pages((unsigned long)start, order);
+ free_pages((unsigned long)start, order);
return rc;
}
+#ifdef CONFIG_X86
+void __init xen_swiotlb_init_early(void)
+{
+ unsigned long nslabs, bytes;
+ unsigned int repeat = 3;
+ char *start;
+ int rc;
+
+ nslabs = swiotlb_nr_tbl();
+ if (!nslabs)
+ nslabs = DEFAULT_NSLABS;
+retry:
+ /*
+ * Get IO TLB memory from any location.
+ */
+ bytes = nslabs << IO_TLB_SHIFT;
+ start = memblock_alloc(PAGE_ALIGN(bytes), PAGE_SIZE);
+ if (!start)
+ panic("%s: Failed to allocate %lu bytes align=0x%lx\n",
+ __func__, PAGE_ALIGN(bytes), PAGE_SIZE);
+
+ /*
+ * And replace that memory with pages under 4GB.
+ */
+ rc = xen_swiotlb_fixup(start, bytes, nslabs);
+ if (rc) {
+ memblock_free(__pa(start), PAGE_ALIGN(bytes));
+ if (repeat--) {
+ /* Min is 2MB */
+ nslabs = max(1024UL, (nslabs >> 1));
+ pr_info("Lowering to %luMB\n",
+ (nslabs << IO_TLB_SHIFT) >> 20);
+ goto retry;
+ }
+ panic("%s (rc:%d)", xen_swiotlb_error(XEN_SWIOTLB_EFIXUP), rc);
+ }
+
+ if (swiotlb_init_with_tbl(start, nslabs, false))
+ panic("Cannot allocate SWIOTLB buffer");
+ swiotlb_set_max_segment(PAGE_SIZE);
+}
+#endif /* CONFIG_X86 */
+
static void *
xen_swiotlb_alloc_coherent(struct device *hwdev, size_t size,
dma_addr_t *dma_handle, gfp_t flags,
diff --git a/include/xen/swiotlb-xen.h b/include/xen/swiotlb-xen.h
index d5eaf9d682b804..6206b1ec99168a 100644
--- a/include/xen/swiotlb-xen.h
+++ b/include/xen/swiotlb-xen.h
@@ -9,7 +9,8 @@ void xen_dma_sync_for_cpu(struct device *dev, dma_addr_t handle,
void xen_dma_sync_for_device(struct device *dev, dma_addr_t handle,
size_t size, enum dma_data_direction dir);
-extern int xen_swiotlb_init(int verbose, bool early);
+int xen_swiotlb_init(void);
+void __init xen_swiotlb_init_early(void);
extern const struct dma_map_ops xen_swiotlb_dma_ops;
#endif /* __LINUX_SWIOTLB_XEN_H */
--
2.29.2
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH 8/8] xen-swiotlb: remove the unused size argument from xen_swiotlb_fixup
2021-02-07 16:09 swiotlb cleanups Christoph Hellwig
` (6 preceding siblings ...)
2021-02-07 16:09 ` [PATCH 7/8] xen-swiotlb: split xen_swiotlb_init Christoph Hellwig
@ 2021-02-07 16:09 ` Christoph Hellwig
7 siblings, 0 replies; 11+ messages in thread
From: Christoph Hellwig @ 2021-02-07 16:09 UTC (permalink / raw)
To: Konrad Rzeszutek Wilk
Cc: Michael Ellerman, Dongli Zhang, Claire Chang, xen-devel,
linuxppc-dev, iommu
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
drivers/xen/swiotlb-xen.c | 7 +++----
1 file changed, 3 insertions(+), 4 deletions(-)
diff --git a/drivers/xen/swiotlb-xen.c b/drivers/xen/swiotlb-xen.c
index b2d9e77059bf5a..621a20c1143597 100644
--- a/drivers/xen/swiotlb-xen.c
+++ b/drivers/xen/swiotlb-xen.c
@@ -104,8 +104,7 @@ static int is_xen_swiotlb_buffer(struct device *dev, dma_addr_t dma_addr)
return 0;
}
-static int
-xen_swiotlb_fixup(void *buf, size_t size, unsigned long nslabs)
+static int xen_swiotlb_fixup(void *buf, unsigned long nslabs)
{
int i, rc;
int dma_bits;
@@ -195,7 +194,7 @@ int __ref xen_swiotlb_init(void)
/*
* And replace that memory with pages under 4GB.
*/
- rc = xen_swiotlb_fixup(start, bytes, nslabs);
+ rc = xen_swiotlb_fixup(start, nslabs);
if (rc) {
free_pages((unsigned long)start, order);
m_ret = XEN_SWIOTLB_EFIXUP;
@@ -243,7 +242,7 @@ void __init xen_swiotlb_init_early(void)
/*
* And replace that memory with pages under 4GB.
*/
- rc = xen_swiotlb_fixup(start, bytes, nslabs);
+ rc = xen_swiotlb_fixup(start, nslabs);
if (rc) {
memblock_free(__pa(start), PAGE_ALIGN(bytes));
if (repeat--) {
--
2.29.2
^ permalink raw reply related [flat|nested] 11+ messages in thread
* Re: [PATCH 2/8] xen-swiotlb: use is_swiotlb_buffer in is_xen_swiotlb_buffer
2021-02-07 16:09 ` [PATCH 2/8] xen-swiotlb: use is_swiotlb_buffer in is_xen_swiotlb_buffer Christoph Hellwig
@ 2021-02-19 20:43 ` Konrad Rzeszutek Wilk
0 siblings, 0 replies; 11+ messages in thread
From: Konrad Rzeszutek Wilk @ 2021-02-19 20:43 UTC (permalink / raw)
To: Christoph Hellwig
Cc: Michael Ellerman, Dongli Zhang, Claire Chang, xen-devel,
linuxppc-dev, iommu
On Sun, Feb 07, 2021 at 05:09:28PM +0100, Christoph Hellwig wrote:
> Use the is_swiotlb_buffer to check if a physical address is
> a swiotlb buffer. This works because xen-swiotlb does use the
> same buffer as the main swiotlb code, and xen_io_tlb_{start,end}
> are just the addresses for it that went through phys_to_virt.
>
Reviewed-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
> Signed-off-by: Christoph Hellwig <hch@lst.de>
> ---
> drivers/xen/swiotlb-xen.c | 6 ++----
> 1 file changed, 2 insertions(+), 4 deletions(-)
>
> diff --git a/drivers/xen/swiotlb-xen.c b/drivers/xen/swiotlb-xen.c
> index 2b385c1b4a99cb..a4026822a889f7 100644
> --- a/drivers/xen/swiotlb-xen.c
> +++ b/drivers/xen/swiotlb-xen.c
> @@ -111,10 +111,8 @@ static int is_xen_swiotlb_buffer(struct device *dev, dma_addr_t dma_addr)
> * have the same virtual address as another address
> * in our domain. Therefore _only_ check address within our domain.
> */
> - if (pfn_valid(PFN_DOWN(paddr))) {
> - return paddr >= virt_to_phys(xen_io_tlb_start) &&
> - paddr < virt_to_phys(xen_io_tlb_end);
> - }
> + if (pfn_valid(PFN_DOWN(paddr)))
> + return is_swiotlb_buffer(paddr);
> return 0;
> }
>
> --
> 2.29.2
>
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH 3/8] xen-swiotlb: use io_tlb_end in xen_swiotlb_dma_supported
2021-02-07 16:09 ` [PATCH 3/8] xen-swiotlb: use io_tlb_end in xen_swiotlb_dma_supported Christoph Hellwig
@ 2021-02-19 21:00 ` Konrad Rzeszutek Wilk
0 siblings, 0 replies; 11+ messages in thread
From: Konrad Rzeszutek Wilk @ 2021-02-19 21:00 UTC (permalink / raw)
To: Christoph Hellwig
Cc: Michael Ellerman, Dongli Zhang, Claire Chang, xen-devel,
linuxppc-dev, iommu
On Sun, Feb 07, 2021 at 05:09:29PM +0100, Christoph Hellwig wrote:
> Use the existing variable that holds the physical address for
> xen_io_tlb_end to simplify xen_swiotlb_dma_supported a bit, and remove
> the otherwise unused xen_io_tlb_end variable and the xen_virt_to_bus
> helper.
>
Reviewed-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
> Signed-off-by: Christoph Hellwig <hch@lst.de>
> ---
> drivers/xen/swiotlb-xen.c | 10 ++--------
> 1 file changed, 2 insertions(+), 8 deletions(-)
>
> diff --git a/drivers/xen/swiotlb-xen.c b/drivers/xen/swiotlb-xen.c
> index a4026822a889f7..4298f74a083985 100644
> --- a/drivers/xen/swiotlb-xen.c
> +++ b/drivers/xen/swiotlb-xen.c
> @@ -46,7 +46,7 @@
> * API.
> */
>
> -static char *xen_io_tlb_start, *xen_io_tlb_end;
> +static char *xen_io_tlb_start;
> static unsigned long xen_io_tlb_nslabs;
> /*
> * Quick lookup value of the bus address of the IOTLB.
> @@ -82,11 +82,6 @@ static inline phys_addr_t xen_dma_to_phys(struct device *dev,
> return xen_bus_to_phys(dev, dma_to_phys(dev, dma_addr));
> }
>
> -static inline dma_addr_t xen_virt_to_bus(struct device *dev, void *address)
> -{
> - return xen_phys_to_dma(dev, virt_to_phys(address));
> -}
> -
> static inline int range_straddles_page_boundary(phys_addr_t p, size_t size)
> {
> unsigned long next_bfn, xen_pfn = XEN_PFN_DOWN(p);
> @@ -250,7 +245,6 @@ int __ref xen_swiotlb_init(int verbose, bool early)
> rc = swiotlb_late_init_with_tbl(xen_io_tlb_start, xen_io_tlb_nslabs);
>
> end:
> - xen_io_tlb_end = xen_io_tlb_start + bytes;
> if (!rc)
> swiotlb_set_max_segment(PAGE_SIZE);
>
> @@ -558,7 +552,7 @@ xen_swiotlb_sync_sg_for_device(struct device *dev, struct scatterlist *sgl,
> static int
> xen_swiotlb_dma_supported(struct device *hwdev, u64 mask)
> {
> - return xen_virt_to_bus(hwdev, xen_io_tlb_end - 1) <= mask;
> + return xen_phys_to_dma(hwdev, io_tlb_end - 1) <= mask;
> }
>
> const struct dma_map_ops xen_swiotlb_dma_ops = {
> --
> 2.29.2
>
^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2021-02-19 21:00 UTC | newest]
Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-02-07 16:09 swiotlb cleanups Christoph Hellwig
2021-02-07 16:09 ` [PATCH 1/8] powerpc/svm: stop using io_tlb_start Christoph Hellwig
2021-02-07 16:09 ` [PATCH 2/8] xen-swiotlb: use is_swiotlb_buffer in is_xen_swiotlb_buffer Christoph Hellwig
2021-02-19 20:43 ` Konrad Rzeszutek Wilk
2021-02-07 16:09 ` [PATCH 3/8] xen-swiotlb: use io_tlb_end in xen_swiotlb_dma_supported Christoph Hellwig
2021-02-19 21:00 ` Konrad Rzeszutek Wilk
2021-02-07 16:09 ` [PATCH 4/8] xen-swiotlb: remove xen_set_nslabs Christoph Hellwig
2021-02-07 16:09 ` [PATCH 5/8] xen-swiotlb: remove xen_io_tlb_start and xen_io_tlb_nslabs Christoph Hellwig
2021-02-07 16:09 ` [PATCH 6/8] swiotlb: lift the double initialization protection from xen-swiotlb Christoph Hellwig
2021-02-07 16:09 ` [PATCH 7/8] xen-swiotlb: split xen_swiotlb_init Christoph Hellwig
2021-02-07 16:09 ` [PATCH 8/8] xen-swiotlb: remove the unused size argument from xen_swiotlb_fixup Christoph Hellwig
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).