* cleanup swiotlb initialization v2
@ 2022-02-27 14:30 ` Christoph Hellwig
0 siblings, 0 replies; 118+ messages in thread
From: Christoph Hellwig @ 2022-02-27 14:30 UTC (permalink / raw)
To: iommu
Cc: x86, Anshuman Khandual, Konrad Rzeszutek Wilk,
Stefano Stabellini, Boris Ostrovsky, Juergen Gross, Joerg Roedel,
David Woodhouse, Lu Baolu, Robin Murphy, linux-arm-kernel,
xen-devel, linux-ia64, linux-mips, linuxppc-dev, linux-riscv,
linux-s390, linux-hyperv, tboot-devel, linux-pci
Hi all,
this series tries to clean up the swiotlb initialization, including
that of swiotlb-xen. To get there is also removes the x86 iommu table
infrastructure that massively obsfucates the initialization path.
Git tree:
git://git.infradead.org/users/hch/misc.git swiotlb-init-cleanup
Gitweb:
http://git.infradead.org/users/hch/misc.git/shortlog/refs/heads/swiotlb-init-cleanup
Changes since v1:
- skip IOMMU initialization on Xen PV kernels
- various small whitespace / typo fixes
Diffstat:
arch/ia64/include/asm/iommu_table.h | 7 -
arch/x86/include/asm/iommu_table.h | 102 -------------------
arch/x86/include/asm/swiotlb.h | 30 -----
arch/x86/kernel/pci-iommu_table.c | 77 --------------
arch/x86/kernel/pci-swiotlb.c | 77 --------------
arch/x86/xen/pci-swiotlb-xen.c | 96 ------------------
b/arch/arm/mm/init.c | 6 -
b/arch/arm/xen/mm.c | 23 ++--
b/arch/arm64/mm/init.c | 6 -
b/arch/ia64/mm/init.c | 4
b/arch/mips/cavium-octeon/dma-octeon.c | 15 --
b/arch/mips/loongson64/dma.c | 2
b/arch/mips/pci/pci-octeon.c | 2
b/arch/mips/sibyte/common/dma.c | 2
b/arch/powerpc/include/asm/svm.h | 4
b/arch/powerpc/include/asm/swiotlb.h | 1
b/arch/powerpc/mm/mem.c | 6 -
b/arch/powerpc/platforms/pseries/setup.c | 3
b/arch/powerpc/platforms/pseries/svm.c | 26 ----
b/arch/riscv/mm/init.c | 8 -
b/arch/s390/mm/init.c | 3
b/arch/x86/include/asm/dma-mapping.h | 12 --
b/arch/x86/include/asm/gart.h | 5
b/arch/x86/include/asm/iommu.h | 8 +
b/arch/x86/include/asm/xen/page.h | 5
b/arch/x86/include/asm/xen/swiotlb-xen.h | 2
b/arch/x86/kernel/Makefile | 2
b/arch/x86/kernel/amd_gart_64.c | 5
b/arch/x86/kernel/aperture_64.c | 14 --
b/arch/x86/kernel/cpu/mshyperv.c | 8 -
b/arch/x86/kernel/pci-dma.c | 109 ++++++++++++++++----
b/arch/x86/kernel/tboot.c | 1
b/arch/x86/kernel/vmlinux.lds.S | 12 --
b/arch/x86/mm/mem_encrypt_amd.c | 3
b/arch/x86/pci/sta2x11-fixup.c | 2
b/arch/x86/xen/Makefile | 2
b/drivers/iommu/amd/init.c | 6 -
b/drivers/iommu/amd/iommu.c | 5
b/drivers/iommu/intel/dmar.c | 6 -
b/drivers/xen/swiotlb-xen.c | 132 -------------------------
b/include/linux/dmar.h | 6 -
b/include/linux/swiotlb.h | 22 ++--
b/include/trace/events/swiotlb.h | 29 +----
b/include/xen/arm/page.h | 1
b/include/xen/swiotlb-xen.h | 8 +
b/kernel/dma/direct.h | 2
b/kernel/dma/swiotlb.c | 163 +++++++++++++++----------------
47 files changed, 253 insertions(+), 817 deletions(-)
^ permalink raw reply [flat|nested] 118+ messages in thread
* cleanup swiotlb initialization v2
@ 2022-02-27 14:30 ` Christoph Hellwig
0 siblings, 0 replies; 118+ messages in thread
From: Christoph Hellwig @ 2022-02-27 14:30 UTC (permalink / raw)
To: iommu
Cc: x86, Anshuman Khandual, Konrad Rzeszutek Wilk,
Stefano Stabellini, Boris Ostrovsky, Juergen Gross, Joerg Roedel,
David Woodhouse, Lu Baolu, Robin Murphy, linux-arm-kernel,
xen-devel, linux-ia64, linux-mips, linuxppc-dev, linux-riscv,
linux-s390, linux-hyperv, tboot-devel, linux-pci
Hi all,
this series tries to clean up the swiotlb initialization, including
that of swiotlb-xen. To get there is also removes the x86 iommu table
infrastructure that massively obsfucates the initialization path.
Git tree:
git://git.infradead.org/users/hch/misc.git swiotlb-init-cleanup
Gitweb:
http://git.infradead.org/users/hch/misc.git/shortlog/refs/heads/swiotlb-init-cleanup
Changes since v1:
- skip IOMMU initialization on Xen PV kernels
- various small whitespace / typo fixes
Diffstat:
arch/ia64/include/asm/iommu_table.h | 7 -
arch/x86/include/asm/iommu_table.h | 102 -------------------
arch/x86/include/asm/swiotlb.h | 30 -----
arch/x86/kernel/pci-iommu_table.c | 77 --------------
arch/x86/kernel/pci-swiotlb.c | 77 --------------
arch/x86/xen/pci-swiotlb-xen.c | 96 ------------------
b/arch/arm/mm/init.c | 6 -
b/arch/arm/xen/mm.c | 23 ++--
b/arch/arm64/mm/init.c | 6 -
b/arch/ia64/mm/init.c | 4
b/arch/mips/cavium-octeon/dma-octeon.c | 15 --
b/arch/mips/loongson64/dma.c | 2
b/arch/mips/pci/pci-octeon.c | 2
b/arch/mips/sibyte/common/dma.c | 2
b/arch/powerpc/include/asm/svm.h | 4
b/arch/powerpc/include/asm/swiotlb.h | 1
b/arch/powerpc/mm/mem.c | 6 -
b/arch/powerpc/platforms/pseries/setup.c | 3
b/arch/powerpc/platforms/pseries/svm.c | 26 ----
b/arch/riscv/mm/init.c | 8 -
b/arch/s390/mm/init.c | 3
b/arch/x86/include/asm/dma-mapping.h | 12 --
b/arch/x86/include/asm/gart.h | 5
b/arch/x86/include/asm/iommu.h | 8 +
b/arch/x86/include/asm/xen/page.h | 5
b/arch/x86/include/asm/xen/swiotlb-xen.h | 2
b/arch/x86/kernel/Makefile | 2
b/arch/x86/kernel/amd_gart_64.c | 5
b/arch/x86/kernel/aperture_64.c | 14 --
b/arch/x86/kernel/cpu/mshyperv.c | 8 -
b/arch/x86/kernel/pci-dma.c | 109 ++++++++++++++++----
b/arch/x86/kernel/tboot.c | 1
b/arch/x86/kernel/vmlinux.lds.S | 12 --
b/arch/x86/mm/mem_encrypt_amd.c | 3
b/arch/x86/pci/sta2x11-fixup.c | 2
b/arch/x86/xen/Makefile | 2
b/drivers/iommu/amd/init.c | 6 -
b/drivers/iommu/amd/iommu.c | 5
b/drivers/iommu/intel/dmar.c | 6 -
b/drivers/xen/swiotlb-xen.c | 132 -------------------------
b/include/linux/dmar.h | 6 -
b/include/linux/swiotlb.h | 22 ++--
b/include/trace/events/swiotlb.h | 29 +----
b/include/xen/arm/page.h | 1
b/include/xen/swiotlb-xen.h | 8 +
b/kernel/dma/direct.h | 2
b/kernel/dma/swiotlb.c | 163 +++++++++++++++----------------
47 files changed, 253 insertions(+), 817 deletions(-)
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 118+ messages in thread
* cleanup swiotlb initialization v2
@ 2022-02-27 14:30 ` Christoph Hellwig
0 siblings, 0 replies; 118+ messages in thread
From: Christoph Hellwig @ 2022-02-27 14:30 UTC (permalink / raw)
To: iommu
Cc: Juergen Gross, linux-s390, linux-hyperv, Stefano Stabellini,
linux-ia64, Anshuman Khandual, Konrad Rzeszutek Wilk,
Robin Murphy, x86, linux-mips, linuxppc-dev, tboot-devel,
linux-pci, xen-devel, Boris Ostrovsky, David Woodhouse,
linux-riscv, linux-arm-kernel
Hi all,
this series tries to clean up the swiotlb initialization, including
that of swiotlb-xen. To get there is also removes the x86 iommu table
infrastructure that massively obsfucates the initialization path.
Git tree:
git://git.infradead.org/users/hch/misc.git swiotlb-init-cleanup
Gitweb:
http://git.infradead.org/users/hch/misc.git/shortlog/refs/heads/swiotlb-init-cleanup
Changes since v1:
- skip IOMMU initialization on Xen PV kernels
- various small whitespace / typo fixes
Diffstat:
arch/ia64/include/asm/iommu_table.h | 7 -
arch/x86/include/asm/iommu_table.h | 102 -------------------
arch/x86/include/asm/swiotlb.h | 30 -----
arch/x86/kernel/pci-iommu_table.c | 77 --------------
arch/x86/kernel/pci-swiotlb.c | 77 --------------
arch/x86/xen/pci-swiotlb-xen.c | 96 ------------------
b/arch/arm/mm/init.c | 6 -
b/arch/arm/xen/mm.c | 23 ++--
b/arch/arm64/mm/init.c | 6 -
b/arch/ia64/mm/init.c | 4
b/arch/mips/cavium-octeon/dma-octeon.c | 15 --
b/arch/mips/loongson64/dma.c | 2
b/arch/mips/pci/pci-octeon.c | 2
b/arch/mips/sibyte/common/dma.c | 2
b/arch/powerpc/include/asm/svm.h | 4
b/arch/powerpc/include/asm/swiotlb.h | 1
b/arch/powerpc/mm/mem.c | 6 -
b/arch/powerpc/platforms/pseries/setup.c | 3
b/arch/powerpc/platforms/pseries/svm.c | 26 ----
b/arch/riscv/mm/init.c | 8 -
b/arch/s390/mm/init.c | 3
b/arch/x86/include/asm/dma-mapping.h | 12 --
b/arch/x86/include/asm/gart.h | 5
b/arch/x86/include/asm/iommu.h | 8 +
b/arch/x86/include/asm/xen/page.h | 5
b/arch/x86/include/asm/xen/swiotlb-xen.h | 2
b/arch/x86/kernel/Makefile | 2
b/arch/x86/kernel/amd_gart_64.c | 5
b/arch/x86/kernel/aperture_64.c | 14 --
b/arch/x86/kernel/cpu/mshyperv.c | 8 -
b/arch/x86/kernel/pci-dma.c | 109 ++++++++++++++++----
b/arch/x86/kernel/tboot.c | 1
b/arch/x86/kernel/vmlinux.lds.S | 12 --
b/arch/x86/mm/mem_encrypt_amd.c | 3
b/arch/x86/pci/sta2x11-fixup.c | 2
b/arch/x86/xen/Makefile | 2
b/drivers/iommu/amd/init.c | 6 -
b/drivers/iommu/amd/iommu.c | 5
b/drivers/iommu/intel/dmar.c | 6 -
b/drivers/xen/swiotlb-xen.c | 132 -------------------------
b/include/linux/dmar.h | 6 -
b/include/linux/swiotlb.h | 22 ++--
b/include/trace/events/swiotlb.h | 29 +----
b/include/xen/arm/page.h | 1
b/include/xen/swiotlb-xen.h | 8 +
b/kernel/dma/direct.h | 2
b/kernel/dma/swiotlb.c | 163 +++++++++++++++----------------
47 files changed, 253 insertions(+), 817 deletions(-)
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply [flat|nested] 118+ messages in thread
* cleanup swiotlb initialization v2
@ 2022-02-27 14:30 ` Christoph Hellwig
0 siblings, 0 replies; 118+ messages in thread
From: Christoph Hellwig @ 2022-02-27 14:30 UTC (permalink / raw)
To: iommu
Cc: Juergen Gross, linux-s390, linux-hyperv, Stefano Stabellini,
linux-ia64, Anshuman Khandual, Konrad Rzeszutek Wilk,
Robin Murphy, Joerg Roedel, x86, linux-mips, linuxppc-dev,
tboot-devel, linux-pci, xen-devel, Boris Ostrovsky,
David Woodhouse, linux-riscv, linux-arm-kernel, Lu Baolu
Hi all,
this series tries to clean up the swiotlb initialization, including
that of swiotlb-xen. To get there is also removes the x86 iommu table
infrastructure that massively obsfucates the initialization path.
Git tree:
git://git.infradead.org/users/hch/misc.git swiotlb-init-cleanup
Gitweb:
http://git.infradead.org/users/hch/misc.git/shortlog/refs/heads/swiotlb-init-cleanup
Changes since v1:
- skip IOMMU initialization on Xen PV kernels
- various small whitespace / typo fixes
Diffstat:
arch/ia64/include/asm/iommu_table.h | 7 -
arch/x86/include/asm/iommu_table.h | 102 -------------------
arch/x86/include/asm/swiotlb.h | 30 -----
arch/x86/kernel/pci-iommu_table.c | 77 --------------
arch/x86/kernel/pci-swiotlb.c | 77 --------------
arch/x86/xen/pci-swiotlb-xen.c | 96 ------------------
b/arch/arm/mm/init.c | 6 -
b/arch/arm/xen/mm.c | 23 ++--
b/arch/arm64/mm/init.c | 6 -
b/arch/ia64/mm/init.c | 4
b/arch/mips/cavium-octeon/dma-octeon.c | 15 --
b/arch/mips/loongson64/dma.c | 2
b/arch/mips/pci/pci-octeon.c | 2
b/arch/mips/sibyte/common/dma.c | 2
b/arch/powerpc/include/asm/svm.h | 4
b/arch/powerpc/include/asm/swiotlb.h | 1
b/arch/powerpc/mm/mem.c | 6 -
b/arch/powerpc/platforms/pseries/setup.c | 3
b/arch/powerpc/platforms/pseries/svm.c | 26 ----
b/arch/riscv/mm/init.c | 8 -
b/arch/s390/mm/init.c | 3
b/arch/x86/include/asm/dma-mapping.h | 12 --
b/arch/x86/include/asm/gart.h | 5
b/arch/x86/include/asm/iommu.h | 8 +
b/arch/x86/include/asm/xen/page.h | 5
b/arch/x86/include/asm/xen/swiotlb-xen.h | 2
b/arch/x86/kernel/Makefile | 2
b/arch/x86/kernel/amd_gart_64.c | 5
b/arch/x86/kernel/aperture_64.c | 14 --
b/arch/x86/kernel/cpu/mshyperv.c | 8 -
b/arch/x86/kernel/pci-dma.c | 109 ++++++++++++++++----
b/arch/x86/kernel/tboot.c | 1
b/arch/x86/kernel/vmlinux.lds.S | 12 --
b/arch/x86/mm/mem_encrypt_amd.c | 3
b/arch/x86/pci/sta2x11-fixup.c | 2
b/arch/x86/xen/Makefile | 2
b/drivers/iommu/amd/init.c | 6 -
b/drivers/iommu/amd/iommu.c | 5
b/drivers/iommu/intel/dmar.c | 6 -
b/drivers/xen/swiotlb-xen.c | 132 -------------------------
b/include/linux/dmar.h | 6 -
b/include/linux/swiotlb.h | 22 ++--
b/include/trace/events/swiotlb.h | 29 +----
b/include/xen/arm/page.h | 1
b/include/xen/swiotlb-xen.h | 8 +
b/kernel/dma/direct.h | 2
b/kernel/dma/swiotlb.c | 163 +++++++++++++++----------------
47 files changed, 253 insertions(+), 817 deletions(-)
^ permalink raw reply [flat|nested] 118+ messages in thread
* cleanup swiotlb initialization v2
@ 2022-02-27 14:30 ` Christoph Hellwig
0 siblings, 0 replies; 118+ messages in thread
From: Christoph Hellwig @ 2022-02-27 14:30 UTC (permalink / raw)
To: iommu
Cc: x86, Anshuman Khandual, Konrad Rzeszutek Wilk,
Stefano Stabellini, Boris Ostrovsky, Juergen Gross, Joerg Roedel,
David Woodhouse, Lu Baolu, Robin Murphy, linux-arm-kernel,
xen-devel, linux-ia64, linux-mips, linuxppc-dev, linux-riscv,
linux-s390, linux-hyperv, tboot-devel, linux-pci
Hi all,
this series tries to clean up the swiotlb initialization, including
that of swiotlb-xen. To get there is also removes the x86 iommu table
infrastructure that massively obsfucates the initialization path.
Git tree:
git://git.infradead.org/users/hch/misc.git swiotlb-init-cleanup
Gitweb:
http://git.infradead.org/users/hch/misc.git/shortlog/refs/heads/swiotlb-init-cleanup
Changes since v1:
- skip IOMMU initialization on Xen PV kernels
- various small whitespace / typo fixes
Diffstat:
arch/ia64/include/asm/iommu_table.h | 7 -
arch/x86/include/asm/iommu_table.h | 102 -------------------
arch/x86/include/asm/swiotlb.h | 30 -----
arch/x86/kernel/pci-iommu_table.c | 77 --------------
arch/x86/kernel/pci-swiotlb.c | 77 --------------
arch/x86/xen/pci-swiotlb-xen.c | 96 ------------------
b/arch/arm/mm/init.c | 6 -
b/arch/arm/xen/mm.c | 23 ++--
b/arch/arm64/mm/init.c | 6 -
b/arch/ia64/mm/init.c | 4
b/arch/mips/cavium-octeon/dma-octeon.c | 15 --
b/arch/mips/loongson64/dma.c | 2
b/arch/mips/pci/pci-octeon.c | 2
b/arch/mips/sibyte/common/dma.c | 2
b/arch/powerpc/include/asm/svm.h | 4
b/arch/powerpc/include/asm/swiotlb.h | 1
b/arch/powerpc/mm/mem.c | 6 -
b/arch/powerpc/platforms/pseries/setup.c | 3
b/arch/powerpc/platforms/pseries/svm.c | 26 ----
b/arch/riscv/mm/init.c | 8 -
b/arch/s390/mm/init.c | 3
b/arch/x86/include/asm/dma-mapping.h | 12 --
b/arch/x86/include/asm/gart.h | 5
b/arch/x86/include/asm/iommu.h | 8 +
b/arch/x86/include/asm/xen/page.h | 5
b/arch/x86/include/asm/xen/swiotlb-xen.h | 2
b/arch/x86/kernel/Makefile | 2
b/arch/x86/kernel/amd_gart_64.c | 5
b/arch/x86/kernel/aperture_64.c | 14 --
b/arch/x86/kernel/cpu/mshyperv.c | 8 -
b/arch/x86/kernel/pci-dma.c | 109 ++++++++++++++++----
b/arch/x86/kernel/tboot.c | 1
b/arch/x86/kernel/vmlinux.lds.S | 12 --
b/arch/x86/mm/mem_encrypt_amd.c | 3
b/arch/x86/pci/sta2x11-fixup.c | 2
b/arch/x86/xen/Makefile | 2
b/drivers/iommu/amd/init.c | 6 -
b/drivers/iommu/amd/iommu.c | 5
b/drivers/iommu/intel/dmar.c | 6 -
b/drivers/xen/swiotlb-xen.c | 132 -------------------------
b/include/linux/dmar.h | 6 -
b/include/linux/swiotlb.h | 22 ++--
b/include/trace/events/swiotlb.h | 29 +----
b/include/xen/arm/page.h | 1
b/include/xen/swiotlb-xen.h | 8 +
b/kernel/dma/direct.h | 2
b/kernel/dma/swiotlb.c | 163 +++++++++++++++----------------
47 files changed, 253 insertions(+), 817 deletions(-)
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 118+ messages in thread
* cleanup swiotlb initialization v2
@ 2022-02-27 14:30 ` Christoph Hellwig
0 siblings, 0 replies; 118+ messages in thread
From: Christoph Hellwig @ 2022-02-27 14:30 UTC (permalink / raw)
To: iommu
Cc: x86, Anshuman Khandual, Konrad Rzeszutek Wilk,
Stefano Stabellini, Boris Ostrovsky, Juergen Gross, Joerg Roedel,
David Woodhouse, Lu Baolu, Robin Murphy, linux-arm-kernel,
xen-devel, linux-ia64, linux-mips, linuxppc-dev, linux-riscv,
linux-s390, linux-hyperv, tboot-devel, linux-pci
Hi all,
this series tries to clean up the swiotlb initialization, including
that of swiotlb-xen. To get there is also removes the x86 iommu table
infrastructure that massively obsfucates the initialization path.
Git tree:
git://git.infradead.org/users/hch/misc.git swiotlb-init-cleanup
Gitweb:
http://git.infradead.org/users/hch/misc.git/shortlog/refs/heads/swiotlb-init-cleanup
Changes since v1:
- skip IOMMU initialization on Xen PV kernels
- various small whitespace / typo fixes
Diffstat:
arch/ia64/include/asm/iommu_table.h | 7 -
arch/x86/include/asm/iommu_table.h | 102 -------------------
arch/x86/include/asm/swiotlb.h | 30 -----
arch/x86/kernel/pci-iommu_table.c | 77 --------------
arch/x86/kernel/pci-swiotlb.c | 77 --------------
arch/x86/xen/pci-swiotlb-xen.c | 96 ------------------
b/arch/arm/mm/init.c | 6 -
b/arch/arm/xen/mm.c | 23 ++--
b/arch/arm64/mm/init.c | 6 -
b/arch/ia64/mm/init.c | 4
b/arch/mips/cavium-octeon/dma-octeon.c | 15 --
b/arch/mips/loongson64/dma.c | 2
b/arch/mips/pci/pci-octeon.c | 2
b/arch/mips/sibyte/common/dma.c | 2
b/arch/powerpc/include/asm/svm.h | 4
b/arch/powerpc/include/asm/swiotlb.h | 1
b/arch/powerpc/mm/mem.c | 6 -
b/arch/powerpc/platforms/pseries/setup.c | 3
b/arch/powerpc/platforms/pseries/svm.c | 26 ----
b/arch/riscv/mm/init.c | 8 -
b/arch/s390/mm/init.c | 3
b/arch/x86/include/asm/dma-mapping.h | 12 --
b/arch/x86/include/asm/gart.h | 5
b/arch/x86/include/asm/iommu.h | 8 +
b/arch/x86/include/asm/xen/page.h | 5
b/arch/x86/include/asm/xen/swiotlb-xen.h | 2
b/arch/x86/kernel/Makefile | 2
b/arch/x86/kernel/amd_gart_64.c | 5
b/arch/x86/kernel/aperture_64.c | 14 --
b/arch/x86/kernel/cpu/mshyperv.c | 8 -
b/arch/x86/kernel/pci-dma.c | 109 ++++++++++++++++----
b/arch/x86/kernel/tboot.c | 1
b/arch/x86/kernel/vmlinux.lds.S | 12 --
b/arch/x86/mm/mem_encrypt_amd.c | 3
b/arch/x86/pci/sta2x11-fixup.c | 2
b/arch/x86/xen/Makefile | 2
b/drivers/iommu/amd/init.c | 6 -
b/drivers/iommu/amd/iommu.c | 5
b/drivers/iommu/intel/dmar.c | 6 -
b/drivers/xen/swiotlb-xen.c | 132 -------------------------
b/include/linux/dmar.h | 6 -
b/include/linux/swiotlb.h | 22 ++--
b/include/trace/events/swiotlb.h | 29 +----
b/include/xen/arm/page.h | 1
b/include/xen/swiotlb-xen.h | 8 +
b/kernel/dma/direct.h | 2
b/kernel/dma/swiotlb.c | 163 +++++++++++++++----------------
47 files changed, 253 insertions(+), 817 deletions(-)
^ permalink raw reply [flat|nested] 118+ messages in thread
* [PATCH 01/11] dma-direct: use is_swiotlb_active in dma_direct_map_page
2022-02-27 14:30 ` Christoph Hellwig
` (3 preceding siblings ...)
(?)
@ 2022-02-27 14:30 ` Christoph Hellwig
-1 siblings, 0 replies; 118+ messages in thread
From: Christoph Hellwig @ 2022-02-27 14:30 UTC (permalink / raw)
To: iommu
Cc: x86, Anshuman Khandual, Konrad Rzeszutek Wilk,
Stefano Stabellini, Boris Ostrovsky, Juergen Gross, Joerg Roedel,
David Woodhouse, Lu Baolu, Robin Murphy, linux-arm-kernel,
xen-devel, linux-ia64, linux-mips, linuxppc-dev, linux-riscv,
linux-s390, linux-hyperv, tboot-devel, linux-pci
Use the more specific is_swiotlb_active check instead of checking the
global swiotlb_force variable.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Anshuman Khandual <anshuman.khandual@arm.com>
---
kernel/dma/direct.h | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/kernel/dma/direct.h b/kernel/dma/direct.h
index 4632b0f4f72eb..4dc16e08c7e1a 100644
--- a/kernel/dma/direct.h
+++ b/kernel/dma/direct.h
@@ -91,7 +91,7 @@ static inline dma_addr_t dma_direct_map_page(struct device *dev,
return swiotlb_map(dev, phys, size, dir, attrs);
if (unlikely(!dma_capable(dev, dma_addr, size, true))) {
- if (swiotlb_force != SWIOTLB_NO_FORCE)
+ if (is_swiotlb_active(dev))
return swiotlb_map(dev, phys, size, dir, attrs);
dev_WARN_ONCE(dev, 1,
--
2.30.2
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply related [flat|nested] 118+ messages in thread
* [PATCH 01/11] dma-direct: use is_swiotlb_active in dma_direct_map_page
@ 2022-02-27 14:30 ` Christoph Hellwig
0 siblings, 0 replies; 118+ messages in thread
From: Christoph Hellwig @ 2022-02-27 14:30 UTC (permalink / raw)
To: iommu
Cc: Juergen Gross, linux-s390, linux-hyperv, Stefano Stabellini,
linux-ia64, Anshuman Khandual, Konrad Rzeszutek Wilk,
Robin Murphy, x86, linux-mips, linuxppc-dev, tboot-devel,
linux-pci, xen-devel, Boris Ostrovsky, David Woodhouse,
linux-riscv, linux-arm-kernel
Use the more specific is_swiotlb_active check instead of checking the
global swiotlb_force variable.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Anshuman Khandual <anshuman.khandual@arm.com>
---
kernel/dma/direct.h | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/kernel/dma/direct.h b/kernel/dma/direct.h
index 4632b0f4f72eb..4dc16e08c7e1a 100644
--- a/kernel/dma/direct.h
+++ b/kernel/dma/direct.h
@@ -91,7 +91,7 @@ static inline dma_addr_t dma_direct_map_page(struct device *dev,
return swiotlb_map(dev, phys, size, dir, attrs);
if (unlikely(!dma_capable(dev, dma_addr, size, true))) {
- if (swiotlb_force != SWIOTLB_NO_FORCE)
+ if (is_swiotlb_active(dev))
return swiotlb_map(dev, phys, size, dir, attrs);
dev_WARN_ONCE(dev, 1,
--
2.30.2
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply related [flat|nested] 118+ messages in thread
* [PATCH 01/11] dma-direct: use is_swiotlb_active in dma_direct_map_page
@ 2022-02-27 14:30 ` Christoph Hellwig
0 siblings, 0 replies; 118+ messages in thread
From: Christoph Hellwig @ 2022-02-27 14:30 UTC (permalink / raw)
To: iommu
Cc: x86, Anshuman Khandual, Konrad Rzeszutek Wilk,
Stefano Stabellini, Boris Ostrovsky, Juergen Gross, Joerg Roedel,
David Woodhouse, Lu Baolu, Robin Murphy, linux-arm-kernel,
xen-devel, linux-ia64, linux-mips, linuxppc-dev, linux-riscv,
linux-s390, linux-hyperv, tboot-devel, linux-pci
Use the more specific is_swiotlb_active check instead of checking the
global swiotlb_force variable.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Anshuman Khandual <anshuman.khandual@arm.com>
---
kernel/dma/direct.h | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/kernel/dma/direct.h b/kernel/dma/direct.h
index 4632b0f4f72eb..4dc16e08c7e1a 100644
--- a/kernel/dma/direct.h
+++ b/kernel/dma/direct.h
@@ -91,7 +91,7 @@ static inline dma_addr_t dma_direct_map_page(struct device *dev,
return swiotlb_map(dev, phys, size, dir, attrs);
if (unlikely(!dma_capable(dev, dma_addr, size, true))) {
- if (swiotlb_force != SWIOTLB_NO_FORCE)
+ if (is_swiotlb_active(dev))
return swiotlb_map(dev, phys, size, dir, attrs);
dev_WARN_ONCE(dev, 1,
--
2.30.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 118+ messages in thread
* [PATCH 01/11] dma-direct: use is_swiotlb_active in dma_direct_map_page
@ 2022-02-27 14:30 ` Christoph Hellwig
0 siblings, 0 replies; 118+ messages in thread
From: Christoph Hellwig @ 2022-02-27 14:30 UTC (permalink / raw)
To: iommu
Cc: x86, Anshuman Khandual, Konrad Rzeszutek Wilk,
Stefano Stabellini, Boris Ostrovsky, Juergen Gross, Joerg Roedel,
David Woodhouse, Lu Baolu, Robin Murphy, linux-arm-kernel,
xen-devel, linux-ia64, linux-mips, linuxppc-dev, linux-riscv,
linux-s390, linux-hyperv, tboot-devel, linux-pci
Use the more specific is_swiotlb_active check instead of checking the
global swiotlb_force variable.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Anshuman Khandual <anshuman.khandual@arm.com>
---
kernel/dma/direct.h | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/kernel/dma/direct.h b/kernel/dma/direct.h
index 4632b0f4f72eb..4dc16e08c7e1a 100644
--- a/kernel/dma/direct.h
+++ b/kernel/dma/direct.h
@@ -91,7 +91,7 @@ static inline dma_addr_t dma_direct_map_page(struct device *dev,
return swiotlb_map(dev, phys, size, dir, attrs);
if (unlikely(!dma_capable(dev, dma_addr, size, true))) {
- if (swiotlb_force != SWIOTLB_NO_FORCE)
+ if (is_swiotlb_active(dev))
return swiotlb_map(dev, phys, size, dir, attrs);
dev_WARN_ONCE(dev, 1,
--
2.30.2
^ permalink raw reply related [flat|nested] 118+ messages in thread
* [PATCH 01/11] dma-direct: use is_swiotlb_active in dma_direct_map_page
@ 2022-02-27 14:30 ` Christoph Hellwig
0 siblings, 0 replies; 118+ messages in thread
From: Christoph Hellwig @ 2022-02-27 14:30 UTC (permalink / raw)
To: iommu
Cc: Juergen Gross, linux-s390, linux-hyperv, Stefano Stabellini,
linux-ia64, Anshuman Khandual, Konrad Rzeszutek Wilk,
Robin Murphy, Joerg Roedel, x86, linux-mips, linuxppc-dev,
tboot-devel, linux-pci, xen-devel, Boris Ostrovsky,
David Woodhouse, linux-riscv, linux-arm-kernel, Lu Baolu
Use the more specific is_swiotlb_active check instead of checking the
global swiotlb_force variable.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Anshuman Khandual <anshuman.khandual@arm.com>
---
kernel/dma/direct.h | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/kernel/dma/direct.h b/kernel/dma/direct.h
index 4632b0f4f72eb..4dc16e08c7e1a 100644
--- a/kernel/dma/direct.h
+++ b/kernel/dma/direct.h
@@ -91,7 +91,7 @@ static inline dma_addr_t dma_direct_map_page(struct device *dev,
return swiotlb_map(dev, phys, size, dir, attrs);
if (unlikely(!dma_capable(dev, dma_addr, size, true))) {
- if (swiotlb_force != SWIOTLB_NO_FORCE)
+ if (is_swiotlb_active(dev))
return swiotlb_map(dev, phys, size, dir, attrs);
dev_WARN_ONCE(dev, 1,
--
2.30.2
^ permalink raw reply related [flat|nested] 118+ messages in thread
* [PATCH 01/11] dma-direct: use is_swiotlb_active in dma_direct_map_page
@ 2022-02-27 14:30 ` Christoph Hellwig
0 siblings, 0 replies; 118+ messages in thread
From: Christoph Hellwig @ 2022-02-27 14:30 UTC (permalink / raw)
To: iommu
Cc: x86, Anshuman Khandual, Konrad Rzeszutek Wilk,
Stefano Stabellini, Boris Ostrovsky, Juergen Gross, Joerg Roedel,
David Woodhouse, Lu Baolu, Robin Murphy, linux-arm-kernel,
xen-devel, linux-ia64, linux-mips, linuxppc-dev, linux-riscv,
linux-s390, linux-hyperv, tboot-devel, linux-pci
Use the more specific is_swiotlb_active check instead of checking the
global swiotlb_force variable.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Anshuman Khandual <anshuman.khandual@arm.com>
---
kernel/dma/direct.h | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/kernel/dma/direct.h b/kernel/dma/direct.h
index 4632b0f4f72eb..4dc16e08c7e1a 100644
--- a/kernel/dma/direct.h
+++ b/kernel/dma/direct.h
@@ -91,7 +91,7 @@ static inline dma_addr_t dma_direct_map_page(struct device *dev,
return swiotlb_map(dev, phys, size, dir, attrs);
if (unlikely(!dma_capable(dev, dma_addr, size, true))) {
- if (swiotlb_force != SWIOTLB_NO_FORCE)
+ if (is_swiotlb_active(dev))
return swiotlb_map(dev, phys, size, dir, attrs);
dev_WARN_ONCE(dev, 1,
--
2.30.2
^ permalink raw reply related [flat|nested] 118+ messages in thread
* [PATCH 02/11] swiotlb: make swiotlb_exit a no-op if SWIOTLB_FORCE is set
2022-02-27 14:30 ` Christoph Hellwig
` (3 preceding siblings ...)
(?)
@ 2022-02-27 14:30 ` Christoph Hellwig
-1 siblings, 0 replies; 118+ messages in thread
From: Christoph Hellwig @ 2022-02-27 14:30 UTC (permalink / raw)
To: iommu
Cc: x86, Anshuman Khandual, Konrad Rzeszutek Wilk,
Stefano Stabellini, Boris Ostrovsky, Juergen Gross, Joerg Roedel,
David Woodhouse, Lu Baolu, Robin Murphy, linux-arm-kernel,
xen-devel, linux-ia64, linux-mips, linuxppc-dev, linux-riscv,
linux-s390, linux-hyperv, tboot-devel, linux-pci
If force bouncing is enabled we can't release the buffers.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Anshuman Khandual <anshuman.khandual@arm.com>
---
kernel/dma/swiotlb.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
index f1e7ea160b433..36fbf1181d285 100644
--- a/kernel/dma/swiotlb.c
+++ b/kernel/dma/swiotlb.c
@@ -378,6 +378,9 @@ void __init swiotlb_exit(void)
unsigned long tbl_vaddr;
size_t tbl_size, slots_size;
+ if (swiotlb_force == SWIOTLB_FORCE)
+ return;
+
if (!mem->nslabs)
return;
--
2.30.2
^ permalink raw reply related [flat|nested] 118+ messages in thread
* [PATCH 02/11] swiotlb: make swiotlb_exit a no-op if SWIOTLB_FORCE is set
@ 2022-02-27 14:30 ` Christoph Hellwig
0 siblings, 0 replies; 118+ messages in thread
From: Christoph Hellwig @ 2022-02-27 14:30 UTC (permalink / raw)
To: iommu
Cc: Juergen Gross, linux-s390, linux-hyperv, Stefano Stabellini,
linux-ia64, Anshuman Khandual, Konrad Rzeszutek Wilk,
Robin Murphy, x86, linux-mips, linuxppc-dev, tboot-devel,
linux-pci, xen-devel, Boris Ostrovsky, David Woodhouse,
linux-riscv, linux-arm-kernel
If force bouncing is enabled we can't release the buffers.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Anshuman Khandual <anshuman.khandual@arm.com>
---
kernel/dma/swiotlb.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
index f1e7ea160b433..36fbf1181d285 100644
--- a/kernel/dma/swiotlb.c
+++ b/kernel/dma/swiotlb.c
@@ -378,6 +378,9 @@ void __init swiotlb_exit(void)
unsigned long tbl_vaddr;
size_t tbl_size, slots_size;
+ if (swiotlb_force == SWIOTLB_FORCE)
+ return;
+
if (!mem->nslabs)
return;
--
2.30.2
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply related [flat|nested] 118+ messages in thread
* [PATCH 02/11] swiotlb: make swiotlb_exit a no-op if SWIOTLB_FORCE is set
@ 2022-02-27 14:30 ` Christoph Hellwig
0 siblings, 0 replies; 118+ messages in thread
From: Christoph Hellwig @ 2022-02-27 14:30 UTC (permalink / raw)
To: iommu
Cc: x86, Anshuman Khandual, Konrad Rzeszutek Wilk,
Stefano Stabellini, Boris Ostrovsky, Juergen Gross, Joerg Roedel,
David Woodhouse, Lu Baolu, Robin Murphy, linux-arm-kernel,
xen-devel, linux-ia64, linux-mips, linuxppc-dev, linux-riscv,
linux-s390, linux-hyperv, tboot-devel, linux-pci
If force bouncing is enabled we can't release the buffers.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Anshuman Khandual <anshuman.khandual@arm.com>
---
kernel/dma/swiotlb.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
index f1e7ea160b433..36fbf1181d285 100644
--- a/kernel/dma/swiotlb.c
+++ b/kernel/dma/swiotlb.c
@@ -378,6 +378,9 @@ void __init swiotlb_exit(void)
unsigned long tbl_vaddr;
size_t tbl_size, slots_size;
+ if (swiotlb_force == SWIOTLB_FORCE)
+ return;
+
if (!mem->nslabs)
return;
--
2.30.2
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply related [flat|nested] 118+ messages in thread
* [PATCH 02/11] swiotlb: make swiotlb_exit a no-op if SWIOTLB_FORCE is set
@ 2022-02-27 14:30 ` Christoph Hellwig
0 siblings, 0 replies; 118+ messages in thread
From: Christoph Hellwig @ 2022-02-27 14:30 UTC (permalink / raw)
To: iommu
Cc: Juergen Gross, linux-s390, linux-hyperv, Stefano Stabellini,
linux-ia64, Anshuman Khandual, Konrad Rzeszutek Wilk,
Robin Murphy, Joerg Roedel, x86, linux-mips, linuxppc-dev,
tboot-devel, linux-pci, xen-devel, Boris Ostrovsky,
David Woodhouse, linux-riscv, linux-arm-kernel, Lu Baolu
If force bouncing is enabled we can't release the buffers.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Anshuman Khandual <anshuman.khandual@arm.com>
---
kernel/dma/swiotlb.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
index f1e7ea160b433..36fbf1181d285 100644
--- a/kernel/dma/swiotlb.c
+++ b/kernel/dma/swiotlb.c
@@ -378,6 +378,9 @@ void __init swiotlb_exit(void)
unsigned long tbl_vaddr;
size_t tbl_size, slots_size;
+ if (swiotlb_force == SWIOTLB_FORCE)
+ return;
+
if (!mem->nslabs)
return;
--
2.30.2
^ permalink raw reply related [flat|nested] 118+ messages in thread
* [PATCH 02/11] swiotlb: make swiotlb_exit a no-op if SWIOTLB_FORCE is set
@ 2022-02-27 14:30 ` Christoph Hellwig
0 siblings, 0 replies; 118+ messages in thread
From: Christoph Hellwig @ 2022-02-27 14:30 UTC (permalink / raw)
To: iommu
Cc: x86, Anshuman Khandual, Konrad Rzeszutek Wilk,
Stefano Stabellini, Boris Ostrovsky, Juergen Gross, Joerg Roedel,
David Woodhouse, Lu Baolu, Robin Murphy, linux-arm-kernel,
xen-devel, linux-ia64, linux-mips, linuxppc-dev, linux-riscv,
linux-s390, linux-hyperv, tboot-devel, linux-pci
If force bouncing is enabled we can't release the buffers.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Anshuman Khandual <anshuman.khandual@arm.com>
---
kernel/dma/swiotlb.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
index f1e7ea160b433..36fbf1181d285 100644
--- a/kernel/dma/swiotlb.c
+++ b/kernel/dma/swiotlb.c
@@ -378,6 +378,9 @@ void __init swiotlb_exit(void)
unsigned long tbl_vaddr;
size_t tbl_size, slots_size;
+ if (swiotlb_force == SWIOTLB_FORCE)
+ return;
+
if (!mem->nslabs)
return;
--
2.30.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 118+ messages in thread
* [PATCH 02/11] swiotlb: make swiotlb_exit a no-op if SWIOTLB_FORCE is set
@ 2022-02-27 14:30 ` Christoph Hellwig
0 siblings, 0 replies; 118+ messages in thread
From: Christoph Hellwig @ 2022-02-27 14:30 UTC (permalink / raw)
To: iommu
Cc: x86, Anshuman Khandual, Konrad Rzeszutek Wilk,
Stefano Stabellini, Boris Ostrovsky, Juergen Gross, Joerg Roedel,
David Woodhouse, Lu Baolu, Robin Murphy, linux-arm-kernel,
xen-devel, linux-ia64, linux-mips, linuxppc-dev, linux-riscv,
linux-s390, linux-hyperv, tboot-devel, linux-pci
If force bouncing is enabled we can't release the buffers.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Anshuman Khandual <anshuman.khandual@arm.com>
---
kernel/dma/swiotlb.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
index f1e7ea160b433..36fbf1181d285 100644
--- a/kernel/dma/swiotlb.c
+++ b/kernel/dma/swiotlb.c
@@ -378,6 +378,9 @@ void __init swiotlb_exit(void)
unsigned long tbl_vaddr;
size_t tbl_size, slots_size;
+ if (swiotlb_force = SWIOTLB_FORCE)
+ return;
+
if (!mem->nslabs)
return;
--
2.30.2
^ permalink raw reply related [flat|nested] 118+ messages in thread
* [PATCH 03/11] swiotlb: simplify swiotlb_max_segment
2022-02-27 14:30 ` Christoph Hellwig
` (3 preceding siblings ...)
(?)
@ 2022-02-27 14:30 ` Christoph Hellwig
-1 siblings, 0 replies; 118+ messages in thread
From: Christoph Hellwig @ 2022-02-27 14:30 UTC (permalink / raw)
To: iommu
Cc: x86, Anshuman Khandual, Konrad Rzeszutek Wilk,
Stefano Stabellini, Boris Ostrovsky, Juergen Gross, Joerg Roedel,
David Woodhouse, Lu Baolu, Robin Murphy, linux-arm-kernel,
xen-devel, linux-ia64, linux-mips, linuxppc-dev, linux-riscv,
linux-s390, linux-hyperv, tboot-devel, linux-pci
Remove the bogus Xen override that was usually larger than the actual
size and just calculate the value on demand. Note that
swiotlb_max_segment still doesn't make sense as an interface and should
eventually be removed.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Anshuman Khandual <anshuman.khandual@arm.com>
---
drivers/xen/swiotlb-xen.c | 2 --
include/linux/swiotlb.h | 1 -
kernel/dma/swiotlb.c | 20 +++-----------------
3 files changed, 3 insertions(+), 20 deletions(-)
diff --git a/drivers/xen/swiotlb-xen.c b/drivers/xen/swiotlb-xen.c
index 47aebd98f52f5..485cd06ed39e7 100644
--- a/drivers/xen/swiotlb-xen.c
+++ b/drivers/xen/swiotlb-xen.c
@@ -202,7 +202,6 @@ int xen_swiotlb_init(void)
rc = swiotlb_late_init_with_tbl(start, nslabs);
if (rc)
return rc;
- swiotlb_set_max_segment(PAGE_SIZE);
return 0;
error:
if (nslabs > 1024 && repeat--) {
@@ -254,7 +253,6 @@ void __init xen_swiotlb_init_early(void)
if (swiotlb_init_with_tbl(start, nslabs, true))
panic("Cannot allocate SWIOTLB buffer");
- swiotlb_set_max_segment(PAGE_SIZE);
}
#endif /* CONFIG_X86 */
diff --git a/include/linux/swiotlb.h b/include/linux/swiotlb.h
index f6c3638255d54..9fb3a568f0c51 100644
--- a/include/linux/swiotlb.h
+++ b/include/linux/swiotlb.h
@@ -164,7 +164,6 @@ static inline void swiotlb_adjust_size(unsigned long size)
#endif /* CONFIG_SWIOTLB */
extern void swiotlb_print_info(void);
-extern void swiotlb_set_max_segment(unsigned int);
#ifdef CONFIG_DMA_RESTRICTED_POOL
struct page *swiotlb_alloc(struct device *dev, size_t size);
diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
index 36fbf1181d285..519e363097190 100644
--- a/kernel/dma/swiotlb.c
+++ b/kernel/dma/swiotlb.c
@@ -75,12 +75,6 @@ struct io_tlb_mem io_tlb_default_mem;
phys_addr_t swiotlb_unencrypted_base;
-/*
- * Max segment that we can provide which (if pages are contingous) will
- * not be bounced (unless SWIOTLB_FORCE is set).
- */
-static unsigned int max_segment;
-
static unsigned long default_nslabs = IO_TLB_DEFAULT_SIZE >> IO_TLB_SHIFT;
static int __init
@@ -104,18 +98,12 @@ early_param("swiotlb", setup_io_tlb_npages);
unsigned int swiotlb_max_segment(void)
{
- return io_tlb_default_mem.nslabs ? max_segment : 0;
+ if (!io_tlb_default_mem.nslabs)
+ return 0;
+ return rounddown(io_tlb_default_mem.nslabs << IO_TLB_SHIFT, PAGE_SIZE);
}
EXPORT_SYMBOL_GPL(swiotlb_max_segment);
-void swiotlb_set_max_segment(unsigned int val)
-{
- if (swiotlb_force == SWIOTLB_FORCE)
- max_segment = 1;
- else
- max_segment = rounddown(val, PAGE_SIZE);
-}
-
unsigned long swiotlb_size_or_default(void)
{
return default_nslabs << IO_TLB_SHIFT;
@@ -267,7 +255,6 @@ int __init swiotlb_init_with_tbl(char *tlb, unsigned long nslabs, int verbose)
if (verbose)
swiotlb_print_info();
- swiotlb_set_max_segment(mem->nslabs << IO_TLB_SHIFT);
return 0;
}
@@ -368,7 +355,6 @@ swiotlb_late_init_with_tbl(char *tlb, unsigned long nslabs)
swiotlb_init_io_tlb_mem(mem, virt_to_phys(tlb), nslabs, true);
swiotlb_print_info();
- swiotlb_set_max_segment(mem->nslabs << IO_TLB_SHIFT);
return 0;
}
--
2.30.2
^ permalink raw reply related [flat|nested] 118+ messages in thread
* [PATCH 03/11] swiotlb: simplify swiotlb_max_segment
@ 2022-02-27 14:30 ` Christoph Hellwig
0 siblings, 0 replies; 118+ messages in thread
From: Christoph Hellwig @ 2022-02-27 14:30 UTC (permalink / raw)
To: iommu
Cc: Juergen Gross, linux-s390, linux-hyperv, Stefano Stabellini,
linux-ia64, Anshuman Khandual, Konrad Rzeszutek Wilk,
Robin Murphy, x86, linux-mips, linuxppc-dev, tboot-devel,
linux-pci, xen-devel, Boris Ostrovsky, David Woodhouse,
linux-riscv, linux-arm-kernel
Remove the bogus Xen override that was usually larger than the actual
size and just calculate the value on demand. Note that
swiotlb_max_segment still doesn't make sense as an interface and should
eventually be removed.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Anshuman Khandual <anshuman.khandual@arm.com>
---
drivers/xen/swiotlb-xen.c | 2 --
include/linux/swiotlb.h | 1 -
kernel/dma/swiotlb.c | 20 +++-----------------
3 files changed, 3 insertions(+), 20 deletions(-)
diff --git a/drivers/xen/swiotlb-xen.c b/drivers/xen/swiotlb-xen.c
index 47aebd98f52f5..485cd06ed39e7 100644
--- a/drivers/xen/swiotlb-xen.c
+++ b/drivers/xen/swiotlb-xen.c
@@ -202,7 +202,6 @@ int xen_swiotlb_init(void)
rc = swiotlb_late_init_with_tbl(start, nslabs);
if (rc)
return rc;
- swiotlb_set_max_segment(PAGE_SIZE);
return 0;
error:
if (nslabs > 1024 && repeat--) {
@@ -254,7 +253,6 @@ void __init xen_swiotlb_init_early(void)
if (swiotlb_init_with_tbl(start, nslabs, true))
panic("Cannot allocate SWIOTLB buffer");
- swiotlb_set_max_segment(PAGE_SIZE);
}
#endif /* CONFIG_X86 */
diff --git a/include/linux/swiotlb.h b/include/linux/swiotlb.h
index f6c3638255d54..9fb3a568f0c51 100644
--- a/include/linux/swiotlb.h
+++ b/include/linux/swiotlb.h
@@ -164,7 +164,6 @@ static inline void swiotlb_adjust_size(unsigned long size)
#endif /* CONFIG_SWIOTLB */
extern void swiotlb_print_info(void);
-extern void swiotlb_set_max_segment(unsigned int);
#ifdef CONFIG_DMA_RESTRICTED_POOL
struct page *swiotlb_alloc(struct device *dev, size_t size);
diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
index 36fbf1181d285..519e363097190 100644
--- a/kernel/dma/swiotlb.c
+++ b/kernel/dma/swiotlb.c
@@ -75,12 +75,6 @@ struct io_tlb_mem io_tlb_default_mem;
phys_addr_t swiotlb_unencrypted_base;
-/*
- * Max segment that we can provide which (if pages are contingous) will
- * not be bounced (unless SWIOTLB_FORCE is set).
- */
-static unsigned int max_segment;
-
static unsigned long default_nslabs = IO_TLB_DEFAULT_SIZE >> IO_TLB_SHIFT;
static int __init
@@ -104,18 +98,12 @@ early_param("swiotlb", setup_io_tlb_npages);
unsigned int swiotlb_max_segment(void)
{
- return io_tlb_default_mem.nslabs ? max_segment : 0;
+ if (!io_tlb_default_mem.nslabs)
+ return 0;
+ return rounddown(io_tlb_default_mem.nslabs << IO_TLB_SHIFT, PAGE_SIZE);
}
EXPORT_SYMBOL_GPL(swiotlb_max_segment);
-void swiotlb_set_max_segment(unsigned int val)
-{
- if (swiotlb_force == SWIOTLB_FORCE)
- max_segment = 1;
- else
- max_segment = rounddown(val, PAGE_SIZE);
-}
-
unsigned long swiotlb_size_or_default(void)
{
return default_nslabs << IO_TLB_SHIFT;
@@ -267,7 +255,6 @@ int __init swiotlb_init_with_tbl(char *tlb, unsigned long nslabs, int verbose)
if (verbose)
swiotlb_print_info();
- swiotlb_set_max_segment(mem->nslabs << IO_TLB_SHIFT);
return 0;
}
@@ -368,7 +355,6 @@ swiotlb_late_init_with_tbl(char *tlb, unsigned long nslabs)
swiotlb_init_io_tlb_mem(mem, virt_to_phys(tlb), nslabs, true);
swiotlb_print_info();
- swiotlb_set_max_segment(mem->nslabs << IO_TLB_SHIFT);
return 0;
}
--
2.30.2
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply related [flat|nested] 118+ messages in thread
* [PATCH 03/11] swiotlb: simplify swiotlb_max_segment
@ 2022-02-27 14:30 ` Christoph Hellwig
0 siblings, 0 replies; 118+ messages in thread
From: Christoph Hellwig @ 2022-02-27 14:30 UTC (permalink / raw)
To: iommu
Cc: x86, Anshuman Khandual, Konrad Rzeszutek Wilk,
Stefano Stabellini, Boris Ostrovsky, Juergen Gross, Joerg Roedel,
David Woodhouse, Lu Baolu, Robin Murphy, linux-arm-kernel,
xen-devel, linux-ia64, linux-mips, linuxppc-dev, linux-riscv,
linux-s390, linux-hyperv, tboot-devel, linux-pci
Remove the bogus Xen override that was usually larger than the actual
size and just calculate the value on demand. Note that
swiotlb_max_segment still doesn't make sense as an interface and should
eventually be removed.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Anshuman Khandual <anshuman.khandual@arm.com>
---
drivers/xen/swiotlb-xen.c | 2 --
include/linux/swiotlb.h | 1 -
kernel/dma/swiotlb.c | 20 +++-----------------
3 files changed, 3 insertions(+), 20 deletions(-)
diff --git a/drivers/xen/swiotlb-xen.c b/drivers/xen/swiotlb-xen.c
index 47aebd98f52f5..485cd06ed39e7 100644
--- a/drivers/xen/swiotlb-xen.c
+++ b/drivers/xen/swiotlb-xen.c
@@ -202,7 +202,6 @@ int xen_swiotlb_init(void)
rc = swiotlb_late_init_with_tbl(start, nslabs);
if (rc)
return rc;
- swiotlb_set_max_segment(PAGE_SIZE);
return 0;
error:
if (nslabs > 1024 && repeat--) {
@@ -254,7 +253,6 @@ void __init xen_swiotlb_init_early(void)
if (swiotlb_init_with_tbl(start, nslabs, true))
panic("Cannot allocate SWIOTLB buffer");
- swiotlb_set_max_segment(PAGE_SIZE);
}
#endif /* CONFIG_X86 */
diff --git a/include/linux/swiotlb.h b/include/linux/swiotlb.h
index f6c3638255d54..9fb3a568f0c51 100644
--- a/include/linux/swiotlb.h
+++ b/include/linux/swiotlb.h
@@ -164,7 +164,6 @@ static inline void swiotlb_adjust_size(unsigned long size)
#endif /* CONFIG_SWIOTLB */
extern void swiotlb_print_info(void);
-extern void swiotlb_set_max_segment(unsigned int);
#ifdef CONFIG_DMA_RESTRICTED_POOL
struct page *swiotlb_alloc(struct device *dev, size_t size);
diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
index 36fbf1181d285..519e363097190 100644
--- a/kernel/dma/swiotlb.c
+++ b/kernel/dma/swiotlb.c
@@ -75,12 +75,6 @@ struct io_tlb_mem io_tlb_default_mem;
phys_addr_t swiotlb_unencrypted_base;
-/*
- * Max segment that we can provide which (if pages are contingous) will
- * not be bounced (unless SWIOTLB_FORCE is set).
- */
-static unsigned int max_segment;
-
static unsigned long default_nslabs = IO_TLB_DEFAULT_SIZE >> IO_TLB_SHIFT;
static int __init
@@ -104,18 +98,12 @@ early_param("swiotlb", setup_io_tlb_npages);
unsigned int swiotlb_max_segment(void)
{
- return io_tlb_default_mem.nslabs ? max_segment : 0;
+ if (!io_tlb_default_mem.nslabs)
+ return 0;
+ return rounddown(io_tlb_default_mem.nslabs << IO_TLB_SHIFT, PAGE_SIZE);
}
EXPORT_SYMBOL_GPL(swiotlb_max_segment);
-void swiotlb_set_max_segment(unsigned int val)
-{
- if (swiotlb_force == SWIOTLB_FORCE)
- max_segment = 1;
- else
- max_segment = rounddown(val, PAGE_SIZE);
-}
-
unsigned long swiotlb_size_or_default(void)
{
return default_nslabs << IO_TLB_SHIFT;
@@ -267,7 +255,6 @@ int __init swiotlb_init_with_tbl(char *tlb, unsigned long nslabs, int verbose)
if (verbose)
swiotlb_print_info();
- swiotlb_set_max_segment(mem->nslabs << IO_TLB_SHIFT);
return 0;
}
@@ -368,7 +355,6 @@ swiotlb_late_init_with_tbl(char *tlb, unsigned long nslabs)
swiotlb_init_io_tlb_mem(mem, virt_to_phys(tlb), nslabs, true);
swiotlb_print_info();
- swiotlb_set_max_segment(mem->nslabs << IO_TLB_SHIFT);
return 0;
}
--
2.30.2
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply related [flat|nested] 118+ messages in thread
* [PATCH 03/11] swiotlb: simplify swiotlb_max_segment
@ 2022-02-27 14:30 ` Christoph Hellwig
0 siblings, 0 replies; 118+ messages in thread
From: Christoph Hellwig @ 2022-02-27 14:30 UTC (permalink / raw)
To: iommu
Cc: x86, Anshuman Khandual, Konrad Rzeszutek Wilk,
Stefano Stabellini, Boris Ostrovsky, Juergen Gross, Joerg Roedel,
David Woodhouse, Lu Baolu, Robin Murphy, linux-arm-kernel,
xen-devel, linux-ia64, linux-mips, linuxppc-dev, linux-riscv,
linux-s390, linux-hyperv, tboot-devel, linux-pci
Remove the bogus Xen override that was usually larger than the actual
size and just calculate the value on demand. Note that
swiotlb_max_segment still doesn't make sense as an interface and should
eventually be removed.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Anshuman Khandual <anshuman.khandual@arm.com>
---
drivers/xen/swiotlb-xen.c | 2 --
include/linux/swiotlb.h | 1 -
kernel/dma/swiotlb.c | 20 +++-----------------
3 files changed, 3 insertions(+), 20 deletions(-)
diff --git a/drivers/xen/swiotlb-xen.c b/drivers/xen/swiotlb-xen.c
index 47aebd98f52f5..485cd06ed39e7 100644
--- a/drivers/xen/swiotlb-xen.c
+++ b/drivers/xen/swiotlb-xen.c
@@ -202,7 +202,6 @@ int xen_swiotlb_init(void)
rc = swiotlb_late_init_with_tbl(start, nslabs);
if (rc)
return rc;
- swiotlb_set_max_segment(PAGE_SIZE);
return 0;
error:
if (nslabs > 1024 && repeat--) {
@@ -254,7 +253,6 @@ void __init xen_swiotlb_init_early(void)
if (swiotlb_init_with_tbl(start, nslabs, true))
panic("Cannot allocate SWIOTLB buffer");
- swiotlb_set_max_segment(PAGE_SIZE);
}
#endif /* CONFIG_X86 */
diff --git a/include/linux/swiotlb.h b/include/linux/swiotlb.h
index f6c3638255d54..9fb3a568f0c51 100644
--- a/include/linux/swiotlb.h
+++ b/include/linux/swiotlb.h
@@ -164,7 +164,6 @@ static inline void swiotlb_adjust_size(unsigned long size)
#endif /* CONFIG_SWIOTLB */
extern void swiotlb_print_info(void);
-extern void swiotlb_set_max_segment(unsigned int);
#ifdef CONFIG_DMA_RESTRICTED_POOL
struct page *swiotlb_alloc(struct device *dev, size_t size);
diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
index 36fbf1181d285..519e363097190 100644
--- a/kernel/dma/swiotlb.c
+++ b/kernel/dma/swiotlb.c
@@ -75,12 +75,6 @@ struct io_tlb_mem io_tlb_default_mem;
phys_addr_t swiotlb_unencrypted_base;
-/*
- * Max segment that we can provide which (if pages are contingous) will
- * not be bounced (unless SWIOTLB_FORCE is set).
- */
-static unsigned int max_segment;
-
static unsigned long default_nslabs = IO_TLB_DEFAULT_SIZE >> IO_TLB_SHIFT;
static int __init
@@ -104,18 +98,12 @@ early_param("swiotlb", setup_io_tlb_npages);
unsigned int swiotlb_max_segment(void)
{
- return io_tlb_default_mem.nslabs ? max_segment : 0;
+ if (!io_tlb_default_mem.nslabs)
+ return 0;
+ return rounddown(io_tlb_default_mem.nslabs << IO_TLB_SHIFT, PAGE_SIZE);
}
EXPORT_SYMBOL_GPL(swiotlb_max_segment);
-void swiotlb_set_max_segment(unsigned int val)
-{
- if (swiotlb_force == SWIOTLB_FORCE)
- max_segment = 1;
- else
- max_segment = rounddown(val, PAGE_SIZE);
-}
-
unsigned long swiotlb_size_or_default(void)
{
return default_nslabs << IO_TLB_SHIFT;
@@ -267,7 +255,6 @@ int __init swiotlb_init_with_tbl(char *tlb, unsigned long nslabs, int verbose)
if (verbose)
swiotlb_print_info();
- swiotlb_set_max_segment(mem->nslabs << IO_TLB_SHIFT);
return 0;
}
@@ -368,7 +355,6 @@ swiotlb_late_init_with_tbl(char *tlb, unsigned long nslabs)
swiotlb_init_io_tlb_mem(mem, virt_to_phys(tlb), nslabs, true);
swiotlb_print_info();
- swiotlb_set_max_segment(mem->nslabs << IO_TLB_SHIFT);
return 0;
}
--
2.30.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 118+ messages in thread
* [PATCH 03/11] swiotlb: simplify swiotlb_max_segment
@ 2022-02-27 14:30 ` Christoph Hellwig
0 siblings, 0 replies; 118+ messages in thread
From: Christoph Hellwig @ 2022-02-27 14:30 UTC (permalink / raw)
To: iommu
Cc: Juergen Gross, linux-s390, linux-hyperv, Stefano Stabellini,
linux-ia64, Anshuman Khandual, Konrad Rzeszutek Wilk,
Robin Murphy, Joerg Roedel, x86, linux-mips, linuxppc-dev,
tboot-devel, linux-pci, xen-devel, Boris Ostrovsky,
David Woodhouse, linux-riscv, linux-arm-kernel, Lu Baolu
Remove the bogus Xen override that was usually larger than the actual
size and just calculate the value on demand. Note that
swiotlb_max_segment still doesn't make sense as an interface and should
eventually be removed.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Anshuman Khandual <anshuman.khandual@arm.com>
---
drivers/xen/swiotlb-xen.c | 2 --
include/linux/swiotlb.h | 1 -
kernel/dma/swiotlb.c | 20 +++-----------------
3 files changed, 3 insertions(+), 20 deletions(-)
diff --git a/drivers/xen/swiotlb-xen.c b/drivers/xen/swiotlb-xen.c
index 47aebd98f52f5..485cd06ed39e7 100644
--- a/drivers/xen/swiotlb-xen.c
+++ b/drivers/xen/swiotlb-xen.c
@@ -202,7 +202,6 @@ int xen_swiotlb_init(void)
rc = swiotlb_late_init_with_tbl(start, nslabs);
if (rc)
return rc;
- swiotlb_set_max_segment(PAGE_SIZE);
return 0;
error:
if (nslabs > 1024 && repeat--) {
@@ -254,7 +253,6 @@ void __init xen_swiotlb_init_early(void)
if (swiotlb_init_with_tbl(start, nslabs, true))
panic("Cannot allocate SWIOTLB buffer");
- swiotlb_set_max_segment(PAGE_SIZE);
}
#endif /* CONFIG_X86 */
diff --git a/include/linux/swiotlb.h b/include/linux/swiotlb.h
index f6c3638255d54..9fb3a568f0c51 100644
--- a/include/linux/swiotlb.h
+++ b/include/linux/swiotlb.h
@@ -164,7 +164,6 @@ static inline void swiotlb_adjust_size(unsigned long size)
#endif /* CONFIG_SWIOTLB */
extern void swiotlb_print_info(void);
-extern void swiotlb_set_max_segment(unsigned int);
#ifdef CONFIG_DMA_RESTRICTED_POOL
struct page *swiotlb_alloc(struct device *dev, size_t size);
diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
index 36fbf1181d285..519e363097190 100644
--- a/kernel/dma/swiotlb.c
+++ b/kernel/dma/swiotlb.c
@@ -75,12 +75,6 @@ struct io_tlb_mem io_tlb_default_mem;
phys_addr_t swiotlb_unencrypted_base;
-/*
- * Max segment that we can provide which (if pages are contingous) will
- * not be bounced (unless SWIOTLB_FORCE is set).
- */
-static unsigned int max_segment;
-
static unsigned long default_nslabs = IO_TLB_DEFAULT_SIZE >> IO_TLB_SHIFT;
static int __init
@@ -104,18 +98,12 @@ early_param("swiotlb", setup_io_tlb_npages);
unsigned int swiotlb_max_segment(void)
{
- return io_tlb_default_mem.nslabs ? max_segment : 0;
+ if (!io_tlb_default_mem.nslabs)
+ return 0;
+ return rounddown(io_tlb_default_mem.nslabs << IO_TLB_SHIFT, PAGE_SIZE);
}
EXPORT_SYMBOL_GPL(swiotlb_max_segment);
-void swiotlb_set_max_segment(unsigned int val)
-{
- if (swiotlb_force == SWIOTLB_FORCE)
- max_segment = 1;
- else
- max_segment = rounddown(val, PAGE_SIZE);
-}
-
unsigned long swiotlb_size_or_default(void)
{
return default_nslabs << IO_TLB_SHIFT;
@@ -267,7 +255,6 @@ int __init swiotlb_init_with_tbl(char *tlb, unsigned long nslabs, int verbose)
if (verbose)
swiotlb_print_info();
- swiotlb_set_max_segment(mem->nslabs << IO_TLB_SHIFT);
return 0;
}
@@ -368,7 +355,6 @@ swiotlb_late_init_with_tbl(char *tlb, unsigned long nslabs)
swiotlb_init_io_tlb_mem(mem, virt_to_phys(tlb), nslabs, true);
swiotlb_print_info();
- swiotlb_set_max_segment(mem->nslabs << IO_TLB_SHIFT);
return 0;
}
--
2.30.2
^ permalink raw reply related [flat|nested] 118+ messages in thread
* [PATCH 03/11] swiotlb: simplify swiotlb_max_segment
@ 2022-02-27 14:30 ` Christoph Hellwig
0 siblings, 0 replies; 118+ messages in thread
From: Christoph Hellwig @ 2022-02-27 14:30 UTC (permalink / raw)
To: iommu
Cc: x86, Anshuman Khandual, Konrad Rzeszutek Wilk,
Stefano Stabellini, Boris Ostrovsky, Juergen Gross, Joerg Roedel,
David Woodhouse, Lu Baolu, Robin Murphy, linux-arm-kernel,
xen-devel, linux-ia64, linux-mips, linuxppc-dev, linux-riscv,
linux-s390, linux-hyperv, tboot-devel, linux-pci
Remove the bogus Xen override that was usually larger than the actual
size and just calculate the value on demand. Note that
swiotlb_max_segment still doesn't make sense as an interface and should
eventually be removed.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Anshuman Khandual <anshuman.khandual@arm.com>
---
drivers/xen/swiotlb-xen.c | 2 --
include/linux/swiotlb.h | 1 -
kernel/dma/swiotlb.c | 20 +++-----------------
3 files changed, 3 insertions(+), 20 deletions(-)
diff --git a/drivers/xen/swiotlb-xen.c b/drivers/xen/swiotlb-xen.c
index 47aebd98f52f5..485cd06ed39e7 100644
--- a/drivers/xen/swiotlb-xen.c
+++ b/drivers/xen/swiotlb-xen.c
@@ -202,7 +202,6 @@ int xen_swiotlb_init(void)
rc = swiotlb_late_init_with_tbl(start, nslabs);
if (rc)
return rc;
- swiotlb_set_max_segment(PAGE_SIZE);
return 0;
error:
if (nslabs > 1024 && repeat--) {
@@ -254,7 +253,6 @@ void __init xen_swiotlb_init_early(void)
if (swiotlb_init_with_tbl(start, nslabs, true))
panic("Cannot allocate SWIOTLB buffer");
- swiotlb_set_max_segment(PAGE_SIZE);
}
#endif /* CONFIG_X86 */
diff --git a/include/linux/swiotlb.h b/include/linux/swiotlb.h
index f6c3638255d54..9fb3a568f0c51 100644
--- a/include/linux/swiotlb.h
+++ b/include/linux/swiotlb.h
@@ -164,7 +164,6 @@ static inline void swiotlb_adjust_size(unsigned long size)
#endif /* CONFIG_SWIOTLB */
extern void swiotlb_print_info(void);
-extern void swiotlb_set_max_segment(unsigned int);
#ifdef CONFIG_DMA_RESTRICTED_POOL
struct page *swiotlb_alloc(struct device *dev, size_t size);
diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
index 36fbf1181d285..519e363097190 100644
--- a/kernel/dma/swiotlb.c
+++ b/kernel/dma/swiotlb.c
@@ -75,12 +75,6 @@ struct io_tlb_mem io_tlb_default_mem;
phys_addr_t swiotlb_unencrypted_base;
-/*
- * Max segment that we can provide which (if pages are contingous) will
- * not be bounced (unless SWIOTLB_FORCE is set).
- */
-static unsigned int max_segment;
-
static unsigned long default_nslabs = IO_TLB_DEFAULT_SIZE >> IO_TLB_SHIFT;
static int __init
@@ -104,18 +98,12 @@ early_param("swiotlb", setup_io_tlb_npages);
unsigned int swiotlb_max_segment(void)
{
- return io_tlb_default_mem.nslabs ? max_segment : 0;
+ if (!io_tlb_default_mem.nslabs)
+ return 0;
+ return rounddown(io_tlb_default_mem.nslabs << IO_TLB_SHIFT, PAGE_SIZE);
}
EXPORT_SYMBOL_GPL(swiotlb_max_segment);
-void swiotlb_set_max_segment(unsigned int val)
-{
- if (swiotlb_force = SWIOTLB_FORCE)
- max_segment = 1;
- else
- max_segment = rounddown(val, PAGE_SIZE);
-}
-
unsigned long swiotlb_size_or_default(void)
{
return default_nslabs << IO_TLB_SHIFT;
@@ -267,7 +255,6 @@ int __init swiotlb_init_with_tbl(char *tlb, unsigned long nslabs, int verbose)
if (verbose)
swiotlb_print_info();
- swiotlb_set_max_segment(mem->nslabs << IO_TLB_SHIFT);
return 0;
}
@@ -368,7 +355,6 @@ swiotlb_late_init_with_tbl(char *tlb, unsigned long nslabs)
swiotlb_init_io_tlb_mem(mem, virt_to_phys(tlb), nslabs, true);
swiotlb_print_info();
- swiotlb_set_max_segment(mem->nslabs << IO_TLB_SHIFT);
return 0;
}
--
2.30.2
^ permalink raw reply related [flat|nested] 118+ messages in thread
* [PATCH 04/11] swiotlb: rename swiotlb_late_init_with_default_size
2022-02-27 14:30 ` Christoph Hellwig
` (3 preceding siblings ...)
(?)
@ 2022-02-27 14:30 ` Christoph Hellwig
-1 siblings, 0 replies; 118+ messages in thread
From: Christoph Hellwig @ 2022-02-27 14:30 UTC (permalink / raw)
To: iommu
Cc: x86, Anshuman Khandual, Konrad Rzeszutek Wilk,
Stefano Stabellini, Boris Ostrovsky, Juergen Gross, Joerg Roedel,
David Woodhouse, Lu Baolu, Robin Murphy, linux-arm-kernel,
xen-devel, linux-ia64, linux-mips, linuxppc-dev, linux-riscv,
linux-s390, linux-hyperv, tboot-devel, linux-pci
swiotlb_late_init_with_default_size is an overly verbose name that
doesn't even catch what the function is doing, given that the size is
not just a default but the actual requested size.
Rename it to swiotlb_init_late.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Anshuman Khandual <anshuman.khandual@arm.com>
---
arch/x86/pci/sta2x11-fixup.c | 2 +-
include/linux/swiotlb.h | 2 +-
kernel/dma/swiotlb.c | 6 ++----
3 files changed, 4 insertions(+), 6 deletions(-)
diff --git a/arch/x86/pci/sta2x11-fixup.c b/arch/x86/pci/sta2x11-fixup.c
index 101081ad64b6d..e0c039a75b2db 100644
--- a/arch/x86/pci/sta2x11-fixup.c
+++ b/arch/x86/pci/sta2x11-fixup.c
@@ -57,7 +57,7 @@ static void sta2x11_new_instance(struct pci_dev *pdev)
int size = STA2X11_SWIOTLB_SIZE;
/* First instance: register your own swiotlb area */
dev_info(&pdev->dev, "Using SWIOTLB (size %i)\n", size);
- if (swiotlb_late_init_with_default_size(size))
+ if (swiotlb_init_late(size))
dev_emerg(&pdev->dev, "init swiotlb failed\n");
}
list_add(&instance->list, &sta2x11_instance_list);
diff --git a/include/linux/swiotlb.h b/include/linux/swiotlb.h
index 9fb3a568f0c51..b48b26bfa0edb 100644
--- a/include/linux/swiotlb.h
+++ b/include/linux/swiotlb.h
@@ -40,7 +40,7 @@ extern void swiotlb_init(int verbose);
int swiotlb_init_with_tbl(char *tlb, unsigned long nslabs, int verbose);
unsigned long swiotlb_size_or_default(void);
extern int swiotlb_late_init_with_tbl(char *tlb, unsigned long nslabs);
-extern int swiotlb_late_init_with_default_size(size_t default_size);
+int swiotlb_init_late(size_t size);
extern void __init swiotlb_update_mem_attributes(void);
phys_addr_t swiotlb_tbl_map_single(struct device *hwdev, phys_addr_t phys,
diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
index 519e363097190..5f64b02fbb732 100644
--- a/kernel/dma/swiotlb.c
+++ b/kernel/dma/swiotlb.c
@@ -290,11 +290,9 @@ swiotlb_init(int verbose)
* initialize the swiotlb later using the slab allocator if needed.
* This should be just like above, but with some error catching.
*/
-int
-swiotlb_late_init_with_default_size(size_t default_size)
+int swiotlb_init_late(size_t size)
{
- unsigned long nslabs =
- ALIGN(default_size >> IO_TLB_SHIFT, IO_TLB_SEGSIZE);
+ unsigned long nslabs = ALIGN(size >> IO_TLB_SHIFT, IO_TLB_SEGSIZE);
unsigned long bytes;
unsigned char *vstart = NULL;
unsigned int order;
--
2.30.2
^ permalink raw reply related [flat|nested] 118+ messages in thread
* [PATCH 04/11] swiotlb: rename swiotlb_late_init_with_default_size
@ 2022-02-27 14:30 ` Christoph Hellwig
0 siblings, 0 replies; 118+ messages in thread
From: Christoph Hellwig @ 2022-02-27 14:30 UTC (permalink / raw)
To: iommu
Cc: Juergen Gross, linux-s390, linux-hyperv, Stefano Stabellini,
linux-ia64, Anshuman Khandual, Konrad Rzeszutek Wilk,
Robin Murphy, x86, linux-mips, linuxppc-dev, tboot-devel,
linux-pci, xen-devel, Boris Ostrovsky, David Woodhouse,
linux-riscv, linux-arm-kernel
swiotlb_late_init_with_default_size is an overly verbose name that
doesn't even catch what the function is doing, given that the size is
not just a default but the actual requested size.
Rename it to swiotlb_init_late.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Anshuman Khandual <anshuman.khandual@arm.com>
---
arch/x86/pci/sta2x11-fixup.c | 2 +-
include/linux/swiotlb.h | 2 +-
kernel/dma/swiotlb.c | 6 ++----
3 files changed, 4 insertions(+), 6 deletions(-)
diff --git a/arch/x86/pci/sta2x11-fixup.c b/arch/x86/pci/sta2x11-fixup.c
index 101081ad64b6d..e0c039a75b2db 100644
--- a/arch/x86/pci/sta2x11-fixup.c
+++ b/arch/x86/pci/sta2x11-fixup.c
@@ -57,7 +57,7 @@ static void sta2x11_new_instance(struct pci_dev *pdev)
int size = STA2X11_SWIOTLB_SIZE;
/* First instance: register your own swiotlb area */
dev_info(&pdev->dev, "Using SWIOTLB (size %i)\n", size);
- if (swiotlb_late_init_with_default_size(size))
+ if (swiotlb_init_late(size))
dev_emerg(&pdev->dev, "init swiotlb failed\n");
}
list_add(&instance->list, &sta2x11_instance_list);
diff --git a/include/linux/swiotlb.h b/include/linux/swiotlb.h
index 9fb3a568f0c51..b48b26bfa0edb 100644
--- a/include/linux/swiotlb.h
+++ b/include/linux/swiotlb.h
@@ -40,7 +40,7 @@ extern void swiotlb_init(int verbose);
int swiotlb_init_with_tbl(char *tlb, unsigned long nslabs, int verbose);
unsigned long swiotlb_size_or_default(void);
extern int swiotlb_late_init_with_tbl(char *tlb, unsigned long nslabs);
-extern int swiotlb_late_init_with_default_size(size_t default_size);
+int swiotlb_init_late(size_t size);
extern void __init swiotlb_update_mem_attributes(void);
phys_addr_t swiotlb_tbl_map_single(struct device *hwdev, phys_addr_t phys,
diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
index 519e363097190..5f64b02fbb732 100644
--- a/kernel/dma/swiotlb.c
+++ b/kernel/dma/swiotlb.c
@@ -290,11 +290,9 @@ swiotlb_init(int verbose)
* initialize the swiotlb later using the slab allocator if needed.
* This should be just like above, but with some error catching.
*/
-int
-swiotlb_late_init_with_default_size(size_t default_size)
+int swiotlb_init_late(size_t size)
{
- unsigned long nslabs =
- ALIGN(default_size >> IO_TLB_SHIFT, IO_TLB_SEGSIZE);
+ unsigned long nslabs = ALIGN(size >> IO_TLB_SHIFT, IO_TLB_SEGSIZE);
unsigned long bytes;
unsigned char *vstart = NULL;
unsigned int order;
--
2.30.2
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply related [flat|nested] 118+ messages in thread
* [PATCH 04/11] swiotlb: rename swiotlb_late_init_with_default_size
@ 2022-02-27 14:30 ` Christoph Hellwig
0 siblings, 0 replies; 118+ messages in thread
From: Christoph Hellwig @ 2022-02-27 14:30 UTC (permalink / raw)
To: iommu
Cc: x86, Anshuman Khandual, Konrad Rzeszutek Wilk,
Stefano Stabellini, Boris Ostrovsky, Juergen Gross, Joerg Roedel,
David Woodhouse, Lu Baolu, Robin Murphy, linux-arm-kernel,
xen-devel, linux-ia64, linux-mips, linuxppc-dev, linux-riscv,
linux-s390, linux-hyperv, tboot-devel, linux-pci
swiotlb_late_init_with_default_size is an overly verbose name that
doesn't even catch what the function is doing, given that the size is
not just a default but the actual requested size.
Rename it to swiotlb_init_late.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Anshuman Khandual <anshuman.khandual@arm.com>
---
arch/x86/pci/sta2x11-fixup.c | 2 +-
include/linux/swiotlb.h | 2 +-
kernel/dma/swiotlb.c | 6 ++----
3 files changed, 4 insertions(+), 6 deletions(-)
diff --git a/arch/x86/pci/sta2x11-fixup.c b/arch/x86/pci/sta2x11-fixup.c
index 101081ad64b6d..e0c039a75b2db 100644
--- a/arch/x86/pci/sta2x11-fixup.c
+++ b/arch/x86/pci/sta2x11-fixup.c
@@ -57,7 +57,7 @@ static void sta2x11_new_instance(struct pci_dev *pdev)
int size = STA2X11_SWIOTLB_SIZE;
/* First instance: register your own swiotlb area */
dev_info(&pdev->dev, "Using SWIOTLB (size %i)\n", size);
- if (swiotlb_late_init_with_default_size(size))
+ if (swiotlb_init_late(size))
dev_emerg(&pdev->dev, "init swiotlb failed\n");
}
list_add(&instance->list, &sta2x11_instance_list);
diff --git a/include/linux/swiotlb.h b/include/linux/swiotlb.h
index 9fb3a568f0c51..b48b26bfa0edb 100644
--- a/include/linux/swiotlb.h
+++ b/include/linux/swiotlb.h
@@ -40,7 +40,7 @@ extern void swiotlb_init(int verbose);
int swiotlb_init_with_tbl(char *tlb, unsigned long nslabs, int verbose);
unsigned long swiotlb_size_or_default(void);
extern int swiotlb_late_init_with_tbl(char *tlb, unsigned long nslabs);
-extern int swiotlb_late_init_with_default_size(size_t default_size);
+int swiotlb_init_late(size_t size);
extern void __init swiotlb_update_mem_attributes(void);
phys_addr_t swiotlb_tbl_map_single(struct device *hwdev, phys_addr_t phys,
diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
index 519e363097190..5f64b02fbb732 100644
--- a/kernel/dma/swiotlb.c
+++ b/kernel/dma/swiotlb.c
@@ -290,11 +290,9 @@ swiotlb_init(int verbose)
* initialize the swiotlb later using the slab allocator if needed.
* This should be just like above, but with some error catching.
*/
-int
-swiotlb_late_init_with_default_size(size_t default_size)
+int swiotlb_init_late(size_t size)
{
- unsigned long nslabs =
- ALIGN(default_size >> IO_TLB_SHIFT, IO_TLB_SEGSIZE);
+ unsigned long nslabs = ALIGN(size >> IO_TLB_SHIFT, IO_TLB_SEGSIZE);
unsigned long bytes;
unsigned char *vstart = NULL;
unsigned int order;
--
2.30.2
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply related [flat|nested] 118+ messages in thread
* [PATCH 04/11] swiotlb: rename swiotlb_late_init_with_default_size
@ 2022-02-27 14:30 ` Christoph Hellwig
0 siblings, 0 replies; 118+ messages in thread
From: Christoph Hellwig @ 2022-02-27 14:30 UTC (permalink / raw)
To: iommu
Cc: Juergen Gross, linux-s390, linux-hyperv, Stefano Stabellini,
linux-ia64, Anshuman Khandual, Konrad Rzeszutek Wilk,
Robin Murphy, Joerg Roedel, x86, linux-mips, linuxppc-dev,
tboot-devel, linux-pci, xen-devel, Boris Ostrovsky,
David Woodhouse, linux-riscv, linux-arm-kernel, Lu Baolu
swiotlb_late_init_with_default_size is an overly verbose name that
doesn't even catch what the function is doing, given that the size is
not just a default but the actual requested size.
Rename it to swiotlb_init_late.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Anshuman Khandual <anshuman.khandual@arm.com>
---
arch/x86/pci/sta2x11-fixup.c | 2 +-
include/linux/swiotlb.h | 2 +-
kernel/dma/swiotlb.c | 6 ++----
3 files changed, 4 insertions(+), 6 deletions(-)
diff --git a/arch/x86/pci/sta2x11-fixup.c b/arch/x86/pci/sta2x11-fixup.c
index 101081ad64b6d..e0c039a75b2db 100644
--- a/arch/x86/pci/sta2x11-fixup.c
+++ b/arch/x86/pci/sta2x11-fixup.c
@@ -57,7 +57,7 @@ static void sta2x11_new_instance(struct pci_dev *pdev)
int size = STA2X11_SWIOTLB_SIZE;
/* First instance: register your own swiotlb area */
dev_info(&pdev->dev, "Using SWIOTLB (size %i)\n", size);
- if (swiotlb_late_init_with_default_size(size))
+ if (swiotlb_init_late(size))
dev_emerg(&pdev->dev, "init swiotlb failed\n");
}
list_add(&instance->list, &sta2x11_instance_list);
diff --git a/include/linux/swiotlb.h b/include/linux/swiotlb.h
index 9fb3a568f0c51..b48b26bfa0edb 100644
--- a/include/linux/swiotlb.h
+++ b/include/linux/swiotlb.h
@@ -40,7 +40,7 @@ extern void swiotlb_init(int verbose);
int swiotlb_init_with_tbl(char *tlb, unsigned long nslabs, int verbose);
unsigned long swiotlb_size_or_default(void);
extern int swiotlb_late_init_with_tbl(char *tlb, unsigned long nslabs);
-extern int swiotlb_late_init_with_default_size(size_t default_size);
+int swiotlb_init_late(size_t size);
extern void __init swiotlb_update_mem_attributes(void);
phys_addr_t swiotlb_tbl_map_single(struct device *hwdev, phys_addr_t phys,
diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
index 519e363097190..5f64b02fbb732 100644
--- a/kernel/dma/swiotlb.c
+++ b/kernel/dma/swiotlb.c
@@ -290,11 +290,9 @@ swiotlb_init(int verbose)
* initialize the swiotlb later using the slab allocator if needed.
* This should be just like above, but with some error catching.
*/
-int
-swiotlb_late_init_with_default_size(size_t default_size)
+int swiotlb_init_late(size_t size)
{
- unsigned long nslabs =
- ALIGN(default_size >> IO_TLB_SHIFT, IO_TLB_SEGSIZE);
+ unsigned long nslabs = ALIGN(size >> IO_TLB_SHIFT, IO_TLB_SEGSIZE);
unsigned long bytes;
unsigned char *vstart = NULL;
unsigned int order;
--
2.30.2
^ permalink raw reply related [flat|nested] 118+ messages in thread
* [PATCH 04/11] swiotlb: rename swiotlb_late_init_with_default_size
@ 2022-02-27 14:30 ` Christoph Hellwig
0 siblings, 0 replies; 118+ messages in thread
From: Christoph Hellwig @ 2022-02-27 14:30 UTC (permalink / raw)
To: iommu
Cc: x86, Anshuman Khandual, Konrad Rzeszutek Wilk,
Stefano Stabellini, Boris Ostrovsky, Juergen Gross, Joerg Roedel,
David Woodhouse, Lu Baolu, Robin Murphy, linux-arm-kernel,
xen-devel, linux-ia64, linux-mips, linuxppc-dev, linux-riscv,
linux-s390, linux-hyperv, tboot-devel, linux-pci
swiotlb_late_init_with_default_size is an overly verbose name that
doesn't even catch what the function is doing, given that the size is
not just a default but the actual requested size.
Rename it to swiotlb_init_late.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Anshuman Khandual <anshuman.khandual@arm.com>
---
arch/x86/pci/sta2x11-fixup.c | 2 +-
include/linux/swiotlb.h | 2 +-
kernel/dma/swiotlb.c | 6 ++----
3 files changed, 4 insertions(+), 6 deletions(-)
diff --git a/arch/x86/pci/sta2x11-fixup.c b/arch/x86/pci/sta2x11-fixup.c
index 101081ad64b6d..e0c039a75b2db 100644
--- a/arch/x86/pci/sta2x11-fixup.c
+++ b/arch/x86/pci/sta2x11-fixup.c
@@ -57,7 +57,7 @@ static void sta2x11_new_instance(struct pci_dev *pdev)
int size = STA2X11_SWIOTLB_SIZE;
/* First instance: register your own swiotlb area */
dev_info(&pdev->dev, "Using SWIOTLB (size %i)\n", size);
- if (swiotlb_late_init_with_default_size(size))
+ if (swiotlb_init_late(size))
dev_emerg(&pdev->dev, "init swiotlb failed\n");
}
list_add(&instance->list, &sta2x11_instance_list);
diff --git a/include/linux/swiotlb.h b/include/linux/swiotlb.h
index 9fb3a568f0c51..b48b26bfa0edb 100644
--- a/include/linux/swiotlb.h
+++ b/include/linux/swiotlb.h
@@ -40,7 +40,7 @@ extern void swiotlb_init(int verbose);
int swiotlb_init_with_tbl(char *tlb, unsigned long nslabs, int verbose);
unsigned long swiotlb_size_or_default(void);
extern int swiotlb_late_init_with_tbl(char *tlb, unsigned long nslabs);
-extern int swiotlb_late_init_with_default_size(size_t default_size);
+int swiotlb_init_late(size_t size);
extern void __init swiotlb_update_mem_attributes(void);
phys_addr_t swiotlb_tbl_map_single(struct device *hwdev, phys_addr_t phys,
diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
index 519e363097190..5f64b02fbb732 100644
--- a/kernel/dma/swiotlb.c
+++ b/kernel/dma/swiotlb.c
@@ -290,11 +290,9 @@ swiotlb_init(int verbose)
* initialize the swiotlb later using the slab allocator if needed.
* This should be just like above, but with some error catching.
*/
-int
-swiotlb_late_init_with_default_size(size_t default_size)
+int swiotlb_init_late(size_t size)
{
- unsigned long nslabs =
- ALIGN(default_size >> IO_TLB_SHIFT, IO_TLB_SEGSIZE);
+ unsigned long nslabs = ALIGN(size >> IO_TLB_SHIFT, IO_TLB_SEGSIZE);
unsigned long bytes;
unsigned char *vstart = NULL;
unsigned int order;
--
2.30.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 118+ messages in thread
* [PATCH 04/11] swiotlb: rename swiotlb_late_init_with_default_size
@ 2022-02-27 14:30 ` Christoph Hellwig
0 siblings, 0 replies; 118+ messages in thread
From: Christoph Hellwig @ 2022-02-27 14:30 UTC (permalink / raw)
To: iommu
Cc: x86, Anshuman Khandual, Konrad Rzeszutek Wilk,
Stefano Stabellini, Boris Ostrovsky, Juergen Gross, Joerg Roedel,
David Woodhouse, Lu Baolu, Robin Murphy, linux-arm-kernel,
xen-devel, linux-ia64, linux-mips, linuxppc-dev, linux-riscv,
linux-s390, linux-hyperv, tboot-devel, linux-pci
swiotlb_late_init_with_default_size is an overly verbose name that
doesn't even catch what the function is doing, given that the size is
not just a default but the actual requested size.
Rename it to swiotlb_init_late.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Anshuman Khandual <anshuman.khandual@arm.com>
---
arch/x86/pci/sta2x11-fixup.c | 2 +-
include/linux/swiotlb.h | 2 +-
kernel/dma/swiotlb.c | 6 ++----
3 files changed, 4 insertions(+), 6 deletions(-)
diff --git a/arch/x86/pci/sta2x11-fixup.c b/arch/x86/pci/sta2x11-fixup.c
index 101081ad64b6d..e0c039a75b2db 100644
--- a/arch/x86/pci/sta2x11-fixup.c
+++ b/arch/x86/pci/sta2x11-fixup.c
@@ -57,7 +57,7 @@ static void sta2x11_new_instance(struct pci_dev *pdev)
int size = STA2X11_SWIOTLB_SIZE;
/* First instance: register your own swiotlb area */
dev_info(&pdev->dev, "Using SWIOTLB (size %i)\n", size);
- if (swiotlb_late_init_with_default_size(size))
+ if (swiotlb_init_late(size))
dev_emerg(&pdev->dev, "init swiotlb failed\n");
}
list_add(&instance->list, &sta2x11_instance_list);
diff --git a/include/linux/swiotlb.h b/include/linux/swiotlb.h
index 9fb3a568f0c51..b48b26bfa0edb 100644
--- a/include/linux/swiotlb.h
+++ b/include/linux/swiotlb.h
@@ -40,7 +40,7 @@ extern void swiotlb_init(int verbose);
int swiotlb_init_with_tbl(char *tlb, unsigned long nslabs, int verbose);
unsigned long swiotlb_size_or_default(void);
extern int swiotlb_late_init_with_tbl(char *tlb, unsigned long nslabs);
-extern int swiotlb_late_init_with_default_size(size_t default_size);
+int swiotlb_init_late(size_t size);
extern void __init swiotlb_update_mem_attributes(void);
phys_addr_t swiotlb_tbl_map_single(struct device *hwdev, phys_addr_t phys,
diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
index 519e363097190..5f64b02fbb732 100644
--- a/kernel/dma/swiotlb.c
+++ b/kernel/dma/swiotlb.c
@@ -290,11 +290,9 @@ swiotlb_init(int verbose)
* initialize the swiotlb later using the slab allocator if needed.
* This should be just like above, but with some error catching.
*/
-int
-swiotlb_late_init_with_default_size(size_t default_size)
+int swiotlb_init_late(size_t size)
{
- unsigned long nslabs - ALIGN(default_size >> IO_TLB_SHIFT, IO_TLB_SEGSIZE);
+ unsigned long nslabs = ALIGN(size >> IO_TLB_SHIFT, IO_TLB_SEGSIZE);
unsigned long bytes;
unsigned char *vstart = NULL;
unsigned int order;
--
2.30.2
^ permalink raw reply related [flat|nested] 118+ messages in thread
* [PATCH 05/11] swiotlb: pass a gfp_mask argument to swiotlb_init_late
2022-02-27 14:30 ` Christoph Hellwig
` (3 preceding siblings ...)
(?)
@ 2022-02-27 14:30 ` Christoph Hellwig
-1 siblings, 0 replies; 118+ messages in thread
From: Christoph Hellwig @ 2022-02-27 14:30 UTC (permalink / raw)
To: iommu
Cc: x86, Anshuman Khandual, Konrad Rzeszutek Wilk,
Stefano Stabellini, Boris Ostrovsky, Juergen Gross, Joerg Roedel,
David Woodhouse, Lu Baolu, Robin Murphy, linux-arm-kernel,
xen-devel, linux-ia64, linux-mips, linuxppc-dev, linux-riscv,
linux-s390, linux-hyperv, tboot-devel, linux-pci
Let the caller chose a zone to allocate from. This will be used
later on by the xen-swiotlb initialization on arm.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Anshuman Khandual <anshuman.khandual@arm.com>
---
arch/x86/pci/sta2x11-fixup.c | 2 +-
include/linux/swiotlb.h | 2 +-
kernel/dma/swiotlb.c | 4 ++--
3 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/arch/x86/pci/sta2x11-fixup.c b/arch/x86/pci/sta2x11-fixup.c
index e0c039a75b2db..c7e6faf59a861 100644
--- a/arch/x86/pci/sta2x11-fixup.c
+++ b/arch/x86/pci/sta2x11-fixup.c
@@ -57,7 +57,7 @@ static void sta2x11_new_instance(struct pci_dev *pdev)
int size = STA2X11_SWIOTLB_SIZE;
/* First instance: register your own swiotlb area */
dev_info(&pdev->dev, "Using SWIOTLB (size %i)\n", size);
- if (swiotlb_init_late(size))
+ if (swiotlb_init_late(size, GFP_DMA))
dev_emerg(&pdev->dev, "init swiotlb failed\n");
}
list_add(&instance->list, &sta2x11_instance_list);
diff --git a/include/linux/swiotlb.h b/include/linux/swiotlb.h
index b48b26bfa0edb..1befd6b2ccf5e 100644
--- a/include/linux/swiotlb.h
+++ b/include/linux/swiotlb.h
@@ -40,7 +40,7 @@ extern void swiotlb_init(int verbose);
int swiotlb_init_with_tbl(char *tlb, unsigned long nslabs, int verbose);
unsigned long swiotlb_size_or_default(void);
extern int swiotlb_late_init_with_tbl(char *tlb, unsigned long nslabs);
-int swiotlb_init_late(size_t size);
+int swiotlb_init_late(size_t size, gfp_t gfp_mask);
extern void __init swiotlb_update_mem_attributes(void);
phys_addr_t swiotlb_tbl_map_single(struct device *hwdev, phys_addr_t phys,
diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
index 5f64b02fbb732..a653fcf1fe6c2 100644
--- a/kernel/dma/swiotlb.c
+++ b/kernel/dma/swiotlb.c
@@ -290,7 +290,7 @@ swiotlb_init(int verbose)
* initialize the swiotlb later using the slab allocator if needed.
* This should be just like above, but with some error catching.
*/
-int swiotlb_init_late(size_t size)
+int swiotlb_init_late(size_t size, gfp_t gfp_mask)
{
unsigned long nslabs = ALIGN(size >> IO_TLB_SHIFT, IO_TLB_SEGSIZE);
unsigned long bytes;
@@ -309,7 +309,7 @@ int swiotlb_init_late(size_t size)
bytes = nslabs << IO_TLB_SHIFT;
while ((SLABS_PER_PAGE << order) > IO_TLB_MIN_SLABS) {
- vstart = (void *)__get_free_pages(GFP_DMA | __GFP_NOWARN,
+ vstart = (void *)__get_free_pages(gfp_mask | __GFP_NOWARN,
order);
if (vstart)
break;
--
2.30.2
^ permalink raw reply related [flat|nested] 118+ messages in thread
* [PATCH 05/11] swiotlb: pass a gfp_mask argument to swiotlb_init_late
@ 2022-02-27 14:30 ` Christoph Hellwig
0 siblings, 0 replies; 118+ messages in thread
From: Christoph Hellwig @ 2022-02-27 14:30 UTC (permalink / raw)
To: iommu
Cc: Juergen Gross, linux-s390, linux-hyperv, Stefano Stabellini,
linux-ia64, Anshuman Khandual, Konrad Rzeszutek Wilk,
Robin Murphy, x86, linux-mips, linuxppc-dev, tboot-devel,
linux-pci, xen-devel, Boris Ostrovsky, David Woodhouse,
linux-riscv, linux-arm-kernel
Let the caller chose a zone to allocate from. This will be used
later on by the xen-swiotlb initialization on arm.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Anshuman Khandual <anshuman.khandual@arm.com>
---
arch/x86/pci/sta2x11-fixup.c | 2 +-
include/linux/swiotlb.h | 2 +-
kernel/dma/swiotlb.c | 4 ++--
3 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/arch/x86/pci/sta2x11-fixup.c b/arch/x86/pci/sta2x11-fixup.c
index e0c039a75b2db..c7e6faf59a861 100644
--- a/arch/x86/pci/sta2x11-fixup.c
+++ b/arch/x86/pci/sta2x11-fixup.c
@@ -57,7 +57,7 @@ static void sta2x11_new_instance(struct pci_dev *pdev)
int size = STA2X11_SWIOTLB_SIZE;
/* First instance: register your own swiotlb area */
dev_info(&pdev->dev, "Using SWIOTLB (size %i)\n", size);
- if (swiotlb_init_late(size))
+ if (swiotlb_init_late(size, GFP_DMA))
dev_emerg(&pdev->dev, "init swiotlb failed\n");
}
list_add(&instance->list, &sta2x11_instance_list);
diff --git a/include/linux/swiotlb.h b/include/linux/swiotlb.h
index b48b26bfa0edb..1befd6b2ccf5e 100644
--- a/include/linux/swiotlb.h
+++ b/include/linux/swiotlb.h
@@ -40,7 +40,7 @@ extern void swiotlb_init(int verbose);
int swiotlb_init_with_tbl(char *tlb, unsigned long nslabs, int verbose);
unsigned long swiotlb_size_or_default(void);
extern int swiotlb_late_init_with_tbl(char *tlb, unsigned long nslabs);
-int swiotlb_init_late(size_t size);
+int swiotlb_init_late(size_t size, gfp_t gfp_mask);
extern void __init swiotlb_update_mem_attributes(void);
phys_addr_t swiotlb_tbl_map_single(struct device *hwdev, phys_addr_t phys,
diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
index 5f64b02fbb732..a653fcf1fe6c2 100644
--- a/kernel/dma/swiotlb.c
+++ b/kernel/dma/swiotlb.c
@@ -290,7 +290,7 @@ swiotlb_init(int verbose)
* initialize the swiotlb later using the slab allocator if needed.
* This should be just like above, but with some error catching.
*/
-int swiotlb_init_late(size_t size)
+int swiotlb_init_late(size_t size, gfp_t gfp_mask)
{
unsigned long nslabs = ALIGN(size >> IO_TLB_SHIFT, IO_TLB_SEGSIZE);
unsigned long bytes;
@@ -309,7 +309,7 @@ int swiotlb_init_late(size_t size)
bytes = nslabs << IO_TLB_SHIFT;
while ((SLABS_PER_PAGE << order) > IO_TLB_MIN_SLABS) {
- vstart = (void *)__get_free_pages(GFP_DMA | __GFP_NOWARN,
+ vstart = (void *)__get_free_pages(gfp_mask | __GFP_NOWARN,
order);
if (vstart)
break;
--
2.30.2
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply related [flat|nested] 118+ messages in thread
* [PATCH 05/11] swiotlb: pass a gfp_mask argument to swiotlb_init_late
@ 2022-02-27 14:30 ` Christoph Hellwig
0 siblings, 0 replies; 118+ messages in thread
From: Christoph Hellwig @ 2022-02-27 14:30 UTC (permalink / raw)
To: iommu
Cc: x86, Anshuman Khandual, Konrad Rzeszutek Wilk,
Stefano Stabellini, Boris Ostrovsky, Juergen Gross, Joerg Roedel,
David Woodhouse, Lu Baolu, Robin Murphy, linux-arm-kernel,
xen-devel, linux-ia64, linux-mips, linuxppc-dev, linux-riscv,
linux-s390, linux-hyperv, tboot-devel, linux-pci
Let the caller chose a zone to allocate from. This will be used
later on by the xen-swiotlb initialization on arm.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Anshuman Khandual <anshuman.khandual@arm.com>
---
arch/x86/pci/sta2x11-fixup.c | 2 +-
include/linux/swiotlb.h | 2 +-
kernel/dma/swiotlb.c | 4 ++--
3 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/arch/x86/pci/sta2x11-fixup.c b/arch/x86/pci/sta2x11-fixup.c
index e0c039a75b2db..c7e6faf59a861 100644
--- a/arch/x86/pci/sta2x11-fixup.c
+++ b/arch/x86/pci/sta2x11-fixup.c
@@ -57,7 +57,7 @@ static void sta2x11_new_instance(struct pci_dev *pdev)
int size = STA2X11_SWIOTLB_SIZE;
/* First instance: register your own swiotlb area */
dev_info(&pdev->dev, "Using SWIOTLB (size %i)\n", size);
- if (swiotlb_init_late(size))
+ if (swiotlb_init_late(size, GFP_DMA))
dev_emerg(&pdev->dev, "init swiotlb failed\n");
}
list_add(&instance->list, &sta2x11_instance_list);
diff --git a/include/linux/swiotlb.h b/include/linux/swiotlb.h
index b48b26bfa0edb..1befd6b2ccf5e 100644
--- a/include/linux/swiotlb.h
+++ b/include/linux/swiotlb.h
@@ -40,7 +40,7 @@ extern void swiotlb_init(int verbose);
int swiotlb_init_with_tbl(char *tlb, unsigned long nslabs, int verbose);
unsigned long swiotlb_size_or_default(void);
extern int swiotlb_late_init_with_tbl(char *tlb, unsigned long nslabs);
-int swiotlb_init_late(size_t size);
+int swiotlb_init_late(size_t size, gfp_t gfp_mask);
extern void __init swiotlb_update_mem_attributes(void);
phys_addr_t swiotlb_tbl_map_single(struct device *hwdev, phys_addr_t phys,
diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
index 5f64b02fbb732..a653fcf1fe6c2 100644
--- a/kernel/dma/swiotlb.c
+++ b/kernel/dma/swiotlb.c
@@ -290,7 +290,7 @@ swiotlb_init(int verbose)
* initialize the swiotlb later using the slab allocator if needed.
* This should be just like above, but with some error catching.
*/
-int swiotlb_init_late(size_t size)
+int swiotlb_init_late(size_t size, gfp_t gfp_mask)
{
unsigned long nslabs = ALIGN(size >> IO_TLB_SHIFT, IO_TLB_SEGSIZE);
unsigned long bytes;
@@ -309,7 +309,7 @@ int swiotlb_init_late(size_t size)
bytes = nslabs << IO_TLB_SHIFT;
while ((SLABS_PER_PAGE << order) > IO_TLB_MIN_SLABS) {
- vstart = (void *)__get_free_pages(GFP_DMA | __GFP_NOWARN,
+ vstart = (void *)__get_free_pages(gfp_mask | __GFP_NOWARN,
order);
if (vstart)
break;
--
2.30.2
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply related [flat|nested] 118+ messages in thread
* [PATCH 05/11] swiotlb: pass a gfp_mask argument to swiotlb_init_late
@ 2022-02-27 14:30 ` Christoph Hellwig
0 siblings, 0 replies; 118+ messages in thread
From: Christoph Hellwig @ 2022-02-27 14:30 UTC (permalink / raw)
To: iommu
Cc: Juergen Gross, linux-s390, linux-hyperv, Stefano Stabellini,
linux-ia64, Anshuman Khandual, Konrad Rzeszutek Wilk,
Robin Murphy, Joerg Roedel, x86, linux-mips, linuxppc-dev,
tboot-devel, linux-pci, xen-devel, Boris Ostrovsky,
David Woodhouse, linux-riscv, linux-arm-kernel, Lu Baolu
Let the caller chose a zone to allocate from. This will be used
later on by the xen-swiotlb initialization on arm.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Anshuman Khandual <anshuman.khandual@arm.com>
---
arch/x86/pci/sta2x11-fixup.c | 2 +-
include/linux/swiotlb.h | 2 +-
kernel/dma/swiotlb.c | 4 ++--
3 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/arch/x86/pci/sta2x11-fixup.c b/arch/x86/pci/sta2x11-fixup.c
index e0c039a75b2db..c7e6faf59a861 100644
--- a/arch/x86/pci/sta2x11-fixup.c
+++ b/arch/x86/pci/sta2x11-fixup.c
@@ -57,7 +57,7 @@ static void sta2x11_new_instance(struct pci_dev *pdev)
int size = STA2X11_SWIOTLB_SIZE;
/* First instance: register your own swiotlb area */
dev_info(&pdev->dev, "Using SWIOTLB (size %i)\n", size);
- if (swiotlb_init_late(size))
+ if (swiotlb_init_late(size, GFP_DMA))
dev_emerg(&pdev->dev, "init swiotlb failed\n");
}
list_add(&instance->list, &sta2x11_instance_list);
diff --git a/include/linux/swiotlb.h b/include/linux/swiotlb.h
index b48b26bfa0edb..1befd6b2ccf5e 100644
--- a/include/linux/swiotlb.h
+++ b/include/linux/swiotlb.h
@@ -40,7 +40,7 @@ extern void swiotlb_init(int verbose);
int swiotlb_init_with_tbl(char *tlb, unsigned long nslabs, int verbose);
unsigned long swiotlb_size_or_default(void);
extern int swiotlb_late_init_with_tbl(char *tlb, unsigned long nslabs);
-int swiotlb_init_late(size_t size);
+int swiotlb_init_late(size_t size, gfp_t gfp_mask);
extern void __init swiotlb_update_mem_attributes(void);
phys_addr_t swiotlb_tbl_map_single(struct device *hwdev, phys_addr_t phys,
diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
index 5f64b02fbb732..a653fcf1fe6c2 100644
--- a/kernel/dma/swiotlb.c
+++ b/kernel/dma/swiotlb.c
@@ -290,7 +290,7 @@ swiotlb_init(int verbose)
* initialize the swiotlb later using the slab allocator if needed.
* This should be just like above, but with some error catching.
*/
-int swiotlb_init_late(size_t size)
+int swiotlb_init_late(size_t size, gfp_t gfp_mask)
{
unsigned long nslabs = ALIGN(size >> IO_TLB_SHIFT, IO_TLB_SEGSIZE);
unsigned long bytes;
@@ -309,7 +309,7 @@ int swiotlb_init_late(size_t size)
bytes = nslabs << IO_TLB_SHIFT;
while ((SLABS_PER_PAGE << order) > IO_TLB_MIN_SLABS) {
- vstart = (void *)__get_free_pages(GFP_DMA | __GFP_NOWARN,
+ vstart = (void *)__get_free_pages(gfp_mask | __GFP_NOWARN,
order);
if (vstart)
break;
--
2.30.2
^ permalink raw reply related [flat|nested] 118+ messages in thread
* [PATCH 05/11] swiotlb: pass a gfp_mask argument to swiotlb_init_late
@ 2022-02-27 14:30 ` Christoph Hellwig
0 siblings, 0 replies; 118+ messages in thread
From: Christoph Hellwig @ 2022-02-27 14:30 UTC (permalink / raw)
To: iommu
Cc: x86, Anshuman Khandual, Konrad Rzeszutek Wilk,
Stefano Stabellini, Boris Ostrovsky, Juergen Gross, Joerg Roedel,
David Woodhouse, Lu Baolu, Robin Murphy, linux-arm-kernel,
xen-devel, linux-ia64, linux-mips, linuxppc-dev, linux-riscv,
linux-s390, linux-hyperv, tboot-devel, linux-pci
Let the caller chose a zone to allocate from. This will be used
later on by the xen-swiotlb initialization on arm.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Anshuman Khandual <anshuman.khandual@arm.com>
---
arch/x86/pci/sta2x11-fixup.c | 2 +-
include/linux/swiotlb.h | 2 +-
kernel/dma/swiotlb.c | 4 ++--
3 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/arch/x86/pci/sta2x11-fixup.c b/arch/x86/pci/sta2x11-fixup.c
index e0c039a75b2db..c7e6faf59a861 100644
--- a/arch/x86/pci/sta2x11-fixup.c
+++ b/arch/x86/pci/sta2x11-fixup.c
@@ -57,7 +57,7 @@ static void sta2x11_new_instance(struct pci_dev *pdev)
int size = STA2X11_SWIOTLB_SIZE;
/* First instance: register your own swiotlb area */
dev_info(&pdev->dev, "Using SWIOTLB (size %i)\n", size);
- if (swiotlb_init_late(size))
+ if (swiotlb_init_late(size, GFP_DMA))
dev_emerg(&pdev->dev, "init swiotlb failed\n");
}
list_add(&instance->list, &sta2x11_instance_list);
diff --git a/include/linux/swiotlb.h b/include/linux/swiotlb.h
index b48b26bfa0edb..1befd6b2ccf5e 100644
--- a/include/linux/swiotlb.h
+++ b/include/linux/swiotlb.h
@@ -40,7 +40,7 @@ extern void swiotlb_init(int verbose);
int swiotlb_init_with_tbl(char *tlb, unsigned long nslabs, int verbose);
unsigned long swiotlb_size_or_default(void);
extern int swiotlb_late_init_with_tbl(char *tlb, unsigned long nslabs);
-int swiotlb_init_late(size_t size);
+int swiotlb_init_late(size_t size, gfp_t gfp_mask);
extern void __init swiotlb_update_mem_attributes(void);
phys_addr_t swiotlb_tbl_map_single(struct device *hwdev, phys_addr_t phys,
diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
index 5f64b02fbb732..a653fcf1fe6c2 100644
--- a/kernel/dma/swiotlb.c
+++ b/kernel/dma/swiotlb.c
@@ -290,7 +290,7 @@ swiotlb_init(int verbose)
* initialize the swiotlb later using the slab allocator if needed.
* This should be just like above, but with some error catching.
*/
-int swiotlb_init_late(size_t size)
+int swiotlb_init_late(size_t size, gfp_t gfp_mask)
{
unsigned long nslabs = ALIGN(size >> IO_TLB_SHIFT, IO_TLB_SEGSIZE);
unsigned long bytes;
@@ -309,7 +309,7 @@ int swiotlb_init_late(size_t size)
bytes = nslabs << IO_TLB_SHIFT;
while ((SLABS_PER_PAGE << order) > IO_TLB_MIN_SLABS) {
- vstart = (void *)__get_free_pages(GFP_DMA | __GFP_NOWARN,
+ vstart = (void *)__get_free_pages(gfp_mask | __GFP_NOWARN,
order);
if (vstart)
break;
--
2.30.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 118+ messages in thread
* [PATCH 05/11] swiotlb: pass a gfp_mask argument to swiotlb_init_late
@ 2022-02-27 14:30 ` Christoph Hellwig
0 siblings, 0 replies; 118+ messages in thread
From: Christoph Hellwig @ 2022-02-27 14:30 UTC (permalink / raw)
To: iommu
Cc: x86, Anshuman Khandual, Konrad Rzeszutek Wilk,
Stefano Stabellini, Boris Ostrovsky, Juergen Gross, Joerg Roedel,
David Woodhouse, Lu Baolu, Robin Murphy, linux-arm-kernel,
xen-devel, linux-ia64, linux-mips, linuxppc-dev, linux-riscv,
linux-s390, linux-hyperv, tboot-devel, linux-pci
Let the caller chose a zone to allocate from. This will be used
later on by the xen-swiotlb initialization on arm.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Anshuman Khandual <anshuman.khandual@arm.com>
---
arch/x86/pci/sta2x11-fixup.c | 2 +-
include/linux/swiotlb.h | 2 +-
kernel/dma/swiotlb.c | 4 ++--
3 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/arch/x86/pci/sta2x11-fixup.c b/arch/x86/pci/sta2x11-fixup.c
index e0c039a75b2db..c7e6faf59a861 100644
--- a/arch/x86/pci/sta2x11-fixup.c
+++ b/arch/x86/pci/sta2x11-fixup.c
@@ -57,7 +57,7 @@ static void sta2x11_new_instance(struct pci_dev *pdev)
int size = STA2X11_SWIOTLB_SIZE;
/* First instance: register your own swiotlb area */
dev_info(&pdev->dev, "Using SWIOTLB (size %i)\n", size);
- if (swiotlb_init_late(size))
+ if (swiotlb_init_late(size, GFP_DMA))
dev_emerg(&pdev->dev, "init swiotlb failed\n");
}
list_add(&instance->list, &sta2x11_instance_list);
diff --git a/include/linux/swiotlb.h b/include/linux/swiotlb.h
index b48b26bfa0edb..1befd6b2ccf5e 100644
--- a/include/linux/swiotlb.h
+++ b/include/linux/swiotlb.h
@@ -40,7 +40,7 @@ extern void swiotlb_init(int verbose);
int swiotlb_init_with_tbl(char *tlb, unsigned long nslabs, int verbose);
unsigned long swiotlb_size_or_default(void);
extern int swiotlb_late_init_with_tbl(char *tlb, unsigned long nslabs);
-int swiotlb_init_late(size_t size);
+int swiotlb_init_late(size_t size, gfp_t gfp_mask);
extern void __init swiotlb_update_mem_attributes(void);
phys_addr_t swiotlb_tbl_map_single(struct device *hwdev, phys_addr_t phys,
diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
index 5f64b02fbb732..a653fcf1fe6c2 100644
--- a/kernel/dma/swiotlb.c
+++ b/kernel/dma/swiotlb.c
@@ -290,7 +290,7 @@ swiotlb_init(int verbose)
* initialize the swiotlb later using the slab allocator if needed.
* This should be just like above, but with some error catching.
*/
-int swiotlb_init_late(size_t size)
+int swiotlb_init_late(size_t size, gfp_t gfp_mask)
{
unsigned long nslabs = ALIGN(size >> IO_TLB_SHIFT, IO_TLB_SEGSIZE);
unsigned long bytes;
@@ -309,7 +309,7 @@ int swiotlb_init_late(size_t size)
bytes = nslabs << IO_TLB_SHIFT;
while ((SLABS_PER_PAGE << order) > IO_TLB_MIN_SLABS) {
- vstart = (void *)__get_free_pages(GFP_DMA | __GFP_NOWARN,
+ vstart = (void *)__get_free_pages(gfp_mask | __GFP_NOWARN,
order);
if (vstart)
break;
--
2.30.2
^ permalink raw reply related [flat|nested] 118+ messages in thread
* [PATCH 06/11] MIPS/octeon: use swiotlb_init instead of open coding it
2022-02-27 14:30 ` Christoph Hellwig
` (3 preceding siblings ...)
(?)
@ 2022-02-27 14:30 ` Christoph Hellwig
-1 siblings, 0 replies; 118+ messages in thread
From: Christoph Hellwig @ 2022-02-27 14:30 UTC (permalink / raw)
To: iommu
Cc: x86, Anshuman Khandual, Konrad Rzeszutek Wilk,
Stefano Stabellini, Boris Ostrovsky, Juergen Gross, Joerg Roedel,
David Woodhouse, Lu Baolu, Robin Murphy, linux-arm-kernel,
xen-devel, linux-ia64, linux-mips, linuxppc-dev, linux-riscv,
linux-s390, linux-hyperv, tboot-devel, linux-pci
Use the generic swiotlb initialization helper instead of open coding it.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
arch/mips/cavium-octeon/dma-octeon.c | 15 ++-------------
arch/mips/pci/pci-octeon.c | 2 +-
2 files changed, 3 insertions(+), 14 deletions(-)
diff --git a/arch/mips/cavium-octeon/dma-octeon.c b/arch/mips/cavium-octeon/dma-octeon.c
index df70308db0e69..fb7547e217263 100644
--- a/arch/mips/cavium-octeon/dma-octeon.c
+++ b/arch/mips/cavium-octeon/dma-octeon.c
@@ -186,15 +186,12 @@ phys_addr_t dma_to_phys(struct device *dev, dma_addr_t daddr)
return daddr;
}
-char *octeon_swiotlb;
-
void __init plat_swiotlb_setup(void)
{
phys_addr_t start, end;
phys_addr_t max_addr;
phys_addr_t addr_size;
size_t swiotlbsize;
- unsigned long swiotlb_nslabs;
u64 i;
max_addr = 0;
@@ -236,15 +233,7 @@ void __init plat_swiotlb_setup(void)
if (OCTEON_IS_OCTEON2() && max_addr >= 0x100000000ul)
swiotlbsize = 64 * (1<<20);
#endif
- swiotlb_nslabs = swiotlbsize >> IO_TLB_SHIFT;
- swiotlb_nslabs = ALIGN(swiotlb_nslabs, IO_TLB_SEGSIZE);
- swiotlbsize = swiotlb_nslabs << IO_TLB_SHIFT;
-
- octeon_swiotlb = memblock_alloc_low(swiotlbsize, PAGE_SIZE);
- if (!octeon_swiotlb)
- panic("%s: Failed to allocate %zu bytes align=%lx\n",
- __func__, swiotlbsize, PAGE_SIZE);
- if (swiotlb_init_with_tbl(octeon_swiotlb, swiotlb_nslabs, 1) == -ENOMEM)
- panic("Cannot allocate SWIOTLB buffer");
+ swiotlb_adjust_size(swiotlbsize);
+ swiotlb_init(1);
}
diff --git a/arch/mips/pci/pci-octeon.c b/arch/mips/pci/pci-octeon.c
index fc29b85cfa926..e457a18cbdc59 100644
--- a/arch/mips/pci/pci-octeon.c
+++ b/arch/mips/pci/pci-octeon.c
@@ -664,7 +664,7 @@ static int __init octeon_pci_setup(void)
/* BAR1 movable regions contiguous to cover the swiotlb */
octeon_bar1_pci_phys =
- virt_to_phys(octeon_swiotlb) & ~((1ull << 22) - 1);
+ io_tlb_default_mem.start & ~((1ull << 22) - 1);
for (index = 0; index < 32; index++) {
union cvmx_pci_bar1_indexx bar1_index;
--
2.30.2
^ permalink raw reply related [flat|nested] 118+ messages in thread
* [PATCH 06/11] MIPS/octeon: use swiotlb_init instead of open coding it
@ 2022-02-27 14:30 ` Christoph Hellwig
0 siblings, 0 replies; 118+ messages in thread
From: Christoph Hellwig @ 2022-02-27 14:30 UTC (permalink / raw)
To: iommu
Cc: Juergen Gross, linux-s390, linux-hyperv, Stefano Stabellini,
linux-ia64, Anshuman Khandual, Konrad Rzeszutek Wilk,
Robin Murphy, x86, linux-mips, linuxppc-dev, tboot-devel,
linux-pci, xen-devel, Boris Ostrovsky, David Woodhouse,
linux-riscv, linux-arm-kernel
Use the generic swiotlb initialization helper instead of open coding it.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
arch/mips/cavium-octeon/dma-octeon.c | 15 ++-------------
arch/mips/pci/pci-octeon.c | 2 +-
2 files changed, 3 insertions(+), 14 deletions(-)
diff --git a/arch/mips/cavium-octeon/dma-octeon.c b/arch/mips/cavium-octeon/dma-octeon.c
index df70308db0e69..fb7547e217263 100644
--- a/arch/mips/cavium-octeon/dma-octeon.c
+++ b/arch/mips/cavium-octeon/dma-octeon.c
@@ -186,15 +186,12 @@ phys_addr_t dma_to_phys(struct device *dev, dma_addr_t daddr)
return daddr;
}
-char *octeon_swiotlb;
-
void __init plat_swiotlb_setup(void)
{
phys_addr_t start, end;
phys_addr_t max_addr;
phys_addr_t addr_size;
size_t swiotlbsize;
- unsigned long swiotlb_nslabs;
u64 i;
max_addr = 0;
@@ -236,15 +233,7 @@ void __init plat_swiotlb_setup(void)
if (OCTEON_IS_OCTEON2() && max_addr >= 0x100000000ul)
swiotlbsize = 64 * (1<<20);
#endif
- swiotlb_nslabs = swiotlbsize >> IO_TLB_SHIFT;
- swiotlb_nslabs = ALIGN(swiotlb_nslabs, IO_TLB_SEGSIZE);
- swiotlbsize = swiotlb_nslabs << IO_TLB_SHIFT;
-
- octeon_swiotlb = memblock_alloc_low(swiotlbsize, PAGE_SIZE);
- if (!octeon_swiotlb)
- panic("%s: Failed to allocate %zu bytes align=%lx\n",
- __func__, swiotlbsize, PAGE_SIZE);
- if (swiotlb_init_with_tbl(octeon_swiotlb, swiotlb_nslabs, 1) == -ENOMEM)
- panic("Cannot allocate SWIOTLB buffer");
+ swiotlb_adjust_size(swiotlbsize);
+ swiotlb_init(1);
}
diff --git a/arch/mips/pci/pci-octeon.c b/arch/mips/pci/pci-octeon.c
index fc29b85cfa926..e457a18cbdc59 100644
--- a/arch/mips/pci/pci-octeon.c
+++ b/arch/mips/pci/pci-octeon.c
@@ -664,7 +664,7 @@ static int __init octeon_pci_setup(void)
/* BAR1 movable regions contiguous to cover the swiotlb */
octeon_bar1_pci_phys =
- virt_to_phys(octeon_swiotlb) & ~((1ull << 22) - 1);
+ io_tlb_default_mem.start & ~((1ull << 22) - 1);
for (index = 0; index < 32; index++) {
union cvmx_pci_bar1_indexx bar1_index;
--
2.30.2
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply related [flat|nested] 118+ messages in thread
* [PATCH 06/11] MIPS/octeon: use swiotlb_init instead of open coding it
@ 2022-02-27 14:30 ` Christoph Hellwig
0 siblings, 0 replies; 118+ messages in thread
From: Christoph Hellwig @ 2022-02-27 14:30 UTC (permalink / raw)
To: iommu
Cc: x86, Anshuman Khandual, Konrad Rzeszutek Wilk,
Stefano Stabellini, Boris Ostrovsky, Juergen Gross, Joerg Roedel,
David Woodhouse, Lu Baolu, Robin Murphy, linux-arm-kernel,
xen-devel, linux-ia64, linux-mips, linuxppc-dev, linux-riscv,
linux-s390, linux-hyperv, tboot-devel, linux-pci
Use the generic swiotlb initialization helper instead of open coding it.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
arch/mips/cavium-octeon/dma-octeon.c | 15 ++-------------
arch/mips/pci/pci-octeon.c | 2 +-
2 files changed, 3 insertions(+), 14 deletions(-)
diff --git a/arch/mips/cavium-octeon/dma-octeon.c b/arch/mips/cavium-octeon/dma-octeon.c
index df70308db0e69..fb7547e217263 100644
--- a/arch/mips/cavium-octeon/dma-octeon.c
+++ b/arch/mips/cavium-octeon/dma-octeon.c
@@ -186,15 +186,12 @@ phys_addr_t dma_to_phys(struct device *dev, dma_addr_t daddr)
return daddr;
}
-char *octeon_swiotlb;
-
void __init plat_swiotlb_setup(void)
{
phys_addr_t start, end;
phys_addr_t max_addr;
phys_addr_t addr_size;
size_t swiotlbsize;
- unsigned long swiotlb_nslabs;
u64 i;
max_addr = 0;
@@ -236,15 +233,7 @@ void __init plat_swiotlb_setup(void)
if (OCTEON_IS_OCTEON2() && max_addr >= 0x100000000ul)
swiotlbsize = 64 * (1<<20);
#endif
- swiotlb_nslabs = swiotlbsize >> IO_TLB_SHIFT;
- swiotlb_nslabs = ALIGN(swiotlb_nslabs, IO_TLB_SEGSIZE);
- swiotlbsize = swiotlb_nslabs << IO_TLB_SHIFT;
-
- octeon_swiotlb = memblock_alloc_low(swiotlbsize, PAGE_SIZE);
- if (!octeon_swiotlb)
- panic("%s: Failed to allocate %zu bytes align=%lx\n",
- __func__, swiotlbsize, PAGE_SIZE);
- if (swiotlb_init_with_tbl(octeon_swiotlb, swiotlb_nslabs, 1) == -ENOMEM)
- panic("Cannot allocate SWIOTLB buffer");
+ swiotlb_adjust_size(swiotlbsize);
+ swiotlb_init(1);
}
diff --git a/arch/mips/pci/pci-octeon.c b/arch/mips/pci/pci-octeon.c
index fc29b85cfa926..e457a18cbdc59 100644
--- a/arch/mips/pci/pci-octeon.c
+++ b/arch/mips/pci/pci-octeon.c
@@ -664,7 +664,7 @@ static int __init octeon_pci_setup(void)
/* BAR1 movable regions contiguous to cover the swiotlb */
octeon_bar1_pci_phys =
- virt_to_phys(octeon_swiotlb) & ~((1ull << 22) - 1);
+ io_tlb_default_mem.start & ~((1ull << 22) - 1);
for (index = 0; index < 32; index++) {
union cvmx_pci_bar1_indexx bar1_index;
--
2.30.2
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply related [flat|nested] 118+ messages in thread
* [PATCH 06/11] MIPS/octeon: use swiotlb_init instead of open coding it
@ 2022-02-27 14:30 ` Christoph Hellwig
0 siblings, 0 replies; 118+ messages in thread
From: Christoph Hellwig @ 2022-02-27 14:30 UTC (permalink / raw)
To: iommu
Cc: x86, Anshuman Khandual, Konrad Rzeszutek Wilk,
Stefano Stabellini, Boris Ostrovsky, Juergen Gross, Joerg Roedel,
David Woodhouse, Lu Baolu, Robin Murphy, linux-arm-kernel,
xen-devel, linux-ia64, linux-mips, linuxppc-dev, linux-riscv,
linux-s390, linux-hyperv, tboot-devel, linux-pci
Use the generic swiotlb initialization helper instead of open coding it.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
arch/mips/cavium-octeon/dma-octeon.c | 15 ++-------------
arch/mips/pci/pci-octeon.c | 2 +-
2 files changed, 3 insertions(+), 14 deletions(-)
diff --git a/arch/mips/cavium-octeon/dma-octeon.c b/arch/mips/cavium-octeon/dma-octeon.c
index df70308db0e69..fb7547e217263 100644
--- a/arch/mips/cavium-octeon/dma-octeon.c
+++ b/arch/mips/cavium-octeon/dma-octeon.c
@@ -186,15 +186,12 @@ phys_addr_t dma_to_phys(struct device *dev, dma_addr_t daddr)
return daddr;
}
-char *octeon_swiotlb;
-
void __init plat_swiotlb_setup(void)
{
phys_addr_t start, end;
phys_addr_t max_addr;
phys_addr_t addr_size;
size_t swiotlbsize;
- unsigned long swiotlb_nslabs;
u64 i;
max_addr = 0;
@@ -236,15 +233,7 @@ void __init plat_swiotlb_setup(void)
if (OCTEON_IS_OCTEON2() && max_addr >= 0x100000000ul)
swiotlbsize = 64 * (1<<20);
#endif
- swiotlb_nslabs = swiotlbsize >> IO_TLB_SHIFT;
- swiotlb_nslabs = ALIGN(swiotlb_nslabs, IO_TLB_SEGSIZE);
- swiotlbsize = swiotlb_nslabs << IO_TLB_SHIFT;
-
- octeon_swiotlb = memblock_alloc_low(swiotlbsize, PAGE_SIZE);
- if (!octeon_swiotlb)
- panic("%s: Failed to allocate %zu bytes align=%lx\n",
- __func__, swiotlbsize, PAGE_SIZE);
- if (swiotlb_init_with_tbl(octeon_swiotlb, swiotlb_nslabs, 1) == -ENOMEM)
- panic("Cannot allocate SWIOTLB buffer");
+ swiotlb_adjust_size(swiotlbsize);
+ swiotlb_init(1);
}
diff --git a/arch/mips/pci/pci-octeon.c b/arch/mips/pci/pci-octeon.c
index fc29b85cfa926..e457a18cbdc59 100644
--- a/arch/mips/pci/pci-octeon.c
+++ b/arch/mips/pci/pci-octeon.c
@@ -664,7 +664,7 @@ static int __init octeon_pci_setup(void)
/* BAR1 movable regions contiguous to cover the swiotlb */
octeon_bar1_pci_phys =
- virt_to_phys(octeon_swiotlb) & ~((1ull << 22) - 1);
+ io_tlb_default_mem.start & ~((1ull << 22) - 1);
for (index = 0; index < 32; index++) {
union cvmx_pci_bar1_indexx bar1_index;
--
2.30.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 118+ messages in thread
* [PATCH 06/11] MIPS/octeon: use swiotlb_init instead of open coding it
@ 2022-02-27 14:30 ` Christoph Hellwig
0 siblings, 0 replies; 118+ messages in thread
From: Christoph Hellwig @ 2022-02-27 14:30 UTC (permalink / raw)
To: iommu
Cc: Juergen Gross, linux-s390, linux-hyperv, Stefano Stabellini,
linux-ia64, Anshuman Khandual, Konrad Rzeszutek Wilk,
Robin Murphy, Joerg Roedel, x86, linux-mips, linuxppc-dev,
tboot-devel, linux-pci, xen-devel, Boris Ostrovsky,
David Woodhouse, linux-riscv, linux-arm-kernel, Lu Baolu
Use the generic swiotlb initialization helper instead of open coding it.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
arch/mips/cavium-octeon/dma-octeon.c | 15 ++-------------
arch/mips/pci/pci-octeon.c | 2 +-
2 files changed, 3 insertions(+), 14 deletions(-)
diff --git a/arch/mips/cavium-octeon/dma-octeon.c b/arch/mips/cavium-octeon/dma-octeon.c
index df70308db0e69..fb7547e217263 100644
--- a/arch/mips/cavium-octeon/dma-octeon.c
+++ b/arch/mips/cavium-octeon/dma-octeon.c
@@ -186,15 +186,12 @@ phys_addr_t dma_to_phys(struct device *dev, dma_addr_t daddr)
return daddr;
}
-char *octeon_swiotlb;
-
void __init plat_swiotlb_setup(void)
{
phys_addr_t start, end;
phys_addr_t max_addr;
phys_addr_t addr_size;
size_t swiotlbsize;
- unsigned long swiotlb_nslabs;
u64 i;
max_addr = 0;
@@ -236,15 +233,7 @@ void __init plat_swiotlb_setup(void)
if (OCTEON_IS_OCTEON2() && max_addr >= 0x100000000ul)
swiotlbsize = 64 * (1<<20);
#endif
- swiotlb_nslabs = swiotlbsize >> IO_TLB_SHIFT;
- swiotlb_nslabs = ALIGN(swiotlb_nslabs, IO_TLB_SEGSIZE);
- swiotlbsize = swiotlb_nslabs << IO_TLB_SHIFT;
-
- octeon_swiotlb = memblock_alloc_low(swiotlbsize, PAGE_SIZE);
- if (!octeon_swiotlb)
- panic("%s: Failed to allocate %zu bytes align=%lx\n",
- __func__, swiotlbsize, PAGE_SIZE);
- if (swiotlb_init_with_tbl(octeon_swiotlb, swiotlb_nslabs, 1) == -ENOMEM)
- panic("Cannot allocate SWIOTLB buffer");
+ swiotlb_adjust_size(swiotlbsize);
+ swiotlb_init(1);
}
diff --git a/arch/mips/pci/pci-octeon.c b/arch/mips/pci/pci-octeon.c
index fc29b85cfa926..e457a18cbdc59 100644
--- a/arch/mips/pci/pci-octeon.c
+++ b/arch/mips/pci/pci-octeon.c
@@ -664,7 +664,7 @@ static int __init octeon_pci_setup(void)
/* BAR1 movable regions contiguous to cover the swiotlb */
octeon_bar1_pci_phys =
- virt_to_phys(octeon_swiotlb) & ~((1ull << 22) - 1);
+ io_tlb_default_mem.start & ~((1ull << 22) - 1);
for (index = 0; index < 32; index++) {
union cvmx_pci_bar1_indexx bar1_index;
--
2.30.2
^ permalink raw reply related [flat|nested] 118+ messages in thread
* [PATCH 06/11] MIPS/octeon: use swiotlb_init instead of open coding it
@ 2022-02-27 14:30 ` Christoph Hellwig
0 siblings, 0 replies; 118+ messages in thread
From: Christoph Hellwig @ 2022-02-27 14:30 UTC (permalink / raw)
To: iommu
Cc: x86, Anshuman Khandual, Konrad Rzeszutek Wilk,
Stefano Stabellini, Boris Ostrovsky, Juergen Gross, Joerg Roedel,
David Woodhouse, Lu Baolu, Robin Murphy, linux-arm-kernel,
xen-devel, linux-ia64, linux-mips, linuxppc-dev, linux-riscv,
linux-s390, linux-hyperv, tboot-devel, linux-pci
Use the generic swiotlb initialization helper instead of open coding it.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
arch/mips/cavium-octeon/dma-octeon.c | 15 ++-------------
arch/mips/pci/pci-octeon.c | 2 +-
2 files changed, 3 insertions(+), 14 deletions(-)
diff --git a/arch/mips/cavium-octeon/dma-octeon.c b/arch/mips/cavium-octeon/dma-octeon.c
index df70308db0e69..fb7547e217263 100644
--- a/arch/mips/cavium-octeon/dma-octeon.c
+++ b/arch/mips/cavium-octeon/dma-octeon.c
@@ -186,15 +186,12 @@ phys_addr_t dma_to_phys(struct device *dev, dma_addr_t daddr)
return daddr;
}
-char *octeon_swiotlb;
-
void __init plat_swiotlb_setup(void)
{
phys_addr_t start, end;
phys_addr_t max_addr;
phys_addr_t addr_size;
size_t swiotlbsize;
- unsigned long swiotlb_nslabs;
u64 i;
max_addr = 0;
@@ -236,15 +233,7 @@ void __init plat_swiotlb_setup(void)
if (OCTEON_IS_OCTEON2() && max_addr >= 0x100000000ul)
swiotlbsize = 64 * (1<<20);
#endif
- swiotlb_nslabs = swiotlbsize >> IO_TLB_SHIFT;
- swiotlb_nslabs = ALIGN(swiotlb_nslabs, IO_TLB_SEGSIZE);
- swiotlbsize = swiotlb_nslabs << IO_TLB_SHIFT;
-
- octeon_swiotlb = memblock_alloc_low(swiotlbsize, PAGE_SIZE);
- if (!octeon_swiotlb)
- panic("%s: Failed to allocate %zu bytes align=%lx\n",
- __func__, swiotlbsize, PAGE_SIZE);
- if (swiotlb_init_with_tbl(octeon_swiotlb, swiotlb_nslabs, 1) = -ENOMEM)
- panic("Cannot allocate SWIOTLB buffer");
+ swiotlb_adjust_size(swiotlbsize);
+ swiotlb_init(1);
}
diff --git a/arch/mips/pci/pci-octeon.c b/arch/mips/pci/pci-octeon.c
index fc29b85cfa926..e457a18cbdc59 100644
--- a/arch/mips/pci/pci-octeon.c
+++ b/arch/mips/pci/pci-octeon.c
@@ -664,7 +664,7 @@ static int __init octeon_pci_setup(void)
/* BAR1 movable regions contiguous to cover the swiotlb */
octeon_bar1_pci_phys - virt_to_phys(octeon_swiotlb) & ~((1ull << 22) - 1);
+ io_tlb_default_mem.start & ~((1ull << 22) - 1);
for (index = 0; index < 32; index++) {
union cvmx_pci_bar1_indexx bar1_index;
--
2.30.2
^ permalink raw reply related [flat|nested] 118+ messages in thread
* [PATCH 07/11] x86: remove the IOMMU table infrastructure
2022-02-27 14:30 ` Christoph Hellwig
` (3 preceding siblings ...)
(?)
@ 2022-02-27 14:30 ` Christoph Hellwig
-1 siblings, 0 replies; 118+ messages in thread
From: Christoph Hellwig @ 2022-02-27 14:30 UTC (permalink / raw)
To: iommu
Cc: Juergen Gross, linux-s390, linux-hyperv, Stefano Stabellini,
linux-ia64, Anshuman Khandual, Konrad Rzeszutek Wilk,
Robin Murphy, x86, linux-mips, linuxppc-dev, tboot-devel,
linux-pci, xen-devel, Boris Ostrovsky, David Woodhouse,
linux-riscv, linux-arm-kernel
The IOMMU table tries to separate the different IOMMUs into different
backends, but actually requires various cross calls.
Rewrite the code to do the generic swiotlb/swiotlb-xen setup directly
in pci-dma.c and then just call into the IOMMU drivers.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
arch/ia64/include/asm/iommu_table.h | 7 --
arch/x86/include/asm/dma-mapping.h | 1 -
arch/x86/include/asm/gart.h | 5 +-
arch/x86/include/asm/iommu.h | 6 ++
arch/x86/include/asm/iommu_table.h | 102 -----------------------
arch/x86/include/asm/swiotlb.h | 30 -------
arch/x86/include/asm/xen/swiotlb-xen.h | 2 -
arch/x86/kernel/Makefile | 2 -
arch/x86/kernel/amd_gart_64.c | 5 +-
arch/x86/kernel/aperture_64.c | 14 ++--
arch/x86/kernel/pci-dma.c | 111 ++++++++++++++++++++-----
arch/x86/kernel/pci-iommu_table.c | 77 -----------------
arch/x86/kernel/pci-swiotlb.c | 77 -----------------
arch/x86/kernel/tboot.c | 1 -
arch/x86/kernel/vmlinux.lds.S | 12 ---
arch/x86/xen/Makefile | 2 -
arch/x86/xen/pci-swiotlb-xen.c | 96 ---------------------
drivers/iommu/amd/init.c | 6 --
drivers/iommu/amd/iommu.c | 5 +-
drivers/iommu/intel/dmar.c | 6 +-
include/linux/dmar.h | 6 +-
21 files changed, 114 insertions(+), 459 deletions(-)
delete mode 100644 arch/ia64/include/asm/iommu_table.h
delete mode 100644 arch/x86/include/asm/iommu_table.h
delete mode 100644 arch/x86/include/asm/swiotlb.h
delete mode 100644 arch/x86/kernel/pci-iommu_table.c
delete mode 100644 arch/x86/kernel/pci-swiotlb.c
delete mode 100644 arch/x86/xen/pci-swiotlb-xen.c
diff --git a/arch/ia64/include/asm/iommu_table.h b/arch/ia64/include/asm/iommu_table.h
deleted file mode 100644
index cc96116ac276a..0000000000000
--- a/arch/ia64/include/asm/iommu_table.h
+++ /dev/null
@@ -1,7 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0 */
-#ifndef _ASM_IA64_IOMMU_TABLE_H
-#define _ASM_IA64_IOMMU_TABLE_H
-
-#define IOMMU_INIT_POST(_detect)
-
-#endif /* _ASM_IA64_IOMMU_TABLE_H */
diff --git a/arch/x86/include/asm/dma-mapping.h b/arch/x86/include/asm/dma-mapping.h
index bb1654fe0ce74..256fd8115223d 100644
--- a/arch/x86/include/asm/dma-mapping.h
+++ b/arch/x86/include/asm/dma-mapping.h
@@ -9,7 +9,6 @@
#include <linux/scatterlist.h>
#include <asm/io.h>
-#include <asm/swiotlb.h>
extern int iommu_merge;
extern int panic_on_overflow;
diff --git a/arch/x86/include/asm/gart.h b/arch/x86/include/asm/gart.h
index 3185565743459..5af8088a10df6 100644
--- a/arch/x86/include/asm/gart.h
+++ b/arch/x86/include/asm/gart.h
@@ -38,7 +38,7 @@ extern int gart_iommu_aperture_disabled;
extern void early_gart_iommu_check(void);
extern int gart_iommu_init(void);
extern void __init gart_parse_options(char *);
-extern int gart_iommu_hole_init(void);
+void gart_iommu_hole_init(void);
#else
#define gart_iommu_aperture 0
@@ -51,9 +51,8 @@ static inline void early_gart_iommu_check(void)
static inline void gart_parse_options(char *options)
{
}
-static inline int gart_iommu_hole_init(void)
+static inline void gart_iommu_hole_init(void)
{
- return -ENODEV;
}
#endif
diff --git a/arch/x86/include/asm/iommu.h b/arch/x86/include/asm/iommu.h
index bf1ed2ddc74bd..dba89ed40d38d 100644
--- a/arch/x86/include/asm/iommu.h
+++ b/arch/x86/include/asm/iommu.h
@@ -9,6 +9,12 @@
extern int force_iommu, no_iommu;
extern int iommu_detected;
+#ifdef CONFIG_SWIOTLB
+extern bool x86_swiotlb_enable;
+#else
+#define x86_swiotlb_enable false
+#endif
+
/* 10 seconds */
#define DMAR_OPERATION_TIMEOUT ((cycles_t) tsc_khz*10*1000)
diff --git a/arch/x86/include/asm/iommu_table.h b/arch/x86/include/asm/iommu_table.h
deleted file mode 100644
index 1fb3fd1a83c25..0000000000000
--- a/arch/x86/include/asm/iommu_table.h
+++ /dev/null
@@ -1,102 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0 */
-#ifndef _ASM_X86_IOMMU_TABLE_H
-#define _ASM_X86_IOMMU_TABLE_H
-
-#include <asm/swiotlb.h>
-
-/*
- * History lesson:
- * The execution chain of IOMMUs in 2.6.36 looks as so:
- *
- * [xen-swiotlb]
- * |
- * +----[swiotlb *]--+
- * / | \
- * / | \
- * [GART] [Calgary] [Intel VT-d]
- * /
- * /
- * [AMD-Vi]
- *
- * *: if SWIOTLB detected 'iommu=soft'/'swiotlb=force' it would skip
- * over the rest of IOMMUs and unconditionally initialize the SWIOTLB.
- * Also it would surreptitiously initialize set the swiotlb=1 if there were
- * more than 4GB and if the user did not pass in 'iommu=off'. The swiotlb
- * flag would be turned off by all IOMMUs except the Calgary one.
- *
- * The IOMMU_INIT* macros allow a similar tree (or more complex if desired)
- * to be built by defining who we depend on.
- *
- * And all that needs to be done is to use one of the macros in the IOMMU
- * and the pci-dma.c will take care of the rest.
- */
-
-struct iommu_table_entry {
- initcall_t detect;
- initcall_t depend;
- void (*early_init)(void); /* No memory allocate available. */
- void (*late_init)(void); /* Yes, can allocate memory. */
-#define IOMMU_FINISH_IF_DETECTED (1<<0)
-#define IOMMU_DETECTED (1<<1)
- int flags;
-};
-/*
- * Macro fills out an entry in the .iommu_table that is equivalent
- * to the fields that 'struct iommu_table_entry' has. The entries
- * that are put in the .iommu_table section are not put in any order
- * hence during boot-time we will have to resort them based on
- * dependency. */
-
-
-#define __IOMMU_INIT(_detect, _depend, _early_init, _late_init, _finish)\
- static const struct iommu_table_entry \
- __iommu_entry_##_detect __used \
- __attribute__ ((unused, __section__(".iommu_table"), \
- aligned((sizeof(void *))))) \
- = {_detect, _depend, _early_init, _late_init, \
- _finish ? IOMMU_FINISH_IF_DETECTED : 0}
-/*
- * The simplest IOMMU definition. Provide the detection routine
- * and it will be run after the SWIOTLB and the other IOMMUs
- * that utilize this macro. If the IOMMU is detected (ie, the
- * detect routine returns a positive value), the other IOMMUs
- * are also checked. You can use IOMMU_INIT_POST_FINISH if you prefer
- * to stop detecting the other IOMMUs after yours has been detected.
- */
-#define IOMMU_INIT_POST(_detect) \
- __IOMMU_INIT(_detect, pci_swiotlb_detect_4gb, NULL, NULL, 0)
-
-#define IOMMU_INIT_POST_FINISH(detect) \
- __IOMMU_INIT(_detect, pci_swiotlb_detect_4gb, NULL, NULL, 1)
-
-/*
- * A more sophisticated version of IOMMU_INIT. This variant requires:
- * a). A detection routine function.
- * b). The name of the detection routine we depend on to get called
- * before us.
- * c). The init routine which gets called if the detection routine
- * returns a positive value from the pci_iommu_alloc. This means
- * no presence of a memory allocator.
- * d). Similar to the 'init', except that this gets called from pci_iommu_init
- * where we do have a memory allocator.
- *
- * The standard IOMMU_INIT differs from the IOMMU_INIT_FINISH variant
- * in that the former will continue detecting other IOMMUs in the call
- * list after the detection routine returns a positive number, while the
- * latter will stop the execution chain upon first successful detection.
- * Both variants will still call the 'init' and 'late_init' functions if
- * they are set.
- */
-#define IOMMU_INIT_FINISH(_detect, _depend, _init, _late_init) \
- __IOMMU_INIT(_detect, _depend, _init, _late_init, 1)
-
-#define IOMMU_INIT(_detect, _depend, _init, _late_init) \
- __IOMMU_INIT(_detect, _depend, _init, _late_init, 0)
-
-void sort_iommu_table(struct iommu_table_entry *start,
- struct iommu_table_entry *finish);
-
-void check_iommu_entries(struct iommu_table_entry *start,
- struct iommu_table_entry *finish);
-
-#endif /* _ASM_X86_IOMMU_TABLE_H */
diff --git a/arch/x86/include/asm/swiotlb.h b/arch/x86/include/asm/swiotlb.h
deleted file mode 100644
index ff6c92eff035a..0000000000000
--- a/arch/x86/include/asm/swiotlb.h
+++ /dev/null
@@ -1,30 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0 */
-#ifndef _ASM_X86_SWIOTLB_H
-#define _ASM_X86_SWIOTLB_H
-
-#include <linux/swiotlb.h>
-
-#ifdef CONFIG_SWIOTLB
-extern int swiotlb;
-extern int __init pci_swiotlb_detect_override(void);
-extern int __init pci_swiotlb_detect_4gb(void);
-extern void __init pci_swiotlb_init(void);
-extern void __init pci_swiotlb_late_init(void);
-#else
-#define swiotlb 0
-static inline int pci_swiotlb_detect_override(void)
-{
- return 0;
-}
-static inline int pci_swiotlb_detect_4gb(void)
-{
- return 0;
-}
-static inline void pci_swiotlb_init(void)
-{
-}
-static inline void pci_swiotlb_late_init(void)
-{
-}
-#endif
-#endif /* _ASM_X86_SWIOTLB_H */
diff --git a/arch/x86/include/asm/xen/swiotlb-xen.h b/arch/x86/include/asm/xen/swiotlb-xen.h
index 66b4ddde77430..e5a90b42e4dde 100644
--- a/arch/x86/include/asm/xen/swiotlb-xen.h
+++ b/arch/x86/include/asm/xen/swiotlb-xen.h
@@ -3,10 +3,8 @@
#define _ASM_X86_SWIOTLB_XEN_H
#ifdef CONFIG_SWIOTLB_XEN
-extern int __init pci_xen_swiotlb_detect(void);
extern int pci_xen_swiotlb_init_late(void);
#else
-#define pci_xen_swiotlb_detect NULL
static inline int pci_xen_swiotlb_init_late(void) { return -ENXIO; }
#endif
diff --git a/arch/x86/kernel/Makefile b/arch/x86/kernel/Makefile
index 6aef9ee28a394..2851d4f0aa0d2 100644
--- a/arch/x86/kernel/Makefile
+++ b/arch/x86/kernel/Makefile
@@ -71,7 +71,6 @@ obj-y += bootflag.o e820.o
obj-y += pci-dma.o quirks.o topology.o kdebugfs.o
obj-y += alternative.o i8253.o hw_breakpoint.o
obj-y += tsc.o tsc_msr.o io_delay.o rtc.o
-obj-y += pci-iommu_table.o
obj-y += resource.o
obj-y += irqflags.o
obj-y += static_call.o
@@ -136,7 +135,6 @@ obj-$(CONFIG_PCSPKR_PLATFORM) += pcspeaker.o
obj-$(CONFIG_X86_CHECK_BIOS_CORRUPTION) += check.o
-obj-$(CONFIG_SWIOTLB) += pci-swiotlb.o
obj-$(CONFIG_OF) += devicetree.o
obj-$(CONFIG_UPROBES) += uprobes.o
diff --git a/arch/x86/kernel/amd_gart_64.c b/arch/x86/kernel/amd_gart_64.c
index ed837383de5c8..194d54eed5376 100644
--- a/arch/x86/kernel/amd_gart_64.c
+++ b/arch/x86/kernel/amd_gart_64.c
@@ -38,11 +38,9 @@
#include <asm/iommu.h>
#include <asm/gart.h>
#include <asm/set_memory.h>
-#include <asm/swiotlb.h>
#include <asm/dma.h>
#include <asm/amd_nb.h>
#include <asm/x86_init.h>
-#include <asm/iommu_table.h>
static unsigned long iommu_bus_base; /* GART remapping area (physical) */
static unsigned long iommu_size; /* size of remapping area bytes */
@@ -808,7 +806,7 @@ int __init gart_iommu_init(void)
flush_gart();
dma_ops = &gart_dma_ops;
x86_platform.iommu_shutdown = gart_iommu_shutdown;
- swiotlb = 0;
+ x86_swiotlb_enable = false;
return 0;
}
@@ -842,4 +840,3 @@ void __init gart_parse_options(char *p)
}
}
}
-IOMMU_INIT_POST(gart_iommu_hole_init);
diff --git a/arch/x86/kernel/aperture_64.c b/arch/x86/kernel/aperture_64.c
index af3ba08b684b5..7a5630d904b23 100644
--- a/arch/x86/kernel/aperture_64.c
+++ b/arch/x86/kernel/aperture_64.c
@@ -392,7 +392,7 @@ void __init early_gart_iommu_check(void)
static int __initdata printed_gart_size_msg;
-int __init gart_iommu_hole_init(void)
+void __init gart_iommu_hole_init(void)
{
u32 agp_aper_base = 0, agp_aper_order = 0;
u32 aper_size, aper_alloc = 0, aper_order = 0, last_aper_order = 0;
@@ -401,11 +401,11 @@ int __init gart_iommu_hole_init(void)
int i, node;
if (!amd_gart_present())
- return -ENODEV;
+ return;
if (gart_iommu_aperture_disabled || !fix_aperture ||
!early_pci_allowed())
- return -ENODEV;
+ return;
pr_info("Checking aperture...\n");
@@ -491,10 +491,8 @@ int __init gart_iommu_hole_init(void)
* and fixed up the northbridge
*/
exclude_from_core(last_aper_base, last_aper_order);
-
- return 1;
}
- return 0;
+ return;
}
if (!fallback_aper_force) {
@@ -527,7 +525,7 @@ int __init gart_iommu_hole_init(void)
panic("Not enough memory for aperture");
}
} else {
- return 0;
+ return;
}
/*
@@ -561,6 +559,4 @@ int __init gart_iommu_hole_init(void)
}
set_up_gart_resume(aper_order, aper_alloc);
-
- return 1;
}
diff --git a/arch/x86/kernel/pci-dma.c b/arch/x86/kernel/pci-dma.c
index de234e7a8962e..17bcac942ac49 100644
--- a/arch/x86/kernel/pci-dma.c
+++ b/arch/x86/kernel/pci-dma.c
@@ -7,13 +7,16 @@
#include <linux/memblock.h>
#include <linux/gfp.h>
#include <linux/pci.h>
+#include <linux/amd-iommu.h>
#include <asm/proto.h>
#include <asm/dma.h>
#include <asm/iommu.h>
#include <asm/gart.h>
#include <asm/x86_init.h>
-#include <asm/iommu_table.h>
+
+#include <xen/xen.h>
+#include <xen/swiotlb-xen.h>
static bool disable_dac_quirk __read_mostly;
@@ -34,24 +37,87 @@ int no_iommu __read_mostly;
/* Set this to 1 if there is a HW IOMMU in the system */
int iommu_detected __read_mostly = 0;
-extern struct iommu_table_entry __iommu_table[], __iommu_table_end[];
+#ifdef CONFIG_SWIOTLB
+bool x86_swiotlb_enable;
+
+/*
+ * If 4GB or more detected (and iommu=off not set) or if SME is active
+ * then set swiotlb to 1 and return 1.
+ */
+static void __init pci_swiotlb_detect_4gb(void)
+{
+ /* don't initialize swiotlb if iommu=off (no_iommu=1) */
+ if (!no_iommu && max_possible_pfn > MAX_DMA32_PFN)
+ x86_swiotlb_enable = true;
+
+ /*
+ * Set swiotlb to 1 so that bounce buffers are allocated and used for
+ * devices that can't support DMA to encrypted memory.
+ */
+ if (cc_platform_has(CC_ATTR_HOST_MEM_ENCRYPT))
+ x86_swiotlb_enable = true;
+
+ if (swiotlb_force == SWIOTLB_FORCE)
+ x86_swiotlb_enable = true;
+}
+#else
+static inline void __init pci_swiotlb_detect_4gb(void)
+{
+}
+#endif /* CONFIG_SWIOTLB */
+
+#ifdef CONFIG_SWIOTLB_XEN
+static bool xen_swiotlb;
+
+static void __init pci_xen_swiotlb_init(void)
+{
+ if (!xen_initial_domain() && !x86_swiotlb_enable &&
+ swiotlb_force != SWIOTLB_FORCE)
+ return;
+ x86_swiotlb_enable = false;
+ xen_swiotlb = true;
+ xen_swiotlb_init_early();
+ dma_ops = &xen_swiotlb_dma_ops;
+ if (IS_ENABLED(CONFIG_PCI))
+ pci_request_acs();
+}
+
+int pci_xen_swiotlb_init_late(void)
+{
+ int rc;
+
+ if (xen_swiotlb)
+ return 0;
+
+ rc = xen_swiotlb_init();
+ if (rc)
+ return rc;
+
+ /* XXX: this switches the dma ops under live devices! */
+ dma_ops = &xen_swiotlb_dma_ops;
+ if (IS_ENABLED(CONFIG_PCI))
+ pci_request_acs();
+ return 0;
+}
+EXPORT_SYMBOL_GPL(pci_xen_swiotlb_init_late);
+#else
+static inline void __init pci_xen_swiotlb_init(void)
+{
+}
+#endif /* CONFIG_SWIOTLB_XEN */
void __init pci_iommu_alloc(void)
{
- struct iommu_table_entry *p;
-
- sort_iommu_table(__iommu_table, __iommu_table_end);
- check_iommu_entries(__iommu_table, __iommu_table_end);
-
- for (p = __iommu_table; p < __iommu_table_end; p++) {
- if (p && p->detect && p->detect() > 0) {
- p->flags |= IOMMU_DETECTED;
- if (p->early_init)
- p->early_init();
- if (p->flags & IOMMU_FINISH_IF_DETECTED)
- break;
- }
+ if (xen_pv_domain()) {
+ pci_xen_swiotlb_init();
+ return;
}
+ pci_swiotlb_detect_4gb();
+ gart_iommu_hole_init();
+ amd_iommu_detect();
+ detect_intel_iommu();
+ if (x86_swiotlb_enable)
+ swiotlb_init(0);
}
/*
@@ -102,7 +168,7 @@ static __init int iommu_setup(char *p)
}
#ifdef CONFIG_SWIOTLB
if (!strncmp(p, "soft", 4))
- swiotlb = 1;
+ x86_swiotlb_enable = 1;
#endif
if (!strncmp(p, "pt", 2))
iommu_set_default_passthrough(true);
@@ -121,14 +187,17 @@ early_param("iommu", iommu_setup);
static int __init pci_iommu_init(void)
{
- struct iommu_table_entry *p;
-
x86_init.iommu.iommu_init();
- for (p = __iommu_table; p < __iommu_table_end; p++) {
- if (p && (p->flags & IOMMU_DETECTED) && p->late_init)
- p->late_init();
+#ifdef CONFIG_SWIOTLB
+ /* An IOMMU turned us off. */
+ if (x86_swiotlb_enable) {
+ pr_info("PCI-DMA: Using software bounce buffering for IO (SWIOTLB)\n");
+ swiotlb_print_info();
+ } else {
+ swiotlb_exit();
}
+#endif
return 0;
}
diff --git a/arch/x86/kernel/pci-iommu_table.c b/arch/x86/kernel/pci-iommu_table.c
deleted file mode 100644
index 42e92ec62973b..0000000000000
--- a/arch/x86/kernel/pci-iommu_table.c
+++ /dev/null
@@ -1,77 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0
-#include <linux/dma-mapping.h>
-#include <asm/iommu_table.h>
-#include <linux/string.h>
-#include <linux/kallsyms.h>
-
-static struct iommu_table_entry * __init
-find_dependents_of(struct iommu_table_entry *start,
- struct iommu_table_entry *finish,
- struct iommu_table_entry *q)
-{
- struct iommu_table_entry *p;
-
- if (!q)
- return NULL;
-
- for (p = start; p < finish; p++)
- if (p->detect == q->depend)
- return p;
-
- return NULL;
-}
-
-
-void __init sort_iommu_table(struct iommu_table_entry *start,
- struct iommu_table_entry *finish) {
-
- struct iommu_table_entry *p, *q, tmp;
-
- for (p = start; p < finish; p++) {
-again:
- q = find_dependents_of(start, finish, p);
- /* We are bit sneaky here. We use the memory address to figure
- * out if the node we depend on is past our point, if so, swap.
- */
- if (q > p) {
- tmp = *p;
- memmove(p, q, sizeof(*p));
- *q = tmp;
- goto again;
- }
- }
-
-}
-
-#ifdef DEBUG
-void __init check_iommu_entries(struct iommu_table_entry *start,
- struct iommu_table_entry *finish)
-{
- struct iommu_table_entry *p, *q, *x;
-
- /* Simple cyclic dependency checker. */
- for (p = start; p < finish; p++) {
- q = find_dependents_of(start, finish, p);
- x = find_dependents_of(start, finish, q);
- if (p == x) {
- printk(KERN_ERR "CYCLIC DEPENDENCY FOUND! %pS depends on %pS and vice-versa. BREAKING IT.\n",
- p->detect, q->detect);
- /* Heavy handed way..*/
- x->depend = NULL;
- }
- }
-
- for (p = start; p < finish; p++) {
- q = find_dependents_of(p, finish, p);
- if (q && q > p) {
- printk(KERN_ERR "EXECUTION ORDER INVALID! %pS should be called before %pS!\n",
- p->detect, q->detect);
- }
- }
-}
-#else
-void __init check_iommu_entries(struct iommu_table_entry *start,
- struct iommu_table_entry *finish)
-{
-}
-#endif
diff --git a/arch/x86/kernel/pci-swiotlb.c b/arch/x86/kernel/pci-swiotlb.c
deleted file mode 100644
index 814ab46a0dada..0000000000000
--- a/arch/x86/kernel/pci-swiotlb.c
+++ /dev/null
@@ -1,77 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0
-
-#include <linux/pci.h>
-#include <linux/cache.h>
-#include <linux/init.h>
-#include <linux/swiotlb.h>
-#include <linux/memblock.h>
-#include <linux/dma-direct.h>
-#include <linux/cc_platform.h>
-
-#include <asm/iommu.h>
-#include <asm/swiotlb.h>
-#include <asm/dma.h>
-#include <asm/xen/swiotlb-xen.h>
-#include <asm/iommu_table.h>
-
-int swiotlb __read_mostly;
-
-/*
- * pci_swiotlb_detect_override - set swiotlb to 1 if necessary
- *
- * This returns non-zero if we are forced to use swiotlb (by the boot
- * option).
- */
-int __init pci_swiotlb_detect_override(void)
-{
- if (swiotlb_force == SWIOTLB_FORCE)
- swiotlb = 1;
-
- return swiotlb;
-}
-IOMMU_INIT_FINISH(pci_swiotlb_detect_override,
- pci_xen_swiotlb_detect,
- pci_swiotlb_init,
- pci_swiotlb_late_init);
-
-/*
- * If 4GB or more detected (and iommu=off not set) or if SME is active
- * then set swiotlb to 1 and return 1.
- */
-int __init pci_swiotlb_detect_4gb(void)
-{
- /* don't initialize swiotlb if iommu=off (no_iommu=1) */
- if (!no_iommu && max_possible_pfn > MAX_DMA32_PFN)
- swiotlb = 1;
-
- /*
- * Set swiotlb to 1 so that bounce buffers are allocated and used for
- * devices that can't support DMA to encrypted memory.
- */
- if (cc_platform_has(CC_ATTR_HOST_MEM_ENCRYPT))
- swiotlb = 1;
-
- return swiotlb;
-}
-IOMMU_INIT(pci_swiotlb_detect_4gb,
- pci_swiotlb_detect_override,
- pci_swiotlb_init,
- pci_swiotlb_late_init);
-
-void __init pci_swiotlb_init(void)
-{
- if (swiotlb)
- swiotlb_init(0);
-}
-
-void __init pci_swiotlb_late_init(void)
-{
- /* An IOMMU turned us off. */
- if (!swiotlb)
- swiotlb_exit();
- else {
- printk(KERN_INFO "PCI-DMA: "
- "Using software bounce buffering for IO (SWIOTLB)\n");
- swiotlb_print_info();
- }
-}
diff --git a/arch/x86/kernel/tboot.c b/arch/x86/kernel/tboot.c
index f9af561c3cd4f..0c1154a1c4032 100644
--- a/arch/x86/kernel/tboot.c
+++ b/arch/x86/kernel/tboot.c
@@ -24,7 +24,6 @@
#include <asm/processor.h>
#include <asm/bootparam.h>
#include <asm/pgalloc.h>
-#include <asm/swiotlb.h>
#include <asm/fixmap.h>
#include <asm/proto.h>
#include <asm/setup.h>
diff --git a/arch/x86/kernel/vmlinux.lds.S b/arch/x86/kernel/vmlinux.lds.S
index 27f830345b6f0..bbe910c15b293 100644
--- a/arch/x86/kernel/vmlinux.lds.S
+++ b/arch/x86/kernel/vmlinux.lds.S
@@ -306,18 +306,6 @@ SECTIONS
*(.altinstr_replacement)
}
- /*
- * struct iommu_table_entry entries are injected in this section.
- * It is an array of IOMMUs which during run time gets sorted depending
- * on its dependency order. After rootfs_initcall is complete
- * this section can be safely removed.
- */
- .iommu_table : AT(ADDR(.iommu_table) - LOAD_OFFSET) {
- __iommu_table = .;
- *(.iommu_table)
- __iommu_table_end = .;
- }
-
. = ALIGN(8);
.apicdrivers : AT(ADDR(.apicdrivers) - LOAD_OFFSET) {
__apicdrivers = .;
diff --git a/arch/x86/xen/Makefile b/arch/x86/xen/Makefile
index 4953260e281c3..3c5b52fbe4a7f 100644
--- a/arch/x86/xen/Makefile
+++ b/arch/x86/xen/Makefile
@@ -47,6 +47,4 @@ obj-$(CONFIG_XEN_DEBUG_FS) += debugfs.o
obj-$(CONFIG_XEN_PV_DOM0) += vga.o
-obj-$(CONFIG_SWIOTLB_XEN) += pci-swiotlb-xen.o
-
obj-$(CONFIG_XEN_EFI) += efi.o
diff --git a/arch/x86/xen/pci-swiotlb-xen.c b/arch/x86/xen/pci-swiotlb-xen.c
deleted file mode 100644
index 46df59aeaa06a..0000000000000
--- a/arch/x86/xen/pci-swiotlb-xen.c
+++ /dev/null
@@ -1,96 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0
-
-/* Glue code to lib/swiotlb-xen.c */
-
-#include <linux/dma-map-ops.h>
-#include <linux/pci.h>
-#include <xen/swiotlb-xen.h>
-
-#include <asm/xen/hypervisor.h>
-#include <xen/xen.h>
-#include <asm/iommu_table.h>
-
-
-#include <asm/xen/swiotlb-xen.h>
-#ifdef CONFIG_X86_64
-#include <asm/iommu.h>
-#include <asm/dma.h>
-#endif
-#include <linux/export.h>
-
-static int xen_swiotlb __read_mostly;
-
-/*
- * pci_xen_swiotlb_detect - set xen_swiotlb to 1 if necessary
- *
- * This returns non-zero if we are forced to use xen_swiotlb (by the boot
- * option).
- */
-int __init pci_xen_swiotlb_detect(void)
-{
-
- if (!xen_pv_domain())
- return 0;
-
- /* If running as PV guest, either iommu=soft, or swiotlb=force will
- * activate this IOMMU. If running as PV privileged, activate it
- * irregardless.
- */
- if (xen_initial_domain() || swiotlb || swiotlb_force == SWIOTLB_FORCE)
- xen_swiotlb = 1;
-
- /* If we are running under Xen, we MUST disable the native SWIOTLB.
- * Don't worry about swiotlb_force flag activating the native, as
- * the 'swiotlb' flag is the only one turning it on. */
- swiotlb = 0;
-
-#ifdef CONFIG_X86_64
- /* pci_swiotlb_detect_4gb turns on native SWIOTLB if no_iommu == 0
- * (so no iommu=X command line over-writes).
- * Considering that PV guests do not want the *native SWIOTLB* but
- * only Xen SWIOTLB it is not useful to us so set no_iommu=1 here.
- */
- if (max_pfn > MAX_DMA32_PFN)
- no_iommu = 1;
-#endif
- return xen_swiotlb;
-}
-
-static void __init pci_xen_swiotlb_init(void)
-{
- if (xen_swiotlb) {
- xen_swiotlb_init_early();
- dma_ops = &xen_swiotlb_dma_ops;
-
-#ifdef CONFIG_PCI
- /* Make sure ACS will be enabled */
- pci_request_acs();
-#endif
- }
-}
-
-int pci_xen_swiotlb_init_late(void)
-{
- int rc;
-
- if (xen_swiotlb)
- return 0;
-
- rc = xen_swiotlb_init();
- if (rc)
- return rc;
-
- dma_ops = &xen_swiotlb_dma_ops;
-#ifdef CONFIG_PCI
- /* Make sure ACS will be enabled */
- pci_request_acs();
-#endif
-
- return 0;
-}
-EXPORT_SYMBOL_GPL(pci_xen_swiotlb_init_late);
-
-IOMMU_INIT_FINISH(pci_xen_swiotlb_detect,
- NULL,
- pci_xen_swiotlb_init,
- NULL);
diff --git a/drivers/iommu/amd/init.c b/drivers/iommu/amd/init.c
index b10fb52ea4428..721300cf90207 100644
--- a/drivers/iommu/amd/init.c
+++ b/drivers/iommu/amd/init.c
@@ -27,7 +27,6 @@
#include <asm/apic.h>
#include <asm/gart.h>
#include <asm/x86_init.h>
-#include <asm/iommu_table.h>
#include <asm/io_apic.h>
#include <asm/irq_remapping.h>
#include <asm/set_memory.h>
@@ -3237,11 +3236,6 @@ __setup("ivrs_ioapic", parse_ivrs_ioapic);
__setup("ivrs_hpet", parse_ivrs_hpet);
__setup("ivrs_acpihid", parse_ivrs_acpihid);
-IOMMU_INIT_FINISH(amd_iommu_detect,
- gart_iommu_hole_init,
- NULL,
- NULL);
-
bool amd_iommu_v2_supported(void)
{
return amd_iommu_v2_present;
diff --git a/drivers/iommu/amd/iommu.c b/drivers/iommu/amd/iommu.c
index 461f1844ed1fb..541a7b8315a12 100644
--- a/drivers/iommu/amd/iommu.c
+++ b/drivers/iommu/amd/iommu.c
@@ -1834,7 +1834,10 @@ void amd_iommu_domain_update(struct protection_domain *domain)
static void __init amd_iommu_init_dma_ops(void)
{
- swiotlb = (iommu_default_passthrough() || sme_me_mask) ? 1 : 0;
+ if (iommu_default_passthrough() || sme_me_mask)
+ x86_swiotlb_enable = true;
+ else
+ x86_swiotlb_enable = false;
}
int __init amd_iommu_init_api(void)
diff --git a/drivers/iommu/intel/dmar.c b/drivers/iommu/intel/dmar.c
index 915bff76fe965..29bee4b210c5b 100644
--- a/drivers/iommu/intel/dmar.c
+++ b/drivers/iommu/intel/dmar.c
@@ -30,7 +30,6 @@
#include <linux/numa.h>
#include <linux/limits.h>
#include <asm/irq_remapping.h>
-#include <asm/iommu_table.h>
#include <trace/events/intel_iommu.h>
#include "../irq_remapping.h"
@@ -913,7 +912,7 @@ dmar_validate_one_drhd(struct acpi_dmar_header *entry, void *arg)
return 0;
}
-int __init detect_intel_iommu(void)
+void __init detect_intel_iommu(void)
{
int ret;
struct dmar_res_callback validate_drhd_cb = {
@@ -946,8 +945,6 @@ int __init detect_intel_iommu(void)
dmar_tbl = NULL;
}
up_write(&dmar_global_lock);
-
- return ret ? ret : 1;
}
static void unmap_iommu(struct intel_iommu *iommu)
@@ -2165,7 +2162,6 @@ static int __init dmar_free_unused_resources(void)
}
late_initcall(dmar_free_unused_resources);
-IOMMU_INIT_POST(detect_intel_iommu);
/*
* DMAR Hotplug Support
diff --git a/include/linux/dmar.h b/include/linux/dmar.h
index 45e903d847335..cbd714a198a0a 100644
--- a/include/linux/dmar.h
+++ b/include/linux/dmar.h
@@ -121,7 +121,7 @@ extern int dmar_remove_dev_scope(struct dmar_pci_notify_info *info,
u16 segment, struct dmar_dev_scope *devices,
int count);
/* Intel IOMMU detection */
-extern int detect_intel_iommu(void);
+void detect_intel_iommu(void);
extern int enable_drhd_fault_handling(void);
extern int dmar_device_add(acpi_handle handle);
extern int dmar_device_remove(acpi_handle handle);
@@ -197,6 +197,10 @@ static inline bool dmar_platform_optin(void)
return false;
}
+static inline void detect_intel_iommu(void)
+{
+}
+
#endif /* CONFIG_DMAR_TABLE */
struct irte {
--
2.30.2
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply related [flat|nested] 118+ messages in thread
* [PATCH 07/11] x86: remove the IOMMU table infrastructure
@ 2022-02-27 14:30 ` Christoph Hellwig
0 siblings, 0 replies; 118+ messages in thread
From: Christoph Hellwig @ 2022-02-27 14:30 UTC (permalink / raw)
To: iommu
Cc: x86, Anshuman Khandual, Konrad Rzeszutek Wilk,
Stefano Stabellini, Boris Ostrovsky, Juergen Gross, Joerg Roedel,
David Woodhouse, Lu Baolu, Robin Murphy, linux-arm-kernel,
xen-devel, linux-ia64, linux-mips, linuxppc-dev, linux-riscv,
linux-s390, linux-hyperv, tboot-devel, linux-pci
The IOMMU table tries to separate the different IOMMUs into different
backends, but actually requires various cross calls.
Rewrite the code to do the generic swiotlb/swiotlb-xen setup directly
in pci-dma.c and then just call into the IOMMU drivers.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
arch/ia64/include/asm/iommu_table.h | 7 --
arch/x86/include/asm/dma-mapping.h | 1 -
arch/x86/include/asm/gart.h | 5 +-
arch/x86/include/asm/iommu.h | 6 ++
arch/x86/include/asm/iommu_table.h | 102 -----------------------
arch/x86/include/asm/swiotlb.h | 30 -------
arch/x86/include/asm/xen/swiotlb-xen.h | 2 -
arch/x86/kernel/Makefile | 2 -
arch/x86/kernel/amd_gart_64.c | 5 +-
arch/x86/kernel/aperture_64.c | 14 ++--
arch/x86/kernel/pci-dma.c | 111 ++++++++++++++++++++-----
arch/x86/kernel/pci-iommu_table.c | 77 -----------------
arch/x86/kernel/pci-swiotlb.c | 77 -----------------
arch/x86/kernel/tboot.c | 1 -
arch/x86/kernel/vmlinux.lds.S | 12 ---
arch/x86/xen/Makefile | 2 -
arch/x86/xen/pci-swiotlb-xen.c | 96 ---------------------
drivers/iommu/amd/init.c | 6 --
drivers/iommu/amd/iommu.c | 5 +-
drivers/iommu/intel/dmar.c | 6 +-
include/linux/dmar.h | 6 +-
21 files changed, 114 insertions(+), 459 deletions(-)
delete mode 100644 arch/ia64/include/asm/iommu_table.h
delete mode 100644 arch/x86/include/asm/iommu_table.h
delete mode 100644 arch/x86/include/asm/swiotlb.h
delete mode 100644 arch/x86/kernel/pci-iommu_table.c
delete mode 100644 arch/x86/kernel/pci-swiotlb.c
delete mode 100644 arch/x86/xen/pci-swiotlb-xen.c
diff --git a/arch/ia64/include/asm/iommu_table.h b/arch/ia64/include/asm/iommu_table.h
deleted file mode 100644
index cc96116ac276a..0000000000000
--- a/arch/ia64/include/asm/iommu_table.h
+++ /dev/null
@@ -1,7 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0 */
-#ifndef _ASM_IA64_IOMMU_TABLE_H
-#define _ASM_IA64_IOMMU_TABLE_H
-
-#define IOMMU_INIT_POST(_detect)
-
-#endif /* _ASM_IA64_IOMMU_TABLE_H */
diff --git a/arch/x86/include/asm/dma-mapping.h b/arch/x86/include/asm/dma-mapping.h
index bb1654fe0ce74..256fd8115223d 100644
--- a/arch/x86/include/asm/dma-mapping.h
+++ b/arch/x86/include/asm/dma-mapping.h
@@ -9,7 +9,6 @@
#include <linux/scatterlist.h>
#include <asm/io.h>
-#include <asm/swiotlb.h>
extern int iommu_merge;
extern int panic_on_overflow;
diff --git a/arch/x86/include/asm/gart.h b/arch/x86/include/asm/gart.h
index 3185565743459..5af8088a10df6 100644
--- a/arch/x86/include/asm/gart.h
+++ b/arch/x86/include/asm/gart.h
@@ -38,7 +38,7 @@ extern int gart_iommu_aperture_disabled;
extern void early_gart_iommu_check(void);
extern int gart_iommu_init(void);
extern void __init gart_parse_options(char *);
-extern int gart_iommu_hole_init(void);
+void gart_iommu_hole_init(void);
#else
#define gart_iommu_aperture 0
@@ -51,9 +51,8 @@ static inline void early_gart_iommu_check(void)
static inline void gart_parse_options(char *options)
{
}
-static inline int gart_iommu_hole_init(void)
+static inline void gart_iommu_hole_init(void)
{
- return -ENODEV;
}
#endif
diff --git a/arch/x86/include/asm/iommu.h b/arch/x86/include/asm/iommu.h
index bf1ed2ddc74bd..dba89ed40d38d 100644
--- a/arch/x86/include/asm/iommu.h
+++ b/arch/x86/include/asm/iommu.h
@@ -9,6 +9,12 @@
extern int force_iommu, no_iommu;
extern int iommu_detected;
+#ifdef CONFIG_SWIOTLB
+extern bool x86_swiotlb_enable;
+#else
+#define x86_swiotlb_enable false
+#endif
+
/* 10 seconds */
#define DMAR_OPERATION_TIMEOUT ((cycles_t) tsc_khz*10*1000)
diff --git a/arch/x86/include/asm/iommu_table.h b/arch/x86/include/asm/iommu_table.h
deleted file mode 100644
index 1fb3fd1a83c25..0000000000000
--- a/arch/x86/include/asm/iommu_table.h
+++ /dev/null
@@ -1,102 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0 */
-#ifndef _ASM_X86_IOMMU_TABLE_H
-#define _ASM_X86_IOMMU_TABLE_H
-
-#include <asm/swiotlb.h>
-
-/*
- * History lesson:
- * The execution chain of IOMMUs in 2.6.36 looks as so:
- *
- * [xen-swiotlb]
- * |
- * +----[swiotlb *]--+
- * / | \
- * / | \
- * [GART] [Calgary] [Intel VT-d]
- * /
- * /
- * [AMD-Vi]
- *
- * *: if SWIOTLB detected 'iommu=soft'/'swiotlb=force' it would skip
- * over the rest of IOMMUs and unconditionally initialize the SWIOTLB.
- * Also it would surreptitiously initialize set the swiotlb=1 if there were
- * more than 4GB and if the user did not pass in 'iommu=off'. The swiotlb
- * flag would be turned off by all IOMMUs except the Calgary one.
- *
- * The IOMMU_INIT* macros allow a similar tree (or more complex if desired)
- * to be built by defining who we depend on.
- *
- * And all that needs to be done is to use one of the macros in the IOMMU
- * and the pci-dma.c will take care of the rest.
- */
-
-struct iommu_table_entry {
- initcall_t detect;
- initcall_t depend;
- void (*early_init)(void); /* No memory allocate available. */
- void (*late_init)(void); /* Yes, can allocate memory. */
-#define IOMMU_FINISH_IF_DETECTED (1<<0)
-#define IOMMU_DETECTED (1<<1)
- int flags;
-};
-/*
- * Macro fills out an entry in the .iommu_table that is equivalent
- * to the fields that 'struct iommu_table_entry' has. The entries
- * that are put in the .iommu_table section are not put in any order
- * hence during boot-time we will have to resort them based on
- * dependency. */
-
-
-#define __IOMMU_INIT(_detect, _depend, _early_init, _late_init, _finish)\
- static const struct iommu_table_entry \
- __iommu_entry_##_detect __used \
- __attribute__ ((unused, __section__(".iommu_table"), \
- aligned((sizeof(void *))))) \
- = {_detect, _depend, _early_init, _late_init, \
- _finish ? IOMMU_FINISH_IF_DETECTED : 0}
-/*
- * The simplest IOMMU definition. Provide the detection routine
- * and it will be run after the SWIOTLB and the other IOMMUs
- * that utilize this macro. If the IOMMU is detected (ie, the
- * detect routine returns a positive value), the other IOMMUs
- * are also checked. You can use IOMMU_INIT_POST_FINISH if you prefer
- * to stop detecting the other IOMMUs after yours has been detected.
- */
-#define IOMMU_INIT_POST(_detect) \
- __IOMMU_INIT(_detect, pci_swiotlb_detect_4gb, NULL, NULL, 0)
-
-#define IOMMU_INIT_POST_FINISH(detect) \
- __IOMMU_INIT(_detect, pci_swiotlb_detect_4gb, NULL, NULL, 1)
-
-/*
- * A more sophisticated version of IOMMU_INIT. This variant requires:
- * a). A detection routine function.
- * b). The name of the detection routine we depend on to get called
- * before us.
- * c). The init routine which gets called if the detection routine
- * returns a positive value from the pci_iommu_alloc. This means
- * no presence of a memory allocator.
- * d). Similar to the 'init', except that this gets called from pci_iommu_init
- * where we do have a memory allocator.
- *
- * The standard IOMMU_INIT differs from the IOMMU_INIT_FINISH variant
- * in that the former will continue detecting other IOMMUs in the call
- * list after the detection routine returns a positive number, while the
- * latter will stop the execution chain upon first successful detection.
- * Both variants will still call the 'init' and 'late_init' functions if
- * they are set.
- */
-#define IOMMU_INIT_FINISH(_detect, _depend, _init, _late_init) \
- __IOMMU_INIT(_detect, _depend, _init, _late_init, 1)
-
-#define IOMMU_INIT(_detect, _depend, _init, _late_init) \
- __IOMMU_INIT(_detect, _depend, _init, _late_init, 0)
-
-void sort_iommu_table(struct iommu_table_entry *start,
- struct iommu_table_entry *finish);
-
-void check_iommu_entries(struct iommu_table_entry *start,
- struct iommu_table_entry *finish);
-
-#endif /* _ASM_X86_IOMMU_TABLE_H */
diff --git a/arch/x86/include/asm/swiotlb.h b/arch/x86/include/asm/swiotlb.h
deleted file mode 100644
index ff6c92eff035a..0000000000000
--- a/arch/x86/include/asm/swiotlb.h
+++ /dev/null
@@ -1,30 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0 */
-#ifndef _ASM_X86_SWIOTLB_H
-#define _ASM_X86_SWIOTLB_H
-
-#include <linux/swiotlb.h>
-
-#ifdef CONFIG_SWIOTLB
-extern int swiotlb;
-extern int __init pci_swiotlb_detect_override(void);
-extern int __init pci_swiotlb_detect_4gb(void);
-extern void __init pci_swiotlb_init(void);
-extern void __init pci_swiotlb_late_init(void);
-#else
-#define swiotlb 0
-static inline int pci_swiotlb_detect_override(void)
-{
- return 0;
-}
-static inline int pci_swiotlb_detect_4gb(void)
-{
- return 0;
-}
-static inline void pci_swiotlb_init(void)
-{
-}
-static inline void pci_swiotlb_late_init(void)
-{
-}
-#endif
-#endif /* _ASM_X86_SWIOTLB_H */
diff --git a/arch/x86/include/asm/xen/swiotlb-xen.h b/arch/x86/include/asm/xen/swiotlb-xen.h
index 66b4ddde77430..e5a90b42e4dde 100644
--- a/arch/x86/include/asm/xen/swiotlb-xen.h
+++ b/arch/x86/include/asm/xen/swiotlb-xen.h
@@ -3,10 +3,8 @@
#define _ASM_X86_SWIOTLB_XEN_H
#ifdef CONFIG_SWIOTLB_XEN
-extern int __init pci_xen_swiotlb_detect(void);
extern int pci_xen_swiotlb_init_late(void);
#else
-#define pci_xen_swiotlb_detect NULL
static inline int pci_xen_swiotlb_init_late(void) { return -ENXIO; }
#endif
diff --git a/arch/x86/kernel/Makefile b/arch/x86/kernel/Makefile
index 6aef9ee28a394..2851d4f0aa0d2 100644
--- a/arch/x86/kernel/Makefile
+++ b/arch/x86/kernel/Makefile
@@ -71,7 +71,6 @@ obj-y += bootflag.o e820.o
obj-y += pci-dma.o quirks.o topology.o kdebugfs.o
obj-y += alternative.o i8253.o hw_breakpoint.o
obj-y += tsc.o tsc_msr.o io_delay.o rtc.o
-obj-y += pci-iommu_table.o
obj-y += resource.o
obj-y += irqflags.o
obj-y += static_call.o
@@ -136,7 +135,6 @@ obj-$(CONFIG_PCSPKR_PLATFORM) += pcspeaker.o
obj-$(CONFIG_X86_CHECK_BIOS_CORRUPTION) += check.o
-obj-$(CONFIG_SWIOTLB) += pci-swiotlb.o
obj-$(CONFIG_OF) += devicetree.o
obj-$(CONFIG_UPROBES) += uprobes.o
diff --git a/arch/x86/kernel/amd_gart_64.c b/arch/x86/kernel/amd_gart_64.c
index ed837383de5c8..194d54eed5376 100644
--- a/arch/x86/kernel/amd_gart_64.c
+++ b/arch/x86/kernel/amd_gart_64.c
@@ -38,11 +38,9 @@
#include <asm/iommu.h>
#include <asm/gart.h>
#include <asm/set_memory.h>
-#include <asm/swiotlb.h>
#include <asm/dma.h>
#include <asm/amd_nb.h>
#include <asm/x86_init.h>
-#include <asm/iommu_table.h>
static unsigned long iommu_bus_base; /* GART remapping area (physical) */
static unsigned long iommu_size; /* size of remapping area bytes */
@@ -808,7 +806,7 @@ int __init gart_iommu_init(void)
flush_gart();
dma_ops = &gart_dma_ops;
x86_platform.iommu_shutdown = gart_iommu_shutdown;
- swiotlb = 0;
+ x86_swiotlb_enable = false;
return 0;
}
@@ -842,4 +840,3 @@ void __init gart_parse_options(char *p)
}
}
}
-IOMMU_INIT_POST(gart_iommu_hole_init);
diff --git a/arch/x86/kernel/aperture_64.c b/arch/x86/kernel/aperture_64.c
index af3ba08b684b5..7a5630d904b23 100644
--- a/arch/x86/kernel/aperture_64.c
+++ b/arch/x86/kernel/aperture_64.c
@@ -392,7 +392,7 @@ void __init early_gart_iommu_check(void)
static int __initdata printed_gart_size_msg;
-int __init gart_iommu_hole_init(void)
+void __init gart_iommu_hole_init(void)
{
u32 agp_aper_base = 0, agp_aper_order = 0;
u32 aper_size, aper_alloc = 0, aper_order = 0, last_aper_order = 0;
@@ -401,11 +401,11 @@ int __init gart_iommu_hole_init(void)
int i, node;
if (!amd_gart_present())
- return -ENODEV;
+ return;
if (gart_iommu_aperture_disabled || !fix_aperture ||
!early_pci_allowed())
- return -ENODEV;
+ return;
pr_info("Checking aperture...\n");
@@ -491,10 +491,8 @@ int __init gart_iommu_hole_init(void)
* and fixed up the northbridge
*/
exclude_from_core(last_aper_base, last_aper_order);
-
- return 1;
}
- return 0;
+ return;
}
if (!fallback_aper_force) {
@@ -527,7 +525,7 @@ int __init gart_iommu_hole_init(void)
panic("Not enough memory for aperture");
}
} else {
- return 0;
+ return;
}
/*
@@ -561,6 +559,4 @@ int __init gart_iommu_hole_init(void)
}
set_up_gart_resume(aper_order, aper_alloc);
-
- return 1;
}
diff --git a/arch/x86/kernel/pci-dma.c b/arch/x86/kernel/pci-dma.c
index de234e7a8962e..17bcac942ac49 100644
--- a/arch/x86/kernel/pci-dma.c
+++ b/arch/x86/kernel/pci-dma.c
@@ -7,13 +7,16 @@
#include <linux/memblock.h>
#include <linux/gfp.h>
#include <linux/pci.h>
+#include <linux/amd-iommu.h>
#include <asm/proto.h>
#include <asm/dma.h>
#include <asm/iommu.h>
#include <asm/gart.h>
#include <asm/x86_init.h>
-#include <asm/iommu_table.h>
+
+#include <xen/xen.h>
+#include <xen/swiotlb-xen.h>
static bool disable_dac_quirk __read_mostly;
@@ -34,24 +37,87 @@ int no_iommu __read_mostly;
/* Set this to 1 if there is a HW IOMMU in the system */
int iommu_detected __read_mostly = 0;
-extern struct iommu_table_entry __iommu_table[], __iommu_table_end[];
+#ifdef CONFIG_SWIOTLB
+bool x86_swiotlb_enable;
+
+/*
+ * If 4GB or more detected (and iommu=off not set) or if SME is active
+ * then set swiotlb to 1 and return 1.
+ */
+static void __init pci_swiotlb_detect_4gb(void)
+{
+ /* don't initialize swiotlb if iommu=off (no_iommu=1) */
+ if (!no_iommu && max_possible_pfn > MAX_DMA32_PFN)
+ x86_swiotlb_enable = true;
+
+ /*
+ * Set swiotlb to 1 so that bounce buffers are allocated and used for
+ * devices that can't support DMA to encrypted memory.
+ */
+ if (cc_platform_has(CC_ATTR_HOST_MEM_ENCRYPT))
+ x86_swiotlb_enable = true;
+
+ if (swiotlb_force == SWIOTLB_FORCE)
+ x86_swiotlb_enable = true;
+}
+#else
+static inline void __init pci_swiotlb_detect_4gb(void)
+{
+}
+#endif /* CONFIG_SWIOTLB */
+
+#ifdef CONFIG_SWIOTLB_XEN
+static bool xen_swiotlb;
+
+static void __init pci_xen_swiotlb_init(void)
+{
+ if (!xen_initial_domain() && !x86_swiotlb_enable &&
+ swiotlb_force != SWIOTLB_FORCE)
+ return;
+ x86_swiotlb_enable = false;
+ xen_swiotlb = true;
+ xen_swiotlb_init_early();
+ dma_ops = &xen_swiotlb_dma_ops;
+ if (IS_ENABLED(CONFIG_PCI))
+ pci_request_acs();
+}
+
+int pci_xen_swiotlb_init_late(void)
+{
+ int rc;
+
+ if (xen_swiotlb)
+ return 0;
+
+ rc = xen_swiotlb_init();
+ if (rc)
+ return rc;
+
+ /* XXX: this switches the dma ops under live devices! */
+ dma_ops = &xen_swiotlb_dma_ops;
+ if (IS_ENABLED(CONFIG_PCI))
+ pci_request_acs();
+ return 0;
+}
+EXPORT_SYMBOL_GPL(pci_xen_swiotlb_init_late);
+#else
+static inline void __init pci_xen_swiotlb_init(void)
+{
+}
+#endif /* CONFIG_SWIOTLB_XEN */
void __init pci_iommu_alloc(void)
{
- struct iommu_table_entry *p;
-
- sort_iommu_table(__iommu_table, __iommu_table_end);
- check_iommu_entries(__iommu_table, __iommu_table_end);
-
- for (p = __iommu_table; p < __iommu_table_end; p++) {
- if (p && p->detect && p->detect() > 0) {
- p->flags |= IOMMU_DETECTED;
- if (p->early_init)
- p->early_init();
- if (p->flags & IOMMU_FINISH_IF_DETECTED)
- break;
- }
+ if (xen_pv_domain()) {
+ pci_xen_swiotlb_init();
+ return;
}
+ pci_swiotlb_detect_4gb();
+ gart_iommu_hole_init();
+ amd_iommu_detect();
+ detect_intel_iommu();
+ if (x86_swiotlb_enable)
+ swiotlb_init(0);
}
/*
@@ -102,7 +168,7 @@ static __init int iommu_setup(char *p)
}
#ifdef CONFIG_SWIOTLB
if (!strncmp(p, "soft", 4))
- swiotlb = 1;
+ x86_swiotlb_enable = 1;
#endif
if (!strncmp(p, "pt", 2))
iommu_set_default_passthrough(true);
@@ -121,14 +187,17 @@ early_param("iommu", iommu_setup);
static int __init pci_iommu_init(void)
{
- struct iommu_table_entry *p;
-
x86_init.iommu.iommu_init();
- for (p = __iommu_table; p < __iommu_table_end; p++) {
- if (p && (p->flags & IOMMU_DETECTED) && p->late_init)
- p->late_init();
+#ifdef CONFIG_SWIOTLB
+ /* An IOMMU turned us off. */
+ if (x86_swiotlb_enable) {
+ pr_info("PCI-DMA: Using software bounce buffering for IO (SWIOTLB)\n");
+ swiotlb_print_info();
+ } else {
+ swiotlb_exit();
}
+#endif
return 0;
}
diff --git a/arch/x86/kernel/pci-iommu_table.c b/arch/x86/kernel/pci-iommu_table.c
deleted file mode 100644
index 42e92ec62973b..0000000000000
--- a/arch/x86/kernel/pci-iommu_table.c
+++ /dev/null
@@ -1,77 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0
-#include <linux/dma-mapping.h>
-#include <asm/iommu_table.h>
-#include <linux/string.h>
-#include <linux/kallsyms.h>
-
-static struct iommu_table_entry * __init
-find_dependents_of(struct iommu_table_entry *start,
- struct iommu_table_entry *finish,
- struct iommu_table_entry *q)
-{
- struct iommu_table_entry *p;
-
- if (!q)
- return NULL;
-
- for (p = start; p < finish; p++)
- if (p->detect == q->depend)
- return p;
-
- return NULL;
-}
-
-
-void __init sort_iommu_table(struct iommu_table_entry *start,
- struct iommu_table_entry *finish) {
-
- struct iommu_table_entry *p, *q, tmp;
-
- for (p = start; p < finish; p++) {
-again:
- q = find_dependents_of(start, finish, p);
- /* We are bit sneaky here. We use the memory address to figure
- * out if the node we depend on is past our point, if so, swap.
- */
- if (q > p) {
- tmp = *p;
- memmove(p, q, sizeof(*p));
- *q = tmp;
- goto again;
- }
- }
-
-}
-
-#ifdef DEBUG
-void __init check_iommu_entries(struct iommu_table_entry *start,
- struct iommu_table_entry *finish)
-{
- struct iommu_table_entry *p, *q, *x;
-
- /* Simple cyclic dependency checker. */
- for (p = start; p < finish; p++) {
- q = find_dependents_of(start, finish, p);
- x = find_dependents_of(start, finish, q);
- if (p == x) {
- printk(KERN_ERR "CYCLIC DEPENDENCY FOUND! %pS depends on %pS and vice-versa. BREAKING IT.\n",
- p->detect, q->detect);
- /* Heavy handed way..*/
- x->depend = NULL;
- }
- }
-
- for (p = start; p < finish; p++) {
- q = find_dependents_of(p, finish, p);
- if (q && q > p) {
- printk(KERN_ERR "EXECUTION ORDER INVALID! %pS should be called before %pS!\n",
- p->detect, q->detect);
- }
- }
-}
-#else
-void __init check_iommu_entries(struct iommu_table_entry *start,
- struct iommu_table_entry *finish)
-{
-}
-#endif
diff --git a/arch/x86/kernel/pci-swiotlb.c b/arch/x86/kernel/pci-swiotlb.c
deleted file mode 100644
index 814ab46a0dada..0000000000000
--- a/arch/x86/kernel/pci-swiotlb.c
+++ /dev/null
@@ -1,77 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0
-
-#include <linux/pci.h>
-#include <linux/cache.h>
-#include <linux/init.h>
-#include <linux/swiotlb.h>
-#include <linux/memblock.h>
-#include <linux/dma-direct.h>
-#include <linux/cc_platform.h>
-
-#include <asm/iommu.h>
-#include <asm/swiotlb.h>
-#include <asm/dma.h>
-#include <asm/xen/swiotlb-xen.h>
-#include <asm/iommu_table.h>
-
-int swiotlb __read_mostly;
-
-/*
- * pci_swiotlb_detect_override - set swiotlb to 1 if necessary
- *
- * This returns non-zero if we are forced to use swiotlb (by the boot
- * option).
- */
-int __init pci_swiotlb_detect_override(void)
-{
- if (swiotlb_force == SWIOTLB_FORCE)
- swiotlb = 1;
-
- return swiotlb;
-}
-IOMMU_INIT_FINISH(pci_swiotlb_detect_override,
- pci_xen_swiotlb_detect,
- pci_swiotlb_init,
- pci_swiotlb_late_init);
-
-/*
- * If 4GB or more detected (and iommu=off not set) or if SME is active
- * then set swiotlb to 1 and return 1.
- */
-int __init pci_swiotlb_detect_4gb(void)
-{
- /* don't initialize swiotlb if iommu=off (no_iommu=1) */
- if (!no_iommu && max_possible_pfn > MAX_DMA32_PFN)
- swiotlb = 1;
-
- /*
- * Set swiotlb to 1 so that bounce buffers are allocated and used for
- * devices that can't support DMA to encrypted memory.
- */
- if (cc_platform_has(CC_ATTR_HOST_MEM_ENCRYPT))
- swiotlb = 1;
-
- return swiotlb;
-}
-IOMMU_INIT(pci_swiotlb_detect_4gb,
- pci_swiotlb_detect_override,
- pci_swiotlb_init,
- pci_swiotlb_late_init);
-
-void __init pci_swiotlb_init(void)
-{
- if (swiotlb)
- swiotlb_init(0);
-}
-
-void __init pci_swiotlb_late_init(void)
-{
- /* An IOMMU turned us off. */
- if (!swiotlb)
- swiotlb_exit();
- else {
- printk(KERN_INFO "PCI-DMA: "
- "Using software bounce buffering for IO (SWIOTLB)\n");
- swiotlb_print_info();
- }
-}
diff --git a/arch/x86/kernel/tboot.c b/arch/x86/kernel/tboot.c
index f9af561c3cd4f..0c1154a1c4032 100644
--- a/arch/x86/kernel/tboot.c
+++ b/arch/x86/kernel/tboot.c
@@ -24,7 +24,6 @@
#include <asm/processor.h>
#include <asm/bootparam.h>
#include <asm/pgalloc.h>
-#include <asm/swiotlb.h>
#include <asm/fixmap.h>
#include <asm/proto.h>
#include <asm/setup.h>
diff --git a/arch/x86/kernel/vmlinux.lds.S b/arch/x86/kernel/vmlinux.lds.S
index 27f830345b6f0..bbe910c15b293 100644
--- a/arch/x86/kernel/vmlinux.lds.S
+++ b/arch/x86/kernel/vmlinux.lds.S
@@ -306,18 +306,6 @@ SECTIONS
*(.altinstr_replacement)
}
- /*
- * struct iommu_table_entry entries are injected in this section.
- * It is an array of IOMMUs which during run time gets sorted depending
- * on its dependency order. After rootfs_initcall is complete
- * this section can be safely removed.
- */
- .iommu_table : AT(ADDR(.iommu_table) - LOAD_OFFSET) {
- __iommu_table = .;
- *(.iommu_table)
- __iommu_table_end = .;
- }
-
. = ALIGN(8);
.apicdrivers : AT(ADDR(.apicdrivers) - LOAD_OFFSET) {
__apicdrivers = .;
diff --git a/arch/x86/xen/Makefile b/arch/x86/xen/Makefile
index 4953260e281c3..3c5b52fbe4a7f 100644
--- a/arch/x86/xen/Makefile
+++ b/arch/x86/xen/Makefile
@@ -47,6 +47,4 @@ obj-$(CONFIG_XEN_DEBUG_FS) += debugfs.o
obj-$(CONFIG_XEN_PV_DOM0) += vga.o
-obj-$(CONFIG_SWIOTLB_XEN) += pci-swiotlb-xen.o
-
obj-$(CONFIG_XEN_EFI) += efi.o
diff --git a/arch/x86/xen/pci-swiotlb-xen.c b/arch/x86/xen/pci-swiotlb-xen.c
deleted file mode 100644
index 46df59aeaa06a..0000000000000
--- a/arch/x86/xen/pci-swiotlb-xen.c
+++ /dev/null
@@ -1,96 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0
-
-/* Glue code to lib/swiotlb-xen.c */
-
-#include <linux/dma-map-ops.h>
-#include <linux/pci.h>
-#include <xen/swiotlb-xen.h>
-
-#include <asm/xen/hypervisor.h>
-#include <xen/xen.h>
-#include <asm/iommu_table.h>
-
-
-#include <asm/xen/swiotlb-xen.h>
-#ifdef CONFIG_X86_64
-#include <asm/iommu.h>
-#include <asm/dma.h>
-#endif
-#include <linux/export.h>
-
-static int xen_swiotlb __read_mostly;
-
-/*
- * pci_xen_swiotlb_detect - set xen_swiotlb to 1 if necessary
- *
- * This returns non-zero if we are forced to use xen_swiotlb (by the boot
- * option).
- */
-int __init pci_xen_swiotlb_detect(void)
-{
-
- if (!xen_pv_domain())
- return 0;
-
- /* If running as PV guest, either iommu=soft, or swiotlb=force will
- * activate this IOMMU. If running as PV privileged, activate it
- * irregardless.
- */
- if (xen_initial_domain() || swiotlb || swiotlb_force == SWIOTLB_FORCE)
- xen_swiotlb = 1;
-
- /* If we are running under Xen, we MUST disable the native SWIOTLB.
- * Don't worry about swiotlb_force flag activating the native, as
- * the 'swiotlb' flag is the only one turning it on. */
- swiotlb = 0;
-
-#ifdef CONFIG_X86_64
- /* pci_swiotlb_detect_4gb turns on native SWIOTLB if no_iommu == 0
- * (so no iommu=X command line over-writes).
- * Considering that PV guests do not want the *native SWIOTLB* but
- * only Xen SWIOTLB it is not useful to us so set no_iommu=1 here.
- */
- if (max_pfn > MAX_DMA32_PFN)
- no_iommu = 1;
-#endif
- return xen_swiotlb;
-}
-
-static void __init pci_xen_swiotlb_init(void)
-{
- if (xen_swiotlb) {
- xen_swiotlb_init_early();
- dma_ops = &xen_swiotlb_dma_ops;
-
-#ifdef CONFIG_PCI
- /* Make sure ACS will be enabled */
- pci_request_acs();
-#endif
- }
-}
-
-int pci_xen_swiotlb_init_late(void)
-{
- int rc;
-
- if (xen_swiotlb)
- return 0;
-
- rc = xen_swiotlb_init();
- if (rc)
- return rc;
-
- dma_ops = &xen_swiotlb_dma_ops;
-#ifdef CONFIG_PCI
- /* Make sure ACS will be enabled */
- pci_request_acs();
-#endif
-
- return 0;
-}
-EXPORT_SYMBOL_GPL(pci_xen_swiotlb_init_late);
-
-IOMMU_INIT_FINISH(pci_xen_swiotlb_detect,
- NULL,
- pci_xen_swiotlb_init,
- NULL);
diff --git a/drivers/iommu/amd/init.c b/drivers/iommu/amd/init.c
index b10fb52ea4428..721300cf90207 100644
--- a/drivers/iommu/amd/init.c
+++ b/drivers/iommu/amd/init.c
@@ -27,7 +27,6 @@
#include <asm/apic.h>
#include <asm/gart.h>
#include <asm/x86_init.h>
-#include <asm/iommu_table.h>
#include <asm/io_apic.h>
#include <asm/irq_remapping.h>
#include <asm/set_memory.h>
@@ -3237,11 +3236,6 @@ __setup("ivrs_ioapic", parse_ivrs_ioapic);
__setup("ivrs_hpet", parse_ivrs_hpet);
__setup("ivrs_acpihid", parse_ivrs_acpihid);
-IOMMU_INIT_FINISH(amd_iommu_detect,
- gart_iommu_hole_init,
- NULL,
- NULL);
-
bool amd_iommu_v2_supported(void)
{
return amd_iommu_v2_present;
diff --git a/drivers/iommu/amd/iommu.c b/drivers/iommu/amd/iommu.c
index 461f1844ed1fb..541a7b8315a12 100644
--- a/drivers/iommu/amd/iommu.c
+++ b/drivers/iommu/amd/iommu.c
@@ -1834,7 +1834,10 @@ void amd_iommu_domain_update(struct protection_domain *domain)
static void __init amd_iommu_init_dma_ops(void)
{
- swiotlb = (iommu_default_passthrough() || sme_me_mask) ? 1 : 0;
+ if (iommu_default_passthrough() || sme_me_mask)
+ x86_swiotlb_enable = true;
+ else
+ x86_swiotlb_enable = false;
}
int __init amd_iommu_init_api(void)
diff --git a/drivers/iommu/intel/dmar.c b/drivers/iommu/intel/dmar.c
index 915bff76fe965..29bee4b210c5b 100644
--- a/drivers/iommu/intel/dmar.c
+++ b/drivers/iommu/intel/dmar.c
@@ -30,7 +30,6 @@
#include <linux/numa.h>
#include <linux/limits.h>
#include <asm/irq_remapping.h>
-#include <asm/iommu_table.h>
#include <trace/events/intel_iommu.h>
#include "../irq_remapping.h"
@@ -913,7 +912,7 @@ dmar_validate_one_drhd(struct acpi_dmar_header *entry, void *arg)
return 0;
}
-int __init detect_intel_iommu(void)
+void __init detect_intel_iommu(void)
{
int ret;
struct dmar_res_callback validate_drhd_cb = {
@@ -946,8 +945,6 @@ int __init detect_intel_iommu(void)
dmar_tbl = NULL;
}
up_write(&dmar_global_lock);
-
- return ret ? ret : 1;
}
static void unmap_iommu(struct intel_iommu *iommu)
@@ -2165,7 +2162,6 @@ static int __init dmar_free_unused_resources(void)
}
late_initcall(dmar_free_unused_resources);
-IOMMU_INIT_POST(detect_intel_iommu);
/*
* DMAR Hotplug Support
diff --git a/include/linux/dmar.h b/include/linux/dmar.h
index 45e903d847335..cbd714a198a0a 100644
--- a/include/linux/dmar.h
+++ b/include/linux/dmar.h
@@ -121,7 +121,7 @@ extern int dmar_remove_dev_scope(struct dmar_pci_notify_info *info,
u16 segment, struct dmar_dev_scope *devices,
int count);
/* Intel IOMMU detection */
-extern int detect_intel_iommu(void);
+void detect_intel_iommu(void);
extern int enable_drhd_fault_handling(void);
extern int dmar_device_add(acpi_handle handle);
extern int dmar_device_remove(acpi_handle handle);
@@ -197,6 +197,10 @@ static inline bool dmar_platform_optin(void)
return false;
}
+static inline void detect_intel_iommu(void)
+{
+}
+
#endif /* CONFIG_DMAR_TABLE */
struct irte {
--
2.30.2
^ permalink raw reply related [flat|nested] 118+ messages in thread
* [PATCH 07/11] x86: remove the IOMMU table infrastructure
@ 2022-02-27 14:30 ` Christoph Hellwig
0 siblings, 0 replies; 118+ messages in thread
From: Christoph Hellwig @ 2022-02-27 14:30 UTC (permalink / raw)
To: iommu
Cc: x86, Anshuman Khandual, Konrad Rzeszutek Wilk,
Stefano Stabellini, Boris Ostrovsky, Juergen Gross, Joerg Roedel,
David Woodhouse, Lu Baolu, Robin Murphy, linux-arm-kernel,
xen-devel, linux-ia64, linux-mips, linuxppc-dev, linux-riscv,
linux-s390, linux-hyperv, tboot-devel, linux-pci
The IOMMU table tries to separate the different IOMMUs into different
backends, but actually requires various cross calls.
Rewrite the code to do the generic swiotlb/swiotlb-xen setup directly
in pci-dma.c and then just call into the IOMMU drivers.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
arch/ia64/include/asm/iommu_table.h | 7 --
arch/x86/include/asm/dma-mapping.h | 1 -
arch/x86/include/asm/gart.h | 5 +-
arch/x86/include/asm/iommu.h | 6 ++
arch/x86/include/asm/iommu_table.h | 102 -----------------------
arch/x86/include/asm/swiotlb.h | 30 -------
arch/x86/include/asm/xen/swiotlb-xen.h | 2 -
arch/x86/kernel/Makefile | 2 -
arch/x86/kernel/amd_gart_64.c | 5 +-
arch/x86/kernel/aperture_64.c | 14 ++--
arch/x86/kernel/pci-dma.c | 111 ++++++++++++++++++++-----
arch/x86/kernel/pci-iommu_table.c | 77 -----------------
arch/x86/kernel/pci-swiotlb.c | 77 -----------------
arch/x86/kernel/tboot.c | 1 -
arch/x86/kernel/vmlinux.lds.S | 12 ---
arch/x86/xen/Makefile | 2 -
arch/x86/xen/pci-swiotlb-xen.c | 96 ---------------------
drivers/iommu/amd/init.c | 6 --
drivers/iommu/amd/iommu.c | 5 +-
drivers/iommu/intel/dmar.c | 6 +-
include/linux/dmar.h | 6 +-
21 files changed, 114 insertions(+), 459 deletions(-)
delete mode 100644 arch/ia64/include/asm/iommu_table.h
delete mode 100644 arch/x86/include/asm/iommu_table.h
delete mode 100644 arch/x86/include/asm/swiotlb.h
delete mode 100644 arch/x86/kernel/pci-iommu_table.c
delete mode 100644 arch/x86/kernel/pci-swiotlb.c
delete mode 100644 arch/x86/xen/pci-swiotlb-xen.c
diff --git a/arch/ia64/include/asm/iommu_table.h b/arch/ia64/include/asm/iommu_table.h
deleted file mode 100644
index cc96116ac276a..0000000000000
--- a/arch/ia64/include/asm/iommu_table.h
+++ /dev/null
@@ -1,7 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0 */
-#ifndef _ASM_IA64_IOMMU_TABLE_H
-#define _ASM_IA64_IOMMU_TABLE_H
-
-#define IOMMU_INIT_POST(_detect)
-
-#endif /* _ASM_IA64_IOMMU_TABLE_H */
diff --git a/arch/x86/include/asm/dma-mapping.h b/arch/x86/include/asm/dma-mapping.h
index bb1654fe0ce74..256fd8115223d 100644
--- a/arch/x86/include/asm/dma-mapping.h
+++ b/arch/x86/include/asm/dma-mapping.h
@@ -9,7 +9,6 @@
#include <linux/scatterlist.h>
#include <asm/io.h>
-#include <asm/swiotlb.h>
extern int iommu_merge;
extern int panic_on_overflow;
diff --git a/arch/x86/include/asm/gart.h b/arch/x86/include/asm/gart.h
index 3185565743459..5af8088a10df6 100644
--- a/arch/x86/include/asm/gart.h
+++ b/arch/x86/include/asm/gart.h
@@ -38,7 +38,7 @@ extern int gart_iommu_aperture_disabled;
extern void early_gart_iommu_check(void);
extern int gart_iommu_init(void);
extern void __init gart_parse_options(char *);
-extern int gart_iommu_hole_init(void);
+void gart_iommu_hole_init(void);
#else
#define gart_iommu_aperture 0
@@ -51,9 +51,8 @@ static inline void early_gart_iommu_check(void)
static inline void gart_parse_options(char *options)
{
}
-static inline int gart_iommu_hole_init(void)
+static inline void gart_iommu_hole_init(void)
{
- return -ENODEV;
}
#endif
diff --git a/arch/x86/include/asm/iommu.h b/arch/x86/include/asm/iommu.h
index bf1ed2ddc74bd..dba89ed40d38d 100644
--- a/arch/x86/include/asm/iommu.h
+++ b/arch/x86/include/asm/iommu.h
@@ -9,6 +9,12 @@
extern int force_iommu, no_iommu;
extern int iommu_detected;
+#ifdef CONFIG_SWIOTLB
+extern bool x86_swiotlb_enable;
+#else
+#define x86_swiotlb_enable false
+#endif
+
/* 10 seconds */
#define DMAR_OPERATION_TIMEOUT ((cycles_t) tsc_khz*10*1000)
diff --git a/arch/x86/include/asm/iommu_table.h b/arch/x86/include/asm/iommu_table.h
deleted file mode 100644
index 1fb3fd1a83c25..0000000000000
--- a/arch/x86/include/asm/iommu_table.h
+++ /dev/null
@@ -1,102 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0 */
-#ifndef _ASM_X86_IOMMU_TABLE_H
-#define _ASM_X86_IOMMU_TABLE_H
-
-#include <asm/swiotlb.h>
-
-/*
- * History lesson:
- * The execution chain of IOMMUs in 2.6.36 looks as so:
- *
- * [xen-swiotlb]
- * |
- * +----[swiotlb *]--+
- * / | \
- * / | \
- * [GART] [Calgary] [Intel VT-d]
- * /
- * /
- * [AMD-Vi]
- *
- * *: if SWIOTLB detected 'iommu=soft'/'swiotlb=force' it would skip
- * over the rest of IOMMUs and unconditionally initialize the SWIOTLB.
- * Also it would surreptitiously initialize set the swiotlb=1 if there were
- * more than 4GB and if the user did not pass in 'iommu=off'. The swiotlb
- * flag would be turned off by all IOMMUs except the Calgary one.
- *
- * The IOMMU_INIT* macros allow a similar tree (or more complex if desired)
- * to be built by defining who we depend on.
- *
- * And all that needs to be done is to use one of the macros in the IOMMU
- * and the pci-dma.c will take care of the rest.
- */
-
-struct iommu_table_entry {
- initcall_t detect;
- initcall_t depend;
- void (*early_init)(void); /* No memory allocate available. */
- void (*late_init)(void); /* Yes, can allocate memory. */
-#define IOMMU_FINISH_IF_DETECTED (1<<0)
-#define IOMMU_DETECTED (1<<1)
- int flags;
-};
-/*
- * Macro fills out an entry in the .iommu_table that is equivalent
- * to the fields that 'struct iommu_table_entry' has. The entries
- * that are put in the .iommu_table section are not put in any order
- * hence during boot-time we will have to resort them based on
- * dependency. */
-
-
-#define __IOMMU_INIT(_detect, _depend, _early_init, _late_init, _finish)\
- static const struct iommu_table_entry \
- __iommu_entry_##_detect __used \
- __attribute__ ((unused, __section__(".iommu_table"), \
- aligned((sizeof(void *))))) \
- = {_detect, _depend, _early_init, _late_init, \
- _finish ? IOMMU_FINISH_IF_DETECTED : 0}
-/*
- * The simplest IOMMU definition. Provide the detection routine
- * and it will be run after the SWIOTLB and the other IOMMUs
- * that utilize this macro. If the IOMMU is detected (ie, the
- * detect routine returns a positive value), the other IOMMUs
- * are also checked. You can use IOMMU_INIT_POST_FINISH if you prefer
- * to stop detecting the other IOMMUs after yours has been detected.
- */
-#define IOMMU_INIT_POST(_detect) \
- __IOMMU_INIT(_detect, pci_swiotlb_detect_4gb, NULL, NULL, 0)
-
-#define IOMMU_INIT_POST_FINISH(detect) \
- __IOMMU_INIT(_detect, pci_swiotlb_detect_4gb, NULL, NULL, 1)
-
-/*
- * A more sophisticated version of IOMMU_INIT. This variant requires:
- * a). A detection routine function.
- * b). The name of the detection routine we depend on to get called
- * before us.
- * c). The init routine which gets called if the detection routine
- * returns a positive value from the pci_iommu_alloc. This means
- * no presence of a memory allocator.
- * d). Similar to the 'init', except that this gets called from pci_iommu_init
- * where we do have a memory allocator.
- *
- * The standard IOMMU_INIT differs from the IOMMU_INIT_FINISH variant
- * in that the former will continue detecting other IOMMUs in the call
- * list after the detection routine returns a positive number, while the
- * latter will stop the execution chain upon first successful detection.
- * Both variants will still call the 'init' and 'late_init' functions if
- * they are set.
- */
-#define IOMMU_INIT_FINISH(_detect, _depend, _init, _late_init) \
- __IOMMU_INIT(_detect, _depend, _init, _late_init, 1)
-
-#define IOMMU_INIT(_detect, _depend, _init, _late_init) \
- __IOMMU_INIT(_detect, _depend, _init, _late_init, 0)
-
-void sort_iommu_table(struct iommu_table_entry *start,
- struct iommu_table_entry *finish);
-
-void check_iommu_entries(struct iommu_table_entry *start,
- struct iommu_table_entry *finish);
-
-#endif /* _ASM_X86_IOMMU_TABLE_H */
diff --git a/arch/x86/include/asm/swiotlb.h b/arch/x86/include/asm/swiotlb.h
deleted file mode 100644
index ff6c92eff035a..0000000000000
--- a/arch/x86/include/asm/swiotlb.h
+++ /dev/null
@@ -1,30 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0 */
-#ifndef _ASM_X86_SWIOTLB_H
-#define _ASM_X86_SWIOTLB_H
-
-#include <linux/swiotlb.h>
-
-#ifdef CONFIG_SWIOTLB
-extern int swiotlb;
-extern int __init pci_swiotlb_detect_override(void);
-extern int __init pci_swiotlb_detect_4gb(void);
-extern void __init pci_swiotlb_init(void);
-extern void __init pci_swiotlb_late_init(void);
-#else
-#define swiotlb 0
-static inline int pci_swiotlb_detect_override(void)
-{
- return 0;
-}
-static inline int pci_swiotlb_detect_4gb(void)
-{
- return 0;
-}
-static inline void pci_swiotlb_init(void)
-{
-}
-static inline void pci_swiotlb_late_init(void)
-{
-}
-#endif
-#endif /* _ASM_X86_SWIOTLB_H */
diff --git a/arch/x86/include/asm/xen/swiotlb-xen.h b/arch/x86/include/asm/xen/swiotlb-xen.h
index 66b4ddde77430..e5a90b42e4dde 100644
--- a/arch/x86/include/asm/xen/swiotlb-xen.h
+++ b/arch/x86/include/asm/xen/swiotlb-xen.h
@@ -3,10 +3,8 @@
#define _ASM_X86_SWIOTLB_XEN_H
#ifdef CONFIG_SWIOTLB_XEN
-extern int __init pci_xen_swiotlb_detect(void);
extern int pci_xen_swiotlb_init_late(void);
#else
-#define pci_xen_swiotlb_detect NULL
static inline int pci_xen_swiotlb_init_late(void) { return -ENXIO; }
#endif
diff --git a/arch/x86/kernel/Makefile b/arch/x86/kernel/Makefile
index 6aef9ee28a394..2851d4f0aa0d2 100644
--- a/arch/x86/kernel/Makefile
+++ b/arch/x86/kernel/Makefile
@@ -71,7 +71,6 @@ obj-y += bootflag.o e820.o
obj-y += pci-dma.o quirks.o topology.o kdebugfs.o
obj-y += alternative.o i8253.o hw_breakpoint.o
obj-y += tsc.o tsc_msr.o io_delay.o rtc.o
-obj-y += pci-iommu_table.o
obj-y += resource.o
obj-y += irqflags.o
obj-y += static_call.o
@@ -136,7 +135,6 @@ obj-$(CONFIG_PCSPKR_PLATFORM) += pcspeaker.o
obj-$(CONFIG_X86_CHECK_BIOS_CORRUPTION) += check.o
-obj-$(CONFIG_SWIOTLB) += pci-swiotlb.o
obj-$(CONFIG_OF) += devicetree.o
obj-$(CONFIG_UPROBES) += uprobes.o
diff --git a/arch/x86/kernel/amd_gart_64.c b/arch/x86/kernel/amd_gart_64.c
index ed837383de5c8..194d54eed5376 100644
--- a/arch/x86/kernel/amd_gart_64.c
+++ b/arch/x86/kernel/amd_gart_64.c
@@ -38,11 +38,9 @@
#include <asm/iommu.h>
#include <asm/gart.h>
#include <asm/set_memory.h>
-#include <asm/swiotlb.h>
#include <asm/dma.h>
#include <asm/amd_nb.h>
#include <asm/x86_init.h>
-#include <asm/iommu_table.h>
static unsigned long iommu_bus_base; /* GART remapping area (physical) */
static unsigned long iommu_size; /* size of remapping area bytes */
@@ -808,7 +806,7 @@ int __init gart_iommu_init(void)
flush_gart();
dma_ops = &gart_dma_ops;
x86_platform.iommu_shutdown = gart_iommu_shutdown;
- swiotlb = 0;
+ x86_swiotlb_enable = false;
return 0;
}
@@ -842,4 +840,3 @@ void __init gart_parse_options(char *p)
}
}
}
-IOMMU_INIT_POST(gart_iommu_hole_init);
diff --git a/arch/x86/kernel/aperture_64.c b/arch/x86/kernel/aperture_64.c
index af3ba08b684b5..7a5630d904b23 100644
--- a/arch/x86/kernel/aperture_64.c
+++ b/arch/x86/kernel/aperture_64.c
@@ -392,7 +392,7 @@ void __init early_gart_iommu_check(void)
static int __initdata printed_gart_size_msg;
-int __init gart_iommu_hole_init(void)
+void __init gart_iommu_hole_init(void)
{
u32 agp_aper_base = 0, agp_aper_order = 0;
u32 aper_size, aper_alloc = 0, aper_order = 0, last_aper_order = 0;
@@ -401,11 +401,11 @@ int __init gart_iommu_hole_init(void)
int i, node;
if (!amd_gart_present())
- return -ENODEV;
+ return;
if (gart_iommu_aperture_disabled || !fix_aperture ||
!early_pci_allowed())
- return -ENODEV;
+ return;
pr_info("Checking aperture...\n");
@@ -491,10 +491,8 @@ int __init gart_iommu_hole_init(void)
* and fixed up the northbridge
*/
exclude_from_core(last_aper_base, last_aper_order);
-
- return 1;
}
- return 0;
+ return;
}
if (!fallback_aper_force) {
@@ -527,7 +525,7 @@ int __init gart_iommu_hole_init(void)
panic("Not enough memory for aperture");
}
} else {
- return 0;
+ return;
}
/*
@@ -561,6 +559,4 @@ int __init gart_iommu_hole_init(void)
}
set_up_gart_resume(aper_order, aper_alloc);
-
- return 1;
}
diff --git a/arch/x86/kernel/pci-dma.c b/arch/x86/kernel/pci-dma.c
index de234e7a8962e..17bcac942ac49 100644
--- a/arch/x86/kernel/pci-dma.c
+++ b/arch/x86/kernel/pci-dma.c
@@ -7,13 +7,16 @@
#include <linux/memblock.h>
#include <linux/gfp.h>
#include <linux/pci.h>
+#include <linux/amd-iommu.h>
#include <asm/proto.h>
#include <asm/dma.h>
#include <asm/iommu.h>
#include <asm/gart.h>
#include <asm/x86_init.h>
-#include <asm/iommu_table.h>
+
+#include <xen/xen.h>
+#include <xen/swiotlb-xen.h>
static bool disable_dac_quirk __read_mostly;
@@ -34,24 +37,87 @@ int no_iommu __read_mostly;
/* Set this to 1 if there is a HW IOMMU in the system */
int iommu_detected __read_mostly = 0;
-extern struct iommu_table_entry __iommu_table[], __iommu_table_end[];
+#ifdef CONFIG_SWIOTLB
+bool x86_swiotlb_enable;
+
+/*
+ * If 4GB or more detected (and iommu=off not set) or if SME is active
+ * then set swiotlb to 1 and return 1.
+ */
+static void __init pci_swiotlb_detect_4gb(void)
+{
+ /* don't initialize swiotlb if iommu=off (no_iommu=1) */
+ if (!no_iommu && max_possible_pfn > MAX_DMA32_PFN)
+ x86_swiotlb_enable = true;
+
+ /*
+ * Set swiotlb to 1 so that bounce buffers are allocated and used for
+ * devices that can't support DMA to encrypted memory.
+ */
+ if (cc_platform_has(CC_ATTR_HOST_MEM_ENCRYPT))
+ x86_swiotlb_enable = true;
+
+ if (swiotlb_force == SWIOTLB_FORCE)
+ x86_swiotlb_enable = true;
+}
+#else
+static inline void __init pci_swiotlb_detect_4gb(void)
+{
+}
+#endif /* CONFIG_SWIOTLB */
+
+#ifdef CONFIG_SWIOTLB_XEN
+static bool xen_swiotlb;
+
+static void __init pci_xen_swiotlb_init(void)
+{
+ if (!xen_initial_domain() && !x86_swiotlb_enable &&
+ swiotlb_force != SWIOTLB_FORCE)
+ return;
+ x86_swiotlb_enable = false;
+ xen_swiotlb = true;
+ xen_swiotlb_init_early();
+ dma_ops = &xen_swiotlb_dma_ops;
+ if (IS_ENABLED(CONFIG_PCI))
+ pci_request_acs();
+}
+
+int pci_xen_swiotlb_init_late(void)
+{
+ int rc;
+
+ if (xen_swiotlb)
+ return 0;
+
+ rc = xen_swiotlb_init();
+ if (rc)
+ return rc;
+
+ /* XXX: this switches the dma ops under live devices! */
+ dma_ops = &xen_swiotlb_dma_ops;
+ if (IS_ENABLED(CONFIG_PCI))
+ pci_request_acs();
+ return 0;
+}
+EXPORT_SYMBOL_GPL(pci_xen_swiotlb_init_late);
+#else
+static inline void __init pci_xen_swiotlb_init(void)
+{
+}
+#endif /* CONFIG_SWIOTLB_XEN */
void __init pci_iommu_alloc(void)
{
- struct iommu_table_entry *p;
-
- sort_iommu_table(__iommu_table, __iommu_table_end);
- check_iommu_entries(__iommu_table, __iommu_table_end);
-
- for (p = __iommu_table; p < __iommu_table_end; p++) {
- if (p && p->detect && p->detect() > 0) {
- p->flags |= IOMMU_DETECTED;
- if (p->early_init)
- p->early_init();
- if (p->flags & IOMMU_FINISH_IF_DETECTED)
- break;
- }
+ if (xen_pv_domain()) {
+ pci_xen_swiotlb_init();
+ return;
}
+ pci_swiotlb_detect_4gb();
+ gart_iommu_hole_init();
+ amd_iommu_detect();
+ detect_intel_iommu();
+ if (x86_swiotlb_enable)
+ swiotlb_init(0);
}
/*
@@ -102,7 +168,7 @@ static __init int iommu_setup(char *p)
}
#ifdef CONFIG_SWIOTLB
if (!strncmp(p, "soft", 4))
- swiotlb = 1;
+ x86_swiotlb_enable = 1;
#endif
if (!strncmp(p, "pt", 2))
iommu_set_default_passthrough(true);
@@ -121,14 +187,17 @@ early_param("iommu", iommu_setup);
static int __init pci_iommu_init(void)
{
- struct iommu_table_entry *p;
-
x86_init.iommu.iommu_init();
- for (p = __iommu_table; p < __iommu_table_end; p++) {
- if (p && (p->flags & IOMMU_DETECTED) && p->late_init)
- p->late_init();
+#ifdef CONFIG_SWIOTLB
+ /* An IOMMU turned us off. */
+ if (x86_swiotlb_enable) {
+ pr_info("PCI-DMA: Using software bounce buffering for IO (SWIOTLB)\n");
+ swiotlb_print_info();
+ } else {
+ swiotlb_exit();
}
+#endif
return 0;
}
diff --git a/arch/x86/kernel/pci-iommu_table.c b/arch/x86/kernel/pci-iommu_table.c
deleted file mode 100644
index 42e92ec62973b..0000000000000
--- a/arch/x86/kernel/pci-iommu_table.c
+++ /dev/null
@@ -1,77 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0
-#include <linux/dma-mapping.h>
-#include <asm/iommu_table.h>
-#include <linux/string.h>
-#include <linux/kallsyms.h>
-
-static struct iommu_table_entry * __init
-find_dependents_of(struct iommu_table_entry *start,
- struct iommu_table_entry *finish,
- struct iommu_table_entry *q)
-{
- struct iommu_table_entry *p;
-
- if (!q)
- return NULL;
-
- for (p = start; p < finish; p++)
- if (p->detect == q->depend)
- return p;
-
- return NULL;
-}
-
-
-void __init sort_iommu_table(struct iommu_table_entry *start,
- struct iommu_table_entry *finish) {
-
- struct iommu_table_entry *p, *q, tmp;
-
- for (p = start; p < finish; p++) {
-again:
- q = find_dependents_of(start, finish, p);
- /* We are bit sneaky here. We use the memory address to figure
- * out if the node we depend on is past our point, if so, swap.
- */
- if (q > p) {
- tmp = *p;
- memmove(p, q, sizeof(*p));
- *q = tmp;
- goto again;
- }
- }
-
-}
-
-#ifdef DEBUG
-void __init check_iommu_entries(struct iommu_table_entry *start,
- struct iommu_table_entry *finish)
-{
- struct iommu_table_entry *p, *q, *x;
-
- /* Simple cyclic dependency checker. */
- for (p = start; p < finish; p++) {
- q = find_dependents_of(start, finish, p);
- x = find_dependents_of(start, finish, q);
- if (p == x) {
- printk(KERN_ERR "CYCLIC DEPENDENCY FOUND! %pS depends on %pS and vice-versa. BREAKING IT.\n",
- p->detect, q->detect);
- /* Heavy handed way..*/
- x->depend = NULL;
- }
- }
-
- for (p = start; p < finish; p++) {
- q = find_dependents_of(p, finish, p);
- if (q && q > p) {
- printk(KERN_ERR "EXECUTION ORDER INVALID! %pS should be called before %pS!\n",
- p->detect, q->detect);
- }
- }
-}
-#else
-void __init check_iommu_entries(struct iommu_table_entry *start,
- struct iommu_table_entry *finish)
-{
-}
-#endif
diff --git a/arch/x86/kernel/pci-swiotlb.c b/arch/x86/kernel/pci-swiotlb.c
deleted file mode 100644
index 814ab46a0dada..0000000000000
--- a/arch/x86/kernel/pci-swiotlb.c
+++ /dev/null
@@ -1,77 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0
-
-#include <linux/pci.h>
-#include <linux/cache.h>
-#include <linux/init.h>
-#include <linux/swiotlb.h>
-#include <linux/memblock.h>
-#include <linux/dma-direct.h>
-#include <linux/cc_platform.h>
-
-#include <asm/iommu.h>
-#include <asm/swiotlb.h>
-#include <asm/dma.h>
-#include <asm/xen/swiotlb-xen.h>
-#include <asm/iommu_table.h>
-
-int swiotlb __read_mostly;
-
-/*
- * pci_swiotlb_detect_override - set swiotlb to 1 if necessary
- *
- * This returns non-zero if we are forced to use swiotlb (by the boot
- * option).
- */
-int __init pci_swiotlb_detect_override(void)
-{
- if (swiotlb_force == SWIOTLB_FORCE)
- swiotlb = 1;
-
- return swiotlb;
-}
-IOMMU_INIT_FINISH(pci_swiotlb_detect_override,
- pci_xen_swiotlb_detect,
- pci_swiotlb_init,
- pci_swiotlb_late_init);
-
-/*
- * If 4GB or more detected (and iommu=off not set) or if SME is active
- * then set swiotlb to 1 and return 1.
- */
-int __init pci_swiotlb_detect_4gb(void)
-{
- /* don't initialize swiotlb if iommu=off (no_iommu=1) */
- if (!no_iommu && max_possible_pfn > MAX_DMA32_PFN)
- swiotlb = 1;
-
- /*
- * Set swiotlb to 1 so that bounce buffers are allocated and used for
- * devices that can't support DMA to encrypted memory.
- */
- if (cc_platform_has(CC_ATTR_HOST_MEM_ENCRYPT))
- swiotlb = 1;
-
- return swiotlb;
-}
-IOMMU_INIT(pci_swiotlb_detect_4gb,
- pci_swiotlb_detect_override,
- pci_swiotlb_init,
- pci_swiotlb_late_init);
-
-void __init pci_swiotlb_init(void)
-{
- if (swiotlb)
- swiotlb_init(0);
-}
-
-void __init pci_swiotlb_late_init(void)
-{
- /* An IOMMU turned us off. */
- if (!swiotlb)
- swiotlb_exit();
- else {
- printk(KERN_INFO "PCI-DMA: "
- "Using software bounce buffering for IO (SWIOTLB)\n");
- swiotlb_print_info();
- }
-}
diff --git a/arch/x86/kernel/tboot.c b/arch/x86/kernel/tboot.c
index f9af561c3cd4f..0c1154a1c4032 100644
--- a/arch/x86/kernel/tboot.c
+++ b/arch/x86/kernel/tboot.c
@@ -24,7 +24,6 @@
#include <asm/processor.h>
#include <asm/bootparam.h>
#include <asm/pgalloc.h>
-#include <asm/swiotlb.h>
#include <asm/fixmap.h>
#include <asm/proto.h>
#include <asm/setup.h>
diff --git a/arch/x86/kernel/vmlinux.lds.S b/arch/x86/kernel/vmlinux.lds.S
index 27f830345b6f0..bbe910c15b293 100644
--- a/arch/x86/kernel/vmlinux.lds.S
+++ b/arch/x86/kernel/vmlinux.lds.S
@@ -306,18 +306,6 @@ SECTIONS
*(.altinstr_replacement)
}
- /*
- * struct iommu_table_entry entries are injected in this section.
- * It is an array of IOMMUs which during run time gets sorted depending
- * on its dependency order. After rootfs_initcall is complete
- * this section can be safely removed.
- */
- .iommu_table : AT(ADDR(.iommu_table) - LOAD_OFFSET) {
- __iommu_table = .;
- *(.iommu_table)
- __iommu_table_end = .;
- }
-
. = ALIGN(8);
.apicdrivers : AT(ADDR(.apicdrivers) - LOAD_OFFSET) {
__apicdrivers = .;
diff --git a/arch/x86/xen/Makefile b/arch/x86/xen/Makefile
index 4953260e281c3..3c5b52fbe4a7f 100644
--- a/arch/x86/xen/Makefile
+++ b/arch/x86/xen/Makefile
@@ -47,6 +47,4 @@ obj-$(CONFIG_XEN_DEBUG_FS) += debugfs.o
obj-$(CONFIG_XEN_PV_DOM0) += vga.o
-obj-$(CONFIG_SWIOTLB_XEN) += pci-swiotlb-xen.o
-
obj-$(CONFIG_XEN_EFI) += efi.o
diff --git a/arch/x86/xen/pci-swiotlb-xen.c b/arch/x86/xen/pci-swiotlb-xen.c
deleted file mode 100644
index 46df59aeaa06a..0000000000000
--- a/arch/x86/xen/pci-swiotlb-xen.c
+++ /dev/null
@@ -1,96 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0
-
-/* Glue code to lib/swiotlb-xen.c */
-
-#include <linux/dma-map-ops.h>
-#include <linux/pci.h>
-#include <xen/swiotlb-xen.h>
-
-#include <asm/xen/hypervisor.h>
-#include <xen/xen.h>
-#include <asm/iommu_table.h>
-
-
-#include <asm/xen/swiotlb-xen.h>
-#ifdef CONFIG_X86_64
-#include <asm/iommu.h>
-#include <asm/dma.h>
-#endif
-#include <linux/export.h>
-
-static int xen_swiotlb __read_mostly;
-
-/*
- * pci_xen_swiotlb_detect - set xen_swiotlb to 1 if necessary
- *
- * This returns non-zero if we are forced to use xen_swiotlb (by the boot
- * option).
- */
-int __init pci_xen_swiotlb_detect(void)
-{
-
- if (!xen_pv_domain())
- return 0;
-
- /* If running as PV guest, either iommu=soft, or swiotlb=force will
- * activate this IOMMU. If running as PV privileged, activate it
- * irregardless.
- */
- if (xen_initial_domain() || swiotlb || swiotlb_force == SWIOTLB_FORCE)
- xen_swiotlb = 1;
-
- /* If we are running under Xen, we MUST disable the native SWIOTLB.
- * Don't worry about swiotlb_force flag activating the native, as
- * the 'swiotlb' flag is the only one turning it on. */
- swiotlb = 0;
-
-#ifdef CONFIG_X86_64
- /* pci_swiotlb_detect_4gb turns on native SWIOTLB if no_iommu == 0
- * (so no iommu=X command line over-writes).
- * Considering that PV guests do not want the *native SWIOTLB* but
- * only Xen SWIOTLB it is not useful to us so set no_iommu=1 here.
- */
- if (max_pfn > MAX_DMA32_PFN)
- no_iommu = 1;
-#endif
- return xen_swiotlb;
-}
-
-static void __init pci_xen_swiotlb_init(void)
-{
- if (xen_swiotlb) {
- xen_swiotlb_init_early();
- dma_ops = &xen_swiotlb_dma_ops;
-
-#ifdef CONFIG_PCI
- /* Make sure ACS will be enabled */
- pci_request_acs();
-#endif
- }
-}
-
-int pci_xen_swiotlb_init_late(void)
-{
- int rc;
-
- if (xen_swiotlb)
- return 0;
-
- rc = xen_swiotlb_init();
- if (rc)
- return rc;
-
- dma_ops = &xen_swiotlb_dma_ops;
-#ifdef CONFIG_PCI
- /* Make sure ACS will be enabled */
- pci_request_acs();
-#endif
-
- return 0;
-}
-EXPORT_SYMBOL_GPL(pci_xen_swiotlb_init_late);
-
-IOMMU_INIT_FINISH(pci_xen_swiotlb_detect,
- NULL,
- pci_xen_swiotlb_init,
- NULL);
diff --git a/drivers/iommu/amd/init.c b/drivers/iommu/amd/init.c
index b10fb52ea4428..721300cf90207 100644
--- a/drivers/iommu/amd/init.c
+++ b/drivers/iommu/amd/init.c
@@ -27,7 +27,6 @@
#include <asm/apic.h>
#include <asm/gart.h>
#include <asm/x86_init.h>
-#include <asm/iommu_table.h>
#include <asm/io_apic.h>
#include <asm/irq_remapping.h>
#include <asm/set_memory.h>
@@ -3237,11 +3236,6 @@ __setup("ivrs_ioapic", parse_ivrs_ioapic);
__setup("ivrs_hpet", parse_ivrs_hpet);
__setup("ivrs_acpihid", parse_ivrs_acpihid);
-IOMMU_INIT_FINISH(amd_iommu_detect,
- gart_iommu_hole_init,
- NULL,
- NULL);
-
bool amd_iommu_v2_supported(void)
{
return amd_iommu_v2_present;
diff --git a/drivers/iommu/amd/iommu.c b/drivers/iommu/amd/iommu.c
index 461f1844ed1fb..541a7b8315a12 100644
--- a/drivers/iommu/amd/iommu.c
+++ b/drivers/iommu/amd/iommu.c
@@ -1834,7 +1834,10 @@ void amd_iommu_domain_update(struct protection_domain *domain)
static void __init amd_iommu_init_dma_ops(void)
{
- swiotlb = (iommu_default_passthrough() || sme_me_mask) ? 1 : 0;
+ if (iommu_default_passthrough() || sme_me_mask)
+ x86_swiotlb_enable = true;
+ else
+ x86_swiotlb_enable = false;
}
int __init amd_iommu_init_api(void)
diff --git a/drivers/iommu/intel/dmar.c b/drivers/iommu/intel/dmar.c
index 915bff76fe965..29bee4b210c5b 100644
--- a/drivers/iommu/intel/dmar.c
+++ b/drivers/iommu/intel/dmar.c
@@ -30,7 +30,6 @@
#include <linux/numa.h>
#include <linux/limits.h>
#include <asm/irq_remapping.h>
-#include <asm/iommu_table.h>
#include <trace/events/intel_iommu.h>
#include "../irq_remapping.h"
@@ -913,7 +912,7 @@ dmar_validate_one_drhd(struct acpi_dmar_header *entry, void *arg)
return 0;
}
-int __init detect_intel_iommu(void)
+void __init detect_intel_iommu(void)
{
int ret;
struct dmar_res_callback validate_drhd_cb = {
@@ -946,8 +945,6 @@ int __init detect_intel_iommu(void)
dmar_tbl = NULL;
}
up_write(&dmar_global_lock);
-
- return ret ? ret : 1;
}
static void unmap_iommu(struct intel_iommu *iommu)
@@ -2165,7 +2162,6 @@ static int __init dmar_free_unused_resources(void)
}
late_initcall(dmar_free_unused_resources);
-IOMMU_INIT_POST(detect_intel_iommu);
/*
* DMAR Hotplug Support
diff --git a/include/linux/dmar.h b/include/linux/dmar.h
index 45e903d847335..cbd714a198a0a 100644
--- a/include/linux/dmar.h
+++ b/include/linux/dmar.h
@@ -121,7 +121,7 @@ extern int dmar_remove_dev_scope(struct dmar_pci_notify_info *info,
u16 segment, struct dmar_dev_scope *devices,
int count);
/* Intel IOMMU detection */
-extern int detect_intel_iommu(void);
+void detect_intel_iommu(void);
extern int enable_drhd_fault_handling(void);
extern int dmar_device_add(acpi_handle handle);
extern int dmar_device_remove(acpi_handle handle);
@@ -197,6 +197,10 @@ static inline bool dmar_platform_optin(void)
return false;
}
+static inline void detect_intel_iommu(void)
+{
+}
+
#endif /* CONFIG_DMAR_TABLE */
struct irte {
--
2.30.2
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply related [flat|nested] 118+ messages in thread
* [PATCH 07/11] x86: remove the IOMMU table infrastructure
@ 2022-02-27 14:30 ` Christoph Hellwig
0 siblings, 0 replies; 118+ messages in thread
From: Christoph Hellwig @ 2022-02-27 14:30 UTC (permalink / raw)
To: iommu
Cc: x86, Anshuman Khandual, Konrad Rzeszutek Wilk,
Stefano Stabellini, Boris Ostrovsky, Juergen Gross, Joerg Roedel,
David Woodhouse, Lu Baolu, Robin Murphy, linux-arm-kernel,
xen-devel, linux-ia64, linux-mips, linuxppc-dev, linux-riscv,
linux-s390, linux-hyperv, tboot-devel, linux-pci
The IOMMU table tries to separate the different IOMMUs into different
backends, but actually requires various cross calls.
Rewrite the code to do the generic swiotlb/swiotlb-xen setup directly
in pci-dma.c and then just call into the IOMMU drivers.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
arch/ia64/include/asm/iommu_table.h | 7 --
arch/x86/include/asm/dma-mapping.h | 1 -
arch/x86/include/asm/gart.h | 5 +-
arch/x86/include/asm/iommu.h | 6 ++
arch/x86/include/asm/iommu_table.h | 102 -----------------------
arch/x86/include/asm/swiotlb.h | 30 -------
arch/x86/include/asm/xen/swiotlb-xen.h | 2 -
arch/x86/kernel/Makefile | 2 -
arch/x86/kernel/amd_gart_64.c | 5 +-
arch/x86/kernel/aperture_64.c | 14 ++--
arch/x86/kernel/pci-dma.c | 111 ++++++++++++++++++++-----
arch/x86/kernel/pci-iommu_table.c | 77 -----------------
arch/x86/kernel/pci-swiotlb.c | 77 -----------------
arch/x86/kernel/tboot.c | 1 -
arch/x86/kernel/vmlinux.lds.S | 12 ---
arch/x86/xen/Makefile | 2 -
arch/x86/xen/pci-swiotlb-xen.c | 96 ---------------------
drivers/iommu/amd/init.c | 6 --
drivers/iommu/amd/iommu.c | 5 +-
drivers/iommu/intel/dmar.c | 6 +-
include/linux/dmar.h | 6 +-
21 files changed, 114 insertions(+), 459 deletions(-)
delete mode 100644 arch/ia64/include/asm/iommu_table.h
delete mode 100644 arch/x86/include/asm/iommu_table.h
delete mode 100644 arch/x86/include/asm/swiotlb.h
delete mode 100644 arch/x86/kernel/pci-iommu_table.c
delete mode 100644 arch/x86/kernel/pci-swiotlb.c
delete mode 100644 arch/x86/xen/pci-swiotlb-xen.c
diff --git a/arch/ia64/include/asm/iommu_table.h b/arch/ia64/include/asm/iommu_table.h
deleted file mode 100644
index cc96116ac276a..0000000000000
--- a/arch/ia64/include/asm/iommu_table.h
+++ /dev/null
@@ -1,7 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0 */
-#ifndef _ASM_IA64_IOMMU_TABLE_H
-#define _ASM_IA64_IOMMU_TABLE_H
-
-#define IOMMU_INIT_POST(_detect)
-
-#endif /* _ASM_IA64_IOMMU_TABLE_H */
diff --git a/arch/x86/include/asm/dma-mapping.h b/arch/x86/include/asm/dma-mapping.h
index bb1654fe0ce74..256fd8115223d 100644
--- a/arch/x86/include/asm/dma-mapping.h
+++ b/arch/x86/include/asm/dma-mapping.h
@@ -9,7 +9,6 @@
#include <linux/scatterlist.h>
#include <asm/io.h>
-#include <asm/swiotlb.h>
extern int iommu_merge;
extern int panic_on_overflow;
diff --git a/arch/x86/include/asm/gart.h b/arch/x86/include/asm/gart.h
index 3185565743459..5af8088a10df6 100644
--- a/arch/x86/include/asm/gart.h
+++ b/arch/x86/include/asm/gart.h
@@ -38,7 +38,7 @@ extern int gart_iommu_aperture_disabled;
extern void early_gart_iommu_check(void);
extern int gart_iommu_init(void);
extern void __init gart_parse_options(char *);
-extern int gart_iommu_hole_init(void);
+void gart_iommu_hole_init(void);
#else
#define gart_iommu_aperture 0
@@ -51,9 +51,8 @@ static inline void early_gart_iommu_check(void)
static inline void gart_parse_options(char *options)
{
}
-static inline int gart_iommu_hole_init(void)
+static inline void gart_iommu_hole_init(void)
{
- return -ENODEV;
}
#endif
diff --git a/arch/x86/include/asm/iommu.h b/arch/x86/include/asm/iommu.h
index bf1ed2ddc74bd..dba89ed40d38d 100644
--- a/arch/x86/include/asm/iommu.h
+++ b/arch/x86/include/asm/iommu.h
@@ -9,6 +9,12 @@
extern int force_iommu, no_iommu;
extern int iommu_detected;
+#ifdef CONFIG_SWIOTLB
+extern bool x86_swiotlb_enable;
+#else
+#define x86_swiotlb_enable false
+#endif
+
/* 10 seconds */
#define DMAR_OPERATION_TIMEOUT ((cycles_t) tsc_khz*10*1000)
diff --git a/arch/x86/include/asm/iommu_table.h b/arch/x86/include/asm/iommu_table.h
deleted file mode 100644
index 1fb3fd1a83c25..0000000000000
--- a/arch/x86/include/asm/iommu_table.h
+++ /dev/null
@@ -1,102 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0 */
-#ifndef _ASM_X86_IOMMU_TABLE_H
-#define _ASM_X86_IOMMU_TABLE_H
-
-#include <asm/swiotlb.h>
-
-/*
- * History lesson:
- * The execution chain of IOMMUs in 2.6.36 looks as so:
- *
- * [xen-swiotlb]
- * |
- * +----[swiotlb *]--+
- * / | \
- * / | \
- * [GART] [Calgary] [Intel VT-d]
- * /
- * /
- * [AMD-Vi]
- *
- * *: if SWIOTLB detected 'iommu=soft'/'swiotlb=force' it would skip
- * over the rest of IOMMUs and unconditionally initialize the SWIOTLB.
- * Also it would surreptitiously initialize set the swiotlb=1 if there were
- * more than 4GB and if the user did not pass in 'iommu=off'. The swiotlb
- * flag would be turned off by all IOMMUs except the Calgary one.
- *
- * The IOMMU_INIT* macros allow a similar tree (or more complex if desired)
- * to be built by defining who we depend on.
- *
- * And all that needs to be done is to use one of the macros in the IOMMU
- * and the pci-dma.c will take care of the rest.
- */
-
-struct iommu_table_entry {
- initcall_t detect;
- initcall_t depend;
- void (*early_init)(void); /* No memory allocate available. */
- void (*late_init)(void); /* Yes, can allocate memory. */
-#define IOMMU_FINISH_IF_DETECTED (1<<0)
-#define IOMMU_DETECTED (1<<1)
- int flags;
-};
-/*
- * Macro fills out an entry in the .iommu_table that is equivalent
- * to the fields that 'struct iommu_table_entry' has. The entries
- * that are put in the .iommu_table section are not put in any order
- * hence during boot-time we will have to resort them based on
- * dependency. */
-
-
-#define __IOMMU_INIT(_detect, _depend, _early_init, _late_init, _finish)\
- static const struct iommu_table_entry \
- __iommu_entry_##_detect __used \
- __attribute__ ((unused, __section__(".iommu_table"), \
- aligned((sizeof(void *))))) \
- = {_detect, _depend, _early_init, _late_init, \
- _finish ? IOMMU_FINISH_IF_DETECTED : 0}
-/*
- * The simplest IOMMU definition. Provide the detection routine
- * and it will be run after the SWIOTLB and the other IOMMUs
- * that utilize this macro. If the IOMMU is detected (ie, the
- * detect routine returns a positive value), the other IOMMUs
- * are also checked. You can use IOMMU_INIT_POST_FINISH if you prefer
- * to stop detecting the other IOMMUs after yours has been detected.
- */
-#define IOMMU_INIT_POST(_detect) \
- __IOMMU_INIT(_detect, pci_swiotlb_detect_4gb, NULL, NULL, 0)
-
-#define IOMMU_INIT_POST_FINISH(detect) \
- __IOMMU_INIT(_detect, pci_swiotlb_detect_4gb, NULL, NULL, 1)
-
-/*
- * A more sophisticated version of IOMMU_INIT. This variant requires:
- * a). A detection routine function.
- * b). The name of the detection routine we depend on to get called
- * before us.
- * c). The init routine which gets called if the detection routine
- * returns a positive value from the pci_iommu_alloc. This means
- * no presence of a memory allocator.
- * d). Similar to the 'init', except that this gets called from pci_iommu_init
- * where we do have a memory allocator.
- *
- * The standard IOMMU_INIT differs from the IOMMU_INIT_FINISH variant
- * in that the former will continue detecting other IOMMUs in the call
- * list after the detection routine returns a positive number, while the
- * latter will stop the execution chain upon first successful detection.
- * Both variants will still call the 'init' and 'late_init' functions if
- * they are set.
- */
-#define IOMMU_INIT_FINISH(_detect, _depend, _init, _late_init) \
- __IOMMU_INIT(_detect, _depend, _init, _late_init, 1)
-
-#define IOMMU_INIT(_detect, _depend, _init, _late_init) \
- __IOMMU_INIT(_detect, _depend, _init, _late_init, 0)
-
-void sort_iommu_table(struct iommu_table_entry *start,
- struct iommu_table_entry *finish);
-
-void check_iommu_entries(struct iommu_table_entry *start,
- struct iommu_table_entry *finish);
-
-#endif /* _ASM_X86_IOMMU_TABLE_H */
diff --git a/arch/x86/include/asm/swiotlb.h b/arch/x86/include/asm/swiotlb.h
deleted file mode 100644
index ff6c92eff035a..0000000000000
--- a/arch/x86/include/asm/swiotlb.h
+++ /dev/null
@@ -1,30 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0 */
-#ifndef _ASM_X86_SWIOTLB_H
-#define _ASM_X86_SWIOTLB_H
-
-#include <linux/swiotlb.h>
-
-#ifdef CONFIG_SWIOTLB
-extern int swiotlb;
-extern int __init pci_swiotlb_detect_override(void);
-extern int __init pci_swiotlb_detect_4gb(void);
-extern void __init pci_swiotlb_init(void);
-extern void __init pci_swiotlb_late_init(void);
-#else
-#define swiotlb 0
-static inline int pci_swiotlb_detect_override(void)
-{
- return 0;
-}
-static inline int pci_swiotlb_detect_4gb(void)
-{
- return 0;
-}
-static inline void pci_swiotlb_init(void)
-{
-}
-static inline void pci_swiotlb_late_init(void)
-{
-}
-#endif
-#endif /* _ASM_X86_SWIOTLB_H */
diff --git a/arch/x86/include/asm/xen/swiotlb-xen.h b/arch/x86/include/asm/xen/swiotlb-xen.h
index 66b4ddde77430..e5a90b42e4dde 100644
--- a/arch/x86/include/asm/xen/swiotlb-xen.h
+++ b/arch/x86/include/asm/xen/swiotlb-xen.h
@@ -3,10 +3,8 @@
#define _ASM_X86_SWIOTLB_XEN_H
#ifdef CONFIG_SWIOTLB_XEN
-extern int __init pci_xen_swiotlb_detect(void);
extern int pci_xen_swiotlb_init_late(void);
#else
-#define pci_xen_swiotlb_detect NULL
static inline int pci_xen_swiotlb_init_late(void) { return -ENXIO; }
#endif
diff --git a/arch/x86/kernel/Makefile b/arch/x86/kernel/Makefile
index 6aef9ee28a394..2851d4f0aa0d2 100644
--- a/arch/x86/kernel/Makefile
+++ b/arch/x86/kernel/Makefile
@@ -71,7 +71,6 @@ obj-y += bootflag.o e820.o
obj-y += pci-dma.o quirks.o topology.o kdebugfs.o
obj-y += alternative.o i8253.o hw_breakpoint.o
obj-y += tsc.o tsc_msr.o io_delay.o rtc.o
-obj-y += pci-iommu_table.o
obj-y += resource.o
obj-y += irqflags.o
obj-y += static_call.o
@@ -136,7 +135,6 @@ obj-$(CONFIG_PCSPKR_PLATFORM) += pcspeaker.o
obj-$(CONFIG_X86_CHECK_BIOS_CORRUPTION) += check.o
-obj-$(CONFIG_SWIOTLB) += pci-swiotlb.o
obj-$(CONFIG_OF) += devicetree.o
obj-$(CONFIG_UPROBES) += uprobes.o
diff --git a/arch/x86/kernel/amd_gart_64.c b/arch/x86/kernel/amd_gart_64.c
index ed837383de5c8..194d54eed5376 100644
--- a/arch/x86/kernel/amd_gart_64.c
+++ b/arch/x86/kernel/amd_gart_64.c
@@ -38,11 +38,9 @@
#include <asm/iommu.h>
#include <asm/gart.h>
#include <asm/set_memory.h>
-#include <asm/swiotlb.h>
#include <asm/dma.h>
#include <asm/amd_nb.h>
#include <asm/x86_init.h>
-#include <asm/iommu_table.h>
static unsigned long iommu_bus_base; /* GART remapping area (physical) */
static unsigned long iommu_size; /* size of remapping area bytes */
@@ -808,7 +806,7 @@ int __init gart_iommu_init(void)
flush_gart();
dma_ops = &gart_dma_ops;
x86_platform.iommu_shutdown = gart_iommu_shutdown;
- swiotlb = 0;
+ x86_swiotlb_enable = false;
return 0;
}
@@ -842,4 +840,3 @@ void __init gart_parse_options(char *p)
}
}
}
-IOMMU_INIT_POST(gart_iommu_hole_init);
diff --git a/arch/x86/kernel/aperture_64.c b/arch/x86/kernel/aperture_64.c
index af3ba08b684b5..7a5630d904b23 100644
--- a/arch/x86/kernel/aperture_64.c
+++ b/arch/x86/kernel/aperture_64.c
@@ -392,7 +392,7 @@ void __init early_gart_iommu_check(void)
static int __initdata printed_gart_size_msg;
-int __init gart_iommu_hole_init(void)
+void __init gart_iommu_hole_init(void)
{
u32 agp_aper_base = 0, agp_aper_order = 0;
u32 aper_size, aper_alloc = 0, aper_order = 0, last_aper_order = 0;
@@ -401,11 +401,11 @@ int __init gart_iommu_hole_init(void)
int i, node;
if (!amd_gart_present())
- return -ENODEV;
+ return;
if (gart_iommu_aperture_disabled || !fix_aperture ||
!early_pci_allowed())
- return -ENODEV;
+ return;
pr_info("Checking aperture...\n");
@@ -491,10 +491,8 @@ int __init gart_iommu_hole_init(void)
* and fixed up the northbridge
*/
exclude_from_core(last_aper_base, last_aper_order);
-
- return 1;
}
- return 0;
+ return;
}
if (!fallback_aper_force) {
@@ -527,7 +525,7 @@ int __init gart_iommu_hole_init(void)
panic("Not enough memory for aperture");
}
} else {
- return 0;
+ return;
}
/*
@@ -561,6 +559,4 @@ int __init gart_iommu_hole_init(void)
}
set_up_gart_resume(aper_order, aper_alloc);
-
- return 1;
}
diff --git a/arch/x86/kernel/pci-dma.c b/arch/x86/kernel/pci-dma.c
index de234e7a8962e..17bcac942ac49 100644
--- a/arch/x86/kernel/pci-dma.c
+++ b/arch/x86/kernel/pci-dma.c
@@ -7,13 +7,16 @@
#include <linux/memblock.h>
#include <linux/gfp.h>
#include <linux/pci.h>
+#include <linux/amd-iommu.h>
#include <asm/proto.h>
#include <asm/dma.h>
#include <asm/iommu.h>
#include <asm/gart.h>
#include <asm/x86_init.h>
-#include <asm/iommu_table.h>
+
+#include <xen/xen.h>
+#include <xen/swiotlb-xen.h>
static bool disable_dac_quirk __read_mostly;
@@ -34,24 +37,87 @@ int no_iommu __read_mostly;
/* Set this to 1 if there is a HW IOMMU in the system */
int iommu_detected __read_mostly = 0;
-extern struct iommu_table_entry __iommu_table[], __iommu_table_end[];
+#ifdef CONFIG_SWIOTLB
+bool x86_swiotlb_enable;
+
+/*
+ * If 4GB or more detected (and iommu=off not set) or if SME is active
+ * then set swiotlb to 1 and return 1.
+ */
+static void __init pci_swiotlb_detect_4gb(void)
+{
+ /* don't initialize swiotlb if iommu=off (no_iommu=1) */
+ if (!no_iommu && max_possible_pfn > MAX_DMA32_PFN)
+ x86_swiotlb_enable = true;
+
+ /*
+ * Set swiotlb to 1 so that bounce buffers are allocated and used for
+ * devices that can't support DMA to encrypted memory.
+ */
+ if (cc_platform_has(CC_ATTR_HOST_MEM_ENCRYPT))
+ x86_swiotlb_enable = true;
+
+ if (swiotlb_force == SWIOTLB_FORCE)
+ x86_swiotlb_enable = true;
+}
+#else
+static inline void __init pci_swiotlb_detect_4gb(void)
+{
+}
+#endif /* CONFIG_SWIOTLB */
+
+#ifdef CONFIG_SWIOTLB_XEN
+static bool xen_swiotlb;
+
+static void __init pci_xen_swiotlb_init(void)
+{
+ if (!xen_initial_domain() && !x86_swiotlb_enable &&
+ swiotlb_force != SWIOTLB_FORCE)
+ return;
+ x86_swiotlb_enable = false;
+ xen_swiotlb = true;
+ xen_swiotlb_init_early();
+ dma_ops = &xen_swiotlb_dma_ops;
+ if (IS_ENABLED(CONFIG_PCI))
+ pci_request_acs();
+}
+
+int pci_xen_swiotlb_init_late(void)
+{
+ int rc;
+
+ if (xen_swiotlb)
+ return 0;
+
+ rc = xen_swiotlb_init();
+ if (rc)
+ return rc;
+
+ /* XXX: this switches the dma ops under live devices! */
+ dma_ops = &xen_swiotlb_dma_ops;
+ if (IS_ENABLED(CONFIG_PCI))
+ pci_request_acs();
+ return 0;
+}
+EXPORT_SYMBOL_GPL(pci_xen_swiotlb_init_late);
+#else
+static inline void __init pci_xen_swiotlb_init(void)
+{
+}
+#endif /* CONFIG_SWIOTLB_XEN */
void __init pci_iommu_alloc(void)
{
- struct iommu_table_entry *p;
-
- sort_iommu_table(__iommu_table, __iommu_table_end);
- check_iommu_entries(__iommu_table, __iommu_table_end);
-
- for (p = __iommu_table; p < __iommu_table_end; p++) {
- if (p && p->detect && p->detect() > 0) {
- p->flags |= IOMMU_DETECTED;
- if (p->early_init)
- p->early_init();
- if (p->flags & IOMMU_FINISH_IF_DETECTED)
- break;
- }
+ if (xen_pv_domain()) {
+ pci_xen_swiotlb_init();
+ return;
}
+ pci_swiotlb_detect_4gb();
+ gart_iommu_hole_init();
+ amd_iommu_detect();
+ detect_intel_iommu();
+ if (x86_swiotlb_enable)
+ swiotlb_init(0);
}
/*
@@ -102,7 +168,7 @@ static __init int iommu_setup(char *p)
}
#ifdef CONFIG_SWIOTLB
if (!strncmp(p, "soft", 4))
- swiotlb = 1;
+ x86_swiotlb_enable = 1;
#endif
if (!strncmp(p, "pt", 2))
iommu_set_default_passthrough(true);
@@ -121,14 +187,17 @@ early_param("iommu", iommu_setup);
static int __init pci_iommu_init(void)
{
- struct iommu_table_entry *p;
-
x86_init.iommu.iommu_init();
- for (p = __iommu_table; p < __iommu_table_end; p++) {
- if (p && (p->flags & IOMMU_DETECTED) && p->late_init)
- p->late_init();
+#ifdef CONFIG_SWIOTLB
+ /* An IOMMU turned us off. */
+ if (x86_swiotlb_enable) {
+ pr_info("PCI-DMA: Using software bounce buffering for IO (SWIOTLB)\n");
+ swiotlb_print_info();
+ } else {
+ swiotlb_exit();
}
+#endif
return 0;
}
diff --git a/arch/x86/kernel/pci-iommu_table.c b/arch/x86/kernel/pci-iommu_table.c
deleted file mode 100644
index 42e92ec62973b..0000000000000
--- a/arch/x86/kernel/pci-iommu_table.c
+++ /dev/null
@@ -1,77 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0
-#include <linux/dma-mapping.h>
-#include <asm/iommu_table.h>
-#include <linux/string.h>
-#include <linux/kallsyms.h>
-
-static struct iommu_table_entry * __init
-find_dependents_of(struct iommu_table_entry *start,
- struct iommu_table_entry *finish,
- struct iommu_table_entry *q)
-{
- struct iommu_table_entry *p;
-
- if (!q)
- return NULL;
-
- for (p = start; p < finish; p++)
- if (p->detect == q->depend)
- return p;
-
- return NULL;
-}
-
-
-void __init sort_iommu_table(struct iommu_table_entry *start,
- struct iommu_table_entry *finish) {
-
- struct iommu_table_entry *p, *q, tmp;
-
- for (p = start; p < finish; p++) {
-again:
- q = find_dependents_of(start, finish, p);
- /* We are bit sneaky here. We use the memory address to figure
- * out if the node we depend on is past our point, if so, swap.
- */
- if (q > p) {
- tmp = *p;
- memmove(p, q, sizeof(*p));
- *q = tmp;
- goto again;
- }
- }
-
-}
-
-#ifdef DEBUG
-void __init check_iommu_entries(struct iommu_table_entry *start,
- struct iommu_table_entry *finish)
-{
- struct iommu_table_entry *p, *q, *x;
-
- /* Simple cyclic dependency checker. */
- for (p = start; p < finish; p++) {
- q = find_dependents_of(start, finish, p);
- x = find_dependents_of(start, finish, q);
- if (p == x) {
- printk(KERN_ERR "CYCLIC DEPENDENCY FOUND! %pS depends on %pS and vice-versa. BREAKING IT.\n",
- p->detect, q->detect);
- /* Heavy handed way..*/
- x->depend = NULL;
- }
- }
-
- for (p = start; p < finish; p++) {
- q = find_dependents_of(p, finish, p);
- if (q && q > p) {
- printk(KERN_ERR "EXECUTION ORDER INVALID! %pS should be called before %pS!\n",
- p->detect, q->detect);
- }
- }
-}
-#else
-void __init check_iommu_entries(struct iommu_table_entry *start,
- struct iommu_table_entry *finish)
-{
-}
-#endif
diff --git a/arch/x86/kernel/pci-swiotlb.c b/arch/x86/kernel/pci-swiotlb.c
deleted file mode 100644
index 814ab46a0dada..0000000000000
--- a/arch/x86/kernel/pci-swiotlb.c
+++ /dev/null
@@ -1,77 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0
-
-#include <linux/pci.h>
-#include <linux/cache.h>
-#include <linux/init.h>
-#include <linux/swiotlb.h>
-#include <linux/memblock.h>
-#include <linux/dma-direct.h>
-#include <linux/cc_platform.h>
-
-#include <asm/iommu.h>
-#include <asm/swiotlb.h>
-#include <asm/dma.h>
-#include <asm/xen/swiotlb-xen.h>
-#include <asm/iommu_table.h>
-
-int swiotlb __read_mostly;
-
-/*
- * pci_swiotlb_detect_override - set swiotlb to 1 if necessary
- *
- * This returns non-zero if we are forced to use swiotlb (by the boot
- * option).
- */
-int __init pci_swiotlb_detect_override(void)
-{
- if (swiotlb_force == SWIOTLB_FORCE)
- swiotlb = 1;
-
- return swiotlb;
-}
-IOMMU_INIT_FINISH(pci_swiotlb_detect_override,
- pci_xen_swiotlb_detect,
- pci_swiotlb_init,
- pci_swiotlb_late_init);
-
-/*
- * If 4GB or more detected (and iommu=off not set) or if SME is active
- * then set swiotlb to 1 and return 1.
- */
-int __init pci_swiotlb_detect_4gb(void)
-{
- /* don't initialize swiotlb if iommu=off (no_iommu=1) */
- if (!no_iommu && max_possible_pfn > MAX_DMA32_PFN)
- swiotlb = 1;
-
- /*
- * Set swiotlb to 1 so that bounce buffers are allocated and used for
- * devices that can't support DMA to encrypted memory.
- */
- if (cc_platform_has(CC_ATTR_HOST_MEM_ENCRYPT))
- swiotlb = 1;
-
- return swiotlb;
-}
-IOMMU_INIT(pci_swiotlb_detect_4gb,
- pci_swiotlb_detect_override,
- pci_swiotlb_init,
- pci_swiotlb_late_init);
-
-void __init pci_swiotlb_init(void)
-{
- if (swiotlb)
- swiotlb_init(0);
-}
-
-void __init pci_swiotlb_late_init(void)
-{
- /* An IOMMU turned us off. */
- if (!swiotlb)
- swiotlb_exit();
- else {
- printk(KERN_INFO "PCI-DMA: "
- "Using software bounce buffering for IO (SWIOTLB)\n");
- swiotlb_print_info();
- }
-}
diff --git a/arch/x86/kernel/tboot.c b/arch/x86/kernel/tboot.c
index f9af561c3cd4f..0c1154a1c4032 100644
--- a/arch/x86/kernel/tboot.c
+++ b/arch/x86/kernel/tboot.c
@@ -24,7 +24,6 @@
#include <asm/processor.h>
#include <asm/bootparam.h>
#include <asm/pgalloc.h>
-#include <asm/swiotlb.h>
#include <asm/fixmap.h>
#include <asm/proto.h>
#include <asm/setup.h>
diff --git a/arch/x86/kernel/vmlinux.lds.S b/arch/x86/kernel/vmlinux.lds.S
index 27f830345b6f0..bbe910c15b293 100644
--- a/arch/x86/kernel/vmlinux.lds.S
+++ b/arch/x86/kernel/vmlinux.lds.S
@@ -306,18 +306,6 @@ SECTIONS
*(.altinstr_replacement)
}
- /*
- * struct iommu_table_entry entries are injected in this section.
- * It is an array of IOMMUs which during run time gets sorted depending
- * on its dependency order. After rootfs_initcall is complete
- * this section can be safely removed.
- */
- .iommu_table : AT(ADDR(.iommu_table) - LOAD_OFFSET) {
- __iommu_table = .;
- *(.iommu_table)
- __iommu_table_end = .;
- }
-
. = ALIGN(8);
.apicdrivers : AT(ADDR(.apicdrivers) - LOAD_OFFSET) {
__apicdrivers = .;
diff --git a/arch/x86/xen/Makefile b/arch/x86/xen/Makefile
index 4953260e281c3..3c5b52fbe4a7f 100644
--- a/arch/x86/xen/Makefile
+++ b/arch/x86/xen/Makefile
@@ -47,6 +47,4 @@ obj-$(CONFIG_XEN_DEBUG_FS) += debugfs.o
obj-$(CONFIG_XEN_PV_DOM0) += vga.o
-obj-$(CONFIG_SWIOTLB_XEN) += pci-swiotlb-xen.o
-
obj-$(CONFIG_XEN_EFI) += efi.o
diff --git a/arch/x86/xen/pci-swiotlb-xen.c b/arch/x86/xen/pci-swiotlb-xen.c
deleted file mode 100644
index 46df59aeaa06a..0000000000000
--- a/arch/x86/xen/pci-swiotlb-xen.c
+++ /dev/null
@@ -1,96 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0
-
-/* Glue code to lib/swiotlb-xen.c */
-
-#include <linux/dma-map-ops.h>
-#include <linux/pci.h>
-#include <xen/swiotlb-xen.h>
-
-#include <asm/xen/hypervisor.h>
-#include <xen/xen.h>
-#include <asm/iommu_table.h>
-
-
-#include <asm/xen/swiotlb-xen.h>
-#ifdef CONFIG_X86_64
-#include <asm/iommu.h>
-#include <asm/dma.h>
-#endif
-#include <linux/export.h>
-
-static int xen_swiotlb __read_mostly;
-
-/*
- * pci_xen_swiotlb_detect - set xen_swiotlb to 1 if necessary
- *
- * This returns non-zero if we are forced to use xen_swiotlb (by the boot
- * option).
- */
-int __init pci_xen_swiotlb_detect(void)
-{
-
- if (!xen_pv_domain())
- return 0;
-
- /* If running as PV guest, either iommu=soft, or swiotlb=force will
- * activate this IOMMU. If running as PV privileged, activate it
- * irregardless.
- */
- if (xen_initial_domain() || swiotlb || swiotlb_force == SWIOTLB_FORCE)
- xen_swiotlb = 1;
-
- /* If we are running under Xen, we MUST disable the native SWIOTLB.
- * Don't worry about swiotlb_force flag activating the native, as
- * the 'swiotlb' flag is the only one turning it on. */
- swiotlb = 0;
-
-#ifdef CONFIG_X86_64
- /* pci_swiotlb_detect_4gb turns on native SWIOTLB if no_iommu == 0
- * (so no iommu=X command line over-writes).
- * Considering that PV guests do not want the *native SWIOTLB* but
- * only Xen SWIOTLB it is not useful to us so set no_iommu=1 here.
- */
- if (max_pfn > MAX_DMA32_PFN)
- no_iommu = 1;
-#endif
- return xen_swiotlb;
-}
-
-static void __init pci_xen_swiotlb_init(void)
-{
- if (xen_swiotlb) {
- xen_swiotlb_init_early();
- dma_ops = &xen_swiotlb_dma_ops;
-
-#ifdef CONFIG_PCI
- /* Make sure ACS will be enabled */
- pci_request_acs();
-#endif
- }
-}
-
-int pci_xen_swiotlb_init_late(void)
-{
- int rc;
-
- if (xen_swiotlb)
- return 0;
-
- rc = xen_swiotlb_init();
- if (rc)
- return rc;
-
- dma_ops = &xen_swiotlb_dma_ops;
-#ifdef CONFIG_PCI
- /* Make sure ACS will be enabled */
- pci_request_acs();
-#endif
-
- return 0;
-}
-EXPORT_SYMBOL_GPL(pci_xen_swiotlb_init_late);
-
-IOMMU_INIT_FINISH(pci_xen_swiotlb_detect,
- NULL,
- pci_xen_swiotlb_init,
- NULL);
diff --git a/drivers/iommu/amd/init.c b/drivers/iommu/amd/init.c
index b10fb52ea4428..721300cf90207 100644
--- a/drivers/iommu/amd/init.c
+++ b/drivers/iommu/amd/init.c
@@ -27,7 +27,6 @@
#include <asm/apic.h>
#include <asm/gart.h>
#include <asm/x86_init.h>
-#include <asm/iommu_table.h>
#include <asm/io_apic.h>
#include <asm/irq_remapping.h>
#include <asm/set_memory.h>
@@ -3237,11 +3236,6 @@ __setup("ivrs_ioapic", parse_ivrs_ioapic);
__setup("ivrs_hpet", parse_ivrs_hpet);
__setup("ivrs_acpihid", parse_ivrs_acpihid);
-IOMMU_INIT_FINISH(amd_iommu_detect,
- gart_iommu_hole_init,
- NULL,
- NULL);
-
bool amd_iommu_v2_supported(void)
{
return amd_iommu_v2_present;
diff --git a/drivers/iommu/amd/iommu.c b/drivers/iommu/amd/iommu.c
index 461f1844ed1fb..541a7b8315a12 100644
--- a/drivers/iommu/amd/iommu.c
+++ b/drivers/iommu/amd/iommu.c
@@ -1834,7 +1834,10 @@ void amd_iommu_domain_update(struct protection_domain *domain)
static void __init amd_iommu_init_dma_ops(void)
{
- swiotlb = (iommu_default_passthrough() || sme_me_mask) ? 1 : 0;
+ if (iommu_default_passthrough() || sme_me_mask)
+ x86_swiotlb_enable = true;
+ else
+ x86_swiotlb_enable = false;
}
int __init amd_iommu_init_api(void)
diff --git a/drivers/iommu/intel/dmar.c b/drivers/iommu/intel/dmar.c
index 915bff76fe965..29bee4b210c5b 100644
--- a/drivers/iommu/intel/dmar.c
+++ b/drivers/iommu/intel/dmar.c
@@ -30,7 +30,6 @@
#include <linux/numa.h>
#include <linux/limits.h>
#include <asm/irq_remapping.h>
-#include <asm/iommu_table.h>
#include <trace/events/intel_iommu.h>
#include "../irq_remapping.h"
@@ -913,7 +912,7 @@ dmar_validate_one_drhd(struct acpi_dmar_header *entry, void *arg)
return 0;
}
-int __init detect_intel_iommu(void)
+void __init detect_intel_iommu(void)
{
int ret;
struct dmar_res_callback validate_drhd_cb = {
@@ -946,8 +945,6 @@ int __init detect_intel_iommu(void)
dmar_tbl = NULL;
}
up_write(&dmar_global_lock);
-
- return ret ? ret : 1;
}
static void unmap_iommu(struct intel_iommu *iommu)
@@ -2165,7 +2162,6 @@ static int __init dmar_free_unused_resources(void)
}
late_initcall(dmar_free_unused_resources);
-IOMMU_INIT_POST(detect_intel_iommu);
/*
* DMAR Hotplug Support
diff --git a/include/linux/dmar.h b/include/linux/dmar.h
index 45e903d847335..cbd714a198a0a 100644
--- a/include/linux/dmar.h
+++ b/include/linux/dmar.h
@@ -121,7 +121,7 @@ extern int dmar_remove_dev_scope(struct dmar_pci_notify_info *info,
u16 segment, struct dmar_dev_scope *devices,
int count);
/* Intel IOMMU detection */
-extern int detect_intel_iommu(void);
+void detect_intel_iommu(void);
extern int enable_drhd_fault_handling(void);
extern int dmar_device_add(acpi_handle handle);
extern int dmar_device_remove(acpi_handle handle);
@@ -197,6 +197,10 @@ static inline bool dmar_platform_optin(void)
return false;
}
+static inline void detect_intel_iommu(void)
+{
+}
+
#endif /* CONFIG_DMAR_TABLE */
struct irte {
--
2.30.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 118+ messages in thread
* [PATCH 07/11] x86: remove the IOMMU table infrastructure
@ 2022-02-27 14:30 ` Christoph Hellwig
0 siblings, 0 replies; 118+ messages in thread
From: Christoph Hellwig @ 2022-02-27 14:30 UTC (permalink / raw)
To: iommu
Cc: Juergen Gross, linux-s390, linux-hyperv, Stefano Stabellini,
linux-ia64, Anshuman Khandual, Konrad Rzeszutek Wilk,
Robin Murphy, Joerg Roedel, x86, linux-mips, linuxppc-dev,
tboot-devel, linux-pci, xen-devel, Boris Ostrovsky,
David Woodhouse, linux-riscv, linux-arm-kernel, Lu Baolu
The IOMMU table tries to separate the different IOMMUs into different
backends, but actually requires various cross calls.
Rewrite the code to do the generic swiotlb/swiotlb-xen setup directly
in pci-dma.c and then just call into the IOMMU drivers.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
arch/ia64/include/asm/iommu_table.h | 7 --
arch/x86/include/asm/dma-mapping.h | 1 -
arch/x86/include/asm/gart.h | 5 +-
arch/x86/include/asm/iommu.h | 6 ++
arch/x86/include/asm/iommu_table.h | 102 -----------------------
arch/x86/include/asm/swiotlb.h | 30 -------
arch/x86/include/asm/xen/swiotlb-xen.h | 2 -
arch/x86/kernel/Makefile | 2 -
arch/x86/kernel/amd_gart_64.c | 5 +-
arch/x86/kernel/aperture_64.c | 14 ++--
arch/x86/kernel/pci-dma.c | 111 ++++++++++++++++++++-----
arch/x86/kernel/pci-iommu_table.c | 77 -----------------
arch/x86/kernel/pci-swiotlb.c | 77 -----------------
arch/x86/kernel/tboot.c | 1 -
arch/x86/kernel/vmlinux.lds.S | 12 ---
arch/x86/xen/Makefile | 2 -
arch/x86/xen/pci-swiotlb-xen.c | 96 ---------------------
drivers/iommu/amd/init.c | 6 --
drivers/iommu/amd/iommu.c | 5 +-
drivers/iommu/intel/dmar.c | 6 +-
include/linux/dmar.h | 6 +-
21 files changed, 114 insertions(+), 459 deletions(-)
delete mode 100644 arch/ia64/include/asm/iommu_table.h
delete mode 100644 arch/x86/include/asm/iommu_table.h
delete mode 100644 arch/x86/include/asm/swiotlb.h
delete mode 100644 arch/x86/kernel/pci-iommu_table.c
delete mode 100644 arch/x86/kernel/pci-swiotlb.c
delete mode 100644 arch/x86/xen/pci-swiotlb-xen.c
diff --git a/arch/ia64/include/asm/iommu_table.h b/arch/ia64/include/asm/iommu_table.h
deleted file mode 100644
index cc96116ac276a..0000000000000
--- a/arch/ia64/include/asm/iommu_table.h
+++ /dev/null
@@ -1,7 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0 */
-#ifndef _ASM_IA64_IOMMU_TABLE_H
-#define _ASM_IA64_IOMMU_TABLE_H
-
-#define IOMMU_INIT_POST(_detect)
-
-#endif /* _ASM_IA64_IOMMU_TABLE_H */
diff --git a/arch/x86/include/asm/dma-mapping.h b/arch/x86/include/asm/dma-mapping.h
index bb1654fe0ce74..256fd8115223d 100644
--- a/arch/x86/include/asm/dma-mapping.h
+++ b/arch/x86/include/asm/dma-mapping.h
@@ -9,7 +9,6 @@
#include <linux/scatterlist.h>
#include <asm/io.h>
-#include <asm/swiotlb.h>
extern int iommu_merge;
extern int panic_on_overflow;
diff --git a/arch/x86/include/asm/gart.h b/arch/x86/include/asm/gart.h
index 3185565743459..5af8088a10df6 100644
--- a/arch/x86/include/asm/gart.h
+++ b/arch/x86/include/asm/gart.h
@@ -38,7 +38,7 @@ extern int gart_iommu_aperture_disabled;
extern void early_gart_iommu_check(void);
extern int gart_iommu_init(void);
extern void __init gart_parse_options(char *);
-extern int gart_iommu_hole_init(void);
+void gart_iommu_hole_init(void);
#else
#define gart_iommu_aperture 0
@@ -51,9 +51,8 @@ static inline void early_gart_iommu_check(void)
static inline void gart_parse_options(char *options)
{
}
-static inline int gart_iommu_hole_init(void)
+static inline void gart_iommu_hole_init(void)
{
- return -ENODEV;
}
#endif
diff --git a/arch/x86/include/asm/iommu.h b/arch/x86/include/asm/iommu.h
index bf1ed2ddc74bd..dba89ed40d38d 100644
--- a/arch/x86/include/asm/iommu.h
+++ b/arch/x86/include/asm/iommu.h
@@ -9,6 +9,12 @@
extern int force_iommu, no_iommu;
extern int iommu_detected;
+#ifdef CONFIG_SWIOTLB
+extern bool x86_swiotlb_enable;
+#else
+#define x86_swiotlb_enable false
+#endif
+
/* 10 seconds */
#define DMAR_OPERATION_TIMEOUT ((cycles_t) tsc_khz*10*1000)
diff --git a/arch/x86/include/asm/iommu_table.h b/arch/x86/include/asm/iommu_table.h
deleted file mode 100644
index 1fb3fd1a83c25..0000000000000
--- a/arch/x86/include/asm/iommu_table.h
+++ /dev/null
@@ -1,102 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0 */
-#ifndef _ASM_X86_IOMMU_TABLE_H
-#define _ASM_X86_IOMMU_TABLE_H
-
-#include <asm/swiotlb.h>
-
-/*
- * History lesson:
- * The execution chain of IOMMUs in 2.6.36 looks as so:
- *
- * [xen-swiotlb]
- * |
- * +----[swiotlb *]--+
- * / | \
- * / | \
- * [GART] [Calgary] [Intel VT-d]
- * /
- * /
- * [AMD-Vi]
- *
- * *: if SWIOTLB detected 'iommu=soft'/'swiotlb=force' it would skip
- * over the rest of IOMMUs and unconditionally initialize the SWIOTLB.
- * Also it would surreptitiously initialize set the swiotlb=1 if there were
- * more than 4GB and if the user did not pass in 'iommu=off'. The swiotlb
- * flag would be turned off by all IOMMUs except the Calgary one.
- *
- * The IOMMU_INIT* macros allow a similar tree (or more complex if desired)
- * to be built by defining who we depend on.
- *
- * And all that needs to be done is to use one of the macros in the IOMMU
- * and the pci-dma.c will take care of the rest.
- */
-
-struct iommu_table_entry {
- initcall_t detect;
- initcall_t depend;
- void (*early_init)(void); /* No memory allocate available. */
- void (*late_init)(void); /* Yes, can allocate memory. */
-#define IOMMU_FINISH_IF_DETECTED (1<<0)
-#define IOMMU_DETECTED (1<<1)
- int flags;
-};
-/*
- * Macro fills out an entry in the .iommu_table that is equivalent
- * to the fields that 'struct iommu_table_entry' has. The entries
- * that are put in the .iommu_table section are not put in any order
- * hence during boot-time we will have to resort them based on
- * dependency. */
-
-
-#define __IOMMU_INIT(_detect, _depend, _early_init, _late_init, _finish)\
- static const struct iommu_table_entry \
- __iommu_entry_##_detect __used \
- __attribute__ ((unused, __section__(".iommu_table"), \
- aligned((sizeof(void *))))) \
- = {_detect, _depend, _early_init, _late_init, \
- _finish ? IOMMU_FINISH_IF_DETECTED : 0}
-/*
- * The simplest IOMMU definition. Provide the detection routine
- * and it will be run after the SWIOTLB and the other IOMMUs
- * that utilize this macro. If the IOMMU is detected (ie, the
- * detect routine returns a positive value), the other IOMMUs
- * are also checked. You can use IOMMU_INIT_POST_FINISH if you prefer
- * to stop detecting the other IOMMUs after yours has been detected.
- */
-#define IOMMU_INIT_POST(_detect) \
- __IOMMU_INIT(_detect, pci_swiotlb_detect_4gb, NULL, NULL, 0)
-
-#define IOMMU_INIT_POST_FINISH(detect) \
- __IOMMU_INIT(_detect, pci_swiotlb_detect_4gb, NULL, NULL, 1)
-
-/*
- * A more sophisticated version of IOMMU_INIT. This variant requires:
- * a). A detection routine function.
- * b). The name of the detection routine we depend on to get called
- * before us.
- * c). The init routine which gets called if the detection routine
- * returns a positive value from the pci_iommu_alloc. This means
- * no presence of a memory allocator.
- * d). Similar to the 'init', except that this gets called from pci_iommu_init
- * where we do have a memory allocator.
- *
- * The standard IOMMU_INIT differs from the IOMMU_INIT_FINISH variant
- * in that the former will continue detecting other IOMMUs in the call
- * list after the detection routine returns a positive number, while the
- * latter will stop the execution chain upon first successful detection.
- * Both variants will still call the 'init' and 'late_init' functions if
- * they are set.
- */
-#define IOMMU_INIT_FINISH(_detect, _depend, _init, _late_init) \
- __IOMMU_INIT(_detect, _depend, _init, _late_init, 1)
-
-#define IOMMU_INIT(_detect, _depend, _init, _late_init) \
- __IOMMU_INIT(_detect, _depend, _init, _late_init, 0)
-
-void sort_iommu_table(struct iommu_table_entry *start,
- struct iommu_table_entry *finish);
-
-void check_iommu_entries(struct iommu_table_entry *start,
- struct iommu_table_entry *finish);
-
-#endif /* _ASM_X86_IOMMU_TABLE_H */
diff --git a/arch/x86/include/asm/swiotlb.h b/arch/x86/include/asm/swiotlb.h
deleted file mode 100644
index ff6c92eff035a..0000000000000
--- a/arch/x86/include/asm/swiotlb.h
+++ /dev/null
@@ -1,30 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0 */
-#ifndef _ASM_X86_SWIOTLB_H
-#define _ASM_X86_SWIOTLB_H
-
-#include <linux/swiotlb.h>
-
-#ifdef CONFIG_SWIOTLB
-extern int swiotlb;
-extern int __init pci_swiotlb_detect_override(void);
-extern int __init pci_swiotlb_detect_4gb(void);
-extern void __init pci_swiotlb_init(void);
-extern void __init pci_swiotlb_late_init(void);
-#else
-#define swiotlb 0
-static inline int pci_swiotlb_detect_override(void)
-{
- return 0;
-}
-static inline int pci_swiotlb_detect_4gb(void)
-{
- return 0;
-}
-static inline void pci_swiotlb_init(void)
-{
-}
-static inline void pci_swiotlb_late_init(void)
-{
-}
-#endif
-#endif /* _ASM_X86_SWIOTLB_H */
diff --git a/arch/x86/include/asm/xen/swiotlb-xen.h b/arch/x86/include/asm/xen/swiotlb-xen.h
index 66b4ddde77430..e5a90b42e4dde 100644
--- a/arch/x86/include/asm/xen/swiotlb-xen.h
+++ b/arch/x86/include/asm/xen/swiotlb-xen.h
@@ -3,10 +3,8 @@
#define _ASM_X86_SWIOTLB_XEN_H
#ifdef CONFIG_SWIOTLB_XEN
-extern int __init pci_xen_swiotlb_detect(void);
extern int pci_xen_swiotlb_init_late(void);
#else
-#define pci_xen_swiotlb_detect NULL
static inline int pci_xen_swiotlb_init_late(void) { return -ENXIO; }
#endif
diff --git a/arch/x86/kernel/Makefile b/arch/x86/kernel/Makefile
index 6aef9ee28a394..2851d4f0aa0d2 100644
--- a/arch/x86/kernel/Makefile
+++ b/arch/x86/kernel/Makefile
@@ -71,7 +71,6 @@ obj-y += bootflag.o e820.o
obj-y += pci-dma.o quirks.o topology.o kdebugfs.o
obj-y += alternative.o i8253.o hw_breakpoint.o
obj-y += tsc.o tsc_msr.o io_delay.o rtc.o
-obj-y += pci-iommu_table.o
obj-y += resource.o
obj-y += irqflags.o
obj-y += static_call.o
@@ -136,7 +135,6 @@ obj-$(CONFIG_PCSPKR_PLATFORM) += pcspeaker.o
obj-$(CONFIG_X86_CHECK_BIOS_CORRUPTION) += check.o
-obj-$(CONFIG_SWIOTLB) += pci-swiotlb.o
obj-$(CONFIG_OF) += devicetree.o
obj-$(CONFIG_UPROBES) += uprobes.o
diff --git a/arch/x86/kernel/amd_gart_64.c b/arch/x86/kernel/amd_gart_64.c
index ed837383de5c8..194d54eed5376 100644
--- a/arch/x86/kernel/amd_gart_64.c
+++ b/arch/x86/kernel/amd_gart_64.c
@@ -38,11 +38,9 @@
#include <asm/iommu.h>
#include <asm/gart.h>
#include <asm/set_memory.h>
-#include <asm/swiotlb.h>
#include <asm/dma.h>
#include <asm/amd_nb.h>
#include <asm/x86_init.h>
-#include <asm/iommu_table.h>
static unsigned long iommu_bus_base; /* GART remapping area (physical) */
static unsigned long iommu_size; /* size of remapping area bytes */
@@ -808,7 +806,7 @@ int __init gart_iommu_init(void)
flush_gart();
dma_ops = &gart_dma_ops;
x86_platform.iommu_shutdown = gart_iommu_shutdown;
- swiotlb = 0;
+ x86_swiotlb_enable = false;
return 0;
}
@@ -842,4 +840,3 @@ void __init gart_parse_options(char *p)
}
}
}
-IOMMU_INIT_POST(gart_iommu_hole_init);
diff --git a/arch/x86/kernel/aperture_64.c b/arch/x86/kernel/aperture_64.c
index af3ba08b684b5..7a5630d904b23 100644
--- a/arch/x86/kernel/aperture_64.c
+++ b/arch/x86/kernel/aperture_64.c
@@ -392,7 +392,7 @@ void __init early_gart_iommu_check(void)
static int __initdata printed_gart_size_msg;
-int __init gart_iommu_hole_init(void)
+void __init gart_iommu_hole_init(void)
{
u32 agp_aper_base = 0, agp_aper_order = 0;
u32 aper_size, aper_alloc = 0, aper_order = 0, last_aper_order = 0;
@@ -401,11 +401,11 @@ int __init gart_iommu_hole_init(void)
int i, node;
if (!amd_gart_present())
- return -ENODEV;
+ return;
if (gart_iommu_aperture_disabled || !fix_aperture ||
!early_pci_allowed())
- return -ENODEV;
+ return;
pr_info("Checking aperture...\n");
@@ -491,10 +491,8 @@ int __init gart_iommu_hole_init(void)
* and fixed up the northbridge
*/
exclude_from_core(last_aper_base, last_aper_order);
-
- return 1;
}
- return 0;
+ return;
}
if (!fallback_aper_force) {
@@ -527,7 +525,7 @@ int __init gart_iommu_hole_init(void)
panic("Not enough memory for aperture");
}
} else {
- return 0;
+ return;
}
/*
@@ -561,6 +559,4 @@ int __init gart_iommu_hole_init(void)
}
set_up_gart_resume(aper_order, aper_alloc);
-
- return 1;
}
diff --git a/arch/x86/kernel/pci-dma.c b/arch/x86/kernel/pci-dma.c
index de234e7a8962e..17bcac942ac49 100644
--- a/arch/x86/kernel/pci-dma.c
+++ b/arch/x86/kernel/pci-dma.c
@@ -7,13 +7,16 @@
#include <linux/memblock.h>
#include <linux/gfp.h>
#include <linux/pci.h>
+#include <linux/amd-iommu.h>
#include <asm/proto.h>
#include <asm/dma.h>
#include <asm/iommu.h>
#include <asm/gart.h>
#include <asm/x86_init.h>
-#include <asm/iommu_table.h>
+
+#include <xen/xen.h>
+#include <xen/swiotlb-xen.h>
static bool disable_dac_quirk __read_mostly;
@@ -34,24 +37,87 @@ int no_iommu __read_mostly;
/* Set this to 1 if there is a HW IOMMU in the system */
int iommu_detected __read_mostly = 0;
-extern struct iommu_table_entry __iommu_table[], __iommu_table_end[];
+#ifdef CONFIG_SWIOTLB
+bool x86_swiotlb_enable;
+
+/*
+ * If 4GB or more detected (and iommu=off not set) or if SME is active
+ * then set swiotlb to 1 and return 1.
+ */
+static void __init pci_swiotlb_detect_4gb(void)
+{
+ /* don't initialize swiotlb if iommu=off (no_iommu=1) */
+ if (!no_iommu && max_possible_pfn > MAX_DMA32_PFN)
+ x86_swiotlb_enable = true;
+
+ /*
+ * Set swiotlb to 1 so that bounce buffers are allocated and used for
+ * devices that can't support DMA to encrypted memory.
+ */
+ if (cc_platform_has(CC_ATTR_HOST_MEM_ENCRYPT))
+ x86_swiotlb_enable = true;
+
+ if (swiotlb_force == SWIOTLB_FORCE)
+ x86_swiotlb_enable = true;
+}
+#else
+static inline void __init pci_swiotlb_detect_4gb(void)
+{
+}
+#endif /* CONFIG_SWIOTLB */
+
+#ifdef CONFIG_SWIOTLB_XEN
+static bool xen_swiotlb;
+
+static void __init pci_xen_swiotlb_init(void)
+{
+ if (!xen_initial_domain() && !x86_swiotlb_enable &&
+ swiotlb_force != SWIOTLB_FORCE)
+ return;
+ x86_swiotlb_enable = false;
+ xen_swiotlb = true;
+ xen_swiotlb_init_early();
+ dma_ops = &xen_swiotlb_dma_ops;
+ if (IS_ENABLED(CONFIG_PCI))
+ pci_request_acs();
+}
+
+int pci_xen_swiotlb_init_late(void)
+{
+ int rc;
+
+ if (xen_swiotlb)
+ return 0;
+
+ rc = xen_swiotlb_init();
+ if (rc)
+ return rc;
+
+ /* XXX: this switches the dma ops under live devices! */
+ dma_ops = &xen_swiotlb_dma_ops;
+ if (IS_ENABLED(CONFIG_PCI))
+ pci_request_acs();
+ return 0;
+}
+EXPORT_SYMBOL_GPL(pci_xen_swiotlb_init_late);
+#else
+static inline void __init pci_xen_swiotlb_init(void)
+{
+}
+#endif /* CONFIG_SWIOTLB_XEN */
void __init pci_iommu_alloc(void)
{
- struct iommu_table_entry *p;
-
- sort_iommu_table(__iommu_table, __iommu_table_end);
- check_iommu_entries(__iommu_table, __iommu_table_end);
-
- for (p = __iommu_table; p < __iommu_table_end; p++) {
- if (p && p->detect && p->detect() > 0) {
- p->flags |= IOMMU_DETECTED;
- if (p->early_init)
- p->early_init();
- if (p->flags & IOMMU_FINISH_IF_DETECTED)
- break;
- }
+ if (xen_pv_domain()) {
+ pci_xen_swiotlb_init();
+ return;
}
+ pci_swiotlb_detect_4gb();
+ gart_iommu_hole_init();
+ amd_iommu_detect();
+ detect_intel_iommu();
+ if (x86_swiotlb_enable)
+ swiotlb_init(0);
}
/*
@@ -102,7 +168,7 @@ static __init int iommu_setup(char *p)
}
#ifdef CONFIG_SWIOTLB
if (!strncmp(p, "soft", 4))
- swiotlb = 1;
+ x86_swiotlb_enable = 1;
#endif
if (!strncmp(p, "pt", 2))
iommu_set_default_passthrough(true);
@@ -121,14 +187,17 @@ early_param("iommu", iommu_setup);
static int __init pci_iommu_init(void)
{
- struct iommu_table_entry *p;
-
x86_init.iommu.iommu_init();
- for (p = __iommu_table; p < __iommu_table_end; p++) {
- if (p && (p->flags & IOMMU_DETECTED) && p->late_init)
- p->late_init();
+#ifdef CONFIG_SWIOTLB
+ /* An IOMMU turned us off. */
+ if (x86_swiotlb_enable) {
+ pr_info("PCI-DMA: Using software bounce buffering for IO (SWIOTLB)\n");
+ swiotlb_print_info();
+ } else {
+ swiotlb_exit();
}
+#endif
return 0;
}
diff --git a/arch/x86/kernel/pci-iommu_table.c b/arch/x86/kernel/pci-iommu_table.c
deleted file mode 100644
index 42e92ec62973b..0000000000000
--- a/arch/x86/kernel/pci-iommu_table.c
+++ /dev/null
@@ -1,77 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0
-#include <linux/dma-mapping.h>
-#include <asm/iommu_table.h>
-#include <linux/string.h>
-#include <linux/kallsyms.h>
-
-static struct iommu_table_entry * __init
-find_dependents_of(struct iommu_table_entry *start,
- struct iommu_table_entry *finish,
- struct iommu_table_entry *q)
-{
- struct iommu_table_entry *p;
-
- if (!q)
- return NULL;
-
- for (p = start; p < finish; p++)
- if (p->detect == q->depend)
- return p;
-
- return NULL;
-}
-
-
-void __init sort_iommu_table(struct iommu_table_entry *start,
- struct iommu_table_entry *finish) {
-
- struct iommu_table_entry *p, *q, tmp;
-
- for (p = start; p < finish; p++) {
-again:
- q = find_dependents_of(start, finish, p);
- /* We are bit sneaky here. We use the memory address to figure
- * out if the node we depend on is past our point, if so, swap.
- */
- if (q > p) {
- tmp = *p;
- memmove(p, q, sizeof(*p));
- *q = tmp;
- goto again;
- }
- }
-
-}
-
-#ifdef DEBUG
-void __init check_iommu_entries(struct iommu_table_entry *start,
- struct iommu_table_entry *finish)
-{
- struct iommu_table_entry *p, *q, *x;
-
- /* Simple cyclic dependency checker. */
- for (p = start; p < finish; p++) {
- q = find_dependents_of(start, finish, p);
- x = find_dependents_of(start, finish, q);
- if (p == x) {
- printk(KERN_ERR "CYCLIC DEPENDENCY FOUND! %pS depends on %pS and vice-versa. BREAKING IT.\n",
- p->detect, q->detect);
- /* Heavy handed way..*/
- x->depend = NULL;
- }
- }
-
- for (p = start; p < finish; p++) {
- q = find_dependents_of(p, finish, p);
- if (q && q > p) {
- printk(KERN_ERR "EXECUTION ORDER INVALID! %pS should be called before %pS!\n",
- p->detect, q->detect);
- }
- }
-}
-#else
-void __init check_iommu_entries(struct iommu_table_entry *start,
- struct iommu_table_entry *finish)
-{
-}
-#endif
diff --git a/arch/x86/kernel/pci-swiotlb.c b/arch/x86/kernel/pci-swiotlb.c
deleted file mode 100644
index 814ab46a0dada..0000000000000
--- a/arch/x86/kernel/pci-swiotlb.c
+++ /dev/null
@@ -1,77 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0
-
-#include <linux/pci.h>
-#include <linux/cache.h>
-#include <linux/init.h>
-#include <linux/swiotlb.h>
-#include <linux/memblock.h>
-#include <linux/dma-direct.h>
-#include <linux/cc_platform.h>
-
-#include <asm/iommu.h>
-#include <asm/swiotlb.h>
-#include <asm/dma.h>
-#include <asm/xen/swiotlb-xen.h>
-#include <asm/iommu_table.h>
-
-int swiotlb __read_mostly;
-
-/*
- * pci_swiotlb_detect_override - set swiotlb to 1 if necessary
- *
- * This returns non-zero if we are forced to use swiotlb (by the boot
- * option).
- */
-int __init pci_swiotlb_detect_override(void)
-{
- if (swiotlb_force == SWIOTLB_FORCE)
- swiotlb = 1;
-
- return swiotlb;
-}
-IOMMU_INIT_FINISH(pci_swiotlb_detect_override,
- pci_xen_swiotlb_detect,
- pci_swiotlb_init,
- pci_swiotlb_late_init);
-
-/*
- * If 4GB or more detected (and iommu=off not set) or if SME is active
- * then set swiotlb to 1 and return 1.
- */
-int __init pci_swiotlb_detect_4gb(void)
-{
- /* don't initialize swiotlb if iommu=off (no_iommu=1) */
- if (!no_iommu && max_possible_pfn > MAX_DMA32_PFN)
- swiotlb = 1;
-
- /*
- * Set swiotlb to 1 so that bounce buffers are allocated and used for
- * devices that can't support DMA to encrypted memory.
- */
- if (cc_platform_has(CC_ATTR_HOST_MEM_ENCRYPT))
- swiotlb = 1;
-
- return swiotlb;
-}
-IOMMU_INIT(pci_swiotlb_detect_4gb,
- pci_swiotlb_detect_override,
- pci_swiotlb_init,
- pci_swiotlb_late_init);
-
-void __init pci_swiotlb_init(void)
-{
- if (swiotlb)
- swiotlb_init(0);
-}
-
-void __init pci_swiotlb_late_init(void)
-{
- /* An IOMMU turned us off. */
- if (!swiotlb)
- swiotlb_exit();
- else {
- printk(KERN_INFO "PCI-DMA: "
- "Using software bounce buffering for IO (SWIOTLB)\n");
- swiotlb_print_info();
- }
-}
diff --git a/arch/x86/kernel/tboot.c b/arch/x86/kernel/tboot.c
index f9af561c3cd4f..0c1154a1c4032 100644
--- a/arch/x86/kernel/tboot.c
+++ b/arch/x86/kernel/tboot.c
@@ -24,7 +24,6 @@
#include <asm/processor.h>
#include <asm/bootparam.h>
#include <asm/pgalloc.h>
-#include <asm/swiotlb.h>
#include <asm/fixmap.h>
#include <asm/proto.h>
#include <asm/setup.h>
diff --git a/arch/x86/kernel/vmlinux.lds.S b/arch/x86/kernel/vmlinux.lds.S
index 27f830345b6f0..bbe910c15b293 100644
--- a/arch/x86/kernel/vmlinux.lds.S
+++ b/arch/x86/kernel/vmlinux.lds.S
@@ -306,18 +306,6 @@ SECTIONS
*(.altinstr_replacement)
}
- /*
- * struct iommu_table_entry entries are injected in this section.
- * It is an array of IOMMUs which during run time gets sorted depending
- * on its dependency order. After rootfs_initcall is complete
- * this section can be safely removed.
- */
- .iommu_table : AT(ADDR(.iommu_table) - LOAD_OFFSET) {
- __iommu_table = .;
- *(.iommu_table)
- __iommu_table_end = .;
- }
-
. = ALIGN(8);
.apicdrivers : AT(ADDR(.apicdrivers) - LOAD_OFFSET) {
__apicdrivers = .;
diff --git a/arch/x86/xen/Makefile b/arch/x86/xen/Makefile
index 4953260e281c3..3c5b52fbe4a7f 100644
--- a/arch/x86/xen/Makefile
+++ b/arch/x86/xen/Makefile
@@ -47,6 +47,4 @@ obj-$(CONFIG_XEN_DEBUG_FS) += debugfs.o
obj-$(CONFIG_XEN_PV_DOM0) += vga.o
-obj-$(CONFIG_SWIOTLB_XEN) += pci-swiotlb-xen.o
-
obj-$(CONFIG_XEN_EFI) += efi.o
diff --git a/arch/x86/xen/pci-swiotlb-xen.c b/arch/x86/xen/pci-swiotlb-xen.c
deleted file mode 100644
index 46df59aeaa06a..0000000000000
--- a/arch/x86/xen/pci-swiotlb-xen.c
+++ /dev/null
@@ -1,96 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0
-
-/* Glue code to lib/swiotlb-xen.c */
-
-#include <linux/dma-map-ops.h>
-#include <linux/pci.h>
-#include <xen/swiotlb-xen.h>
-
-#include <asm/xen/hypervisor.h>
-#include <xen/xen.h>
-#include <asm/iommu_table.h>
-
-
-#include <asm/xen/swiotlb-xen.h>
-#ifdef CONFIG_X86_64
-#include <asm/iommu.h>
-#include <asm/dma.h>
-#endif
-#include <linux/export.h>
-
-static int xen_swiotlb __read_mostly;
-
-/*
- * pci_xen_swiotlb_detect - set xen_swiotlb to 1 if necessary
- *
- * This returns non-zero if we are forced to use xen_swiotlb (by the boot
- * option).
- */
-int __init pci_xen_swiotlb_detect(void)
-{
-
- if (!xen_pv_domain())
- return 0;
-
- /* If running as PV guest, either iommu=soft, or swiotlb=force will
- * activate this IOMMU. If running as PV privileged, activate it
- * irregardless.
- */
- if (xen_initial_domain() || swiotlb || swiotlb_force == SWIOTLB_FORCE)
- xen_swiotlb = 1;
-
- /* If we are running under Xen, we MUST disable the native SWIOTLB.
- * Don't worry about swiotlb_force flag activating the native, as
- * the 'swiotlb' flag is the only one turning it on. */
- swiotlb = 0;
-
-#ifdef CONFIG_X86_64
- /* pci_swiotlb_detect_4gb turns on native SWIOTLB if no_iommu == 0
- * (so no iommu=X command line over-writes).
- * Considering that PV guests do not want the *native SWIOTLB* but
- * only Xen SWIOTLB it is not useful to us so set no_iommu=1 here.
- */
- if (max_pfn > MAX_DMA32_PFN)
- no_iommu = 1;
-#endif
- return xen_swiotlb;
-}
-
-static void __init pci_xen_swiotlb_init(void)
-{
- if (xen_swiotlb) {
- xen_swiotlb_init_early();
- dma_ops = &xen_swiotlb_dma_ops;
-
-#ifdef CONFIG_PCI
- /* Make sure ACS will be enabled */
- pci_request_acs();
-#endif
- }
-}
-
-int pci_xen_swiotlb_init_late(void)
-{
- int rc;
-
- if (xen_swiotlb)
- return 0;
-
- rc = xen_swiotlb_init();
- if (rc)
- return rc;
-
- dma_ops = &xen_swiotlb_dma_ops;
-#ifdef CONFIG_PCI
- /* Make sure ACS will be enabled */
- pci_request_acs();
-#endif
-
- return 0;
-}
-EXPORT_SYMBOL_GPL(pci_xen_swiotlb_init_late);
-
-IOMMU_INIT_FINISH(pci_xen_swiotlb_detect,
- NULL,
- pci_xen_swiotlb_init,
- NULL);
diff --git a/drivers/iommu/amd/init.c b/drivers/iommu/amd/init.c
index b10fb52ea4428..721300cf90207 100644
--- a/drivers/iommu/amd/init.c
+++ b/drivers/iommu/amd/init.c
@@ -27,7 +27,6 @@
#include <asm/apic.h>
#include <asm/gart.h>
#include <asm/x86_init.h>
-#include <asm/iommu_table.h>
#include <asm/io_apic.h>
#include <asm/irq_remapping.h>
#include <asm/set_memory.h>
@@ -3237,11 +3236,6 @@ __setup("ivrs_ioapic", parse_ivrs_ioapic);
__setup("ivrs_hpet", parse_ivrs_hpet);
__setup("ivrs_acpihid", parse_ivrs_acpihid);
-IOMMU_INIT_FINISH(amd_iommu_detect,
- gart_iommu_hole_init,
- NULL,
- NULL);
-
bool amd_iommu_v2_supported(void)
{
return amd_iommu_v2_present;
diff --git a/drivers/iommu/amd/iommu.c b/drivers/iommu/amd/iommu.c
index 461f1844ed1fb..541a7b8315a12 100644
--- a/drivers/iommu/amd/iommu.c
+++ b/drivers/iommu/amd/iommu.c
@@ -1834,7 +1834,10 @@ void amd_iommu_domain_update(struct protection_domain *domain)
static void __init amd_iommu_init_dma_ops(void)
{
- swiotlb = (iommu_default_passthrough() || sme_me_mask) ? 1 : 0;
+ if (iommu_default_passthrough() || sme_me_mask)
+ x86_swiotlb_enable = true;
+ else
+ x86_swiotlb_enable = false;
}
int __init amd_iommu_init_api(void)
diff --git a/drivers/iommu/intel/dmar.c b/drivers/iommu/intel/dmar.c
index 915bff76fe965..29bee4b210c5b 100644
--- a/drivers/iommu/intel/dmar.c
+++ b/drivers/iommu/intel/dmar.c
@@ -30,7 +30,6 @@
#include <linux/numa.h>
#include <linux/limits.h>
#include <asm/irq_remapping.h>
-#include <asm/iommu_table.h>
#include <trace/events/intel_iommu.h>
#include "../irq_remapping.h"
@@ -913,7 +912,7 @@ dmar_validate_one_drhd(struct acpi_dmar_header *entry, void *arg)
return 0;
}
-int __init detect_intel_iommu(void)
+void __init detect_intel_iommu(void)
{
int ret;
struct dmar_res_callback validate_drhd_cb = {
@@ -946,8 +945,6 @@ int __init detect_intel_iommu(void)
dmar_tbl = NULL;
}
up_write(&dmar_global_lock);
-
- return ret ? ret : 1;
}
static void unmap_iommu(struct intel_iommu *iommu)
@@ -2165,7 +2162,6 @@ static int __init dmar_free_unused_resources(void)
}
late_initcall(dmar_free_unused_resources);
-IOMMU_INIT_POST(detect_intel_iommu);
/*
* DMAR Hotplug Support
diff --git a/include/linux/dmar.h b/include/linux/dmar.h
index 45e903d847335..cbd714a198a0a 100644
--- a/include/linux/dmar.h
+++ b/include/linux/dmar.h
@@ -121,7 +121,7 @@ extern int dmar_remove_dev_scope(struct dmar_pci_notify_info *info,
u16 segment, struct dmar_dev_scope *devices,
int count);
/* Intel IOMMU detection */
-extern int detect_intel_iommu(void);
+void detect_intel_iommu(void);
extern int enable_drhd_fault_handling(void);
extern int dmar_device_add(acpi_handle handle);
extern int dmar_device_remove(acpi_handle handle);
@@ -197,6 +197,10 @@ static inline bool dmar_platform_optin(void)
return false;
}
+static inline void detect_intel_iommu(void)
+{
+}
+
#endif /* CONFIG_DMAR_TABLE */
struct irte {
--
2.30.2
^ permalink raw reply related [flat|nested] 118+ messages in thread
* [PATCH 07/11] x86: remove the IOMMU table infrastructure
@ 2022-02-27 14:30 ` Christoph Hellwig
0 siblings, 0 replies; 118+ messages in thread
From: Christoph Hellwig @ 2022-02-27 14:30 UTC (permalink / raw)
To: iommu
Cc: Juergen Gross, linux-s390, linux-hyperv, Stefano Stabellini,
linux-ia64, Anshuman Khandual, Konrad Rzeszutek Wilk,
Robin Murphy, x86, linux-mips, linuxppc-dev, tboot-devel,
linux-pci, xen-devel, Boris Ostrovsky, David Woodhouse,
linux-riscv, linux-arm-kernel
The IOMMU table tries to separate the different IOMMUs into different
backends, but actually requires various cross calls.
Rewrite the code to do the generic swiotlb/swiotlb-xen setup directly
in pci-dma.c and then just call into the IOMMU drivers.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
arch/ia64/include/asm/iommu_table.h | 7 --
arch/x86/include/asm/dma-mapping.h | 1 -
arch/x86/include/asm/gart.h | 5 +-
arch/x86/include/asm/iommu.h | 6 ++
arch/x86/include/asm/iommu_table.h | 102 -----------------------
arch/x86/include/asm/swiotlb.h | 30 -------
arch/x86/include/asm/xen/swiotlb-xen.h | 2 -
arch/x86/kernel/Makefile | 2 -
arch/x86/kernel/amd_gart_64.c | 5 +-
arch/x86/kernel/aperture_64.c | 14 ++--
arch/x86/kernel/pci-dma.c | 111 ++++++++++++++++++++-----
arch/x86/kernel/pci-iommu_table.c | 77 -----------------
arch/x86/kernel/pci-swiotlb.c | 77 -----------------
arch/x86/kernel/tboot.c | 1 -
arch/x86/kernel/vmlinux.lds.S | 12 ---
arch/x86/xen/Makefile | 2 -
arch/x86/xen/pci-swiotlb-xen.c | 96 ---------------------
drivers/iommu/amd/init.c | 6 --
drivers/iommu/amd/iommu.c | 5 +-
drivers/iommu/intel/dmar.c | 6 +-
include/linux/dmar.h | 6 +-
21 files changed, 114 insertions(+), 459 deletions(-)
delete mode 100644 arch/ia64/include/asm/iommu_table.h
delete mode 100644 arch/x86/include/asm/iommu_table.h
delete mode 100644 arch/x86/include/asm/swiotlb.h
delete mode 100644 arch/x86/kernel/pci-iommu_table.c
delete mode 100644 arch/x86/kernel/pci-swiotlb.c
delete mode 100644 arch/x86/xen/pci-swiotlb-xen.c
diff --git a/arch/ia64/include/asm/iommu_table.h b/arch/ia64/include/asm/iommu_table.h
deleted file mode 100644
index cc96116ac276a..0000000000000
--- a/arch/ia64/include/asm/iommu_table.h
+++ /dev/null
@@ -1,7 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0 */
-#ifndef _ASM_IA64_IOMMU_TABLE_H
-#define _ASM_IA64_IOMMU_TABLE_H
-
-#define IOMMU_INIT_POST(_detect)
-
-#endif /* _ASM_IA64_IOMMU_TABLE_H */
diff --git a/arch/x86/include/asm/dma-mapping.h b/arch/x86/include/asm/dma-mapping.h
index bb1654fe0ce74..256fd8115223d 100644
--- a/arch/x86/include/asm/dma-mapping.h
+++ b/arch/x86/include/asm/dma-mapping.h
@@ -9,7 +9,6 @@
#include <linux/scatterlist.h>
#include <asm/io.h>
-#include <asm/swiotlb.h>
extern int iommu_merge;
extern int panic_on_overflow;
diff --git a/arch/x86/include/asm/gart.h b/arch/x86/include/asm/gart.h
index 3185565743459..5af8088a10df6 100644
--- a/arch/x86/include/asm/gart.h
+++ b/arch/x86/include/asm/gart.h
@@ -38,7 +38,7 @@ extern int gart_iommu_aperture_disabled;
extern void early_gart_iommu_check(void);
extern int gart_iommu_init(void);
extern void __init gart_parse_options(char *);
-extern int gart_iommu_hole_init(void);
+void gart_iommu_hole_init(void);
#else
#define gart_iommu_aperture 0
@@ -51,9 +51,8 @@ static inline void early_gart_iommu_check(void)
static inline void gart_parse_options(char *options)
{
}
-static inline int gart_iommu_hole_init(void)
+static inline void gart_iommu_hole_init(void)
{
- return -ENODEV;
}
#endif
diff --git a/arch/x86/include/asm/iommu.h b/arch/x86/include/asm/iommu.h
index bf1ed2ddc74bd..dba89ed40d38d 100644
--- a/arch/x86/include/asm/iommu.h
+++ b/arch/x86/include/asm/iommu.h
@@ -9,6 +9,12 @@
extern int force_iommu, no_iommu;
extern int iommu_detected;
+#ifdef CONFIG_SWIOTLB
+extern bool x86_swiotlb_enable;
+#else
+#define x86_swiotlb_enable false
+#endif
+
/* 10 seconds */
#define DMAR_OPERATION_TIMEOUT ((cycles_t) tsc_khz*10*1000)
diff --git a/arch/x86/include/asm/iommu_table.h b/arch/x86/include/asm/iommu_table.h
deleted file mode 100644
index 1fb3fd1a83c25..0000000000000
--- a/arch/x86/include/asm/iommu_table.h
+++ /dev/null
@@ -1,102 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0 */
-#ifndef _ASM_X86_IOMMU_TABLE_H
-#define _ASM_X86_IOMMU_TABLE_H
-
-#include <asm/swiotlb.h>
-
-/*
- * History lesson:
- * The execution chain of IOMMUs in 2.6.36 looks as so:
- *
- * [xen-swiotlb]
- * |
- * +----[swiotlb *]--+
- * / | \
- * / | \
- * [GART] [Calgary] [Intel VT-d]
- * /
- * /
- * [AMD-Vi]
- *
- * *: if SWIOTLB detected 'iommu=soft'/'swiotlb=force' it would skip
- * over the rest of IOMMUs and unconditionally initialize the SWIOTLB.
- * Also it would surreptitiously initialize set the swiotlb=1 if there were
- * more than 4GB and if the user did not pass in 'iommu=off'. The swiotlb
- * flag would be turned off by all IOMMUs except the Calgary one.
- *
- * The IOMMU_INIT* macros allow a similar tree (or more complex if desired)
- * to be built by defining who we depend on.
- *
- * And all that needs to be done is to use one of the macros in the IOMMU
- * and the pci-dma.c will take care of the rest.
- */
-
-struct iommu_table_entry {
- initcall_t detect;
- initcall_t depend;
- void (*early_init)(void); /* No memory allocate available. */
- void (*late_init)(void); /* Yes, can allocate memory. */
-#define IOMMU_FINISH_IF_DETECTED (1<<0)
-#define IOMMU_DETECTED (1<<1)
- int flags;
-};
-/*
- * Macro fills out an entry in the .iommu_table that is equivalent
- * to the fields that 'struct iommu_table_entry' has. The entries
- * that are put in the .iommu_table section are not put in any order
- * hence during boot-time we will have to resort them based on
- * dependency. */
-
-
-#define __IOMMU_INIT(_detect, _depend, _early_init, _late_init, _finish)\
- static const struct iommu_table_entry \
- __iommu_entry_##_detect __used \
- __attribute__ ((unused, __section__(".iommu_table"), \
- aligned((sizeof(void *))))) \
- = {_detect, _depend, _early_init, _late_init, \
- _finish ? IOMMU_FINISH_IF_DETECTED : 0}
-/*
- * The simplest IOMMU definition. Provide the detection routine
- * and it will be run after the SWIOTLB and the other IOMMUs
- * that utilize this macro. If the IOMMU is detected (ie, the
- * detect routine returns a positive value), the other IOMMUs
- * are also checked. You can use IOMMU_INIT_POST_FINISH if you prefer
- * to stop detecting the other IOMMUs after yours has been detected.
- */
-#define IOMMU_INIT_POST(_detect) \
- __IOMMU_INIT(_detect, pci_swiotlb_detect_4gb, NULL, NULL, 0)
-
-#define IOMMU_INIT_POST_FINISH(detect) \
- __IOMMU_INIT(_detect, pci_swiotlb_detect_4gb, NULL, NULL, 1)
-
-/*
- * A more sophisticated version of IOMMU_INIT. This variant requires:
- * a). A detection routine function.
- * b). The name of the detection routine we depend on to get called
- * before us.
- * c). The init routine which gets called if the detection routine
- * returns a positive value from the pci_iommu_alloc. This means
- * no presence of a memory allocator.
- * d). Similar to the 'init', except that this gets called from pci_iommu_init
- * where we do have a memory allocator.
- *
- * The standard IOMMU_INIT differs from the IOMMU_INIT_FINISH variant
- * in that the former will continue detecting other IOMMUs in the call
- * list after the detection routine returns a positive number, while the
- * latter will stop the execution chain upon first successful detection.
- * Both variants will still call the 'init' and 'late_init' functions if
- * they are set.
- */
-#define IOMMU_INIT_FINISH(_detect, _depend, _init, _late_init) \
- __IOMMU_INIT(_detect, _depend, _init, _late_init, 1)
-
-#define IOMMU_INIT(_detect, _depend, _init, _late_init) \
- __IOMMU_INIT(_detect, _depend, _init, _late_init, 0)
-
-void sort_iommu_table(struct iommu_table_entry *start,
- struct iommu_table_entry *finish);
-
-void check_iommu_entries(struct iommu_table_entry *start,
- struct iommu_table_entry *finish);
-
-#endif /* _ASM_X86_IOMMU_TABLE_H */
diff --git a/arch/x86/include/asm/swiotlb.h b/arch/x86/include/asm/swiotlb.h
deleted file mode 100644
index ff6c92eff035a..0000000000000
--- a/arch/x86/include/asm/swiotlb.h
+++ /dev/null
@@ -1,30 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0 */
-#ifndef _ASM_X86_SWIOTLB_H
-#define _ASM_X86_SWIOTLB_H
-
-#include <linux/swiotlb.h>
-
-#ifdef CONFIG_SWIOTLB
-extern int swiotlb;
-extern int __init pci_swiotlb_detect_override(void);
-extern int __init pci_swiotlb_detect_4gb(void);
-extern void __init pci_swiotlb_init(void);
-extern void __init pci_swiotlb_late_init(void);
-#else
-#define swiotlb 0
-static inline int pci_swiotlb_detect_override(void)
-{
- return 0;
-}
-static inline int pci_swiotlb_detect_4gb(void)
-{
- return 0;
-}
-static inline void pci_swiotlb_init(void)
-{
-}
-static inline void pci_swiotlb_late_init(void)
-{
-}
-#endif
-#endif /* _ASM_X86_SWIOTLB_H */
diff --git a/arch/x86/include/asm/xen/swiotlb-xen.h b/arch/x86/include/asm/xen/swiotlb-xen.h
index 66b4ddde77430..e5a90b42e4dde 100644
--- a/arch/x86/include/asm/xen/swiotlb-xen.h
+++ b/arch/x86/include/asm/xen/swiotlb-xen.h
@@ -3,10 +3,8 @@
#define _ASM_X86_SWIOTLB_XEN_H
#ifdef CONFIG_SWIOTLB_XEN
-extern int __init pci_xen_swiotlb_detect(void);
extern int pci_xen_swiotlb_init_late(void);
#else
-#define pci_xen_swiotlb_detect NULL
static inline int pci_xen_swiotlb_init_late(void) { return -ENXIO; }
#endif
diff --git a/arch/x86/kernel/Makefile b/arch/x86/kernel/Makefile
index 6aef9ee28a394..2851d4f0aa0d2 100644
--- a/arch/x86/kernel/Makefile
+++ b/arch/x86/kernel/Makefile
@@ -71,7 +71,6 @@ obj-y += bootflag.o e820.o
obj-y += pci-dma.o quirks.o topology.o kdebugfs.o
obj-y += alternative.o i8253.o hw_breakpoint.o
obj-y += tsc.o tsc_msr.o io_delay.o rtc.o
-obj-y += pci-iommu_table.o
obj-y += resource.o
obj-y += irqflags.o
obj-y += static_call.o
@@ -136,7 +135,6 @@ obj-$(CONFIG_PCSPKR_PLATFORM) += pcspeaker.o
obj-$(CONFIG_X86_CHECK_BIOS_CORRUPTION) += check.o
-obj-$(CONFIG_SWIOTLB) += pci-swiotlb.o
obj-$(CONFIG_OF) += devicetree.o
obj-$(CONFIG_UPROBES) += uprobes.o
diff --git a/arch/x86/kernel/amd_gart_64.c b/arch/x86/kernel/amd_gart_64.c
index ed837383de5c8..194d54eed5376 100644
--- a/arch/x86/kernel/amd_gart_64.c
+++ b/arch/x86/kernel/amd_gart_64.c
@@ -38,11 +38,9 @@
#include <asm/iommu.h>
#include <asm/gart.h>
#include <asm/set_memory.h>
-#include <asm/swiotlb.h>
#include <asm/dma.h>
#include <asm/amd_nb.h>
#include <asm/x86_init.h>
-#include <asm/iommu_table.h>
static unsigned long iommu_bus_base; /* GART remapping area (physical) */
static unsigned long iommu_size; /* size of remapping area bytes */
@@ -808,7 +806,7 @@ int __init gart_iommu_init(void)
flush_gart();
dma_ops = &gart_dma_ops;
x86_platform.iommu_shutdown = gart_iommu_shutdown;
- swiotlb = 0;
+ x86_swiotlb_enable = false;
return 0;
}
@@ -842,4 +840,3 @@ void __init gart_parse_options(char *p)
}
}
}
-IOMMU_INIT_POST(gart_iommu_hole_init);
diff --git a/arch/x86/kernel/aperture_64.c b/arch/x86/kernel/aperture_64.c
index af3ba08b684b5..7a5630d904b23 100644
--- a/arch/x86/kernel/aperture_64.c
+++ b/arch/x86/kernel/aperture_64.c
@@ -392,7 +392,7 @@ void __init early_gart_iommu_check(void)
static int __initdata printed_gart_size_msg;
-int __init gart_iommu_hole_init(void)
+void __init gart_iommu_hole_init(void)
{
u32 agp_aper_base = 0, agp_aper_order = 0;
u32 aper_size, aper_alloc = 0, aper_order = 0, last_aper_order = 0;
@@ -401,11 +401,11 @@ int __init gart_iommu_hole_init(void)
int i, node;
if (!amd_gart_present())
- return -ENODEV;
+ return;
if (gart_iommu_aperture_disabled || !fix_aperture ||
!early_pci_allowed())
- return -ENODEV;
+ return;
pr_info("Checking aperture...\n");
@@ -491,10 +491,8 @@ int __init gart_iommu_hole_init(void)
* and fixed up the northbridge
*/
exclude_from_core(last_aper_base, last_aper_order);
-
- return 1;
}
- return 0;
+ return;
}
if (!fallback_aper_force) {
@@ -527,7 +525,7 @@ int __init gart_iommu_hole_init(void)
panic("Not enough memory for aperture");
}
} else {
- return 0;
+ return;
}
/*
@@ -561,6 +559,4 @@ int __init gart_iommu_hole_init(void)
}
set_up_gart_resume(aper_order, aper_alloc);
-
- return 1;
}
diff --git a/arch/x86/kernel/pci-dma.c b/arch/x86/kernel/pci-dma.c
index de234e7a8962e..17bcac942ac49 100644
--- a/arch/x86/kernel/pci-dma.c
+++ b/arch/x86/kernel/pci-dma.c
@@ -7,13 +7,16 @@
#include <linux/memblock.h>
#include <linux/gfp.h>
#include <linux/pci.h>
+#include <linux/amd-iommu.h>
#include <asm/proto.h>
#include <asm/dma.h>
#include <asm/iommu.h>
#include <asm/gart.h>
#include <asm/x86_init.h>
-#include <asm/iommu_table.h>
+
+#include <xen/xen.h>
+#include <xen/swiotlb-xen.h>
static bool disable_dac_quirk __read_mostly;
@@ -34,24 +37,87 @@ int no_iommu __read_mostly;
/* Set this to 1 if there is a HW IOMMU in the system */
int iommu_detected __read_mostly = 0;
-extern struct iommu_table_entry __iommu_table[], __iommu_table_end[];
+#ifdef CONFIG_SWIOTLB
+bool x86_swiotlb_enable;
+
+/*
+ * If 4GB or more detected (and iommu=off not set) or if SME is active
+ * then set swiotlb to 1 and return 1.
+ */
+static void __init pci_swiotlb_detect_4gb(void)
+{
+ /* don't initialize swiotlb if iommu=off (no_iommu=1) */
+ if (!no_iommu && max_possible_pfn > MAX_DMA32_PFN)
+ x86_swiotlb_enable = true;
+
+ /*
+ * Set swiotlb to 1 so that bounce buffers are allocated and used for
+ * devices that can't support DMA to encrypted memory.
+ */
+ if (cc_platform_has(CC_ATTR_HOST_MEM_ENCRYPT))
+ x86_swiotlb_enable = true;
+
+ if (swiotlb_force = SWIOTLB_FORCE)
+ x86_swiotlb_enable = true;
+}
+#else
+static inline void __init pci_swiotlb_detect_4gb(void)
+{
+}
+#endif /* CONFIG_SWIOTLB */
+
+#ifdef CONFIG_SWIOTLB_XEN
+static bool xen_swiotlb;
+
+static void __init pci_xen_swiotlb_init(void)
+{
+ if (!xen_initial_domain() && !x86_swiotlb_enable &&
+ swiotlb_force != SWIOTLB_FORCE)
+ return;
+ x86_swiotlb_enable = false;
+ xen_swiotlb = true;
+ xen_swiotlb_init_early();
+ dma_ops = &xen_swiotlb_dma_ops;
+ if (IS_ENABLED(CONFIG_PCI))
+ pci_request_acs();
+}
+
+int pci_xen_swiotlb_init_late(void)
+{
+ int rc;
+
+ if (xen_swiotlb)
+ return 0;
+
+ rc = xen_swiotlb_init();
+ if (rc)
+ return rc;
+
+ /* XXX: this switches the dma ops under live devices! */
+ dma_ops = &xen_swiotlb_dma_ops;
+ if (IS_ENABLED(CONFIG_PCI))
+ pci_request_acs();
+ return 0;
+}
+EXPORT_SYMBOL_GPL(pci_xen_swiotlb_init_late);
+#else
+static inline void __init pci_xen_swiotlb_init(void)
+{
+}
+#endif /* CONFIG_SWIOTLB_XEN */
void __init pci_iommu_alloc(void)
{
- struct iommu_table_entry *p;
-
- sort_iommu_table(__iommu_table, __iommu_table_end);
- check_iommu_entries(__iommu_table, __iommu_table_end);
-
- for (p = __iommu_table; p < __iommu_table_end; p++) {
- if (p && p->detect && p->detect() > 0) {
- p->flags |= IOMMU_DETECTED;
- if (p->early_init)
- p->early_init();
- if (p->flags & IOMMU_FINISH_IF_DETECTED)
- break;
- }
+ if (xen_pv_domain()) {
+ pci_xen_swiotlb_init();
+ return;
}
+ pci_swiotlb_detect_4gb();
+ gart_iommu_hole_init();
+ amd_iommu_detect();
+ detect_intel_iommu();
+ if (x86_swiotlb_enable)
+ swiotlb_init(0);
}
/*
@@ -102,7 +168,7 @@ static __init int iommu_setup(char *p)
}
#ifdef CONFIG_SWIOTLB
if (!strncmp(p, "soft", 4))
- swiotlb = 1;
+ x86_swiotlb_enable = 1;
#endif
if (!strncmp(p, "pt", 2))
iommu_set_default_passthrough(true);
@@ -121,14 +187,17 @@ early_param("iommu", iommu_setup);
static int __init pci_iommu_init(void)
{
- struct iommu_table_entry *p;
-
x86_init.iommu.iommu_init();
- for (p = __iommu_table; p < __iommu_table_end; p++) {
- if (p && (p->flags & IOMMU_DETECTED) && p->late_init)
- p->late_init();
+#ifdef CONFIG_SWIOTLB
+ /* An IOMMU turned us off. */
+ if (x86_swiotlb_enable) {
+ pr_info("PCI-DMA: Using software bounce buffering for IO (SWIOTLB)\n");
+ swiotlb_print_info();
+ } else {
+ swiotlb_exit();
}
+#endif
return 0;
}
diff --git a/arch/x86/kernel/pci-iommu_table.c b/arch/x86/kernel/pci-iommu_table.c
deleted file mode 100644
index 42e92ec62973b..0000000000000
--- a/arch/x86/kernel/pci-iommu_table.c
+++ /dev/null
@@ -1,77 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0
-#include <linux/dma-mapping.h>
-#include <asm/iommu_table.h>
-#include <linux/string.h>
-#include <linux/kallsyms.h>
-
-static struct iommu_table_entry * __init
-find_dependents_of(struct iommu_table_entry *start,
- struct iommu_table_entry *finish,
- struct iommu_table_entry *q)
-{
- struct iommu_table_entry *p;
-
- if (!q)
- return NULL;
-
- for (p = start; p < finish; p++)
- if (p->detect = q->depend)
- return p;
-
- return NULL;
-}
-
-
-void __init sort_iommu_table(struct iommu_table_entry *start,
- struct iommu_table_entry *finish) {
-
- struct iommu_table_entry *p, *q, tmp;
-
- for (p = start; p < finish; p++) {
-again:
- q = find_dependents_of(start, finish, p);
- /* We are bit sneaky here. We use the memory address to figure
- * out if the node we depend on is past our point, if so, swap.
- */
- if (q > p) {
- tmp = *p;
- memmove(p, q, sizeof(*p));
- *q = tmp;
- goto again;
- }
- }
-
-}
-
-#ifdef DEBUG
-void __init check_iommu_entries(struct iommu_table_entry *start,
- struct iommu_table_entry *finish)
-{
- struct iommu_table_entry *p, *q, *x;
-
- /* Simple cyclic dependency checker. */
- for (p = start; p < finish; p++) {
- q = find_dependents_of(start, finish, p);
- x = find_dependents_of(start, finish, q);
- if (p = x) {
- printk(KERN_ERR "CYCLIC DEPENDENCY FOUND! %pS depends on %pS and vice-versa. BREAKING IT.\n",
- p->detect, q->detect);
- /* Heavy handed way..*/
- x->depend = NULL;
- }
- }
-
- for (p = start; p < finish; p++) {
- q = find_dependents_of(p, finish, p);
- if (q && q > p) {
- printk(KERN_ERR "EXECUTION ORDER INVALID! %pS should be called before %pS!\n",
- p->detect, q->detect);
- }
- }
-}
-#else
-void __init check_iommu_entries(struct iommu_table_entry *start,
- struct iommu_table_entry *finish)
-{
-}
-#endif
diff --git a/arch/x86/kernel/pci-swiotlb.c b/arch/x86/kernel/pci-swiotlb.c
deleted file mode 100644
index 814ab46a0dada..0000000000000
--- a/arch/x86/kernel/pci-swiotlb.c
+++ /dev/null
@@ -1,77 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0
-
-#include <linux/pci.h>
-#include <linux/cache.h>
-#include <linux/init.h>
-#include <linux/swiotlb.h>
-#include <linux/memblock.h>
-#include <linux/dma-direct.h>
-#include <linux/cc_platform.h>
-
-#include <asm/iommu.h>
-#include <asm/swiotlb.h>
-#include <asm/dma.h>
-#include <asm/xen/swiotlb-xen.h>
-#include <asm/iommu_table.h>
-
-int swiotlb __read_mostly;
-
-/*
- * pci_swiotlb_detect_override - set swiotlb to 1 if necessary
- *
- * This returns non-zero if we are forced to use swiotlb (by the boot
- * option).
- */
-int __init pci_swiotlb_detect_override(void)
-{
- if (swiotlb_force = SWIOTLB_FORCE)
- swiotlb = 1;
-
- return swiotlb;
-}
-IOMMU_INIT_FINISH(pci_swiotlb_detect_override,
- pci_xen_swiotlb_detect,
- pci_swiotlb_init,
- pci_swiotlb_late_init);
-
-/*
- * If 4GB or more detected (and iommu=off not set) or if SME is active
- * then set swiotlb to 1 and return 1.
- */
-int __init pci_swiotlb_detect_4gb(void)
-{
- /* don't initialize swiotlb if iommu=off (no_iommu=1) */
- if (!no_iommu && max_possible_pfn > MAX_DMA32_PFN)
- swiotlb = 1;
-
- /*
- * Set swiotlb to 1 so that bounce buffers are allocated and used for
- * devices that can't support DMA to encrypted memory.
- */
- if (cc_platform_has(CC_ATTR_HOST_MEM_ENCRYPT))
- swiotlb = 1;
-
- return swiotlb;
-}
-IOMMU_INIT(pci_swiotlb_detect_4gb,
- pci_swiotlb_detect_override,
- pci_swiotlb_init,
- pci_swiotlb_late_init);
-
-void __init pci_swiotlb_init(void)
-{
- if (swiotlb)
- swiotlb_init(0);
-}
-
-void __init pci_swiotlb_late_init(void)
-{
- /* An IOMMU turned us off. */
- if (!swiotlb)
- swiotlb_exit();
- else {
- printk(KERN_INFO "PCI-DMA: "
- "Using software bounce buffering for IO (SWIOTLB)\n");
- swiotlb_print_info();
- }
-}
diff --git a/arch/x86/kernel/tboot.c b/arch/x86/kernel/tboot.c
index f9af561c3cd4f..0c1154a1c4032 100644
--- a/arch/x86/kernel/tboot.c
+++ b/arch/x86/kernel/tboot.c
@@ -24,7 +24,6 @@
#include <asm/processor.h>
#include <asm/bootparam.h>
#include <asm/pgalloc.h>
-#include <asm/swiotlb.h>
#include <asm/fixmap.h>
#include <asm/proto.h>
#include <asm/setup.h>
diff --git a/arch/x86/kernel/vmlinux.lds.S b/arch/x86/kernel/vmlinux.lds.S
index 27f830345b6f0..bbe910c15b293 100644
--- a/arch/x86/kernel/vmlinux.lds.S
+++ b/arch/x86/kernel/vmlinux.lds.S
@@ -306,18 +306,6 @@ SECTIONS
*(.altinstr_replacement)
}
- /*
- * struct iommu_table_entry entries are injected in this section.
- * It is an array of IOMMUs which during run time gets sorted depending
- * on its dependency order. After rootfs_initcall is complete
- * this section can be safely removed.
- */
- .iommu_table : AT(ADDR(.iommu_table) - LOAD_OFFSET) {
- __iommu_table = .;
- *(.iommu_table)
- __iommu_table_end = .;
- }
-
. = ALIGN(8);
.apicdrivers : AT(ADDR(.apicdrivers) - LOAD_OFFSET) {
__apicdrivers = .;
diff --git a/arch/x86/xen/Makefile b/arch/x86/xen/Makefile
index 4953260e281c3..3c5b52fbe4a7f 100644
--- a/arch/x86/xen/Makefile
+++ b/arch/x86/xen/Makefile
@@ -47,6 +47,4 @@ obj-$(CONFIG_XEN_DEBUG_FS) += debugfs.o
obj-$(CONFIG_XEN_PV_DOM0) += vga.o
-obj-$(CONFIG_SWIOTLB_XEN) += pci-swiotlb-xen.o
-
obj-$(CONFIG_XEN_EFI) += efi.o
diff --git a/arch/x86/xen/pci-swiotlb-xen.c b/arch/x86/xen/pci-swiotlb-xen.c
deleted file mode 100644
index 46df59aeaa06a..0000000000000
--- a/arch/x86/xen/pci-swiotlb-xen.c
+++ /dev/null
@@ -1,96 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0
-
-/* Glue code to lib/swiotlb-xen.c */
-
-#include <linux/dma-map-ops.h>
-#include <linux/pci.h>
-#include <xen/swiotlb-xen.h>
-
-#include <asm/xen/hypervisor.h>
-#include <xen/xen.h>
-#include <asm/iommu_table.h>
-
-
-#include <asm/xen/swiotlb-xen.h>
-#ifdef CONFIG_X86_64
-#include <asm/iommu.h>
-#include <asm/dma.h>
-#endif
-#include <linux/export.h>
-
-static int xen_swiotlb __read_mostly;
-
-/*
- * pci_xen_swiotlb_detect - set xen_swiotlb to 1 if necessary
- *
- * This returns non-zero if we are forced to use xen_swiotlb (by the boot
- * option).
- */
-int __init pci_xen_swiotlb_detect(void)
-{
-
- if (!xen_pv_domain())
- return 0;
-
- /* If running as PV guest, either iommu=soft, or swiotlb=force will
- * activate this IOMMU. If running as PV privileged, activate it
- * irregardless.
- */
- if (xen_initial_domain() || swiotlb || swiotlb_force = SWIOTLB_FORCE)
- xen_swiotlb = 1;
-
- /* If we are running under Xen, we MUST disable the native SWIOTLB.
- * Don't worry about swiotlb_force flag activating the native, as
- * the 'swiotlb' flag is the only one turning it on. */
- swiotlb = 0;
-
-#ifdef CONFIG_X86_64
- /* pci_swiotlb_detect_4gb turns on native SWIOTLB if no_iommu = 0
- * (so no iommu=X command line over-writes).
- * Considering that PV guests do not want the *native SWIOTLB* but
- * only Xen SWIOTLB it is not useful to us so set no_iommu=1 here.
- */
- if (max_pfn > MAX_DMA32_PFN)
- no_iommu = 1;
-#endif
- return xen_swiotlb;
-}
-
-static void __init pci_xen_swiotlb_init(void)
-{
- if (xen_swiotlb) {
- xen_swiotlb_init_early();
- dma_ops = &xen_swiotlb_dma_ops;
-
-#ifdef CONFIG_PCI
- /* Make sure ACS will be enabled */
- pci_request_acs();
-#endif
- }
-}
-
-int pci_xen_swiotlb_init_late(void)
-{
- int rc;
-
- if (xen_swiotlb)
- return 0;
-
- rc = xen_swiotlb_init();
- if (rc)
- return rc;
-
- dma_ops = &xen_swiotlb_dma_ops;
-#ifdef CONFIG_PCI
- /* Make sure ACS will be enabled */
- pci_request_acs();
-#endif
-
- return 0;
-}
-EXPORT_SYMBOL_GPL(pci_xen_swiotlb_init_late);
-
-IOMMU_INIT_FINISH(pci_xen_swiotlb_detect,
- NULL,
- pci_xen_swiotlb_init,
- NULL);
diff --git a/drivers/iommu/amd/init.c b/drivers/iommu/amd/init.c
index b10fb52ea4428..721300cf90207 100644
--- a/drivers/iommu/amd/init.c
+++ b/drivers/iommu/amd/init.c
@@ -27,7 +27,6 @@
#include <asm/apic.h>
#include <asm/gart.h>
#include <asm/x86_init.h>
-#include <asm/iommu_table.h>
#include <asm/io_apic.h>
#include <asm/irq_remapping.h>
#include <asm/set_memory.h>
@@ -3237,11 +3236,6 @@ __setup("ivrs_ioapic", parse_ivrs_ioapic);
__setup("ivrs_hpet", parse_ivrs_hpet);
__setup("ivrs_acpihid", parse_ivrs_acpihid);
-IOMMU_INIT_FINISH(amd_iommu_detect,
- gart_iommu_hole_init,
- NULL,
- NULL);
-
bool amd_iommu_v2_supported(void)
{
return amd_iommu_v2_present;
diff --git a/drivers/iommu/amd/iommu.c b/drivers/iommu/amd/iommu.c
index 461f1844ed1fb..541a7b8315a12 100644
--- a/drivers/iommu/amd/iommu.c
+++ b/drivers/iommu/amd/iommu.c
@@ -1834,7 +1834,10 @@ void amd_iommu_domain_update(struct protection_domain *domain)
static void __init amd_iommu_init_dma_ops(void)
{
- swiotlb = (iommu_default_passthrough() || sme_me_mask) ? 1 : 0;
+ if (iommu_default_passthrough() || sme_me_mask)
+ x86_swiotlb_enable = true;
+ else
+ x86_swiotlb_enable = false;
}
int __init amd_iommu_init_api(void)
diff --git a/drivers/iommu/intel/dmar.c b/drivers/iommu/intel/dmar.c
index 915bff76fe965..29bee4b210c5b 100644
--- a/drivers/iommu/intel/dmar.c
+++ b/drivers/iommu/intel/dmar.c
@@ -30,7 +30,6 @@
#include <linux/numa.h>
#include <linux/limits.h>
#include <asm/irq_remapping.h>
-#include <asm/iommu_table.h>
#include <trace/events/intel_iommu.h>
#include "../irq_remapping.h"
@@ -913,7 +912,7 @@ dmar_validate_one_drhd(struct acpi_dmar_header *entry, void *arg)
return 0;
}
-int __init detect_intel_iommu(void)
+void __init detect_intel_iommu(void)
{
int ret;
struct dmar_res_callback validate_drhd_cb = {
@@ -946,8 +945,6 @@ int __init detect_intel_iommu(void)
dmar_tbl = NULL;
}
up_write(&dmar_global_lock);
-
- return ret ? ret : 1;
}
static void unmap_iommu(struct intel_iommu *iommu)
@@ -2165,7 +2162,6 @@ static int __init dmar_free_unused_resources(void)
}
late_initcall(dmar_free_unused_resources);
-IOMMU_INIT_POST(detect_intel_iommu);
/*
* DMAR Hotplug Support
diff --git a/include/linux/dmar.h b/include/linux/dmar.h
index 45e903d847335..cbd714a198a0a 100644
--- a/include/linux/dmar.h
+++ b/include/linux/dmar.h
@@ -121,7 +121,7 @@ extern int dmar_remove_dev_scope(struct dmar_pci_notify_info *info,
u16 segment, struct dmar_dev_scope *devices,
int count);
/* Intel IOMMU detection */
-extern int detect_intel_iommu(void);
+void detect_intel_iommu(void);
extern int enable_drhd_fault_handling(void);
extern int dmar_device_add(acpi_handle handle);
extern int dmar_device_remove(acpi_handle handle);
@@ -197,6 +197,10 @@ static inline bool dmar_platform_optin(void)
return false;
}
+static inline void detect_intel_iommu(void)
+{
+}
+
#endif /* CONFIG_DMAR_TABLE */
struct irte {
--
2.30.2
^ permalink raw reply related [flat|nested] 118+ messages in thread
* [PATCH 08/11] swiotlb: make the swiotlb_init interface more useful
2022-02-27 14:30 ` Christoph Hellwig
` (3 preceding siblings ...)
(?)
@ 2022-02-27 14:30 ` Christoph Hellwig
-1 siblings, 0 replies; 118+ messages in thread
From: Christoph Hellwig @ 2022-02-27 14:30 UTC (permalink / raw)
To: iommu
Cc: x86, Anshuman Khandual, Konrad Rzeszutek Wilk,
Stefano Stabellini, Boris Ostrovsky, Juergen Gross, Joerg Roedel,
David Woodhouse, Lu Baolu, Robin Murphy, linux-arm-kernel,
xen-devel, linux-ia64, linux-mips, linuxppc-dev, linux-riscv,
linux-s390, linux-hyperv, tboot-devel, linux-pci
Pass a bool to pass if swiotlb needs to be enabled based on the
addressing needs and replace the verbose argument with a set of
flags, including one to force enable bounce buffering.
Note that this patch removes the possibility to force xen-swiotlb
use using swiotlb=force on the command line on x86 (arm and arm64
never supported that), but this interface will be restored shortly.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
arch/arm/mm/init.c | 6 +----
arch/arm64/mm/init.c | 6 +----
arch/ia64/mm/init.c | 4 +--
arch/mips/cavium-octeon/dma-octeon.c | 2 +-
arch/mips/loongson64/dma.c | 2 +-
arch/mips/sibyte/common/dma.c | 2 +-
arch/powerpc/include/asm/swiotlb.h | 1 +
arch/powerpc/mm/mem.c | 3 ++-
arch/powerpc/platforms/pseries/setup.c | 3 ---
arch/riscv/mm/init.c | 8 +-----
arch/s390/mm/init.c | 3 +--
arch/x86/kernel/cpu/mshyperv.c | 8 ------
arch/x86/kernel/pci-dma.c | 15 ++++++-----
arch/x86/mm/mem_encrypt_amd.c | 3 ---
drivers/xen/swiotlb-xen.c | 4 +--
include/linux/swiotlb.h | 15 ++++++-----
include/trace/events/swiotlb.h | 29 ++++++++-------------
kernel/dma/swiotlb.c | 35 ++++++++++++++------------
18 files changed, 56 insertions(+), 93 deletions(-)
diff --git a/arch/arm/mm/init.c b/arch/arm/mm/init.c
index 6d0cb0f7bc54b..73f30d278b565 100644
--- a/arch/arm/mm/init.c
+++ b/arch/arm/mm/init.c
@@ -312,11 +312,7 @@ static void __init free_highpages(void)
void __init mem_init(void)
{
#ifdef CONFIG_ARM_LPAE
- if (swiotlb_force == SWIOTLB_FORCE ||
- max_pfn > arm_dma_pfn_limit)
- swiotlb_init(1);
- else
- swiotlb_force = SWIOTLB_NO_FORCE;
+ swiotlb_init(max_pfn > arm_dma_pfn_limit, SWIOTLB_VERBOSE);
#endif
set_max_mapnr(pfn_to_page(max_pfn) - mem_map);
diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c
index db63cc885771a..52102adda3d28 100644
--- a/arch/arm64/mm/init.c
+++ b/arch/arm64/mm/init.c
@@ -373,11 +373,7 @@ void __init bootmem_init(void)
*/
void __init mem_init(void)
{
- if (swiotlb_force == SWIOTLB_FORCE ||
- max_pfn > PFN_DOWN(arm64_dma_phys_limit))
- swiotlb_init(1);
- else if (!xen_swiotlb_detect())
- swiotlb_force = SWIOTLB_NO_FORCE;
+ swiotlb_init(max_pfn > PFN_DOWN(arm64_dma_phys_limit), SWIOTLB_VERBOSE);
/* this will put all unused low memory onto the freelists */
memblock_free_all();
diff --git a/arch/ia64/mm/init.c b/arch/ia64/mm/init.c
index 5d165607bf354..3c3e15b22608f 100644
--- a/arch/ia64/mm/init.c
+++ b/arch/ia64/mm/init.c
@@ -437,9 +437,7 @@ mem_init (void)
if (iommu_detected)
break;
#endif
-#ifdef CONFIG_SWIOTLB
- swiotlb_init(1);
-#endif
+ swiotlb_init(true, SWIOTLB_VERBOSE);
} while (0);
#ifdef CONFIG_FLATMEM
diff --git a/arch/mips/cavium-octeon/dma-octeon.c b/arch/mips/cavium-octeon/dma-octeon.c
index fb7547e217263..9fbba6a8fa4c5 100644
--- a/arch/mips/cavium-octeon/dma-octeon.c
+++ b/arch/mips/cavium-octeon/dma-octeon.c
@@ -235,5 +235,5 @@ void __init plat_swiotlb_setup(void)
#endif
swiotlb_adjust_size(swiotlbsize);
- swiotlb_init(1);
+ swiotlb_init(true, SWIOTLB_VERBOSE);
}
diff --git a/arch/mips/loongson64/dma.c b/arch/mips/loongson64/dma.c
index 364f2f27c8723..8220a1bc0db64 100644
--- a/arch/mips/loongson64/dma.c
+++ b/arch/mips/loongson64/dma.c
@@ -24,5 +24,5 @@ phys_addr_t dma_to_phys(struct device *dev, dma_addr_t daddr)
void __init plat_swiotlb_setup(void)
{
- swiotlb_init(1);
+ swiotlb_init(true, SWIOTLB_VERBOSE);
}
diff --git a/arch/mips/sibyte/common/dma.c b/arch/mips/sibyte/common/dma.c
index eb47a94f3583e..c5c2c782aff68 100644
--- a/arch/mips/sibyte/common/dma.c
+++ b/arch/mips/sibyte/common/dma.c
@@ -10,5 +10,5 @@
void __init plat_swiotlb_setup(void)
{
- swiotlb_init(1);
+ swiotlb_init(true, SWIOTLB_VERBOSE);
}
diff --git a/arch/powerpc/include/asm/swiotlb.h b/arch/powerpc/include/asm/swiotlb.h
index 3c1a1cd161286..4203b5e0a88ed 100644
--- a/arch/powerpc/include/asm/swiotlb.h
+++ b/arch/powerpc/include/asm/swiotlb.h
@@ -9,6 +9,7 @@
#include <linux/swiotlb.h>
extern unsigned int ppc_swiotlb_enable;
+extern unsigned int ppc_swiotlb_flags;
#ifdef CONFIG_SWIOTLB
void swiotlb_detect_4g(void);
diff --git a/arch/powerpc/mm/mem.c b/arch/powerpc/mm/mem.c
index 8e301cd8925b2..d99b8b5b40ca6 100644
--- a/arch/powerpc/mm/mem.c
+++ b/arch/powerpc/mm/mem.c
@@ -17,6 +17,7 @@
#include <linux/suspend.h>
#include <linux/dma-direct.h>
+#include <asm/swiotlb.h>
#include <asm/machdep.h>
#include <asm/rtas.h>
#include <asm/kasan.h>
@@ -251,7 +252,7 @@ void __init mem_init(void)
if (is_secure_guest())
svm_swiotlb_init();
else
- swiotlb_init(0);
+ swiotlb_init(ppc_swiotlb_enable, ppc_swiotlb_flags);
#endif
high_memory = (void *) __va(max_low_pfn * PAGE_SIZE);
diff --git a/arch/powerpc/platforms/pseries/setup.c b/arch/powerpc/platforms/pseries/setup.c
index 83a04d967a59f..45d637ab58261 100644
--- a/arch/powerpc/platforms/pseries/setup.c
+++ b/arch/powerpc/platforms/pseries/setup.c
@@ -838,9 +838,6 @@ static void __init pSeries_setup_arch(void)
}
ppc_md.pcibios_root_bridge_prepare = pseries_root_bridge_prepare;
-
- if (swiotlb_force == SWIOTLB_FORCE)
- ppc_swiotlb_enable = 1;
}
static void pseries_panic(char *str)
diff --git a/arch/riscv/mm/init.c b/arch/riscv/mm/init.c
index c27294128e182..6cdbb62672fe5 100644
--- a/arch/riscv/mm/init.c
+++ b/arch/riscv/mm/init.c
@@ -118,13 +118,7 @@ void __init mem_init(void)
BUG_ON(!mem_map);
#endif /* CONFIG_FLATMEM */
-#ifdef CONFIG_SWIOTLB
- if (swiotlb_force == SWIOTLB_FORCE ||
- max_pfn > PFN_DOWN(dma32_phys_limit))
- swiotlb_init(1);
- else
- swiotlb_force = SWIOTLB_NO_FORCE;
-#endif
+ swiotlb_init(max_pfn > PFN_DOWN(dma32_phys_limit), SWIOTLB_VERBOSE);
high_memory = (void *)(__va(PFN_PHYS(max_low_pfn)));
memblock_free_all();
diff --git a/arch/s390/mm/init.c b/arch/s390/mm/init.c
index 86ffd0d51fd59..6fb6bf64326f9 100644
--- a/arch/s390/mm/init.c
+++ b/arch/s390/mm/init.c
@@ -185,8 +185,7 @@ static void pv_init(void)
return;
/* make sure bounce buffers are shared */
- swiotlb_force = SWIOTLB_FORCE;
- swiotlb_init(1);
+ swiotlb_init(true, SWIOTLB_FORCE | SWIOTLB_VERBOSE);
swiotlb_update_mem_attributes();
}
diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c
index 5a99f993e6392..568274917f1cd 100644
--- a/arch/x86/kernel/cpu/mshyperv.c
+++ b/arch/x86/kernel/cpu/mshyperv.c
@@ -336,14 +336,6 @@ static void __init ms_hyperv_init_platform(void)
swiotlb_unencrypted_base = ms_hyperv.shared_gpa_boundary;
#endif
}
-
-#ifdef CONFIG_SWIOTLB
- /*
- * Enable swiotlb force mode in Isolation VM to
- * use swiotlb bounce buffer for dma transaction.
- */
- swiotlb_force = SWIOTLB_FORCE;
-#endif
}
if (hv_max_functions_eax >= HYPERV_CPUID_NESTED_FEATURES) {
diff --git a/arch/x86/kernel/pci-dma.c b/arch/x86/kernel/pci-dma.c
index 17bcac942ac49..bb08184a50e3a 100644
--- a/arch/x86/kernel/pci-dma.c
+++ b/arch/x86/kernel/pci-dma.c
@@ -39,6 +39,7 @@ int iommu_detected __read_mostly = 0;
#ifdef CONFIG_SWIOTLB
bool x86_swiotlb_enable;
+static unsigned int x86_swiotlb_flags;
/*
* If 4GB or more detected (and iommu=off not set) or if SME is active
@@ -54,16 +55,16 @@ static void __init pci_swiotlb_detect_4gb(void)
* Set swiotlb to 1 so that bounce buffers are allocated and used for
* devices that can't support DMA to encrypted memory.
*/
- if (cc_platform_has(CC_ATTR_HOST_MEM_ENCRYPT))
- x86_swiotlb_enable = true;
-
- if (swiotlb_force == SWIOTLB_FORCE)
+ if (cc_platform_has(CC_ATTR_HOST_MEM_ENCRYPT)) {
x86_swiotlb_enable = true;
+ x86_swiotlb_flags |= SWIOTLB_FORCE;
+ }
}
#else
static inline void __init pci_swiotlb_detect_4gb(void)
{
}
+#define x86_swiotlb_flags 0
#endif /* CONFIG_SWIOTLB */
#ifdef CONFIG_SWIOTLB_XEN
@@ -71,8 +72,7 @@ static bool xen_swiotlb;
static void __init pci_xen_swiotlb_init(void)
{
- if (!xen_initial_domain() && !x86_swiotlb_enable &&
- swiotlb_force != SWIOTLB_FORCE)
+ if (!xen_initial_domain() && !x86_swiotlb_enable)
return;
x86_swiotlb_enable = false;
xen_swiotlb = true;
@@ -116,8 +116,7 @@ void __init pci_iommu_alloc(void)
gart_iommu_hole_init();
amd_iommu_detect();
detect_intel_iommu();
- if (x86_swiotlb_enable)
- swiotlb_init(0);
+ swiotlb_init(x86_swiotlb_enable, x86_swiotlb_flags);
}
/*
diff --git a/arch/x86/mm/mem_encrypt_amd.c b/arch/x86/mm/mem_encrypt_amd.c
index 2b2d018ea3450..a72942d569cf9 100644
--- a/arch/x86/mm/mem_encrypt_amd.c
+++ b/arch/x86/mm/mem_encrypt_amd.c
@@ -191,9 +191,6 @@ void __init sme_early_init(void)
/* Update the protection map with memory encryption mask */
for (i = 0; i < ARRAY_SIZE(protection_map); i++)
protection_map[i] = pgprot_encrypted(protection_map[i]);
-
- if (cc_platform_has(CC_ATTR_GUEST_MEM_ENCRYPT))
- swiotlb_force = SWIOTLB_FORCE;
}
void __init sev_setup_arch(void)
diff --git a/drivers/xen/swiotlb-xen.c b/drivers/xen/swiotlb-xen.c
index 485cd06ed39e7..c2da3eb4826e8 100644
--- a/drivers/xen/swiotlb-xen.c
+++ b/drivers/xen/swiotlb-xen.c
@@ -251,7 +251,7 @@ void __init xen_swiotlb_init_early(void)
panic("%s (rc:%d)", xen_swiotlb_error(XEN_SWIOTLB_EFIXUP), rc);
}
- if (swiotlb_init_with_tbl(start, nslabs, true))
+ if (swiotlb_init_with_tbl(start, nslabs, SWIOTLB_VERBOSE))
panic("Cannot allocate SWIOTLB buffer");
}
#endif /* CONFIG_X86 */
@@ -376,7 +376,7 @@ static dma_addr_t xen_swiotlb_map_page(struct device *dev, struct page *page,
/*
* Oh well, have to allocate and map a bounce buffer.
*/
- trace_swiotlb_bounced(dev, dev_addr, size, swiotlb_force);
+ trace_swiotlb_bounced(dev, dev_addr, size);
map = swiotlb_tbl_map_single(dev, phys, size, size, 0, dir, attrs);
if (map == (phys_addr_t)DMA_MAPPING_ERROR)
diff --git a/include/linux/swiotlb.h b/include/linux/swiotlb.h
index 1befd6b2ccf5e..dcecf953f7997 100644
--- a/include/linux/swiotlb.h
+++ b/include/linux/swiotlb.h
@@ -13,11 +13,8 @@ struct device;
struct page;
struct scatterlist;
-enum swiotlb_force {
- SWIOTLB_NORMAL, /* Default - depending on HW DMA mask etc. */
- SWIOTLB_FORCE, /* swiotlb=force */
- SWIOTLB_NO_FORCE, /* swiotlb=noforce */
-};
+#define SWIOTLB_VERBOSE (1 << 0) /* verbose initialization */
+#define SWIOTLB_FORCE (1 << 1) /* force bounce buffering */
/*
* Maximum allowable number of contiguous slabs to map,
@@ -36,8 +33,7 @@ enum swiotlb_force {
/* default to 64MB */
#define IO_TLB_DEFAULT_SIZE (64UL<<20)
-extern void swiotlb_init(int verbose);
-int swiotlb_init_with_tbl(char *tlb, unsigned long nslabs, int verbose);
+int swiotlb_init_with_tbl(char *tlb, unsigned long nslabs, unsigned int flags);
unsigned long swiotlb_size_or_default(void);
extern int swiotlb_late_init_with_tbl(char *tlb, unsigned long nslabs);
int swiotlb_init_late(size_t size, gfp_t gfp_mask);
@@ -126,13 +122,16 @@ static inline bool is_swiotlb_force_bounce(struct device *dev)
return mem && mem->force_bounce;
}
+void swiotlb_init(bool addressing_limited, unsigned int flags);
void __init swiotlb_exit(void);
unsigned int swiotlb_max_segment(void);
size_t swiotlb_max_mapping_size(struct device *dev);
bool is_swiotlb_active(struct device *dev);
void __init swiotlb_adjust_size(unsigned long size);
#else
-#define swiotlb_force SWIOTLB_NO_FORCE
+static inline void swiotlb_init(bool addressing_limited, unsigned int flags)
+{
+}
static inline bool is_swiotlb_buffer(struct device *dev, phys_addr_t paddr)
{
return false;
diff --git a/include/trace/events/swiotlb.h b/include/trace/events/swiotlb.h
index 705be43b71ab0..da05c9ebd224a 100644
--- a/include/trace/events/swiotlb.h
+++ b/include/trace/events/swiotlb.h
@@ -8,20 +8,15 @@
#include <linux/tracepoint.h>
TRACE_EVENT(swiotlb_bounced,
-
- TP_PROTO(struct device *dev,
- dma_addr_t dev_addr,
- size_t size,
- enum swiotlb_force swiotlb_force),
-
- TP_ARGS(dev, dev_addr, size, swiotlb_force),
+ TP_PROTO(struct device *dev, dma_addr_t dev_addr, size_t size),
+ TP_ARGS(dev, dev_addr, size),
TP_STRUCT__entry(
- __string( dev_name, dev_name(dev) )
- __field( u64, dma_mask )
- __field( dma_addr_t, dev_addr )
- __field( size_t, size )
- __field( enum swiotlb_force, swiotlb_force )
+ __string(dev_name, dev_name(dev))
+ __field(u64, dma_mask)
+ __field(dma_addr_t, dev_addr)
+ __field(size_t, size)
+ __field(bool, force)
),
TP_fast_assign(
@@ -29,19 +24,15 @@ TRACE_EVENT(swiotlb_bounced,
__entry->dma_mask = (dev->dma_mask ? *dev->dma_mask : 0);
__entry->dev_addr = dev_addr;
__entry->size = size;
- __entry->swiotlb_force = swiotlb_force;
+ __entry->force = is_swiotlb_force_bounce(dev);
),
- TP_printk("dev_name: %s dma_mask=%llx dev_addr=%llx "
- "size=%zu %s",
+ TP_printk("dev_name: %s dma_mask=%llx dev_addr=%llx size=%zu %s",
__get_str(dev_name),
__entry->dma_mask,
(unsigned long long)__entry->dev_addr,
__entry->size,
- __print_symbolic(__entry->swiotlb_force,
- { SWIOTLB_NORMAL, "NORMAL" },
- { SWIOTLB_FORCE, "FORCE" },
- { SWIOTLB_NO_FORCE, "NO_FORCE" }))
+ __entry->force ? "FORCE" : "NORMAL")
);
#endif /* _TRACE_SWIOTLB_H */
diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
index a653fcf1fe6c2..ad604e5a0983d 100644
--- a/kernel/dma/swiotlb.c
+++ b/kernel/dma/swiotlb.c
@@ -69,7 +69,8 @@
#define INVALID_PHYS_ADDR (~(phys_addr_t)0)
-enum swiotlb_force swiotlb_force;
+static bool swiotlb_force_bounce;
+static bool swiotlb_force_disable;
struct io_tlb_mem io_tlb_default_mem;
@@ -88,9 +89,9 @@ setup_io_tlb_npages(char *str)
if (*str == ',')
++str;
if (!strcmp(str, "force"))
- swiotlb_force = SWIOTLB_FORCE;
+ swiotlb_force_bounce = true;
else if (!strcmp(str, "noforce"))
- swiotlb_force = SWIOTLB_NO_FORCE;
+ swiotlb_force_disable = true;
return 0;
}
@@ -211,7 +212,7 @@ static void swiotlb_init_io_tlb_mem(struct io_tlb_mem *mem, phys_addr_t start,
mem->index = 0;
mem->late_alloc = late_alloc;
- if (swiotlb_force == SWIOTLB_FORCE)
+ if (swiotlb_force_bounce)
mem->force_bounce = true;
spin_lock_init(&mem->lock);
@@ -233,12 +234,13 @@ static void swiotlb_init_io_tlb_mem(struct io_tlb_mem *mem, phys_addr_t start,
return;
}
-int __init swiotlb_init_with_tbl(char *tlb, unsigned long nslabs, int verbose)
+int __init swiotlb_init_with_tbl(char *tlb, unsigned long nslabs,
+ unsigned int flags)
{
struct io_tlb_mem *mem = &io_tlb_default_mem;
size_t alloc_size;
- if (swiotlb_force == SWIOTLB_NO_FORCE)
+ if (swiotlb_force_disable)
return 0;
/* protect against double initialization */
@@ -252,8 +254,9 @@ int __init swiotlb_init_with_tbl(char *tlb, unsigned long nslabs, int verbose)
__func__, alloc_size, PAGE_SIZE);
swiotlb_init_io_tlb_mem(mem, __pa(tlb), nslabs, false);
+ mem->force_bounce = flags & SWIOTLB_FORCE;
- if (verbose)
+ if (flags & SWIOTLB_VERBOSE)
swiotlb_print_info();
return 0;
}
@@ -262,20 +265,21 @@ int __init swiotlb_init_with_tbl(char *tlb, unsigned long nslabs, int verbose)
* Statically reserve bounce buffer space and initialize bounce buffer data
* structures for the software IO TLB used to implement the DMA API.
*/
-void __init
-swiotlb_init(int verbose)
+void __init swiotlb_init(bool addressing_limit, unsigned int flags)
{
size_t bytes = PAGE_ALIGN(default_nslabs << IO_TLB_SHIFT);
void *tlb;
- if (swiotlb_force == SWIOTLB_NO_FORCE)
+ if (!addressing_limit && !swiotlb_force_bounce)
+ return;
+ if (swiotlb_force_disable)
return;
/* Get IO TLB memory from the low pages */
tlb = memblock_alloc_low(bytes, PAGE_SIZE);
if (!tlb)
goto fail;
- if (swiotlb_init_with_tbl(tlb, default_nslabs, verbose))
+ if (swiotlb_init_with_tbl(tlb, default_nslabs, flags))
goto fail_free_mem;
return;
@@ -298,7 +302,7 @@ int swiotlb_init_late(size_t size, gfp_t gfp_mask)
unsigned int order;
int rc = 0;
- if (swiotlb_force == SWIOTLB_NO_FORCE)
+ if (swiotlb_force_disable)
return 0;
/*
@@ -337,7 +341,7 @@ swiotlb_late_init_with_tbl(char *tlb, unsigned long nslabs)
struct io_tlb_mem *mem = &io_tlb_default_mem;
unsigned long bytes = nslabs << IO_TLB_SHIFT;
- if (swiotlb_force == SWIOTLB_NO_FORCE)
+ if (swiotlb_force_disable)
return 0;
/* protect against double initialization */
@@ -362,7 +366,7 @@ void __init swiotlb_exit(void)
unsigned long tbl_vaddr;
size_t tbl_size, slots_size;
- if (swiotlb_force == SWIOTLB_FORCE)
+ if (swiotlb_force_bounce)
return;
if (!mem->nslabs)
@@ -708,8 +712,7 @@ dma_addr_t swiotlb_map(struct device *dev, phys_addr_t paddr, size_t size,
phys_addr_t swiotlb_addr;
dma_addr_t dma_addr;
- trace_swiotlb_bounced(dev, phys_to_dma(dev, paddr), size,
- swiotlb_force);
+ trace_swiotlb_bounced(dev, phys_to_dma(dev, paddr), size);
swiotlb_addr = swiotlb_tbl_map_single(dev, paddr, size, size, 0, dir,
attrs);
--
2.30.2
^ permalink raw reply related [flat|nested] 118+ messages in thread
* [PATCH 08/11] swiotlb: make the swiotlb_init interface more useful
@ 2022-02-27 14:30 ` Christoph Hellwig
0 siblings, 0 replies; 118+ messages in thread
From: Christoph Hellwig @ 2022-02-27 14:30 UTC (permalink / raw)
To: iommu
Cc: Juergen Gross, linux-s390, linux-hyperv, Stefano Stabellini,
linux-ia64, Anshuman Khandual, Konrad Rzeszutek Wilk,
Robin Murphy, x86, linux-mips, linuxppc-dev, tboot-devel,
linux-pci, xen-devel, Boris Ostrovsky, David Woodhouse,
linux-riscv, linux-arm-kernel
Pass a bool to pass if swiotlb needs to be enabled based on the
addressing needs and replace the verbose argument with a set of
flags, including one to force enable bounce buffering.
Note that this patch removes the possibility to force xen-swiotlb
use using swiotlb=force on the command line on x86 (arm and arm64
never supported that), but this interface will be restored shortly.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
arch/arm/mm/init.c | 6 +----
arch/arm64/mm/init.c | 6 +----
arch/ia64/mm/init.c | 4 +--
arch/mips/cavium-octeon/dma-octeon.c | 2 +-
arch/mips/loongson64/dma.c | 2 +-
arch/mips/sibyte/common/dma.c | 2 +-
arch/powerpc/include/asm/swiotlb.h | 1 +
arch/powerpc/mm/mem.c | 3 ++-
arch/powerpc/platforms/pseries/setup.c | 3 ---
arch/riscv/mm/init.c | 8 +-----
arch/s390/mm/init.c | 3 +--
arch/x86/kernel/cpu/mshyperv.c | 8 ------
arch/x86/kernel/pci-dma.c | 15 ++++++-----
arch/x86/mm/mem_encrypt_amd.c | 3 ---
drivers/xen/swiotlb-xen.c | 4 +--
include/linux/swiotlb.h | 15 ++++++-----
include/trace/events/swiotlb.h | 29 ++++++++-------------
kernel/dma/swiotlb.c | 35 ++++++++++++++------------
18 files changed, 56 insertions(+), 93 deletions(-)
diff --git a/arch/arm/mm/init.c b/arch/arm/mm/init.c
index 6d0cb0f7bc54b..73f30d278b565 100644
--- a/arch/arm/mm/init.c
+++ b/arch/arm/mm/init.c
@@ -312,11 +312,7 @@ static void __init free_highpages(void)
void __init mem_init(void)
{
#ifdef CONFIG_ARM_LPAE
- if (swiotlb_force == SWIOTLB_FORCE ||
- max_pfn > arm_dma_pfn_limit)
- swiotlb_init(1);
- else
- swiotlb_force = SWIOTLB_NO_FORCE;
+ swiotlb_init(max_pfn > arm_dma_pfn_limit, SWIOTLB_VERBOSE);
#endif
set_max_mapnr(pfn_to_page(max_pfn) - mem_map);
diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c
index db63cc885771a..52102adda3d28 100644
--- a/arch/arm64/mm/init.c
+++ b/arch/arm64/mm/init.c
@@ -373,11 +373,7 @@ void __init bootmem_init(void)
*/
void __init mem_init(void)
{
- if (swiotlb_force == SWIOTLB_FORCE ||
- max_pfn > PFN_DOWN(arm64_dma_phys_limit))
- swiotlb_init(1);
- else if (!xen_swiotlb_detect())
- swiotlb_force = SWIOTLB_NO_FORCE;
+ swiotlb_init(max_pfn > PFN_DOWN(arm64_dma_phys_limit), SWIOTLB_VERBOSE);
/* this will put all unused low memory onto the freelists */
memblock_free_all();
diff --git a/arch/ia64/mm/init.c b/arch/ia64/mm/init.c
index 5d165607bf354..3c3e15b22608f 100644
--- a/arch/ia64/mm/init.c
+++ b/arch/ia64/mm/init.c
@@ -437,9 +437,7 @@ mem_init (void)
if (iommu_detected)
break;
#endif
-#ifdef CONFIG_SWIOTLB
- swiotlb_init(1);
-#endif
+ swiotlb_init(true, SWIOTLB_VERBOSE);
} while (0);
#ifdef CONFIG_FLATMEM
diff --git a/arch/mips/cavium-octeon/dma-octeon.c b/arch/mips/cavium-octeon/dma-octeon.c
index fb7547e217263..9fbba6a8fa4c5 100644
--- a/arch/mips/cavium-octeon/dma-octeon.c
+++ b/arch/mips/cavium-octeon/dma-octeon.c
@@ -235,5 +235,5 @@ void __init plat_swiotlb_setup(void)
#endif
swiotlb_adjust_size(swiotlbsize);
- swiotlb_init(1);
+ swiotlb_init(true, SWIOTLB_VERBOSE);
}
diff --git a/arch/mips/loongson64/dma.c b/arch/mips/loongson64/dma.c
index 364f2f27c8723..8220a1bc0db64 100644
--- a/arch/mips/loongson64/dma.c
+++ b/arch/mips/loongson64/dma.c
@@ -24,5 +24,5 @@ phys_addr_t dma_to_phys(struct device *dev, dma_addr_t daddr)
void __init plat_swiotlb_setup(void)
{
- swiotlb_init(1);
+ swiotlb_init(true, SWIOTLB_VERBOSE);
}
diff --git a/arch/mips/sibyte/common/dma.c b/arch/mips/sibyte/common/dma.c
index eb47a94f3583e..c5c2c782aff68 100644
--- a/arch/mips/sibyte/common/dma.c
+++ b/arch/mips/sibyte/common/dma.c
@@ -10,5 +10,5 @@
void __init plat_swiotlb_setup(void)
{
- swiotlb_init(1);
+ swiotlb_init(true, SWIOTLB_VERBOSE);
}
diff --git a/arch/powerpc/include/asm/swiotlb.h b/arch/powerpc/include/asm/swiotlb.h
index 3c1a1cd161286..4203b5e0a88ed 100644
--- a/arch/powerpc/include/asm/swiotlb.h
+++ b/arch/powerpc/include/asm/swiotlb.h
@@ -9,6 +9,7 @@
#include <linux/swiotlb.h>
extern unsigned int ppc_swiotlb_enable;
+extern unsigned int ppc_swiotlb_flags;
#ifdef CONFIG_SWIOTLB
void swiotlb_detect_4g(void);
diff --git a/arch/powerpc/mm/mem.c b/arch/powerpc/mm/mem.c
index 8e301cd8925b2..d99b8b5b40ca6 100644
--- a/arch/powerpc/mm/mem.c
+++ b/arch/powerpc/mm/mem.c
@@ -17,6 +17,7 @@
#include <linux/suspend.h>
#include <linux/dma-direct.h>
+#include <asm/swiotlb.h>
#include <asm/machdep.h>
#include <asm/rtas.h>
#include <asm/kasan.h>
@@ -251,7 +252,7 @@ void __init mem_init(void)
if (is_secure_guest())
svm_swiotlb_init();
else
- swiotlb_init(0);
+ swiotlb_init(ppc_swiotlb_enable, ppc_swiotlb_flags);
#endif
high_memory = (void *) __va(max_low_pfn * PAGE_SIZE);
diff --git a/arch/powerpc/platforms/pseries/setup.c b/arch/powerpc/platforms/pseries/setup.c
index 83a04d967a59f..45d637ab58261 100644
--- a/arch/powerpc/platforms/pseries/setup.c
+++ b/arch/powerpc/platforms/pseries/setup.c
@@ -838,9 +838,6 @@ static void __init pSeries_setup_arch(void)
}
ppc_md.pcibios_root_bridge_prepare = pseries_root_bridge_prepare;
-
- if (swiotlb_force == SWIOTLB_FORCE)
- ppc_swiotlb_enable = 1;
}
static void pseries_panic(char *str)
diff --git a/arch/riscv/mm/init.c b/arch/riscv/mm/init.c
index c27294128e182..6cdbb62672fe5 100644
--- a/arch/riscv/mm/init.c
+++ b/arch/riscv/mm/init.c
@@ -118,13 +118,7 @@ void __init mem_init(void)
BUG_ON(!mem_map);
#endif /* CONFIG_FLATMEM */
-#ifdef CONFIG_SWIOTLB
- if (swiotlb_force == SWIOTLB_FORCE ||
- max_pfn > PFN_DOWN(dma32_phys_limit))
- swiotlb_init(1);
- else
- swiotlb_force = SWIOTLB_NO_FORCE;
-#endif
+ swiotlb_init(max_pfn > PFN_DOWN(dma32_phys_limit), SWIOTLB_VERBOSE);
high_memory = (void *)(__va(PFN_PHYS(max_low_pfn)));
memblock_free_all();
diff --git a/arch/s390/mm/init.c b/arch/s390/mm/init.c
index 86ffd0d51fd59..6fb6bf64326f9 100644
--- a/arch/s390/mm/init.c
+++ b/arch/s390/mm/init.c
@@ -185,8 +185,7 @@ static void pv_init(void)
return;
/* make sure bounce buffers are shared */
- swiotlb_force = SWIOTLB_FORCE;
- swiotlb_init(1);
+ swiotlb_init(true, SWIOTLB_FORCE | SWIOTLB_VERBOSE);
swiotlb_update_mem_attributes();
}
diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c
index 5a99f993e6392..568274917f1cd 100644
--- a/arch/x86/kernel/cpu/mshyperv.c
+++ b/arch/x86/kernel/cpu/mshyperv.c
@@ -336,14 +336,6 @@ static void __init ms_hyperv_init_platform(void)
swiotlb_unencrypted_base = ms_hyperv.shared_gpa_boundary;
#endif
}
-
-#ifdef CONFIG_SWIOTLB
- /*
- * Enable swiotlb force mode in Isolation VM to
- * use swiotlb bounce buffer for dma transaction.
- */
- swiotlb_force = SWIOTLB_FORCE;
-#endif
}
if (hv_max_functions_eax >= HYPERV_CPUID_NESTED_FEATURES) {
diff --git a/arch/x86/kernel/pci-dma.c b/arch/x86/kernel/pci-dma.c
index 17bcac942ac49..bb08184a50e3a 100644
--- a/arch/x86/kernel/pci-dma.c
+++ b/arch/x86/kernel/pci-dma.c
@@ -39,6 +39,7 @@ int iommu_detected __read_mostly = 0;
#ifdef CONFIG_SWIOTLB
bool x86_swiotlb_enable;
+static unsigned int x86_swiotlb_flags;
/*
* If 4GB or more detected (and iommu=off not set) or if SME is active
@@ -54,16 +55,16 @@ static void __init pci_swiotlb_detect_4gb(void)
* Set swiotlb to 1 so that bounce buffers are allocated and used for
* devices that can't support DMA to encrypted memory.
*/
- if (cc_platform_has(CC_ATTR_HOST_MEM_ENCRYPT))
- x86_swiotlb_enable = true;
-
- if (swiotlb_force == SWIOTLB_FORCE)
+ if (cc_platform_has(CC_ATTR_HOST_MEM_ENCRYPT)) {
x86_swiotlb_enable = true;
+ x86_swiotlb_flags |= SWIOTLB_FORCE;
+ }
}
#else
static inline void __init pci_swiotlb_detect_4gb(void)
{
}
+#define x86_swiotlb_flags 0
#endif /* CONFIG_SWIOTLB */
#ifdef CONFIG_SWIOTLB_XEN
@@ -71,8 +72,7 @@ static bool xen_swiotlb;
static void __init pci_xen_swiotlb_init(void)
{
- if (!xen_initial_domain() && !x86_swiotlb_enable &&
- swiotlb_force != SWIOTLB_FORCE)
+ if (!xen_initial_domain() && !x86_swiotlb_enable)
return;
x86_swiotlb_enable = false;
xen_swiotlb = true;
@@ -116,8 +116,7 @@ void __init pci_iommu_alloc(void)
gart_iommu_hole_init();
amd_iommu_detect();
detect_intel_iommu();
- if (x86_swiotlb_enable)
- swiotlb_init(0);
+ swiotlb_init(x86_swiotlb_enable, x86_swiotlb_flags);
}
/*
diff --git a/arch/x86/mm/mem_encrypt_amd.c b/arch/x86/mm/mem_encrypt_amd.c
index 2b2d018ea3450..a72942d569cf9 100644
--- a/arch/x86/mm/mem_encrypt_amd.c
+++ b/arch/x86/mm/mem_encrypt_amd.c
@@ -191,9 +191,6 @@ void __init sme_early_init(void)
/* Update the protection map with memory encryption mask */
for (i = 0; i < ARRAY_SIZE(protection_map); i++)
protection_map[i] = pgprot_encrypted(protection_map[i]);
-
- if (cc_platform_has(CC_ATTR_GUEST_MEM_ENCRYPT))
- swiotlb_force = SWIOTLB_FORCE;
}
void __init sev_setup_arch(void)
diff --git a/drivers/xen/swiotlb-xen.c b/drivers/xen/swiotlb-xen.c
index 485cd06ed39e7..c2da3eb4826e8 100644
--- a/drivers/xen/swiotlb-xen.c
+++ b/drivers/xen/swiotlb-xen.c
@@ -251,7 +251,7 @@ void __init xen_swiotlb_init_early(void)
panic("%s (rc:%d)", xen_swiotlb_error(XEN_SWIOTLB_EFIXUP), rc);
}
- if (swiotlb_init_with_tbl(start, nslabs, true))
+ if (swiotlb_init_with_tbl(start, nslabs, SWIOTLB_VERBOSE))
panic("Cannot allocate SWIOTLB buffer");
}
#endif /* CONFIG_X86 */
@@ -376,7 +376,7 @@ static dma_addr_t xen_swiotlb_map_page(struct device *dev, struct page *page,
/*
* Oh well, have to allocate and map a bounce buffer.
*/
- trace_swiotlb_bounced(dev, dev_addr, size, swiotlb_force);
+ trace_swiotlb_bounced(dev, dev_addr, size);
map = swiotlb_tbl_map_single(dev, phys, size, size, 0, dir, attrs);
if (map == (phys_addr_t)DMA_MAPPING_ERROR)
diff --git a/include/linux/swiotlb.h b/include/linux/swiotlb.h
index 1befd6b2ccf5e..dcecf953f7997 100644
--- a/include/linux/swiotlb.h
+++ b/include/linux/swiotlb.h
@@ -13,11 +13,8 @@ struct device;
struct page;
struct scatterlist;
-enum swiotlb_force {
- SWIOTLB_NORMAL, /* Default - depending on HW DMA mask etc. */
- SWIOTLB_FORCE, /* swiotlb=force */
- SWIOTLB_NO_FORCE, /* swiotlb=noforce */
-};
+#define SWIOTLB_VERBOSE (1 << 0) /* verbose initialization */
+#define SWIOTLB_FORCE (1 << 1) /* force bounce buffering */
/*
* Maximum allowable number of contiguous slabs to map,
@@ -36,8 +33,7 @@ enum swiotlb_force {
/* default to 64MB */
#define IO_TLB_DEFAULT_SIZE (64UL<<20)
-extern void swiotlb_init(int verbose);
-int swiotlb_init_with_tbl(char *tlb, unsigned long nslabs, int verbose);
+int swiotlb_init_with_tbl(char *tlb, unsigned long nslabs, unsigned int flags);
unsigned long swiotlb_size_or_default(void);
extern int swiotlb_late_init_with_tbl(char *tlb, unsigned long nslabs);
int swiotlb_init_late(size_t size, gfp_t gfp_mask);
@@ -126,13 +122,16 @@ static inline bool is_swiotlb_force_bounce(struct device *dev)
return mem && mem->force_bounce;
}
+void swiotlb_init(bool addressing_limited, unsigned int flags);
void __init swiotlb_exit(void);
unsigned int swiotlb_max_segment(void);
size_t swiotlb_max_mapping_size(struct device *dev);
bool is_swiotlb_active(struct device *dev);
void __init swiotlb_adjust_size(unsigned long size);
#else
-#define swiotlb_force SWIOTLB_NO_FORCE
+static inline void swiotlb_init(bool addressing_limited, unsigned int flags)
+{
+}
static inline bool is_swiotlb_buffer(struct device *dev, phys_addr_t paddr)
{
return false;
diff --git a/include/trace/events/swiotlb.h b/include/trace/events/swiotlb.h
index 705be43b71ab0..da05c9ebd224a 100644
--- a/include/trace/events/swiotlb.h
+++ b/include/trace/events/swiotlb.h
@@ -8,20 +8,15 @@
#include <linux/tracepoint.h>
TRACE_EVENT(swiotlb_bounced,
-
- TP_PROTO(struct device *dev,
- dma_addr_t dev_addr,
- size_t size,
- enum swiotlb_force swiotlb_force),
-
- TP_ARGS(dev, dev_addr, size, swiotlb_force),
+ TP_PROTO(struct device *dev, dma_addr_t dev_addr, size_t size),
+ TP_ARGS(dev, dev_addr, size),
TP_STRUCT__entry(
- __string( dev_name, dev_name(dev) )
- __field( u64, dma_mask )
- __field( dma_addr_t, dev_addr )
- __field( size_t, size )
- __field( enum swiotlb_force, swiotlb_force )
+ __string(dev_name, dev_name(dev))
+ __field(u64, dma_mask)
+ __field(dma_addr_t, dev_addr)
+ __field(size_t, size)
+ __field(bool, force)
),
TP_fast_assign(
@@ -29,19 +24,15 @@ TRACE_EVENT(swiotlb_bounced,
__entry->dma_mask = (dev->dma_mask ? *dev->dma_mask : 0);
__entry->dev_addr = dev_addr;
__entry->size = size;
- __entry->swiotlb_force = swiotlb_force;
+ __entry->force = is_swiotlb_force_bounce(dev);
),
- TP_printk("dev_name: %s dma_mask=%llx dev_addr=%llx "
- "size=%zu %s",
+ TP_printk("dev_name: %s dma_mask=%llx dev_addr=%llx size=%zu %s",
__get_str(dev_name),
__entry->dma_mask,
(unsigned long long)__entry->dev_addr,
__entry->size,
- __print_symbolic(__entry->swiotlb_force,
- { SWIOTLB_NORMAL, "NORMAL" },
- { SWIOTLB_FORCE, "FORCE" },
- { SWIOTLB_NO_FORCE, "NO_FORCE" }))
+ __entry->force ? "FORCE" : "NORMAL")
);
#endif /* _TRACE_SWIOTLB_H */
diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
index a653fcf1fe6c2..ad604e5a0983d 100644
--- a/kernel/dma/swiotlb.c
+++ b/kernel/dma/swiotlb.c
@@ -69,7 +69,8 @@
#define INVALID_PHYS_ADDR (~(phys_addr_t)0)
-enum swiotlb_force swiotlb_force;
+static bool swiotlb_force_bounce;
+static bool swiotlb_force_disable;
struct io_tlb_mem io_tlb_default_mem;
@@ -88,9 +89,9 @@ setup_io_tlb_npages(char *str)
if (*str == ',')
++str;
if (!strcmp(str, "force"))
- swiotlb_force = SWIOTLB_FORCE;
+ swiotlb_force_bounce = true;
else if (!strcmp(str, "noforce"))
- swiotlb_force = SWIOTLB_NO_FORCE;
+ swiotlb_force_disable = true;
return 0;
}
@@ -211,7 +212,7 @@ static void swiotlb_init_io_tlb_mem(struct io_tlb_mem *mem, phys_addr_t start,
mem->index = 0;
mem->late_alloc = late_alloc;
- if (swiotlb_force == SWIOTLB_FORCE)
+ if (swiotlb_force_bounce)
mem->force_bounce = true;
spin_lock_init(&mem->lock);
@@ -233,12 +234,13 @@ static void swiotlb_init_io_tlb_mem(struct io_tlb_mem *mem, phys_addr_t start,
return;
}
-int __init swiotlb_init_with_tbl(char *tlb, unsigned long nslabs, int verbose)
+int __init swiotlb_init_with_tbl(char *tlb, unsigned long nslabs,
+ unsigned int flags)
{
struct io_tlb_mem *mem = &io_tlb_default_mem;
size_t alloc_size;
- if (swiotlb_force == SWIOTLB_NO_FORCE)
+ if (swiotlb_force_disable)
return 0;
/* protect against double initialization */
@@ -252,8 +254,9 @@ int __init swiotlb_init_with_tbl(char *tlb, unsigned long nslabs, int verbose)
__func__, alloc_size, PAGE_SIZE);
swiotlb_init_io_tlb_mem(mem, __pa(tlb), nslabs, false);
+ mem->force_bounce = flags & SWIOTLB_FORCE;
- if (verbose)
+ if (flags & SWIOTLB_VERBOSE)
swiotlb_print_info();
return 0;
}
@@ -262,20 +265,21 @@ int __init swiotlb_init_with_tbl(char *tlb, unsigned long nslabs, int verbose)
* Statically reserve bounce buffer space and initialize bounce buffer data
* structures for the software IO TLB used to implement the DMA API.
*/
-void __init
-swiotlb_init(int verbose)
+void __init swiotlb_init(bool addressing_limit, unsigned int flags)
{
size_t bytes = PAGE_ALIGN(default_nslabs << IO_TLB_SHIFT);
void *tlb;
- if (swiotlb_force == SWIOTLB_NO_FORCE)
+ if (!addressing_limit && !swiotlb_force_bounce)
+ return;
+ if (swiotlb_force_disable)
return;
/* Get IO TLB memory from the low pages */
tlb = memblock_alloc_low(bytes, PAGE_SIZE);
if (!tlb)
goto fail;
- if (swiotlb_init_with_tbl(tlb, default_nslabs, verbose))
+ if (swiotlb_init_with_tbl(tlb, default_nslabs, flags))
goto fail_free_mem;
return;
@@ -298,7 +302,7 @@ int swiotlb_init_late(size_t size, gfp_t gfp_mask)
unsigned int order;
int rc = 0;
- if (swiotlb_force == SWIOTLB_NO_FORCE)
+ if (swiotlb_force_disable)
return 0;
/*
@@ -337,7 +341,7 @@ swiotlb_late_init_with_tbl(char *tlb, unsigned long nslabs)
struct io_tlb_mem *mem = &io_tlb_default_mem;
unsigned long bytes = nslabs << IO_TLB_SHIFT;
- if (swiotlb_force == SWIOTLB_NO_FORCE)
+ if (swiotlb_force_disable)
return 0;
/* protect against double initialization */
@@ -362,7 +366,7 @@ void __init swiotlb_exit(void)
unsigned long tbl_vaddr;
size_t tbl_size, slots_size;
- if (swiotlb_force == SWIOTLB_FORCE)
+ if (swiotlb_force_bounce)
return;
if (!mem->nslabs)
@@ -708,8 +712,7 @@ dma_addr_t swiotlb_map(struct device *dev, phys_addr_t paddr, size_t size,
phys_addr_t swiotlb_addr;
dma_addr_t dma_addr;
- trace_swiotlb_bounced(dev, phys_to_dma(dev, paddr), size,
- swiotlb_force);
+ trace_swiotlb_bounced(dev, phys_to_dma(dev, paddr), size);
swiotlb_addr = swiotlb_tbl_map_single(dev, paddr, size, size, 0, dir,
attrs);
--
2.30.2
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply related [flat|nested] 118+ messages in thread
* [PATCH 08/11] swiotlb: make the swiotlb_init interface more useful
@ 2022-02-27 14:30 ` Christoph Hellwig
0 siblings, 0 replies; 118+ messages in thread
From: Christoph Hellwig @ 2022-02-27 14:30 UTC (permalink / raw)
To: iommu
Cc: x86, Anshuman Khandual, Konrad Rzeszutek Wilk,
Stefano Stabellini, Boris Ostrovsky, Juergen Gross, Joerg Roedel,
David Woodhouse, Lu Baolu, Robin Murphy, linux-arm-kernel,
xen-devel, linux-ia64, linux-mips, linuxppc-dev, linux-riscv,
linux-s390, linux-hyperv, tboot-devel, linux-pci
Pass a bool to pass if swiotlb needs to be enabled based on the
addressing needs and replace the verbose argument with a set of
flags, including one to force enable bounce buffering.
Note that this patch removes the possibility to force xen-swiotlb
use using swiotlb=force on the command line on x86 (arm and arm64
never supported that), but this interface will be restored shortly.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
arch/arm/mm/init.c | 6 +----
arch/arm64/mm/init.c | 6 +----
arch/ia64/mm/init.c | 4 +--
arch/mips/cavium-octeon/dma-octeon.c | 2 +-
arch/mips/loongson64/dma.c | 2 +-
arch/mips/sibyte/common/dma.c | 2 +-
arch/powerpc/include/asm/swiotlb.h | 1 +
arch/powerpc/mm/mem.c | 3 ++-
arch/powerpc/platforms/pseries/setup.c | 3 ---
arch/riscv/mm/init.c | 8 +-----
arch/s390/mm/init.c | 3 +--
arch/x86/kernel/cpu/mshyperv.c | 8 ------
arch/x86/kernel/pci-dma.c | 15 ++++++-----
arch/x86/mm/mem_encrypt_amd.c | 3 ---
drivers/xen/swiotlb-xen.c | 4 +--
include/linux/swiotlb.h | 15 ++++++-----
include/trace/events/swiotlb.h | 29 ++++++++-------------
kernel/dma/swiotlb.c | 35 ++++++++++++++------------
18 files changed, 56 insertions(+), 93 deletions(-)
diff --git a/arch/arm/mm/init.c b/arch/arm/mm/init.c
index 6d0cb0f7bc54b..73f30d278b565 100644
--- a/arch/arm/mm/init.c
+++ b/arch/arm/mm/init.c
@@ -312,11 +312,7 @@ static void __init free_highpages(void)
void __init mem_init(void)
{
#ifdef CONFIG_ARM_LPAE
- if (swiotlb_force == SWIOTLB_FORCE ||
- max_pfn > arm_dma_pfn_limit)
- swiotlb_init(1);
- else
- swiotlb_force = SWIOTLB_NO_FORCE;
+ swiotlb_init(max_pfn > arm_dma_pfn_limit, SWIOTLB_VERBOSE);
#endif
set_max_mapnr(pfn_to_page(max_pfn) - mem_map);
diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c
index db63cc885771a..52102adda3d28 100644
--- a/arch/arm64/mm/init.c
+++ b/arch/arm64/mm/init.c
@@ -373,11 +373,7 @@ void __init bootmem_init(void)
*/
void __init mem_init(void)
{
- if (swiotlb_force == SWIOTLB_FORCE ||
- max_pfn > PFN_DOWN(arm64_dma_phys_limit))
- swiotlb_init(1);
- else if (!xen_swiotlb_detect())
- swiotlb_force = SWIOTLB_NO_FORCE;
+ swiotlb_init(max_pfn > PFN_DOWN(arm64_dma_phys_limit), SWIOTLB_VERBOSE);
/* this will put all unused low memory onto the freelists */
memblock_free_all();
diff --git a/arch/ia64/mm/init.c b/arch/ia64/mm/init.c
index 5d165607bf354..3c3e15b22608f 100644
--- a/arch/ia64/mm/init.c
+++ b/arch/ia64/mm/init.c
@@ -437,9 +437,7 @@ mem_init (void)
if (iommu_detected)
break;
#endif
-#ifdef CONFIG_SWIOTLB
- swiotlb_init(1);
-#endif
+ swiotlb_init(true, SWIOTLB_VERBOSE);
} while (0);
#ifdef CONFIG_FLATMEM
diff --git a/arch/mips/cavium-octeon/dma-octeon.c b/arch/mips/cavium-octeon/dma-octeon.c
index fb7547e217263..9fbba6a8fa4c5 100644
--- a/arch/mips/cavium-octeon/dma-octeon.c
+++ b/arch/mips/cavium-octeon/dma-octeon.c
@@ -235,5 +235,5 @@ void __init plat_swiotlb_setup(void)
#endif
swiotlb_adjust_size(swiotlbsize);
- swiotlb_init(1);
+ swiotlb_init(true, SWIOTLB_VERBOSE);
}
diff --git a/arch/mips/loongson64/dma.c b/arch/mips/loongson64/dma.c
index 364f2f27c8723..8220a1bc0db64 100644
--- a/arch/mips/loongson64/dma.c
+++ b/arch/mips/loongson64/dma.c
@@ -24,5 +24,5 @@ phys_addr_t dma_to_phys(struct device *dev, dma_addr_t daddr)
void __init plat_swiotlb_setup(void)
{
- swiotlb_init(1);
+ swiotlb_init(true, SWIOTLB_VERBOSE);
}
diff --git a/arch/mips/sibyte/common/dma.c b/arch/mips/sibyte/common/dma.c
index eb47a94f3583e..c5c2c782aff68 100644
--- a/arch/mips/sibyte/common/dma.c
+++ b/arch/mips/sibyte/common/dma.c
@@ -10,5 +10,5 @@
void __init plat_swiotlb_setup(void)
{
- swiotlb_init(1);
+ swiotlb_init(true, SWIOTLB_VERBOSE);
}
diff --git a/arch/powerpc/include/asm/swiotlb.h b/arch/powerpc/include/asm/swiotlb.h
index 3c1a1cd161286..4203b5e0a88ed 100644
--- a/arch/powerpc/include/asm/swiotlb.h
+++ b/arch/powerpc/include/asm/swiotlb.h
@@ -9,6 +9,7 @@
#include <linux/swiotlb.h>
extern unsigned int ppc_swiotlb_enable;
+extern unsigned int ppc_swiotlb_flags;
#ifdef CONFIG_SWIOTLB
void swiotlb_detect_4g(void);
diff --git a/arch/powerpc/mm/mem.c b/arch/powerpc/mm/mem.c
index 8e301cd8925b2..d99b8b5b40ca6 100644
--- a/arch/powerpc/mm/mem.c
+++ b/arch/powerpc/mm/mem.c
@@ -17,6 +17,7 @@
#include <linux/suspend.h>
#include <linux/dma-direct.h>
+#include <asm/swiotlb.h>
#include <asm/machdep.h>
#include <asm/rtas.h>
#include <asm/kasan.h>
@@ -251,7 +252,7 @@ void __init mem_init(void)
if (is_secure_guest())
svm_swiotlb_init();
else
- swiotlb_init(0);
+ swiotlb_init(ppc_swiotlb_enable, ppc_swiotlb_flags);
#endif
high_memory = (void *) __va(max_low_pfn * PAGE_SIZE);
diff --git a/arch/powerpc/platforms/pseries/setup.c b/arch/powerpc/platforms/pseries/setup.c
index 83a04d967a59f..45d637ab58261 100644
--- a/arch/powerpc/platforms/pseries/setup.c
+++ b/arch/powerpc/platforms/pseries/setup.c
@@ -838,9 +838,6 @@ static void __init pSeries_setup_arch(void)
}
ppc_md.pcibios_root_bridge_prepare = pseries_root_bridge_prepare;
-
- if (swiotlb_force == SWIOTLB_FORCE)
- ppc_swiotlb_enable = 1;
}
static void pseries_panic(char *str)
diff --git a/arch/riscv/mm/init.c b/arch/riscv/mm/init.c
index c27294128e182..6cdbb62672fe5 100644
--- a/arch/riscv/mm/init.c
+++ b/arch/riscv/mm/init.c
@@ -118,13 +118,7 @@ void __init mem_init(void)
BUG_ON(!mem_map);
#endif /* CONFIG_FLATMEM */
-#ifdef CONFIG_SWIOTLB
- if (swiotlb_force == SWIOTLB_FORCE ||
- max_pfn > PFN_DOWN(dma32_phys_limit))
- swiotlb_init(1);
- else
- swiotlb_force = SWIOTLB_NO_FORCE;
-#endif
+ swiotlb_init(max_pfn > PFN_DOWN(dma32_phys_limit), SWIOTLB_VERBOSE);
high_memory = (void *)(__va(PFN_PHYS(max_low_pfn)));
memblock_free_all();
diff --git a/arch/s390/mm/init.c b/arch/s390/mm/init.c
index 86ffd0d51fd59..6fb6bf64326f9 100644
--- a/arch/s390/mm/init.c
+++ b/arch/s390/mm/init.c
@@ -185,8 +185,7 @@ static void pv_init(void)
return;
/* make sure bounce buffers are shared */
- swiotlb_force = SWIOTLB_FORCE;
- swiotlb_init(1);
+ swiotlb_init(true, SWIOTLB_FORCE | SWIOTLB_VERBOSE);
swiotlb_update_mem_attributes();
}
diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c
index 5a99f993e6392..568274917f1cd 100644
--- a/arch/x86/kernel/cpu/mshyperv.c
+++ b/arch/x86/kernel/cpu/mshyperv.c
@@ -336,14 +336,6 @@ static void __init ms_hyperv_init_platform(void)
swiotlb_unencrypted_base = ms_hyperv.shared_gpa_boundary;
#endif
}
-
-#ifdef CONFIG_SWIOTLB
- /*
- * Enable swiotlb force mode in Isolation VM to
- * use swiotlb bounce buffer for dma transaction.
- */
- swiotlb_force = SWIOTLB_FORCE;
-#endif
}
if (hv_max_functions_eax >= HYPERV_CPUID_NESTED_FEATURES) {
diff --git a/arch/x86/kernel/pci-dma.c b/arch/x86/kernel/pci-dma.c
index 17bcac942ac49..bb08184a50e3a 100644
--- a/arch/x86/kernel/pci-dma.c
+++ b/arch/x86/kernel/pci-dma.c
@@ -39,6 +39,7 @@ int iommu_detected __read_mostly = 0;
#ifdef CONFIG_SWIOTLB
bool x86_swiotlb_enable;
+static unsigned int x86_swiotlb_flags;
/*
* If 4GB or more detected (and iommu=off not set) or if SME is active
@@ -54,16 +55,16 @@ static void __init pci_swiotlb_detect_4gb(void)
* Set swiotlb to 1 so that bounce buffers are allocated and used for
* devices that can't support DMA to encrypted memory.
*/
- if (cc_platform_has(CC_ATTR_HOST_MEM_ENCRYPT))
- x86_swiotlb_enable = true;
-
- if (swiotlb_force == SWIOTLB_FORCE)
+ if (cc_platform_has(CC_ATTR_HOST_MEM_ENCRYPT)) {
x86_swiotlb_enable = true;
+ x86_swiotlb_flags |= SWIOTLB_FORCE;
+ }
}
#else
static inline void __init pci_swiotlb_detect_4gb(void)
{
}
+#define x86_swiotlb_flags 0
#endif /* CONFIG_SWIOTLB */
#ifdef CONFIG_SWIOTLB_XEN
@@ -71,8 +72,7 @@ static bool xen_swiotlb;
static void __init pci_xen_swiotlb_init(void)
{
- if (!xen_initial_domain() && !x86_swiotlb_enable &&
- swiotlb_force != SWIOTLB_FORCE)
+ if (!xen_initial_domain() && !x86_swiotlb_enable)
return;
x86_swiotlb_enable = false;
xen_swiotlb = true;
@@ -116,8 +116,7 @@ void __init pci_iommu_alloc(void)
gart_iommu_hole_init();
amd_iommu_detect();
detect_intel_iommu();
- if (x86_swiotlb_enable)
- swiotlb_init(0);
+ swiotlb_init(x86_swiotlb_enable, x86_swiotlb_flags);
}
/*
diff --git a/arch/x86/mm/mem_encrypt_amd.c b/arch/x86/mm/mem_encrypt_amd.c
index 2b2d018ea3450..a72942d569cf9 100644
--- a/arch/x86/mm/mem_encrypt_amd.c
+++ b/arch/x86/mm/mem_encrypt_amd.c
@@ -191,9 +191,6 @@ void __init sme_early_init(void)
/* Update the protection map with memory encryption mask */
for (i = 0; i < ARRAY_SIZE(protection_map); i++)
protection_map[i] = pgprot_encrypted(protection_map[i]);
-
- if (cc_platform_has(CC_ATTR_GUEST_MEM_ENCRYPT))
- swiotlb_force = SWIOTLB_FORCE;
}
void __init sev_setup_arch(void)
diff --git a/drivers/xen/swiotlb-xen.c b/drivers/xen/swiotlb-xen.c
index 485cd06ed39e7..c2da3eb4826e8 100644
--- a/drivers/xen/swiotlb-xen.c
+++ b/drivers/xen/swiotlb-xen.c
@@ -251,7 +251,7 @@ void __init xen_swiotlb_init_early(void)
panic("%s (rc:%d)", xen_swiotlb_error(XEN_SWIOTLB_EFIXUP), rc);
}
- if (swiotlb_init_with_tbl(start, nslabs, true))
+ if (swiotlb_init_with_tbl(start, nslabs, SWIOTLB_VERBOSE))
panic("Cannot allocate SWIOTLB buffer");
}
#endif /* CONFIG_X86 */
@@ -376,7 +376,7 @@ static dma_addr_t xen_swiotlb_map_page(struct device *dev, struct page *page,
/*
* Oh well, have to allocate and map a bounce buffer.
*/
- trace_swiotlb_bounced(dev, dev_addr, size, swiotlb_force);
+ trace_swiotlb_bounced(dev, dev_addr, size);
map = swiotlb_tbl_map_single(dev, phys, size, size, 0, dir, attrs);
if (map == (phys_addr_t)DMA_MAPPING_ERROR)
diff --git a/include/linux/swiotlb.h b/include/linux/swiotlb.h
index 1befd6b2ccf5e..dcecf953f7997 100644
--- a/include/linux/swiotlb.h
+++ b/include/linux/swiotlb.h
@@ -13,11 +13,8 @@ struct device;
struct page;
struct scatterlist;
-enum swiotlb_force {
- SWIOTLB_NORMAL, /* Default - depending on HW DMA mask etc. */
- SWIOTLB_FORCE, /* swiotlb=force */
- SWIOTLB_NO_FORCE, /* swiotlb=noforce */
-};
+#define SWIOTLB_VERBOSE (1 << 0) /* verbose initialization */
+#define SWIOTLB_FORCE (1 << 1) /* force bounce buffering */
/*
* Maximum allowable number of contiguous slabs to map,
@@ -36,8 +33,7 @@ enum swiotlb_force {
/* default to 64MB */
#define IO_TLB_DEFAULT_SIZE (64UL<<20)
-extern void swiotlb_init(int verbose);
-int swiotlb_init_with_tbl(char *tlb, unsigned long nslabs, int verbose);
+int swiotlb_init_with_tbl(char *tlb, unsigned long nslabs, unsigned int flags);
unsigned long swiotlb_size_or_default(void);
extern int swiotlb_late_init_with_tbl(char *tlb, unsigned long nslabs);
int swiotlb_init_late(size_t size, gfp_t gfp_mask);
@@ -126,13 +122,16 @@ static inline bool is_swiotlb_force_bounce(struct device *dev)
return mem && mem->force_bounce;
}
+void swiotlb_init(bool addressing_limited, unsigned int flags);
void __init swiotlb_exit(void);
unsigned int swiotlb_max_segment(void);
size_t swiotlb_max_mapping_size(struct device *dev);
bool is_swiotlb_active(struct device *dev);
void __init swiotlb_adjust_size(unsigned long size);
#else
-#define swiotlb_force SWIOTLB_NO_FORCE
+static inline void swiotlb_init(bool addressing_limited, unsigned int flags)
+{
+}
static inline bool is_swiotlb_buffer(struct device *dev, phys_addr_t paddr)
{
return false;
diff --git a/include/trace/events/swiotlb.h b/include/trace/events/swiotlb.h
index 705be43b71ab0..da05c9ebd224a 100644
--- a/include/trace/events/swiotlb.h
+++ b/include/trace/events/swiotlb.h
@@ -8,20 +8,15 @@
#include <linux/tracepoint.h>
TRACE_EVENT(swiotlb_bounced,
-
- TP_PROTO(struct device *dev,
- dma_addr_t dev_addr,
- size_t size,
- enum swiotlb_force swiotlb_force),
-
- TP_ARGS(dev, dev_addr, size, swiotlb_force),
+ TP_PROTO(struct device *dev, dma_addr_t dev_addr, size_t size),
+ TP_ARGS(dev, dev_addr, size),
TP_STRUCT__entry(
- __string( dev_name, dev_name(dev) )
- __field( u64, dma_mask )
- __field( dma_addr_t, dev_addr )
- __field( size_t, size )
- __field( enum swiotlb_force, swiotlb_force )
+ __string(dev_name, dev_name(dev))
+ __field(u64, dma_mask)
+ __field(dma_addr_t, dev_addr)
+ __field(size_t, size)
+ __field(bool, force)
),
TP_fast_assign(
@@ -29,19 +24,15 @@ TRACE_EVENT(swiotlb_bounced,
__entry->dma_mask = (dev->dma_mask ? *dev->dma_mask : 0);
__entry->dev_addr = dev_addr;
__entry->size = size;
- __entry->swiotlb_force = swiotlb_force;
+ __entry->force = is_swiotlb_force_bounce(dev);
),
- TP_printk("dev_name: %s dma_mask=%llx dev_addr=%llx "
- "size=%zu %s",
+ TP_printk("dev_name: %s dma_mask=%llx dev_addr=%llx size=%zu %s",
__get_str(dev_name),
__entry->dma_mask,
(unsigned long long)__entry->dev_addr,
__entry->size,
- __print_symbolic(__entry->swiotlb_force,
- { SWIOTLB_NORMAL, "NORMAL" },
- { SWIOTLB_FORCE, "FORCE" },
- { SWIOTLB_NO_FORCE, "NO_FORCE" }))
+ __entry->force ? "FORCE" : "NORMAL")
);
#endif /* _TRACE_SWIOTLB_H */
diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
index a653fcf1fe6c2..ad604e5a0983d 100644
--- a/kernel/dma/swiotlb.c
+++ b/kernel/dma/swiotlb.c
@@ -69,7 +69,8 @@
#define INVALID_PHYS_ADDR (~(phys_addr_t)0)
-enum swiotlb_force swiotlb_force;
+static bool swiotlb_force_bounce;
+static bool swiotlb_force_disable;
struct io_tlb_mem io_tlb_default_mem;
@@ -88,9 +89,9 @@ setup_io_tlb_npages(char *str)
if (*str == ',')
++str;
if (!strcmp(str, "force"))
- swiotlb_force = SWIOTLB_FORCE;
+ swiotlb_force_bounce = true;
else if (!strcmp(str, "noforce"))
- swiotlb_force = SWIOTLB_NO_FORCE;
+ swiotlb_force_disable = true;
return 0;
}
@@ -211,7 +212,7 @@ static void swiotlb_init_io_tlb_mem(struct io_tlb_mem *mem, phys_addr_t start,
mem->index = 0;
mem->late_alloc = late_alloc;
- if (swiotlb_force == SWIOTLB_FORCE)
+ if (swiotlb_force_bounce)
mem->force_bounce = true;
spin_lock_init(&mem->lock);
@@ -233,12 +234,13 @@ static void swiotlb_init_io_tlb_mem(struct io_tlb_mem *mem, phys_addr_t start,
return;
}
-int __init swiotlb_init_with_tbl(char *tlb, unsigned long nslabs, int verbose)
+int __init swiotlb_init_with_tbl(char *tlb, unsigned long nslabs,
+ unsigned int flags)
{
struct io_tlb_mem *mem = &io_tlb_default_mem;
size_t alloc_size;
- if (swiotlb_force == SWIOTLB_NO_FORCE)
+ if (swiotlb_force_disable)
return 0;
/* protect against double initialization */
@@ -252,8 +254,9 @@ int __init swiotlb_init_with_tbl(char *tlb, unsigned long nslabs, int verbose)
__func__, alloc_size, PAGE_SIZE);
swiotlb_init_io_tlb_mem(mem, __pa(tlb), nslabs, false);
+ mem->force_bounce = flags & SWIOTLB_FORCE;
- if (verbose)
+ if (flags & SWIOTLB_VERBOSE)
swiotlb_print_info();
return 0;
}
@@ -262,20 +265,21 @@ int __init swiotlb_init_with_tbl(char *tlb, unsigned long nslabs, int verbose)
* Statically reserve bounce buffer space and initialize bounce buffer data
* structures for the software IO TLB used to implement the DMA API.
*/
-void __init
-swiotlb_init(int verbose)
+void __init swiotlb_init(bool addressing_limit, unsigned int flags)
{
size_t bytes = PAGE_ALIGN(default_nslabs << IO_TLB_SHIFT);
void *tlb;
- if (swiotlb_force == SWIOTLB_NO_FORCE)
+ if (!addressing_limit && !swiotlb_force_bounce)
+ return;
+ if (swiotlb_force_disable)
return;
/* Get IO TLB memory from the low pages */
tlb = memblock_alloc_low(bytes, PAGE_SIZE);
if (!tlb)
goto fail;
- if (swiotlb_init_with_tbl(tlb, default_nslabs, verbose))
+ if (swiotlb_init_with_tbl(tlb, default_nslabs, flags))
goto fail_free_mem;
return;
@@ -298,7 +302,7 @@ int swiotlb_init_late(size_t size, gfp_t gfp_mask)
unsigned int order;
int rc = 0;
- if (swiotlb_force == SWIOTLB_NO_FORCE)
+ if (swiotlb_force_disable)
return 0;
/*
@@ -337,7 +341,7 @@ swiotlb_late_init_with_tbl(char *tlb, unsigned long nslabs)
struct io_tlb_mem *mem = &io_tlb_default_mem;
unsigned long bytes = nslabs << IO_TLB_SHIFT;
- if (swiotlb_force == SWIOTLB_NO_FORCE)
+ if (swiotlb_force_disable)
return 0;
/* protect against double initialization */
@@ -362,7 +366,7 @@ void __init swiotlb_exit(void)
unsigned long tbl_vaddr;
size_t tbl_size, slots_size;
- if (swiotlb_force == SWIOTLB_FORCE)
+ if (swiotlb_force_bounce)
return;
if (!mem->nslabs)
@@ -708,8 +712,7 @@ dma_addr_t swiotlb_map(struct device *dev, phys_addr_t paddr, size_t size,
phys_addr_t swiotlb_addr;
dma_addr_t dma_addr;
- trace_swiotlb_bounced(dev, phys_to_dma(dev, paddr), size,
- swiotlb_force);
+ trace_swiotlb_bounced(dev, phys_to_dma(dev, paddr), size);
swiotlb_addr = swiotlb_tbl_map_single(dev, paddr, size, size, 0, dir,
attrs);
--
2.30.2
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply related [flat|nested] 118+ messages in thread
* [PATCH 08/11] swiotlb: make the swiotlb_init interface more useful
@ 2022-02-27 14:30 ` Christoph Hellwig
0 siblings, 0 replies; 118+ messages in thread
From: Christoph Hellwig @ 2022-02-27 14:30 UTC (permalink / raw)
To: iommu
Cc: x86, Anshuman Khandual, Konrad Rzeszutek Wilk,
Stefano Stabellini, Boris Ostrovsky, Juergen Gross, Joerg Roedel,
David Woodhouse, Lu Baolu, Robin Murphy, linux-arm-kernel,
xen-devel, linux-ia64, linux-mips, linuxppc-dev, linux-riscv,
linux-s390, linux-hyperv, tboot-devel, linux-pci
Pass a bool to pass if swiotlb needs to be enabled based on the
addressing needs and replace the verbose argument with a set of
flags, including one to force enable bounce buffering.
Note that this patch removes the possibility to force xen-swiotlb
use using swiotlb=force on the command line on x86 (arm and arm64
never supported that), but this interface will be restored shortly.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
arch/arm/mm/init.c | 6 +----
arch/arm64/mm/init.c | 6 +----
arch/ia64/mm/init.c | 4 +--
arch/mips/cavium-octeon/dma-octeon.c | 2 +-
arch/mips/loongson64/dma.c | 2 +-
arch/mips/sibyte/common/dma.c | 2 +-
arch/powerpc/include/asm/swiotlb.h | 1 +
arch/powerpc/mm/mem.c | 3 ++-
arch/powerpc/platforms/pseries/setup.c | 3 ---
arch/riscv/mm/init.c | 8 +-----
arch/s390/mm/init.c | 3 +--
arch/x86/kernel/cpu/mshyperv.c | 8 ------
arch/x86/kernel/pci-dma.c | 15 ++++++-----
arch/x86/mm/mem_encrypt_amd.c | 3 ---
drivers/xen/swiotlb-xen.c | 4 +--
include/linux/swiotlb.h | 15 ++++++-----
include/trace/events/swiotlb.h | 29 ++++++++-------------
kernel/dma/swiotlb.c | 35 ++++++++++++++------------
18 files changed, 56 insertions(+), 93 deletions(-)
diff --git a/arch/arm/mm/init.c b/arch/arm/mm/init.c
index 6d0cb0f7bc54b..73f30d278b565 100644
--- a/arch/arm/mm/init.c
+++ b/arch/arm/mm/init.c
@@ -312,11 +312,7 @@ static void __init free_highpages(void)
void __init mem_init(void)
{
#ifdef CONFIG_ARM_LPAE
- if (swiotlb_force == SWIOTLB_FORCE ||
- max_pfn > arm_dma_pfn_limit)
- swiotlb_init(1);
- else
- swiotlb_force = SWIOTLB_NO_FORCE;
+ swiotlb_init(max_pfn > arm_dma_pfn_limit, SWIOTLB_VERBOSE);
#endif
set_max_mapnr(pfn_to_page(max_pfn) - mem_map);
diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c
index db63cc885771a..52102adda3d28 100644
--- a/arch/arm64/mm/init.c
+++ b/arch/arm64/mm/init.c
@@ -373,11 +373,7 @@ void __init bootmem_init(void)
*/
void __init mem_init(void)
{
- if (swiotlb_force == SWIOTLB_FORCE ||
- max_pfn > PFN_DOWN(arm64_dma_phys_limit))
- swiotlb_init(1);
- else if (!xen_swiotlb_detect())
- swiotlb_force = SWIOTLB_NO_FORCE;
+ swiotlb_init(max_pfn > PFN_DOWN(arm64_dma_phys_limit), SWIOTLB_VERBOSE);
/* this will put all unused low memory onto the freelists */
memblock_free_all();
diff --git a/arch/ia64/mm/init.c b/arch/ia64/mm/init.c
index 5d165607bf354..3c3e15b22608f 100644
--- a/arch/ia64/mm/init.c
+++ b/arch/ia64/mm/init.c
@@ -437,9 +437,7 @@ mem_init (void)
if (iommu_detected)
break;
#endif
-#ifdef CONFIG_SWIOTLB
- swiotlb_init(1);
-#endif
+ swiotlb_init(true, SWIOTLB_VERBOSE);
} while (0);
#ifdef CONFIG_FLATMEM
diff --git a/arch/mips/cavium-octeon/dma-octeon.c b/arch/mips/cavium-octeon/dma-octeon.c
index fb7547e217263..9fbba6a8fa4c5 100644
--- a/arch/mips/cavium-octeon/dma-octeon.c
+++ b/arch/mips/cavium-octeon/dma-octeon.c
@@ -235,5 +235,5 @@ void __init plat_swiotlb_setup(void)
#endif
swiotlb_adjust_size(swiotlbsize);
- swiotlb_init(1);
+ swiotlb_init(true, SWIOTLB_VERBOSE);
}
diff --git a/arch/mips/loongson64/dma.c b/arch/mips/loongson64/dma.c
index 364f2f27c8723..8220a1bc0db64 100644
--- a/arch/mips/loongson64/dma.c
+++ b/arch/mips/loongson64/dma.c
@@ -24,5 +24,5 @@ phys_addr_t dma_to_phys(struct device *dev, dma_addr_t daddr)
void __init plat_swiotlb_setup(void)
{
- swiotlb_init(1);
+ swiotlb_init(true, SWIOTLB_VERBOSE);
}
diff --git a/arch/mips/sibyte/common/dma.c b/arch/mips/sibyte/common/dma.c
index eb47a94f3583e..c5c2c782aff68 100644
--- a/arch/mips/sibyte/common/dma.c
+++ b/arch/mips/sibyte/common/dma.c
@@ -10,5 +10,5 @@
void __init plat_swiotlb_setup(void)
{
- swiotlb_init(1);
+ swiotlb_init(true, SWIOTLB_VERBOSE);
}
diff --git a/arch/powerpc/include/asm/swiotlb.h b/arch/powerpc/include/asm/swiotlb.h
index 3c1a1cd161286..4203b5e0a88ed 100644
--- a/arch/powerpc/include/asm/swiotlb.h
+++ b/arch/powerpc/include/asm/swiotlb.h
@@ -9,6 +9,7 @@
#include <linux/swiotlb.h>
extern unsigned int ppc_swiotlb_enable;
+extern unsigned int ppc_swiotlb_flags;
#ifdef CONFIG_SWIOTLB
void swiotlb_detect_4g(void);
diff --git a/arch/powerpc/mm/mem.c b/arch/powerpc/mm/mem.c
index 8e301cd8925b2..d99b8b5b40ca6 100644
--- a/arch/powerpc/mm/mem.c
+++ b/arch/powerpc/mm/mem.c
@@ -17,6 +17,7 @@
#include <linux/suspend.h>
#include <linux/dma-direct.h>
+#include <asm/swiotlb.h>
#include <asm/machdep.h>
#include <asm/rtas.h>
#include <asm/kasan.h>
@@ -251,7 +252,7 @@ void __init mem_init(void)
if (is_secure_guest())
svm_swiotlb_init();
else
- swiotlb_init(0);
+ swiotlb_init(ppc_swiotlb_enable, ppc_swiotlb_flags);
#endif
high_memory = (void *) __va(max_low_pfn * PAGE_SIZE);
diff --git a/arch/powerpc/platforms/pseries/setup.c b/arch/powerpc/platforms/pseries/setup.c
index 83a04d967a59f..45d637ab58261 100644
--- a/arch/powerpc/platforms/pseries/setup.c
+++ b/arch/powerpc/platforms/pseries/setup.c
@@ -838,9 +838,6 @@ static void __init pSeries_setup_arch(void)
}
ppc_md.pcibios_root_bridge_prepare = pseries_root_bridge_prepare;
-
- if (swiotlb_force == SWIOTLB_FORCE)
- ppc_swiotlb_enable = 1;
}
static void pseries_panic(char *str)
diff --git a/arch/riscv/mm/init.c b/arch/riscv/mm/init.c
index c27294128e182..6cdbb62672fe5 100644
--- a/arch/riscv/mm/init.c
+++ b/arch/riscv/mm/init.c
@@ -118,13 +118,7 @@ void __init mem_init(void)
BUG_ON(!mem_map);
#endif /* CONFIG_FLATMEM */
-#ifdef CONFIG_SWIOTLB
- if (swiotlb_force == SWIOTLB_FORCE ||
- max_pfn > PFN_DOWN(dma32_phys_limit))
- swiotlb_init(1);
- else
- swiotlb_force = SWIOTLB_NO_FORCE;
-#endif
+ swiotlb_init(max_pfn > PFN_DOWN(dma32_phys_limit), SWIOTLB_VERBOSE);
high_memory = (void *)(__va(PFN_PHYS(max_low_pfn)));
memblock_free_all();
diff --git a/arch/s390/mm/init.c b/arch/s390/mm/init.c
index 86ffd0d51fd59..6fb6bf64326f9 100644
--- a/arch/s390/mm/init.c
+++ b/arch/s390/mm/init.c
@@ -185,8 +185,7 @@ static void pv_init(void)
return;
/* make sure bounce buffers are shared */
- swiotlb_force = SWIOTLB_FORCE;
- swiotlb_init(1);
+ swiotlb_init(true, SWIOTLB_FORCE | SWIOTLB_VERBOSE);
swiotlb_update_mem_attributes();
}
diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c
index 5a99f993e6392..568274917f1cd 100644
--- a/arch/x86/kernel/cpu/mshyperv.c
+++ b/arch/x86/kernel/cpu/mshyperv.c
@@ -336,14 +336,6 @@ static void __init ms_hyperv_init_platform(void)
swiotlb_unencrypted_base = ms_hyperv.shared_gpa_boundary;
#endif
}
-
-#ifdef CONFIG_SWIOTLB
- /*
- * Enable swiotlb force mode in Isolation VM to
- * use swiotlb bounce buffer for dma transaction.
- */
- swiotlb_force = SWIOTLB_FORCE;
-#endif
}
if (hv_max_functions_eax >= HYPERV_CPUID_NESTED_FEATURES) {
diff --git a/arch/x86/kernel/pci-dma.c b/arch/x86/kernel/pci-dma.c
index 17bcac942ac49..bb08184a50e3a 100644
--- a/arch/x86/kernel/pci-dma.c
+++ b/arch/x86/kernel/pci-dma.c
@@ -39,6 +39,7 @@ int iommu_detected __read_mostly = 0;
#ifdef CONFIG_SWIOTLB
bool x86_swiotlb_enable;
+static unsigned int x86_swiotlb_flags;
/*
* If 4GB or more detected (and iommu=off not set) or if SME is active
@@ -54,16 +55,16 @@ static void __init pci_swiotlb_detect_4gb(void)
* Set swiotlb to 1 so that bounce buffers are allocated and used for
* devices that can't support DMA to encrypted memory.
*/
- if (cc_platform_has(CC_ATTR_HOST_MEM_ENCRYPT))
- x86_swiotlb_enable = true;
-
- if (swiotlb_force == SWIOTLB_FORCE)
+ if (cc_platform_has(CC_ATTR_HOST_MEM_ENCRYPT)) {
x86_swiotlb_enable = true;
+ x86_swiotlb_flags |= SWIOTLB_FORCE;
+ }
}
#else
static inline void __init pci_swiotlb_detect_4gb(void)
{
}
+#define x86_swiotlb_flags 0
#endif /* CONFIG_SWIOTLB */
#ifdef CONFIG_SWIOTLB_XEN
@@ -71,8 +72,7 @@ static bool xen_swiotlb;
static void __init pci_xen_swiotlb_init(void)
{
- if (!xen_initial_domain() && !x86_swiotlb_enable &&
- swiotlb_force != SWIOTLB_FORCE)
+ if (!xen_initial_domain() && !x86_swiotlb_enable)
return;
x86_swiotlb_enable = false;
xen_swiotlb = true;
@@ -116,8 +116,7 @@ void __init pci_iommu_alloc(void)
gart_iommu_hole_init();
amd_iommu_detect();
detect_intel_iommu();
- if (x86_swiotlb_enable)
- swiotlb_init(0);
+ swiotlb_init(x86_swiotlb_enable, x86_swiotlb_flags);
}
/*
diff --git a/arch/x86/mm/mem_encrypt_amd.c b/arch/x86/mm/mem_encrypt_amd.c
index 2b2d018ea3450..a72942d569cf9 100644
--- a/arch/x86/mm/mem_encrypt_amd.c
+++ b/arch/x86/mm/mem_encrypt_amd.c
@@ -191,9 +191,6 @@ void __init sme_early_init(void)
/* Update the protection map with memory encryption mask */
for (i = 0; i < ARRAY_SIZE(protection_map); i++)
protection_map[i] = pgprot_encrypted(protection_map[i]);
-
- if (cc_platform_has(CC_ATTR_GUEST_MEM_ENCRYPT))
- swiotlb_force = SWIOTLB_FORCE;
}
void __init sev_setup_arch(void)
diff --git a/drivers/xen/swiotlb-xen.c b/drivers/xen/swiotlb-xen.c
index 485cd06ed39e7..c2da3eb4826e8 100644
--- a/drivers/xen/swiotlb-xen.c
+++ b/drivers/xen/swiotlb-xen.c
@@ -251,7 +251,7 @@ void __init xen_swiotlb_init_early(void)
panic("%s (rc:%d)", xen_swiotlb_error(XEN_SWIOTLB_EFIXUP), rc);
}
- if (swiotlb_init_with_tbl(start, nslabs, true))
+ if (swiotlb_init_with_tbl(start, nslabs, SWIOTLB_VERBOSE))
panic("Cannot allocate SWIOTLB buffer");
}
#endif /* CONFIG_X86 */
@@ -376,7 +376,7 @@ static dma_addr_t xen_swiotlb_map_page(struct device *dev, struct page *page,
/*
* Oh well, have to allocate and map a bounce buffer.
*/
- trace_swiotlb_bounced(dev, dev_addr, size, swiotlb_force);
+ trace_swiotlb_bounced(dev, dev_addr, size);
map = swiotlb_tbl_map_single(dev, phys, size, size, 0, dir, attrs);
if (map == (phys_addr_t)DMA_MAPPING_ERROR)
diff --git a/include/linux/swiotlb.h b/include/linux/swiotlb.h
index 1befd6b2ccf5e..dcecf953f7997 100644
--- a/include/linux/swiotlb.h
+++ b/include/linux/swiotlb.h
@@ -13,11 +13,8 @@ struct device;
struct page;
struct scatterlist;
-enum swiotlb_force {
- SWIOTLB_NORMAL, /* Default - depending on HW DMA mask etc. */
- SWIOTLB_FORCE, /* swiotlb=force */
- SWIOTLB_NO_FORCE, /* swiotlb=noforce */
-};
+#define SWIOTLB_VERBOSE (1 << 0) /* verbose initialization */
+#define SWIOTLB_FORCE (1 << 1) /* force bounce buffering */
/*
* Maximum allowable number of contiguous slabs to map,
@@ -36,8 +33,7 @@ enum swiotlb_force {
/* default to 64MB */
#define IO_TLB_DEFAULT_SIZE (64UL<<20)
-extern void swiotlb_init(int verbose);
-int swiotlb_init_with_tbl(char *tlb, unsigned long nslabs, int verbose);
+int swiotlb_init_with_tbl(char *tlb, unsigned long nslabs, unsigned int flags);
unsigned long swiotlb_size_or_default(void);
extern int swiotlb_late_init_with_tbl(char *tlb, unsigned long nslabs);
int swiotlb_init_late(size_t size, gfp_t gfp_mask);
@@ -126,13 +122,16 @@ static inline bool is_swiotlb_force_bounce(struct device *dev)
return mem && mem->force_bounce;
}
+void swiotlb_init(bool addressing_limited, unsigned int flags);
void __init swiotlb_exit(void);
unsigned int swiotlb_max_segment(void);
size_t swiotlb_max_mapping_size(struct device *dev);
bool is_swiotlb_active(struct device *dev);
void __init swiotlb_adjust_size(unsigned long size);
#else
-#define swiotlb_force SWIOTLB_NO_FORCE
+static inline void swiotlb_init(bool addressing_limited, unsigned int flags)
+{
+}
static inline bool is_swiotlb_buffer(struct device *dev, phys_addr_t paddr)
{
return false;
diff --git a/include/trace/events/swiotlb.h b/include/trace/events/swiotlb.h
index 705be43b71ab0..da05c9ebd224a 100644
--- a/include/trace/events/swiotlb.h
+++ b/include/trace/events/swiotlb.h
@@ -8,20 +8,15 @@
#include <linux/tracepoint.h>
TRACE_EVENT(swiotlb_bounced,
-
- TP_PROTO(struct device *dev,
- dma_addr_t dev_addr,
- size_t size,
- enum swiotlb_force swiotlb_force),
-
- TP_ARGS(dev, dev_addr, size, swiotlb_force),
+ TP_PROTO(struct device *dev, dma_addr_t dev_addr, size_t size),
+ TP_ARGS(dev, dev_addr, size),
TP_STRUCT__entry(
- __string( dev_name, dev_name(dev) )
- __field( u64, dma_mask )
- __field( dma_addr_t, dev_addr )
- __field( size_t, size )
- __field( enum swiotlb_force, swiotlb_force )
+ __string(dev_name, dev_name(dev))
+ __field(u64, dma_mask)
+ __field(dma_addr_t, dev_addr)
+ __field(size_t, size)
+ __field(bool, force)
),
TP_fast_assign(
@@ -29,19 +24,15 @@ TRACE_EVENT(swiotlb_bounced,
__entry->dma_mask = (dev->dma_mask ? *dev->dma_mask : 0);
__entry->dev_addr = dev_addr;
__entry->size = size;
- __entry->swiotlb_force = swiotlb_force;
+ __entry->force = is_swiotlb_force_bounce(dev);
),
- TP_printk("dev_name: %s dma_mask=%llx dev_addr=%llx "
- "size=%zu %s",
+ TP_printk("dev_name: %s dma_mask=%llx dev_addr=%llx size=%zu %s",
__get_str(dev_name),
__entry->dma_mask,
(unsigned long long)__entry->dev_addr,
__entry->size,
- __print_symbolic(__entry->swiotlb_force,
- { SWIOTLB_NORMAL, "NORMAL" },
- { SWIOTLB_FORCE, "FORCE" },
- { SWIOTLB_NO_FORCE, "NO_FORCE" }))
+ __entry->force ? "FORCE" : "NORMAL")
);
#endif /* _TRACE_SWIOTLB_H */
diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
index a653fcf1fe6c2..ad604e5a0983d 100644
--- a/kernel/dma/swiotlb.c
+++ b/kernel/dma/swiotlb.c
@@ -69,7 +69,8 @@
#define INVALID_PHYS_ADDR (~(phys_addr_t)0)
-enum swiotlb_force swiotlb_force;
+static bool swiotlb_force_bounce;
+static bool swiotlb_force_disable;
struct io_tlb_mem io_tlb_default_mem;
@@ -88,9 +89,9 @@ setup_io_tlb_npages(char *str)
if (*str == ',')
++str;
if (!strcmp(str, "force"))
- swiotlb_force = SWIOTLB_FORCE;
+ swiotlb_force_bounce = true;
else if (!strcmp(str, "noforce"))
- swiotlb_force = SWIOTLB_NO_FORCE;
+ swiotlb_force_disable = true;
return 0;
}
@@ -211,7 +212,7 @@ static void swiotlb_init_io_tlb_mem(struct io_tlb_mem *mem, phys_addr_t start,
mem->index = 0;
mem->late_alloc = late_alloc;
- if (swiotlb_force == SWIOTLB_FORCE)
+ if (swiotlb_force_bounce)
mem->force_bounce = true;
spin_lock_init(&mem->lock);
@@ -233,12 +234,13 @@ static void swiotlb_init_io_tlb_mem(struct io_tlb_mem *mem, phys_addr_t start,
return;
}
-int __init swiotlb_init_with_tbl(char *tlb, unsigned long nslabs, int verbose)
+int __init swiotlb_init_with_tbl(char *tlb, unsigned long nslabs,
+ unsigned int flags)
{
struct io_tlb_mem *mem = &io_tlb_default_mem;
size_t alloc_size;
- if (swiotlb_force == SWIOTLB_NO_FORCE)
+ if (swiotlb_force_disable)
return 0;
/* protect against double initialization */
@@ -252,8 +254,9 @@ int __init swiotlb_init_with_tbl(char *tlb, unsigned long nslabs, int verbose)
__func__, alloc_size, PAGE_SIZE);
swiotlb_init_io_tlb_mem(mem, __pa(tlb), nslabs, false);
+ mem->force_bounce = flags & SWIOTLB_FORCE;
- if (verbose)
+ if (flags & SWIOTLB_VERBOSE)
swiotlb_print_info();
return 0;
}
@@ -262,20 +265,21 @@ int __init swiotlb_init_with_tbl(char *tlb, unsigned long nslabs, int verbose)
* Statically reserve bounce buffer space and initialize bounce buffer data
* structures for the software IO TLB used to implement the DMA API.
*/
-void __init
-swiotlb_init(int verbose)
+void __init swiotlb_init(bool addressing_limit, unsigned int flags)
{
size_t bytes = PAGE_ALIGN(default_nslabs << IO_TLB_SHIFT);
void *tlb;
- if (swiotlb_force == SWIOTLB_NO_FORCE)
+ if (!addressing_limit && !swiotlb_force_bounce)
+ return;
+ if (swiotlb_force_disable)
return;
/* Get IO TLB memory from the low pages */
tlb = memblock_alloc_low(bytes, PAGE_SIZE);
if (!tlb)
goto fail;
- if (swiotlb_init_with_tbl(tlb, default_nslabs, verbose))
+ if (swiotlb_init_with_tbl(tlb, default_nslabs, flags))
goto fail_free_mem;
return;
@@ -298,7 +302,7 @@ int swiotlb_init_late(size_t size, gfp_t gfp_mask)
unsigned int order;
int rc = 0;
- if (swiotlb_force == SWIOTLB_NO_FORCE)
+ if (swiotlb_force_disable)
return 0;
/*
@@ -337,7 +341,7 @@ swiotlb_late_init_with_tbl(char *tlb, unsigned long nslabs)
struct io_tlb_mem *mem = &io_tlb_default_mem;
unsigned long bytes = nslabs << IO_TLB_SHIFT;
- if (swiotlb_force == SWIOTLB_NO_FORCE)
+ if (swiotlb_force_disable)
return 0;
/* protect against double initialization */
@@ -362,7 +366,7 @@ void __init swiotlb_exit(void)
unsigned long tbl_vaddr;
size_t tbl_size, slots_size;
- if (swiotlb_force == SWIOTLB_FORCE)
+ if (swiotlb_force_bounce)
return;
if (!mem->nslabs)
@@ -708,8 +712,7 @@ dma_addr_t swiotlb_map(struct device *dev, phys_addr_t paddr, size_t size,
phys_addr_t swiotlb_addr;
dma_addr_t dma_addr;
- trace_swiotlb_bounced(dev, phys_to_dma(dev, paddr), size,
- swiotlb_force);
+ trace_swiotlb_bounced(dev, phys_to_dma(dev, paddr), size);
swiotlb_addr = swiotlb_tbl_map_single(dev, paddr, size, size, 0, dir,
attrs);
--
2.30.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 118+ messages in thread
* [PATCH 08/11] swiotlb: make the swiotlb_init interface more useful
@ 2022-02-27 14:30 ` Christoph Hellwig
0 siblings, 0 replies; 118+ messages in thread
From: Christoph Hellwig @ 2022-02-27 14:30 UTC (permalink / raw)
To: iommu
Cc: Juergen Gross, linux-s390, linux-hyperv, Stefano Stabellini,
linux-ia64, Anshuman Khandual, Konrad Rzeszutek Wilk,
Robin Murphy, Joerg Roedel, x86, linux-mips, linuxppc-dev,
tboot-devel, linux-pci, xen-devel, Boris Ostrovsky,
David Woodhouse, linux-riscv, linux-arm-kernel, Lu Baolu
Pass a bool to pass if swiotlb needs to be enabled based on the
addressing needs and replace the verbose argument with a set of
flags, including one to force enable bounce buffering.
Note that this patch removes the possibility to force xen-swiotlb
use using swiotlb=force on the command line on x86 (arm and arm64
never supported that), but this interface will be restored shortly.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
arch/arm/mm/init.c | 6 +----
arch/arm64/mm/init.c | 6 +----
arch/ia64/mm/init.c | 4 +--
arch/mips/cavium-octeon/dma-octeon.c | 2 +-
arch/mips/loongson64/dma.c | 2 +-
arch/mips/sibyte/common/dma.c | 2 +-
arch/powerpc/include/asm/swiotlb.h | 1 +
arch/powerpc/mm/mem.c | 3 ++-
arch/powerpc/platforms/pseries/setup.c | 3 ---
arch/riscv/mm/init.c | 8 +-----
arch/s390/mm/init.c | 3 +--
arch/x86/kernel/cpu/mshyperv.c | 8 ------
arch/x86/kernel/pci-dma.c | 15 ++++++-----
arch/x86/mm/mem_encrypt_amd.c | 3 ---
drivers/xen/swiotlb-xen.c | 4 +--
include/linux/swiotlb.h | 15 ++++++-----
include/trace/events/swiotlb.h | 29 ++++++++-------------
kernel/dma/swiotlb.c | 35 ++++++++++++++------------
18 files changed, 56 insertions(+), 93 deletions(-)
diff --git a/arch/arm/mm/init.c b/arch/arm/mm/init.c
index 6d0cb0f7bc54b..73f30d278b565 100644
--- a/arch/arm/mm/init.c
+++ b/arch/arm/mm/init.c
@@ -312,11 +312,7 @@ static void __init free_highpages(void)
void __init mem_init(void)
{
#ifdef CONFIG_ARM_LPAE
- if (swiotlb_force == SWIOTLB_FORCE ||
- max_pfn > arm_dma_pfn_limit)
- swiotlb_init(1);
- else
- swiotlb_force = SWIOTLB_NO_FORCE;
+ swiotlb_init(max_pfn > arm_dma_pfn_limit, SWIOTLB_VERBOSE);
#endif
set_max_mapnr(pfn_to_page(max_pfn) - mem_map);
diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c
index db63cc885771a..52102adda3d28 100644
--- a/arch/arm64/mm/init.c
+++ b/arch/arm64/mm/init.c
@@ -373,11 +373,7 @@ void __init bootmem_init(void)
*/
void __init mem_init(void)
{
- if (swiotlb_force == SWIOTLB_FORCE ||
- max_pfn > PFN_DOWN(arm64_dma_phys_limit))
- swiotlb_init(1);
- else if (!xen_swiotlb_detect())
- swiotlb_force = SWIOTLB_NO_FORCE;
+ swiotlb_init(max_pfn > PFN_DOWN(arm64_dma_phys_limit), SWIOTLB_VERBOSE);
/* this will put all unused low memory onto the freelists */
memblock_free_all();
diff --git a/arch/ia64/mm/init.c b/arch/ia64/mm/init.c
index 5d165607bf354..3c3e15b22608f 100644
--- a/arch/ia64/mm/init.c
+++ b/arch/ia64/mm/init.c
@@ -437,9 +437,7 @@ mem_init (void)
if (iommu_detected)
break;
#endif
-#ifdef CONFIG_SWIOTLB
- swiotlb_init(1);
-#endif
+ swiotlb_init(true, SWIOTLB_VERBOSE);
} while (0);
#ifdef CONFIG_FLATMEM
diff --git a/arch/mips/cavium-octeon/dma-octeon.c b/arch/mips/cavium-octeon/dma-octeon.c
index fb7547e217263..9fbba6a8fa4c5 100644
--- a/arch/mips/cavium-octeon/dma-octeon.c
+++ b/arch/mips/cavium-octeon/dma-octeon.c
@@ -235,5 +235,5 @@ void __init plat_swiotlb_setup(void)
#endif
swiotlb_adjust_size(swiotlbsize);
- swiotlb_init(1);
+ swiotlb_init(true, SWIOTLB_VERBOSE);
}
diff --git a/arch/mips/loongson64/dma.c b/arch/mips/loongson64/dma.c
index 364f2f27c8723..8220a1bc0db64 100644
--- a/arch/mips/loongson64/dma.c
+++ b/arch/mips/loongson64/dma.c
@@ -24,5 +24,5 @@ phys_addr_t dma_to_phys(struct device *dev, dma_addr_t daddr)
void __init plat_swiotlb_setup(void)
{
- swiotlb_init(1);
+ swiotlb_init(true, SWIOTLB_VERBOSE);
}
diff --git a/arch/mips/sibyte/common/dma.c b/arch/mips/sibyte/common/dma.c
index eb47a94f3583e..c5c2c782aff68 100644
--- a/arch/mips/sibyte/common/dma.c
+++ b/arch/mips/sibyte/common/dma.c
@@ -10,5 +10,5 @@
void __init plat_swiotlb_setup(void)
{
- swiotlb_init(1);
+ swiotlb_init(true, SWIOTLB_VERBOSE);
}
diff --git a/arch/powerpc/include/asm/swiotlb.h b/arch/powerpc/include/asm/swiotlb.h
index 3c1a1cd161286..4203b5e0a88ed 100644
--- a/arch/powerpc/include/asm/swiotlb.h
+++ b/arch/powerpc/include/asm/swiotlb.h
@@ -9,6 +9,7 @@
#include <linux/swiotlb.h>
extern unsigned int ppc_swiotlb_enable;
+extern unsigned int ppc_swiotlb_flags;
#ifdef CONFIG_SWIOTLB
void swiotlb_detect_4g(void);
diff --git a/arch/powerpc/mm/mem.c b/arch/powerpc/mm/mem.c
index 8e301cd8925b2..d99b8b5b40ca6 100644
--- a/arch/powerpc/mm/mem.c
+++ b/arch/powerpc/mm/mem.c
@@ -17,6 +17,7 @@
#include <linux/suspend.h>
#include <linux/dma-direct.h>
+#include <asm/swiotlb.h>
#include <asm/machdep.h>
#include <asm/rtas.h>
#include <asm/kasan.h>
@@ -251,7 +252,7 @@ void __init mem_init(void)
if (is_secure_guest())
svm_swiotlb_init();
else
- swiotlb_init(0);
+ swiotlb_init(ppc_swiotlb_enable, ppc_swiotlb_flags);
#endif
high_memory = (void *) __va(max_low_pfn * PAGE_SIZE);
diff --git a/arch/powerpc/platforms/pseries/setup.c b/arch/powerpc/platforms/pseries/setup.c
index 83a04d967a59f..45d637ab58261 100644
--- a/arch/powerpc/platforms/pseries/setup.c
+++ b/arch/powerpc/platforms/pseries/setup.c
@@ -838,9 +838,6 @@ static void __init pSeries_setup_arch(void)
}
ppc_md.pcibios_root_bridge_prepare = pseries_root_bridge_prepare;
-
- if (swiotlb_force == SWIOTLB_FORCE)
- ppc_swiotlb_enable = 1;
}
static void pseries_panic(char *str)
diff --git a/arch/riscv/mm/init.c b/arch/riscv/mm/init.c
index c27294128e182..6cdbb62672fe5 100644
--- a/arch/riscv/mm/init.c
+++ b/arch/riscv/mm/init.c
@@ -118,13 +118,7 @@ void __init mem_init(void)
BUG_ON(!mem_map);
#endif /* CONFIG_FLATMEM */
-#ifdef CONFIG_SWIOTLB
- if (swiotlb_force == SWIOTLB_FORCE ||
- max_pfn > PFN_DOWN(dma32_phys_limit))
- swiotlb_init(1);
- else
- swiotlb_force = SWIOTLB_NO_FORCE;
-#endif
+ swiotlb_init(max_pfn > PFN_DOWN(dma32_phys_limit), SWIOTLB_VERBOSE);
high_memory = (void *)(__va(PFN_PHYS(max_low_pfn)));
memblock_free_all();
diff --git a/arch/s390/mm/init.c b/arch/s390/mm/init.c
index 86ffd0d51fd59..6fb6bf64326f9 100644
--- a/arch/s390/mm/init.c
+++ b/arch/s390/mm/init.c
@@ -185,8 +185,7 @@ static void pv_init(void)
return;
/* make sure bounce buffers are shared */
- swiotlb_force = SWIOTLB_FORCE;
- swiotlb_init(1);
+ swiotlb_init(true, SWIOTLB_FORCE | SWIOTLB_VERBOSE);
swiotlb_update_mem_attributes();
}
diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c
index 5a99f993e6392..568274917f1cd 100644
--- a/arch/x86/kernel/cpu/mshyperv.c
+++ b/arch/x86/kernel/cpu/mshyperv.c
@@ -336,14 +336,6 @@ static void __init ms_hyperv_init_platform(void)
swiotlb_unencrypted_base = ms_hyperv.shared_gpa_boundary;
#endif
}
-
-#ifdef CONFIG_SWIOTLB
- /*
- * Enable swiotlb force mode in Isolation VM to
- * use swiotlb bounce buffer for dma transaction.
- */
- swiotlb_force = SWIOTLB_FORCE;
-#endif
}
if (hv_max_functions_eax >= HYPERV_CPUID_NESTED_FEATURES) {
diff --git a/arch/x86/kernel/pci-dma.c b/arch/x86/kernel/pci-dma.c
index 17bcac942ac49..bb08184a50e3a 100644
--- a/arch/x86/kernel/pci-dma.c
+++ b/arch/x86/kernel/pci-dma.c
@@ -39,6 +39,7 @@ int iommu_detected __read_mostly = 0;
#ifdef CONFIG_SWIOTLB
bool x86_swiotlb_enable;
+static unsigned int x86_swiotlb_flags;
/*
* If 4GB or more detected (and iommu=off not set) or if SME is active
@@ -54,16 +55,16 @@ static void __init pci_swiotlb_detect_4gb(void)
* Set swiotlb to 1 so that bounce buffers are allocated and used for
* devices that can't support DMA to encrypted memory.
*/
- if (cc_platform_has(CC_ATTR_HOST_MEM_ENCRYPT))
- x86_swiotlb_enable = true;
-
- if (swiotlb_force == SWIOTLB_FORCE)
+ if (cc_platform_has(CC_ATTR_HOST_MEM_ENCRYPT)) {
x86_swiotlb_enable = true;
+ x86_swiotlb_flags |= SWIOTLB_FORCE;
+ }
}
#else
static inline void __init pci_swiotlb_detect_4gb(void)
{
}
+#define x86_swiotlb_flags 0
#endif /* CONFIG_SWIOTLB */
#ifdef CONFIG_SWIOTLB_XEN
@@ -71,8 +72,7 @@ static bool xen_swiotlb;
static void __init pci_xen_swiotlb_init(void)
{
- if (!xen_initial_domain() && !x86_swiotlb_enable &&
- swiotlb_force != SWIOTLB_FORCE)
+ if (!xen_initial_domain() && !x86_swiotlb_enable)
return;
x86_swiotlb_enable = false;
xen_swiotlb = true;
@@ -116,8 +116,7 @@ void __init pci_iommu_alloc(void)
gart_iommu_hole_init();
amd_iommu_detect();
detect_intel_iommu();
- if (x86_swiotlb_enable)
- swiotlb_init(0);
+ swiotlb_init(x86_swiotlb_enable, x86_swiotlb_flags);
}
/*
diff --git a/arch/x86/mm/mem_encrypt_amd.c b/arch/x86/mm/mem_encrypt_amd.c
index 2b2d018ea3450..a72942d569cf9 100644
--- a/arch/x86/mm/mem_encrypt_amd.c
+++ b/arch/x86/mm/mem_encrypt_amd.c
@@ -191,9 +191,6 @@ void __init sme_early_init(void)
/* Update the protection map with memory encryption mask */
for (i = 0; i < ARRAY_SIZE(protection_map); i++)
protection_map[i] = pgprot_encrypted(protection_map[i]);
-
- if (cc_platform_has(CC_ATTR_GUEST_MEM_ENCRYPT))
- swiotlb_force = SWIOTLB_FORCE;
}
void __init sev_setup_arch(void)
diff --git a/drivers/xen/swiotlb-xen.c b/drivers/xen/swiotlb-xen.c
index 485cd06ed39e7..c2da3eb4826e8 100644
--- a/drivers/xen/swiotlb-xen.c
+++ b/drivers/xen/swiotlb-xen.c
@@ -251,7 +251,7 @@ void __init xen_swiotlb_init_early(void)
panic("%s (rc:%d)", xen_swiotlb_error(XEN_SWIOTLB_EFIXUP), rc);
}
- if (swiotlb_init_with_tbl(start, nslabs, true))
+ if (swiotlb_init_with_tbl(start, nslabs, SWIOTLB_VERBOSE))
panic("Cannot allocate SWIOTLB buffer");
}
#endif /* CONFIG_X86 */
@@ -376,7 +376,7 @@ static dma_addr_t xen_swiotlb_map_page(struct device *dev, struct page *page,
/*
* Oh well, have to allocate and map a bounce buffer.
*/
- trace_swiotlb_bounced(dev, dev_addr, size, swiotlb_force);
+ trace_swiotlb_bounced(dev, dev_addr, size);
map = swiotlb_tbl_map_single(dev, phys, size, size, 0, dir, attrs);
if (map == (phys_addr_t)DMA_MAPPING_ERROR)
diff --git a/include/linux/swiotlb.h b/include/linux/swiotlb.h
index 1befd6b2ccf5e..dcecf953f7997 100644
--- a/include/linux/swiotlb.h
+++ b/include/linux/swiotlb.h
@@ -13,11 +13,8 @@ struct device;
struct page;
struct scatterlist;
-enum swiotlb_force {
- SWIOTLB_NORMAL, /* Default - depending on HW DMA mask etc. */
- SWIOTLB_FORCE, /* swiotlb=force */
- SWIOTLB_NO_FORCE, /* swiotlb=noforce */
-};
+#define SWIOTLB_VERBOSE (1 << 0) /* verbose initialization */
+#define SWIOTLB_FORCE (1 << 1) /* force bounce buffering */
/*
* Maximum allowable number of contiguous slabs to map,
@@ -36,8 +33,7 @@ enum swiotlb_force {
/* default to 64MB */
#define IO_TLB_DEFAULT_SIZE (64UL<<20)
-extern void swiotlb_init(int verbose);
-int swiotlb_init_with_tbl(char *tlb, unsigned long nslabs, int verbose);
+int swiotlb_init_with_tbl(char *tlb, unsigned long nslabs, unsigned int flags);
unsigned long swiotlb_size_or_default(void);
extern int swiotlb_late_init_with_tbl(char *tlb, unsigned long nslabs);
int swiotlb_init_late(size_t size, gfp_t gfp_mask);
@@ -126,13 +122,16 @@ static inline bool is_swiotlb_force_bounce(struct device *dev)
return mem && mem->force_bounce;
}
+void swiotlb_init(bool addressing_limited, unsigned int flags);
void __init swiotlb_exit(void);
unsigned int swiotlb_max_segment(void);
size_t swiotlb_max_mapping_size(struct device *dev);
bool is_swiotlb_active(struct device *dev);
void __init swiotlb_adjust_size(unsigned long size);
#else
-#define swiotlb_force SWIOTLB_NO_FORCE
+static inline void swiotlb_init(bool addressing_limited, unsigned int flags)
+{
+}
static inline bool is_swiotlb_buffer(struct device *dev, phys_addr_t paddr)
{
return false;
diff --git a/include/trace/events/swiotlb.h b/include/trace/events/swiotlb.h
index 705be43b71ab0..da05c9ebd224a 100644
--- a/include/trace/events/swiotlb.h
+++ b/include/trace/events/swiotlb.h
@@ -8,20 +8,15 @@
#include <linux/tracepoint.h>
TRACE_EVENT(swiotlb_bounced,
-
- TP_PROTO(struct device *dev,
- dma_addr_t dev_addr,
- size_t size,
- enum swiotlb_force swiotlb_force),
-
- TP_ARGS(dev, dev_addr, size, swiotlb_force),
+ TP_PROTO(struct device *dev, dma_addr_t dev_addr, size_t size),
+ TP_ARGS(dev, dev_addr, size),
TP_STRUCT__entry(
- __string( dev_name, dev_name(dev) )
- __field( u64, dma_mask )
- __field( dma_addr_t, dev_addr )
- __field( size_t, size )
- __field( enum swiotlb_force, swiotlb_force )
+ __string(dev_name, dev_name(dev))
+ __field(u64, dma_mask)
+ __field(dma_addr_t, dev_addr)
+ __field(size_t, size)
+ __field(bool, force)
),
TP_fast_assign(
@@ -29,19 +24,15 @@ TRACE_EVENT(swiotlb_bounced,
__entry->dma_mask = (dev->dma_mask ? *dev->dma_mask : 0);
__entry->dev_addr = dev_addr;
__entry->size = size;
- __entry->swiotlb_force = swiotlb_force;
+ __entry->force = is_swiotlb_force_bounce(dev);
),
- TP_printk("dev_name: %s dma_mask=%llx dev_addr=%llx "
- "size=%zu %s",
+ TP_printk("dev_name: %s dma_mask=%llx dev_addr=%llx size=%zu %s",
__get_str(dev_name),
__entry->dma_mask,
(unsigned long long)__entry->dev_addr,
__entry->size,
- __print_symbolic(__entry->swiotlb_force,
- { SWIOTLB_NORMAL, "NORMAL" },
- { SWIOTLB_FORCE, "FORCE" },
- { SWIOTLB_NO_FORCE, "NO_FORCE" }))
+ __entry->force ? "FORCE" : "NORMAL")
);
#endif /* _TRACE_SWIOTLB_H */
diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
index a653fcf1fe6c2..ad604e5a0983d 100644
--- a/kernel/dma/swiotlb.c
+++ b/kernel/dma/swiotlb.c
@@ -69,7 +69,8 @@
#define INVALID_PHYS_ADDR (~(phys_addr_t)0)
-enum swiotlb_force swiotlb_force;
+static bool swiotlb_force_bounce;
+static bool swiotlb_force_disable;
struct io_tlb_mem io_tlb_default_mem;
@@ -88,9 +89,9 @@ setup_io_tlb_npages(char *str)
if (*str == ',')
++str;
if (!strcmp(str, "force"))
- swiotlb_force = SWIOTLB_FORCE;
+ swiotlb_force_bounce = true;
else if (!strcmp(str, "noforce"))
- swiotlb_force = SWIOTLB_NO_FORCE;
+ swiotlb_force_disable = true;
return 0;
}
@@ -211,7 +212,7 @@ static void swiotlb_init_io_tlb_mem(struct io_tlb_mem *mem, phys_addr_t start,
mem->index = 0;
mem->late_alloc = late_alloc;
- if (swiotlb_force == SWIOTLB_FORCE)
+ if (swiotlb_force_bounce)
mem->force_bounce = true;
spin_lock_init(&mem->lock);
@@ -233,12 +234,13 @@ static void swiotlb_init_io_tlb_mem(struct io_tlb_mem *mem, phys_addr_t start,
return;
}
-int __init swiotlb_init_with_tbl(char *tlb, unsigned long nslabs, int verbose)
+int __init swiotlb_init_with_tbl(char *tlb, unsigned long nslabs,
+ unsigned int flags)
{
struct io_tlb_mem *mem = &io_tlb_default_mem;
size_t alloc_size;
- if (swiotlb_force == SWIOTLB_NO_FORCE)
+ if (swiotlb_force_disable)
return 0;
/* protect against double initialization */
@@ -252,8 +254,9 @@ int __init swiotlb_init_with_tbl(char *tlb, unsigned long nslabs, int verbose)
__func__, alloc_size, PAGE_SIZE);
swiotlb_init_io_tlb_mem(mem, __pa(tlb), nslabs, false);
+ mem->force_bounce = flags & SWIOTLB_FORCE;
- if (verbose)
+ if (flags & SWIOTLB_VERBOSE)
swiotlb_print_info();
return 0;
}
@@ -262,20 +265,21 @@ int __init swiotlb_init_with_tbl(char *tlb, unsigned long nslabs, int verbose)
* Statically reserve bounce buffer space and initialize bounce buffer data
* structures for the software IO TLB used to implement the DMA API.
*/
-void __init
-swiotlb_init(int verbose)
+void __init swiotlb_init(bool addressing_limit, unsigned int flags)
{
size_t bytes = PAGE_ALIGN(default_nslabs << IO_TLB_SHIFT);
void *tlb;
- if (swiotlb_force == SWIOTLB_NO_FORCE)
+ if (!addressing_limit && !swiotlb_force_bounce)
+ return;
+ if (swiotlb_force_disable)
return;
/* Get IO TLB memory from the low pages */
tlb = memblock_alloc_low(bytes, PAGE_SIZE);
if (!tlb)
goto fail;
- if (swiotlb_init_with_tbl(tlb, default_nslabs, verbose))
+ if (swiotlb_init_with_tbl(tlb, default_nslabs, flags))
goto fail_free_mem;
return;
@@ -298,7 +302,7 @@ int swiotlb_init_late(size_t size, gfp_t gfp_mask)
unsigned int order;
int rc = 0;
- if (swiotlb_force == SWIOTLB_NO_FORCE)
+ if (swiotlb_force_disable)
return 0;
/*
@@ -337,7 +341,7 @@ swiotlb_late_init_with_tbl(char *tlb, unsigned long nslabs)
struct io_tlb_mem *mem = &io_tlb_default_mem;
unsigned long bytes = nslabs << IO_TLB_SHIFT;
- if (swiotlb_force == SWIOTLB_NO_FORCE)
+ if (swiotlb_force_disable)
return 0;
/* protect against double initialization */
@@ -362,7 +366,7 @@ void __init swiotlb_exit(void)
unsigned long tbl_vaddr;
size_t tbl_size, slots_size;
- if (swiotlb_force == SWIOTLB_FORCE)
+ if (swiotlb_force_bounce)
return;
if (!mem->nslabs)
@@ -708,8 +712,7 @@ dma_addr_t swiotlb_map(struct device *dev, phys_addr_t paddr, size_t size,
phys_addr_t swiotlb_addr;
dma_addr_t dma_addr;
- trace_swiotlb_bounced(dev, phys_to_dma(dev, paddr), size,
- swiotlb_force);
+ trace_swiotlb_bounced(dev, phys_to_dma(dev, paddr), size);
swiotlb_addr = swiotlb_tbl_map_single(dev, paddr, size, size, 0, dir,
attrs);
--
2.30.2
^ permalink raw reply related [flat|nested] 118+ messages in thread
* [PATCH 08/11] swiotlb: make the swiotlb_init interface more useful
@ 2022-02-27 14:30 ` Christoph Hellwig
0 siblings, 0 replies; 118+ messages in thread
From: Christoph Hellwig @ 2022-02-27 14:30 UTC (permalink / raw)
To: iommu
Cc: x86, Anshuman Khandual, Konrad Rzeszutek Wilk,
Stefano Stabellini, Boris Ostrovsky, Juergen Gross, Joerg Roedel,
David Woodhouse, Lu Baolu, Robin Murphy, linux-arm-kernel,
xen-devel, linux-ia64, linux-mips, linuxppc-dev, linux-riscv,
linux-s390, linux-hyperv, tboot-devel, linux-pci
Pass a bool to pass if swiotlb needs to be enabled based on the
addressing needs and replace the verbose argument with a set of
flags, including one to force enable bounce buffering.
Note that this patch removes the possibility to force xen-swiotlb
use using swiotlb=force on the command line on x86 (arm and arm64
never supported that), but this interface will be restored shortly.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
arch/arm/mm/init.c | 6 +----
arch/arm64/mm/init.c | 6 +----
arch/ia64/mm/init.c | 4 +--
arch/mips/cavium-octeon/dma-octeon.c | 2 +-
arch/mips/loongson64/dma.c | 2 +-
arch/mips/sibyte/common/dma.c | 2 +-
arch/powerpc/include/asm/swiotlb.h | 1 +
arch/powerpc/mm/mem.c | 3 ++-
arch/powerpc/platforms/pseries/setup.c | 3 ---
arch/riscv/mm/init.c | 8 +-----
arch/s390/mm/init.c | 3 +--
arch/x86/kernel/cpu/mshyperv.c | 8 ------
arch/x86/kernel/pci-dma.c | 15 ++++++-----
arch/x86/mm/mem_encrypt_amd.c | 3 ---
drivers/xen/swiotlb-xen.c | 4 +--
include/linux/swiotlb.h | 15 ++++++-----
include/trace/events/swiotlb.h | 29 ++++++++-------------
kernel/dma/swiotlb.c | 35 ++++++++++++++------------
18 files changed, 56 insertions(+), 93 deletions(-)
diff --git a/arch/arm/mm/init.c b/arch/arm/mm/init.c
index 6d0cb0f7bc54b..73f30d278b565 100644
--- a/arch/arm/mm/init.c
+++ b/arch/arm/mm/init.c
@@ -312,11 +312,7 @@ static void __init free_highpages(void)
void __init mem_init(void)
{
#ifdef CONFIG_ARM_LPAE
- if (swiotlb_force = SWIOTLB_FORCE ||
- max_pfn > arm_dma_pfn_limit)
- swiotlb_init(1);
- else
- swiotlb_force = SWIOTLB_NO_FORCE;
+ swiotlb_init(max_pfn > arm_dma_pfn_limit, SWIOTLB_VERBOSE);
#endif
set_max_mapnr(pfn_to_page(max_pfn) - mem_map);
diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c
index db63cc885771a..52102adda3d28 100644
--- a/arch/arm64/mm/init.c
+++ b/arch/arm64/mm/init.c
@@ -373,11 +373,7 @@ void __init bootmem_init(void)
*/
void __init mem_init(void)
{
- if (swiotlb_force = SWIOTLB_FORCE ||
- max_pfn > PFN_DOWN(arm64_dma_phys_limit))
- swiotlb_init(1);
- else if (!xen_swiotlb_detect())
- swiotlb_force = SWIOTLB_NO_FORCE;
+ swiotlb_init(max_pfn > PFN_DOWN(arm64_dma_phys_limit), SWIOTLB_VERBOSE);
/* this will put all unused low memory onto the freelists */
memblock_free_all();
diff --git a/arch/ia64/mm/init.c b/arch/ia64/mm/init.c
index 5d165607bf354..3c3e15b22608f 100644
--- a/arch/ia64/mm/init.c
+++ b/arch/ia64/mm/init.c
@@ -437,9 +437,7 @@ mem_init (void)
if (iommu_detected)
break;
#endif
-#ifdef CONFIG_SWIOTLB
- swiotlb_init(1);
-#endif
+ swiotlb_init(true, SWIOTLB_VERBOSE);
} while (0);
#ifdef CONFIG_FLATMEM
diff --git a/arch/mips/cavium-octeon/dma-octeon.c b/arch/mips/cavium-octeon/dma-octeon.c
index fb7547e217263..9fbba6a8fa4c5 100644
--- a/arch/mips/cavium-octeon/dma-octeon.c
+++ b/arch/mips/cavium-octeon/dma-octeon.c
@@ -235,5 +235,5 @@ void __init plat_swiotlb_setup(void)
#endif
swiotlb_adjust_size(swiotlbsize);
- swiotlb_init(1);
+ swiotlb_init(true, SWIOTLB_VERBOSE);
}
diff --git a/arch/mips/loongson64/dma.c b/arch/mips/loongson64/dma.c
index 364f2f27c8723..8220a1bc0db64 100644
--- a/arch/mips/loongson64/dma.c
+++ b/arch/mips/loongson64/dma.c
@@ -24,5 +24,5 @@ phys_addr_t dma_to_phys(struct device *dev, dma_addr_t daddr)
void __init plat_swiotlb_setup(void)
{
- swiotlb_init(1);
+ swiotlb_init(true, SWIOTLB_VERBOSE);
}
diff --git a/arch/mips/sibyte/common/dma.c b/arch/mips/sibyte/common/dma.c
index eb47a94f3583e..c5c2c782aff68 100644
--- a/arch/mips/sibyte/common/dma.c
+++ b/arch/mips/sibyte/common/dma.c
@@ -10,5 +10,5 @@
void __init plat_swiotlb_setup(void)
{
- swiotlb_init(1);
+ swiotlb_init(true, SWIOTLB_VERBOSE);
}
diff --git a/arch/powerpc/include/asm/swiotlb.h b/arch/powerpc/include/asm/swiotlb.h
index 3c1a1cd161286..4203b5e0a88ed 100644
--- a/arch/powerpc/include/asm/swiotlb.h
+++ b/arch/powerpc/include/asm/swiotlb.h
@@ -9,6 +9,7 @@
#include <linux/swiotlb.h>
extern unsigned int ppc_swiotlb_enable;
+extern unsigned int ppc_swiotlb_flags;
#ifdef CONFIG_SWIOTLB
void swiotlb_detect_4g(void);
diff --git a/arch/powerpc/mm/mem.c b/arch/powerpc/mm/mem.c
index 8e301cd8925b2..d99b8b5b40ca6 100644
--- a/arch/powerpc/mm/mem.c
+++ b/arch/powerpc/mm/mem.c
@@ -17,6 +17,7 @@
#include <linux/suspend.h>
#include <linux/dma-direct.h>
+#include <asm/swiotlb.h>
#include <asm/machdep.h>
#include <asm/rtas.h>
#include <asm/kasan.h>
@@ -251,7 +252,7 @@ void __init mem_init(void)
if (is_secure_guest())
svm_swiotlb_init();
else
- swiotlb_init(0);
+ swiotlb_init(ppc_swiotlb_enable, ppc_swiotlb_flags);
#endif
high_memory = (void *) __va(max_low_pfn * PAGE_SIZE);
diff --git a/arch/powerpc/platforms/pseries/setup.c b/arch/powerpc/platforms/pseries/setup.c
index 83a04d967a59f..45d637ab58261 100644
--- a/arch/powerpc/platforms/pseries/setup.c
+++ b/arch/powerpc/platforms/pseries/setup.c
@@ -838,9 +838,6 @@ static void __init pSeries_setup_arch(void)
}
ppc_md.pcibios_root_bridge_prepare = pseries_root_bridge_prepare;
-
- if (swiotlb_force = SWIOTLB_FORCE)
- ppc_swiotlb_enable = 1;
}
static void pseries_panic(char *str)
diff --git a/arch/riscv/mm/init.c b/arch/riscv/mm/init.c
index c27294128e182..6cdbb62672fe5 100644
--- a/arch/riscv/mm/init.c
+++ b/arch/riscv/mm/init.c
@@ -118,13 +118,7 @@ void __init mem_init(void)
BUG_ON(!mem_map);
#endif /* CONFIG_FLATMEM */
-#ifdef CONFIG_SWIOTLB
- if (swiotlb_force = SWIOTLB_FORCE ||
- max_pfn > PFN_DOWN(dma32_phys_limit))
- swiotlb_init(1);
- else
- swiotlb_force = SWIOTLB_NO_FORCE;
-#endif
+ swiotlb_init(max_pfn > PFN_DOWN(dma32_phys_limit), SWIOTLB_VERBOSE);
high_memory = (void *)(__va(PFN_PHYS(max_low_pfn)));
memblock_free_all();
diff --git a/arch/s390/mm/init.c b/arch/s390/mm/init.c
index 86ffd0d51fd59..6fb6bf64326f9 100644
--- a/arch/s390/mm/init.c
+++ b/arch/s390/mm/init.c
@@ -185,8 +185,7 @@ static void pv_init(void)
return;
/* make sure bounce buffers are shared */
- swiotlb_force = SWIOTLB_FORCE;
- swiotlb_init(1);
+ swiotlb_init(true, SWIOTLB_FORCE | SWIOTLB_VERBOSE);
swiotlb_update_mem_attributes();
}
diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c
index 5a99f993e6392..568274917f1cd 100644
--- a/arch/x86/kernel/cpu/mshyperv.c
+++ b/arch/x86/kernel/cpu/mshyperv.c
@@ -336,14 +336,6 @@ static void __init ms_hyperv_init_platform(void)
swiotlb_unencrypted_base = ms_hyperv.shared_gpa_boundary;
#endif
}
-
-#ifdef CONFIG_SWIOTLB
- /*
- * Enable swiotlb force mode in Isolation VM to
- * use swiotlb bounce buffer for dma transaction.
- */
- swiotlb_force = SWIOTLB_FORCE;
-#endif
}
if (hv_max_functions_eax >= HYPERV_CPUID_NESTED_FEATURES) {
diff --git a/arch/x86/kernel/pci-dma.c b/arch/x86/kernel/pci-dma.c
index 17bcac942ac49..bb08184a50e3a 100644
--- a/arch/x86/kernel/pci-dma.c
+++ b/arch/x86/kernel/pci-dma.c
@@ -39,6 +39,7 @@ int iommu_detected __read_mostly = 0;
#ifdef CONFIG_SWIOTLB
bool x86_swiotlb_enable;
+static unsigned int x86_swiotlb_flags;
/*
* If 4GB or more detected (and iommu=off not set) or if SME is active
@@ -54,16 +55,16 @@ static void __init pci_swiotlb_detect_4gb(void)
* Set swiotlb to 1 so that bounce buffers are allocated and used for
* devices that can't support DMA to encrypted memory.
*/
- if (cc_platform_has(CC_ATTR_HOST_MEM_ENCRYPT))
- x86_swiotlb_enable = true;
-
- if (swiotlb_force = SWIOTLB_FORCE)
+ if (cc_platform_has(CC_ATTR_HOST_MEM_ENCRYPT)) {
x86_swiotlb_enable = true;
+ x86_swiotlb_flags |= SWIOTLB_FORCE;
+ }
}
#else
static inline void __init pci_swiotlb_detect_4gb(void)
{
}
+#define x86_swiotlb_flags 0
#endif /* CONFIG_SWIOTLB */
#ifdef CONFIG_SWIOTLB_XEN
@@ -71,8 +72,7 @@ static bool xen_swiotlb;
static void __init pci_xen_swiotlb_init(void)
{
- if (!xen_initial_domain() && !x86_swiotlb_enable &&
- swiotlb_force != SWIOTLB_FORCE)
+ if (!xen_initial_domain() && !x86_swiotlb_enable)
return;
x86_swiotlb_enable = false;
xen_swiotlb = true;
@@ -116,8 +116,7 @@ void __init pci_iommu_alloc(void)
gart_iommu_hole_init();
amd_iommu_detect();
detect_intel_iommu();
- if (x86_swiotlb_enable)
- swiotlb_init(0);
+ swiotlb_init(x86_swiotlb_enable, x86_swiotlb_flags);
}
/*
diff --git a/arch/x86/mm/mem_encrypt_amd.c b/arch/x86/mm/mem_encrypt_amd.c
index 2b2d018ea3450..a72942d569cf9 100644
--- a/arch/x86/mm/mem_encrypt_amd.c
+++ b/arch/x86/mm/mem_encrypt_amd.c
@@ -191,9 +191,6 @@ void __init sme_early_init(void)
/* Update the protection map with memory encryption mask */
for (i = 0; i < ARRAY_SIZE(protection_map); i++)
protection_map[i] = pgprot_encrypted(protection_map[i]);
-
- if (cc_platform_has(CC_ATTR_GUEST_MEM_ENCRYPT))
- swiotlb_force = SWIOTLB_FORCE;
}
void __init sev_setup_arch(void)
diff --git a/drivers/xen/swiotlb-xen.c b/drivers/xen/swiotlb-xen.c
index 485cd06ed39e7..c2da3eb4826e8 100644
--- a/drivers/xen/swiotlb-xen.c
+++ b/drivers/xen/swiotlb-xen.c
@@ -251,7 +251,7 @@ void __init xen_swiotlb_init_early(void)
panic("%s (rc:%d)", xen_swiotlb_error(XEN_SWIOTLB_EFIXUP), rc);
}
- if (swiotlb_init_with_tbl(start, nslabs, true))
+ if (swiotlb_init_with_tbl(start, nslabs, SWIOTLB_VERBOSE))
panic("Cannot allocate SWIOTLB buffer");
}
#endif /* CONFIG_X86 */
@@ -376,7 +376,7 @@ static dma_addr_t xen_swiotlb_map_page(struct device *dev, struct page *page,
/*
* Oh well, have to allocate and map a bounce buffer.
*/
- trace_swiotlb_bounced(dev, dev_addr, size, swiotlb_force);
+ trace_swiotlb_bounced(dev, dev_addr, size);
map = swiotlb_tbl_map_single(dev, phys, size, size, 0, dir, attrs);
if (map = (phys_addr_t)DMA_MAPPING_ERROR)
diff --git a/include/linux/swiotlb.h b/include/linux/swiotlb.h
index 1befd6b2ccf5e..dcecf953f7997 100644
--- a/include/linux/swiotlb.h
+++ b/include/linux/swiotlb.h
@@ -13,11 +13,8 @@ struct device;
struct page;
struct scatterlist;
-enum swiotlb_force {
- SWIOTLB_NORMAL, /* Default - depending on HW DMA mask etc. */
- SWIOTLB_FORCE, /* swiotlb=force */
- SWIOTLB_NO_FORCE, /* swiotlb=noforce */
-};
+#define SWIOTLB_VERBOSE (1 << 0) /* verbose initialization */
+#define SWIOTLB_FORCE (1 << 1) /* force bounce buffering */
/*
* Maximum allowable number of contiguous slabs to map,
@@ -36,8 +33,7 @@ enum swiotlb_force {
/* default to 64MB */
#define IO_TLB_DEFAULT_SIZE (64UL<<20)
-extern void swiotlb_init(int verbose);
-int swiotlb_init_with_tbl(char *tlb, unsigned long nslabs, int verbose);
+int swiotlb_init_with_tbl(char *tlb, unsigned long nslabs, unsigned int flags);
unsigned long swiotlb_size_or_default(void);
extern int swiotlb_late_init_with_tbl(char *tlb, unsigned long nslabs);
int swiotlb_init_late(size_t size, gfp_t gfp_mask);
@@ -126,13 +122,16 @@ static inline bool is_swiotlb_force_bounce(struct device *dev)
return mem && mem->force_bounce;
}
+void swiotlb_init(bool addressing_limited, unsigned int flags);
void __init swiotlb_exit(void);
unsigned int swiotlb_max_segment(void);
size_t swiotlb_max_mapping_size(struct device *dev);
bool is_swiotlb_active(struct device *dev);
void __init swiotlb_adjust_size(unsigned long size);
#else
-#define swiotlb_force SWIOTLB_NO_FORCE
+static inline void swiotlb_init(bool addressing_limited, unsigned int flags)
+{
+}
static inline bool is_swiotlb_buffer(struct device *dev, phys_addr_t paddr)
{
return false;
diff --git a/include/trace/events/swiotlb.h b/include/trace/events/swiotlb.h
index 705be43b71ab0..da05c9ebd224a 100644
--- a/include/trace/events/swiotlb.h
+++ b/include/trace/events/swiotlb.h
@@ -8,20 +8,15 @@
#include <linux/tracepoint.h>
TRACE_EVENT(swiotlb_bounced,
-
- TP_PROTO(struct device *dev,
- dma_addr_t dev_addr,
- size_t size,
- enum swiotlb_force swiotlb_force),
-
- TP_ARGS(dev, dev_addr, size, swiotlb_force),
+ TP_PROTO(struct device *dev, dma_addr_t dev_addr, size_t size),
+ TP_ARGS(dev, dev_addr, size),
TP_STRUCT__entry(
- __string( dev_name, dev_name(dev) )
- __field( u64, dma_mask )
- __field( dma_addr_t, dev_addr )
- __field( size_t, size )
- __field( enum swiotlb_force, swiotlb_force )
+ __string(dev_name, dev_name(dev))
+ __field(u64, dma_mask)
+ __field(dma_addr_t, dev_addr)
+ __field(size_t, size)
+ __field(bool, force)
),
TP_fast_assign(
@@ -29,19 +24,15 @@ TRACE_EVENT(swiotlb_bounced,
__entry->dma_mask = (dev->dma_mask ? *dev->dma_mask : 0);
__entry->dev_addr = dev_addr;
__entry->size = size;
- __entry->swiotlb_force = swiotlb_force;
+ __entry->force = is_swiotlb_force_bounce(dev);
),
- TP_printk("dev_name: %s dma_mask=%llx dev_addr=%llx "
- "size=%zu %s",
+ TP_printk("dev_name: %s dma_mask=%llx dev_addr=%llx size=%zu %s",
__get_str(dev_name),
__entry->dma_mask,
(unsigned long long)__entry->dev_addr,
__entry->size,
- __print_symbolic(__entry->swiotlb_force,
- { SWIOTLB_NORMAL, "NORMAL" },
- { SWIOTLB_FORCE, "FORCE" },
- { SWIOTLB_NO_FORCE, "NO_FORCE" }))
+ __entry->force ? "FORCE" : "NORMAL")
);
#endif /* _TRACE_SWIOTLB_H */
diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
index a653fcf1fe6c2..ad604e5a0983d 100644
--- a/kernel/dma/swiotlb.c
+++ b/kernel/dma/swiotlb.c
@@ -69,7 +69,8 @@
#define INVALID_PHYS_ADDR (~(phys_addr_t)0)
-enum swiotlb_force swiotlb_force;
+static bool swiotlb_force_bounce;
+static bool swiotlb_force_disable;
struct io_tlb_mem io_tlb_default_mem;
@@ -88,9 +89,9 @@ setup_io_tlb_npages(char *str)
if (*str = ',')
++str;
if (!strcmp(str, "force"))
- swiotlb_force = SWIOTLB_FORCE;
+ swiotlb_force_bounce = true;
else if (!strcmp(str, "noforce"))
- swiotlb_force = SWIOTLB_NO_FORCE;
+ swiotlb_force_disable = true;
return 0;
}
@@ -211,7 +212,7 @@ static void swiotlb_init_io_tlb_mem(struct io_tlb_mem *mem, phys_addr_t start,
mem->index = 0;
mem->late_alloc = late_alloc;
- if (swiotlb_force = SWIOTLB_FORCE)
+ if (swiotlb_force_bounce)
mem->force_bounce = true;
spin_lock_init(&mem->lock);
@@ -233,12 +234,13 @@ static void swiotlb_init_io_tlb_mem(struct io_tlb_mem *mem, phys_addr_t start,
return;
}
-int __init swiotlb_init_with_tbl(char *tlb, unsigned long nslabs, int verbose)
+int __init swiotlb_init_with_tbl(char *tlb, unsigned long nslabs,
+ unsigned int flags)
{
struct io_tlb_mem *mem = &io_tlb_default_mem;
size_t alloc_size;
- if (swiotlb_force = SWIOTLB_NO_FORCE)
+ if (swiotlb_force_disable)
return 0;
/* protect against double initialization */
@@ -252,8 +254,9 @@ int __init swiotlb_init_with_tbl(char *tlb, unsigned long nslabs, int verbose)
__func__, alloc_size, PAGE_SIZE);
swiotlb_init_io_tlb_mem(mem, __pa(tlb), nslabs, false);
+ mem->force_bounce = flags & SWIOTLB_FORCE;
- if (verbose)
+ if (flags & SWIOTLB_VERBOSE)
swiotlb_print_info();
return 0;
}
@@ -262,20 +265,21 @@ int __init swiotlb_init_with_tbl(char *tlb, unsigned long nslabs, int verbose)
* Statically reserve bounce buffer space and initialize bounce buffer data
* structures for the software IO TLB used to implement the DMA API.
*/
-void __init
-swiotlb_init(int verbose)
+void __init swiotlb_init(bool addressing_limit, unsigned int flags)
{
size_t bytes = PAGE_ALIGN(default_nslabs << IO_TLB_SHIFT);
void *tlb;
- if (swiotlb_force = SWIOTLB_NO_FORCE)
+ if (!addressing_limit && !swiotlb_force_bounce)
+ return;
+ if (swiotlb_force_disable)
return;
/* Get IO TLB memory from the low pages */
tlb = memblock_alloc_low(bytes, PAGE_SIZE);
if (!tlb)
goto fail;
- if (swiotlb_init_with_tbl(tlb, default_nslabs, verbose))
+ if (swiotlb_init_with_tbl(tlb, default_nslabs, flags))
goto fail_free_mem;
return;
@@ -298,7 +302,7 @@ int swiotlb_init_late(size_t size, gfp_t gfp_mask)
unsigned int order;
int rc = 0;
- if (swiotlb_force = SWIOTLB_NO_FORCE)
+ if (swiotlb_force_disable)
return 0;
/*
@@ -337,7 +341,7 @@ swiotlb_late_init_with_tbl(char *tlb, unsigned long nslabs)
struct io_tlb_mem *mem = &io_tlb_default_mem;
unsigned long bytes = nslabs << IO_TLB_SHIFT;
- if (swiotlb_force = SWIOTLB_NO_FORCE)
+ if (swiotlb_force_disable)
return 0;
/* protect against double initialization */
@@ -362,7 +366,7 @@ void __init swiotlb_exit(void)
unsigned long tbl_vaddr;
size_t tbl_size, slots_size;
- if (swiotlb_force = SWIOTLB_FORCE)
+ if (swiotlb_force_bounce)
return;
if (!mem->nslabs)
@@ -708,8 +712,7 @@ dma_addr_t swiotlb_map(struct device *dev, phys_addr_t paddr, size_t size,
phys_addr_t swiotlb_addr;
dma_addr_t dma_addr;
- trace_swiotlb_bounced(dev, phys_to_dma(dev, paddr), size,
- swiotlb_force);
+ trace_swiotlb_bounced(dev, phys_to_dma(dev, paddr), size);
swiotlb_addr = swiotlb_tbl_map_single(dev, paddr, size, size, 0, dir,
attrs);
--
2.30.2
^ permalink raw reply related [flat|nested] 118+ messages in thread
* [PATCH 09/11] swiotlb: add a SWIOTLB_ANY flag to lift the low memory restriction
2022-02-27 14:30 ` Christoph Hellwig
` (3 preceding siblings ...)
(?)
@ 2022-02-27 14:30 ` Christoph Hellwig
-1 siblings, 0 replies; 118+ messages in thread
From: Christoph Hellwig @ 2022-02-27 14:30 UTC (permalink / raw)
To: iommu
Cc: x86, Anshuman Khandual, Konrad Rzeszutek Wilk,
Stefano Stabellini, Boris Ostrovsky, Juergen Gross, Joerg Roedel,
David Woodhouse, Lu Baolu, Robin Murphy, linux-arm-kernel,
xen-devel, linux-ia64, linux-mips, linuxppc-dev, linux-riscv,
linux-s390, linux-hyperv, tboot-devel, linux-pci
Power SVM wants to allocate a swiotlb buffer that is not restricted to
low memory for the trusted hypervisor scheme. Consolidate the support
for this into the swiotlb_init interface by adding a new flag.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
arch/powerpc/include/asm/svm.h | 4 ----
arch/powerpc/mm/mem.c | 5 +----
arch/powerpc/platforms/pseries/svm.c | 26 +-------------------------
include/linux/swiotlb.h | 1 +
kernel/dma/swiotlb.c | 9 +++++++--
5 files changed, 10 insertions(+), 35 deletions(-)
diff --git a/arch/powerpc/include/asm/svm.h b/arch/powerpc/include/asm/svm.h
index 7546402d796af..85580b30aba48 100644
--- a/arch/powerpc/include/asm/svm.h
+++ b/arch/powerpc/include/asm/svm.h
@@ -15,8 +15,6 @@ static inline bool is_secure_guest(void)
return mfmsr() & MSR_S;
}
-void __init svm_swiotlb_init(void);
-
void dtl_cache_ctor(void *addr);
#define get_dtl_cache_ctor() (is_secure_guest() ? dtl_cache_ctor : NULL)
@@ -27,8 +25,6 @@ static inline bool is_secure_guest(void)
return false;
}
-static inline void svm_swiotlb_init(void) {}
-
#define get_dtl_cache_ctor() NULL
#endif /* CONFIG_PPC_SVM */
diff --git a/arch/powerpc/mm/mem.c b/arch/powerpc/mm/mem.c
index d99b8b5b40ca6..a4d65418c30a9 100644
--- a/arch/powerpc/mm/mem.c
+++ b/arch/powerpc/mm/mem.c
@@ -249,10 +249,7 @@ void __init mem_init(void)
* back to to-down.
*/
memblock_set_bottom_up(true);
- if (is_secure_guest())
- svm_swiotlb_init();
- else
- swiotlb_init(ppc_swiotlb_enable, ppc_swiotlb_flags);
+ swiotlb_init(ppc_swiotlb_enable, ppc_swiotlb_flags);
#endif
high_memory = (void *) __va(max_low_pfn * PAGE_SIZE);
diff --git a/arch/powerpc/platforms/pseries/svm.c b/arch/powerpc/platforms/pseries/svm.c
index c5228f4969eb2..3b4045d508ec8 100644
--- a/arch/powerpc/platforms/pseries/svm.c
+++ b/arch/powerpc/platforms/pseries/svm.c
@@ -28,7 +28,7 @@ static int __init init_svm(void)
* need to use the SWIOTLB buffer for DMA even if dma_capable() says
* otherwise.
*/
- swiotlb_force = SWIOTLB_FORCE;
+ ppc_swiotlb_flags |= SWIOTLB_ANY | SWIOTLB_FORCE;
/* Share the SWIOTLB buffer with the host. */
swiotlb_update_mem_attributes();
@@ -37,30 +37,6 @@ static int __init init_svm(void)
}
machine_early_initcall(pseries, init_svm);
-/*
- * Initialize SWIOTLB. Essentially the same as swiotlb_init(), except that it
- * can allocate the buffer anywhere in memory. Since the hypervisor doesn't have
- * any addressing limitation, we don't need to allocate it in low addresses.
- */
-void __init svm_swiotlb_init(void)
-{
- unsigned char *vstart;
- unsigned long bytes, io_tlb_nslabs;
-
- io_tlb_nslabs = (swiotlb_size_or_default() >> IO_TLB_SHIFT);
- io_tlb_nslabs = ALIGN(io_tlb_nslabs, IO_TLB_SEGSIZE);
-
- bytes = io_tlb_nslabs << IO_TLB_SHIFT;
-
- vstart = memblock_alloc(PAGE_ALIGN(bytes), PAGE_SIZE);
- if (vstart && !swiotlb_init_with_tbl(vstart, io_tlb_nslabs, false))
- return;
-
-
- memblock_free(vstart, PAGE_ALIGN(io_tlb_nslabs << IO_TLB_SHIFT));
- panic("SVM: Cannot allocate SWIOTLB buffer");
-}
-
int set_memory_encrypted(unsigned long addr, int numpages)
{
if (!cc_platform_has(CC_ATTR_MEM_ENCRYPT))
diff --git a/include/linux/swiotlb.h b/include/linux/swiotlb.h
index dcecf953f7997..ee655f2e4d28b 100644
--- a/include/linux/swiotlb.h
+++ b/include/linux/swiotlb.h
@@ -15,6 +15,7 @@ struct scatterlist;
#define SWIOTLB_VERBOSE (1 << 0) /* verbose initialization */
#define SWIOTLB_FORCE (1 << 1) /* force bounce buffering */
+#define SWIOTLB_ANY (1 << 2) /* allow any memory for the buffer */
/*
* Maximum allowable number of contiguous slabs to map,
diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
index ad604e5a0983d..ec200e40fc397 100644
--- a/kernel/dma/swiotlb.c
+++ b/kernel/dma/swiotlb.c
@@ -275,8 +275,13 @@ void __init swiotlb_init(bool addressing_limit, unsigned int flags)
if (swiotlb_force_disable)
return;
- /* Get IO TLB memory from the low pages */
- tlb = memblock_alloc_low(bytes, PAGE_SIZE);
+ /*
+ * By default allocate the bonuce buffer memory from low memory.
+ */
+ if (flags & SWIOTLB_ANY)
+ tlb = memblock_alloc(bytes, PAGE_SIZE);
+ else
+ tlb = memblock_alloc_low(bytes, PAGE_SIZE);
if (!tlb)
goto fail;
if (swiotlb_init_with_tbl(tlb, default_nslabs, flags))
--
2.30.2
^ permalink raw reply related [flat|nested] 118+ messages in thread
* [PATCH 09/11] swiotlb: add a SWIOTLB_ANY flag to lift the low memory restriction
@ 2022-02-27 14:30 ` Christoph Hellwig
0 siblings, 0 replies; 118+ messages in thread
From: Christoph Hellwig @ 2022-02-27 14:30 UTC (permalink / raw)
To: iommu
Cc: Juergen Gross, linux-s390, linux-hyperv, Stefano Stabellini,
linux-ia64, Anshuman Khandual, Konrad Rzeszutek Wilk,
Robin Murphy, x86, linux-mips, linuxppc-dev, tboot-devel,
linux-pci, xen-devel, Boris Ostrovsky, David Woodhouse,
linux-riscv, linux-arm-kernel
Power SVM wants to allocate a swiotlb buffer that is not restricted to
low memory for the trusted hypervisor scheme. Consolidate the support
for this into the swiotlb_init interface by adding a new flag.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
arch/powerpc/include/asm/svm.h | 4 ----
arch/powerpc/mm/mem.c | 5 +----
arch/powerpc/platforms/pseries/svm.c | 26 +-------------------------
include/linux/swiotlb.h | 1 +
kernel/dma/swiotlb.c | 9 +++++++--
5 files changed, 10 insertions(+), 35 deletions(-)
diff --git a/arch/powerpc/include/asm/svm.h b/arch/powerpc/include/asm/svm.h
index 7546402d796af..85580b30aba48 100644
--- a/arch/powerpc/include/asm/svm.h
+++ b/arch/powerpc/include/asm/svm.h
@@ -15,8 +15,6 @@ static inline bool is_secure_guest(void)
return mfmsr() & MSR_S;
}
-void __init svm_swiotlb_init(void);
-
void dtl_cache_ctor(void *addr);
#define get_dtl_cache_ctor() (is_secure_guest() ? dtl_cache_ctor : NULL)
@@ -27,8 +25,6 @@ static inline bool is_secure_guest(void)
return false;
}
-static inline void svm_swiotlb_init(void) {}
-
#define get_dtl_cache_ctor() NULL
#endif /* CONFIG_PPC_SVM */
diff --git a/arch/powerpc/mm/mem.c b/arch/powerpc/mm/mem.c
index d99b8b5b40ca6..a4d65418c30a9 100644
--- a/arch/powerpc/mm/mem.c
+++ b/arch/powerpc/mm/mem.c
@@ -249,10 +249,7 @@ void __init mem_init(void)
* back to to-down.
*/
memblock_set_bottom_up(true);
- if (is_secure_guest())
- svm_swiotlb_init();
- else
- swiotlb_init(ppc_swiotlb_enable, ppc_swiotlb_flags);
+ swiotlb_init(ppc_swiotlb_enable, ppc_swiotlb_flags);
#endif
high_memory = (void *) __va(max_low_pfn * PAGE_SIZE);
diff --git a/arch/powerpc/platforms/pseries/svm.c b/arch/powerpc/platforms/pseries/svm.c
index c5228f4969eb2..3b4045d508ec8 100644
--- a/arch/powerpc/platforms/pseries/svm.c
+++ b/arch/powerpc/platforms/pseries/svm.c
@@ -28,7 +28,7 @@ static int __init init_svm(void)
* need to use the SWIOTLB buffer for DMA even if dma_capable() says
* otherwise.
*/
- swiotlb_force = SWIOTLB_FORCE;
+ ppc_swiotlb_flags |= SWIOTLB_ANY | SWIOTLB_FORCE;
/* Share the SWIOTLB buffer with the host. */
swiotlb_update_mem_attributes();
@@ -37,30 +37,6 @@ static int __init init_svm(void)
}
machine_early_initcall(pseries, init_svm);
-/*
- * Initialize SWIOTLB. Essentially the same as swiotlb_init(), except that it
- * can allocate the buffer anywhere in memory. Since the hypervisor doesn't have
- * any addressing limitation, we don't need to allocate it in low addresses.
- */
-void __init svm_swiotlb_init(void)
-{
- unsigned char *vstart;
- unsigned long bytes, io_tlb_nslabs;
-
- io_tlb_nslabs = (swiotlb_size_or_default() >> IO_TLB_SHIFT);
- io_tlb_nslabs = ALIGN(io_tlb_nslabs, IO_TLB_SEGSIZE);
-
- bytes = io_tlb_nslabs << IO_TLB_SHIFT;
-
- vstart = memblock_alloc(PAGE_ALIGN(bytes), PAGE_SIZE);
- if (vstart && !swiotlb_init_with_tbl(vstart, io_tlb_nslabs, false))
- return;
-
-
- memblock_free(vstart, PAGE_ALIGN(io_tlb_nslabs << IO_TLB_SHIFT));
- panic("SVM: Cannot allocate SWIOTLB buffer");
-}
-
int set_memory_encrypted(unsigned long addr, int numpages)
{
if (!cc_platform_has(CC_ATTR_MEM_ENCRYPT))
diff --git a/include/linux/swiotlb.h b/include/linux/swiotlb.h
index dcecf953f7997..ee655f2e4d28b 100644
--- a/include/linux/swiotlb.h
+++ b/include/linux/swiotlb.h
@@ -15,6 +15,7 @@ struct scatterlist;
#define SWIOTLB_VERBOSE (1 << 0) /* verbose initialization */
#define SWIOTLB_FORCE (1 << 1) /* force bounce buffering */
+#define SWIOTLB_ANY (1 << 2) /* allow any memory for the buffer */
/*
* Maximum allowable number of contiguous slabs to map,
diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
index ad604e5a0983d..ec200e40fc397 100644
--- a/kernel/dma/swiotlb.c
+++ b/kernel/dma/swiotlb.c
@@ -275,8 +275,13 @@ void __init swiotlb_init(bool addressing_limit, unsigned int flags)
if (swiotlb_force_disable)
return;
- /* Get IO TLB memory from the low pages */
- tlb = memblock_alloc_low(bytes, PAGE_SIZE);
+ /*
+ * By default allocate the bonuce buffer memory from low memory.
+ */
+ if (flags & SWIOTLB_ANY)
+ tlb = memblock_alloc(bytes, PAGE_SIZE);
+ else
+ tlb = memblock_alloc_low(bytes, PAGE_SIZE);
if (!tlb)
goto fail;
if (swiotlb_init_with_tbl(tlb, default_nslabs, flags))
--
2.30.2
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply related [flat|nested] 118+ messages in thread
* [PATCH 09/11] swiotlb: add a SWIOTLB_ANY flag to lift the low memory restriction
@ 2022-02-27 14:30 ` Christoph Hellwig
0 siblings, 0 replies; 118+ messages in thread
From: Christoph Hellwig @ 2022-02-27 14:30 UTC (permalink / raw)
To: iommu
Cc: x86, Anshuman Khandual, Konrad Rzeszutek Wilk,
Stefano Stabellini, Boris Ostrovsky, Juergen Gross, Joerg Roedel,
David Woodhouse, Lu Baolu, Robin Murphy, linux-arm-kernel,
xen-devel, linux-ia64, linux-mips, linuxppc-dev, linux-riscv,
linux-s390, linux-hyperv, tboot-devel, linux-pci
Power SVM wants to allocate a swiotlb buffer that is not restricted to
low memory for the trusted hypervisor scheme. Consolidate the support
for this into the swiotlb_init interface by adding a new flag.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
arch/powerpc/include/asm/svm.h | 4 ----
arch/powerpc/mm/mem.c | 5 +----
arch/powerpc/platforms/pseries/svm.c | 26 +-------------------------
include/linux/swiotlb.h | 1 +
kernel/dma/swiotlb.c | 9 +++++++--
5 files changed, 10 insertions(+), 35 deletions(-)
diff --git a/arch/powerpc/include/asm/svm.h b/arch/powerpc/include/asm/svm.h
index 7546402d796af..85580b30aba48 100644
--- a/arch/powerpc/include/asm/svm.h
+++ b/arch/powerpc/include/asm/svm.h
@@ -15,8 +15,6 @@ static inline bool is_secure_guest(void)
return mfmsr() & MSR_S;
}
-void __init svm_swiotlb_init(void);
-
void dtl_cache_ctor(void *addr);
#define get_dtl_cache_ctor() (is_secure_guest() ? dtl_cache_ctor : NULL)
@@ -27,8 +25,6 @@ static inline bool is_secure_guest(void)
return false;
}
-static inline void svm_swiotlb_init(void) {}
-
#define get_dtl_cache_ctor() NULL
#endif /* CONFIG_PPC_SVM */
diff --git a/arch/powerpc/mm/mem.c b/arch/powerpc/mm/mem.c
index d99b8b5b40ca6..a4d65418c30a9 100644
--- a/arch/powerpc/mm/mem.c
+++ b/arch/powerpc/mm/mem.c
@@ -249,10 +249,7 @@ void __init mem_init(void)
* back to to-down.
*/
memblock_set_bottom_up(true);
- if (is_secure_guest())
- svm_swiotlb_init();
- else
- swiotlb_init(ppc_swiotlb_enable, ppc_swiotlb_flags);
+ swiotlb_init(ppc_swiotlb_enable, ppc_swiotlb_flags);
#endif
high_memory = (void *) __va(max_low_pfn * PAGE_SIZE);
diff --git a/arch/powerpc/platforms/pseries/svm.c b/arch/powerpc/platforms/pseries/svm.c
index c5228f4969eb2..3b4045d508ec8 100644
--- a/arch/powerpc/platforms/pseries/svm.c
+++ b/arch/powerpc/platforms/pseries/svm.c
@@ -28,7 +28,7 @@ static int __init init_svm(void)
* need to use the SWIOTLB buffer for DMA even if dma_capable() says
* otherwise.
*/
- swiotlb_force = SWIOTLB_FORCE;
+ ppc_swiotlb_flags |= SWIOTLB_ANY | SWIOTLB_FORCE;
/* Share the SWIOTLB buffer with the host. */
swiotlb_update_mem_attributes();
@@ -37,30 +37,6 @@ static int __init init_svm(void)
}
machine_early_initcall(pseries, init_svm);
-/*
- * Initialize SWIOTLB. Essentially the same as swiotlb_init(), except that it
- * can allocate the buffer anywhere in memory. Since the hypervisor doesn't have
- * any addressing limitation, we don't need to allocate it in low addresses.
- */
-void __init svm_swiotlb_init(void)
-{
- unsigned char *vstart;
- unsigned long bytes, io_tlb_nslabs;
-
- io_tlb_nslabs = (swiotlb_size_or_default() >> IO_TLB_SHIFT);
- io_tlb_nslabs = ALIGN(io_tlb_nslabs, IO_TLB_SEGSIZE);
-
- bytes = io_tlb_nslabs << IO_TLB_SHIFT;
-
- vstart = memblock_alloc(PAGE_ALIGN(bytes), PAGE_SIZE);
- if (vstart && !swiotlb_init_with_tbl(vstart, io_tlb_nslabs, false))
- return;
-
-
- memblock_free(vstart, PAGE_ALIGN(io_tlb_nslabs << IO_TLB_SHIFT));
- panic("SVM: Cannot allocate SWIOTLB buffer");
-}
-
int set_memory_encrypted(unsigned long addr, int numpages)
{
if (!cc_platform_has(CC_ATTR_MEM_ENCRYPT))
diff --git a/include/linux/swiotlb.h b/include/linux/swiotlb.h
index dcecf953f7997..ee655f2e4d28b 100644
--- a/include/linux/swiotlb.h
+++ b/include/linux/swiotlb.h
@@ -15,6 +15,7 @@ struct scatterlist;
#define SWIOTLB_VERBOSE (1 << 0) /* verbose initialization */
#define SWIOTLB_FORCE (1 << 1) /* force bounce buffering */
+#define SWIOTLB_ANY (1 << 2) /* allow any memory for the buffer */
/*
* Maximum allowable number of contiguous slabs to map,
diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
index ad604e5a0983d..ec200e40fc397 100644
--- a/kernel/dma/swiotlb.c
+++ b/kernel/dma/swiotlb.c
@@ -275,8 +275,13 @@ void __init swiotlb_init(bool addressing_limit, unsigned int flags)
if (swiotlb_force_disable)
return;
- /* Get IO TLB memory from the low pages */
- tlb = memblock_alloc_low(bytes, PAGE_SIZE);
+ /*
+ * By default allocate the bonuce buffer memory from low memory.
+ */
+ if (flags & SWIOTLB_ANY)
+ tlb = memblock_alloc(bytes, PAGE_SIZE);
+ else
+ tlb = memblock_alloc_low(bytes, PAGE_SIZE);
if (!tlb)
goto fail;
if (swiotlb_init_with_tbl(tlb, default_nslabs, flags))
--
2.30.2
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply related [flat|nested] 118+ messages in thread
* [PATCH 09/11] swiotlb: add a SWIOTLB_ANY flag to lift the low memory restriction
@ 2022-02-27 14:30 ` Christoph Hellwig
0 siblings, 0 replies; 118+ messages in thread
From: Christoph Hellwig @ 2022-02-27 14:30 UTC (permalink / raw)
To: iommu
Cc: Juergen Gross, linux-s390, linux-hyperv, Stefano Stabellini,
linux-ia64, Anshuman Khandual, Konrad Rzeszutek Wilk,
Robin Murphy, Joerg Roedel, x86, linux-mips, linuxppc-dev,
tboot-devel, linux-pci, xen-devel, Boris Ostrovsky,
David Woodhouse, linux-riscv, linux-arm-kernel, Lu Baolu
Power SVM wants to allocate a swiotlb buffer that is not restricted to
low memory for the trusted hypervisor scheme. Consolidate the support
for this into the swiotlb_init interface by adding a new flag.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
arch/powerpc/include/asm/svm.h | 4 ----
arch/powerpc/mm/mem.c | 5 +----
arch/powerpc/platforms/pseries/svm.c | 26 +-------------------------
include/linux/swiotlb.h | 1 +
kernel/dma/swiotlb.c | 9 +++++++--
5 files changed, 10 insertions(+), 35 deletions(-)
diff --git a/arch/powerpc/include/asm/svm.h b/arch/powerpc/include/asm/svm.h
index 7546402d796af..85580b30aba48 100644
--- a/arch/powerpc/include/asm/svm.h
+++ b/arch/powerpc/include/asm/svm.h
@@ -15,8 +15,6 @@ static inline bool is_secure_guest(void)
return mfmsr() & MSR_S;
}
-void __init svm_swiotlb_init(void);
-
void dtl_cache_ctor(void *addr);
#define get_dtl_cache_ctor() (is_secure_guest() ? dtl_cache_ctor : NULL)
@@ -27,8 +25,6 @@ static inline bool is_secure_guest(void)
return false;
}
-static inline void svm_swiotlb_init(void) {}
-
#define get_dtl_cache_ctor() NULL
#endif /* CONFIG_PPC_SVM */
diff --git a/arch/powerpc/mm/mem.c b/arch/powerpc/mm/mem.c
index d99b8b5b40ca6..a4d65418c30a9 100644
--- a/arch/powerpc/mm/mem.c
+++ b/arch/powerpc/mm/mem.c
@@ -249,10 +249,7 @@ void __init mem_init(void)
* back to to-down.
*/
memblock_set_bottom_up(true);
- if (is_secure_guest())
- svm_swiotlb_init();
- else
- swiotlb_init(ppc_swiotlb_enable, ppc_swiotlb_flags);
+ swiotlb_init(ppc_swiotlb_enable, ppc_swiotlb_flags);
#endif
high_memory = (void *) __va(max_low_pfn * PAGE_SIZE);
diff --git a/arch/powerpc/platforms/pseries/svm.c b/arch/powerpc/platforms/pseries/svm.c
index c5228f4969eb2..3b4045d508ec8 100644
--- a/arch/powerpc/platforms/pseries/svm.c
+++ b/arch/powerpc/platforms/pseries/svm.c
@@ -28,7 +28,7 @@ static int __init init_svm(void)
* need to use the SWIOTLB buffer for DMA even if dma_capable() says
* otherwise.
*/
- swiotlb_force = SWIOTLB_FORCE;
+ ppc_swiotlb_flags |= SWIOTLB_ANY | SWIOTLB_FORCE;
/* Share the SWIOTLB buffer with the host. */
swiotlb_update_mem_attributes();
@@ -37,30 +37,6 @@ static int __init init_svm(void)
}
machine_early_initcall(pseries, init_svm);
-/*
- * Initialize SWIOTLB. Essentially the same as swiotlb_init(), except that it
- * can allocate the buffer anywhere in memory. Since the hypervisor doesn't have
- * any addressing limitation, we don't need to allocate it in low addresses.
- */
-void __init svm_swiotlb_init(void)
-{
- unsigned char *vstart;
- unsigned long bytes, io_tlb_nslabs;
-
- io_tlb_nslabs = (swiotlb_size_or_default() >> IO_TLB_SHIFT);
- io_tlb_nslabs = ALIGN(io_tlb_nslabs, IO_TLB_SEGSIZE);
-
- bytes = io_tlb_nslabs << IO_TLB_SHIFT;
-
- vstart = memblock_alloc(PAGE_ALIGN(bytes), PAGE_SIZE);
- if (vstart && !swiotlb_init_with_tbl(vstart, io_tlb_nslabs, false))
- return;
-
-
- memblock_free(vstart, PAGE_ALIGN(io_tlb_nslabs << IO_TLB_SHIFT));
- panic("SVM: Cannot allocate SWIOTLB buffer");
-}
-
int set_memory_encrypted(unsigned long addr, int numpages)
{
if (!cc_platform_has(CC_ATTR_MEM_ENCRYPT))
diff --git a/include/linux/swiotlb.h b/include/linux/swiotlb.h
index dcecf953f7997..ee655f2e4d28b 100644
--- a/include/linux/swiotlb.h
+++ b/include/linux/swiotlb.h
@@ -15,6 +15,7 @@ struct scatterlist;
#define SWIOTLB_VERBOSE (1 << 0) /* verbose initialization */
#define SWIOTLB_FORCE (1 << 1) /* force bounce buffering */
+#define SWIOTLB_ANY (1 << 2) /* allow any memory for the buffer */
/*
* Maximum allowable number of contiguous slabs to map,
diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
index ad604e5a0983d..ec200e40fc397 100644
--- a/kernel/dma/swiotlb.c
+++ b/kernel/dma/swiotlb.c
@@ -275,8 +275,13 @@ void __init swiotlb_init(bool addressing_limit, unsigned int flags)
if (swiotlb_force_disable)
return;
- /* Get IO TLB memory from the low pages */
- tlb = memblock_alloc_low(bytes, PAGE_SIZE);
+ /*
+ * By default allocate the bonuce buffer memory from low memory.
+ */
+ if (flags & SWIOTLB_ANY)
+ tlb = memblock_alloc(bytes, PAGE_SIZE);
+ else
+ tlb = memblock_alloc_low(bytes, PAGE_SIZE);
if (!tlb)
goto fail;
if (swiotlb_init_with_tbl(tlb, default_nslabs, flags))
--
2.30.2
^ permalink raw reply related [flat|nested] 118+ messages in thread
* [PATCH 09/11] swiotlb: add a SWIOTLB_ANY flag to lift the low memory restriction
@ 2022-02-27 14:30 ` Christoph Hellwig
0 siblings, 0 replies; 118+ messages in thread
From: Christoph Hellwig @ 2022-02-27 14:30 UTC (permalink / raw)
To: iommu
Cc: x86, Anshuman Khandual, Konrad Rzeszutek Wilk,
Stefano Stabellini, Boris Ostrovsky, Juergen Gross, Joerg Roedel,
David Woodhouse, Lu Baolu, Robin Murphy, linux-arm-kernel,
xen-devel, linux-ia64, linux-mips, linuxppc-dev, linux-riscv,
linux-s390, linux-hyperv, tboot-devel, linux-pci
Power SVM wants to allocate a swiotlb buffer that is not restricted to
low memory for the trusted hypervisor scheme. Consolidate the support
for this into the swiotlb_init interface by adding a new flag.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
arch/powerpc/include/asm/svm.h | 4 ----
arch/powerpc/mm/mem.c | 5 +----
arch/powerpc/platforms/pseries/svm.c | 26 +-------------------------
include/linux/swiotlb.h | 1 +
kernel/dma/swiotlb.c | 9 +++++++--
5 files changed, 10 insertions(+), 35 deletions(-)
diff --git a/arch/powerpc/include/asm/svm.h b/arch/powerpc/include/asm/svm.h
index 7546402d796af..85580b30aba48 100644
--- a/arch/powerpc/include/asm/svm.h
+++ b/arch/powerpc/include/asm/svm.h
@@ -15,8 +15,6 @@ static inline bool is_secure_guest(void)
return mfmsr() & MSR_S;
}
-void __init svm_swiotlb_init(void);
-
void dtl_cache_ctor(void *addr);
#define get_dtl_cache_ctor() (is_secure_guest() ? dtl_cache_ctor : NULL)
@@ -27,8 +25,6 @@ static inline bool is_secure_guest(void)
return false;
}
-static inline void svm_swiotlb_init(void) {}
-
#define get_dtl_cache_ctor() NULL
#endif /* CONFIG_PPC_SVM */
diff --git a/arch/powerpc/mm/mem.c b/arch/powerpc/mm/mem.c
index d99b8b5b40ca6..a4d65418c30a9 100644
--- a/arch/powerpc/mm/mem.c
+++ b/arch/powerpc/mm/mem.c
@@ -249,10 +249,7 @@ void __init mem_init(void)
* back to to-down.
*/
memblock_set_bottom_up(true);
- if (is_secure_guest())
- svm_swiotlb_init();
- else
- swiotlb_init(ppc_swiotlb_enable, ppc_swiotlb_flags);
+ swiotlb_init(ppc_swiotlb_enable, ppc_swiotlb_flags);
#endif
high_memory = (void *) __va(max_low_pfn * PAGE_SIZE);
diff --git a/arch/powerpc/platforms/pseries/svm.c b/arch/powerpc/platforms/pseries/svm.c
index c5228f4969eb2..3b4045d508ec8 100644
--- a/arch/powerpc/platforms/pseries/svm.c
+++ b/arch/powerpc/platforms/pseries/svm.c
@@ -28,7 +28,7 @@ static int __init init_svm(void)
* need to use the SWIOTLB buffer for DMA even if dma_capable() says
* otherwise.
*/
- swiotlb_force = SWIOTLB_FORCE;
+ ppc_swiotlb_flags |= SWIOTLB_ANY | SWIOTLB_FORCE;
/* Share the SWIOTLB buffer with the host. */
swiotlb_update_mem_attributes();
@@ -37,30 +37,6 @@ static int __init init_svm(void)
}
machine_early_initcall(pseries, init_svm);
-/*
- * Initialize SWIOTLB. Essentially the same as swiotlb_init(), except that it
- * can allocate the buffer anywhere in memory. Since the hypervisor doesn't have
- * any addressing limitation, we don't need to allocate it in low addresses.
- */
-void __init svm_swiotlb_init(void)
-{
- unsigned char *vstart;
- unsigned long bytes, io_tlb_nslabs;
-
- io_tlb_nslabs = (swiotlb_size_or_default() >> IO_TLB_SHIFT);
- io_tlb_nslabs = ALIGN(io_tlb_nslabs, IO_TLB_SEGSIZE);
-
- bytes = io_tlb_nslabs << IO_TLB_SHIFT;
-
- vstart = memblock_alloc(PAGE_ALIGN(bytes), PAGE_SIZE);
- if (vstart && !swiotlb_init_with_tbl(vstart, io_tlb_nslabs, false))
- return;
-
-
- memblock_free(vstart, PAGE_ALIGN(io_tlb_nslabs << IO_TLB_SHIFT));
- panic("SVM: Cannot allocate SWIOTLB buffer");
-}
-
int set_memory_encrypted(unsigned long addr, int numpages)
{
if (!cc_platform_has(CC_ATTR_MEM_ENCRYPT))
diff --git a/include/linux/swiotlb.h b/include/linux/swiotlb.h
index dcecf953f7997..ee655f2e4d28b 100644
--- a/include/linux/swiotlb.h
+++ b/include/linux/swiotlb.h
@@ -15,6 +15,7 @@ struct scatterlist;
#define SWIOTLB_VERBOSE (1 << 0) /* verbose initialization */
#define SWIOTLB_FORCE (1 << 1) /* force bounce buffering */
+#define SWIOTLB_ANY (1 << 2) /* allow any memory for the buffer */
/*
* Maximum allowable number of contiguous slabs to map,
diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
index ad604e5a0983d..ec200e40fc397 100644
--- a/kernel/dma/swiotlb.c
+++ b/kernel/dma/swiotlb.c
@@ -275,8 +275,13 @@ void __init swiotlb_init(bool addressing_limit, unsigned int flags)
if (swiotlb_force_disable)
return;
- /* Get IO TLB memory from the low pages */
- tlb = memblock_alloc_low(bytes, PAGE_SIZE);
+ /*
+ * By default allocate the bonuce buffer memory from low memory.
+ */
+ if (flags & SWIOTLB_ANY)
+ tlb = memblock_alloc(bytes, PAGE_SIZE);
+ else
+ tlb = memblock_alloc_low(bytes, PAGE_SIZE);
if (!tlb)
goto fail;
if (swiotlb_init_with_tbl(tlb, default_nslabs, flags))
--
2.30.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 118+ messages in thread
* [PATCH 09/11] swiotlb: add a SWIOTLB_ANY flag to lift the low memory restriction
@ 2022-02-27 14:30 ` Christoph Hellwig
0 siblings, 0 replies; 118+ messages in thread
From: Christoph Hellwig @ 2022-02-27 14:30 UTC (permalink / raw)
To: iommu
Cc: x86, Anshuman Khandual, Konrad Rzeszutek Wilk,
Stefano Stabellini, Boris Ostrovsky, Juergen Gross, Joerg Roedel,
David Woodhouse, Lu Baolu, Robin Murphy, linux-arm-kernel,
xen-devel, linux-ia64, linux-mips, linuxppc-dev, linux-riscv,
linux-s390, linux-hyperv, tboot-devel, linux-pci
Power SVM wants to allocate a swiotlb buffer that is not restricted to
low memory for the trusted hypervisor scheme. Consolidate the support
for this into the swiotlb_init interface by adding a new flag.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
arch/powerpc/include/asm/svm.h | 4 ----
arch/powerpc/mm/mem.c | 5 +----
arch/powerpc/platforms/pseries/svm.c | 26 +-------------------------
include/linux/swiotlb.h | 1 +
kernel/dma/swiotlb.c | 9 +++++++--
5 files changed, 10 insertions(+), 35 deletions(-)
diff --git a/arch/powerpc/include/asm/svm.h b/arch/powerpc/include/asm/svm.h
index 7546402d796af..85580b30aba48 100644
--- a/arch/powerpc/include/asm/svm.h
+++ b/arch/powerpc/include/asm/svm.h
@@ -15,8 +15,6 @@ static inline bool is_secure_guest(void)
return mfmsr() & MSR_S;
}
-void __init svm_swiotlb_init(void);
-
void dtl_cache_ctor(void *addr);
#define get_dtl_cache_ctor() (is_secure_guest() ? dtl_cache_ctor : NULL)
@@ -27,8 +25,6 @@ static inline bool is_secure_guest(void)
return false;
}
-static inline void svm_swiotlb_init(void) {}
-
#define get_dtl_cache_ctor() NULL
#endif /* CONFIG_PPC_SVM */
diff --git a/arch/powerpc/mm/mem.c b/arch/powerpc/mm/mem.c
index d99b8b5b40ca6..a4d65418c30a9 100644
--- a/arch/powerpc/mm/mem.c
+++ b/arch/powerpc/mm/mem.c
@@ -249,10 +249,7 @@ void __init mem_init(void)
* back to to-down.
*/
memblock_set_bottom_up(true);
- if (is_secure_guest())
- svm_swiotlb_init();
- else
- swiotlb_init(ppc_swiotlb_enable, ppc_swiotlb_flags);
+ swiotlb_init(ppc_swiotlb_enable, ppc_swiotlb_flags);
#endif
high_memory = (void *) __va(max_low_pfn * PAGE_SIZE);
diff --git a/arch/powerpc/platforms/pseries/svm.c b/arch/powerpc/platforms/pseries/svm.c
index c5228f4969eb2..3b4045d508ec8 100644
--- a/arch/powerpc/platforms/pseries/svm.c
+++ b/arch/powerpc/platforms/pseries/svm.c
@@ -28,7 +28,7 @@ static int __init init_svm(void)
* need to use the SWIOTLB buffer for DMA even if dma_capable() says
* otherwise.
*/
- swiotlb_force = SWIOTLB_FORCE;
+ ppc_swiotlb_flags |= SWIOTLB_ANY | SWIOTLB_FORCE;
/* Share the SWIOTLB buffer with the host. */
swiotlb_update_mem_attributes();
@@ -37,30 +37,6 @@ static int __init init_svm(void)
}
machine_early_initcall(pseries, init_svm);
-/*
- * Initialize SWIOTLB. Essentially the same as swiotlb_init(), except that it
- * can allocate the buffer anywhere in memory. Since the hypervisor doesn't have
- * any addressing limitation, we don't need to allocate it in low addresses.
- */
-void __init svm_swiotlb_init(void)
-{
- unsigned char *vstart;
- unsigned long bytes, io_tlb_nslabs;
-
- io_tlb_nslabs = (swiotlb_size_or_default() >> IO_TLB_SHIFT);
- io_tlb_nslabs = ALIGN(io_tlb_nslabs, IO_TLB_SEGSIZE);
-
- bytes = io_tlb_nslabs << IO_TLB_SHIFT;
-
- vstart = memblock_alloc(PAGE_ALIGN(bytes), PAGE_SIZE);
- if (vstart && !swiotlb_init_with_tbl(vstart, io_tlb_nslabs, false))
- return;
-
-
- memblock_free(vstart, PAGE_ALIGN(io_tlb_nslabs << IO_TLB_SHIFT));
- panic("SVM: Cannot allocate SWIOTLB buffer");
-}
-
int set_memory_encrypted(unsigned long addr, int numpages)
{
if (!cc_platform_has(CC_ATTR_MEM_ENCRYPT))
diff --git a/include/linux/swiotlb.h b/include/linux/swiotlb.h
index dcecf953f7997..ee655f2e4d28b 100644
--- a/include/linux/swiotlb.h
+++ b/include/linux/swiotlb.h
@@ -15,6 +15,7 @@ struct scatterlist;
#define SWIOTLB_VERBOSE (1 << 0) /* verbose initialization */
#define SWIOTLB_FORCE (1 << 1) /* force bounce buffering */
+#define SWIOTLB_ANY (1 << 2) /* allow any memory for the buffer */
/*
* Maximum allowable number of contiguous slabs to map,
diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
index ad604e5a0983d..ec200e40fc397 100644
--- a/kernel/dma/swiotlb.c
+++ b/kernel/dma/swiotlb.c
@@ -275,8 +275,13 @@ void __init swiotlb_init(bool addressing_limit, unsigned int flags)
if (swiotlb_force_disable)
return;
- /* Get IO TLB memory from the low pages */
- tlb = memblock_alloc_low(bytes, PAGE_SIZE);
+ /*
+ * By default allocate the bonuce buffer memory from low memory.
+ */
+ if (flags & SWIOTLB_ANY)
+ tlb = memblock_alloc(bytes, PAGE_SIZE);
+ else
+ tlb = memblock_alloc_low(bytes, PAGE_SIZE);
if (!tlb)
goto fail;
if (swiotlb_init_with_tbl(tlb, default_nslabs, flags))
--
2.30.2
^ permalink raw reply related [flat|nested] 118+ messages in thread
* [PATCH 10/11] swiotlb: merge swiotlb-xen initialization into swiotlb
2022-02-27 14:30 ` Christoph Hellwig
` (3 preceding siblings ...)
(?)
@ 2022-02-27 14:30 ` Christoph Hellwig
-1 siblings, 0 replies; 118+ messages in thread
From: Christoph Hellwig @ 2022-02-27 14:30 UTC (permalink / raw)
To: iommu
Cc: x86, Anshuman Khandual, Konrad Rzeszutek Wilk,
Stefano Stabellini, Boris Ostrovsky, Juergen Gross, Joerg Roedel,
David Woodhouse, Lu Baolu, Robin Murphy, linux-arm-kernel,
xen-devel, linux-ia64, linux-mips, linuxppc-dev, linux-riscv,
linux-s390, linux-hyperv, tboot-devel, linux-pci
Allow to pass a remap argument to the swiotlb initialization functions
to handle the Xen/x86 remap case. ARM/ARM64 never did any remapping
from xen_swiotlb_fixup, so we don't even need that quirk.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
arch/arm/xen/mm.c | 23 +++---
arch/x86/include/asm/xen/page.h | 5 --
arch/x86/kernel/pci-dma.c | 19 +++--
arch/x86/pci/sta2x11-fixup.c | 2 +-
drivers/xen/swiotlb-xen.c | 128 +-------------------------------
include/linux/swiotlb.h | 7 +-
include/xen/arm/page.h | 1 -
include/xen/swiotlb-xen.h | 8 +-
kernel/dma/swiotlb.c | 120 +++++++++++++++---------------
9 files changed, 96 insertions(+), 217 deletions(-)
diff --git a/arch/arm/xen/mm.c b/arch/arm/xen/mm.c
index a7e54a087b802..58b40f87617d3 100644
--- a/arch/arm/xen/mm.c
+++ b/arch/arm/xen/mm.c
@@ -23,22 +23,20 @@
#include <asm/xen/hypercall.h>
#include <asm/xen/interface.h>
-unsigned long xen_get_swiotlb_free_pages(unsigned int order)
+static gfp_t xen_swiotlb_gfp(void)
{
phys_addr_t base;
- gfp_t flags = __GFP_NOWARN|__GFP_KSWAPD_RECLAIM;
u64 i;
for_each_mem_range(i, &base, NULL) {
if (base < (phys_addr_t)0xffffffff) {
if (IS_ENABLED(CONFIG_ZONE_DMA32))
- flags |= __GFP_DMA32;
- else
- flags |= __GFP_DMA;
- break;
+ return __GFP_DMA32;
+ return __GFP_DMA;
}
}
- return __get_free_pages(flags, order);
+
+ return GFP_KERNEL;
}
static bool hypercall_cflush = false;
@@ -143,10 +141,15 @@ static int __init xen_mm_init(void)
if (!xen_swiotlb_detect())
return 0;
- rc = xen_swiotlb_init();
/* we can work with the default swiotlb */
- if (rc < 0 && rc != -EEXIST)
- return rc;
+ if (!io_tlb_default_mem.nslabs) {
+ if (!xen_initial_domain())
+ return -EINVAL;
+ rc = swiotlb_init_late(swiotlb_size_or_default(),
+ xen_swiotlb_gfp(), NULL);
+ if (rc < 0)
+ return rc;
+ }
cflush.op = 0;
cflush.a.dev_bus_addr = 0;
diff --git a/arch/x86/include/asm/xen/page.h b/arch/x86/include/asm/xen/page.h
index e989bc2269f54..1fc67df500145 100644
--- a/arch/x86/include/asm/xen/page.h
+++ b/arch/x86/include/asm/xen/page.h
@@ -357,9 +357,4 @@ static inline bool xen_arch_need_swiotlb(struct device *dev,
return false;
}
-static inline unsigned long xen_get_swiotlb_free_pages(unsigned int order)
-{
- return __get_free_pages(__GFP_NOWARN, order);
-}
-
#endif /* _ASM_X86_XEN_PAGE_H */
diff --git a/arch/x86/kernel/pci-dma.c b/arch/x86/kernel/pci-dma.c
index bb08184a50e3a..b0901f027cdcd 100644
--- a/arch/x86/kernel/pci-dma.c
+++ b/arch/x86/kernel/pci-dma.c
@@ -68,15 +68,12 @@ static inline void __init pci_swiotlb_detect_4gb(void)
#endif /* CONFIG_SWIOTLB */
#ifdef CONFIG_SWIOTLB_XEN
-static bool xen_swiotlb;
-
static void __init pci_xen_swiotlb_init(void)
{
if (!xen_initial_domain() && !x86_swiotlb_enable)
return;
x86_swiotlb_enable = false;
- xen_swiotlb = true;
- xen_swiotlb_init_early();
+ swiotlb_init_remap(true, x86_swiotlb_flags, xen_swiotlb_fixup);
dma_ops = &xen_swiotlb_dma_ops;
if (IS_ENABLED(CONFIG_PCI))
pci_request_acs();
@@ -84,14 +81,16 @@ static void __init pci_xen_swiotlb_init(void)
int pci_xen_swiotlb_init_late(void)
{
- int rc;
-
- if (xen_swiotlb)
+ if (dma_ops == &xen_swiotlb_dma_ops)
return 0;
- rc = xen_swiotlb_init();
- if (rc)
- return rc;
+ /* we can work with the default swiotlb */
+ if (!io_tlb_default_mem.nslabs) {
+ int rc = swiotlb_init_late(swiotlb_size_or_default(),
+ GFP_KERNEL, xen_swiotlb_fixup);
+ if (rc < 0)
+ return rc;
+ }
/* XXX: this switches the dma ops under live devices! */
dma_ops = &xen_swiotlb_dma_ops;
diff --git a/arch/x86/pci/sta2x11-fixup.c b/arch/x86/pci/sta2x11-fixup.c
index c7e6faf59a861..7368afc039987 100644
--- a/arch/x86/pci/sta2x11-fixup.c
+++ b/arch/x86/pci/sta2x11-fixup.c
@@ -57,7 +57,7 @@ static void sta2x11_new_instance(struct pci_dev *pdev)
int size = STA2X11_SWIOTLB_SIZE;
/* First instance: register your own swiotlb area */
dev_info(&pdev->dev, "Using SWIOTLB (size %i)\n", size);
- if (swiotlb_init_late(size, GFP_DMA))
+ if (swiotlb_init_late(size, GFP_DMA, NULL))
dev_emerg(&pdev->dev, "init swiotlb failed\n");
}
list_add(&instance->list, &sta2x11_instance_list);
diff --git a/drivers/xen/swiotlb-xen.c b/drivers/xen/swiotlb-xen.c
index c2da3eb4826e8..df8085b50df10 100644
--- a/drivers/xen/swiotlb-xen.c
+++ b/drivers/xen/swiotlb-xen.c
@@ -104,7 +104,7 @@ static int is_xen_swiotlb_buffer(struct device *dev, dma_addr_t dma_addr)
return 0;
}
-static int xen_swiotlb_fixup(void *buf, unsigned long nslabs)
+int xen_swiotlb_fixup(void *buf, unsigned long nslabs)
{
int rc;
unsigned int order = get_order(IO_TLB_SEGSIZE << IO_TLB_SHIFT);
@@ -130,132 +130,6 @@ static int xen_swiotlb_fixup(void *buf, unsigned long nslabs)
return 0;
}
-enum xen_swiotlb_err {
- XEN_SWIOTLB_UNKNOWN = 0,
- XEN_SWIOTLB_ENOMEM,
- XEN_SWIOTLB_EFIXUP
-};
-
-static const char *xen_swiotlb_error(enum xen_swiotlb_err err)
-{
- switch (err) {
- case XEN_SWIOTLB_ENOMEM:
- return "Cannot allocate Xen-SWIOTLB buffer\n";
- case XEN_SWIOTLB_EFIXUP:
- return "Failed to get contiguous memory for DMA from Xen!\n"\
- "You either: don't have the permissions, do not have"\
- " enough free memory under 4GB, or the hypervisor memory"\
- " is too fragmented!";
- default:
- break;
- }
- return "";
-}
-
-int xen_swiotlb_init(void)
-{
- enum xen_swiotlb_err m_ret = XEN_SWIOTLB_UNKNOWN;
- unsigned long bytes = swiotlb_size_or_default();
- unsigned long nslabs = bytes >> IO_TLB_SHIFT;
- unsigned int order, repeat = 3;
- int rc = -ENOMEM;
- char *start;
-
- if (io_tlb_default_mem.nslabs) {
- pr_warn("swiotlb buffer already initialized\n");
- return -EEXIST;
- }
-
-retry:
- m_ret = XEN_SWIOTLB_ENOMEM;
- order = get_order(bytes);
-
- /*
- * Get IO TLB memory from any location.
- */
-#define SLABS_PER_PAGE (1 << (PAGE_SHIFT - IO_TLB_SHIFT))
-#define IO_TLB_MIN_SLABS ((1<<20) >> IO_TLB_SHIFT)
- while ((SLABS_PER_PAGE << order) > IO_TLB_MIN_SLABS) {
- start = (void *)xen_get_swiotlb_free_pages(order);
- if (start)
- break;
- order--;
- }
- if (!start)
- goto exit;
- if (order != get_order(bytes)) {
- pr_warn("Warning: only able to allocate %ld MB for software IO TLB\n",
- (PAGE_SIZE << order) >> 20);
- nslabs = SLABS_PER_PAGE << order;
- bytes = nslabs << IO_TLB_SHIFT;
- }
-
- /*
- * And replace that memory with pages under 4GB.
- */
- rc = xen_swiotlb_fixup(start, nslabs);
- if (rc) {
- free_pages((unsigned long)start, order);
- m_ret = XEN_SWIOTLB_EFIXUP;
- goto error;
- }
- rc = swiotlb_late_init_with_tbl(start, nslabs);
- if (rc)
- return rc;
- return 0;
-error:
- if (nslabs > 1024 && repeat--) {
- /* Min is 2MB */
- nslabs = max(1024UL, ALIGN(nslabs >> 1, IO_TLB_SEGSIZE));
- bytes = nslabs << IO_TLB_SHIFT;
- pr_info("Lowering to %luMB\n", bytes >> 20);
- goto retry;
- }
-exit:
- pr_err("%s (rc:%d)\n", xen_swiotlb_error(m_ret), rc);
- return rc;
-}
-
-#ifdef CONFIG_X86
-void __init xen_swiotlb_init_early(void)
-{
- unsigned long bytes = swiotlb_size_or_default();
- unsigned long nslabs = bytes >> IO_TLB_SHIFT;
- unsigned int repeat = 3;
- char *start;
- int rc;
-
-retry:
- /*
- * Get IO TLB memory from any location.
- */
- start = memblock_alloc(PAGE_ALIGN(bytes),
- IO_TLB_SEGSIZE << IO_TLB_SHIFT);
- if (!start)
- panic("%s: Failed to allocate %lu bytes\n",
- __func__, PAGE_ALIGN(bytes));
-
- /*
- * And replace that memory with pages under 4GB.
- */
- rc = xen_swiotlb_fixup(start, nslabs);
- if (rc) {
- memblock_free(start, PAGE_ALIGN(bytes));
- if (nslabs > 1024 && repeat--) {
- /* Min is 2MB */
- nslabs = max(1024UL, ALIGN(nslabs >> 1, IO_TLB_SEGSIZE));
- bytes = nslabs << IO_TLB_SHIFT;
- pr_info("Lowering to %luMB\n", bytes >> 20);
- goto retry;
- }
- panic("%s (rc:%d)", xen_swiotlb_error(XEN_SWIOTLB_EFIXUP), rc);
- }
-
- if (swiotlb_init_with_tbl(start, nslabs, SWIOTLB_VERBOSE))
- panic("Cannot allocate SWIOTLB buffer");
-}
-#endif /* CONFIG_X86 */
-
static void *
xen_swiotlb_alloc_coherent(struct device *hwdev, size_t size,
dma_addr_t *dma_handle, gfp_t flags,
diff --git a/include/linux/swiotlb.h b/include/linux/swiotlb.h
index ee655f2e4d28b..919cf82ed978e 100644
--- a/include/linux/swiotlb.h
+++ b/include/linux/swiotlb.h
@@ -34,10 +34,11 @@ struct scatterlist;
/* default to 64MB */
#define IO_TLB_DEFAULT_SIZE (64UL<<20)
-int swiotlb_init_with_tbl(char *tlb, unsigned long nslabs, unsigned int flags);
unsigned long swiotlb_size_or_default(void);
-extern int swiotlb_late_init_with_tbl(char *tlb, unsigned long nslabs);
-int swiotlb_init_late(size_t size, gfp_t gfp_mask);
+int swiotlb_init_late(size_t size, gfp_t gfp_mask,
+ int (*remap)(void *tlb, unsigned long nslabs));
+void __init swiotlb_init_remap(bool addressing_limit, unsigned int flags,
+ int (*remap)(void *tlb, unsigned long nslabs));
extern void __init swiotlb_update_mem_attributes(void);
phys_addr_t swiotlb_tbl_map_single(struct device *hwdev, phys_addr_t phys,
diff --git a/include/xen/arm/page.h b/include/xen/arm/page.h
index ac1b654705631..7e199c6656b90 100644
--- a/include/xen/arm/page.h
+++ b/include/xen/arm/page.h
@@ -115,6 +115,5 @@ static inline bool set_phys_to_machine(unsigned long pfn, unsigned long mfn)
bool xen_arch_need_swiotlb(struct device *dev,
phys_addr_t phys,
dma_addr_t dev_addr);
-unsigned long xen_get_swiotlb_free_pages(unsigned int order);
#endif /* _ASM_ARM_XEN_PAGE_H */
diff --git a/include/xen/swiotlb-xen.h b/include/xen/swiotlb-xen.h
index b3e647f86e3e2..590ceb923f0c8 100644
--- a/include/xen/swiotlb-xen.h
+++ b/include/xen/swiotlb-xen.h
@@ -10,8 +10,12 @@ void xen_dma_sync_for_cpu(struct device *dev, dma_addr_t handle,
void xen_dma_sync_for_device(struct device *dev, dma_addr_t handle,
size_t size, enum dma_data_direction dir);
-int xen_swiotlb_init(void);
-void __init xen_swiotlb_init_early(void);
+#ifdef CONFIG_SWIOTLB_XEN
+int xen_swiotlb_fixup(void *buf, unsigned long nslabs);
+#else
+#define xen_swiotlb_fixup NULL
+#endif
+
extern const struct dma_map_ops xen_swiotlb_dma_ops;
#endif /* __LINUX_SWIOTLB_XEN_H */
diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
index ec200e40fc397..35ee2318ea563 100644
--- a/kernel/dma/swiotlb.c
+++ b/kernel/dma/swiotlb.c
@@ -234,40 +234,17 @@ static void swiotlb_init_io_tlb_mem(struct io_tlb_mem *mem, phys_addr_t start,
return;
}
-int __init swiotlb_init_with_tbl(char *tlb, unsigned long nslabs,
- unsigned int flags)
-{
- struct io_tlb_mem *mem = &io_tlb_default_mem;
- size_t alloc_size;
-
- if (swiotlb_force_disable)
- return 0;
-
- /* protect against double initialization */
- if (WARN_ON_ONCE(mem->nslabs))
- return -ENOMEM;
-
- alloc_size = PAGE_ALIGN(array_size(sizeof(*mem->slots), nslabs));
- mem->slots = memblock_alloc(alloc_size, PAGE_SIZE);
- if (!mem->slots)
- panic("%s: Failed to allocate %zu bytes align=0x%lx\n",
- __func__, alloc_size, PAGE_SIZE);
-
- swiotlb_init_io_tlb_mem(mem, __pa(tlb), nslabs, false);
- mem->force_bounce = flags & SWIOTLB_FORCE;
-
- if (flags & SWIOTLB_VERBOSE)
- swiotlb_print_info();
- return 0;
-}
-
/*
* Statically reserve bounce buffer space and initialize bounce buffer data
* structures for the software IO TLB used to implement the DMA API.
*/
-void __init swiotlb_init(bool addressing_limit, unsigned int flags)
+void __init swiotlb_init_remap(bool addressing_limit, unsigned int flags,
+ int (*remap)(void *tlb, unsigned long nslabs))
{
- size_t bytes = PAGE_ALIGN(default_nslabs << IO_TLB_SHIFT);
+ struct io_tlb_mem *mem = &io_tlb_default_mem;
+ unsigned long nslabs = default_nslabs;
+ size_t alloc_size = PAGE_ALIGN(array_size(sizeof(*mem->slots), nslabs));
+ size_t bytes;
void *tlb;
if (!addressing_limit && !swiotlb_force_bounce)
@@ -275,23 +252,48 @@ void __init swiotlb_init(bool addressing_limit, unsigned int flags)
if (swiotlb_force_disable)
return;
+ /* protect against double initialization */
+ if (WARN_ON_ONCE(mem->nslabs))
+ return;
+
/*
* By default allocate the bonuce buffer memory from low memory.
*/
+retry:
+ bytes = PAGE_ALIGN(default_nslabs << IO_TLB_SHIFT);
if (flags & SWIOTLB_ANY)
tlb = memblock_alloc(bytes, PAGE_SIZE);
else
tlb = memblock_alloc_low(bytes, PAGE_SIZE);
if (!tlb)
- goto fail;
- if (swiotlb_init_with_tbl(tlb, default_nslabs, flags))
- goto fail_free_mem;
- return;
+ panic("%s: failed to allocate tlb structure\n", __func__);
+
+ if (remap && remap(tlb, nslabs) < 0) {
+ memblock_free(tlb, PAGE_ALIGN(bytes));
+
+ /* Min is 2MB */
+ if (nslabs <= 1024)
+ panic("%s: Failed to remap %zu bytes\n",
+ __func__, bytes);
+ nslabs = max(1024UL, ALIGN(nslabs >> 1, IO_TLB_SEGSIZE));
+ goto retry;
+ }
+
+ mem->slots = memblock_alloc(alloc_size, PAGE_SIZE);
+ if (!mem->slots)
+ panic("%s: Failed to allocate %zu bytes align=0x%lx\n",
+ __func__, alloc_size, PAGE_SIZE);
-fail_free_mem:
- memblock_free(tlb, bytes);
-fail:
- pr_warn("Cannot allocate buffer");
+ swiotlb_init_io_tlb_mem(mem, __pa(tlb), default_nslabs, false);
+ mem->force_bounce = flags & SWIOTLB_FORCE;
+
+ if (flags & SWIOTLB_VERBOSE)
+ swiotlb_print_info();
+}
+
+void __init swiotlb_init(bool addressing_limit, unsigned int flags)
+{
+ return swiotlb_init_remap(addressing_limit, flags, NULL);
}
/*
@@ -299,8 +301,10 @@ void __init swiotlb_init(bool addressing_limit, unsigned int flags)
* initialize the swiotlb later using the slab allocator if needed.
* This should be just like above, but with some error catching.
*/
-int swiotlb_init_late(size_t size, gfp_t gfp_mask)
+int swiotlb_init_late(size_t size, gfp_t gfp_mask,
+ int (*remap)(void *tlb, unsigned long nslabs))
{
+ struct io_tlb_mem *mem = &io_tlb_default_mem;
unsigned long nslabs = ALIGN(size >> IO_TLB_SHIFT, IO_TLB_SEGSIZE);
unsigned long bytes;
unsigned char *vstart = NULL;
@@ -310,9 +314,14 @@ int swiotlb_init_late(size_t size, gfp_t gfp_mask)
if (swiotlb_force_disable)
return 0;
+ /* protect against double initialization */
+ if (WARN_ON_ONCE(mem->nslabs))
+ return -ENOMEM;
+
/*
* Get IO TLB memory from the low pages
*/
+retry:
order = get_order(nslabs << IO_TLB_SHIFT);
nslabs = SLABS_PER_PAGE << order;
bytes = nslabs << IO_TLB_SHIFT;
@@ -333,33 +342,28 @@ int swiotlb_init_late(size_t size, gfp_t gfp_mask)
(PAGE_SIZE << order) >> 20);
nslabs = SLABS_PER_PAGE << order;
}
- rc = swiotlb_late_init_with_tbl(vstart, nslabs);
- if (rc)
- free_pages((unsigned long)vstart, order);
-
- return rc;
-}
-
-int
-swiotlb_late_init_with_tbl(char *tlb, unsigned long nslabs)
-{
- struct io_tlb_mem *mem = &io_tlb_default_mem;
- unsigned long bytes = nslabs << IO_TLB_SHIFT;
- if (swiotlb_force_disable)
- return 0;
+ if (remap)
+ rc = remap(vstart, nslabs);
+ if (rc) {
+ free_pages((unsigned long)vstart, order);
- /* protect against double initialization */
- if (WARN_ON_ONCE(mem->nslabs))
- return -ENOMEM;
+ /* Min is 2MB */
+ if (nslabs <= 1024)
+ return rc;
+ nslabs = max(1024UL, ALIGN(nslabs >> 1, IO_TLB_SEGSIZE));
+ goto retry;
+ }
mem->slots = (void *)__get_free_pages(GFP_KERNEL | __GFP_ZERO,
get_order(array_size(sizeof(*mem->slots), nslabs)));
- if (!mem->slots)
+ if (!mem->slots) {
+ free_pages((unsigned long)vstart, order);
return -ENOMEM;
+ }
- set_memory_decrypted((unsigned long)tlb, bytes >> PAGE_SHIFT);
- swiotlb_init_io_tlb_mem(mem, virt_to_phys(tlb), nslabs, true);
+ set_memory_decrypted((unsigned long)vstart, bytes >> PAGE_SHIFT);
+ swiotlb_init_io_tlb_mem(mem, virt_to_phys(vstart), nslabs, true);
swiotlb_print_info();
return 0;
--
2.30.2
^ permalink raw reply related [flat|nested] 118+ messages in thread
* [PATCH 10/11] swiotlb: merge swiotlb-xen initialization into swiotlb
@ 2022-02-27 14:30 ` Christoph Hellwig
0 siblings, 0 replies; 118+ messages in thread
From: Christoph Hellwig @ 2022-02-27 14:30 UTC (permalink / raw)
To: iommu
Cc: Juergen Gross, linux-s390, linux-hyperv, Stefano Stabellini,
linux-ia64, Anshuman Khandual, Konrad Rzeszutek Wilk,
Robin Murphy, x86, linux-mips, linuxppc-dev, tboot-devel,
linux-pci, xen-devel, Boris Ostrovsky, David Woodhouse,
linux-riscv, linux-arm-kernel
Allow to pass a remap argument to the swiotlb initialization functions
to handle the Xen/x86 remap case. ARM/ARM64 never did any remapping
from xen_swiotlb_fixup, so we don't even need that quirk.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
arch/arm/xen/mm.c | 23 +++---
arch/x86/include/asm/xen/page.h | 5 --
arch/x86/kernel/pci-dma.c | 19 +++--
arch/x86/pci/sta2x11-fixup.c | 2 +-
drivers/xen/swiotlb-xen.c | 128 +-------------------------------
include/linux/swiotlb.h | 7 +-
include/xen/arm/page.h | 1 -
include/xen/swiotlb-xen.h | 8 +-
kernel/dma/swiotlb.c | 120 +++++++++++++++---------------
9 files changed, 96 insertions(+), 217 deletions(-)
diff --git a/arch/arm/xen/mm.c b/arch/arm/xen/mm.c
index a7e54a087b802..58b40f87617d3 100644
--- a/arch/arm/xen/mm.c
+++ b/arch/arm/xen/mm.c
@@ -23,22 +23,20 @@
#include <asm/xen/hypercall.h>
#include <asm/xen/interface.h>
-unsigned long xen_get_swiotlb_free_pages(unsigned int order)
+static gfp_t xen_swiotlb_gfp(void)
{
phys_addr_t base;
- gfp_t flags = __GFP_NOWARN|__GFP_KSWAPD_RECLAIM;
u64 i;
for_each_mem_range(i, &base, NULL) {
if (base < (phys_addr_t)0xffffffff) {
if (IS_ENABLED(CONFIG_ZONE_DMA32))
- flags |= __GFP_DMA32;
- else
- flags |= __GFP_DMA;
- break;
+ return __GFP_DMA32;
+ return __GFP_DMA;
}
}
- return __get_free_pages(flags, order);
+
+ return GFP_KERNEL;
}
static bool hypercall_cflush = false;
@@ -143,10 +141,15 @@ static int __init xen_mm_init(void)
if (!xen_swiotlb_detect())
return 0;
- rc = xen_swiotlb_init();
/* we can work with the default swiotlb */
- if (rc < 0 && rc != -EEXIST)
- return rc;
+ if (!io_tlb_default_mem.nslabs) {
+ if (!xen_initial_domain())
+ return -EINVAL;
+ rc = swiotlb_init_late(swiotlb_size_or_default(),
+ xen_swiotlb_gfp(), NULL);
+ if (rc < 0)
+ return rc;
+ }
cflush.op = 0;
cflush.a.dev_bus_addr = 0;
diff --git a/arch/x86/include/asm/xen/page.h b/arch/x86/include/asm/xen/page.h
index e989bc2269f54..1fc67df500145 100644
--- a/arch/x86/include/asm/xen/page.h
+++ b/arch/x86/include/asm/xen/page.h
@@ -357,9 +357,4 @@ static inline bool xen_arch_need_swiotlb(struct device *dev,
return false;
}
-static inline unsigned long xen_get_swiotlb_free_pages(unsigned int order)
-{
- return __get_free_pages(__GFP_NOWARN, order);
-}
-
#endif /* _ASM_X86_XEN_PAGE_H */
diff --git a/arch/x86/kernel/pci-dma.c b/arch/x86/kernel/pci-dma.c
index bb08184a50e3a..b0901f027cdcd 100644
--- a/arch/x86/kernel/pci-dma.c
+++ b/arch/x86/kernel/pci-dma.c
@@ -68,15 +68,12 @@ static inline void __init pci_swiotlb_detect_4gb(void)
#endif /* CONFIG_SWIOTLB */
#ifdef CONFIG_SWIOTLB_XEN
-static bool xen_swiotlb;
-
static void __init pci_xen_swiotlb_init(void)
{
if (!xen_initial_domain() && !x86_swiotlb_enable)
return;
x86_swiotlb_enable = false;
- xen_swiotlb = true;
- xen_swiotlb_init_early();
+ swiotlb_init_remap(true, x86_swiotlb_flags, xen_swiotlb_fixup);
dma_ops = &xen_swiotlb_dma_ops;
if (IS_ENABLED(CONFIG_PCI))
pci_request_acs();
@@ -84,14 +81,16 @@ static void __init pci_xen_swiotlb_init(void)
int pci_xen_swiotlb_init_late(void)
{
- int rc;
-
- if (xen_swiotlb)
+ if (dma_ops == &xen_swiotlb_dma_ops)
return 0;
- rc = xen_swiotlb_init();
- if (rc)
- return rc;
+ /* we can work with the default swiotlb */
+ if (!io_tlb_default_mem.nslabs) {
+ int rc = swiotlb_init_late(swiotlb_size_or_default(),
+ GFP_KERNEL, xen_swiotlb_fixup);
+ if (rc < 0)
+ return rc;
+ }
/* XXX: this switches the dma ops under live devices! */
dma_ops = &xen_swiotlb_dma_ops;
diff --git a/arch/x86/pci/sta2x11-fixup.c b/arch/x86/pci/sta2x11-fixup.c
index c7e6faf59a861..7368afc039987 100644
--- a/arch/x86/pci/sta2x11-fixup.c
+++ b/arch/x86/pci/sta2x11-fixup.c
@@ -57,7 +57,7 @@ static void sta2x11_new_instance(struct pci_dev *pdev)
int size = STA2X11_SWIOTLB_SIZE;
/* First instance: register your own swiotlb area */
dev_info(&pdev->dev, "Using SWIOTLB (size %i)\n", size);
- if (swiotlb_init_late(size, GFP_DMA))
+ if (swiotlb_init_late(size, GFP_DMA, NULL))
dev_emerg(&pdev->dev, "init swiotlb failed\n");
}
list_add(&instance->list, &sta2x11_instance_list);
diff --git a/drivers/xen/swiotlb-xen.c b/drivers/xen/swiotlb-xen.c
index c2da3eb4826e8..df8085b50df10 100644
--- a/drivers/xen/swiotlb-xen.c
+++ b/drivers/xen/swiotlb-xen.c
@@ -104,7 +104,7 @@ static int is_xen_swiotlb_buffer(struct device *dev, dma_addr_t dma_addr)
return 0;
}
-static int xen_swiotlb_fixup(void *buf, unsigned long nslabs)
+int xen_swiotlb_fixup(void *buf, unsigned long nslabs)
{
int rc;
unsigned int order = get_order(IO_TLB_SEGSIZE << IO_TLB_SHIFT);
@@ -130,132 +130,6 @@ static int xen_swiotlb_fixup(void *buf, unsigned long nslabs)
return 0;
}
-enum xen_swiotlb_err {
- XEN_SWIOTLB_UNKNOWN = 0,
- XEN_SWIOTLB_ENOMEM,
- XEN_SWIOTLB_EFIXUP
-};
-
-static const char *xen_swiotlb_error(enum xen_swiotlb_err err)
-{
- switch (err) {
- case XEN_SWIOTLB_ENOMEM:
- return "Cannot allocate Xen-SWIOTLB buffer\n";
- case XEN_SWIOTLB_EFIXUP:
- return "Failed to get contiguous memory for DMA from Xen!\n"\
- "You either: don't have the permissions, do not have"\
- " enough free memory under 4GB, or the hypervisor memory"\
- " is too fragmented!";
- default:
- break;
- }
- return "";
-}
-
-int xen_swiotlb_init(void)
-{
- enum xen_swiotlb_err m_ret = XEN_SWIOTLB_UNKNOWN;
- unsigned long bytes = swiotlb_size_or_default();
- unsigned long nslabs = bytes >> IO_TLB_SHIFT;
- unsigned int order, repeat = 3;
- int rc = -ENOMEM;
- char *start;
-
- if (io_tlb_default_mem.nslabs) {
- pr_warn("swiotlb buffer already initialized\n");
- return -EEXIST;
- }
-
-retry:
- m_ret = XEN_SWIOTLB_ENOMEM;
- order = get_order(bytes);
-
- /*
- * Get IO TLB memory from any location.
- */
-#define SLABS_PER_PAGE (1 << (PAGE_SHIFT - IO_TLB_SHIFT))
-#define IO_TLB_MIN_SLABS ((1<<20) >> IO_TLB_SHIFT)
- while ((SLABS_PER_PAGE << order) > IO_TLB_MIN_SLABS) {
- start = (void *)xen_get_swiotlb_free_pages(order);
- if (start)
- break;
- order--;
- }
- if (!start)
- goto exit;
- if (order != get_order(bytes)) {
- pr_warn("Warning: only able to allocate %ld MB for software IO TLB\n",
- (PAGE_SIZE << order) >> 20);
- nslabs = SLABS_PER_PAGE << order;
- bytes = nslabs << IO_TLB_SHIFT;
- }
-
- /*
- * And replace that memory with pages under 4GB.
- */
- rc = xen_swiotlb_fixup(start, nslabs);
- if (rc) {
- free_pages((unsigned long)start, order);
- m_ret = XEN_SWIOTLB_EFIXUP;
- goto error;
- }
- rc = swiotlb_late_init_with_tbl(start, nslabs);
- if (rc)
- return rc;
- return 0;
-error:
- if (nslabs > 1024 && repeat--) {
- /* Min is 2MB */
- nslabs = max(1024UL, ALIGN(nslabs >> 1, IO_TLB_SEGSIZE));
- bytes = nslabs << IO_TLB_SHIFT;
- pr_info("Lowering to %luMB\n", bytes >> 20);
- goto retry;
- }
-exit:
- pr_err("%s (rc:%d)\n", xen_swiotlb_error(m_ret), rc);
- return rc;
-}
-
-#ifdef CONFIG_X86
-void __init xen_swiotlb_init_early(void)
-{
- unsigned long bytes = swiotlb_size_or_default();
- unsigned long nslabs = bytes >> IO_TLB_SHIFT;
- unsigned int repeat = 3;
- char *start;
- int rc;
-
-retry:
- /*
- * Get IO TLB memory from any location.
- */
- start = memblock_alloc(PAGE_ALIGN(bytes),
- IO_TLB_SEGSIZE << IO_TLB_SHIFT);
- if (!start)
- panic("%s: Failed to allocate %lu bytes\n",
- __func__, PAGE_ALIGN(bytes));
-
- /*
- * And replace that memory with pages under 4GB.
- */
- rc = xen_swiotlb_fixup(start, nslabs);
- if (rc) {
- memblock_free(start, PAGE_ALIGN(bytes));
- if (nslabs > 1024 && repeat--) {
- /* Min is 2MB */
- nslabs = max(1024UL, ALIGN(nslabs >> 1, IO_TLB_SEGSIZE));
- bytes = nslabs << IO_TLB_SHIFT;
- pr_info("Lowering to %luMB\n", bytes >> 20);
- goto retry;
- }
- panic("%s (rc:%d)", xen_swiotlb_error(XEN_SWIOTLB_EFIXUP), rc);
- }
-
- if (swiotlb_init_with_tbl(start, nslabs, SWIOTLB_VERBOSE))
- panic("Cannot allocate SWIOTLB buffer");
-}
-#endif /* CONFIG_X86 */
-
static void *
xen_swiotlb_alloc_coherent(struct device *hwdev, size_t size,
dma_addr_t *dma_handle, gfp_t flags,
diff --git a/include/linux/swiotlb.h b/include/linux/swiotlb.h
index ee655f2e4d28b..919cf82ed978e 100644
--- a/include/linux/swiotlb.h
+++ b/include/linux/swiotlb.h
@@ -34,10 +34,11 @@ struct scatterlist;
/* default to 64MB */
#define IO_TLB_DEFAULT_SIZE (64UL<<20)
-int swiotlb_init_with_tbl(char *tlb, unsigned long nslabs, unsigned int flags);
unsigned long swiotlb_size_or_default(void);
-extern int swiotlb_late_init_with_tbl(char *tlb, unsigned long nslabs);
-int swiotlb_init_late(size_t size, gfp_t gfp_mask);
+int swiotlb_init_late(size_t size, gfp_t gfp_mask,
+ int (*remap)(void *tlb, unsigned long nslabs));
+void __init swiotlb_init_remap(bool addressing_limit, unsigned int flags,
+ int (*remap)(void *tlb, unsigned long nslabs));
extern void __init swiotlb_update_mem_attributes(void);
phys_addr_t swiotlb_tbl_map_single(struct device *hwdev, phys_addr_t phys,
diff --git a/include/xen/arm/page.h b/include/xen/arm/page.h
index ac1b654705631..7e199c6656b90 100644
--- a/include/xen/arm/page.h
+++ b/include/xen/arm/page.h
@@ -115,6 +115,5 @@ static inline bool set_phys_to_machine(unsigned long pfn, unsigned long mfn)
bool xen_arch_need_swiotlb(struct device *dev,
phys_addr_t phys,
dma_addr_t dev_addr);
-unsigned long xen_get_swiotlb_free_pages(unsigned int order);
#endif /* _ASM_ARM_XEN_PAGE_H */
diff --git a/include/xen/swiotlb-xen.h b/include/xen/swiotlb-xen.h
index b3e647f86e3e2..590ceb923f0c8 100644
--- a/include/xen/swiotlb-xen.h
+++ b/include/xen/swiotlb-xen.h
@@ -10,8 +10,12 @@ void xen_dma_sync_for_cpu(struct device *dev, dma_addr_t handle,
void xen_dma_sync_for_device(struct device *dev, dma_addr_t handle,
size_t size, enum dma_data_direction dir);
-int xen_swiotlb_init(void);
-void __init xen_swiotlb_init_early(void);
+#ifdef CONFIG_SWIOTLB_XEN
+int xen_swiotlb_fixup(void *buf, unsigned long nslabs);
+#else
+#define xen_swiotlb_fixup NULL
+#endif
+
extern const struct dma_map_ops xen_swiotlb_dma_ops;
#endif /* __LINUX_SWIOTLB_XEN_H */
diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
index ec200e40fc397..35ee2318ea563 100644
--- a/kernel/dma/swiotlb.c
+++ b/kernel/dma/swiotlb.c
@@ -234,40 +234,17 @@ static void swiotlb_init_io_tlb_mem(struct io_tlb_mem *mem, phys_addr_t start,
return;
}
-int __init swiotlb_init_with_tbl(char *tlb, unsigned long nslabs,
- unsigned int flags)
-{
- struct io_tlb_mem *mem = &io_tlb_default_mem;
- size_t alloc_size;
-
- if (swiotlb_force_disable)
- return 0;
-
- /* protect against double initialization */
- if (WARN_ON_ONCE(mem->nslabs))
- return -ENOMEM;
-
- alloc_size = PAGE_ALIGN(array_size(sizeof(*mem->slots), nslabs));
- mem->slots = memblock_alloc(alloc_size, PAGE_SIZE);
- if (!mem->slots)
- panic("%s: Failed to allocate %zu bytes align=0x%lx\n",
- __func__, alloc_size, PAGE_SIZE);
-
- swiotlb_init_io_tlb_mem(mem, __pa(tlb), nslabs, false);
- mem->force_bounce = flags & SWIOTLB_FORCE;
-
- if (flags & SWIOTLB_VERBOSE)
- swiotlb_print_info();
- return 0;
-}
-
/*
* Statically reserve bounce buffer space and initialize bounce buffer data
* structures for the software IO TLB used to implement the DMA API.
*/
-void __init swiotlb_init(bool addressing_limit, unsigned int flags)
+void __init swiotlb_init_remap(bool addressing_limit, unsigned int flags,
+ int (*remap)(void *tlb, unsigned long nslabs))
{
- size_t bytes = PAGE_ALIGN(default_nslabs << IO_TLB_SHIFT);
+ struct io_tlb_mem *mem = &io_tlb_default_mem;
+ unsigned long nslabs = default_nslabs;
+ size_t alloc_size = PAGE_ALIGN(array_size(sizeof(*mem->slots), nslabs));
+ size_t bytes;
void *tlb;
if (!addressing_limit && !swiotlb_force_bounce)
@@ -275,23 +252,48 @@ void __init swiotlb_init(bool addressing_limit, unsigned int flags)
if (swiotlb_force_disable)
return;
+ /* protect against double initialization */
+ if (WARN_ON_ONCE(mem->nslabs))
+ return;
+
/*
* By default allocate the bonuce buffer memory from low memory.
*/
+retry:
+ bytes = PAGE_ALIGN(default_nslabs << IO_TLB_SHIFT);
if (flags & SWIOTLB_ANY)
tlb = memblock_alloc(bytes, PAGE_SIZE);
else
tlb = memblock_alloc_low(bytes, PAGE_SIZE);
if (!tlb)
- goto fail;
- if (swiotlb_init_with_tbl(tlb, default_nslabs, flags))
- goto fail_free_mem;
- return;
+ panic("%s: failed to allocate tlb structure\n", __func__);
+
+ if (remap && remap(tlb, nslabs) < 0) {
+ memblock_free(tlb, PAGE_ALIGN(bytes));
+
+ /* Min is 2MB */
+ if (nslabs <= 1024)
+ panic("%s: Failed to remap %zu bytes\n",
+ __func__, bytes);
+ nslabs = max(1024UL, ALIGN(nslabs >> 1, IO_TLB_SEGSIZE));
+ goto retry;
+ }
+
+ mem->slots = memblock_alloc(alloc_size, PAGE_SIZE);
+ if (!mem->slots)
+ panic("%s: Failed to allocate %zu bytes align=0x%lx\n",
+ __func__, alloc_size, PAGE_SIZE);
-fail_free_mem:
- memblock_free(tlb, bytes);
-fail:
- pr_warn("Cannot allocate buffer");
+ swiotlb_init_io_tlb_mem(mem, __pa(tlb), default_nslabs, false);
+ mem->force_bounce = flags & SWIOTLB_FORCE;
+
+ if (flags & SWIOTLB_VERBOSE)
+ swiotlb_print_info();
+}
+
+void __init swiotlb_init(bool addressing_limit, unsigned int flags)
+{
+ return swiotlb_init_remap(addressing_limit, flags, NULL);
}
/*
@@ -299,8 +301,10 @@ void __init swiotlb_init(bool addressing_limit, unsigned int flags)
* initialize the swiotlb later using the slab allocator if needed.
* This should be just like above, but with some error catching.
*/
-int swiotlb_init_late(size_t size, gfp_t gfp_mask)
+int swiotlb_init_late(size_t size, gfp_t gfp_mask,
+ int (*remap)(void *tlb, unsigned long nslabs))
{
+ struct io_tlb_mem *mem = &io_tlb_default_mem;
unsigned long nslabs = ALIGN(size >> IO_TLB_SHIFT, IO_TLB_SEGSIZE);
unsigned long bytes;
unsigned char *vstart = NULL;
@@ -310,9 +314,14 @@ int swiotlb_init_late(size_t size, gfp_t gfp_mask)
if (swiotlb_force_disable)
return 0;
+ /* protect against double initialization */
+ if (WARN_ON_ONCE(mem->nslabs))
+ return -ENOMEM;
+
/*
* Get IO TLB memory from the low pages
*/
+retry:
order = get_order(nslabs << IO_TLB_SHIFT);
nslabs = SLABS_PER_PAGE << order;
bytes = nslabs << IO_TLB_SHIFT;
@@ -333,33 +342,28 @@ int swiotlb_init_late(size_t size, gfp_t gfp_mask)
(PAGE_SIZE << order) >> 20);
nslabs = SLABS_PER_PAGE << order;
}
- rc = swiotlb_late_init_with_tbl(vstart, nslabs);
- if (rc)
- free_pages((unsigned long)vstart, order);
-
- return rc;
-}
-
-int
-swiotlb_late_init_with_tbl(char *tlb, unsigned long nslabs)
-{
- struct io_tlb_mem *mem = &io_tlb_default_mem;
- unsigned long bytes = nslabs << IO_TLB_SHIFT;
- if (swiotlb_force_disable)
- return 0;
+ if (remap)
+ rc = remap(vstart, nslabs);
+ if (rc) {
+ free_pages((unsigned long)vstart, order);
- /* protect against double initialization */
- if (WARN_ON_ONCE(mem->nslabs))
- return -ENOMEM;
+ /* Min is 2MB */
+ if (nslabs <= 1024)
+ return rc;
+ nslabs = max(1024UL, ALIGN(nslabs >> 1, IO_TLB_SEGSIZE));
+ goto retry;
+ }
mem->slots = (void *)__get_free_pages(GFP_KERNEL | __GFP_ZERO,
get_order(array_size(sizeof(*mem->slots), nslabs)));
- if (!mem->slots)
+ if (!mem->slots) {
+ free_pages((unsigned long)vstart, order);
return -ENOMEM;
+ }
- set_memory_decrypted((unsigned long)tlb, bytes >> PAGE_SHIFT);
- swiotlb_init_io_tlb_mem(mem, virt_to_phys(tlb), nslabs, true);
+ set_memory_decrypted((unsigned long)vstart, bytes >> PAGE_SHIFT);
+ swiotlb_init_io_tlb_mem(mem, virt_to_phys(vstart), nslabs, true);
swiotlb_print_info();
return 0;
--
2.30.2
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply related [flat|nested] 118+ messages in thread
* [PATCH 10/11] swiotlb: merge swiotlb-xen initialization into swiotlb
@ 2022-02-27 14:30 ` Christoph Hellwig
0 siblings, 0 replies; 118+ messages in thread
From: Christoph Hellwig @ 2022-02-27 14:30 UTC (permalink / raw)
To: iommu
Cc: x86, Anshuman Khandual, Konrad Rzeszutek Wilk,
Stefano Stabellini, Boris Ostrovsky, Juergen Gross, Joerg Roedel,
David Woodhouse, Lu Baolu, Robin Murphy, linux-arm-kernel,
xen-devel, linux-ia64, linux-mips, linuxppc-dev, linux-riscv,
linux-s390, linux-hyperv, tboot-devel, linux-pci
Allow to pass a remap argument to the swiotlb initialization functions
to handle the Xen/x86 remap case. ARM/ARM64 never did any remapping
from xen_swiotlb_fixup, so we don't even need that quirk.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
arch/arm/xen/mm.c | 23 +++---
arch/x86/include/asm/xen/page.h | 5 --
arch/x86/kernel/pci-dma.c | 19 +++--
arch/x86/pci/sta2x11-fixup.c | 2 +-
drivers/xen/swiotlb-xen.c | 128 +-------------------------------
include/linux/swiotlb.h | 7 +-
include/xen/arm/page.h | 1 -
include/xen/swiotlb-xen.h | 8 +-
kernel/dma/swiotlb.c | 120 +++++++++++++++---------------
9 files changed, 96 insertions(+), 217 deletions(-)
diff --git a/arch/arm/xen/mm.c b/arch/arm/xen/mm.c
index a7e54a087b802..58b40f87617d3 100644
--- a/arch/arm/xen/mm.c
+++ b/arch/arm/xen/mm.c
@@ -23,22 +23,20 @@
#include <asm/xen/hypercall.h>
#include <asm/xen/interface.h>
-unsigned long xen_get_swiotlb_free_pages(unsigned int order)
+static gfp_t xen_swiotlb_gfp(void)
{
phys_addr_t base;
- gfp_t flags = __GFP_NOWARN|__GFP_KSWAPD_RECLAIM;
u64 i;
for_each_mem_range(i, &base, NULL) {
if (base < (phys_addr_t)0xffffffff) {
if (IS_ENABLED(CONFIG_ZONE_DMA32))
- flags |= __GFP_DMA32;
- else
- flags |= __GFP_DMA;
- break;
+ return __GFP_DMA32;
+ return __GFP_DMA;
}
}
- return __get_free_pages(flags, order);
+
+ return GFP_KERNEL;
}
static bool hypercall_cflush = false;
@@ -143,10 +141,15 @@ static int __init xen_mm_init(void)
if (!xen_swiotlb_detect())
return 0;
- rc = xen_swiotlb_init();
/* we can work with the default swiotlb */
- if (rc < 0 && rc != -EEXIST)
- return rc;
+ if (!io_tlb_default_mem.nslabs) {
+ if (!xen_initial_domain())
+ return -EINVAL;
+ rc = swiotlb_init_late(swiotlb_size_or_default(),
+ xen_swiotlb_gfp(), NULL);
+ if (rc < 0)
+ return rc;
+ }
cflush.op = 0;
cflush.a.dev_bus_addr = 0;
diff --git a/arch/x86/include/asm/xen/page.h b/arch/x86/include/asm/xen/page.h
index e989bc2269f54..1fc67df500145 100644
--- a/arch/x86/include/asm/xen/page.h
+++ b/arch/x86/include/asm/xen/page.h
@@ -357,9 +357,4 @@ static inline bool xen_arch_need_swiotlb(struct device *dev,
return false;
}
-static inline unsigned long xen_get_swiotlb_free_pages(unsigned int order)
-{
- return __get_free_pages(__GFP_NOWARN, order);
-}
-
#endif /* _ASM_X86_XEN_PAGE_H */
diff --git a/arch/x86/kernel/pci-dma.c b/arch/x86/kernel/pci-dma.c
index bb08184a50e3a..b0901f027cdcd 100644
--- a/arch/x86/kernel/pci-dma.c
+++ b/arch/x86/kernel/pci-dma.c
@@ -68,15 +68,12 @@ static inline void __init pci_swiotlb_detect_4gb(void)
#endif /* CONFIG_SWIOTLB */
#ifdef CONFIG_SWIOTLB_XEN
-static bool xen_swiotlb;
-
static void __init pci_xen_swiotlb_init(void)
{
if (!xen_initial_domain() && !x86_swiotlb_enable)
return;
x86_swiotlb_enable = false;
- xen_swiotlb = true;
- xen_swiotlb_init_early();
+ swiotlb_init_remap(true, x86_swiotlb_flags, xen_swiotlb_fixup);
dma_ops = &xen_swiotlb_dma_ops;
if (IS_ENABLED(CONFIG_PCI))
pci_request_acs();
@@ -84,14 +81,16 @@ static void __init pci_xen_swiotlb_init(void)
int pci_xen_swiotlb_init_late(void)
{
- int rc;
-
- if (xen_swiotlb)
+ if (dma_ops == &xen_swiotlb_dma_ops)
return 0;
- rc = xen_swiotlb_init();
- if (rc)
- return rc;
+ /* we can work with the default swiotlb */
+ if (!io_tlb_default_mem.nslabs) {
+ int rc = swiotlb_init_late(swiotlb_size_or_default(),
+ GFP_KERNEL, xen_swiotlb_fixup);
+ if (rc < 0)
+ return rc;
+ }
/* XXX: this switches the dma ops under live devices! */
dma_ops = &xen_swiotlb_dma_ops;
diff --git a/arch/x86/pci/sta2x11-fixup.c b/arch/x86/pci/sta2x11-fixup.c
index c7e6faf59a861..7368afc039987 100644
--- a/arch/x86/pci/sta2x11-fixup.c
+++ b/arch/x86/pci/sta2x11-fixup.c
@@ -57,7 +57,7 @@ static void sta2x11_new_instance(struct pci_dev *pdev)
int size = STA2X11_SWIOTLB_SIZE;
/* First instance: register your own swiotlb area */
dev_info(&pdev->dev, "Using SWIOTLB (size %i)\n", size);
- if (swiotlb_init_late(size, GFP_DMA))
+ if (swiotlb_init_late(size, GFP_DMA, NULL))
dev_emerg(&pdev->dev, "init swiotlb failed\n");
}
list_add(&instance->list, &sta2x11_instance_list);
diff --git a/drivers/xen/swiotlb-xen.c b/drivers/xen/swiotlb-xen.c
index c2da3eb4826e8..df8085b50df10 100644
--- a/drivers/xen/swiotlb-xen.c
+++ b/drivers/xen/swiotlb-xen.c
@@ -104,7 +104,7 @@ static int is_xen_swiotlb_buffer(struct device *dev, dma_addr_t dma_addr)
return 0;
}
-static int xen_swiotlb_fixup(void *buf, unsigned long nslabs)
+int xen_swiotlb_fixup(void *buf, unsigned long nslabs)
{
int rc;
unsigned int order = get_order(IO_TLB_SEGSIZE << IO_TLB_SHIFT);
@@ -130,132 +130,6 @@ static int xen_swiotlb_fixup(void *buf, unsigned long nslabs)
return 0;
}
-enum xen_swiotlb_err {
- XEN_SWIOTLB_UNKNOWN = 0,
- XEN_SWIOTLB_ENOMEM,
- XEN_SWIOTLB_EFIXUP
-};
-
-static const char *xen_swiotlb_error(enum xen_swiotlb_err err)
-{
- switch (err) {
- case XEN_SWIOTLB_ENOMEM:
- return "Cannot allocate Xen-SWIOTLB buffer\n";
- case XEN_SWIOTLB_EFIXUP:
- return "Failed to get contiguous memory for DMA from Xen!\n"\
- "You either: don't have the permissions, do not have"\
- " enough free memory under 4GB, or the hypervisor memory"\
- " is too fragmented!";
- default:
- break;
- }
- return "";
-}
-
-int xen_swiotlb_init(void)
-{
- enum xen_swiotlb_err m_ret = XEN_SWIOTLB_UNKNOWN;
- unsigned long bytes = swiotlb_size_or_default();
- unsigned long nslabs = bytes >> IO_TLB_SHIFT;
- unsigned int order, repeat = 3;
- int rc = -ENOMEM;
- char *start;
-
- if (io_tlb_default_mem.nslabs) {
- pr_warn("swiotlb buffer already initialized\n");
- return -EEXIST;
- }
-
-retry:
- m_ret = XEN_SWIOTLB_ENOMEM;
- order = get_order(bytes);
-
- /*
- * Get IO TLB memory from any location.
- */
-#define SLABS_PER_PAGE (1 << (PAGE_SHIFT - IO_TLB_SHIFT))
-#define IO_TLB_MIN_SLABS ((1<<20) >> IO_TLB_SHIFT)
- while ((SLABS_PER_PAGE << order) > IO_TLB_MIN_SLABS) {
- start = (void *)xen_get_swiotlb_free_pages(order);
- if (start)
- break;
- order--;
- }
- if (!start)
- goto exit;
- if (order != get_order(bytes)) {
- pr_warn("Warning: only able to allocate %ld MB for software IO TLB\n",
- (PAGE_SIZE << order) >> 20);
- nslabs = SLABS_PER_PAGE << order;
- bytes = nslabs << IO_TLB_SHIFT;
- }
-
- /*
- * And replace that memory with pages under 4GB.
- */
- rc = xen_swiotlb_fixup(start, nslabs);
- if (rc) {
- free_pages((unsigned long)start, order);
- m_ret = XEN_SWIOTLB_EFIXUP;
- goto error;
- }
- rc = swiotlb_late_init_with_tbl(start, nslabs);
- if (rc)
- return rc;
- return 0;
-error:
- if (nslabs > 1024 && repeat--) {
- /* Min is 2MB */
- nslabs = max(1024UL, ALIGN(nslabs >> 1, IO_TLB_SEGSIZE));
- bytes = nslabs << IO_TLB_SHIFT;
- pr_info("Lowering to %luMB\n", bytes >> 20);
- goto retry;
- }
-exit:
- pr_err("%s (rc:%d)\n", xen_swiotlb_error(m_ret), rc);
- return rc;
-}
-
-#ifdef CONFIG_X86
-void __init xen_swiotlb_init_early(void)
-{
- unsigned long bytes = swiotlb_size_or_default();
- unsigned long nslabs = bytes >> IO_TLB_SHIFT;
- unsigned int repeat = 3;
- char *start;
- int rc;
-
-retry:
- /*
- * Get IO TLB memory from any location.
- */
- start = memblock_alloc(PAGE_ALIGN(bytes),
- IO_TLB_SEGSIZE << IO_TLB_SHIFT);
- if (!start)
- panic("%s: Failed to allocate %lu bytes\n",
- __func__, PAGE_ALIGN(bytes));
-
- /*
- * And replace that memory with pages under 4GB.
- */
- rc = xen_swiotlb_fixup(start, nslabs);
- if (rc) {
- memblock_free(start, PAGE_ALIGN(bytes));
- if (nslabs > 1024 && repeat--) {
- /* Min is 2MB */
- nslabs = max(1024UL, ALIGN(nslabs >> 1, IO_TLB_SEGSIZE));
- bytes = nslabs << IO_TLB_SHIFT;
- pr_info("Lowering to %luMB\n", bytes >> 20);
- goto retry;
- }
- panic("%s (rc:%d)", xen_swiotlb_error(XEN_SWIOTLB_EFIXUP), rc);
- }
-
- if (swiotlb_init_with_tbl(start, nslabs, SWIOTLB_VERBOSE))
- panic("Cannot allocate SWIOTLB buffer");
-}
-#endif /* CONFIG_X86 */
-
static void *
xen_swiotlb_alloc_coherent(struct device *hwdev, size_t size,
dma_addr_t *dma_handle, gfp_t flags,
diff --git a/include/linux/swiotlb.h b/include/linux/swiotlb.h
index ee655f2e4d28b..919cf82ed978e 100644
--- a/include/linux/swiotlb.h
+++ b/include/linux/swiotlb.h
@@ -34,10 +34,11 @@ struct scatterlist;
/* default to 64MB */
#define IO_TLB_DEFAULT_SIZE (64UL<<20)
-int swiotlb_init_with_tbl(char *tlb, unsigned long nslabs, unsigned int flags);
unsigned long swiotlb_size_or_default(void);
-extern int swiotlb_late_init_with_tbl(char *tlb, unsigned long nslabs);
-int swiotlb_init_late(size_t size, gfp_t gfp_mask);
+int swiotlb_init_late(size_t size, gfp_t gfp_mask,
+ int (*remap)(void *tlb, unsigned long nslabs));
+void __init swiotlb_init_remap(bool addressing_limit, unsigned int flags,
+ int (*remap)(void *tlb, unsigned long nslabs));
extern void __init swiotlb_update_mem_attributes(void);
phys_addr_t swiotlb_tbl_map_single(struct device *hwdev, phys_addr_t phys,
diff --git a/include/xen/arm/page.h b/include/xen/arm/page.h
index ac1b654705631..7e199c6656b90 100644
--- a/include/xen/arm/page.h
+++ b/include/xen/arm/page.h
@@ -115,6 +115,5 @@ static inline bool set_phys_to_machine(unsigned long pfn, unsigned long mfn)
bool xen_arch_need_swiotlb(struct device *dev,
phys_addr_t phys,
dma_addr_t dev_addr);
-unsigned long xen_get_swiotlb_free_pages(unsigned int order);
#endif /* _ASM_ARM_XEN_PAGE_H */
diff --git a/include/xen/swiotlb-xen.h b/include/xen/swiotlb-xen.h
index b3e647f86e3e2..590ceb923f0c8 100644
--- a/include/xen/swiotlb-xen.h
+++ b/include/xen/swiotlb-xen.h
@@ -10,8 +10,12 @@ void xen_dma_sync_for_cpu(struct device *dev, dma_addr_t handle,
void xen_dma_sync_for_device(struct device *dev, dma_addr_t handle,
size_t size, enum dma_data_direction dir);
-int xen_swiotlb_init(void);
-void __init xen_swiotlb_init_early(void);
+#ifdef CONFIG_SWIOTLB_XEN
+int xen_swiotlb_fixup(void *buf, unsigned long nslabs);
+#else
+#define xen_swiotlb_fixup NULL
+#endif
+
extern const struct dma_map_ops xen_swiotlb_dma_ops;
#endif /* __LINUX_SWIOTLB_XEN_H */
diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
index ec200e40fc397..35ee2318ea563 100644
--- a/kernel/dma/swiotlb.c
+++ b/kernel/dma/swiotlb.c
@@ -234,40 +234,17 @@ static void swiotlb_init_io_tlb_mem(struct io_tlb_mem *mem, phys_addr_t start,
return;
}
-int __init swiotlb_init_with_tbl(char *tlb, unsigned long nslabs,
- unsigned int flags)
-{
- struct io_tlb_mem *mem = &io_tlb_default_mem;
- size_t alloc_size;
-
- if (swiotlb_force_disable)
- return 0;
-
- /* protect against double initialization */
- if (WARN_ON_ONCE(mem->nslabs))
- return -ENOMEM;
-
- alloc_size = PAGE_ALIGN(array_size(sizeof(*mem->slots), nslabs));
- mem->slots = memblock_alloc(alloc_size, PAGE_SIZE);
- if (!mem->slots)
- panic("%s: Failed to allocate %zu bytes align=0x%lx\n",
- __func__, alloc_size, PAGE_SIZE);
-
- swiotlb_init_io_tlb_mem(mem, __pa(tlb), nslabs, false);
- mem->force_bounce = flags & SWIOTLB_FORCE;
-
- if (flags & SWIOTLB_VERBOSE)
- swiotlb_print_info();
- return 0;
-}
-
/*
* Statically reserve bounce buffer space and initialize bounce buffer data
* structures for the software IO TLB used to implement the DMA API.
*/
-void __init swiotlb_init(bool addressing_limit, unsigned int flags)
+void __init swiotlb_init_remap(bool addressing_limit, unsigned int flags,
+ int (*remap)(void *tlb, unsigned long nslabs))
{
- size_t bytes = PAGE_ALIGN(default_nslabs << IO_TLB_SHIFT);
+ struct io_tlb_mem *mem = &io_tlb_default_mem;
+ unsigned long nslabs = default_nslabs;
+ size_t alloc_size = PAGE_ALIGN(array_size(sizeof(*mem->slots), nslabs));
+ size_t bytes;
void *tlb;
if (!addressing_limit && !swiotlb_force_bounce)
@@ -275,23 +252,48 @@ void __init swiotlb_init(bool addressing_limit, unsigned int flags)
if (swiotlb_force_disable)
return;
+ /* protect against double initialization */
+ if (WARN_ON_ONCE(mem->nslabs))
+ return;
+
/*
* By default allocate the bonuce buffer memory from low memory.
*/
+retry:
+ bytes = PAGE_ALIGN(default_nslabs << IO_TLB_SHIFT);
if (flags & SWIOTLB_ANY)
tlb = memblock_alloc(bytes, PAGE_SIZE);
else
tlb = memblock_alloc_low(bytes, PAGE_SIZE);
if (!tlb)
- goto fail;
- if (swiotlb_init_with_tbl(tlb, default_nslabs, flags))
- goto fail_free_mem;
- return;
+ panic("%s: failed to allocate tlb structure\n", __func__);
+
+ if (remap && remap(tlb, nslabs) < 0) {
+ memblock_free(tlb, PAGE_ALIGN(bytes));
+
+ /* Min is 2MB */
+ if (nslabs <= 1024)
+ panic("%s: Failed to remap %zu bytes\n",
+ __func__, bytes);
+ nslabs = max(1024UL, ALIGN(nslabs >> 1, IO_TLB_SEGSIZE));
+ goto retry;
+ }
+
+ mem->slots = memblock_alloc(alloc_size, PAGE_SIZE);
+ if (!mem->slots)
+ panic("%s: Failed to allocate %zu bytes align=0x%lx\n",
+ __func__, alloc_size, PAGE_SIZE);
-fail_free_mem:
- memblock_free(tlb, bytes);
-fail:
- pr_warn("Cannot allocate buffer");
+ swiotlb_init_io_tlb_mem(mem, __pa(tlb), default_nslabs, false);
+ mem->force_bounce = flags & SWIOTLB_FORCE;
+
+ if (flags & SWIOTLB_VERBOSE)
+ swiotlb_print_info();
+}
+
+void __init swiotlb_init(bool addressing_limit, unsigned int flags)
+{
+ return swiotlb_init_remap(addressing_limit, flags, NULL);
}
/*
@@ -299,8 +301,10 @@ void __init swiotlb_init(bool addressing_limit, unsigned int flags)
* initialize the swiotlb later using the slab allocator if needed.
* This should be just like above, but with some error catching.
*/
-int swiotlb_init_late(size_t size, gfp_t gfp_mask)
+int swiotlb_init_late(size_t size, gfp_t gfp_mask,
+ int (*remap)(void *tlb, unsigned long nslabs))
{
+ struct io_tlb_mem *mem = &io_tlb_default_mem;
unsigned long nslabs = ALIGN(size >> IO_TLB_SHIFT, IO_TLB_SEGSIZE);
unsigned long bytes;
unsigned char *vstart = NULL;
@@ -310,9 +314,14 @@ int swiotlb_init_late(size_t size, gfp_t gfp_mask)
if (swiotlb_force_disable)
return 0;
+ /* protect against double initialization */
+ if (WARN_ON_ONCE(mem->nslabs))
+ return -ENOMEM;
+
/*
* Get IO TLB memory from the low pages
*/
+retry:
order = get_order(nslabs << IO_TLB_SHIFT);
nslabs = SLABS_PER_PAGE << order;
bytes = nslabs << IO_TLB_SHIFT;
@@ -333,33 +342,28 @@ int swiotlb_init_late(size_t size, gfp_t gfp_mask)
(PAGE_SIZE << order) >> 20);
nslabs = SLABS_PER_PAGE << order;
}
- rc = swiotlb_late_init_with_tbl(vstart, nslabs);
- if (rc)
- free_pages((unsigned long)vstart, order);
-
- return rc;
-}
-
-int
-swiotlb_late_init_with_tbl(char *tlb, unsigned long nslabs)
-{
- struct io_tlb_mem *mem = &io_tlb_default_mem;
- unsigned long bytes = nslabs << IO_TLB_SHIFT;
- if (swiotlb_force_disable)
- return 0;
+ if (remap)
+ rc = remap(vstart, nslabs);
+ if (rc) {
+ free_pages((unsigned long)vstart, order);
- /* protect against double initialization */
- if (WARN_ON_ONCE(mem->nslabs))
- return -ENOMEM;
+ /* Min is 2MB */
+ if (nslabs <= 1024)
+ return rc;
+ nslabs = max(1024UL, ALIGN(nslabs >> 1, IO_TLB_SEGSIZE));
+ goto retry;
+ }
mem->slots = (void *)__get_free_pages(GFP_KERNEL | __GFP_ZERO,
get_order(array_size(sizeof(*mem->slots), nslabs)));
- if (!mem->slots)
+ if (!mem->slots) {
+ free_pages((unsigned long)vstart, order);
return -ENOMEM;
+ }
- set_memory_decrypted((unsigned long)tlb, bytes >> PAGE_SHIFT);
- swiotlb_init_io_tlb_mem(mem, virt_to_phys(tlb), nslabs, true);
+ set_memory_decrypted((unsigned long)vstart, bytes >> PAGE_SHIFT);
+ swiotlb_init_io_tlb_mem(mem, virt_to_phys(vstart), nslabs, true);
swiotlb_print_info();
return 0;
--
2.30.2
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply related [flat|nested] 118+ messages in thread
* [PATCH 10/11] swiotlb: merge swiotlb-xen initialization into swiotlb
@ 2022-02-27 14:30 ` Christoph Hellwig
0 siblings, 0 replies; 118+ messages in thread
From: Christoph Hellwig @ 2022-02-27 14:30 UTC (permalink / raw)
To: iommu
Cc: x86, Anshuman Khandual, Konrad Rzeszutek Wilk,
Stefano Stabellini, Boris Ostrovsky, Juergen Gross, Joerg Roedel,
David Woodhouse, Lu Baolu, Robin Murphy, linux-arm-kernel,
xen-devel, linux-ia64, linux-mips, linuxppc-dev, linux-riscv,
linux-s390, linux-hyperv, tboot-devel, linux-pci
Allow to pass a remap argument to the swiotlb initialization functions
to handle the Xen/x86 remap case. ARM/ARM64 never did any remapping
from xen_swiotlb_fixup, so we don't even need that quirk.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
arch/arm/xen/mm.c | 23 +++---
arch/x86/include/asm/xen/page.h | 5 --
arch/x86/kernel/pci-dma.c | 19 +++--
arch/x86/pci/sta2x11-fixup.c | 2 +-
drivers/xen/swiotlb-xen.c | 128 +-------------------------------
include/linux/swiotlb.h | 7 +-
include/xen/arm/page.h | 1 -
include/xen/swiotlb-xen.h | 8 +-
kernel/dma/swiotlb.c | 120 +++++++++++++++---------------
9 files changed, 96 insertions(+), 217 deletions(-)
diff --git a/arch/arm/xen/mm.c b/arch/arm/xen/mm.c
index a7e54a087b802..58b40f87617d3 100644
--- a/arch/arm/xen/mm.c
+++ b/arch/arm/xen/mm.c
@@ -23,22 +23,20 @@
#include <asm/xen/hypercall.h>
#include <asm/xen/interface.h>
-unsigned long xen_get_swiotlb_free_pages(unsigned int order)
+static gfp_t xen_swiotlb_gfp(void)
{
phys_addr_t base;
- gfp_t flags = __GFP_NOWARN|__GFP_KSWAPD_RECLAIM;
u64 i;
for_each_mem_range(i, &base, NULL) {
if (base < (phys_addr_t)0xffffffff) {
if (IS_ENABLED(CONFIG_ZONE_DMA32))
- flags |= __GFP_DMA32;
- else
- flags |= __GFP_DMA;
- break;
+ return __GFP_DMA32;
+ return __GFP_DMA;
}
}
- return __get_free_pages(flags, order);
+
+ return GFP_KERNEL;
}
static bool hypercall_cflush = false;
@@ -143,10 +141,15 @@ static int __init xen_mm_init(void)
if (!xen_swiotlb_detect())
return 0;
- rc = xen_swiotlb_init();
/* we can work with the default swiotlb */
- if (rc < 0 && rc != -EEXIST)
- return rc;
+ if (!io_tlb_default_mem.nslabs) {
+ if (!xen_initial_domain())
+ return -EINVAL;
+ rc = swiotlb_init_late(swiotlb_size_or_default(),
+ xen_swiotlb_gfp(), NULL);
+ if (rc < 0)
+ return rc;
+ }
cflush.op = 0;
cflush.a.dev_bus_addr = 0;
diff --git a/arch/x86/include/asm/xen/page.h b/arch/x86/include/asm/xen/page.h
index e989bc2269f54..1fc67df500145 100644
--- a/arch/x86/include/asm/xen/page.h
+++ b/arch/x86/include/asm/xen/page.h
@@ -357,9 +357,4 @@ static inline bool xen_arch_need_swiotlb(struct device *dev,
return false;
}
-static inline unsigned long xen_get_swiotlb_free_pages(unsigned int order)
-{
- return __get_free_pages(__GFP_NOWARN, order);
-}
-
#endif /* _ASM_X86_XEN_PAGE_H */
diff --git a/arch/x86/kernel/pci-dma.c b/arch/x86/kernel/pci-dma.c
index bb08184a50e3a..b0901f027cdcd 100644
--- a/arch/x86/kernel/pci-dma.c
+++ b/arch/x86/kernel/pci-dma.c
@@ -68,15 +68,12 @@ static inline void __init pci_swiotlb_detect_4gb(void)
#endif /* CONFIG_SWIOTLB */
#ifdef CONFIG_SWIOTLB_XEN
-static bool xen_swiotlb;
-
static void __init pci_xen_swiotlb_init(void)
{
if (!xen_initial_domain() && !x86_swiotlb_enable)
return;
x86_swiotlb_enable = false;
- xen_swiotlb = true;
- xen_swiotlb_init_early();
+ swiotlb_init_remap(true, x86_swiotlb_flags, xen_swiotlb_fixup);
dma_ops = &xen_swiotlb_dma_ops;
if (IS_ENABLED(CONFIG_PCI))
pci_request_acs();
@@ -84,14 +81,16 @@ static void __init pci_xen_swiotlb_init(void)
int pci_xen_swiotlb_init_late(void)
{
- int rc;
-
- if (xen_swiotlb)
+ if (dma_ops == &xen_swiotlb_dma_ops)
return 0;
- rc = xen_swiotlb_init();
- if (rc)
- return rc;
+ /* we can work with the default swiotlb */
+ if (!io_tlb_default_mem.nslabs) {
+ int rc = swiotlb_init_late(swiotlb_size_or_default(),
+ GFP_KERNEL, xen_swiotlb_fixup);
+ if (rc < 0)
+ return rc;
+ }
/* XXX: this switches the dma ops under live devices! */
dma_ops = &xen_swiotlb_dma_ops;
diff --git a/arch/x86/pci/sta2x11-fixup.c b/arch/x86/pci/sta2x11-fixup.c
index c7e6faf59a861..7368afc039987 100644
--- a/arch/x86/pci/sta2x11-fixup.c
+++ b/arch/x86/pci/sta2x11-fixup.c
@@ -57,7 +57,7 @@ static void sta2x11_new_instance(struct pci_dev *pdev)
int size = STA2X11_SWIOTLB_SIZE;
/* First instance: register your own swiotlb area */
dev_info(&pdev->dev, "Using SWIOTLB (size %i)\n", size);
- if (swiotlb_init_late(size, GFP_DMA))
+ if (swiotlb_init_late(size, GFP_DMA, NULL))
dev_emerg(&pdev->dev, "init swiotlb failed\n");
}
list_add(&instance->list, &sta2x11_instance_list);
diff --git a/drivers/xen/swiotlb-xen.c b/drivers/xen/swiotlb-xen.c
index c2da3eb4826e8..df8085b50df10 100644
--- a/drivers/xen/swiotlb-xen.c
+++ b/drivers/xen/swiotlb-xen.c
@@ -104,7 +104,7 @@ static int is_xen_swiotlb_buffer(struct device *dev, dma_addr_t dma_addr)
return 0;
}
-static int xen_swiotlb_fixup(void *buf, unsigned long nslabs)
+int xen_swiotlb_fixup(void *buf, unsigned long nslabs)
{
int rc;
unsigned int order = get_order(IO_TLB_SEGSIZE << IO_TLB_SHIFT);
@@ -130,132 +130,6 @@ static int xen_swiotlb_fixup(void *buf, unsigned long nslabs)
return 0;
}
-enum xen_swiotlb_err {
- XEN_SWIOTLB_UNKNOWN = 0,
- XEN_SWIOTLB_ENOMEM,
- XEN_SWIOTLB_EFIXUP
-};
-
-static const char *xen_swiotlb_error(enum xen_swiotlb_err err)
-{
- switch (err) {
- case XEN_SWIOTLB_ENOMEM:
- return "Cannot allocate Xen-SWIOTLB buffer\n";
- case XEN_SWIOTLB_EFIXUP:
- return "Failed to get contiguous memory for DMA from Xen!\n"\
- "You either: don't have the permissions, do not have"\
- " enough free memory under 4GB, or the hypervisor memory"\
- " is too fragmented!";
- default:
- break;
- }
- return "";
-}
-
-int xen_swiotlb_init(void)
-{
- enum xen_swiotlb_err m_ret = XEN_SWIOTLB_UNKNOWN;
- unsigned long bytes = swiotlb_size_or_default();
- unsigned long nslabs = bytes >> IO_TLB_SHIFT;
- unsigned int order, repeat = 3;
- int rc = -ENOMEM;
- char *start;
-
- if (io_tlb_default_mem.nslabs) {
- pr_warn("swiotlb buffer already initialized\n");
- return -EEXIST;
- }
-
-retry:
- m_ret = XEN_SWIOTLB_ENOMEM;
- order = get_order(bytes);
-
- /*
- * Get IO TLB memory from any location.
- */
-#define SLABS_PER_PAGE (1 << (PAGE_SHIFT - IO_TLB_SHIFT))
-#define IO_TLB_MIN_SLABS ((1<<20) >> IO_TLB_SHIFT)
- while ((SLABS_PER_PAGE << order) > IO_TLB_MIN_SLABS) {
- start = (void *)xen_get_swiotlb_free_pages(order);
- if (start)
- break;
- order--;
- }
- if (!start)
- goto exit;
- if (order != get_order(bytes)) {
- pr_warn("Warning: only able to allocate %ld MB for software IO TLB\n",
- (PAGE_SIZE << order) >> 20);
- nslabs = SLABS_PER_PAGE << order;
- bytes = nslabs << IO_TLB_SHIFT;
- }
-
- /*
- * And replace that memory with pages under 4GB.
- */
- rc = xen_swiotlb_fixup(start, nslabs);
- if (rc) {
- free_pages((unsigned long)start, order);
- m_ret = XEN_SWIOTLB_EFIXUP;
- goto error;
- }
- rc = swiotlb_late_init_with_tbl(start, nslabs);
- if (rc)
- return rc;
- return 0;
-error:
- if (nslabs > 1024 && repeat--) {
- /* Min is 2MB */
- nslabs = max(1024UL, ALIGN(nslabs >> 1, IO_TLB_SEGSIZE));
- bytes = nslabs << IO_TLB_SHIFT;
- pr_info("Lowering to %luMB\n", bytes >> 20);
- goto retry;
- }
-exit:
- pr_err("%s (rc:%d)\n", xen_swiotlb_error(m_ret), rc);
- return rc;
-}
-
-#ifdef CONFIG_X86
-void __init xen_swiotlb_init_early(void)
-{
- unsigned long bytes = swiotlb_size_or_default();
- unsigned long nslabs = bytes >> IO_TLB_SHIFT;
- unsigned int repeat = 3;
- char *start;
- int rc;
-
-retry:
- /*
- * Get IO TLB memory from any location.
- */
- start = memblock_alloc(PAGE_ALIGN(bytes),
- IO_TLB_SEGSIZE << IO_TLB_SHIFT);
- if (!start)
- panic("%s: Failed to allocate %lu bytes\n",
- __func__, PAGE_ALIGN(bytes));
-
- /*
- * And replace that memory with pages under 4GB.
- */
- rc = xen_swiotlb_fixup(start, nslabs);
- if (rc) {
- memblock_free(start, PAGE_ALIGN(bytes));
- if (nslabs > 1024 && repeat--) {
- /* Min is 2MB */
- nslabs = max(1024UL, ALIGN(nslabs >> 1, IO_TLB_SEGSIZE));
- bytes = nslabs << IO_TLB_SHIFT;
- pr_info("Lowering to %luMB\n", bytes >> 20);
- goto retry;
- }
- panic("%s (rc:%d)", xen_swiotlb_error(XEN_SWIOTLB_EFIXUP), rc);
- }
-
- if (swiotlb_init_with_tbl(start, nslabs, SWIOTLB_VERBOSE))
- panic("Cannot allocate SWIOTLB buffer");
-}
-#endif /* CONFIG_X86 */
-
static void *
xen_swiotlb_alloc_coherent(struct device *hwdev, size_t size,
dma_addr_t *dma_handle, gfp_t flags,
diff --git a/include/linux/swiotlb.h b/include/linux/swiotlb.h
index ee655f2e4d28b..919cf82ed978e 100644
--- a/include/linux/swiotlb.h
+++ b/include/linux/swiotlb.h
@@ -34,10 +34,11 @@ struct scatterlist;
/* default to 64MB */
#define IO_TLB_DEFAULT_SIZE (64UL<<20)
-int swiotlb_init_with_tbl(char *tlb, unsigned long nslabs, unsigned int flags);
unsigned long swiotlb_size_or_default(void);
-extern int swiotlb_late_init_with_tbl(char *tlb, unsigned long nslabs);
-int swiotlb_init_late(size_t size, gfp_t gfp_mask);
+int swiotlb_init_late(size_t size, gfp_t gfp_mask,
+ int (*remap)(void *tlb, unsigned long nslabs));
+void __init swiotlb_init_remap(bool addressing_limit, unsigned int flags,
+ int (*remap)(void *tlb, unsigned long nslabs));
extern void __init swiotlb_update_mem_attributes(void);
phys_addr_t swiotlb_tbl_map_single(struct device *hwdev, phys_addr_t phys,
diff --git a/include/xen/arm/page.h b/include/xen/arm/page.h
index ac1b654705631..7e199c6656b90 100644
--- a/include/xen/arm/page.h
+++ b/include/xen/arm/page.h
@@ -115,6 +115,5 @@ static inline bool set_phys_to_machine(unsigned long pfn, unsigned long mfn)
bool xen_arch_need_swiotlb(struct device *dev,
phys_addr_t phys,
dma_addr_t dev_addr);
-unsigned long xen_get_swiotlb_free_pages(unsigned int order);
#endif /* _ASM_ARM_XEN_PAGE_H */
diff --git a/include/xen/swiotlb-xen.h b/include/xen/swiotlb-xen.h
index b3e647f86e3e2..590ceb923f0c8 100644
--- a/include/xen/swiotlb-xen.h
+++ b/include/xen/swiotlb-xen.h
@@ -10,8 +10,12 @@ void xen_dma_sync_for_cpu(struct device *dev, dma_addr_t handle,
void xen_dma_sync_for_device(struct device *dev, dma_addr_t handle,
size_t size, enum dma_data_direction dir);
-int xen_swiotlb_init(void);
-void __init xen_swiotlb_init_early(void);
+#ifdef CONFIG_SWIOTLB_XEN
+int xen_swiotlb_fixup(void *buf, unsigned long nslabs);
+#else
+#define xen_swiotlb_fixup NULL
+#endif
+
extern const struct dma_map_ops xen_swiotlb_dma_ops;
#endif /* __LINUX_SWIOTLB_XEN_H */
diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
index ec200e40fc397..35ee2318ea563 100644
--- a/kernel/dma/swiotlb.c
+++ b/kernel/dma/swiotlb.c
@@ -234,40 +234,17 @@ static void swiotlb_init_io_tlb_mem(struct io_tlb_mem *mem, phys_addr_t start,
return;
}
-int __init swiotlb_init_with_tbl(char *tlb, unsigned long nslabs,
- unsigned int flags)
-{
- struct io_tlb_mem *mem = &io_tlb_default_mem;
- size_t alloc_size;
-
- if (swiotlb_force_disable)
- return 0;
-
- /* protect against double initialization */
- if (WARN_ON_ONCE(mem->nslabs))
- return -ENOMEM;
-
- alloc_size = PAGE_ALIGN(array_size(sizeof(*mem->slots), nslabs));
- mem->slots = memblock_alloc(alloc_size, PAGE_SIZE);
- if (!mem->slots)
- panic("%s: Failed to allocate %zu bytes align=0x%lx\n",
- __func__, alloc_size, PAGE_SIZE);
-
- swiotlb_init_io_tlb_mem(mem, __pa(tlb), nslabs, false);
- mem->force_bounce = flags & SWIOTLB_FORCE;
-
- if (flags & SWIOTLB_VERBOSE)
- swiotlb_print_info();
- return 0;
-}
-
/*
* Statically reserve bounce buffer space and initialize bounce buffer data
* structures for the software IO TLB used to implement the DMA API.
*/
-void __init swiotlb_init(bool addressing_limit, unsigned int flags)
+void __init swiotlb_init_remap(bool addressing_limit, unsigned int flags,
+ int (*remap)(void *tlb, unsigned long nslabs))
{
- size_t bytes = PAGE_ALIGN(default_nslabs << IO_TLB_SHIFT);
+ struct io_tlb_mem *mem = &io_tlb_default_mem;
+ unsigned long nslabs = default_nslabs;
+ size_t alloc_size = PAGE_ALIGN(array_size(sizeof(*mem->slots), nslabs));
+ size_t bytes;
void *tlb;
if (!addressing_limit && !swiotlb_force_bounce)
@@ -275,23 +252,48 @@ void __init swiotlb_init(bool addressing_limit, unsigned int flags)
if (swiotlb_force_disable)
return;
+ /* protect against double initialization */
+ if (WARN_ON_ONCE(mem->nslabs))
+ return;
+
/*
* By default allocate the bonuce buffer memory from low memory.
*/
+retry:
+ bytes = PAGE_ALIGN(default_nslabs << IO_TLB_SHIFT);
if (flags & SWIOTLB_ANY)
tlb = memblock_alloc(bytes, PAGE_SIZE);
else
tlb = memblock_alloc_low(bytes, PAGE_SIZE);
if (!tlb)
- goto fail;
- if (swiotlb_init_with_tbl(tlb, default_nslabs, flags))
- goto fail_free_mem;
- return;
+ panic("%s: failed to allocate tlb structure\n", __func__);
+
+ if (remap && remap(tlb, nslabs) < 0) {
+ memblock_free(tlb, PAGE_ALIGN(bytes));
+
+ /* Min is 2MB */
+ if (nslabs <= 1024)
+ panic("%s: Failed to remap %zu bytes\n",
+ __func__, bytes);
+ nslabs = max(1024UL, ALIGN(nslabs >> 1, IO_TLB_SEGSIZE));
+ goto retry;
+ }
+
+ mem->slots = memblock_alloc(alloc_size, PAGE_SIZE);
+ if (!mem->slots)
+ panic("%s: Failed to allocate %zu bytes align=0x%lx\n",
+ __func__, alloc_size, PAGE_SIZE);
-fail_free_mem:
- memblock_free(tlb, bytes);
-fail:
- pr_warn("Cannot allocate buffer");
+ swiotlb_init_io_tlb_mem(mem, __pa(tlb), default_nslabs, false);
+ mem->force_bounce = flags & SWIOTLB_FORCE;
+
+ if (flags & SWIOTLB_VERBOSE)
+ swiotlb_print_info();
+}
+
+void __init swiotlb_init(bool addressing_limit, unsigned int flags)
+{
+ return swiotlb_init_remap(addressing_limit, flags, NULL);
}
/*
@@ -299,8 +301,10 @@ void __init swiotlb_init(bool addressing_limit, unsigned int flags)
* initialize the swiotlb later using the slab allocator if needed.
* This should be just like above, but with some error catching.
*/
-int swiotlb_init_late(size_t size, gfp_t gfp_mask)
+int swiotlb_init_late(size_t size, gfp_t gfp_mask,
+ int (*remap)(void *tlb, unsigned long nslabs))
{
+ struct io_tlb_mem *mem = &io_tlb_default_mem;
unsigned long nslabs = ALIGN(size >> IO_TLB_SHIFT, IO_TLB_SEGSIZE);
unsigned long bytes;
unsigned char *vstart = NULL;
@@ -310,9 +314,14 @@ int swiotlb_init_late(size_t size, gfp_t gfp_mask)
if (swiotlb_force_disable)
return 0;
+ /* protect against double initialization */
+ if (WARN_ON_ONCE(mem->nslabs))
+ return -ENOMEM;
+
/*
* Get IO TLB memory from the low pages
*/
+retry:
order = get_order(nslabs << IO_TLB_SHIFT);
nslabs = SLABS_PER_PAGE << order;
bytes = nslabs << IO_TLB_SHIFT;
@@ -333,33 +342,28 @@ int swiotlb_init_late(size_t size, gfp_t gfp_mask)
(PAGE_SIZE << order) >> 20);
nslabs = SLABS_PER_PAGE << order;
}
- rc = swiotlb_late_init_with_tbl(vstart, nslabs);
- if (rc)
- free_pages((unsigned long)vstart, order);
-
- return rc;
-}
-
-int
-swiotlb_late_init_with_tbl(char *tlb, unsigned long nslabs)
-{
- struct io_tlb_mem *mem = &io_tlb_default_mem;
- unsigned long bytes = nslabs << IO_TLB_SHIFT;
- if (swiotlb_force_disable)
- return 0;
+ if (remap)
+ rc = remap(vstart, nslabs);
+ if (rc) {
+ free_pages((unsigned long)vstart, order);
- /* protect against double initialization */
- if (WARN_ON_ONCE(mem->nslabs))
- return -ENOMEM;
+ /* Min is 2MB */
+ if (nslabs <= 1024)
+ return rc;
+ nslabs = max(1024UL, ALIGN(nslabs >> 1, IO_TLB_SEGSIZE));
+ goto retry;
+ }
mem->slots = (void *)__get_free_pages(GFP_KERNEL | __GFP_ZERO,
get_order(array_size(sizeof(*mem->slots), nslabs)));
- if (!mem->slots)
+ if (!mem->slots) {
+ free_pages((unsigned long)vstart, order);
return -ENOMEM;
+ }
- set_memory_decrypted((unsigned long)tlb, bytes >> PAGE_SHIFT);
- swiotlb_init_io_tlb_mem(mem, virt_to_phys(tlb), nslabs, true);
+ set_memory_decrypted((unsigned long)vstart, bytes >> PAGE_SHIFT);
+ swiotlb_init_io_tlb_mem(mem, virt_to_phys(vstart), nslabs, true);
swiotlb_print_info();
return 0;
--
2.30.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 118+ messages in thread
* [PATCH 10/11] swiotlb: merge swiotlb-xen initialization into swiotlb
@ 2022-02-27 14:30 ` Christoph Hellwig
0 siblings, 0 replies; 118+ messages in thread
From: Christoph Hellwig @ 2022-02-27 14:30 UTC (permalink / raw)
To: iommu
Cc: Juergen Gross, linux-s390, linux-hyperv, Stefano Stabellini,
linux-ia64, Anshuman Khandual, Konrad Rzeszutek Wilk,
Robin Murphy, Joerg Roedel, x86, linux-mips, linuxppc-dev,
tboot-devel, linux-pci, xen-devel, Boris Ostrovsky,
David Woodhouse, linux-riscv, linux-arm-kernel, Lu Baolu
Allow to pass a remap argument to the swiotlb initialization functions
to handle the Xen/x86 remap case. ARM/ARM64 never did any remapping
from xen_swiotlb_fixup, so we don't even need that quirk.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
arch/arm/xen/mm.c | 23 +++---
arch/x86/include/asm/xen/page.h | 5 --
arch/x86/kernel/pci-dma.c | 19 +++--
arch/x86/pci/sta2x11-fixup.c | 2 +-
drivers/xen/swiotlb-xen.c | 128 +-------------------------------
include/linux/swiotlb.h | 7 +-
include/xen/arm/page.h | 1 -
include/xen/swiotlb-xen.h | 8 +-
kernel/dma/swiotlb.c | 120 +++++++++++++++---------------
9 files changed, 96 insertions(+), 217 deletions(-)
diff --git a/arch/arm/xen/mm.c b/arch/arm/xen/mm.c
index a7e54a087b802..58b40f87617d3 100644
--- a/arch/arm/xen/mm.c
+++ b/arch/arm/xen/mm.c
@@ -23,22 +23,20 @@
#include <asm/xen/hypercall.h>
#include <asm/xen/interface.h>
-unsigned long xen_get_swiotlb_free_pages(unsigned int order)
+static gfp_t xen_swiotlb_gfp(void)
{
phys_addr_t base;
- gfp_t flags = __GFP_NOWARN|__GFP_KSWAPD_RECLAIM;
u64 i;
for_each_mem_range(i, &base, NULL) {
if (base < (phys_addr_t)0xffffffff) {
if (IS_ENABLED(CONFIG_ZONE_DMA32))
- flags |= __GFP_DMA32;
- else
- flags |= __GFP_DMA;
- break;
+ return __GFP_DMA32;
+ return __GFP_DMA;
}
}
- return __get_free_pages(flags, order);
+
+ return GFP_KERNEL;
}
static bool hypercall_cflush = false;
@@ -143,10 +141,15 @@ static int __init xen_mm_init(void)
if (!xen_swiotlb_detect())
return 0;
- rc = xen_swiotlb_init();
/* we can work with the default swiotlb */
- if (rc < 0 && rc != -EEXIST)
- return rc;
+ if (!io_tlb_default_mem.nslabs) {
+ if (!xen_initial_domain())
+ return -EINVAL;
+ rc = swiotlb_init_late(swiotlb_size_or_default(),
+ xen_swiotlb_gfp(), NULL);
+ if (rc < 0)
+ return rc;
+ }
cflush.op = 0;
cflush.a.dev_bus_addr = 0;
diff --git a/arch/x86/include/asm/xen/page.h b/arch/x86/include/asm/xen/page.h
index e989bc2269f54..1fc67df500145 100644
--- a/arch/x86/include/asm/xen/page.h
+++ b/arch/x86/include/asm/xen/page.h
@@ -357,9 +357,4 @@ static inline bool xen_arch_need_swiotlb(struct device *dev,
return false;
}
-static inline unsigned long xen_get_swiotlb_free_pages(unsigned int order)
-{
- return __get_free_pages(__GFP_NOWARN, order);
-}
-
#endif /* _ASM_X86_XEN_PAGE_H */
diff --git a/arch/x86/kernel/pci-dma.c b/arch/x86/kernel/pci-dma.c
index bb08184a50e3a..b0901f027cdcd 100644
--- a/arch/x86/kernel/pci-dma.c
+++ b/arch/x86/kernel/pci-dma.c
@@ -68,15 +68,12 @@ static inline void __init pci_swiotlb_detect_4gb(void)
#endif /* CONFIG_SWIOTLB */
#ifdef CONFIG_SWIOTLB_XEN
-static bool xen_swiotlb;
-
static void __init pci_xen_swiotlb_init(void)
{
if (!xen_initial_domain() && !x86_swiotlb_enable)
return;
x86_swiotlb_enable = false;
- xen_swiotlb = true;
- xen_swiotlb_init_early();
+ swiotlb_init_remap(true, x86_swiotlb_flags, xen_swiotlb_fixup);
dma_ops = &xen_swiotlb_dma_ops;
if (IS_ENABLED(CONFIG_PCI))
pci_request_acs();
@@ -84,14 +81,16 @@ static void __init pci_xen_swiotlb_init(void)
int pci_xen_swiotlb_init_late(void)
{
- int rc;
-
- if (xen_swiotlb)
+ if (dma_ops == &xen_swiotlb_dma_ops)
return 0;
- rc = xen_swiotlb_init();
- if (rc)
- return rc;
+ /* we can work with the default swiotlb */
+ if (!io_tlb_default_mem.nslabs) {
+ int rc = swiotlb_init_late(swiotlb_size_or_default(),
+ GFP_KERNEL, xen_swiotlb_fixup);
+ if (rc < 0)
+ return rc;
+ }
/* XXX: this switches the dma ops under live devices! */
dma_ops = &xen_swiotlb_dma_ops;
diff --git a/arch/x86/pci/sta2x11-fixup.c b/arch/x86/pci/sta2x11-fixup.c
index c7e6faf59a861..7368afc039987 100644
--- a/arch/x86/pci/sta2x11-fixup.c
+++ b/arch/x86/pci/sta2x11-fixup.c
@@ -57,7 +57,7 @@ static void sta2x11_new_instance(struct pci_dev *pdev)
int size = STA2X11_SWIOTLB_SIZE;
/* First instance: register your own swiotlb area */
dev_info(&pdev->dev, "Using SWIOTLB (size %i)\n", size);
- if (swiotlb_init_late(size, GFP_DMA))
+ if (swiotlb_init_late(size, GFP_DMA, NULL))
dev_emerg(&pdev->dev, "init swiotlb failed\n");
}
list_add(&instance->list, &sta2x11_instance_list);
diff --git a/drivers/xen/swiotlb-xen.c b/drivers/xen/swiotlb-xen.c
index c2da3eb4826e8..df8085b50df10 100644
--- a/drivers/xen/swiotlb-xen.c
+++ b/drivers/xen/swiotlb-xen.c
@@ -104,7 +104,7 @@ static int is_xen_swiotlb_buffer(struct device *dev, dma_addr_t dma_addr)
return 0;
}
-static int xen_swiotlb_fixup(void *buf, unsigned long nslabs)
+int xen_swiotlb_fixup(void *buf, unsigned long nslabs)
{
int rc;
unsigned int order = get_order(IO_TLB_SEGSIZE << IO_TLB_SHIFT);
@@ -130,132 +130,6 @@ static int xen_swiotlb_fixup(void *buf, unsigned long nslabs)
return 0;
}
-enum xen_swiotlb_err {
- XEN_SWIOTLB_UNKNOWN = 0,
- XEN_SWIOTLB_ENOMEM,
- XEN_SWIOTLB_EFIXUP
-};
-
-static const char *xen_swiotlb_error(enum xen_swiotlb_err err)
-{
- switch (err) {
- case XEN_SWIOTLB_ENOMEM:
- return "Cannot allocate Xen-SWIOTLB buffer\n";
- case XEN_SWIOTLB_EFIXUP:
- return "Failed to get contiguous memory for DMA from Xen!\n"\
- "You either: don't have the permissions, do not have"\
- " enough free memory under 4GB, or the hypervisor memory"\
- " is too fragmented!";
- default:
- break;
- }
- return "";
-}
-
-int xen_swiotlb_init(void)
-{
- enum xen_swiotlb_err m_ret = XEN_SWIOTLB_UNKNOWN;
- unsigned long bytes = swiotlb_size_or_default();
- unsigned long nslabs = bytes >> IO_TLB_SHIFT;
- unsigned int order, repeat = 3;
- int rc = -ENOMEM;
- char *start;
-
- if (io_tlb_default_mem.nslabs) {
- pr_warn("swiotlb buffer already initialized\n");
- return -EEXIST;
- }
-
-retry:
- m_ret = XEN_SWIOTLB_ENOMEM;
- order = get_order(bytes);
-
- /*
- * Get IO TLB memory from any location.
- */
-#define SLABS_PER_PAGE (1 << (PAGE_SHIFT - IO_TLB_SHIFT))
-#define IO_TLB_MIN_SLABS ((1<<20) >> IO_TLB_SHIFT)
- while ((SLABS_PER_PAGE << order) > IO_TLB_MIN_SLABS) {
- start = (void *)xen_get_swiotlb_free_pages(order);
- if (start)
- break;
- order--;
- }
- if (!start)
- goto exit;
- if (order != get_order(bytes)) {
- pr_warn("Warning: only able to allocate %ld MB for software IO TLB\n",
- (PAGE_SIZE << order) >> 20);
- nslabs = SLABS_PER_PAGE << order;
- bytes = nslabs << IO_TLB_SHIFT;
- }
-
- /*
- * And replace that memory with pages under 4GB.
- */
- rc = xen_swiotlb_fixup(start, nslabs);
- if (rc) {
- free_pages((unsigned long)start, order);
- m_ret = XEN_SWIOTLB_EFIXUP;
- goto error;
- }
- rc = swiotlb_late_init_with_tbl(start, nslabs);
- if (rc)
- return rc;
- return 0;
-error:
- if (nslabs > 1024 && repeat--) {
- /* Min is 2MB */
- nslabs = max(1024UL, ALIGN(nslabs >> 1, IO_TLB_SEGSIZE));
- bytes = nslabs << IO_TLB_SHIFT;
- pr_info("Lowering to %luMB\n", bytes >> 20);
- goto retry;
- }
-exit:
- pr_err("%s (rc:%d)\n", xen_swiotlb_error(m_ret), rc);
- return rc;
-}
-
-#ifdef CONFIG_X86
-void __init xen_swiotlb_init_early(void)
-{
- unsigned long bytes = swiotlb_size_or_default();
- unsigned long nslabs = bytes >> IO_TLB_SHIFT;
- unsigned int repeat = 3;
- char *start;
- int rc;
-
-retry:
- /*
- * Get IO TLB memory from any location.
- */
- start = memblock_alloc(PAGE_ALIGN(bytes),
- IO_TLB_SEGSIZE << IO_TLB_SHIFT);
- if (!start)
- panic("%s: Failed to allocate %lu bytes\n",
- __func__, PAGE_ALIGN(bytes));
-
- /*
- * And replace that memory with pages under 4GB.
- */
- rc = xen_swiotlb_fixup(start, nslabs);
- if (rc) {
- memblock_free(start, PAGE_ALIGN(bytes));
- if (nslabs > 1024 && repeat--) {
- /* Min is 2MB */
- nslabs = max(1024UL, ALIGN(nslabs >> 1, IO_TLB_SEGSIZE));
- bytes = nslabs << IO_TLB_SHIFT;
- pr_info("Lowering to %luMB\n", bytes >> 20);
- goto retry;
- }
- panic("%s (rc:%d)", xen_swiotlb_error(XEN_SWIOTLB_EFIXUP), rc);
- }
-
- if (swiotlb_init_with_tbl(start, nslabs, SWIOTLB_VERBOSE))
- panic("Cannot allocate SWIOTLB buffer");
-}
-#endif /* CONFIG_X86 */
-
static void *
xen_swiotlb_alloc_coherent(struct device *hwdev, size_t size,
dma_addr_t *dma_handle, gfp_t flags,
diff --git a/include/linux/swiotlb.h b/include/linux/swiotlb.h
index ee655f2e4d28b..919cf82ed978e 100644
--- a/include/linux/swiotlb.h
+++ b/include/linux/swiotlb.h
@@ -34,10 +34,11 @@ struct scatterlist;
/* default to 64MB */
#define IO_TLB_DEFAULT_SIZE (64UL<<20)
-int swiotlb_init_with_tbl(char *tlb, unsigned long nslabs, unsigned int flags);
unsigned long swiotlb_size_or_default(void);
-extern int swiotlb_late_init_with_tbl(char *tlb, unsigned long nslabs);
-int swiotlb_init_late(size_t size, gfp_t gfp_mask);
+int swiotlb_init_late(size_t size, gfp_t gfp_mask,
+ int (*remap)(void *tlb, unsigned long nslabs));
+void __init swiotlb_init_remap(bool addressing_limit, unsigned int flags,
+ int (*remap)(void *tlb, unsigned long nslabs));
extern void __init swiotlb_update_mem_attributes(void);
phys_addr_t swiotlb_tbl_map_single(struct device *hwdev, phys_addr_t phys,
diff --git a/include/xen/arm/page.h b/include/xen/arm/page.h
index ac1b654705631..7e199c6656b90 100644
--- a/include/xen/arm/page.h
+++ b/include/xen/arm/page.h
@@ -115,6 +115,5 @@ static inline bool set_phys_to_machine(unsigned long pfn, unsigned long mfn)
bool xen_arch_need_swiotlb(struct device *dev,
phys_addr_t phys,
dma_addr_t dev_addr);
-unsigned long xen_get_swiotlb_free_pages(unsigned int order);
#endif /* _ASM_ARM_XEN_PAGE_H */
diff --git a/include/xen/swiotlb-xen.h b/include/xen/swiotlb-xen.h
index b3e647f86e3e2..590ceb923f0c8 100644
--- a/include/xen/swiotlb-xen.h
+++ b/include/xen/swiotlb-xen.h
@@ -10,8 +10,12 @@ void xen_dma_sync_for_cpu(struct device *dev, dma_addr_t handle,
void xen_dma_sync_for_device(struct device *dev, dma_addr_t handle,
size_t size, enum dma_data_direction dir);
-int xen_swiotlb_init(void);
-void __init xen_swiotlb_init_early(void);
+#ifdef CONFIG_SWIOTLB_XEN
+int xen_swiotlb_fixup(void *buf, unsigned long nslabs);
+#else
+#define xen_swiotlb_fixup NULL
+#endif
+
extern const struct dma_map_ops xen_swiotlb_dma_ops;
#endif /* __LINUX_SWIOTLB_XEN_H */
diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
index ec200e40fc397..35ee2318ea563 100644
--- a/kernel/dma/swiotlb.c
+++ b/kernel/dma/swiotlb.c
@@ -234,40 +234,17 @@ static void swiotlb_init_io_tlb_mem(struct io_tlb_mem *mem, phys_addr_t start,
return;
}
-int __init swiotlb_init_with_tbl(char *tlb, unsigned long nslabs,
- unsigned int flags)
-{
- struct io_tlb_mem *mem = &io_tlb_default_mem;
- size_t alloc_size;
-
- if (swiotlb_force_disable)
- return 0;
-
- /* protect against double initialization */
- if (WARN_ON_ONCE(mem->nslabs))
- return -ENOMEM;
-
- alloc_size = PAGE_ALIGN(array_size(sizeof(*mem->slots), nslabs));
- mem->slots = memblock_alloc(alloc_size, PAGE_SIZE);
- if (!mem->slots)
- panic("%s: Failed to allocate %zu bytes align=0x%lx\n",
- __func__, alloc_size, PAGE_SIZE);
-
- swiotlb_init_io_tlb_mem(mem, __pa(tlb), nslabs, false);
- mem->force_bounce = flags & SWIOTLB_FORCE;
-
- if (flags & SWIOTLB_VERBOSE)
- swiotlb_print_info();
- return 0;
-}
-
/*
* Statically reserve bounce buffer space and initialize bounce buffer data
* structures for the software IO TLB used to implement the DMA API.
*/
-void __init swiotlb_init(bool addressing_limit, unsigned int flags)
+void __init swiotlb_init_remap(bool addressing_limit, unsigned int flags,
+ int (*remap)(void *tlb, unsigned long nslabs))
{
- size_t bytes = PAGE_ALIGN(default_nslabs << IO_TLB_SHIFT);
+ struct io_tlb_mem *mem = &io_tlb_default_mem;
+ unsigned long nslabs = default_nslabs;
+ size_t alloc_size = PAGE_ALIGN(array_size(sizeof(*mem->slots), nslabs));
+ size_t bytes;
void *tlb;
if (!addressing_limit && !swiotlb_force_bounce)
@@ -275,23 +252,48 @@ void __init swiotlb_init(bool addressing_limit, unsigned int flags)
if (swiotlb_force_disable)
return;
+ /* protect against double initialization */
+ if (WARN_ON_ONCE(mem->nslabs))
+ return;
+
/*
* By default allocate the bonuce buffer memory from low memory.
*/
+retry:
+ bytes = PAGE_ALIGN(default_nslabs << IO_TLB_SHIFT);
if (flags & SWIOTLB_ANY)
tlb = memblock_alloc(bytes, PAGE_SIZE);
else
tlb = memblock_alloc_low(bytes, PAGE_SIZE);
if (!tlb)
- goto fail;
- if (swiotlb_init_with_tbl(tlb, default_nslabs, flags))
- goto fail_free_mem;
- return;
+ panic("%s: failed to allocate tlb structure\n", __func__);
+
+ if (remap && remap(tlb, nslabs) < 0) {
+ memblock_free(tlb, PAGE_ALIGN(bytes));
+
+ /* Min is 2MB */
+ if (nslabs <= 1024)
+ panic("%s: Failed to remap %zu bytes\n",
+ __func__, bytes);
+ nslabs = max(1024UL, ALIGN(nslabs >> 1, IO_TLB_SEGSIZE));
+ goto retry;
+ }
+
+ mem->slots = memblock_alloc(alloc_size, PAGE_SIZE);
+ if (!mem->slots)
+ panic("%s: Failed to allocate %zu bytes align=0x%lx\n",
+ __func__, alloc_size, PAGE_SIZE);
-fail_free_mem:
- memblock_free(tlb, bytes);
-fail:
- pr_warn("Cannot allocate buffer");
+ swiotlb_init_io_tlb_mem(mem, __pa(tlb), default_nslabs, false);
+ mem->force_bounce = flags & SWIOTLB_FORCE;
+
+ if (flags & SWIOTLB_VERBOSE)
+ swiotlb_print_info();
+}
+
+void __init swiotlb_init(bool addressing_limit, unsigned int flags)
+{
+ return swiotlb_init_remap(addressing_limit, flags, NULL);
}
/*
@@ -299,8 +301,10 @@ void __init swiotlb_init(bool addressing_limit, unsigned int flags)
* initialize the swiotlb later using the slab allocator if needed.
* This should be just like above, but with some error catching.
*/
-int swiotlb_init_late(size_t size, gfp_t gfp_mask)
+int swiotlb_init_late(size_t size, gfp_t gfp_mask,
+ int (*remap)(void *tlb, unsigned long nslabs))
{
+ struct io_tlb_mem *mem = &io_tlb_default_mem;
unsigned long nslabs = ALIGN(size >> IO_TLB_SHIFT, IO_TLB_SEGSIZE);
unsigned long bytes;
unsigned char *vstart = NULL;
@@ -310,9 +314,14 @@ int swiotlb_init_late(size_t size, gfp_t gfp_mask)
if (swiotlb_force_disable)
return 0;
+ /* protect against double initialization */
+ if (WARN_ON_ONCE(mem->nslabs))
+ return -ENOMEM;
+
/*
* Get IO TLB memory from the low pages
*/
+retry:
order = get_order(nslabs << IO_TLB_SHIFT);
nslabs = SLABS_PER_PAGE << order;
bytes = nslabs << IO_TLB_SHIFT;
@@ -333,33 +342,28 @@ int swiotlb_init_late(size_t size, gfp_t gfp_mask)
(PAGE_SIZE << order) >> 20);
nslabs = SLABS_PER_PAGE << order;
}
- rc = swiotlb_late_init_with_tbl(vstart, nslabs);
- if (rc)
- free_pages((unsigned long)vstart, order);
-
- return rc;
-}
-
-int
-swiotlb_late_init_with_tbl(char *tlb, unsigned long nslabs)
-{
- struct io_tlb_mem *mem = &io_tlb_default_mem;
- unsigned long bytes = nslabs << IO_TLB_SHIFT;
- if (swiotlb_force_disable)
- return 0;
+ if (remap)
+ rc = remap(vstart, nslabs);
+ if (rc) {
+ free_pages((unsigned long)vstart, order);
- /* protect against double initialization */
- if (WARN_ON_ONCE(mem->nslabs))
- return -ENOMEM;
+ /* Min is 2MB */
+ if (nslabs <= 1024)
+ return rc;
+ nslabs = max(1024UL, ALIGN(nslabs >> 1, IO_TLB_SEGSIZE));
+ goto retry;
+ }
mem->slots = (void *)__get_free_pages(GFP_KERNEL | __GFP_ZERO,
get_order(array_size(sizeof(*mem->slots), nslabs)));
- if (!mem->slots)
+ if (!mem->slots) {
+ free_pages((unsigned long)vstart, order);
return -ENOMEM;
+ }
- set_memory_decrypted((unsigned long)tlb, bytes >> PAGE_SHIFT);
- swiotlb_init_io_tlb_mem(mem, virt_to_phys(tlb), nslabs, true);
+ set_memory_decrypted((unsigned long)vstart, bytes >> PAGE_SHIFT);
+ swiotlb_init_io_tlb_mem(mem, virt_to_phys(vstart), nslabs, true);
swiotlb_print_info();
return 0;
--
2.30.2
^ permalink raw reply related [flat|nested] 118+ messages in thread
* [PATCH 10/11] swiotlb: merge swiotlb-xen initialization into swiotlb
@ 2022-02-27 14:30 ` Christoph Hellwig
0 siblings, 0 replies; 118+ messages in thread
From: Christoph Hellwig @ 2022-02-27 14:30 UTC (permalink / raw)
To: iommu
Cc: x86, Anshuman Khandual, Konrad Rzeszutek Wilk,
Stefano Stabellini, Boris Ostrovsky, Juergen Gross, Joerg Roedel,
David Woodhouse, Lu Baolu, Robin Murphy, linux-arm-kernel,
xen-devel, linux-ia64, linux-mips, linuxppc-dev, linux-riscv,
linux-s390, linux-hyperv, tboot-devel, linux-pci
Allow to pass a remap argument to the swiotlb initialization functions
to handle the Xen/x86 remap case. ARM/ARM64 never did any remapping
from xen_swiotlb_fixup, so we don't even need that quirk.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
arch/arm/xen/mm.c | 23 +++---
arch/x86/include/asm/xen/page.h | 5 --
arch/x86/kernel/pci-dma.c | 19 +++--
arch/x86/pci/sta2x11-fixup.c | 2 +-
drivers/xen/swiotlb-xen.c | 128 +-------------------------------
include/linux/swiotlb.h | 7 +-
include/xen/arm/page.h | 1 -
include/xen/swiotlb-xen.h | 8 +-
kernel/dma/swiotlb.c | 120 +++++++++++++++---------------
9 files changed, 96 insertions(+), 217 deletions(-)
diff --git a/arch/arm/xen/mm.c b/arch/arm/xen/mm.c
index a7e54a087b802..58b40f87617d3 100644
--- a/arch/arm/xen/mm.c
+++ b/arch/arm/xen/mm.c
@@ -23,22 +23,20 @@
#include <asm/xen/hypercall.h>
#include <asm/xen/interface.h>
-unsigned long xen_get_swiotlb_free_pages(unsigned int order)
+static gfp_t xen_swiotlb_gfp(void)
{
phys_addr_t base;
- gfp_t flags = __GFP_NOWARN|__GFP_KSWAPD_RECLAIM;
u64 i;
for_each_mem_range(i, &base, NULL) {
if (base < (phys_addr_t)0xffffffff) {
if (IS_ENABLED(CONFIG_ZONE_DMA32))
- flags |= __GFP_DMA32;
- else
- flags |= __GFP_DMA;
- break;
+ return __GFP_DMA32;
+ return __GFP_DMA;
}
}
- return __get_free_pages(flags, order);
+
+ return GFP_KERNEL;
}
static bool hypercall_cflush = false;
@@ -143,10 +141,15 @@ static int __init xen_mm_init(void)
if (!xen_swiotlb_detect())
return 0;
- rc = xen_swiotlb_init();
/* we can work with the default swiotlb */
- if (rc < 0 && rc != -EEXIST)
- return rc;
+ if (!io_tlb_default_mem.nslabs) {
+ if (!xen_initial_domain())
+ return -EINVAL;
+ rc = swiotlb_init_late(swiotlb_size_or_default(),
+ xen_swiotlb_gfp(), NULL);
+ if (rc < 0)
+ return rc;
+ }
cflush.op = 0;
cflush.a.dev_bus_addr = 0;
diff --git a/arch/x86/include/asm/xen/page.h b/arch/x86/include/asm/xen/page.h
index e989bc2269f54..1fc67df500145 100644
--- a/arch/x86/include/asm/xen/page.h
+++ b/arch/x86/include/asm/xen/page.h
@@ -357,9 +357,4 @@ static inline bool xen_arch_need_swiotlb(struct device *dev,
return false;
}
-static inline unsigned long xen_get_swiotlb_free_pages(unsigned int order)
-{
- return __get_free_pages(__GFP_NOWARN, order);
-}
-
#endif /* _ASM_X86_XEN_PAGE_H */
diff --git a/arch/x86/kernel/pci-dma.c b/arch/x86/kernel/pci-dma.c
index bb08184a50e3a..b0901f027cdcd 100644
--- a/arch/x86/kernel/pci-dma.c
+++ b/arch/x86/kernel/pci-dma.c
@@ -68,15 +68,12 @@ static inline void __init pci_swiotlb_detect_4gb(void)
#endif /* CONFIG_SWIOTLB */
#ifdef CONFIG_SWIOTLB_XEN
-static bool xen_swiotlb;
-
static void __init pci_xen_swiotlb_init(void)
{
if (!xen_initial_domain() && !x86_swiotlb_enable)
return;
x86_swiotlb_enable = false;
- xen_swiotlb = true;
- xen_swiotlb_init_early();
+ swiotlb_init_remap(true, x86_swiotlb_flags, xen_swiotlb_fixup);
dma_ops = &xen_swiotlb_dma_ops;
if (IS_ENABLED(CONFIG_PCI))
pci_request_acs();
@@ -84,14 +81,16 @@ static void __init pci_xen_swiotlb_init(void)
int pci_xen_swiotlb_init_late(void)
{
- int rc;
-
- if (xen_swiotlb)
+ if (dma_ops = &xen_swiotlb_dma_ops)
return 0;
- rc = xen_swiotlb_init();
- if (rc)
- return rc;
+ /* we can work with the default swiotlb */
+ if (!io_tlb_default_mem.nslabs) {
+ int rc = swiotlb_init_late(swiotlb_size_or_default(),
+ GFP_KERNEL, xen_swiotlb_fixup);
+ if (rc < 0)
+ return rc;
+ }
/* XXX: this switches the dma ops under live devices! */
dma_ops = &xen_swiotlb_dma_ops;
diff --git a/arch/x86/pci/sta2x11-fixup.c b/arch/x86/pci/sta2x11-fixup.c
index c7e6faf59a861..7368afc039987 100644
--- a/arch/x86/pci/sta2x11-fixup.c
+++ b/arch/x86/pci/sta2x11-fixup.c
@@ -57,7 +57,7 @@ static void sta2x11_new_instance(struct pci_dev *pdev)
int size = STA2X11_SWIOTLB_SIZE;
/* First instance: register your own swiotlb area */
dev_info(&pdev->dev, "Using SWIOTLB (size %i)\n", size);
- if (swiotlb_init_late(size, GFP_DMA))
+ if (swiotlb_init_late(size, GFP_DMA, NULL))
dev_emerg(&pdev->dev, "init swiotlb failed\n");
}
list_add(&instance->list, &sta2x11_instance_list);
diff --git a/drivers/xen/swiotlb-xen.c b/drivers/xen/swiotlb-xen.c
index c2da3eb4826e8..df8085b50df10 100644
--- a/drivers/xen/swiotlb-xen.c
+++ b/drivers/xen/swiotlb-xen.c
@@ -104,7 +104,7 @@ static int is_xen_swiotlb_buffer(struct device *dev, dma_addr_t dma_addr)
return 0;
}
-static int xen_swiotlb_fixup(void *buf, unsigned long nslabs)
+int xen_swiotlb_fixup(void *buf, unsigned long nslabs)
{
int rc;
unsigned int order = get_order(IO_TLB_SEGSIZE << IO_TLB_SHIFT);
@@ -130,132 +130,6 @@ static int xen_swiotlb_fixup(void *buf, unsigned long nslabs)
return 0;
}
-enum xen_swiotlb_err {
- XEN_SWIOTLB_UNKNOWN = 0,
- XEN_SWIOTLB_ENOMEM,
- XEN_SWIOTLB_EFIXUP
-};
-
-static const char *xen_swiotlb_error(enum xen_swiotlb_err err)
-{
- switch (err) {
- case XEN_SWIOTLB_ENOMEM:
- return "Cannot allocate Xen-SWIOTLB buffer\n";
- case XEN_SWIOTLB_EFIXUP:
- return "Failed to get contiguous memory for DMA from Xen!\n"\
- "You either: don't have the permissions, do not have"\
- " enough free memory under 4GB, or the hypervisor memory"\
- " is too fragmented!";
- default:
- break;
- }
- return "";
-}
-
-int xen_swiotlb_init(void)
-{
- enum xen_swiotlb_err m_ret = XEN_SWIOTLB_UNKNOWN;
- unsigned long bytes = swiotlb_size_or_default();
- unsigned long nslabs = bytes >> IO_TLB_SHIFT;
- unsigned int order, repeat = 3;
- int rc = -ENOMEM;
- char *start;
-
- if (io_tlb_default_mem.nslabs) {
- pr_warn("swiotlb buffer already initialized\n");
- return -EEXIST;
- }
-
-retry:
- m_ret = XEN_SWIOTLB_ENOMEM;
- order = get_order(bytes);
-
- /*
- * Get IO TLB memory from any location.
- */
-#define SLABS_PER_PAGE (1 << (PAGE_SHIFT - IO_TLB_SHIFT))
-#define IO_TLB_MIN_SLABS ((1<<20) >> IO_TLB_SHIFT)
- while ((SLABS_PER_PAGE << order) > IO_TLB_MIN_SLABS) {
- start = (void *)xen_get_swiotlb_free_pages(order);
- if (start)
- break;
- order--;
- }
- if (!start)
- goto exit;
- if (order != get_order(bytes)) {
- pr_warn("Warning: only able to allocate %ld MB for software IO TLB\n",
- (PAGE_SIZE << order) >> 20);
- nslabs = SLABS_PER_PAGE << order;
- bytes = nslabs << IO_TLB_SHIFT;
- }
-
- /*
- * And replace that memory with pages under 4GB.
- */
- rc = xen_swiotlb_fixup(start, nslabs);
- if (rc) {
- free_pages((unsigned long)start, order);
- m_ret = XEN_SWIOTLB_EFIXUP;
- goto error;
- }
- rc = swiotlb_late_init_with_tbl(start, nslabs);
- if (rc)
- return rc;
- return 0;
-error:
- if (nslabs > 1024 && repeat--) {
- /* Min is 2MB */
- nslabs = max(1024UL, ALIGN(nslabs >> 1, IO_TLB_SEGSIZE));
- bytes = nslabs << IO_TLB_SHIFT;
- pr_info("Lowering to %luMB\n", bytes >> 20);
- goto retry;
- }
-exit:
- pr_err("%s (rc:%d)\n", xen_swiotlb_error(m_ret), rc);
- return rc;
-}
-
-#ifdef CONFIG_X86
-void __init xen_swiotlb_init_early(void)
-{
- unsigned long bytes = swiotlb_size_or_default();
- unsigned long nslabs = bytes >> IO_TLB_SHIFT;
- unsigned int repeat = 3;
- char *start;
- int rc;
-
-retry:
- /*
- * Get IO TLB memory from any location.
- */
- start = memblock_alloc(PAGE_ALIGN(bytes),
- IO_TLB_SEGSIZE << IO_TLB_SHIFT);
- if (!start)
- panic("%s: Failed to allocate %lu bytes\n",
- __func__, PAGE_ALIGN(bytes));
-
- /*
- * And replace that memory with pages under 4GB.
- */
- rc = xen_swiotlb_fixup(start, nslabs);
- if (rc) {
- memblock_free(start, PAGE_ALIGN(bytes));
- if (nslabs > 1024 && repeat--) {
- /* Min is 2MB */
- nslabs = max(1024UL, ALIGN(nslabs >> 1, IO_TLB_SEGSIZE));
- bytes = nslabs << IO_TLB_SHIFT;
- pr_info("Lowering to %luMB\n", bytes >> 20);
- goto retry;
- }
- panic("%s (rc:%d)", xen_swiotlb_error(XEN_SWIOTLB_EFIXUP), rc);
- }
-
- if (swiotlb_init_with_tbl(start, nslabs, SWIOTLB_VERBOSE))
- panic("Cannot allocate SWIOTLB buffer");
-}
-#endif /* CONFIG_X86 */
-
static void *
xen_swiotlb_alloc_coherent(struct device *hwdev, size_t size,
dma_addr_t *dma_handle, gfp_t flags,
diff --git a/include/linux/swiotlb.h b/include/linux/swiotlb.h
index ee655f2e4d28b..919cf82ed978e 100644
--- a/include/linux/swiotlb.h
+++ b/include/linux/swiotlb.h
@@ -34,10 +34,11 @@ struct scatterlist;
/* default to 64MB */
#define IO_TLB_DEFAULT_SIZE (64UL<<20)
-int swiotlb_init_with_tbl(char *tlb, unsigned long nslabs, unsigned int flags);
unsigned long swiotlb_size_or_default(void);
-extern int swiotlb_late_init_with_tbl(char *tlb, unsigned long nslabs);
-int swiotlb_init_late(size_t size, gfp_t gfp_mask);
+int swiotlb_init_late(size_t size, gfp_t gfp_mask,
+ int (*remap)(void *tlb, unsigned long nslabs));
+void __init swiotlb_init_remap(bool addressing_limit, unsigned int flags,
+ int (*remap)(void *tlb, unsigned long nslabs));
extern void __init swiotlb_update_mem_attributes(void);
phys_addr_t swiotlb_tbl_map_single(struct device *hwdev, phys_addr_t phys,
diff --git a/include/xen/arm/page.h b/include/xen/arm/page.h
index ac1b654705631..7e199c6656b90 100644
--- a/include/xen/arm/page.h
+++ b/include/xen/arm/page.h
@@ -115,6 +115,5 @@ static inline bool set_phys_to_machine(unsigned long pfn, unsigned long mfn)
bool xen_arch_need_swiotlb(struct device *dev,
phys_addr_t phys,
dma_addr_t dev_addr);
-unsigned long xen_get_swiotlb_free_pages(unsigned int order);
#endif /* _ASM_ARM_XEN_PAGE_H */
diff --git a/include/xen/swiotlb-xen.h b/include/xen/swiotlb-xen.h
index b3e647f86e3e2..590ceb923f0c8 100644
--- a/include/xen/swiotlb-xen.h
+++ b/include/xen/swiotlb-xen.h
@@ -10,8 +10,12 @@ void xen_dma_sync_for_cpu(struct device *dev, dma_addr_t handle,
void xen_dma_sync_for_device(struct device *dev, dma_addr_t handle,
size_t size, enum dma_data_direction dir);
-int xen_swiotlb_init(void);
-void __init xen_swiotlb_init_early(void);
+#ifdef CONFIG_SWIOTLB_XEN
+int xen_swiotlb_fixup(void *buf, unsigned long nslabs);
+#else
+#define xen_swiotlb_fixup NULL
+#endif
+
extern const struct dma_map_ops xen_swiotlb_dma_ops;
#endif /* __LINUX_SWIOTLB_XEN_H */
diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
index ec200e40fc397..35ee2318ea563 100644
--- a/kernel/dma/swiotlb.c
+++ b/kernel/dma/swiotlb.c
@@ -234,40 +234,17 @@ static void swiotlb_init_io_tlb_mem(struct io_tlb_mem *mem, phys_addr_t start,
return;
}
-int __init swiotlb_init_with_tbl(char *tlb, unsigned long nslabs,
- unsigned int flags)
-{
- struct io_tlb_mem *mem = &io_tlb_default_mem;
- size_t alloc_size;
-
- if (swiotlb_force_disable)
- return 0;
-
- /* protect against double initialization */
- if (WARN_ON_ONCE(mem->nslabs))
- return -ENOMEM;
-
- alloc_size = PAGE_ALIGN(array_size(sizeof(*mem->slots), nslabs));
- mem->slots = memblock_alloc(alloc_size, PAGE_SIZE);
- if (!mem->slots)
- panic("%s: Failed to allocate %zu bytes align=0x%lx\n",
- __func__, alloc_size, PAGE_SIZE);
-
- swiotlb_init_io_tlb_mem(mem, __pa(tlb), nslabs, false);
- mem->force_bounce = flags & SWIOTLB_FORCE;
-
- if (flags & SWIOTLB_VERBOSE)
- swiotlb_print_info();
- return 0;
-}
-
/*
* Statically reserve bounce buffer space and initialize bounce buffer data
* structures for the software IO TLB used to implement the DMA API.
*/
-void __init swiotlb_init(bool addressing_limit, unsigned int flags)
+void __init swiotlb_init_remap(bool addressing_limit, unsigned int flags,
+ int (*remap)(void *tlb, unsigned long nslabs))
{
- size_t bytes = PAGE_ALIGN(default_nslabs << IO_TLB_SHIFT);
+ struct io_tlb_mem *mem = &io_tlb_default_mem;
+ unsigned long nslabs = default_nslabs;
+ size_t alloc_size = PAGE_ALIGN(array_size(sizeof(*mem->slots), nslabs));
+ size_t bytes;
void *tlb;
if (!addressing_limit && !swiotlb_force_bounce)
@@ -275,23 +252,48 @@ void __init swiotlb_init(bool addressing_limit, unsigned int flags)
if (swiotlb_force_disable)
return;
+ /* protect against double initialization */
+ if (WARN_ON_ONCE(mem->nslabs))
+ return;
+
/*
* By default allocate the bonuce buffer memory from low memory.
*/
+retry:
+ bytes = PAGE_ALIGN(default_nslabs << IO_TLB_SHIFT);
if (flags & SWIOTLB_ANY)
tlb = memblock_alloc(bytes, PAGE_SIZE);
else
tlb = memblock_alloc_low(bytes, PAGE_SIZE);
if (!tlb)
- goto fail;
- if (swiotlb_init_with_tbl(tlb, default_nslabs, flags))
- goto fail_free_mem;
- return;
+ panic("%s: failed to allocate tlb structure\n", __func__);
+
+ if (remap && remap(tlb, nslabs) < 0) {
+ memblock_free(tlb, PAGE_ALIGN(bytes));
+
+ /* Min is 2MB */
+ if (nslabs <= 1024)
+ panic("%s: Failed to remap %zu bytes\n",
+ __func__, bytes);
+ nslabs = max(1024UL, ALIGN(nslabs >> 1, IO_TLB_SEGSIZE));
+ goto retry;
+ }
+
+ mem->slots = memblock_alloc(alloc_size, PAGE_SIZE);
+ if (!mem->slots)
+ panic("%s: Failed to allocate %zu bytes align=0x%lx\n",
+ __func__, alloc_size, PAGE_SIZE);
-fail_free_mem:
- memblock_free(tlb, bytes);
-fail:
- pr_warn("Cannot allocate buffer");
+ swiotlb_init_io_tlb_mem(mem, __pa(tlb), default_nslabs, false);
+ mem->force_bounce = flags & SWIOTLB_FORCE;
+
+ if (flags & SWIOTLB_VERBOSE)
+ swiotlb_print_info();
+}
+
+void __init swiotlb_init(bool addressing_limit, unsigned int flags)
+{
+ return swiotlb_init_remap(addressing_limit, flags, NULL);
}
/*
@@ -299,8 +301,10 @@ void __init swiotlb_init(bool addressing_limit, unsigned int flags)
* initialize the swiotlb later using the slab allocator if needed.
* This should be just like above, but with some error catching.
*/
-int swiotlb_init_late(size_t size, gfp_t gfp_mask)
+int swiotlb_init_late(size_t size, gfp_t gfp_mask,
+ int (*remap)(void *tlb, unsigned long nslabs))
{
+ struct io_tlb_mem *mem = &io_tlb_default_mem;
unsigned long nslabs = ALIGN(size >> IO_TLB_SHIFT, IO_TLB_SEGSIZE);
unsigned long bytes;
unsigned char *vstart = NULL;
@@ -310,9 +314,14 @@ int swiotlb_init_late(size_t size, gfp_t gfp_mask)
if (swiotlb_force_disable)
return 0;
+ /* protect against double initialization */
+ if (WARN_ON_ONCE(mem->nslabs))
+ return -ENOMEM;
+
/*
* Get IO TLB memory from the low pages
*/
+retry:
order = get_order(nslabs << IO_TLB_SHIFT);
nslabs = SLABS_PER_PAGE << order;
bytes = nslabs << IO_TLB_SHIFT;
@@ -333,33 +342,28 @@ int swiotlb_init_late(size_t size, gfp_t gfp_mask)
(PAGE_SIZE << order) >> 20);
nslabs = SLABS_PER_PAGE << order;
}
- rc = swiotlb_late_init_with_tbl(vstart, nslabs);
- if (rc)
- free_pages((unsigned long)vstart, order);
-
- return rc;
-}
-
-int
-swiotlb_late_init_with_tbl(char *tlb, unsigned long nslabs)
-{
- struct io_tlb_mem *mem = &io_tlb_default_mem;
- unsigned long bytes = nslabs << IO_TLB_SHIFT;
- if (swiotlb_force_disable)
- return 0;
+ if (remap)
+ rc = remap(vstart, nslabs);
+ if (rc) {
+ free_pages((unsigned long)vstart, order);
- /* protect against double initialization */
- if (WARN_ON_ONCE(mem->nslabs))
- return -ENOMEM;
+ /* Min is 2MB */
+ if (nslabs <= 1024)
+ return rc;
+ nslabs = max(1024UL, ALIGN(nslabs >> 1, IO_TLB_SEGSIZE));
+ goto retry;
+ }
mem->slots = (void *)__get_free_pages(GFP_KERNEL | __GFP_ZERO,
get_order(array_size(sizeof(*mem->slots), nslabs)));
- if (!mem->slots)
+ if (!mem->slots) {
+ free_pages((unsigned long)vstart, order);
return -ENOMEM;
+ }
- set_memory_decrypted((unsigned long)tlb, bytes >> PAGE_SHIFT);
- swiotlb_init_io_tlb_mem(mem, virt_to_phys(tlb), nslabs, true);
+ set_memory_decrypted((unsigned long)vstart, bytes >> PAGE_SHIFT);
+ swiotlb_init_io_tlb_mem(mem, virt_to_phys(vstart), nslabs, true);
swiotlb_print_info();
return 0;
--
2.30.2
^ permalink raw reply related [flat|nested] 118+ messages in thread
* [PATCH 11/11] x86: remove cruft from <asm/dma-mapping.h>
2022-02-27 14:30 ` Christoph Hellwig
` (3 preceding siblings ...)
(?)
@ 2022-02-27 14:30 ` Christoph Hellwig
-1 siblings, 0 replies; 118+ messages in thread
From: Christoph Hellwig @ 2022-02-27 14:30 UTC (permalink / raw)
To: iommu
Cc: x86, Anshuman Khandual, Konrad Rzeszutek Wilk,
Stefano Stabellini, Boris Ostrovsky, Juergen Gross, Joerg Roedel,
David Woodhouse, Lu Baolu, Robin Murphy, linux-arm-kernel,
xen-devel, linux-ia64, linux-mips, linuxppc-dev, linux-riscv,
linux-s390, linux-hyperv, tboot-devel, linux-pci
<asm/dma-mapping.h> gets pulled in by all drivers using the DMA API.
Remove x86 internal variables and unnecessary includes from it.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
arch/x86/include/asm/dma-mapping.h | 11 -----------
arch/x86/include/asm/iommu.h | 2 ++
2 files changed, 2 insertions(+), 11 deletions(-)
diff --git a/arch/x86/include/asm/dma-mapping.h b/arch/x86/include/asm/dma-mapping.h
index 256fd8115223d..1c66708e30623 100644
--- a/arch/x86/include/asm/dma-mapping.h
+++ b/arch/x86/include/asm/dma-mapping.h
@@ -2,17 +2,6 @@
#ifndef _ASM_X86_DMA_MAPPING_H
#define _ASM_X86_DMA_MAPPING_H
-/*
- * IOMMU interface. See Documentation/core-api/dma-api-howto.rst and
- * Documentation/core-api/dma-api.rst for documentation.
- */
-
-#include <linux/scatterlist.h>
-#include <asm/io.h>
-
-extern int iommu_merge;
-extern int panic_on_overflow;
-
extern const struct dma_map_ops *dma_ops;
static inline const struct dma_map_ops *get_arch_dma_ops(struct bus_type *bus)
diff --git a/arch/x86/include/asm/iommu.h b/arch/x86/include/asm/iommu.h
index dba89ed40d38d..0bef44d30a278 100644
--- a/arch/x86/include/asm/iommu.h
+++ b/arch/x86/include/asm/iommu.h
@@ -8,6 +8,8 @@
extern int force_iommu, no_iommu;
extern int iommu_detected;
+extern int iommu_merge;
+extern int panic_on_overflow;
#ifdef CONFIG_SWIOTLB
extern bool x86_swiotlb_enable;
--
2.30.2
^ permalink raw reply related [flat|nested] 118+ messages in thread
* [PATCH 11/11] x86: remove cruft from <asm/dma-mapping.h>
@ 2022-02-27 14:30 ` Christoph Hellwig
0 siblings, 0 replies; 118+ messages in thread
From: Christoph Hellwig @ 2022-02-27 14:30 UTC (permalink / raw)
To: iommu
Cc: Juergen Gross, linux-s390, linux-hyperv, Stefano Stabellini,
linux-ia64, Anshuman Khandual, Konrad Rzeszutek Wilk,
Robin Murphy, x86, linux-mips, linuxppc-dev, tboot-devel,
linux-pci, xen-devel, Boris Ostrovsky, David Woodhouse,
linux-riscv, linux-arm-kernel
<asm/dma-mapping.h> gets pulled in by all drivers using the DMA API.
Remove x86 internal variables and unnecessary includes from it.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
arch/x86/include/asm/dma-mapping.h | 11 -----------
arch/x86/include/asm/iommu.h | 2 ++
2 files changed, 2 insertions(+), 11 deletions(-)
diff --git a/arch/x86/include/asm/dma-mapping.h b/arch/x86/include/asm/dma-mapping.h
index 256fd8115223d..1c66708e30623 100644
--- a/arch/x86/include/asm/dma-mapping.h
+++ b/arch/x86/include/asm/dma-mapping.h
@@ -2,17 +2,6 @@
#ifndef _ASM_X86_DMA_MAPPING_H
#define _ASM_X86_DMA_MAPPING_H
-/*
- * IOMMU interface. See Documentation/core-api/dma-api-howto.rst and
- * Documentation/core-api/dma-api.rst for documentation.
- */
-
-#include <linux/scatterlist.h>
-#include <asm/io.h>
-
-extern int iommu_merge;
-extern int panic_on_overflow;
-
extern const struct dma_map_ops *dma_ops;
static inline const struct dma_map_ops *get_arch_dma_ops(struct bus_type *bus)
diff --git a/arch/x86/include/asm/iommu.h b/arch/x86/include/asm/iommu.h
index dba89ed40d38d..0bef44d30a278 100644
--- a/arch/x86/include/asm/iommu.h
+++ b/arch/x86/include/asm/iommu.h
@@ -8,6 +8,8 @@
extern int force_iommu, no_iommu;
extern int iommu_detected;
+extern int iommu_merge;
+extern int panic_on_overflow;
#ifdef CONFIG_SWIOTLB
extern bool x86_swiotlb_enable;
--
2.30.2
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply related [flat|nested] 118+ messages in thread
* [PATCH 11/11] x86: remove cruft from <asm/dma-mapping.h>
@ 2022-02-27 14:30 ` Christoph Hellwig
0 siblings, 0 replies; 118+ messages in thread
From: Christoph Hellwig @ 2022-02-27 14:30 UTC (permalink / raw)
To: iommu
Cc: x86, Anshuman Khandual, Konrad Rzeszutek Wilk,
Stefano Stabellini, Boris Ostrovsky, Juergen Gross, Joerg Roedel,
David Woodhouse, Lu Baolu, Robin Murphy, linux-arm-kernel,
xen-devel, linux-ia64, linux-mips, linuxppc-dev, linux-riscv,
linux-s390, linux-hyperv, tboot-devel, linux-pci
<asm/dma-mapping.h> gets pulled in by all drivers using the DMA API.
Remove x86 internal variables and unnecessary includes from it.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
arch/x86/include/asm/dma-mapping.h | 11 -----------
arch/x86/include/asm/iommu.h | 2 ++
2 files changed, 2 insertions(+), 11 deletions(-)
diff --git a/arch/x86/include/asm/dma-mapping.h b/arch/x86/include/asm/dma-mapping.h
index 256fd8115223d..1c66708e30623 100644
--- a/arch/x86/include/asm/dma-mapping.h
+++ b/arch/x86/include/asm/dma-mapping.h
@@ -2,17 +2,6 @@
#ifndef _ASM_X86_DMA_MAPPING_H
#define _ASM_X86_DMA_MAPPING_H
-/*
- * IOMMU interface. See Documentation/core-api/dma-api-howto.rst and
- * Documentation/core-api/dma-api.rst for documentation.
- */
-
-#include <linux/scatterlist.h>
-#include <asm/io.h>
-
-extern int iommu_merge;
-extern int panic_on_overflow;
-
extern const struct dma_map_ops *dma_ops;
static inline const struct dma_map_ops *get_arch_dma_ops(struct bus_type *bus)
diff --git a/arch/x86/include/asm/iommu.h b/arch/x86/include/asm/iommu.h
index dba89ed40d38d..0bef44d30a278 100644
--- a/arch/x86/include/asm/iommu.h
+++ b/arch/x86/include/asm/iommu.h
@@ -8,6 +8,8 @@
extern int force_iommu, no_iommu;
extern int iommu_detected;
+extern int iommu_merge;
+extern int panic_on_overflow;
#ifdef CONFIG_SWIOTLB
extern bool x86_swiotlb_enable;
--
2.30.2
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply related [flat|nested] 118+ messages in thread
* [PATCH 11/11] x86: remove cruft from <asm/dma-mapping.h>
@ 2022-02-27 14:30 ` Christoph Hellwig
0 siblings, 0 replies; 118+ messages in thread
From: Christoph Hellwig @ 2022-02-27 14:30 UTC (permalink / raw)
To: iommu
Cc: x86, Anshuman Khandual, Konrad Rzeszutek Wilk,
Stefano Stabellini, Boris Ostrovsky, Juergen Gross, Joerg Roedel,
David Woodhouse, Lu Baolu, Robin Murphy, linux-arm-kernel,
xen-devel, linux-ia64, linux-mips, linuxppc-dev, linux-riscv,
linux-s390, linux-hyperv, tboot-devel, linux-pci
<asm/dma-mapping.h> gets pulled in by all drivers using the DMA API.
Remove x86 internal variables and unnecessary includes from it.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
arch/x86/include/asm/dma-mapping.h | 11 -----------
arch/x86/include/asm/iommu.h | 2 ++
2 files changed, 2 insertions(+), 11 deletions(-)
diff --git a/arch/x86/include/asm/dma-mapping.h b/arch/x86/include/asm/dma-mapping.h
index 256fd8115223d..1c66708e30623 100644
--- a/arch/x86/include/asm/dma-mapping.h
+++ b/arch/x86/include/asm/dma-mapping.h
@@ -2,17 +2,6 @@
#ifndef _ASM_X86_DMA_MAPPING_H
#define _ASM_X86_DMA_MAPPING_H
-/*
- * IOMMU interface. See Documentation/core-api/dma-api-howto.rst and
- * Documentation/core-api/dma-api.rst for documentation.
- */
-
-#include <linux/scatterlist.h>
-#include <asm/io.h>
-
-extern int iommu_merge;
-extern int panic_on_overflow;
-
extern const struct dma_map_ops *dma_ops;
static inline const struct dma_map_ops *get_arch_dma_ops(struct bus_type *bus)
diff --git a/arch/x86/include/asm/iommu.h b/arch/x86/include/asm/iommu.h
index dba89ed40d38d..0bef44d30a278 100644
--- a/arch/x86/include/asm/iommu.h
+++ b/arch/x86/include/asm/iommu.h
@@ -8,6 +8,8 @@
extern int force_iommu, no_iommu;
extern int iommu_detected;
+extern int iommu_merge;
+extern int panic_on_overflow;
#ifdef CONFIG_SWIOTLB
extern bool x86_swiotlb_enable;
--
2.30.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 118+ messages in thread
* [PATCH 11/11] x86: remove cruft from <asm/dma-mapping.h>
@ 2022-02-27 14:30 ` Christoph Hellwig
0 siblings, 0 replies; 118+ messages in thread
From: Christoph Hellwig @ 2022-02-27 14:30 UTC (permalink / raw)
To: iommu
Cc: Juergen Gross, linux-s390, linux-hyperv, Stefano Stabellini,
linux-ia64, Anshuman Khandual, Konrad Rzeszutek Wilk,
Robin Murphy, Joerg Roedel, x86, linux-mips, linuxppc-dev,
tboot-devel, linux-pci, xen-devel, Boris Ostrovsky,
David Woodhouse, linux-riscv, linux-arm-kernel, Lu Baolu
<asm/dma-mapping.h> gets pulled in by all drivers using the DMA API.
Remove x86 internal variables and unnecessary includes from it.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
arch/x86/include/asm/dma-mapping.h | 11 -----------
arch/x86/include/asm/iommu.h | 2 ++
2 files changed, 2 insertions(+), 11 deletions(-)
diff --git a/arch/x86/include/asm/dma-mapping.h b/arch/x86/include/asm/dma-mapping.h
index 256fd8115223d..1c66708e30623 100644
--- a/arch/x86/include/asm/dma-mapping.h
+++ b/arch/x86/include/asm/dma-mapping.h
@@ -2,17 +2,6 @@
#ifndef _ASM_X86_DMA_MAPPING_H
#define _ASM_X86_DMA_MAPPING_H
-/*
- * IOMMU interface. See Documentation/core-api/dma-api-howto.rst and
- * Documentation/core-api/dma-api.rst for documentation.
- */
-
-#include <linux/scatterlist.h>
-#include <asm/io.h>
-
-extern int iommu_merge;
-extern int panic_on_overflow;
-
extern const struct dma_map_ops *dma_ops;
static inline const struct dma_map_ops *get_arch_dma_ops(struct bus_type *bus)
diff --git a/arch/x86/include/asm/iommu.h b/arch/x86/include/asm/iommu.h
index dba89ed40d38d..0bef44d30a278 100644
--- a/arch/x86/include/asm/iommu.h
+++ b/arch/x86/include/asm/iommu.h
@@ -8,6 +8,8 @@
extern int force_iommu, no_iommu;
extern int iommu_detected;
+extern int iommu_merge;
+extern int panic_on_overflow;
#ifdef CONFIG_SWIOTLB
extern bool x86_swiotlb_enable;
--
2.30.2
^ permalink raw reply related [flat|nested] 118+ messages in thread
* [PATCH 11/11] x86: remove cruft from <asm/dma-mapping.h>
@ 2022-02-27 14:30 ` Christoph Hellwig
0 siblings, 0 replies; 118+ messages in thread
From: Christoph Hellwig @ 2022-02-27 14:30 UTC (permalink / raw)
To: iommu
Cc: x86, Anshuman Khandual, Konrad Rzeszutek Wilk,
Stefano Stabellini, Boris Ostrovsky, Juergen Gross, Joerg Roedel,
David Woodhouse, Lu Baolu, Robin Murphy, linux-arm-kernel,
xen-devel, linux-ia64, linux-mips, linuxppc-dev, linux-riscv,
linux-s390, linux-hyperv, tboot-devel, linux-pci
<asm/dma-mapping.h> gets pulled in by all drivers using the DMA API.
Remove x86 internal variables and unnecessary includes from it.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
arch/x86/include/asm/dma-mapping.h | 11 -----------
arch/x86/include/asm/iommu.h | 2 ++
2 files changed, 2 insertions(+), 11 deletions(-)
diff --git a/arch/x86/include/asm/dma-mapping.h b/arch/x86/include/asm/dma-mapping.h
index 256fd8115223d..1c66708e30623 100644
--- a/arch/x86/include/asm/dma-mapping.h
+++ b/arch/x86/include/asm/dma-mapping.h
@@ -2,17 +2,6 @@
#ifndef _ASM_X86_DMA_MAPPING_H
#define _ASM_X86_DMA_MAPPING_H
-/*
- * IOMMU interface. See Documentation/core-api/dma-api-howto.rst and
- * Documentation/core-api/dma-api.rst for documentation.
- */
-
-#include <linux/scatterlist.h>
-#include <asm/io.h>
-
-extern int iommu_merge;
-extern int panic_on_overflow;
-
extern const struct dma_map_ops *dma_ops;
static inline const struct dma_map_ops *get_arch_dma_ops(struct bus_type *bus)
diff --git a/arch/x86/include/asm/iommu.h b/arch/x86/include/asm/iommu.h
index dba89ed40d38d..0bef44d30a278 100644
--- a/arch/x86/include/asm/iommu.h
+++ b/arch/x86/include/asm/iommu.h
@@ -8,6 +8,8 @@
extern int force_iommu, no_iommu;
extern int iommu_detected;
+extern int iommu_merge;
+extern int panic_on_overflow;
#ifdef CONFIG_SWIOTLB
extern bool x86_swiotlb_enable;
--
2.30.2
^ permalink raw reply related [flat|nested] 118+ messages in thread
* Re: [PATCH 08/11] swiotlb: make the swiotlb_init interface more useful
2022-02-27 14:30 ` Christoph Hellwig
` (3 preceding siblings ...)
(?)
@ 2022-02-27 18:15 ` Christophe Leroy
-1 siblings, 0 replies; 118+ messages in thread
From: Christophe Leroy @ 2022-02-27 18:15 UTC (permalink / raw)
To: Christoph Hellwig, iommu
Cc: Juergen Gross, linux-s390, linux-hyperv, Stefano Stabellini,
linux-ia64, Anshuman Khandual, Konrad Rzeszutek Wilk,
Robin Murphy, Joerg Roedel, x86, linux-mips, linuxppc-dev,
tboot-devel, linux-pci, xen-devel, Boris Ostrovsky,
David Woodhouse, linux-riscv, linux-arm-kernel, Lu Baolu
Le 27/02/2022 à 15:30, Christoph Hellwig a écrit :
> Pass a bool to pass if swiotlb needs to be enabled based on the
> addressing needs and replace the verbose argument with a set of
> flags, including one to force enable bounce buffering.
>
> Note that this patch removes the possibility to force xen-swiotlb
> use using swiotlb=force on the command line on x86 (arm and arm64
> never supported that), but this interface will be restored shortly.
>
> Signed-off-by: Christoph Hellwig <hch@lst.de>
> ---
> arch/arm/mm/init.c | 6 +----
> arch/arm64/mm/init.c | 6 +----
> arch/ia64/mm/init.c | 4 +--
> arch/mips/cavium-octeon/dma-octeon.c | 2 +-
> arch/mips/loongson64/dma.c | 2 +-
> arch/mips/sibyte/common/dma.c | 2 +-
> arch/powerpc/include/asm/swiotlb.h | 1 +
> arch/powerpc/mm/mem.c | 3 ++-
arch/powerpc/mm/mem.o:(.toc+0x0): undefined reference to `ppc_swiotlb_flags'
make[1]: *** [vmlinux] Error 1
/linux/Makefile:1155: recipe for target 'vmlinux' failed
> arch/powerpc/platforms/pseries/setup.c | 3 ---
> arch/riscv/mm/init.c | 8 +-----
> arch/s390/mm/init.c | 3 +--
> arch/x86/kernel/cpu/mshyperv.c | 8 ------
> arch/x86/kernel/pci-dma.c | 15 ++++++-----
> arch/x86/mm/mem_encrypt_amd.c | 3 ---
> drivers/xen/swiotlb-xen.c | 4 +--
> include/linux/swiotlb.h | 15 ++++++-----
> include/trace/events/swiotlb.h | 29 ++++++++-------------
> kernel/dma/swiotlb.c | 35 ++++++++++++++------------
> 18 files changed, 56 insertions(+), 93 deletions(-)
>
^ permalink raw reply [flat|nested] 118+ messages in thread
* Re: [PATCH 08/11] swiotlb: make the swiotlb_init interface more useful
@ 2022-02-27 18:15 ` Christophe Leroy
0 siblings, 0 replies; 118+ messages in thread
From: Christophe Leroy @ 2022-02-27 18:15 UTC (permalink / raw)
To: Christoph Hellwig, iommu
Cc: Juergen Gross, linux-s390, linux-hyperv, Stefano Stabellini,
linux-ia64, Anshuman Khandual, Konrad Rzeszutek Wilk,
Robin Murphy, Joerg Roedel, x86, linux-mips, linuxppc-dev,
tboot-devel, linux-pci, xen-devel, Boris Ostrovsky,
David Woodhouse, linux-riscv, linux-arm-kernel, Lu Baolu
Le 27/02/2022 à 15:30, Christoph Hellwig a écrit :
> Pass a bool to pass if swiotlb needs to be enabled based on the
> addressing needs and replace the verbose argument with a set of
> flags, including one to force enable bounce buffering.
>
> Note that this patch removes the possibility to force xen-swiotlb
> use using swiotlb=force on the command line on x86 (arm and arm64
> never supported that), but this interface will be restored shortly.
>
> Signed-off-by: Christoph Hellwig <hch@lst.de>
> ---
> arch/arm/mm/init.c | 6 +----
> arch/arm64/mm/init.c | 6 +----
> arch/ia64/mm/init.c | 4 +--
> arch/mips/cavium-octeon/dma-octeon.c | 2 +-
> arch/mips/loongson64/dma.c | 2 +-
> arch/mips/sibyte/common/dma.c | 2 +-
> arch/powerpc/include/asm/swiotlb.h | 1 +
> arch/powerpc/mm/mem.c | 3 ++-
arch/powerpc/mm/mem.o:(.toc+0x0): undefined reference to `ppc_swiotlb_flags'
make[1]: *** [vmlinux] Error 1
/linux/Makefile:1155: recipe for target 'vmlinux' failed
> arch/powerpc/platforms/pseries/setup.c | 3 ---
> arch/riscv/mm/init.c | 8 +-----
> arch/s390/mm/init.c | 3 +--
> arch/x86/kernel/cpu/mshyperv.c | 8 ------
> arch/x86/kernel/pci-dma.c | 15 ++++++-----
> arch/x86/mm/mem_encrypt_amd.c | 3 ---
> drivers/xen/swiotlb-xen.c | 4 +--
> include/linux/swiotlb.h | 15 ++++++-----
> include/trace/events/swiotlb.h | 29 ++++++++-------------
> kernel/dma/swiotlb.c | 35 ++++++++++++++------------
> 18 files changed, 56 insertions(+), 93 deletions(-)
>
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 118+ messages in thread
* Re: [PATCH 08/11] swiotlb: make the swiotlb_init interface more useful
@ 2022-02-27 18:15 ` Christophe Leroy
0 siblings, 0 replies; 118+ messages in thread
From: Christophe Leroy @ 2022-02-27 18:15 UTC (permalink / raw)
To: Christoph Hellwig, iommu
Cc: Juergen Gross, linux-s390, linux-hyperv, Stefano Stabellini,
linux-ia64, Anshuman Khandual, linuxppc-dev, Joerg Roedel,
Konrad Rzeszutek Wilk, x86, linux-riscv, linux-mips, tboot-devel,
linux-pci, xen-devel, Boris Ostrovsky, Robin Murphy,
David Woodhouse, linux-arm-kernel, Lu Baolu
Le 27/02/2022 à 15:30, Christoph Hellwig a écrit :
> Pass a bool to pass if swiotlb needs to be enabled based on the
> addressing needs and replace the verbose argument with a set of
> flags, including one to force enable bounce buffering.
>
> Note that this patch removes the possibility to force xen-swiotlb
> use using swiotlb=force on the command line on x86 (arm and arm64
> never supported that), but this interface will be restored shortly.
>
> Signed-off-by: Christoph Hellwig <hch@lst.de>
> ---
> arch/arm/mm/init.c | 6 +----
> arch/arm64/mm/init.c | 6 +----
> arch/ia64/mm/init.c | 4 +--
> arch/mips/cavium-octeon/dma-octeon.c | 2 +-
> arch/mips/loongson64/dma.c | 2 +-
> arch/mips/sibyte/common/dma.c | 2 +-
> arch/powerpc/include/asm/swiotlb.h | 1 +
> arch/powerpc/mm/mem.c | 3 ++-
arch/powerpc/mm/mem.o:(.toc+0x0): undefined reference to `ppc_swiotlb_flags'
make[1]: *** [vmlinux] Error 1
/linux/Makefile:1155: recipe for target 'vmlinux' failed
> arch/powerpc/platforms/pseries/setup.c | 3 ---
> arch/riscv/mm/init.c | 8 +-----
> arch/s390/mm/init.c | 3 +--
> arch/x86/kernel/cpu/mshyperv.c | 8 ------
> arch/x86/kernel/pci-dma.c | 15 ++++++-----
> arch/x86/mm/mem_encrypt_amd.c | 3 ---
> drivers/xen/swiotlb-xen.c | 4 +--
> include/linux/swiotlb.h | 15 ++++++-----
> include/trace/events/swiotlb.h | 29 ++++++++-------------
> kernel/dma/swiotlb.c | 35 ++++++++++++++------------
> 18 files changed, 56 insertions(+), 93 deletions(-)
>
^ permalink raw reply [flat|nested] 118+ messages in thread
* Re: [PATCH 08/11] swiotlb: make the swiotlb_init interface more useful
@ 2022-02-27 18:15 ` Christophe Leroy
0 siblings, 0 replies; 118+ messages in thread
From: Christophe Leroy @ 2022-02-27 18:15 UTC (permalink / raw)
To: Christoph Hellwig, iommu
Cc: Juergen Gross, linux-s390, linux-hyperv, Stefano Stabellini,
linux-ia64, Anshuman Khandual, linuxppc-dev,
Konrad Rzeszutek Wilk, x86, linux-riscv, linux-mips, tboot-devel,
linux-pci, xen-devel, Boris Ostrovsky, Robin Murphy,
David Woodhouse, linux-arm-kernel
Le 27/02/2022 à 15:30, Christoph Hellwig a écrit :
> Pass a bool to pass if swiotlb needs to be enabled based on the
> addressing needs and replace the verbose argument with a set of
> flags, including one to force enable bounce buffering.
>
> Note that this patch removes the possibility to force xen-swiotlb
> use using swiotlb=force on the command line on x86 (arm and arm64
> never supported that), but this interface will be restored shortly.
>
> Signed-off-by: Christoph Hellwig <hch@lst.de>
> ---
> arch/arm/mm/init.c | 6 +----
> arch/arm64/mm/init.c | 6 +----
> arch/ia64/mm/init.c | 4 +--
> arch/mips/cavium-octeon/dma-octeon.c | 2 +-
> arch/mips/loongson64/dma.c | 2 +-
> arch/mips/sibyte/common/dma.c | 2 +-
> arch/powerpc/include/asm/swiotlb.h | 1 +
> arch/powerpc/mm/mem.c | 3 ++-
arch/powerpc/mm/mem.o:(.toc+0x0): undefined reference to `ppc_swiotlb_flags'
make[1]: *** [vmlinux] Error 1
/linux/Makefile:1155: recipe for target 'vmlinux' failed
> arch/powerpc/platforms/pseries/setup.c | 3 ---
> arch/riscv/mm/init.c | 8 +-----
> arch/s390/mm/init.c | 3 +--
> arch/x86/kernel/cpu/mshyperv.c | 8 ------
> arch/x86/kernel/pci-dma.c | 15 ++++++-----
> arch/x86/mm/mem_encrypt_amd.c | 3 ---
> drivers/xen/swiotlb-xen.c | 4 +--
> include/linux/swiotlb.h | 15 ++++++-----
> include/trace/events/swiotlb.h | 29 ++++++++-------------
> kernel/dma/swiotlb.c | 35 ++++++++++++++------------
> 18 files changed, 56 insertions(+), 93 deletions(-)
>
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply [flat|nested] 118+ messages in thread
* Re: [PATCH 08/11] swiotlb: make the swiotlb_init interface more useful
@ 2022-02-27 18:15 ` Christophe Leroy
0 siblings, 0 replies; 118+ messages in thread
From: Christophe Leroy @ 2022-02-27 18:15 UTC (permalink / raw)
To: Christoph Hellwig, iommu
Cc: Juergen Gross, linux-s390, linux-hyperv, Stefano Stabellini,
linux-ia64, Anshuman Khandual, Konrad Rzeszutek Wilk,
Robin Murphy, Joerg Roedel, x86, linux-mips, linuxppc-dev,
tboot-devel, linux-pci, xen-devel, Boris Ostrovsky,
David Woodhouse, linux-riscv, linux-arm-kernel, Lu Baolu
Le 27/02/2022 à 15:30, Christoph Hellwig a écrit :
> Pass a bool to pass if swiotlb needs to be enabled based on the
> addressing needs and replace the verbose argument with a set of
> flags, including one to force enable bounce buffering.
>
> Note that this patch removes the possibility to force xen-swiotlb
> use using swiotlb=force on the command line on x86 (arm and arm64
> never supported that), but this interface will be restored shortly.
>
> Signed-off-by: Christoph Hellwig <hch@lst.de>
> ---
> arch/arm/mm/init.c | 6 +----
> arch/arm64/mm/init.c | 6 +----
> arch/ia64/mm/init.c | 4 +--
> arch/mips/cavium-octeon/dma-octeon.c | 2 +-
> arch/mips/loongson64/dma.c | 2 +-
> arch/mips/sibyte/common/dma.c | 2 +-
> arch/powerpc/include/asm/swiotlb.h | 1 +
> arch/powerpc/mm/mem.c | 3 ++-
arch/powerpc/mm/mem.o:(.toc+0x0): undefined reference to `ppc_swiotlb_flags'
make[1]: *** [vmlinux] Error 1
/linux/Makefile:1155: recipe for target 'vmlinux' failed
> arch/powerpc/platforms/pseries/setup.c | 3 ---
> arch/riscv/mm/init.c | 8 +-----
> arch/s390/mm/init.c | 3 +--
> arch/x86/kernel/cpu/mshyperv.c | 8 ------
> arch/x86/kernel/pci-dma.c | 15 ++++++-----
> arch/x86/mm/mem_encrypt_amd.c | 3 ---
> drivers/xen/swiotlb-xen.c | 4 +--
> include/linux/swiotlb.h | 15 ++++++-----
> include/trace/events/swiotlb.h | 29 ++++++++-------------
> kernel/dma/swiotlb.c | 35 ++++++++++++++------------
> 18 files changed, 56 insertions(+), 93 deletions(-)
>
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 118+ messages in thread
* Re: [PATCH 08/11] swiotlb: make the swiotlb_init interface more useful
@ 2022-02-27 18:15 ` Christophe Leroy
0 siblings, 0 replies; 118+ messages in thread
From: Christophe Leroy @ 2022-02-27 18:15 UTC (permalink / raw)
To: Christoph Hellwig, iommu
Cc: Juergen Gross, linux-s390, linux-hyperv, Stefano Stabellini,
linux-ia64, Anshuman Khandual, Konrad Rzeszutek Wilk,
Robin Murphy, Joerg Roedel, x86, linux-mips, linuxppc-dev,
tboot-devel, linux-pci, xen-devel, Boris Ostrovsky,
David Woodhouse, linux-riscv, linux-arm-kernel, Lu Baolu
DQoNCkxlIDI3LzAyLzIwMjIgw6AgMTU6MzAsIENocmlzdG9waCBIZWxsd2lnIGEgw6ljcml0wqA6
DQo+IFBhc3MgYSBib29sIHRvIHBhc3MgaWYgc3dpb3RsYiBuZWVkcyB0byBiZSBlbmFibGVkIGJh
c2VkIG9uIHRoZQ0KPiBhZGRyZXNzaW5nIG5lZWRzIGFuZCByZXBsYWNlIHRoZSB2ZXJib3NlIGFy
Z3VtZW50IHdpdGggYSBzZXQgb2YNCj4gZmxhZ3MsIGluY2x1ZGluZyBvbmUgdG8gZm9yY2UgZW5h
YmxlIGJvdW5jZSBidWZmZXJpbmcuDQo+IA0KPiBOb3RlIHRoYXQgdGhpcyBwYXRjaCByZW1vdmVz
IHRoZSBwb3NzaWJpbGl0eSB0byBmb3JjZSB4ZW4tc3dpb3RsYg0KPiB1c2UgdXNpbmcgc3dpb3Rs
Yj1mb3JjZSBvbiB0aGUgY29tbWFuZCBsaW5lIG9uIHg4NiAoYXJtIGFuZCBhcm02NA0KPiBuZXZl
ciBzdXBwb3J0ZWQgdGhhdCksIGJ1dCB0aGlzIGludGVyZmFjZSB3aWxsIGJlIHJlc3RvcmVkIHNo
b3J0bHkuDQo+IA0KPiBTaWduZWQtb2ZmLWJ5OiBDaHJpc3RvcGggSGVsbHdpZyA8aGNoQGxzdC5k
ZT4NCj4gLS0tDQo+ICAgYXJjaC9hcm0vbW0vaW5pdC5jICAgICAgICAgICAgICAgICAgICAgfCAg
NiArLS0tLQ0KPiAgIGFyY2gvYXJtNjQvbW0vaW5pdC5jICAgICAgICAgICAgICAgICAgIHwgIDYg
Ky0tLS0NCj4gICBhcmNoL2lhNjQvbW0vaW5pdC5jICAgICAgICAgICAgICAgICAgICB8ICA0ICst
LQ0KPiAgIGFyY2gvbWlwcy9jYXZpdW0tb2N0ZW9uL2RtYS1vY3Rlb24uYyAgIHwgIDIgKy0NCj4g
ICBhcmNoL21pcHMvbG9vbmdzb242NC9kbWEuYyAgICAgICAgICAgICB8ICAyICstDQo+ICAgYXJj
aC9taXBzL3NpYnl0ZS9jb21tb24vZG1hLmMgICAgICAgICAgfCAgMiArLQ0KPiAgIGFyY2gvcG93
ZXJwYy9pbmNsdWRlL2FzbS9zd2lvdGxiLmggICAgIHwgIDEgKw0KPiAgIGFyY2gvcG93ZXJwYy9t
bS9tZW0uYyAgICAgICAgICAgICAgICAgIHwgIDMgKystDQoNCmFyY2gvcG93ZXJwYy9tbS9tZW0u
bzooLnRvYysweDApOiB1bmRlZmluZWQgcmVmZXJlbmNlIHRvIGBwcGNfc3dpb3RsYl9mbGFncycN
Cm1ha2VbMV06ICoqKiBbdm1saW51eF0gRXJyb3IgMQ0KL2xpbnV4L01ha2VmaWxlOjExNTU6IHJl
Y2lwZSBmb3IgdGFyZ2V0ICd2bWxpbnV4JyBmYWlsZWQNCg0KDQo+ICAgYXJjaC9wb3dlcnBjL3Bs
YXRmb3Jtcy9wc2VyaWVzL3NldHVwLmMgfCAgMyAtLS0NCj4gICBhcmNoL3Jpc2N2L21tL2luaXQu
YyAgICAgICAgICAgICAgICAgICB8ICA4ICstLS0tLQ0KPiAgIGFyY2gvczM5MC9tbS9pbml0LmMg
ICAgICAgICAgICAgICAgICAgIHwgIDMgKy0tDQo+ICAgYXJjaC94ODYva2VybmVsL2NwdS9tc2h5
cGVydi5jICAgICAgICAgfCAgOCAtLS0tLS0NCj4gICBhcmNoL3g4Ni9rZXJuZWwvcGNpLWRtYS5j
ICAgICAgICAgICAgICB8IDE1ICsrKysrKy0tLS0tDQo+ICAgYXJjaC94ODYvbW0vbWVtX2VuY3J5
cHRfYW1kLmMgICAgICAgICAgfCAgMyAtLS0NCj4gICBkcml2ZXJzL3hlbi9zd2lvdGxiLXhlbi5j
ICAgICAgICAgICAgICB8ICA0ICstLQ0KPiAgIGluY2x1ZGUvbGludXgvc3dpb3RsYi5oICAgICAg
ICAgICAgICAgIHwgMTUgKysrKysrLS0tLS0NCj4gICBpbmNsdWRlL3RyYWNlL2V2ZW50cy9zd2lv
dGxiLmggICAgICAgICB8IDI5ICsrKysrKysrLS0tLS0tLS0tLS0tLQ0KPiAgIGtlcm5lbC9kbWEv
c3dpb3RsYi5jICAgICAgICAgICAgICAgICAgIHwgMzUgKysrKysrKysrKysrKystLS0tLS0tLS0t
LS0NCj4gICAxOCBmaWxlcyBjaGFuZ2VkLCA1NiBpbnNlcnRpb25zKCspLCA5MyBkZWxldGlvbnMo
LSkNCj4g
^ permalink raw reply [flat|nested] 118+ messages in thread
* RE: [PATCH 08/11] swiotlb: make the swiotlb_init interface more useful
2022-02-27 14:30 ` Christoph Hellwig
` (2 preceding siblings ...)
(?)
@ 2022-02-28 2:53 ` Michael Kelley (LINUX) via iommu
-1 siblings, 0 replies; 118+ messages in thread
From: Michael Kelley (LINUX) @ 2022-02-28 2:53 UTC (permalink / raw)
To: Christoph Hellwig, iommu
Cc: x86, Anshuman Khandual, Konrad Rzeszutek Wilk,
Stefano Stabellini, Boris Ostrovsky, Juergen Gross, Joerg Roedel,
David Woodhouse, Lu Baolu, Robin Murphy, linux-arm-kernel,
xen-devel, linux-ia64, linux-mips, linuxppc-dev, linux-riscv,
linux-s390, linux-hyperv, tboot-devel, linux-pci
From: Christoph Hellwig <hch@lst.de> Sent: Sunday, February 27, 2022 6:31 AM
>
> Pass a bool to pass if swiotlb needs to be enabled based on the
> addressing needs and replace the verbose argument with a set of
> flags, including one to force enable bounce buffering.
>
> Note that this patch removes the possibility to force xen-swiotlb
> use using swiotlb=force on the command line on x86 (arm and arm64
> never supported that), but this interface will be restored shortly.
>
> Signed-off-by: Christoph Hellwig <hch@lst.de>
> ---
> arch/arm/mm/init.c | 6 +----
> arch/arm64/mm/init.c | 6 +----
> arch/ia64/mm/init.c | 4 +--
> arch/mips/cavium-octeon/dma-octeon.c | 2 +-
> arch/mips/loongson64/dma.c | 2 +-
> arch/mips/sibyte/common/dma.c | 2 +-
> arch/powerpc/include/asm/swiotlb.h | 1 +
> arch/powerpc/mm/mem.c | 3 ++-
> arch/powerpc/platforms/pseries/setup.c | 3 ---
> arch/riscv/mm/init.c | 8 +-----
> arch/s390/mm/init.c | 3 +--
> arch/x86/kernel/cpu/mshyperv.c | 8 ------
> arch/x86/kernel/pci-dma.c | 15 ++++++-----
> arch/x86/mm/mem_encrypt_amd.c | 3 ---
> drivers/xen/swiotlb-xen.c | 4 +--
> include/linux/swiotlb.h | 15 ++++++-----
> include/trace/events/swiotlb.h | 29 ++++++++-------------
> kernel/dma/swiotlb.c | 35 ++++++++++++++------------
> 18 files changed, 56 insertions(+), 93 deletions(-)
[snip]
>
> diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c
> index 5a99f993e6392..568274917f1cd 100644
> --- a/arch/x86/kernel/cpu/mshyperv.c
> +++ b/arch/x86/kernel/cpu/mshyperv.c
> @@ -336,14 +336,6 @@ static void __init ms_hyperv_init_platform(void)
> swiotlb_unencrypted_base = ms_hyperv.shared_gpa_boundary;
> #endif
> }
> -
> -#ifdef CONFIG_SWIOTLB
> - /*
> - * Enable swiotlb force mode in Isolation VM to
> - * use swiotlb bounce buffer for dma transaction.
> - */
> - swiotlb_force = SWIOTLB_FORCE;
> -#endif
With this code removed, it's not clear to me what forces the use of the
swiotlb in a Hyper-V isolated VM. The code in pci_swiotlb_detect_4g() doesn't
catch this case because cc_platform_has(CC_ATTR_HOST_MEM_ENCRYPT)
returns "false" in a Hyper-V guest. In the Hyper-V guest, it's only
cc_platform_has(CC_ATTR_GUEST_MEM_ENCRYPT) that returns "true". I'm
looking more closely at the meaning of the CC_ATTR_* values, and it may
be that Hyper-V should also return "true" for CC_ATTR_MEM_ENCRYPT,
but I don't think CC_ATTR_HOST_MEM_ENCRYPT should return "true".
Michael
^ permalink raw reply [flat|nested] 118+ messages in thread
* RE: [PATCH 08/11] swiotlb: make the swiotlb_init interface more useful
@ 2022-02-28 2:53 ` Michael Kelley (LINUX) via iommu
0 siblings, 0 replies; 118+ messages in thread
From: Michael Kelley (LINUX) via iommu @ 2022-02-28 2:53 UTC (permalink / raw)
To: Christoph Hellwig, iommu
Cc: Juergen Gross, linux-s390, linux-hyperv, Stefano Stabellini,
linux-ia64, Anshuman Khandual, Konrad Rzeszutek Wilk,
Robin Murphy, x86, linux-mips, linuxppc-dev, tboot-devel,
linux-pci, xen-devel, Boris Ostrovsky, David Woodhouse,
linux-riscv, linux-arm-kernel
From: Christoph Hellwig <hch@lst.de> Sent: Sunday, February 27, 2022 6:31 AM
>
> Pass a bool to pass if swiotlb needs to be enabled based on the
> addressing needs and replace the verbose argument with a set of
> flags, including one to force enable bounce buffering.
>
> Note that this patch removes the possibility to force xen-swiotlb
> use using swiotlb=force on the command line on x86 (arm and arm64
> never supported that), but this interface will be restored shortly.
>
> Signed-off-by: Christoph Hellwig <hch@lst.de>
> ---
> arch/arm/mm/init.c | 6 +----
> arch/arm64/mm/init.c | 6 +----
> arch/ia64/mm/init.c | 4 +--
> arch/mips/cavium-octeon/dma-octeon.c | 2 +-
> arch/mips/loongson64/dma.c | 2 +-
> arch/mips/sibyte/common/dma.c | 2 +-
> arch/powerpc/include/asm/swiotlb.h | 1 +
> arch/powerpc/mm/mem.c | 3 ++-
> arch/powerpc/platforms/pseries/setup.c | 3 ---
> arch/riscv/mm/init.c | 8 +-----
> arch/s390/mm/init.c | 3 +--
> arch/x86/kernel/cpu/mshyperv.c | 8 ------
> arch/x86/kernel/pci-dma.c | 15 ++++++-----
> arch/x86/mm/mem_encrypt_amd.c | 3 ---
> drivers/xen/swiotlb-xen.c | 4 +--
> include/linux/swiotlb.h | 15 ++++++-----
> include/trace/events/swiotlb.h | 29 ++++++++-------------
> kernel/dma/swiotlb.c | 35 ++++++++++++++------------
> 18 files changed, 56 insertions(+), 93 deletions(-)
[snip]
>
> diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c
> index 5a99f993e6392..568274917f1cd 100644
> --- a/arch/x86/kernel/cpu/mshyperv.c
> +++ b/arch/x86/kernel/cpu/mshyperv.c
> @@ -336,14 +336,6 @@ static void __init ms_hyperv_init_platform(void)
> swiotlb_unencrypted_base = ms_hyperv.shared_gpa_boundary;
> #endif
> }
> -
> -#ifdef CONFIG_SWIOTLB
> - /*
> - * Enable swiotlb force mode in Isolation VM to
> - * use swiotlb bounce buffer for dma transaction.
> - */
> - swiotlb_force = SWIOTLB_FORCE;
> -#endif
With this code removed, it's not clear to me what forces the use of the
swiotlb in a Hyper-V isolated VM. The code in pci_swiotlb_detect_4g() doesn't
catch this case because cc_platform_has(CC_ATTR_HOST_MEM_ENCRYPT)
returns "false" in a Hyper-V guest. In the Hyper-V guest, it's only
cc_platform_has(CC_ATTR_GUEST_MEM_ENCRYPT) that returns "true". I'm
looking more closely at the meaning of the CC_ATTR_* values, and it may
be that Hyper-V should also return "true" for CC_ATTR_MEM_ENCRYPT,
but I don't think CC_ATTR_HOST_MEM_ENCRYPT should return "true".
Michael
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply [flat|nested] 118+ messages in thread
* RE: [PATCH 08/11] swiotlb: make the swiotlb_init interface more useful
@ 2022-02-28 2:53 ` Michael Kelley (LINUX) via iommu
0 siblings, 0 replies; 118+ messages in thread
From: Michael Kelley (LINUX) @ 2022-02-28 2:53 UTC (permalink / raw)
To: Christoph Hellwig, iommu
Cc: Juergen Gross, linux-s390, linux-hyperv, Stefano Stabellini,
linux-ia64, Anshuman Khandual, Konrad Rzeszutek Wilk,
Robin Murphy, Joerg Roedel, x86, linux-mips, linuxppc-dev,
tboot-devel, linux-pci, xen-devel, Boris Ostrovsky,
David Woodhouse, linux-riscv, linux-arm-kernel, Lu Baolu
From: Christoph Hellwig <hch@lst.de> Sent: Sunday, February 27, 2022 6:31 AM
>
> Pass a bool to pass if swiotlb needs to be enabled based on the
> addressing needs and replace the verbose argument with a set of
> flags, including one to force enable bounce buffering.
>
> Note that this patch removes the possibility to force xen-swiotlb
> use using swiotlb=force on the command line on x86 (arm and arm64
> never supported that), but this interface will be restored shortly.
>
> Signed-off-by: Christoph Hellwig <hch@lst.de>
> ---
> arch/arm/mm/init.c | 6 +----
> arch/arm64/mm/init.c | 6 +----
> arch/ia64/mm/init.c | 4 +--
> arch/mips/cavium-octeon/dma-octeon.c | 2 +-
> arch/mips/loongson64/dma.c | 2 +-
> arch/mips/sibyte/common/dma.c | 2 +-
> arch/powerpc/include/asm/swiotlb.h | 1 +
> arch/powerpc/mm/mem.c | 3 ++-
> arch/powerpc/platforms/pseries/setup.c | 3 ---
> arch/riscv/mm/init.c | 8 +-----
> arch/s390/mm/init.c | 3 +--
> arch/x86/kernel/cpu/mshyperv.c | 8 ------
> arch/x86/kernel/pci-dma.c | 15 ++++++-----
> arch/x86/mm/mem_encrypt_amd.c | 3 ---
> drivers/xen/swiotlb-xen.c | 4 +--
> include/linux/swiotlb.h | 15 ++++++-----
> include/trace/events/swiotlb.h | 29 ++++++++-------------
> kernel/dma/swiotlb.c | 35 ++++++++++++++------------
> 18 files changed, 56 insertions(+), 93 deletions(-)
[snip]
>
> diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c
> index 5a99f993e6392..568274917f1cd 100644
> --- a/arch/x86/kernel/cpu/mshyperv.c
> +++ b/arch/x86/kernel/cpu/mshyperv.c
> @@ -336,14 +336,6 @@ static void __init ms_hyperv_init_platform(void)
> swiotlb_unencrypted_base = ms_hyperv.shared_gpa_boundary;
> #endif
> }
> -
> -#ifdef CONFIG_SWIOTLB
> - /*
> - * Enable swiotlb force mode in Isolation VM to
> - * use swiotlb bounce buffer for dma transaction.
> - */
> - swiotlb_force = SWIOTLB_FORCE;
> -#endif
With this code removed, it's not clear to me what forces the use of the
swiotlb in a Hyper-V isolated VM. The code in pci_swiotlb_detect_4g() doesn't
catch this case because cc_platform_has(CC_ATTR_HOST_MEM_ENCRYPT)
returns "false" in a Hyper-V guest. In the Hyper-V guest, it's only
cc_platform_has(CC_ATTR_GUEST_MEM_ENCRYPT) that returns "true". I'm
looking more closely at the meaning of the CC_ATTR_* values, and it may
be that Hyper-V should also return "true" for CC_ATTR_MEM_ENCRYPT,
but I don't think CC_ATTR_HOST_MEM_ENCRYPT should return "true".
Michael
^ permalink raw reply [flat|nested] 118+ messages in thread
* RE: [PATCH 08/11] swiotlb: make the swiotlb_init interface more useful
@ 2022-02-28 2:53 ` Michael Kelley (LINUX) via iommu
0 siblings, 0 replies; 118+ messages in thread
From: Michael Kelley (LINUX) @ 2022-02-28 2:53 UTC (permalink / raw)
To: Christoph Hellwig, iommu
Cc: x86, Anshuman Khandual, Konrad Rzeszutek Wilk,
Stefano Stabellini, Boris Ostrovsky, Juergen Gross, Joerg Roedel,
David Woodhouse, Lu Baolu, Robin Murphy, linux-arm-kernel,
xen-devel, linux-ia64, linux-mips, linuxppc-dev, linux-riscv,
linux-s390, linux-hyperv, tboot-devel, linux-pci
From: Christoph Hellwig <hch@lst.de> Sent: Sunday, February 27, 2022 6:31 AM
>
> Pass a bool to pass if swiotlb needs to be enabled based on the
> addressing needs and replace the verbose argument with a set of
> flags, including one to force enable bounce buffering.
>
> Note that this patch removes the possibility to force xen-swiotlb
> use using swiotlb=force on the command line on x86 (arm and arm64
> never supported that), but this interface will be restored shortly.
>
> Signed-off-by: Christoph Hellwig <hch@lst.de>
> ---
> arch/arm/mm/init.c | 6 +----
> arch/arm64/mm/init.c | 6 +----
> arch/ia64/mm/init.c | 4 +--
> arch/mips/cavium-octeon/dma-octeon.c | 2 +-
> arch/mips/loongson64/dma.c | 2 +-
> arch/mips/sibyte/common/dma.c | 2 +-
> arch/powerpc/include/asm/swiotlb.h | 1 +
> arch/powerpc/mm/mem.c | 3 ++-
> arch/powerpc/platforms/pseries/setup.c | 3 ---
> arch/riscv/mm/init.c | 8 +-----
> arch/s390/mm/init.c | 3 +--
> arch/x86/kernel/cpu/mshyperv.c | 8 ------
> arch/x86/kernel/pci-dma.c | 15 ++++++-----
> arch/x86/mm/mem_encrypt_amd.c | 3 ---
> drivers/xen/swiotlb-xen.c | 4 +--
> include/linux/swiotlb.h | 15 ++++++-----
> include/trace/events/swiotlb.h | 29 ++++++++-------------
> kernel/dma/swiotlb.c | 35 ++++++++++++++------------
> 18 files changed, 56 insertions(+), 93 deletions(-)
[snip]
>
> diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c
> index 5a99f993e6392..568274917f1cd 100644
> --- a/arch/x86/kernel/cpu/mshyperv.c
> +++ b/arch/x86/kernel/cpu/mshyperv.c
> @@ -336,14 +336,6 @@ static void __init ms_hyperv_init_platform(void)
> swiotlb_unencrypted_base = ms_hyperv.shared_gpa_boundary;
> #endif
> }
> -
> -#ifdef CONFIG_SWIOTLB
> - /*
> - * Enable swiotlb force mode in Isolation VM to
> - * use swiotlb bounce buffer for dma transaction.
> - */
> - swiotlb_force = SWIOTLB_FORCE;
> -#endif
With this code removed, it's not clear to me what forces the use of the
swiotlb in a Hyper-V isolated VM. The code in pci_swiotlb_detect_4g() doesn't
catch this case because cc_platform_has(CC_ATTR_HOST_MEM_ENCRYPT)
returns "false" in a Hyper-V guest. In the Hyper-V guest, it's only
cc_platform_has(CC_ATTR_GUEST_MEM_ENCRYPT) that returns "true". I'm
looking more closely at the meaning of the CC_ATTR_* values, and it may
be that Hyper-V should also return "true" for CC_ATTR_MEM_ENCRYPT,
but I don't think CC_ATTR_HOST_MEM_ENCRYPT should return "true".
Michael
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 118+ messages in thread
* RE: [PATCH 08/11] swiotlb: make the swiotlb_init interface more useful
@ 2022-02-28 2:53 ` Michael Kelley (LINUX) via iommu
0 siblings, 0 replies; 118+ messages in thread
From: Michael Kelley (LINUX) @ 2022-02-28 2:53 UTC (permalink / raw)
To: Christoph Hellwig, iommu
Cc: x86, Anshuman Khandual, Konrad Rzeszutek Wilk,
Stefano Stabellini, Boris Ostrovsky, Juergen Gross, Joerg Roedel,
David Woodhouse, Lu Baolu, Robin Murphy, linux-arm-kernel,
xen-devel, linux-ia64, linux-mips, linuxppc-dev, linux-riscv,
linux-s390, linux-hyperv, tboot-devel, linux-pci
From: Christoph Hellwig <hch@lst.de> Sent: Sunday, February 27, 2022 6:31 AM
>
> Pass a bool to pass if swiotlb needs to be enabled based on the
> addressing needs and replace the verbose argument with a set of
> flags, including one to force enable bounce buffering.
>
> Note that this patch removes the possibility to force xen-swiotlb
> use using swiotlb=force on the command line on x86 (arm and arm64
> never supported that), but this interface will be restored shortly.
>
> Signed-off-by: Christoph Hellwig <hch@lst.de>
> ---
> arch/arm/mm/init.c | 6 +----
> arch/arm64/mm/init.c | 6 +----
> arch/ia64/mm/init.c | 4 +--
> arch/mips/cavium-octeon/dma-octeon.c | 2 +-
> arch/mips/loongson64/dma.c | 2 +-
> arch/mips/sibyte/common/dma.c | 2 +-
> arch/powerpc/include/asm/swiotlb.h | 1 +
> arch/powerpc/mm/mem.c | 3 ++-
> arch/powerpc/platforms/pseries/setup.c | 3 ---
> arch/riscv/mm/init.c | 8 +-----
> arch/s390/mm/init.c | 3 +--
> arch/x86/kernel/cpu/mshyperv.c | 8 ------
> arch/x86/kernel/pci-dma.c | 15 ++++++-----
> arch/x86/mm/mem_encrypt_amd.c | 3 ---
> drivers/xen/swiotlb-xen.c | 4 +--
> include/linux/swiotlb.h | 15 ++++++-----
> include/trace/events/swiotlb.h | 29 ++++++++-------------
> kernel/dma/swiotlb.c | 35 ++++++++++++++------------
> 18 files changed, 56 insertions(+), 93 deletions(-)
[snip]
>
> diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c
> index 5a99f993e6392..568274917f1cd 100644
> --- a/arch/x86/kernel/cpu/mshyperv.c
> +++ b/arch/x86/kernel/cpu/mshyperv.c
> @@ -336,14 +336,6 @@ static void __init ms_hyperv_init_platform(void)
> swiotlb_unencrypted_base = ms_hyperv.shared_gpa_boundary;
> #endif
> }
> -
> -#ifdef CONFIG_SWIOTLB
> - /*
> - * Enable swiotlb force mode in Isolation VM to
> - * use swiotlb bounce buffer for dma transaction.
> - */
> - swiotlb_force = SWIOTLB_FORCE;
> -#endif
With this code removed, it's not clear to me what forces the use of the
swiotlb in a Hyper-V isolated VM. The code in pci_swiotlb_detect_4g() doesn't
catch this case because cc_platform_has(CC_ATTR_HOST_MEM_ENCRYPT)
returns "false" in a Hyper-V guest. In the Hyper-V guest, it's only
cc_platform_has(CC_ATTR_GUEST_MEM_ENCRYPT) that returns "true". I'm
looking more closely at the meaning of the CC_ATTR_* values, and it may
be that Hyper-V should also return "true" for CC_ATTR_MEM_ENCRYPT,
but I don't think CC_ATTR_HOST_MEM_ENCRYPT should return "true".
Michael
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 118+ messages in thread
* Re: [PATCH 08/11] swiotlb: make the swiotlb_init interface more useful
2022-02-28 2:53 ` Michael Kelley (LINUX) via iommu
` (3 preceding siblings ...)
(?)
@ 2022-02-28 11:30 ` Christoph Hellwig
-1 siblings, 0 replies; 118+ messages in thread
From: Christoph Hellwig @ 2022-02-28 11:30 UTC (permalink / raw)
To: Michael Kelley (LINUX)
Cc: Christoph Hellwig, iommu, x86, Anshuman Khandual,
Konrad Rzeszutek Wilk, Stefano Stabellini, Boris Ostrovsky,
Juergen Gross, Joerg Roedel, David Woodhouse, Lu Baolu,
Robin Murphy, linux-arm-kernel, xen-devel, linux-ia64,
linux-mips, linuxppc-dev, linux-riscv, linux-s390, linux-hyperv,
tboot-devel, linux-pci
On Mon, Feb 28, 2022 at 02:53:39AM +0000, Michael Kelley (LINUX) wrote:
> From: Christoph Hellwig <hch@lst.de> Sent: Sunday, February 27, 2022 6:31 AM
> >
> > Pass a bool to pass if swiotlb needs to be enabled based on the
> > addressing needs and replace the verbose argument with a set of
> > flags, including one to force enable bounce buffering.
> >
> > Note that this patch removes the possibility to force xen-swiotlb
> > use using swiotlb=force on the command line on x86 (arm and arm64
> > never supported that), but this interface will be restored shortly.
> >
> > Signed-off-by: Christoph Hellwig <hch@lst.de>
> > ---
> > arch/arm/mm/init.c | 6 +----
> > arch/arm64/mm/init.c | 6 +----
> > arch/ia64/mm/init.c | 4 +--
> > arch/mips/cavium-octeon/dma-octeon.c | 2 +-
> > arch/mips/loongson64/dma.c | 2 +-
> > arch/mips/sibyte/common/dma.c | 2 +-
> > arch/powerpc/include/asm/swiotlb.h | 1 +
> > arch/powerpc/mm/mem.c | 3 ++-
> > arch/powerpc/platforms/pseries/setup.c | 3 ---
> > arch/riscv/mm/init.c | 8 +-----
> > arch/s390/mm/init.c | 3 +--
> > arch/x86/kernel/cpu/mshyperv.c | 8 ------
> > arch/x86/kernel/pci-dma.c | 15 ++++++-----
> > arch/x86/mm/mem_encrypt_amd.c | 3 ---
> > drivers/xen/swiotlb-xen.c | 4 +--
> > include/linux/swiotlb.h | 15 ++++++-----
> > include/trace/events/swiotlb.h | 29 ++++++++-------------
> > kernel/dma/swiotlb.c | 35 ++++++++++++++------------
> > 18 files changed, 56 insertions(+), 93 deletions(-)
>
> [snip]
>
> >
> > diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c
> > index 5a99f993e6392..568274917f1cd 100644
> > --- a/arch/x86/kernel/cpu/mshyperv.c
> > +++ b/arch/x86/kernel/cpu/mshyperv.c
> > @@ -336,14 +336,6 @@ static void __init ms_hyperv_init_platform(void)
> > swiotlb_unencrypted_base = ms_hyperv.shared_gpa_boundary;
> > #endif
> > }
> > -
> > -#ifdef CONFIG_SWIOTLB
> > - /*
> > - * Enable swiotlb force mode in Isolation VM to
> > - * use swiotlb bounce buffer for dma transaction.
> > - */
> > - swiotlb_force = SWIOTLB_FORCE;
> > -#endif
>
> With this code removed, it's not clear to me what forces the use of the
> swiotlb in a Hyper-V isolated VM. The code in pci_swiotlb_detect_4g() doesn't
> catch this case because cc_platform_has(CC_ATTR_HOST_MEM_ENCRYPT)
> returns "false" in a Hyper-V guest. In the Hyper-V guest, it's only
> cc_platform_has(CC_ATTR_GUEST_MEM_ENCRYPT) that returns "true". I'm
> looking more closely at the meaning of the CC_ATTR_* values, and it may
> be that Hyper-V should also return "true" for CC_ATTR_MEM_ENCRYPT,
> but I don't think CC_ATTR_HOST_MEM_ENCRYPT should return "true".
Ok, I assumed that CC_ATTR_HOST_MEM_ENCRYPT returned true in this case.
I guess we just need to check for CC_ATTR_GUEST_MEM_ENCRYPT as well
there?
^ permalink raw reply [flat|nested] 118+ messages in thread
* Re: [PATCH 08/11] swiotlb: make the swiotlb_init interface more useful
@ 2022-02-28 11:30 ` Christoph Hellwig
0 siblings, 0 replies; 118+ messages in thread
From: Christoph Hellwig @ 2022-02-28 11:30 UTC (permalink / raw)
To: Michael Kelley (LINUX)
Cc: linux-hyperv, x86, linux-ia64, linux-pci, linux-riscv,
Christoph Hellwig, linux-s390, Stefano Stabellini, Joerg Roedel,
Konrad Rzeszutek Wilk, tboot-devel, xen-devel, David Woodhouse,
Anshuman Khandual, Boris Ostrovsky, linux-arm-kernel,
Juergen Gross, linuxppc-dev, linux-mips, iommu, Robin Murphy,
Lu Baolu
On Mon, Feb 28, 2022 at 02:53:39AM +0000, Michael Kelley (LINUX) wrote:
> From: Christoph Hellwig <hch@lst.de> Sent: Sunday, February 27, 2022 6:31 AM
> >
> > Pass a bool to pass if swiotlb needs to be enabled based on the
> > addressing needs and replace the verbose argument with a set of
> > flags, including one to force enable bounce buffering.
> >
> > Note that this patch removes the possibility to force xen-swiotlb
> > use using swiotlb=force on the command line on x86 (arm and arm64
> > never supported that), but this interface will be restored shortly.
> >
> > Signed-off-by: Christoph Hellwig <hch@lst.de>
> > ---
> > arch/arm/mm/init.c | 6 +----
> > arch/arm64/mm/init.c | 6 +----
> > arch/ia64/mm/init.c | 4 +--
> > arch/mips/cavium-octeon/dma-octeon.c | 2 +-
> > arch/mips/loongson64/dma.c | 2 +-
> > arch/mips/sibyte/common/dma.c | 2 +-
> > arch/powerpc/include/asm/swiotlb.h | 1 +
> > arch/powerpc/mm/mem.c | 3 ++-
> > arch/powerpc/platforms/pseries/setup.c | 3 ---
> > arch/riscv/mm/init.c | 8 +-----
> > arch/s390/mm/init.c | 3 +--
> > arch/x86/kernel/cpu/mshyperv.c | 8 ------
> > arch/x86/kernel/pci-dma.c | 15 ++++++-----
> > arch/x86/mm/mem_encrypt_amd.c | 3 ---
> > drivers/xen/swiotlb-xen.c | 4 +--
> > include/linux/swiotlb.h | 15 ++++++-----
> > include/trace/events/swiotlb.h | 29 ++++++++-------------
> > kernel/dma/swiotlb.c | 35 ++++++++++++++------------
> > 18 files changed, 56 insertions(+), 93 deletions(-)
>
> [snip]
>
> >
> > diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c
> > index 5a99f993e6392..568274917f1cd 100644
> > --- a/arch/x86/kernel/cpu/mshyperv.c
> > +++ b/arch/x86/kernel/cpu/mshyperv.c
> > @@ -336,14 +336,6 @@ static void __init ms_hyperv_init_platform(void)
> > swiotlb_unencrypted_base = ms_hyperv.shared_gpa_boundary;
> > #endif
> > }
> > -
> > -#ifdef CONFIG_SWIOTLB
> > - /*
> > - * Enable swiotlb force mode in Isolation VM to
> > - * use swiotlb bounce buffer for dma transaction.
> > - */
> > - swiotlb_force = SWIOTLB_FORCE;
> > -#endif
>
> With this code removed, it's not clear to me what forces the use of the
> swiotlb in a Hyper-V isolated VM. The code in pci_swiotlb_detect_4g() doesn't
> catch this case because cc_platform_has(CC_ATTR_HOST_MEM_ENCRYPT)
> returns "false" in a Hyper-V guest. In the Hyper-V guest, it's only
> cc_platform_has(CC_ATTR_GUEST_MEM_ENCRYPT) that returns "true". I'm
> looking more closely at the meaning of the CC_ATTR_* values, and it may
> be that Hyper-V should also return "true" for CC_ATTR_MEM_ENCRYPT,
> but I don't think CC_ATTR_HOST_MEM_ENCRYPT should return "true".
Ok, I assumed that CC_ATTR_HOST_MEM_ENCRYPT returned true in this case.
I guess we just need to check for CC_ATTR_GUEST_MEM_ENCRYPT as well
there?
^ permalink raw reply [flat|nested] 118+ messages in thread
* Re: [PATCH 08/11] swiotlb: make the swiotlb_init interface more useful
@ 2022-02-28 11:30 ` Christoph Hellwig
0 siblings, 0 replies; 118+ messages in thread
From: Christoph Hellwig @ 2022-02-28 11:30 UTC (permalink / raw)
To: Michael Kelley (LINUX)
Cc: linux-hyperv, x86, linux-ia64, linux-pci, linux-riscv,
Christoph Hellwig, linux-s390, Stefano Stabellini,
Konrad Rzeszutek Wilk, tboot-devel, xen-devel, David Woodhouse,
Anshuman Khandual, Boris Ostrovsky, linux-arm-kernel,
Juergen Gross, linuxppc-dev, linux-mips, iommu, Robin Murphy
On Mon, Feb 28, 2022 at 02:53:39AM +0000, Michael Kelley (LINUX) wrote:
> From: Christoph Hellwig <hch@lst.de> Sent: Sunday, February 27, 2022 6:31 AM
> >
> > Pass a bool to pass if swiotlb needs to be enabled based on the
> > addressing needs and replace the verbose argument with a set of
> > flags, including one to force enable bounce buffering.
> >
> > Note that this patch removes the possibility to force xen-swiotlb
> > use using swiotlb=force on the command line on x86 (arm and arm64
> > never supported that), but this interface will be restored shortly.
> >
> > Signed-off-by: Christoph Hellwig <hch@lst.de>
> > ---
> > arch/arm/mm/init.c | 6 +----
> > arch/arm64/mm/init.c | 6 +----
> > arch/ia64/mm/init.c | 4 +--
> > arch/mips/cavium-octeon/dma-octeon.c | 2 +-
> > arch/mips/loongson64/dma.c | 2 +-
> > arch/mips/sibyte/common/dma.c | 2 +-
> > arch/powerpc/include/asm/swiotlb.h | 1 +
> > arch/powerpc/mm/mem.c | 3 ++-
> > arch/powerpc/platforms/pseries/setup.c | 3 ---
> > arch/riscv/mm/init.c | 8 +-----
> > arch/s390/mm/init.c | 3 +--
> > arch/x86/kernel/cpu/mshyperv.c | 8 ------
> > arch/x86/kernel/pci-dma.c | 15 ++++++-----
> > arch/x86/mm/mem_encrypt_amd.c | 3 ---
> > drivers/xen/swiotlb-xen.c | 4 +--
> > include/linux/swiotlb.h | 15 ++++++-----
> > include/trace/events/swiotlb.h | 29 ++++++++-------------
> > kernel/dma/swiotlb.c | 35 ++++++++++++++------------
> > 18 files changed, 56 insertions(+), 93 deletions(-)
>
> [snip]
>
> >
> > diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c
> > index 5a99f993e6392..568274917f1cd 100644
> > --- a/arch/x86/kernel/cpu/mshyperv.c
> > +++ b/arch/x86/kernel/cpu/mshyperv.c
> > @@ -336,14 +336,6 @@ static void __init ms_hyperv_init_platform(void)
> > swiotlb_unencrypted_base = ms_hyperv.shared_gpa_boundary;
> > #endif
> > }
> > -
> > -#ifdef CONFIG_SWIOTLB
> > - /*
> > - * Enable swiotlb force mode in Isolation VM to
> > - * use swiotlb bounce buffer for dma transaction.
> > - */
> > - swiotlb_force = SWIOTLB_FORCE;
> > -#endif
>
> With this code removed, it's not clear to me what forces the use of the
> swiotlb in a Hyper-V isolated VM. The code in pci_swiotlb_detect_4g() doesn't
> catch this case because cc_platform_has(CC_ATTR_HOST_MEM_ENCRYPT)
> returns "false" in a Hyper-V guest. In the Hyper-V guest, it's only
> cc_platform_has(CC_ATTR_GUEST_MEM_ENCRYPT) that returns "true". I'm
> looking more closely at the meaning of the CC_ATTR_* values, and it may
> be that Hyper-V should also return "true" for CC_ATTR_MEM_ENCRYPT,
> but I don't think CC_ATTR_HOST_MEM_ENCRYPT should return "true".
Ok, I assumed that CC_ATTR_HOST_MEM_ENCRYPT returned true in this case.
I guess we just need to check for CC_ATTR_GUEST_MEM_ENCRYPT as well
there?
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply [flat|nested] 118+ messages in thread
* Re: [PATCH 08/11] swiotlb: make the swiotlb_init interface more useful
@ 2022-02-28 11:30 ` Christoph Hellwig
0 siblings, 0 replies; 118+ messages in thread
From: Christoph Hellwig @ 2022-02-28 11:30 UTC (permalink / raw)
To: Michael Kelley (LINUX)
Cc: Christoph Hellwig, iommu, x86, Anshuman Khandual,
Konrad Rzeszutek Wilk, Stefano Stabellini, Boris Ostrovsky,
Juergen Gross, Joerg Roedel, David Woodhouse, Lu Baolu,
Robin Murphy, linux-arm-kernel, xen-devel, linux-ia64,
linux-mips, linuxppc-dev, linux-riscv, linux-s390, linux-hyperv,
tboot-devel, linux-pci
On Mon, Feb 28, 2022 at 02:53:39AM +0000, Michael Kelley (LINUX) wrote:
> From: Christoph Hellwig <hch@lst.de> Sent: Sunday, February 27, 2022 6:31 AM
> >
> > Pass a bool to pass if swiotlb needs to be enabled based on the
> > addressing needs and replace the verbose argument with a set of
> > flags, including one to force enable bounce buffering.
> >
> > Note that this patch removes the possibility to force xen-swiotlb
> > use using swiotlb=force on the command line on x86 (arm and arm64
> > never supported that), but this interface will be restored shortly.
> >
> > Signed-off-by: Christoph Hellwig <hch@lst.de>
> > ---
> > arch/arm/mm/init.c | 6 +----
> > arch/arm64/mm/init.c | 6 +----
> > arch/ia64/mm/init.c | 4 +--
> > arch/mips/cavium-octeon/dma-octeon.c | 2 +-
> > arch/mips/loongson64/dma.c | 2 +-
> > arch/mips/sibyte/common/dma.c | 2 +-
> > arch/powerpc/include/asm/swiotlb.h | 1 +
> > arch/powerpc/mm/mem.c | 3 ++-
> > arch/powerpc/platforms/pseries/setup.c | 3 ---
> > arch/riscv/mm/init.c | 8 +-----
> > arch/s390/mm/init.c | 3 +--
> > arch/x86/kernel/cpu/mshyperv.c | 8 ------
> > arch/x86/kernel/pci-dma.c | 15 ++++++-----
> > arch/x86/mm/mem_encrypt_amd.c | 3 ---
> > drivers/xen/swiotlb-xen.c | 4 +--
> > include/linux/swiotlb.h | 15 ++++++-----
> > include/trace/events/swiotlb.h | 29 ++++++++-------------
> > kernel/dma/swiotlb.c | 35 ++++++++++++++------------
> > 18 files changed, 56 insertions(+), 93 deletions(-)
>
> [snip]
>
> >
> > diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c
> > index 5a99f993e6392..568274917f1cd 100644
> > --- a/arch/x86/kernel/cpu/mshyperv.c
> > +++ b/arch/x86/kernel/cpu/mshyperv.c
> > @@ -336,14 +336,6 @@ static void __init ms_hyperv_init_platform(void)
> > swiotlb_unencrypted_base = ms_hyperv.shared_gpa_boundary;
> > #endif
> > }
> > -
> > -#ifdef CONFIG_SWIOTLB
> > - /*
> > - * Enable swiotlb force mode in Isolation VM to
> > - * use swiotlb bounce buffer for dma transaction.
> > - */
> > - swiotlb_force = SWIOTLB_FORCE;
> > -#endif
>
> With this code removed, it's not clear to me what forces the use of the
> swiotlb in a Hyper-V isolated VM. The code in pci_swiotlb_detect_4g() doesn't
> catch this case because cc_platform_has(CC_ATTR_HOST_MEM_ENCRYPT)
> returns "false" in a Hyper-V guest. In the Hyper-V guest, it's only
> cc_platform_has(CC_ATTR_GUEST_MEM_ENCRYPT) that returns "true". I'm
> looking more closely at the meaning of the CC_ATTR_* values, and it may
> be that Hyper-V should also return "true" for CC_ATTR_MEM_ENCRYPT,
> but I don't think CC_ATTR_HOST_MEM_ENCRYPT should return "true".
Ok, I assumed that CC_ATTR_HOST_MEM_ENCRYPT returned true in this case.
I guess we just need to check for CC_ATTR_GUEST_MEM_ENCRYPT as well
there?
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 118+ messages in thread
* Re: [PATCH 08/11] swiotlb: make the swiotlb_init interface more useful
@ 2022-02-28 11:30 ` Christoph Hellwig
0 siblings, 0 replies; 118+ messages in thread
From: Christoph Hellwig @ 2022-02-28 11:30 UTC (permalink / raw)
To: Michael Kelley (LINUX)
Cc: Christoph Hellwig, iommu, x86, Anshuman Khandual,
Konrad Rzeszutek Wilk, Stefano Stabellini, Boris Ostrovsky,
Juergen Gross, Joerg Roedel, David Woodhouse, Lu Baolu,
Robin Murphy, linux-arm-kernel, xen-devel, linux-ia64,
linux-mips, linuxppc-dev, linux-riscv, linux-s390, linux-hyperv,
tboot-devel, linux-pci
On Mon, Feb 28, 2022 at 02:53:39AM +0000, Michael Kelley (LINUX) wrote:
> From: Christoph Hellwig <hch@lst.de> Sent: Sunday, February 27, 2022 6:31 AM
> >
> > Pass a bool to pass if swiotlb needs to be enabled based on the
> > addressing needs and replace the verbose argument with a set of
> > flags, including one to force enable bounce buffering.
> >
> > Note that this patch removes the possibility to force xen-swiotlb
> > use using swiotlb=force on the command line on x86 (arm and arm64
> > never supported that), but this interface will be restored shortly.
> >
> > Signed-off-by: Christoph Hellwig <hch@lst.de>
> > ---
> > arch/arm/mm/init.c | 6 +----
> > arch/arm64/mm/init.c | 6 +----
> > arch/ia64/mm/init.c | 4 +--
> > arch/mips/cavium-octeon/dma-octeon.c | 2 +-
> > arch/mips/loongson64/dma.c | 2 +-
> > arch/mips/sibyte/common/dma.c | 2 +-
> > arch/powerpc/include/asm/swiotlb.h | 1 +
> > arch/powerpc/mm/mem.c | 3 ++-
> > arch/powerpc/platforms/pseries/setup.c | 3 ---
> > arch/riscv/mm/init.c | 8 +-----
> > arch/s390/mm/init.c | 3 +--
> > arch/x86/kernel/cpu/mshyperv.c | 8 ------
> > arch/x86/kernel/pci-dma.c | 15 ++++++-----
> > arch/x86/mm/mem_encrypt_amd.c | 3 ---
> > drivers/xen/swiotlb-xen.c | 4 +--
> > include/linux/swiotlb.h | 15 ++++++-----
> > include/trace/events/swiotlb.h | 29 ++++++++-------------
> > kernel/dma/swiotlb.c | 35 ++++++++++++++------------
> > 18 files changed, 56 insertions(+), 93 deletions(-)
>
> [snip]
>
> >
> > diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c
> > index 5a99f993e6392..568274917f1cd 100644
> > --- a/arch/x86/kernel/cpu/mshyperv.c
> > +++ b/arch/x86/kernel/cpu/mshyperv.c
> > @@ -336,14 +336,6 @@ static void __init ms_hyperv_init_platform(void)
> > swiotlb_unencrypted_base = ms_hyperv.shared_gpa_boundary;
> > #endif
> > }
> > -
> > -#ifdef CONFIG_SWIOTLB
> > - /*
> > - * Enable swiotlb force mode in Isolation VM to
> > - * use swiotlb bounce buffer for dma transaction.
> > - */
> > - swiotlb_force = SWIOTLB_FORCE;
> > -#endif
>
> With this code removed, it's not clear to me what forces the use of the
> swiotlb in a Hyper-V isolated VM. The code in pci_swiotlb_detect_4g() doesn't
> catch this case because cc_platform_has(CC_ATTR_HOST_MEM_ENCRYPT)
> returns "false" in a Hyper-V guest. In the Hyper-V guest, it's only
> cc_platform_has(CC_ATTR_GUEST_MEM_ENCRYPT) that returns "true". I'm
> looking more closely at the meaning of the CC_ATTR_* values, and it may
> be that Hyper-V should also return "true" for CC_ATTR_MEM_ENCRYPT,
> but I don't think CC_ATTR_HOST_MEM_ENCRYPT should return "true".
Ok, I assumed that CC_ATTR_HOST_MEM_ENCRYPT returned true in this case.
I guess we just need to check for CC_ATTR_GUEST_MEM_ENCRYPT as well
there?
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 118+ messages in thread
* Re: [PATCH 08/11] swiotlb: make the swiotlb_init interface more useful
@ 2022-02-28 11:30 ` Christoph Hellwig
0 siblings, 0 replies; 118+ messages in thread
From: Christoph Hellwig @ 2022-02-28 11:30 UTC (permalink / raw)
To: Michael Kelley (LINUX)
Cc: Christoph Hellwig, iommu, x86, Anshuman Khandual,
Konrad Rzeszutek Wilk, Stefano Stabellini, Boris Ostrovsky,
Juergen Gross, Joerg Roedel, David Woodhouse, Lu Baolu,
Robin Murphy, linux-arm-kernel, xen-devel, linux-ia64,
linux-mips, linuxppc-dev, linux-riscv, linux-s390, linux-hyperv,
tboot-devel, linux-pci
On Mon, Feb 28, 2022 at 02:53:39AM +0000, Michael Kelley (LINUX) wrote:
> From: Christoph Hellwig <hch@lst.de> Sent: Sunday, February 27, 2022 6:31 AM
> >
> > Pass a bool to pass if swiotlb needs to be enabled based on the
> > addressing needs and replace the verbose argument with a set of
> > flags, including one to force enable bounce buffering.
> >
> > Note that this patch removes the possibility to force xen-swiotlb
> > use using swiotlb=force on the command line on x86 (arm and arm64
> > never supported that), but this interface will be restored shortly.
> >
> > Signed-off-by: Christoph Hellwig <hch@lst.de>
> > ---
> > arch/arm/mm/init.c | 6 +----
> > arch/arm64/mm/init.c | 6 +----
> > arch/ia64/mm/init.c | 4 +--
> > arch/mips/cavium-octeon/dma-octeon.c | 2 +-
> > arch/mips/loongson64/dma.c | 2 +-
> > arch/mips/sibyte/common/dma.c | 2 +-
> > arch/powerpc/include/asm/swiotlb.h | 1 +
> > arch/powerpc/mm/mem.c | 3 ++-
> > arch/powerpc/platforms/pseries/setup.c | 3 ---
> > arch/riscv/mm/init.c | 8 +-----
> > arch/s390/mm/init.c | 3 +--
> > arch/x86/kernel/cpu/mshyperv.c | 8 ------
> > arch/x86/kernel/pci-dma.c | 15 ++++++-----
> > arch/x86/mm/mem_encrypt_amd.c | 3 ---
> > drivers/xen/swiotlb-xen.c | 4 +--
> > include/linux/swiotlb.h | 15 ++++++-----
> > include/trace/events/swiotlb.h | 29 ++++++++-------------
> > kernel/dma/swiotlb.c | 35 ++++++++++++++------------
> > 18 files changed, 56 insertions(+), 93 deletions(-)
>
> [snip]
>
> >
> > diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c
> > index 5a99f993e6392..568274917f1cd 100644
> > --- a/arch/x86/kernel/cpu/mshyperv.c
> > +++ b/arch/x86/kernel/cpu/mshyperv.c
> > @@ -336,14 +336,6 @@ static void __init ms_hyperv_init_platform(void)
> > swiotlb_unencrypted_base = ms_hyperv.shared_gpa_boundary;
> > #endif
> > }
> > -
> > -#ifdef CONFIG_SWIOTLB
> > - /*
> > - * Enable swiotlb force mode in Isolation VM to
> > - * use swiotlb bounce buffer for dma transaction.
> > - */
> > - swiotlb_force = SWIOTLB_FORCE;
> > -#endif
>
> With this code removed, it's not clear to me what forces the use of the
> swiotlb in a Hyper-V isolated VM. The code in pci_swiotlb_detect_4g() doesn't
> catch this case because cc_platform_has(CC_ATTR_HOST_MEM_ENCRYPT)
> returns "false" in a Hyper-V guest. In the Hyper-V guest, it's only
> cc_platform_has(CC_ATTR_GUEST_MEM_ENCRYPT) that returns "true". I'm
> looking more closely at the meaning of the CC_ATTR_* values, and it may
> be that Hyper-V should also return "true" for CC_ATTR_MEM_ENCRYPT,
> but I don't think CC_ATTR_HOST_MEM_ENCRYPT should return "true".
Ok, I assumed that CC_ATTR_HOST_MEM_ENCRYPT returned true in this case.
I guess we just need to check for CC_ATTR_GUEST_MEM_ENCRYPT as well
there?
^ permalink raw reply [flat|nested] 118+ messages in thread
* RE: [PATCH 08/11] swiotlb: make the swiotlb_init interface more useful
2022-02-28 11:30 ` Christoph Hellwig
` (2 preceding siblings ...)
(?)
@ 2022-02-28 17:21 ` Michael Kelley (LINUX) via iommu
-1 siblings, 0 replies; 118+ messages in thread
From: Michael Kelley (LINUX) @ 2022-02-28 17:21 UTC (permalink / raw)
To: Christoph Hellwig
Cc: iommu, x86, Anshuman Khandual, Konrad Rzeszutek Wilk,
Stefano Stabellini, Boris Ostrovsky, Juergen Gross, Joerg Roedel,
David Woodhouse, Lu Baolu, Robin Murphy, linux-arm-kernel,
xen-devel, linux-ia64, linux-mips, linuxppc-dev, linux-riscv,
linux-s390, linux-hyperv, tboot-devel, linux-pci
From: Christoph Hellwig <hch@lst.de> Sent: Monday, February 28, 2022 3:31 AM
>
> On Mon, Feb 28, 2022 at 02:53:39AM +0000, Michael Kelley (LINUX) wrote:
> > From: Christoph Hellwig <hch@lst.de> Sent: Sunday, February 27, 2022 6:31 AM
> > >
> > > Pass a bool to pass if swiotlb needs to be enabled based on the
> > > addressing needs and replace the verbose argument with a set of
> > > flags, including one to force enable bounce buffering.
> > >
> > > Note that this patch removes the possibility to force xen-swiotlb
> > > use using swiotlb=force on the command line on x86 (arm and arm64
> > > never supported that), but this interface will be restored shortly.
> > >
> > > Signed-off-by: Christoph Hellwig <hch@lst.de>
> > > ---
> > > arch/arm/mm/init.c | 6 +----
> > > arch/arm64/mm/init.c | 6 +----
> > > arch/ia64/mm/init.c | 4 +--
> > > arch/mips/cavium-octeon/dma-octeon.c | 2 +-
> > > arch/mips/loongson64/dma.c | 2 +-
> > > arch/mips/sibyte/common/dma.c | 2 +-
> > > arch/powerpc/include/asm/swiotlb.h | 1 +
> > > arch/powerpc/mm/mem.c | 3 ++-
> > > arch/powerpc/platforms/pseries/setup.c | 3 ---
> > > arch/riscv/mm/init.c | 8 +-----
> > > arch/s390/mm/init.c | 3 +--
> > > arch/x86/kernel/cpu/mshyperv.c | 8 ------
> > > arch/x86/kernel/pci-dma.c | 15 ++++++-----
> > > arch/x86/mm/mem_encrypt_amd.c | 3 ---
> > > drivers/xen/swiotlb-xen.c | 4 +--
> > > include/linux/swiotlb.h | 15 ++++++-----
> > > include/trace/events/swiotlb.h | 29 ++++++++-------------
> > > kernel/dma/swiotlb.c | 35 ++++++++++++++------------
> > > 18 files changed, 56 insertions(+), 93 deletions(-)
> >
> > [snip]
> >
> > >
> > > diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c
> > > index 5a99f993e6392..568274917f1cd 100644
> > > --- a/arch/x86/kernel/cpu/mshyperv.c
> > > +++ b/arch/x86/kernel/cpu/mshyperv.c
> > > @@ -336,14 +336,6 @@ static void __init ms_hyperv_init_platform(void)
> > > swiotlb_unencrypted_base =
> ms_hyperv.shared_gpa_boundary;
> > > #endif
> > > }
> > > -
> > > -#ifdef CONFIG_SWIOTLB
> > > - /*
> > > - * Enable swiotlb force mode in Isolation VM to
> > > - * use swiotlb bounce buffer for dma transaction.
> > > - */
> > > - swiotlb_force = SWIOTLB_FORCE;
> > > -#endif
> >
> > With this code removed, it's not clear to me what forces the use of the
> > swiotlb in a Hyper-V isolated VM. The code in pci_swiotlb_detect_4g() doesn't
> > catch this case because cc_platform_has(CC_ATTR_HOST_MEM_ENCRYPT)
> > returns "false" in a Hyper-V guest. In the Hyper-V guest, it's only
> > cc_platform_has(CC_ATTR_GUEST_MEM_ENCRYPT) that returns "true". I'm
> > looking more closely at the meaning of the CC_ATTR_* values, and it may
> > be that Hyper-V should also return "true" for CC_ATTR_MEM_ENCRYPT,
> > but I don't think CC_ATTR_HOST_MEM_ENCRYPT should return "true".
>
> Ok, I assumed that CC_ATTR_HOST_MEM_ENCRYPT returned true in this case.
> I guess we just need to check for CC_ATTR_GUEST_MEM_ENCRYPT as well
> there?
I'm unsure.
The comments for CC_ATTR_HOST_MEM_ENCRYPT indicates that it is for
SME. The comments for both CC_ATTR_MEM_ENCRYPT and
CC_ATTR_GUEST_MEM_ENCRYPT mention SEV and SEV-ES (and presumably
SEV-SNP). But I haven't looked at the details of the core SNP patches from
the AMD folks. I'd say that they need to weigh in on the right approach
here that will work for both SME and the various SEV flavors, and then
hopefully the Hyper-V case will fit in.
Michael
^ permalink raw reply [flat|nested] 118+ messages in thread
* RE: [PATCH 08/11] swiotlb: make the swiotlb_init interface more useful
@ 2022-02-28 17:21 ` Michael Kelley (LINUX) via iommu
0 siblings, 0 replies; 118+ messages in thread
From: Michael Kelley (LINUX) via iommu @ 2022-02-28 17:21 UTC (permalink / raw)
To: Christoph Hellwig
Cc: Juergen Gross, linux-s390, linux-hyperv, Stefano Stabellini,
linux-ia64, Anshuman Khandual, Konrad Rzeszutek Wilk,
Robin Murphy, x86, linux-mips, linuxppc-dev, iommu, tboot-devel,
linux-pci, xen-devel, Boris Ostrovsky, David Woodhouse,
linux-riscv, linux-arm-kernel
From: Christoph Hellwig <hch@lst.de> Sent: Monday, February 28, 2022 3:31 AM
>
> On Mon, Feb 28, 2022 at 02:53:39AM +0000, Michael Kelley (LINUX) wrote:
> > From: Christoph Hellwig <hch@lst.de> Sent: Sunday, February 27, 2022 6:31 AM
> > >
> > > Pass a bool to pass if swiotlb needs to be enabled based on the
> > > addressing needs and replace the verbose argument with a set of
> > > flags, including one to force enable bounce buffering.
> > >
> > > Note that this patch removes the possibility to force xen-swiotlb
> > > use using swiotlb=force on the command line on x86 (arm and arm64
> > > never supported that), but this interface will be restored shortly.
> > >
> > > Signed-off-by: Christoph Hellwig <hch@lst.de>
> > > ---
> > > arch/arm/mm/init.c | 6 +----
> > > arch/arm64/mm/init.c | 6 +----
> > > arch/ia64/mm/init.c | 4 +--
> > > arch/mips/cavium-octeon/dma-octeon.c | 2 +-
> > > arch/mips/loongson64/dma.c | 2 +-
> > > arch/mips/sibyte/common/dma.c | 2 +-
> > > arch/powerpc/include/asm/swiotlb.h | 1 +
> > > arch/powerpc/mm/mem.c | 3 ++-
> > > arch/powerpc/platforms/pseries/setup.c | 3 ---
> > > arch/riscv/mm/init.c | 8 +-----
> > > arch/s390/mm/init.c | 3 +--
> > > arch/x86/kernel/cpu/mshyperv.c | 8 ------
> > > arch/x86/kernel/pci-dma.c | 15 ++++++-----
> > > arch/x86/mm/mem_encrypt_amd.c | 3 ---
> > > drivers/xen/swiotlb-xen.c | 4 +--
> > > include/linux/swiotlb.h | 15 ++++++-----
> > > include/trace/events/swiotlb.h | 29 ++++++++-------------
> > > kernel/dma/swiotlb.c | 35 ++++++++++++++------------
> > > 18 files changed, 56 insertions(+), 93 deletions(-)
> >
> > [snip]
> >
> > >
> > > diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c
> > > index 5a99f993e6392..568274917f1cd 100644
> > > --- a/arch/x86/kernel/cpu/mshyperv.c
> > > +++ b/arch/x86/kernel/cpu/mshyperv.c
> > > @@ -336,14 +336,6 @@ static void __init ms_hyperv_init_platform(void)
> > > swiotlb_unencrypted_base =
> ms_hyperv.shared_gpa_boundary;
> > > #endif
> > > }
> > > -
> > > -#ifdef CONFIG_SWIOTLB
> > > - /*
> > > - * Enable swiotlb force mode in Isolation VM to
> > > - * use swiotlb bounce buffer for dma transaction.
> > > - */
> > > - swiotlb_force = SWIOTLB_FORCE;
> > > -#endif
> >
> > With this code removed, it's not clear to me what forces the use of the
> > swiotlb in a Hyper-V isolated VM. The code in pci_swiotlb_detect_4g() doesn't
> > catch this case because cc_platform_has(CC_ATTR_HOST_MEM_ENCRYPT)
> > returns "false" in a Hyper-V guest. In the Hyper-V guest, it's only
> > cc_platform_has(CC_ATTR_GUEST_MEM_ENCRYPT) that returns "true". I'm
> > looking more closely at the meaning of the CC_ATTR_* values, and it may
> > be that Hyper-V should also return "true" for CC_ATTR_MEM_ENCRYPT,
> > but I don't think CC_ATTR_HOST_MEM_ENCRYPT should return "true".
>
> Ok, I assumed that CC_ATTR_HOST_MEM_ENCRYPT returned true in this case.
> I guess we just need to check for CC_ATTR_GUEST_MEM_ENCRYPT as well
> there?
I'm unsure.
The comments for CC_ATTR_HOST_MEM_ENCRYPT indicates that it is for
SME. The comments for both CC_ATTR_MEM_ENCRYPT and
CC_ATTR_GUEST_MEM_ENCRYPT mention SEV and SEV-ES (and presumably
SEV-SNP). But I haven't looked at the details of the core SNP patches from
the AMD folks. I'd say that they need to weigh in on the right approach
here that will work for both SME and the various SEV flavors, and then
hopefully the Hyper-V case will fit in.
Michael
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply [flat|nested] 118+ messages in thread
* RE: [PATCH 08/11] swiotlb: make the swiotlb_init interface more useful
@ 2022-02-28 17:21 ` Michael Kelley (LINUX) via iommu
0 siblings, 0 replies; 118+ messages in thread
From: Michael Kelley (LINUX) @ 2022-02-28 17:21 UTC (permalink / raw)
To: Christoph Hellwig
Cc: Juergen Gross, linux-s390, linux-hyperv, Stefano Stabellini,
linux-ia64, Anshuman Khandual, Konrad Rzeszutek Wilk,
Robin Murphy, Joerg Roedel, x86, linux-mips, linuxppc-dev, iommu,
tboot-devel, linux-pci, xen-devel, Boris Ostrovsky,
David Woodhouse, linux-riscv, linux-arm-kernel, Lu Baolu
From: Christoph Hellwig <hch@lst.de> Sent: Monday, February 28, 2022 3:31 AM
>
> On Mon, Feb 28, 2022 at 02:53:39AM +0000, Michael Kelley (LINUX) wrote:
> > From: Christoph Hellwig <hch@lst.de> Sent: Sunday, February 27, 2022 6:31 AM
> > >
> > > Pass a bool to pass if swiotlb needs to be enabled based on the
> > > addressing needs and replace the verbose argument with a set of
> > > flags, including one to force enable bounce buffering.
> > >
> > > Note that this patch removes the possibility to force xen-swiotlb
> > > use using swiotlb=force on the command line on x86 (arm and arm64
> > > never supported that), but this interface will be restored shortly.
> > >
> > > Signed-off-by: Christoph Hellwig <hch@lst.de>
> > > ---
> > > arch/arm/mm/init.c | 6 +----
> > > arch/arm64/mm/init.c | 6 +----
> > > arch/ia64/mm/init.c | 4 +--
> > > arch/mips/cavium-octeon/dma-octeon.c | 2 +-
> > > arch/mips/loongson64/dma.c | 2 +-
> > > arch/mips/sibyte/common/dma.c | 2 +-
> > > arch/powerpc/include/asm/swiotlb.h | 1 +
> > > arch/powerpc/mm/mem.c | 3 ++-
> > > arch/powerpc/platforms/pseries/setup.c | 3 ---
> > > arch/riscv/mm/init.c | 8 +-----
> > > arch/s390/mm/init.c | 3 +--
> > > arch/x86/kernel/cpu/mshyperv.c | 8 ------
> > > arch/x86/kernel/pci-dma.c | 15 ++++++-----
> > > arch/x86/mm/mem_encrypt_amd.c | 3 ---
> > > drivers/xen/swiotlb-xen.c | 4 +--
> > > include/linux/swiotlb.h | 15 ++++++-----
> > > include/trace/events/swiotlb.h | 29 ++++++++-------------
> > > kernel/dma/swiotlb.c | 35 ++++++++++++++------------
> > > 18 files changed, 56 insertions(+), 93 deletions(-)
> >
> > [snip]
> >
> > >
> > > diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c
> > > index 5a99f993e6392..568274917f1cd 100644
> > > --- a/arch/x86/kernel/cpu/mshyperv.c
> > > +++ b/arch/x86/kernel/cpu/mshyperv.c
> > > @@ -336,14 +336,6 @@ static void __init ms_hyperv_init_platform(void)
> > > swiotlb_unencrypted_base =
> ms_hyperv.shared_gpa_boundary;
> > > #endif
> > > }
> > > -
> > > -#ifdef CONFIG_SWIOTLB
> > > - /*
> > > - * Enable swiotlb force mode in Isolation VM to
> > > - * use swiotlb bounce buffer for dma transaction.
> > > - */
> > > - swiotlb_force = SWIOTLB_FORCE;
> > > -#endif
> >
> > With this code removed, it's not clear to me what forces the use of the
> > swiotlb in a Hyper-V isolated VM. The code in pci_swiotlb_detect_4g() doesn't
> > catch this case because cc_platform_has(CC_ATTR_HOST_MEM_ENCRYPT)
> > returns "false" in a Hyper-V guest. In the Hyper-V guest, it's only
> > cc_platform_has(CC_ATTR_GUEST_MEM_ENCRYPT) that returns "true". I'm
> > looking more closely at the meaning of the CC_ATTR_* values, and it may
> > be that Hyper-V should also return "true" for CC_ATTR_MEM_ENCRYPT,
> > but I don't think CC_ATTR_HOST_MEM_ENCRYPT should return "true".
>
> Ok, I assumed that CC_ATTR_HOST_MEM_ENCRYPT returned true in this case.
> I guess we just need to check for CC_ATTR_GUEST_MEM_ENCRYPT as well
> there?
I'm unsure.
The comments for CC_ATTR_HOST_MEM_ENCRYPT indicates that it is for
SME. The comments for both CC_ATTR_MEM_ENCRYPT and
CC_ATTR_GUEST_MEM_ENCRYPT mention SEV and SEV-ES (and presumably
SEV-SNP). But I haven't looked at the details of the core SNP patches from
the AMD folks. I'd say that they need to weigh in on the right approach
here that will work for both SME and the various SEV flavors, and then
hopefully the Hyper-V case will fit in.
Michael
^ permalink raw reply [flat|nested] 118+ messages in thread
* RE: [PATCH 08/11] swiotlb: make the swiotlb_init interface more useful
@ 2022-02-28 17:21 ` Michael Kelley (LINUX) via iommu
0 siblings, 0 replies; 118+ messages in thread
From: Michael Kelley (LINUX) @ 2022-02-28 17:21 UTC (permalink / raw)
To: Christoph Hellwig
Cc: iommu, x86, Anshuman Khandual, Konrad Rzeszutek Wilk,
Stefano Stabellini, Boris Ostrovsky, Juergen Gross, Joerg Roedel,
David Woodhouse, Lu Baolu, Robin Murphy, linux-arm-kernel,
xen-devel, linux-ia64, linux-mips, linuxppc-dev, linux-riscv,
linux-s390, linux-hyperv, tboot-devel, linux-pci
From: Christoph Hellwig <hch@lst.de> Sent: Monday, February 28, 2022 3:31 AM
>
> On Mon, Feb 28, 2022 at 02:53:39AM +0000, Michael Kelley (LINUX) wrote:
> > From: Christoph Hellwig <hch@lst.de> Sent: Sunday, February 27, 2022 6:31 AM
> > >
> > > Pass a bool to pass if swiotlb needs to be enabled based on the
> > > addressing needs and replace the verbose argument with a set of
> > > flags, including one to force enable bounce buffering.
> > >
> > > Note that this patch removes the possibility to force xen-swiotlb
> > > use using swiotlb=force on the command line on x86 (arm and arm64
> > > never supported that), but this interface will be restored shortly.
> > >
> > > Signed-off-by: Christoph Hellwig <hch@lst.de>
> > > ---
> > > arch/arm/mm/init.c | 6 +----
> > > arch/arm64/mm/init.c | 6 +----
> > > arch/ia64/mm/init.c | 4 +--
> > > arch/mips/cavium-octeon/dma-octeon.c | 2 +-
> > > arch/mips/loongson64/dma.c | 2 +-
> > > arch/mips/sibyte/common/dma.c | 2 +-
> > > arch/powerpc/include/asm/swiotlb.h | 1 +
> > > arch/powerpc/mm/mem.c | 3 ++-
> > > arch/powerpc/platforms/pseries/setup.c | 3 ---
> > > arch/riscv/mm/init.c | 8 +-----
> > > arch/s390/mm/init.c | 3 +--
> > > arch/x86/kernel/cpu/mshyperv.c | 8 ------
> > > arch/x86/kernel/pci-dma.c | 15 ++++++-----
> > > arch/x86/mm/mem_encrypt_amd.c | 3 ---
> > > drivers/xen/swiotlb-xen.c | 4 +--
> > > include/linux/swiotlb.h | 15 ++++++-----
> > > include/trace/events/swiotlb.h | 29 ++++++++-------------
> > > kernel/dma/swiotlb.c | 35 ++++++++++++++------------
> > > 18 files changed, 56 insertions(+), 93 deletions(-)
> >
> > [snip]
> >
> > >
> > > diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c
> > > index 5a99f993e6392..568274917f1cd 100644
> > > --- a/arch/x86/kernel/cpu/mshyperv.c
> > > +++ b/arch/x86/kernel/cpu/mshyperv.c
> > > @@ -336,14 +336,6 @@ static void __init ms_hyperv_init_platform(void)
> > > swiotlb_unencrypted_base =
> ms_hyperv.shared_gpa_boundary;
> > > #endif
> > > }
> > > -
> > > -#ifdef CONFIG_SWIOTLB
> > > - /*
> > > - * Enable swiotlb force mode in Isolation VM to
> > > - * use swiotlb bounce buffer for dma transaction.
> > > - */
> > > - swiotlb_force = SWIOTLB_FORCE;
> > > -#endif
> >
> > With this code removed, it's not clear to me what forces the use of the
> > swiotlb in a Hyper-V isolated VM. The code in pci_swiotlb_detect_4g() doesn't
> > catch this case because cc_platform_has(CC_ATTR_HOST_MEM_ENCRYPT)
> > returns "false" in a Hyper-V guest. In the Hyper-V guest, it's only
> > cc_platform_has(CC_ATTR_GUEST_MEM_ENCRYPT) that returns "true". I'm
> > looking more closely at the meaning of the CC_ATTR_* values, and it may
> > be that Hyper-V should also return "true" for CC_ATTR_MEM_ENCRYPT,
> > but I don't think CC_ATTR_HOST_MEM_ENCRYPT should return "true".
>
> Ok, I assumed that CC_ATTR_HOST_MEM_ENCRYPT returned true in this case.
> I guess we just need to check for CC_ATTR_GUEST_MEM_ENCRYPT as well
> there?
I'm unsure.
The comments for CC_ATTR_HOST_MEM_ENCRYPT indicates that it is for
SME. The comments for both CC_ATTR_MEM_ENCRYPT and
CC_ATTR_GUEST_MEM_ENCRYPT mention SEV and SEV-ES (and presumably
SEV-SNP). But I haven't looked at the details of the core SNP patches from
the AMD folks. I'd say that they need to weigh in on the right approach
here that will work for both SME and the various SEV flavors, and then
hopefully the Hyper-V case will fit in.
Michael
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 118+ messages in thread
* RE: [PATCH 08/11] swiotlb: make the swiotlb_init interface more useful
@ 2022-02-28 17:21 ` Michael Kelley (LINUX) via iommu
0 siblings, 0 replies; 118+ messages in thread
From: Michael Kelley (LINUX) @ 2022-02-28 17:21 UTC (permalink / raw)
To: Christoph Hellwig
Cc: iommu, x86, Anshuman Khandual, Konrad Rzeszutek Wilk,
Stefano Stabellini, Boris Ostrovsky, Juergen Gross, Joerg Roedel,
David Woodhouse, Lu Baolu, Robin Murphy, linux-arm-kernel,
xen-devel, linux-ia64, linux-mips, linuxppc-dev, linux-riscv,
linux-s390, linux-hyperv, tboot-devel, linux-pci
From: Christoph Hellwig <hch@lst.de> Sent: Monday, February 28, 2022 3:31 AM
>
> On Mon, Feb 28, 2022 at 02:53:39AM +0000, Michael Kelley (LINUX) wrote:
> > From: Christoph Hellwig <hch@lst.de> Sent: Sunday, February 27, 2022 6:31 AM
> > >
> > > Pass a bool to pass if swiotlb needs to be enabled based on the
> > > addressing needs and replace the verbose argument with a set of
> > > flags, including one to force enable bounce buffering.
> > >
> > > Note that this patch removes the possibility to force xen-swiotlb
> > > use using swiotlb=force on the command line on x86 (arm and arm64
> > > never supported that), but this interface will be restored shortly.
> > >
> > > Signed-off-by: Christoph Hellwig <hch@lst.de>
> > > ---
> > > arch/arm/mm/init.c | 6 +----
> > > arch/arm64/mm/init.c | 6 +----
> > > arch/ia64/mm/init.c | 4 +--
> > > arch/mips/cavium-octeon/dma-octeon.c | 2 +-
> > > arch/mips/loongson64/dma.c | 2 +-
> > > arch/mips/sibyte/common/dma.c | 2 +-
> > > arch/powerpc/include/asm/swiotlb.h | 1 +
> > > arch/powerpc/mm/mem.c | 3 ++-
> > > arch/powerpc/platforms/pseries/setup.c | 3 ---
> > > arch/riscv/mm/init.c | 8 +-----
> > > arch/s390/mm/init.c | 3 +--
> > > arch/x86/kernel/cpu/mshyperv.c | 8 ------
> > > arch/x86/kernel/pci-dma.c | 15 ++++++-----
> > > arch/x86/mm/mem_encrypt_amd.c | 3 ---
> > > drivers/xen/swiotlb-xen.c | 4 +--
> > > include/linux/swiotlb.h | 15 ++++++-----
> > > include/trace/events/swiotlb.h | 29 ++++++++-------------
> > > kernel/dma/swiotlb.c | 35 ++++++++++++++------------
> > > 18 files changed, 56 insertions(+), 93 deletions(-)
> >
> > [snip]
> >
> > >
> > > diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c
> > > index 5a99f993e6392..568274917f1cd 100644
> > > --- a/arch/x86/kernel/cpu/mshyperv.c
> > > +++ b/arch/x86/kernel/cpu/mshyperv.c
> > > @@ -336,14 +336,6 @@ static void __init ms_hyperv_init_platform(void)
> > > swiotlb_unencrypted_base =
> ms_hyperv.shared_gpa_boundary;
> > > #endif
> > > }
> > > -
> > > -#ifdef CONFIG_SWIOTLB
> > > - /*
> > > - * Enable swiotlb force mode in Isolation VM to
> > > - * use swiotlb bounce buffer for dma transaction.
> > > - */
> > > - swiotlb_force = SWIOTLB_FORCE;
> > > -#endif
> >
> > With this code removed, it's not clear to me what forces the use of the
> > swiotlb in a Hyper-V isolated VM. The code in pci_swiotlb_detect_4g() doesn't
> > catch this case because cc_platform_has(CC_ATTR_HOST_MEM_ENCRYPT)
> > returns "false" in a Hyper-V guest. In the Hyper-V guest, it's only
> > cc_platform_has(CC_ATTR_GUEST_MEM_ENCRYPT) that returns "true". I'm
> > looking more closely at the meaning of the CC_ATTR_* values, and it may
> > be that Hyper-V should also return "true" for CC_ATTR_MEM_ENCRYPT,
> > but I don't think CC_ATTR_HOST_MEM_ENCRYPT should return "true".
>
> Ok, I assumed that CC_ATTR_HOST_MEM_ENCRYPT returned true in this case.
> I guess we just need to check for CC_ATTR_GUEST_MEM_ENCRYPT as well
> there?
I'm unsure.
The comments for CC_ATTR_HOST_MEM_ENCRYPT indicates that it is for
SME. The comments for both CC_ATTR_MEM_ENCRYPT and
CC_ATTR_GUEST_MEM_ENCRYPT mention SEV and SEV-ES (and presumably
SEV-SNP). But I haven't looked at the details of the core SNP patches from
the AMD folks. I'd say that they need to weigh in on the right approach
here that will work for both SME and the various SEV flavors, and then
hopefully the Hyper-V case will fit in.
Michael
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 118+ messages in thread
* Re: [PATCH 07/11] x86: remove the IOMMU table infrastructure
2022-02-27 14:30 ` Christoph Hellwig
` (3 preceding siblings ...)
(?)
@ 2022-03-01 18:20 ` Konrad Rzeszutek Wilk
-1 siblings, 0 replies; 118+ messages in thread
From: Konrad Rzeszutek Wilk @ 2022-03-01 18:20 UTC (permalink / raw)
To: Christoph Hellwig
Cc: iommu, x86, Anshuman Khandual, Stefano Stabellini,
Boris Ostrovsky, Juergen Gross, Joerg Roedel, David Woodhouse,
Lu Baolu, Robin Murphy, linux-arm-kernel, xen-devel, linux-ia64,
linux-mips, linuxppc-dev, linux-riscv, linux-s390, linux-hyperv,
tboot-devel, linux-pci
> -#include <asm/swiotlb.h>
> -
> -/*
> - * History lesson:
> - * The execution chain of IOMMUs in 2.6.36 looks as so:
> - *
> - * [xen-swiotlb]
> - * |
> - * +----[swiotlb *]--+
> - * / | \
> - * / | \
> - * [GART] [Calgary] [Intel VT-d]
> - * /
> - * /
> - * [AMD-Vi]
.. snip..
> - *
> void __init pci_iommu_alloc(void)
> {
> - struct iommu_table_entry *p;
> -
> - sort_iommu_table(__iommu_table, __iommu_table_end);
> - check_iommu_entries(__iommu_table, __iommu_table_end);
> -
> - for (p = __iommu_table; p < __iommu_table_end; p++) {
> - if (p && p->detect && p->detect() > 0) {
> - p->flags |= IOMMU_DETECTED;
> - if (p->early_init)
> - p->early_init();
> - if (p->flags & IOMMU_FINISH_IF_DETECTED)
> - break;
> - }
> + if (xen_pv_domain()) {
> + pci_xen_swiotlb_init();
> + return;
> }
> + pci_swiotlb_detect_4gb();
I think you also need to check for IBM Calgary?
> + gart_iommu_hole_init();
> + amd_iommu_detect();
> + detect_intel_iommu();
> + if (x86_swiotlb_enable)
> + swiotlb_init(0);
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 118+ messages in thread
* Re: [PATCH 07/11] x86: remove the IOMMU table infrastructure
@ 2022-03-01 18:20 ` Konrad Rzeszutek Wilk
0 siblings, 0 replies; 118+ messages in thread
From: Konrad Rzeszutek Wilk @ 2022-03-01 18:20 UTC (permalink / raw)
To: Christoph Hellwig
Cc: Juergen Gross, linux-s390, linux-hyperv, Stefano Stabellini,
linux-ia64, Anshuman Khandual, Robin Murphy, x86, linux-mips,
linuxppc-dev, iommu, tboot-devel, linux-pci, xen-devel,
Boris Ostrovsky, David Woodhouse, linux-riscv, linux-arm-kernel
> -#include <asm/swiotlb.h>
> -
> -/*
> - * History lesson:
> - * The execution chain of IOMMUs in 2.6.36 looks as so:
> - *
> - * [xen-swiotlb]
> - * |
> - * +----[swiotlb *]--+
> - * / | \
> - * / | \
> - * [GART] [Calgary] [Intel VT-d]
> - * /
> - * /
> - * [AMD-Vi]
.. snip..
> - *
> void __init pci_iommu_alloc(void)
> {
> - struct iommu_table_entry *p;
> -
> - sort_iommu_table(__iommu_table, __iommu_table_end);
> - check_iommu_entries(__iommu_table, __iommu_table_end);
> -
> - for (p = __iommu_table; p < __iommu_table_end; p++) {
> - if (p && p->detect && p->detect() > 0) {
> - p->flags |= IOMMU_DETECTED;
> - if (p->early_init)
> - p->early_init();
> - if (p->flags & IOMMU_FINISH_IF_DETECTED)
> - break;
> - }
> + if (xen_pv_domain()) {
> + pci_xen_swiotlb_init();
> + return;
> }
> + pci_swiotlb_detect_4gb();
I think you also need to check for IBM Calgary?
> + gart_iommu_hole_init();
> + amd_iommu_detect();
> + detect_intel_iommu();
> + if (x86_swiotlb_enable)
> + swiotlb_init(0);
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply [flat|nested] 118+ messages in thread
* Re: [PATCH 07/11] x86: remove the IOMMU table infrastructure
@ 2022-03-01 18:20 ` Konrad Rzeszutek Wilk
0 siblings, 0 replies; 118+ messages in thread
From: Konrad Rzeszutek Wilk @ 2022-03-01 18:20 UTC (permalink / raw)
To: Christoph Hellwig
Cc: iommu, x86, Anshuman Khandual, Stefano Stabellini,
Boris Ostrovsky, Juergen Gross, Joerg Roedel, David Woodhouse,
Lu Baolu, Robin Murphy, linux-arm-kernel, xen-devel, linux-ia64,
linux-mips, linuxppc-dev, linux-riscv, linux-s390, linux-hyperv,
tboot-devel, linux-pci
> -#include <asm/swiotlb.h>
> -
> -/*
> - * History lesson:
> - * The execution chain of IOMMUs in 2.6.36 looks as so:
> - *
> - * [xen-swiotlb]
> - * |
> - * +----[swiotlb *]--+
> - * / | \
> - * / | \
> - * [GART] [Calgary] [Intel VT-d]
> - * /
> - * /
> - * [AMD-Vi]
.. snip..
> - *
> void __init pci_iommu_alloc(void)
> {
> - struct iommu_table_entry *p;
> -
> - sort_iommu_table(__iommu_table, __iommu_table_end);
> - check_iommu_entries(__iommu_table, __iommu_table_end);
> -
> - for (p = __iommu_table; p < __iommu_table_end; p++) {
> - if (p && p->detect && p->detect() > 0) {
> - p->flags |= IOMMU_DETECTED;
> - if (p->early_init)
> - p->early_init();
> - if (p->flags & IOMMU_FINISH_IF_DETECTED)
> - break;
> - }
> + if (xen_pv_domain()) {
> + pci_xen_swiotlb_init();
> + return;
> }
> + pci_swiotlb_detect_4gb();
I think you also need to check for IBM Calgary?
> + gart_iommu_hole_init();
> + amd_iommu_detect();
> + detect_intel_iommu();
> + if (x86_swiotlb_enable)
> + swiotlb_init(0);
^ permalink raw reply [flat|nested] 118+ messages in thread
* Re: [PATCH 07/11] x86: remove the IOMMU table infrastructure
@ 2022-03-01 18:20 ` Konrad Rzeszutek Wilk
0 siblings, 0 replies; 118+ messages in thread
From: Konrad Rzeszutek Wilk @ 2022-03-01 18:20 UTC (permalink / raw)
To: Christoph Hellwig
Cc: Juergen Gross, linux-s390, linux-hyperv, Stefano Stabellini,
linux-ia64, Anshuman Khandual, Robin Murphy, Joerg Roedel, x86,
linux-mips, linuxppc-dev, iommu, tboot-devel, linux-pci,
xen-devel, Boris Ostrovsky, David Woodhouse, linux-riscv,
linux-arm-kernel, Lu Baolu
> -#include <asm/swiotlb.h>
> -
> -/*
> - * History lesson:
> - * The execution chain of IOMMUs in 2.6.36 looks as so:
> - *
> - * [xen-swiotlb]
> - * |
> - * +----[swiotlb *]--+
> - * / | \
> - * / | \
> - * [GART] [Calgary] [Intel VT-d]
> - * /
> - * /
> - * [AMD-Vi]
.. snip..
> - *
> void __init pci_iommu_alloc(void)
> {
> - struct iommu_table_entry *p;
> -
> - sort_iommu_table(__iommu_table, __iommu_table_end);
> - check_iommu_entries(__iommu_table, __iommu_table_end);
> -
> - for (p = __iommu_table; p < __iommu_table_end; p++) {
> - if (p && p->detect && p->detect() > 0) {
> - p->flags |= IOMMU_DETECTED;
> - if (p->early_init)
> - p->early_init();
> - if (p->flags & IOMMU_FINISH_IF_DETECTED)
> - break;
> - }
> + if (xen_pv_domain()) {
> + pci_xen_swiotlb_init();
> + return;
> }
> + pci_swiotlb_detect_4gb();
I think you also need to check for IBM Calgary?
> + gart_iommu_hole_init();
> + amd_iommu_detect();
> + detect_intel_iommu();
> + if (x86_swiotlb_enable)
> + swiotlb_init(0);
^ permalink raw reply [flat|nested] 118+ messages in thread
* Re: [PATCH 07/11] x86: remove the IOMMU table infrastructure
@ 2022-03-01 18:20 ` Konrad Rzeszutek Wilk
0 siblings, 0 replies; 118+ messages in thread
From: Konrad Rzeszutek Wilk @ 2022-03-01 18:20 UTC (permalink / raw)
To: Christoph Hellwig
Cc: iommu, x86, Anshuman Khandual, Stefano Stabellini,
Boris Ostrovsky, Juergen Gross, Joerg Roedel, David Woodhouse,
Lu Baolu, Robin Murphy, linux-arm-kernel, xen-devel, linux-ia64,
linux-mips, linuxppc-dev, linux-riscv, linux-s390, linux-hyperv,
tboot-devel, linux-pci
> -#include <asm/swiotlb.h>
> -
> -/*
> - * History lesson:
> - * The execution chain of IOMMUs in 2.6.36 looks as so:
> - *
> - * [xen-swiotlb]
> - * |
> - * +----[swiotlb *]--+
> - * / | \
> - * / | \
> - * [GART] [Calgary] [Intel VT-d]
> - * /
> - * /
> - * [AMD-Vi]
.. snip..
> - *
> void __init pci_iommu_alloc(void)
> {
> - struct iommu_table_entry *p;
> -
> - sort_iommu_table(__iommu_table, __iommu_table_end);
> - check_iommu_entries(__iommu_table, __iommu_table_end);
> -
> - for (p = __iommu_table; p < __iommu_table_end; p++) {
> - if (p && p->detect && p->detect() > 0) {
> - p->flags |= IOMMU_DETECTED;
> - if (p->early_init)
> - p->early_init();
> - if (p->flags & IOMMU_FINISH_IF_DETECTED)
> - break;
> - }
> + if (xen_pv_domain()) {
> + pci_xen_swiotlb_init();
> + return;
> }
> + pci_swiotlb_detect_4gb();
I think you also need to check for IBM Calgary?
> + gart_iommu_hole_init();
> + amd_iommu_detect();
> + detect_intel_iommu();
> + if (x86_swiotlb_enable)
> + swiotlb_init(0);
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 118+ messages in thread
* Re: [PATCH 07/11] x86: remove the IOMMU table infrastructure
@ 2022-03-01 18:20 ` Konrad Rzeszutek Wilk
0 siblings, 0 replies; 118+ messages in thread
From: Konrad Rzeszutek Wilk @ 2022-03-01 18:20 UTC (permalink / raw)
To: Christoph Hellwig
Cc: iommu, x86, Anshuman Khandual, Stefano Stabellini,
Boris Ostrovsky, Juergen Gross, Joerg Roedel, David Woodhouse,
Lu Baolu, Robin Murphy, linux-arm-kernel, xen-devel, linux-ia64,
linux-mips, linuxppc-dev, linux-riscv, linux-s390, linux-hyperv,
tboot-devel, linux-pci
> -#include <asm/swiotlb.h>
> -
> -/*
> - * History lesson:
> - * The execution chain of IOMMUs in 2.6.36 looks as so:
> - *
> - * [xen-swiotlb]
> - * |
> - * +----[swiotlb *]--+
> - * / | \
> - * / | \
> - * [GART] [Calgary] [Intel VT-d]
> - * /
> - * /
> - * [AMD-Vi]
.. snip..
> - *
> void __init pci_iommu_alloc(void)
> {
> - struct iommu_table_entry *p;
> -
> - sort_iommu_table(__iommu_table, __iommu_table_end);
> - check_iommu_entries(__iommu_table, __iommu_table_end);
> -
> - for (p = __iommu_table; p < __iommu_table_end; p++) {
> - if (p && p->detect && p->detect() > 0) {
> - p->flags |= IOMMU_DETECTED;
> - if (p->early_init)
> - p->early_init();
> - if (p->flags & IOMMU_FINISH_IF_DETECTED)
> - break;
> - }
> + if (xen_pv_domain()) {
> + pci_xen_swiotlb_init();
> + return;
> }
> + pci_swiotlb_detect_4gb();
I think you also need to check for IBM Calgary?
> + gart_iommu_hole_init();
> + amd_iommu_detect();
> + detect_intel_iommu();
> + if (x86_swiotlb_enable)
> + swiotlb_init(0);
^ permalink raw reply [flat|nested] 118+ messages in thread
* Re: [PATCH 07/11] x86: remove the IOMMU table infrastructure
2022-03-01 18:20 ` Konrad Rzeszutek Wilk
` (3 preceding siblings ...)
(?)
@ 2022-03-02 8:07 ` Christoph Hellwig
-1 siblings, 0 replies; 118+ messages in thread
From: Christoph Hellwig @ 2022-03-02 8:07 UTC (permalink / raw)
To: Konrad Rzeszutek Wilk
Cc: Christoph Hellwig, iommu, x86, Anshuman Khandual,
Stefano Stabellini, Boris Ostrovsky, Juergen Gross, Joerg Roedel,
David Woodhouse, Lu Baolu, Robin Murphy, linux-arm-kernel,
xen-devel, linux-ia64, linux-mips, linuxppc-dev, linux-riscv,
linux-s390, linux-hyperv, tboot-devel, linux-pci
On Tue, Mar 01, 2022 at 01:20:22PM -0500, Konrad Rzeszutek Wilk wrote:
> I think you also need to check for IBM Calgary?
The IBM Calgary IOMMU support is long gone.
^ permalink raw reply [flat|nested] 118+ messages in thread
* Re: [PATCH 07/11] x86: remove the IOMMU table infrastructure
@ 2022-03-02 8:07 ` Christoph Hellwig
0 siblings, 0 replies; 118+ messages in thread
From: Christoph Hellwig @ 2022-03-02 8:07 UTC (permalink / raw)
To: Konrad Rzeszutek Wilk
Cc: Christoph Hellwig, iommu, x86, Anshuman Khandual,
Stefano Stabellini, Boris Ostrovsky, Juergen Gross, Joerg Roedel,
David Woodhouse, Lu Baolu, Robin Murphy, linux-arm-kernel,
xen-devel, linux-ia64, linux-mips, linuxppc-dev, linux-riscv,
linux-s390, linux-hyperv, tboot-devel, linux-pci
On Tue, Mar 01, 2022 at 01:20:22PM -0500, Konrad Rzeszutek Wilk wrote:
> I think you also need to check for IBM Calgary?
The IBM Calgary IOMMU support is long gone.
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 118+ messages in thread
* Re: [PATCH 07/11] x86: remove the IOMMU table infrastructure
@ 2022-03-02 8:07 ` Christoph Hellwig
0 siblings, 0 replies; 118+ messages in thread
From: Christoph Hellwig @ 2022-03-02 8:07 UTC (permalink / raw)
To: Konrad Rzeszutek Wilk
Cc: Juergen Gross, linux-s390, linux-hyperv, Stefano Stabellini,
linux-ia64, Anshuman Khandual, Robin Murphy, Joerg Roedel, x86,
linux-mips, linux-riscv, linuxppc-dev, iommu, tboot-devel,
linux-pci, xen-devel, Boris Ostrovsky, David Woodhouse,
Christoph Hellwig, linux-arm-kernel, Lu Baolu
On Tue, Mar 01, 2022 at 01:20:22PM -0500, Konrad Rzeszutek Wilk wrote:
> I think you also need to check for IBM Calgary?
The IBM Calgary IOMMU support is long gone.
^ permalink raw reply [flat|nested] 118+ messages in thread
* Re: [PATCH 07/11] x86: remove the IOMMU table infrastructure
@ 2022-03-02 8:07 ` Christoph Hellwig
0 siblings, 0 replies; 118+ messages in thread
From: Christoph Hellwig @ 2022-03-02 8:07 UTC (permalink / raw)
To: Konrad Rzeszutek Wilk
Cc: Juergen Gross, linux-s390, linux-hyperv, Stefano Stabellini,
linux-ia64, Anshuman Khandual, Robin Murphy, x86, linux-mips,
linux-riscv, linuxppc-dev, iommu, tboot-devel, linux-pci,
xen-devel, Boris Ostrovsky, David Woodhouse, Christoph Hellwig,
linux-arm-kernel
On Tue, Mar 01, 2022 at 01:20:22PM -0500, Konrad Rzeszutek Wilk wrote:
> I think you also need to check for IBM Calgary?
The IBM Calgary IOMMU support is long gone.
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply [flat|nested] 118+ messages in thread
* Re: [PATCH 07/11] x86: remove the IOMMU table infrastructure
@ 2022-03-02 8:07 ` Christoph Hellwig
0 siblings, 0 replies; 118+ messages in thread
From: Christoph Hellwig @ 2022-03-02 8:07 UTC (permalink / raw)
To: Konrad Rzeszutek Wilk
Cc: Christoph Hellwig, iommu, x86, Anshuman Khandual,
Stefano Stabellini, Boris Ostrovsky, Juergen Gross, Joerg Roedel,
David Woodhouse, Lu Baolu, Robin Murphy, linux-arm-kernel,
xen-devel, linux-ia64, linux-mips, linuxppc-dev, linux-riscv,
linux-s390, linux-hyperv, tboot-devel, linux-pci
On Tue, Mar 01, 2022 at 01:20:22PM -0500, Konrad Rzeszutek Wilk wrote:
> I think you also need to check for IBM Calgary?
The IBM Calgary IOMMU support is long gone.
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 118+ messages in thread
* Re: [PATCH 07/11] x86: remove the IOMMU table infrastructure
@ 2022-03-02 8:07 ` Christoph Hellwig
0 siblings, 0 replies; 118+ messages in thread
From: Christoph Hellwig @ 2022-03-02 8:07 UTC (permalink / raw)
To: Konrad Rzeszutek Wilk
Cc: Christoph Hellwig, iommu, x86, Anshuman Khandual,
Stefano Stabellini, Boris Ostrovsky, Juergen Gross, Joerg Roedel,
David Woodhouse, Lu Baolu, Robin Murphy, linux-arm-kernel,
xen-devel, linux-ia64, linux-mips, linuxppc-dev, linux-riscv,
linux-s390, linux-hyperv, tboot-devel, linux-pci
On Tue, Mar 01, 2022 at 01:20:22PM -0500, Konrad Rzeszutek Wilk wrote:
> I think you also need to check for IBM Calgary?
The IBM Calgary IOMMU support is long gone.
^ permalink raw reply [flat|nested] 118+ messages in thread
* Re: [PATCH 07/11] x86: remove the IOMMU table infrastructure
2022-02-27 14:30 ` Christoph Hellwig
` (3 preceding siblings ...)
(?)
@ 2022-03-02 9:18 ` Anatoly Pugachev
-1 siblings, 0 replies; 118+ messages in thread
From: Anatoly Pugachev @ 2022-03-02 9:18 UTC (permalink / raw)
To: Christoph Hellwig
Cc: iommu, x86, Anshuman Khandual, Konrad Rzeszutek Wilk,
Stefano Stabellini, Boris Ostrovsky, Juergen Gross, Joerg Roedel,
David Woodhouse, Lu Baolu, Robin Murphy, linux-arm-kernel,
xen-devel, linux-ia64, linux-mips, linuxppc-dev, linux-riscv,
linux-s390, linux-hyperv, tboot-devel, linux-pci
On Sun, Feb 27, 2022 at 7:31 PM Christoph Hellwig <hch@lst.de> wrote:
>
> The IOMMU table tries to separate the different IOMMUs into different
> backends, but actually requires various cross calls.
>
> Rewrite the code to do the generic swiotlb/swiotlb-xen setup directly
> in pci-dma.c and then just call into the IOMMU drivers.
...
> --- a/arch/x86/include/asm/iommu_table.h
> +++ /dev/null
> @@ -1,102 +0,0 @@
> -/* SPDX-License-Identifier: GPL-2.0 */
> -#ifndef _ASM_X86_IOMMU_TABLE_H
> -#define _ASM_X86_IOMMU_TABLE_H
> -
> -#include <asm/swiotlb.h>
> -
> -/*
> - * History lesson:
> - * The execution chain of IOMMUs in 2.6.36 looks as so:
> - *
> - * [xen-swiotlb]
> - * |
> - * +----[swiotlb *]--+
> - * / | \
> - * / | \
> - * [GART] [Calgary] [Intel VT-d]
> - * /
> - * /
> - * [AMD-Vi]
> - *
> - * *: if SWIOTLB detected 'iommu=soft'/'swiotlb=force' it would skip
> - * over the rest of IOMMUs and unconditionally initialize the SWIOTLB.
> - * Also it would surreptitiously initialize set the swiotlb=1 if there were
> - * more than 4GB and if the user did not pass in 'iommu=off'. The swiotlb
> - * flag would be turned off by all IOMMUs except the Calgary one.
> - *
> - * The IOMMU_INIT* macros allow a similar tree (or more complex if desired)
> - * to be built by defining who we depend on.
> - *
> - * And all that needs to be done is to use one of the macros in the IOMMU
> - * and the pci-dma.c will take care of the rest.
> - */
Christoph,
Is it possible to keep documentation comments in source files? Or are
they completely irrelevant now?
Thanks.
^ permalink raw reply [flat|nested] 118+ messages in thread
* Re: [PATCH 07/11] x86: remove the IOMMU table infrastructure
@ 2022-03-02 9:18 ` Anatoly Pugachev
0 siblings, 0 replies; 118+ messages in thread
From: Anatoly Pugachev @ 2022-03-02 9:18 UTC (permalink / raw)
To: Christoph Hellwig
Cc: iommu, x86, Anshuman Khandual, Konrad Rzeszutek Wilk,
Stefano Stabellini, Boris Ostrovsky, Juergen Gross, Joerg Roedel,
David Woodhouse, Lu Baolu, Robin Murphy, linux-arm-kernel,
xen-devel, linux-ia64, linux-mips, linuxppc-dev, linux-riscv,
linux-s390, linux-hyperv, tboot-devel, linux-pci
On Sun, Feb 27, 2022 at 7:31 PM Christoph Hellwig <hch@lst.de> wrote:
>
> The IOMMU table tries to separate the different IOMMUs into different
> backends, but actually requires various cross calls.
>
> Rewrite the code to do the generic swiotlb/swiotlb-xen setup directly
> in pci-dma.c and then just call into the IOMMU drivers.
...
> --- a/arch/x86/include/asm/iommu_table.h
> +++ /dev/null
> @@ -1,102 +0,0 @@
> -/* SPDX-License-Identifier: GPL-2.0 */
> -#ifndef _ASM_X86_IOMMU_TABLE_H
> -#define _ASM_X86_IOMMU_TABLE_H
> -
> -#include <asm/swiotlb.h>
> -
> -/*
> - * History lesson:
> - * The execution chain of IOMMUs in 2.6.36 looks as so:
> - *
> - * [xen-swiotlb]
> - * |
> - * +----[swiotlb *]--+
> - * / | \
> - * / | \
> - * [GART] [Calgary] [Intel VT-d]
> - * /
> - * /
> - * [AMD-Vi]
> - *
> - * *: if SWIOTLB detected 'iommu=soft'/'swiotlb=force' it would skip
> - * over the rest of IOMMUs and unconditionally initialize the SWIOTLB.
> - * Also it would surreptitiously initialize set the swiotlb=1 if there were
> - * more than 4GB and if the user did not pass in 'iommu=off'. The swiotlb
> - * flag would be turned off by all IOMMUs except the Calgary one.
> - *
> - * The IOMMU_INIT* macros allow a similar tree (or more complex if desired)
> - * to be built by defining who we depend on.
> - *
> - * And all that needs to be done is to use one of the macros in the IOMMU
> - * and the pci-dma.c will take care of the rest.
> - */
Christoph,
Is it possible to keep documentation comments in source files? Or are
they completely irrelevant now?
Thanks.
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 118+ messages in thread
* Re: [PATCH 07/11] x86: remove the IOMMU table infrastructure
@ 2022-03-02 9:18 ` Anatoly Pugachev
0 siblings, 0 replies; 118+ messages in thread
From: Anatoly Pugachev @ 2022-03-02 9:18 UTC (permalink / raw)
To: Christoph Hellwig
Cc: Juergen Gross, linux-s390, linux-hyperv, Stefano Stabellini,
linux-ia64, Anshuman Khandual, Konrad Rzeszutek Wilk,
Robin Murphy, Joerg Roedel, x86, linux-mips, linuxppc-dev, iommu,
tboot-devel, linux-pci, xen-devel, Boris Ostrovsky,
David Woodhouse, linux-riscv, linux-arm-kernel, Lu Baolu
On Sun, Feb 27, 2022 at 7:31 PM Christoph Hellwig <hch@lst.de> wrote:
>
> The IOMMU table tries to separate the different IOMMUs into different
> backends, but actually requires various cross calls.
>
> Rewrite the code to do the generic swiotlb/swiotlb-xen setup directly
> in pci-dma.c and then just call into the IOMMU drivers.
...
> --- a/arch/x86/include/asm/iommu_table.h
> +++ /dev/null
> @@ -1,102 +0,0 @@
> -/* SPDX-License-Identifier: GPL-2.0 */
> -#ifndef _ASM_X86_IOMMU_TABLE_H
> -#define _ASM_X86_IOMMU_TABLE_H
> -
> -#include <asm/swiotlb.h>
> -
> -/*
> - * History lesson:
> - * The execution chain of IOMMUs in 2.6.36 looks as so:
> - *
> - * [xen-swiotlb]
> - * |
> - * +----[swiotlb *]--+
> - * / | \
> - * / | \
> - * [GART] [Calgary] [Intel VT-d]
> - * /
> - * /
> - * [AMD-Vi]
> - *
> - * *: if SWIOTLB detected 'iommu=soft'/'swiotlb=force' it would skip
> - * over the rest of IOMMUs and unconditionally initialize the SWIOTLB.
> - * Also it would surreptitiously initialize set the swiotlb=1 if there were
> - * more than 4GB and if the user did not pass in 'iommu=off'. The swiotlb
> - * flag would be turned off by all IOMMUs except the Calgary one.
> - *
> - * The IOMMU_INIT* macros allow a similar tree (or more complex if desired)
> - * to be built by defining who we depend on.
> - *
> - * And all that needs to be done is to use one of the macros in the IOMMU
> - * and the pci-dma.c will take care of the rest.
> - */
Christoph,
Is it possible to keep documentation comments in source files? Or are
they completely irrelevant now?
Thanks.
^ permalink raw reply [flat|nested] 118+ messages in thread
* Re: [PATCH 07/11] x86: remove the IOMMU table infrastructure
@ 2022-03-02 9:18 ` Anatoly Pugachev
0 siblings, 0 replies; 118+ messages in thread
From: Anatoly Pugachev @ 2022-03-02 9:18 UTC (permalink / raw)
To: Christoph Hellwig
Cc: iommu, x86, Anshuman Khandual, Konrad Rzeszutek Wilk,
Stefano Stabellini, Boris Ostrovsky, Juergen Gross, Joerg Roedel,
David Woodhouse, Lu Baolu, Robin Murphy, linux-arm-kernel,
xen-devel, linux-ia64, linux-mips, linuxppc-dev, linux-riscv,
linux-s390, linux-hyperv, tboot-devel, linux-pci
On Sun, Feb 27, 2022 at 7:31 PM Christoph Hellwig <hch@lst.de> wrote:
>
> The IOMMU table tries to separate the different IOMMUs into different
> backends, but actually requires various cross calls.
>
> Rewrite the code to do the generic swiotlb/swiotlb-xen setup directly
> in pci-dma.c and then just call into the IOMMU drivers.
...
> --- a/arch/x86/include/asm/iommu_table.h
> +++ /dev/null
> @@ -1,102 +0,0 @@
> -/* SPDX-License-Identifier: GPL-2.0 */
> -#ifndef _ASM_X86_IOMMU_TABLE_H
> -#define _ASM_X86_IOMMU_TABLE_H
> -
> -#include <asm/swiotlb.h>
> -
> -/*
> - * History lesson:
> - * The execution chain of IOMMUs in 2.6.36 looks as so:
> - *
> - * [xen-swiotlb]
> - * |
> - * +----[swiotlb *]--+
> - * / | \
> - * / | \
> - * [GART] [Calgary] [Intel VT-d]
> - * /
> - * /
> - * [AMD-Vi]
> - *
> - * *: if SWIOTLB detected 'iommu=soft'/'swiotlb=force' it would skip
> - * over the rest of IOMMUs and unconditionally initialize the SWIOTLB.
> - * Also it would surreptitiously initialize set the swiotlb=1 if there were
> - * more than 4GB and if the user did not pass in 'iommu=off'. The swiotlb
> - * flag would be turned off by all IOMMUs except the Calgary one.
> - *
> - * The IOMMU_INIT* macros allow a similar tree (or more complex if desired)
> - * to be built by defining who we depend on.
> - *
> - * And all that needs to be done is to use one of the macros in the IOMMU
> - * and the pci-dma.c will take care of the rest.
> - */
Christoph,
Is it possible to keep documentation comments in source files? Or are
they completely irrelevant now?
Thanks.
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 118+ messages in thread
* Re: [PATCH 07/11] x86: remove the IOMMU table infrastructure
@ 2022-03-02 9:18 ` Anatoly Pugachev
0 siblings, 0 replies; 118+ messages in thread
From: Anatoly Pugachev @ 2022-03-02 9:18 UTC (permalink / raw)
To: Christoph Hellwig
Cc: Juergen Gross, linux-s390, linux-hyperv, Stefano Stabellini,
linux-ia64, Anshuman Khandual, Konrad Rzeszutek Wilk,
Robin Murphy, x86, linux-mips, linuxppc-dev, iommu, tboot-devel,
linux-pci, xen-devel, Boris Ostrovsky, David Woodhouse,
linux-riscv, linux-arm-kernel
On Sun, Feb 27, 2022 at 7:31 PM Christoph Hellwig <hch@lst.de> wrote:
>
> The IOMMU table tries to separate the different IOMMUs into different
> backends, but actually requires various cross calls.
>
> Rewrite the code to do the generic swiotlb/swiotlb-xen setup directly
> in pci-dma.c and then just call into the IOMMU drivers.
...
> --- a/arch/x86/include/asm/iommu_table.h
> +++ /dev/null
> @@ -1,102 +0,0 @@
> -/* SPDX-License-Identifier: GPL-2.0 */
> -#ifndef _ASM_X86_IOMMU_TABLE_H
> -#define _ASM_X86_IOMMU_TABLE_H
> -
> -#include <asm/swiotlb.h>
> -
> -/*
> - * History lesson:
> - * The execution chain of IOMMUs in 2.6.36 looks as so:
> - *
> - * [xen-swiotlb]
> - * |
> - * +----[swiotlb *]--+
> - * / | \
> - * / | \
> - * [GART] [Calgary] [Intel VT-d]
> - * /
> - * /
> - * [AMD-Vi]
> - *
> - * *: if SWIOTLB detected 'iommu=soft'/'swiotlb=force' it would skip
> - * over the rest of IOMMUs and unconditionally initialize the SWIOTLB.
> - * Also it would surreptitiously initialize set the swiotlb=1 if there were
> - * more than 4GB and if the user did not pass in 'iommu=off'. The swiotlb
> - * flag would be turned off by all IOMMUs except the Calgary one.
> - *
> - * The IOMMU_INIT* macros allow a similar tree (or more complex if desired)
> - * to be built by defining who we depend on.
> - *
> - * And all that needs to be done is to use one of the macros in the IOMMU
> - * and the pci-dma.c will take care of the rest.
> - */
Christoph,
Is it possible to keep documentation comments in source files? Or are
they completely irrelevant now?
Thanks.
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply [flat|nested] 118+ messages in thread
* Re: [PATCH 07/11] x86: remove the IOMMU table infrastructure
@ 2022-03-02 9:18 ` Anatoly Pugachev
0 siblings, 0 replies; 118+ messages in thread
From: Anatoly Pugachev @ 2022-03-02 9:18 UTC (permalink / raw)
To: Christoph Hellwig
Cc: iommu, x86, Anshuman Khandual, Konrad Rzeszutek Wilk,
Stefano Stabellini, Boris Ostrovsky, Juergen Gross, Joerg Roedel,
David Woodhouse, Lu Baolu, Robin Murphy, linux-arm-kernel,
xen-devel, linux-ia64, linux-mips, linuxppc-dev, linux-riscv,
linux-s390, linux-hyperv, tboot-devel, linux-pci
On Sun, Feb 27, 2022 at 7:31 PM Christoph Hellwig <hch@lst.de> wrote:
>
> The IOMMU table tries to separate the different IOMMUs into different
> backends, but actually requires various cross calls.
>
> Rewrite the code to do the generic swiotlb/swiotlb-xen setup directly
> in pci-dma.c and then just call into the IOMMU drivers.
...
> --- a/arch/x86/include/asm/iommu_table.h
> +++ /dev/null
> @@ -1,102 +0,0 @@
> -/* SPDX-License-Identifier: GPL-2.0 */
> -#ifndef _ASM_X86_IOMMU_TABLE_H
> -#define _ASM_X86_IOMMU_TABLE_H
> -
> -#include <asm/swiotlb.h>
> -
> -/*
> - * History lesson:
> - * The execution chain of IOMMUs in 2.6.36 looks as so:
> - *
> - * [xen-swiotlb]
> - * |
> - * +----[swiotlb *]--+
> - * / | \
> - * / | \
> - * [GART] [Calgary] [Intel VT-d]
> - * /
> - * /
> - * [AMD-Vi]
> - *
> - * *: if SWIOTLB detected 'iommu=soft'/'swiotlb=force' it would skip
> - * over the rest of IOMMUs and unconditionally initialize the SWIOTLB.
> - * Also it would surreptitiously initialize set the swiotlb=1 if there were
> - * more than 4GB and if the user did not pass in 'iommu=off'. The swiotlb
> - * flag would be turned off by all IOMMUs except the Calgary one.
> - *
> - * The IOMMU_INIT* macros allow a similar tree (or more complex if desired)
> - * to be built by defining who we depend on.
> - *
> - * And all that needs to be done is to use one of the macros in the IOMMU
> - * and the pci-dma.c will take care of the rest.
> - */
Christoph,
Is it possible to keep documentation comments in source files? Or are
they completely irrelevant now?
Thanks.
^ permalink raw reply [flat|nested] 118+ messages in thread
* Re: [PATCH 07/11] x86: remove the IOMMU table infrastructure
2022-03-02 9:18 ` Anatoly Pugachev
` (3 preceding siblings ...)
(?)
@ 2022-03-02 9:22 ` Christoph Hellwig
-1 siblings, 0 replies; 118+ messages in thread
From: Christoph Hellwig @ 2022-03-02 9:22 UTC (permalink / raw)
To: Anatoly Pugachev
Cc: Christoph Hellwig, iommu, x86, Anshuman Khandual,
Konrad Rzeszutek Wilk, Stefano Stabellini, Boris Ostrovsky,
Juergen Gross, Joerg Roedel, David Woodhouse, Lu Baolu,
Robin Murphy, linux-arm-kernel, xen-devel, linux-ia64,
linux-mips, linuxppc-dev, linux-riscv, linux-s390, linux-hyperv,
tboot-devel, linux-pci
On Wed, Mar 02, 2022 at 12:18:26PM +0300, Anatoly Pugachev wrote:
> Is it possible to keep documentation comments in source files? Or are
> they completely irrelevant now?
That ones you quoted are very much irrelevant now. And the behaviour
of the swiotlb disabling will have to change (this patchset is a bit
of a preparation for now) as we now use per-device dma_ops and the
dma-iommu can dip into the swiotlb pool for untrusted devices. In
practive we'll basically have to always initialize the swiotlb buffer
now.
^ permalink raw reply [flat|nested] 118+ messages in thread
* Re: [PATCH 07/11] x86: remove the IOMMU table infrastructure
@ 2022-03-02 9:22 ` Christoph Hellwig
0 siblings, 0 replies; 118+ messages in thread
From: Christoph Hellwig @ 2022-03-02 9:22 UTC (permalink / raw)
To: Anatoly Pugachev
Cc: linux-hyperv, x86, linux-ia64, linux-pci, linux-riscv,
Christoph Hellwig, linux-s390, Stefano Stabellini,
Konrad Rzeszutek Wilk, tboot-devel, xen-devel, David Woodhouse,
Anshuman Khandual, Boris Ostrovsky, linux-arm-kernel,
Juergen Gross, linuxppc-dev, linux-mips, iommu, Robin Murphy
On Wed, Mar 02, 2022 at 12:18:26PM +0300, Anatoly Pugachev wrote:
> Is it possible to keep documentation comments in source files? Or are
> they completely irrelevant now?
That ones you quoted are very much irrelevant now. And the behaviour
of the swiotlb disabling will have to change (this patchset is a bit
of a preparation for now) as we now use per-device dma_ops and the
dma-iommu can dip into the swiotlb pool for untrusted devices. In
practive we'll basically have to always initialize the swiotlb buffer
now.
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply [flat|nested] 118+ messages in thread
* Re: [PATCH 07/11] x86: remove the IOMMU table infrastructure
@ 2022-03-02 9:22 ` Christoph Hellwig
0 siblings, 0 replies; 118+ messages in thread
From: Christoph Hellwig @ 2022-03-02 9:22 UTC (permalink / raw)
To: Anatoly Pugachev
Cc: Christoph Hellwig, iommu, x86, Anshuman Khandual,
Konrad Rzeszutek Wilk, Stefano Stabellini, Boris Ostrovsky,
Juergen Gross, Joerg Roedel, David Woodhouse, Lu Baolu,
Robin Murphy, linux-arm-kernel, xen-devel, linux-ia64,
linux-mips, linuxppc-dev, linux-riscv, linux-s390, linux-hyperv,
tboot-devel, linux-pci
On Wed, Mar 02, 2022 at 12:18:26PM +0300, Anatoly Pugachev wrote:
> Is it possible to keep documentation comments in source files? Or are
> they completely irrelevant now?
That ones you quoted are very much irrelevant now. And the behaviour
of the swiotlb disabling will have to change (this patchset is a bit
of a preparation for now) as we now use per-device dma_ops and the
dma-iommu can dip into the swiotlb pool for untrusted devices. In
practive we'll basically have to always initialize the swiotlb buffer
now.
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 118+ messages in thread
* Re: [PATCH 07/11] x86: remove the IOMMU table infrastructure
@ 2022-03-02 9:22 ` Christoph Hellwig
0 siblings, 0 replies; 118+ messages in thread
From: Christoph Hellwig @ 2022-03-02 9:22 UTC (permalink / raw)
To: Anatoly Pugachev
Cc: linux-hyperv, x86, linux-ia64, linux-pci, linux-riscv,
Christoph Hellwig, linux-s390, Stefano Stabellini, Joerg Roedel,
Konrad Rzeszutek Wilk, tboot-devel, xen-devel, David Woodhouse,
Anshuman Khandual, Boris Ostrovsky, linux-arm-kernel,
Juergen Gross, linuxppc-dev, linux-mips, iommu, Robin Murphy,
Lu Baolu
On Wed, Mar 02, 2022 at 12:18:26PM +0300, Anatoly Pugachev wrote:
> Is it possible to keep documentation comments in source files? Or are
> they completely irrelevant now?
That ones you quoted are very much irrelevant now. And the behaviour
of the swiotlb disabling will have to change (this patchset is a bit
of a preparation for now) as we now use per-device dma_ops and the
dma-iommu can dip into the swiotlb pool for untrusted devices. In
practive we'll basically have to always initialize the swiotlb buffer
now.
^ permalink raw reply [flat|nested] 118+ messages in thread
* Re: [PATCH 07/11] x86: remove the IOMMU table infrastructure
@ 2022-03-02 9:22 ` Christoph Hellwig
0 siblings, 0 replies; 118+ messages in thread
From: Christoph Hellwig @ 2022-03-02 9:22 UTC (permalink / raw)
To: Anatoly Pugachev
Cc: Christoph Hellwig, iommu, x86, Anshuman Khandual,
Konrad Rzeszutek Wilk, Stefano Stabellini, Boris Ostrovsky,
Juergen Gross, Joerg Roedel, David Woodhouse, Lu Baolu,
Robin Murphy, linux-arm-kernel, xen-devel, linux-ia64,
linux-mips, linuxppc-dev, linux-riscv, linux-s390, linux-hyperv,
tboot-devel, linux-pci
On Wed, Mar 02, 2022 at 12:18:26PM +0300, Anatoly Pugachev wrote:
> Is it possible to keep documentation comments in source files? Or are
> they completely irrelevant now?
That ones you quoted are very much irrelevant now. And the behaviour
of the swiotlb disabling will have to change (this patchset is a bit
of a preparation for now) as we now use per-device dma_ops and the
dma-iommu can dip into the swiotlb pool for untrusted devices. In
practive we'll basically have to always initialize the swiotlb buffer
now.
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 118+ messages in thread
* Re: [PATCH 07/11] x86: remove the IOMMU table infrastructure
@ 2022-03-02 9:22 ` Christoph Hellwig
0 siblings, 0 replies; 118+ messages in thread
From: Christoph Hellwig @ 2022-03-02 9:22 UTC (permalink / raw)
To: Anatoly Pugachev
Cc: Christoph Hellwig, iommu, x86, Anshuman Khandual,
Konrad Rzeszutek Wilk, Stefano Stabellini, Boris Ostrovsky,
Juergen Gross, Joerg Roedel, David Woodhouse, Lu Baolu,
Robin Murphy, linux-arm-kernel, xen-devel, linux-ia64,
linux-mips, linuxppc-dev, linux-riscv, linux-s390, linux-hyperv,
tboot-devel, linux-pci
On Wed, Mar 02, 2022 at 12:18:26PM +0300, Anatoly Pugachev wrote:
> Is it possible to keep documentation comments in source files? Or are
> they completely irrelevant now?
That ones you quoted are very much irrelevant now. And the behaviour
of the swiotlb disabling will have to change (this patchset is a bit
of a preparation for now) as we now use per-device dma_ops and the
dma-iommu can dip into the swiotlb pool for untrusted devices. In
practive we'll basically have to always initialize the swiotlb buffer
now.
^ permalink raw reply [flat|nested] 118+ messages in thread
end of thread, other threads:[~2022-03-02 9:47 UTC | newest]
Thread overview: 118+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-02-27 14:30 cleanup swiotlb initialization v2 Christoph Hellwig
2022-02-27 14:30 ` Christoph Hellwig
2022-02-27 14:30 ` Christoph Hellwig
2022-02-27 14:30 ` Christoph Hellwig
2022-02-27 14:30 ` Christoph Hellwig
2022-02-27 14:30 ` Christoph Hellwig
2022-02-27 14:30 ` [PATCH 01/11] dma-direct: use is_swiotlb_active in dma_direct_map_page Christoph Hellwig
2022-02-27 14:30 ` Christoph Hellwig
2022-02-27 14:30 ` Christoph Hellwig
2022-02-27 14:30 ` Christoph Hellwig
2022-02-27 14:30 ` Christoph Hellwig
2022-02-27 14:30 ` Christoph Hellwig
2022-02-27 14:30 ` [PATCH 02/11] swiotlb: make swiotlb_exit a no-op if SWIOTLB_FORCE is set Christoph Hellwig
2022-02-27 14:30 ` Christoph Hellwig
2022-02-27 14:30 ` Christoph Hellwig
2022-02-27 14:30 ` Christoph Hellwig
2022-02-27 14:30 ` Christoph Hellwig
2022-02-27 14:30 ` Christoph Hellwig
2022-02-27 14:30 ` [PATCH 03/11] swiotlb: simplify swiotlb_max_segment Christoph Hellwig
2022-02-27 14:30 ` Christoph Hellwig
2022-02-27 14:30 ` Christoph Hellwig
2022-02-27 14:30 ` Christoph Hellwig
2022-02-27 14:30 ` Christoph Hellwig
2022-02-27 14:30 ` Christoph Hellwig
2022-02-27 14:30 ` [PATCH 04/11] swiotlb: rename swiotlb_late_init_with_default_size Christoph Hellwig
2022-02-27 14:30 ` Christoph Hellwig
2022-02-27 14:30 ` Christoph Hellwig
2022-02-27 14:30 ` Christoph Hellwig
2022-02-27 14:30 ` Christoph Hellwig
2022-02-27 14:30 ` Christoph Hellwig
2022-02-27 14:30 ` [PATCH 05/11] swiotlb: pass a gfp_mask argument to swiotlb_init_late Christoph Hellwig
2022-02-27 14:30 ` Christoph Hellwig
2022-02-27 14:30 ` Christoph Hellwig
2022-02-27 14:30 ` Christoph Hellwig
2022-02-27 14:30 ` Christoph Hellwig
2022-02-27 14:30 ` Christoph Hellwig
2022-02-27 14:30 ` [PATCH 06/11] MIPS/octeon: use swiotlb_init instead of open coding it Christoph Hellwig
2022-02-27 14:30 ` Christoph Hellwig
2022-02-27 14:30 ` Christoph Hellwig
2022-02-27 14:30 ` Christoph Hellwig
2022-02-27 14:30 ` Christoph Hellwig
2022-02-27 14:30 ` Christoph Hellwig
2022-02-27 14:30 ` [PATCH 07/11] x86: remove the IOMMU table infrastructure Christoph Hellwig
2022-02-27 14:30 ` Christoph Hellwig
2022-02-27 14:30 ` Christoph Hellwig
2022-02-27 14:30 ` Christoph Hellwig
2022-02-27 14:30 ` Christoph Hellwig
2022-02-27 14:30 ` Christoph Hellwig
2022-03-01 18:20 ` Konrad Rzeszutek Wilk
2022-03-01 18:20 ` Konrad Rzeszutek Wilk
2022-03-01 18:20 ` Konrad Rzeszutek Wilk
2022-03-01 18:20 ` Konrad Rzeszutek Wilk
2022-03-01 18:20 ` Konrad Rzeszutek Wilk
2022-03-01 18:20 ` Konrad Rzeszutek Wilk
2022-03-02 8:07 ` Christoph Hellwig
2022-03-02 8:07 ` Christoph Hellwig
2022-03-02 8:07 ` Christoph Hellwig
2022-03-02 8:07 ` Christoph Hellwig
2022-03-02 8:07 ` Christoph Hellwig
2022-03-02 8:07 ` Christoph Hellwig
2022-03-02 9:18 ` Anatoly Pugachev
2022-03-02 9:18 ` Anatoly Pugachev
2022-03-02 9:18 ` Anatoly Pugachev
2022-03-02 9:18 ` Anatoly Pugachev
2022-03-02 9:18 ` Anatoly Pugachev
2022-03-02 9:18 ` Anatoly Pugachev
2022-03-02 9:22 ` Christoph Hellwig
2022-03-02 9:22 ` Christoph Hellwig
2022-03-02 9:22 ` Christoph Hellwig
2022-03-02 9:22 ` Christoph Hellwig
2022-03-02 9:22 ` Christoph Hellwig
2022-03-02 9:22 ` Christoph Hellwig
2022-02-27 14:30 ` [PATCH 08/11] swiotlb: make the swiotlb_init interface more useful Christoph Hellwig
2022-02-27 14:30 ` Christoph Hellwig
2022-02-27 14:30 ` Christoph Hellwig
2022-02-27 14:30 ` Christoph Hellwig
2022-02-27 14:30 ` Christoph Hellwig
2022-02-27 14:30 ` Christoph Hellwig
2022-02-27 18:15 ` Christophe Leroy
2022-02-27 18:15 ` Christophe Leroy
2022-02-27 18:15 ` Christophe Leroy
2022-02-27 18:15 ` Christophe Leroy
2022-02-27 18:15 ` Christophe Leroy
2022-02-27 18:15 ` Christophe Leroy
2022-02-28 2:53 ` Michael Kelley (LINUX)
2022-02-28 2:53 ` Michael Kelley (LINUX)
2022-02-28 2:53 ` Michael Kelley (LINUX)
2022-02-28 2:53 ` Michael Kelley (LINUX)
2022-02-28 2:53 ` Michael Kelley (LINUX) via iommu
2022-02-28 11:30 ` Christoph Hellwig
2022-02-28 11:30 ` Christoph Hellwig
2022-02-28 11:30 ` Christoph Hellwig
2022-02-28 11:30 ` Christoph Hellwig
2022-02-28 11:30 ` Christoph Hellwig
2022-02-28 11:30 ` Christoph Hellwig
2022-02-28 17:21 ` Michael Kelley (LINUX)
2022-02-28 17:21 ` Michael Kelley (LINUX)
2022-02-28 17:21 ` Michael Kelley (LINUX)
2022-02-28 17:21 ` Michael Kelley (LINUX)
2022-02-28 17:21 ` Michael Kelley (LINUX) via iommu
2022-02-27 14:30 ` [PATCH 09/11] swiotlb: add a SWIOTLB_ANY flag to lift the low memory restriction Christoph Hellwig
2022-02-27 14:30 ` Christoph Hellwig
2022-02-27 14:30 ` Christoph Hellwig
2022-02-27 14:30 ` Christoph Hellwig
2022-02-27 14:30 ` Christoph Hellwig
2022-02-27 14:30 ` Christoph Hellwig
2022-02-27 14:30 ` [PATCH 10/11] swiotlb: merge swiotlb-xen initialization into swiotlb Christoph Hellwig
2022-02-27 14:30 ` Christoph Hellwig
2022-02-27 14:30 ` Christoph Hellwig
2022-02-27 14:30 ` Christoph Hellwig
2022-02-27 14:30 ` Christoph Hellwig
2022-02-27 14:30 ` Christoph Hellwig
2022-02-27 14:30 ` [PATCH 11/11] x86: remove cruft from <asm/dma-mapping.h> Christoph Hellwig
2022-02-27 14:30 ` Christoph Hellwig
2022-02-27 14:30 ` Christoph Hellwig
2022-02-27 14:30 ` Christoph Hellwig
2022-02-27 14:30 ` Christoph Hellwig
2022-02-27 14:30 ` Christoph Hellwig
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.