* [PATCH] Documentation: x86: add documenation for AMD IOMMU @ 2022-03-08 19:04 Alex Deucher 2022-03-08 20:09 ` Robin Murphy 0 siblings, 1 reply; 3+ messages in thread From: Alex Deucher @ 2022-03-08 19:04 UTC (permalink / raw) To: linux-doc, linux-kernel, corbet, hpa, x86, dave.hansen, bp, mingo, tglx, joro, suravee.suthikulpanit, will, iommu Cc: Alex Deucher Add preliminary documenation for AMD IOMMU. Signed-off-by: Alex Deucher <alexander.deucher@amd.com> --- Documentation/x86/amd-iommu.rst | 85 +++++++++++++++++++++++++++++++ Documentation/x86/index.rst | 1 + Documentation/x86/intel-iommu.rst | 2 +- 3 files changed, 87 insertions(+), 1 deletion(-) create mode 100644 Documentation/x86/amd-iommu.rst diff --git a/Documentation/x86/amd-iommu.rst b/Documentation/x86/amd-iommu.rst new file mode 100644 index 000000000000..89820140fefa --- /dev/null +++ b/Documentation/x86/amd-iommu.rst @@ -0,0 +1,85 @@ +================= +AMD IOMMU Support +================= + +The architecture spec can be obtained from the below location. + +https://www.amd.com/system/files/TechDocs/48882_IOMMU.pdf + +This guide gives a quick cheat sheet for some basic understanding. + +Some Keywords + +- IVRS - I/O Virtualization Reporting Structure +- IVDB - I/O Virtualization Definition Block +- IVHD - I/O Virtualization Hardware Definition +- IOVA - I/O Virtual Address. + +Basic stuff +----------- + +ACPI enumerates and lists the different DMA engines in the platform, and +device scope relationships between PCI devices and which DMA engine controls +them. + +What is IVRS? +------------- + +The architecture defines an ACPI-compatible data structure called an I/O +Virtualization Reporting Structure (IVRS) that is used to convey information +related to I/O virtualization to system software. The IVRS describes the +configuration and capabilities of the IOMMUs contained in the platform as +well as information about the devices that each IOMMU virtualizes. + +The IVRS provides information about the following: +- IOMMUs present in the platform including their capabilities and proper configuration +- System I/O topology relevant to each IOMMU +- Peripheral devices that cannot be otherwise enumerated +- Memory regions used by SMI/SMM, platform firmware, and platform hardware. These are +generally exclusion ranges to be configured by system software. + +How is IOVA generated? +---------------------- + +Well behaved drivers call pci_map_*() calls before sending command to device +that needs to perform DMA. Once DMA is completed and mapping is no longer +required, device performs a pci_unmap_*() calls to unmap the region. + +The AMD IOMMU driver allocates a virtual address per domain. Each PCIE +device has its own domain (hence protection). Devices under p2p bridges +share the virtual address with all devices under the p2p bridge due to +transaction id aliasing for p2p bridges. + +IOVA generation is pretty generic. We used the same technique as vmalloc() +but these are not global address spaces, but separate for each domain. +Different DMA engines may support different number of domains. + + +Fault reporting +--------------- +When errors are reported, the DMA engine signals via an interrupt. The fault +reason and device that caused it with fault reason is printed on console. + +See below for sample. + + +Boot Message Sample +------------------- + +Something like this gets printed indicating presence of the IOMMU. + + iommu: Default domain type: Translated + iommu: DMA domain TLB invalidation policy: lazy mode + + +PCI-DMA: Using AMD IOMMU +------------------------ + +Fault reporting +^^^^^^^^^^^^^^^ + +:: + + AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x0007 address=0xffffc02000 flags=0x0000] + AMD-Vi: Event logged [IO_PAGE_FAULT device=07:00.0 domain=0x0007 address=0xffffc02000 flags=0x0000] + diff --git a/Documentation/x86/index.rst b/Documentation/x86/index.rst index f498f1d36cd3..15711134eb68 100644 --- a/Documentation/x86/index.rst +++ b/Documentation/x86/index.rst @@ -22,6 +22,7 @@ x86-specific Documentation mtrr pat intel-iommu + amd-iommu intel_txt amd-memory-encryption pti diff --git a/Documentation/x86/intel-iommu.rst b/Documentation/x86/intel-iommu.rst index 099f13d51d5f..4d3391c7bd3f 100644 --- a/Documentation/x86/intel-iommu.rst +++ b/Documentation/x86/intel-iommu.rst @@ -1,5 +1,5 @@ =================== -Linux IOMMU Support +Intel IOMMU Support =================== The architecture spec can be obtained from the below location. -- 2.35.1 ^ permalink raw reply related [flat|nested] 3+ messages in thread
* Re: [PATCH] Documentation: x86: add documenation for AMD IOMMU 2022-03-08 19:04 [PATCH] Documentation: x86: add documenation for AMD IOMMU Alex Deucher @ 2022-03-08 20:09 ` Robin Murphy 2022-03-09 17:53 ` Deucher, Alexander 0 siblings, 1 reply; 3+ messages in thread From: Robin Murphy @ 2022-03-08 20:09 UTC (permalink / raw) To: Alex Deucher, linux-doc, linux-kernel, corbet, hpa, x86, dave.hansen, bp, mingo, tglx, joro, suravee.suthikulpanit, will, iommu On 2022-03-08 19:04, Alex Deucher via iommu wrote: > Add preliminary documenation for AMD IOMMU. > > Signed-off-by: Alex Deucher <alexander.deucher@amd.com> > --- > Documentation/x86/amd-iommu.rst | 85 +++++++++++++++++++++++++++++++ > Documentation/x86/index.rst | 1 + > Documentation/x86/intel-iommu.rst | 2 +- > 3 files changed, 87 insertions(+), 1 deletion(-) > create mode 100644 Documentation/x86/amd-iommu.rst > > diff --git a/Documentation/x86/amd-iommu.rst b/Documentation/x86/amd-iommu.rst > new file mode 100644 > index 000000000000..89820140fefa > --- /dev/null > +++ b/Documentation/x86/amd-iommu.rst > @@ -0,0 +1,85 @@ > +================= > +AMD IOMMU Support > +================= > + > +The architecture spec can be obtained from the below location. > + > +https://www.amd.com/system/files/TechDocs/48882_IOMMU.pdf > + > +This guide gives a quick cheat sheet for some basic understanding. > + > +Some Keywords > + > +- IVRS - I/O Virtualization Reporting Structure > +- IVDB - I/O Virtualization Definition Block > +- IVHD - I/O Virtualization Hardware Definition > +- IOVA - I/O Virtual Address. > + > +Basic stuff > +----------- > + > +ACPI enumerates and lists the different DMA engines in the platform, and > +device scope relationships between PCI devices and which DMA engine controls > +them. "DMA engine" typically means a dedicated device for peripheral-to-memory or memory-to-memory transfers, or the responsible block within a general DMA-capable endpoint. In the context of the original Intel doc from whence I see this is copied, this probably should have said "DMAR unit" or similar; here I'd suggest picking your favourite vendor-appropriate term for "instance of IOMMU translation hardware". Let's not promote confusion more than necessary. > + > +What is IVRS? > +------------- > + > +The architecture defines an ACPI-compatible data structure called an I/O > +Virtualization Reporting Structure (IVRS) that is used to convey information > +related to I/O virtualization to system software. The IVRS describes the > +configuration and capabilities of the IOMMUs contained in the platform as > +well as information about the devices that each IOMMU virtualizes. > + > +The IVRS provides information about the following: > +- IOMMUs present in the platform including their capabilities and proper configuration > +- System I/O topology relevant to each IOMMU > +- Peripheral devices that cannot be otherwise enumerated > +- Memory regions used by SMI/SMM, platform firmware, and platform hardware. These are > +generally exclusion ranges to be configured by system software. > + > +How is IOVA generated? > +---------------------- > + > +Well behaved drivers call pci_map_*() calls before sending command to device Horribly out-of-date drivers call pci_map_*(). Modern well-behaved drivers call dma_map_*() ;) > +that needs to perform DMA. Once DMA is completed and mapping is no longer > +required, device performs a pci_unmap_*() calls to unmap the region. > + > +The AMD IOMMU driver allocates a virtual address per domain. Each PCIE > +device has its own domain (hence protection). Devices under p2p bridges > +share the virtual address with all devices under the p2p bridge due to > +transaction id aliasing for p2p bridges. > + > +IOVA generation is pretty generic. We used the same technique as vmalloc() > +but these are not global address spaces, but separate for each domain. > +Different DMA engines may support different number of domains. I'm not sure about this whole section, really - IOVA management was entirely decoupled from drivers some time ago. If there's value in having some overview documentation, then it's probably worth consolidating into a common "IOMMU API" doc that can be cross-referenced for a summary of domains, groups, and iommu_dma_ops. > + > + > +Fault reporting > +--------------- > +When errors are reported, the DMA engine signals via an interrupt. The fault Again, here I instinctively read "DMA engine" as being the endpoint device *making* the DMA transaction that faulted, and indeed that might happen to raise its own error interrupt if it gets an unexpected abort back from the IOMMU, which is coincidental to a thoroughly misleading degree... Thanks, Robin. > +reason and device that caused it with fault reason is printed on console. > + > +See below for sample. > + > + > +Boot Message Sample > +------------------- > + > +Something like this gets printed indicating presence of the IOMMU. > + > + iommu: Default domain type: Translated > + iommu: DMA domain TLB invalidation policy: lazy mode > + > + > +PCI-DMA: Using AMD IOMMU > +------------------------ > + > +Fault reporting > +^^^^^^^^^^^^^^^ > + > +:: > + > + AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x0007 address=0xffffc02000 flags=0x0000] > + AMD-Vi: Event logged [IO_PAGE_FAULT device=07:00.0 domain=0x0007 address=0xffffc02000 flags=0x0000] > + > diff --git a/Documentation/x86/index.rst b/Documentation/x86/index.rst > index f498f1d36cd3..15711134eb68 100644 > --- a/Documentation/x86/index.rst > +++ b/Documentation/x86/index.rst > @@ -22,6 +22,7 @@ x86-specific Documentation > mtrr > pat > intel-iommu > + amd-iommu > intel_txt > amd-memory-encryption > pti > diff --git a/Documentation/x86/intel-iommu.rst b/Documentation/x86/intel-iommu.rst > index 099f13d51d5f..4d3391c7bd3f 100644 > --- a/Documentation/x86/intel-iommu.rst > +++ b/Documentation/x86/intel-iommu.rst > @@ -1,5 +1,5 @@ > =================== > -Linux IOMMU Support > +Intel IOMMU Support > =================== > > The architecture spec can be obtained from the below location. ^ permalink raw reply [flat|nested] 3+ messages in thread
* RE: [PATCH] Documentation: x86: add documenation for AMD IOMMU 2022-03-08 20:09 ` Robin Murphy @ 2022-03-09 17:53 ` Deucher, Alexander 0 siblings, 0 replies; 3+ messages in thread From: Deucher, Alexander @ 2022-03-09 17:53 UTC (permalink / raw) To: Robin Murphy, linux-doc, linux-kernel, corbet, hpa, x86, dave.hansen, bp, mingo, tglx, joro, Suthikulpanit, Suravee, will, iommu [Public] > -----Original Message----- > From: Robin Murphy <robin.murphy@arm.com> > Sent: Tuesday, March 8, 2022 3:09 PM > To: Deucher, Alexander <Alexander.Deucher@amd.com>; linux- > doc@vger.kernel.org; linux-kernel@vger.kernel.org; corbet@lwn.net; > hpa@zytor.com; x86@kernel.org; dave.hansen@linux.intel.com; > bp@alien8.de; mingo@redhat.com; tglx@linutronix.de; joro@8bytes.org; > Suthikulpanit, Suravee <Suravee.Suthikulpanit@amd.com>; will@kernel.org; > iommu@lists.linux-foundation.org > Subject: Re: [PATCH] Documentation: x86: add documenation for AMD > IOMMU > > On 2022-03-08 19:04, Alex Deucher via iommu wrote: > > Add preliminary documenation for AMD IOMMU. > > > > Signed-off-by: Alex Deucher <alexander.deucher@amd.com> > > --- > > Documentation/x86/amd-iommu.rst | 85 > +++++++++++++++++++++++++++++++ > > Documentation/x86/index.rst | 1 + > > Documentation/x86/intel-iommu.rst | 2 +- > > 3 files changed, 87 insertions(+), 1 deletion(-) > > create mode 100644 Documentation/x86/amd-iommu.rst > > > > diff --git a/Documentation/x86/amd-iommu.rst > > b/Documentation/x86/amd-iommu.rst new file mode 100644 index > > 000000000000..89820140fefa > > --- /dev/null > > +++ b/Documentation/x86/amd-iommu.rst > > @@ -0,0 +1,85 @@ > > +================= > > +AMD IOMMU Support > > +================= > > + > > +The architecture spec can be obtained from the below location. > > + > > > +https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fw > ww > > > +.amd.com%2Fsystem%2Ffiles%2FTechDocs%2F48882_IOMMU.pdf&da > ta=04%7C > > > +01%7Calexander.deucher%40amd.com%7C3adb51f8c3f1435e0deb08da013f > 8172%7 > > > +C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637823669974023501 > %7CUnkn > > > +own%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6I > k1haWw > > > +iLCJXVCI6Mn0%3D%7C3000&sdata=9Wq07GM%2BdT9xt%2FCZ3xhue > %2BrNIe6CnD > > +cG32kwqosUEZ8%3D&reserved=0 > > + > > +This guide gives a quick cheat sheet for some basic understanding. > > + > > +Some Keywords > > + > > +- IVRS - I/O Virtualization Reporting Structure > > +- IVDB - I/O Virtualization Definition Block > > +- IVHD - I/O Virtualization Hardware Definition > > +- IOVA - I/O Virtual Address. > > + > > +Basic stuff > > +----------- > > + > > +ACPI enumerates and lists the different DMA engines in the platform, > > +and device scope relationships between PCI devices and which DMA > > +engine controls them. > > "DMA engine" typically means a dedicated device for peripheral-to-memory > or memory-to-memory transfers, or the responsible block within a general > DMA-capable endpoint. In the context of the original Intel doc from whence I > see this is copied, this probably should have said "DMAR unit" > or similar; here I'd suggest picking your favourite vendor-appropriate term > for "instance of IOMMU translation hardware". Let's not promote confusion > more than necessary. > > > + > > +What is IVRS? > > +------------- > > + > > +The architecture defines an ACPI-compatible data structure called an > > +I/O Virtualization Reporting Structure (IVRS) that is used to convey > > +information related to I/O virtualization to system software. The > > +IVRS describes the configuration and capabilities of the IOMMUs > > +contained in the platform as well as information about the devices that > each IOMMU virtualizes. > > + > > +The IVRS provides information about the following: > > +- IOMMUs present in the platform including their capabilities and > > +proper configuration > > +- System I/O topology relevant to each IOMMU > > +- Peripheral devices that cannot be otherwise enumerated > > +- Memory regions used by SMI/SMM, platform firmware, and platform > > +hardware. These are generally exclusion ranges to be configured by > system software. > > + > > +How is IOVA generated? > > +---------------------- > > + > > +Well behaved drivers call pci_map_*() calls before sending command to > > +device > > Horribly out-of-date drivers call pci_map_*(). Modern well-behaved drivers > call dma_map_*() ;) > > > +that needs to perform DMA. Once DMA is completed and mapping is no > > +longer required, device performs a pci_unmap_*() calls to unmap the > region. > > + > > +The AMD IOMMU driver allocates a virtual address per domain. Each > > +PCIE device has its own domain (hence protection). Devices under p2p > > +bridges share the virtual address with all devices under the p2p > > +bridge due to transaction id aliasing for p2p bridges. > > + > > +IOVA generation is pretty generic. We used the same technique as > > +vmalloc() but these are not global address spaces, but separate for each > domain. > > +Different DMA engines may support different number of domains. > > I'm not sure about this whole section, really - IOVA management was entirely > decoupled from drivers some time ago. If there's value in having some > overview documentation, then it's probably worth consolidating into a > common "IOMMU API" doc that can be cross-referenced for a summary of > domains, groups, and iommu_dma_ops. > > > + > > + > > +Fault reporting > > +--------------- > > +When errors are reported, the DMA engine signals via an interrupt. The > fault > > Again, here I instinctively read "DMA engine" as being the endpoint > device *making* the DMA transaction that faulted, and indeed that might > happen to raise its own error interrupt if it gets an unexpected abort > back from the IOMMU, which is coincidental to a thoroughly misleading > degree... Thanks for the review Robin. All good points. I've fixed them up and also fixed up the Intel documentation. Will send out v2 shortly. Thanks, Alex > > Thanks, > Robin. > > > +reason and device that caused it with fault reason is printed on console. > > + > > +See below for sample. > > + > > + > > +Boot Message Sample > > +------------------- > > + > > +Something like this gets printed indicating presence of the IOMMU. > > + > > + iommu: Default domain type: Translated > > + iommu: DMA domain TLB invalidation policy: lazy mode > > + > > + > > +PCI-DMA: Using AMD IOMMU > > +------------------------ > > + > > +Fault reporting > > +^^^^^^^^^^^^^^^ > > + > > +:: > > + > > + AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x0007 > address=0xffffc02000 flags=0x0000] > > + AMD-Vi: Event logged [IO_PAGE_FAULT device=07:00.0 > domain=0x0007 address=0xffffc02000 flags=0x0000] > > + > > diff --git a/Documentation/x86/index.rst b/Documentation/x86/index.rst > > index f498f1d36cd3..15711134eb68 100644 > > --- a/Documentation/x86/index.rst > > +++ b/Documentation/x86/index.rst > > @@ -22,6 +22,7 @@ x86-specific Documentation > > mtrr > > pat > > intel-iommu > > + amd-iommu > > intel_txt > > amd-memory-encryption > > pti > > diff --git a/Documentation/x86/intel-iommu.rst > b/Documentation/x86/intel-iommu.rst > > index 099f13d51d5f..4d3391c7bd3f 100644 > > --- a/Documentation/x86/intel-iommu.rst > > +++ b/Documentation/x86/intel-iommu.rst > > @@ -1,5 +1,5 @@ > > =================== > > -Linux IOMMU Support > > +Intel IOMMU Support > > =================== > > > > The architecture spec can be obtained from the below location. ^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2022-03-09 17:53 UTC | newest] Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2022-03-08 19:04 [PATCH] Documentation: x86: add documenation for AMD IOMMU Alex Deucher 2022-03-08 20:09 ` Robin Murphy 2022-03-09 17:53 ` Deucher, Alexander
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).