* [PATCH v1 1/3] iommu/vt-d: Only clear real DMA device's context entries
2020-05-27 16:56 [PATCH v1 0/3] iommu/vt-d: real DMA sub-device info allocation Jon Derrick
@ 2020-05-27 16:56 ` Jon Derrick
2020-05-28 6:59 ` Lu Baolu
2020-05-27 16:56 ` [PATCH v1 2/3] iommu/vt-d: Allocate domain info for real DMA sub-devices Jon Derrick
` (2 subsequent siblings)
3 siblings, 1 reply; 9+ messages in thread
From: Jon Derrick @ 2020-05-27 16:56 UTC (permalink / raw)
To: iommu
Cc: linux-pci, Joerg Roedel, Lu Baolu, Jacob Pan,
Sai Praneeth Prakhya, Ashok Raj, Jon Derrick
Domain context mapping can encounter issues with sub-devices of a real
DMA device. A sub-device cannot have a valid context entry due to it
potentially aliasing another device's 16-bit ID. It's expected that
sub-devices of the real DMA device uses the real DMA device's requester
when context mapping.
This is an issue when a sub-device is removed where the context entry is
cleared for all aliases. Other sub-devices are still valid, resulting in
those sub-devices being stranded without valid context entries.
The correct approach is to use the real DMA device when programming the
context entries. The insertion path is correct because device_to_iommu()
will return the bus and devfn of the real DMA device. The removal path
needs to only operate on the real DMA device, otherwise the entire
context entry would be cleared for all sub-devices of the real DMA
device.
This patch also adds a helper to determine if a struct device is a
sub-device of a real DMA device.
Signed-off-by: Jon Derrick <jonathan.derrick@intel.com>
---
drivers/iommu/intel-iommu.c | 9 ++++++++-
1 file changed, 8 insertions(+), 1 deletion(-)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index ff5a30a..1ff45b2 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -2500,6 +2500,12 @@ static int domain_setup_first_level(struct intel_iommu *iommu,
flags);
}
+static bool dev_is_real_dma_subdevice(struct device *dev)
+{
+ return dev && dev_is_pci(dev) &&
+ pci_real_dma_dev(to_pci_dev(dev)) != to_pci_dev(dev);
+}
+
static struct dmar_domain *dmar_insert_one_dev_info(struct intel_iommu *iommu,
int bus, int devfn,
struct device *dev,
@@ -4975,7 +4981,8 @@ static void __dmar_remove_one_dev_info(struct device_domain_info *info)
PASID_RID2PASID, false);
iommu_disable_dev_iotlb(info);
- domain_context_clear(iommu, info->dev);
+ if (!dev_is_real_dma_subdevice(info->dev))
+ domain_context_clear(iommu, info->dev);
intel_pasid_free_table(info->dev);
}
--
1.8.3.1
^ permalink raw reply related [flat|nested] 9+ messages in thread
* Re: [PATCH v1 1/3] iommu/vt-d: Only clear real DMA device's context entries
2020-05-27 16:56 ` [PATCH v1 1/3] iommu/vt-d: Only clear real DMA device's context entries Jon Derrick
@ 2020-05-28 6:59 ` Lu Baolu
0 siblings, 0 replies; 9+ messages in thread
From: Lu Baolu @ 2020-05-28 6:59 UTC (permalink / raw)
To: Jon Derrick, iommu
Cc: baolu.lu, linux-pci, Joerg Roedel, Jacob Pan,
Sai Praneeth Prakhya, Ashok Raj
On 2020/5/28 0:56, Jon Derrick wrote:
> Domain context mapping can encounter issues with sub-devices of a real
> DMA device. A sub-device cannot have a valid context entry due to it
> potentially aliasing another device's 16-bit ID. It's expected that
> sub-devices of the real DMA device uses the real DMA device's requester
> when context mapping.
>
> This is an issue when a sub-device is removed where the context entry is
> cleared for all aliases. Other sub-devices are still valid, resulting in
> those sub-devices being stranded without valid context entries.
>
> The correct approach is to use the real DMA device when programming the
> context entries. The insertion path is correct because device_to_iommu()
> will return the bus and devfn of the real DMA device. The removal path
> needs to only operate on the real DMA device, otherwise the entire
> context entry would be cleared for all sub-devices of the real DMA
> device.
>
> This patch also adds a helper to determine if a struct device is a
> sub-device of a real DMA device.
>
> Signed-off-by: Jon Derrick <jonathan.derrick@intel.com>
Fixes: 2b0140c69637e ("iommu/vt-d: Use pci_real_dma_dev() for mapping")
Acked-by: Lu Baolu <baolu.lu@linux.intel.com>
Best regards,
baolu
^ permalink raw reply [flat|nested] 9+ messages in thread
* [PATCH v1 2/3] iommu/vt-d: Allocate domain info for real DMA sub-devices
2020-05-27 16:56 [PATCH v1 0/3] iommu/vt-d: real DMA sub-device info allocation Jon Derrick
2020-05-27 16:56 ` [PATCH v1 1/3] iommu/vt-d: Only clear real DMA device's context entries Jon Derrick
@ 2020-05-27 16:56 ` Jon Derrick
2020-05-28 7:01 ` Lu Baolu
2020-05-27 16:56 ` [PATCH v1 3/3] iommu/vt-d: Remove real DMA lookup in find_domain Jon Derrick
2020-05-29 13:12 ` [PATCH v1 0/3] iommu/vt-d: real DMA sub-device info allocation Joerg Roedel
3 siblings, 1 reply; 9+ messages in thread
From: Jon Derrick @ 2020-05-27 16:56 UTC (permalink / raw)
To: iommu
Cc: linux-pci, Joerg Roedel, Lu Baolu, Jacob Pan,
Sai Praneeth Prakhya, Ashok Raj, Jon Derrick
Sub-devices of a real DMA device might exist on a separate segment than
the real DMA device and its IOMMU. These devices should still have a
valid device_domain_info, but the current dma alias model won't
allocate info for the subdevice.
This patch adds a segment member to struct device_domain_info and uses
the sub-device's BDF so that these sub-devices won't alias to other
devices.
Signed-off-by: Jon Derrick <jonathan.derrick@intel.com>
---
drivers/iommu/intel-iommu.c | 19 +++++++++++++++----
include/linux/intel-iommu.h | 1 +
2 files changed, 16 insertions(+), 4 deletions(-)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 1ff45b2..6d39b9b 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -2463,7 +2463,7 @@ static void do_deferred_attach(struct device *dev)
struct device_domain_info *info;
list_for_each_entry(info, &device_domain_list, global)
- if (info->iommu->segment == segment && info->bus == bus &&
+ if (info->segment == segment && info->bus == bus &&
info->devfn == devfn)
return info;
@@ -2520,8 +2520,18 @@ static struct dmar_domain *dmar_insert_one_dev_info(struct intel_iommu *iommu,
if (!info)
return NULL;
- info->bus = bus;
- info->devfn = devfn;
+ if (!dev_is_real_dma_subdevice(dev)) {
+ info->bus = bus;
+ info->devfn = devfn;
+ info->segment = iommu->segment;
+ } else {
+ struct pci_dev *pdev = to_pci_dev(dev);
+
+ info->bus = pdev->bus->number;
+ info->devfn = pdev->devfn;
+ info->segment = pci_domain_nr(pdev->bus);
+ }
+
info->ats_supported = info->pasid_supported = info->pri_supported = 0;
info->ats_enabled = info->pasid_enabled = info->pri_enabled = 0;
info->ats_qdep = 0;
@@ -2561,7 +2571,8 @@ static struct dmar_domain *dmar_insert_one_dev_info(struct intel_iommu *iommu,
if (!found) {
struct device_domain_info *info2;
- info2 = dmar_search_domain_by_dev_info(iommu->segment, bus, devfn);
+ info2 = dmar_search_domain_by_dev_info(info->segment, info->bus,
+ info->devfn);
if (info2) {
found = info2->domain;
info2->dev = dev;
diff --git a/include/linux/intel-iommu.h b/include/linux/intel-iommu.h
index 21633ce..4100bd2 100644
--- a/include/linux/intel-iommu.h
+++ b/include/linux/intel-iommu.h
@@ -609,6 +609,7 @@ struct device_domain_info {
struct list_head auxiliary_domains; /* auxiliary domains
* attached to this device
*/
+ u32 segment; /* PCI segment number */
u8 bus; /* PCI bus number */
u8 devfn; /* PCI devfn number */
u16 pfsid; /* SRIOV physical function source ID */
--
1.8.3.1
^ permalink raw reply related [flat|nested] 9+ messages in thread
* Re: [PATCH v1 2/3] iommu/vt-d: Allocate domain info for real DMA sub-devices
2020-05-27 16:56 ` [PATCH v1 2/3] iommu/vt-d: Allocate domain info for real DMA sub-devices Jon Derrick
@ 2020-05-28 7:01 ` Lu Baolu
2020-05-28 7:14 ` Lu Baolu
0 siblings, 1 reply; 9+ messages in thread
From: Lu Baolu @ 2020-05-28 7:01 UTC (permalink / raw)
To: Jon Derrick, iommu
Cc: baolu.lu, linux-pci, Joerg Roedel, Jacob Pan,
Sai Praneeth Prakhya, Ashok Raj
On 2020/5/28 0:56, Jon Derrick wrote:
> Sub-devices of a real DMA device might exist on a separate segment than
> the real DMA device and its IOMMU. These devices should still have a
> valid device_domain_info, but the current dma alias model won't
> allocate info for the subdevice.
>
> This patch adds a segment member to struct device_domain_info and uses
> the sub-device's BDF so that these sub-devices won't alias to other
> devices.
>
> Signed-off-by: Jon Derrick <jonathan.derrick@intel.com>
Acked-by: Lu Baolu <baolu.lu@linux.intel.com>
Best regards,
baolu
> ---
> drivers/iommu/intel-iommu.c | 19 +++++++++++++++----
> include/linux/intel-iommu.h | 1 +
> 2 files changed, 16 insertions(+), 4 deletions(-)
>
> diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
> index 1ff45b2..6d39b9b 100644
> --- a/drivers/iommu/intel-iommu.c
> +++ b/drivers/iommu/intel-iommu.c
> @@ -2463,7 +2463,7 @@ static void do_deferred_attach(struct device *dev)
> struct device_domain_info *info;
>
> list_for_each_entry(info, &device_domain_list, global)
> - if (info->iommu->segment == segment && info->bus == bus &&
> + if (info->segment == segment && info->bus == bus &&
> info->devfn == devfn)
> return info;
>
> @@ -2520,8 +2520,18 @@ static struct dmar_domain *dmar_insert_one_dev_info(struct intel_iommu *iommu,
> if (!info)
> return NULL;
>
> - info->bus = bus;
> - info->devfn = devfn;
> + if (!dev_is_real_dma_subdevice(dev)) {
> + info->bus = bus;
> + info->devfn = devfn;
> + info->segment = iommu->segment;
> + } else {
> + struct pci_dev *pdev = to_pci_dev(dev);
> +
> + info->bus = pdev->bus->number;
> + info->devfn = pdev->devfn;
> + info->segment = pci_domain_nr(pdev->bus);
> + }
> +
> info->ats_supported = info->pasid_supported = info->pri_supported = 0;
> info->ats_enabled = info->pasid_enabled = info->pri_enabled = 0;
> info->ats_qdep = 0;
> @@ -2561,7 +2571,8 @@ static struct dmar_domain *dmar_insert_one_dev_info(struct intel_iommu *iommu,
>
> if (!found) {
> struct device_domain_info *info2;
> - info2 = dmar_search_domain_by_dev_info(iommu->segment, bus, devfn);
> + info2 = dmar_search_domain_by_dev_info(info->segment, info->bus,
> + info->devfn);
> if (info2) {
> found = info2->domain;
> info2->dev = dev;
> diff --git a/include/linux/intel-iommu.h b/include/linux/intel-iommu.h
> index 21633ce..4100bd2 100644
> --- a/include/linux/intel-iommu.h
> +++ b/include/linux/intel-iommu.h
> @@ -609,6 +609,7 @@ struct device_domain_info {
> struct list_head auxiliary_domains; /* auxiliary domains
> * attached to this device
> */
> + u32 segment; /* PCI segment number */
> u8 bus; /* PCI bus number */
> u8 devfn; /* PCI devfn number */
> u16 pfsid; /* SRIOV physical function source ID */
>
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH v1 2/3] iommu/vt-d: Allocate domain info for real DMA sub-devices
2020-05-28 7:01 ` Lu Baolu
@ 2020-05-28 7:14 ` Lu Baolu
0 siblings, 0 replies; 9+ messages in thread
From: Lu Baolu @ 2020-05-28 7:14 UTC (permalink / raw)
To: Jon Derrick, iommu
Cc: baolu.lu, linux-pci, Joerg Roedel, Jacob Pan,
Sai Praneeth Prakhya, Ashok Raj
On 2020/5/28 15:01, Lu Baolu wrote:
> On 2020/5/28 0:56, Jon Derrick wrote:
>> Sub-devices of a real DMA device might exist on a separate segment than
>> the real DMA device and its IOMMU. These devices should still have a
>> valid device_domain_info, but the current dma alias model won't
>> allocate info for the subdevice.
>>
>> This patch adds a segment member to struct device_domain_info and uses
>> the sub-device's BDF so that these sub-devices won't alias to other
>> devices.
>>
>> Signed-off-by: Jon Derrick <jonathan.derrick@intel.com>
>
> Acked-by: Lu Baolu <baolu.lu@linux.intel.com>
And a fix tag, so that it could be picked up for v5.6+ stable kernel.
Fixes: 2b0140c69637e ("iommu/vt-d: Use pci_real_dma_dev() for mapping")
Best regards,
baolu
^ permalink raw reply [flat|nested] 9+ messages in thread
* [PATCH v1 3/3] iommu/vt-d: Remove real DMA lookup in find_domain
2020-05-27 16:56 [PATCH v1 0/3] iommu/vt-d: real DMA sub-device info allocation Jon Derrick
2020-05-27 16:56 ` [PATCH v1 1/3] iommu/vt-d: Only clear real DMA device's context entries Jon Derrick
2020-05-27 16:56 ` [PATCH v1 2/3] iommu/vt-d: Allocate domain info for real DMA sub-devices Jon Derrick
@ 2020-05-27 16:56 ` Jon Derrick
2020-05-28 7:02 ` Lu Baolu
2020-05-29 13:12 ` [PATCH v1 0/3] iommu/vt-d: real DMA sub-device info allocation Joerg Roedel
3 siblings, 1 reply; 9+ messages in thread
From: Jon Derrick @ 2020-05-27 16:56 UTC (permalink / raw)
To: iommu
Cc: linux-pci, Joerg Roedel, Lu Baolu, Jacob Pan,
Sai Praneeth Prakhya, Ashok Raj, Jon Derrick
By removing the real DMA indirection in find_domain(), we can allow
sub-devices of a real DMA device to have their own valid
device_domain_info. The dmar lookup and context entry removal paths have
been fixed to account for sub-devices.
Fixes: 2b0140c69637 ("iommu/vt-d: Use pci_real_dma_dev() for mapping")
Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=207575
Signed-off-by: Jon Derrick <jonathan.derrick@intel.com>
---
drivers/iommu/intel-iommu.c | 3 ---
1 file changed, 3 deletions(-)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 6d39b9b..5767882 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -2436,9 +2436,6 @@ struct dmar_domain *find_domain(struct device *dev)
if (unlikely(attach_deferred(dev) || iommu_dummy(dev)))
return NULL;
- if (dev_is_pci(dev))
- dev = &pci_real_dma_dev(to_pci_dev(dev))->dev;
-
/* No lock here, assumes no domain exit in normal case */
info = get_domain_info(dev);
if (likely(info))
--
1.8.3.1
^ permalink raw reply related [flat|nested] 9+ messages in thread
* Re: [PATCH v1 3/3] iommu/vt-d: Remove real DMA lookup in find_domain
2020-05-27 16:56 ` [PATCH v1 3/3] iommu/vt-d: Remove real DMA lookup in find_domain Jon Derrick
@ 2020-05-28 7:02 ` Lu Baolu
0 siblings, 0 replies; 9+ messages in thread
From: Lu Baolu @ 2020-05-28 7:02 UTC (permalink / raw)
To: Jon Derrick, iommu
Cc: baolu.lu, linux-pci, Joerg Roedel, Jacob Pan,
Sai Praneeth Prakhya, Ashok Raj
On 2020/5/28 0:56, Jon Derrick wrote:
> By removing the real DMA indirection in find_domain(), we can allow
> sub-devices of a real DMA device to have their own valid
> device_domain_info. The dmar lookup and context entry removal paths have
> been fixed to account for sub-devices.
>
> Fixes: 2b0140c69637 ("iommu/vt-d: Use pci_real_dma_dev() for mapping")
> Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=207575
> Signed-off-by: Jon Derrick <jonathan.derrick@intel.com>
Acked-by: Lu Baolu <baolu.lu@linux.intel.com>
Best regards,
baolu
> ---
> drivers/iommu/intel-iommu.c | 3 ---
> 1 file changed, 3 deletions(-)
>
> diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
> index 6d39b9b..5767882 100644
> --- a/drivers/iommu/intel-iommu.c
> +++ b/drivers/iommu/intel-iommu.c
> @@ -2436,9 +2436,6 @@ struct dmar_domain *find_domain(struct device *dev)
> if (unlikely(attach_deferred(dev) || iommu_dummy(dev)))
> return NULL;
>
> - if (dev_is_pci(dev))
> - dev = &pci_real_dma_dev(to_pci_dev(dev))->dev;
> -
> /* No lock here, assumes no domain exit in normal case */
> info = get_domain_info(dev);
> if (likely(info))
>
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH v1 0/3] iommu/vt-d: real DMA sub-device info allocation
2020-05-27 16:56 [PATCH v1 0/3] iommu/vt-d: real DMA sub-device info allocation Jon Derrick
` (2 preceding siblings ...)
2020-05-27 16:56 ` [PATCH v1 3/3] iommu/vt-d: Remove real DMA lookup in find_domain Jon Derrick
@ 2020-05-29 13:12 ` Joerg Roedel
3 siblings, 0 replies; 9+ messages in thread
From: Joerg Roedel @ 2020-05-29 13:12 UTC (permalink / raw)
To: Jon Derrick
Cc: iommu, linux-pci, Lu Baolu, Jacob Pan, Sai Praneeth Prakhya, Ashok Raj
On Wed, May 27, 2020 at 10:56:14AM -0600, Jon Derrick wrote:
> Jon Derrick (3):
> iommu/vt-d: Only clear real DMA device's context entries
> iommu/vt-d: Allocate domain info for real DMA sub-devices
> iommu/vt-d: Remove real DMA lookup in find_domain
>
> drivers/iommu/intel-iommu.c | 31 +++++++++++++++++++++++--------
> include/linux/intel-iommu.h | 1 +
> 2 files changed, 24 insertions(+), 8 deletions(-)
Applied, thanks.
^ permalink raw reply [flat|nested] 9+ messages in thread