* [PATCH v2 1/6] PCI/ATS: Export symbols of PASID functions
2020-02-24 16:58 [PATCH v2 0/6] iommu/arm-smmu-v3: Finish PASID support and command queue batching Jean-Philippe Brucker
@ 2020-02-24 16:58 ` Jean-Philippe Brucker
2020-03-18 18:36 ` Bjorn Helgaas
2020-02-24 16:58 ` [PATCH v2 2/6] iommu/arm-smmu-v3: Add support for PCI PASID Jean-Philippe Brucker
` (4 subsequent siblings)
5 siblings, 1 reply; 9+ messages in thread
From: Jean-Philippe Brucker @ 2020-02-24 16:58 UTC (permalink / raw)
To: linux-pci, linux-arm-kernel, iommu, will, bhelgaas
Cc: zhangfei.gao, robin.murphy, robh
The Arm SMMUv3 driver uses pci_{enable,disable}_pasid() and related
functions. Export them to allow the driver to be built as a module.
Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
---
drivers/pci/ats.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/drivers/pci/ats.c b/drivers/pci/ats.c
index 3ef0bb281e7c..390e92f2d8d1 100644
--- a/drivers/pci/ats.c
+++ b/drivers/pci/ats.c
@@ -366,6 +366,7 @@ int pci_enable_pasid(struct pci_dev *pdev, int features)
return 0;
}
+EXPORT_SYMBOL_GPL(pci_enable_pasid);
/**
* pci_disable_pasid - Disable the PASID capability
@@ -390,6 +391,7 @@ void pci_disable_pasid(struct pci_dev *pdev)
pdev->pasid_enabled = 0;
}
+EXPORT_SYMBOL_GPL(pci_disable_pasid);
/**
* pci_restore_pasid_state - Restore PASID capabilities
@@ -441,6 +443,7 @@ int pci_pasid_features(struct pci_dev *pdev)
return supported;
}
+EXPORT_SYMBOL_GPL(pci_pasid_features);
#define PASID_NUMBER_SHIFT 8
#define PASID_NUMBER_MASK (0x1f << PASID_NUMBER_SHIFT)
@@ -469,4 +472,5 @@ int pci_max_pasids(struct pci_dev *pdev)
return (1 << supported);
}
+EXPORT_SYMBOL_GPL(pci_max_pasids);
#endif /* CONFIG_PCI_PASID */
--
2.25.0
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply related [flat|nested] 9+ messages in thread
* Re: [PATCH v2 1/6] PCI/ATS: Export symbols of PASID functions
2020-02-24 16:58 ` [PATCH v2 1/6] PCI/ATS: Export symbols of PASID functions Jean-Philippe Brucker
@ 2020-03-18 18:36 ` Bjorn Helgaas
2020-03-18 21:12 ` Will Deacon
0 siblings, 1 reply; 9+ messages in thread
From: Bjorn Helgaas @ 2020-03-18 18:36 UTC (permalink / raw)
To: Jean-Philippe Brucker
Cc: robh, linux-pci, robin.murphy, iommu, zhangfei.gao, will,
linux-arm-kernel
On Mon, Feb 24, 2020 at 05:58:41PM +0100, Jean-Philippe Brucker wrote:
> The Arm SMMUv3 driver uses pci_{enable,disable}_pasid() and related
> functions. Export them to allow the driver to be built as a module.
>
> Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
Acked-by: Bjorn Helgaas <bhelgaas@google.com>
> ---
> drivers/pci/ats.c | 4 ++++
> 1 file changed, 4 insertions(+)
>
> diff --git a/drivers/pci/ats.c b/drivers/pci/ats.c
> index 3ef0bb281e7c..390e92f2d8d1 100644
> --- a/drivers/pci/ats.c
> +++ b/drivers/pci/ats.c
> @@ -366,6 +366,7 @@ int pci_enable_pasid(struct pci_dev *pdev, int features)
>
> return 0;
> }
> +EXPORT_SYMBOL_GPL(pci_enable_pasid);
>
> /**
> * pci_disable_pasid - Disable the PASID capability
> @@ -390,6 +391,7 @@ void pci_disable_pasid(struct pci_dev *pdev)
>
> pdev->pasid_enabled = 0;
> }
> +EXPORT_SYMBOL_GPL(pci_disable_pasid);
>
> /**
> * pci_restore_pasid_state - Restore PASID capabilities
> @@ -441,6 +443,7 @@ int pci_pasid_features(struct pci_dev *pdev)
>
> return supported;
> }
> +EXPORT_SYMBOL_GPL(pci_pasid_features);
>
> #define PASID_NUMBER_SHIFT 8
> #define PASID_NUMBER_MASK (0x1f << PASID_NUMBER_SHIFT)
> @@ -469,4 +472,5 @@ int pci_max_pasids(struct pci_dev *pdev)
>
> return (1 << supported);
> }
> +EXPORT_SYMBOL_GPL(pci_max_pasids);
> #endif /* CONFIG_PCI_PASID */
> --
> 2.25.0
>
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH v2 1/6] PCI/ATS: Export symbols of PASID functions
2020-03-18 18:36 ` Bjorn Helgaas
@ 2020-03-18 21:12 ` Will Deacon
0 siblings, 0 replies; 9+ messages in thread
From: Will Deacon @ 2020-03-18 21:12 UTC (permalink / raw)
To: Bjorn Helgaas
Cc: Jean-Philippe Brucker, robh, linux-pci, iommu, zhangfei.gao,
robin.murphy, linux-arm-kernel
On Wed, Mar 18, 2020 at 01:36:06PM -0500, Bjorn Helgaas wrote:
> On Mon, Feb 24, 2020 at 05:58:41PM +0100, Jean-Philippe Brucker wrote:
> > The Arm SMMUv3 driver uses pci_{enable,disable}_pasid() and related
> > functions. Export them to allow the driver to be built as a module.
> >
> > Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
>
> Acked-by: Bjorn Helgaas <bhelgaas@google.com>
Cheers, Bjorn. I'll queue this series in the Arm SMMU tree.
Will
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply [flat|nested] 9+ messages in thread
* [PATCH v2 2/6] iommu/arm-smmu-v3: Add support for PCI PASID
2020-02-24 16:58 [PATCH v2 0/6] iommu/arm-smmu-v3: Finish PASID support and command queue batching Jean-Philippe Brucker
2020-02-24 16:58 ` [PATCH v2 1/6] PCI/ATS: Export symbols of PASID functions Jean-Philippe Brucker
@ 2020-02-24 16:58 ` Jean-Philippe Brucker
2020-02-24 16:58 ` [PATCH v2 3/6] iommu/arm-smmu-v3: Write level-1 descriptors atomically Jean-Philippe Brucker
` (3 subsequent siblings)
5 siblings, 0 replies; 9+ messages in thread
From: Jean-Philippe Brucker @ 2020-02-24 16:58 UTC (permalink / raw)
To: linux-pci, linux-arm-kernel, iommu, will, bhelgaas
Cc: zhangfei.gao, robin.murphy, robh
Enable PASID for PCI devices that support it. Initialize PASID early in
add_device() because it must be enabled before ATS.
Tested-by: Zhangfei Gao <zhangfei.gao@linaro.org>
Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
---
drivers/iommu/arm-smmu-v3.c | 62 ++++++++++++++++++++++++++++++++++++-
1 file changed, 61 insertions(+), 1 deletion(-)
diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c
index aa3ac2a03807..6b76df37025e 100644
--- a/drivers/iommu/arm-smmu-v3.c
+++ b/drivers/iommu/arm-smmu-v3.c
@@ -2628,6 +2628,53 @@ static void arm_smmu_disable_ats(struct arm_smmu_master *master)
atomic_dec(&smmu_domain->nr_ats_masters);
}
+static int arm_smmu_enable_pasid(struct arm_smmu_master *master)
+{
+ int ret;
+ int features;
+ int num_pasids;
+ struct pci_dev *pdev;
+
+ if (!dev_is_pci(master->dev))
+ return -ENODEV;
+
+ pdev = to_pci_dev(master->dev);
+
+ features = pci_pasid_features(pdev);
+ if (features < 0)
+ return features;
+
+ num_pasids = pci_max_pasids(pdev);
+ if (num_pasids <= 0)
+ return num_pasids;
+
+ ret = pci_enable_pasid(pdev, features);
+ if (ret) {
+ dev_err(&pdev->dev, "Failed to enable PASID\n");
+ return ret;
+ }
+
+ master->ssid_bits = min_t(u8, ilog2(num_pasids),
+ master->smmu->ssid_bits);
+ return 0;
+}
+
+static void arm_smmu_disable_pasid(struct arm_smmu_master *master)
+{
+ struct pci_dev *pdev;
+
+ if (!dev_is_pci(master->dev))
+ return;
+
+ pdev = to_pci_dev(master->dev);
+
+ if (!pdev->pasid_enabled)
+ return;
+
+ master->ssid_bits = 0;
+ pci_disable_pasid(pdev);
+}
+
static void arm_smmu_detach_dev(struct arm_smmu_master *master)
{
unsigned long flags;
@@ -2831,13 +2878,23 @@ static int arm_smmu_add_device(struct device *dev)
master->ssid_bits = min(smmu->ssid_bits, fwspec->num_pasid_bits);
+ /*
+ * Note that PASID must be enabled before, and disabled after ATS:
+ * PCI Express Base 4.0r1.0 - 10.5.1.3 ATS Control Register
+ *
+ * Behavior is undefined if this bit is Set and the value of the PASID
+ * Enable, Execute Requested Enable, or Privileged Mode Requested bits
+ * are changed.
+ */
+ arm_smmu_enable_pasid(master);
+
if (!(smmu->features & ARM_SMMU_FEAT_2_LVL_CDTAB))
master->ssid_bits = min_t(u8, master->ssid_bits,
CTXDESC_LINEAR_CDMAX);
ret = iommu_device_link(&smmu->iommu, dev);
if (ret)
- goto err_free_master;
+ goto err_disable_pasid;
group = iommu_group_get_for_dev(dev);
if (IS_ERR(group)) {
@@ -2850,6 +2907,8 @@ static int arm_smmu_add_device(struct device *dev)
err_unlink:
iommu_device_unlink(&smmu->iommu, dev);
+err_disable_pasid:
+ arm_smmu_disable_pasid(master);
err_free_master:
kfree(master);
fwspec->iommu_priv = NULL;
@@ -2870,6 +2929,7 @@ static void arm_smmu_remove_device(struct device *dev)
arm_smmu_detach_dev(master);
iommu_group_remove_device(dev);
iommu_device_unlink(&smmu->iommu, dev);
+ arm_smmu_disable_pasid(master);
kfree(master);
iommu_fwspec_free(dev);
}
--
2.25.0
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply related [flat|nested] 9+ messages in thread
* [PATCH v2 3/6] iommu/arm-smmu-v3: Write level-1 descriptors atomically
2020-02-24 16:58 [PATCH v2 0/6] iommu/arm-smmu-v3: Finish PASID support and command queue batching Jean-Philippe Brucker
2020-02-24 16:58 ` [PATCH v2 1/6] PCI/ATS: Export symbols of PASID functions Jean-Philippe Brucker
2020-02-24 16:58 ` [PATCH v2 2/6] iommu/arm-smmu-v3: Add support for PCI PASID Jean-Philippe Brucker
@ 2020-02-24 16:58 ` Jean-Philippe Brucker
2020-02-24 16:58 ` [PATCH v2 4/6] iommu/arm-smmu-v3: Add command queue batching helpers Jean-Philippe Brucker
` (2 subsequent siblings)
5 siblings, 0 replies; 9+ messages in thread
From: Jean-Philippe Brucker @ 2020-02-24 16:58 UTC (permalink / raw)
To: linux-pci, linux-arm-kernel, iommu, will, bhelgaas
Cc: zhangfei.gao, robin.murphy, robh
Use WRITE_ONCE() to make sure that the SMMU doesn't read incomplete
stream table descriptors. Refer to the comment about 64-bit accesses,
and add the comment to the equivalent context descriptor code.
Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
---
drivers/iommu/arm-smmu-v3.c | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c
index 6b76df37025e..068a16d0eabe 100644
--- a/drivers/iommu/arm-smmu-v3.c
+++ b/drivers/iommu/arm-smmu-v3.c
@@ -1531,6 +1531,7 @@ static void arm_smmu_write_cd_l1_desc(__le64 *dst,
u64 val = (l1_desc->l2ptr_dma & CTXDESC_L1_DESC_L2PTR_MASK) |
CTXDESC_L1_DESC_V;
+ /* See comment in arm_smmu_write_ctx_desc() */
WRITE_ONCE(*dst, cpu_to_le64(val));
}
@@ -1726,7 +1727,8 @@ arm_smmu_write_strtab_l1_desc(__le64 *dst, struct arm_smmu_strtab_l1_desc *desc)
val |= FIELD_PREP(STRTAB_L1_DESC_SPAN, desc->span);
val |= desc->l2ptr_dma & STRTAB_L1_DESC_L2PTR_MASK;
- *dst = cpu_to_le64(val);
+ /* See comment in arm_smmu_write_ctx_desc() */
+ WRITE_ONCE(*dst, cpu_to_le64(val));
}
static void arm_smmu_sync_ste_for_sid(struct arm_smmu_device *smmu, u32 sid)
--
2.25.0
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply related [flat|nested] 9+ messages in thread
* [PATCH v2 4/6] iommu/arm-smmu-v3: Add command queue batching helpers
2020-02-24 16:58 [PATCH v2 0/6] iommu/arm-smmu-v3: Finish PASID support and command queue batching Jean-Philippe Brucker
` (2 preceding siblings ...)
2020-02-24 16:58 ` [PATCH v2 3/6] iommu/arm-smmu-v3: Write level-1 descriptors atomically Jean-Philippe Brucker
@ 2020-02-24 16:58 ` Jean-Philippe Brucker
2020-02-24 16:58 ` [PATCH v2 5/6] iommu/arm-smmu-v3: Batch context descriptor invalidation Jean-Philippe Brucker
2020-02-24 16:58 ` [PATCH v2 6/6] iommu/arm-smmu-v3: Batch ATC invalidation commands Jean-Philippe Brucker
5 siblings, 0 replies; 9+ messages in thread
From: Jean-Philippe Brucker @ 2020-02-24 16:58 UTC (permalink / raw)
To: linux-pci, linux-arm-kernel, iommu, will, bhelgaas
Cc: zhangfei.gao, robin.murphy, robh
As more functions will implement command queue batching, add two helpers
to simplify building a command list.
Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
---
drivers/iommu/arm-smmu-v3.c | 37 ++++++++++++++++++++++++++-----------
1 file changed, 26 insertions(+), 11 deletions(-)
diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c
index 068a16d0eabe..beeec366bc41 100644
--- a/drivers/iommu/arm-smmu-v3.c
+++ b/drivers/iommu/arm-smmu-v3.c
@@ -548,6 +548,11 @@ struct arm_smmu_cmdq {
atomic_t lock;
};
+struct arm_smmu_cmdq_batch {
+ u64 cmds[CMDQ_BATCH_ENTRIES * CMDQ_ENT_DWORDS];
+ int num;
+};
+
struct arm_smmu_evtq {
struct arm_smmu_queue q;
u32 max_stalls;
@@ -1482,6 +1487,24 @@ static int arm_smmu_cmdq_issue_sync(struct arm_smmu_device *smmu)
return arm_smmu_cmdq_issue_cmdlist(smmu, NULL, 0, true);
}
+static void arm_smmu_cmdq_batch_add(struct arm_smmu_device *smmu,
+ struct arm_smmu_cmdq_batch *cmds,
+ struct arm_smmu_cmdq_ent *cmd)
+{
+ if (cmds->num == CMDQ_BATCH_ENTRIES) {
+ arm_smmu_cmdq_issue_cmdlist(smmu, cmds->cmds, cmds->num, false);
+ cmds->num = 0;
+ }
+ arm_smmu_cmdq_build_cmd(&cmds->cmds[cmds->num * CMDQ_ENT_DWORDS], cmd);
+ cmds->num++;
+}
+
+static int arm_smmu_cmdq_batch_submit(struct arm_smmu_device *smmu,
+ struct arm_smmu_cmdq_batch *cmds)
+{
+ return arm_smmu_cmdq_issue_cmdlist(smmu, cmds->cmds, cmds->num, true);
+}
+
/* Context descriptor manipulation functions */
static void arm_smmu_sync_cd(struct arm_smmu_domain *smmu_domain,
int ssid, bool leaf)
@@ -2220,10 +2243,9 @@ static void arm_smmu_tlb_inv_range(unsigned long iova, size_t size,
size_t granule, bool leaf,
struct arm_smmu_domain *smmu_domain)
{
- u64 cmds[CMDQ_BATCH_ENTRIES * CMDQ_ENT_DWORDS];
struct arm_smmu_device *smmu = smmu_domain->smmu;
unsigned long start = iova, end = iova + size;
- int i = 0;
+ struct arm_smmu_cmdq_batch cmds = {};
struct arm_smmu_cmdq_ent cmd = {
.tlbi = {
.leaf = leaf,
@@ -2242,18 +2264,11 @@ static void arm_smmu_tlb_inv_range(unsigned long iova, size_t size,
}
while (iova < end) {
- if (i == CMDQ_BATCH_ENTRIES) {
- arm_smmu_cmdq_issue_cmdlist(smmu, cmds, i, false);
- i = 0;
- }
-
cmd.tlbi.addr = iova;
- arm_smmu_cmdq_build_cmd(&cmds[i * CMDQ_ENT_DWORDS], &cmd);
+ arm_smmu_cmdq_batch_add(smmu, &cmds, &cmd);
iova += granule;
- i++;
}
-
- arm_smmu_cmdq_issue_cmdlist(smmu, cmds, i, true);
+ arm_smmu_cmdq_batch_submit(smmu, &cmds);
/*
* Unfortunately, this can't be leaf-only since we may have
--
2.25.0
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply related [flat|nested] 9+ messages in thread
* [PATCH v2 5/6] iommu/arm-smmu-v3: Batch context descriptor invalidation
2020-02-24 16:58 [PATCH v2 0/6] iommu/arm-smmu-v3: Finish PASID support and command queue batching Jean-Philippe Brucker
` (3 preceding siblings ...)
2020-02-24 16:58 ` [PATCH v2 4/6] iommu/arm-smmu-v3: Add command queue batching helpers Jean-Philippe Brucker
@ 2020-02-24 16:58 ` Jean-Philippe Brucker
2020-02-24 16:58 ` [PATCH v2 6/6] iommu/arm-smmu-v3: Batch ATC invalidation commands Jean-Philippe Brucker
5 siblings, 0 replies; 9+ messages in thread
From: Jean-Philippe Brucker @ 2020-02-24 16:58 UTC (permalink / raw)
To: linux-pci, linux-arm-kernel, iommu, will, bhelgaas
Cc: zhangfei.gao, robin.murphy, robh
Rather than publishing one command at a time when invalidating a context
descriptor, batch the commands for all SIDs in the domain.
Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
---
drivers/iommu/arm-smmu-v3.c | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c
index beeec366bc41..12b2a0fa747e 100644
--- a/drivers/iommu/arm-smmu-v3.c
+++ b/drivers/iommu/arm-smmu-v3.c
@@ -1512,6 +1512,7 @@ static void arm_smmu_sync_cd(struct arm_smmu_domain *smmu_domain,
size_t i;
unsigned long flags;
struct arm_smmu_master *master;
+ struct arm_smmu_cmdq_batch cmds = {};
struct arm_smmu_device *smmu = smmu_domain->smmu;
struct arm_smmu_cmdq_ent cmd = {
.opcode = CMDQ_OP_CFGI_CD,
@@ -1525,12 +1526,12 @@ static void arm_smmu_sync_cd(struct arm_smmu_domain *smmu_domain,
list_for_each_entry(master, &smmu_domain->devices, domain_head) {
for (i = 0; i < master->num_sids; i++) {
cmd.cfgi.sid = master->sids[i];
- arm_smmu_cmdq_issue_cmd(smmu, &cmd);
+ arm_smmu_cmdq_batch_add(smmu, &cmds, &cmd);
}
}
spin_unlock_irqrestore(&smmu_domain->devices_lock, flags);
- arm_smmu_cmdq_issue_sync(smmu);
+ arm_smmu_cmdq_batch_submit(smmu, &cmds);
}
static int arm_smmu_alloc_cd_leaf_table(struct arm_smmu_device *smmu,
--
2.25.0
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply related [flat|nested] 9+ messages in thread
* [PATCH v2 6/6] iommu/arm-smmu-v3: Batch ATC invalidation commands
2020-02-24 16:58 [PATCH v2 0/6] iommu/arm-smmu-v3: Finish PASID support and command queue batching Jean-Philippe Brucker
` (4 preceding siblings ...)
2020-02-24 16:58 ` [PATCH v2 5/6] iommu/arm-smmu-v3: Batch context descriptor invalidation Jean-Philippe Brucker
@ 2020-02-24 16:58 ` Jean-Philippe Brucker
5 siblings, 0 replies; 9+ messages in thread
From: Jean-Philippe Brucker @ 2020-02-24 16:58 UTC (permalink / raw)
To: linux-pci, linux-arm-kernel, iommu, will, bhelgaas
Cc: zhangfei.gao, robin.murphy, robh
From: Rob Herring <robh@kernel.org>
Similar to commit 2af2e72b18b4 ("iommu/arm-smmu-v3: Defer TLB
invalidation until ->iotlb_sync()"), build up a list of ATC invalidation
commands and submit them all at once to the command queue instead of
one-by-one.
As there is only one caller of arm_smmu_atc_inv_master() left, we can
simplify it and avoid passing in struct arm_smmu_cmdq_ent.
Cc: Jean-Philippe Brucker <jean-philippe@linaro.org>
Cc: Will Deacon <will@kernel.org>
Cc: Robin Murphy <robin.murphy@arm.com>
Cc: Joerg Roedel <joro@8bytes.org>
Signed-off-by: Rob Herring <robh@kernel.org>
Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
---
drivers/iommu/arm-smmu-v3.c | 31 ++++++++++++++++++-------------
1 file changed, 18 insertions(+), 13 deletions(-)
diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c
index 12b2a0fa747e..4f0a38dae6db 100644
--- a/drivers/iommu/arm-smmu-v3.c
+++ b/drivers/iommu/arm-smmu-v3.c
@@ -2158,17 +2158,16 @@ arm_smmu_atc_inv_to_cmd(int ssid, unsigned long iova, size_t size,
cmd->atc.size = log2_span;
}
-static int arm_smmu_atc_inv_master(struct arm_smmu_master *master,
- struct arm_smmu_cmdq_ent *cmd)
+static int arm_smmu_atc_inv_master(struct arm_smmu_master *master)
{
int i;
+ struct arm_smmu_cmdq_ent cmd;
- if (!master->ats_enabled)
- return 0;
+ arm_smmu_atc_inv_to_cmd(0, 0, 0, &cmd);
for (i = 0; i < master->num_sids; i++) {
- cmd->atc.sid = master->sids[i];
- arm_smmu_cmdq_issue_cmd(master->smmu, cmd);
+ cmd.atc.sid = master->sids[i];
+ arm_smmu_cmdq_issue_cmd(master->smmu, &cmd);
}
return arm_smmu_cmdq_issue_sync(master->smmu);
@@ -2177,10 +2176,11 @@ static int arm_smmu_atc_inv_master(struct arm_smmu_master *master,
static int arm_smmu_atc_inv_domain(struct arm_smmu_domain *smmu_domain,
int ssid, unsigned long iova, size_t size)
{
- int ret = 0;
+ int i;
unsigned long flags;
struct arm_smmu_cmdq_ent cmd;
struct arm_smmu_master *master;
+ struct arm_smmu_cmdq_batch cmds = {};
if (!(smmu_domain->smmu->features & ARM_SMMU_FEAT_ATS))
return 0;
@@ -2205,11 +2205,18 @@ static int arm_smmu_atc_inv_domain(struct arm_smmu_domain *smmu_domain,
arm_smmu_atc_inv_to_cmd(ssid, iova, size, &cmd);
spin_lock_irqsave(&smmu_domain->devices_lock, flags);
- list_for_each_entry(master, &smmu_domain->devices, domain_head)
- ret |= arm_smmu_atc_inv_master(master, &cmd);
+ list_for_each_entry(master, &smmu_domain->devices, domain_head) {
+ if (!master->ats_enabled)
+ continue;
+
+ for (i = 0; i < master->num_sids; i++) {
+ cmd.atc.sid = master->sids[i];
+ arm_smmu_cmdq_batch_add(smmu_domain->smmu, &cmds, &cmd);
+ }
+ }
spin_unlock_irqrestore(&smmu_domain->devices_lock, flags);
- return ret ? -ETIMEDOUT : 0;
+ return arm_smmu_cmdq_batch_submit(smmu_domain->smmu, &cmds);
}
/* IO_PGTABLE API */
@@ -2629,7 +2636,6 @@ static void arm_smmu_enable_ats(struct arm_smmu_master *master)
static void arm_smmu_disable_ats(struct arm_smmu_master *master)
{
- struct arm_smmu_cmdq_ent cmd;
struct arm_smmu_domain *smmu_domain = master->domain;
if (!master->ats_enabled)
@@ -2641,8 +2647,7 @@ static void arm_smmu_disable_ats(struct arm_smmu_master *master)
* ATC invalidation via the SMMU.
*/
wmb();
- arm_smmu_atc_inv_to_cmd(0, 0, 0, &cmd);
- arm_smmu_atc_inv_master(master, &cmd);
+ arm_smmu_atc_inv_master(master);
atomic_dec(&smmu_domain->nr_ats_masters);
}
--
2.25.0
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply related [flat|nested] 9+ messages in thread