From: Jean-Philippe Brucker <jean-philippe@linaro.org>
To: Will Deacon <will@kernel.org>
Cc: linux-pci@vger.kernel.org, linux-arm-kernel@lists.infradead.org,
linux-acpi@vger.kernel.org, devicetree@vger.kernel.org,
iommu@lists.linux-foundation.org, joro@8bytes.org,
robh+dt@kernel.org, mark.rutland@arm.com,
lorenzo.pieralisi@arm.com, guohanjun@huawei.com,
sudeep.holla@arm.com, rjw@rjwysocki.net, lenb@kernel.org,
robin.murphy@arm.com, bhelgaas@google.com, eric.auger@redhat.com,
jonathan.cameron@huawei.com, zhangfei.gao@linaro.org
Subject: Re: [PATCH v4 07/13] iommu/arm-smmu-v3: Add support for Substream IDs
Date: Tue, 14 Jan 2020 17:30:09 +0100 [thread overview]
Message-ID: <20200114163009.GA5319@myrica> (raw)
In-Reply-To: <20200114123819.GC29222@willie-the-truck>
On Tue, Jan 14, 2020 at 12:38:19PM +0000, Will Deacon wrote:
> > +static void arm_smmu_sync_cd(struct arm_smmu_domain *smmu_domain,
> > + int ssid, bool leaf)
> > +{
> > + size_t i;
> > + unsigned long flags;
> > + struct arm_smmu_master *master;
> > + struct arm_smmu_device *smmu = smmu_domain->smmu;
> > + struct arm_smmu_cmdq_ent cmd = {
> > + .opcode = CMDQ_OP_CFGI_CD,
> > + .cfgi = {
> > + .ssid = ssid,
> > + .leaf = leaf,
> > + },
> > + };
> > +
> > + spin_lock_irqsave(&smmu_domain->devices_lock, flags);
> > + list_for_each_entry(master, &smmu_domain->devices, domain_head) {
> > + for (i = 0; i < master->num_sids; i++) {
> > + cmd.cfgi.sid = master->sids[i];
> > + arm_smmu_cmdq_issue_cmd(smmu, &cmd);
> > + }
> > + }
> > + spin_unlock_irqrestore(&smmu_domain->devices_lock, flags);
> > +
> > + arm_smmu_cmdq_issue_sync(smmu);
>
> Can you send a follow-up patch converting this to batch submission, please?
Ok
> > +}
> > +
> > static int arm_smmu_alloc_cd_leaf_table(struct arm_smmu_device *smmu,
> > struct arm_smmu_cd_table *table,
> > size_t num_entries)
> > @@ -1498,34 +1541,65 @@ static u64 arm_smmu_cpu_tcr_to_cd(u64 tcr)
> > return val;
> > }
> >
> > -static void arm_smmu_write_ctx_desc(struct arm_smmu_device *smmu,
> > - struct arm_smmu_s1_cfg *cfg)
> > +static int arm_smmu_write_ctx_desc(struct arm_smmu_domain *smmu_domain,
> > + int ssid, struct arm_smmu_ctx_desc *cd)
> > {
> > - u64 val;
> > - __le64 *cdptr = cfg->table.ptr;
> > -
> > /*
> > - * We don't need to issue any invalidation here, as we'll invalidate
> > - * the STE when installing the new entry anyway.
> > + * This function handles the following cases:
> > + *
> > + * (1) Install primary CD, for normal DMA traffic (SSID = 0).
> > + * (2) Install a secondary CD, for SID+SSID traffic.
> > + * (3) Update ASID of a CD. Atomically write the first 64 bits of the
> > + * CD, then invalidate the old entry and mappings.
> > + * (4) Remove a secondary CD.
> > */
> > - val = arm_smmu_cpu_tcr_to_cd(cfg->cd.tcr) |
> > -#ifdef __BIG_ENDIAN
> > - CTXDESC_CD_0_ENDI |
> > -#endif
> > - CTXDESC_CD_0_R | CTXDESC_CD_0_A | CTXDESC_CD_0_ASET |
> > - CTXDESC_CD_0_AA64 | FIELD_PREP(CTXDESC_CD_0_ASID, cfg->cd.asid) |
> > - CTXDESC_CD_0_V;
> > + u64 val;
> > + bool cd_live;
> > + struct arm_smmu_device *smmu = smmu_domain->smmu;
> > + __le64 *cdptr = smmu_domain->s1_cfg.table.ptr + ssid *
> > + CTXDESC_CD_DWORDS;
> >
> > - /* STALL_MODEL==0b10 && CD.S==0 is ILLEGAL */
> > - if (smmu->features & ARM_SMMU_FEAT_STALL_FORCE)
> > - val |= CTXDESC_CD_0_S;
> > + val = le64_to_cpu(cdptr[0]);
> > + cd_live = !!(val & CTXDESC_CD_0_V);
> >
> > - cdptr[0] = cpu_to_le64(val);
> > + if (!cd) { /* (4) */
> > + val = 0;
> > + } else if (cd_live) { /* (3) */
> > + val &= ~CTXDESC_CD_0_ASID;
> > + val |= FIELD_PREP(CTXDESC_CD_0_ASID, cd->asid);
> > + /*
> > + * Until CD+TLB invalidation, both ASIDs may be used for tagging
> > + * this substream's traffic
> > + */
>
> I don't think you need to change anything here, but I do find it a little
> scary that we can modify live CDs like this. However, given that the
> hardware is permitted to cache the structures regardless of validity, it
> appears to be the only option. Terrifying!
>
> > + } else { /* (1) and (2) */
> > + cdptr[1] = cpu_to_le64(cd->ttbr & CTXDESC_CD_1_TTB0_MASK);
>
> Can you use FIELD_PREP here too?
No, FIELD_PREP will shift ttbr left by 4 bits
> > + cdptr[2] = 0;
> > + cdptr[3] = cpu_to_le64(cd->mair);
> > +
> > + /*
> > + * STE is live, and the SMMU might read dwords of this CD in any
> > + * order. Ensure that it observes valid values before reading
> > + * V=1.
> > + */
> > + arm_smmu_sync_cd(smmu_domain, ssid, true);
> >
> > - val = cfg->cd.ttbr & CTXDESC_CD_1_TTB0_MASK;
> > - cdptr[1] = cpu_to_le64(val);
> > + val = arm_smmu_cpu_tcr_to_cd(cd->tcr) |
> > +#ifdef __BIG_ENDIAN
> > + CTXDESC_CD_0_ENDI |
> > +#endif
> > + CTXDESC_CD_0_R | CTXDESC_CD_0_A | CTXDESC_CD_0_ASET |
> > + CTXDESC_CD_0_AA64 |
> > + FIELD_PREP(CTXDESC_CD_0_ASID, cd->asid) |
> > + CTXDESC_CD_0_V;
> >
> > - cdptr[3] = cpu_to_le64(cfg->cd.mair);
> > + /* STALL_MODEL==0b10 && CD.S==0 is ILLEGAL */
> > + if (smmu->features & ARM_SMMU_FEAT_STALL_FORCE)
> > + val |= CTXDESC_CD_0_S;
> > + }
> > +
> > + WRITE_ONCE(cdptr[0], cpu_to_le64(val));
>
> Can you add a comment here citing 3.21.3 ("Configuration structures and
> configuration invalidation completion") please? Specifically, the note that
> states:
>
> | The size of single-copy atomic reads made by the SMMU is IMPLEMENTATION
> | DEFINED but must be at least 64 bits.
>
> Because that's really crucial to the WRITE_ONCE() above!
>
> Shouldn't we also do the same thing for the STE side of things? I think so,
> and you can just comment of them with the quote and cite the comment from
> the other callsite.
Yes, makes sense
Thanks,
Jean
next prev parent reply other threads:[~2020-01-14 16:30 UTC|newest]
Thread overview: 34+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-12-19 16:30 [PATCH v4 00/13] iommu: Add PASID support to Arm SMMUv3 Jean-Philippe Brucker
2019-12-19 16:30 ` [PATCH v4 01/13] iommu/arm-smmu-v3: Drop __GFP_ZERO flag from DMA allocation Jean-Philippe Brucker
2019-12-19 16:30 ` [PATCH v4 02/13] dt-bindings: document PASID property for IOMMU masters Jean-Philippe Brucker
2019-12-19 16:30 ` [PATCH v4 03/13] iommu/arm-smmu-v3: Parse PASID devicetree property of platform devices Jean-Philippe Brucker
2019-12-19 16:30 ` [PATCH v4 04/13] ACPI/IORT: Parse SSID property of named component node Jean-Philippe Brucker
2019-12-19 16:30 ` [PATCH v4 05/13] iommu/arm-smmu-v3: Prepare arm_smmu_s1_cfg for SSID support Jean-Philippe Brucker
2019-12-19 16:30 ` [PATCH v4 06/13] iommu/arm-smmu-v3: Add context descriptor tables allocators Jean-Philippe Brucker
2020-01-14 11:06 ` Will Deacon
2020-01-14 11:52 ` Jean-Philippe Brucker
2020-01-14 11:56 ` Will Deacon
2019-12-19 16:30 ` [PATCH v4 07/13] iommu/arm-smmu-v3: Add support for Substream IDs Jean-Philippe Brucker
2020-01-14 12:38 ` Will Deacon
2020-01-14 16:30 ` Jean-Philippe Brucker [this message]
2019-12-19 16:30 ` [PATCH v4 08/13] iommu/arm-smmu-v3: Propagate ssid_bits Jean-Philippe Brucker
2019-12-19 16:30 ` [PATCH v4 09/13] iommu/arm-smmu-v3: Prepare for handling arm_smmu_write_ctx_desc() failure Jean-Philippe Brucker
2020-01-14 12:42 ` Will Deacon
2020-01-14 16:31 ` Jean-Philippe Brucker
2019-12-19 16:30 ` [PATCH v4 10/13] iommu/arm-smmu-v3: Add second level of context descriptor table Jean-Philippe Brucker
2019-12-20 7:37 ` Auger Eric
2020-01-14 15:04 ` Will Deacon
2020-01-15 9:45 ` Jean-Philippe Brucker
2019-12-19 16:30 ` [PATCH v4 11/13] iommu/arm-smmu-v3: Improve add_device() error handling Jean-Philippe Brucker
2020-01-14 15:25 ` Will Deacon
2020-01-15 16:17 ` Will Deacon
2020-01-15 17:44 ` Robin Murphy
2019-12-19 16:30 ` [PATCH v4 12/13] PCI/ATS: Add PASID stubs Jean-Philippe Brucker
2019-12-19 16:30 ` [PATCH v4 13/13] iommu/arm-smmu-v3: Add support for PCI PASID Jean-Philippe Brucker
2019-12-20 7:37 ` Auger Eric
2020-01-14 12:45 ` Will Deacon
2020-01-15 7:57 ` Jean-Philippe Brucker
2020-01-09 14:36 ` [PATCH v4 00/13] iommu: Add PASID support to Arm SMMUv3 Jean-Philippe Brucker
2020-01-09 14:41 ` Will Deacon
2020-01-10 7:15 ` Jean-Philippe Brucker
2020-01-14 15:40 ` Will Deacon
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200114163009.GA5319@myrica \
--to=jean-philippe@linaro.org \
--cc=bhelgaas@google.com \
--cc=devicetree@vger.kernel.org \
--cc=eric.auger@redhat.com \
--cc=guohanjun@huawei.com \
--cc=iommu@lists.linux-foundation.org \
--cc=jonathan.cameron@huawei.com \
--cc=joro@8bytes.org \
--cc=lenb@kernel.org \
--cc=linux-acpi@vger.kernel.org \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-pci@vger.kernel.org \
--cc=lorenzo.pieralisi@arm.com \
--cc=mark.rutland@arm.com \
--cc=rjw@rjwysocki.net \
--cc=robh+dt@kernel.org \
--cc=robin.murphy@arm.com \
--cc=sudeep.holla@arm.com \
--cc=will@kernel.org \
--cc=zhangfei.gao@linaro.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).