From: Rob Clark <email@example.com>
To: Will Deacon <firstname.lastname@example.org>, Rob Clark <email@example.com>,
Linux Kernel Mailing List <firstname.lastname@example.org>,
"email@example.com:IOMMU DRIVERS <firstname.lastname@example.org>,
Joerg Roedel <email@example.com>,
Robin Murphy <firstname.lastname@example.org>,
"moderated list:ARM/FREESCALE IMX / MXC ARM ARCHITECTURE"
Subject: Re: [PATCH v1 2/6] arm/smmu: Add auxiliary domain support for arm-smmuv2
Date: Wed, 20 May 2020 09:35:42 -0700 [thread overview]
Message-ID: <CAF6AEGtF+_Q3=GxaVrAtqsPC3B6p_Dc5PB+DQ=btsg3wAej5bg@mail.gmail.com> (raw)
On Wed, May 20, 2020 at 8:13 AM Jordan Crouse <email@example.com> wrote:
> On Wed, May 20, 2020 at 01:57:01PM +0100, Will Deacon wrote:
> > On Mon, May 18, 2020 at 08:50:27AM -0700, Rob Clark wrote:
> > > On Mon, May 18, 2020 at 8:18 AM Will Deacon <firstname.lastname@example.org> wrote:
> > > > On Wed, Mar 18, 2020 at 04:43:07PM -0700, Rob Clark wrote:
> > > > > We do in fact need live domain switching, that is really the whole
> > > > > point. The GPU CP (command processor/parser) is directly updating
> > > > > TTBR0 and triggering TLB flush, asynchronously from the CPU.
> > > > >
> > > > > And I think the answer about ASID is easy (on current hw).. it must be zero[*].
> > > >
> > > > Using ASID zero is really bad, because it means that you will end up sharing
> > > > TLB entries with whichever device is using context bank 0.
> > > >
> > > > Is the SMMU only used by the GPU in your SoC?
> > > >
> > >
> > > yes, the snapdragon SoCs have two SMMU instances, one used by the GPU,
> > > where ASID0/cb0 is the gpu itself, and another cb is the GMU
> > > (basically power control for the gpu), and the second SMMU is
> > > everything else.
> > Right, in which case I'm starting to think that we should treat this GPU
> > SMMU instance specially. Give it its own compatible string (looks like you
> > need this for HUPCFG anyway) and hook in via arm_smmu_impl_init(). You can
> > then set IO_PGTABLE_QUIRK_ARM_TTBR1 when talking to the io-pgtable code
> > without having to add a domain attribute.
> If we did this via a special GPU SMMU instance then we could also create and
> register a dummy TTBR0 instance along with the TTBR1 instance and then we
> wouldn't need to worry about the aux domains at all.
> > With that. you'll need to find a way to allow the GPU driver to call into
> > your own hooks for getting at the TTBR0 tables -- given that you're
> > programming these in the hardware, I don't think it makes sense to expose
> > that in the IOMMU API, since most devices won't be able to do anything with
> > that data. Perhaps you could install a couple of function pointers
> > (subdomain_alloc/subdomain_free) in the GPU device when you see it appear
> > from the SMMU driver? Alternatively, you could make an io_pgtable_cfg
> > available so that the GPU driver can interface with io-pgtable directly.
> I don't want to speak for Rob but I think that this is the same direction we've
> landed on. If we use the implementation specific code to initialize the base
> pagetables then the GPU driver can use io-pgtable directly. We can easily
> construct an io_pgtable_cfg. This feature will only be available for opt-in
> GPU targets that will have a known configuration.
Agreed about using io-pgtable helpers directly.. the gpu's use-case is
pretty far different from anything normal/sane, and I don't think it
is worth designing some generic iommu interfaces with precisely one
user[*]. We just need enough in arm-smmu(/-impl) to bootstrap things
when we power up the gpu.
[*] all the other gpu's that I've seen so far, even if they sit behind
an iommu, they have their own internal mmu
> The only gotcha is TLB maintenance but Rob and I have ideas about coordinating
> with the GPU hardware (which has to do a TLBIALL during a switch anyway) and we
> can always use the iommu_tlb_flush_all() hammer from software if we really need
> it. It might take a bit of thought, but it is doable.
> > Yes, it's ugly, but I don't think it's worth trying to abstract this.
> I'm not sure how ugly it is. I've always operated under the assumption that the
> GPU SMMU was special (though it had generic registers) just because of where it
> was and how it it was used. In the long run baking in a implementation specific
> solution would probably be preferable to lots of domain attributes and aux
> domains that would never be used except by us.
> > Thoughts? It's taken me a long time to figure out what's going on here,
> > so sorry if it feels like I'm leading you round the houses.
> I'll hack on this and try to get something in place. It might be dumber on the
> GPU side than we would like but it would at least spur some more conversation.
> > Will
> The Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
> a Linux Foundation Collaborative Project
iommu mailing list
next prev parent reply other threads:[~2020-05-20 16:35 UTC|newest]
Thread overview: 15+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-01-28 22:16 [PATCH v1 0/6] iommu/arm-smmu: Auxiliary domain and per instance pagetables Jordan Crouse
2020-01-28 22:16 ` [PATCH v1 1/6] iommu: Add DOMAIN_ATTR_PTBASE Jordan Crouse
2020-01-28 22:16 ` [PATCH v1 2/6] arm/smmu: Add auxiliary domain support for arm-smmuv2 Jordan Crouse
2020-03-18 22:48 ` Will Deacon
2020-03-18 23:43 ` Rob Clark
2020-03-19 15:23 ` Jordan Crouse
2020-05-18 15:18 ` Will Deacon
2020-05-18 15:50 ` Rob Clark
2020-05-20 12:57 ` Will Deacon
2020-05-20 15:13 ` Jordan Crouse
2020-05-20 16:35 ` Rob Clark [this message]
2020-01-28 22:16 ` [PATCH v1 3/6] drm/msm/adreno: ADd support for IOMMU auxiliary domains Jordan Crouse
2020-01-28 22:16 ` [PATCH v1 4/6] drm/msm: Add support to create target specific address spaces Jordan Crouse
2020-01-28 22:16 ` [PATCH v1 5/6] drm/msm/gpu: Add ttbr0 to the memptrs Jordan Crouse
2020-01-28 22:16 ` [PATCH v1 6/6] drm/msm/a6xx: Support per-instance pagetables Jordan Crouse
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).