IOMMU Archive on lore.kernel.org
 help / color / Atom feed
From: Guo Ren <guoren@kernel.org>
To: Will Deacon <will@kernel.org>
Cc: julien.thierry@arm.com, Catalin Marinas <catalin.marinas@arm.com>,
	Palmer Dabbelt <palmer@sifive.com>,
	Will Deacon <will.deacon@arm.com>,
	christoffer.dall@arm.com, Atish Patra <Atish.Patra@wdc.com>,
	Julien Grall <julien.grall@arm.com>,
	gary@garyguo.net, linux-riscv@lists.infradead.org,
	kvmarm@lists.cs.columbia.edu, Mike Rapoport <rppt@linux.ibm.com>,
	aou@eecs.berkeley.edu, Arnd Bergmann <arnd@arndb.de>,
	suzuki.poulose@arm.com, Marc Zyngier <marc.zyngier@arm.com>,
	Paul Walmsley <paul.walmsley@sifive.com>,
	linux-arm-kernel@lists.infradead.org,
	Anup Patel <anup.Patel@wdc.com>,
	Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
	iommu@lists.linux-foundation.org, james.morse@arm.com
Subject: Re: [PATCH RFC 11/14] arm64: Move the ASID allocator code in a separate file
Date: Fri, 13 Sep 2019 15:13:10 +0800
Message-ID: <CAJF2gTTsHCsSpf1ncVb=ZJS2d=r+AdDi2=5z-REVS=uUg9138A@mail.gmail.com> (raw)
In-Reply-To: <CAJF2gTT2c45HRfATF+=zs-HNToFAKgq1inKRmJMV3uPYBo4iVg@mail.gmail.com>

[-- Attachment #1.1: Type: text/plain, Size: 2641 bytes --]

Another idea is seperate remote TLB invalidate into two instructions:

 - sfence.vma.b.asyc
 - sfence.vma.b.barrier // wait all async TLB invalidate operations
finished for all harts.

(I remember who mentioned me separate them into two instructions after
session. Anup? Is the idea right ?)

Actually, I never consider asyc TLB invalidate before, because current our
light iommu did not need it.

Thx all people attend the session :) Let's continue the talk.


Guo Ren <guoren@kernel.org> 于 2019年9月12日周四 22:59写道:

> Thx Will for reply.
>
> On Thu, Sep 12, 2019 at 3:03 PM Will Deacon <will@kernel.org> wrote:
> >
> > On Sun, Sep 08, 2019 at 07:52:55AM +0800, Guo Ren wrote:
> > > On Mon, Jun 24, 2019 at 6:40 PM Will Deacon <will@kernel.org> wrote:
> > > > > I'll keep my system use the same ASID for SMP + IOMMU :P
> > > >
> > > > You will want a separate allocator for that:
> > > >
> > > >
> https://lkml.kernel.org/r/20190610184714.6786-2-jean-philippe.brucker@arm.com
> > >
> > > Yes, it is hard to maintain ASID between IOMMU and CPUMMU or different
> > > system, because it's difficult to synchronize the IO_ASID when the CPU
> > > ASID is rollover.
> > > But we could still use hardware broadcast TLB invalidation instruction
> > > to uniformly manage the ASID and IO_ASID, or OTHER_ASID in our IOMMU.
> >
> > That's probably a bad idea, because you'll likely stall execution on the
> > CPU until the IOTLB has completed invalidation. In the case of ATS, I
> think
> > an endpoint ATC is permitted to take over a minute to respond. In
> reality, I
> > suspect the worst you'll ever see would be in the msec range, but that's
> > still an unacceptable period of time to hold a CPU.
> Just as I've said in the session that IOTLB invalidate delay is
> another topic, My main proposal is to introduce stage1.pgd and
> stage2.pgd as address space identifiers between different TLB systems
> based on vmid, asid. My last part of sildes will show you how to
> translate stage1/2.pgd to as/vmid in PCI ATS system and the method
> could work with SMMU-v3 and intel Vt-d. (It's regret for me there is
> no time to show you the whole slides.)
>
> In our light IOMMU implementation, there's no IOTLB invalidate delay
> problem. Becasue IOMMU is very close to CPU MMU and interconnect's
> delay is the same with SMP CPUs MMU (no PCI, VM supported).
>
> To solve the problem, we could define a async mode in sfence.vma.b to
> slove the problem and finished with per_cpu_irq/exception.
>
> --
> Best Regards
>  Guo Ren
>
> ML: https://lore.kernel.org/linux-csky/
>

[-- Attachment #1.2: Type: text/html, Size: 3733 bytes --]

<div dir="auto"><div>Another idea is seperate remote TLB invalidate into two instructions:<div dir="auto"><br></div><div dir="auto"> - sfence.vma.b.asyc</div><div dir="auto"> - sfence.vma.b.barrier  // wait all async TLB invalidate operations finished for all harts.</div><div dir="auto"><br></div><div dir="auto">(I remember who mentioned me separate them into two instructions after session. Anup? Is the idea right ?) </div><div dir="auto"><br></div><div dir="auto">Actually,  I never consider asyc TLB invalidate before,  because current our light iommu did not need it.</div><div dir="auto"><br></div><div dir="auto">Thx all people attend the session :)  Let&#39;s continue the talk. </div><br><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">Guo Ren &lt;<a href="mailto:guoren@kernel.org">guoren@kernel.org</a>&gt; 于 2019年9月12日周四 22:59写道:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Thx Will for reply.<br>
<br>
On Thu, Sep 12, 2019 at 3:03 PM Will Deacon &lt;<a href="mailto:will@kernel.org" target="_blank" rel="noreferrer">will@kernel.org</a>&gt; wrote:<br>
&gt;<br>
&gt; On Sun, Sep 08, 2019 at 07:52:55AM +0800, Guo Ren wrote:<br>
&gt; &gt; On Mon, Jun 24, 2019 at 6:40 PM Will Deacon &lt;<a href="mailto:will@kernel.org" target="_blank" rel="noreferrer">will@kernel.org</a>&gt; wrote:<br>
&gt; &gt; &gt; &gt; I&#39;ll keep my system use the same ASID for SMP + IOMMU :P<br>
&gt; &gt; &gt;<br>
&gt; &gt; &gt; You will want a separate allocator for that:<br>
&gt; &gt; &gt;<br>
&gt; &gt; &gt; <a href="https://lkml.kernel.org/r/20190610184714.6786-2-jean-philippe.brucker@arm.com" rel="noreferrer noreferrer" target="_blank">https://lkml.kernel.org/r/20190610184714.6786-2-jean-philippe.brucker@arm.com</a><br>
&gt; &gt;<br>
&gt; &gt; Yes, it is hard to maintain ASID between IOMMU and CPUMMU or different<br>
&gt; &gt; system, because it&#39;s difficult to synchronize the IO_ASID when the CPU<br>
&gt; &gt; ASID is rollover.<br>
&gt; &gt; But we could still use hardware broadcast TLB invalidation instruction<br>
&gt; &gt; to uniformly manage the ASID and IO_ASID, or OTHER_ASID in our IOMMU.<br>
&gt;<br>
&gt; That&#39;s probably a bad idea, because you&#39;ll likely stall execution on the<br>
&gt; CPU until the IOTLB has completed invalidation. In the case of ATS, I think<br>
&gt; an endpoint ATC is permitted to take over a minute to respond. In reality, I<br>
&gt; suspect the worst you&#39;ll ever see would be in the msec range, but that&#39;s<br>
&gt; still an unacceptable period of time to hold a CPU.<br>
Just as I&#39;ve said in the session that IOTLB invalidate delay is<br>
another topic, My main proposal is to introduce stage1.pgd and<br>
stage2.pgd as address space identifiers between different TLB systems<br>
based on vmid, asid. My last part of sildes will show you how to<br>
translate stage1/2.pgd to as/vmid in PCI ATS system and the method<br>
could work with SMMU-v3 and intel Vt-d. (It&#39;s regret for me there is<br>
no time to show you the whole slides.)<br>
<br>
In our light IOMMU implementation, there&#39;s no IOTLB invalidate delay<br>
problem. Becasue IOMMU is very close to CPU MMU and interconnect&#39;s<br>
delay is the same with SMP CPUs MMU (no PCI, VM supported).<br>
<br>
To solve the problem, we could define a async mode in sfence.vma.b to<br>
slove the problem and finished with per_cpu_irq/exception.<br>
<br>
-- <br>
Best Regards<br>
 Guo Ren<br>
<br>
ML: <a href="https://lore.kernel.org/linux-csky/" rel="noreferrer noreferrer" target="_blank">https://lore.kernel.org/linux-csky/</a><br>
</blockquote></div></div></div>

[-- Attachment #2: Type: text/plain, Size: 156 bytes --]

_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

  reply index

Thread overview: 6+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
     [not found] <20190321163623.20219-1-julien.grall@arm.com>
     [not found] ` <20190321163623.20219-12-julien.grall@arm.com>
     [not found]   ` <0dfe120b-066a-2ac8-13bc-3f5a29e2caa3@arm.com>
     [not found]     ` <CAJF2gTTXHHgDboaexdHA284y6kNZVSjLis5-Q2rDnXCxr4RSmA@mail.gmail.com>
     [not found]       ` <c871a5ae-914f-a8bb-9474-1dcfec5d45bf@arm.com>
     [not found]         ` <20190619091219.GB7767@fuggles.cambridge.arm.com>
     [not found]           ` <CAJF2gTTmFq3yYa9UrdZRAFwJgC=KmKTe2_NFy_UZBUQovqQJPg@mail.gmail.com>
     [not found]             ` <20190619123939.GF7767@fuggles.cambridge.arm.com>
     [not found]               ` <CAJF2gTSiiiewTLwVAXvPLO7rTSUw1rg8VtFLzANdP2S2EEbTjg@mail.gmail.com>
     [not found]                 ` <20190624104006.lvm32nahemaqklxc@willie-the-truck>
2019-09-07 23:52                   ` Guo Ren
2019-09-12 14:02                     ` Will Deacon
2019-09-12 14:59                       ` Guo Ren
2019-09-13  7:13                         ` Guo Ren [this message]
2019-09-14  8:49                           ` Guo Ren
2019-09-14 14:01                       ` Palmer Dabbelt

Reply instructions:

You may reply publically to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='CAJF2gTTsHCsSpf1ncVb=ZJS2d=r+AdDi2=5z-REVS=uUg9138A@mail.gmail.com' \
    --to=guoren@kernel.org \
    --cc=Atish.Patra@wdc.com \
    --cc=anup.Patel@wdc.com \
    --cc=aou@eecs.berkeley.edu \
    --cc=arnd@arndb.de \
    --cc=catalin.marinas@arm.com \
    --cc=christoffer.dall@arm.com \
    --cc=gary@garyguo.net \
    --cc=iommu@lists.linux-foundation.org \
    --cc=james.morse@arm.com \
    --cc=julien.grall@arm.com \
    --cc=julien.thierry@arm.com \
    --cc=kvmarm@lists.cs.columbia.edu \
    --cc=linux-arm-kernel@lists.infradead.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-riscv@lists.infradead.org \
    --cc=marc.zyngier@arm.com \
    --cc=palmer@sifive.com \
    --cc=paul.walmsley@sifive.com \
    --cc=rppt@linux.ibm.com \
    --cc=suzuki.poulose@arm.com \
    --cc=will.deacon@arm.com \
    --cc=will@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

IOMMU Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-iommu/0 linux-iommu/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-iommu linux-iommu/ https://lore.kernel.org/linux-iommu \
		iommu@lists.linux-foundation.org iommu@archiver.kernel.org
	public-inbox-index linux-iommu


Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.linux-foundation.lists.iommu


AGPL code for this site: git clone https://public-inbox.org/ public-inbox