KVM ARM Archive on lore.kernel.org
 help / color / Atom feed
From: Guo Ren <guoren@kernel.org>
To: Will Deacon <will@kernel.org>
Cc: Catalin Marinas <catalin.marinas@arm.com>,
	Palmer Dabbelt <palmer@sifive.com>,
	Will Deacon <will.deacon@arm.com>,
	Atish Patra <Atish.Patra@wdc.com>,
	Julien Grall <julien.grall@arm.com>,
	gary@garyguo.net, linux-riscv@lists.infradead.org,
	kvmarm@lists.cs.columbia.edu, Mike Rapoport <rppt@linux.ibm.com>,
	Christoph Hellwig <hch@infradead.org>,
	aou@eecs.berkeley.edu, Arnd Bergmann <arnd@arndb.de>,
	Marc Zyngier <marc.zyngier@arm.com>,
	Paul Walmsley <paul.walmsley@sifive.com>,
	linux-arm-kernel@lists.infradead.org,
	Anup Patel <anup.Patel@wdc.com>,
	Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
	iommu@lists.linux-foundation.org
Subject: Re: [PATCH RFC 11/14] arm64: Move the ASID allocator code in a separate file
Date: Fri, 13 Sep 2019 15:13:10 +0800
Message-ID: <CAJF2gTTsHCsSpf1ncVb=ZJS2d=r+AdDi2=5z-REVS=uUg9138A@mail.gmail.com> (raw)
In-Reply-To: <CAJF2gTT2c45HRfATF+=zs-HNToFAKgq1inKRmJMV3uPYBo4iVg@mail.gmail.com>

[-- Attachment #1.1: Type: text/plain, Size: 2641 bytes --]

Another idea is seperate remote TLB invalidate into two instructions:

 - sfence.vma.b.asyc
 - sfence.vma.b.barrier // wait all async TLB invalidate operations
finished for all harts.

(I remember who mentioned me separate them into two instructions after
session. Anup? Is the idea right ?)

Actually, I never consider asyc TLB invalidate before, because current our
light iommu did not need it.

Thx all people attend the session :) Let's continue the talk.


Guo Ren <guoren@kernel.org> 于 2019年9月12日周四 22:59写道:

> Thx Will for reply.
>
> On Thu, Sep 12, 2019 at 3:03 PM Will Deacon <will@kernel.org> wrote:
> >
> > On Sun, Sep 08, 2019 at 07:52:55AM +0800, Guo Ren wrote:
> > > On Mon, Jun 24, 2019 at 6:40 PM Will Deacon <will@kernel.org> wrote:
> > > > > I'll keep my system use the same ASID for SMP + IOMMU :P
> > > >
> > > > You will want a separate allocator for that:
> > > >
> > > >
> https://lkml.kernel.org/r/20190610184714.6786-2-jean-philippe.brucker@arm.com
> > >
> > > Yes, it is hard to maintain ASID between IOMMU and CPUMMU or different
> > > system, because it's difficult to synchronize the IO_ASID when the CPU
> > > ASID is rollover.
> > > But we could still use hardware broadcast TLB invalidation instruction
> > > to uniformly manage the ASID and IO_ASID, or OTHER_ASID in our IOMMU.
> >
> > That's probably a bad idea, because you'll likely stall execution on the
> > CPU until the IOTLB has completed invalidation. In the case of ATS, I
> think
> > an endpoint ATC is permitted to take over a minute to respond. In
> reality, I
> > suspect the worst you'll ever see would be in the msec range, but that's
> > still an unacceptable period of time to hold a CPU.
> Just as I've said in the session that IOTLB invalidate delay is
> another topic, My main proposal is to introduce stage1.pgd and
> stage2.pgd as address space identifiers between different TLB systems
> based on vmid, asid. My last part of sildes will show you how to
> translate stage1/2.pgd to as/vmid in PCI ATS system and the method
> could work with SMMU-v3 and intel Vt-d. (It's regret for me there is
> no time to show you the whole slides.)
>
> In our light IOMMU implementation, there's no IOTLB invalidate delay
> problem. Becasue IOMMU is very close to CPU MMU and interconnect's
> delay is the same with SMP CPUs MMU (no PCI, VM supported).
>
> To solve the problem, we could define a async mode in sfence.vma.b to
> slove the problem and finished with per_cpu_irq/exception.
>
> --
> Best Regards
>  Guo Ren
>
> ML: https://lore.kernel.org/linux-csky/
>

[-- Attachment #1.2: Type: text/html, Size: 3733 bytes --]

<div dir="auto"><div>Another idea is seperate remote TLB invalidate into two instructions:<div dir="auto"><br></div><div dir="auto"> - sfence.vma.b.asyc</div><div dir="auto"> - sfence.vma.b.barrier  // wait all async TLB invalidate operations finished for all harts.</div><div dir="auto"><br></div><div dir="auto">(I remember who mentioned me separate them into two instructions after session. Anup? Is the idea right ?) </div><div dir="auto"><br></div><div dir="auto">Actually,  I never consider asyc TLB invalidate before,  because current our light iommu did not need it.</div><div dir="auto"><br></div><div dir="auto">Thx all people attend the session :)  Let&#39;s continue the talk. </div><br><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">Guo Ren &lt;<a href="mailto:guoren@kernel.org">guoren@kernel.org</a>&gt; 于 2019年9月12日周四 22:59写道:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Thx Will for reply.<br>
<br>
On Thu, Sep 12, 2019 at 3:03 PM Will Deacon &lt;<a href="mailto:will@kernel.org" target="_blank" rel="noreferrer">will@kernel.org</a>&gt; wrote:<br>
&gt;<br>
&gt; On Sun, Sep 08, 2019 at 07:52:55AM +0800, Guo Ren wrote:<br>
&gt; &gt; On Mon, Jun 24, 2019 at 6:40 PM Will Deacon &lt;<a href="mailto:will@kernel.org" target="_blank" rel="noreferrer">will@kernel.org</a>&gt; wrote:<br>
&gt; &gt; &gt; &gt; I&#39;ll keep my system use the same ASID for SMP + IOMMU :P<br>
&gt; &gt; &gt;<br>
&gt; &gt; &gt; You will want a separate allocator for that:<br>
&gt; &gt; &gt;<br>
&gt; &gt; &gt; <a href="https://lkml.kernel.org/r/20190610184714.6786-2-jean-philippe.brucker@arm.com" rel="noreferrer noreferrer" target="_blank">https://lkml.kernel.org/r/20190610184714.6786-2-jean-philippe.brucker@arm.com</a><br>
&gt; &gt;<br>
&gt; &gt; Yes, it is hard to maintain ASID between IOMMU and CPUMMU or different<br>
&gt; &gt; system, because it&#39;s difficult to synchronize the IO_ASID when the CPU<br>
&gt; &gt; ASID is rollover.<br>
&gt; &gt; But we could still use hardware broadcast TLB invalidation instruction<br>
&gt; &gt; to uniformly manage the ASID and IO_ASID, or OTHER_ASID in our IOMMU.<br>
&gt;<br>
&gt; That&#39;s probably a bad idea, because you&#39;ll likely stall execution on the<br>
&gt; CPU until the IOTLB has completed invalidation. In the case of ATS, I think<br>
&gt; an endpoint ATC is permitted to take over a minute to respond. In reality, I<br>
&gt; suspect the worst you&#39;ll ever see would be in the msec range, but that&#39;s<br>
&gt; still an unacceptable period of time to hold a CPU.<br>
Just as I&#39;ve said in the session that IOTLB invalidate delay is<br>
another topic, My main proposal is to introduce stage1.pgd and<br>
stage2.pgd as address space identifiers between different TLB systems<br>
based on vmid, asid. My last part of sildes will show you how to<br>
translate stage1/2.pgd to as/vmid in PCI ATS system and the method<br>
could work with SMMU-v3 and intel Vt-d. (It&#39;s regret for me there is<br>
no time to show you the whole slides.)<br>
<br>
In our light IOMMU implementation, there&#39;s no IOTLB invalidate delay<br>
problem. Becasue IOMMU is very close to CPU MMU and interconnect&#39;s<br>
delay is the same with SMP CPUs MMU (no PCI, VM supported).<br>
<br>
To solve the problem, we could define a async mode in sfence.vma.b to<br>
slove the problem and finished with per_cpu_irq/exception.<br>
<br>
-- <br>
Best Regards<br>
 Guo Ren<br>
<br>
ML: <a href="https://lore.kernel.org/linux-csky/" rel="noreferrer noreferrer" target="_blank">https://lore.kernel.org/linux-csky/</a><br>
</blockquote></div></div></div>

[-- Attachment #2: Type: text/plain, Size: 151 bytes --]

_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

  reply index

Thread overview: 51+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-03-21 16:36 [PATCH RFC 00/14] kvm/arm: Align the VMID allocation with the arm64 ASID one Julien Grall
2019-03-21 16:36 ` [PATCH RFC 01/14] arm64/mm: Introduce asid_info structure and move asid_generation/asid_map to it Julien Grall
2019-03-21 17:03   ` Suzuki K Poulose
2019-03-21 17:27     ` Julien Grall
2019-03-21 16:36 ` [PATCH RFC 02/14] arm64/mm: Move active_asids and reserved_asids to asid_info Julien Grall
2019-03-21 16:36 ` [PATCH RFC 03/14] arm64/mm: Move bits " Julien Grall
2019-03-21 16:36 ` [PATCH RFC 04/14] arm64/mm: Move the variable lock and tlb_flush_pending " Julien Grall
2019-03-21 16:36 ` [PATCH RFC 05/14] arm64/mm: Remove dependency on MM in new_context Julien Grall
2019-03-21 16:36 ` [PATCH RFC 06/14] arm64/mm: Store the number of asid allocated per context Julien Grall
2019-03-21 16:36 ` [PATCH RFC 07/14] arm64/mm: Introduce NUM_ASIDS Julien Grall
2019-03-21 16:36 ` [PATCH RFC 08/14] arm64/mm: Split asid_inits in 2 parts Julien Grall
2019-03-21 16:36 ` [PATCH RFC 09/14] arm64/mm: Split the function check_and_switch_context in 3 parts Julien Grall
2019-03-21 16:36 ` [PATCH RFC 10/14] arm64/mm: Introduce a callback to flush the local context Julien Grall
2019-03-21 16:36 ` [PATCH RFC 11/14] arm64: Move the ASID allocator code in a separate file Julien Grall
2019-06-05 16:56   ` Julien Grall
2019-06-05 20:41     ` Palmer Dabbelt
2019-06-11  1:56       ` Gary Guo
2019-06-19  8:07     ` Guo Ren
2019-06-19  8:54       ` Julien Grall
2019-06-19  9:12         ` Will Deacon
2019-06-19 12:18           ` Guo Ren
2019-06-19 12:39             ` Will Deacon
2019-06-20  9:33               ` Guo Ren
2019-06-24 10:40                 ` Will Deacon
2019-06-25  7:25                   ` Palmer Dabbelt
2019-09-07 23:52                   ` Guo Ren
2019-09-12 14:02                     ` Will Deacon
2019-09-12 14:59                       ` Guo Ren
2019-09-13  7:13                         ` Guo Ren [this message]
2019-09-14  8:49                           ` Guo Ren
2019-09-16 12:57                           ` Jean-Philippe Brucker
2019-09-14 14:01                       ` Palmer Dabbelt
2019-09-15  5:03                         ` Anup Patel
2019-09-16 18:18                           ` Will Deacon
2019-09-16 18:28                             ` Palmer Dabbelt
2019-09-17  3:42                             ` Anup Patel
2019-06-19 11:51         ` Guo Ren
2019-06-19 12:52           ` Julien Grall
2019-06-21 14:16           ` Catalin Marinas
2019-06-23 16:35             ` Guo Ren
2019-06-24 10:22               ` Will Deacon
2019-06-27  9:41                 ` qi.fuli
2019-06-27 10:26                   ` Will Deacon
2019-06-24 15:38               ` Catalin Marinas
2019-06-30  4:29                 ` Guo Ren
2019-07-01  9:17                   ` Catalin Marinas
2019-07-16  3:31                     ` Guo Ren
2019-07-22 16:38                       ` Catalin Marinas
2019-03-21 16:36 ` [PATCH RFC 12/14] arm64/lib: asid: Allow user to update the context under the lock Julien Grall
2019-03-21 16:36 ` [PATCH RFC 13/14] arm/kvm: Introduce a new VMID allocator Julien Grall
2019-03-21 16:36 ` [PATCH RFC 14/14] kvm/arm: Align the VMID allocation with the arm64 ASID one Julien Grall

Reply instructions:

You may reply publically to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='CAJF2gTTsHCsSpf1ncVb=ZJS2d=r+AdDi2=5z-REVS=uUg9138A@mail.gmail.com' \
    --to=guoren@kernel.org \
    --cc=Atish.Patra@wdc.com \
    --cc=anup.Patel@wdc.com \
    --cc=aou@eecs.berkeley.edu \
    --cc=arnd@arndb.de \
    --cc=catalin.marinas@arm.com \
    --cc=gary@garyguo.net \
    --cc=hch@infradead.org \
    --cc=iommu@lists.linux-foundation.org \
    --cc=julien.grall@arm.com \
    --cc=kvmarm@lists.cs.columbia.edu \
    --cc=linux-arm-kernel@lists.infradead.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-riscv@lists.infradead.org \
    --cc=marc.zyngier@arm.com \
    --cc=palmer@sifive.com \
    --cc=paul.walmsley@sifive.com \
    --cc=rppt@linux.ibm.com \
    --cc=will.deacon@arm.com \
    --cc=will@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

KVM ARM Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/kvmarm/0 kvmarm/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 kvmarm kvmarm/ https://lore.kernel.org/kvmarm \
		kvmarm@lists.cs.columbia.edu kvmarm@archiver.kernel.org
	public-inbox-index kvmarm


Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/edu.columbia.cs.lists.kvmarm


AGPL code for this site: git clone https://public-inbox.org/ public-inbox