From: Jean-Philippe Brucker <jean-philippe@linaro.org> To: Guo Ren <guoren@kernel.org>, Will Deacon <will@kernel.org> Cc: aou@eecs.berkeley.edu, Linux Kernel Mailing List <linux-kernel@vger.kernel.org>, Arnd Bergmann <arnd@arndb.de>, Marc Zyngier <marc.zyngier@arm.com>, Catalin Marinas <catalin.marinas@arm.com>, Palmer Dabbelt <palmer@sifive.com>, iommu@lists.linux-foundation.org, Mike Rapoport <rppt@linux.ibm.com>, Anup Patel <anup.Patel@wdc.com>, Atish Patra <Atish.Patra@wdc.com>, Julien Grall <julien.grall@arm.com>, gary@garyguo.net, Paul Walmsley <paul.walmsley@sifive.com>, linux-riscv@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org Subject: Re: [PATCH RFC 11/14] arm64: Move the ASID allocator code in a separate file Date: Mon, 16 Sep 2019 14:57:16 +0200 Message-ID: <057a0af3-93f7-271c-170e-4b31e6894c3c@linaro.org> (raw) In-Reply-To: <CAJF2gTTsHCsSpf1ncVb=ZJS2d=r+AdDi2=5z-REVS=uUg9138A@mail.gmail.com> Hi, On 13/09/2019 09:13, Guo Ren wrote: > Another idea is seperate remote TLB invalidate into two instructions: > > - sfence.vma.b.asyc > - sfence.vma.b.barrier // wait all async TLB invalidate operations > finished for all harts. It's not clear to me how this helps, but I probably don't have the whole picture. If you have a place where it is safe to wait for the barrier to complete, why not do the whole invalidate there? > (I remember who mentioned me separate them into two instructions after > session. Anup? Is the idea right ?) > > Actually, I never consider asyc TLB invalidate before, because current our > light iommu did not need it. > > Thx all people attend the session :) Let's continue the talk. > > > Guo Ren <guoren@kernel.org <mailto:guoren@kernel.org>> 于 2019年9月12日周 > 四 22:59写道: > > Thx Will for reply. > > On Thu, Sep 12, 2019 at 3:03 PM Will Deacon <will@kernel.org > <mailto:will@kernel.org>> wrote: > > > > On Sun, Sep 08, 2019 at 07:52:55AM +0800, Guo Ren wrote: > > > On Mon, Jun 24, 2019 at 6:40 PM Will Deacon <will@kernel.org > <mailto:will@kernel.org>> wrote: > > > > > I'll keep my system use the same ASID for SMP + IOMMU :P > > > > > > > > You will want a separate allocator for that: > > > > > > > > > https://lkml.kernel.org/r/20190610184714.6786-2-jean-philippe.brucker@arm.com > > > > > > Yes, it is hard to maintain ASID between IOMMU and CPUMMU or different > > > system, because it's difficult to synchronize the IO_ASID when the CPU > > > ASID is rollover. > > > But we could still use hardware broadcast TLB invalidation instruction > > > to uniformly manage the ASID and IO_ASID, or OTHER_ASID in our IOMMU. > > > > That's probably a bad idea, because you'll likely stall execution on the > > CPU until the IOTLB has completed invalidation. In the case of ATS, > I think > > an endpoint ATC is permitted to take over a minute to respond. In > reality, I > > suspect the worst you'll ever see would be in the msec range, but that's > > still an unacceptable period of time to hold a CPU. > Just as I've said in the session that IOTLB invalidate delay is > another topic, My main proposal is to introduce stage1.pgd and > stage2.pgd as address space identifiers between different TLB systems > based on vmid, asid. My last part of sildes will show you how to > translate stage1/2.pgd to as/vmid in PCI ATS system and the method > could work with SMMU-v3 and intel Vt-d. (It's regret for me there is > no time to show you the whole slides.) > > In our light IOMMU implementation, there's no IOTLB invalidate delay > problem. Becasue IOMMU is very close to CPU MMU and interconnect's > delay is the same with SMP CPUs MMU (no PCI, VM supported). > > To solve the problem, we could define a async mode in sfence.vma.b to > slove the problem and finished with per_cpu_irq/exception. The solution I had to this problem is pinning the ASID [1] used by the IOMMU, to prevent the CPU from recycling the ASID on rollover. This way the CPU doesn't have to wait for IOMMU invalidations to complete, when scheduling a task that might not even have anything to do with the IOMMU. In the Arm SMMU, ASID and IOASID (PASID) are separate identifiers. IOASID indexes an entry in the context descriptor table, which contains the ASID. So with unpinned shared ASID you don't need to invalidate the ATC on rollover, since the IOASID doesn't change, but you do need to modify the context descriptor and invalidate cached versions of it. Once you have pinned ASIDs, you could also declare that IOASID = ASID. I don't remember finding an argument to strictly forbid it, even though ASID and IOASID have different sizes on Arm (respectively 8/16 and 20 bits). Thanks, Jean [1] https://lore.kernel.org/linux-iommu/20180511190641.23008-17-jean-philippe.brucker@arm.com/ _______________________________________________ kvmarm mailing list kvmarm@lists.cs.columbia.edu https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
next prev parent reply index Thread overview: 56+ messages / expand[flat|nested] mbox.gz Atom feed top 2019-03-21 16:36 [PATCH RFC 00/14] kvm/arm: Align the VMID allocation with the arm64 ASID one Julien Grall 2019-03-21 16:36 ` [PATCH RFC 01/14] arm64/mm: Introduce asid_info structure and move asid_generation/asid_map to it Julien Grall 2019-03-21 17:03 ` Suzuki K Poulose 2019-03-21 17:27 ` Julien Grall 2019-03-21 16:36 ` [PATCH RFC 02/14] arm64/mm: Move active_asids and reserved_asids to asid_info Julien Grall 2019-03-21 16:36 ` [PATCH RFC 03/14] arm64/mm: Move bits " Julien Grall 2019-03-21 16:36 ` [PATCH RFC 04/14] arm64/mm: Move the variable lock and tlb_flush_pending " Julien Grall 2019-03-21 16:36 ` [PATCH RFC 05/14] arm64/mm: Remove dependency on MM in new_context Julien Grall 2019-03-21 16:36 ` [PATCH RFC 06/14] arm64/mm: Store the number of asid allocated per context Julien Grall 2019-03-21 16:36 ` [PATCH RFC 07/14] arm64/mm: Introduce NUM_ASIDS Julien Grall 2019-03-21 16:36 ` [PATCH RFC 08/14] arm64/mm: Split asid_inits in 2 parts Julien Grall 2019-03-21 16:36 ` [PATCH RFC 09/14] arm64/mm: Split the function check_and_switch_context in 3 parts Julien Grall 2019-03-21 16:36 ` [PATCH RFC 10/14] arm64/mm: Introduce a callback to flush the local context Julien Grall 2019-03-21 16:36 ` [PATCH RFC 11/14] arm64: Move the ASID allocator code in a separate file Julien Grall 2019-06-05 16:56 ` Julien Grall 2019-06-05 20:41 ` Palmer Dabbelt 2019-06-11 1:56 ` Gary Guo 2019-06-19 8:07 ` Guo Ren 2019-06-19 8:54 ` Julien Grall 2019-06-19 9:12 ` Will Deacon 2019-06-19 12:18 ` Guo Ren 2019-06-19 12:39 ` Will Deacon 2019-06-20 9:33 ` Guo Ren 2019-06-24 10:40 ` Will Deacon 2019-06-25 7:25 ` Palmer Dabbelt 2019-09-07 23:52 ` Guo Ren 2019-09-12 14:02 ` Will Deacon 2019-09-12 14:59 ` Guo Ren 2019-09-13 7:13 ` Guo Ren 2019-09-14 8:49 ` Guo Ren 2019-09-16 12:57 ` Jean-Philippe Brucker [this message] 2019-09-19 13:07 ` Guo Ren 2019-09-19 15:18 ` Jean-Philippe Brucker 2019-09-20 0:07 ` Guo Ren 2019-09-20 7:18 ` Jean-Philippe Brucker 2019-09-14 14:01 ` Palmer Dabbelt 2019-09-15 5:03 ` Anup Patel 2019-09-16 18:18 ` Will Deacon 2019-09-16 18:28 ` Palmer Dabbelt 2019-09-17 3:42 ` Anup Patel 2019-09-19 13:36 ` Guo Ren 2019-06-19 11:51 ` Guo Ren 2019-06-19 12:52 ` Julien Grall 2019-06-21 14:16 ` Catalin Marinas 2019-06-23 16:35 ` Guo Ren 2019-06-24 10:22 ` Will Deacon 2019-06-27 9:41 ` qi.fuli 2019-06-27 10:26 ` Will Deacon 2019-06-24 15:38 ` Catalin Marinas 2019-06-30 4:29 ` Guo Ren 2019-07-01 9:17 ` Catalin Marinas 2019-07-16 3:31 ` Guo Ren 2019-07-22 16:38 ` Catalin Marinas 2019-03-21 16:36 ` [PATCH RFC 12/14] arm64/lib: asid: Allow user to update the context under the lock Julien Grall 2019-03-21 16:36 ` [PATCH RFC 13/14] arm/kvm: Introduce a new VMID allocator Julien Grall 2019-03-21 16:36 ` [PATCH RFC 14/14] kvm/arm: Align the VMID allocation with the arm64 ASID one Julien Grall
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=057a0af3-93f7-271c-170e-4b31e6894c3c@linaro.org \ --to=jean-philippe@linaro.org \ --cc=Atish.Patra@wdc.com \ --cc=anup.Patel@wdc.com \ --cc=aou@eecs.berkeley.edu \ --cc=arnd@arndb.de \ --cc=catalin.marinas@arm.com \ --cc=gary@garyguo.net \ --cc=guoren@kernel.org \ --cc=iommu@lists.linux-foundation.org \ --cc=julien.grall@arm.com \ --cc=kvmarm@lists.cs.columbia.edu \ --cc=linux-arm-kernel@lists.infradead.org \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-riscv@lists.infradead.org \ --cc=marc.zyngier@arm.com \ --cc=palmer@sifive.com \ --cc=paul.walmsley@sifive.com \ --cc=rppt@linux.ibm.com \ --cc=will@kernel.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
KVM ARM Archive on lore.kernel.org Archives are clonable: git clone --mirror https://lore.kernel.org/kvmarm/0 kvmarm/git/0.git # If you have public-inbox 1.1+ installed, you may # initialize and index your mirror using the following commands: public-inbox-init -V2 kvmarm kvmarm/ https://lore.kernel.org/kvmarm \ kvmarm@lists.cs.columbia.edu public-inbox-index kvmarm Example config snippet for mirrors Newsgroup available over NNTP: nntp://nntp.lore.kernel.org/edu.columbia.cs.lists.kvmarm AGPL code for this site: git clone https://public-inbox.org/public-inbox.git