From: Will Deacon <will@kernel.org>
To: Guo Ren <guoren@kernel.org>
Cc: Julien Grall <julien.grall@arm.com>,
aou@eecs.berkeley.edu, Arnd Bergmann <arnd@arndb.de>,
Marc Zyngier <marc.zyngier@arm.com>,
catalin.marinas@arm.com, Will Deacon <will.deacon@arm.com>,
linux-kernel@vger.kernel.org, rppt@linux.ibm.com,
hch@infradead.org, Atish.Patra@wdc.com,
Anup Patel <anup.Patel@wdc.com>,
gary@garyguo.net, Palmer Dabbelt <palmer@sifive.com>,
paul.walmsley@sifive.com, linux-riscv@lists.infradead.org,
kvmarm@lists.cs.columbia.edu,
linux-arm-kernel@lists.infradead.org
Subject: Re: [PATCH RFC 11/14] arm64: Move the ASID allocator code in a separate file
Date: Mon, 24 Jun 2019 11:40:07 +0100 [thread overview]
Message-ID: <20190624104006.lvm32nahemaqklxc@willie-the-truck> (raw)
In-Reply-To: <CAJF2gTSiiiewTLwVAXvPLO7rTSUw1rg8VtFLzANdP2S2EEbTjg@mail.gmail.com>
On Thu, Jun 20, 2019 at 05:33:03PM +0800, Guo Ren wrote:
> On Wed, Jun 19, 2019 at 8:39 PM Will Deacon <will.deacon@arm.com> wrote:
> >
> > On Wed, Jun 19, 2019 at 08:18:04PM +0800, Guo Ren wrote:
> > > On Wed, Jun 19, 2019 at 5:12 PM Will Deacon <will.deacon@arm.com> wrote:
> > > > This is one place where I'd actually prefer not to go down the route of
> > > > making the code generic. Context-switching and low-level TLB management
> > > > is deeply architecture-specific and I worry that by trying to make this
> > > > code common, we run the real risk of introducing subtle bugs on some
> > > > architecture every time it is changed.
> > > "Add generic asid code" and "move arm's into generic" are two things.
> > > We could do
> > > first and let architecture's maintainer to choose.
> >
> > If I understand the proposal being discussed, it involves basing that
> > generic ASID allocation code around the arm64 implementation which I don't
> > necessarily think is a good starting point.
> ...
> >
> > > > Furthermore, the algorithm we use
> > > > on arm64 is designed to scale to large systems using DVM and may well be
> > > > too complex and/or sub-optimal for architectures with different system
> > > > topologies or TLB invalidation mechanisms.
> > > It's just a asid algorithm not very complex and there is a callback
> > > for architecture to define their
> > > own local hart tlb flush. Seems it has nothing with DVM or tlb
> > > broadcast mechanism.
> >
> > I'm pleased that you think the algorithm is not very complex, but I'm also
> > worried that you might not have fully understood some of its finer details.
> I understand your concern about my less understanding of asid
> technology. Here is
> my short-description of arm64 asid allocator: (If you find anything
> wrong, please
> correct me directly, thx :)
The complexity mainly comes from the fact that this thing runs concurrently
with itself without synchronization on the fast-path. Coupled with the
need to use the same ASID for all threads of a task, you end up in fiddly
situations where rollover can occur on one CPU whilst another CPU is trying
to schedule a thread of a task that already has threads running in
userspace.
However, it's architecture-specific whether or not you care about that
scenario.
> > The reason I mention DVM and TLB broadcasting is because, depending on
> > the mechanisms in your architecture relating to those, it may be strictly
> > required that all concurrently running threads of a process have the same
> > ASID at any given point in time, or it may be that you really don't care.
> >
> > If you don't care, then the arm64 allocator is over-engineered and likely
> > inefficient for your system. If you do care, then it's worth considering
> > whether a lock is sufficient around the allocator if you don't expect high
> > core counts. Another possibility is that you end up using only one ASID and
> > invalidating the local TLB on every context switch. Yet another design
> > would be to manage per-cpu ASID pools.
> I'll keep my system use the same ASID for SMP + IOMMU :P
You will want a separate allocator for that:
https://lkml.kernel.org/r/20190610184714.6786-2-jean-philippe.brucker@arm.com
> Yes, there are two styles of asid allocator: per-cpu ASID (MIPS) or
> same ASID (ARM).
> If the CPU couldn't support cache/tlb coherency maintian in hardware,
> it should use
> per-cpu ASID style because IPI is expensive and per-cpu ASID style
> need more software
> mechanism to improve performance (eg: delay cache flush). From software view the
> same ASID is clearer and easier to build bigger system with more TLB caches.
>
> I think the same ASID style is a more sensible choice for modern
> processor and let it be
> one of generic is reasonable.
I'm not sure I agree. x86, for example, is better off using a different
algorithm for allocating its PCIDs.
> > So rather than blindly copying the arm64 code, I suggest sitting down and
> > designing something that fits to your architecture instead. You may end up
> > with something that is both simpler and more efficient.
> In fact, riscv folks have discussed a lot about arm's asid allocator
> and I learned
> a lot from the discussion:
> https://lore.kernel.org/linux-riscv/20190327100201.32220-1-anup.patel@wdc.com/
If you require all threads of the same process to have the same ASID, then
that patch looks broken to me.
Will
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
next prev parent reply other threads:[~2019-06-24 11:20 UTC|newest]
Thread overview: 56+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-03-21 16:36 [PATCH RFC 00/14] kvm/arm: Align the VMID allocation with the arm64 ASID one Julien Grall
2019-03-21 16:36 ` [PATCH RFC 01/14] arm64/mm: Introduce asid_info structure and move asid_generation/asid_map to it Julien Grall
2019-03-21 17:03 ` Suzuki K Poulose
2019-03-21 17:27 ` Julien Grall
2019-03-21 16:36 ` [PATCH RFC 02/14] arm64/mm: Move active_asids and reserved_asids to asid_info Julien Grall
2019-03-21 16:36 ` [PATCH RFC 03/14] arm64/mm: Move bits " Julien Grall
2019-03-21 16:36 ` [PATCH RFC 04/14] arm64/mm: Move the variable lock and tlb_flush_pending " Julien Grall
2019-03-21 16:36 ` [PATCH RFC 05/14] arm64/mm: Remove dependency on MM in new_context Julien Grall
2019-03-21 16:36 ` [PATCH RFC 06/14] arm64/mm: Store the number of asid allocated per context Julien Grall
2019-03-21 16:36 ` [PATCH RFC 07/14] arm64/mm: Introduce NUM_ASIDS Julien Grall
2019-03-21 16:36 ` [PATCH RFC 08/14] arm64/mm: Split asid_inits in 2 parts Julien Grall
2019-03-21 16:36 ` [PATCH RFC 09/14] arm64/mm: Split the function check_and_switch_context in 3 parts Julien Grall
2019-03-21 16:36 ` [PATCH RFC 10/14] arm64/mm: Introduce a callback to flush the local context Julien Grall
2019-03-21 16:36 ` [PATCH RFC 11/14] arm64: Move the ASID allocator code in a separate file Julien Grall
2019-06-05 16:56 ` Julien Grall
2019-06-05 20:41 ` Palmer Dabbelt
2019-06-11 1:56 ` Gary Guo
2019-06-19 8:07 ` Guo Ren
2019-06-19 8:54 ` Julien Grall
2019-06-19 9:12 ` Will Deacon
2019-06-19 12:18 ` Guo Ren
2019-06-19 12:39 ` Will Deacon
2019-06-20 9:33 ` Guo Ren
2019-06-24 10:40 ` Will Deacon [this message]
2019-06-25 7:25 ` Palmer Dabbelt
2019-09-07 23:52 ` Guo Ren
2019-09-12 14:02 ` Will Deacon
2019-09-12 14:59 ` Guo Ren
2019-09-13 7:13 ` Guo Ren
2019-09-14 8:49 ` Guo Ren
2019-09-16 12:57 ` Jean-Philippe Brucker
2019-09-19 13:07 ` Guo Ren
2019-09-19 15:18 ` Jean-Philippe Brucker
2019-09-20 0:07 ` Guo Ren
2019-09-20 7:18 ` Jean-Philippe Brucker
2019-09-14 14:01 ` Palmer Dabbelt
2019-09-15 5:03 ` Anup Patel
2019-09-16 18:18 ` Will Deacon
2019-09-16 18:28 ` Palmer Dabbelt
2019-09-17 3:42 ` Anup Patel
2019-09-19 13:36 ` Guo Ren
2019-06-19 11:51 ` Guo Ren
2019-06-19 12:52 ` Julien Grall
2019-06-21 14:16 ` Catalin Marinas
2019-06-23 16:35 ` Guo Ren
2019-06-24 10:22 ` Will Deacon
2019-06-27 9:41 ` qi.fuli
2019-06-27 10:26 ` Will Deacon
2019-06-24 15:38 ` Catalin Marinas
2019-06-30 4:29 ` Guo Ren
2019-07-01 9:17 ` Catalin Marinas
2019-07-16 3:31 ` Guo Ren
2019-07-22 16:38 ` Catalin Marinas
2019-03-21 16:36 ` [PATCH RFC 12/14] arm64/lib: asid: Allow user to update the context under the lock Julien Grall
2019-03-21 16:36 ` [PATCH RFC 13/14] arm/kvm: Introduce a new VMID allocator Julien Grall
2019-03-21 16:36 ` [PATCH RFC 14/14] kvm/arm: Align the VMID allocation with the arm64 ASID one Julien Grall
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20190624104006.lvm32nahemaqklxc@willie-the-truck \
--to=will@kernel.org \
--cc=Atish.Patra@wdc.com \
--cc=anup.Patel@wdc.com \
--cc=aou@eecs.berkeley.edu \
--cc=arnd@arndb.de \
--cc=catalin.marinas@arm.com \
--cc=gary@garyguo.net \
--cc=guoren@kernel.org \
--cc=hch@infradead.org \
--cc=julien.grall@arm.com \
--cc=kvmarm@lists.cs.columbia.edu \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-riscv@lists.infradead.org \
--cc=marc.zyngier@arm.com \
--cc=palmer@sifive.com \
--cc=paul.walmsley@sifive.com \
--cc=rppt@linux.ibm.com \
--cc=will.deacon@arm.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).