From: Ley Foon Tan <lftan.linux@gmail.com>
To: Greentime Hu <greentime.hu@sifive.com>
Cc: Darius Rad <darius@bluespec.com>,
linux-riscv <linux-riscv@lists.infradead.org>,
Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
Albert Ou <aou@eecs.berkeley.edu>,
Palmer Dabbelt <palmer@dabbelt.com>,
Paul Walmsley <paul.walmsley@sifive.com>,
Vincent Chen <vincent.chen@sifive.com>
Subject: Re: [RFC PATCH v8 09/21] riscv: Add task switch support for vector
Date: Thu, 7 Oct 2021 18:10:26 +0800 [thread overview]
Message-ID: <CAFiDJ5-GwxCOOW4O6NL=8UCubt0j6NzxaBT7Cm5yBmcFA_RmyA@mail.gmail.com> (raw)
In-Reply-To: <CAHCEehLooHmL9Zhg4Bsr2Ea7wvMZkLzMv9093p9KvYJv6H4DAw@mail.gmail.com>
On Tue, Oct 5, 2021 at 11:47 PM Greentime Hu <greentime.hu@sifive.com> wrote:
>
> Ley Foon Tan <lftan.linux@gmail.com> 於 2021年10月5日 週二 上午10:12寫道:
> >
> > On Mon, Oct 4, 2021 at 8:41 PM Greentime Hu <greentime.hu@sifive.com> wrote:
> > >
> > > Ley Foon Tan <lftan.linux@gmail.com> 於 2021年10月1日 週五 上午10:46寫道:
> > > >
> > > > On Wed, Sep 29, 2021 at 11:54 PM Darius Rad <darius@bluespec.com> wrote:
> > > > >
> > > > > On Tue, Sep 28, 2021 at 10:56:52PM +0800, Greentime Hu wrote:
> > > > > > Darius Rad <darius@bluespec.com> 於 2021年9月13日 週一 下午8:21寫道:
> > > > > > >
> > [....]
> >
> >
> > > > > > >
> > > > > > > So this will unconditionally enable vector instructions, and allocate
> > > > > > > memory for vector state, for all processes, regardless of whether vector
> > > > > > > instructions are used?
> > > > > > >
> > > > > >
> > > > > > Hi Darius,
> > > > > >
> > > > > > Yes, it will enable vector if has_vector() is true. The reason that we
> > > > > > choose to enable and allocate memory for user space program is because
> > > > > > we also implement some common functions in the glibc such as memcpy
> > > > > > vector version and it is called very often by every process. So that
> > > > > > we assume if the user program is running in a CPU with vector ISA
> > > > > > would like to use vector by default. If we disable it by default and
> > > > > > make it trigger the illegal instruction, that might be a burden since
> > > > > > almost every process will use vector glibc memcpy or something like
> > > > > > that.
> > > > >
> > > > > Do you have any evidence to support the assertion that almost every process
> > > > > would use vector operations? One could easily argue that the converse is
> > > > > true: no existing software uses the vector extension now, so most likely a
> > > > > process will not be using it.
> > > > >
> > > > > >
> > > > > > > Given the size of the vector state and potential power and performance
> > > > > > > implications of enabling the vector engine, it seems like this should
> > > > > > > treated similarly to Intel AMX on x86. The full discussion of that is
> > > > > > > here:
> > > > > > >
> > > > > > > https://lore.kernel.org/lkml/CALCETrW2QHa2TLvnUuVxAAheqcbSZ-5_WRXtDSAGcbG8N+gtdQ-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org/
> > > > > > >
> > > > > > > The cover letter for recent Intel AMX patches has a summary of the x86
> > > > > > > implementation:
> > > > > > >
> > > > > > > https://lore.kernel.org/lkml/20210825155413.19673-1-chang.seok.bae@intel.com/
> > > > > > >
> > > > > > > If RISC-V were to adopt a similar approach, I think the significant
> > > > > > > points are:
> > > > > > >
> > > > > > > 1. A process (or thread) must specifically request the desire to use
> > > > > > > vector extensions (perhaps with some new arch_prctl() API),
> > > > > > >
> > > > > > > 2. The kernel is free to deny permission, perhaps based on
> > > > > > > administrative rules or for other reasons, and
> > > > > > >
> > > > > > > 3. If a process attempts to use vector extensions before doing the
> > > > > > > above, the process will die due to an illegal instruction.
> > > > > >
> > > > > > Thank you for sharing this, but I am not sure if we should treat
> > > > > > vector like AMX on x86. IMHO, compiler might generate code with vector
> > > > > > instructions automatically someday, maybe we should treat vector
> > > > > > extensions like other extensions.
> > > > > > If user knows the vector extension is supported in this CPU and he
> > > > > > would like to use it, it seems we should let user use it directly just
> > > > > > like other extensions.
> > > > > > If user don't know it exists or not, user should use the library API
> > > > > > transparently and let glibc or other library deal with it. The glibc
> > > > > > ifunc feature or multi-lib should be able to choose the correct
> > > > > > implementation.
> > > > >
> > > > > What makes me think that the vector extension should be treated like AMX is
> > > > > that they both (1) have a significant amount of architectural state, and
> > > > > (2) likely have a significant power and/or area impact on (non-emulated)
> > > > > designs.
> > > > >
> > > > > For example, I think it is possible, maybe even likely, that vector
> > > > > implementations will have one or more of the following behaviors:
> > > > >
> > > > > 1. A single vector unit shared among two or more harts,
> > > > >
> > > > > 2. Additional power consumption when the vector unit is enabled and idle
> > > > > versus not being enabled at all,
> > > > >
> > > > > 3. For a system which supports variable operating frequency, a reduction
> > > > > in the maximum frequency when the vector unit is enabled, and/or
> > > > >
> > > > > 4. The inability to enter low power states and/or delays to low power
> > > > > states transitions when the vector unit is enabled.
> > > > >
> > > > > None of the above constraints apply to more ordinary extensions like
> > > > > compressed or the various bit manipulation extensions.
> > > > >
> > > > > The discussion I linked to has some well reasoned arguments on why
> > > > > substantial extensions should have a mechanism to request using them by
> > > > > user space. The discussion was in the context of Intel AMX, but applies to
> > > > > further x86 extensions, and I think should also apply to similar extensions
> > > > > on RISC-V, like vector here.
> > > > >
> > > > There is possible use case where not all cores support vector
> > > > extension due to size, area and power.
> > > > Perhaps can have the mechanism or flow to determine the
> > > > application/thread require vector extension or it specifically request
> > > > the desire to use
> > > > vector extensions. Then this app/thread run on cpu with vector
> > > > extension capability only.
> > > >
> > >
> > > IIRC, we assume all harts has the same ability in Linux because of SMP
> > > assumption.
> > > If we have more information of hw capability and we may use this
> > > information for scheduler to switch the correct process to the correct
> > > CPU.
> > > Do you have any idea how to implement it in Linux kernel? Maybe we can
> > > list in the TODO list.
> > I think we can refer to other arch implementations as reference:
> >
> > 1. ARM64 supports 32-bit thread on asymmetric AArch32 systems. There
> > is a flag in ELF to check, then start the thread on the core that
> > supports 32-bit execution. This patchset is merged to mainline 5.15.
> > https://lore.kernel.org/linux-arm-kernel/20210730112443.23245-8-will@kernel.org/T/
>
> Wow! This is useful for AMP.
>
> >
> > 2. Link shared by Darius, on-demand request implementation on Intel AMX
> > https://lore.kernel.org/lkml/20210825155413.19673-1-chang.seok.bae@intel.com/
> >
> > glibc support optimized library functions with vector, this is enabled
> > by default if compiler is with vector extension enabled? If yes, then
> > most of the app required vector core.
>
> As I mentioned earlier, glibc ifunc will solve this issue. The
> Linux/glibc can run on platform with vector or without vector and
> glibc will use the information get from Linux kernel and using ifunc
> to decide whether it should use the vector version or not.
> Which means even your toolchain has vector glibc support and your
> Linux kernel told the glibc this platform doesn't support vector then
> the ifunc mechanism will choose the non-vector version ones.
Okay. Then Linux kernel needs to report vector capability as per core
feature, if not all SMP cores support vector.
next prev parent reply other threads:[~2021-10-07 10:10 UTC|newest]
Thread overview: 55+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-09-08 17:45 [RFC PATCH v8 00/21] riscv: Add vector ISA support Greentime Hu
2021-09-08 17:45 ` [RFC PATCH v8 01/21] riscv: Separate patch for cflags and aflags Greentime Hu
2021-09-08 17:45 ` [RFC PATCH v8 02/21] riscv: Rename __switch_to_aux -> fpu Greentime Hu
2021-09-08 17:45 ` [RFC PATCH v8 03/21] riscv: Extending cpufeature.c to detect V-extension Greentime Hu
2021-09-08 17:45 ` [RFC PATCH v8 04/21] riscv: Add new csr defines related to vector extension Greentime Hu
2021-09-08 17:45 ` [RFC PATCH v8 05/21] riscv: Add vector feature to compile Greentime Hu
2021-09-08 17:45 ` [RFC PATCH v8 06/21] riscv: Add has_vector/riscv_vsize to save vector features Greentime Hu
2021-09-08 17:45 ` [RFC PATCH v8 07/21] riscv: Reset vector register Greentime Hu
2021-09-08 17:45 ` [RFC PATCH v8 08/21] riscv: Add vector struct and assembler definitions Greentime Hu
2021-09-08 17:45 ` [RFC PATCH v8 09/21] riscv: Add task switch support for vector Greentime Hu
2021-09-13 12:21 ` Darius Rad
2021-09-28 14:56 ` Greentime Hu
2021-09-29 13:28 ` Darius Rad
2021-10-01 2:46 ` Ley Foon Tan
2021-10-04 12:41 ` Greentime Hu
2021-10-05 2:12 ` Ley Foon Tan
2021-10-05 15:46 ` Greentime Hu
2021-10-07 10:10 ` Ley Foon Tan [this message]
2021-10-04 12:36 ` Greentime Hu
2021-10-05 13:57 ` Darius Rad
2021-10-21 1:01 ` Paul Walmsley
2021-10-21 10:50 ` Darius Rad
2021-10-22 3:52 ` Vincent Chen
2021-10-22 10:40 ` Darius Rad
2021-10-25 4:47 ` Greentime Hu
2021-10-25 16:22 ` Darius Rad
2021-10-26 4:44 ` Greentime Hu
2021-10-27 12:58 ` Darius Rad
2021-11-09 9:49 ` Greentime Hu
2021-11-09 19:21 ` Darius Rad
2021-10-26 14:58 ` Heiko Stübner
2021-09-08 17:45 ` [RFC PATCH v8 10/21] riscv: Add ptrace vector support Greentime Hu
2021-09-08 17:45 ` [RFC PATCH v8 11/21] riscv: Add sigcontext save/restore for vector Greentime Hu
2021-09-30 2:37 ` Ley Foon Tan
2021-09-08 17:45 ` [RFC PATCH v8 12/21] riscv: signal: Report signal frame size to userspace via auxv Greentime Hu
2021-09-08 17:45 ` [RFC PATCH v8 13/21] riscv: Add support for kernel mode vector Greentime Hu
2021-09-09 6:17 ` Christoph Hellwig
2021-09-08 17:45 ` [RFC PATCH v8 14/21] riscv: Use CSR_STATUS to replace sstatus in vector.S Greentime Hu
2021-09-08 17:45 ` [RFC PATCH v8 15/21] riscv: Add vector extension XOR implementation Greentime Hu
2021-09-09 6:12 ` Christoph Hellwig
2021-09-28 7:00 ` Greentime Hu
2021-09-14 8:29 ` Ley Foon Tan
2021-09-28 7:01 ` Greentime Hu
2021-09-08 17:45 ` [RFC PATCH v8 16/21] riscv: Initialize vector registers with proper vsetvli then it can work normally Greentime Hu
2021-09-08 17:45 ` [RFC PATCH v8 17/21] riscv: Optimize vector registers initialization Greentime Hu
2021-09-08 17:45 ` [RFC PATCH v8 18/21] riscv: Fix an illegal instruction exception when accessing vlenb without enable vector first Greentime Hu
2021-09-08 17:45 ` [RFC PATCH v8 19/21] riscv: Allocate space for vector registers in start_thread() Greentime Hu
2021-09-08 17:45 ` [RFC PATCH v8 20/21] riscv: Optimize task switch codes of vector Greentime Hu
2021-09-15 14:29 ` Jisheng Zhang
2021-10-04 14:13 ` Greentime Hu
2021-09-08 17:45 ` [RFC PATCH v8 21/21] riscv: Turn has_vector into a static key if VECTOR=y Greentime Hu
2021-09-15 14:24 ` Jisheng Zhang
2021-10-04 15:04 ` Greentime Hu
2021-09-13 1:47 ` [RFC PATCH v8 00/21] riscv: Add vector ISA support Vincent Chen
2021-09-13 17:18 ` Vineet Gupta
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CAFiDJ5-GwxCOOW4O6NL=8UCubt0j6NzxaBT7Cm5yBmcFA_RmyA@mail.gmail.com' \
--to=lftan.linux@gmail.com \
--cc=aou@eecs.berkeley.edu \
--cc=darius@bluespec.com \
--cc=greentime.hu@sifive.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-riscv@lists.infradead.org \
--cc=palmer@dabbelt.com \
--cc=paul.walmsley@sifive.com \
--cc=vincent.chen@sifive.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).