All of lore.kernel.org
 help / color / mirror / Atom feed
From: Vincent Chen <vincent.chen@sifive.com>
To: Greentime Hu <greentime.hu@sifive.com>
Cc: linux-riscv <linux-riscv@lists.infradead.org>,
	"linux-kernel@vger.kernel.org List"
	<linux-kernel@vger.kernel.org>, Albert Ou <aou@eecs.berkeley.edu>,
	Palmer Dabbelt <palmer@dabbelt.com>,
	Paul Walmsley <paul.walmsley@sifive.com>
Subject: Re: [RFC PATCH v8 00/21] riscv: Add vector ISA support
Date: Mon, 13 Sep 2021 09:47:30 +0800	[thread overview]
Message-ID: <CABvJ_xjiD0Vf1SqR2vtLDpmvY79wmG=yyyFU=M8JjuEWs0n_Pw@mail.gmail.com> (raw)
In-Reply-To: <cover.1631121222.git.greentime.hu@sifive.com>

Hi all,

The associated Glibc vector patches could be found here
https://sourceware.org/pipermail/libc-alpha/2021-September/130897.html
Thanks

On Thu, Sep 9, 2021 at 1:45 AM Greentime Hu <greentime.hu@sifive.com> wrote:
>
> This patchset is implemented based on vector 1.0-rc1 spec to add vector
> support in riscv Linux kernel. To make this happen, we defined a new
> structure __riscv_v_state to save the vector related registers. It is used
> for both kernel space and user space.
>
>  - In kernel space, the datap pointer in __riscv_v_state will be allocated
>    dynamically to save vector registers.
>  - In user space,
>         - In signal handler of user space, datap will point to the address
>           of the __riscv_v_state data structure to save vector
>           registers in stack. We also create a __reserved[] array for
>           future extensions.
>         - In ptrace, the data will be put in ubuf in which we use
>           riscv_vr_get()/riscv_vr_set() to get or set the
>           __riscv_v_state data structure from/to it, datap pointer
>           would be zeroed and vector registers will be copied to the
>           address right after the __riscv_v_state structure in ubuf.
>
> This patchset also adds support for kernel mode vector, kernel XOR
> implementation with vector ISA and includes several bug fixes and code
> refinement.
>
> This patchset is rebased to v5.14 and it is tested by running several
> vector programs simultaneously. It also can get the correct ucontext_t in
> signal handler and restore correct context after sigreturn. It is also
> tested with ptrace() syscall to use PTRACE_GETREGSET/PTRACE_SETREGSET to
> get/set the vector registers. I have tested vlen=128 and vlen=256 cases in
> qemu-system-riscv64 provided by Frank Chang.
>
> We have sent patches to glibc mailing list for ifunc support and sigcontext
> changes. We will send patches for vector support to glibc mailing list
> recently.
>
>  [1] https://github.com/riscv/riscv-v-spec/blob/master/v-spec.adoc
>
> ---
> Changelog V8
>  - Rebase to v5.14
>  - Refine struct __riscv_v_state with struct __riscv_ctx_hdr
>  - Refine has_vector into a static key
>  - Defined __reserved space in struct sigcontext for vector and future extensions
>
> Changelog V7
>  - Add support for kernel mode vector
>  - Add vector extension XOR implementation
>  - Optimize task switch codes of vector
>  - Allocate space for vector registers in start_thread()
>  - Fix an illegal instruction exception when accessing vlenb
>  - Optimize vector registers initialization
>  - Initialize vector registers with proper vsetvli then it can work normally
>  - Refine ptrace porting due to generic API changed
>  - Code clean up
>
> Changelog V6
>  - Replace vle.v/vse.v instructions with vle8.v/vse8.v based on 0.9 spec
>  - Add comments based on mailinglist feedback
>  - Fix rv32 build error
>
> Changelog V5
>  - Using regset_size() correctly in generic ptrace
>  - Fix the ptrace porting
>  - Fix compile warning
>
> Changelog V4
>  - Support dynamic vlen
>  - Fix bugs: lazy save/resotre, not saving vtype
>  - Update VS bit offset based on latest vector spec
>  - Add new vector csr based on latest vector spec
>  - Code refine and removed unused macros
>
> Changelog V3
>  - Rebase linux-5.6-rc3 and tested with qemu
>  - Seperate patches with Anup's advice
>  - Give out a ABI puzzle with unlimited vlen
>
> Changelog V2
>  - Fixup typo "vecotr, fstate_save->vstate_save".
>  - Fixup wrong saved registers' length in vector.S.
>  - Seperate unrelated patches from this one.
>
> Greentime Hu (15):
>   riscv: Add new csr defines related to vector extension
>   riscv: Add has_vector/riscv_vsize to save vector features.
>   riscv: Add vector struct and assembler definitions
>   riscv: Add task switch support for vector
>   riscv: Add ptrace vector support
>   riscv: Add sigcontext save/restore for vector
>   riscv: Add support for kernel mode vector
>   riscv: Use CSR_STATUS to replace sstatus in vector.S
>   riscv: Add vector extension XOR implementation
>   riscv: Initialize vector registers with proper vsetvli then it can
>     work normally
>   riscv: Optimize vector registers initialization
>   riscv: Fix an illegal instruction exception when accessing vlenb
>     without enable vector first
>   riscv: Allocate space for vector registers in start_thread()
>   riscv: Optimize task switch codes of vector
>   riscv: Turn has_vector into a static key if VECTOR=y
>
> Guo Ren (5):
>   riscv: Separate patch for cflags and aflags
>   riscv: Rename __switch_to_aux -> fpu
>   riscv: Extending cpufeature.c to detect V-extension
>   riscv: Add vector feature to compile
>   riscv: Reset vector register
>
> Vincent Chen (1):
>   riscv: signal: Report signal frame size to userspace via auxv
>
>  arch/riscv/Kconfig                       |   9 ++
>  arch/riscv/Makefile                      |  19 ++-
>  arch/riscv/include/asm/csr.h             |  16 ++-
>  arch/riscv/include/asm/elf.h             |  41 +++---
>  arch/riscv/include/asm/processor.h       |   3 +
>  arch/riscv/include/asm/switch_to.h       |  71 +++++++++-
>  arch/riscv/include/asm/vector.h          |  16 +++
>  arch/riscv/include/asm/xor.h             |  74 ++++++++++
>  arch/riscv/include/uapi/asm/auxvec.h     |   1 +
>  arch/riscv/include/uapi/asm/hwcap.h      |   1 +
>  arch/riscv/include/uapi/asm/ptrace.h     |  25 ++++
>  arch/riscv/include/uapi/asm/sigcontext.h |  24 ++++
>  arch/riscv/kernel/Makefile               |   7 +
>  arch/riscv/kernel/asm-offsets.c          |   8 ++
>  arch/riscv/kernel/cpufeature.c           |  16 +++
>  arch/riscv/kernel/entry.S                |   6 +-
>  arch/riscv/kernel/head.S                 |  22 ++-
>  arch/riscv/kernel/kernel_mode_vector.c   | 158 +++++++++++++++++++++
>  arch/riscv/kernel/process.c              |  49 +++++++
>  arch/riscv/kernel/ptrace.c               |  71 ++++++++++
>  arch/riscv/kernel/setup.c                |   4 +
>  arch/riscv/kernel/signal.c               | 172 ++++++++++++++++++++++-
>  arch/riscv/kernel/vector.S               |  81 +++++++++++
>  arch/riscv/lib/Makefile                  |   1 +
>  arch/riscv/lib/xor.S                     |  81 +++++++++++
>  include/uapi/linux/elf.h                 |   1 +
>  26 files changed, 941 insertions(+), 36 deletions(-)
>  create mode 100644 arch/riscv/include/asm/vector.h
>  create mode 100644 arch/riscv/include/asm/xor.h
>  create mode 100644 arch/riscv/kernel/kernel_mode_vector.c
>  create mode 100644 arch/riscv/kernel/vector.S
>  create mode 100644 arch/riscv/lib/xor.S
>
> --
> 2.31.1
>

WARNING: multiple messages have this Message-ID (diff)
From: Vincent Chen <vincent.chen@sifive.com>
To: Greentime Hu <greentime.hu@sifive.com>
Cc: linux-riscv <linux-riscv@lists.infradead.org>,
	 "linux-kernel@vger.kernel.org List"
	<linux-kernel@vger.kernel.org>, Albert Ou <aou@eecs.berkeley.edu>,
	 Palmer Dabbelt <palmer@dabbelt.com>,
	Paul Walmsley <paul.walmsley@sifive.com>
Subject: Re: [RFC PATCH v8 00/21] riscv: Add vector ISA support
Date: Mon, 13 Sep 2021 09:47:30 +0800	[thread overview]
Message-ID: <CABvJ_xjiD0Vf1SqR2vtLDpmvY79wmG=yyyFU=M8JjuEWs0n_Pw@mail.gmail.com> (raw)
In-Reply-To: <cover.1631121222.git.greentime.hu@sifive.com>

Hi all,

The associated Glibc vector patches could be found here
https://sourceware.org/pipermail/libc-alpha/2021-September/130897.html
Thanks

On Thu, Sep 9, 2021 at 1:45 AM Greentime Hu <greentime.hu@sifive.com> wrote:
>
> This patchset is implemented based on vector 1.0-rc1 spec to add vector
> support in riscv Linux kernel. To make this happen, we defined a new
> structure __riscv_v_state to save the vector related registers. It is used
> for both kernel space and user space.
>
>  - In kernel space, the datap pointer in __riscv_v_state will be allocated
>    dynamically to save vector registers.
>  - In user space,
>         - In signal handler of user space, datap will point to the address
>           of the __riscv_v_state data structure to save vector
>           registers in stack. We also create a __reserved[] array for
>           future extensions.
>         - In ptrace, the data will be put in ubuf in which we use
>           riscv_vr_get()/riscv_vr_set() to get or set the
>           __riscv_v_state data structure from/to it, datap pointer
>           would be zeroed and vector registers will be copied to the
>           address right after the __riscv_v_state structure in ubuf.
>
> This patchset also adds support for kernel mode vector, kernel XOR
> implementation with vector ISA and includes several bug fixes and code
> refinement.
>
> This patchset is rebased to v5.14 and it is tested by running several
> vector programs simultaneously. It also can get the correct ucontext_t in
> signal handler and restore correct context after sigreturn. It is also
> tested with ptrace() syscall to use PTRACE_GETREGSET/PTRACE_SETREGSET to
> get/set the vector registers. I have tested vlen=128 and vlen=256 cases in
> qemu-system-riscv64 provided by Frank Chang.
>
> We have sent patches to glibc mailing list for ifunc support and sigcontext
> changes. We will send patches for vector support to glibc mailing list
> recently.
>
>  [1] https://github.com/riscv/riscv-v-spec/blob/master/v-spec.adoc
>
> ---
> Changelog V8
>  - Rebase to v5.14
>  - Refine struct __riscv_v_state with struct __riscv_ctx_hdr
>  - Refine has_vector into a static key
>  - Defined __reserved space in struct sigcontext for vector and future extensions
>
> Changelog V7
>  - Add support for kernel mode vector
>  - Add vector extension XOR implementation
>  - Optimize task switch codes of vector
>  - Allocate space for vector registers in start_thread()
>  - Fix an illegal instruction exception when accessing vlenb
>  - Optimize vector registers initialization
>  - Initialize vector registers with proper vsetvli then it can work normally
>  - Refine ptrace porting due to generic API changed
>  - Code clean up
>
> Changelog V6
>  - Replace vle.v/vse.v instructions with vle8.v/vse8.v based on 0.9 spec
>  - Add comments based on mailinglist feedback
>  - Fix rv32 build error
>
> Changelog V5
>  - Using regset_size() correctly in generic ptrace
>  - Fix the ptrace porting
>  - Fix compile warning
>
> Changelog V4
>  - Support dynamic vlen
>  - Fix bugs: lazy save/resotre, not saving vtype
>  - Update VS bit offset based on latest vector spec
>  - Add new vector csr based on latest vector spec
>  - Code refine and removed unused macros
>
> Changelog V3
>  - Rebase linux-5.6-rc3 and tested with qemu
>  - Seperate patches with Anup's advice
>  - Give out a ABI puzzle with unlimited vlen
>
> Changelog V2
>  - Fixup typo "vecotr, fstate_save->vstate_save".
>  - Fixup wrong saved registers' length in vector.S.
>  - Seperate unrelated patches from this one.
>
> Greentime Hu (15):
>   riscv: Add new csr defines related to vector extension
>   riscv: Add has_vector/riscv_vsize to save vector features.
>   riscv: Add vector struct and assembler definitions
>   riscv: Add task switch support for vector
>   riscv: Add ptrace vector support
>   riscv: Add sigcontext save/restore for vector
>   riscv: Add support for kernel mode vector
>   riscv: Use CSR_STATUS to replace sstatus in vector.S
>   riscv: Add vector extension XOR implementation
>   riscv: Initialize vector registers with proper vsetvli then it can
>     work normally
>   riscv: Optimize vector registers initialization
>   riscv: Fix an illegal instruction exception when accessing vlenb
>     without enable vector first
>   riscv: Allocate space for vector registers in start_thread()
>   riscv: Optimize task switch codes of vector
>   riscv: Turn has_vector into a static key if VECTOR=y
>
> Guo Ren (5):
>   riscv: Separate patch for cflags and aflags
>   riscv: Rename __switch_to_aux -> fpu
>   riscv: Extending cpufeature.c to detect V-extension
>   riscv: Add vector feature to compile
>   riscv: Reset vector register
>
> Vincent Chen (1):
>   riscv: signal: Report signal frame size to userspace via auxv
>
>  arch/riscv/Kconfig                       |   9 ++
>  arch/riscv/Makefile                      |  19 ++-
>  arch/riscv/include/asm/csr.h             |  16 ++-
>  arch/riscv/include/asm/elf.h             |  41 +++---
>  arch/riscv/include/asm/processor.h       |   3 +
>  arch/riscv/include/asm/switch_to.h       |  71 +++++++++-
>  arch/riscv/include/asm/vector.h          |  16 +++
>  arch/riscv/include/asm/xor.h             |  74 ++++++++++
>  arch/riscv/include/uapi/asm/auxvec.h     |   1 +
>  arch/riscv/include/uapi/asm/hwcap.h      |   1 +
>  arch/riscv/include/uapi/asm/ptrace.h     |  25 ++++
>  arch/riscv/include/uapi/asm/sigcontext.h |  24 ++++
>  arch/riscv/kernel/Makefile               |   7 +
>  arch/riscv/kernel/asm-offsets.c          |   8 ++
>  arch/riscv/kernel/cpufeature.c           |  16 +++
>  arch/riscv/kernel/entry.S                |   6 +-
>  arch/riscv/kernel/head.S                 |  22 ++-
>  arch/riscv/kernel/kernel_mode_vector.c   | 158 +++++++++++++++++++++
>  arch/riscv/kernel/process.c              |  49 +++++++
>  arch/riscv/kernel/ptrace.c               |  71 ++++++++++
>  arch/riscv/kernel/setup.c                |   4 +
>  arch/riscv/kernel/signal.c               | 172 ++++++++++++++++++++++-
>  arch/riscv/kernel/vector.S               |  81 +++++++++++
>  arch/riscv/lib/Makefile                  |   1 +
>  arch/riscv/lib/xor.S                     |  81 +++++++++++
>  include/uapi/linux/elf.h                 |   1 +
>  26 files changed, 941 insertions(+), 36 deletions(-)
>  create mode 100644 arch/riscv/include/asm/vector.h
>  create mode 100644 arch/riscv/include/asm/xor.h
>  create mode 100644 arch/riscv/kernel/kernel_mode_vector.c
>  create mode 100644 arch/riscv/kernel/vector.S
>  create mode 100644 arch/riscv/lib/xor.S
>
> --
> 2.31.1
>

_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

  parent reply	other threads:[~2021-09-13  1:47 UTC|newest]

Thread overview: 112+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-09-08 17:45 [RFC PATCH v8 00/21] riscv: Add vector ISA support Greentime Hu
2021-09-08 17:45 ` Greentime Hu
2021-09-08 17:45 ` [RFC PATCH v8 01/21] riscv: Separate patch for cflags and aflags Greentime Hu
2021-09-08 17:45   ` Greentime Hu
2021-09-08 17:45 ` [RFC PATCH v8 02/21] riscv: Rename __switch_to_aux -> fpu Greentime Hu
2021-09-08 17:45   ` Greentime Hu
2021-09-08 17:45 ` [RFC PATCH v8 03/21] riscv: Extending cpufeature.c to detect V-extension Greentime Hu
2021-09-08 17:45   ` Greentime Hu
2021-09-08 17:45 ` [RFC PATCH v8 04/21] riscv: Add new csr defines related to vector extension Greentime Hu
2021-09-08 17:45   ` Greentime Hu
2021-09-08 17:45 ` [RFC PATCH v8 05/21] riscv: Add vector feature to compile Greentime Hu
2021-09-08 17:45   ` Greentime Hu
2021-09-08 17:45 ` [RFC PATCH v8 06/21] riscv: Add has_vector/riscv_vsize to save vector features Greentime Hu
2021-09-08 17:45   ` Greentime Hu
2021-09-08 17:45 ` [RFC PATCH v8 07/21] riscv: Reset vector register Greentime Hu
2021-09-08 17:45   ` Greentime Hu
2021-09-08 17:45 ` [RFC PATCH v8 08/21] riscv: Add vector struct and assembler definitions Greentime Hu
2021-09-08 17:45   ` Greentime Hu
2021-09-08 17:45 ` [RFC PATCH v8 09/21] riscv: Add task switch support for vector Greentime Hu
2021-09-08 17:45   ` Greentime Hu
2021-09-08 20:47   ` kernel test robot
2021-09-08 20:47     ` kernel test robot
2021-09-13 12:21   ` Darius Rad
2021-09-13 12:21     ` Darius Rad
2021-09-28 14:56     ` Greentime Hu
2021-09-28 14:56       ` Greentime Hu
2021-09-29 13:28       ` Darius Rad
2021-09-29 13:28         ` Darius Rad
2021-10-01  2:46         ` Ley Foon Tan
2021-10-01  2:46           ` Ley Foon Tan
2021-10-04 12:41           ` Greentime Hu
2021-10-04 12:41             ` Greentime Hu
2021-10-05  2:12             ` Ley Foon Tan
2021-10-05  2:12               ` Ley Foon Tan
2021-10-05 15:46               ` Greentime Hu
2021-10-05 15:46                 ` Greentime Hu
2021-10-07 10:10                 ` Ley Foon Tan
2021-10-07 10:10                   ` Ley Foon Tan
2021-10-04 12:36         ` Greentime Hu
2021-10-04 12:36           ` Greentime Hu
2021-10-05 13:57           ` Darius Rad
2021-10-05 13:57             ` Darius Rad
2021-10-21  1:01             ` Paul Walmsley
2021-10-21  1:01               ` Paul Walmsley
2021-10-21 10:50               ` Darius Rad
2021-10-21 10:50                 ` Darius Rad
2021-10-22  3:52                 ` Vincent Chen
2021-10-22  3:52                   ` Vincent Chen
2021-10-22 10:40                   ` Darius Rad
2021-10-22 10:40                     ` Darius Rad
2021-10-25  4:47                     ` Greentime Hu
2021-10-25  4:47                       ` Greentime Hu
2021-10-25 16:22                       ` Darius Rad
2021-10-25 16:22                         ` Darius Rad
2021-10-26  4:44                         ` Greentime Hu
2021-10-26  4:44                           ` Greentime Hu
2021-10-27 12:58                           ` Darius Rad
2021-10-27 12:58                             ` Darius Rad
2021-11-09  9:49                             ` Greentime Hu
2021-11-09  9:49                               ` Greentime Hu
2021-11-09 19:21                               ` Darius Rad
2021-11-09 19:21                                 ` Darius Rad
2021-10-26 14:58                     ` Heiko Stübner
2021-10-26 14:58                       ` Heiko Stübner
2021-09-08 17:45 ` [RFC PATCH v8 10/21] riscv: Add ptrace vector support Greentime Hu
2021-09-08 17:45   ` Greentime Hu
2021-09-08 17:45 ` [RFC PATCH v8 11/21] riscv: Add sigcontext save/restore for vector Greentime Hu
2021-09-08 17:45   ` Greentime Hu
2021-09-30  2:37   ` Ley Foon Tan
2021-09-30  2:37     ` Ley Foon Tan
2021-09-08 17:45 ` [RFC PATCH v8 12/21] riscv: signal: Report signal frame size to userspace via auxv Greentime Hu
2021-09-08 17:45   ` Greentime Hu
2021-09-08 17:45 ` [RFC PATCH v8 13/21] riscv: Add support for kernel mode vector Greentime Hu
2021-09-08 17:45   ` Greentime Hu
2021-09-09  6:17   ` Christoph Hellwig
2021-09-09  6:17     ` Christoph Hellwig
2021-09-08 17:45 ` [RFC PATCH v8 14/21] riscv: Use CSR_STATUS to replace sstatus in vector.S Greentime Hu
2021-09-08 17:45   ` Greentime Hu
2021-09-08 17:45 ` [RFC PATCH v8 15/21] riscv: Add vector extension XOR implementation Greentime Hu
2021-09-08 17:45   ` Greentime Hu
2021-09-09  6:12   ` Christoph Hellwig
2021-09-09  6:12     ` Christoph Hellwig
2021-09-28  7:00     ` Greentime Hu
2021-09-28  7:00       ` Greentime Hu
2021-09-14  8:29   ` Ley Foon Tan
2021-09-14  8:29     ` Ley Foon Tan
2021-09-28  7:01     ` Greentime Hu
2021-09-28  7:01       ` Greentime Hu
2021-09-08 17:45 ` [RFC PATCH v8 16/21] riscv: Initialize vector registers with proper vsetvli then it can work normally Greentime Hu
2021-09-08 17:45   ` Greentime Hu
2021-09-08 17:45 ` [RFC PATCH v8 17/21] riscv: Optimize vector registers initialization Greentime Hu
2021-09-08 17:45   ` Greentime Hu
2021-09-08 17:45 ` [RFC PATCH v8 18/21] riscv: Fix an illegal instruction exception when accessing vlenb without enable vector first Greentime Hu
2021-09-08 17:45   ` Greentime Hu
2021-09-08 17:45 ` [RFC PATCH v8 19/21] riscv: Allocate space for vector registers in start_thread() Greentime Hu
2021-09-08 17:45   ` Greentime Hu
2021-09-08 17:45 ` [RFC PATCH v8 20/21] riscv: Optimize task switch codes of vector Greentime Hu
2021-09-08 17:45   ` Greentime Hu
2021-09-15 14:29   ` Jisheng Zhang
2021-09-15 14:29     ` Jisheng Zhang
2021-10-04 14:13     ` Greentime Hu
2021-10-04 14:13       ` Greentime Hu
2021-09-08 17:45 ` [RFC PATCH v8 21/21] riscv: Turn has_vector into a static key if VECTOR=y Greentime Hu
2021-09-08 17:45   ` Greentime Hu
2021-09-15 14:24   ` Jisheng Zhang
2021-09-15 14:24     ` Jisheng Zhang
2021-10-04 15:04     ` Greentime Hu
2021-10-04 15:04       ` Greentime Hu
2021-09-13  1:47 ` Vincent Chen [this message]
2021-09-13  1:47   ` [RFC PATCH v8 00/21] riscv: Add vector ISA support Vincent Chen
2021-09-13 17:18 ` Vineet Gupta
2021-09-13 17:18   ` Vineet Gupta

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='CABvJ_xjiD0Vf1SqR2vtLDpmvY79wmG=yyyFU=M8JjuEWs0n_Pw@mail.gmail.com' \
    --to=vincent.chen@sifive.com \
    --cc=aou@eecs.berkeley.edu \
    --cc=greentime.hu@sifive.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-riscv@lists.infradead.org \
    --cc=palmer@dabbelt.com \
    --cc=paul.walmsley@sifive.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.