All of lore.kernel.org
 help / color / mirror / Atom feed
From: Vincenzo Frascino <vincenzo.frascino@arm.com>
To: Evgenii Stepanov <eugenis@google.com>,
	Andrey Konovalov <andreyknvl@google.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>,
	Will Deacon <will.deacon@arm.com>,
	Mark Rutland <mark.rutland@arm.com>,
	Robin Murphy <robin.murphy@arm.com>,
	Kees Cook <keescook@chromium.org>,
	Kate Stewart <kstewart@linuxfoundation.org>,
	Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
	Andrew Morton <akpm@linux-foundation.org>,
	Ingo Molnar <mingo@kernel.org>,
	"Kirill A . Shutemov" <kirill.shutemov@linux.intel.com>,
	Shuah Khan <shuah@kernel.org>,
	Linux ARM <linux-arm-kernel@lists.infradead.org>,
	"open list:DOCUMENTATION" <linux-doc@vger.kernel.org>,
	Linux Memory Management List <linux-mm@kvack.org>,
	linux-arch <linux-arch@vger.kernel.org>,
	"open list:KERNEL SELFTEST FRAMEWORK" 
	<linux-kselftest@vger.kernel.org>,
	LKML <linux-kernel@vger.kernel.org>,
	Chintan Pandya <cpandya@codeaurora.org>,
	Jacob Bramley <Jacob.Bramley@arm.com>,
	Ruben Ayrapetyan <Ruben.Ayrapetyan@arm.com>,
	Lee Smith <Lee.Smith@arm.com>, Kostya Serebryany <kcc@google.com>,
	Dmitry Vyukov <dvyukov@google.com>,
	Ramana Radhakrishnan <Ramana.Radhakrishnan@arm.com>,
	Luc Van Oostenryck <luc.vanoostenryck@gmail.com>
Subject: Re: [PATCH v7 0/8] arm64: untag user pointers passed to the kernel
Date: Fri, 19 Oct 2018 10:04:42 +0100	[thread overview]
Message-ID: <9bb7fefd-3f8f-266a-3cc9-cc64f8927206@arm.com> (raw)
In-Reply-To: <CAFKCwrh4-BvFB_R1J0LWcbfeR=d02OazowFuMU+hmq8Y=Dx+4w@mail.gmail.com>

On 10/17/18 9:25 PM, Evgenii Stepanov wrote:
> On Wed, Oct 17, 2018 at 7:20 AM, Andrey Konovalov <andreyknvl@google.com> wrote:
>> On Wed, Oct 17, 2018 at 4:06 PM, Vincenzo Frascino
>> <vincenzo.frascino@arm.com> wrote:
>>> Hi Andrey,
>>> I have been thinking a bit lately on how to address the problem of user tagged pointers passed to the kernel through syscalls, and IMHO probably the best way we have to catch them all and make sure that the approach is maintainable in the long term is to introduce shims that tag/untag the pointers passed to the kernel.
>>>
>>> In details, what I am proposing can live either in userspace (preferred solution so that we do not have to relax the ABI) or in kernel space and can be summarized as follows:
>>>  - A shim is specific to a syscall and is called by the libc when it needs to invoke the respective syscall.
>>>  - It is required only if the syscall accepts pointers.
>>>  - It saves the tags of a pointers passed to the syscall in memory (same approach if the we are passing a struct that contains pointers to the kernel, with the difference that all the tags of the pointers in the struct need to be saved singularly)
>>>  - Untags the pointers
>>>  - Invokes the syscall
>>>  - Retags the pointers with the tags stored in memory
>>>  - Returns
>>>
>>> What do you think?
>>
>> Hi Vincenzo,
>>
>> If I correctly understand what you are proposing, I'm not sure if that
>> would work with the countless number of different ioctl calls. For
>> example when an ioctl accepts a struct with a bunch of pointer fields.
>> In this case a shim like the one you propose can't live in userspace,
>> since libc doesn't know about the interface of all ioctls, so it can't
>> know which fields to untag. The kernel knows about those interfaces
>> (since the kernel implements them), but then we would need a custom
>> shim for each ioctl variation, which doesn't seem practical.
> 
> The current patchset handles majority of pointers in a just a few
> common places, like copy_from_user. Userspace shims will need to untag
> & retag all pointer arguments - we are looking at hundreds if not
> thousands of shims. They will also be located in a different code base
> from the syscall / ioctl implementations, which would make them
> impossible to keep up to date.
> 

I agree with both of you, ioctl is the real show stopper for this approach. Thanks for pointing this out.

-- 
Regards,
Vincenzo

WARNING: multiple messages have this Message-ID (diff)
From: vincenzo.frascino at arm.com (Vincenzo Frascino)
Subject: [PATCH v7 0/8] arm64: untag user pointers passed to the kernel
Date: Fri, 19 Oct 2018 10:04:42 +0100	[thread overview]
Message-ID: <9bb7fefd-3f8f-266a-3cc9-cc64f8927206@arm.com> (raw)
In-Reply-To: <CAFKCwrh4-BvFB_R1J0LWcbfeR=d02OazowFuMU+hmq8Y=Dx+4w@mail.gmail.com>

On 10/17/18 9:25 PM, Evgenii Stepanov wrote:
> On Wed, Oct 17, 2018 at 7:20 AM, Andrey Konovalov <andreyknvl at google.com> wrote:
>> On Wed, Oct 17, 2018 at 4:06 PM, Vincenzo Frascino
>> <vincenzo.frascino at arm.com> wrote:
>>> Hi Andrey,
>>> I have been thinking a bit lately on how to address the problem of user tagged pointers passed to the kernel through syscalls, and IMHO probably the best way we have to catch them all and make sure that the approach is maintainable in the long term is to introduce shims that tag/untag the pointers passed to the kernel.
>>>
>>> In details, what I am proposing can live either in userspace (preferred solution so that we do not have to relax the ABI) or in kernel space and can be summarized as follows:
>>>  - A shim is specific to a syscall and is called by the libc when it needs to invoke the respective syscall.
>>>  - It is required only if the syscall accepts pointers.
>>>  - It saves the tags of a pointers passed to the syscall in memory (same approach if the we are passing a struct that contains pointers to the kernel, with the difference that all the tags of the pointers in the struct need to be saved singularly)
>>>  - Untags the pointers
>>>  - Invokes the syscall
>>>  - Retags the pointers with the tags stored in memory
>>>  - Returns
>>>
>>> What do you think?
>>
>> Hi Vincenzo,
>>
>> If I correctly understand what you are proposing, I'm not sure if that
>> would work with the countless number of different ioctl calls. For
>> example when an ioctl accepts a struct with a bunch of pointer fields.
>> In this case a shim like the one you propose can't live in userspace,
>> since libc doesn't know about the interface of all ioctls, so it can't
>> know which fields to untag. The kernel knows about those interfaces
>> (since the kernel implements them), but then we would need a custom
>> shim for each ioctl variation, which doesn't seem practical.
> 
> The current patchset handles majority of pointers in a just a few
> common places, like copy_from_user. Userspace shims will need to untag
> & retag all pointer arguments - we are looking at hundreds if not
> thousands of shims. They will also be located in a different code base
> from the syscall / ioctl implementations, which would make them
> impossible to keep up to date.
> 

I agree with both of you, ioctl is the real show stopper for this approach. Thanks for pointing this out.

-- 
Regards,
Vincenzo

WARNING: multiple messages have this Message-ID (diff)
From: vincenzo.frascino@arm.com (Vincenzo Frascino)
Subject: [PATCH v7 0/8] arm64: untag user pointers passed to the kernel
Date: Fri, 19 Oct 2018 10:04:42 +0100	[thread overview]
Message-ID: <9bb7fefd-3f8f-266a-3cc9-cc64f8927206@arm.com> (raw)
Message-ID: <20181019090442.jLU31m_5HTy2_0TALEzl2PHBxbWwJfYmD2eG9I5z6_o@z> (raw)
In-Reply-To: <CAFKCwrh4-BvFB_R1J0LWcbfeR=d02OazowFuMU+hmq8Y=Dx+4w@mail.gmail.com>

On 10/17/18 9:25 PM, Evgenii Stepanov wrote:
> On Wed, Oct 17, 2018@7:20 AM, Andrey Konovalov <andreyknvl@google.com> wrote:
>> On Wed, Oct 17, 2018 at 4:06 PM, Vincenzo Frascino
>> <vincenzo.frascino@arm.com> wrote:
>>> Hi Andrey,
>>> I have been thinking a bit lately on how to address the problem of user tagged pointers passed to the kernel through syscalls, and IMHO probably the best way we have to catch them all and make sure that the approach is maintainable in the long term is to introduce shims that tag/untag the pointers passed to the kernel.
>>>
>>> In details, what I am proposing can live either in userspace (preferred solution so that we do not have to relax the ABI) or in kernel space and can be summarized as follows:
>>>  - A shim is specific to a syscall and is called by the libc when it needs to invoke the respective syscall.
>>>  - It is required only if the syscall accepts pointers.
>>>  - It saves the tags of a pointers passed to the syscall in memory (same approach if the we are passing a struct that contains pointers to the kernel, with the difference that all the tags of the pointers in the struct need to be saved singularly)
>>>  - Untags the pointers
>>>  - Invokes the syscall
>>>  - Retags the pointers with the tags stored in memory
>>>  - Returns
>>>
>>> What do you think?
>>
>> Hi Vincenzo,
>>
>> If I correctly understand what you are proposing, I'm not sure if that
>> would work with the countless number of different ioctl calls. For
>> example when an ioctl accepts a struct with a bunch of pointer fields.
>> In this case a shim like the one you propose can't live in userspace,
>> since libc doesn't know about the interface of all ioctls, so it can't
>> know which fields to untag. The kernel knows about those interfaces
>> (since the kernel implements them), but then we would need a custom
>> shim for each ioctl variation, which doesn't seem practical.
> 
> The current patchset handles majority of pointers in a just a few
> common places, like copy_from_user. Userspace shims will need to untag
> & retag all pointer arguments - we are looking at hundreds if not
> thousands of shims. They will also be located in a different code base
> from the syscall / ioctl implementations, which would make them
> impossible to keep up to date.
> 

I agree with both of you, ioctl is the real show stopper for this approach. Thanks for pointing this out.

-- 
Regards,
Vincenzo

WARNING: multiple messages have this Message-ID (diff)
From: Vincenzo Frascino <vincenzo.frascino@arm.com>
To: Evgenii Stepanov <eugenis@google.com>,
	Andrey Konovalov <andreyknvl@google.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>,
	Will Deacon <will.deacon@arm.com>,
	Mark Rutland <mark.rutland@arm.com>,
	Robin Murphy <robin.murphy@arm.com>,
	Kees Cook <keescook@chromium.org>,
	Kate Stewart <kstewart@linuxfoundation.org>,
	Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
	Andrew Morton <akpm@linux-foundation.org>,
	Ingo Molnar <mingo@kernel.org>,
	"Kirill A . Shutemov" <kirill.shutemov@linux.intel.com>,
	Shuah Khan <shuah@kernel.org>,
	Linux ARM <linux-arm-kernel@lists.infradead.org>,
	"open list:DOCUMENTATION" <linux-doc@vger.kernel.org>,
	Linux Memory Management List <linux-mm@kvack.org>,
	linux-arch <linux-arch@vger.kernel.org>,
	"open list:KERNEL SELFTEST FRAMEWORK"
	<linux-kselftest@vger.kernel.org>,
	LKML <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH v7 0/8] arm64: untag user pointers passed to the kernel
Date: Fri, 19 Oct 2018 10:04:42 +0100	[thread overview]
Message-ID: <9bb7fefd-3f8f-266a-3cc9-cc64f8927206@arm.com> (raw)
In-Reply-To: <CAFKCwrh4-BvFB_R1J0LWcbfeR=d02OazowFuMU+hmq8Y=Dx+4w@mail.gmail.com>

On 10/17/18 9:25 PM, Evgenii Stepanov wrote:
> On Wed, Oct 17, 2018 at 7:20 AM, Andrey Konovalov <andreyknvl@google.com> wrote:
>> On Wed, Oct 17, 2018 at 4:06 PM, Vincenzo Frascino
>> <vincenzo.frascino@arm.com> wrote:
>>> Hi Andrey,
>>> I have been thinking a bit lately on how to address the problem of user tagged pointers passed to the kernel through syscalls, and IMHO probably the best way we have to catch them all and make sure that the approach is maintainable in the long term is to introduce shims that tag/untag the pointers passed to the kernel.
>>>
>>> In details, what I am proposing can live either in userspace (preferred solution so that we do not have to relax the ABI) or in kernel space and can be summarized as follows:
>>>  - A shim is specific to a syscall and is called by the libc when it needs to invoke the respective syscall.
>>>  - It is required only if the syscall accepts pointers.
>>>  - It saves the tags of a pointers passed to the syscall in memory (same approach if the we are passing a struct that contains pointers to the kernel, with the difference that all the tags of the pointers in the struct need to be saved singularly)
>>>  - Untags the pointers
>>>  - Invokes the syscall
>>>  - Retags the pointers with the tags stored in memory
>>>  - Returns
>>>
>>> What do you think?
>>
>> Hi Vincenzo,
>>
>> If I correctly understand what you are proposing, I'm not sure if that
>> would work with the countless number of different ioctl calls. For
>> example when an ioctl accepts a struct with a bunch of pointer fields.
>> In this case a shim like the one you propose can't live in userspace,
>> since libc doesn't know about the interface of all ioctls, so it can't
>> know which fields to untag. The kernel knows about those interfaces
>> (since the kernel implements them), but then we would need a custom
>> shim for each ioctl variation, which doesn't seem practical.
> 
> The current patchset handles majority of pointers in a just a few
> common places, like copy_from_user. Userspace shims will need to untag
> & retag all pointer arguments - we are looking at hundreds if not
> thousands of shims. They will also be located in a different code base
> from the syscall / ioctl implementations, which would make them
> impossible to keep up to date.
> 

I agree with both of you, ioctl is the real show stopper for this approach. Thanks for pointing this out.

-- 
Regards,
Vincenzo

WARNING: multiple messages have this Message-ID (diff)
From: vincenzo.frascino@arm.com (Vincenzo Frascino)
To: linux-arm-kernel@lists.infradead.org
Subject: [PATCH v7 0/8] arm64: untag user pointers passed to the kernel
Date: Fri, 19 Oct 2018 10:04:42 +0100	[thread overview]
Message-ID: <9bb7fefd-3f8f-266a-3cc9-cc64f8927206@arm.com> (raw)
In-Reply-To: <CAFKCwrh4-BvFB_R1J0LWcbfeR=d02OazowFuMU+hmq8Y=Dx+4w@mail.gmail.com>

On 10/17/18 9:25 PM, Evgenii Stepanov wrote:
> On Wed, Oct 17, 2018 at 7:20 AM, Andrey Konovalov <andreyknvl@google.com> wrote:
>> On Wed, Oct 17, 2018 at 4:06 PM, Vincenzo Frascino
>> <vincenzo.frascino@arm.com> wrote:
>>> Hi Andrey,
>>> I have been thinking a bit lately on how to address the problem of user tagged pointers passed to the kernel through syscalls, and IMHO probably the best way we have to catch them all and make sure that the approach is maintainable in the long term is to introduce shims that tag/untag the pointers passed to the kernel.
>>>
>>> In details, what I am proposing can live either in userspace (preferred solution so that we do not have to relax the ABI) or in kernel space and can be summarized as follows:
>>>  - A shim is specific to a syscall and is called by the libc when it needs to invoke the respective syscall.
>>>  - It is required only if the syscall accepts pointers.
>>>  - It saves the tags of a pointers passed to the syscall in memory (same approach if the we are passing a struct that contains pointers to the kernel, with the difference that all the tags of the pointers in the struct need to be saved singularly)
>>>  - Untags the pointers
>>>  - Invokes the syscall
>>>  - Retags the pointers with the tags stored in memory
>>>  - Returns
>>>
>>> What do you think?
>>
>> Hi Vincenzo,
>>
>> If I correctly understand what you are proposing, I'm not sure if that
>> would work with the countless number of different ioctl calls. For
>> example when an ioctl accepts a struct with a bunch of pointer fields.
>> In this case a shim like the one you propose can't live in userspace,
>> since libc doesn't know about the interface of all ioctls, so it can't
>> know which fields to untag. The kernel knows about those interfaces
>> (since the kernel implements them), but then we would need a custom
>> shim for each ioctl variation, which doesn't seem practical.
> 
> The current patchset handles majority of pointers in a just a few
> common places, like copy_from_user. Userspace shims will need to untag
> & retag all pointer arguments - we are looking at hundreds if not
> thousands of shims. They will also be located in a different code base
> from the syscall / ioctl implementations, which would make them
> impossible to keep up to date.
> 

I agree with both of you, ioctl is the real show stopper for this approach. Thanks for pointing this out.

-- 
Regards,
Vincenzo

  parent reply	other threads:[~2018-10-19  9:04 UTC|newest]

Thread overview: 86+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-10-02 13:12 [PATCH v7 0/8] arm64: untag user pointers passed to the kernel Andrey Konovalov
2018-10-02 13:12 ` Andrey Konovalov
2018-10-02 13:12 ` Andrey Konovalov
2018-10-02 13:12 ` andreyknvl
2018-10-02 13:12 ` [PATCH v7 1/8] arm64: add type casts to untagged_addr macro Andrey Konovalov
2018-10-02 13:12   ` Andrey Konovalov
2018-10-02 13:12   ` Andrey Konovalov
2018-10-02 13:12   ` andreyknvl
2018-10-02 13:12 ` [PATCH v7 2/8] uaccess: add untagged_addr definition for other arches Andrey Konovalov
2018-10-02 13:12   ` Andrey Konovalov
2018-10-02 13:12   ` Andrey Konovalov
2018-10-02 13:12   ` andreyknvl
2018-10-02 13:12 ` [PATCH v7 3/8] arm64: untag user addresses in access_ok and __uaccess_mask_ptr Andrey Konovalov
2018-10-02 13:12   ` Andrey Konovalov
2018-10-02 13:12   ` Andrey Konovalov
2018-10-02 13:12   ` andreyknvl
2018-10-02 13:12 ` [PATCH v7 4/8] mm, arm64: untag user addresses in mm/gup.c Andrey Konovalov
2018-10-02 13:12   ` Andrey Konovalov
2018-10-02 13:12   ` Andrey Konovalov
2018-10-02 13:12   ` andreyknvl
2018-10-02 13:12 ` [PATCH v7 5/8] lib, arm64: untag addrs passed to strncpy_from_user and strnlen_user Andrey Konovalov
2018-10-02 13:12   ` Andrey Konovalov
2018-10-02 13:12   ` Andrey Konovalov
2018-10-02 13:12   ` andreyknvl
2018-10-02 13:12 ` [PATCH v7 6/8] fs, arm64: untag user address in copy_mount_options Andrey Konovalov
2018-10-02 13:12   ` Andrey Konovalov
2018-10-02 13:12   ` Andrey Konovalov
2018-10-02 13:12   ` andreyknvl
2018-10-02 13:12 ` [PATCH v7 7/8] arm64: update Documentation/arm64/tagged-pointers.txt Andrey Konovalov
2018-10-02 13:12   ` Andrey Konovalov
2018-10-02 13:12   ` Andrey Konovalov
2018-10-02 13:12   ` andreyknvl
2018-10-03 17:32   ` Catalin Marinas
2018-10-03 17:32     ` Catalin Marinas
2018-10-03 17:32     ` Catalin Marinas
2018-10-03 17:32     ` Catalin Marinas
2018-10-03 17:32     ` catalin.marinas
2018-10-10 14:09     ` Andrey Konovalov
2018-10-10 14:09       ` Andrey Konovalov
2018-10-10 14:09       ` Andrey Konovalov
2018-10-10 14:09       ` Andrey Konovalov
2018-10-10 14:09       ` Andrey Konovalov
2018-10-10 14:09       ` andreyknvl
2018-10-18 17:31       ` Catalin Marinas
2018-10-18 17:31         ` Catalin Marinas
2018-10-18 17:31         ` Catalin Marinas
2018-10-18 17:31         ` Catalin Marinas
2018-10-18 17:31         ` Catalin Marinas
2018-10-18 17:31         ` catalin.marinas
2018-10-02 13:12 ` [PATCH v7 8/8] selftests, arm64: add a selftest for passing tagged pointers to kernel Andrey Konovalov
2018-10-02 13:12   ` Andrey Konovalov
2018-10-02 13:12   ` Andrey Konovalov
2018-10-02 13:12   ` andreyknvl
2018-10-03 13:31 ` [PATCH v7 0/8] arm64: untag user pointers passed to the kernel Luc Van Oostenryck
2018-10-03 13:31   ` Luc Van Oostenryck
2018-10-03 13:31   ` Luc Van Oostenryck
2018-10-03 13:31   ` Luc Van Oostenryck
2018-10-03 13:31   ` luc.vanoostenryck
2018-10-17 14:06 ` Vincenzo Frascino
2018-10-17 14:06   ` Vincenzo Frascino
2018-10-17 14:06   ` Vincenzo Frascino
2018-10-17 14:06   ` vincenzo.frascino
2018-10-17 14:20   ` Andrey Konovalov
2018-10-17 14:20     ` Andrey Konovalov
2018-10-17 14:20     ` Andrey Konovalov
2018-10-17 14:20     ` Andrey Konovalov
2018-10-17 14:20     ` Andrey Konovalov
2018-10-17 14:20     ` andreyknvl
2018-10-17 20:25     ` Evgenii Stepanov
2018-10-17 20:25       ` Evgenii Stepanov
2018-10-17 20:25       ` Evgenii Stepanov
2018-10-17 20:25       ` Evgenii Stepanov
2018-10-17 20:25       ` Evgenii Stepanov
2018-10-17 20:25       ` eugenis
2018-10-18 17:33       ` Catalin Marinas
2018-10-18 17:33         ` Catalin Marinas
2018-10-18 17:33         ` Catalin Marinas
2018-10-18 17:33         ` Catalin Marinas
2018-10-18 17:33         ` Catalin Marinas
2018-10-18 17:33         ` catalin.marinas
2018-10-19  9:04       ` Vincenzo Frascino [this message]
2018-10-19  9:04         ` Vincenzo Frascino
2018-10-19  9:04         ` Vincenzo Frascino
2018-10-19  9:04         ` Vincenzo Frascino
2018-10-19  9:04         ` Vincenzo Frascino
2018-10-19  9:04         ` vincenzo.frascino

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=9bb7fefd-3f8f-266a-3cc9-cc64f8927206@arm.com \
    --to=vincenzo.frascino@arm.com \
    --cc=Jacob.Bramley@arm.com \
    --cc=Lee.Smith@arm.com \
    --cc=Ramana.Radhakrishnan@arm.com \
    --cc=Ruben.Ayrapetyan@arm.com \
    --cc=akpm@linux-foundation.org \
    --cc=andreyknvl@google.com \
    --cc=catalin.marinas@arm.com \
    --cc=cpandya@codeaurora.org \
    --cc=dvyukov@google.com \
    --cc=eugenis@google.com \
    --cc=gregkh@linuxfoundation.org \
    --cc=kcc@google.com \
    --cc=keescook@chromium.org \
    --cc=kirill.shutemov@linux.intel.com \
    --cc=kstewart@linuxfoundation.org \
    --cc=linux-arch@vger.kernel.org \
    --cc=linux-arm-kernel@lists.infradead.org \
    --cc=linux-doc@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-kselftest@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=luc.vanoostenryck@gmail.com \
    --cc=mark.rutland@arm.com \
    --cc=mingo@kernel.org \
    --cc=robin.murphy@arm.com \
    --cc=shuah@kernel.org \
    --cc=will.deacon@arm.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.