All of lore.kernel.org
 help / color / mirror / Atom feed
From: Raghavendra K T <raghavendra.kt@linux.vnet.ibm.com>
To: <gleb@redhat.com>, <mingo@redhat.com>, <jeremy@goop.org>,
	<x86@kernel.org>, <konrad.wilk@oracle.com>, <hpa@zytor.com>,
	<pbonzini@redhat.com>
Cc: linux-doc@vger.kernel.org, <habanero@linux.vnet.ibm.com>,
	Raghavendra K T <raghavendra.kt@linux.vnet.ibm.com>,
	<xen-devel@lists.xensource.com>, <peterz@infradead.org>,
	<mtosatti@redhat.com>, <stefano.stabellini@eu.citrix.com>,
	<andi@firstfloor.org>, <attilio.rao@citrix.com>,
	<ouyang@cs.pitt.edu>, <gregkh@suse.de>, <agraf@suse.de>,
	<chegu_vinod@hp.com>, <torvalds@linux-foundation.org>,
	<avi.kivity@gmail.com>, <tglx@linutronix.de>,
	<kvm@vger.kernel.org>, <linux-kernel@vger.kernel.org>,
	<riel@redhat.com>, <drjones@redhat.com>,
	<virtualization@lists.linux-foundation.org>,
	<srivatsa.vaddagiri@gmail.com>
Subject: [PATCH RFC V11 17/18] Documentation/kvm : Add documentation on Hypercalls and features used for PV spinlock
Date: Mon, 22 Jul 2013 11:50:38 +0530	[thread overview]
Message-ID: <20130722062038.24737.35006.sendpatchset@codeblue> (raw)
In-Reply-To: <20130722061631.24737.75508.sendpatchset@codeblue>

Documentation/kvm : Add documentation on Hypercalls and features used for PV spinlock

From: Raghavendra K T <raghavendra.kt@linux.vnet.ibm.com>

KVM_HC_KICK_CPU  hypercall added to wakeup halted vcpu in paravirtual spinlock
enabled guest.

KVM_FEATURE_PV_UNHALT enables guest to check whether pv spinlock can be enabled
in guest.

Thanks Vatsa for rewriting KVM_HC_KICK_CPU

Signed-off-by: Srivatsa Vaddagiri <vatsa@linux.vnet.ibm.com>
Signed-off-by: Raghavendra K T <raghavendra.kt@linux.vnet.ibm.com>
---
 Documentation/virtual/kvm/cpuid.txt      |    4 ++++
 Documentation/virtual/kvm/hypercalls.txt |   14 ++++++++++++++
 2 files changed, 18 insertions(+)

diff --git a/Documentation/virtual/kvm/cpuid.txt b/Documentation/virtual/kvm/cpuid.txt
index 83afe65..654f43c 100644
--- a/Documentation/virtual/kvm/cpuid.txt
+++ b/Documentation/virtual/kvm/cpuid.txt
@@ -43,6 +43,10 @@ KVM_FEATURE_CLOCKSOURCE2           ||     3 || kvmclock available at msrs
 KVM_FEATURE_ASYNC_PF               ||     4 || async pf can be enabled by
                                    ||       || writing to msr 0x4b564d02
 ------------------------------------------------------------------------------
+KVM_FEATURE_PV_UNHALT              ||     6 || guest checks this feature bit
+                                   ||       || before enabling paravirtualized
+                                   ||       || spinlock support.
+------------------------------------------------------------------------------
 KVM_FEATURE_CLOCKSOURCE_STABLE_BIT ||    24 || host will warn if no guest-side
                                    ||       || per-cpu warps are expected in
                                    ||       || kvmclock.
diff --git a/Documentation/virtual/kvm/hypercalls.txt b/Documentation/virtual/kvm/hypercalls.txt
index ea113b5..022198e 100644
--- a/Documentation/virtual/kvm/hypercalls.txt
+++ b/Documentation/virtual/kvm/hypercalls.txt
@@ -64,3 +64,17 @@ Purpose: To enable communication between the hypervisor and guest there is a
 shared page that contains parts of supervisor visible register state.
 The guest can map this shared page to access its supervisor register through
 memory using this hypercall.
+
+5. KVM_HC_KICK_CPU
+------------------------
+Architecture: x86
+Status: active
+Purpose: Hypercall used to wakeup a vcpu from HLT state
+Usage example : A vcpu of a paravirtualized guest that is busywaiting in guest
+kernel mode for an event to occur (ex: a spinlock to become available) can
+execute HLT instruction once it has busy-waited for more than a threshold
+time-interval. Execution of HLT instruction would cause the hypervisor to put
+the vcpu to sleep until occurence of an appropriate event. Another vcpu of the
+same guest can wakeup the sleeping vcpu by issuing KVM_HC_KICK_CPU hypercall,
+specifying APIC ID (a1) of the vcpu to be woken up. An additional argument (a0)
+is used in the hypercall for future use.


WARNING: multiple messages have this Message-ID (diff)
From: Raghavendra K T <raghavendra.kt@linux.vnet.ibm.com>
To: <gleb@redhat.com>, <mingo@redhat.com>, <jeremy@goop.org>,
	<x86@kernel.org>, <konrad.wilk@oracle.com>, <hpa@zytor.com>,
	<pbonzini@redhat.com>
Cc: gregkh@suse.de, kvm@vger.kernel.org, linux-doc@vger.kernel.org,
	peterz@infradead.org, drjones@redhat.com,
	virtualization@lists.linux-foundation.org, andi@firstfloor.org,
	xen-devel@lists.xensource.com,
	Raghavendra K T <raghavendra.kt@linux.vnet.ibm.com>,
	habanero@linux.vnet.ibm.com, riel@redhat.com,
	stefano.stabellini@eu.citrix.com, ouyang@cs.pitt.edu,
	avi.kivity@gmail.com, tglx@linutronix.de, chegu_vinod@hp.com,
	linux-kernel@vger.kernel.org, srivatsa.vaddagiri@gmail.com,
	attilio.rao@citrix.com, torvalds@linux-foundation.org
Subject: [PATCH RFC V11 17/18] Documentation/kvm : Add documentation on Hypercalls and features used for PV spinlock
Date: Mon, 22 Jul 2013 11:50:38 +0530	[thread overview]
Message-ID: <20130722062038.24737.35006.sendpatchset@codeblue> (raw)
In-Reply-To: <20130722061631.24737.75508.sendpatchset@codeblue>

Documentation/kvm : Add documentation on Hypercalls and features used for PV spinlock

From: Raghavendra K T <raghavendra.kt@linux.vnet.ibm.com>

KVM_HC_KICK_CPU  hypercall added to wakeup halted vcpu in paravirtual spinlock
enabled guest.

KVM_FEATURE_PV_UNHALT enables guest to check whether pv spinlock can be enabled
in guest.

Thanks Vatsa for rewriting KVM_HC_KICK_CPU

Signed-off-by: Srivatsa Vaddagiri <vatsa@linux.vnet.ibm.com>
Signed-off-by: Raghavendra K T <raghavendra.kt@linux.vnet.ibm.com>
---
 Documentation/virtual/kvm/cpuid.txt      |    4 ++++
 Documentation/virtual/kvm/hypercalls.txt |   14 ++++++++++++++
 2 files changed, 18 insertions(+)

diff --git a/Documentation/virtual/kvm/cpuid.txt b/Documentation/virtual/kvm/cpuid.txt
index 83afe65..654f43c 100644
--- a/Documentation/virtual/kvm/cpuid.txt
+++ b/Documentation/virtual/kvm/cpuid.txt
@@ -43,6 +43,10 @@ KVM_FEATURE_CLOCKSOURCE2           ||     3 || kvmclock available at msrs
 KVM_FEATURE_ASYNC_PF               ||     4 || async pf can be enabled by
                                    ||       || writing to msr 0x4b564d02
 ------------------------------------------------------------------------------
+KVM_FEATURE_PV_UNHALT              ||     6 || guest checks this feature bit
+                                   ||       || before enabling paravirtualized
+                                   ||       || spinlock support.
+------------------------------------------------------------------------------
 KVM_FEATURE_CLOCKSOURCE_STABLE_BIT ||    24 || host will warn if no guest-side
                                    ||       || per-cpu warps are expected in
                                    ||       || kvmclock.
diff --git a/Documentation/virtual/kvm/hypercalls.txt b/Documentation/virtual/kvm/hypercalls.txt
index ea113b5..022198e 100644
--- a/Documentation/virtual/kvm/hypercalls.txt
+++ b/Documentation/virtual/kvm/hypercalls.txt
@@ -64,3 +64,17 @@ Purpose: To enable communication between the hypervisor and guest there is a
 shared page that contains parts of supervisor visible register state.
 The guest can map this shared page to access its supervisor register through
 memory using this hypercall.
+
+5. KVM_HC_KICK_CPU
+------------------------
+Architecture: x86
+Status: active
+Purpose: Hypercall used to wakeup a vcpu from HLT state
+Usage example : A vcpu of a paravirtualized guest that is busywaiting in guest
+kernel mode for an event to occur (ex: a spinlock to become available) can
+execute HLT instruction once it has busy-waited for more than a threshold
+time-interval. Execution of HLT instruction would cause the hypervisor to put
+the vcpu to sleep until occurence of an appropriate event. Another vcpu of the
+same guest can wakeup the sleeping vcpu by issuing KVM_HC_KICK_CPU hypercall,
+specifying APIC ID (a1) of the vcpu to be woken up. An additional argument (a0)
+is used in the hypercall for future use.

WARNING: multiple messages have this Message-ID (diff)
From: Raghavendra K T <raghavendra.kt@linux.vnet.ibm.com>
To: gleb@redhat.com, mingo@redhat.com, jeremy@goop.org,
	x86@kernel.org, konrad.wilk@oracle.com, hpa@zytor.com,
	pbonzini@redhat.com
Cc: gregkh@suse.de, kvm@vger.kernel.org, linux-doc@vger.kernel.org,
	peterz@infradead.org, drjones@redhat.com,
	virtualization@lists.linux-foundation.org, andi@firstfloor.org,
	xen-devel@lists.xensource.com,
	Raghavendra K T <raghavendra.kt@linux.vnet.ibm.com>,
	habanero@linux.vnet.ibm.com, riel@redhat.com,
	stefano.stabellini@eu.citrix.com, ouyang@cs.pitt.edu,
	avi.kivity@gmail.com, tglx@linutronix.de, chegu_vinod@hp.com,
	linux-kernel@vger.kernel.org, srivatsa.vaddagiri@gmail.com,
	attilio.rao@citrix.com, torvalds@linux-foundation.org
Subject: [PATCH RFC V11 17/18] Documentation/kvm : Add documentation on Hypercalls and features used for PV spinlock
Date: Mon, 22 Jul 2013 11:50:38 +0530	[thread overview]
Message-ID: <20130722062038.24737.35006.sendpatchset@codeblue> (raw)
In-Reply-To: <20130722061631.24737.75508.sendpatchset@codeblue>

Documentation/kvm : Add documentation on Hypercalls and features used for PV spinlock

From: Raghavendra K T <raghavendra.kt@linux.vnet.ibm.com>

KVM_HC_KICK_CPU  hypercall added to wakeup halted vcpu in paravirtual spinlock
enabled guest.

KVM_FEATURE_PV_UNHALT enables guest to check whether pv spinlock can be enabled
in guest.

Thanks Vatsa for rewriting KVM_HC_KICK_CPU

Signed-off-by: Srivatsa Vaddagiri <vatsa@linux.vnet.ibm.com>
Signed-off-by: Raghavendra K T <raghavendra.kt@linux.vnet.ibm.com>
---
 Documentation/virtual/kvm/cpuid.txt      |    4 ++++
 Documentation/virtual/kvm/hypercalls.txt |   14 ++++++++++++++
 2 files changed, 18 insertions(+)

diff --git a/Documentation/virtual/kvm/cpuid.txt b/Documentation/virtual/kvm/cpuid.txt
index 83afe65..654f43c 100644
--- a/Documentation/virtual/kvm/cpuid.txt
+++ b/Documentation/virtual/kvm/cpuid.txt
@@ -43,6 +43,10 @@ KVM_FEATURE_CLOCKSOURCE2           ||     3 || kvmclock available at msrs
 KVM_FEATURE_ASYNC_PF               ||     4 || async pf can be enabled by
                                    ||       || writing to msr 0x4b564d02
 ------------------------------------------------------------------------------
+KVM_FEATURE_PV_UNHALT              ||     6 || guest checks this feature bit
+                                   ||       || before enabling paravirtualized
+                                   ||       || spinlock support.
+------------------------------------------------------------------------------
 KVM_FEATURE_CLOCKSOURCE_STABLE_BIT ||    24 || host will warn if no guest-side
                                    ||       || per-cpu warps are expected in
                                    ||       || kvmclock.
diff --git a/Documentation/virtual/kvm/hypercalls.txt b/Documentation/virtual/kvm/hypercalls.txt
index ea113b5..022198e 100644
--- a/Documentation/virtual/kvm/hypercalls.txt
+++ b/Documentation/virtual/kvm/hypercalls.txt
@@ -64,3 +64,17 @@ Purpose: To enable communication between the hypervisor and guest there is a
 shared page that contains parts of supervisor visible register state.
 The guest can map this shared page to access its supervisor register through
 memory using this hypercall.
+
+5. KVM_HC_KICK_CPU
+------------------------
+Architecture: x86
+Status: active
+Purpose: Hypercall used to wakeup a vcpu from HLT state
+Usage example : A vcpu of a paravirtualized guest that is busywaiting in guest
+kernel mode for an event to occur (ex: a spinlock to become available) can
+execute HLT instruction once it has busy-waited for more than a threshold
+time-interval. Execution of HLT instruction would cause the hypervisor to put
+the vcpu to sleep until occurence of an appropriate event. Another vcpu of the
+same guest can wakeup the sleeping vcpu by issuing KVM_HC_KICK_CPU hypercall,
+specifying APIC ID (a1) of the vcpu to be woken up. An additional argument (a0)
+is used in the hypercall for future use.

  parent reply	other threads:[~2013-07-22  6:13 UTC|newest]

Thread overview: 120+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2013-07-22  6:16 [PATCH RFC V11 0/18] Paravirtualized ticket spinlocks Raghavendra K T
2013-07-22  6:16 ` Raghavendra K T
2013-07-22  6:16 ` [PATCH RFC V11 1/18] x86/spinlock: Replace pv spinlocks with pv ticketlocks Raghavendra K T
2013-07-22  6:16   ` Raghavendra K T
2013-07-22  6:16   ` Raghavendra K T
2013-07-22  6:17 ` [PATCH RFC V11 2/18] x86/ticketlock: Don't inline _spin_unlock when using paravirt spinlocks Raghavendra K T
2013-07-22  6:17   ` Raghavendra K T
2013-07-22  6:17 ` Raghavendra K T
2013-07-22  6:17 ` [PATCH RFC V11 3/18] x86/ticketlock: Collapse a layer of functions Raghavendra K T
2013-07-22  6:17   ` Raghavendra K T
2013-07-22  6:17   ` Raghavendra K T
2013-07-22  6:17 ` [PATCH RFC V11 4/18] xen: Defer spinlock setup until boot CPU setup Raghavendra K T
2013-07-22  6:17   ` Raghavendra K T
2013-07-22  6:17   ` Raghavendra K T
2013-07-22  6:17 ` [PATCH RFC V11 5/18] xen/pvticketlock: Xen implementation for PV ticket locks Raghavendra K T
2013-07-22  6:17   ` Raghavendra K T
2013-07-22  6:17   ` Raghavendra K T
2013-07-22  6:17 ` [PATCH RFC V11 6/18] xen/pvticketlocks: Add xen_nopvspin parameter to disable xen pv ticketlocks Raghavendra K T
2013-07-22  6:17   ` Raghavendra K T
2013-07-22  6:17   ` Raghavendra K T
2013-07-22  6:18 ` [PATCH RFC V11 7/18] x86/pvticketlock: Use callee-save for lock_spinning Raghavendra K T
2013-07-22  6:18   ` Raghavendra K T
2013-07-22  6:18   ` Raghavendra K T
2013-07-22  6:18 ` [PATCH RFC V11 8/18] x86/pvticketlock: When paravirtualizing ticket locks, increment by 2 Raghavendra K T
2013-07-22  6:18   ` Raghavendra K T
2013-07-22  6:18   ` Raghavendra K T
2013-07-22  6:18 ` [PATCH RFC V11 9/18] jump_label: Split out rate limiting from jump_label.h Raghavendra K T
2013-07-22  6:18   ` Raghavendra K T
2013-07-22  6:18   ` Raghavendra K T
2013-07-22  6:18 ` [PATCH RFC V11 10/18] x86/ticketlock: Add slowpath logic Raghavendra K T
2013-07-22  6:18   ` Raghavendra K T
2013-07-22  6:18   ` Raghavendra K T
2013-07-22  6:19 ` [PATCH RFC V11 11/18] xen/pvticketlock: Allow interrupts to be enabled while blocking Raghavendra K T
2013-07-22  6:19 ` Raghavendra K T
2013-07-22  6:19   ` Raghavendra K T
2013-07-22  6:19 ` [PATCH RFC V11 12/18] kvm hypervisor : Add a hypercall to KVM hypervisor to support pv-ticketlocks Raghavendra K T
2013-07-22  6:19   ` Raghavendra K T
2013-07-22  6:19   ` Raghavendra K T
2013-07-22  6:19 ` [PATCH RFC V11 13/18] kvm : Fold pv_unhalt flag into GET_MP_STATE ioctl to aid migration Raghavendra K T
2013-07-22  6:19   ` Raghavendra K T
2013-07-22  6:19   ` Raghavendra K T
2013-07-22  6:20 ` [PATCH RFC V11 14/18] kvm guest : Add configuration support to enable debug information for KVM Guests Raghavendra K T
2013-07-22  6:20   ` Raghavendra K T
2013-07-22  6:20   ` Raghavendra K T
2013-07-22  6:20 ` [PATCH RFC V11 15/18] kvm : Paravirtual ticketlocks support for linux guests running on KVM hypervisor Raghavendra K T
2013-07-22  6:20   ` Raghavendra K T
2013-07-22  6:20   ` Raghavendra K T
2013-07-23 15:07   ` Gleb Natapov
2013-07-23 15:07     ` Gleb Natapov
2013-07-24  9:24     ` [PATCH RESEND " Raghavendra K T
2013-07-24  9:24       ` Raghavendra K T
2013-07-24  9:45     ` [PATCH " Raghavendra K T
2013-07-24  9:45       ` Raghavendra K T
2013-07-24 10:39       ` Gleb Natapov
2013-07-24 10:39         ` Gleb Natapov
2013-07-24 12:00         ` Raghavendra K T
2013-07-24 12:00         ` Raghavendra K T
2013-07-24 12:06           ` Gleb Natapov
2013-07-24 12:06           ` Gleb Natapov
2013-07-24 12:36             ` Raghavendra K T
2013-07-24 12:36             ` Raghavendra K T
2013-07-25  9:17               ` Raghavendra K T
2013-07-25  9:17                 ` Raghavendra K T
2013-07-25  9:15                 ` Gleb Natapov
2013-07-25  9:15                   ` Gleb Natapov
2013-07-25  9:38                   ` Raghavendra K T
2013-07-25  9:38                     ` Raghavendra K T
2013-07-30 16:43                     ` Raghavendra K T
2013-07-30 16:43                       ` Raghavendra K T
2013-07-31  6:24                       ` Gleb Natapov
2013-07-31  6:24                         ` Gleb Natapov
2013-08-01  7:38                         ` Raghavendra K T
2013-08-01  7:38                           ` Raghavendra K T
2013-08-01  7:45                           ` Gleb Natapov
2013-08-01  7:45                             ` Gleb Natapov
2013-08-01  9:04                             ` Raghavendra K T
2013-08-02  3:22                               ` Raghavendra K T
2013-08-02  3:22                                 ` Raghavendra K T
2013-08-02  9:23                                 ` Ingo Molnar
2013-08-02  9:23                                   ` Ingo Molnar
2013-08-02  9:44                                   ` Raghavendra K T
2013-08-02  9:44                                     ` Raghavendra K T
2013-08-01  9:04                             ` Raghavendra K T
2013-08-02  9:25                           ` Ingo Molnar
2013-08-02  9:25                             ` Ingo Molnar
2013-08-02  9:54                             ` Gleb Natapov
2013-08-02  9:54                               ` Gleb Natapov
2013-08-02 10:57                               ` Raghavendra K T
2013-08-02 10:57                                 ` Raghavendra K T
2013-08-05  9:46                               ` Ingo Molnar
2013-08-05  9:46                                 ` Ingo Molnar
2013-08-05 10:42                                 ` Raghavendra K T
2013-08-05 10:42                                   ` Raghavendra K T
     [not found]                                 ` <20130805095901.GL2258@redhat.com>
2013-08-05 13:52                                   ` Ingo Molnar
2013-08-05 14:05                                     ` Paolo Bonzini
2013-08-05 14:05                                       ` Paolo Bonzini
2013-08-05 14:39                                       ` Raghavendra K T
2013-08-05 14:45                                         ` Paolo Bonzini
2013-08-05 14:45                                         ` Paolo Bonzini
2013-08-05 14:39                                       ` Raghavendra K T
2013-08-05 13:52                                   ` Ingo Molnar
2013-08-05 15:37                                 ` Konrad Rzeszutek Wilk
2013-08-05 15:37                                   ` Konrad Rzeszutek Wilk
2013-07-22  6:20 ` [PATCH RFC V11 16/18] kvm hypervisor : Simplify kvm_for_each_vcpu with kvm_irq_delivery_to_apic Raghavendra K T
2013-07-22  6:20   ` Raghavendra K T
2013-07-22  6:20   ` Raghavendra K T
2013-07-22  6:20 ` Raghavendra K T [this message]
2013-07-22  6:20   ` [PATCH RFC V11 17/18] Documentation/kvm : Add documentation on Hypercalls and features used for PV spinlock Raghavendra K T
2013-07-22  6:20   ` Raghavendra K T
2013-07-22  6:20 ` [PATCH RFC V11 18/18] kvm hypervisor: Add directed yield in vcpu block path Raghavendra K T
2013-07-22  6:20   ` Raghavendra K T
2013-07-22  6:20   ` Raghavendra K T
2013-07-22 19:36 ` [PATCH RFC V11 0/18] Paravirtualized ticket spinlocks Konrad Rzeszutek Wilk
2013-07-22 19:36   ` Konrad Rzeszutek Wilk
2013-07-23  2:50   ` Raghavendra K T
2013-07-23  2:50     ` Raghavendra K T
2013-08-05 22:50 ` H. Peter Anvin
2013-08-05 22:50   ` H. Peter Anvin
2013-08-06  2:50   ` Raghavendra K T
2013-08-06  2:50     ` Raghavendra K T

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20130722062038.24737.35006.sendpatchset@codeblue \
    --to=raghavendra.kt@linux.vnet.ibm.com \
    --cc=agraf@suse.de \
    --cc=andi@firstfloor.org \
    --cc=attilio.rao@citrix.com \
    --cc=avi.kivity@gmail.com \
    --cc=chegu_vinod@hp.com \
    --cc=drjones@redhat.com \
    --cc=gleb@redhat.com \
    --cc=gregkh@suse.de \
    --cc=habanero@linux.vnet.ibm.com \
    --cc=hpa@zytor.com \
    --cc=jeremy@goop.org \
    --cc=konrad.wilk@oracle.com \
    --cc=kvm@vger.kernel.org \
    --cc=linux-doc@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mingo@redhat.com \
    --cc=mtosatti@redhat.com \
    --cc=ouyang@cs.pitt.edu \
    --cc=pbonzini@redhat.com \
    --cc=peterz@infradead.org \
    --cc=riel@redhat.com \
    --cc=srivatsa.vaddagiri@gmail.com \
    --cc=stefano.stabellini@eu.citrix.com \
    --cc=tglx@linutronix.de \
    --cc=torvalds@linux-foundation.org \
    --cc=virtualization@lists.linux-foundation.org \
    --cc=x86@kernel.org \
    --cc=xen-devel@lists.xensource.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.