All of lore.kernel.org
 help / color / mirror / Atom feed
From: Alexander Graf <agraf@suse.de>
To: kvm-ppc@vger.kernel.org
Subject: Re: [PATCH 00/27] Add KVM support for Book3s_64 (PPC64) hosts v4
Date: Wed, 30 Sep 2009 09:11:00 +0000	[thread overview]
Message-ID: <0218D264-917A-4BA7-B39A-DBFFB96ACF97@suse.de> (raw)
In-Reply-To: <1254212303-8737-1-git-send-email-agraf@suse.de>


On 30.09.2009, at 10:59, Avi Kivity wrote:

> On 09/30/2009 10:47 AM, Alexander Graf wrote:
>>>
>>> What's the plan here?  While not a requirement for merging, that's  
>>> one of the kvm points of strength and I'd like to see it supported  
>>> across the board.
>>
>>
>> I'm having a deja vu :-).
>
> Will probably get one on every repost.

Yippie :)

>> The plan is to get qemu ppc64 guest support in a shape where it can  
>> actually use the KVM support. As it is it's rather useless.
>> When we have that, a PV interface would be needed to get things  
>> fast and then the next thing on my list is the MMU notifiers.
>
> Um.  How slow is it today?  What paths are problematic? mmu, context  
> switch?

Instruction emulation.

X86 with virtualization extensions doesn't trap often, as most of the  
state can be safely handled within the guest mode.
Now with PPC we're basically running in "ring 3" (called "problem  
state" in ppc speech) which traps all the time because guests change  
the IF or access some SPRs that we don't really need to trap on, but  
only need to sync state with on #VMEXIT.

So the PV idea here is to have a shared page between host and guest  
that contains guest specific SPRs and other state (an MSR shadow for  
example). That way the guest can patch itself to use that shared page  
and KVM always knows about the most current state on #VMEXIT. At the  
same time we're reducing exits by a _lot_.

A short kvm_stat during boot of a ppc32 guest on ppc64 shows what I'm  
talking about:

  dec                       3224     168
  exits                 18957500 1037240
  ext_intr                    75       5
  halt_wakeup               6874       0
  inst_emu               8570503  818597
  ld                           0       0
  ld_slow                      0       0
  mmio                   8719444   26249
  pf_instruc              302572   35379
  pf_storage             9215970   86750
  queue_intr              354020   31482
  sig                       7244     188
  sp_instruc              302541   35365
  sp_storage              370002   45370
  st                           0       0
  st_slow                      0       0
  sysc                     57907    5342


As you can see the bulk of exits are from MMIO and emulation.

We certainly won't be able to get rid of all the emulation exits, but  
quite a bunch of them aren't really that useful.

For MMIO we'll hopefully be able to use virtio.

> Our experience with pv on x86 has been mostly negative.  It's not  
> trivial to get security right, it ended up slower than non-pv, and  
> hardware obsoleted it fairly quickly.

Yes, and I really don't want to overdo it. PV for mfmsr/mtmsr and  
mfspr/mtspr is really necessary. X86 simply has that in hardware.

Alex

  parent reply	other threads:[~2009-09-30  9:11 UTC|newest]

Thread overview: 244+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2009-09-29  8:17 [PATCH 00/27] Add KVM support for Book3s_64 (PPC64) hosts v4 Alexander Graf
2009-09-30  8:42 ` Avi Kivity
2009-09-30  8:47 ` Alexander Graf
2009-09-30  8:59 ` Avi Kivity
2009-09-30  9:11 ` Alexander Graf [this message]
2009-09-30  9:24 ` Avi Kivity
2009-09-30  9:37 ` Alexander Graf
2009-10-02  0:26 ` Benjamin Herrenschmidt
2009-10-02  0:32 ` Benjamin Herrenschmidt
2009-10-03 10:08 ` Avi Kivity
2009-10-03 10:58 ` Benjamin Herrenschmidt
2009-10-03 11:10 ` Benjamin Herrenschmidt
2009-10-21 15:03 [PATCH 00/27] Add KVM support for Book3s_64 (PPC64) hosts v5 Alexander Graf
2009-10-21 15:03 ` Alexander Graf
2009-10-21 15:03 ` [PATCH 01/27] Move dirty logging code to sub-arch Alexander Graf
2009-10-21 15:03   ` Alexander Graf
     [not found]   ` <1256137413-15256-2-git-send-email-agraf-l3A5Bk7waGM@public.gmane.org>
2009-10-21 15:03     ` [PATCH 02/27] Pass PVR in sregs Alexander Graf
2009-10-21 15:03       ` Alexander Graf
2009-10-21 15:03       ` [PATCH 03/27] Add Book3s definitions Alexander Graf
2009-10-21 15:03         ` Alexander Graf
2009-10-21 15:03         ` [PATCH 04/27] Add Book3s fields to vcpu structs Alexander Graf
2009-10-21 15:03           ` Alexander Graf
2009-10-21 15:03           ` [PATCH 05/27] Add asm/kvm_book3s.h Alexander Graf
2009-10-21 15:03             ` Alexander Graf
2009-10-21 15:03             ` [PATCH 06/27] Add Book3s_64 intercept helpers Alexander Graf
2009-10-21 15:03               ` Alexander Graf
2009-10-21 15:03               ` [PATCH 07/27] Add book3s_64 highmem asm code Alexander Graf
2009-10-21 15:03                 ` Alexander Graf
     [not found]                 ` <1256137413-15256-8-git-send-email-agraf-l3A5Bk7waGM@public.gmane.org>
2009-10-21 15:03                   ` [PATCH 08/27] Add SLB switching code for entry/exit Alexander Graf
2009-10-21 15:03                     ` Alexander Graf
     [not found]                     ` <1256137413-15256-9-git-send-email-agraf-l3A5Bk7waGM@public.gmane.org>
2009-10-21 15:03                       ` [PATCH 09/27] Add interrupt handling code Alexander Graf
2009-10-21 15:03                         ` Alexander Graf
     [not found]                         ` <1256137413-15256-10-git-send-email-agraf-l3A5Bk7waGM@public.gmane.org>
2009-10-21 15:03                           ` [PATCH 10/27] Add book3s.c Alexander Graf
2009-10-21 15:03                             ` Alexander Graf
2009-10-21 15:03                             ` [PATCH 11/27] Add book3s_64 Host MMU handling Alexander Graf
2009-10-21 15:03                               ` Alexander Graf
2009-10-21 15:03                               ` [PATCH 12/27] Add book3s_64 guest MMU Alexander Graf
2009-10-21 15:03                                 ` Alexander Graf
     [not found]                                 ` <1256137413-15256-13-git-send-email-agraf-l3A5Bk7waGM@public.gmane.org>
2009-10-21 15:03                                   ` [PATCH 13/27] Add book3s_32 " Alexander Graf
2009-10-21 15:03                                     ` Alexander Graf
     [not found]                                     ` <1256137413-15256-14-git-send-email-agraf-l3A5Bk7waGM@public.gmane.org>
2009-10-21 15:03                                       ` [PATCH 14/27] Add book3s_64 specific opcode emulation Alexander Graf
2009-10-21 15:03                                         ` Alexander Graf
     [not found]                                         ` <1256137413-15256-15-git-send-email-agraf-l3A5Bk7waGM@public.gmane.org>
2009-10-21 15:03                                           ` [PATCH 15/27] Add mfdec emulation Alexander Graf
2009-10-21 15:03                                             ` Alexander Graf
     [not found]                                             ` <1256137413-15256-16-git-send-email-agraf-l3A5Bk7waGM@public.gmane.org>
2009-10-21 15:03                                               ` [PATCH 16/27] Add desktop PowerPC specific emulation Alexander Graf
2009-10-21 15:03                                                 ` Alexander Graf
     [not found]                                                 ` <1256137413-15256-17-git-send-email-agraf-l3A5Bk7waGM@public.gmane.org>
2009-10-21 15:03                                                   ` [PATCH 17/27] Make head_64.S aware of KVM real mode code Alexander Graf
2009-10-21 15:03                                                     ` Alexander Graf
2009-10-21 15:03                                                     ` [PATCH 18/27] Add Book3s_64 offsets to asm-offsets.c Alexander Graf
2009-10-21 15:03                                                       ` Alexander Graf
     [not found]                                                       ` <1256137413-15256-19-git-send-email-agraf-l3A5Bk7waGM@public.gmane.org>
2009-10-21 15:03                                                         ` [PATCH 19/27] Export symbols for KVM module Alexander Graf
2009-10-21 15:03                                                           ` Alexander Graf
     [not found]                                                           ` <1256137413-15256-20-git-send-email-agraf-l3A5Bk7waGM@public.gmane.org>
2009-10-21 15:03                                                             ` [PATCH 20/27] Split init_new_context and destroy_context Alexander Graf
2009-10-21 15:03                                                               ` Alexander Graf
     [not found]                                                               ` <1256137413-15256-21-git-send-email-agraf-l3A5Bk7waGM@public.gmane.org>
2009-10-21 15:03                                                                 ` [PATCH 21/27] Export KVM symbols for module Alexander Graf
2009-10-21 15:03                                                                   ` Alexander Graf
2009-10-21 15:03                                                                   ` [PATCH 22/27] Add fields to PACA Alexander Graf
2009-10-21 15:03                                                                     ` Alexander Graf
     [not found]                                                                     ` <1256137413-15256-23-git-send-email-agraf-l3A5Bk7waGM@public.gmane.org>
2009-10-21 15:03                                                                       ` [PATCH 23/27] Export new PACA constants in asm-offsets Alexander Graf
2009-10-21 15:03                                                                         ` Alexander Graf
2009-10-21 15:03                                                                         ` [PATCH 24/27] Include Book3s_64 target in buildsystem Alexander Graf
2009-10-21 15:03                                                                           ` Alexander Graf
     [not found]                                                                           ` <1256137413-15256-25-git-send-email-agraf-l3A5Bk7waGM@public.gmane.org>
2009-10-21 15:03                                                                             ` [PATCH 25/27] Fix trace.h Alexander Graf
2009-10-21 15:03                                                                               ` Alexander Graf
2009-10-21 15:03                                                                               ` [PATCH 26/27] Use Little Endian for Dirty Bitmap Alexander Graf
2009-10-21 15:03                                                                                 ` Alexander Graf
     [not found]                                                                                 ` <1256137413-15256-27-git-send-email-agraf-l3A5Bk7waGM@public.gmane.org>
2009-10-21 15:03                                                                                   ` [PATCH 27/27] Use hrtimers for the decrementer Alexander Graf
2009-10-21 15:03                                                                                     ` Alexander Graf
     [not found]                                                                         ` <1256137413-15256-24-git-send-email-agraf-l3A5Bk7waGM@public.gmane.org>
2009-10-29  2:50                                                                           ` [PATCH 23/27] Export new PACA constants in asm-offsets Benjamin Herrenschmidt
2009-10-29  2:50                                                                             ` Benjamin Herrenschmidt
2009-10-29  2:50                                                                       ` [PATCH 22/27] Add fields to PACA Benjamin Herrenschmidt
2009-10-29  2:50                                                                         ` Benjamin Herrenschmidt
2009-10-29  2:48                                                                 ` [PATCH 20/27] Split init_new_context and destroy_context Benjamin Herrenschmidt
2009-10-29  2:48                                                                   ` Benjamin Herrenschmidt
2009-10-29  2:46                                                             ` [PATCH 19/27] Export symbols for KVM module Benjamin Herrenschmidt
2009-10-29  2:46                                                               ` Benjamin Herrenschmidt
2009-10-29  2:53                                                               ` Alexander Graf
2009-10-29  2:53                                                                 ` Alexander Graf
2009-10-29  2:45                                                         ` [PATCH 18/27] Add Book3s_64 offsets to asm-offsets.c Benjamin Herrenschmidt
2009-10-29  2:45                                                           ` Benjamin Herrenschmidt
     [not found]                                                     ` <1256137413-15256-18-git-send-email-agraf-l3A5Bk7waGM@public.gmane.org>
2009-10-29  2:45                                                       ` [PATCH 17/27] Make head_64.S aware of KVM real mode code Benjamin Herrenschmidt
2009-10-29  2:45                                                         ` Benjamin Herrenschmidt
2009-10-21 15:22 ` [PATCH 00/27] Add KVM support for Book3s_64 (PPC64) hosts v5 Alexander Graf
2009-10-21 15:22   ` Alexander Graf
     [not found] ` <1256137413-15256-1-git-send-email-agraf-l3A5Bk7waGM@public.gmane.org>
2009-10-22 13:26   ` Arnd Bergmann
2009-10-22 13:26     ` Arnd Bergmann
2009-10-23  0:33   ` Hollis Blanchard
2009-10-23  0:33     ` Hollis Blanchard
     [not found]     ` <1256258028.7495.34.camel-6XWu2dSDoRTcKpUcGLbliUEOCMrvLtNR@public.gmane.org>
2009-10-25 13:01       ` Avi Kivity
2009-10-25 13:01         ` Avi Kivity
     [not found]         ` <4AE44C14.8040507-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
2009-10-26 21:18           ` Hollis Blanchard
2009-10-26 21:18             ` Hollis Blanchard
2009-10-29  2:55           ` Benjamin Herrenschmidt
2009-10-29  2:55             ` Benjamin Herrenschmidt
2009-10-26 22:46 ` Olof Johansson
2009-10-26 23:06   ` Olof Johansson
     [not found]   ` <20091026230632.GB5366-nZhT3qVonbNeoWH0uzbU5w@public.gmane.org>
2009-10-26 23:20     ` Hollis Blanchard
2009-10-26 23:20       ` Hollis Blanchard
2009-10-26 23:21       ` Olof Johansson
2009-10-26 23:21         ` Olof Johansson
2009-10-27  8:56         ` Avi Kivity
2009-10-27  8:56           ` Avi Kivity
2009-10-27 13:42           ` Alexander Graf
2009-10-27 13:42             ` Alexander Graf
     [not found]             ` <8E92E3B9-39D5-4D71-8B8E-96B49430B67B-l3A5Bk7waGM@public.gmane.org>
2009-10-27 15:49               ` Avi Kivity
2009-10-27 15:49                 ` Avi Kivity
2009-10-30 15:47 [PATCH 00/27] Add KVM support for Book3s_64 (PPC64) hosts v6 Alexander Graf
2009-10-30 15:47 ` Alexander Graf
2009-10-30 15:47 ` Alexander Graf
2009-10-30 15:47 ` [PATCH 02/27] Pass PVR in sregs Alexander Graf
2009-10-30 15:47   ` Alexander Graf
2009-10-30 15:47   ` Alexander Graf
2009-10-30 15:47 ` [PATCH 04/27] Add Book3s fields to vcpu structs Alexander Graf
2009-10-30 15:47   ` Alexander Graf
2009-10-30 15:47   ` Alexander Graf
2009-10-30 15:47 ` [PATCH 05/27] Add asm/kvm_book3s.h Alexander Graf
2009-10-30 15:47   ` Alexander Graf
2009-10-30 15:47   ` Alexander Graf
2009-10-30 15:47 ` [PATCH 06/27] Add Book3s_64 intercept helpers Alexander Graf
2009-10-30 15:47   ` Alexander Graf
2009-10-30 15:47   ` Alexander Graf
2009-10-30 15:47 ` [PATCH 11/27] Add book3s_64 Host MMU handling Alexander Graf
2009-10-30 15:47   ` Alexander Graf
2009-10-30 15:47   ` Alexander Graf
     [not found]   ` <1256917647-6200-12-git-send-email-agraf-l3A5Bk7waGM@public.gmane.org>
2009-11-01 23:39     ` Michael Neuling
2009-11-01 23:39       ` Michael Neuling
2009-11-01 23:39       ` Michael Neuling
2009-11-02  9:26       ` Alexander Graf
2009-11-02  9:26         ` Alexander Graf
2009-11-02  9:26         ` Alexander Graf
2009-10-30 15:47 ` [PATCH 17/27] Make head_64.S aware of KVM real mode code Alexander Graf
2009-10-30 15:47   ` Alexander Graf
2009-10-30 15:47   ` Alexander Graf
2009-10-30 15:47 ` [PATCH 18/27] Add Book3s_64 offsets to asm-offsets.c Alexander Graf
2009-10-30 15:47   ` Alexander Graf
2009-10-30 15:47   ` Alexander Graf
     [not found] ` <1256917647-6200-1-git-send-email-agraf-l3A5Bk7waGM@public.gmane.org>
2009-10-30 15:47   ` [PATCH 01/27] Move dirty logging code to sub-arch Alexander Graf
2009-10-30 15:47     ` Alexander Graf
2009-10-30 15:47     ` Alexander Graf
2009-10-30 15:47   ` [PATCH 03/27] Add Book3s definitions Alexander Graf
2009-10-30 15:47     ` Alexander Graf
2009-10-30 15:47     ` Alexander Graf
2009-10-30 15:47   ` [PATCH 07/27] Add book3s_64 highmem asm code Alexander Graf
2009-10-30 15:47     ` Alexander Graf
2009-10-30 15:47     ` Alexander Graf
2009-10-30 15:47   ` [PATCH 08/27] Add SLB switching code for entry/exit Alexander Graf
2009-10-30 15:47     ` Alexander Graf
2009-10-30 15:47     ` Alexander Graf
2009-11-01 23:23     ` Michael Neuling
2009-11-01 23:23       ` Michael Neuling
2009-11-01 23:23       ` Michael Neuling
     [not found]       ` <6695.1257117827-/owAOxkjmzZAfugRpC6u6w@public.gmane.org>
2009-11-02  9:23         ` Alexander Graf
2009-11-02  9:23           ` Alexander Graf
2009-11-02  9:23           ` Alexander Graf
     [not found]           ` <00BF2D99-F2CE-4204-B4B4-0D113FD54CE6-l3A5Bk7waGM@public.gmane.org>
2009-11-02  9:39             ` Michael Neuling
2009-11-02  9:39               ` Michael Neuling
2009-11-02  9:39               ` Michael Neuling
2009-11-02  9:59               ` Alexander Graf
2009-11-02  9:59                 ` Alexander Graf
2009-11-02  9:59                 ` Alexander Graf
2009-10-30 15:47   ` [PATCH 09/27] Add interrupt handling code Alexander Graf
2009-10-30 15:47     ` Alexander Graf
2009-10-30 15:47     ` Alexander Graf
2009-10-30 15:47   ` [PATCH 10/27] Add book3s.c Alexander Graf
2009-10-30 15:47     ` Alexander Graf
2009-10-30 15:47     ` Alexander Graf
2009-10-30 15:47   ` [PATCH 12/27] Add book3s_64 guest MMU Alexander Graf
2009-10-30 15:47     ` Alexander Graf
2009-10-30 15:47     ` Alexander Graf
2009-10-30 15:47   ` [PATCH 13/27] Add book3s_32 " Alexander Graf
2009-10-30 15:47     ` Alexander Graf
2009-10-30 15:47     ` Alexander Graf
2009-10-30 15:47   ` [PATCH 14/27] Add book3s_64 specific opcode emulation Alexander Graf
2009-10-30 15:47     ` Alexander Graf
2009-10-30 15:47     ` Alexander Graf
2009-11-03  8:47     ` Segher Boessenkool
2009-11-03  8:47       ` Segher Boessenkool
     [not found]       ` <A1CBD511-FF08-48BB-A8D6-9F66E20F770B-XVmvHMARGAS8U2dJNN8I7kB+6BGkLq7r@public.gmane.org>
2009-11-03  9:06         ` Alexander Graf
2009-11-03  9:06           ` Alexander Graf
2009-11-03  9:06           ` Alexander Graf
2009-11-03 21:38           ` Benjamin Herrenschmidt
2009-11-03 21:38             ` Benjamin Herrenschmidt
2009-11-04  8:43             ` Arnd Bergmann
2009-11-04  8:43               ` Arnd Bergmann
2009-11-04  8:47               ` Benjamin Herrenschmidt
2009-11-04  8:47                 ` Benjamin Herrenschmidt
2009-11-04 11:35                 ` Alexander Graf
2009-11-04 11:35                   ` Alexander Graf
2009-11-04 11:35                   ` Alexander Graf
2009-11-05  0:53           ` Segher Boessenkool
2009-11-05  0:53             ` Segher Boessenkool
2009-11-05 10:09             ` Alexander Graf
2009-11-05 10:09               ` Alexander Graf
2009-11-05 10:09               ` Alexander Graf
2009-10-30 15:47   ` [PATCH 15/27] Add mfdec emulation Alexander Graf
2009-10-30 15:47     ` Alexander Graf
2009-10-30 15:47     ` Alexander Graf
2009-10-30 15:47   ` [PATCH 16/27] Add desktop PowerPC specific emulation Alexander Graf
2009-10-30 15:47     ` Alexander Graf
2009-10-30 15:47     ` Alexander Graf
2009-10-30 15:47   ` [PATCH 19/27] Export symbols for KVM module Alexander Graf
2009-10-30 15:47     ` Alexander Graf
2009-10-30 15:47     ` Alexander Graf
     [not found]     ` <1256917647-6200-20-git-send-email-agraf-l3A5Bk7waGM@public.gmane.org>
2009-10-31  4:37       ` Stephen Rothwell
2009-10-31  4:37         ` Stephen Rothwell
2009-10-31  4:37         ` Stephen Rothwell
     [not found]         ` <20091031153719.10a4e61b.sfr-3FnU+UHB4dNDw9hX6IcOSA@public.gmane.org>
2009-10-31 12:02           ` Alexander Graf
2009-10-31 12:02             ` Alexander Graf
2009-10-31 12:02             ` Alexander Graf
2009-10-30 15:47   ` [PATCH 20/27] Split init_new_context and destroy_context Alexander Graf
2009-10-30 15:47     ` Alexander Graf
2009-10-30 15:47     ` Alexander Graf
2009-10-31  4:40     ` Stephen Rothwell
2009-10-31  4:40       ` Stephen Rothwell
2009-10-31  4:40       ` Stephen Rothwell
2009-10-31 21:20       ` Alexander Graf
2009-10-31 21:20         ` Alexander Graf
2009-10-31 21:20         ` Alexander Graf
2009-10-31 21:37         ` Benjamin Herrenschmidt
2009-10-31 21:37           ` Benjamin Herrenschmidt
2009-10-30 15:47   ` [PATCH 21/27] Export KVM symbols for module Alexander Graf
2009-10-30 15:47     ` Alexander Graf
2009-10-30 15:47     ` Alexander Graf
2009-10-30 15:47   ` [PATCH 22/27] Add fields to PACA Alexander Graf
2009-10-30 15:47     ` Alexander Graf
2009-10-30 15:47     ` Alexander Graf
2009-10-30 15:47   ` [PATCH 27/27] Use hrtimers for the decrementer Alexander Graf
2009-10-30 15:47     ` Alexander Graf
2009-10-30 15:47     ` Alexander Graf
2009-11-05  6:03   ` [PATCH 00/27] Add KVM support for Book3s_64 (PPC64) hosts v6 Benjamin Herrenschmidt
2009-11-05  6:03     ` Benjamin Herrenschmidt
2009-11-05  6:03     ` Benjamin Herrenschmidt
2009-10-30 15:47 ` [PATCH 23/27] Export new PACA constants in asm-offsets Alexander Graf
2009-10-30 15:47   ` Alexander Graf
2009-10-30 15:47   ` Alexander Graf
2009-10-30 15:47 ` [PATCH 24/27] Include Book3s_64 target in buildsystem Alexander Graf
2009-10-30 15:47   ` Alexander Graf
2009-10-30 15:47   ` Alexander Graf
2009-10-30 15:47 ` [PATCH 25/27] Fix trace.h Alexander Graf
2009-10-30 15:47   ` Alexander Graf
2009-10-30 15:47   ` Alexander Graf
2009-10-30 15:47 ` [PATCH 26/27] Use Little Endian for Dirty Bitmap Alexander Graf
2009-10-30 15:47   ` Alexander Graf
2009-10-30 15:47   ` Alexander Graf

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=0218D264-917A-4BA7-B39A-DBFFB96ACF97@suse.de \
    --to=agraf@suse.de \
    --cc=kvm-ppc@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.