All of lore.kernel.org
 help / color / mirror / Atom feed
From: "Chen, Tiejun" <tiejun.chen@intel.com>
To: Anthony PERARD <anthony.perard@citrix.com>
Cc: "peter.maydell@linaro.org" <peter.maydell@linaro.org>,
	"xen-devel@lists.xensource.com" <xen-devel@lists.xensource.com>,
	"mst@redhat.com" <mst@redhat.com>,
	"stefano.stabellini@eu.citrix.com"
	<stefano.stabellini@eu.citrix.com>,
	"Kay, Allen M" <allen.m.kay@intel.com>,
	"Kelly.Zytaruk@amd.com" <Kelly.Zytaruk@amd.com>,
	"qemu-devel@nongnu.org" <qemu-devel@nongnu.org>,
	Gerd Hoffmann <kraxel@redhat.com>,
	"anthony@codemonkey.ws" <anthony@codemonkey.ws>,
	"Zhang, Yang Z" <yang.z.zhang@intel.com>
Subject: Re: [Qemu-devel] [v2][PATCH 4/8] xen, gfx passthrough: reserve 00:02.0 for INTEL IGD
Date: Wed, 21 May 2014 01:25:03 +0000	[thread overview]
Message-ID: <CCCF29FA0F52DC4B8E2D1EF38AE07FF7B3778D@SHSMSX101.ccr.corp.intel.com> (raw)
In-Reply-To: <20140520144510.GD4121@perard.uk.xensource.com>

> -----Original Message-----
> From: Anthony PERARD [mailto:anthony.perard@citrix.com]
> Sent: Tuesday, May 20, 2014 10:45 PM
> To: Chen, Tiejun
> Cc: Gerd Hoffmann; stefano.stabellini@eu.citrix.com; mst@redhat.com;
> Kelly.Zytaruk@amd.com; peter.maydell@linaro.org;
> xen-devel@lists.xensource.com; Kay, Allen M; qemu-devel@nongnu.org;
> anthony@codemonkey.ws; Zhang, Yang Z
> Subject: Re: [Qemu-devel] [v2][PATCH 4/8] xen, gfx passthrough: reserve
> 00:02.0 for INTEL IGD
> 
> On Mon, May 19, 2014 at 12:04:14PM +0000, Chen, Tiejun wrote:
> > > -----Original Message-----
> > > From: Gerd Hoffmann [mailto:kraxel@redhat.com]
> > > Sent: Monday, May 19, 2014 7:23 PM
> > > To: Chen, Tiejun
> > > Cc: anthony.perard@citrix.com; stefano.stabellini@eu.citrix.com;
> > > mst@redhat.com; Kelly.Zytaruk@amd.com; peter.maydell@linaro.org;
> > > xen-devel@lists.xensource.com; Kay, Allen M; qemu-devel@nongnu.org;
> > > jean.guyader@eu.citrix.com; anthony@codemonkey.ws; Zhang, Yang Z
> > > Subject: Re: [Qemu-devel] [v2][PATCH 4/8] xen, gfx passthrough:
> > > reserve
> > > 00:02.0 for INTEL IGD
> > >
> > >   Hi,
> > >
> > > > I think '-vga none' just guarantees the qemu vga cards doesn't
> > > > occupy 00:02.0, but this doesn't mean others use this specific
> > > > slot since in qemu internal, we always pass -1 to assign a slot
> > > > automatically to register a PCI device. So in some cases, we can't
> > > > get this slot as we expect since that is already assigned previously before
> we need this.
> > >
> > > Yes, -vga, -net nic, -drive if=scsi (maybe more) can internally
> > > create pci devices with auto slot assignment, which will occupy slot 2
> indeed.
> > > Use -device instead to create the devices.
> > >
> >
> > Are you saying we have to create the devices explicitly when we want to work
> IGD vga with passthrough? But how to make sure all user know this workable
> way? Maybe you suggest we should document somewhere.
> >
> > > > PCI: slot 2 function 0 not available for xen-pci-passthrough, in
> > > > use by xen-platform
> > >
> > > Ah, the xen platform device.  /me looks.  Ah, pc_xen_hvm_init
> > > creates this automatically.  Two options here IMHO:
> > >
> > >   (1) Just move it somewhere else explicitly.  For example slot 3, or
> > >       make it a southbridge function (say 00:01.7).
> > >   (2) Don't create it automatically, instead expect management add it
> > >       if needed, using -device xen-plaform,addr=...
> > >
> > > I personally would suggest to go for #2.  As far I know the platform
> > > device is only needed if you want attach xenbus devices to the guest
> > > (correct?), so creating virtual machines without the xen platform
> > > device is a valid use case and you should allow it.
> > >
> >
> > Looks you recommend we should change current xen platform design, I'm not
> sure if something in libxl also need to be modified. Especially, this may not be
> compatible with those old xen version. And especially, how to guarantee no one
> occupy 00:02.0 in the future with auto assign?
> 
> Hi,
> 
> Libxl knows how to create a guest without xen-platform, but I think this can
> work only from QEMU 1.6 (which Xen 4.4 use).
> 
> So, using the options below with libxl would put the xen-platform device away
> from the 00:02 slot.
> xen_platform_pci=0
> device_model_args_hvm = [ '-device', 'xen-platform,addr=0x3' ]
> 

I added these two lines in domU's configuration file, but failed:

tchen0@tchen0-linux:~/workspace$ sudo xl cr domu-cfg                                                                                                       
Parsing config from domu-cfg
libxl: error: libxl_qmp.c:287:qmp_handle_error_response: received an error message from QMP server: Unsupported bus. Bus doesn't have property 'acpi-pcihp-bsel' set
libxl: error: libxl_create.c:1277:domcreate_attach_pci: libxl_device_pci_add failed: -3
tchen0@tchen0-linux:~/workspace$ cat /var/log/xen/qemu-dm-HVM.log                                                                                                      
char device redirected to /dev/pts/49 (label serial0)
qemu: terminating on signal 1 from pid 3117

Anything else I'm still missing? Or this mean this way can't work based on the latest qemu/xen?

Thanks
Tiejun

WARNING: multiple messages have this Message-ID (diff)
From: "Chen, Tiejun" <tiejun.chen@intel.com>
To: Anthony PERARD <anthony.perard@citrix.com>
Cc: "peter.maydell@linaro.org" <peter.maydell@linaro.org>,
	"xen-devel@lists.xensource.com" <xen-devel@lists.xensource.com>,
	"mst@redhat.com" <mst@redhat.com>,
	"stefano.stabellini@eu.citrix.com"
	<stefano.stabellini@eu.citrix.com>,
	"Kay, Allen M" <allen.m.kay@intel.com>,
	"Kelly.Zytaruk@amd.com" <Kelly.Zytaruk@amd.com>,
	"qemu-devel@nongnu.org" <qemu-devel@nongnu.org>,
	Gerd Hoffmann <kraxel@redhat.com>,
	"anthony@codemonkey.ws" <anthony@codemonkey.ws>,
	"Zhang, Yang Z" <yang.z.zhang@intel.com>
Subject: Re: [v2][PATCH 4/8] xen, gfx passthrough: reserve 00:02.0 for INTEL IGD
Date: Wed, 21 May 2014 01:25:03 +0000	[thread overview]
Message-ID: <CCCF29FA0F52DC4B8E2D1EF38AE07FF7B3778D@SHSMSX101.ccr.corp.intel.com> (raw)
In-Reply-To: <20140520144510.GD4121@perard.uk.xensource.com>

> -----Original Message-----
> From: Anthony PERARD [mailto:anthony.perard@citrix.com]
> Sent: Tuesday, May 20, 2014 10:45 PM
> To: Chen, Tiejun
> Cc: Gerd Hoffmann; stefano.stabellini@eu.citrix.com; mst@redhat.com;
> Kelly.Zytaruk@amd.com; peter.maydell@linaro.org;
> xen-devel@lists.xensource.com; Kay, Allen M; qemu-devel@nongnu.org;
> anthony@codemonkey.ws; Zhang, Yang Z
> Subject: Re: [Qemu-devel] [v2][PATCH 4/8] xen, gfx passthrough: reserve
> 00:02.0 for INTEL IGD
> 
> On Mon, May 19, 2014 at 12:04:14PM +0000, Chen, Tiejun wrote:
> > > -----Original Message-----
> > > From: Gerd Hoffmann [mailto:kraxel@redhat.com]
> > > Sent: Monday, May 19, 2014 7:23 PM
> > > To: Chen, Tiejun
> > > Cc: anthony.perard@citrix.com; stefano.stabellini@eu.citrix.com;
> > > mst@redhat.com; Kelly.Zytaruk@amd.com; peter.maydell@linaro.org;
> > > xen-devel@lists.xensource.com; Kay, Allen M; qemu-devel@nongnu.org;
> > > jean.guyader@eu.citrix.com; anthony@codemonkey.ws; Zhang, Yang Z
> > > Subject: Re: [Qemu-devel] [v2][PATCH 4/8] xen, gfx passthrough:
> > > reserve
> > > 00:02.0 for INTEL IGD
> > >
> > >   Hi,
> > >
> > > > I think '-vga none' just guarantees the qemu vga cards doesn't
> > > > occupy 00:02.0, but this doesn't mean others use this specific
> > > > slot since in qemu internal, we always pass -1 to assign a slot
> > > > automatically to register a PCI device. So in some cases, we can't
> > > > get this slot as we expect since that is already assigned previously before
> we need this.
> > >
> > > Yes, -vga, -net nic, -drive if=scsi (maybe more) can internally
> > > create pci devices with auto slot assignment, which will occupy slot 2
> indeed.
> > > Use -device instead to create the devices.
> > >
> >
> > Are you saying we have to create the devices explicitly when we want to work
> IGD vga with passthrough? But how to make sure all user know this workable
> way? Maybe you suggest we should document somewhere.
> >
> > > > PCI: slot 2 function 0 not available for xen-pci-passthrough, in
> > > > use by xen-platform
> > >
> > > Ah, the xen platform device.  /me looks.  Ah, pc_xen_hvm_init
> > > creates this automatically.  Two options here IMHO:
> > >
> > >   (1) Just move it somewhere else explicitly.  For example slot 3, or
> > >       make it a southbridge function (say 00:01.7).
> > >   (2) Don't create it automatically, instead expect management add it
> > >       if needed, using -device xen-plaform,addr=...
> > >
> > > I personally would suggest to go for #2.  As far I know the platform
> > > device is only needed if you want attach xenbus devices to the guest
> > > (correct?), so creating virtual machines without the xen platform
> > > device is a valid use case and you should allow it.
> > >
> >
> > Looks you recommend we should change current xen platform design, I'm not
> sure if something in libxl also need to be modified. Especially, this may not be
> compatible with those old xen version. And especially, how to guarantee no one
> occupy 00:02.0 in the future with auto assign?
> 
> Hi,
> 
> Libxl knows how to create a guest without xen-platform, but I think this can
> work only from QEMU 1.6 (which Xen 4.4 use).
> 
> So, using the options below with libxl would put the xen-platform device away
> from the 00:02 slot.
> xen_platform_pci=0
> device_model_args_hvm = [ '-device', 'xen-platform,addr=0x3' ]
> 

I added these two lines in domU's configuration file, but failed:

tchen0@tchen0-linux:~/workspace$ sudo xl cr domu-cfg                                                                                                       
Parsing config from domu-cfg
libxl: error: libxl_qmp.c:287:qmp_handle_error_response: received an error message from QMP server: Unsupported bus. Bus doesn't have property 'acpi-pcihp-bsel' set
libxl: error: libxl_create.c:1277:domcreate_attach_pci: libxl_device_pci_add failed: -3
tchen0@tchen0-linux:~/workspace$ cat /var/log/xen/qemu-dm-HVM.log                                                                                                      
char device redirected to /dev/pts/49 (label serial0)
qemu: terminating on signal 1 from pid 3117

Anything else I'm still missing? Or this mean this way can't work based on the latest qemu/xen?

Thanks
Tiejun

  reply	other threads:[~2014-05-21  1:25 UTC|newest]

Thread overview: 150+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2014-05-16 10:53 [Qemu-devel] [v2][PATCH 0/8] xen: add Intel IGD passthrough support Tiejun Chen
2014-05-16 10:53 ` Tiejun Chen
2014-05-16 10:53 ` [Qemu-devel] [v2][PATCH 1/8] pci: use bitmap to manage registe/runregister pci device Tiejun Chen
2014-05-16 10:53   ` Tiejun Chen
2014-05-16 10:53 ` [Qemu-devel] [v2][PATCH 2/8] pci: provide a way to reserve some specific devfn Tiejun Chen
2014-05-16 10:53   ` Tiejun Chen
2014-05-16 14:07   ` [Qemu-devel] [Xen-devel] " Konrad Rzeszutek Wilk
2014-05-16 14:07     ` Konrad Rzeszutek Wilk
2014-05-19  9:43     ` [Qemu-devel] " Chen, Tiejun
2014-05-19  9:43       ` Chen, Tiejun
2014-05-16 10:53 ` [Qemu-devel] [v2][PATCH 3/8] xen, gfx passthrough: basic graphics passthrough support Tiejun Chen
2014-05-16 10:53   ` Tiejun Chen
2014-05-16 14:06   ` [Qemu-devel] [Xen-devel] " Konrad Rzeszutek Wilk
2014-05-16 14:06     ` Konrad Rzeszutek Wilk
2014-05-19  9:42     ` [Qemu-devel] " Chen, Tiejun
2014-05-19  9:42       ` Chen, Tiejun
2014-05-19 13:35       ` [Qemu-devel] " Konrad Rzeszutek Wilk
2014-05-19 13:35         ` Konrad Rzeszutek Wilk
2014-05-20  9:32         ` [Qemu-devel] " Chen, Tiejun
2014-05-20  9:32           ` Chen, Tiejun
2014-05-19 12:10     ` [Qemu-devel] " Stefano Stabellini
2014-05-19 12:10       ` Stefano Stabellini
2014-05-20  5:09       ` [Qemu-devel] " Chen, Tiejun
2014-05-20  5:09         ` Chen, Tiejun
2014-05-16 10:53 ` [Qemu-devel] [v2][PATCH 4/8] xen, gfx passthrough: reserve 00:02.0 for INTEL IGD Tiejun Chen
2014-05-16 10:53   ` Tiejun Chen
2014-05-16 14:08   ` [Qemu-devel] [Xen-devel] " Konrad Rzeszutek Wilk
2014-05-16 14:08     ` Konrad Rzeszutek Wilk
2014-05-19  9:54     ` [Qemu-devel] " Chen, Tiejun
2014-05-19  9:54       ` Chen, Tiejun
2014-05-19  6:44   ` [Qemu-devel] " Gerd Hoffmann
2014-05-19  6:44     ` Gerd Hoffmann
2014-05-19  7:48     ` [Qemu-devel] [Xen-devel] " Fabio Fantoni
2014-05-19  7:48       ` Fabio Fantoni
2014-05-19  8:15       ` [Qemu-devel] " Zhang, Yang Z
2014-05-19  8:15         ` Zhang, Yang Z
2014-05-19  9:34       ` [Qemu-devel] " Chen, Tiejun
2014-05-19  9:34         ` Chen, Tiejun
2014-05-19  9:25     ` [Qemu-devel] " Chen, Tiejun
2014-05-19  9:25       ` Chen, Tiejun
2014-05-19 10:13       ` [Qemu-devel] " Michael S. Tsirkin
2014-05-19 10:13         ` Michael S. Tsirkin
2014-05-20  9:34         ` [Qemu-devel] " Chen, Tiejun
2014-05-20  9:34           ` Chen, Tiejun
2014-05-20 11:36           ` [Qemu-devel] " Michael S. Tsirkin
2014-05-20 11:36             ` Michael S. Tsirkin
2014-05-19 11:22       ` [Qemu-devel] " Gerd Hoffmann
2014-05-19 11:22         ` Gerd Hoffmann
2014-05-19 12:04         ` [Qemu-devel] " Chen, Tiejun
2014-05-19 12:04           ` Chen, Tiejun
2014-05-19 13:50           ` [Qemu-devel] " Gerd Hoffmann
2014-05-19 13:50             ` Gerd Hoffmann
2014-05-19 14:00             ` [Qemu-devel] " Daniel P. Berrange
2014-05-19 14:00               ` Daniel P. Berrange
2014-05-21  7:07             ` [Qemu-devel] " Chen, Tiejun
2014-05-21  7:07               ` Chen, Tiejun
2014-05-22  0:31               ` [Qemu-devel] " Chen, Tiejun
2014-05-22  0:31                 ` Chen, Tiejun
2014-05-22  5:39                 ` [Qemu-devel] " Gerd Hoffmann
2014-05-22  5:39                   ` Gerd Hoffmann
2014-05-22  6:18                   ` [Qemu-devel] " Chen, Tiejun
2014-05-22  6:18                     ` Chen, Tiejun
2014-05-22  6:44                     ` [Qemu-devel] " Gerd Hoffmann
2014-05-22  6:44                       ` Gerd Hoffmann
2014-05-22  6:49                       ` [Qemu-devel] " Michael S. Tsirkin
2014-05-22  6:49                         ` Michael S. Tsirkin
2014-05-22  7:11                       ` [Qemu-devel] " Chen, Tiejun
2014-05-22  7:11                         ` Chen, Tiejun
2014-05-22 10:50                       ` [Qemu-devel] " Chen, Tiejun
2014-05-22 10:50                         ` Chen, Tiejun
2014-05-22 11:03                         ` [Qemu-devel] " Gonglei (Arei)
2014-05-22 11:03                           ` Gonglei (Arei)
2014-05-22 11:22                         ` [Qemu-devel] " Gerd Hoffmann
2014-05-22 11:22                           ` Gerd Hoffmann
2014-05-23  1:07                           ` Chen, Tiejun
2014-05-23  1:07                             ` Chen, Tiejun
2014-05-22 11:25                         ` [Qemu-devel] " Michael S. Tsirkin
2014-05-22 11:25                           ` Michael S. Tsirkin
2014-05-22 14:20                           ` [Qemu-devel] [Xen-devel] " Igor Mammedov
2014-05-22 14:20                             ` Igor Mammedov
2014-05-23  1:18                             ` [Qemu-devel] " Chen, Tiejun
2014-05-23  1:18                               ` Chen, Tiejun
2014-05-23  7:38                               ` [Qemu-devel] " Igor Mammedov
2014-05-23  7:38                                 ` Igor Mammedov
2014-05-23 10:52                                 ` [Qemu-devel] " Anthony PERARD
2014-05-23 10:52                                   ` Anthony PERARD
2014-05-23 11:40                                   ` [Qemu-devel] " Stefano Stabellini
2014-05-23 11:40                                     ` Stefano Stabellini
2014-05-23 11:53                                     ` [Qemu-devel] " Gerd Hoffmann
2014-05-23 11:53                                       ` Gerd Hoffmann
2014-05-23 12:06                                   ` [Qemu-devel] " Igor Mammedov
2014-05-23 12:06                                     ` Igor Mammedov
2014-05-23 12:16                                   ` [Qemu-devel] " Igor Mammedov
2014-05-23 12:16                                     ` Igor Mammedov
2014-05-22  9:58                   ` [Qemu-devel] " Konrad Rzeszutek Wilk
2014-05-22  9:58                     ` Konrad Rzeszutek Wilk
2014-05-20 14:45           ` [Qemu-devel] " Anthony PERARD
2014-05-20 14:45             ` Anthony PERARD
2014-05-21  1:25             ` Chen, Tiejun [this message]
2014-05-21  1:25               ` Chen, Tiejun
2014-06-25  2:49         ` [Qemu-devel] " Chen, Tiejun
2014-06-25  2:49           ` Chen, Tiejun
2014-06-25 23:04           ` [Qemu-devel] " Slutz, Donald Christopher
2014-06-25 23:04             ` Slutz, Donald Christopher
2014-06-26  2:00             ` [Qemu-devel] " Chen, Tiejun
2014-06-26  2:00               ` Chen, Tiejun
2014-06-26  8:23               ` [Qemu-devel] " Chen, Tiejun
2014-06-26  8:23                 ` Chen, Tiejun
2014-06-26 16:58                 ` [Qemu-devel] " Don Slutz
2014-06-26 16:58                   ` Don Slutz
2014-06-30  9:29                   ` [Qemu-devel] " Gerd Hoffmann
2014-06-30  9:29                     ` Gerd Hoffmann
2014-06-30 10:23                     ` [Qemu-devel] " Chen, Tiejun
2014-06-30 10:23                       ` Chen, Tiejun
2014-05-16 10:53 ` [Qemu-devel] [v2][PATCH 5/8] xen, gfx passthrough: create intel isa bridge Tiejun Chen
2014-05-16 10:53   ` Tiejun Chen
2014-05-16 14:11   ` [Qemu-devel] [Xen-devel] " Konrad Rzeszutek Wilk
2014-05-16 14:11     ` Konrad Rzeszutek Wilk
2014-05-19  9:59     ` [Qemu-devel] " Chen, Tiejun
2014-05-19  9:59       ` Chen, Tiejun
2014-05-16 10:53 ` [Qemu-devel] [v2][PATCH 6/8] xen, gfx passthrough: support Intel IGD passthrough with VT-D Tiejun Chen
2014-05-16 10:53   ` Tiejun Chen
2014-05-16 14:35   ` [Qemu-devel] [Xen-devel] " Konrad Rzeszutek Wilk
2014-05-16 14:35     ` Konrad Rzeszutek Wilk
2014-05-19  0:58     ` [Qemu-devel] " Zhang, Yang Z
2014-05-19  0:58       ` Zhang, Yang Z
2014-05-19 13:34       ` [Qemu-devel] " Konrad Rzeszutek Wilk
2014-05-19 13:34         ` Konrad Rzeszutek Wilk
2014-05-20  5:13         ` [Qemu-devel] " Chen, Tiejun
2014-05-20  5:13           ` Chen, Tiejun
2014-05-20 13:39           ` [Qemu-devel] " Konrad Rzeszutek Wilk
2014-05-20 13:39             ` Konrad Rzeszutek Wilk
2014-05-19 10:02     ` [Qemu-devel] [Xen-devel] " Chen, Tiejun
2014-05-19 10:02       ` Chen, Tiejun
2014-05-16 10:53 ` [Qemu-devel] [v2][PATCH 7/8] xen, gfx passthrough: create host bridge to passthrough Tiejun Chen
2014-05-16 10:53   ` Tiejun Chen
2014-05-16 14:37   ` [Qemu-devel] [Xen-devel] " Konrad Rzeszutek Wilk
2014-05-16 14:37     ` Konrad Rzeszutek Wilk
2014-05-19 10:03     ` [Qemu-devel] " Chen, Tiejun
2014-05-19 10:03       ` Chen, Tiejun
2014-05-16 10:53 ` [Qemu-devel] [v2][PATCH 8/8] xen, gfx passthrough: add opregion mapping Tiejun Chen
2014-05-16 10:53   ` Tiejun Chen
2014-05-19 11:53   ` [Qemu-devel] " Stefano Stabellini
2014-05-19 11:53     ` Stefano Stabellini
2014-05-20  9:24     ` [Qemu-devel] " Chen, Tiejun
2014-05-20  9:24       ` Chen, Tiejun
2014-05-20 10:50       ` [Qemu-devel] " Stefano Stabellini
2014-05-20 10:50         ` Stefano Stabellini
2014-05-21  0:57         ` [Qemu-devel] " Chen, Tiejun
2014-05-21  0:57           ` Chen, Tiejun

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=CCCF29FA0F52DC4B8E2D1EF38AE07FF7B3778D@SHSMSX101.ccr.corp.intel.com \
    --to=tiejun.chen@intel.com \
    --cc=Kelly.Zytaruk@amd.com \
    --cc=allen.m.kay@intel.com \
    --cc=anthony.perard@citrix.com \
    --cc=anthony@codemonkey.ws \
    --cc=kraxel@redhat.com \
    --cc=mst@redhat.com \
    --cc=peter.maydell@linaro.org \
    --cc=qemu-devel@nongnu.org \
    --cc=stefano.stabellini@eu.citrix.com \
    --cc=xen-devel@lists.xensource.com \
    --cc=yang.z.zhang@intel.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.