From: Marek Szyprowski <m.szyprowski-Sze3O3UU22JBDgjK7y7TUQ@public.gmane.org> To: Sricharan R <sricharan-sgV2jX0FEOL9JmXXK+q4OQ@public.gmane.org>, will.deacon-5wv7dgnIgG8@public.gmane.org, robin.murphy-5wv7dgnIgG8@public.gmane.org, joro-zLv9SwRftAIdnm+yROfE0A@public.gmane.org, iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org, linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org, linux-arm-msm-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, laurent.pinchart-ryLnwIuWjnjg/C1BVhZhaw@public.gmane.org, tfiga-F7+t8E8rja9g9hUCZPvPmw@public.gmane.org, srinivas.kandagatla-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org Subject: Re: [PATCH V3 0/8] IOMMU probe deferral support Date: Mon, 10 Oct 2016 14:36:41 +0200 [thread overview] Message-ID: <12cfb59f-f7ca-d4df-eb7f-42348e357979@samsung.com> (raw) In-Reply-To: <1475600632-21289-1-git-send-email-sricharan-sgV2jX0FEOL9JmXXK+q4OQ@public.gmane.org> Hi Sricharan, On 2016-10-04 19:03, Sricharan R wrote: > Initial post from Laurent Pinchart[1]. This is > series calls the dma ops configuration for the devices > at a generic place so that it works for all busses. > The dma_configure_ops for a device is now called during > the device_attach callback just before the probe of the > bus/driver is called. Similarly dma_deconfigure is called during > device/driver_detach path. > > > pci_bus_add_devices (platform/amba)(_device_create/driver_register) > | | > pci_bus_add_device (device_add/driver_register) > | | > device_attach device_initial_probe > | | > __device_attach_driver __device_attach_driver > | > driver_probe_device > | > really_probe > | > dma_configure > > Similarly on the device/driver_unregister path __device_release_driver is > called which inturn calls dma_deconfigure. > > If the ACPI bus code follows the same, we can add acpi_dma_configure > at the same place as of_dma_configure. > > This series is based on the recently merged Generic DT bindings for > PCI IOMMUs and ARM SMMU from Robin Murphy robin.murphy-5wv7dgnIgG8@public.gmane.org [2] > > This time tested this with platform and pci device for probe deferral > and reprobe on arm64 based platform. There is an issue on the cleanup > path for arm64 though, where there is WARN_ON if the dma_ops is reset while > device is attached to an domain in arch_teardown_dma_ops. > But with iommu_groups created from the iommu driver, the device is always > attached to a domain/default_domain. So so the WARN has to be removed/handled > probably. Thanks for continuing work on this feature! Your can add my: Tested-by: Marek Szyprowski <m.szyprowski-Sze3O3UU22JBDgjK7y7TUQ@public.gmane.org> It works fine with Exynos SYSMMU driver, although a patch is needed to fix infinite loop due to list corruption (same element is added twice if master device fails with deferred probe): From: Marek Szyprowski <m.szyprowski-Sze3O3UU22JBDgjK7y7TUQ@public.gmane.org> Date: Mon, 10 Oct 2016 14:22:42 +0200 Subject: [PATCH] iommu/exynos: ensure that sysmmu is added only once to its master Since adding IOMMU deferred probing support, of_xlate() callback might be called more than once for given master device (for example it happens when masters device driver fails with EPROBE_DEFER), so ensure that SYSMMU controller is added to its master device (owner) only once. Signed-off-by: Marek Szyprowski <m.szyprowski-Sze3O3UU22JBDgjK7y7TUQ@public.gmane.org> --- drivers/iommu/exynos-iommu.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/drivers/iommu/exynos-iommu.c b/drivers/iommu/exynos-iommu.c index 30808e91b775..1525a86eb829 100644 --- a/drivers/iommu/exynos-iommu.c +++ b/drivers/iommu/exynos-iommu.c @@ -1253,7 +1253,7 @@ static int exynos_iommu_of_xlate(struct device *dev, { struct exynos_iommu_owner *owner = dev->archdata.iommu; struct platform_device *sysmmu = of_find_device_by_node(spec->np); - struct sysmmu_drvdata *data; + struct sysmmu_drvdata *data, *entry; if (!sysmmu) return -ENODEV; @@ -1271,6 +1271,10 @@ static int exynos_iommu_of_xlate(struct device *dev, dev->archdata.iommu = owner; } + list_for_each_entry(entry, &owner->controllers, owner_node) + if (entry == data) + return 0; + list_add_tail(&data->owner_node, &owner->controllers); return 0; } -- 1.9.1 > > Previous post of this series [3]. > > [V3] > * Removed the patch to split dma_masks/dma_ops configuration separately > based on review comments that both masks and ops are required only > during the device probe time. > > * Reworked the series based on Generic DT bindings series [2]. > > * Added call to iommu's remove_device in the cleanup path for arm and arm64. > > * Removed the notifier trick in arm64 to handle early device registration. > > * Added reset of dma_ops in cleanup path for arm based on comments. > > * Fixed the pci_iommu_configure path and tested with PCI device as well. > > * Fixed a bug to return the correct iommu_ops from patch 7 [4] in last post. > > * Fixed few other cosmetic comments. > > [V2] > * Updated the Initial post to call dma_configure/deconfigure from generic code > > * Added iommu add_device callback from of_iommu_configure path > > [V1] > * Initial post > > [1] http://lists.linuxfoundation.org/pipermail/iommu/2015-May/013016.html > [2] http://www.spinics.net/lists/devicetree/msg142943.html > [3] https://www.mail-archive.com/iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org/msg13941.html > [4] https://www.mail-archive.com/iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org/msg13940.html > > > > Laurent Pinchart (4): > arm: dma-mapping: Don't override dma_ops in arch_setup_dma_ops() > of: dma: Move range size workaround to of_dma_get_range() > of: dma: Make of_dma_deconfigure() public > iommu: of: Handle IOMMU lookup failure with deferred probing or error > > Sricharan R (4): > drivers: platform: Configure dma operations at probe time > arm: dma-mapping: Reset the device's dma_ops > arm/arm64: dma-mapping: Call iommu's remove_device callback during > device detach > arm64: dma-mapping: Remove the notifier trick to handle early setting > of dma_ops > > arch/arm/mm/dma-mapping.c | 18 ++++++++ > arch/arm64/mm/dma-mapping.c | 107 +++++--------------------------------------- > drivers/base/dd.c | 10 +++++ > drivers/base/dma-mapping.c | 11 +++++ > drivers/iommu/of_iommu.c | 47 +++++++++++++++++-- > drivers/of/address.c | 20 ++++++++- > drivers/of/device.c | 34 +++++++------- > drivers/of/platform.c | 9 ---- > drivers/pci/probe.c | 5 +-- > include/linux/dma-mapping.h | 3 ++ > include/linux/of_device.h | 7 ++- > 11 files changed, 138 insertions(+), 133 deletions(-) > Best regards -- Marek Szyprowski, PhD Samsung R&D Institute Poland
WARNING: multiple messages have this Message-ID (diff)
From: m.szyprowski@samsung.com (Marek Szyprowski) To: linux-arm-kernel@lists.infradead.org Subject: [PATCH V3 0/8] IOMMU probe deferral support Date: Mon, 10 Oct 2016 14:36:41 +0200 [thread overview] Message-ID: <12cfb59f-f7ca-d4df-eb7f-42348e357979@samsung.com> (raw) In-Reply-To: <1475600632-21289-1-git-send-email-sricharan@codeaurora.org> Hi Sricharan, On 2016-10-04 19:03, Sricharan R wrote: > Initial post from Laurent Pinchart[1]. This is > series calls the dma ops configuration for the devices > at a generic place so that it works for all busses. > The dma_configure_ops for a device is now called during > the device_attach callback just before the probe of the > bus/driver is called. Similarly dma_deconfigure is called during > device/driver_detach path. > > > pci_bus_add_devices (platform/amba)(_device_create/driver_register) > | | > pci_bus_add_device (device_add/driver_register) > | | > device_attach device_initial_probe > | | > __device_attach_driver __device_attach_driver > | > driver_probe_device > | > really_probe > | > dma_configure > > Similarly on the device/driver_unregister path __device_release_driver is > called which inturn calls dma_deconfigure. > > If the ACPI bus code follows the same, we can add acpi_dma_configure > at the same place as of_dma_configure. > > This series is based on the recently merged Generic DT bindings for > PCI IOMMUs and ARM SMMU from Robin Murphy robin.murphy at arm.com [2] > > This time tested this with platform and pci device for probe deferral > and reprobe on arm64 based platform. There is an issue on the cleanup > path for arm64 though, where there is WARN_ON if the dma_ops is reset while > device is attached to an domain in arch_teardown_dma_ops. > But with iommu_groups created from the iommu driver, the device is always > attached to a domain/default_domain. So so the WARN has to be removed/handled > probably. Thanks for continuing work on this feature! Your can add my: Tested-by: Marek Szyprowski <m.szyprowski@samsung.com> It works fine with Exynos SYSMMU driver, although a patch is needed to fix infinite loop due to list corruption (same element is added twice if master device fails with deferred probe): From: Marek Szyprowski <m.szyprowski@samsung.com> Date: Mon, 10 Oct 2016 14:22:42 +0200 Subject: [PATCH] iommu/exynos: ensure that sysmmu is added only once to its master Since adding IOMMU deferred probing support, of_xlate() callback might be called more than once for given master device (for example it happens when masters device driver fails with EPROBE_DEFER), so ensure that SYSMMU controller is added to its master device (owner) only once. Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com> --- drivers/iommu/exynos-iommu.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/drivers/iommu/exynos-iommu.c b/drivers/iommu/exynos-iommu.c index 30808e91b775..1525a86eb829 100644 --- a/drivers/iommu/exynos-iommu.c +++ b/drivers/iommu/exynos-iommu.c @@ -1253,7 +1253,7 @@ static int exynos_iommu_of_xlate(struct device *dev, { struct exynos_iommu_owner *owner = dev->archdata.iommu; struct platform_device *sysmmu = of_find_device_by_node(spec->np); - struct sysmmu_drvdata *data; + struct sysmmu_drvdata *data, *entry; if (!sysmmu) return -ENODEV; @@ -1271,6 +1271,10 @@ static int exynos_iommu_of_xlate(struct device *dev, dev->archdata.iommu = owner; } + list_for_each_entry(entry, &owner->controllers, owner_node) + if (entry == data) + return 0; + list_add_tail(&data->owner_node, &owner->controllers); return 0; } -- 1.9.1 > > Previous post of this series [3]. > > [V3] > * Removed the patch to split dma_masks/dma_ops configuration separately > based on review comments that both masks and ops are required only > during the device probe time. > > * Reworked the series based on Generic DT bindings series [2]. > > * Added call to iommu's remove_device in the cleanup path for arm and arm64. > > * Removed the notifier trick in arm64 to handle early device registration. > > * Added reset of dma_ops in cleanup path for arm based on comments. > > * Fixed the pci_iommu_configure path and tested with PCI device as well. > > * Fixed a bug to return the correct iommu_ops from patch 7 [4] in last post. > > * Fixed few other cosmetic comments. > > [V2] > * Updated the Initial post to call dma_configure/deconfigure from generic code > > * Added iommu add_device callback from of_iommu_configure path > > [V1] > * Initial post > > [1] http://lists.linuxfoundation.org/pipermail/iommu/2015-May/013016.html > [2] http://www.spinics.net/lists/devicetree/msg142943.html > [3] https://www.mail-archive.com/iommu at lists.linux-foundation.org/msg13941.html > [4] https://www.mail-archive.com/iommu at lists.linux-foundation.org/msg13940.html > > > > Laurent Pinchart (4): > arm: dma-mapping: Don't override dma_ops in arch_setup_dma_ops() > of: dma: Move range size workaround to of_dma_get_range() > of: dma: Make of_dma_deconfigure() public > iommu: of: Handle IOMMU lookup failure with deferred probing or error > > Sricharan R (4): > drivers: platform: Configure dma operations at probe time > arm: dma-mapping: Reset the device's dma_ops > arm/arm64: dma-mapping: Call iommu's remove_device callback during > device detach > arm64: dma-mapping: Remove the notifier trick to handle early setting > of dma_ops > > arch/arm/mm/dma-mapping.c | 18 ++++++++ > arch/arm64/mm/dma-mapping.c | 107 +++++--------------------------------------- > drivers/base/dd.c | 10 +++++ > drivers/base/dma-mapping.c | 11 +++++ > drivers/iommu/of_iommu.c | 47 +++++++++++++++++-- > drivers/of/address.c | 20 ++++++++- > drivers/of/device.c | 34 +++++++------- > drivers/of/platform.c | 9 ---- > drivers/pci/probe.c | 5 +-- > include/linux/dma-mapping.h | 3 ++ > include/linux/of_device.h | 7 ++- > 11 files changed, 138 insertions(+), 133 deletions(-) > Best regards -- Marek Szyprowski, PhD Samsung R&D Institute Poland
next prev parent reply other threads:[~2016-10-10 12:36 UTC|newest] Thread overview: 118+ messages / expand[flat|nested] mbox.gz Atom feed top [not found] <CGME20161004170414eucas1p141bebe16e1bf241862833e7ad0270c72@eucas1p1.samsung.com> 2016-10-04 17:03 ` [PATCH V3 0/8] IOMMU probe deferral support Sricharan R 2016-10-04 17:03 ` Sricharan R [not found] ` <1475600632-21289-1-git-send-email-sricharan-sgV2jX0FEOL9JmXXK+q4OQ@public.gmane.org> 2016-10-04 17:03 ` [PATCH V3 1/8] arm: dma-mapping: Don't override dma_ops in arch_setup_dma_ops() Sricharan R 2016-10-04 17:03 ` Sricharan R 2016-10-04 17:03 ` [PATCH V3 4/8] drivers: platform: Configure dma operations at probe time Sricharan R 2016-10-04 17:03 ` Sricharan R 2016-10-26 14:07 ` Robin Murphy 2016-10-26 14:07 ` Robin Murphy 2016-10-26 15:04 ` Sricharan 2016-10-26 15:04 ` Sricharan 2016-10-27 10:49 ` Lorenzo Pieralisi 2016-10-27 10:49 ` Lorenzo Pieralisi 2016-11-02 7:05 ` Sricharan 2016-11-02 7:05 ` Sricharan 2016-10-04 17:03 ` [PATCH V3 6/8] arm: dma-mapping: Reset the device's dma_ops Sricharan R 2016-10-04 17:03 ` Sricharan R 2016-10-26 15:07 ` Robin Murphy 2016-10-26 15:07 ` Robin Murphy [not found] ` <a3d4533f-165d-f444-7681-141479617a18-5wv7dgnIgG8@public.gmane.org> 2016-10-27 3:37 ` Sricharan 2016-10-27 3:37 ` Sricharan 2017-05-23 16:25 ` Russell King - ARM Linux 2017-05-23 16:25 ` Russell King - ARM Linux [not found] ` <20170523162507.GA1729-l+eeeJia6m9URfEZ8mYm6t73F7V6hmMc@public.gmane.org> 2017-05-23 16:55 ` Robin Murphy 2017-05-23 16:55 ` Robin Murphy 2017-05-23 17:53 ` Russell King - ARM Linux 2017-05-23 17:53 ` Russell King - ARM Linux [not found] ` <20170523175319.GA22219-l+eeeJia6m9URfEZ8mYm6t73F7V6hmMc@public.gmane.org> 2017-05-23 21:46 ` Laurent Pinchart 2017-05-23 21:46 ` Laurent Pinchart 2017-05-23 22:42 ` Russell King - ARM Linux 2017-05-23 22:42 ` Russell King - ARM Linux [not found] ` <20170523224216.GI22219-l+eeeJia6m9URfEZ8mYm6t73F7V6hmMc@public.gmane.org> 2017-05-24 10:31 ` Sricharan R 2017-05-24 10:31 ` Sricharan R [not found] ` <c4ad7341-fa9f-81b7-a41c-417144c4f842-sgV2jX0FEOL9JmXXK+q4OQ@public.gmane.org> 2017-05-24 11:26 ` Laurent Pinchart 2017-05-24 11:26 ` Laurent Pinchart 2017-05-24 11:38 ` Sricharan R 2017-05-24 11:38 ` Sricharan R 2017-05-25 15:05 ` Russell King - ARM Linux 2017-05-25 15:05 ` Russell King - ARM Linux [not found] ` <20170525150540.GJ22219-l+eeeJia6m9URfEZ8mYm6t73F7V6hmMc@public.gmane.org> 2017-05-26 5:18 ` Sricharan R 2017-05-26 5:18 ` Sricharan R 2017-05-26 14:04 ` Laurent Pinchart 2017-05-26 14:04 ` Laurent Pinchart 2016-10-10 12:36 ` Marek Szyprowski [this message] 2016-10-10 12:36 ` [PATCH V3 0/8] IOMMU probe deferral support Marek Szyprowski 2016-10-17 6:58 ` Sricharan 2016-10-17 6:58 ` Sricharan [not found] ` <12cfb59f-f7ca-d4df-eb7f-42348e357979-Sze3O3UU22JBDgjK7y7TUQ@public.gmane.org> 2016-10-12 6:24 ` Sricharan 2016-10-12 6:24 ` Sricharan 2016-10-24 6:34 ` Marek Szyprowski 2016-10-24 6:34 ` Marek Szyprowski [not found] ` <b9e4e81f-3b3e-951f-df62-d640275aae71-Sze3O3UU22JBDgjK7y7TUQ@public.gmane.org> 2016-10-24 12:30 ` Sricharan 2016-10-24 12:30 ` Sricharan 2016-10-17 7:02 ` Sricharan 2016-10-17 7:02 ` Sricharan 2016-10-25 6:25 ` Archit Taneja 2016-10-25 6:25 ` Archit Taneja 2016-10-04 17:03 ` [PATCH V3 2/8] of: dma: Move range size workaround to of_dma_get_range() Sricharan R 2016-10-04 17:03 ` Sricharan R 2016-10-04 17:03 ` [PATCH V3 3/8] of: dma: Make of_dma_deconfigure() public Sricharan R 2016-10-04 17:03 ` Sricharan R 2016-10-04 17:03 ` [PATCH V3 5/8] iommu: of: Handle IOMMU lookup failure with deferred probing or error Sricharan R 2016-10-04 17:03 ` Sricharan R [not found] ` <1475600632-21289-6-git-send-email-sricharan-sgV2jX0FEOL9JmXXK+q4OQ@public.gmane.org> 2016-10-26 14:52 ` Robin Murphy 2016-10-26 14:52 ` Robin Murphy [not found] ` <f08e65b4-f755-897c-f776-40f0d6788251-5wv7dgnIgG8@public.gmane.org> 2016-10-27 2:55 ` Sricharan 2016-10-27 2:55 ` Sricharan 2016-10-04 17:03 ` [PATCH V3 7/8] arm/arm64: dma-mapping: Call iommu's remove_device callback during device detach Sricharan R 2016-10-04 17:03 ` Sricharan R [not found] ` <1475600632-21289-8-git-send-email-sricharan-sgV2jX0FEOL9JmXXK+q4OQ@public.gmane.org> 2016-10-26 15:16 ` Robin Murphy 2016-10-26 15:16 ` Robin Murphy 2016-10-27 5:16 ` Sricharan 2016-10-27 5:16 ` Sricharan 2016-10-04 17:03 ` [PATCH V3 8/8] arm64: dma-mapping: Remove the notifier trick to handle early setting of dma_ops Sricharan R 2016-10-04 17:03 ` Sricharan R [not found] ` <1475600632-21289-9-git-send-email-sricharan-sgV2jX0FEOL9JmXXK+q4OQ@public.gmane.org> 2016-10-07 15:40 ` Sricharan 2016-10-07 15:40 ` Sricharan 2016-10-26 15:34 ` Robin Murphy 2016-10-26 15:34 ` Robin Murphy 2016-10-27 5:19 ` Sricharan 2016-10-27 5:19 ` Sricharan 2016-10-25 14:35 ` [PATCH V3 0/8] IOMMU probe deferral support Robin Murphy 2016-10-25 14:35 ` Robin Murphy [not found] ` <60ee8066-f167-e9df-ae3e-4138f1133bad-5wv7dgnIgG8@public.gmane.org> 2016-10-26 14:44 ` Sricharan 2016-10-26 14:44 ` Sricharan 2016-10-26 17:14 ` Robin Murphy 2016-10-26 17:14 ` Robin Murphy [not found] ` <421e2b14-0231-d376-02a0-097423120b3d-5wv7dgnIgG8@public.gmane.org> 2016-10-27 8:37 ` Sricharan 2016-10-27 8:37 ` Sricharan 2016-11-03 22:25 ` Sricharan 2016-11-03 22:25 ` Sricharan 2016-11-04 15:16 ` Sricharan 2016-11-04 15:16 ` Sricharan 2016-11-07 19:13 ` Will Deacon 2016-11-07 19:13 ` Will Deacon 2016-11-07 19:22 ` Robin Murphy 2016-11-07 19:22 ` Robin Murphy 2016-11-09 6:24 ` Sricharan 2016-11-09 6:24 ` Sricharan 2016-11-09 16:59 ` Will Deacon 2016-11-09 16:59 ` Will Deacon 2016-11-14 3:41 ` Sricharan 2016-11-14 3:41 ` Sricharan 2016-11-20 15:11 ` Sricharan 2016-11-20 15:11 ` Sricharan 2016-11-23 19:54 ` Robin Murphy 2016-11-23 19:54 ` Robin Murphy [not found] ` <918128b9-cdb0-1454-000a-146cee7a05ea-5wv7dgnIgG8@public.gmane.org> 2016-11-24 16:10 ` Sricharan 2016-11-24 16:10 ` Sricharan 2016-11-24 19:11 ` Robin Murphy 2016-11-24 19:11 ` Robin Murphy 2016-11-28 17:42 ` Sricharan 2016-11-28 17:42 ` Sricharan 2016-11-28 18:13 ` Lorenzo Pieralisi 2016-11-28 18:13 ` Lorenzo Pieralisi 2016-11-30 0:34 ` Sricharan 2016-11-30 0:34 ` Sricharan 2016-11-30 12:07 ` Lorenzo Pieralisi 2016-11-30 12:07 ` Lorenzo Pieralisi
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=12cfb59f-f7ca-d4df-eb7f-42348e357979@samsung.com \ --to=m.szyprowski-sze3o3uu22jbdgjk7y7tuq@public.gmane.org \ --cc=iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org \ --cc=joro-zLv9SwRftAIdnm+yROfE0A@public.gmane.org \ --cc=laurent.pinchart-ryLnwIuWjnjg/C1BVhZhaw@public.gmane.org \ --cc=linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org \ --cc=linux-arm-msm-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \ --cc=robin.murphy-5wv7dgnIgG8@public.gmane.org \ --cc=sricharan-sgV2jX0FEOL9JmXXK+q4OQ@public.gmane.org \ --cc=srinivas.kandagatla-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org \ --cc=tfiga-F7+t8E8rja9g9hUCZPvPmw@public.gmane.org \ --cc=will.deacon-5wv7dgnIgG8@public.gmane.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.