From: Suzuki K Poulose <suzuki.poulose@arm.com> To: Marc Zyngier <maz@kernel.org> Cc: kvmarm@lists.linux.dev, kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, Alexandru Elisei <alexandru.elisei@arm.com>, Andre Przywara <andre.przywara@arm.com>, Chase Conklin <chase.conklin@arm.com>, Christoffer Dall <christoffer.dall@arm.com>, Ganapatrao Kulkarni <gankulkarni@os.amperecomputing.com>, Jintack Lim <jintack@cs.columbia.edu>, Russell King <rmk+kernel@armlinux.org.uk>, James Morse <james.morse@arm.com>, Oliver Upton <oliver.upton@linux.dev>, Zenghui Yu <yuzenghui@huawei.com> Subject: Re: [PATCH v8 01/69] arm64: Add ARM64_HAS_NESTED_VIRT cpufeature Date: Tue, 31 Jan 2023 17:34:39 +0000 [thread overview] Message-ID: <3c15760c-c76f-3d5d-a661-442459ce4e07@arm.com> (raw) In-Reply-To: <86cz6u248j.wl-maz@kernel.org> Hi Marc, On 31/01/2023 14:00, Marc Zyngier wrote: > Hi Suzuki, > > On Tue, 31 Jan 2023 13:47:31 +0000, > Suzuki K Poulose <suzuki.poulose@arm.com> wrote: >> >> Hi Marc >> >> On 31/01/2023 09:23, Marc Zyngier wrote: >>> From: Jintack Lim <jintack.lim@linaro.org> >>> >>> Add a new ARM64_HAS_NESTED_VIRT feature to indicate that the >>> CPU has the ARMv8.3 nested virtualization capability, together >>> with the 'kvm-arm.mode=nested' command line option. >>> >>> This will be used to support nested virtualization in KVM. >>> >>> Reviewed-by: Russell King (Oracle) <rmk+kernel@armlinux.org.uk> >>> Signed-off-by: Jintack Lim <jintack.lim@linaro.org> >>> Signed-off-by: Andre Przywara <andre.przywara@arm.com> >>> Signed-off-by: Christoffer Dall <christoffer.dall@arm.com> >>> [maz: moved the command-line option to kvm-arm.mode] >> >> Should this be separate kvm-arm mode ? Or can this be tied to >> is_kernel_in_hyp_mode() ? Given this mode (from my limited >> review) doesn't conflict with normal VHE mode (and RME support), >> adding this explicit mode could confuse the user. > > What is exactly the objection here? NV is more or less a VHE++ mode, > but is also completely experimental and incomplete. I am all in for making this an "optional", only enabled it when "I know what I want". kvm-arm.mode=nv kind of seems that the KVM driver is conditioned mainly for running NV (comparing with the other existing options for kvm-arm.mode). In reality, as you confirmed, NV is an *additional* capability of a VHE hypervisor. So it would be good to "opt" in for "nv" capability support. e.g, kvm-arm.nv=on Thinking more about it, either is fine. > >> In case we need a command line to turn the NV mode on/off, >> we could always use the id-override and simply leave this out ? > > I really want an explicit user buy-in. There is absolutely no way this > can be enabled by default, the risks are way too high. Just look at > the x86 story: it took them 10 years to enable NV by default. I don't > expect to do any better. Of course, I am with you on that. I realise that we may not be able to disable nv by default using idreg-override (i.e., without any kernel command line option). So we may need something outside of that, like the option mentioned above. Suzuki > > Thanks, > > M. >
WARNING: multiple messages have this Message-ID (diff)
From: Suzuki K Poulose <suzuki.poulose@arm.com> To: Marc Zyngier <maz@kernel.org> Cc: kvmarm@lists.linux.dev, kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, Alexandru Elisei <alexandru.elisei@arm.com>, Andre Przywara <andre.przywara@arm.com>, Chase Conklin <chase.conklin@arm.com>, Christoffer Dall <christoffer.dall@arm.com>, Ganapatrao Kulkarni <gankulkarni@os.amperecomputing.com>, Jintack Lim <jintack@cs.columbia.edu>, Russell King <rmk+kernel@armlinux.org.uk>, James Morse <james.morse@arm.com>, Oliver Upton <oliver.upton@linux.dev>, Zenghui Yu <yuzenghui@huawei.com> Subject: Re: [PATCH v8 01/69] arm64: Add ARM64_HAS_NESTED_VIRT cpufeature Date: Tue, 31 Jan 2023 17:34:39 +0000 [thread overview] Message-ID: <3c15760c-c76f-3d5d-a661-442459ce4e07@arm.com> (raw) In-Reply-To: <86cz6u248j.wl-maz@kernel.org> Hi Marc, On 31/01/2023 14:00, Marc Zyngier wrote: > Hi Suzuki, > > On Tue, 31 Jan 2023 13:47:31 +0000, > Suzuki K Poulose <suzuki.poulose@arm.com> wrote: >> >> Hi Marc >> >> On 31/01/2023 09:23, Marc Zyngier wrote: >>> From: Jintack Lim <jintack.lim@linaro.org> >>> >>> Add a new ARM64_HAS_NESTED_VIRT feature to indicate that the >>> CPU has the ARMv8.3 nested virtualization capability, together >>> with the 'kvm-arm.mode=nested' command line option. >>> >>> This will be used to support nested virtualization in KVM. >>> >>> Reviewed-by: Russell King (Oracle) <rmk+kernel@armlinux.org.uk> >>> Signed-off-by: Jintack Lim <jintack.lim@linaro.org> >>> Signed-off-by: Andre Przywara <andre.przywara@arm.com> >>> Signed-off-by: Christoffer Dall <christoffer.dall@arm.com> >>> [maz: moved the command-line option to kvm-arm.mode] >> >> Should this be separate kvm-arm mode ? Or can this be tied to >> is_kernel_in_hyp_mode() ? Given this mode (from my limited >> review) doesn't conflict with normal VHE mode (and RME support), >> adding this explicit mode could confuse the user. > > What is exactly the objection here? NV is more or less a VHE++ mode, > but is also completely experimental and incomplete. I am all in for making this an "optional", only enabled it when "I know what I want". kvm-arm.mode=nv kind of seems that the KVM driver is conditioned mainly for running NV (comparing with the other existing options for kvm-arm.mode). In reality, as you confirmed, NV is an *additional* capability of a VHE hypervisor. So it would be good to "opt" in for "nv" capability support. e.g, kvm-arm.nv=on Thinking more about it, either is fine. > >> In case we need a command line to turn the NV mode on/off, >> we could always use the id-override and simply leave this out ? > > I really want an explicit user buy-in. There is absolutely no way this > can be enabled by default, the risks are way too high. Just look at > the x86 story: it took them 10 years to enable NV by default. I don't > expect to do any better. Of course, I am with you on that. I realise that we may not be able to disable nv by default using idreg-override (i.e., without any kernel command line option). So we may need something outside of that, like the option mentioned above. Suzuki > > Thanks, > > M. > _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
next prev parent reply other threads:[~2023-01-31 17:35 UTC|newest] Thread overview: 170+ messages / expand[flat|nested] mbox.gz Atom feed top 2023-01-31 9:23 [PATCH v8 00/69] KVM: arm64: ARMv8.3/8.4 Nested Virtualization support Marc Zyngier 2023-01-31 9:23 ` Marc Zyngier 2023-01-31 9:23 ` [PATCH v8 01/69] arm64: Add ARM64_HAS_NESTED_VIRT cpufeature Marc Zyngier 2023-01-31 9:23 ` Marc Zyngier 2023-01-31 12:03 ` Catalin Marinas 2023-01-31 12:03 ` Catalin Marinas 2023-01-31 13:21 ` Marc Zyngier 2023-01-31 13:21 ` Marc Zyngier 2023-01-31 13:47 ` Suzuki K Poulose 2023-01-31 13:47 ` Suzuki K Poulose 2023-01-31 14:00 ` Marc Zyngier 2023-01-31 14:00 ` Marc Zyngier 2023-01-31 17:34 ` Suzuki K Poulose [this message] 2023-01-31 17:34 ` Suzuki K Poulose 2023-01-31 20:04 ` Oliver Upton 2023-01-31 20:04 ` Oliver Upton 2023-01-31 21:26 ` Marc Zyngier 2023-01-31 21:26 ` Marc Zyngier 2023-01-31 22:01 ` Oliver Upton 2023-01-31 22:01 ` Oliver Upton 2023-01-31 20:44 ` Marc Zyngier 2023-01-31 20:44 ` Marc Zyngier 2023-01-31 9:23 ` [PATCH v8 02/69] KVM: arm64: Use the S2 MMU context to iterate over S2 table Marc Zyngier 2023-01-31 9:23 ` Marc Zyngier 2023-01-31 9:23 ` [PATCH v8 03/69] KVM: arm64: nv: Introduce nested virtualization VCPU feature Marc Zyngier 2023-01-31 9:23 ` Marc Zyngier 2023-01-31 9:23 ` [PATCH v8 04/69] KVM: arm64: nv: Reset VCPU to EL2 registers if VCPU nested virt is set Marc Zyngier 2023-01-31 9:23 ` Marc Zyngier 2023-01-31 9:24 ` [PATCH v8 05/69] KVM: arm64: nv: Allow userspace to set PSR_MODE_EL2x Marc Zyngier 2023-01-31 9:24 ` Marc Zyngier 2023-01-31 9:24 ` [PATCH v8 06/69] KVM: arm64: nv: Add EL2 system registers to vcpu context Marc Zyngier 2023-01-31 9:24 ` Marc Zyngier 2023-01-31 9:24 ` [PATCH v8 07/69] KVM: arm64: nv: Add nested virt VCPU primitives for vEL2 VCPU state Marc Zyngier 2023-01-31 9:24 ` Marc Zyngier 2023-01-31 9:24 ` [PATCH v8 08/69] KVM: arm64: nv: Handle HCR_EL2.NV system register traps Marc Zyngier 2023-01-31 9:24 ` Marc Zyngier 2023-01-31 9:24 ` [PATCH v8 09/69] KVM: arm64: nv: Reset VMPIDR_EL2 and VPIDR_EL2 to sane values Marc Zyngier 2023-01-31 9:24 ` Marc Zyngier 2023-01-31 20:17 ` Oliver Upton 2023-01-31 20:17 ` Oliver Upton 2023-01-31 22:04 ` Marc Zyngier 2023-01-31 22:04 ` Marc Zyngier 2023-01-31 22:09 ` Oliver Upton 2023-01-31 22:09 ` Oliver Upton 2023-01-31 22:16 ` Marc Zyngier 2023-01-31 22:16 ` Marc Zyngier 2023-01-31 9:24 ` [PATCH v8 10/69] KVM: arm64: nv: Support virtual EL2 exceptions Marc Zyngier 2023-01-31 9:24 ` Marc Zyngier 2023-01-31 9:24 ` [PATCH v8 11/69] KVM: arm64: nv: Inject HVC exceptions to the virtual EL2 Marc Zyngier 2023-01-31 9:24 ` Marc Zyngier 2023-01-31 9:24 ` [PATCH v8 12/69] KVM: arm64: nv: Handle trapped ERET from " Marc Zyngier 2023-01-31 9:24 ` Marc Zyngier 2023-01-31 9:24 ` [PATCH v8 13/69] KVM: arm64: nv: Add non-VHE-EL2->EL1 translation helpers Marc Zyngier 2023-01-31 9:24 ` Marc Zyngier 2023-01-31 9:24 ` [PATCH v8 14/69] KVM: arm64: nv: Handle virtual EL2 registers in vcpu_read/write_sys_reg() Marc Zyngier 2023-01-31 9:24 ` Marc Zyngier 2023-01-31 9:24 ` [PATCH v8 15/69] KVM: arm64: nv: Handle SPSR_EL2 specially Marc Zyngier 2023-01-31 9:24 ` Marc Zyngier 2023-01-31 9:24 ` [PATCH v8 16/69] KVM: arm64: nv: Handle HCR_EL2.E2H specially Marc Zyngier 2023-01-31 9:24 ` Marc Zyngier 2023-01-31 9:24 ` [PATCH v8 17/69] KVM: arm64: nv: Save/Restore vEL2 sysregs Marc Zyngier 2023-01-31 9:24 ` Marc Zyngier 2023-01-31 9:24 ` [PATCH v8 18/69] KVM: arm64: nv: Emulate PSTATE.M for a guest hypervisor Marc Zyngier 2023-01-31 9:24 ` Marc Zyngier 2023-01-31 9:24 ` [PATCH v8 19/69] KVM: arm64: nv: Trap EL1 VM register accesses in virtual EL2 Marc Zyngier 2023-01-31 9:24 ` Marc Zyngier 2023-01-31 9:24 ` [PATCH v8 20/69] KVM: arm64: nv: Add accessors for SPSR_EL1, ELR_EL1 and VBAR_EL1 from " Marc Zyngier 2023-01-31 9:24 ` Marc Zyngier 2023-01-31 9:24 ` [PATCH v8 21/69] KVM: arm64: nv: Trap CPACR_EL1 access in " Marc Zyngier 2023-01-31 9:24 ` Marc Zyngier 2023-01-31 9:24 ` [PATCH v8 22/69] KVM: arm64: nv: Handle PSCI call via smc from the guest Marc Zyngier 2023-01-31 9:24 ` Marc Zyngier 2023-01-31 9:24 ` [PATCH v8 23/69] KVM: arm64: nv: Respect virtual HCR_EL2.TWX setting Marc Zyngier 2023-01-31 9:24 ` Marc Zyngier 2023-01-31 9:24 ` [PATCH v8 24/69] KVM: arm64: nv: Respect virtual CPTR_EL2.{TFP,FPEN} settings Marc Zyngier 2023-01-31 9:24 ` Marc Zyngier 2023-01-31 9:24 ` [PATCH v8 25/69] KVM: arm64: nv: Respect the virtual HCR_EL2.NV bit setting Marc Zyngier 2023-01-31 9:24 ` Marc Zyngier 2023-01-31 9:24 ` [PATCH v8 26/69] KVM: arm64: nv: Respect virtual HCR_EL2.TVM and TRVM settings Marc Zyngier 2023-01-31 9:24 ` Marc Zyngier 2023-01-31 9:24 ` [PATCH v8 27/69] KVM: arm64: nv: Respect the virtual HCR_EL2.NV1 bit setting Marc Zyngier 2023-01-31 9:24 ` Marc Zyngier 2023-01-31 9:24 ` [PATCH v8 28/69] KVM: arm64: nv: Allow a sysreg to be hidden from userspace only Marc Zyngier 2023-01-31 9:24 ` Marc Zyngier 2023-01-31 9:24 ` [PATCH v8 29/69] KVM: arm64: nv: Emulate EL12 register accesses from the virtual EL2 Marc Zyngier 2023-01-31 9:24 ` Marc Zyngier 2023-01-31 9:24 ` [PATCH v8 30/69] KVM: arm64: nv: Forward debug traps to the nested guest Marc Zyngier 2023-01-31 9:24 ` Marc Zyngier 2023-01-31 9:24 ` [PATCH v8 31/69] KVM: arm64: nv: Configure HCR_EL2 for nested virtualization Marc Zyngier 2023-01-31 9:24 ` Marc Zyngier 2023-01-31 9:24 ` [PATCH v8 32/69] KVM: arm64: nv: Only toggle cache for virtual EL2 when SCTLR_EL2 changes Marc Zyngier 2023-01-31 9:24 ` Marc Zyngier 2023-01-31 9:24 ` [PATCH v8 33/69] KVM: arm64: nv: Filter out unsupported features from ID regs Marc Zyngier 2023-01-31 9:24 ` Marc Zyngier 2023-01-31 9:24 ` [PATCH v8 34/69] KVM: arm64: nv: Hide RAS from nested guests Marc Zyngier 2023-01-31 9:24 ` Marc Zyngier 2023-01-31 9:24 ` [PATCH v8 35/69] KVM: arm64: nv: Support multiple nested Stage-2 mmu structures Marc Zyngier 2023-01-31 9:24 ` Marc Zyngier 2023-01-31 9:24 ` [PATCH v8 36/69] KVM: arm64: nv: Implement nested Stage-2 page table walk logic Marc Zyngier 2023-01-31 9:24 ` Marc Zyngier 2023-01-31 9:24 ` [PATCH v8 37/69] KVM: arm64: nv: Handle shadow stage 2 page faults Marc Zyngier 2023-01-31 9:24 ` Marc Zyngier 2023-01-31 9:24 ` [PATCH v8 38/69] KVM: arm64: nv: Restrict S2 RD/WR permissions to match the guest's Marc Zyngier 2023-01-31 9:24 ` Marc Zyngier 2023-01-31 9:24 ` [PATCH v8 39/69] KVM: arm64: nv: Unmap/flush shadow stage 2 page tables Marc Zyngier 2023-01-31 9:24 ` Marc Zyngier 2023-01-31 9:24 ` [PATCH v8 40/69] KVM: arm64: nv: Set a handler for the system instruction traps Marc Zyngier 2023-01-31 9:24 ` Marc Zyngier 2023-01-31 9:24 ` [PATCH v8 41/69] KVM: arm64: nv: Trap and emulate AT instructions from virtual EL2 Marc Zyngier 2023-01-31 9:24 ` Marc Zyngier 2023-01-31 9:24 ` [PATCH v8 42/69] KVM: arm64: nv: Trap and emulate TLBI " Marc Zyngier 2023-01-31 9:24 ` Marc Zyngier 2023-01-31 9:24 ` [PATCH v8 43/69] KVM: arm64: nv: Fold guest's HCR_EL2 configuration into the host's Marc Zyngier 2023-01-31 9:24 ` Marc Zyngier 2023-01-31 9:24 ` [PATCH v8 44/69] KVM: arm64: nv: arch_timer: Support hyp timer emulation Marc Zyngier 2023-01-31 9:24 ` Marc Zyngier 2023-01-31 9:24 ` [PATCH v8 45/69] KVM: arm64: nv: Add handling of EL2-specific timer registers Marc Zyngier 2023-01-31 9:24 ` Marc Zyngier 2023-01-31 9:24 ` [PATCH v8 46/69] KVM: arm64: nv: Load timer before the GIC Marc Zyngier 2023-01-31 9:24 ` Marc Zyngier 2023-01-31 9:24 ` [PATCH v8 47/69] KVM: arm64: nv: Nested GICv3 Support Marc Zyngier 2023-01-31 9:24 ` Marc Zyngier 2023-01-31 9:24 ` [PATCH v8 48/69] KVM: arm64: nv: Don't load the GICv4 context on entering a nested guest Marc Zyngier 2023-01-31 9:24 ` Marc Zyngier 2023-01-31 9:24 ` [PATCH v8 49/69] KVM: arm64: nv: vgic: Emulate the HW bit in software Marc Zyngier 2023-01-31 9:24 ` Marc Zyngier 2023-01-31 9:24 ` [PATCH v8 50/69] KVM: arm64: nv: vgic: Allow userland to set VGIC maintenance IRQ Marc Zyngier 2023-01-31 9:24 ` Marc Zyngier 2023-01-31 9:24 ` [PATCH v8 51/69] KVM: arm64: nv: Implement maintenance interrupt forwarding Marc Zyngier 2023-01-31 9:24 ` Marc Zyngier 2023-01-31 9:24 ` [PATCH v8 52/69] KVM: arm64: nv: Deal with broken VGIC on maintenance interrupt delivery Marc Zyngier 2023-01-31 9:24 ` Marc Zyngier 2023-01-31 9:24 ` [PATCH v8 53/69] KVM: arm64: nv: Add nested GICv3 tracepoints Marc Zyngier 2023-01-31 9:24 ` Marc Zyngier 2023-01-31 9:24 ` [PATCH v8 54/69] KVM: arm64: nv: Allow userspace to request KVM_ARM_VCPU_NESTED_VIRT Marc Zyngier 2023-01-31 9:24 ` Marc Zyngier 2023-01-31 9:24 ` [PATCH v8 55/69] KVM: arm64: nv: Add handling of FEAT_TTL TLB invalidation Marc Zyngier 2023-01-31 9:24 ` Marc Zyngier 2023-01-31 9:24 ` [PATCH v8 56/69] KVM: arm64: nv: Invalidate TLBs based on shadow S2 TTL-like information Marc Zyngier 2023-01-31 9:24 ` Marc Zyngier 2023-01-31 9:24 ` [PATCH v8 57/69] KVM: arm64: nv: Tag shadow S2 entries with nested level Marc Zyngier 2023-01-31 9:24 ` Marc Zyngier 2023-01-31 9:24 ` [PATCH v8 58/69] KVM: arm64: nv: Add include containing the VNCR_EL2 offsets Marc Zyngier 2023-01-31 9:24 ` Marc Zyngier 2023-01-31 9:24 ` [PATCH v8 59/69] KVM: arm64: nv: Map VNCR-capable registers to a separate page Marc Zyngier 2023-01-31 9:24 ` Marc Zyngier 2023-01-31 9:24 ` [PATCH v8 60/69] KVM: arm64: nv: Move nested vgic state into the sysreg file Marc Zyngier 2023-01-31 9:24 ` Marc Zyngier 2023-01-31 9:24 ` [PATCH v8 61/69] KVM: arm64: Add FEAT_NV2 cpu feature Marc Zyngier 2023-01-31 9:24 ` Marc Zyngier 2023-01-31 9:24 ` [PATCH v8 62/69] KVM: arm64: nv: Sync nested timer state with FEAT_NV2 Marc Zyngier 2023-01-31 9:24 ` Marc Zyngier 2023-01-31 9:24 ` [PATCH v8 63/69] KVM: arm64: nv: Publish emulated timer interrupt state in the in-memory state Marc Zyngier 2023-01-31 9:24 ` Marc Zyngier 2023-01-31 9:24 ` [PATCH v8 64/69] KVM: arm64: nv: Allocate VNCR page when required Marc Zyngier 2023-01-31 9:24 ` Marc Zyngier 2023-01-31 9:25 ` [PATCH v8 65/69] KVM: arm64: nv: Enable ARMv8.4-NV support Marc Zyngier 2023-01-31 9:25 ` Marc Zyngier 2023-01-31 9:25 ` [PATCH v8 66/69] KVM: arm64: nv: Fast-track 'InHost' exception returns Marc Zyngier 2023-01-31 9:25 ` Marc Zyngier 2023-01-31 9:25 ` [PATCH v8 67/69] KVM: arm64: nv: Fast-track EL1 TLBIs for VHE guests Marc Zyngier 2023-01-31 9:25 ` Marc Zyngier 2023-01-31 9:25 ` [PATCH v8 68/69] KVM: arm64: nv: Use FEAT_ECV to trap access to EL0 timers Marc Zyngier 2023-01-31 9:25 ` Marc Zyngier 2023-01-31 9:25 ` [PATCH v8 69/69] KVM: arm64: nv: Accelerate EL0 timer read accesses when FEAT_ECV is on Marc Zyngier 2023-01-31 9:25 ` Marc Zyngier 2023-01-31 22:13 ` [PATCH v8 00/69] KVM: arm64: ARMv8.3/8.4 Nested Virtualization support Oliver Upton 2023-01-31 22:13 ` Oliver Upton 2023-01-31 22:20 ` Marc Zyngier 2023-01-31 22:20 ` Marc Zyngier
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=3c15760c-c76f-3d5d-a661-442459ce4e07@arm.com \ --to=suzuki.poulose@arm.com \ --cc=alexandru.elisei@arm.com \ --cc=andre.przywara@arm.com \ --cc=chase.conklin@arm.com \ --cc=christoffer.dall@arm.com \ --cc=gankulkarni@os.amperecomputing.com \ --cc=james.morse@arm.com \ --cc=jintack@cs.columbia.edu \ --cc=kvm@vger.kernel.org \ --cc=kvmarm@lists.linux.dev \ --cc=linux-arm-kernel@lists.infradead.org \ --cc=maz@kernel.org \ --cc=oliver.upton@linux.dev \ --cc=rmk+kernel@armlinux.org.uk \ --cc=yuzenghui@huawei.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.