All of lore.kernel.org
 help / color / mirror / Atom feed
From: Kees Cook <keescook@chromium.org>
To: "Theodore Ts'o" <tytso@mit.edu>,
	Kees Cook <keescook@chromium.org>,
	Linus Torvalds <torvalds@linux-foundation.org>,
	Djalal Harouni <tixxdz@gmail.com>,
	Jonathan Corbet <corbet@lwn.net>,
	James Morris <james.l.morris@oracle.com>,
	LSM List <linux-security-module@vger.kernel.org>,
	Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
	"kernel-hardening@lists.openwall.com" 
	<kernel-hardening@lists.openwall.com>,
	Geo Kozey <geokozey@mailfence.com>,
	Tycho Andersen <tycho@docker.com>
Subject: Re: [kernel-hardening] Re: [PATCH v5 next 5/5] net: modules: use request_module_cap() to load 'netdev-%s' modules
Date: Tue, 28 Nov 2017 16:18:59 -0800	[thread overview]
Message-ID: <CAGXu5j+kXxkctzH79YxOu=45xgamV-5qyR2zL7OiC4POL6EcyA@mail.gmail.com> (raw)
In-Reply-To: <20171128234920.awfwicihuudw5ogx@thunk.org>

On Tue, Nov 28, 2017 at 3:49 PM, Theodore Ts'o <tytso@mit.edu> wrote:
> OK, but if the goal is to protect users who are running distro
> kernels, then a kernel config that breaks some percentage of users is
> ****highly**** unlikely to be enabled by Red Hat and SuSE, right?  And
> many of these users either can't (from a skills perspective) or won't
> (because they lose all support from the enterprise distro's help desk)
> recompile their kernel to enable an "might break 3% of all users ---
> oh, well" config option.

There's a spectrum across "insane not to be done everywhere"
(STRICT_KERNEL_RWX), "this is a good idea for nearly all Linux
systems" (link restrictions), "this can break some common use-cases,
but protects systems without that use-case" (user-namespace
disabling), "this breaks most systems, but specialized deployments
really need it" (modules_disabled).

There's also a difference between immutable CONFIG options that cannot
be disabled at runtime, those that can, global sysctls, per-namespace
controls, etc etc. The kernel is all about providing admins with knobs
to tweak their performance and security. Suddenly being told that we
can't create optional improvements is very odd.

Now, being told "make it easier to audit all the module loading we're
already doing so we can further reduce needless exposures for everyone
even without this feature enabled", THAT makes sense. My point there
is that we've already done those kinds of things; see commits like:

  7f78e0351394 ("fs: Limit sys_mount to only request filesystem modules.")
  5d26a105b5a7 ("crypto: prefix module autoloading with "crypto-"")
  4943ba16bbc2 ("crypto: include crypto- module prefix in template")

> Which argues for being extremely conservative about making something
> that has an extremely low probability of breaking users, and it points
> out why Geo Kozey's "who cares about breaking users; security is
> IMPORTANT" argument is so wrong-headed.

I don't want to break users either. I want to provide meaningful ways
for admins to reduce their kernel attack surface. Djalal and I aren't
advocating for on-by-default removal of module auto-loading (that
would have been a very small patch!). The idea was to provide a
dynamic control over it, and make that available to distros. As shown
in the patch series, it would be immediately put to use with systemd
for process-tree isolation and for container restriction.

> If the goal is to protect distro kernels, but a sloppy approach
> guarantees that distro kernels will never enable it, is it really
> worth it?

I don't think this is sloppy and of course distros would see use,
since systemd would be using it.

> P.S.  This is where it might be useful to get some input from the Red
> Hat and SuSE support teams.  How many angry user calls to their help
> desk are they willing to field before they'll just turn off the kernel
> config option for their kernels?

This isn't about giant-hammer CONFIGs -- this is like more like
PR_SET_DUMPABLE or seccomp.

-Kees

-- 
Kees Cook
Pixel Security

WARNING: multiple messages have this Message-ID (diff)
From: keescook@chromium.org (Kees Cook)
To: linux-security-module@vger.kernel.org
Subject: [kernel-hardening] Re: [PATCH v5 next 5/5] net: modules: use request_module_cap() to load 'netdev-%s' modules
Date: Tue, 28 Nov 2017 16:18:59 -0800	[thread overview]
Message-ID: <CAGXu5j+kXxkctzH79YxOu=45xgamV-5qyR2zL7OiC4POL6EcyA@mail.gmail.com> (raw)
In-Reply-To: <20171128234920.awfwicihuudw5ogx@thunk.org>

On Tue, Nov 28, 2017 at 3:49 PM, Theodore Ts'o <tytso@mit.edu> wrote:
> OK, but if the goal is to protect users who are running distro
> kernels, then a kernel config that breaks some percentage of users is
> ****highly**** unlikely to be enabled by Red Hat and SuSE, right?  And
> many of these users either can't (from a skills perspective) or won't
> (because they lose all support from the enterprise distro's help desk)
> recompile their kernel to enable an "might break 3% of all users ---
> oh, well" config option.

There's a spectrum across "insane not to be done everywhere"
(STRICT_KERNEL_RWX), "this is a good idea for nearly all Linux
systems" (link restrictions), "this can break some common use-cases,
but protects systems without that use-case" (user-namespace
disabling), "this breaks most systems, but specialized deployments
really need it" (modules_disabled).

There's also a difference between immutable CONFIG options that cannot
be disabled at runtime, those that can, global sysctls, per-namespace
controls, etc etc. The kernel is all about providing admins with knobs
to tweak their performance and security. Suddenly being told that we
can't create optional improvements is very odd.

Now, being told "make it easier to audit all the module loading we're
already doing so we can further reduce needless exposures for everyone
even without this feature enabled", THAT makes sense. My point there
is that we've already done those kinds of things; see commits like:

  7f78e0351394 ("fs: Limit sys_mount to only request filesystem modules.")
  5d26a105b5a7 ("crypto: prefix module autoloading with "crypto-"")
  4943ba16bbc2 ("crypto: include crypto- module prefix in template")

> Which argues for being extremely conservative about making something
> that has an extremely low probability of breaking users, and it points
> out why Geo Kozey's "who cares about breaking users; security is
> IMPORTANT" argument is so wrong-headed.

I don't want to break users either. I want to provide meaningful ways
for admins to reduce their kernel attack surface. Djalal and I aren't
advocating for on-by-default removal of module auto-loading (that
would have been a very small patch!). The idea was to provide a
dynamic control over it, and make that available to distros. As shown
in the patch series, it would be immediately put to use with systemd
for process-tree isolation and for container restriction.

> If the goal is to protect distro kernels, but a sloppy approach
> guarantees that distro kernels will never enable it, is it really
> worth it?

I don't think this is sloppy and of course distros would see use,
since systemd would be using it.

> P.S.  This is where it might be useful to get some input from the Red
> Hat and SuSE support teams.  How many angry user calls to their help
> desk are they willing to field before they'll just turn off the kernel
> config option for their kernels?

This isn't about giant-hammer CONFIGs -- this is like more like
PR_SET_DUMPABLE or seccomp.

-Kees

-- 
Kees Cook
Pixel Security
--
To unsubscribe from this list: send the line "unsubscribe linux-security-module" in
the body of a message to majordomo at vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

  reply	other threads:[~2017-11-29  0:19 UTC|newest]

Thread overview: 266+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2017-11-27 17:18 [PATCH v5 next 0/5] Improve Module autoloading infrastructure Djalal Harouni
2017-11-27 17:18 ` [kernel-hardening] " Djalal Harouni
2017-11-27 17:18 ` Djalal Harouni
2017-11-27 17:18 ` [PATCH v5 next 1/5] modules:capabilities: add request_module_cap() Djalal Harouni
2017-11-27 17:18   ` [kernel-hardening] " Djalal Harouni
2017-11-27 17:18   ` Djalal Harouni
2017-11-27 18:48   ` Randy Dunlap
2017-11-27 18:48     ` [kernel-hardening] " Randy Dunlap
2017-11-27 18:48     ` Randy Dunlap
2017-11-27 21:35     ` Djalal Harouni
2017-11-27 21:35       ` [kernel-hardening] " Djalal Harouni
2017-11-27 21:35       ` Djalal Harouni
2017-11-28 19:14   ` Luis R. Rodriguez
2017-11-28 19:14     ` [kernel-hardening] " Luis R. Rodriguez
2017-11-28 19:14     ` Luis R. Rodriguez
2017-11-28 20:11     ` Kees Cook
2017-11-28 20:11       ` [kernel-hardening] " Kees Cook
2017-11-28 20:11       ` Kees Cook
2017-11-28 21:16       ` Luis R. Rodriguez
2017-11-28 21:16         ` [kernel-hardening] " Luis R. Rodriguez
2017-11-28 21:16         ` Luis R. Rodriguez
2017-11-28 21:33         ` Djalal Harouni
2017-11-28 21:33           ` [kernel-hardening] " Djalal Harouni
2017-11-28 21:33           ` Djalal Harouni
2017-11-28 22:18           ` Luis R. Rodriguez
2017-11-28 22:18             ` [kernel-hardening] " Luis R. Rodriguez
2017-11-28 22:18             ` Luis R. Rodriguez
2017-11-28 22:52             ` Djalal Harouni
2017-11-28 22:52               ` [kernel-hardening] " Djalal Harouni
2017-11-28 22:52               ` Djalal Harouni
2017-11-28 21:39         ` Kees Cook
2017-11-28 21:39           ` [kernel-hardening] " Kees Cook
2017-11-28 21:39           ` Kees Cook
2017-11-28 22:12           ` Luis R. Rodriguez
2017-11-28 22:12             ` [kernel-hardening] " Luis R. Rodriguez
2017-11-28 22:12             ` Luis R. Rodriguez
2017-11-28 22:18             ` Kees Cook
2017-11-28 22:18               ` [kernel-hardening] " Kees Cook
2017-11-28 22:18               ` Kees Cook
2017-11-28 22:48               ` Luis R. Rodriguez
2017-11-28 22:48                 ` [kernel-hardening] " Luis R. Rodriguez
2017-11-28 22:48                 ` Luis R. Rodriguez
2017-11-29  7:49                 ` Michal Kubecek
2017-11-29  7:49                   ` [kernel-hardening] " Michal Kubecek
2017-11-29  7:49                   ` Michal Kubecek
2017-11-29 13:46           ` Alan Cox
2017-11-29 13:46             ` [kernel-hardening] " Alan Cox
2017-11-29 13:46             ` Alan Cox
2017-11-29 14:50             ` David Miller
2017-11-29 14:50               ` [kernel-hardening] " David Miller
2017-11-29 14:50               ` David Miller
2017-11-29 15:54               ` Theodore Ts'o
2017-11-29 15:54                 ` [kernel-hardening] " Theodore Ts'o
2017-11-29 15:54                 ` Theodore Ts'o
2017-11-29 15:58                 ` David Miller
2017-11-29 15:58                   ` [kernel-hardening] " David Miller
2017-11-29 15:58                   ` David Miller
2017-11-29 16:29                   ` Theodore Ts'o
2017-11-29 16:29                     ` [kernel-hardening] " Theodore Ts'o
2017-11-29 16:29                     ` Theodore Ts'o
2017-11-29 22:45                   ` Linus Torvalds
2017-11-29 22:45                     ` [kernel-hardening] " Linus Torvalds
2017-11-29 22:45                     ` Linus Torvalds
2017-11-29 22:45                     ` Linus Torvalds
2017-11-30  0:06                     ` Kees Cook
2017-11-30  0:06                       ` [kernel-hardening] " Kees Cook
2017-11-30  0:06                       ` Kees Cook
2017-11-30  0:06                       ` Kees Cook
2017-11-29 17:28                 ` Serge E. Hallyn
2017-11-29 17:28                   ` [kernel-hardening] " Serge E. Hallyn
2017-11-29 17:28                   ` Serge E. Hallyn
2017-11-30  0:35                   ` Theodore Ts'o
2017-11-30  0:35                     ` [kernel-hardening] " Theodore Ts'o
2017-11-30  0:35                     ` Theodore Ts'o
2017-11-30 17:17                     ` Serge E. Hallyn
2017-11-30 17:17                       ` [kernel-hardening] " Serge E. Hallyn
2017-11-30 17:17                       ` Serge E. Hallyn
2017-11-28 20:18     ` Djalal Harouni
2017-11-28 20:18       ` [kernel-hardening] " Djalal Harouni
2017-11-28 20:18       ` Djalal Harouni
2017-11-27 17:18 ` [PATCH v5 next 2/5] modules:capabilities: add cap_kernel_module_request() permission check Djalal Harouni
2017-11-27 17:18   ` [kernel-hardening] " Djalal Harouni
2017-11-27 17:18   ` Djalal Harouni
2017-11-30  2:05   ` Luis R. Rodriguez
2017-11-30  2:05     ` [kernel-hardening] " Luis R. Rodriguez
2017-11-30  2:05     ` Luis R. Rodriguez
2017-11-27 17:18 ` [PATCH v5 next 3/5] modules:capabilities: automatic module loading restriction Djalal Harouni
2017-11-27 17:18   ` [kernel-hardening] " Djalal Harouni
2017-11-27 17:18   ` Djalal Harouni
2017-11-30  1:23   ` Luis R. Rodriguez
2017-11-30  1:23     ` [kernel-hardening] " Luis R. Rodriguez
2017-11-30  1:23     ` Luis R. Rodriguez
2017-11-30 12:22     ` Djalal Harouni
2017-11-30 12:22       ` [kernel-hardening] " Djalal Harouni
2017-11-30 12:22       ` Djalal Harouni
2017-11-27 17:18 ` [PATCH v5 next 4/5] modules:capabilities: add a per-task modules auto-load mode Djalal Harouni
2017-11-27 17:18   ` [kernel-hardening] " Djalal Harouni
2017-11-27 17:18   ` Djalal Harouni
2017-11-27 17:18 ` [PATCH v5 next 5/5] net: modules: use request_module_cap() to load 'netdev-%s' modules Djalal Harouni
2017-11-27 17:18   ` [kernel-hardening] " Djalal Harouni
2017-11-27 17:18   ` Djalal Harouni
2017-11-27 18:44   ` Linus Torvalds
2017-11-27 18:44     ` [kernel-hardening] " Linus Torvalds
2017-11-27 18:44     ` Linus Torvalds
2017-11-27 18:44     ` Linus Torvalds
2017-11-27 21:41     ` Djalal Harouni
2017-11-27 21:41       ` [kernel-hardening] " Djalal Harouni
2017-11-27 21:41       ` Djalal Harouni
2017-11-27 21:41       ` Djalal Harouni
2017-11-27 22:04       ` Linus Torvalds
2017-11-27 22:04         ` [kernel-hardening] " Linus Torvalds
2017-11-27 22:04         ` Linus Torvalds
2017-11-27 22:04         ` Linus Torvalds
2017-11-27 22:59         ` Kees Cook
2017-11-27 22:59           ` [kernel-hardening] " Kees Cook
2017-11-27 22:59           ` Kees Cook
2017-11-27 22:59           ` Kees Cook
2017-11-27 23:14           ` Linus Torvalds
2017-11-27 23:14             ` [kernel-hardening] " Linus Torvalds
2017-11-27 23:14             ` Linus Torvalds
2017-11-27 23:14             ` Linus Torvalds
2017-11-27 23:19             ` Kees Cook
2017-11-27 23:19               ` [kernel-hardening] " Kees Cook
2017-11-27 23:19               ` Kees Cook
2017-11-27 23:19               ` Kees Cook
2017-11-27 23:35               ` Linus Torvalds
2017-11-27 23:35                 ` [kernel-hardening] " Linus Torvalds
2017-11-27 23:35                 ` Linus Torvalds
2017-11-27 23:35                 ` Linus Torvalds
2017-11-28  1:23             ` Kees Cook
2017-11-28  1:23               ` [kernel-hardening] " Kees Cook
2017-11-28  1:23               ` Kees Cook
2017-11-28  1:23               ` Kees Cook
2017-11-28 12:16         ` [kernel-hardening] " Geo Kozey
2017-11-28 12:16           ` Geo Kozey
2017-11-28 12:16           ` Geo Kozey
2017-11-28 19:32           ` Theodore Ts'o
2017-11-28 19:32             ` Theodore Ts'o
2017-11-28 19:32             ` Theodore Ts'o
2017-11-28 20:08             ` Kees Cook
2017-11-28 20:08               ` Kees Cook
2017-11-28 20:08               ` Kees Cook
2017-11-28 20:12               ` Linus Torvalds
2017-11-28 20:12                 ` Linus Torvalds
2017-11-28 20:12                 ` Linus Torvalds
2017-11-28 20:20                 ` Kees Cook
2017-11-28 20:20                   ` Kees Cook
2017-11-28 20:20                   ` Kees Cook
2017-11-28 20:33                   ` Linus Torvalds
2017-11-28 20:33                     ` Linus Torvalds
2017-11-28 20:33                     ` Linus Torvalds
2017-11-28 21:10                     ` Djalal Harouni
2017-11-28 21:10                       ` Djalal Harouni
2017-11-28 21:10                       ` Djalal Harouni
2017-11-28 21:33                     ` Kees Cook
2017-11-28 21:33                       ` Kees Cook
2017-11-28 21:33                       ` Kees Cook
2017-11-28 23:23                       ` Theodore Ts'o
2017-11-28 23:23                         ` Theodore Ts'o
2017-11-28 23:23                         ` Theodore Ts'o
2017-11-28 23:29                         ` Kees Cook
2017-11-28 23:29                           ` Kees Cook
2017-11-28 23:29                           ` Kees Cook
2017-11-28 23:49                           ` Theodore Ts'o
2017-11-28 23:49                             ` Theodore Ts'o
2017-11-28 23:49                             ` Theodore Ts'o
2017-11-29  0:18                             ` Kees Cook [this message]
2017-11-29  0:18                               ` Kees Cook
2017-11-29  0:18                               ` Kees Cook
2017-11-29  6:36                               ` Theodore Ts'o
2017-11-29  6:36                                 ` Theodore Ts'o
2017-11-29  6:36                                 ` Theodore Ts'o
2017-11-29 14:46                             ` Geo Kozey
2017-11-29 14:46                               ` Geo Kozey
2017-11-29 14:46                               ` Geo Kozey
2017-12-01 15:22                             ` Marcus Meissner
2017-12-01 15:22                               ` Marcus Meissner
2017-12-01 15:22                               ` Marcus Meissner
2017-11-28 23:53                         ` Djalal Harouni
2017-11-28 23:53                           ` Djalal Harouni
2017-11-28 23:53                           ` Djalal Harouni
2017-11-28 21:51                     ` Geo Kozey
2017-11-28 21:51                       ` Geo Kozey
2017-11-28 21:51                       ` Geo Kozey
2017-11-28 23:51                       ` Linus Torvalds
2017-11-28 23:51                         ` Linus Torvalds
2017-11-28 23:51                         ` Linus Torvalds
2017-11-29  0:17                         ` Linus Torvalds
2017-11-29  0:17                           ` Linus Torvalds
2017-11-29  0:17                           ` Linus Torvalds
2017-11-29  0:26                           ` Kees Cook
2017-11-29  0:26                             ` Kees Cook
2017-11-29  0:26                             ` Kees Cook
2017-11-29  0:50                             ` Linus Torvalds
2017-11-29  0:50                               ` Linus Torvalds
2017-11-29  0:50                               ` Linus Torvalds
2017-11-29  4:26                               ` Eric W. Biederman
2017-11-29  4:26                                 ` Eric W. Biederman
2017-11-29  4:26                                 ` Eric W. Biederman
2017-11-29 18:30                               ` Kees Cook
2017-11-29 18:30                                 ` Kees Cook
2017-11-29 18:30                                 ` Kees Cook
2017-11-29 18:46                                 ` Linus Torvalds
2017-11-29 18:46                                   ` Linus Torvalds
2017-11-29 18:46                                   ` Linus Torvalds
2017-11-29 18:53                                   ` Linus Torvalds
2017-11-29 18:53                                     ` Linus Torvalds
2017-11-29 18:53                                     ` Linus Torvalds
2017-11-29 21:17                                   ` Kees Cook
2017-11-29 21:17                                     ` Kees Cook
2017-11-29 21:17                                     ` Kees Cook
2017-11-29 22:14                                     ` Linus Torvalds
2017-11-29 22:14                                       ` Linus Torvalds
2017-11-29 22:14                                       ` Linus Torvalds
2017-11-30  0:44                                       ` Kees Cook
2017-11-30  0:44                                         ` Kees Cook
2017-11-30  0:44                                         ` Kees Cook
2017-11-30  2:08                                         ` Linus Torvalds
2017-11-30  2:08                                           ` Linus Torvalds
2017-11-30  2:08                                           ` Linus Torvalds
2017-11-30  6:51                                       ` Daniel Micay
2017-11-30  6:51                                         ` Daniel Micay
2017-11-30  6:51                                         ` Daniel Micay
2017-11-30  8:50                                         ` Djalal Harouni
2017-11-30  8:50                                           ` Djalal Harouni
2017-11-30  8:50                                           ` Djalal Harouni
2017-11-30 14:16                                           ` Theodore Ts'o
2017-11-30 14:16                                             ` Theodore Ts'o
2017-11-30 14:16                                             ` Theodore Ts'o
2017-11-30 14:51                                             ` Djalal Harouni
2017-11-30 14:51                                               ` Djalal Harouni
2017-11-30 14:51                                               ` Djalal Harouni
2017-12-01  6:39                                           ` Daniel Micay
2017-12-01  6:39                                             ` Daniel Micay
2017-12-01  6:39                                             ` Daniel Micay
2017-11-29 15:28                           ` Geo Kozey
2017-11-29 15:28                             ` Geo Kozey
2017-11-29 15:28                             ` Geo Kozey
2017-11-27 18:41 ` [PATCH v5 next 0/5] Improve Module autoloading infrastructure Linus Torvalds
2017-11-27 18:41   ` [kernel-hardening] " Linus Torvalds
2017-11-27 18:41   ` Linus Torvalds
2017-11-27 18:41   ` Linus Torvalds
2017-11-27 19:02   ` Linus Torvalds
2017-11-27 19:02     ` [kernel-hardening] " Linus Torvalds
2017-11-27 19:02     ` Linus Torvalds
2017-11-27 19:02     ` Linus Torvalds
2017-11-27 19:12     ` Linus Torvalds
2017-11-27 19:12       ` [kernel-hardening] " Linus Torvalds
2017-11-27 19:12       ` Linus Torvalds
2017-11-27 19:12       ` Linus Torvalds
2017-11-27 21:31       ` Djalal Harouni
2017-11-27 21:31         ` [kernel-hardening] " Djalal Harouni
2017-11-27 21:31         ` Djalal Harouni
2017-11-27 21:31         ` Djalal Harouni
2017-11-27 19:14   ` David Miller
2017-11-27 19:14     ` [kernel-hardening] " David Miller
2017-11-27 19:14     ` David Miller
2017-11-27 22:31     ` James Morris
2017-11-27 22:31       ` [kernel-hardening] " James Morris
2017-11-27 22:31       ` James Morris
2017-11-27 23:04       ` Kees Cook
2017-11-27 23:04         ` [kernel-hardening] " Kees Cook
2017-11-27 23:04         ` Kees Cook
2017-11-27 23:44         ` James Morris
2017-11-27 23:44           ` [kernel-hardening] " James Morris
2017-11-27 23:44           ` James Morris

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='CAGXu5j+kXxkctzH79YxOu=45xgamV-5qyR2zL7OiC4POL6EcyA@mail.gmail.com' \
    --to=keescook@chromium.org \
    --cc=corbet@lwn.net \
    --cc=geokozey@mailfence.com \
    --cc=james.l.morris@oracle.com \
    --cc=kernel-hardening@lists.openwall.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-security-module@vger.kernel.org \
    --cc=tixxdz@gmail.com \
    --cc=torvalds@linux-foundation.org \
    --cc=tycho@docker.com \
    --cc=tytso@mit.edu \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.