linux-ext4.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: "Theodore Y. Ts'o" <tytso@mit.edu>
To: "Alexander E. Patrakov" <patrakov@gmail.com>
Cc: "Ahmed S. Darwish" <darwish.07@gmail.com>,
	Linus Torvalds <torvalds@linux-foundation.org>,
	Michael Kerrisk <mtk.manpages@gmail.com>,
	Andreas Dilger <adilger.kernel@dilger.ca>,
	Jan Kara <jack@suse.cz>, Ray Strode <rstrode@redhat.com>,
	William Jon McCann <mccann@jhu.edu>,
	zhangjs <zachary@baishancloud.com>,
	linux-ext4@vger.kernel.org, lkml <linux-kernel@vger.kernel.org>
Subject: [PATCH RFC v2] random: optionally block in getrandom(2) when the CRNG is uninitialized
Date: Sun, 15 Sep 2019 01:22:42 -0400	[thread overview]
Message-ID: <20190915052242.GG19710@mit.edu> (raw)
In-Reply-To: <008f17bc-102b-e762-a17c-e2766d48f515@gmail.com>

getrandom() has been created as a new and more secure interface for
pseudorandom data requests.  Unlike /dev/urandom, it unconditionally
blocks until the entropy pool has been properly initialized.

While getrandom() has no guaranteed upper bound for its waiting time,
user-space has been abusing it by issuing the syscall, from shared
libraries no less, during the main system boot sequence.

Thus, on certain setups where there is no hwrng (embedded), or the
hwrng is not trusted by some users (intel RDRAND), or sometimes it's
just broken (amd RDRAND), the system boot can be *reliably* blocked.

The issue is further exaggerated by recent file-system optimizations,
e.g. b03755ad6f33 (ext4: make __ext4_get_inode_loc plug), which
merges directory lookup code inode table IO, and thus minimizes the
number of disk interrupts and entropy during boot. After that commit,
a blocked boot can be reliably reproduced on a Thinkpad E480 laptop
with standard ArchLinux user-space.

Thus, add an optional configuration option which stops getrandom(2)
from blocking, but instead returns "best efforts" randomness, which
might not be random or secure at all.  This can be controlled via
random.getrandom_block boot command line option, and the
CONFIG_RANDOM_BLOCK can be used to set the default to be blocking.
Since according to the Great Penguin, only incompetent system
designers would value "security" ahead of "usability", the default is
to be non-blocking.

In addition, modify getrandom(2) to complain loudly with a kernel
warning when some userspace process is erroneously calling
getrandom(2) too early during the boot process.

Link: https://lkml.kernel.org/r/CAHk-=wjyH910+JRBdZf_Y9G54c1M=LBF8NKXB6vJcm9XjLnRfg@mail.gmail.com
Link: https://lkml.kernel.org/r/20190912034421.GA2085@darwi-home-pc
Link: https://lkml.kernel.org/r/20190911173624.GI2740@mit.edu
Link: https://lkml.kernel.org/r/20180514003034.GI14763@thunk.org

[ Modified by tytso@mit.edu to make the change of getrandom(2) to be
  non-blocking to be optional. ]

Suggested-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Ahmed S. Darwish <darwish.07@gmail.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
---

Here's my take on the patch.  I really very strongly believe that the
idea of making getrandom(2) non-blocking and to blindly assume that we
can load up the buffer with "best efforts" randomness to be a
terrible, terrible idea that is going to cause major security problems
that we will potentially regret very badly.  Linus Torvalds believes I
am an incompetent systems designer.

So let's do it both ways, and push the decision on the distributor
and/or product manufacturer

 drivers/char/Kconfig  | 33 +++++++++++++++++++++++++++++++--
 drivers/char/random.c | 34 +++++++++++++++++++++++++++++-----
 2 files changed, 60 insertions(+), 7 deletions(-)

diff --git a/drivers/char/Kconfig b/drivers/char/Kconfig
index 3e866885a405..337baeca5ebc 100644
--- a/drivers/char/Kconfig
+++ b/drivers/char/Kconfig
@@ -557,8 +557,6 @@ config ADI
 	  and SSM (Silicon Secured Memory).  Intended consumers of this
 	  driver include crash and makedumpfile.
 
-endmenu
-
 config RANDOM_TRUST_CPU
 	bool "Trust the CPU manufacturer to initialize Linux's CRNG"
 	depends on X86 || S390 || PPC
@@ -573,3 +571,34 @@ config RANDOM_TRUST_CPU
 	has not installed a hidden back door to compromise the CPU's
 	random number generation facilities. This can also be configured
 	at boot with "random.trust_cpu=on/off".
+
+config RANDOM_BLOCK
+	bool "Block if getrandom is called before CRNG is initialized"
+	help
+	  Say Y here if you want userspace programs which call
+	  getrandom(2) before the Cryptographic Random Number
+	  Generator (CRNG) is initialized to block until
+	  secure random numbers are available.
+
+	  Say N if you believe usability is more important than
+	  security, so if getrandom(2) is called before the CRNG is
+	  initialized, it should not block, but instead return "best
+	  effort" randomness which might not be very secure or random
+	  at all; but at least the system boot will not be delayed by
+	  minutes or hours.
+
+	  This can also be controlled at boot with
+	  "random.getrandom_block=on/off".
+
+	  Ideally, systems would be configured with hardware random
+	  number generators, and/or configured to trust CPU-provided
+	  RNG's.  In addition, userspace should generate cryptographic
+	  keys only as late as possible, when they are needed, instead
+	  of during early boot.  (For non-cryptographic use cases,
+	  such as dictionary seeds or MIT Magic Cookies, other
+	  mechanisms such as /dev/urandom or random(3) may be more
+	  appropropriate.)  This config option controls what the
+	  kernel should do as a fallback when the non-ideal case
+	  presents itself.
+
+endmenu
diff --git a/drivers/char/random.c b/drivers/char/random.c
index 5d5ea4ce1442..243fb4a4535f 100644
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -511,6 +511,8 @@ static struct ratelimit_state unseeded_warning =
 	RATELIMIT_STATE_INIT("warn_unseeded_randomness", HZ, 3);
 static struct ratelimit_state urandom_warning =
 	RATELIMIT_STATE_INIT("warn_urandom_randomness", HZ, 3);
+static struct ratelimit_state getrandom_warning =
+	RATELIMIT_STATE_INIT("warn_getrandom_randomness", HZ, 3);
 
 static int ratelimit_disable __read_mostly;
 
@@ -854,12 +856,19 @@ static void invalidate_batched_entropy(void);
 static void numa_crng_init(void);
 
 static bool trust_cpu __ro_after_init = IS_ENABLED(CONFIG_RANDOM_TRUST_CPU);
+static bool getrandom_block __ro_after_init = IS_ENABLED(CONFIG_RANDOM_BLOCK);
 static int __init parse_trust_cpu(char *arg)
 {
 	return kstrtobool(arg, &trust_cpu);
 }
 early_param("random.trust_cpu", parse_trust_cpu);
 
+static int __init parse_block(char *arg)
+{
+	return kstrtobool(arg, &getrandom_block);
+}
+early_param("random.getrandom_block", parse_block);
+
 static void crng_initialize(struct crng_state *crng)
 {
 	int		i;
@@ -1045,6 +1054,12 @@ static void crng_reseed(struct crng_state *crng, struct entropy_store *r)
 				  urandom_warning.missed);
 			urandom_warning.missed = 0;
 		}
+		if (getrandom_warning.missed) {
+			pr_notice("random: %d getrandom warning(s) missed "
+				  "due to ratelimiting\n",
+				  getrandom_warning.missed);
+			getrandom_warning.missed = 0;
+		}
 	}
 }
 
@@ -1900,6 +1915,7 @@ int __init rand_initialize(void)
 	crng_global_init_time = jiffies;
 	if (ratelimit_disable) {
 		urandom_warning.interval = 0;
+		getrandom_warning.interval = 0;
 		unseeded_warning.interval = 0;
 	}
 	return 0;
@@ -1969,8 +1985,8 @@ urandom_read(struct file *file, char __user *buf, size_t nbytes, loff_t *ppos)
 	if (!crng_ready() && maxwarn > 0) {
 		maxwarn--;
 		if (__ratelimit(&urandom_warning))
-			printk(KERN_NOTICE "random: %s: uninitialized "
-			       "urandom read (%zd bytes read)\n",
+			pr_err("random: %s: CRNG uninitialized "
+			       "(%zd bytes read)\n",
 			       current->comm, nbytes);
 		spin_lock_irqsave(&primary_crng.lock, flags);
 		crng_init_cnt = 0;
@@ -2135,9 +2151,17 @@ SYSCALL_DEFINE3(getrandom, char __user *, buf, size_t, count,
 	if (!crng_ready()) {
 		if (flags & GRND_NONBLOCK)
 			return -EAGAIN;
-		ret = wait_for_random_bytes();
-		if (unlikely(ret))
-			return ret;
+		WARN_ON_ONCE(1);
+		if (getrandom_block) {
+			if (__ratelimit(&getrandom_warning))
+				pr_err("random: %s: getrandom blocking for CRNG initialization\n",
+				       current->comm);
+			ret = wait_for_random_bytes();
+			if (unlikely(ret))
+				return ret;
+		} else if (__ratelimit(&getrandom_warning))
+			pr_err("random: %s: getrandom called too early\n",
+			       current->comm);
 	}
 	return urandom_read(NULL, buf, count, NULL);
 }
-- 
2.23.0


  reply	other threads:[~2019-09-15  5:23 UTC|newest]

Thread overview: 211+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
     [not found] <CAHk-=whBQ+6c-h+htiv6pp8ndtv97+45AH9WvdZougDRM6M4VQ@mail.gmail.com>
2019-09-10  4:21 ` Linux 5.3-rc8 Ahmed S. Darwish
2019-09-10 11:33   ` Linus Torvalds
2019-09-10 12:21     ` Linus Torvalds
2019-09-10 17:33     ` Ahmed S. Darwish
2019-09-10 17:47       ` Reindl Harald
2019-09-10 18:21       ` Linus Torvalds
2019-09-11 16:07         ` Theodore Y. Ts'o
2019-09-11 16:45           ` Linus Torvalds
2019-09-11 17:00             ` Linus Torvalds
2019-09-11 17:36               ` Theodore Y. Ts'o
2019-09-12  3:44                 ` Ahmed S. Darwish
2019-09-12  8:25                   ` Theodore Y. Ts'o
2019-09-12 11:34                     ` Linus Torvalds
2019-09-12 11:58                       ` Willy Tarreau
2019-09-14 12:25                       ` [PATCH RFC] random: getrandom(2): don't block on non-initialized entropy pool Ahmed S. Darwish
2019-09-14 14:08                         ` Alexander E. Patrakov
2019-09-15  5:22                           ` Theodore Y. Ts'o [this message]
2019-09-15  8:17                             ` [PATCH RFC v3] random: getrandom(2): optionally block when CRNG is uninitialized Ahmed S. Darwish
2019-09-15  8:59                               ` Lennart Poettering
2019-09-15  9:30                                 ` Willy Tarreau
2019-09-15 10:02                                   ` Ahmed S. Darwish
2019-09-15 10:40                                     ` Willy Tarreau
2019-09-15 10:55                                       ` Ahmed S. Darwish
2019-09-15 11:17                                         ` Willy Tarreau
2019-09-15 17:32                             ` [PATCH RFC v2] random: optionally block in getrandom(2) when the " Linus Torvalds
2019-09-15 18:32                               ` Willy Tarreau
2019-09-15 18:36                                 ` Willy Tarreau
2019-09-15 19:08                                   ` Linus Torvalds
2019-09-15 19:18                                     ` Willy Tarreau
2019-09-15 19:31                                       ` Linus Torvalds
2019-09-15 19:54                                         ` Willy Tarreau
2019-09-15 18:59                                 ` Linus Torvalds
2019-09-15 19:12                                   ` Willy Tarreau
2019-09-16  2:45                                   ` Ahmed S. Darwish
2019-09-16 18:08                               ` Lennart Poettering
2019-09-16 19:16                                 ` Willy Tarreau
2019-09-18 21:15                               ` [PATCH RFC v4 0/1] random: WARN on large getrandom() waits and introduce getrandom2() Ahmed S. Darwish
2019-09-18 21:17                                 ` [PATCH RFC v4 1/1] " Ahmed S. Darwish
2019-09-18 23:57                                   ` Linus Torvalds
2019-09-19 14:34                                     ` Theodore Y. Ts'o
2019-09-19 15:20                                       ` Linus Torvalds
2019-09-19 15:50                                         ` Linus Torvalds
2019-09-20 13:13                                           ` Theodore Y. Ts'o
2019-09-19 20:04                                         ` Linus Torvalds
2019-09-19 20:45                                           ` Alexander E. Patrakov
2019-09-19 21:47                                             ` Linus Torvalds
2019-09-19 22:23                                               ` Alexander E. Patrakov
2019-09-19 23:44                                                 ` Alexander E. Patrakov
2019-09-20 13:16                                                 ` Theodore Y. Ts'o
2019-09-23 11:55                                           ` David Laight
2019-09-20 13:08                                         ` Theodore Y. Ts'o
2019-09-20 13:46                                     ` Ahmed S. Darwish
2019-09-20 14:33                                       ` Andy Lutomirski
2019-09-20 16:29                                         ` Linus Torvalds
2019-09-20 17:52                                           ` Andy Lutomirski
2019-09-20 18:09                                             ` Linus Torvalds
2019-09-20 18:16                                               ` Willy Tarreau
2019-09-20 19:12                                               ` Andy Lutomirski
2019-09-20 19:51                                                 ` Linus Torvalds
2019-09-20 20:11                                                   ` Alexander E. Patrakov
2019-09-20 20:17                                                   ` Matthew Garrett
2019-09-20 20:51                                                   ` Andy Lutomirski
2019-09-20 22:44                                                     ` Linus Torvalds
2019-09-20 23:30                                                       ` Andy Lutomirski
2019-09-21  3:05                                                         ` Willy Tarreau
2019-09-21  6:07                                               ` Florian Weimer
2019-09-23 18:33                                                 ` Andy Lutomirski
2019-09-26 21:11                                                   ` Ahmed S. Darwish
2019-09-20 18:12                                             ` Willy Tarreau
2019-09-20 19:22                                               ` Andy Lutomirski
2019-09-20 19:37                                                 ` Willy Tarreau
2019-09-20 19:52                                                   ` Andy Lutomirski
2019-09-20 20:02                                                 ` Linus Torvalds
2019-09-20 18:15                                             ` Alexander E. Patrakov
2019-09-20 18:29                                               ` Andy Lutomirski
2019-09-20 17:26                                       ` Willy Tarreau
2019-09-20 17:56                                         ` Ahmed S. Darwish
2019-09-26 20:42                                     ` [PATCH v5 0/1] random: getrandom(2): warn on large CRNG waits, introduce new flags Ahmed S. Darwish
2019-09-26 20:44                                       ` [PATCH v5 1/1] " Ahmed S. Darwish
2019-09-26 21:39                                         ` Andy Lutomirski
2019-09-28  9:30                                           ` Ahmed S. Darwish
2019-09-14 15:02                       ` Linux 5.3-rc8 Ahmed S. Darwish
2019-09-14 16:30                         ` Linus Torvalds
2019-09-14 16:35                           ` Alexander E. Patrakov
2019-09-14 16:52                             ` Linus Torvalds
2019-09-14 17:09                               ` Alexander E. Patrakov
2019-09-14 19:19                                 ` Linus Torvalds
2019-09-15  6:56                               ` Lennart Poettering
2019-09-15  7:01                                 ` Willy Tarreau
2019-09-15  7:05                                   ` Lennart Poettering
2019-09-15  7:07                                     ` Willy Tarreau
2019-09-15  8:34                                       ` Lennart Poettering
2019-09-15 17:02                                 ` Linus Torvalds
2019-09-16  3:23                                   ` Theodore Y. Ts'o
2019-09-16  3:40                                     ` Linus Torvalds
2019-09-16  3:56                                       ` Linus Torvalds
2019-09-16 17:00                                       ` Theodore Y. Ts'o
2019-09-16 17:07                                         ` Linus Torvalds
2019-09-14 21:11                           ` Ahmed S. Darwish
2019-09-14 22:05                             ` Martin Steigerwald
2019-09-14 22:24                             ` Theodore Y. Ts'o
2019-09-14 22:32                               ` Linus Torvalds
2019-09-15  1:00                                 ` Theodore Y. Ts'o
2019-09-15  1:10                                   ` Linus Torvalds
2019-09-15  2:05                                     ` Theodore Y. Ts'o
2019-09-15  2:11                                       ` Linus Torvalds
2019-09-15  6:33                                       ` Willy Tarreau
2019-09-15  6:53                                       ` Willy Tarreau
2019-09-15  6:51                           ` Lennart Poettering
2019-09-15  7:27                             ` Ahmed S. Darwish
2019-09-15  8:48                               ` Lennart Poettering
2019-09-15 16:29                             ` Linus Torvalds
2019-09-16  1:40                               ` Ahmed S. Darwish
2019-09-16  1:48                                 ` Vito Caputo
2019-09-16  2:49                                   ` Theodore Y. Ts'o
2019-09-16  4:29                                     ` Willy Tarreau
2019-09-16  5:02                                       ` Linus Torvalds
2019-09-16  6:12                                         ` Willy Tarreau
2019-09-16 16:17                                           ` Linus Torvalds
2019-09-16 17:21                                             ` Theodore Y. Ts'o
2019-09-16 17:44                                               ` Linus Torvalds
2019-09-16 17:55                                                 ` Serge Belyshev
2019-09-16 19:08                                                 ` Willy Tarreau
2019-09-16 23:02                                                 ` Matthew Garrett
2019-09-16 23:05                                                   ` Linus Torvalds
2019-09-16 23:11                                                     ` Matthew Garrett
2019-09-16 23:13                                                       ` Alexander E. Patrakov
2019-09-16 23:15                                                         ` Matthew Garrett
2019-09-16 23:18                                                       ` Linus Torvalds
2019-09-16 23:29                                                         ` Ahmed S. Darwish
2019-09-17  1:05                                                           ` Linus Torvalds
2019-09-17  1:23                                                             ` Matthew Garrett
2019-09-17  1:41                                                               ` Linus Torvalds
2019-09-17  1:46                                                                 ` Matthew Garrett
2019-09-17  5:24                                                                   ` Willy Tarreau
2019-09-17  7:33                                                                     ` Martin Steigerwald
2019-09-17  8:35                                                                       ` Willy Tarreau
2019-09-17  8:44                                                                         ` Martin Steigerwald
2019-09-17 12:11                                                                       ` Theodore Y. Ts'o
2019-09-17 12:30                                                                         ` Ahmed S. Darwish
2019-09-17 12:46                                                                           ` Alexander E. Patrakov
2019-09-17 12:47                                                                           ` Willy Tarreau
2019-09-17 16:08                                                                           ` Lennart Poettering
2019-09-17 16:23                                                                             ` Linus Torvalds
2019-09-17 16:34                                                                               ` Reindl Harald
2019-09-17 17:42                                                                               ` Lennart Poettering
2019-09-17 18:01                                                                                 ` Linus Torvalds
2019-09-17 20:28                                                                                   ` Martin Steigerwald
2019-09-17 20:52                                                                                     ` Ahmed S. Darwish
2019-09-17 21:38                                                                                       ` Martin Steigerwald
2019-09-17 21:52                                                                                         ` Matthew Garrett
2019-09-17 22:10                                                                                           ` Martin Steigerwald
2019-09-18 13:53                                                                                             ` Lennart Poettering
2019-09-19  7:28                                                                                               ` Martin Steigerwald
2019-09-17 23:08                                                                                           ` Linus Torvalds
2019-09-18 13:40                                                                                         ` Lennart Poettering
2019-09-17 20:58                                                                                   ` Linus Torvalds
2019-09-18  9:33                                                                                     ` Rasmus Villemoes
2019-09-18 10:16                                                                                       ` Willy Tarreau
2019-09-18 10:25                                                                                         ` Alexander E. Patrakov
2019-09-18 10:42                                                                                           ` Willy Tarreau
2019-09-18 19:31                                                                                       ` Linus Torvalds
2019-09-18 19:56                                                                                 ` Eric W. Biederman
2019-09-18 20:13                                                                                   ` Linus Torvalds
2019-09-18 20:15                                                                                   ` Alexander E. Patrakov
2019-09-18 20:26                                                                                     ` Linus Torvalds
2019-09-18 22:12                                                                                       ` Willy Tarreau
2019-09-27 13:57                                                                                       ` Lennart Poettering
2019-09-27 15:58                                                                                         ` Linus Torvalds
2019-09-29  9:05                                                                                           ` Lennart Poettering
2019-09-17 13:11                                                                         ` Alexander E. Patrakov
2019-09-17 13:37                                                                           ` Alexander E. Patrakov
2019-09-17 15:57                                                                         ` Lennart Poettering
2019-09-17 16:21                                                                           ` Willy Tarreau
2019-09-17 17:13                                                                             ` Lennart Poettering
2019-09-17 17:29                                                                               ` Willy Tarreau
2019-09-17 20:42                                                                                 ` Martin Steigerwald
2019-09-18 13:38                                                                                 ` Lennart Poettering
2019-09-18 13:59                                                                                   ` Alexander E. Patrakov
2019-09-18 14:50                                                                                     ` Alexander E. Patrakov
2019-09-17 20:36                                                                             ` Martin Steigerwald
2019-09-17 16:27                                                                       ` Linus Torvalds
2019-09-17 16:34                                                                         ` Matthew Garrett
2019-09-17 17:16                                                                           ` Willy Tarreau
2019-09-17 17:20                                                                             ` Matthew Garrett
2019-09-17 17:23                                                                               ` Matthew Garrett
2019-09-17 17:57                                                                               ` Willy Tarreau
2019-09-17 16:58                                                                         ` Alexander E. Patrakov
2019-09-17 17:30                                                                           ` Lennart Poettering
2019-09-17 17:32                                                                             ` Willy Tarreau
2019-09-17 17:41                                                                               ` Alexander E. Patrakov
2019-09-17 17:28                                                                         ` Lennart Poettering
2019-09-17  0:03                                                         ` Matthew Garrett
2019-09-17  0:40                                                         ` Matthew Garrett
2019-09-17  7:15                                                     ` a sane approach to random numbers (was: Re: Linux 5.3-rc8) Martin Steigerwald
2019-09-16 18:00                                               ` Linux 5.3-rc8 Alexander E. Patrakov
2019-09-16 19:53                                               ` Ahmed S. Darwish
2019-09-17 15:32                                               ` Lennart Poettering
2019-09-16  3:31                                 ` Linus Torvalds
2019-09-23 20:49                           ` chaos generating driver was " Pavel Machek
2019-09-14  9:25                     ` Ahmed S. Darwish
2019-09-14 16:27                       ` Theodore Y. Ts'o
2019-09-11 21:41             ` Ahmed S. Darwish
2019-09-11 22:37               ` Ahmed S. Darwish
2019-09-16  3:52           ` Herbert Xu
2019-09-16  4:21             ` Linus Torvalds
2019-09-16  4:53               ` Willy Tarreau
2019-09-10 11:56   ` Theodore Y. Ts'o
2019-09-16 10:33     ` Christoph Hellwig
2019-10-03 21:10   ` Jon Masters
2019-10-03 21:31   ` Jon Masters

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20190915052242.GG19710@mit.edu \
    --to=tytso@mit.edu \
    --cc=adilger.kernel@dilger.ca \
    --cc=darwish.07@gmail.com \
    --cc=jack@suse.cz \
    --cc=linux-ext4@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mccann@jhu.edu \
    --cc=mtk.manpages@gmail.com \
    --cc=patrakov@gmail.com \
    --cc=rstrode@redhat.com \
    --cc=torvalds@linux-foundation.org \
    --cc=zachary@baishancloud.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).