Linux-ext4 Archive on lore.kernel.org
 help / color / Atom feed
From: Linus Torvalds <torvalds@linux-foundation.org>
To: Andy Lutomirski <luto@kernel.org>
Cc: "Ahmed S. Darwish" <darwish.07@gmail.com>,
	Lennart Poettering <mzxreary@0pointer.de>,
	"Theodore Y. Ts'o" <tytso@mit.edu>,
	"Eric W. Biederman" <ebiederm@xmission.com>,
	"Alexander E. Patrakov" <patrakov@gmail.com>,
	Michael Kerrisk <mtk.manpages@gmail.com>,
	Willy Tarreau <w@1wt.eu>, Matthew Garrett <mjg59@srcf.ucam.org>,
	lkml <linux-kernel@vger.kernel.org>,
	Ext4 Developers List <linux-ext4@vger.kernel.org>,
	Linux API <linux-api@vger.kernel.org>,
	linux-man <linux-man@vger.kernel.org>
Subject: Re: [PATCH RFC v4 1/1] random: WARN on large getrandom() waits and introduce getrandom2()
Date: Fri, 20 Sep 2019 12:51:12 -0700
Message-ID: <CAHk-=whJ3kmcZp=Ws+uXnRB9KokG6nXSQCSuBnerG--jkAfP5w@mail.gmail.com> (raw)
In-Reply-To: <CALCETrUEqjFmPvpcJQwJe3dNbz8eaJ4k3_AV2u0v96MffjLn+g@mail.gmail.com>

On Fri, Sep 20, 2019 at 12:12 PM Andy Lutomirski <luto@kernel.org> wrote:
>
> The problem is that new programs will have to try the new flag value
> and, if it returns -EINVAL, fall back to 0.  This isn't so great.

Don't be silly.

Of course they will do that, but so what? With a new kernel, they'll
get the behavior they expect. And with an old kernel, they'll get the
behavior they expect.

They'd never fall back to to "0 means something I didn't want",
exactly because we'd make this new flag be the first change.

> Wait, are you suggesting that 0 means invoke jitter-entropy or
> whatever and GRND_SECURE_BLOCKING means not wait forever and deadlock?
>  That's no good -- people will want to continue using 0 because the
> behavior is better.

I assume that "not wait forever" was meant to be "wait forever".

So the one thing we have to do is break the "0 waits forever".  I
guarantee that will happen. I will override Ted if he just NAk's it,
because we simply _cannot_ continue with it.

So we absolutely _will_ come up with some way 0 ends the wait. Whether
it's _just_ a timeout, or whether it's jitter-entropy or whatever, it
will happen.

But we'll also make getrandom(0) do the annoying warning, because it's
just ambiguous. And I suspect you'll find that a lot of security
people don't really like jitter-entropy, at least not in whatever
cut-down format we'll likely have to use in the kernel.

And we'll also have to make getrandom(0) be really _timely_. Security
people would likely rather wait for minutes before they are happy with
it. But because it's a boot constraint as things are now, it will not
just be jitter-entropy, it will be _accelerated_ jitter-entropy in 15
seconds or whatever, and since it can't use up all of CPU time, it's
realistically more like "15 second timeout, but less of actual CPU
time for jitter".

We can try to be clever with a background thread and a lot of
yielding(), so that if the CPU is actually idle we'll get most of that
15 seconds for whatever jitter, but end result is that it's still
accelerated.

Do I believe we can do a good job in that kind of timeframe?
Absolutely. The whole point should be that it's still "good enough",
and as has been pointed out, that same jitter entropy that people are
worried about is just done in user space right now instead.

But do I believe that security people would prefer a non-accelerated
GRND_SECURE_BLOCKING? Yes I do. That doesn't mean that
GRND_SECURE_BLOCKING shouldn't use jitter entropy too, but it doesn't
need the same kind of "let's hurry this up because it might be during
early boot and block things".

That said, if we can all convince everybody (hah!) that jitter entropy
in the kernel would be sufficient, then we can make the whole point
entirely moot, and just say "we'll just change crng_wait() to do
jitter entropy instead and be done with it. Then any getrandom() user
will just basically wait for a (very limited) time and the system will
be happy.

If that is the case we wouldn't need new flags at all. But I don't
think you can make everybody agree to that, which is why I suspect
we'll need the new flag, and I'll just take the heat for saying "0 is
now off limits, because it does this thing that a lot of people
dislike".

> IMO this is confusing.  The GRND_RANDOM flag was IMO a mistake and
> should just be retired.  Let's enumerate useful cases and then give
> them sane values.


That's basically what I'm doing. I enumerate the new values.

But the enumerations have hidden meaning, because the actual bits do
matter. The GRND_EXPLICIT bit isn't supposed to be used by any user,
but it has the value it has because it makes old kernels return
-EINVAL.

But if people hate the bit names, we can just do an enum and be done with it:

   enum grnd_flags {
      GRND_NONBLOCK = 1,
      GRND_RANDOM, // Don't use!
      GRND_RANDOM_NONBLOCK, // Don't use
      GRND_UNUSED,
      GRND_INSECURE,
      GRND_SECURE_BLOCKING,
      GRND_SECURE_NONBLOCKING,
  };

but the values now have a _hidden_ pattern (because we currently have
that "| GRND_NONBLOCK" pattern that I want to make sure still
continues to work, rather than give unexpected behavior in case
somebody continues to use it).

So the _only_ difference between the above and what I suggested is
that I made the bit pattern explicit rather than hidden in the value.

> And the only real question is how to map existing users to these
> semantics.  I see two sensible choices:
>
> 1. 0 means "secure, blocking". I think this is not what we'd do if we
> could go back in time and chage the ABI from day 1, but I think it's
> actually good enough.  As long as this mode won't deadlock, it's not
> *that* bad if programs are using it when they wanted "insecure".

It's exactly that "as long as it won't deadlock" that is our current problem.

It *does* deadlock.

So it can't mean "blocking" in any long-term meaning.

It can mean "blocks for up to 15 seconds" or something like that. I'd
honestly prefer a smaller number, but I think 15 seconds is an
acceptable "your user space is buggy, but we won't make you think the
machine hung".

> 2. 0 means "secure, blocking, but warn".  Some new value means
> "secure, blocking, don't warn".  The problem is that new applications
> will have to fall back to 0 to continue supporting old kernels.

The same comment about blocking.

Maybe you came in in the middle, and didn't see the whole "reduced IO
patterns means that boot blocks forever" part of the original problem.

THAT is why 0 will absolutely change behaviour.

                Linus

  reply index

Thread overview: 211+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
     [not found] <CAHk-=whBQ+6c-h+htiv6pp8ndtv97+45AH9WvdZougDRM6M4VQ@mail.gmail.com>
2019-09-10  4:21 ` Linux 5.3-rc8 Ahmed S. Darwish
2019-09-10 11:33   ` Linus Torvalds
2019-09-10 12:21     ` Linus Torvalds
2019-09-10 17:33     ` Ahmed S. Darwish
2019-09-10 17:47       ` Reindl Harald
2019-09-10 18:21       ` Linus Torvalds
2019-09-11 16:07         ` Theodore Y. Ts'o
2019-09-11 16:45           ` Linus Torvalds
2019-09-11 17:00             ` Linus Torvalds
2019-09-11 17:36               ` Theodore Y. Ts'o
2019-09-12  3:44                 ` Ahmed S. Darwish
2019-09-12  8:25                   ` Theodore Y. Ts'o
2019-09-12 11:34                     ` Linus Torvalds
2019-09-12 11:58                       ` Willy Tarreau
2019-09-14 12:25                       ` [PATCH RFC] random: getrandom(2): don't block on non-initialized entropy pool Ahmed S. Darwish
2019-09-14 14:08                         ` Alexander E. Patrakov
2019-09-15  5:22                           ` [PATCH RFC v2] random: optionally block in getrandom(2) when the CRNG is uninitialized Theodore Y. Ts'o
2019-09-15  8:17                             ` [PATCH RFC v3] random: getrandom(2): optionally block when " Ahmed S. Darwish
2019-09-15  8:59                               ` Lennart Poettering
2019-09-15  9:30                                 ` Willy Tarreau
2019-09-15 10:02                                   ` Ahmed S. Darwish
2019-09-15 10:40                                     ` Willy Tarreau
2019-09-15 10:55                                       ` Ahmed S. Darwish
2019-09-15 11:17                                         ` Willy Tarreau
2019-09-15 17:32                             ` [PATCH RFC v2] random: optionally block in getrandom(2) when the " Linus Torvalds
2019-09-15 18:32                               ` Willy Tarreau
2019-09-15 18:36                                 ` Willy Tarreau
2019-09-15 19:08                                   ` Linus Torvalds
2019-09-15 19:18                                     ` Willy Tarreau
2019-09-15 19:31                                       ` Linus Torvalds
2019-09-15 19:54                                         ` Willy Tarreau
2019-09-15 18:59                                 ` Linus Torvalds
2019-09-15 19:12                                   ` Willy Tarreau
2019-09-16  2:45                                   ` Ahmed S. Darwish
2019-09-16 18:08                               ` Lennart Poettering
2019-09-16 19:16                                 ` Willy Tarreau
2019-09-18 21:15                               ` [PATCH RFC v4 0/1] random: WARN on large getrandom() waits and introduce getrandom2() Ahmed S. Darwish
2019-09-18 21:17                                 ` [PATCH RFC v4 1/1] " Ahmed S. Darwish
2019-09-18 23:57                                   ` Linus Torvalds
2019-09-19 14:34                                     ` Theodore Y. Ts'o
2019-09-19 15:20                                       ` Linus Torvalds
2019-09-19 15:50                                         ` Linus Torvalds
2019-09-20 13:13                                           ` Theodore Y. Ts'o
2019-09-19 20:04                                         ` Linus Torvalds
2019-09-19 20:45                                           ` Alexander E. Patrakov
2019-09-19 21:47                                             ` Linus Torvalds
2019-09-19 22:23                                               ` Alexander E. Patrakov
2019-09-19 23:44                                                 ` Alexander E. Patrakov
2019-09-20 13:16                                                 ` Theodore Y. Ts'o
2019-09-23 11:55                                           ` David Laight
2019-09-20 13:08                                         ` Theodore Y. Ts'o
2019-09-20 13:46                                     ` Ahmed S. Darwish
2019-09-20 14:33                                       ` Andy Lutomirski
2019-09-20 16:29                                         ` Linus Torvalds
2019-09-20 17:52                                           ` Andy Lutomirski
2019-09-20 18:09                                             ` Linus Torvalds
2019-09-20 18:16                                               ` Willy Tarreau
2019-09-20 19:12                                               ` Andy Lutomirski
2019-09-20 19:51                                                 ` Linus Torvalds [this message]
2019-09-20 20:11                                                   ` Alexander E. Patrakov
2019-09-20 20:17                                                   ` Matthew Garrett
2019-09-20 20:51                                                   ` Andy Lutomirski
2019-09-20 22:44                                                     ` Linus Torvalds
2019-09-20 23:30                                                       ` Andy Lutomirski
2019-09-21  3:05                                                         ` Willy Tarreau
2019-09-21  6:07                                               ` Florian Weimer
2019-09-23 18:33                                                 ` Andy Lutomirski
2019-09-26 21:11                                                   ` Ahmed S. Darwish
2019-09-20 18:12                                             ` Willy Tarreau
2019-09-20 19:22                                               ` Andy Lutomirski
2019-09-20 19:37                                                 ` Willy Tarreau
2019-09-20 19:52                                                   ` Andy Lutomirski
2019-09-20 20:02                                                 ` Linus Torvalds
2019-09-20 18:15                                             ` Alexander E. Patrakov
2019-09-20 18:29                                               ` Andy Lutomirski
2019-09-20 17:26                                       ` Willy Tarreau
2019-09-20 17:56                                         ` Ahmed S. Darwish
2019-09-26 20:42                                     ` [PATCH v5 0/1] random: getrandom(2): warn on large CRNG waits, introduce new flags Ahmed S. Darwish
2019-09-26 20:44                                       ` [PATCH v5 1/1] " Ahmed S. Darwish
2019-09-26 21:39                                         ` Andy Lutomirski
2019-09-28  9:30                                           ` Ahmed S. Darwish
2019-09-14 15:02                       ` Linux 5.3-rc8 Ahmed S. Darwish
2019-09-14 16:30                         ` Linus Torvalds
2019-09-14 16:35                           ` Alexander E. Patrakov
2019-09-14 16:52                             ` Linus Torvalds
2019-09-14 17:09                               ` Alexander E. Patrakov
2019-09-14 19:19                                 ` Linus Torvalds
2019-09-15  6:56                               ` Lennart Poettering
2019-09-15  7:01                                 ` Willy Tarreau
2019-09-15  7:05                                   ` Lennart Poettering
2019-09-15  7:07                                     ` Willy Tarreau
2019-09-15  8:34                                       ` Lennart Poettering
2019-09-15 17:02                                 ` Linus Torvalds
2019-09-16  3:23                                   ` Theodore Y. Ts'o
2019-09-16  3:40                                     ` Linus Torvalds
2019-09-16  3:56                                       ` Linus Torvalds
2019-09-16 17:00                                       ` Theodore Y. Ts'o
2019-09-16 17:07                                         ` Linus Torvalds
2019-09-14 21:11                           ` Ahmed S. Darwish
2019-09-14 22:05                             ` Martin Steigerwald
2019-09-14 22:24                             ` Theodore Y. Ts'o
2019-09-14 22:32                               ` Linus Torvalds
2019-09-15  1:00                                 ` Theodore Y. Ts'o
2019-09-15  1:10                                   ` Linus Torvalds
2019-09-15  2:05                                     ` Theodore Y. Ts'o
2019-09-15  2:11                                       ` Linus Torvalds
2019-09-15  6:33                                       ` Willy Tarreau
2019-09-15  6:53                                       ` Willy Tarreau
2019-09-15  6:51                           ` Lennart Poettering
2019-09-15  7:27                             ` Ahmed S. Darwish
2019-09-15  8:48                               ` Lennart Poettering
2019-09-15 16:29                             ` Linus Torvalds
2019-09-16  1:40                               ` Ahmed S. Darwish
2019-09-16  1:48                                 ` Vito Caputo
2019-09-16  2:49                                   ` Theodore Y. Ts'o
2019-09-16  4:29                                     ` Willy Tarreau
2019-09-16  5:02                                       ` Linus Torvalds
2019-09-16  6:12                                         ` Willy Tarreau
2019-09-16 16:17                                           ` Linus Torvalds
2019-09-16 17:21                                             ` Theodore Y. Ts'o
2019-09-16 17:44                                               ` Linus Torvalds
2019-09-16 17:55                                                 ` Serge Belyshev
2019-09-16 19:08                                                 ` Willy Tarreau
2019-09-16 23:02                                                 ` Matthew Garrett
2019-09-16 23:05                                                   ` Linus Torvalds
2019-09-16 23:11                                                     ` Matthew Garrett
2019-09-16 23:13                                                       ` Alexander E. Patrakov
2019-09-16 23:15                                                         ` Matthew Garrett
2019-09-16 23:18                                                       ` Linus Torvalds
2019-09-16 23:29                                                         ` Ahmed S. Darwish
2019-09-17  1:05                                                           ` Linus Torvalds
2019-09-17  1:23                                                             ` Matthew Garrett
2019-09-17  1:41                                                               ` Linus Torvalds
2019-09-17  1:46                                                                 ` Matthew Garrett
2019-09-17  5:24                                                                   ` Willy Tarreau
2019-09-17  7:33                                                                     ` Martin Steigerwald
2019-09-17  8:35                                                                       ` Willy Tarreau
2019-09-17  8:44                                                                         ` Martin Steigerwald
2019-09-17 12:11                                                                       ` Theodore Y. Ts'o
2019-09-17 12:30                                                                         ` Ahmed S. Darwish
2019-09-17 12:46                                                                           ` Alexander E. Patrakov
2019-09-17 12:47                                                                           ` Willy Tarreau
2019-09-17 16:08                                                                           ` Lennart Poettering
2019-09-17 16:23                                                                             ` Linus Torvalds
2019-09-17 16:34                                                                               ` Reindl Harald
2019-09-17 17:42                                                                               ` Lennart Poettering
2019-09-17 18:01                                                                                 ` Linus Torvalds
2019-09-17 20:28                                                                                   ` Martin Steigerwald
2019-09-17 20:52                                                                                     ` Ahmed S. Darwish
2019-09-17 21:38                                                                                       ` Martin Steigerwald
2019-09-17 21:52                                                                                         ` Matthew Garrett
2019-09-17 22:10                                                                                           ` Martin Steigerwald
2019-09-18 13:53                                                                                             ` Lennart Poettering
2019-09-19  7:28                                                                                               ` Martin Steigerwald
2019-09-17 23:08                                                                                           ` Linus Torvalds
2019-09-18 13:40                                                                                         ` Lennart Poettering
2019-09-17 20:58                                                                                   ` Linus Torvalds
2019-09-18  9:33                                                                                     ` Rasmus Villemoes
2019-09-18 10:16                                                                                       ` Willy Tarreau
2019-09-18 10:25                                                                                         ` Alexander E. Patrakov
2019-09-18 10:42                                                                                           ` Willy Tarreau
2019-09-18 19:31                                                                                       ` Linus Torvalds
2019-09-18 19:56                                                                                 ` Eric W. Biederman
2019-09-18 20:13                                                                                   ` Linus Torvalds
2019-09-18 20:15                                                                                   ` Alexander E. Patrakov
2019-09-18 20:26                                                                                     ` Linus Torvalds
2019-09-18 22:12                                                                                       ` Willy Tarreau
2019-09-27 13:57                                                                                       ` Lennart Poettering
2019-09-27 15:58                                                                                         ` Linus Torvalds
2019-09-29  9:05                                                                                           ` Lennart Poettering
2019-09-17 13:11                                                                         ` Alexander E. Patrakov
2019-09-17 13:37                                                                           ` Alexander E. Patrakov
2019-09-17 15:57                                                                         ` Lennart Poettering
2019-09-17 16:21                                                                           ` Willy Tarreau
2019-09-17 17:13                                                                             ` Lennart Poettering
2019-09-17 17:29                                                                               ` Willy Tarreau
2019-09-17 20:42                                                                                 ` Martin Steigerwald
2019-09-18 13:38                                                                                 ` Lennart Poettering
2019-09-18 13:59                                                                                   ` Alexander E. Patrakov
2019-09-18 14:50                                                                                     ` Alexander E. Patrakov
2019-09-17 20:36                                                                             ` Martin Steigerwald
2019-09-17 16:27                                                                       ` Linus Torvalds
2019-09-17 16:34                                                                         ` Matthew Garrett
2019-09-17 17:16                                                                           ` Willy Tarreau
2019-09-17 17:20                                                                             ` Matthew Garrett
2019-09-17 17:23                                                                               ` Matthew Garrett
2019-09-17 17:57                                                                               ` Willy Tarreau
2019-09-17 16:58                                                                         ` Alexander E. Patrakov
2019-09-17 17:30                                                                           ` Lennart Poettering
2019-09-17 17:32                                                                             ` Willy Tarreau
2019-09-17 17:41                                                                               ` Alexander E. Patrakov
2019-09-17 17:28                                                                         ` Lennart Poettering
2019-09-17  0:03                                                         ` Matthew Garrett
2019-09-17  0:40                                                         ` Matthew Garrett
2019-09-17  7:15                                                     ` a sane approach to random numbers (was: Re: Linux 5.3-rc8) Martin Steigerwald
2019-09-16 18:00                                               ` Linux 5.3-rc8 Alexander E. Patrakov
2019-09-16 19:53                                               ` Ahmed S. Darwish
2019-09-17 15:32                                               ` Lennart Poettering
2019-09-16  3:31                                 ` Linus Torvalds
2019-09-23 20:49                           ` chaos generating driver was " Pavel Machek
2019-09-14  9:25                     ` Ahmed S. Darwish
2019-09-14 16:27                       ` Theodore Y. Ts'o
2019-09-11 21:41             ` Ahmed S. Darwish
2019-09-11 22:37               ` Ahmed S. Darwish
2019-09-16  3:52           ` Herbert Xu
2019-09-16  4:21             ` Linus Torvalds
2019-09-16  4:53               ` Willy Tarreau
2019-09-10 11:56   ` Theodore Y. Ts'o
2019-09-16 10:33     ` Christoph Hellwig
2019-10-03 21:10   ` Jon Masters
2019-10-03 21:31   ` Jon Masters

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='CAHk-=whJ3kmcZp=Ws+uXnRB9KokG6nXSQCSuBnerG--jkAfP5w@mail.gmail.com' \
    --to=torvalds@linux-foundation.org \
    --cc=darwish.07@gmail.com \
    --cc=ebiederm@xmission.com \
    --cc=linux-api@vger.kernel.org \
    --cc=linux-ext4@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-man@vger.kernel.org \
    --cc=luto@kernel.org \
    --cc=mjg59@srcf.ucam.org \
    --cc=mtk.manpages@gmail.com \
    --cc=mzxreary@0pointer.de \
    --cc=patrakov@gmail.com \
    --cc=tytso@mit.edu \
    --cc=w@1wt.eu \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Linux-ext4 Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-ext4/0 linux-ext4/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-ext4 linux-ext4/ https://lore.kernel.org/linux-ext4 \
		linux-ext4@vger.kernel.org
	public-inbox-index linux-ext4

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-ext4


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git