[PATCH] Update the random(4) documentation towards a more accurate view on /dev/urandom

[PATCH] Update the random(4) documentation towards a more accurate view on /dev/urandom

[Nikos Mavrogiannopoulos]

This documents the "property" of /dev/urandom of being able to serve numbers
prior to pool being initialized, and removes any suggested usages of /dev/random
which are disputable (i.e., one-time pad).
Document the fact /dev/random is only suitable for applications which can afford
indeterminate delays since very few applications can do so.
Smooth the alarming language about a theoretical attack, and mention that its
security depends on the cryptographic primitives used by the kernel, as well
as the total entropy gathered.

Signed-off-by: Nikos Mavrogiannopoulos <nmav-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
---
 man4/random.4 | 32 +++++++++++++++++---------------
 1 file changed, 17 insertions(+), 15 deletions(-)

diff --git a/man4/random.4 b/man4/random.4
index b6fdd8c..27a78c6 100644
--- a/man4/random.4
+++ b/man4/random.4
@@ -38,10 +38,10 @@ number of bits of noise in the entropy pool.
 From this entropy pool random numbers are created.
 .LP
 When read, the \fI/dev/random\fP device will return random bytes
-only within the estimated number of bits of noise in the entropy
+only within the estimated number of bits of fresh noise in the entropy
 pool.
-\fI/dev/random\fP should be suitable for uses that need very
-high quality randomness such as one-time pad or key generation.
+\fI/dev/random\fP is suitable for uses that need very
+high quality randomness, and can afford indeterminate delays.
 When the entropy pool is empty, reads from \fI/dev/random\fP will block
 until additional environmental noise is gathered.
 If
@@ -63,15 +63,20 @@ will be set to
 .LP
 A read from the \fI/dev/urandom\fP device will not block
 waiting for more entropy.
-If there is not sufficient entropy, a pseudorandom number generator is used
-to create the requested bytes.
-As a result, in this case the returned values are theoretically vulnerable to a
-cryptographic attack on the algorithms used by the driver.
-Knowledge of how to do this is not available in the current unclassified
-literature, but it is theoretically possible that such an attack may
-exist.
-If this is a concern in your application, use \fI/dev/random\fP
-instead.
+If the estimated fresh entropy is not sufficient, a pseudorandom number generator is
+used to create the requested bytes. This is typically of no concern to applications,
+except for applications that initialize early at boot time, in systems which cannot
+accumulate sufficient entropy at boot.
+
+As a result, in this case the returned values' security depends on
+the cryptographic primitives used by the driver, and the entropy accumulated
+by the system. An attack on the cryptographic primitives is not available in the
+current unclassified literature, but it is theoretically possible that such an attack
+exists.
+
+If these are a concern in your application, use
+.BR getrandom(2)
+or \fI/dev/random\fP instead.
 .B O_NONBLOCK
 has no effect when opening
 .IR /dev/urandom .
@@ -109,9 +114,6 @@ If you are unsure about whether you should use
 or
 .IR /dev/urandom ,
 then probably you want to use the latter.
-As a general rule,
-.IR /dev/urandom
-should be used for everything except long-lived GPG/SSL/SSH keys.
 
 If a seed file is saved across reboots as recommended below (all major
 Linux distributions have done this since 2000 at least), the output is
-- 
2.5.5







--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] Update the random(4) documentation towards a more accurate view on /dev/urandom

[George Spelvin]

I like the bit about indeterminate delays.

Removing the examples of high quality randomness I'm less fond of;
the whole idea is to inform people who don't quite understand what the
general terms mean.  Good enough for one-time pads is a design goal
of /dev/random.

The whole language about "if not X. then a pseudorandom number generator
is used" is actually pretty confusing now that I read it.  It implies
some sort of mode switch which doen't exist.

The bit about early boot is actually not as much of an issue as you think.
Even /dev/urandom will stall early on boot until a minimum initial seed
(128 bits at present) has been acumulated.  (grep for "urandom_init_wait")

How about something more like (draft, not final edit):

 A read from the \fI/dev/urandom\fP device will not block
 waiting for more entropy.
+If the estimated fresh entropy is not sufficient, a \fI/dev/urandom\fP
+will produce output anyway, relying on the cryptographic primitives in
+the driver's pseudo-random number generator to ensure that the output,
+although correlated with previous output in an information theoretic
+sense (it exceeds the unicity distance), is secure for all practical
+purposes.
+
+(One exception: even \fI/dev/urandom\fP will block during early boot until
+a minimum initial seed is accumulated.
+This is currently 128 bits.
+The message "random: nonblocking pool is initialized" appears in the
+kernel logs when this is complete.)
+
+Although the CPRNG is quite robust, and no attack is available in the
+current unclassified literature, it is theoretically possible that such
+an attack exists.

I don't like the bit about "use /dev/random or getrandom(2)"; while
getrandom(2) should be mentioned in "see also", the equivalent is
"getrandom(..., GRND_RANDOM)".  It's the flag, no the syscall.

Either don't mention it here, or mention the flag.

I strongly dislike the deletion of the "as a general rule" advice.
Specific recommendations are very valuable.
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] Update the random(4) documentation towards a more accurate view on /dev/urandom

[Nikos Mavrogiannopoulos]

On Mon, 2016-04-25 at 11:46 -0400, George Spelvin wrote:
> I like the bit about indeterminate delays.
> 
> Removing the examples of high quality randomness I'm less fond of;
> the whole idea is to inform people who don't quite understand what
> the general terms mean.  Good enough for one-time pads is a design
> goal of /dev/random.

If that's about documenting a design goal I'd prefer to move it out of
the main text for 2 reasons. (a) There is no practical crypto system
using one time pads, thus mentioning it in the main body only creates
confusion (b), one time pad is such a theoretical construction that any
real algorithm wouldn't implement it.

> The whole language about "if not X. then a pseudorandom number
> generator
> is used" is actually pretty confusing now that I read it.  It implies
> some sort of mode switch which doen't exist.
> 
> The bit about early boot is actually not as much of an issue as you
> think.
> Even /dev/urandom will stall early on boot until a minimum initial
> seed
> (128 bits at present) has been acumulated.  (grep for
> "urandom_init_wait")

No it will not. We notice often the keys for sshd being generated
*before* the kernel logs that the random pool has been
initialized.

> How about something more like (draft, not final edit):
> 
>  A read from the \fI/dev/urandom\fP device will not block
>  waiting for more entropy.
> +If the estimated fresh entropy is not sufficient, a
> \fI/dev/urandom\fP
> +will produce output anyway, relying on the cryptographic primitives
> in
> +the driver's pseudo-random number generator to ensure that the
> output,
> +although correlated with previous output in an information theoretic
> +sense (it exceeds the unicity distance), is secure for all practical
> +purposes.

What is the purpose of this text? To whom does it target? I wouldn't
like to get into such details about the device in the manpage, but if
you would like a section studying the theoretical properties of
/dev/urandom I'd again suggest to keep it separate and elaborate. What
is on the text above is certainly not complete analysis and is
certainly not targetting administrators and developers who would like
to understand what this device does.

> I don't like the bit about "use /dev/random or getrandom(2)"; while
> getrandom(2) should be mentioned in "see also", the equivalent is
> "getrandom(..., GRND_RANDOM)".  It's the flag, no the syscall.

It is the syscall. According the description in getrandom(2):
"If the pool has not yet been initialized, then the call blocks,
unless  GRND_RANDOM  is  specified  in flags."

> I strongly dislike the deletion of the "as a general rule" advice.
> Specific recommendations are very valuable.

This advice despite being present for so long, is widely ignored as
/dev/urandom is used unconditionally by all software generating keys
(SSH/SSL), gnupg being the exception.

regards,
Nikos

--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] Update the random(4) documentation towards a more accurate view on /dev/urandom

[George Spelvin]

Nikos Mavrogiannopoulos wrote:
> On Mon, 2016-04-25 at 11:46 -0400, George Spelvin wrote:
>> Removing the examples of high quality randomness I'm less fond of;
>> the whole idea is to inform people who don't quite understand what
>> the general terms mean.  Good enough for one-time pads is a design
>> goal of /dev/random.

> If that's about documenting a design goal I'd prefer to move it out of
> the main text for 2 reasons. (a) There is no practical crypto system
> using one time pads, thus mentioning it in the main body only creates
> confusion (b), one time pad is such a theoretical construction that any
> real algorithm wouldn't implement it.

The original (removed by your patch) line was:

-high quality randomness such as one-time pad or key generation.

It's not the words "one-time pad" I'm attached to, but the specific
examples of when "high-quality randomness" is required.  A big point is
to teach people *how* to use it, and without those examples, when would
anyone think "my application wants low-quality randomness"?

You're right that a one-time pad is impractical, but it's still
a common and familiar pedagogical example, and more importantly
something that a person wondering which to use can see that their
application is NOT.

Your proposed patch *also* deleted the other usage example at the end:

-should be used for everything except long-lived GPG/SSL/SSH keys.

which really reduces the value of the man page as a guide to people
who aren't crypto experts.

>> The bit about early boot is actually not as much of an issue as you
>> think.
>>
>> Even /dev/urandom will stall early on boot until a minimum initial seed
>> (128 bits at present) has been acumulated.  (grep for "urandom_init_wait")

> No it will not. We notice often the keys for sshd being generated
> *before* the kernel logs that the random pool has been
> initialized.

H'm... observation definitely trumps theoretical predictions based on
reading the code.

Is that a documentation problem or a code bug, or something I'm not
understanding properly?  If you look for that symbol in the source
it definitely looks like it's supposed to wait for initial seeding.
And ssh-keygen uses /dev/urandom.

The getrandom(2) man page also documents the block-on-startup behavior.

The driver wakes up the sleeping readers *before* printing
the message.  Is it possible that syslog is just losing the race?

>> How about something more like (draft, not final edit):
>> 
>>  A read from the \fI/dev/urandom\fP device will not block
>>  waiting for more entropy.
>> +If the estimated fresh entropy is not sufficient, a
>> \fI/dev/urandom\fP
>> +will produce output anyway, relying on the cryptographic primitives
>> in
>> +the driver's pseudo-random number generator to ensure that the
>> output,
>> +although correlated with previous output in an information theoretic
>> +sense (it exceeds the unicity distance), is secure for all practical
>> +purposes.

> What is the purpose of this text? To whom does it target?

To replace the text

-If there is not sufficient entropy, a pseudorandom number generator is used
-to create the requested bytes.

or

+If the estimated fresh entropy is not sufficient, a pseudorandom number generator is
+used to create the requested bytes.

with something that doesn't imply a mode switch.

I labelled it "draft" because I wasn't really thrilled with the wording,
myself, but I thought it gave the general idea and wasn't worth massaging
into editorial perfection since it was due to get torn apart anyway.

Can you think of better wording?  I'm all for keeping it simple, but
not at the expense of seriously misleading people.

> I wouldn't like to get into such details about the device in the manpage,
> but if you would like a section studying the theoretical properties of
> /dev/urandom I'd again suggest to keep it separate and elaborate. What
> is on the text above is certainly not complete analysis and is certainly
> not targetting administrators and developers who would like to understand
> what this device does.

A reorganization might indeed be a good way forward; I was examining 
your changes without stepping back and considering the whole forest.

Shall I take a stab at it?

>> I don't like the bit about "use /dev/random or getrandom(2)"; while
>> getrandom(2) should be mentioned in "see also", the equivalent is
>> "getrandom(..., GRND_RANDOM)".   It's the flag, no the syscall.

> It is the syscall. According the description in getrandom(2):
> "If the pool has not yet been initialized, then the call blocks,
> unless GRND_RANDOM is specified in flags."

1. You have a buggy man page.  The corrected one says "If the pool has
   not yet been initialized, then the call blocks, unless GRND_NONBLOCK
   is specified in flags."

2. I stand by what I wrote above.  Without the GRND_RANDOM flag,
   getrandom() access the nonblocking pool (/dev/urandom).

>> I strongly dislike the deletion of the "as a general rule" advice.
>> Specific recommendations are very valuable.

> This advice despite being present for so long, is widely ignored as
> /dev/urandom is used unconditionally by all software generating keys
> (SSH/SSL), gnupg being the exception.

No, it's not being ignored.  The advice isn't "use /dev/random for
SSH keys", but "*don't* use /dev/random for anything *except* SSH
keys".  The "(and maybe not even then)" part is implicit, but much
less of a concern.

The audience is not the authors of ssh, libssl, or gnupg; they know
what they're doing.  The audience is everyone *else*, and I think
specific examples really help there.
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] Update the random(4) documentation towards a more accurate view on /dev/urandom

[Nikos Mavrogiannopoulos]

[-- Attachment #1: Type: text/plain, Size: 809 bytes --]

On Mon, 2016-04-25 at 10:48 +0200, Nikos Mavrogiannopoulos wrote:
> This documents the "property" of /dev/urandom of being able to serve
> numbers
> prior to pool being initialized, and removes any suggested usages of
> /dev/random
> which are disputable (i.e., one-time pad).
> Document the fact /dev/random is only suitable for applications which
> can afford
> indeterminate delays since very few applications can do so.
> Smooth the alarming language about a theoretical attack, and mention
> that its
> security depends on the cryptographic primitives used by the kernel,
> as well
> as the total entropy gathered.

This is an updated patch reflecting the recent discussion in linux-
crypto:

http://www.mail-archive.com/linux-crypto-u79uwXL29TY76Z2rM5mHXA@public.gmane.org/msg20400.html

regards,
Nikos

[-- Attachment #2: 0001-Update-the-random-4-documentation-towards-a-more-acc.patch --]
[-- Type: text/x-patch, Size: 4420 bytes --]

From ed2c270b4c5ba96c262eaf1ac8bb2e24a918e2af Mon Sep 17 00:00:00 2001
From: Nikos Mavrogiannopoulos <nmav@redhat.com>
Date: Thu, 7 Apr 2016 09:08:14 +0200
Subject: [PATCH] Update the random(4) documentation towards a more accurate
 view on /dev/urandom

This documents the "property" of /dev/urandom of being able to serve numbers
prior to pool being initialized, and removes any suggested usages of /dev/random
which are disputable (i.e., one-time pad).
Document the fact /dev/random is a legacy interface and only suitable for
applications which can afford indeterminate delays since very few applications
can do so.

Signed-off-by: Nikos Mavrogiannopoulos <nmav@redhat.com>
---
 man4/random.4 | 51 ++++++++++++++++++++++++++-------------------------
 1 file changed, 26 insertions(+), 25 deletions(-)

diff --git a/man4/random.4 b/man4/random.4
index b6fdd8c..e2b975b 100644
--- a/man4/random.4
+++ b/man4/random.4
@@ -13,8 +13,9 @@
 .\" 2004-04-08, AEB, Improved description of read from /dev/urandom
 .\" 2008-06-20, George Spelvin <linux@horizon.com>,
 .\"             Matt Mackall <mpm@selenic.com>
-.\"     Add a Usage subsection that recommends most users to use
-.\"     /dev/urandom, and emphasizes parsimonious usage of /dev/random.
+.\" 2016-08-01, Nikos Mavrogiannopoulos <nmav@redhat.com>
+.\"     Mention that /dev/random is a legacy interface and removed suggested
+.\"     uses of /dev/random.
 .\"
 .TH RANDOM 4 2015-12-28 "Linux" "Linux Programmer's Manual"
 .SH NAME
@@ -37,11 +38,22 @@ The generator also keeps an estimate of the
 number of bits of noise in the entropy pool.
 From this entropy pool random numbers are created.
 .LP
-When read, the \fI/dev/random\fP device will return random bytes
-only within the estimated number of bits of noise in the entropy
-pool.
-\fI/dev/random\fP should be suitable for uses that need very
-high quality randomness such as one-time pad or key generation.
+When read, the \fI/dev/urandom\fP device return random bytes using a pseudorandom 
+number generator seeded from the entropy pool. That operation is
+non-blocking. When used during early boot time, this device may return
+data prior to the entropy pool being initialization.
+If this is of concern in your application, use
+.BR getrandom(2)
+or \fI/dev/random\fP instead.
+
+.LP
+The \fI/dev/random\fP device is a legacy interface which dates back to
+a time where the cryptographic primitives used in the implementation
+were not widely trusted. It will return random bytes
+only within the estimated number of bits of fresh noise in the entropy
+pool, blocking if necessary.
+\fI/dev/random\fP is suitable for applications that need very
+high quality randomness, and can afford indeterminate delays.
 When the entropy pool is empty, reads from \fI/dev/random\fP will block
 until additional environmental noise is gathered.
 If
@@ -60,18 +72,8 @@ will return -1 and
 .I errno
 will be set to
 .BR EAGAIN .
-.LP
-A read from the \fI/dev/urandom\fP device will not block
-waiting for more entropy.
-If there is not sufficient entropy, a pseudorandom number generator is used
-to create the requested bytes.
-As a result, in this case the returned values are theoretically vulnerable to a
-cryptographic attack on the algorithms used by the driver.
-Knowledge of how to do this is not available in the current unclassified
-literature, but it is theoretically possible that such an attack may
-exist.
-If this is a concern in your application, use \fI/dev/random\fP
-instead.
+
+The flag
 .B O_NONBLOCK
 has no effect when opening
 .IR /dev/urandom .
@@ -82,6 +84,8 @@ for the device
 signals will not be handled until after the requested random bytes
 have been generated.
 
+
+
 Since Linux 3.16,
 .\" commit 79a8468747c5f95ed3d5ce8376a3e82e0c5857fc
 a
@@ -104,14 +108,11 @@ This means that it will impact the contents
 read from both files, but it will not make reads from
 \fI/dev/random\fP faster.
 .SS Usage
-If you are unsure about whether you should use
+The 
 .IR /dev/random
-or
+interface is considered a legacy interface, and 
 .IR /dev/urandom ,
-then probably you want to use the latter.
-As a general rule,
-.IR /dev/urandom
-should be used for everything except long-lived GPG/SSL/SSH keys.
+is recommended for general use.
 
 If a seed file is saved across reboots as recommended below (all major
 Linux distributions have done this since 2000 at least), the output is
-- 
2.7.4

Re: [PATCH] Update the random(4) documentation towards a more accurate view on /dev/urandom

[Nikos Mavrogiannopoulos]

[-- Attachment #1: Type: text/plain, Size: 412 bytes --]

On Mon, 2016-08-01 at 13:48 +0200, Nikos Mavrogiannopoulos wrote:
> This is an updated patch reflecting the recent discussion in linux-
> crypto:
> http://www.mail-archive.com/linux-crypto-u79uwXL29TY76Z2rM5mHXA@public.gmane.org/msg20400.html

Hi,
 I'm resending the patch with few typo fixes, and adding Ted in CC for
review. Ted would you like to review this patch for the random(4)
manpage?

regards,
Nikos

[-- Attachment #2: 0001-Update-the-random-4-documentation-towards-a-more-acc.patch --]
[-- Type: text/x-patch, Size: 4417 bytes --]

From 7952cf6bbabf36a4be50a3ba953278077f7b5157 Mon Sep 17 00:00:00 2001
From: Nikos Mavrogiannopoulos <nmav@redhat.com>
Date: Thu, 7 Apr 2016 09:08:14 +0200
Subject: [PATCH] Update the random(4) documentation towards a more accurate
 view on /dev/urandom

This documents the "property" of /dev/urandom of being able to serve numbers
prior to pool being initialized, and removes any suggested usages of /dev/random
which are disputable (i.e., one-time pad).
Document the fact /dev/random is a legacy interface and only suitable for
applications which can afford indeterminate delays since very few applications
can do so.

Signed-off-by: Nikos Mavrogiannopoulos <nmav@redhat.com>
---
 man4/random.4 | 51 ++++++++++++++++++++++++++-------------------------
 1 file changed, 26 insertions(+), 25 deletions(-)

diff --git a/man4/random.4 b/man4/random.4
index b67c46f..c7afaf4 100644
--- a/man4/random.4
+++ b/man4/random.4
@@ -13,8 +13,9 @@
 .\" 2004-04-08, AEB, Improved description of read from /dev/urandom
 .\" 2008-06-20, George Spelvin <linux@horizon.com>,
 .\"             Matt Mackall <mpm@selenic.com>
-.\"     Add a Usage subsection that recommends most users to use
-.\"     /dev/urandom, and emphasizes parsimonious usage of /dev/random.
+.\" 2016-10-20, Nikos Mavrogiannopoulos <nmav@redhat.com>
+.\"     Mention that /dev/random is a legacy interface and removed suggested
+.\"     uses of /dev/random.
 .\"
 .TH RANDOM 4 2016-10-08 "Linux" "Linux Programmer's Manual"
 .SH NAME
@@ -37,11 +38,22 @@ The generator also keeps an estimate of the
 number of bits of noise in the entropy pool.
 From this entropy pool random numbers are created.
 .LP
-When read, the \fI/dev/random\fP device will return random bytes
-only within the estimated number of bits of noise in the entropy
-pool.
-\fI/dev/random\fP should be suitable for uses that need very
-high quality randomness such as one-time pad or key generation.
+When read, the \fI/dev/urandom\fP device return random bytes using a pseudorandom 
+number generator seeded from the entropy pool. That operation is
+non-blocking. When used during early boot time, this device may return
+data prior to the entropy pool being initialized.
+If this is of concern in your application, use
+.BR getrandom(2)
+or \fI/dev/random\fP instead.
+
+.LP
+The \fI/dev/random\fP device is a legacy interface which dates back to
+a time where the cryptographic primitives used in the implementation
+were not widely trusted. It will return random bytes
+only within the estimated number of bits of fresh noise in the entropy
+pool, blocking if necessary.
+\fI/dev/random\fP is suitable for applications that need very
+high quality randomness, and can afford indeterminate delays.
 When the entropy pool is empty, reads from \fI/dev/random\fP will block
 until additional environmental noise is gathered.
 If
@@ -60,18 +72,8 @@ will return -1 and
 .I errno
 will be set to
 .BR EAGAIN .
-.LP
-A read from the \fI/dev/urandom\fP device will not block
-waiting for more entropy.
-If there is not sufficient entropy, a pseudorandom number generator is used
-to create the requested bytes.
-As a result, in this case the returned values are theoretically vulnerable to a
-cryptographic attack on the algorithms used by the driver.
-Knowledge of how to do this is not available in the current unclassified
-literature, but it is theoretically possible that such an attack may
-exist.
-If this is a concern in your application, use \fI/dev/random\fP
-instead.
+
+The flag
 .B O_NONBLOCK
 has no effect when opening
 .IR /dev/urandom .
@@ -82,6 +84,8 @@ for the device
 signals will not be handled until after the requested random bytes
 have been generated.
 
+
+
 Since Linux 3.16,
 .\" commit 79a8468747c5f95ed3d5ce8376a3e82e0c5857fc
 a
@@ -104,14 +108,11 @@ This means that it will impact the contents
 read from both files, but it will not make reads from
 \fI/dev/random\fP faster.
 .SS Usage
-If you are unsure about whether you should use
+The 
 .IR /dev/random
-or
+interface is considered a legacy interface, and 
 .IR /dev/urandom ,
-then probably you want to use the latter.
-As a general rule,
-.IR /dev/urandom
-should be used for everything except long-lived GPG/SSL/SSH keys.
+is recommended for general use.
 
 If a seed file is saved across reboots as recommended below (all major
 Linux distributions have done this since 2000 at least), the output is
-- 
2.7.4

Re: [PATCH] Update the random(4) documentation towards a more accurate view on /dev/urandom

[Michael Kerrisk (man-pages)]

[Dropping a couple of other people into CC who may be able / willing
to review.]

Folks, I'd really appreciate some review help here!

Cheers,

Michael

On 10/20/2016 10:37 AM, Nikos Mavrogiannopoulos wrote:
> On Mon, 2016-08-01 at 13:48 +0200, Nikos Mavrogiannopoulos wrote:
>> > This is an updated patch reflecting the recent discussion in linux-
>> > crypto:
>> > http://www.mail-archive.com/linux-crypto-u79uwXL29TY76Z2rM5mHXA@public.gmane.org/msg20400.html
> Hi,
>  I'm resending the patch with few typo fixes, and adding Ted in CC for
> review. Ted would you like to review this patch for the random(4)
> manpage?
> 
> regards,
> Nikos
> 
> 
> 0001-Update-the-random-4-documentation-towards-a-more-acc.patch
> 
> 
> From 7952cf6bbabf36a4be50a3ba953278077f7b5157 Mon Sep 17 00:00:00 2001
> From: Nikos Mavrogiannopoulos <nmav-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
> Date: Thu, 7 Apr 2016 09:08:14 +0200
> Subject: [PATCH] Update the random(4) documentation towards a more accurate
>  view on /dev/urandom
> 
> This documents the "property" of /dev/urandom of being able to serve numbers
> prior to pool being initialized, and removes any suggested usages of /dev/random
> which are disputable (i.e., one-time pad).
> Document the fact /dev/random is a legacy interface and only suitable for
> applications which can afford indeterminate delays since very few applications
> can do so.
> 
> Signed-off-by: Nikos Mavrogiannopoulos <nmav-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
> ---
>  man4/random.4 | 51 ++++++++++++++++++++++++++-------------------------
>  1 file changed, 26 insertions(+), 25 deletions(-)
> 
> diff --git a/man4/random.4 b/man4/random.4
> index b67c46f..c7afaf4 100644
> --- a/man4/random.4
> +++ b/man4/random.4
> @@ -13,8 +13,9 @@
>  .\" 2004-04-08, AEB, Improved description of read from /dev/urandom
>  .\" 2008-06-20, George Spelvin <linux-gpGsJRJZ3PBBDgjK7y7TUQ@public.gmane.org>,
>  .\"             Matt Mackall <mpm-VDJrAJ4Gl5ZBDgjK7y7TUQ@public.gmane.org>
> -.\"     Add a Usage subsection that recommends most users to use
> -.\"     /dev/urandom, and emphasizes parsimonious usage of /dev/random.
> +.\" 2016-10-20, Nikos Mavrogiannopoulos <nmav-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
> +.\"     Mention that /dev/random is a legacy interface and removed suggested
> +.\"     uses of /dev/random.
>  .\"
>  .TH RANDOM 4 2016-10-08 "Linux" "Linux Programmer's Manual"
>  .SH NAME
> @@ -37,11 +38,22 @@ The generator also keeps an estimate of the
>  number of bits of noise in the entropy pool.
>  From this entropy pool random numbers are created.
>  .LP
> -When read, the \fI/dev/random\fP device will return random bytes
> -only within the estimated number of bits of noise in the entropy
> -pool.
> -\fI/dev/random\fP should be suitable for uses that need very
> -high quality randomness such as one-time pad or key generation.
> +When read, the \fI/dev/urandom\fP device return random bytes using a pseudorandom 
> +number generator seeded from the entropy pool. That operation is
> +non-blocking. When used during early boot time, this device may return
> +data prior to the entropy pool being initialized.
> +If this is of concern in your application, use
> +.BR getrandom(2)
> +or \fI/dev/random\fP instead.
> +
> +.LP
> +The \fI/dev/random\fP device is a legacy interface which dates back to
> +a time where the cryptographic primitives used in the implementation
> +were not widely trusted. It will return random bytes
> +only within the estimated number of bits of fresh noise in the entropy
> +pool, blocking if necessary.
> +\fI/dev/random\fP is suitable for applications that need very
> +high quality randomness, and can afford indeterminate delays.
>  When the entropy pool is empty, reads from \fI/dev/random\fP will block
>  until additional environmental noise is gathered.
>  If
> @@ -60,18 +72,8 @@ will return -1 and
>  .I errno
>  will be set to
>  .BR EAGAIN .
> -.LP
> -A read from the \fI/dev/urandom\fP device will not block
> -waiting for more entropy.
> -If there is not sufficient entropy, a pseudorandom number generator is used
> -to create the requested bytes.
> -As a result, in this case the returned values are theoretically vulnerable to a
> -cryptographic attack on the algorithms used by the driver.
> -Knowledge of how to do this is not available in the current unclassified
> -literature, but it is theoretically possible that such an attack may
> -exist.
> -If this is a concern in your application, use \fI/dev/random\fP
> -instead.
> +
> +The flag
>  .B O_NONBLOCK
>  has no effect when opening
>  .IR /dev/urandom .
> @@ -82,6 +84,8 @@ for the device
>  signals will not be handled until after the requested random bytes
>  have been generated.
>  
> +
> +
>  Since Linux 3.16,
>  .\" commit 79a8468747c5f95ed3d5ce8376a3e82e0c5857fc
>  a
> @@ -104,14 +108,11 @@ This means that it will impact the contents
>  read from both files, but it will not make reads from
>  \fI/dev/random\fP faster.
>  .SS Usage
> -If you are unsure about whether you should use
> +The 
>  .IR /dev/random
> -or
> +interface is considered a legacy interface, and 
>  .IR /dev/urandom ,
> -then probably you want to use the latter.
> -As a general rule,
> -.IR /dev/urandom
> -should be used for everything except long-lived GPG/SSL/SSH keys.
> +is recommended for general use.
>  
>  If a seed file is saved across reboots as recommended below (all major
>  Linux distributions have done this since 2000 at least), the output is
> -- 2.7.4
> 


-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] Update the random(4) documentation towards a more accurate view on /dev/urandom

[Stephan Mueller]

Am Freitag, 21. Oktober 2016, 09:21:28 CEST schrieb Michael Kerrisk (man-
pages):

Hi Michael,

> [Dropping a couple of other people into CC who may be able / willing
> to review.]
> 
> Folks, I'd really appreciate some review help here!
> 
> Cheers,
> 
> Michael
> 
> On 10/20/2016 10:37 AM, Nikos Mavrogiannopoulos wrote:
> > On Mon, 2016-08-01 at 13:48 +0200, Nikos Mavrogiannopoulos wrote:
> >> > This is an updated patch reflecting the recent discussion in linux-
> >> > crypto:
> >> > http://www.mail-archive.com/linux-crypto-u79uwXL29TY76Z2rM5mHXA@public.gmane.org/msg20400.html
> > 
> > Hi,
> > 
> >  I'm resending the patch with few typo fixes, and adding Ted in CC for
> > 
> > review. Ted would you like to review this patch for the random(4)
> > manpage?
> > 
> > regards,
> > Nikos
> > 
> > 
> > 0001-Update-the-random-4-documentation-towards-a-more-acc.patch
> > 
> > 
> > From 7952cf6bbabf36a4be50a3ba953278077f7b5157 Mon Sep 17 00:00:00 2001
> > From: Nikos Mavrogiannopoulos <nmav-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
> > Date: Thu, 7 Apr 2016 09:08:14 +0200
> > Subject: [PATCH] Update the random(4) documentation towards a more
> > accurate
> > 
> >  view on /dev/urandom
> > 
> > This documents the "property" of /dev/urandom of being able to serve
> > numbers prior to pool being initialized, and removes any suggested usages
> > of /dev/random which are disputable (i.e., one-time pad).
> > Document the fact /dev/random is a legacy interface and only suitable for
> > applications which can afford indeterminate delays since very few
> > applications can do so.
> > 
> > Signed-off-by: Nikos Mavrogiannopoulos <nmav-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
> > ---
> > 
> >  man4/random.4 | 51 ++++++++++++++++++++++++++-------------------------
> >  1 file changed, 26 insertions(+), 25 deletions(-)
> > 
> > diff --git a/man4/random.4 b/man4/random.4
> > index b67c46f..c7afaf4 100644
> > --- a/man4/random.4
> > +++ b/man4/random.4
> > @@ -13,8 +13,9 @@
> > 
> >  .\" 2004-04-08, AEB, Improved description of read from /dev/urandom
> >  .\" 2008-06-20, George Spelvin <linux-gpGsJRJZ3PBBDgjK7y7TUQ@public.gmane.org>,
> >  .\"             Matt Mackall <mpm-VDJrAJ4Gl5ZBDgjK7y7TUQ@public.gmane.org>
> > 
> > -.\"     Add a Usage subsection that recommends most users to use
> > -.\"     /dev/urandom, and emphasizes parsimonious usage of /dev/random.
> > +.\" 2016-10-20, Nikos Mavrogiannopoulos <nmav-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
> > +.\"     Mention that /dev/random is a legacy interface and removed
> > suggested +.\"     uses of /dev/random.
> > 
> >  .\"
> >  .TH RANDOM 4 2016-10-08 "Linux" "Linux Programmer's Manual"
> >  .SH NAME
> > 
> > @@ -37,11 +38,22 @@ The generator also keeps an estimate of the
> > 
> >  number of bits of noise in the entropy pool.
> >  From this entropy pool random numbers are created.
> >  .LP
> > 
> > -When read, the \fI/dev/random\fP device will return random bytes
> > -only within the estimated number of bits of noise in the entropy
> > -pool.
> > -\fI/dev/random\fP should be suitable for uses that need very
> > -high quality randomness such as one-time pad or key generation.
> > +When read, the \fI/dev/urandom\fP device return random bytes using a
> > pseudorandom +number generator seeded from the entropy pool. That

Starting with 4.8, there is no nonblocking_pool any more. Please refer to the 
ChaCha20 DRNG.

> > operation is
> > +non-blocking. When used during early boot time, this device may return
> > +data prior to the entropy pool being initialized.
> > +If this is of concern in your application, use
> > +.BR getrandom(2)
> > +or \fI/dev/random\fP instead.
> > +
> > +.LP
> > +The \fI/dev/random\fP device is a legacy interface which dates back to
> > +a time where the cryptographic primitives used in the implementation
> > +were not widely trusted. It will return random bytes
> > +only within the estimated number of bits of fresh noise in the entropy
> > +pool, blocking if necessary.
> > +\fI/dev/random\fP is suitable for applications that need very
> > +high quality randomness, and can afford indeterminate delays.

Would it be possible to add something around getrandom stating that it blocks 
until initially 128 bits of entropy are measured before it unblocks and 
behaves like /dev/urandom? Maybe it makes even sense to add a recommendation 
to use getrandom in favor of /dev/urandom?
> > 
> >  When the entropy pool is empty, reads from \fI/dev/random\fP will block
> >  until additional environmental noise is gathered.
> >  If
> > 
> > @@ -60,18 +72,8 @@ will return -1 and
> > 
> >  .I errno
> >  will be set to
> >  .BR EAGAIN .
> > 
> > -.LP
> > -A read from the \fI/dev/urandom\fP device will not block
> > -waiting for more entropy.
> > -If there is not sufficient entropy, a pseudorandom number generator is
> > used -to create the requested bytes.
> > -As a result, in this case the returned values are theoretically
> > vulnerable to a -cryptographic attack on the algorithms used by the
> > driver.
> > -Knowledge of how to do this is not available in the current unclassified
> > -literature, but it is theoretically possible that such an attack may
> > -exist.
> > -If this is a concern in your application, use \fI/dev/random\fP
> > -instead.
> > +
> > +The flag
> > 
> >  .B O_NONBLOCK
> >  has no effect when opening
> >  .IR /dev/urandom .
> > 
> > @@ -82,6 +84,8 @@ for the device
> > 
> >  signals will not be handled until after the requested random bytes
> >  have been generated.
> > 
> > +
> > +
> > 
> >  Since Linux 3.16,
> >  .\" commit 79a8468747c5f95ed3d5ce8376a3e82e0c5857fc
> >  a
> > 
> > @@ -104,14 +108,11 @@ This means that it will impact the contents
> > 
> >  read from both files, but it will not make reads from
> >  \fI/dev/random\fP faster.
> >  .SS Usage
> > 
> > -If you are unsure about whether you should use
> > +The
> > 
> >  .IR /dev/random
> > 
> > -or
> > +interface is considered a legacy interface, and
> > 
> >  .IR /dev/urandom ,
> > 
> > -then probably you want to use the latter.
> > -As a general rule,
> > -.IR /dev/urandom
> > -should be used for everything except long-lived GPG/SSL/SSH keys.
> > +is recommended for general use.
> > 
> >  If a seed file is saved across reboots as recommended below (all major
> >  Linux distributions have done this since 2000 at least), the output is
> > 
> > -- 2.7.4



Ciao
Stephan

--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] Update the random(4) documentation towards a more accurate view on /dev/urandom

[Nikos Mavrogiannopoulos]

On Fri, 2016-10-21 at 16:07 +0200, Stephan Mueller wrote:

> > > -When read, the \fI/dev/random\fP device will return random bytes
> > > -only within the estimated number of bits of noise in the entropy
> > > -pool.
> > > -\fI/dev/random\fP should be suitable for uses that need very
> > > -high quality randomness such as one-time pad or key generation.
> > > +When read, the \fI/dev/urandom\fP device return random bytes
> > > using a
> > > pseudorandom +number generator seeded from the entropy pool. That
> 
> Starting with 4.8, there is no nonblocking_pool any more. Please
> refer to the ChaCha20 DRNG.

Hi Stephan,
 I am not sure the suggestion above is clear to me. The text above (nor
the rest of the manpage) doesn't mention details about non-
blocking/blocking pools. I intentionally left such details out as they
do not provide information to a reader who is not familiar with the
actual implementation behind it. The CHACHA20 DRNG is another detail
that I wouldn't like the manpage to mention since it is a technical
detail and may even change in the future (e.g., to a faster stream
cipher).

Nevertheless, I find this suggestion orthogonal to my text above. There
may be another update of the manpage to add these details (even though
I wouldn't really like it).

> > +.LP
> > > +The \fI/dev/random\fP device is a legacy interface which dates
> > > back to
> > > +a time where the cryptographic primitives used in the
> > > implementation
> > > +were not widely trusted. It will return random bytes
> > > +only within the estimated number of bits of fresh noise in the
> > > entropy
> > > +pool, blocking if necessary.
> > > +\fI/dev/random\fP is suitable for applications that need very
> > > +high quality randomness, and can afford indeterminate delays.
> 
> Would it be possible to add something around getrandom stating that
> it blocks 
> until initially 128 bits of entropy are measured before it unblocks
> and 
> behaves like /dev/urandom? Maybe it makes even sense to add a
> recommendation 
> to use getrandom in favor of /dev/urandom?

Note that this is the manpage on /dev/urandom and /dev/random only.
getrandom() has a separate manpage which lists (or should list) such
information. Said that, indeed this suggestion is right, but I think
this recommendation is mildly already there (the quotes may hide it). It is on the 3rd paragraph:
"When used during early boot time,       this device may return data
prior to the entropy pool being initialized.  If this is of
concern       in your application, use getrandom(2) or /dev/random
instead."

regards,
Nikos

--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] Update the random(4) documentation towards a more accurate view on /dev/urandom

[Stephan Mueller]

Am Freitag, 21. Oktober 2016, 16:38:30 CEST schrieb Nikos Mavrogiannopoulos:

Hi Nikos,

> On Fri, 2016-10-21 at 16:07 +0200, Stephan Mueller wrote:
> > > > -When read, the \fI/dev/random\fP device will return random bytes
> > > > -only within the estimated number of bits of noise in the entropy
> > > > -pool.
> > > > -\fI/dev/random\fP should be suitable for uses that need very
> > > > -high quality randomness such as one-time pad or key generation.
> > > > +When read, the \fI/dev/urandom\fP device return random bytes
> > > > using a
> > > > pseudorandom +number generator seeded from the entropy pool. That
> > 
> > Starting with 4.8, there is no nonblocking_pool any more. Please
> > refer to the ChaCha20 DRNG.
> 
> Hi Stephan,
>  I am not sure the suggestion above is clear to me. The text above (nor
> the rest of the manpage) doesn't mention details about non-
> blocking/blocking pools. I intentionally left such details out as they
> do not provide information to a reader who is not familiar with the
> actual implementation behind it. The CHACHA20 DRNG is another detail
> that I wouldn't like the manpage to mention since it is a technical
> detail and may even change in the future (e.g., to a faster stream
> cipher).

Thanks for the clarification. It just occurred to me due to the use of the 
term "entropy pool". IMHO we cannot speak about such pool any more for the 
component behind /dev/urandom as this now is a real DRNG.

But if you feel that these are too many details, please disregard my comments. 
Then, I see your changes are good.
> 
> Nevertheless, I find this suggestion orthogonal to my text above. There
> may be another update of the manpage to add these details (even though
> I wouldn't really like it).
> 
> > > +.LP
> > > 
> > > > +The \fI/dev/random\fP device is a legacy interface which dates
> > > > back to
> > > > +a time where the cryptographic primitives used in the
> > > > implementation
> > > > +were not widely trusted. It will return random bytes
> > > > +only within the estimated number of bits of fresh noise in the
> > > > entropy
> > > > +pool, blocking if necessary.
> > > > +\fI/dev/random\fP is suitable for applications that need very
> > > > +high quality randomness, and can afford indeterminate delays.
> > 
> > Would it be possible to add something around getrandom stating that
> > it blocks 
> > until initially 128 bits of entropy are measured before it unblocks
> > and 
> > behaves like /dev/urandom? Maybe it makes even sense to add a
> > recommendation 
> > to use getrandom in favor of /dev/urandom?
> 
> Note that this is the manpage on /dev/urandom and /dev/random only.
> getrandom() has a separate manpage which lists (or should list) such
> information. Said that, indeed this suggestion is right, but I think
> this recommendation is mildly already there (the quotes may hide it). It is
> on the 3rd paragraph: "When used during early boot time,       this device
> may return data prior to the entropy pool being initialized.  If this is of
> concern       in your application, use getrandom(2) or /dev/random
> instead."

Ok, I see. If you feel that my suggestion is already covered to the extent 
needed, I would be fine with the original patch here too.

Ciao
Stephan
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] Update the random(4) documentation towards a more accurate view on /dev/urandom

[Theodore Ts'o]

On Fri, Oct 21, 2016 at 04:07:05PM +0200, Stephan Mueller wrote:
> > > -When read, the \fI/dev/random\fP device will return random bytes
> > > -only within the estimated number of bits of noise in the entropy
> > > -pool.
> > > -\fI/dev/random\fP should be suitable for uses that need very
> > > -high quality randomness such as one-time pad or key generation.
> > > +When read, the \fI/dev/urandom\fP device return random bytes using a
> > > pseudorandom +number generator seeded from the entropy pool. That
> 
> Starting with 4.8, there is no nonblocking_pool any more. Please refer to the 
> ChaCha20 DRNG.

That's true, although I think the suggested text is fine.  It doesn't
reference the nonblocking_ool as near as I can tell.

> Would it be possible to add something around getrandom stating that it blocks 
> until initially 128 bits of entropy are measured before it unblocks and 
> behaves like /dev/urandom? Maybe it makes even sense to add a recommendation 
> to use getrandom in favor of /dev/urandom?

It would certainly be a good idea to suggest the use of getrandom(2),
and the fact that the only difference between getrandom(2) and
/dev/urandom is that getrandom(2) will block until it can safely
generate random numbers.  Unfortunately, there are far too many
programs (including udev and systemd!) that try to use /dev/urandom at
boot time, and making a change to /dev/urandom would break users.  If
any of these use cases are security sensitive, they should really
change it so that users aren't vulnerable to security attacks.  (The
use of systemd on IOT devices especially terrifies me in this regard.)

So anytihng we can do to at least nudge man page readers that they
shouldn't be trying to use any of these interfaces during the boot
sequence would be a Really Good Thing.

						- Ted
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] Update the random(4) documentation towards a more accurate view on /dev/urandom

[Stephan Mueller]

Am Freitag, 21. Oktober 2016, 12:33:14 CEST schrieb Theodore Ts'o:

Hi Theodore,

> 
> It would certainly be a good idea to suggest the use of getrandom(2),
> and the fact that the only difference between getrandom(2) and
> /dev/urandom is that getrandom(2) will block until it can safely
> generate random numbers.  Unfortunately, there are far too many
> programs (including udev and systemd!) that try to use /dev/urandom at
> boot time, and making a change to /dev/urandom would break users.  If
> any of these use cases are security sensitive, they should really
> change it so that users aren't vulnerable to security attacks.  (The
> use of systemd on IOT devices especially terrifies me in this regard.)

Although I this is not related to the man page, please note that I have tried 
this approach during the development of my LRNG. systemd did not like that at 
all. The system did not come up, even though there was only 0.5 seconds it 
would need to wait until getrandom would unblock during my tests.

Ciao
Stephan
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] Update the random(4) documentation towards a more accurate view on /dev/urandom

[Theodore Ts'o]

On Fri, Oct 21, 2016 at 06:50:58PM +0200, Stephan Mueller wrote:
> > It would certainly be a good idea to suggest the use of getrandom(2),
> > and the fact that the only difference between getrandom(2) and
> > /dev/urandom is that getrandom(2) will block until it can safely
> > generate random numbers.  Unfortunately, there are far too many
> > programs (including udev and systemd!) that try to use /dev/urandom at
> > boot time, and making a change to /dev/urandom would break users.  If
> > any of these use cases are security sensitive, they should really
> > change it so that users aren't vulnerable to security attacks.  (The
> > use of systemd on IOT devices especially terrifies me in this regard.)
> 
> Although I this is not related to the man page, please note that I have tried 
> this approach during the development of my LRNG. systemd did not like that at 
> all. The system did not come up, even though there was only 0.5 seconds it 
> would need to wait until getrandom would unblock during my tests.

Although I'd argue this is a systemd bug, to be fair to systemd, when
I briefly tried this approach, the zero day kernel testing system, let
me know than an older version of Ubuntu (which wasn't using systemd),
and CeroWrt (which also doesn't use systemd) were also hanging.

I haven't checked to see if that was because Cerowrt was trying to do
something security sensitive such as creating ssh host keys.  I've
been too busy, and I was afriad the result would be too depressing....
Besides, I don't use Cerowrt thes days, since I'm using Google OnHub
now.  :-)

     	       	     	 	    	   - Ted
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] Update the random(4) documentation towards a more accurate view on /dev/urandom

[Stephan Mueller]

Am Freitag, 21. Oktober 2016, 13:56:33 CEST schrieb Theodore Ts'o:

Hi Theodore,

> > Although I this is not related to the man page, please note that I have
> > tried this approach during the development of my LRNG. systemd did not
> > like that at all. The system did not come up, even though there was only
> > 0.5 seconds it would need to wait until getrandom would unblock during my
> > tests.
> 
> Although I'd argue this is a systemd bug, to be fair to systemd, when
> I briefly tried this approach, the zero day kernel testing system, let
> me know than an older version of Ubuntu (which wasn't using systemd),
> and CeroWrt (which also doesn't use systemd) were also hanging.
> 
> I haven't checked to see if that was because Cerowrt was trying to do
> something security sensitive such as creating ssh host keys.  I've
> been too busy, and I was afriad the result would be too depressing....
> Besides, I don't use Cerowrt thes days, since I'm using Google OnHub
> now.  :-)

I will dig into that matter a bit further as I need an answer for this 
behavior for other purposes.

When I find the cause, I will report it.

Ciao
Stephan
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] Update the random(4) documentation towards a more accurate view on /dev/urandom

[Nikos Mavrogiannopoulos]

On Fri, 2016-10-21 at 09:21 +0200, Michael Kerrisk (man-pages) wrote:
> [Dropping a couple of other people into CC who may be able / willing
> to review.]
> 
> Folks, I'd really appreciate some review help here!

Hi guys,
 If I understand correctly Ted Ts'o, and Stephan Mueller agree with the
updated text. Do we need addition review someone else? Sorry for
pressing here, but I believe that the update on the text is quite
important to be ignored.

regards,
Nikos

--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] Update the random(4) documentation towards a more accurate view on /dev/urandom

[Michael Kerrisk (man-pages)]

Hello Nikos,

On 11/01/2016 10:35 AM, Nikos Mavrogiannopoulos wrote:
> On Fri, 2016-10-21 at 09:21 +0200, Michael Kerrisk (man-pages) wrote:
>> [Dropping a couple of other people into CC who may be able / willing
>> to review.]
>>
>> Folks, I'd really appreciate some review help here!
> 
> Hi guys,
>  If I understand correctly Ted Ts'o, and Stephan Mueller agree with the
> updated text. Do we need addition review someone else? Sorry for
> pressing here, but I believe that the update on the text is quite
> important to be ignored.

I'll reply to your other mail shortly.

Cheers,

Michael



-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] Update the random(4) documentation towards a more accurate view on /dev/urandom

[Michael Kerrisk (man-pages)]

[I'm looping a few people into this mail who previously commented
on this page. Nikos, I will also thread you into an earlier mail 
by Laurent Georget.]

Hello Nikos,

Sorry that I have been so slow to follow up on this.
Thanks for your persistence. I have some comments 
that probably require some tweaks to your patch.
I also have some questions about a couple of other 
earlier discussions.

On 10/20/2016 10:37 AM, Nikos Mavrogiannopoulos wrote:
> On Mon, 2016-08-01 at 13:48 +0200, Nikos Mavrogiannopoulos wrote:
>> > This is an updated patch reflecting the recent discussion in linux-
>> > crypto:
>> > http://www.mail-archive.com/linux-crypto-u79uwXL29TY76Z2rM5mHXA@public.gmane.org/msg20400.html
> Hi,
>  I'm resending the patch with few typo fixes, and adding Ted in CC for
> review. Ted would you like to review this patch for the random(4)
> manpage?

Some comments below.

But one question first. You didn't further reply to George 
Spelvin's comments on 26 April. Did you consider those comments
irrelevant or already addressed or something else?

By the way, inline patches are rather easier for me to deal with.

> 0001-Update-the-random-4-documentation-towards-a-more-acc.patch
> 
> 
> From 7952cf6bbabf36a4be50a3ba953278077f7b5157 Mon Sep 17 00:00:00 2001
> From: Nikos Mavrogiannopoulos <nmav-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
> Date: Thu, 7 Apr 2016 09:08:14 +0200
> Subject: [PATCH] Update the random(4) documentation towards a more accurate
>  view on /dev/urandom
> 
> This documents the "property" of /dev/urandom of being able to serve numbers
> prior to pool being initialized, and removes any suggested usages of /dev/random
> which are disputable (i.e., one-time pad).
> Document the fact /dev/random is a legacy interface and only suitable for
> applications which can afford indeterminate delays since very few applications
> can do so.
> 
> Signed-off-by: Nikos Mavrogiannopoulos <nmav-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
> ---
>  man4/random.4 | 51 ++++++++++++++++++++++++++-------------------------
>  1 file changed, 26 insertions(+), 25 deletions(-)
> 
> diff --git a/man4/random.4 b/man4/random.4
> index b67c46f..c7afaf4 100644
> --- a/man4/random.4
> +++ b/man4/random.4
> @@ -13,8 +13,9 @@
>  .\" 2004-04-08, AEB, Improved description of read from /dev/urandom
>  .\" 2008-06-20, George Spelvin <linux-gpGsJRJZ3PBBDgjK7y7TUQ@public.gmane.org>,
>  .\"             Matt Mackall <mpm-VDJrAJ4Gl5ZBDgjK7y7TUQ@public.gmane.org>
> -.\"     Add a Usage subsection that recommends most users to use
> -.\"     /dev/urandom, and emphasizes parsimonious usage of /dev/random.
> +.\" 2016-10-20, Nikos Mavrogiannopoulos <nmav-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
> +.\"     Mention that /dev/random is a legacy interface and removed suggested
> +.\"     uses of /dev/random.

No need to update this in-page changelog. We use git these days.

>  .\"
>  .TH RANDOM 4 2016-10-08 "Linux" "Linux Programmer's Manual"
>  .SH NAME
> @@ -37,11 +38,22 @@ The generator also keeps an estimate of the
>  number of bits of noise in the entropy pool.
>  From this entropy pool random numbers are created.
>  .LP
> -When read, the \fI/dev/random\fP device will return random bytes
> -only within the estimated number of bits of noise in the entropy
> -pool.
> -\fI/dev/random\fP should be suitable for uses that need very
> -high quality randomness such as one-time pad or key generation.
> +When read, the \fI/dev/urandom\fP device return random bytes using a pseudorandom 
> +number generator seeded from the entropy pool. That operation is
> +non-blocking. When used during early boot time, this device may return
> +data prior to the entropy pool being initialized.
> +If this is of concern in your application, use
> +.BR getrandom(2)
> +or \fI/dev/random\fP instead.
> +
> +.LP
> +The \fI/dev/random\fP device is a legacy interface which dates back to
> +a time where the cryptographic primitives used in the implementation

s%in the implementation%in the implementation of /dev/urandom% ?

If that's not correct, can you please say more precisely
what "implementation" you are referring to.

> +were not widely trusted. It will return random bytes
> +only within the estimated number of bits of fresh noise in the entropy
> +pool, blocking if necessary.
> +\fI/dev/random\fP is suitable for applications that need very
> +high quality randomness, and can afford indeterminate delays.
>  When the entropy pool is empty, reads from \fI/dev/random\fP will block
>  until additional environmental noise is gathered.
>  If
> @@ -60,18 +72,8 @@ will return -1 and
>  .I errno
>  will be set to
>  .BR EAGAIN .
> -.LP
> -A read from the \fI/dev/urandom\fP device will not block
> -waiting for more entropy.
> -If there is not sufficient entropy, a pseudorandom number generator is used
> -to create the requested bytes.
> -As a result, in this case the returned values are theoretically vulnerable to a
> -cryptographic attack on the algorithms used by the driver.
> -Knowledge of how to do this is not available in the current unclassified
> -literature, but it is theoretically possible that such an attack may
> -exist.
> -If this is a concern in your application, use \fI/dev/random\fP
> -instead.
> +
> +The flag
>  .B O_NONBLOCK
>  has no effect when opening
>  .IR /dev/urandom .
> @@ -82,6 +84,8 @@ for the device
>  signals will not be handled until after the requested random bytes
>  have been generated.
>  
> +
> +

Please remove these two blank lines.

>  Since Linux 3.16,
>  .\" commit 79a8468747c5f95ed3d5ce8376a3e82e0c5857fc
>  a
> @@ -104,14 +108,11 @@ This means that it will impact the contents
>  read from both files, but it will not make reads from
>  \fI/dev/random\fP faster.
>  .SS Usage
> -If you are unsure about whether you should use
> +The 
>  .IR /dev/random
> -or
> +interface is considered a legacy interface, and

I'm a little uncomfortable with the term "legacy". To me it implies 
that there is *no* legitimate use of /dev/random these days. I'm
no expert on randomness, but I wonder if that is true. Is it?
If it's not, then I would prefer simply a strong statement that
"/dev/urandom is preferred and sufficient in all use cases".

>  .IR /dev/urandom ,
> -then probably you want to use the latter.
> -As a general rule,
> -.IR /dev/urandom
> -should be used for everything except long-lived GPG/SSL/SSH keys.
> +is recommended for general use.
>  
>  If a seed file is saved across reboots as recommended below (all major
>  Linux distributions have done this since 2000 at least), the output is
> -- 2.7.4

Laurent Georget also commented on this page in a mail last year.
I'm going to thread you (and the other people on this mail) into
that mail discussion in case there's something there that you
might incorporate into a revised patch.

Cheers,

Michael


-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] Update the random(4) documentation towards a more accurate view on /dev/urandom

[Michael Kerrisk (man-pages)]

Hello Nikos,

This was the mail from George that I just referred to in my 
other mail. Is there anything that needs to be covered here?
Some specific comments below.

On 04/26/2016 06:58 PM, George Spelvin wrote:
> Nikos Mavrogiannopoulos wrote:
>> On Mon, 2016-04-25 at 11:46 -0400, George Spelvin wrote:
>>> Removing the examples of high quality randomness I'm less fond of;
>>> the whole idea is to inform people who don't quite understand what
>>> the general terms mean.  Good enough for one-time pads is a design
>>> goal of /dev/random.
> 
>> If that's about documenting a design goal I'd prefer to move it out of
>> the main text for 2 reasons. (a) There is no practical crypto system
>> using one time pads, thus mentioning it in the main body only creates
>> confusion (b), one time pad is such a theoretical construction that any
>> real algorithm wouldn't implement it.
> 
> The original (removed by your patch) line was:
> 
> -high quality randomness such as one-time pad or key generation.
> 
> It's not the words "one-time pad" I'm attached to, but the specific
> examples of when "high-quality randomness" is required.  A big point is
> to teach people *how* to use it, and without those examples, when would
> anyone think "my application wants low-quality randomness"?
> 
> You're right that a one-time pad is impractical, but it's still
> a common and familiar pedagogical example, and more importantly
> something that a person wondering which to use can see that their
> application is NOT.
> 
> Your proposed patch *also* deleted the other usage example at the end:
> 
> -should be used for everything except long-lived GPG/SSL/SSH keys.
> 
> which really reduces the value of the man page as a guide to people
> who aren't crypto experts.

Nikos, what are your thoughts on the above comments?

>>> The bit about early boot is actually not as much of an issue as you
>>> think.
>>>
>>> Even /dev/urandom will stall early on boot until a minimum initial seed
>>> (128 bits at present) has been acumulated.  (grep for "urandom_init_wait")
> 
>> No it will not. We notice often the keys for sshd being generated
>> *before* the kernel logs that the random pool has been
>> initialized.
> 
> H'm... observation definitely trumps theoretical predictions based on
> reading the code.
> 
> Is that a documentation problem or a code bug, or something I'm not
> understanding properly?  If you look for that symbol in the source
> it definitely looks like it's supposed to wait for initial seeding.
> And ssh-keygen uses /dev/urandom.
> 
> The getrandom(2) man page also documents the block-on-startup behavior.
> 
> The driver wakes up the sleeping readers *before* printing
> the message.  Is it possible that syslog is just losing the race?

Anything from the above that is relevant to add to your patch?

>>> How about something more like (draft, not final edit):
>>>
>>>  A read from the \fI/dev/urandom\fP device will not block
>>>  waiting for more entropy.
>>> +If the estimated fresh entropy is not sufficient, a
>>> \fI/dev/urandom\fP
>>> +will produce output anyway, relying on the cryptographic primitives
>>> in
>>> +the driver's pseudo-random number generator to ensure that the
>>> output,
>>> +although correlated with previous output in an information theoretic
>>> +sense (it exceeds the unicity distance), is secure for all practical
>>> +purposes.
> 
>> What is the purpose of this text? To whom does it target?
> 
> To replace the text
> 
> -If there is not sufficient entropy, a pseudorandom number generator is used
> -to create the requested bytes.
> 
> or
> 
> +If the estimated fresh entropy is not sufficient, a pseudorandom number generator is
> +used to create the requested bytes.
> 
> with something that doesn't imply a mode switch.
> 
> I labelled it "draft" because I wasn't really thrilled with the wording,
> myself, but I thought it gave the general idea and wasn't worth massaging
> into editorial perfection since it was due to get torn apart anyway.
> 
> Can you think of better wording?  I'm all for keeping it simple, but
> not at the expense of seriously misleading people.

Comments?

>> I wouldn't like to get into such details about the device in the manpage,
>> but if you would like a section studying the theoretical properties of
>> /dev/urandom I'd again suggest to keep it separate and elaborate. What
>> is on the text above is certainly not complete analysis and is certainly
>> not targetting administrators and developers who would like to understand
>> what this device does.
> 
> A reorganization might indeed be a good way forward; I was examining 
> your changes without stepping back and considering the whole forest.
> 
> Shall I take a stab at it?

Comments? Nikos? George?

>>> I don't like the bit about "use /dev/random or getrandom(2)"; while
>>> getrandom(2) should be mentioned in "see also", the equivalent is
>>> "getrandom(..., GRND_RANDOM)".   It's the flag, no the syscall.
> 
>> It is the syscall. According the description in getrandom(2):
>> "If the pool has not yet been initialized, then the call blocks,
>> unless GRND_RANDOM is specified in flags."
> 
> 1. You have a buggy man page.  The corrected one says "If the pool has
>    not yet been initialized, then the call blocks, unless GRND_NONBLOCK
>    is specified in flags."
> 
> 2. I stand by what I wrote above.  Without the GRND_RANDOM flag,
>    getrandom() access the nonblocking pool (/dev/urandom).
> 
>>> I strongly dislike the deletion of the "as a general rule" advice.
>>> Specific recommendations are very valuable.
> 
>> This advice despite being present for so long, is widely ignored as
>> /dev/urandom is used unconditionally by all software generating keys
>> (SSH/SSL), gnupg being the exception.
> 
> No, it's not being ignored.  The advice isn't "use /dev/random for
> SSH keys", but "*don't* use /dev/random for anything *except* SSH
> keys".  The "(and maybe not even then)" part is implicit, but much
> less of a concern.
> 
> The audience is not the authors of ssh, libssl, or gnupg; they know
> what they're doing.  The audience is everyone *else*, and I think
> specific examples really help there.

Comments?

Cheers,

Michael


-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] Update the random(4) documentation towards a more accurate view on /dev/urandom

[Nikos Mavrogiannopoulos]

[-- Attachment #1: Type: text/plain, Size: 627 bytes --]

On Wed, 2016-11-09 at 16:26 +0100, Michael Kerrisk (man-pages) wrote:
> Hello Nikos,
> 
> This was the mail from George that I just referred to in my 
> other mail. Is there anything that needs to be covered here?
> Some specific comments below.

Hi Michael,
 I think George and I disagree on what the text would recommend. I'm
against keeping the existing text which implies (wrongly IMHO) that
/dev/random should be used for generating keys.

I attach my reply at the time (I had agreed on George updating the
manpage and discussing it further - since that didn't happen I'm back
with an updated proposal).

regards,
Nikos

[-- Attachment #2: urandom --]
[-- Type: text/plain, Size: 8026 bytes --]

Message-ID: <1462178545.15353.37.camel@redhat.com>
Subject: Re: [PATCH] Update the random(4) documentation towards a more
 accurate view on /dev/urandom
From: Nikos Mavrogiannopoulos <nmav@redhat.com>
To: George Spelvin <linux@horizon.com>
Date: Mon, 02 May 2016 10:42:25 +0200
In-Reply-To: <20160426165847.5804.qmail@ns.horizon.com>
References: <20160426165847.5804.qmail@ns.horizon.com>
Content-Type: text/plain; charset="UTF-8"
X-Mailer: Evolution 3.18.5.2 (3.18.5.2-1.fc23) 
Mime-Version: 1.0
Content-Transfer-Encoding: 8bit

On Tue, 2016-04-26 at 12:58 -0400, George Spelvin wrote:

> > If that's about documenting a design goal I'd prefer to move it out
> > of
> > the main text for 2 reasons. (a) There is no practical crypto
> > system
> > using one time pads, thus mentioning it in the main body only
> > creates
> > confusion (b), one time pad is such a theoretical construction that
> > any
> > real algorithm wouldn't implement it.
> The original (removed by your patch) line was:
> -high quality randomness such as one-time pad or key generation.
> It's not the words "one-time pad" I'm attached to, but the specific
> examples of when "high-quality randomness" is required.  A big point
> is
> to teach people *how* to use it, and without those examples, when
> would
> anyone think "my application wants low-quality randomness"?
> 
> You're right that a one-time pad is impractical, but it's still
> a common and familiar pedagogical example, and more importantly
> something that a person wondering which to use can see that their
> application is NOT.
> 
> Your proposed patch *also* deleted the other usage example at the
> end:
> 
> -should be used for everything except long-lived GPG/SSL/SSH keys.
> which really reduces the value of the man page as a guide to people
> who aren't crypto experts.

Well I think that since it contradicts how SSL and SSH keys are
generated in practice it may not be a good example to show. If you
change "should be" to "is intended to be" I may be more sympathetic to
the cause. But then it should also be added "by a user-interacting
application" since the fact that /dev/random has indefinite delays it
makes it unsuitable to generate SSL or SSH keys in an unattended
matter.




> > > The bit about early boot is actually not as much of an issue as
> > > you
> > > think.
> > > 
> > > Even /dev/urandom will stall early on boot until a minimum
> > > initial seed
> > > (128 bits at present) has been acumulated.  (grep for
> > > "urandom_init_wait")
> > 
> > No it will not. We notice often the keys for sshd being generated
> > *before* the kernel logs that the random pool has been
> > initialized.
> H'm... observation definitely trumps theoretical predictions based on
> reading the code.
[...]
> The driver wakes up the sleeping readers *before* printing
> the message.  Is it possible that syslog is just losing the race?

Could be. But then could you point me to the code you are referring to.
I'm certainly interested to figure the issue.


> > > How about something more like (draft, not final edit):
> > > 
> > >  A read from the \fI/dev/urandom\fP device will not block
> > >  waiting for more entropy.
> > > +If the estimated fresh entropy is not sufficient, a
> > > \fI/dev/urandom\fP
> > > +will produce output anyway, relying on the cryptographic
> > > primitives
> > > in
> > > +the driver's pseudo-random number generator to ensure that the
> > > output,
> > > +although correlated with previous output in an information
> > > theoretic
> > > +sense (it exceeds the unicity distance), is secure for all
> > > practical
> > > +purposes.
> > 
> > What is the purpose of this text? To whom does it target?
> To replace the text
> 
> -If there is not sufficient entropy, a pseudorandom number generator
> is used
> -to create the requested bytes.
> 
> or
> 
> +If the estimated fresh entropy is not sufficient, a pseudorandom
> number generator is
> +used to create the requested bytes.
> 
> with something that doesn't imply a mode switch.
> 
> I labelled it "draft" because I wasn't really thrilled with the
> wording,
> myself, but I thought it gave the general idea and wasn't worth
> massaging
> into editorial perfection since it was due to get torn apart anyway.
> 
> Can you think of better wording?  I'm all for keeping it simple, but
> not at the expense of seriously misleading people.

Well by using academic language in a manpage you will certainly mislead
:) since the audience cannot follow. I think the language of the text
should be targeted to the audience, i.e., administrators, developers.
Mentioning information-theoretic correlation will only scare people as
they will believe there is some actual correlation.

Note, that we live with this kind of information theoretic correlation
for our cryptosystems and pretty much for about everything. However it
is not mentioned in any crypto manpage because it will be misunderstood
by the reader (as information-theoretic correlation is far from being
correlation in the everyday sense, and in practice it is not, however
that is not apparent to someone without clear theoretic knowledge).

An internal analysis of the device can be done separately in a
published paper in some reviewed conference/journal (ideally).


> > I wouldn't like to get into such details about the device in the
> > manpage,
> > but if you would like a section studying the theoretical properties
> > of
> > /dev/urandom I'd again suggest to keep it separate and elaborate.
> > What
> > is on the text above is certainly not complete analysis and is
> > certainly
> > not targetting administrators and developers who would like to
> > understand
> > what this device does.
> A reorganization might indeed be a good way forward; I was examining 
> your changes without stepping back and considering the whole forest.
> Shall I take a stab at it?

Please do, I'd be grateful.



> > > I don't like the bit about "use /dev/random or getrandom(2)";
> > > while
> > > getrandom(2) should be mentioned in "see also", the equivalent is
> > > "getrandom(..., GRND_RANDOM)".   It's the flag, no the syscall.
> > It is the syscall. According the description in getrandom(2):
> > "If the pool has not yet been initialized, then the call blocks,
> > unless GRND_RANDOM is specified in flags."
> 1. You have a buggy man page.  The corrected one says "If the pool
> has
>    not yet been initialized, then the call blocks, unless
> GRND_NONBLOCK
>    is specified in flags."
> 
> 2. I stand by what I wrote above.  Without the GRND_RANDOM flag,
>    getrandom() access the nonblocking pool (/dev/urandom).

I misunderstood what you wrote. My point was that getrandom() will
block if the generator is not seeded (see my first point above).

> > > I strongly dislike the deletion of the "as a general rule"
> > > advice.
> > > Specific recommendations are very valuable.
> > 
> > This advice despite being present for so long, is widely ignored as
> > /dev/urandom is used unconditionally by all software generating
> > keys
> > (SSH/SSL), gnupg being the exception.
> No, it's not being ignored.  The advice isn't "use /dev/random for
> SSH keys", but "*don't* use /dev/random for anything *except* SSH
> keys".  The "(and maybe not even then)" part is implicit, but much
> less of a concern.
> 
> The audience is not the authors of ssh, libssl, or gnupg; they know
> what they're doing.  The audience is everyone *else*, and I think
> specific examples really help there.

Could be, but note that an advice that goes against what the experts do
is not really a good advice. I'd be concrete here any say:
 /dev/urandom is intended to be used by non-interactive applications
generating long term keys which cannot afford unpredictable delays.
Examples are SSL, or SSH keys.
 /dev/random is intended to be used by interactive applications, that
can afford long delays, generating long term keys with very high
requirements on the random data quality. A prominent example is gnupg.

regards,
Nikos

Re: [PATCH] Update the random(4) documentation towards a more accurate view on /dev/urandom

[Nikos Mavrogiannopoulos]

[-- Attachment #1: Type: text/plain, Size: 4101 bytes --]

On Wed, 2016-11-09 at 16:23 +0100, Michael Kerrisk (man-pages) wrote:
> [I'm looping a few people into this mail who previously commented
> on this page. Nikos, I will also thread you into an earlier mail 
> by Laurent Georget.]
> 
> Hello Nikos,
> 
> Sorry that I have been so slow to follow up on this.
> Thanks for your persistence. I have some comments 
> that probably require some tweaks to your patch.
> I also have some questions about a couple of other 
> earlier discussions.

The comments should be addressed now (see inline for more info or the
attached patch). I have not included my proposed fix for Laurent's
issue (my proposal was to drop that text, though it can be done
independent of this patch).

https://bugzilla.kernel.org/show_bug.cgi?id=71211




> 
> On 10/20/2016 10:37 AM, Nikos Mavrogiannopoulos wrote:
> > 
> > On Mon, 2016-08-01 at 13:48 +0200, Nikos Mavrogiannopoulos wrote:
> > > 
> > > > 
> > > > This is an updated patch reflecting the recent discussion in
> > > > linux-
> > > > crypto:
> > > > http://www.mail-archive.com/linux-crypto-u79uwXL29TY76Z2rM5mHXA@public.gmane.org/msg204
> > > > 00.html
> > Hi,
> >  I'm resending the patch with few typo fixes, and adding Ted in CC
> > for
> > review. Ted would you like to review this patch for the random(4)
> > manpage?
> 
> Some comments below.
> 
> But one question first. You didn't further reply to George 
> Spelvin's comments on 26 April. Did you consider those comments
> irrelevant or already addressed or something else?

I believe we disagree on some points with George (see other mail).

> By the way, inline patches are rather easier for me to deal with.

(sorry for the attachment, I have no figured a way to make my mailer
send consistently the right data when inline)


> > +.\" 2016-10-20, Nikos Mavrogiannopoulos <nmav-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
> > +.\"     Mention that /dev/random is a legacy interface and removed
> > suggested
> > +.\"     uses of /dev/random.
> 
> No need to update this in-page changelog. We use git these days.

done

> > +.LP
> > +The \fI/dev/random\fP device is a legacy interface which dates
> > back to
> > +a time where the cryptographic primitives used in the
> > implementation
> 
> s%in the implementation%in the implementation of /dev/urandom% ?
> 

done.

> >  has no effect when opening
> >  .IR /dev/urandom .
> > @@ -82,6 +84,8 @@ for the device
> >  signals will not be handled until after the requested random bytes
> >  have been generated.
> >  
> > +
> > +
> 
> Please remove these two blank lines.

done.


> >  Since Linux 3.16,
> >  .\" commit 79a8468747c5f95ed3d5ce8376a3e82e0c5857fc
> >  a
> > @@ -104,14 +108,11 @@ This means that it will impact the contents
> >  read from both files, but it will not make reads from
> >  \fI/dev/random\fP faster.
> >  .SS Usage
> > -If you are unsure about whether you should use
> > +The 
> >  .IR /dev/random
> > -or
> > +interface is considered a legacy interface, and
> 
> I'm a little uncomfortable with the term "legacy". To me it implies 
> that there is *no* legitimate use of /dev/random these days. I'm
> no expert on randomness, but I wonder if that is true. Is it?
> If it's not, then I would prefer simply a strong statement that
> "/dev/urandom is preferred and sufficient in all use cases".

That's a tough one to handle. Yes /dev/urandom is preferred and
sufficient in all use cases, with the exception of early boot time.
More in the text. I've also added a section "KNOWN ISSUES" to state
clearly that issue, and mention getrandom() early in the page.

 
> >  If a seed file is saved across reboots as recommended below (all
> > major
> >  Linux distributions have done this since 2000 at least), the
> > output is
> > -- 2.7.4
> Laurent Georget also commented on this page in a mail last year.
> I'm going to thread you (and the other people on this mail) into
> that mail discussion in case there's something there that you
> might incorporate into a revised patch.

I think it was a different paragraph. Replied in the other email.

regards,
Nikos

[-- Attachment #2: 0001-Update-the-random-4-documentation-towards-a-more-acc.patch --]
[-- Type: text/x-patch, Size: 4793 bytes --]

From 652621c5601d788bd25cb05ee730e27f851e3b94 Mon Sep 17 00:00:00 2001
From: Nikos Mavrogiannopoulos <nmav@redhat.com>
Date: Thu, 7 Apr 2016 09:08:14 +0200
Subject: [PATCH] Update the random(4) documentation towards a more accurate
 view on /dev/urandom

This documents the "property" of /dev/urandom of being able to serve numbers
prior to pool being initialized, and removes any suggested usages of /dev/random
which are disputable (i.e., one-time pad).
Document the fact /dev/random is a legacy interface and only suitable for
applications which can afford indeterminate delays since very few applications
can do so.

Signed-off-by: Nikos Mavrogiannopoulos <nmav@redhat.com>
---
 man4/random.4 | 57 ++++++++++++++++++++++++++++++++-------------------------
 1 file changed, 32 insertions(+), 25 deletions(-)

diff --git a/man4/random.4 b/man4/random.4
index b67c46f..8609b84 100644
--- a/man4/random.4
+++ b/man4/random.4
@@ -13,8 +13,6 @@
 .\" 2004-04-08, AEB, Improved description of read from /dev/urandom
 .\" 2008-06-20, George Spelvin <linux@horizon.com>,
 .\"             Matt Mackall <mpm@selenic.com>
-.\"     Add a Usage subsection that recommends most users to use
-.\"     /dev/urandom, and emphasizes parsimonious usage of /dev/random.
 .\"
 .TH RANDOM 4 2016-10-08 "Linux" "Linux Programmer's Manual"
 .SH NAME
@@ -37,11 +35,26 @@ The generator also keeps an estimate of the
 number of bits of noise in the entropy pool.
 From this entropy pool random numbers are created.
 .LP
-When read, the \fI/dev/random\fP device will return random bytes
-only within the estimated number of bits of noise in the entropy
-pool.
-\fI/dev/random\fP should be suitable for uses that need very
-high quality randomness such as one-time pad or key generation.
+Linux 3.17 and later provides the simpler and safer (see below)
+.BR getrandom(2)
+interface which requires no special files.
+.LP
+When read, the \fI/dev/urandom\fP device return random bytes using a pseudorandom
+number generator seeded from the entropy pool. That operation is
+non-blocking. When used during early boot time, this device may return
+data prior to the entropy pool being initialized.
+If this is of concern in your application, use
+.BR getrandom(2)
+or \fI/dev/random\fP instead.
+
+.LP
+The \fI/dev/random\fP device is a legacy interface which dates back to
+a time where the cryptographic primitives used in the implementation
+of \fI/dev/urandom\fP were not widely trusted. It will return random bytes
+only within the estimated number of bits of fresh noise in the entropy
+pool, blocking if necessary.
+\fI/dev/random\fP is suitable for applications that need very
+high quality randomness, and can afford indeterminate delays.
 When the entropy pool is empty, reads from \fI/dev/random\fP will block
 until additional environmental noise is gathered.
 If
@@ -60,18 +73,8 @@ will return -1 and
 .I errno
 will be set to
 .BR EAGAIN .
-.LP
-A read from the \fI/dev/urandom\fP device will not block
-waiting for more entropy.
-If there is not sufficient entropy, a pseudorandom number generator is used
-to create the requested bytes.
-As a result, in this case the returned values are theoretically vulnerable to a
-cryptographic attack on the algorithms used by the driver.
-Knowledge of how to do this is not available in the current unclassified
-literature, but it is theoretically possible that such an attack may
-exist.
-If this is a concern in your application, use \fI/dev/random\fP
-instead.
+
+The flag
 .B O_NONBLOCK
 has no effect when opening
 .IR /dev/urandom .
@@ -104,14 +107,15 @@ This means that it will impact the contents
 read from both files, but it will not make reads from
 \fI/dev/random\fP faster.
 .SS Usage
-If you are unsure about whether you should use
+The
 .IR /dev/random
-or
+interface is considered a legacy interface, and
 .IR /dev/urandom ,
-then probably you want to use the latter.
-As a general rule,
-.IR /dev/urandom
-should be used for everything except long-lived GPG/SSL/SSH keys.
+is preferred and sufficient in all use cases, with the exception of
+applications which require randomness during early boot time; for
+these applications, the system call
+.BR getrandom(2)
+must be used instead, because will block until the entropy pool is initialized.
 
 If a seed file is saved across reboots as recommended below (all major
 Linux distributions have done this since 2000 at least), the output is
@@ -319,6 +323,9 @@ which gets added to the entropy pool.
 .BR RNDZAPENTCNT ", " RNDCLEARPOOL
 Zero the entropy count of all pools and add some system data (such as
 wall clock) to the pools.
+.SH KNOWN ISSUES
+When used during early boot, \fI/dev/urandom\fP may return data prior to the entropy pool being initialized.
+
 .SH FILES
 /dev/random
 .br
-- 
2.7.4

Re: [PATCH] Update the random(4) documentation towards a more accurate view on /dev/urandom

[Laurent Georget]

[-- Attachment #1.1: Type: text/plain, Size: 4899 bytes --]

Hi everyone,

I very much like the proposed patch. It nicely describes the new
interfaces and it's much more clear. I agree with the use of the
word "legacy" you don't seem to like, Michael, I think this is
accurate (although we might receive a new fair amount of bug reports
about this, when the patch is applied).

By the way, my mail was intended as an answer to the bug report but
I wonder if Corey Csuhta (the original bug poster) saw it because I
replied by email. What should we do about this bug now?

Cheers,
Laurent

Le 10/11/2016 à 09:54, Nikos Mavrogiannopoulos a écrit :
> On Wed, 2016-11-09 at 16:23 +0100, Michael Kerrisk (man-pages) wrote:
>> [I'm looping a few people into this mail who previously commented
>> on this page. Nikos, I will also thread you into an earlier mail 
>> by Laurent Georget.]
>>
>> Hello Nikos,
>>
>> Sorry that I have been so slow to follow up on this.
>> Thanks for your persistence. I have some comments 
>> that probably require some tweaks to your patch.
>> I also have some questions about a couple of other 
>> earlier discussions.
> 
> The comments should be addressed now (see inline for more info or the
> attached patch). I have not included my proposed fix for Laurent's
> issue (my proposal was to drop that text, though it can be done
> independent of this patch).
> 
> https://bugzilla.kernel.org/show_bug.cgi?id=71211
> 
> 
> 
> 
>>
>> On 10/20/2016 10:37 AM, Nikos Mavrogiannopoulos wrote:
>>>
>>> On Mon, 2016-08-01 at 13:48 +0200, Nikos Mavrogiannopoulos wrote:
>>>>
>>>>>
>>>>> This is an updated patch reflecting the recent discussion in
>>>>> linux-
>>>>> crypto:
>>>>> http://www.mail-archive.com/linux-crypto-u79uwXL29TY76Z2rM5mHXA@public.gmane.org/msg204
>>>>> 00.html
>>> Hi,
>>>  I'm resending the patch with few typo fixes, and adding Ted in CC
>>> for
>>> review. Ted would you like to review this patch for the random(4)
>>> manpage?
>>
>> Some comments below.
>>
>> But one question first. You didn't further reply to George 
>> Spelvin's comments on 26 April. Did you consider those comments
>> irrelevant or already addressed or something else?
> 
> I believe we disagree on some points with George (see other mail).
> 
>> By the way, inline patches are rather easier for me to deal with.
> 
> (sorry for the attachment, I have no figured a way to make my mailer
> send consistently the right data when inline)
> 
> 
>>> +.\" 2016-10-20, Nikos Mavrogiannopoulos <nmav-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
>>> +.\"     Mention that /dev/random is a legacy interface and removed
>>> suggested
>>> +.\"     uses of /dev/random.
>>
>> No need to update this in-page changelog. We use git these days.
> 
> done
> 
>>> +.LP
>>> +The \fI/dev/random\fP device is a legacy interface which dates
>>> back to
>>> +a time where the cryptographic primitives used in the
>>> implementation
>>
>> s%in the implementation%in the implementation of /dev/urandom% ?
>>
> 
> done.
> 
>>>  has no effect when opening
>>>  .IR /dev/urandom .
>>> @@ -82,6 +84,8 @@ for the device
>>>  signals will not be handled until after the requested random bytes
>>>  have been generated.
>>>  
>>> +
>>> +
>>
>> Please remove these two blank lines.
> 
> done.
> 
> 
>>>  Since Linux 3.16,
>>>  .\" commit 79a8468747c5f95ed3d5ce8376a3e82e0c5857fc
>>>  a
>>> @@ -104,14 +108,11 @@ This means that it will impact the contents
>>>  read from both files, but it will not make reads from
>>>  \fI/dev/random\fP faster.
>>>  .SS Usage
>>> -If you are unsure about whether you should use
>>> +The 
>>>  .IR /dev/random
>>> -or
>>> +interface is considered a legacy interface, and
>>
>> I'm a little uncomfortable with the term "legacy". To me it implies 
>> that there is *no* legitimate use of /dev/random these days. I'm
>> no expert on randomness, but I wonder if that is true. Is it?
>> If it's not, then I would prefer simply a strong statement that
>> "/dev/urandom is preferred and sufficient in all use cases".
> 
> That's a tough one to handle. Yes /dev/urandom is preferred and
> sufficient in all use cases, with the exception of early boot time.
> More in the text. I've also added a section "KNOWN ISSUES" to state
> clearly that issue, and mention getrandom() early in the page.
> 
>  
>>>  If a seed file is saved across reboots as recommended below (all
>>> major
>>>  Linux distributions have done this since 2000 at least), the
>>> output is
>>> -- 2.7.4
>> Laurent Georget also commented on this page in a mail last year.
>> I'm going to thread you (and the other people on this mail) into
>> that mail discussion in case there's something there that you
>> might incorporate into a revised patch.
> 
> I think it was a different paragraph. Replied in the other email.
> 
> regards,
> Nikos
> 


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: [PATCH] Update the random(4) documentation towards a more accurate view on /dev/urandom

[Michael Kerrisk (man-pages)]

Hi Nikos,

On 11/10/2016 09:21 AM, Nikos Mavrogiannopoulos wrote:
> On Wed, 2016-11-09 at 16:26 +0100, Michael Kerrisk (man-pages) wrote:
>> Hello Nikos,
>>
>> This was the mail from George that I just referred to in my 
>> other mail. Is there anything that needs to be covered here?
>> Some specific comments below.
> 
> Hi Michael,
>  I think George and I disagree on what the text would recommend. I'm
> against keeping the existing text which implies (wrongly IMHO) that
> /dev/random should be used for generating keys.
> 
> I attach my reply at the time (I had agreed on George updating the
> manpage and discussing it further - since that didn't happen I'm back
> with an updated proposal).

Was that reply to George offlist? It didn't land in my mailbox, at 
least.

Anyway, thanks for the context. See coming replies re your 
latest patch.

Cheers,

Michael

Message-ID: <1462178545.15353.37.camel-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
Subject: Re: [PATCH] Update the random(4) documentation towards a more
 accurate view on /dev/urandom
From: Nikos Mavrogiannopoulos <nmav-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
To: George Spelvin <linux-gpGsJRJZ3PBBDgjK7y7TUQ@public.gmane.org>
Date: Mon, 02 May 2016 10:42:25 +0200
In-Reply-To: <20160426165847.5804.qmail-HzZAx2gCgqrSUeElwK9/Pw@public.gmane.org>
References: <20160426165847.5804.qmail-HzZAx2gCgqrSUeElwK9/Pw@public.gmane.org>
Content-Type: text/plain; charset="UTF-8"
X-Mailer: Evolution 3.18.5.2 (3.18.5.2-1.fc23) 
Mime-Version: 1.0
Content-Transfer-Encoding: 8bit

On Tue, 2016-04-26 at 12:58 -0400, George Spelvin wrote:

> > If that's about documenting a design goal I'd prefer to move it out
> > of
> > the main text for 2 reasons. (a) There is no practical crypto
> > system
> > using one time pads, thus mentioning it in the main body only
> > creates
> > confusion (b), one time pad is such a theoretical construction that
> > any
> > real algorithm wouldn't implement it.
> The original (removed by your patch) line was:
> -high quality randomness such as one-time pad or key generation.
> It's not the words "one-time pad" I'm attached to, but the specific
> examples of when "high-quality randomness" is required.  A big point
> is
> to teach people *how* to use it, and without those examples, when
> would
> anyone think "my application wants low-quality randomness"?
> 
> You're right that a one-time pad is impractical, but it's still
> a common and familiar pedagogical example, and more importantly
> something that a person wondering which to use can see that their
> application is NOT.
> 
> Your proposed patch *also* deleted the other usage example at the
> end:
> 
> -should be used for everything except long-lived GPG/SSL/SSH keys.
> which really reduces the value of the man page as a guide to people
> who aren't crypto experts.

Well I think that since it contradicts how SSL and SSH keys are
generated in practice it may not be a good example to show. If you
change "should be" to "is intended to be" I may be more sympathetic to
the cause. But then it should also be added "by a user-interacting
application" since the fact that /dev/random has indefinite delays it
makes it unsuitable to generate SSL or SSH keys in an unattended
matter.




> > > The bit about early boot is actually not as much of an issue as
> > > you
> > > think.
> > > 
> > > Even /dev/urandom will stall early on boot until a minimum
> > > initial seed
> > > (128 bits at present) has been acumulated.  (grep for
> > > "urandom_init_wait")
> > 
> > No it will not. We notice often the keys for sshd being generated
> > *before* the kernel logs that the random pool has been
> > initialized.
> H'm... observation definitely trumps theoretical predictions based on
> reading the code.
[...]
> The driver wakes up the sleeping readers *before* printing
> the message.  Is it possible that syslog is just losing the race?

Could be. But then could you point me to the code you are referring to.
I'm certainly interested to figure the issue.


> > > How about something more like (draft, not final edit):
> > > 
> > >  A read from the \fI/dev/urandom\fP device will not block
> > >  waiting for more entropy.
> > > +If the estimated fresh entropy is not sufficient, a
> > > \fI/dev/urandom\fP
> > > +will produce output anyway, relying on the cryptographic
> > > primitives
> > > in
> > > +the driver's pseudo-random number generator to ensure that the
> > > output,
> > > +although correlated with previous output in an information
> > > theoretic
> > > +sense (it exceeds the unicity distance), is secure for all
> > > practical
> > > +purposes.
> > 
> > What is the purpose of this text? To whom does it target?
> To replace the text
> 
> -If there is not sufficient entropy, a pseudorandom number generator
> is used
> -to create the requested bytes.
> 
> or
> 
> +If the estimated fresh entropy is not sufficient, a pseudorandom
> number generator is
> +used to create the requested bytes.
> 
> with something that doesn't imply a mode switch.
> 
> I labelled it "draft" because I wasn't really thrilled with the
> wording,
> myself, but I thought it gave the general idea and wasn't worth
> massaging
> into editorial perfection since it was due to get torn apart anyway.
> 
> Can you think of better wording?  I'm all for keeping it simple, but
> not at the expense of seriously misleading people.

Well by using academic language in a manpage you will certainly mislead
:) since the audience cannot follow. I think the language of the text
should be targeted to the audience, i.e., administrators, developers.
Mentioning information-theoretic correlation will only scare people as
they will believe there is some actual correlation.

Note, that we live with this kind of information theoretic correlation
for our cryptosystems and pretty much for about everything. However it
is not mentioned in any crypto manpage because it will be misunderstood
by the reader (as information-theoretic correlation is far from being
correlation in the everyday sense, and in practice it is not, however
that is not apparent to someone without clear theoretic knowledge).

An internal analysis of the device can be done separately in a
published paper in some reviewed conference/journal (ideally).


> > I wouldn't like to get into such details about the device in the
> > manpage,
> > but if you would like a section studying the theoretical properties
> > of
> > /dev/urandom I'd again suggest to keep it separate and elaborate.
> > What
> > is on the text above is certainly not complete analysis and is
> > certainly
> > not targetting administrators and developers who would like to
> > understand
> > what this device does.
> A reorganization might indeed be a good way forward; I was examining 
> your changes without stepping back and considering the whole forest.
> Shall I take a stab at it?

Please do, I'd be grateful.



> > > I don't like the bit about "use /dev/random or getrandom(2)";
> > > while
> > > getrandom(2) should be mentioned in "see also", the equivalent is
> > > "getrandom(..., GRND_RANDOM)".   It's the flag, no the syscall.
> > It is the syscall. According the description in getrandom(2):
> > "If the pool has not yet been initialized, then the call blocks,
> > unless GRND_RANDOM is specified in flags."
> 1. You have a buggy man page.  The corrected one says "If the pool
> has
>    not yet been initialized, then the call blocks, unless
> GRND_NONBLOCK
>    is specified in flags."
> 
> 2. I stand by what I wrote above.  Without the GRND_RANDOM flag,
>    getrandom() access the nonblocking pool (/dev/urandom).

I misunderstood what you wrote. My point was that getrandom() will
block if the generator is not seeded (see my first point above).

> > > I strongly dislike the deletion of the "as a general rule"
> > > advice.
> > > Specific recommendations are very valuable.
> > 
> > This advice despite being present for so long, is widely ignored as
> > /dev/urandom is used unconditionally by all software generating
> > keys
> > (SSH/SSL), gnupg being the exception.
> No, it's not being ignored.  The advice isn't "use /dev/random for
> SSH keys", but "*don't* use /dev/random for anything *except* SSH
> keys".  The "(and maybe not even then)" part is implicit, but much
> less of a concern.
> 
> The audience is not the authors of ssh, libssl, or gnupg; they know
> what they're doing.  The audience is everyone *else*, and I think
> specific examples really help there.

Could be, but note that an advice that goes against what the experts do
is not really a good advice. I'd be concrete here any say:
 /dev/urandom is intended to be used by non-interactive applications
generating long term keys which cannot afford unpredictable delays.
Examples are SSL, or SSH keys.
 /dev/random is intended to be used by interactive applications, that
can afford long delays, generating long term keys with very high
requirements on the random data quality. A prominent example is gnupg.

regards,
Nikos

-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] Update the random(4) documentation towards a more accurate view on /dev/urandom

[Michael Kerrisk (man-pages)]

Hello Nikos,

On 11/10/2016 09:54 AM, Nikos Mavrogiannopoulos wrote:
> On Wed, 2016-11-09 at 16:23 +0100, Michael Kerrisk (man-pages) wrote:
>> [I'm looping a few people into this mail who previously commented
>> on this page. Nikos, I will also thread you into an earlier mail 
>> by Laurent Georget.]
>>
>> Hello Nikos,
>>
>> Sorry that I have been so slow to follow up on this.
>> Thanks for your persistence. I have some comments 
>> that probably require some tweaks to your patch.
>> I also have some questions about a couple of other 
>> earlier discussions.
> 
> The comments should be addressed now (see inline for more info or the
> attached patch). I have not included my proposed fix for Laurent's
> issue (my proposal was to drop that text, though it can be done
> independent of this patch).
> 
> https://bugzilla.kernel.org/show_bug.cgi?id=71211

Okay. I'll reply to that separately.

>> On 10/20/2016 10:37 AM, Nikos Mavrogiannopoulos wrote:
>>>
>>> On Mon, 2016-08-01 at 13:48 +0200, Nikos Mavrogiannopoulos wrote:
>>>>
>>>>>
>>>>> This is an updated patch reflecting the recent discussion in
>>>>> linux-
>>>>> crypto:
>>>>> http://www.mail-archive.com/linux-crypto-u79uwXL29TY76Z2rM5mHXA@public.gmane.org/msg204
>>>>> 00.html
>>> Hi,
>>>  I'm resending the patch with few typo fixes, and adding Ted in CC
>>> for
>>> review. Ted would you like to review this patch for the random(4)
>>> manpage?
>>
>> Some comments below.
>>
>> But one question first. You didn't further reply to George 
>> Spelvin's comments on 26 April. Did you consider those comments
>> irrelevant or already addressed or something else?
> 
> I believe we disagree on some points with George (see other mail).
> 
>> By the way, inline patches are rather easier for me to deal with.
> 
> (sorry for the attachment, I have no figured a way to make my mailer
> send consistently the right data when inline)

Okay. But, have you read "MUA-SPECIFIC HINTS" in the 
git-format-patch(1) man page?

[...]

>>>  Since Linux 3.16,
>>>  .\" commit 79a8468747c5f95ed3d5ce8376a3e82e0c5857fc
>>>  a
>>> @@ -104,14 +108,11 @@ This means that it will impact the contents
>>>  read from both files, but it will not make reads from
>>>  \fI/dev/random\fP faster.
>>>  .SS Usage
>>> -If you are unsure about whether you should use
>>> +The 
>>>  .IR /dev/random
>>> -or
>>> +interface is considered a legacy interface, and
>>
>> I'm a little uncomfortable with the term "legacy". To me it implies 
>> that there is *no* legitimate use of /dev/random these days. I'm
>> no expert on randomness, but I wonder if that is true. Is it?
>> If it's not, then I would prefer simply a strong statement that
>> "/dev/urandom is preferred and sufficient in all use cases".
> 
> That's a tough one to handle. Yes /dev/urandom is preferred and
> sufficient in all use cases, with the exception of early boot time.
> More in the text. I've also added a section "KNOWN ISSUES" to state
> clearly that issue, and mention getrandom() early in the page.

Okay. I can go with your suggestion now, especially since Laurent 
also chimed in to agree with you.

>>>  If a seed file is saved across reboots as recommended below (all
>>> major
>>>  Linux distributions have done this since 2000 at least), the
>>> output is
>>> -- 2.7.4
>> Laurent Georget also commented on this page in a mail last year.
>> I'm going to thread you (and the other people on this mail) into
>> that mail discussion in case there's something there that you
>> might incorporate into a revised patch.
> 
> I think it was a different paragraph. Replied in the other email.

Okay.

I've applied your patch. Thanks for persisting with this.
After massaging the text a little, I have also applied 
the following wording fix, which I hope is okay:

===
diff --git a/man4/random.4 b/man4/random.4
index 0363dac..a0c10fa 100644
--- a/man4/random.4
+++ b/man4/random.4
@@ -46,8 +46,8 @@ When read, the
 .I /dev/urandom
 device returns random bytes using a pseudorandom
 number generator seeded from the entropy pool.
-That operation is non-blocking.
-When used during early boot time, this device may return
+Reads from this device are nonblocking.
+When read during early boot time, this device may return
 data prior to the entropy pool being initialized.
 If this is of concern in your application, use
 .BR getrandom (2)
@@ -331,7 +331,7 @@ which gets added to the entropy pool.
 Zero the entropy count of all pools and add some system data (such as
 wall clock) to the pools.
 .SH BUGS
-When used during early boot,
+During early boot time, reads from
 .I /dev/urandom
 may return data prior to the entropy pool being initialized.
 .SH FILES
====

The updates are currently sitting in a local branch on my machine
just in case we need more tweaks, but the patch will go out
with the next man-pages release.

Cheers,

Michael


=====
>From 652621c5601d788bd25cb05ee730e27f851e3b94 Mon Sep 17 00:00:00 2001
From: Nikos Mavrogiannopoulos <nmav-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
Date: Thu, 7 Apr 2016 09:08:14 +0200
Subject: [PATCH] Update the random(4) documentation towards a more accurate
 view on /dev/urandom

This documents the "property" of /dev/urandom of being able to serve numbers
prior to pool being initialized, and removes any suggested usages of /dev/random
which are disputable (i.e., one-time pad).
Document the fact /dev/random is a legacy interface and only suitable for
applications which can afford indeterminate delays since very few applications
can do so.

Signed-off-by: Nikos Mavrogiannopoulos <nmav-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
---
 man4/random.4 | 57 ++++++++++++++++++++++++++++++++-------------------------
 1 file changed, 32 insertions(+), 25 deletions(-)

diff --git a/man4/random.4 b/man4/random.4
index b67c46f..8609b84 100644
--- a/man4/random.4
+++ b/man4/random.4
@@ -13,8 +13,6 @@
 .\" 2004-04-08, AEB, Improved description of read from /dev/urandom
 .\" 2008-06-20, George Spelvin <linux-gpGsJRJZ3PBBDgjK7y7TUQ@public.gmane.org>,
 .\"             Matt Mackall <mpm-VDJrAJ4Gl5ZBDgjK7y7TUQ@public.gmane.org>
-.\"     Add a Usage subsection that recommends most users to use
-.\"     /dev/urandom, and emphasizes parsimonious usage of /dev/random.
 .\"
 .TH RANDOM 4 2016-10-08 "Linux" "Linux Programmer's Manual"
 .SH NAME
@@ -37,11 +35,26 @@ The generator also keeps an estimate of the
 number of bits of noise in the entropy pool.
 From this entropy pool random numbers are created.
 .LP
-When read, the \fI/dev/random\fP device will return random bytes
-only within the estimated number of bits of noise in the entropy
-pool.
-\fI/dev/random\fP should be suitable for uses that need very
-high quality randomness such as one-time pad or key generation.
+Linux 3.17 and later provides the simpler and safer (see below)
+.BR getrandom(2)
+interface which requires no special files.
+.LP
+When read, the \fI/dev/urandom\fP device return random bytes using a pseudorandom
+number generator seeded from the entropy pool. That operation is
+non-blocking. When used during early boot time, this device may return
+data prior to the entropy pool being initialized.
+If this is of concern in your application, use
+.BR getrandom(2)
+or \fI/dev/random\fP instead.
+
+.LP
+The \fI/dev/random\fP device is a legacy interface which dates back to
+a time where the cryptographic primitives used in the implementation
+of \fI/dev/urandom\fP were not widely trusted. It will return random bytes
+only within the estimated number of bits of fresh noise in the entropy
+pool, blocking if necessary.
+\fI/dev/random\fP is suitable for applications that need very
+high quality randomness, and can afford indeterminate delays.
 When the entropy pool is empty, reads from \fI/dev/random\fP will block
 until additional environmental noise is gathered.
 If
@@ -60,18 +73,8 @@ will return -1 and
 .I errno
 will be set to
 .BR EAGAIN .
-.LP
-A read from the \fI/dev/urandom\fP device will not block
-waiting for more entropy.
-If there is not sufficient entropy, a pseudorandom number generator is used
-to create the requested bytes.
-As a result, in this case the returned values are theoretically vulnerable to a
-cryptographic attack on the algorithms used by the driver.
-Knowledge of how to do this is not available in the current unclassified
-literature, but it is theoretically possible that such an attack may
-exist.
-If this is a concern in your application, use \fI/dev/random\fP
-instead.
+
+The flag
 .B O_NONBLOCK
 has no effect when opening
 .IR /dev/urandom .
@@ -104,14 +107,15 @@ This means that it will impact the contents
 read from both files, but it will not make reads from
 \fI/dev/random\fP faster.
 .SS Usage
-If you are unsure about whether you should use
+The
 .IR /dev/random
-or
+interface is considered a legacy interface, and
 .IR /dev/urandom ,
-then probably you want to use the latter.
-As a general rule,
-.IR /dev/urandom
-should be used for everything except long-lived GPG/SSL/SSH keys.
+is preferred and sufficient in all use cases, with the exception of
+applications which require randomness during early boot time; for
+these applications, the system call
+.BR getrandom(2)
+must be used instead, because will block until the entropy pool is initialized.
 
 If a seed file is saved across reboots as recommended below (all major
 Linux distributions have done this since 2000 at least), the output is
@@ -319,6 +323,9 @@ which gets added to the entropy pool.
 .BR RNDZAPENTCNT ", " RNDCLEARPOOL
 Zero the entropy count of all pools and add some system data (such as
 wall clock) to the pools.
+.SH KNOWN ISSUES
+When used during early boot, \fI/dev/urandom\fP may return data prior to the entropy pool being initialized.
+
 .SH FILES
 /dev/random
 .br
-- 

-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] Update the random(4) documentation towards a more accurate view on /dev/urandom

[Nikos Mavrogiannopoulos]

On Thu, 2016-11-10 at 12:50 +0100, Michael Kerrisk (man-pages) wrote:
> Hello Nikos,

> > > >  If a seed file is saved across reboots as recommended below
> > > > (all
> > > > major
> > > >  Linux distributions have done this since 2000 at least), the
> > > > output is
> > > > -- 2.7.4
> > > Laurent Georget also commented on this page in a mail last year.
> > > I'm going to thread you (and the other people on this mail) into
> > > that mail discussion in case there's something there that you
> > > might incorporate into a revised patch.
> > 
> > I think it was a different paragraph. Replied in the other email.
> 
> Okay.
> 
> I've applied your patch. Thanks for persisting with this.
> After massaging the text a little, I have also applied 
> the following wording fix, which I hope is okay:

The new text is fine with me.

regards,
Nikos

--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] Update the random(4) documentation towards a more accurate view on /dev/urandom

[Michael Kerrisk (man-pages)]

Hello Laurent,

On 11/10/2016 10:11 AM, Laurent Georget wrote:
> Hi everyone,
> 
> I very much like the proposed patch. It nicely describes the new
> interfaces and it's much more clear. I agree with the use of the
> word "legacy" you don't seem to like, Michael, I think this is
> accurate (although we might receive a new fair amount of bug reports
> about this, when the patch is applied).

Thanks for your comments Laurent! It's really helpful to
have a second voice confirming a patch to a man page where
my background is weak. I've applied the patch, and added a
Reviewed-by: tag from you.

> By the way, my mail was intended as an answer to the bug report but
> I wonder if Corey Csuhta (the original bug poster) saw it because I
> replied by email. What should we do about this bug now?

I'll take this to the other thread.

Cheers,

Michael

> Le 10/11/2016 à 09:54, Nikos Mavrogiannopoulos a écrit :
>> On Wed, 2016-11-09 at 16:23 +0100, Michael Kerrisk (man-pages) wrote:
>>> [I'm looping a few people into this mail who previously commented
>>> on this page. Nikos, I will also thread you into an earlier mail 
>>> by Laurent Georget.]
>>>
>>> Hello Nikos,
>>>
>>> Sorry that I have been so slow to follow up on this.
>>> Thanks for your persistence. I have some comments 
>>> that probably require some tweaks to your patch.
>>> I also have some questions about a couple of other 
>>> earlier discussions.
>>
>> The comments should be addressed now (see inline for more info or the
>> attached patch). I have not included my proposed fix for Laurent's
>> issue (my proposal was to drop that text, though it can be done
>> independent of this patch).
>>
>> https://bugzilla.kernel.org/show_bug.cgi?id=71211
>>
>>
>>
>>
>>>
>>> On 10/20/2016 10:37 AM, Nikos Mavrogiannopoulos wrote:
>>>>
>>>> On Mon, 2016-08-01 at 13:48 +0200, Nikos Mavrogiannopoulos wrote:
>>>>>
>>>>>>
>>>>>> This is an updated patch reflecting the recent discussion in
>>>>>> linux-
>>>>>> crypto:
>>>>>> http://www.mail-archive.com/linux-crypto-u79uwXL29TY76Z2rM5mHXA@public.gmane.org/msg204
>>>>>> 00.html
>>>> Hi,
>>>>  I'm resending the patch with few typo fixes, and adding Ted in CC
>>>> for
>>>> review. Ted would you like to review this patch for the random(4)
>>>> manpage?
>>>
>>> Some comments below.
>>>
>>> But one question first. You didn't further reply to George 
>>> Spelvin's comments on 26 April. Did you consider those comments
>>> irrelevant or already addressed or something else?
>>
>> I believe we disagree on some points with George (see other mail).
>>
>>> By the way, inline patches are rather easier for me to deal with.
>>
>> (sorry for the attachment, I have no figured a way to make my mailer
>> send consistently the right data when inline)
>>
>>
>>>> +.\" 2016-10-20, Nikos Mavrogiannopoulos <nmav-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
>>>> +.\"     Mention that /dev/random is a legacy interface and removed
>>>> suggested
>>>> +.\"     uses of /dev/random.
>>>
>>> No need to update this in-page changelog. We use git these days.
>>
>> done
>>
>>>> +.LP
>>>> +The \fI/dev/random\fP device is a legacy interface which dates
>>>> back to
>>>> +a time where the cryptographic primitives used in the
>>>> implementation
>>>
>>> s%in the implementation%in the implementation of /dev/urandom% ?
>>>
>>
>> done.
>>
>>>>  has no effect when opening
>>>>  .IR /dev/urandom .
>>>> @@ -82,6 +84,8 @@ for the device
>>>>  signals will not be handled until after the requested random bytes
>>>>  have been generated.
>>>>  
>>>> +
>>>> +
>>>
>>> Please remove these two blank lines.
>>
>> done.
>>
>>
>>>>  Since Linux 3.16,
>>>>  .\" commit 79a8468747c5f95ed3d5ce8376a3e82e0c5857fc
>>>>  a
>>>> @@ -104,14 +108,11 @@ This means that it will impact the contents
>>>>  read from both files, but it will not make reads from
>>>>  \fI/dev/random\fP faster.
>>>>  .SS Usage
>>>> -If you are unsure about whether you should use
>>>> +The 
>>>>  .IR /dev/random
>>>> -or
>>>> +interface is considered a legacy interface, and
>>>
>>> I'm a little uncomfortable with the term "legacy". To me it implies 
>>> that there is *no* legitimate use of /dev/random these days. I'm
>>> no expert on randomness, but I wonder if that is true. Is it?
>>> If it's not, then I would prefer simply a strong statement that
>>> "/dev/urandom is preferred and sufficient in all use cases".
>>
>> That's a tough one to handle. Yes /dev/urandom is preferred and
>> sufficient in all use cases, with the exception of early boot time.
>> More in the text. I've also added a section "KNOWN ISSUES" to state
>> clearly that issue, and mention getrandom() early in the page.
>>
>>  
>>>>  If a seed file is saved across reboots as recommended below (all
>>>> major
>>>>  Linux distributions have done this since 2000 at least), the
>>>> output is
>>>> -- 2.7.4
>>> Laurent Georget also commented on this page in a mail last year.
>>> I'm going to thread you (and the other people on this mail) into
>>> that mail discussion in case there's something there that you
>>> might incorporate into a revised patch.
>>
>> I think it was a different paragraph. Replied in the other email.
>>
>> regards,
>> Nikos
>>
> 


-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] Update the random(4) documentation towards a more accurate view on /dev/urandom

[Michael Kerrisk (man-pages)]

On 11/10/2016 12:53 PM, Nikos Mavrogiannopoulos wrote:
> On Thu, 2016-11-10 at 12:50 +0100, Michael Kerrisk (man-pages) wrote:
>> Hello Nikos,
> 
>>>>>  If a seed file is saved across reboots as recommended below
>>>>> (all
>>>>> major
>>>>>  Linux distributions have done this since 2000 at least), the
>>>>> output is
>>>>> -- 2.7.4
>>>> Laurent Georget also commented on this page in a mail last year.
>>>> I'm going to thread you (and the other people on this mail) into
>>>> that mail discussion in case there's something there that you
>>>> might incorporate into a revised patch.
>>>
>>> I think it was a different paragraph. Replied in the other email.
>>
>> Okay.
>>
>> I've applied your patch. Thanks for persisting with this.
>> After massaging the text a little, I have also applied 
>> the following wording fix, which I hope is okay:
> 
> The new text is fine with me.

Thanks, Nikos.

Cheers,

Michael


-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] Update the random(4) documentation towards a more accurate view on /dev/urandom

[Michael Kerrisk (man-pages)]

On 11/10/2016 12:56 PM, Michael Kerrisk (man-pages) wrote:
> On 11/10/2016 12:53 PM, Nikos Mavrogiannopoulos wrote:
>> On Thu, 2016-11-10 at 12:50 +0100, Michael Kerrisk (man-pages) wrote:
>>> Hello Nikos,
>>
>>>>>>  If a seed file is saved across reboots as recommended below
>>>>>> (all
>>>>>> major
>>>>>>  Linux distributions have done this since 2000 at least), the
>>>>>> output is
>>>>>> -- 2.7.4
>>>>> Laurent Georget also commented on this page in a mail last year.
>>>>> I'm going to thread you (and the other people on this mail) into
>>>>> that mail discussion in case there's something there that you
>>>>> might incorporate into a revised patch.
>>>>
>>>> I think it was a different paragraph. Replied in the other email.
>>>
>>> Okay.
>>>
>>> I've applied your patch. Thanks for persisting with this.
>>> After massaging the text a little, I have also applied 
>>> the following wording fix, which I hope is okay:
>>
>> The new text is fine with me.
> 
> Thanks, Nikos.

All the changes we discussed so far are now pushed to Git.
I'll now send out a couple of proposed patches.

Cheers,

Michael



-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH] random.4: Improve discussion or urandom, blocking reads, and signals

[Michael Kerrisk (man-pages)]

Hello all,

While reworking the text of the random(4) page, I found
what appear to be some inaccuracies. Could I get some feedback
on the patch below?

Thanks,

Michael

    random.4: Improve discussion or urandom, blocking reads, and signals
    
    The text currently states that O_NONBLOCK has no effect for
    /dev/urandom, which is true.  It also says that reads from
    /dev/urandom are nonblocking.  This is at the least confusing.
    If one attempts large reads (say 10MB) from /dev/urandom
    there is an appreciable delay, and interruption by a signal
    handler will result in a short read. Amend the text to
    reflect this.
    
    Signed-off-by: Michael Kerrisk <mtk.manpages-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>

diff --git a/man4/random.4 b/man4/random.4
index db5d0a2..736cbde 100644
--- a/man4/random.4
+++ b/man4/random.4
@@ -46,7 +46,6 @@ When read, the
 .I /dev/urandom
 device returns random bytes using a pseudorandom
 number generator seeded from the entropy pool.
-Reads from this device are nonblocking.
 When read during early boot time, this device may return
 data prior to the entropy pool being initialized.
 If this is of concern in your application, use
@@ -88,8 +87,11 @@ When calling
 .BR read (2)
 for the device
 .IR /dev/urandom ,
-signals will not be handled until after the requested random bytes
-have been generated.
+reads of up to 256 bytes will return as many bytes as are requested
+and will not be interrupted by a signal handler.
+Reads with a buffer over this limit may return less than the
+requested number of bytes or fail with the error
+.BR EINTR .

--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] random.4: Improve discussion or urandom, blocking reads, and signals

[Laurent Georget]

[-- Attachment #1.1: Type: text/plain, Size: 2311 bytes --]

Hello,

Le 10/11/2016 à 15:20, Michael Kerrisk (man-pages) a écrit :
> Hello all,
> 
> While reworking the text of the random(4) page, I found
> what appear to be some inaccuracies. Could I get some feedback
> on the patch below?
> 
> Thanks,
> 
> Michael
> 
>     random.4: Improve discussion or urandom, blocking reads, and signals
>     
>     The text currently states that O_NONBLOCK has no effect for
>     /dev/urandom, which is true.  It also says that reads from
>     /dev/urandom are nonblocking.  This is at the least confusing.
>     If one attempts large reads (say 10MB) from /dev/urandom
>     there is an appreciable delay, and interruption by a signal
>     handler will result in a short read. Amend the text to
>     reflect this.
>     
>     Signed-off-by: Michael Kerrisk <mtk.manpages-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
> 
> diff --git a/man4/random.4 b/man4/random.4
> index db5d0a2..736cbde 100644
> --- a/man4/random.4
> +++ b/man4/random.4
> @@ -46,7 +46,6 @@ When read, the
>  .I /dev/urandom
>  device returns random bytes using a pseudorandom
>  number generator seeded from the entropy pool.
> -Reads from this device are nonblocking.

Being non blocking is not incompatible with being interruptible, though.
Non blocking only means that the reading will never explicitly yield the
CPU to wait for data, which is true. This is consistent with the use of
"non blocking" elsewhere in the filesystem, as far as I understand it.

But I guess this can be confusing indeed, and that the precision is not
useful in the man page.

>  When read during early boot time, this device may return
>  data prior to the entropy pool being initialized.
>  If this is of concern in your application, use
> @@ -88,8 +87,11 @@ When calling
>  .BR read (2)
>  for the device
>  .IR /dev/urandom ,
> -signals will not be handled until after the requested random bytes
> -have been generated.
> +reads of up to 256 bytes will return as many bytes as are requested
> +and will not be interrupted by a signal handler.
> +Reads with a buffer over this limit may return less than the
> +requested number of bytes or fail with the error
> +.BR EINTR .

Yes, this is a useful precision.

This patch looks good to me.

Cheers,
Laurent


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: [PATCH] random.4: Improve discussion or urandom, blocking reads, and signals

[Michael Kerrisk (man-pages)]

Hello Laurent,

On 11/11/2016 11:28 AM, Laurent Georget wrote:
> Hello,
> 
> Le 10/11/2016 à 15:20, Michael Kerrisk (man-pages) a écrit :
>> Hello all,
>>
>> While reworking the text of the random(4) page, I found
>> what appear to be some inaccuracies. Could I get some feedback
>> on the patch below?
>>
>> Thanks,
>>
>> Michael
>>
>>     random.4: Improve discussion or urandom, blocking reads, and signals
>>     
>>     The text currently states that O_NONBLOCK has no effect for
>>     /dev/urandom, which is true.  It also says that reads from
>>     /dev/urandom are nonblocking.  This is at the least confusing.
>>     If one attempts large reads (say 10MB) from /dev/urandom
>>     there is an appreciable delay, and interruption by a signal
>>     handler will result in a short read. Amend the text to
>>     reflect this.
>>     
>>     Signed-off-by: Michael Kerrisk <mtk.manpages-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
>>
>> diff --git a/man4/random.4 b/man4/random.4
>> index db5d0a2..736cbde 100644
>> --- a/man4/random.4
>> +++ b/man4/random.4
>> @@ -46,7 +46,6 @@ When read, the
>>  .I /dev/urandom
>>  device returns random bytes using a pseudorandom
>>  number generator seeded from the entropy pool.
>> -Reads from this device are nonblocking.
> 
> Being non blocking is not incompatible with being interruptible, though.
> Non blocking only means that the reading will never explicitly yield the
> CPU to wait for data, which is true. This is consistent with the use of
> "non blocking" elsewhere in the filesystem, as far as I understand it.

Yes, I see. Perhaps we can find a middle ground. I restored the removed 
line, but elaborated a little:

[[
Reads from this device do not block (i.e., the CPU is not yielded),
but may incur an appreciable delay when requesting
a large amount (megabytes) of data.
]]

> But I guess this can be confusing indeed, and that the precision is not
> useful in the man page.
> 
>>  When read during early boot time, this device may return
>>  data prior to the entropy pool being initialized.
>>  If this is of concern in your application, use
>> @@ -88,8 +87,11 @@ When calling
>>  .BR read (2)
>>  for the device
>>  .IR /dev/urandom ,
>> -signals will not be handled until after the requested random bytes
>> -have been generated.
>> +reads of up to 256 bytes will return as many bytes as are requested
>> +and will not be interrupted by a signal handler.
>> +Reads with a buffer over this limit may return less than the
>> +requested number of bytes or fail with the error
>> +.BR EINTR .
> 
> Yes, this is a useful precision.
> 
> This patch looks good to me.

Thanks.

Cheers,

Michael


-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] random.4: Improve discussion or urandom, blocking reads, and signals

[Theodore Ts'o]

On Thu, Nov 10, 2016 at 03:20:59PM +0100, Michael Kerrisk (man-pages) wrote:
>     random.4: Improve discussion or urandom, blocking reads, and signals
>     
>     The text currently states that O_NONBLOCK has no effect for
>     /dev/urandom, which is true.  It also says that reads from
>     /dev/urandom are nonblocking.  This is at the least confusing.
>     If one attempts large reads (say 10MB) from /dev/urandom
>     there is an appreciable delay, and interruption by a signal
>     handler will result in a short read. Amend the text to
>     reflect this.

Nonblocking has a very clear and definied meaning in kernel
programming.  So it's technically a true statement.  As far as whether
it is confusing, I wonder if there are any other statements we discuss
what "non-blocking" means where we might want change things.

as far as deleting this line:

> --- a/man4/random.4
> +++ b/man4/random.4
> @@ -46,7 +46,6 @@ When read, the
>  .I /dev/urandom
>  device returns random bytes using a pseudorandom
>  number generator seeded from the entropy pool.
> -Reads from this device are nonblocking.
>  When read during early boot time, this device may return
>  data prior to the entropy pool being initialized.
>  If this is of concern in your application, use

Meh.  I don't really care one way or another.  We have a *huge* amount
of detail about the internal implementation of /dev/urandom, so a
novice programmer who doesn't understand the formal meaning of what it
means for an OS system call to block is likely going to be confused by
other things anyway.

There's sort of a larger philosophical question of how much detail to
include, and how much tutorial instruction to include.  For example, we could say:

    The O_NONBLOCK flag has no effect when opening /dev/urandom or if
    set by fcntl(2), since all reads and writes are non-blocking ---
    that is, the process will not lose control of the CPU and some
    other process will not be scheduled to run during the read or
    write system call.

    *However*, if the user program requests a large number of bytes
    (which is an abuse of the interface, and if a user program does
    this the use program is bad and should feel bad, and we reserve
    the right to artifically cap the number of bytes returned by a
    read to /dev/urandom in the future), the kernel may spend a long
    time doing precisely what the user program requested, and so the
    system call may not return for a long time.

... but some might argue that's probably too much information.  :-)

OTOH, given how much other exhaustive detail is in the man page today,
perhaps it's OK.  I dunno.  Sorry, it's hard for me to get too excited
one way or another, especially since I continuously get questions and
see debates where it's obvious people aren't reading the man page as
it exists today, so I sometimes get a little cynical about too much
bike-sheddding on man pages.  :-/

>  .BR read (2)
>  for the device
>  .IR /dev/urandom ,
> -signals will not be handled until after the requested random bytes
> -have been generated.
> +reads of up to 256 bytes will return as many bytes as are requested
> +and will not be interrupted by a signal handler.
> +Reads with a buffer over this limit may return less than the
> +requested number of bytes or fail with the error
> +.BR EINTR .
> 

Yes, that's a good change to make.

						- Ted
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] random.4: Improve discussion or urandom, blocking reads, and signals

[Michael Kerrisk (man-pages)]

Hi Ted,

On 11/11/2016 05:05 PM, Theodore Ts'o wrote:
> On Thu, Nov 10, 2016 at 03:20:59PM +0100, Michael Kerrisk (man-pages) wrote:
>>     random.4: Improve discussion or urandom, blocking reads, and signals
>>     
>>     The text currently states that O_NONBLOCK has no effect for
>>     /dev/urandom, which is true.  It also says that reads from
>>     /dev/urandom are nonblocking.  This is at the least confusing.
>>     If one attempts large reads (say 10MB) from /dev/urandom
>>     there is an appreciable delay, and interruption by a signal
>>     handler will result in a short read. Amend the text to
>>     reflect this.
> 
> Nonblocking has a very clear and definied meaning in kernel
> programming.  So it's technically a true statement.  As far as whether
> it is confusing, I wonder if there are any other statements we discuss
> what "non-blocking" means where we might want change things.
> 
> as far as deleting this line:
> 
>> --- a/man4/random.4
>> +++ b/man4/random.4
>> @@ -46,7 +46,6 @@ When read, the
>>  .I /dev/urandom
>>  device returns random bytes using a pseudorandom
>>  number generator seeded from the entropy pool.
>> -Reads from this device are nonblocking.
>>  When read during early boot time, this device may return
>>  data prior to the entropy pool being initialized.
>>  If this is of concern in your application, use
> 
> Meh.  I don't really care one way or another.  We have a *huge* amount
> of detail about the internal implementation of /dev/urandom, so a
> novice programmer who doesn't understand the formal meaning of what it
> means for an OS system call to block is likely going to be confused by
> other things anyway.
> 
> There's sort of a larger philosophical question of how much detail to
> include, and how much tutorial instruction to include.  For example, we could say:
> 
>     The O_NONBLOCK flag has no effect when opening /dev/urandom or if
>     set by fcntl(2), since all reads and writes are non-blocking ---
>     that is, the process will not lose control of the CPU and some
>     other process will not be scheduled to run during the read or
>     write system call.
> 
>     *However*, if the user program requests a large number of bytes
>     (which is an abuse of the interface, and if a user program does
>     this the use program is bad and should feel bad, and we reserve
>     the right to artifically cap the number of bytes returned by a
>     read to /dev/urandom in the future), the kernel may spend a long
>     time doing precisely what the user program requested, and so the
>     system call may not return for a long time.
> 
> ... but some might argue that's probably too much information.  :-)

Well, as noted in another reply, I did restore that line, but
elaborated a little:

[[
Reads from this device do not block (i.e., the CPU is not yielded),
but can incur an appreciable delay when requesting large amounts 
of data.
]]

> OTOH, given how much other exhaustive detail is in the man page today,
> perhaps it's OK.  I dunno.  Sorry, it's hard for me to get too excited
> one way or another, especially since I continuously get questions and
> see debates where it's obvious people aren't reading the man page as
> it exists today, so I sometimes get a little cynical about too much
> bike-sheddding on man pages.  :-/

Possible confirmation bias? Perhaps the people that *do* read the
man page don't bother with debates?

> 
>>  .BR read (2)
>>  for the device
>>  .IR /dev/urandom ,
>> -signals will not be handled until after the requested random bytes
>> -have been generated.
>> +reads of up to 256 bytes will return as many bytes as are requested
>> +and will not be interrupted by a signal handler.
>> +Reads with a buffer over this limit may return less than the
>> +requested number of bytes or fail with the error
>> +.BR EINTR .
>>
> 
> Yes, that's a good change to make.

Thanks for checking it.

Cheers,

Michael


-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

New random(7) page for review

[Michael Kerrisk (man-pages)]

[-- Attachment #1: Type: text/plain, Size: 8764 bytes --]

Hello all,

Following on from our discussions about the random(4)
page, and a suggestion from Laurent, I've created
a random(7) page that provides a short overview of
the APIs for obtaining randomness, and some comparisons
and recommendations for using those APIs. A good part of
the text is relocated from the existing getrandom(2) and
random(4) pages. I also incorporated a table that was
proposed by Laurent that compares the APIs on a number
of points.

The rendered version of the page is below. The source is
attached (and already pushed to Git.) 

Does anyone see any further tweaks that might be required?

Cheers,

Michael


RANDOM(7)              Linux Programmer's Manual             RANDOM(7)

NAME
    random - overview of interfaces for obtaining randomness

DESCRIPTION
    The kernel provides the following interfaces to the kernel's cryp‐
    tographically secure pseudorandom number generator (CSPRNG):

    *  The /dev/urandom and /dev/random  devices,  both  described  in
       random(4).   These  devices  have  been  present on Linux since
       early times.

    *  The getrandom(2) system call, available since Linux 3.17.  This
       system  call  provides  access  either  to  the  same source as
       /dev/urandom (called the urandom source in this page) or to the
       same  source  as  /dev/random (called the random source in this
       page).  The default is the urandom source; the random source is
       selected by specifying the GRND_RANDOM flag to the system call.

Initialization of the entropy pool
    The  kernel collects bits of entropy from the environment.  When a
    sufficient number of random bits has been collected,  the  urandom
    entropy pool is considered to be initialized.

Choice of random device
    Unless you are doing long-term key generation (and most likely not
    even then), you probably shouldn't be using getrandom(2) with  the
    GRND_RANDOM flag or the /dev/random device.

    Instead,  use  either getrandom(2) without the GRND_RANDOM flag or
    the /dev/urandom device.  The cryptographic  algorithms  used  for
    the urandom source are quite conservative, and so should be suffi‐
    cient for all purposes.

    The disadvantage of GRND_RANDOM and reads from /dev/random is that
    the  operation can block.  Furthermore, dealing with the partially
    fulfilled requests that can occur when using GRND_RANDOM  or  when
    reading from /dev/random increases code complexity.

Usage recommendations
    The kernel random-number generator relies on entropy gathered from
    device drivers and other sources of environmental  noise.   It  is
    designed  to  produce a small amount of high-quality seed material
    to seed a cryptographically secure pseudorandom  number  generator
    (CSPRNG).   It  is designed for security, not speed, and is poorly
    suited to generating large amounts of cryptographic  random  data.
    Users  should  be  economical  in the amount of seed material that
    they consume  via  getrandom(2),  /dev/urandom,  and  /dev/random.
    Consuming  unnecessarily large quantities of data via these inter‐
    faces will have a negative impact on other  consumers  of  random‐
    ness.

    These interfaces should not be used to provide large quantities of
    data for Monte  Carlo  simulations  or  other  programs/algorithms
    which  are  doing  probabilistic  sampling.  Indeed, such usage is
    unnecessary (and will be slow).  Instead, use these interfaces  to
    provide  a  small amount of data used to seed a user-space pseudo‐
    random number generator for use by such applications.

Comparison between getrandom, /dev/urandom, and /dev/random
    The following table summarizes the behavior of the various  inter‐
    faces  that  can be used to obtain randomness.  GRND_NONBLOCK is a
    flag that can be used to control the blocking behavior of  getran‐
    dom(2).

    ┌──────────────┬──────────────┬──────────────────┬────────────────────┐
    │Interface     │ Pool         │ Blocking         │ Behavior in early  │
    │              │              │ behavior         │ boot time          │
    ├──────────────┼──────────────┼──────────────────┼────────────────────┤
    │/dev/random   │ Blocking     │ If entropy too   │ Blocks until       │
    │              │ pool         │ low, block until │ enough entropy     │
    │              │              │ there is enough  │ gathered           │
    │              │              │ entropy again    │                    │
    ├──────────────┼──────────────┼──────────────────┼────────────────────┤
    │/dev/urandom  │ CSPRNG out‐  │ Never blocks     │ Returns output     │
    │              │ put          │                  │ from uninitialized │
    │              │              │                  │ CSPRNG (may be low │
    │              │              │                  │ entropy and        │
    │              │              │                  │ unsuitable for     │
    │              │              │                  │ cryptography)      │
    ├──────────────┼──────────────┼──────────────────┼────────────────────┤
    │getrandom()   │ Same as      │ Does not block   │ Blocks until pool  │
    │              │ /dev/urandom │ once pool ready  │ ready              │
    ├──────────────┼──────────────┼──────────────────┼────────────────────┤
    │getrandom()   │ Same as      │ If entropy too   │ Blocks until pool  │
    │GRND_RANDOM   │ /dev/random  │ low, block until │ ready              │
    │              │              │ there is enough  │                    │
    │              │              │ entropy again    │                    │
    ├──────────────┼──────────────┼──────────────────┼────────────────────┤
    │getrandom()   │ Same as      │ Does not block   │ EAGAIN if pool not │
    │GRND_NONBLOCK │ /dev/urandom │                  │ ready              │
    ├──────────────┼──────────────┼──────────────────┼────────────────────┤
    │getrandom()   │ Same as      │ EAGAIN if not    │ EAGAIN if pool not │
    │GRND_RANDOM + │ /dev/random  │ enough entropy   │ ready              │
    │GRND_NONBLOCK │              │ available        │                    │
    └──────────────┴──────────────┴──────────────────┴────────────────────┘
Generating cryptographic keys
    The  amount  of seed material required to generate a cryptographic
    key equals the effective key size of  the  key.   For  example,  a
    3072-bit  RSA  or  Diffie-Hellman private key has an effective key
    size of 128 bits (it requires about 2^128 operations to break)  so
    a  key  generator  needs only 128 bits (16 bytes) of seed material
    from /dev/random.

    While some safety margin above that minimum is  reasonable,  as  a
    guard  against  flaws  in  the  CSPRNG algorithm, no cryptographic
    primitive available today can hope to promise more than  256  bits
    of security, so if any program reads more than 256 bits (32 bytes)
    from the kernel random pool  per  invocation,  or  per  reasonable
    reseed  interval  (not less than one minute), that should be taken
    as a sign that its cryptography is not skillfully implemented.

SEE ALSO
    getrandom(2), random(4), urandom(4), signal(7)



-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/

[-- Attachment #2: random.7 --]
[-- Type: application/x-troff-man, Size: 6524 bytes --]

Re: New random(7) page for review

[Laurent Georget]

[-- Attachment #1.1: Type: text/plain, Size: 9769 bytes --]

Hello,

Le 12/11/2016 à 13:25, Michael Kerrisk (man-pages) a écrit :
> [...]
> 
> Does anyone see any further tweaks that might be required?
> 

I put my comments directly in the rendered text below.

> 
> 
> RANDOM(7)              Linux Programmer's Manual             RANDOM(7)
> 
> NAME
>     random - overview of interfaces for obtaining randomness
> 
> DESCRIPTION
>     The kernel provides the following interfaces to the kernel's cryp‐
>     tographically secure pseudorandom number generator (CSPRNG):
> 
>     *  The /dev/urandom and /dev/random  devices,  both  described  in
>        random(4).   These  devices  have  been  present on Linux since
>        early times.
> 
>     *  The getrandom(2) system call, available since Linux 3.17.  This
>        system  call  provides  access  either  to  the  same source as
>        /dev/urandom (called the urandom source in this page) or to the
>        same  source  as  /dev/random (called the random source in this
>        page).  The default is the urandom source; the random source is
>        selected by specifying the GRND_RANDOM flag to the system call.
> 

We could highlight here that /dev/urandom and /dev/random exist in many
UNIX-like systems while getrandom(2) is Linux-specific. This can be
important for developers writing cross-platform code.

> Initialization of the entropy pool
>     The  kernel collects bits of entropy from the environment.  When a
>     sufficient number of random bits has been collected,  the  urandom
>     entropy pool is considered to be initialized.
>

the *random* entropy pool? Or maybe just the entropy pool?

> Choice of random device
>     Unless you are doing long-term key generation (and most likely not
>     even then), you probably shouldn't be using getrandom(2) with  the
>     GRND_RANDOM flag or the /dev/random device.
> 
>     Instead,  use  either getrandom(2) without the GRND_RANDOM flag or
>     the /dev/urandom device.  The cryptographic  algorithms  used  for
>     the urandom source are quite conservative, and so should be suffi‐
>     cient for all purposes.
> 
>     The disadvantage of GRND_RANDOM and reads from /dev/random is that
>     the  operation can block.  Furthermore, dealing with the partially
>     fulfilled requests that can occur when using GRND_RANDOM  or  when
>     reading from /dev/random increases code complexity.
>

This is not a precision that was present in the previous version of the
pages but we should write here "The disadvantage of GRND_RANDOM and reads
from /dev/random is that the  operation can block *for an indefinite
amount of time*".

I don't think we can reach the extreme case where no bits of entropy are
ever collected, but we can imagine a situation with two processes, one
having hig priority and the other low priority, both reading from
/dev/random. If the high priority process consumes entropy quicklier
than it's collected, the low priority process won't wake up ever.
This is how you get robots stuck on Mars!

> Usage recommendations
>     The kernel random-number generator relies on entropy gathered from
>     device drivers and other sources of environmental  noise.   It  is
>     designed  to  produce a small amount of high-quality seed material
>     to seed a cryptographically secure pseudorandom  number  generator
>     (CSPRNG).   It  is designed for security, not speed, and is poorly
>     suited to generating large amounts of cryptographic  random  data.
>     Users  should  be  economical  in the amount of seed material that
>     they consume  via  getrandom(2),  /dev/urandom,  and  /dev/random.
>     Consuming  unnecessarily large quantities of data via these inter‐
>     faces will have a negative impact on other  consumers  of  random‐
>     ness.
> 
>     These interfaces should not be used to provide large quantities of
>     data for Monte  Carlo  simulations  or  other  programs/algorithms
>     which  are  doing  probabilistic  sampling.  Indeed, such usage is
>     unnecessary (and will be slow).  Instead, use these interfaces  to
>     provide  a  small amount of data used to seed a user-space pseudo‐
>     random number generator for use by such applications.
> 
> Comparison between getrandom, /dev/urandom, and /dev/random
>     The following table summarizes the behavior of the various  inter‐
>     faces  that  can be used to obtain randomness.  GRND_NONBLOCK is a
>     flag that can be used to control the blocking behavior of  getran‐
>     dom(2).
> 
>     ┌──────────────┬──────────────┬──────────────────┬────────────────────┐
>     │Interface     │ Pool         │ Blocking         │ Behavior in early  │
>     │              │              │ behavior         │ boot time          │
>     ├──────────────┼──────────────┼──────────────────┼────────────────────┤
>     │/dev/random   │ Blocking     │ If entropy too   │ Blocks until       │
>     │              │ pool         │ low, block until │ enough entropy     │
>     │              │              │ there is enough  │ gathered           │
>     │              │              │ entropy again    │                    │
>     ├──────────────┼──────────────┼──────────────────┼────────────────────┤

"If entropy too low, *blocks* until there is enough entropy again"

>     ├──────────────┼──────────────┼──────────────────┼────────────────────┤
>     │/dev/urandom  │ CSPRNG out‐  │ Never blocks     │ Returns output     │
>     │              │ put          │                  │ from uninitialized │
>     │              │              │                  │ CSPRNG (may be low │
>     │              │              │                  │ entropy and        │
>     │              │              │                  │ unsuitable for     │
>     │              │              │                  │ cryptography)      │
>     ├──────────────┼──────────────┼──────────────────┼────────────────────┤
>     │getrandom()   │ Same as      │ Does not block   │ Blocks until pool  │
>     │              │ /dev/urandom │ once pool ready  │ ready              │
>     ├──────────────┼──────────────┼──────────────────┼────────────────────┤
>     │getrandom()   │ Same as      │ If entropy too   │ Blocks until pool  │
>     │GRND_RANDOM   │ /dev/random  │ low, block until │ ready              │
>     │              │              │ there is enough  │                    │
>     │              │              │ entropy again    │                    │
>     ├──────────────┼──────────────┼──────────────────┼────────────────────┤
>     │getrandom()   │ Same as      │ Does not block   │ EAGAIN if pool not │
>     │GRND_NONBLOCK │ /dev/urandom │                  │ ready              │
>     ├──────────────┼──────────────┼──────────────────┼────────────────────┤
>     │getrandom()   │ Same as      │ EAGAIN if not    │ EAGAIN if pool not │
>     │GRND_RANDOM + │ /dev/random  │ enough entropy   │ ready              │
>     │GRND_NONBLOCK │              │ available        │                    │
>     └──────────────┴──────────────┴──────────────────┴────────────────────┘
> Generating cryptographic keys
>     The  amount  of seed material required to generate a cryptographic
>     key equals the effective key size of  the  key.   For  example,  a
>     3072-bit  RSA  or  Diffie-Hellman private key has an effective key
>     size of 128 bits (it requires about 2^128 operations to break)  so
>     a  key  generator  needs only 128 bits (16 bytes) of seed material
>     from /dev/random.
> 
>     While some safety margin above that minimum is  reasonable,  as  a
>     guard  against  flaws  in  the  CSPRNG algorithm, no cryptographic
>     primitive available today can hope to promise more than  256  bits
>     of security, so if any program reads more than 256 bits (32 bytes)
>     from the kernel random pool  per  invocation,  or  per  reasonable
>     reseed  interval  (not less than one minute), that should be taken
>     as a sign that its cryptography is not skillfully implemented.
> 
> SEE ALSO
>     getrandom(2), random(4), urandom(4), signal(7)
> 
> 

Cheers,
Laurent

 


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: New random(7) page for review

[Theodore Ts'o]

On Sat, Nov 12, 2016 at 01:25:11PM +0100, Michael Kerrisk (man-pages) wrote:
>     ┌──────────────┬──────────────┬──────────────────┬────────────────────┐
>     │Interface     │ Pool         │ Blocking         │ Behavior in early  │
>     │              │              │ behavior         │ boot time          │
>     ├──────────────┼──────────────┼──────────────────┼────────────────────┤
>     │/dev/random   │ Blocking     │ If entropy too   │ Blocks until       │
>     │              │ pool         │ low, block until │ enough entropy     │
>     │              │              │ there is enough  │ gathered           │
>     │              │              │ entropy again    │                    │
>     ├──────────────┼──────────────┼──────────────────┼────────────────────┤
>     │/dev/urandom  │ CSPRNG out‐  │ Never blocks     │ Returns output     │
>     │              │ put          │                  │ from uninitialized │
>     │              │              │                  │ CSPRNG (may be low │
>     │              │              │                  │ entropy and        │
>     │              │              │                  │ unsuitable for     │
>     │              │              │                  │ cryptography)      │
>     ├──────────────┼──────────────┼──────────────────┼────────────────────┤
>     │getrandom()   │ Same as      │ Does not block   │ Blocks until pool  │
>     │              │ /dev/urandom │ once pool ready  │ ready              │
>     ├──────────────┼──────────────┼──────────────────┼────────────────────┤
>     │getrandom()   │ Same as      │ If entropy too   │ Blocks until pool  │
>     │GRND_RANDOM   │ /dev/random  │ low, block until │ ready              │
>     │              │              │ there is enough  │                    │
>     │              │              │ entropy again    │                    │
>     ├──────────────┼──────────────┼──────────────────┼────────────────────┤
>     │getrandom()   │ Same as      │ Does not block   │ EAGAIN if pool not │
>     │GRND_NONBLOCK │ /dev/urandom │                  │ ready              │
>     ├──────────────┼──────────────┼──────────────────┼────────────────────┤
>     │getrandom()   │ Same as      │ EAGAIN if not    │ EAGAIN if pool not │
>     │GRND_RANDOM + │ /dev/random  │ enough entropy   │ ready              │
>     │GRND_NONBLOCK │              │ available        │                    │
>     └──────────────┴──────────────┴──────────────────┴────────────────────┘

I would change the rightmost column to be "Behavior when pool not yet
ready", and just the text in that column accordingly.

And the cell, getrandom() GRND_NONBLOCK and blocking behavior, "does
not block" is not quite right.  It's EAGAIN if pool not ready.

This distinguishes this from /dev/urandom & blocking behavior's "Never
blocks", in that it will return potentially not fully secure
randomness if the pool is not initialized.

Cheers,

						- Ted
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: New random(7) page for review

[Nikos Mavrogiannopoulos]

On Sat, 2016-11-12 at 13:25 +0100, Michael Kerrisk (man-pages) wrote:
> Hello all,
> 
> Following on from our discussions about the random(4)
> page, and a suggestion from Laurent, I've created
> a random(7) page that provides a short overview of
> the APIs for obtaining randomness, and some comparisons
> and recommendations for using those APIs. A good part of
> the text is relocated from the existing getrandom(2) and
> random(4) pages. I also incorporated a table that was
> proposed by Laurent that compares the APIs on a number
> of points.
> 
> The rendered version of the page is below. The source is
> attached (and already pushed to Git.) 
> 
> Does anyone see any further tweaks that might be required?

Looks fine to me.

--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: New random(7) page for review

[Michael Kerrisk (man-pages)]

On 11/14/2016 09:06 AM, Nikos Mavrogiannopoulos wrote:
> On Sat, 2016-11-12 at 13:25 +0100, Michael Kerrisk (man-pages) wrote:
>> Hello all,
>>
>> Following on from our discussions about the random(4)
>> page, and a suggestion from Laurent, I've created
>> a random(7) page that provides a short overview of
>> the APIs for obtaining randomness, and some comparisons
>> and recommendations for using those APIs. A good part of
>> the text is relocated from the existing getrandom(2) and
>> random(4) pages. I also incorporated a table that was
>> proposed by Laurent that compares the APIs on a number
>> of points.
>>
>> The rendered version of the page is below. The source is
>> attached (and already pushed to Git.) 
>>
>> Does anyone see any further tweaks that might be required?
> 
> Looks fine to me.

Thanks, Nikos.

Cheers,

Michael



-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: New random(7) page for review

[Michael Kerrisk (man-pages)]

Hi Laurent,

On 11/12/2016 03:03 PM, Laurent Georget wrote:
> Hello,
> 
> Le 12/11/2016 à 13:25, Michael Kerrisk (man-pages) a écrit :
>> [...]
>>
>> Does anyone see any further tweaks that might be required?
>>
> 
> I put my comments directly in the rendered text below.
> 
>>
>>
>> RANDOM(7)              Linux Programmer's Manual             RANDOM(7)
>>
>> NAME
>>     random - overview of interfaces for obtaining randomness
>>
>> DESCRIPTION
>>     The kernel provides the following interfaces to the kernel's cryp‐
>>     tographically secure pseudorandom number generator (CSPRNG):
>>
>>     *  The /dev/urandom and /dev/random  devices,  both  described  in
>>        random(4).   These  devices  have  been  present on Linux since
>>        early times.
>>
>>     *  The getrandom(2) system call, available since Linux 3.17.  This
>>        system  call  provides  access  either  to  the  same source as
>>        /dev/urandom (called the urandom source in this page) or to the
>>        same  source  as  /dev/random (called the random source in this
>>        page).  The default is the urandom source; the random source is
>>        selected by specifying the GRND_RANDOM flag to the system call.
>>
> 
> We could highlight here that /dev/urandom and /dev/random exist in many
> UNIX-like systems while getrandom(2) is Linux-specific. This can be
> important for developers writing cross-platform code.

Agreed. I added some words to address this.

>> Initialization of the entropy pool
>>     The  kernel collects bits of entropy from the environment.  When a
>>     sufficient number of random bits has been collected,  the  urandom
>>     entropy pool is considered to be initialized.
>>
> 
> the *random* entropy pool? Or maybe just the entropy pool?

Yes. Changed to just "entropy pool"

>> Choice of random device
>>     Unless you are doing long-term key generation (and most likely not
>>     even then), you probably shouldn't be using getrandom(2) with  the
>>     GRND_RANDOM flag or the /dev/random device.
>>
>>     Instead,  use  either getrandom(2) without the GRND_RANDOM flag or
>>     the /dev/urandom device.  The cryptographic  algorithms  used  for
>>     the urandom source are quite conservative, and so should be suffi‐
>>     cient for all purposes.
>>
>>     The disadvantage of GRND_RANDOM and reads from /dev/random is that
>>     the  operation can block.  Furthermore, dealing with the partially
>>     fulfilled requests that can occur when using GRND_RANDOM  or  when
>>     reading from /dev/random increases code complexity.
>>
> 
> This is not a precision that was present in the previous version of the
> pages but we should write here "The disadvantage of GRND_RANDOM and reads
> from /dev/random is that the  operation can block *for an indefinite
> amount of time*".

Yes. Added.

> I don't think we can reach the extreme case where no bits of entropy are
> ever collected, but we can imagine a situation with two processes, one
> having hig priority and the other low priority, both reading from
> /dev/random. If the high priority process consumes entropy quicklier
> than it's collected, the low priority process won't wake up ever.
> This is how you get robots stuck on Mars!
> 
>> Usage recommendations
>>     The kernel random-number generator relies on entropy gathered from
>>     device drivers and other sources of environmental  noise.   It  is
>>     designed  to  produce a small amount of high-quality seed material
>>     to seed a cryptographically secure pseudorandom  number  generator
>>     (CSPRNG).   It  is designed for security, not speed, and is poorly
>>     suited to generating large amounts of cryptographic  random  data.
>>     Users  should  be  economical  in the amount of seed material that
>>     they consume  via  getrandom(2),  /dev/urandom,  and  /dev/random.
>>     Consuming  unnecessarily large quantities of data via these inter‐
>>     faces will have a negative impact on other  consumers  of  random‐
>>     ness.
>>
>>     These interfaces should not be used to provide large quantities of
>>     data for Monte  Carlo  simulations  or  other  programs/algorithms
>>     which  are  doing  probabilistic  sampling.  Indeed, such usage is
>>     unnecessary (and will be slow).  Instead, use these interfaces  to
>>     provide  a  small amount of data used to seed a user-space pseudo‐
>>     random number generator for use by such applications.
>>
>> Comparison between getrandom, /dev/urandom, and /dev/random
>>     The following table summarizes the behavior of the various  inter‐
>>     faces  that  can be used to obtain randomness.  GRND_NONBLOCK is a
>>     flag that can be used to control the blocking behavior of  getran‐
>>     dom(2).
>>
>>     ┌──────────────┬──────────────┬──────────────────┬────────────────────┐
>>     │Interface     │ Pool         │ Blocking         │ Behavior in early  │
>>     │              │              │ behavior         │ boot time          │
>>     ├──────────────┼──────────────┼──────────────────┼────────────────────┤
>>     │/dev/random   │ Blocking     │ If entropy too   │ Blocks until       │
>>     │              │ pool         │ low, block until │ enough entropy     │
>>     │              │              │ there is enough  │ gathered           │
>>     │              │              │ entropy again    │                    │
>>     ├──────────────┼──────────────┼──────────────────┼────────────────────┤
> 
> "If entropy too low, *blocks* until there is enough entropy again"

Fixed.

>>     ├──────────────┼──────────────┼──────────────────┼────────────────────┤
>>     │/dev/urandom  │ CSPRNG out‐  │ Never blocks     │ Returns output     │
>>     │              │ put          │                  │ from uninitialized │
>>     │              │              │                  │ CSPRNG (may be low │
>>     │              │              │                  │ entropy and        │
>>     │              │              │                  │ unsuitable for     │
>>     │              │              │                  │ cryptography)      │
>>     ├──────────────┼──────────────┼──────────────────┼────────────────────┤
>>     │getrandom()   │ Same as      │ Does not block   │ Blocks until pool  │
>>     │              │ /dev/urandom │ once pool ready  │ ready              │
>>     ├──────────────┼──────────────┼──────────────────┼────────────────────┤
>>     │getrandom()   │ Same as      │ If entropy too   │ Blocks until pool  │
>>     │GRND_RANDOM   │ /dev/random  │ low, block until │ ready              │
>>     │              │              │ there is enough  │                    │
>>     │              │              │ entropy again    │                    │
>>     ├──────────────┼──────────────┼──────────────────┼────────────────────┤
>>     │getrandom()   │ Same as      │ Does not block   │ EAGAIN if pool not │
>>     │GRND_NONBLOCK │ /dev/urandom │                  │ ready              │
>>     ├──────────────┼──────────────┼──────────────────┼────────────────────┤
>>     │getrandom()   │ Same as      │ EAGAIN if not    │ EAGAIN if pool not │
>>     │GRND_RANDOM + │ /dev/random  │ enough entropy   │ ready              │
>>     │GRND_NONBLOCK │              │ available        │                    │
>>     └──────────────┴──────────────┴──────────────────┴────────────────────┘
>> Generating cryptographic keys
>>     The  amount  of seed material required to generate a cryptographic
>>     key equals the effective key size of  the  key.   For  example,  a
>>     3072-bit  RSA  or  Diffie-Hellman private key has an effective key
>>     size of 128 bits (it requires about 2^128 operations to break)  so
>>     a  key  generator  needs only 128 bits (16 bytes) of seed material
>>     from /dev/random.
>>
>>     While some safety margin above that minimum is  reasonable,  as  a
>>     guard  against  flaws  in  the  CSPRNG algorithm, no cryptographic
>>     primitive available today can hope to promise more than  256  bits
>>     of security, so if any program reads more than 256 bits (32 bytes)
>>     from the kernel random pool  per  invocation,  or  per  reasonable
>>     reseed  interval  (not less than one minute), that should be taken
>>     as a sign that its cryptography is not skillfully implemented.
>>
>> SEE ALSO
>>     getrandom(2), random(4), urandom(4), signal(7)
>>
>>
> 
> Cheers,
> Laurent
> 
>  
> 


-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: New random(7) page for review

[Michael Kerrisk (man-pages)]

Hi Ted,

On 11/13/2016 11:20 PM, Theodore Ts'o wrote:
> On Sat, Nov 12, 2016 at 01:25:11PM +0100, Michael Kerrisk (man-pages) wrote:
>>     ┌──────────────┬──────────────┬──────────────────┬────────────────────┐
>>     │Interface     │ Pool         │ Blocking         │ Behavior in early  │
>>     │              │              │ behavior         │ boot time          │
>>     ├──────────────┼──────────────┼──────────────────┼────────────────────┤
>>     │/dev/random   │ Blocking     │ If entropy too   │ Blocks until       │
>>     │              │ pool         │ low, block until │ enough entropy     │
>>     │              │              │ there is enough  │ gathered           │
>>     │              │              │ entropy again    │                    │
>>     ├──────────────┼──────────────┼──────────────────┼────────────────────┤
>>     │/dev/urandom  │ CSPRNG out‐  │ Never blocks     │ Returns output     │
>>     │              │ put          │                  │ from uninitialized │
>>     │              │              │                  │ CSPRNG (may be low │
>>     │              │              │                  │ entropy and        │
>>     │              │              │                  │ unsuitable for     │
>>     │              │              │                  │ cryptography)      │
>>     ├──────────────┼──────────────┼──────────────────┼────────────────────┤
>>     │getrandom()   │ Same as      │ Does not block   │ Blocks until pool  │
>>     │              │ /dev/urandom │ once pool ready  │ ready              │
>>     ├──────────────┼──────────────┼──────────────────┼────────────────────┤
>>     │getrandom()   │ Same as      │ If entropy too   │ Blocks until pool  │
>>     │GRND_RANDOM   │ /dev/random  │ low, block until │ ready              │
>>     │              │              │ there is enough  │                    │
>>     │              │              │ entropy again    │                    │
>>     ├──────────────┼──────────────┼──────────────────┼────────────────────┤
>>     │getrandom()   │ Same as      │ Does not block   │ EAGAIN if pool not │
>>     │GRND_NONBLOCK │ /dev/urandom │                  │ ready              │
>>     ├──────────────┼──────────────┼──────────────────┼────────────────────┤
>>     │getrandom()   │ Same as      │ EAGAIN if not    │ EAGAIN if pool not │
>>     │GRND_RANDOM + │ /dev/random  │ enough entropy   │ ready              │
>>     │GRND_NONBLOCK │              │ available        │                    │
>>     └──────────────┴──────────────┴──────────────────┴────────────────────┘
> 
> I would change the rightmost column to be "Behavior when pool not yet
> ready", and just the text in that column accordingly.

Yes, better. Changed.

> And the cell, getrandom() GRND_NONBLOCK and blocking behavior, "does
> not block" is not quite right.  It's EAGAIN if pool not ready.
>
> This distinguishes this from /dev/urandom & blocking behavior's "Never
> blocks", in that it will return potentially not fully secure
> randomness if the pool is not initialized.

I see what you mean, but I think Laurent meant that point to be covered
in the rightmost column. So, to try to make this clearer, I changed the
text in that cell to "Does not block once the pool is read". If this still
seems problematic, let me know for the next version of the page. (I'll
send out a new draft in a moment.)
 
Cheers,

Michael


-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html